2024-09-19T07:32:47.132 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T07:32:47.153 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T07:32:47.245 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911819 branch: squid description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/client-limits} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911819' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - responding to mclientcaps\(revoke\) - not advance its oldest_client_tid - failing to advance its oldest client/flush tid - Too many inodes in cache - failing to respond to cache pressure - slow requests are blocked - failing to respond to capability release - MDS cache is too large - \(MDS_CLIENT_OLDEST_TID\) - \(MDS_CACHE_OVERSIZED\) sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDUKm2Esm24Sp8vfW+fdjdvoN5O6hymkgnj6oO32BJHjs6/UU39o50O2jqCJem9LJ236GcTM+Ufj48UoP8xG8J8= smithi146.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDbnwXGecZyDr09cvJDx+f6F7AIdacwpm+x5UrTPPtwJ416LUd0dJuP9nlqRWCItgCim+t+qaf2vgcXemrVqpLw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_client_limits teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T07:32:47.245 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T07:32:47.246 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T07:32:47.246 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T07:32:47.248 INFO:teuthology.task.internal:Checking packages... 2024-09-19T07:32:47.269 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T07:32:47.270 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T07:32:47.270 INFO:teuthology.packaging:ref: None 2024-09-19T07:32:47.270 INFO:teuthology.packaging:tag: None 2024-09-19T07:32:47.270 INFO:teuthology.packaging:branch: squid 2024-09-19T07:32:47.270 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:32:47.270 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T07:32:47.478 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T07:32:47.480 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T07:32:47.488 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T07:32:47.488 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T07:32:47.513 INFO:teuthology.task.internal:Saving configuration 2024-09-19T07:32:47.530 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T07:32:47.541 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T07:32:47.562 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911819', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 07:27:43.705605', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDUKm2Esm24Sp8vfW+fdjdvoN5O6hymkgnj6oO32BJHjs6/UU39o50O2jqCJem9LJ236GcTM+Ufj48UoP8xG8J8='} 2024-09-19T07:32:47.586 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911819', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 07:27:43.707036', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDbnwXGecZyDr09cvJDx+f6F7AIdacwpm+x5UrTPPtwJ416LUd0dJuP9nlqRWCItgCim+t+qaf2vgcXemrVqpLw='} 2024-09-19T07:32:47.587 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T07:32:47.594 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-19T07:32:47.594 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-19T07:32:47.595 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T07:32:47.681 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f59c5743be0>, signals=[15]) 2024-09-19T07:32:47.682 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T07:32:47.689 INFO:teuthology.task.internal:Opening connections... 2024-09-19T07:32:47.689 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:32:47.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:32:47.764 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:32:47.765 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:32:47.842 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T07:32:47.850 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-09-19T07:32:47.868 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-09-19T07:32:47.869 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-09-19T07:32:47.925 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2024-09-19T07:32:47.926 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2024-09-19T07:32:47.926 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2024-09-19T07:32:47.926 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2024-09-19T07:32:47.926 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2024-09-19T07:32:47.926 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2024-09-19T07:32:47.926 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T07:32:47.926 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-09-19T07:32:47.926 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2024-09-19T07:32:47.926 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T07:32:47.926 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2024-09-19T07:32:47.926 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T07:32:47.927 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T07:32:47.927 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T07:32:47.927 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-09-19T07:32:47.951 DEBUG:teuthology.orchestra.run.smithi146:> uname -m 2024-09-19T07:32:47.970 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2024-09-19T07:32:47.970 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2024-09-19T07:32:48.027 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream" 2024-09-19T07:32:48.027 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="9" 2024-09-19T07:32:48.028 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos" 2024-09-19T07:32:48.028 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora" 2024-09-19T07:32:48.028 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="9" 2024-09-19T07:32:48.028 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el9" 2024-09-19T07:32:48.028 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T07:32:48.028 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2024-09-19T07:32:48.028 INFO:teuthology.orchestra.run.smithi146.stdout:LOGO="fedora-logo-icon" 2024-09-19T07:32:48.028 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T07:32:48.028 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/" 2024-09-19T07:32:48.028 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T07:32:48.028 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T07:32:48.028 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T07:32:48.029 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2024-09-19T07:32:48.051 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T07:32:48.064 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T07:32:48.112 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T07:32:48.113 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-09-19T07:32:48.115 DEBUG:teuthology.orchestra.run.smithi146:> test '!' -e /home/ubuntu/cephtest 2024-09-19T07:32:48.132 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T07:32:48.139 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T07:32:48.139 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-09-19T07:32:48.172 DEBUG:teuthology.orchestra.run.smithi146:> test -z $(ls -A /var/lib/ceph) 2024-09-19T07:32:48.200 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T07:32:48.254 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T07:32:48.270 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-19T07:32:48.270 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-19T07:32:48.270 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T07:32:48.271 DEBUG:teuthology.orchestra.run.smithi146:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T07:32:48.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:32:48.288 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-19T07:32:48.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:32:48.289 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2024-09-19T07:32:48.345 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-503.el9.x86_64 2024-09-19T07:32:48.345 INFO:teuthology.task.kernel:Running kernel on smithi016: 5.14.0-503.el9.x86_64 2024-09-19T07:32:48.345 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2024-09-19T07:32:48.347 INFO:teuthology.orchestra.run.smithi146.stdout:5.14.0-503.el9.x86_64 2024-09-19T07:32:48.347 INFO:teuthology.task.kernel:Running kernel on smithi146: 5.14.0-503.el9.x86_64 2024-09-19T07:32:48.347 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum install -y kernel 2024-09-19T07:32:50.832 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.3 MB 00:00 2024-09-19T07:32:51.818 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.3 MB 00:01 2024-09-19T07:32:52.617 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 32 MB/s | 20 MB 00:00 2024-09-19T07:32:53.786 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-19T07:32:57.251 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-19T07:32:58.220 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-19T07:32:58.996 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-19T07:32:59.712 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 19 kB 00:00 2024-09-19T07:32:59.989 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-19T07:33:00.724 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-19T07:33:05.703 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-19T07:33:06.105 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-19T07:33:07.710 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T07:33:07.710 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T07:33:07.808 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-09-19T07:33:07.818 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-19T07:33:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-09-19T07:33:07.818 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-19T07:33:07.818 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-09-19T07:33:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T07:33:07.818 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-09-19T07:33:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T07:33:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T07:33:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T07:33:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:33:07.819 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-09-19T07:33:07.819 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-19T07:33:07.819 INFO:teuthology.orchestra.run.smithi016.stdout:Install 4 Packages 2024-09-19T07:33:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:33:07.819 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 79 M 2024-09-19T07:33:07.820 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 126 M 2024-09-19T07:33:07.820 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-09-19T07:33:07.987 INFO:teuthology.orchestra.run.smithi146.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T07:33:07.987 INFO:teuthology.orchestra.run.smithi146.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T07:33:08.083 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-19T07:33:08.093 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-19T07:33:08.093 INFO:teuthology.orchestra.run.smithi146.stdout: Package Architecture Version Repository Size 2024-09-19T07:33:08.093 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-19T07:33:08.093 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-19T07:33:08.093 INFO:teuthology.orchestra.run.smithi146.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T07:33:08.093 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-19T07:33:08.093 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T07:33:08.093 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T07:33:08.094 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T07:33:08.094 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:33:08.094 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-19T07:33:08.094 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-19T07:33:08.094 INFO:teuthology.orchestra.run.smithi146.stdout:Install 4 Packages 2024-09-19T07:33:08.094 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:33:08.094 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 79 M 2024-09-19T07:33:08.095 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 126 M 2024-09-19T07:33:08.095 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-19T07:33:08.146 INFO:teuthology.orchestra.run.smithi016.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 128 kB/s | 36 kB 00:00 2024-09-19T07:33:08.448 INFO:teuthology.orchestra.run.smithi146.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 364 kB/s | 36 kB 00:00 2024-09-19T07:33:08.824 INFO:teuthology.orchestra.run.smithi146.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-19T07:33:09.190 INFO:teuthology.orchestra.run.smithi146.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 39 MB/s | 29 MB 00:00 2024-09-19T07:33:09.499 INFO:teuthology.orchestra.run.smithi146.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 30 MB/s | 35 MB 00:01 2024-09-19T07:33:09.499 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-19T07:33:09.499 INFO:teuthology.orchestra.run.smithi146.stdout:Total 56 MB/s | 79 MB 00:01 2024-09-19T07:33:10.181 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-19T07:33:10.547 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-19T07:33:10.547 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-19T07:33:11.491 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-19T07:33:11.491 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-19T07:33:13.938 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-19T07:33:14.645 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T07:33:14.724 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T07:33:17.054 INFO:teuthology.orchestra.run.smithi016.stdout:(2/4): kernel-modules-core-5.14.0-508.el9.x86_6 3.2 MB/s | 29 MB 00:08 2024-09-19T07:33:17.500 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T07:33:17.627 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T07:33:22.436 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T07:33:22.477 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:33:23.637 INFO:teuthology.orchestra.run.smithi016.stdout:(3/4): kernel-core-5.14.0-508.el9.x86_64.rpm 1.0 MB/s | 16 MB 00:15 2024-09-19T07:33:27.341 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:33:50.768 INFO:teuthology.orchestra.run.smithi016.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 824 kB/s | 35 MB 00:42 2024-09-19T07:33:50.768 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-19T07:33:50.769 INFO:teuthology.orchestra.run.smithi016.stdout:Total 1.8 MB/s | 79 MB 00:42 2024-09-19T07:33:51.479 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-09-19T07:33:51.878 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-09-19T07:33:51.879 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-09-19T07:33:52.835 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-09-19T07:33:52.835 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-09-19T07:33:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-09-19T07:33:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T07:33:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T07:33:58.919 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T07:33:59.056 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T07:34:04.186 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T07:34:04.240 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:34:05.909 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:34:05.955 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:34:06.313 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:34:06.314 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T07:34:06.314 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T07:34:06.314 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T07:34:07.120 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:34:07.120 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:34:07.120 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-19T07:34:07.120 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T07:34:07.121 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T07:34:07.121 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T07:34:07.121 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T07:34:07.121 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:34:07.121 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-19T07:34:07.490 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T07:34:07.574 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T07:34:07.575 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T07:34:07.575 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T07:34:07.575 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T07:34:07.575 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T07:34:07.575 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-19T07:34:07.575 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi146.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T07:34:07.576 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T07:34:07.634 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T07:34:07.635 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T07:34:07.635 DEBUG:teuthology.orchestra.run.smithi146:> sudo rpm -qi grub2-tools 2024-09-19T07:34:07.667 INFO:teuthology.orchestra.run.smithi146.stdout:Name : grub2-tools 2024-09-19T07:34:07.667 INFO:teuthology.orchestra.run.smithi146.stdout:Epoch : 1 2024-09-19T07:34:07.667 INFO:teuthology.orchestra.run.smithi146.stdout:Version : 2.06 2024-09-19T07:34:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:Release : 82.el9 2024-09-19T07:34:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:Architecture: x86_64 2024-09-19T07:34:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T07:34:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:Group : Unspecified 2024-09-19T07:34:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:Size : 8274670 2024-09-19T07:34:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:License : GPLv3+ 2024-09-19T07:34:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T07:34:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T07:34:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T07:34:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T07:34:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:Packager : builder@centos.org 2024-09-19T07:34:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:Vendor : CentOS 2024-09-19T07:34:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T07:34:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:Summary : Support tools for GRUB. 2024-09-19T07:34:07.669 INFO:teuthology.orchestra.run.smithi146.stdout:Description : 2024-09-19T07:34:07.669 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:34:07.669 INFO:teuthology.orchestra.run.smithi146.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T07:34:07.669 INFO:teuthology.orchestra.run.smithi146.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T07:34:07.669 INFO:teuthology.orchestra.run.smithi146.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T07:34:07.669 INFO:teuthology.orchestra.run.smithi146.stdout:hardware devices. 2024-09-19T07:34:07.669 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:34:07.669 INFO:teuthology.orchestra.run.smithi146.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T07:34:07.670 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T07:34:07.670 INFO:teuthology.task.kernel:Updating grub on smithi146 to boot 5.14.0-508.el9.x86_64 2024-09-19T07:34:07.670 DEBUG:teuthology.orchestra.run.smithi146:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T07:34:08.277 INFO:teuthology.orchestra.run.smithi146.stderr:Generating grub configuration file ... 2024-09-19T07:34:09.371 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:34:09.603 INFO:teuthology.orchestra.run.smithi146.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T07:34:09.638 INFO:teuthology.orchestra.run.smithi146.stderr:done 2024-09-19T07:34:09.640 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2024-09-19T07:34:09.656 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.zN1HwKMPfn 2024-09-19T07:34:09.656 DEBUG:teuthology.orchestra.run.smithi146:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.zN1HwKMPfn 2024-09-19T07:34:09.723 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.zN1HwKMPfn 2024-09-19T07:34:09.877 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.zN1HwKMPfn is 6KB 2024-09-19T07:34:09.897 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.zN1HwKMPfn 2024-09-19T07:34:09.912 DEBUG:teuthology.orchestra.run.smithi146:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T07:34:09.974 INFO:teuthology.orchestra.run.smithi146.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T07:34:09.975 INFO:teuthology.orchestra.run.smithi146.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T07:34:09.975 INFO:teuthology.orchestra.run.smithi146.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T07:34:09.975 INFO:teuthology.orchestra.run.smithi146.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T07:34:09.977 DEBUG:teuthology.orchestra.run.smithi146:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T07:34:10.134 DEBUG:teuthology.orchestra.run.smithi146:> sudo shutdown -r now 2024-09-19T07:34:40.137 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-19T07:34:40.137 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-19T07:34:40.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:34:48.385 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:34:48.425 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:34:48.805 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:34:48.805 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T07:34:48.805 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T07:34:48.805 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T07:34:49.648 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:34:49.648 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:34:49.648 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-09-19T07:34:49.648 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T07:34:49.648 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T07:34:49.648 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T07:34:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T07:34:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:34:49.649 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-09-19T07:34:49.998 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T07:34:50.089 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T07:34:50.090 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T07:34:50.090 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T07:34:50.090 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T07:34:50.090 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T07:34:50.090 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-19T07:34:50.090 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T07:34:50.090 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T07:34:50.148 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T07:34:50.149 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T07:34:50.149 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2024-09-19T07:34:50.183 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2024-09-19T07:34:50.184 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2024-09-19T07:34:50.184 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.06 2024-09-19T07:34:50.184 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 82.el9 2024-09-19T07:34:50.184 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2024-09-19T07:34:50.184 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T07:34:50.184 INFO:teuthology.orchestra.run.smithi016.stdout:Group : Unspecified 2024-09-19T07:34:50.184 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 8274670 2024-09-19T07:34:50.184 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2024-09-19T07:34:50.184 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T07:34:50.184 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T07:34:50.184 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T07:34:50.185 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T07:34:50.185 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : builder@centos.org 2024-09-19T07:34:50.185 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : CentOS 2024-09-19T07:34:50.185 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T07:34:50.185 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2024-09-19T07:34:50.185 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2024-09-19T07:34:50.185 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:34:50.185 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T07:34:50.185 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T07:34:50.185 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T07:34:50.185 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2024-09-19T07:34:50.185 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:34:50.186 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T07:34:50.186 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T07:34:50.187 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 5.14.0-508.el9.x86_64 2024-09-19T07:34:50.187 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T07:34:50.892 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-09-19T07:34:52.137 INFO:teuthology.orchestra.run.smithi016.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T07:34:52.178 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-09-19T07:34:52.180 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-09-19T07:34:52.195 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.6Nr1EAwU0Y 2024-09-19T07:34:52.195 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6Nr1EAwU0Y 2024-09-19T07:34:52.264 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.6Nr1EAwU0Y 2024-09-19T07:34:52.400 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.6Nr1EAwU0Y is 6KB 2024-09-19T07:34:52.449 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.6Nr1EAwU0Y 2024-09-19T07:34:52.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T07:34:52.525 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T07:34:52.526 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T07:34:52.526 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T07:34:52.526 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T07:34:52.527 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T07:34:52.682 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2024-09-19T07:35:14.753 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-09-19T07:35:22.693 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-19T07:35:22.693 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-19T07:35:22.694 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:35:23.755 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-19T07:35:23.756 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:35:42.141 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-09-19T07:35:54.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-19T07:35:54.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:35:55.836 DEBUG:teuthology.orchestra.run.smithi146:> true 2024-09-19T07:35:56.387 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-19T07:35:56.388 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-19T07:35:56.388 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2024-09-19T07:35:56.403 INFO:teuthology.orchestra.run.smithi146.stdout:5.14.0-508.el9.x86_64 2024-09-19T07:35:56.403 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T07:35:56.403 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T07:35:56.403 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T07:35:57.404 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-19T07:35:57.404 DEBUG:teuthology.orchestra.run.smithi146:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T07:35:57.492 INFO:teuthology.orchestra.run.smithi146.stdout:ttyS1 2024-09-19T07:35:57.525 DEBUG:teuthology.parallel:result is None 2024-09-19T07:36:22.697 DEBUG:teuthology.orchestra.remote:timed out 2024-09-19T07:36:31.698 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-19T07:36:31.700 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:36:31.982 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-19T07:36:32.335 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-19T07:36:32.336 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-19T07:36:32.336 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-19T07:36:32.351 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-508.el9.x86_64 2024-09-19T07:36:32.351 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T07:36:32.352 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T07:36:32.352 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T07:36:33.353 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-19T07:36:33.353 DEBUG:teuthology.orchestra.run.smithi016:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T07:36:33.448 INFO:teuthology.orchestra.run.smithi016.stdout:ttyS1 2024-09-19T07:36:33.486 DEBUG:teuthology.parallel:result is None 2024-09-19T07:36:33.486 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T07:36:33.494 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T07:36:33.494 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T07:36:33.496 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T07:36:33.517 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T07:36:33.537 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T07:36:33.571 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T07:36:33.572 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T07:36:33.574 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T07:36:33.616 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T07:36:33.622 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T07:36:33.623 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T07:36:33.635 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T07:36:33.661 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:36:33.672 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:36:33.698 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:36:33.713 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:36:33.713 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T07:36:33.720 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T07:36:33.720 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T07:36:33.722 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T07:36:33.751 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T07:36:33.761 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T07:36:33.761 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T07:36:33.794 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T07:36:33.811 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:36:33.896 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:36:33.947 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:36:34.034 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:36:34.077 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:36:34.077 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T07:36:34.146 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:36:34.191 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:36:34.247 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:36:34.333 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:36:34.375 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:36:34.375 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T07:36:34.444 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-09-19T07:36:34.447 DEBUG:teuthology.orchestra.run.smithi146:> sudo service rsyslog restart 2024-09-19T07:36:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:36:34.557 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:36:34.972 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T07:36:34.980 INFO:teuthology.task.internal:Starting timer... 2024-09-19T07:36:34.980 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T07:36:35.026 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T07:36:35.057 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-19T07:36:35.057 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-09-19T07:36:35.131 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-09-19T07:36:35.133 DEBUG:teuthology.orchestra.run.smithi146:> sudo service auditd rotate 2024-09-19T07:36:35.204 INFO:teuthology.orchestra.run.smithi146.stdout:Rotating logs: 2024-09-19T07:36:35.206 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T07:36:35.207 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-09-19T07:36:35.226 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-09-19T07:36:35.227 DEBUG:teuthology.orchestra.run.smithi146:> /usr/sbin/getenforce 2024-09-19T07:36:35.250 INFO:teuthology.orchestra.run.smithi146.stdout:Permissive 2024-09-19T07:36:35.251 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi146.front.sepia.ceph.com': 'permissive'} 2024-09-19T07:36:35.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:36:35.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:36:35.297 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:36:35.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:36:35.326 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T07:36:35.326 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-09-19T07:36:35.371 DEBUG:teuthology.orchestra.run.smithi146:> sudo /usr/sbin/setenforce permissive 2024-09-19T07:36:35.404 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T07:36:35.413 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T07:36:35.428 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T07:36:35.429 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi146.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T07:40:23.644 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi146.front.sepia.ceph.com')] 2024-09-19T07:40:23.645 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-19T07:40:23.646 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:40:23.722 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-19T07:40:23.796 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-19T07:40:23.796 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-19T07:40:23.797 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:40:23.873 DEBUG:teuthology.orchestra.run.smithi146:> true 2024-09-19T07:40:23.953 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-19T07:40:23.953 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T07:40:23.964 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T07:40:23.964 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T07:40:23.965 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:40:23.967 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T07:40:23.967 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:40:23.994 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T07:40:24.009 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T07:40:24.023 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T07:40:24.034 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-09-19T07:40:24.038 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T07:40:24.046 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-09-19T07:40:24.061 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T07:40:24.062 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: ntpd: command not found 2024-09-19T07:40:24.074 INFO:teuthology.orchestra.run.smithi146.stdout:506 Cannot talk to daemon 2024-09-19T07:40:24.075 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T07:40:24.089 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T07:40:24.103 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T07:40:24.132 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:40:24.135 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:40:24.136 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-09-19T07:40:24.136 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:40:24.136 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:40:24.136 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:40:24.136 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:40:24.157 INFO:teuthology.orchestra.run.smithi146.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:40:24.161 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:40:24.161 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2024-09-19T07:40:24.161 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:40:24.161 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:40:24.161 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:40:24.161 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:40:24.162 INFO:teuthology.run_tasks:Running task install... 2024-09-19T07:40:24.169 DEBUG:teuthology.task.install:project ceph 2024-09-19T07:40:24.170 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T07:40:24.170 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T07:40:24.170 INFO:teuthology.task.install:Using flavor: default 2024-09-19T07:40:24.175 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T07:40:24.175 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T07:40:24.176 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T07:40:24.176 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:40:24.178 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T07:40:24.179 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:40:24.315 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T07:40:24.315 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T07:40:24.325 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T07:40:24.325 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T07:40:24.435 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T07:40:24.436 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:40:24.436 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T07:40:24.438 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T07:40:24.438 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:40:24.438 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T07:40:24.464 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T07:40:24.464 DEBUG:teuthology.orchestra.run.smithi146:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T07:40:24.465 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T07:40:24.466 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T07:40:24.530 DEBUG:teuthology.orchestra.run.smithi146:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T07:40:24.532 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T07:40:24.609 DEBUG:teuthology.orchestra.run.smithi146:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T07:40:24.611 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T07:40:24.678 INFO:teuthology.orchestra.run.smithi146.stdout:check_obsoletes = 1 2024-09-19T07:40:24.680 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum clean all 2024-09-19T07:40:24.681 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-09-19T07:40:24.681 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-09-19T07:40:24.917 INFO:teuthology.orchestra.run.smithi146.stdout:45 files removed 2024-09-19T07:40:24.918 INFO:teuthology.orchestra.run.smithi016.stdout:45 files removed 2024-09-19T07:40:24.945 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T07:40:24.946 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T07:40:25.499 INFO:teuthology.orchestra.run.smithi146.stdout:ceph packages for x86_64 315 kB/s | 84 kB 00:00 2024-09-19T07:40:25.503 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 326 kB/s | 84 kB 00:00 2024-09-19T07:40:25.779 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 51 kB/s | 12 kB 00:00 2024-09-19T07:40:25.790 INFO:teuthology.orchestra.run.smithi146.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-19T07:40:26.046 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 8.3 kB/s | 1.9 kB 00:00 2024-09-19T07:40:26.066 INFO:teuthology.orchestra.run.smithi146.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-19T07:40:26.738 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-19T07:40:26.849 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.3 MB 00:00 2024-09-19T07:40:28.447 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 34 MB/s | 20 MB 00:00 2024-09-19T07:40:30.084 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - AppStream 9.6 MB/s | 20 MB 00:02 2024-09-19T07:40:34.476 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-19T07:40:34.606 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.5 MB 00:02 2024-09-19T07:40:36.093 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 99 kB/s | 19 kB 00:00 2024-09-19T07:40:36.143 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-19T07:40:37.281 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-19T07:40:37.347 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-19T07:40:42.612 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-09-19T07:40:42.953 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-19T07:40:44.772 INFO:teuthology.orchestra.run.smithi146.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T07:40:44.773 INFO:teuthology.orchestra.run.smithi146.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T07:40:44.784 INFO:teuthology.orchestra.run.smithi146.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T07:40:44.785 INFO:teuthology.orchestra.run.smithi146.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T07:40:44.785 INFO:teuthology.orchestra.run.smithi146.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T07:40:44.786 INFO:teuthology.orchestra.run.smithi146.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T07:40:44.788 INFO:teuthology.orchestra.run.smithi146.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T07:40:44.790 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T07:40:44.791 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T07:40:44.853 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-19T07:40:44.873 INFO:teuthology.orchestra.run.smithi146.stdout:====================================================================================== 2024-09-19T07:40:44.874 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-19T07:40:44.874 INFO:teuthology.orchestra.run.smithi146.stdout:====================================================================================== 2024-09-19T07:40:44.874 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-19T07:40:44.874 INFO:teuthology.orchestra.run.smithi146.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T07:40:44.874 INFO:teuthology.orchestra.run.smithi146.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T07:40:44.874 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T07:40:44.874 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T07:40:44.874 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T07:40:44.874 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T07:40:44.874 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T07:40:44.875 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T07:40:44.875 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T07:40:44.875 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T07:40:44.875 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T07:40:44.875 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T07:40:44.875 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T07:40:44.875 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T07:40:44.875 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T07:40:44.875 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T07:40:44.875 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T07:40:44.875 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T07:40:44.875 INFO:teuthology.orchestra.run.smithi146.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T07:40:44.875 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T07:40:44.876 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T07:40:44.876 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T07:40:44.876 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T07:40:44.876 INFO:teuthology.orchestra.run.smithi146.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T07:40:44.876 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T07:40:44.876 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T07:40:44.876 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T07:40:44.876 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T07:40:44.876 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T07:40:44.876 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T07:40:44.876 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T07:40:44.876 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T07:40:44.877 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T07:40:44.877 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T07:40:44.877 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T07:40:44.877 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T07:40:44.877 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T07:40:44.877 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrading: 2024-09-19T07:40:44.877 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T07:40:44.877 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T07:40:44.877 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T07:40:44.877 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T07:40:44.877 INFO:teuthology.orchestra.run.smithi146.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T07:40:44.877 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T07:40:44.877 INFO:teuthology.orchestra.run.smithi146.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T07:40:44.878 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T07:40:44.878 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-19T07:40:44.878 INFO:teuthology.orchestra.run.smithi146.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T07:40:44.878 INFO:teuthology.orchestra.run.smithi146.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T07:40:44.878 INFO:teuthology.orchestra.run.smithi146.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T07:40:44.878 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T07:40:44.878 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T07:40:44.878 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T07:40:44.878 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T07:40:44.878 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T07:40:44.878 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T07:40:44.878 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T07:40:44.878 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T07:40:44.879 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T07:40:44.879 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T07:40:44.879 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T07:40:44.879 INFO:teuthology.orchestra.run.smithi146.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T07:40:44.879 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T07:40:44.879 INFO:teuthology.orchestra.run.smithi146.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T07:40:44.879 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T07:40:44.879 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T07:40:44.879 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T07:40:44.879 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T07:40:44.879 INFO:teuthology.orchestra.run.smithi146.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T07:40:44.879 INFO:teuthology.orchestra.run.smithi146.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T07:40:44.879 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T07:40:44.880 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T07:40:44.880 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T07:40:44.880 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T07:40:44.880 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T07:40:44.880 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T07:40:44.880 INFO:teuthology.orchestra.run.smithi146.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T07:40:44.880 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T07:40:44.880 INFO:teuthology.orchestra.run.smithi146.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T07:40:44.880 INFO:teuthology.orchestra.run.smithi146.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T07:40:44.881 INFO:teuthology.orchestra.run.smithi146.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T07:40:44.881 INFO:teuthology.orchestra.run.smithi146.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T07:40:44.881 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T07:40:44.881 INFO:teuthology.orchestra.run.smithi146.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T07:40:44.881 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T07:40:44.881 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T07:40:44.881 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T07:40:44.881 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T07:40:44.881 INFO:teuthology.orchestra.run.smithi146.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T07:40:44.881 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T07:40:44.881 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T07:40:44.881 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T07:40:44.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T07:40:44.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T07:40:44.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T07:40:44.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T07:40:44.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T07:40:44.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T07:40:44.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T07:40:44.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T07:40:44.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T07:40:44.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T07:40:44.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T07:40:44.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T07:40:44.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T07:40:44.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T07:40:44.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T07:40:44.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T07:40:44.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T07:40:44.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T07:40:44.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T07:40:44.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T07:40:44.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T07:40:44.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T07:40:44.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T07:40:44.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T07:40:44.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T07:40:44.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T07:40:44.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T07:40:44.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T07:40:44.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T07:40:44.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T07:40:44.885 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T07:40:44.885 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T07:40:44.885 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T07:40:44.885 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T07:40:44.885 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T07:40:44.885 INFO:teuthology.orchestra.run.smithi146.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T07:40:44.885 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T07:40:44.885 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T07:40:44.885 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T07:40:44.885 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T07:40:44.886 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T07:40:44.886 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T07:40:44.886 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T07:40:44.886 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T07:40:44.886 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T07:40:44.886 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T07:40:44.886 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T07:40:44.886 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T07:40:44.886 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T07:40:44.886 INFO:teuthology.orchestra.run.smithi146.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T07:40:44.886 INFO:teuthology.orchestra.run.smithi146.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T07:40:44.886 INFO:teuthology.orchestra.run.smithi146.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T07:40:44.886 INFO:teuthology.orchestra.run.smithi146.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T07:40:44.886 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T07:40:44.887 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T07:40:44.887 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T07:40:44.887 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T07:40:44.887 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T07:40:44.887 INFO:teuthology.orchestra.run.smithi146.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T07:40:44.887 INFO:teuthology.orchestra.run.smithi146.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T07:40:44.887 INFO:teuthology.orchestra.run.smithi146.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T07:40:44.887 INFO:teuthology.orchestra.run.smithi146.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T07:40:44.887 INFO:teuthology.orchestra.run.smithi146.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T07:40:44.887 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T07:40:44.887 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T07:40:44.887 INFO:teuthology.orchestra.run.smithi146.stdout:Installing weak dependencies: 2024-09-19T07:40:44.887 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T07:40:44.888 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:40:44.888 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-19T07:40:44.888 INFO:teuthology.orchestra.run.smithi146.stdout:====================================================================================== 2024-09-19T07:40:44.888 INFO:teuthology.orchestra.run.smithi146.stdout:Install 148 Packages 2024-09-19T07:40:44.888 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrade 8 Packages 2024-09-19T07:40:44.888 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:40:44.888 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 226 M 2024-09-19T07:40:44.889 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-19T07:40:45.197 INFO:teuthology.orchestra.run.smithi146.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 48 kB/s | 6.6 kB 00:00 2024-09-19T07:40:45.242 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T07:40:45.243 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T07:40:45.270 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T07:40:45.270 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T07:40:45.270 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T07:40:45.271 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T07:40:45.271 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T07:40:45.271 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T07:40:45.271 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T07:40:45.333 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-09-19T07:40:45.353 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-19T07:40:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-09-19T07:40:45.353 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-19T07:40:45.354 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-09-19T07:40:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T07:40:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T07:40:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T07:40:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T07:40:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T07:40:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T07:40:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T07:40:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T07:40:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T07:40:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T07:40:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T07:40:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T07:40:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T07:40:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T07:40:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T07:40:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T07:40:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T07:40:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T07:40:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T07:40:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T07:40:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T07:40:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T07:40:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T07:40:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T07:40:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T07:40:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T07:40:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T07:40:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T07:40:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T07:40:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T07:40:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T07:40:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T07:40:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T07:40:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T07:40:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T07:40:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T07:40:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T07:40:45.356 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-09-19T07:40:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T07:40:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T07:40:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T07:40:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T07:40:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T07:40:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T07:40:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T07:40:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T07:40:45.357 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-09-19T07:40:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T07:40:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T07:40:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T07:40:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T07:40:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T07:40:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T07:40:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T07:40:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T07:40:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T07:40:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T07:40:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T07:40:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T07:40:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T07:40:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T07:40:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T07:40:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T07:40:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T07:40:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T07:40:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T07:40:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T07:40:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T07:40:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T07:40:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T07:40:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T07:40:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T07:40:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T07:40:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T07:40:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T07:40:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T07:40:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T07:40:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T07:40:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T07:40:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T07:40:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T07:40:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T07:40:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T07:40:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T07:40:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T07:40:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T07:40:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T07:40:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T07:40:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T07:40:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T07:40:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T07:40:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T07:40:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T07:40:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T07:40:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T07:40:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T07:40:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T07:40:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T07:40:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T07:40:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T07:40:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T07:40:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T07:40:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T07:40:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T07:40:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T07:40:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T07:40:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T07:40:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T07:40:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T07:40:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T07:40:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T07:40:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T07:40:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T07:40:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T07:40:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T07:40:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T07:40:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T07:40:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T07:40:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T07:40:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T07:40:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T07:40:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T07:40:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T07:40:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T07:40:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T07:40:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T07:40:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T07:40:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T07:40:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T07:40:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T07:40:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T07:40:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T07:40:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T07:40:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T07:40:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T07:40:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T07:40:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T07:40:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T07:40:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T07:40:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T07:40:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T07:40:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T07:40:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T07:40:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T07:40:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T07:40:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T07:40:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T07:40:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T07:40:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T07:40:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T07:40:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T07:40:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T07:40:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T07:40:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T07:40:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T07:40:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T07:40:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T07:40:45.366 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2024-09-19T07:40:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T07:40:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:40:45.367 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-09-19T07:40:45.367 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-19T07:40:45.367 INFO:teuthology.orchestra.run.smithi016.stdout:Install 148 Packages 2024-09-19T07:40:45.367 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 8 Packages 2024-09-19T07:40:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:40:45.367 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 226 M 2024-09-19T07:40:45.368 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-09-19T07:40:45.380 INFO:teuthology.orchestra.run.smithi146.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 4.8 MB/s | 896 kB 00:00 2024-09-19T07:40:45.439 INFO:teuthology.orchestra.run.smithi146.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.4 MB/s | 145 kB 00:00 2024-09-19T07:40:45.589 INFO:teuthology.orchestra.run.smithi146.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 10 MB/s | 5.4 MB 00:00 2024-09-19T07:40:45.672 INFO:teuthology.orchestra.run.smithi146.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 10 MB/s | 2.4 MB 00:00 2024-09-19T07:40:45.714 INFO:teuthology.orchestra.run.smithi146.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 12 MB/s | 1.5 MB 00:00 2024-09-19T07:40:46.065 INFO:teuthology.orchestra.run.smithi146.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-09-19T07:40:46.304 INFO:teuthology.orchestra.run.smithi016.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 31 kB/s | 6.6 kB 00:00 2024-09-19T07:40:46.390 INFO:teuthology.orchestra.run.smithi146.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 16 MB/s | 21 MB 00:01 2024-09-19T07:40:46.431 INFO:teuthology.orchestra.run.smithi146.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 608 kB/s | 25 kB 00:00 2024-09-19T07:40:46.715 INFO:teuthology.orchestra.run.smithi146.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 17 MB/s | 17 MB 00:01 2024-09-19T07:40:46.771 INFO:teuthology.orchestra.run.smithi016.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 1.9 MB/s | 896 kB 00:00 2024-09-19T07:40:46.838 INFO:teuthology.orchestra.run.smithi016.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.1 MB/s | 145 kB 00:00 2024-09-19T07:40:46.840 INFO:teuthology.orchestra.run.smithi146.stdout:(11/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 14 MB/s | 11 MB 00:00 2024-09-19T07:40:46.865 INFO:teuthology.orchestra.run.smithi146.stdout:(12/156): cephfs-mirror-19.1.1-361.g2f599a46.el 1.5 MB/s | 228 kB 00:00 2024-09-19T07:40:46.891 INFO:teuthology.orchestra.run.smithi146.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 538 kB/s | 27 kB 00:00 2024-09-19T07:40:46.916 INFO:teuthology.orchestra.run.smithi146.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 631 kB/s | 32 kB 00:00 2024-09-19T07:40:46.950 INFO:teuthology.orchestra.run.smithi146.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 13 MB/s | 751 kB 00:00 2024-09-19T07:40:46.975 INFO:teuthology.orchestra.run.smithi146.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.7 MB/s | 163 kB 00:00 2024-09-19T07:40:47.000 INFO:teuthology.orchestra.run.smithi146.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T07:40:47.025 INFO:teuthology.orchestra.run.smithi146.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 9.9 MB/s | 506 kB 00:00 2024-09-19T07:40:47.078 INFO:teuthology.orchestra.run.smithi146.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 862 kB/s | 45 kB 00:00 2024-09-19T07:40:47.170 INFO:teuthology.orchestra.run.smithi146.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.5 MB/s | 136 kB 00:00 2024-09-19T07:40:47.253 INFO:teuthology.orchestra.run.smithi146.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.9 MB/s | 161 kB 00:00 2024-09-19T07:40:47.379 INFO:teuthology.orchestra.run.smithi146.stdout:(22/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 14 MB/s | 5.3 MB 00:00 2024-09-19T07:40:47.404 INFO:teuthology.orchestra.run.smithi146.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 2.1 MB/s | 320 kB 00:00 2024-09-19T07:40:47.422 INFO:teuthology.orchestra.run.smithi016.stdout:(4/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 4.1 MB/s | 2.4 MB 00:00 2024-09-19T07:40:47.438 INFO:teuthology.orchestra.run.smithi146.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.0 MB/s | 300 kB 00:00 2024-09-19T07:40:47.463 INFO:teuthology.orchestra.run.smithi146.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.7 MB/s | 99 kB 00:00 2024-09-19T07:40:47.488 INFO:teuthology.orchestra.run.smithi146.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T07:40:47.555 INFO:teuthology.orchestra.run.smithi146.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-19T07:40:47.630 INFO:teuthology.orchestra.run.smithi146.stdout:(28/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 19 MB/s | 3.1 MB 00:00 2024-09-19T07:40:47.647 INFO:teuthology.orchestra.run.smithi146.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 346 kB/s | 32 kB 00:00 2024-09-19T07:40:47.681 INFO:teuthology.orchestra.run.smithi146.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.8 MB/s | 146 kB 00:00 2024-09-19T07:40:47.689 INFO:teuthology.orchestra.run.smithi016.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 5.7 MB/s | 1.5 MB 00:00 2024-09-19T07:40:47.981 INFO:teuthology.orchestra.run.smithi146.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 11 MB/s | 3.6 MB 00:00 2024-09-19T07:40:48.073 INFO:teuthology.orchestra.run.smithi146.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.7 MB/s | 249 kB 00:00 2024-09-19T07:40:48.165 INFO:teuthology.orchestra.run.smithi146.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 15 MB/s | 7.4 MB 00:00 2024-09-19T07:40:48.190 INFO:teuthology.orchestra.run.smithi146.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 422 kB/s | 49 kB 00:00 2024-09-19T07:40:48.215 INFO:teuthology.orchestra.run.smithi146.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 329 kB/s | 16 kB 00:00 2024-09-19T07:40:48.241 INFO:teuthology.orchestra.run.smithi146.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 5.6 MB/s | 285 kB 00:00 2024-09-19T07:40:48.274 INFO:teuthology.orchestra.run.smithi146.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 13 MB/s | 765 kB 00:00 2024-09-19T07:40:48.299 INFO:teuthology.orchestra.run.smithi146.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 312 kB/s | 18 kB 00:00 2024-09-19T07:40:48.417 INFO:teuthology.orchestra.run.smithi146.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 562 kB/s | 80 kB 00:00 2024-09-19T07:40:48.764 INFO:teuthology.orchestra.run.smithi016.stdout:(6/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 2.0 MB/s | 5.4 MB 00:02 2024-09-19T07:40:48.831 INFO:teuthology.orchestra.run.smithi016.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 4.1 MB/s | 4.7 MB 00:01 2024-09-19T07:40:48.842 INFO:teuthology.orchestra.run.smithi146.stdout:(40/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 20 MB/s | 49 MB 00:02 2024-09-19T07:40:48.867 INFO:teuthology.orchestra.run.smithi146.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 425 kB/s | 191 kB 00:00 2024-09-19T07:40:48.909 INFO:teuthology.orchestra.run.smithi146.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 801 kB 00:00 2024-09-19T07:40:48.951 INFO:teuthology.orchestra.run.smithi146.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-19T07:40:49.001 INFO:teuthology.orchestra.run.smithi146.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-09-19T07:40:49.026 INFO:teuthology.orchestra.run.smithi146.stdout:(45/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-19T07:40:49.052 INFO:teuthology.orchestra.run.smithi146.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-09-19T07:40:49.077 INFO:teuthology.orchestra.run.smithi146.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-19T07:40:49.102 INFO:teuthology.orchestra.run.smithi146.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-19T07:40:49.803 INFO:teuthology.orchestra.run.smithi146.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 153 kB/s | 107 kB 00:00 2024-09-19T07:40:49.856 INFO:teuthology.orchestra.run.smithi016.stdout:(8/156): ceph-radosgw-19.1.1-361.g2f599a46.el9. 10 MB/s | 11 MB 00:01 2024-09-19T07:40:49.906 INFO:teuthology.orchestra.run.smithi016.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 508 kB/s | 25 kB 00:00 2024-09-19T07:40:49.936 INFO:teuthology.orchestra.run.smithi146.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 779 kB/s | 689 kB 00:00 2024-09-19T07:40:49.987 INFO:teuthology.orchestra.run.smithi146.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-09-19T07:40:50.345 INFO:teuthology.orchestra.run.smithi146.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 40 kB/s | 22 kB 00:00 2024-09-19T07:40:50.537 INFO:teuthology.orchestra.run.smithi146.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 59 kB/s | 33 kB 00:00 2024-09-19T07:40:50.762 INFO:teuthology.orchestra.run.smithi146.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 380 kB/s | 313 kB 00:00 2024-09-19T07:40:51.029 INFO:teuthology.orchestra.run.smithi146.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 37 kB/s | 18 kB 00:00 2024-09-19T07:40:51.271 INFO:teuthology.orchestra.run.smithi146.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-09-19T07:40:51.379 INFO:teuthology.orchestra.run.smithi146.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.9 MB/s | 3.0 MB 00:01 2024-09-19T07:40:51.440 INFO:teuthology.orchestra.run.smithi016.stdout:(10/156): ceph-common-19.1.1-361.g2f599a46.el9. 4.0 MB/s | 21 MB 00:05 2024-09-19T07:40:51.507 INFO:teuthology.orchestra.run.smithi016.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 3.3 MB/s | 228 kB 00:00 2024-09-19T07:40:51.557 INFO:teuthology.orchestra.run.smithi016.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 543 kB/s | 27 kB 00:00 2024-09-19T07:40:51.608 INFO:teuthology.orchestra.run.smithi016.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 636 kB/s | 32 kB 00:00 2024-09-19T07:40:51.730 INFO:teuthology.orchestra.run.smithi146.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 229 kB/s | 160 kB 00:00 2024-09-19T07:40:51.741 INFO:teuthology.orchestra.run.smithi016.stdout:(14/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 5.5 MB/s | 751 kB 00:00 2024-09-19T07:40:51.792 INFO:teuthology.orchestra.run.smithi016.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 3.2 MB/s | 163 kB 00:00 2024-09-19T07:40:51.843 INFO:teuthology.orchestra.run.smithi016.stdout:(16/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T07:40:51.880 INFO:teuthology.orchestra.run.smithi146.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 74 kB/s | 45 kB 00:00 2024-09-19T07:40:51.934 INFO:teuthology.orchestra.run.smithi016.stdout:(17/156): libradosstriper1-19.1.1-361.g2f599a46 5.4 MB/s | 506 kB 00:00 2024-09-19T07:40:52.264 INFO:teuthology.orchestra.run.smithi146.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 750 kB/s | 662 kB 00:00 2024-09-19T07:40:52.293 INFO:teuthology.orchestra.run.smithi016.stdout:(18/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 4.8 MB/s | 17 MB 00:03 2024-09-19T07:40:52.343 INFO:teuthology.orchestra.run.smithi016.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 910 kB/s | 45 kB 00:00 2024-09-19T07:40:52.393 INFO:teuthology.orchestra.run.smithi016.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.7 MB/s | 136 kB 00:00 2024-09-19T07:40:52.438 INFO:teuthology.orchestra.run.smithi146.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 44 kB/s | 24 kB 00:00 2024-09-19T07:40:52.443 INFO:teuthology.orchestra.run.smithi016.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 3.2 MB/s | 161 kB 00:00 2024-09-19T07:40:52.519 INFO:teuthology.orchestra.run.smithi016.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 4.2 MB/s | 320 kB 00:00 2024-09-19T07:40:52.585 INFO:teuthology.orchestra.run.smithi016.stdout:(23/156): python3-rbd-19.1.1-361.g2f599a46.el9. 4.4 MB/s | 300 kB 00:00 2024-09-19T07:40:52.597 INFO:teuthology.orchestra.run.smithi146.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 670 kB/s | 581 kB 00:00 2024-09-19T07:40:52.636 INFO:teuthology.orchestra.run.smithi016.stdout:(24/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T07:40:52.686 INFO:teuthology.orchestra.run.smithi016.stdout:(25/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T07:40:52.762 INFO:teuthology.orchestra.run.smithi016.stdout:(26/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 6.5 MB/s | 5.3 MB 00:00 2024-09-19T07:40:52.812 INFO:teuthology.orchestra.run.smithi016.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-19T07:40:52.854 INFO:teuthology.orchestra.run.smithi016.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 768 kB/s | 32 kB 00:00 2024-09-19T07:40:52.881 INFO:teuthology.orchestra.run.smithi146.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 77 kB/s | 47 kB 00:00 2024-09-19T07:40:52.904 INFO:teuthology.orchestra.run.smithi016.stdout:(29/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.9 MB/s | 146 kB 00:00 2024-09-19T07:40:53.188 INFO:teuthology.orchestra.run.smithi016.stdout:(30/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 6.2 MB/s | 3.1 MB 00:00 2024-09-19T07:40:53.239 INFO:teuthology.orchestra.run.smithi146.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 365 kB/s | 292 kB 00:00 2024-09-19T07:40:53.331 INFO:teuthology.orchestra.run.smithi146.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 257 kB/s | 188 kB 00:00 2024-09-19T07:40:53.346 INFO:teuthology.orchestra.run.smithi016.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 8.1 MB/s | 3.6 MB 00:00 2024-09-19T07:40:53.413 INFO:teuthology.orchestra.run.smithi016.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 3.7 MB/s | 249 kB 00:00 2024-09-19T07:40:53.448 INFO:teuthology.orchestra.run.smithi146.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 68 kB/s | 38 kB 00:00 2024-09-19T07:40:53.464 INFO:teuthology.orchestra.run.smithi016.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 990 kB/s | 49 kB 00:00 2024-09-19T07:40:53.514 INFO:teuthology.orchestra.run.smithi016.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 330 kB/s | 16 kB 00:00 2024-09-19T07:40:53.581 INFO:teuthology.orchestra.run.smithi016.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 4.2 MB/s | 285 kB 00:00 2024-09-19T07:40:53.698 INFO:teuthology.orchestra.run.smithi016.stdout:(36/156): cephadm-19.1.1-361.g2f599a46.el9.noar 6.4 MB/s | 765 kB 00:00 2024-09-19T07:40:53.748 INFO:teuthology.orchestra.run.smithi016.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 366 kB/s | 18 kB 00:00 2024-09-19T07:40:54.341 INFO:teuthology.orchestra.run.smithi016.stdout:(38/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 11 MB/s | 49 MB 00:04 2024-09-19T07:40:54.415 INFO:teuthology.orchestra.run.smithi146.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.2 MB/s | 4.9 MB 00:01 2024-09-19T07:40:54.520 INFO:teuthology.orchestra.run.smithi146.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2024-09-19T07:40:54.533 INFO:teuthology.orchestra.run.smithi016.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 5.5 MB/s | 7.4 MB 00:01 2024-09-19T07:40:54.557 INFO:teuthology.orchestra.run.smithi146.stdout:(69/156): postgresql-13.16-1.el9.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 2024-09-19T07:40:54.558 INFO:teuthology.orchestra.run.smithi016.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 99 kB/s | 80 kB 00:00 2024-09-19T07:40:54.650 INFO:teuthology.orchestra.run.smithi016.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 191 kB 00:00 2024-09-19T07:40:54.675 INFO:teuthology.orchestra.run.smithi016.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.3 MB/s | 801 kB 00:00 2024-09-19T07:40:54.725 INFO:teuthology.orchestra.run.smithi016.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-19T07:40:54.784 INFO:teuthology.orchestra.run.smithi016.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-19T07:40:54.809 INFO:teuthology.orchestra.run.smithi016.stdout:(45/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-09-19T07:40:54.835 INFO:teuthology.orchestra.run.smithi016.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-19T07:40:54.868 INFO:teuthology.orchestra.run.smithi016.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 5.6 MB/s | 1.2 MB 00:00 2024-09-19T07:40:54.919 INFO:teuthology.orchestra.run.smithi016.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-09-19T07:40:55.085 INFO:teuthology.orchestra.run.smithi016.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 642 kB/s | 107 kB 00:00 2024-09-19T07:40:55.111 INFO:teuthology.orchestra.run.smithi016.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-19T07:40:55.136 INFO:teuthology.orchestra.run.smithi016.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 431 kB/s | 22 kB 00:00 2024-09-19T07:40:55.178 INFO:teuthology.orchestra.run.smithi016.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 943 kB 00:00 2024-09-19T07:40:55.207 INFO:teuthology.orchestra.run.smithi146.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 200 kB/s | 137 kB 00:00 2024-09-19T07:40:55.220 INFO:teuthology.orchestra.run.smithi016.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 391 kB/s | 33 kB 00:00 2024-09-19T07:40:55.254 INFO:teuthology.orchestra.run.smithi016.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.1 MB/s | 313 kB 00:00 2024-09-19T07:40:55.270 INFO:teuthology.orchestra.run.smithi016.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-19T07:40:55.296 INFO:teuthology.orchestra.run.smithi016.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-09-19T07:40:55.338 INFO:teuthology.orchestra.run.smithi016.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-19T07:40:55.341 INFO:teuthology.orchestra.run.smithi146.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 983 kB/s | 910 kB 00:00 2024-09-19T07:40:55.363 INFO:teuthology.orchestra.run.smithi016.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-19T07:40:55.455 INFO:teuthology.orchestra.run.smithi016.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-09-19T07:40:55.481 INFO:teuthology.orchestra.run.smithi016.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-09-19T07:40:55.514 INFO:teuthology.orchestra.run.smithi016.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 3.8 MB/s | 581 kB 00:00 2024-09-19T07:40:55.539 INFO:teuthology.orchestra.run.smithi016.stdout:(62/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 291 kB/s | 24 kB 00:00 2024-09-19T07:40:55.564 INFO:teuthology.orchestra.run.smithi016.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-09-19T07:40:55.598 INFO:teuthology.orchestra.run.smithi016.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-19T07:40:55.632 INFO:teuthology.orchestra.run.smithi016.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 572 kB/s | 38 kB 00:00 2024-09-19T07:40:55.666 INFO:teuthology.orchestra.run.smithi016.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-19T07:40:55.700 INFO:teuthology.orchestra.run.smithi146.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 5.1 MB/s | 5.8 MB 00:01 2024-09-19T07:40:55.825 INFO:teuthology.orchestra.run.smithi016.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-09-19T07:40:55.925 INFO:teuthology.orchestra.run.smithi016.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-19T07:40:55.991 INFO:teuthology.orchestra.run.smithi146.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 134 kB/s | 87 kB 00:00 2024-09-19T07:40:55.992 INFO:teuthology.orchestra.run.smithi016.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 13 MB/s | 4.9 MB 00:00 2024-09-19T07:40:56.017 INFO:teuthology.orchestra.run.smithi016.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-19T07:40:56.050 INFO:teuthology.orchestra.run.smithi016.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 3.9 MB/s | 910 kB 00:00 2024-09-19T07:40:56.118 INFO:teuthology.orchestra.run.smithi016.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-09-19T07:40:56.143 INFO:teuthology.orchestra.run.smithi016.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 951 kB/s | 87 kB 00:00 2024-09-19T07:40:56.150 INFO:teuthology.orchestra.run.smithi146.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2024-09-19T07:40:56.193 INFO:teuthology.orchestra.run.smithi016.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-19T07:40:56.260 INFO:teuthology.orchestra.run.smithi016.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-09-19T07:40:56.327 INFO:teuthology.orchestra.run.smithi016.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-09-19T07:40:56.402 INFO:teuthology.orchestra.run.smithi016.stdout:(77/156): postgresql-server-13.16-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-09-19T07:40:56.427 INFO:teuthology.orchestra.run.smithi016.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-09-19T07:40:56.453 INFO:teuthology.orchestra.run.smithi016.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-19T07:40:56.478 INFO:teuthology.orchestra.run.smithi016.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-19T07:40:56.553 INFO:teuthology.orchestra.run.smithi016.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-19T07:40:56.579 INFO:teuthology.orchestra.run.smithi016.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 277 kB/s | 35 kB 00:00 2024-09-19T07:40:56.612 INFO:teuthology.orchestra.run.smithi016.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-19T07:40:56.625 INFO:teuthology.orchestra.run.smithi146.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 97 kB/s | 61 kB 00:00 2024-09-19T07:40:56.638 INFO:teuthology.orchestra.run.smithi016.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-19T07:40:56.663 INFO:teuthology.orchestra.run.smithi016.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-19T07:40:56.688 INFO:teuthology.orchestra.run.smithi016.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-19T07:40:56.730 INFO:teuthology.orchestra.run.smithi016.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-09-19T07:40:56.756 INFO:teuthology.orchestra.run.smithi016.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-19T07:40:56.789 INFO:teuthology.orchestra.run.smithi016.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-19T07:40:56.806 INFO:teuthology.orchestra.run.smithi016.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-19T07:40:56.890 INFO:teuthology.orchestra.run.smithi016.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-19T07:40:56.907 INFO:teuthology.orchestra.run.smithi016.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 458 kB/s | 54 kB 00:00 2024-09-19T07:40:56.909 INFO:teuthology.orchestra.run.smithi146.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.9 MB/s | 6.0 MB 00:01 2024-09-19T07:40:56.940 INFO:teuthology.orchestra.run.smithi016.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-09-19T07:40:56.942 INFO:teuthology.orchestra.run.smithi146.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 309 kB/s | 245 kB 00:00 2024-09-19T07:40:56.966 INFO:teuthology.orchestra.run.smithi016.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 750 kB/s | 44 kB 00:00 2024-09-19T07:40:56.999 INFO:teuthology.orchestra.run.smithi016.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-19T07:40:57.016 INFO:teuthology.orchestra.run.smithi016.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-09-19T07:40:57.041 INFO:teuthology.orchestra.run.smithi016.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-19T07:40:57.075 INFO:teuthology.orchestra.run.smithi016.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-19T07:40:57.092 INFO:teuthology.orchestra.run.smithi016.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-19T07:40:57.134 INFO:teuthology.orchestra.run.smithi016.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-19T07:40:57.151 INFO:teuthology.orchestra.run.smithi016.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-19T07:40:57.176 INFO:teuthology.orchestra.run.smithi016.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-09-19T07:40:57.227 INFO:teuthology.orchestra.run.smithi016.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-09-19T07:40:57.294 INFO:teuthology.orchestra.run.smithi016.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 8.1 MB/s | 548 kB 00:00 2024-09-19T07:40:57.338 INFO:teuthology.orchestra.run.smithi016.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 7.3 MB/s | 308 kB 00:00 2024-09-19T07:40:57.361 INFO:teuthology.orchestra.run.smithi016.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 782 kB/s | 19 kB 00:00 2024-09-19T07:40:57.403 INFO:teuthology.orchestra.run.smithi016.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-09-19T07:40:57.426 INFO:teuthology.orchestra.run.smithi146.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 40 kB/s | 19 kB 00:00 2024-09-19T07:40:57.468 INFO:teuthology.orchestra.run.smithi146.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 47 kB/s | 26 kB 00:00 2024-09-19T07:40:57.510 INFO:teuthology.orchestra.run.smithi146.stdout:(80/156): python3-jinja2-2.11.3-6.el9.noarch.rp 281 kB/s | 248 kB 00:00 2024-09-19T07:40:57.587 INFO:teuthology.orchestra.run.smithi016.stdout:(108/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-09-19T07:40:57.620 INFO:teuthology.orchestra.run.smithi016.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 120 kB/s | 26 kB 00:00 2024-09-19T07:40:57.646 INFO:teuthology.orchestra.run.smithi016.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 866 kB/s | 49 kB 00:00 2024-09-19T07:40:57.671 INFO:teuthology.orchestra.run.smithi016.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.4 MB/s | 67 kB 00:00 2024-09-19T07:40:57.697 INFO:teuthology.orchestra.run.smithi016.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-19T07:40:57.747 INFO:teuthology.orchestra.run.smithi016.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-19T07:40:57.772 INFO:teuthology.orchestra.run.smithi016.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T07:40:57.814 INFO:teuthology.orchestra.run.smithi016.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.7 MB/s | 839 kB 00:00 2024-09-19T07:40:57.839 INFO:teuthology.orchestra.run.smithi016.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 481 kB/s | 32 kB 00:00 2024-09-19T07:40:57.864 INFO:teuthology.orchestra.run.smithi016.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-19T07:40:57.890 INFO:teuthology.orchestra.run.smithi016.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-19T07:40:57.932 INFO:teuthology.orchestra.run.smithi016.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-19T07:40:57.957 INFO:teuthology.orchestra.run.smithi016.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-19T07:40:57.982 INFO:teuthology.orchestra.run.smithi016.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T07:40:58.027 INFO:teuthology.orchestra.run.smithi146.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 62 kB/s | 35 kB 00:00 2024-09-19T07:40:58.050 INFO:teuthology.orchestra.run.smithi016.stdout:(122/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-09-19T07:40:58.075 INFO:teuthology.orchestra.run.smithi016.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-09-19T07:40:58.100 INFO:teuthology.orchestra.run.smithi016.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-19T07:40:58.142 INFO:teuthology.orchestra.run.smithi016.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-09-19T07:40:58.160 INFO:teuthology.orchestra.run.smithi146.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 235 kB/s | 172 kB 00:00 2024-09-19T07:40:58.167 INFO:teuthology.orchestra.run.smithi016.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 347 kB/s | 23 kB 00:00 2024-09-19T07:40:58.193 INFO:teuthology.orchestra.run.smithi016.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-19T07:40:58.226 INFO:teuthology.orchestra.run.smithi016.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 337 kB/s | 20 kB 00:00 2024-09-19T07:40:58.252 INFO:teuthology.orchestra.run.smithi016.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T07:40:58.285 INFO:teuthology.orchestra.run.smithi016.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-19T07:40:58.327 INFO:teuthology.orchestra.run.smithi016.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-09-19T07:40:58.352 INFO:teuthology.orchestra.run.smithi016.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 862 kB/s | 58 kB 00:00 2024-09-19T07:40:58.378 INFO:teuthology.orchestra.run.smithi016.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-09-19T07:40:58.403 INFO:teuthology.orchestra.run.smithi016.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-09-19T07:40:58.428 INFO:teuthology.orchestra.run.smithi016.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-19T07:40:58.445 INFO:teuthology.orchestra.run.smithi016.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 672 kB/s | 28 kB 00:00 2024-09-19T07:40:58.462 INFO:teuthology.orchestra.run.smithi016.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 913 kB/s | 31 kB 00:00 2024-09-19T07:40:58.513 INFO:teuthology.orchestra.run.smithi016.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.7 MB/s | 2.0 MB 00:00 2024-09-19T07:40:58.530 INFO:teuthology.orchestra.run.smithi016.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 885 kB/s | 59 kB 00:00 2024-09-19T07:40:58.547 INFO:teuthology.orchestra.run.smithi016.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-09-19T07:40:58.572 INFO:teuthology.orchestra.run.smithi016.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-19T07:40:58.597 INFO:teuthology.orchestra.run.smithi016.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-19T07:40:58.619 INFO:teuthology.orchestra.run.smithi146.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.6 MB/s | 5.1 MB 00:01 2024-09-19T07:40:58.622 INFO:teuthology.orchestra.run.smithi016.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-19T07:40:58.648 INFO:teuthology.orchestra.run.smithi016.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-19T07:40:58.673 INFO:teuthology.orchestra.run.smithi016.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-19T07:40:58.698 INFO:teuthology.orchestra.run.smithi016.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T07:40:58.724 INFO:teuthology.orchestra.run.smithi016.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-19T07:40:58.777 INFO:teuthology.orchestra.run.smithi146.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 334 kB/s | 250 kB 00:00 2024-09-19T07:40:58.833 INFO:teuthology.orchestra.run.smithi016.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 9.9 MB/s | 1.6 MB 00:00 2024-09-19T07:40:58.911 INFO:teuthology.orchestra.run.smithi146.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 296 kB/s | 222 kB 00:00 2024-09-19T07:40:59.183 INFO:teuthology.orchestra.run.smithi016.stdout:(149/156): librbd1-19.1.1-361.g2f599a46.el9.x86 6.9 MB/s | 3.2 MB 00:00 2024-09-19T07:40:59.241 INFO:teuthology.orchestra.run.smithi016.stdout:(150/156): librados2-19.1.1-361.g2f599a46.el9.x 6.3 MB/s | 3.4 MB 00:00 2024-09-19T07:40:59.278 INFO:teuthology.orchestra.run.smithi146.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 117 kB/s | 77 kB 00:00 2024-09-19T07:40:59.345 INFO:teuthology.orchestra.run.smithi146.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 75 kB/s | 42 kB 00:00 2024-09-19T07:40:59.387 INFO:teuthology.orchestra.run.smithi016.stdout:(151/156): NetworkManager-1.51.0-1.el9.x86_64.r 4.2 MB/s | 2.3 MB 00:00 2024-09-19T07:40:59.412 INFO:teuthology.orchestra.run.smithi016.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 240 kB/s | 41 kB 00:00 2024-09-19T07:40:59.537 INFO:teuthology.orchestra.run.smithi016.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 1.6 MB/s | 249 kB 00:00 2024-09-19T07:40:59.637 INFO:teuthology.orchestra.run.smithi016.stdout:(154/156): NetworkManager-libnm-1.51.0-1.el9.x8 4.0 MB/s | 1.8 MB 00:00 2024-09-19T07:40:59.688 INFO:teuthology.orchestra.run.smithi016.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 4.9 MB/s | 1.4 MB 00:00 2024-09-19T07:40:59.745 INFO:teuthology.orchestra.run.smithi146.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 323 kB/s | 269 kB 00:00 2024-09-19T07:41:00.020 INFO:teuthology.orchestra.run.smithi146.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 214 kB/s | 159 kB 00:00 2024-09-19T07:41:00.129 INFO:teuthology.orchestra.run.smithi146.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 356 kB/s | 279 kB 00:00 2024-09-19T07:41:00.395 INFO:teuthology.orchestra.run.smithi146.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 78 kB/s | 51 kB 00:00 2024-09-19T07:41:00.629 INFO:teuthology.orchestra.run.smithi146.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 89 kB/s | 54 kB 00:00 2024-09-19T07:41:00.763 INFO:teuthology.orchestra.run.smithi016.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 2.4 MB 00:01 2024-09-19T07:41:00.769 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-19T07:41:00.769 INFO:teuthology.orchestra.run.smithi016.stdout:Total 15 MB/s | 226 MB 00:15 2024-09-19T07:41:00.980 INFO:teuthology.orchestra.run.smithi146.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 71 kB/s | 42 kB 00:00 2024-09-19T07:41:01.246 INFO:teuthology.orchestra.run.smithi146.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 72 kB/s | 44 kB 00:00 2024-09-19T07:41:01.555 INFO:teuthology.orchestra.run.smithi146.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-19T07:41:01.822 INFO:teuthology.orchestra.run.smithi146.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 262 kB/s | 221 kB 00:00 2024-09-19T07:41:01.922 INFO:teuthology.orchestra.run.smithi146.stdout:(97/156): qatlib-service-24.02.0-1.el9.x86_64.r 53 kB/s | 36 kB 00:00 2024-09-19T07:41:02.036 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-09-19T07:41:02.131 INFO:teuthology.orchestra.run.smithi146.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 84 kB/s | 47 kB 00:00 2024-09-19T07:41:02.208 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-09-19T07:41:02.208 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-09-19T07:41:02.523 INFO:teuthology.orchestra.run.smithi146.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 96 kB/s | 58 kB 00:00 2024-09-19T07:41:02.598 INFO:teuthology.orchestra.run.smithi146.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 392 kB/s | 304 kB 00:00 2024-09-19T07:41:02.848 INFO:teuthology.orchestra.run.smithi146.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 134 kB/s | 96 kB 00:00 2024-09-19T07:41:02.931 INFO:teuthology.orchestra.run.smithi146.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 298 kB/s | 22 kB 00:00 2024-09-19T07:41:03.157 INFO:teuthology.orchestra.run.smithi146.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.8 MB/s | 863 kB 00:00 2024-09-19T07:41:03.207 INFO:teuthology.orchestra.run.smithi146.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-19T07:41:03.232 INFO:teuthology.orchestra.run.smithi146.stdout:(105/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 90 kB/s | 64 kB 00:00 2024-09-19T07:41:03.249 INFO:teuthology.orchestra.run.smithi146.stdout:(106/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 69 kB/s | 45 kB 00:00 2024-09-19T07:41:03.283 INFO:teuthology.orchestra.run.smithi146.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 396 kB/s | 19 kB 00:00 2024-09-19T07:41:03.299 INFO:teuthology.orchestra.run.smithi146.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.7 MB/s | 26 kB 00:00 2024-09-19T07:41:03.325 INFO:teuthology.orchestra.run.smithi146.stdout:(109/156): gperftools-libs-2.9.1-3.el9.x86_64.r 2.6 MB/s | 308 kB 00:00 2024-09-19T07:41:03.350 INFO:teuthology.orchestra.run.smithi146.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 967 kB/s | 49 kB 00:00 2024-09-19T07:41:03.375 INFO:teuthology.orchestra.run.smithi146.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-19T07:41:03.401 INFO:teuthology.orchestra.run.smithi146.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-19T07:41:03.468 INFO:teuthology.orchestra.run.smithi146.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-09-19T07:41:03.493 INFO:teuthology.orchestra.run.smithi146.stdout:(114/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-19T07:41:03.518 INFO:teuthology.orchestra.run.smithi146.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 872 kB/s | 43 kB 00:00 2024-09-19T07:41:03.544 INFO:teuthology.orchestra.run.smithi146.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-09-19T07:41:03.569 INFO:teuthology.orchestra.run.smithi146.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-19T07:41:03.594 INFO:teuthology.orchestra.run.smithi146.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-19T07:41:03.628 INFO:teuthology.orchestra.run.smithi146.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-19T07:41:03.662 INFO:teuthology.orchestra.run.smithi146.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-19T07:41:03.686 INFO:teuthology.orchestra.run.smithi146.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-19T07:41:03.720 INFO:teuthology.orchestra.run.smithi146.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-19T07:41:03.745 INFO:teuthology.orchestra.run.smithi146.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-19T07:41:03.771 INFO:teuthology.orchestra.run.smithi146.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-09-19T07:41:03.796 INFO:teuthology.orchestra.run.smithi146.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 934 kB/s | 23 kB 00:00 2024-09-19T07:41:03.821 INFO:teuthology.orchestra.run.smithi146.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 782 kB/s | 19 kB 00:00 2024-09-19T07:41:03.847 INFO:teuthology.orchestra.run.smithi146.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 791 kB/s | 20 kB 00:00 2024-09-19T07:41:03.897 INFO:teuthology.orchestra.run.smithi146.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-09-19T07:41:03.964 INFO:teuthology.orchestra.run.smithi146.stdout:(129/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.2 MB/s | 4.4 MB 00:00 2024-09-19T07:41:03.980 INFO:teuthology.orchestra.run.smithi146.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 555 kB/s | 46 kB 00:00 2024-09-19T07:41:04.006 INFO:teuthology.orchestra.run.smithi146.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-09-19T07:41:04.023 INFO:teuthology.orchestra.run.smithi146.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-09-19T07:41:04.040 INFO:teuthology.orchestra.run.smithi146.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 996 kB/s | 16 kB 00:00 2024-09-19T07:41:04.065 INFO:teuthology.orchestra.run.smithi146.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-19T07:41:04.090 INFO:teuthology.orchestra.run.smithi146.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-19T07:41:04.107 INFO:teuthology.orchestra.run.smithi146.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 674 kB/s | 28 kB 00:00 2024-09-19T07:41:04.124 INFO:teuthology.orchestra.run.smithi146.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 923 kB/s | 31 kB 00:00 2024-09-19T07:41:04.158 INFO:teuthology.orchestra.run.smithi146.stdout:(138/156): python3-kubernetes-26.1.0-2.el9.noar 3.3 MB/s | 1.0 MB 00:00 2024-09-19T07:41:04.183 INFO:teuthology.orchestra.run.smithi146.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-19T07:41:04.208 INFO:teuthology.orchestra.run.smithi146.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-19T07:41:04.233 INFO:teuthology.orchestra.run.smithi146.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-19T07:41:04.275 INFO:teuthology.orchestra.run.smithi146.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 809 kB/s | 74 kB 00:00 2024-09-19T07:41:04.301 INFO:teuthology.orchestra.run.smithi146.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-09-19T07:41:04.317 INFO:teuthology.orchestra.run.smithi146.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-19T07:41:04.351 INFO:teuthology.orchestra.run.smithi146.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-09-19T07:41:04.376 INFO:teuthology.orchestra.run.smithi146.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-19T07:41:04.402 INFO:teuthology.orchestra.run.smithi146.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-19T07:41:04.452 INFO:teuthology.orchestra.run.smithi146.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-19T07:41:04.564 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-09-19T07:41:04.564 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-09-19T07:41:04.636 INFO:teuthology.orchestra.run.smithi146.stdout:(149/156): NetworkManager-1.51.0-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-09-19T07:41:04.711 INFO:teuthology.orchestra.run.smithi146.stdout:(150/156): librbd1-19.1.1-361.g2f599a46.el9.x86 10 MB/s | 3.2 MB 00:00 2024-09-19T07:41:04.769 INFO:teuthology.orchestra.run.smithi146.stdout:(151/156): librados2-19.1.1-361.g2f599a46.el9.x 8.7 MB/s | 3.4 MB 00:00 2024-09-19T07:41:04.803 INFO:teuthology.orchestra.run.smithi146.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 447 kB/s | 41 kB 00:00 2024-09-19T07:41:04.845 INFO:teuthology.orchestra.run.smithi146.stdout:(153/156): NetworkManager-libnm-1.51.0-1.el9.x8 8.7 MB/s | 1.8 MB 00:00 2024-09-19T07:41:04.878 INFO:teuthology.orchestra.run.smithi146.stdout:(154/156): NetworkManager-tui-1.51.0-1.el9.x86_ 2.2 MB/s | 249 kB 00:00 2024-09-19T07:41:04.946 INFO:teuthology.orchestra.run.smithi146.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-09-19T07:41:04.987 INFO:teuthology.orchestra.run.smithi146.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 7.4 MB/s | 1.4 MB 00:00 2024-09-19T07:41:04.993 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-19T07:41:04.994 INFO:teuthology.orchestra.run.smithi146.stdout:Total 11 MB/s | 226 MB 00:20 2024-09-19T07:41:06.169 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-19T07:41:06.321 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-19T07:41:06.322 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-19T07:41:08.534 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-19T07:41:08.535 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-19T07:41:11.326 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-09-19T07:41:11.426 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T07:41:11.468 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T07:41:11.544 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T07:41:11.585 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T07:41:11.610 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T07:41:11.674 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T07:41:12.018 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T07:41:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:41:12.245 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:41:12.270 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:41:12.364 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:41:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T07:41:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T07:41:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T07:41:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:41:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:41:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T07:41:12.848 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T07:41:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T07:41:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T07:41:13.223 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T07:41:13.408 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T07:41:13.557 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T07:41:13.662 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:41:13.763 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:41:13.996 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:41:14.139 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T07:41:14.241 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T07:41:14.282 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T07:41:14.326 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T07:41:14.403 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T07:41:14.453 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T07:41:14.493 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T07:41:14.537 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T07:41:14.570 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T07:41:14.597 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T07:41:14.669 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T07:41:14.729 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T07:41:14.759 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T07:41:14.802 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T07:41:14.880 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T07:41:14.913 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T07:41:14.957 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T07:41:14.985 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T07:41:15.007 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-19T07:41:15.059 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T07:41:15.109 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T07:41:15.141 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T07:41:15.180 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T07:41:15.195 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T07:41:15.233 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T07:41:15.266 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T07:41:15.292 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T07:41:15.340 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T07:41:15.362 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T07:41:15.404 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T07:41:15.749 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T07:41:15.817 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:41:15.922 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:41:15.958 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T07:41:15.959 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:41:16.042 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T07:41:16.051 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:41:16.077 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T07:41:16.081 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T07:41:16.121 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T07:41:16.159 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T07:41:16.219 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T07:41:16.259 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T07:41:16.325 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T07:41:16.355 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:41:16.379 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T07:41:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T07:41:16.460 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:41:16.485 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T07:41:16.489 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T07:41:16.552 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T07:41:16.558 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T07:41:16.582 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T07:41:16.604 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T07:41:16.605 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T07:41:16.640 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T07:41:16.688 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T07:41:16.723 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T07:41:16.767 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T07:41:16.801 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T07:41:16.809 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T07:41:16.952 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T07:41:16.964 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T07:41:17.088 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T07:41:17.169 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T07:41:17.205 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:41:17.288 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T07:41:17.315 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:41:17.400 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T07:41:17.436 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T07:41:17.493 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T07:41:17.526 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T07:41:17.545 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:41:17.550 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T07:41:17.642 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T07:41:17.661 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T07:41:17.756 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T07:41:17.778 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T07:41:17.795 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T07:41:17.812 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T07:41:17.829 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T07:41:17.852 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T07:41:17.893 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T07:41:17.899 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T07:41:17.938 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T07:41:17.939 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T07:41:17.970 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T07:41:17.987 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T07:41:18.014 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T07:41:18.025 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T07:41:18.046 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T07:41:18.068 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T07:41:18.074 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T07:41:18.101 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T07:41:18.143 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T07:41:18.144 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T07:41:18.177 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T07:41:18.196 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T07:41:18.236 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T07:41:18.266 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T07:41:18.279 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T07:41:18.317 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T07:41:18.347 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T07:41:18.355 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T07:41:18.389 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T07:41:18.433 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T07:41:18.461 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T07:41:18.464 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T07:41:18.525 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T07:41:18.531 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T07:41:18.584 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T07:41:18.659 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T07:41:18.798 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T07:41:18.861 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T07:41:19.288 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T07:41:19.329 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T07:41:19.363 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T07:41:19.378 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T07:41:19.465 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T07:41:19.465 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T07:41:19.494 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T07:41:19.514 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T07:41:19.537 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T07:41:19.559 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T07:41:19.575 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T07:41:19.618 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T07:41:19.649 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T07:41:19.694 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T07:41:19.773 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T07:41:19.827 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T07:41:19.881 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T07:41:19.886 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T07:41:19.917 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T07:41:19.954 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T07:41:19.987 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T07:41:19.997 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T07:41:20.023 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T07:41:20.069 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T07:41:20.105 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T07:41:20.148 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T07:41:20.215 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T07:41:20.351 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T07:41:20.400 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T07:41:20.526 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T07:41:20.627 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T07:41:20.659 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T07:41:20.693 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T07:41:20.699 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T07:41:20.732 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T07:41:20.757 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T07:41:20.819 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T07:41:20.859 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T07:41:20.985 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T07:41:21.014 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T07:41:21.051 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T07:41:21.085 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T07:41:21.123 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T07:41:21.155 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T07:41:21.185 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T07:41:21.211 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T07:41:21.244 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T07:41:21.294 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T07:41:21.329 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T07:41:21.415 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T07:41:21.459 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T07:41:21.493 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T07:41:21.596 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T07:41:21.644 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T07:41:22.154 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T07:41:22.238 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T07:41:22.266 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T07:41:22.299 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T07:41:22.369 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T07:41:22.416 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T07:41:22.450 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T07:41:22.557 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T07:41:22.613 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T07:41:22.634 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T07:41:22.658 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T07:41:22.679 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T07:41:22.753 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T07:41:22.779 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T07:41:22.786 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T07:41:22.810 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T07:41:22.854 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T07:41:22.970 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T07:41:23.009 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T07:41:23.086 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T07:41:23.363 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T07:41:23.390 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T07:41:23.454 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T07:41:23.484 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T07:41:23.737 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T07:41:23.840 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T07:41:25.105 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T07:41:25.194 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T07:41:25.214 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T07:41:25.238 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T07:41:25.549 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T07:41:25.593 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T07:41:25.676 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T07:41:25.706 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T07:41:25.747 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T07:41:25.942 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:41:25.980 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:41:26.227 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T07:41:26.352 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T07:41:26.352 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T07:41:26.364 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T07:41:26.565 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:41:26.609 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T07:41:26.701 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T07:41:26.701 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T07:41:26.701 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T07:41:26.701 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:41:26.746 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T07:41:28.690 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:41:28.727 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:41:29.274 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:41:29.314 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T07:41:29.394 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T07:41:29.394 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T07:41:29.394 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T07:41:29.394 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:41:29.432 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T07:41:39.098 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T07:41:39.099 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-09-19T07:41:39.099 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-09-19T07:41:39.099 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-09-19T07:41:39.099 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-09-19T07:41:39.099 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-09-19T07:41:39.099 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2024-09-19T07:41:39.099 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-09-19T07:41:39.099 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:41:39.276 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T07:41:39.314 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T07:41:39.314 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:41:39.314 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T07:41:39.315 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:41:40.846 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T07:41:40.874 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T07:41:40.874 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:41:40.874 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T07:41:40.874 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T07:41:40.875 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T07:41:40.875 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:41:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T07:41:40.916 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /sys 2024-09-19T07:41:40.916 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /proc 2024-09-19T07:41:40.916 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /mnt 2024-09-19T07:41:40.917 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /var/tmp 2024-09-19T07:41:40.917 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /home 2024-09-19T07:41:40.917 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /root 2024-09-19T07:41:40.917 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /tmp 2024-09-19T07:41:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:41:41.091 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T07:41:41.106 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T07:41:41.107 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:41:41.107 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T07:41:41.107 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:41:41.113 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T07:41:41.138 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T07:41:41.138 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:41:41.138 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T07:41:41.138 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T07:41:41.139 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T07:41:41.139 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:41:41.606 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T07:41:41.631 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T07:41:41.631 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:41:41.631 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T07:41:41.631 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T07:41:41.631 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T07:41:41.631 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:41:41.745 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T07:41:41.803 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T07:41:41.867 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T07:41:42.062 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T07:41:42.124 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T07:41:42.204 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T07:41:42.305 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T07:41:42.444 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T07:41:42.492 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T07:41:42.522 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T07:41:42.550 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T07:41:42.576 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T07:41:42.604 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T07:41:42.604 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:41:42.604 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T07:41:42.604 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T07:41:42.604 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T07:41:42.604 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:41:42.846 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T07:41:42.869 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T07:41:42.869 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:41:42.870 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T07:41:42.870 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T07:41:42.870 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T07:41:42.870 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:41:42.999 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T07:41:43.024 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T07:41:43.065 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T07:41:43.129 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T07:41:43.155 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T07:41:43.323 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T07:41:43.346 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T07:41:43.347 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:41:43.347 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T07:41:43.347 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T07:41:43.347 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T07:41:43.347 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:41:43.444 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T07:41:43.501 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T07:41:43.556 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T07:41:43.752 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T07:41:43.798 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T07:41:43.876 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T07:41:43.966 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T07:41:44.105 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T07:41:44.163 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T07:41:44.197 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T07:41:44.224 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T07:41:44.267 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T07:41:44.299 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T07:41:44.653 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T07:41:44.681 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T07:41:44.712 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T07:41:44.775 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T07:41:44.804 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T07:41:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T07:41:45.341 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T07:41:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T07:41:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T07:41:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T07:41:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T07:41:45.677 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:41:45.677 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T07:41:45.677 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T07:41:45.677 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T07:41:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:41:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T07:41:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T07:41:45.842 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T07:41:45.851 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T07:41:45.893 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T07:41:45.905 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T07:41:46.852 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T07:41:46.873 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T07:41:46.880 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T07:41:46.922 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T07:41:46.948 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T07:41:46.948 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:41:46.948 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T07:41:46.948 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T07:41:46.948 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T07:41:46.948 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:41:46.976 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T07:41:46.989 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T07:41:47.013 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T07:41:47.013 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:41:47.014 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T07:41:47.014 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:41:47.065 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T07:41:47.089 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T07:41:47.089 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:41:47.089 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T07:41:47.089 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T07:41:47.090 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T07:41:47.090 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:41:47.135 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T07:41:47.166 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T07:41:47.191 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T07:41:47.191 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:41:47.191 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T07:41:47.191 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T07:41:47.191 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T07:41:47.191 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:41:47.242 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T07:41:47.254 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T07:41:47.358 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T07:41:47.407 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T07:41:47.410 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T07:41:47.432 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T07:41:47.432 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:41:47.432 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T07:41:47.433 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T07:41:47.433 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T07:41:47.433 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:41:48.331 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T07:41:48.362 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T07:41:48.384 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T07:41:48.385 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:41:48.385 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T07:41:48.385 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T07:41:48.385 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T07:41:48.385 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:41:48.428 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T07:41:48.451 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T07:41:48.451 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:41:48.451 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T07:41:48.452 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:41:48.496 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T07:41:48.519 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T07:41:48.519 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:41:48.520 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T07:41:48.520 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T07:41:48.520 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T07:41:48.520 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:41:48.830 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T07:41:48.853 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T07:41:48.853 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:41:48.853 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T07:41:48.853 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T07:41:48.853 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T07:41:48.853 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:41:52.287 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T07:41:52.337 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T07:41:52.378 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T07:41:52.410 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T07:41:52.437 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T07:41:52.653 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T07:41:52.701 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T07:41:52.731 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T07:41:52.766 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T07:41:52.815 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T07:41:53.400 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:41:53.409 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T07:41:53.447 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:41:53.450 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T07:41:53.468 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:41:53.491 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T07:41:53.523 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T07:41:53.561 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T07:41:53.763 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T07:41:53.814 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T07:41:53.844 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T07:41:53.876 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T07:41:53.920 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T07:41:54.368 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T07:41:54.452 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T07:41:54.505 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T07:41:54.537 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:41:54.575 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:41:54.596 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:41:54.599 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T07:41:54.630 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T07:41:54.656 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T07:41:54.690 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T07:41:54.690 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T07:41:54.783 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T07:41:54.783 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T07:41:54.875 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T07:41:54.900 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T07:41:54.909 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:41:54.909 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:41:55.093 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:41:55.094 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T07:41:55.186 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T07:41:55.263 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:41:55.435 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T07:41:55.521 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T07:41:55.571 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T07:41:55.661 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T07:41:55.691 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T07:41:55.716 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T07:41:55.751 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T07:41:55.751 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T07:41:55.843 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T07:41:55.843 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T07:41:55.927 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T07:41:55.993 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T07:41:55.998 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:41:55.999 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:41:56.187 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:41:56.187 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T07:41:56.280 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T07:41:56.344 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:42:00.871 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:42:00.871 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T07:42:00.871 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T07:42:00.871 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T07:42:00.871 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T07:42:00.871 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T07:42:00.871 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T07:42:00.872 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T07:42:00.872 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:42:00.872 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:42:00.872 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T07:42:00.872 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T07:42:00.872 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T07:42:00.872 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:42:00.872 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T07:42:00.872 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T07:42:00.873 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T07:42:00.873 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T07:42:00.873 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T07:42:00.873 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T07:42:00.873 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T07:42:00.873 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T07:42:00.873 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T07:42:00.873 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T07:42:00.873 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T07:42:00.874 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T07:42:00.874 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T07:42:00.874 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T07:42:00.874 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T07:42:00.874 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T07:42:00.887 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T07:42:00.887 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T07:42:00.887 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T07:42:00.887 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T07:42:00.887 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T07:42:00.887 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T07:42:00.887 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T07:42:00.887 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T07:42:00.887 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T07:42:00.888 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T07:42:00.888 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T07:42:00.888 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T07:42:00.888 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T07:42:00.888 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T07:42:00.888 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T07:42:00.888 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T07:42:00.888 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T07:42:00.888 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T07:42:00.888 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T07:42:00.888 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T07:42:00.889 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T07:42:00.889 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T07:42:00.889 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T07:42:00.889 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T07:42:00.889 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T07:42:00.889 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T07:42:00.889 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T07:42:00.889 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T07:42:00.889 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T07:42:00.889 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T07:42:00.889 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T07:42:00.890 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T07:42:00.890 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T07:42:00.890 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T07:42:00.890 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T07:42:00.890 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T07:42:00.890 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T07:42:00.890 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T07:42:00.890 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T07:42:00.890 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T07:42:00.890 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T07:42:00.890 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T07:42:00.890 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T07:42:00.890 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T07:42:00.890 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T07:42:00.891 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T07:42:00.891 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T07:42:00.891 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T07:42:00.891 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T07:42:00.891 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T07:42:00.891 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T07:42:00.891 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T07:42:00.891 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T07:42:00.891 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T07:42:00.891 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T07:42:00.891 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T07:42:00.891 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T07:42:00.891 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T07:42:00.892 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T07:42:00.892 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T07:42:00.892 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T07:42:00.892 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T07:42:00.892 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T07:42:00.892 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T07:42:00.892 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T07:42:00.892 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T07:42:00.892 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T07:42:00.892 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T07:42:00.892 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T07:42:00.892 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T07:42:00.892 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T07:42:00.893 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T07:42:00.893 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T07:42:00.893 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T07:42:00.893 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T07:42:00.893 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T07:42:00.893 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T07:42:00.893 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T07:42:00.893 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T07:42:00.893 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T07:42:00.893 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T07:42:00.893 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T07:42:00.893 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T07:42:00.893 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T07:42:00.894 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T07:42:00.894 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T07:42:00.894 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T07:42:00.894 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T07:42:00.894 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T07:42:00.894 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T07:42:00.894 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T07:42:00.894 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T07:42:00.895 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T07:42:00.895 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T07:42:00.895 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T07:42:00.895 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T07:42:00.895 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T07:42:00.895 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T07:42:00.895 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T07:42:00.895 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T07:42:00.895 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T07:42:00.895 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T07:42:00.896 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T07:42:00.896 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T07:42:00.896 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T07:42:00.896 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T07:42:00.896 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T07:42:00.896 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T07:42:00.896 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T07:42:00.896 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T07:42:00.896 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T07:42:00.896 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T07:42:00.896 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T07:42:00.897 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T07:42:00.897 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T07:42:00.897 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T07:42:00.897 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T07:42:00.897 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T07:42:00.897 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T07:42:00.897 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T07:42:00.897 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T07:42:00.897 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T07:42:00.897 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T07:42:00.897 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T07:42:00.897 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T07:42:00.898 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T07:42:00.898 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T07:42:00.898 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T07:42:00.898 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T07:42:00.898 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T07:42:00.898 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T07:42:00.898 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T07:42:00.898 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T07:42:00.898 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T07:42:01.906 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:42:01.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T07:42:01.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T07:42:01.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T07:42:01.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T07:42:01.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T07:42:01.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T07:42:01.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T07:42:01.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:42:01.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:42:01.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T07:42:01.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T07:42:01.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T07:42:01.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:42:01.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T07:42:01.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T07:42:01.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T07:42:01.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T07:42:01.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T07:42:01.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T07:42:01.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T07:42:01.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T07:42:01.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T07:42:01.909 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T07:42:01.909 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T07:42:01.909 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T07:42:01.909 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T07:42:01.909 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T07:42:01.922 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T07:42:01.922 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T07:42:01.922 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T07:42:01.922 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T07:42:01.922 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T07:42:01.922 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T07:42:01.922 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T07:42:01.922 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T07:42:01.922 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T07:42:01.922 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T07:42:01.923 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T07:42:01.923 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T07:42:01.923 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T07:42:01.923 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T07:42:01.923 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T07:42:01.923 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T07:42:01.923 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T07:42:01.923 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T07:42:01.923 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T07:42:01.923 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T07:42:01.923 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T07:42:01.923 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T07:42:01.923 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T07:42:01.924 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T07:42:01.924 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T07:42:01.924 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T07:42:01.924 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T07:42:01.924 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T07:42:01.924 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T07:42:01.924 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T07:42:01.924 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T07:42:01.924 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T07:42:01.924 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T07:42:01.924 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T07:42:01.925 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T07:42:01.925 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T07:42:01.925 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T07:42:01.925 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T07:42:01.925 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T07:42:01.925 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T07:42:01.925 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T07:42:01.925 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T07:42:01.925 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T07:42:01.925 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T07:42:01.925 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T07:42:01.926 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T07:42:01.926 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T07:42:01.926 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T07:42:01.926 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T07:42:01.926 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T07:42:01.926 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T07:42:01.926 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T07:42:01.926 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T07:42:01.926 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T07:42:01.926 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T07:42:01.926 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T07:42:01.926 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T07:42:01.926 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T07:42:01.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T07:42:01.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T07:42:01.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T07:42:01.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T07:42:01.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T07:42:01.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T07:42:01.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T07:42:01.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T07:42:01.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T07:42:01.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T07:42:01.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T07:42:01.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T07:42:01.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T07:42:01.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T07:42:01.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T07:42:01.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T07:42:01.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T07:42:01.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T07:42:01.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T07:42:01.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T07:42:01.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T07:42:01.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T07:42:01.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T07:42:01.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T07:42:01.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T07:42:01.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T07:42:01.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T07:42:01.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T07:42:01.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T07:42:01.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T07:42:01.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T07:42:01.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T07:42:01.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T07:42:01.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T07:42:01.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T07:42:01.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T07:42:01.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T07:42:01.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T07:42:01.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T07:42:01.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T07:42:01.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T07:42:01.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T07:42:01.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T07:42:01.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T07:42:01.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T07:42:01.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T07:42:01.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T07:42:01.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T07:42:01.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T07:42:01.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T07:42:01.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T07:42:01.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T07:42:01.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T07:42:01.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T07:42:01.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T07:42:01.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T07:42:01.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T07:42:01.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T07:42:01.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T07:42:01.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T07:42:01.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T07:42:01.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T07:42:01.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T07:42:01.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T07:42:01.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T07:42:01.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T07:42:01.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T07:42:01.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T07:42:01.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T07:42:01.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T07:42:01.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T07:42:01.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T07:42:01.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T07:42:01.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T07:42:01.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T07:42:01.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T07:42:01.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T07:42:01.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T07:42:03.541 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:42:03.541 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:03.541 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-09-19T07:42:03.541 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T07:42:03.542 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T07:42:03.542 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T07:42:03.542 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T07:42:03.542 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.542 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.542 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T07:42:03.542 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T07:42:03.542 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-09-19T07:42:03.542 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T07:42:03.542 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T07:42:03.542 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T07:42:03.542 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T07:42:03.542 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.543 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.543 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.543 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.543 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:03.543 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.543 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.543 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.543 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:03.543 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:03.543 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:03.543 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:03.543 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:03.543 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.544 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.544 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:03.544 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.544 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.544 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.544 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:03.544 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:03.544 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.544 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.544 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:03.544 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T07:42:03.544 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T07:42:03.545 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T07:42:03.545 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T07:42:03.545 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T07:42:03.545 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T07:42:03.545 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T07:42:03.545 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T07:42:03.545 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T07:42:03.545 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T07:42:03.545 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T07:42:03.545 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.545 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.545 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.545 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T07:42:03.546 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T07:42:03.546 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T07:42:03.546 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T07:42:03.546 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T07:42:03.546 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.546 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.546 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T07:42:03.546 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.546 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T07:42:03.546 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T07:42:03.546 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T07:42:03.546 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T07:42:03.546 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T07:42:03.547 INFO:teuthology.orchestra.run.smithi016.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T07:42:03.547 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T07:42:03.547 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T07:42:03.547 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T07:42:03.547 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T07:42:03.547 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T07:42:03.547 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T07:42:03.547 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T07:42:03.547 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T07:42:03.547 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T07:42:03.547 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T07:42:03.547 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T07:42:03.547 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T07:42:03.547 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T07:42:03.548 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T07:42:03.548 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T07:42:03.548 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T07:42:03.548 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T07:42:03.548 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.548 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.548 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.548 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T07:42:03.548 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T07:42:03.548 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T07:42:03.548 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T07:42:03.548 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T07:42:03.548 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T07:42:03.548 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T07:42:03.549 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T07:42:03.549 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T07:42:03.549 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T07:42:03.549 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T07:42:03.549 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T07:42:03.549 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T07:42:03.549 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T07:42:03.549 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T07:42:03.549 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T07:42:03.549 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T07:42:03.549 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T07:42:03.549 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T07:42:03.549 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T07:42:03.550 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T07:42:03.550 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T07:42:03.550 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T07:42:03.550 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T07:42:03.550 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T07:42:03.550 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T07:42:03.550 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T07:42:03.550 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T07:42:03.550 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T07:42:03.550 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T07:42:03.550 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T07:42:03.550 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T07:42:03.550 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T07:42:03.551 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T07:42:03.551 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T07:42:03.551 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T07:42:03.551 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T07:42:03.551 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T07:42:03.551 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T07:42:03.551 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T07:42:03.551 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.551 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.551 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T07:42:03.551 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T07:42:03.551 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T07:42:03.551 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.552 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T07:42:03.552 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T07:42:03.552 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T07:42:03.552 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T07:42:03.552 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T07:42:03.552 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T07:42:03.552 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T07:42:03.552 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T07:42:03.552 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T07:42:03.552 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T07:42:03.552 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T07:42:03.552 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T07:42:03.552 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T07:42:03.553 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T07:42:03.553 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T07:42:03.553 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.553 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.553 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:03.553 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T07:42:03.553 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T07:42:03.553 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T07:42:03.553 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T07:42:03.553 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T07:42:03.553 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T07:42:03.554 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T07:42:03.554 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:03.554 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-09-19T07:42:04.122 DEBUG:teuthology.parallel:result is None 2024-09-19T07:42:04.586 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:42:04.586 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:04.586 INFO:teuthology.orchestra.run.smithi146.stdout:Upgraded: 2024-09-19T07:42:04.587 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T07:42:04.587 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T07:42:04.587 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T07:42:04.587 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T07:42:04.587 INFO:teuthology.orchestra.run.smithi146.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.587 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.587 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T07:42:04.587 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T07:42:04.587 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-19T07:42:04.587 INFO:teuthology.orchestra.run.smithi146.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T07:42:04.587 INFO:teuthology.orchestra.run.smithi146.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T07:42:04.587 INFO:teuthology.orchestra.run.smithi146.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T07:42:04.587 INFO:teuthology.orchestra.run.smithi146.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T07:42:04.587 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.588 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.588 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.588 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.588 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:04.588 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.588 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.588 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.588 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:04.588 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:04.588 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:04.588 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:04.588 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:04.589 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.589 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.589 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:04.589 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.589 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.589 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.589 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:04.589 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:04.589 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.589 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.589 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:42:04.589 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T07:42:04.589 INFO:teuthology.orchestra.run.smithi146.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T07:42:04.590 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T07:42:04.590 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T07:42:04.590 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T07:42:04.590 INFO:teuthology.orchestra.run.smithi146.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T07:42:04.590 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T07:42:04.590 INFO:teuthology.orchestra.run.smithi146.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T07:42:04.590 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T07:42:04.590 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T07:42:04.590 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T07:42:04.590 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.590 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.591 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.591 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T07:42:04.591 INFO:teuthology.orchestra.run.smithi146.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T07:42:04.591 INFO:teuthology.orchestra.run.smithi146.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T07:42:04.591 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T07:42:04.591 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T07:42:04.591 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.591 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.591 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T07:42:04.591 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.591 INFO:teuthology.orchestra.run.smithi146.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T07:42:04.591 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T07:42:04.591 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T07:42:04.592 INFO:teuthology.orchestra.run.smithi146.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T07:42:04.592 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T07:42:04.592 INFO:teuthology.orchestra.run.smithi146.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T07:42:04.592 INFO:teuthology.orchestra.run.smithi146.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T07:42:04.592 INFO:teuthology.orchestra.run.smithi146.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T07:42:04.592 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T07:42:04.592 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T07:42:04.592 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T07:42:04.592 INFO:teuthology.orchestra.run.smithi146.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T07:42:04.592 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T07:42:04.592 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T07:42:04.592 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T07:42:04.593 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T07:42:04.593 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T07:42:04.593 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T07:42:04.593 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T07:42:04.593 INFO:teuthology.orchestra.run.smithi146.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T07:42:04.593 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T07:42:04.593 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T07:42:04.593 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T07:42:04.593 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.593 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.593 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.593 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T07:42:04.593 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T07:42:04.594 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T07:42:04.594 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T07:42:04.594 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T07:42:04.594 INFO:teuthology.orchestra.run.smithi146.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T07:42:04.594 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T07:42:04.594 INFO:teuthology.orchestra.run.smithi146.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T07:42:04.594 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T07:42:04.594 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T07:42:04.594 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T07:42:04.594 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T07:42:04.594 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T07:42:04.594 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T07:42:04.594 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T07:42:04.595 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T07:42:04.595 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T07:42:04.595 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T07:42:04.595 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T07:42:04.595 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T07:42:04.595 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T07:42:04.595 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T07:42:04.595 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T07:42:04.595 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T07:42:04.595 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T07:42:04.595 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T07:42:04.595 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T07:42:04.595 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T07:42:04.596 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T07:42:04.596 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T07:42:04.596 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T07:42:04.596 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T07:42:04.596 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T07:42:04.596 INFO:teuthology.orchestra.run.smithi146.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T07:42:04.596 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T07:42:04.596 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T07:42:04.596 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T07:42:04.596 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T07:42:04.596 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T07:42:04.596 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T07:42:04.596 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.596 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.597 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T07:42:04.597 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T07:42:04.597 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T07:42:04.597 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.597 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T07:42:04.597 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T07:42:04.597 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T07:42:04.597 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T07:42:04.597 INFO:teuthology.orchestra.run.smithi146.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T07:42:04.597 INFO:teuthology.orchestra.run.smithi146.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T07:42:04.598 INFO:teuthology.orchestra.run.smithi146.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T07:42:04.598 INFO:teuthology.orchestra.run.smithi146.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T07:42:04.598 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T07:42:04.598 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T07:42:04.598 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T07:42:04.598 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T07:42:04.598 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T07:42:04.598 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T07:42:04.598 INFO:teuthology.orchestra.run.smithi146.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T07:42:04.598 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.598 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.598 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:42:04.598 INFO:teuthology.orchestra.run.smithi146.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T07:42:04.599 INFO:teuthology.orchestra.run.smithi146.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T07:42:04.599 INFO:teuthology.orchestra.run.smithi146.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T07:42:04.599 INFO:teuthology.orchestra.run.smithi146.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T07:42:04.599 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T07:42:04.599 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T07:42:04.599 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T07:42:04.599 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:04.599 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-19T07:42:05.050 DEBUG:teuthology.parallel:result is None 2024-09-19T07:42:05.051 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:42:05.213 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T07:42:05.240 INFO:teuthology.orchestra.run.smithi016.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T07:42:05.241 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T07:42:05.241 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T07:42:05.243 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:42:05.394 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T07:42:05.417 INFO:teuthology.orchestra.run.smithi146.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T07:42:05.417 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T07:42:05.417 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T07:42:05.419 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T07:42:05.420 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:05.420 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T07:42:05.454 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:05.454 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T07:42:05.485 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T07:42:05.485 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:05.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T07:42:05.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T07:42:05.595 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:05.595 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T07:42:05.621 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T07:42:05.688 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T07:42:05.688 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:05.688 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T07:42:05.717 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T07:42:05.786 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:05.786 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T07:42:05.812 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T07:42:05.877 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T07:42:05.877 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:05.877 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T07:42:05.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T07:42:05.973 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:05.973 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T07:42:05.998 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T07:42:06.063 INFO:teuthology.run_tasks:Running task ceph... 2024-09-19T07:42:06.145 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-19T07:42:06.146 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /var/log/ceph 2024-09-19T07:42:06.148 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 777 /var/log/ceph 2024-09-19T07:42:06.174 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-19T07:42:06.174 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-19T07:42:06.217 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-19T07:42:06.242 INFO:tasks.ceph:Creating extra log directories... 2024-09-19T07:42:06.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-19T07:42:06.285 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-19T07:42:06.315 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-19T07:42:06.315 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-19T07:42:06.316 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-19T07:42:06.316 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:06.316 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-19T07:42:06.366 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-19T07:42:06.480 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-19T07:42:06.507 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:06.507 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-19T07:42:06.522 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-19T07:42:06.631 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-19T07:42:06.661 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-19T07:42:06.661 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-19T07:42:06.661 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911819', 'branch': 'squid', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/client-limits}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7911819', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '6fad5dd73c93545e52f57f681e140a9c3529c11e'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1747, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'sleep_before_teardown': 0, 'subset': '176/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '6fad5dd73c93545e52f57f681e140a9c3529c11e', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi016.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDUKm2Esm24Sp8vfW+fdjdvoN5O6hymkgnj6oO32BJHjs6/UU39o50O2jqCJem9LJ236GcTM+Ufj48UoP8xG8J8=', 'smithi146.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDbnwXGecZyDr09cvJDx+f6F7AIdacwpm+x5UrTPPtwJ416LUd0dJuP9nlqRWCItgCim+t+qaf2vgcXemrVqpLw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_client_limits']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-18_21:24:17', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-19T07:42:06.662 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-19T07:42:06.664 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-19T07:42:06.711 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/run/ceph 2024-09-19T07:42:06.720 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m0777 -- /var/run/ceph 2024-09-19T07:42:06.775 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:06.776 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T07:42:06.801 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T07:42:06.801 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-09-19T07:42:06.858 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T07:42:06.858 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:42:06.858 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-19T07:42:06.858 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:42:06.858 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:42:06.858 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-19 07:41:55.626415764 +0000 2024-09-19T07:42:06.858 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-19 07:40:05.114138958 +0000 2024-09-19T07:42:06.858 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-19 07:40:05.114138958 +0000 2024-09-19T07:42:06.858 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-19 07:40:05.114138958 +0000 2024-09-19T07:42:06.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T07:42:06.922 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-19T07:42:06.922 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-19T07:42:06.922 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 8.8211e-05 s, 5.8 MB/s 2024-09-19T07:42:06.923 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T07:42:06.980 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-09-19T07:42:07.036 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T07:42:07.036 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:42:07.036 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-19T07:42:07.036 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:42:07.036 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:42:07.036 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-19 07:41:55.626415764 +0000 2024-09-19T07:42:07.036 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-19 07:40:05.450134350 +0000 2024-09-19T07:42:07.036 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-19 07:40:05.450134350 +0000 2024-09-19T07:42:07.036 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-19 07:40:05.450134350 +0000 2024-09-19T07:42:07.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T07:42:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-19T07:42:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-19T07:42:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 9.0769e-05 s, 5.6 MB/s 2024-09-19T07:42:07.101 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T07:42:07.159 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-09-19T07:42:07.215 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T07:42:07.215 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:42:07.215 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-19T07:42:07.215 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:42:07.215 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:42:07.215 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-19 07:41:55.626415764 +0000 2024-09-19T07:42:07.215 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-19 07:40:05.761130085 +0000 2024-09-19T07:42:07.216 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-19 07:40:05.761130085 +0000 2024-09-19T07:42:07.216 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-19 07:40:05.761130085 +0000 2024-09-19T07:42:07.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T07:42:07.279 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-19T07:42:07.279 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-19T07:42:07.279 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.00010003 s, 5.1 MB/s 2024-09-19T07:42:07.280 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T07:42:07.336 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-09-19T07:42:07.392 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T07:42:07.392 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:42:07.392 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-19T07:42:07.392 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:42:07.392 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:42:07.392 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-19 07:41:55.627415750 +0000 2024-09-19T07:42:07.392 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-19 07:40:06.095125504 +0000 2024-09-19T07:42:07.393 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-19 07:40:06.095125504 +0000 2024-09-19T07:42:07.393 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-19 07:40:06.095125504 +0000 2024-09-19T07:42:07.393 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T07:42:07.456 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-19T07:42:07.457 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-19T07:42:07.457 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 9.9939e-05 s, 5.1 MB/s 2024-09-19T07:42:07.458 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T07:42:07.520 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T07:42:07.521 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:07.521 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T07:42:07.538 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T07:42:07.538 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_1 2024-09-19T07:42:07.598 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T07:42:07.599 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:42:07.599 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-19T07:42:07.599 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:42:07.599 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:42:07.599 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-19 07:41:56.711879955 +0000 2024-09-19T07:42:07.599 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-19 07:40:03.444512200 +0000 2024-09-19T07:42:07.599 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-19 07:40:03.444512200 +0000 2024-09-19T07:42:07.599 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-09-19 07:40:03.444512200 +0000 2024-09-19T07:42:07.600 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T07:42:07.675 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-19T07:42:07.676 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-19T07:42:07.676 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000208596 s, 2.5 MB/s 2024-09-19T07:42:07.677 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T07:42:07.738 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_2 2024-09-19T07:42:07.796 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T07:42:07.796 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:42:07.796 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-19T07:42:07.797 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:42:07.797 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:42:07.797 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-19 07:41:56.711879955 +0000 2024-09-19T07:42:07.797 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-19 07:40:03.783507238 +0000 2024-09-19T07:42:07.797 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-19 07:40:03.783507238 +0000 2024-09-19T07:42:07.797 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-09-19 07:40:03.783507238 +0000 2024-09-19T07:42:07.797 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T07:42:07.861 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-19T07:42:07.861 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-19T07:42:07.862 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000265475 s, 1.9 MB/s 2024-09-19T07:42:07.863 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T07:42:07.919 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_3 2024-09-19T07:42:07.974 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T07:42:07.974 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:42:07.975 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-19T07:42:07.975 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:42:07.975 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:42:07.975 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-19 07:41:56.712879940 +0000 2024-09-19T07:42:07.975 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-19 07:40:04.109502466 +0000 2024-09-19T07:42:07.975 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-19 07:40:04.109502466 +0000 2024-09-19T07:42:07.975 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-09-19 07:40:04.109502466 +0000 2024-09-19T07:42:07.975 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T07:42:08.039 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-19T07:42:08.039 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-19T07:42:08.039 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.0002254 s, 2.3 MB/s 2024-09-19T07:42:08.040 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T07:42:08.096 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_4 2024-09-19T07:42:08.151 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T07:42:08.151 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:42:08.151 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-19T07:42:08.151 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:42:08.151 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:42:08.151 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-19 07:41:56.712879940 +0000 2024-09-19T07:42:08.151 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-19 07:40:04.437497665 +0000 2024-09-19T07:42:08.151 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-19 07:40:04.437497665 +0000 2024-09-19T07:42:08.151 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-09-19 07:40:04.437497665 +0000 2024-09-19T07:42:08.152 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T07:42:08.214 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-19T07:42:08.215 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-19T07:42:08.215 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000238796 s, 2.1 MB/s 2024-09-19T07:42:08.216 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T07:42:08.272 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T07:42:08.272 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi146.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-19T07:42:08.273 INFO:tasks.ceph:Generating config... 2024-09-19T07:42:08.274 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-19T07:42:08.274 INFO:tasks.ceph:[client] debug client = 20 2024-09-19T07:42:08.275 INFO:tasks.ceph:[client] debug ms = 1 2024-09-19T07:42:08.275 INFO:tasks.ceph:[client] pid file = 2024-09-19T07:42:08.275 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-19T07:42:08.275 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-19T07:42:08.275 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-19T07:42:08.275 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-19T07:42:08.275 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-19T07:42:08.275 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-19T07:42:08.275 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-19T07:42:08.275 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-19T07:42:08.275 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-19T07:42:08.275 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-19T07:42:08.275 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-19T07:42:08.275 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-19T07:42:08.276 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-19T07:42:08.276 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-19T07:42:08.276 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-19T07:42:08.276 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-19T07:42:08.276 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-19T07:42:08.276 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-19T07:42:08.276 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-19T07:42:08.276 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-19T07:42:08.276 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-19T07:42:08.276 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-19T07:42:08.276 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-19T07:42:08.276 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-19T07:42:08.276 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-19T07:42:08.276 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-19T07:42:08.277 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-19T07:42:08.277 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-19T07:42:08.277 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-19T07:42:08.277 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-19T07:42:08.277 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-19T07:42:08.277 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-19T07:42:08.277 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-19T07:42:08.277 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-19T07:42:08.277 INFO:tasks.ceph:Setting up mon.a... 2024-09-19T07:42:08.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-19T07:42:08.343 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.keyring 2024-09-19T07:42:08.346 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-19T07:42:08.392 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-19T07:42:08.418 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.16'), ('mon.b', '172.21.15.146'), ('mon.c', '[v2:172.21.15.146:3301,v1:172.21.15.146:6790]')] 2024-09-19T07:42:08.419 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.16,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-19T07:42:08.420 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:08.420 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-19T07:42:08.476 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.16 --add b 172.21.15.146 --addv c '[v2:172.21.15.146:3301,v1:172.21.15.146:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-19T07:42:08.566 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-19T07:42:08.566 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: generated fsid 6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb 2024-09-19T07:42:08.566 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = quincy 2024-09-19T07:42:08.567 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 0 2024-09-19T07:42:08.567 INFO:teuthology.orchestra.run.smithi016.stdout:fsid 6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb 2024-09-19T07:42:08.567 INFO:teuthology.orchestra.run.smithi016.stdout:last_changed 2024-09-19T07:42:08.567128+0000 2024-09-19T07:42:08.567 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-09-19T07:42:08.567128+0000 2024-09-19T07:42:08.567 INFO:teuthology.orchestra.run.smithi016.stdout:min_mon_release 17 (quincy) 2024-09-19T07:42:08.567 INFO:teuthology.orchestra.run.smithi016.stdout:election_strategy: 1 2024-09-19T07:42:08.567 INFO:teuthology.orchestra.run.smithi016.stdout:0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-19T07:42:08.567 INFO:teuthology.orchestra.run.smithi016.stdout:1: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.b 2024-09-19T07:42:08.567 INFO:teuthology.orchestra.run.smithi016.stdout:2: [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] mon.c 2024-09-19T07:42:08.567 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-19T07:42:08.569 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-19T07:42:08.584 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb... 2024-09-19T07:42:08.585 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T07:42:08.626 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T07:42:08.669 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-09-19T07:42:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2024-09-19T07:42:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T07:42:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2024-09-19T07:42:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2024-09-19T07:42:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2024-09-19T07:42:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2024-09-19T07:42:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2024-09-19T07:42:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2024-09-19T07:42:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2024-09-19T07:42:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T07:42:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2024-09-19T07:42:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T07:42:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T07:42:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2024-09-19T07:42:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2024-09-19T07:42:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2024-09-19T07:42:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2024-09-19T07:42:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2024-09-19T07:42:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2024-09-19T07:42:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2024-09-19T07:42:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2024-09-19T07:42:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T07:42:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T07:42:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2024-09-19T07:42:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T07:42:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2024-09-19T07:42:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2024-09-19T07:42:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2024-09-19T07:42:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2024-09-19T07:42:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2024-09-19T07:42:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-09-19T07:42:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb 2024-09-19T07:42:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.678 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2024-09-19T07:42:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2024-09-19T07:42:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2024-09-19T07:42:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2024-09-19T07:42:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2024-09-19T07:42:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2024-09-19T07:42:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2024-09-19T07:42:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2024-09-19T07:42:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2024-09-19T07:42:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2024-09-19T07:42:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2024-09-19T07:42:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2024-09-19T07:42:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2024-09-19T07:42:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2024-09-19T07:42:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2024-09-19T07:42:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2024-09-19T07:42:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2024-09-19T07:42:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2024-09-19T07:42:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2024-09-19T07:42:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2024-09-19T07:42:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2024-09-19T07:42:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2024-09-19T07:42:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2024-09-19T07:42:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2024-09-19T07:42:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:42:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2024-09-19T07:42:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2024-09-19T07:42:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T07:42:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2024-09-19T07:42:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2024-09-19T07:42:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2024-09-19T07:42:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2024-09-19T07:42:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-09-19T07:42:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.683 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2024-09-19T07:42:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:42:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2024-09-19T07:42:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-19T07:42:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-19T07:42:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:42:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:42:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2024-09-19T07:42:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-09-19T07:42:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.684 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2024-09-19T07:42:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:42:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-19T07:42:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2024-09-19T07:42:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-19T07:42:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2024-09-19T07:42:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2024-09-19T07:42:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:42:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2024-09-19T07:42:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2024-09-19T07:42:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:42:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T07:42:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T07:42:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T07:42:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T07:42:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T07:42:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T07:42:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2024-09-19T07:42:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2024-09-19T07:42:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2024-09-19T07:42:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: mon down mkfs grace = 300 2024-09-19T07:42:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2024-09-19T07:42:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on pool no app = False 2024-09-19T07:42:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:08.686 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2024-09-19T07:42:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2024-09-19T07:42:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2024-09-19T07:42:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2024-09-19T07:42:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T07:42:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T07:42:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2024-09-19T07:42:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-09-19T07:42:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:42:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2024-09-19T07:42:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-09-19T07:42:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-09-19T07:42:08.687 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2024-09-19T07:42:08.687 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2024-09-19T07:42:08.687 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2024-09-19T07:42:08.687 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-09-19T07:42:08.687 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-09-19T07:42:08.687 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-09-19T07:42:08.687 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-09-19T07:42:08.687 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2024-09-19T07:42:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2024-09-19T07:42:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2024-09-19T07:42:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:42:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2024-09-19T07:42:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2024-09-19T07:42:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2024-09-19T07:42:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = True 2024-09-19T07:42:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-09-19T07:42:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-09-19T07:42:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-09-19T07:42:08.691 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2024-09-19T07:42:08.691 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2024-09-19T07:42:08.691 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T07:42:08.691 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2024-09-19T07:42:08.691 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.691 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2024-09-19T07:42:08.691 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.691 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2024-09-19T07:42:08.691 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.691 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2024-09-19T07:42:08.691 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2024-09-19T07:42:08.691 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.692 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2024-09-19T07:42:08.692 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2024-09-19T07:42:08.692 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.692 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T07:42:08.692 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2024-09-19T07:42:08.692 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.692 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T07:42:08.692 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T07:42:08.692 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.692 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2024-09-19T07:42:08.692 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.693 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2024-09-19T07:42:08.693 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2024-09-19T07:42:08.693 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2024-09-19T07:42:08.693 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2024-09-19T07:42:08.693 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2024-09-19T07:42:08.693 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2024-09-19T07:42:08.693 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2024-09-19T07:42:08.693 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T07:42:08.693 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T07:42:08.693 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2024-09-19T07:42:08.693 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.693 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T07:42:08.693 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.693 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2024-09-19T07:42:08.694 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.694 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2024-09-19T07:42:08.694 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.694 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2024-09-19T07:42:08.694 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2024-09-19T07:42:08.694 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2024-09-19T07:42:08.694 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.16,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-09-19T07:42:08.694 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = 6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb 2024-09-19T07:42:08.694 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.694 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2024-09-19T07:42:08.694 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2024-09-19T07:42:08.695 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.695 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2024-09-19T07:42:08.695 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2024-09-19T07:42:08.695 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2024-09-19T07:42:08.695 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.695 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2024-09-19T07:42:08.695 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2024-09-19T07:42:08.696 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.696 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2024-09-19T07:42:08.696 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2024-09-19T07:42:08.696 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2024-09-19T07:42:08.696 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug trim objects = true 2024-09-19T07:42:08.696 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.696 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2024-09-19T07:42:08.696 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2024-09-19T07:42:08.696 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.696 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2024-09-19T07:42:08.696 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.696 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2024-09-19T07:42:08.696 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.696 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2024-09-19T07:42:08.697 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.697 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2024-09-19T07:42:08.697 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2024-09-19T07:42:08.698 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.698 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2024-09-19T07:42:08.698 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2024-09-19T07:42:08.698 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2024-09-19T07:42:08.698 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2024-09-19T07:42:08.698 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 20 2024-09-19T07:42:08.698 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 20 2024-09-19T07:42:08.699 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:42:08.699 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2024-09-19T07:42:08.699 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 10 2024-09-19T07:42:08.699 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T07:42:08.699 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2024-09-19T07:42:08.699 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2024-09-19T07:42:08.700 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2024-09-19T07:42:08.700 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2024-09-19T07:42:08.700 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-09-19T07:42:08.700 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.700 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2024-09-19T07:42:08.700 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:42:08.700 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2024-09-19T07:42:08.700 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-19T07:42:08.701 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-19T07:42:08.701 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:42:08.701 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:42:08.701 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2024-09-19T07:42:08.701 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-09-19T07:42:08.701 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.701 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2024-09-19T07:42:08.701 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:42:08.701 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-19T07:42:08.701 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2024-09-19T07:42:08.701 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-19T07:42:08.701 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2024-09-19T07:42:08.701 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2024-09-19T07:42:08.702 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:42:08.702 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2024-09-19T07:42:08.702 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2024-09-19T07:42:08.702 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:42:08.702 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.702 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T07:42:08.702 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T07:42:08.702 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T07:42:08.702 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.702 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T07:42:08.702 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T07:42:08.702 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T07:42:08.702 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.703 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2024-09-19T07:42:08.703 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2024-09-19T07:42:08.703 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.703 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2024-09-19T07:42:08.703 INFO:teuthology.orchestra.run.smithi146.stdout: mon down mkfs grace = 300 2024-09-19T07:42:08.703 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2024-09-19T07:42:08.703 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on pool no app = False 2024-09-19T07:42:08.703 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:42:08.703 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2024-09-19T07:42:08.703 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2024-09-19T07:42:08.703 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2024-09-19T07:42:08.703 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2024-09-19T07:42:08.703 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T07:42:08.703 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T07:42:08.704 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2024-09-19T07:42:08.704 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-09-19T07:42:08.704 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:42:08.704 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = "" 2024-09-19T07:42:08.704 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-09-19T07:42:08.704 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-09-19T07:42:08.704 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2024-09-19T07:42:08.704 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2024-09-19T07:42:08.704 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2024-09-19T07:42:08.704 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2024-09-19T07:42:08.704 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2024-09-19T07:42:08.704 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2024-09-19T07:42:08.704 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2024-09-19T07:42:08.704 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2024-09-19T07:42:08.705 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2024-09-19T07:42:08.705 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2024-09-19T07:42:08.705 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:42:08.705 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2024-09-19T07:42:08.705 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2024-09-19T07:42:08.705 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2024-09-19T07:42:08.705 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2024-09-19T07:42:08.705 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-09-19T07:42:08.705 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-09-19T07:42:08.705 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-09-19T07:42:08.705 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-19T07:42:08.706 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-19T07:42:08.787 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-19T07:42:08.787 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:08.788 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-19T07:42:08.803 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:08.803 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-19T07:42:08.859 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:42:08.859 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:08.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-19T07:42:08.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-19T07:42:08.935 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:08.935 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-19T07:42:08.991 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:42:08.991 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:08.991 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-19T07:42:08.991 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-19T07:42:09.026 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:09.026 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-19T07:42:09.081 INFO:tasks.ceph:Setting up mon nodes... 2024-09-19T07:42:09.081 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-19T07:42:09.082 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-19T07:42:09.134 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-19T07:42:09.137 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-19T07:42:09.196 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-19T07:42:09.198 INFO:tasks.ceph:Setting up mds nodes... 2024-09-19T07:42:09.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-19T07:42:09.251 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-19T07:42:09.254 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-19T07:42:09.279 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-19T07:42:09.373 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-19T07:42:09.375 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-19T07:42:09.401 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-19T07:42:09.453 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-19T07:42:09.455 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-19T07:42:09.480 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-19T07:42:09.572 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-19T07:42:09.574 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-19T07:42:09.599 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-19T07:42:09.599 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T07:42:09.642 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-19T07:42:09.655 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-19T07:42:09.737 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-19T07:42:09.751 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-19T07:42:09.792 INFO:teuthology.orchestra.run.smithi146.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-19T07:42:09.805 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-19T07:42:09.886 INFO:teuthology.orchestra.run.smithi146.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-19T07:42:09.899 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-19T07:42:09.899 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi146.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-19T07:42:09.899 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-19T07:42:09.925 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T07:42:09.925 INFO:tasks.ceph:role: osd.0 2024-09-19T07:42:09.925 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:42:09.926 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-19T07:42:09.990 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T07:42:09.991 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T07:42:09.991 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T07:42:09.991 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T07:42:09.991 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T07:42:09.991 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-09-19T07:42:09.991 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T07:42:09.991 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T07:42:09.991 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T07:42:09.991 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T07:42:10.023 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-09-19T07:42:10.025 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-09-19T07:42:10.025 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-19T07:42:10.096 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-19T07:42:10.166 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-19T07:42:10.232 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T07:42:10.233 INFO:tasks.ceph:role: osd.1 2024-09-19T07:42:10.233 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:42:10.233 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-19T07:42:10.299 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T07:42:10.300 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T07:42:10.300 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T07:42:10.300 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T07:42:10.300 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T07:42:10.300 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-09-19T07:42:10.300 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T07:42:10.300 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T07:42:10.300 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T07:42:10.300 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T07:42:10.332 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-09-19T07:42:10.334 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-09-19T07:42:10.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-19T07:42:10.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-19T07:42:10.476 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-19T07:42:10.543 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T07:42:10.543 INFO:tasks.ceph:role: osd.2 2024-09-19T07:42:10.543 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:42:10.543 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-19T07:42:10.608 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T07:42:10.608 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T07:42:10.609 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T07:42:10.609 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T07:42:10.609 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T07:42:10.609 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-09-19T07:42:10.609 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T07:42:10.609 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T07:42:10.609 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T07:42:10.609 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T07:42:10.641 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-09-19T07:42:10.643 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-09-19T07:42:10.644 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-19T07:42:10.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-19T07:42:10.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-19T07:42:10.853 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T07:42:10.853 INFO:tasks.ceph:role: osd.3 2024-09-19T07:42:10.853 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:42:10.853 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-19T07:42:10.918 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T07:42:10.918 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T07:42:10.918 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T07:42:10.919 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T07:42:10.919 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T07:42:10.919 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-09-19T07:42:10.919 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T07:42:10.919 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T07:42:10.919 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T07:42:10.919 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T07:42:10.951 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-09-19T07:42:10.953 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-09-19T07:42:10.953 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-19T07:42:11.022 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-19T07:42:11.092 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T07:42:11.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:11.190+0000 7f77ec8b0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-19T07:42:11.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:11.190+0000 7f77ec8b0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-19T07:42:11.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:11.190+0000 7f77ec8b0640 -1 bdev(0x55d513a11800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-19T07:42:11.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:11.190+0000 7f77ec8b0640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-19T07:42:14.366 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-19T07:42:14.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T07:42:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:14.488+0000 7fd6e1a0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-19T07:42:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:14.489+0000 7fd6e1a0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-19T07:42:14.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:14.489+0000 7fd6e1a0f640 -1 bdev(0x557026eab800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-19T07:42:14.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:14.489+0000 7fd6e1a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-19T07:42:17.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-19T07:42:17.436 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T07:42:17.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:17.528+0000 7ff307945640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-19T07:42:17.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:17.528+0000 7ff307945640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-19T07:42:17.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:17.529+0000 7ff307945640 -1 bdev(0x5592ce481800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-19T07:42:17.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:17.529+0000 7ff307945640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-19T07:42:20.448 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-19T07:42:20.478 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T07:42:20.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:20.570+0000 7f91faf4f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-19T07:42:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:20.570+0000 7f91faf4f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-19T07:42:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:20.570+0000 7f91faf4f640 -1 bdev(0x56366afd3800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-19T07:42:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:20.570+0000 7f91faf4f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-19T07:42:23.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-19T07:42:23.532 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-19T07:42:23.563 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T07:42:23.563 INFO:tasks.ceph:role: osd.4 2024-09-19T07:42:23.563 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:42:23.564 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-19T07:42:23.639 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T07:42:23.640 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T07:42:23.640 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T07:42:23.640 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T07:42:23.640 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T07:42:23.640 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-09-19T07:42:23.640 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T07:42:23.640 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T07:42:23.640 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T07:42:23.641 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T07:42:24.142 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2024-09-19T07:42:24.145 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-09-19T07:42:24.145 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-19T07:42:24.281 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-19T07:42:24.315 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-19T07:42:24.382 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T07:42:24.382 INFO:tasks.ceph:role: osd.5 2024-09-19T07:42:24.382 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:42:24.382 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-19T07:42:24.455 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T07:42:24.456 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T07:42:24.456 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T07:42:24.456 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T07:42:24.456 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T07:42:24.456 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-09-19T07:42:24.456 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T07:42:24.456 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T07:42:24.456 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T07:42:24.457 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T07:42:24.911 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2024-09-19T07:42:24.913 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-09-19T07:42:24.913 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-19T07:42:25.005 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-19T07:42:25.035 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-19T07:42:25.101 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T07:42:25.101 INFO:tasks.ceph:role: osd.6 2024-09-19T07:42:25.101 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:42:25.101 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-19T07:42:25.174 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T07:42:25.174 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T07:42:25.174 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T07:42:25.174 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T07:42:25.174 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T07:42:25.174 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-09-19T07:42:25.174 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T07:42:25.174 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T07:42:25.175 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T07:42:25.175 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T07:42:25.657 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2024-09-19T07:42:25.659 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-09-19T07:42:25.659 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-19T07:42:25.802 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-19T07:42:25.833 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-19T07:42:25.898 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T07:42:25.898 INFO:tasks.ceph:role: osd.7 2024-09-19T07:42:25.898 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:42:25.898 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-19T07:42:25.970 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T07:42:25.970 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T07:42:25.970 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T07:42:25.970 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T07:42:25.970 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T07:42:25.971 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-09-19T07:42:25.971 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T07:42:25.971 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T07:42:25.971 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T07:42:25.971 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T07:42:26.485 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2024-09-19T07:42:26.488 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-09-19T07:42:26.488 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-19T07:42:26.630 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-19T07:42:26.660 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T07:42:26.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:26.760+0000 7fdc20c0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-19T07:42:26.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:26.760+0000 7fdc20c0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-19T07:42:26.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:26.760+0000 7fdc20c0f640 -1 bdev(0x55c1d643b800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-19T07:42:26.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:26.760+0000 7fdc20c0f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-19T07:42:29.911 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-19T07:42:29.941 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T07:42:30.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:30.031+0000 7fde4a00d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-19T07:42:30.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:30.032+0000 7fde4a00d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-19T07:42:30.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:30.032+0000 7fde4a00d640 -1 bdev(0x55fc421e9800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-19T07:42:30.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:30.032+0000 7fde4a00d640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-19T07:42:32.942 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-19T07:42:32.972 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T07:42:33.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:33.062+0000 7f1108814640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-19T07:42:33.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:33.062+0000 7f1108814640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-19T07:42:33.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:33.062+0000 7f1108814640 -1 bdev(0x55da0a30f800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-19T07:42:33.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:33.062+0000 7f1108814640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-19T07:42:35.991 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-19T07:42:36.020 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T07:42:36.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:36.113+0000 7f8ce760d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-19T07:42:36.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:36.113+0000 7f8ce760d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-19T07:42:36.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:36.113+0000 7f8ce760d640 -1 bdev(0x556bcfc09800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-19T07:42:36.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:36.113+0000 7f8ce760d640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-19T07:42:36.663 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:42:36.666 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:42:39.291 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-19T07:42:39.320 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-19T07:42:39.321 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:39.321 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-19T07:42:39.352 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:39.352 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-19T07:42:39.423 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:39.423 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-19T07:42:39.493 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:39.493 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-19T07:42:39.563 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:39.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-19T07:42:39.633 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:39.633 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-19T07:42:39.702 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:39.702 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-19T07:42:39.771 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:39.771 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-19T07:42:39.801 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:39.801 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-19T07:42:39.871 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:39.871 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-19T07:42:39.940 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:39.940 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-19T07:42:40.008 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:40.009 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-19T07:42:40.077 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:40.077 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-19T07:42:40.143 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:40.144 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-19T07:42:40.211 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:40.211 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-19T07:42:40.230 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:40.231 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-19T07:42:40.290 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:40.290 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-19T07:42:40.306 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:40.306 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-19T07:42:40.363 INFO:tasks.ceph:Adding keys to all mons... 2024-09-19T07:42:40.363 DEBUG:teuthology.orchestra.run.smithi016:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-19T07:42:40.366 DEBUG:teuthology.orchestra.run.smithi146:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-19T07:42:40.406 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.y] 2024-09-19T07:42:40.406 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etmp+gSCBAAGXzmKfFNd6K2fWeqYhlHAg== 2024-09-19T07:42:40.406 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-09-19T07:42:40.406 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etms80KDxAAqTrG88LTEJIj9TWJu0witQ== 2024-09-19T07:42:40.406 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-09-19T07:42:40.407 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etmNDBGFhAAtBioJJ1+NtIDLhd/Hg7aZA== 2024-09-19T07:42:40.407 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.0] 2024-09-19T07:42:40.407 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDT1etm09plCxAAGvJYF6JrKw49FtXb71ox3A== 2024-09-19T07:42:40.407 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.1] 2024-09-19T07:42:40.407 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDW1etmGy8yHRAAlcDsHLnkjtS6sCCR8Olv2A== 2024-09-19T07:42:40.407 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.2] 2024-09-19T07:42:40.407 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDZ1etmMhGTHxAATQaW2kgJhHKftutQQtUa6A== 2024-09-19T07:42:40.407 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.3] 2024-09-19T07:42:40.407 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDc1etmD3sLIhAAIdY00buvb3bIFA5ZpgxxjA== 2024-09-19T07:42:40.407 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.x] 2024-09-19T07:42:40.407 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etmNQ68CxAANkK+D3u/oTEz8Su4iuZrCg== 2024-09-19T07:42:40.407 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-09-19T07:42:40.407 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etm4T0KGxAAcfDOKrSdKzhJ4xvTFIyxcQ== 2024-09-19T07:42:40.407 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-09-19T07:42:40.407 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etmaHEiIhAACqqjCRMV+xlGh9WqYsj+MA== 2024-09-19T07:42:40.408 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.4] 2024-09-19T07:42:40.408 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDi1etmgutcLRAAHJEl5SfQ7N0T4YoYeBfrQg== 2024-09-19T07:42:40.408 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.5] 2024-09-19T07:42:40.408 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDm1etmCV3xARAA+5yhQEmIpyNId3RL1Xsxhw== 2024-09-19T07:42:40.408 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.6] 2024-09-19T07:42:40.408 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDp1etm9XC+AxAA+Lrs0ExfSKaixgOxfX8Nug== 2024-09-19T07:42:40.408 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.7] 2024-09-19T07:42:40.408 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDs1etmhmvMBhAAhvW/udIE2gCd8IrJCSFuvw== 2024-09-19T07:42:40.408 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-19T07:42:40.408 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA== 2024-09-19T07:42:40.408 INFO:teuthology.orchestra.run.smithi016.stdout:[client.2] 2024-09-19T07:42:40.408 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etmjFsCLBAANa7gCMvDw84Ovo7u6RRJ3Q== 2024-09-19T07:42:40.408 INFO:teuthology.orchestra.run.smithi016.stdout:[client.1] 2024-09-19T07:42:40.408 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etmc05KLxAAPQl6wOHqFIf8LVxskiuLVg== 2024-09-19T07:42:40.409 INFO:teuthology.orchestra.run.smithi016.stdout:[client.3] 2024-09-19T07:42:40.409 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etmdUbgNBAAfwstyOeeoZfetgVZu4z5xg== 2024-09-19T07:42:40.427 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr.y] 2024-09-19T07:42:40.427 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDR1etmp+gSCBAAGXzmKfFNd6K2fWeqYhlHAg== 2024-09-19T07:42:40.427 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2024-09-19T07:42:40.427 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDR1etms80KDxAAqTrG88LTEJIj9TWJu0witQ== 2024-09-19T07:42:40.427 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2024-09-19T07:42:40.427 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDR1etmNDBGFhAAtBioJJ1+NtIDLhd/Hg7aZA== 2024-09-19T07:42:40.427 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.0] 2024-09-19T07:42:40.427 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDT1etm09plCxAAGvJYF6JrKw49FtXb71ox3A== 2024-09-19T07:42:40.427 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.1] 2024-09-19T07:42:40.427 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDW1etmGy8yHRAAlcDsHLnkjtS6sCCR8Olv2A== 2024-09-19T07:42:40.427 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.2] 2024-09-19T07:42:40.427 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDZ1etmMhGTHxAATQaW2kgJhHKftutQQtUa6A== 2024-09-19T07:42:40.428 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.3] 2024-09-19T07:42:40.428 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDc1etmD3sLIhAAIdY00buvb3bIFA5ZpgxxjA== 2024-09-19T07:42:40.428 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr.x] 2024-09-19T07:42:40.428 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDR1etmNQ68CxAANkK+D3u/oTEz8Su4iuZrCg== 2024-09-19T07:42:40.428 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2024-09-19T07:42:40.428 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDR1etm4T0KGxAAcfDOKrSdKzhJ4xvTFIyxcQ== 2024-09-19T07:42:40.428 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2024-09-19T07:42:40.428 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDR1etmaHEiIhAACqqjCRMV+xlGh9WqYsj+MA== 2024-09-19T07:42:40.428 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.4] 2024-09-19T07:42:40.428 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDi1etmgutcLRAAHJEl5SfQ7N0T4YoYeBfrQg== 2024-09-19T07:42:40.428 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.5] 2024-09-19T07:42:40.428 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDm1etmCV3xARAA+5yhQEmIpyNId3RL1Xsxhw== 2024-09-19T07:42:40.428 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.6] 2024-09-19T07:42:40.429 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDp1etm9XC+AxAA+Lrs0ExfSKaixgOxfX8Nug== 2024-09-19T07:42:40.429 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.7] 2024-09-19T07:42:40.429 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDs1etmhmvMBhAAhvW/udIE2gCd8IrJCSFuvw== 2024-09-19T07:42:40.429 INFO:teuthology.orchestra.run.smithi146.stdout:[client.0] 2024-09-19T07:42:40.429 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA== 2024-09-19T07:42:40.429 INFO:teuthology.orchestra.run.smithi146.stdout:[client.2] 2024-09-19T07:42:40.429 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDR1etmjFsCLBAANa7gCMvDw84Ovo7u6RRJ3Q== 2024-09-19T07:42:40.429 INFO:teuthology.orchestra.run.smithi146.stdout:[client.1] 2024-09-19T07:42:40.429 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDR1etmc05KLxAAPQl6wOHqFIf8LVxskiuLVg== 2024-09-19T07:42:40.429 INFO:teuthology.orchestra.run.smithi146.stdout:[client.3] 2024-09-19T07:42:40.429 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDR1etmdUbgNBAAfwstyOeeoZfetgVZu4z5xg== 2024-09-19T07:42:40.430 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-19T07:42:40.453 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-19T07:42:40.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T07:42:40.524 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T07:42:40.579 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T07:42:40.581 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T07:42:40.636 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T07:42:40.638 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T07:42:40.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T07:42:40.693 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T07:42:40.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T07:42:40.746 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T07:42:40.797 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T07:42:40.799 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T07:42:40.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-19T07:42:40.852 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-19T07:42:40.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T07:42:40.906 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T07:42:40.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T07:42:40.963 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T07:42:41.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T07:42:41.019 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T07:42:41.070 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T07:42:41.072 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T07:42:41.123 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T07:42:41.126 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T07:42:41.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T07:42:41.180 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T07:42:41.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T07:42:41.236 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T07:42:41.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T07:42:41.292 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T07:42:41.347 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T07:42:41.349 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T07:42:41.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T07:42:41.406 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T07:42:41.460 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-19T07:42:41.461 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-19T07:42:41.487 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-19T07:42:41.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-19T07:42:41.610 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-19T07:42:41.637 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-19T07:42:41.732 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-19T07:42:41.757 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-19T07:42:41.823 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-19T07:42:41.919 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-19T07:42:41.943 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-19T07:42:41.946 DEBUG:teuthology.orchestra.run.smithi146:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-19T07:42:41.999 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-19T07:42:42.000 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-19T07:42:42.000 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-19T07:42:42.002 INFO:tasks.ceph.mon.a:Started 2024-09-19T07:42:42.003 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-19T07:42:42.003 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-19T07:42:42.041 INFO:tasks.ceph.mon.b:Started 2024-09-19T07:42:42.042 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-19T07:42:42.042 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-19T07:42:42.044 INFO:tasks.ceph.mon.c:Started 2024-09-19T07:42:42.044 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-19T07:42:42.044 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-19T07:42:42.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-19T07:42:42.046 INFO:tasks.ceph.mgr.y:Started 2024-09-19T07:42:42.047 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-19T07:42:42.047 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-19T07:42:42.049 INFO:tasks.ceph.mgr.x:Started 2024-09-19T07:42:42.049 INFO:tasks.ceph:Setting crush tunables to default 2024-09-19T07:42:42.049 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd crush tunables default 2024-09-19T07:42:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.151+0000 7f28f38ba640 1 Processor -- start 2024-09-19T07:42:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.151+0000 7f28f38ba640 1 -- start start 2024-09-19T07:42:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.152+0000 7f28f38ba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28ec15f0d0 0x7f28ec15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.152+0000 7f28f38ba640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f28ec0d4130 0x7f28ec0d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.152+0000 7f28f38ba640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28ec0d4b20 con 0x7f28ec0d0070 2024-09-19T07:42:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.152+0000 7f28f38ba640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f28ec0d1b10 con 0x7f28ec0d4130 2024-09-19T07:42:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.152+0000 7f28f38ba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f28ec0cf2b0 con 0x7f28ec15f0d0 2024-09-19T07:42:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.152+0000 7f28f1656640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28ec15f0d0 0x7f28ec15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.152+0000 7f28f1e57640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f28ec0d4130 0x7f28ec0d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.152+0000 7f28f0e55640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f28ec0d0070 0x7f28ec0d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:47866/0 (socket says 172.21.15.16:47866) 2024-09-19T07:42:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.152+0000 7f28f0e55640 1 -- 172.21.15.16:0/3295901062 learned_addr learned my addr 172.21.15.16:0/3295901062 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:42:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.152+0000 7f28f1e57640 1 -- 172.21.15.16:0/3295901062 >> v2:172.21.15.146:3300/0 conn(0x7f28ec0d4130 msgr2=0x7f28ec0d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-19T07:42:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.152+0000 7f28f1e57640 1 -- 172.21.15.16:0/3295901062 >> v2:172.21.15.146:3300/0 conn(0x7f28ec0d4130 msgr2=0x7f28ec0d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-19T07:42:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.152+0000 7f28f1e57640 1 --2- 172.21.15.16:0/3295901062 >> v2:172.21.15.146:3300/0 conn(0x7f28ec0d4130 0x7f28ec0d45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T07:42:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.152+0000 7f28f1656640 1 -- 172.21.15.16:0/3295901062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28ec15f0d0 msgr2=0x7f28ec15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-19T07:42:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.152+0000 7f28f1656640 1 -- 172.21.15.16:0/3295901062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28ec15f0d0 msgr2=0x7f28ec15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-19T07:42:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.152+0000 7f28f1656640 1 --2- 172.21.15.16:0/3295901062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28ec15f0d0 0x7f28ec15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T07:42:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.152+0000 7f28f1656640 1 --2- 172.21.15.16:0/3295901062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28ec15f0d0 0x7f28ec15f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:42:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.152+0000 7f28f1e57640 1 --2- 172.21.15.16:0/3295901062 >> v2:172.21.15.146:3300/0 conn(0x7f28ec0d4130 0x7f28ec0d45e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:42:42.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.313+0000 7f28e3fff640 1 -- 172.21.15.16:0/3295901062 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2484851420 0 0) 0x7f28dc001110 con 0x7f28ec0d0070 2024-09-19T07:42:42.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.313+0000 7f28e3fff640 1 -- 172.21.15.16:0/3295901062 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28d8003520 con 0x7f28ec0d0070 2024-09-19T07:42:42.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28e3fff640 1 -- 172.21.15.16:0/3295901062 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 223106565 0 0) 0x7f28d8003520 con 0x7f28ec0d0070 2024-09-19T07:42:42.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28e3fff640 1 -- 172.21.15.16:0/3295901062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28ec15f0d0 msgr2=0x7f28ec15f4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:42:42.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28e3fff640 1 --2- 172.21.15.16:0/3295901062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28ec15f0d0 0x7f28ec15f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:42.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28e3fff640 1 -- 172.21.15.16:0/3295901062 >> v2:172.21.15.146:3300/0 conn(0x7f28ec0d4130 msgr2=0x7f28ec0d45e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:42:42.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28e3fff640 1 --2- 172.21.15.16:0/3295901062 >> v2:172.21.15.146:3300/0 conn(0x7f28ec0d4130 0x7f28ec0d45e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:42.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28e3fff640 1 -- 172.21.15.16:0/3295901062 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ec0cf590 con 0x7f28ec0d0070 2024-09-19T07:42:42.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28e3fff640 1 -- 172.21.15.16:0/3295901062 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f28dc001840 con 0x7f28ec0d0070 2024-09-19T07:42:42.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28e3fff640 1 -- 172.21.15.16:0/3295901062 >> v1:172.21.15.146:6789/0 conn(0x7f28ec0d0070 legacy=0x7f28ec0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:42.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28e3fff640 1 --2- 172.21.15.16:0/3295901062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28ec15f0d0 0x7f28d80039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:42.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28e3fff640 1 --2- 172.21.15.16:0/3295901062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d8004050 0x7f28d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:42.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28e3fff640 1 --2- 172.21.15.16:0/3295901062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d8004d40 0x7f28d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:42.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28e3fff640 1 -- 172.21.15.16:0/3295901062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec0d1b10 con 0x7f28d8004050 2024-09-19T07:42:42.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28e3fff640 1 -- 172.21.15.16:0/3295901062 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec0cf2b0 con 0x7f28d8004d40 2024-09-19T07:42:42.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28e3fff640 1 -- 172.21.15.16:0/3295901062 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f28d800bc00 con 0x7f28ec15f0d0 2024-09-19T07:42:42.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28f0e55640 1 --2- 172.21.15.16:0/3295901062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d8004050 0x7f28d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:42.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28f1656640 1 --2- 172.21.15.16:0/3295901062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28ec15f0d0 0x7f28d80039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:42.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28f1e57640 1 --2- 172.21.15.16:0/3295901062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d8004d40 0x7f28d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:42.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28f1656640 1 -- 172.21.15.16:0/3295901062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d8004d40 msgr2=0x7f28d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28f1656640 1 --2- 172.21.15.16:0/3295901062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d8004d40 0x7f28d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28f1656640 1 -- 172.21.15.16:0/3295901062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d8004050 msgr2=0x7f28d8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28f1656640 1 --2- 172.21.15.16:0/3295901062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d8004050 0x7f28d8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28f1656640 1 -- 172.21.15.16:0/3295901062 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d8003520 con 0x7f28ec15f0d0 2024-09-19T07:42:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.317+0000 7f28f0e55640 1 --2- 172.21.15.16:0/3295901062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d8004050 0x7f28d8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.318+0000 7f28f1656640 1 --2- 172.21.15.16:0/3295901062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28ec15f0d0 0x7f28d80039d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f28e800b9e0 tx=0x7f28d8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa1b374419288155 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:42.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.319+0000 7f28e3fff640 1 -- 172.21.15.16:0/3295901062 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8030070 con 0x7f28ec15f0d0 2024-09-19T07:42:42.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.319+0000 7f28e3fff640 1 -- 172.21.15.16:0/3295901062 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e802adb0 con 0x7f28ec15f0d0 2024-09-19T07:42:42.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.319+0000 7f28e3fff640 1 -- 172.21.15.16:0/3295901062 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e800b5e0 con 0x7f28ec15f0d0 2024-09-19T07:42:42.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.320+0000 7f28f38ba640 1 -- 172.21.15.16:0/3295901062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28ec15f0d0 msgr2=0x7f28d80039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:42.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.320+0000 7f28f38ba640 1 --2- 172.21.15.16:0/3295901062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28ec15f0d0 0x7f28d80039d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f28e800b9e0 tx=0x7f28d8005830 comp rx=0 tx=0).stop 2024-09-19T07:42:42.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.320+0000 7f28f1e57640 1 -- 172.21.15.16:0/3295901062 reap_dead start 2024-09-19T07:42:42.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.320+0000 7f28f38ba640 1 -- 172.21.15.16:0/3295901062 shutdown_connections 2024-09-19T07:42:42.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.320+0000 7f28f38ba640 1 -- 172.21.15.16:0/3295901062 >> 172.21.15.16:0/3295901062 conn(0x7f28ec09d760 msgr2=0x7f28ec08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:42.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.320+0000 7f28f38ba640 1 -- 172.21.15.16:0/3295901062 shutdown_connections 2024-09-19T07:42:42.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.320+0000 7f28f38ba640 1 -- 172.21.15.16:0/3295901062 wait complete. 2024-09-19T07:42:42.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.322+0000 7f28f38ba640 1 Processor -- start 2024-09-19T07:42:42.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.322+0000 7f28f38ba640 1 -- start start 2024-09-19T07:42:42.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.322+0000 7f28f38ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ec15f0d0 0x7f28ec0b99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:42.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.322+0000 7f28f38ba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28ec15b410 0x7f28ec160f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:42.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.322+0000 7f28f38ba640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28ec161460 0x7f28ec1e8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:42.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.322+0000 7f28f38ba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec0cef60 con 0x7f28ec15f0d0 2024-09-19T07:42:42.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.322+0000 7f28f38ba640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec0cfb50 con 0x7f28ec161460 2024-09-19T07:42:42.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.322+0000 7f28f38ba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f28ec15f4b0 con 0x7f28ec15b410 2024-09-19T07:42:42.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.323+0000 7f28f1656640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ec15f0d0 0x7f28ec0b99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:42.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.323+0000 7f28f1656640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ec15f0d0 0x7f28ec0b99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59514/0 (socket says 172.21.15.16:59514) 2024-09-19T07:42:42.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.323+0000 7f28f1656640 1 -- 172.21.15.16:0/1210759099 learned_addr learned my addr 172.21.15.16:0/1210759099 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:42:42.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.323+0000 7f28f0e55640 1 --2- 172.21.15.16:0/1210759099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28ec15b410 0x7f28ec160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:42.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.323+0000 7f28f1e57640 1 --2- 172.21.15.16:0/1210759099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28ec161460 0x7f28ec1e8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:42.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.323+0000 7f28f1656640 1 -- 172.21.15.16:0/1210759099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28ec15b410 msgr2=0x7f28ec160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:42.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.323+0000 7f28f1656640 1 --2- 172.21.15.16:0/1210759099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28ec15b410 0x7f28ec160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:42.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.323+0000 7f28f1656640 1 -- 172.21.15.16:0/1210759099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28ec161460 msgr2=0x7f28ec1e8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:42.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.323+0000 7f28f1656640 1 --2- 172.21.15.16:0/1210759099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28ec161460 0x7f28ec1e8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:42.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.323+0000 7f28f1656640 1 -- 172.21.15.16:0/1210759099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ec0b73b0 con 0x7f28ec15f0d0 2024-09-19T07:42:42.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.323+0000 7f28f1656640 1 --2- 172.21.15.16:0/1210759099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ec15f0d0 0x7f28ec0b99e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f28e800bb00 tx=0x7f28e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:42.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.323+0000 7f28e1ffb640 1 -- 172.21.15.16:0/1210759099 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8043020 con 0x7f28ec15f0d0 2024-09-19T07:42:42.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.323+0000 7f28e1ffb640 1 -- 172.21.15.16:0/1210759099 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e803f680 con 0x7f28ec15f0d0 2024-09-19T07:42:42.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.323+0000 7f28f38ba640 1 -- 172.21.15.16:0/1210759099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ec0b7690 con 0x7f28ec15f0d0 2024-09-19T07:42:42.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.323+0000 7f28e1ffb640 1 -- 172.21.15.16:0/1210759099 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e800b5e0 con 0x7f28ec15f0d0 2024-09-19T07:42:42.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.323+0000 7f28f38ba640 1 -- 172.21.15.16:0/1210759099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ec0b7c50 con 0x7f28ec15f0d0 2024-09-19T07:42:42.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.324+0000 7f28e1ffb640 1 -- 172.21.15.16:0/1210759099 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f28e8046020 con 0x7f28ec15f0d0 2024-09-19T07:42:42.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.324+0000 7f28e1ffb640 1 -- 172.21.15.16:0/1210759099 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f28e803dcb0 con 0x7f28ec15f0d0 2024-09-19T07:42:42.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.325+0000 7f28f38ba640 1 -- 172.21.15.16:0/1210759099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28ec0d45e0 con 0x7f28ec15f0d0 2024-09-19T07:42:42.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.327+0000 7f28e1ffb640 1 -- 172.21.15.16:0/1210759099 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f28e804d030 con 0x7f28ec15f0d0 2024-09-19T07:42:42.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.392+0000 7f28f38ba640 1 -- 172.21.15.16:0/1210759099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f28ec0b8db0 con 0x7f28ec15f0d0 2024-09-19T07:42:42.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.393+0000 7f28e1ffb640 1 -- 172.21.15.16:0/1210759099 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f28e803d440 con 0x7f28ec15f0d0 2024-09-19T07:42:42.394 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2024-09-19T07:42:42.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.394+0000 7f28f38ba640 1 -- 172.21.15.16:0/1210759099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ec15f0d0 msgr2=0x7f28ec0b99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:42.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.394+0000 7f28f38ba640 1 --2- 172.21.15.16:0/1210759099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ec15f0d0 0x7f28ec0b99e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f28e800bb00 tx=0x7f28e8004730 comp rx=0 tx=0).stop 2024-09-19T07:42:42.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.395+0000 7f28f38ba640 1 -- 172.21.15.16:0/1210759099 shutdown_connections 2024-09-19T07:42:42.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.395+0000 7f28f38ba640 1 --2- 172.21.15.16:0/1210759099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28ec161460 0x7f28ec1e8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:42.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.395+0000 7f28f38ba640 1 --2- 172.21.15.16:0/1210759099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28ec15b410 0x7f28ec160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:42.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.395+0000 7f28f38ba640 1 --2- 172.21.15.16:0/1210759099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ec15f0d0 0x7f28ec0b99e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:42.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.395+0000 7f28f38ba640 1 -- 172.21.15.16:0/1210759099 >> 172.21.15.16:0/1210759099 conn(0x7f28ec09d760 msgr2=0x7f28ec0cd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:42.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.395+0000 7f28f38ba640 1 -- 172.21.15.16:0/1210759099 shutdown_connections 2024-09-19T07:42:42.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:42.395+0000 7f28f38ba640 1 -- 172.21.15.16:0/1210759099 wait complete. 2024-09-19T07:42:42.406 INFO:tasks.ceph:check_enable_crimson: False 2024-09-19T07:42:42.406 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-19T07:42:42.406 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:42.406 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-19T07:42:42.420 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:42.420+0000 7feda872f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T07:42:42.426 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:42.425+0000 7fdcf962f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T07:42:42.432 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:42.432 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-19T07:42:42.461 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:42.460+0000 7feda872f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T07:42:42.469 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:42.469+0000 7fdcf962f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T07:42:42.494 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:42.494 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-19T07:42:42.560 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:42:42.560 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-19T07:42:42.605 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:42.605+0000 7feda872f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T07:42:42.626 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:42.626 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-19T07:42:42.629 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:42.629+0000 7fdcf962f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T07:42:42.651 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:42.652 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-19T07:42:42.658 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:42.658+0000 7feda872f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T07:42:42.682 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:42.682 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-19T07:42:42.682 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:42.682+0000 7fdcf962f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T07:42:42.747 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:42:42.747 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-19T07:42:42.812 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 90068366-8d2f-411b-a051-3eb4e2cc6032 0 2024-09-19T07:42:42.840 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:42.840+0000 7feda872f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:42:42.849 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:42.849+0000 7fdcf962f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:42:42.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.939+0000 7f060f253640 1 Processor -- start 2024-09-19T07:42:42.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.939+0000 7f060f253640 1 -- start start 2024-09-19T07:42:42.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.940+0000 7f060f253640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f060815e560 0x7f06080d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:42.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.940+0000 7f060f253640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0608071090 0x7f0608162030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:42.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.940+0000 7f060f253640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f060815e940 con 0x7f06080d1410 2024-09-19T07:42:42.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.940+0000 7f060f253640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f0608070010 con 0x7f0608071090 2024-09-19T07:42:42.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.940+0000 7f060f253640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f06080702d0 con 0x7f060815e560 2024-09-19T07:42:42.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.940+0000 7f060cfef640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f060815e560 0x7f06080d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:42.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.940+0000 7f060cfef640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f060815e560 0x7f06080d0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57978/0 (socket says 172.21.15.146:57978) 2024-09-19T07:42:42.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.940+0000 7f060cfef640 1 -- 172.21.15.146:0/2020207087 learned_addr learned my addr 172.21.15.146:0/2020207087 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-19T07:42:42.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.940+0000 7f0607fff640 1 --2- 172.21.15.146:0/2020207087 >> v2:172.21.15.16:3300/0 conn(0x7f0608071090 0x7f0608162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:42.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.941+0000 7f060cfef640 1 -- 172.21.15.146:0/2020207087 >> v2:172.21.15.16:3300/0 conn(0x7f0608071090 msgr2=0x7f0608162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:42.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.941+0000 7f060cfef640 1 --2- 172.21.15.146:0/2020207087 >> v2:172.21.15.16:3300/0 conn(0x7f0608071090 0x7f0608162030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:42.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.941+0000 7f060cfef640 1 -- 172.21.15.146:0/2020207087 >> v1:172.21.15.16:6789/0 conn(0x7f06080d1410 legacy=0x7f060809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:42.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.941+0000 7f060cfef640 1 -- 172.21.15.146:0/2020207087 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06080705b0 con 0x7f060815e560 2024-09-19T07:42:42.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.941+0000 7f060cfef640 1 --2- 172.21.15.146:0/2020207087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f060815e560 0x7f06080d0ed0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0608073c60 tx=0x7f05fc02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51e9d1f55408ac16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:42.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.941+0000 7f06077fe640 1 -- 172.21.15.146:0/2020207087 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc037070 con 0x7f060815e560 2024-09-19T07:42:42.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.942+0000 7f06077fe640 1 -- 172.21.15.146:0/2020207087 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05fc032440 con 0x7f060815e560 2024-09-19T07:42:42.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.942+0000 7f06077fe640 1 -- 172.21.15.146:0/2020207087 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc032810 con 0x7f060815e560 2024-09-19T07:42:42.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.942+0000 7f060f253640 1 -- 172.21.15.146:0/2020207087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f060815e560 msgr2=0x7f06080d0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:42.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.942+0000 7f060f253640 1 --2- 172.21.15.146:0/2020207087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f060815e560 0x7f06080d0ed0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0608073c60 tx=0x7f05fc02a270 comp rx=0 tx=0).stop 2024-09-19T07:42:42.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.942+0000 7f060f253640 1 -- 172.21.15.146:0/2020207087 shutdown_connections 2024-09-19T07:42:42.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.942+0000 7f060f253640 1 --2- 172.21.15.146:0/2020207087 >> v2:172.21.15.16:3300/0 conn(0x7f0608071090 0x7f0608162030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:42.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.942+0000 7f060f253640 1 --2- 172.21.15.146:0/2020207087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f060815e560 0x7f06080d0ed0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:42.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.942+0000 7f060f253640 1 -- 172.21.15.146:0/2020207087 >> 172.21.15.146:0/2020207087 conn(0x7f06080d2230 msgr2=0x7f06080d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:42.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.942+0000 7f060f253640 1 -- 172.21.15.146:0/2020207087 shutdown_connections 2024-09-19T07:42:42.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.942+0000 7f060f253640 1 -- 172.21.15.146:0/2020207087 wait complete. 2024-09-19T07:42:42.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.943+0000 7f060f253640 1 Processor -- start 2024-09-19T07:42:42.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.944+0000 7f060f253640 1 -- start start 2024-09-19T07:42:42.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.944+0000 7f060f253640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0608071090 0x7f06080b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:42.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.944+0000 7f060f253640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06080d1410 0x7f060814d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:42.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.944+0000 7f060f253640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f060815e560 0x7f06080b7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:42.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.944+0000 7f060f253640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0608162d90 con 0x7f060815e560 2024-09-19T07:42:42.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.944+0000 7f060f253640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0608070b70 con 0x7f0608071090 2024-09-19T07:42:42.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.944+0000 7f060f253640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0608162570 con 0x7f06080d1410 2024-09-19T07:42:42.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.944+0000 7f060cfef640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0608071090 0x7f06080b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:42.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.944+0000 7f0607fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06080d1410 0x7f060814d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:42.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.945+0000 7f060cfef640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0608071090 0x7f06080b50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57982/0 (socket says 172.21.15.146:57982) 2024-09-19T07:42:42.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.945+0000 7f0607fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06080d1410 0x7f060814d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:50536/0 (socket says 172.21.15.146:50536) 2024-09-19T07:42:42.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.945+0000 7f060cfef640 1 -- 172.21.15.146:0/2828818578 learned_addr learned my addr 172.21.15.146:0/2828818578 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-19T07:42:42.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.945+0000 7f060d7f0640 1 --2- 172.21.15.146:0/2828818578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f060815e560 0x7f06080b7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:42.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.945+0000 7f060cfef640 1 -- 172.21.15.146:0/2828818578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06080d1410 msgr2=0x7f060814d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:42.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.945+0000 7f060cfef640 1 --2- 172.21.15.146:0/2828818578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06080d1410 0x7f060814d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:42.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.945+0000 7f060cfef640 1 -- 172.21.15.146:0/2828818578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f060815e560 msgr2=0x7f06080b7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:42.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.945+0000 7f060cfef640 1 --2- 172.21.15.146:0/2828818578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f060815e560 0x7f06080b7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:42.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.945+0000 7f060cfef640 1 -- 172.21.15.146:0/2828818578 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06080b02a0 con 0x7f0608071090 2024-09-19T07:42:42.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.945+0000 7f0607fff640 1 --2- 172.21.15.146:0/2828818578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06080d1410 0x7f060814d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:42.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.945+0000 7f060cfef640 1 --2- 172.21.15.146:0/2828818578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0608071090 0x7f06080b50a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f05fc02a2b0 tx=0x7f05fc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:42.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.945+0000 7f06057fa640 1 -- 172.21.15.146:0/2828818578 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc037040 con 0x7f0608071090 2024-09-19T07:42:42.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.945+0000 7f06057fa640 1 -- 172.21.15.146:0/2828818578 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05fc042070 con 0x7f0608071090 2024-09-19T07:42:42.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.945+0000 7f06057fa640 1 -- 172.21.15.146:0/2828818578 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc030880 con 0x7f0608071090 2024-09-19T07:42:42.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.945+0000 7f060f253640 1 -- 172.21.15.146:0/2828818578 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06080b0580 con 0x7f0608071090 2024-09-19T07:42:42.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.945+0000 7f060f253640 1 -- 172.21.15.146:0/2828818578 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06080b0b40 con 0x7f0608071090 2024-09-19T07:42:42.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.945+0000 7f06057fa640 1 -- 172.21.15.146:0/2828818578 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f05fc030a20 con 0x7f0608071090 2024-09-19T07:42:42.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.945+0000 7f06057fa640 1 -- 172.21.15.146:0/2828818578 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f05fc047bc0 con 0x7f0608071090 2024-09-19T07:42:42.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.946+0000 7f060f253640 1 -- 172.21.15.146:0/2828818578 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06080b0d80 con 0x7f0608071090 2024-09-19T07:42:42.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:42.950+0000 7f06057fa640 1 -- 172.21.15.146:0/2828818578 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f06080b0d80 con 0x7f0608071090 2024-09-19T07:42:42.959 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:42.959+0000 7feda872f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T07:42:42.965 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:42.965+0000 7fdcf962f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T07:42:43.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.020+0000 7f060f253640 1 -- 172.21.15.146:0/2828818578 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "90068366-8d2f-411b-a051-3eb4e2cc6032", "id": 0} v 0) -- 0x7f0608162900 con 0x7f0608071090 2024-09-19T07:42:43.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.023+0000 7f06057fa640 1 -- 172.21.15.146:0/2828818578 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "90068366-8d2f-411b-a051-3eb4e2cc6032", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f05fc047460 con 0x7f0608071090 2024-09-19T07:42:43.024 INFO:teuthology.orchestra.run.smithi146.stdout:0 2024-09-19T07:42:43.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.024+0000 7f060f253640 1 -- 172.21.15.146:0/2828818578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0608071090 msgr2=0x7f06080b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.024+0000 7f060f253640 1 --2- 172.21.15.146:0/2828818578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0608071090 0x7f06080b50a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f05fc02a2b0 tx=0x7f05fc0046c0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.024+0000 7f060f253640 1 -- 172.21.15.146:0/2828818578 shutdown_connections 2024-09-19T07:42:43.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.024+0000 7f060f253640 1 --2- 172.21.15.146:0/2828818578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f060815e560 0x7f06080b7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.024+0000 7f060f253640 1 --2- 172.21.15.146:0/2828818578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06080d1410 0x7f060814d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.024+0000 7f060f253640 1 --2- 172.21.15.146:0/2828818578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0608071090 0x7f06080b50a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.024+0000 7f060f253640 1 -- 172.21.15.146:0/2828818578 >> 172.21.15.146:0/2828818578 conn(0x7f06080d2230 msgr2=0x7f060809a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:43.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.025+0000 7f060f253640 1 -- 172.21.15.146:0/2828818578 shutdown_connections 2024-09-19T07:42:43.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.025+0000 7f060f253640 1 -- 172.21.15.146:0/2828818578 wait complete. 2024-09-19T07:42:43.035 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new df8b4cc6-fe35-4f67-9422-260cb7702f5f 1 2024-09-19T07:42:43.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.133+0000 7f6157549640 1 Processor -- start 2024-09-19T07:42:43.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.134+0000 7f6157549640 1 -- start start 2024-09-19T07:42:43.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.134+0000 7f6157549640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f615015c3c0 0x7f61500cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.134+0000 7f6157549640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61500cd030 con 0x7f615009b100 2024-09-19T07:42:43.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.134+0000 7f6157549640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61500cd310 con 0x7f61500d0790 2024-09-19T07:42:43.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.134+0000 7f6157549640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f61500cd5f0 con 0x7f615015c3c0 2024-09-19T07:42:43.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.135+0000 7f61552e5640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f615015c3c0 0x7f61500cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.135+0000 7f61552e5640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f615015c3c0 0x7f61500cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57992/0 (socket says 172.21.15.146:57992) 2024-09-19T07:42:43.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.135+0000 7f61552e5640 1 -- 172.21.15.146:0/3856729505 learned_addr learned my addr 172.21.15.146:0/3856729505 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-19T07:42:43.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.135+0000 7f61552e5640 1 -- 172.21.15.146:0/3856729505 >> v1:172.21.15.146:6789/0 conn(0x7f61500d0790 legacy=0x7f615009a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.135+0000 7f61552e5640 1 -- 172.21.15.146:0/3856729505 >> v1:172.21.15.16:6789/0 conn(0x7f615009b100 legacy=0x7f61500c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:43.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.135+0000 7f61552e5640 1 -- 172.21.15.146:0/3856729505 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61500cd8d0 con 0x7f615015c3c0 2024-09-19T07:42:43.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.135+0000 7f61552e5640 1 --2- 172.21.15.146:0/3856729505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f615015c3c0 0x7f61500cddf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f61480098a0 tx=0x7f614802a2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae58ace3461ce2a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:43.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.136+0000 7f614ffff640 1 -- 172.21.15.146:0/3856729505 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148037070 con 0x7f615015c3c0 2024-09-19T07:42:43.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.136+0000 7f614ffff640 1 -- 172.21.15.146:0/3856729505 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6148032440 con 0x7f615015c3c0 2024-09-19T07:42:43.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.136+0000 7f614ffff640 1 -- 172.21.15.146:0/3856729505 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148032810 con 0x7f615015c3c0 2024-09-19T07:42:43.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.136+0000 7f6157549640 1 -- 172.21.15.146:0/3856729505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f615015c3c0 msgr2=0x7f61500cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.136+0000 7f6157549640 1 --2- 172.21.15.146:0/3856729505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f615015c3c0 0x7f61500cddf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f61480098a0 tx=0x7f614802a2e0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.136+0000 7f6157549640 1 -- 172.21.15.146:0/3856729505 shutdown_connections 2024-09-19T07:42:43.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.136+0000 7f6157549640 1 --2- 172.21.15.146:0/3856729505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f615015c3c0 0x7f61500cddf0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.136+0000 7f6157549640 1 -- 172.21.15.146:0/3856729505 >> 172.21.15.146:0/3856729505 conn(0x7f61500b3ca0 msgr2=0x7f61500b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:43.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.136+0000 7f6157549640 1 -- 172.21.15.146:0/3856729505 shutdown_connections 2024-09-19T07:42:43.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.136+0000 7f6157549640 1 -- 172.21.15.146:0/3856729505 wait complete. 2024-09-19T07:42:43.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.138+0000 7f6157549640 1 Processor -- start 2024-09-19T07:42:43.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.138+0000 7f6157549640 1 -- start start 2024-09-19T07:42:43.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.138+0000 7f6157549640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f615009b100 0x7f61500cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.138+0000 7f6157549640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61500d0790 0x7f61501542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.138+0000 7f6157549640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f615015c3c0 0x7f61501f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.138+0000 7f6157549640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f615009bd30 con 0x7f61500d0790 2024-09-19T07:42:43.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.138+0000 7f6157549640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f615009bbb0 con 0x7f615015c3c0 2024-09-19T07:42:43.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.138+0000 7f6157549640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f61500ccc50 con 0x7f615009b100 2024-09-19T07:42:43.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.138+0000 7f61552e5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f615009b100 0x7f61500cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.139+0000 7f61552e5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f615009b100 0x7f61500cbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:50548/0 (socket says 172.21.15.146:50548) 2024-09-19T07:42:43.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.139+0000 7f6155ae6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f615015c3c0 0x7f61501f7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.139+0000 7f61552e5640 1 -- 172.21.15.146:0/4218545384 learned_addr learned my addr 172.21.15.146:0/4218545384 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-19T07:42:43.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.139+0000 7f6154ae4640 1 --2- 172.21.15.146:0/4218545384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61500d0790 0x7f61501542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.139+0000 7f61552e5640 1 -- 172.21.15.146:0/4218545384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f615015c3c0 msgr2=0x7f61501f7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.139+0000 7f61552e5640 1 --2- 172.21.15.146:0/4218545384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f615015c3c0 0x7f61501f7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.139+0000 7f61552e5640 1 -- 172.21.15.146:0/4218545384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61500d0790 msgr2=0x7f61501542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.139+0000 7f61552e5640 1 --2- 172.21.15.146:0/4218545384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61500d0790 0x7f61501542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.139+0000 7f61552e5640 1 -- 172.21.15.146:0/4218545384 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615006dd30 con 0x7f615009b100 2024-09-19T07:42:43.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.139+0000 7f6155ae6640 1 --2- 172.21.15.146:0/4218545384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f615015c3c0 0x7f61501f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:43.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.139+0000 7f61552e5640 1 --2- 172.21.15.146:0/4218545384 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f615009b100 0x7f61500cbef0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6148000c00 tx=0x7f61480046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:43.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.139+0000 7f614dffb640 1 -- 172.21.15.146:0/4218545384 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148037040 con 0x7f615009b100 2024-09-19T07:42:43.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.139+0000 7f614dffb640 1 -- 172.21.15.146:0/4218545384 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6148004250 con 0x7f615009b100 2024-09-19T07:42:43.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.139+0000 7f614dffb640 1 -- 172.21.15.146:0/4218545384 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148003610 con 0x7f615009b100 2024-09-19T07:42:43.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.139+0000 7f6157549640 1 -- 172.21.15.146:0/4218545384 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615006e010 con 0x7f615009b100 2024-09-19T07:42:43.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.139+0000 7f6157549640 1 -- 172.21.15.146:0/4218545384 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f615006e5d0 con 0x7f615009b100 2024-09-19T07:42:43.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.140+0000 7f614dffb640 1 -- 172.21.15.146:0/4218545384 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6148003b40 con 0x7f615009b100 2024-09-19T07:42:43.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.140+0000 7f614dffb640 1 -- 172.21.15.146:0/4218545384 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f614803dd50 con 0x7f615009b100 2024-09-19T07:42:43.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.140+0000 7f6157549640 1 -- 172.21.15.146:0/4218545384 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6150070f40 con 0x7f615009b100 2024-09-19T07:42:43.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.143+0000 7f614dffb640 1 -- 172.21.15.146:0/4218545384 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6150070f40 con 0x7f615009b100 2024-09-19T07:42:43.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.212+0000 7f6157549640 1 -- 172.21.15.146:0/4218545384 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd new", "uuid": "df8b4cc6-fe35-4f67-9422-260cb7702f5f", "id": 1} v 0) -- 0x7f61500cddf0 con 0x7f615009b100 2024-09-19T07:42:43.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.215+0000 7f614dffb640 1 -- 172.21.15.146:0/4218545384 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "df8b4cc6-fe35-4f67-9422-260cb7702f5f", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f614803d4e0 con 0x7f615009b100 2024-09-19T07:42:43.216 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-09-19T07:42:43.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.216+0000 7f6157549640 1 -- 172.21.15.146:0/4218545384 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f615009b100 msgr2=0x7f61500cbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.216+0000 7f6157549640 1 --2- 172.21.15.146:0/4218545384 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f615009b100 0x7f61500cbef0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6148000c00 tx=0x7f61480046c0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.216+0000 7f6157549640 1 -- 172.21.15.146:0/4218545384 shutdown_connections 2024-09-19T07:42:43.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.216+0000 7f6157549640 1 --2- 172.21.15.146:0/4218545384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f615015c3c0 0x7f61501f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.216+0000 7f6157549640 1 --2- 172.21.15.146:0/4218545384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61500d0790 0x7f61501542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.216+0000 7f6157549640 1 --2- 172.21.15.146:0/4218545384 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f615009b100 0x7f61500cbef0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.216+0000 7f6157549640 1 -- 172.21.15.146:0/4218545384 >> 172.21.15.146:0/4218545384 conn(0x7f61500b3ca0 msgr2=0x7f6150070830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:43.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.216+0000 7f6157549640 1 -- 172.21.15.146:0/4218545384 shutdown_connections 2024-09-19T07:42:43.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.216+0000 7f6157549640 1 -- 172.21.15.146:0/4218545384 wait complete. 2024-09-19T07:42:43.227 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 049b9893-2f23-45f9-af8e-2e70d2f7ceb0 2 2024-09-19T07:42:43.306 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:43.306+0000 7feda872f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T07:42:43.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.327+0000 7f26ced66640 1 Processor -- start 2024-09-19T07:42:43.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.327+0000 7f26ced66640 1 -- start start 2024-09-19T07:42:43.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.328+0000 7f26ced66640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f26c80b4d90 0x7f26c80b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.328+0000 7f26ced66640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f26c80b45c0 0x7f26c815ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.328+0000 7f26ced66640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26c815c610 con 0x7f26c80b5170 2024-09-19T07:42:43.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.328+0000 7f26ced66640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f26c815d220 con 0x7f26c80b4d90 2024-09-19T07:42:43.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.328+0000 7f26ced66640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f26c815c8f0 con 0x7f26c80b45c0 2024-09-19T07:42:43.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.328+0000 7f26c7fff640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f26c80b45c0 0x7f26c815ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.328+0000 7f26c7fff640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f26c80b45c0 0x7f26c815ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:58020/0 (socket says 172.21.15.146:58020) 2024-09-19T07:42:43.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.328+0000 7f26c7fff640 1 -- 172.21.15.146:0/3414851566 learned_addr learned my addr 172.21.15.146:0/3414851566 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-19T07:42:43.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.329+0000 7f26ccb02640 1 --2- 172.21.15.146:0/3414851566 >> v2:172.21.15.16:3300/0 conn(0x7f26c80b4d90 0x7f26c80b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.329+0000 7f26c7fff640 1 -- 172.21.15.146:0/3414851566 >> v2:172.21.15.16:3300/0 conn(0x7f26c80b4d90 msgr2=0x7f26c80b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:43.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.329+0000 7f26c7fff640 1 --2- 172.21.15.146:0/3414851566 >> v2:172.21.15.16:3300/0 conn(0x7f26c80b4d90 0x7f26c80b4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.329+0000 7f26c7fff640 1 -- 172.21.15.146:0/3414851566 >> v1:172.21.15.146:6789/0 conn(0x7f26c80b5170 legacy=0x7f26c815eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.329+0000 7f26c7fff640 1 -- 172.21.15.146:0/3414851566 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c815cbd0 con 0x7f26c80b45c0 2024-09-19T07:42:43.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.329+0000 7f26c7fff640 1 --2- 172.21.15.146:0/3414851566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26c80b45c0 0x7f26c815ac50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f26c8073c10 tx=0x7f26b402a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aae2971358d1629d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:43.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.329+0000 7f26c77fe640 1 -- 172.21.15.146:0/3414851566 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b4037070 con 0x7f26c80b45c0 2024-09-19T07:42:43.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.330+0000 7f26c77fe640 1 -- 172.21.15.146:0/3414851566 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b402ae50 con 0x7f26c80b45c0 2024-09-19T07:42:43.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.330+0000 7f26c77fe640 1 -- 172.21.15.146:0/3414851566 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b4032860 con 0x7f26c80b45c0 2024-09-19T07:42:43.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.330+0000 7f26ced66640 1 -- 172.21.15.146:0/3414851566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26c80b45c0 msgr2=0x7f26c815ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.330+0000 7f26ced66640 1 --2- 172.21.15.146:0/3414851566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26c80b45c0 0x7f26c815ac50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f26c8073c10 tx=0x7f26b402a300 comp rx=0 tx=0).stop 2024-09-19T07:42:43.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.330+0000 7f26ced66640 1 -- 172.21.15.146:0/3414851566 shutdown_connections 2024-09-19T07:42:43.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.330+0000 7f26ced66640 1 --2- 172.21.15.146:0/3414851566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26c80b45c0 0x7f26c815ac50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.330+0000 7f26ced66640 1 --2- 172.21.15.146:0/3414851566 >> v2:172.21.15.16:3300/0 conn(0x7f26c80b4d90 0x7f26c80b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.330+0000 7f26ced66640 1 -- 172.21.15.146:0/3414851566 >> 172.21.15.146:0/3414851566 conn(0x7f26c809a9f0 msgr2=0x7f26c80b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:43.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.331+0000 7f26ced66640 1 -- 172.21.15.146:0/3414851566 shutdown_connections 2024-09-19T07:42:43.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.331+0000 7f26ced66640 1 -- 172.21.15.146:0/3414851566 wait complete. 2024-09-19T07:42:43.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.332+0000 7f26ced66640 1 Processor -- start 2024-09-19T07:42:43.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.332+0000 7f26ced66640 1 -- start start 2024-09-19T07:42:43.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.334+0000 7f26ced66640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26c80b4d90 0x7f26c8148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.334+0000 7f26ced66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c80b5170 0x7f26c806fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.334+0000 7f26ced66640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26c809d510 0x7f26c809d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.334+0000 7f26ced66640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26c815c3d0 con 0x7f26c80b5170 2024-09-19T07:42:43.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.334+0000 7f26ced66640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f26c815c250 con 0x7f26c809d510 2024-09-19T07:42:43.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.334+0000 7f26ced66640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f26c815e6c0 con 0x7f26c80b4d90 2024-09-19T07:42:43.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.334+0000 7f26ccb02640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26c80b4d90 0x7f26c8148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.334+0000 7f26ccb02640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26c80b4d90 0x7f26c8148ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:50550/0 (socket says 172.21.15.146:50550) 2024-09-19T07:42:43.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.334+0000 7f26ccb02640 1 -- 172.21.15.146:0/1473084921 learned_addr learned my addr 172.21.15.146:0/1473084921 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-19T07:42:43.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.334+0000 7f26cd303640 1 --2- 172.21.15.146:0/1473084921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26c809d510 0x7f26c809d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.334+0000 7f26c7fff640 1 --2- 172.21.15.146:0/1473084921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c80b5170 0x7f26c806fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.334+0000 7f26ccb02640 1 -- 172.21.15.146:0/1473084921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26c809d510 msgr2=0x7f26c809d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.334+0000 7f26ccb02640 1 --2- 172.21.15.146:0/1473084921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26c809d510 0x7f26c809d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.334+0000 7f26ccb02640 1 -- 172.21.15.146:0/1473084921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c80b5170 msgr2=0x7f26c806fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.334+0000 7f26ccb02640 1 --2- 172.21.15.146:0/1473084921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c80b5170 0x7f26c806fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.334+0000 7f26ccb02640 1 -- 172.21.15.146:0/1473084921 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c80ae6c0 con 0x7f26c80b4d90 2024-09-19T07:42:43.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.334+0000 7f26ccb02640 1 --2- 172.21.15.146:0/1473084921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26c80b4d90 0x7f26c8148ca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00d800 tx=0x7f26bc00dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:43.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.335+0000 7f26c57fa640 1 -- 172.21.15.146:0/1473084921 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc00b6c0 con 0x7f26c80b4d90 2024-09-19T07:42:43.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.335+0000 7f26c57fa640 1 -- 172.21.15.146:0/1473084921 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26bc019070 con 0x7f26c80b4d90 2024-09-19T07:42:43.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.335+0000 7f26c57fa640 1 -- 172.21.15.146:0/1473084921 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc00ba80 con 0x7f26c80b4d90 2024-09-19T07:42:43.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.335+0000 7f26ced66640 1 -- 172.21.15.146:0/1473084921 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c80ae9a0 con 0x7f26c80b4d90 2024-09-19T07:42:43.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.335+0000 7f26ced66640 1 -- 172.21.15.146:0/1473084921 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c80aef60 con 0x7f26c80b4d90 2024-09-19T07:42:43.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.335+0000 7f26c57fa640 1 -- 172.21.15.146:0/1473084921 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f26bc00bc20 con 0x7f26c80b4d90 2024-09-19T07:42:43.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.335+0000 7f26c57fa640 1 -- 172.21.15.146:0/1473084921 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f26bc01e880 con 0x7f26c80b4d90 2024-09-19T07:42:43.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.336+0000 7f26ced66640 1 -- 172.21.15.146:0/1473084921 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c80b4080 con 0x7f26c80b4d90 2024-09-19T07:42:43.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.338+0000 7f26c57fa640 1 -- 172.21.15.146:0/1473084921 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f26bc01ea80 con 0x7f26c80b4d90 2024-09-19T07:42:43.340 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:43.340+0000 7fdcf962f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T07:42:43.359 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:43.358+0000 7feda872f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T07:42:43.398 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:43.398+0000 7fdcf962f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T07:42:43.407 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.407+0000 7f26ced66640 1 -- 172.21.15.146:0/1473084921 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd new", "uuid": "049b9893-2f23-45f9-af8e-2e70d2f7ceb0", "id": 2} v 0) -- 0x7f26c8154fc0 con 0x7f26c80b4d90 2024-09-19T07:42:43.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.410+0000 7f26c57fa640 1 -- 172.21.15.146:0/1473084921 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "049b9893-2f23-45f9-af8e-2e70d2f7ceb0", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f26bc01e460 con 0x7f26c80b4d90 2024-09-19T07:42:43.411 INFO:teuthology.orchestra.run.smithi146.stdout:2 2024-09-19T07:42:43.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.411+0000 7f26ced66640 1 -- 172.21.15.146:0/1473084921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26c80b4d90 msgr2=0x7f26c8148ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.411+0000 7f26ced66640 1 --2- 172.21.15.146:0/1473084921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26c80b4d90 0x7f26c8148ca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00d800 tx=0x7f26bc00dcc0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.411+0000 7f26ced66640 1 -- 172.21.15.146:0/1473084921 shutdown_connections 2024-09-19T07:42:43.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.411+0000 7f26ced66640 1 --2- 172.21.15.146:0/1473084921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26c809d510 0x7f26c809d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.411+0000 7f26ced66640 1 --2- 172.21.15.146:0/1473084921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c80b5170 0x7f26c806fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.411+0000 7f26ced66640 1 --2- 172.21.15.146:0/1473084921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26c80b4d90 0x7f26c8148ca0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.411+0000 7f26ced66640 1 -- 172.21.15.146:0/1473084921 >> 172.21.15.146:0/1473084921 conn(0x7f26c809a9f0 msgr2=0x7f26c8158ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:43.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.412+0000 7f26ced66640 1 -- 172.21.15.146:0/1473084921 shutdown_connections 2024-09-19T07:42:43.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.412+0000 7f26ced66640 1 -- 172.21.15.146:0/1473084921 wait complete. 2024-09-19T07:42:43.422 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 1c8063a9-686f-4a4d-9604-41ddd6ee0660 3 2024-09-19T07:42:43.517 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:43.517+0000 7feda872f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T07:42:43.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.520+0000 7faa5406e640 1 Processor -- start 2024-09-19T07:42:43.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.520+0000 7faa5406e640 1 -- start start 2024-09-19T07:42:43.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.521+0000 7faa5406e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faa4c0af860 0x7faa4c06f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.521+0000 7faa5406e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa4c070ab0 0x7faa4c0afc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.521+0000 7faa5406e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa4c0b0180 con 0x7faa4c0ad860 2024-09-19T07:42:43.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.521+0000 7faa5406e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7faa4c06f7e0 con 0x7faa4c070ab0 2024-09-19T07:42:43.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.521+0000 7faa5406e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7faa4c06fac0 con 0x7faa4c0af860 2024-09-19T07:42:43.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.521+0000 7faa51e0a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faa4c0af860 0x7faa4c06f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.521+0000 7faa51e0a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faa4c0af860 0x7faa4c06f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:58034/0 (socket says 172.21.15.146:58034) 2024-09-19T07:42:43.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.521+0000 7faa51e0a640 1 -- 172.21.15.146:0/3391641569 learned_addr learned my addr 172.21.15.146:0/3391641569 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-19T07:42:43.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.521+0000 7faa51609640 1 --2- 172.21.15.146:0/3391641569 >> v2:172.21.15.16:3300/0 conn(0x7faa4c070ab0 0x7faa4c0afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.521+0000 7faa51e0a640 1 -- 172.21.15.146:0/3391641569 >> v2:172.21.15.16:3300/0 conn(0x7faa4c070ab0 msgr2=0x7faa4c0afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:43.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.521+0000 7faa51e0a640 1 --2- 172.21.15.146:0/3391641569 >> v2:172.21.15.16:3300/0 conn(0x7faa4c070ab0 0x7faa4c0afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.521+0000 7faa51e0a640 1 -- 172.21.15.146:0/3391641569 >> v1:172.21.15.146:6789/0 conn(0x7faa4c0ad860 legacy=0x7faa4c0adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.521+0000 7faa51e0a640 1 -- 172.21.15.146:0/3391641569 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4c06fda0 con 0x7faa4c0af860 2024-09-19T07:42:43.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.521+0000 7faa51609640 1 --2- 172.21.15.146:0/3391641569 >> v2:172.21.15.16:3300/0 conn(0x7faa4c070ab0 0x7faa4c0afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:42:43.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.522+0000 7faa51e0a640 1 --2- 172.21.15.146:0/3391641569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa4c0af860 0x7faa4c06f0b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0737e0 tx=0x7faa3c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14b949d1ce56239d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:43.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.522+0000 7faa50e08640 1 -- 172.21.15.146:0/3391641569 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c037070 con 0x7faa4c0af860 2024-09-19T07:42:43.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.522+0000 7faa50e08640 1 -- 172.21.15.146:0/3391641569 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa3c032440 con 0x7faa4c0af860 2024-09-19T07:42:43.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.522+0000 7faa50e08640 1 -- 172.21.15.146:0/3391641569 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c032810 con 0x7faa4c0af860 2024-09-19T07:42:43.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.523+0000 7faa5406e640 1 -- 172.21.15.146:0/3391641569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa4c0af860 msgr2=0x7faa4c06f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.523+0000 7faa5406e640 1 --2- 172.21.15.146:0/3391641569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa4c0af860 0x7faa4c06f0b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0737e0 tx=0x7faa3c02a270 comp rx=0 tx=0).stop 2024-09-19T07:42:43.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.523+0000 7faa5406e640 1 -- 172.21.15.146:0/3391641569 shutdown_connections 2024-09-19T07:42:43.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.523+0000 7faa5406e640 1 --2- 172.21.15.146:0/3391641569 >> v2:172.21.15.16:3300/0 conn(0x7faa4c070ab0 0x7faa4c0afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.523+0000 7faa5406e640 1 --2- 172.21.15.146:0/3391641569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa4c0af860 0x7faa4c06f0b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.523+0000 7faa5406e640 1 -- 172.21.15.146:0/3391641569 >> 172.21.15.146:0/3391641569 conn(0x7faa4c073010 msgr2=0x7faa4c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:43.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.523+0000 7faa5406e640 1 -- 172.21.15.146:0/3391641569 shutdown_connections 2024-09-19T07:42:43.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.523+0000 7faa5406e640 1 -- 172.21.15.146:0/3391641569 wait complete. 2024-09-19T07:42:43.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.524+0000 7faa5406e640 1 Processor -- start 2024-09-19T07:42:43.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.524+0000 7faa5406e640 1 -- start start 2024-09-19T07:42:43.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.525+0000 7faa5406e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa4c070ab0 0x7faa4c14e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.525+0000 7faa5406e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa4c0ad860 0x7faa4c14d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.525+0000 7faa5406e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa4c0af860 0x7faa4c14d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.525+0000 7faa5406e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c070f20 con 0x7faa4c0af860 2024-09-19T07:42:43.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.525+0000 7faa5406e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c0b5150 con 0x7faa4c070ab0 2024-09-19T07:42:43.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.525+0000 7faa5406e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faa4c15e4f0 con 0x7faa4c0ad860 2024-09-19T07:42:43.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.525+0000 7faa51e0a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa4c070ab0 0x7faa4c14e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.525+0000 7faa51e0a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa4c070ab0 0x7faa4c14e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:58040/0 (socket says 172.21.15.146:58040) 2024-09-19T07:42:43.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.525+0000 7faa51e0a640 1 -- 172.21.15.146:0/3203813107 learned_addr learned my addr 172.21.15.146:0/3203813107 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-19T07:42:43.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.525+0000 7faa51609640 1 --2- 172.21.15.146:0/3203813107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa4c0ad860 0x7faa4c14d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.525+0000 7faa5260b640 1 --2- 172.21.15.146:0/3203813107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa4c0af860 0x7faa4c14d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.525+0000 7faa51e0a640 1 -- 172.21.15.146:0/3203813107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa4c0ad860 msgr2=0x7faa4c14d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.525+0000 7faa51e0a640 1 --2- 172.21.15.146:0/3203813107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa4c0ad860 0x7faa4c14d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.525+0000 7faa51e0a640 1 -- 172.21.15.146:0/3203813107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa4c0af860 msgr2=0x7faa4c14d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.525+0000 7faa51e0a640 1 --2- 172.21.15.146:0/3203813107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa4c0af860 0x7faa4c14d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.525+0000 7faa51e0a640 1 -- 172.21.15.146:0/3203813107 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4c0cec90 con 0x7faa4c070ab0 2024-09-19T07:42:43.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.525+0000 7faa51e0a640 1 --2- 172.21.15.146:0/3203813107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa4c070ab0 0x7faa4c14e780 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faa3c02a2b0 tx=0x7faa3c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:43.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.526+0000 7faa427fc640 1 -- 172.21.15.146:0/3203813107 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c041020 con 0x7faa4c070ab0 2024-09-19T07:42:43.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.526+0000 7faa427fc640 1 -- 172.21.15.146:0/3203813107 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa3c0306c0 con 0x7faa4c070ab0 2024-09-19T07:42:43.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.526+0000 7faa427fc640 1 -- 172.21.15.146:0/3203813107 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c037040 con 0x7faa4c070ab0 2024-09-19T07:42:43.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.526+0000 7faa5406e640 1 -- 172.21.15.146:0/3203813107 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa4c0cef70 con 0x7faa4c070ab0 2024-09-19T07:42:43.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.526+0000 7faa5406e640 1 -- 172.21.15.146:0/3203813107 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa4c1e3930 con 0x7faa4c070ab0 2024-09-19T07:42:43.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.526+0000 7faa427fc640 1 -- 172.21.15.146:0/3203813107 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faa3c044020 con 0x7faa4c070ab0 2024-09-19T07:42:43.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.526+0000 7faa427fc640 1 -- 172.21.15.146:0/3203813107 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7faa3c03dc90 con 0x7faa4c070ab0 2024-09-19T07:42:43.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.527+0000 7faa5406e640 1 -- 172.21.15.146:0/3203813107 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa4c0afc40 con 0x7faa4c070ab0 2024-09-19T07:42:43.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.529+0000 7faa427fc640 1 -- 172.21.15.146:0/3203813107 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7faa3c04b030 con 0x7faa4c070ab0 2024-09-19T07:42:43.555 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:43.555+0000 7fdcf962f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T07:42:43.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.595+0000 7faa5406e640 1 -- 172.21.15.146:0/3203813107 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1c8063a9-686f-4a4d-9604-41ddd6ee0660", "id": 3} v 0) -- 0x7faa4c0b17e0 con 0x7faa4c070ab0 2024-09-19T07:42:43.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.598+0000 7faa427fc640 1 -- 172.21.15.146:0/3203813107 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1c8063a9-686f-4a4d-9604-41ddd6ee0660", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7faa3c03d420 con 0x7faa4c070ab0 2024-09-19T07:42:43.599 INFO:teuthology.orchestra.run.smithi146.stdout:3 2024-09-19T07:42:43.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.599+0000 7faa5406e640 1 -- 172.21.15.146:0/3203813107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa4c070ab0 msgr2=0x7faa4c14e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.599+0000 7faa5406e640 1 --2- 172.21.15.146:0/3203813107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa4c070ab0 0x7faa4c14e780 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faa3c02a2b0 tx=0x7faa3c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.599+0000 7faa5406e640 1 -- 172.21.15.146:0/3203813107 shutdown_connections 2024-09-19T07:42:43.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.599+0000 7faa5406e640 1 --2- 172.21.15.146:0/3203813107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa4c0af860 0x7faa4c14d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.599+0000 7faa5406e640 1 --2- 172.21.15.146:0/3203813107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa4c0ad860 0x7faa4c14d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.599+0000 7faa5406e640 1 --2- 172.21.15.146:0/3203813107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa4c070ab0 0x7faa4c14e780 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.599+0000 7faa5406e640 1 -- 172.21.15.146:0/3203813107 >> 172.21.15.146:0/3203813107 conn(0x7faa4c073010 msgr2=0x7faa4c0b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:43.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.600+0000 7faa5406e640 1 -- 172.21.15.146:0/3203813107 shutdown_connections 2024-09-19T07:42:43.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.600+0000 7faa5406e640 1 -- 172.21.15.146:0/3203813107 wait complete. 2024-09-19T07:42:43.610 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 0e08bb0f-6f52-4c20-b8d2-8716f38382aa 4 2024-09-19T07:42:43.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.709+0000 7f1ec328d640 1 Processor -- start 2024-09-19T07:42:43.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.709+0000 7f1ec328d640 1 -- start start 2024-09-19T07:42:43.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.710+0000 7f1ec328d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ebc15e6c0 0x7f1ebc166130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.710+0000 7f1ec328d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ebc0b4790 con 0x7f1ebc0b4d90 2024-09-19T07:42:43.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.710+0000 7f1ec328d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ebc15c610 con 0x7f1ebc0b5170 2024-09-19T07:42:43.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.710+0000 7f1ec328d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f1ebc15d220 con 0x7f1ebc15e6c0 2024-09-19T07:42:43.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.710+0000 7f1ec0828640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f1ebc0b5170 0x7f1ebc15ac50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:59440/0 (socket says 172.21.15.146:59440) 2024-09-19T07:42:43.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.710+0000 7f1ec0828640 1 -- 172.21.15.146:0/1567205570 learned_addr learned my addr 172.21.15.146:0/1567205570 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-19T07:42:43.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.710+0000 7f1ec182a640 1 --2- 172.21.15.146:0/1567205570 >> v2:172.21.15.16:3300/0 conn(0x7f1ebc15e6c0 0x7f1ebc166130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1eb3fff640 1 -- 172.21.15.146:0/1567205570 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2596680748 0 0) 0x7f1ebc15c610 con 0x7f1ebc0b5170 2024-09-19T07:42:43.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1eb3fff640 1 -- 172.21.15.146:0/1567205570 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ea8003520 con 0x7f1ebc0b5170 2024-09-19T07:42:43.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1eb3fff640 1 -- 172.21.15.146:0/1567205570 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 737134346 0 0) 0x7f1ea8003520 con 0x7f1ebc0b5170 2024-09-19T07:42:43.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1eb3fff640 1 -- 172.21.15.146:0/1567205570 >> v2:172.21.15.16:3300/0 conn(0x7f1ebc15e6c0 msgr2=0x7f1ebc166130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:43.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1eb3fff640 1 --2- 172.21.15.146:0/1567205570 >> v2:172.21.15.16:3300/0 conn(0x7f1ebc15e6c0 0x7f1ebc166130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1eb3fff640 1 -- 172.21.15.146:0/1567205570 >> v1:172.21.15.16:6789/0 conn(0x7f1ebc0b4d90 legacy=0x7f1ebc0b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1eb3fff640 1 -- 172.21.15.146:0/1567205570 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ebc15c8f0 con 0x7f1ebc0b5170 2024-09-19T07:42:43.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1eb3fff640 1 -- 172.21.15.146:0/1567205570 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f1eb8001670 con 0x7f1ebc0b5170 2024-09-19T07:42:43.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1eb3fff640 1 -- 172.21.15.146:0/1567205570 >> v1:172.21.15.146:6789/0 conn(0x7f1ebc0b5170 legacy=0x7f1ebc15ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1eb3fff640 1 --2- 172.21.15.146:0/1567205570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea80039b0 0x7f1ea8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1eb3fff640 1 --2- 172.21.15.146:0/1567205570 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ea8004800 0x7f1ea8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1eb3fff640 1 --2- 172.21.15.146:0/1567205570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea80051c0 0x7f1ea800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1eb3fff640 1 -- 172.21.15.146:0/1567205570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc15d220 con 0x7f1ea80051c0 2024-09-19T07:42:43.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1eb3fff640 1 -- 172.21.15.146:0/1567205570 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea800c060 con 0x7f1ea80039b0 2024-09-19T07:42:43.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1eb3fff640 1 -- 172.21.15.146:0/1567205570 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea800c260 con 0x7f1ea8004800 2024-09-19T07:42:43.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1ec0828640 1 --2- 172.21.15.146:0/1567205570 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ea8004800 0x7f1ea8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1ec1029640 1 --2- 172.21.15.146:0/1567205570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea80039b0 0x7f1ea8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1ec182a640 1 --2- 172.21.15.146:0/1567205570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea80051c0 0x7f1ea800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1ec1029640 1 -- 172.21.15.146:0/1567205570 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ea8004800 msgr2=0x7f1ea8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:43.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1ec1029640 1 --2- 172.21.15.146:0/1567205570 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ea8004800 0x7f1ea8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1ec1029640 1 -- 172.21.15.146:0/1567205570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea80051c0 msgr2=0x7f1ea800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:43.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1ec1029640 1 --2- 172.21.15.146:0/1567205570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea80051c0 0x7f1ea800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1ec1029640 1 -- 172.21.15.146:0/1567205570 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea8003520 con 0x7f1ea80039b0 2024-09-19T07:42:43.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1ec182a640 1 --2- 172.21.15.146:0/1567205570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea80051c0 0x7f1ea800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:42:43.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.711+0000 7f1ec182a640 1 -- 172.21.15.146:0/1567205570 reap_dead start 2024-09-19T07:42:43.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.712+0000 7f1ec1029640 1 --2- 172.21.15.146:0/1567205570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea80039b0 0x7f1ea8003dd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1eac001750 tx=0x7f1ea8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43ab82545345f44a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:43.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.712+0000 7f1eb3fff640 1 -- 172.21.15.146:0/1567205570 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac001370 con 0x7f1ea80039b0 2024-09-19T07:42:43.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.712+0000 7f1eb3fff640 1 -- 172.21.15.146:0/1567205570 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eac0052d0 con 0x7f1ea80039b0 2024-09-19T07:42:43.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.712+0000 7f1eb3fff640 1 -- 172.21.15.146:0/1567205570 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac02abc0 con 0x7f1ea80039b0 2024-09-19T07:42:43.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.713+0000 7f1ec328d640 1 -- 172.21.15.146:0/1567205570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea80039b0 msgr2=0x7f1ea8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.713+0000 7f1ec328d640 1 --2- 172.21.15.146:0/1567205570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea80039b0 0x7f1ea8003dd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1eac001750 tx=0x7f1ea8005830 comp rx=0 tx=0).stop 2024-09-19T07:42:43.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.713+0000 7f1ec328d640 1 -- 172.21.15.146:0/1567205570 shutdown_connections 2024-09-19T07:42:43.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.713+0000 7f1ec328d640 1 --2- 172.21.15.146:0/1567205570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ea80039b0 0x7f1ea8003dd0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.713+0000 7f1ec328d640 1 -- 172.21.15.146:0/1567205570 >> 172.21.15.146:0/1567205570 conn(0x7f1ebc09a9f0 msgr2=0x7f1ebc0b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:43.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.713+0000 7f1ec328d640 1 -- 172.21.15.146:0/1567205570 shutdown_connections 2024-09-19T07:42:43.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.713+0000 7f1ec328d640 1 -- 172.21.15.146:0/1567205570 wait complete. 2024-09-19T07:42:43.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.714+0000 7f1ec328d640 1 Processor -- start 2024-09-19T07:42:43.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.714+0000 7f1ec328d640 1 -- start start 2024-09-19T07:42:43.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.715+0000 7f1ec328d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ea80039b0 0x7f1ebc13cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.715+0000 7f1ec328d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ebc14f480 0x7f1ebc159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.715+0000 7f1ec328d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ebc14f900 0x7f1ebc159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.715+0000 7f1ec328d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc15c250 con 0x7f1ebc14f900 2024-09-19T07:42:43.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.715+0000 7f1ec328d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1ebc15ceb0 con 0x7f1ebc14f480 2024-09-19T07:42:43.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.715+0000 7f1ec328d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1ebc15c3d0 con 0x7f1ea80039b0 2024-09-19T07:42:43.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.715+0000 7f1ec1029640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ea80039b0 0x7f1ebc13cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.715+0000 7f1ec1029640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ea80039b0 0x7f1ebc13cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:50558/0 (socket says 172.21.15.146:50558) 2024-09-19T07:42:43.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.715+0000 7f1ec1029640 1 -- 172.21.15.146:0/331764877 learned_addr learned my addr 172.21.15.146:0/331764877 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-19T07:42:43.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.715+0000 7f1ec0828640 1 --2- 172.21.15.146:0/331764877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ebc14f480 0x7f1ebc159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.715+0000 7f1ec182a640 1 --2- 172.21.15.146:0/331764877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ebc14f900 0x7f1ebc159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.715+0000 7f1ec1029640 1 -- 172.21.15.146:0/331764877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ebc14f480 msgr2=0x7f1ebc159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.715+0000 7f1ec1029640 1 --2- 172.21.15.146:0/331764877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ebc14f480 0x7f1ebc159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.715+0000 7f1ec1029640 1 -- 172.21.15.146:0/331764877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ebc14f900 msgr2=0x7f1ebc159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.715+0000 7f1ec1029640 1 --2- 172.21.15.146:0/331764877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ebc14f900 0x7f1ebc159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.715+0000 7f1ec1029640 1 -- 172.21.15.146:0/331764877 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ebc144270 con 0x7f1ea80039b0 2024-09-19T07:42:43.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.715+0000 7f1ec1029640 1 --2- 172.21.15.146:0/331764877 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ea80039b0 0x7f1ebc13cc70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1eac000c00 tx=0x7f1eac001120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:43.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.716+0000 7f1eb1ffb640 1 -- 172.21.15.146:0/331764877 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac03f030 con 0x7f1ea80039b0 2024-09-19T07:42:43.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.716+0000 7f1eb1ffb640 1 -- 172.21.15.146:0/331764877 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eac005480 con 0x7f1ea80039b0 2024-09-19T07:42:43.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.716+0000 7f1ec328d640 1 -- 172.21.15.146:0/331764877 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ebc144550 con 0x7f1ea80039b0 2024-09-19T07:42:43.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.716+0000 7f1eb1ffb640 1 -- 172.21.15.146:0/331764877 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac037040 con 0x7f1ea80039b0 2024-09-19T07:42:43.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.716+0000 7f1ec328d640 1 -- 172.21.15.146:0/331764877 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ebc144b10 con 0x7f1ea80039b0 2024-09-19T07:42:43.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.716+0000 7f1eb1ffb640 1 -- 172.21.15.146:0/331764877 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1eac042070 con 0x7f1ea80039b0 2024-09-19T07:42:43.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.716+0000 7f1eb1ffb640 1 -- 172.21.15.146:0/331764877 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f1eac006cc0 con 0x7f1ea80039b0 2024-09-19T07:42:43.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.717+0000 7f1ec328d640 1 -- 172.21.15.146:0/331764877 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ebc0b4080 con 0x7f1ea80039b0 2024-09-19T07:42:43.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.719+0000 7f1eb1ffb640 1 -- 172.21.15.146:0/331764877 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1eac001370 con 0x7f1ea80039b0 2024-09-19T07:42:43.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.788+0000 7f1ec328d640 1 -- 172.21.15.146:0/331764877 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0e08bb0f-6f52-4c20-b8d2-8716f38382aa", "id": 4} v 0) -- 0x7f1ebc06e9d0 con 0x7f1ea80039b0 2024-09-19T07:42:43.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.791+0000 7f1eb1ffb640 1 -- 172.21.15.146:0/331764877 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0e08bb0f-6f52-4c20-b8d2-8716f38382aa", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f1eac0067d0 con 0x7f1ea80039b0 2024-09-19T07:42:43.791 INFO:teuthology.orchestra.run.smithi146.stdout:4 2024-09-19T07:42:43.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.792+0000 7f1ec328d640 1 -- 172.21.15.146:0/331764877 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ea80039b0 msgr2=0x7f1ebc13cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.792+0000 7f1ec328d640 1 --2- 172.21.15.146:0/331764877 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ea80039b0 0x7f1ebc13cc70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1eac000c00 tx=0x7f1eac001120 comp rx=0 tx=0).stop 2024-09-19T07:42:43.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.792+0000 7f1ec328d640 1 -- 172.21.15.146:0/331764877 shutdown_connections 2024-09-19T07:42:43.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.792+0000 7f1ec328d640 1 --2- 172.21.15.146:0/331764877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ebc14f900 0x7f1ebc159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.792+0000 7f1ec328d640 1 --2- 172.21.15.146:0/331764877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ebc14f480 0x7f1ebc159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.792+0000 7f1ec328d640 1 --2- 172.21.15.146:0/331764877 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ea80039b0 0x7f1ebc13cc70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.792+0000 7f1ec328d640 1 -- 172.21.15.146:0/331764877 >> 172.21.15.146:0/331764877 conn(0x7f1ebc09a9f0 msgr2=0x7f1ebc15b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:43.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.792+0000 7f1ec328d640 1 -- 172.21.15.146:0/331764877 shutdown_connections 2024-09-19T07:42:43.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.792+0000 7f1ec328d640 1 -- 172.21.15.146:0/331764877 wait complete. 2024-09-19T07:42:43.802 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new bebed515-c757-4a04-9890-1e2cfc521983 5 2024-09-19T07:42:43.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.904+0000 7fa094ab7640 1 Processor -- start 2024-09-19T07:42:43.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.904+0000 7fa094ab7640 1 -- start start 2024-09-19T07:42:43.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.905+0000 7fa094ab7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa0900ae060 0x7fa0900b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.905+0000 7fa094ab7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0900b5b50 con 0x7fa0900afeb0 2024-09-19T07:42:43.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.905+0000 7fa094ab7640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa09015e8d0 con 0x7fa09015e4f0 2024-09-19T07:42:43.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.905+0000 7fa094ab7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa0900b4720 con 0x7fa0900ae060 2024-09-19T07:42:43.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.905+0000 7fa08e59c640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa09015e4f0 0x7fa0900b1600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:59446/0 (socket says 172.21.15.146:59446) 2024-09-19T07:42:43.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.905+0000 7fa08e59c640 1 -- 172.21.15.146:0/1535765563 learned_addr learned my addr 172.21.15.146:0/1535765563 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-19T07:42:43.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.905+0000 7fa085d9b640 1 --2- 172.21.15.146:0/1535765563 >> v2:172.21.15.16:3300/0 conn(0x7fa0900ae060 0x7fa0900b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.905+0000 7fa08dd9b640 1 -- 172.21.15.146:0/1535765563 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 405326141 0 0) 0x7fa09015e8d0 con 0x7fa09015e4f0 2024-09-19T07:42:43.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.905+0000 7fa08dd9b640 1 -- 172.21.15.146:0/1535765563 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa07c003520 con 0x7fa09015e4f0 2024-09-19T07:42:43.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.905+0000 7fa085d9b640 1 -- 172.21.15.146:0/1535765563 >> v1:172.21.15.146:6789/0 conn(0x7fa09015e4f0 legacy=0x7fa0900b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.905+0000 7fa085d9b640 1 -- 172.21.15.146:0/1535765563 >> v1:172.21.15.16:6789/0 conn(0x7fa0900afeb0 legacy=0x7fa09006bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.905+0000 7fa085d9b640 1 -- 172.21.15.146:0/1535765563 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0900b4440 con 0x7fa0900ae060 2024-09-19T07:42:43.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.906+0000 7fa085d9b640 1 --2- 172.21.15.146:0/1535765563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0900ae060 0x7fa0900b2720 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa09015eaa0 tx=0x7fa07802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd901157048757c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:43.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.906+0000 7fa08dd9b640 1 -- 172.21.15.146:0/1535765563 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078037070 con 0x7fa0900ae060 2024-09-19T07:42:43.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.906+0000 7fa08dd9b640 1 -- 172.21.15.146:0/1535765563 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa078032440 con 0x7fa0900ae060 2024-09-19T07:42:43.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.906+0000 7fa08dd9b640 1 -- 172.21.15.146:0/1535765563 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0780327f0 con 0x7fa0900ae060 2024-09-19T07:42:43.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.907+0000 7fa094ab7640 1 -- 172.21.15.146:0/1535765563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0900ae060 msgr2=0x7fa0900b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.907+0000 7fa094ab7640 1 --2- 172.21.15.146:0/1535765563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0900ae060 0x7fa0900b2720 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa09015eaa0 tx=0x7fa07802a180 comp rx=0 tx=0).stop 2024-09-19T07:42:43.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.907+0000 7fa094ab7640 1 -- 172.21.15.146:0/1535765563 shutdown_connections 2024-09-19T07:42:43.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.907+0000 7fa094ab7640 1 --2- 172.21.15.146:0/1535765563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0900ae060 0x7fa0900b2720 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.907+0000 7fa094ab7640 1 -- 172.21.15.146:0/1535765563 >> 172.21.15.146:0/1535765563 conn(0x7fa09009a9f0 msgr2=0x7fa0900b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:43.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.907+0000 7fa094ab7640 1 -- 172.21.15.146:0/1535765563 shutdown_connections 2024-09-19T07:42:43.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.907+0000 7fa094ab7640 1 -- 172.21.15.146:0/1535765563 wait complete. 2024-09-19T07:42:43.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.908+0000 7fa094ab7640 1 Processor -- start 2024-09-19T07:42:43.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.908+0000 7fa094ab7640 1 -- start start 2024-09-19T07:42:43.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.909+0000 7fa094ab7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0900ae060 0x7fa09009e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.909+0000 7fa094ab7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0900afeb0 0x7fa09015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.909+0000 7fa094ab7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa09015e4f0 0x7fa0901f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:43.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.909+0000 7fa094ab7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0900b02e0 con 0x7fa09015e4f0 2024-09-19T07:42:43.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.909+0000 7fa094ab7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa0900b60b0 con 0x7fa0900afeb0 2024-09-19T07:42:43.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.909+0000 7fa094ab7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa0900b4080 con 0x7fa0900ae060 2024-09-19T07:42:43.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.909+0000 7fa085d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0900afeb0 0x7fa09015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.909+0000 7fa085d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0900afeb0 0x7fa09015c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:58074/0 (socket says 172.21.15.146:58074) 2024-09-19T07:42:43.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.909+0000 7fa085d9b640 1 -- 172.21.15.146:0/3680686080 learned_addr learned my addr 172.21.15.146:0/3680686080 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-19T07:42:43.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.909+0000 7fa08e59c640 1 --2- 172.21.15.146:0/3680686080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0900ae060 0x7fa09009e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.909+0000 7fa08ed9d640 1 --2- 172.21.15.146:0/3680686080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa09015e4f0 0x7fa0901f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:43.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.909+0000 7fa08e59c640 1 -- 172.21.15.146:0/3680686080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0900afeb0 msgr2=0x7fa09015c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.909+0000 7fa08e59c640 1 --2- 172.21.15.146:0/3680686080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0900afeb0 0x7fa09015c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.909+0000 7fa08e59c640 1 -- 172.21.15.146:0/3680686080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa09015e4f0 msgr2=0x7fa0901f49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.909+0000 7fa08e59c640 1 --2- 172.21.15.146:0/3680686080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa09015e4f0 0x7fa0901f49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.909+0000 7fa08e59c640 1 -- 172.21.15.146:0/3680686080 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09015cde0 con 0x7fa0900ae060 2024-09-19T07:42:43.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.909+0000 7fa085d9b640 1 --2- 172.21.15.146:0/3680686080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0900afeb0 0x7fa09015c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:43.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.909+0000 7fa08ed9d640 1 --2- 172.21.15.146:0/3680686080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa09015e4f0 0x7fa0901f49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:43.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.910+0000 7fa08e59c640 1 --2- 172.21.15.146:0/3680686080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0900ae060 0x7fa09009e210 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa08000c3c0 tx=0x7fa080001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:43.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.910+0000 7fa0877fe640 1 -- 172.21.15.146:0/3680686080 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080017070 con 0x7fa0900ae060 2024-09-19T07:42:43.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.910+0000 7fa0877fe640 1 -- 172.21.15.146:0/3680686080 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa080005ce0 con 0x7fa0900ae060 2024-09-19T07:42:43.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.910+0000 7fa0877fe640 1 -- 172.21.15.146:0/3680686080 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0800072d0 con 0x7fa0900ae060 2024-09-19T07:42:43.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.910+0000 7fa094ab7640 1 -- 172.21.15.146:0/3680686080 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa09009ef60 con 0x7fa0900ae060 2024-09-19T07:42:43.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.910+0000 7fa094ab7640 1 -- 172.21.15.146:0/3680686080 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa09009ec80 con 0x7fa0900ae060 2024-09-19T07:42:43.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.910+0000 7fa0877fe640 1 -- 172.21.15.146:0/3680686080 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa08001a070 con 0x7fa0900ae060 2024-09-19T07:42:43.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.910+0000 7fa0877fe640 1 -- 172.21.15.146:0/3680686080 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fa08001cd50 con 0x7fa0900ae060 2024-09-19T07:42:43.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.911+0000 7fa094ab7640 1 -- 172.21.15.146:0/3680686080 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0900b0760 con 0x7fa0900ae060 2024-09-19T07:42:43.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.913+0000 7fa0877fe640 1 -- 172.21.15.146:0/3680686080 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fa080007570 con 0x7fa0900ae060 2024-09-19T07:42:43.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.982+0000 7fa094ab7640 1 -- 172.21.15.146:0/3680686080 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bebed515-c757-4a04-9890-1e2cfc521983", "id": 5} v 0) -- 0x7fa0900afa20 con 0x7fa0900ae060 2024-09-19T07:42:43.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.986+0000 7fa0877fe640 1 -- 172.21.15.146:0/3680686080 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bebed515-c757-4a04-9890-1e2cfc521983", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fa08001c440 con 0x7fa0900ae060 2024-09-19T07:42:43.986 INFO:teuthology.orchestra.run.smithi146.stdout:5 2024-09-19T07:42:43.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.986+0000 7fa094ab7640 1 -- 172.21.15.146:0/3680686080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0900ae060 msgr2=0x7fa09009e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:43.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.986+0000 7fa094ab7640 1 --2- 172.21.15.146:0/3680686080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0900ae060 0x7fa09009e210 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa08000c3c0 tx=0x7fa080001e70 comp rx=0 tx=0).stop 2024-09-19T07:42:43.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.986+0000 7fa094ab7640 1 -- 172.21.15.146:0/3680686080 shutdown_connections 2024-09-19T07:42:43.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.986+0000 7fa094ab7640 1 --2- 172.21.15.146:0/3680686080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa09015e4f0 0x7fa0901f49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.986+0000 7fa094ab7640 1 --2- 172.21.15.146:0/3680686080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0900afeb0 0x7fa09015c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.986+0000 7fa094ab7640 1 --2- 172.21.15.146:0/3680686080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0900ae060 0x7fa09009e210 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:43.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.987+0000 7fa094ab7640 1 -- 172.21.15.146:0/3680686080 >> 172.21.15.146:0/3680686080 conn(0x7fa09009a9f0 msgr2=0x7fa09006ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:43.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.987+0000 7fa094ab7640 1 -- 172.21.15.146:0/3680686080 shutdown_connections 2024-09-19T07:42:43.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:43.987+0000 7fa094ab7640 1 -- 172.21.15.146:0/3680686080 wait complete. 2024-09-19T07:42:43.997 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 676c8305-aaee-45b7-9c3a-ad1a35785d17 6 2024-09-19T07:42:44.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.096+0000 7f047790a640 1 Processor -- start 2024-09-19T07:42:44.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.096+0000 7f047790a640 1 -- start start 2024-09-19T07:42:44.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.097+0000 7f047790a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0470159de0 0x7f0470070950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:44.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.097+0000 7f047790a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f047015d420 0x7f04700b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:44.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.097+0000 7f047790a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f047006e940 con 0x7f04700b62e0 2024-09-19T07:42:44.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.097+0000 7f047790a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f04700b6730 con 0x7f0470159de0 2024-09-19T07:42:44.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.097+0000 7f047790a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f047006e6f0 con 0x7f047015d420 2024-09-19T07:42:44.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.097+0000 7f04756a6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0470159de0 0x7f0470070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:44.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.097+0000 7f0474ea5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f047015d420 0x7f04700b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:44.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.097+0000 7f04756a6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0470159de0 0x7f0470070950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:58090/0 (socket says 172.21.15.146:58090) 2024-09-19T07:42:44.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.097+0000 7f04756a6640 1 -- 172.21.15.146:0/2918002413 learned_addr learned my addr 172.21.15.146:0/2918002413 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-19T07:42:44.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.097+0000 7f0474ea5640 1 -- 172.21.15.146:0/2918002413 >> v2:172.21.15.146:3300/0 conn(0x7f0470159de0 msgr2=0x7f0470070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:44.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.097+0000 7f0474ea5640 1 --2- 172.21.15.146:0/2918002413 >> v2:172.21.15.146:3300/0 conn(0x7f0470159de0 0x7f0470070950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.097+0000 7f0474ea5640 1 -- 172.21.15.146:0/2918002413 >> v1:172.21.15.146:6789/0 conn(0x7f04700b62e0 legacy=0x7f047015fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:44.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.097+0000 7f0474ea5640 1 -- 172.21.15.146:0/2918002413 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047006ec20 con 0x7f047015d420 2024-09-19T07:42:44.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.097+0000 7f04756a6640 1 --2- 172.21.15.146:0/2918002413 >> v2:172.21.15.146:3300/0 conn(0x7f0470159de0 0x7f0470070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:44.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.098+0000 7f0474ea5640 1 --2- 172.21.15.146:0/2918002413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f047015d420 0x7f04700b5da0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f04600097a0 tx=0x7f046002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=994e3d1259e087a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:44.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.098+0000 7f0467fff640 1 -- 172.21.15.146:0/2918002413 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460037070 con 0x7f047015d420 2024-09-19T07:42:44.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.098+0000 7f0467fff640 1 -- 172.21.15.146:0/2918002413 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0460032440 con 0x7f047015d420 2024-09-19T07:42:44.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.098+0000 7f0467fff640 1 -- 172.21.15.146:0/2918002413 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460032810 con 0x7f047015d420 2024-09-19T07:42:44.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.099+0000 7f047790a640 1 -- 172.21.15.146:0/2918002413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f047015d420 msgr2=0x7f04700b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:44.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.099+0000 7f047790a640 1 --2- 172.21.15.146:0/2918002413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f047015d420 0x7f04700b5da0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f04600097a0 tx=0x7f046002a180 comp rx=0 tx=0).stop 2024-09-19T07:42:44.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.099+0000 7f047790a640 1 -- 172.21.15.146:0/2918002413 shutdown_connections 2024-09-19T07:42:44.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.099+0000 7f047790a640 1 --2- 172.21.15.146:0/2918002413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f047015d420 0x7f04700b5da0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.099+0000 7f047790a640 1 --2- 172.21.15.146:0/2918002413 >> v2:172.21.15.146:3300/0 conn(0x7f0470159de0 0x7f0470070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.099+0000 7f047790a640 1 -- 172.21.15.146:0/2918002413 >> 172.21.15.146:0/2918002413 conn(0x7f0470073010 msgr2=0x7f047006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:44.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.099+0000 7f047790a640 1 -- 172.21.15.146:0/2918002413 shutdown_connections 2024-09-19T07:42:44.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.099+0000 7f047790a640 1 -- 172.21.15.146:0/2918002413 wait complete. 2024-09-19T07:42:44.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.100+0000 7f047790a640 1 Processor -- start 2024-09-19T07:42:44.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.100+0000 7f047790a640 1 -- start start 2024-09-19T07:42:44.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.101+0000 7f047790a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04700b62e0 0x7f04701546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:44.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.101+0000 7f047790a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0470159de0 0x7f04701464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:44.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.101+0000 7f047790a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f047015d420 0x7f04701567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:44.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.101+0000 7f047790a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f047006f4c0 con 0x7f04700b62e0 2024-09-19T07:42:44.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.101+0000 7f047790a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f047006f1e0 con 0x7f0470159de0 2024-09-19T07:42:44.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.101+0000 7f047790a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0470070e90 con 0x7f047015d420 2024-09-19T07:42:44.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.101+0000 7f0475ea7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f047015d420 0x7f04701567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:44.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.101+0000 7f0474ea5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0470159de0 0x7f04701464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:44.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.101+0000 7f0475ea7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f047015d420 0x7f04701567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:50578/0 (socket says 172.21.15.146:50578) 2024-09-19T07:42:44.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.101+0000 7f0475ea7640 1 -- 172.21.15.146:0/2887388622 learned_addr learned my addr 172.21.15.146:0/2887388622 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-19T07:42:44.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.101+0000 7f04756a6640 1 --2- 172.21.15.146:0/2887388622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04700b62e0 0x7f04701546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:44.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.101+0000 7f0474ea5640 1 -- 172.21.15.146:0/2887388622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f047015d420 msgr2=0x7f04701567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:44.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.101+0000 7f0474ea5640 1 --2- 172.21.15.146:0/2887388622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f047015d420 0x7f04701567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.101+0000 7f0474ea5640 1 -- 172.21.15.146:0/2887388622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04700b62e0 msgr2=0x7f04701546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:44.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.101+0000 7f0474ea5640 1 --2- 172.21.15.146:0/2887388622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04700b62e0 0x7f04701546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.101+0000 7f0474ea5640 1 -- 172.21.15.146:0/2887388622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0470155d20 con 0x7f0470159de0 2024-09-19T07:42:44.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.101+0000 7f0475ea7640 1 --2- 172.21.15.146:0/2887388622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f047015d420 0x7f04701567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:44.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.101+0000 7f04756a6640 1 --2- 172.21.15.146:0/2887388622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04700b62e0 0x7f04701546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:42:44.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.101+0000 7f0474ea5640 1 --2- 172.21.15.146:0/2887388622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0470159de0 0x7f04701464b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f047006e870 tx=0x7f04600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:44.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.102+0000 7f0465ffb640 1 -- 172.21.15.146:0/2887388622 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460041020 con 0x7f0470159de0 2024-09-19T07:42:44.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.102+0000 7f0465ffb640 1 -- 172.21.15.146:0/2887388622 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0460030700 con 0x7f0470159de0 2024-09-19T07:42:44.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.102+0000 7f047790a640 1 -- 172.21.15.146:0/2887388622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0470154c80 con 0x7f0470159de0 2024-09-19T07:42:44.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.102+0000 7f0465ffb640 1 -- 172.21.15.146:0/2887388622 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460037040 con 0x7f0470159de0 2024-09-19T07:42:44.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.102+0000 7f047790a640 1 -- 172.21.15.146:0/2887388622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0470155240 con 0x7f0470159de0 2024-09-19T07:42:44.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.102+0000 7f0465ffb640 1 -- 172.21.15.146:0/2887388622 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0460044020 con 0x7f0470159de0 2024-09-19T07:42:44.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.102+0000 7f0465ffb640 1 -- 172.21.15.146:0/2887388622 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f04600067a0 con 0x7f0470159de0 2024-09-19T07:42:44.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.103+0000 7f047790a640 1 -- 172.21.15.146:0/2887388622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0470070950 con 0x7f0470159de0 2024-09-19T07:42:44.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.105+0000 7f0465ffb640 1 -- 172.21.15.146:0/2887388622 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f046004b070 con 0x7f0470159de0 2024-09-19T07:42:44.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.175+0000 7f047790a640 1 -- 172.21.15.146:0/2887388622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "676c8305-aaee-45b7-9c3a-ad1a35785d17", "id": 6} v 0) -- 0x7f047006caa0 con 0x7f0470159de0 2024-09-19T07:42:44.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.178+0000 7f0465ffb640 1 -- 172.21.15.146:0/2887388622 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "676c8305-aaee-45b7-9c3a-ad1a35785d17", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f0460005990 con 0x7f0470159de0 2024-09-19T07:42:44.178 INFO:teuthology.orchestra.run.smithi146.stdout:6 2024-09-19T07:42:44.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.179+0000 7f047790a640 1 -- 172.21.15.146:0/2887388622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0470159de0 msgr2=0x7f04701464b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:44.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.179+0000 7f047790a640 1 --2- 172.21.15.146:0/2887388622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0470159de0 0x7f04701464b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f047006e870 tx=0x7f04600046c0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.179+0000 7f047790a640 1 -- 172.21.15.146:0/2887388622 shutdown_connections 2024-09-19T07:42:44.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.179+0000 7f047790a640 1 --2- 172.21.15.146:0/2887388622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f047015d420 0x7f04701567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.179+0000 7f047790a640 1 --2- 172.21.15.146:0/2887388622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0470159de0 0x7f04701464b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.179+0000 7f047790a640 1 --2- 172.21.15.146:0/2887388622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04700b62e0 0x7f04701546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.179+0000 7f047790a640 1 -- 172.21.15.146:0/2887388622 >> 172.21.15.146:0/2887388622 conn(0x7f0470073010 msgr2=0x7f04700b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:44.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.179+0000 7f047790a640 1 -- 172.21.15.146:0/2887388622 shutdown_connections 2024-09-19T07:42:44.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.179+0000 7f047790a640 1 -- 172.21.15.146:0/2887388622 wait complete. 2024-09-19T07:42:44.189 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new afdf6bda-674b-4dfb-a698-ba28e38a22e6 7 2024-09-19T07:42:44.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.288+0000 7f4eb54e4640 1 Processor -- start 2024-09-19T07:42:44.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.288+0000 7f4eb54e4640 1 -- start start 2024-09-19T07:42:44.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.289+0000 7f4eb54e4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4eb015e4f0 0x7f4eb00b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:44.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.289+0000 7f4eb54e4640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4eb00afeb0 0x7f4eb006bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:44.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.289+0000 7f4eb54e4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb00b5b50 con 0x7f4eb00ae060 2024-09-19T07:42:44.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.289+0000 7f4eb54e4640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4eb00b4720 con 0x7f4eb00afeb0 2024-09-19T07:42:44.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.289+0000 7f4eb54e4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb015e8d0 con 0x7f4eb015e4f0 2024-09-19T07:42:44.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.289+0000 7f4eaeffd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4eb015e4f0 0x7f4eb00b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:44.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.289+0000 7f4eaf7fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4eb00afeb0 0x7f4eb006bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:44.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.289+0000 7f4eaeffd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4eb015e4f0 0x7f4eb00b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:50584/0 (socket says 172.21.15.146:50584) 2024-09-19T07:42:44.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.289+0000 7f4eaf7fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4eb00afeb0 0x7f4eb006bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:58104/0 (socket says 172.21.15.146:58104) 2024-09-19T07:42:44.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.289+0000 7f4eaeffd640 1 -- 172.21.15.146:0/1797701424 learned_addr learned my addr 172.21.15.146:0/1797701424 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-19T07:42:44.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.290+0000 7f4eaf7fe640 1 -- 172.21.15.146:0/1797701424 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4eb015e4f0 msgr2=0x7f4eb00b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:44.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.290+0000 7f4eaf7fe640 1 --2- 172.21.15.146:0/1797701424 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4eb015e4f0 0x7f4eb00b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.290+0000 7f4eaf7fe640 1 -- 172.21.15.146:0/1797701424 >> v1:172.21.15.16:6789/0 conn(0x7f4eb00ae060 legacy=0x7f4eb00b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:44.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.290+0000 7f4eaf7fe640 1 -- 172.21.15.146:0/1797701424 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb00b0360 con 0x7f4eb00afeb0 2024-09-19T07:42:44.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.290+0000 7f4eaeffd640 1 --2- 172.21.15.146:0/1797701424 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4eb015e4f0 0x7f4eb00b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:44.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.290+0000 7f4eaf7fe640 1 --2- 172.21.15.146:0/1797701424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4eb00afeb0 0x7f4eb006bc10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009d90 tx=0x7f4ea802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9013432400ef33bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:44.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.290+0000 7f4eadffb640 1 -- 172.21.15.146:0/1797701424 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8031600 con 0x7f4eb00afeb0 2024-09-19T07:42:44.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.291+0000 7f4eadffb640 1 -- 172.21.15.146:0/1797701424 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea80317a0 con 0x7f4eb00afeb0 2024-09-19T07:42:44.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.291+0000 7f4eadffb640 1 -- 172.21.15.146:0/1797701424 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8031b70 con 0x7f4eb00afeb0 2024-09-19T07:42:44.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.291+0000 7f4eb54e4640 1 -- 172.21.15.146:0/1797701424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4eb00afeb0 msgr2=0x7f4eb006bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:44.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.291+0000 7f4eb54e4640 1 --2- 172.21.15.146:0/1797701424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4eb00afeb0 0x7f4eb006bc10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009d90 tx=0x7f4ea802a5d0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.291+0000 7f4eb54e4640 1 -- 172.21.15.146:0/1797701424 shutdown_connections 2024-09-19T07:42:44.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.291+0000 7f4eb54e4640 1 --2- 172.21.15.146:0/1797701424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4eb00afeb0 0x7f4eb006bc10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.291+0000 7f4eb54e4640 1 --2- 172.21.15.146:0/1797701424 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4eb015e4f0 0x7f4eb00b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.291+0000 7f4eb54e4640 1 -- 172.21.15.146:0/1797701424 >> 172.21.15.146:0/1797701424 conn(0x7f4eb009a9f0 msgr2=0x7f4eb00b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:44.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.291+0000 7f4eb54e4640 1 -- 172.21.15.146:0/1797701424 shutdown_connections 2024-09-19T07:42:44.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.291+0000 7f4eb54e4640 1 -- 172.21.15.146:0/1797701424 wait complete. 2024-09-19T07:42:44.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.293+0000 7f4eb54e4640 1 Processor -- start 2024-09-19T07:42:44.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.293+0000 7f4eb54e4640 1 -- start start 2024-09-19T07:42:44.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.293+0000 7f4eb54e4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4eb00ae060 0x7f4eb009b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:44.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.293+0000 7f4eb54e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4eb00afeb0 0x7f4eb014f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:44.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.293+0000 7f4eb54e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4eb015e4f0 0x7f4eb00cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:44.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.293+0000 7f4eb54e4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb00b60b0 con 0x7f4eb00afeb0 2024-09-19T07:42:44.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.293+0000 7f4eb54e4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb00b5dd0 con 0x7f4eb015e4f0 2024-09-19T07:42:44.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.293+0000 7f4eb54e4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb00b4080 con 0x7f4eb00ae060 2024-09-19T07:42:44.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.293+0000 7f4eaeffd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4eb00ae060 0x7f4eb009b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:44.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.293+0000 7f4eaeffd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4eb00ae060 0x7f4eb009b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:50598/0 (socket says 172.21.15.146:50598) 2024-09-19T07:42:44.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.293+0000 7f4eaeffd640 1 -- 172.21.15.146:0/2487555099 learned_addr learned my addr 172.21.15.146:0/2487555099 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-19T07:42:44.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.293+0000 7f4eaf7fe640 1 --2- 172.21.15.146:0/2487555099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4eb015e4f0 0x7f4eb00cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:44.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.293+0000 7f4eae7fc640 1 --2- 172.21.15.146:0/2487555099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4eb00afeb0 0x7f4eb014f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:44.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.294+0000 7f4eaeffd640 1 -- 172.21.15.146:0/2487555099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4eb015e4f0 msgr2=0x7f4eb00cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:44.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.294+0000 7f4eaeffd640 1 --2- 172.21.15.146:0/2487555099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4eb015e4f0 0x7f4eb00cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.294+0000 7f4eaeffd640 1 -- 172.21.15.146:0/2487555099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4eb00afeb0 msgr2=0x7f4eb014f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:44.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.294+0000 7f4eaeffd640 1 --2- 172.21.15.146:0/2487555099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4eb00afeb0 0x7f4eb014f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.294+0000 7f4eaeffd640 1 -- 172.21.15.146:0/2487555099 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb009e280 con 0x7f4eb00ae060 2024-09-19T07:42:44.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.294+0000 7f4eaeffd640 1 --2- 172.21.15.146:0/2487555099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4eb00ae060 0x7f4eb009b170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000e9d0 tx=0x7f4ea000ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:44.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.294+0000 7f4ea77fe640 1 -- 172.21.15.146:0/2487555099 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0017070 con 0x7f4eb00ae060 2024-09-19T07:42:44.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.294+0000 7f4ea77fe640 1 -- 172.21.15.146:0/2487555099 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea000ca60 con 0x7f4eb00ae060 2024-09-19T07:42:44.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.294+0000 7f4ea77fe640 1 -- 172.21.15.146:0/2487555099 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea000ce10 con 0x7f4eb00ae060 2024-09-19T07:42:44.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.294+0000 7f4eb54e4640 1 -- 172.21.15.146:0/2487555099 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb009e560 con 0x7f4eb00ae060 2024-09-19T07:42:44.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.294+0000 7f4eb54e4640 1 -- 172.21.15.146:0/2487555099 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb009eb20 con 0x7f4eb00ae060 2024-09-19T07:42:44.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.294+0000 7f4ea77fe640 1 -- 172.21.15.146:0/2487555099 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4ea001a070 con 0x7f4eb00ae060 2024-09-19T07:42:44.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.294+0000 7f4ea77fe640 1 -- 172.21.15.146:0/2487555099 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f4ea001cab0 con 0x7f4eb00ae060 2024-09-19T07:42:44.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.295+0000 7f4eb54e4640 1 -- 172.21.15.146:0/2487555099 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb00b0760 con 0x7f4eb00ae060 2024-09-19T07:42:44.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.299+0000 7f4ea77fe640 1 -- 172.21.15.146:0/2487555099 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4ea001c450 con 0x7f4eb00ae060 2024-09-19T07:42:44.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.369+0000 7f4eb54e4640 1 -- 172.21.15.146:0/2487555099 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd new", "uuid": "afdf6bda-674b-4dfb-a698-ba28e38a22e6", "id": 7} v 0) -- 0x7f4eb01e6d70 con 0x7f4eb00ae060 2024-09-19T07:42:44.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.373+0000 7f4ea77fe640 1 -- 172.21.15.146:0/2487555099 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "afdf6bda-674b-4dfb-a698-ba28e38a22e6", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f4ea001c700 con 0x7f4eb00ae060 2024-09-19T07:42:44.373 INFO:teuthology.orchestra.run.smithi146.stdout:7 2024-09-19T07:42:44.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.374+0000 7f4eb54e4640 1 -- 172.21.15.146:0/2487555099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4eb00ae060 msgr2=0x7f4eb009b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:44.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.374+0000 7f4eb54e4640 1 --2- 172.21.15.146:0/2487555099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4eb00ae060 0x7f4eb009b170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000e9d0 tx=0x7f4ea000ee90 comp rx=0 tx=0).stop 2024-09-19T07:42:44.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.374+0000 7f4eb54e4640 1 -- 172.21.15.146:0/2487555099 shutdown_connections 2024-09-19T07:42:44.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.374+0000 7f4eb54e4640 1 --2- 172.21.15.146:0/2487555099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4eb015e4f0 0x7f4eb00cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.374+0000 7f4eb54e4640 1 --2- 172.21.15.146:0/2487555099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4eb00afeb0 0x7f4eb014f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.374+0000 7f4eb54e4640 1 --2- 172.21.15.146:0/2487555099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4eb00ae060 0x7f4eb009b170 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.374+0000 7f4eb54e4640 1 -- 172.21.15.146:0/2487555099 >> 172.21.15.146:0/2487555099 conn(0x7f4eb009a9f0 msgr2=0x7f4eb006ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:44.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.374+0000 7f4eb54e4640 1 -- 172.21.15.146:0/2487555099 shutdown_connections 2024-09-19T07:42:44.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:42:44.374+0000 7f4eb54e4640 1 -- 172.21.15.146:0/2487555099 wait complete. 2024-09-19T07:42:44.384 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-19T07:42:44.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-19T07:42:44.387 INFO:tasks.ceph.osd.0:Started 2024-09-19T07:42:44.387 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-19T07:42:44.387 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-19T07:42:44.389 INFO:tasks.ceph.osd.1:Started 2024-09-19T07:42:44.389 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-19T07:42:44.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-19T07:42:44.391 INFO:tasks.ceph.osd.2:Started 2024-09-19T07:42:44.391 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-19T07:42:44.391 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-19T07:42:44.394 INFO:tasks.ceph.osd.3:Started 2024-09-19T07:42:44.394 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-19T07:42:44.394 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-19T07:42:44.396 INFO:tasks.ceph.osd.4:Started 2024-09-19T07:42:44.396 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-19T07:42:44.396 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-19T07:42:44.398 INFO:tasks.ceph.osd.5:Started 2024-09-19T07:42:44.398 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-19T07:42:44.399 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-19T07:42:44.400 INFO:tasks.ceph.osd.6:Started 2024-09-19T07:42:44.401 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-19T07:42:44.401 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-19T07:42:44.403 INFO:tasks.ceph.osd.7:Started 2024-09-19T07:42:44.403 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:42:44.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.525+0000 7f0b5574b640 1 Processor -- start 2024-09-19T07:42:44.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.525+0000 7f0b5574b640 1 -- start start 2024-09-19T07:42:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.526+0000 7f0b5574b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b50145ec0 0x7f0b5015d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.526+0000 7f0b5574b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0b500b6c40 0x7f0b5006bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.526+0000 7f0b5574b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b5015dae0 con 0x7f0b5006c150 2024-09-19T07:42:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.526+0000 7f0b5574b640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f0b5015d8b0 con 0x7f0b500b6c40 2024-09-19T07:42:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.526+0000 7f0b5574b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0b5015ddc0 con 0x7f0b50145ec0 2024-09-19T07:42:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.526+0000 7f0b4f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0b5006c150 0x7f0b500d10c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34488/0 (socket says 172.21.15.16:34488) 2024-09-19T07:42:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.526+0000 7f0b4f7fe640 1 -- 172.21.15.16:0/2845097253 learned_addr learned my addr 172.21.15.16:0/2845097253 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:42:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.527+0000 7f0b4effd640 1 --2- 172.21.15.16:0/2845097253 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b50145ec0 0x7f0b5015d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.527+0000 7f0b4e7fc640 1 --2- 172.21.15.16:0/2845097253 >> v2:172.21.15.146:3300/0 conn(0x7f0b500b6c40 0x7f0b5006bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.527+0000 7f0b4dffb640 1 -- 172.21.15.16:0/2845097253 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 498156305 0 0) 0x7f0b5015dae0 con 0x7f0b5006c150 2024-09-19T07:42:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.527+0000 7f0b4dffb640 1 -- 172.21.15.16:0/2845097253 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b38003520 con 0x7f0b5006c150 2024-09-19T07:42:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.527+0000 7f0b4effd640 1 -- 172.21.15.16:0/2845097253 >> v2:172.21.15.146:3300/0 conn(0x7f0b500b6c40 msgr2=0x7f0b5006bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.527+0000 7f0b4effd640 1 --2- 172.21.15.16:0/2845097253 >> v2:172.21.15.146:3300/0 conn(0x7f0b500b6c40 0x7f0b5006bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.527+0000 7f0b4effd640 1 -- 172.21.15.16:0/2845097253 >> v1:172.21.15.16:6789/0 conn(0x7f0b5006c150 legacy=0x7f0b500d10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.527+0000 7f0b4effd640 1 -- 172.21.15.16:0/2845097253 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5015e0a0 con 0x7f0b50145ec0 2024-09-19T07:42:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.528+0000 7f0b4effd640 1 --2- 172.21.15.16:0/2845097253 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b50145ec0 0x7f0b5015d170 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0b400097a0 tx=0x7f0b4002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79b40810ce2cccff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.528+0000 7f0b4dffb640 1 -- 172.21.15.16:0/2845097253 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40037070 con 0x7f0b50145ec0 2024-09-19T07:42:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.528+0000 7f0b4dffb640 1 -- 172.21.15.16:0/2845097253 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b4002ae50 con 0x7f0b50145ec0 2024-09-19T07:42:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.528+0000 7f0b4dffb640 1 -- 172.21.15.16:0/2845097253 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40032640 con 0x7f0b50145ec0 2024-09-19T07:42:44.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.528+0000 7f0b5574b640 1 -- 172.21.15.16:0/2845097253 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b50145ec0 msgr2=0x7f0b5015d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:44.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.528+0000 7f0b5574b640 1 --2- 172.21.15.16:0/2845097253 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b50145ec0 0x7f0b5015d170 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0b400097a0 tx=0x7f0b4002a100 comp rx=0 tx=0).stop 2024-09-19T07:42:44.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.529+0000 7f0b5574b640 1 -- 172.21.15.16:0/2845097253 shutdown_connections 2024-09-19T07:42:44.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.529+0000 7f0b5574b640 1 --2- 172.21.15.16:0/2845097253 >> v2:172.21.15.146:3300/0 conn(0x7f0b500b6c40 0x7f0b5006bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.529+0000 7f0b5574b640 1 --2- 172.21.15.16:0/2845097253 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b50145ec0 0x7f0b5015d170 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.529+0000 7f0b5574b640 1 -- 172.21.15.16:0/2845097253 >> 172.21.15.16:0/2845097253 conn(0x7f0b5009a9f0 msgr2=0x7f0b500b6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.529+0000 7f0b5574b640 1 -- 172.21.15.16:0/2845097253 shutdown_connections 2024-09-19T07:42:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.529+0000 7f0b5574b640 1 -- 172.21.15.16:0/2845097253 wait complete. 2024-09-19T07:42:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.530+0000 7f0b5574b640 1 Processor -- start 2024-09-19T07:42:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.530+0000 7f0b5574b640 1 -- start start 2024-09-19T07:42:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.531+0000 7f0b5574b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b5006c150 0x7f0b50158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.531+0000 7f0b5574b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b500b6c40 0x7f0b5009b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.531+0000 7f0b5574b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b50145ec0 0x7f0b5009baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.531+0000 7f0b5574b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5015f8b0 con 0x7f0b50145ec0 2024-09-19T07:42:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.531+0000 7f0b5574b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0b500b7080 con 0x7f0b500b6c40 2024-09-19T07:42:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.531+0000 7f0b5574b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0b5015fa30 con 0x7f0b5006c150 2024-09-19T07:42:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.531+0000 7f0b4f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b50145ec0 0x7f0b5009baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.531+0000 7f0b4f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b50145ec0 0x7f0b5009baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41788/0 (socket says 172.21.15.16:41788) 2024-09-19T07:42:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.531+0000 7f0b4f7fe640 1 -- 172.21.15.16:0/1413587385 learned_addr learned my addr 172.21.15.16:0/1413587385 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:42:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.531+0000 7f0b4effd640 1 --2- 172.21.15.16:0/1413587385 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b5006c150 0x7f0b50158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.531+0000 7f0b4e7fc640 1 --2- 172.21.15.16:0/1413587385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b500b6c40 0x7f0b5009b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.531+0000 7f0b4f7fe640 1 -- 172.21.15.16:0/1413587385 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b5006c150 msgr2=0x7f0b50158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.531+0000 7f0b4f7fe640 1 --2- 172.21.15.16:0/1413587385 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b5006c150 0x7f0b50158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.531+0000 7f0b4f7fe640 1 -- 172.21.15.16:0/1413587385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b500b6c40 msgr2=0x7f0b5009b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.531+0000 7f0b4f7fe640 1 --2- 172.21.15.16:0/1413587385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b500b6c40 0x7f0b5009b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.531+0000 7f0b4f7fe640 1 -- 172.21.15.16:0/1413587385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5009e170 con 0x7f0b50145ec0 2024-09-19T07:42:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.531+0000 7f0b4effd640 1 --2- 172.21.15.16:0/1413587385 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b5006c150 0x7f0b50158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.531+0000 7f0b4f7fe640 1 --2- 172.21.15.16:0/1413587385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b50145ec0 0x7f0b5009baa0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800ca20 tx=0x7f0b4800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.531+0000 7f0b377fe640 1 -- 172.21.15.16:0/1413587385 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48017070 con 0x7f0b50145ec0 2024-09-19T07:42:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.532+0000 7f0b377fe640 1 -- 172.21.15.16:0/1413587385 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b4800abd0 con 0x7f0b50145ec0 2024-09-19T07:42:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.532+0000 7f0b5574b640 1 -- 172.21.15.16:0/1413587385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b5009e450 con 0x7f0b50145ec0 2024-09-19T07:42:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.532+0000 7f0b377fe640 1 -- 172.21.15.16:0/1413587385 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48012400 con 0x7f0b50145ec0 2024-09-19T07:42:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.532+0000 7f0b5574b640 1 -- 172.21.15.16:0/1413587385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b5009ea10 con 0x7f0b50145ec0 2024-09-19T07:42:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.532+0000 7f0b377fe640 1 -- 172.21.15.16:0/1413587385 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0b4801a070 con 0x7f0b50145ec0 2024-09-19T07:42:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.532+0000 7f0b377fe640 1 -- 172.21.15.16:0/1413587385 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f0b48007740 con 0x7f0b50145ec0 2024-09-19T07:42:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.533+0000 7f0b5574b640 1 -- 172.21.15.16:0/1413587385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b5006bc10 con 0x7f0b50145ec0 2024-09-19T07:42:44.536 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:44.535+0000 7feda872f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T07:42:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.535+0000 7f0b377fe640 1 -- 172.21.15.16:0/1413587385 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f0b48021070 con 0x7f0b50145ec0 2024-09-19T07:42:44.582 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:44.582+0000 7fdcf962f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T07:42:44.588 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:44.587+0000 7feda872f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T07:42:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.594+0000 7f0b5574b640 1 -- 172.21.15.16:0/1413587385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b5006faa0 con 0x7f0b50145ec0 2024-09-19T07:42:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.595+0000 7f0b377fe640 1 -- 172.21.15.16:0/1413587385 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f0b48007990 con 0x7f0b50145ec0 2024-09-19T07:42:44.595 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:42:44.596 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:42:44.371446+0000","last_up_change":"0.000000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:42:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.596+0000 7f0b5574b640 1 -- 172.21.15.16:0/1413587385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b50145ec0 msgr2=0x7f0b5009baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.596+0000 7f0b5574b640 1 --2- 172.21.15.16:0/1413587385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b50145ec0 0x7f0b5009baa0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800ca20 tx=0x7f0b4800cee0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.596+0000 7f0b5574b640 1 -- 172.21.15.16:0/1413587385 shutdown_connections 2024-09-19T07:42:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.596+0000 7f0b5574b640 1 --2- 172.21.15.16:0/1413587385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b50145ec0 0x7f0b5009baa0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.596+0000 7f0b5574b640 1 --2- 172.21.15.16:0/1413587385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b500b6c40 0x7f0b5009b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.596+0000 7f0b5574b640 1 --2- 172.21.15.16:0/1413587385 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b5006c150 0x7f0b50158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.596+0000 7f0b5574b640 1 -- 172.21.15.16:0/1413587385 >> 172.21.15.16:0/1413587385 conn(0x7f0b5009a9f0 msgr2=0x7f0b50070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.596+0000 7f0b5574b640 1 -- 172.21.15.16:0/1413587385 shutdown_connections 2024-09-19T07:42:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:42:44.596+0000 7f0b5574b640 1 -- 172.21.15.16:0/1413587385 wait complete. 2024-09-19T07:42:44.607 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-19T07:42:44.607 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-19T07:42:44.639 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:44.638+0000 7fdcf962f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T07:42:44.954 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:44.954+0000 7feda872f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T07:42:45.023 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:45.023+0000 7fdcf962f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T07:42:45.039 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:45.039+0000 7feda872f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T07:42:45.112 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:45.112+0000 7fdcf962f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T07:42:45.148 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:45.148+0000 7feda872f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:42:45.194 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:45.193+0000 7feda872f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T07:42:45.224 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:45.224+0000 7fdcf962f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:42:45.272 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:45.272+0000 7fdcf962f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T07:42:45.306 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:45.306+0000 7feda872f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T07:42:45.362 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:45.361+0000 7feda872f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T07:42:45.365 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-09-19T07:42:45.365+0000 7fc8bdeef640 -1 Falling back to public interface 2024-09-19T07:42:45.391 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:45.391+0000 7fdcf962f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T07:42:45.407 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:45.407+0000 7feda872f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T07:42:45.415 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-19T07:42:45.447 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:45.447+0000 7fdcf962f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T07:42:45.496 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:45.495+0000 7fdcf962f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T07:42:45.508 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.508+0000 7f7ee9d9f640 1 Processor -- start 2024-09-19T07:42:45.508 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.508+0000 7f7ee9d9f640 1 -- start start 2024-09-19T07:42:45.510 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.509+0000 7f7ee9d9f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ee415c140 0x7f7ee415c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:45.510 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.509+0000 7f7ee9d9f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7ee40aedb0 0x7f7ee406bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:45.510 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.509+0000 7f7ee9d9f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ee4160a60 con 0x7f7ee40b34a0 2024-09-19T07:42:45.510 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.509+0000 7f7ee9d9f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f7ee4160500 con 0x7f7ee415c140 2024-09-19T07:42:45.510 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.509+0000 7f7ee9d9f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f7ee415ca60 con 0x7f7ee40aedb0 2024-09-19T07:42:45.510 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.509+0000 7f7ee8d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ee415c140 0x7f7ee415c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:45.510 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.509+0000 7f7ee3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7ee40b34a0 0x7f7ee40ae6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34502/0 (socket says 172.21.15.16:34502) 2024-09-19T07:42:45.510 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.509+0000 7f7ee8d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ee415c140 0x7f7ee415c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41804/0 (socket says 172.21.15.16:41804) 2024-09-19T07:42:45.510 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.509+0000 7f7ee3fff640 1 -- 172.21.15.16:0/3577873760 learned_addr learned my addr 172.21.15.16:0/3577873760 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:42:45.510 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.509+0000 7f7ee959e640 1 --2- 172.21.15.16:0/3577873760 >> v2:172.21.15.146:3300/0 conn(0x7f7ee40aedb0 0x7f7ee406bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:45.510 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.509+0000 7f7ee8d9d640 1 -- 172.21.15.16:0/3577873760 >> v2:172.21.15.146:3300/0 conn(0x7f7ee40aedb0 msgr2=0x7f7ee406bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:45.511 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.509+0000 7f7ee8d9d640 1 --2- 172.21.15.16:0/3577873760 >> v2:172.21.15.146:3300/0 conn(0x7f7ee40aedb0 0x7f7ee406bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:45.511 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.509+0000 7f7ee8d9d640 1 -- 172.21.15.16:0/3577873760 >> v1:172.21.15.16:6789/0 conn(0x7f7ee40b34a0 legacy=0x7f7ee40ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:45.511 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.509+0000 7f7ee8d9d640 1 -- 172.21.15.16:0/3577873760 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee409aa70 con 0x7f7ee415c140 2024-09-19T07:42:45.511 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.509+0000 7f7ee37fe640 1 -- 172.21.15.16:0/3577873760 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3195709008 0 0) 0x7f7ee4160a60 con 0x7f7ee40b34a0 2024-09-19T07:42:45.512 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.510+0000 7f7ee8d9d640 1 --2- 172.21.15.16:0/3577873760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee415c140 0x7f7ee415c520 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4068cb0 tx=0x7f7ed802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c56c63ed88952ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:45.512 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.510+0000 7f7ee37fe640 1 -- 172.21.15.16:0/3577873760 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8037070 con 0x7f7ee415c140 2024-09-19T07:42:45.512 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.510+0000 7f7ee37fe640 1 -- 172.21.15.16:0/3577873760 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed802add0 con 0x7f7ee415c140 2024-09-19T07:42:45.512 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.510+0000 7f7ee37fe640 1 -- 172.21.15.16:0/3577873760 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8032660 con 0x7f7ee415c140 2024-09-19T07:42:45.513 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.511+0000 7f7ee9d9f640 1 -- 172.21.15.16:0/3577873760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee415c140 msgr2=0x7f7ee415c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:45.513 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.511+0000 7f7ee9d9f640 1 --2- 172.21.15.16:0/3577873760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee415c140 0x7f7ee415c520 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4068cb0 tx=0x7f7ed802a080 comp rx=0 tx=0).stop 2024-09-19T07:42:45.513 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.511+0000 7f7ee9d9f640 1 -- 172.21.15.16:0/3577873760 shutdown_connections 2024-09-19T07:42:45.513 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.511+0000 7f7ee9d9f640 1 --2- 172.21.15.16:0/3577873760 >> v2:172.21.15.146:3300/0 conn(0x7f7ee40aedb0 0x7f7ee406bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:45.513 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.511+0000 7f7ee9d9f640 1 --2- 172.21.15.16:0/3577873760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee415c140 0x7f7ee415c520 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:45.513 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.511+0000 7f7ee9d9f640 1 -- 172.21.15.16:0/3577873760 >> 172.21.15.16:0/3577873760 conn(0x7f7ee40b6d20 msgr2=0x7f7ee40b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:45.513 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.511+0000 7f7ee9d9f640 1 -- 172.21.15.16:0/3577873760 shutdown_connections 2024-09-19T07:42:45.513 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.511+0000 7f7ee9d9f640 1 -- 172.21.15.16:0/3577873760 wait complete. 2024-09-19T07:42:45.513 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.512+0000 7f7ee9d9f640 1 Processor -- start 2024-09-19T07:42:45.513 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.512+0000 7f7ee9d9f640 1 -- start start 2024-09-19T07:42:45.515 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.513+0000 7f7ee9d9f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ee40aedb0 0x7f7ee41f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:45.515 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.513+0000 7f7ee9d9f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ee40b34a0 0x7f7ee41f3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:45.516 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.513+0000 7f7ee9d9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee415c140 0x7f7ee415a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:45.516 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.513+0000 7f7ee9d9f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee40ae180 con 0x7f7ee415c140 2024-09-19T07:42:45.516 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.513+0000 7f7ee9d9f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee4160780 con 0x7f7ee40aedb0 2024-09-19T07:42:45.516 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.513+0000 7f7ee9d9f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee40af240 con 0x7f7ee40b34a0 2024-09-19T07:42:45.516 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.513+0000 7f7ee959e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee415c140 0x7f7ee415a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:45.516 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.513+0000 7f7ee959e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee415c140 0x7f7ee415a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41806/0 (socket says 172.21.15.16:41806) 2024-09-19T07:42:45.516 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.513+0000 7f7ee959e640 1 -- 172.21.15.16:0/280657056 learned_addr learned my addr 172.21.15.16:0/280657056 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:42:45.516 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.513+0000 7f7ee8d9d640 1 --2- 172.21.15.16:0/280657056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ee40aedb0 0x7f7ee41f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:45.516 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.513+0000 7f7ee3fff640 1 --2- 172.21.15.16:0/280657056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ee40b34a0 0x7f7ee41f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:45.516 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.514+0000 7f7ee959e640 1 -- 172.21.15.16:0/280657056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ee40b34a0 msgr2=0x7f7ee41f3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:45.516 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.514+0000 7f7ee959e640 1 --2- 172.21.15.16:0/280657056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ee40b34a0 0x7f7ee41f3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:45.516 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.514+0000 7f7ee959e640 1 -- 172.21.15.16:0/280657056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ee40aedb0 msgr2=0x7f7ee41f55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:45.516 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.514+0000 7f7ee959e640 1 --2- 172.21.15.16:0/280657056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ee40aedb0 0x7f7ee41f55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:45.517 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.514+0000 7f7ee959e640 1 -- 172.21.15.16:0/280657056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee41f6120 con 0x7f7ee415c140 2024-09-19T07:42:45.517 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.514+0000 7f7ee959e640 1 --2- 172.21.15.16:0/280657056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee415c140 0x7f7ee415a720 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7edc007e60 tx=0x7f7edc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:45.517 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.514+0000 7f7ee17fa640 1 -- 172.21.15.16:0/280657056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc014420 con 0x7f7ee415c140 2024-09-19T07:42:45.517 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.514+0000 7f7ee17fa640 1 -- 172.21.15.16:0/280657056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7edc019070 con 0x7f7ee415c140 2024-09-19T07:42:45.517 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.514+0000 7f7ee17fa640 1 -- 172.21.15.16:0/280657056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc014820 con 0x7f7ee415c140 2024-09-19T07:42:45.517 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.514+0000 7f7ee9d9f640 1 -- 172.21.15.16:0/280657056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee41f6400 con 0x7f7ee415c140 2024-09-19T07:42:45.517 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.514+0000 7f7ee9d9f640 1 -- 172.21.15.16:0/280657056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee41f69c0 con 0x7f7ee415c140 2024-09-19T07:42:45.517 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.514+0000 7f7ee17fa640 1 -- 172.21.15.16:0/280657056 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7edc0149c0 con 0x7f7ee415c140 2024-09-19T07:42:45.517 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.514+0000 7f7ee17fa640 1 -- 172.21.15.16:0/280657056 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7edc014cb0 con 0x7f7ee415c140 2024-09-19T07:42:45.517 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.515+0000 7f7ee9d9f640 1 -- 172.21.15.16:0/280657056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee415cc40 con 0x7f7ee415c140 2024-09-19T07:42:45.518 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.517+0000 7f7ee17fa640 1 -- 172.21.15.16:0/280657056 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7ee415cc40 con 0x7f7ee415c140 2024-09-19T07:42:45.519 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:45.519+0000 7feda872f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T07:42:45.569 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:45.569+0000 7feda872f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T07:42:45.577 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.577+0000 7f7ee9d9f640 1 -- 172.21.15.16:0/280657056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ee415e560 con 0x7f7ee415c140 2024-09-19T07:42:45.577 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.577+0000 7f7ee17fa640 1 -- 172.21.15.16:0/280657056 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7ee415e560 con 0x7f7ee415c140 2024-09-19T07:42:45.578 INFO:teuthology.misc.health.smithi016.stdout: 2024-09-19T07:42:45.578 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":10,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:42:44.371446+0000","last_up_change":"0.000000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:42:45.579 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.578+0000 7f7ee9d9f640 1 -- 172.21.15.16:0/280657056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee415c140 msgr2=0x7f7ee415a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:45.579 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.578+0000 7f7ee9d9f640 1 --2- 172.21.15.16:0/280657056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee415c140 0x7f7ee415a720 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7edc007e60 tx=0x7f7edc00e4b0 comp rx=0 tx=0).stop 2024-09-19T07:42:45.579 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.578+0000 7f7ee9d9f640 1 -- 172.21.15.16:0/280657056 shutdown_connections 2024-09-19T07:42:45.579 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.578+0000 7f7ee9d9f640 1 --2- 172.21.15.16:0/280657056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ee415c140 0x7f7ee415a720 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:45.579 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.578+0000 7f7ee9d9f640 1 --2- 172.21.15.16:0/280657056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ee40b34a0 0x7f7ee41f3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:45.579 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.578+0000 7f7ee9d9f640 1 --2- 172.21.15.16:0/280657056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ee40aedb0 0x7f7ee41f55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:45.579 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.578+0000 7f7ee9d9f640 1 -- 172.21.15.16:0/280657056 >> 172.21.15.16:0/280657056 conn(0x7f7ee40b6d20 msgr2=0x7f7ee40b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:45.579 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.578+0000 7f7ee9d9f640 1 -- 172.21.15.16:0/280657056 shutdown_connections 2024-09-19T07:42:45.579 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:45.578+0000 7f7ee9d9f640 1 -- 172.21.15.16:0/280657056 wait complete. 2024-09-19T07:42:45.588 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-19T07:42:45.606 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-09-19T07:42:45.606+0000 7f0261327640 -1 Falling back to public interface 2024-09-19T07:42:45.616 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:45.616+0000 7fdcf962f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T07:42:45.621 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:45.621+0000 7feda872f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T07:42:45.669 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:45.668+0000 7fdcf962f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T07:42:45.689 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:45.689+0000 7feda872f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T07:42:45.722 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:45.722+0000 7fdcf962f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T07:42:45.733 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:45.733+0000 7feda872f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T07:42:45.777 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:45.776+0000 7feda872f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T07:42:45.793 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:45.793+0000 7fdcf962f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T07:42:45.840 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:45.839+0000 7fdcf962f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T07:42:45.851 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-09-19T07:42:45.850+0000 7f38cb219640 -1 Falling back to public interface 2024-09-19T07:42:45.855 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-09-19T07:42:45.855+0000 7fe18982a640 -1 Falling back to public interface 2024-09-19T07:42:45.862 INFO:tasks.ceph.osd.6.smithi146.stderr:2024-09-19T07:42:45.862+0000 7f40dd224640 -1 Falling back to public interface 2024-09-19T07:42:45.868 INFO:tasks.ceph.osd.4.smithi146.stderr:2024-09-19T07:42:45.868+0000 7ff141fc3640 -1 Falling back to public interface 2024-09-19T07:42:45.870 INFO:tasks.ceph.osd.5.smithi146.stderr:2024-09-19T07:42:45.870+0000 7fdde6734640 -1 Falling back to public interface 2024-09-19T07:42:45.890 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:45.889+0000 7fdcf962f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T07:42:46.071 INFO:tasks.ceph.osd.7.smithi146.stderr:2024-09-19T07:42:46.070+0000 7f33127c5640 -1 Falling back to public interface 2024-09-19T07:42:46.441 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:46.441+0000 7feda872f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T07:42:46.581 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-09-19T07:42:46.580+0000 7fdcf962f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T07:42:47.457 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:47.457+0000 7fed48ff5640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-19T07:42:47.457 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-09-19T07:42:47.457+0000 7fed48ff5640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-19T07:42:49.181 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-09-19T07:42:49.181+0000 7fc8bdeef640 -1 osd.2 0 log_to_monitors true 2024-09-19T07:42:49.664 INFO:tasks.ceph.osd.4.smithi146.stderr:2024-09-19T07:42:49.664+0000 7ff141fc3640 -1 osd.4 0 log_to_monitors true 2024-09-19T07:42:49.687 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-09-19T07:42:49.686+0000 7fe18982a640 -1 osd.1 0 log_to_monitors true 2024-09-19T07:42:49.899 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-09-19T07:42:49.899+0000 7f38cb219640 -1 osd.3 0 log_to_monitors true 2024-09-19T07:42:49.944 INFO:tasks.ceph.osd.6.smithi146.stderr:2024-09-19T07:42:49.944+0000 7f40dd224640 -1 osd.6 0 log_to_monitors true 2024-09-19T07:42:49.949 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-09-19T07:42:49.949+0000 7f0261327640 -1 osd.0 0 log_to_monitors true 2024-09-19T07:42:49.973 INFO:tasks.ceph.osd.5.smithi146.stderr:2024-09-19T07:42:49.973+0000 7fdde6734640 -1 osd.5 0 log_to_monitors true 2024-09-19T07:42:50.071 INFO:tasks.ceph.osd.7.smithi146.stderr:2024-09-19T07:42:50.071+0000 7f33127c5640 -1 osd.7 0 log_to_monitors true 2024-09-19T07:42:51.074 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-09-19T07:42:51.074+0000 7fc8b9e7e640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T07:42:51.085 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-09-19T07:42:51.085+0000 7fc8b1c9e640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:42:52.395 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-19T07:42:52.543 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.542+0000 7f0e2a531640 1 Processor -- start 2024-09-19T07:42:52.543 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.542+0000 7f0e2a531640 1 -- start start 2024-09-19T07:42:52.544 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.544+0000 7f0e2a531640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e240b4210 0x7f0e240b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:52.544 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.544+0000 7f0e2a531640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0e241645c0 0x7f0e2406bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:52.544 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.544+0000 7f0e2a531640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e2415d1b0 con 0x7f0e2415cdd0 2024-09-19T07:42:52.544 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.544+0000 7f0e2a531640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f0e240b5c70 con 0x7f0e241645c0 2024-09-19T07:42:52.545 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.544+0000 7f0e2a531640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0e240b2d60 con 0x7f0e240b4210 2024-09-19T07:42:52.545 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.544+0000 7f0e28ace640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0e241645c0 0x7f0e2406bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:52.545 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.544+0000 7f0e28ace640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0e241645c0 0x7f0e2406bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:51812/0 (socket says 172.21.15.16:51812) 2024-09-19T07:42:52.545 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.544+0000 7f0e28ace640 1 -- 172.21.15.16:0/3944770588 learned_addr learned my addr 172.21.15.16:0/3944770588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:42:52.545 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.544+0000 7f0e23fff640 1 --1- 172.21.15.16:0/3944770588 >> v1:172.21.15.146:6789/0 conn(0x7f0e2415cdd0 0x7f0e24163eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:44350/0 (socket says 172.21.15.16:44350) 2024-09-19T07:42:52.545 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.544+0000 7f0e237fe640 1 --2- 172.21.15.16:0/3944770588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e240b4210 0x7f0e240b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:52.545 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.544+0000 7f0e237fe640 1 -- 172.21.15.16:0/3944770588 >> v2:172.21.15.146:3300/0 conn(0x7f0e241645c0 msgr2=0x7f0e2406bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:52.545 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.544+0000 7f0e237fe640 1 --2- 172.21.15.16:0/3944770588 >> v2:172.21.15.146:3300/0 conn(0x7f0e241645c0 0x7f0e2406bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:52.546 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.544+0000 7f0e237fe640 1 -- 172.21.15.16:0/3944770588 >> v1:172.21.15.146:6789/0 conn(0x7f0e2415cdd0 legacy=0x7f0e24163eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:52.546 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.544+0000 7f0e237fe640 1 -- 172.21.15.16:0/3944770588 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e24163aa0 con 0x7f0e240b4210 2024-09-19T07:42:52.546 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.544+0000 7f0e28ace640 1 --2- 172.21.15.16:0/3944770588 >> v2:172.21.15.146:3300/0 conn(0x7f0e241645c0 0x7f0e2406bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:52.546 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.545+0000 7f0e237fe640 1 --2- 172.21.15.16:0/3944770588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e240b4210 0x7f0e240b60e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c0097a0 tx=0x7f0e0c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df5a5e1f4e4cc933 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:52.621 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-09-19T07:42:52.621+0000 7f38c731a640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T07:42:52.634 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-09-19T07:42:52.634+0000 7f38be926640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:42:52.643 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-09-19T07:42:52.643+0000 7fe1857b9640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T07:42:52.645 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-09-19T07:42:52.645+0000 7f025d2b6640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T07:42:52.652 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-09-19T07:42:52.652+0000 7fe17d5d9640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:42:52.748 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-09-19T07:42:52.748+0000 7f02550d6640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:42:52.823 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.822+0000 7f0e22ffd640 1 -- 172.21.15.16:0/3944770588 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c037070 con 0x7f0e240b4210 2024-09-19T07:42:52.823 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.822+0000 7f0e22ffd640 1 -- 172.21.15.16:0/3944770588 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e0c032440 con 0x7f0e240b4210 2024-09-19T07:42:52.823 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.822+0000 7f0e22ffd640 1 -- 172.21.15.16:0/3944770588 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c032810 con 0x7f0e240b4210 2024-09-19T07:42:52.824 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.823+0000 7f0e2a531640 1 -- 172.21.15.16:0/3944770588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e240b4210 msgr2=0x7f0e240b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:52.824 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.823+0000 7f0e2a531640 1 --2- 172.21.15.16:0/3944770588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e240b4210 0x7f0e240b60e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c0097a0 tx=0x7f0e0c02a180 comp rx=0 tx=0).stop 2024-09-19T07:42:52.824 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.823+0000 7f0e2a531640 1 -- 172.21.15.16:0/3944770588 shutdown_connections 2024-09-19T07:42:52.824 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.823+0000 7f0e2a531640 1 --2- 172.21.15.16:0/3944770588 >> v2:172.21.15.146:3300/0 conn(0x7f0e241645c0 0x7f0e2406bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:52.825 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.823+0000 7f0e2a531640 1 --2- 172.21.15.16:0/3944770588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e240b4210 0x7f0e240b60e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:52.825 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.823+0000 7f0e2a531640 1 -- 172.21.15.16:0/3944770588 >> 172.21.15.16:0/3944770588 conn(0x7f0e240b6730 msgr2=0x7f0e240b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:52.825 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.823+0000 7f0e2a531640 1 -- 172.21.15.16:0/3944770588 shutdown_connections 2024-09-19T07:42:52.825 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.823+0000 7f0e2a531640 1 -- 172.21.15.16:0/3944770588 wait complete. 2024-09-19T07:42:52.825 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.825+0000 7f0e2a531640 1 Processor -- start 2024-09-19T07:42:52.825 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.825+0000 7f0e2a531640 1 -- start start 2024-09-19T07:42:52.827 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.826+0000 7f0e2a531640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e240b4210 0x7f0e24152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:52.828 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.826+0000 7f0e2a531640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e2415cdd0 0x7f0e2414e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:52.828 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.826+0000 7f0e2a531640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e241645c0 0x7f0e2409e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:52.828 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.826+0000 7f0e2a531640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e24163720 con 0x7f0e240b4210 2024-09-19T07:42:52.828 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.826+0000 7f0e2a531640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0e240b2a80 con 0x7f0e2415cdd0 2024-09-19T07:42:52.828 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.826+0000 7f0e2a531640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0e24164a30 con 0x7f0e241645c0 2024-09-19T07:42:52.828 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.826+0000 7f0e23fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e240b4210 0x7f0e24152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:52.828 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.826+0000 7f0e23fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e240b4210 0x7f0e24152e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35638/0 (socket says 172.21.15.16:35638) 2024-09-19T07:42:52.828 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.826+0000 7f0e23fff640 1 -- 172.21.15.16:0/1426289312 learned_addr learned my addr 172.21.15.16:0/1426289312 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:42:52.829 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.826+0000 7f0e28ace640 1 --2- 172.21.15.16:0/1426289312 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e241645c0 0x7f0e2409e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:52.829 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.826+0000 7f0e237fe640 1 --2- 172.21.15.16:0/1426289312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e2415cdd0 0x7f0e2414e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:52.829 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.826+0000 7f0e23fff640 1 -- 172.21.15.16:0/1426289312 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e241645c0 msgr2=0x7f0e2409e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:52.829 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.826+0000 7f0e23fff640 1 --2- 172.21.15.16:0/1426289312 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e241645c0 0x7f0e2409e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:52.829 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.826+0000 7f0e23fff640 1 -- 172.21.15.16:0/1426289312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e2415cdd0 msgr2=0x7f0e2414e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:52.829 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.826+0000 7f0e23fff640 1 --2- 172.21.15.16:0/1426289312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e2415cdd0 0x7f0e2414e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:52.829 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.826+0000 7f0e23fff640 1 -- 172.21.15.16:0/1426289312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2409bd70 con 0x7f0e240b4210 2024-09-19T07:42:52.829 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.827+0000 7f0e23fff640 1 --2- 172.21.15.16:0/1426289312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e240b4210 0x7f0e24152e80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00c7b0 tx=0x7f0e1c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:52.829 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.827+0000 7f0e20ff9640 1 -- 172.21.15.16:0/1426289312 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c017070 con 0x7f0e240b4210 2024-09-19T07:42:52.829 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.827+0000 7f0e20ff9640 1 -- 172.21.15.16:0/1426289312 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e1c00a640 con 0x7f0e240b4210 2024-09-19T07:42:52.830 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.827+0000 7f0e20ff9640 1 -- 172.21.15.16:0/1426289312 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c00aa70 con 0x7f0e240b4210 2024-09-19T07:42:52.830 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.827+0000 7f0e2a531640 1 -- 172.21.15.16:0/1426289312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e241536b0 con 0x7f0e240b4210 2024-09-19T07:42:52.830 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.827+0000 7f0e2a531640 1 -- 172.21.15.16:0/1426289312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e241533c0 con 0x7f0e240b4210 2024-09-19T07:42:52.831 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.828+0000 7f0e2a531640 1 -- 172.21.15.16:0/1426289312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e24153990 con 0x7f0e240b4210 2024-09-19T07:42:52.831 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.828+0000 7f0e20ff9640 1 -- 172.21.15.16:0/1426289312 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) ==== 90790+0+0 (secure 0 0 0) 0x7f0e1c01a070 con 0x7f0e240b4210 2024-09-19T07:42:52.831 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.829+0000 7f0e20ff9640 1 --2- 172.21.15.16:0/1426289312 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0e0806c220 0x7f0e0806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:52.831 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.829+0000 7f0e20ff9640 1 -- 172.21.15.16:0/1426289312 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2511+0+0 (secure 0 0 0) 0x7f0e1c089c50 con 0x7f0e240b4210 2024-09-19T07:42:52.831 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.829+0000 7f0e237fe640 1 --2- 172.21.15.16:0/1426289312 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0e0806c220 0x7f0e0806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:52.831 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.829+0000 7f0e237fe640 1 --2- 172.21.15.16:0/1426289312 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0e0806c220 0x7f0e0806e6e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c034000 tx=0x7f0e0c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:52.833 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:52.833+0000 7f0e20ff9640 1 -- 172.21.15.16:0/1426289312 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f0e24153990 con 0x7f0e240b4210 2024-09-19T07:42:53.012 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:53.011+0000 7f0e2a531640 1 -- 172.21.15.16:0/1426289312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e240b60e0 con 0x7f0e240b4210 2024-09-19T07:42:53.012 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:53.012+0000 7f0e20ff9640 1 -- 172.21.15.16:0/1426289312 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+8526 (secure 0 0 0) 0x7f0e1c058ac0 con 0x7f0e240b4210 2024-09-19T07:42:53.013 INFO:teuthology.misc.health.smithi016.stdout: 2024-09-19T07:42:53.013 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":14,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:42:52.760455+0000","last_up_change":"2024-09-19T07:42:52.760455+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:42:53.015 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:53.014+0000 7f0e2a531640 1 -- 172.21.15.16:0/1426289312 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0e0806c220 msgr2=0x7f0e0806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:53.015 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:53.014+0000 7f0e2a531640 1 --2- 172.21.15.16:0/1426289312 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0e0806c220 0x7f0e0806e6e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c034000 tx=0x7f0e0c02e000 comp rx=0 tx=0).stop 2024-09-19T07:42:53.015 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:53.014+0000 7f0e2a531640 1 -- 172.21.15.16:0/1426289312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e240b4210 msgr2=0x7f0e24152e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:53.015 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:53.014+0000 7f0e2a531640 1 --2- 172.21.15.16:0/1426289312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e240b4210 0x7f0e24152e80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00c7b0 tx=0x7f0e1c00cc70 comp rx=0 tx=0).stop 2024-09-19T07:42:53.015 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:53.014+0000 7f0e2a531640 1 -- 172.21.15.16:0/1426289312 shutdown_connections 2024-09-19T07:42:53.015 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:53.014+0000 7f0e2a531640 1 --2- 172.21.15.16:0/1426289312 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0e0806c220 0x7f0e0806e6e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:53.015 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:53.014+0000 7f0e2a531640 1 --2- 172.21.15.16:0/1426289312 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e241645c0 0x7f0e2409e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:53.015 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:53.014+0000 7f0e2a531640 1 --2- 172.21.15.16:0/1426289312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e2415cdd0 0x7f0e2414e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:53.015 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:53.014+0000 7f0e2a531640 1 --2- 172.21.15.16:0/1426289312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e240b4210 0x7f0e24152e80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:53.015 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:53.014+0000 7f0e2a531640 1 -- 172.21.15.16:0/1426289312 >> 172.21.15.16:0/1426289312 conn(0x7f0e240b6730 msgr2=0x7f0e24150470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:53.016 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:53.014+0000 7f0e2a531640 1 -- 172.21.15.16:0/1426289312 shutdown_connections 2024-09-19T07:42:53.016 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:53.014+0000 7f0e2a531640 1 -- 172.21.15.16:0/1426289312 wait complete. 2024-09-19T07:42:53.024 DEBUG:teuthology.misc:3 of 8 OSDs are up 2024-09-19T07:42:53.463 INFO:tasks.ceph.osd.4.smithi146.stderr:2024-09-19T07:42:53.463+0000 7ff13df52640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T07:42:53.483 INFO:tasks.ceph.osd.4.smithi146.stderr:2024-09-19T07:42:53.483+0000 7ff135d72640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:42:53.585 INFO:tasks.ceph.osd.5.smithi146.stderr:2024-09-19T07:42:53.585+0000 7fdde26c3640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T07:42:53.604 INFO:tasks.ceph.osd.5.smithi146.stderr:2024-09-19T07:42:53.604+0000 7fddda4e3640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:42:53.632 INFO:tasks.ceph.osd.7.smithi146.stderr:2024-09-19T07:42:53.632+0000 7f330e754640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T07:42:53.638 INFO:tasks.ceph.osd.6.smithi146.stderr:2024-09-19T07:42:53.638+0000 7f40d8c0f640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T07:42:53.646 INFO:tasks.ceph.osd.7.smithi146.stderr:2024-09-19T07:42:53.646+0000 7f3306574640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:42:53.652 INFO:tasks.ceph.osd.6.smithi146.stderr:2024-09-19T07:42:53.652+0000 7f40d0a2f640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:42:59.831 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-19T07:42:59.936 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.936+0000 7f52c4c42640 1 Processor -- start 2024-09-19T07:42:59.937 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.936+0000 7f52c4c42640 1 -- start start 2024-09-19T07:42:59.938 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.937+0000 7f52c4c42640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52c015e1f0 0x7f52c00af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.938 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.937+0000 7f52c4c42640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f52c00b6380 0x7f52c00afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.938 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.937+0000 7f52c4c42640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52c00b4eb0 con 0x7f52c006f030 2024-09-19T07:42:59.938 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.937+0000 7f52c4c42640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f52c015e5d0 con 0x7f52c00b6380 2024-09-19T07:42:59.939 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.937+0000 7f52c4c42640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f52c006f5a0 con 0x7f52c015e1f0 2024-09-19T07:42:59.939 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.937+0000 7f52be59c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52c015e1f0 0x7f52c00af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.939 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.937+0000 7f52be59c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52c015e1f0 0x7f52c00af630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:44484/0 (socket says 172.21.15.16:44484) 2024-09-19T07:42:59.939 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.937+0000 7f52bed9d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f52c00b6380 0x7f52c00afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.939 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.937+0000 7f52be59c640 1 -- 172.21.15.16:0/881417932 learned_addr learned my addr 172.21.15.16:0/881417932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:42:59.939 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.938+0000 7f52be59c640 1 -- 172.21.15.16:0/881417932 >> v2:172.21.15.146:3300/0 conn(0x7f52c00b6380 msgr2=0x7f52c00afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:59.939 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.938+0000 7f52be59c640 1 --2- 172.21.15.16:0/881417932 >> v2:172.21.15.146:3300/0 conn(0x7f52c00b6380 0x7f52c00afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.939 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.938+0000 7f52be59c640 1 -- 172.21.15.16:0/881417932 >> v1:172.21.15.146:6789/0 conn(0x7f52c006f030 legacy=0x7f52c00ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:59.939 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.938+0000 7f52be59c640 1 -- 172.21.15.16:0/881417932 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c006f880 con 0x7f52c015e1f0 2024-09-19T07:42:59.941 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.939+0000 7f52be59c640 1 --2- 172.21.15.16:0/881417932 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52c015e1f0 0x7f52c00af630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009e50 tx=0x7f52b002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3244e1c247708096 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:59.941 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.939+0000 7f52bd59a640 1 -- 172.21.15.16:0/881417932 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0036070 con 0x7f52c015e1f0 2024-09-19T07:42:59.941 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.939+0000 7f52bd59a640 1 -- 172.21.15.16:0/881417932 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b0031600 con 0x7f52c015e1f0 2024-09-19T07:42:59.941 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.939+0000 7f52bd59a640 1 -- 172.21.15.16:0/881417932 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b00319d0 con 0x7f52c015e1f0 2024-09-19T07:42:59.941 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.939+0000 7f52c4c42640 1 -- 172.21.15.16:0/881417932 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52c015e1f0 msgr2=0x7f52c00af630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:59.941 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.939+0000 7f52c4c42640 1 --2- 172.21.15.16:0/881417932 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52c015e1f0 0x7f52c00af630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009e50 tx=0x7f52b002a680 comp rx=0 tx=0).stop 2024-09-19T07:42:59.941 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.939+0000 7f52c4c42640 1 -- 172.21.15.16:0/881417932 shutdown_connections 2024-09-19T07:42:59.941 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.940+0000 7f52c4c42640 1 --2- 172.21.15.16:0/881417932 >> v2:172.21.15.146:3300/0 conn(0x7f52c00b6380 0x7f52c00afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.941 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.940+0000 7f52c4c42640 1 --2- 172.21.15.16:0/881417932 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52c015e1f0 0x7f52c00af630 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.941 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.940+0000 7f52c4c42640 1 -- 172.21.15.16:0/881417932 >> 172.21.15.16:0/881417932 conn(0x7f52c00734c0 msgr2=0x7f52c006bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:59.941 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.940+0000 7f52c4c42640 1 -- 172.21.15.16:0/881417932 shutdown_connections 2024-09-19T07:42:59.942 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.940+0000 7f52c4c42640 1 -- 172.21.15.16:0/881417932 wait complete. 2024-09-19T07:42:59.944 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.941+0000 7f52c4c42640 1 Processor -- start 2024-09-19T07:42:59.944 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.941+0000 7f52c4c42640 1 -- start start 2024-09-19T07:42:59.944 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.942+0000 7f52c4c42640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52c006f030 0x7f52c014d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.944 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.942+0000 7f52c4c42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c00b6380 0x7f52c01605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.944 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.942+0000 7f52c4c42640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52c015e1f0 0x7f52c015f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.944 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.942+0000 7f52c4c42640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f52c00af0a0 con 0x7f52c00b6380 2024-09-19T07:42:59.944 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.942+0000 7f52c4c42640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f52c006fe40 con 0x7f52c015e1f0 2024-09-19T07:42:59.944 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.942+0000 7f52c4c42640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f52c00af220 con 0x7f52c006f030 2024-09-19T07:42:59.944 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.942+0000 7f52be59c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52c006f030 0x7f52c014d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.944 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.942+0000 7f52bdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c00b6380 0x7f52c01605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.944 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.942+0000 7f52be59c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52c006f030 0x7f52c014d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:44490/0 (socket says 172.21.15.16:44490) 2024-09-19T07:42:59.945 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.942+0000 7f52bed9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52c015e1f0 0x7f52c015f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.945 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.942+0000 7f52be59c640 1 -- 172.21.15.16:0/2069547088 learned_addr learned my addr 172.21.15.16:0/2069547088 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:42:59.945 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.942+0000 7f52be59c640 1 -- 172.21.15.16:0/2069547088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52c015e1f0 msgr2=0x7f52c015f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:59.945 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.942+0000 7f52be59c640 1 --2- 172.21.15.16:0/2069547088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52c015e1f0 0x7f52c015f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.945 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.942+0000 7f52be59c640 1 -- 172.21.15.16:0/2069547088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c00b6380 msgr2=0x7f52c01605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:59.945 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.942+0000 7f52be59c640 1 --2- 172.21.15.16:0/2069547088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c00b6380 0x7f52c01605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.945 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.942+0000 7f52be59c640 1 -- 172.21.15.16:0/2069547088 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c014d9e0 con 0x7f52c006f030 2024-09-19T07:42:59.945 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.942+0000 7f52bdd9b640 1 --2- 172.21.15.16:0/2069547088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c00b6380 0x7f52c01605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:59.945 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.943+0000 7f52be59c640 1 --2- 172.21.15.16:0/2069547088 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52c006f030 0x7f52c014d4a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f52b002ac70 tx=0x7f52b0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:59.945 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.943+0000 7f52b6ffd640 1 -- 172.21.15.16:0/2069547088 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0041020 con 0x7f52c006f030 2024-09-19T07:42:59.945 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.943+0000 7f52b6ffd640 1 -- 172.21.15.16:0/2069547088 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b0007820 con 0x7f52c006f030 2024-09-19T07:42:59.945 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.943+0000 7f52c4c42640 1 -- 172.21.15.16:0/2069547088 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c015e880 con 0x7f52c006f030 2024-09-19T07:42:59.945 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.943+0000 7f52b6ffd640 1 -- 172.21.15.16:0/2069547088 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0036040 con 0x7f52c006f030 2024-09-19T07:42:59.946 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.943+0000 7f52c4c42640 1 -- 172.21.15.16:0/2069547088 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c015ee40 con 0x7f52c006f030 2024-09-19T07:42:59.946 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.944+0000 7f52c4c42640 1 -- 172.21.15.16:0/2069547088 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c00706c0 con 0x7f52c006f030 2024-09-19T07:42:59.946 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.944+0000 7f52b6ffd640 1 -- 172.21.15.16:0/2069547088 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f52b0044020 con 0x7f52c006f030 2024-09-19T07:42:59.947 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.945+0000 7f52b6ffd640 1 --2- 172.21.15.16:0/2069547088 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f52a006c2e0 0x7f52a006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.947 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.945+0000 7f52b6ffd640 1 -- 172.21.15.16:0/2069547088 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f52b00abc40 con 0x7f52c006f030 2024-09-19T07:42:59.947 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.945+0000 7f52bdd9b640 1 --2- 172.21.15.16:0/2069547088 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f52a006c2e0 0x7f52a006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.947 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.946+0000 7f52bdd9b640 1 --2- 172.21.15.16:0/2069547088 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f52a006c2e0 0x7f52a006e7a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f52c01617e0 tx=0x7f52a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:59.951 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:42:59.951+0000 7f52b6ffd640 1 -- 172.21.15.16:0/2069547088 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f52b004b030 con 0x7f52c006f030 2024-09-19T07:43:00.126 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:43:00.125+0000 7f52c4c42640 1 -- 172.21.15.16:0/2069547088 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52c006cb40 con 0x7f52c006f030 2024-09-19T07:43:00.127 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:43:00.127+0000 7f52b6ffd640 1 -- 172.21.15.16:0/2069547088 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12920 (secure 0 0 0) 0x7f52c014d9e0 con 0x7f52c006f030 2024-09-19T07:43:00.127 INFO:teuthology.misc.health.smithi016.stdout: 2024-09-19T07:43:00.127 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":18,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:42:56.948267+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:43:00.129 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:43:00.129+0000 7f52c4c42640 1 -- 172.21.15.16:0/2069547088 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f52a006c2e0 msgr2=0x7f52a006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:00.130 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:43:00.129+0000 7f52c4c42640 1 --2- 172.21.15.16:0/2069547088 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f52a006c2e0 0x7f52a006e7a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f52c01617e0 tx=0x7f52a8009040 comp rx=0 tx=0).stop 2024-09-19T07:43:00.130 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:43:00.129+0000 7f52c4c42640 1 -- 172.21.15.16:0/2069547088 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52c006f030 msgr2=0x7f52c014d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:00.130 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:43:00.129+0000 7f52c4c42640 1 --2- 172.21.15.16:0/2069547088 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52c006f030 0x7f52c014d4a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f52b002ac70 tx=0x7f52b0004a40 comp rx=0 tx=0).stop 2024-09-19T07:43:00.130 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:43:00.129+0000 7f52c4c42640 1 -- 172.21.15.16:0/2069547088 shutdown_connections 2024-09-19T07:43:00.130 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:43:00.129+0000 7f52c4c42640 1 --2- 172.21.15.16:0/2069547088 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f52a006c2e0 0x7f52a006e7a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.130 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:43:00.129+0000 7f52c4c42640 1 --2- 172.21.15.16:0/2069547088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52c015e1f0 0x7f52c015f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.130 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:43:00.129+0000 7f52c4c42640 1 --2- 172.21.15.16:0/2069547088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c00b6380 0x7f52c01605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.130 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:43:00.129+0000 7f52c4c42640 1 --2- 172.21.15.16:0/2069547088 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52c006f030 0x7f52c014d4a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.130 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:43:00.129+0000 7f52c4c42640 1 -- 172.21.15.16:0/2069547088 >> 172.21.15.16:0/2069547088 conn(0x7f52c00734c0 msgr2=0x7f52c00af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:00.130 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:43:00.129+0000 7f52c4c42640 1 -- 172.21.15.16:0/2069547088 shutdown_connections 2024-09-19T07:43:00.130 INFO:teuthology.misc.health.smithi016.stderr:2024-09-19T07:43:00.129+0000 7f52c4c42640 1 -- 172.21.15.16:0/2069547088 wait complete. 2024-09-19T07:43:00.139 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-19T07:43:00.139 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-19T07:43:00.140 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T07:43:00.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T07:43:00.142 INFO:tasks.ceph.mds.a:Started 2024-09-19T07:43:00.142 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T07:43:00.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T07:43:00.144 INFO:tasks.ceph.mds.c:Started 2024-09-19T07:43:00.144 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T07:43:00.145 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T07:43:00.147 INFO:tasks.ceph.mds.b:Started 2024-09-19T07:43:00.147 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T07:43:00.148 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T07:43:00.150 INFO:tasks.ceph.mds.d:Started 2024-09-19T07:43:00.150 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-19T07:43:00.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:43:00.219 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-19T07:43:00.220 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-19T07:43:00.232 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2024-09-19T07:43:00.233 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2024-09-19T07:43:00.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.256+0000 7fbadce65640 1 Processor -- start 2024-09-19T07:43:00.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.256+0000 7fbadce65640 1 -- start start 2024-09-19T07:43:00.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.257+0000 7fbadce65640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbad8145a20 0x7fbad80b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.257+0000 7fbadce65640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbad806c320 0x7fbad80b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.257+0000 7fbadce65640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbad806ca00 con 0x7fbad80b0460 2024-09-19T07:43:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.257+0000 7fbadce65640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fbad80b1570 con 0x7fbad806c320 2024-09-19T07:43:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.257+0000 7fbadce65640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbad80b0960 con 0x7fbad8145a20 2024-09-19T07:43:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.257+0000 7fbad5d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbad80b0460 0x7fbad806bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36954/0 (socket says 172.21.15.16:36954) 2024-09-19T07:43:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.257+0000 7fbad5d9b640 1 -- 172.21.15.16:0/863975181 learned_addr learned my addr 172.21.15.16:0/863975181 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.257+0000 7fbad6d9d640 1 --2- 172.21.15.16:0/863975181 >> v2:172.21.15.146:3300/0 conn(0x7fbad806c320 0x7fbad80b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.257+0000 7fbad659c640 1 --2- 172.21.15.16:0/863975181 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbad8145a20 0x7fbad80b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad559a640 1 -- 172.21.15.16:0/863975181 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3355727846 0 0) 0x7fbad806ca00 con 0x7fbad80b0460 2024-09-19T07:43:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad559a640 1 -- 172.21.15.16:0/863975181 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbac0003520 con 0x7fbad80b0460 2024-09-19T07:43:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad559a640 1 -- 172.21.15.16:0/863975181 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3961308640 0 0) 0x7fbac0003520 con 0x7fbad80b0460 2024-09-19T07:43:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad559a640 1 -- 172.21.15.16:0/863975181 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbad8145a20 msgr2=0x7fbad80b6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad559a640 1 --2- 172.21.15.16:0/863975181 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbad8145a20 0x7fbad80b6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad559a640 1 -- 172.21.15.16:0/863975181 >> v2:172.21.15.146:3300/0 conn(0x7fbad806c320 msgr2=0x7fbad80b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:00.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad559a640 1 --2- 172.21.15.16:0/863975181 >> v2:172.21.15.146:3300/0 conn(0x7fbad806c320 0x7fbad80b1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad559a640 1 -- 172.21.15.16:0/863975181 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad80b0c40 con 0x7fbad80b0460 2024-09-19T07:43:00.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad559a640 1 -- 172.21.15.16:0/863975181 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fbacc001690 con 0x7fbad80b0460 2024-09-19T07:43:00.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad659c640 1 --2- 172.21.15.16:0/863975181 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbad8145a20 0x7fbad80b6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:00.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad6d9d640 1 --2- 172.21.15.16:0/863975181 >> v2:172.21.15.146:3300/0 conn(0x7fbad806c320 0x7fbad80b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:00.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad559a640 1 -- 172.21.15.16:0/863975181 >> v1:172.21.15.16:6789/0 conn(0x7fbad80b0460 legacy=0x7fbad806bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:00.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad559a640 1 --2- 172.21.15.16:0/863975181 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbad8145a20 0x7fbac00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad559a640 1 --2- 172.21.15.16:0/863975181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbac0003ef0 0x7fbac0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad559a640 1 --2- 172.21.15.16:0/863975181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbac0004d40 0x7fbac00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad559a640 1 -- 172.21.15.16:0/863975181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbad80b1570 con 0x7fbac0004d40 2024-09-19T07:43:00.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad559a640 1 -- 172.21.15.16:0/863975181 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbad80b0960 con 0x7fbac0003ef0 2024-09-19T07:43:00.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad559a640 1 -- 172.21.15.16:0/863975181 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbac000b9c0 con 0x7fbad8145a20 2024-09-19T07:43:00.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad6d9d640 1 --2- 172.21.15.16:0/863975181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbac0004d40 0x7fbac00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad5d9b640 1 --2- 172.21.15.16:0/863975181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbac0003ef0 0x7fbac0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad659c640 1 --2- 172.21.15.16:0/863975181 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbad8145a20 0x7fbac00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad6d9d640 1 -- 172.21.15.16:0/863975181 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbad8145a20 msgr2=0x7fbac00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:00.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad6d9d640 1 --2- 172.21.15.16:0/863975181 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbad8145a20 0x7fbac00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad6d9d640 1 -- 172.21.15.16:0/863975181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbac0003ef0 msgr2=0x7fbac0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:00.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad6d9d640 1 --2- 172.21.15.16:0/863975181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbac0003ef0 0x7fbac0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.258+0000 7fbad6d9d640 1 -- 172.21.15.16:0/863975181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac0003520 con 0x7fbac0004d40 2024-09-19T07:43:00.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.259+0000 7fbad6d9d640 1 --2- 172.21.15.16:0/863975181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbac0004d40 0x7fbac00051c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbad000e320 tx=0x7fbac0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3628a71a757d4af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:00.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.259+0000 7fbad559a640 1 -- 172.21.15.16:0/863975181 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0009de0 con 0x7fbac0004d40 2024-09-19T07:43:00.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.259+0000 7fbad559a640 1 -- 172.21.15.16:0/863975181 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbad002e620 con 0x7fbac0004d40 2024-09-19T07:43:00.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.259+0000 7fbad559a640 1 -- 172.21.15.16:0/863975181 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad002e9d0 con 0x7fbac0004d40 2024-09-19T07:43:00.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.260+0000 7fbadce65640 1 -- 172.21.15.16:0/863975181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbac0004d40 msgr2=0x7fbac00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:00.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.260+0000 7fbadce65640 1 --2- 172.21.15.16:0/863975181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbac0004d40 0x7fbac00051c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbad000e320 tx=0x7fbac0005830 comp rx=0 tx=0).stop 2024-09-19T07:43:00.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.260+0000 7fbad6d9d640 1 -- 172.21.15.16:0/863975181 reap_dead start 2024-09-19T07:43:00.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.260+0000 7fbadce65640 1 -- 172.21.15.16:0/863975181 shutdown_connections 2024-09-19T07:43:00.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.260+0000 7fbadce65640 1 -- 172.21.15.16:0/863975181 >> 172.21.15.16:0/863975181 conn(0x7fbad809a9f0 msgr2=0x7fbad80b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:00.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.260+0000 7fbadce65640 1 -- 172.21.15.16:0/863975181 shutdown_connections 2024-09-19T07:43:00.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.260+0000 7fbadce65640 1 -- 172.21.15.16:0/863975181 wait complete. 2024-09-19T07:43:00.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.261+0000 7fbadce65640 1 Processor -- start 2024-09-19T07:43:00.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.261+0000 7fbadce65640 1 -- start start 2024-09-19T07:43:00.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.262+0000 7fbadce65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbac0004d40 0x7fbad81420f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.262+0000 7fbadce65640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbad80cd220 0x7fbad80cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.262+0000 7fbadce65640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbad80d03e0 0x7fbad809bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.262+0000 7fbadce65640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbad8145e00 con 0x7fbac0004d40 2024-09-19T07:43:00.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.262+0000 7fbadce65640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbad80b1200 con 0x7fbad80cd220 2024-09-19T07:43:00.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.262+0000 7fbadce65640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbad8145f80 con 0x7fbad80d03e0 2024-09-19T07:43:00.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.262+0000 7fbad659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbac0004d40 0x7fbad81420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.262+0000 7fbad659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbac0004d40 0x7fbad81420f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35700/0 (socket says 172.21.15.16:35700) 2024-09-19T07:43:00.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.262+0000 7fbad659c640 1 -- 172.21.15.16:0/3118068806 learned_addr learned my addr 172.21.15.16:0/3118068806 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:00.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.262+0000 7fbad5d9b640 1 --2- 172.21.15.16:0/3118068806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbad80cd220 0x7fbad80cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.262+0000 7fbad6d9d640 1 --2- 172.21.15.16:0/3118068806 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbad80d03e0 0x7fbad809bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.262+0000 7fbad659c640 1 -- 172.21.15.16:0/3118068806 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbad80d03e0 msgr2=0x7fbad809bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:00.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.262+0000 7fbad659c640 1 --2- 172.21.15.16:0/3118068806 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbad80d03e0 0x7fbad809bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.262+0000 7fbad659c640 1 -- 172.21.15.16:0/3118068806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbad80cd220 msgr2=0x7fbad80cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.262+0000 7fbad659c640 1 --2- 172.21.15.16:0/3118068806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbad80cd220 0x7fbad80cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.262+0000 7fbad659c640 1 -- 172.21.15.16:0/3118068806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad80cf1c0 con 0x7fbac0004d40 2024-09-19T07:43:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.262+0000 7fbad6d9d640 1 --2- 172.21.15.16:0/3118068806 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbad80d03e0 0x7fbad809bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.263+0000 7fbad659c640 1 --2- 172.21.15.16:0/3118068806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbac0004d40 0x7fbad81420f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbac80119d0 tx=0x7fbac8011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.263+0000 7fbac6ffd640 1 -- 172.21.15.16:0/3118068806 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac800fa90 con 0x7fbac0004d40 2024-09-19T07:43:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.263+0000 7fbac6ffd640 1 -- 172.21.15.16:0/3118068806 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac800fc30 con 0x7fbac0004d40 2024-09-19T07:43:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.263+0000 7fbadce65640 1 -- 172.21.15.16:0/3118068806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad809b210 con 0x7fbac0004d40 2024-09-19T07:43:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.263+0000 7fbac6ffd640 1 -- 172.21.15.16:0/3118068806 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8004030 con 0x7fbac0004d40 2024-09-19T07:43:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.263+0000 7fbadce65640 1 -- 172.21.15.16:0/3118068806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad809b7d0 con 0x7fbac0004d40 2024-09-19T07:43:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.264+0000 7fbadce65640 1 -- 172.21.15.16:0/3118068806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad80d07f0 con 0x7fbac0004d40 2024-09-19T07:43:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.264+0000 7fbac6ffd640 1 -- 172.21.15.16:0/3118068806 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbac8012040 con 0x7fbac0004d40 2024-09-19T07:43:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.265+0000 7fbac6ffd640 1 --2- 172.21.15.16:0/3118068806 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbab806c2e0 0x7fbab806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.265+0000 7fbac6ffd640 1 -- 172.21.15.16:0/3118068806 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fbac808b390 con 0x7fbac0004d40 2024-09-19T07:43:00.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.265+0000 7fbad5d9b640 1 --2- 172.21.15.16:0/3118068806 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbab806c2e0 0x7fbab806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.266+0000 7fbad5d9b640 1 --2- 172.21.15.16:0/3118068806 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbab806c2e0 0x7fbab806e7a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbad80cd660 tx=0x7fbacc001e70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:00.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.270+0000 7fbac6ffd640 1 -- 172.21.15.16:0/3118068806 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fbad80d07f0 con 0x7fbac0004d40 2024-09-19T07:43:00.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.485+0000 7fbadce65640 1 -- 172.21.15.16:0/3118068806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbad80b6cb0 con 0x7fbac0004d40 2024-09-19T07:43:00.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.485+0000 7fbac6ffd640 1 -- 172.21.15.16:0/3118068806 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1431 (secure 0 0 0) 0x7fbac8019070 con 0x7fbac0004d40 2024-09-19T07:43:00.486 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:00.486 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":2,"btime":"2024-09-19T07:43:00:224322+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/340172279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":340172279},{"type":"v1","addr":"172.21.15.16:6833","nonce":340172279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T07:43:00.486 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 2 2024-09-19T07:43:00.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.487+0000 7fbadce65640 1 -- 172.21.15.16:0/3118068806 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbab806c2e0 msgr2=0x7fbab806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:00.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.487+0000 7fbadce65640 1 --2- 172.21.15.16:0/3118068806 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbab806c2e0 0x7fbab806e7a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbad80cd660 tx=0x7fbacc001e70 comp rx=0 tx=0).stop 2024-09-19T07:43:00.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.487+0000 7fbadce65640 1 -- 172.21.15.16:0/3118068806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbac0004d40 msgr2=0x7fbad81420f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:00.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.487+0000 7fbadce65640 1 --2- 172.21.15.16:0/3118068806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbac0004d40 0x7fbad81420f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbac80119d0 tx=0x7fbac8011dd0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.487+0000 7fbadce65640 1 -- 172.21.15.16:0/3118068806 shutdown_connections 2024-09-19T07:43:00.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.487+0000 7fbadce65640 1 --2- 172.21.15.16:0/3118068806 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbab806c2e0 0x7fbab806e7a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.487+0000 7fbadce65640 1 --2- 172.21.15.16:0/3118068806 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbad80d03e0 0x7fbad809bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.487+0000 7fbadce65640 1 --2- 172.21.15.16:0/3118068806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbad80cd220 0x7fbad80cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.487+0000 7fbadce65640 1 --2- 172.21.15.16:0/3118068806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbac0004d40 0x7fbad81420f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.487+0000 7fbadce65640 1 -- 172.21.15.16:0/3118068806 >> 172.21.15.16:0/3118068806 conn(0x7fbad809a9f0 msgr2=0x7fbad80b3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:00.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.487+0000 7fbadce65640 1 -- 172.21.15.16:0/3118068806 shutdown_connections 2024-09-19T07:43:00.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:00.487+0000 7fbadce65640 1 -- 172.21.15.16:0/3118068806 wait complete. 2024-09-19T07:43:02.499 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:43:02.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.614+0000 7f2abcdb7640 1 Processor -- start 2024-09-19T07:43:02.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.614+0000 7f2abcdb7640 1 -- start start 2024-09-19T07:43:02.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.615+0000 7f2abcdb7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab815f0d0 0x7f2ab815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.615+0000 7f2abcdb7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2ab80d4130 0x7f2ab80d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.615+0000 7f2abcdb7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ab80d4b20 con 0x7f2ab80d0070 2024-09-19T07:43:02.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.615+0000 7f2abcdb7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2ab80d1b10 con 0x7f2ab80d4130 2024-09-19T07:43:02.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.615+0000 7f2abcdb7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab80cf2b0 con 0x7f2ab815f0d0 2024-09-19T07:43:02.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.615+0000 7f2ab5d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2ab80d0070 0x7f2ab80d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42352/0 (socket says 172.21.15.16:42352) 2024-09-19T07:43:02.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.615+0000 7f2ab5d9b640 1 -- 172.21.15.16:0/1153577826 learned_addr learned my addr 172.21.15.16:0/1153577826 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:02.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.615+0000 7f2ab659c640 1 --2- 172.21.15.16:0/1153577826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab815f0d0 0x7f2ab815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.615+0000 7f2ab6d9d640 1 --2- 172.21.15.16:0/1153577826 >> v2:172.21.15.146:3300/0 conn(0x7f2ab80d4130 0x7f2ab80d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.615+0000 7f2ab559a640 1 -- 172.21.15.16:0/1153577826 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4104822993 0 0) 0x7f2aac001110 con 0x7f2ab80d0070 2024-09-19T07:43:02.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.615+0000 7f2ab559a640 1 -- 172.21.15.16:0/1153577826 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2aa0003520 con 0x7f2ab80d0070 2024-09-19T07:43:02.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.616+0000 7f2ab6d9d640 1 -- 172.21.15.16:0/1153577826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab815f0d0 msgr2=0x7f2ab815f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:02.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.616+0000 7f2ab6d9d640 1 --2- 172.21.15.16:0/1153577826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab815f0d0 0x7f2ab815f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.616+0000 7f2ab6d9d640 1 -- 172.21.15.16:0/1153577826 >> v1:172.21.15.16:6789/0 conn(0x7f2ab80d0070 legacy=0x7f2ab80d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:02.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.616+0000 7f2ab6d9d640 1 -- 172.21.15.16:0/1153577826 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab80cf590 con 0x7f2ab80d4130 2024-09-19T07:43:02.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.616+0000 7f2ab6d9d640 1 --2- 172.21.15.16:0/1153577826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab80d4130 0x7f2ab80d45e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80d2fc0 tx=0x7f2ab002a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d599f28a05a0257b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:02.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.617+0000 7f2ab559a640 1 -- 172.21.15.16:0/1153577826 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0007740 con 0x7f2ab80d4130 2024-09-19T07:43:02.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.617+0000 7f2ab559a640 1 -- 172.21.15.16:0/1153577826 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab00078e0 con 0x7f2ab80d4130 2024-09-19T07:43:02.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.617+0000 7f2ab559a640 1 -- 172.21.15.16:0/1153577826 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0007cb0 con 0x7f2ab80d4130 2024-09-19T07:43:02.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.617+0000 7f2abcdb7640 1 -- 172.21.15.16:0/1153577826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab80d4130 msgr2=0x7f2ab80d45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:02.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.617+0000 7f2abcdb7640 1 --2- 172.21.15.16:0/1153577826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab80d4130 0x7f2ab80d45e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80d2fc0 tx=0x7f2ab002a5b0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.617+0000 7f2abcdb7640 1 -- 172.21.15.16:0/1153577826 shutdown_connections 2024-09-19T07:43:02.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.617+0000 7f2abcdb7640 1 --2- 172.21.15.16:0/1153577826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab80d4130 0x7f2ab80d45e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.617+0000 7f2abcdb7640 1 --2- 172.21.15.16:0/1153577826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab815f0d0 0x7f2ab815f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.617+0000 7f2abcdb7640 1 -- 172.21.15.16:0/1153577826 >> 172.21.15.16:0/1153577826 conn(0x7f2ab809d760 msgr2=0x7f2ab808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:02.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.617+0000 7f2abcdb7640 1 -- 172.21.15.16:0/1153577826 shutdown_connections 2024-09-19T07:43:02.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.617+0000 7f2abcdb7640 1 -- 172.21.15.16:0/1153577826 wait complete. 2024-09-19T07:43:02.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.619+0000 7f2abcdb7640 1 Processor -- start 2024-09-19T07:43:02.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.619+0000 7f2abcdb7640 1 -- start start 2024-09-19T07:43:02.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.619+0000 7f2abcdb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab80d0070 0x7f2ab814b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.619+0000 7f2abcdb7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab80d4130 0x7f2ab80cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.619+0000 7f2abcdb7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab815f0d0 0x7f2ab80b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.620+0000 7f2abcdb7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab80cef60 con 0x7f2ab80d0070 2024-09-19T07:43:02.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.620+0000 7f2abcdb7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab80cfb50 con 0x7f2ab80d4130 2024-09-19T07:43:02.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.620+0000 7f2abcdb7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab80b2f70 con 0x7f2ab815f0d0 2024-09-19T07:43:02.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.620+0000 7f2ab659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab80d0070 0x7f2ab814b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.620+0000 7f2ab659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab80d0070 0x7f2ab814b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51598/0 (socket says 172.21.15.16:51598) 2024-09-19T07:43:02.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.620+0000 7f2ab659c640 1 -- 172.21.15.16:0/1197643363 learned_addr learned my addr 172.21.15.16:0/1197643363 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:02.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.620+0000 7f2ab5d9b640 1 --2- 172.21.15.16:0/1197643363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab80d4130 0x7f2ab80cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.620+0000 7f2ab6d9d640 1 --2- 172.21.15.16:0/1197643363 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab815f0d0 0x7f2ab80b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.620+0000 7f2ab659c640 1 -- 172.21.15.16:0/1197643363 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab815f0d0 msgr2=0x7f2ab80b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:02.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.620+0000 7f2ab659c640 1 --2- 172.21.15.16:0/1197643363 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab815f0d0 0x7f2ab80b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.620+0000 7f2ab659c640 1 -- 172.21.15.16:0/1197643363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab80d4130 msgr2=0x7f2ab80cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:02.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.620+0000 7f2ab659c640 1 --2- 172.21.15.16:0/1197643363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab80d4130 0x7f2ab80cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.620+0000 7f2ab659c640 1 -- 172.21.15.16:0/1197643363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab8148650 con 0x7f2ab80d0070 2024-09-19T07:43:02.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.620+0000 7f2ab659c640 1 --2- 172.21.15.16:0/1197643363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab80d0070 0x7f2ab814b200 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800e870 tx=0x7f2aa800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:02.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.620+0000 7f2aa6ffd640 1 -- 172.21.15.16:0/1197643363 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8017070 con 0x7f2ab80d0070 2024-09-19T07:43:02.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.620+0000 7f2aa6ffd640 1 -- 172.21.15.16:0/1197643363 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa800c940 con 0x7f2ab80d0070 2024-09-19T07:43:02.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.620+0000 7f2abcdb7640 1 -- 172.21.15.16:0/1197643363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab8148930 con 0x7f2ab80d0070 2024-09-19T07:43:02.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.620+0000 7f2aa6ffd640 1 -- 172.21.15.16:0/1197643363 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa800cd30 con 0x7f2ab80d0070 2024-09-19T07:43:02.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.620+0000 7f2abcdb7640 1 -- 172.21.15.16:0/1197643363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab8148ef0 con 0x7f2ab80d0070 2024-09-19T07:43:02.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.622+0000 7f2abcdb7640 1 -- 172.21.15.16:0/1197643363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab8161a30 con 0x7f2ab80d0070 2024-09-19T07:43:02.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.622+0000 7f2aa6ffd640 1 -- 172.21.15.16:0/1197643363 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2aa801a070 con 0x7f2ab80d0070 2024-09-19T07:43:02.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.623+0000 7f2aa6ffd640 1 --2- 172.21.15.16:0/1197643363 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2a9806c2b0 0x7f2a9806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.623+0000 7f2aa6ffd640 1 -- 172.21.15.16:0/1197643363 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f2aa808b340 con 0x7f2ab80d0070 2024-09-19T07:43:02.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.623+0000 7f2ab5d9b640 1 --2- 172.21.15.16:0/1197643363 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2a9806c2b0 0x7f2a9806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.623+0000 7f2ab5d9b640 1 --2- 172.21.15.16:0/1197643363 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2a9806c2b0 0x7f2a9806e770 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80b2f40 tx=0x7f2aac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:02.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.627+0000 7f2aa6ffd640 1 -- 172.21.15.16:0/1197643363 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2ab8161a30 con 0x7f2ab80d0070 2024-09-19T07:43:02.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.842+0000 7f2abcdb7640 1 -- 172.21.15.16:0/1197643363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ab806cf30 con 0x7f2ab80d0070 2024-09-19T07:43:02.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.842+0000 7f2aa6ffd640 1 -- 172.21.15.16:0/1197643363 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3909 (secure 0 0 0) 0x7f2aa8059710 con 0x7f2ab80d0070 2024-09-19T07:43:02.843 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:02.843 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":3,"btime":"2024-09-19T07:43:01:225512+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2772948330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2772948330},{"type":"v1","addr":"172.21.15.146:6835","nonce":2772948330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/340172279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":340172279},{"type":"v1","addr":"172.21.15.16:6833","nonce":340172279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1204811459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1204811459},{"type":"v1","addr":"172.21.15.16:6835","nonce":1204811459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1303703249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1303703249},{"type":"v1","addr":"172.21.15.146:6837","nonce":1303703249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-19T07:43:02.843 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 3 2024-09-19T07:43:02.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.844+0000 7f2abcdb7640 1 -- 172.21.15.16:0/1197643363 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2a9806c2b0 msgr2=0x7f2a9806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:02.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.844+0000 7f2abcdb7640 1 --2- 172.21.15.16:0/1197643363 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2a9806c2b0 0x7f2a9806e770 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80b2f40 tx=0x7f2aac009040 comp rx=0 tx=0).stop 2024-09-19T07:43:02.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.844+0000 7f2abcdb7640 1 -- 172.21.15.16:0/1197643363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab80d0070 msgr2=0x7f2ab814b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:02.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.844+0000 7f2abcdb7640 1 --2- 172.21.15.16:0/1197643363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab80d0070 0x7f2ab814b200 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800e870 tx=0x7f2aa800ed30 comp rx=0 tx=0).stop 2024-09-19T07:43:02.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.844+0000 7f2abcdb7640 1 -- 172.21.15.16:0/1197643363 shutdown_connections 2024-09-19T07:43:02.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.844+0000 7f2abcdb7640 1 --2- 172.21.15.16:0/1197643363 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2a9806c2b0 0x7f2a9806e770 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.844+0000 7f2abcdb7640 1 --2- 172.21.15.16:0/1197643363 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab815f0d0 0x7f2ab80b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.844+0000 7f2abcdb7640 1 --2- 172.21.15.16:0/1197643363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab80d4130 0x7f2ab80cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.844+0000 7f2abcdb7640 1 --2- 172.21.15.16:0/1197643363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab80d0070 0x7f2ab814b200 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.844+0000 7f2abcdb7640 1 -- 172.21.15.16:0/1197643363 >> 172.21.15.16:0/1197643363 conn(0x7f2ab809d760 msgr2=0x7f2ab808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:02.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.844+0000 7f2abcdb7640 1 -- 172.21.15.16:0/1197643363 shutdown_connections 2024-09-19T07:43:02.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.844+0000 7f2abcdb7640 1 -- 172.21.15.16:0/1197643363 wait complete. 2024-09-19T07:43:02.856 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T07:43:02.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.957+0000 7fac302e6640 1 Processor -- start 2024-09-19T07:43:02.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.957+0000 7fac302e6640 1 -- start start 2024-09-19T07:43:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.958+0000 7fac302e6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac281619b0 0x7fac280d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.958+0000 7fac302e6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fac280b0a80 0x7fac280d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.958+0000 7fac302e6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac2806c990 con 0x7fac28161d90 2024-09-19T07:43:02.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.958+0000 7fac302e6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fac280d0ec0 con 0x7fac280b0a80 2024-09-19T07:43:02.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.958+0000 7fac302e6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fac280cec10 con 0x7fac281619b0 2024-09-19T07:43:02.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.958+0000 7fac2d881640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fac280b0a80 0x7fac280d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.958+0000 7fac2d881640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fac280b0a80 0x7fac280d0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51608/0 (socket says 172.21.15.16:51608) 2024-09-19T07:43:02.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.958+0000 7fac2d881640 1 -- 172.21.15.16:0/3557095921 learned_addr learned my addr 172.21.15.16:0/3557095921 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:02.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.958+0000 7fac2e082640 1 --2- 172.21.15.16:0/3557095921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac281619b0 0x7fac280d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.958+0000 7fac2d881640 1 -- 172.21.15.16:0/3557095921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac281619b0 msgr2=0x7fac280d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:02.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.958+0000 7fac2d881640 1 --2- 172.21.15.16:0/3557095921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac281619b0 0x7fac280d0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.958+0000 7fac2d881640 1 -- 172.21.15.16:0/3557095921 >> v1:172.21.15.146:6789/0 conn(0x7fac28161d90 legacy=0x7fac280cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:02.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.958+0000 7fac2d881640 1 -- 172.21.15.16:0/3557095921 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac280ce930 con 0x7fac280b0a80 2024-09-19T07:43:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.959+0000 7fac2d881640 1 --2- 172.21.15.16:0/3557095921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac280b0a80 0x7fac280d0980 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fac18009870 tx=0x7fac1802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404dda8e6d591f9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.959+0000 7fac2d080640 1 -- 172.21.15.16:0/3557095921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18037070 con 0x7fac280b0a80 2024-09-19T07:43:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.959+0000 7fac2d080640 1 -- 172.21.15.16:0/3557095921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac18032440 con 0x7fac280b0a80 2024-09-19T07:43:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.959+0000 7fac2d080640 1 -- 172.21.15.16:0/3557095921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18032810 con 0x7fac280b0a80 2024-09-19T07:43:02.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.960+0000 7fac302e6640 1 -- 172.21.15.16:0/3557095921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac280b0a80 msgr2=0x7fac280d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:02.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.960+0000 7fac302e6640 1 --2- 172.21.15.16:0/3557095921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac280b0a80 0x7fac280d0980 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fac18009870 tx=0x7fac1802a210 comp rx=0 tx=0).stop 2024-09-19T07:43:02.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.960+0000 7fac302e6640 1 -- 172.21.15.16:0/3557095921 shutdown_connections 2024-09-19T07:43:02.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.960+0000 7fac302e6640 1 --2- 172.21.15.16:0/3557095921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac280b0a80 0x7fac280d0980 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.960+0000 7fac302e6640 1 --2- 172.21.15.16:0/3557095921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac281619b0 0x7fac280d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.960+0000 7fac302e6640 1 -- 172.21.15.16:0/3557095921 >> 172.21.15.16:0/3557095921 conn(0x7fac2809a9f0 msgr2=0x7fac280b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:02.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.960+0000 7fac302e6640 1 -- 172.21.15.16:0/3557095921 shutdown_connections 2024-09-19T07:43:02.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.960+0000 7fac302e6640 1 -- 172.21.15.16:0/3557095921 wait complete. 2024-09-19T07:43:02.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.961+0000 7fac302e6640 1 Processor -- start 2024-09-19T07:43:02.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.961+0000 7fac302e6640 1 -- start start 2024-09-19T07:43:02.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.962+0000 7fac302e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac280b0a80 0x7fac281f7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.962+0000 7fac302e6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac281619b0 0x7fac2815bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.962+0000 7fac302e6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac28161d90 0x7fac281494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.962+0000 7fac302e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac2806c620 con 0x7fac280b0a80 2024-09-19T07:43:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.962+0000 7fac302e6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fac280b0820 con 0x7fac281619b0 2024-09-19T07:43:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.962+0000 7fac302e6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fac280ccce0 con 0x7fac28161d90 2024-09-19T07:43:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.962+0000 7fac2e082640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac280b0a80 0x7fac281f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.962+0000 7fac2e082640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac280b0a80 0x7fac281f7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51618/0 (socket says 172.21.15.16:51618) 2024-09-19T07:43:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.962+0000 7fac2e082640 1 -- 172.21.15.16:0/282960245 learned_addr learned my addr 172.21.15.16:0/282960245 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.962+0000 7fac2e883640 1 --2- 172.21.15.16:0/282960245 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac28161d90 0x7fac281494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.962+0000 7fac2d881640 1 --2- 172.21.15.16:0/282960245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac281619b0 0x7fac2815bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.962+0000 7fac2e082640 1 -- 172.21.15.16:0/282960245 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac28161d90 msgr2=0x7fac281494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.962+0000 7fac2e082640 1 --2- 172.21.15.16:0/282960245 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac28161d90 0x7fac281494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.962+0000 7fac2e082640 1 -- 172.21.15.16:0/282960245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac281619b0 msgr2=0x7fac2815bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.962+0000 7fac2e082640 1 --2- 172.21.15.16:0/282960245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac281619b0 0x7fac2815bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.962+0000 7fac2e082640 1 -- 172.21.15.16:0/282960245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac280c60a0 con 0x7fac280b0a80 2024-09-19T07:43:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.962+0000 7fac2e082640 1 --2- 172.21.15.16:0/282960245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac280b0a80 0x7fac281f7280 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fac2400c820 tx=0x7fac2400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.962+0000 7fac1effd640 1 -- 172.21.15.16:0/282960245 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24015070 con 0x7fac280b0a80 2024-09-19T07:43:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.963+0000 7fac1effd640 1 -- 172.21.15.16:0/282960245 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac24004240 con 0x7fac280b0a80 2024-09-19T07:43:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.963+0000 7fac302e6640 1 -- 172.21.15.16:0/282960245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac280c6380 con 0x7fac280b0a80 2024-09-19T07:43:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.963+0000 7fac1effd640 1 -- 172.21.15.16:0/282960245 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2400ce10 con 0x7fac280b0a80 2024-09-19T07:43:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.963+0000 7fac302e6640 1 -- 172.21.15.16:0/282960245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac280c6940 con 0x7fac280b0a80 2024-09-19T07:43:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.964+0000 7fac302e6640 1 -- 172.21.15.16:0/282960245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac280cd330 con 0x7fac280b0a80 2024-09-19T07:43:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.964+0000 7fac1effd640 1 -- 172.21.15.16:0/282960245 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fac24007830 con 0x7fac280b0a80 2024-09-19T07:43:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.964+0000 7fac1effd640 1 --2- 172.21.15.16:0/282960245 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fac0c06c2e0 0x7fac0c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.965+0000 7fac1effd640 1 -- 172.21.15.16:0/282960245 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fac2408aee0 con 0x7fac280b0a80 2024-09-19T07:43:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.965+0000 7fac2d881640 1 --2- 172.21.15.16:0/282960245 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fac0c06c2e0 0x7fac0c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.965+0000 7fac2d881640 1 --2- 172.21.15.16:0/282960245 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fac0c06c2e0 0x7fac0c06e7a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fac18034000 tx=0x7fac1802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:02.969+0000 7fac1effd640 1 -- 172.21.15.16:0/282960245 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fac24059350 con 0x7fac280b0a80 2024-09-19T07:43:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.146+0000 7fac302e6640 1 -- 172.21.15.16:0/282960245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fac28070d10 con 0x7fac280b0a80 2024-09-19T07:43:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.146+0000 7fac1effd640 1 -- 172.21.15.16:0/282960245 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7fac2405d650 con 0x7fac280b0a80 2024-09-19T07:43:03.146 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:03.147 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-19T07:43:03.147 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:03.147 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-19T07:43:03.147 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-19T07:43:03.147 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:03.147 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-19T07:43:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.148+0000 7fac302e6640 1 -- 172.21.15.16:0/282960245 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fac0c06c2e0 msgr2=0x7fac0c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.148+0000 7fac302e6640 1 --2- 172.21.15.16:0/282960245 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fac0c06c2e0 0x7fac0c06e7a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fac18034000 tx=0x7fac1802e000 comp rx=0 tx=0).stop 2024-09-19T07:43:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.148+0000 7fac302e6640 1 -- 172.21.15.16:0/282960245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac280b0a80 msgr2=0x7fac281f7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.148+0000 7fac302e6640 1 --2- 172.21.15.16:0/282960245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac280b0a80 0x7fac281f7280 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fac2400c820 tx=0x7fac2400cce0 comp rx=0 tx=0).stop 2024-09-19T07:43:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.148+0000 7fac302e6640 1 -- 172.21.15.16:0/282960245 shutdown_connections 2024-09-19T07:43:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.148+0000 7fac302e6640 1 --2- 172.21.15.16:0/282960245 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fac0c06c2e0 0x7fac0c06e7a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.148+0000 7fac302e6640 1 --2- 172.21.15.16:0/282960245 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac28161d90 0x7fac281494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.148+0000 7fac302e6640 1 --2- 172.21.15.16:0/282960245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac281619b0 0x7fac2815bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.148+0000 7fac302e6640 1 --2- 172.21.15.16:0/282960245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac280b0a80 0x7fac281f7280 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.148+0000 7fac302e6640 1 -- 172.21.15.16:0/282960245 >> 172.21.15.16:0/282960245 conn(0x7fac2809a9f0 msgr2=0x7fac280d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.148+0000 7fac302e6640 1 -- 172.21.15.16:0/282960245 shutdown_connections 2024-09-19T07:43:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.148+0000 7fac302e6640 1 -- 172.21.15.16:0/282960245 wait complete. 2024-09-19T07:43:03.159 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T07:43:03.159 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T07:43:03.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.264+0000 7f25358e4640 1 Processor -- start 2024-09-19T07:43:03.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.264+0000 7f25358e4640 1 -- start start 2024-09-19T07:43:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.265+0000 7f25358e4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f25300d1090 0x7f2530070b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.265+0000 7f25358e4640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f25300afed0 0x7f253006f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.265+0000 7f25358e4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25300ae460 con 0x7f25300adff0 2024-09-19T07:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.265+0000 7f25358e4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f25300b46d0 con 0x7f25300d1090 2024-09-19T07:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.265+0000 7f25358e4640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f25300b43f0 con 0x7f25300afed0 2024-09-19T07:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.265+0000 7f252effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f25300d1090 0x7f2530070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.265+0000 7f252e7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f25300adff0 0x7f25300d1470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42362/0 (socket says 172.21.15.16:42362) 2024-09-19T07:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.266+0000 7f252e7fc640 1 -- 172.21.15.16:0/1140187426 learned_addr learned my addr 172.21.15.16:0/1140187426 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.266+0000 7f252f7fe640 1 --2- 172.21.15.16:0/1140187426 >> v2:172.21.15.146:3300/0 conn(0x7f25300afed0 0x7f253006f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.266+0000 7f252effd640 1 -- 172.21.15.16:0/1140187426 >> v2:172.21.15.146:3300/0 conn(0x7f25300afed0 msgr2=0x7f253006f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.266+0000 7f252effd640 1 --2- 172.21.15.16:0/1140187426 >> v2:172.21.15.146:3300/0 conn(0x7f25300afed0 0x7f253006f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.266+0000 7f252effd640 1 -- 172.21.15.16:0/1140187426 >> v1:172.21.15.16:6789/0 conn(0x7f25300adff0 legacy=0x7f25300d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.266+0000 7f252effd640 1 -- 172.21.15.16:0/1140187426 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25300b5e30 con 0x7f25300d1090 2024-09-19T07:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.266+0000 7f252f7fe640 1 --2- 172.21.15.16:0/1140187426 >> v2:172.21.15.146:3300/0 conn(0x7f25300afed0 0x7f253006f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.266+0000 7f252effd640 1 --2- 172.21.15.16:0/1140187426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25300d1090 0x7f2530070b10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f25280097a0 tx=0x7f252802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b16dfdae9382e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.267+0000 7f252dffb640 1 -- 172.21.15.16:0/1140187426 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528037070 con 0x7f25300d1090 2024-09-19T07:43:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.267+0000 7f252dffb640 1 -- 172.21.15.16:0/1140187426 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2528032440 con 0x7f25300d1090 2024-09-19T07:43:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.267+0000 7f252dffb640 1 -- 172.21.15.16:0/1140187426 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528032810 con 0x7f25300d1090 2024-09-19T07:43:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.267+0000 7f25358e4640 1 -- 172.21.15.16:0/1140187426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25300d1090 msgr2=0x7f2530070b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.267+0000 7f25358e4640 1 --2- 172.21.15.16:0/1140187426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25300d1090 0x7f2530070b10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f25280097a0 tx=0x7f252802a1e0 comp rx=0 tx=0).stop 2024-09-19T07:43:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.267+0000 7f25358e4640 1 -- 172.21.15.16:0/1140187426 shutdown_connections 2024-09-19T07:43:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.267+0000 7f25358e4640 1 --2- 172.21.15.16:0/1140187426 >> v2:172.21.15.146:3300/0 conn(0x7f25300afed0 0x7f253006f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.267+0000 7f25358e4640 1 --2- 172.21.15.16:0/1140187426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25300d1090 0x7f2530070b10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.267+0000 7f25358e4640 1 -- 172.21.15.16:0/1140187426 >> 172.21.15.16:0/1140187426 conn(0x7f25300b6530 msgr2=0x7f25300b6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.267+0000 7f25358e4640 1 -- 172.21.15.16:0/1140187426 shutdown_connections 2024-09-19T07:43:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.267+0000 7f25358e4640 1 -- 172.21.15.16:0/1140187426 wait complete. 2024-09-19T07:43:03.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.268+0000 7f25358e4640 1 Processor -- start 2024-09-19T07:43:03.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.269+0000 7f25358e4640 1 -- start start 2024-09-19T07:43:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.269+0000 7f25358e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25300adff0 0x7f2530147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.269+0000 7f25358e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25300afed0 0x7f253015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:03.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.269+0000 7f25358e4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f25300d1090 0x7f25301f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.269+0000 7f25358e4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f25300b5ae0 con 0x7f25300adff0 2024-09-19T07:43:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.269+0000 7f25358e4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f25300b0320 con 0x7f25300afed0 2024-09-19T07:43:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.269+0000 7f25358e4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f25300b4010 con 0x7f25300d1090 2024-09-19T07:43:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.269+0000 7f252effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25300adff0 0x7f2530147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.269+0000 7f252effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25300adff0 0x7f2530147bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51642/0 (socket says 172.21.15.16:51642) 2024-09-19T07:43:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.269+0000 7f252effd640 1 -- 172.21.15.16:0/898240737 learned_addr learned my addr 172.21.15.16:0/898240737 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.270+0000 7f252e7fc640 1 --2- 172.21.15.16:0/898240737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25300afed0 0x7f253015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.270+0000 7f252f7fe640 1 --2- 172.21.15.16:0/898240737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f25300d1090 0x7f25301f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.270+0000 7f252effd640 1 -- 172.21.15.16:0/898240737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f25300d1090 msgr2=0x7f25301f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.270+0000 7f252effd640 1 --2- 172.21.15.16:0/898240737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f25300d1090 0x7f25301f41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.270+0000 7f252effd640 1 -- 172.21.15.16:0/898240737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25300afed0 msgr2=0x7f253015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:03.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.270+0000 7f252effd640 1 --2- 172.21.15.16:0/898240737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25300afed0 0x7f253015c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.270+0000 7f252effd640 1 -- 172.21.15.16:0/898240737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2530160690 con 0x7f25300adff0 2024-09-19T07:43:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.270+0000 7f252e7fc640 1 --2- 172.21.15.16:0/898240737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25300afed0 0x7f253015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.270+0000 7f252effd640 1 --2- 172.21.15.16:0/898240737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25300adff0 0x7f2530147bf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2528035040 tx=0x7f25280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.270+0000 7f2527fff640 1 -- 172.21.15.16:0/898240737 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528037040 con 0x7f25300adff0 2024-09-19T07:43:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.270+0000 7f2527fff640 1 -- 172.21.15.16:0/898240737 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2528030770 con 0x7f25300adff0 2024-09-19T07:43:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.270+0000 7f25358e4640 1 -- 172.21.15.16:0/898240737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2530148130 con 0x7f25300adff0 2024-09-19T07:43:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.270+0000 7f2527fff640 1 -- 172.21.15.16:0/898240737 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528030b80 con 0x7f25300adff0 2024-09-19T07:43:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.270+0000 7f25358e4640 1 -- 172.21.15.16:0/898240737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f253015f5f0 con 0x7f25300adff0 2024-09-19T07:43:03.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.271+0000 7f25358e4640 1 -- 172.21.15.16:0/898240737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f253015e780 con 0x7f25300adff0 2024-09-19T07:43:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.272+0000 7f2527fff640 1 -- 172.21.15.16:0/898240737 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2528044020 con 0x7f25300adff0 2024-09-19T07:43:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.272+0000 7f2527fff640 1 --2- 172.21.15.16:0/898240737 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f251006c2b0 0x7f251006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.272+0000 7f2527fff640 1 -- 172.21.15.16:0/898240737 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f25280ab5e0 con 0x7f25300adff0 2024-09-19T07:43:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.272+0000 7f252e7fc640 1 --2- 172.21.15.16:0/898240737 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f251006c2b0 0x7f251006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.273+0000 7f252e7fc640 1 --2- 172.21.15.16:0/898240737 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f251006c2b0 0x7f251006e770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2530075670 tx=0x7f2518009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.276+0000 7f2527fff640 1 -- 172.21.15.16:0/898240737 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2528032e20 con 0x7f25300adff0 2024-09-19T07:43:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:03.454+0000 7f25358e4640 1 -- 172.21.15.16:0/898240737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2530075230 con 0x7f25300adff0 2024-09-19T07:43:04.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.236+0000 7f2527fff640 1 -- 172.21.15.16:0/898240737 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7f252807a130 con 0x7f25300adff0 2024-09-19T07:43:04.237 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-19T07:43:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.240+0000 7f25358e4640 1 -- 172.21.15.16:0/898240737 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f251006c2b0 msgr2=0x7f251006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.240+0000 7f25358e4640 1 --2- 172.21.15.16:0/898240737 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f251006c2b0 0x7f251006e770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2530075670 tx=0x7f2518009040 comp rx=0 tx=0).stop 2024-09-19T07:43:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.240+0000 7f25358e4640 1 -- 172.21.15.16:0/898240737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25300adff0 msgr2=0x7f2530147bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.240+0000 7f25358e4640 1 --2- 172.21.15.16:0/898240737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25300adff0 0x7f2530147bf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2528035040 tx=0x7f25280046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.241+0000 7f25358e4640 1 -- 172.21.15.16:0/898240737 shutdown_connections 2024-09-19T07:43:04.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.241+0000 7f25358e4640 1 --2- 172.21.15.16:0/898240737 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f251006c2b0 0x7f251006e770 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.241+0000 7f25358e4640 1 --2- 172.21.15.16:0/898240737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f25300d1090 0x7f25301f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.241+0000 7f25358e4640 1 --2- 172.21.15.16:0/898240737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25300afed0 0x7f253015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.241+0000 7f25358e4640 1 --2- 172.21.15.16:0/898240737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25300adff0 0x7f2530147bf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.241+0000 7f25358e4640 1 -- 172.21.15.16:0/898240737 >> 172.21.15.16:0/898240737 conn(0x7f25300b6530 msgr2=0x7f253006f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:04.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.242+0000 7f25358e4640 1 -- 172.21.15.16:0/898240737 shutdown_connections 2024-09-19T07:43:04.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.242+0000 7f25358e4640 1 -- 172.21.15.16:0/898240737 wait complete. 2024-09-19T07:43:04.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-19T07:43:04.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.383+0000 7f4c46bd8640 1 Processor -- start 2024-09-19T07:43:04.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.383+0000 7f4c46bd8640 1 -- start start 2024-09-19T07:43:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.384+0000 7f4c46bd8640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4c4015e0a0 0x7f4c40162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.384+0000 7f4c46bd8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c4015f250 con 0x7f4c400b5280 2024-09-19T07:43:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.384+0000 7f4c46bd8640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c4015f530 con 0x7f4c400d43e0 2024-09-19T07:43:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.384+0000 7f4c46bd8640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4c4015f810 con 0x7f4c4015e0a0 2024-09-19T07:43:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.384+0000 7f4c45175640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4c400b5280 0x7f4c4015e480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42378/0 (socket says 172.21.15.16:42378) 2024-09-19T07:43:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.384+0000 7f4c45175640 1 -- 172.21.15.16:0/2652059175 learned_addr learned my addr 172.21.15.16:0/2652059175 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.384+0000 7f4c44974640 1 --2- 172.21.15.16:0/2652059175 >> v2:172.21.15.146:3300/0 conn(0x7f4c4015e0a0 0x7f4c40162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.384+0000 7f4c377fe640 1 -- 172.21.15.16:0/2652059175 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1994825206 0 0) 0x7f4c4015f250 con 0x7f4c400b5280 2024-09-19T07:43:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.384+0000 7f4c377fe640 1 -- 172.21.15.16:0/2652059175 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c28003520 con 0x7f4c400b5280 2024-09-19T07:43:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.384+0000 7f4c377fe640 1 -- 172.21.15.16:0/2652059175 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2975897644 0 0) 0x7f4c28003520 con 0x7f4c400b5280 2024-09-19T07:43:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c377fe640 1 -- 172.21.15.16:0/2652059175 >> v2:172.21.15.146:3300/0 conn(0x7f4c4015e0a0 msgr2=0x7f4c40162a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c377fe640 1 --2- 172.21.15.16:0/2652059175 >> v2:172.21.15.146:3300/0 conn(0x7f4c4015e0a0 0x7f4c40162a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c377fe640 1 -- 172.21.15.16:0/2652059175 >> v1:172.21.15.146:6789/0 conn(0x7f4c400d43e0 legacy=0x7f4c400b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c377fe640 1 -- 172.21.15.16:0/2652059175 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c400b4650 con 0x7f4c400b5280 2024-09-19T07:43:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c377fe640 1 -- 172.21.15.16:0/2652059175 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f4c380019f0 con 0x7f4c400b5280 2024-09-19T07:43:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c377fe640 1 -- 172.21.15.16:0/2652059175 >> v1:172.21.15.16:6789/0 conn(0x7f4c400b5280 legacy=0x7f4c4015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c377fe640 1 --2- 172.21.15.16:0/2652059175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c280039b0 0x7f4c28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c377fe640 1 --2- 172.21.15.16:0/2652059175 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c28004800 0x7f4c28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c377fe640 1 --2- 172.21.15.16:0/2652059175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c280051c0 0x7f4c2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c377fe640 1 -- 172.21.15.16:0/2652059175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4015f810 con 0x7f4c280039b0 2024-09-19T07:43:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c377fe640 1 -- 172.21.15.16:0/2652059175 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2800c060 con 0x7f4c280051c0 2024-09-19T07:43:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c377fe640 1 -- 172.21.15.16:0/2652059175 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4c2800c260 con 0x7f4c28004800 2024-09-19T07:43:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c44974640 1 --2- 172.21.15.16:0/2652059175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c280039b0 0x7f4c28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c45175640 1 --2- 172.21.15.16:0/2652059175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c280051c0 0x7f4c2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c37fff640 1 --2- 172.21.15.16:0/2652059175 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c28004800 0x7f4c28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c44974640 1 -- 172.21.15.16:0/2652059175 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c28004800 msgr2=0x7f4c28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c44974640 1 --2- 172.21.15.16:0/2652059175 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c28004800 0x7f4c28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c44974640 1 -- 172.21.15.16:0/2652059175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c280051c0 msgr2=0x7f4c2800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c44974640 1 --2- 172.21.15.16:0/2652059175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c280051c0 0x7f4c2800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c45175640 1 -- 172.21.15.16:0/2652059175 reap_dead start 2024-09-19T07:43:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.385+0000 7f4c44974640 1 -- 172.21.15.16:0/2652059175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c28003520 con 0x7f4c280039b0 2024-09-19T07:43:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.386+0000 7f4c44974640 1 --2- 172.21.15.16:0/2652059175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c280039b0 0x7f4c28003dd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009ac0 tx=0x7f4c28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4356fcf4fb5f4c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.386+0000 7f4c377fe640 1 -- 172.21.15.16:0/2652059175 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c036070 con 0x7f4c280039b0 2024-09-19T07:43:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.386+0000 7f4c377fe640 1 -- 172.21.15.16:0/2652059175 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c3c031400 con 0x7f4c280039b0 2024-09-19T07:43:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.386+0000 7f4c377fe640 1 -- 172.21.15.16:0/2652059175 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c009690 con 0x7f4c280039b0 2024-09-19T07:43:04.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.386+0000 7f4c46bd8640 1 -- 172.21.15.16:0/2652059175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c280039b0 msgr2=0x7f4c28003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:04.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.386+0000 7f4c46bd8640 1 --2- 172.21.15.16:0/2652059175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c280039b0 0x7f4c28003dd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009ac0 tx=0x7f4c28005830 comp rx=0 tx=0).stop 2024-09-19T07:43:04.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.387+0000 7f4c46bd8640 1 -- 172.21.15.16:0/2652059175 shutdown_connections 2024-09-19T07:43:04.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.387+0000 7f4c46bd8640 1 --2- 172.21.15.16:0/2652059175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c280039b0 0x7f4c28003dd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.387+0000 7f4c46bd8640 1 -- 172.21.15.16:0/2652059175 >> 172.21.15.16:0/2652059175 conn(0x7f4c4009d770 msgr2=0x7f4c4008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:04.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.387+0000 7f4c46bd8640 1 -- 172.21.15.16:0/2652059175 shutdown_connections 2024-09-19T07:43:04.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.387+0000 7f4c46bd8640 1 -- 172.21.15.16:0/2652059175 wait complete. 2024-09-19T07:43:04.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.388+0000 7f4c46bd8640 1 Processor -- start 2024-09-19T07:43:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.388+0000 7f4c46bd8640 1 -- start start 2024-09-19T07:43:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.389+0000 7f4c46bd8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c280039b0 0x7f4c400b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.389+0000 7f4c46bd8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c401599d0 0x7f4c400b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.389+0000 7f4c46bd8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c400b2ee0 0x7f4c4015b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.389+0000 7f4c46bd8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c40162e00 con 0x7f4c280039b0 2024-09-19T07:43:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.389+0000 7f4c46bd8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4c40162c80 con 0x7f4c400b2ee0 2024-09-19T07:43:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.389+0000 7f4c46bd8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4c400d4820 con 0x7f4c401599d0 2024-09-19T07:43:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.389+0000 7f4c44974640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c280039b0 0x7f4c400b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.389+0000 7f4c44974640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c280039b0 0x7f4c400b20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51648/0 (socket says 172.21.15.16:51648) 2024-09-19T07:43:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.389+0000 7f4c44974640 1 -- 172.21.15.16:0/202959684 learned_addr learned my addr 172.21.15.16:0/202959684 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.389+0000 7f4c45175640 1 --2- 172.21.15.16:0/202959684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c400b2ee0 0x7f4c4015b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.389+0000 7f4c37fff640 1 --2- 172.21.15.16:0/202959684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c401599d0 0x7f4c400b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.389+0000 7f4c44974640 1 -- 172.21.15.16:0/202959684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c401599d0 msgr2=0x7f4c400b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.389+0000 7f4c44974640 1 --2- 172.21.15.16:0/202959684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c401599d0 0x7f4c400b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.389+0000 7f4c44974640 1 -- 172.21.15.16:0/202959684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c400b2ee0 msgr2=0x7f4c4015b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.389+0000 7f4c44974640 1 --2- 172.21.15.16:0/202959684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c400b2ee0 0x7f4c4015b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.389+0000 7f4c44974640 1 -- 172.21.15.16:0/202959684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c400b1020 con 0x7f4c280039b0 2024-09-19T07:43:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.389+0000 7f4c44974640 1 --2- 172.21.15.16:0/202959684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c280039b0 0x7f4c400b20c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009be0 tx=0x7f4c3c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.389+0000 7f4c357fa640 1 -- 172.21.15.16:0/202959684 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c03f020 con 0x7f4c280039b0 2024-09-19T07:43:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.389+0000 7f4c357fa640 1 -- 172.21.15.16:0/202959684 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c3c037040 con 0x7f4c280039b0 2024-09-19T07:43:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.390+0000 7f4c46bd8640 1 -- 172.21.15.16:0/202959684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c400b1300 con 0x7f4c280039b0 2024-09-19T07:43:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.390+0000 7f4c357fa640 1 -- 172.21.15.16:0/202959684 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c036070 con 0x7f4c280039b0 2024-09-19T07:43:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.390+0000 7f4c46bd8640 1 -- 172.21.15.16:0/202959684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c400b18c0 con 0x7f4c280039b0 2024-09-19T07:43:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.390+0000 7f4c46bd8640 1 -- 172.21.15.16:0/202959684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c400d4bb0 con 0x7f4c280039b0 2024-09-19T07:43:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.391+0000 7f4c357fa640 1 -- 172.21.15.16:0/202959684 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4c3c042020 con 0x7f4c280039b0 2024-09-19T07:43:04.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.391+0000 7f4c357fa640 1 --2- 172.21.15.16:0/202959684 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4c2c06c2e0 0x7f4c2c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.392+0000 7f4c357fa640 1 -- 172.21.15.16:0/202959684 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7f4c3c0ab0f0 con 0x7f4c280039b0 2024-09-19T07:43:04.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.392+0000 7f4c37fff640 1 --2- 172.21.15.16:0/202959684 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4c2c06c2e0 0x7f4c2c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.392+0000 7f4c37fff640 1 --2- 172.21.15.16:0/202959684 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4c2c06c2e0 0x7f4c2c06e7a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4c4006e5e0 tx=0x7f4c30009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:04.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.396+0000 7f4c357fa640 1 -- 172.21.15.16:0/202959684 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4c3c049030 con 0x7f4c280039b0 2024-09-19T07:43:04.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:04.572+0000 7f4c46bd8640 1 -- 172.21.15.16:0/202959684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f4c400c7d20 con 0x7f4c280039b0 2024-09-19T07:43:05.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.242+0000 7f4c357fa640 1 -- 172.21.15.16:0/202959684 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v20) ==== 171+0+0 (secure 0 0 0) 0x7f4c3c079540 con 0x7f4c280039b0 2024-09-19T07:43:05.243 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-19T07:43:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.245+0000 7f4c46bd8640 1 -- 172.21.15.16:0/202959684 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4c2c06c2e0 msgr2=0x7f4c2c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.245+0000 7f4c46bd8640 1 --2- 172.21.15.16:0/202959684 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4c2c06c2e0 0x7f4c2c06e7a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4c4006e5e0 tx=0x7f4c30009040 comp rx=0 tx=0).stop 2024-09-19T07:43:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.245+0000 7f4c46bd8640 1 -- 172.21.15.16:0/202959684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c280039b0 msgr2=0x7f4c400b20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.245+0000 7f4c46bd8640 1 --2- 172.21.15.16:0/202959684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c280039b0 0x7f4c400b20c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009be0 tx=0x7f4c3c004730 comp rx=0 tx=0).stop 2024-09-19T07:43:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.245+0000 7f4c46bd8640 1 -- 172.21.15.16:0/202959684 shutdown_connections 2024-09-19T07:43:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.245+0000 7f4c46bd8640 1 --2- 172.21.15.16:0/202959684 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4c2c06c2e0 0x7f4c2c06e7a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.245+0000 7f4c46bd8640 1 --2- 172.21.15.16:0/202959684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c400b2ee0 0x7f4c4015b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.245+0000 7f4c46bd8640 1 --2- 172.21.15.16:0/202959684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c401599d0 0x7f4c400b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.245+0000 7f4c46bd8640 1 --2- 172.21.15.16:0/202959684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c280039b0 0x7f4c400b20c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.246+0000 7f4c46bd8640 1 -- 172.21.15.16:0/202959684 >> 172.21.15.16:0/202959684 conn(0x7f4c4009d770 msgr2=0x7f4c4015e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.246+0000 7f4c46bd8640 1 -- 172.21.15.16:0/202959684 shutdown_connections 2024-09-19T07:43:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.246+0000 7f4c46bd8640 1 -- 172.21.15.16:0/202959684 wait complete. 2024-09-19T07:43:05.275 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T07:43:05.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.385+0000 7ff2ac6d0640 1 Processor -- start 2024-09-19T07:43:05.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.385+0000 7ff2ac6d0640 1 -- start start 2024-09-19T07:43:05.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.386+0000 7ff2ac6d0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2a415e2a0 0x7ff2a406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:05.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.386+0000 7ff2ac6d0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2a415e680 0x7ff2a40cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:05.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.386+0000 7ff2ac6d0640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff2a406d960 0x7ff2a40d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:05.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.386+0000 7ff2ac6d0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff2a40d1880 con 0x7ff2a415e2a0 2024-09-19T07:43:05.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.386+0000 7ff2ac6d0640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff2a40d1b60 con 0x7ff2a406d960 2024-09-19T07:43:05.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.386+0000 7ff2ac6d0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a40cf2d0 con 0x7ff2a415e680 2024-09-19T07:43:05.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.386+0000 7ff2aa46c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2a415e2a0 0x7ff2a406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:05.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.386+0000 7ff2aa46c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2a415e2a0 0x7ff2a406d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51660/0 (socket says 172.21.15.16:51660) 2024-09-19T07:43:05.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.386+0000 7ff2aa46c640 1 -- 172.21.15.16:0/1240969170 learned_addr learned my addr 172.21.15.16:0/1240969170 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:05.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.386+0000 7ff2aa46c640 1 -- 172.21.15.16:0/1240969170 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2a415e680 msgr2=0x7ff2a40cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.386+0000 7ff2aac6d640 1 --2- 172.21.15.16:0/1240969170 >> v2:172.21.15.146:3300/0 conn(0x7ff2a406d960 0x7ff2a40d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.386+0000 7ff2a9c6b640 1 --2- 172.21.15.16:0/1240969170 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2a415e680 0x7ff2a40cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.386+0000 7ff2aa46c640 1 --2- 172.21.15.16:0/1240969170 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2a415e680 0x7ff2a40cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.386+0000 7ff2aa46c640 1 -- 172.21.15.16:0/1240969170 >> v2:172.21.15.146:3300/0 conn(0x7ff2a406d960 msgr2=0x7ff2a40d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.386+0000 7ff2aa46c640 1 --2- 172.21.15.16:0/1240969170 >> v2:172.21.15.146:3300/0 conn(0x7ff2a406d960 0x7ff2a40d0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.386+0000 7ff2aa46c640 1 -- 172.21.15.16:0/1240969170 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a40cf5b0 con 0x7ff2a415e2a0 2024-09-19T07:43:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.387+0000 7ff2aa46c640 1 --2- 172.21.15.16:0/1240969170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a415e2a0 0x7ff2a406d420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff2a406cd80 tx=0x7ff29402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df0c12aba71ec54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.387+0000 7ff2a946a640 1 -- 172.21.15.16:0/1240969170 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294037070 con 0x7ff2a415e2a0 2024-09-19T07:43:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.387+0000 7ff2a946a640 1 -- 172.21.15.16:0/1240969170 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff294032440 con 0x7ff2a415e2a0 2024-09-19T07:43:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.387+0000 7ff2a946a640 1 -- 172.21.15.16:0/1240969170 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294032810 con 0x7ff2a415e2a0 2024-09-19T07:43:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.388+0000 7ff2ac6d0640 1 -- 172.21.15.16:0/1240969170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a415e2a0 msgr2=0x7ff2a406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.388+0000 7ff2ac6d0640 1 --2- 172.21.15.16:0/1240969170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a415e2a0 0x7ff2a406d420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff2a406cd80 tx=0x7ff29402a1e0 comp rx=0 tx=0).stop 2024-09-19T07:43:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.388+0000 7ff2ac6d0640 1 -- 172.21.15.16:0/1240969170 shutdown_connections 2024-09-19T07:43:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.388+0000 7ff2ac6d0640 1 --2- 172.21.15.16:0/1240969170 >> v2:172.21.15.146:3300/0 conn(0x7ff2a406d960 0x7ff2a40d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.388+0000 7ff2ac6d0640 1 --2- 172.21.15.16:0/1240969170 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2a415e680 0x7ff2a40cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.388+0000 7ff2ac6d0640 1 --2- 172.21.15.16:0/1240969170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a415e2a0 0x7ff2a406d420 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.388+0000 7ff2ac6d0640 1 -- 172.21.15.16:0/1240969170 >> 172.21.15.16:0/1240969170 conn(0x7ff2a409d760 msgr2=0x7ff2a408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.388+0000 7ff2ac6d0640 1 -- 172.21.15.16:0/1240969170 shutdown_connections 2024-09-19T07:43:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.388+0000 7ff2ac6d0640 1 -- 172.21.15.16:0/1240969170 wait complete. 2024-09-19T07:43:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.389+0000 7ff2ac6d0640 1 Processor -- start 2024-09-19T07:43:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.389+0000 7ff2ac6d0640 1 -- start start 2024-09-19T07:43:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.390+0000 7ff2ac6d0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2a406d960 0x7ff2a40cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.390+0000 7ff2ac6d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a415e2a0 0x7ff2a40b92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.390+0000 7ff2ac6d0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2a415e680 0x7ff2a40b9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.390+0000 7ff2ac6d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a40d44f0 con 0x7ff2a415e2a0 2024-09-19T07:43:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.390+0000 7ff2ac6d0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a40d4370 con 0x7ff2a415e680 2024-09-19T07:43:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.390+0000 7ff2ac6d0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a40d4670 con 0x7ff2a406d960 2024-09-19T07:43:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.390+0000 7ff2a9c6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a415e2a0 0x7ff2a40b92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.390+0000 7ff2a9c6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a415e2a0 0x7ff2a40b92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51664/0 (socket says 172.21.15.16:51664) 2024-09-19T07:43:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.390+0000 7ff2a9c6b640 1 -- 172.21.15.16:0/473078674 learned_addr learned my addr 172.21.15.16:0/473078674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.390+0000 7ff2aac6d640 1 --2- 172.21.15.16:0/473078674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2a415e680 0x7ff2a40b9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.390+0000 7ff2aa46c640 1 --2- 172.21.15.16:0/473078674 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2a406d960 0x7ff2a40cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.390+0000 7ff2a9c6b640 1 -- 172.21.15.16:0/473078674 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2a406d960 msgr2=0x7ff2a40cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.390+0000 7ff2a9c6b640 1 --2- 172.21.15.16:0/473078674 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2a406d960 0x7ff2a40cdf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.390+0000 7ff2a9c6b640 1 -- 172.21.15.16:0/473078674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2a415e680 msgr2=0x7ff2a40b9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.390+0000 7ff2a9c6b640 1 --2- 172.21.15.16:0/473078674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2a415e680 0x7ff2a40b9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.390+0000 7ff2a9c6b640 1 -- 172.21.15.16:0/473078674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a40b8260 con 0x7ff2a415e2a0 2024-09-19T07:43:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.390+0000 7ff2a9c6b640 1 --2- 172.21.15.16:0/473078674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a415e2a0 0x7ff2a40b92c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000c5c0 tx=0x7ff2a000ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.391+0000 7ff29affd640 1 -- 172.21.15.16:0/473078674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0015070 con 0x7ff2a415e2a0 2024-09-19T07:43:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.391+0000 7ff29affd640 1 -- 172.21.15.16:0/473078674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a0004220 con 0x7ff2a415e2a0 2024-09-19T07:43:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.391+0000 7ff29affd640 1 -- 172.21.15.16:0/473078674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0003610 con 0x7ff2a415e2a0 2024-09-19T07:43:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.391+0000 7ff2ac6d0640 1 -- 172.21.15.16:0/473078674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a40b8500 con 0x7ff2a415e2a0 2024-09-19T07:43:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.391+0000 7ff2ac6d0640 1 -- 172.21.15.16:0/473078674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a40b8ac0 con 0x7ff2a415e2a0 2024-09-19T07:43:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.392+0000 7ff2ac6d0640 1 -- 172.21.15.16:0/473078674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a40d4a40 con 0x7ff2a415e2a0 2024-09-19T07:43:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.392+0000 7ff29affd640 1 -- 172.21.15.16:0/473078674 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff2a00037b0 con 0x7ff2a415e2a0 2024-09-19T07:43:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.393+0000 7ff29affd640 1 --2- 172.21.15.16:0/473078674 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff28806c2b0 0x7ff28806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.393+0000 7ff29affd640 1 -- 172.21.15.16:0/473078674 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7ff2a008b060 con 0x7ff2a415e2a0 2024-09-19T07:43:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.393+0000 7ff2aa46c640 1 --2- 172.21.15.16:0/473078674 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff28806c2b0 0x7ff28806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.393+0000 7ff2aa46c640 1 --2- 172.21.15.16:0/473078674 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff28806c2b0 0x7ff28806e770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff294034000 tx=0x7ff29402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.397+0000 7ff29affd640 1 -- 172.21.15.16:0/473078674 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff2a0059790 con 0x7ff2a415e2a0 2024-09-19T07:43:05.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:05.608+0000 7ff2ac6d0640 1 -- 172.21.15.16:0/473078674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff2a406cf90 con 0x7ff2a415e2a0 2024-09-19T07:43:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.243+0000 7ff29affd640 1 -- 172.21.15.16:0/473078674 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7ff2a0059090 con 0x7ff2a415e2a0 2024-09-19T07:43:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T07:43:06.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.247+0000 7ff2ac6d0640 1 -- 172.21.15.16:0/473078674 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff28806c2b0 msgr2=0x7ff28806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:06.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.247+0000 7ff2ac6d0640 1 --2- 172.21.15.16:0/473078674 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff28806c2b0 0x7ff28806e770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff294034000 tx=0x7ff29402e000 comp rx=0 tx=0).stop 2024-09-19T07:43:06.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.247+0000 7ff2ac6d0640 1 -- 172.21.15.16:0/473078674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a415e2a0 msgr2=0x7ff2a40b92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:06.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.247+0000 7ff2ac6d0640 1 --2- 172.21.15.16:0/473078674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a415e2a0 0x7ff2a40b92c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000c5c0 tx=0x7ff2a000ca80 comp rx=0 tx=0).stop 2024-09-19T07:43:06.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.248+0000 7ff2ac6d0640 1 -- 172.21.15.16:0/473078674 shutdown_connections 2024-09-19T07:43:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.248+0000 7ff2ac6d0640 1 --2- 172.21.15.16:0/473078674 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff28806c2b0 0x7ff28806e770 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.248+0000 7ff2ac6d0640 1 --2- 172.21.15.16:0/473078674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2a415e680 0x7ff2a40b9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.248+0000 7ff2ac6d0640 1 --2- 172.21.15.16:0/473078674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a415e2a0 0x7ff2a40b92c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.248+0000 7ff2ac6d0640 1 --2- 172.21.15.16:0/473078674 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2a406d960 0x7ff2a40cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.248+0000 7ff2ac6d0640 1 -- 172.21.15.16:0/473078674 >> 172.21.15.16:0/473078674 conn(0x7ff2a409d760 msgr2=0x7ff2a40d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.249+0000 7ff2ac6d0640 1 -- 172.21.15.16:0/473078674 shutdown_connections 2024-09-19T07:43:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.249+0000 7ff2ac6d0640 1 -- 172.21.15.16:0/473078674 wait complete. 2024-09-19T07:43:06.282 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-19T07:43:06.282 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-19T07:43:06.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.392+0000 7f8f1eaaf640 1 Processor -- start 2024-09-19T07:43:06.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.392+0000 7f8f1eaaf640 1 -- start start 2024-09-19T07:43:06.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.393+0000 7f8f1eaaf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8f1815f0d0 0x7f8f1815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.393+0000 7f8f1eaaf640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8f180d0070 0x7f8f180d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.393+0000 7f8f1eaaf640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f180cf2b0 con 0x7f8f180d1940 2024-09-19T07:43:06.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.393+0000 7f8f1eaaf640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f8f180cf590 con 0x7f8f1815f0d0 2024-09-19T07:43:06.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.393+0000 7f8f1eaaf640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f8f180cf870 con 0x7f8f180d0070 2024-09-19T07:43:06.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.393+0000 7f8f1c84b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8f1815f0d0 0x7f8f1815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.393+0000 7f8f1c84b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8f1815f0d0 0x7f8f1815f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51674/0 (socket says 172.21.15.16:51674) 2024-09-19T07:43:06.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.393+0000 7f8f1c84b640 1 -- 172.21.15.16:0/3151832993 learned_addr learned my addr 172.21.15.16:0/3151832993 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:06.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.393+0000 7f8f17fff640 1 --2- 172.21.15.16:0/3151832993 >> v2:172.21.15.146:3300/0 conn(0x7f8f180d0070 0x7f8f180d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.393+0000 7f8f1c84b640 1 -- 172.21.15.16:0/3151832993 >> v2:172.21.15.146:3300/0 conn(0x7f8f180d0070 msgr2=0x7f8f180d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:06.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.393+0000 7f8f1c84b640 1 --2- 172.21.15.16:0/3151832993 >> v2:172.21.15.146:3300/0 conn(0x7f8f180d0070 0x7f8f180d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.393+0000 7f8f1c84b640 1 -- 172.21.15.16:0/3151832993 >> v1:172.21.15.146:6789/0 conn(0x7f8f180d1940 legacy=0x7f8f180d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:06.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.393+0000 7f8f1c84b640 1 -- 172.21.15.16:0/3151832993 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f180cfb50 con 0x7f8f1815f0d0 2024-09-19T07:43:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.394+0000 7f8f1c84b640 1 --2- 172.21.15.16:0/3151832993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f1815f0d0 0x7f8f1815f4b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8f180b6f40 tx=0x7f8f0802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0f3029b60a2d19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.394+0000 7f8f177fe640 1 -- 172.21.15.16:0/3151832993 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08037070 con 0x7f8f1815f0d0 2024-09-19T07:43:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.394+0000 7f8f177fe640 1 -- 172.21.15.16:0/3151832993 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f08032440 con 0x7f8f1815f0d0 2024-09-19T07:43:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.394+0000 7f8f177fe640 1 -- 172.21.15.16:0/3151832993 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08032810 con 0x7f8f1815f0d0 2024-09-19T07:43:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.395+0000 7f8f1eaaf640 1 -- 172.21.15.16:0/3151832993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f1815f0d0 msgr2=0x7f8f1815f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.395+0000 7f8f1eaaf640 1 --2- 172.21.15.16:0/3151832993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f1815f0d0 0x7f8f1815f4b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8f180b6f40 tx=0x7f8f0802a180 comp rx=0 tx=0).stop 2024-09-19T07:43:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.395+0000 7f8f1eaaf640 1 -- 172.21.15.16:0/3151832993 shutdown_connections 2024-09-19T07:43:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.395+0000 7f8f1eaaf640 1 --2- 172.21.15.16:0/3151832993 >> v2:172.21.15.146:3300/0 conn(0x7f8f180d0070 0x7f8f180d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.395+0000 7f8f1eaaf640 1 --2- 172.21.15.16:0/3151832993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f1815f0d0 0x7f8f1815f4b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.395+0000 7f8f1eaaf640 1 -- 172.21.15.16:0/3151832993 >> 172.21.15.16:0/3151832993 conn(0x7f8f1809d760 msgr2=0x7f8f1808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.395+0000 7f8f1eaaf640 1 -- 172.21.15.16:0/3151832993 shutdown_connections 2024-09-19T07:43:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.395+0000 7f8f1eaaf640 1 -- 172.21.15.16:0/3151832993 wait complete. 2024-09-19T07:43:06.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.396+0000 7f8f1eaaf640 1 Processor -- start 2024-09-19T07:43:06.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.396+0000 7f8f1eaaf640 1 -- start start 2024-09-19T07:43:06.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.397+0000 7f8f1eaaf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f180d0070 0x7f8f181ea000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.397+0000 7f8f1eaaf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f180d1940 0x7f8f1806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.397+0000 7f8f1eaaf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f1815f0d0 0x7f8f180c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.397+0000 7f8f1eaaf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f180d49c0 con 0x7f8f180d1940 2024-09-19T07:43:06.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.397+0000 7f8f1eaaf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8f180d4840 con 0x7f8f1815f0d0 2024-09-19T07:43:06.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.397+0000 7f8f1eaaf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8f180d4b40 con 0x7f8f180d0070 2024-09-19T07:43:06.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.397+0000 7f8f17fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f180d1940 0x7f8f1806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.397+0000 7f8f17fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f180d1940 0x7f8f1806d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51676/0 (socket says 172.21.15.16:51676) 2024-09-19T07:43:06.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.397+0000 7f8f17fff640 1 -- 172.21.15.16:0/3585203039 learned_addr learned my addr 172.21.15.16:0/3585203039 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:06.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.397+0000 7f8f17fff640 1 -- 172.21.15.16:0/3585203039 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f180d0070 msgr2=0x7f8f181ea000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:06.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.397+0000 7f8f17fff640 1 --2- 172.21.15.16:0/3585203039 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f180d0070 0x7f8f181ea000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.397+0000 7f8f17fff640 1 -- 172.21.15.16:0/3585203039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f1815f0d0 msgr2=0x7f8f180c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:06.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.397+0000 7f8f1d04c640 1 --2- 172.21.15.16:0/3585203039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f1815f0d0 0x7f8f180c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.397+0000 7f8f17fff640 1 --2- 172.21.15.16:0/3585203039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f1815f0d0 0x7f8f180c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.397+0000 7f8f17fff640 1 -- 172.21.15.16:0/3585203039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f181e8f60 con 0x7f8f180d1940 2024-09-19T07:43:06.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.397+0000 7f8f1d04c640 1 --2- 172.21.15.16:0/3585203039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f1815f0d0 0x7f8f180c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:06.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.397+0000 7f8f17fff640 1 --2- 172.21.15.16:0/3585203039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f180d1940 0x7f8f1806d4c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00d800 tx=0x7f8f0c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:06.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.398+0000 7f8f157fa640 1 -- 172.21.15.16:0/3585203039 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c00b6c0 con 0x7f8f180d1940 2024-09-19T07:43:06.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.398+0000 7f8f157fa640 1 -- 172.21.15.16:0/3585203039 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f0c019070 con 0x7f8f180d1940 2024-09-19T07:43:06.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.398+0000 7f8f1eaaf640 1 -- 172.21.15.16:0/3585203039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f181e9240 con 0x7f8f180d1940 2024-09-19T07:43:06.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.398+0000 7f8f157fa640 1 -- 172.21.15.16:0/3585203039 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c00ba80 con 0x7f8f180d1940 2024-09-19T07:43:06.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.398+0000 7f8f1eaaf640 1 -- 172.21.15.16:0/3585203039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f181e9800 con 0x7f8f180d1940 2024-09-19T07:43:06.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.399+0000 7f8f1eaaf640 1 -- 172.21.15.16:0/3585203039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f180d1400 con 0x7f8f180d1940 2024-09-19T07:43:06.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.399+0000 7f8f157fa640 1 -- 172.21.15.16:0/3585203039 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8f0c00bc20 con 0x7f8f180d1940 2024-09-19T07:43:06.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.400+0000 7f8f157fa640 1 --2- 172.21.15.16:0/3585203039 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8f0406c2b0 0x7f8f0406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.400+0000 7f8f157fa640 1 -- 172.21.15.16:0/3585203039 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8f0c08b870 con 0x7f8f180d1940 2024-09-19T07:43:06.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.400+0000 7f8f1c84b640 1 --2- 172.21.15.16:0/3585203039 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8f0406c2b0 0x7f8f0406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.401+0000 7f8f1c84b640 1 --2- 172.21.15.16:0/3585203039 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8f0406c2b0 0x7f8f0406e770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8f08034000 tx=0x7f8f0802c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.404+0000 7f8f157fa640 1 -- 172.21.15.16:0/3585203039 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8f0c01c360 con 0x7f8f180d1940 2024-09-19T07:43:06.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:06.592+0000 7f8f1eaaf640 1 -- 172.21.15.16:0/3585203039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f8f1806cf30 con 0x7f8f180d1940 2024-09-19T07:43:06.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:43:06.771 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:43:07.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.261+0000 7f8f157fa640 1 -- 172.21.15.16:0/3585203039 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v22) ==== 157+0+0 (secure 0 0 0) 0x7f8f0c05a070 con 0x7f8f180d1940 2024-09-19T07:43:07.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.264+0000 7f8f1eaaf640 1 -- 172.21.15.16:0/3585203039 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8f0406c2b0 msgr2=0x7f8f0406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:07.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.264+0000 7f8f1eaaf640 1 --2- 172.21.15.16:0/3585203039 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8f0406c2b0 0x7f8f0406e770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8f08034000 tx=0x7f8f0802c000 comp rx=0 tx=0).stop 2024-09-19T07:43:07.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.264+0000 7f8f1eaaf640 1 -- 172.21.15.16:0/3585203039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f180d1940 msgr2=0x7f8f1806d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:07.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.264+0000 7f8f1eaaf640 1 --2- 172.21.15.16:0/3585203039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f180d1940 0x7f8f1806d4c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00d800 tx=0x7f8f0c00dcc0 comp rx=0 tx=0).stop 2024-09-19T07:43:07.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.264+0000 7f8f1eaaf640 1 -- 172.21.15.16:0/3585203039 shutdown_connections 2024-09-19T07:43:07.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.264+0000 7f8f1eaaf640 1 --2- 172.21.15.16:0/3585203039 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8f0406c2b0 0x7f8f0406e770 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:07.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.264+0000 7f8f1eaaf640 1 --2- 172.21.15.16:0/3585203039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f1815f0d0 0x7f8f180c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:07.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.264+0000 7f8f1eaaf640 1 --2- 172.21.15.16:0/3585203039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f180d1940 0x7f8f1806d4c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:07.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.264+0000 7f8f1eaaf640 1 --2- 172.21.15.16:0/3585203039 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f180d0070 0x7f8f181ea000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:07.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.264+0000 7f8f1eaaf640 1 -- 172.21.15.16:0/3585203039 >> 172.21.15.16:0/3585203039 conn(0x7f8f1809d760 msgr2=0x7f8f1815aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:07.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.264+0000 7f8f1eaaf640 1 -- 172.21.15.16:0/3585203039 shutdown_connections 2024-09-19T07:43:07.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.265+0000 7f8f1eaaf640 1 -- 172.21.15.16:0/3585203039 wait complete. 2024-09-19T07:43:07.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T07:43:07.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.396+0000 7f7250bcf640 1 Processor -- start 2024-09-19T07:43:07.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.396+0000 7f7250bcf640 1 -- start start 2024-09-19T07:43:07.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f7250bcf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f724c0b51f0 0x7f724c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f7250bcf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f724c0b5740 con 0x7f724c162d50 2024-09-19T07:43:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f7250bcf640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f724c0d05b0 con 0x7f724c0b3c60 2024-09-19T07:43:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f7250bcf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f724c163130 con 0x7f724c0b51f0 2024-09-19T07:43:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f724a59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f724c162d50 0x7f724c0d29d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42392/0 (socket says 172.21.15.16:42392) 2024-09-19T07:43:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f724a59c640 1 -- 172.21.15.16:0/1958664799 learned_addr learned my addr 172.21.15.16:0/1958664799 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f7249d9b640 1 --2- 172.21.15.16:0/1958664799 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f724c0b51f0 0x7f724c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f724959a640 1 -- 172.21.15.16:0/1958664799 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1931044069 0 0) 0x7f724c0b5740 con 0x7f724c162d50 2024-09-19T07:43:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f724959a640 1 -- 172.21.15.16:0/1958664799 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7234003520 con 0x7f724c162d50 2024-09-19T07:43:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f724959a640 1 -- 172.21.15.16:0/1958664799 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1325353321 0 0) 0x7f724c0d05b0 con 0x7f724c0b3c60 2024-09-19T07:43:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f724959a640 1 -- 172.21.15.16:0/1958664799 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f724c0b5740 con 0x7f724c0b3c60 2024-09-19T07:43:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f724959a640 1 -- 172.21.15.16:0/1958664799 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 751351351 0 0) 0x7f7234003520 con 0x7f724c162d50 2024-09-19T07:43:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f724959a640 1 -- 172.21.15.16:0/1958664799 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f724c0b51f0 msgr2=0x7f724c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f724959a640 1 --2- 172.21.15.16:0/1958664799 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f724c0b51f0 0x7f724c0cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f724959a640 1 -- 172.21.15.16:0/1958664799 >> v1:172.21.15.146:6789/0 conn(0x7f724c0b3c60 legacy=0x7f724c0d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f724959a640 1 -- 172.21.15.16:0/1958664799 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724c0d0890 con 0x7f724c162d50 2024-09-19T07:43:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f724959a640 1 -- 172.21.15.16:0/1958664799 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f723c001670 con 0x7f724c162d50 2024-09-19T07:43:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f724959a640 1 -- 172.21.15.16:0/1958664799 >> v1:172.21.15.16:6789/0 conn(0x7f724c162d50 legacy=0x7f724c0d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f724959a640 1 --2- 172.21.15.16:0/1958664799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7234003b70 0x7f7234003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f724959a640 1 --2- 172.21.15.16:0/1958664799 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f724c0b51f0 0x7f7234004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f724959a640 1 --2- 172.21.15.16:0/1958664799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7234004d40 0x7f72340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f724959a640 1 -- 172.21.15.16:0/1958664799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f723400bac0 con 0x7f7234004d40 2024-09-19T07:43:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f724959a640 1 -- 172.21.15.16:0/1958664799 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f723400bc70 con 0x7f7234003b70 2024-09-19T07:43:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.397+0000 7f724959a640 1 -- 172.21.15.16:0/1958664799 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f723400be70 con 0x7f724c0b51f0 2024-09-19T07:43:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.398+0000 7f724ad9d640 1 --2- 172.21.15.16:0/1958664799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7234004d40 0x7f72340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.398+0000 7f7249d9b640 1 --2- 172.21.15.16:0/1958664799 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f724c0b51f0 0x7f7234004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.398+0000 7f724a59c640 1 --2- 172.21.15.16:0/1958664799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7234003b70 0x7f7234003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.398+0000 7f724ad9d640 1 -- 172.21.15.16:0/1958664799 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f724c0b51f0 msgr2=0x7f7234004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.398+0000 7f724ad9d640 1 --2- 172.21.15.16:0/1958664799 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f724c0b51f0 0x7f7234004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.398+0000 7f724ad9d640 1 -- 172.21.15.16:0/1958664799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7234003b70 msgr2=0x7f7234003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.398+0000 7f724ad9d640 1 --2- 172.21.15.16:0/1958664799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7234003b70 0x7f7234003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.398+0000 7f724ad9d640 1 -- 172.21.15.16:0/1958664799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7234003520 con 0x7f7234004d40 2024-09-19T07:43:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.399+0000 7f724ad9d640 1 --2- 172.21.15.16:0/1958664799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7234004d40 0x7f72340051c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7244001fd0 tx=0x7f7234005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745a7d01f021e82c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.399+0000 7f724959a640 1 -- 172.21.15.16:0/1958664799 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7244001bf0 con 0x7f7234004d40 2024-09-19T07:43:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.399+0000 7f724959a640 1 -- 172.21.15.16:0/1958664799 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7244003b70 con 0x7f7234004d40 2024-09-19T07:43:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.399+0000 7f724959a640 1 -- 172.21.15.16:0/1958664799 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72440050d0 con 0x7f7234004d40 2024-09-19T07:43:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.399+0000 7f7250bcf640 1 -- 172.21.15.16:0/1958664799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7234004d40 msgr2=0x7f72340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.399+0000 7f7250bcf640 1 --2- 172.21.15.16:0/1958664799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7234004d40 0x7f72340051c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7244001fd0 tx=0x7f7234005830 comp rx=0 tx=0).stop 2024-09-19T07:43:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.399+0000 7f724ad9d640 1 -- 172.21.15.16:0/1958664799 reap_dead start 2024-09-19T07:43:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.399+0000 7f7250bcf640 1 -- 172.21.15.16:0/1958664799 shutdown_connections 2024-09-19T07:43:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.399+0000 7f7250bcf640 1 -- 172.21.15.16:0/1958664799 >> 172.21.15.16:0/1958664799 conn(0x7f724c08bf80 msgr2=0x7f724c0d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.399+0000 7f7250bcf640 1 -- 172.21.15.16:0/1958664799 shutdown_connections 2024-09-19T07:43:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.399+0000 7f7250bcf640 1 -- 172.21.15.16:0/1958664799 wait complete. 2024-09-19T07:43:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.401+0000 7f7250bcf640 1 Processor -- start 2024-09-19T07:43:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.401+0000 7f7250bcf640 1 -- start start 2024-09-19T07:43:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f7250bcf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7234004d40 0x7f724c153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f7250bcf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f724c15e040 0x7f724c0b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f7250bcf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f724c0cd0b0 0x7f724c0cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f7250bcf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f724c0d3260 con 0x7f724c0cd0b0 2024-09-19T07:43:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f7250bcf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f724c0d30e0 con 0x7f7234004d40 2024-09-19T07:43:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f7250bcf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f724c08c360 con 0x7f724c15e040 2024-09-19T07:43:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f724ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f724c0cd0b0 0x7f724c0cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f724ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f724c0cd0b0 0x7f724c0cd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51694/0 (socket says 172.21.15.16:51694) 2024-09-19T07:43:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f724ad9d640 1 -- 172.21.15.16:0/2547742740 learned_addr learned my addr 172.21.15.16:0/2547742740 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f724a59c640 1 --2- 172.21.15.16:0/2547742740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7234004d40 0x7f724c153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f7249d9b640 1 --2- 172.21.15.16:0/2547742740 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f724c15e040 0x7f724c0b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f724ad9d640 1 -- 172.21.15.16:0/2547742740 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f724c15e040 msgr2=0x7f724c0b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f724ad9d640 1 --2- 172.21.15.16:0/2547742740 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f724c15e040 0x7f724c0b71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f724ad9d640 1 -- 172.21.15.16:0/2547742740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7234004d40 msgr2=0x7f724c153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f724ad9d640 1 --2- 172.21.15.16:0/2547742740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7234004d40 0x7f724c153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f724ad9d640 1 -- 172.21.15.16:0/2547742740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724c0cda70 con 0x7f724c0cd0b0 2024-09-19T07:43:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f724a59c640 1 --2- 172.21.15.16:0/2547742740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7234004d40 0x7f724c153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f724ad9d640 1 --2- 172.21.15.16:0/2547742740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f724c0cd0b0 0x7f724c0cd530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7244005040 tx=0x7f7244001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f723affd640 1 -- 172.21.15.16:0/2547742740 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7244001bf0 con 0x7f724c0cd0b0 2024-09-19T07:43:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f723affd640 1 -- 172.21.15.16:0/2547742740 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7244043070 con 0x7f724c0cd0b0 2024-09-19T07:43:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f723affd640 1 -- 172.21.15.16:0/2547742740 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7244005610 con 0x7f724c0cd0b0 2024-09-19T07:43:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f7250bcf640 1 -- 172.21.15.16:0/2547742740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f724c153f80 con 0x7f724c0cd0b0 2024-09-19T07:43:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.402+0000 7f7250bcf640 1 -- 172.21.15.16:0/2547742740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f724c0b79b0 con 0x7f724c0cd0b0 2024-09-19T07:43:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.404+0000 7f7250bcf640 1 -- 172.21.15.16:0/2547742740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f724c0b4110 con 0x7f724c0cd0b0 2024-09-19T07:43:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.404+0000 7f723affd640 1 -- 172.21.15.16:0/2547742740 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f724403a040 con 0x7f724c0cd0b0 2024-09-19T07:43:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.404+0000 7f723affd640 1 --2- 172.21.15.16:0/2547742740 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f722c06c2b0 0x7f722c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.404+0000 7f723affd640 1 -- 172.21.15.16:0/2547742740 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6007+0+0 (secure 0 0 0) 0x7f72440ab9c0 con 0x7f724c0cd0b0 2024-09-19T07:43:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.405+0000 7f724a59c640 1 --2- 172.21.15.16:0/2547742740 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f722c06c2b0 0x7f722c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.405+0000 7f724a59c640 1 --2- 172.21.15.16:0/2547742740 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f722c06c2b0 0x7f722c06e770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f723c001eb0 tx=0x7f723c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.409+0000 7f723affd640 1 -- 172.21.15.16:0/2547742740 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f72440ade20 con 0x7f724c0cd0b0 2024-09-19T07:43:07.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:07.585+0000 7f7250bcf640 1 -- 172.21.15.16:0/2547742740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f724c0d0b90 con 0x7f724c0cd0b0 2024-09-19T07:43:09.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.265+0000 7f723affd640 1 -- 172.21.15.16:0/2547742740 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v24) ==== 228+0+0 (secure 0 0 0) 0x7f724c153f80 con 0x7f724c0cd0b0 2024-09-19T07:43:09.266 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' created 2024-09-19T07:43:09.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.268+0000 7f7250bcf640 1 -- 172.21.15.16:0/2547742740 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f722c06c2b0 msgr2=0x7f722c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:09.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.268+0000 7f7250bcf640 1 --2- 172.21.15.16:0/2547742740 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f722c06c2b0 0x7f722c06e770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f723c001eb0 tx=0x7f723c009040 comp rx=0 tx=0).stop 2024-09-19T07:43:09.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.268+0000 7f7250bcf640 1 -- 172.21.15.16:0/2547742740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f724c0cd0b0 msgr2=0x7f724c0cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:09.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.268+0000 7f7250bcf640 1 --2- 172.21.15.16:0/2547742740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f724c0cd0b0 0x7f724c0cd530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7244005040 tx=0x7f7244001650 comp rx=0 tx=0).stop 2024-09-19T07:43:09.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.269+0000 7f7250bcf640 1 -- 172.21.15.16:0/2547742740 shutdown_connections 2024-09-19T07:43:09.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.269+0000 7f7250bcf640 1 --2- 172.21.15.16:0/2547742740 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f722c06c2b0 0x7f722c06e770 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:09.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.269+0000 7f7250bcf640 1 --2- 172.21.15.16:0/2547742740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f724c0cd0b0 0x7f724c0cd530 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:09.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.269+0000 7f7250bcf640 1 --2- 172.21.15.16:0/2547742740 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f724c15e040 0x7f724c0b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:09.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.269+0000 7f7250bcf640 1 --2- 172.21.15.16:0/2547742740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7234004d40 0x7f724c153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:09.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.269+0000 7f7250bcf640 1 -- 172.21.15.16:0/2547742740 >> 172.21.15.16:0/2547742740 conn(0x7f724c08bf80 msgr2=0x7f724c06e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:09.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.269+0000 7f7250bcf640 1 -- 172.21.15.16:0/2547742740 shutdown_connections 2024-09-19T07:43:09.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.269+0000 7f7250bcf640 1 -- 172.21.15.16:0/2547742740 wait complete. 2024-09-19T07:43:09.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-19T07:43:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.401+0000 7fa178850640 1 Processor -- start 2024-09-19T07:43:09.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.401+0000 7fa178850640 1 -- start start 2024-09-19T07:43:09.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.402+0000 7fa178850640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1740d4840 0x7fa17415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:09.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.402+0000 7fa178850640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1740d1b10 con 0x7fa17415f0d0 2024-09-19T07:43:09.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.402+0000 7fa178850640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1740cf2b0 con 0x7fa1740d0070 2024-09-19T07:43:09.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.402+0000 7fa178850640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa1740cf590 con 0x7fa1740d4840 2024-09-19T07:43:09.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.402+0000 7fa17259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa17415f0d0 0x7fa1740d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42394/0 (socket says 172.21.15.16:42394) 2024-09-19T07:43:09.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.402+0000 7fa17259c640 1 -- 172.21.15.16:0/1975996442 learned_addr learned my addr 172.21.15.16:0/1975996442 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:09.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.402+0000 7fa172d9d640 1 --2- 172.21.15.16:0/1975996442 >> v2:172.21.15.16:3300/0 conn(0x7fa1740d4840 0x7fa17415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:09.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.402+0000 7fa172d9d640 1 -- 172.21.15.16:0/1975996442 >> v1:172.21.15.146:6789/0 conn(0x7fa1740d0070 legacy=0x7fa1740d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:09.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.402+0000 7fa172d9d640 1 -- 172.21.15.16:0/1975996442 >> v1:172.21.15.16:6789/0 conn(0x7fa17415f0d0 legacy=0x7fa1740d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:09.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.402+0000 7fa172d9d640 1 -- 172.21.15.16:0/1975996442 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1740cf870 con 0x7fa1740d4840 2024-09-19T07:43:09.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.403+0000 7fa172d9d640 1 --2- 172.21.15.16:0/1975996442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1740d4840 0x7fa17415f4b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009dc0 tx=0x7fa16c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39a5bd887a2f78 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:09.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.403+0000 7fa17159a640 1 -- 172.21.15.16:0/1975996442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c007740 con 0x7fa1740d4840 2024-09-19T07:43:09.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.403+0000 7fa17159a640 1 -- 172.21.15.16:0/1975996442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa16c0078e0 con 0x7fa1740d4840 2024-09-19T07:43:09.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.403+0000 7fa17159a640 1 -- 172.21.15.16:0/1975996442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c007cb0 con 0x7fa1740d4840 2024-09-19T07:43:09.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.404+0000 7fa178850640 1 -- 172.21.15.16:0/1975996442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1740d4840 msgr2=0x7fa17415f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:09.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.404+0000 7fa178850640 1 --2- 172.21.15.16:0/1975996442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1740d4840 0x7fa17415f4b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009dc0 tx=0x7fa16c02a6a0 comp rx=0 tx=0).stop 2024-09-19T07:43:09.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.404+0000 7fa178850640 1 -- 172.21.15.16:0/1975996442 shutdown_connections 2024-09-19T07:43:09.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.404+0000 7fa178850640 1 --2- 172.21.15.16:0/1975996442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1740d4840 0x7fa17415f4b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:09.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.404+0000 7fa178850640 1 -- 172.21.15.16:0/1975996442 >> 172.21.15.16:0/1975996442 conn(0x7fa17409d760 msgr2=0x7fa17408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:09.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.404+0000 7fa178850640 1 -- 172.21.15.16:0/1975996442 shutdown_connections 2024-09-19T07:43:09.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.404+0000 7fa178850640 1 -- 172.21.15.16:0/1975996442 wait complete. 2024-09-19T07:43:09.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.405+0000 7fa178850640 1 Processor -- start 2024-09-19T07:43:09.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.405+0000 7fa178850640 1 -- start start 2024-09-19T07:43:09.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.406+0000 7fa178850640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1740d0070 0x7fa17415b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:09.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.406+0000 7fa178850640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1740d4840 0x7fa174161270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:09.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.406+0000 7fa178850640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa17415f0d0 0x7fa1740b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:09.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.406+0000 7fa178850640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1740b2f70 con 0x7fa17415f0d0 2024-09-19T07:43:09.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.406+0000 7fa178850640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa1740cef60 con 0x7fa1740d0070 2024-09-19T07:43:09.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.406+0000 7fa178850640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa1740b30f0 con 0x7fa1740d4840 2024-09-19T07:43:09.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.406+0000 7fa172d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa17415f0d0 0x7fa1740b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:09.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.406+0000 7fa172d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa17415f0d0 0x7fa1740b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51706/0 (socket says 172.21.15.16:51706) 2024-09-19T07:43:09.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.406+0000 7fa172d9d640 1 -- 172.21.15.16:0/1412176802 learned_addr learned my addr 172.21.15.16:0/1412176802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:09.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.407+0000 7fa17259c640 1 --2- 172.21.15.16:0/1412176802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1740d0070 0x7fa17415b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:09.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.407+0000 7fa171d9b640 1 --2- 172.21.15.16:0/1412176802 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1740d4840 0x7fa174161270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:09.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.407+0000 7fa172d9d640 1 -- 172.21.15.16:0/1412176802 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1740d4840 msgr2=0x7fa174161270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:09.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.407+0000 7fa172d9d640 1 --2- 172.21.15.16:0/1412176802 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1740d4840 0x7fa174161270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:09.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.407+0000 7fa172d9d640 1 -- 172.21.15.16:0/1412176802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1740d0070 msgr2=0x7fa17415b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:09.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.407+0000 7fa172d9d640 1 --2- 172.21.15.16:0/1412176802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1740d0070 0x7fa17415b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:09.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.407+0000 7fa172d9d640 1 -- 172.21.15.16:0/1412176802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17415bb00 con 0x7fa17415f0d0 2024-09-19T07:43:09.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.407+0000 7fa171d9b640 1 --2- 172.21.15.16:0/1412176802 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1740d4840 0x7fa174161270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:09.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.407+0000 7fa172d9d640 1 --2- 172.21.15.16:0/1412176802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa17415f0d0 0x7fa1740b4630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009ee0 tx=0x7fa16c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:09.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.407+0000 7fa162ffd640 1 -- 172.21.15.16:0/1412176802 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c040070 con 0x7fa17415f0d0 2024-09-19T07:43:09.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.407+0000 7fa162ffd640 1 -- 172.21.15.16:0/1412176802 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa16c031bc0 con 0x7fa17415f0d0 2024-09-19T07:43:09.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.407+0000 7fa178850640 1 -- 172.21.15.16:0/1412176802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1740cd790 con 0x7fa17415f0d0 2024-09-19T07:43:09.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.407+0000 7fa162ffd640 1 -- 172.21.15.16:0/1412176802 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c045460 con 0x7fa17415f0d0 2024-09-19T07:43:09.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.407+0000 7fa178850640 1 -- 172.21.15.16:0/1412176802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1740cdd50 con 0x7fa17415f0d0 2024-09-19T07:43:09.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.408+0000 7fa178850640 1 -- 172.21.15.16:0/1412176802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa17415f4b0 con 0x7fa17415f0d0 2024-09-19T07:43:09.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.409+0000 7fa162ffd640 1 -- 172.21.15.16:0/1412176802 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa16c039040 con 0x7fa17415f0d0 2024-09-19T07:43:09.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.409+0000 7fa162ffd640 1 --2- 172.21.15.16:0/1412176802 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa15406c2b0 0x7fa15406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:09.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.409+0000 7fa162ffd640 1 -- 172.21.15.16:0/1412176802 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7fa16c0ad160 con 0x7fa17415f0d0 2024-09-19T07:43:09.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.409+0000 7fa17259c640 1 --2- 172.21.15.16:0/1412176802 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa15406c2b0 0x7fa15406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:09.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.410+0000 7fa17259c640 1 --2- 172.21.15.16:0/1412176802 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa15406c2b0 0x7fa15406e770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa164005880 tx=0x7fa164009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:09.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.414+0000 7fa162ffd640 1 -- 172.21.15.16:0/1412176802 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa16c07b5b0 con 0x7fa17415f0d0 2024-09-19T07:43:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:09.588+0000 7fa178850640 1 -- 172.21.15.16:0/1412176802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fa17406cf30 con 0x7fa17415f0d0 2024-09-19T07:43:10.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.267+0000 7fa162ffd640 1 -- 172.21.15.16:0/1412176802 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v25) ==== 169+0+0 (secure 0 0 0) 0x7fa16c07aeb0 con 0x7fa17415f0d0 2024-09-19T07:43:10.267 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 4 allow_ec_overwrites to true 2024-09-19T07:43:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.271+0000 7fa178850640 1 -- 172.21.15.16:0/1412176802 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa15406c2b0 msgr2=0x7fa15406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.272+0000 7fa178850640 1 --2- 172.21.15.16:0/1412176802 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa15406c2b0 0x7fa15406e770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa164005880 tx=0x7fa164009040 comp rx=0 tx=0).stop 2024-09-19T07:43:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.272+0000 7fa178850640 1 -- 172.21.15.16:0/1412176802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa17415f0d0 msgr2=0x7fa1740b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.272+0000 7fa178850640 1 --2- 172.21.15.16:0/1412176802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa17415f0d0 0x7fa1740b4630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009ee0 tx=0x7fa16c004a40 comp rx=0 tx=0).stop 2024-09-19T07:43:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.272+0000 7fa178850640 1 -- 172.21.15.16:0/1412176802 shutdown_connections 2024-09-19T07:43:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.272+0000 7fa178850640 1 --2- 172.21.15.16:0/1412176802 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa15406c2b0 0x7fa15406e770 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.272+0000 7fa178850640 1 --2- 172.21.15.16:0/1412176802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa17415f0d0 0x7fa1740b4630 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.272+0000 7fa178850640 1 --2- 172.21.15.16:0/1412176802 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1740d4840 0x7fa174161270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.272+0000 7fa178850640 1 --2- 172.21.15.16:0/1412176802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1740d0070 0x7fa17415b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.272+0000 7fa178850640 1 -- 172.21.15.16:0/1412176802 >> 172.21.15.16:0/1412176802 conn(0x7fa17409d760 msgr2=0x7fa17406ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.272+0000 7fa178850640 1 -- 172.21.15.16:0/1412176802 shutdown_connections 2024-09-19T07:43:10.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.272+0000 7fa178850640 1 -- 172.21.15.16:0/1412176802 wait complete. 2024-09-19T07:43:10.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-19T07:43:10.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.410+0000 7fc1e30e6640 1 Processor -- start 2024-09-19T07:43:10.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.410+0000 7fc1e30e6640 1 -- start start 2024-09-19T07:43:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.411+0000 7fc1e30e6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc1dc15b120 0x7fc1dc06f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.411+0000 7fc1e30e6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc1dc06dec0 0x7fc1dc06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.411+0000 7fc1e30e6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1dc15b500 con 0x7fc1dc0d04b0 2024-09-19T07:43:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.411+0000 7fc1e30e6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc1dc06e940 con 0x7fc1dc06dec0 2024-09-19T07:43:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.411+0000 7fc1e30e6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc1dc06ec20 con 0x7fc1dc15b120 2024-09-19T07:43:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.411+0000 7fc1dbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc1dc06dec0 0x7fc1dc06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.411+0000 7fc1dbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc1dc06dec0 0x7fc1dc06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51722/0 (socket says 172.21.15.16:51722) 2024-09-19T07:43:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.411+0000 7fc1dbfff640 1 -- 172.21.15.16:0/3386566884 learned_addr learned my addr 172.21.15.16:0/3386566884 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.411+0000 7fc1e0e82640 1 --2- 172.21.15.16:0/3386566884 >> v2:172.21.15.146:3300/0 conn(0x7fc1dc15b120 0x7fc1dc06f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.411+0000 7fc1dbfff640 1 -- 172.21.15.16:0/3386566884 >> v2:172.21.15.146:3300/0 conn(0x7fc1dc15b120 msgr2=0x7fc1dc06f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:10.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.411+0000 7fc1dbfff640 1 --2- 172.21.15.16:0/3386566884 >> v2:172.21.15.146:3300/0 conn(0x7fc1dc15b120 0x7fc1dc06f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:10.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.411+0000 7fc1dbfff640 1 -- 172.21.15.16:0/3386566884 >> v1:172.21.15.146:6789/0 conn(0x7fc1dc0d04b0 legacy=0x7fc1dc0c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:10.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.411+0000 7fc1dbfff640 1 -- 172.21.15.16:0/3386566884 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1dc06ef00 con 0x7fc1dc06dec0 2024-09-19T07:43:10.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.412+0000 7fc1dbfff640 1 --2- 172.21.15.16:0/3386566884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1dc06dec0 0x7fc1dc06e330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc06cb50 tx=0x7fc1cc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abb56a4177e9cb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:10.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.412+0000 7fc1db7fe640 1 -- 172.21.15.16:0/3386566884 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc036070 con 0x7fc1dc06dec0 2024-09-19T07:43:10.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.412+0000 7fc1db7fe640 1 -- 172.21.15.16:0/3386566884 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1cc031600 con 0x7fc1dc06dec0 2024-09-19T07:43:10.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.412+0000 7fc1db7fe640 1 -- 172.21.15.16:0/3386566884 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0319d0 con 0x7fc1dc06dec0 2024-09-19T07:43:10.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.412+0000 7fc1e30e6640 1 -- 172.21.15.16:0/3386566884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1dc06dec0 msgr2=0x7fc1dc06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:10.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.412+0000 7fc1e30e6640 1 --2- 172.21.15.16:0/3386566884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1dc06dec0 0x7fc1dc06e330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc06cb50 tx=0x7fc1cc02a6f0 comp rx=0 tx=0).stop 2024-09-19T07:43:10.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.413+0000 7fc1e30e6640 1 -- 172.21.15.16:0/3386566884 shutdown_connections 2024-09-19T07:43:10.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.413+0000 7fc1e30e6640 1 --2- 172.21.15.16:0/3386566884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1dc06dec0 0x7fc1dc06e330 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:10.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.413+0000 7fc1e30e6640 1 --2- 172.21.15.16:0/3386566884 >> v2:172.21.15.146:3300/0 conn(0x7fc1dc15b120 0x7fc1dc06f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:10.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.413+0000 7fc1e30e6640 1 -- 172.21.15.16:0/3386566884 >> 172.21.15.16:0/3386566884 conn(0x7fc1dc06bdf0 msgr2=0x7fc1dc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:10.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.413+0000 7fc1e30e6640 1 -- 172.21.15.16:0/3386566884 shutdown_connections 2024-09-19T07:43:10.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.413+0000 7fc1e30e6640 1 -- 172.21.15.16:0/3386566884 wait complete. 2024-09-19T07:43:10.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.414+0000 7fc1e30e6640 1 Processor -- start 2024-09-19T07:43:10.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.414+0000 7fc1e30e6640 1 -- start start 2024-09-19T07:43:10.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.415+0000 7fc1e30e6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1dc06dec0 0x7fc1dc154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:10.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.415+0000 7fc1e30e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1dc0d04b0 0x7fc1dc0b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:10.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.415+0000 7fc1e30e6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1dc15b120 0x7fc1dc0c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:10.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.415+0000 7fc1e30e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc0ce060 con 0x7fc1dc0d04b0 2024-09-19T07:43:10.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.415+0000 7fc1e30e6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc06f4c0 con 0x7fc1dc06dec0 2024-09-19T07:43:10.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.415+0000 7fc1e30e6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc1dc0d0900 con 0x7fc1dc15b120 2024-09-19T07:43:10.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.415+0000 7fc1dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1dc0d04b0 0x7fc1dc0b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:10.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.415+0000 7fc1dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1dc0d04b0 0x7fc1dc0b0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51730/0 (socket says 172.21.15.16:51730) 2024-09-19T07:43:10.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.415+0000 7fc1dbfff640 1 -- 172.21.15.16:0/3167553725 learned_addr learned my addr 172.21.15.16:0/3167553725 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:10.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.415+0000 7fc1e0e82640 1 --2- 172.21.15.16:0/3167553725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1dc06dec0 0x7fc1dc154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:10.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.415+0000 7fc1e1683640 1 --2- 172.21.15.16:0/3167553725 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1dc15b120 0x7fc1dc0c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:10.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.415+0000 7fc1dbfff640 1 -- 172.21.15.16:0/3167553725 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1dc15b120 msgr2=0x7fc1dc0c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:10.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.415+0000 7fc1dbfff640 1 --2- 172.21.15.16:0/3167553725 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1dc15b120 0x7fc1dc0c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:10.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.415+0000 7fc1dbfff640 1 -- 172.21.15.16:0/3167553725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1dc06dec0 msgr2=0x7fc1dc154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:10.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.415+0000 7fc1dbfff640 1 --2- 172.21.15.16:0/3167553725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1dc06dec0 0x7fc1dc154ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:10.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.415+0000 7fc1dbfff640 1 -- 172.21.15.16:0/3167553725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1dc0b1830 con 0x7fc1dc0d04b0 2024-09-19T07:43:10.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.415+0000 7fc1dbfff640 1 --2- 172.21.15.16:0/3167553725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1dc0d04b0 0x7fc1dc0b0f50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc000f80 tx=0x7fc1cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:10.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.416+0000 7fc1d97fa640 1 -- 172.21.15.16:0/3167553725 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc036050 con 0x7fc1dc0d04b0 2024-09-19T07:43:10.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.416+0000 7fc1d97fa640 1 -- 172.21.15.16:0/3167553725 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1cc042070 con 0x7fc1dc0d04b0 2024-09-19T07:43:10.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.416+0000 7fc1e30e6640 1 -- 172.21.15.16:0/3167553725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1dc0c5c50 con 0x7fc1dc0d04b0 2024-09-19T07:43:10.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.416+0000 7fc1d97fa640 1 -- 172.21.15.16:0/3167553725 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc009510 con 0x7fc1dc0d04b0 2024-09-19T07:43:10.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.416+0000 7fc1e30e6640 1 -- 172.21.15.16:0/3167553725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1dc1551e0 con 0x7fc1dc0d04b0 2024-09-19T07:43:10.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.417+0000 7fc1e30e6640 1 -- 172.21.15.16:0/3167553725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1dc0d0cd0 con 0x7fc1dc0d04b0 2024-09-19T07:43:10.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.417+0000 7fc1d97fa640 1 -- 172.21.15.16:0/3167553725 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc1cc007c70 con 0x7fc1dc0d04b0 2024-09-19T07:43:10.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.417+0000 7fc1d97fa640 1 --2- 172.21.15.16:0/3167553725 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc1bc06c2b0 0x7fc1bc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:10.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.418+0000 7fc1d97fa640 1 -- 172.21.15.16:0/3167553725 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6500+0+0 (secure 0 0 0) 0x7fc1cc0ac540 con 0x7fc1dc0d04b0 2024-09-19T07:43:10.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.418+0000 7fc1e0e82640 1 --2- 172.21.15.16:0/3167553725 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc1bc06c2b0 0x7fc1bc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:10.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.418+0000 7fc1e0e82640 1 --2- 172.21.15.16:0/3167553725 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc1bc06c2b0 0x7fc1bc06e770 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc1d40079a0 tx=0x7fc1d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:10.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.422+0000 7fc1d97fa640 1 -- 172.21.15.16:0/3167553725 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc1cc07a340 con 0x7fc1dc0d04b0 2024-09-19T07:43:10.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:10.640+0000 7fc1e30e6640 1 -- 172.21.15.16:0/3167553725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fc1dc06e510 con 0x7fc1dc0d04b0 2024-09-19T07:43:11.233 INFO:tasks.ceph.mds.d.smithi146.stderr:2024-09-19T07:43:11.232+0000 7fa2cc324640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T07:43:11.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.312+0000 7fc1d97fa640 1 -- 172.21.15.16:0/3167553725 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) ==== 137+0+0 (secure 0 0 0) 0x7fc1cc07e640 con 0x7fc1dc0d04b0 2024-09-19T07:43:11.312 INFO:teuthology.orchestra.run.smithi016.stderr:added data pool 4 to fsmap 2024-09-19T07:43:11.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.315+0000 7fc1e30e6640 1 -- 172.21.15.16:0/3167553725 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc1bc06c2b0 msgr2=0x7fc1bc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:11.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.315+0000 7fc1e30e6640 1 --2- 172.21.15.16:0/3167553725 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc1bc06c2b0 0x7fc1bc06e770 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc1d40079a0 tx=0x7fc1d4008040 comp rx=0 tx=0).stop 2024-09-19T07:43:11.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.315+0000 7fc1e30e6640 1 -- 172.21.15.16:0/3167553725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1dc0d04b0 msgr2=0x7fc1dc0b0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:11.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.315+0000 7fc1e30e6640 1 --2- 172.21.15.16:0/3167553725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1dc0d04b0 0x7fc1dc0b0f50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc000f80 tx=0x7fc1cc004a40 comp rx=0 tx=0).stop 2024-09-19T07:43:11.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.315+0000 7fc1e30e6640 1 -- 172.21.15.16:0/3167553725 shutdown_connections 2024-09-19T07:43:11.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.315+0000 7fc1e30e6640 1 --2- 172.21.15.16:0/3167553725 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc1bc06c2b0 0x7fc1bc06e770 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.315+0000 7fc1e30e6640 1 --2- 172.21.15.16:0/3167553725 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1dc15b120 0x7fc1dc0c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.315+0000 7fc1e30e6640 1 --2- 172.21.15.16:0/3167553725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1dc0d04b0 0x7fc1dc0b0f50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.315+0000 7fc1e30e6640 1 --2- 172.21.15.16:0/3167553725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1dc06dec0 0x7fc1dc154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.315+0000 7fc1e30e6640 1 -- 172.21.15.16:0/3167553725 >> 172.21.15.16:0/3167553725 conn(0x7fc1dc06bdf0 msgr2=0x7fc1dc075180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:11.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.315+0000 7fc1e30e6640 1 -- 172.21.15.16:0/3167553725 shutdown_connections 2024-09-19T07:43:11.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.315+0000 7fc1e30e6640 1 -- 172.21.15.16:0/3167553725 wait complete. 2024-09-19T07:43:11.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:43:11.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.436+0000 7fe02279f640 1 Processor -- start 2024-09-19T07:43:11.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.436+0000 7fe02279f640 1 -- start start 2024-09-19T07:43:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.437+0000 7fe02279f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe01c0d0070 0x7fe01c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.437+0000 7fe02279f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe01c0d4130 0x7fe01c0d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.437+0000 7fe02279f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe01c0d4ae0 con 0x7fe01c15f0d0 2024-09-19T07:43:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.437+0000 7fe02279f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fe01c0d1b10 con 0x7fe01c0d0070 2024-09-19T07:43:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.437+0000 7fe02279f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe01c0cf2b0 con 0x7fe01c0d4130 2024-09-19T07:43:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.437+0000 7fe01b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe01c0d0070 0x7fe01c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.437+0000 7fe01b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe01c0d0070 0x7fe01c15f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51738/0 (socket says 172.21.15.16:51738) 2024-09-19T07:43:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.437+0000 7fe01b7fe640 1 -- 172.21.15.16:0/3567300679 learned_addr learned my addr 172.21.15.16:0/3567300679 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.437+0000 7fe020d3c640 1 --2- 172.21.15.16:0/3567300679 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe01c0d4130 0x7fe01c0d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.437+0000 7fe01b7fe640 1 -- 172.21.15.16:0/3567300679 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe01c0d4130 msgr2=0x7fe01c0d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.438+0000 7fe01b7fe640 1 --2- 172.21.15.16:0/3567300679 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe01c0d4130 0x7fe01c0d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.438+0000 7fe01b7fe640 1 -- 172.21.15.16:0/3567300679 >> v1:172.21.15.146:6789/0 conn(0x7fe01c15f0d0 legacy=0x7fe01c0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.438+0000 7fe01b7fe640 1 -- 172.21.15.16:0/3567300679 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01c0cf590 con 0x7fe01c0d0070 2024-09-19T07:43:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.438+0000 7fe01b7fe640 1 --2- 172.21.15.16:0/3567300679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c0d0070 0x7fe01c15f4b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe010009870 tx=0x7fe01002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7889aca0cb240d6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.438+0000 7fe01affd640 1 -- 172.21.15.16:0/3567300679 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010037070 con 0x7fe01c0d0070 2024-09-19T07:43:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.438+0000 7fe01affd640 1 -- 172.21.15.16:0/3567300679 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe010032440 con 0x7fe01c0d0070 2024-09-19T07:43:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.438+0000 7fe01affd640 1 -- 172.21.15.16:0/3567300679 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010032810 con 0x7fe01c0d0070 2024-09-19T07:43:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.439+0000 7fe02279f640 1 -- 172.21.15.16:0/3567300679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c0d0070 msgr2=0x7fe01c15f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.439+0000 7fe02279f640 1 --2- 172.21.15.16:0/3567300679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c0d0070 0x7fe01c15f4b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe010009870 tx=0x7fe01002a1c0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.439+0000 7fe02279f640 1 -- 172.21.15.16:0/3567300679 shutdown_connections 2024-09-19T07:43:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.439+0000 7fe02279f640 1 --2- 172.21.15.16:0/3567300679 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe01c0d4130 0x7fe01c0d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.439+0000 7fe02279f640 1 --2- 172.21.15.16:0/3567300679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c0d0070 0x7fe01c15f4b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.439+0000 7fe02279f640 1 -- 172.21.15.16:0/3567300679 >> 172.21.15.16:0/3567300679 conn(0x7fe01c09d760 msgr2=0x7fe01c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.439+0000 7fe02279f640 1 -- 172.21.15.16:0/3567300679 shutdown_connections 2024-09-19T07:43:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.439+0000 7fe02279f640 1 -- 172.21.15.16:0/3567300679 wait complete. 2024-09-19T07:43:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.440+0000 7fe02279f640 1 Processor -- start 2024-09-19T07:43:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.440+0000 7fe02279f640 1 -- start start 2024-09-19T07:43:11.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.441+0000 7fe02279f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c0d0070 0x7fe01c15b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:11.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.441+0000 7fe02279f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe01c0d4130 0x7fe01c0ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:11.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.441+0000 7fe02279f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe01c15f0d0 0x7fe01c1518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:11.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.441+0000 7fe02279f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c0cef60 con 0x7fe01c0d0070 2024-09-19T07:43:11.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.441+0000 7fe02279f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c0cfb50 con 0x7fe01c0d4130 2024-09-19T07:43:11.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.441+0000 7fe02279f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe01c0b2f70 con 0x7fe01c15f0d0 2024-09-19T07:43:11.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.441+0000 7fe01bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c0d0070 0x7fe01c15b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:11.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.441+0000 7fe01bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c0d0070 0x7fe01c15b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51746/0 (socket says 172.21.15.16:51746) 2024-09-19T07:43:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.441+0000 7fe01bfff640 1 -- 172.21.15.16:0/1315370044 learned_addr learned my addr 172.21.15.16:0/1315370044 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.441+0000 7fe01bfff640 1 -- 172.21.15.16:0/1315370044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe01c15f0d0 msgr2=0x7fe01c1518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.441+0000 7fe01bfff640 1 --2- 172.21.15.16:0/1315370044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe01c15f0d0 0x7fe01c1518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.441+0000 7fe01bfff640 1 -- 172.21.15.16:0/1315370044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe01c0d4130 msgr2=0x7fe01c0ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.441+0000 7fe01bfff640 1 --2- 172.21.15.16:0/1315370044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe01c0d4130 0x7fe01c0ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.441+0000 7fe01bfff640 1 -- 172.21.15.16:0/1315370044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01c0b2850 con 0x7fe01c0d0070 2024-09-19T07:43:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.442+0000 7fe01bfff640 1 --2- 172.21.15.16:0/1315370044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c0d0070 0x7fe01c15b6d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00c3c0 tx=0x7fe00c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.442+0000 7fe018ff9640 1 -- 172.21.15.16:0/1315370044 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c0052d0 con 0x7fe01c0d0070 2024-09-19T07:43:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.442+0000 7fe018ff9640 1 -- 172.21.15.16:0/1315370044 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe00c005490 con 0x7fe01c0d0070 2024-09-19T07:43:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.442+0000 7fe02279f640 1 -- 172.21.15.16:0/1315370044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01c0b2b30 con 0x7fe01c0d0070 2024-09-19T07:43:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.442+0000 7fe018ff9640 1 -- 172.21.15.16:0/1315370044 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c007260 con 0x7fe01c0d0070 2024-09-19T07:43:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.442+0000 7fe02279f640 1 -- 172.21.15.16:0/1315370044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe01c0b4420 con 0x7fe01c0d0070 2024-09-19T07:43:11.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.443+0000 7fe02279f640 1 -- 172.21.15.16:0/1315370044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe01c0d45a0 con 0x7fe01c0d0070 2024-09-19T07:43:11.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.443+0000 7fe018ff9640 1 -- 172.21.15.16:0/1315370044 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe00c007400 con 0x7fe01c0d0070 2024-09-19T07:43:11.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.444+0000 7fe018ff9640 1 --2- 172.21.15.16:0/1315370044 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe00806c2b0 0x7fe00806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:11.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.444+0000 7fe018ff9640 1 -- 172.21.15.16:0/1315370044 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe00c08ab20 con 0x7fe01c0d0070 2024-09-19T07:43:11.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.444+0000 7fe01b7fe640 1 --2- 172.21.15.16:0/1315370044 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe00806c2b0 0x7fe00806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:11.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.445+0000 7fe01b7fe640 1 --2- 172.21.15.16:0/1315370044 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe00806c2b0 0x7fe00806e770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe010034000 tx=0x7fe01002c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:11.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.448+0000 7fe018ff9640 1 -- 172.21.15.16:0/1315370044 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe00c058fe0 con 0x7fe01c0d0070 2024-09-19T07:43:11.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.663+0000 7fe02279f640 1 -- 172.21.15.16:0/1315370044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe01c06cf30 con 0x7fe01c0d0070 2024-09-19T07:43:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.663+0000 7fe018ff9640 1 -- 172.21.15.16:0/1315370044 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5222 (secure 0 0 0) 0x7fe00c05de20 con 0x7fe01c0d0070 2024-09-19T07:43:11.664 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:11.664 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"btime":"2024-09-19T07:43:11:306999+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2772948330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2772948330},{"type":"v1","addr":"172.21.15.146:6835","nonce":2772948330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/340172279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":340172279},{"type":"v1","addr":"172.21.15.16:6833","nonce":340172279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1204811459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1204811459},{"type":"v1","addr":"172.21.15.16:6835","nonce":1204811459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:43:05.610038+0000","modified":"2024-09-19T07:43:10.642236+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/1303703249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1303703249},{"type":"v1","addr":"172.21.15.146:6837","nonce":1303703249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-09-19T07:43:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2024-09-19T07:43:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.665+0000 7fe02279f640 1 -- 172.21.15.16:0/1315370044 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe00806c2b0 msgr2=0x7fe00806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.665+0000 7fe02279f640 1 --2- 172.21.15.16:0/1315370044 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe00806c2b0 0x7fe00806e770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe010034000 tx=0x7fe01002c000 comp rx=0 tx=0).stop 2024-09-19T07:43:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.665+0000 7fe02279f640 1 -- 172.21.15.16:0/1315370044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c0d0070 msgr2=0x7fe01c15b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.665+0000 7fe02279f640 1 --2- 172.21.15.16:0/1315370044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c0d0070 0x7fe01c15b6d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00c3c0 tx=0x7fe00c00c880 comp rx=0 tx=0).stop 2024-09-19T07:43:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.665+0000 7fe02279f640 1 -- 172.21.15.16:0/1315370044 shutdown_connections 2024-09-19T07:43:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.665+0000 7fe02279f640 1 --2- 172.21.15.16:0/1315370044 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe00806c2b0 0x7fe00806e770 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.665+0000 7fe02279f640 1 --2- 172.21.15.16:0/1315370044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe01c15f0d0 0x7fe01c1518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.665+0000 7fe02279f640 1 --2- 172.21.15.16:0/1315370044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe01c0d4130 0x7fe01c0ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.665+0000 7fe02279f640 1 --2- 172.21.15.16:0/1315370044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe01c0d0070 0x7fe01c15b6d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.665+0000 7fe02279f640 1 -- 172.21.15.16:0/1315370044 >> 172.21.15.16:0/1315370044 conn(0x7fe01c09d760 msgr2=0x7fe01c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.665+0000 7fe02279f640 1 -- 172.21.15.16:0/1315370044 shutdown_connections 2024-09-19T07:43:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.665+0000 7fe02279f640 1 -- 172.21.15.16:0/1315370044 wait complete. 2024-09-19T07:43:11.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:43:11.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.777+0000 7fad89149640 1 Processor -- start 2024-09-19T07:43:11.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.777+0000 7fad89149640 1 -- start start 2024-09-19T07:43:11.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.778+0000 7fad89149640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fad8415f0d0 0x7fad8415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:11.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.778+0000 7fad89149640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fad840d0070 0x7fad840d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:11.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.778+0000 7fad89149640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad840cf2b0 con 0x7fad840d1940 2024-09-19T07:43:11.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.778+0000 7fad89149640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fad840cf590 con 0x7fad8415f0d0 2024-09-19T07:43:11.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.778+0000 7fad89149640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fad840cf870 con 0x7fad840d0070 2024-09-19T07:43:11.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.778+0000 7fad8359e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fad840d1940 0x7fad840d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42408/0 (socket says 172.21.15.16:42408) 2024-09-19T07:43:11.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.778+0000 7fad8359e640 1 -- 172.21.15.16:0/1255556773 learned_addr learned my addr 172.21.15.16:0/1255556773 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:11.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad82d9d640 1 --2- 172.21.15.16:0/1255556773 >> v2:172.21.15.146:3300/0 conn(0x7fad8415f0d0 0x7fad8415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:11.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad81d9b640 1 -- 172.21.15.16:0/1255556773 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2022748021 0 0) 0x7fad840cf2b0 con 0x7fad840d1940 2024-09-19T07:43:11.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad8259c640 1 --2- 172.21.15.16:0/1255556773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fad840d0070 0x7fad840d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:11.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad81d9b640 1 -- 172.21.15.16:0/1255556773 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad70003520 con 0x7fad840d1940 2024-09-19T07:43:11.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad81d9b640 1 -- 172.21.15.16:0/1255556773 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4035326649 0 0) 0x7fad70003520 con 0x7fad840d1940 2024-09-19T07:43:11.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad81d9b640 1 -- 172.21.15.16:0/1255556773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fad840d0070 msgr2=0x7fad840d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:11.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad81d9b640 1 --2- 172.21.15.16:0/1255556773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fad840d0070 0x7fad840d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad81d9b640 1 -- 172.21.15.16:0/1255556773 >> v2:172.21.15.146:3300/0 conn(0x7fad8415f0d0 msgr2=0x7fad8415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:11.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad81d9b640 1 --2- 172.21.15.16:0/1255556773 >> v2:172.21.15.146:3300/0 conn(0x7fad8415f0d0 0x7fad8415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad81d9b640 1 -- 172.21.15.16:0/1255556773 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad840cfb50 con 0x7fad840d1940 2024-09-19T07:43:11.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad81d9b640 1 -- 172.21.15.16:0/1255556773 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fad7c001690 con 0x7fad840d1940 2024-09-19T07:43:11.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad81d9b640 1 -- 172.21.15.16:0/1255556773 >> v1:172.21.15.16:6789/0 conn(0x7fad840d1940 legacy=0x7fad840d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:11.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad81d9b640 1 --2- 172.21.15.16:0/1255556773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad700039b0 0x7fad70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:11.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad81d9b640 1 --2- 172.21.15.16:0/1255556773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004800 0x7fad70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:11.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad81d9b640 1 --2- 172.21.15.16:0/1255556773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fad840d0070 0x7fad700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:11.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad81d9b640 1 -- 172.21.15.16:0/1255556773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad840cf590 con 0x7fad70004800 2024-09-19T07:43:11.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad81d9b640 1 -- 172.21.15.16:0/1255556773 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fad840cf870 con 0x7fad700039b0 2024-09-19T07:43:11.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad81d9b640 1 -- 172.21.15.16:0/1255556773 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fad7000b9f0 con 0x7fad840d0070 2024-09-19T07:43:11.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad8259c640 1 --2- 172.21.15.16:0/1255556773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004800 0x7fad70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:11.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad82d9d640 1 --2- 172.21.15.16:0/1255556773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad700039b0 0x7fad70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:11.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad8359e640 1 --2- 172.21.15.16:0/1255556773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fad840d0070 0x7fad700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:11.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad8259c640 1 -- 172.21.15.16:0/1255556773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fad840d0070 msgr2=0x7fad700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:11.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad8259c640 1 --2- 172.21.15.16:0/1255556773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fad840d0070 0x7fad700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad8259c640 1 -- 172.21.15.16:0/1255556773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad700039b0 msgr2=0x7fad70003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:11.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad8259c640 1 --2- 172.21.15.16:0/1255556773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad700039b0 0x7fad70003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad8259c640 1 -- 172.21.15.16:0/1255556773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad70003520 con 0x7fad70004800 2024-09-19T07:43:11.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad82d9d640 1 --2- 172.21.15.16:0/1255556773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad700039b0 0x7fad70003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:11.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.779+0000 7fad8359e640 1 --2- 172.21.15.16:0/1255556773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fad840d0070 0x7fad700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:11.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.780+0000 7fad8259c640 1 --2- 172.21.15.16:0/1255556773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004800 0x7fad70004c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009ac0 tx=0x7fad70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102025b09c9488b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:11.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.780+0000 7fad81d9b640 1 -- 172.21.15.16:0/1255556773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c036070 con 0x7fad70004800 2024-09-19T07:43:11.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.780+0000 7fad81d9b640 1 -- 172.21.15.16:0/1255556773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad6c031400 con 0x7fad70004800 2024-09-19T07:43:11.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.780+0000 7fad81d9b640 1 -- 172.21.15.16:0/1255556773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c009690 con 0x7fad70004800 2024-09-19T07:43:11.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.781+0000 7fad89149640 1 -- 172.21.15.16:0/1255556773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004800 msgr2=0x7fad70004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.781+0000 7fad89149640 1 --2- 172.21.15.16:0/1255556773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004800 0x7fad70004c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009ac0 tx=0x7fad70005830 comp rx=0 tx=0).stop 2024-09-19T07:43:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.781+0000 7fad8359e640 1 -- 172.21.15.16:0/1255556773 reap_dead start 2024-09-19T07:43:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.781+0000 7fad89149640 1 -- 172.21.15.16:0/1255556773 shutdown_connections 2024-09-19T07:43:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.781+0000 7fad89149640 1 -- 172.21.15.16:0/1255556773 >> 172.21.15.16:0/1255556773 conn(0x7fad8409d760 msgr2=0x7fad8408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.781+0000 7fad89149640 1 -- 172.21.15.16:0/1255556773 shutdown_connections 2024-09-19T07:43:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.781+0000 7fad89149640 1 -- 172.21.15.16:0/1255556773 wait complete. 2024-09-19T07:43:11.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.782+0000 7fad89149640 1 Processor -- start 2024-09-19T07:43:11.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.782+0000 7fad89149640 1 -- start start 2024-09-19T07:43:11.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.783+0000 7fad89149640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004800 0x7fad840a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:11.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.783+0000 7fad89149640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fad8409e0d0 0x7fad840b9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:11.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.783+0000 7fad89149640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad840b7600 0x7fad840b7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:11.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.783+0000 7fad89149640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad840d49c0 con 0x7fad70004800 2024-09-19T07:43:11.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.783+0000 7fad89149640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fad840d4840 con 0x7fad840b7600 2024-09-19T07:43:11.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.783+0000 7fad89149640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fad840d4b40 con 0x7fad8409e0d0 2024-09-19T07:43:11.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.783+0000 7fad82d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004800 0x7fad840a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:11.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.783+0000 7fad82d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004800 0x7fad840a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51754/0 (socket says 172.21.15.16:51754) 2024-09-19T07:43:11.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.783+0000 7fad82d9d640 1 -- 172.21.15.16:0/2357683931 learned_addr learned my addr 172.21.15.16:0/2357683931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:11.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.783+0000 7fad8359e640 1 --2- 172.21.15.16:0/2357683931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad840b7600 0x7fad840b7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:11.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.783+0000 7fad8259c640 1 --2- 172.21.15.16:0/2357683931 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fad8409e0d0 0x7fad840b9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:11.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.783+0000 7fad82d9d640 1 -- 172.21.15.16:0/2357683931 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fad8409e0d0 msgr2=0x7fad840b9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:11.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.783+0000 7fad82d9d640 1 --2- 172.21.15.16:0/2357683931 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fad8409e0d0 0x7fad840b9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.783+0000 7fad82d9d640 1 -- 172.21.15.16:0/2357683931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad840b7600 msgr2=0x7fad840b7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.783+0000 7fad82d9d640 1 --2- 172.21.15.16:0/2357683931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad840b7600 0x7fad840b7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.783+0000 7fad82d9d640 1 -- 172.21.15.16:0/2357683931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad840a0bd0 con 0x7fad70004800 2024-09-19T07:43:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.784+0000 7fad82d9d640 1 --2- 172.21.15.16:0/2357683931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004800 0x7fad840a0690 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fad740101f0 tx=0x7fad740105f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.784+0000 7fad7b7fe640 1 -- 172.21.15.16:0/2357683931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7401a030 con 0x7fad70004800 2024-09-19T07:43:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.784+0000 7fad7b7fe640 1 -- 172.21.15.16:0/2357683931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad74003990 con 0x7fad70004800 2024-09-19T07:43:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.784+0000 7fad7b7fe640 1 -- 172.21.15.16:0/2357683931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74003d60 con 0x7fad70004800 2024-09-19T07:43:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.784+0000 7fad89149640 1 -- 172.21.15.16:0/2357683931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad840a0e50 con 0x7fad70004800 2024-09-19T07:43:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.784+0000 7fad89149640 1 -- 172.21.15.16:0/2357683931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad840a1410 con 0x7fad70004800 2024-09-19T07:43:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.785+0000 7fad89149640 1 -- 172.21.15.16:0/2357683931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad8415f4b0 con 0x7fad70004800 2024-09-19T07:43:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.785+0000 7fad7b7fe640 1 -- 172.21.15.16:0/2357683931 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fad7401d070 con 0x7fad70004800 2024-09-19T07:43:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.786+0000 7fad7b7fe640 1 --2- 172.21.15.16:0/2357683931 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fad6406c2b0 0x7fad6406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.786+0000 7fad7b7fe640 1 -- 172.21.15.16:0/2357683931 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fad7408c750 con 0x7fad70004800 2024-09-19T07:43:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.786+0000 7fad8259c640 1 --2- 172.21.15.16:0/2357683931 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fad6406c2b0 0x7fad6406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.786+0000 7fad8259c640 1 --2- 172.21.15.16:0/2357683931 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fad6406c2b0 0x7fad6406e770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fad6c033000 tx=0x7fad6c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:11.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.790+0000 7fad7b7fe640 1 -- 172.21.15.16:0/2357683931 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fad840a0bd0 con 0x7fad70004800 2024-09-19T07:43:11.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.967+0000 7fad89149640 1 -- 172.21.15.16:0/2357683931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad8406cf30 con 0x7fad70004800 2024-09-19T07:43:11.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.968+0000 7fad7b7fe640 1 -- 172.21.15.16:0/2357683931 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7fad7405ac10 con 0x7fad70004800 2024-09-19T07:43:11.969 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:11.969 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:43:10.642082+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:43:03.485582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:43:04.603961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:43:08.299591+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:43:11.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.970+0000 7fad89149640 1 -- 172.21.15.16:0/2357683931 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fad6406c2b0 msgr2=0x7fad6406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:11.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.971+0000 7fad89149640 1 --2- 172.21.15.16:0/2357683931 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fad6406c2b0 0x7fad6406e770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fad6c033000 tx=0x7fad6c008030 comp rx=0 tx=0).stop 2024-09-19T07:43:11.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.971+0000 7fad89149640 1 -- 172.21.15.16:0/2357683931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004800 msgr2=0x7fad840a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:11.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.971+0000 7fad89149640 1 --2- 172.21.15.16:0/2357683931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004800 0x7fad840a0690 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fad740101f0 tx=0x7fad740105f0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.971+0000 7fad89149640 1 -- 172.21.15.16:0/2357683931 shutdown_connections 2024-09-19T07:43:11.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.971+0000 7fad89149640 1 --2- 172.21.15.16:0/2357683931 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fad6406c2b0 0x7fad6406e770 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.971+0000 7fad89149640 1 --2- 172.21.15.16:0/2357683931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad840b7600 0x7fad840b7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.971+0000 7fad89149640 1 --2- 172.21.15.16:0/2357683931 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fad8409e0d0 0x7fad840b9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.971+0000 7fad89149640 1 --2- 172.21.15.16:0/2357683931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004800 0x7fad840a0690 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:11.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.971+0000 7fad89149640 1 -- 172.21.15.16:0/2357683931 >> 172.21.15.16:0/2357683931 conn(0x7fad8409d760 msgr2=0x7fad840b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:11.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.971+0000 7fad89149640 1 -- 172.21.15.16:0/2357683931 shutdown_connections 2024-09-19T07:43:11.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:11.971+0000 7fad89149640 1 -- 172.21.15.16:0/2357683931 wait complete. 2024-09-19T07:43:11.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:43:12.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.083+0000 7f719bf2c640 1 Processor -- start 2024-09-19T07:43:12.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.083+0000 7f719bf2c640 1 -- start start 2024-09-19T07:43:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.084+0000 7f719bf2c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71940c8d20 0x7f719406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.084+0000 7f719bf2c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71940c7140 con 0x7f719406d260 2024-09-19T07:43:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.084+0000 7f719bf2c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71940c9100 con 0x7f7194144100 2024-09-19T07:43:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.084+0000 7f719bf2c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f719406d6d0 con 0x7f71940c8d20 2024-09-19T07:43:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.084+0000 7f7199cc8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71940c8d20 0x7f719406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.084+0000 7f71994c7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f719406d260 0x7f71941439f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42412/0 (socket says 172.21.15.16:42412) 2024-09-19T07:43:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.084+0000 7f71994c7640 1 -- 172.21.15.16:0/4118220078 learned_addr learned my addr 172.21.15.16:0/4118220078 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.084+0000 7f7199cc8640 1 -- 172.21.15.16:0/4118220078 >> v1:172.21.15.146:6789/0 conn(0x7f7194144100 legacy=0x7f71940d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.084+0000 7f7199cc8640 1 -- 172.21.15.16:0/4118220078 >> v1:172.21.15.16:6789/0 conn(0x7f719406d260 legacy=0x7f71941439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.084+0000 7f7199cc8640 1 -- 172.21.15.16:0/4118220078 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7194144760 con 0x7f71940c8d20 2024-09-19T07:43:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.084+0000 7f7198cc6640 1 -- 172.21.15.16:0/4118220078 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 643443143 0 0) 0x7f7184001110 con 0x7f719406d260 2024-09-19T07:43:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.085+0000 7f7199cc8640 1 --2- 172.21.15.16:0/4118220078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71940c8d20 0x7f719406cd20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f71900097a0 tx=0x7f719002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40157329805c81c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.085+0000 7f7198cc6640 1 -- 172.21.15.16:0/4118220078 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190037070 con 0x7f71940c8d20 2024-09-19T07:43:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.085+0000 7f7198cc6640 1 -- 172.21.15.16:0/4118220078 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f719002adc0 con 0x7f71940c8d20 2024-09-19T07:43:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.085+0000 7f7198cc6640 1 -- 172.21.15.16:0/4118220078 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190032660 con 0x7f71940c8d20 2024-09-19T07:43:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.086+0000 7f719bf2c640 1 -- 172.21.15.16:0/4118220078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71940c8d20 msgr2=0x7f719406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.086+0000 7f719bf2c640 1 --2- 172.21.15.16:0/4118220078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71940c8d20 0x7f719406cd20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f71900097a0 tx=0x7f719002a0b0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.086+0000 7f719bf2c640 1 -- 172.21.15.16:0/4118220078 shutdown_connections 2024-09-19T07:43:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.086+0000 7f719bf2c640 1 --2- 172.21.15.16:0/4118220078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71940c8d20 0x7f719406cd20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.086+0000 7f719bf2c640 1 -- 172.21.15.16:0/4118220078 >> 172.21.15.16:0/4118220078 conn(0x7f71940b3ed0 msgr2=0x7f71940b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.086+0000 7f719bf2c640 1 -- 172.21.15.16:0/4118220078 shutdown_connections 2024-09-19T07:43:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.086+0000 7f719bf2c640 1 -- 172.21.15.16:0/4118220078 wait complete. 2024-09-19T07:43:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.087+0000 7f719bf2c640 1 Processor -- start 2024-09-19T07:43:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.087+0000 7f719bf2c640 1 -- start start 2024-09-19T07:43:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f719bf2c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f719406d260 0x7f7194140790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f719bf2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71940c8d20 0x7f71940b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f719bf2c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7194144100 0x7f719415fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f719bf2c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7194142200 con 0x7f71940c8d20 2024-09-19T07:43:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f719bf2c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7194141960 con 0x7f7194144100 2024-09-19T07:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f719bf2c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7194144510 con 0x7f719406d260 2024-09-19T07:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f71994c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71940c8d20 0x7f71940b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f71994c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71940c8d20 0x7f71940b7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51772/0 (socket says 172.21.15.16:51772) 2024-09-19T07:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f71994c7640 1 -- 172.21.15.16:0/1375310604 learned_addr learned my addr 172.21.15.16:0/1375310604 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f7199cc8640 1 --2- 172.21.15.16:0/1375310604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f719406d260 0x7f7194140790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f719a4c9640 1 --2- 172.21.15.16:0/1375310604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7194144100 0x7f719415fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f71994c7640 1 -- 172.21.15.16:0/1375310604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f719406d260 msgr2=0x7f7194140790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f71994c7640 1 --2- 172.21.15.16:0/1375310604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f719406d260 0x7f7194140790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f71994c7640 1 -- 172.21.15.16:0/1375310604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7194144100 msgr2=0x7f719415fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f71994c7640 1 --2- 172.21.15.16:0/1375310604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7194144100 0x7f719415fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f71994c7640 1 -- 172.21.15.16:0/1375310604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71941557a0 con 0x7f71940c8d20 2024-09-19T07:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f7199cc8640 1 --2- 172.21.15.16:0/1375310604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f719406d260 0x7f7194140790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f719a4c9640 1 --2- 172.21.15.16:0/1375310604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7194144100 0x7f719415fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f71994c7640 1 --2- 172.21.15.16:0/1375310604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71940c8d20 0x7f71940b7330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f718400c460 tx=0x7f718400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.088+0000 7f718a7fc640 1 -- 172.21.15.16:0/1375310604 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184018070 con 0x7f71940c8d20 2024-09-19T07:43:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.089+0000 7f718a7fc640 1 -- 172.21.15.16:0/1375310604 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7184007500 con 0x7f71940c8d20 2024-09-19T07:43:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.089+0000 7f719bf2c640 1 -- 172.21.15.16:0/1375310604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7194155a80 con 0x7f71940c8d20 2024-09-19T07:43:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.089+0000 7f718a7fc640 1 -- 172.21.15.16:0/1375310604 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184004a20 con 0x7f71940c8d20 2024-09-19T07:43:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.089+0000 7f719bf2c640 1 -- 172.21.15.16:0/1375310604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7194156040 con 0x7f71940c8d20 2024-09-19T07:43:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.090+0000 7f719bf2c640 1 -- 172.21.15.16:0/1375310604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f719406cd20 con 0x7f71940c8d20 2024-09-19T07:43:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.090+0000 7f718a7fc640 1 -- 172.21.15.16:0/1375310604 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f718401b070 con 0x7f71940c8d20 2024-09-19T07:43:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.090+0000 7f718a7fc640 1 --2- 172.21.15.16:0/1375310604 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f717806c2b0 0x7f717806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.091+0000 7f718a7fc640 1 -- 172.21.15.16:0/1375310604 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f718408aa30 con 0x7f71940c8d20 2024-09-19T07:43:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.091+0000 7f7199cc8640 1 --2- 172.21.15.16:0/1375310604 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f717806c2b0 0x7f717806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.091+0000 7f7199cc8640 1 --2- 172.21.15.16:0/1375310604 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f717806c2b0 0x7f717806e770 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7190034000 tx=0x7f719002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.095+0000 7f718a7fc640 1 -- 172.21.15.16:0/1375310604 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f71940c7140 con 0x7f71940c8d20 2024-09-19T07:43:12.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.272+0000 7f719bf2c640 1 -- 172.21.15.16:0/1375310604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7194072290 con 0x7f71940c8d20 2024-09-19T07:43:12.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.273+0000 7f718a7fc640 1 -- 172.21.15.16:0/1375310604 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7f7184058e50 con 0x7f71940c8d20 2024-09-19T07:43:12.274 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:12.274 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:43:10.642082+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:43:03.485582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:43:04.603961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:43:08.299591+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:43:12.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.275+0000 7f719bf2c640 1 -- 172.21.15.16:0/1375310604 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f717806c2b0 msgr2=0x7f717806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:12.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.275+0000 7f719bf2c640 1 --2- 172.21.15.16:0/1375310604 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f717806c2b0 0x7f717806e770 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7190034000 tx=0x7f719002e000 comp rx=0 tx=0).stop 2024-09-19T07:43:12.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.275+0000 7f719bf2c640 1 -- 172.21.15.16:0/1375310604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71940c8d20 msgr2=0x7f71940b7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:12.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.275+0000 7f719bf2c640 1 --2- 172.21.15.16:0/1375310604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71940c8d20 0x7f71940b7330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f718400c460 tx=0x7f718400c920 comp rx=0 tx=0).stop 2024-09-19T07:43:12.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.275+0000 7f719bf2c640 1 -- 172.21.15.16:0/1375310604 shutdown_connections 2024-09-19T07:43:12.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.275+0000 7f719bf2c640 1 --2- 172.21.15.16:0/1375310604 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f717806c2b0 0x7f717806e770 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.275+0000 7f719bf2c640 1 --2- 172.21.15.16:0/1375310604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7194144100 0x7f719415fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.275+0000 7f719bf2c640 1 --2- 172.21.15.16:0/1375310604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71940c8d20 0x7f71940b7330 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.275+0000 7f719bf2c640 1 --2- 172.21.15.16:0/1375310604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f719406d260 0x7f7194140790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.275+0000 7f719bf2c640 1 -- 172.21.15.16:0/1375310604 >> 172.21.15.16:0/1375310604 conn(0x7f71940b3ed0 msgr2=0x7f71940b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.275+0000 7f719bf2c640 1 -- 172.21.15.16:0/1375310604 shutdown_connections 2024-09-19T07:43:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.275+0000 7f719bf2c640 1 -- 172.21.15.16:0/1375310604 wait complete. 2024-09-19T07:43:12.304 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:43:12.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.406+0000 7f6983bc4640 1 Processor -- start 2024-09-19T07:43:12.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.406+0000 7f6983bc4640 1 -- start start 2024-09-19T07:43:12.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.407+0000 7f6983bc4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f697c15e680 0x7f697c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:12.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.407+0000 7f6983bc4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f697c0d1880 con 0x7f697c15e2a0 2024-09-19T07:43:12.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.407+0000 7f6983bc4640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f697c0d1b60 con 0x7f697c0d0010 2024-09-19T07:43:12.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.407+0000 7f6983bc4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f697c0cf2d0 con 0x7f697c15e680 2024-09-19T07:43:12.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.407+0000 7f698115f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f697c15e680 0x7f697c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:12.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.407+0000 7f6981960640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f697c15e2a0 0x7f697c06d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42424/0 (socket says 172.21.15.16:42424) 2024-09-19T07:43:12.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.407+0000 7f6981960640 1 -- 172.21.15.16:0/442023514 learned_addr learned my addr 172.21.15.16:0/442023514 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:12.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.407+0000 7f698115f640 1 -- 172.21.15.16:0/442023514 >> v1:172.21.15.146:6789/0 conn(0x7f697c0d0010 legacy=0x7f697c0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:12.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.407+0000 7f698115f640 1 -- 172.21.15.16:0/442023514 >> v1:172.21.15.16:6789/0 conn(0x7f697c15e2a0 legacy=0x7f697c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:12.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.407+0000 7f698115f640 1 -- 172.21.15.16:0/442023514 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c0cf5b0 con 0x7f697c15e680 2024-09-19T07:43:12.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.408+0000 7f698115f640 1 --2- 172.21.15.16:0/442023514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f697c15e680 0x7f697c0cfad0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f696c0097a0 tx=0x7f696c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d761afcdbf36a8f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:12.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.408+0000 7f698095e640 1 -- 172.21.15.16:0/442023514 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c037070 con 0x7f697c15e680 2024-09-19T07:43:12.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.408+0000 7f698095e640 1 -- 172.21.15.16:0/442023514 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f696c032440 con 0x7f697c15e680 2024-09-19T07:43:12.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.408+0000 7f698095e640 1 -- 172.21.15.16:0/442023514 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c032810 con 0x7f697c15e680 2024-09-19T07:43:12.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.409+0000 7f6983bc4640 1 -- 172.21.15.16:0/442023514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f697c15e680 msgr2=0x7f697c0cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:12.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.409+0000 7f6983bc4640 1 --2- 172.21.15.16:0/442023514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f697c15e680 0x7f697c0cfad0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f696c0097a0 tx=0x7f696c02a1a0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.409+0000 7f6983bc4640 1 -- 172.21.15.16:0/442023514 shutdown_connections 2024-09-19T07:43:12.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.409+0000 7f6983bc4640 1 --2- 172.21.15.16:0/442023514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f697c15e680 0x7f697c0cfad0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.409+0000 7f6983bc4640 1 -- 172.21.15.16:0/442023514 >> 172.21.15.16:0/442023514 conn(0x7f697c09d760 msgr2=0x7f697c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:12.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.409+0000 7f6983bc4640 1 -- 172.21.15.16:0/442023514 shutdown_connections 2024-09-19T07:43:12.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.409+0000 7f6983bc4640 1 -- 172.21.15.16:0/442023514 wait complete. 2024-09-19T07:43:12.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.410+0000 7f6983bc4640 1 Processor -- start 2024-09-19T07:43:12.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.410+0000 7f6983bc4640 1 -- start start 2024-09-19T07:43:12.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f6983bc4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f697c0d0010 0x7f697c09f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:12.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f6983bc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f697c15e2a0 0x7f697c1f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:12.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f6983bc4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f697c15e680 0x7f697c1f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:12.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f6983bc4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f697c06dcb0 con 0x7f697c15e2a0 2024-09-19T07:43:12.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f6983bc4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f697c06db30 con 0x7f697c0d0010 2024-09-19T07:43:12.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f6983bc4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f697c0d4a80 con 0x7f697c15e680 2024-09-19T07:43:12.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f698115f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f697c15e2a0 0x7f697c1f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:12.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f698115f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f697c15e2a0 0x7f697c1f3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51794/0 (socket says 172.21.15.16:51794) 2024-09-19T07:43:12.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f698115f640 1 -- 172.21.15.16:0/2582435284 learned_addr learned my addr 172.21.15.16:0/2582435284 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:12.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f6981960640 1 --2- 172.21.15.16:0/2582435284 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f697c0d0010 0x7f697c09f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:12.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f6982161640 1 --2- 172.21.15.16:0/2582435284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f697c15e680 0x7f697c1f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:12.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f698115f640 1 -- 172.21.15.16:0/2582435284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f697c15e680 msgr2=0x7f697c1f43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:12.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f698115f640 1 --2- 172.21.15.16:0/2582435284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f697c15e680 0x7f697c1f43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f698115f640 1 -- 172.21.15.16:0/2582435284 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f697c0d0010 msgr2=0x7f697c09f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:12.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f698115f640 1 --2- 172.21.15.16:0/2582435284 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f697c0d0010 0x7f697c09f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f698115f640 1 -- 172.21.15.16:0/2582435284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c150540 con 0x7f697c15e2a0 2024-09-19T07:43:12.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f6982161640 1 --2- 172.21.15.16:0/2582435284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f697c15e680 0x7f697c1f43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:12.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f698115f640 1 --2- 172.21.15.16:0/2582435284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f697c15e2a0 0x7f697c1f3e70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f696c000c00 tx=0x7f696c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:12.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f69727fc640 1 -- 172.21.15.16:0/2582435284 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c037040 con 0x7f697c15e2a0 2024-09-19T07:43:12.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f69727fc640 1 -- 172.21.15.16:0/2582435284 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f696c042070 con 0x7f697c15e2a0 2024-09-19T07:43:12.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f6983bc4640 1 -- 172.21.15.16:0/2582435284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f697c09fcd0 con 0x7f697c15e2a0 2024-09-19T07:43:12.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f69727fc640 1 -- 172.21.15.16:0/2582435284 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c030bd0 con 0x7f697c15e2a0 2024-09-19T07:43:12.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.411+0000 7f6983bc4640 1 -- 172.21.15.16:0/2582435284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f697c0a0290 con 0x7f697c15e2a0 2024-09-19T07:43:12.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.413+0000 7f6983bc4640 1 -- 172.21.15.16:0/2582435284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f697c0cfad0 con 0x7f697c15e2a0 2024-09-19T07:43:12.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.413+0000 7f69727fc640 1 -- 172.21.15.16:0/2582435284 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f696c047460 con 0x7f697c15e2a0 2024-09-19T07:43:12.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.414+0000 7f69727fc640 1 --2- 172.21.15.16:0/2582435284 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f696006c2b0 0x7f696006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:12.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.414+0000 7f69727fc640 1 -- 172.21.15.16:0/2582435284 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f696c0ac360 con 0x7f697c15e2a0 2024-09-19T07:43:12.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.414+0000 7f6981960640 1 --2- 172.21.15.16:0/2582435284 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f696006c2b0 0x7f696006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:12.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.415+0000 7f6981960640 1 --2- 172.21.15.16:0/2582435284 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f696006c2b0 0x7f696006e770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6978005880 tx=0x7f6978009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:12.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.418+0000 7f69727fc640 1 -- 172.21.15.16:0/2582435284 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f696c032e20 con 0x7f697c15e2a0 2024-09-19T07:43:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.629+0000 7f6983bc4640 1 -- 172.21.15.16:0/2582435284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f697c06cf90 con 0x7f697c15e2a0 2024-09-19T07:43:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.629+0000 7f69727fc640 1 -- 172.21.15.16:0/2582435284 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5222 (secure 0 0 0) 0x7f696c07a790 con 0x7f697c15e2a0 2024-09-19T07:43:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:12.630 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"btime":"2024-09-19T07:43:11:306999+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2772948330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2772948330},{"type":"v1","addr":"172.21.15.146:6835","nonce":2772948330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/340172279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":340172279},{"type":"v1","addr":"172.21.15.16:6833","nonce":340172279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1204811459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1204811459},{"type":"v1","addr":"172.21.15.16:6835","nonce":1204811459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:43:05.610038+0000","modified":"2024-09-19T07:43:10.642236+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/1303703249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1303703249},{"type":"v1","addr":"172.21.15.146:6837","nonce":1303703249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-09-19T07:43:12.630 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2024-09-19T07:43:12.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.631+0000 7f6983bc4640 1 -- 172.21.15.16:0/2582435284 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f696006c2b0 msgr2=0x7f696006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:12.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.631+0000 7f6983bc4640 1 --2- 172.21.15.16:0/2582435284 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f696006c2b0 0x7f696006e770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6978005880 tx=0x7f6978009040 comp rx=0 tx=0).stop 2024-09-19T07:43:12.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.631+0000 7f6983bc4640 1 -- 172.21.15.16:0/2582435284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f697c15e2a0 msgr2=0x7f697c1f3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:12.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.631+0000 7f6983bc4640 1 --2- 172.21.15.16:0/2582435284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f697c15e2a0 0x7f697c1f3e70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f696c000c00 tx=0x7f696c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.631+0000 7f6983bc4640 1 -- 172.21.15.16:0/2582435284 shutdown_connections 2024-09-19T07:43:12.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.631+0000 7f6983bc4640 1 --2- 172.21.15.16:0/2582435284 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f696006c2b0 0x7f696006e770 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.631+0000 7f6983bc4640 1 --2- 172.21.15.16:0/2582435284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f697c15e680 0x7f697c1f43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.631+0000 7f6983bc4640 1 --2- 172.21.15.16:0/2582435284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f697c15e2a0 0x7f697c1f3e70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.631+0000 7f6983bc4640 1 --2- 172.21.15.16:0/2582435284 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f697c0d0010 0x7f697c09f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.631+0000 7f6983bc4640 1 -- 172.21.15.16:0/2582435284 >> 172.21.15.16:0/2582435284 conn(0x7f697c09d760 msgr2=0x7f697c0d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.631+0000 7f6983bc4640 1 -- 172.21.15.16:0/2582435284 shutdown_connections 2024-09-19T07:43:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.631+0000 7f6983bc4640 1 -- 172.21.15.16:0/2582435284 wait complete. 2024-09-19T07:43:12.643 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:43:12.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.745+0000 7f916912c640 1 Processor -- start 2024-09-19T07:43:12.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.745+0000 7f916912c640 1 -- start start 2024-09-19T07:43:12.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.746+0000 7f916912c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f916415f0d0 0x7f916415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:12.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.746+0000 7f916912c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91640d0070 0x7f91640d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:12.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.746+0000 7f916912c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f91640d1940 0x7f91640d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:12.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.746+0000 7f916912c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f91640cf2b0 con 0x7f916415f0d0 2024-09-19T07:43:12.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.746+0000 7f916912c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f91640cf590 con 0x7f91640d1940 2024-09-19T07:43:12.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.746+0000 7f916912c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f91640cf870 con 0x7f91640d0070 2024-09-19T07:43:12.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.746+0000 7f9162d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f916415f0d0 0x7f916415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:12.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.746+0000 7f9162d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f916415f0d0 0x7f916415f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38954/0 (socket says 172.21.15.16:38954) 2024-09-19T07:43:12.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.746+0000 7f9162d9d640 1 -- 172.21.15.16:0/402660257 learned_addr learned my addr 172.21.15.16:0/402660257 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:12.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.746+0000 7f9162d9d640 1 -- 172.21.15.16:0/402660257 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91640d0070 msgr2=0x7f91640d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:12.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.746+0000 7f9162d9d640 1 --2- 172.21.15.16:0/402660257 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91640d0070 0x7f91640d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.746+0000 7f9162d9d640 1 -- 172.21.15.16:0/402660257 >> v2:172.21.15.146:3300/0 conn(0x7f91640d1940 msgr2=0x7f91640d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:12.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.746+0000 7f9162d9d640 1 --2- 172.21.15.16:0/402660257 >> v2:172.21.15.146:3300/0 conn(0x7f91640d1940 0x7f91640d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.746+0000 7f9162d9d640 1 -- 172.21.15.16:0/402660257 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91640cfb50 con 0x7f916415f0d0 2024-09-19T07:43:12.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.747+0000 7f9162d9d640 1 --2- 172.21.15.16:0/402660257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916415f0d0 0x7f916415f4b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9164161e50 tx=0x7f915402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7754b85ed8d7ab1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:12.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.747+0000 7f9161d9b640 1 -- 172.21.15.16:0/402660257 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9154037070 con 0x7f916415f0d0 2024-09-19T07:43:12.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.747+0000 7f9161d9b640 1 -- 172.21.15.16:0/402660257 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9154032440 con 0x7f916415f0d0 2024-09-19T07:43:12.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.747+0000 7f9161d9b640 1 -- 172.21.15.16:0/402660257 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9154032810 con 0x7f916415f0d0 2024-09-19T07:43:12.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.748+0000 7f916912c640 1 -- 172.21.15.16:0/402660257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916415f0d0 msgr2=0x7f916415f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:12.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.748+0000 7f916912c640 1 --2- 172.21.15.16:0/402660257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916415f0d0 0x7f916415f4b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9164161e50 tx=0x7f915402a140 comp rx=0 tx=0).stop 2024-09-19T07:43:12.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.748+0000 7f916912c640 1 -- 172.21.15.16:0/402660257 shutdown_connections 2024-09-19T07:43:12.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.748+0000 7f916912c640 1 --2- 172.21.15.16:0/402660257 >> v2:172.21.15.146:3300/0 conn(0x7f91640d1940 0x7f91640d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.748+0000 7f916912c640 1 --2- 172.21.15.16:0/402660257 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91640d0070 0x7f91640d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.748+0000 7f916912c640 1 --2- 172.21.15.16:0/402660257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916415f0d0 0x7f916415f4b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.748+0000 7f916912c640 1 -- 172.21.15.16:0/402660257 >> 172.21.15.16:0/402660257 conn(0x7f916409d760 msgr2=0x7f916408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:12.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.748+0000 7f916912c640 1 -- 172.21.15.16:0/402660257 shutdown_connections 2024-09-19T07:43:12.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.748+0000 7f916912c640 1 -- 172.21.15.16:0/402660257 wait complete. 2024-09-19T07:43:12.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.749+0000 7f916912c640 1 Processor -- start 2024-09-19T07:43:12.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.749+0000 7f916912c640 1 -- start start 2024-09-19T07:43:12.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.750+0000 7f916912c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91640d0070 0x7f91640b9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:12.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.750+0000 7f916912c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91640d1940 0x7f91640b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.750+0000 7f916912c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f916415f0d0 0x7f91641503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.750+0000 7f916912c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f91640d47f0 con 0x7f91640d0070 2024-09-19T07:43:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.750+0000 7f916912c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f91640d4670 con 0x7f916415f0d0 2024-09-19T07:43:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.750+0000 7f916912c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f91640d4970 con 0x7f91640d1940 2024-09-19T07:43:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.750+0000 7f9162d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91640d0070 0x7f91640b9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.750+0000 7f9162d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91640d0070 0x7f91640b9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38962/0 (socket says 172.21.15.16:38962) 2024-09-19T07:43:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.750+0000 7f9162d9d640 1 -- 172.21.15.16:0/1009348591 learned_addr learned my addr 172.21.15.16:0/1009348591 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.750+0000 7f916259c640 1 --2- 172.21.15.16:0/1009348591 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91640d1940 0x7f91640b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.750+0000 7f916359e640 1 --2- 172.21.15.16:0/1009348591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f916415f0d0 0x7f91641503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.750+0000 7f9162d9d640 1 -- 172.21.15.16:0/1009348591 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91640d1940 msgr2=0x7f91640b81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.750+0000 7f9162d9d640 1 --2- 172.21.15.16:0/1009348591 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91640d1940 0x7f91640b81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.750+0000 7f9162d9d640 1 -- 172.21.15.16:0/1009348591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f916415f0d0 msgr2=0x7f91641503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.750+0000 7f9162d9d640 1 --2- 172.21.15.16:0/1009348591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f916415f0d0 0x7f91641503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.750+0000 7f9162d9d640 1 -- 172.21.15.16:0/1009348591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91640b7b10 con 0x7f91640d0070 2024-09-19T07:43:12.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.750+0000 7f916259c640 1 --2- 172.21.15.16:0/1009348591 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91640d1940 0x7f91640b81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:12.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.750+0000 7f916359e640 1 --2- 172.21.15.16:0/1009348591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f916415f0d0 0x7f91641503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:12.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.751+0000 7f9162d9d640 1 --2- 172.21.15.16:0/1009348591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91640d0070 0x7f91640b9c90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9154000c00 tx=0x7f91540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:12.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.751+0000 7f914b7fe640 1 -- 172.21.15.16:0/1009348591 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9154041070 con 0x7f91640d0070 2024-09-19T07:43:12.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.751+0000 7f914b7fe640 1 -- 172.21.15.16:0/1009348591 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f915403c870 con 0x7f91640d0070 2024-09-19T07:43:12.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.751+0000 7f916912c640 1 -- 172.21.15.16:0/1009348591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91640cac90 con 0x7f91640d0070 2024-09-19T07:43:12.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.751+0000 7f914b7fe640 1 -- 172.21.15.16:0/1009348591 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9154037040 con 0x7f91640d0070 2024-09-19T07:43:12.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.751+0000 7f916912c640 1 -- 172.21.15.16:0/1009348591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91640cb250 con 0x7f91640d0070 2024-09-19T07:43:12.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.752+0000 7f916912c640 1 -- 172.21.15.16:0/1009348591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91640d1400 con 0x7f91640d0070 2024-09-19T07:43:12.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.752+0000 7f914b7fe640 1 -- 172.21.15.16:0/1009348591 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9154044070 con 0x7f91640d0070 2024-09-19T07:43:12.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.753+0000 7f914b7fe640 1 --2- 172.21.15.16:0/1009348591 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f915006c2b0 0x7f915006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:12.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.753+0000 7f914b7fe640 1 -- 172.21.15.16:0/1009348591 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f91540ac4e0 con 0x7f91640d0070 2024-09-19T07:43:12.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.753+0000 7f916259c640 1 --2- 172.21.15.16:0/1009348591 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f915006c2b0 0x7f915006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:12.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.754+0000 7f916259c640 1 --2- 172.21.15.16:0/1009348591 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f915006c2b0 0x7f915006e770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f91641f7d90 tx=0x7f9158008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:12.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.757+0000 7f914b7fe640 1 -- 172.21.15.16:0/1009348591 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f915407aa40 con 0x7f91640d0070 2024-09-19T07:43:12.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.932+0000 7f916912c640 1 -- 172.21.15.16:0/1009348591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91640d4430 con 0x7f91640d0070 2024-09-19T07:43:12.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.933+0000 7f914b7fe640 1 -- 172.21.15.16:0/1009348591 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7f915407a340 con 0x7f91640d0070 2024-09-19T07:43:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:12.934 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:43:10.642082+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:43:03.485582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:43:04.603961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:43:08.299591+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:43:12.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.935+0000 7f916912c640 1 -- 172.21.15.16:0/1009348591 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f915006c2b0 msgr2=0x7f915006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:12.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.935+0000 7f916912c640 1 --2- 172.21.15.16:0/1009348591 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f915006c2b0 0x7f915006e770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f91641f7d90 tx=0x7f9158008040 comp rx=0 tx=0).stop 2024-09-19T07:43:12.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.935+0000 7f916912c640 1 -- 172.21.15.16:0/1009348591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91640d0070 msgr2=0x7f91640b9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:12.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.935+0000 7f916912c640 1 --2- 172.21.15.16:0/1009348591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91640d0070 0x7f91640b9c90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9154000c00 tx=0x7f91540046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.936+0000 7f916912c640 1 -- 172.21.15.16:0/1009348591 shutdown_connections 2024-09-19T07:43:12.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.936+0000 7f916912c640 1 --2- 172.21.15.16:0/1009348591 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f915006c2b0 0x7f915006e770 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.936+0000 7f916912c640 1 --2- 172.21.15.16:0/1009348591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f916415f0d0 0x7f91641503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.936+0000 7f916912c640 1 --2- 172.21.15.16:0/1009348591 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91640d1940 0x7f91640b81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.936+0000 7f916912c640 1 --2- 172.21.15.16:0/1009348591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91640d0070 0x7f91640b9c90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:12.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.936+0000 7f916912c640 1 -- 172.21.15.16:0/1009348591 >> 172.21.15.16:0/1009348591 conn(0x7f916409d760 msgr2=0x7f91641614c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:12.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.936+0000 7f916912c640 1 -- 172.21.15.16:0/1009348591 shutdown_connections 2024-09-19T07:43:12.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:12.936+0000 7f916912c640 1 -- 172.21.15.16:0/1009348591 wait complete. 2024-09-19T07:43:13.350 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T07:43:05.610038+0000', 'modified': '2024-09-19T07:43:10.642236+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4278}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4278': {'gid': 4278, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.146:6837/1303703249', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6836', 'nonce': 1303703249}, {'type': 'v1', 'addr': '172.21.15.146:6837', 'nonce': 1303703249}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4278, 'qdb_cluster': [4278]} 2024-09-19T07:43:13.350 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T07:43:13.350 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-19T07:43:13.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.457+0000 7f5bcd274640 1 Processor -- start 2024-09-19T07:43:13.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.457+0000 7f5bcd274640 1 -- start start 2024-09-19T07:43:13.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.458+0000 7f5bcd274640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5bc815cb00 0x7f5bc80d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.458+0000 7f5bcd274640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bc80d22d0 con 0x7f5bc80d04c0 2024-09-19T07:43:13.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.458+0000 7f5bcd274640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bc80d25b0 con 0x7f5bc815cee0 2024-09-19T07:43:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.458+0000 7f5bcd274640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5bc80b40b0 con 0x7f5bc815cb00 2024-09-19T07:43:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.458+0000 7f5bc77fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5bc80d04c0 0x7f5bc81562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49492/0 (socket says 172.21.15.16:49492) 2024-09-19T07:43:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.458+0000 7f5bc77fe640 1 -- 172.21.15.16:0/657082524 learned_addr learned my addr 172.21.15.16:0/657082524 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.458+0000 7f5bc5ffb640 1 -- 172.21.15.16:0/657082524 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2687521628 0 0) 0x7f5bc80d22d0 con 0x7f5bc80d04c0 2024-09-19T07:43:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.458+0000 7f5bc5ffb640 1 -- 172.21.15.16:0/657082524 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bb4003520 con 0x7f5bc80d04c0 2024-09-19T07:43:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.458+0000 7f5bc6ffd640 1 --2- 172.21.15.16:0/657082524 >> v2:172.21.15.146:3300/0 conn(0x7f5bc815cb00 0x7f5bc80d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.458+0000 7f5bc5ffb640 1 -- 172.21.15.16:0/657082524 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3040101148 0 0) 0x7f5bb4003520 con 0x7f5bc80d04c0 2024-09-19T07:43:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc5ffb640 1 -- 172.21.15.16:0/657082524 >> v2:172.21.15.146:3300/0 conn(0x7f5bc815cb00 msgr2=0x7f5bc80d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc5ffb640 1 --2- 172.21.15.16:0/657082524 >> v2:172.21.15.146:3300/0 conn(0x7f5bc815cb00 0x7f5bc80d1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc5ffb640 1 -- 172.21.15.16:0/657082524 >> v1:172.21.15.146:6789/0 conn(0x7f5bc815cee0 legacy=0x7f5bc80cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:13.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc5ffb640 1 -- 172.21.15.16:0/657082524 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc80d2890 con 0x7f5bc80d04c0 2024-09-19T07:43:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc5ffb640 1 -- 172.21.15.16:0/657082524 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f5bb8001670 con 0x7f5bc80d04c0 2024-09-19T07:43:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc5ffb640 1 -- 172.21.15.16:0/657082524 >> v1:172.21.15.16:6789/0 conn(0x7f5bc80d04c0 legacy=0x7f5bc81562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc5ffb640 1 --2- 172.21.15.16:0/657082524 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bb40039b0 0x7f5bb4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc5ffb640 1 --2- 172.21.15.16:0/657082524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bb4004800 0x7f5bb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc5ffb640 1 --2- 172.21.15.16:0/657082524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bb40051c0 0x7f5bb400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc5ffb640 1 -- 172.21.15.16:0/657082524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc80b40b0 con 0x7f5bb4004800 2024-09-19T07:43:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc5ffb640 1 -- 172.21.15.16:0/657082524 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb400c160 con 0x7f5bb40051c0 2024-09-19T07:43:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc5ffb640 1 -- 172.21.15.16:0/657082524 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb400c360 con 0x7f5bb40039b0 2024-09-19T07:43:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc67fc640 1 --2- 172.21.15.16:0/657082524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bb4004800 0x7f5bb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc6ffd640 1 --2- 172.21.15.16:0/657082524 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bb40039b0 0x7f5bb4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc77fe640 1 --2- 172.21.15.16:0/657082524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bb40051c0 0x7f5bb400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc67fc640 1 -- 172.21.15.16:0/657082524 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bb40039b0 msgr2=0x7f5bb4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc67fc640 1 --2- 172.21.15.16:0/657082524 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bb40039b0 0x7f5bb4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc67fc640 1 -- 172.21.15.16:0/657082524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bb40051c0 msgr2=0x7f5bb400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:13.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc67fc640 1 --2- 172.21.15.16:0/657082524 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bb40051c0 0x7f5bb400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc77fe640 1 -- 172.21.15.16:0/657082524 reap_dead start 2024-09-19T07:43:13.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.459+0000 7f5bc67fc640 1 -- 172.21.15.16:0/657082524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb4003520 con 0x7f5bb4004800 2024-09-19T07:43:13.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.460+0000 7f5bc67fc640 1 --2- 172.21.15.16:0/657082524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bb4004800 0x7f5bb4004c80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0001750 tx=0x7f5bb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce8d0c37e22cf81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:13.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.460+0000 7f5bc5ffb640 1 -- 172.21.15.16:0/657082524 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0001370 con 0x7f5bb4004800 2024-09-19T07:43:13.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.460+0000 7f5bc5ffb640 1 -- 172.21.15.16:0/657082524 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb00052d0 con 0x7f5bb4004800 2024-09-19T07:43:13.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.460+0000 7f5bc5ffb640 1 -- 172.21.15.16:0/657082524 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb002abc0 con 0x7f5bb4004800 2024-09-19T07:43:13.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.460+0000 7f5bcd274640 1 -- 172.21.15.16:0/657082524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bb4004800 msgr2=0x7f5bb4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:13.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.460+0000 7f5bcd274640 1 --2- 172.21.15.16:0/657082524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bb4004800 0x7f5bb4004c80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0001750 tx=0x7f5bb4005830 comp rx=0 tx=0).stop 2024-09-19T07:43:13.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.461+0000 7f5bcd274640 1 -- 172.21.15.16:0/657082524 shutdown_connections 2024-09-19T07:43:13.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.461+0000 7f5bcd274640 1 --2- 172.21.15.16:0/657082524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bb4004800 0x7f5bb4004c80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.461+0000 7f5bcd274640 1 -- 172.21.15.16:0/657082524 >> 172.21.15.16:0/657082524 conn(0x7f5bc8067fe0 msgr2=0x7f5bc806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.461+0000 7f5bcd274640 1 -- 172.21.15.16:0/657082524 shutdown_connections 2024-09-19T07:43:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.461+0000 7f5bcd274640 1 -- 172.21.15.16:0/657082524 wait complete. 2024-09-19T07:43:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.462+0000 7f5bcd274640 1 Processor -- start 2024-09-19T07:43:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.462+0000 7f5bcd274640 1 -- start start 2024-09-19T07:43:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bcd274640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bb4004800 0x7f5bc81f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bcd274640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bc8157830 0x7f5bc8155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bcd274640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc8155fd0 0x7f5bc815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bcd274640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc80d1e20 con 0x7f5bc8155fd0 2024-09-19T07:43:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bcd274640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc80d1ca0 con 0x7f5bb4004800 2024-09-19T07:43:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bcd274640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5bc80d1fa0 con 0x7f5bc8157830 2024-09-19T07:43:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bc77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc8155fd0 0x7f5bc815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bc77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc8155fd0 0x7f5bc815e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38976/0 (socket says 172.21.15.16:38976) 2024-09-19T07:43:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bc77fe640 1 -- 172.21.15.16:0/3601912365 learned_addr learned my addr 172.21.15.16:0/3601912365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bc6ffd640 1 --2- 172.21.15.16:0/3601912365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bb4004800 0x7f5bc81f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bc67fc640 1 --2- 172.21.15.16:0/3601912365 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bc8157830 0x7f5bc8155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bc77fe640 1 -- 172.21.15.16:0/3601912365 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bc8157830 msgr2=0x7f5bc8155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bc77fe640 1 --2- 172.21.15.16:0/3601912365 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bc8157830 0x7f5bc8155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bc77fe640 1 -- 172.21.15.16:0/3601912365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bb4004800 msgr2=0x7f5bc81f2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bc77fe640 1 --2- 172.21.15.16:0/3601912365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bb4004800 0x7f5bc81f2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bc77fe640 1 -- 172.21.15.16:0/3601912365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc80b62c0 con 0x7f5bc8155fd0 2024-09-19T07:43:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bc77fe640 1 --2- 172.21.15.16:0/3601912365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc8155fd0 0x7f5bc815e5d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8000c00 tx=0x7f5bb8007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bbf7fe640 1 -- 172.21.15.16:0/3601912365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb800f6a0 con 0x7f5bc8155fd0 2024-09-19T07:43:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bbf7fe640 1 -- 172.21.15.16:0/3601912365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb800f840 con 0x7f5bc8155fd0 2024-09-19T07:43:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bbf7fe640 1 -- 172.21.15.16:0/3601912365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb800fc10 con 0x7f5bc8155fd0 2024-09-19T07:43:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bcd274640 1 -- 172.21.15.16:0/3601912365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc80b5220 con 0x7f5bc8155fd0 2024-09-19T07:43:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.463+0000 7f5bcd274640 1 -- 172.21.15.16:0/3601912365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc80b57e0 con 0x7f5bc8155fd0 2024-09-19T07:43:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.465+0000 7f5bbf7fe640 1 -- 172.21.15.16:0/3601912365 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5bb80133a0 con 0x7f5bc8155fd0 2024-09-19T07:43:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.465+0000 7f5bbf7fe640 1 --2- 172.21.15.16:0/3601912365 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5ba806c2b0 0x7f5ba806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.465+0000 7f5bbf7fe640 1 -- 172.21.15.16:0/3601912365 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5bb808abc0 con 0x7f5bc8155fd0 2024-09-19T07:43:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.466+0000 7f5bc6ffd640 1 --2- 172.21.15.16:0/3601912365 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5ba806c2b0 0x7f5ba806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.466+0000 7f5bc6ffd640 1 --2- 172.21.15.16:0/3601912365 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5ba806c2b0 0x7f5ba806e770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0014510 tx=0x7f5bc000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.466+0000 7f5bd0016740 1 Processor -- start 2024-09-19T07:43:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.466+0000 7f5bd0016740 1 -- start start 2024-09-19T07:43:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.467+0000 7f5bd0016740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5565734e22c0 0x556573527a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.467+0000 7f5bd0016740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565734dfa80 0x5565734e26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.467+0000 7f5bd0016740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 0x5565734dc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.467+0000 7f5bd0016740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556573444520 con 0x5565734e3a50 2024-09-19T07:43:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.467+0000 7f5bd0016740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5565733ec3e0 con 0x5565734dfa80 2024-09-19T07:43:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.467+0000 7f5bd0016740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x5565734404e0 con 0x5565734e22c0 2024-09-19T07:43:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.467+0000 7f5bc77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 0x5565734dc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.467+0000 7f5bc6ffd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5565734e22c0 0x556573527a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.467+0000 7f5bc77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 0x5565734dc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38984/0 (socket says 172.21.15.16:38984) 2024-09-19T07:43:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.467+0000 7f5bc67fc640 1 --2- 172.21.15.16:0/1012337186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565734dfa80 0x5565734e26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.467+0000 7f5bc77fe640 1 -- 172.21.15.16:0/1012337186 learned_addr learned my addr 172.21.15.16:0/1012337186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:13.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.468+0000 7f5bc77fe640 1 -- 172.21.15.16:0/1012337186 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5565734e22c0 msgr2=0x556573527a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:13.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.468+0000 7f5bc77fe640 1 --2- 172.21.15.16:0/1012337186 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5565734e22c0 0x556573527a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.468+0000 7f5bc77fe640 1 -- 172.21.15.16:0/1012337186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565734dfa80 msgr2=0x5565734e26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:13.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.468+0000 7f5bc77fe640 1 --2- 172.21.15.16:0/1012337186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565734dfa80 0x5565734e26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.468+0000 7f5bc77fe640 1 -- 172.21.15.16:0/1012337186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565734072f0 con 0x5565734e3a50 2024-09-19T07:43:13.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.468+0000 7f5bc77fe640 1 --2- 172.21.15.16:0/1012337186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 0x5565734dc8b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80586d0 tx=0x7f5bb8054e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb08dfb8db37c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:13.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.468+0000 7f5bbcff9640 1 -- 172.21.15.16:0/1012337186 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8056ad0 con 0x5565734e3a50 2024-09-19T07:43:13.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.468+0000 7f5bbcff9640 1 -- 172.21.15.16:0/1012337186 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb8056d30 con 0x5565734e3a50 2024-09-19T07:43:13.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.468+0000 7f5bbcff9640 1 -- 172.21.15.16:0/1012337186 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb80550a0 con 0x5565734e3a50 2024-09-19T07:43:13.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.468+0000 7f5bd0016740 1 -- 172.21.15.16:0/1012337186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 msgr2=0x5565734dc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:13.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.469+0000 7f5bd0016740 1 --2- 172.21.15.16:0/1012337186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 0x5565734dc8b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80586d0 tx=0x7f5bb8054e00 comp rx=0 tx=0).stop 2024-09-19T07:43:13.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.469+0000 7f5bd0016740 1 -- 172.21.15.16:0/1012337186 shutdown_connections 2024-09-19T07:43:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.469+0000 7f5bd0016740 1 --2- 172.21.15.16:0/1012337186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 0x5565734dc8b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.469+0000 7f5bd0016740 1 --2- 172.21.15.16:0/1012337186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565734dfa80 0x5565734e26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.469+0000 7f5bd0016740 1 --2- 172.21.15.16:0/1012337186 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5565734e22c0 0x556573527a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.469+0000 7f5bd0016740 1 -- 172.21.15.16:0/1012337186 >> 172.21.15.16:0/1012337186 conn(0x5565734dea80 msgr2=0x556573526f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.469+0000 7f5bd0016740 1 -- 172.21.15.16:0/1012337186 shutdown_connections 2024-09-19T07:43:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.469+0000 7f5bd0016740 1 -- 172.21.15.16:0/1012337186 wait complete. 2024-09-19T07:43:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.469+0000 7f5bd0016740 1 -- start start 2024-09-19T07:43:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.469+0000 7f5bd0016740 1 Processor -- start 2024-09-19T07:43:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.470+0000 7f5bd0016740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5565734e22c0 0x556573536e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.470+0000 7f5bd0016740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 0x556573539b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.470+0000 7f5bd0016740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556573526270 0x5565735266b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.470+0000 7f5bd0016740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5565734ddf20 con 0x5565734e3a50 2024-09-19T07:43:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.470+0000 7f5bd0016740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5565733cd450 con 0x556573526270 2024-09-19T07:43:13.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.470+0000 7f5bd0016740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x556573528290 con 0x5565734e22c0 2024-09-19T07:43:13.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.470+0000 7f5bd0016740 10 client.0 _ll_register_callbacks cb 0x556573156ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:13.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.470+0000 7f5bc67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 0x556573539b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.470+0000 7f5bc6ffd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5565734e22c0 0x556573536e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.470+0000 7f5bc67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 0x556573539b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38992/0 (socket says 172.21.15.16:38992) 2024-09-19T07:43:13.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.470+0000 7f5bc77fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556573526270 0x5565735266b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.470+0000 7f5bc67fc640 1 -- 172.21.15.16:0/2284732230 learned_addr learned my addr 172.21.15.16:0/2284732230 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:13.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5bc67fc640 1 -- 172.21.15.16:0/2284732230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5565734e22c0 msgr2=0x556573536e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:13.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5bc67fc640 1 --2- 172.21.15.16:0/2284732230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5565734e22c0 0x556573536e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5bc67fc640 1 -- 172.21.15.16:0/2284732230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556573526270 msgr2=0x5565735266b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:13.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5bc67fc640 1 --2- 172.21.15.16:0/2284732230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556573526270 0x5565735266b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5bc67fc640 1 -- 172.21.15.16:0/2284732230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565731572d0 con 0x5565734e3a50 2024-09-19T07:43:13.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5bc67fc640 1 --2- 172.21.15.16:0/2284732230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 0x556573539b10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0000c00 tx=0x7f5bb0001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea204d32b4daea63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:13.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5ba5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:43:13.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5ba5ffb640 1 -- 172.21.15.16:0/2284732230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0037040 con 0x5565734e3a50 2024-09-19T07:43:13.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5ba5ffb640 1 -- 172.21.15.16:0/2284732230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5bb002ad90 con 0x5565734e3a50 2024-09-19T07:43:13.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5bd0016740 1 -- 172.21.15.16:0/2284732230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bb8056d30 con 0x5565734e3a50 2024-09-19T07:43:13.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5ba5ffb640 1 -- 172.21.15.16:0/2284732230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f5bb0001430 con 0x5565734e3a50 2024-09-19T07:43:13.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5bd0016740 10 client.4392 fetch_fsmap learned FSMap version 6 2024-09-19T07:43:13.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5bd0016740 1 -- 172.21.15.16:0/2284732230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x5565732216b0 con 0x5565734e3a50 2024-09-19T07:43:13.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5ba5ffb640 1 -- 172.21.15.16:0/2284732230 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb8056d30 con 0x5565734e3a50 2024-09-19T07:43:13.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5ba5ffb640 1 -- 172.21.15.16:0/2284732230 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 6) ==== 3510+0+0 (secure 0 0 0) 0x7f5bb0043020 con 0x5565734e3a50 2024-09-19T07:43:13.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5bd0016740 10 client.4392 fetch_fsmap finished waiting for FSMap version 6 2024-09-19T07:43:13.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5bd0016740 10 client.4392 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-19T07:43:13.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5bd0016740 20 client.4392 populate_metadata read hostname 'smithi016' 2024-09-19T07:43:13.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5bd0016740 1 --2- 172.21.15.16:0/2284732230 >> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] conn(0x556573509d90 0x55657350a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5bd0016740 4 client.4392 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:43:13.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.471+0000 7f5bd0016740 1 -- 172.21.15.16:0/2284732230 --> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556573451c50 con 0x556573509d90 2024-09-19T07:43:13.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.472+0000 7f5bc6ffd640 1 --2- 172.21.15.16:0/2284732230 >> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] conn(0x556573509d90 0x55657350a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.472+0000 7f5ba5ffb640 10 client.4392 ms_handle_connect on v2:172.21.15.146:6836/1303703249 2024-09-19T07:43:13.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.472+0000 7f5bc6ffd640 1 --2- 172.21.15.16:0/2284732230 >> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] conn(0x556573509d90 0x55657350a170 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a87b78e0e28ece0d server_cookie=6ac4e90d202eba85 in_seq=0 out_seq=0 2024-09-19T07:43:13.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.473+0000 7f5ba5ffb640 1 -- 172.21.15.16:0/2284732230 <== mds.0 v2:172.21.15.146:6836/1303703249 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556573451c50 con 0x556573509d90 2024-09-19T07:43:13.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.473+0000 7f5ba5ffb640 10 client.4392 handle_command_reply: tid=0 2024-09-19T07:43:13.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.473+0000 7f5bd0016740 1 client.4392 shutdown 2024-09-19T07:43:13.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.473+0000 7f5bd0016740 1 -- 172.21.15.16:0/2284732230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 msgr2=0x556573539b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:13.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.473+0000 7f5bd0016740 1 --2- 172.21.15.16:0/2284732230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 0x556573539b10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0000c00 tx=0x7f5bb0001190 comp rx=0 tx=0).stop 2024-09-19T07:43:13.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.473+0000 7f5bd0016740 1 -- 172.21.15.16:0/2284732230 shutdown_connections 2024-09-19T07:43:13.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.473+0000 7f5bd0016740 1 --2- 172.21.15.16:0/2284732230 >> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] conn(0x556573509d90 0x55657350a170 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.473+0000 7f5bd0016740 1 --2- 172.21.15.16:0/2284732230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556573526270 0x5565735266b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.473+0000 7f5bd0016740 1 --2- 172.21.15.16:0/2284732230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 0x556573539b10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.473+0000 7f5ba5ffb640 0 client.4392 ms_handle_reset on v2:172.21.15.146:6836/1303703249 2024-09-19T07:43:13.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.473+0000 7f5bd0016740 1 --2- 172.21.15.16:0/2284732230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5565734e22c0 0x556573536e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.473+0000 7f5bd0016740 1 -- 172.21.15.16:0/2284732230 >> 172.21.15.16:0/2284732230 conn(0x5565734dfa80 msgr2=0x556573528a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:13.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.473+0000 7f5bd0016740 1 -- 172.21.15.16:0/2284732230 shutdown_connections 2024-09-19T07:43:13.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.473+0000 7f5bd0016740 1 -- 172.21.15.16:0/2284732230 wait complete. 2024-09-19T07:43:13.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.474+0000 7f5bd0016740 20 client.4392 trim_cache size 0 max 16384 2024-09-19T07:43:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.491+0000 7f5bd0016740 1 Processor -- start 2024-09-19T07:43:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.492+0000 7f5bd0016740 1 -- start start 2024-09-19T07:43:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.492+0000 7f5bd0016740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5565734e22c0 0x55657350c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.492+0000 7f5bd0016740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 0x556573524fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.492+0000 7f5bd0016740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556573526270 0x5565735254f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.492+0000 7f5bd0016740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81ff390 con 0x5565734e3a50 2024-09-19T07:43:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.492+0000 7f5bd0016740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81fe700 con 0x556573526270 2024-09-19T07:43:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.492+0000 7f5bd0016740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5bc81fed50 con 0x5565734e22c0 2024-09-19T07:43:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.492+0000 7f5bc67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 0x556573524fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.492+0000 7f5bc67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 0x556573524fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39000/0 (socket says 172.21.15.16:39000) 2024-09-19T07:43:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.492+0000 7f5bc67fc640 1 -- 172.21.15.16:0/2625241946 learned_addr learned my addr 172.21.15.16:0/2625241946 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.492+0000 7f5bc67fc640 1 -- 172.21.15.16:0/2625241946 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5565734e22c0 msgr2=0x55657350c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.493+0000 7f5bc67fc640 1 --2- 172.21.15.16:0/2625241946 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5565734e22c0 0x55657350c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.493+0000 7f5bc67fc640 1 -- 172.21.15.16:0/2625241946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556573526270 msgr2=0x5565735254f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.493+0000 7f5bc67fc640 1 --2- 172.21.15.16:0/2625241946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556573526270 0x5565735254f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.493+0000 7f5bc67fc640 1 -- 172.21.15.16:0/2625241946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55657350c260 con 0x5565734e3a50 2024-09-19T07:43:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.493+0000 7f5bc67fc640 1 --2- 172.21.15.16:0/2625241946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 0x556573524fb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0000ec0 tx=0x7f5bb003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d919e50701d5187 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.493+0000 7f5ba7fff640 1 -- 172.21.15.16:0/2625241946 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0005470 con 0x5565734e3a50 2024-09-19T07:43:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.493+0000 7f5ba7fff640 1 -- 172.21.15.16:0/2625241946 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb002ab90 con 0x5565734e3a50 2024-09-19T07:43:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.493+0000 7f5ba7fff640 1 -- 172.21.15.16:0/2625241946 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0007390 con 0x5565734e3a50 2024-09-19T07:43:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.493+0000 7f5bd0016740 1 -- 172.21.15.16:0/2625241946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 msgr2=0x556573524fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.493+0000 7f5bd0016740 1 --2- 172.21.15.16:0/2625241946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 0x556573524fb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0000ec0 tx=0x7f5bb003e000 comp rx=0 tx=0).stop 2024-09-19T07:43:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.493+0000 7f5bd0016740 1 -- 172.21.15.16:0/2625241946 shutdown_connections 2024-09-19T07:43:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.493+0000 7f5bd0016740 1 --2- 172.21.15.16:0/2625241946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556573526270 0x5565735254f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.493+0000 7f5bd0016740 1 --2- 172.21.15.16:0/2625241946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565734e3a50 0x556573524fb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.493+0000 7f5bd0016740 1 --2- 172.21.15.16:0/2625241946 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5565734e22c0 0x55657350c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.493+0000 7f5bd0016740 1 -- 172.21.15.16:0/2625241946 >> 172.21.15.16:0/2625241946 conn(0x5565734dfa80 msgr2=0x5565734d8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.493+0000 7f5bd0016740 1 -- 172.21.15.16:0/2625241946 shutdown_connections 2024-09-19T07:43:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.493+0000 7f5bd0016740 1 -- 172.21.15.16:0/2625241946 wait complete. 2024-09-19T07:43:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.494+0000 7f5bd0016740 1 -- start start 2024-09-19T07:43:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.494+0000 7f5bd0016740 1 Processor -- start 2024-09-19T07:43:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5bd0016740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5565734e22c0 0x556573524fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5bd0016740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565734e3a50 0x5565735254f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5bd0016740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556573526270 0x5565735355c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5bd0016740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55657350cac0 con 0x556573526270 2024-09-19T07:43:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5bd0016740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5565735380e0 con 0x5565734e3a50 2024-09-19T07:43:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5bd0016740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55657350e0e0 con 0x5565734e22c0 2024-09-19T07:43:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5bd0016740 10 client.0 _ll_register_callbacks cb 0x7f5bc00095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5bc77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556573526270 0x5565735355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5bc77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556573526270 0x5565735355c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39002/0 (socket says 172.21.15.16:39002) 2024-09-19T07:43:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5bc77fe640 1 -- 172.21.15.16:0/1865672625 learned_addr learned my addr 172.21.15.16:0/1865672625 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5bc67fc640 1 --2- 172.21.15.16:0/1865672625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565734e3a50 0x5565735254f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5bc6ffd640 1 --2- 172.21.15.16:0/1865672625 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5565734e22c0 0x556573524fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5bc77fe640 1 -- 172.21.15.16:0/1865672625 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5565734e22c0 msgr2=0x556573524fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5bc77fe640 1 --2- 172.21.15.16:0/1865672625 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5565734e22c0 0x556573524fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5bc77fe640 1 -- 172.21.15.16:0/1865672625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565734e3a50 msgr2=0x5565735254f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5bc77fe640 1 --2- 172.21.15.16:0/1865672625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565734e3a50 0x5565735254f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5bc77fe640 1 -- 172.21.15.16:0/1865672625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565735499b0 con 0x556573526270 2024-09-19T07:43:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5bc67fc640 1 --2- 172.21.15.16:0/1865672625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565734e3a50 0x5565735254f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5bc77fe640 1 --2- 172.21.15.16:0/1865672625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556573526270 0x5565735355c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8006040 tx=0x7f5bb8058bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ca2f10ee8a1386 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.495+0000 7f5ba5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:43:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.496+0000 7f5ba5ffb640 1 -- 172.21.15.16:0/1865672625 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb800fdb0 con 0x556573526270 2024-09-19T07:43:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.496+0000 7f5ba5ffb640 1 -- 172.21.15.16:0/1865672625 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5bb805f0c0 con 0x556573526270 2024-09-19T07:43:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.496+0000 7f5bd0016740 1 -- 172.21.15.16:0/1865672625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bb002ab90 con 0x556573526270 2024-09-19T07:43:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.496+0000 7f5ba5ffb640 1 -- 172.21.15.16:0/1865672625 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f5bb8070be0 con 0x556573526270 2024-09-19T07:43:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.496+0000 7f5bd0016740 10 client.4398 fetch_fsmap learned FSMap version 6 2024-09-19T07:43:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.496+0000 7f5bd0016740 1 -- 172.21.15.16:0/1865672625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x556573536220 con 0x556573526270 2024-09-19T07:43:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.496+0000 7f5ba5ffb640 1 -- 172.21.15.16:0/1865672625 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb002ab90 con 0x556573526270 2024-09-19T07:43:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.496+0000 7f5ba5ffb640 1 -- 172.21.15.16:0/1865672625 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 6) ==== 3510+0+0 (secure 0 0 0) 0x7f5bb80610d0 con 0x556573526270 2024-09-19T07:43:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.496+0000 7f5bd0016740 10 client.4398 fetch_fsmap finished waiting for FSMap version 6 2024-09-19T07:43:13.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.496+0000 7f5bd0016740 10 client.4398 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-19T07:43:13.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.496+0000 7f5bd0016740 20 client.4398 populate_metadata read hostname 'smithi016' 2024-09-19T07:43:13.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.496+0000 7f5bd0016740 1 --2- 172.21.15.16:0/1865672625 >> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] conn(0x556573535b00 0x5565735a8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.496+0000 7f5bd0016740 4 client.4398 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:43:13.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.496+0000 7f5bd0016740 1 -- 172.21.15.16:0/1865672625 --> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] -- command(tid 0: {"prefix": "status"}) -- 0x55657344bfa0 con 0x556573535b00 2024-09-19T07:43:13.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.496+0000 7f5bc6ffd640 1 --2- 172.21.15.16:0/1865672625 >> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] conn(0x556573535b00 0x5565735a8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.497+0000 7f5ba5ffb640 10 client.4398 ms_handle_connect on v2:172.21.15.146:6836/1303703249 2024-09-19T07:43:13.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.497+0000 7f5bc6ffd640 1 --2- 172.21.15.16:0/1865672625 >> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] conn(0x556573535b00 0x5565735a8fb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e4b4d36aa4d29e server_cookie=7ebe9bd4acd8d4d3 in_seq=0 out_seq=0 2024-09-19T07:43:13.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.497+0000 7f5ba5ffb640 1 -- 172.21.15.16:0/1865672625 <== mds.0 v2:172.21.15.146:6836/1303703249 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55657344bfa0 con 0x556573535b00 2024-09-19T07:43:13.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.497+0000 7f5ba5ffb640 10 client.4398 handle_command_reply: tid=0 2024-09-19T07:43:13.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.497+0000 7f5bd0016740 1 client.4398 shutdown 2024-09-19T07:43:13.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.497+0000 7f5bd0016740 1 -- 172.21.15.16:0/1865672625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556573526270 msgr2=0x5565735355c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:13.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.497+0000 7f5bd0016740 1 --2- 172.21.15.16:0/1865672625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556573526270 0x5565735355c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8006040 tx=0x7f5bb8058bb0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.497+0000 7f5bd0016740 1 -- 172.21.15.16:0/1865672625 shutdown_connections 2024-09-19T07:43:13.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.497+0000 7f5bd0016740 1 --2- 172.21.15.16:0/1865672625 >> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] conn(0x556573535b00 0x5565735a8fb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.497+0000 7f5bd0016740 1 --2- 172.21.15.16:0/1865672625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556573526270 0x5565735355c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.497+0000 7f5bd0016740 1 --2- 172.21.15.16:0/1865672625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565734e3a50 0x5565735254f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.497+0000 7f5ba5ffb640 0 client.4398 ms_handle_reset on v2:172.21.15.146:6836/1303703249 2024-09-19T07:43:13.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.497+0000 7f5bd0016740 1 --2- 172.21.15.16:0/1865672625 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5565734e22c0 0x556573524fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.497+0000 7f5bd0016740 1 -- 172.21.15.16:0/1865672625 >> 172.21.15.16:0/1865672625 conn(0x5565734dfa80 msgr2=0x556573509010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:13.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.497+0000 7f5bd0016740 1 -- 172.21.15.16:0/1865672625 shutdown_connections 2024-09-19T07:43:13.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.497+0000 7f5bd0016740 1 -- 172.21.15.16:0/1865672625 wait complete. 2024-09-19T07:43:13.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.497+0000 7f5bd0016740 20 client.4398 trim_cache size 0 max 16384 2024-09-19T07:43:13.504 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:43:13.505 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb", 2024-09-19T07:43:13.505 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-19T07:43:13.505 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4278, 2024-09-19T07:43:13.505 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T07:43:13.505 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T07:43:13.505 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T07:43:13.505 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 7.2525832670000003, 2024-09-19T07:43:13.506 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 6, 2024-09-19T07:43:13.506 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 26, 2024-09-19T07:43:13.506 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 26, 2024-09-19T07:43:13.506 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 13.261399786 2024-09-19T07:43:13.506 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:43:13.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.498+0000 7f5bcd274640 1 -- 172.21.15.16:0/3601912365 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5ba806c2b0 msgr2=0x7f5ba806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:13.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.498+0000 7f5bcd274640 1 --2- 172.21.15.16:0/3601912365 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5ba806c2b0 0x7f5ba806e770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0014510 tx=0x7f5bc000b040 comp rx=0 tx=0).stop 2024-09-19T07:43:13.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.498+0000 7f5bcd274640 1 -- 172.21.15.16:0/3601912365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc8155fd0 msgr2=0x7f5bc815e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:13.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.498+0000 7f5bcd274640 1 --2- 172.21.15.16:0/3601912365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc8155fd0 0x7f5bc815e5d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8000c00 tx=0x7f5bb8007430 comp rx=0 tx=0).stop 2024-09-19T07:43:13.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.498+0000 7f5bcd274640 1 -- 172.21.15.16:0/3601912365 shutdown_connections 2024-09-19T07:43:13.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.498+0000 7f5bcd274640 1 --2- 172.21.15.16:0/3601912365 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5ba806c2b0 0x7f5ba806e770 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.498+0000 7f5bcd274640 1 --2- 172.21.15.16:0/3601912365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc8155fd0 0x7f5bc815e5d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.498+0000 7f5bcd274640 1 --2- 172.21.15.16:0/3601912365 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bc8157830 0x7f5bc8155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.498+0000 7f5bcd274640 1 --2- 172.21.15.16:0/3601912365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bb4004800 0x7f5bc81f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.498+0000 7f5bcd274640 1 -- 172.21.15.16:0/3601912365 >> 172.21.15.16:0/3601912365 conn(0x7f5bc8067fe0 msgr2=0x7f5bc80cc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:13.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.498+0000 7f5bcd274640 1 -- 172.21.15.16:0/3601912365 shutdown_connections 2024-09-19T07:43:13.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.498+0000 7f5bcd274640 1 -- 172.21.15.16:0/3601912365 wait complete. 2024-09-19T07:43:13.509 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T07:43:13.509 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-19T07:43:13.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:43:13.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.610+0000 7fae0775b640 1 Processor -- start 2024-09-19T07:43:13.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.611+0000 7fae0775b640 1 -- start start 2024-09-19T07:43:13.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.612+0000 7fae0775b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fae000b1550 0x7fae000aec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.612+0000 7fae0775b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae000b19e0 con 0x7fae000b27c0 2024-09-19T07:43:13.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.612+0000 7fae0775b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae000b1cc0 con 0x7fae0015b1d0 2024-09-19T07:43:13.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.612+0000 7fae0775b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fae000b2560 con 0x7fae000b1550 2024-09-19T07:43:13.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.612+0000 7fae04cf6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fae000b1550 0x7fae000aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.612+0000 7fae05cf8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fae000b27c0 0x7fae000af140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49498/0 (socket says 172.21.15.16:49498) 2024-09-19T07:43:13.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.612+0000 7fae05cf8640 1 -- 172.21.15.16:0/483817494 learned_addr learned my addr 172.21.15.16:0/483817494 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:13.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.612+0000 7fae04cf6640 1 -- 172.21.15.16:0/483817494 >> v1:172.21.15.146:6789/0 conn(0x7fae0015b1d0 legacy=0x7fae000b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:13.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.612+0000 7fae04cf6640 1 -- 172.21.15.16:0/483817494 >> v1:172.21.15.16:6789/0 conn(0x7fae000b27c0 legacy=0x7fae000af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:13.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.612+0000 7fae04cf6640 1 -- 172.21.15.16:0/483817494 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae000b1fa0 con 0x7fae000b1550 2024-09-19T07:43:13.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.612+0000 7fadfffff640 1 -- 172.21.15.16:0/483817494 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1974464313 0 0) 0x7fae000b19e0 con 0x7fae000b27c0 2024-09-19T07:43:13.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.613+0000 7fae04cf6640 1 --2- 172.21.15.16:0/483817494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae000b1550 0x7fae000aec00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fadf40097a0 tx=0x7fadf402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84f97b841a8b334 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:13.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.613+0000 7fadfffff640 1 -- 172.21.15.16:0/483817494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4037070 con 0x7fae000b1550 2024-09-19T07:43:13.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.613+0000 7fadfffff640 1 -- 172.21.15.16:0/483817494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf4032440 con 0x7fae000b1550 2024-09-19T07:43:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.613+0000 7fadfffff640 1 -- 172.21.15.16:0/483817494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4032810 con 0x7fae000b1550 2024-09-19T07:43:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.613+0000 7fae0775b640 1 -- 172.21.15.16:0/483817494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae000b1550 msgr2=0x7fae000aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.613+0000 7fae0775b640 1 --2- 172.21.15.16:0/483817494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae000b1550 0x7fae000aec00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fadf40097a0 tx=0x7fadf402a230 comp rx=0 tx=0).stop 2024-09-19T07:43:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.613+0000 7fae0775b640 1 -- 172.21.15.16:0/483817494 shutdown_connections 2024-09-19T07:43:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.613+0000 7fae0775b640 1 --2- 172.21.15.16:0/483817494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae000b1550 0x7fae000aec00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.613+0000 7fae0775b640 1 -- 172.21.15.16:0/483817494 >> 172.21.15.16:0/483817494 conn(0x7fae0009a9f0 msgr2=0x7fae000b6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.614+0000 7fae0775b640 1 -- 172.21.15.16:0/483817494 shutdown_connections 2024-09-19T07:43:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.614+0000 7fae0775b640 1 -- 172.21.15.16:0/483817494 wait complete. 2024-09-19T07:43:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.615+0000 7fae0775b640 1 Processor -- start 2024-09-19T07:43:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.615+0000 7fae0775b640 1 -- start start 2024-09-19T07:43:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.615+0000 7fae0775b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae000b27c0 0x7fae001576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fae0775b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae0015b1d0 0x7fae0009bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fae0775b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae0009c310 0x7fae0009cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fae0775b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fae000b5130 con 0x7fae000b27c0 2024-09-19T07:43:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fae0775b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fae0015b5b0 con 0x7fae0009c310 2024-09-19T07:43:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fae0775b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fae000b52b0 con 0x7fae0015b1d0 2024-09-19T07:43:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fae054f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae000b27c0 0x7fae001576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fae054f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae000b27c0 0x7fae001576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39030/0 (socket says 172.21.15.16:39030) 2024-09-19T07:43:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fae054f7640 1 -- 172.21.15.16:0/3929077207 learned_addr learned my addr 172.21.15.16:0/3929077207 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fae05cf8640 1 --2- 172.21.15.16:0/3929077207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae0009c310 0x7fae0009cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fae054f7640 1 -- 172.21.15.16:0/3929077207 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae0015b1d0 msgr2=0x7fae0009bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fae054f7640 1 --2- 172.21.15.16:0/3929077207 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae0015b1d0 0x7fae0009bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fae054f7640 1 -- 172.21.15.16:0/3929077207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae0009c310 msgr2=0x7fae0009cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fae054f7640 1 --2- 172.21.15.16:0/3929077207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae0009c310 0x7fae0009cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fae054f7640 1 -- 172.21.15.16:0/3929077207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0009d8e0 con 0x7fae000b27c0 2024-09-19T07:43:13.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fae05cf8640 1 --2- 172.21.15.16:0/3929077207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae0009c310 0x7fae0009cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:13.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fae054f7640 1 --2- 172.21.15.16:0/3929077207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae000b27c0 0x7fae001576f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fadf800c450 tx=0x7fadf800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:13.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fadfdffb640 1 -- 172.21.15.16:0/3929077207 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8017070 con 0x7fae000b27c0 2024-09-19T07:43:13.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fadfdffb640 1 -- 172.21.15.16:0/3929077207 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf80052d0 con 0x7fae000b27c0 2024-09-19T07:43:13.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fadfdffb640 1 -- 172.21.15.16:0/3929077207 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8003830 con 0x7fae000b27c0 2024-09-19T07:43:13.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.616+0000 7fae0775b640 1 -- 172.21.15.16:0/3929077207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0009b2f0 con 0x7fae000b27c0 2024-09-19T07:43:13.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.617+0000 7fae0775b640 1 -- 172.21.15.16:0/3929077207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0009b8b0 con 0x7fae000b27c0 2024-09-19T07:43:13.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.618+0000 7fae0775b640 1 -- 172.21.15.16:0/3929077207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae0009bab0 con 0x7fae000b27c0 2024-09-19T07:43:13.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.618+0000 7fadfdffb640 1 -- 172.21.15.16:0/3929077207 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fadf801a070 con 0x7fae000b27c0 2024-09-19T07:43:13.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.618+0000 7fadfdffb640 1 --2- 172.21.15.16:0/3929077207 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fadec06c2b0 0x7fadec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:13.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.618+0000 7fadfdffb640 1 -- 172.21.15.16:0/3929077207 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fadf808ab50 con 0x7fae000b27c0 2024-09-19T07:43:13.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.619+0000 7fae04cf6640 1 --2- 172.21.15.16:0/3929077207 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fadec06c2b0 0x7fadec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:13.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.619+0000 7fae04cf6640 1 --2- 172.21.15.16:0/3929077207 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fadec06c2b0 0x7fadec06e770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fadf4034000 tx=0x7fadf402c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:13.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.623+0000 7fadfdffb640 1 -- 172.21.15.16:0/3929077207 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fadf8058f80 con 0x7fae000b27c0 2024-09-19T07:43:13.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.832+0000 7fae0775b640 1 -- 172.21.15.16:0/3929077207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fae000b4010 con 0x7fae000b27c0 2024-09-19T07:43:13.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.832+0000 7fadfdffb640 1 -- 172.21.15.16:0/3929077207 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+88 (secure 0 0 0) 0x7fadf8058880 con 0x7fae000b27c0 2024-09-19T07:43:13.833 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-19T07:43:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.834+0000 7fae0775b640 1 -- 172.21.15.16:0/3929077207 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fadec06c2b0 msgr2=0x7fadec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.834+0000 7fae0775b640 1 --2- 172.21.15.16:0/3929077207 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fadec06c2b0 0x7fadec06e770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fadf4034000 tx=0x7fadf402c000 comp rx=0 tx=0).stop 2024-09-19T07:43:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.834+0000 7fae0775b640 1 -- 172.21.15.16:0/3929077207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae000b27c0 msgr2=0x7fae001576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.834+0000 7fae0775b640 1 --2- 172.21.15.16:0/3929077207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae000b27c0 0x7fae001576f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fadf800c450 tx=0x7fadf800c910 comp rx=0 tx=0).stop 2024-09-19T07:43:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.834+0000 7fae0775b640 1 -- 172.21.15.16:0/3929077207 shutdown_connections 2024-09-19T07:43:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.834+0000 7fae0775b640 1 --2- 172.21.15.16:0/3929077207 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fadec06c2b0 0x7fadec06e770 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.834+0000 7fae0775b640 1 --2- 172.21.15.16:0/3929077207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae0009c310 0x7fae0009cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.834+0000 7fae0775b640 1 --2- 172.21.15.16:0/3929077207 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae0015b1d0 0x7fae0009bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.834+0000 7fae0775b640 1 --2- 172.21.15.16:0/3929077207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae000b27c0 0x7fae001576f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.834+0000 7fae0775b640 1 -- 172.21.15.16:0/3929077207 >> 172.21.15.16:0/3929077207 conn(0x7fae0009a9f0 msgr2=0x7fae00156700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.834+0000 7fae0775b640 1 -- 172.21.15.16:0/3929077207 shutdown_connections 2024-09-19T07:43:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13.834+0000 7fae0775b640 1 -- 172.21.15.16:0/3929077207 wait complete. 2024-09-19T07:43:13.846 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-19T07:43:13.846 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:43:13.846 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T07:43:13.846 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-19T07:43:13.846 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-19T07:43:13.846 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-19T07:43:13.846 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:43:13.846 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:43:13.846 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-19T07:43:13.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:13.847 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T07:43:13.863 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:43:13.863 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:43:13.863 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:43:13.863 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:13.863 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:43:13.863 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:13.863 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:43:13.863 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:13.863 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T07:43:13.864 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:43:13.864 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:13.864 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T07:43:13.864 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:43:13.864 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:13.864 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T07:43:13.864 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T07:43:13.864 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42795sec preferred_lft 42795sec 2024-09-19T07:43:13.864 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T07:43:13.864 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:13.864 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:43:13.864 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:13.864 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T07:43:13.865 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T07:43:13.865 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:13.866 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:43:13.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-09-19T07:43:13.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-09-19T07:43:13.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-09-19T07:43:13.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T07:43:13.866 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:43:13.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:14.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:14.075 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:43:14.101 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:43:14.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:14.103 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T07:43:14.172 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T07:43:14.173 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:43:14.173 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T07:43:14.173 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T07:43:14.173 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:43:14.173 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:14.173 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:43:14.173 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T07:43:14.174 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T07:43:14.174 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T07:43:14.174 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:43:14.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:14.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:14.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:14.479 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:43:14.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:14.496 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-19T07:43:14.554 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:14.554 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:43:14.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T07:43:14.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-19T07:43:14.555 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:43:14.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:14.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:14.675 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-19T07:43:14.676 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:14.676 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:43:14.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-19T07:43:14.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T07:43:14.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-19T07:43:14.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-19T07:43:14.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-19T07:43:14.676 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:43:14.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:14.805 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:14.805 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:43:14.805 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-19T07:43:14.805 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T07:43:14.805 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:43:14.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:14.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:14.884 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-19T07:43:14.884 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:43:14.884 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-19T07:43:14.900 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-19T07:43:14.900 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:43:14.901 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-19T07:43:14.956 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:43:15.023 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:43:15.080 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:43:15.080 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:43:15.080 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:43:15.080 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:43:15.080 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:43:15.080 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:43:15.080 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:43:15.080 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:43:15.080 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:43:15.080 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:43:15.081 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:43:15.081 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:43:15.081 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:43:15.081 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:43:15.081 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:43:15.081 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:43:15.081 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:43:15.081 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:43:15.081 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:43:15.081 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:43:15.081 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:43:15.081 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:43:15.082 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:43:15.082 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:43:15.082 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:43:15.082 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:43:15.082 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:43:15.082 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:43:15.082 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:43:15.082 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:43:15.082 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T07:43:15.082 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T07:43:15.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:15.083 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:43:15.139 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:43:15.140 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-19T07:43:15.183 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:43:15.209 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:43:15.266 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:43:15.266 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:43:15.267 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:43:15.267 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:43:15.267 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:43:15.267 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:43:15.267 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:43:15.267 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:43:15.267 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:43:15.268 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:43:15.268 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:43:15.268 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:43:15.268 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:43:15.268 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:43:15.268 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:43:15.269 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:43:15.269 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:43:15.269 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:43:15.269 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:43:15.269 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:43:15.269 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:43:15.269 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:43:15.269 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:43:15.270 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:43:15.270 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:43:15.270 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:43:15.270 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:43:15.270 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:43:15.270 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:43:15.270 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:43:15.270 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T07:43:15.270 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T07:43:15.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:15.271 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:43:15.278 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:2024-09-19T07:43:15.277+0000 7f062e036480 -1 init, newargv = 0x560d9e7817f0 newargc=15 2024-09-19T07:43:15.278 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[42124]: starting ceph client 2024-09-19T07:43:15.284 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[42124]: starting fuse 2024-09-19T07:43:15.294 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:43:15.294 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-19T07:43:15.294 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:43:15.295 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:43:15.295 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:43:15.295 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:43:15.295 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:43:15.295 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:43:15.295 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:43:15.295 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-19T07:43:15.295 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:43:15.295 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-19T07:43:15.295 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:43:15.295 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:43:15.295 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:43:15.295 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:43:15.295 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:43:15.296 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:43:15.296 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:43:15.296 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:43:15.296 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:43:15.296 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:43:15.296 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:43:15.296 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:43:15.296 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:43:15.296 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:43:15.296 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:43:15.296 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.admin")) 2024-09-19T07:43:15.296 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:43:15.414 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.admin.42124.asok 2024-09-19T07:43:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:15.425 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.42124.asok 2024-09-19T07:43:15.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:15.426 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.42124.asok status 2024-09-19T07:43:15.563 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:43:15.563 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:43:15.563 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:43:15.563 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:43:15.564 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "admin", 2024-09-19T07:43:15.564 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:43:15.564 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-19T07:43:15.564 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "42124", 2024-09-19T07:43:15.564 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:43:15.564 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:15.564 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-19T07:43:15.564 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-19T07:43:15.564 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4413, 2024-09-19T07:43:15.564 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:43:15.565 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:43:15.565 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:43:15.565 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4413 2024-09-19T07:43:15.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:15.565 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:43:15.565 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:43:15.565 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:43:15.565 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 412603654 2024-09-19T07:43:15.565 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:15.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:15.566 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:43:15.566 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:43:15.566 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:43:15.566 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 412603654 2024-09-19T07:43:15.566 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:15.566 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.4413 192.168.0.1:0/412603654", 2024-09-19T07:43:15.567 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/412603654", 2024-09-19T07:43:15.567 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-19T07:43:15.567 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 6, 2024-09-19T07:43:15.567 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 26, 2024-09-19T07:43:15.567 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:43:15.567 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:43:15.568 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:43:15.568 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:43:15.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:15.572 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-19T07:43:15.572 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-19T07:43:15.589 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-19T07:43:15.589 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-19T07:43:15.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:15.589 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-19T07:43:15.657 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-19T07:43:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T07:43:15.767 INFO:teuthology.orchestra.run.smithi016.stdout:# file: . 2024-09-19T07:43:15.768 INFO:teuthology.orchestra.run.smithi016.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-19T07:43:15.768 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:15.774 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:43:15.774 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:43:15.791 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:43:15.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.934+0000 7ff1a759e640 1 Processor -- start 2024-09-19T07:43:15.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.934+0000 7ff1a759e640 1 -- start start 2024-09-19T07:43:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.935+0000 7ff1a759e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1a81618d0 0x7ff1a80cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.935+0000 7ff1a759e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff1a80b2e60 0x7ff1a80d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.935+0000 7ff1a759e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1a80cf430 con 0x7ff1a80d2940 2024-09-19T07:43:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.935+0000 7ff1a759e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff1a80cf710 con 0x7ff1a80b2e60 2024-09-19T07:43:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.935+0000 7ff1a759e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff1a80cf9f0 con 0x7ff1a81618d0 2024-09-19T07:43:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.935+0000 7ff1a5d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff1a80b2e60 0x7ff1a80d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.935+0000 7ff1a6d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff1a80d2940 0x7ff1a80d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49506/0 (socket says 172.21.15.16:49506) 2024-09-19T07:43:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.935+0000 7ff1a6d9d640 1 -- 172.21.15.16:0/1656268 learned_addr learned my addr 172.21.15.16:0/1656268 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.935+0000 7ff1a659c640 1 --2- 172.21.15.16:0/1656268 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1a81618d0 0x7ff1a80cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:15.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.935+0000 7ff1a5d9b640 1 -- 172.21.15.16:0/1656268 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1a81618d0 msgr2=0x7ff1a80cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:15.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.935+0000 7ff1a5d9b640 1 --2- 172.21.15.16:0/1656268 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1a81618d0 0x7ff1a80cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:15.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.935+0000 7ff1a5d9b640 1 -- 172.21.15.16:0/1656268 >> v1:172.21.15.16:6789/0 conn(0x7ff1a80d2940 legacy=0x7ff1a80d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:15.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.935+0000 7ff1a5d9b640 1 -- 172.21.15.16:0/1656268 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a80cfcd0 con 0x7ff1a80b2e60 2024-09-19T07:43:15.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.935+0000 7ff1a559a640 1 -- 172.21.15.16:0/1656268 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4250715013 0 0) 0x7ff1a80cf430 con 0x7ff1a80d2940 2024-09-19T07:43:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.936+0000 7ff1a5d9b640 1 --2- 172.21.15.16:0/1656268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a80b2e60 0x7ff1a80d2400 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0097a0 tx=0x7ff19c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4823c9412a8260 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.936+0000 7ff1a559a640 1 -- 172.21.15.16:0/1656268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c037070 con 0x7ff1a80b2e60 2024-09-19T07:43:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.936+0000 7ff1a559a640 1 -- 172.21.15.16:0/1656268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff19c032440 con 0x7ff1a80b2e60 2024-09-19T07:43:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.936+0000 7ff1a559a640 1 -- 172.21.15.16:0/1656268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c032810 con 0x7ff1a80b2e60 2024-09-19T07:43:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.937+0000 7ff1a759e640 1 -- 172.21.15.16:0/1656268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a80b2e60 msgr2=0x7ff1a80d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.937+0000 7ff1a759e640 1 --2- 172.21.15.16:0/1656268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a80b2e60 0x7ff1a80d2400 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0097a0 tx=0x7ff19c02a180 comp rx=0 tx=0).stop 2024-09-19T07:43:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.937+0000 7ff1a759e640 1 -- 172.21.15.16:0/1656268 shutdown_connections 2024-09-19T07:43:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.937+0000 7ff1a759e640 1 --2- 172.21.15.16:0/1656268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a80b2e60 0x7ff1a80d2400 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.937+0000 7ff1a759e640 1 --2- 172.21.15.16:0/1656268 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1a81618d0 0x7ff1a80cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.937+0000 7ff1a759e640 1 -- 172.21.15.16:0/1656268 >> 172.21.15.16:0/1656268 conn(0x7ff1a809d510 msgr2=0x7ff1a80d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.937+0000 7ff1a759e640 1 -- 172.21.15.16:0/1656268 shutdown_connections 2024-09-19T07:43:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.937+0000 7ff1a759e640 1 -- 172.21.15.16:0/1656268 wait complete. 2024-09-19T07:43:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.938+0000 7ff1a759e640 1 Processor -- start 2024-09-19T07:43:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.938+0000 7ff1a759e640 1 -- start start 2024-09-19T07:43:15.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.939+0000 7ff1a759e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1a80b2e60 0x7ff1a8153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.939+0000 7ff1a759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a80d2940 0x7ff1a80cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.939+0000 7ff1a759e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1a81618d0 0x7ff1a80b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.939+0000 7ff1a759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a80d3200 con 0x7ff1a80d2940 2024-09-19T07:43:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.939+0000 7ff1a759e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a80d3080 con 0x7ff1a81618d0 2024-09-19T07:43:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.939+0000 7ff1a759e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff1a80b1e10 con 0x7ff1a80b2e60 2024-09-19T07:43:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.939+0000 7ff1a5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a80d2940 0x7ff1a80cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.939+0000 7ff1a5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a80d2940 0x7ff1a80cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39036/0 (socket says 172.21.15.16:39036) 2024-09-19T07:43:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.939+0000 7ff1a5d9b640 1 -- 172.21.15.16:0/2871844523 learned_addr learned my addr 172.21.15.16:0/2871844523 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.939+0000 7ff1a659c640 1 --2- 172.21.15.16:0/2871844523 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1a80b2e60 0x7ff1a8153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.939+0000 7ff1a6d9d640 1 --2- 172.21.15.16:0/2871844523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1a81618d0 0x7ff1a80b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.939+0000 7ff1a5d9b640 1 -- 172.21.15.16:0/2871844523 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1a80b2e60 msgr2=0x7ff1a8153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.939+0000 7ff1a5d9b640 1 --2- 172.21.15.16:0/2871844523 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1a80b2e60 0x7ff1a8153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.939+0000 7ff1a5d9b640 1 -- 172.21.15.16:0/2871844523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1a81618d0 msgr2=0x7ff1a80b8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.939+0000 7ff1a5d9b640 1 --2- 172.21.15.16:0/2871844523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1a81618d0 0x7ff1a80b8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.939+0000 7ff1a5d9b640 1 -- 172.21.15.16:0/2871844523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a80ccdd0 con 0x7ff1a80d2940 2024-09-19T07:43:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.939+0000 7ff1a6d9d640 1 --2- 172.21.15.16:0/2871844523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1a81618d0 0x7ff1a80b8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.939+0000 7ff1a5d9b640 1 --2- 172.21.15.16:0/2871844523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a80d2940 0x7ff1a80cc890 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0098c0 tx=0x7ff19c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.940+0000 7ff196ffd640 1 -- 172.21.15.16:0/2871844523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c037040 con 0x7ff1a80d2940 2024-09-19T07:43:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.940+0000 7ff196ffd640 1 -- 172.21.15.16:0/2871844523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff19c030770 con 0x7ff1a80d2940 2024-09-19T07:43:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.940+0000 7ff196ffd640 1 -- 172.21.15.16:0/2871844523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c030b40 con 0x7ff1a80d2940 2024-09-19T07:43:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.940+0000 7ff1a759e640 1 -- 172.21.15.16:0/2871844523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a80cb7f0 con 0x7ff1a80d2940 2024-09-19T07:43:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.940+0000 7ff1a759e640 1 -- 172.21.15.16:0/2871844523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a80cbdb0 con 0x7ff1a80d2940 2024-09-19T07:43:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.941+0000 7ff1a759e640 1 -- 172.21.15.16:0/2871844523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1a8161cb0 con 0x7ff1a80d2940 2024-09-19T07:43:15.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.941+0000 7ff196ffd640 1 -- 172.21.15.16:0/2871844523 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff19c044020 con 0x7ff1a80d2940 2024-09-19T07:43:15.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.942+0000 7ff196ffd640 1 --2- 172.21.15.16:0/2871844523 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff18806c2b0 0x7ff18806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:15.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.942+0000 7ff196ffd640 1 -- 172.21.15.16:0/2871844523 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff19c0abbf0 con 0x7ff1a80d2940 2024-09-19T07:43:15.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.942+0000 7ff1a659c640 1 --2- 172.21.15.16:0/2871844523 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff18806c2b0 0x7ff18806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:15.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.942+0000 7ff1a659c640 1 --2- 172.21.15.16:0/2871844523 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff18806c2b0 0x7ff18806e770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff19800aac0 tx=0x7ff198008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:15.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:15.946+0000 7ff196ffd640 1 -- 172.21.15.16:0/2871844523 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff19c032e20 con 0x7ff1a80d2940 2024-09-19T07:43:16.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.137+0000 7ff1a759e640 1 -- 172.21.15.16:0/2871844523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff1a80754c0 con 0x7ff1a80d2940 2024-09-19T07:43:16.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.137+0000 7ff196ffd640 1 -- 172.21.15.16:0/2871844523 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7ff19c07a0b0 con 0x7ff1a80d2940 2024-09-19T07:43:16.138 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:43:16.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.139+0000 7ff1a759e640 1 -- 172.21.15.16:0/2871844523 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff18806c2b0 msgr2=0x7ff18806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:16.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.139+0000 7ff1a759e640 1 --2- 172.21.15.16:0/2871844523 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff18806c2b0 0x7ff18806e770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff19800aac0 tx=0x7ff198008040 comp rx=0 tx=0).stop 2024-09-19T07:43:16.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.139+0000 7ff1a759e640 1 -- 172.21.15.16:0/2871844523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a80d2940 msgr2=0x7ff1a80cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:16.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.139+0000 7ff1a759e640 1 --2- 172.21.15.16:0/2871844523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a80d2940 0x7ff1a80cc890 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0098c0 tx=0x7ff19c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:16.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.139+0000 7ff1a759e640 1 -- 172.21.15.16:0/2871844523 shutdown_connections 2024-09-19T07:43:16.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.139+0000 7ff1a759e640 1 --2- 172.21.15.16:0/2871844523 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff18806c2b0 0x7ff18806e770 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:16.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.139+0000 7ff1a759e640 1 --2- 172.21.15.16:0/2871844523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1a81618d0 0x7ff1a80b8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:16.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.139+0000 7ff1a759e640 1 --2- 172.21.15.16:0/2871844523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a80d2940 0x7ff1a80cc890 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:16.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.139+0000 7ff1a759e640 1 --2- 172.21.15.16:0/2871844523 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1a80b2e60 0x7ff1a8153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:16.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.139+0000 7ff1a759e640 1 -- 172.21.15.16:0/2871844523 >> 172.21.15.16:0/2871844523 conn(0x7ff1a809d510 msgr2=0x7ff1a806e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:16.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.139+0000 7ff1a759e640 1 -- 172.21.15.16:0/2871844523 shutdown_connections 2024-09-19T07:43:16.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.139+0000 7ff1a759e640 1 -- 172.21.15.16:0/2871844523 wait complete. 2024-09-19T07:43:16.151 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:43:16.151 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:43:16.167 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:43:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.310+0000 7f5e49a2e640 1 Processor -- start 2024-09-19T07:43:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.310+0000 7f5e49a2e640 1 -- start start 2024-09-19T07:43:16.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.311+0000 7f5e49a2e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e4415ee60 0x7f5e44161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:16.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.311+0000 7f5e49a2e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e4415f240 0x7f5e440b4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:16.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.311+0000 7f5e49a2e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5e440b4660 0x7f5e4406e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:16.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.311+0000 7f5e49a2e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f5e440b39b0 con 0x7f5e4415ee60 2024-09-19T07:43:16.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.311+0000 7f5e49a2e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5e44162340 con 0x7f5e440b4660 2024-09-19T07:43:16.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.311+0000 7f5e49a2e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5e44162610 con 0x7f5e4415f240 2024-09-19T07:43:16.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.311+0000 7f5e42ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e4415ee60 0x7f5e44161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:16.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.311+0000 7f5e42ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e4415ee60 0x7f5e44161e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39046/0 (socket says 172.21.15.16:39046) 2024-09-19T07:43:16.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.311+0000 7f5e42ffd640 1 -- 172.21.15.16:0/2417333345 learned_addr learned my addr 172.21.15.16:0/2417333345 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:16.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.311+0000 7f5e437fe640 1 --2- 172.21.15.16:0/2417333345 >> v2:172.21.15.146:3300/0 conn(0x7f5e440b4660 0x7f5e4406e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:16.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.311+0000 7f5e427fc640 1 --2- 172.21.15.16:0/2417333345 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e4415f240 0x7f5e440b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:16.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.312+0000 7f5e42ffd640 1 -- 172.21.15.16:0/2417333345 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e4415f240 msgr2=0x7f5e440b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:16.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.312+0000 7f5e42ffd640 1 --2- 172.21.15.16:0/2417333345 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e4415f240 0x7f5e440b4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:16.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.312+0000 7f5e42ffd640 1 -- 172.21.15.16:0/2417333345 >> v2:172.21.15.146:3300/0 conn(0x7f5e440b4660 msgr2=0x7f5e4406e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:16.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.312+0000 7f5e42ffd640 1 --2- 172.21.15.16:0/2417333345 >> v2:172.21.15.146:3300/0 conn(0x7f5e440b4660 0x7f5e4406e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:16.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.312+0000 7f5e42ffd640 1 -- 172.21.15.16:0/2417333345 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e441628f0 con 0x7f5e4415ee60 2024-09-19T07:43:16.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.312+0000 7f5e427fc640 1 --2- 172.21.15.16:0/2417333345 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e4415f240 0x7f5e440b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:16.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.312+0000 7f5e42ffd640 1 --2- 172.21.15.16:0/2417333345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4415ee60 0x7f5e44161e00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5e440ccb40 tx=0x7f5e3c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd906483af45fae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:16.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.312+0000 7f5e41ffb640 1 -- 172.21.15.16:0/2417333345 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c037070 con 0x7f5e4415ee60 2024-09-19T07:43:16.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.313+0000 7f5e41ffb640 1 -- 172.21.15.16:0/2417333345 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e3c032440 con 0x7f5e4415ee60 2024-09-19T07:43:16.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.313+0000 7f5e41ffb640 1 -- 172.21.15.16:0/2417333345 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c032810 con 0x7f5e4415ee60 2024-09-19T07:43:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.313+0000 7f5e49a2e640 1 -- 172.21.15.16:0/2417333345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4415ee60 msgr2=0x7f5e44161e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.313+0000 7f5e49a2e640 1 --2- 172.21.15.16:0/2417333345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4415ee60 0x7f5e44161e00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5e440ccb40 tx=0x7f5e3c02a140 comp rx=0 tx=0).stop 2024-09-19T07:43:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.313+0000 7f5e49a2e640 1 -- 172.21.15.16:0/2417333345 shutdown_connections 2024-09-19T07:43:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.313+0000 7f5e49a2e640 1 --2- 172.21.15.16:0/2417333345 >> v2:172.21.15.146:3300/0 conn(0x7f5e440b4660 0x7f5e4406e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.313+0000 7f5e49a2e640 1 --2- 172.21.15.16:0/2417333345 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e4415f240 0x7f5e440b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.313+0000 7f5e49a2e640 1 --2- 172.21.15.16:0/2417333345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4415ee60 0x7f5e44161e00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.313+0000 7f5e49a2e640 1 -- 172.21.15.16:0/2417333345 >> 172.21.15.16:0/2417333345 conn(0x7f5e440729a0 msgr2=0x7f5e44072db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:16.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.313+0000 7f5e49a2e640 1 -- 172.21.15.16:0/2417333345 shutdown_connections 2024-09-19T07:43:16.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.313+0000 7f5e49a2e640 1 -- 172.21.15.16:0/2417333345 wait complete. 2024-09-19T07:43:16.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.314+0000 7f5e49a2e640 1 Processor -- start 2024-09-19T07:43:16.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.314+0000 7f5e49a2e640 1 -- start start 2024-09-19T07:43:16.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.315+0000 7f5e49a2e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e440b4660 0x7f5e441554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:16.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.315+0000 7f5e49a2e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e4415ee60 0x7f5e4406ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:16.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.315+0000 7f5e49a2e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4415f240 0x7f5e4406f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:16.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.315+0000 7f5e49a2e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4415f6a0 con 0x7f5e4415f240 2024-09-19T07:43:16.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.315+0000 7f5e49a2e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5e440b3c00 con 0x7f5e4415ee60 2024-09-19T07:43:16.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.315+0000 7f5e49a2e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5e441559f0 con 0x7f5e440b4660 2024-09-19T07:43:16.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.315+0000 7f5e437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4415f240 0x7f5e4406f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:16.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.315+0000 7f5e437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4415f240 0x7f5e4406f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39052/0 (socket says 172.21.15.16:39052) 2024-09-19T07:43:16.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.315+0000 7f5e437fe640 1 -- 172.21.15.16:0/1004787165 learned_addr learned my addr 172.21.15.16:0/1004787165 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:16.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.315+0000 7f5e42ffd640 1 --2- 172.21.15.16:0/1004787165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e440b4660 0x7f5e441554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:16.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.315+0000 7f5e427fc640 1 --2- 172.21.15.16:0/1004787165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e4415ee60 0x7f5e4406ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:16.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.316+0000 7f5e437fe640 1 -- 172.21.15.16:0/1004787165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e440b4660 msgr2=0x7f5e441554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:16.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.316+0000 7f5e437fe640 1 --2- 172.21.15.16:0/1004787165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e440b4660 0x7f5e441554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:16.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.316+0000 7f5e437fe640 1 -- 172.21.15.16:0/1004787165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e4415ee60 msgr2=0x7f5e4406ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:16.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.316+0000 7f5e437fe640 1 --2- 172.21.15.16:0/1004787165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e4415ee60 0x7f5e4406ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:16.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.316+0000 7f5e437fe640 1 -- 172.21.15.16:0/1004787165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e44159bc0 con 0x7f5e4415f240 2024-09-19T07:43:16.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.316+0000 7f5e437fe640 1 --2- 172.21.15.16:0/1004787165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4415f240 0x7f5e4406f360 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400efc0 tx=0x7f5e3400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:16.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.318+0000 7f5e3bfff640 1 -- 172.21.15.16:0/1004787165 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34018020 con 0x7f5e4415f240 2024-09-19T07:43:16.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.318+0000 7f5e3bfff640 1 -- 172.21.15.16:0/1004787165 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e34014480 con 0x7f5e4415f240 2024-09-19T07:43:16.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.318+0000 7f5e49a2e640 1 -- 172.21.15.16:0/1004787165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e44159ea0 con 0x7f5e4415f240 2024-09-19T07:43:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.318+0000 7f5e3bfff640 1 -- 172.21.15.16:0/1004787165 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34014870 con 0x7f5e4415f240 2024-09-19T07:43:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.318+0000 7f5e49a2e640 1 -- 172.21.15.16:0/1004787165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e4415a460 con 0x7f5e4415f240 2024-09-19T07:43:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.319+0000 7f5e49a2e640 1 -- 172.21.15.16:0/1004787165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e44068d40 con 0x7f5e4415f240 2024-09-19T07:43:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.320+0000 7f5e3bfff640 1 -- 172.21.15.16:0/1004787165 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5e34005d10 con 0x7f5e4415f240 2024-09-19T07:43:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.320+0000 7f5e3bfff640 1 --2- 172.21.15.16:0/1004787165 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5e2406c2b0 0x7f5e2406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:16.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.320+0000 7f5e3bfff640 1 -- 172.21.15.16:0/1004787165 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5e3408c130 con 0x7f5e4415f240 2024-09-19T07:43:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.320+0000 7f5e42ffd640 1 --2- 172.21.15.16:0/1004787165 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5e2406c2b0 0x7f5e2406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:16.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.321+0000 7f5e42ffd640 1 --2- 172.21.15.16:0/1004787165 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5e2406c2b0 0x7f5e2406e770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c034000 tx=0x7f5e3c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:16.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.325+0000 7f5e3bfff640 1 -- 172.21.15.16:0/1004787165 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5e44068d40 con 0x7f5e4415f240 2024-09-19T07:43:16.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.511+0000 7f5e49a2e640 1 -- 172.21.15.16:0/1004787165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5e440d0670 con 0x7f5e4415f240 2024-09-19T07:43:16.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.511+0000 7f5e3bfff640 1 -- 172.21.15.16:0/1004787165 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f5e34022070 con 0x7f5e4415f240 2024-09-19T07:43:16.512 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:43:16.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.513+0000 7f5e49a2e640 1 -- 172.21.15.16:0/1004787165 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5e2406c2b0 msgr2=0x7f5e2406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:16.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.513+0000 7f5e49a2e640 1 --2- 172.21.15.16:0/1004787165 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5e2406c2b0 0x7f5e2406e770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c034000 tx=0x7f5e3c02e000 comp rx=0 tx=0).stop 2024-09-19T07:43:16.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.513+0000 7f5e49a2e640 1 -- 172.21.15.16:0/1004787165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4415f240 msgr2=0x7f5e4406f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:16.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.513+0000 7f5e49a2e640 1 --2- 172.21.15.16:0/1004787165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4415f240 0x7f5e4406f360 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400efc0 tx=0x7f5e3400c490 comp rx=0 tx=0).stop 2024-09-19T07:43:16.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.513+0000 7f5e49a2e640 1 -- 172.21.15.16:0/1004787165 shutdown_connections 2024-09-19T07:43:16.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.513+0000 7f5e49a2e640 1 --2- 172.21.15.16:0/1004787165 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5e2406c2b0 0x7f5e2406e770 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:16.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.513+0000 7f5e49a2e640 1 --2- 172.21.15.16:0/1004787165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e4415f240 0x7f5e4406f360 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:16.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.513+0000 7f5e49a2e640 1 --2- 172.21.15.16:0/1004787165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e4415ee60 0x7f5e4406ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:16.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.513+0000 7f5e49a2e640 1 --2- 172.21.15.16:0/1004787165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e440b4660 0x7f5e441554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:16.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.513+0000 7f5e49a2e640 1 -- 172.21.15.16:0/1004787165 >> 172.21.15.16:0/1004787165 conn(0x7f5e440729a0 msgr2=0x7f5e440b0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:16.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.513+0000 7f5e49a2e640 1 -- 172.21.15.16:0/1004787165 shutdown_connections 2024-09-19T07:43:16.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:16.513+0000 7f5e49a2e640 1 -- 172.21.15.16:0/1004787165 wait complete. 2024-09-19T07:43:16.525 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-19T07:43:16.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:16.526 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-19T07:43:16.550 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[42124]: fuse finished with error 0 and tester_r 0 2024-09-19T07:43:16.552 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:43:22.654 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:43:22.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:22.655 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-19T07:43:22.687 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:43:22.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.841+0000 7ff5d0d6d640 1 Processor -- start 2024-09-19T07:43:22.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.841+0000 7ff5d0d6d640 1 -- start start 2024-09-19T07:43:22.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.842+0000 7ff5d0d6d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff5cc06f9e0 0x7ff5cc0d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:22.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.842+0000 7ff5d0d6d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5cc0d0e00 con 0x7ff5cc15b120 2024-09-19T07:43:22.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.842+0000 7ff5d0d6d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5cc15b500 con 0x7ff5cc0d09f0 2024-09-19T07:43:22.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.842+0000 7ff5d0d6d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff5cc06e940 con 0x7ff5cc06f9e0 2024-09-19T07:43:22.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.842+0000 7ff5ca59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff5cc15b120 0x7ff5cc06dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44772/0 (socket says 172.21.15.16:44772) 2024-09-19T07:43:22.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.842+0000 7ff5ca59c640 1 -- 172.21.15.16:0/4232024342 learned_addr learned my addr 172.21.15.16:0/4232024342 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:22.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.842+0000 7ff5c959a640 1 -- 172.21.15.16:0/4232024342 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 157730807 0 0) 0x7ff5bc001110 con 0x7ff5cc15b120 2024-09-19T07:43:22.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.842+0000 7ff5c959a640 1 -- 172.21.15.16:0/4232024342 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5b4003520 con 0x7ff5cc15b120 2024-09-19T07:43:22.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5c9d9b640 1 --2- 172.21.15.16:0/4232024342 >> v2:172.21.15.146:3300/0 conn(0x7ff5cc06f9e0 0x7ff5cc0d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:22.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5c959a640 1 -- 172.21.15.16:0/4232024342 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4282610916 0 0) 0x7ff5b4003520 con 0x7ff5cc15b120 2024-09-19T07:43:22.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5c959a640 1 -- 172.21.15.16:0/4232024342 >> v2:172.21.15.146:3300/0 conn(0x7ff5cc06f9e0 msgr2=0x7ff5cc0d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:22.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5c959a640 1 --2- 172.21.15.16:0/4232024342 >> v2:172.21.15.146:3300/0 conn(0x7ff5cc06f9e0 0x7ff5cc0d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:22.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5c959a640 1 -- 172.21.15.16:0/4232024342 >> v1:172.21.15.146:6789/0 conn(0x7ff5cc0d09f0 legacy=0x7ff5cc0c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:22.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5c959a640 1 -- 172.21.15.16:0/4232024342 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5cc06ec20 con 0x7ff5cc15b120 2024-09-19T07:43:22.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5c959a640 1 -- 172.21.15.16:0/4232024342 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7ff5bc001840 con 0x7ff5cc15b120 2024-09-19T07:43:22.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5c9d9b640 1 --2- 172.21.15.16:0/4232024342 >> v2:172.21.15.146:3300/0 conn(0x7ff5cc06f9e0 0x7ff5cc0d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:22.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5c959a640 1 -- 172.21.15.16:0/4232024342 >> v1:172.21.15.16:6789/0 conn(0x7ff5cc15b120 legacy=0x7ff5cc06dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:22.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5c959a640 1 --2- 172.21.15.16:0/4232024342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b40039b0 0x7ff5b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:22.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5c959a640 1 --2- 172.21.15.16:0/4232024342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5b4004800 0x7ff5b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:22.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5c959a640 1 --2- 172.21.15.16:0/4232024342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5b40051c0 0x7ff5b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:22.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5c959a640 1 -- 172.21.15.16:0/4232024342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc06e940 con 0x7ff5b40039b0 2024-09-19T07:43:22.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5c959a640 1 -- 172.21.15.16:0/4232024342 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b400c110 con 0x7ff5b40051c0 2024-09-19T07:43:22.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5c959a640 1 -- 172.21.15.16:0/4232024342 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff5b400c310 con 0x7ff5b4004800 2024-09-19T07:43:22.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5ca59c640 1 --2- 172.21.15.16:0/4232024342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b40039b0 0x7ff5b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:22.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5c9d9b640 1 --2- 172.21.15.16:0/4232024342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5b4004800 0x7ff5b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:22.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5cad9d640 1 --2- 172.21.15.16:0/4232024342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5b40051c0 0x7ff5b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:22.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5ca59c640 1 -- 172.21.15.16:0/4232024342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5b4004800 msgr2=0x7ff5b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5ca59c640 1 --2- 172.21.15.16:0/4232024342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5b4004800 0x7ff5b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5ca59c640 1 -- 172.21.15.16:0/4232024342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5b40051c0 msgr2=0x7ff5b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5ca59c640 1 --2- 172.21.15.16:0/4232024342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5b40051c0 0x7ff5b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5ca59c640 1 -- 172.21.15.16:0/4232024342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b4003520 con 0x7ff5b40039b0 2024-09-19T07:43:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5cad9d640 1 --2- 172.21.15.16:0/4232024342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5b40051c0 0x7ff5b400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.843+0000 7ff5cad9d640 1 -- 172.21.15.16:0/4232024342 reap_dead start 2024-09-19T07:43:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.844+0000 7ff5ca59c640 1 --2- 172.21.15.16:0/4232024342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b40039b0 0x7ff5b4003dd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00c4c0 tx=0x7ff5b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2014e027f4bda61f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.844+0000 7ff5c959a640 1 -- 172.21.15.16:0/4232024342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc00c0e0 con 0x7ff5b40039b0 2024-09-19T07:43:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.844+0000 7ff5c959a640 1 -- 172.21.15.16:0/4232024342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5bc034070 con 0x7ff5b40039b0 2024-09-19T07:43:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.844+0000 7ff5c959a640 1 -- 172.21.15.16:0/4232024342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc005d70 con 0x7ff5b40039b0 2024-09-19T07:43:22.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.845+0000 7ff5d0d6d640 1 -- 172.21.15.16:0/4232024342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b40039b0 msgr2=0x7ff5b4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:22.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.845+0000 7ff5d0d6d640 1 --2- 172.21.15.16:0/4232024342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b40039b0 0x7ff5b4003dd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00c4c0 tx=0x7ff5b4005830 comp rx=0 tx=0).stop 2024-09-19T07:43:22.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.845+0000 7ff5d0d6d640 1 -- 172.21.15.16:0/4232024342 shutdown_connections 2024-09-19T07:43:22.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.845+0000 7ff5d0d6d640 1 --2- 172.21.15.16:0/4232024342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b40039b0 0x7ff5b4003dd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.845+0000 7ff5d0d6d640 1 -- 172.21.15.16:0/4232024342 >> 172.21.15.16:0/4232024342 conn(0x7ff5cc06bdf0 msgr2=0x7ff5cc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.845+0000 7ff5d0d6d640 1 -- 172.21.15.16:0/4232024342 shutdown_connections 2024-09-19T07:43:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.845+0000 7ff5d0d6d640 1 -- 172.21.15.16:0/4232024342 wait complete. 2024-09-19T07:43:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.846+0000 7ff5d0d6d640 1 Processor -- start 2024-09-19T07:43:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.846+0000 7ff5d0d6d640 1 -- start start 2024-09-19T07:43:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.847+0000 7ff5d0d6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b40039b0 0x7ff5cc1e5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.847+0000 7ff5d0d6d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5cc152be0 0x7ff5cc1509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.847+0000 7ff5d0d6d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5cc1e4f30 0x7ff5cc1e53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.847+0000 7ff5d0d6d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc06f4c0 con 0x7ff5b40039b0 2024-09-19T07:43:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.847+0000 7ff5d0d6d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc06f1e0 con 0x7ff5cc152be0 2024-09-19T07:43:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.847+0000 7ff5d0d6d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff5cc0ce060 con 0x7ff5cc1e4f30 2024-09-19T07:43:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.847+0000 7ff5ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b40039b0 0x7ff5cc1e5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.847+0000 7ff5ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b40039b0 0x7ff5cc1e5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57172/0 (socket says 172.21.15.16:57172) 2024-09-19T07:43:22.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.847+0000 7ff5ca59c640 1 -- 172.21.15.16:0/2973080164 learned_addr learned my addr 172.21.15.16:0/2973080164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:22.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.847+0000 7ff5c9d9b640 1 --2- 172.21.15.16:0/2973080164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5cc152be0 0x7ff5cc1509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:22.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.847+0000 7ff5cad9d640 1 --2- 172.21.15.16:0/2973080164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5cc1e4f30 0x7ff5cc1e53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:22.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.848+0000 7ff5ca59c640 1 -- 172.21.15.16:0/2973080164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5cc1e4f30 msgr2=0x7ff5cc1e53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:22.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.848+0000 7ff5ca59c640 1 --2- 172.21.15.16:0/2973080164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5cc1e4f30 0x7ff5cc1e53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:22.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.848+0000 7ff5ca59c640 1 -- 172.21.15.16:0/2973080164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5cc152be0 msgr2=0x7ff5cc1509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:22.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.848+0000 7ff5ca59c640 1 --2- 172.21.15.16:0/2973080164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5cc152be0 0x7ff5cc1509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:22.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.848+0000 7ff5ca59c640 1 -- 172.21.15.16:0/2973080164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5cc1e6db0 con 0x7ff5b40039b0 2024-09-19T07:43:22.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.848+0000 7ff5ca59c640 1 --2- 172.21.15.16:0/2973080164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b40039b0 0x7ff5cc1e5f60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc031040 tx=0x7ff5cc06edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:22.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.848+0000 7ff5baffd640 1 -- 172.21.15.16:0/2973080164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc007480 con 0x7ff5b40039b0 2024-09-19T07:43:22.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.848+0000 7ff5baffd640 1 -- 172.21.15.16:0/2973080164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5bc03f030 con 0x7ff5b40039b0 2024-09-19T07:43:22.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.848+0000 7ff5d0d6d640 1 -- 172.21.15.16:0/2973080164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5cc1e67f0 con 0x7ff5b40039b0 2024-09-19T07:43:22.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.848+0000 7ff5baffd640 1 -- 172.21.15.16:0/2973080164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc000ed0 con 0x7ff5b40039b0 2024-09-19T07:43:22.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.848+0000 7ff5d0d6d640 1 -- 172.21.15.16:0/2973080164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5cc1e4730 con 0x7ff5b40039b0 2024-09-19T07:43:22.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.849+0000 7ff5d0d6d640 1 -- 172.21.15.16:0/2973080164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5cc0d04b0 con 0x7ff5b40039b0 2024-09-19T07:43:22.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.850+0000 7ff5baffd640 1 -- 172.21.15.16:0/2973080164 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff5bc02f460 con 0x7ff5b40039b0 2024-09-19T07:43:22.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.850+0000 7ff5baffd640 1 --2- 172.21.15.16:0/2973080164 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff5ac06c450 0x7ff5ac06e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:22.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.850+0000 7ff5baffd640 1 -- 172.21.15.16:0/2973080164 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff5bc0ac000 con 0x7ff5b40039b0 2024-09-19T07:43:22.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.850+0000 7ff5c9d9b640 1 --2- 172.21.15.16:0/2973080164 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff5ac06c450 0x7ff5ac06e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:22.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.851+0000 7ff5c9d9b640 1 --2- 172.21.15.16:0/2973080164 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff5ac06c450 0x7ff5ac06e910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc159d70 tx=0x7ff5c0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:22.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:22.855+0000 7ff5baffd640 1 -- 172.21.15.16:0/2973080164 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff5bc049030 con 0x7ff5b40039b0 2024-09-19T07:43:23.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.032+0000 7ff5d0d6d640 1 -- 172.21.15.16:0/2973080164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5cc0cac20 con 0x7ff5b40039b0 2024-09-19T07:43:23.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.033+0000 7ff5baffd640 1 -- 172.21.15.16:0/2973080164 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7ff5cc0d0e00 con 0x7ff5b40039b0 2024-09-19T07:43:23.033 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:23.034 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:43:10.642082+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:43:03.485582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:43:04.603961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:43:08.299591+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:43:23.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.035+0000 7ff5d0d6d640 1 -- 172.21.15.16:0/2973080164 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff5ac06c450 msgr2=0x7ff5ac06e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.035+0000 7ff5d0d6d640 1 --2- 172.21.15.16:0/2973080164 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff5ac06c450 0x7ff5ac06e910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc159d70 tx=0x7ff5c0008040 comp rx=0 tx=0).stop 2024-09-19T07:43:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.035+0000 7ff5d0d6d640 1 -- 172.21.15.16:0/2973080164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b40039b0 msgr2=0x7ff5cc1e5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.035+0000 7ff5d0d6d640 1 --2- 172.21.15.16:0/2973080164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b40039b0 0x7ff5cc1e5f60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc031040 tx=0x7ff5cc06edf0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.035+0000 7ff5d0d6d640 1 -- 172.21.15.16:0/2973080164 shutdown_connections 2024-09-19T07:43:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.035+0000 7ff5d0d6d640 1 --2- 172.21.15.16:0/2973080164 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff5ac06c450 0x7ff5ac06e910 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.035+0000 7ff5d0d6d640 1 --2- 172.21.15.16:0/2973080164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5cc1e4f30 0x7ff5cc1e53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.035+0000 7ff5d0d6d640 1 --2- 172.21.15.16:0/2973080164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5cc152be0 0x7ff5cc1509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.035+0000 7ff5d0d6d640 1 --2- 172.21.15.16:0/2973080164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5b40039b0 0x7ff5cc1e5f60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.035+0000 7ff5d0d6d640 1 -- 172.21.15.16:0/2973080164 >> 172.21.15.16:0/2973080164 conn(0x7ff5cc06bdf0 msgr2=0x7ff5cc0cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.035+0000 7ff5d0d6d640 1 -- 172.21.15.16:0/2973080164 shutdown_connections 2024-09-19T07:43:23.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.035+0000 7ff5d0d6d640 1 -- 172.21.15.16:0/2973080164 wait complete. 2024-09-19T07:43:23.047 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:43:23.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.148+0000 7f410888d640 1 Processor -- start 2024-09-19T07:43:23.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.148+0000 7f410888d640 1 -- start start 2024-09-19T07:43:23.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.149+0000 7f410888d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4104072250 0x7f41040cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.149+0000 7f410888d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4104072690 0x7f41040d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.149+0000 7f410888d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41040d2b30 con 0x7f41040d2750 2024-09-19T07:43:23.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.149+0000 7f410888d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4104145bd0 con 0x7f4104072690 2024-09-19T07:43:23.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.149+0000 7f410888d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f41041427e0 con 0x7f4104072250 2024-09-19T07:43:23.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.150+0000 7f410259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f41040d2750 0x7f410413f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44784/0 (socket says 172.21.15.16:44784) 2024-09-19T07:43:23.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.150+0000 7f410259c640 1 -- 172.21.15.16:0/2208815497 learned_addr learned my addr 172.21.15.16:0/2208815497 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:23.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.150+0000 7f4102d9d640 1 --2- 172.21.15.16:0/2208815497 >> v2:172.21.15.146:3300/0 conn(0x7f4104072690 0x7f41040d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.150+0000 7f4101d9b640 1 --2- 172.21.15.16:0/2208815497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4104072250 0x7f41040cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.150+0000 7f410159a640 1 -- 172.21.15.16:0/2208815497 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3179504700 0 0) 0x7f41040d2b30 con 0x7f41040d2750 2024-09-19T07:43:23.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.150+0000 7f410159a640 1 -- 172.21.15.16:0/2208815497 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40f0003520 con 0x7f41040d2750 2024-09-19T07:43:23.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.150+0000 7f4102d9d640 1 -- 172.21.15.16:0/2208815497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4104072250 msgr2=0x7f41040cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:23.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.150+0000 7f4102d9d640 1 --2- 172.21.15.16:0/2208815497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4104072250 0x7f41040cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.150+0000 7f4102d9d640 1 -- 172.21.15.16:0/2208815497 >> v1:172.21.15.16:6789/0 conn(0x7f41040d2750 legacy=0x7f410413f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.150+0000 7f4102d9d640 1 -- 172.21.15.16:0/2208815497 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4104141c60 con 0x7f4104072690 2024-09-19T07:43:23.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.151+0000 7f4102d9d640 1 --2- 172.21.15.16:0/2208815497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4104072690 0x7f41040d0300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009d90 tx=0x7f40f402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22f09cf939e1721d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:23.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.151+0000 7f410159a640 1 -- 172.21.15.16:0/2208815497 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f4031600 con 0x7f4104072690 2024-09-19T07:43:23.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.151+0000 7f410159a640 1 -- 172.21.15.16:0/2208815497 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f40317a0 con 0x7f4104072690 2024-09-19T07:43:23.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.151+0000 7f410159a640 1 -- 172.21.15.16:0/2208815497 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f4031b70 con 0x7f4104072690 2024-09-19T07:43:23.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.151+0000 7f410888d640 1 -- 172.21.15.16:0/2208815497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4104072690 msgr2=0x7f41040d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.151+0000 7f410888d640 1 --2- 172.21.15.16:0/2208815497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4104072690 0x7f41040d0300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009d90 tx=0x7f40f402a5d0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.152+0000 7f410888d640 1 -- 172.21.15.16:0/2208815497 shutdown_connections 2024-09-19T07:43:23.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.152+0000 7f410888d640 1 --2- 172.21.15.16:0/2208815497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4104072690 0x7f41040d0300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.152+0000 7f410888d640 1 --2- 172.21.15.16:0/2208815497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4104072250 0x7f41040cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.152+0000 7f410888d640 1 -- 172.21.15.16:0/2208815497 >> 172.21.15.16:0/2208815497 conn(0x7f41040b4040 msgr2=0x7f41040b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:23.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.152+0000 7f410888d640 1 -- 172.21.15.16:0/2208815497 shutdown_connections 2024-09-19T07:43:23.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.152+0000 7f410888d640 1 -- 172.21.15.16:0/2208815497 wait complete. 2024-09-19T07:43:23.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.153+0000 7f410888d640 1 Processor -- start 2024-09-19T07:43:23.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.153+0000 7f410888d640 1 -- start start 2024-09-19T07:43:23.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.154+0000 7f410888d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4104072250 0x7f41041f1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.154+0000 7f410888d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4104072690 0x7f4104145190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.154+0000 7f410888d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41040d2750 0x7f410415e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.154+0000 7f410888d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4104142500 con 0x7f41040d2750 2024-09-19T07:43:23.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.154+0000 7f410888d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4104142220 con 0x7f4104072690 2024-09-19T07:43:23.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.154+0000 7f410888d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f41041418b0 con 0x7f4104072250 2024-09-19T07:43:23.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.154+0000 7f4102d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41040d2750 0x7f410415e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.154+0000 7f4102d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41040d2750 0x7f410415e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57178/0 (socket says 172.21.15.16:57178) 2024-09-19T07:43:23.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.154+0000 7f4102d9d640 1 -- 172.21.15.16:0/2347788323 learned_addr learned my addr 172.21.15.16:0/2347788323 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:23.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.154+0000 7f410259c640 1 --2- 172.21.15.16:0/2347788323 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4104072250 0x7f41041f1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.154+0000 7f4101d9b640 1 --2- 172.21.15.16:0/2347788323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4104072690 0x7f4104145190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.154+0000 7f4102d9d640 1 -- 172.21.15.16:0/2347788323 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4104072250 msgr2=0x7f41041f1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.154+0000 7f4102d9d640 1 --2- 172.21.15.16:0/2347788323 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4104072250 0x7f41041f1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.154+0000 7f4102d9d640 1 -- 172.21.15.16:0/2347788323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4104072690 msgr2=0x7f4104145190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.154+0000 7f4102d9d640 1 --2- 172.21.15.16:0/2347788323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4104072690 0x7f4104145190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.154+0000 7f4102d9d640 1 -- 172.21.15.16:0/2347788323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f410415d220 con 0x7f41040d2750 2024-09-19T07:43:23.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.154+0000 7f4102d9d640 1 --2- 172.21.15.16:0/2347788323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41040d2750 0x7f410415e280 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009eb0 tx=0x7f40f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:23.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.155+0000 7f40faffd640 1 -- 172.21.15.16:0/2347788323 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f4040070 con 0x7f41040d2750 2024-09-19T07:43:23.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.155+0000 7f40faffd640 1 -- 172.21.15.16:0/2347788323 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f403c830 con 0x7f41040d2750 2024-09-19T07:43:23.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.155+0000 7f40faffd640 1 -- 172.21.15.16:0/2347788323 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f403cbe0 con 0x7f41040d2750 2024-09-19T07:43:23.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.155+0000 7f410888d640 1 -- 172.21.15.16:0/2347788323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f410415d500 con 0x7f41040d2750 2024-09-19T07:43:23.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.155+0000 7f410888d640 1 -- 172.21.15.16:0/2347788323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41041456d0 con 0x7f41040d2750 2024-09-19T07:43:23.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.156+0000 7f410888d640 1 -- 172.21.15.16:0/2347788323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f410415d6d0 con 0x7f41040d2750 2024-09-19T07:43:23.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.156+0000 7f40faffd640 1 -- 172.21.15.16:0/2347788323 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f40f4039040 con 0x7f41040d2750 2024-09-19T07:43:23.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.157+0000 7f40faffd640 1 --2- 172.21.15.16:0/2347788323 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f40e406c2b0 0x7f40e406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.157+0000 7f40faffd640 1 -- 172.21.15.16:0/2347788323 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f40f40acde0 con 0x7f41040d2750 2024-09-19T07:43:23.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.157+0000 7f410259c640 1 --2- 172.21.15.16:0/2347788323 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f40e406c2b0 0x7f40e406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.157+0000 7f410259c640 1 --2- 172.21.15.16:0/2347788323 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f40e406c2b0 0x7f40e406e770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f40fc005880 tx=0x7f40fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:23.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.161+0000 7f40faffd640 1 -- 172.21.15.16:0/2347788323 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f40f407ac40 con 0x7f41040d2750 2024-09-19T07:43:23.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.339+0000 7f410888d640 1 -- 172.21.15.16:0/2347788323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41040cd6b0 con 0x7f41040d2750 2024-09-19T07:43:23.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.340+0000 7f40faffd640 1 -- 172.21.15.16:0/2347788323 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7f40f407ef40 con 0x7f41040d2750 2024-09-19T07:43:23.341 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:23.341 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:43:10.642082+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:43:03.485582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:43:04.603961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:43:08.299591+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:43:23.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.342+0000 7f410888d640 1 -- 172.21.15.16:0/2347788323 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f40e406c2b0 msgr2=0x7f40e406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.342+0000 7f410888d640 1 --2- 172.21.15.16:0/2347788323 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f40e406c2b0 0x7f40e406e770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f40fc005880 tx=0x7f40fc009040 comp rx=0 tx=0).stop 2024-09-19T07:43:23.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.342+0000 7f410888d640 1 -- 172.21.15.16:0/2347788323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41040d2750 msgr2=0x7f410415e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.342+0000 7f410888d640 1 --2- 172.21.15.16:0/2347788323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41040d2750 0x7f410415e280 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009eb0 tx=0x7f40f4004a40 comp rx=0 tx=0).stop 2024-09-19T07:43:23.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.342+0000 7f410888d640 1 -- 172.21.15.16:0/2347788323 shutdown_connections 2024-09-19T07:43:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.342+0000 7f410888d640 1 --2- 172.21.15.16:0/2347788323 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f40e406c2b0 0x7f40e406e770 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.342+0000 7f410888d640 1 --2- 172.21.15.16:0/2347788323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41040d2750 0x7f410415e280 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.342+0000 7f410888d640 1 --2- 172.21.15.16:0/2347788323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4104072690 0x7f4104145190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.342+0000 7f410888d640 1 --2- 172.21.15.16:0/2347788323 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4104072250 0x7f41041f1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.342+0000 7f410888d640 1 -- 172.21.15.16:0/2347788323 >> 172.21.15.16:0/2347788323 conn(0x7f41040b4040 msgr2=0x7f41040ce110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.343+0000 7f410888d640 1 -- 172.21.15.16:0/2347788323 shutdown_connections 2024-09-19T07:43:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.343+0000 7f410888d640 1 -- 172.21.15.16:0/2347788323 wait complete. 2024-09-19T07:43:23.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T07:43:23.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.459+0000 7f78ddb86640 1 Processor -- start 2024-09-19T07:43:23.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.459+0000 7f78ddb86640 1 -- start start 2024-09-19T07:43:23.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.460+0000 7f78ddb86640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78d815f0d0 0x7f78d815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.460+0000 7f78ddb86640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78d80d1b10 con 0x7f78d80d4130 2024-09-19T07:43:23.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.460+0000 7f78ddb86640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78d80cf2b0 con 0x7f78d80d0070 2024-09-19T07:43:23.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.460+0000 7f78ddb86640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f78d80cf590 con 0x7f78d815f0d0 2024-09-19T07:43:23.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.460+0000 7f78d7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f78d80d4130 0x7f78d80d45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44792/0 (socket says 172.21.15.16:44792) 2024-09-19T07:43:23.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.460+0000 7f78d7fff640 1 -- 172.21.15.16:0/2280498150 learned_addr learned my addr 172.21.15.16:0/2280498150 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:23.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.460+0000 7f78d77fe640 1 --2- 172.21.15.16:0/2280498150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78d815f0d0 0x7f78d815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.460+0000 7f78d67fc640 1 -- 172.21.15.16:0/2280498150 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1193286057 0 0) 0x7f78d80d1b10 con 0x7f78d80d4130 2024-09-19T07:43:23.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.460+0000 7f78d67fc640 1 -- 172.21.15.16:0/2280498150 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78c0003520 con 0x7f78d80d4130 2024-09-19T07:43:23.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d67fc640 1 -- 172.21.15.16:0/2280498150 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2639303341 0 0) 0x7f78c0003520 con 0x7f78d80d4130 2024-09-19T07:43:23.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d67fc640 1 -- 172.21.15.16:0/2280498150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78d815f0d0 msgr2=0x7f78d815f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:23.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d67fc640 1 --2- 172.21.15.16:0/2280498150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78d815f0d0 0x7f78d815f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d67fc640 1 -- 172.21.15.16:0/2280498150 >> v1:172.21.15.146:6789/0 conn(0x7f78d80d0070 legacy=0x7f78d80d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d67fc640 1 -- 172.21.15.16:0/2280498150 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d80cf870 con 0x7f78d80d4130 2024-09-19T07:43:23.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d67fc640 1 -- 172.21.15.16:0/2280498150 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f78d00019f0 con 0x7f78d80d4130 2024-09-19T07:43:23.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d67fc640 1 -- 172.21.15.16:0/2280498150 >> v1:172.21.15.16:6789/0 conn(0x7f78d80d4130 legacy=0x7f78d80d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d67fc640 1 --2- 172.21.15.16:0/2280498150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c00039e0 0x7f78c0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d67fc640 1 --2- 172.21.15.16:0/2280498150 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78c0004800 0x7f78c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d67fc640 1 --2- 172.21.15.16:0/2280498150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78d815f0d0 0x7f78c0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d67fc640 1 -- 172.21.15.16:0/2280498150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78d80cf590 con 0x7f78c00039e0 2024-09-19T07:43:23.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d67fc640 1 -- 172.21.15.16:0/2280498150 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f78c000baf0 con 0x7f78c0004800 2024-09-19T07:43:23.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d67fc640 1 -- 172.21.15.16:0/2280498150 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f78c000bcf0 con 0x7f78d815f0d0 2024-09-19T07:43:23.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d77fe640 1 --2- 172.21.15.16:0/2280498150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c00039e0 0x7f78c0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d6ffd640 1 --2- 172.21.15.16:0/2280498150 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78c0004800 0x7f78c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d7fff640 1 --2- 172.21.15.16:0/2280498150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78d815f0d0 0x7f78c0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d77fe640 1 -- 172.21.15.16:0/2280498150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78d815f0d0 msgr2=0x7f78c0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:23.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d77fe640 1 --2- 172.21.15.16:0/2280498150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78d815f0d0 0x7f78c0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d77fe640 1 -- 172.21.15.16:0/2280498150 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78c0004800 msgr2=0x7f78c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:23.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d77fe640 1 --2- 172.21.15.16:0/2280498150 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78c0004800 0x7f78c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.461+0000 7f78d77fe640 1 -- 172.21.15.16:0/2280498150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c0003520 con 0x7f78c00039e0 2024-09-19T07:43:23.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.462+0000 7f78d77fe640 1 --2- 172.21.15.16:0/2280498150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c00039e0 0x7f78c0003e00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009ac0 tx=0x7f78c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5231b7dd33b3a02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:23.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.462+0000 7f78d67fc640 1 -- 172.21.15.16:0/2280498150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8036070 con 0x7f78c00039e0 2024-09-19T07:43:23.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.462+0000 7f78d67fc640 1 -- 172.21.15.16:0/2280498150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c8031400 con 0x7f78c00039e0 2024-09-19T07:43:23.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.462+0000 7f78d67fc640 1 -- 172.21.15.16:0/2280498150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8009690 con 0x7f78c00039e0 2024-09-19T07:43:23.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.463+0000 7f78ddb86640 1 -- 172.21.15.16:0/2280498150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c00039e0 msgr2=0x7f78c0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.463+0000 7f78ddb86640 1 --2- 172.21.15.16:0/2280498150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c00039e0 0x7f78c0003e00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009ac0 tx=0x7f78c0005830 comp rx=0 tx=0).stop 2024-09-19T07:43:23.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.463+0000 7f78d7fff640 1 -- 172.21.15.16:0/2280498150 reap_dead start 2024-09-19T07:43:23.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.463+0000 7f78ddb86640 1 -- 172.21.15.16:0/2280498150 shutdown_connections 2024-09-19T07:43:23.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.463+0000 7f78ddb86640 1 -- 172.21.15.16:0/2280498150 >> 172.21.15.16:0/2280498150 conn(0x7f78d809d760 msgr2=0x7f78d808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:23.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.463+0000 7f78ddb86640 1 -- 172.21.15.16:0/2280498150 shutdown_connections 2024-09-19T07:43:23.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.463+0000 7f78ddb86640 1 -- 172.21.15.16:0/2280498150 wait complete. 2024-09-19T07:43:23.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.464+0000 7f78ddb86640 1 Processor -- start 2024-09-19T07:43:23.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.464+0000 7f78ddb86640 1 -- start start 2024-09-19T07:43:23.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.465+0000 7f78ddb86640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78c00039e0 0x7f78d80cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.465+0000 7f78ddb86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d81586d0 0x7f78d8158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.465+0000 7f78ddb86640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78d8154080 0x7f78d8154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.465+0000 7f78ddb86640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78d815f4b0 con 0x7f78d81586d0 2024-09-19T07:43:23.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.465+0000 7f78ddb86640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f78d80cef60 con 0x7f78d8154080 2024-09-19T07:43:23.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.465+0000 7f78ddb86640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f78d815f630 con 0x7f78c00039e0 2024-09-19T07:43:23.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.465+0000 7f78d6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d81586d0 0x7f78d8158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.465+0000 7f78d6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d81586d0 0x7f78d8158b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57186/0 (socket says 172.21.15.16:57186) 2024-09-19T07:43:23.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.465+0000 7f78d6ffd640 1 -- 172.21.15.16:0/277264303 learned_addr learned my addr 172.21.15.16:0/277264303 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:23.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.465+0000 7f78d77fe640 1 --2- 172.21.15.16:0/277264303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78c00039e0 0x7f78d80cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.465+0000 7f78d7fff640 1 --2- 172.21.15.16:0/277264303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78d8154080 0x7f78d8154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.465+0000 7f78d6ffd640 1 -- 172.21.15.16:0/277264303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78c00039e0 msgr2=0x7f78d80cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.465+0000 7f78d6ffd640 1 --2- 172.21.15.16:0/277264303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78c00039e0 0x7f78d80cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.465+0000 7f78d6ffd640 1 -- 172.21.15.16:0/277264303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78d8154080 msgr2=0x7f78d8154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.465+0000 7f78d6ffd640 1 --2- 172.21.15.16:0/277264303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78d8154080 0x7f78d8154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.465+0000 7f78d6ffd640 1 -- 172.21.15.16:0/277264303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d8159240 con 0x7f78d81586d0 2024-09-19T07:43:23.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.465+0000 7f78d6ffd640 1 --2- 172.21.15.16:0/277264303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d81586d0 0x7f78d8158b50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00f5c0 tx=0x7f78cc00fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:23.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.466+0000 7f78bffff640 1 -- 172.21.15.16:0/277264303 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc0052e0 con 0x7f78d81586d0 2024-09-19T07:43:23.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.466+0000 7f78bffff640 1 -- 172.21.15.16:0/277264303 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78cc018070 con 0x7f78d81586d0 2024-09-19T07:43:23.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.466+0000 7f78bffff640 1 -- 172.21.15.16:0/277264303 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc003810 con 0x7f78d81586d0 2024-09-19T07:43:23.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.466+0000 7f78ddb86640 1 -- 172.21.15.16:0/277264303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d8159520 con 0x7f78d81586d0 2024-09-19T07:43:23.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.466+0000 7f78ddb86640 1 -- 172.21.15.16:0/277264303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d8159ae0 con 0x7f78d81586d0 2024-09-19T07:43:23.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.467+0000 7f78ddb86640 1 -- 172.21.15.16:0/277264303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d80d45e0 con 0x7f78d81586d0 2024-09-19T07:43:23.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.467+0000 7f78bffff640 1 -- 172.21.15.16:0/277264303 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f78cc0039b0 con 0x7f78d81586d0 2024-09-19T07:43:23.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.468+0000 7f78bffff640 1 --2- 172.21.15.16:0/277264303 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f78c406c2b0 0x7f78c406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.468+0000 7f78bffff640 1 -- 172.21.15.16:0/277264303 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f78cc08b3a0 con 0x7f78d81586d0 2024-09-19T07:43:23.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.468+0000 7f78d77fe640 1 --2- 172.21.15.16:0/277264303 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f78c406c2b0 0x7f78c406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.468+0000 7f78d77fe640 1 --2- 172.21.15.16:0/277264303 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f78c406c2b0 0x7f78c406e770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f78c8033000 tx=0x7f78c8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:23.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.472+0000 7f78bffff640 1 -- 172.21.15.16:0/277264303 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f78cc059210 con 0x7f78d81586d0 2024-09-19T07:43:23.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:23.686+0000 7f78ddb86640 1 -- 172.21.15.16:0/277264303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f78d806cf30 con 0x7f78d81586d0 2024-09-19T07:43:24.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.333+0000 7f78bffff640 1 -- 172.21.15.16:0/277264303 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f78cc05d510 con 0x7f78d81586d0 2024-09-19T07:43:24.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.336+0000 7f78ddb86640 1 -- 172.21.15.16:0/277264303 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f78c406c2b0 msgr2=0x7f78c406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:24.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.336+0000 7f78ddb86640 1 --2- 172.21.15.16:0/277264303 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f78c406c2b0 0x7f78c406e770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f78c8033000 tx=0x7f78c8008030 comp rx=0 tx=0).stop 2024-09-19T07:43:24.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.336+0000 7f78ddb86640 1 -- 172.21.15.16:0/277264303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d81586d0 msgr2=0x7f78d8158b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:24.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.336+0000 7f78ddb86640 1 --2- 172.21.15.16:0/277264303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d81586d0 0x7f78d8158b50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00f5c0 tx=0x7f78cc00fa80 comp rx=0 tx=0).stop 2024-09-19T07:43:24.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.336+0000 7f78ddb86640 1 -- 172.21.15.16:0/277264303 shutdown_connections 2024-09-19T07:43:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.336+0000 7f78ddb86640 1 --2- 172.21.15.16:0/277264303 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f78c406c2b0 0x7f78c406e770 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.336+0000 7f78ddb86640 1 --2- 172.21.15.16:0/277264303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78d8154080 0x7f78d8154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.336+0000 7f78ddb86640 1 --2- 172.21.15.16:0/277264303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d81586d0 0x7f78d8158b50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.336+0000 7f78ddb86640 1 --2- 172.21.15.16:0/277264303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78c00039e0 0x7f78d80cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.336+0000 7f78ddb86640 1 -- 172.21.15.16:0/277264303 >> 172.21.15.16:0/277264303 conn(0x7f78d809d760 msgr2=0x7f78d80b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.336+0000 7f78ddb86640 1 -- 172.21.15.16:0/277264303 shutdown_connections 2024-09-19T07:43:24.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.336+0000 7f78ddb86640 1 -- 172.21.15.16:0/277264303 wait complete. 2024-09-19T07:43:24.350 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-09-19T07:43:24.350 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T07:43:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.463+0000 7f301d99f640 1 Processor -- start 2024-09-19T07:43:24.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.463+0000 7f301d99f640 1 -- start start 2024-09-19T07:43:24.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f301d99f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3018162b00 0x7f30180d2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:24.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f301d99f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30180d1de0 con 0x7f30180d41a0 2024-09-19T07:43:24.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f301d99f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30180d20c0 con 0x7f301815e300 2024-09-19T07:43:24.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f301d99f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f30180d29f0 con 0x7f3018162b00 2024-09-19T07:43:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f301d19e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f30180d41a0 0x7f30180cedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44802/0 (socket says 172.21.15.16:44802) 2024-09-19T07:43:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f301d19e640 1 -- 172.21.15.16:0/4235577653 learned_addr learned my addr 172.21.15.16:0/4235577653 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f3017fff640 1 --2- 172.21.15.16:0/4235577653 >> v2:172.21.15.146:3300/0 conn(0x7f3018162b00 0x7f30180d2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f30177fe640 1 -- 172.21.15.16:0/4235577653 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2594365541 0 0) 0x7f30180d1de0 con 0x7f30180d41a0 2024-09-19T07:43:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f30177fe640 1 -- 172.21.15.16:0/4235577653 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3008003520 con 0x7f30180d41a0 2024-09-19T07:43:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f30177fe640 1 -- 172.21.15.16:0/4235577653 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2391459667 0 0) 0x7f30180d20c0 con 0x7f301815e300 2024-09-19T07:43:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f30177fe640 1 -- 172.21.15.16:0/4235577653 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30180d1de0 con 0x7f301815e300 2024-09-19T07:43:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f30177fe640 1 -- 172.21.15.16:0/4235577653 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1915406081 0 0) 0x7f3008003520 con 0x7f30180d41a0 2024-09-19T07:43:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f30177fe640 1 -- 172.21.15.16:0/4235577653 >> v2:172.21.15.146:3300/0 conn(0x7f3018162b00 msgr2=0x7f30180d2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f30177fe640 1 --2- 172.21.15.16:0/4235577653 >> v2:172.21.15.146:3300/0 conn(0x7f3018162b00 0x7f30180d2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f30177fe640 1 -- 172.21.15.16:0/4235577653 >> v1:172.21.15.146:6789/0 conn(0x7f301815e300 legacy=0x7f30180b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f30177fe640 1 -- 172.21.15.16:0/4235577653 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30180d23a0 con 0x7f30180d41a0 2024-09-19T07:43:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f30177fe640 1 -- 172.21.15.16:0/4235577653 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f30100019f0 con 0x7f30180d41a0 2024-09-19T07:43:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f30177fe640 1 -- 172.21.15.16:0/4235577653 >> v1:172.21.15.16:6789/0 conn(0x7f30180d41a0 legacy=0x7f30180cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f30177fe640 1 --2- 172.21.15.16:0/4235577653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3008003b40 0x7f3008003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f30177fe640 1 --2- 172.21.15.16:0/4235577653 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3008004800 0x7f3008004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.464+0000 7f30177fe640 1 --2- 172.21.15.16:0/4235577653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30080051c0 0x7f300800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.465+0000 7f30177fe640 1 -- 172.21.15.16:0/4235577653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30180d29f0 con 0x7f3008003b40 2024-09-19T07:43:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.465+0000 7f30177fe640 1 -- 172.21.15.16:0/4235577653 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f300800c040 con 0x7f30080051c0 2024-09-19T07:43:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.465+0000 7f30177fe640 1 -- 172.21.15.16:0/4235577653 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f300800c240 con 0x7f3008004800 2024-09-19T07:43:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.465+0000 7f301c99d640 1 --2- 172.21.15.16:0/4235577653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3008003b40 0x7f3008003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.465+0000 7f301d19e640 1 --2- 172.21.15.16:0/4235577653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30080051c0 0x7f300800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.465+0000 7f3017fff640 1 --2- 172.21.15.16:0/4235577653 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3008004800 0x7f3008004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.465+0000 7f301c99d640 1 -- 172.21.15.16:0/4235577653 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3008004800 msgr2=0x7f3008004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.465+0000 7f301c99d640 1 --2- 172.21.15.16:0/4235577653 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3008004800 0x7f3008004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.465+0000 7f301c99d640 1 -- 172.21.15.16:0/4235577653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30080051c0 msgr2=0x7f300800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.465+0000 7f301c99d640 1 --2- 172.21.15.16:0/4235577653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30080051c0 0x7f300800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.465+0000 7f301d19e640 1 -- 172.21.15.16:0/4235577653 reap_dead start 2024-09-19T07:43:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.465+0000 7f301c99d640 1 -- 172.21.15.16:0/4235577653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3008003520 con 0x7f3008003b40 2024-09-19T07:43:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.466+0000 7f301c99d640 1 --2- 172.21.15.16:0/4235577653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3008003b40 0x7f3008003f40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f300c0018d0 tx=0x7f3008005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f62d5df6603f56c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.466+0000 7f30177fe640 1 -- 172.21.15.16:0/4235577653 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c0014f0 con 0x7f3008003b40 2024-09-19T07:43:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.466+0000 7f30177fe640 1 -- 172.21.15.16:0/4235577653 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f300c0052d0 con 0x7f3008003b40 2024-09-19T07:43:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.466+0000 7f30177fe640 1 -- 172.21.15.16:0/4235577653 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c02ac70 con 0x7f3008003b40 2024-09-19T07:43:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.466+0000 7f301d99f640 1 -- 172.21.15.16:0/4235577653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3008003b40 msgr2=0x7f3008003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.466+0000 7f301d99f640 1 --2- 172.21.15.16:0/4235577653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3008003b40 0x7f3008003f40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f300c0018d0 tx=0x7f3008005830 comp rx=0 tx=0).stop 2024-09-19T07:43:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.466+0000 7f301d99f640 1 -- 172.21.15.16:0/4235577653 shutdown_connections 2024-09-19T07:43:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.466+0000 7f301d99f640 1 --2- 172.21.15.16:0/4235577653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3008003b40 0x7f3008003f40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:24.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.466+0000 7f301d99f640 1 -- 172.21.15.16:0/4235577653 >> 172.21.15.16:0/4235577653 conn(0x7f301808bf80 msgr2=0x7f301809d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:24.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.467+0000 7f301d99f640 1 -- 172.21.15.16:0/4235577653 shutdown_connections 2024-09-19T07:43:24.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.467+0000 7f301d99f640 1 -- 172.21.15.16:0/4235577653 wait complete. 2024-09-19T07:43:24.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.468+0000 7f301d99f640 1 Processor -- start 2024-09-19T07:43:24.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.468+0000 7f301d99f640 1 -- start start 2024-09-19T07:43:24.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f301d99f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3008003b40 0x7f3018156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:24.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f301d99f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018160060 0x7f30181604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:24.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f301d99f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30180b2300 0x7f30180b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f301d99f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f301815e860 con 0x7f3018160060 2024-09-19T07:43:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f301d99f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f301815e6e0 con 0x7f30180b2300 2024-09-19T07:43:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f301d99f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f30180d2e80 con 0x7f3008003b40 2024-09-19T07:43:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f3017fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018160060 0x7f30181604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f3017fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018160060 0x7f30181604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57202/0 (socket says 172.21.15.16:57202) 2024-09-19T07:43:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f3017fff640 1 -- 172.21.15.16:0/187824271 learned_addr learned my addr 172.21.15.16:0/187824271 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f301d19e640 1 --2- 172.21.15.16:0/187824271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30180b2300 0x7f30180b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f301c99d640 1 --2- 172.21.15.16:0/187824271 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3008003b40 0x7f3018156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f3017fff640 1 -- 172.21.15.16:0/187824271 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3008003b40 msgr2=0x7f3018156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f3017fff640 1 --2- 172.21.15.16:0/187824271 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3008003b40 0x7f3018156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f3017fff640 1 -- 172.21.15.16:0/187824271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30180b2300 msgr2=0x7f30180b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:24.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f3017fff640 1 --2- 172.21.15.16:0/187824271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30180b2300 0x7f30180b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:24.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f3017fff640 1 -- 172.21.15.16:0/187824271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30180b3350 con 0x7f3018160060 2024-09-19T07:43:24.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f301d19e640 1 --2- 172.21.15.16:0/187824271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30180b2300 0x7f30180b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:24.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f3017fff640 1 --2- 172.21.15.16:0/187824271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018160060 0x7f30181604e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f30040119f0 tx=0x7f3004011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:24.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f30157fa640 1 -- 172.21.15.16:0/187824271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300400faf0 con 0x7f3018160060 2024-09-19T07:43:24.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f30157fa640 1 -- 172.21.15.16:0/187824271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f300400fc90 con 0x7f3018160060 2024-09-19T07:43:24.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f30157fa640 1 -- 172.21.15.16:0/187824271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004005bb0 con 0x7f3018160060 2024-09-19T07:43:24.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f301d99f640 1 -- 172.21.15.16:0/187824271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30180b7900 con 0x7f3018160060 2024-09-19T07:43:24.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.469+0000 7f301d99f640 1 -- 172.21.15.16:0/187824271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30180b3070 con 0x7f3018160060 2024-09-19T07:43:24.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.470+0000 7f301d99f640 1 -- 172.21.15.16:0/187824271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3018069110 con 0x7f3018160060 2024-09-19T07:43:24.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.471+0000 7f30157fa640 1 -- 172.21.15.16:0/187824271 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3004012040 con 0x7f3018160060 2024-09-19T07:43:24.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.471+0000 7f30157fa640 1 --2- 172.21.15.16:0/187824271 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2ffc06c2e0 0x7f2ffc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:24.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.472+0000 7f30157fa640 1 -- 172.21.15.16:0/187824271 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f300408bb40 con 0x7f3018160060 2024-09-19T07:43:24.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.472+0000 7f301c99d640 1 --2- 172.21.15.16:0/187824271 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2ffc06c2e0 0x7f2ffc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:24.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.472+0000 7f301c99d640 1 --2- 172.21.15.16:0/187824271 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2ffc06c2e0 0x7f2ffc06e7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f300c032000 tx=0x7f300c009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:24.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.476+0000 7f30157fa640 1 -- 172.21.15.16:0/187824271 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3004019030 con 0x7f3018160060 2024-09-19T07:43:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:24.690+0000 7f301d99f640 1 -- 172.21.15.16:0/187824271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3018162190 con 0x7f3018160060 2024-09-19T07:43:25.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.337+0000 7f30157fa640 1 -- 172.21.15.16:0/187824271 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f3004059f70 con 0x7f3018160060 2024-09-19T07:43:25.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.340+0000 7f301d99f640 1 -- 172.21.15.16:0/187824271 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2ffc06c2e0 msgr2=0x7f2ffc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:25.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.340+0000 7f301d99f640 1 --2- 172.21.15.16:0/187824271 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2ffc06c2e0 0x7f2ffc06e7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f300c032000 tx=0x7f300c009030 comp rx=0 tx=0).stop 2024-09-19T07:43:25.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.340+0000 7f301d99f640 1 -- 172.21.15.16:0/187824271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018160060 msgr2=0x7f30181604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:25.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.340+0000 7f301d99f640 1 --2- 172.21.15.16:0/187824271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018160060 0x7f30181604e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f30040119f0 tx=0x7f3004011eb0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.341+0000 7f301d99f640 1 -- 172.21.15.16:0/187824271 shutdown_connections 2024-09-19T07:43:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.341+0000 7f301d99f640 1 --2- 172.21.15.16:0/187824271 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2ffc06c2e0 0x7f2ffc06e7a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.341+0000 7f301d99f640 1 --2- 172.21.15.16:0/187824271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30180b2300 0x7f30180b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.341+0000 7f301d99f640 1 --2- 172.21.15.16:0/187824271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018160060 0x7f30181604e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.341+0000 7f301d99f640 1 --2- 172.21.15.16:0/187824271 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3008003b40 0x7f3018156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.341+0000 7f301d99f640 1 -- 172.21.15.16:0/187824271 >> 172.21.15.16:0/187824271 conn(0x7f301808bf80 msgr2=0x7f301806de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:25.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.341+0000 7f301d99f640 1 -- 172.21.15.16:0/187824271 shutdown_connections 2024-09-19T07:43:25.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.341+0000 7f301d99f640 1 -- 172.21.15.16:0/187824271 wait complete. 2024-09-19T07:43:25.354 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T07:43:25.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:43:25.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.461+0000 7f76d4add640 1 Processor -- start 2024-09-19T07:43:25.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.462+0000 7f76d4add640 1 -- start start 2024-09-19T07:43:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.462+0000 7f76d4add640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76d0072250 0x7f76d00cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.462+0000 7f76d4add640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76d0072690 0x7f76d00d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.462+0000 7f76d4add640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76d00d2b30 con 0x7f76d00d2750 2024-09-19T07:43:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.462+0000 7f76d4add640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f76d0145bd0 con 0x7f76d0072690 2024-09-19T07:43:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.462+0000 7f76d4add640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f76d01427e0 con 0x7f76d0072250 2024-09-19T07:43:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.462+0000 7f76ced9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76d0072690 0x7f76d00d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.463+0000 7f76ce59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f76d00d2750 0x7f76d013f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44810/0 (socket says 172.21.15.16:44810) 2024-09-19T07:43:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.463+0000 7f76ce59c640 1 -- 172.21.15.16:0/1871338973 learned_addr learned my addr 172.21.15.16:0/1871338973 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.463+0000 7f76c7fff640 1 --2- 172.21.15.16:0/1871338973 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76d0072250 0x7f76d00cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.463+0000 7f76ced9d640 1 -- 172.21.15.16:0/1871338973 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76d0072250 msgr2=0x7f76d00cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.463+0000 7f76ced9d640 1 --2- 172.21.15.16:0/1871338973 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76d0072250 0x7f76d00cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.463+0000 7f76ced9d640 1 -- 172.21.15.16:0/1871338973 >> v1:172.21.15.16:6789/0 conn(0x7f76d00d2750 legacy=0x7f76d013f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.463+0000 7f76ced9d640 1 -- 172.21.15.16:0/1871338973 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d0141c60 con 0x7f76d0072690 2024-09-19T07:43:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.463+0000 7f76c7fff640 1 --2- 172.21.15.16:0/1871338973 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76d0072250 0x7f76d00cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.464+0000 7f76ced9d640 1 --2- 172.21.15.16:0/1871338973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d0072690 0x7f76d00d0300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f76c8009d90 tx=0x7f76c802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4c30d2e904f7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.464+0000 7f76cdd9b640 1 -- 172.21.15.16:0/1871338973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8031600 con 0x7f76d0072690 2024-09-19T07:43:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.464+0000 7f76cdd9b640 1 -- 172.21.15.16:0/1871338973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c80317a0 con 0x7f76d0072690 2024-09-19T07:43:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.464+0000 7f76cdd9b640 1 -- 172.21.15.16:0/1871338973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8031b70 con 0x7f76d0072690 2024-09-19T07:43:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.464+0000 7f76d4add640 1 -- 172.21.15.16:0/1871338973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d0072690 msgr2=0x7f76d00d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.464+0000 7f76d4add640 1 --2- 172.21.15.16:0/1871338973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d0072690 0x7f76d00d0300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f76c8009d90 tx=0x7f76c802a5d0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.464+0000 7f76d4add640 1 -- 172.21.15.16:0/1871338973 shutdown_connections 2024-09-19T07:43:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.464+0000 7f76d4add640 1 --2- 172.21.15.16:0/1871338973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d0072690 0x7f76d00d0300 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.464+0000 7f76d4add640 1 --2- 172.21.15.16:0/1871338973 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76d0072250 0x7f76d00cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.464+0000 7f76d4add640 1 -- 172.21.15.16:0/1871338973 >> 172.21.15.16:0/1871338973 conn(0x7f76d00b4040 msgr2=0x7f76d00b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.464+0000 7f76d4add640 1 -- 172.21.15.16:0/1871338973 shutdown_connections 2024-09-19T07:43:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.464+0000 7f76d4add640 1 -- 172.21.15.16:0/1871338973 wait complete. 2024-09-19T07:43:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.466+0000 7f76d4add640 1 Processor -- start 2024-09-19T07:43:25.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.466+0000 7f76d4add640 1 -- start start 2024-09-19T07:43:25.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.466+0000 7f76d4add640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76d0072250 0x7f76d00cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:25.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.466+0000 7f76d4add640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d0072690 0x7f76d013ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:25.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76d4add640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76d00d2750 0x7f76d00c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:25.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76d4add640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76d0142500 con 0x7f76d0072690 2024-09-19T07:43:25.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76d4add640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f76d0142220 con 0x7f76d00d2750 2024-09-19T07:43:25.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76d4add640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f76d01418b0 con 0x7f76d0072250 2024-09-19T07:43:25.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76c7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d0072690 0x7f76d013ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:25.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76c7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d0072690 0x7f76d013ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57216/0 (socket says 172.21.15.16:57216) 2024-09-19T07:43:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76c7fff640 1 -- 172.21.15.16:0/2402125703 learned_addr learned my addr 172.21.15.16:0/2402125703 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76ce59c640 1 --2- 172.21.15.16:0/2402125703 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76d0072250 0x7f76d00cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76ced9d640 1 --2- 172.21.15.16:0/2402125703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76d00d2750 0x7f76d00c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76c7fff640 1 -- 172.21.15.16:0/2402125703 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76d0072250 msgr2=0x7f76d00cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76c7fff640 1 --2- 172.21.15.16:0/2402125703 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76d0072250 0x7f76d00cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76c7fff640 1 -- 172.21.15.16:0/2402125703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76d00d2750 msgr2=0x7f76d00c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76c7fff640 1 --2- 172.21.15.16:0/2402125703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76d00d2750 0x7f76d00c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76c7fff640 1 -- 172.21.15.16:0/2402125703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d00c9800 con 0x7f76d0072690 2024-09-19T07:43:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76c7fff640 1 --2- 172.21.15.16:0/2402125703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d0072690 0x7f76d013ec70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f76b800e760 tx=0x7f76b800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76c6ffd640 1 -- 172.21.15.16:0/2402125703 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b800f040 con 0x7f76d0072690 2024-09-19T07:43:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76c6ffd640 1 -- 172.21.15.16:0/2402125703 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76b8004230 con 0x7f76d0072690 2024-09-19T07:43:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76d4add640 1 -- 172.21.15.16:0/2402125703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d00c9ad0 con 0x7f76d0072690 2024-09-19T07:43:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76c6ffd640 1 -- 172.21.15.16:0/2402125703 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b800ee70 con 0x7f76d0072690 2024-09-19T07:43:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.467+0000 7f76d4add640 1 -- 172.21.15.16:0/2402125703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d00ca090 con 0x7f76d0072690 2024-09-19T07:43:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.468+0000 7f76d4add640 1 -- 172.21.15.16:0/2402125703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d00cd6f0 con 0x7f76d0072690 2024-09-19T07:43:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.469+0000 7f76c6ffd640 1 -- 172.21.15.16:0/2402125703 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f76b800c6d0 con 0x7f76d0072690 2024-09-19T07:43:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.469+0000 7f76c6ffd640 1 --2- 172.21.15.16:0/2402125703 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f76b006c2e0 0x7f76b006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.470+0000 7f76c6ffd640 1 -- 172.21.15.16:0/2402125703 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f76b808b340 con 0x7f76d0072690 2024-09-19T07:43:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.470+0000 7f76ce59c640 1 --2- 172.21.15.16:0/2402125703 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f76b006c2e0 0x7f76b006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.470+0000 7f76ce59c640 1 --2- 172.21.15.16:0/2402125703 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f76b006c2e0 0x7f76b006e7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f76c0007fd0 tx=0x7f76c0007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:25.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.474+0000 7f76c6ffd640 1 -- 172.21.15.16:0/2402125703 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f76b8017020 con 0x7f76d0072690 2024-09-19T07:43:25.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.685+0000 7f76d4add640 1 -- 172.21.15.16:0/2402125703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76d00cd410 con 0x7f76d0072690 2024-09-19T07:43:25.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.686+0000 7f76c6ffd640 1 -- 172.21.15.16:0/2402125703 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5222 (secure 0 0 0) 0x7f76b8059120 con 0x7f76d0072690 2024-09-19T07:43:25.687 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:25.687 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-19T07:43:25:334589+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2772948330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2772948330},{"type":"v1","addr":"172.21.15.146:6835","nonce":2772948330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/340172279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":340172279},{"type":"v1","addr":"172.21.15.16:6833","nonce":340172279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1204811459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1204811459},{"type":"v1","addr":"172.21.15.16:6835","nonce":1204811459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:43:05.610038+0000","modified":"2024-09-19T07:43:24.691417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/1303703249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1303703249},{"type":"v1","addr":"172.21.15.146:6837","nonce":1303703249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-09-19T07:43:25.687 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-19T07:43:25.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.688+0000 7f76d4add640 1 -- 172.21.15.16:0/2402125703 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f76b006c2e0 msgr2=0x7f76b006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:25.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.688+0000 7f76d4add640 1 --2- 172.21.15.16:0/2402125703 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f76b006c2e0 0x7f76b006e7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f76c0007fd0 tx=0x7f76c0007e80 comp rx=0 tx=0).stop 2024-09-19T07:43:25.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.688+0000 7f76d4add640 1 -- 172.21.15.16:0/2402125703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d0072690 msgr2=0x7f76d013ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.688+0000 7f76d4add640 1 --2- 172.21.15.16:0/2402125703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d0072690 0x7f76d013ec70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f76b800e760 tx=0x7f76b800ec20 comp rx=0 tx=0).stop 2024-09-19T07:43:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.688+0000 7f76d4add640 1 -- 172.21.15.16:0/2402125703 shutdown_connections 2024-09-19T07:43:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.688+0000 7f76d4add640 1 --2- 172.21.15.16:0/2402125703 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f76b006c2e0 0x7f76b006e7a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.688+0000 7f76d4add640 1 --2- 172.21.15.16:0/2402125703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76d00d2750 0x7f76d00c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.688+0000 7f76d4add640 1 --2- 172.21.15.16:0/2402125703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d0072690 0x7f76d013ec70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.688+0000 7f76d4add640 1 --2- 172.21.15.16:0/2402125703 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76d0072250 0x7f76d00cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.688+0000 7f76d4add640 1 -- 172.21.15.16:0/2402125703 >> 172.21.15.16:0/2402125703 conn(0x7f76d00b4040 msgr2=0x7f76d00cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.688+0000 7f76d4add640 1 -- 172.21.15.16:0/2402125703 shutdown_connections 2024-09-19T07:43:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.688+0000 7f76d4add640 1 -- 172.21.15.16:0/2402125703 wait complete. 2024-09-19T07:43:25.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:43:25.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.801+0000 7f467409b640 1 Processor -- start 2024-09-19T07:43:25.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.801+0000 7f467409b640 1 -- start start 2024-09-19T07:43:25.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.802+0000 7f467409b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f466c15f0d0 0x7f466c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:25.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.802+0000 7f467409b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f466c0d4130 0x7f466c0d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:25.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.802+0000 7f467409b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f466c0d4ae0 con 0x7f466c0d0070 2024-09-19T07:43:25.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.802+0000 7f467409b640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f466c0d1b10 con 0x7f466c15f0d0 2024-09-19T07:43:25.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.802+0000 7f467409b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f466c0cf2b0 con 0x7f466c0d4130 2024-09-19T07:43:25.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.802+0000 7f4671636640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f466c0d0070 0x7f466c0d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44812/0 (socket says 172.21.15.16:44812) 2024-09-19T07:43:25.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.802+0000 7f4671636640 1 -- 172.21.15.16:0/596671246 learned_addr learned my addr 172.21.15.16:0/596671246 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:25.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.802+0000 7f4671e37640 1 --2- 172.21.15.16:0/596671246 >> v2:172.21.15.146:3300/0 conn(0x7f466c15f0d0 0x7f466c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.802+0000 7f4672638640 1 --2- 172.21.15.16:0/596671246 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f466c0d4130 0x7f466c0d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.802+0000 7f4670e35640 1 -- 172.21.15.16:0/596671246 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2253217509 0 0) 0x7f466c0d4ae0 con 0x7f466c0d0070 2024-09-19T07:43:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.802+0000 7f4670e35640 1 -- 172.21.15.16:0/596671246 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4658003520 con 0x7f466c0d0070 2024-09-19T07:43:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.803+0000 7f4671e37640 1 -- 172.21.15.16:0/596671246 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f466c0d4130 msgr2=0x7f466c0d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.803+0000 7f4671e37640 1 --2- 172.21.15.16:0/596671246 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f466c0d4130 0x7f466c0d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.803+0000 7f4671e37640 1 -- 172.21.15.16:0/596671246 >> v1:172.21.15.16:6789/0 conn(0x7f466c0d0070 legacy=0x7f466c0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.803+0000 7f4670e35640 1 -- 172.21.15.16:0/596671246 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2606514579 0 0) 0x7f4658003520 con 0x7f466c0d0070 2024-09-19T07:43:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.803+0000 7f4671e37640 1 -- 172.21.15.16:0/596671246 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466c0cf590 con 0x7f466c15f0d0 2024-09-19T07:43:25.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.803+0000 7f4671e37640 1 --2- 172.21.15.16:0/596671246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f466c15f0d0 0x7f466c15f4b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f46680097a0 tx=0x7f466802a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f076499a09df1f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:25.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.804+0000 7f4670e35640 1 -- 172.21.15.16:0/596671246 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668037070 con 0x7f466c15f0d0 2024-09-19T07:43:25.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.804+0000 7f4670e35640 1 -- 172.21.15.16:0/596671246 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f466802ab10 con 0x7f466c15f0d0 2024-09-19T07:43:25.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.804+0000 7f4670e35640 1 -- 172.21.15.16:0/596671246 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660001670 con 0x7f466c15f0d0 2024-09-19T07:43:25.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.804+0000 7f467409b640 1 -- 172.21.15.16:0/596671246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f466c15f0d0 msgr2=0x7f466c15f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:25.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.804+0000 7f467409b640 1 --2- 172.21.15.16:0/596671246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f466c15f0d0 0x7f466c15f4b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f46680097a0 tx=0x7f466802a080 comp rx=0 tx=0).stop 2024-09-19T07:43:25.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.804+0000 7f467409b640 1 -- 172.21.15.16:0/596671246 shutdown_connections 2024-09-19T07:43:25.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.804+0000 7f467409b640 1 --2- 172.21.15.16:0/596671246 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f466c0d4130 0x7f466c0d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.804+0000 7f467409b640 1 --2- 172.21.15.16:0/596671246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f466c15f0d0 0x7f466c15f4b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.804+0000 7f467409b640 1 -- 172.21.15.16:0/596671246 >> 172.21.15.16:0/596671246 conn(0x7f466c09d760 msgr2=0x7f466c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:25.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.804+0000 7f467409b640 1 -- 172.21.15.16:0/596671246 shutdown_connections 2024-09-19T07:43:25.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.804+0000 7f467409b640 1 -- 172.21.15.16:0/596671246 wait complete. 2024-09-19T07:43:25.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.805+0000 7f467409b640 1 Processor -- start 2024-09-19T07:43:25.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.805+0000 7f467409b640 1 -- start start 2024-09-19T07:43:25.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.806+0000 7f467409b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f466c0d0070 0x7f466c157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.806+0000 7f467409b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f466c0d4130 0x7f466c160f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.806+0000 7f467409b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f466c15f0d0 0x7f466c0b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.806+0000 7f467409b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f466c0cef60 con 0x7f466c0d4130 2024-09-19T07:43:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.806+0000 7f467409b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f466c0cfb50 con 0x7f466c15f0d0 2024-09-19T07:43:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.806+0000 7f467409b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f466c0b2f70 con 0x7f466c0d0070 2024-09-19T07:43:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.806+0000 7f4671636640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f466c0d4130 0x7f466c160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.806+0000 7f4671636640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f466c0d4130 0x7f466c160f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57230/0 (socket says 172.21.15.16:57230) 2024-09-19T07:43:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.806+0000 7f4671636640 1 -- 172.21.15.16:0/1646440038 learned_addr learned my addr 172.21.15.16:0/1646440038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.807+0000 7f4671e37640 1 --2- 172.21.15.16:0/1646440038 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f466c0d0070 0x7f466c157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.807+0000 7f4672638640 1 --2- 172.21.15.16:0/1646440038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f466c15f0d0 0x7f466c0b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.807+0000 7f4671636640 1 -- 172.21.15.16:0/1646440038 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f466c0d0070 msgr2=0x7f466c157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.807+0000 7f4671636640 1 --2- 172.21.15.16:0/1646440038 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f466c0d0070 0x7f466c157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.807+0000 7f4671636640 1 -- 172.21.15.16:0/1646440038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f466c15f0d0 msgr2=0x7f466c0b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:25.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.807+0000 7f4671636640 1 --2- 172.21.15.16:0/1646440038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f466c15f0d0 0x7f466c0b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.807+0000 7f4671636640 1 -- 172.21.15.16:0/1646440038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466c0c93d0 con 0x7f466c0d4130 2024-09-19T07:43:25.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.807+0000 7f4672638640 1 --2- 172.21.15.16:0/1646440038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f466c15f0d0 0x7f466c0b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:25.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.807+0000 7f4671e37640 1 --2- 172.21.15.16:0/1646440038 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f466c0d0070 0x7f466c157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:25.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.807+0000 7f4671636640 1 --2- 172.21.15.16:0/1646440038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f466c0d4130 0x7f466c160f20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f466000c780 tx=0x7f466000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:25.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.807+0000 7f465e7fc640 1 -- 172.21.15.16:0/1646440038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660017070 con 0x7f466c0d4130 2024-09-19T07:43:25.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.807+0000 7f465e7fc640 1 -- 172.21.15.16:0/1646440038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4660005ad0 con 0x7f466c0d4130 2024-09-19T07:43:25.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.807+0000 7f465e7fc640 1 -- 172.21.15.16:0/1646440038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660007070 con 0x7f466c0d4130 2024-09-19T07:43:25.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.807+0000 7f467409b640 1 -- 172.21.15.16:0/1646440038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466c0c96b0 con 0x7f466c0d4130 2024-09-19T07:43:25.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.807+0000 7f467409b640 1 -- 172.21.15.16:0/1646440038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f466c0c9c70 con 0x7f466c0d4130 2024-09-19T07:43:25.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.808+0000 7f467409b640 1 -- 172.21.15.16:0/1646440038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f466c15f4b0 con 0x7f466c0d4130 2024-09-19T07:43:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.809+0000 7f465e7fc640 1 -- 172.21.15.16:0/1646440038 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f466001a070 con 0x7f466c0d4130 2024-09-19T07:43:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.809+0000 7f465e7fc640 1 --2- 172.21.15.16:0/1646440038 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f465006c2b0 0x7f465006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.809+0000 7f465e7fc640 1 -- 172.21.15.16:0/1646440038 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f466008ad20 con 0x7f466c0d4130 2024-09-19T07:43:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.810+0000 7f4671e37640 1 --2- 172.21.15.16:0/1646440038 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f465006c2b0 0x7f465006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.810+0000 7f4671e37640 1 --2- 172.21.15.16:0/1646440038 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f465006c2b0 0x7f465006e770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4668034000 tx=0x7f466802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:25.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.814+0000 7f465e7fc640 1 -- 172.21.15.16:0/1646440038 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4660059150 con 0x7f466c0d4130 2024-09-19T07:43:25.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.990+0000 7f467409b640 1 -- 172.21.15.16:0/1646440038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f466c06cf30 con 0x7f466c0d4130 2024-09-19T07:43:25.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.991+0000 7f465e7fc640 1 -- 172.21.15.16:0/1646440038 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7f4660058a50 con 0x7f466c0d4130 2024-09-19T07:43:25.992 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:25.992 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:43:10.642082+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:43:03.485582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:43:04.603961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:43:08.299591+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:43:25.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.993+0000 7f467409b640 1 -- 172.21.15.16:0/1646440038 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f465006c2b0 msgr2=0x7f465006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:25.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.993+0000 7f467409b640 1 --2- 172.21.15.16:0/1646440038 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f465006c2b0 0x7f465006e770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4668034000 tx=0x7f466802e000 comp rx=0 tx=0).stop 2024-09-19T07:43:25.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.993+0000 7f467409b640 1 -- 172.21.15.16:0/1646440038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f466c0d4130 msgr2=0x7f466c160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:25.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.993+0000 7f467409b640 1 --2- 172.21.15.16:0/1646440038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f466c0d4130 0x7f466c160f20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f466000c780 tx=0x7f466000cc40 comp rx=0 tx=0).stop 2024-09-19T07:43:25.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.993+0000 7f467409b640 1 -- 172.21.15.16:0/1646440038 shutdown_connections 2024-09-19T07:43:25.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.993+0000 7f467409b640 1 --2- 172.21.15.16:0/1646440038 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f465006c2b0 0x7f465006e770 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.993+0000 7f467409b640 1 --2- 172.21.15.16:0/1646440038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f466c15f0d0 0x7f466c0b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.993+0000 7f467409b640 1 --2- 172.21.15.16:0/1646440038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f466c0d4130 0x7f466c160f20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.993+0000 7f467409b640 1 --2- 172.21.15.16:0/1646440038 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f466c0d0070 0x7f466c157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:25.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.993+0000 7f467409b640 1 -- 172.21.15.16:0/1646440038 >> 172.21.15.16:0/1646440038 conn(0x7f466c09d760 msgr2=0x7f466c0d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:25.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.993+0000 7f467409b640 1 -- 172.21.15.16:0/1646440038 shutdown_connections 2024-09-19T07:43:25.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:25.993+0000 7f467409b640 1 -- 172.21.15.16:0/1646440038 wait complete. 2024-09-19T07:43:26.005 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T07:43:26.005 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:43:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.107+0000 7fe8fc9c4640 1 Processor -- start 2024-09-19T07:43:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.107+0000 7fe8fc9c4640 1 -- start start 2024-09-19T07:43:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.108+0000 7fe8fc9c4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8f8071090 0x7fe8f8162030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.108+0000 7fe8fc9c4640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe8f8162570 0x7fe8f806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.108+0000 7fe8fc9c4640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8f815e940 con 0x7fe8f815e560 2024-09-19T07:43:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.108+0000 7fe8fc9c4640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe8f81629e0 con 0x7fe8f8162570 2024-09-19T07:43:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.108+0000 7fe8fc9c4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe8f8070010 con 0x7fe8f8071090 2024-09-19T07:43:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.108+0000 7fe8effff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8f8071090 0x7fe8f8162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.108+0000 7fe8f6d9d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe8f8162570 0x7fe8f806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.108+0000 7fe8effff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8f8071090 0x7fe8f8162030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:33966/0 (socket says 172.21.15.16:33966) 2024-09-19T07:43:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.108+0000 7fe8effff640 1 -- 172.21.15.16:0/16790731 learned_addr learned my addr 172.21.15.16:0/16790731 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.108+0000 7fe8effff640 1 -- 172.21.15.16:0/16790731 >> v2:172.21.15.146:3300/0 conn(0x7fe8f8162570 msgr2=0x7fe8f806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.108+0000 7fe8effff640 1 --2- 172.21.15.16:0/16790731 >> v2:172.21.15.146:3300/0 conn(0x7fe8f8162570 0x7fe8f806cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.108+0000 7fe8effff640 1 -- 172.21.15.16:0/16790731 >> v1:172.21.15.146:6789/0 conn(0x7fe8f815e560 legacy=0x7fe8f80d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:26.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.108+0000 7fe8effff640 1 -- 172.21.15.16:0/16790731 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f80702d0 con 0x7fe8f8071090 2024-09-19T07:43:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.109+0000 7fe8effff640 1 --2- 172.21.15.16:0/16790731 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8f8071090 0x7fe8f8162030 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009870 tx=0x7fe8e002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a4978fa522594f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.109+0000 7fe8f5d9b640 1 -- 172.21.15.16:0/16790731 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0037070 con 0x7fe8f8071090 2024-09-19T07:43:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.109+0000 7fe8f5d9b640 1 -- 172.21.15.16:0/16790731 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e0032440 con 0x7fe8f8071090 2024-09-19T07:43:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.109+0000 7fe8f5d9b640 1 -- 172.21.15.16:0/16790731 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0032810 con 0x7fe8f8071090 2024-09-19T07:43:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.110+0000 7fe8fc9c4640 1 -- 172.21.15.16:0/16790731 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8f8071090 msgr2=0x7fe8f8162030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.110+0000 7fe8fc9c4640 1 --2- 172.21.15.16:0/16790731 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8f8071090 0x7fe8f8162030 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009870 tx=0x7fe8e002a1c0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.110+0000 7fe8fc9c4640 1 -- 172.21.15.16:0/16790731 shutdown_connections 2024-09-19T07:43:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.110+0000 7fe8fc9c4640 1 --2- 172.21.15.16:0/16790731 >> v2:172.21.15.146:3300/0 conn(0x7fe8f8162570 0x7fe8f806cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.110+0000 7fe8fc9c4640 1 --2- 172.21.15.16:0/16790731 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8f8071090 0x7fe8f8162030 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.110+0000 7fe8fc9c4640 1 -- 172.21.15.16:0/16790731 >> 172.21.15.16:0/16790731 conn(0x7fe8f80d2230 msgr2=0x7fe8f80d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.110+0000 7fe8fc9c4640 1 -- 172.21.15.16:0/16790731 shutdown_connections 2024-09-19T07:43:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.110+0000 7fe8fc9c4640 1 -- 172.21.15.16:0/16790731 wait complete. 2024-09-19T07:43:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.111+0000 7fe8fc9c4640 1 Processor -- start 2024-09-19T07:43:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.111+0000 7fe8fc9c4640 1 -- start start 2024-09-19T07:43:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.112+0000 7fe8fc9c4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8f8071090 0x7fe8f80b7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.112+0000 7fe8fc9c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f815e560 0x7fe8f80b6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.112+0000 7fe8fc9c4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8f8162570 0x7fe8f80b6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.112+0000 7fe8fc9c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f8070b70 con 0x7fe8f815e560 2024-09-19T07:43:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.112+0000 7fe8fc9c4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f8070890 con 0x7fe8f8071090 2024-09-19T07:43:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.112+0000 7fe8fc9c4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe8f8162d90 con 0x7fe8f8162570 2024-09-19T07:43:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.112+0000 7fe8effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f815e560 0x7fe8f80b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.112+0000 7fe8effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f815e560 0x7fe8f80b6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57234/0 (socket says 172.21.15.16:57234) 2024-09-19T07:43:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.112+0000 7fe8effff640 1 -- 172.21.15.16:0/4190899845 learned_addr learned my addr 172.21.15.16:0/4190899845 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.112+0000 7fe8f659c640 1 --2- 172.21.15.16:0/4190899845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8f8071090 0x7fe8f80b7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.112+0000 7fe8f6d9d640 1 --2- 172.21.15.16:0/4190899845 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8f8162570 0x7fe8f80b6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.113+0000 7fe8effff640 1 -- 172.21.15.16:0/4190899845 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8f8162570 msgr2=0x7fe8f80b6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.113+0000 7fe8effff640 1 --2- 172.21.15.16:0/4190899845 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8f8162570 0x7fe8f80b6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.113+0000 7fe8effff640 1 -- 172.21.15.16:0/4190899845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8f8071090 msgr2=0x7fe8f80b7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.113+0000 7fe8effff640 1 --2- 172.21.15.16:0/4190899845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8f8071090 0x7fe8f80b7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.113+0000 7fe8effff640 1 -- 172.21.15.16:0/4190899845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f815ed20 con 0x7fe8f815e560 2024-09-19T07:43:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.113+0000 7fe8f659c640 1 --2- 172.21.15.16:0/4190899845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8f8071090 0x7fe8f80b7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.113+0000 7fe8effff640 1 --2- 172.21.15.16:0/4190899845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f815e560 0x7fe8f80b6810 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0000c00 tx=0x7fe8e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.113+0000 7fe8eeffd640 1 -- 172.21.15.16:0/4190899845 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0037040 con 0x7fe8f815e560 2024-09-19T07:43:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.113+0000 7fe8eeffd640 1 -- 172.21.15.16:0/4190899845 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e0042070 con 0x7fe8f815e560 2024-09-19T07:43:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.113+0000 7fe8fc9c4640 1 -- 172.21.15.16:0/4190899845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f815f000 con 0x7fe8f815e560 2024-09-19T07:43:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.113+0000 7fe8eeffd640 1 -- 172.21.15.16:0/4190899845 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0030860 con 0x7fe8f815e560 2024-09-19T07:43:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.113+0000 7fe8fc9c4640 1 -- 172.21.15.16:0/4190899845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f815f5c0 con 0x7fe8f815e560 2024-09-19T07:43:26.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.114+0000 7fe8fc9c4640 1 -- 172.21.15.16:0/4190899845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f8162030 con 0x7fe8f815e560 2024-09-19T07:43:26.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.115+0000 7fe8eeffd640 1 -- 172.21.15.16:0/4190899845 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe8e002ad10 con 0x7fe8f815e560 2024-09-19T07:43:26.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.115+0000 7fe8eeffd640 1 --2- 172.21.15.16:0/4190899845 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe8d806c2b0 0x7fe8d806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:26.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.115+0000 7fe8eeffd640 1 -- 172.21.15.16:0/4190899845 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe8e00abbf0 con 0x7fe8f815e560 2024-09-19T07:43:26.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.115+0000 7fe8f659c640 1 --2- 172.21.15.16:0/4190899845 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe8d806c2b0 0x7fe8d806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:26.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.116+0000 7fe8f659c640 1 --2- 172.21.15.16:0/4190899845 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe8d806c2b0 0x7fe8d806e770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8001ad0 tx=0x7fe8e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.123+0000 7fe8eeffd640 1 -- 172.21.15.16:0/4190899845 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe8e00799d0 con 0x7fe8f815e560 2024-09-19T07:43:26.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.299+0000 7fe8fc9c4640 1 -- 172.21.15.16:0/4190899845 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe8f80d2e90 con 0x7fe8d806c2b0 2024-09-19T07:43:26.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.308+0000 7fe8eeffd640 1 -- 172.21.15.16:0/4190899845 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fe8f80d2e90 con 0x7fe8d806c2b0 2024-09-19T07:43:26.313 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:43:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.312+0000 7fe8fc9c4640 1 -- 172.21.15.16:0/4190899845 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe8d806c2b0 msgr2=0x7fe8d806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.312+0000 7fe8fc9c4640 1 --2- 172.21.15.16:0/4190899845 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe8d806c2b0 0x7fe8d806e770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8001ad0 tx=0x7fe8e8009040 comp rx=0 tx=0).stop 2024-09-19T07:43:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.312+0000 7fe8fc9c4640 1 -- 172.21.15.16:0/4190899845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f815e560 msgr2=0x7fe8f80b6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.312+0000 7fe8fc9c4640 1 --2- 172.21.15.16:0/4190899845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f815e560 0x7fe8f80b6810 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0000c00 tx=0x7fe8e00046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.312+0000 7fe8fc9c4640 1 -- 172.21.15.16:0/4190899845 shutdown_connections 2024-09-19T07:43:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.312+0000 7fe8fc9c4640 1 --2- 172.21.15.16:0/4190899845 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe8d806c2b0 0x7fe8d806e770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.312+0000 7fe8fc9c4640 1 --2- 172.21.15.16:0/4190899845 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8f8162570 0x7fe8f80b6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.312+0000 7fe8fc9c4640 1 --2- 172.21.15.16:0/4190899845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f815e560 0x7fe8f80b6810 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.312+0000 7fe8fc9c4640 1 --2- 172.21.15.16:0/4190899845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8f8071090 0x7fe8f80b7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.312+0000 7fe8fc9c4640 1 -- 172.21.15.16:0/4190899845 >> 172.21.15.16:0/4190899845 conn(0x7fe8f80d2230 msgr2=0x7fe8f815d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.312+0000 7fe8fc9c4640 1 -- 172.21.15.16:0/4190899845 shutdown_connections 2024-09-19T07:43:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.312+0000 7fe8fc9c4640 1 -- 172.21.15.16:0/4190899845 wait complete. 2024-09-19T07:43:26.325 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-19T07:43:24.462010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236216,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738504,"statfs":{"total":773094113280,"available":772852228096,"internally_reserved":0,"allocated":3416064,"data_stored":2041588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003522"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656052+0000","last_change":"2024-09-19T07:43:05.265202+0000","last_active":"2024-09-19T07:43:12.656052+0000","last_peered":"2024-09-19T07:43:12.656052+0000","last_clean":"2024-09-19T07:43:12.656052+0000","last_became_active":"2024-09-19T07:43:05.264871+0000","last_became_peered":"2024-09-19T07:43:05.264871+0000","last_unstale":"2024-09-19T07:43:12.656052+0000","last_undegraded":"2024-09-19T07:43:12.656052+0000","last_fullsized":"2024-09-19T07:43:12.656052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:36:39.982941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310781+0000","last_change":"2024-09-19T07:43:06.264003+0000","last_active":"2024-09-19T07:43:11.310781+0000","last_peered":"2024-09-19T07:43:11.310781+0000","last_clean":"2024-09-19T07:43:11.310781+0000","last_became_active":"2024-09-19T07:43:06.262313+0000","last_became_peered":"2024-09-19T07:43:06.262313+0000","last_unstale":"2024-09-19T07:43:11.310781+0000","last_undegraded":"2024-09-19T07:43:11.310781+0000","last_fullsized":"2024-09-19T07:43:11.310781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.309855+0000","last_change":"2024-09-19T07:43:10.483058+0000","last_active":"2024-09-19T07:43:11.309855+0000","last_peered":"2024-09-19T07:43:11.309855+0000","last_clean":"2024-09-19T07:43:11.309855+0000","last_became_active":"2024-09-19T07:43:10.482621+0000","last_became_peered":"2024-09-19T07:43:10.482621+0000","last_unstale":"2024-09-19T07:43:11.309855+0000","last_undegraded":"2024-09-19T07:43:11.309855+0000","last_fullsized":"2024-09-19T07:43:11.309855+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:27.168500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310785+0000","last_change":"2024-09-19T07:43:05.265029+0000","last_active":"2024-09-19T07:43:11.310785+0000","last_peered":"2024-09-19T07:43:11.310785+0000","last_clean":"2024-09-19T07:43:11.310785+0000","last_became_active":"2024-09-19T07:43:05.264529+0000","last_became_peered":"2024-09-19T07:43:05.264529+0000","last_unstale":"2024-09-19T07:43:11.310785+0000","last_undegraded":"2024-09-19T07:43:11.310785+0000","last_fullsized":"2024-09-19T07:43:11.310785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:12.998698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655877+0000","last_change":"2024-09-19T07:43:06.272531+0000","last_active":"2024-09-19T07:43:12.655877+0000","last_peered":"2024-09-19T07:43:12.655877+0000","last_clean":"2024-09-19T07:43:12.655877+0000","last_became_active":"2024-09-19T07:43:06.272225+0000","last_became_peered":"2024-09-19T07:43:06.272225+0000","last_unstale":"2024-09-19T07:43:12.655877+0000","last_undegraded":"2024-09-19T07:43:12.655877+0000","last_fullsized":"2024-09-19T07:43:12.655877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:06.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311381+0000","last_change":"2024-09-19T07:43:10.288276+0000","last_active":"2024-09-19T07:43:11.311381+0000","last_peered":"2024-09-19T07:43:11.311381+0000","last_clean":"2024-09-19T07:43:11.311381+0000","last_became_active":"2024-09-19T07:43:10.287816+0000","last_became_peered":"2024-09-19T07:43:10.287816+0000","last_unstale":"2024-09-19T07:43:11.311381+0000","last_undegraded":"2024-09-19T07:43:11.311381+0000","last_fullsized":"2024-09-19T07:43:11.311381+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:26.830106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681715+0000","last_change":"2024-09-19T07:43:05.266982+0000","last_active":"2024-09-19T07:43:12.681715+0000","last_peered":"2024-09-19T07:43:12.681715+0000","last_clean":"2024-09-19T07:43:12.681715+0000","last_became_active":"2024-09-19T07:43:05.266667+0000","last_became_peered":"2024-09-19T07:43:05.266667+0000","last_unstale":"2024-09-19T07:43:12.681715+0000","last_undegraded":"2024-09-19T07:43:12.681715+0000","last_fullsized":"2024-09-19T07:43:12.681715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:02.488778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312020+0000","last_change":"2024-09-19T07:43:06.273456+0000","last_active":"2024-09-19T07:43:11.312020+0000","last_peered":"2024-09-19T07:43:11.312020+0000","last_clean":"2024-09-19T07:43:11.312020+0000","last_became_active":"2024-09-19T07:43:06.273046+0000","last_became_peered":"2024-09-19T07:43:06.273046+0000","last_unstale":"2024-09-19T07:43:11.312020+0000","last_undegraded":"2024-09-19T07:43:11.312020+0000","last_fullsized":"2024-09-19T07:43:11.312020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:09.222719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310977+0000","last_change":"2024-09-19T07:43:10.293267+0000","last_active":"2024-09-19T07:43:11.310977+0000","last_peered":"2024-09-19T07:43:11.310977+0000","last_clean":"2024-09-19T07:43:11.310977+0000","last_became_active":"2024-09-19T07:43:10.292870+0000","last_became_peered":"2024-09-19T07:43:10.292870+0000","last_unstale":"2024-09-19T07:43:11.310977+0000","last_undegraded":"2024-09-19T07:43:11.310977+0000","last_fullsized":"2024-09-19T07:43:11.310977+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:54.570005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681497+0000","last_change":"2024-09-19T07:43:05.267905+0000","last_active":"2024-09-19T07:43:12.681497+0000","last_peered":"2024-09-19T07:43:12.681497+0000","last_clean":"2024-09-19T07:43:12.681497+0000","last_became_active":"2024-09-19T07:43:05.267563+0000","last_became_peered":"2024-09-19T07:43:05.267563+0000","last_unstale":"2024-09-19T07:43:12.681497+0000","last_undegraded":"2024-09-19T07:43:12.681497+0000","last_fullsized":"2024-09-19T07:43:12.681497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:47.531702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311784+0000","last_change":"2024-09-19T07:43:06.262966+0000","last_active":"2024-09-19T07:43:11.311784+0000","last_peered":"2024-09-19T07:43:11.311784+0000","last_clean":"2024-09-19T07:43:11.311784+0000","last_became_active":"2024-09-19T07:43:06.262698+0000","last_became_peered":"2024-09-19T07:43:06.262698+0000","last_unstale":"2024-09-19T07:43:11.311784+0000","last_undegraded":"2024-09-19T07:43:11.311784+0000","last_fullsized":"2024-09-19T07:43:11.311784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:47.918580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310630+0000","last_change":"2024-09-19T07:43:10.287084+0000","last_active":"2024-09-19T07:43:11.310630+0000","last_peered":"2024-09-19T07:43:11.310630+0000","last_clean":"2024-09-19T07:43:11.310630+0000","last_became_active":"2024-09-19T07:43:10.286693+0000","last_became_peered":"2024-09-19T07:43:10.286693+0000","last_unstale":"2024-09-19T07:43:11.310630+0000","last_undegraded":"2024-09-19T07:43:11.310630+0000","last_fullsized":"2024-09-19T07:43:11.310630+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:17.611361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450546+0000","last_change":"2024-09-19T07:43:06.262664+0000","last_active":"2024-09-19T07:43:11.450546+0000","last_peered":"2024-09-19T07:43:11.450546+0000","last_clean":"2024-09-19T07:43:11.450546+0000","last_became_active":"2024-09-19T07:43:06.262389+0000","last_became_peered":"2024-09-19T07:43:06.262389+0000","last_unstale":"2024-09-19T07:43:11.450546+0000","last_undegraded":"2024-09-19T07:43:11.450546+0000","last_fullsized":"2024-09-19T07:43:11.450546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:06.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681454+0000","last_change":"2024-09-19T07:43:05.262431+0000","last_active":"2024-09-19T07:43:12.681454+0000","last_peered":"2024-09-19T07:43:12.681454+0000","last_clean":"2024-09-19T07:43:12.681454+0000","last_became_active":"2024-09-19T07:43:05.262156+0000","last_became_peered":"2024-09-19T07:43:05.262156+0000","last_unstale":"2024-09-19T07:43:12.681454+0000","last_undegraded":"2024-09-19T07:43:12.681454+0000","last_fullsized":"2024-09-19T07:43:12.681454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:27.338560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312544+0000","last_change":"2024-09-19T07:43:10.287687+0000","last_active":"2024-09-19T07:43:11.312544+0000","last_peered":"2024-09-19T07:43:11.312544+0000","last_clean":"2024-09-19T07:43:11.312544+0000","last_became_active":"2024-09-19T07:43:10.287301+0000","last_became_peered":"2024-09-19T07:43:10.287301+0000","last_unstale":"2024-09-19T07:43:11.312544+0000","last_undegraded":"2024-09-19T07:43:11.312544+0000","last_fullsized":"2024-09-19T07:43:11.312544+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:43.684480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310291+0000","last_change":"2024-09-19T07:43:06.274918+0000","last_active":"2024-09-19T07:43:11.310291+0000","last_peered":"2024-09-19T07:43:11.310291+0000","last_clean":"2024-09-19T07:43:11.310291+0000","last_became_active":"2024-09-19T07:43:06.274639+0000","last_became_peered":"2024-09-19T07:43:06.274639+0000","last_unstale":"2024-09-19T07:43:11.310291+0000","last_undegraded":"2024-09-19T07:43:11.310291+0000","last_fullsized":"2024-09-19T07:43:11.310291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:22.804370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680919+0000","last_change":"2024-09-19T07:43:05.258127+0000","last_active":"2024-09-19T07:43:12.680919+0000","last_peered":"2024-09-19T07:43:12.680919+0000","last_clean":"2024-09-19T07:43:12.680919+0000","last_became_active":"2024-09-19T07:43:05.257766+0000","last_became_peered":"2024-09-19T07:43:05.257766+0000","last_unstale":"2024-09-19T07:43:12.680919+0000","last_undegraded":"2024-09-19T07:43:12.680919+0000","last_fullsized":"2024-09-19T07:43:12.680919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:02.488778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311080+0000","last_change":"2024-09-19T07:43:10.294352+0000","last_active":"2024-09-19T07:43:11.311080+0000","last_peered":"2024-09-19T07:43:11.311080+0000","last_clean":"2024-09-19T07:43:11.311080+0000","last_became_active":"2024-09-19T07:43:10.293957+0000","last_became_peered":"2024-09-19T07:43:10.293957+0000","last_unstale":"2024-09-19T07:43:11.311080+0000","last_undegraded":"2024-09-19T07:43:11.311080+0000","last_fullsized":"2024-09-19T07:43:11.311080+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:13.248454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310160+0000","last_change":"2024-09-19T07:43:06.256705+0000","last_active":"2024-09-19T07:43:11.310160+0000","last_peered":"2024-09-19T07:43:11.310160+0000","last_clean":"2024-09-19T07:43:11.310160+0000","last_became_active":"2024-09-19T07:43:06.256423+0000","last_became_peered":"2024-09-19T07:43:06.256423+0000","last_unstale":"2024-09-19T07:43:11.310160+0000","last_undegraded":"2024-09-19T07:43:11.310160+0000","last_fullsized":"2024-09-19T07:43:11.310160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:47.918580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680712+0000","last_change":"2024-09-19T07:43:05.260367+0000","last_active":"2024-09-19T07:43:12.680712+0000","last_peered":"2024-09-19T07:43:12.680712+0000","last_clean":"2024-09-19T07:43:12.680712+0000","last_became_active":"2024-09-19T07:43:05.260056+0000","last_became_peered":"2024-09-19T07:43:05.260056+0000","last_unstale":"2024-09-19T07:43:12.680712+0000","last_undegraded":"2024-09-19T07:43:12.680712+0000","last_fullsized":"2024-09-19T07:43:12.680712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:38:29.212025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681342+0000","last_change":"2024-09-19T07:43:10.482020+0000","last_active":"2024-09-19T07:43:12.681342+0000","last_peered":"2024-09-19T07:43:12.681342+0000","last_clean":"2024-09-19T07:43:12.681342+0000","last_became_active":"2024-09-19T07:43:10.480883+0000","last_became_peered":"2024-09-19T07:43:10.480883+0000","last_unstale":"2024-09-19T07:43:12.681342+0000","last_undegraded":"2024-09-19T07:43:12.681342+0000","last_fullsized":"2024-09-19T07:43:12.681342+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:01.954262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312259+0000","last_change":"2024-09-19T07:43:06.268719+0000","last_active":"2024-09-19T07:43:11.312259+0000","last_peered":"2024-09-19T07:43:11.312259+0000","last_clean":"2024-09-19T07:43:11.312259+0000","last_became_active":"2024-09-19T07:43:06.268405+0000","last_became_peered":"2024-09-19T07:43:06.268405+0000","last_unstale":"2024-09-19T07:43:11.312259+0000","last_undegraded":"2024-09-19T07:43:11.312259+0000","last_fullsized":"2024-09-19T07:43:11.312259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:45.364876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681025+0000","last_change":"2024-09-19T07:43:05.267962+0000","last_active":"2024-09-19T07:43:12.681025+0000","last_peered":"2024-09-19T07:43:12.681025+0000","last_clean":"2024-09-19T07:43:12.681025+0000","last_became_active":"2024-09-19T07:43:05.267670+0000","last_became_peered":"2024-09-19T07:43:05.267670+0000","last_unstale":"2024-09-19T07:43:12.681025+0000","last_undegraded":"2024-09-19T07:43:12.681025+0000","last_fullsized":"2024-09-19T07:43:12.681025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:13.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312584+0000","last_change":"2024-09-19T07:43:10.289357+0000","last_active":"2024-09-19T07:43:11.312584+0000","last_peered":"2024-09-19T07:43:11.312584+0000","last_clean":"2024-09-19T07:43:11.312584+0000","last_became_active":"2024-09-19T07:43:10.288596+0000","last_became_peered":"2024-09-19T07:43:10.288596+0000","last_unstale":"2024-09-19T07:43:11.312584+0000","last_undegraded":"2024-09-19T07:43:11.312584+0000","last_fullsized":"2024-09-19T07:43:11.312584+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:08.193959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656239+0000","last_change":"2024-09-19T07:43:06.270766+0000","last_active":"2024-09-19T07:43:12.656239+0000","last_peered":"2024-09-19T07:43:12.656239+0000","last_clean":"2024-09-19T07:43:12.656239+0000","last_became_active":"2024-09-19T07:43:06.270494+0000","last_became_peered":"2024-09-19T07:43:06.270494+0000","last_unstale":"2024-09-19T07:43:12.656239+0000","last_undegraded":"2024-09-19T07:43:12.656239+0000","last_fullsized":"2024-09-19T07:43:12.656239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451242+0000","last_change":"2024-09-19T07:43:05.263058+0000","last_active":"2024-09-19T07:43:11.451242+0000","last_peered":"2024-09-19T07:43:11.451242+0000","last_clean":"2024-09-19T07:43:11.451242+0000","last_became_active":"2024-09-19T07:43:05.262762+0000","last_became_peered":"2024-09-19T07:43:05.262762+0000","last_unstale":"2024-09-19T07:43:11.451242+0000","last_undegraded":"2024-09-19T07:43:11.451242+0000","last_fullsized":"2024-09-19T07:43:11.451242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:38:29.212025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312088+0000","last_change":"2024-09-19T07:43:10.482293+0000","last_active":"2024-09-19T07:43:11.312088+0000","last_peered":"2024-09-19T07:43:11.312088+0000","last_clean":"2024-09-19T07:43:11.312088+0000","last_became_active":"2024-09-19T07:43:10.481268+0000","last_became_peered":"2024-09-19T07:43:10.481268+0000","last_unstale":"2024-09-19T07:43:11.312088+0000","last_undegraded":"2024-09-19T07:43:11.312088+0000","last_fullsized":"2024-09-19T07:43:11.312088+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:51.944316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311136+0000","last_change":"2024-09-19T07:43:06.268026+0000","last_active":"2024-09-19T07:43:11.311136+0000","last_peered":"2024-09-19T07:43:11.311136+0000","last_clean":"2024-09-19T07:43:11.311136+0000","last_became_active":"2024-09-19T07:43:06.267713+0000","last_became_peered":"2024-09-19T07:43:06.267713+0000","last_unstale":"2024-09-19T07:43:11.311136+0000","last_undegraded":"2024-09-19T07:43:11.311136+0000","last_fullsized":"2024-09-19T07:43:11.311136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:56.380341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681209+0000","last_change":"2024-09-19T07:43:05.263248+0000","last_active":"2024-09-19T07:43:12.681209+0000","last_peered":"2024-09-19T07:43:12.681209+0000","last_clean":"2024-09-19T07:43:12.681209+0000","last_became_active":"2024-09-19T07:43:05.262892+0000","last_became_peered":"2024-09-19T07:43:05.262892+0000","last_unstale":"2024-09-19T07:43:12.681209+0000","last_undegraded":"2024-09-19T07:43:12.681209+0000","last_fullsized":"2024-09-19T07:43:12.681209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:47.531702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.309826+0000","last_change":"2024-09-19T07:43:10.482975+0000","last_active":"2024-09-19T07:43:11.309826+0000","last_peered":"2024-09-19T07:43:11.309826+0000","last_clean":"2024-09-19T07:43:11.309826+0000","last_became_active":"2024-09-19T07:43:10.482584+0000","last_became_peered":"2024-09-19T07:43:10.482584+0000","last_unstale":"2024-09-19T07:43:11.309826+0000","last_undegraded":"2024-09-19T07:43:11.309826+0000","last_fullsized":"2024-09-19T07:43:11.309826+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:10.437589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680841+0000","last_change":"2024-09-19T07:43:06.258826+0000","last_active":"2024-09-19T07:43:12.680841+0000","last_peered":"2024-09-19T07:43:12.680841+0000","last_clean":"2024-09-19T07:43:12.680841+0000","last_became_active":"2024-09-19T07:43:06.258518+0000","last_became_peered":"2024-09-19T07:43:06.258518+0000","last_unstale":"2024-09-19T07:43:12.680841+0000","last_undegraded":"2024-09-19T07:43:12.680841+0000","last_fullsized":"2024-09-19T07:43:12.680841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681832+0000","last_change":"2024-09-19T07:43:05.267167+0000","last_active":"2024-09-19T07:43:12.681832+0000","last_peered":"2024-09-19T07:43:12.681832+0000","last_clean":"2024-09-19T07:43:12.681832+0000","last_became_active":"2024-09-19T07:43:05.266892+0000","last_became_peered":"2024-09-19T07:43:05.266892+0000","last_unstale":"2024-09-19T07:43:12.681832+0000","last_undegraded":"2024-09-19T07:43:12.681832+0000","last_fullsized":"2024-09-19T07:43:12.681832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:06.889703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655769+0000","last_change":"2024-09-19T07:43:10.480657+0000","last_active":"2024-09-19T07:43:12.655769+0000","last_peered":"2024-09-19T07:43:12.655769+0000","last_clean":"2024-09-19T07:43:12.655769+0000","last_became_active":"2024-09-19T07:43:10.480186+0000","last_became_peered":"2024-09-19T07:43:10.480186+0000","last_unstale":"2024-09-19T07:43:12.655769+0000","last_undegraded":"2024-09-19T07:43:12.655769+0000","last_fullsized":"2024-09-19T07:43:12.655769+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:43.490603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311921+0000","last_change":"2024-09-19T07:43:06.265266+0000","last_active":"2024-09-19T07:43:11.311921+0000","last_peered":"2024-09-19T07:43:11.311921+0000","last_clean":"2024-09-19T07:43:11.311921+0000","last_became_active":"2024-09-19T07:43:06.265001+0000","last_became_peered":"2024-09-19T07:43:06.265001+0000","last_unstale":"2024-09-19T07:43:11.311921+0000","last_undegraded":"2024-09-19T07:43:11.311921+0000","last_fullsized":"2024-09-19T07:43:11.311921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:39.464868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680965+0000","last_change":"2024-09-19T07:43:05.259114+0000","last_active":"2024-09-19T07:43:12.680965+0000","last_peered":"2024-09-19T07:43:12.680965+0000","last_clean":"2024-09-19T07:43:12.680965+0000","last_became_active":"2024-09-19T07:43:05.258794+0000","last_became_peered":"2024-09-19T07:43:05.258794+0000","last_unstale":"2024-09-19T07:43:12.680965+0000","last_undegraded":"2024-09-19T07:43:12.680965+0000","last_fullsized":"2024-09-19T07:43:12.680965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:13.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680778+0000","last_change":"2024-09-19T07:43:10.295795+0000","last_active":"2024-09-19T07:43:12.680778+0000","last_peered":"2024-09-19T07:43:12.680778+0000","last_clean":"2024-09-19T07:43:12.680778+0000","last_became_active":"2024-09-19T07:43:10.295379+0000","last_became_peered":"2024-09-19T07:43:10.295379+0000","last_unstale":"2024-09-19T07:43:12.680778+0000","last_undegraded":"2024-09-19T07:43:12.680778+0000","last_fullsized":"2024-09-19T07:43:12.680778+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:43.490603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311661+0000","last_change":"2024-09-19T07:43:06.271554+0000","last_active":"2024-09-19T07:43:11.311661+0000","last_peered":"2024-09-19T07:43:11.311661+0000","last_clean":"2024-09-19T07:43:11.311661+0000","last_became_active":"2024-09-19T07:43:06.271304+0000","last_became_peered":"2024-09-19T07:43:06.271304+0000","last_unstale":"2024-09-19T07:43:11.311661+0000","last_undegraded":"2024-09-19T07:43:11.311661+0000","last_fullsized":"2024-09-19T07:43:11.311661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:03.494318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311677+0000","last_change":"2024-09-19T07:43:05.264548+0000","last_active":"2024-09-19T07:43:11.311677+0000","last_peered":"2024-09-19T07:43:11.311677+0000","last_clean":"2024-09-19T07:43:11.311677+0000","last_became_active":"2024-09-19T07:43:05.261674+0000","last_became_peered":"2024-09-19T07:43:05.261674+0000","last_unstale":"2024-09-19T07:43:11.311677+0000","last_undegraded":"2024-09-19T07:43:11.311677+0000","last_fullsized":"2024-09-19T07:43:11.311677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:47.531702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681063+0000","last_change":"2024-09-19T07:43:10.288751+0000","last_active":"2024-09-19T07:43:12.681063+0000","last_peered":"2024-09-19T07:43:12.681063+0000","last_clean":"2024-09-19T07:43:12.681063+0000","last_became_active":"2024-09-19T07:43:10.288383+0000","last_became_peered":"2024-09-19T07:43:10.288383+0000","last_unstale":"2024-09-19T07:43:12.681063+0000","last_undegraded":"2024-09-19T07:43:12.681063+0000","last_fullsized":"2024-09-19T07:43:12.681063+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:43.944005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680858+0000","last_change":"2024-09-19T07:43:06.264468+0000","last_active":"2024-09-19T07:43:12.680858+0000","last_peered":"2024-09-19T07:43:12.680858+0000","last_clean":"2024-09-19T07:43:12.680858+0000","last_became_active":"2024-09-19T07:43:06.263801+0000","last_became_peered":"2024-09-19T07:43:06.263801+0000","last_unstale":"2024-09-19T07:43:12.680858+0000","last_undegraded":"2024-09-19T07:43:12.680858+0000","last_fullsized":"2024-09-19T07:43:12.680858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:06.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588628+0000","last_change":"2024-09-19T07:43:05.259922+0000","last_active":"2024-09-19T07:43:11.588628+0000","last_peered":"2024-09-19T07:43:11.588628+0000","last_clean":"2024-09-19T07:43:11.588628+0000","last_became_active":"2024-09-19T07:43:05.259552+0000","last_became_peered":"2024-09-19T07:43:05.259552+0000","last_unstale":"2024-09-19T07:43:11.588628+0000","last_undegraded":"2024-09-19T07:43:11.588628+0000","last_fullsized":"2024-09-19T07:43:11.588628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:55.737753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681498+0000","last_change":"2024-09-19T07:43:10.296047+0000","last_active":"2024-09-19T07:43:12.681498+0000","last_peered":"2024-09-19T07:43:12.681498+0000","last_clean":"2024-09-19T07:43:12.681498+0000","last_became_active":"2024-09-19T07:43:10.295636+0000","last_became_peered":"2024-09-19T07:43:10.295636+0000","last_unstale":"2024-09-19T07:43:12.681498+0000","last_undegraded":"2024-09-19T07:43:12.681498+0000","last_fullsized":"2024-09-19T07:43:12.681498+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:27.138783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655596+0000","last_change":"2024-09-19T07:43:06.273785+0000","last_active":"2024-09-19T07:43:12.655596+0000","last_peered":"2024-09-19T07:43:12.655596+0000","last_clean":"2024-09-19T07:43:12.655596+0000","last_became_active":"2024-09-19T07:43:06.273514+0000","last_became_peered":"2024-09-19T07:43:06.273514+0000","last_unstale":"2024-09-19T07:43:12.655596+0000","last_undegraded":"2024-09-19T07:43:12.655596+0000","last_fullsized":"2024-09-19T07:43:12.655596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:09.399092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681059+0000","last_change":"2024-09-19T07:43:05.263952+0000","last_active":"2024-09-19T07:43:12.681059+0000","last_peered":"2024-09-19T07:43:12.681059+0000","last_clean":"2024-09-19T07:43:12.681059+0000","last_became_active":"2024-09-19T07:43:05.263677+0000","last_became_peered":"2024-09-19T07:43:05.263677+0000","last_unstale":"2024-09-19T07:43:12.681059+0000","last_undegraded":"2024-09-19T07:43:12.681059+0000","last_fullsized":"2024-09-19T07:43:12.681059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:44.359336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656084+0000","last_change":"2024-09-19T07:43:10.480721+0000","last_active":"2024-09-19T07:43:12.656084+0000","last_peered":"2024-09-19T07:43:12.656084+0000","last_clean":"2024-09-19T07:43:12.656084+0000","last_became_active":"2024-09-19T07:43:10.480321+0000","last_became_peered":"2024-09-19T07:43:10.480321+0000","last_unstale":"2024-09-19T07:43:12.656084+0000","last_undegraded":"2024-09-19T07:43:12.656084+0000","last_fullsized":"2024-09-19T07:43:12.656084+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:01.954262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310005+0000","last_change":"2024-09-19T07:43:06.271482+0000","last_active":"2024-09-19T07:43:11.310005+0000","last_peered":"2024-09-19T07:43:11.310005+0000","last_clean":"2024-09-19T07:43:11.310005+0000","last_became_active":"2024-09-19T07:43:06.271212+0000","last_became_peered":"2024-09-19T07:43:06.271212+0000","last_unstale":"2024-09-19T07:43:11.310005+0000","last_undegraded":"2024-09-19T07:43:11.310005+0000","last_fullsized":"2024-09-19T07:43:11.310005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:27.185190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311209+0000","last_change":"2024-09-19T07:43:05.261378+0000","last_active":"2024-09-19T07:43:11.311209+0000","last_peered":"2024-09-19T07:43:11.311209+0000","last_clean":"2024-09-19T07:43:11.311209+0000","last_became_active":"2024-09-19T07:43:05.261055+0000","last_became_peered":"2024-09-19T07:43:05.261055+0000","last_unstale":"2024-09-19T07:43:11.311209+0000","last_undegraded":"2024-09-19T07:43:11.311209+0000","last_fullsized":"2024-09-19T07:43:11.311209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:12.998698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681162+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:43:12.681162+0000","last_peered":"2024-09-19T07:43:12.681162+0000","last_clean":"2024-09-19T07:43:12.681162+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:43:12.681162+0000","last_undegraded":"2024-09-19T07:43:12.681162+0000","last_fullsized":"2024-09-19T07:43:12.681162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450864+0000","last_change":"2024-09-19T07:43:10.481567+0000","last_active":"2024-09-19T07:43:11.450864+0000","last_peered":"2024-09-19T07:43:11.450864+0000","last_clean":"2024-09-19T07:43:11.450864+0000","last_became_active":"2024-09-19T07:43:10.481210+0000","last_became_peered":"2024-09-19T07:43:10.481210+0000","last_unstale":"2024-09-19T07:43:11.450864+0000","last_undegraded":"2024-09-19T07:43:11.450864+0000","last_fullsized":"2024-09-19T07:43:11.450864+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:08.193959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681552+0000","last_change":"2024-09-19T07:43:06.268655+0000","last_active":"2024-09-19T07:43:12.681552+0000","last_peered":"2024-09-19T07:43:12.681552+0000","last_clean":"2024-09-19T07:43:12.681552+0000","last_became_active":"2024-09-19T07:43:06.268387+0000","last_became_peered":"2024-09-19T07:43:06.268387+0000","last_unstale":"2024-09-19T07:43:12.681552+0000","last_undegraded":"2024-09-19T07:43:12.681552+0000","last_fullsized":"2024-09-19T07:43:12.681552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:14.347245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588993+0000","last_change":"2024-09-19T07:43:05.261374+0000","last_active":"2024-09-19T07:43:11.588993+0000","last_peered":"2024-09-19T07:43:11.588993+0000","last_clean":"2024-09-19T07:43:11.588993+0000","last_became_active":"2024-09-19T07:43:05.261088+0000","last_became_peered":"2024-09-19T07:43:05.261088+0000","last_unstale":"2024-09-19T07:43:11.588993+0000","last_undegraded":"2024-09-19T07:43:11.588993+0000","last_fullsized":"2024-09-19T07:43:11.588993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:39.630700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450612+0000","last_change":"2024-09-19T07:43:10.480274+0000","last_active":"2024-09-19T07:43:11.450612+0000","last_peered":"2024-09-19T07:43:11.450612+0000","last_clean":"2024-09-19T07:43:11.450612+0000","last_became_active":"2024-09-19T07:43:10.479848+0000","last_became_peered":"2024-09-19T07:43:10.479848+0000","last_unstale":"2024-09-19T07:43:11.450612+0000","last_undegraded":"2024-09-19T07:43:11.450612+0000","last_fullsized":"2024-09-19T07:43:11.450612+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:43.684480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312126+0000","last_change":"2024-09-19T07:43:06.264529+0000","last_active":"2024-09-19T07:43:11.312126+0000","last_peered":"2024-09-19T07:43:11.312126+0000","last_clean":"2024-09-19T07:43:11.312126+0000","last_became_active":"2024-09-19T07:43:06.264270+0000","last_became_peered":"2024-09-19T07:43:06.264270+0000","last_unstale":"2024-09-19T07:43:11.312126+0000","last_undegraded":"2024-09-19T07:43:11.312126+0000","last_fullsized":"2024-09-19T07:43:11.312126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:04.168224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681259+0000","last_change":"2024-09-19T07:43:05.254996+0000","last_active":"2024-09-19T07:43:12.681259+0000","last_peered":"2024-09-19T07:43:12.681259+0000","last_clean":"2024-09-19T07:43:12.681259+0000","last_became_active":"2024-09-19T07:43:05.254430+0000","last_became_peered":"2024-09-19T07:43:05.254430+0000","last_unstale":"2024-09-19T07:43:12.681259+0000","last_undegraded":"2024-09-19T07:43:12.681259+0000","last_fullsized":"2024-09-19T07:43:12.681259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:06.889703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656122+0000","last_change":"2024-09-19T07:43:10.295667+0000","last_active":"2024-09-19T07:43:12.656122+0000","last_peered":"2024-09-19T07:43:12.656122+0000","last_clean":"2024-09-19T07:43:12.656122+0000","last_became_active":"2024-09-19T07:43:10.295241+0000","last_became_peered":"2024-09-19T07:43:10.295241+0000","last_unstale":"2024-09-19T07:43:12.656122+0000","last_undegraded":"2024-09-19T07:43:12.656122+0000","last_fullsized":"2024-09-19T07:43:12.656122+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:27.138783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681804+0000","last_change":"2024-09-19T07:43:06.275720+0000","last_active":"2024-09-19T07:43:12.681804+0000","last_peered":"2024-09-19T07:43:12.681804+0000","last_clean":"2024-09-19T07:43:12.681804+0000","last_became_active":"2024-09-19T07:43:06.275403+0000","last_became_peered":"2024-09-19T07:43:06.275403+0000","last_unstale":"2024-09-19T07:43:12.681804+0000","last_undegraded":"2024-09-19T07:43:12.681804+0000","last_fullsized":"2024-09-19T07:43:12.681804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:39.918269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588845+0000","last_change":"2024-09-19T07:43:05.260217+0000","last_active":"2024-09-19T07:43:11.588845+0000","last_peered":"2024-09-19T07:43:11.588845+0000","last_clean":"2024-09-19T07:43:11.588845+0000","last_became_active":"2024-09-19T07:43:05.259482+0000","last_became_peered":"2024-09-19T07:43:05.259482+0000","last_unstale":"2024-09-19T07:43:11.588845+0000","last_undegraded":"2024-09-19T07:43:11.588845+0000","last_fullsized":"2024-09-19T07:43:11.588845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:56.644264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310513+0000","last_change":"2024-09-19T07:43:10.480657+0000","last_active":"2024-09-19T07:43:11.310513+0000","last_peered":"2024-09-19T07:43:11.310513+0000","last_clean":"2024-09-19T07:43:11.310513+0000","last_became_active":"2024-09-19T07:43:10.480279+0000","last_became_peered":"2024-09-19T07:43:10.480279+0000","last_unstale":"2024-09-19T07:43:11.310513+0000","last_undegraded":"2024-09-19T07:43:11.310513+0000","last_fullsized":"2024-09-19T07:43:11.310513+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:26.933433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681655+0000","last_change":"2024-09-19T07:43:06.275354+0000","last_active":"2024-09-19T07:43:12.681655+0000","last_peered":"2024-09-19T07:43:12.681655+0000","last_clean":"2024-09-19T07:43:12.681655+0000","last_became_active":"2024-09-19T07:43:06.275039+0000","last_became_peered":"2024-09-19T07:43:06.275039+0000","last_unstale":"2024-09-19T07:43:12.681655+0000","last_undegraded":"2024-09-19T07:43:12.681655+0000","last_fullsized":"2024-09-19T07:43:12.681655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:47.918580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311619+0000","last_change":"2024-09-19T07:43:05.265977+0000","last_active":"2024-09-19T07:43:11.311619+0000","last_peered":"2024-09-19T07:43:11.311619+0000","last_clean":"2024-09-19T07:43:11.311619+0000","last_became_active":"2024-09-19T07:43:05.265615+0000","last_became_peered":"2024-09-19T07:43:05.265615+0000","last_unstale":"2024-09-19T07:43:11.311619+0000","last_undegraded":"2024-09-19T07:43:11.311619+0000","last_fullsized":"2024-09-19T07:43:11.311619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:34.475024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310868+0000","last_change":"2024-09-19T07:43:10.283896+0000","last_active":"2024-09-19T07:43:11.310868+0000","last_peered":"2024-09-19T07:43:11.310868+0000","last_clean":"2024-09-19T07:43:11.310868+0000","last_became_active":"2024-09-19T07:43:10.283587+0000","last_became_peered":"2024-09-19T07:43:10.283587+0000","last_unstale":"2024-09-19T07:43:11.310868+0000","last_undegraded":"2024-09-19T07:43:11.310868+0000","last_fullsized":"2024-09-19T07:43:11.310868+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:41.387226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681270+0000","last_change":"2024-09-19T07:43:06.274315+0000","last_active":"2024-09-19T07:43:12.681270+0000","last_peered":"2024-09-19T07:43:12.681270+0000","last_clean":"2024-09-19T07:43:12.681270+0000","last_became_active":"2024-09-19T07:43:06.274064+0000","last_became_peered":"2024-09-19T07:43:06.274064+0000","last_unstale":"2024-09-19T07:43:12.681270+0000","last_undegraded":"2024-09-19T07:43:12.681270+0000","last_fullsized":"2024-09-19T07:43:12.681270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:22.804370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588519+0000","last_change":"2024-09-19T07:43:05.259083+0000","last_active":"2024-09-19T07:43:11.588519+0000","last_peered":"2024-09-19T07:43:11.588519+0000","last_clean":"2024-09-19T07:43:11.588519+0000","last_became_active":"2024-09-19T07:43:05.258769+0000","last_became_peered":"2024-09-19T07:43:05.258769+0000","last_unstale":"2024-09-19T07:43:11.588519+0000","last_undegraded":"2024-09-19T07:43:11.588519+0000","last_fullsized":"2024-09-19T07:43:11.588519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:51:24.580949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656048+0000","last_change":"2024-09-19T07:43:10.290261+0000","last_active":"2024-09-19T07:43:12.656048+0000","last_peered":"2024-09-19T07:43:12.656048+0000","last_clean":"2024-09-19T07:43:12.656048+0000","last_became_active":"2024-09-19T07:43:10.289428+0000","last_became_peered":"2024-09-19T07:43:10.289428+0000","last_unstale":"2024-09-19T07:43:12.656048+0000","last_undegraded":"2024-09-19T07:43:12.656048+0000","last_fullsized":"2024-09-19T07:43:12.656048+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:43.944005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450559+0000","last_change":"2024-09-19T07:43:06.276364+0000","last_active":"2024-09-19T07:43:11.450559+0000","last_peered":"2024-09-19T07:43:11.450559+0000","last_clean":"2024-09-19T07:43:11.450559+0000","last_became_active":"2024-09-19T07:43:06.275564+0000","last_became_peered":"2024-09-19T07:43:06.275564+0000","last_unstale":"2024-09-19T07:43:11.450559+0000","last_undegraded":"2024-09-19T07:43:11.450559+0000","last_fullsized":"2024-09-19T07:43:11.450559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450611+0000","last_change":"2024-09-19T07:43:05.264487+0000","last_active":"2024-09-19T07:43:11.450611+0000","last_peered":"2024-09-19T07:43:11.450611+0000","last_clean":"2024-09-19T07:43:11.450611+0000","last_became_active":"2024-09-19T07:43:05.264196+0000","last_became_peered":"2024-09-19T07:43:05.264196+0000","last_unstale":"2024-09-19T07:43:11.450611+0000","last_undegraded":"2024-09-19T07:43:11.450611+0000","last_fullsized":"2024-09-19T07:43:11.450611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:47.531702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311055+0000","last_change":"2024-09-19T07:43:10.481930+0000","last_active":"2024-09-19T07:43:11.311055+0000","last_peered":"2024-09-19T07:43:11.311055+0000","last_clean":"2024-09-19T07:43:11.311055+0000","last_became_active":"2024-09-19T07:43:10.481552+0000","last_became_peered":"2024-09-19T07:43:10.481552+0000","last_unstale":"2024-09-19T07:43:11.311055+0000","last_undegraded":"2024-09-19T07:43:11.311055+0000","last_fullsized":"2024-09-19T07:43:11.311055+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:19.523345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681307+0000","last_change":"2024-09-19T07:43:06.275131+0000","last_active":"2024-09-19T07:43:12.681307+0000","last_peered":"2024-09-19T07:43:12.681307+0000","last_clean":"2024-09-19T07:43:12.681307+0000","last_became_active":"2024-09-19T07:43:06.274857+0000","last_became_peered":"2024-09-19T07:43:06.274857+0000","last_unstale":"2024-09-19T07:43:12.681307+0000","last_undegraded":"2024-09-19T07:43:12.681307+0000","last_fullsized":"2024-09-19T07:43:12.681307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:57.928527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311569+0000","last_change":"2024-09-19T07:43:05.263838+0000","last_active":"2024-09-19T07:43:11.311569+0000","last_peered":"2024-09-19T07:43:11.311569+0000","last_clean":"2024-09-19T07:43:11.311569+0000","last_became_active":"2024-09-19T07:43:05.263484+0000","last_became_peered":"2024-09-19T07:43:05.263484+0000","last_unstale":"2024-09-19T07:43:11.311569+0000","last_undegraded":"2024-09-19T07:43:11.311569+0000","last_fullsized":"2024-09-19T07:43:11.311569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:38.098052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310652+0000","last_change":"2024-09-19T07:43:10.480636+0000","last_active":"2024-09-19T07:43:11.310652+0000","last_peered":"2024-09-19T07:43:11.310652+0000","last_clean":"2024-09-19T07:43:11.310652+0000","last_became_active":"2024-09-19T07:43:10.480217+0000","last_became_peered":"2024-09-19T07:43:10.480217+0000","last_unstale":"2024-09-19T07:43:11.310652+0000","last_undegraded":"2024-09-19T07:43:11.310652+0000","last_fullsized":"2024-09-19T07:43:11.310652+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:29.825328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656111+0000","last_change":"2024-09-19T07:43:06.272020+0000","last_active":"2024-09-19T07:43:12.656111+0000","last_peered":"2024-09-19T07:43:12.656111+0000","last_clean":"2024-09-19T07:43:12.656111+0000","last_became_active":"2024-09-19T07:43:06.271671+0000","last_became_peered":"2024-09-19T07:43:06.271671+0000","last_unstale":"2024-09-19T07:43:12.656111+0000","last_undegraded":"2024-09-19T07:43:12.656111+0000","last_fullsized":"2024-09-19T07:43:12.656111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:24:44.612992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656246+0000","last_change":"2024-09-19T07:43:05.265898+0000","last_active":"2024-09-19T07:43:12.656246+0000","last_peered":"2024-09-19T07:43:12.656246+0000","last_clean":"2024-09-19T07:43:12.656246+0000","last_became_active":"2024-09-19T07:43:05.265595+0000","last_became_peered":"2024-09-19T07:43:05.265595+0000","last_unstale":"2024-09-19T07:43:12.656246+0000","last_undegraded":"2024-09-19T07:43:12.656246+0000","last_fullsized":"2024-09-19T07:43:12.656246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:22.696534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311727+0000","last_change":"2024-09-19T07:43:10.482166+0000","last_active":"2024-09-19T07:43:11.311727+0000","last_peered":"2024-09-19T07:43:11.311727+0000","last_clean":"2024-09-19T07:43:11.311727+0000","last_became_active":"2024-09-19T07:43:10.481667+0000","last_became_peered":"2024-09-19T07:43:10.481667+0000","last_unstale":"2024-09-19T07:43:11.311727+0000","last_undegraded":"2024-09-19T07:43:11.311727+0000","last_fullsized":"2024-09-19T07:43:11.311727+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:10.437589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681470+0000","last_change":"2024-09-19T07:43:06.277189+0000","last_active":"2024-09-19T07:43:12.681470+0000","last_peered":"2024-09-19T07:43:12.681470+0000","last_clean":"2024-09-19T07:43:12.681470+0000","last_became_active":"2024-09-19T07:43:06.276924+0000","last_became_peered":"2024-09-19T07:43:06.276924+0000","last_unstale":"2024-09-19T07:43:12.681470+0000","last_undegraded":"2024-09-19T07:43:12.681470+0000","last_fullsized":"2024-09-19T07:43:12.681470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:39.658744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:15.766406+0000","last_change":"2024-09-19T07:43:05.266360+0000","last_active":"2024-09-19T07:43:15.766406+0000","last_peered":"2024-09-19T07:43:15.766406+0000","last_clean":"2024-09-19T07:43:15.766406+0000","last_became_active":"2024-09-19T07:43:05.266038+0000","last_became_peered":"2024-09-19T07:43:05.266038+0000","last_unstale":"2024-09-19T07:43:15.766406+0000","last_undegraded":"2024-09-19T07:43:15.766406+0000","last_fullsized":"2024-09-19T07:43:15.766406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:13.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680836+0000","last_change":"2024-09-19T07:43:10.295894+0000","last_active":"2024-09-19T07:43:12.680836+0000","last_peered":"2024-09-19T07:43:12.680836+0000","last_clean":"2024-09-19T07:43:12.680836+0000","last_became_active":"2024-09-19T07:43:10.295496+0000","last_became_peered":"2024-09-19T07:43:10.295496+0000","last_unstale":"2024-09-19T07:43:12.680836+0000","last_undegraded":"2024-09-19T07:43:12.680836+0000","last_fullsized":"2024-09-19T07:43:12.680836+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:18.372981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312405+0000","last_change":"2024-09-19T07:43:06.277639+0000","last_active":"2024-09-19T07:43:11.312405+0000","last_peered":"2024-09-19T07:43:11.312405+0000","last_clean":"2024-09-19T07:43:11.312405+0000","last_became_active":"2024-09-19T07:43:06.277186+0000","last_became_peered":"2024-09-19T07:43:06.277186+0000","last_unstale":"2024-09-19T07:43:11.312405+0000","last_undegraded":"2024-09-19T07:43:11.312405+0000","last_fullsized":"2024-09-19T07:43:11.312405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:07.895243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681210+0000","last_change":"2024-09-19T07:43:05.257416+0000","last_active":"2024-09-19T07:43:12.681210+0000","last_peered":"2024-09-19T07:43:12.681210+0000","last_clean":"2024-09-19T07:43:12.681210+0000","last_became_active":"2024-09-19T07:43:05.256941+0000","last_became_peered":"2024-09-19T07:43:05.256941+0000","last_unstale":"2024-09-19T07:43:12.681210+0000","last_undegraded":"2024-09-19T07:43:12.681210+0000","last_fullsized":"2024-09-19T07:43:12.681210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:44.359336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311217+0000","last_change":"2024-09-19T07:43:10.288931+0000","last_active":"2024-09-19T07:43:11.311217+0000","last_peered":"2024-09-19T07:43:11.311217+0000","last_clean":"2024-09-19T07:43:11.311217+0000","last_became_active":"2024-09-19T07:43:10.288548+0000","last_became_peered":"2024-09-19T07:43:10.288548+0000","last_unstale":"2024-09-19T07:43:11.311217+0000","last_undegraded":"2024-09-19T07:43:11.311217+0000","last_fullsized":"2024-09-19T07:43:11.311217+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:00.406076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656008+0000","last_change":"2024-09-19T07:43:06.270017+0000","last_active":"2024-09-19T07:43:12.656008+0000","last_peered":"2024-09-19T07:43:12.656008+0000","last_clean":"2024-09-19T07:43:12.656008+0000","last_became_active":"2024-09-19T07:43:06.269698+0000","last_became_peered":"2024-09-19T07:43:06.269698+0000","last_unstale":"2024-09-19T07:43:12.656008+0000","last_undegraded":"2024-09-19T07:43:12.656008+0000","last_fullsized":"2024-09-19T07:43:12.656008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:27.185190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311784+0000","last_change":"2024-09-19T07:43:05.265424+0000","last_active":"2024-09-19T07:43:11.311784+0000","last_peered":"2024-09-19T07:43:11.311784+0000","last_clean":"2024-09-19T07:43:11.311784+0000","last_became_active":"2024-09-19T07:43:05.265165+0000","last_became_peered":"2024-09-19T07:43:05.265165+0000","last_unstale":"2024-09-19T07:43:11.311784+0000","last_undegraded":"2024-09-19T07:43:11.311784+0000","last_fullsized":"2024-09-19T07:43:11.311784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:38:29.212025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310075+0000","last_change":"2024-09-19T07:43:10.290285+0000","last_active":"2024-09-19T07:43:11.310075+0000","last_peered":"2024-09-19T07:43:11.310075+0000","last_clean":"2024-09-19T07:43:11.310075+0000","last_became_active":"2024-09-19T07:43:10.289718+0000","last_became_peered":"2024-09-19T07:43:10.289718+0000","last_unstale":"2024-09-19T07:43:11.310075+0000","last_undegraded":"2024-09-19T07:43:11.310075+0000","last_fullsized":"2024-09-19T07:43:11.310075+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:08.693513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311662+0000","last_change":"2024-09-19T07:43:06.265419+0000","last_active":"2024-09-19T07:43:11.311662+0000","last_peered":"2024-09-19T07:43:11.311662+0000","last_clean":"2024-09-19T07:43:11.311662+0000","last_became_active":"2024-09-19T07:43:06.265170+0000","last_became_peered":"2024-09-19T07:43:06.265170+0000","last_unstale":"2024-09-19T07:43:11.311662+0000","last_undegraded":"2024-09-19T07:43:11.311662+0000","last_fullsized":"2024-09-19T07:43:11.311662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451157+0000","last_change":"2024-09-19T07:43:05.264095+0000","last_active":"2024-09-19T07:43:11.451157+0000","last_peered":"2024-09-19T07:43:11.451157+0000","last_clean":"2024-09-19T07:43:11.451157+0000","last_became_active":"2024-09-19T07:43:05.262895+0000","last_became_peered":"2024-09-19T07:43:05.262895+0000","last_unstale":"2024-09-19T07:43:11.451157+0000","last_undegraded":"2024-09-19T07:43:11.451157+0000","last_fullsized":"2024-09-19T07:43:11.451157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:38.098052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310066+0000","last_change":"2024-09-19T07:43:10.482892+0000","last_active":"2024-09-19T07:43:11.310066+0000","last_peered":"2024-09-19T07:43:11.310066+0000","last_clean":"2024-09-19T07:43:11.310066+0000","last_became_active":"2024-09-19T07:43:10.482523+0000","last_became_peered":"2024-09-19T07:43:10.482523+0000","last_unstale":"2024-09-19T07:43:11.310066+0000","last_undegraded":"2024-09-19T07:43:11.310066+0000","last_fullsized":"2024-09-19T07:43:11.310066+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:07.059894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655916+0000","last_change":"2024-09-19T07:43:10.480793+0000","last_active":"2024-09-19T07:43:12.655916+0000","last_peered":"2024-09-19T07:43:12.655916+0000","last_clean":"2024-09-19T07:43:12.655916+0000","last_became_active":"2024-09-19T07:43:10.480423+0000","last_became_peered":"2024-09-19T07:43:10.480423+0000","last_unstale":"2024-09-19T07:43:12.655916+0000","last_undegraded":"2024-09-19T07:43:12.655916+0000","last_fullsized":"2024-09-19T07:43:12.655916+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:18.372981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680526+0000","last_change":"2024-09-19T07:43:06.258001+0000","last_active":"2024-09-19T07:43:12.680526+0000","last_peered":"2024-09-19T07:43:12.680526+0000","last_clean":"2024-09-19T07:43:12.680526+0000","last_became_active":"2024-09-19T07:43:06.257731+0000","last_became_peered":"2024-09-19T07:43:06.257731+0000","last_unstale":"2024-09-19T07:43:12.680526+0000","last_undegraded":"2024-09-19T07:43:12.680526+0000","last_fullsized":"2024-09-19T07:43:12.680526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:14.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310662+0000","last_change":"2024-09-19T07:43:05.260911+0000","last_active":"2024-09-19T07:43:11.310662+0000","last_peered":"2024-09-19T07:43:11.310662+0000","last_clean":"2024-09-19T07:43:11.310662+0000","last_became_active":"2024-09-19T07:43:05.258827+0000","last_became_peered":"2024-09-19T07:43:05.258827+0000","last_unstale":"2024-09-19T07:43:11.310662+0000","last_undegraded":"2024-09-19T07:43:11.310662+0000","last_fullsized":"2024-09-19T07:43:11.310662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:13.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451139+0000","last_change":"2024-09-19T07:43:10.293630+0000","last_active":"2024-09-19T07:43:11.451139+0000","last_peered":"2024-09-19T07:43:11.451139+0000","last_clean":"2024-09-19T07:43:11.451139+0000","last_became_active":"2024-09-19T07:43:10.292979+0000","last_became_peered":"2024-09-19T07:43:10.292979+0000","last_unstale":"2024-09-19T07:43:11.451139+0000","last_undegraded":"2024-09-19T07:43:11.451139+0000","last_fullsized":"2024-09-19T07:43:11.451139+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:43.490603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310026+0000","last_change":"2024-09-19T07:43:06.274060+0000","last_active":"2024-09-19T07:43:11.310026+0000","last_peered":"2024-09-19T07:43:11.310026+0000","last_clean":"2024-09-19T07:43:11.310026+0000","last_became_active":"2024-09-19T07:43:06.272704+0000","last_became_peered":"2024-09-19T07:43:06.272704+0000","last_unstale":"2024-09-19T07:43:11.310026+0000","last_undegraded":"2024-09-19T07:43:11.310026+0000","last_fullsized":"2024-09-19T07:43:11.310026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:06.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310182+0000","last_change":"2024-09-19T07:43:05.264989+0000","last_active":"2024-09-19T07:43:11.310182+0000","last_peered":"2024-09-19T07:43:11.310182+0000","last_clean":"2024-09-19T07:43:11.310182+0000","last_became_active":"2024-09-19T07:43:05.264673+0000","last_became_peered":"2024-09-19T07:43:05.264673+0000","last_unstale":"2024-09-19T07:43:11.310182+0000","last_undegraded":"2024-09-19T07:43:11.310182+0000","last_fullsized":"2024-09-19T07:43:11.310182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:22.696534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450834+0000","last_change":"2024-09-19T07:43:05.262438+0000","last_active":"2024-09-19T07:43:11.450834+0000","last_peered":"2024-09-19T07:43:11.450834+0000","last_clean":"2024-09-19T07:43:11.450834+0000","last_became_active":"2024-09-19T07:43:05.262129+0000","last_became_peered":"2024-09-19T07:43:05.262129+0000","last_unstale":"2024-09-19T07:43:11.450834+0000","last_undegraded":"2024-09-19T07:43:11.450834+0000","last_fullsized":"2024-09-19T07:43:11.450834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:44.359336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656191+0000","last_change":"2024-09-19T07:43:06.270402+0000","last_active":"2024-09-19T07:43:12.656191+0000","last_peered":"2024-09-19T07:43:12.656191+0000","last_clean":"2024-09-19T07:43:12.656191+0000","last_became_active":"2024-09-19T07:43:06.270087+0000","last_became_peered":"2024-09-19T07:43:06.270087+0000","last_unstale":"2024-09-19T07:43:12.656191+0000","last_undegraded":"2024-09-19T07:43:12.656191+0000","last_fullsized":"2024-09-19T07:43:12.656191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588782+0000","last_change":"2024-09-19T07:43:10.288597+0000","last_active":"2024-09-19T07:43:11.588782+0000","last_peered":"2024-09-19T07:43:11.588782+0000","last_clean":"2024-09-19T07:43:11.588782+0000","last_became_active":"2024-09-19T07:43:10.288207+0000","last_became_peered":"2024-09-19T07:43:10.288207+0000","last_unstale":"2024-09-19T07:43:11.588782+0000","last_undegraded":"2024-09-19T07:43:11.588782+0000","last_fullsized":"2024-09-19T07:43:11.588782+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:43.318582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310346+0000","last_change":"2024-09-19T07:43:05.261298+0000","last_active":"2024-09-19T07:43:11.310346+0000","last_peered":"2024-09-19T07:43:11.310346+0000","last_clean":"2024-09-19T07:43:11.310346+0000","last_became_active":"2024-09-19T07:43:05.261000+0000","last_became_peered":"2024-09-19T07:43:05.261000+0000","last_unstale":"2024-09-19T07:43:11.310346+0000","last_undegraded":"2024-09-19T07:43:11.310346+0000","last_fullsized":"2024-09-19T07:43:11.310346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:38.098052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310888+0000","last_change":"2024-09-19T07:43:06.267976+0000","last_active":"2024-09-19T07:43:11.310888+0000","last_peered":"2024-09-19T07:43:11.310888+0000","last_clean":"2024-09-19T07:43:11.310888+0000","last_became_active":"2024-09-19T07:43:06.267214+0000","last_became_peered":"2024-09-19T07:43:06.267214+0000","last_unstale":"2024-09-19T07:43:11.310888+0000","last_undegraded":"2024-09-19T07:43:11.310888+0000","last_fullsized":"2024-09-19T07:43:11.310888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:48.537241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310742+0000","last_change":"2024-09-19T07:43:10.288619+0000","last_active":"2024-09-19T07:43:11.310742+0000","last_peered":"2024-09-19T07:43:11.310742+0000","last_clean":"2024-09-19T07:43:11.310742+0000","last_became_active":"2024-09-19T07:43:10.288213+0000","last_became_peered":"2024-09-19T07:43:10.288213+0000","last_unstale":"2024-09-19T07:43:11.310742+0000","last_undegraded":"2024-09-19T07:43:11.310742+0000","last_fullsized":"2024-09-19T07:43:11.310742+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:30.165579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450437+0000","last_change":"2024-09-19T07:43:06.276792+0000","last_active":"2024-09-19T07:43:11.450437+0000","last_peered":"2024-09-19T07:43:11.450437+0000","last_clean":"2024-09-19T07:43:11.450437+0000","last_became_active":"2024-09-19T07:43:06.276518+0000","last_became_peered":"2024-09-19T07:43:06.276518+0000","last_unstale":"2024-09-19T07:43:11.450437+0000","last_undegraded":"2024-09-19T07:43:11.450437+0000","last_fullsized":"2024-09-19T07:43:11.450437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:12.618593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588673+0000","last_change":"2024-09-19T07:43:06.268099+0000","last_active":"2024-09-19T07:43:11.588673+0000","last_peered":"2024-09-19T07:43:11.588673+0000","last_clean":"2024-09-19T07:43:11.588673+0000","last_became_active":"2024-09-19T07:43:06.267803+0000","last_became_peered":"2024-09-19T07:43:06.267803+0000","last_unstale":"2024-09-19T07:43:11.588673+0000","last_undegraded":"2024-09-19T07:43:11.588673+0000","last_fullsized":"2024-09-19T07:43:11.588673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:31.966840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311465+0000","last_change":"2024-09-19T07:43:06.263757+0000","last_active":"2024-09-19T07:43:11.311465+0000","last_peered":"2024-09-19T07:43:11.311465+0000","last_clean":"2024-09-19T07:43:11.311465+0000","last_became_active":"2024-09-19T07:43:06.263480+0000","last_became_peered":"2024-09-19T07:43:06.263480+0000","last_unstale":"2024-09-19T07:43:11.311465+0000","last_undegraded":"2024-09-19T07:43:11.311465+0000","last_fullsized":"2024-09-19T07:43:11.311465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:39.918269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656177+0000","last_change":"2024-09-19T07:43:06.278694+0000","last_active":"2024-09-19T07:43:12.656177+0000","last_peered":"2024-09-19T07:43:12.656177+0000","last_clean":"2024-09-19T07:43:12.656177+0000","last_became_active":"2024-09-19T07:43:06.277280+0000","last_became_peered":"2024-09-19T07:43:06.277280+0000","last_unstale":"2024-09-19T07:43:12.656177+0000","last_undegraded":"2024-09-19T07:43:12.656177+0000","last_fullsized":"2024-09-19T07:43:12.656177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:51.595301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310764+0000","last_change":"2024-09-19T07:43:06.272967+0000","last_active":"2024-09-19T07:43:11.310764+0000","last_peered":"2024-09-19T07:43:11.310764+0000","last_clean":"2024-09-19T07:43:11.310764+0000","last_became_active":"2024-09-19T07:43:06.272476+0000","last_became_peered":"2024-09-19T07:43:06.272476+0000","last_unstale":"2024-09-19T07:43:11.310764+0000","last_undegraded":"2024-09-19T07:43:11.310764+0000","last_fullsized":"2024-09-19T07:43:11.310764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:09.222719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681113+0000","last_change":"2024-09-19T07:43:06.260655+0000","last_active":"2024-09-19T07:43:12.681113+0000","last_peered":"2024-09-19T07:43:12.681113+0000","last_clean":"2024-09-19T07:43:12.681113+0000","last_became_active":"2024-09-19T07:43:06.260388+0000","last_became_peered":"2024-09-19T07:43:06.260388+0000","last_unstale":"2024-09-19T07:43:12.681113+0000","last_undegraded":"2024-09-19T07:43:12.681113+0000","last_fullsized":"2024-09-19T07:43:12.681113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451084+0000","last_change":"2024-09-19T07:43:06.270719+0000","last_active":"2024-09-19T07:43:11.451084+0000","last_peered":"2024-09-19T07:43:11.451084+0000","last_clean":"2024-09-19T07:43:11.451084+0000","last_became_active":"2024-09-19T07:43:06.270022+0000","last_became_peered":"2024-09-19T07:43:06.270022+0000","last_unstale":"2024-09-19T07:43:11.451084+0000","last_undegraded":"2024-09-19T07:43:11.451084+0000","last_fullsized":"2024-09-19T07:43:11.451084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:27.185190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310886+0000","last_change":"2024-09-19T07:43:06.266798+0000","last_active":"2024-09-19T07:43:11.310886+0000","last_peered":"2024-09-19T07:43:11.310886+0000","last_clean":"2024-09-19T07:43:11.310886+0000","last_became_active":"2024-09-19T07:43:06.266461+0000","last_became_peered":"2024-09-19T07:43:06.266461+0000","last_unstale":"2024-09-19T07:43:11.310886+0000","last_undegraded":"2024-09-19T07:43:11.310886+0000","last_fullsized":"2024-09-19T07:43:11.310886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:50.544269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310636+0000","last_change":"2024-09-19T07:43:06.263830+0000","last_active":"2024-09-19T07:43:11.310636+0000","last_peered":"2024-09-19T07:43:11.310636+0000","last_clean":"2024-09-19T07:43:11.310636+0000","last_became_active":"2024-09-19T07:43:06.263539+0000","last_became_peered":"2024-09-19T07:43:06.263539+0000","last_unstale":"2024-09-19T07:43:11.310636+0000","last_undegraded":"2024-09-19T07:43:11.310636+0000","last_fullsized":"2024-09-19T07:43:11.310636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:50.544269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310166+0000","last_change":"2024-09-19T07:43:06.276371+0000","last_active":"2024-09-19T07:43:11.310166+0000","last_peered":"2024-09-19T07:43:11.310166+0000","last_clean":"2024-09-19T07:43:11.310166+0000","last_became_active":"2024-09-19T07:43:06.276103+0000","last_became_peered":"2024-09-19T07:43:06.276103+0000","last_unstale":"2024-09-19T07:43:11.310166+0000","last_undegraded":"2024-09-19T07:43:11.310166+0000","last_fullsized":"2024-09-19T07:43:11.310166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:51.595301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311371+0000","last_change":"2024-09-19T07:43:06.265375+0000","last_active":"2024-09-19T07:43:11.311371+0000","last_peered":"2024-09-19T07:43:11.311371+0000","last_clean":"2024-09-19T07:43:11.311371+0000","last_became_active":"2024-09-19T07:43:06.265118+0000","last_became_peered":"2024-09-19T07:43:06.265118+0000","last_unstale":"2024-09-19T07:43:11.311371+0000","last_undegraded":"2024-09-19T07:43:11.311371+0000","last_fullsized":"2024-09-19T07:43:11.311371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:14.347245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655877+0000","last_change":"2024-09-19T07:43:06.279273+0000","last_active":"2024-09-19T07:43:12.655877+0000","last_peered":"2024-09-19T07:43:12.655877+0000","last_clean":"2024-09-19T07:43:12.655877+0000","last_became_active":"2024-09-19T07:43:06.278276+0000","last_became_peered":"2024-09-19T07:43:06.278276+0000","last_unstale":"2024-09-19T07:43:12.655877+0000","last_undegraded":"2024-09-19T07:43:12.655877+0000","last_fullsized":"2024-09-19T07:43:12.655877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:14.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450830+0000","last_change":"2024-09-19T07:43:06.268360+0000","last_active":"2024-09-19T07:43:11.450830+0000","last_peered":"2024-09-19T07:43:11.450830+0000","last_clean":"2024-09-19T07:43:11.450830+0000","last_became_active":"2024-09-19T07:43:06.268096+0000","last_became_peered":"2024-09-19T07:43:06.268096+0000","last_unstale":"2024-09-19T07:43:11.450830+0000","last_undegraded":"2024-09-19T07:43:11.450830+0000","last_fullsized":"2024-09-19T07:43:11.450830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681729+0000","last_change":"2024-09-19T07:43:06.272110+0000","last_active":"2024-09-19T07:43:12.681729+0000","last_peered":"2024-09-19T07:43:12.681729+0000","last_clean":"2024-09-19T07:43:12.681729+0000","last_became_active":"2024-09-19T07:43:06.271804+0000","last_became_peered":"2024-09-19T07:43:06.271804+0000","last_unstale":"2024-09-19T07:43:12.681729+0000","last_undegraded":"2024-09-19T07:43:12.681729+0000","last_fullsized":"2024-09-19T07:43:12.681729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:23.113048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588806+0000","last_change":"2024-09-19T07:43:06.260936+0000","last_active":"2024-09-19T07:43:11.588806+0000","last_peered":"2024-09-19T07:43:11.588806+0000","last_clean":"2024-09-19T07:43:11.588806+0000","last_became_active":"2024-09-19T07:43:06.260663+0000","last_became_peered":"2024-09-19T07:43:06.260663+0000","last_unstale":"2024-09-19T07:43:11.588806+0000","last_undegraded":"2024-09-19T07:43:11.588806+0000","last_fullsized":"2024-09-19T07:43:11.588806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:06.425112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312242+0000","last_change":"2024-09-19T07:43:06.268104+0000","last_active":"2024-09-19T07:43:11.312242+0000","last_peered":"2024-09-19T07:43:11.312242+0000","last_clean":"2024-09-19T07:43:11.312242+0000","last_became_active":"2024-09-19T07:43:06.267830+0000","last_became_peered":"2024-09-19T07:43:06.267830+0000","last_unstale":"2024-09-19T07:43:11.312242+0000","last_undegraded":"2024-09-19T07:43:11.312242+0000","last_fullsized":"2024-09-19T07:43:11.312242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:22.804370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310551+0000","last_change":"2024-09-19T07:43:06.269948+0000","last_active":"2024-09-19T07:43:11.310551+0000","last_peered":"2024-09-19T07:43:11.310551+0000","last_clean":"2024-09-19T07:43:11.310551+0000","last_became_active":"2024-09-19T07:43:06.269684+0000","last_became_peered":"2024-09-19T07:43:06.269684+0000","last_unstale":"2024-09-19T07:43:11.310551+0000","last_undegraded":"2024-09-19T07:43:11.310551+0000","last_fullsized":"2024-09-19T07:43:11.310551+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680603+0000","last_change":"2024-09-19T07:43:06.269924+0000","last_active":"2024-09-19T07:43:12.680603+0000","last_peered":"2024-09-19T07:43:12.680603+0000","last_clean":"2024-09-19T07:43:12.680603+0000","last_became_active":"2024-09-19T07:43:06.269602+0000","last_became_peered":"2024-09-19T07:43:06.269602+0000","last_unstale":"2024-09-19T07:43:12.680603+0000","last_undegraded":"2024-09-19T07:43:12.680603+0000","last_fullsized":"2024-09-19T07:43:12.680603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:51.595301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655995+0000","last_change":"2024-09-19T07:43:06.271422+0000","last_active":"2024-09-19T07:43:12.655995+0000","last_peered":"2024-09-19T07:43:12.655995+0000","last_clean":"2024-09-19T07:43:12.655995+0000","last_became_active":"2024-09-19T07:43:06.271096+0000","last_became_peered":"2024-09-19T07:43:06.271096+0000","last_unstale":"2024-09-19T07:43:12.655995+0000","last_undegraded":"2024-09-19T07:43:12.655995+0000","last_fullsized":"2024-09-19T07:43:12.655995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:23.743784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310215+0000","last_change":"2024-09-19T07:43:06.264090+0000","last_active":"2024-09-19T07:43:11.310215+0000","last_peered":"2024-09-19T07:43:11.310215+0000","last_clean":"2024-09-19T07:43:11.310215+0000","last_became_active":"2024-09-19T07:43:06.262071+0000","last_became_peered":"2024-09-19T07:43:06.262071+0000","last_unstale":"2024-09-19T07:43:11.310215+0000","last_undegraded":"2024-09-19T07:43:11.310215+0000","last_fullsized":"2024-09-19T07:43:11.310215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312316+0000","last_change":"2024-09-19T07:43:06.265567+0000","last_active":"2024-09-19T07:43:11.312316+0000","last_peered":"2024-09-19T07:43:11.312316+0000","last_clean":"2024-09-19T07:43:11.312316+0000","last_became_active":"2024-09-19T07:43:06.265290+0000","last_became_peered":"2024-09-19T07:43:06.265290+0000","last_unstale":"2024-09-19T07:43:11.312316+0000","last_undegraded":"2024-09-19T07:43:11.312316+0000","last_fullsized":"2024-09-19T07:43:11.312316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:14.597733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450953+0000","last_change":"2024-09-19T07:43:06.267949+0000","last_active":"2024-09-19T07:43:11.450953+0000","last_peered":"2024-09-19T07:43:11.450953+0000","last_clean":"2024-09-19T07:43:11.450953+0000","last_became_active":"2024-09-19T07:43:06.267640+0000","last_became_peered":"2024-09-19T07:43:06.267640+0000","last_unstale":"2024-09-19T07:43:11.450953+0000","last_undegraded":"2024-09-19T07:43:11.450953+0000","last_fullsized":"2024-09-19T07:43:11.450953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:14.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451030+0000","last_change":"2024-09-19T07:43:06.268718+0000","last_active":"2024-09-19T07:43:11.451030+0000","last_peered":"2024-09-19T07:43:11.451030+0000","last_clean":"2024-09-19T07:43:11.451030+0000","last_became_active":"2024-09-19T07:43:06.268398+0000","last_became_peered":"2024-09-19T07:43:06.268398+0000","last_unstale":"2024-09-19T07:43:11.451030+0000","last_undegraded":"2024-09-19T07:43:11.451030+0000","last_fullsized":"2024-09-19T07:43:11.451030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:51.595301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681743+0000","last_change":"2024-09-19T07:43:06.273520+0000","last_active":"2024-09-19T07:43:12.681743+0000","last_peered":"2024-09-19T07:43:12.681743+0000","last_clean":"2024-09-19T07:43:12.681743+0000","last_became_active":"2024-09-19T07:43:06.273150+0000","last_became_peered":"2024-09-19T07:43:06.273150+0000","last_unstale":"2024-09-19T07:43:12.681743+0000","last_undegraded":"2024-09-19T07:43:12.681743+0000","last_fullsized":"2024-09-19T07:43:12.681743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:39.464868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.309860+0000","last_change":"2024-09-19T07:43:06.272990+0000","last_active":"2024-09-19T07:43:11.309860+0000","last_peered":"2024-09-19T07:43:11.309860+0000","last_clean":"2024-09-19T07:43:11.309860+0000","last_became_active":"2024-09-19T07:43:06.272323+0000","last_became_peered":"2024-09-19T07:43:06.272323+0000","last_unstale":"2024-09-19T07:43:11.309860+0000","last_undegraded":"2024-09-19T07:43:11.309860+0000","last_fullsized":"2024-09-19T07:43:11.309860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:14.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681143+0000","last_change":"2024-09-19T07:43:06.273648+0000","last_active":"2024-09-19T07:43:12.681143+0000","last_peered":"2024-09-19T07:43:12.681143+0000","last_clean":"2024-09-19T07:43:12.681143+0000","last_became_active":"2024-09-19T07:43:06.273387+0000","last_became_peered":"2024-09-19T07:43:06.273387+0000","last_unstale":"2024-09-19T07:43:12.681143+0000","last_undegraded":"2024-09-19T07:43:12.681143+0000","last_fullsized":"2024-09-19T07:43:12.681143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:56.380341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311270+0000","last_change":"2024-09-19T07:43:06.274896+0000","last_active":"2024-09-19T07:43:11.311270+0000","last_peered":"2024-09-19T07:43:11.311270+0000","last_clean":"2024-09-19T07:43:11.311270+0000","last_became_active":"2024-09-19T07:43:06.274551+0000","last_became_peered":"2024-09-19T07:43:06.274551+0000","last_unstale":"2024-09-19T07:43:11.311270+0000","last_undegraded":"2024-09-19T07:43:11.311270+0000","last_fullsized":"2024-09-19T07:43:11.311270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:39.658744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681148+0000","last_change":"2024-09-19T07:43:06.271789+0000","last_active":"2024-09-19T07:43:12.681148+0000","last_peered":"2024-09-19T07:43:12.681148+0000","last_clean":"2024-09-19T07:43:12.681148+0000","last_became_active":"2024-09-19T07:43:06.269978+0000","last_became_peered":"2024-09-19T07:43:06.269978+0000","last_unstale":"2024-09-19T07:43:12.681148+0000","last_undegraded":"2024-09-19T07:43:12.681148+0000","last_fullsized":"2024-09-19T07:43:12.681148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:04.168224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311913+0000","last_change":"2024-09-19T07:43:06.274164+0000","last_active":"2024-09-19T07:43:11.311913+0000","last_peered":"2024-09-19T07:43:11.311913+0000","last_clean":"2024-09-19T07:43:11.311913+0000","last_became_active":"2024-09-19T07:43:06.273730+0000","last_became_peered":"2024-09-19T07:43:06.273730+0000","last_unstale":"2024-09-19T07:43:11.311913+0000","last_undegraded":"2024-09-19T07:43:11.311913+0000","last_fullsized":"2024-09-19T07:43:11.311913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:56.380341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655874+0000","last_change":"2024-09-19T07:43:06.266043+0000","last_active":"2024-09-19T07:43:12.655874+0000","last_peered":"2024-09-19T07:43:12.655874+0000","last_clean":"2024-09-19T07:43:12.655874+0000","last_became_active":"2024-09-19T07:43:06.265717+0000","last_became_peered":"2024-09-19T07:43:06.265717+0000","last_unstale":"2024-09-19T07:43:12.655874+0000","last_undegraded":"2024-09-19T07:43:12.655874+0000","last_fullsized":"2024-09-19T07:43:12.655874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:12.618593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681153+0000","last_change":"2024-09-19T07:43:06.270945+0000","last_active":"2024-09-19T07:43:12.681153+0000","last_peered":"2024-09-19T07:43:12.681153+0000","last_clean":"2024-09-19T07:43:12.681153+0000","last_became_active":"2024-09-19T07:43:06.270641+0000","last_became_peered":"2024-09-19T07:43:06.270641+0000","last_unstale":"2024-09-19T07:43:12.681153+0000","last_undegraded":"2024-09-19T07:43:12.681153+0000","last_fullsized":"2024-09-19T07:43:12.681153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:27.185190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681632+0000","last_change":"2024-09-19T07:43:06.277928+0000","last_active":"2024-09-19T07:43:12.681632+0000","last_peered":"2024-09-19T07:43:12.681632+0000","last_clean":"2024-09-19T07:43:12.681632+0000","last_became_active":"2024-09-19T07:43:06.277632+0000","last_became_peered":"2024-09-19T07:43:06.277632+0000","last_unstale":"2024-09-19T07:43:12.681632+0000","last_undegraded":"2024-09-19T07:43:12.681632+0000","last_fullsized":"2024-09-19T07:43:12.681632+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:09.222719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":868352,"data_stored":697396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":110290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":106536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":106572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":876544,"data_stored":697452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":282624,"data_stored":110290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":278528,"data_stored":106606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":270336,"data_stored":106446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:43:26.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:43:26.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.470+0000 7fb8267fd640 1 Processor -- start 2024-09-19T07:43:26.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.470+0000 7fb8267fd640 1 -- start start 2024-09-19T07:43:26.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.471+0000 7fb8267fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb82015f0d0 0x7fb82015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:26.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.471+0000 7fb8267fd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8200d4130 0x7fb8200d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:26.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.471+0000 7fb8267fd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8200d4b20 con 0x7fb8200d0070 2024-09-19T07:43:26.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.471+0000 7fb8267fd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb8200d1b10 con 0x7fb8200d4130 2024-09-19T07:43:26.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.471+0000 7fb8267fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb8200cf2b0 con 0x7fb82015f0d0 2024-09-19T07:43:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.471+0000 7fb824d9a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8200d4130 0x7fb8200d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.471+0000 7fb81f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb8200d0070 0x7fb8200d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44828/0 (socket says 172.21.15.16:44828) 2024-09-19T07:43:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.471+0000 7fb824d9a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8200d4130 0x7fb8200d45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57238/0 (socket says 172.21.15.16:57238) 2024-09-19T07:43:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.471+0000 7fb81f7fe640 1 -- 172.21.15.16:0/1755329282 learned_addr learned my addr 172.21.15.16:0/1755329282 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.471+0000 7fb81ffff640 1 --2- 172.21.15.16:0/1755329282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb82015f0d0 0x7fb82015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.472+0000 7fb824d9a640 1 -- 172.21.15.16:0/1755329282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb82015f0d0 msgr2=0x7fb82015f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.472+0000 7fb824d9a640 1 --2- 172.21.15.16:0/1755329282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb82015f0d0 0x7fb82015f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.472+0000 7fb824d9a640 1 -- 172.21.15.16:0/1755329282 >> v1:172.21.15.16:6789/0 conn(0x7fb8200d0070 legacy=0x7fb8200d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.472+0000 7fb824d9a640 1 -- 172.21.15.16:0/1755329282 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8200cf590 con 0x7fb8200d4130 2024-09-19T07:43:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.472+0000 7fb824d9a640 1 --2- 172.21.15.16:0/1755329282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200d4130 0x7fb8200d45e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb8200d2fc0 tx=0x7fb81802a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb08b456050381e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.473+0000 7fb81effd640 1 -- 172.21.15.16:0/1755329282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818007740 con 0x7fb8200d4130 2024-09-19T07:43:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.473+0000 7fb81effd640 1 -- 172.21.15.16:0/1755329282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8180078e0 con 0x7fb8200d4130 2024-09-19T07:43:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.473+0000 7fb81effd640 1 -- 172.21.15.16:0/1755329282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818007cb0 con 0x7fb8200d4130 2024-09-19T07:43:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.473+0000 7fb8267fd640 1 -- 172.21.15.16:0/1755329282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200d4130 msgr2=0x7fb8200d45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.473+0000 7fb8267fd640 1 --2- 172.21.15.16:0/1755329282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200d4130 0x7fb8200d45e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb8200d2fc0 tx=0x7fb81802a5b0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.473+0000 7fb8267fd640 1 -- 172.21.15.16:0/1755329282 shutdown_connections 2024-09-19T07:43:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.473+0000 7fb8267fd640 1 --2- 172.21.15.16:0/1755329282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200d4130 0x7fb8200d45e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.473+0000 7fb8267fd640 1 --2- 172.21.15.16:0/1755329282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb82015f0d0 0x7fb82015f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.473+0000 7fb8267fd640 1 -- 172.21.15.16:0/1755329282 >> 172.21.15.16:0/1755329282 conn(0x7fb82009d760 msgr2=0x7fb82008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.473+0000 7fb8267fd640 1 -- 172.21.15.16:0/1755329282 shutdown_connections 2024-09-19T07:43:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.473+0000 7fb8267fd640 1 -- 172.21.15.16:0/1755329282 wait complete. 2024-09-19T07:43:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.475+0000 7fb8267fd640 1 Processor -- start 2024-09-19T07:43:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.475+0000 7fb8267fd640 1 -- start start 2024-09-19T07:43:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.476+0000 7fb8267fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200d0070 0x7fb82015ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.476+0000 7fb8267fd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8200d4130 0x7fb8200b9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.476+0000 7fb8267fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb82015f0d0 0x7fb8200b7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.476+0000 7fb8267fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8200cef60 con 0x7fb8200d0070 2024-09-19T07:43:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.476+0000 7fb8267fd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb8200cfb50 con 0x7fb8200d4130 2024-09-19T07:43:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.476+0000 7fb8267fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb8200b2f70 con 0x7fb82015f0d0 2024-09-19T07:43:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.476+0000 7fb81ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200d0070 0x7fb82015ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.476+0000 7fb81ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200d0070 0x7fb82015ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57252/0 (socket says 172.21.15.16:57252) 2024-09-19T07:43:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.476+0000 7fb81ffff640 1 -- 172.21.15.16:0/371032979 learned_addr learned my addr 172.21.15.16:0/371032979 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.476+0000 7fb824d9a640 1 --2- 172.21.15.16:0/371032979 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb82015f0d0 0x7fb8200b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.476+0000 7fb81f7fe640 1 --2- 172.21.15.16:0/371032979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8200d4130 0x7fb8200b9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.476+0000 7fb81ffff640 1 -- 172.21.15.16:0/371032979 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb82015f0d0 msgr2=0x7fb8200b7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.476+0000 7fb81ffff640 1 --2- 172.21.15.16:0/371032979 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb82015f0d0 0x7fb8200b7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.476+0000 7fb81ffff640 1 -- 172.21.15.16:0/371032979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8200d4130 msgr2=0x7fb8200b9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.476+0000 7fb81ffff640 1 --2- 172.21.15.16:0/371032979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8200d4130 0x7fb8200b9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.476+0000 7fb81ffff640 1 -- 172.21.15.16:0/371032979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8200ba2c0 con 0x7fb8200d0070 2024-09-19T07:43:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.476+0000 7fb81ffff640 1 --2- 172.21.15.16:0/371032979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200d0070 0x7fb82015ba60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb81000d5f0 tx=0x7fb81000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.477+0000 7fb81cff9640 1 -- 172.21.15.16:0/371032979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8100125f0 con 0x7fb8200d0070 2024-09-19T07:43:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.477+0000 7fb81cff9640 1 -- 172.21.15.16:0/371032979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb810019070 con 0x7fb8200d0070 2024-09-19T07:43:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.477+0000 7fb8267fd640 1 -- 172.21.15.16:0/371032979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8200b9fe0 con 0x7fb8200d0070 2024-09-19T07:43:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.477+0000 7fb81cff9640 1 -- 172.21.15.16:0/371032979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8100129d0 con 0x7fb8200d0070 2024-09-19T07:43:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.477+0000 7fb8267fd640 1 -- 172.21.15.16:0/371032979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8200cbb60 con 0x7fb8200d0070 2024-09-19T07:43:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.478+0000 7fb8267fd640 1 -- 172.21.15.16:0/371032979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8200d45e0 con 0x7fb8200d0070 2024-09-19T07:43:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.478+0000 7fb81cff9640 1 -- 172.21.15.16:0/371032979 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb810012b70 con 0x7fb8200d0070 2024-09-19T07:43:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.478+0000 7fb81cff9640 1 --2- 172.21.15.16:0/371032979 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb80c06c2b0 0x7fb80c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.479+0000 7fb81cff9640 1 -- 172.21.15.16:0/371032979 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb81008b9e0 con 0x7fb8200d0070 2024-09-19T07:43:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.479+0000 7fb81f7fe640 1 --2- 172.21.15.16:0/371032979 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb80c06c2b0 0x7fb80c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.479+0000 7fb81f7fe640 1 --2- 172.21.15.16:0/371032979 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb80c06c2b0 0x7fb80c06e770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb8200b5bf0 tx=0x7fb814007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:26.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.484+0000 7fb81cff9640 1 -- 172.21.15.16:0/371032979 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb810059850 con 0x7fb8200d0070 2024-09-19T07:43:26.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.659+0000 7fb8267fd640 1 -- 172.21.15.16:0/371032979 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb82006dd00 con 0x7fb80c06c2b0 2024-09-19T07:43:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.667+0000 7fb81cff9640 1 -- 172.21.15.16:0/371032979 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fb82006dd00 con 0x7fb80c06c2b0 2024-09-19T07:43:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:26.672 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:43:26.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.670+0000 7fb8267fd640 1 -- 172.21.15.16:0/371032979 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb80c06c2b0 msgr2=0x7fb80c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:26.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.670+0000 7fb8267fd640 1 --2- 172.21.15.16:0/371032979 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb80c06c2b0 0x7fb80c06e770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb8200b5bf0 tx=0x7fb814007e80 comp rx=0 tx=0).stop 2024-09-19T07:43:26.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.670+0000 7fb8267fd640 1 -- 172.21.15.16:0/371032979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200d0070 msgr2=0x7fb82015ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:26.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.670+0000 7fb8267fd640 1 --2- 172.21.15.16:0/371032979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200d0070 0x7fb82015ba60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb81000d5f0 tx=0x7fb81000dab0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.670+0000 7fb8267fd640 1 -- 172.21.15.16:0/371032979 shutdown_connections 2024-09-19T07:43:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.670+0000 7fb8267fd640 1 --2- 172.21.15.16:0/371032979 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb80c06c2b0 0x7fb80c06e770 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.670+0000 7fb8267fd640 1 --2- 172.21.15.16:0/371032979 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb82015f0d0 0x7fb8200b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.670+0000 7fb8267fd640 1 --2- 172.21.15.16:0/371032979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8200d4130 0x7fb8200b9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.670+0000 7fb8267fd640 1 --2- 172.21.15.16:0/371032979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8200d0070 0x7fb82015ba60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.670+0000 7fb8267fd640 1 -- 172.21.15.16:0/371032979 >> 172.21.15.16:0/371032979 conn(0x7fb82009d760 msgr2=0x7fb8200b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.670+0000 7fb8267fd640 1 -- 172.21.15.16:0/371032979 shutdown_connections 2024-09-19T07:43:26.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.670+0000 7fb8267fd640 1 -- 172.21.15.16:0/371032979 wait complete. 2024-09-19T07:43:26.682 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-09-19T07:43:26.462548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236216,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738504,"statfs":{"total":773094113280,"available":772852228096,"internally_reserved":0,"allocated":3416064,"data_stored":2041588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003496"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656052+0000","last_change":"2024-09-19T07:43:05.265202+0000","last_active":"2024-09-19T07:43:12.656052+0000","last_peered":"2024-09-19T07:43:12.656052+0000","last_clean":"2024-09-19T07:43:12.656052+0000","last_became_active":"2024-09-19T07:43:05.264871+0000","last_became_peered":"2024-09-19T07:43:05.264871+0000","last_unstale":"2024-09-19T07:43:12.656052+0000","last_undegraded":"2024-09-19T07:43:12.656052+0000","last_fullsized":"2024-09-19T07:43:12.656052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:36:39.982941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310781+0000","last_change":"2024-09-19T07:43:06.264003+0000","last_active":"2024-09-19T07:43:11.310781+0000","last_peered":"2024-09-19T07:43:11.310781+0000","last_clean":"2024-09-19T07:43:11.310781+0000","last_became_active":"2024-09-19T07:43:06.262313+0000","last_became_peered":"2024-09-19T07:43:06.262313+0000","last_unstale":"2024-09-19T07:43:11.310781+0000","last_undegraded":"2024-09-19T07:43:11.310781+0000","last_fullsized":"2024-09-19T07:43:11.310781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.309855+0000","last_change":"2024-09-19T07:43:10.483058+0000","last_active":"2024-09-19T07:43:11.309855+0000","last_peered":"2024-09-19T07:43:11.309855+0000","last_clean":"2024-09-19T07:43:11.309855+0000","last_became_active":"2024-09-19T07:43:10.482621+0000","last_became_peered":"2024-09-19T07:43:10.482621+0000","last_unstale":"2024-09-19T07:43:11.309855+0000","last_undegraded":"2024-09-19T07:43:11.309855+0000","last_fullsized":"2024-09-19T07:43:11.309855+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:27.168500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310785+0000","last_change":"2024-09-19T07:43:05.265029+0000","last_active":"2024-09-19T07:43:11.310785+0000","last_peered":"2024-09-19T07:43:11.310785+0000","last_clean":"2024-09-19T07:43:11.310785+0000","last_became_active":"2024-09-19T07:43:05.264529+0000","last_became_peered":"2024-09-19T07:43:05.264529+0000","last_unstale":"2024-09-19T07:43:11.310785+0000","last_undegraded":"2024-09-19T07:43:11.310785+0000","last_fullsized":"2024-09-19T07:43:11.310785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:12.998698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655877+0000","last_change":"2024-09-19T07:43:06.272531+0000","last_active":"2024-09-19T07:43:12.655877+0000","last_peered":"2024-09-19T07:43:12.655877+0000","last_clean":"2024-09-19T07:43:12.655877+0000","last_became_active":"2024-09-19T07:43:06.272225+0000","last_became_peered":"2024-09-19T07:43:06.272225+0000","last_unstale":"2024-09-19T07:43:12.655877+0000","last_undegraded":"2024-09-19T07:43:12.655877+0000","last_fullsized":"2024-09-19T07:43:12.655877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:06.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311381+0000","last_change":"2024-09-19T07:43:10.288276+0000","last_active":"2024-09-19T07:43:11.311381+0000","last_peered":"2024-09-19T07:43:11.311381+0000","last_clean":"2024-09-19T07:43:11.311381+0000","last_became_active":"2024-09-19T07:43:10.287816+0000","last_became_peered":"2024-09-19T07:43:10.287816+0000","last_unstale":"2024-09-19T07:43:11.311381+0000","last_undegraded":"2024-09-19T07:43:11.311381+0000","last_fullsized":"2024-09-19T07:43:11.311381+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:26.830106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681715+0000","last_change":"2024-09-19T07:43:05.266982+0000","last_active":"2024-09-19T07:43:12.681715+0000","last_peered":"2024-09-19T07:43:12.681715+0000","last_clean":"2024-09-19T07:43:12.681715+0000","last_became_active":"2024-09-19T07:43:05.266667+0000","last_became_peered":"2024-09-19T07:43:05.266667+0000","last_unstale":"2024-09-19T07:43:12.681715+0000","last_undegraded":"2024-09-19T07:43:12.681715+0000","last_fullsized":"2024-09-19T07:43:12.681715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:02.488778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312020+0000","last_change":"2024-09-19T07:43:06.273456+0000","last_active":"2024-09-19T07:43:11.312020+0000","last_peered":"2024-09-19T07:43:11.312020+0000","last_clean":"2024-09-19T07:43:11.312020+0000","last_became_active":"2024-09-19T07:43:06.273046+0000","last_became_peered":"2024-09-19T07:43:06.273046+0000","last_unstale":"2024-09-19T07:43:11.312020+0000","last_undegraded":"2024-09-19T07:43:11.312020+0000","last_fullsized":"2024-09-19T07:43:11.312020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:09.222719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310977+0000","last_change":"2024-09-19T07:43:10.293267+0000","last_active":"2024-09-19T07:43:11.310977+0000","last_peered":"2024-09-19T07:43:11.310977+0000","last_clean":"2024-09-19T07:43:11.310977+0000","last_became_active":"2024-09-19T07:43:10.292870+0000","last_became_peered":"2024-09-19T07:43:10.292870+0000","last_unstale":"2024-09-19T07:43:11.310977+0000","last_undegraded":"2024-09-19T07:43:11.310977+0000","last_fullsized":"2024-09-19T07:43:11.310977+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:54.570005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681497+0000","last_change":"2024-09-19T07:43:05.267905+0000","last_active":"2024-09-19T07:43:12.681497+0000","last_peered":"2024-09-19T07:43:12.681497+0000","last_clean":"2024-09-19T07:43:12.681497+0000","last_became_active":"2024-09-19T07:43:05.267563+0000","last_became_peered":"2024-09-19T07:43:05.267563+0000","last_unstale":"2024-09-19T07:43:12.681497+0000","last_undegraded":"2024-09-19T07:43:12.681497+0000","last_fullsized":"2024-09-19T07:43:12.681497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:47.531702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311784+0000","last_change":"2024-09-19T07:43:06.262966+0000","last_active":"2024-09-19T07:43:11.311784+0000","last_peered":"2024-09-19T07:43:11.311784+0000","last_clean":"2024-09-19T07:43:11.311784+0000","last_became_active":"2024-09-19T07:43:06.262698+0000","last_became_peered":"2024-09-19T07:43:06.262698+0000","last_unstale":"2024-09-19T07:43:11.311784+0000","last_undegraded":"2024-09-19T07:43:11.311784+0000","last_fullsized":"2024-09-19T07:43:11.311784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:47.918580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310630+0000","last_change":"2024-09-19T07:43:10.287084+0000","last_active":"2024-09-19T07:43:11.310630+0000","last_peered":"2024-09-19T07:43:11.310630+0000","last_clean":"2024-09-19T07:43:11.310630+0000","last_became_active":"2024-09-19T07:43:10.286693+0000","last_became_peered":"2024-09-19T07:43:10.286693+0000","last_unstale":"2024-09-19T07:43:11.310630+0000","last_undegraded":"2024-09-19T07:43:11.310630+0000","last_fullsized":"2024-09-19T07:43:11.310630+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:17.611361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450546+0000","last_change":"2024-09-19T07:43:06.262664+0000","last_active":"2024-09-19T07:43:11.450546+0000","last_peered":"2024-09-19T07:43:11.450546+0000","last_clean":"2024-09-19T07:43:11.450546+0000","last_became_active":"2024-09-19T07:43:06.262389+0000","last_became_peered":"2024-09-19T07:43:06.262389+0000","last_unstale":"2024-09-19T07:43:11.450546+0000","last_undegraded":"2024-09-19T07:43:11.450546+0000","last_fullsized":"2024-09-19T07:43:11.450546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:06.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681454+0000","last_change":"2024-09-19T07:43:05.262431+0000","last_active":"2024-09-19T07:43:12.681454+0000","last_peered":"2024-09-19T07:43:12.681454+0000","last_clean":"2024-09-19T07:43:12.681454+0000","last_became_active":"2024-09-19T07:43:05.262156+0000","last_became_peered":"2024-09-19T07:43:05.262156+0000","last_unstale":"2024-09-19T07:43:12.681454+0000","last_undegraded":"2024-09-19T07:43:12.681454+0000","last_fullsized":"2024-09-19T07:43:12.681454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:27.338560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312544+0000","last_change":"2024-09-19T07:43:10.287687+0000","last_active":"2024-09-19T07:43:11.312544+0000","last_peered":"2024-09-19T07:43:11.312544+0000","last_clean":"2024-09-19T07:43:11.312544+0000","last_became_active":"2024-09-19T07:43:10.287301+0000","last_became_peered":"2024-09-19T07:43:10.287301+0000","last_unstale":"2024-09-19T07:43:11.312544+0000","last_undegraded":"2024-09-19T07:43:11.312544+0000","last_fullsized":"2024-09-19T07:43:11.312544+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:43.684480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310291+0000","last_change":"2024-09-19T07:43:06.274918+0000","last_active":"2024-09-19T07:43:11.310291+0000","last_peered":"2024-09-19T07:43:11.310291+0000","last_clean":"2024-09-19T07:43:11.310291+0000","last_became_active":"2024-09-19T07:43:06.274639+0000","last_became_peered":"2024-09-19T07:43:06.274639+0000","last_unstale":"2024-09-19T07:43:11.310291+0000","last_undegraded":"2024-09-19T07:43:11.310291+0000","last_fullsized":"2024-09-19T07:43:11.310291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:22.804370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680919+0000","last_change":"2024-09-19T07:43:05.258127+0000","last_active":"2024-09-19T07:43:12.680919+0000","last_peered":"2024-09-19T07:43:12.680919+0000","last_clean":"2024-09-19T07:43:12.680919+0000","last_became_active":"2024-09-19T07:43:05.257766+0000","last_became_peered":"2024-09-19T07:43:05.257766+0000","last_unstale":"2024-09-19T07:43:12.680919+0000","last_undegraded":"2024-09-19T07:43:12.680919+0000","last_fullsized":"2024-09-19T07:43:12.680919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:02.488778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311080+0000","last_change":"2024-09-19T07:43:10.294352+0000","last_active":"2024-09-19T07:43:11.311080+0000","last_peered":"2024-09-19T07:43:11.311080+0000","last_clean":"2024-09-19T07:43:11.311080+0000","last_became_active":"2024-09-19T07:43:10.293957+0000","last_became_peered":"2024-09-19T07:43:10.293957+0000","last_unstale":"2024-09-19T07:43:11.311080+0000","last_undegraded":"2024-09-19T07:43:11.311080+0000","last_fullsized":"2024-09-19T07:43:11.311080+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:13.248454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310160+0000","last_change":"2024-09-19T07:43:06.256705+0000","last_active":"2024-09-19T07:43:11.310160+0000","last_peered":"2024-09-19T07:43:11.310160+0000","last_clean":"2024-09-19T07:43:11.310160+0000","last_became_active":"2024-09-19T07:43:06.256423+0000","last_became_peered":"2024-09-19T07:43:06.256423+0000","last_unstale":"2024-09-19T07:43:11.310160+0000","last_undegraded":"2024-09-19T07:43:11.310160+0000","last_fullsized":"2024-09-19T07:43:11.310160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:47.918580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680712+0000","last_change":"2024-09-19T07:43:05.260367+0000","last_active":"2024-09-19T07:43:12.680712+0000","last_peered":"2024-09-19T07:43:12.680712+0000","last_clean":"2024-09-19T07:43:12.680712+0000","last_became_active":"2024-09-19T07:43:05.260056+0000","last_became_peered":"2024-09-19T07:43:05.260056+0000","last_unstale":"2024-09-19T07:43:12.680712+0000","last_undegraded":"2024-09-19T07:43:12.680712+0000","last_fullsized":"2024-09-19T07:43:12.680712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:38:29.212025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681342+0000","last_change":"2024-09-19T07:43:10.482020+0000","last_active":"2024-09-19T07:43:12.681342+0000","last_peered":"2024-09-19T07:43:12.681342+0000","last_clean":"2024-09-19T07:43:12.681342+0000","last_became_active":"2024-09-19T07:43:10.480883+0000","last_became_peered":"2024-09-19T07:43:10.480883+0000","last_unstale":"2024-09-19T07:43:12.681342+0000","last_undegraded":"2024-09-19T07:43:12.681342+0000","last_fullsized":"2024-09-19T07:43:12.681342+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:01.954262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312259+0000","last_change":"2024-09-19T07:43:06.268719+0000","last_active":"2024-09-19T07:43:11.312259+0000","last_peered":"2024-09-19T07:43:11.312259+0000","last_clean":"2024-09-19T07:43:11.312259+0000","last_became_active":"2024-09-19T07:43:06.268405+0000","last_became_peered":"2024-09-19T07:43:06.268405+0000","last_unstale":"2024-09-19T07:43:11.312259+0000","last_undegraded":"2024-09-19T07:43:11.312259+0000","last_fullsized":"2024-09-19T07:43:11.312259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:45.364876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681025+0000","last_change":"2024-09-19T07:43:05.267962+0000","last_active":"2024-09-19T07:43:12.681025+0000","last_peered":"2024-09-19T07:43:12.681025+0000","last_clean":"2024-09-19T07:43:12.681025+0000","last_became_active":"2024-09-19T07:43:05.267670+0000","last_became_peered":"2024-09-19T07:43:05.267670+0000","last_unstale":"2024-09-19T07:43:12.681025+0000","last_undegraded":"2024-09-19T07:43:12.681025+0000","last_fullsized":"2024-09-19T07:43:12.681025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:13.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312584+0000","last_change":"2024-09-19T07:43:10.289357+0000","last_active":"2024-09-19T07:43:11.312584+0000","last_peered":"2024-09-19T07:43:11.312584+0000","last_clean":"2024-09-19T07:43:11.312584+0000","last_became_active":"2024-09-19T07:43:10.288596+0000","last_became_peered":"2024-09-19T07:43:10.288596+0000","last_unstale":"2024-09-19T07:43:11.312584+0000","last_undegraded":"2024-09-19T07:43:11.312584+0000","last_fullsized":"2024-09-19T07:43:11.312584+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:08.193959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656239+0000","last_change":"2024-09-19T07:43:06.270766+0000","last_active":"2024-09-19T07:43:12.656239+0000","last_peered":"2024-09-19T07:43:12.656239+0000","last_clean":"2024-09-19T07:43:12.656239+0000","last_became_active":"2024-09-19T07:43:06.270494+0000","last_became_peered":"2024-09-19T07:43:06.270494+0000","last_unstale":"2024-09-19T07:43:12.656239+0000","last_undegraded":"2024-09-19T07:43:12.656239+0000","last_fullsized":"2024-09-19T07:43:12.656239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451242+0000","last_change":"2024-09-19T07:43:05.263058+0000","last_active":"2024-09-19T07:43:11.451242+0000","last_peered":"2024-09-19T07:43:11.451242+0000","last_clean":"2024-09-19T07:43:11.451242+0000","last_became_active":"2024-09-19T07:43:05.262762+0000","last_became_peered":"2024-09-19T07:43:05.262762+0000","last_unstale":"2024-09-19T07:43:11.451242+0000","last_undegraded":"2024-09-19T07:43:11.451242+0000","last_fullsized":"2024-09-19T07:43:11.451242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:38:29.212025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312088+0000","last_change":"2024-09-19T07:43:10.482293+0000","last_active":"2024-09-19T07:43:11.312088+0000","last_peered":"2024-09-19T07:43:11.312088+0000","last_clean":"2024-09-19T07:43:11.312088+0000","last_became_active":"2024-09-19T07:43:10.481268+0000","last_became_peered":"2024-09-19T07:43:10.481268+0000","last_unstale":"2024-09-19T07:43:11.312088+0000","last_undegraded":"2024-09-19T07:43:11.312088+0000","last_fullsized":"2024-09-19T07:43:11.312088+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:51.944316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311136+0000","last_change":"2024-09-19T07:43:06.268026+0000","last_active":"2024-09-19T07:43:11.311136+0000","last_peered":"2024-09-19T07:43:11.311136+0000","last_clean":"2024-09-19T07:43:11.311136+0000","last_became_active":"2024-09-19T07:43:06.267713+0000","last_became_peered":"2024-09-19T07:43:06.267713+0000","last_unstale":"2024-09-19T07:43:11.311136+0000","last_undegraded":"2024-09-19T07:43:11.311136+0000","last_fullsized":"2024-09-19T07:43:11.311136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:56.380341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681209+0000","last_change":"2024-09-19T07:43:05.263248+0000","last_active":"2024-09-19T07:43:12.681209+0000","last_peered":"2024-09-19T07:43:12.681209+0000","last_clean":"2024-09-19T07:43:12.681209+0000","last_became_active":"2024-09-19T07:43:05.262892+0000","last_became_peered":"2024-09-19T07:43:05.262892+0000","last_unstale":"2024-09-19T07:43:12.681209+0000","last_undegraded":"2024-09-19T07:43:12.681209+0000","last_fullsized":"2024-09-19T07:43:12.681209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:47.531702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.309826+0000","last_change":"2024-09-19T07:43:10.482975+0000","last_active":"2024-09-19T07:43:11.309826+0000","last_peered":"2024-09-19T07:43:11.309826+0000","last_clean":"2024-09-19T07:43:11.309826+0000","last_became_active":"2024-09-19T07:43:10.482584+0000","last_became_peered":"2024-09-19T07:43:10.482584+0000","last_unstale":"2024-09-19T07:43:11.309826+0000","last_undegraded":"2024-09-19T07:43:11.309826+0000","last_fullsized":"2024-09-19T07:43:11.309826+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:10.437589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680841+0000","last_change":"2024-09-19T07:43:06.258826+0000","last_active":"2024-09-19T07:43:12.680841+0000","last_peered":"2024-09-19T07:43:12.680841+0000","last_clean":"2024-09-19T07:43:12.680841+0000","last_became_active":"2024-09-19T07:43:06.258518+0000","last_became_peered":"2024-09-19T07:43:06.258518+0000","last_unstale":"2024-09-19T07:43:12.680841+0000","last_undegraded":"2024-09-19T07:43:12.680841+0000","last_fullsized":"2024-09-19T07:43:12.680841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681832+0000","last_change":"2024-09-19T07:43:05.267167+0000","last_active":"2024-09-19T07:43:12.681832+0000","last_peered":"2024-09-19T07:43:12.681832+0000","last_clean":"2024-09-19T07:43:12.681832+0000","last_became_active":"2024-09-19T07:43:05.266892+0000","last_became_peered":"2024-09-19T07:43:05.266892+0000","last_unstale":"2024-09-19T07:43:12.681832+0000","last_undegraded":"2024-09-19T07:43:12.681832+0000","last_fullsized":"2024-09-19T07:43:12.681832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:06.889703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655769+0000","last_change":"2024-09-19T07:43:10.480657+0000","last_active":"2024-09-19T07:43:12.655769+0000","last_peered":"2024-09-19T07:43:12.655769+0000","last_clean":"2024-09-19T07:43:12.655769+0000","last_became_active":"2024-09-19T07:43:10.480186+0000","last_became_peered":"2024-09-19T07:43:10.480186+0000","last_unstale":"2024-09-19T07:43:12.655769+0000","last_undegraded":"2024-09-19T07:43:12.655769+0000","last_fullsized":"2024-09-19T07:43:12.655769+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:43.490603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311921+0000","last_change":"2024-09-19T07:43:06.265266+0000","last_active":"2024-09-19T07:43:11.311921+0000","last_peered":"2024-09-19T07:43:11.311921+0000","last_clean":"2024-09-19T07:43:11.311921+0000","last_became_active":"2024-09-19T07:43:06.265001+0000","last_became_peered":"2024-09-19T07:43:06.265001+0000","last_unstale":"2024-09-19T07:43:11.311921+0000","last_undegraded":"2024-09-19T07:43:11.311921+0000","last_fullsized":"2024-09-19T07:43:11.311921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:39.464868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680965+0000","last_change":"2024-09-19T07:43:05.259114+0000","last_active":"2024-09-19T07:43:12.680965+0000","last_peered":"2024-09-19T07:43:12.680965+0000","last_clean":"2024-09-19T07:43:12.680965+0000","last_became_active":"2024-09-19T07:43:05.258794+0000","last_became_peered":"2024-09-19T07:43:05.258794+0000","last_unstale":"2024-09-19T07:43:12.680965+0000","last_undegraded":"2024-09-19T07:43:12.680965+0000","last_fullsized":"2024-09-19T07:43:12.680965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:13.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680778+0000","last_change":"2024-09-19T07:43:10.295795+0000","last_active":"2024-09-19T07:43:12.680778+0000","last_peered":"2024-09-19T07:43:12.680778+0000","last_clean":"2024-09-19T07:43:12.680778+0000","last_became_active":"2024-09-19T07:43:10.295379+0000","last_became_peered":"2024-09-19T07:43:10.295379+0000","last_unstale":"2024-09-19T07:43:12.680778+0000","last_undegraded":"2024-09-19T07:43:12.680778+0000","last_fullsized":"2024-09-19T07:43:12.680778+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:43.490603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311661+0000","last_change":"2024-09-19T07:43:06.271554+0000","last_active":"2024-09-19T07:43:11.311661+0000","last_peered":"2024-09-19T07:43:11.311661+0000","last_clean":"2024-09-19T07:43:11.311661+0000","last_became_active":"2024-09-19T07:43:06.271304+0000","last_became_peered":"2024-09-19T07:43:06.271304+0000","last_unstale":"2024-09-19T07:43:11.311661+0000","last_undegraded":"2024-09-19T07:43:11.311661+0000","last_fullsized":"2024-09-19T07:43:11.311661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:03.494318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311677+0000","last_change":"2024-09-19T07:43:05.264548+0000","last_active":"2024-09-19T07:43:11.311677+0000","last_peered":"2024-09-19T07:43:11.311677+0000","last_clean":"2024-09-19T07:43:11.311677+0000","last_became_active":"2024-09-19T07:43:05.261674+0000","last_became_peered":"2024-09-19T07:43:05.261674+0000","last_unstale":"2024-09-19T07:43:11.311677+0000","last_undegraded":"2024-09-19T07:43:11.311677+0000","last_fullsized":"2024-09-19T07:43:11.311677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:47.531702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681063+0000","last_change":"2024-09-19T07:43:10.288751+0000","last_active":"2024-09-19T07:43:12.681063+0000","last_peered":"2024-09-19T07:43:12.681063+0000","last_clean":"2024-09-19T07:43:12.681063+0000","last_became_active":"2024-09-19T07:43:10.288383+0000","last_became_peered":"2024-09-19T07:43:10.288383+0000","last_unstale":"2024-09-19T07:43:12.681063+0000","last_undegraded":"2024-09-19T07:43:12.681063+0000","last_fullsized":"2024-09-19T07:43:12.681063+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:43.944005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680858+0000","last_change":"2024-09-19T07:43:06.264468+0000","last_active":"2024-09-19T07:43:12.680858+0000","last_peered":"2024-09-19T07:43:12.680858+0000","last_clean":"2024-09-19T07:43:12.680858+0000","last_became_active":"2024-09-19T07:43:06.263801+0000","last_became_peered":"2024-09-19T07:43:06.263801+0000","last_unstale":"2024-09-19T07:43:12.680858+0000","last_undegraded":"2024-09-19T07:43:12.680858+0000","last_fullsized":"2024-09-19T07:43:12.680858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:06.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588628+0000","last_change":"2024-09-19T07:43:05.259922+0000","last_active":"2024-09-19T07:43:11.588628+0000","last_peered":"2024-09-19T07:43:11.588628+0000","last_clean":"2024-09-19T07:43:11.588628+0000","last_became_active":"2024-09-19T07:43:05.259552+0000","last_became_peered":"2024-09-19T07:43:05.259552+0000","last_unstale":"2024-09-19T07:43:11.588628+0000","last_undegraded":"2024-09-19T07:43:11.588628+0000","last_fullsized":"2024-09-19T07:43:11.588628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:55.737753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681498+0000","last_change":"2024-09-19T07:43:10.296047+0000","last_active":"2024-09-19T07:43:12.681498+0000","last_peered":"2024-09-19T07:43:12.681498+0000","last_clean":"2024-09-19T07:43:12.681498+0000","last_became_active":"2024-09-19T07:43:10.295636+0000","last_became_peered":"2024-09-19T07:43:10.295636+0000","last_unstale":"2024-09-19T07:43:12.681498+0000","last_undegraded":"2024-09-19T07:43:12.681498+0000","last_fullsized":"2024-09-19T07:43:12.681498+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:27.138783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655596+0000","last_change":"2024-09-19T07:43:06.273785+0000","last_active":"2024-09-19T07:43:12.655596+0000","last_peered":"2024-09-19T07:43:12.655596+0000","last_clean":"2024-09-19T07:43:12.655596+0000","last_became_active":"2024-09-19T07:43:06.273514+0000","last_became_peered":"2024-09-19T07:43:06.273514+0000","last_unstale":"2024-09-19T07:43:12.655596+0000","last_undegraded":"2024-09-19T07:43:12.655596+0000","last_fullsized":"2024-09-19T07:43:12.655596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:09.399092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681059+0000","last_change":"2024-09-19T07:43:05.263952+0000","last_active":"2024-09-19T07:43:12.681059+0000","last_peered":"2024-09-19T07:43:12.681059+0000","last_clean":"2024-09-19T07:43:12.681059+0000","last_became_active":"2024-09-19T07:43:05.263677+0000","last_became_peered":"2024-09-19T07:43:05.263677+0000","last_unstale":"2024-09-19T07:43:12.681059+0000","last_undegraded":"2024-09-19T07:43:12.681059+0000","last_fullsized":"2024-09-19T07:43:12.681059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:44.359336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656084+0000","last_change":"2024-09-19T07:43:10.480721+0000","last_active":"2024-09-19T07:43:12.656084+0000","last_peered":"2024-09-19T07:43:12.656084+0000","last_clean":"2024-09-19T07:43:12.656084+0000","last_became_active":"2024-09-19T07:43:10.480321+0000","last_became_peered":"2024-09-19T07:43:10.480321+0000","last_unstale":"2024-09-19T07:43:12.656084+0000","last_undegraded":"2024-09-19T07:43:12.656084+0000","last_fullsized":"2024-09-19T07:43:12.656084+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:01.954262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310005+0000","last_change":"2024-09-19T07:43:06.271482+0000","last_active":"2024-09-19T07:43:11.310005+0000","last_peered":"2024-09-19T07:43:11.310005+0000","last_clean":"2024-09-19T07:43:11.310005+0000","last_became_active":"2024-09-19T07:43:06.271212+0000","last_became_peered":"2024-09-19T07:43:06.271212+0000","last_unstale":"2024-09-19T07:43:11.310005+0000","last_undegraded":"2024-09-19T07:43:11.310005+0000","last_fullsized":"2024-09-19T07:43:11.310005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:27.185190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311209+0000","last_change":"2024-09-19T07:43:05.261378+0000","last_active":"2024-09-19T07:43:11.311209+0000","last_peered":"2024-09-19T07:43:11.311209+0000","last_clean":"2024-09-19T07:43:11.311209+0000","last_became_active":"2024-09-19T07:43:05.261055+0000","last_became_peered":"2024-09-19T07:43:05.261055+0000","last_unstale":"2024-09-19T07:43:11.311209+0000","last_undegraded":"2024-09-19T07:43:11.311209+0000","last_fullsized":"2024-09-19T07:43:11.311209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:12.998698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681162+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:43:12.681162+0000","last_peered":"2024-09-19T07:43:12.681162+0000","last_clean":"2024-09-19T07:43:12.681162+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:43:12.681162+0000","last_undegraded":"2024-09-19T07:43:12.681162+0000","last_fullsized":"2024-09-19T07:43:12.681162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450864+0000","last_change":"2024-09-19T07:43:10.481567+0000","last_active":"2024-09-19T07:43:11.450864+0000","last_peered":"2024-09-19T07:43:11.450864+0000","last_clean":"2024-09-19T07:43:11.450864+0000","last_became_active":"2024-09-19T07:43:10.481210+0000","last_became_peered":"2024-09-19T07:43:10.481210+0000","last_unstale":"2024-09-19T07:43:11.450864+0000","last_undegraded":"2024-09-19T07:43:11.450864+0000","last_fullsized":"2024-09-19T07:43:11.450864+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:08.193959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681552+0000","last_change":"2024-09-19T07:43:06.268655+0000","last_active":"2024-09-19T07:43:12.681552+0000","last_peered":"2024-09-19T07:43:12.681552+0000","last_clean":"2024-09-19T07:43:12.681552+0000","last_became_active":"2024-09-19T07:43:06.268387+0000","last_became_peered":"2024-09-19T07:43:06.268387+0000","last_unstale":"2024-09-19T07:43:12.681552+0000","last_undegraded":"2024-09-19T07:43:12.681552+0000","last_fullsized":"2024-09-19T07:43:12.681552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:14.347245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588993+0000","last_change":"2024-09-19T07:43:05.261374+0000","last_active":"2024-09-19T07:43:11.588993+0000","last_peered":"2024-09-19T07:43:11.588993+0000","last_clean":"2024-09-19T07:43:11.588993+0000","last_became_active":"2024-09-19T07:43:05.261088+0000","last_became_peered":"2024-09-19T07:43:05.261088+0000","last_unstale":"2024-09-19T07:43:11.588993+0000","last_undegraded":"2024-09-19T07:43:11.588993+0000","last_fullsized":"2024-09-19T07:43:11.588993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:39.630700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450612+0000","last_change":"2024-09-19T07:43:10.480274+0000","last_active":"2024-09-19T07:43:11.450612+0000","last_peered":"2024-09-19T07:43:11.450612+0000","last_clean":"2024-09-19T07:43:11.450612+0000","last_became_active":"2024-09-19T07:43:10.479848+0000","last_became_peered":"2024-09-19T07:43:10.479848+0000","last_unstale":"2024-09-19T07:43:11.450612+0000","last_undegraded":"2024-09-19T07:43:11.450612+0000","last_fullsized":"2024-09-19T07:43:11.450612+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:43.684480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312126+0000","last_change":"2024-09-19T07:43:06.264529+0000","last_active":"2024-09-19T07:43:11.312126+0000","last_peered":"2024-09-19T07:43:11.312126+0000","last_clean":"2024-09-19T07:43:11.312126+0000","last_became_active":"2024-09-19T07:43:06.264270+0000","last_became_peered":"2024-09-19T07:43:06.264270+0000","last_unstale":"2024-09-19T07:43:11.312126+0000","last_undegraded":"2024-09-19T07:43:11.312126+0000","last_fullsized":"2024-09-19T07:43:11.312126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:04.168224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681259+0000","last_change":"2024-09-19T07:43:05.254996+0000","last_active":"2024-09-19T07:43:12.681259+0000","last_peered":"2024-09-19T07:43:12.681259+0000","last_clean":"2024-09-19T07:43:12.681259+0000","last_became_active":"2024-09-19T07:43:05.254430+0000","last_became_peered":"2024-09-19T07:43:05.254430+0000","last_unstale":"2024-09-19T07:43:12.681259+0000","last_undegraded":"2024-09-19T07:43:12.681259+0000","last_fullsized":"2024-09-19T07:43:12.681259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:06.889703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656122+0000","last_change":"2024-09-19T07:43:10.295667+0000","last_active":"2024-09-19T07:43:12.656122+0000","last_peered":"2024-09-19T07:43:12.656122+0000","last_clean":"2024-09-19T07:43:12.656122+0000","last_became_active":"2024-09-19T07:43:10.295241+0000","last_became_peered":"2024-09-19T07:43:10.295241+0000","last_unstale":"2024-09-19T07:43:12.656122+0000","last_undegraded":"2024-09-19T07:43:12.656122+0000","last_fullsized":"2024-09-19T07:43:12.656122+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:27.138783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681804+0000","last_change":"2024-09-19T07:43:06.275720+0000","last_active":"2024-09-19T07:43:12.681804+0000","last_peered":"2024-09-19T07:43:12.681804+0000","last_clean":"2024-09-19T07:43:12.681804+0000","last_became_active":"2024-09-19T07:43:06.275403+0000","last_became_peered":"2024-09-19T07:43:06.275403+0000","last_unstale":"2024-09-19T07:43:12.681804+0000","last_undegraded":"2024-09-19T07:43:12.681804+0000","last_fullsized":"2024-09-19T07:43:12.681804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:39.918269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588845+0000","last_change":"2024-09-19T07:43:05.260217+0000","last_active":"2024-09-19T07:43:11.588845+0000","last_peered":"2024-09-19T07:43:11.588845+0000","last_clean":"2024-09-19T07:43:11.588845+0000","last_became_active":"2024-09-19T07:43:05.259482+0000","last_became_peered":"2024-09-19T07:43:05.259482+0000","last_unstale":"2024-09-19T07:43:11.588845+0000","last_undegraded":"2024-09-19T07:43:11.588845+0000","last_fullsized":"2024-09-19T07:43:11.588845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:56.644264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310513+0000","last_change":"2024-09-19T07:43:10.480657+0000","last_active":"2024-09-19T07:43:11.310513+0000","last_peered":"2024-09-19T07:43:11.310513+0000","last_clean":"2024-09-19T07:43:11.310513+0000","last_became_active":"2024-09-19T07:43:10.480279+0000","last_became_peered":"2024-09-19T07:43:10.480279+0000","last_unstale":"2024-09-19T07:43:11.310513+0000","last_undegraded":"2024-09-19T07:43:11.310513+0000","last_fullsized":"2024-09-19T07:43:11.310513+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:26.933433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681655+0000","last_change":"2024-09-19T07:43:06.275354+0000","last_active":"2024-09-19T07:43:12.681655+0000","last_peered":"2024-09-19T07:43:12.681655+0000","last_clean":"2024-09-19T07:43:12.681655+0000","last_became_active":"2024-09-19T07:43:06.275039+0000","last_became_peered":"2024-09-19T07:43:06.275039+0000","last_unstale":"2024-09-19T07:43:12.681655+0000","last_undegraded":"2024-09-19T07:43:12.681655+0000","last_fullsized":"2024-09-19T07:43:12.681655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:47.918580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311619+0000","last_change":"2024-09-19T07:43:05.265977+0000","last_active":"2024-09-19T07:43:11.311619+0000","last_peered":"2024-09-19T07:43:11.311619+0000","last_clean":"2024-09-19T07:43:11.311619+0000","last_became_active":"2024-09-19T07:43:05.265615+0000","last_became_peered":"2024-09-19T07:43:05.265615+0000","last_unstale":"2024-09-19T07:43:11.311619+0000","last_undegraded":"2024-09-19T07:43:11.311619+0000","last_fullsized":"2024-09-19T07:43:11.311619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:34.475024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310868+0000","last_change":"2024-09-19T07:43:10.283896+0000","last_active":"2024-09-19T07:43:11.310868+0000","last_peered":"2024-09-19T07:43:11.310868+0000","last_clean":"2024-09-19T07:43:11.310868+0000","last_became_active":"2024-09-19T07:43:10.283587+0000","last_became_peered":"2024-09-19T07:43:10.283587+0000","last_unstale":"2024-09-19T07:43:11.310868+0000","last_undegraded":"2024-09-19T07:43:11.310868+0000","last_fullsized":"2024-09-19T07:43:11.310868+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:41.387226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681270+0000","last_change":"2024-09-19T07:43:06.274315+0000","last_active":"2024-09-19T07:43:12.681270+0000","last_peered":"2024-09-19T07:43:12.681270+0000","last_clean":"2024-09-19T07:43:12.681270+0000","last_became_active":"2024-09-19T07:43:06.274064+0000","last_became_peered":"2024-09-19T07:43:06.274064+0000","last_unstale":"2024-09-19T07:43:12.681270+0000","last_undegraded":"2024-09-19T07:43:12.681270+0000","last_fullsized":"2024-09-19T07:43:12.681270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:22.804370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588519+0000","last_change":"2024-09-19T07:43:05.259083+0000","last_active":"2024-09-19T07:43:11.588519+0000","last_peered":"2024-09-19T07:43:11.588519+0000","last_clean":"2024-09-19T07:43:11.588519+0000","last_became_active":"2024-09-19T07:43:05.258769+0000","last_became_peered":"2024-09-19T07:43:05.258769+0000","last_unstale":"2024-09-19T07:43:11.588519+0000","last_undegraded":"2024-09-19T07:43:11.588519+0000","last_fullsized":"2024-09-19T07:43:11.588519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:51:24.580949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656048+0000","last_change":"2024-09-19T07:43:10.290261+0000","last_active":"2024-09-19T07:43:12.656048+0000","last_peered":"2024-09-19T07:43:12.656048+0000","last_clean":"2024-09-19T07:43:12.656048+0000","last_became_active":"2024-09-19T07:43:10.289428+0000","last_became_peered":"2024-09-19T07:43:10.289428+0000","last_unstale":"2024-09-19T07:43:12.656048+0000","last_undegraded":"2024-09-19T07:43:12.656048+0000","last_fullsized":"2024-09-19T07:43:12.656048+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:43.944005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450559+0000","last_change":"2024-09-19T07:43:06.276364+0000","last_active":"2024-09-19T07:43:11.450559+0000","last_peered":"2024-09-19T07:43:11.450559+0000","last_clean":"2024-09-19T07:43:11.450559+0000","last_became_active":"2024-09-19T07:43:06.275564+0000","last_became_peered":"2024-09-19T07:43:06.275564+0000","last_unstale":"2024-09-19T07:43:11.450559+0000","last_undegraded":"2024-09-19T07:43:11.450559+0000","last_fullsized":"2024-09-19T07:43:11.450559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450611+0000","last_change":"2024-09-19T07:43:05.264487+0000","last_active":"2024-09-19T07:43:11.450611+0000","last_peered":"2024-09-19T07:43:11.450611+0000","last_clean":"2024-09-19T07:43:11.450611+0000","last_became_active":"2024-09-19T07:43:05.264196+0000","last_became_peered":"2024-09-19T07:43:05.264196+0000","last_unstale":"2024-09-19T07:43:11.450611+0000","last_undegraded":"2024-09-19T07:43:11.450611+0000","last_fullsized":"2024-09-19T07:43:11.450611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:47.531702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311055+0000","last_change":"2024-09-19T07:43:10.481930+0000","last_active":"2024-09-19T07:43:11.311055+0000","last_peered":"2024-09-19T07:43:11.311055+0000","last_clean":"2024-09-19T07:43:11.311055+0000","last_became_active":"2024-09-19T07:43:10.481552+0000","last_became_peered":"2024-09-19T07:43:10.481552+0000","last_unstale":"2024-09-19T07:43:11.311055+0000","last_undegraded":"2024-09-19T07:43:11.311055+0000","last_fullsized":"2024-09-19T07:43:11.311055+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:19.523345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681307+0000","last_change":"2024-09-19T07:43:06.275131+0000","last_active":"2024-09-19T07:43:12.681307+0000","last_peered":"2024-09-19T07:43:12.681307+0000","last_clean":"2024-09-19T07:43:12.681307+0000","last_became_active":"2024-09-19T07:43:06.274857+0000","last_became_peered":"2024-09-19T07:43:06.274857+0000","last_unstale":"2024-09-19T07:43:12.681307+0000","last_undegraded":"2024-09-19T07:43:12.681307+0000","last_fullsized":"2024-09-19T07:43:12.681307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:57.928527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311569+0000","last_change":"2024-09-19T07:43:05.263838+0000","last_active":"2024-09-19T07:43:11.311569+0000","last_peered":"2024-09-19T07:43:11.311569+0000","last_clean":"2024-09-19T07:43:11.311569+0000","last_became_active":"2024-09-19T07:43:05.263484+0000","last_became_peered":"2024-09-19T07:43:05.263484+0000","last_unstale":"2024-09-19T07:43:11.311569+0000","last_undegraded":"2024-09-19T07:43:11.311569+0000","last_fullsized":"2024-09-19T07:43:11.311569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:38.098052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310652+0000","last_change":"2024-09-19T07:43:10.480636+0000","last_active":"2024-09-19T07:43:11.310652+0000","last_peered":"2024-09-19T07:43:11.310652+0000","last_clean":"2024-09-19T07:43:11.310652+0000","last_became_active":"2024-09-19T07:43:10.480217+0000","last_became_peered":"2024-09-19T07:43:10.480217+0000","last_unstale":"2024-09-19T07:43:11.310652+0000","last_undegraded":"2024-09-19T07:43:11.310652+0000","last_fullsized":"2024-09-19T07:43:11.310652+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:29.825328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656111+0000","last_change":"2024-09-19T07:43:06.272020+0000","last_active":"2024-09-19T07:43:12.656111+0000","last_peered":"2024-09-19T07:43:12.656111+0000","last_clean":"2024-09-19T07:43:12.656111+0000","last_became_active":"2024-09-19T07:43:06.271671+0000","last_became_peered":"2024-09-19T07:43:06.271671+0000","last_unstale":"2024-09-19T07:43:12.656111+0000","last_undegraded":"2024-09-19T07:43:12.656111+0000","last_fullsized":"2024-09-19T07:43:12.656111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:24:44.612992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656246+0000","last_change":"2024-09-19T07:43:05.265898+0000","last_active":"2024-09-19T07:43:12.656246+0000","last_peered":"2024-09-19T07:43:12.656246+0000","last_clean":"2024-09-19T07:43:12.656246+0000","last_became_active":"2024-09-19T07:43:05.265595+0000","last_became_peered":"2024-09-19T07:43:05.265595+0000","last_unstale":"2024-09-19T07:43:12.656246+0000","last_undegraded":"2024-09-19T07:43:12.656246+0000","last_fullsized":"2024-09-19T07:43:12.656246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:22.696534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311727+0000","last_change":"2024-09-19T07:43:10.482166+0000","last_active":"2024-09-19T07:43:11.311727+0000","last_peered":"2024-09-19T07:43:11.311727+0000","last_clean":"2024-09-19T07:43:11.311727+0000","last_became_active":"2024-09-19T07:43:10.481667+0000","last_became_peered":"2024-09-19T07:43:10.481667+0000","last_unstale":"2024-09-19T07:43:11.311727+0000","last_undegraded":"2024-09-19T07:43:11.311727+0000","last_fullsized":"2024-09-19T07:43:11.311727+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:10.437589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681470+0000","last_change":"2024-09-19T07:43:06.277189+0000","last_active":"2024-09-19T07:43:12.681470+0000","last_peered":"2024-09-19T07:43:12.681470+0000","last_clean":"2024-09-19T07:43:12.681470+0000","last_became_active":"2024-09-19T07:43:06.276924+0000","last_became_peered":"2024-09-19T07:43:06.276924+0000","last_unstale":"2024-09-19T07:43:12.681470+0000","last_undegraded":"2024-09-19T07:43:12.681470+0000","last_fullsized":"2024-09-19T07:43:12.681470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:39.658744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:15.766406+0000","last_change":"2024-09-19T07:43:05.266360+0000","last_active":"2024-09-19T07:43:15.766406+0000","last_peered":"2024-09-19T07:43:15.766406+0000","last_clean":"2024-09-19T07:43:15.766406+0000","last_became_active":"2024-09-19T07:43:05.266038+0000","last_became_peered":"2024-09-19T07:43:05.266038+0000","last_unstale":"2024-09-19T07:43:15.766406+0000","last_undegraded":"2024-09-19T07:43:15.766406+0000","last_fullsized":"2024-09-19T07:43:15.766406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:13.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680836+0000","last_change":"2024-09-19T07:43:10.295894+0000","last_active":"2024-09-19T07:43:12.680836+0000","last_peered":"2024-09-19T07:43:12.680836+0000","last_clean":"2024-09-19T07:43:12.680836+0000","last_became_active":"2024-09-19T07:43:10.295496+0000","last_became_peered":"2024-09-19T07:43:10.295496+0000","last_unstale":"2024-09-19T07:43:12.680836+0000","last_undegraded":"2024-09-19T07:43:12.680836+0000","last_fullsized":"2024-09-19T07:43:12.680836+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:18.372981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312405+0000","last_change":"2024-09-19T07:43:06.277639+0000","last_active":"2024-09-19T07:43:11.312405+0000","last_peered":"2024-09-19T07:43:11.312405+0000","last_clean":"2024-09-19T07:43:11.312405+0000","last_became_active":"2024-09-19T07:43:06.277186+0000","last_became_peered":"2024-09-19T07:43:06.277186+0000","last_unstale":"2024-09-19T07:43:11.312405+0000","last_undegraded":"2024-09-19T07:43:11.312405+0000","last_fullsized":"2024-09-19T07:43:11.312405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:07.895243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681210+0000","last_change":"2024-09-19T07:43:05.257416+0000","last_active":"2024-09-19T07:43:12.681210+0000","last_peered":"2024-09-19T07:43:12.681210+0000","last_clean":"2024-09-19T07:43:12.681210+0000","last_became_active":"2024-09-19T07:43:05.256941+0000","last_became_peered":"2024-09-19T07:43:05.256941+0000","last_unstale":"2024-09-19T07:43:12.681210+0000","last_undegraded":"2024-09-19T07:43:12.681210+0000","last_fullsized":"2024-09-19T07:43:12.681210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:44.359336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311217+0000","last_change":"2024-09-19T07:43:10.288931+0000","last_active":"2024-09-19T07:43:11.311217+0000","last_peered":"2024-09-19T07:43:11.311217+0000","last_clean":"2024-09-19T07:43:11.311217+0000","last_became_active":"2024-09-19T07:43:10.288548+0000","last_became_peered":"2024-09-19T07:43:10.288548+0000","last_unstale":"2024-09-19T07:43:11.311217+0000","last_undegraded":"2024-09-19T07:43:11.311217+0000","last_fullsized":"2024-09-19T07:43:11.311217+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:00.406076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656008+0000","last_change":"2024-09-19T07:43:06.270017+0000","last_active":"2024-09-19T07:43:12.656008+0000","last_peered":"2024-09-19T07:43:12.656008+0000","last_clean":"2024-09-19T07:43:12.656008+0000","last_became_active":"2024-09-19T07:43:06.269698+0000","last_became_peered":"2024-09-19T07:43:06.269698+0000","last_unstale":"2024-09-19T07:43:12.656008+0000","last_undegraded":"2024-09-19T07:43:12.656008+0000","last_fullsized":"2024-09-19T07:43:12.656008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:27.185190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311784+0000","last_change":"2024-09-19T07:43:05.265424+0000","last_active":"2024-09-19T07:43:11.311784+0000","last_peered":"2024-09-19T07:43:11.311784+0000","last_clean":"2024-09-19T07:43:11.311784+0000","last_became_active":"2024-09-19T07:43:05.265165+0000","last_became_peered":"2024-09-19T07:43:05.265165+0000","last_unstale":"2024-09-19T07:43:11.311784+0000","last_undegraded":"2024-09-19T07:43:11.311784+0000","last_fullsized":"2024-09-19T07:43:11.311784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:38:29.212025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310075+0000","last_change":"2024-09-19T07:43:10.290285+0000","last_active":"2024-09-19T07:43:11.310075+0000","last_peered":"2024-09-19T07:43:11.310075+0000","last_clean":"2024-09-19T07:43:11.310075+0000","last_became_active":"2024-09-19T07:43:10.289718+0000","last_became_peered":"2024-09-19T07:43:10.289718+0000","last_unstale":"2024-09-19T07:43:11.310075+0000","last_undegraded":"2024-09-19T07:43:11.310075+0000","last_fullsized":"2024-09-19T07:43:11.310075+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:08.693513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311662+0000","last_change":"2024-09-19T07:43:06.265419+0000","last_active":"2024-09-19T07:43:11.311662+0000","last_peered":"2024-09-19T07:43:11.311662+0000","last_clean":"2024-09-19T07:43:11.311662+0000","last_became_active":"2024-09-19T07:43:06.265170+0000","last_became_peered":"2024-09-19T07:43:06.265170+0000","last_unstale":"2024-09-19T07:43:11.311662+0000","last_undegraded":"2024-09-19T07:43:11.311662+0000","last_fullsized":"2024-09-19T07:43:11.311662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451157+0000","last_change":"2024-09-19T07:43:05.264095+0000","last_active":"2024-09-19T07:43:11.451157+0000","last_peered":"2024-09-19T07:43:11.451157+0000","last_clean":"2024-09-19T07:43:11.451157+0000","last_became_active":"2024-09-19T07:43:05.262895+0000","last_became_peered":"2024-09-19T07:43:05.262895+0000","last_unstale":"2024-09-19T07:43:11.451157+0000","last_undegraded":"2024-09-19T07:43:11.451157+0000","last_fullsized":"2024-09-19T07:43:11.451157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:38.098052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310066+0000","last_change":"2024-09-19T07:43:10.482892+0000","last_active":"2024-09-19T07:43:11.310066+0000","last_peered":"2024-09-19T07:43:11.310066+0000","last_clean":"2024-09-19T07:43:11.310066+0000","last_became_active":"2024-09-19T07:43:10.482523+0000","last_became_peered":"2024-09-19T07:43:10.482523+0000","last_unstale":"2024-09-19T07:43:11.310066+0000","last_undegraded":"2024-09-19T07:43:11.310066+0000","last_fullsized":"2024-09-19T07:43:11.310066+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:07.059894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655916+0000","last_change":"2024-09-19T07:43:10.480793+0000","last_active":"2024-09-19T07:43:12.655916+0000","last_peered":"2024-09-19T07:43:12.655916+0000","last_clean":"2024-09-19T07:43:12.655916+0000","last_became_active":"2024-09-19T07:43:10.480423+0000","last_became_peered":"2024-09-19T07:43:10.480423+0000","last_unstale":"2024-09-19T07:43:12.655916+0000","last_undegraded":"2024-09-19T07:43:12.655916+0000","last_fullsized":"2024-09-19T07:43:12.655916+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:18.372981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680526+0000","last_change":"2024-09-19T07:43:06.258001+0000","last_active":"2024-09-19T07:43:12.680526+0000","last_peered":"2024-09-19T07:43:12.680526+0000","last_clean":"2024-09-19T07:43:12.680526+0000","last_became_active":"2024-09-19T07:43:06.257731+0000","last_became_peered":"2024-09-19T07:43:06.257731+0000","last_unstale":"2024-09-19T07:43:12.680526+0000","last_undegraded":"2024-09-19T07:43:12.680526+0000","last_fullsized":"2024-09-19T07:43:12.680526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:14.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310662+0000","last_change":"2024-09-19T07:43:05.260911+0000","last_active":"2024-09-19T07:43:11.310662+0000","last_peered":"2024-09-19T07:43:11.310662+0000","last_clean":"2024-09-19T07:43:11.310662+0000","last_became_active":"2024-09-19T07:43:05.258827+0000","last_became_peered":"2024-09-19T07:43:05.258827+0000","last_unstale":"2024-09-19T07:43:11.310662+0000","last_undegraded":"2024-09-19T07:43:11.310662+0000","last_fullsized":"2024-09-19T07:43:11.310662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:13.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451139+0000","last_change":"2024-09-19T07:43:10.293630+0000","last_active":"2024-09-19T07:43:11.451139+0000","last_peered":"2024-09-19T07:43:11.451139+0000","last_clean":"2024-09-19T07:43:11.451139+0000","last_became_active":"2024-09-19T07:43:10.292979+0000","last_became_peered":"2024-09-19T07:43:10.292979+0000","last_unstale":"2024-09-19T07:43:11.451139+0000","last_undegraded":"2024-09-19T07:43:11.451139+0000","last_fullsized":"2024-09-19T07:43:11.451139+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:43.490603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310026+0000","last_change":"2024-09-19T07:43:06.274060+0000","last_active":"2024-09-19T07:43:11.310026+0000","last_peered":"2024-09-19T07:43:11.310026+0000","last_clean":"2024-09-19T07:43:11.310026+0000","last_became_active":"2024-09-19T07:43:06.272704+0000","last_became_peered":"2024-09-19T07:43:06.272704+0000","last_unstale":"2024-09-19T07:43:11.310026+0000","last_undegraded":"2024-09-19T07:43:11.310026+0000","last_fullsized":"2024-09-19T07:43:11.310026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:06.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310182+0000","last_change":"2024-09-19T07:43:05.264989+0000","last_active":"2024-09-19T07:43:11.310182+0000","last_peered":"2024-09-19T07:43:11.310182+0000","last_clean":"2024-09-19T07:43:11.310182+0000","last_became_active":"2024-09-19T07:43:05.264673+0000","last_became_peered":"2024-09-19T07:43:05.264673+0000","last_unstale":"2024-09-19T07:43:11.310182+0000","last_undegraded":"2024-09-19T07:43:11.310182+0000","last_fullsized":"2024-09-19T07:43:11.310182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:22.696534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450834+0000","last_change":"2024-09-19T07:43:05.262438+0000","last_active":"2024-09-19T07:43:11.450834+0000","last_peered":"2024-09-19T07:43:11.450834+0000","last_clean":"2024-09-19T07:43:11.450834+0000","last_became_active":"2024-09-19T07:43:05.262129+0000","last_became_peered":"2024-09-19T07:43:05.262129+0000","last_unstale":"2024-09-19T07:43:11.450834+0000","last_undegraded":"2024-09-19T07:43:11.450834+0000","last_fullsized":"2024-09-19T07:43:11.450834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:44.359336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656191+0000","last_change":"2024-09-19T07:43:06.270402+0000","last_active":"2024-09-19T07:43:12.656191+0000","last_peered":"2024-09-19T07:43:12.656191+0000","last_clean":"2024-09-19T07:43:12.656191+0000","last_became_active":"2024-09-19T07:43:06.270087+0000","last_became_peered":"2024-09-19T07:43:06.270087+0000","last_unstale":"2024-09-19T07:43:12.656191+0000","last_undegraded":"2024-09-19T07:43:12.656191+0000","last_fullsized":"2024-09-19T07:43:12.656191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588782+0000","last_change":"2024-09-19T07:43:10.288597+0000","last_active":"2024-09-19T07:43:11.588782+0000","last_peered":"2024-09-19T07:43:11.588782+0000","last_clean":"2024-09-19T07:43:11.588782+0000","last_became_active":"2024-09-19T07:43:10.288207+0000","last_became_peered":"2024-09-19T07:43:10.288207+0000","last_unstale":"2024-09-19T07:43:11.588782+0000","last_undegraded":"2024-09-19T07:43:11.588782+0000","last_fullsized":"2024-09-19T07:43:11.588782+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:43.318582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310346+0000","last_change":"2024-09-19T07:43:05.261298+0000","last_active":"2024-09-19T07:43:11.310346+0000","last_peered":"2024-09-19T07:43:11.310346+0000","last_clean":"2024-09-19T07:43:11.310346+0000","last_became_active":"2024-09-19T07:43:05.261000+0000","last_became_peered":"2024-09-19T07:43:05.261000+0000","last_unstale":"2024-09-19T07:43:11.310346+0000","last_undegraded":"2024-09-19T07:43:11.310346+0000","last_fullsized":"2024-09-19T07:43:11.310346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:38.098052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310888+0000","last_change":"2024-09-19T07:43:06.267976+0000","last_active":"2024-09-19T07:43:11.310888+0000","last_peered":"2024-09-19T07:43:11.310888+0000","last_clean":"2024-09-19T07:43:11.310888+0000","last_became_active":"2024-09-19T07:43:06.267214+0000","last_became_peered":"2024-09-19T07:43:06.267214+0000","last_unstale":"2024-09-19T07:43:11.310888+0000","last_undegraded":"2024-09-19T07:43:11.310888+0000","last_fullsized":"2024-09-19T07:43:11.310888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:48.537241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310742+0000","last_change":"2024-09-19T07:43:10.288619+0000","last_active":"2024-09-19T07:43:11.310742+0000","last_peered":"2024-09-19T07:43:11.310742+0000","last_clean":"2024-09-19T07:43:11.310742+0000","last_became_active":"2024-09-19T07:43:10.288213+0000","last_became_peered":"2024-09-19T07:43:10.288213+0000","last_unstale":"2024-09-19T07:43:11.310742+0000","last_undegraded":"2024-09-19T07:43:11.310742+0000","last_fullsized":"2024-09-19T07:43:11.310742+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:30.165579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450437+0000","last_change":"2024-09-19T07:43:06.276792+0000","last_active":"2024-09-19T07:43:11.450437+0000","last_peered":"2024-09-19T07:43:11.450437+0000","last_clean":"2024-09-19T07:43:11.450437+0000","last_became_active":"2024-09-19T07:43:06.276518+0000","last_became_peered":"2024-09-19T07:43:06.276518+0000","last_unstale":"2024-09-19T07:43:11.450437+0000","last_undegraded":"2024-09-19T07:43:11.450437+0000","last_fullsized":"2024-09-19T07:43:11.450437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:12.618593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588673+0000","last_change":"2024-09-19T07:43:06.268099+0000","last_active":"2024-09-19T07:43:11.588673+0000","last_peered":"2024-09-19T07:43:11.588673+0000","last_clean":"2024-09-19T07:43:11.588673+0000","last_became_active":"2024-09-19T07:43:06.267803+0000","last_became_peered":"2024-09-19T07:43:06.267803+0000","last_unstale":"2024-09-19T07:43:11.588673+0000","last_undegraded":"2024-09-19T07:43:11.588673+0000","last_fullsized":"2024-09-19T07:43:11.588673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:31.966840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311465+0000","last_change":"2024-09-19T07:43:06.263757+0000","last_active":"2024-09-19T07:43:11.311465+0000","last_peered":"2024-09-19T07:43:11.311465+0000","last_clean":"2024-09-19T07:43:11.311465+0000","last_became_active":"2024-09-19T07:43:06.263480+0000","last_became_peered":"2024-09-19T07:43:06.263480+0000","last_unstale":"2024-09-19T07:43:11.311465+0000","last_undegraded":"2024-09-19T07:43:11.311465+0000","last_fullsized":"2024-09-19T07:43:11.311465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:39.918269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656177+0000","last_change":"2024-09-19T07:43:06.278694+0000","last_active":"2024-09-19T07:43:12.656177+0000","last_peered":"2024-09-19T07:43:12.656177+0000","last_clean":"2024-09-19T07:43:12.656177+0000","last_became_active":"2024-09-19T07:43:06.277280+0000","last_became_peered":"2024-09-19T07:43:06.277280+0000","last_unstale":"2024-09-19T07:43:12.656177+0000","last_undegraded":"2024-09-19T07:43:12.656177+0000","last_fullsized":"2024-09-19T07:43:12.656177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:51.595301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310764+0000","last_change":"2024-09-19T07:43:06.272967+0000","last_active":"2024-09-19T07:43:11.310764+0000","last_peered":"2024-09-19T07:43:11.310764+0000","last_clean":"2024-09-19T07:43:11.310764+0000","last_became_active":"2024-09-19T07:43:06.272476+0000","last_became_peered":"2024-09-19T07:43:06.272476+0000","last_unstale":"2024-09-19T07:43:11.310764+0000","last_undegraded":"2024-09-19T07:43:11.310764+0000","last_fullsized":"2024-09-19T07:43:11.310764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:09.222719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681113+0000","last_change":"2024-09-19T07:43:06.260655+0000","last_active":"2024-09-19T07:43:12.681113+0000","last_peered":"2024-09-19T07:43:12.681113+0000","last_clean":"2024-09-19T07:43:12.681113+0000","last_became_active":"2024-09-19T07:43:06.260388+0000","last_became_peered":"2024-09-19T07:43:06.260388+0000","last_unstale":"2024-09-19T07:43:12.681113+0000","last_undegraded":"2024-09-19T07:43:12.681113+0000","last_fullsized":"2024-09-19T07:43:12.681113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451084+0000","last_change":"2024-09-19T07:43:06.270719+0000","last_active":"2024-09-19T07:43:11.451084+0000","last_peered":"2024-09-19T07:43:11.451084+0000","last_clean":"2024-09-19T07:43:11.451084+0000","last_became_active":"2024-09-19T07:43:06.270022+0000","last_became_peered":"2024-09-19T07:43:06.270022+0000","last_unstale":"2024-09-19T07:43:11.451084+0000","last_undegraded":"2024-09-19T07:43:11.451084+0000","last_fullsized":"2024-09-19T07:43:11.451084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:27.185190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310886+0000","last_change":"2024-09-19T07:43:06.266798+0000","last_active":"2024-09-19T07:43:11.310886+0000","last_peered":"2024-09-19T07:43:11.310886+0000","last_clean":"2024-09-19T07:43:11.310886+0000","last_became_active":"2024-09-19T07:43:06.266461+0000","last_became_peered":"2024-09-19T07:43:06.266461+0000","last_unstale":"2024-09-19T07:43:11.310886+0000","last_undegraded":"2024-09-19T07:43:11.310886+0000","last_fullsized":"2024-09-19T07:43:11.310886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:50.544269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310636+0000","last_change":"2024-09-19T07:43:06.263830+0000","last_active":"2024-09-19T07:43:11.310636+0000","last_peered":"2024-09-19T07:43:11.310636+0000","last_clean":"2024-09-19T07:43:11.310636+0000","last_became_active":"2024-09-19T07:43:06.263539+0000","last_became_peered":"2024-09-19T07:43:06.263539+0000","last_unstale":"2024-09-19T07:43:11.310636+0000","last_undegraded":"2024-09-19T07:43:11.310636+0000","last_fullsized":"2024-09-19T07:43:11.310636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:50.544269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310166+0000","last_change":"2024-09-19T07:43:06.276371+0000","last_active":"2024-09-19T07:43:11.310166+0000","last_peered":"2024-09-19T07:43:11.310166+0000","last_clean":"2024-09-19T07:43:11.310166+0000","last_became_active":"2024-09-19T07:43:06.276103+0000","last_became_peered":"2024-09-19T07:43:06.276103+0000","last_unstale":"2024-09-19T07:43:11.310166+0000","last_undegraded":"2024-09-19T07:43:11.310166+0000","last_fullsized":"2024-09-19T07:43:11.310166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:51.595301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311371+0000","last_change":"2024-09-19T07:43:06.265375+0000","last_active":"2024-09-19T07:43:11.311371+0000","last_peered":"2024-09-19T07:43:11.311371+0000","last_clean":"2024-09-19T07:43:11.311371+0000","last_became_active":"2024-09-19T07:43:06.265118+0000","last_became_peered":"2024-09-19T07:43:06.265118+0000","last_unstale":"2024-09-19T07:43:11.311371+0000","last_undegraded":"2024-09-19T07:43:11.311371+0000","last_fullsized":"2024-09-19T07:43:11.311371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:14.347245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655877+0000","last_change":"2024-09-19T07:43:06.279273+0000","last_active":"2024-09-19T07:43:12.655877+0000","last_peered":"2024-09-19T07:43:12.655877+0000","last_clean":"2024-09-19T07:43:12.655877+0000","last_became_active":"2024-09-19T07:43:06.278276+0000","last_became_peered":"2024-09-19T07:43:06.278276+0000","last_unstale":"2024-09-19T07:43:12.655877+0000","last_undegraded":"2024-09-19T07:43:12.655877+0000","last_fullsized":"2024-09-19T07:43:12.655877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:14.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450830+0000","last_change":"2024-09-19T07:43:06.268360+0000","last_active":"2024-09-19T07:43:11.450830+0000","last_peered":"2024-09-19T07:43:11.450830+0000","last_clean":"2024-09-19T07:43:11.450830+0000","last_became_active":"2024-09-19T07:43:06.268096+0000","last_became_peered":"2024-09-19T07:43:06.268096+0000","last_unstale":"2024-09-19T07:43:11.450830+0000","last_undegraded":"2024-09-19T07:43:11.450830+0000","last_fullsized":"2024-09-19T07:43:11.450830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681729+0000","last_change":"2024-09-19T07:43:06.272110+0000","last_active":"2024-09-19T07:43:12.681729+0000","last_peered":"2024-09-19T07:43:12.681729+0000","last_clean":"2024-09-19T07:43:12.681729+0000","last_became_active":"2024-09-19T07:43:06.271804+0000","last_became_peered":"2024-09-19T07:43:06.271804+0000","last_unstale":"2024-09-19T07:43:12.681729+0000","last_undegraded":"2024-09-19T07:43:12.681729+0000","last_fullsized":"2024-09-19T07:43:12.681729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:23.113048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588806+0000","last_change":"2024-09-19T07:43:06.260936+0000","last_active":"2024-09-19T07:43:11.588806+0000","last_peered":"2024-09-19T07:43:11.588806+0000","last_clean":"2024-09-19T07:43:11.588806+0000","last_became_active":"2024-09-19T07:43:06.260663+0000","last_became_peered":"2024-09-19T07:43:06.260663+0000","last_unstale":"2024-09-19T07:43:11.588806+0000","last_undegraded":"2024-09-19T07:43:11.588806+0000","last_fullsized":"2024-09-19T07:43:11.588806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:06.425112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312242+0000","last_change":"2024-09-19T07:43:06.268104+0000","last_active":"2024-09-19T07:43:11.312242+0000","last_peered":"2024-09-19T07:43:11.312242+0000","last_clean":"2024-09-19T07:43:11.312242+0000","last_became_active":"2024-09-19T07:43:06.267830+0000","last_became_peered":"2024-09-19T07:43:06.267830+0000","last_unstale":"2024-09-19T07:43:11.312242+0000","last_undegraded":"2024-09-19T07:43:11.312242+0000","last_fullsized":"2024-09-19T07:43:11.312242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:22.804370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310551+0000","last_change":"2024-09-19T07:43:06.269948+0000","last_active":"2024-09-19T07:43:11.310551+0000","last_peered":"2024-09-19T07:43:11.310551+0000","last_clean":"2024-09-19T07:43:11.310551+0000","last_became_active":"2024-09-19T07:43:06.269684+0000","last_became_peered":"2024-09-19T07:43:06.269684+0000","last_unstale":"2024-09-19T07:43:11.310551+0000","last_undegraded":"2024-09-19T07:43:11.310551+0000","last_fullsized":"2024-09-19T07:43:11.310551+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680603+0000","last_change":"2024-09-19T07:43:06.269924+0000","last_active":"2024-09-19T07:43:12.680603+0000","last_peered":"2024-09-19T07:43:12.680603+0000","last_clean":"2024-09-19T07:43:12.680603+0000","last_became_active":"2024-09-19T07:43:06.269602+0000","last_became_peered":"2024-09-19T07:43:06.269602+0000","last_unstale":"2024-09-19T07:43:12.680603+0000","last_undegraded":"2024-09-19T07:43:12.680603+0000","last_fullsized":"2024-09-19T07:43:12.680603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:51.595301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655995+0000","last_change":"2024-09-19T07:43:06.271422+0000","last_active":"2024-09-19T07:43:12.655995+0000","last_peered":"2024-09-19T07:43:12.655995+0000","last_clean":"2024-09-19T07:43:12.655995+0000","last_became_active":"2024-09-19T07:43:06.271096+0000","last_became_peered":"2024-09-19T07:43:06.271096+0000","last_unstale":"2024-09-19T07:43:12.655995+0000","last_undegraded":"2024-09-19T07:43:12.655995+0000","last_fullsized":"2024-09-19T07:43:12.655995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:23.743784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310215+0000","last_change":"2024-09-19T07:43:06.264090+0000","last_active":"2024-09-19T07:43:11.310215+0000","last_peered":"2024-09-19T07:43:11.310215+0000","last_clean":"2024-09-19T07:43:11.310215+0000","last_became_active":"2024-09-19T07:43:06.262071+0000","last_became_peered":"2024-09-19T07:43:06.262071+0000","last_unstale":"2024-09-19T07:43:11.310215+0000","last_undegraded":"2024-09-19T07:43:11.310215+0000","last_fullsized":"2024-09-19T07:43:11.310215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312316+0000","last_change":"2024-09-19T07:43:06.265567+0000","last_active":"2024-09-19T07:43:11.312316+0000","last_peered":"2024-09-19T07:43:11.312316+0000","last_clean":"2024-09-19T07:43:11.312316+0000","last_became_active":"2024-09-19T07:43:06.265290+0000","last_became_peered":"2024-09-19T07:43:06.265290+0000","last_unstale":"2024-09-19T07:43:11.312316+0000","last_undegraded":"2024-09-19T07:43:11.312316+0000","last_fullsized":"2024-09-19T07:43:11.312316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:14.597733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450953+0000","last_change":"2024-09-19T07:43:06.267949+0000","last_active":"2024-09-19T07:43:11.450953+0000","last_peered":"2024-09-19T07:43:11.450953+0000","last_clean":"2024-09-19T07:43:11.450953+0000","last_became_active":"2024-09-19T07:43:06.267640+0000","last_became_peered":"2024-09-19T07:43:06.267640+0000","last_unstale":"2024-09-19T07:43:11.450953+0000","last_undegraded":"2024-09-19T07:43:11.450953+0000","last_fullsized":"2024-09-19T07:43:11.450953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:14.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451030+0000","last_change":"2024-09-19T07:43:06.268718+0000","last_active":"2024-09-19T07:43:11.451030+0000","last_peered":"2024-09-19T07:43:11.451030+0000","last_clean":"2024-09-19T07:43:11.451030+0000","last_became_active":"2024-09-19T07:43:06.268398+0000","last_became_peered":"2024-09-19T07:43:06.268398+0000","last_unstale":"2024-09-19T07:43:11.451030+0000","last_undegraded":"2024-09-19T07:43:11.451030+0000","last_fullsized":"2024-09-19T07:43:11.451030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:51.595301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681743+0000","last_change":"2024-09-19T07:43:06.273520+0000","last_active":"2024-09-19T07:43:12.681743+0000","last_peered":"2024-09-19T07:43:12.681743+0000","last_clean":"2024-09-19T07:43:12.681743+0000","last_became_active":"2024-09-19T07:43:06.273150+0000","last_became_peered":"2024-09-19T07:43:06.273150+0000","last_unstale":"2024-09-19T07:43:12.681743+0000","last_undegraded":"2024-09-19T07:43:12.681743+0000","last_fullsized":"2024-09-19T07:43:12.681743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:39.464868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.309860+0000","last_change":"2024-09-19T07:43:06.272990+0000","last_active":"2024-09-19T07:43:11.309860+0000","last_peered":"2024-09-19T07:43:11.309860+0000","last_clean":"2024-09-19T07:43:11.309860+0000","last_became_active":"2024-09-19T07:43:06.272323+0000","last_became_peered":"2024-09-19T07:43:06.272323+0000","last_unstale":"2024-09-19T07:43:11.309860+0000","last_undegraded":"2024-09-19T07:43:11.309860+0000","last_fullsized":"2024-09-19T07:43:11.309860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:14.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681143+0000","last_change":"2024-09-19T07:43:06.273648+0000","last_active":"2024-09-19T07:43:12.681143+0000","last_peered":"2024-09-19T07:43:12.681143+0000","last_clean":"2024-09-19T07:43:12.681143+0000","last_became_active":"2024-09-19T07:43:06.273387+0000","last_became_peered":"2024-09-19T07:43:06.273387+0000","last_unstale":"2024-09-19T07:43:12.681143+0000","last_undegraded":"2024-09-19T07:43:12.681143+0000","last_fullsized":"2024-09-19T07:43:12.681143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:56.380341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311270+0000","last_change":"2024-09-19T07:43:06.274896+0000","last_active":"2024-09-19T07:43:11.311270+0000","last_peered":"2024-09-19T07:43:11.311270+0000","last_clean":"2024-09-19T07:43:11.311270+0000","last_became_active":"2024-09-19T07:43:06.274551+0000","last_became_peered":"2024-09-19T07:43:06.274551+0000","last_unstale":"2024-09-19T07:43:11.311270+0000","last_undegraded":"2024-09-19T07:43:11.311270+0000","last_fullsized":"2024-09-19T07:43:11.311270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:39.658744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681148+0000","last_change":"2024-09-19T07:43:06.271789+0000","last_active":"2024-09-19T07:43:12.681148+0000","last_peered":"2024-09-19T07:43:12.681148+0000","last_clean":"2024-09-19T07:43:12.681148+0000","last_became_active":"2024-09-19T07:43:06.269978+0000","last_became_peered":"2024-09-19T07:43:06.269978+0000","last_unstale":"2024-09-19T07:43:12.681148+0000","last_undegraded":"2024-09-19T07:43:12.681148+0000","last_fullsized":"2024-09-19T07:43:12.681148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:04.168224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311913+0000","last_change":"2024-09-19T07:43:06.274164+0000","last_active":"2024-09-19T07:43:11.311913+0000","last_peered":"2024-09-19T07:43:11.311913+0000","last_clean":"2024-09-19T07:43:11.311913+0000","last_became_active":"2024-09-19T07:43:06.273730+0000","last_became_peered":"2024-09-19T07:43:06.273730+0000","last_unstale":"2024-09-19T07:43:11.311913+0000","last_undegraded":"2024-09-19T07:43:11.311913+0000","last_fullsized":"2024-09-19T07:43:11.311913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:56.380341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655874+0000","last_change":"2024-09-19T07:43:06.266043+0000","last_active":"2024-09-19T07:43:12.655874+0000","last_peered":"2024-09-19T07:43:12.655874+0000","last_clean":"2024-09-19T07:43:12.655874+0000","last_became_active":"2024-09-19T07:43:06.265717+0000","last_became_peered":"2024-09-19T07:43:06.265717+0000","last_unstale":"2024-09-19T07:43:12.655874+0000","last_undegraded":"2024-09-19T07:43:12.655874+0000","last_fullsized":"2024-09-19T07:43:12.655874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:12.618593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681153+0000","last_change":"2024-09-19T07:43:06.270945+0000","last_active":"2024-09-19T07:43:12.681153+0000","last_peered":"2024-09-19T07:43:12.681153+0000","last_clean":"2024-09-19T07:43:12.681153+0000","last_became_active":"2024-09-19T07:43:06.270641+0000","last_became_peered":"2024-09-19T07:43:06.270641+0000","last_unstale":"2024-09-19T07:43:12.681153+0000","last_undegraded":"2024-09-19T07:43:12.681153+0000","last_fullsized":"2024-09-19T07:43:12.681153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:27.185190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681632+0000","last_change":"2024-09-19T07:43:06.277928+0000","last_active":"2024-09-19T07:43:12.681632+0000","last_peered":"2024-09-19T07:43:12.681632+0000","last_clean":"2024-09-19T07:43:12.681632+0000","last_became_active":"2024-09-19T07:43:06.277632+0000","last_became_peered":"2024-09-19T07:43:06.277632+0000","last_unstale":"2024-09-19T07:43:12.681632+0000","last_undegraded":"2024-09-19T07:43:12.681632+0000","last_fullsized":"2024-09-19T07:43:12.681632+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:09.222719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":868352,"data_stored":697396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":110290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":106536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":106572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":876544,"data_stored":697452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":282624,"data_stored":110290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":278528,"data_stored":106606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":270336,"data_stored":106446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:43:26.687 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T07:43:26.687 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T07:43:26.687 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-19T07:43:26.688 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-19T07:43:26.688 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-19T07:43:26.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.825+0000 7f77c30d0640 1 Processor -- start 2024-09-19T07:43:26.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.825+0000 7f77c30d0640 1 -- start start 2024-09-19T07:43:26.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.826+0000 7f77c30d0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f77bc145a20 0x7f77bc0b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:26.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.826+0000 7f77c30d0640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f77bc06c320 0x7f77bc0b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:26.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.826+0000 7f77c30d0640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77bc06ca00 con 0x7f77bc0b0460 2024-09-19T07:43:26.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.826+0000 7f77c30d0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f77bc0b1570 con 0x7f77bc145a20 2024-09-19T07:43:26.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.826+0000 7f77c30d0640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f77bc0b0960 con 0x7f77bc06c320 2024-09-19T07:43:26.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.826+0000 7f77c0e6c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f77bc145a20 0x7f77bc0b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:26.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.826+0000 7f77c0e6c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f77bc145a20 0x7f77bc0b6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57266/0 (socket says 172.21.15.16:57266) 2024-09-19T07:43:26.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.826+0000 7f77c0e6c640 1 -- 172.21.15.16:0/722086495 learned_addr learned my addr 172.21.15.16:0/722086495 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:26.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.826+0000 7f77c166d640 1 --2- 172.21.15.16:0/722086495 >> v2:172.21.15.146:3300/0 conn(0x7f77bc06c320 0x7f77bc0b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:26.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.827+0000 7f77c0e6c640 1 -- 172.21.15.16:0/722086495 >> v2:172.21.15.146:3300/0 conn(0x7f77bc06c320 msgr2=0x7f77bc0b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:26.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.827+0000 7f77c0e6c640 1 --2- 172.21.15.16:0/722086495 >> v2:172.21.15.146:3300/0 conn(0x7f77bc06c320 0x7f77bc0b1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.827+0000 7f77c0e6c640 1 -- 172.21.15.16:0/722086495 >> v1:172.21.15.146:6789/0 conn(0x7f77bc0b0460 legacy=0x7f77bc06bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:26.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.827+0000 7f77c0e6c640 1 -- 172.21.15.16:0/722086495 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77bc0b0c40 con 0x7f77bc145a20 2024-09-19T07:43:26.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.827+0000 7f77c166d640 1 --2- 172.21.15.16:0/722086495 >> v2:172.21.15.146:3300/0 conn(0x7f77bc06c320 0x7f77bc0b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:26.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.827+0000 7f77c0e6c640 1 --2- 172.21.15.16:0/722086495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77bc145a20 0x7f77bc0b6cb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0b16f0 tx=0x7f77b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449844e9bf48b4f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:26.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.827+0000 7f77b37fe640 1 -- 172.21.15.16:0/722086495 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8037070 con 0x7f77bc145a20 2024-09-19T07:43:26.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.828+0000 7f77b37fe640 1 -- 172.21.15.16:0/722086495 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77b8032440 con 0x7f77bc145a20 2024-09-19T07:43:26.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.828+0000 7f77b37fe640 1 -- 172.21.15.16:0/722086495 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8032810 con 0x7f77bc145a20 2024-09-19T07:43:26.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.828+0000 7f77c30d0640 1 -- 172.21.15.16:0/722086495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77bc145a20 msgr2=0x7f77bc0b6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:26.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.828+0000 7f77c30d0640 1 --2- 172.21.15.16:0/722086495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77bc145a20 0x7f77bc0b6cb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0b16f0 tx=0x7f77b802a270 comp rx=0 tx=0).stop 2024-09-19T07:43:26.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.828+0000 7f77c30d0640 1 -- 172.21.15.16:0/722086495 shutdown_connections 2024-09-19T07:43:26.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.828+0000 7f77c30d0640 1 --2- 172.21.15.16:0/722086495 >> v2:172.21.15.146:3300/0 conn(0x7f77bc06c320 0x7f77bc0b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.828+0000 7f77c30d0640 1 --2- 172.21.15.16:0/722086495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77bc145a20 0x7f77bc0b6cb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.828+0000 7f77c30d0640 1 -- 172.21.15.16:0/722086495 >> 172.21.15.16:0/722086495 conn(0x7f77bc09a9f0 msgr2=0x7f77bc0b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:26.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.828+0000 7f77c30d0640 1 -- 172.21.15.16:0/722086495 shutdown_connections 2024-09-19T07:43:26.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.828+0000 7f77c30d0640 1 -- 172.21.15.16:0/722086495 wait complete. 2024-09-19T07:43:26.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.829+0000 7f77c30d0640 1 Processor -- start 2024-09-19T07:43:26.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.829+0000 7f77c30d0640 1 -- start start 2024-09-19T07:43:26.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.830+0000 7f77c30d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77bc06c320 0x7f77bc13eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:26.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.830+0000 7f77c30d0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77bc0b0460 0x7f77bc0cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:26.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.830+0000 7f77c30d0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77bc145a20 0x7f77bc15aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:26.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.830+0000 7f77c30d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc145e00 con 0x7f77bc06c320 2024-09-19T07:43:26.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.830+0000 7f77c30d0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc0b1200 con 0x7f77bc145a20 2024-09-19T07:43:26.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.830+0000 7f77c30d0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f77bc145f80 con 0x7f77bc0b0460 2024-09-19T07:43:26.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.830+0000 7f77c0e6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77bc06c320 0x7f77bc13eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:26.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.830+0000 7f77c0e6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77bc06c320 0x7f77bc13eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57268/0 (socket says 172.21.15.16:57268) 2024-09-19T07:43:26.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.830+0000 7f77c0e6c640 1 -- 172.21.15.16:0/420792137 learned_addr learned my addr 172.21.15.16:0/420792137 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:26.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.830+0000 7f77b3fff640 1 --2- 172.21.15.16:0/420792137 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77bc0b0460 0x7f77bc0cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:26.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.830+0000 7f77c166d640 1 --2- 172.21.15.16:0/420792137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77bc145a20 0x7f77bc15aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:26.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.831+0000 7f77c0e6c640 1 -- 172.21.15.16:0/420792137 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77bc0b0460 msgr2=0x7f77bc0cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:26.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.831+0000 7f77c0e6c640 1 --2- 172.21.15.16:0/420792137 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77bc0b0460 0x7f77bc0cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.831+0000 7f77c0e6c640 1 -- 172.21.15.16:0/420792137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77bc145a20 msgr2=0x7f77bc15aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:26.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.831+0000 7f77c0e6c640 1 --2- 172.21.15.16:0/420792137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77bc145a20 0x7f77bc15aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:26.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.831+0000 7f77c0e6c640 1 -- 172.21.15.16:0/420792137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77bc13f520 con 0x7f77bc06c320 2024-09-19T07:43:26.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.831+0000 7f77c0e6c640 1 --2- 172.21.15.16:0/420792137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77bc06c320 0x7f77bc13eeb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f77b8000c00 tx=0x7f77b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:26.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.831+0000 7f77b17fa640 1 -- 172.21.15.16:0/420792137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8041020 con 0x7f77bc06c320 2024-09-19T07:43:26.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.831+0000 7f77b17fa640 1 -- 172.21.15.16:0/420792137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77b80306c0 con 0x7f77bc06c320 2024-09-19T07:43:26.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.831+0000 7f77c30d0640 1 -- 172.21.15.16:0/420792137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77bc13f800 con 0x7f77bc06c320 2024-09-19T07:43:26.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.831+0000 7f77b17fa640 1 -- 172.21.15.16:0/420792137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8037040 con 0x7f77bc06c320 2024-09-19T07:43:26.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.831+0000 7f77c30d0640 1 -- 172.21.15.16:0/420792137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77bc13fdc0 con 0x7f77bc06c320 2024-09-19T07:43:26.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.832+0000 7f77c30d0640 1 -- 172.21.15.16:0/420792137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77bc0b6cb0 con 0x7f77bc06c320 2024-09-19T07:43:26.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.833+0000 7f77b17fa640 1 -- 172.21.15.16:0/420792137 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f77b8044020 con 0x7f77bc06c320 2024-09-19T07:43:26.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.833+0000 7f77b17fa640 1 --2- 172.21.15.16:0/420792137 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f77a806c2b0 0x7f77a806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:26.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.833+0000 7f77b17fa640 1 -- 172.21.15.16:0/420792137 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f77b80abdd0 con 0x7f77bc06c320 2024-09-19T07:43:26.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.834+0000 7f77b3fff640 1 --2- 172.21.15.16:0/420792137 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f77a806c2b0 0x7f77a806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:26.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.834+0000 7f77b3fff640 1 --2- 172.21.15.16:0/420792137 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f77a806c2b0 0x7f77a806e770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f77bc15c5e0 tx=0x7f77ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:26.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:26.838+0000 7f77b17fa640 1 -- 172.21.15.16:0/420792137 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f77b804b030 con 0x7f77bc06c320 2024-09-19T07:43:27.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.060+0000 7f77c30d0640 1 -- 172.21.15.16:0/420792137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f77bc0d0c10 con 0x7f77bc06c320 2024-09-19T07:43:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.064+0000 7f77b17fa640 1 -- 172.21.15.16:0/420792137 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+173507 (secure 0 0 0) 0x7f77b80ad090 con 0x7f77bc06c320 2024-09-19T07:43:27.068 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.066+0000 7f77c30d0640 1 -- 172.21.15.16:0/420792137 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f77a806c2b0 msgr2=0x7f77a806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.066+0000 7f77c30d0640 1 --2- 172.21.15.16:0/420792137 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f77a806c2b0 0x7f77a806e770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f77bc15c5e0 tx=0x7f77ac009040 comp rx=0 tx=0).stop 2024-09-19T07:43:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.066+0000 7f77c30d0640 1 -- 172.21.15.16:0/420792137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77bc06c320 msgr2=0x7f77bc13eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.066+0000 7f77c30d0640 1 --2- 172.21.15.16:0/420792137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77bc06c320 0x7f77bc13eeb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f77b8000c00 tx=0x7f77b80046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.066+0000 7f77c30d0640 1 -- 172.21.15.16:0/420792137 shutdown_connections 2024-09-19T07:43:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.066+0000 7f77c30d0640 1 --2- 172.21.15.16:0/420792137 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f77a806c2b0 0x7f77a806e770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.066+0000 7f77c30d0640 1 --2- 172.21.15.16:0/420792137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77bc145a20 0x7f77bc15aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.066+0000 7f77c30d0640 1 --2- 172.21.15.16:0/420792137 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77bc0b0460 0x7f77bc0cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.066+0000 7f77c30d0640 1 --2- 172.21.15.16:0/420792137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77bc06c320 0x7f77bc13eeb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.066+0000 7f77c30d0640 1 -- 172.21.15.16:0/420792137 >> 172.21.15.16:0/420792137 conn(0x7f77bc09a9f0 msgr2=0x7f77bc0b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.066+0000 7f77c30d0640 1 -- 172.21.15.16:0/420792137 shutdown_connections 2024-09-19T07:43:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.066+0000 7f77c30d0640 1 -- 172.21.15.16:0/420792137 wait complete. 2024-09-19T07:43:27.078 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":413611562},{"type":"v1","addr":"172.21.15.146:6833","nonce":413611562}]},"active_addr":"172.21.15.146:6833/413611562","active_change":"2024-09-19T07:42:46.443918+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4100,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.146:0","nonce":3429828922}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.146:0","nonce":3774536188}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.146:0","nonce":922274030}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.146:0","nonce":381747868}]}]} 2024-09-19T07:43:27.083 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-19T07:43:27.083 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-19T07:43:27.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:43:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.185+0000 7fc024591640 1 Processor -- start 2024-09-19T07:43:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.185+0000 7fc024591640 1 -- start start 2024-09-19T07:43:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.186+0000 7fc024591640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc01c06c320 0x7fc01c15d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.186+0000 7fc024591640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c0b6c40 0x7fc01c15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.186+0000 7fc024591640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc01c15dae0 con 0x7fc01c145ec0 2024-09-19T07:43:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.186+0000 7fc024591640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc01c15d8b0 con 0x7fc01c06c320 2024-09-19T07:43:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.186+0000 7fc024591640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc01c15ddc0 con 0x7fc01c0b6c40 2024-09-19T07:43:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.186+0000 7fc022b2e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c0b6c40 0x7fc01c15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.186+0000 7fc021b2c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc01c06c320 0x7fc01c15d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.186+0000 7fc02232d640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fc01c145ec0 0x7fc01c06bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:56384/0 (socket says 172.21.15.16:56384) 2024-09-19T07:43:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.186+0000 7fc022b2e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c0b6c40 0x7fc01c15e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:33988/0 (socket says 172.21.15.16:33988) 2024-09-19T07:43:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.186+0000 7fc02232d640 1 -- 172.21.15.16:0/1027355822 learned_addr learned my addr 172.21.15.16:0/1027355822 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.186+0000 7fc022b2e640 1 -- 172.21.15.16:0/1027355822 >> v2:172.21.15.146:3300/0 conn(0x7fc01c06c320 msgr2=0x7fc01c15d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.186+0000 7fc022b2e640 1 --2- 172.21.15.16:0/1027355822 >> v2:172.21.15.146:3300/0 conn(0x7fc01c06c320 0x7fc01c15d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.186+0000 7fc022b2e640 1 -- 172.21.15.16:0/1027355822 >> v1:172.21.15.146:6789/0 conn(0x7fc01c145ec0 legacy=0x7fc01c06bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.186+0000 7fc022b2e640 1 -- 172.21.15.16:0/1027355822 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c15e0a0 con 0x7fc01c0b6c40 2024-09-19T07:43:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.187+0000 7fc022b2e640 1 --2- 172.21.15.16:0/1027355822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c0b6c40 0x7fc01c15e8a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc014009d90 tx=0x7fc01402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af13088fd8e4ea66 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.187+0000 7fc02132b640 1 -- 172.21.15.16:0/1027355822 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014031600 con 0x7fc01c0b6c40 2024-09-19T07:43:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.187+0000 7fc02132b640 1 -- 172.21.15.16:0/1027355822 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0140317a0 con 0x7fc01c0b6c40 2024-09-19T07:43:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.187+0000 7fc02132b640 1 -- 172.21.15.16:0/1027355822 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014031b70 con 0x7fc01c0b6c40 2024-09-19T07:43:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.188+0000 7fc024591640 1 -- 172.21.15.16:0/1027355822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c0b6c40 msgr2=0x7fc01c15e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.188+0000 7fc024591640 1 --2- 172.21.15.16:0/1027355822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c0b6c40 0x7fc01c15e8a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc014009d90 tx=0x7fc01402a5d0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.188+0000 7fc024591640 1 -- 172.21.15.16:0/1027355822 shutdown_connections 2024-09-19T07:43:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.188+0000 7fc024591640 1 --2- 172.21.15.16:0/1027355822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c0b6c40 0x7fc01c15e8a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.188+0000 7fc024591640 1 --2- 172.21.15.16:0/1027355822 >> v2:172.21.15.146:3300/0 conn(0x7fc01c06c320 0x7fc01c15d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.188+0000 7fc024591640 1 -- 172.21.15.16:0/1027355822 >> 172.21.15.16:0/1027355822 conn(0x7fc01c09a9f0 msgr2=0x7fc01c0b6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.188+0000 7fc024591640 1 -- 172.21.15.16:0/1027355822 shutdown_connections 2024-09-19T07:43:27.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.188+0000 7fc024591640 1 -- 172.21.15.16:0/1027355822 wait complete. 2024-09-19T07:43:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.189+0000 7fc024591640 1 Processor -- start 2024-09-19T07:43:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.189+0000 7fc024591640 1 -- start start 2024-09-19T07:43:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.190+0000 7fc024591640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc01c06c320 0x7fc01c070070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.190+0000 7fc024591640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c0b6c40 0x7fc01c0705b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.190+0000 7fc024591640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c145ec0 0x7fc01c0b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.190+0000 7fc024591640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c15ede0 con 0x7fc01c145ec0 2024-09-19T07:43:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.190+0000 7fc024591640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c0b70b0 con 0x7fc01c06c320 2024-09-19T07:43:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.190+0000 7fc024591640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc01c15ef60 con 0x7fc01c0b6c40 2024-09-19T07:43:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.190+0000 7fc022b2e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c145ec0 0x7fc01c0b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.190+0000 7fc022b2e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c145ec0 0x7fc01c0b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57280/0 (socket says 172.21.15.16:57280) 2024-09-19T07:43:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.190+0000 7fc022b2e640 1 -- 172.21.15.16:0/1244670059 learned_addr learned my addr 172.21.15.16:0/1244670059 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.190+0000 7fc02232d640 1 --2- 172.21.15.16:0/1244670059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc01c06c320 0x7fc01c070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.190+0000 7fc021b2c640 1 --2- 172.21.15.16:0/1244670059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c0b6c40 0x7fc01c0705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.191+0000 7fc022b2e640 1 -- 172.21.15.16:0/1244670059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c0b6c40 msgr2=0x7fc01c0705b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.191+0000 7fc022b2e640 1 --2- 172.21.15.16:0/1244670059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c0b6c40 0x7fc01c0705b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.191+0000 7fc022b2e640 1 -- 172.21.15.16:0/1244670059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc01c06c320 msgr2=0x7fc01c070070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.191+0000 7fc022b2e640 1 --2- 172.21.15.16:0/1244670059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc01c06c320 0x7fc01c070070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.191+0000 7fc022b2e640 1 -- 172.21.15.16:0/1244670059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c1f64b0 con 0x7fc01c145ec0 2024-09-19T07:43:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.191+0000 7fc021b2c640 1 --2- 172.21.15.16:0/1244670059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c0b6c40 0x7fc01c0705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.191+0000 7fc022b2e640 1 --2- 172.21.15.16:0/1244670059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c145ec0 0x7fc01c0b1d60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc01402abc0 tx=0x7fc014004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.191+0000 7fc012ffd640 1 -- 172.21.15.16:0/1244670059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014040070 con 0x7fc01c145ec0 2024-09-19T07:43:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.191+0000 7fc012ffd640 1 -- 172.21.15.16:0/1244670059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc014030b60 con 0x7fc01c145ec0 2024-09-19T07:43:27.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.191+0000 7fc024591640 1 -- 172.21.15.16:0/1244670059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01c1f56f0 con 0x7fc01c145ec0 2024-09-19T07:43:27.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.191+0000 7fc012ffd640 1 -- 172.21.15.16:0/1244670059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014009450 con 0x7fc01c145ec0 2024-09-19T07:43:27.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.191+0000 7fc024591640 1 -- 172.21.15.16:0/1244670059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01c1f5cb0 con 0x7fc01c145ec0 2024-09-19T07:43:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.192+0000 7fc024591640 1 -- 172.21.15.16:0/1244670059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01c15d170 con 0x7fc01c145ec0 2024-09-19T07:43:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.192+0000 7fc012ffd640 1 -- 172.21.15.16:0/1244670059 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc014039040 con 0x7fc01c145ec0 2024-09-19T07:43:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.193+0000 7fc012ffd640 1 --2- 172.21.15.16:0/1244670059 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc00006c2b0 0x7fc00006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.193+0000 7fc012ffd640 1 -- 172.21.15.16:0/1244670059 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc0140acc20 con 0x7fc01c145ec0 2024-09-19T07:43:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.193+0000 7fc02232d640 1 --2- 172.21.15.16:0/1244670059 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc00006c2b0 0x7fc00006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.194+0000 7fc02232d640 1 --2- 172.21.15.16:0/1244670059 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc00006c2b0 0x7fc00006e770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc018007fd0 tx=0x7fc018007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.197+0000 7fc012ffd640 1 -- 172.21.15.16:0/1244670059 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc01407aa90 con 0x7fc01c145ec0 2024-09-19T07:43:27.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.376+0000 7fc024591640 1 -- 172.21.15.16:0/1244670059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc01c09f360 con 0x7fc01c145ec0 2024-09-19T07:43:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.377+0000 7fc012ffd640 1 -- 172.21.15.16:0/1244670059 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7fc01407ed90 con 0x7fc01c145ec0 2024-09-19T07:43:27.378 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:27.378 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:43:10.642082+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:43:03.485582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:43:04.603961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:43:08.299591+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:43:27.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.379+0000 7fc024591640 1 -- 172.21.15.16:0/1244670059 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc00006c2b0 msgr2=0x7fc00006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.379+0000 7fc024591640 1 --2- 172.21.15.16:0/1244670059 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc00006c2b0 0x7fc00006e770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc018007fd0 tx=0x7fc018007e80 comp rx=0 tx=0).stop 2024-09-19T07:43:27.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.379+0000 7fc024591640 1 -- 172.21.15.16:0/1244670059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c145ec0 msgr2=0x7fc01c0b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.379+0000 7fc024591640 1 --2- 172.21.15.16:0/1244670059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c145ec0 0x7fc01c0b1d60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc01402abc0 tx=0x7fc014004a40 comp rx=0 tx=0).stop 2024-09-19T07:43:27.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.379+0000 7fc024591640 1 -- 172.21.15.16:0/1244670059 shutdown_connections 2024-09-19T07:43:27.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.379+0000 7fc024591640 1 --2- 172.21.15.16:0/1244670059 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc00006c2b0 0x7fc00006e770 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.379+0000 7fc024591640 1 --2- 172.21.15.16:0/1244670059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c145ec0 0x7fc01c0b1d60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.379+0000 7fc024591640 1 --2- 172.21.15.16:0/1244670059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c0b6c40 0x7fc01c0705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.379+0000 7fc024591640 1 --2- 172.21.15.16:0/1244670059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc01c06c320 0x7fc01c070070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.379+0000 7fc024591640 1 -- 172.21.15.16:0/1244670059 >> 172.21.15.16:0/1244670059 conn(0x7fc01c09a9f0 msgr2=0x7fc01c15be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.380+0000 7fc024591640 1 -- 172.21.15.16:0/1244670059 shutdown_connections 2024-09-19T07:43:27.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.380+0000 7fc024591640 1 -- 172.21.15.16:0/1244670059 wait complete. 2024-09-19T07:43:27.391 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-19T07:43:27.391 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:43:27.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.492+0000 7fb7dcd42640 1 Processor -- start 2024-09-19T07:43:27.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.492+0000 7fb7dcd42640 1 -- start start 2024-09-19T07:43:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.493+0000 7fb7dcd42640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7d80d0070 0x7fb7d815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.493+0000 7fb7dcd42640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb7d80d4130 0x7fb7d80d45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.493+0000 7fb7dcd42640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7d80d4b00 con 0x7fb7d815f0d0 2024-09-19T07:43:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.493+0000 7fb7dcd42640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb7d80d1b10 con 0x7fb7d80d4130 2024-09-19T07:43:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.493+0000 7fb7dcd42640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb7d80cf2b0 con 0x7fb7d80d0070 2024-09-19T07:43:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.493+0000 7fb7d659c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb7d815f0d0 0x7fb7d80d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44832/0 (socket says 172.21.15.16:44832) 2024-09-19T07:43:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.493+0000 7fb7d659c640 1 -- 172.21.15.16:0/1714658189 learned_addr learned my addr 172.21.15.16:0/1714658189 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.493+0000 7fb7d6d9d640 1 --2- 172.21.15.16:0/1714658189 >> v2:172.21.15.146:3300/0 conn(0x7fb7d80d4130 0x7fb7d80d45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.493+0000 7fb7d5d9b640 1 --2- 172.21.15.16:0/1714658189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7d80d0070 0x7fb7d815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.493+0000 7fb7d559a640 1 -- 172.21.15.16:0/1714658189 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1843900587 0 0) 0x7fb7d80d4b00 con 0x7fb7d815f0d0 2024-09-19T07:43:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.493+0000 7fb7d559a640 1 -- 172.21.15.16:0/1714658189 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7c0003520 con 0x7fb7d815f0d0 2024-09-19T07:43:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d559a640 1 -- 172.21.15.16:0/1714658189 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3854418798 0 0) 0x7fb7c0003520 con 0x7fb7d815f0d0 2024-09-19T07:43:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d559a640 1 -- 172.21.15.16:0/1714658189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7d80d0070 msgr2=0x7fb7d815f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d559a640 1 --2- 172.21.15.16:0/1714658189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7d80d0070 0x7fb7d815f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d559a640 1 -- 172.21.15.16:0/1714658189 >> v2:172.21.15.146:3300/0 conn(0x7fb7d80d4130 msgr2=0x7fb7d80d45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d559a640 1 --2- 172.21.15.16:0/1714658189 >> v2:172.21.15.146:3300/0 conn(0x7fb7d80d4130 0x7fb7d80d45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d559a640 1 -- 172.21.15.16:0/1714658189 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d80cf590 con 0x7fb7d815f0d0 2024-09-19T07:43:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d559a640 1 -- 172.21.15.16:0/1714658189 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fb7c8001670 con 0x7fb7d815f0d0 2024-09-19T07:43:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d559a640 1 -- 172.21.15.16:0/1714658189 >> v1:172.21.15.16:6789/0 conn(0x7fb7d815f0d0 legacy=0x7fb7d80d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d559a640 1 --2- 172.21.15.16:0/1714658189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c00039b0 0x7fb7c0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d559a640 1 --2- 172.21.15.16:0/1714658189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7c0004800 0x7fb7c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d559a640 1 --2- 172.21.15.16:0/1714658189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7d80d0070 0x7fb7c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d559a640 1 -- 172.21.15.16:0/1714658189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d80d1b10 con 0x7fb7c00039b0 2024-09-19T07:43:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d559a640 1 -- 172.21.15.16:0/1714658189 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d80cf2b0 con 0x7fb7c0004800 2024-09-19T07:43:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d559a640 1 -- 172.21.15.16:0/1714658189 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb7c000ba10 con 0x7fb7d80d0070 2024-09-19T07:43:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d659c640 1 --2- 172.21.15.16:0/1714658189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c00039b0 0x7fb7c0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d5d9b640 1 --2- 172.21.15.16:0/1714658189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7c0004800 0x7fb7c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d6d9d640 1 --2- 172.21.15.16:0/1714658189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7d80d0070 0x7fb7c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d659c640 1 -- 172.21.15.16:0/1714658189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7d80d0070 msgr2=0x7fb7c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d659c640 1 --2- 172.21.15.16:0/1714658189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7d80d0070 0x7fb7c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d659c640 1 -- 172.21.15.16:0/1714658189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7c0004800 msgr2=0x7fb7c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d659c640 1 --2- 172.21.15.16:0/1714658189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7c0004800 0x7fb7c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.494+0000 7fb7d659c640 1 -- 172.21.15.16:0/1714658189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c0003520 con 0x7fb7c00039b0 2024-09-19T07:43:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.495+0000 7fb7d659c640 1 --2- 172.21.15.16:0/1714658189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c00039b0 0x7fb7c0003df0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800c210 tx=0x7fb7c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69734e567c6c9cea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.495+0000 7fb7d559a640 1 -- 172.21.15.16:0/1714658189 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8007d50 con 0x7fb7c00039b0 2024-09-19T07:43:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.495+0000 7fb7d559a640 1 -- 172.21.15.16:0/1714658189 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7c8005ad0 con 0x7fb7c00039b0 2024-09-19T07:43:27.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.495+0000 7fb7d559a640 1 -- 172.21.15.16:0/1714658189 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8007070 con 0x7fb7c00039b0 2024-09-19T07:43:27.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.496+0000 7fb7dcd42640 1 -- 172.21.15.16:0/1714658189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c00039b0 msgr2=0x7fb7c0003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.496+0000 7fb7dcd42640 1 --2- 172.21.15.16:0/1714658189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c00039b0 0x7fb7c0003df0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800c210 tx=0x7fb7c0005830 comp rx=0 tx=0).stop 2024-09-19T07:43:27.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.496+0000 7fb7d6d9d640 1 -- 172.21.15.16:0/1714658189 reap_dead start 2024-09-19T07:43:27.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.496+0000 7fb7dcd42640 1 -- 172.21.15.16:0/1714658189 shutdown_connections 2024-09-19T07:43:27.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.496+0000 7fb7dcd42640 1 -- 172.21.15.16:0/1714658189 >> 172.21.15.16:0/1714658189 conn(0x7fb7d809d760 msgr2=0x7fb7d808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.496+0000 7fb7dcd42640 1 -- 172.21.15.16:0/1714658189 shutdown_connections 2024-09-19T07:43:27.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.496+0000 7fb7dcd42640 1 -- 172.21.15.16:0/1714658189 wait complete. 2024-09-19T07:43:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.497+0000 7fb7dcd42640 1 Processor -- start 2024-09-19T07:43:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.497+0000 7fb7dcd42640 1 -- start start 2024-09-19T07:43:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.498+0000 7fb7dcd42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c00039b0 0x7fb7d81597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.498+0000 7fb7dcd42640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7d8160f50 0x7fb7d809e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.498+0000 7fb7dcd42640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7d81613b0 0x7fb7d809e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.498+0000 7fb7dcd42640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d80cef60 con 0x7fb7c00039b0 2024-09-19T07:43:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.498+0000 7fb7dcd42640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d80cfb50 con 0x7fb7d81613b0 2024-09-19T07:43:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.498+0000 7fb7dcd42640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb7d815f4b0 con 0x7fb7d8160f50 2024-09-19T07:43:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.498+0000 7fb7d659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c00039b0 0x7fb7d81597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.498+0000 7fb7d659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c00039b0 0x7fb7d81597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57306/0 (socket says 172.21.15.16:57306) 2024-09-19T07:43:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.498+0000 7fb7d659c640 1 -- 172.21.15.16:0/42127942 learned_addr learned my addr 172.21.15.16:0/42127942 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.498+0000 7fb7d6d9d640 1 --2- 172.21.15.16:0/42127942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7d81613b0 0x7fb7d809e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.498+0000 7fb7d5d9b640 1 --2- 172.21.15.16:0/42127942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7d8160f50 0x7fb7d809e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.498+0000 7fb7d659c640 1 -- 172.21.15.16:0/42127942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7d8160f50 msgr2=0x7fb7d809e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.498+0000 7fb7d659c640 1 --2- 172.21.15.16:0/42127942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7d8160f50 0x7fb7d809e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.499+0000 7fb7d659c640 1 -- 172.21.15.16:0/42127942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7d81613b0 msgr2=0x7fb7d809e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.499+0000 7fb7d659c640 1 --2- 172.21.15.16:0/42127942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7d81613b0 0x7fb7d809e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.499+0000 7fb7d659c640 1 -- 172.21.15.16:0/42127942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d80a00a0 con 0x7fb7c00039b0 2024-09-19T07:43:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.499+0000 7fb7d659c640 1 --2- 172.21.15.16:0/42127942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c00039b0 0x7fb7d81597a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800c330 tx=0x7fb7d815e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.499+0000 7fb7c6ffd640 1 -- 172.21.15.16:0/42127942 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8001cb0 con 0x7fb7c00039b0 2024-09-19T07:43:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.499+0000 7fb7c6ffd640 1 -- 172.21.15.16:0/42127942 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7c8043070 con 0x7fb7c00039b0 2024-09-19T07:43:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.499+0000 7fb7dcd42640 1 -- 172.21.15.16:0/42127942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d80a0380 con 0x7fb7c00039b0 2024-09-19T07:43:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.499+0000 7fb7c6ffd640 1 -- 172.21.15.16:0/42127942 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c80076a0 con 0x7fb7c00039b0 2024-09-19T07:43:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.499+0000 7fb7dcd42640 1 -- 172.21.15.16:0/42127942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d80a0940 con 0x7fb7c00039b0 2024-09-19T07:43:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.500+0000 7fb7dcd42640 1 -- 172.21.15.16:0/42127942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7d815f8c0 con 0x7fb7c00039b0 2024-09-19T07:43:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.500+0000 7fb7c6ffd640 1 -- 172.21.15.16:0/42127942 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb7c8033440 con 0x7fb7c00039b0 2024-09-19T07:43:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.501+0000 7fb7c6ffd640 1 --2- 172.21.15.16:0/42127942 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb7b806c2e0 0x7fb7b806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.501+0000 7fb7c6ffd640 1 -- 172.21.15.16:0/42127942 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb7c80abb60 con 0x7fb7c00039b0 2024-09-19T07:43:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.501+0000 7fb7d5d9b640 1 --2- 172.21.15.16:0/42127942 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb7b806c2e0 0x7fb7b806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.502+0000 7fb7d5d9b640 1 --2- 172.21.15.16:0/42127942 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb7b806c2e0 0x7fb7b806e7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80d3ee0 tx=0x7fb7cc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.505+0000 7fb7c6ffd640 1 -- 172.21.15.16:0/42127942 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb7c8079fa0 con 0x7fb7c00039b0 2024-09-19T07:43:27.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.681+0000 7fb7dcd42640 1 -- 172.21.15.16:0/42127942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7d806cf30 con 0x7fb7c00039b0 2024-09-19T07:43:27.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.682+0000 7fb7c6ffd640 1 -- 172.21.15.16:0/42127942 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7fb7c80798a0 con 0x7fb7c00039b0 2024-09-19T07:43:27.683 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:27.683 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:43:10.642082+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:43:03.485582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:43:04.603961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:43:08.299591+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:43:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.684+0000 7fb7dcd42640 1 -- 172.21.15.16:0/42127942 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb7b806c2e0 msgr2=0x7fb7b806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.684+0000 7fb7dcd42640 1 --2- 172.21.15.16:0/42127942 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb7b806c2e0 0x7fb7b806e7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80d3ee0 tx=0x7fb7cc009190 comp rx=0 tx=0).stop 2024-09-19T07:43:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.684+0000 7fb7dcd42640 1 -- 172.21.15.16:0/42127942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c00039b0 msgr2=0x7fb7d81597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.684+0000 7fb7dcd42640 1 --2- 172.21.15.16:0/42127942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c00039b0 0x7fb7d81597a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800c330 tx=0x7fb7d815e930 comp rx=0 tx=0).stop 2024-09-19T07:43:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.684+0000 7fb7dcd42640 1 -- 172.21.15.16:0/42127942 shutdown_connections 2024-09-19T07:43:27.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.685+0000 7fb7dcd42640 1 --2- 172.21.15.16:0/42127942 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb7b806c2e0 0x7fb7b806e7a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.685+0000 7fb7dcd42640 1 --2- 172.21.15.16:0/42127942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7d81613b0 0x7fb7d809e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.685+0000 7fb7dcd42640 1 --2- 172.21.15.16:0/42127942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7d8160f50 0x7fb7d809e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.685+0000 7fb7dcd42640 1 --2- 172.21.15.16:0/42127942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c00039b0 0x7fb7d81597a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.685+0000 7fb7dcd42640 1 -- 172.21.15.16:0/42127942 >> 172.21.15.16:0/42127942 conn(0x7fb7d809d760 msgr2=0x7fb7d80d2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.685+0000 7fb7dcd42640 1 -- 172.21.15.16:0/42127942 shutdown_connections 2024-09-19T07:43:27.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.685+0000 7fb7dcd42640 1 -- 172.21.15.16:0/42127942 wait complete. 2024-09-19T07:43:27.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-19T07:43:27.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-19T07:43:27.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-19T07:43:27.698 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-19T07:43:27.698 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-19T07:43:27.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-19T07:43:27.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-19T07:43:27.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-19T07:43:27.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.875+0000 7f3c2eaba640 1 Processor -- start 2024-09-19T07:43:27.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.874+0000 7fa32d67b640 1 Processor -- start 2024-09-19T07:43:27.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.875+0000 7fa32d67b640 1 -- start start 2024-09-19T07:43:27.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.876+0000 7fa32d67b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa3280d0df0 0x7fa3280d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.876+0000 7fa32d67b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa328163270 con 0x7fa3280d2980 2024-09-19T07:43:27.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.876+0000 7fa32d67b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa328163550 con 0x7fa3281626c0 2024-09-19T07:43:27.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.876+0000 7fa32d67b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa3280d1780 con 0x7fa3280d0df0 2024-09-19T07:43:27.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.875+0000 7f3c2eaba640 1 -- start start 2024-09-19T07:43:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.876+0000 7f493eca5640 1 Processor -- start 2024-09-19T07:43:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.876+0000 7f493eca5640 1 -- start start 2024-09-19T07:43:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.876+0000 7f3c2eaba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3c2815c960 0x7f3c28162a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.876+0000 7f3c2eaba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c280b30c0 0x7f3c2806dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.876+0000 7f3c2eaba640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3c2806e0f0 0x7f3c28165ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.876+0000 7f3c2eaba640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f3c28163a40 con 0x7f3c2815c960 2024-09-19T07:43:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.876+0000 7f3c2eaba640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f3c2806e540 con 0x7f3c2806e0f0 2024-09-19T07:43:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.876+0000 7f3c2eaba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3c28071e00 con 0x7f3c280b30c0 2024-09-19T07:43:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.876+0000 7fec810af640 1 Processor -- start 2024-09-19T07:43:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.877+0000 7fa3277fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa3280d0df0 0x7fa3280d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.877+0000 7fa3277fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa3280d0df0 0x7fa3280d1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57326/0 (socket says 172.21.15.16:57326) 2024-09-19T07:43:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.877+0000 7fa3277fe640 1 -- 172.21.15.16:0/2223959142 learned_addr learned my addr 172.21.15.16:0/2223959142 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.877+0000 7fec810af640 1 -- start start 2024-09-19T07:43:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.877+0000 7f3c2c856640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3c2815c960 0x7f3c28162a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.877+0000 7f3c2c856640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3c2815c960 0x7f3c28162a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57312/0 (socket says 172.21.15.16:57312) 2024-09-19T07:43:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.877+0000 7f3c27fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c280b30c0 0x7f3c2806dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.877+0000 7f3c2c856640 1 -- 172.21.15.16:0/1991028276 learned_addr learned my addr 172.21.15.16:0/1991028276 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.877+0000 7f3c2d858640 1 --2- 172.21.15.16:0/1991028276 >> v2:172.21.15.146:3300/0 conn(0x7f3c2806e0f0 0x7f3c28165ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.877+0000 7efe79617640 1 Processor -- start 2024-09-19T07:43:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.877+0000 7fc710c96640 1 Processor -- start 2024-09-19T07:43:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.877+0000 7efe79617640 1 -- start start 2024-09-19T07:43:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.877+0000 7f493eca5640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f49380b5010 0x7f49380c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.877+0000 7f493eca5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49380c5630 0x7f49380ad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.877+0000 7f3c2d858640 1 -- 172.21.15.16:0/1991028276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c280b30c0 msgr2=0x7f3c2806dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.877+0000 7f3c2d858640 1 --2- 172.21.15.16:0/1991028276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c280b30c0 0x7f3c2806dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.877+0000 7f3c2d858640 1 -- 172.21.15.16:0/1991028276 >> v2:172.21.15.16:3300/0 conn(0x7f3c2815c960 msgr2=0x7f3c28162a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.877+0000 7f3c2d858640 1 --2- 172.21.15.16:0/1991028276 >> v2:172.21.15.16:3300/0 conn(0x7f3c2815c960 0x7f3c28162a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.877+0000 7f3c2d858640 1 -- 172.21.15.16:0/1991028276 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c281631a0 con 0x7f3c2806e0f0 2024-09-19T07:43:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7f493eca5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49380ad6a0 0x7f4938098ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7f493eca5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f49380c5a90 con 0x7f49380ad6a0 2024-09-19T07:43:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7f493eca5640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f49380aeaa0 con 0x7f49380b5010 2024-09-19T07:43:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7f493eca5640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f49380ade90 con 0x7f49380c5630 2024-09-19T07:43:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7f4937fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49380c5630 0x7f49380ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7f493ca41640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f49380b5010 0x7f49380c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7f4937fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49380c5630 0x7f49380ad160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:34044/0 (socket says 172.21.15.16:34044) 2024-09-19T07:43:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7f4937fff640 1 -- 172.21.15.16:0/3741648697 learned_addr learned my addr 172.21.15.16:0/3741648697 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7f493d242640 1 --2- 172.21.15.16:0/3741648697 >> v2:172.21.15.16:3300/0 conn(0x7f49380ad6a0 0x7f4938098ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7fc710c96640 1 -- start start 2024-09-19T07:43:27.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7fa3277fe640 1 -- 172.21.15.16:0/2223959142 >> v1:172.21.15.146:6789/0 conn(0x7fa3281626c0 legacy=0x7fa328162aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:27.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7fa3277fe640 1 -- 172.21.15.16:0/2223959142 >> v1:172.21.15.16:6789/0 conn(0x7fa3280d2980 legacy=0x7fa3280d2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:27.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7fa3277fe640 1 -- 172.21.15.16:0/2223959142 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa328163830 con 0x7fa3280d0df0 2024-09-19T07:43:27.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7fec810af640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fec740e72f0 0x7fec74058340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7fec810af640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fec7403ce20 0x7fec74056d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7fec810af640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec74006930 con 0x7fec740535b0 2024-09-19T07:43:27.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7f4937fff640 1 -- 172.21.15.16:0/3741648697 >> v2:172.21.15.146:3300/0 conn(0x7f49380b5010 msgr2=0x7f49380c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:27.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7f4937fff640 1 --2- 172.21.15.16:0/3741648697 >> v2:172.21.15.146:3300/0 conn(0x7f49380b5010 0x7f49380c50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7f4937fff640 1 -- 172.21.15.16:0/3741648697 >> v2:172.21.15.16:3300/0 conn(0x7f49380ad6a0 msgr2=0x7f4938098ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:27.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7f4937fff640 1 --2- 172.21.15.16:0/3741648697 >> v2:172.21.15.16:3300/0 conn(0x7f49380ad6a0 0x7f4938098ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7f4937fff640 1 -- 172.21.15.16:0/3741648697 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49380ae170 con 0x7f49380c5630 2024-09-19T07:43:27.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7fec810af640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fec74006c10 con 0x7fec740e72f0 2024-09-19T07:43:27.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7fec810af640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fec74006ef0 con 0x7fec7403ce20 2024-09-19T07:43:27.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7f3c2d858640 1 --2- 172.21.15.16:0/1991028276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c2806e0f0 0x7f3c28165ff0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c280cffc0 tx=0x7f3c1402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=556aefffe9babe37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7f0c81d82640 1 Processor -- start 2024-09-19T07:43:27.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.878+0000 7f0c81d82640 1 -- start start 2024-09-19T07:43:27.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7fec7bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fec740535b0 0x7fec74053990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44858/0 (socket says 172.21.15.16:44858) 2024-09-19T07:43:27.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7fec7bfff640 1 -- 172.21.15.16:0/1744006692 learned_addr learned my addr 172.21.15.16:0/1744006692 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:27.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7fc710c96640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc704005a60 0x7fc704053f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7fc710c96640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc7040e7120 0x7fc704005e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7f3c277fe640 1 -- 172.21.15.16:0/1991028276 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14037070 con 0x7f3c2806e0f0 2024-09-19T07:43:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7fec7b7fe640 1 --2- 172.21.15.16:0/1744006692 >> v2:172.21.15.16:3300/0 conn(0x7fec740e72f0 0x7fec74058340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7fec808ae640 1 --2- 172.21.15.16:0/1744006692 >> v2:172.21.15.146:3300/0 conn(0x7fec7403ce20 0x7fec74056d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7fc710c96640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc704006380 con 0x7fc70404eb60 2024-09-19T07:43:27.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7fec7b7fe640 1 -- 172.21.15.16:0/1744006692 >> v2:172.21.15.146:3300/0 conn(0x7fec7403ce20 msgr2=0x7fec74056d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:27.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7fc710c96640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc7040520e0 con 0x7fc704005a60 2024-09-19T07:43:27.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7fc710c96640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc704052330 con 0x7fc7040e7120 2024-09-19T07:43:27.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7fc70affd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc7040e7120 0x7fc704005e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7f3c277fe640 1 -- 172.21.15.16:0/1991028276 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c1402ae50 con 0x7f3c2806e0f0 2024-09-19T07:43:27.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7f3c277fe640 1 -- 172.21.15.16:0/1991028276 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14032660 con 0x7f3c2806e0f0 2024-09-19T07:43:27.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7fec7b7fe640 1 --2- 172.21.15.16:0/1744006692 >> v2:172.21.15.146:3300/0 conn(0x7fec7403ce20 0x7fec74056d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7fec7b7fe640 1 -- 172.21.15.16:0/1744006692 >> v1:172.21.15.16:6789/0 conn(0x7fec740535b0 legacy=0x7fec74053990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7fec7b7fe640 1 -- 172.21.15.16:0/1744006692 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec740071d0 con 0x7fec740e72f0 2024-09-19T07:43:27.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7fc70b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc704005a60 0x7fc704053f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7fa3277fe640 1 --2- 172.21.15.16:0/2223959142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3280d0df0 0x7fa3280d1240 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009870 tx=0x7fa31c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353103aaf50c4a4a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7f4937fff640 1 --2- 172.21.15.16:0/3741648697 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49380c5630 0x7f49380ad160 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f49380c5d00 tx=0x7f492c02a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dab80970660abcdc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7fc70affd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc7040e7120 0x7fc704005e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:57808/0 (socket says 172.21.15.16:57808) 2024-09-19T07:43:27.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7fc70affd640 1 -- 172.21.15.16:0/1151270466 learned_addr learned my addr 172.21.15.16:0/1151270466 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:27.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7efe79617640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efe74099ae0 0x7efe740d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7efe79617640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efe740d09f0 0x7efe740ad920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7efe79617640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe740ad160 con 0x7efe740c5690 2024-09-19T07:43:27.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7efe79617640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7efe740d0e60 con 0x7efe74099ae0 2024-09-19T07:43:27.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7efe79617640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7efe7415d2d0 con 0x7efe740d09f0 2024-09-19T07:43:27.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7efe72ffd640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7efe740c5690 0x7efe74162d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:56406/0 (socket says 172.21.15.16:56406) 2024-09-19T07:43:27.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7efe72ffd640 1 -- 172.21.15.16:0/3463368792 learned_addr learned my addr 172.21.15.16:0/3463368792 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:27.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.879+0000 7efe737fe640 1 --2- 172.21.15.16:0/3463368792 >> v2:172.21.15.146:3300/0 conn(0x7efe740d09f0 0x7efe740ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7efe727fc640 1 --2- 172.21.15.16:0/3463368792 >> v2:172.21.15.16:3300/0 conn(0x7efe74099ae0 0x7efe740d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7fa325ffb640 1 -- 172.21.15.16:0/2223959142 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c037070 con 0x7fa3280d0df0 2024-09-19T07:43:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7f0c81d82640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c74007d60 0x7f0c7403a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7f0c81d82640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c74008c00 con 0x7f0c74034470 2024-09-19T07:43:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7f0c81d82640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c74008ee0 con 0x7f0c740349c0 2024-09-19T07:43:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7f0c81d82640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0c740380c0 con 0x7f0c74007d60 2024-09-19T07:43:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7fa325ffb640 1 -- 172.21.15.16:0/2223959142 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa31c032440 con 0x7fa3280d0df0 2024-09-19T07:43:27.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7fa325ffb640 1 -- 172.21.15.16:0/2223959142 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c032810 con 0x7fa3280d0df0 2024-09-19T07:43:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7efe737fe640 1 -- 172.21.15.16:0/3463368792 >> v2:172.21.15.16:3300/0 conn(0x7efe74099ae0 msgr2=0x7efe740d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7efe737fe640 1 --2- 172.21.15.16:0/3463368792 >> v2:172.21.15.16:3300/0 conn(0x7efe74099ae0 0x7efe740d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7efe737fe640 1 -- 172.21.15.16:0/3463368792 >> v1:172.21.15.146:6789/0 conn(0x7efe740c5690 legacy=0x7efe74162d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7efe737fe640 1 -- 172.21.15.16:0/3463368792 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe74163680 con 0x7efe740d09f0 2024-09-19T07:43:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7f3c2eaba640 1 -- 172.21.15.16:0/1991028276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c2806e0f0 msgr2=0x7f3c28165ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7f3c2eaba640 1 --2- 172.21.15.16:0/1991028276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c2806e0f0 0x7f3c28165ff0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c280cffc0 tx=0x7f3c1402a100 comp rx=0 tx=0).stop 2024-09-19T07:43:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7f3c2eaba640 1 -- 172.21.15.16:0/1991028276 shutdown_connections 2024-09-19T07:43:27.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7f3c2eaba640 1 --2- 172.21.15.16:0/1991028276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c2806e0f0 0x7f3c28165ff0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7f3c2eaba640 1 --2- 172.21.15.16:0/1991028276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c280b30c0 0x7f3c2806dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7f3c2eaba640 1 --2- 172.21.15.16:0/1991028276 >> v2:172.21.15.16:3300/0 conn(0x7f3c2815c960 0x7f3c28162a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7f3c2eaba640 1 -- 172.21.15.16:0/1991028276 >> 172.21.15.16:0/1991028276 conn(0x7f3c280b5a00 msgr2=0x7f3c2806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7f0c81581640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c74007d60 0x7f0c7403a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7f0c81581640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c74007d60 0x7f0c7403a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:34056/0 (socket says 172.21.15.16:34056) 2024-09-19T07:43:27.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7f0c81581640 1 -- 172.21.15.16:0/2746436550 learned_addr learned my addr 172.21.15.16:0/2746436550 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7fec7b7fe640 1 --2- 172.21.15.16:0/1744006692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec740e72f0 0x7fec74058340 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fec74056720 tx=0x7fec7002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f5197582e1cde6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7fec7affd640 1 -- 172.21.15.16:0/1744006692 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec70037070 con 0x7fec740e72f0 2024-09-19T07:43:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7fec7affd640 1 -- 172.21.15.16:0/1744006692 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec70032440 con 0x7fec740e72f0 2024-09-19T07:43:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7fec7affd640 1 -- 172.21.15.16:0/1744006692 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec70032810 con 0x7fec740e72f0 2024-09-19T07:43:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7f49377fe640 1 -- 172.21.15.16:0/3741648697 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c037070 con 0x7f49380c5630 2024-09-19T07:43:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7f49377fe640 1 -- 172.21.15.16:0/3741648697 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f492c032440 con 0x7f49380c5630 2024-09-19T07:43:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7f49377fe640 1 -- 172.21.15.16:0/3741648697 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c032810 con 0x7f49380c5630 2024-09-19T07:43:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7fc70affd640 1 -- 172.21.15.16:0/1151270466 >> v2:172.21.15.16:3300/0 conn(0x7fc704005a60 msgr2=0x7fc704053f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7fc70affd640 1 --2- 172.21.15.16:0/1151270466 >> v2:172.21.15.16:3300/0 conn(0x7fc704005a60 0x7fc704053f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7fc70affd640 1 -- 172.21.15.16:0/1151270466 >> v1:172.21.15.146:6789/0 conn(0x7fc70404eb60 legacy=0x7fc70401c520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:43:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.880+0000 7fc70affd640 1 -- 172.21.15.16:0/1151270466 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc704006c80 con 0x7fc7040e7120 2024-09-19T07:43:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7fa32d67b640 1 -- 172.21.15.16:0/2223959142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3280d0df0 msgr2=0x7fa3280d1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7fa32d67b640 1 --2- 172.21.15.16:0/2223959142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3280d0df0 0x7fa3280d1240 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009870 tx=0x7fa31c02a270 comp rx=0 tx=0).stop 2024-09-19T07:43:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7f0c81581640 1 -- 172.21.15.16:0/2746436550 >> v1:172.21.15.146:6789/0 conn(0x7f0c740349c0 legacy=0x7f0c74006640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:43:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7f0c81581640 1 -- 172.21.15.16:0/2746436550 >> v1:172.21.15.16:6789/0 conn(0x7f0c74034470 legacy=0x7f0c74055dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7f0c81581640 1 -- 172.21.15.16:0/2746436550 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c740e67b0 con 0x7f0c74007d60 2024-09-19T07:43:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7f3c2eaba640 1 -- 172.21.15.16:0/1991028276 shutdown_connections 2024-09-19T07:43:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7fa32d67b640 1 -- 172.21.15.16:0/2223959142 shutdown_connections 2024-09-19T07:43:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7fa32d67b640 1 --2- 172.21.15.16:0/2223959142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3280d0df0 0x7fa3280d1240 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7fa32d67b640 1 -- 172.21.15.16:0/2223959142 >> 172.21.15.16:0/2223959142 conn(0x7fa32809d770 msgr2=0x7fa32808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7efe737fe640 1 --2- 172.21.15.16:0/3463368792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe740d09f0 0x7efe740ad920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efe7415d450 tx=0x7efe6c02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3118e5e5eed85f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7efe71ffb640 1 -- 172.21.15.16:0/3463368792 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c031600 con 0x7efe740d09f0 2024-09-19T07:43:27.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7efe71ffb640 1 -- 172.21.15.16:0/3463368792 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6c0317a0 con 0x7efe740d09f0 2024-09-19T07:43:27.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7efe71ffb640 1 -- 172.21.15.16:0/3463368792 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c031b70 con 0x7efe740d09f0 2024-09-19T07:43:27.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.881+0000 7f3c2eaba640 1 -- 172.21.15.16:0/1991028276 wait complete. 2024-09-19T07:43:27.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7fa32d67b640 1 -- 172.21.15.16:0/2223959142 shutdown_connections 2024-09-19T07:43:27.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7fc70affd640 1 --2- 172.21.15.16:0/1151270466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7040e7120 0x7fc704005e40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc70400d7c0 tx=0x7fc70002a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a9940fd5c8f412f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7f2a126ba640 1 Processor -- start 2024-09-19T07:43:27.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7f2a126ba640 1 -- start start 2024-09-19T07:43:27.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7f0c81581640 1 --2- 172.21.15.16:0/2746436550 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c74007d60 0x7f0c7403a360 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0c700098a0 tx=0x7f0c7002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5002b54a838edfb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7fec810af640 1 -- 172.21.15.16:0/1744006692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec740e72f0 msgr2=0x7fec74058340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7fec810af640 1 --2- 172.21.15.16:0/1744006692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec740e72f0 0x7fec74058340 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fec74056720 tx=0x7fec7002a1a0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7f493eca5640 1 -- 172.21.15.16:0/3741648697 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49380c5630 msgr2=0x7f49380ad160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7f493eca5640 1 --2- 172.21.15.16:0/3741648697 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49380c5630 0x7f49380ad160 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f49380c5d00 tx=0x7f492c02a1d0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7fa32d67b640 1 -- 172.21.15.16:0/2223959142 wait complete. 2024-09-19T07:43:27.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7fec810af640 1 -- 172.21.15.16:0/1744006692 shutdown_connections 2024-09-19T07:43:27.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7fc70a7fc640 1 -- 172.21.15.16:0/1151270466 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700037070 con 0x7fc7040e7120 2024-09-19T07:43:27.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7fec810af640 1 --2- 172.21.15.16:0/1744006692 >> v2:172.21.15.146:3300/0 conn(0x7fec7403ce20 0x7fec74056d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7fec810af640 1 --2- 172.21.15.16:0/1744006692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec740e72f0 0x7fec74058340 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7fec810af640 1 -- 172.21.15.16:0/1744006692 >> 172.21.15.16:0/1744006692 conn(0x7fec7403a520 msgr2=0x7fec7403a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7fc70a7fc640 1 -- 172.21.15.16:0/1151270466 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc700032440 con 0x7fc7040e7120 2024-09-19T07:43:27.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7f493eca5640 1 -- 172.21.15.16:0/3741648697 shutdown_connections 2024-09-19T07:43:27.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7f493eca5640 1 --2- 172.21.15.16:0/3741648697 >> v2:172.21.15.16:3300/0 conn(0x7f49380ad6a0 0x7f4938098ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7f493eca5640 1 --2- 172.21.15.16:0/3741648697 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49380c5630 0x7f49380ad160 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7f493eca5640 1 --2- 172.21.15.16:0/3741648697 >> v2:172.21.15.146:3300/0 conn(0x7f49380b5010 0x7f49380c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.882+0000 7f493eca5640 1 -- 172.21.15.16:0/3741648697 >> 172.21.15.16:0/3741648697 conn(0x7f493806bdf0 msgr2=0x7f493806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7fec810af640 1 -- 172.21.15.16:0/1744006692 shutdown_connections 2024-09-19T07:43:27.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7fec810af640 1 -- 172.21.15.16:0/1744006692 wait complete. 2024-09-19T07:43:27.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7f493eca5640 1 -- 172.21.15.16:0/3741648697 shutdown_connections 2024-09-19T07:43:27.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7f0c7b7fe640 1 -- 172.21.15.16:0/2746436550 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70037070 con 0x7f0c74007d60 2024-09-19T07:43:27.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7f0c7b7fe640 1 -- 172.21.15.16:0/2746436550 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c7002ae20 con 0x7f0c74007d60 2024-09-19T07:43:27.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7f0c7b7fe640 1 -- 172.21.15.16:0/2746436550 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70032660 con 0x7f0c74007d60 2024-09-19T07:43:27.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7f3c2eaba640 1 Processor -- start 2024-09-19T07:43:27.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7fc70a7fc640 1 -- 172.21.15.16:0/1151270466 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700032810 con 0x7fc7040e7120 2024-09-19T07:43:27.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7fc710c96640 1 -- 172.21.15.16:0/1151270466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7040e7120 msgr2=0x7fc704005e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7fc710c96640 1 --2- 172.21.15.16:0/1151270466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7040e7120 0x7fc704005e40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc70400d7c0 tx=0x7fc70002a2c0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7f2a126ba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a0c0d2750 0x7f2a0c13f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7fc710c96640 1 -- 172.21.15.16:0/1151270466 shutdown_connections 2024-09-19T07:43:27.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7fc710c96640 1 --2- 172.21.15.16:0/1151270466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7040e7120 0x7fc704005e40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7fc710c96640 1 --2- 172.21.15.16:0/1151270466 >> v2:172.21.15.16:3300/0 conn(0x7fc704005a60 0x7fc704053f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7fc710c96640 1 -- 172.21.15.16:0/1151270466 >> 172.21.15.16:0/1151270466 conn(0x7fc704036090 msgr2=0x7fc7040364a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7f3c2eaba640 1 -- start start 2024-09-19T07:43:27.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.884+0000 7fc710c96640 1 -- 172.21.15.16:0/1151270466 shutdown_connections 2024-09-19T07:43:27.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7efe79617640 1 -- 172.21.15.16:0/3463368792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe740d09f0 msgr2=0x7efe740ad920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7efe79617640 1 --2- 172.21.15.16:0/3463368792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe740d09f0 0x7efe740ad920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efe7415d450 tx=0x7efe6c02a5d0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7efe79617640 1 -- 172.21.15.16:0/3463368792 shutdown_connections 2024-09-19T07:43:27.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7efe79617640 1 --2- 172.21.15.16:0/3463368792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe740d09f0 0x7efe740ad920 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7efe79617640 1 --2- 172.21.15.16:0/3463368792 >> v2:172.21.15.16:3300/0 conn(0x7efe74099ae0 0x7efe740d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7efe79617640 1 -- 172.21.15.16:0/3463368792 >> 172.21.15.16:0/3463368792 conn(0x7efe7406bdf0 msgr2=0x7efe7406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7efe79617640 1 -- 172.21.15.16:0/3463368792 shutdown_connections 2024-09-19T07:43:27.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7efe79617640 1 -- 172.21.15.16:0/3463368792 wait complete. 2024-09-19T07:43:27.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.884+0000 7f0c81d82640 1 -- 172.21.15.16:0/2746436550 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c74007d60 msgr2=0x7f0c7403a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.884+0000 7f0c81d82640 1 --2- 172.21.15.16:0/2746436550 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c74007d60 0x7f0c7403a360 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0c700098a0 tx=0x7f0c7002a110 comp rx=0 tx=0).stop 2024-09-19T07:43:27.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.884+0000 7f0c81d82640 1 -- 172.21.15.16:0/2746436550 shutdown_connections 2024-09-19T07:43:27.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.884+0000 7f0c81d82640 1 --2- 172.21.15.16:0/2746436550 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c74007d60 0x7f0c7403a360 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.884+0000 7f0c81d82640 1 -- 172.21.15.16:0/2746436550 >> 172.21.15.16:0/2746436550 conn(0x7f0c7401ea40 msgr2=0x7f0c740556b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.884+0000 7f2a126ba640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2a0c0d04d0 0x7f2a0c160010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.884+0000 7f2a126ba640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a0c13fda0 con 0x7f2a0c072250 2024-09-19T07:43:27.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.884+0000 7f2a0bfff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a0c0d2750 0x7f2a0c13f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.884+0000 7f2a126ba640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2a0c145bd0 con 0x7f2a0c0d04d0 2024-09-19T07:43:27.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.884+0000 7f2a0bfff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a0c0d2750 0x7f2a0c13f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:34068/0 (socket says 172.21.15.16:34068) 2024-09-19T07:43:27.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.884+0000 7f2a0bfff640 1 -- 172.21.15.16:0/3188923630 learned_addr learned my addr 172.21.15.16:0/3188923630 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:27.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.884+0000 7fc710c96640 1 -- 172.21.15.16:0/1151270466 wait complete. 2024-09-19T07:43:27.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.884+0000 7f2a126ba640 1 -- 172.21.15.16:0/3188923630 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2a0c1427e0 con 0x7f2a0c0d2750 2024-09-19T07:43:27.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.884+0000 7f2a10c57640 1 --2- 172.21.15.16:0/3188923630 >> v2:172.21.15.146:3300/0 conn(0x7f2a0c0d04d0 0x7f2a0c160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.883+0000 7f493eca5640 1 -- 172.21.15.16:0/3741648697 wait complete. 2024-09-19T07:43:27.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.885+0000 7f493eca5640 1 Processor -- start 2024-09-19T07:43:27.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.885+0000 7f3c2eaba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c2806e0f0 0x7f3c280ce550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.885+0000 7f2a0bfff640 1 -- 172.21.15.16:0/3188923630 >> v2:172.21.15.146:3300/0 conn(0x7f2a0c0d04d0 msgr2=0x7f2a0c160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:27.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.885+0000 7f2a0bfff640 1 --2- 172.21.15.16:0/3188923630 >> v2:172.21.15.146:3300/0 conn(0x7f2a0c0d04d0 0x7f2a0c160010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.885+0000 7efe79617640 1 Processor -- start 2024-09-19T07:43:27.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.885+0000 7f2a0bfff640 1 -- 172.21.15.16:0/3188923630 >> v1:172.21.15.146:6789/0 conn(0x7f2a0c072250 legacy=0x7f2a0c0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.885+0000 7f2a0bfff640 1 -- 172.21.15.16:0/3188923630 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c0d2b30 con 0x7f2a0c0d2750 2024-09-19T07:43:27.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.885+0000 7efe79617640 1 -- start start 2024-09-19T07:43:27.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.885+0000 7f3c2eaba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c280b30c0 0x7f3c281618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.885+0000 7fec810af640 1 Processor -- start 2024-09-19T07:43:27.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.885+0000 7f493eca5640 1 -- start start 2024-09-19T07:43:27.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.885+0000 7f3c2c856640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c2806e0f0 0x7f3c280ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.885+0000 7f3c2c856640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c2806e0f0 0x7f3c280ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:34074/0 (socket says 172.21.15.16:34074) 2024-09-19T07:43:27.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.885+0000 7f3c2c856640 1 -- 172.21.15.16:0/1020464696 learned_addr learned my addr 172.21.15.16:0/1020464696 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:27.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.885+0000 7f3c2eaba640 1 --2- 172.21.15.16:0/1020464696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c2815c960 0x7f3c28161de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.885+0000 7fec810af640 1 -- start start 2024-09-19T07:43:27.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.886+0000 7f3c2eaba640 1 -- 172.21.15.16:0/1020464696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c28163760 con 0x7f3c280b30c0 2024-09-19T07:43:27.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.886+0000 7fc710c96640 1 Processor -- start 2024-09-19T07:43:27.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.886+0000 7f3c2d858640 1 --2- 172.21.15.16:0/1020464696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c2815c960 0x7f3c28161de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.886+0000 7f0c81d82640 1 -- 172.21.15.16:0/2746436550 shutdown_connections 2024-09-19T07:43:27.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.886+0000 7f3c2eaba640 1 -- 172.21.15.16:0/1020464696 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3c28162f40 con 0x7f3c2815c960 2024-09-19T07:43:27.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.886+0000 7f3c2eaba640 1 -- 172.21.15.16:0/1020464696 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2815cd40 con 0x7f3c2806e0f0 2024-09-19T07:43:27.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.886+0000 7fa32d67b640 1 Processor -- start 2024-09-19T07:43:27.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.886+0000 7fa32d67b640 1 -- start start 2024-09-19T07:43:27.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.886+0000 7f3c27fff640 1 --2- 172.21.15.16:0/1020464696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c280b30c0 0x7f3c281618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.886+0000 7f0c81d82640 1 -- 172.21.15.16:0/2746436550 wait complete. 2024-09-19T07:43:27.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.886+0000 7efe79617640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efe74099ae0 0x7efe7406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.886+0000 7f3c2d858640 1 -- 172.21.15.16:0/1020464696 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c2806e0f0 msgr2=0x7f3c280ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7efe79617640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe740c5690 0x7efe740c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7f3c2d858640 1 --2- 172.21.15.16:0/1020464696 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c2806e0f0 0x7f3c280ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7f3c2d858640 1 -- 172.21.15.16:0/1020464696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c280b30c0 msgr2=0x7f3c281618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7f493eca5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49380ad6a0 0x7f49381fa770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7efe79617640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe740d09f0 0x7efe7415c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7fec810af640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec7403ce20 0x7fec740dd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7f493eca5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49380b5010 0x7f49380ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7efe727fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe740c5690 0x7efe740c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7efe727fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe740c5690 0x7efe740c7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:57826/0 (socket says 172.21.15.16:57826) 2024-09-19T07:43:27.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7efe737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe740d09f0 0x7efe7415c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7efe737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe740d09f0 0x7efe7415c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57378/0 (socket says 172.21.15.16:57378) 2024-09-19T07:43:27.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7efe727fc640 1 -- 172.21.15.16:0/1487276428 learned_addr learned my addr 172.21.15.16:0/1487276428 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:27.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7f493eca5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49380c5630 0x7f4938203f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7efe79617640 1 -- 172.21.15.16:0/1487276428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efe740ad400 con 0x7efe740d09f0 2024-09-19T07:43:27.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7f3c2d858640 1 --2- 172.21.15.16:0/1020464696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c280b30c0 0x7f3c281618a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7f3c2d858640 1 -- 172.21.15.16:0/1020464696 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c28162320 con 0x7f3c2815c960 2024-09-19T07:43:27.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7f493eca5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f49380aef30 con 0x7f49380ad6a0 2024-09-19T07:43:27.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7efe79617640 1 -- 172.21.15.16:0/1487276428 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efe74163c40 con 0x7efe740c5690 2024-09-19T07:43:27.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7f3c27fff640 1 --2- 172.21.15.16:0/1020464696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c280b30c0 0x7f3c281618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:27.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7efe79617640 1 -- 172.21.15.16:0/1487276428 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efe740c5a70 con 0x7efe74099ae0 2024-09-19T07:43:27.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7f4937fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49380b5010 0x7f49380ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7f4937fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49380b5010 0x7f49380ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:57838/0 (socket says 172.21.15.16:57838) 2024-09-19T07:43:27.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7f4937fff640 1 -- 172.21.15.16:0/1559289191 learned_addr learned my addr 172.21.15.16:0/1559289191 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:27.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7f493eca5640 1 -- 172.21.15.16:0/1559289191 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f49380ae730 con 0x7f49380b5010 2024-09-19T07:43:27.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7f493eca5640 1 -- 172.21.15.16:0/1559289191 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f49380adb50 con 0x7f49380c5630 2024-09-19T07:43:27.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7fa32d67b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3280d0df0 0x7fa32809e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7fec810af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec740535b0 0x7fec740393b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7f3c2d858640 1 --2- 172.21.15.16:0/1020464696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c2815c960 0x7f3c28161de0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3c14031000 tx=0x7f3c140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7fa32d67b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa3280d2980 0x7fa32809fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.886+0000 7fc710c96640 1 -- start start 2024-09-19T07:43:27.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7fc710c96640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc704005a60 0x7fc7040db780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7fc710c96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc70404eb60 0x7fc7040e3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7fc710c96640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc70400ac70 0x7fc70400b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7fc710c96640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc704053a10 con 0x7fc70404eb60 2024-09-19T07:43:27.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7fc710c96640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc704007240 con 0x7fc704005a60 2024-09-19T07:43:27.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7fc710c96640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc70401dc40 con 0x7fc70400ac70 2024-09-19T07:43:27.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7fa32d67b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3281626c0 0x7fa3280cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7fa32d67b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3280d2dd0 con 0x7fa3281626c0 2024-09-19T07:43:27.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7fa32d67b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa328163b10 con 0x7fa3280d0df0 2024-09-19T07:43:27.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.887+0000 7fa32d67b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa3280cbe10 con 0x7fa3280d2980 2024-09-19T07:43:27.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7fec810af640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec740e72f0 0x7fec740398f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7fec810af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec740573d0 con 0x7fec740535b0 2024-09-19T07:43:27.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7fec810af640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fec74057250 con 0x7fec7403ce20 2024-09-19T07:43:27.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7f493ca41640 1 --2- 172.21.15.16:0/1559289191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49380ad6a0 0x7f49381fa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7fec810af640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fec74057550 con 0x7fec740e72f0 2024-09-19T07:43:27.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7f2a0bfff640 1 --2- 172.21.15.16:0/3188923630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a0c0d2750 0x7f2a0c13f860 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009e50 tx=0x7f2a0402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f280d54006612f62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7fec7bfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec7403ce20 0x7fec740dd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7f4937fff640 1 -- 172.21.15.16:0/1559289191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49380c5630 msgr2=0x7f4938203f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7f4937fff640 1 --2- 172.21.15.16:0/1559289191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49380c5630 0x7f4938203f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7f4937fff640 1 -- 172.21.15.16:0/1559289191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49380ad6a0 msgr2=0x7f49381fa770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7f3c257fa640 1 -- 172.21.15.16:0/1020464696 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14042020 con 0x7f3c2815c960 2024-09-19T07:43:27.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7f3c257fa640 1 -- 172.21.15.16:0/1020464696 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c1403d470 con 0x7f3c2815c960 2024-09-19T07:43:27.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7f3c257fa640 1 -- 172.21.15.16:0/1020464696 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14037040 con 0x7f3c2815c960 2024-09-19T07:43:27.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7fc70affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc70404eb60 0x7fc7040e3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7fc70affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc70404eb60 0x7fc7040e3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57400/0 (socket says 172.21.15.16:57400) 2024-09-19T07:43:27.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7fc70affd640 1 -- 172.21.15.16:0/3377319321 learned_addr learned my addr 172.21.15.16:0/3377319321 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:27.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7fa326ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3280d0df0 0x7fa32809e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7f0c81d82640 1 Processor -- start 2024-09-19T07:43:27.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7f0c81d82640 1 -- start start 2024-09-19T07:43:27.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7fa3277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3281626c0 0x7fa3280cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7fa3277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3281626c0 0x7fa3280cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57410/0 (socket says 172.21.15.16:57410) 2024-09-19T07:43:27.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7f4937fff640 1 --2- 172.21.15.16:0/1559289191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49380ad6a0 0x7f49381fa770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7f4937fff640 1 -- 172.21.15.16:0/1559289191 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49380cb3e0 con 0x7f49380b5010 2024-09-19T07:43:27.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7fa326ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3280d0df0 0x7fa32809e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:57846/0 (socket says 172.21.15.16:57846) 2024-09-19T07:43:27.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7fa3277fe640 1 -- 172.21.15.16:0/1086342658 learned_addr learned my addr 172.21.15.16:0/1086342658 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:27.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7f3c2eaba640 1 -- 172.21.15.16:0/1020464696 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c28160180 con 0x7f3c2815c960 2024-09-19T07:43:27.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7f3c2eaba640 1 -- 172.21.15.16:0/1020464696 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c28160740 con 0x7f3c2815c960 2024-09-19T07:43:27.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7fc70b7fe640 1 --2- 172.21.15.16:0/3377319321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc704005a60 0x7fc7040db780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7f493ca41640 1 --2- 172.21.15.16:0/1559289191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49380ad6a0 0x7f49381fa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:27.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7efe727fc640 1 -- 172.21.15.16:0/1487276428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efe74099ae0 msgr2=0x7efe7406e8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T07:43:27.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7efe727fc640 1 --2- 172.21.15.16:0/1487276428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efe74099ae0 0x7efe7406e8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7efe727fc640 1 -- 172.21.15.16:0/1487276428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe740d09f0 msgr2=0x7efe7415c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7efe727fc640 1 --2- 172.21.15.16:0/1487276428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe740d09f0 0x7efe7415c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7efe727fc640 1 -- 172.21.15.16:0/1487276428 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7406edf0 con 0x7efe740c5690 2024-09-19T07:43:27.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7f2a0affd640 1 -- 172.21.15.16:0/3188923630 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04036070 con 0x7f2a0c0d2750 2024-09-19T07:43:27.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7fc70bfff640 1 --2- 172.21.15.16:0/3377319321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc70400ac70 0x7fc70400b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7f2a0affd640 1 -- 172.21.15.16:0/3188923630 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a04031600 con 0x7f2a0c0d2750 2024-09-19T07:43:27.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7f4937fff640 1 --2- 172.21.15.16:0/1559289191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49380b5010 0x7f49380ccff0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f492c0090b0 tx=0x7f492c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7efe737fe640 1 --2- 172.21.15.16:0/1487276428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe740d09f0 0x7efe7415c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:27.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7f2a0affd640 1 -- 172.21.15.16:0/3188923630 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04036070 con 0x7f2a0c0d2750 2024-09-19T07:43:27.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7fec7bfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec7403ce20 0x7fec740dd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:57844/0 (socket says 172.21.15.16:57844) 2024-09-19T07:43:27.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7fec7bfff640 1 -- 172.21.15.16:0/2426048415 learned_addr learned my addr 172.21.15.16:0/2426048415 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:27.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7efe727fc640 1 --2- 172.21.15.16:0/1487276428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe740c5690 0x7efe740c7520 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efe6400e9e0 tx=0x7efe6400eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7fa3267fc640 1 --2- 172.21.15.16:0/1086342658 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa3280d2980 0x7fa32809fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7fc70bfff640 1 -- 172.21.15.16:0/3377319321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc704005a60 msgr2=0x7fc7040db780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7fc70bfff640 1 --2- 172.21.15.16:0/3377319321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc704005a60 0x7fc7040db780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7fc70bfff640 1 -- 172.21.15.16:0/3377319321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc70404eb60 msgr2=0x7fc7040e3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7fc70bfff640 1 --2- 172.21.15.16:0/3377319321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc70404eb60 0x7fc7040e3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7fc70bfff640 1 -- 172.21.15.16:0/3377319321 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7040dc210 con 0x7fc70400ac70 2024-09-19T07:43:27.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7efe5b7fe640 1 -- 172.21.15.16:0/1487276428 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64017070 con 0x7efe740c5690 2024-09-19T07:43:27.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7fc70bfff640 1 --2- 172.21.15.16:0/3377319321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc70400ac70 0x7fc70400b120 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc70c06a560 tx=0x7fc70c069710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7efe79617640 1 -- 172.21.15.16:0/1487276428 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe7406f090 con 0x7efe740c5690 2024-09-19T07:43:27.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7fec7b7fe640 1 --2- 172.21.15.16:0/2426048415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec740535b0 0x7fec740393b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7fc6f7fff640 1 -- 172.21.15.16:0/3377319321 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c08f070 con 0x7fc70400ac70 2024-09-19T07:43:27.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7efe79617640 1 -- 172.21.15.16:0/1487276428 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe740ce4c0 con 0x7efe740c5690 2024-09-19T07:43:27.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.890+0000 7efe5b7fe640 1 -- 172.21.15.16:0/1487276428 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6400c970 con 0x7efe740c5690 2024-09-19T07:43:27.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.890+0000 7efe5b7fe640 1 -- 172.21.15.16:0/1487276428 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6400cd60 con 0x7efe740c5690 2024-09-19T07:43:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.890+0000 7f2a126ba640 1 -- 172.21.15.16:0/3188923630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a0c0d2750 msgr2=0x7f2a0c13f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.890+0000 7f2a126ba640 1 --2- 172.21.15.16:0/3188923630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a0c0d2750 0x7f2a0c13f860 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009e50 tx=0x7f2a0402a680 comp rx=0 tx=0).stop 2024-09-19T07:43:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.890+0000 7f2a126ba640 1 -- 172.21.15.16:0/3188923630 shutdown_connections 2024-09-19T07:43:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7fc710c96640 1 -- 172.21.15.16:0/3377319321 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7040dbcc0 con 0x7fc70400ac70 2024-09-19T07:43:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.890+0000 7fc710c96640 1 -- 172.21.15.16:0/3377319321 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70417d640 con 0x7fc70400ac70 2024-09-19T07:43:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.890+0000 7f3c2eaba640 1 -- 172.21.15.16:0/1020464696 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3c08001030 con 0x7f3c2815c960 2024-09-19T07:43:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7f0c81d82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c74007d60 0x7f0c74036df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7f0c81d82640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c74034470 0x7f0c740da4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7f0c81d82640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c740349c0 0x7f0c74035f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7f0c81d82640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0c740564d0 con 0x7f0c74007d60 2024-09-19T07:43:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7f0c81d82640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0c74037b10 con 0x7f0c740349c0 2024-09-19T07:43:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7f0c81d82640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0c7400cb60 con 0x7f0c74034470 2024-09-19T07:43:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.890+0000 7f0c7bfff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c74034470 0x7f0c740da4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.890+0000 7f0c7bfff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c74034470 0x7f0c740da4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:34138/0 (socket says 172.21.15.16:34138) 2024-09-19T07:43:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.890+0000 7f0c7bfff640 1 -- 172.21.15.16:0/4212779306 learned_addr learned my addr 172.21.15.16:0/4212779306 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:27.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.890+0000 7f0c80d80640 1 --2- 172.21.15.16:0/4212779306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c74007d60 0x7f0c74036df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.890+0000 7f0c81581640 1 --2- 172.21.15.16:0/4212779306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c740349c0 0x7f0c74035f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.888+0000 7fec808ae640 1 --2- 172.21.15.16:0/2426048415 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec740e72f0 0x7fec740398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7fec7b7fe640 1 -- 172.21.15.16:0/2426048415 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec740e72f0 msgr2=0x7fec740398f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7fec7b7fe640 1 --2- 172.21.15.16:0/2426048415 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec740e72f0 0x7fec740398f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7fec7b7fe640 1 -- 172.21.15.16:0/2426048415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec7403ce20 msgr2=0x7fec740dd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7fec7b7fe640 1 --2- 172.21.15.16:0/2426048415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec7403ce20 0x7fec740dd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.890+0000 7fec7b7fe640 1 -- 172.21.15.16:0/2426048415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7404d530 con 0x7fec740535b0 2024-09-19T07:43:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7f49357fa640 1 -- 172.21.15.16:0/1559289191 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c040020 con 0x7f49380b5010 2024-09-19T07:43:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7f49357fa640 1 -- 172.21.15.16:0/1559289191 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f492c038040 con 0x7f49380b5010 2024-09-19T07:43:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7f49357fa640 1 -- 172.21.15.16:0/1559289191 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c037070 con 0x7f49380b5010 2024-09-19T07:43:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.890+0000 7f493eca5640 1 -- 172.21.15.16:0/1559289191 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49380cb6c0 con 0x7f49380b5010 2024-09-19T07:43:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7fa3277fe640 1 -- 172.21.15.16:0/1086342658 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa3280d2980 msgr2=0x7fa32809fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7fa3277fe640 1 --2- 172.21.15.16:0/1086342658 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa3280d2980 0x7fa32809fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7fa3277fe640 1 -- 172.21.15.16:0/1086342658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3280d0df0 msgr2=0x7fa32809e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7fa3277fe640 1 --2- 172.21.15.16:0/1086342658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3280d0df0 0x7fa32809e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.889+0000 7fa3277fe640 1 -- 172.21.15.16:0/1086342658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3280cc0f0 con 0x7fa3281626c0 2024-09-19T07:43:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.890+0000 7f493eca5640 1 -- 172.21.15.16:0/1559289191 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49380cbc80 con 0x7f49380b5010 2024-09-19T07:43:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.890+0000 7fa3277fe640 1 --2- 172.21.15.16:0/1086342658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3281626c0 0x7fa3280cceb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa31c02a2b0 tx=0x7fa31c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.891+0000 7f0c80d80640 1 -- 172.21.15.16:0/4212779306 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c74034470 msgr2=0x7f0c740da4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.891+0000 7f0c80d80640 1 --2- 172.21.15.16:0/4212779306 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c74034470 0x7f0c740da4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.891+0000 7f0c80d80640 1 -- 172.21.15.16:0/4212779306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c740349c0 msgr2=0x7f0c74035f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.891+0000 7f0c80d80640 1 --2- 172.21.15.16:0/4212779306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c740349c0 0x7f0c74035f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.891+0000 7f0c80d80640 1 -- 172.21.15.16:0/4212779306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c7401f3b0 con 0x7f0c74007d60 2024-09-19T07:43:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.891+0000 7fec7b7fe640 1 --2- 172.21.15.16:0/2426048415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec740535b0 0x7fec740393b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fec74055410 tx=0x7fec700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.890+0000 7fc6f7fff640 1 -- 172.21.15.16:0/3377319321 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc70c0754a0 con 0x7fc70400ac70 2024-09-19T07:43:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.891+0000 7fc6f7fff640 1 -- 172.21.15.16:0/3377319321 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c067b90 con 0x7fc70400ac70 2024-09-19T07:43:27.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.891+0000 7f0c80d80640 1 --2- 172.21.15.16:0/4212779306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c74007d60 0x7f0c74036df0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0518b0 tx=0x7f0c7c069710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.891+0000 7f2a126ba640 1 --2- 172.21.15.16:0/3188923630 >> v2:172.21.15.146:3300/0 conn(0x7f2a0c0d04d0 0x7f2a0c160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.891+0000 7f2a126ba640 1 --2- 172.21.15.16:0/3188923630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a0c0d2750 0x7f2a0c13f860 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.891+0000 7f2a126ba640 1 -- 172.21.15.16:0/3188923630 >> 172.21.15.16:0/3188923630 conn(0x7f2a0c0b4040 msgr2=0x7f2a0c0b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.891+0000 7fa30f7fe640 1 -- 172.21.15.16:0/1086342658 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c041070 con 0x7fa3281626c0 2024-09-19T07:43:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.891+0000 7fa30f7fe640 1 -- 172.21.15.16:0/1086342658 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa31c03c870 con 0x7fa3281626c0 2024-09-19T07:43:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.891+0000 7fa32d67b640 1 -- 172.21.15.16:0/1086342658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3280cc3d0 con 0x7fa3281626c0 2024-09-19T07:43:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.891+0000 7f0c797fa640 1 -- 172.21.15.16:0/4212779306 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c0754a0 con 0x7f0c74007d60 2024-09-19T07:43:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.892+0000 7f0c797fa640 1 -- 172.21.15.16:0/4212779306 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c7c075640 con 0x7f0c74007d60 2024-09-19T07:43:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.892+0000 7efe5b7fe640 1 -- 172.21.15.16:0/1487276428 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7efe6401a070 con 0x7efe740c5690 2024-09-19T07:43:27.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.892+0000 7fa32d67b640 1 -- 172.21.15.16:0/1086342658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3280cc990 con 0x7fa3281626c0 2024-09-19T07:43:27.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.891+0000 7f3c257fa640 1 -- 172.21.15.16:0/1020464696 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3c14045020 con 0x7f3c2815c960 2024-09-19T07:43:27.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.891+0000 7f3c257fa640 1 --2- 172.21.15.16:0/1020464696 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3c0c06c380 0x7f3c0c06e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.892+0000 7f3c257fa640 1 -- 172.21.15.16:0/1020464696 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3c140abfb0 con 0x7f3c2815c960 2024-09-19T07:43:27.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.892+0000 7f3c257fa640 1 --2- 172.21.15.16:0/1020464696 >> [v2:172.21.15.16:6808/2159059174,v1:172.21.15.16:6809/2159059174] conn(0x7f3c0c075a80 0x7f3c0c077ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.892+0000 7f3c257fa640 1 -- 172.21.15.16:0/1020464696 --> [v2:172.21.15.16:6808/2159059174,v1:172.21.15.16:6809/2159059174] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3c14001d30 con 0x7f3c0c075a80 2024-09-19T07:43:27.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.892+0000 7f2a126ba640 1 -- 172.21.15.16:0/3188923630 shutdown_connections 2024-09-19T07:43:27.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.892+0000 7f3c257fa640 1 -- 172.21.15.16:0/1020464696 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c14075f60 con 0x7f3c2815c960 2024-09-19T07:43:27.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.892+0000 7f3c2c856640 1 --2- 172.21.15.16:0/1020464696 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3c0c06c380 0x7f3c0c06e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.893+0000 7f3c2c856640 1 --2- 172.21.15.16:0/1020464696 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3c0c06c380 0x7f3c0c06e840 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0094a0 tx=0x7f3c1c009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.893+0000 7f3c27fff640 1 --2- 172.21.15.16:0/1020464696 >> [v2:172.21.15.16:6808/2159059174,v1:172.21.15.16:6809/2159059174] conn(0x7f3c0c075a80 0x7f3c0c077ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.892+0000 7f0c81d82640 1 -- 172.21.15.16:0/4212779306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c7401f670 con 0x7f0c74007d60 2024-09-19T07:43:27.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.892+0000 7f0c81d82640 1 -- 172.21.15.16:0/4212779306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c740376f0 con 0x7f0c74007d60 2024-09-19T07:43:27.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.893+0000 7f0c797fa640 1 -- 172.21.15.16:0/4212779306 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c093640 con 0x7f0c74007d60 2024-09-19T07:43:27.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.893+0000 7f0c66ffd640 1 -- 172.21.15.16:0/4212779306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0c54001030 con 0x7f0c74007d60 2024-09-19T07:43:27.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.892+0000 7fec78ff9640 1 -- 172.21.15.16:0/2426048415 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec70037040 con 0x7fec740535b0 2024-09-19T07:43:27.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.892+0000 7fec78ff9640 1 -- 172.21.15.16:0/2426048415 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec70042070 con 0x7fec740535b0 2024-09-19T07:43:27.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.892+0000 7fec78ff9640 1 -- 172.21.15.16:0/2426048415 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec70030bb0 con 0x7fec740535b0 2024-09-19T07:43:27.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.893+0000 7fec810af640 1 -- 172.21.15.16:0/2426048415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7404d810 con 0x7fec740535b0 2024-09-19T07:43:27.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.893+0000 7fec810af640 1 -- 172.21.15.16:0/2426048415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec7404ddd0 con 0x7fec740535b0 2024-09-19T07:43:27.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.893+0000 7f491effd640 1 -- 172.21.15.16:0/1559289191 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4938068da0 con 0x7f49380b5010 2024-09-19T07:43:27.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.893+0000 7fc710c96640 1 -- 172.21.15.16:0/3377319321 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc6ec001030 con 0x7fc70400ac70 2024-09-19T07:43:27.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.892+0000 7efe5b7fe640 1 --2- 172.21.15.16:0/1487276428 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efe6006c380 0x7efe6006e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.892+0000 7efe5b7fe640 1 -- 172.21.15.16:0/1487276428 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7efe6408b590 con 0x7efe740c5690 2024-09-19T07:43:27.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.893+0000 7efe79617640 1 --2- 172.21.15.16:0/1487276428 >> [v2:172.21.15.146:6816/2651212753,v1:172.21.15.146:6817/2651212753] conn(0x7efe54001650 0x7efe54003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.893+0000 7efe79617640 1 -- 172.21.15.16:0/1487276428 --> [v2:172.21.15.146:6816/2651212753,v1:172.21.15.146:6817/2651212753] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efe54006bb0 con 0x7efe54001650 2024-09-19T07:43:27.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.892+0000 7f2a126ba640 1 -- 172.21.15.16:0/3188923630 wait complete. 2024-09-19T07:43:27.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.894+0000 7efe737fe640 1 --2- 172.21.15.16:0/1487276428 >> [v2:172.21.15.146:6816/2651212753,v1:172.21.15.146:6817/2651212753] conn(0x7efe54001650 0x7efe54003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.894+0000 7fc6f7fff640 1 -- 172.21.15.16:0/3377319321 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc70c068020 con 0x7fc70400ac70 2024-09-19T07:43:27.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.893+0000 7fa30f7fe640 1 -- 172.21.15.16:0/1086342658 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c037040 con 0x7fa3281626c0 2024-09-19T07:43:27.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.894+0000 7f2a126ba640 1 Processor -- start 2024-09-19T07:43:27.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.894+0000 7fa32d67b640 1 -- 172.21.15.16:0/1086342658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa3280d1240 con 0x7fa3281626c0 2024-09-19T07:43:27.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.894+0000 7fec810af640 1 -- 172.21.15.16:0/2426048415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec7404e1b0 con 0x7fec740535b0 2024-09-19T07:43:27.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.895+0000 7f2a126ba640 1 -- start start 2024-09-19T07:43:27.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.895+0000 7efe72ffd640 1 --2- 172.21.15.16:0/1487276428 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efe6006c380 0x7efe6006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.895+0000 7fc6f7fff640 1 --2- 172.21.15.16:0/3377319321 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc6f006c3b0 0x7fc6f006e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.895+0000 7fc6f7fff640 1 -- 172.21.15.16:0/3377319321 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc70c106540 con 0x7fc70400ac70 2024-09-19T07:43:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.895+0000 7fc70b7fe640 1 --2- 172.21.15.16:0/3377319321 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc6f006c3b0 0x7fc6f006e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.895+0000 7fc6f7fff640 1 --2- 172.21.15.16:0/3377319321 >> [v2:172.21.15.146:6808/2073273152,v1:172.21.15.146:6809/2073273152] conn(0x7fc6f0075b30 0x7fc6f0077f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.895+0000 7fc6f7fff640 1 -- 172.21.15.16:0/3377319321 --> [v2:172.21.15.146:6808/2073273152,v1:172.21.15.146:6809/2073273152] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc70c070e50 con 0x7fc6f0075b30 2024-09-19T07:43:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.895+0000 7fc6f7fff640 1 -- 172.21.15.16:0/3377319321 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc70c10d050 con 0x7fc70400ac70 2024-09-19T07:43:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.894+0000 7f49357fa640 1 -- 172.21.15.16:0/1559289191 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f492c043020 con 0x7f49380b5010 2024-09-19T07:43:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.895+0000 7f0c797fa640 1 -- 172.21.15.16:0/4212779306 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0c7c0937e0 con 0x7f0c74007d60 2024-09-19T07:43:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7fc70affd640 1 --2- 172.21.15.16:0/3377319321 >> [v2:172.21.15.146:6808/2073273152,v1:172.21.15.146:6809/2073273152] conn(0x7fc6f0075b30 0x7fc6f0077f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7fc70b7fe640 1 --2- 172.21.15.16:0/3377319321 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc6f006c3b0 0x7fc6f006e870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009b90 tx=0x7fc6f8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f0c797fa640 1 --2- 172.21.15.16:0/4212779306 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0c6006c300 0x7f0c6006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7efe72ffd640 1 --2- 172.21.15.16:0/1487276428 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efe6006c380 0x7efe6006e840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efe68001580 tx=0x7efe68007a40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7efe737fe640 1 --2- 172.21.15.16:0/1487276428 >> [v2:172.21.15.146:6816/2651212753,v1:172.21.15.146:6817/2651212753] conn(0x7efe54001650 0x7efe54003b10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f2a126ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c072250 0x7f2a0c15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.894+0000 7f49357fa640 1 --2- 172.21.15.16:0/1559289191 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f492406c380 0x7f492406e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.895+0000 7f49357fa640 1 -- 172.21.15.16:0/1559289191 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f492c0abd10 con 0x7f49380b5010 2024-09-19T07:43:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.895+0000 7f493ca41640 1 --2- 172.21.15.16:0/1559289191 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f492406c380 0x7f492406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f0c797fa640 1 -- 172.21.15.16:0/4212779306 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0c7c1071f0 con 0x7f0c74007d60 2024-09-19T07:43:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f0c797fa640 1 --2- 172.21.15.16:0/4212779306 >> [v2:172.21.15.16:6824/3030558062,v1:172.21.15.16:6825/3030558062] conn(0x7f0c60075a80 0x7f0c60077ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f0c7bfff640 1 --2- 172.21.15.16:0/4212779306 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0c6006c300 0x7f0c6006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f49357fa640 1 --2- 172.21.15.16:0/1559289191 >> [v2:172.21.15.16:6816/3748453908,v1:172.21.15.16:6817/3748453908] conn(0x7f4924075b00 0x7f4924077f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f49357fa640 1 -- 172.21.15.16:0/1559289191 --> [v2:172.21.15.16:6816/3748453908,v1:172.21.15.16:6817/3748453908] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f492c032440 con 0x7f4924075b00 2024-09-19T07:43:27.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f2a126ba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a0c0d04d0 0x7f2a0c15e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f0c797fa640 1 -- 172.21.15.16:0/4212779306 --> [v2:172.21.15.16:6824/3030558062,v1:172.21.15.16:6825/3030558062] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c7c08ce50 con 0x7f0c60075a80 2024-09-19T07:43:27.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f0c797fa640 1 -- 172.21.15.16:0/4212779306 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0c7c1075a0 con 0x7f0c74007d60 2024-09-19T07:43:27.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7fec78ff9640 1 -- 172.21.15.16:0/2426048415 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fec70047460 con 0x7fec740535b0 2024-09-19T07:43:27.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f2a0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c072250 0x7f2a0c15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f2a0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c072250 0x7f2a0c15c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57440/0 (socket says 172.21.15.16:57440) 2024-09-19T07:43:27.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7fc70affd640 1 --2- 172.21.15.16:0/3377319321 >> [v2:172.21.15.146:6808/2073273152,v1:172.21.15.146:6809/2073273152] conn(0x7fc6f0075b30 0x7fc6f0077f70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f2a126ba640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a0c0d2750 0x7f2a0c15b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.895+0000 7f3c27fff640 1 --2- 172.21.15.16:0/1020464696 >> [v2:172.21.15.16:6808/2159059174,v1:172.21.15.16:6809/2159059174] conn(0x7f3c0c075a80 0x7f3c0c077ec0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f493d242640 1 --2- 172.21.15.16:0/1559289191 >> [v2:172.21.15.16:6816/3748453908,v1:172.21.15.16:6817/3748453908] conn(0x7f4924075b00 0x7f4924077f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f49357fa640 1 -- 172.21.15.16:0/1559289191 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f492c0b1300 con 0x7f49380b5010 2024-09-19T07:43:27.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7fa30f7fe640 1 -- 172.21.15.16:0/1086342658 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa31c044070 con 0x7fa3281626c0 2024-09-19T07:43:27.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f2a0b7fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a0c0d04d0 0x7f2a0c15e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f2a126ba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c142220 con 0x7f2a0c072250 2024-09-19T07:43:27.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f0c81581640 1 --2- 172.21.15.16:0/4212779306 >> [v2:172.21.15.16:6824/3030558062,v1:172.21.15.16:6825/3030558062] conn(0x7f0c60075a80 0x7f0c60077ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f2a126ba640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c141f40 con 0x7f2a0c0d2750 2024-09-19T07:43:27.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f2a126ba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2a0c142500 con 0x7f2a0c0d04d0 2024-09-19T07:43:27.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.896+0000 7f2a0bfff640 1 -- 172.21.15.16:0/1839668392 learned_addr learned my addr 172.21.15.16:0/1839668392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:27.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7f3c257fa640 1 -- 172.21.15.16:0/1020464696 <== osd.0 v2:172.21.15.16:6808/2159059174 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3c14001d30 con 0x7f3c0c075a80 2024-09-19T07:43:27.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7f2a10c57640 1 --2- 172.21.15.16:0/1839668392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a0c0d2750 0x7f2a0c15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7fec78ff9640 1 --2- 172.21.15.16:0/2426048415 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fec6806c3b0 0x7fec6806e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7fec78ff9640 1 -- 172.21.15.16:0/2426048415 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fec700acd50 con 0x7fec740535b0 2024-09-19T07:43:27.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7f493ca41640 1 --2- 172.21.15.16:0/1559289191 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f492406c380 0x7f492406e840 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f49380d06e0 tx=0x7f4930007650 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7fa30f7fe640 1 --2- 172.21.15.16:0/1086342658 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa31406c380 0x7fa31406e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7f0c81581640 1 --2- 172.21.15.16:0/4212779306 >> [v2:172.21.15.16:6824/3030558062,v1:172.21.15.16:6825/3030558062] conn(0x7f0c60075a80 0x7f0c60077ec0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7f0c7bfff640 1 --2- 172.21.15.16:0/4212779306 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0c6006c300 0x7f0c6006e7c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400bb90 tx=0x7f0c68009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7fec78ff9640 1 --2- 172.21.15.16:0/2426048415 >> [v2:172.21.15.16:6800/4074420552,v1:172.21.15.16:6801/4074420552] conn(0x7fec68075b30 0x7fec68077f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7fec78ff9640 1 -- 172.21.15.16:0/2426048415 --> [v2:172.21.15.16:6800/4074420552,v1:172.21.15.16:6801/4074420552] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec70036e90 con 0x7fec68075b30 2024-09-19T07:43:27.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7fec78ff9640 1 -- 172.21.15.16:0/2426048415 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec700ad140 con 0x7fec740535b0 2024-09-19T07:43:27.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7fec808ae640 1 --2- 172.21.15.16:0/2426048415 >> [v2:172.21.15.16:6800/4074420552,v1:172.21.15.16:6801/4074420552] conn(0x7fec68075b30 0x7fec68077f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7fc6f7fff640 1 -- 172.21.15.16:0/3377319321 <== osd.4 v2:172.21.15.146:6808/2073273152 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc70c070e50 con 0x7fc6f0075b30 2024-09-19T07:43:27.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7fa30f7fe640 1 -- 172.21.15.16:0/1086342658 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa31c0ac4b0 con 0x7fa3281626c0 2024-09-19T07:43:27.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7fa326ffd640 1 --2- 172.21.15.16:0/1086342658 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa31406c380 0x7fa31406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7f2a0bfff640 1 -- 172.21.15.16:0/1839668392 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a0c0d04d0 msgr2=0x7f2a0c15e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7f2a0bfff640 1 --2- 172.21.15.16:0/1839668392 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a0c0d04d0 0x7f2a0c15e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7f2a0bfff640 1 -- 172.21.15.16:0/1839668392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a0c0d2750 msgr2=0x7f2a0c15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7f2a0bfff640 1 --2- 172.21.15.16:0/1839668392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a0c0d2750 0x7f2a0c15b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7f2a0bfff640 1 -- 172.21.15.16:0/1839668392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c0b60a0 con 0x7f2a0c072250 2024-09-19T07:43:27.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.898+0000 7fa30f7fe640 1 --2- 172.21.15.16:0/1086342658 >> [v2:172.21.15.146:6800/2692587126,v1:172.21.15.146:6801/2692587126] conn(0x7fa314075af0 0x7fa314077f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.898+0000 7fa30f7fe640 1 -- 172.21.15.16:0/1086342658 --> [v2:172.21.15.146:6800/2692587126,v1:172.21.15.146:6801/2692587126] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa31c032440 con 0x7fa314075af0 2024-09-19T07:43:27.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.898+0000 7fa30f7fe640 1 -- 172.21.15.16:0/1086342658 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa31c0b3050 con 0x7fa3281626c0 2024-09-19T07:43:27.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.898+0000 7efe5b7fe640 1 -- 172.21.15.16:0/1487276428 <== osd.5 v2:172.21.15.146:6816/2651212753 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe54006bb0 con 0x7efe54001650 2024-09-19T07:43:27.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.897+0000 7f2a0b7fe640 1 --2- 172.21.15.16:0/1839668392 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a0c0d04d0 0x7f2a0c15e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:27.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.898+0000 7f2a0bfff640 1 --2- 172.21.15.16:0/1839668392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c072250 0x7f2a0c15c3e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009f70 tx=0x7f2a04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.898+0000 7fa3267fc640 1 --2- 172.21.15.16:0/1086342658 >> [v2:172.21.15.146:6800/2692587126,v1:172.21.15.146:6801/2692587126] conn(0x7fa314075af0 0x7fa314077f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.898+0000 7fec7bfff640 1 --2- 172.21.15.16:0/2426048415 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fec6806c3b0 0x7fec6806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.898+0000 7fec808ae640 1 --2- 172.21.15.16:0/2426048415 >> [v2:172.21.15.16:6800/4074420552,v1:172.21.15.16:6801/4074420552] conn(0x7fec68075b30 0x7fec68077f70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.898+0000 7f493d242640 1 --2- 172.21.15.16:0/1559289191 >> [v2:172.21.15.16:6816/3748453908,v1:172.21.15.16:6817/3748453908] conn(0x7f4924075b00 0x7f4924077f40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.898+0000 7f0c797fa640 1 -- 172.21.15.16:0/4212779306 <== osd.1 v2:172.21.15.16:6824/3030558062 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0c7c08ce50 con 0x7f0c60075a80 2024-09-19T07:43:27.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.899+0000 7fa3267fc640 1 --2- 172.21.15.16:0/1086342658 >> [v2:172.21.15.146:6800/2692587126,v1:172.21.15.146:6801/2692587126] conn(0x7fa314075af0 0x7fa314077f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.899+0000 7fec7bfff640 1 --2- 172.21.15.16:0/2426048415 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fec6806c3b0 0x7fec6806e870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec6c005880 tx=0x7fec6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.898+0000 7f2a08ff9640 1 -- 172.21.15.16:0/1839668392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04041070 con 0x7f2a0c072250 2024-09-19T07:43:27.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.898+0000 7f2a08ff9640 1 -- 172.21.15.16:0/1839668392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a04007a30 con 0x7f2a0c072250 2024-09-19T07:43:27.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.898+0000 7f2a08ff9640 1 -- 172.21.15.16:0/1839668392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04036070 con 0x7f2a0c072250 2024-09-19T07:43:27.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.899+0000 7f2a126ba640 1 -- 172.21.15.16:0/1839668392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0c0cbac0 con 0x7f2a0c072250 2024-09-19T07:43:27.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.899+0000 7f2a126ba640 1 -- 172.21.15.16:0/1839668392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0c15cd10 con 0x7f2a0c072250 2024-09-19T07:43:27.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.899+0000 7fec78ff9640 1 -- 172.21.15.16:0/2426048415 <== osd.2 v2:172.21.15.16:6800/4074420552 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec70036e90 con 0x7fec68075b30 2024-09-19T07:43:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.899+0000 7fa326ffd640 1 --2- 172.21.15.16:0/1086342658 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa31406c380 0x7fa31406e840 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa320006fd0 tx=0x7fa320009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.899+0000 7fa30f7fe640 1 -- 172.21.15.16:0/1086342658 <== osd.6 v2:172.21.15.146:6800/2692587126 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa31c032440 con 0x7fa314075af0 2024-09-19T07:43:27.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.900+0000 7f49357fa640 1 -- 172.21.15.16:0/1559289191 <== osd.3 v2:172.21.15.16:6816/3748453908 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f492c032440 con 0x7f4924075b00 2024-09-19T07:43:27.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.901+0000 7f29f27fc640 1 -- 172.21.15.16:0/1839668392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2a0c1418b0 con 0x7f2a0c072250 2024-09-19T07:43:27.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.901+0000 7f2a08ff9640 1 -- 172.21.15.16:0/1839668392 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2a04044070 con 0x7f2a0c072250 2024-09-19T07:43:27.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.902+0000 7f2a08ff9640 1 --2- 172.21.15.16:0/1839668392 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f29f806c300 0x7f29f806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.902+0000 7f2a0b7fe640 1 --2- 172.21.15.16:0/1839668392 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f29f806c300 0x7f29f806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.903+0000 7f2a08ff9640 1 -- 172.21.15.16:0/1839668392 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2a04007e00 con 0x7f2a0c072250 2024-09-19T07:43:27.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.903+0000 7f2a08ff9640 1 --2- 172.21.15.16:0/1839668392 >> [v2:172.21.15.146:6824/1757996733,v1:172.21.15.146:6825/1757996733] conn(0x7f29f80766d0 0x7f29f8078b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:27.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.903+0000 7f2a08ff9640 1 -- 172.21.15.16:0/1839668392 --> [v2:172.21.15.146:6824/1757996733,v1:172.21.15.146:6825/1757996733] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f29f8079240 con 0x7f29f80766d0 2024-09-19T07:43:27.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.904+0000 7f2a0b7fe640 1 --2- 172.21.15.16:0/1839668392 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f29f806c300 0x7f29f806e7c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c15d7d0 tx=0x7f2a00009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.904+0000 7f2a08ff9640 1 -- 172.21.15.16:0/1839668392 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2a04076960 con 0x7f2a0c072250 2024-09-19T07:43:27.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.904+0000 7f2a10c57640 1 --2- 172.21.15.16:0/1839668392 >> [v2:172.21.15.146:6824/1757996733,v1:172.21.15.146:6825/1757996733] conn(0x7f29f80766d0 0x7f29f8078b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:27.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.905+0000 7f2a10c57640 1 --2- 172.21.15.16:0/1839668392 >> [v2:172.21.15.146:6824/1757996733,v1:172.21.15.146:6825/1757996733] conn(0x7f29f80766d0 0x7f29f8078b70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:27.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.906+0000 7f2a08ff9640 1 -- 172.21.15.16:0/1839668392 <== osd.7 v2:172.21.15.146:6824/1757996733 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f29f8079240 con 0x7f29f80766d0 2024-09-19T07:43:27.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.928+0000 7f3c2eaba640 1 -- 172.21.15.16:0/1020464696 --> [v2:172.21.15.16:6808/2159059174,v1:172.21.15.16:6809/2159059174] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3c08002dd0 con 0x7f3c0c075a80 2024-09-19T07:43:27.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.929+0000 7f3c257fa640 1 -- 172.21.15.16:0/1020464696 <== osd.0 v2:172.21.15.16:6808/2159059174 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3c08002dd0 con 0x7f3c0c075a80 2024-09-19T07:43:27.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.929+0000 7f3c12ffd640 1 -- 172.21.15.16:0/1020464696 >> [v2:172.21.15.16:6808/2159059174,v1:172.21.15.16:6809/2159059174] conn(0x7f3c0c075a80 msgr2=0x7f3c0c077ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.929+0000 7f3c12ffd640 1 --2- 172.21.15.16:0/1020464696 >> [v2:172.21.15.16:6808/2159059174,v1:172.21.15.16:6809/2159059174] conn(0x7f3c0c075a80 0x7f3c0c077ec0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.929+0000 7fc710c96640 1 -- 172.21.15.16:0/3377319321 --> [v2:172.21.15.146:6808/2073273152,v1:172.21.15.146:6809/2073273152] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc6ec002dd0 con 0x7fc6f0075b30 2024-09-19T07:43:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.929+0000 7efe79617640 1 -- 172.21.15.16:0/1487276428 --> [v2:172.21.15.146:6816/2651212753,v1:172.21.15.146:6817/2651212753] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efe54005c40 con 0x7efe54001650 2024-09-19T07:43:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.930+0000 7f3c12ffd640 1 -- 172.21.15.16:0/1020464696 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3c0c06c380 msgr2=0x7f3c0c06e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.930+0000 7f3c12ffd640 1 --2- 172.21.15.16:0/1020464696 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3c0c06c380 0x7f3c0c06e840 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0094a0 tx=0x7f3c1c009240 comp rx=0 tx=0).stop 2024-09-19T07:43:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.930+0000 7f3c12ffd640 1 -- 172.21.15.16:0/1020464696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c2815c960 msgr2=0x7f3c28161de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.930+0000 7f3c12ffd640 1 --2- 172.21.15.16:0/1020464696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c2815c960 0x7f3c28161de0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3c14031000 tx=0x7f3c140046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.930+0000 7f491effd640 1 -- 172.21.15.16:0/1559289191 --> [v2:172.21.15.16:6816/3748453908,v1:172.21.15.16:6817/3748453908] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f493806c1d0 con 0x7f4924075b00 2024-09-19T07:43:27.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.930+0000 7f3c2d858640 1 -- 172.21.15.16:0/1020464696 reap_dead start 2024-09-19T07:43:27.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.930+0000 7f3c12ffd640 1 -- 172.21.15.16:0/1020464696 shutdown_connections 2024-09-19T07:43:27.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.930+0000 7f3c12ffd640 1 -- 172.21.15.16:0/1020464696 >> 172.21.15.16:0/1020464696 conn(0x7f3c280b5a00 msgr2=0x7f3c280b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.930+0000 7f3c12ffd640 1 -- 172.21.15.16:0/1020464696 shutdown_connections 2024-09-19T07:43:27.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.930+0000 7f3c12ffd640 1 -- 172.21.15.16:0/1020464696 wait complete. 2024-09-19T07:43:27.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.930+0000 7fc6f7fff640 1 -- 172.21.15.16:0/3377319321 <== osd.4 v2:172.21.15.146:6808/2073273152 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc6ec002dd0 con 0x7fc6f0075b30 2024-09-19T07:43:27.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.930+0000 7f0c66ffd640 1 -- 172.21.15.16:0/4212779306 --> [v2:172.21.15.16:6824/3030558062,v1:172.21.15.16:6825/3030558062] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0c54002dd0 con 0x7f0c60075a80 2024-09-19T07:43:27.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.930+0000 7f49357fa640 1 -- 172.21.15.16:0/1559289191 <== osd.3 v2:172.21.15.16:6816/3748453908 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f493806c1d0 con 0x7f4924075b00 2024-09-19T07:43:27.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.930+0000 7efe5b7fe640 1 -- 172.21.15.16:0/1487276428 <== osd.5 v2:172.21.15.146:6816/2651212753 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efe54005c40 con 0x7efe54001650 2024-09-19T07:43:27.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7fc710c96640 1 -- 172.21.15.16:0/3377319321 >> [v2:172.21.15.146:6808/2073273152,v1:172.21.15.146:6809/2073273152] conn(0x7fc6f0075b30 msgr2=0x7fc6f0077f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7f491effd640 1 -- 172.21.15.16:0/1559289191 >> [v2:172.21.15.16:6816/3748453908,v1:172.21.15.16:6817/3748453908] conn(0x7f4924075b00 msgr2=0x7f4924077f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7f491effd640 1 --2- 172.21.15.16:0/1559289191 >> [v2:172.21.15.16:6816/3748453908,v1:172.21.15.16:6817/3748453908] conn(0x7f4924075b00 0x7f4924077f40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7f0c797fa640 1 -- 172.21.15.16:0/4212779306 <== osd.1 v2:172.21.15.16:6824/3030558062 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0c54002dd0 con 0x7f0c60075a80 2024-09-19T07:43:27.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7fc710c96640 1 --2- 172.21.15.16:0/3377319321 >> [v2:172.21.15.146:6808/2073273152,v1:172.21.15.146:6809/2073273152] conn(0x7fc6f0075b30 0x7fc6f0077f70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7f491effd640 1 -- 172.21.15.16:0/1559289191 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f492406c380 msgr2=0x7f492406e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7f491effd640 1 --2- 172.21.15.16:0/1559289191 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f492406c380 0x7f492406e840 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f49380d06e0 tx=0x7f4930007650 comp rx=0 tx=0).stop 2024-09-19T07:43:27.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7f491effd640 1 -- 172.21.15.16:0/1559289191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49380b5010 msgr2=0x7f49380ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7f491effd640 1 --2- 172.21.15.16:0/1559289191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49380b5010 0x7f49380ccff0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f492c0090b0 tx=0x7f492c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7efe79617640 1 -- 172.21.15.16:0/1487276428 >> [v2:172.21.15.146:6816/2651212753,v1:172.21.15.146:6817/2651212753] conn(0x7efe54001650 msgr2=0x7efe54003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7efe79617640 1 --2- 172.21.15.16:0/1487276428 >> [v2:172.21.15.146:6816/2651212753,v1:172.21.15.146:6817/2651212753] conn(0x7efe54001650 0x7efe54003b10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7fa32d67b640 1 -- 172.21.15.16:0/1086342658 --> [v2:172.21.15.146:6800/2692587126,v1:172.21.15.146:6801/2692587126] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa3280cf220 con 0x7fa314075af0 2024-09-19T07:43:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7fc710c96640 1 -- 172.21.15.16:0/3377319321 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc6f006c3b0 msgr2=0x7fc6f006e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7fc710c96640 1 --2- 172.21.15.16:0/3377319321 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc6f006c3b0 0x7fc6f006e870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009b90 tx=0x7fc6f8009190 comp rx=0 tx=0).stop 2024-09-19T07:43:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7f493d242640 1 -- 172.21.15.16:0/1559289191 reap_dead start 2024-09-19T07:43:27.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7fc710c96640 1 -- 172.21.15.16:0/3377319321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc70400ac70 msgr2=0x7fc70400b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7fc710c96640 1 --2- 172.21.15.16:0/3377319321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc70400ac70 0x7fc70400b120 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc70c06a560 tx=0x7fc70c069710 comp rx=0 tx=0).stop 2024-09-19T07:43:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7efe79617640 1 -- 172.21.15.16:0/1487276428 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efe6006c380 msgr2=0x7efe6006e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7efe79617640 1 --2- 172.21.15.16:0/1487276428 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efe6006c380 0x7efe6006e840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efe68001580 tx=0x7efe68007a40 comp rx=0 tx=0).stop 2024-09-19T07:43:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7f491effd640 1 -- 172.21.15.16:0/1559289191 shutdown_connections 2024-09-19T07:43:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7f491effd640 1 -- 172.21.15.16:0/1559289191 >> 172.21.15.16:0/1559289191 conn(0x7f493806bdf0 msgr2=0x7f49380737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.931+0000 7fc70bfff640 1 -- 172.21.15.16:0/3377319321 reap_dead start 2024-09-19T07:43:27.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7f0c66ffd640 1 -- 172.21.15.16:0/4212779306 >> [v2:172.21.15.16:6824/3030558062,v1:172.21.15.16:6825/3030558062] conn(0x7f0c60075a80 msgr2=0x7f0c60077ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7efe79617640 1 -- 172.21.15.16:0/1487276428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe740c5690 msgr2=0x7efe740c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7efe79617640 1 --2- 172.21.15.16:0/1487276428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe740c5690 0x7efe740c7520 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efe6400e9e0 tx=0x7efe6400eea0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7f0c66ffd640 1 --2- 172.21.15.16:0/4212779306 >> [v2:172.21.15.16:6824/3030558062,v1:172.21.15.16:6825/3030558062] conn(0x7f0c60075a80 0x7f0c60077ec0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7efe737fe640 1 -- 172.21.15.16:0/1487276428 reap_dead start 2024-09-19T07:43:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7f0c66ffd640 1 -- 172.21.15.16:0/4212779306 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0c6006c300 msgr2=0x7f0c6006e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7f0c66ffd640 1 --2- 172.21.15.16:0/4212779306 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0c6006c300 0x7f0c6006e7c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400bb90 tx=0x7f0c68009190 comp rx=0 tx=0).stop 2024-09-19T07:43:27.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7f491effd640 1 -- 172.21.15.16:0/1559289191 shutdown_connections 2024-09-19T07:43:27.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7fc710c96640 1 -- 172.21.15.16:0/3377319321 shutdown_connections 2024-09-19T07:43:27.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7efe79617640 1 -- 172.21.15.16:0/1487276428 shutdown_connections 2024-09-19T07:43:27.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7fc710c96640 1 -- 172.21.15.16:0/3377319321 >> 172.21.15.16:0/3377319321 conn(0x7fc704036090 msgr2=0x7fc704031e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7efe79617640 1 -- 172.21.15.16:0/1487276428 >> 172.21.15.16:0/1487276428 conn(0x7efe7406bdf0 msgr2=0x7efe74166d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7f0c66ffd640 1 -- 172.21.15.16:0/4212779306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c74007d60 msgr2=0x7f0c74036df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7f0c66ffd640 1 --2- 172.21.15.16:0/4212779306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c74007d60 0x7f0c74036df0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0518b0 tx=0x7f0c7c069710 comp rx=0 tx=0).stop 2024-09-19T07:43:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7f491effd640 1 -- 172.21.15.16:0/1559289191 wait complete. 2024-09-19T07:43:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7efe79617640 1 -- 172.21.15.16:0/1487276428 shutdown_connections 2024-09-19T07:43:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7f0c81581640 1 -- 172.21.15.16:0/4212779306 reap_dead start 2024-09-19T07:43:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7efe79617640 1 -- 172.21.15.16:0/1487276428 wait complete. 2024-09-19T07:43:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7fc710c96640 1 -- 172.21.15.16:0/3377319321 shutdown_connections 2024-09-19T07:43:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7fc710c96640 1 -- 172.21.15.16:0/3377319321 wait complete. 2024-09-19T07:43:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7fa30f7fe640 1 -- 172.21.15.16:0/1086342658 <== osd.6 v2:172.21.15.146:6800/2692587126 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa3280cf220 con 0x7fa314075af0 2024-09-19T07:43:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7f0c66ffd640 1 -- 172.21.15.16:0/4212779306 shutdown_connections 2024-09-19T07:43:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7f0c66ffd640 1 -- 172.21.15.16:0/4212779306 >> 172.21.15.16:0/4212779306 conn(0x7f0c7401ea40 msgr2=0x7f0c740e2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.932+0000 7f0c66ffd640 1 -- 172.21.15.16:0/4212779306 shutdown_connections 2024-09-19T07:43:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.933+0000 7f0c66ffd640 1 -- 172.21.15.16:0/4212779306 wait complete. 2024-09-19T07:43:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.933+0000 7fa32d67b640 1 -- 172.21.15.16:0/1086342658 >> [v2:172.21.15.146:6800/2692587126,v1:172.21.15.146:6801/2692587126] conn(0x7fa314075af0 msgr2=0x7fa314077f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.933+0000 7fa32d67b640 1 --2- 172.21.15.16:0/1086342658 >> [v2:172.21.15.146:6800/2692587126,v1:172.21.15.146:6801/2692587126] conn(0x7fa314075af0 0x7fa314077f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.933+0000 7fec810af640 1 -- 172.21.15.16:0/2426048415 --> [v2:172.21.15.16:6800/4074420552,v1:172.21.15.16:6801/4074420552] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fec7404e360 con 0x7fec68075b30 2024-09-19T07:43:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.933+0000 7fa32d67b640 1 -- 172.21.15.16:0/1086342658 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa31406c380 msgr2=0x7fa31406e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.933+0000 7fa32d67b640 1 --2- 172.21.15.16:0/1086342658 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa31406c380 0x7fa31406e840 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa320006fd0 tx=0x7fa320009060 comp rx=0 tx=0).stop 2024-09-19T07:43:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.933+0000 7fa32d67b640 1 -- 172.21.15.16:0/1086342658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3281626c0 msgr2=0x7fa3280cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.933+0000 7fa32d67b640 1 --2- 172.21.15.16:0/1086342658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3281626c0 0x7fa3280cceb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa31c02a2b0 tx=0x7fa31c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.933+0000 7fa3277fe640 1 -- 172.21.15.16:0/1086342658 reap_dead start 2024-09-19T07:43:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.933+0000 7fec78ff9640 1 -- 172.21.15.16:0/2426048415 <== osd.2 v2:172.21.15.16:6800/4074420552 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fec7404e360 con 0x7fec68075b30 2024-09-19T07:43:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.933+0000 7fa32d67b640 1 -- 172.21.15.16:0/1086342658 shutdown_connections 2024-09-19T07:43:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.933+0000 7fa32d67b640 1 -- 172.21.15.16:0/1086342658 >> 172.21.15.16:0/1086342658 conn(0x7fa32809d770 msgr2=0x7fa3280750f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.934+0000 7fa32d67b640 1 -- 172.21.15.16:0/1086342658 shutdown_connections 2024-09-19T07:43:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.934+0000 7fa32d67b640 1 -- 172.21.15.16:0/1086342658 wait complete. 2024-09-19T07:43:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.934+0000 7fec810af640 1 -- 172.21.15.16:0/2426048415 >> [v2:172.21.15.16:6800/4074420552,v1:172.21.15.16:6801/4074420552] conn(0x7fec68075b30 msgr2=0x7fec68077f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.934+0000 7fec810af640 1 --2- 172.21.15.16:0/2426048415 >> [v2:172.21.15.16:6800/4074420552,v1:172.21.15.16:6801/4074420552] conn(0x7fec68075b30 0x7fec68077f70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.935+0000 7fec810af640 1 -- 172.21.15.16:0/2426048415 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fec6806c3b0 msgr2=0x7fec6806e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.935+0000 7fec810af640 1 --2- 172.21.15.16:0/2426048415 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fec6806c3b0 0x7fec6806e870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec6c005880 tx=0x7fec6c009040 comp rx=0 tx=0).stop 2024-09-19T07:43:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.935+0000 7fec810af640 1 -- 172.21.15.16:0/2426048415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec740535b0 msgr2=0x7fec740393b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.935+0000 7fec810af640 1 --2- 172.21.15.16:0/2426048415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec740535b0 0x7fec740393b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fec74055410 tx=0x7fec700046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.935+0000 7fec808ae640 1 -- 172.21.15.16:0/2426048415 reap_dead start 2024-09-19T07:43:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.935+0000 7fec810af640 1 -- 172.21.15.16:0/2426048415 shutdown_connections 2024-09-19T07:43:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.935+0000 7fec810af640 1 -- 172.21.15.16:0/2426048415 >> 172.21.15.16:0/2426048415 conn(0x7fec7403a520 msgr2=0x7fec74038470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.935+0000 7fec810af640 1 -- 172.21.15.16:0/2426048415 shutdown_connections 2024-09-19T07:43:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.936+0000 7fec810af640 1 -- 172.21.15.16:0/2426048415 wait complete. 2024-09-19T07:43:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.936+0000 7f29f27fc640 1 -- 172.21.15.16:0/1839668392 --> [v2:172.21.15.146:6824/1757996733,v1:172.21.15.146:6825/1757996733] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2a0c15d1d0 con 0x7f29f80766d0 2024-09-19T07:43:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.937+0000 7f2a08ff9640 1 -- 172.21.15.16:0/1839668392 <== osd.7 v2:172.21.15.146:6824/1757996733 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2a0c15d1d0 con 0x7f29f80766d0 2024-09-19T07:43:27.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.938+0000 7f29f27fc640 1 -- 172.21.15.16:0/1839668392 >> [v2:172.21.15.146:6824/1757996733,v1:172.21.15.146:6825/1757996733] conn(0x7f29f80766d0 msgr2=0x7f29f8078b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.938+0000 7f29f27fc640 1 --2- 172.21.15.16:0/1839668392 >> [v2:172.21.15.146:6824/1757996733,v1:172.21.15.146:6825/1757996733] conn(0x7f29f80766d0 0x7f29f8078b70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.938+0000 7f29f27fc640 1 -- 172.21.15.16:0/1839668392 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f29f806c300 msgr2=0x7f29f806e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.938+0000 7f29f27fc640 1 --2- 172.21.15.16:0/1839668392 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f29f806c300 0x7f29f806e7c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c15d7d0 tx=0x7f2a00009040 comp rx=0 tx=0).stop 2024-09-19T07:43:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.938+0000 7f29f27fc640 1 -- 172.21.15.16:0/1839668392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c072250 msgr2=0x7f2a0c15c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.938+0000 7f29f27fc640 1 --2- 172.21.15.16:0/1839668392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c072250 0x7f2a0c15c3e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009f70 tx=0x7f2a04004a40 comp rx=0 tx=0).stop 2024-09-19T07:43:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.939+0000 7f2a10c57640 1 -- 172.21.15.16:0/1839668392 reap_dead start 2024-09-19T07:43:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.939+0000 7f29f27fc640 1 -- 172.21.15.16:0/1839668392 shutdown_connections 2024-09-19T07:43:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.939+0000 7f29f27fc640 1 -- 172.21.15.16:0/1839668392 >> 172.21.15.16:0/1839668392 conn(0x7f2a0c0b4040 msgr2=0x7f2a0c0c89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.939+0000 7f29f27fc640 1 -- 172.21.15.16:0/1839668392 shutdown_connections 2024-09-19T07:43:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:27.940+0000 7f29f27fc640 1 -- 172.21.15.16:0/1839668392 wait complete. 2024-09-19T07:43:28.000 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-09-19T07:43:28.000 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-19T07:43:28.001 INFO:teuthology.orchestra.run.smithi016.stdout:60129542152 2024-09-19T07:43:28.001 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-09-19T07:43:28.002 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-09-19T07:43:28.002 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-19T07:43:28.003 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-19T07:43:28.003 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-19T07:43:28.004 INFO:teuthology.orchestra.run.smithi016.stdout:60129542152 2024-09-19T07:43:28.004 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-19T07:43:28.005 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-09-19T07:43:28.005 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-19T07:43:28.006 INFO:teuthology.orchestra.run.smithi016.stdout:55834574858 2024-09-19T07:43:28.006 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-19T07:43:28.007 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-09-19T07:43:28.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-19T07:43:28.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.217+0000 7f27a5b31640 1 Processor -- start 2024-09-19T07:43:28.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.218+0000 7f27a5b31640 1 -- start start 2024-09-19T07:43:28.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.217+0000 7ff73759e640 1 Processor -- start 2024-09-19T07:43:28.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.218+0000 7ff73759e640 1 -- start start 2024-09-19T07:43:28.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.219+0000 7ff73759e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff7300e2b90 0x7ff730053a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.219+0000 7ff73759e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff730006d40 0x7ff730007190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.219+0000 7ff73759e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7300e2f70 con 0x7ff730053f40 2024-09-19T07:43:28.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.219+0000 7ff73759e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff7300076d0 con 0x7ff7300e2b90 2024-09-19T07:43:28.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.219+0000 7ff73759e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff7300577a0 con 0x7ff730006d40 2024-09-19T07:43:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.219+0000 7f27a5b31640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a00b1550 0x7f27a015b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.219+0000 7f27a5b31640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27a00b19e0 con 0x7f27a00b27c0 2024-09-19T07:43:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.219+0000 7f27a5b31640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27a00b1cc0 con 0x7f27a00b5130 2024-09-19T07:43:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.219+0000 7f27a5b31640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f27a015b710 con 0x7f27a00b1550 2024-09-19T07:43:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.219+0000 7f279effd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f27a00b27c0 0x7f27a00b4a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44878/0 (socket says 172.21.15.16:44878) 2024-09-19T07:43:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.219+0000 7f279effd640 1 -- 172.21.15.16:0/425779113 learned_addr learned my addr 172.21.15.16:0/425779113 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.219+0000 7f406ffff640 1 Processor -- start 2024-09-19T07:43:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.220+0000 7ff73659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff7300e2b90 0x7ff730053a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.220+0000 7ff736d9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff730006d40 0x7ff730007190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.220+0000 7ff735d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff730053f40 0x7ff73003be40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44890/0 (socket says 172.21.15.16:44890) 2024-09-19T07:43:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.220+0000 7ff735d9b640 1 -- 172.21.15.16:0/4164906822 learned_addr learned my addr 172.21.15.16:0/4164906822 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.219+0000 7f406ffff640 1 -- start start 2024-09-19T07:43:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.220+0000 7f279e7fc640 1 -- 172.21.15.16:0/425779113 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2096504987 0 0) 0x7f27a00b19e0 con 0x7f27a00b27c0 2024-09-19T07:43:28.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.220+0000 7f406ffff640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4068005a60 0x7f4068053f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.220+0000 7f406ffff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40680e7120 0x7f4068005e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.220+0000 7f279e7fc640 1 -- 172.21.15.16:0/425779113 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2788003520 con 0x7f27a00b27c0 2024-09-19T07:43:28.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.220+0000 7f406ffff640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4068006380 con 0x7f406804eb60 2024-09-19T07:43:28.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.220+0000 7f406ffff640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f40680520e0 con 0x7f4068005a60 2024-09-19T07:43:28.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.220+0000 7f406ffff640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4068052330 con 0x7f40680e7120 2024-09-19T07:43:28.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7f279e7fc640 1 -- 172.21.15.16:0/425779113 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3319351672 0 0) 0x7f2788003520 con 0x7f27a00b27c0 2024-09-19T07:43:28.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7f279f7fe640 1 --2- 172.21.15.16:0/425779113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a00b1550 0x7f27a015b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7f279e7fc640 1 -- 172.21.15.16:0/425779113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a00b1550 msgr2=0x7f27a015b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:28.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7f279e7fc640 1 --2- 172.21.15.16:0/425779113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a00b1550 0x7f27a015b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7f279e7fc640 1 -- 172.21.15.16:0/425779113 >> v1:172.21.15.146:6789/0 conn(0x7f27a00b5130 legacy=0x7f27a006bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7fe6dca26640 1 Processor -- start 2024-09-19T07:43:28.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7f279e7fc640 1 -- 172.21.15.16:0/425779113 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a01620b0 con 0x7f27a00b27c0 2024-09-19T07:43:28.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7f406f7fe640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f406804eb60 0x7f406801c520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:56442/0 (socket says 172.21.15.16:56442) 2024-09-19T07:43:28.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7f406f7fe640 1 -- 172.21.15.16:0/3060600558 learned_addr learned my addr 172.21.15.16:0/3060600558 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:28.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7f279e7fc640 1 -- 172.21.15.16:0/425779113 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f2790001670 con 0x7f27a00b27c0 2024-09-19T07:43:28.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7f279e7fc640 1 -- 172.21.15.16:0/425779113 >> v1:172.21.15.16:6789/0 conn(0x7f27a00b27c0 legacy=0x7f27a00b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7f279e7fc640 1 --2- 172.21.15.16:0/425779113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27880039e0 0x7f2788003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7f279e7fc640 1 --2- 172.21.15.16:0/425779113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2788004800 0x7f2788004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7f279e7fc640 1 --2- 172.21.15.16:0/425779113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27880052e0 0x7f278800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7f279e7fc640 1 -- 172.21.15.16:0/425779113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27a015b710 con 0x7f27880039e0 2024-09-19T07:43:28.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7f279e7fc640 1 -- 172.21.15.16:0/425779113 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f278800c010 con 0x7f2788004800 2024-09-19T07:43:28.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7f279e7fc640 1 -- 172.21.15.16:0/425779113 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f278800c210 con 0x7f27880052e0 2024-09-19T07:43:28.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7f279effd640 1 --2- 172.21.15.16:0/425779113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2788004800 0x7f2788004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.220+0000 7ff73659c640 1 -- 172.21.15.16:0/4164906822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff730006d40 msgr2=0x7ff730007190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:28.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.220+0000 7ff73659c640 1 --2- 172.21.15.16:0/4164906822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff730006d40 0x7ff730007190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.220+0000 7ff73659c640 1 -- 172.21.15.16:0/4164906822 >> v1:172.21.15.16:6789/0 conn(0x7ff730053f40 legacy=0x7ff73003be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.220+0000 7ff73659c640 1 -- 172.21.15.16:0/4164906822 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff730057a60 con 0x7ff7300e2b90 2024-09-19T07:43:28.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7f279f7fe640 1 --2- 172.21.15.16:0/425779113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27880039e0 0x7f2788003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7f279ffff640 1 --2- 172.21.15.16:0/425779113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27880052e0 0x7f278800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f2a1264a640 1 Processor -- start 2024-09-19T07:43:28.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f406e7fc640 1 --2- 172.21.15.16:0/3060600558 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40680e7120 0x7f4068005e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7ff73659c640 1 --2- 172.21.15.16:0/4164906822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7300e2b90 0x7ff730053a00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0097a0 tx=0x7ff72c02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b28778c2f17c97 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f279effd640 1 -- 172.21.15.16:0/425779113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27880052e0 msgr2=0x7f278800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f279effd640 1 --2- 172.21.15.16:0/425779113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27880052e0 0x7f278800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f279effd640 1 -- 172.21.15.16:0/425779113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27880039e0 msgr2=0x7f2788003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f279effd640 1 --2- 172.21.15.16:0/425779113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27880039e0 0x7f2788003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f279effd640 1 -- 172.21.15.16:0/425779113 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2788003520 con 0x7f2788004800 2024-09-19T07:43:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.221+0000 7fe6dca26640 1 -- start start 2024-09-19T07:43:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f406effd640 1 --2- 172.21.15.16:0/3060600558 >> v2:172.21.15.146:3300/0 conn(0x7f4068005a60 0x7f4068053f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f406dffb640 1 -- 172.21.15.16:0/3060600558 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2934987865 0 0) 0x7f4068006380 con 0x7f406804eb60 2024-09-19T07:43:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7ff73559a640 1 -- 172.21.15.16:0/4164906822 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c037070 con 0x7ff7300e2b90 2024-09-19T07:43:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7ff73559a640 1 -- 172.21.15.16:0/4164906822 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff72c02add0 con 0x7ff7300e2b90 2024-09-19T07:43:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7ff73559a640 1 -- 172.21.15.16:0/4164906822 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c032660 con 0x7ff7300e2b90 2024-09-19T07:43:28.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f406dffb640 1 -- 172.21.15.16:0/3060600558 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4058003520 con 0x7f406804eb60 2024-09-19T07:43:28.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f406dffb640 1 -- 172.21.15.16:0/3060600558 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1228531367 0 0) 0x7f4058003520 con 0x7f406804eb60 2024-09-19T07:43:28.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f406dffb640 1 -- 172.21.15.16:0/3060600558 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40680e7120 msgr2=0x7f4068005e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:28.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f406dffb640 1 --2- 172.21.15.16:0/3060600558 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40680e7120 0x7f4068005e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f406dffb640 1 -- 172.21.15.16:0/3060600558 >> v2:172.21.15.146:3300/0 conn(0x7f4068005a60 msgr2=0x7f4068053f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:28.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f406dffb640 1 --2- 172.21.15.16:0/3060600558 >> v2:172.21.15.146:3300/0 conn(0x7f4068005a60 0x7f4068053f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f406dffb640 1 -- 172.21.15.16:0/3060600558 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4068006c80 con 0x7f406804eb60 2024-09-19T07:43:28.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f406dffb640 1 -- 172.21.15.16:0/3060600558 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f4064001670 con 0x7f406804eb60 2024-09-19T07:43:28.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f406dffb640 1 -- 172.21.15.16:0/3060600558 >> v1:172.21.15.146:6789/0 conn(0x7f406804eb60 legacy=0x7f406801c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f406dffb640 1 --2- 172.21.15.16:0/3060600558 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40580039b0 0x7f4058003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f406dffb640 1 --2- 172.21.15.16:0/3060600558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40580043a0 0x7f4058004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f406dffb640 1 --2- 172.21.15.16:0/3060600558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4058004fd0 0x7f405800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f406dffb640 1 -- 172.21.15.16:0/3060600558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40680520e0 con 0x7f40580043a0 2024-09-19T07:43:28.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f406dffb640 1 -- 172.21.15.16:0/3060600558 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4068052330 con 0x7f4058004fd0 2024-09-19T07:43:28.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f406dffb640 1 -- 172.21.15.16:0/3060600558 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f405800bf50 con 0x7f40580039b0 2024-09-19T07:43:28.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7fe6dca26640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe6d0005a60 0x7fe6d0053f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7fe6dca26640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe6d00e7120 0x7fe6d0005e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7fe6dca26640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6d0006380 con 0x7fe6d004eb60 2024-09-19T07:43:28.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7fe6dca26640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fe6d00520e0 con 0x7fe6d00e7120 2024-09-19T07:43:28.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7fe6dca26640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe6d0052330 con 0x7fe6d0005a60 2024-09-19T07:43:28.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.223+0000 7fe6d6d9d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe6d0005a60 0x7fe6d0053f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.223+0000 7fe6d6d9d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe6d0005a60 0x7fe6d0053f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:57882/0 (socket says 172.21.15.16:57882) 2024-09-19T07:43:28.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.223+0000 7fe6d6d9d640 1 -- 172.21.15.16:0/3055948713 learned_addr learned my addr 172.21.15.16:0/3055948713 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:28.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.223+0000 7fe6d659c640 1 --2- 172.21.15.16:0/3055948713 >> v2:172.21.15.16:3300/0 conn(0x7fe6d00e7120 0x7fe6d0005e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.222+0000 7f2a1264a640 1 -- start start 2024-09-19T07:43:28.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.223+0000 7f406f7fe640 1 --2- 172.21.15.16:0/3060600558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4058004fd0 0x7f405800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.223+0000 7f406e7fc640 1 --2- 172.21.15.16:0/3060600558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40580043a0 0x7f4058004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.223+0000 7ff73759e640 1 -- 172.21.15.16:0/4164906822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7300e2b90 msgr2=0x7ff730053a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.223+0000 7fe6d659c640 1 -- 172.21.15.16:0/3055948713 >> v2:172.21.15.146:3300/0 conn(0x7fe6d0005a60 msgr2=0x7fe6d0053f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:28.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.223+0000 7f406f7fe640 1 -- 172.21.15.16:0/3060600558 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40580039b0 msgr2=0x7f4058003dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:43:28.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.223+0000 7fe6d659c640 1 --2- 172.21.15.16:0/3055948713 >> v2:172.21.15.146:3300/0 conn(0x7fe6d0005a60 0x7fe6d0053f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.223+0000 7f406f7fe640 1 --2- 172.21.15.16:0/3060600558 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40580039b0 0x7f4058003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.223+0000 7fe6d659c640 1 -- 172.21.15.16:0/3055948713 >> v1:172.21.15.16:6789/0 conn(0x7fe6d004eb60 legacy=0x7fe6d001c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.223+0000 7f406f7fe640 1 -- 172.21.15.16:0/3060600558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40580043a0 msgr2=0x7f4058004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:28.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.223+0000 7f406f7fe640 1 --2- 172.21.15.16:0/3060600558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40580043a0 0x7f4058004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.223+0000 7fe6d659c640 1 -- 172.21.15.16:0/3055948713 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d0006c80 con 0x7fe6d00e7120 2024-09-19T07:43:28.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.223+0000 7ff73759e640 1 --2- 172.21.15.16:0/4164906822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7300e2b90 0x7ff730053a00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0097a0 tx=0x7ff72c02a080 comp rx=0 tx=0).stop 2024-09-19T07:43:28.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.224+0000 7ff73759e640 1 -- 172.21.15.16:0/4164906822 shutdown_connections 2024-09-19T07:43:28.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.224+0000 7ff73759e640 1 --2- 172.21.15.16:0/4164906822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff730006d40 0x7ff730007190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.224+0000 7ff73759e640 1 --2- 172.21.15.16:0/4164906822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7300e2b90 0x7ff730053a00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.224+0000 7ff73759e640 1 -- 172.21.15.16:0/4164906822 >> 172.21.15.16:0/4164906822 conn(0x7ff73003a520 msgr2=0x7ff73003a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:28.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.224+0000 7f279effd640 1 --2- 172.21.15.16:0/425779113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2788004800 0x7f2788004c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f27a009f340 tx=0x7f2788005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96a3ad4b4d002063 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.224+0000 7f406f7fe640 1 -- 172.21.15.16:0/3060600558 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4068006c80 con 0x7f4058004fd0 2024-09-19T07:43:28.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.224+0000 7f2a1264a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2a0c15cc70 0x7f2a0c0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.224+0000 7f2a1264a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2a0c0b5500 0x7f2a0c0b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.224+0000 7f2a1264a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a0c071b60 con 0x7f2a0c0b37c0 2024-09-19T07:43:28.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.224+0000 7f2a1264a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2a0c0b5970 con 0x7f2a0c15cc70 2024-09-19T07:43:28.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.224+0000 7f2a1264a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2a0c072190 con 0x7f2a0c0b5500 2024-09-19T07:43:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.224+0000 7f279e7fc640 1 -- 172.21.15.16:0/425779113 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279003f070 con 0x7f2788004800 2024-09-19T07:43:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.224+0000 7f279e7fc640 1 -- 172.21.15.16:0/425779113 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279003a620 con 0x7f2788004800 2024-09-19T07:43:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.224+0000 7efce29cc640 1 Processor -- start 2024-09-19T07:43:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.224+0000 7f2a11648640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2a0c15cc70 0x7f2a0c0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.224+0000 7f2a11648640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2a0c15cc70 0x7f2a0c0b4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57482/0 (socket says 172.21.15.16:57482) 2024-09-19T07:43:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.224+0000 7f2a11648640 1 -- 172.21.15.16:0/173064960 learned_addr learned my addr 172.21.15.16:0/173064960 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.225+0000 7efce29cc640 1 -- start start 2024-09-19T07:43:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.225+0000 7f2a10e47640 1 --2- 172.21.15.16:0/173064960 >> v2:172.21.15.146:3300/0 conn(0x7f2a0c0b5500 0x7f2a0c0b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.225+0000 7fe6d659c640 1 --2- 172.21.15.16:0/3055948713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d00e7120 0x7fe6d0005e40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8074480 tx=0x7fe6d80a6060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7d86fcae5fedc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.225+0000 7f2a11648640 1 -- 172.21.15.16:0/173064960 >> v2:172.21.15.146:3300/0 conn(0x7f2a0c0b5500 msgr2=0x7f2a0c0b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:28.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.225+0000 7f2a11648640 1 --2- 172.21.15.16:0/173064960 >> v2:172.21.15.146:3300/0 conn(0x7f2a0c0b5500 0x7f2a0c0b3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.225+0000 7f2a11648640 1 -- 172.21.15.16:0/173064960 >> v1:172.21.15.146:6789/0 conn(0x7f2a0c0b37c0 legacy=0x7f2a0c072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:28.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.225+0000 7f406f7fe640 1 --2- 172.21.15.16:0/3060600558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4058004fd0 0x7f405800b9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f406400c260 tx=0x7f4058005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f55e455caac271b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.225+0000 7fe6d5d9b640 1 -- 172.21.15.16:0/3055948713 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d80b3070 con 0x7fe6d00e7120 2024-09-19T07:43:28.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.225+0000 7f279e7fc640 1 -- 172.21.15.16:0/425779113 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790012030 con 0x7f2788004800 2024-09-19T07:43:28.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.225+0000 7f27a5b31640 1 -- 172.21.15.16:0/425779113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2788004800 msgr2=0x7f2788004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.225+0000 7f27a5b31640 1 --2- 172.21.15.16:0/425779113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2788004800 0x7f2788004c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f27a009f340 tx=0x7f2788005830 comp rx=0 tx=0).stop 2024-09-19T07:43:28.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.225+0000 7fe6d5d9b640 1 -- 172.21.15.16:0/3055948713 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d80ae440 con 0x7fe6d00e7120 2024-09-19T07:43:28.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.225+0000 7f279ffff640 1 -- 172.21.15.16:0/425779113 reap_dead start 2024-09-19T07:43:28.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.225+0000 7f5714c74640 1 Processor -- start 2024-09-19T07:43:28.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.225+0000 7f5714c74640 1 -- start start 2024-09-19T07:43:28.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.226+0000 7f27a5b31640 1 -- 172.21.15.16:0/425779113 shutdown_connections 2024-09-19T07:43:28.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.226+0000 7f27a5b31640 1 -- 172.21.15.16:0/425779113 >> 172.21.15.16:0/425779113 conn(0x7f27a009a9f0 msgr2=0x7f27a00b6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:28.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.224+0000 7ff73759e640 1 -- 172.21.15.16:0/4164906822 shutdown_connections 2024-09-19T07:43:28.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.225+0000 7f2a11648640 1 -- 172.21.15.16:0/173064960 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c15d050 con 0x7f2a0c15cc70 2024-09-19T07:43:28.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.224+0000 7ff73759e640 1 -- 172.21.15.16:0/4164906822 wait complete. 2024-09-19T07:43:28.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.226+0000 7f27a5b31640 1 -- 172.21.15.16:0/425779113 shutdown_connections 2024-09-19T07:43:28.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.226+0000 7f27a5b31640 1 -- 172.21.15.16:0/425779113 wait complete. 2024-09-19T07:43:28.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.226+0000 7f2a11648640 1 --2- 172.21.15.16:0/173064960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c15cc70 0x7f2a0c0b4fc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0ccb30 tx=0x7f2a0802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d02455a6c5fe0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.226+0000 7f406dffb640 1 -- 172.21.15.16:0/3060600558 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064007d70 con 0x7f4058004fd0 2024-09-19T07:43:28.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.226+0000 7f406dffb640 1 -- 172.21.15.16:0/3060600558 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40640058d0 con 0x7f4058004fd0 2024-09-19T07:43:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.226+0000 7fe6dca26640 1 -- 172.21.15.16:0/3055948713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d00e7120 msgr2=0x7fe6d0005e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.226+0000 7fe6d5d9b640 1 -- 172.21.15.16:0/3055948713 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d80b3070 con 0x7fe6d00e7120 2024-09-19T07:43:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.226+0000 7fe6dca26640 1 --2- 172.21.15.16:0/3055948713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d00e7120 0x7fe6d0005e40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8074480 tx=0x7fe6d80a6060 comp rx=0 tx=0).stop 2024-09-19T07:43:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.226+0000 7efce29cc640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efcdc06bc10 0x7efcdc0ae200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.226+0000 7efce29cc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efcdc0ae740 0x7efcdc0b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7efce29cc640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcdc06d380 con 0x7efcdc15e160 2024-09-19T07:43:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7efce29cc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7efcdc0aeb70 con 0x7efcdc0ae740 2024-09-19T07:43:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7efce29cc640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7efcdc0b3c90 con 0x7efcdc06bc10 2024-09-19T07:43:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7efce11c9640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efcdc06bc10 0x7efcdc0ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7f2a03fff640 1 -- 172.21.15.16:0/173064960 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08037070 con 0x7f2a0c15cc70 2024-09-19T07:43:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7f5714c74640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f57101626c0 0x7f5710162aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7f406dffb640 1 -- 172.21.15.16:0/3060600558 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064005d20 con 0x7f4058004fd0 2024-09-19T07:43:28.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7efce21cb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efcdc0ae740 0x7efcdc0b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7efce21cb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efcdc0ae740 0x7efcdc0b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57494/0 (socket says 172.21.15.16:57494) 2024-09-19T07:43:28.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7efce21cb640 1 -- 172.21.15.16:0/431283769 learned_addr learned my addr 172.21.15.16:0/431283769 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:28.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7fe6dca26640 1 -- 172.21.15.16:0/3055948713 shutdown_connections 2024-09-19T07:43:28.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7f2a03fff640 1 -- 172.21.15.16:0/173064960 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a08032440 con 0x7f2a0c15cc70 2024-09-19T07:43:28.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7f2a03fff640 1 -- 172.21.15.16:0/173064960 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08032810 con 0x7f2a0c15cc70 2024-09-19T07:43:28.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7fe6dca26640 1 --2- 172.21.15.16:0/3055948713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d00e7120 0x7fe6d0005e40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7fe6dca26640 1 --2- 172.21.15.16:0/3055948713 >> v2:172.21.15.146:3300/0 conn(0x7fe6d0005a60 0x7fe6d0053f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7fe6dca26640 1 -- 172.21.15.16:0/3055948713 >> 172.21.15.16:0/3055948713 conn(0x7fe6d0036090 msgr2=0x7fe6d00364a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:28.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7f5714c74640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57100d0df0 0x7f57100d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7f5714c74640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5710162fe0 con 0x7f57100d2980 2024-09-19T07:43:28.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7f406ffff640 1 -- 172.21.15.16:0/3060600558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4058004fd0 msgr2=0x7f405800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7f406ffff640 1 --2- 172.21.15.16:0/3060600558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4058004fd0 0x7f405800b9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f406400c260 tx=0x7f4058005830 comp rx=0 tx=0).stop 2024-09-19T07:43:28.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7f5714c74640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f5710163270 con 0x7f57101626c0 2024-09-19T07:43:28.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7f406f7fe640 1 -- 172.21.15.16:0/3060600558 reap_dead start 2024-09-19T07:43:28.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7f5714c74640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5710163550 con 0x7f57100d0df0 2024-09-19T07:43:28.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7ff73759e640 1 Processor -- start 2024-09-19T07:43:28.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7ff73759e640 1 -- start start 2024-09-19T07:43:28.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7fbdb2e5e640 1 Processor -- start 2024-09-19T07:43:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7f570ed9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57100d0df0 0x7f57100d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7f570ed9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57100d0df0 0x7f57100d1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:34196/0 (socket says 172.21.15.16:34196) 2024-09-19T07:43:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.227+0000 7f570ed9d640 1 -- 172.21.15.16:0/3100509297 learned_addr learned my addr 172.21.15.16:0/3100509297 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7fe6dca26640 1 -- 172.21.15.16:0/3055948713 shutdown_connections 2024-09-19T07:43:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7fe6dca26640 1 -- 172.21.15.16:0/3055948713 wait complete. 2024-09-19T07:43:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7fbdb2e5e640 1 -- start start 2024-09-19T07:43:28.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7f570e59c640 1 --2- 172.21.15.16:0/3100509297 >> v2:172.21.15.16:3300/0 conn(0x7f57101626c0 0x7f5710162aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7f570ed9d640 1 -- 172.21.15.16:0/3100509297 >> v2:172.21.15.16:3300/0 conn(0x7f57101626c0 msgr2=0x7f5710162aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:28.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7ff73759e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff730006d40 0x7ff73000d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7f570ed9d640 1 --2- 172.21.15.16:0/3100509297 >> v2:172.21.15.16:3300/0 conn(0x7f57101626c0 0x7f5710162aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7f570ed9d640 1 -- 172.21.15.16:0/3100509297 >> v1:172.21.15.16:6789/0 conn(0x7f57100d2980 legacy=0x7f57100d2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:28.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7f570ed9d640 1 -- 172.21.15.16:0/3100509297 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5710163830 con 0x7f57100d0df0 2024-09-19T07:43:28.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7f27a5b31640 1 Processor -- start 2024-09-19T07:43:28.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7f2a1264a640 1 -- 172.21.15.16:0/173064960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c15cc70 msgr2=0x7f2a0c0b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7f2a1264a640 1 --2- 172.21.15.16:0/173064960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c15cc70 0x7f2a0c0b4fc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0ccb30 tx=0x7f2a0802a230 comp rx=0 tx=0).stop 2024-09-19T07:43:28.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7ff73759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff730053f40 0x7ff7300d6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7f2a1264a640 1 -- 172.21.15.16:0/173064960 shutdown_connections 2024-09-19T07:43:28.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7f2a1264a640 1 --2- 172.21.15.16:0/173064960 >> v2:172.21.15.146:3300/0 conn(0x7f2a0c0b5500 0x7f2a0c0b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7f2a1264a640 1 --2- 172.21.15.16:0/173064960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c15cc70 0x7f2a0c0b4fc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7f2a1264a640 1 -- 172.21.15.16:0/173064960 >> 172.21.15.16:0/173064960 conn(0x7f2a0c0b1dc0 msgr2=0x7f2a0c0b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:28.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7ff73759e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7300e2b90 0x7ff7300cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7ff73759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff730058300 con 0x7ff730053f40 2024-09-19T07:43:28.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7ff73759e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff730058020 con 0x7ff7300e2b90 2024-09-19T07:43:28.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7ff73759e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff73003c550 con 0x7ff730006d40 2024-09-19T07:43:28.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7ff736d9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7300e2b90 0x7ff7300cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7ff73659c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff730006d40 0x7ff73000d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7ff736d9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7300e2b90 0x7ff7300cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:57922/0 (socket says 172.21.15.16:57922) 2024-09-19T07:43:28.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7ff736d9d640 1 -- 172.21.15.16:0/1799469670 learned_addr learned my addr 172.21.15.16:0/1799469670 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:28.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7f27a5b31640 1 -- start start 2024-09-19T07:43:28.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7efce21cb640 1 -- 172.21.15.16:0/431283769 >> v2:172.21.15.146:3300/0 conn(0x7efcdc06bc10 msgr2=0x7efcdc0ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:28.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7efce21cb640 1 --2- 172.21.15.16:0/431283769 >> v2:172.21.15.146:3300/0 conn(0x7efcdc06bc10 0x7efcdc0ae200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7efce21cb640 1 -- 172.21.15.16:0/431283769 >> v1:172.21.15.146:6789/0 conn(0x7efcdc15e160 legacy=0x7efcdc06cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7efce21cb640 1 -- 172.21.15.16:0/431283769 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcdc0b3f70 con 0x7efcdc0ae740 2024-09-19T07:43:28.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7efce21cb640 1 --2- 172.21.15.16:0/431283769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcdc0ae740 0x7efcdc0b4770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0aff10 tx=0x7efcd402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858601eb2bba236f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7fbdb2e5e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbdac0d2290 0x7fbdac161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7fbdb2e5e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbdac0d3ec0 0x7fbdac0d4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7fbdb2e5e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdac0d4d40 con 0x7fbdac15e490 2024-09-19T07:43:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7fbdb2e5e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fbdac0ced20 con 0x7fbdac0d3ec0 2024-09-19T07:43:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7fbdb2e5e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fbdac1625e0 con 0x7fbdac0d2290 2024-09-19T07:43:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7f2a1264a640 1 -- 172.21.15.16:0/173064960 shutdown_connections 2024-09-19T07:43:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7f406ffff640 1 -- 172.21.15.16:0/3060600558 shutdown_connections 2024-09-19T07:43:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.228+0000 7f406ffff640 1 -- 172.21.15.16:0/3060600558 >> 172.21.15.16:0/3060600558 conn(0x7f4068036090 msgr2=0x7f40680364a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:28.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7ff735d9b640 1 --2- 172.21.15.16:0/1799469670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff730053f40 0x7ff7300d6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7f570ed9d640 1 --2- 172.21.15.16:0/3100509297 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57100d0df0 0x7f57100d1280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5708009e50 tx=0x7f570802a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3da28373fa40a80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7ff736d9d640 1 -- 172.21.15.16:0/1799469670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff730006d40 msgr2=0x7ff73000d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7ff736d9d640 1 --2- 172.21.15.16:0/1799469670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff730006d40 0x7ff73000d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7ff736d9d640 1 -- 172.21.15.16:0/1799469670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff730053f40 msgr2=0x7ff7300d6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7ff736d9d640 1 --2- 172.21.15.16:0/1799469670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff730053f40 0x7ff7300d6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7ff736d9d640 1 -- 172.21.15.16:0/1799469670 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7300d4750 con 0x7ff7300e2b90 2024-09-19T07:43:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7f406ffff640 1 -- 172.21.15.16:0/3060600558 shutdown_connections 2024-09-19T07:43:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7efce09c8640 1 -- 172.21.15.16:0/431283769 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd4036070 con 0x7efcdc0ae740 2024-09-19T07:43:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7fbdb13fb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbdac0d3ec0 0x7fbdac0d4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7fbdb13fb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbdac0d3ec0 0x7fbdac0d4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57532/0 (socket says 172.21.15.16:57532) 2024-09-19T07:43:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7fbdb13fb640 1 -- 172.21.15.16:0/4053684220 learned_addr learned my addr 172.21.15.16:0/4053684220 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7fbdb0bfa640 1 --1- 172.21.15.16:0/4053684220 >> v1:172.21.15.146:6789/0 conn(0x7fbdac15e490 0x7fbdac0b4400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:56460/0 (socket says 172.21.15.16:56460) 2024-09-19T07:43:28.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7fbdabfff640 1 --2- 172.21.15.16:0/4053684220 >> v2:172.21.15.146:3300/0 conn(0x7fbdac0d2290 0x7fbdac161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7ff736d9d640 1 --2- 172.21.15.16:0/1799469670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7300e2b90 0x7ff7300cfca0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff73806aa20 tx=0x7ff738069710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7efce09c8640 1 -- 172.21.15.16:0/431283769 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcd4031600 con 0x7efcdc0ae740 2024-09-19T07:43:28.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7f27a5b31640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2788004800 0x7f27a0150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7fe6dca26640 1 Processor -- start 2024-09-19T07:43:28.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7efce09c8640 1 -- 172.21.15.16:0/431283769 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd40319d0 con 0x7efcdc0ae740 2024-09-19T07:43:28.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7f27a5b31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a0157da0 0x7f27a0200e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7f27a5b31640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a0201360 0x7f27a0201810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7f27a5b31640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27a00b2280 con 0x7f27a0157da0 2024-09-19T07:43:28.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7f27a5b31640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f27a00b2560 con 0x7f2788004800 2024-09-19T07:43:28.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7f27a5b31640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f27a00afc10 con 0x7f27a0201360 2024-09-19T07:43:28.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7fe6dca26640 1 -- start start 2024-09-19T07:43:28.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7f279f7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2788004800 0x7f27a0150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7f406ffff640 1 -- 172.21.15.16:0/3060600558 wait complete. 2024-09-19T07:43:28.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7fbdb13fb640 1 -- 172.21.15.16:0/4053684220 >> v2:172.21.15.146:3300/0 conn(0x7fbdac0d2290 msgr2=0x7fbdac161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:28.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7fbdb13fb640 1 --2- 172.21.15.16:0/4053684220 >> v2:172.21.15.146:3300/0 conn(0x7fbdac0d2290 0x7fbdac161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7fbdb13fb640 1 -- 172.21.15.16:0/4053684220 >> v1:172.21.15.146:6789/0 conn(0x7fbdac15e490 legacy=0x7fbdac0b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:28.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7fbdb13fb640 1 -- 172.21.15.16:0/4053684220 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdac1628c0 con 0x7fbdac0d3ec0 2024-09-19T07:43:28.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7f279effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a0157da0 0x7f27a0200e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7fbdabfff640 1 --2- 172.21.15.16:0/4053684220 >> v2:172.21.15.146:3300/0 conn(0x7fbdac0d2290 0x7fbdac161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:28.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7f279effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a0157da0 0x7f27a0200e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57538/0 (socket says 172.21.15.16:57538) 2024-09-19T07:43:28.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7f279effd640 1 -- 172.21.15.16:0/532676006 learned_addr learned my addr 172.21.15.16:0/532676006 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:28.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7f279ffff640 1 --2- 172.21.15.16:0/532676006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a0201360 0x7f27a0201810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7efce29cc640 1 -- 172.21.15.16:0/431283769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcdc0ae740 msgr2=0x7efcdc0b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7efce29cc640 1 --2- 172.21.15.16:0/431283769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcdc0ae740 0x7efcdc0b4770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0aff10 tx=0x7efcd402a730 comp rx=0 tx=0).stop 2024-09-19T07:43:28.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f279effd640 1 -- 172.21.15.16:0/532676006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a0201360 msgr2=0x7f27a0201810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7f570d59a640 1 -- 172.21.15.16:0/3100509297 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5708036070 con 0x7f57100d0df0 2024-09-19T07:43:28.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.230+0000 7f570d59a640 1 -- 172.21.15.16:0/3100509297 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5708031600 con 0x7f57100d0df0 2024-09-19T07:43:28.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f5714c74640 1 -- 172.21.15.16:0/3100509297 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57100d0df0 msgr2=0x7f57100d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f570d59a640 1 -- 172.21.15.16:0/3100509297 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57080319d0 con 0x7f57100d0df0 2024-09-19T07:43:28.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f5714c74640 1 --2- 172.21.15.16:0/3100509297 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57100d0df0 0x7f57100d1280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5708009e50 tx=0x7f570802a6c0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f279effd640 1 --2- 172.21.15.16:0/532676006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a0201360 0x7f27a0201810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f279effd640 1 -- 172.21.15.16:0/532676006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2788004800 msgr2=0x7f27a0150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f279effd640 1 --2- 172.21.15.16:0/532676006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2788004800 0x7f27a0150730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f279effd640 1 -- 172.21.15.16:0/532676006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a01fb340 con 0x7f27a0157da0 2024-09-19T07:43:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.229+0000 7f2a1264a640 1 -- 172.21.15.16:0/173064960 wait complete. 2024-09-19T07:43:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f5714c74640 1 -- 172.21.15.16:0/3100509297 shutdown_connections 2024-09-19T07:43:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f5714c74640 1 --2- 172.21.15.16:0/3100509297 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57100d0df0 0x7f57100d1280 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f5714c74640 1 --2- 172.21.15.16:0/3100509297 >> v2:172.21.15.16:3300/0 conn(0x7f57101626c0 0x7f5710162aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f5714c74640 1 -- 172.21.15.16:0/3100509297 >> 172.21.15.16:0/3100509297 conn(0x7f571009d770 msgr2=0x7f571008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7efce29cc640 1 -- 172.21.15.16:0/431283769 shutdown_connections 2024-09-19T07:43:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7efce29cc640 1 --2- 172.21.15.16:0/431283769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcdc0ae740 0x7efcdc0b4770 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7efce29cc640 1 --2- 172.21.15.16:0/431283769 >> v2:172.21.15.146:3300/0 conn(0x7efcdc06bc10 0x7efcdc0ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7efce29cc640 1 -- 172.21.15.16:0/431283769 >> 172.21.15.16:0/431283769 conn(0x7efcdc0d0890 msgr2=0x7efcdc0d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7fe6dca26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005a60 0x7fe6d017d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f279effd640 1 --2- 172.21.15.16:0/532676006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a0157da0 0x7f27a0200e20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f27a015fe50 tx=0x7f27a00b2ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f2a1264a640 1 Processor -- start 2024-09-19T07:43:28.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f5714c74640 1 -- 172.21.15.16:0/3100509297 shutdown_connections 2024-09-19T07:43:28.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f279ffff640 1 --2- 172.21.15.16:0/532676006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a0201360 0x7f27a0201810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:28.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f2787fff640 1 -- 172.21.15.16:0/532676006 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279003f030 con 0x7f27a0157da0 2024-09-19T07:43:28.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7fbdb13fb640 1 --2- 172.21.15.16:0/4053684220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdac0d3ec0 0x7fbdac0d4300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbdac08e4c0 tx=0x7fbda402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce63ff02d48f981 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f5714c74640 1 -- 172.21.15.16:0/3100509297 wait complete. 2024-09-19T07:43:28.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f2787fff640 1 -- 172.21.15.16:0/532676006 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27900049f0 con 0x7f27a0157da0 2024-09-19T07:43:28.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f2787fff640 1 -- 172.21.15.16:0/532676006 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790012070 con 0x7f27a0157da0 2024-09-19T07:43:28.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f27a5b31640 1 -- 172.21.15.16:0/532676006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a0150d80 con 0x7f27a0157da0 2024-09-19T07:43:28.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7ff726ffd640 1 -- 172.21.15.16:0/1799469670 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff738099070 con 0x7ff7300e2b90 2024-09-19T07:43:28.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7f27a5b31640 1 -- 172.21.15.16:0/532676006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a0151340 con 0x7f27a0157da0 2024-09-19T07:43:28.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7ff726ffd640 1 -- 172.21.15.16:0/1799469670 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff738068fe0 con 0x7ff7300e2b90 2024-09-19T07:43:28.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7fe6d6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005a60 0x7fe6d017d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7fe6d6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005a60 0x7fe6d017d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57544/0 (socket says 172.21.15.16:57544) 2024-09-19T07:43:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7fe6d6d9d640 1 -- 172.21.15.16:0/3235775578 learned_addr learned my addr 172.21.15.16:0/3235775578 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7fe6dca26640 1 --2- 172.21.15.16:0/3235775578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6d004eb60 0x7fe6d00e4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7efce29cc640 1 -- 172.21.15.16:0/431283769 shutdown_connections 2024-09-19T07:43:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7fe6dca26640 1 --2- 172.21.15.16:0/3235775578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6d00e7120 0x7fe6d00e98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7ff726ffd640 1 -- 172.21.15.16:0/1799469670 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff738075750 con 0x7ff7300e2b90 2024-09-19T07:43:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7ff73759e640 1 -- 172.21.15.16:0/1799469670 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7300d72b0 con 0x7ff7300e2b90 2024-09-19T07:43:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7ff73759e640 1 -- 172.21.15.16:0/1799469670 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7300d7870 con 0x7ff7300e2b90 2024-09-19T07:43:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7f406ffff640 1 Processor -- start 2024-09-19T07:43:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7f406ffff640 1 -- start start 2024-09-19T07:43:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7fe6dca26640 1 -- 172.21.15.16:0/3235775578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d0053a10 con 0x7fe6d0005a60 2024-09-19T07:43:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7fe6d659c640 1 --2- 172.21.15.16:0/3235775578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6d004eb60 0x7fe6d00e4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7fe6dca26640 1 -- 172.21.15.16:0/3235775578 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d0007240 con 0x7fe6d00e7120 2024-09-19T07:43:28.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7fe6d759e640 1 --2- 172.21.15.16:0/3235775578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6d00e7120 0x7fe6d00e98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7fe6dca26640 1 -- 172.21.15.16:0/3235775578 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d001dc40 con 0x7fe6d004eb60 2024-09-19T07:43:28.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7efce29cc640 1 -- 172.21.15.16:0/431283769 wait complete. 2024-09-19T07:43:28.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.231+0000 7f2a1264a640 1 -- start start 2024-09-19T07:43:28.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7fbdab7fe640 1 -- 172.21.15.16:0/4053684220 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4036070 con 0x7fbdac0d3ec0 2024-09-19T07:43:28.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.233+0000 7f2a1264a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a0c0b37c0 0x7f2a0c15bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.233+0000 7f2a1264a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c0b5500 0x7f2a0c161120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.233+0000 7f2a1264a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a0c15cc70 0x7f2a0c158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.233+0000 7f2a1264a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c071e20 con 0x7f2a0c0b5500 2024-09-19T07:43:28.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7fbdab7fe640 1 -- 172.21.15.16:0/4053684220 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbda4031600 con 0x7fbdac0d3ec0 2024-09-19T07:43:28.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.232+0000 7fbdab7fe640 1 -- 172.21.15.16:0/4053684220 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda40319d0 con 0x7fbdac0d3ec0 2024-09-19T07:43:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.233+0000 7f2a1264a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c0b42d0 con 0x7f2a0c0b37c0 2024-09-19T07:43:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.233+0000 7f2a1264a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2a0c0b3bd0 con 0x7f2a0c15cc70 2024-09-19T07:43:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.233+0000 7fe6d6d9d640 1 -- 172.21.15.16:0/3235775578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6d004eb60 msgr2=0x7fe6d00e4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.233+0000 7fe6d6d9d640 1 --2- 172.21.15.16:0/3235775578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6d004eb60 0x7fe6d00e4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.233+0000 7fe6d6d9d640 1 -- 172.21.15.16:0/3235775578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6d00e7120 msgr2=0x7fe6d00e98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.233+0000 7fe6d6d9d640 1 --2- 172.21.15.16:0/3235775578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6d00e7120 0x7fe6d00e98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.233+0000 7fe6d6d9d640 1 -- 172.21.15.16:0/3235775578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d00e8550 con 0x7fe6d0005a60 2024-09-19T07:43:28.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.233+0000 7fbdb2e5e640 1 -- 172.21.15.16:0/4053684220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdac0d3ec0 msgr2=0x7fbdac0d4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.233+0000 7fbdb2e5e640 1 --2- 172.21.15.16:0/4053684220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdac0d3ec0 0x7fbdac0d4300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbdac08e4c0 tx=0x7fbda402a6f0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.233+0000 7fe6d659c640 1 --2- 172.21.15.16:0/3235775578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6d004eb60 0x7fe6d00e4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.234+0000 7f406ffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4058004fd0 0x7f40680dd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.234+0000 7f406ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f406804b8f0 0x7f406804bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.234+0000 7fe6d6d9d640 1 --2- 172.21.15.16:0/3235775578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005a60 0x7fe6d017d200 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800e8e0 tx=0x7fe6c800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.234+0000 7f406effd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4058004fd0 0x7f40680dd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.234+0000 7f406effd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4058004fd0 0x7f40680dd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:57962/0 (socket says 172.21.15.16:57962) 2024-09-19T07:43:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.234+0000 7f406effd640 1 -- 172.21.15.16:0/915321831 learned_addr learned my addr 172.21.15.16:0/915321831 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.234+0000 7f5714c74640 1 Processor -- start 2024-09-19T07:43:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.234+0000 7f406ffff640 1 --2- 172.21.15.16:0/915321831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4068049540 0x7f40680499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7f5714c74640 1 -- start start 2024-09-19T07:43:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.234+0000 7f2787fff640 1 -- 172.21.15.16:0/532676006 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2790042070 con 0x7f27a0157da0 2024-09-19T07:43:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7f406e7fc640 1 --2- 172.21.15.16:0/915321831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f406804b8f0 0x7f406804bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.233+0000 7f2a11e49640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a0c15cc70 0x7f2a0c158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7ff73759e640 1 -- 172.21.15.16:0/1799469670 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff730007190 con 0x7ff7300e2b90 2024-09-19T07:43:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7fbdb2e5e640 1 -- 172.21.15.16:0/4053684220 shutdown_connections 2024-09-19T07:43:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7fbdb2e5e640 1 --2- 172.21.15.16:0/4053684220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdac0d3ec0 0x7fbdac0d4300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7fbdb2e5e640 1 --2- 172.21.15.16:0/4053684220 >> v2:172.21.15.146:3300/0 conn(0x7fbdac0d2290 0x7fbdac161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7fbdb2e5e640 1 -- 172.21.15.16:0/4053684220 >> 172.21.15.16:0/4053684220 conn(0x7fbdac08bf80 msgr2=0x7fbdac09d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7f406f7fe640 1 --2- 172.21.15.16:0/915321831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4068049540 0x7f40680499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7f406ffff640 1 -- 172.21.15.16:0/915321831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4068053a10 con 0x7f406804b8f0 2024-09-19T07:43:28.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7f406ffff640 1 -- 172.21.15.16:0/915321831 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4068007240 con 0x7f4058004fd0 2024-09-19T07:43:28.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7f406ffff640 1 -- 172.21.15.16:0/915321831 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f406801dc40 con 0x7f4068049540 2024-09-19T07:43:28.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7ff726ffd640 1 -- 172.21.15.16:0/1799469670 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff73809c070 con 0x7ff7300e2b90 2024-09-19T07:43:28.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.234+0000 7efce29cc640 1 Processor -- start 2024-09-19T07:43:28.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.234+0000 7efce29cc640 1 -- start start 2024-09-19T07:43:28.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.233+0000 7f2a11e49640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a0c15cc70 0x7f2a0c158de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:34242/0 (socket says 172.21.15.16:34242) 2024-09-19T07:43:28.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7fe6bf7fe640 1 -- 172.21.15.16:0/3235775578 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8017070 con 0x7fe6d0005a60 2024-09-19T07:43:28.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.233+0000 7f2a11e49640 1 -- 172.21.15.16:0/179194477 learned_addr learned my addr 172.21.15.16:0/179194477 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:28.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.233+0000 7f2a11648640 1 --2- 172.21.15.16:0/179194477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a0c0b37c0 0x7f2a0c15bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.234+0000 7f2a11e49640 1 -- 172.21.15.16:0/179194477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a0c0b37c0 msgr2=0x7f2a0c15bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7f406effd640 1 -- 172.21.15.16:0/915321831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4068049540 msgr2=0x7f40680499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7f406effd640 1 --2- 172.21.15.16:0/915321831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4068049540 0x7f40680499c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7f406effd640 1 -- 172.21.15.16:0/915321831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f406804b8f0 msgr2=0x7f406804bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7f406effd640 1 --2- 172.21.15.16:0/915321831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f406804b8f0 0x7f406804bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7f406effd640 1 -- 172.21.15.16:0/915321831 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40680dde50 con 0x7f4058004fd0 2024-09-19T07:43:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7f406e7fc640 1 --2- 172.21.15.16:0/915321831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f406804b8f0 0x7f406804bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7fe6dca26640 1 -- 172.21.15.16:0/3235775578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d00e8830 con 0x7fe6d0005a60 2024-09-19T07:43:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7fe6bf7fe640 1 -- 172.21.15.16:0/3235775578 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6c800c940 con 0x7fe6d0005a60 2024-09-19T07:43:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.234+0000 7f2a11e49640 1 --2- 172.21.15.16:0/179194477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a0c0b37c0 0x7f2a0c15bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.234+0000 7f2a11e49640 1 -- 172.21.15.16:0/179194477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c0b5500 msgr2=0x7f2a0c161120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:43:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7f2a10e47640 1 --2- 172.21.15.16:0/179194477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c0b5500 0x7f2a0c161120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7f2a11e49640 1 --2- 172.21.15.16:0/179194477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c0b5500 0x7f2a0c161120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.235+0000 7f2a11e49640 1 -- 172.21.15.16:0/179194477 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c15b140 con 0x7f2a0c15cc70 2024-09-19T07:43:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7ff726ffd640 1 --2- 172.21.15.16:0/1799469670 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff71806c300 0x7ff71806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7ff726ffd640 1 -- 172.21.15.16:0/1799469670 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff738106b50 con 0x7ff7300e2b90 2024-09-19T07:43:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7efce29cc640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efcdc06bc10 0x7efcdc145190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7efce29cc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcdc0ae740 0x7efcdc157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7efce29cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcdc15e160 0x7efcdc0b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7efce29cc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc0b4cb0 con 0x7efcdc15e160 2024-09-19T07:43:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7efce29cc640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc15e540 con 0x7efcdc0ae740 2024-09-19T07:43:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7f5714c74640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57100d0df0 0x7f57101593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7efce29cc640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efcdc0b4e30 con 0x7efcdc06bc10 2024-09-19T07:43:28.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7f406effd640 1 --2- 172.21.15.16:0/915321831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4058004fd0 0x7f40680dd910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f406000e930 tx=0x7f406000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7f5714c74640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57100d2980 0x7f57100b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7efce11c9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcdc0ae740 0x7efcdc157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7efce11c9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcdc0ae740 0x7efcdc157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:57972/0 (socket says 172.21.15.16:57972) 2024-09-19T07:43:28.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7efce11c9640 1 -- 172.21.15.16:0/1549458071 learned_addr learned my addr 172.21.15.16:0/1549458071 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:28.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7ff73659c640 1 --2- 172.21.15.16:0/1799469670 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff71806c300 0x7ff71806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7f2a11e49640 1 --2- 172.21.15.16:0/179194477 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a0c15cc70 0x7f2a0c158de0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f29fc00b410 tx=0x7f29fc00b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7efce19ca640 1 --2- 172.21.15.16:0/1549458071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efcdc06bc10 0x7efcdc145190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7efce21cb640 1 --2- 172.21.15.16:0/1549458071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcdc15e160 0x7efcdc0b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7f27a5b31640 1 -- 172.21.15.16:0/532676006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2780005290 con 0x7f27a0157da0 2024-09-19T07:43:28.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7f5714c74640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57101626c0 0x7f57100b7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7f2a01ffb640 1 -- 172.21.15.16:0/179194477 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc015030 con 0x7f2a0c15cc70 2024-09-19T07:43:28.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7f2a01ffb640 1 -- 172.21.15.16:0/179194477 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29fc004020 con 0x7f2a0c15cc70 2024-09-19T07:43:28.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7f2a01ffb640 1 -- 172.21.15.16:0/179194477 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc004410 con 0x7f2a0c15cc70 2024-09-19T07:43:28.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7efce11c9640 1 -- 172.21.15.16:0/1549458071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efcdc06bc10 msgr2=0x7efcdc145190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7efce11c9640 1 --2- 172.21.15.16:0/1549458071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efcdc06bc10 0x7efcdc145190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7efce11c9640 1 -- 172.21.15.16:0/1549458071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcdc15e160 msgr2=0x7efcdc0b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7efce11c9640 1 --2- 172.21.15.16:0/1549458071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcdc15e160 0x7efcdc0b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.237+0000 7efce11c9640 1 -- 172.21.15.16:0/1549458071 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcdc1608f0 con 0x7efcdc0ae740 2024-09-19T07:43:28.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.237+0000 7efce19ca640 1 --2- 172.21.15.16:0/1549458071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efcdc06bc10 0x7efcdc145190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:28.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.237+0000 7f570dd9b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57100d2980 0x7f57100b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.237+0000 7f570dd9b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57100d2980 0x7f57100b7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:34280/0 (socket says 172.21.15.16:34280) 2024-09-19T07:43:28.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.237+0000 7f2787fff640 1 --2- 172.21.15.16:0/532676006 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f278c06c3b0 0x7f278c06e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.237+0000 7f570dd9b640 1 -- 172.21.15.16:0/84060489 learned_addr learned my addr 172.21.15.16:0/84060489 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:28.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.237+0000 7f5714c74640 1 -- 172.21.15.16:0/84060489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57100d2df0 con 0x7f57100d0df0 2024-09-19T07:43:28.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.237+0000 7efce11c9640 1 --2- 172.21.15.16:0/1549458071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcdc0ae740 0x7efcdc157680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efccc00e9e0 tx=0x7efccc00eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.237+0000 7f570ed9d640 1 --2- 172.21.15.16:0/84060489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57101626c0 0x7f57100b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.237+0000 7f5714c74640 1 -- 172.21.15.16:0/84060489 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5710163b10 con 0x7f57101626c0 2024-09-19T07:43:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.237+0000 7f5714c74640 1 -- 172.21.15.16:0/84060489 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f57100d43c0 con 0x7f57100d2980 2024-09-19T07:43:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.237+0000 7f279f7fe640 1 --2- 172.21.15.16:0/532676006 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f278c06c3b0 0x7f278c06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.237+0000 7f2787fff640 1 -- 172.21.15.16:0/532676006 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f27900ab220 con 0x7f27a0157da0 2024-09-19T07:43:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.237+0000 7fbdb2e5e640 1 -- 172.21.15.16:0/4053684220 shutdown_connections 2024-09-19T07:43:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.237+0000 7fbdb2e5e640 1 -- 172.21.15.16:0/4053684220 wait complete. 2024-09-19T07:43:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.237+0000 7f2a1264a640 1 -- 172.21.15.16:0/179194477 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0c15b480 con 0x7f2a0c15cc70 2024-09-19T07:43:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.237+0000 7f570e59c640 1 --2- 172.21.15.16:0/84060489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57100d0df0 0x7f57101593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.237+0000 7f2a1264a640 1 -- 172.21.15.16:0/179194477 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0c15c440 con 0x7f2a0c15cc70 2024-09-19T07:43:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.236+0000 7fe6dca26640 1 -- 172.21.15.16:0/3235775578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d00e8df0 con 0x7fe6d0005a60 2024-09-19T07:43:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.237+0000 7fe6dca26640 1 -- 172.21.15.16:0/3235775578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d0047b50 con 0x7fe6d0005a60 2024-09-19T07:43:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.238+0000 7f570dd9b640 1 -- 172.21.15.16:0/84060489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57101626c0 msgr2=0x7f57100b7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.238+0000 7f570dd9b640 1 --2- 172.21.15.16:0/84060489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57101626c0 0x7f57100b7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.238+0000 7f570dd9b640 1 -- 172.21.15.16:0/84060489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57100d0df0 msgr2=0x7f57101593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.238+0000 7f570dd9b640 1 --2- 172.21.15.16:0/84060489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57100d0df0 0x7f57101593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.238+0000 7f570dd9b640 1 -- 172.21.15.16:0/84060489 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57100b81b0 con 0x7f57100d2980 2024-09-19T07:43:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.238+0000 7f570e59c640 1 --2- 172.21.15.16:0/84060489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57100d0df0 0x7f57101593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.238+0000 7f279f7fe640 1 --2- 172.21.15.16:0/532676006 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f278c06c3b0 0x7f278c06e870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f27a00b65d0 tx=0x7f2798007660 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.238+0000 7f570dd9b640 1 --2- 172.21.15.16:0/84060489 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57100d2980 0x7f57100b7210 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f570400c480 tx=0x7f570400c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.238+0000 7f56feffd640 1 -- 172.21.15.16:0/84060489 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704017070 con 0x7f57100d2980 2024-09-19T07:43:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.238+0000 7f56feffd640 1 -- 172.21.15.16:0/84060489 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57040035f0 con 0x7f57100d2980 2024-09-19T07:43:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.238+0000 7f56feffd640 1 -- 172.21.15.16:0/84060489 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704003a20 con 0x7f57100d2980 2024-09-19T07:43:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.238+0000 7f5714c74640 1 -- 172.21.15.16:0/84060489 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5710165bf0 con 0x7f57100d2980 2024-09-19T07:43:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.238+0000 7f5714c74640 1 -- 172.21.15.16:0/84060489 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5710159f30 con 0x7f57100d2980 2024-09-19T07:43:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.239+0000 7f2a1264a640 1 -- 172.21.15.16:0/179194477 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a0c0b3280 con 0x7f2a0c15cc70 2024-09-19T07:43:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.239+0000 7ff73659c640 1 --2- 172.21.15.16:0/1799469670 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff71806c300 0x7ff71806e7c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff72c034000 tx=0x7ff72c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.239+0000 7fe6bf7fe640 1 -- 172.21.15.16:0/3235775578 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c800cd30 con 0x7fe6d0005a60 2024-09-19T07:43:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.239+0000 7fe6bf7fe640 1 -- 172.21.15.16:0/3235775578 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe6c801a070 con 0x7fe6d0005a60 2024-09-19T07:43:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.239+0000 7fbdb2e5e640 1 Processor -- start 2024-09-19T07:43:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.239+0000 7fbdb2e5e640 1 -- start start 2024-09-19T07:43:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.240+0000 7fe6bf7fe640 1 --2- 172.21.15.16:0/3235775578 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe6c406c3b0 0x7fe6c406e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.240+0000 7f2a01ffb640 1 -- 172.21.15.16:0/179194477 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f29fc018070 con 0x7f2a0c15cc70 2024-09-19T07:43:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.240+0000 7fe6d659c640 1 --2- 172.21.15.16:0/3235775578 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe6c406c3b0 0x7fe6c406e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.240+0000 7fe6bf7fe640 1 -- 172.21.15.16:0/3235775578 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe6c808bc10 con 0x7fe6d0005a60 2024-09-19T07:43:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.240+0000 7f5714c74640 1 -- 172.21.15.16:0/84060489 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5710162aa0 con 0x7f57100d2980 2024-09-19T07:43:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.240+0000 7f2a01ffb640 1 --2- 172.21.15.16:0/179194477 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f29f006c2b0 0x7f29f006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.241+0000 7f56feffd640 1 -- 172.21.15.16:0/84060489 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5704003bc0 con 0x7f57100d2980 2024-09-19T07:43:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.241+0000 7fe6d659c640 1 --2- 172.21.15.16:0/3235775578 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe6c406c3b0 0x7fe6c406e870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0008020 tx=0x7fe6d80aa000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.241+0000 7fbdb2e5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdac0d2290 0x7fbdac0cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.241+0000 7f2a01ffb640 1 -- 172.21.15.16:0/179194477 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f29fc08b2e0 con 0x7f2a0c15cc70 2024-09-19T07:43:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.241+0000 7fbdb2e5e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdac0d3ec0 0x7fbdac0cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.241+0000 7fbdb0bfa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdac0d2290 0x7fbdac0cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.241+0000 7fbdb2e5e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdac15e490 0x7fbdac0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.241+0000 7fbdb2e5e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbdac0d4840 con 0x7fbdac0d2290 2024-09-19T07:43:28.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.241+0000 7fbdb0bfa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdac0d2290 0x7fbdac0cb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57590/0 (socket says 172.21.15.16:57590) 2024-09-19T07:43:28.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.241+0000 7fbdb0bfa640 1 -- 172.21.15.16:0/1709272706 learned_addr learned my addr 172.21.15.16:0/1709272706 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:28.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.241+0000 7fbdb2e5e640 1 -- 172.21.15.16:0/1709272706 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbdac0cefc0 con 0x7fbdac0d3ec0 2024-09-19T07:43:28.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.241+0000 7fbdb2e5e640 1 -- 172.21.15.16:0/1709272706 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbdac0cf4e0 con 0x7fbdac15e490 2024-09-19T07:43:28.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.241+0000 7fbdabfff640 1 --2- 172.21.15.16:0/1709272706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdac0d3ec0 0x7fbdac0cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.241+0000 7f56feffd640 1 --2- 172.21.15.16:0/84060489 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f56f006c2b0 0x7f56f006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.241+0000 7f2a11648640 1 --2- 172.21.15.16:0/179194477 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f29f006c2b0 0x7f29f006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.241+0000 7fbdb13fb640 1 --2- 172.21.15.16:0/1709272706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdac15e490 0x7fbdac0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.242+0000 7f40577fe640 1 -- 172.21.15.16:0/915321831 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060017070 con 0x7f4058004fd0 2024-09-19T07:43:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.242+0000 7f40577fe640 1 -- 172.21.15.16:0/915321831 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f406000c8e0 con 0x7f4058004fd0 2024-09-19T07:43:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.242+0000 7f40577fe640 1 -- 172.21.15.16:0/915321831 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406000ccb0 con 0x7f4058004fd0 2024-09-19T07:43:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.242+0000 7f56feffd640 1 -- 172.21.15.16:0/84060489 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f570408aa00 con 0x7f57100d2980 2024-09-19T07:43:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.242+0000 7efcd27fc640 1 -- 172.21.15.16:0/1549458071 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc017070 con 0x7efcdc0ae740 2024-09-19T07:43:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.242+0000 7f570e59c640 1 --2- 172.21.15.16:0/84060489 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f56f006c2b0 0x7f56f006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.242+0000 7ff726ffd640 1 -- 172.21.15.16:0/1799469670 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff738091070 con 0x7ff7300e2b90 2024-09-19T07:43:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.242+0000 7f406ffff640 1 -- 172.21.15.16:0/915321831 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406800b470 con 0x7f4058004fd0 2024-09-19T07:43:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.242+0000 7f406ffff640 1 -- 172.21.15.16:0/915321831 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4068048a60 con 0x7f4058004fd0 2024-09-19T07:43:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.242+0000 7efce29cc640 1 -- 172.21.15.16:0/1549458071 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcdc160b90 con 0x7efcdc0ae740 2024-09-19T07:43:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.242+0000 7efcd27fc640 1 -- 172.21.15.16:0/1549458071 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efccc00c9d0 con 0x7efcdc0ae740 2024-09-19T07:43:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.242+0000 7efce29cc640 1 -- 172.21.15.16:0/1549458071 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcdc161150 con 0x7efcdc0ae740 2024-09-19T07:43:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.243+0000 7efcd27fc640 1 -- 172.21.15.16:0/1549458071 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc00cdc0 con 0x7efcdc0ae740 2024-09-19T07:43:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.242+0000 7fbdb13fb640 1 -- 172.21.15.16:0/1709272706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdac0d3ec0 msgr2=0x7fbdac0cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.242+0000 7fbdb13fb640 1 --2- 172.21.15.16:0/1709272706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdac0d3ec0 0x7fbdac0cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.242+0000 7fbdb13fb640 1 -- 172.21.15.16:0/1709272706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdac0d2290 msgr2=0x7fbdac0cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.242+0000 7fbdb13fb640 1 --2- 172.21.15.16:0/1709272706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdac0d2290 0x7fbdac0cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.242+0000 7fbdb13fb640 1 -- 172.21.15.16:0/1709272706 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdac0cc8e0 con 0x7fbdac15e490 2024-09-19T07:43:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.242+0000 7fbdb13fb640 1 --2- 172.21.15.16:0/1709272706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdac15e490 0x7fbdac0cc230 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbdac0d4f10 tx=0x7fbda4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.243+0000 7f406ffff640 1 -- 172.21.15.16:0/915321831 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4068047b50 con 0x7f4058004fd0 2024-09-19T07:43:28.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.243+0000 7f2a11648640 1 --2- 172.21.15.16:0/179194477 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f29f006c2b0 0x7f29f006e770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a08034000 tx=0x7f2a0802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.244+0000 7efcbffff640 1 -- 172.21.15.16:0/1549458071 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcdc159ba0 con 0x7efcdc0ae740 2024-09-19T07:43:28.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.244+0000 7f570e59c640 1 --2- 172.21.15.16:0/84060489 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f56f006c2b0 0x7f56f006e770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5700006fd0 tx=0x7f5700008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.244+0000 7f40577fe640 1 -- 172.21.15.16:0/915321831 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f406001a070 con 0x7f4058004fd0 2024-09-19T07:43:28.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.245+0000 7efcd27fc640 1 -- 172.21.15.16:0/1549458071 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7efccc01a070 con 0x7efcdc0ae740 2024-09-19T07:43:28.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.245+0000 7f40577fe640 1 --2- 172.21.15.16:0/915321831 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f405c06c330 0x7f405c06e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.245+0000 7efcd27fc640 1 --2- 172.21.15.16:0/1549458071 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efcc006c2b0 0x7efcc006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.245+0000 7efcd27fc640 1 -- 172.21.15.16:0/1549458071 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7efccc08b710 con 0x7efcdc0ae740 2024-09-19T07:43:28.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.245+0000 7f40577fe640 1 -- 172.21.15.16:0/915321831 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f406008b840 con 0x7f4058004fd0 2024-09-19T07:43:28.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.245+0000 7f406e7fc640 1 --2- 172.21.15.16:0/915321831 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f405c06c330 0x7f405c06e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.246+0000 7efce19ca640 1 --2- 172.21.15.16:0/1549458071 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efcc006c2b0 0x7efcc006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.246+0000 7f2a01ffb640 1 -- 172.21.15.16:0/179194477 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2a0c15b480 con 0x7f2a0c15cc70 2024-09-19T07:43:28.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.246+0000 7f406e7fc640 1 --2- 172.21.15.16:0/915321831 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f405c06c330 0x7f405c06e7f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f407006b500 tx=0x7f407008e040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.247+0000 7f2787fff640 1 -- 172.21.15.16:0/532676006 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f279000b070 con 0x7f27a0157da0 2024-09-19T07:43:28.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.247+0000 7efce19ca640 1 --2- 172.21.15.16:0/1549458071 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efcc006c2b0 0x7efcc006e770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efcd80017a0 tx=0x7efcd8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.250+0000 7f40577fe640 1 -- 172.21.15.16:0/915321831 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4060059700 con 0x7f4058004fd0 2024-09-19T07:43:28.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.250+0000 7f56feffd640 1 -- 172.21.15.16:0/84060489 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5704058870 con 0x7f57100d2980 2024-09-19T07:43:28.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.250+0000 7fbda97fa640 1 -- 172.21.15.16:0/1709272706 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4041020 con 0x7fbdac15e490 2024-09-19T07:43:28.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.251+0000 7fbda97fa640 1 -- 172.21.15.16:0/1709272706 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbda4008710 con 0x7fbdac15e490 2024-09-19T07:43:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.251+0000 7fbda97fa640 1 -- 172.21.15.16:0/1709272706 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4041020 con 0x7fbdac15e490 2024-09-19T07:43:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.251+0000 7fbdb2e5e640 1 -- 172.21.15.16:0/1709272706 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdac0b1b30 con 0x7fbdac15e490 2024-09-19T07:43:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.252+0000 7fbdb2e5e640 1 -- 172.21.15.16:0/1709272706 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdac0b20f0 con 0x7fbdac15e490 2024-09-19T07:43:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.253+0000 7fbdb2e5e640 1 -- 172.21.15.16:0/1709272706 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdac161930 con 0x7fbdac15e490 2024-09-19T07:43:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.254+0000 7fbda97fa640 1 -- 172.21.15.16:0/1709272706 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbda4044020 con 0x7fbdac15e490 2024-09-19T07:43:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.255+0000 7efcd27fc640 1 -- 172.21.15.16:0/1549458071 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7efccc0594f0 con 0x7efcdc0ae740 2024-09-19T07:43:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.255+0000 7fbda97fa640 1 --2- 172.21.15.16:0/1709272706 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbd9006c4b0 0x7fbd9006e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:28.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.255+0000 7fbda97fa640 1 -- 172.21.15.16:0/1709272706 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbda40abcb0 con 0x7fbdac15e490 2024-09-19T07:43:28.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.256+0000 7fe6bf7fe640 1 -- 172.21.15.16:0/3235775578 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe6c80599d0 con 0x7fe6d0005a60 2024-09-19T07:43:28.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.256+0000 7fbdb0bfa640 1 --2- 172.21.15.16:0/1709272706 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbd9006c4b0 0x7fbd9006e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:28.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.256+0000 7fbdb0bfa640 1 --2- 172.21.15.16:0/1709272706 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbd9006c4b0 0x7fbd9006e970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbda00099b0 tx=0x7fbda0009110 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:28.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.259+0000 7fbda97fa640 1 -- 172.21.15.16:0/1709272706 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fbdac161930 con 0x7fbdac15e490 2024-09-19T07:43:28.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.648+0000 7ff73759e640 1 -- 172.21.15.16:0/1799469670 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff730039a20 con 0x7ff7300e2b90 2024-09-19T07:43:28.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.649+0000 7ff726ffd640 1 -- 172.21.15.16:0/1799469670 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff7380d4910 con 0x7ff7300e2b90 2024-09-19T07:43:28.650 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-09-19T07:43:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.653+0000 7ff724ff9640 1 -- 172.21.15.16:0/1799469670 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff71806c300 msgr2=0x7ff71806e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.653+0000 7ff724ff9640 1 --2- 172.21.15.16:0/1799469670 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff71806c300 0x7ff71806e7c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff72c034000 tx=0x7ff72c02e000 comp rx=0 tx=0).stop 2024-09-19T07:43:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.654+0000 7ff724ff9640 1 -- 172.21.15.16:0/1799469670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7300e2b90 msgr2=0x7ff7300cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.654+0000 7ff724ff9640 1 --2- 172.21.15.16:0/1799469670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7300e2b90 0x7ff7300cfca0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff73806aa20 tx=0x7ff738069710 comp rx=0 tx=0).stop 2024-09-19T07:43:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.654+0000 7ff724ff9640 1 -- 172.21.15.16:0/1799469670 shutdown_connections 2024-09-19T07:43:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.654+0000 7ff724ff9640 1 --2- 172.21.15.16:0/1799469670 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff71806c300 0x7ff71806e7c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.654+0000 7ff724ff9640 1 --2- 172.21.15.16:0/1799469670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7300e2b90 0x7ff7300cfca0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.654+0000 7ff724ff9640 1 --2- 172.21.15.16:0/1799469670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff730053f40 0x7ff7300d6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.654+0000 7ff724ff9640 1 --2- 172.21.15.16:0/1799469670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff730006d40 0x7ff73000d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.654+0000 7ff724ff9640 1 -- 172.21.15.16:0/1799469670 >> 172.21.15.16:0/1799469670 conn(0x7ff73003a520 msgr2=0x7ff730038970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.654+0000 7ff724ff9640 1 -- 172.21.15.16:0/1799469670 shutdown_connections 2024-09-19T07:43:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.655+0000 7ff724ff9640 1 -- 172.21.15.16:0/1799469670 wait complete. 2024-09-19T07:43:28.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.661+0000 7f5714c74640 1 -- 172.21.15.16:0/84060489 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f57100ba360 con 0x7f57100d2980 2024-09-19T07:43:28.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.661+0000 7f406ffff640 1 -- 172.21.15.16:0/915321831 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f406801ce60 con 0x7f4058004fd0 2024-09-19T07:43:28.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.661+0000 7f56feffd640 1 -- 172.21.15.16:0/84060489 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f570405cb70 con 0x7f57100d2980 2024-09-19T07:43:28.662 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-09-19T07:43:28.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.661+0000 7f40577fe640 1 -- 172.21.15.16:0/915321831 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f406005da00 con 0x7f4058004fd0 2024-09-19T07:43:28.662 INFO:teuthology.orchestra.run.smithi016.stdout:60129542151 2024-09-19T07:43:28.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.662+0000 7f2a1264a640 1 -- 172.21.15.16:0/179194477 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f2a0c074f50 con 0x7f2a0c15cc70 2024-09-19T07:43:28.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.662+0000 7f2a01ffb640 1 -- 172.21.15.16:0/179194477 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f29fc059670 con 0x7f2a0c15cc70 2024-09-19T07:43:28.664 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-09-19T07:43:28.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.664+0000 7f27a5b31640 1 -- 172.21.15.16:0/532676006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f2780005580 con 0x7f27a0157da0 2024-09-19T07:43:28.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f2787fff640 1 -- 172.21.15.16:0/532676006 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f27900795e0 con 0x7f27a0157da0 2024-09-19T07:43:28.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f56fcff9640 1 -- 172.21.15.16:0/84060489 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f56f006c2b0 msgr2=0x7f56f006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f56fcff9640 1 --2- 172.21.15.16:0/84060489 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f56f006c2b0 0x7f56f006e770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5700006fd0 tx=0x7f5700008040 comp rx=0 tx=0).stop 2024-09-19T07:43:28.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f56fcff9640 1 -- 172.21.15.16:0/84060489 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57100d2980 msgr2=0x7f57100b7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f56fcff9640 1 --2- 172.21.15.16:0/84060489 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57100d2980 0x7f57100b7210 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f570400c480 tx=0x7f570400c940 comp rx=0 tx=0).stop 2024-09-19T07:43:28.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f56fcff9640 1 -- 172.21.15.16:0/84060489 shutdown_connections 2024-09-19T07:43:28.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f56fcff9640 1 --2- 172.21.15.16:0/84060489 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f56f006c2b0 0x7f56f006e770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f56fcff9640 1 --2- 172.21.15.16:0/84060489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57101626c0 0x7f57100b7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f56fcff9640 1 --2- 172.21.15.16:0/84060489 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57100d2980 0x7f57100b7210 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f56fcff9640 1 --2- 172.21.15.16:0/84060489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57100d0df0 0x7f57101593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f56fcff9640 1 -- 172.21.15.16:0/84060489 >> 172.21.15.16:0/84060489 conn(0x7f571009d770 msgr2=0x7f57100d11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:28.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f56fcff9640 1 -- 172.21.15.16:0/84060489 shutdown_connections 2024-09-19T07:43:28.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f56fcff9640 1 -- 172.21.15.16:0/84060489 wait complete. 2024-09-19T07:43:28.667 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-09-19T07:43:28.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f406ffff640 1 -- 172.21.15.16:0/915321831 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f405c06c330 msgr2=0x7f405c06e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f406ffff640 1 --2- 172.21.15.16:0/915321831 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f405c06c330 0x7f405c06e7f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f407006b500 tx=0x7f407008e040 comp rx=0 tx=0).stop 2024-09-19T07:43:28.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f406ffff640 1 -- 172.21.15.16:0/915321831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4058004fd0 msgr2=0x7f40680dd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f406ffff640 1 --2- 172.21.15.16:0/915321831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4058004fd0 0x7f40680dd910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f406000e930 tx=0x7f406000ed30 comp rx=0 tx=0).stop 2024-09-19T07:43:28.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f406ffff640 1 -- 172.21.15.16:0/915321831 shutdown_connections 2024-09-19T07:43:28.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f406ffff640 1 --2- 172.21.15.16:0/915321831 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f405c06c330 0x7f405c06e7f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f406ffff640 1 --2- 172.21.15.16:0/915321831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4068049540 0x7f40680499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f406ffff640 1 --2- 172.21.15.16:0/915321831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f406804b8f0 0x7f406804bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f406ffff640 1 --2- 172.21.15.16:0/915321831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4058004fd0 0x7f40680dd910 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.665+0000 7f406ffff640 1 -- 172.21.15.16:0/915321831 >> 172.21.15.16:0/915321831 conn(0x7f4068036090 msgr2=0x7f40680355d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:28.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.666+0000 7f406ffff640 1 -- 172.21.15.16:0/915321831 shutdown_connections 2024-09-19T07:43:28.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.666+0000 7f406ffff640 1 -- 172.21.15.16:0/915321831 wait complete. 2024-09-19T07:43:28.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.666+0000 7f2a1264a640 1 -- 172.21.15.16:0/179194477 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f29f006c2b0 msgr2=0x7f29f006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.666+0000 7f2a1264a640 1 --2- 172.21.15.16:0/179194477 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f29f006c2b0 0x7f29f006e770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a08034000 tx=0x7f2a0802e000 comp rx=0 tx=0).stop 2024-09-19T07:43:28.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.666+0000 7f2a1264a640 1 -- 172.21.15.16:0/179194477 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a0c15cc70 msgr2=0x7f2a0c158de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.666+0000 7f2a1264a640 1 --2- 172.21.15.16:0/179194477 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a0c15cc70 0x7f2a0c158de0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f29fc00b410 tx=0x7f29fc00b8d0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.666+0000 7f2a1264a640 1 -- 172.21.15.16:0/179194477 shutdown_connections 2024-09-19T07:43:28.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.666+0000 7f2a1264a640 1 --2- 172.21.15.16:0/179194477 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f29f006c2b0 0x7f29f006e770 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a08034000 tx=0x7f2a0802e000 comp rx=0 tx=0).stop 2024-09-19T07:43:28.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.666+0000 7f2a1264a640 1 --2- 172.21.15.16:0/179194477 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a0c15cc70 0x7f2a0c158de0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.666+0000 7f2a1264a640 1 --2- 172.21.15.16:0/179194477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c0b5500 0x7f2a0c161120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.666+0000 7f2a1264a640 1 --2- 172.21.15.16:0/179194477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a0c0b37c0 0x7f2a0c15bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.666+0000 7f2a1264a640 1 -- 172.21.15.16:0/179194477 >> 172.21.15.16:0/179194477 conn(0x7f2a0c0b1dc0 msgr2=0x7f2a0c165730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.667+0000 7f2a1264a640 1 -- 172.21.15.16:0/179194477 shutdown_connections 2024-09-19T07:43:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.667+0000 7f2a1264a640 1 -- 172.21.15.16:0/179194477 wait complete. 2024-09-19T07:43:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.668+0000 7f27a5b31640 1 -- 172.21.15.16:0/532676006 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f278c06c3b0 msgr2=0x7f278c06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.668+0000 7f27a5b31640 1 --2- 172.21.15.16:0/532676006 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f278c06c3b0 0x7f278c06e870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f27a00b65d0 tx=0x7f2798007660 comp rx=0 tx=0).stop 2024-09-19T07:43:28.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.669+0000 7f27a5b31640 1 -- 172.21.15.16:0/532676006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a0157da0 msgr2=0x7f27a0200e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.669+0000 7f27a5b31640 1 --2- 172.21.15.16:0/532676006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a0157da0 0x7f27a0200e20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f27a015fe50 tx=0x7f27a00b2ca0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.669+0000 7f27a5b31640 1 -- 172.21.15.16:0/532676006 shutdown_connections 2024-09-19T07:43:28.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.669+0000 7f27a5b31640 1 --2- 172.21.15.16:0/532676006 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f278c06c3b0 0x7f278c06e870 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.669+0000 7f27a5b31640 1 --2- 172.21.15.16:0/532676006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a0201360 0x7f27a0201810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.669+0000 7f27a5b31640 1 --2- 172.21.15.16:0/532676006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a0157da0 0x7f27a0200e20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.669+0000 7fe6dca26640 1 -- 172.21.15.16:0/3235775578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe6d0053ff0 con 0x7fe6d0005a60 2024-09-19T07:43:28.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.669+0000 7f27a5b31640 1 --2- 172.21.15.16:0/532676006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2788004800 0x7f27a0150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.669+0000 7f27a5b31640 1 -- 172.21.15.16:0/532676006 >> 172.21.15.16:0/532676006 conn(0x7f27a009a9f0 msgr2=0x7f27a015f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:28.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.669+0000 7f27a5b31640 1 -- 172.21.15.16:0/532676006 shutdown_connections 2024-09-19T07:43:28.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.669+0000 7f27a5b31640 1 -- 172.21.15.16:0/532676006 wait complete. 2024-09-19T07:43:28.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.669+0000 7fe6bf7fe640 1 -- 172.21.15.16:0/3235775578 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe6c805dcd0 con 0x7fe6d0005a60 2024-09-19T07:43:28.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.670+0000 7fbdb2e5e640 1 -- 172.21.15.16:0/1709272706 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fbdac0d4300 con 0x7fbdac15e490 2024-09-19T07:43:28.673 INFO:teuthology.orchestra.run.smithi016.stdout:60129542151 2024-09-19T07:43:28.673 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2024-09-19T07:43:28.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.670+0000 7fbda97fa640 1 -- 172.21.15.16:0/1709272706 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbda404b070 con 0x7fbdac15e490 2024-09-19T07:43:28.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.673+0000 7efcbffff640 1 -- 172.21.15.16:0/1549458071 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7efcdc06f760 con 0x7efcdc0ae740 2024-09-19T07:43:28.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.674+0000 7fe6dca26640 1 -- 172.21.15.16:0/3235775578 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe6c406c3b0 msgr2=0x7fe6c406e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.674+0000 7fe6dca26640 1 --2- 172.21.15.16:0/3235775578 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe6c406c3b0 0x7fe6c406e870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0008020 tx=0x7fe6d80aa000 comp rx=0 tx=0).stop 2024-09-19T07:43:28.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.674+0000 7fe6dca26640 1 -- 172.21.15.16:0/3235775578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005a60 msgr2=0x7fe6d017d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.674+0000 7fe6dca26640 1 --2- 172.21.15.16:0/3235775578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005a60 0x7fe6d017d200 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800e8e0 tx=0x7fe6c800eda0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.674+0000 7fe6dca26640 1 -- 172.21.15.16:0/3235775578 shutdown_connections 2024-09-19T07:43:28.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.674+0000 7fe6dca26640 1 --2- 172.21.15.16:0/3235775578 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe6c406c3b0 0x7fe6c406e870 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.674+0000 7fe6dca26640 1 --2- 172.21.15.16:0/3235775578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6d00e7120 0x7fe6d00e98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.674+0000 7fe6dca26640 1 --2- 172.21.15.16:0/3235775578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6d004eb60 0x7fe6d00e4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.674+0000 7fe6dca26640 1 --2- 172.21.15.16:0/3235775578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005a60 0x7fe6d017d200 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.674+0000 7fe6dca26640 1 -- 172.21.15.16:0/3235775578 >> 172.21.15.16:0/3235775578 conn(0x7fe6d0036090 msgr2=0x7fe6d0031190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:28.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.674+0000 7fe6dca26640 1 -- 172.21.15.16:0/3235775578 shutdown_connections 2024-09-19T07:43:28.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.674+0000 7fe6dca26640 1 -- 172.21.15.16:0/3235775578 wait complete. 2024-09-19T07:43:28.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.674+0000 7efcd27fc640 1 -- 172.21.15.16:0/1549458071 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efccc05d7f0 con 0x7efcdc0ae740 2024-09-19T07:43:28.676 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-09-19T07:43:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.674+0000 7fbdb2e5e640 1 -- 172.21.15.16:0/1709272706 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbd9006c4b0 msgr2=0x7fbd9006e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.674+0000 7fbdb2e5e640 1 --2- 172.21.15.16:0/1709272706 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbd9006c4b0 0x7fbd9006e970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbda00099b0 tx=0x7fbda0009110 comp rx=0 tx=0).stop 2024-09-19T07:43:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.675+0000 7fbdb2e5e640 1 -- 172.21.15.16:0/1709272706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdac15e490 msgr2=0x7fbdac0cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.675+0000 7fbdb2e5e640 1 --2- 172.21.15.16:0/1709272706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdac15e490 0x7fbdac0cc230 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbdac0d4f10 tx=0x7fbda4004a40 comp rx=0 tx=0).stop 2024-09-19T07:43:28.678 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.4 2024-09-19T07:43:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.675+0000 7fbdb2e5e640 1 -- 172.21.15.16:0/1709272706 shutdown_connections 2024-09-19T07:43:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.675+0000 7fbdb2e5e640 1 --2- 172.21.15.16:0/1709272706 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbd9006c4b0 0x7fbd9006e970 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.675+0000 7fbdb2e5e640 1 --2- 172.21.15.16:0/1709272706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdac15e490 0x7fbdac0cc230 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.675+0000 7fbdb2e5e640 1 --2- 172.21.15.16:0/1709272706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdac0d3ec0 0x7fbdac0cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.675+0000 7fbdb2e5e640 1 --2- 172.21.15.16:0/1709272706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdac0d2290 0x7fbdac0cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.675+0000 7fbdb2e5e640 1 -- 172.21.15.16:0/1709272706 >> 172.21.15.16:0/1709272706 conn(0x7fbdac08bf80 msgr2=0x7fbdac0d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:28.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.675+0000 7fbdb2e5e640 1 -- 172.21.15.16:0/1709272706 shutdown_connections 2024-09-19T07:43:28.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.675+0000 7fbdb2e5e640 1 -- 172.21.15.16:0/1709272706 wait complete. 2024-09-19T07:43:28.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.677+0000 7efcbffff640 1 -- 172.21.15.16:0/1549458071 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efcc006c2b0 msgr2=0x7efcc006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.677+0000 7efcbffff640 1 --2- 172.21.15.16:0/1549458071 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efcc006c2b0 0x7efcc006e770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efcd80017a0 tx=0x7efcd8009040 comp rx=0 tx=0).stop 2024-09-19T07:43:28.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.677+0000 7efcbffff640 1 -- 172.21.15.16:0/1549458071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcdc0ae740 msgr2=0x7efcdc157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:28.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.677+0000 7efcbffff640 1 --2- 172.21.15.16:0/1549458071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcdc0ae740 0x7efcdc157680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efccc00e9e0 tx=0x7efccc00eea0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.677+0000 7efcbffff640 1 -- 172.21.15.16:0/1549458071 shutdown_connections 2024-09-19T07:43:28.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.677+0000 7efcbffff640 1 --2- 172.21.15.16:0/1549458071 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efcc006c2b0 0x7efcc006e770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.677+0000 7efcbffff640 1 --2- 172.21.15.16:0/1549458071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcdc15e160 0x7efcdc0b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.677+0000 7efcbffff640 1 --2- 172.21.15.16:0/1549458071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcdc0ae740 0x7efcdc157680 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.677+0000 7efcbffff640 1 --2- 172.21.15.16:0/1549458071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efcdc06bc10 0x7efcdc145190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:28.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.677+0000 7efcbffff640 1 -- 172.21.15.16:0/1549458071 >> 172.21.15.16:0/1549458071 conn(0x7efcdc0d0890 msgr2=0x7efcdc1588c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:28.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.677+0000 7efcbffff640 1 -- 172.21.15.16:0/1549458071 shutdown_connections 2024-09-19T07:43:28.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:28.677+0000 7efcbffff640 1 -- 172.21.15.16:0/1549458071 wait complete. 2024-09-19T07:43:28.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.7 2024-09-19T07:43:28.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.0 2024-09-19T07:43:28.692 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.3 2024-09-19T07:43:28.693 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.5 2024-09-19T07:43:28.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.1 2024-09-19T07:43:28.699 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.2 2024-09-19T07:43:28.699 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.6 2024-09-19T07:43:29.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-19T07:43:29.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-19T07:43:29.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-19T07:43:29.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-19T07:43:29.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-19T07:43:29.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-19T07:43:29.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-19T07:43:29.700 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-19T07:43:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.847+0000 7efc9bfe4640 1 Processor -- start 2024-09-19T07:43:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.847+0000 7efc9bfe4640 1 -- start start 2024-09-19T07:43:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.848+0000 7efc9bfe4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efc940cd2d0 0x7efc941638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.848+0000 7efc9bfe4640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efc94073960 0x7efc94161f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.848+0000 7efc9bfe4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc94163de0 con 0x7efc940b2aa0 2024-09-19T07:43:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.848+0000 7efc9bfe4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7efc94098ba0 con 0x7efc940cd2d0 2024-09-19T07:43:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.848+0000 7efc9bfe4640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7efc94098e40 con 0x7efc94073960 2024-09-19T07:43:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.849+0000 7efc9957f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efc940b2aa0 0x7efc94073250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44916/0 (socket says 172.21.15.16:44916) 2024-09-19T07:43:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.849+0000 7efc9957f640 1 -- 172.21.15.16:0/1865378645 learned_addr learned my addr 172.21.15.16:0/1865378645 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.849+0000 7efc99d80640 1 --2- 172.21.15.16:0/1865378645 >> v2:172.21.15.16:3300/0 conn(0x7efc940cd2d0 0x7efc941638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.850+0000 7efc99d80640 1 -- 172.21.15.16:0/1865378645 >> v2:172.21.15.146:3300/0 conn(0x7efc94073960 msgr2=0x7efc94161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.850+0000 7efc99d80640 1 --2- 172.21.15.16:0/1865378645 >> v2:172.21.15.146:3300/0 conn(0x7efc94073960 0x7efc94161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.850+0000 7efc99d80640 1 -- 172.21.15.16:0/1865378645 >> v1:172.21.15.16:6789/0 conn(0x7efc940b2aa0 legacy=0x7efc94073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.850+0000 7efc99d80640 1 -- 172.21.15.16:0/1865378645 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc94099120 con 0x7efc940cd2d0 2024-09-19T07:43:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.851+0000 7efc99d80640 1 --2- 172.21.15.16:0/1865378645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc940cd2d0 0x7efc941638a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efc940ca570 tx=0x7efc8402a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef49cb35ee8fe0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.853+0000 7efc98d7e640 1 -- 172.21.15.16:0/1865378645 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84037070 con 0x7efc940cd2d0 2024-09-19T07:43:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.853+0000 7efc98d7e640 1 -- 172.21.15.16:0/1865378645 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc8402ad60 con 0x7efc940cd2d0 2024-09-19T07:43:29.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.854+0000 7efc9bfe4640 1 -- 172.21.15.16:0/1865378645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc940cd2d0 msgr2=0x7efc941638a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.854+0000 7efc9bfe4640 1 --2- 172.21.15.16:0/1865378645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc940cd2d0 0x7efc941638a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efc940ca570 tx=0x7efc8402a0d0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.854+0000 7efc9bfe4640 1 -- 172.21.15.16:0/1865378645 shutdown_connections 2024-09-19T07:43:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.854+0000 7efc9bfe4640 1 --2- 172.21.15.16:0/1865378645 >> v2:172.21.15.146:3300/0 conn(0x7efc94073960 0x7efc94161f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.854+0000 7efc9bfe4640 1 --2- 172.21.15.16:0/1865378645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc940cd2d0 0x7efc941638a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.854+0000 7efc9bfe4640 1 -- 172.21.15.16:0/1865378645 >> 172.21.15.16:0/1865378645 conn(0x7efc940b1e80 msgr2=0x7efc940b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.855+0000 7efc9bfe4640 1 -- 172.21.15.16:0/1865378645 shutdown_connections 2024-09-19T07:43:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.855+0000 7efc9bfe4640 1 -- 172.21.15.16:0/1865378645 wait complete. 2024-09-19T07:43:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.857+0000 7efc9bfe4640 1 Processor -- start 2024-09-19T07:43:29.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.857+0000 7efc9bfe4640 1 -- start start 2024-09-19T07:43:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.859+0000 7efc9bfe4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc94073960 0x7efc940b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.859+0000 7efc9bfe4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc940b2aa0 0x7efc9414fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.859+0000 7efc9bfe4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc940cd2d0 0x7efc940afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.859+0000 7efc9bfe4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc940999c0 con 0x7efc94073960 2024-09-19T07:43:29.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.859+0000 7efc9bfe4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efc940996e0 con 0x7efc940b2aa0 2024-09-19T07:43:29.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.859+0000 7efc9bfe4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efc94162480 con 0x7efc940cd2d0 2024-09-19T07:43:29.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.859+0000 7efc9957f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc940b2aa0 0x7efc9414fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.859+0000 7efc9957f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc940b2aa0 0x7efc9414fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:58004/0 (socket says 172.21.15.16:58004) 2024-09-19T07:43:29.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.859+0000 7efc9957f640 1 -- 172.21.15.16:0/3729920700 learned_addr learned my addr 172.21.15.16:0/3729920700 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:29.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.859+0000 7efc9a581640 1 --2- 172.21.15.16:0/3729920700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc940cd2d0 0x7efc940afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.860+0000 7efc9957f640 1 -- 172.21.15.16:0/3729920700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc940cd2d0 msgr2=0x7efc940afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.860+0000 7efc9957f640 1 --2- 172.21.15.16:0/3729920700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc940cd2d0 0x7efc940afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.860+0000 7efc9957f640 1 -- 172.21.15.16:0/3729920700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc94073960 msgr2=0x7efc940b4760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:43:29.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.860+0000 7efc9957f640 1 --2- 172.21.15.16:0/3729920700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc94073960 0x7efc940b4760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.860+0000 7efc9957f640 1 -- 172.21.15.16:0/3729920700 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9414ffd0 con 0x7efc940b2aa0 2024-09-19T07:43:29.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.860+0000 7efc9957f640 1 --2- 172.21.15.16:0/3729920700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc940b2aa0 0x7efc9414fa90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efc9000c810 tx=0x7efc9000ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.861+0000 7efc8a7fc640 1 -- 172.21.15.16:0/3729920700 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90018070 con 0x7efc940b2aa0 2024-09-19T07:43:29.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.861+0000 7efc9bfe4640 1 -- 172.21.15.16:0/3729920700 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc940b3660 con 0x7efc940b2aa0 2024-09-19T07:43:29.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.861+0000 7efc9bfe4640 1 -- 172.21.15.16:0/3729920700 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc940b3c20 con 0x7efc940b2aa0 2024-09-19T07:43:29.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.862+0000 7efc8a7fc640 1 -- 172.21.15.16:0/3729920700 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc900074e0 con 0x7efc940b2aa0 2024-09-19T07:43:29.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.863+0000 7efc8a7fc640 1 -- 172.21.15.16:0/3729920700 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc900049e0 con 0x7efc940b2aa0 2024-09-19T07:43:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.863+0000 7efc8a7fc640 1 -- 172.21.15.16:0/3729920700 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7efc9001b070 con 0x7efc940b2aa0 2024-09-19T07:43:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.864+0000 7efc8a7fc640 1 --2- 172.21.15.16:0/3729920700 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efc7806c380 0x7efc7806e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.864+0000 7efc8a7fc640 1 -- 172.21.15.16:0/3729920700 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7efc9008acb0 con 0x7efc940b2aa0 2024-09-19T07:43:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.865+0000 7efc9bfe4640 1 -- 172.21.15.16:0/3729920700 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc94161f40 con 0x7efc940b2aa0 2024-09-19T07:43:29.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.865+0000 7efc99d80640 1 --2- 172.21.15.16:0/3729920700 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efc7806c380 0x7efc7806e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.867+0000 7efc99d80640 1 --2- 172.21.15.16:0/3729920700 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efc7806c380 0x7efc7806e840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efc84034000 tx=0x7efc8402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.870+0000 7efc8a7fc640 1 -- 172.21.15.16:0/3729920700 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7efc94163de0 con 0x7efc940b2aa0 2024-09-19T07:43:29.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.906+0000 7f3270d91640 1 Processor -- start 2024-09-19T07:43:29.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.906+0000 7f58fa6af640 1 Processor -- start 2024-09-19T07:43:29.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.906+0000 7f58fa6af640 1 -- start start 2024-09-19T07:43:29.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.906+0000 7f3270d91640 1 -- start start 2024-09-19T07:43:29.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.907+0000 7f95be1fd640 1 Processor -- start 2024-09-19T07:43:29.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.907+0000 7fbff4d8d640 1 Processor -- start 2024-09-19T07:43:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.907+0000 7f3270d91640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3264033a40 0x7f3264009b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.908+0000 7f3270d91640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32640eb3a0 0x7f326404eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.908+0000 7f3270d91640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32640071e0 0x7f326404f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.908+0000 7f3270d91640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f326401cf20 con 0x7f32640071e0 2024-09-19T07:43:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.908+0000 7f3270d91640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f326401c5f0 con 0x7f3264033a40 2024-09-19T07:43:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.908+0000 7f3270d91640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f326401c8d0 con 0x7f32640eb3a0 2024-09-19T07:43:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.908+0000 7f58fa6af640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58f406f3b0 0x7f58f415b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.908+0000 7f66d67da640 1 Processor -- start 2024-09-19T07:43:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.908+0000 7f66d67da640 1 -- start start 2024-09-19T07:43:29.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.908+0000 7f58fa6af640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58f4140410 0x7f58f415bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.908+0000 7f3269d9b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32640eb3a0 0x7f326404eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.908+0000 7f58fa6af640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58f415c240 con 0x7f58f4140030 2024-09-19T07:43:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.908+0000 7f3269d9b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32640eb3a0 0x7f326404eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:34310/0 (socket says 172.21.15.16:34310) 2024-09-19T07:43:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.908+0000 7f3269d9b640 1 -- 172.21.15.16:0/1560087464 learned_addr learned my addr 172.21.15.16:0/1560087464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f58fa6af640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f58f4140920 con 0x7f58f406f3b0 2024-09-19T07:43:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f326a59c640 1 --2- 172.21.15.16:0/1560087464 >> v2:172.21.15.146:3300/0 conn(0x7f3264033a40 0x7f3264009b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f58fa6af640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f58f4140c00 con 0x7f58f4140410 2024-09-19T07:43:29.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f3269d9b640 1 -- 172.21.15.16:0/1560087464 >> v2:172.21.15.146:3300/0 conn(0x7f3264033a40 msgr2=0x7f3264009b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:29.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.908+0000 7f95be1fd640 1 -- start start 2024-09-19T07:43:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.907+0000 7fbff4d8d640 1 -- start start 2024-09-19T07:43:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7fbff4d8d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbff00b5b30 0x7fbff015a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7fbff4d8d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbff00cdce0 0x7fbff00b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7fbff4d8d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbff00ce470 con 0x7fbff0154680 2024-09-19T07:43:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7fbff4d8d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fbff00ce750 con 0x7fbff00b5b30 2024-09-19T07:43:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7fbff4d8d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbff00cea30 con 0x7fbff00cdce0 2024-09-19T07:43:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f58f96ad640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f58f4140030 0x7f58f406eca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:56474/0 (socket says 172.21.15.16:56474) 2024-09-19T07:43:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f58f96ad640 1 -- 172.21.15.16:0/2929482549 learned_addr learned my addr 172.21.15.16:0/2929482549 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f95be1fd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f95b815ec70 0x7f95b80cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f95be1fd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95b815b320 con 0x7f95b80b29a0 2024-09-19T07:43:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f95be1fd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95b815b600 con 0x7f95b8158220 2024-09-19T07:43:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f95be1fd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f95b80c5670 con 0x7f95b815ec70 2024-09-19T07:43:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f3269d9b640 1 --2- 172.21.15.16:0/1560087464 >> v2:172.21.15.146:3300/0 conn(0x7f3264033a40 0x7f3264009b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7fbfef7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbff0154680 0x7fbff00b5420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44928/0 (socket says 172.21.15.16:44928) 2024-09-19T07:43:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7fbfef7fe640 1 -- 172.21.15.16:0/447750413 learned_addr learned my addr 172.21.15.16:0/447750413 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f3269d9b640 1 -- 172.21.15.16:0/1560087464 >> v2:172.21.15.16:3300/0 conn(0x7f32640071e0 msgr2=0x7f326404f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f95b77fe640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f95b8158220 0x7f95b815aa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:56480/0 (socket says 172.21.15.16:56480) 2024-09-19T07:43:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f95b77fe640 1 -- 172.21.15.16:0/860117336 learned_addr learned my addr 172.21.15.16:0/860117336 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f66d67da640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f66d0158860 0x7f66d015b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f66d67da640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f66d0158c40 0x7f66d0070110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7fbfeffff640 1 --2- 172.21.15.16:0/447750413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbff00cdce0 0x7fbff00b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f66d67da640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66d00cfa30 0x7f66d0070650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f66d67da640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f66d00cff80 con 0x7f66d0158c40 2024-09-19T07:43:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f66d67da640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f66d00d0260 con 0x7f66d0158860 2024-09-19T07:43:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f66d67da640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f66d00d0540 con 0x7f66d00cfa30 2024-09-19T07:43:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7f95b6ffd640 1 --2- 172.21.15.16:0/860117336 >> v2:172.21.15.146:3300/0 conn(0x7f95b815ec70 0x7f95b80cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7f326ad9d640 1 --2- 172.21.15.16:0/1560087464 >> v2:172.21.15.16:3300/0 conn(0x7f32640071e0 0x7f326404f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7f3269d9b640 1 --2- 172.21.15.16:0/1560087464 >> v2:172.21.15.16:3300/0 conn(0x7f32640071e0 0x7f326404f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7f3269d9b640 1 -- 172.21.15.16:0/1560087464 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326401cbb0 con 0x7f32640eb3a0 2024-09-19T07:43:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7f58ebfff640 1 -- 172.21.15.16:0/2929482549 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 508606307 0 0) 0x7f58f415c240 con 0x7f58f4140030 2024-09-19T07:43:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7f58ebfff640 1 -- 172.21.15.16:0/2929482549 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58e0003520 con 0x7f58f4140030 2024-09-19T07:43:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7f66d5fd9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66d00cfa30 0x7f66d0070650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7f66d5fd9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66d00cfa30 0x7f66d0070650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:34338/0 (socket says 172.21.15.16:34338) 2024-09-19T07:43:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7f66d5fd9640 1 -- 172.21.15.16:0/3603812009 learned_addr learned my addr 172.21.15.16:0/3603812009 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7f66d57d8640 1 --2- 172.21.15.16:0/3603812009 >> v2:172.21.15.146:3300/0 conn(0x7f66d0158860 0x7f66d015b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7f66d4fd7640 1 --2- 172.21.15.16:0/3603812009 >> v2:172.21.15.16:3300/0 conn(0x7f66d0158c40 0x7f66d0070110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7f58ebfff640 1 -- 172.21.15.16:0/2929482549 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4076826823 0 0) 0x7f58e0003520 con 0x7f58f4140030 2024-09-19T07:43:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7f58ebfff640 1 -- 172.21.15.16:0/2929482549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58f4140410 msgr2=0x7f58f415bd00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:43:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7f58ebfff640 1 --2- 172.21.15.16:0/2929482549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58f4140410 0x7f58f415bd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f58ebfff640 1 -- 172.21.15.16:0/2929482549 >> v2:172.21.15.16:3300/0 conn(0x7f58f406f3b0 msgr2=0x7f58f415b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f58ebfff640 1 --2- 172.21.15.16:0/2929482549 >> v2:172.21.15.16:3300/0 conn(0x7f58f406f3b0 0x7f58f415b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f58ebfff640 1 -- 172.21.15.16:0/2929482549 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f4140ee0 con 0x7f58f4140030 2024-09-19T07:43:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7f66d5fd9640 1 -- 172.21.15.16:0/3603812009 >> v2:172.21.15.146:3300/0 conn(0x7f66d0158860 msgr2=0x7f66d015b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7f66d5fd9640 1 --2- 172.21.15.16:0/3603812009 >> v2:172.21.15.146:3300/0 conn(0x7f66d0158860 0x7f66d015b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7f66d5fd9640 1 -- 172.21.15.16:0/3603812009 >> v2:172.21.15.16:3300/0 conn(0x7f66d0158c40 msgr2=0x7f66d0070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7f66d5fd9640 1 --2- 172.21.15.16:0/3603812009 >> v2:172.21.15.16:3300/0 conn(0x7f66d0158c40 0x7f66d0070110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7f66d5fd9640 1 -- 172.21.15.16:0/3603812009 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d00d0820 con 0x7f66d00cfa30 2024-09-19T07:43:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7f66d57d8640 1 --2- 172.21.15.16:0/3603812009 >> v2:172.21.15.146:3300/0 conn(0x7f66d0158860 0x7f66d015b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f326ad9d640 1 --2- 172.21.15.16:0/1560087464 >> v2:172.21.15.16:3300/0 conn(0x7f32640071e0 0x7f326404f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f58ebfff640 1 -- 172.21.15.16:0/2929482549 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f58e4001670 con 0x7f58f4140030 2024-09-19T07:43:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7fbfeffff640 1 -- 172.21.15.16:0/447750413 >> v2:172.21.15.16:3300/0 conn(0x7fbff00b5b30 msgr2=0x7fbff015a700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:43:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7fbfeffff640 1 --2- 172.21.15.16:0/447750413 >> v2:172.21.15.16:3300/0 conn(0x7fbff00b5b30 0x7fbff015a700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7fbfeffff640 1 -- 172.21.15.16:0/447750413 >> v1:172.21.15.16:6789/0 conn(0x7fbff0154680 legacy=0x7fbff00b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7fbfeffff640 1 -- 172.21.15.16:0/447750413 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff00ced10 con 0x7fbff00cdce0 2024-09-19T07:43:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f3269d9b640 1 --2- 172.21.15.16:0/1560087464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32640eb3a0 0x7f326404eb90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f326c02dd50 tx=0x7f326c0a60a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac4721b5b2d25cde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f58ebfff640 1 -- 172.21.15.16:0/2929482549 >> v1:172.21.15.146:6789/0 conn(0x7f58f4140030 legacy=0x7f58f406eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f58ebfff640 1 --2- 172.21.15.16:0/2929482549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58e00039b0 0x7f58e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f58ebfff640 1 --2- 172.21.15.16:0/2929482549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58e0004800 0x7f58e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f58ebfff640 1 --2- 172.21.15.16:0/2929482549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58f4140410 0x7f58e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f58ebfff640 1 -- 172.21.15.16:0/2929482549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58f4140920 con 0x7f58e00039b0 2024-09-19T07:43:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f58ebfff640 1 -- 172.21.15.16:0/2929482549 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f58f4140c00 con 0x7f58e0004800 2024-09-19T07:43:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f58ebfff640 1 -- 172.21.15.16:0/2929482549 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f58e00055a0 con 0x7f58f4140410 2024-09-19T07:43:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7fbfeffff640 1 --2- 172.21.15.16:0/447750413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbff00cdce0 0x7fbff00b46a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000a940 tx=0x7fbfe002c690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62c18d6c97531468 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f326959a640 1 -- 172.21.15.16:0/1560087464 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c0b3070 con 0x7f32640eb3a0 2024-09-19T07:43:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f326959a640 1 -- 172.21.15.16:0/1560087464 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f326c0ae440 con 0x7f32640eb3a0 2024-09-19T07:43:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f326959a640 1 -- 172.21.15.16:0/1560087464 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c0ae8b0 con 0x7f32640eb3a0 2024-09-19T07:43:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f95b6ffd640 1 -- 172.21.15.16:0/860117336 >> v1:172.21.15.146:6789/0 conn(0x7f95b8158220 legacy=0x7f95b815aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f95b6ffd640 1 -- 172.21.15.16:0/860117336 >> v1:172.21.15.16:6789/0 conn(0x7f95b80b29a0 legacy=0x7f95b80c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f58f8eac640 1 --2- 172.21.15.16:0/2929482549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58e0004800 0x7f58e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f58f9eae640 1 --2- 172.21.15.16:0/2929482549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58f4140410 0x7f58e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.908+0000 7fc3c29c5640 1 Processor -- start 2024-09-19T07:43:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.909+0000 7fc3c29c5640 1 -- start start 2024-09-19T07:43:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7fc3c29c5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3b4036020 0x7fc3b4009f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7fc3c29c5640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc3b400b770 0x7fc3b4036400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7fc3c29c5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3b400bc20 con 0x7fc3b404eb60 2024-09-19T07:43:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7fc3c29c5640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc3b4007480 con 0x7fc3b400b770 2024-09-19T07:43:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.910+0000 7fc3c29c5640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b40063e0 con 0x7fc3b4036020 2024-09-19T07:43:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7fc3c11c2640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc3b404eb60 0x7fc3b4054cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44942/0 (socket says 172.21.15.16:44942) 2024-09-19T07:43:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7fc3c11c2640 1 -- 172.21.15.16:0/1929434346 learned_addr learned my addr 172.21.15.16:0/1929434346 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7fc3c21c4640 1 --2- 172.21.15.16:0/1929434346 >> v2:172.21.15.146:3300/0 conn(0x7fc3b400b770 0x7fc3b4036400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.911+0000 7f95b6ffd640 1 -- 172.21.15.16:0/860117336 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b815b8e0 con 0x7f95b815ec70 2024-09-19T07:43:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.912+0000 7f58f96ad640 1 --2- 172.21.15.16:0/2929482549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58e00039b0 0x7f58e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.912+0000 7f58f8eac640 1 -- 172.21.15.16:0/2929482549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58f4140410 msgr2=0x7f58e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:29.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.912+0000 7f58f8eac640 1 --2- 172.21.15.16:0/2929482549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58f4140410 0x7f58e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.912+0000 7f58f8eac640 1 -- 172.21.15.16:0/2929482549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58e00039b0 msgr2=0x7f58e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:29.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.912+0000 7f58f8eac640 1 --2- 172.21.15.16:0/2929482549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58e00039b0 0x7f58e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.912+0000 7f58f8eac640 1 -- 172.21.15.16:0/2929482549 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e0003520 con 0x7f58e0004800 2024-09-19T07:43:29.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.912+0000 7f58f9eae640 1 --2- 172.21.15.16:0/2929482549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58f4140410 0x7f58e000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:29.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.912+0000 7fbfee7fc640 1 -- 172.21.15.16:0/447750413 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe000b040 con 0x7fbff00cdce0 2024-09-19T07:43:29.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.912+0000 7fbfee7fc640 1 -- 172.21.15.16:0/447750413 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfe0031710 con 0x7fbff00cdce0 2024-09-19T07:43:29.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.912+0000 7f3270d91640 1 -- 172.21.15.16:0/1560087464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32640eb3a0 msgr2=0x7f326404eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.912+0000 7f3270d91640 1 --2- 172.21.15.16:0/1560087464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32640eb3a0 0x7f326404eb90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f326c02dd50 tx=0x7f326c0a60a0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.912+0000 7fbfee7fc640 1 -- 172.21.15.16:0/447750413 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0031b20 con 0x7fbff00cdce0 2024-09-19T07:43:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7f3270d91640 1 -- 172.21.15.16:0/1560087464 shutdown_connections 2024-09-19T07:43:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7f3270d91640 1 --2- 172.21.15.16:0/1560087464 >> v2:172.21.15.16:3300/0 conn(0x7f32640071e0 0x7f326404f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7f3270d91640 1 --2- 172.21.15.16:0/1560087464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32640eb3a0 0x7f326404eb90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7f3270d91640 1 --2- 172.21.15.16:0/1560087464 >> v2:172.21.15.146:3300/0 conn(0x7f3264033a40 0x7f3264009b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7f3270d91640 1 -- 172.21.15.16:0/1560087464 >> 172.21.15.16:0/1560087464 conn(0x7f32640360c0 msgr2=0x7f32640364d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7f66d5fd9640 1 --2- 172.21.15.16:0/3603812009 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66d00cfa30 0x7f66d0070650 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f66d00728d0 tx=0x7f66c002a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=539d5849afaccd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.912+0000 7fc3c19c3640 1 --2- 172.21.15.16:0/1929434346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3b4036020 0x7fc3b4009f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.912+0000 7fc3c21c4640 1 -- 172.21.15.16:0/1929434346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3b4036020 msgr2=0x7fc3b4009f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.912+0000 7fc3c21c4640 1 --2- 172.21.15.16:0/1929434346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3b4036020 0x7fc3b4009f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.912+0000 7fc3c21c4640 1 -- 172.21.15.16:0/1929434346 >> v1:172.21.15.16:6789/0 conn(0x7fc3b404eb60 legacy=0x7fc3b4054cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.912+0000 7fc3c21c4640 1 -- 172.21.15.16:0/1929434346 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b4006940 con 0x7fc3b400b770 2024-09-19T07:43:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.912+0000 7fc3c19c3640 1 --2- 172.21.15.16:0/1929434346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3b4036020 0x7fc3b4009f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7f95b6ffd640 1 --2- 172.21.15.16:0/860117336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95b815ec70 0x7f95b80cde40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009720 tx=0x7f95a802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14ce4cb594e363f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7fbff4d8d640 1 -- 172.21.15.16:0/447750413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbff00cdce0 msgr2=0x7fbff00b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7fbff4d8d640 1 --2- 172.21.15.16:0/447750413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbff00cdce0 0x7fbff00b46a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000a940 tx=0x7fbfe002c690 comp rx=0 tx=0).stop 2024-09-19T07:43:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7f3270d91640 1 -- 172.21.15.16:0/1560087464 shutdown_connections 2024-09-19T07:43:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7f66c7fff640 1 -- 172.21.15.16:0/3603812009 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0037070 con 0x7f66d00cfa30 2024-09-19T07:43:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7f3270d91640 1 -- 172.21.15.16:0/1560087464 wait complete. 2024-09-19T07:43:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7fbff4d8d640 1 -- 172.21.15.16:0/447750413 shutdown_connections 2024-09-19T07:43:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7fbff4d8d640 1 --2- 172.21.15.16:0/447750413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbff00cdce0 0x7fbff00b46a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7fbff4d8d640 1 --2- 172.21.15.16:0/447750413 >> v2:172.21.15.16:3300/0 conn(0x7fbff00b5b30 0x7fbff015a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7fbff4d8d640 1 -- 172.21.15.16:0/447750413 >> 172.21.15.16:0/447750413 conn(0x7fbff006dae0 msgr2=0x7fbff006dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:29.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7f66c7fff640 1 -- 172.21.15.16:0/3603812009 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66c0032440 con 0x7f66d00cfa30 2024-09-19T07:43:29.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7f95b67fc640 1 -- 172.21.15.16:0/860117336 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8037070 con 0x7f95b815ec70 2024-09-19T07:43:29.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7f95b67fc640 1 -- 172.21.15.16:0/860117336 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95a802ae10 con 0x7f95b815ec70 2024-09-19T07:43:29.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7f95b67fc640 1 -- 172.21.15.16:0/860117336 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8032660 con 0x7f95b815ec70 2024-09-19T07:43:29.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7fc3c21c4640 1 --2- 172.21.15.16:0/1929434346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3b400b770 0x7fc3b4036400 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00c050 tx=0x7fc3ac02e510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6895421f9bfcef20 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7fbff4d8d640 1 -- 172.21.15.16:0/447750413 shutdown_connections 2024-09-19T07:43:29.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7fc3c09c1640 1 -- 172.21.15.16:0/1929434346 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac03a070 con 0x7fc3b400b770 2024-09-19T07:43:29.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7fc3c09c1640 1 -- 172.21.15.16:0/1929434346 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3ac00ae50 con 0x7fc3b400b770 2024-09-19T07:43:29.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7fc3c09c1640 1 -- 172.21.15.16:0/1929434346 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac00a700 con 0x7fc3b400b770 2024-09-19T07:43:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7f58f8eac640 1 --2- 172.21.15.16:0/2929482549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58e0004800 0x7f58e0004c80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f58f000bb00 tx=0x7f58e0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e30d1f1958c9b1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7f9b2f4c8640 1 Processor -- start 2024-09-19T07:43:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7f95be1fd640 1 -- 172.21.15.16:0/860117336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95b815ec70 msgr2=0x7f95b80cde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7f95be1fd640 1 --2- 172.21.15.16:0/860117336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95b815ec70 0x7f95b80cde40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009720 tx=0x7f95a802a100 comp rx=0 tx=0).stop 2024-09-19T07:43:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7f9b2f4c8640 1 -- start start 2024-09-19T07:43:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7f66d67da640 1 -- 172.21.15.16:0/3603812009 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66d00cfa30 msgr2=0x7f66d0070650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7f66d67da640 1 --2- 172.21.15.16:0/3603812009 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66d00cfa30 0x7f66d0070650 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f66d00728d0 tx=0x7f66c002a1e0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7f58ebfff640 1 -- 172.21.15.16:0/2929482549 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f000b700 con 0x7f58e0004800 2024-09-19T07:43:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7f66d67da640 1 -- 172.21.15.16:0/3603812009 shutdown_connections 2024-09-19T07:43:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7f66d67da640 1 --2- 172.21.15.16:0/3603812009 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66d00cfa30 0x7f66d0070650 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7f66d67da640 1 --2- 172.21.15.16:0/3603812009 >> v2:172.21.15.16:3300/0 conn(0x7f66d0158c40 0x7f66d0070110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7f66d67da640 1 --2- 172.21.15.16:0/3603812009 >> v2:172.21.15.146:3300/0 conn(0x7f66d0158860 0x7f66d015b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7f66d67da640 1 -- 172.21.15.16:0/3603812009 >> 172.21.15.16:0/3603812009 conn(0x7f66d006ebb0 msgr2=0x7f66d006efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7f58ebfff640 1 -- 172.21.15.16:0/2929482549 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f0033620 con 0x7f58e0004800 2024-09-19T07:43:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7fc3c29c5640 1 -- 172.21.15.16:0/1929434346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3b400b770 msgr2=0x7fc3b4036400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7fc3c29c5640 1 --2- 172.21.15.16:0/1929434346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3b400b770 0x7fc3b4036400 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00c050 tx=0x7fc3ac02e510 comp rx=0 tx=0).stop 2024-09-19T07:43:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7fc3c29c5640 1 -- 172.21.15.16:0/1929434346 shutdown_connections 2024-09-19T07:43:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7fc3c29c5640 1 --2- 172.21.15.16:0/1929434346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3b400b770 0x7fc3b4036400 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7fc3c29c5640 1 --2- 172.21.15.16:0/1929434346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3b4036020 0x7fc3b4009f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.914+0000 7fc3c29c5640 1 -- 172.21.15.16:0/1929434346 >> 172.21.15.16:0/1929434346 conn(0x7fc3b401d930 msgr2=0x7fc3b40545a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.913+0000 7fbff4d8d640 1 -- 172.21.15.16:0/447750413 wait complete. 2024-09-19T07:43:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.915+0000 7f66d67da640 1 -- 172.21.15.16:0/3603812009 shutdown_connections 2024-09-19T07:43:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.915+0000 7f58ebfff640 1 -- 172.21.15.16:0/2929482549 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0033a50 con 0x7f58e0004800 2024-09-19T07:43:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.915+0000 7f3270d91640 1 Processor -- start 2024-09-19T07:43:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.915+0000 7f3270d91640 1 -- start start 2024-09-19T07:43:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.915+0000 7fbff4d8d640 1 Processor -- start 2024-09-19T07:43:29.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.915+0000 7fc3c29c5640 1 -- 172.21.15.16:0/1929434346 shutdown_connections 2024-09-19T07:43:29.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.915+0000 7fc3c29c5640 1 -- 172.21.15.16:0/1929434346 wait complete. 2024-09-19T07:43:29.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7f95be1fd640 1 -- 172.21.15.16:0/860117336 shutdown_connections 2024-09-19T07:43:29.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7f95be1fd640 1 --2- 172.21.15.16:0/860117336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95b815ec70 0x7f95b80cde40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7f95be1fd640 1 -- 172.21.15.16:0/860117336 >> 172.21.15.16:0/860117336 conn(0x7f95b80b1e80 msgr2=0x7f95b80b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7f58fa6af640 1 -- 172.21.15.16:0/2929482549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58e0004800 msgr2=0x7f58e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7f58fa6af640 1 --2- 172.21.15.16:0/2929482549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58e0004800 0x7f58e0004c80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f58f000bb00 tx=0x7f58e0005830 comp rx=0 tx=0).stop 2024-09-19T07:43:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7f58f9eae640 1 -- 172.21.15.16:0/2929482549 reap_dead start 2024-09-19T07:43:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7f9b2f4c8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b20005a60 0x7f9b20053f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7f9b2f4c8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b20052330 con 0x7f9b2004eb60 2024-09-19T07:43:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7f9b2f4c8640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b20006c80 con 0x7f9b200e7120 2024-09-19T07:43:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7f9b2f4c8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9b200520e0 con 0x7f9b20005a60 2024-09-19T07:43:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7f58fa6af640 1 -- 172.21.15.16:0/2929482549 shutdown_connections 2024-09-19T07:43:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7f58fa6af640 1 -- 172.21.15.16:0/2929482549 >> 172.21.15.16:0/2929482549 conn(0x7f58f4067b30 msgr2=0x7f58f406cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7f9b2ecc7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9b2004eb60 0x7f9b2001d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44954/0 (socket says 172.21.15.16:44954) 2024-09-19T07:43:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7f58fa6af640 1 -- 172.21.15.16:0/2929482549 shutdown_connections 2024-09-19T07:43:29.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7f3270d91640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32640071e0 0x7f32640defd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7f9b2ecc7640 1 -- 172.21.15.16:0/628099787 learned_addr learned my addr 172.21.15.16:0/628099787 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:29.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7f9b2d4c4640 1 -- 172.21.15.16:0/628099787 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1399617691 0 0) 0x7f9b20052330 con 0x7f9b2004eb60 2024-09-19T07:43:29.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7f9b2d4c4640 1 -- 172.21.15.16:0/628099787 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b14003520 con 0x7f9b2004eb60 2024-09-19T07:43:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.915+0000 7f66d67da640 1 -- 172.21.15.16:0/3603812009 wait complete. 2024-09-19T07:43:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7f66d67da640 1 Processor -- start 2024-09-19T07:43:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7f3270d91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3264033a40 0x7f3264038d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.915+0000 7fbff4d8d640 1 -- start start 2024-09-19T07:43:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7fbff4d8d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbff00b5b30 0x7fbff0159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7fbff4d8d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbff00cdce0 0x7fbff014e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7fbff4d8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff0154680 0x7fbff0157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7fbff4d8d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbff00ce150 con 0x7fbff0154680 2024-09-19T07:43:29.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7fbff4d8d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0073ae0 con 0x7fbff00cdce0 2024-09-19T07:43:29.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7fbff4d8d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbff0154a60 con 0x7fbff00b5b30 2024-09-19T07:43:29.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7f3270d91640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32640392d0 0x7f32640e6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7f9b2d4c4640 1 -- 172.21.15.16:0/628099787 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3422341606 0 0) 0x7f9b14003520 con 0x7f9b2004eb60 2024-09-19T07:43:29.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7fbfeeffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbff00cdce0 0x7fbff014e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7fbfeeffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbff00cdce0 0x7fbff014e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:58066/0 (socket says 172.21.15.16:58066) 2024-09-19T07:43:29.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7fbfeeffd640 1 -- 172.21.15.16:0/643810409 learned_addr learned my addr 172.21.15.16:0/643810409 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:29.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7fbfef7fe640 1 --2- 172.21.15.16:0/643810409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbff00b5b30 0x7fbff0159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7f9b2d4c4640 1 -- 172.21.15.16:0/628099787 >> v2:172.21.15.16:3300/0 conn(0x7f9b20005a60 msgr2=0x7f9b20053f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:29.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.916+0000 7f58fa6af640 1 -- 172.21.15.16:0/2929482549 wait complete. 2024-09-19T07:43:29.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7f9b2e4c6640 1 --2- 172.21.15.16:0/628099787 >> v2:172.21.15.16:3300/0 conn(0x7f9b20005a60 0x7f9b20053f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7f9b2d4c4640 1 --2- 172.21.15.16:0/628099787 >> v2:172.21.15.16:3300/0 conn(0x7f9b20005a60 0x7f9b20053f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7f9b2d4c4640 1 -- 172.21.15.16:0/628099787 >> v1:172.21.15.146:6789/0 conn(0x7f9b200e7120 legacy=0x7f9b20005e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7f9b2d4c4640 1 -- 172.21.15.16:0/628099787 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b20006f60 con 0x7f9b2004eb60 2024-09-19T07:43:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7f9b2d4c4640 1 -- 172.21.15.16:0/628099787 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f9b28050800 con 0x7f9b2004eb60 2024-09-19T07:43:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7f9b2e4c6640 1 --2- 172.21.15.16:0/628099787 >> v2:172.21.15.16:3300/0 conn(0x7f9b20005a60 0x7f9b20053f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7f326a59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32640071e0 0x7f32640defd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7f9b2d4c4640 1 -- 172.21.15.16:0/628099787 >> v1:172.21.15.16:6789/0 conn(0x7f9b2004eb60 legacy=0x7f9b2001d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7f326ad9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32640392d0 0x7f32640e6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7f9b2d4c4640 1 --2- 172.21.15.16:0/628099787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b140039b0 0x7f9b14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7f326a59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32640071e0 0x7f32640defd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:58068/0 (socket says 172.21.15.16:58068) 2024-09-19T07:43:29.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7f326a59c640 1 -- 172.21.15.16:0/2287678069 learned_addr learned my addr 172.21.15.16:0/2287678069 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:29.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f9b2d4c4640 1 --2- 172.21.15.16:0/628099787 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b14004800 0x7f9b14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f9b2d4c4640 1 --2- 172.21.15.16:0/628099787 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b140051c0 0x7f9b1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f3270d91640 1 -- 172.21.15.16:0/2287678069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32640475a0 con 0x7f3264033a40 2024-09-19T07:43:29.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f3270d91640 1 -- 172.21.15.16:0/2287678069 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f326400a060 con 0x7f32640071e0 2024-09-19T07:43:29.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7fc3c29c5640 1 Processor -- start 2024-09-19T07:43:29.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7fc3c29c5640 1 -- start start 2024-09-19T07:43:29.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f9b2e4c6640 1 --2- 172.21.15.16:0/628099787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b140039b0 0x7f9b14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f3270d91640 1 -- 172.21.15.16:0/2287678069 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3264047720 con 0x7f32640392d0 2024-09-19T07:43:29.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.917+0000 7f66d67da640 1 -- start start 2024-09-19T07:43:29.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f66d67da640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66d00cfa30 0x7f66d0203a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f66d67da640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66d0158860 0x7f66d0200aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f66d67da640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66d0158c40 0x7f66d0147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f66d67da640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66d006fd00 con 0x7f66d0158c40 2024-09-19T07:43:29.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f66d67da640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f66d006fb80 con 0x7f66d0158860 2024-09-19T07:43:29.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f66d67da640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f66d015eaa0 con 0x7f66d00cfa30 2024-09-19T07:43:29.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7fbfeffff640 1 --2- 172.21.15.16:0/643810409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff0154680 0x7fbff0157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7fbfef7fe640 1 -- 172.21.15.16:0/643810409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbff00cdce0 msgr2=0x7fbff014e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7fbfef7fe640 1 --2- 172.21.15.16:0/643810409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbff00cdce0 0x7fbff014e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7fbfef7fe640 1 -- 172.21.15.16:0/643810409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff0154680 msgr2=0x7fbff0157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7fbfef7fe640 1 --2- 172.21.15.16:0/643810409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff0154680 0x7fbff0157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7fbfef7fe640 1 -- 172.21.15.16:0/643810409 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff014eeb0 con 0x7fbff00b5b30 2024-09-19T07:43:29.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7fbfeeffd640 1 --2- 172.21.15.16:0/643810409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbff00cdce0 0x7fbff014e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:29.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7fbfef7fe640 1 --2- 172.21.15.16:0/643810409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbff00b5b30 0x7fbff0159820 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400c5f0 tx=0x7fbfe400cab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f3269d9b640 1 --2- 172.21.15.16:0/2287678069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3264033a40 0x7f3264038d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7f326a59c640 1 -- 172.21.15.16:0/2287678069 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32640392d0 msgr2=0x7f32640e6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7f326a59c640 1 --2- 172.21.15.16:0/2287678069 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32640392d0 0x7f32640e6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7f326a59c640 1 -- 172.21.15.16:0/2287678069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3264033a40 msgr2=0x7f3264038d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7f326a59c640 1 --2- 172.21.15.16:0/2287678069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3264033a40 0x7f3264038d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7f326a59c640 1 -- 172.21.15.16:0/2287678069 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32640e7c90 con 0x7f32640071e0 2024-09-19T07:43:29.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7f3269d9b640 1 --2- 172.21.15.16:0/2287678069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3264033a40 0x7f3264038d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:29.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f95be1fd640 1 -- 172.21.15.16:0/860117336 shutdown_connections 2024-09-19T07:43:29.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f58fa6af640 1 Processor -- start 2024-09-19T07:43:29.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f58fa6af640 1 -- start start 2024-09-19T07:43:29.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7fc3c29c5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3b400b770 0x7fc3b41020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7fc3c29c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3b4036020 0x7fc3b4032310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7fc3c29c5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3b404eb60 0x7fc3b4032850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7fc3c29c5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b4100a80 con 0x7fc3b4036020 2024-09-19T07:43:29.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7fc3c29c5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b40553c0 con 0x7fc3b404eb60 2024-09-19T07:43:29.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7fc3c29c5640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b4006e70 con 0x7fc3b400b770 2024-09-19T07:43:29.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7fbfd7fff640 1 -- 172.21.15.16:0/643810409 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4008040 con 0x7fbff00b5b30 2024-09-19T07:43:29.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7fbfd7fff640 1 -- 172.21.15.16:0/643810409 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfe4015e20 con 0x7fbff00b5b30 2024-09-19T07:43:29.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7fbfd7fff640 1 -- 172.21.15.16:0/643810409 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4008040 con 0x7fbff00b5b30 2024-09-19T07:43:29.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7f326a59c640 1 --2- 172.21.15.16:0/2287678069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32640071e0 0x7f32640defd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f325c00e870 tx=0x7f325c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f9b2d4c4640 1 -- 172.21.15.16:0/628099787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b200520e0 con 0x7f9b140039b0 2024-09-19T07:43:29.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f9b2d4c4640 1 -- 172.21.15.16:0/628099787 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1400c140 con 0x7f9b14004800 2024-09-19T07:43:29.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7f325affd640 1 -- 172.21.15.16:0/2287678069 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c017070 con 0x7f32640071e0 2024-09-19T07:43:29.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7f325affd640 1 -- 172.21.15.16:0/2287678069 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f325c00c8e0 con 0x7f32640071e0 2024-09-19T07:43:29.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f9b2d4c4640 1 -- 172.21.15.16:0/628099787 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9b1400c340 con 0x7f9b140051c0 2024-09-19T07:43:29.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f9b2dcc5640 1 --2- 172.21.15.16:0/628099787 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b14004800 0x7f9b14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f58fa6af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f413c880 0x7f58f4157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f9b2dcc5640 1 -- 172.21.15.16:0/628099787 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b140051c0 msgr2=0x7f9b1400b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:43:29.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f58fa6af640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58f413cc60 0x7f58f415a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f58fa6af640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58f4155340 0x7f58f415ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f9b2dcc5640 1 --2- 172.21.15.16:0/628099787 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b140051c0 0x7f9b1400b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f9b2dcc5640 1 -- 172.21.15.16:0/628099787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b140039b0 msgr2=0x7f9b14003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f9b2dcc5640 1 --2- 172.21.15.16:0/628099787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b140039b0 0x7f9b14003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f58fa6af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58f415be80 con 0x7f58f413c880 2024-09-19T07:43:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f9b2dcc5640 1 -- 172.21.15.16:0/628099787 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b14003520 con 0x7f9b14004800 2024-09-19T07:43:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f58fa6af640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f58f415bd00 con 0x7f58f4155340 2024-09-19T07:43:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7f9b2e4c6640 1 --2- 172.21.15.16:0/628099787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b140039b0 0x7f9b14003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f58f9eae640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58f4155340 0x7f58f415ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f58f9eae640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58f4155340 0x7f58f415ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:58088/0 (socket says 172.21.15.16:58088) 2024-09-19T07:43:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f58f9eae640 1 -- 172.21.15.16:0/2222766379 learned_addr learned my addr 172.21.15.16:0/2222766379 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f58fa6af640 1 -- 172.21.15.16:0/2222766379 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f58f415c000 con 0x7f58f413cc60 2024-09-19T07:43:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f58f96ad640 1 --2- 172.21.15.16:0/2222766379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f413c880 0x7f58f4157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f9b2dcc5640 1 --2- 172.21.15.16:0/628099787 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b14004800 0x7f9b14004c80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9b24001750 tx=0x7f9b14005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c0ee12859857fb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f95be1fd640 1 -- 172.21.15.16:0/860117336 wait complete. 2024-09-19T07:43:29.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f95be1fd640 1 Processor -- start 2024-09-19T07:43:29.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f9b2ecc7640 1 -- 172.21.15.16:0/628099787 reap_dead start 2024-09-19T07:43:29.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f3270d91640 1 -- 172.21.15.16:0/2287678069 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32640e7f70 con 0x7f32640071e0 2024-09-19T07:43:29.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7fbff4d8d640 1 -- 172.21.15.16:0/643810409 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff0156ce0 con 0x7fbff00b5b30 2024-09-19T07:43:29.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7fbff4d8d640 1 -- 172.21.15.16:0/643810409 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff01572a0 con 0x7fbff00b5b30 2024-09-19T07:43:29.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f3270d91640 1 -- 172.21.15.16:0/2287678069 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32640e8530 con 0x7f32640071e0 2024-09-19T07:43:29.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f325affd640 1 -- 172.21.15.16:0/2287678069 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c017070 con 0x7f32640071e0 2024-09-19T07:43:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f95be1fd640 1 -- start start 2024-09-19T07:43:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.921+0000 7f58f8eac640 1 --2- 172.21.15.16:0/2222766379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58f413cc60 0x7f58f415a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f66d4fd7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66d0158860 0x7f66d0200aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f66d4fd7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66d0158860 0x7f66d0200aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:58080/0 (socket says 172.21.15.16:58080) 2024-09-19T07:43:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.918+0000 7f66d4fd7640 1 -- 172.21.15.16:0/1887888189 learned_addr learned my addr 172.21.15.16:0/1887888189 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f66d4fd7640 1 -- 172.21.15.16:0/1887888189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66d00cfa30 msgr2=0x7f66d0203a20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T07:43:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f66d4fd7640 1 --2- 172.21.15.16:0/1887888189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66d00cfa30 0x7f66d0203a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f66d4fd7640 1 -- 172.21.15.16:0/1887888189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66d0158c40 msgr2=0x7f66d0147b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f66d4fd7640 1 --2- 172.21.15.16:0/1887888189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66d0158c40 0x7f66d0147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f66d4fd7640 1 -- 172.21.15.16:0/1887888189 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d0148430 con 0x7f66d0158860 2024-09-19T07:43:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f66d4fd7640 1 --2- 172.21.15.16:0/1887888189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66d0158860 0x7f66d0200aa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f66c8007e90 tx=0x7f66c800e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f66c5ffb640 1 -- 172.21.15.16:0/1887888189 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c801a070 con 0x7f66d0158860 2024-09-19T07:43:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7f66c5ffb640 1 -- 172.21.15.16:0/1887888189 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66c8003b90 con 0x7f66d0158860 2024-09-19T07:43:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.921+0000 7f9b2d4c4640 1 -- 172.21.15.16:0/628099787 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24001370 con 0x7f9b14004800 2024-09-19T07:43:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.921+0000 7f9b2d4c4640 1 -- 172.21.15.16:0/628099787 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b240052d0 con 0x7f9b14004800 2024-09-19T07:43:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.921+0000 7f58f9eae640 1 -- 172.21.15.16:0/2222766379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58f413cc60 msgr2=0x7f58f415a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.921+0000 7f58f9eae640 1 --2- 172.21.15.16:0/2222766379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58f413cc60 0x7f58f415a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.921+0000 7f66c5ffb640 1 -- 172.21.15.16:0/1887888189 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c800ee70 con 0x7f66d0158860 2024-09-19T07:43:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.921+0000 7f58f9eae640 1 -- 172.21.15.16:0/2222766379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f413c880 msgr2=0x7f58f4157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.921+0000 7f58f9eae640 1 --2- 172.21.15.16:0/2222766379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f413c880 0x7f58f4157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.921+0000 7f58f9eae640 1 -- 172.21.15.16:0/2222766379 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f4158830 con 0x7f58f4155340 2024-09-19T07:43:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.921+0000 7f9b2d4c4640 1 -- 172.21.15.16:0/628099787 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2402ac00 con 0x7f9b14004800 2024-09-19T07:43:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.921+0000 7f66d67da640 1 -- 172.21.15.16:0/1887888189 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d0148710 con 0x7f66d0158860 2024-09-19T07:43:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.921+0000 7f66d67da640 1 -- 172.21.15.16:0/1887888189 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d0148cd0 con 0x7f66d0158860 2024-09-19T07:43:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7fc3c21c4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3b404eb60 0x7fc3b4032850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7fc3c21c4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3b404eb60 0x7fc3b4032850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:58084/0 (socket says 172.21.15.16:58084) 2024-09-19T07:43:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7fc3c21c4640 1 -- 172.21.15.16:0/707409867 learned_addr learned my addr 172.21.15.16:0/707409867 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.919+0000 7fc3c11c2640 1 --2- 172.21.15.16:0/707409867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3b4036020 0x7fc3b4032310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7fc3c21c4640 1 -- 172.21.15.16:0/707409867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3b400b770 msgr2=0x7fc3b41020b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T07:43:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7fc3c21c4640 1 --2- 172.21.15.16:0/707409867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3b400b770 0x7fc3b41020b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7fc3c21c4640 1 -- 172.21.15.16:0/707409867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3b4036020 msgr2=0x7fc3b4032310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7fc3c21c4640 1 --2- 172.21.15.16:0/707409867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3b4036020 0x7fc3b4032310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7fc3c21c4640 1 -- 172.21.15.16:0/707409867 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b41025f0 con 0x7fc3b404eb60 2024-09-19T07:43:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.920+0000 7fc3c11c2640 1 --2- 172.21.15.16:0/707409867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3b4036020 0x7fc3b4032310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.922+0000 7f95be1fd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95b80b29a0 0x7f95b8099860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.922+0000 7f95be1fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95b8158220 0x7f95b8146bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.922+0000 7f9b2f4c8640 1 -- 172.21.15.16:0/628099787 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b14004800 msgr2=0x7f9b14004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.922+0000 7f58f9eae640 1 --2- 172.21.15.16:0/2222766379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58f4155340 0x7f58f415ab90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f58f4155a20 tx=0x7f58ec00c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.922+0000 7f9b2f4c8640 1 --2- 172.21.15.16:0/628099787 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b14004800 0x7f9b14004c80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9b24001750 tx=0x7f9b14005830 comp rx=0 tx=0).stop 2024-09-19T07:43:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.922+0000 7fc3c21c4640 1 --2- 172.21.15.16:0/707409867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3b404eb60 0x7fc3b4032850 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00c050 tx=0x7fc3ac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.922+0000 7f95b6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95b8158220 0x7f95b8146bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.922+0000 7f3258ff9640 1 -- 172.21.15.16:0/2287678069 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3244005290 con 0x7f32640071e0 2024-09-19T07:43:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.922+0000 7fc3b27fc640 1 -- 172.21.15.16:0/707409867 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac03a030 con 0x7fc3b404eb60 2024-09-19T07:43:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.922+0000 7fc3b27fc640 1 -- 172.21.15.16:0/707409867 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3ac03c070 con 0x7fc3b404eb60 2024-09-19T07:43:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.922+0000 7fc3b27fc640 1 -- 172.21.15.16:0/707409867 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac0359d0 con 0x7fc3b404eb60 2024-09-19T07:43:29.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.922+0000 7f9b2f4c8640 1 -- 172.21.15.16:0/628099787 shutdown_connections 2024-09-19T07:43:29.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.922+0000 7f9b2f4c8640 1 --2- 172.21.15.16:0/628099787 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b14004800 0x7f9b14004c80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.922+0000 7f9b2f4c8640 1 -- 172.21.15.16:0/628099787 >> 172.21.15.16:0/628099787 conn(0x7f9b20036090 msgr2=0x7f9b200364a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:29.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.922+0000 7f95b6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95b8158220 0x7f95b8146bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57740/0 (socket says 172.21.15.16:57740) 2024-09-19T07:43:29.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.923+0000 7f95b6ffd640 1 -- 172.21.15.16:0/3486143677 learned_addr learned my addr 172.21.15.16:0/3486143677 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:29.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.923+0000 7f95be1fd640 1 --2- 172.21.15.16:0/3486143677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95b815ec70 0x7f95b80ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.923+0000 7f95be1fd640 1 -- 172.21.15.16:0/3486143677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f95b80c6390 con 0x7f95b8158220 2024-09-19T07:43:29.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.923+0000 7f95be1fd640 1 -- 172.21.15.16:0/3486143677 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f95b80c6210 con 0x7f95b80b29a0 2024-09-19T07:43:29.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.923+0000 7f95be1fd640 1 -- 172.21.15.16:0/3486143677 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f95b80ce380 con 0x7f95b815ec70 2024-09-19T07:43:29.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.923+0000 7fc3c29c5640 1 -- 172.21.15.16:0/707409867 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b419ebb0 con 0x7fc3b404eb60 2024-09-19T07:43:29.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.923+0000 7fc3c29c5640 1 -- 172.21.15.16:0/707409867 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b419f170 con 0x7fc3b404eb60 2024-09-19T07:43:29.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.923+0000 7f325affd640 1 -- 172.21.15.16:0/2287678069 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f325c01a070 con 0x7f32640071e0 2024-09-19T07:43:29.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.923+0000 7f66d67da640 1 -- 172.21.15.16:0/1887888189 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d015b4b0 con 0x7f66d0158860 2024-09-19T07:43:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.923+0000 7f58e9ffb640 1 -- 172.21.15.16:0/2222766379 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ec008030 con 0x7f58f4155340 2024-09-19T07:43:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.922+0000 7fbff4d8d640 1 -- 172.21.15.16:0/643810409 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbff006e8d0 con 0x7fbff00b5b30 2024-09-19T07:43:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.923+0000 7f58e9ffb640 1 -- 172.21.15.16:0/2222766379 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58ec00cd30 con 0x7f58f4155340 2024-09-19T07:43:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.924+0000 7f95b77fe640 1 --2- 172.21.15.16:0/3486143677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95b80b29a0 0x7f95b8099860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.924+0000 7f9b2f4c8640 1 -- 172.21.15.16:0/628099787 shutdown_connections 2024-09-19T07:43:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.924+0000 7f58e9ffb640 1 -- 172.21.15.16:0/2222766379 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ec009a30 con 0x7f58f4155340 2024-09-19T07:43:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.924+0000 7f325affd640 1 --2- 172.21.15.16:0/2287678069 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f324c06c6e0 0x7f324c06eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.924+0000 7f325affd640 1 -- 172.21.15.16:0/2287678069 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f325c00f040 con 0x7f32640071e0 2024-09-19T07:43:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.924+0000 7f58fa6af640 1 -- 172.21.15.16:0/2222766379 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f4158b10 con 0x7f58f4155340 2024-09-19T07:43:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.924+0000 7fc3c29c5640 1 -- 172.21.15.16:0/707409867 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b4006010 con 0x7fc3b404eb60 2024-09-19T07:43:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.924+0000 7f66c5ffb640 1 -- 172.21.15.16:0/1887888189 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f66c801d070 con 0x7f66d0158860 2024-09-19T07:43:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.923+0000 7fbfd7fff640 1 -- 172.21.15.16:0/643810409 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbfe4004a00 con 0x7fbff00b5b30 2024-09-19T07:43:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.924+0000 7fbfd7fff640 1 --2- 172.21.15.16:0/643810409 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbfdc06c6b0 0x7fbfdc06eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.924+0000 7fbfd7fff640 1 -- 172.21.15.16:0/643810409 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbfe4015c20 con 0x7fbff00b5b30 2024-09-19T07:43:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.925+0000 7f3269d9b640 1 --2- 172.21.15.16:0/2287678069 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f324c06c6e0 0x7f324c06eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.925+0000 7f95b7fff640 1 --2- 172.21.15.16:0/3486143677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95b815ec70 0x7f95b80ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.925+0000 7fbfeeffd640 1 --2- 172.21.15.16:0/643810409 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbfdc06c6b0 0x7fbfdc06eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.925+0000 7f95b6ffd640 1 -- 172.21.15.16:0/3486143677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95b815ec70 msgr2=0x7f95b80ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.924+0000 7f9b2f4c8640 1 -- 172.21.15.16:0/628099787 wait complete. 2024-09-19T07:43:29.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.925+0000 7f66c5ffb640 1 --2- 172.21.15.16:0/1887888189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f66b406c380 0x7f66b406e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.925+0000 7f95b6ffd640 1 --2- 172.21.15.16:0/3486143677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95b815ec70 0x7f95b80ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.925+0000 7f95b6ffd640 1 -- 172.21.15.16:0/3486143677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95b80b29a0 msgr2=0x7f95b8099860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.925+0000 7f66c5ffb640 1 -- 172.21.15.16:0/1887888189 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f66c808b940 con 0x7f66d0158860 2024-09-19T07:43:29.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.925+0000 7fc3b27fc640 1 -- 172.21.15.16:0/707409867 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc3ac03e030 con 0x7fc3b404eb60 2024-09-19T07:43:29.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.925+0000 7f95b6ffd640 1 --2- 172.21.15.16:0/3486143677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95b80b29a0 0x7f95b8099860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.925+0000 7f95b6ffd640 1 -- 172.21.15.16:0/3486143677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b815ce40 con 0x7f95b8158220 2024-09-19T07:43:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.925+0000 7f58fa6af640 1 -- 172.21.15.16:0/2222766379 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f41590d0 con 0x7f58f4155340 2024-09-19T07:43:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.925+0000 7f95b77fe640 1 --2- 172.21.15.16:0/3486143677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95b80b29a0 0x7f95b8099860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.926+0000 7fbfeeffd640 1 --2- 172.21.15.16:0/643810409 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbfdc06c6b0 0x7fbfdc06eb70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbff00cc370 tx=0x7fbfe800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.926+0000 7f66d57d8640 1 --2- 172.21.15.16:0/1887888189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f66b406c380 0x7f66b406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.926+0000 7f95b6ffd640 1 --2- 172.21.15.16:0/3486143677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95b8158220 0x7f95b8146bb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f95a8000c00 tx=0x7f95a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.926+0000 7f9b2f4c8640 1 Processor -- start 2024-09-19T07:43:29.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.926+0000 7f9b2f4c8640 1 -- start start 2024-09-19T07:43:29.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.926+0000 7fc3b27fc640 1 --2- 172.21.15.16:0/707409867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc3a006c300 0x7fc3a006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.926+0000 7f959ffff640 1 -- 172.21.15.16:0/3486143677 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8037040 con 0x7f95b8158220 2024-09-19T07:43:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.926+0000 7f3269d9b640 1 --2- 172.21.15.16:0/2287678069 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f324c06c6e0 0x7f324c06eba0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f326401ca50 tx=0x7f326c0aa000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.926+0000 7f95be1fd640 1 -- 172.21.15.16:0/3486143677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b815d120 con 0x7f95b8158220 2024-09-19T07:43:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.926+0000 7f959ffff640 1 -- 172.21.15.16:0/3486143677 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95a8004250 con 0x7f95b8158220 2024-09-19T07:43:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.926+0000 7f95be1fd640 1 -- 172.21.15.16:0/3486143677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b815d6e0 con 0x7f95b8158220 2024-09-19T07:43:29.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.926+0000 7fc3b27fc640 1 -- 172.21.15.16:0/707409867 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc3ac0abea0 con 0x7fc3b404eb60 2024-09-19T07:43:29.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.926+0000 7f66d57d8640 1 --2- 172.21.15.16:0/1887888189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f66b406c380 0x7f66b406e840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f66d00b0850 tx=0x7f66cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.926+0000 7fc3c19c3640 1 --2- 172.21.15.16:0/707409867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc3a006c300 0x7fc3a006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.926+0000 7f959ffff640 1 -- 172.21.15.16:0/3486143677 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8003610 con 0x7f95b8158220 2024-09-19T07:43:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.927+0000 7f9b2f4c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b14004800 0x7f9b2004a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.927+0000 7fc3c19c3640 1 --2- 172.21.15.16:0/707409867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc3a006c300 0x7fc3a006e7c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800a810 tx=0x7fc3b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.927+0000 7f58d77fe640 1 -- 172.21.15.16:0/2222766379 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d0005290 con 0x7f58f4155340 2024-09-19T07:43:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.927+0000 7f9b2f4c8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b200e37e0 0x7f9b2004b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.927+0000 7f9b2f4c8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2004a6a0 0x7f9b2004be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.927+0000 7f9b2f4c8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b20047b50 con 0x7f9b14004800 2024-09-19T07:43:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.927+0000 7f9b2f4c8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9b20053a10 con 0x7f9b200e37e0 2024-09-19T07:43:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.927+0000 7f9b2f4c8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9b20047cd0 con 0x7f9b2004a6a0 2024-09-19T07:43:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.927+0000 7f9b2e4c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b14004800 0x7f9b2004a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.927+0000 7f9b2e4c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b14004800 0x7f9b2004a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57742/0 (socket says 172.21.15.16:57742) 2024-09-19T07:43:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.927+0000 7f9b2e4c6640 1 -- 172.21.15.16:0/844250585 learned_addr learned my addr 172.21.15.16:0/844250585 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.927+0000 7f9b2dcc5640 1 --2- 172.21.15.16:0/844250585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b200e37e0 0x7f9b2004b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.928+0000 7f9b2ecc7640 1 --2- 172.21.15.16:0/844250585 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2004a6a0 0x7f9b2004be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.928+0000 7f95be1fd640 1 -- 172.21.15.16:0/3486143677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9598005290 con 0x7f95b8158220 2024-09-19T07:43:29.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.928+0000 7f9b2e4c6640 1 -- 172.21.15.16:0/844250585 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2004a6a0 msgr2=0x7f9b2004be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.928+0000 7f9b2e4c6640 1 --2- 172.21.15.16:0/844250585 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2004a6a0 0x7f9b2004be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.928+0000 7f9b2e4c6640 1 -- 172.21.15.16:0/844250585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b200e37e0 msgr2=0x7f9b2004b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:29.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.928+0000 7f9b2e4c6640 1 --2- 172.21.15.16:0/844250585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b200e37e0 0x7f9b2004b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:29.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.928+0000 7f9b2e4c6640 1 -- 172.21.15.16:0/844250585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b20037550 con 0x7f9b14004800 2024-09-19T07:43:29.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.928+0000 7f9b2e4c6640 1 --2- 172.21.15.16:0/844250585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b14004800 0x7f9b2004a160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800d890 tx=0x7f9b1800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.928+0000 7f9b1effd640 1 -- 172.21.15.16:0/844250585 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b18016070 con 0x7f9b14004800 2024-09-19T07:43:29.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.928+0000 7f9b1effd640 1 -- 172.21.15.16:0/844250585 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b18008ac0 con 0x7f9b14004800 2024-09-19T07:43:29.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.928+0000 7f9b2f4c8640 1 -- 172.21.15.16:0/844250585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b20037830 con 0x7f9b14004800 2024-09-19T07:43:29.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.928+0000 7f9b2f4c8640 1 -- 172.21.15.16:0/844250585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b20049c40 con 0x7f9b14004800 2024-09-19T07:43:29.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.929+0000 7f325affd640 1 -- 172.21.15.16:0/2287678069 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f325c058e50 con 0x7f32640071e0 2024-09-19T07:43:29.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.929+0000 7f9b1effd640 1 -- 172.21.15.16:0/844250585 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1800de80 con 0x7f9b14004800 2024-09-19T07:43:29.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.929+0000 7f959ffff640 1 -- 172.21.15.16:0/3486143677 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f95a803a040 con 0x7f95b8158220 2024-09-19T07:43:29.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.929+0000 7f959ffff640 1 --2- 172.21.15.16:0/3486143677 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f95a406c300 0x7f95a406e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.930+0000 7f9b2f4c8640 1 -- 172.21.15.16:0/844250585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b20047600 con 0x7f9b14004800 2024-09-19T07:43:29.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.930+0000 7f959ffff640 1 -- 172.21.15.16:0/3486143677 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f95a80abaa0 con 0x7f95b8158220 2024-09-19T07:43:29.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.930+0000 7fbfd7fff640 1 -- 172.21.15.16:0/643810409 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fbff006e8d0 con 0x7fbff00b5b30 2024-09-19T07:43:29.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.930+0000 7f95b77fe640 1 --2- 172.21.15.16:0/3486143677 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f95a406c300 0x7f95a406e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.930+0000 7f95b77fe640 1 --2- 172.21.15.16:0/3486143677 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f95a406c300 0x7f95a406e7c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f95ac007fd0 tx=0x7f95ac007d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.931+0000 7f9b1effd640 1 -- 172.21.15.16:0/844250585 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9b18019070 con 0x7f9b14004800 2024-09-19T07:43:29.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.932+0000 7f9b1effd640 1 --2- 172.21.15.16:0/844250585 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b0c06c2b0 0x7f9b0c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:29.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.932+0000 7f9b1effd640 1 -- 172.21.15.16:0/844250585 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9b1808bdc0 con 0x7f9b14004800 2024-09-19T07:43:29.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.932+0000 7f9b2dcc5640 1 --2- 172.21.15.16:0/844250585 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b0c06c2b0 0x7f9b0c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:29.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.932+0000 7f9b2dcc5640 1 --2- 172.21.15.16:0/844250585 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b0c06c2b0 0x7f9b0c06e770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9b24032000 tx=0x7f9b2402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:29.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.933+0000 7f66c5ffb640 1 -- 172.21.15.16:0/1887888189 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f66c8059700 con 0x7f66d0158860 2024-09-19T07:43:29.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.935+0000 7f959ffff640 1 -- 172.21.15.16:0/3486143677 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f95a8042020 con 0x7f95b8158220 2024-09-19T07:43:29.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.937+0000 7fc3b27fc640 1 -- 172.21.15.16:0/707409867 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc3ac079db0 con 0x7fc3b404eb60 2024-09-19T07:43:30.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.939+0000 7f58e9ffb640 1 -- 172.21.15.16:0/2222766379 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f58ec013650 con 0x7f58f4155340 2024-09-19T07:43:30.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.940+0000 7f58e9ffb640 1 --2- 172.21.15.16:0/2222766379 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58d806c380 0x7f58d806e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:30.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.940+0000 7f58e9ffb640 1 -- 172.21.15.16:0/2222766379 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f58ec090610 con 0x7f58f4155340 2024-09-19T07:43:30.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.940+0000 7f58f96ad640 1 --2- 172.21.15.16:0/2222766379 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58d806c380 0x7f58d806e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:30.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.941+0000 7f58f96ad640 1 --2- 172.21.15.16:0/2222766379 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58d806c380 0x7f58d806e840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f58f40c6190 tx=0x7f58e4007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:30.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.942+0000 7f58e9ffb640 1 -- 172.21.15.16:0/2222766379 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f58ec05e3d0 con 0x7f58f4155340 2024-09-19T07:43:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:29.944+0000 7f9b1effd640 1 -- 172.21.15.16:0/844250585 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9b18059b00 con 0x7f9b14004800 2024-09-19T07:43:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.256+0000 7efc9bfe4640 1 -- 172.21.15.16:0/3729920700 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7efc9406ff60 con 0x7efc940b2aa0 2024-09-19T07:43:30.260 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-09-19T07:43:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.257+0000 7efc8a7fc640 1 -- 172.21.15.16:0/3729920700 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efc90058b00 con 0x7efc940b2aa0 2024-09-19T07:43:30.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.262+0000 7efc77fff640 1 -- 172.21.15.16:0/3729920700 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efc7806c380 msgr2=0x7efc7806e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.262+0000 7efc77fff640 1 --2- 172.21.15.16:0/3729920700 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efc7806c380 0x7efc7806e840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efc84034000 tx=0x7efc8402e000 comp rx=0 tx=0).stop 2024-09-19T07:43:30.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.263+0000 7efc77fff640 1 -- 172.21.15.16:0/3729920700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc940b2aa0 msgr2=0x7efc9414fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.264+0000 7efc77fff640 1 --2- 172.21.15.16:0/3729920700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc940b2aa0 0x7efc9414fa90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efc9000c810 tx=0x7efc9000ccd0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.264+0000 7efc77fff640 1 -- 172.21.15.16:0/3729920700 shutdown_connections 2024-09-19T07:43:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.264+0000 7efc77fff640 1 --2- 172.21.15.16:0/3729920700 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efc7806c380 0x7efc7806e840 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.264+0000 7efc77fff640 1 --2- 172.21.15.16:0/3729920700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc940cd2d0 0x7efc940afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.264+0000 7efc77fff640 1 --2- 172.21.15.16:0/3729920700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc940b2aa0 0x7efc9414fa90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.265+0000 7efc77fff640 1 --2- 172.21.15.16:0/3729920700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc94073960 0x7efc940b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.265+0000 7efc77fff640 1 -- 172.21.15.16:0/3729920700 >> 172.21.15.16:0/3729920700 conn(0x7efc940b1e80 msgr2=0x7efc94075140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:30.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.265+0000 7efc77fff640 1 -- 172.21.15.16:0/3729920700 shutdown_connections 2024-09-19T07:43:30.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.265+0000 7efc77fff640 1 -- 172.21.15.16:0/3729920700 wait complete. 2024-09-19T07:43:30.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.4 2024-09-19T07:43:30.291 DEBUG:teuthology.parallel:result is None 2024-09-19T07:43:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.316+0000 7f9b2f4c8640 1 -- 172.21.15.16:0/844250585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f9b20034360 con 0x7f9b14004800 2024-09-19T07:43:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.317+0000 7f9b1effd640 1 -- 172.21.15.16:0/844250585 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9b1805de00 con 0x7f9b14004800 2024-09-19T07:43:30.318 INFO:teuthology.orchestra.run.smithi016.stdout:60129542152 2024-09-19T07:43:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.319+0000 7f9b2f4c8640 1 -- 172.21.15.16:0/844250585 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b0c06c2b0 msgr2=0x7f9b0c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.319+0000 7f9b2f4c8640 1 --2- 172.21.15.16:0/844250585 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b0c06c2b0 0x7f9b0c06e770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9b24032000 tx=0x7f9b2402e000 comp rx=0 tx=0).stop 2024-09-19T07:43:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.319+0000 7f9b2f4c8640 1 -- 172.21.15.16:0/844250585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b14004800 msgr2=0x7f9b2004a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.319+0000 7f9b2f4c8640 1 --2- 172.21.15.16:0/844250585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b14004800 0x7f9b2004a160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800d890 tx=0x7f9b1800dc90 comp rx=0 tx=0).stop 2024-09-19T07:43:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.319+0000 7f9b2f4c8640 1 -- 172.21.15.16:0/844250585 shutdown_connections 2024-09-19T07:43:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.319+0000 7f9b2f4c8640 1 --2- 172.21.15.16:0/844250585 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b0c06c2b0 0x7f9b0c06e770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.319+0000 7f9b2f4c8640 1 --2- 172.21.15.16:0/844250585 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2004a6a0 0x7f9b2004be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.319+0000 7f9b2f4c8640 1 --2- 172.21.15.16:0/844250585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b200e37e0 0x7f9b2004b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.319+0000 7f9b2f4c8640 1 --2- 172.21.15.16:0/844250585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b14004800 0x7f9b2004a160 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.319+0000 7f9b2f4c8640 1 -- 172.21.15.16:0/844250585 >> 172.21.15.16:0/844250585 conn(0x7f9b20036090 msgr2=0x7f9b200e60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.320+0000 7f9b2f4c8640 1 -- 172.21.15.16:0/844250585 shutdown_connections 2024-09-19T07:43:30.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.320+0000 7f9b2f4c8640 1 -- 172.21.15.16:0/844250585 wait complete. 2024-09-19T07:43:30.331 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.3 2024-09-19T07:43:30.331 DEBUG:teuthology.parallel:result is None 2024-09-19T07:43:30.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.336+0000 7fbff4d8d640 1 -- 172.21.15.16:0/643810409 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fbff015a7f0 con 0x7fbff00b5b30 2024-09-19T07:43:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.337+0000 7fbfd7fff640 1 -- 172.21.15.16:0/643810409 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbfe4061150 con 0x7fbff00b5b30 2024-09-19T07:43:30.337 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-09-19T07:43:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.339+0000 7fbff4d8d640 1 -- 172.21.15.16:0/643810409 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbfdc06c6b0 msgr2=0x7fbfdc06eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.339+0000 7fbff4d8d640 1 --2- 172.21.15.16:0/643810409 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbfdc06c6b0 0x7fbfdc06eb70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbff00cc370 tx=0x7fbfe800b040 comp rx=0 tx=0).stop 2024-09-19T07:43:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.339+0000 7fbff4d8d640 1 -- 172.21.15.16:0/643810409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbff00b5b30 msgr2=0x7fbff0159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.339+0000 7fbff4d8d640 1 --2- 172.21.15.16:0/643810409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbff00b5b30 0x7fbff0159820 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400c5f0 tx=0x7fbfe400cab0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.339+0000 7fbff4d8d640 1 -- 172.21.15.16:0/643810409 shutdown_connections 2024-09-19T07:43:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.339+0000 7fbff4d8d640 1 --2- 172.21.15.16:0/643810409 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbfdc06c6b0 0x7fbfdc06eb70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.339+0000 7fbff4d8d640 1 --2- 172.21.15.16:0/643810409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff0154680 0x7fbff0157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.339+0000 7fbff4d8d640 1 --2- 172.21.15.16:0/643810409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbff00cdce0 0x7fbff014e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.339+0000 7fbff4d8d640 1 --2- 172.21.15.16:0/643810409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbff00b5b30 0x7fbff0159820 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.339+0000 7fbff4d8d640 1 -- 172.21.15.16:0/643810409 >> 172.21.15.16:0/643810409 conn(0x7fbff006dae0 msgr2=0x7fbff015d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.339+0000 7fbff4d8d640 1 -- 172.21.15.16:0/643810409 shutdown_connections 2024-09-19T07:43:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.339+0000 7fbff4d8d640 1 -- 172.21.15.16:0/643810409 wait complete. 2024-09-19T07:43:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.341+0000 7f66d67da640 1 -- 172.21.15.16:0/1887888189 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f66d00ced50 con 0x7f66d0158860 2024-09-19T07:43:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.341+0000 7f66c5ffb640 1 -- 172.21.15.16:0/1887888189 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f66c805da00 con 0x7f66d0158860 2024-09-19T07:43:30.342 INFO:teuthology.orchestra.run.smithi016.stdout:55834574858 2024-09-19T07:43:30.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.343+0000 7f3258ff9640 1 -- 172.21.15.16:0/2287678069 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3244005580 con 0x7f32640071e0 2024-09-19T07:43:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.343+0000 7f66d67da640 1 -- 172.21.15.16:0/1887888189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f66b406c380 msgr2=0x7f66b406e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.343+0000 7f66d67da640 1 --2- 172.21.15.16:0/1887888189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f66b406c380 0x7f66b406e840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f66d00b0850 tx=0x7f66cc008040 comp rx=0 tx=0).stop 2024-09-19T07:43:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.343+0000 7f66d67da640 1 -- 172.21.15.16:0/1887888189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66d0158860 msgr2=0x7f66d0200aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.343+0000 7f66d67da640 1 --2- 172.21.15.16:0/1887888189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66d0158860 0x7f66d0200aa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f66c8007e90 tx=0x7f66c800e4b0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.344+0000 7f66d67da640 1 -- 172.21.15.16:0/1887888189 shutdown_connections 2024-09-19T07:43:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.344+0000 7f66d67da640 1 --2- 172.21.15.16:0/1887888189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f66b406c380 0x7f66b406e840 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.344+0000 7f66d67da640 1 --2- 172.21.15.16:0/1887888189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66d0158c40 0x7f66d0147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.344+0000 7f66d67da640 1 --2- 172.21.15.16:0/1887888189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66d0158860 0x7f66d0200aa0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.344+0000 7f66d67da640 1 --2- 172.21.15.16:0/1887888189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66d00cfa30 0x7f66d0203a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.344+0000 7f66d67da640 1 -- 172.21.15.16:0/1887888189 >> 172.21.15.16:0/1887888189 conn(0x7f66d006ebb0 msgr2=0x7f66d006fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.344+0000 7f66d67da640 1 -- 172.21.15.16:0/1887888189 shutdown_connections 2024-09-19T07:43:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.344+0000 7f66d67da640 1 -- 172.21.15.16:0/1887888189 wait complete. 2024-09-19T07:43:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.344+0000 7f325affd640 1 -- 172.21.15.16:0/2287678069 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f325c05d150 con 0x7f32640071e0 2024-09-19T07:43:30.346 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-09-19T07:43:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.346+0000 7f58d77fe640 1 -- 172.21.15.16:0/2222766379 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f58d0005850 con 0x7f58f4155340 2024-09-19T07:43:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.346+0000 7f58e9ffb640 1 -- 172.21.15.16:0/2222766379 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f58ec0626d0 con 0x7f58f4155340 2024-09-19T07:43:30.347 INFO:teuthology.orchestra.run.smithi016.stdout:60129542152 2024-09-19T07:43:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.347+0000 7f95be1fd640 1 -- 172.21.15.16:0/3486143677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9598005850 con 0x7f95b8158220 2024-09-19T07:43:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.348+0000 7f959ffff640 1 -- 172.21.15.16:0/3486143677 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f95a8079960 con 0x7f95b8158220 2024-09-19T07:43:30.348 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-09-19T07:43:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.348+0000 7f3258ff9640 1 -- 172.21.15.16:0/2287678069 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f324c06c6e0 msgr2=0x7f324c06eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.348+0000 7f3258ff9640 1 --2- 172.21.15.16:0/2287678069 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f324c06c6e0 0x7f324c06eba0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f326401ca50 tx=0x7f326c0aa000 comp rx=0 tx=0).stop 2024-09-19T07:43:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.348+0000 7f3258ff9640 1 -- 172.21.15.16:0/2287678069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32640071e0 msgr2=0x7f32640defd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.348+0000 7f3258ff9640 1 --2- 172.21.15.16:0/2287678069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32640071e0 0x7f32640defd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f325c00e870 tx=0x7f325c00ed30 comp rx=0 tx=0).stop 2024-09-19T07:43:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.348+0000 7f3258ff9640 1 -- 172.21.15.16:0/2287678069 shutdown_connections 2024-09-19T07:43:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.348+0000 7f3258ff9640 1 --2- 172.21.15.16:0/2287678069 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f324c06c6e0 0x7f324c06eba0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.348+0000 7f3258ff9640 1 --2- 172.21.15.16:0/2287678069 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32640392d0 0x7f32640e6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.348+0000 7f3258ff9640 1 --2- 172.21.15.16:0/2287678069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3264033a40 0x7f3264038d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.348+0000 7f3258ff9640 1 --2- 172.21.15.16:0/2287678069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32640071e0 0x7f32640defd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.348+0000 7f3258ff9640 1 -- 172.21.15.16:0/2287678069 >> 172.21.15.16:0/2287678069 conn(0x7f32640360c0 msgr2=0x7f3264034010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.348+0000 7f3258ff9640 1 -- 172.21.15.16:0/2287678069 shutdown_connections 2024-09-19T07:43:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.349+0000 7f3258ff9640 1 -- 172.21.15.16:0/2287678069 wait complete. 2024-09-19T07:43:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.350+0000 7f58d77fe640 1 -- 172.21.15.16:0/2222766379 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58d806c380 msgr2=0x7f58d806e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.350+0000 7f58d77fe640 1 --2- 172.21.15.16:0/2222766379 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58d806c380 0x7f58d806e840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f58f40c6190 tx=0x7f58e4007040 comp rx=0 tx=0).stop 2024-09-19T07:43:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.350+0000 7f58d77fe640 1 -- 172.21.15.16:0/2222766379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58f4155340 msgr2=0x7f58f415ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.350+0000 7f58d77fe640 1 --2- 172.21.15.16:0/2222766379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58f4155340 0x7f58f415ab90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f58f4155a20 tx=0x7f58ec00c460 comp rx=0 tx=0).stop 2024-09-19T07:43:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.350+0000 7f58d77fe640 1 -- 172.21.15.16:0/2222766379 shutdown_connections 2024-09-19T07:43:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.350+0000 7f58d77fe640 1 --2- 172.21.15.16:0/2222766379 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58d806c380 0x7f58d806e840 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.350+0000 7f58d77fe640 1 --2- 172.21.15.16:0/2222766379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58f4155340 0x7f58f415ab90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.350+0000 7f58d77fe640 1 --2- 172.21.15.16:0/2222766379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58f413cc60 0x7f58f415a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.350+0000 7f58d77fe640 1 --2- 172.21.15.16:0/2222766379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f413c880 0x7f58f4157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.350+0000 7f58d77fe640 1 -- 172.21.15.16:0/2222766379 >> 172.21.15.16:0/2222766379 conn(0x7f58f4067b30 msgr2=0x7f58f40b56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.351+0000 7f58d77fe640 1 -- 172.21.15.16:0/2222766379 shutdown_connections 2024-09-19T07:43:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.351+0000 7f95be1fd640 1 -- 172.21.15.16:0/3486143677 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f95a406c300 msgr2=0x7f95a406e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.351+0000 7f58d77fe640 1 -- 172.21.15.16:0/2222766379 wait complete. 2024-09-19T07:43:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.351+0000 7f95be1fd640 1 --2- 172.21.15.16:0/3486143677 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f95a406c300 0x7f95a406e7c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f95ac007fd0 tx=0x7f95ac007d60 comp rx=0 tx=0).stop 2024-09-19T07:43:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.351+0000 7f95be1fd640 1 -- 172.21.15.16:0/3486143677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95b8158220 msgr2=0x7f95b8146bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.351+0000 7f95be1fd640 1 --2- 172.21.15.16:0/3486143677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95b8158220 0x7f95b8146bb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f95a8000c00 tx=0x7f95a80046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.351+0000 7f95be1fd640 1 -- 172.21.15.16:0/3486143677 shutdown_connections 2024-09-19T07:43:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.351+0000 7f95be1fd640 1 --2- 172.21.15.16:0/3486143677 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f95a406c300 0x7f95a406e7c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.351+0000 7f95be1fd640 1 --2- 172.21.15.16:0/3486143677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95b815ec70 0x7f95b80ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.351+0000 7f95be1fd640 1 --2- 172.21.15.16:0/3486143677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95b8158220 0x7f95b8146bb0 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f95a8000c00 tx=0x7f95a80046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.351+0000 7f95be1fd640 1 --2- 172.21.15.16:0/3486143677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95b80b29a0 0x7f95b8099860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.351+0000 7f95be1fd640 1 -- 172.21.15.16:0/3486143677 >> 172.21.15.16:0/3486143677 conn(0x7f95b80b1e80 msgr2=0x7f95b80b0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.351+0000 7f95be1fd640 1 -- 172.21.15.16:0/3486143677 shutdown_connections 2024-09-19T07:43:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.352+0000 7f95be1fd640 1 -- 172.21.15.16:0/3486143677 wait complete. 2024-09-19T07:43:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.353+0000 7fc3c29c5640 1 -- 172.21.15.16:0/707409867 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fc3b4008ad0 con 0x7fc3b404eb60 2024-09-19T07:43:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.354+0000 7fc3b27fc640 1 -- 172.21.15.16:0/707409867 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc3ac07e0b0 con 0x7fc3b404eb60 2024-09-19T07:43:30.354 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-09-19T07:43:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.356+0000 7fc3c29c5640 1 -- 172.21.15.16:0/707409867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc3a006c300 msgr2=0x7fc3a006e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.356+0000 7fc3c29c5640 1 --2- 172.21.15.16:0/707409867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc3a006c300 0x7fc3a006e7c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800a810 tx=0x7fc3b8008040 comp rx=0 tx=0).stop 2024-09-19T07:43:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.356+0000 7fc3c29c5640 1 -- 172.21.15.16:0/707409867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3b404eb60 msgr2=0x7fc3b4032850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.356+0000 7fc3c29c5640 1 --2- 172.21.15.16:0/707409867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3b404eb60 0x7fc3b4032850 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00c050 tx=0x7fc3ac0046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.356+0000 7fc3c29c5640 1 -- 172.21.15.16:0/707409867 shutdown_connections 2024-09-19T07:43:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.356+0000 7fc3c29c5640 1 --2- 172.21.15.16:0/707409867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc3a006c300 0x7fc3a006e7c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.356+0000 7fc3c29c5640 1 --2- 172.21.15.16:0/707409867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3b404eb60 0x7fc3b4032850 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.356+0000 7fc3c29c5640 1 --2- 172.21.15.16:0/707409867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3b4036020 0x7fc3b4032310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.356+0000 7fc3c29c5640 1 --2- 172.21.15.16:0/707409867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3b400b770 0x7fc3b41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.356+0000 7fc3c29c5640 1 -- 172.21.15.16:0/707409867 >> 172.21.15.16:0/707409867 conn(0x7fc3b401d930 msgr2=0x7fc3b40518a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:30.358 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.7 2024-09-19T07:43:30.359 DEBUG:teuthology.parallel:result is None 2024-09-19T07:43:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.356+0000 7fc3c29c5640 1 -- 172.21.15.16:0/707409867 shutdown_connections 2024-09-19T07:43:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.357+0000 7fc3c29c5640 1 -- 172.21.15.16:0/707409867 wait complete. 2024-09-19T07:43:30.361 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2024-09-19T07:43:30.361 DEBUG:teuthology.parallel:result is None 2024-09-19T07:43:30.367 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.0 2024-09-19T07:43:30.368 DEBUG:teuthology.parallel:result is None 2024-09-19T07:43:30.371 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.6 2024-09-19T07:43:30.371 DEBUG:teuthology.parallel:result is None 2024-09-19T07:43:30.372 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.1 2024-09-19T07:43:30.372 DEBUG:teuthology.parallel:result is None 2024-09-19T07:43:30.373 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.5 2024-09-19T07:43:30.373 DEBUG:teuthology.parallel:result is None 2024-09-19T07:43:30.373 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T07:43:30.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:43:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.514+0000 7f868176f640 1 Processor -- start 2024-09-19T07:43:30.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.515+0000 7f868176f640 1 -- start start 2024-09-19T07:43:30.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.515+0000 7f868176f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f867c06cd20 0x7f867c06d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:30.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.515+0000 7f868176f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f867c06d640 con 0x7f867c161eb0 2024-09-19T07:43:30.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.515+0000 7f868176f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f867c162b70 con 0x7f867c1425e0 2024-09-19T07:43:30.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.515+0000 7f868176f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f867c162e50 con 0x7f867c06cd20 2024-09-19T07:43:30.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.516+0000 7f867a7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f867c161eb0 0x7f867c1622f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44958/0 (socket says 172.21.15.16:44958) 2024-09-19T07:43:30.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.516+0000 7f867a7fc640 1 -- 172.21.15.16:0/202241342 learned_addr learned my addr 172.21.15.16:0/202241342 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:30.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.516+0000 7f867affd640 1 --2- 172.21.15.16:0/202241342 >> v2:172.21.15.146:3300/0 conn(0x7f867c06cd20 0x7f867c06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:30.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.516+0000 7f8679ffb640 1 -- 172.21.15.16:0/202241342 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2655210830 0 0) 0x7f867c06d640 con 0x7f867c161eb0 2024-09-19T07:43:30.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.516+0000 7f8679ffb640 1 -- 172.21.15.16:0/202241342 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8668003520 con 0x7f867c161eb0 2024-09-19T07:43:30.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.516+0000 7f8679ffb640 1 -- 172.21.15.16:0/202241342 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2625824932 0 0) 0x7f867c162b70 con 0x7f867c1425e0 2024-09-19T07:43:30.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.516+0000 7f8679ffb640 1 -- 172.21.15.16:0/202241342 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f867c06d640 con 0x7f867c1425e0 2024-09-19T07:43:30.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.516+0000 7f867affd640 1 -- 172.21.15.16:0/202241342 >> v1:172.21.15.146:6789/0 conn(0x7f867c1425e0 legacy=0x7f867c142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.516+0000 7f867affd640 1 -- 172.21.15.16:0/202241342 >> v1:172.21.15.16:6789/0 conn(0x7f867c161eb0 legacy=0x7f867c1622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.516+0000 7f8679ffb640 1 -- 172.21.15.16:0/202241342 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1327480133 0 0) 0x7f8668003520 con 0x7f867c161eb0 2024-09-19T07:43:30.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.516+0000 7f867affd640 1 -- 172.21.15.16:0/202241342 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867c141b00 con 0x7f867c06cd20 2024-09-19T07:43:30.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.517+0000 7f867affd640 1 --2- 172.21.15.16:0/202241342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f867c06cd20 0x7f867c06d100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f86740097a0 tx=0x7f867402a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17bd3acdb1cac38d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:30.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.517+0000 7f8679ffb640 1 -- 172.21.15.16:0/202241342 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674037070 con 0x7f867c06cd20 2024-09-19T07:43:30.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.517+0000 7f8679ffb640 1 -- 172.21.15.16:0/202241342 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f867402abc0 con 0x7f867c06cd20 2024-09-19T07:43:30.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.517+0000 7f8679ffb640 1 -- 172.21.15.16:0/202241342 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664001670 con 0x7f867c06cd20 2024-09-19T07:43:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.518+0000 7f868176f640 1 -- 172.21.15.16:0/202241342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f867c06cd20 msgr2=0x7f867c06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.518+0000 7f868176f640 1 --2- 172.21.15.16:0/202241342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f867c06cd20 0x7f867c06d100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f86740097a0 tx=0x7f867402a130 comp rx=0 tx=0).stop 2024-09-19T07:43:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.518+0000 7f868176f640 1 -- 172.21.15.16:0/202241342 shutdown_connections 2024-09-19T07:43:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.518+0000 7f868176f640 1 --2- 172.21.15.16:0/202241342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f867c06cd20 0x7f867c06d100 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.518+0000 7f868176f640 1 -- 172.21.15.16:0/202241342 >> 172.21.15.16:0/202241342 conn(0x7f867c0b3ed0 msgr2=0x7f867c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.518+0000 7f868176f640 1 -- 172.21.15.16:0/202241342 shutdown_connections 2024-09-19T07:43:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.518+0000 7f868176f640 1 -- 172.21.15.16:0/202241342 wait complete. 2024-09-19T07:43:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.519+0000 7f868176f640 1 Processor -- start 2024-09-19T07:43:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.519+0000 7f868176f640 1 -- start start 2024-09-19T07:43:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.520+0000 7f868176f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f867c06cd20 0x7f867c15ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.520+0000 7f868176f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f867c1425e0 0x7f867c158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.520+0000 7f868176f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c161eb0 0x7f867c0d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.520+0000 7f868176f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f867c141600 con 0x7f867c161eb0 2024-09-19T07:43:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.520+0000 7f868176f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f867c142150 con 0x7f867c1425e0 2024-09-19T07:43:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.520+0000 7f868176f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f867c143180 con 0x7f867c06cd20 2024-09-19T07:43:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.520+0000 7f867b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c161eb0 0x7f867c0d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.520+0000 7f867b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c161eb0 0x7f867c0d1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57752/0 (socket says 172.21.15.16:57752) 2024-09-19T07:43:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.520+0000 7f867b7fe640 1 -- 172.21.15.16:0/1824145571 learned_addr learned my addr 172.21.15.16:0/1824145571 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.520+0000 7f867a7fc640 1 --2- 172.21.15.16:0/1824145571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f867c1425e0 0x7f867c158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.520+0000 7f867affd640 1 --2- 172.21.15.16:0/1824145571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f867c06cd20 0x7f867c15ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.520+0000 7f867b7fe640 1 -- 172.21.15.16:0/1824145571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f867c06cd20 msgr2=0x7f867c15ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.520+0000 7f867b7fe640 1 --2- 172.21.15.16:0/1824145571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f867c06cd20 0x7f867c15ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.520+0000 7f867b7fe640 1 -- 172.21.15.16:0/1824145571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f867c1425e0 msgr2=0x7f867c158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.520+0000 7f867b7fe640 1 --2- 172.21.15.16:0/1824145571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f867c1425e0 0x7f867c158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.520+0000 7f867b7fe640 1 -- 172.21.15.16:0/1824145571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867c0ccd90 con 0x7f867c161eb0 2024-09-19T07:43:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.520+0000 7f867b7fe640 1 --2- 172.21.15.16:0/1824145571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c161eb0 0x7f867c0d1c90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f866c00c790 tx=0x7f866c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.521+0000 7f86737fe640 1 -- 172.21.15.16:0/1824145571 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c003b70 con 0x7f867c161eb0 2024-09-19T07:43:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.521+0000 7f86737fe640 1 -- 172.21.15.16:0/1824145571 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f866c003d10 con 0x7f867c161eb0 2024-09-19T07:43:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.521+0000 7f868176f640 1 -- 172.21.15.16:0/1824145571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867c0cd070 con 0x7f867c161eb0 2024-09-19T07:43:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.521+0000 7f86737fe640 1 -- 172.21.15.16:0/1824145571 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c00aa50 con 0x7f867c161eb0 2024-09-19T07:43:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.521+0000 7f868176f640 1 -- 172.21.15.16:0/1824145571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f867c0cd630 con 0x7f867c161eb0 2024-09-19T07:43:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.522+0000 7f868176f640 1 -- 172.21.15.16:0/1824145571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f867c162b70 con 0x7f867c161eb0 2024-09-19T07:43:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.522+0000 7f86737fe640 1 -- 172.21.15.16:0/1824145571 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f866c00abf0 con 0x7f867c161eb0 2024-09-19T07:43:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.522+0000 7f86737fe640 1 --2- 172.21.15.16:0/1824145571 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f865c06c2b0 0x7f865c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.523+0000 7f86737fe640 1 -- 172.21.15.16:0/1824145571 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f866c08b780 con 0x7f867c161eb0 2024-09-19T07:43:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.523+0000 7f867affd640 1 --2- 172.21.15.16:0/1824145571 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f865c06c2b0 0x7f865c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:30.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.523+0000 7f867affd640 1 --2- 172.21.15.16:0/1824145571 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f865c06c2b0 0x7f865c06e770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8674034000 tx=0x7f867402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:30.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.527+0000 7f86737fe640 1 -- 172.21.15.16:0/1824145571 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f866c059b10 con 0x7f867c161eb0 2024-09-19T07:43:30.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.700+0000 7f868176f640 1 -- 172.21.15.16:0/1824145571 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f867c0690c0 con 0x7f865c06c2b0 2024-09-19T07:43:30.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.709+0000 7f86737fe640 1 -- 172.21.15.16:0/1824145571 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f867c0690c0 con 0x7f865c06c2b0 2024-09-19T07:43:30.714 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:30.715 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:43:30.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.712+0000 7f868176f640 1 -- 172.21.15.16:0/1824145571 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f865c06c2b0 msgr2=0x7f865c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.712+0000 7f868176f640 1 --2- 172.21.15.16:0/1824145571 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f865c06c2b0 0x7f865c06e770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8674034000 tx=0x7f867402e000 comp rx=0 tx=0).stop 2024-09-19T07:43:30.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.712+0000 7f868176f640 1 -- 172.21.15.16:0/1824145571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c161eb0 msgr2=0x7f867c0d1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.712+0000 7f868176f640 1 --2- 172.21.15.16:0/1824145571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c161eb0 0x7f867c0d1c90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f866c00c790 tx=0x7f866c00cc50 comp rx=0 tx=0).stop 2024-09-19T07:43:30.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.712+0000 7f868176f640 1 -- 172.21.15.16:0/1824145571 shutdown_connections 2024-09-19T07:43:30.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.712+0000 7f868176f640 1 --2- 172.21.15.16:0/1824145571 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f865c06c2b0 0x7f865c06e770 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.712+0000 7f868176f640 1 --2- 172.21.15.16:0/1824145571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c161eb0 0x7f867c0d1c90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.712+0000 7f868176f640 1 --2- 172.21.15.16:0/1824145571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f867c1425e0 0x7f867c158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.712+0000 7f868176f640 1 --2- 172.21.15.16:0/1824145571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f867c06cd20 0x7f867c15ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.712+0000 7f868176f640 1 -- 172.21.15.16:0/1824145571 >> 172.21.15.16:0/1824145571 conn(0x7f867c0b3ed0 msgr2=0x7f867c1603c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:30.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.712+0000 7f868176f640 1 -- 172.21.15.16:0/1824145571 shutdown_connections 2024-09-19T07:43:30.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.712+0000 7f868176f640 1 -- 172.21.15.16:0/1824145571 wait complete. 2024-09-19T07:43:30.723 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-09-19T07:43:30.463820+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236216,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738504,"statfs":{"total":773094113280,"available":772852228096,"internally_reserved":0,"allocated":3416064,"data_stored":2041588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003525"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656052+0000","last_change":"2024-09-19T07:43:05.265202+0000","last_active":"2024-09-19T07:43:12.656052+0000","last_peered":"2024-09-19T07:43:12.656052+0000","last_clean":"2024-09-19T07:43:12.656052+0000","last_became_active":"2024-09-19T07:43:05.264871+0000","last_became_peered":"2024-09-19T07:43:05.264871+0000","last_unstale":"2024-09-19T07:43:12.656052+0000","last_undegraded":"2024-09-19T07:43:12.656052+0000","last_fullsized":"2024-09-19T07:43:12.656052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:36:39.982941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310781+0000","last_change":"2024-09-19T07:43:06.264003+0000","last_active":"2024-09-19T07:43:11.310781+0000","last_peered":"2024-09-19T07:43:11.310781+0000","last_clean":"2024-09-19T07:43:11.310781+0000","last_became_active":"2024-09-19T07:43:06.262313+0000","last_became_peered":"2024-09-19T07:43:06.262313+0000","last_unstale":"2024-09-19T07:43:11.310781+0000","last_undegraded":"2024-09-19T07:43:11.310781+0000","last_fullsized":"2024-09-19T07:43:11.310781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.309855+0000","last_change":"2024-09-19T07:43:10.483058+0000","last_active":"2024-09-19T07:43:11.309855+0000","last_peered":"2024-09-19T07:43:11.309855+0000","last_clean":"2024-09-19T07:43:11.309855+0000","last_became_active":"2024-09-19T07:43:10.482621+0000","last_became_peered":"2024-09-19T07:43:10.482621+0000","last_unstale":"2024-09-19T07:43:11.309855+0000","last_undegraded":"2024-09-19T07:43:11.309855+0000","last_fullsized":"2024-09-19T07:43:11.309855+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:27.168500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310785+0000","last_change":"2024-09-19T07:43:05.265029+0000","last_active":"2024-09-19T07:43:11.310785+0000","last_peered":"2024-09-19T07:43:11.310785+0000","last_clean":"2024-09-19T07:43:11.310785+0000","last_became_active":"2024-09-19T07:43:05.264529+0000","last_became_peered":"2024-09-19T07:43:05.264529+0000","last_unstale":"2024-09-19T07:43:11.310785+0000","last_undegraded":"2024-09-19T07:43:11.310785+0000","last_fullsized":"2024-09-19T07:43:11.310785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:12.998698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655877+0000","last_change":"2024-09-19T07:43:06.272531+0000","last_active":"2024-09-19T07:43:12.655877+0000","last_peered":"2024-09-19T07:43:12.655877+0000","last_clean":"2024-09-19T07:43:12.655877+0000","last_became_active":"2024-09-19T07:43:06.272225+0000","last_became_peered":"2024-09-19T07:43:06.272225+0000","last_unstale":"2024-09-19T07:43:12.655877+0000","last_undegraded":"2024-09-19T07:43:12.655877+0000","last_fullsized":"2024-09-19T07:43:12.655877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:06.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311381+0000","last_change":"2024-09-19T07:43:10.288276+0000","last_active":"2024-09-19T07:43:11.311381+0000","last_peered":"2024-09-19T07:43:11.311381+0000","last_clean":"2024-09-19T07:43:11.311381+0000","last_became_active":"2024-09-19T07:43:10.287816+0000","last_became_peered":"2024-09-19T07:43:10.287816+0000","last_unstale":"2024-09-19T07:43:11.311381+0000","last_undegraded":"2024-09-19T07:43:11.311381+0000","last_fullsized":"2024-09-19T07:43:11.311381+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:26.830106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681715+0000","last_change":"2024-09-19T07:43:05.266982+0000","last_active":"2024-09-19T07:43:12.681715+0000","last_peered":"2024-09-19T07:43:12.681715+0000","last_clean":"2024-09-19T07:43:12.681715+0000","last_became_active":"2024-09-19T07:43:05.266667+0000","last_became_peered":"2024-09-19T07:43:05.266667+0000","last_unstale":"2024-09-19T07:43:12.681715+0000","last_undegraded":"2024-09-19T07:43:12.681715+0000","last_fullsized":"2024-09-19T07:43:12.681715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:02.488778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312020+0000","last_change":"2024-09-19T07:43:06.273456+0000","last_active":"2024-09-19T07:43:11.312020+0000","last_peered":"2024-09-19T07:43:11.312020+0000","last_clean":"2024-09-19T07:43:11.312020+0000","last_became_active":"2024-09-19T07:43:06.273046+0000","last_became_peered":"2024-09-19T07:43:06.273046+0000","last_unstale":"2024-09-19T07:43:11.312020+0000","last_undegraded":"2024-09-19T07:43:11.312020+0000","last_fullsized":"2024-09-19T07:43:11.312020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:09.222719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310977+0000","last_change":"2024-09-19T07:43:10.293267+0000","last_active":"2024-09-19T07:43:11.310977+0000","last_peered":"2024-09-19T07:43:11.310977+0000","last_clean":"2024-09-19T07:43:11.310977+0000","last_became_active":"2024-09-19T07:43:10.292870+0000","last_became_peered":"2024-09-19T07:43:10.292870+0000","last_unstale":"2024-09-19T07:43:11.310977+0000","last_undegraded":"2024-09-19T07:43:11.310977+0000","last_fullsized":"2024-09-19T07:43:11.310977+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:54.570005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681497+0000","last_change":"2024-09-19T07:43:05.267905+0000","last_active":"2024-09-19T07:43:12.681497+0000","last_peered":"2024-09-19T07:43:12.681497+0000","last_clean":"2024-09-19T07:43:12.681497+0000","last_became_active":"2024-09-19T07:43:05.267563+0000","last_became_peered":"2024-09-19T07:43:05.267563+0000","last_unstale":"2024-09-19T07:43:12.681497+0000","last_undegraded":"2024-09-19T07:43:12.681497+0000","last_fullsized":"2024-09-19T07:43:12.681497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:47.531702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311784+0000","last_change":"2024-09-19T07:43:06.262966+0000","last_active":"2024-09-19T07:43:11.311784+0000","last_peered":"2024-09-19T07:43:11.311784+0000","last_clean":"2024-09-19T07:43:11.311784+0000","last_became_active":"2024-09-19T07:43:06.262698+0000","last_became_peered":"2024-09-19T07:43:06.262698+0000","last_unstale":"2024-09-19T07:43:11.311784+0000","last_undegraded":"2024-09-19T07:43:11.311784+0000","last_fullsized":"2024-09-19T07:43:11.311784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:47.918580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310630+0000","last_change":"2024-09-19T07:43:10.287084+0000","last_active":"2024-09-19T07:43:11.310630+0000","last_peered":"2024-09-19T07:43:11.310630+0000","last_clean":"2024-09-19T07:43:11.310630+0000","last_became_active":"2024-09-19T07:43:10.286693+0000","last_became_peered":"2024-09-19T07:43:10.286693+0000","last_unstale":"2024-09-19T07:43:11.310630+0000","last_undegraded":"2024-09-19T07:43:11.310630+0000","last_fullsized":"2024-09-19T07:43:11.310630+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:17.611361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450546+0000","last_change":"2024-09-19T07:43:06.262664+0000","last_active":"2024-09-19T07:43:11.450546+0000","last_peered":"2024-09-19T07:43:11.450546+0000","last_clean":"2024-09-19T07:43:11.450546+0000","last_became_active":"2024-09-19T07:43:06.262389+0000","last_became_peered":"2024-09-19T07:43:06.262389+0000","last_unstale":"2024-09-19T07:43:11.450546+0000","last_undegraded":"2024-09-19T07:43:11.450546+0000","last_fullsized":"2024-09-19T07:43:11.450546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:06.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681454+0000","last_change":"2024-09-19T07:43:05.262431+0000","last_active":"2024-09-19T07:43:12.681454+0000","last_peered":"2024-09-19T07:43:12.681454+0000","last_clean":"2024-09-19T07:43:12.681454+0000","last_became_active":"2024-09-19T07:43:05.262156+0000","last_became_peered":"2024-09-19T07:43:05.262156+0000","last_unstale":"2024-09-19T07:43:12.681454+0000","last_undegraded":"2024-09-19T07:43:12.681454+0000","last_fullsized":"2024-09-19T07:43:12.681454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:27.338560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312544+0000","last_change":"2024-09-19T07:43:10.287687+0000","last_active":"2024-09-19T07:43:11.312544+0000","last_peered":"2024-09-19T07:43:11.312544+0000","last_clean":"2024-09-19T07:43:11.312544+0000","last_became_active":"2024-09-19T07:43:10.287301+0000","last_became_peered":"2024-09-19T07:43:10.287301+0000","last_unstale":"2024-09-19T07:43:11.312544+0000","last_undegraded":"2024-09-19T07:43:11.312544+0000","last_fullsized":"2024-09-19T07:43:11.312544+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:43.684480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310291+0000","last_change":"2024-09-19T07:43:06.274918+0000","last_active":"2024-09-19T07:43:11.310291+0000","last_peered":"2024-09-19T07:43:11.310291+0000","last_clean":"2024-09-19T07:43:11.310291+0000","last_became_active":"2024-09-19T07:43:06.274639+0000","last_became_peered":"2024-09-19T07:43:06.274639+0000","last_unstale":"2024-09-19T07:43:11.310291+0000","last_undegraded":"2024-09-19T07:43:11.310291+0000","last_fullsized":"2024-09-19T07:43:11.310291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:22.804370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680919+0000","last_change":"2024-09-19T07:43:05.258127+0000","last_active":"2024-09-19T07:43:12.680919+0000","last_peered":"2024-09-19T07:43:12.680919+0000","last_clean":"2024-09-19T07:43:12.680919+0000","last_became_active":"2024-09-19T07:43:05.257766+0000","last_became_peered":"2024-09-19T07:43:05.257766+0000","last_unstale":"2024-09-19T07:43:12.680919+0000","last_undegraded":"2024-09-19T07:43:12.680919+0000","last_fullsized":"2024-09-19T07:43:12.680919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:02.488778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311080+0000","last_change":"2024-09-19T07:43:10.294352+0000","last_active":"2024-09-19T07:43:11.311080+0000","last_peered":"2024-09-19T07:43:11.311080+0000","last_clean":"2024-09-19T07:43:11.311080+0000","last_became_active":"2024-09-19T07:43:10.293957+0000","last_became_peered":"2024-09-19T07:43:10.293957+0000","last_unstale":"2024-09-19T07:43:11.311080+0000","last_undegraded":"2024-09-19T07:43:11.311080+0000","last_fullsized":"2024-09-19T07:43:11.311080+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:13.248454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310160+0000","last_change":"2024-09-19T07:43:06.256705+0000","last_active":"2024-09-19T07:43:11.310160+0000","last_peered":"2024-09-19T07:43:11.310160+0000","last_clean":"2024-09-19T07:43:11.310160+0000","last_became_active":"2024-09-19T07:43:06.256423+0000","last_became_peered":"2024-09-19T07:43:06.256423+0000","last_unstale":"2024-09-19T07:43:11.310160+0000","last_undegraded":"2024-09-19T07:43:11.310160+0000","last_fullsized":"2024-09-19T07:43:11.310160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:47.918580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680712+0000","last_change":"2024-09-19T07:43:05.260367+0000","last_active":"2024-09-19T07:43:12.680712+0000","last_peered":"2024-09-19T07:43:12.680712+0000","last_clean":"2024-09-19T07:43:12.680712+0000","last_became_active":"2024-09-19T07:43:05.260056+0000","last_became_peered":"2024-09-19T07:43:05.260056+0000","last_unstale":"2024-09-19T07:43:12.680712+0000","last_undegraded":"2024-09-19T07:43:12.680712+0000","last_fullsized":"2024-09-19T07:43:12.680712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:38:29.212025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681342+0000","last_change":"2024-09-19T07:43:10.482020+0000","last_active":"2024-09-19T07:43:12.681342+0000","last_peered":"2024-09-19T07:43:12.681342+0000","last_clean":"2024-09-19T07:43:12.681342+0000","last_became_active":"2024-09-19T07:43:10.480883+0000","last_became_peered":"2024-09-19T07:43:10.480883+0000","last_unstale":"2024-09-19T07:43:12.681342+0000","last_undegraded":"2024-09-19T07:43:12.681342+0000","last_fullsized":"2024-09-19T07:43:12.681342+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:01.954262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312259+0000","last_change":"2024-09-19T07:43:06.268719+0000","last_active":"2024-09-19T07:43:11.312259+0000","last_peered":"2024-09-19T07:43:11.312259+0000","last_clean":"2024-09-19T07:43:11.312259+0000","last_became_active":"2024-09-19T07:43:06.268405+0000","last_became_peered":"2024-09-19T07:43:06.268405+0000","last_unstale":"2024-09-19T07:43:11.312259+0000","last_undegraded":"2024-09-19T07:43:11.312259+0000","last_fullsized":"2024-09-19T07:43:11.312259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:45.364876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681025+0000","last_change":"2024-09-19T07:43:05.267962+0000","last_active":"2024-09-19T07:43:12.681025+0000","last_peered":"2024-09-19T07:43:12.681025+0000","last_clean":"2024-09-19T07:43:12.681025+0000","last_became_active":"2024-09-19T07:43:05.267670+0000","last_became_peered":"2024-09-19T07:43:05.267670+0000","last_unstale":"2024-09-19T07:43:12.681025+0000","last_undegraded":"2024-09-19T07:43:12.681025+0000","last_fullsized":"2024-09-19T07:43:12.681025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:13.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312584+0000","last_change":"2024-09-19T07:43:10.289357+0000","last_active":"2024-09-19T07:43:11.312584+0000","last_peered":"2024-09-19T07:43:11.312584+0000","last_clean":"2024-09-19T07:43:11.312584+0000","last_became_active":"2024-09-19T07:43:10.288596+0000","last_became_peered":"2024-09-19T07:43:10.288596+0000","last_unstale":"2024-09-19T07:43:11.312584+0000","last_undegraded":"2024-09-19T07:43:11.312584+0000","last_fullsized":"2024-09-19T07:43:11.312584+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:08.193959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656239+0000","last_change":"2024-09-19T07:43:06.270766+0000","last_active":"2024-09-19T07:43:12.656239+0000","last_peered":"2024-09-19T07:43:12.656239+0000","last_clean":"2024-09-19T07:43:12.656239+0000","last_became_active":"2024-09-19T07:43:06.270494+0000","last_became_peered":"2024-09-19T07:43:06.270494+0000","last_unstale":"2024-09-19T07:43:12.656239+0000","last_undegraded":"2024-09-19T07:43:12.656239+0000","last_fullsized":"2024-09-19T07:43:12.656239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451242+0000","last_change":"2024-09-19T07:43:05.263058+0000","last_active":"2024-09-19T07:43:11.451242+0000","last_peered":"2024-09-19T07:43:11.451242+0000","last_clean":"2024-09-19T07:43:11.451242+0000","last_became_active":"2024-09-19T07:43:05.262762+0000","last_became_peered":"2024-09-19T07:43:05.262762+0000","last_unstale":"2024-09-19T07:43:11.451242+0000","last_undegraded":"2024-09-19T07:43:11.451242+0000","last_fullsized":"2024-09-19T07:43:11.451242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:38:29.212025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312088+0000","last_change":"2024-09-19T07:43:10.482293+0000","last_active":"2024-09-19T07:43:11.312088+0000","last_peered":"2024-09-19T07:43:11.312088+0000","last_clean":"2024-09-19T07:43:11.312088+0000","last_became_active":"2024-09-19T07:43:10.481268+0000","last_became_peered":"2024-09-19T07:43:10.481268+0000","last_unstale":"2024-09-19T07:43:11.312088+0000","last_undegraded":"2024-09-19T07:43:11.312088+0000","last_fullsized":"2024-09-19T07:43:11.312088+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:51.944316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311136+0000","last_change":"2024-09-19T07:43:06.268026+0000","last_active":"2024-09-19T07:43:11.311136+0000","last_peered":"2024-09-19T07:43:11.311136+0000","last_clean":"2024-09-19T07:43:11.311136+0000","last_became_active":"2024-09-19T07:43:06.267713+0000","last_became_peered":"2024-09-19T07:43:06.267713+0000","last_unstale":"2024-09-19T07:43:11.311136+0000","last_undegraded":"2024-09-19T07:43:11.311136+0000","last_fullsized":"2024-09-19T07:43:11.311136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:56.380341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681209+0000","last_change":"2024-09-19T07:43:05.263248+0000","last_active":"2024-09-19T07:43:12.681209+0000","last_peered":"2024-09-19T07:43:12.681209+0000","last_clean":"2024-09-19T07:43:12.681209+0000","last_became_active":"2024-09-19T07:43:05.262892+0000","last_became_peered":"2024-09-19T07:43:05.262892+0000","last_unstale":"2024-09-19T07:43:12.681209+0000","last_undegraded":"2024-09-19T07:43:12.681209+0000","last_fullsized":"2024-09-19T07:43:12.681209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:47.531702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.309826+0000","last_change":"2024-09-19T07:43:10.482975+0000","last_active":"2024-09-19T07:43:11.309826+0000","last_peered":"2024-09-19T07:43:11.309826+0000","last_clean":"2024-09-19T07:43:11.309826+0000","last_became_active":"2024-09-19T07:43:10.482584+0000","last_became_peered":"2024-09-19T07:43:10.482584+0000","last_unstale":"2024-09-19T07:43:11.309826+0000","last_undegraded":"2024-09-19T07:43:11.309826+0000","last_fullsized":"2024-09-19T07:43:11.309826+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:10.437589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680841+0000","last_change":"2024-09-19T07:43:06.258826+0000","last_active":"2024-09-19T07:43:12.680841+0000","last_peered":"2024-09-19T07:43:12.680841+0000","last_clean":"2024-09-19T07:43:12.680841+0000","last_became_active":"2024-09-19T07:43:06.258518+0000","last_became_peered":"2024-09-19T07:43:06.258518+0000","last_unstale":"2024-09-19T07:43:12.680841+0000","last_undegraded":"2024-09-19T07:43:12.680841+0000","last_fullsized":"2024-09-19T07:43:12.680841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681832+0000","last_change":"2024-09-19T07:43:05.267167+0000","last_active":"2024-09-19T07:43:12.681832+0000","last_peered":"2024-09-19T07:43:12.681832+0000","last_clean":"2024-09-19T07:43:12.681832+0000","last_became_active":"2024-09-19T07:43:05.266892+0000","last_became_peered":"2024-09-19T07:43:05.266892+0000","last_unstale":"2024-09-19T07:43:12.681832+0000","last_undegraded":"2024-09-19T07:43:12.681832+0000","last_fullsized":"2024-09-19T07:43:12.681832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:06.889703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655769+0000","last_change":"2024-09-19T07:43:10.480657+0000","last_active":"2024-09-19T07:43:12.655769+0000","last_peered":"2024-09-19T07:43:12.655769+0000","last_clean":"2024-09-19T07:43:12.655769+0000","last_became_active":"2024-09-19T07:43:10.480186+0000","last_became_peered":"2024-09-19T07:43:10.480186+0000","last_unstale":"2024-09-19T07:43:12.655769+0000","last_undegraded":"2024-09-19T07:43:12.655769+0000","last_fullsized":"2024-09-19T07:43:12.655769+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:43.490603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311921+0000","last_change":"2024-09-19T07:43:06.265266+0000","last_active":"2024-09-19T07:43:11.311921+0000","last_peered":"2024-09-19T07:43:11.311921+0000","last_clean":"2024-09-19T07:43:11.311921+0000","last_became_active":"2024-09-19T07:43:06.265001+0000","last_became_peered":"2024-09-19T07:43:06.265001+0000","last_unstale":"2024-09-19T07:43:11.311921+0000","last_undegraded":"2024-09-19T07:43:11.311921+0000","last_fullsized":"2024-09-19T07:43:11.311921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:39.464868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680965+0000","last_change":"2024-09-19T07:43:05.259114+0000","last_active":"2024-09-19T07:43:12.680965+0000","last_peered":"2024-09-19T07:43:12.680965+0000","last_clean":"2024-09-19T07:43:12.680965+0000","last_became_active":"2024-09-19T07:43:05.258794+0000","last_became_peered":"2024-09-19T07:43:05.258794+0000","last_unstale":"2024-09-19T07:43:12.680965+0000","last_undegraded":"2024-09-19T07:43:12.680965+0000","last_fullsized":"2024-09-19T07:43:12.680965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:13.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680778+0000","last_change":"2024-09-19T07:43:10.295795+0000","last_active":"2024-09-19T07:43:12.680778+0000","last_peered":"2024-09-19T07:43:12.680778+0000","last_clean":"2024-09-19T07:43:12.680778+0000","last_became_active":"2024-09-19T07:43:10.295379+0000","last_became_peered":"2024-09-19T07:43:10.295379+0000","last_unstale":"2024-09-19T07:43:12.680778+0000","last_undegraded":"2024-09-19T07:43:12.680778+0000","last_fullsized":"2024-09-19T07:43:12.680778+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:43.490603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311661+0000","last_change":"2024-09-19T07:43:06.271554+0000","last_active":"2024-09-19T07:43:11.311661+0000","last_peered":"2024-09-19T07:43:11.311661+0000","last_clean":"2024-09-19T07:43:11.311661+0000","last_became_active":"2024-09-19T07:43:06.271304+0000","last_became_peered":"2024-09-19T07:43:06.271304+0000","last_unstale":"2024-09-19T07:43:11.311661+0000","last_undegraded":"2024-09-19T07:43:11.311661+0000","last_fullsized":"2024-09-19T07:43:11.311661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:03.494318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311677+0000","last_change":"2024-09-19T07:43:05.264548+0000","last_active":"2024-09-19T07:43:11.311677+0000","last_peered":"2024-09-19T07:43:11.311677+0000","last_clean":"2024-09-19T07:43:11.311677+0000","last_became_active":"2024-09-19T07:43:05.261674+0000","last_became_peered":"2024-09-19T07:43:05.261674+0000","last_unstale":"2024-09-19T07:43:11.311677+0000","last_undegraded":"2024-09-19T07:43:11.311677+0000","last_fullsized":"2024-09-19T07:43:11.311677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:47.531702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681063+0000","last_change":"2024-09-19T07:43:10.288751+0000","last_active":"2024-09-19T07:43:12.681063+0000","last_peered":"2024-09-19T07:43:12.681063+0000","last_clean":"2024-09-19T07:43:12.681063+0000","last_became_active":"2024-09-19T07:43:10.288383+0000","last_became_peered":"2024-09-19T07:43:10.288383+0000","last_unstale":"2024-09-19T07:43:12.681063+0000","last_undegraded":"2024-09-19T07:43:12.681063+0000","last_fullsized":"2024-09-19T07:43:12.681063+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:43.944005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680858+0000","last_change":"2024-09-19T07:43:06.264468+0000","last_active":"2024-09-19T07:43:12.680858+0000","last_peered":"2024-09-19T07:43:12.680858+0000","last_clean":"2024-09-19T07:43:12.680858+0000","last_became_active":"2024-09-19T07:43:06.263801+0000","last_became_peered":"2024-09-19T07:43:06.263801+0000","last_unstale":"2024-09-19T07:43:12.680858+0000","last_undegraded":"2024-09-19T07:43:12.680858+0000","last_fullsized":"2024-09-19T07:43:12.680858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:06.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588628+0000","last_change":"2024-09-19T07:43:05.259922+0000","last_active":"2024-09-19T07:43:11.588628+0000","last_peered":"2024-09-19T07:43:11.588628+0000","last_clean":"2024-09-19T07:43:11.588628+0000","last_became_active":"2024-09-19T07:43:05.259552+0000","last_became_peered":"2024-09-19T07:43:05.259552+0000","last_unstale":"2024-09-19T07:43:11.588628+0000","last_undegraded":"2024-09-19T07:43:11.588628+0000","last_fullsized":"2024-09-19T07:43:11.588628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:55.737753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681498+0000","last_change":"2024-09-19T07:43:10.296047+0000","last_active":"2024-09-19T07:43:12.681498+0000","last_peered":"2024-09-19T07:43:12.681498+0000","last_clean":"2024-09-19T07:43:12.681498+0000","last_became_active":"2024-09-19T07:43:10.295636+0000","last_became_peered":"2024-09-19T07:43:10.295636+0000","last_unstale":"2024-09-19T07:43:12.681498+0000","last_undegraded":"2024-09-19T07:43:12.681498+0000","last_fullsized":"2024-09-19T07:43:12.681498+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:27.138783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655596+0000","last_change":"2024-09-19T07:43:06.273785+0000","last_active":"2024-09-19T07:43:12.655596+0000","last_peered":"2024-09-19T07:43:12.655596+0000","last_clean":"2024-09-19T07:43:12.655596+0000","last_became_active":"2024-09-19T07:43:06.273514+0000","last_became_peered":"2024-09-19T07:43:06.273514+0000","last_unstale":"2024-09-19T07:43:12.655596+0000","last_undegraded":"2024-09-19T07:43:12.655596+0000","last_fullsized":"2024-09-19T07:43:12.655596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:09.399092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681059+0000","last_change":"2024-09-19T07:43:05.263952+0000","last_active":"2024-09-19T07:43:12.681059+0000","last_peered":"2024-09-19T07:43:12.681059+0000","last_clean":"2024-09-19T07:43:12.681059+0000","last_became_active":"2024-09-19T07:43:05.263677+0000","last_became_peered":"2024-09-19T07:43:05.263677+0000","last_unstale":"2024-09-19T07:43:12.681059+0000","last_undegraded":"2024-09-19T07:43:12.681059+0000","last_fullsized":"2024-09-19T07:43:12.681059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:44.359336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656084+0000","last_change":"2024-09-19T07:43:10.480721+0000","last_active":"2024-09-19T07:43:12.656084+0000","last_peered":"2024-09-19T07:43:12.656084+0000","last_clean":"2024-09-19T07:43:12.656084+0000","last_became_active":"2024-09-19T07:43:10.480321+0000","last_became_peered":"2024-09-19T07:43:10.480321+0000","last_unstale":"2024-09-19T07:43:12.656084+0000","last_undegraded":"2024-09-19T07:43:12.656084+0000","last_fullsized":"2024-09-19T07:43:12.656084+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:01.954262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310005+0000","last_change":"2024-09-19T07:43:06.271482+0000","last_active":"2024-09-19T07:43:11.310005+0000","last_peered":"2024-09-19T07:43:11.310005+0000","last_clean":"2024-09-19T07:43:11.310005+0000","last_became_active":"2024-09-19T07:43:06.271212+0000","last_became_peered":"2024-09-19T07:43:06.271212+0000","last_unstale":"2024-09-19T07:43:11.310005+0000","last_undegraded":"2024-09-19T07:43:11.310005+0000","last_fullsized":"2024-09-19T07:43:11.310005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:27.185190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311209+0000","last_change":"2024-09-19T07:43:05.261378+0000","last_active":"2024-09-19T07:43:11.311209+0000","last_peered":"2024-09-19T07:43:11.311209+0000","last_clean":"2024-09-19T07:43:11.311209+0000","last_became_active":"2024-09-19T07:43:05.261055+0000","last_became_peered":"2024-09-19T07:43:05.261055+0000","last_unstale":"2024-09-19T07:43:11.311209+0000","last_undegraded":"2024-09-19T07:43:11.311209+0000","last_fullsized":"2024-09-19T07:43:11.311209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:12.998698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681162+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:43:12.681162+0000","last_peered":"2024-09-19T07:43:12.681162+0000","last_clean":"2024-09-19T07:43:12.681162+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:43:12.681162+0000","last_undegraded":"2024-09-19T07:43:12.681162+0000","last_fullsized":"2024-09-19T07:43:12.681162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450864+0000","last_change":"2024-09-19T07:43:10.481567+0000","last_active":"2024-09-19T07:43:11.450864+0000","last_peered":"2024-09-19T07:43:11.450864+0000","last_clean":"2024-09-19T07:43:11.450864+0000","last_became_active":"2024-09-19T07:43:10.481210+0000","last_became_peered":"2024-09-19T07:43:10.481210+0000","last_unstale":"2024-09-19T07:43:11.450864+0000","last_undegraded":"2024-09-19T07:43:11.450864+0000","last_fullsized":"2024-09-19T07:43:11.450864+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:08.193959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681552+0000","last_change":"2024-09-19T07:43:06.268655+0000","last_active":"2024-09-19T07:43:12.681552+0000","last_peered":"2024-09-19T07:43:12.681552+0000","last_clean":"2024-09-19T07:43:12.681552+0000","last_became_active":"2024-09-19T07:43:06.268387+0000","last_became_peered":"2024-09-19T07:43:06.268387+0000","last_unstale":"2024-09-19T07:43:12.681552+0000","last_undegraded":"2024-09-19T07:43:12.681552+0000","last_fullsized":"2024-09-19T07:43:12.681552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:14.347245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588993+0000","last_change":"2024-09-19T07:43:05.261374+0000","last_active":"2024-09-19T07:43:11.588993+0000","last_peered":"2024-09-19T07:43:11.588993+0000","last_clean":"2024-09-19T07:43:11.588993+0000","last_became_active":"2024-09-19T07:43:05.261088+0000","last_became_peered":"2024-09-19T07:43:05.261088+0000","last_unstale":"2024-09-19T07:43:11.588993+0000","last_undegraded":"2024-09-19T07:43:11.588993+0000","last_fullsized":"2024-09-19T07:43:11.588993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:39.630700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450612+0000","last_change":"2024-09-19T07:43:10.480274+0000","last_active":"2024-09-19T07:43:11.450612+0000","last_peered":"2024-09-19T07:43:11.450612+0000","last_clean":"2024-09-19T07:43:11.450612+0000","last_became_active":"2024-09-19T07:43:10.479848+0000","last_became_peered":"2024-09-19T07:43:10.479848+0000","last_unstale":"2024-09-19T07:43:11.450612+0000","last_undegraded":"2024-09-19T07:43:11.450612+0000","last_fullsized":"2024-09-19T07:43:11.450612+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:43.684480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312126+0000","last_change":"2024-09-19T07:43:06.264529+0000","last_active":"2024-09-19T07:43:11.312126+0000","last_peered":"2024-09-19T07:43:11.312126+0000","last_clean":"2024-09-19T07:43:11.312126+0000","last_became_active":"2024-09-19T07:43:06.264270+0000","last_became_peered":"2024-09-19T07:43:06.264270+0000","last_unstale":"2024-09-19T07:43:11.312126+0000","last_undegraded":"2024-09-19T07:43:11.312126+0000","last_fullsized":"2024-09-19T07:43:11.312126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:04.168224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681259+0000","last_change":"2024-09-19T07:43:05.254996+0000","last_active":"2024-09-19T07:43:12.681259+0000","last_peered":"2024-09-19T07:43:12.681259+0000","last_clean":"2024-09-19T07:43:12.681259+0000","last_became_active":"2024-09-19T07:43:05.254430+0000","last_became_peered":"2024-09-19T07:43:05.254430+0000","last_unstale":"2024-09-19T07:43:12.681259+0000","last_undegraded":"2024-09-19T07:43:12.681259+0000","last_fullsized":"2024-09-19T07:43:12.681259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:06.889703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656122+0000","last_change":"2024-09-19T07:43:10.295667+0000","last_active":"2024-09-19T07:43:12.656122+0000","last_peered":"2024-09-19T07:43:12.656122+0000","last_clean":"2024-09-19T07:43:12.656122+0000","last_became_active":"2024-09-19T07:43:10.295241+0000","last_became_peered":"2024-09-19T07:43:10.295241+0000","last_unstale":"2024-09-19T07:43:12.656122+0000","last_undegraded":"2024-09-19T07:43:12.656122+0000","last_fullsized":"2024-09-19T07:43:12.656122+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:27.138783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681804+0000","last_change":"2024-09-19T07:43:06.275720+0000","last_active":"2024-09-19T07:43:12.681804+0000","last_peered":"2024-09-19T07:43:12.681804+0000","last_clean":"2024-09-19T07:43:12.681804+0000","last_became_active":"2024-09-19T07:43:06.275403+0000","last_became_peered":"2024-09-19T07:43:06.275403+0000","last_unstale":"2024-09-19T07:43:12.681804+0000","last_undegraded":"2024-09-19T07:43:12.681804+0000","last_fullsized":"2024-09-19T07:43:12.681804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:39.918269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588845+0000","last_change":"2024-09-19T07:43:05.260217+0000","last_active":"2024-09-19T07:43:11.588845+0000","last_peered":"2024-09-19T07:43:11.588845+0000","last_clean":"2024-09-19T07:43:11.588845+0000","last_became_active":"2024-09-19T07:43:05.259482+0000","last_became_peered":"2024-09-19T07:43:05.259482+0000","last_unstale":"2024-09-19T07:43:11.588845+0000","last_undegraded":"2024-09-19T07:43:11.588845+0000","last_fullsized":"2024-09-19T07:43:11.588845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:56.644264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310513+0000","last_change":"2024-09-19T07:43:10.480657+0000","last_active":"2024-09-19T07:43:11.310513+0000","last_peered":"2024-09-19T07:43:11.310513+0000","last_clean":"2024-09-19T07:43:11.310513+0000","last_became_active":"2024-09-19T07:43:10.480279+0000","last_became_peered":"2024-09-19T07:43:10.480279+0000","last_unstale":"2024-09-19T07:43:11.310513+0000","last_undegraded":"2024-09-19T07:43:11.310513+0000","last_fullsized":"2024-09-19T07:43:11.310513+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:26.933433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681655+0000","last_change":"2024-09-19T07:43:06.275354+0000","last_active":"2024-09-19T07:43:12.681655+0000","last_peered":"2024-09-19T07:43:12.681655+0000","last_clean":"2024-09-19T07:43:12.681655+0000","last_became_active":"2024-09-19T07:43:06.275039+0000","last_became_peered":"2024-09-19T07:43:06.275039+0000","last_unstale":"2024-09-19T07:43:12.681655+0000","last_undegraded":"2024-09-19T07:43:12.681655+0000","last_fullsized":"2024-09-19T07:43:12.681655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:47.918580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311619+0000","last_change":"2024-09-19T07:43:05.265977+0000","last_active":"2024-09-19T07:43:11.311619+0000","last_peered":"2024-09-19T07:43:11.311619+0000","last_clean":"2024-09-19T07:43:11.311619+0000","last_became_active":"2024-09-19T07:43:05.265615+0000","last_became_peered":"2024-09-19T07:43:05.265615+0000","last_unstale":"2024-09-19T07:43:11.311619+0000","last_undegraded":"2024-09-19T07:43:11.311619+0000","last_fullsized":"2024-09-19T07:43:11.311619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:34.475024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310868+0000","last_change":"2024-09-19T07:43:10.283896+0000","last_active":"2024-09-19T07:43:11.310868+0000","last_peered":"2024-09-19T07:43:11.310868+0000","last_clean":"2024-09-19T07:43:11.310868+0000","last_became_active":"2024-09-19T07:43:10.283587+0000","last_became_peered":"2024-09-19T07:43:10.283587+0000","last_unstale":"2024-09-19T07:43:11.310868+0000","last_undegraded":"2024-09-19T07:43:11.310868+0000","last_fullsized":"2024-09-19T07:43:11.310868+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:41.387226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681270+0000","last_change":"2024-09-19T07:43:06.274315+0000","last_active":"2024-09-19T07:43:12.681270+0000","last_peered":"2024-09-19T07:43:12.681270+0000","last_clean":"2024-09-19T07:43:12.681270+0000","last_became_active":"2024-09-19T07:43:06.274064+0000","last_became_peered":"2024-09-19T07:43:06.274064+0000","last_unstale":"2024-09-19T07:43:12.681270+0000","last_undegraded":"2024-09-19T07:43:12.681270+0000","last_fullsized":"2024-09-19T07:43:12.681270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:22.804370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588519+0000","last_change":"2024-09-19T07:43:05.259083+0000","last_active":"2024-09-19T07:43:11.588519+0000","last_peered":"2024-09-19T07:43:11.588519+0000","last_clean":"2024-09-19T07:43:11.588519+0000","last_became_active":"2024-09-19T07:43:05.258769+0000","last_became_peered":"2024-09-19T07:43:05.258769+0000","last_unstale":"2024-09-19T07:43:11.588519+0000","last_undegraded":"2024-09-19T07:43:11.588519+0000","last_fullsized":"2024-09-19T07:43:11.588519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:51:24.580949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656048+0000","last_change":"2024-09-19T07:43:10.290261+0000","last_active":"2024-09-19T07:43:12.656048+0000","last_peered":"2024-09-19T07:43:12.656048+0000","last_clean":"2024-09-19T07:43:12.656048+0000","last_became_active":"2024-09-19T07:43:10.289428+0000","last_became_peered":"2024-09-19T07:43:10.289428+0000","last_unstale":"2024-09-19T07:43:12.656048+0000","last_undegraded":"2024-09-19T07:43:12.656048+0000","last_fullsized":"2024-09-19T07:43:12.656048+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:43.944005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450559+0000","last_change":"2024-09-19T07:43:06.276364+0000","last_active":"2024-09-19T07:43:11.450559+0000","last_peered":"2024-09-19T07:43:11.450559+0000","last_clean":"2024-09-19T07:43:11.450559+0000","last_became_active":"2024-09-19T07:43:06.275564+0000","last_became_peered":"2024-09-19T07:43:06.275564+0000","last_unstale":"2024-09-19T07:43:11.450559+0000","last_undegraded":"2024-09-19T07:43:11.450559+0000","last_fullsized":"2024-09-19T07:43:11.450559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450611+0000","last_change":"2024-09-19T07:43:05.264487+0000","last_active":"2024-09-19T07:43:11.450611+0000","last_peered":"2024-09-19T07:43:11.450611+0000","last_clean":"2024-09-19T07:43:11.450611+0000","last_became_active":"2024-09-19T07:43:05.264196+0000","last_became_peered":"2024-09-19T07:43:05.264196+0000","last_unstale":"2024-09-19T07:43:11.450611+0000","last_undegraded":"2024-09-19T07:43:11.450611+0000","last_fullsized":"2024-09-19T07:43:11.450611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:47.531702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311055+0000","last_change":"2024-09-19T07:43:10.481930+0000","last_active":"2024-09-19T07:43:11.311055+0000","last_peered":"2024-09-19T07:43:11.311055+0000","last_clean":"2024-09-19T07:43:11.311055+0000","last_became_active":"2024-09-19T07:43:10.481552+0000","last_became_peered":"2024-09-19T07:43:10.481552+0000","last_unstale":"2024-09-19T07:43:11.311055+0000","last_undegraded":"2024-09-19T07:43:11.311055+0000","last_fullsized":"2024-09-19T07:43:11.311055+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:19.523345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681307+0000","last_change":"2024-09-19T07:43:06.275131+0000","last_active":"2024-09-19T07:43:12.681307+0000","last_peered":"2024-09-19T07:43:12.681307+0000","last_clean":"2024-09-19T07:43:12.681307+0000","last_became_active":"2024-09-19T07:43:06.274857+0000","last_became_peered":"2024-09-19T07:43:06.274857+0000","last_unstale":"2024-09-19T07:43:12.681307+0000","last_undegraded":"2024-09-19T07:43:12.681307+0000","last_fullsized":"2024-09-19T07:43:12.681307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:57.928527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311569+0000","last_change":"2024-09-19T07:43:05.263838+0000","last_active":"2024-09-19T07:43:11.311569+0000","last_peered":"2024-09-19T07:43:11.311569+0000","last_clean":"2024-09-19T07:43:11.311569+0000","last_became_active":"2024-09-19T07:43:05.263484+0000","last_became_peered":"2024-09-19T07:43:05.263484+0000","last_unstale":"2024-09-19T07:43:11.311569+0000","last_undegraded":"2024-09-19T07:43:11.311569+0000","last_fullsized":"2024-09-19T07:43:11.311569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:38.098052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310652+0000","last_change":"2024-09-19T07:43:10.480636+0000","last_active":"2024-09-19T07:43:11.310652+0000","last_peered":"2024-09-19T07:43:11.310652+0000","last_clean":"2024-09-19T07:43:11.310652+0000","last_became_active":"2024-09-19T07:43:10.480217+0000","last_became_peered":"2024-09-19T07:43:10.480217+0000","last_unstale":"2024-09-19T07:43:11.310652+0000","last_undegraded":"2024-09-19T07:43:11.310652+0000","last_fullsized":"2024-09-19T07:43:11.310652+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:29.825328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656111+0000","last_change":"2024-09-19T07:43:06.272020+0000","last_active":"2024-09-19T07:43:12.656111+0000","last_peered":"2024-09-19T07:43:12.656111+0000","last_clean":"2024-09-19T07:43:12.656111+0000","last_became_active":"2024-09-19T07:43:06.271671+0000","last_became_peered":"2024-09-19T07:43:06.271671+0000","last_unstale":"2024-09-19T07:43:12.656111+0000","last_undegraded":"2024-09-19T07:43:12.656111+0000","last_fullsized":"2024-09-19T07:43:12.656111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:24:44.612992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656246+0000","last_change":"2024-09-19T07:43:05.265898+0000","last_active":"2024-09-19T07:43:12.656246+0000","last_peered":"2024-09-19T07:43:12.656246+0000","last_clean":"2024-09-19T07:43:12.656246+0000","last_became_active":"2024-09-19T07:43:05.265595+0000","last_became_peered":"2024-09-19T07:43:05.265595+0000","last_unstale":"2024-09-19T07:43:12.656246+0000","last_undegraded":"2024-09-19T07:43:12.656246+0000","last_fullsized":"2024-09-19T07:43:12.656246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:22.696534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311727+0000","last_change":"2024-09-19T07:43:10.482166+0000","last_active":"2024-09-19T07:43:11.311727+0000","last_peered":"2024-09-19T07:43:11.311727+0000","last_clean":"2024-09-19T07:43:11.311727+0000","last_became_active":"2024-09-19T07:43:10.481667+0000","last_became_peered":"2024-09-19T07:43:10.481667+0000","last_unstale":"2024-09-19T07:43:11.311727+0000","last_undegraded":"2024-09-19T07:43:11.311727+0000","last_fullsized":"2024-09-19T07:43:11.311727+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:10.437589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681470+0000","last_change":"2024-09-19T07:43:06.277189+0000","last_active":"2024-09-19T07:43:12.681470+0000","last_peered":"2024-09-19T07:43:12.681470+0000","last_clean":"2024-09-19T07:43:12.681470+0000","last_became_active":"2024-09-19T07:43:06.276924+0000","last_became_peered":"2024-09-19T07:43:06.276924+0000","last_unstale":"2024-09-19T07:43:12.681470+0000","last_undegraded":"2024-09-19T07:43:12.681470+0000","last_fullsized":"2024-09-19T07:43:12.681470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:39.658744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:15.766406+0000","last_change":"2024-09-19T07:43:05.266360+0000","last_active":"2024-09-19T07:43:15.766406+0000","last_peered":"2024-09-19T07:43:15.766406+0000","last_clean":"2024-09-19T07:43:15.766406+0000","last_became_active":"2024-09-19T07:43:05.266038+0000","last_became_peered":"2024-09-19T07:43:05.266038+0000","last_unstale":"2024-09-19T07:43:15.766406+0000","last_undegraded":"2024-09-19T07:43:15.766406+0000","last_fullsized":"2024-09-19T07:43:15.766406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:13.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680836+0000","last_change":"2024-09-19T07:43:10.295894+0000","last_active":"2024-09-19T07:43:12.680836+0000","last_peered":"2024-09-19T07:43:12.680836+0000","last_clean":"2024-09-19T07:43:12.680836+0000","last_became_active":"2024-09-19T07:43:10.295496+0000","last_became_peered":"2024-09-19T07:43:10.295496+0000","last_unstale":"2024-09-19T07:43:12.680836+0000","last_undegraded":"2024-09-19T07:43:12.680836+0000","last_fullsized":"2024-09-19T07:43:12.680836+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:18.372981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312405+0000","last_change":"2024-09-19T07:43:06.277639+0000","last_active":"2024-09-19T07:43:11.312405+0000","last_peered":"2024-09-19T07:43:11.312405+0000","last_clean":"2024-09-19T07:43:11.312405+0000","last_became_active":"2024-09-19T07:43:06.277186+0000","last_became_peered":"2024-09-19T07:43:06.277186+0000","last_unstale":"2024-09-19T07:43:11.312405+0000","last_undegraded":"2024-09-19T07:43:11.312405+0000","last_fullsized":"2024-09-19T07:43:11.312405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:07.895243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681210+0000","last_change":"2024-09-19T07:43:05.257416+0000","last_active":"2024-09-19T07:43:12.681210+0000","last_peered":"2024-09-19T07:43:12.681210+0000","last_clean":"2024-09-19T07:43:12.681210+0000","last_became_active":"2024-09-19T07:43:05.256941+0000","last_became_peered":"2024-09-19T07:43:05.256941+0000","last_unstale":"2024-09-19T07:43:12.681210+0000","last_undegraded":"2024-09-19T07:43:12.681210+0000","last_fullsized":"2024-09-19T07:43:12.681210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:44.359336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311217+0000","last_change":"2024-09-19T07:43:10.288931+0000","last_active":"2024-09-19T07:43:11.311217+0000","last_peered":"2024-09-19T07:43:11.311217+0000","last_clean":"2024-09-19T07:43:11.311217+0000","last_became_active":"2024-09-19T07:43:10.288548+0000","last_became_peered":"2024-09-19T07:43:10.288548+0000","last_unstale":"2024-09-19T07:43:11.311217+0000","last_undegraded":"2024-09-19T07:43:11.311217+0000","last_fullsized":"2024-09-19T07:43:11.311217+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:00.406076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656008+0000","last_change":"2024-09-19T07:43:06.270017+0000","last_active":"2024-09-19T07:43:12.656008+0000","last_peered":"2024-09-19T07:43:12.656008+0000","last_clean":"2024-09-19T07:43:12.656008+0000","last_became_active":"2024-09-19T07:43:06.269698+0000","last_became_peered":"2024-09-19T07:43:06.269698+0000","last_unstale":"2024-09-19T07:43:12.656008+0000","last_undegraded":"2024-09-19T07:43:12.656008+0000","last_fullsized":"2024-09-19T07:43:12.656008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:27.185190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311784+0000","last_change":"2024-09-19T07:43:05.265424+0000","last_active":"2024-09-19T07:43:11.311784+0000","last_peered":"2024-09-19T07:43:11.311784+0000","last_clean":"2024-09-19T07:43:11.311784+0000","last_became_active":"2024-09-19T07:43:05.265165+0000","last_became_peered":"2024-09-19T07:43:05.265165+0000","last_unstale":"2024-09-19T07:43:11.311784+0000","last_undegraded":"2024-09-19T07:43:11.311784+0000","last_fullsized":"2024-09-19T07:43:11.311784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:38:29.212025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310075+0000","last_change":"2024-09-19T07:43:10.290285+0000","last_active":"2024-09-19T07:43:11.310075+0000","last_peered":"2024-09-19T07:43:11.310075+0000","last_clean":"2024-09-19T07:43:11.310075+0000","last_became_active":"2024-09-19T07:43:10.289718+0000","last_became_peered":"2024-09-19T07:43:10.289718+0000","last_unstale":"2024-09-19T07:43:11.310075+0000","last_undegraded":"2024-09-19T07:43:11.310075+0000","last_fullsized":"2024-09-19T07:43:11.310075+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:08.693513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311662+0000","last_change":"2024-09-19T07:43:06.265419+0000","last_active":"2024-09-19T07:43:11.311662+0000","last_peered":"2024-09-19T07:43:11.311662+0000","last_clean":"2024-09-19T07:43:11.311662+0000","last_became_active":"2024-09-19T07:43:06.265170+0000","last_became_peered":"2024-09-19T07:43:06.265170+0000","last_unstale":"2024-09-19T07:43:11.311662+0000","last_undegraded":"2024-09-19T07:43:11.311662+0000","last_fullsized":"2024-09-19T07:43:11.311662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451157+0000","last_change":"2024-09-19T07:43:05.264095+0000","last_active":"2024-09-19T07:43:11.451157+0000","last_peered":"2024-09-19T07:43:11.451157+0000","last_clean":"2024-09-19T07:43:11.451157+0000","last_became_active":"2024-09-19T07:43:05.262895+0000","last_became_peered":"2024-09-19T07:43:05.262895+0000","last_unstale":"2024-09-19T07:43:11.451157+0000","last_undegraded":"2024-09-19T07:43:11.451157+0000","last_fullsized":"2024-09-19T07:43:11.451157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:38.098052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310066+0000","last_change":"2024-09-19T07:43:10.482892+0000","last_active":"2024-09-19T07:43:11.310066+0000","last_peered":"2024-09-19T07:43:11.310066+0000","last_clean":"2024-09-19T07:43:11.310066+0000","last_became_active":"2024-09-19T07:43:10.482523+0000","last_became_peered":"2024-09-19T07:43:10.482523+0000","last_unstale":"2024-09-19T07:43:11.310066+0000","last_undegraded":"2024-09-19T07:43:11.310066+0000","last_fullsized":"2024-09-19T07:43:11.310066+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:07.059894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655916+0000","last_change":"2024-09-19T07:43:10.480793+0000","last_active":"2024-09-19T07:43:12.655916+0000","last_peered":"2024-09-19T07:43:12.655916+0000","last_clean":"2024-09-19T07:43:12.655916+0000","last_became_active":"2024-09-19T07:43:10.480423+0000","last_became_peered":"2024-09-19T07:43:10.480423+0000","last_unstale":"2024-09-19T07:43:12.655916+0000","last_undegraded":"2024-09-19T07:43:12.655916+0000","last_fullsized":"2024-09-19T07:43:12.655916+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:18.372981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680526+0000","last_change":"2024-09-19T07:43:06.258001+0000","last_active":"2024-09-19T07:43:12.680526+0000","last_peered":"2024-09-19T07:43:12.680526+0000","last_clean":"2024-09-19T07:43:12.680526+0000","last_became_active":"2024-09-19T07:43:06.257731+0000","last_became_peered":"2024-09-19T07:43:06.257731+0000","last_unstale":"2024-09-19T07:43:12.680526+0000","last_undegraded":"2024-09-19T07:43:12.680526+0000","last_fullsized":"2024-09-19T07:43:12.680526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:14.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310662+0000","last_change":"2024-09-19T07:43:05.260911+0000","last_active":"2024-09-19T07:43:11.310662+0000","last_peered":"2024-09-19T07:43:11.310662+0000","last_clean":"2024-09-19T07:43:11.310662+0000","last_became_active":"2024-09-19T07:43:05.258827+0000","last_became_peered":"2024-09-19T07:43:05.258827+0000","last_unstale":"2024-09-19T07:43:11.310662+0000","last_undegraded":"2024-09-19T07:43:11.310662+0000","last_fullsized":"2024-09-19T07:43:11.310662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:13.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451139+0000","last_change":"2024-09-19T07:43:10.293630+0000","last_active":"2024-09-19T07:43:11.451139+0000","last_peered":"2024-09-19T07:43:11.451139+0000","last_clean":"2024-09-19T07:43:11.451139+0000","last_became_active":"2024-09-19T07:43:10.292979+0000","last_became_peered":"2024-09-19T07:43:10.292979+0000","last_unstale":"2024-09-19T07:43:11.451139+0000","last_undegraded":"2024-09-19T07:43:11.451139+0000","last_fullsized":"2024-09-19T07:43:11.451139+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:43.490603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310026+0000","last_change":"2024-09-19T07:43:06.274060+0000","last_active":"2024-09-19T07:43:11.310026+0000","last_peered":"2024-09-19T07:43:11.310026+0000","last_clean":"2024-09-19T07:43:11.310026+0000","last_became_active":"2024-09-19T07:43:06.272704+0000","last_became_peered":"2024-09-19T07:43:06.272704+0000","last_unstale":"2024-09-19T07:43:11.310026+0000","last_undegraded":"2024-09-19T07:43:11.310026+0000","last_fullsized":"2024-09-19T07:43:11.310026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:06.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310182+0000","last_change":"2024-09-19T07:43:05.264989+0000","last_active":"2024-09-19T07:43:11.310182+0000","last_peered":"2024-09-19T07:43:11.310182+0000","last_clean":"2024-09-19T07:43:11.310182+0000","last_became_active":"2024-09-19T07:43:05.264673+0000","last_became_peered":"2024-09-19T07:43:05.264673+0000","last_unstale":"2024-09-19T07:43:11.310182+0000","last_undegraded":"2024-09-19T07:43:11.310182+0000","last_fullsized":"2024-09-19T07:43:11.310182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:22.696534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450834+0000","last_change":"2024-09-19T07:43:05.262438+0000","last_active":"2024-09-19T07:43:11.450834+0000","last_peered":"2024-09-19T07:43:11.450834+0000","last_clean":"2024-09-19T07:43:11.450834+0000","last_became_active":"2024-09-19T07:43:05.262129+0000","last_became_peered":"2024-09-19T07:43:05.262129+0000","last_unstale":"2024-09-19T07:43:11.450834+0000","last_undegraded":"2024-09-19T07:43:11.450834+0000","last_fullsized":"2024-09-19T07:43:11.450834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:44.359336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656191+0000","last_change":"2024-09-19T07:43:06.270402+0000","last_active":"2024-09-19T07:43:12.656191+0000","last_peered":"2024-09-19T07:43:12.656191+0000","last_clean":"2024-09-19T07:43:12.656191+0000","last_became_active":"2024-09-19T07:43:06.270087+0000","last_became_peered":"2024-09-19T07:43:06.270087+0000","last_unstale":"2024-09-19T07:43:12.656191+0000","last_undegraded":"2024-09-19T07:43:12.656191+0000","last_fullsized":"2024-09-19T07:43:12.656191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588782+0000","last_change":"2024-09-19T07:43:10.288597+0000","last_active":"2024-09-19T07:43:11.588782+0000","last_peered":"2024-09-19T07:43:11.588782+0000","last_clean":"2024-09-19T07:43:11.588782+0000","last_became_active":"2024-09-19T07:43:10.288207+0000","last_became_peered":"2024-09-19T07:43:10.288207+0000","last_unstale":"2024-09-19T07:43:11.588782+0000","last_undegraded":"2024-09-19T07:43:11.588782+0000","last_fullsized":"2024-09-19T07:43:11.588782+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:43.318582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310346+0000","last_change":"2024-09-19T07:43:05.261298+0000","last_active":"2024-09-19T07:43:11.310346+0000","last_peered":"2024-09-19T07:43:11.310346+0000","last_clean":"2024-09-19T07:43:11.310346+0000","last_became_active":"2024-09-19T07:43:05.261000+0000","last_became_peered":"2024-09-19T07:43:05.261000+0000","last_unstale":"2024-09-19T07:43:11.310346+0000","last_undegraded":"2024-09-19T07:43:11.310346+0000","last_fullsized":"2024-09-19T07:43:11.310346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:38.098052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310888+0000","last_change":"2024-09-19T07:43:06.267976+0000","last_active":"2024-09-19T07:43:11.310888+0000","last_peered":"2024-09-19T07:43:11.310888+0000","last_clean":"2024-09-19T07:43:11.310888+0000","last_became_active":"2024-09-19T07:43:06.267214+0000","last_became_peered":"2024-09-19T07:43:06.267214+0000","last_unstale":"2024-09-19T07:43:11.310888+0000","last_undegraded":"2024-09-19T07:43:11.310888+0000","last_fullsized":"2024-09-19T07:43:11.310888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:48.537241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310742+0000","last_change":"2024-09-19T07:43:10.288619+0000","last_active":"2024-09-19T07:43:11.310742+0000","last_peered":"2024-09-19T07:43:11.310742+0000","last_clean":"2024-09-19T07:43:11.310742+0000","last_became_active":"2024-09-19T07:43:10.288213+0000","last_became_peered":"2024-09-19T07:43:10.288213+0000","last_unstale":"2024-09-19T07:43:11.310742+0000","last_undegraded":"2024-09-19T07:43:11.310742+0000","last_fullsized":"2024-09-19T07:43:11.310742+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:30.165579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450437+0000","last_change":"2024-09-19T07:43:06.276792+0000","last_active":"2024-09-19T07:43:11.450437+0000","last_peered":"2024-09-19T07:43:11.450437+0000","last_clean":"2024-09-19T07:43:11.450437+0000","last_became_active":"2024-09-19T07:43:06.276518+0000","last_became_peered":"2024-09-19T07:43:06.276518+0000","last_unstale":"2024-09-19T07:43:11.450437+0000","last_undegraded":"2024-09-19T07:43:11.450437+0000","last_fullsized":"2024-09-19T07:43:11.450437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:12.618593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588673+0000","last_change":"2024-09-19T07:43:06.268099+0000","last_active":"2024-09-19T07:43:11.588673+0000","last_peered":"2024-09-19T07:43:11.588673+0000","last_clean":"2024-09-19T07:43:11.588673+0000","last_became_active":"2024-09-19T07:43:06.267803+0000","last_became_peered":"2024-09-19T07:43:06.267803+0000","last_unstale":"2024-09-19T07:43:11.588673+0000","last_undegraded":"2024-09-19T07:43:11.588673+0000","last_fullsized":"2024-09-19T07:43:11.588673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:31.966840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311465+0000","last_change":"2024-09-19T07:43:06.263757+0000","last_active":"2024-09-19T07:43:11.311465+0000","last_peered":"2024-09-19T07:43:11.311465+0000","last_clean":"2024-09-19T07:43:11.311465+0000","last_became_active":"2024-09-19T07:43:06.263480+0000","last_became_peered":"2024-09-19T07:43:06.263480+0000","last_unstale":"2024-09-19T07:43:11.311465+0000","last_undegraded":"2024-09-19T07:43:11.311465+0000","last_fullsized":"2024-09-19T07:43:11.311465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:39.918269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656177+0000","last_change":"2024-09-19T07:43:06.278694+0000","last_active":"2024-09-19T07:43:12.656177+0000","last_peered":"2024-09-19T07:43:12.656177+0000","last_clean":"2024-09-19T07:43:12.656177+0000","last_became_active":"2024-09-19T07:43:06.277280+0000","last_became_peered":"2024-09-19T07:43:06.277280+0000","last_unstale":"2024-09-19T07:43:12.656177+0000","last_undegraded":"2024-09-19T07:43:12.656177+0000","last_fullsized":"2024-09-19T07:43:12.656177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:51.595301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310764+0000","last_change":"2024-09-19T07:43:06.272967+0000","last_active":"2024-09-19T07:43:11.310764+0000","last_peered":"2024-09-19T07:43:11.310764+0000","last_clean":"2024-09-19T07:43:11.310764+0000","last_became_active":"2024-09-19T07:43:06.272476+0000","last_became_peered":"2024-09-19T07:43:06.272476+0000","last_unstale":"2024-09-19T07:43:11.310764+0000","last_undegraded":"2024-09-19T07:43:11.310764+0000","last_fullsized":"2024-09-19T07:43:11.310764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:09.222719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681113+0000","last_change":"2024-09-19T07:43:06.260655+0000","last_active":"2024-09-19T07:43:12.681113+0000","last_peered":"2024-09-19T07:43:12.681113+0000","last_clean":"2024-09-19T07:43:12.681113+0000","last_became_active":"2024-09-19T07:43:06.260388+0000","last_became_peered":"2024-09-19T07:43:06.260388+0000","last_unstale":"2024-09-19T07:43:12.681113+0000","last_undegraded":"2024-09-19T07:43:12.681113+0000","last_fullsized":"2024-09-19T07:43:12.681113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451084+0000","last_change":"2024-09-19T07:43:06.270719+0000","last_active":"2024-09-19T07:43:11.451084+0000","last_peered":"2024-09-19T07:43:11.451084+0000","last_clean":"2024-09-19T07:43:11.451084+0000","last_became_active":"2024-09-19T07:43:06.270022+0000","last_became_peered":"2024-09-19T07:43:06.270022+0000","last_unstale":"2024-09-19T07:43:11.451084+0000","last_undegraded":"2024-09-19T07:43:11.451084+0000","last_fullsized":"2024-09-19T07:43:11.451084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:27.185190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310886+0000","last_change":"2024-09-19T07:43:06.266798+0000","last_active":"2024-09-19T07:43:11.310886+0000","last_peered":"2024-09-19T07:43:11.310886+0000","last_clean":"2024-09-19T07:43:11.310886+0000","last_became_active":"2024-09-19T07:43:06.266461+0000","last_became_peered":"2024-09-19T07:43:06.266461+0000","last_unstale":"2024-09-19T07:43:11.310886+0000","last_undegraded":"2024-09-19T07:43:11.310886+0000","last_fullsized":"2024-09-19T07:43:11.310886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:50.544269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310636+0000","last_change":"2024-09-19T07:43:06.263830+0000","last_active":"2024-09-19T07:43:11.310636+0000","last_peered":"2024-09-19T07:43:11.310636+0000","last_clean":"2024-09-19T07:43:11.310636+0000","last_became_active":"2024-09-19T07:43:06.263539+0000","last_became_peered":"2024-09-19T07:43:06.263539+0000","last_unstale":"2024-09-19T07:43:11.310636+0000","last_undegraded":"2024-09-19T07:43:11.310636+0000","last_fullsized":"2024-09-19T07:43:11.310636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:50.544269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310166+0000","last_change":"2024-09-19T07:43:06.276371+0000","last_active":"2024-09-19T07:43:11.310166+0000","last_peered":"2024-09-19T07:43:11.310166+0000","last_clean":"2024-09-19T07:43:11.310166+0000","last_became_active":"2024-09-19T07:43:06.276103+0000","last_became_peered":"2024-09-19T07:43:06.276103+0000","last_unstale":"2024-09-19T07:43:11.310166+0000","last_undegraded":"2024-09-19T07:43:11.310166+0000","last_fullsized":"2024-09-19T07:43:11.310166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:51.595301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311371+0000","last_change":"2024-09-19T07:43:06.265375+0000","last_active":"2024-09-19T07:43:11.311371+0000","last_peered":"2024-09-19T07:43:11.311371+0000","last_clean":"2024-09-19T07:43:11.311371+0000","last_became_active":"2024-09-19T07:43:06.265118+0000","last_became_peered":"2024-09-19T07:43:06.265118+0000","last_unstale":"2024-09-19T07:43:11.311371+0000","last_undegraded":"2024-09-19T07:43:11.311371+0000","last_fullsized":"2024-09-19T07:43:11.311371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:14.347245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655877+0000","last_change":"2024-09-19T07:43:06.279273+0000","last_active":"2024-09-19T07:43:12.655877+0000","last_peered":"2024-09-19T07:43:12.655877+0000","last_clean":"2024-09-19T07:43:12.655877+0000","last_became_active":"2024-09-19T07:43:06.278276+0000","last_became_peered":"2024-09-19T07:43:06.278276+0000","last_unstale":"2024-09-19T07:43:12.655877+0000","last_undegraded":"2024-09-19T07:43:12.655877+0000","last_fullsized":"2024-09-19T07:43:12.655877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:14.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450830+0000","last_change":"2024-09-19T07:43:06.268360+0000","last_active":"2024-09-19T07:43:11.450830+0000","last_peered":"2024-09-19T07:43:11.450830+0000","last_clean":"2024-09-19T07:43:11.450830+0000","last_became_active":"2024-09-19T07:43:06.268096+0000","last_became_peered":"2024-09-19T07:43:06.268096+0000","last_unstale":"2024-09-19T07:43:11.450830+0000","last_undegraded":"2024-09-19T07:43:11.450830+0000","last_fullsized":"2024-09-19T07:43:11.450830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681729+0000","last_change":"2024-09-19T07:43:06.272110+0000","last_active":"2024-09-19T07:43:12.681729+0000","last_peered":"2024-09-19T07:43:12.681729+0000","last_clean":"2024-09-19T07:43:12.681729+0000","last_became_active":"2024-09-19T07:43:06.271804+0000","last_became_peered":"2024-09-19T07:43:06.271804+0000","last_unstale":"2024-09-19T07:43:12.681729+0000","last_undegraded":"2024-09-19T07:43:12.681729+0000","last_fullsized":"2024-09-19T07:43:12.681729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:23.113048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588806+0000","last_change":"2024-09-19T07:43:06.260936+0000","last_active":"2024-09-19T07:43:11.588806+0000","last_peered":"2024-09-19T07:43:11.588806+0000","last_clean":"2024-09-19T07:43:11.588806+0000","last_became_active":"2024-09-19T07:43:06.260663+0000","last_became_peered":"2024-09-19T07:43:06.260663+0000","last_unstale":"2024-09-19T07:43:11.588806+0000","last_undegraded":"2024-09-19T07:43:11.588806+0000","last_fullsized":"2024-09-19T07:43:11.588806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:06.425112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312242+0000","last_change":"2024-09-19T07:43:06.268104+0000","last_active":"2024-09-19T07:43:11.312242+0000","last_peered":"2024-09-19T07:43:11.312242+0000","last_clean":"2024-09-19T07:43:11.312242+0000","last_became_active":"2024-09-19T07:43:06.267830+0000","last_became_peered":"2024-09-19T07:43:06.267830+0000","last_unstale":"2024-09-19T07:43:11.312242+0000","last_undegraded":"2024-09-19T07:43:11.312242+0000","last_fullsized":"2024-09-19T07:43:11.312242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:22.804370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310551+0000","last_change":"2024-09-19T07:43:06.269948+0000","last_active":"2024-09-19T07:43:11.310551+0000","last_peered":"2024-09-19T07:43:11.310551+0000","last_clean":"2024-09-19T07:43:11.310551+0000","last_became_active":"2024-09-19T07:43:06.269684+0000","last_became_peered":"2024-09-19T07:43:06.269684+0000","last_unstale":"2024-09-19T07:43:11.310551+0000","last_undegraded":"2024-09-19T07:43:11.310551+0000","last_fullsized":"2024-09-19T07:43:11.310551+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680603+0000","last_change":"2024-09-19T07:43:06.269924+0000","last_active":"2024-09-19T07:43:12.680603+0000","last_peered":"2024-09-19T07:43:12.680603+0000","last_clean":"2024-09-19T07:43:12.680603+0000","last_became_active":"2024-09-19T07:43:06.269602+0000","last_became_peered":"2024-09-19T07:43:06.269602+0000","last_unstale":"2024-09-19T07:43:12.680603+0000","last_undegraded":"2024-09-19T07:43:12.680603+0000","last_fullsized":"2024-09-19T07:43:12.680603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:51.595301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655995+0000","last_change":"2024-09-19T07:43:06.271422+0000","last_active":"2024-09-19T07:43:12.655995+0000","last_peered":"2024-09-19T07:43:12.655995+0000","last_clean":"2024-09-19T07:43:12.655995+0000","last_became_active":"2024-09-19T07:43:06.271096+0000","last_became_peered":"2024-09-19T07:43:06.271096+0000","last_unstale":"2024-09-19T07:43:12.655995+0000","last_undegraded":"2024-09-19T07:43:12.655995+0000","last_fullsized":"2024-09-19T07:43:12.655995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:23.743784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310215+0000","last_change":"2024-09-19T07:43:06.264090+0000","last_active":"2024-09-19T07:43:11.310215+0000","last_peered":"2024-09-19T07:43:11.310215+0000","last_clean":"2024-09-19T07:43:11.310215+0000","last_became_active":"2024-09-19T07:43:06.262071+0000","last_became_peered":"2024-09-19T07:43:06.262071+0000","last_unstale":"2024-09-19T07:43:11.310215+0000","last_undegraded":"2024-09-19T07:43:11.310215+0000","last_fullsized":"2024-09-19T07:43:11.310215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312316+0000","last_change":"2024-09-19T07:43:06.265567+0000","last_active":"2024-09-19T07:43:11.312316+0000","last_peered":"2024-09-19T07:43:11.312316+0000","last_clean":"2024-09-19T07:43:11.312316+0000","last_became_active":"2024-09-19T07:43:06.265290+0000","last_became_peered":"2024-09-19T07:43:06.265290+0000","last_unstale":"2024-09-19T07:43:11.312316+0000","last_undegraded":"2024-09-19T07:43:11.312316+0000","last_fullsized":"2024-09-19T07:43:11.312316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:14.597733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450953+0000","last_change":"2024-09-19T07:43:06.267949+0000","last_active":"2024-09-19T07:43:11.450953+0000","last_peered":"2024-09-19T07:43:11.450953+0000","last_clean":"2024-09-19T07:43:11.450953+0000","last_became_active":"2024-09-19T07:43:06.267640+0000","last_became_peered":"2024-09-19T07:43:06.267640+0000","last_unstale":"2024-09-19T07:43:11.450953+0000","last_undegraded":"2024-09-19T07:43:11.450953+0000","last_fullsized":"2024-09-19T07:43:11.450953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:14.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451030+0000","last_change":"2024-09-19T07:43:06.268718+0000","last_active":"2024-09-19T07:43:11.451030+0000","last_peered":"2024-09-19T07:43:11.451030+0000","last_clean":"2024-09-19T07:43:11.451030+0000","last_became_active":"2024-09-19T07:43:06.268398+0000","last_became_peered":"2024-09-19T07:43:06.268398+0000","last_unstale":"2024-09-19T07:43:11.451030+0000","last_undegraded":"2024-09-19T07:43:11.451030+0000","last_fullsized":"2024-09-19T07:43:11.451030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:51.595301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681743+0000","last_change":"2024-09-19T07:43:06.273520+0000","last_active":"2024-09-19T07:43:12.681743+0000","last_peered":"2024-09-19T07:43:12.681743+0000","last_clean":"2024-09-19T07:43:12.681743+0000","last_became_active":"2024-09-19T07:43:06.273150+0000","last_became_peered":"2024-09-19T07:43:06.273150+0000","last_unstale":"2024-09-19T07:43:12.681743+0000","last_undegraded":"2024-09-19T07:43:12.681743+0000","last_fullsized":"2024-09-19T07:43:12.681743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:39.464868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.309860+0000","last_change":"2024-09-19T07:43:06.272990+0000","last_active":"2024-09-19T07:43:11.309860+0000","last_peered":"2024-09-19T07:43:11.309860+0000","last_clean":"2024-09-19T07:43:11.309860+0000","last_became_active":"2024-09-19T07:43:06.272323+0000","last_became_peered":"2024-09-19T07:43:06.272323+0000","last_unstale":"2024-09-19T07:43:11.309860+0000","last_undegraded":"2024-09-19T07:43:11.309860+0000","last_fullsized":"2024-09-19T07:43:11.309860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:14.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681143+0000","last_change":"2024-09-19T07:43:06.273648+0000","last_active":"2024-09-19T07:43:12.681143+0000","last_peered":"2024-09-19T07:43:12.681143+0000","last_clean":"2024-09-19T07:43:12.681143+0000","last_became_active":"2024-09-19T07:43:06.273387+0000","last_became_peered":"2024-09-19T07:43:06.273387+0000","last_unstale":"2024-09-19T07:43:12.681143+0000","last_undegraded":"2024-09-19T07:43:12.681143+0000","last_fullsized":"2024-09-19T07:43:12.681143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:56.380341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311270+0000","last_change":"2024-09-19T07:43:06.274896+0000","last_active":"2024-09-19T07:43:11.311270+0000","last_peered":"2024-09-19T07:43:11.311270+0000","last_clean":"2024-09-19T07:43:11.311270+0000","last_became_active":"2024-09-19T07:43:06.274551+0000","last_became_peered":"2024-09-19T07:43:06.274551+0000","last_unstale":"2024-09-19T07:43:11.311270+0000","last_undegraded":"2024-09-19T07:43:11.311270+0000","last_fullsized":"2024-09-19T07:43:11.311270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:39.658744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681148+0000","last_change":"2024-09-19T07:43:06.271789+0000","last_active":"2024-09-19T07:43:12.681148+0000","last_peered":"2024-09-19T07:43:12.681148+0000","last_clean":"2024-09-19T07:43:12.681148+0000","last_became_active":"2024-09-19T07:43:06.269978+0000","last_became_peered":"2024-09-19T07:43:06.269978+0000","last_unstale":"2024-09-19T07:43:12.681148+0000","last_undegraded":"2024-09-19T07:43:12.681148+0000","last_fullsized":"2024-09-19T07:43:12.681148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:04.168224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311913+0000","last_change":"2024-09-19T07:43:06.274164+0000","last_active":"2024-09-19T07:43:11.311913+0000","last_peered":"2024-09-19T07:43:11.311913+0000","last_clean":"2024-09-19T07:43:11.311913+0000","last_became_active":"2024-09-19T07:43:06.273730+0000","last_became_peered":"2024-09-19T07:43:06.273730+0000","last_unstale":"2024-09-19T07:43:11.311913+0000","last_undegraded":"2024-09-19T07:43:11.311913+0000","last_fullsized":"2024-09-19T07:43:11.311913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:56.380341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655874+0000","last_change":"2024-09-19T07:43:06.266043+0000","last_active":"2024-09-19T07:43:12.655874+0000","last_peered":"2024-09-19T07:43:12.655874+0000","last_clean":"2024-09-19T07:43:12.655874+0000","last_became_active":"2024-09-19T07:43:06.265717+0000","last_became_peered":"2024-09-19T07:43:06.265717+0000","last_unstale":"2024-09-19T07:43:12.655874+0000","last_undegraded":"2024-09-19T07:43:12.655874+0000","last_fullsized":"2024-09-19T07:43:12.655874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:12.618593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681153+0000","last_change":"2024-09-19T07:43:06.270945+0000","last_active":"2024-09-19T07:43:12.681153+0000","last_peered":"2024-09-19T07:43:12.681153+0000","last_clean":"2024-09-19T07:43:12.681153+0000","last_became_active":"2024-09-19T07:43:06.270641+0000","last_became_peered":"2024-09-19T07:43:06.270641+0000","last_unstale":"2024-09-19T07:43:12.681153+0000","last_undegraded":"2024-09-19T07:43:12.681153+0000","last_fullsized":"2024-09-19T07:43:12.681153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:27.185190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681632+0000","last_change":"2024-09-19T07:43:06.277928+0000","last_active":"2024-09-19T07:43:12.681632+0000","last_peered":"2024-09-19T07:43:12.681632+0000","last_clean":"2024-09-19T07:43:12.681632+0000","last_became_active":"2024-09-19T07:43:06.277632+0000","last_became_peered":"2024-09-19T07:43:06.277632+0000","last_unstale":"2024-09-19T07:43:12.681632+0000","last_undegraded":"2024-09-19T07:43:12.681632+0000","last_fullsized":"2024-09-19T07:43:12.681632+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:09.222719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":868352,"data_stored":697396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":110290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":106536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":106572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":876544,"data_stored":697452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":282624,"data_stored":110290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":278528,"data_stored":106606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":270336,"data_stored":106446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:43:30.729 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:43:30.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.865+0000 7f1a9dfc4640 1 Processor -- start 2024-09-19T07:43:30.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.865+0000 7f1a9dfc4640 1 -- start start 2024-09-19T07:43:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.866+0000 7f1a9dfc4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a9815e2a0 0x7f1a9806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.866+0000 7f1a9dfc4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a980d1880 con 0x7f1a9806d960 2024-09-19T07:43:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.866+0000 7f1a9dfc4640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a980d1b60 con 0x7f1a9815e680 2024-09-19T07:43:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.866+0000 7f1a9dfc4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1a980cf2d0 con 0x7f1a9815e2a0 2024-09-19T07:43:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.866+0000 7f1a97fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1a9806d960 0x7f1a980d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44968/0 (socket says 172.21.15.16:44968) 2024-09-19T07:43:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.866+0000 7f1a97fff640 1 -- 172.21.15.16:0/2957169342 learned_addr learned my addr 172.21.15.16:0/2957169342 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.866+0000 7f1a977fe640 1 --2- 172.21.15.16:0/2957169342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a9815e2a0 0x7f1a9806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.866+0000 7f1a967fc640 1 -- 172.21.15.16:0/2957169342 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2237258049 0 0) 0x7f1a980d1880 con 0x7f1a9806d960 2024-09-19T07:43:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.866+0000 7f1a967fc640 1 -- 172.21.15.16:0/2957169342 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a80003520 con 0x7f1a9806d960 2024-09-19T07:43:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.866+0000 7f1a967fc640 1 -- 172.21.15.16:0/2957169342 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 676631407 0 0) 0x7f1a80003520 con 0x7f1a9806d960 2024-09-19T07:43:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.866+0000 7f1a967fc640 1 -- 172.21.15.16:0/2957169342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a9815e2a0 msgr2=0x7f1a9806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.866+0000 7f1a967fc640 1 --2- 172.21.15.16:0/2957169342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a9815e2a0 0x7f1a9806d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.866+0000 7f1a967fc640 1 -- 172.21.15.16:0/2957169342 >> v1:172.21.15.146:6789/0 conn(0x7f1a9815e680 legacy=0x7f1a980cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.866+0000 7f1a967fc640 1 -- 172.21.15.16:0/2957169342 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a980cf5b0 con 0x7f1a9806d960 2024-09-19T07:43:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.866+0000 7f1a967fc640 1 -- 172.21.15.16:0/2957169342 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f1a900019f0 con 0x7f1a9806d960 2024-09-19T07:43:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.866+0000 7f1a967fc640 1 -- 172.21.15.16:0/2957169342 >> v1:172.21.15.16:6789/0 conn(0x7f1a9806d960 legacy=0x7f1a980d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.867+0000 7f1a967fc640 1 --2- 172.21.15.16:0/2957169342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a800039e0 0x7f1a80003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.867+0000 7f1a967fc640 1 --2- 172.21.15.16:0/2957169342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a9815e2a0 0x7f1a80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.867+0000 7f1a967fc640 1 --2- 172.21.15.16:0/2957169342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a80004d40 0x7f1a800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.867+0000 7f1a967fc640 1 -- 172.21.15.16:0/2957169342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a980cf2d0 con 0x7f1a800039e0 2024-09-19T07:43:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.867+0000 7f1a967fc640 1 -- 172.21.15.16:0/2957169342 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8000ba70 con 0x7f1a80004d40 2024-09-19T07:43:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.867+0000 7f1a967fc640 1 -- 172.21.15.16:0/2957169342 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8000bc70 con 0x7f1a9815e2a0 2024-09-19T07:43:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.867+0000 7f1a977fe640 1 --2- 172.21.15.16:0/2957169342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a800039e0 0x7f1a80003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.867+0000 7f1a96ffd640 1 --2- 172.21.15.16:0/2957169342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a9815e2a0 0x7f1a80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.867+0000 7f1a97fff640 1 --2- 172.21.15.16:0/2957169342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a80004d40 0x7f1a800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.867+0000 7f1a977fe640 1 -- 172.21.15.16:0/2957169342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a9815e2a0 msgr2=0x7f1a80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.867+0000 7f1a977fe640 1 --2- 172.21.15.16:0/2957169342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a9815e2a0 0x7f1a80004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.867+0000 7f1a977fe640 1 -- 172.21.15.16:0/2957169342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a80004d40 msgr2=0x7f1a800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.867+0000 7f1a977fe640 1 --2- 172.21.15.16:0/2957169342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a80004d40 0x7f1a800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.867+0000 7f1a977fe640 1 -- 172.21.15.16:0/2957169342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a80003520 con 0x7f1a800039e0 2024-09-19T07:43:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.868+0000 7f1a977fe640 1 --2- 172.21.15.16:0/2957169342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a800039e0 0x7f1a80003e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1a88009a90 tx=0x7f1a80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a73fc51ffaebf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.868+0000 7f1a967fc640 1 -- 172.21.15.16:0/2957169342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88036070 con 0x7f1a800039e0 2024-09-19T07:43:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.868+0000 7f1a967fc640 1 -- 172.21.15.16:0/2957169342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a88031400 con 0x7f1a800039e0 2024-09-19T07:43:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.868+0000 7f1a967fc640 1 -- 172.21.15.16:0/2957169342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88009690 con 0x7f1a800039e0 2024-09-19T07:43:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.868+0000 7f1a9dfc4640 1 -- 172.21.15.16:0/2957169342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a800039e0 msgr2=0x7f1a80003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.868+0000 7f1a9dfc4640 1 --2- 172.21.15.16:0/2957169342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a800039e0 0x7f1a80003e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1a88009a90 tx=0x7f1a80005830 comp rx=0 tx=0).stop 2024-09-19T07:43:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.868+0000 7f1a97fff640 1 -- 172.21.15.16:0/2957169342 reap_dead start 2024-09-19T07:43:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.868+0000 7f1a9dfc4640 1 -- 172.21.15.16:0/2957169342 shutdown_connections 2024-09-19T07:43:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.868+0000 7f1a9dfc4640 1 -- 172.21.15.16:0/2957169342 >> 172.21.15.16:0/2957169342 conn(0x7f1a9809d760 msgr2=0x7f1a9808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.868+0000 7f1a9dfc4640 1 -- 172.21.15.16:0/2957169342 shutdown_connections 2024-09-19T07:43:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.868+0000 7f1a9dfc4640 1 -- 172.21.15.16:0/2957169342 wait complete. 2024-09-19T07:43:30.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.870+0000 7f1a9dfc4640 1 Processor -- start 2024-09-19T07:43:30.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.870+0000 7f1a9dfc4640 1 -- start start 2024-09-19T07:43:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.870+0000 7f1a9dfc4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a800039e0 0x7f1a9815d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.870+0000 7f1a9dfc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a980cdf50 0x7f1a980ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.870+0000 7f1a9dfc4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a980ca320 0x7f1a980ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.870+0000 7f1a9dfc4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a980d0360 con 0x7f1a980cdf50 2024-09-19T07:43:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.870+0000 7f1a9dfc4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1a980d01e0 con 0x7f1a980ca320 2024-09-19T07:43:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.871+0000 7f1a9dfc4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1a980d4a80 con 0x7f1a800039e0 2024-09-19T07:43:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.871+0000 7f1a96ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a980cdf50 0x7f1a980ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.871+0000 7f1a96ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a980cdf50 0x7f1a980ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57772/0 (socket says 172.21.15.16:57772) 2024-09-19T07:43:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.871+0000 7f1a96ffd640 1 -- 172.21.15.16:0/2009988345 learned_addr learned my addr 172.21.15.16:0/2009988345 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.871+0000 7f1a97fff640 1 --2- 172.21.15.16:0/2009988345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a980ca320 0x7f1a980ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.871+0000 7f1a977fe640 1 --2- 172.21.15.16:0/2009988345 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a800039e0 0x7f1a9815d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.871+0000 7f1a96ffd640 1 -- 172.21.15.16:0/2009988345 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a800039e0 msgr2=0x7f1a9815d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.871+0000 7f1a96ffd640 1 --2- 172.21.15.16:0/2009988345 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a800039e0 0x7f1a9815d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.871+0000 7f1a96ffd640 1 -- 172.21.15.16:0/2009988345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a980ca320 msgr2=0x7f1a980ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.871+0000 7f1a96ffd640 1 --2- 172.21.15.16:0/2009988345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a980ca320 0x7f1a980ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.871+0000 7f1a96ffd640 1 -- 172.21.15.16:0/2009988345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a980c9280 con 0x7f1a980cdf50 2024-09-19T07:43:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.871+0000 7f1a96ffd640 1 --2- 172.21.15.16:0/2009988345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a980cdf50 0x7f1a980ce3d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00f610 tx=0x7f1a8c00fad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.871+0000 7f1a7ffff640 1 -- 172.21.15.16:0/2009988345 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c0052e0 con 0x7f1a980cdf50 2024-09-19T07:43:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.871+0000 7f1a7ffff640 1 -- 172.21.15.16:0/2009988345 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8c018070 con 0x7f1a980cdf50 2024-09-19T07:43:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.871+0000 7f1a9dfc4640 1 -- 172.21.15.16:0/2009988345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a980c9560 con 0x7f1a980cdf50 2024-09-19T07:43:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.871+0000 7f1a7ffff640 1 -- 172.21.15.16:0/2009988345 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c003830 con 0x7f1a980cdf50 2024-09-19T07:43:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.871+0000 7f1a9dfc4640 1 -- 172.21.15.16:0/2009988345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a980c9b20 con 0x7f1a980cdf50 2024-09-19T07:43:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.872+0000 7f1a9dfc4640 1 -- 172.21.15.16:0/2009988345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a9806d420 con 0x7f1a980cdf50 2024-09-19T07:43:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.873+0000 7f1a7ffff640 1 -- 172.21.15.16:0/2009988345 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1a8c0039d0 con 0x7f1a980cdf50 2024-09-19T07:43:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.873+0000 7f1a7ffff640 1 --2- 172.21.15.16:0/2009988345 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1a8406c2b0 0x7f1a8406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.873+0000 7f1a7ffff640 1 -- 172.21.15.16:0/2009988345 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1a8c08b460 con 0x7f1a980cdf50 2024-09-19T07:43:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.874+0000 7f1a977fe640 1 --2- 172.21.15.16:0/2009988345 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1a8406c2b0 0x7f1a8406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.874+0000 7f1a977fe640 1 --2- 172.21.15.16:0/2009988345 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1a8406c2b0 0x7f1a8406e770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1a88033000 tx=0x7f1a88008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:30.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:30.878+0000 7f1a7ffff640 1 -- 172.21.15.16:0/2009988345 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1a8c0591a0 con 0x7f1a980cdf50 2024-09-19T07:43:31.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.053+0000 7f1a9dfc4640 1 -- 172.21.15.16:0/2009988345 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1a9806bc10 con 0x7f1a8406c2b0 2024-09-19T07:43:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.062+0000 7f1a7ffff640 1 -- 172.21.15.16:0/2009988345 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f1a9806bc10 con 0x7f1a8406c2b0 2024-09-19T07:43:31.066 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:31.068 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:43:31.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.065+0000 7f1a9dfc4640 1 -- 172.21.15.16:0/2009988345 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1a8406c2b0 msgr2=0x7f1a8406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:31.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.065+0000 7f1a9dfc4640 1 --2- 172.21.15.16:0/2009988345 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1a8406c2b0 0x7f1a8406e770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1a88033000 tx=0x7f1a88008030 comp rx=0 tx=0).stop 2024-09-19T07:43:31.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.065+0000 7f1a9dfc4640 1 -- 172.21.15.16:0/2009988345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a980cdf50 msgr2=0x7f1a980ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:31.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.065+0000 7f1a9dfc4640 1 --2- 172.21.15.16:0/2009988345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a980cdf50 0x7f1a980ce3d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00f610 tx=0x7f1a8c00fad0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.065+0000 7f1a9dfc4640 1 -- 172.21.15.16:0/2009988345 shutdown_connections 2024-09-19T07:43:31.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.065+0000 7f1a9dfc4640 1 --2- 172.21.15.16:0/2009988345 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1a8406c2b0 0x7f1a8406e770 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.065+0000 7f1a9dfc4640 1 --2- 172.21.15.16:0/2009988345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a980ca320 0x7f1a980ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.065+0000 7f1a9dfc4640 1 --2- 172.21.15.16:0/2009988345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a980cdf50 0x7f1a980ce3d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.065+0000 7f1a9dfc4640 1 --2- 172.21.15.16:0/2009988345 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a800039e0 0x7f1a9815d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.065+0000 7f1a9dfc4640 1 -- 172.21.15.16:0/2009988345 >> 172.21.15.16:0/2009988345 conn(0x7f1a9809d760 msgr2=0x7f1a980b9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.065+0000 7f1a9dfc4640 1 -- 172.21.15.16:0/2009988345 shutdown_connections 2024-09-19T07:43:31.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.065+0000 7f1a9dfc4640 1 -- 172.21.15.16:0/2009988345 wait complete. 2024-09-19T07:43:31.077 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-09-19T07:43:30.463820+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236216,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738504,"statfs":{"total":773094113280,"available":772852228096,"internally_reserved":0,"allocated":3416064,"data_stored":2041588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003525"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656052+0000","last_change":"2024-09-19T07:43:05.265202+0000","last_active":"2024-09-19T07:43:12.656052+0000","last_peered":"2024-09-19T07:43:12.656052+0000","last_clean":"2024-09-19T07:43:12.656052+0000","last_became_active":"2024-09-19T07:43:05.264871+0000","last_became_peered":"2024-09-19T07:43:05.264871+0000","last_unstale":"2024-09-19T07:43:12.656052+0000","last_undegraded":"2024-09-19T07:43:12.656052+0000","last_fullsized":"2024-09-19T07:43:12.656052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:36:39.982941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310781+0000","last_change":"2024-09-19T07:43:06.264003+0000","last_active":"2024-09-19T07:43:11.310781+0000","last_peered":"2024-09-19T07:43:11.310781+0000","last_clean":"2024-09-19T07:43:11.310781+0000","last_became_active":"2024-09-19T07:43:06.262313+0000","last_became_peered":"2024-09-19T07:43:06.262313+0000","last_unstale":"2024-09-19T07:43:11.310781+0000","last_undegraded":"2024-09-19T07:43:11.310781+0000","last_fullsized":"2024-09-19T07:43:11.310781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.309855+0000","last_change":"2024-09-19T07:43:10.483058+0000","last_active":"2024-09-19T07:43:11.309855+0000","last_peered":"2024-09-19T07:43:11.309855+0000","last_clean":"2024-09-19T07:43:11.309855+0000","last_became_active":"2024-09-19T07:43:10.482621+0000","last_became_peered":"2024-09-19T07:43:10.482621+0000","last_unstale":"2024-09-19T07:43:11.309855+0000","last_undegraded":"2024-09-19T07:43:11.309855+0000","last_fullsized":"2024-09-19T07:43:11.309855+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:27.168500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310785+0000","last_change":"2024-09-19T07:43:05.265029+0000","last_active":"2024-09-19T07:43:11.310785+0000","last_peered":"2024-09-19T07:43:11.310785+0000","last_clean":"2024-09-19T07:43:11.310785+0000","last_became_active":"2024-09-19T07:43:05.264529+0000","last_became_peered":"2024-09-19T07:43:05.264529+0000","last_unstale":"2024-09-19T07:43:11.310785+0000","last_undegraded":"2024-09-19T07:43:11.310785+0000","last_fullsized":"2024-09-19T07:43:11.310785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:12.998698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655877+0000","last_change":"2024-09-19T07:43:06.272531+0000","last_active":"2024-09-19T07:43:12.655877+0000","last_peered":"2024-09-19T07:43:12.655877+0000","last_clean":"2024-09-19T07:43:12.655877+0000","last_became_active":"2024-09-19T07:43:06.272225+0000","last_became_peered":"2024-09-19T07:43:06.272225+0000","last_unstale":"2024-09-19T07:43:12.655877+0000","last_undegraded":"2024-09-19T07:43:12.655877+0000","last_fullsized":"2024-09-19T07:43:12.655877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:06.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311381+0000","last_change":"2024-09-19T07:43:10.288276+0000","last_active":"2024-09-19T07:43:11.311381+0000","last_peered":"2024-09-19T07:43:11.311381+0000","last_clean":"2024-09-19T07:43:11.311381+0000","last_became_active":"2024-09-19T07:43:10.287816+0000","last_became_peered":"2024-09-19T07:43:10.287816+0000","last_unstale":"2024-09-19T07:43:11.311381+0000","last_undegraded":"2024-09-19T07:43:11.311381+0000","last_fullsized":"2024-09-19T07:43:11.311381+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:26.830106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681715+0000","last_change":"2024-09-19T07:43:05.266982+0000","last_active":"2024-09-19T07:43:12.681715+0000","last_peered":"2024-09-19T07:43:12.681715+0000","last_clean":"2024-09-19T07:43:12.681715+0000","last_became_active":"2024-09-19T07:43:05.266667+0000","last_became_peered":"2024-09-19T07:43:05.266667+0000","last_unstale":"2024-09-19T07:43:12.681715+0000","last_undegraded":"2024-09-19T07:43:12.681715+0000","last_fullsized":"2024-09-19T07:43:12.681715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:02.488778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312020+0000","last_change":"2024-09-19T07:43:06.273456+0000","last_active":"2024-09-19T07:43:11.312020+0000","last_peered":"2024-09-19T07:43:11.312020+0000","last_clean":"2024-09-19T07:43:11.312020+0000","last_became_active":"2024-09-19T07:43:06.273046+0000","last_became_peered":"2024-09-19T07:43:06.273046+0000","last_unstale":"2024-09-19T07:43:11.312020+0000","last_undegraded":"2024-09-19T07:43:11.312020+0000","last_fullsized":"2024-09-19T07:43:11.312020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:09.222719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310977+0000","last_change":"2024-09-19T07:43:10.293267+0000","last_active":"2024-09-19T07:43:11.310977+0000","last_peered":"2024-09-19T07:43:11.310977+0000","last_clean":"2024-09-19T07:43:11.310977+0000","last_became_active":"2024-09-19T07:43:10.292870+0000","last_became_peered":"2024-09-19T07:43:10.292870+0000","last_unstale":"2024-09-19T07:43:11.310977+0000","last_undegraded":"2024-09-19T07:43:11.310977+0000","last_fullsized":"2024-09-19T07:43:11.310977+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:54.570005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681497+0000","last_change":"2024-09-19T07:43:05.267905+0000","last_active":"2024-09-19T07:43:12.681497+0000","last_peered":"2024-09-19T07:43:12.681497+0000","last_clean":"2024-09-19T07:43:12.681497+0000","last_became_active":"2024-09-19T07:43:05.267563+0000","last_became_peered":"2024-09-19T07:43:05.267563+0000","last_unstale":"2024-09-19T07:43:12.681497+0000","last_undegraded":"2024-09-19T07:43:12.681497+0000","last_fullsized":"2024-09-19T07:43:12.681497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:47.531702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311784+0000","last_change":"2024-09-19T07:43:06.262966+0000","last_active":"2024-09-19T07:43:11.311784+0000","last_peered":"2024-09-19T07:43:11.311784+0000","last_clean":"2024-09-19T07:43:11.311784+0000","last_became_active":"2024-09-19T07:43:06.262698+0000","last_became_peered":"2024-09-19T07:43:06.262698+0000","last_unstale":"2024-09-19T07:43:11.311784+0000","last_undegraded":"2024-09-19T07:43:11.311784+0000","last_fullsized":"2024-09-19T07:43:11.311784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:47.918580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310630+0000","last_change":"2024-09-19T07:43:10.287084+0000","last_active":"2024-09-19T07:43:11.310630+0000","last_peered":"2024-09-19T07:43:11.310630+0000","last_clean":"2024-09-19T07:43:11.310630+0000","last_became_active":"2024-09-19T07:43:10.286693+0000","last_became_peered":"2024-09-19T07:43:10.286693+0000","last_unstale":"2024-09-19T07:43:11.310630+0000","last_undegraded":"2024-09-19T07:43:11.310630+0000","last_fullsized":"2024-09-19T07:43:11.310630+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:17.611361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450546+0000","last_change":"2024-09-19T07:43:06.262664+0000","last_active":"2024-09-19T07:43:11.450546+0000","last_peered":"2024-09-19T07:43:11.450546+0000","last_clean":"2024-09-19T07:43:11.450546+0000","last_became_active":"2024-09-19T07:43:06.262389+0000","last_became_peered":"2024-09-19T07:43:06.262389+0000","last_unstale":"2024-09-19T07:43:11.450546+0000","last_undegraded":"2024-09-19T07:43:11.450546+0000","last_fullsized":"2024-09-19T07:43:11.450546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:06.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681454+0000","last_change":"2024-09-19T07:43:05.262431+0000","last_active":"2024-09-19T07:43:12.681454+0000","last_peered":"2024-09-19T07:43:12.681454+0000","last_clean":"2024-09-19T07:43:12.681454+0000","last_became_active":"2024-09-19T07:43:05.262156+0000","last_became_peered":"2024-09-19T07:43:05.262156+0000","last_unstale":"2024-09-19T07:43:12.681454+0000","last_undegraded":"2024-09-19T07:43:12.681454+0000","last_fullsized":"2024-09-19T07:43:12.681454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:27.338560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312544+0000","last_change":"2024-09-19T07:43:10.287687+0000","last_active":"2024-09-19T07:43:11.312544+0000","last_peered":"2024-09-19T07:43:11.312544+0000","last_clean":"2024-09-19T07:43:11.312544+0000","last_became_active":"2024-09-19T07:43:10.287301+0000","last_became_peered":"2024-09-19T07:43:10.287301+0000","last_unstale":"2024-09-19T07:43:11.312544+0000","last_undegraded":"2024-09-19T07:43:11.312544+0000","last_fullsized":"2024-09-19T07:43:11.312544+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:43.684480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310291+0000","last_change":"2024-09-19T07:43:06.274918+0000","last_active":"2024-09-19T07:43:11.310291+0000","last_peered":"2024-09-19T07:43:11.310291+0000","last_clean":"2024-09-19T07:43:11.310291+0000","last_became_active":"2024-09-19T07:43:06.274639+0000","last_became_peered":"2024-09-19T07:43:06.274639+0000","last_unstale":"2024-09-19T07:43:11.310291+0000","last_undegraded":"2024-09-19T07:43:11.310291+0000","last_fullsized":"2024-09-19T07:43:11.310291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:22.804370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680919+0000","last_change":"2024-09-19T07:43:05.258127+0000","last_active":"2024-09-19T07:43:12.680919+0000","last_peered":"2024-09-19T07:43:12.680919+0000","last_clean":"2024-09-19T07:43:12.680919+0000","last_became_active":"2024-09-19T07:43:05.257766+0000","last_became_peered":"2024-09-19T07:43:05.257766+0000","last_unstale":"2024-09-19T07:43:12.680919+0000","last_undegraded":"2024-09-19T07:43:12.680919+0000","last_fullsized":"2024-09-19T07:43:12.680919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:02.488778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311080+0000","last_change":"2024-09-19T07:43:10.294352+0000","last_active":"2024-09-19T07:43:11.311080+0000","last_peered":"2024-09-19T07:43:11.311080+0000","last_clean":"2024-09-19T07:43:11.311080+0000","last_became_active":"2024-09-19T07:43:10.293957+0000","last_became_peered":"2024-09-19T07:43:10.293957+0000","last_unstale":"2024-09-19T07:43:11.311080+0000","last_undegraded":"2024-09-19T07:43:11.311080+0000","last_fullsized":"2024-09-19T07:43:11.311080+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:13.248454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310160+0000","last_change":"2024-09-19T07:43:06.256705+0000","last_active":"2024-09-19T07:43:11.310160+0000","last_peered":"2024-09-19T07:43:11.310160+0000","last_clean":"2024-09-19T07:43:11.310160+0000","last_became_active":"2024-09-19T07:43:06.256423+0000","last_became_peered":"2024-09-19T07:43:06.256423+0000","last_unstale":"2024-09-19T07:43:11.310160+0000","last_undegraded":"2024-09-19T07:43:11.310160+0000","last_fullsized":"2024-09-19T07:43:11.310160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:47.918580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680712+0000","last_change":"2024-09-19T07:43:05.260367+0000","last_active":"2024-09-19T07:43:12.680712+0000","last_peered":"2024-09-19T07:43:12.680712+0000","last_clean":"2024-09-19T07:43:12.680712+0000","last_became_active":"2024-09-19T07:43:05.260056+0000","last_became_peered":"2024-09-19T07:43:05.260056+0000","last_unstale":"2024-09-19T07:43:12.680712+0000","last_undegraded":"2024-09-19T07:43:12.680712+0000","last_fullsized":"2024-09-19T07:43:12.680712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:38:29.212025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681342+0000","last_change":"2024-09-19T07:43:10.482020+0000","last_active":"2024-09-19T07:43:12.681342+0000","last_peered":"2024-09-19T07:43:12.681342+0000","last_clean":"2024-09-19T07:43:12.681342+0000","last_became_active":"2024-09-19T07:43:10.480883+0000","last_became_peered":"2024-09-19T07:43:10.480883+0000","last_unstale":"2024-09-19T07:43:12.681342+0000","last_undegraded":"2024-09-19T07:43:12.681342+0000","last_fullsized":"2024-09-19T07:43:12.681342+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:01.954262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312259+0000","last_change":"2024-09-19T07:43:06.268719+0000","last_active":"2024-09-19T07:43:11.312259+0000","last_peered":"2024-09-19T07:43:11.312259+0000","last_clean":"2024-09-19T07:43:11.312259+0000","last_became_active":"2024-09-19T07:43:06.268405+0000","last_became_peered":"2024-09-19T07:43:06.268405+0000","last_unstale":"2024-09-19T07:43:11.312259+0000","last_undegraded":"2024-09-19T07:43:11.312259+0000","last_fullsized":"2024-09-19T07:43:11.312259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:45.364876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681025+0000","last_change":"2024-09-19T07:43:05.267962+0000","last_active":"2024-09-19T07:43:12.681025+0000","last_peered":"2024-09-19T07:43:12.681025+0000","last_clean":"2024-09-19T07:43:12.681025+0000","last_became_active":"2024-09-19T07:43:05.267670+0000","last_became_peered":"2024-09-19T07:43:05.267670+0000","last_unstale":"2024-09-19T07:43:12.681025+0000","last_undegraded":"2024-09-19T07:43:12.681025+0000","last_fullsized":"2024-09-19T07:43:12.681025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:13.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312584+0000","last_change":"2024-09-19T07:43:10.289357+0000","last_active":"2024-09-19T07:43:11.312584+0000","last_peered":"2024-09-19T07:43:11.312584+0000","last_clean":"2024-09-19T07:43:11.312584+0000","last_became_active":"2024-09-19T07:43:10.288596+0000","last_became_peered":"2024-09-19T07:43:10.288596+0000","last_unstale":"2024-09-19T07:43:11.312584+0000","last_undegraded":"2024-09-19T07:43:11.312584+0000","last_fullsized":"2024-09-19T07:43:11.312584+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:08.193959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656239+0000","last_change":"2024-09-19T07:43:06.270766+0000","last_active":"2024-09-19T07:43:12.656239+0000","last_peered":"2024-09-19T07:43:12.656239+0000","last_clean":"2024-09-19T07:43:12.656239+0000","last_became_active":"2024-09-19T07:43:06.270494+0000","last_became_peered":"2024-09-19T07:43:06.270494+0000","last_unstale":"2024-09-19T07:43:12.656239+0000","last_undegraded":"2024-09-19T07:43:12.656239+0000","last_fullsized":"2024-09-19T07:43:12.656239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451242+0000","last_change":"2024-09-19T07:43:05.263058+0000","last_active":"2024-09-19T07:43:11.451242+0000","last_peered":"2024-09-19T07:43:11.451242+0000","last_clean":"2024-09-19T07:43:11.451242+0000","last_became_active":"2024-09-19T07:43:05.262762+0000","last_became_peered":"2024-09-19T07:43:05.262762+0000","last_unstale":"2024-09-19T07:43:11.451242+0000","last_undegraded":"2024-09-19T07:43:11.451242+0000","last_fullsized":"2024-09-19T07:43:11.451242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:38:29.212025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312088+0000","last_change":"2024-09-19T07:43:10.482293+0000","last_active":"2024-09-19T07:43:11.312088+0000","last_peered":"2024-09-19T07:43:11.312088+0000","last_clean":"2024-09-19T07:43:11.312088+0000","last_became_active":"2024-09-19T07:43:10.481268+0000","last_became_peered":"2024-09-19T07:43:10.481268+0000","last_unstale":"2024-09-19T07:43:11.312088+0000","last_undegraded":"2024-09-19T07:43:11.312088+0000","last_fullsized":"2024-09-19T07:43:11.312088+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:51.944316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311136+0000","last_change":"2024-09-19T07:43:06.268026+0000","last_active":"2024-09-19T07:43:11.311136+0000","last_peered":"2024-09-19T07:43:11.311136+0000","last_clean":"2024-09-19T07:43:11.311136+0000","last_became_active":"2024-09-19T07:43:06.267713+0000","last_became_peered":"2024-09-19T07:43:06.267713+0000","last_unstale":"2024-09-19T07:43:11.311136+0000","last_undegraded":"2024-09-19T07:43:11.311136+0000","last_fullsized":"2024-09-19T07:43:11.311136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:56.380341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681209+0000","last_change":"2024-09-19T07:43:05.263248+0000","last_active":"2024-09-19T07:43:12.681209+0000","last_peered":"2024-09-19T07:43:12.681209+0000","last_clean":"2024-09-19T07:43:12.681209+0000","last_became_active":"2024-09-19T07:43:05.262892+0000","last_became_peered":"2024-09-19T07:43:05.262892+0000","last_unstale":"2024-09-19T07:43:12.681209+0000","last_undegraded":"2024-09-19T07:43:12.681209+0000","last_fullsized":"2024-09-19T07:43:12.681209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:47.531702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.309826+0000","last_change":"2024-09-19T07:43:10.482975+0000","last_active":"2024-09-19T07:43:11.309826+0000","last_peered":"2024-09-19T07:43:11.309826+0000","last_clean":"2024-09-19T07:43:11.309826+0000","last_became_active":"2024-09-19T07:43:10.482584+0000","last_became_peered":"2024-09-19T07:43:10.482584+0000","last_unstale":"2024-09-19T07:43:11.309826+0000","last_undegraded":"2024-09-19T07:43:11.309826+0000","last_fullsized":"2024-09-19T07:43:11.309826+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:10.437589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680841+0000","last_change":"2024-09-19T07:43:06.258826+0000","last_active":"2024-09-19T07:43:12.680841+0000","last_peered":"2024-09-19T07:43:12.680841+0000","last_clean":"2024-09-19T07:43:12.680841+0000","last_became_active":"2024-09-19T07:43:06.258518+0000","last_became_peered":"2024-09-19T07:43:06.258518+0000","last_unstale":"2024-09-19T07:43:12.680841+0000","last_undegraded":"2024-09-19T07:43:12.680841+0000","last_fullsized":"2024-09-19T07:43:12.680841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681832+0000","last_change":"2024-09-19T07:43:05.267167+0000","last_active":"2024-09-19T07:43:12.681832+0000","last_peered":"2024-09-19T07:43:12.681832+0000","last_clean":"2024-09-19T07:43:12.681832+0000","last_became_active":"2024-09-19T07:43:05.266892+0000","last_became_peered":"2024-09-19T07:43:05.266892+0000","last_unstale":"2024-09-19T07:43:12.681832+0000","last_undegraded":"2024-09-19T07:43:12.681832+0000","last_fullsized":"2024-09-19T07:43:12.681832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:06.889703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655769+0000","last_change":"2024-09-19T07:43:10.480657+0000","last_active":"2024-09-19T07:43:12.655769+0000","last_peered":"2024-09-19T07:43:12.655769+0000","last_clean":"2024-09-19T07:43:12.655769+0000","last_became_active":"2024-09-19T07:43:10.480186+0000","last_became_peered":"2024-09-19T07:43:10.480186+0000","last_unstale":"2024-09-19T07:43:12.655769+0000","last_undegraded":"2024-09-19T07:43:12.655769+0000","last_fullsized":"2024-09-19T07:43:12.655769+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:43.490603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311921+0000","last_change":"2024-09-19T07:43:06.265266+0000","last_active":"2024-09-19T07:43:11.311921+0000","last_peered":"2024-09-19T07:43:11.311921+0000","last_clean":"2024-09-19T07:43:11.311921+0000","last_became_active":"2024-09-19T07:43:06.265001+0000","last_became_peered":"2024-09-19T07:43:06.265001+0000","last_unstale":"2024-09-19T07:43:11.311921+0000","last_undegraded":"2024-09-19T07:43:11.311921+0000","last_fullsized":"2024-09-19T07:43:11.311921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:39.464868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680965+0000","last_change":"2024-09-19T07:43:05.259114+0000","last_active":"2024-09-19T07:43:12.680965+0000","last_peered":"2024-09-19T07:43:12.680965+0000","last_clean":"2024-09-19T07:43:12.680965+0000","last_became_active":"2024-09-19T07:43:05.258794+0000","last_became_peered":"2024-09-19T07:43:05.258794+0000","last_unstale":"2024-09-19T07:43:12.680965+0000","last_undegraded":"2024-09-19T07:43:12.680965+0000","last_fullsized":"2024-09-19T07:43:12.680965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:13.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680778+0000","last_change":"2024-09-19T07:43:10.295795+0000","last_active":"2024-09-19T07:43:12.680778+0000","last_peered":"2024-09-19T07:43:12.680778+0000","last_clean":"2024-09-19T07:43:12.680778+0000","last_became_active":"2024-09-19T07:43:10.295379+0000","last_became_peered":"2024-09-19T07:43:10.295379+0000","last_unstale":"2024-09-19T07:43:12.680778+0000","last_undegraded":"2024-09-19T07:43:12.680778+0000","last_fullsized":"2024-09-19T07:43:12.680778+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:43.490603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311661+0000","last_change":"2024-09-19T07:43:06.271554+0000","last_active":"2024-09-19T07:43:11.311661+0000","last_peered":"2024-09-19T07:43:11.311661+0000","last_clean":"2024-09-19T07:43:11.311661+0000","last_became_active":"2024-09-19T07:43:06.271304+0000","last_became_peered":"2024-09-19T07:43:06.271304+0000","last_unstale":"2024-09-19T07:43:11.311661+0000","last_undegraded":"2024-09-19T07:43:11.311661+0000","last_fullsized":"2024-09-19T07:43:11.311661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:03.494318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311677+0000","last_change":"2024-09-19T07:43:05.264548+0000","last_active":"2024-09-19T07:43:11.311677+0000","last_peered":"2024-09-19T07:43:11.311677+0000","last_clean":"2024-09-19T07:43:11.311677+0000","last_became_active":"2024-09-19T07:43:05.261674+0000","last_became_peered":"2024-09-19T07:43:05.261674+0000","last_unstale":"2024-09-19T07:43:11.311677+0000","last_undegraded":"2024-09-19T07:43:11.311677+0000","last_fullsized":"2024-09-19T07:43:11.311677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:47.531702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681063+0000","last_change":"2024-09-19T07:43:10.288751+0000","last_active":"2024-09-19T07:43:12.681063+0000","last_peered":"2024-09-19T07:43:12.681063+0000","last_clean":"2024-09-19T07:43:12.681063+0000","last_became_active":"2024-09-19T07:43:10.288383+0000","last_became_peered":"2024-09-19T07:43:10.288383+0000","last_unstale":"2024-09-19T07:43:12.681063+0000","last_undegraded":"2024-09-19T07:43:12.681063+0000","last_fullsized":"2024-09-19T07:43:12.681063+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:43.944005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680858+0000","last_change":"2024-09-19T07:43:06.264468+0000","last_active":"2024-09-19T07:43:12.680858+0000","last_peered":"2024-09-19T07:43:12.680858+0000","last_clean":"2024-09-19T07:43:12.680858+0000","last_became_active":"2024-09-19T07:43:06.263801+0000","last_became_peered":"2024-09-19T07:43:06.263801+0000","last_unstale":"2024-09-19T07:43:12.680858+0000","last_undegraded":"2024-09-19T07:43:12.680858+0000","last_fullsized":"2024-09-19T07:43:12.680858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:06.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588628+0000","last_change":"2024-09-19T07:43:05.259922+0000","last_active":"2024-09-19T07:43:11.588628+0000","last_peered":"2024-09-19T07:43:11.588628+0000","last_clean":"2024-09-19T07:43:11.588628+0000","last_became_active":"2024-09-19T07:43:05.259552+0000","last_became_peered":"2024-09-19T07:43:05.259552+0000","last_unstale":"2024-09-19T07:43:11.588628+0000","last_undegraded":"2024-09-19T07:43:11.588628+0000","last_fullsized":"2024-09-19T07:43:11.588628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:55.737753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681498+0000","last_change":"2024-09-19T07:43:10.296047+0000","last_active":"2024-09-19T07:43:12.681498+0000","last_peered":"2024-09-19T07:43:12.681498+0000","last_clean":"2024-09-19T07:43:12.681498+0000","last_became_active":"2024-09-19T07:43:10.295636+0000","last_became_peered":"2024-09-19T07:43:10.295636+0000","last_unstale":"2024-09-19T07:43:12.681498+0000","last_undegraded":"2024-09-19T07:43:12.681498+0000","last_fullsized":"2024-09-19T07:43:12.681498+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:27.138783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655596+0000","last_change":"2024-09-19T07:43:06.273785+0000","last_active":"2024-09-19T07:43:12.655596+0000","last_peered":"2024-09-19T07:43:12.655596+0000","last_clean":"2024-09-19T07:43:12.655596+0000","last_became_active":"2024-09-19T07:43:06.273514+0000","last_became_peered":"2024-09-19T07:43:06.273514+0000","last_unstale":"2024-09-19T07:43:12.655596+0000","last_undegraded":"2024-09-19T07:43:12.655596+0000","last_fullsized":"2024-09-19T07:43:12.655596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:09.399092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681059+0000","last_change":"2024-09-19T07:43:05.263952+0000","last_active":"2024-09-19T07:43:12.681059+0000","last_peered":"2024-09-19T07:43:12.681059+0000","last_clean":"2024-09-19T07:43:12.681059+0000","last_became_active":"2024-09-19T07:43:05.263677+0000","last_became_peered":"2024-09-19T07:43:05.263677+0000","last_unstale":"2024-09-19T07:43:12.681059+0000","last_undegraded":"2024-09-19T07:43:12.681059+0000","last_fullsized":"2024-09-19T07:43:12.681059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:44.359336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656084+0000","last_change":"2024-09-19T07:43:10.480721+0000","last_active":"2024-09-19T07:43:12.656084+0000","last_peered":"2024-09-19T07:43:12.656084+0000","last_clean":"2024-09-19T07:43:12.656084+0000","last_became_active":"2024-09-19T07:43:10.480321+0000","last_became_peered":"2024-09-19T07:43:10.480321+0000","last_unstale":"2024-09-19T07:43:12.656084+0000","last_undegraded":"2024-09-19T07:43:12.656084+0000","last_fullsized":"2024-09-19T07:43:12.656084+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:01.954262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310005+0000","last_change":"2024-09-19T07:43:06.271482+0000","last_active":"2024-09-19T07:43:11.310005+0000","last_peered":"2024-09-19T07:43:11.310005+0000","last_clean":"2024-09-19T07:43:11.310005+0000","last_became_active":"2024-09-19T07:43:06.271212+0000","last_became_peered":"2024-09-19T07:43:06.271212+0000","last_unstale":"2024-09-19T07:43:11.310005+0000","last_undegraded":"2024-09-19T07:43:11.310005+0000","last_fullsized":"2024-09-19T07:43:11.310005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:27.185190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311209+0000","last_change":"2024-09-19T07:43:05.261378+0000","last_active":"2024-09-19T07:43:11.311209+0000","last_peered":"2024-09-19T07:43:11.311209+0000","last_clean":"2024-09-19T07:43:11.311209+0000","last_became_active":"2024-09-19T07:43:05.261055+0000","last_became_peered":"2024-09-19T07:43:05.261055+0000","last_unstale":"2024-09-19T07:43:11.311209+0000","last_undegraded":"2024-09-19T07:43:11.311209+0000","last_fullsized":"2024-09-19T07:43:11.311209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:12.998698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681162+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:43:12.681162+0000","last_peered":"2024-09-19T07:43:12.681162+0000","last_clean":"2024-09-19T07:43:12.681162+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:43:12.681162+0000","last_undegraded":"2024-09-19T07:43:12.681162+0000","last_fullsized":"2024-09-19T07:43:12.681162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450864+0000","last_change":"2024-09-19T07:43:10.481567+0000","last_active":"2024-09-19T07:43:11.450864+0000","last_peered":"2024-09-19T07:43:11.450864+0000","last_clean":"2024-09-19T07:43:11.450864+0000","last_became_active":"2024-09-19T07:43:10.481210+0000","last_became_peered":"2024-09-19T07:43:10.481210+0000","last_unstale":"2024-09-19T07:43:11.450864+0000","last_undegraded":"2024-09-19T07:43:11.450864+0000","last_fullsized":"2024-09-19T07:43:11.450864+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:08.193959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681552+0000","last_change":"2024-09-19T07:43:06.268655+0000","last_active":"2024-09-19T07:43:12.681552+0000","last_peered":"2024-09-19T07:43:12.681552+0000","last_clean":"2024-09-19T07:43:12.681552+0000","last_became_active":"2024-09-19T07:43:06.268387+0000","last_became_peered":"2024-09-19T07:43:06.268387+0000","last_unstale":"2024-09-19T07:43:12.681552+0000","last_undegraded":"2024-09-19T07:43:12.681552+0000","last_fullsized":"2024-09-19T07:43:12.681552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:14.347245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588993+0000","last_change":"2024-09-19T07:43:05.261374+0000","last_active":"2024-09-19T07:43:11.588993+0000","last_peered":"2024-09-19T07:43:11.588993+0000","last_clean":"2024-09-19T07:43:11.588993+0000","last_became_active":"2024-09-19T07:43:05.261088+0000","last_became_peered":"2024-09-19T07:43:05.261088+0000","last_unstale":"2024-09-19T07:43:11.588993+0000","last_undegraded":"2024-09-19T07:43:11.588993+0000","last_fullsized":"2024-09-19T07:43:11.588993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:39.630700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450612+0000","last_change":"2024-09-19T07:43:10.480274+0000","last_active":"2024-09-19T07:43:11.450612+0000","last_peered":"2024-09-19T07:43:11.450612+0000","last_clean":"2024-09-19T07:43:11.450612+0000","last_became_active":"2024-09-19T07:43:10.479848+0000","last_became_peered":"2024-09-19T07:43:10.479848+0000","last_unstale":"2024-09-19T07:43:11.450612+0000","last_undegraded":"2024-09-19T07:43:11.450612+0000","last_fullsized":"2024-09-19T07:43:11.450612+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:43.684480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312126+0000","last_change":"2024-09-19T07:43:06.264529+0000","last_active":"2024-09-19T07:43:11.312126+0000","last_peered":"2024-09-19T07:43:11.312126+0000","last_clean":"2024-09-19T07:43:11.312126+0000","last_became_active":"2024-09-19T07:43:06.264270+0000","last_became_peered":"2024-09-19T07:43:06.264270+0000","last_unstale":"2024-09-19T07:43:11.312126+0000","last_undegraded":"2024-09-19T07:43:11.312126+0000","last_fullsized":"2024-09-19T07:43:11.312126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:04.168224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681259+0000","last_change":"2024-09-19T07:43:05.254996+0000","last_active":"2024-09-19T07:43:12.681259+0000","last_peered":"2024-09-19T07:43:12.681259+0000","last_clean":"2024-09-19T07:43:12.681259+0000","last_became_active":"2024-09-19T07:43:05.254430+0000","last_became_peered":"2024-09-19T07:43:05.254430+0000","last_unstale":"2024-09-19T07:43:12.681259+0000","last_undegraded":"2024-09-19T07:43:12.681259+0000","last_fullsized":"2024-09-19T07:43:12.681259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:06.889703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656122+0000","last_change":"2024-09-19T07:43:10.295667+0000","last_active":"2024-09-19T07:43:12.656122+0000","last_peered":"2024-09-19T07:43:12.656122+0000","last_clean":"2024-09-19T07:43:12.656122+0000","last_became_active":"2024-09-19T07:43:10.295241+0000","last_became_peered":"2024-09-19T07:43:10.295241+0000","last_unstale":"2024-09-19T07:43:12.656122+0000","last_undegraded":"2024-09-19T07:43:12.656122+0000","last_fullsized":"2024-09-19T07:43:12.656122+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:27.138783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681804+0000","last_change":"2024-09-19T07:43:06.275720+0000","last_active":"2024-09-19T07:43:12.681804+0000","last_peered":"2024-09-19T07:43:12.681804+0000","last_clean":"2024-09-19T07:43:12.681804+0000","last_became_active":"2024-09-19T07:43:06.275403+0000","last_became_peered":"2024-09-19T07:43:06.275403+0000","last_unstale":"2024-09-19T07:43:12.681804+0000","last_undegraded":"2024-09-19T07:43:12.681804+0000","last_fullsized":"2024-09-19T07:43:12.681804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:39.918269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588845+0000","last_change":"2024-09-19T07:43:05.260217+0000","last_active":"2024-09-19T07:43:11.588845+0000","last_peered":"2024-09-19T07:43:11.588845+0000","last_clean":"2024-09-19T07:43:11.588845+0000","last_became_active":"2024-09-19T07:43:05.259482+0000","last_became_peered":"2024-09-19T07:43:05.259482+0000","last_unstale":"2024-09-19T07:43:11.588845+0000","last_undegraded":"2024-09-19T07:43:11.588845+0000","last_fullsized":"2024-09-19T07:43:11.588845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:56.644264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310513+0000","last_change":"2024-09-19T07:43:10.480657+0000","last_active":"2024-09-19T07:43:11.310513+0000","last_peered":"2024-09-19T07:43:11.310513+0000","last_clean":"2024-09-19T07:43:11.310513+0000","last_became_active":"2024-09-19T07:43:10.480279+0000","last_became_peered":"2024-09-19T07:43:10.480279+0000","last_unstale":"2024-09-19T07:43:11.310513+0000","last_undegraded":"2024-09-19T07:43:11.310513+0000","last_fullsized":"2024-09-19T07:43:11.310513+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:26.933433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681655+0000","last_change":"2024-09-19T07:43:06.275354+0000","last_active":"2024-09-19T07:43:12.681655+0000","last_peered":"2024-09-19T07:43:12.681655+0000","last_clean":"2024-09-19T07:43:12.681655+0000","last_became_active":"2024-09-19T07:43:06.275039+0000","last_became_peered":"2024-09-19T07:43:06.275039+0000","last_unstale":"2024-09-19T07:43:12.681655+0000","last_undegraded":"2024-09-19T07:43:12.681655+0000","last_fullsized":"2024-09-19T07:43:12.681655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:47.918580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311619+0000","last_change":"2024-09-19T07:43:05.265977+0000","last_active":"2024-09-19T07:43:11.311619+0000","last_peered":"2024-09-19T07:43:11.311619+0000","last_clean":"2024-09-19T07:43:11.311619+0000","last_became_active":"2024-09-19T07:43:05.265615+0000","last_became_peered":"2024-09-19T07:43:05.265615+0000","last_unstale":"2024-09-19T07:43:11.311619+0000","last_undegraded":"2024-09-19T07:43:11.311619+0000","last_fullsized":"2024-09-19T07:43:11.311619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:20:34.475024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310868+0000","last_change":"2024-09-19T07:43:10.283896+0000","last_active":"2024-09-19T07:43:11.310868+0000","last_peered":"2024-09-19T07:43:11.310868+0000","last_clean":"2024-09-19T07:43:11.310868+0000","last_became_active":"2024-09-19T07:43:10.283587+0000","last_became_peered":"2024-09-19T07:43:10.283587+0000","last_unstale":"2024-09-19T07:43:11.310868+0000","last_undegraded":"2024-09-19T07:43:11.310868+0000","last_fullsized":"2024-09-19T07:43:11.310868+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:41.387226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681270+0000","last_change":"2024-09-19T07:43:06.274315+0000","last_active":"2024-09-19T07:43:12.681270+0000","last_peered":"2024-09-19T07:43:12.681270+0000","last_clean":"2024-09-19T07:43:12.681270+0000","last_became_active":"2024-09-19T07:43:06.274064+0000","last_became_peered":"2024-09-19T07:43:06.274064+0000","last_unstale":"2024-09-19T07:43:12.681270+0000","last_undegraded":"2024-09-19T07:43:12.681270+0000","last_fullsized":"2024-09-19T07:43:12.681270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:22.804370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588519+0000","last_change":"2024-09-19T07:43:05.259083+0000","last_active":"2024-09-19T07:43:11.588519+0000","last_peered":"2024-09-19T07:43:11.588519+0000","last_clean":"2024-09-19T07:43:11.588519+0000","last_became_active":"2024-09-19T07:43:05.258769+0000","last_became_peered":"2024-09-19T07:43:05.258769+0000","last_unstale":"2024-09-19T07:43:11.588519+0000","last_undegraded":"2024-09-19T07:43:11.588519+0000","last_fullsized":"2024-09-19T07:43:11.588519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:51:24.580949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656048+0000","last_change":"2024-09-19T07:43:10.290261+0000","last_active":"2024-09-19T07:43:12.656048+0000","last_peered":"2024-09-19T07:43:12.656048+0000","last_clean":"2024-09-19T07:43:12.656048+0000","last_became_active":"2024-09-19T07:43:10.289428+0000","last_became_peered":"2024-09-19T07:43:10.289428+0000","last_unstale":"2024-09-19T07:43:12.656048+0000","last_undegraded":"2024-09-19T07:43:12.656048+0000","last_fullsized":"2024-09-19T07:43:12.656048+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:43.944005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450559+0000","last_change":"2024-09-19T07:43:06.276364+0000","last_active":"2024-09-19T07:43:11.450559+0000","last_peered":"2024-09-19T07:43:11.450559+0000","last_clean":"2024-09-19T07:43:11.450559+0000","last_became_active":"2024-09-19T07:43:06.275564+0000","last_became_peered":"2024-09-19T07:43:06.275564+0000","last_unstale":"2024-09-19T07:43:11.450559+0000","last_undegraded":"2024-09-19T07:43:11.450559+0000","last_fullsized":"2024-09-19T07:43:11.450559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450611+0000","last_change":"2024-09-19T07:43:05.264487+0000","last_active":"2024-09-19T07:43:11.450611+0000","last_peered":"2024-09-19T07:43:11.450611+0000","last_clean":"2024-09-19T07:43:11.450611+0000","last_became_active":"2024-09-19T07:43:05.264196+0000","last_became_peered":"2024-09-19T07:43:05.264196+0000","last_unstale":"2024-09-19T07:43:11.450611+0000","last_undegraded":"2024-09-19T07:43:11.450611+0000","last_fullsized":"2024-09-19T07:43:11.450611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:47.531702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311055+0000","last_change":"2024-09-19T07:43:10.481930+0000","last_active":"2024-09-19T07:43:11.311055+0000","last_peered":"2024-09-19T07:43:11.311055+0000","last_clean":"2024-09-19T07:43:11.311055+0000","last_became_active":"2024-09-19T07:43:10.481552+0000","last_became_peered":"2024-09-19T07:43:10.481552+0000","last_unstale":"2024-09-19T07:43:11.311055+0000","last_undegraded":"2024-09-19T07:43:11.311055+0000","last_fullsized":"2024-09-19T07:43:11.311055+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:19.523345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681307+0000","last_change":"2024-09-19T07:43:06.275131+0000","last_active":"2024-09-19T07:43:12.681307+0000","last_peered":"2024-09-19T07:43:12.681307+0000","last_clean":"2024-09-19T07:43:12.681307+0000","last_became_active":"2024-09-19T07:43:06.274857+0000","last_became_peered":"2024-09-19T07:43:06.274857+0000","last_unstale":"2024-09-19T07:43:12.681307+0000","last_undegraded":"2024-09-19T07:43:12.681307+0000","last_fullsized":"2024-09-19T07:43:12.681307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:57.928527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311569+0000","last_change":"2024-09-19T07:43:05.263838+0000","last_active":"2024-09-19T07:43:11.311569+0000","last_peered":"2024-09-19T07:43:11.311569+0000","last_clean":"2024-09-19T07:43:11.311569+0000","last_became_active":"2024-09-19T07:43:05.263484+0000","last_became_peered":"2024-09-19T07:43:05.263484+0000","last_unstale":"2024-09-19T07:43:11.311569+0000","last_undegraded":"2024-09-19T07:43:11.311569+0000","last_fullsized":"2024-09-19T07:43:11.311569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:38.098052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310652+0000","last_change":"2024-09-19T07:43:10.480636+0000","last_active":"2024-09-19T07:43:11.310652+0000","last_peered":"2024-09-19T07:43:11.310652+0000","last_clean":"2024-09-19T07:43:11.310652+0000","last_became_active":"2024-09-19T07:43:10.480217+0000","last_became_peered":"2024-09-19T07:43:10.480217+0000","last_unstale":"2024-09-19T07:43:11.310652+0000","last_undegraded":"2024-09-19T07:43:11.310652+0000","last_fullsized":"2024-09-19T07:43:11.310652+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:29.825328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656111+0000","last_change":"2024-09-19T07:43:06.272020+0000","last_active":"2024-09-19T07:43:12.656111+0000","last_peered":"2024-09-19T07:43:12.656111+0000","last_clean":"2024-09-19T07:43:12.656111+0000","last_became_active":"2024-09-19T07:43:06.271671+0000","last_became_peered":"2024-09-19T07:43:06.271671+0000","last_unstale":"2024-09-19T07:43:12.656111+0000","last_undegraded":"2024-09-19T07:43:12.656111+0000","last_fullsized":"2024-09-19T07:43:12.656111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:24:44.612992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656246+0000","last_change":"2024-09-19T07:43:05.265898+0000","last_active":"2024-09-19T07:43:12.656246+0000","last_peered":"2024-09-19T07:43:12.656246+0000","last_clean":"2024-09-19T07:43:12.656246+0000","last_became_active":"2024-09-19T07:43:05.265595+0000","last_became_peered":"2024-09-19T07:43:05.265595+0000","last_unstale":"2024-09-19T07:43:12.656246+0000","last_undegraded":"2024-09-19T07:43:12.656246+0000","last_fullsized":"2024-09-19T07:43:12.656246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:22.696534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311727+0000","last_change":"2024-09-19T07:43:10.482166+0000","last_active":"2024-09-19T07:43:11.311727+0000","last_peered":"2024-09-19T07:43:11.311727+0000","last_clean":"2024-09-19T07:43:11.311727+0000","last_became_active":"2024-09-19T07:43:10.481667+0000","last_became_peered":"2024-09-19T07:43:10.481667+0000","last_unstale":"2024-09-19T07:43:11.311727+0000","last_undegraded":"2024-09-19T07:43:11.311727+0000","last_fullsized":"2024-09-19T07:43:11.311727+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:10.437589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681470+0000","last_change":"2024-09-19T07:43:06.277189+0000","last_active":"2024-09-19T07:43:12.681470+0000","last_peered":"2024-09-19T07:43:12.681470+0000","last_clean":"2024-09-19T07:43:12.681470+0000","last_became_active":"2024-09-19T07:43:06.276924+0000","last_became_peered":"2024-09-19T07:43:06.276924+0000","last_unstale":"2024-09-19T07:43:12.681470+0000","last_undegraded":"2024-09-19T07:43:12.681470+0000","last_fullsized":"2024-09-19T07:43:12.681470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:39.658744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:15.766406+0000","last_change":"2024-09-19T07:43:05.266360+0000","last_active":"2024-09-19T07:43:15.766406+0000","last_peered":"2024-09-19T07:43:15.766406+0000","last_clean":"2024-09-19T07:43:15.766406+0000","last_became_active":"2024-09-19T07:43:05.266038+0000","last_became_peered":"2024-09-19T07:43:05.266038+0000","last_unstale":"2024-09-19T07:43:15.766406+0000","last_undegraded":"2024-09-19T07:43:15.766406+0000","last_fullsized":"2024-09-19T07:43:15.766406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:13.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680836+0000","last_change":"2024-09-19T07:43:10.295894+0000","last_active":"2024-09-19T07:43:12.680836+0000","last_peered":"2024-09-19T07:43:12.680836+0000","last_clean":"2024-09-19T07:43:12.680836+0000","last_became_active":"2024-09-19T07:43:10.295496+0000","last_became_peered":"2024-09-19T07:43:10.295496+0000","last_unstale":"2024-09-19T07:43:12.680836+0000","last_undegraded":"2024-09-19T07:43:12.680836+0000","last_fullsized":"2024-09-19T07:43:12.680836+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:18.372981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312405+0000","last_change":"2024-09-19T07:43:06.277639+0000","last_active":"2024-09-19T07:43:11.312405+0000","last_peered":"2024-09-19T07:43:11.312405+0000","last_clean":"2024-09-19T07:43:11.312405+0000","last_became_active":"2024-09-19T07:43:06.277186+0000","last_became_peered":"2024-09-19T07:43:06.277186+0000","last_unstale":"2024-09-19T07:43:11.312405+0000","last_undegraded":"2024-09-19T07:43:11.312405+0000","last_fullsized":"2024-09-19T07:43:11.312405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:07.895243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681210+0000","last_change":"2024-09-19T07:43:05.257416+0000","last_active":"2024-09-19T07:43:12.681210+0000","last_peered":"2024-09-19T07:43:12.681210+0000","last_clean":"2024-09-19T07:43:12.681210+0000","last_became_active":"2024-09-19T07:43:05.256941+0000","last_became_peered":"2024-09-19T07:43:05.256941+0000","last_unstale":"2024-09-19T07:43:12.681210+0000","last_undegraded":"2024-09-19T07:43:12.681210+0000","last_fullsized":"2024-09-19T07:43:12.681210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:44.359336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311217+0000","last_change":"2024-09-19T07:43:10.288931+0000","last_active":"2024-09-19T07:43:11.311217+0000","last_peered":"2024-09-19T07:43:11.311217+0000","last_clean":"2024-09-19T07:43:11.311217+0000","last_became_active":"2024-09-19T07:43:10.288548+0000","last_became_peered":"2024-09-19T07:43:10.288548+0000","last_unstale":"2024-09-19T07:43:11.311217+0000","last_undegraded":"2024-09-19T07:43:11.311217+0000","last_fullsized":"2024-09-19T07:43:11.311217+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:00.406076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656008+0000","last_change":"2024-09-19T07:43:06.270017+0000","last_active":"2024-09-19T07:43:12.656008+0000","last_peered":"2024-09-19T07:43:12.656008+0000","last_clean":"2024-09-19T07:43:12.656008+0000","last_became_active":"2024-09-19T07:43:06.269698+0000","last_became_peered":"2024-09-19T07:43:06.269698+0000","last_unstale":"2024-09-19T07:43:12.656008+0000","last_undegraded":"2024-09-19T07:43:12.656008+0000","last_fullsized":"2024-09-19T07:43:12.656008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:27.185190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311784+0000","last_change":"2024-09-19T07:43:05.265424+0000","last_active":"2024-09-19T07:43:11.311784+0000","last_peered":"2024-09-19T07:43:11.311784+0000","last_clean":"2024-09-19T07:43:11.311784+0000","last_became_active":"2024-09-19T07:43:05.265165+0000","last_became_peered":"2024-09-19T07:43:05.265165+0000","last_unstale":"2024-09-19T07:43:11.311784+0000","last_undegraded":"2024-09-19T07:43:11.311784+0000","last_fullsized":"2024-09-19T07:43:11.311784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:38:29.212025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310075+0000","last_change":"2024-09-19T07:43:10.290285+0000","last_active":"2024-09-19T07:43:11.310075+0000","last_peered":"2024-09-19T07:43:11.310075+0000","last_clean":"2024-09-19T07:43:11.310075+0000","last_became_active":"2024-09-19T07:43:10.289718+0000","last_became_peered":"2024-09-19T07:43:10.289718+0000","last_unstale":"2024-09-19T07:43:11.310075+0000","last_undegraded":"2024-09-19T07:43:11.310075+0000","last_fullsized":"2024-09-19T07:43:11.310075+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:08.693513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311662+0000","last_change":"2024-09-19T07:43:06.265419+0000","last_active":"2024-09-19T07:43:11.311662+0000","last_peered":"2024-09-19T07:43:11.311662+0000","last_clean":"2024-09-19T07:43:11.311662+0000","last_became_active":"2024-09-19T07:43:06.265170+0000","last_became_peered":"2024-09-19T07:43:06.265170+0000","last_unstale":"2024-09-19T07:43:11.311662+0000","last_undegraded":"2024-09-19T07:43:11.311662+0000","last_fullsized":"2024-09-19T07:43:11.311662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451157+0000","last_change":"2024-09-19T07:43:05.264095+0000","last_active":"2024-09-19T07:43:11.451157+0000","last_peered":"2024-09-19T07:43:11.451157+0000","last_clean":"2024-09-19T07:43:11.451157+0000","last_became_active":"2024-09-19T07:43:05.262895+0000","last_became_peered":"2024-09-19T07:43:05.262895+0000","last_unstale":"2024-09-19T07:43:11.451157+0000","last_undegraded":"2024-09-19T07:43:11.451157+0000","last_fullsized":"2024-09-19T07:43:11.451157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:38.098052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310066+0000","last_change":"2024-09-19T07:43:10.482892+0000","last_active":"2024-09-19T07:43:11.310066+0000","last_peered":"2024-09-19T07:43:11.310066+0000","last_clean":"2024-09-19T07:43:11.310066+0000","last_became_active":"2024-09-19T07:43:10.482523+0000","last_became_peered":"2024-09-19T07:43:10.482523+0000","last_unstale":"2024-09-19T07:43:11.310066+0000","last_undegraded":"2024-09-19T07:43:11.310066+0000","last_fullsized":"2024-09-19T07:43:11.310066+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:07.059894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655916+0000","last_change":"2024-09-19T07:43:10.480793+0000","last_active":"2024-09-19T07:43:12.655916+0000","last_peered":"2024-09-19T07:43:12.655916+0000","last_clean":"2024-09-19T07:43:12.655916+0000","last_became_active":"2024-09-19T07:43:10.480423+0000","last_became_peered":"2024-09-19T07:43:10.480423+0000","last_unstale":"2024-09-19T07:43:12.655916+0000","last_undegraded":"2024-09-19T07:43:12.655916+0000","last_fullsized":"2024-09-19T07:43:12.655916+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:18.372981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680526+0000","last_change":"2024-09-19T07:43:06.258001+0000","last_active":"2024-09-19T07:43:12.680526+0000","last_peered":"2024-09-19T07:43:12.680526+0000","last_clean":"2024-09-19T07:43:12.680526+0000","last_became_active":"2024-09-19T07:43:06.257731+0000","last_became_peered":"2024-09-19T07:43:06.257731+0000","last_unstale":"2024-09-19T07:43:12.680526+0000","last_undegraded":"2024-09-19T07:43:12.680526+0000","last_fullsized":"2024-09-19T07:43:12.680526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:14.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310662+0000","last_change":"2024-09-19T07:43:05.260911+0000","last_active":"2024-09-19T07:43:11.310662+0000","last_peered":"2024-09-19T07:43:11.310662+0000","last_clean":"2024-09-19T07:43:11.310662+0000","last_became_active":"2024-09-19T07:43:05.258827+0000","last_became_peered":"2024-09-19T07:43:05.258827+0000","last_unstale":"2024-09-19T07:43:11.310662+0000","last_undegraded":"2024-09-19T07:43:11.310662+0000","last_fullsized":"2024-09-19T07:43:11.310662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:13.592193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451139+0000","last_change":"2024-09-19T07:43:10.293630+0000","last_active":"2024-09-19T07:43:11.451139+0000","last_peered":"2024-09-19T07:43:11.451139+0000","last_clean":"2024-09-19T07:43:11.451139+0000","last_became_active":"2024-09-19T07:43:10.292979+0000","last_became_peered":"2024-09-19T07:43:10.292979+0000","last_unstale":"2024-09-19T07:43:11.451139+0000","last_undegraded":"2024-09-19T07:43:11.451139+0000","last_fullsized":"2024-09-19T07:43:11.451139+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:43.490603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310026+0000","last_change":"2024-09-19T07:43:06.274060+0000","last_active":"2024-09-19T07:43:11.310026+0000","last_peered":"2024-09-19T07:43:11.310026+0000","last_clean":"2024-09-19T07:43:11.310026+0000","last_became_active":"2024-09-19T07:43:06.272704+0000","last_became_peered":"2024-09-19T07:43:06.272704+0000","last_unstale":"2024-09-19T07:43:11.310026+0000","last_undegraded":"2024-09-19T07:43:11.310026+0000","last_fullsized":"2024-09-19T07:43:11.310026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:06.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310182+0000","last_change":"2024-09-19T07:43:05.264989+0000","last_active":"2024-09-19T07:43:11.310182+0000","last_peered":"2024-09-19T07:43:11.310182+0000","last_clean":"2024-09-19T07:43:11.310182+0000","last_became_active":"2024-09-19T07:43:05.264673+0000","last_became_peered":"2024-09-19T07:43:05.264673+0000","last_unstale":"2024-09-19T07:43:11.310182+0000","last_undegraded":"2024-09-19T07:43:11.310182+0000","last_fullsized":"2024-09-19T07:43:11.310182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:22.696534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450834+0000","last_change":"2024-09-19T07:43:05.262438+0000","last_active":"2024-09-19T07:43:11.450834+0000","last_peered":"2024-09-19T07:43:11.450834+0000","last_clean":"2024-09-19T07:43:11.450834+0000","last_became_active":"2024-09-19T07:43:05.262129+0000","last_became_peered":"2024-09-19T07:43:05.262129+0000","last_unstale":"2024-09-19T07:43:11.450834+0000","last_undegraded":"2024-09-19T07:43:11.450834+0000","last_fullsized":"2024-09-19T07:43:11.450834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:44.359336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656191+0000","last_change":"2024-09-19T07:43:06.270402+0000","last_active":"2024-09-19T07:43:12.656191+0000","last_peered":"2024-09-19T07:43:12.656191+0000","last_clean":"2024-09-19T07:43:12.656191+0000","last_became_active":"2024-09-19T07:43:06.270087+0000","last_became_peered":"2024-09-19T07:43:06.270087+0000","last_unstale":"2024-09-19T07:43:12.656191+0000","last_undegraded":"2024-09-19T07:43:12.656191+0000","last_fullsized":"2024-09-19T07:43:12.656191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588782+0000","last_change":"2024-09-19T07:43:10.288597+0000","last_active":"2024-09-19T07:43:11.588782+0000","last_peered":"2024-09-19T07:43:11.588782+0000","last_clean":"2024-09-19T07:43:11.588782+0000","last_became_active":"2024-09-19T07:43:10.288207+0000","last_became_peered":"2024-09-19T07:43:10.288207+0000","last_unstale":"2024-09-19T07:43:11.588782+0000","last_undegraded":"2024-09-19T07:43:11.588782+0000","last_fullsized":"2024-09-19T07:43:11.588782+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:43.318582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310346+0000","last_change":"2024-09-19T07:43:05.261298+0000","last_active":"2024-09-19T07:43:11.310346+0000","last_peered":"2024-09-19T07:43:11.310346+0000","last_clean":"2024-09-19T07:43:11.310346+0000","last_became_active":"2024-09-19T07:43:05.261000+0000","last_became_peered":"2024-09-19T07:43:05.261000+0000","last_unstale":"2024-09-19T07:43:11.310346+0000","last_undegraded":"2024-09-19T07:43:11.310346+0000","last_fullsized":"2024-09-19T07:43:11.310346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:04.232875+0000","last_clean_scrub_stamp":"2024-09-19T07:43:04.232875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:38.098052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310888+0000","last_change":"2024-09-19T07:43:06.267976+0000","last_active":"2024-09-19T07:43:11.310888+0000","last_peered":"2024-09-19T07:43:11.310888+0000","last_clean":"2024-09-19T07:43:11.310888+0000","last_became_active":"2024-09-19T07:43:06.267214+0000","last_became_peered":"2024-09-19T07:43:06.267214+0000","last_unstale":"2024-09-19T07:43:11.310888+0000","last_undegraded":"2024-09-19T07:43:11.310888+0000","last_fullsized":"2024-09-19T07:43:11.310888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:48.537241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310742+0000","last_change":"2024-09-19T07:43:10.288619+0000","last_active":"2024-09-19T07:43:11.310742+0000","last_peered":"2024-09-19T07:43:11.310742+0000","last_clean":"2024-09-19T07:43:11.310742+0000","last_became_active":"2024-09-19T07:43:10.288213+0000","last_became_peered":"2024-09-19T07:43:10.288213+0000","last_unstale":"2024-09-19T07:43:11.310742+0000","last_undegraded":"2024-09-19T07:43:11.310742+0000","last_fullsized":"2024-09-19T07:43:11.310742+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:09.264150+0000","last_clean_scrub_stamp":"2024-09-19T07:43:09.264150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:30.165579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450437+0000","last_change":"2024-09-19T07:43:06.276792+0000","last_active":"2024-09-19T07:43:11.450437+0000","last_peered":"2024-09-19T07:43:11.450437+0000","last_clean":"2024-09-19T07:43:11.450437+0000","last_became_active":"2024-09-19T07:43:06.276518+0000","last_became_peered":"2024-09-19T07:43:06.276518+0000","last_unstale":"2024-09-19T07:43:11.450437+0000","last_undegraded":"2024-09-19T07:43:11.450437+0000","last_fullsized":"2024-09-19T07:43:11.450437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:12.618593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588673+0000","last_change":"2024-09-19T07:43:06.268099+0000","last_active":"2024-09-19T07:43:11.588673+0000","last_peered":"2024-09-19T07:43:11.588673+0000","last_clean":"2024-09-19T07:43:11.588673+0000","last_became_active":"2024-09-19T07:43:06.267803+0000","last_became_peered":"2024-09-19T07:43:06.267803+0000","last_unstale":"2024-09-19T07:43:11.588673+0000","last_undegraded":"2024-09-19T07:43:11.588673+0000","last_fullsized":"2024-09-19T07:43:11.588673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:31.966840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311465+0000","last_change":"2024-09-19T07:43:06.263757+0000","last_active":"2024-09-19T07:43:11.311465+0000","last_peered":"2024-09-19T07:43:11.311465+0000","last_clean":"2024-09-19T07:43:11.311465+0000","last_became_active":"2024-09-19T07:43:06.263480+0000","last_became_peered":"2024-09-19T07:43:06.263480+0000","last_unstale":"2024-09-19T07:43:11.311465+0000","last_undegraded":"2024-09-19T07:43:11.311465+0000","last_fullsized":"2024-09-19T07:43:11.311465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:39.918269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.656177+0000","last_change":"2024-09-19T07:43:06.278694+0000","last_active":"2024-09-19T07:43:12.656177+0000","last_peered":"2024-09-19T07:43:12.656177+0000","last_clean":"2024-09-19T07:43:12.656177+0000","last_became_active":"2024-09-19T07:43:06.277280+0000","last_became_peered":"2024-09-19T07:43:06.277280+0000","last_unstale":"2024-09-19T07:43:12.656177+0000","last_undegraded":"2024-09-19T07:43:12.656177+0000","last_fullsized":"2024-09-19T07:43:12.656177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:51.595301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310764+0000","last_change":"2024-09-19T07:43:06.272967+0000","last_active":"2024-09-19T07:43:11.310764+0000","last_peered":"2024-09-19T07:43:11.310764+0000","last_clean":"2024-09-19T07:43:11.310764+0000","last_became_active":"2024-09-19T07:43:06.272476+0000","last_became_peered":"2024-09-19T07:43:06.272476+0000","last_unstale":"2024-09-19T07:43:11.310764+0000","last_undegraded":"2024-09-19T07:43:11.310764+0000","last_fullsized":"2024-09-19T07:43:11.310764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:09.222719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681113+0000","last_change":"2024-09-19T07:43:06.260655+0000","last_active":"2024-09-19T07:43:12.681113+0000","last_peered":"2024-09-19T07:43:12.681113+0000","last_clean":"2024-09-19T07:43:12.681113+0000","last_became_active":"2024-09-19T07:43:06.260388+0000","last_became_peered":"2024-09-19T07:43:06.260388+0000","last_unstale":"2024-09-19T07:43:12.681113+0000","last_undegraded":"2024-09-19T07:43:12.681113+0000","last_fullsized":"2024-09-19T07:43:12.681113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451084+0000","last_change":"2024-09-19T07:43:06.270719+0000","last_active":"2024-09-19T07:43:11.451084+0000","last_peered":"2024-09-19T07:43:11.451084+0000","last_clean":"2024-09-19T07:43:11.451084+0000","last_became_active":"2024-09-19T07:43:06.270022+0000","last_became_peered":"2024-09-19T07:43:06.270022+0000","last_unstale":"2024-09-19T07:43:11.451084+0000","last_undegraded":"2024-09-19T07:43:11.451084+0000","last_fullsized":"2024-09-19T07:43:11.451084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:27.185190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310886+0000","last_change":"2024-09-19T07:43:06.266798+0000","last_active":"2024-09-19T07:43:11.310886+0000","last_peered":"2024-09-19T07:43:11.310886+0000","last_clean":"2024-09-19T07:43:11.310886+0000","last_became_active":"2024-09-19T07:43:06.266461+0000","last_became_peered":"2024-09-19T07:43:06.266461+0000","last_unstale":"2024-09-19T07:43:11.310886+0000","last_undegraded":"2024-09-19T07:43:11.310886+0000","last_fullsized":"2024-09-19T07:43:11.310886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:50.544269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310636+0000","last_change":"2024-09-19T07:43:06.263830+0000","last_active":"2024-09-19T07:43:11.310636+0000","last_peered":"2024-09-19T07:43:11.310636+0000","last_clean":"2024-09-19T07:43:11.310636+0000","last_became_active":"2024-09-19T07:43:06.263539+0000","last_became_peered":"2024-09-19T07:43:06.263539+0000","last_unstale":"2024-09-19T07:43:11.310636+0000","last_undegraded":"2024-09-19T07:43:11.310636+0000","last_fullsized":"2024-09-19T07:43:11.310636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:50.544269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310166+0000","last_change":"2024-09-19T07:43:06.276371+0000","last_active":"2024-09-19T07:43:11.310166+0000","last_peered":"2024-09-19T07:43:11.310166+0000","last_clean":"2024-09-19T07:43:11.310166+0000","last_became_active":"2024-09-19T07:43:06.276103+0000","last_became_peered":"2024-09-19T07:43:06.276103+0000","last_unstale":"2024-09-19T07:43:11.310166+0000","last_undegraded":"2024-09-19T07:43:11.310166+0000","last_fullsized":"2024-09-19T07:43:11.310166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:51.595301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311371+0000","last_change":"2024-09-19T07:43:06.265375+0000","last_active":"2024-09-19T07:43:11.311371+0000","last_peered":"2024-09-19T07:43:11.311371+0000","last_clean":"2024-09-19T07:43:11.311371+0000","last_became_active":"2024-09-19T07:43:06.265118+0000","last_became_peered":"2024-09-19T07:43:06.265118+0000","last_unstale":"2024-09-19T07:43:11.311371+0000","last_undegraded":"2024-09-19T07:43:11.311371+0000","last_fullsized":"2024-09-19T07:43:11.311371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:14.347245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655877+0000","last_change":"2024-09-19T07:43:06.279273+0000","last_active":"2024-09-19T07:43:12.655877+0000","last_peered":"2024-09-19T07:43:12.655877+0000","last_clean":"2024-09-19T07:43:12.655877+0000","last_became_active":"2024-09-19T07:43:06.278276+0000","last_became_peered":"2024-09-19T07:43:06.278276+0000","last_unstale":"2024-09-19T07:43:12.655877+0000","last_undegraded":"2024-09-19T07:43:12.655877+0000","last_fullsized":"2024-09-19T07:43:12.655877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:14.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450830+0000","last_change":"2024-09-19T07:43:06.268360+0000","last_active":"2024-09-19T07:43:11.450830+0000","last_peered":"2024-09-19T07:43:11.450830+0000","last_clean":"2024-09-19T07:43:11.450830+0000","last_became_active":"2024-09-19T07:43:06.268096+0000","last_became_peered":"2024-09-19T07:43:06.268096+0000","last_unstale":"2024-09-19T07:43:11.450830+0000","last_undegraded":"2024-09-19T07:43:11.450830+0000","last_fullsized":"2024-09-19T07:43:11.450830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681729+0000","last_change":"2024-09-19T07:43:06.272110+0000","last_active":"2024-09-19T07:43:12.681729+0000","last_peered":"2024-09-19T07:43:12.681729+0000","last_clean":"2024-09-19T07:43:12.681729+0000","last_became_active":"2024-09-19T07:43:06.271804+0000","last_became_peered":"2024-09-19T07:43:06.271804+0000","last_unstale":"2024-09-19T07:43:12.681729+0000","last_undegraded":"2024-09-19T07:43:12.681729+0000","last_fullsized":"2024-09-19T07:43:12.681729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:23.113048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.588806+0000","last_change":"2024-09-19T07:43:06.260936+0000","last_active":"2024-09-19T07:43:11.588806+0000","last_peered":"2024-09-19T07:43:11.588806+0000","last_clean":"2024-09-19T07:43:11.588806+0000","last_became_active":"2024-09-19T07:43:06.260663+0000","last_became_peered":"2024-09-19T07:43:06.260663+0000","last_unstale":"2024-09-19T07:43:11.588806+0000","last_undegraded":"2024-09-19T07:43:11.588806+0000","last_fullsized":"2024-09-19T07:43:11.588806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:06.425112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312242+0000","last_change":"2024-09-19T07:43:06.268104+0000","last_active":"2024-09-19T07:43:11.312242+0000","last_peered":"2024-09-19T07:43:11.312242+0000","last_clean":"2024-09-19T07:43:11.312242+0000","last_became_active":"2024-09-19T07:43:06.267830+0000","last_became_peered":"2024-09-19T07:43:06.267830+0000","last_unstale":"2024-09-19T07:43:11.312242+0000","last_undegraded":"2024-09-19T07:43:11.312242+0000","last_fullsized":"2024-09-19T07:43:11.312242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:22.804370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310551+0000","last_change":"2024-09-19T07:43:06.269948+0000","last_active":"2024-09-19T07:43:11.310551+0000","last_peered":"2024-09-19T07:43:11.310551+0000","last_clean":"2024-09-19T07:43:11.310551+0000","last_became_active":"2024-09-19T07:43:06.269684+0000","last_became_peered":"2024-09-19T07:43:06.269684+0000","last_unstale":"2024-09-19T07:43:11.310551+0000","last_undegraded":"2024-09-19T07:43:11.310551+0000","last_fullsized":"2024-09-19T07:43:11.310551+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:37.361491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.680603+0000","last_change":"2024-09-19T07:43:06.269924+0000","last_active":"2024-09-19T07:43:12.680603+0000","last_peered":"2024-09-19T07:43:12.680603+0000","last_clean":"2024-09-19T07:43:12.680603+0000","last_became_active":"2024-09-19T07:43:06.269602+0000","last_became_peered":"2024-09-19T07:43:06.269602+0000","last_unstale":"2024-09-19T07:43:12.680603+0000","last_undegraded":"2024-09-19T07:43:12.680603+0000","last_fullsized":"2024-09-19T07:43:12.680603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:51.595301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655995+0000","last_change":"2024-09-19T07:43:06.271422+0000","last_active":"2024-09-19T07:43:12.655995+0000","last_peered":"2024-09-19T07:43:12.655995+0000","last_clean":"2024-09-19T07:43:12.655995+0000","last_became_active":"2024-09-19T07:43:06.271096+0000","last_became_peered":"2024-09-19T07:43:06.271096+0000","last_unstale":"2024-09-19T07:43:12.655995+0000","last_undegraded":"2024-09-19T07:43:12.655995+0000","last_fullsized":"2024-09-19T07:43:12.655995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:23.743784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.310215+0000","last_change":"2024-09-19T07:43:06.264090+0000","last_active":"2024-09-19T07:43:11.310215+0000","last_peered":"2024-09-19T07:43:11.310215+0000","last_clean":"2024-09-19T07:43:11.310215+0000","last_became_active":"2024-09-19T07:43:06.262071+0000","last_became_peered":"2024-09-19T07:43:06.262071+0000","last_unstale":"2024-09-19T07:43:11.310215+0000","last_undegraded":"2024-09-19T07:43:11.310215+0000","last_fullsized":"2024-09-19T07:43:11.310215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:13.585625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.312316+0000","last_change":"2024-09-19T07:43:06.265567+0000","last_active":"2024-09-19T07:43:11.312316+0000","last_peered":"2024-09-19T07:43:11.312316+0000","last_clean":"2024-09-19T07:43:11.312316+0000","last_became_active":"2024-09-19T07:43:06.265290+0000","last_became_peered":"2024-09-19T07:43:06.265290+0000","last_unstale":"2024-09-19T07:43:11.312316+0000","last_undegraded":"2024-09-19T07:43:11.312316+0000","last_fullsized":"2024-09-19T07:43:11.312316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:14.597733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.450953+0000","last_change":"2024-09-19T07:43:06.267949+0000","last_active":"2024-09-19T07:43:11.450953+0000","last_peered":"2024-09-19T07:43:11.450953+0000","last_clean":"2024-09-19T07:43:11.450953+0000","last_became_active":"2024-09-19T07:43:06.267640+0000","last_became_peered":"2024-09-19T07:43:06.267640+0000","last_unstale":"2024-09-19T07:43:11.450953+0000","last_undegraded":"2024-09-19T07:43:11.450953+0000","last_fullsized":"2024-09-19T07:43:11.450953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:14.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.451030+0000","last_change":"2024-09-19T07:43:06.268718+0000","last_active":"2024-09-19T07:43:11.451030+0000","last_peered":"2024-09-19T07:43:11.451030+0000","last_clean":"2024-09-19T07:43:11.451030+0000","last_became_active":"2024-09-19T07:43:06.268398+0000","last_became_peered":"2024-09-19T07:43:06.268398+0000","last_unstale":"2024-09-19T07:43:11.451030+0000","last_undegraded":"2024-09-19T07:43:11.451030+0000","last_fullsized":"2024-09-19T07:43:11.451030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:51.595301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681743+0000","last_change":"2024-09-19T07:43:06.273520+0000","last_active":"2024-09-19T07:43:12.681743+0000","last_peered":"2024-09-19T07:43:12.681743+0000","last_clean":"2024-09-19T07:43:12.681743+0000","last_became_active":"2024-09-19T07:43:06.273150+0000","last_became_peered":"2024-09-19T07:43:06.273150+0000","last_unstale":"2024-09-19T07:43:12.681743+0000","last_undegraded":"2024-09-19T07:43:12.681743+0000","last_fullsized":"2024-09-19T07:43:12.681743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:39.464868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.309860+0000","last_change":"2024-09-19T07:43:06.272990+0000","last_active":"2024-09-19T07:43:11.309860+0000","last_peered":"2024-09-19T07:43:11.309860+0000","last_clean":"2024-09-19T07:43:11.309860+0000","last_became_active":"2024-09-19T07:43:06.272323+0000","last_became_peered":"2024-09-19T07:43:06.272323+0000","last_unstale":"2024-09-19T07:43:11.309860+0000","last_undegraded":"2024-09-19T07:43:11.309860+0000","last_fullsized":"2024-09-19T07:43:11.309860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:14.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681143+0000","last_change":"2024-09-19T07:43:06.273648+0000","last_active":"2024-09-19T07:43:12.681143+0000","last_peered":"2024-09-19T07:43:12.681143+0000","last_clean":"2024-09-19T07:43:12.681143+0000","last_became_active":"2024-09-19T07:43:06.273387+0000","last_became_peered":"2024-09-19T07:43:06.273387+0000","last_unstale":"2024-09-19T07:43:12.681143+0000","last_undegraded":"2024-09-19T07:43:12.681143+0000","last_fullsized":"2024-09-19T07:43:12.681143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:56.380341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311270+0000","last_change":"2024-09-19T07:43:06.274896+0000","last_active":"2024-09-19T07:43:11.311270+0000","last_peered":"2024-09-19T07:43:11.311270+0000","last_clean":"2024-09-19T07:43:11.311270+0000","last_became_active":"2024-09-19T07:43:06.274551+0000","last_became_peered":"2024-09-19T07:43:06.274551+0000","last_unstale":"2024-09-19T07:43:11.311270+0000","last_undegraded":"2024-09-19T07:43:11.311270+0000","last_fullsized":"2024-09-19T07:43:11.311270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:39.658744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681148+0000","last_change":"2024-09-19T07:43:06.271789+0000","last_active":"2024-09-19T07:43:12.681148+0000","last_peered":"2024-09-19T07:43:12.681148+0000","last_clean":"2024-09-19T07:43:12.681148+0000","last_became_active":"2024-09-19T07:43:06.269978+0000","last_became_peered":"2024-09-19T07:43:06.269978+0000","last_unstale":"2024-09-19T07:43:12.681148+0000","last_undegraded":"2024-09-19T07:43:12.681148+0000","last_fullsized":"2024-09-19T07:43:12.681148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:04.168224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:11.311913+0000","last_change":"2024-09-19T07:43:06.274164+0000","last_active":"2024-09-19T07:43:11.311913+0000","last_peered":"2024-09-19T07:43:11.311913+0000","last_clean":"2024-09-19T07:43:11.311913+0000","last_became_active":"2024-09-19T07:43:06.273730+0000","last_became_peered":"2024-09-19T07:43:06.273730+0000","last_unstale":"2024-09-19T07:43:11.311913+0000","last_undegraded":"2024-09-19T07:43:11.311913+0000","last_fullsized":"2024-09-19T07:43:11.311913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:56.380341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.655874+0000","last_change":"2024-09-19T07:43:06.266043+0000","last_active":"2024-09-19T07:43:12.655874+0000","last_peered":"2024-09-19T07:43:12.655874+0000","last_clean":"2024-09-19T07:43:12.655874+0000","last_became_active":"2024-09-19T07:43:06.265717+0000","last_became_peered":"2024-09-19T07:43:06.265717+0000","last_unstale":"2024-09-19T07:43:12.655874+0000","last_undegraded":"2024-09-19T07:43:12.655874+0000","last_fullsized":"2024-09-19T07:43:12.655874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:12.618593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681153+0000","last_change":"2024-09-19T07:43:06.270945+0000","last_active":"2024-09-19T07:43:12.681153+0000","last_peered":"2024-09-19T07:43:12.681153+0000","last_clean":"2024-09-19T07:43:12.681153+0000","last_became_active":"2024-09-19T07:43:06.270641+0000","last_became_peered":"2024-09-19T07:43:06.270641+0000","last_unstale":"2024-09-19T07:43:12.681153+0000","last_undegraded":"2024-09-19T07:43:12.681153+0000","last_fullsized":"2024-09-19T07:43:12.681153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:27.185190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-19T07:43:12.681632+0000","last_change":"2024-09-19T07:43:06.277928+0000","last_active":"2024-09-19T07:43:12.681632+0000","last_peered":"2024-09-19T07:43:12.681632+0000","last_clean":"2024-09-19T07:43:12.681632+0000","last_became_active":"2024-09-19T07:43:06.277632+0000","last_became_peered":"2024-09-19T07:43:06.277632+0000","last_unstale":"2024-09-19T07:43:12.681632+0000","last_undegraded":"2024-09-19T07:43:12.681632+0000","last_fullsized":"2024-09-19T07:43:12.681632+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:05.238415+0000","last_clean_scrub_stamp":"2024-09-19T07:43:05.238415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:09.222719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":868352,"data_stored":697396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":110290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":106536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":106572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":876544,"data_stored":697452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":282624,"data_stored":110290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":278528,"data_stored":106606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":270336,"data_stored":106446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:43:31.082 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T07:43:31.082 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T07:43:31.082 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-19T07:43:31.082 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:43:31.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.218+0000 7fbecd1c4640 1 Processor -- start 2024-09-19T07:43:31.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.218+0000 7fbecd1c4640 1 -- start start 2024-09-19T07:43:31.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.219+0000 7fbecd1c4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbec815e680 0x7fbec80cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.219+0000 7fbecd1c4640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbec80d0010 0x7fbec80d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.219+0000 7fbecd1c4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbec80d1880 con 0x7fbec815e2a0 2024-09-19T07:43:31.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.219+0000 7fbecd1c4640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fbec80d1b60 con 0x7fbec80d0010 2024-09-19T07:43:31.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.219+0000 7fbecd1c4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbec80cf2d0 con 0x7fbec815e680 2024-09-19T07:43:31.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.219+0000 7fbec6d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbec815e2a0 0x7fbec806d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44976/0 (socket says 172.21.15.16:44976) 2024-09-19T07:43:31.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.219+0000 7fbec6d9d640 1 -- 172.21.15.16:0/3927765571 learned_addr learned my addr 172.21.15.16:0/3927765571 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:31.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.219+0000 7fbec759e640 1 --2- 172.21.15.16:0/3927765571 >> v2:172.21.15.146:3300/0 conn(0x7fbec80d0010 0x7fbec80d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.219+0000 7fbec659c640 1 --2- 172.21.15.16:0/3927765571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbec815e680 0x7fbec80cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.219+0000 7fbec5d9b640 1 -- 172.21.15.16:0/3927765571 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4920868 0 0) 0x7fbec80d1880 con 0x7fbec815e2a0 2024-09-19T07:43:31.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.219+0000 7fbec5d9b640 1 -- 172.21.15.16:0/3927765571 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbeb0003520 con 0x7fbec815e2a0 2024-09-19T07:43:31.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.219+0000 7fbec5d9b640 1 -- 172.21.15.16:0/3927765571 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3734234451 0 0) 0x7fbeb0003520 con 0x7fbec815e2a0 2024-09-19T07:43:31.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.219+0000 7fbec5d9b640 1 -- 172.21.15.16:0/3927765571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbec815e680 msgr2=0x7fbec80cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:31.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.219+0000 7fbec5d9b640 1 --2- 172.21.15.16:0/3927765571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbec815e680 0x7fbec80cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.219+0000 7fbec5d9b640 1 -- 172.21.15.16:0/3927765571 >> v2:172.21.15.146:3300/0 conn(0x7fbec80d0010 msgr2=0x7fbec80d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:31.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.219+0000 7fbec5d9b640 1 --2- 172.21.15.16:0/3927765571 >> v2:172.21.15.146:3300/0 conn(0x7fbec80d0010 0x7fbec80d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.219+0000 7fbec5d9b640 1 -- 172.21.15.16:0/3927765571 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec80cf5b0 con 0x7fbec815e2a0 2024-09-19T07:43:31.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.219+0000 7fbec5d9b640 1 -- 172.21.15.16:0/3927765571 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fbeb8001670 con 0x7fbec815e2a0 2024-09-19T07:43:31.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.220+0000 7fbec5d9b640 1 -- 172.21.15.16:0/3927765571 >> v1:172.21.15.16:6789/0 conn(0x7fbec815e2a0 legacy=0x7fbec806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:31.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.220+0000 7fbec5d9b640 1 --2- 172.21.15.16:0/3927765571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb00039b0 0x7fbeb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.220+0000 7fbec5d9b640 1 --2- 172.21.15.16:0/3927765571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbec815e680 0x7fbeb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.220+0000 7fbec5d9b640 1 --2- 172.21.15.16:0/3927765571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeb0004d40 0x7fbeb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.220+0000 7fbec5d9b640 1 -- 172.21.15.16:0/3927765571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbec80d1b60 con 0x7fbeb00039b0 2024-09-19T07:43:31.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.220+0000 7fbec5d9b640 1 -- 172.21.15.16:0/3927765571 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbec80cf2d0 con 0x7fbeb0004d40 2024-09-19T07:43:31.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.220+0000 7fbec5d9b640 1 -- 172.21.15.16:0/3927765571 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb000b9f0 con 0x7fbec815e680 2024-09-19T07:43:31.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.220+0000 7fbec6d9d640 1 --2- 172.21.15.16:0/3927765571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb00039b0 0x7fbeb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.220+0000 7fbec759e640 1 --2- 172.21.15.16:0/3927765571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeb0004d40 0x7fbeb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.220+0000 7fbec659c640 1 --2- 172.21.15.16:0/3927765571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbec815e680 0x7fbeb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.220+0000 7fbec6d9d640 1 -- 172.21.15.16:0/3927765571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbec815e680 msgr2=0x7fbeb0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:31.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.220+0000 7fbec6d9d640 1 --2- 172.21.15.16:0/3927765571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbec815e680 0x7fbeb0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.220+0000 7fbec6d9d640 1 -- 172.21.15.16:0/3927765571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeb0004d40 msgr2=0x7fbeb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:31.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.220+0000 7fbec6d9d640 1 --2- 172.21.15.16:0/3927765571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeb0004d40 0x7fbeb00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.220+0000 7fbec6d9d640 1 -- 172.21.15.16:0/3927765571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb0003520 con 0x7fbeb00039b0 2024-09-19T07:43:31.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.221+0000 7fbec6d9d640 1 --2- 172.21.15.16:0/3927765571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb00039b0 0x7fbeb0003dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800c210 tx=0x7fbeb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d99e6952f7c3c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:31.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.221+0000 7fbec5d9b640 1 -- 172.21.15.16:0/3927765571 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8007d50 con 0x7fbeb00039b0 2024-09-19T07:43:31.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.221+0000 7fbec5d9b640 1 -- 172.21.15.16:0/3927765571 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb8005af0 con 0x7fbeb00039b0 2024-09-19T07:43:31.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.221+0000 7fbec5d9b640 1 -- 172.21.15.16:0/3927765571 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8007070 con 0x7fbeb00039b0 2024-09-19T07:43:31.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.221+0000 7fbecd1c4640 1 -- 172.21.15.16:0/3927765571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb00039b0 msgr2=0x7fbeb0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:31.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.221+0000 7fbecd1c4640 1 --2- 172.21.15.16:0/3927765571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb00039b0 0x7fbeb0003dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800c210 tx=0x7fbeb0005830 comp rx=0 tx=0).stop 2024-09-19T07:43:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.221+0000 7fbec759e640 1 -- 172.21.15.16:0/3927765571 reap_dead start 2024-09-19T07:43:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.221+0000 7fbecd1c4640 1 -- 172.21.15.16:0/3927765571 shutdown_connections 2024-09-19T07:43:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.221+0000 7fbecd1c4640 1 -- 172.21.15.16:0/3927765571 >> 172.21.15.16:0/3927765571 conn(0x7fbec809d760 msgr2=0x7fbec808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.221+0000 7fbecd1c4640 1 -- 172.21.15.16:0/3927765571 shutdown_connections 2024-09-19T07:43:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.221+0000 7fbecd1c4640 1 -- 172.21.15.16:0/3927765571 wait complete. 2024-09-19T07:43:31.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.223+0000 7fbecd1c4640 1 Processor -- start 2024-09-19T07:43:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.223+0000 7fbecd1c4640 1 -- start start 2024-09-19T07:43:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.224+0000 7fbecd1c4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeb00039b0 0x7fbec80b5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.224+0000 7fbecd1c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec80ca200 0x7fbec80d26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.224+0000 7fbecd1c4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbec81f4650 0x7fbec80cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.224+0000 7fbecd1c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbec806dcb0 con 0x7fbec80ca200 2024-09-19T07:43:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.224+0000 7fbecd1c4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbec806db30 con 0x7fbec81f4650 2024-09-19T07:43:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.224+0000 7fbecd1c4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbec80cfad0 con 0x7fbeb00039b0 2024-09-19T07:43:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.224+0000 7fbec659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec80ca200 0x7fbec80d26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.224+0000 7fbec659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec80ca200 0x7fbec80d26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57788/0 (socket says 172.21.15.16:57788) 2024-09-19T07:43:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.224+0000 7fbec659c640 1 -- 172.21.15.16:0/1583062845 learned_addr learned my addr 172.21.15.16:0/1583062845 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.224+0000 7fbec759e640 1 --2- 172.21.15.16:0/1583062845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbec81f4650 0x7fbec80cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.224+0000 7fbec6d9d640 1 --2- 172.21.15.16:0/1583062845 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeb00039b0 0x7fbec80b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.224+0000 7fbec659c640 1 -- 172.21.15.16:0/1583062845 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeb00039b0 msgr2=0x7fbec80b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:31.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.224+0000 7fbec659c640 1 --2- 172.21.15.16:0/1583062845 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeb00039b0 0x7fbec80b5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.224+0000 7fbec659c640 1 -- 172.21.15.16:0/1583062845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbec81f4650 msgr2=0x7fbec80cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:31.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.224+0000 7fbec659c640 1 --2- 172.21.15.16:0/1583062845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbec81f4650 0x7fbec80cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.224+0000 7fbec659c640 1 -- 172.21.15.16:0/1583062845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec80ca7a0 con 0x7fbec80ca200 2024-09-19T07:43:31.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.224+0000 7fbec759e640 1 --2- 172.21.15.16:0/1583062845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbec81f4650 0x7fbec80cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:31.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.224+0000 7fbec6d9d640 1 --2- 172.21.15.16:0/1583062845 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeb00039b0 0x7fbec80b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:31.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.224+0000 7fbec659c640 1 --2- 172.21.15.16:0/1583062845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec80ca200 0x7fbec80d26b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbebc011960 tx=0x7fbebc011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:31.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.225+0000 7fbeaf7fe640 1 -- 172.21.15.16:0/1583062845 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc019070 con 0x7fbec80ca200 2024-09-19T07:43:31.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.225+0000 7fbeaf7fe640 1 -- 172.21.15.16:0/1583062845 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbebc004300 con 0x7fbec80ca200 2024-09-19T07:43:31.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.225+0000 7fbecd1c4640 1 -- 172.21.15.16:0/1583062845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec80caa80 con 0x7fbec80ca200 2024-09-19T07:43:31.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.225+0000 7fbeaf7fe640 1 -- 172.21.15.16:0/1583062845 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc00fa60 con 0x7fbec80ca200 2024-09-19T07:43:31.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.225+0000 7fbecd1c4640 1 -- 172.21.15.16:0/1583062845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec80cb630 con 0x7fbec80ca200 2024-09-19T07:43:31.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.226+0000 7fbecd1c4640 1 -- 172.21.15.16:0/1583062845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec806d420 con 0x7fbec80ca200 2024-09-19T07:43:31.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.226+0000 7fbeaf7fe640 1 -- 172.21.15.16:0/1583062845 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbebc01c070 con 0x7fbec80ca200 2024-09-19T07:43:31.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.227+0000 7fbeaf7fe640 1 --2- 172.21.15.16:0/1583062845 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbeb406c2b0 0x7fbeb406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.227+0000 7fbeaf7fe640 1 -- 172.21.15.16:0/1583062845 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbebc08bd30 con 0x7fbec80ca200 2024-09-19T07:43:31.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.227+0000 7fbec6d9d640 1 --2- 172.21.15.16:0/1583062845 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbeb406c2b0 0x7fbeb406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.228+0000 7fbec6d9d640 1 --2- 172.21.15.16:0/1583062845 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbeb406c2b0 0x7fbeb406e770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8033000 tx=0x7fbeb8009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:31.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.232+0000 7fbeaf7fe640 1 -- 172.21.15.16:0/1583062845 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fbebc05a040 con 0x7fbec80ca200 2024-09-19T07:43:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.444+0000 7fbecd1c4640 1 -- 172.21.15.16:0/1583062845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbec806cf90 con 0x7fbec80ca200 2024-09-19T07:43:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.444+0000 7fbeaf7fe640 1 -- 172.21.15.16:0/1583062845 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fbebc059940 con 0x7fbec80ca200 2024-09-19T07:43:31.445 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:31.445 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T07:43:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.446+0000 7fbecd1c4640 1 -- 172.21.15.16:0/1583062845 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbeb406c2b0 msgr2=0x7fbeb406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.446+0000 7fbecd1c4640 1 --2- 172.21.15.16:0/1583062845 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbeb406c2b0 0x7fbeb406e770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8033000 tx=0x7fbeb8009030 comp rx=0 tx=0).stop 2024-09-19T07:43:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.446+0000 7fbecd1c4640 1 -- 172.21.15.16:0/1583062845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec80ca200 msgr2=0x7fbec80d26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.446+0000 7fbecd1c4640 1 --2- 172.21.15.16:0/1583062845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec80ca200 0x7fbec80d26b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbebc011960 tx=0x7fbebc011d60 comp rx=0 tx=0).stop 2024-09-19T07:43:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.446+0000 7fbecd1c4640 1 -- 172.21.15.16:0/1583062845 shutdown_connections 2024-09-19T07:43:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.446+0000 7fbecd1c4640 1 --2- 172.21.15.16:0/1583062845 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbeb406c2b0 0x7fbeb406e770 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.446+0000 7fbecd1c4640 1 --2- 172.21.15.16:0/1583062845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbec81f4650 0x7fbec80cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.446+0000 7fbecd1c4640 1 --2- 172.21.15.16:0/1583062845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec80ca200 0x7fbec80d26b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.446+0000 7fbecd1c4640 1 --2- 172.21.15.16:0/1583062845 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeb00039b0 0x7fbec80b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.446+0000 7fbecd1c4640 1 -- 172.21.15.16:0/1583062845 >> 172.21.15.16:0/1583062845 conn(0x7fbec809d760 msgr2=0x7fbec815f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.446+0000 7fbecd1c4640 1 -- 172.21.15.16:0/1583062845 shutdown_connections 2024-09-19T07:43:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.446+0000 7fbecd1c4640 1 -- 172.21.15.16:0/1583062845 wait complete. 2024-09-19T07:43:31.457 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-19T07:43:31.458 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:43:31.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.559+0000 7fa846245640 1 Processor -- start 2024-09-19T07:43:31.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.559+0000 7fa846245640 1 -- start start 2024-09-19T07:43:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.560+0000 7fa846245640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa84015e2a0 0x7fa84006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.560+0000 7fa846245640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa84006d960 0x7fa8400d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.560+0000 7fa846245640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8400d1880 con 0x7fa84015e680 2024-09-19T07:43:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.560+0000 7fa846245640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa8400d1b60 con 0x7fa84015e2a0 2024-09-19T07:43:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.560+0000 7fa846245640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa8400cf2d0 con 0x7fa84006d960 2024-09-19T07:43:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.560+0000 7fa83effd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa84015e680 0x7fa8400cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44982/0 (socket says 172.21.15.16:44982) 2024-09-19T07:43:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.560+0000 7fa83effd640 1 -- 172.21.15.16:0/1844546242 learned_addr learned my addr 172.21.15.16:0/1844546242 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83f7fe640 1 --2- 172.21.15.16:0/1844546242 >> v2:172.21.15.146:3300/0 conn(0x7fa84015e2a0 0x7fa84006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83ffff640 1 --2- 172.21.15.16:0/1844546242 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa84006d960 0x7fa8400d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83e7fc640 1 -- 172.21.15.16:0/1844546242 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1708574472 0 0) 0x7fa8400d1880 con 0x7fa84015e680 2024-09-19T07:43:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83e7fc640 1 -- 172.21.15.16:0/1844546242 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa828003520 con 0x7fa84015e680 2024-09-19T07:43:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83e7fc640 1 -- 172.21.15.16:0/1844546242 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1651850255 0 0) 0x7fa828003520 con 0x7fa84015e680 2024-09-19T07:43:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83e7fc640 1 -- 172.21.15.16:0/1844546242 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa84006d960 msgr2=0x7fa8400d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83e7fc640 1 --2- 172.21.15.16:0/1844546242 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa84006d960 0x7fa8400d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83e7fc640 1 -- 172.21.15.16:0/1844546242 >> v2:172.21.15.146:3300/0 conn(0x7fa84015e2a0 msgr2=0x7fa84006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83e7fc640 1 --2- 172.21.15.16:0/1844546242 >> v2:172.21.15.146:3300/0 conn(0x7fa84015e2a0 0x7fa84006d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83e7fc640 1 -- 172.21.15.16:0/1844546242 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8400cf5b0 con 0x7fa84015e680 2024-09-19T07:43:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83e7fc640 1 -- 172.21.15.16:0/1844546242 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fa834001670 con 0x7fa84015e680 2024-09-19T07:43:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83e7fc640 1 -- 172.21.15.16:0/1844546242 >> v1:172.21.15.16:6789/0 conn(0x7fa84015e680 legacy=0x7fa8400cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83e7fc640 1 --2- 172.21.15.16:0/1844546242 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa84006d960 0x7fa8280039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83e7fc640 1 --2- 172.21.15.16:0/1844546242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa828003f30 0x7fa828004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83e7fc640 1 --2- 172.21.15.16:0/1844546242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa828004d40 0x7fa8280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83e7fc640 1 -- 172.21.15.16:0/1844546242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8400d1b60 con 0x7fa828003f30 2024-09-19T07:43:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83e7fc640 1 -- 172.21.15.16:0/1844546242 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa8400cf2d0 con 0x7fa828004d40 2024-09-19T07:43:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83e7fc640 1 -- 172.21.15.16:0/1844546242 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa82800b9c0 con 0x7fa84006d960 2024-09-19T07:43:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83effd640 1 --2- 172.21.15.16:0/1844546242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa828003f30 0x7fa828004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83ffff640 1 --2- 172.21.15.16:0/1844546242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa828004d40 0x7fa8280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.561+0000 7fa83f7fe640 1 --2- 172.21.15.16:0/1844546242 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa84006d960 0x7fa8280039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.562+0000 7fa83effd640 1 -- 172.21.15.16:0/1844546242 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa84006d960 msgr2=0x7fa8280039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.562+0000 7fa83effd640 1 --2- 172.21.15.16:0/1844546242 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa84006d960 0x7fa8280039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.562+0000 7fa83effd640 1 -- 172.21.15.16:0/1844546242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa828004d40 msgr2=0x7fa8280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.562+0000 7fa83effd640 1 --2- 172.21.15.16:0/1844546242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa828004d40 0x7fa8280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.562+0000 7fa83effd640 1 -- 172.21.15.16:0/1844546242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa828003520 con 0x7fa828003f30 2024-09-19T07:43:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.562+0000 7fa83f7fe640 1 --2- 172.21.15.16:0/1844546242 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa84006d960 0x7fa8280039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.562+0000 7fa83effd640 1 --2- 172.21.15.16:0/1844546242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa828003f30 0x7fa828004800 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa834007fd0 tx=0x7fa828005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ced873768dbe276 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.562+0000 7fa83e7fc640 1 -- 172.21.15.16:0/1844546242 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834007d50 con 0x7fa828003f30 2024-09-19T07:43:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.562+0000 7fa83e7fc640 1 -- 172.21.15.16:0/1844546242 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa834005af0 con 0x7fa828003f30 2024-09-19T07:43:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.562+0000 7fa83e7fc640 1 -- 172.21.15.16:0/1844546242 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834007070 con 0x7fa828003f30 2024-09-19T07:43:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.563+0000 7fa846245640 1 -- 172.21.15.16:0/1844546242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa828003f30 msgr2=0x7fa828004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.563+0000 7fa846245640 1 --2- 172.21.15.16:0/1844546242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa828003f30 0x7fa828004800 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa834007fd0 tx=0x7fa828005830 comp rx=0 tx=0).stop 2024-09-19T07:43:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.563+0000 7fa83ffff640 1 -- 172.21.15.16:0/1844546242 reap_dead start 2024-09-19T07:43:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.563+0000 7fa846245640 1 -- 172.21.15.16:0/1844546242 shutdown_connections 2024-09-19T07:43:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.563+0000 7fa846245640 1 -- 172.21.15.16:0/1844546242 >> 172.21.15.16:0/1844546242 conn(0x7fa84009d760 msgr2=0x7fa84008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.563+0000 7fa846245640 1 -- 172.21.15.16:0/1844546242 shutdown_connections 2024-09-19T07:43:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.563+0000 7fa846245640 1 -- 172.21.15.16:0/1844546242 wait complete. 2024-09-19T07:43:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.564+0000 7fa846245640 1 Processor -- start 2024-09-19T07:43:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.564+0000 7fa846245640 1 -- start start 2024-09-19T07:43:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.565+0000 7fa846245640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa828003f30 0x7fa840156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.565+0000 7fa846245640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8401592b0 0x7fa840161f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.565+0000 7fa846245640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa840159730 0x7fa840162480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.565+0000 7fa846245640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8400d0360 con 0x7fa8401592b0 2024-09-19T07:43:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.565+0000 7fa846245640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa8400d01e0 con 0x7fa840159730 2024-09-19T07:43:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.565+0000 7fa846245640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa8400d4370 con 0x7fa828003f30 2024-09-19T07:43:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.565+0000 7fa83effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8401592b0 0x7fa840161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.565+0000 7fa83effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8401592b0 0x7fa840161f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57802/0 (socket says 172.21.15.16:57802) 2024-09-19T07:43:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.565+0000 7fa83effd640 1 -- 172.21.15.16:0/1637445455 learned_addr learned my addr 172.21.15.16:0/1637445455 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.565+0000 7fa83f7fe640 1 --2- 172.21.15.16:0/1637445455 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa828003f30 0x7fa840156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.566+0000 7fa83ffff640 1 --2- 172.21.15.16:0/1637445455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa840159730 0x7fa840162480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.566+0000 7fa83effd640 1 -- 172.21.15.16:0/1637445455 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa828003f30 msgr2=0x7fa840156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.566+0000 7fa83effd640 1 --2- 172.21.15.16:0/1637445455 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa828003f30 0x7fa840156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.566+0000 7fa83effd640 1 -- 172.21.15.16:0/1637445455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa840159730 msgr2=0x7fa840162480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.566+0000 7fa83effd640 1 --2- 172.21.15.16:0/1637445455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa840159730 0x7fa840162480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.566+0000 7fa83effd640 1 -- 172.21.15.16:0/1637445455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8400c8260 con 0x7fa8401592b0 2024-09-19T07:43:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.566+0000 7fa83f7fe640 1 --2- 172.21.15.16:0/1637445455 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa828003f30 0x7fa840156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.566+0000 7fa83ffff640 1 --2- 172.21.15.16:0/1637445455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa840159730 0x7fa840162480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.566+0000 7fa83effd640 1 --2- 172.21.15.16:0/1637445455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8401592b0 0x7fa840161f40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa834001970 tx=0x7fa834001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.566+0000 7fa827fff640 1 -- 172.21.15.16:0/1637445455 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834042020 con 0x7fa8401592b0 2024-09-19T07:43:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.566+0000 7fa827fff640 1 -- 172.21.15.16:0/1637445455 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa834004c30 con 0x7fa8401592b0 2024-09-19T07:43:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.566+0000 7fa827fff640 1 -- 172.21.15.16:0/1637445455 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834001c90 con 0x7fa8401592b0 2024-09-19T07:43:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.566+0000 7fa846245640 1 -- 172.21.15.16:0/1637445455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8400c8540 con 0x7fa8401592b0 2024-09-19T07:43:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.566+0000 7fa846245640 1 -- 172.21.15.16:0/1637445455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8400c8b00 con 0x7fa8401592b0 2024-09-19T07:43:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.567+0000 7fa846245640 1 -- 172.21.15.16:0/1637445455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa84006d420 con 0x7fa8401592b0 2024-09-19T07:43:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.568+0000 7fa827fff640 1 -- 172.21.15.16:0/1637445455 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa834045020 con 0x7fa8401592b0 2024-09-19T07:43:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.568+0000 7fa827fff640 1 --2- 172.21.15.16:0/1637445455 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa82c06c2e0 0x7fa82c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.568+0000 7fa827fff640 1 -- 172.21.15.16:0/1637445455 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa8340abea0 con 0x7fa8401592b0 2024-09-19T07:43:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.568+0000 7fa83f7fe640 1 --2- 172.21.15.16:0/1637445455 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa82c06c2e0 0x7fa82c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.569+0000 7fa83f7fe640 1 --2- 172.21.15.16:0/1637445455 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa82c06c2e0 0x7fa82c06e7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa830009a00 tx=0x7fa830008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.572+0000 7fa827fff640 1 -- 172.21.15.16:0/1637445455 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa83404c030 con 0x7fa8401592b0 2024-09-19T07:43:31.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.783+0000 7fa846245640 1 -- 172.21.15.16:0/1637445455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa84006cf90 con 0x7fa8401592b0 2024-09-19T07:43:31.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.784+0000 7fa827fff640 1 -- 172.21.15.16:0/1637445455 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5222 (secure 0 0 0) 0x7fa83407a360 con 0x7fa8401592b0 2024-09-19T07:43:31.784 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:31.785 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-19T07:43:25:334589+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2772948330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2772948330},{"type":"v1","addr":"172.21.15.146:6835","nonce":2772948330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/340172279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":340172279},{"type":"v1","addr":"172.21.15.16:6833","nonce":340172279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1204811459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1204811459},{"type":"v1","addr":"172.21.15.16:6835","nonce":1204811459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:43:05.610038+0000","modified":"2024-09-19T07:43:24.691417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/1303703249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1303703249},{"type":"v1","addr":"172.21.15.146:6837","nonce":1303703249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-09-19T07:43:31.785 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-19T07:43:31.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.786+0000 7fa846245640 1 -- 172.21.15.16:0/1637445455 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa82c06c2e0 msgr2=0x7fa82c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:31.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.786+0000 7fa846245640 1 --2- 172.21.15.16:0/1637445455 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa82c06c2e0 0x7fa82c06e7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa830009a00 tx=0x7fa830008040 comp rx=0 tx=0).stop 2024-09-19T07:43:31.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.786+0000 7fa846245640 1 -- 172.21.15.16:0/1637445455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8401592b0 msgr2=0x7fa840161f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:31.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.786+0000 7fa846245640 1 --2- 172.21.15.16:0/1637445455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8401592b0 0x7fa840161f40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa834001970 tx=0x7fa834001a40 comp rx=0 tx=0).stop 2024-09-19T07:43:31.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.786+0000 7fa846245640 1 -- 172.21.15.16:0/1637445455 shutdown_connections 2024-09-19T07:43:31.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.786+0000 7fa846245640 1 --2- 172.21.15.16:0/1637445455 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa82c06c2e0 0x7fa82c06e7a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.786+0000 7fa846245640 1 --2- 172.21.15.16:0/1637445455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa840159730 0x7fa840162480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.786+0000 7fa846245640 1 --2- 172.21.15.16:0/1637445455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8401592b0 0x7fa840161f40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.786+0000 7fa846245640 1 --2- 172.21.15.16:0/1637445455 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa828003f30 0x7fa840156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.786+0000 7fa846245640 1 -- 172.21.15.16:0/1637445455 >> 172.21.15.16:0/1637445455 conn(0x7fa84009d760 msgr2=0x7fa84006dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:31.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.786+0000 7fa846245640 1 -- 172.21.15.16:0/1637445455 shutdown_connections 2024-09-19T07:43:31.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.786+0000 7fa846245640 1 -- 172.21.15.16:0/1637445455 wait complete. 2024-09-19T07:43:31.799 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:43:31.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.902+0000 7fc76db91640 1 Processor -- start 2024-09-19T07:43:31.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.902+0000 7fc76db91640 1 -- start start 2024-09-19T07:43:31.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.903+0000 7fc76db91640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc76815e2a0 0x7fc76806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.903+0000 7fc76db91640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc76815e680 0x7fc7680cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.903+0000 7fc76db91640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7680d1880 con 0x7fc76806d960 2024-09-19T07:43:31.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.903+0000 7fc76db91640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc7680d1b60 con 0x7fc76815e680 2024-09-19T07:43:31.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.903+0000 7fc76db91640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc7680cf2d0 con 0x7fc76815e2a0 2024-09-19T07:43:31.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.903+0000 7fc766ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc76815e680 0x7fc7680cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.903+0000 7fc766ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc76815e680 0x7fc7680cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57804/0 (socket says 172.21.15.16:57804) 2024-09-19T07:43:31.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.903+0000 7fc766ffd640 1 -- 172.21.15.16:0/3342714213 learned_addr learned my addr 172.21.15.16:0/3342714213 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:31.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.903+0000 7fc767fff640 1 --1- 172.21.15.16:0/3342714213 >> v1:172.21.15.16:6789/0 conn(0x7fc76806d960 0x7fc7680d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44990/0 (socket says 172.21.15.16:44990) 2024-09-19T07:43:31.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.903+0000 7fc7677fe640 1 --2- 172.21.15.16:0/3342714213 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc76815e2a0 0x7fc76806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.903+0000 7fc766ffd640 1 -- 172.21.15.16:0/3342714213 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc76815e2a0 msgr2=0x7fc76806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:31.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.903+0000 7fc766ffd640 1 --2- 172.21.15.16:0/3342714213 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc76815e2a0 0x7fc76806d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.903+0000 7fc766ffd640 1 -- 172.21.15.16:0/3342714213 >> v1:172.21.15.16:6789/0 conn(0x7fc76806d960 legacy=0x7fc7680d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:31.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.903+0000 7fc766ffd640 1 -- 172.21.15.16:0/3342714213 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7680cf5b0 con 0x7fc76815e680 2024-09-19T07:43:31.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.904+0000 7fc766ffd640 1 --2- 172.21.15.16:0/3342714213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76815e680 0x7fc7680cfad0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009870 tx=0x7fc75c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f503acf3b4aa742 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:31.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.904+0000 7fc7667fc640 1 -- 172.21.15.16:0/3342714213 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c037070 con 0x7fc76815e680 2024-09-19T07:43:31.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.904+0000 7fc7667fc640 1 -- 172.21.15.16:0/3342714213 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75c032440 con 0x7fc76815e680 2024-09-19T07:43:31.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.904+0000 7fc7667fc640 1 -- 172.21.15.16:0/3342714213 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c032810 con 0x7fc76815e680 2024-09-19T07:43:31.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.905+0000 7fc76db91640 1 -- 172.21.15.16:0/3342714213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76815e680 msgr2=0x7fc7680cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:31.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.905+0000 7fc76db91640 1 --2- 172.21.15.16:0/3342714213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76815e680 0x7fc7680cfad0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009870 tx=0x7fc75c02a270 comp rx=0 tx=0).stop 2024-09-19T07:43:31.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.905+0000 7fc76db91640 1 -- 172.21.15.16:0/3342714213 shutdown_connections 2024-09-19T07:43:31.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.905+0000 7fc76db91640 1 --2- 172.21.15.16:0/3342714213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76815e680 0x7fc7680cfad0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.905+0000 7fc76db91640 1 --2- 172.21.15.16:0/3342714213 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc76815e2a0 0x7fc76806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.905+0000 7fc76db91640 1 -- 172.21.15.16:0/3342714213 >> 172.21.15.16:0/3342714213 conn(0x7fc76809d760 msgr2=0x7fc76808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:31.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.905+0000 7fc76db91640 1 -- 172.21.15.16:0/3342714213 shutdown_connections 2024-09-19T07:43:31.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.905+0000 7fc76db91640 1 -- 172.21.15.16:0/3342714213 wait complete. 2024-09-19T07:43:31.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.906+0000 7fc76db91640 1 Processor -- start 2024-09-19T07:43:31.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.906+0000 7fc76db91640 1 -- start start 2024-09-19T07:43:31.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.907+0000 7fc76db91640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc76806d960 0x7fc768161f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.907+0000 7fc76db91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76815e2a0 0x7fc768162480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.907+0000 7fc76db91640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc76815e680 0x7fc7680b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.907+0000 7fc76db91640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7680d4c00 con 0x7fc76815e2a0 2024-09-19T07:43:31.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.907+0000 7fc76db91640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc7680d4a80 con 0x7fc76806d960 2024-09-19T07:43:31.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.907+0000 7fc76db91640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc7680d0010 con 0x7fc76815e680 2024-09-19T07:43:31.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.907+0000 7fc766ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76815e2a0 0x7fc768162480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.907+0000 7fc766ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76815e2a0 0x7fc768162480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57816/0 (socket says 172.21.15.16:57816) 2024-09-19T07:43:31.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.907+0000 7fc766ffd640 1 -- 172.21.15.16:0/1782619256 learned_addr learned my addr 172.21.15.16:0/1782619256 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:31.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.907+0000 7fc7677fe640 1 --2- 172.21.15.16:0/1782619256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc76806d960 0x7fc768161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.907+0000 7fc767fff640 1 --2- 172.21.15.16:0/1782619256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc76815e680 0x7fc7680b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.907+0000 7fc766ffd640 1 -- 172.21.15.16:0/1782619256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc76815e680 msgr2=0x7fc7680b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:31.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.908+0000 7fc766ffd640 1 --2- 172.21.15.16:0/1782619256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc76815e680 0x7fc7680b8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.908+0000 7fc766ffd640 1 -- 172.21.15.16:0/1782619256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc76806d960 msgr2=0x7fc768161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:31.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.908+0000 7fc766ffd640 1 --2- 172.21.15.16:0/1782619256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc76806d960 0x7fc768161f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:31.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.908+0000 7fc766ffd640 1 -- 172.21.15.16:0/1782619256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7680ca0e0 con 0x7fc76815e2a0 2024-09-19T07:43:31.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.908+0000 7fc767fff640 1 --2- 172.21.15.16:0/1782619256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc76815e680 0x7fc7680b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:31.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.908+0000 7fc766ffd640 1 --2- 172.21.15.16:0/1782619256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76815e2a0 0x7fc768162480 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0090b0 tx=0x7fc75c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:31.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.908+0000 7fc74ffff640 1 -- 172.21.15.16:0/1782619256 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c041070 con 0x7fc76815e2a0 2024-09-19T07:43:31.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.908+0000 7fc74ffff640 1 -- 172.21.15.16:0/1782619256 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75c03c8b0 con 0x7fc76815e2a0 2024-09-19T07:43:31.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.908+0000 7fc76db91640 1 -- 172.21.15.16:0/1782619256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7680ca3c0 con 0x7fc76815e2a0 2024-09-19T07:43:31.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.908+0000 7fc74ffff640 1 -- 172.21.15.16:0/1782619256 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c037040 con 0x7fc76815e2a0 2024-09-19T07:43:31.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.908+0000 7fc76db91640 1 -- 172.21.15.16:0/1782619256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7680ca980 con 0x7fc76815e2a0 2024-09-19T07:43:31.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.909+0000 7fc76db91640 1 -- 172.21.15.16:0/1782619256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc76806d420 con 0x7fc76815e2a0 2024-09-19T07:43:31.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.909+0000 7fc74ffff640 1 -- 172.21.15.16:0/1782619256 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc75c044070 con 0x7fc76815e2a0 2024-09-19T07:43:31.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.910+0000 7fc74ffff640 1 --2- 172.21.15.16:0/1782619256 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc75406c2b0 0x7fc75406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:31.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.910+0000 7fc74ffff640 1 -- 172.21.15.16:0/1782619256 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc75c0ac490 con 0x7fc76815e2a0 2024-09-19T07:43:31.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.910+0000 7fc7677fe640 1 --2- 172.21.15.16:0/1782619256 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc75406c2b0 0x7fc75406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:31.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.911+0000 7fc7677fe640 1 --2- 172.21.15.16:0/1782619256 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc75406c2b0 0x7fc75406e770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc758006fd0 tx=0x7fc758008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:31.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:31.914+0000 7fc74ffff640 1 -- 172.21.15.16:0/1782619256 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc75c07a300 con 0x7fc76815e2a0 2024-09-19T07:43:32.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.125+0000 7fc76db91640 1 -- 172.21.15.16:0/1782619256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc76806cf90 con 0x7fc76815e2a0 2024-09-19T07:43:32.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.126+0000 7fc74ffff640 1 -- 172.21.15.16:0/1782619256 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5222 (secure 0 0 0) 0x7fc75c07e600 con 0x7fc76815e2a0 2024-09-19T07:43:32.127 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:32.127 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-19T07:43:25:334589+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2772948330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2772948330},{"type":"v1","addr":"172.21.15.146:6835","nonce":2772948330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/340172279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":340172279},{"type":"v1","addr":"172.21.15.16:6833","nonce":340172279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1204811459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1204811459},{"type":"v1","addr":"172.21.15.16:6835","nonce":1204811459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:43:05.610038+0000","modified":"2024-09-19T07:43:24.691417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/1303703249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1303703249},{"type":"v1","addr":"172.21.15.146:6837","nonce":1303703249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-09-19T07:43:32.127 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-19T07:43:32.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.128+0000 7fc76db91640 1 -- 172.21.15.16:0/1782619256 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc75406c2b0 msgr2=0x7fc75406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:32.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.128+0000 7fc76db91640 1 --2- 172.21.15.16:0/1782619256 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc75406c2b0 0x7fc75406e770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc758006fd0 tx=0x7fc758008040 comp rx=0 tx=0).stop 2024-09-19T07:43:32.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.128+0000 7fc76db91640 1 -- 172.21.15.16:0/1782619256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76815e2a0 msgr2=0x7fc768162480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:32.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.128+0000 7fc76db91640 1 --2- 172.21.15.16:0/1782619256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76815e2a0 0x7fc768162480 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0090b0 tx=0x7fc75c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.128+0000 7fc76db91640 1 -- 172.21.15.16:0/1782619256 shutdown_connections 2024-09-19T07:43:32.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.128+0000 7fc76db91640 1 --2- 172.21.15.16:0/1782619256 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc75406c2b0 0x7fc75406e770 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.128+0000 7fc76db91640 1 --2- 172.21.15.16:0/1782619256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc76815e680 0x7fc7680b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.128+0000 7fc76db91640 1 --2- 172.21.15.16:0/1782619256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76815e2a0 0x7fc768162480 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.128+0000 7fc76db91640 1 --2- 172.21.15.16:0/1782619256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc76806d960 0x7fc768161f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.128+0000 7fc76db91640 1 -- 172.21.15.16:0/1782619256 >> 172.21.15.16:0/1782619256 conn(0x7fc76809d760 msgr2=0x7fc76806ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:32.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.128+0000 7fc76db91640 1 -- 172.21.15.16:0/1782619256 shutdown_connections 2024-09-19T07:43:32.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.128+0000 7fc76db91640 1 -- 172.21.15.16:0/1782619256 wait complete. 2024-09-19T07:43:32.139 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:43:32.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.241+0000 7f0dbd51f640 1 Processor -- start 2024-09-19T07:43:32.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.241+0000 7f0dbd51f640 1 -- start start 2024-09-19T07:43:32.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.242+0000 7f0dbd51f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0db815e2a0 0x7f0db806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:32.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.242+0000 7f0dbd51f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0db815e680 0x7f0db80cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:32.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.242+0000 7f0dbd51f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0db806d960 0x7f0db80d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:32.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.242+0000 7f0dbd51f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f0db80d1880 con 0x7f0db815e2a0 2024-09-19T07:43:32.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.242+0000 7f0dbd51f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f0db80d1b60 con 0x7f0db815e680 2024-09-19T07:43:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.242+0000 7f0dbd51f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0db80cf2d0 con 0x7f0db806d960 2024-09-19T07:43:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.242+0000 7f0db6ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0db815e2a0 0x7f0db806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.242+0000 7f0db6ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0db815e2a0 0x7f0db806d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57830/0 (socket says 172.21.15.16:57830) 2024-09-19T07:43:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.242+0000 7f0db6ffd640 1 -- 172.21.15.16:0/1650904738 learned_addr learned my addr 172.21.15.16:0/1650904738 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.242+0000 7f0db77fe640 1 --2- 172.21.15.16:0/1650904738 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0db806d960 0x7f0db80d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.242+0000 7f0db67fc640 1 --2- 172.21.15.16:0/1650904738 >> v2:172.21.15.146:3300/0 conn(0x7f0db815e680 0x7f0db80cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.242+0000 7f0db6ffd640 1 -- 172.21.15.16:0/1650904738 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0db806d960 msgr2=0x7f0db80d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.242+0000 7f0db6ffd640 1 --2- 172.21.15.16:0/1650904738 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0db806d960 0x7f0db80d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.242+0000 7f0db6ffd640 1 -- 172.21.15.16:0/1650904738 >> v2:172.21.15.146:3300/0 conn(0x7f0db815e680 msgr2=0x7f0db80cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.242+0000 7f0db6ffd640 1 --2- 172.21.15.16:0/1650904738 >> v2:172.21.15.146:3300/0 conn(0x7f0db815e680 0x7f0db80cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.242+0000 7f0db6ffd640 1 -- 172.21.15.16:0/1650904738 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db80cf5b0 con 0x7f0db815e2a0 2024-09-19T07:43:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.243+0000 7f0db6ffd640 1 --2- 172.21.15.16:0/1650904738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db815e2a0 0x7f0db806d420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0db80a22a0 tx=0x7f0db002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb4184b08811d41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.243+0000 7f0db5ffb640 1 -- 172.21.15.16:0/1650904738 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0037070 con 0x7f0db815e2a0 2024-09-19T07:43:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.243+0000 7f0db5ffb640 1 -- 172.21.15.16:0/1650904738 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db0032440 con 0x7f0db815e2a0 2024-09-19T07:43:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.243+0000 7f0db5ffb640 1 -- 172.21.15.16:0/1650904738 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0032810 con 0x7f0db815e2a0 2024-09-19T07:43:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.244+0000 7f0dbd51f640 1 -- 172.21.15.16:0/1650904738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db815e2a0 msgr2=0x7f0db806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.244+0000 7f0dbd51f640 1 --2- 172.21.15.16:0/1650904738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db815e2a0 0x7f0db806d420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0db80a22a0 tx=0x7f0db002a1e0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.244+0000 7f0dbd51f640 1 -- 172.21.15.16:0/1650904738 shutdown_connections 2024-09-19T07:43:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.244+0000 7f0dbd51f640 1 --2- 172.21.15.16:0/1650904738 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0db806d960 0x7f0db80d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.244+0000 7f0dbd51f640 1 --2- 172.21.15.16:0/1650904738 >> v2:172.21.15.146:3300/0 conn(0x7f0db815e680 0x7f0db80cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.244+0000 7f0dbd51f640 1 --2- 172.21.15.16:0/1650904738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db815e2a0 0x7f0db806d420 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.244+0000 7f0dbd51f640 1 -- 172.21.15.16:0/1650904738 >> 172.21.15.16:0/1650904738 conn(0x7f0db809d760 msgr2=0x7f0db808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.244+0000 7f0dbd51f640 1 -- 172.21.15.16:0/1650904738 shutdown_connections 2024-09-19T07:43:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.244+0000 7f0dbd51f640 1 -- 172.21.15.16:0/1650904738 wait complete. 2024-09-19T07:43:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.245+0000 7f0dbd51f640 1 Processor -- start 2024-09-19T07:43:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.245+0000 7f0dbd51f640 1 -- start start 2024-09-19T07:43:32.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.246+0000 7f0dbd51f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0db806d960 0x7f0db80c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.246+0000 7f0dbd51f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0db815e2a0 0x7f0db80c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.246+0000 7f0dbd51f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db815e680 0x7f0db80ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.246+0000 7f0dbd51f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0db80d44f0 con 0x7f0db815e680 2024-09-19T07:43:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.246+0000 7f0dbd51f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0db80d4370 con 0x7f0db806d960 2024-09-19T07:43:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.246+0000 7f0dbd51f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0db80d4670 con 0x7f0db815e2a0 2024-09-19T07:43:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.246+0000 7f0db77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db815e680 0x7f0db80ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.246+0000 7f0db77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db815e680 0x7f0db80ca110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57836/0 (socket says 172.21.15.16:57836) 2024-09-19T07:43:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.246+0000 7f0db77fe640 1 -- 172.21.15.16:0/2583381922 learned_addr learned my addr 172.21.15.16:0/2583381922 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.246+0000 7f0db67fc640 1 --2- 172.21.15.16:0/2583381922 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0db815e2a0 0x7f0db80c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.246+0000 7f0db6ffd640 1 --2- 172.21.15.16:0/2583381922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0db806d960 0x7f0db80c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.246+0000 7f0db77fe640 1 -- 172.21.15.16:0/2583381922 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0db815e2a0 msgr2=0x7f0db80c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.246+0000 7f0db77fe640 1 --2- 172.21.15.16:0/2583381922 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0db815e2a0 0x7f0db80c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.246+0000 7f0db77fe640 1 -- 172.21.15.16:0/2583381922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0db806d960 msgr2=0x7f0db80c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.246+0000 7f0db77fe640 1 --2- 172.21.15.16:0/2583381922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0db806d960 0x7f0db80c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.246+0000 7f0db77fe640 1 -- 172.21.15.16:0/2583381922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db80b9080 con 0x7f0db815e680 2024-09-19T07:43:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.247+0000 7f0db77fe640 1 --2- 172.21.15.16:0/2583381922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db815e680 0x7f0db80ca110 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0da8007e40 tx=0x7f0da800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.247+0000 7f0d9f7fe640 1 -- 172.21.15.16:0/2583381922 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8014620 con 0x7f0db815e680 2024-09-19T07:43:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.247+0000 7f0d9f7fe640 1 -- 172.21.15.16:0/2583381922 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da8019070 con 0x7f0db815e680 2024-09-19T07:43:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.247+0000 7f0dbd51f640 1 -- 172.21.15.16:0/2583381922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db80b9360 con 0x7f0db815e680 2024-09-19T07:43:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.247+0000 7f0d9f7fe640 1 -- 172.21.15.16:0/2583381922 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8014a20 con 0x7f0db815e680 2024-09-19T07:43:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.247+0000 7f0dbd51f640 1 -- 172.21.15.16:0/2583381922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db80cae30 con 0x7f0db815e680 2024-09-19T07:43:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.248+0000 7f0dbd51f640 1 -- 172.21.15.16:0/2583381922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db806d420 con 0x7f0db815e680 2024-09-19T07:43:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.248+0000 7f0d9f7fe640 1 -- 172.21.15.16:0/2583381922 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0da8014bc0 con 0x7f0db815e680 2024-09-19T07:43:32.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.249+0000 7f0d9f7fe640 1 --2- 172.21.15.16:0/2583381922 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0da406c2b0 0x7f0da406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:32.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.249+0000 7f0d9f7fe640 1 -- 172.21.15.16:0/2583381922 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0da808be60 con 0x7f0db815e680 2024-09-19T07:43:32.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.249+0000 7f0db6ffd640 1 --2- 172.21.15.16:0/2583381922 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0da406c2b0 0x7f0da406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:32.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.249+0000 7f0db6ffd640 1 --2- 172.21.15.16:0/2583381922 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0da406c2b0 0x7f0da406e770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0db0034000 tx=0x7f0db002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:32.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.253+0000 7f0d9f7fe640 1 -- 172.21.15.16:0/2583381922 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f0da8059cd0 con 0x7f0db815e680 2024-09-19T07:43:32.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.435+0000 7f0dbd51f640 1 -- 172.21.15.16:0/2583381922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0db80d3190 con 0x7f0db815e680 2024-09-19T07:43:32.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.436+0000 7f0d9f7fe640 1 -- 172.21.15.16:0/2583381922 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7f0da805dfd0 con 0x7f0db815e680 2024-09-19T07:43:32.436 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:32.437 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:43:10.642082+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:43:03.485582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:43:04.603961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:43:08.299591+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:43:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.438+0000 7f0dbd51f640 1 -- 172.21.15.16:0/2583381922 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0da406c2b0 msgr2=0x7f0da406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.438+0000 7f0dbd51f640 1 --2- 172.21.15.16:0/2583381922 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0da406c2b0 0x7f0da406e770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0db0034000 tx=0x7f0db002e000 comp rx=0 tx=0).stop 2024-09-19T07:43:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.438+0000 7f0dbd51f640 1 -- 172.21.15.16:0/2583381922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db815e680 msgr2=0x7f0db80ca110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.438+0000 7f0dbd51f640 1 --2- 172.21.15.16:0/2583381922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db815e680 0x7f0db80ca110 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0da8007e40 tx=0x7f0da800e4b0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.438+0000 7f0dbd51f640 1 -- 172.21.15.16:0/2583381922 shutdown_connections 2024-09-19T07:43:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.438+0000 7f0dbd51f640 1 --2- 172.21.15.16:0/2583381922 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0da406c2b0 0x7f0da406e770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.438+0000 7f0dbd51f640 1 --2- 172.21.15.16:0/2583381922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0db815e680 0x7f0db80ca110 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.438+0000 7f0dbd51f640 1 --2- 172.21.15.16:0/2583381922 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0db815e2a0 0x7f0db80c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.438+0000 7f0dbd51f640 1 --2- 172.21.15.16:0/2583381922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0db806d960 0x7f0db80c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.438+0000 7f0dbd51f640 1 -- 172.21.15.16:0/2583381922 >> 172.21.15.16:0/2583381922 conn(0x7f0db809d760 msgr2=0x7f0db8160910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.438+0000 7f0dbd51f640 1 -- 172.21.15.16:0/2583381922 shutdown_connections 2024-09-19T07:43:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.438+0000 7f0dbd51f640 1 -- 172.21.15.16:0/2583381922 wait complete. 2024-09-19T07:43:32.449 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:43:32.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.551+0000 7fb92a103640 1 Processor -- start 2024-09-19T07:43:32.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.551+0000 7fb92a103640 1 -- start start 2024-09-19T07:43:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.552+0000 7fb92a103640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb924072250 0x7fb9240cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.552+0000 7fb92a103640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9240d2b30 con 0x7fb9240d2750 2024-09-19T07:43:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.552+0000 7fb92a103640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb924141c60 con 0x7fb9240726c0 2024-09-19T07:43:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.552+0000 7fb92a103640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb924145bd0 con 0x7fb924072250 2024-09-19T07:43:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.552+0000 7fb9237fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb9240d2750 0x7fb92413f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38304/0 (socket says 172.21.15.16:38304) 2024-09-19T07:43:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.552+0000 7fb9237fe640 1 -- 172.21.15.16:0/395935562 learned_addr learned my addr 172.21.15.16:0/395935562 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.552+0000 7fb922ffd640 1 --2- 172.21.15.16:0/395935562 >> v2:172.21.15.146:3300/0 conn(0x7fb924072250 0x7fb9240cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.553+0000 7fb9227fc640 1 -- 172.21.15.16:0/395935562 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4026692505 0 0) 0x7fb9240d2b30 con 0x7fb9240d2750 2024-09-19T07:43:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.553+0000 7fb9227fc640 1 -- 172.21.15.16:0/395935562 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb910003520 con 0x7fb9240d2750 2024-09-19T07:43:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.553+0000 7fb922ffd640 1 -- 172.21.15.16:0/395935562 >> v1:172.21.15.146:6789/0 conn(0x7fb9240726c0 legacy=0x7fb924160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.553+0000 7fb922ffd640 1 -- 172.21.15.16:0/395935562 >> v1:172.21.15.16:6789/0 conn(0x7fb9240d2750 legacy=0x7fb92413f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.553+0000 7fb922ffd640 1 -- 172.21.15.16:0/395935562 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb924141f40 con 0x7fb924072250 2024-09-19T07:43:32.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.554+0000 7fb922ffd640 1 --2- 172.21.15.16:0/395935562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb924072250 0x7fb9240cfdc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0097a0 tx=0x7fb90c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa565e980f62f2db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:32.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.554+0000 7fb9227fc640 1 -- 172.21.15.16:0/395935562 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c037070 con 0x7fb924072250 2024-09-19T07:43:32.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.554+0000 7fb9227fc640 1 -- 172.21.15.16:0/395935562 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb90c032440 con 0x7fb924072250 2024-09-19T07:43:32.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.554+0000 7fb9227fc640 1 -- 172.21.15.16:0/395935562 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c032810 con 0x7fb924072250 2024-09-19T07:43:32.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.554+0000 7fb92a103640 1 -- 172.21.15.16:0/395935562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb924072250 msgr2=0x7fb9240cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:32.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.554+0000 7fb92a103640 1 --2- 172.21.15.16:0/395935562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb924072250 0x7fb9240cfdc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0097a0 tx=0x7fb90c02a1a0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.554+0000 7fb92a103640 1 -- 172.21.15.16:0/395935562 shutdown_connections 2024-09-19T07:43:32.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.554+0000 7fb92a103640 1 --2- 172.21.15.16:0/395935562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb924072250 0x7fb9240cfdc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.554+0000 7fb92a103640 1 -- 172.21.15.16:0/395935562 >> 172.21.15.16:0/395935562 conn(0x7fb9240b4040 msgr2=0x7fb9240b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:32.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.554+0000 7fb92a103640 1 -- 172.21.15.16:0/395935562 shutdown_connections 2024-09-19T07:43:32.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.554+0000 7fb92a103640 1 -- 172.21.15.16:0/395935562 wait complete. 2024-09-19T07:43:32.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.555+0000 7fb92a103640 1 Processor -- start 2024-09-19T07:43:32.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.556+0000 7fb92a103640 1 -- start start 2024-09-19T07:43:32.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.556+0000 7fb92a103640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb924072250 0x7fb92415a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:32.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.556+0000 7fb92a103640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9240726c0 0x7fb9241442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:32.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.556+0000 7fb92a103640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9240d2750 0x7fb9241447f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:32.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.556+0000 7fb92a103640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb924142500 con 0x7fb924072250 2024-09-19T07:43:32.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.556+0000 7fb92a103640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241427e0 con 0x7fb9240726c0 2024-09-19T07:43:32.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.556+0000 7fb92a103640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb9241418b0 con 0x7fb9240d2750 2024-09-19T07:43:32.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.557+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb924072250 0x7fb92415a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:32.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.557+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb924072250 0x7fb92415a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55976/0 (socket says 172.21.15.16:55976) 2024-09-19T07:43:32.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.557+0000 7fb9237fe640 1 -- 172.21.15.16:0/199788262 learned_addr learned my addr 172.21.15.16:0/199788262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:32.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.557+0000 7fb922ffd640 1 --2- 172.21.15.16:0/199788262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9240726c0 0x7fb9241442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:32.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.557+0000 7fb923fff640 1 --2- 172.21.15.16:0/199788262 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9240d2750 0x7fb9241447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:32.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.557+0000 7fb9237fe640 1 -- 172.21.15.16:0/199788262 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9240d2750 msgr2=0x7fb9241447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:32.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.557+0000 7fb9237fe640 1 --2- 172.21.15.16:0/199788262 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9240d2750 0x7fb9241447f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.557+0000 7fb9237fe640 1 -- 172.21.15.16:0/199788262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9240726c0 msgr2=0x7fb9241442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:32.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.557+0000 7fb9237fe640 1 --2- 172.21.15.16:0/199788262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9240726c0 0x7fb9241442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.557+0000 7fb9237fe640 1 -- 172.21.15.16:0/199788262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92415aba0 con 0x7fb924072250 2024-09-19T07:43:32.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.557+0000 7fb923fff640 1 --2- 172.21.15.16:0/199788262 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9240d2750 0x7fb9241447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.557+0000 7fb9237fe640 1 --2- 172.21.15.16:0/199788262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb924072250 0x7fb92415a660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb91400c500 tx=0x7fb91400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.557+0000 7fb91bfff640 1 -- 172.21.15.16:0/199788262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914017070 con 0x7fb924072250 2024-09-19T07:43:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.557+0000 7fb91bfff640 1 -- 172.21.15.16:0/199788262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb914005ce0 con 0x7fb924072250 2024-09-19T07:43:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.557+0000 7fb92a103640 1 -- 172.21.15.16:0/199788262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92406cd20 con 0x7fb924072250 2024-09-19T07:43:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.557+0000 7fb91bfff640 1 -- 172.21.15.16:0/199788262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914007330 con 0x7fb924072250 2024-09-19T07:43:32.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.557+0000 7fb92a103640 1 -- 172.21.15.16:0/199788262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb92406d2e0 con 0x7fb924072250 2024-09-19T07:43:32.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.558+0000 7fb92a103640 1 -- 172.21.15.16:0/199788262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9241409e0 con 0x7fb924072250 2024-09-19T07:43:32.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.559+0000 7fb91bfff640 1 -- 172.21.15.16:0/199788262 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb91401a070 con 0x7fb924072250 2024-09-19T07:43:32.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.559+0000 7fb91bfff640 1 --2- 172.21.15.16:0/199788262 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb90406c2b0 0x7fb90406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:32.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.559+0000 7fb91bfff640 1 -- 172.21.15.16:0/199788262 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb91408ab90 con 0x7fb924072250 2024-09-19T07:43:32.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.559+0000 7fb922ffd640 1 --2- 172.21.15.16:0/199788262 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb90406c2b0 0x7fb90406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:32.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.560+0000 7fb922ffd640 1 --2- 172.21.15.16:0/199788262 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb90406c2b0 0x7fb90406e770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb90c034000 tx=0x7fb90c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:32.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.563+0000 7fb91bfff640 1 -- 172.21.15.16:0/199788262 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb914059110 con 0x7fb924072250 2024-09-19T07:43:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.776+0000 7fb92a103640 1 -- 172.21.15.16:0/199788262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb924140e90 con 0x7fb924072250 2024-09-19T07:43:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.776+0000 7fb91bfff640 1 -- 172.21.15.16:0/199788262 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5222 (secure 0 0 0) 0x7fb914058a10 con 0x7fb924072250 2024-09-19T07:43:32.778 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:32.778 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-19T07:43:25:334589+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2772948330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2772948330},{"type":"v1","addr":"172.21.15.146:6835","nonce":2772948330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/340172279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":340172279},{"type":"v1","addr":"172.21.15.16:6833","nonce":340172279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1204811459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1204811459},{"type":"v1","addr":"172.21.15.16:6835","nonce":1204811459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:43:05.610038+0000","modified":"2024-09-19T07:43:24.691417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/1303703249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1303703249},{"type":"v1","addr":"172.21.15.146:6837","nonce":1303703249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-09-19T07:43:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-19T07:43:32.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.779+0000 7fb92a103640 1 -- 172.21.15.16:0/199788262 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb90406c2b0 msgr2=0x7fb90406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:32.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.779+0000 7fb92a103640 1 --2- 172.21.15.16:0/199788262 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb90406c2b0 0x7fb90406e770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb90c034000 tx=0x7fb90c02e000 comp rx=0 tx=0).stop 2024-09-19T07:43:32.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.779+0000 7fb92a103640 1 -- 172.21.15.16:0/199788262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb924072250 msgr2=0x7fb92415a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:32.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.779+0000 7fb92a103640 1 --2- 172.21.15.16:0/199788262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb924072250 0x7fb92415a660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb91400c500 tx=0x7fb91400c9c0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.779+0000 7fb92a103640 1 -- 172.21.15.16:0/199788262 shutdown_connections 2024-09-19T07:43:32.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.779+0000 7fb92a103640 1 --2- 172.21.15.16:0/199788262 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb90406c2b0 0x7fb90406e770 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.779+0000 7fb92a103640 1 --2- 172.21.15.16:0/199788262 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9240d2750 0x7fb9241447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.779+0000 7fb92a103640 1 --2- 172.21.15.16:0/199788262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9240726c0 0x7fb9241442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.779+0000 7fb92a103640 1 --2- 172.21.15.16:0/199788262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb924072250 0x7fb92415a660 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:32.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.779+0000 7fb92a103640 1 -- 172.21.15.16:0/199788262 >> 172.21.15.16:0/199788262 conn(0x7fb9240b4040 msgr2=0x7fb9240cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:32.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.779+0000 7fb92a103640 1 -- 172.21.15.16:0/199788262 shutdown_connections 2024-09-19T07:43:32.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:32.779+0000 7fb92a103640 1 -- 172.21.15.16:0/199788262 wait complete. 2024-09-19T07:43:33.192 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T07:43:05.610038+0000', 'modified': '2024-09-19T07:43:24.691417+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4278}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4278': {'gid': 4278, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.146:6837/1303703249', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6836', 'nonce': 1303703249}, {'type': 'v1', 'addr': '172.21.15.146:6837', 'nonce': 1303703249}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4278, 'qdb_cluster': [4278]} 2024-09-19T07:43:33.192 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T07:43:33.193 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-19T07:43:33.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.298+0000 7f372da42640 1 Processor -- start 2024-09-19T07:43:33.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.298+0000 7f372da42640 1 -- start start 2024-09-19T07:43:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.299+0000 7f372da42640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37280c8d20 0x7f372806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.299+0000 7f372da42640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f372806d260 0x7f37281439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.299+0000 7f372da42640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37280c7140 con 0x7f3728143f30 2024-09-19T07:43:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.299+0000 7f372da42640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f3728141960 con 0x7f372806d260 2024-09-19T07:43:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.299+0000 7f372da42640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3728142200 con 0x7f37280c8d20 2024-09-19T07:43:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.299+0000 7f37267fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f372806d260 0x7f37281439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.299+0000 7f37277fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3728143f30 0x7f37280d1e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38320/0 (socket says 172.21.15.16:38320) 2024-09-19T07:43:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.299+0000 7f37277fe640 1 -- 172.21.15.16:0/230389801 learned_addr learned my addr 172.21.15.16:0/230389801 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:33.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.299+0000 7f3726ffd640 1 --2- 172.21.15.16:0/230389801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37280c8d20 0x7f372806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.299+0000 7f37267fc640 1 -- 172.21.15.16:0/230389801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37280c8d20 msgr2=0x7f372806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:33.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.299+0000 7f37267fc640 1 --2- 172.21.15.16:0/230389801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37280c8d20 0x7f372806cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.299+0000 7f37267fc640 1 -- 172.21.15.16:0/230389801 >> v1:172.21.15.16:6789/0 conn(0x7f3728143f30 legacy=0x7f37280d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:33.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.299+0000 7f37267fc640 1 -- 172.21.15.16:0/230389801 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37280c9100 con 0x7f372806d260 2024-09-19T07:43:33.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.300+0000 7f37267fc640 1 --2- 172.21.15.16:0/230389801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372806d260 0x7f37281439f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f37100097a0 tx=0x7f371002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c842f693278bdd66 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:33.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.300+0000 7f3725ffb640 1 -- 172.21.15.16:0/230389801 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710037070 con 0x7f372806d260 2024-09-19T07:43:33.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.300+0000 7f3725ffb640 1 -- 172.21.15.16:0/230389801 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f371002adb0 con 0x7f372806d260 2024-09-19T07:43:33.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.300+0000 7f3725ffb640 1 -- 172.21.15.16:0/230389801 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710032660 con 0x7f372806d260 2024-09-19T07:43:33.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.301+0000 7f372da42640 1 -- 172.21.15.16:0/230389801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372806d260 msgr2=0x7f37281439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:33.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.301+0000 7f372da42640 1 --2- 172.21.15.16:0/230389801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372806d260 0x7f37281439f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f37100097a0 tx=0x7f371002a060 comp rx=0 tx=0).stop 2024-09-19T07:43:33.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.301+0000 7f372da42640 1 -- 172.21.15.16:0/230389801 shutdown_connections 2024-09-19T07:43:33.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.301+0000 7f372da42640 1 --2- 172.21.15.16:0/230389801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372806d260 0x7f37281439f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.301+0000 7f372da42640 1 --2- 172.21.15.16:0/230389801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37280c8d20 0x7f372806cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.301+0000 7f372da42640 1 -- 172.21.15.16:0/230389801 >> 172.21.15.16:0/230389801 conn(0x7f37280b3ed0 msgr2=0x7f37280b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:33.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.301+0000 7f372da42640 1 -- 172.21.15.16:0/230389801 shutdown_connections 2024-09-19T07:43:33.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.301+0000 7f372da42640 1 -- 172.21.15.16:0/230389801 wait complete. 2024-09-19T07:43:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.302+0000 7f372da42640 1 Processor -- start 2024-09-19T07:43:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.302+0000 7f372da42640 1 -- start start 2024-09-19T07:43:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.303+0000 7f372da42640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f372806d260 0x7f37280cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.303+0000 7f372da42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280c8d20 0x7f3728153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.303+0000 7f372da42640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3728143f30 0x7f37280d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.303+0000 7f372da42640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3728141bb0 con 0x7f37280c8d20 2024-09-19T07:43:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.303+0000 7f372da42640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f372806d6a0 con 0x7f3728143f30 2024-09-19T07:43:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.303+0000 7f372da42640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3728141e90 con 0x7f372806d260 2024-09-19T07:43:33.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.303+0000 7f37267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280c8d20 0x7f3728153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.303+0000 7f37267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280c8d20 0x7f3728153fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55988/0 (socket says 172.21.15.16:55988) 2024-09-19T07:43:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.303+0000 7f37267fc640 1 -- 172.21.15.16:0/604350551 learned_addr learned my addr 172.21.15.16:0/604350551 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.303+0000 7f37277fe640 1 --2- 172.21.15.16:0/604350551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3728143f30 0x7f37280d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.303+0000 7f3726ffd640 1 --2- 172.21.15.16:0/604350551 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f372806d260 0x7f37280cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.304+0000 7f37267fc640 1 -- 172.21.15.16:0/604350551 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f372806d260 msgr2=0x7f37280cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.304+0000 7f37267fc640 1 --2- 172.21.15.16:0/604350551 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f372806d260 0x7f37280cbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.304+0000 7f37267fc640 1 -- 172.21.15.16:0/604350551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3728143f30 msgr2=0x7f37280d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.304+0000 7f37267fc640 1 --2- 172.21.15.16:0/604350551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3728143f30 0x7f37280d0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.304+0000 7f37267fc640 1 -- 172.21.15.16:0/604350551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37280c9460 con 0x7f37280c8d20 2024-09-19T07:43:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.304+0000 7f37277fe640 1 --2- 172.21.15.16:0/604350551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3728143f30 0x7f37280d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.304+0000 7f3726ffd640 1 --2- 172.21.15.16:0/604350551 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f372806d260 0x7f37280cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.304+0000 7f37267fc640 1 --2- 172.21.15.16:0/604350551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280c8d20 0x7f3728153fd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3710035040 tx=0x7f37100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.304+0000 7f371ffff640 1 -- 172.21.15.16:0/604350551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710041070 con 0x7f37280c8d20 2024-09-19T07:43:33.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.304+0000 7f371ffff640 1 -- 172.21.15.16:0/604350551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f371003c670 con 0x7f37280c8d20 2024-09-19T07:43:33.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.304+0000 7f372da42640 1 -- 172.21.15.16:0/604350551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37280c9740 con 0x7f37280c8d20 2024-09-19T07:43:33.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.304+0000 7f371ffff640 1 -- 172.21.15.16:0/604350551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710037040 con 0x7f37280c8d20 2024-09-19T07:43:33.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.304+0000 7f372da42640 1 -- 172.21.15.16:0/604350551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37280c9d00 con 0x7f37280c8d20 2024-09-19T07:43:33.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.306+0000 7f371ffff640 1 -- 172.21.15.16:0/604350551 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f371003a070 con 0x7f37280c8d20 2024-09-19T07:43:33.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.306+0000 7f371ffff640 1 --2- 172.21.15.16:0/604350551 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f370806c2b0 0x7f370806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.306+0000 7f371ffff640 1 -- 172.21.15.16:0/604350551 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f37100abac0 con 0x7f37280c8d20 2024-09-19T07:43:33.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.307+0000 7f3726ffd640 1 --2- 172.21.15.16:0/604350551 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f370806c2b0 0x7f370806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.307+0000 7f37307d3740 1 Processor -- start 2024-09-19T07:43:33.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.307+0000 7f37307d3740 1 -- start start 2024-09-19T07:43:33.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.307+0000 7f3726ffd640 1 --2- 172.21.15.16:0/604350551 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f370806c2b0 0x7f370806e770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3728154ca0 tx=0x7f371800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:33.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.308+0000 7f37307d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559348362c0 0x55593487ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.308+0000 7f37307d3740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934833a80 0x5559348366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.308+0000 7f37307d3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555934837a50 0x5559348308b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.308+0000 7f37307d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555934798520 con 0x5559348362c0 2024-09-19T07:43:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.308+0000 7f37307d3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5559347403e0 con 0x555934837a50 2024-09-19T07:43:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.308+0000 7f37307d3740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x5559347944e0 con 0x555934833a80 2024-09-19T07:43:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.308+0000 7f3726ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559348362c0 0x55593487ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.308+0000 7f3726ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559348362c0 0x55593487ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56000/0 (socket says 172.21.15.16:56000) 2024-09-19T07:43:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.308+0000 7f3726ffd640 1 -- 172.21.15.16:0/3484372997 learned_addr learned my addr 172.21.15.16:0/3484372997 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.308+0000 7f37277fe640 1 --2- 172.21.15.16:0/3484372997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555934837a50 0x5559348308b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.308+0000 7f37267fc640 1 --2- 172.21.15.16:0/3484372997 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934833a80 0x5559348366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.309+0000 7f3726ffd640 1 -- 172.21.15.16:0/3484372997 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934833a80 msgr2=0x5559348366a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.309+0000 7f3726ffd640 1 --2- 172.21.15.16:0/3484372997 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934833a80 0x5559348366a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.309+0000 7f3726ffd640 1 -- 172.21.15.16:0/3484372997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555934837a50 msgr2=0x5559348308b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.309+0000 7f3726ffd640 1 --2- 172.21.15.16:0/3484372997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555934837a50 0x5559348308b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.309+0000 7f3726ffd640 1 -- 172.21.15.16:0/3484372997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55593475b2f0 con 0x5559348362c0 2024-09-19T07:43:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.309+0000 7f3726ffd640 1 --2- 172.21.15.16:0/3484372997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559348362c0 0x55593487ba60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3718016740 tx=0x7f3718012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3dc81ae744a86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.309+0000 7f371d7fa640 1 -- 172.21.15.16:0/3484372997 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718026070 con 0x5559348362c0 2024-09-19T07:43:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.309+0000 7f371d7fa640 1 -- 172.21.15.16:0/3484372997 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3718014a20 con 0x5559348362c0 2024-09-19T07:43:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.309+0000 7f371d7fa640 1 -- 172.21.15.16:0/3484372997 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718014dd0 con 0x5559348362c0 2024-09-19T07:43:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.310+0000 7f37307d3740 1 -- 172.21.15.16:0/3484372997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559348362c0 msgr2=0x55593487ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.310+0000 7f37307d3740 1 --2- 172.21.15.16:0/3484372997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559348362c0 0x55593487ba60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3718016740 tx=0x7f3718012000 comp rx=0 tx=0).stop 2024-09-19T07:43:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.310+0000 7f37307d3740 1 -- 172.21.15.16:0/3484372997 shutdown_connections 2024-09-19T07:43:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.310+0000 7f37307d3740 1 --2- 172.21.15.16:0/3484372997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555934837a50 0x5559348308b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.310+0000 7f37307d3740 1 --2- 172.21.15.16:0/3484372997 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934833a80 0x5559348366a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.310+0000 7f37307d3740 1 --2- 172.21.15.16:0/3484372997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559348362c0 0x55593487ba60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.310+0000 7f37307d3740 1 -- 172.21.15.16:0/3484372997 >> 172.21.15.16:0/3484372997 conn(0x555934832a80 msgr2=0x55593487af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.310+0000 7f37307d3740 1 -- 172.21.15.16:0/3484372997 shutdown_connections 2024-09-19T07:43:33.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.310+0000 7f37307d3740 1 -- 172.21.15.16:0/3484372997 wait complete. 2024-09-19T07:43:33.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.310+0000 7f37307d3740 1 -- start start 2024-09-19T07:43:33.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.310+0000 7f37307d3740 1 Processor -- start 2024-09-19T07:43:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.311+0000 7f37307d3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559348362c0 0x555934854ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.311+0000 7f37307d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 0x55593488f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.311+0000 7f37307d3740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934857140 0x555934834e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.311+0000 7f37307d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555934831f20 con 0x555934837a50 2024-09-19T07:43:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.311+0000 7f37307d3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x555934721450 con 0x5559348362c0 2024-09-19T07:43:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.311+0000 7f37307d3740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55593487c290 con 0x555934857140 2024-09-19T07:43:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.311+0000 7f37307d3740 10 client.0 _ll_register_callbacks cb 0x5559344aaac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.311+0000 7f37267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 0x55593488f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.311+0000 7f37267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 0x55593488f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56010/0 (socket says 172.21.15.16:56010) 2024-09-19T07:43:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.311+0000 7f37267fc640 1 -- 172.21.15.16:0/4200850372 learned_addr learned my addr 172.21.15.16:0/4200850372 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.311+0000 7f3726ffd640 1 --2- 172.21.15.16:0/4200850372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559348362c0 0x555934854ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.311+0000 7f37277fe640 1 --2- 172.21.15.16:0/4200850372 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934857140 0x555934834e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.311+0000 7f37267fc640 1 -- 172.21.15.16:0/4200850372 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934857140 msgr2=0x555934834e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.311+0000 7f37267fc640 1 --2- 172.21.15.16:0/4200850372 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934857140 0x555934834e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.311+0000 7f37267fc640 1 -- 172.21.15.16:0/4200850372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559348362c0 msgr2=0x555934854ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.311+0000 7f37267fc640 1 --2- 172.21.15.16:0/4200850372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559348362c0 0x555934854ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.311+0000 7f37267fc640 1 -- 172.21.15.16:0/4200850372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559344ab2d0 con 0x555934837a50 2024-09-19T07:43:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.312+0000 7f37277fe640 1 --2- 172.21.15.16:0/4200850372 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934857140 0x555934834e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.312+0000 7f37267fc640 1 --2- 172.21.15.16:0/4200850372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 0x55593488f850 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f371002a0a0 tx=0x7f371002adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831e2264d251b096 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.312+0000 7f37067fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:43:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.312+0000 7f37067fc640 1 -- 172.21.15.16:0/4200850372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371004a040 con 0x555934837a50 2024-09-19T07:43:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.312+0000 7f37067fc640 1 -- 172.21.15.16:0/4200850372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3710081f50 con 0x555934837a50 2024-09-19T07:43:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.312+0000 7f37307d3740 1 -- 172.21.15.16:0/4200850372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3718014a20 con 0x555934837a50 2024-09-19T07:43:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.312+0000 7f37067fc640 1 -- 172.21.15.16:0/4200850372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f371007dbb0 con 0x555934837a50 2024-09-19T07:43:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.312+0000 7f37307d3740 10 client.4677 fetch_fsmap learned FSMap version 8 2024-09-19T07:43:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.312+0000 7f37307d3740 1 -- 172.21.15.16:0/4200850372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5559345756b0 con 0x555934837a50 2024-09-19T07:43:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.312+0000 7f37067fc640 1 -- 172.21.15.16:0/4200850372 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3718014a20 con 0x555934837a50 2024-09-19T07:43:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.312+0000 7f37067fc640 1 -- 172.21.15.16:0/4200850372 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 8) ==== 3510+0+0 (secure 0 0 0) 0x7f371007ab30 con 0x555934837a50 2024-09-19T07:43:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.312+0000 7f37307d3740 10 client.4677 fetch_fsmap finished waiting for FSMap version 8 2024-09-19T07:43:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.312+0000 7f37307d3740 10 client.4677 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-19T07:43:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.312+0000 7f37307d3740 20 client.4677 populate_metadata read hostname 'smithi016' 2024-09-19T07:43:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.312+0000 7f37307d3740 1 --2- 172.21.15.16:0/4200850372 >> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] conn(0x5559348554c0 0x5559348558a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.312+0000 7f37307d3740 4 client.4677 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:43:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.312+0000 7f37307d3740 1 -- 172.21.15.16:0/4200850372 --> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559347a5c50 con 0x5559348554c0 2024-09-19T07:43:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.313+0000 7f3726ffd640 1 --2- 172.21.15.16:0/4200850372 >> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] conn(0x5559348554c0 0x5559348558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.313+0000 7f37067fc640 10 client.4677 ms_handle_connect on v2:172.21.15.146:6836/1303703249 2024-09-19T07:43:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.313+0000 7f3726ffd640 1 --2- 172.21.15.16:0/4200850372 >> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] conn(0x5559348554c0 0x5559348558a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd323dc36624badd server_cookie=17698eed48a732d2 in_seq=0 out_seq=0 2024-09-19T07:43:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.314+0000 7f37067fc640 1 -- 172.21.15.16:0/4200850372 <== mds.0 v2:172.21.15.146:6836/1303703249 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559347a5c50 con 0x5559348554c0 2024-09-19T07:43:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.314+0000 7f37067fc640 10 client.4677 handle_command_reply: tid=0 2024-09-19T07:43:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.314+0000 7f37307d3740 1 client.4677 shutdown 2024-09-19T07:43:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.314+0000 7f37307d3740 1 -- 172.21.15.16:0/4200850372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 msgr2=0x55593488f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.314+0000 7f37307d3740 1 --2- 172.21.15.16:0/4200850372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 0x55593488f850 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f371002a0a0 tx=0x7f371002adb0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.314+0000 7f37307d3740 1 -- 172.21.15.16:0/4200850372 shutdown_connections 2024-09-19T07:43:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.314+0000 7f37307d3740 1 --2- 172.21.15.16:0/4200850372 >> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] conn(0x5559348554c0 0x5559348558a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.314+0000 7f37307d3740 1 --2- 172.21.15.16:0/4200850372 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934857140 0x555934834e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.314+0000 7f37067fc640 0 client.4677 ms_handle_reset on v2:172.21.15.146:6836/1303703249 2024-09-19T07:43:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.314+0000 7f37307d3740 1 --2- 172.21.15.16:0/4200850372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 0x55593488f850 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.314+0000 7f37307d3740 1 --2- 172.21.15.16:0/4200850372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559348362c0 0x555934854ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.314+0000 7f37307d3740 1 -- 172.21.15.16:0/4200850372 >> 172.21.15.16:0/4200850372 conn(0x555934833a80 msgr2=0x555934878fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.314+0000 7f37307d3740 1 -- 172.21.15.16:0/4200850372 shutdown_connections 2024-09-19T07:43:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.314+0000 7f37307d3740 1 -- 172.21.15.16:0/4200850372 wait complete. 2024-09-19T07:43:33.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.314+0000 7f37307d3740 20 client.4677 trim_cache size 0 max 16384 2024-09-19T07:43:33.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.332+0000 7f37307d3740 1 Processor -- start 2024-09-19T07:43:33.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.332+0000 7f37307d3740 1 -- start start 2024-09-19T07:43:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.333+0000 7f37307d3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559348362c0 0x555934830b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.333+0000 7f37307d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 0x5559348310c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.333+0000 7f37307d3740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934857140 0x55593485ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.333+0000 7f37307d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37282010c0 con 0x555934837a50 2024-09-19T07:43:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.333+0000 7f37307d3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3728200450 con 0x5559348362c0 2024-09-19T07:43:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.333+0000 7f37307d3740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3728200aa0 con 0x555934857140 2024-09-19T07:43:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.333+0000 7f37267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 0x5559348310c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.333+0000 7f37267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 0x5559348310c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56026/0 (socket says 172.21.15.16:56026) 2024-09-19T07:43:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.333+0000 7f37267fc640 1 -- 172.21.15.16:0/1682215462 learned_addr learned my addr 172.21.15.16:0/1682215462 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.333+0000 7f37267fc640 1 -- 172.21.15.16:0/1682215462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934857140 msgr2=0x55593485ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.333+0000 7f37267fc640 1 --2- 172.21.15.16:0/1682215462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934857140 0x55593485ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.333+0000 7f37267fc640 1 -- 172.21.15.16:0/1682215462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559348362c0 msgr2=0x555934830b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.333+0000 7f37267fc640 1 --2- 172.21.15.16:0/1682215462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559348362c0 0x555934830b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.333+0000 7f37267fc640 1 -- 172.21.15.16:0/1682215462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55593482c6c0 con 0x555934837a50 2024-09-19T07:43:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.334+0000 7f37267fc640 1 --2- 172.21.15.16:0/1682215462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 0x5559348310c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3710076180 tx=0x7f371007b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3fd0e47e5776c91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.334+0000 7f371cff9640 1 -- 172.21.15.16:0/1682215462 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710079160 con 0x555934837a50 2024-09-19T07:43:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.334+0000 7f371cff9640 1 -- 172.21.15.16:0/1682215462 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37100823e0 con 0x555934837a50 2024-09-19T07:43:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.334+0000 7f371cff9640 1 -- 172.21.15.16:0/1682215462 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371007ab30 con 0x555934837a50 2024-09-19T07:43:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.334+0000 7f37307d3740 1 -- 172.21.15.16:0/1682215462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 msgr2=0x5559348310c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.334+0000 7f37307d3740 1 --2- 172.21.15.16:0/1682215462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 0x5559348310c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3710076180 tx=0x7f371007b000 comp rx=0 tx=0).stop 2024-09-19T07:43:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.334+0000 7f37307d3740 1 -- 172.21.15.16:0/1682215462 shutdown_connections 2024-09-19T07:43:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.334+0000 7f37307d3740 1 --2- 172.21.15.16:0/1682215462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934857140 0x55593485ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.334+0000 7f37307d3740 1 --2- 172.21.15.16:0/1682215462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 0x5559348310c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.334+0000 7f37307d3740 1 --2- 172.21.15.16:0/1682215462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559348362c0 0x555934830b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.334+0000 7f37307d3740 1 -- 172.21.15.16:0/1682215462 >> 172.21.15.16:0/1682215462 conn(0x555934833a80 msgr2=0x55593487ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:33.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.334+0000 7f37307d3740 1 -- 172.21.15.16:0/1682215462 shutdown_connections 2024-09-19T07:43:33.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.334+0000 7f37307d3740 1 -- 172.21.15.16:0/1682215462 wait complete. 2024-09-19T07:43:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.335+0000 7f37307d3740 1 -- start start 2024-09-19T07:43:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.335+0000 7f37307d3740 1 Processor -- start 2024-09-19T07:43:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37307d3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559348362c0 0x5559348e5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37307d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 0x55593488d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37307d3740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934857140 0x555934879e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37307d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55593485e1d0 con 0x555934837a50 2024-09-19T07:43:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37307d3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x555934831f20 con 0x5559348362c0 2024-09-19T07:43:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37307d3740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55593482fee0 con 0x555934857140 2024-09-19T07:43:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37307d3740 10 client.0 _ll_register_callbacks cb 0x7f3710032f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 0x55593488d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 0x55593488d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56042/0 (socket says 172.21.15.16:56042) 2024-09-19T07:43:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37267fc640 1 -- 172.21.15.16:0/3409134946 learned_addr learned my addr 172.21.15.16:0/3409134946 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37277fe640 1 --2- 172.21.15.16:0/3409134946 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934857140 0x555934879e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f3726ffd640 1 --2- 172.21.15.16:0/3409134946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559348362c0 0x5559348e5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37267fc640 1 -- 172.21.15.16:0/3409134946 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934857140 msgr2=0x555934879e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37267fc640 1 --2- 172.21.15.16:0/3409134946 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934857140 0x555934879e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37267fc640 1 -- 172.21.15.16:0/3409134946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559348362c0 msgr2=0x5559348e5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37267fc640 1 --2- 172.21.15.16:0/3409134946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559348362c0 0x5559348e5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37267fc640 1 -- 172.21.15.16:0/3409134946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559348e5a40 con 0x555934837a50 2024-09-19T07:43:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f3726ffd640 1 --2- 172.21.15.16:0/3409134946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559348362c0 0x5559348e5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37267fc640 1 --2- 172.21.15.16:0/3409134946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 0x55593488d840 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3710075c60 tx=0x7f371003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7071cb8a3c0e2f79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37067fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:43:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37067fc640 1 -- 172.21.15.16:0/3409134946 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371004a030 con 0x555934837a50 2024-09-19T07:43:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37067fc640 1 -- 172.21.15.16:0/3409134946 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3710082160 con 0x555934837a50 2024-09-19T07:43:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.336+0000 7f37307d3740 1 -- 172.21.15.16:0/3409134946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37100823e0 con 0x555934837a50 2024-09-19T07:43:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.337+0000 7f37067fc640 1 -- 172.21.15.16:0/3409134946 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f3710079160 con 0x555934837a50 2024-09-19T07:43:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.337+0000 7f37307d3740 10 client.4683 fetch_fsmap learned FSMap version 8 2024-09-19T07:43:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.337+0000 7f37307d3740 1 -- 172.21.15.16:0/3409134946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5559348e5a40 con 0x555934837a50 2024-09-19T07:43:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.337+0000 7f37067fc640 1 -- 172.21.15.16:0/3409134946 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37100823e0 con 0x555934837a50 2024-09-19T07:43:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.337+0000 7f37067fc640 1 -- 172.21.15.16:0/3409134946 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 8) ==== 3510+0+0 (secure 0 0 0) 0x7f3710078c50 con 0x555934837a50 2024-09-19T07:43:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.337+0000 7f37307d3740 10 client.4683 fetch_fsmap finished waiting for FSMap version 8 2024-09-19T07:43:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.337+0000 7f37307d3740 10 client.4683 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-19T07:43:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.337+0000 7f37307d3740 20 client.4683 populate_metadata read hostname 'smithi016' 2024-09-19T07:43:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.337+0000 7f37307d3740 1 --2- 172.21.15.16:0/3409134946 >> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] conn(0x5559349060e0 0x5559349064c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.337+0000 7f37307d3740 4 client.4683 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:43:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.337+0000 7f37307d3740 1 -- 172.21.15.16:0/3409134946 --> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] -- command(tid 0: {"prefix": "status"}) -- 0x55593479ffa0 con 0x5559349060e0 2024-09-19T07:43:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.337+0000 7f3726ffd640 1 --2- 172.21.15.16:0/3409134946 >> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] conn(0x5559349060e0 0x5559349064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.337+0000 7f3726ffd640 1 --2- 172.21.15.16:0/3409134946 >> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] conn(0x5559349060e0 0x5559349064c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b196d72f85d3258 server_cookie=4254d37614bbccbf in_seq=0 out_seq=0 2024-09-19T07:43:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.337+0000 7f37067fc640 10 client.4683 ms_handle_connect on v2:172.21.15.146:6836/1303703249 2024-09-19T07:43:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.338+0000 7f37067fc640 1 -- 172.21.15.16:0/3409134946 <== mds.0 v2:172.21.15.146:6836/1303703249 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55593479ffa0 con 0x5559349060e0 2024-09-19T07:43:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.338+0000 7f37067fc640 10 client.4683 handle_command_reply: tid=0 2024-09-19T07:43:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.338+0000 7f37307d3740 1 client.4683 shutdown 2024-09-19T07:43:33.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.338+0000 7f37307d3740 1 -- 172.21.15.16:0/3409134946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 msgr2=0x55593488d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:33.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.338+0000 7f37307d3740 1 --2- 172.21.15.16:0/3409134946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 0x55593488d840 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3710075c60 tx=0x7f371003e000 comp rx=0 tx=0).stop 2024-09-19T07:43:33.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.338+0000 7f37307d3740 1 -- 172.21.15.16:0/3409134946 shutdown_connections 2024-09-19T07:43:33.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.338+0000 7f37307d3740 1 --2- 172.21.15.16:0/3409134946 >> [v2:172.21.15.146:6836/1303703249,v1:172.21.15.146:6837/1303703249] conn(0x5559349060e0 0x5559349064c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.338+0000 7f37307d3740 1 --2- 172.21.15.16:0/3409134946 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555934857140 0x555934879e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.338+0000 7f37067fc640 0 client.4683 ms_handle_reset on v2:172.21.15.146:6836/1303703249 2024-09-19T07:43:33.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.338+0000 7f37307d3740 1 --2- 172.21.15.16:0/3409134946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555934837a50 0x55593488d840 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.338+0000 7f37307d3740 1 --2- 172.21.15.16:0/3409134946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559348362c0 0x5559348e5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.338+0000 7f37307d3740 1 -- 172.21.15.16:0/3409134946 >> 172.21.15.16:0/3409134946 conn(0x555934833a80 msgr2=0x555934834690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:33.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.338+0000 7f37307d3740 1 -- 172.21.15.16:0/3409134946 shutdown_connections 2024-09-19T07:43:33.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.338+0000 7f37307d3740 1 -- 172.21.15.16:0/3409134946 wait complete. 2024-09-19T07:43:33.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.338+0000 7f37307d3740 20 client.4683 trim_cache size 0 max 16384 2024-09-19T07:43:33.345 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:43:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb", 2024-09-19T07:43:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-19T07:43:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4278, 2024-09-19T07:43:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T07:43:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T07:43:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T07:43:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 27.093470043, 2024-09-19T07:43:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 8, 2024-09-19T07:43:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 26, 2024-09-19T07:43:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 26, 2024-09-19T07:43:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 33.102285715999997 2024-09-19T07:43:33.346 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:43:33.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.339+0000 7f372da42640 1 -- 172.21.15.16:0/604350551 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f370806c2b0 msgr2=0x7f370806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:33.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.339+0000 7f372da42640 1 --2- 172.21.15.16:0/604350551 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f370806c2b0 0x7f370806e770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3728154ca0 tx=0x7f371800b040 comp rx=0 tx=0).stop 2024-09-19T07:43:33.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.339+0000 7f372da42640 1 -- 172.21.15.16:0/604350551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280c8d20 msgr2=0x7f3728153fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:33.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.339+0000 7f372da42640 1 --2- 172.21.15.16:0/604350551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280c8d20 0x7f3728153fd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3710035040 tx=0x7f37100046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.339+0000 7f372da42640 1 -- 172.21.15.16:0/604350551 shutdown_connections 2024-09-19T07:43:33.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.339+0000 7f372da42640 1 --2- 172.21.15.16:0/604350551 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f370806c2b0 0x7f370806e770 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.339+0000 7f372da42640 1 --2- 172.21.15.16:0/604350551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3728143f30 0x7f37280d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.339+0000 7f372da42640 1 --2- 172.21.15.16:0/604350551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280c8d20 0x7f3728153fd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.339+0000 7f372da42640 1 --2- 172.21.15.16:0/604350551 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f372806d260 0x7f37280cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.339+0000 7f372da42640 1 -- 172.21.15.16:0/604350551 >> 172.21.15.16:0/604350551 conn(0x7f37280b3ed0 msgr2=0x7f3728140ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:33.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.339+0000 7f372da42640 1 -- 172.21.15.16:0/604350551 shutdown_connections 2024-09-19T07:43:33.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.339+0000 7f372da42640 1 -- 172.21.15.16:0/604350551 wait complete. 2024-09-19T07:43:33.350 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T07:43:33.359 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T07:43:33.360 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T07:43:33.360 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T07:43:33.360 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-19T07:43:33.408 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-19T07:43:33.408 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-19T07:43:33.408 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-19T07:43:33.408 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T07:43:33.408 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:43:33.408 INFO:tasks.ceph_fuse:client.2 config is {} 2024-09-19T07:43:33.408 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T07:43:33.409 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-19T07:43:33.409 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-19T07:43:33.409 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T07:43:33.409 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T07:43:33.409 INFO:tasks.ceph_fuse:client.3 config is {} 2024-09-19T07:43:33.409 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T07:43:33.409 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-19T07:43:33.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:33.409 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:43:33.425 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-19T07:43:33.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:33.427 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T07:43:33.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:33.492 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-19T07:43:33.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:33.601 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-09-19T07:43:33.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:33.621 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link delete ceph-brx 2024-09-19T07:43:33.693 INFO:teuthology.orchestra.run.smithi146.stderr:Cannot find device "ceph-brx" 2024-09-19T07:43:33.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:43:33.695 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-19T07:43:33.695 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-19T07:43:33.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:43:33.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.798+0000 7ff0e99be640 1 Processor -- start 2024-09-19T07:43:33.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.798+0000 7ff0e99be640 1 -- start start 2024-09-19T07:43:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.798+0000 7ff0e99be640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0e415bee0 0x7ff0e40d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.798+0000 7ff0e99be640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff0e40cf910 0x7ff0e40d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.798+0000 7ff0e99be640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0e40cdee0 con 0x7ff0e415c2c0 2024-09-19T07:43:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.798+0000 7ff0e99be640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff0e40ceaf0 con 0x7ff0e40cf910 2024-09-19T07:43:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.798+0000 7ff0e99be640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e40ce1c0 con 0x7ff0e415bee0 2024-09-19T07:43:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.799+0000 7ff0e27fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff0e415c2c0 0x7ff0e40cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38334/0 (socket says 172.21.15.16:38334) 2024-09-19T07:43:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.799+0000 7ff0e27fc640 1 -- 172.21.15.16:0/2531875758 learned_addr learned my addr 172.21.15.16:0/2531875758 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.799+0000 7ff0e37fe640 1 --2- 172.21.15.16:0/2531875758 >> v2:172.21.15.146:3300/0 conn(0x7ff0e40cf910 0x7ff0e40d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.799+0000 7ff0e2ffd640 1 --2- 172.21.15.16:0/2531875758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0e415bee0 0x7ff0e40d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.799+0000 7ff0e1ffb640 1 -- 172.21.15.16:0/2531875758 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1972199028 0 0) 0x7ff0e40cdee0 con 0x7ff0e415c2c0 2024-09-19T07:43:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.799+0000 7ff0e1ffb640 1 -- 172.21.15.16:0/2531875758 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0cc003520 con 0x7ff0e415c2c0 2024-09-19T07:43:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.799+0000 7ff0e2ffd640 1 -- 172.21.15.16:0/2531875758 >> v2:172.21.15.146:3300/0 conn(0x7ff0e40cf910 msgr2=0x7ff0e40d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:33.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.799+0000 7ff0e2ffd640 1 --2- 172.21.15.16:0/2531875758 >> v2:172.21.15.146:3300/0 conn(0x7ff0e40cf910 0x7ff0e40d09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.799+0000 7ff0e2ffd640 1 -- 172.21.15.16:0/2531875758 >> v1:172.21.15.16:6789/0 conn(0x7ff0e415c2c0 legacy=0x7ff0e40cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.799+0000 7ff0e2ffd640 1 -- 172.21.15.16:0/2531875758 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e40ce4a0 con 0x7ff0e415bee0 2024-09-19T07:43:33.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.799+0000 7ff0e37fe640 1 --2- 172.21.15.16:0/2531875758 >> v2:172.21.15.146:3300/0 conn(0x7ff0e40cf910 0x7ff0e40d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.800+0000 7ff0e2ffd640 1 --2- 172.21.15.16:0/2531875758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0e415bee0 0x7ff0e40d04b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40097a0 tx=0x7ff0d402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=effc05bad5cc215b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.800+0000 7ff0e1ffb640 1 -- 172.21.15.16:0/2531875758 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4037070 con 0x7ff0e415bee0 2024-09-19T07:43:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.800+0000 7ff0e1ffb640 1 -- 172.21.15.16:0/2531875758 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0d4032440 con 0x7ff0e415bee0 2024-09-19T07:43:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.800+0000 7ff0e1ffb640 1 -- 172.21.15.16:0/2531875758 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40327f0 con 0x7ff0e415bee0 2024-09-19T07:43:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.801+0000 7ff0e99be640 1 -- 172.21.15.16:0/2531875758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0e415bee0 msgr2=0x7ff0e40d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.801+0000 7ff0e99be640 1 --2- 172.21.15.16:0/2531875758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0e415bee0 0x7ff0e40d04b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40097a0 tx=0x7ff0d402a180 comp rx=0 tx=0).stop 2024-09-19T07:43:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.801+0000 7ff0e99be640 1 -- 172.21.15.16:0/2531875758 shutdown_connections 2024-09-19T07:43:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.801+0000 7ff0e99be640 1 --2- 172.21.15.16:0/2531875758 >> v2:172.21.15.146:3300/0 conn(0x7ff0e40cf910 0x7ff0e40d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.801+0000 7ff0e99be640 1 --2- 172.21.15.16:0/2531875758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0e415bee0 0x7ff0e40d04b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.801+0000 7ff0e99be640 1 -- 172.21.15.16:0/2531875758 >> 172.21.15.16:0/2531875758 conn(0x7ff0e406bdf0 msgr2=0x7ff0e406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:33.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.801+0000 7ff0e99be640 1 -- 172.21.15.16:0/2531875758 shutdown_connections 2024-09-19T07:43:33.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.801+0000 7ff0e99be640 1 -- 172.21.15.16:0/2531875758 wait complete. 2024-09-19T07:43:33.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.802+0000 7ff0e99be640 1 Processor -- start 2024-09-19T07:43:33.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.802+0000 7ff0e99be640 1 -- start start 2024-09-19T07:43:33.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.803+0000 7ff0e99be640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0e40cf910 0x7ff0e40adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.803+0000 7ff0e99be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e415bee0 0x7ff0e41f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.803+0000 7ff0e99be640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0e415c2c0 0x7ff0e40ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.803+0000 7ff0e99be640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e406e040 con 0x7ff0e415bee0 2024-09-19T07:43:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.803+0000 7ff0e99be640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e406dec0 con 0x7ff0e40cf910 2024-09-19T07:43:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.803+0000 7ff0e99be640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e406e1c0 con 0x7ff0e415c2c0 2024-09-19T07:43:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.803+0000 7ff0e27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e415bee0 0x7ff0e41f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.803+0000 7ff0e27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e415bee0 0x7ff0e41f4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56054/0 (socket says 172.21.15.16:56054) 2024-09-19T07:43:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.803+0000 7ff0e27fc640 1 -- 172.21.15.16:0/3895793285 learned_addr learned my addr 172.21.15.16:0/3895793285 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.803+0000 7ff0e2ffd640 1 --2- 172.21.15.16:0/3895793285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0e40cf910 0x7ff0e40adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.803+0000 7ff0e37fe640 1 --2- 172.21.15.16:0/3895793285 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0e415c2c0 0x7ff0e40ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.803+0000 7ff0e27fc640 1 -- 172.21.15.16:0/3895793285 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0e415c2c0 msgr2=0x7ff0e40ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.803+0000 7ff0e27fc640 1 --2- 172.21.15.16:0/3895793285 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0e415c2c0 0x7ff0e40ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.803+0000 7ff0e27fc640 1 -- 172.21.15.16:0/3895793285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0e40cf910 msgr2=0x7ff0e40adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.803+0000 7ff0e27fc640 1 --2- 172.21.15.16:0/3895793285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0e40cf910 0x7ff0e40adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:33.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.803+0000 7ff0e27fc640 1 -- 172.21.15.16:0/3895793285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e40ad200 con 0x7ff0e415bee0 2024-09-19T07:43:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.803+0000 7ff0e2ffd640 1 --2- 172.21.15.16:0/3895793285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0e40cf910 0x7ff0e40adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.803+0000 7ff0e37fe640 1 --2- 172.21.15.16:0/3895793285 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0e415c2c0 0x7ff0e40ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.803+0000 7ff0e27fc640 1 --2- 172.21.15.16:0/3895793285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e415bee0 0x7ff0e41f4720 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800c430 tx=0x7ff0d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.804+0000 7ff0cbfff640 1 -- 172.21.15.16:0/3895793285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8017070 con 0x7ff0e415bee0 2024-09-19T07:43:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.804+0000 7ff0cbfff640 1 -- 172.21.15.16:0/3895793285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0d8005d00 con 0x7ff0e415bee0 2024-09-19T07:43:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.804+0000 7ff0e99be640 1 -- 172.21.15.16:0/3895793285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e40ad4e0 con 0x7ff0e415bee0 2024-09-19T07:43:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.804+0000 7ff0cbfff640 1 -- 172.21.15.16:0/3895793285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8007310 con 0x7ff0e415bee0 2024-09-19T07:43:33.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.804+0000 7ff0e99be640 1 -- 172.21.15.16:0/3895793285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e40af1d0 con 0x7ff0e415bee0 2024-09-19T07:43:33.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.804+0000 7ff0e99be640 1 -- 172.21.15.16:0/3895793285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0e406c1d0 con 0x7ff0e415bee0 2024-09-19T07:43:33.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.805+0000 7ff0cbfff640 1 -- 172.21.15.16:0/3895793285 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff0d801a070 con 0x7ff0e415bee0 2024-09-19T07:43:33.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.805+0000 7ff0cbfff640 1 --2- 172.21.15.16:0/3895793285 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff0d006c2b0 0x7ff0d006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:33.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.806+0000 7ff0cbfff640 1 -- 172.21.15.16:0/3895793285 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff0d808aae0 con 0x7ff0e415bee0 2024-09-19T07:43:33.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.806+0000 7ff0e2ffd640 1 --2- 172.21.15.16:0/3895793285 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff0d006c2b0 0x7ff0d006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:33.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.806+0000 7ff0e2ffd640 1 --2- 172.21.15.16:0/3895793285 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff0d006c2b0 0x7ff0d006e770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4034000 tx=0x7ff0d402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:33.810+0000 7ff0cbfff640 1 -- 172.21.15.16:0/3895793285 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff0e406c1d0 con 0x7ff0e415bee0 2024-09-19T07:43:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34.021+0000 7ff0e99be640 1 -- 172.21.15.16:0/3895793285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff0e415b0e0 con 0x7ff0e415bee0 2024-09-19T07:43:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34.021+0000 7ff0cbfff640 1 -- 172.21.15.16:0/3895793285 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7ff0d8058fa0 con 0x7ff0e415bee0 2024-09-19T07:43:34.022 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-19T07:43:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34.023+0000 7ff0e99be640 1 -- 172.21.15.16:0/3895793285 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff0d006c2b0 msgr2=0x7ff0d006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34.023+0000 7ff0e99be640 1 --2- 172.21.15.16:0/3895793285 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff0d006c2b0 0x7ff0d006e770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4034000 tx=0x7ff0d402e000 comp rx=0 tx=0).stop 2024-09-19T07:43:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34.023+0000 7ff0e99be640 1 -- 172.21.15.16:0/3895793285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e415bee0 msgr2=0x7ff0e41f4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34.023+0000 7ff0e99be640 1 --2- 172.21.15.16:0/3895793285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e415bee0 0x7ff0e41f4720 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800c430 tx=0x7ff0d800c8f0 comp rx=0 tx=0).stop 2024-09-19T07:43:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34.023+0000 7ff0e99be640 1 -- 172.21.15.16:0/3895793285 shutdown_connections 2024-09-19T07:43:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34.023+0000 7ff0e99be640 1 --2- 172.21.15.16:0/3895793285 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff0d006c2b0 0x7ff0d006e770 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:34.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34.023+0000 7ff0e99be640 1 --2- 172.21.15.16:0/3895793285 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0e415c2c0 0x7ff0e40ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:34.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34.023+0000 7ff0e99be640 1 --2- 172.21.15.16:0/3895793285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e415bee0 0x7ff0e41f4720 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:34.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34.023+0000 7ff0e99be640 1 --2- 172.21.15.16:0/3895793285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0e40cf910 0x7ff0e40adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:34.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34.023+0000 7ff0e99be640 1 -- 172.21.15.16:0/3895793285 >> 172.21.15.16:0/3895793285 conn(0x7ff0e406bdf0 msgr2=0x7ff0e40b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:34.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34.024+0000 7ff0e99be640 1 -- 172.21.15.16:0/3895793285 shutdown_connections 2024-09-19T07:43:34.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34.024+0000 7ff0e99be640 1 -- 172.21.15.16:0/3895793285 wait complete. 2024-09-19T07:43:34.035 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:43:34.035 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:43:34.035 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T07:43:34.035 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T07:43:34.035 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:43:34.035 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:43:34.035 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:43:34.035 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:43:34.036 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:43:34.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:34.036 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T07:43:34.052 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:43:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:43:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:43:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:43:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:34.052 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:43:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T07:43:34.053 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:43:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T07:43:34.053 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:43:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T07:43:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T07:43:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42775sec preferred_lft 42775sec 2024-09-19T07:43:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T07:43:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:34.053 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:43:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T07:43:34.054 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T07:43:34.054 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:34.054 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:43:34.054 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-09-19T07:43:34.054 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-09-19T07:43:34.054 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-09-19T07:43:34.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T07:43:34.055 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:43:34.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:34.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:34.193 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:43:34.218 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:43:34.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:34.219 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T07:43:34.276 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T07:43:34.277 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:43:34.277 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T07:43:34.277 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T07:43:34.277 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:43:34.277 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:34.277 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:43:34.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T07:43:34.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T07:43:34.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T07:43:34.278 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:43:34.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:34.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:34.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:34.407 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:43:34.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:34.423 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-19T07:43:34.479 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:34.479 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:43:34.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:43:34.480 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T07:43:34.480 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:43:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:34.599 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T07:43:34.600 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:34.600 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:43:34.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T07:43:34.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T07:43:34.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T07:43:34.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T07:43:34.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T07:43:34.600 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:43:34.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:34.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:34.717 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:34.717 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:43:34.717 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-19T07:43:34.717 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T07:43:34.717 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:43:34.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:34.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:34.796 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T07:43:34.797 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:43:34.797 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T07:43:34.812 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:43:34.812 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:43:34.812 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:43:34.867 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:43:34.934 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:43:34.992 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:43:34.992 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:43:34.992 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:43:34.992 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:43:34.992 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:43:34.992 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:43:34.992 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:43:34.992 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:43:34.992 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:43:34.993 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:43:34.993 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:43:34.993 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:43:34.993 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:43:34.993 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:43:34.993 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:43:34.993 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:43:34.993 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:43:34.993 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:43:34.993 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:43:34.993 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:43:34.993 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:43:34.993 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:43:34.994 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:43:34.994 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:43:34.994 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:43:34.994 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:43:34.994 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:43:34.994 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:43:34.994 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:43:34.994 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:43:34.994 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:43:34.994 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:43:34.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:34.995 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:43:35.048 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:43:35.048 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T07:43:35.091 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:43:35.115 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:43:35.173 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:43:35.173 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:43:35.173 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:43:35.173 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:43:35.173 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:43:35.173 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:43:35.173 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:43:35.173 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:43:35.173 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:43:35.173 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:43:35.173 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:43:35.173 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:43:35.174 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:43:35.174 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:43:35.174 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:43:35.174 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:43:35.174 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:43:35.174 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:43:35.174 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:43:35.174 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:43:35.174 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:43:35.174 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:43:35.174 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:43:35.174 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:43:35.174 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:43:35.174 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:43:35.175 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:43:35.175 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:43:35.175 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:43:35.175 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:43:35.175 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:43:35.175 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:43:35.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:35.175 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:43:35.176 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-19T07:43:35.175+0000 7fe3752ff480 -1 init, newargv = 0x56334df3a620 newargc=15 2024-09-19T07:43:35.176 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[44743]: starting ceph client 2024-09-19T07:43:35.180 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[44743]: starting fuse 2024-09-19T07:43:35.192 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:43:35.192 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-19T07:43:35.192 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:43:35.192 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:43:35.192 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:43:35.192 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:43:35.192 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:43:35.192 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:43:35.192 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:43:35.192 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:43:35.193 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:43:35.193 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:43:35.193 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:43:35.193 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:43:35.193 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:43:35.193 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:43:35.193 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:43:35.193 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:43:35.193 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:43:35.193 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:43:35.193 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:43:35.193 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:43:35.193 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:43:35.193 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:43:35.193 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:43:35.194 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:43:35.194 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:43:35.194 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:43:35.194 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:43:35.312 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.44743.asok 2024-09-19T07:43:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:35.323 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44743.asok 2024-09-19T07:43:35.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:35.324 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44743.asok status 2024-09-19T07:43:35.460 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:43:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:43:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:43:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:43:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:43:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:43:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:43:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "44743", 2024-09-19T07:43:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:43:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-19T07:43:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-19T07:43:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4698, 2024-09-19T07:43:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:43:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:43:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:43:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4698 2024-09-19T07:43:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:43:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:43:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:43:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1188301573 2024-09-19T07:43:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:43:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:43:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:43:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1188301573 2024-09-19T07:43:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.4698 192.168.0.1:0/1188301573", 2024-09-19T07:43:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/1188301573", 2024-09-19T07:43:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-19T07:43:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 8, 2024-09-19T07:43:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 26, 2024-09-19T07:43:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:43:35.463 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:43:35.463 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:43:35.463 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:43:35.468 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-19T07:43:35.468 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:43:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.570+0000 7ffa69dcd640 1 Processor -- start 2024-09-19T07:43:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.570+0000 7ffa69dcd640 1 -- start start 2024-09-19T07:43:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.571+0000 7ffa69dcd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa6415b120 0x7ffa6406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.571+0000 7ffa69dcd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffa640d0bc0 0x7ffa6406e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.571+0000 7ffa69dcd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa6415b500 con 0x7ffa6406dec0 2024-09-19T07:43:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.571+0000 7ffa69dcd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ffa6406e940 con 0x7ffa640d0bc0 2024-09-19T07:43:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.571+0000 7ffa69dcd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ffa6406ec20 con 0x7ffa6415b120 2024-09-19T07:43:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.571+0000 7ffa62ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffa6406dec0 0x7ffa640d04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38336/0 (socket says 172.21.15.16:38336) 2024-09-19T07:43:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.571+0000 7ffa62ffd640 1 -- 172.21.15.16:0/1087071433 learned_addr learned my addr 172.21.15.16:0/1087071433 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.571+0000 7ffa627fc640 1 -- 172.21.15.16:0/1087071433 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1941241369 0 0) 0x7ffa6415b500 con 0x7ffa6406dec0 2024-09-19T07:43:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.571+0000 7ffa63fff640 1 --2- 172.21.15.16:0/1087071433 >> v2:172.21.15.146:3300/0 conn(0x7ffa640d0bc0 0x7ffa6406e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.571+0000 7ffa637fe640 1 --2- 172.21.15.16:0/1087071433 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa6415b120 0x7ffa6406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.571+0000 7ffa627fc640 1 -- 172.21.15.16:0/1087071433 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa50003520 con 0x7ffa6406dec0 2024-09-19T07:43:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.571+0000 7ffa627fc640 1 -- 172.21.15.16:0/1087071433 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2075369386 0 0) 0x7ffa50003520 con 0x7ffa6406dec0 2024-09-19T07:43:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.571+0000 7ffa627fc640 1 -- 172.21.15.16:0/1087071433 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa6415b120 msgr2=0x7ffa6406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.571+0000 7ffa627fc640 1 --2- 172.21.15.16:0/1087071433 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa6415b120 0x7ffa6406f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.571+0000 7ffa627fc640 1 -- 172.21.15.16:0/1087071433 >> v2:172.21.15.146:3300/0 conn(0x7ffa640d0bc0 msgr2=0x7ffa6406e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.571+0000 7ffa627fc640 1 --2- 172.21.15.16:0/1087071433 >> v2:172.21.15.146:3300/0 conn(0x7ffa640d0bc0 0x7ffa6406e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.571+0000 7ffa627fc640 1 -- 172.21.15.16:0/1087071433 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6406ef00 con 0x7ffa6406dec0 2024-09-19T07:43:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.571+0000 7ffa627fc640 1 -- 172.21.15.16:0/1087071433 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7ffa4c001670 con 0x7ffa6406dec0 2024-09-19T07:43:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.571+0000 7ffa627fc640 1 -- 172.21.15.16:0/1087071433 >> v1:172.21.15.16:6789/0 conn(0x7ffa6406dec0 legacy=0x7ffa640d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.572+0000 7ffa627fc640 1 --2- 172.21.15.16:0/1087071433 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa6415b120 0x7ffa500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.572+0000 7ffa627fc640 1 --2- 172.21.15.16:0/1087071433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa50003f30 0x7ffa50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.572+0000 7ffa627fc640 1 --2- 172.21.15.16:0/1087071433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa50004d40 0x7ffa500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.572+0000 7ffa627fc640 1 -- 172.21.15.16:0/1087071433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6406e940 con 0x7ffa50003f30 2024-09-19T07:43:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.572+0000 7ffa627fc640 1 -- 172.21.15.16:0/1087071433 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6406ec20 con 0x7ffa50004d40 2024-09-19T07:43:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.572+0000 7ffa627fc640 1 -- 172.21.15.16:0/1087071433 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ffa5000b9c0 con 0x7ffa6415b120 2024-09-19T07:43:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.572+0000 7ffa62ffd640 1 --2- 172.21.15.16:0/1087071433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa50003f30 0x7ffa50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.572+0000 7ffa637fe640 1 --2- 172.21.15.16:0/1087071433 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa6415b120 0x7ffa500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.572+0000 7ffa63fff640 1 --2- 172.21.15.16:0/1087071433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa50004d40 0x7ffa500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.572+0000 7ffa62ffd640 1 -- 172.21.15.16:0/1087071433 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa6415b120 msgr2=0x7ffa500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.572+0000 7ffa62ffd640 1 --2- 172.21.15.16:0/1087071433 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa6415b120 0x7ffa500039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.572+0000 7ffa62ffd640 1 -- 172.21.15.16:0/1087071433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa50004d40 msgr2=0x7ffa500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.572+0000 7ffa62ffd640 1 --2- 172.21.15.16:0/1087071433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa50004d40 0x7ffa500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.572+0000 7ffa62ffd640 1 -- 172.21.15.16:0/1087071433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa50003520 con 0x7ffa50003f30 2024-09-19T07:43:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.573+0000 7ffa62ffd640 1 --2- 172.21.15.16:0/1087071433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa50003f30 0x7ffa50004800 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c001050 tx=0x7ffa50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20dd354cb26fd86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.573+0000 7ffa627fc640 1 -- 172.21.15.16:0/1087071433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c007d70 con 0x7ffa50003f30 2024-09-19T07:43:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.573+0000 7ffa627fc640 1 -- 172.21.15.16:0/1087071433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa4c0058d0 con 0x7ffa50003f30 2024-09-19T07:43:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.573+0000 7ffa627fc640 1 -- 172.21.15.16:0/1087071433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c005ce0 con 0x7ffa50003f30 2024-09-19T07:43:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.573+0000 7ffa69dcd640 1 -- 172.21.15.16:0/1087071433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa50003f30 msgr2=0x7ffa50004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.573+0000 7ffa69dcd640 1 --2- 172.21.15.16:0/1087071433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa50003f30 0x7ffa50004800 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c001050 tx=0x7ffa50005830 comp rx=0 tx=0).stop 2024-09-19T07:43:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.573+0000 7ffa63fff640 1 -- 172.21.15.16:0/1087071433 reap_dead start 2024-09-19T07:43:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.573+0000 7ffa69dcd640 1 -- 172.21.15.16:0/1087071433 shutdown_connections 2024-09-19T07:43:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.573+0000 7ffa69dcd640 1 -- 172.21.15.16:0/1087071433 >> 172.21.15.16:0/1087071433 conn(0x7ffa6406bdf0 msgr2=0x7ffa6406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.573+0000 7ffa69dcd640 1 -- 172.21.15.16:0/1087071433 shutdown_connections 2024-09-19T07:43:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.573+0000 7ffa69dcd640 1 -- 172.21.15.16:0/1087071433 wait complete. 2024-09-19T07:43:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.575+0000 7ffa69dcd640 1 Processor -- start 2024-09-19T07:43:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.575+0000 7ffa69dcd640 1 -- start start 2024-09-19T07:43:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.575+0000 7ffa69dcd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa50003f30 0x7ffa640afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.575+0000 7ffa69dcd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa6414f7c0 0x7ffa641f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.576+0000 7ffa69dcd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa64072db0 0x7ffa64073230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.576+0000 7ffa69dcd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa640ce060 con 0x7ffa64072db0 2024-09-19T07:43:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.576+0000 7ffa69dcd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6406f4c0 con 0x7ffa6414f7c0 2024-09-19T07:43:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.576+0000 7ffa69dcd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ffa6406f9e0 con 0x7ffa50003f30 2024-09-19T07:43:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.576+0000 7ffa63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa64072db0 0x7ffa64073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.576+0000 7ffa63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa64072db0 0x7ffa64073230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56072/0 (socket says 172.21.15.16:56072) 2024-09-19T07:43:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.576+0000 7ffa63fff640 1 -- 172.21.15.16:0/148869867 learned_addr learned my addr 172.21.15.16:0/148869867 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.576+0000 7ffa62ffd640 1 --2- 172.21.15.16:0/148869867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa6414f7c0 0x7ffa641f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.576+0000 7ffa637fe640 1 --2- 172.21.15.16:0/148869867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa50003f30 0x7ffa640afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.576+0000 7ffa63fff640 1 -- 172.21.15.16:0/148869867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa50003f30 msgr2=0x7ffa640afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.576+0000 7ffa63fff640 1 --2- 172.21.15.16:0/148869867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa50003f30 0x7ffa640afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.576+0000 7ffa63fff640 1 -- 172.21.15.16:0/148869867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa6414f7c0 msgr2=0x7ffa641f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.576+0000 7ffa63fff640 1 --2- 172.21.15.16:0/148869867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa6414f7c0 0x7ffa641f4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.576+0000 7ffa63fff640 1 -- 172.21.15.16:0/148869867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa641f53b0 con 0x7ffa64072db0 2024-09-19T07:43:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.576+0000 7ffa63fff640 1 --2- 172.21.15.16:0/148869867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa64072db0 0x7ffa64073230 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffa6415eb80 tx=0x7ffa5c00f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.576+0000 7ffa5bfff640 1 -- 172.21.15.16:0/148869867 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c019070 con 0x7ffa64072db0 2024-09-19T07:43:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.576+0000 7ffa5bfff640 1 -- 172.21.15.16:0/148869867 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa5c00f930 con 0x7ffa64072db0 2024-09-19T07:43:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.576+0000 7ffa5bfff640 1 -- 172.21.15.16:0/148869867 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c003e60 con 0x7ffa64072db0 2024-09-19T07:43:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.576+0000 7ffa69dcd640 1 -- 172.21.15.16:0/148869867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa641f57d0 con 0x7ffa64072db0 2024-09-19T07:43:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.577+0000 7ffa69dcd640 1 -- 172.21.15.16:0/148869867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa641f5d90 con 0x7ffa64072db0 2024-09-19T07:43:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.578+0000 7ffa69dcd640 1 -- 172.21.15.16:0/148869867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa6406e300 con 0x7ffa64072db0 2024-09-19T07:43:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.578+0000 7ffa5bfff640 1 -- 172.21.15.16:0/148869867 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ffa5c01c070 con 0x7ffa64072db0 2024-09-19T07:43:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.578+0000 7ffa5bfff640 1 --2- 172.21.15.16:0/148869867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffa4406c2e0 0x7ffa4406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.579+0000 7ffa5bfff640 1 -- 172.21.15.16:0/148869867 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ffa5c08bf70 con 0x7ffa64072db0 2024-09-19T07:43:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.579+0000 7ffa637fe640 1 --2- 172.21.15.16:0/148869867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffa4406c2e0 0x7ffa4406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.579+0000 7ffa637fe640 1 --2- 172.21.15.16:0/148869867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffa4406c2e0 0x7ffa4406e7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009ae0 tx=0x7ffa54009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.583+0000 7ffa5bfff640 1 -- 172.21.15.16:0/148869867 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ffa5c010040 con 0x7ffa64072db0 2024-09-19T07:43:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.795+0000 7ffa69dcd640 1 -- 172.21.15.16:0/148869867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffa640cb000 con 0x7ffa64072db0 2024-09-19T07:43:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.796+0000 7ffa5bfff640 1 -- 172.21.15.16:0/148869867 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7ffa5c059de0 con 0x7ffa64072db0 2024-09-19T07:43:35.796 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-19T07:43:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.798+0000 7ffa69dcd640 1 -- 172.21.15.16:0/148869867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffa4406c2e0 msgr2=0x7ffa4406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.798+0000 7ffa69dcd640 1 --2- 172.21.15.16:0/148869867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffa4406c2e0 0x7ffa4406e7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009ae0 tx=0x7ffa54009190 comp rx=0 tx=0).stop 2024-09-19T07:43:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.798+0000 7ffa69dcd640 1 -- 172.21.15.16:0/148869867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa64072db0 msgr2=0x7ffa64073230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.798+0000 7ffa69dcd640 1 --2- 172.21.15.16:0/148869867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa64072db0 0x7ffa64073230 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffa6415eb80 tx=0x7ffa5c00f1b0 comp rx=0 tx=0).stop 2024-09-19T07:43:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.798+0000 7ffa69dcd640 1 -- 172.21.15.16:0/148869867 shutdown_connections 2024-09-19T07:43:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.798+0000 7ffa69dcd640 1 --2- 172.21.15.16:0/148869867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffa4406c2e0 0x7ffa4406e7a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.798+0000 7ffa69dcd640 1 --2- 172.21.15.16:0/148869867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa64072db0 0x7ffa64073230 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.798+0000 7ffa69dcd640 1 --2- 172.21.15.16:0/148869867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa6414f7c0 0x7ffa641f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.798+0000 7ffa69dcd640 1 --2- 172.21.15.16:0/148869867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa50003f30 0x7ffa640afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.798+0000 7ffa69dcd640 1 -- 172.21.15.16:0/148869867 >> 172.21.15.16:0/148869867 conn(0x7ffa6406bdf0 msgr2=0x7ffa640ad8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.798+0000 7ffa69dcd640 1 -- 172.21.15.16:0/148869867 shutdown_connections 2024-09-19T07:43:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:35.798+0000 7ffa69dcd640 1 -- 172.21.15.16:0/148869867 wait complete. 2024-09-19T07:43:35.809 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:43:35.809 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:43:35.809 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T07:43:35.809 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-19T07:43:35.810 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-19T07:43:35.810 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:43:35.810 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:43:35.810 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:43:35.810 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-19T07:43:35.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:35.810 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T07:43:35.826 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:43:35.827 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:43:35.827 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:43:35.827 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:35.827 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:43:35.827 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:35.827 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:43:35.827 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:35.827 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T07:43:35.827 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:43:35.827 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:35.827 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T07:43:35.827 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:43:35.827 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:35.827 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T07:43:35.828 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T07:43:35.828 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42773sec preferred_lft 42773sec 2024-09-19T07:43:35.828 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T07:43:35.828 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:35.828 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:43:35.828 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:35.828 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T07:43:35.828 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:43:35.828 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether e6:fd:21:6a:43:69 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:35.828 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:43:35.828 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:35.828 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::32:7bff:fe92:8895/64 scope link 2024-09-19T07:43:35.828 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:35.828 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:43:35.829 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether e6:fd:21:6a:43:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:43:35.829 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e4fd:21ff:fe6a:4369/64 scope link tentative 2024-09-19T07:43:35.829 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:35.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:35.829 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:43:35.891 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:43:35.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:35.893 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T07:43:35.950 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T07:43:35.951 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:43:35.951 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T07:43:35.951 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T07:43:35.951 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:43:35.951 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:35.951 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:43:35.951 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T07:43:35.952 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T07:43:35.952 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T07:43:35.952 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:43:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:36.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:36.080 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:43:36.096 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:43:36.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:36.096 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-19T07:43:36.152 INFO:teuthology.orchestra.run.smithi016.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-19T07:43:36.152 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:36.152 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:43:36.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:43:36.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-19T07:43:36.153 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:43:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:36.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:36.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:36.270 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-19T07:43:36.295 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:43:36.296 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:43:36.296 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:43:36.296 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:36.296 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:43:36.296 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:36.296 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:43:36.296 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 22:22:37:96:97:a9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-19T07:43:36.296 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-19T07:43:36.296 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:36.296 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::2022:37ff:fe96:97a9/64 scope link tentative 2024-09-19T07:43:36.296 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:36.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:36.298 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-19T07:43:36.363 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:43:36.363 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:43:36.363 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:43:36.363 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:36.363 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:43:36.363 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:36.363 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:43:36.363 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 22:22:37:96:97:a9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-19T07:43:36.363 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-19T07:43:36.364 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:36.364 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::2022:37ff:fe96:97a9/64 scope link tentative 2024-09-19T07:43:36.364 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:36.365 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-19T07:43:36.365 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:36.365 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:43:36.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-19T07:43:36.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-19T07:43:36.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-19T07:43:36.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-19T07:43:36.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-19T07:43:36.365 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:43:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:36.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:36.521 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:36.521 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:43:36.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.1 up 2024-09-19T07:43:36.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.1 master ceph-brx 2024-09-19T07:43:36.521 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:43:36.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:36.601 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-09-19T07:43:36.601 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:43:36.601 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-19T07:43:36.616 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-19T07:43:36.616 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:43:36.616 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-19T07:43:36.672 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:43:36.738 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:43:36.795 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:43:36.795 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:43:36.795 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:43:36.795 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:43:36.795 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:43:36.795 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:43:36.796 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:43:36.796 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:43:36.796 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:43:36.796 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:43:36.796 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:43:36.796 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:43:36.796 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:43:36.796 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:43:36.796 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:43:36.796 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:43:36.796 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:43:36.797 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:43:36.797 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:43:36.797 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:43:36.797 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:43:36.797 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:43:36.797 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:43:36.797 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:43:36.797 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:43:36.797 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:43:36.797 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:43:36.798 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:43:36.798 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:43:36.798 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:43:36.798 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:43:36.798 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:43:36.798 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T07:43:36.798 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:43:36.798 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:43:36.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:36.798 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:43:36.851 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:43:36.851 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-09-19T07:43:36.851 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-09-19T07:43:36.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:43:36.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:43:36.919 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:43:36.961 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:43:36.978 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi016.stderr:2024-09-19T07:43:36.978+0000 7f1ddfb1d480 -1 init, newargv = 0x55b8d9a9f7d0 newargc=15 2024-09-19T07:43:36.978 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi016.stderr:ceph-fuse[45241]: starting ceph client 2024-09-19T07:43:36.979 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:43:36.980 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:43:36.980 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:43:36.980 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:43:36.980 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:43:36.980 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:43:36.980 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:43:36.980 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:43:36.980 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:43:36.980 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:43:36.980 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:43:36.980 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:43:36.980 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:43:36.980 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:43:36.981 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:43:36.981 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:43:36.981 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:43:36.981 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:43:36.981 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:43:36.981 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:43:36.981 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:43:36.981 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:43:36.981 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:43:36.981 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:43:36.982 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:43:36.982 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:43:36.982 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:43:36.982 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:43:36.982 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:43:36.982 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:43:36.982 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:43:36.982 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:43:36.982 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T07:43:36.982 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:43:36.982 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:43:36.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:36.983 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:43:36.983 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi016.stderr:ceph-fuse[45241]: starting fuse 2024-09-19T07:43:36.999 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:43:36.999 INFO:teuthology.orchestra.run.smithi016.stdout:44 2024-09-19T07:43:36.999 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2024-09-19T07:43:37.000 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:43:37.000 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:43:37.000 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:43:37.000 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:43:37.000 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:43:37.000 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:43:37.000 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:43:37.000 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-09-19T07:43:37.000 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:43:37.000 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-09-19T07:43:37.000 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:43:37.000 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:43:37.000 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:43:37.000 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:43:37.000 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:43:37.001 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:43:37.001 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:43:37.001 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:43:37.001 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:43:37.001 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:43:37.001 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:43:37.001 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:43:37.001 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:43:37.001 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:43:37.001 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:43:37.001 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.2")) 2024-09-19T07:43:37.001 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:43:37.134 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.2.45241.asok 2024-09-19T07:43:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:37.145 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.45241.asok 2024-09-19T07:43:37.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:37.145 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.45241.asok status 2024-09-19T07:43:37.283 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:43:37.284 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:43:37.284 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:43:37.284 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:43:37.284 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "2", 2024-09-19T07:43:37.284 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:43:37.284 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-09-19T07:43:37.284 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "45241", 2024-09-19T07:43:37.285 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:43:37.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:37.285 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-19T07:43:37.285 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-19T07:43:37.285 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4710, 2024-09-19T07:43:37.285 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:43:37.286 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:43:37.286 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:43:37.286 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4710 2024-09-19T07:43:37.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:37.286 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:43:37.286 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:43:37.286 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.2:0", 2024-09-19T07:43:37.287 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1094481506 2024-09-19T07:43:37.287 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:37.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:37.287 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:43:37.287 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:43:37.287 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.2:0", 2024-09-19T07:43:37.287 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1094481506 2024-09-19T07:43:37.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:37.288 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.4710 192.168.0.2:0/1094481506", 2024-09-19T07:43:37.288 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.2:0/1094481506", 2024-09-19T07:43:37.288 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-19T07:43:37.288 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 8, 2024-09-19T07:43:37.288 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 26, 2024-09-19T07:43:37.289 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:43:37.289 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:43:37.289 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:43:37.289 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:43:37.292 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-19T07:43:37.292 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:43:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.397+0000 7f02b4703640 1 Processor -- start 2024-09-19T07:43:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.397+0000 7f02b4703640 1 -- start start 2024-09-19T07:43:37.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.398+0000 7f02b4703640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f02ac15e680 0x7f02ac0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:37.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.398+0000 7f02b4703640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02ac0d0010 0x7f02ac0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:37.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.398+0000 7f02b4703640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02ac0d1880 con 0x7f02ac15e2a0 2024-09-19T07:43:37.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.398+0000 7f02b4703640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f02ac0d1b60 con 0x7f02ac15e680 2024-09-19T07:43:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.398+0000 7f02b4703640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f02ac0cf2d0 con 0x7f02ac0d0010 2024-09-19T07:43:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.398+0000 7f02b1c9e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f02ac15e680 0x7f02ac0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.398+0000 7f02b249f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f02ac15e2a0 0x7f02ac06d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38350/0 (socket says 172.21.15.16:38350) 2024-09-19T07:43:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.398+0000 7f02b249f640 1 -- 172.21.15.16:0/1802645492 learned_addr learned my addr 172.21.15.16:0/1802645492 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.398+0000 7f02b2ca0640 1 --2- 172.21.15.16:0/1802645492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02ac0d0010 0x7f02ac0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.398+0000 7f02b1c9e640 1 -- 172.21.15.16:0/1802645492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02ac0d0010 msgr2=0x7f02ac0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.398+0000 7f02b1c9e640 1 --2- 172.21.15.16:0/1802645492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02ac0d0010 0x7f02ac0d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.398+0000 7f02b1c9e640 1 -- 172.21.15.16:0/1802645492 >> v1:172.21.15.16:6789/0 conn(0x7f02ac15e2a0 legacy=0x7f02ac06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.398+0000 7f02b1c9e640 1 -- 172.21.15.16:0/1802645492 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ac0cf5b0 con 0x7f02ac15e680 2024-09-19T07:43:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.398+0000 7f02b149d640 1 -- 172.21.15.16:0/1802645492 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3984875839 0 0) 0x7f02ac0d1880 con 0x7f02ac15e2a0 2024-09-19T07:43:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.398+0000 7f02b2ca0640 1 --2- 172.21.15.16:0/1802645492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02ac0d0010 0x7f02ac0d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.399+0000 7f02b1c9e640 1 --2- 172.21.15.16:0/1802645492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02ac15e680 0x7f02ac0cfad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f029c0097a0 tx=0x7f029c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87dcf392d684b6c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.399+0000 7f02b149d640 1 -- 172.21.15.16:0/1802645492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c037070 con 0x7f02ac15e680 2024-09-19T07:43:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.399+0000 7f02b149d640 1 -- 172.21.15.16:0/1802645492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f029c02ae50 con 0x7f02ac15e680 2024-09-19T07:43:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.399+0000 7f02b149d640 1 -- 172.21.15.16:0/1802645492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c032660 con 0x7f02ac15e680 2024-09-19T07:43:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.400+0000 7f02b4703640 1 -- 172.21.15.16:0/1802645492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02ac15e680 msgr2=0x7f02ac0cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.400+0000 7f02b4703640 1 --2- 172.21.15.16:0/1802645492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02ac15e680 0x7f02ac0cfad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f029c0097a0 tx=0x7f029c02a100 comp rx=0 tx=0).stop 2024-09-19T07:43:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.400+0000 7f02b4703640 1 -- 172.21.15.16:0/1802645492 shutdown_connections 2024-09-19T07:43:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.400+0000 7f02b4703640 1 --2- 172.21.15.16:0/1802645492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02ac0d0010 0x7f02ac0d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.400+0000 7f02b4703640 1 --2- 172.21.15.16:0/1802645492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02ac15e680 0x7f02ac0cfad0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.400+0000 7f02b4703640 1 -- 172.21.15.16:0/1802645492 >> 172.21.15.16:0/1802645492 conn(0x7f02ac09d760 msgr2=0x7f02ac08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:37.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.400+0000 7f02b4703640 1 -- 172.21.15.16:0/1802645492 shutdown_connections 2024-09-19T07:43:37.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.400+0000 7f02b4703640 1 -- 172.21.15.16:0/1802645492 wait complete. 2024-09-19T07:43:37.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.401+0000 7f02b4703640 1 Processor -- start 2024-09-19T07:43:37.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.401+0000 7f02b4703640 1 -- start start 2024-09-19T07:43:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.402+0000 7f02b4703640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02ac0d0010 0x7f02ac1ea690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.402+0000 7f02b4703640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02ac15e2a0 0x7f02ac160da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.402+0000 7f02b4703640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02ac15e680 0x7f02ac1612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.402+0000 7f02b4703640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f02ac06dcb0 con 0x7f02ac15e680 2024-09-19T07:43:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.402+0000 7f02b4703640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f02ac06db30 con 0x7f02ac0d0010 2024-09-19T07:43:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.402+0000 7f02b4703640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f02ac0d48b0 con 0x7f02ac15e2a0 2024-09-19T07:43:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.402+0000 7f02b2ca0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02ac15e680 0x7f02ac1612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.402+0000 7f02b2ca0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02ac15e680 0x7f02ac1612e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56084/0 (socket says 172.21.15.16:56084) 2024-09-19T07:43:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.402+0000 7f02b2ca0640 1 -- 172.21.15.16:0/761451785 learned_addr learned my addr 172.21.15.16:0/761451785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.402+0000 7f02b249f640 1 --2- 172.21.15.16:0/761451785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02ac0d0010 0x7f02ac1ea690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.402+0000 7f02b1c9e640 1 --2- 172.21.15.16:0/761451785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02ac15e2a0 0x7f02ac160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.402+0000 7f02b2ca0640 1 -- 172.21.15.16:0/761451785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02ac15e2a0 msgr2=0x7f02ac160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.402+0000 7f02b2ca0640 1 --2- 172.21.15.16:0/761451785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02ac15e2a0 0x7f02ac160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.402+0000 7f02b2ca0640 1 -- 172.21.15.16:0/761451785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02ac0d0010 msgr2=0x7f02ac1ea690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.402+0000 7f02b2ca0640 1 --2- 172.21.15.16:0/761451785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02ac0d0010 0x7f02ac1ea690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.402+0000 7f02b2ca0640 1 -- 172.21.15.16:0/761451785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ac0b73f0 con 0x7f02ac15e680 2024-09-19T07:43:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.402+0000 7f02b249f640 1 --2- 172.21.15.16:0/761451785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02ac0d0010 0x7f02ac1ea690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.402+0000 7f02b1c9e640 1 --2- 172.21.15.16:0/761451785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02ac15e2a0 0x7f02ac160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.403+0000 7f02b2ca0640 1 --2- 172.21.15.16:0/761451785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02ac15e680 0x7f02ac1612e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f02a400cf20 tx=0x7f02a400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.403+0000 7f02a2ffd640 1 -- 172.21.15.16:0/761451785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a401a070 con 0x7f02ac15e680 2024-09-19T07:43:37.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.403+0000 7f02a2ffd640 1 -- 172.21.15.16:0/761451785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02a400d840 con 0x7f02ac15e680 2024-09-19T07:43:37.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.403+0000 7f02b4703640 1 -- 172.21.15.16:0/761451785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02ac0b76d0 con 0x7f02ac15e680 2024-09-19T07:43:37.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.403+0000 7f02a2ffd640 1 -- 172.21.15.16:0/761451785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a400dc30 con 0x7f02ac15e680 2024-09-19T07:43:37.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.403+0000 7f02b4703640 1 -- 172.21.15.16:0/761451785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02ac1eabd0 con 0x7f02ac15e680 2024-09-19T07:43:37.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.404+0000 7f02b4703640 1 -- 172.21.15.16:0/761451785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02ac0d4c70 con 0x7f02ac15e680 2024-09-19T07:43:37.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.404+0000 7f02a2ffd640 1 -- 172.21.15.16:0/761451785 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f02a401d070 con 0x7f02ac15e680 2024-09-19T07:43:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.405+0000 7f02a2ffd640 1 --2- 172.21.15.16:0/761451785 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f029006c2b0 0x7f029006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.405+0000 7f02a2ffd640 1 -- 172.21.15.16:0/761451785 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f02a408c070 con 0x7f02ac15e680 2024-09-19T07:43:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.405+0000 7f02b249f640 1 --2- 172.21.15.16:0/761451785 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f029006c2b0 0x7f029006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.406+0000 7f02b249f640 1 --2- 172.21.15.16:0/761451785 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f029006c2b0 0x7f029006e770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f02a8000f60 tx=0x7f02a8007c30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.410+0000 7f02a2ffd640 1 -- 172.21.15.16:0/761451785 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f02a405a4a0 con 0x7f02ac15e680 2024-09-19T07:43:37.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.619+0000 7f02b4703640 1 -- 172.21.15.16:0/761451785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f02ac06e4e0 con 0x7f02ac15e680 2024-09-19T07:43:37.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.620+0000 7f02a2ffd640 1 -- 172.21.15.16:0/761451785 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f02a4059da0 con 0x7f02ac15e680 2024-09-19T07:43:37.621 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-19T07:43:37.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.622+0000 7f02b4703640 1 -- 172.21.15.16:0/761451785 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f029006c2b0 msgr2=0x7f029006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:37.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.622+0000 7f02b4703640 1 --2- 172.21.15.16:0/761451785 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f029006c2b0 0x7f029006e770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f02a8000f60 tx=0x7f02a8007c30 comp rx=0 tx=0).stop 2024-09-19T07:43:37.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.622+0000 7f02b4703640 1 -- 172.21.15.16:0/761451785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02ac15e680 msgr2=0x7f02ac1612e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:37.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.622+0000 7f02b4703640 1 --2- 172.21.15.16:0/761451785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02ac15e680 0x7f02ac1612e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f02a400cf20 tx=0x7f02a400a7d0 comp rx=0 tx=0).stop 2024-09-19T07:43:37.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.622+0000 7f02b4703640 1 -- 172.21.15.16:0/761451785 shutdown_connections 2024-09-19T07:43:37.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.622+0000 7f02b4703640 1 --2- 172.21.15.16:0/761451785 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f029006c2b0 0x7f029006e770 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:37.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.622+0000 7f02b4703640 1 --2- 172.21.15.16:0/761451785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02ac15e680 0x7f02ac1612e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:37.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.622+0000 7f02b4703640 1 --2- 172.21.15.16:0/761451785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02ac15e2a0 0x7f02ac160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:37.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.622+0000 7f02b4703640 1 --2- 172.21.15.16:0/761451785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02ac0d0010 0x7f02ac1ea690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:37.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.622+0000 7f02b4703640 1 -- 172.21.15.16:0/761451785 >> 172.21.15.16:0/761451785 conn(0x7f02ac09d760 msgr2=0x7f02ac15f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:37.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.622+0000 7f02b4703640 1 -- 172.21.15.16:0/761451785 shutdown_connections 2024-09-19T07:43:37.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:37.622+0000 7f02b4703640 1 -- 172.21.15.16:0/761451785 wait complete. 2024-09-19T07:43:37.634 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:43:37.634 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:43:37.634 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-09-19T07:43:37.634 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T07:43:37.634 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T07:43:37.634 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:43:37.634 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:43:37.634 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:43:37.634 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:43:37.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:37.635 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-09-19T07:43:37.654 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:43:37.654 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:43:37.654 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:43:37.654 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:37.654 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-09-19T07:43:37.654 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:37.655 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:43:37.655 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:37.655 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2024-09-19T07:43:37.655 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:43:37.655 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:37.655 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2024-09-19T07:43:37.655 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:43:37.655 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:37.655 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:43:37.655 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:37.655 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T07:43:37.655 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42739sec preferred_lft 42739sec 2024-09-19T07:43:37.656 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link noprefixroute 2024-09-19T07:43:37.656 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:37.656 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T07:43:37.656 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:37.656 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-19T07:43:37.656 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link add name ceph-brx type bridge 2024-09-19T07:43:37.656 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip addr flush dev ceph-brx 2024-09-19T07:43:37.656 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set ceph-brx up 2024-09-19T07:43:37.656 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T07:43:37.657 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-19T07:43:37.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:37.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:37.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:37.874 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:43:37.899 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-09-19T07:43:37.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:37.900 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-09-19T07:43:37.970 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-09-19T07:43:37.970 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:43:37.970 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T07:43:37.970 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T07:43:37.970 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:43:37.971 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:37.971 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-19T07:43:37.971 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T07:43:37.971 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T07:43:37.971 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T07:43:37.971 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-19T07:43:38.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:38.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:38.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:38.252 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-09-19T07:43:38.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:38.269 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list-id 2024-09-19T07:43:38.324 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:38.325 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-19T07:43:38.325 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:43:38.325 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-19T07:43:38.325 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-19T07:43:38.414 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:38.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:38.444 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-19T07:43:38.444 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:38.444 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-19T07:43:38.444 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-19T07:43:38.444 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T07:43:38.444 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-19T07:43:38.444 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-19T07:43:38.445 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-19T07:43:38.445 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-19T07:43:38.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:38.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:38.567 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:38.567 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-19T07:43:38.568 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2024-09-19T07:43:38.568 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T07:43:38.568 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-19T07:43:38.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:38.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:38.645 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T07:43:38.645 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:43:38.645 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T07:43:38.660 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:43:38.661 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:43:38.661 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:43:38.717 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:43:38.783 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:43:38.839 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:43:38.839 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:43:38.839 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:43:38.839 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:43:38.839 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:43:38.839 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:43:38.839 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:43:38.839 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:43:38.839 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:43:38.840 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:43:38.840 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:43:38.840 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:43:38.840 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:43:38.840 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:43:38.840 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:43:38.840 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:43:38.840 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:43:38.840 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:43:38.840 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:43:38.840 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:43:38.840 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:43:38.840 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:43:38.841 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:43:38.841 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:43:38.841 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:43:38.841 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:43:38.841 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:43:38.841 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:43:38.841 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:43:38.841 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:43:38.841 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:43:38.841 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:43:38.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:38.842 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:43:38.897 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:43:38.897 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T07:43:38.940 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:43:38.965 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:43:39.021 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:43:39.021 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:43:39.021 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:43:39.021 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:43:39.021 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:43:39.021 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:43:39.021 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:43:39.021 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:43:39.021 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:43:39.022 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:43:39.022 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:43:39.022 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:43:39.022 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:43:39.022 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:43:39.022 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:43:39.022 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:43:39.022 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:43:39.022 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:43:39.022 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:43:39.022 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:43:39.022 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:43:39.022 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:43:39.023 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:43:39.023 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:43:39.023 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:43:39.023 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:43:39.023 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:43:39.023 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:43:39.023 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:43:39.023 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:43:39.023 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:43:39.023 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:43:39.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:39.023 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:43:39.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2024-09-19T07:43:39.061+0000 7f7561c17480 -1 init, newargv = 0x55c6073c2a40 newargc=15 2024-09-19T07:43:39.062 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[43005]: starting ceph client 2024-09-19T07:43:39.066 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[43005]: starting fuse 2024-09-19T07:43:39.077 INFO:teuthology.orchestra.run.smithi146.stdout:42 2024-09-19T07:43:39.077 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-19T07:43:39.077 DEBUG:teuthology.orchestra.run.smithi146:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:43:39.077 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2024-09-19T07:43:39.078 DEBUG:teuthology.orchestra.run.smithi146:> import re 2024-09-19T07:43:39.078 DEBUG:teuthology.orchestra.run.smithi146:> import os 2024-09-19T07:43:39.078 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2024-09-19T07:43:39.078 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:43:39.078 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2024-09-19T07:43:39.078 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T07:43:39.078 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2024-09-19T07:43:39.078 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T07:43:39.078 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:43:39.078 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2024-09-19T07:43:39.078 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2024-09-19T07:43:39.079 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2024-09-19T07:43:39.079 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2024-09-19T07:43:39.079 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:43:39.079 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2024-09-19T07:43:39.079 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:43:39.079 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:43:39.079 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:43:39.079 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2024-09-19T07:43:39.079 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2024-09-19T07:43:39.079 DEBUG:teuthology.orchestra.run.smithi146:> return f 2024-09-19T07:43:39.079 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:43:39.079 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:43:39.079 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2024-09-19T07:43:39.080 DEBUG:teuthology.orchestra.run.smithi146:> ' 2024-09-19T07:43:39.195 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.43005.asok 2024-09-19T07:43:39.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:39.206 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.43005.asok 2024-09-19T07:43:39.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:39.206 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.43005.asok status 2024-09-19T07:43:39.339 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-19T07:43:39.339 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2024-09-19T07:43:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:43:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:43:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2024-09-19T07:43:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2024-09-19T07:43:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T07:43:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "43005", 2024-09-19T07:43:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2024-09-19T07:43:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:43:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2024-09-19T07:43:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2024-09-19T07:43:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 4520, 2024-09-19T07:43:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2024-09-19T07:43:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2024-09-19T07:43:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2024-09-19T07:43:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 4520 2024-09-19T07:43:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:43:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:43:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:43:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:43:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 716422429 2024-09-19T07:43:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-19T07:43:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:43:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:43:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:43:39.342 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:43:39.342 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 716422429 2024-09-19T07:43:39.342 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:43:39.342 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.4520 192.168.0.1:0/716422429", 2024-09-19T07:43:39.342 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/716422429", 2024-09-19T07:43:39.342 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2024-09-19T07:43:39.342 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 8, 2024-09-19T07:43:39.342 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 26, 2024-09-19T07:43:39.342 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:43:39.342 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2024-09-19T07:43:39.342 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2024-09-19T07:43:39.342 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-19T07:43:39.348 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-19T07:43:39.348 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:43:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.462+0000 7f6dcecd7640 1 Processor -- start 2024-09-19T07:43:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.462+0000 7f6dcecd7640 1 -- start start 2024-09-19T07:43:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.463+0000 7f6dcecd7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dc815e2a0 0x7f6dc806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.463+0000 7f6dcecd7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6dc815e680 0x7f6dc80cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.463+0000 7f6dcecd7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dc80d1880 con 0x7f6dc806d960 2024-09-19T07:43:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.463+0000 7f6dcecd7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f6dc80d1b60 con 0x7f6dc815e680 2024-09-19T07:43:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.463+0000 7f6dcecd7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc80cf2d0 con 0x7f6dc815e2a0 2024-09-19T07:43:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.463+0000 7f6dc7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6dc815e680 0x7f6dc80cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.463+0000 7f6dcd274640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6dc806d960 0x7f6dc80d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38362/0 (socket says 172.21.15.16:38362) 2024-09-19T07:43:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.463+0000 7f6dcd274640 1 -- 172.21.15.16:0/307589753 learned_addr learned my addr 172.21.15.16:0/307589753 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.463+0000 7f6dcca73640 1 --2- 172.21.15.16:0/307589753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dc815e2a0 0x7f6dc806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.464+0000 7f6dc7fff640 1 -- 172.21.15.16:0/307589753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dc815e2a0 msgr2=0x7f6dc806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.464+0000 7f6dc7fff640 1 --2- 172.21.15.16:0/307589753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dc815e2a0 0x7f6dc806d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.464+0000 7f6dc7fff640 1 -- 172.21.15.16:0/307589753 >> v1:172.21.15.16:6789/0 conn(0x7f6dc806d960 legacy=0x7f6dc80d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.464+0000 7f6dc77fe640 1 -- 172.21.15.16:0/307589753 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3699530660 0 0) 0x7f6dc80d1880 con 0x7f6dc806d960 2024-09-19T07:43:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.464+0000 7f6dc7fff640 1 -- 172.21.15.16:0/307589753 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc80cf5b0 con 0x7f6dc815e680 2024-09-19T07:43:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.464+0000 7f6dcca73640 1 --2- 172.21.15.16:0/307589753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dc815e2a0 0x7f6dc806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.464+0000 7f6dc7fff640 1 --2- 172.21.15.16:0/307589753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc815e680 0x7f6dc80cfad0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0097a0 tx=0x7f6dbc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a55377cb8086464 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.465+0000 7f6dc77fe640 1 -- 172.21.15.16:0/307589753 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc037070 con 0x7f6dc815e680 2024-09-19T07:43:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.465+0000 7f6dc77fe640 1 -- 172.21.15.16:0/307589753 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dbc032440 con 0x7f6dc815e680 2024-09-19T07:43:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.465+0000 7f6dc77fe640 1 -- 172.21.15.16:0/307589753 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc032810 con 0x7f6dc815e680 2024-09-19T07:43:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.465+0000 7f6dcecd7640 1 -- 172.21.15.16:0/307589753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc815e680 msgr2=0x7f6dc80cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.465+0000 7f6dcecd7640 1 --2- 172.21.15.16:0/307589753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc815e680 0x7f6dc80cfad0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0097a0 tx=0x7f6dbc02a180 comp rx=0 tx=0).stop 2024-09-19T07:43:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.465+0000 7f6dcecd7640 1 -- 172.21.15.16:0/307589753 shutdown_connections 2024-09-19T07:43:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.465+0000 7f6dcecd7640 1 --2- 172.21.15.16:0/307589753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc815e680 0x7f6dc80cfad0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.465+0000 7f6dcecd7640 1 --2- 172.21.15.16:0/307589753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dc815e2a0 0x7f6dc806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.465+0000 7f6dcecd7640 1 -- 172.21.15.16:0/307589753 >> 172.21.15.16:0/307589753 conn(0x7f6dc809d760 msgr2=0x7f6dc808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.465+0000 7f6dcecd7640 1 -- 172.21.15.16:0/307589753 shutdown_connections 2024-09-19T07:43:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.465+0000 7f6dcecd7640 1 -- 172.21.15.16:0/307589753 wait complete. 2024-09-19T07:43:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.466+0000 7f6dcecd7640 1 Processor -- start 2024-09-19T07:43:39.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.467+0000 7f6dcecd7640 1 -- start start 2024-09-19T07:43:39.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.467+0000 7f6dcecd7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dc806d960 0x7f6dc81568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:39.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.467+0000 7f6dcecd7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6dc815e2a0 0x7f6dc80c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:39.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.467+0000 7f6dcecd7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc815e680 0x7f6dc80ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:39.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.467+0000 7f6dcecd7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc80d4c00 con 0x7f6dc815e680 2024-09-19T07:43:39.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.467+0000 7f6dcecd7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc80d4a80 con 0x7f6dc815e2a0 2024-09-19T07:43:39.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.467+0000 7f6dcecd7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc80d0010 con 0x7f6dc806d960 2024-09-19T07:43:39.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.468+0000 7f6dcd274640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc815e680 0x7f6dc80ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:39.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.468+0000 7f6dcca73640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dc806d960 0x7f6dc81568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:39.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.468+0000 7f6dcd274640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc815e680 0x7f6dc80ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56100/0 (socket says 172.21.15.16:56100) 2024-09-19T07:43:39.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.468+0000 7f6dcd274640 1 -- 172.21.15.16:0/1377458468 learned_addr learned my addr 172.21.15.16:0/1377458468 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:39.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.468+0000 7f6dc7fff640 1 --2- 172.21.15.16:0/1377458468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6dc815e2a0 0x7f6dc80c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:39.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.468+0000 7f6dcd274640 1 -- 172.21.15.16:0/1377458468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dc806d960 msgr2=0x7f6dc81568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:39.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.468+0000 7f6dcd274640 1 --2- 172.21.15.16:0/1377458468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dc806d960 0x7f6dc81568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.468+0000 7f6dcd274640 1 -- 172.21.15.16:0/1377458468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6dc815e2a0 msgr2=0x7f6dc80c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:39.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.468+0000 7f6dcd274640 1 --2- 172.21.15.16:0/1377458468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6dc815e2a0 0x7f6dc80c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.468+0000 7f6dcd274640 1 -- 172.21.15.16:0/1377458468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc8156e30 con 0x7f6dc815e680 2024-09-19T07:43:39.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.468+0000 7f6dcd274640 1 --2- 172.21.15.16:0/1377458468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc815e680 0x7f6dc80ca1e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000c7e0 tx=0x7f6dc000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:39.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.468+0000 7f6dc57fa640 1 -- 172.21.15.16:0/1377458468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0018070 con 0x7f6dc815e680 2024-09-19T07:43:39.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.468+0000 7f6dc57fa640 1 -- 172.21.15.16:0/1377458468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc000a640 con 0x7f6dc815e680 2024-09-19T07:43:39.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.468+0000 7f6dcecd7640 1 -- 172.21.15.16:0/1377458468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc80cd6d0 con 0x7f6dc815e680 2024-09-19T07:43:39.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.468+0000 7f6dc57fa640 1 -- 172.21.15.16:0/1377458468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc000aa30 con 0x7f6dc815e680 2024-09-19T07:43:39.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.468+0000 7f6dcecd7640 1 -- 172.21.15.16:0/1377458468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc80b7bf0 con 0x7f6dc815e680 2024-09-19T07:43:39.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.469+0000 7f6dcecd7640 1 -- 172.21.15.16:0/1377458468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dc806d420 con 0x7f6dc815e680 2024-09-19T07:43:39.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.470+0000 7f6dc57fa640 1 -- 172.21.15.16:0/1377458468 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6dc001b070 con 0x7f6dc815e680 2024-09-19T07:43:39.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.470+0000 7f6dc57fa640 1 --2- 172.21.15.16:0/1377458468 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6db406c2b0 0x7f6db406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:39.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.470+0000 7f6dc57fa640 1 -- 172.21.15.16:0/1377458468 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f6dc008ad10 con 0x7f6dc815e680 2024-09-19T07:43:39.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.470+0000 7f6dcca73640 1 --2- 172.21.15.16:0/1377458468 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6db406c2b0 0x7f6db406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:39.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.471+0000 7f6dcca73640 1 --2- 172.21.15.16:0/1377458468 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6db406c2b0 0x7f6db406e770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009560 tx=0x7f6db8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.475+0000 7f6dc57fa640 1 -- 172.21.15.16:0/1377458468 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6dc0090020 con 0x7f6dc815e680 2024-09-19T07:43:39.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.685+0000 7f6dcecd7640 1 -- 172.21.15.16:0/1377458468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6dc8160a80 con 0x7f6dc815e680 2024-09-19T07:43:39.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.685+0000 7f6dc57fa640 1 -- 172.21.15.16:0/1377458468 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f6dc0058ae0 con 0x7f6dc815e680 2024-09-19T07:43:39.687 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-19T07:43:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.687+0000 7f6dcecd7640 1 -- 172.21.15.16:0/1377458468 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6db406c2b0 msgr2=0x7f6db406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.687+0000 7f6dcecd7640 1 --2- 172.21.15.16:0/1377458468 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6db406c2b0 0x7f6db406e770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009560 tx=0x7f6db8009190 comp rx=0 tx=0).stop 2024-09-19T07:43:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.687+0000 7f6dcecd7640 1 -- 172.21.15.16:0/1377458468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc815e680 msgr2=0x7f6dc80ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.687+0000 7f6dcecd7640 1 --2- 172.21.15.16:0/1377458468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc815e680 0x7f6dc80ca1e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000c7e0 tx=0x7f6dc000cca0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.687+0000 7f6dcecd7640 1 -- 172.21.15.16:0/1377458468 shutdown_connections 2024-09-19T07:43:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.687+0000 7f6dcecd7640 1 --2- 172.21.15.16:0/1377458468 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6db406c2b0 0x7f6db406e770 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.687+0000 7f6dcecd7640 1 --2- 172.21.15.16:0/1377458468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc815e680 0x7f6dc80ca1e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.687+0000 7f6dcecd7640 1 --2- 172.21.15.16:0/1377458468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6dc815e2a0 0x7f6dc80c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.687+0000 7f6dcecd7640 1 --2- 172.21.15.16:0/1377458468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dc806d960 0x7f6dc81568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.687+0000 7f6dcecd7640 1 -- 172.21.15.16:0/1377458468 >> 172.21.15.16:0/1377458468 conn(0x7f6dc809d760 msgr2=0x7f6dc81628e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.687+0000 7f6dcecd7640 1 -- 172.21.15.16:0/1377458468 shutdown_connections 2024-09-19T07:43:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:39.687+0000 7f6dcecd7640 1 -- 172.21.15.16:0/1377458468 wait complete. 2024-09-19T07:43:39.700 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:43:39.700 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:43:39.700 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-09-19T07:43:39.700 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-19T07:43:39.700 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-19T07:43:39.700 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:43:39.700 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:43:39.700 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:43:39.700 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-19T07:43:39.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:39.701 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-09-19T07:43:39.717 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:43:39.717 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:43:39.717 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:43:39.717 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:39.717 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-09-19T07:43:39.717 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:39.717 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:43:39.718 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:39.718 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2024-09-19T07:43:39.718 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:43:39.718 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:39.718 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2024-09-19T07:43:39.718 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:43:39.718 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:39.718 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:43:39.718 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:39.718 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T07:43:39.718 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42737sec preferred_lft 42737sec 2024-09-19T07:43:39.718 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link noprefixroute 2024-09-19T07:43:39.719 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:39.719 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:43:39.719 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether f6:1f:22:44:9d:cd brd ff:ff:ff:ff:ff:ff 2024-09-19T07:43:39.719 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:43:39.719 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:39.719 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::e47f:32ff:fe68:31ca/64 scope link 2024-09-19T07:43:39.719 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:39.719 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:43:39.719 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether f6:1f:22:44:9d:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:43:39.719 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::f41f:22ff:fe44:9dcd/64 scope link tentative 2024-09-19T07:43:39.719 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:39.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:39.720 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:43:39.781 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-09-19T07:43:39.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:39.783 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-09-19T07:43:39.840 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-09-19T07:43:39.840 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:43:39.840 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T07:43:39.840 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T07:43:39.840 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:43:39.841 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:39.841 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-19T07:43:39.841 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T07:43:39.841 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T07:43:39.841 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T07:43:39.841 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-19T07:43:39.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:39.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:39.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:39.966 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-09-19T07:43:39.981 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T07:43:39.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:39.982 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list-id 2024-09-19T07:43:40.040 INFO:teuthology.orchestra.run.smithi146.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-19T07:43:40.041 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:40.041 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-19T07:43:40.041 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:43:40.041 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-19T07:43:40.041 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-19T07:43:40.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:40.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:40.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:40.163 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-19T07:43:40.188 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:43:40.188 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:43:40.188 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:43:40.188 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:40.188 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-09-19T07:43:40.188 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:40.188 INFO:teuthology.orchestra.run.smithi146.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:43:40.188 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0a:de:ce:57:a3:8f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-19T07:43:40.188 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-19T07:43:40.188 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:40.189 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::8de:ceff:fe57:a38f/64 scope link 2024-09-19T07:43:40.189 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:40.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:40.190 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-19T07:43:40.254 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:43:40.254 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:43:40.254 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:43:40.254 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:40.254 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-09-19T07:43:40.255 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:40.255 INFO:teuthology.orchestra.run.smithi146.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:43:40.255 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0a:de:ce:57:a3:8f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-19T07:43:40.255 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-19T07:43:40.255 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:40.255 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::8de:ceff:fe57:a38f/64 scope link 2024-09-19T07:43:40.255 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:43:40.256 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-19T07:43:40.256 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:40.256 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-19T07:43:40.256 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-19T07:43:40.257 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-19T07:43:40.257 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-19T07:43:40.257 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-19T07:43:40.257 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-19T07:43:40.257 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-19T07:43:40.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:40.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:40.410 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:43:40.410 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-19T07:43:40.410 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.1 up 2024-09-19T07:43:40.410 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set dev brx.1 master ceph-brx 2024-09-19T07:43:40.410 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-19T07:43:40.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:43:40.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:40.487 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-09-19T07:43:40.487 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:43:40.488 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-19T07:43:40.503 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-19T07:43:40.503 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:43:40.503 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-19T07:43:40.559 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:43:40.624 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:43:40.680 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:43:40.680 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:43:40.680 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:43:40.680 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:43:40.681 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:43:40.681 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:43:40.681 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:43:40.681 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:43:40.681 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:43:40.681 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:43:40.681 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:43:40.681 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:43:40.681 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:43:40.681 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:43:40.681 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:43:40.681 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:43:40.682 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:43:40.682 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:43:40.682 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:43:40.682 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:43:40.682 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:43:40.682 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:43:40.682 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:43:40.682 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:43:40.682 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:43:40.682 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:43:40.682 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:43:40.682 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:43:40.682 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:43:40.682 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:43:40.683 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:43:40.683 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:43:40.683 INFO:teuthology.orchestra.run.smithi146.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T07:43:40.683 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:43:40.683 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:43:40.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:40.683 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:43:40.735 INFO:teuthology.orchestra.run.smithi146.stdout:42 2024-09-19T07:43:40.736 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-09-19T07:43:40.736 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-09-19T07:43:40.778 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:43:40.803 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:43:40.859 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:43:40.859 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:43:40.859 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:43:40.860 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:43:40.860 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:43:40.860 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:43:40.860 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:43:40.860 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:43:40.860 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:43:40.860 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:43:40.860 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:43:40.860 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:43:40.860 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:43:40.860 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:43:40.860 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:43:40.861 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:43:40.861 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:43:40.861 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:43:40.861 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:43:40.861 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:43:40.861 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:43:40.861 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:43:40.861 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:43:40.861 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:43:40.861 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:43:40.861 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:43:40.861 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:43:40.861 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:43:40.862 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:43:40.862 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:43:40.862 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:43:40.862 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:43:40.862 INFO:teuthology.orchestra.run.smithi146.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T07:43:40.862 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:43:40.862 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:43:40.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:40.862 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:43:40.863 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi146.stderr:2024-09-19T07:43:40.859+0000 7f91804ab480 -1 init, newargv = 0x5573e41f2110 newargc=15 2024-09-19T07:43:40.863 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi146.stderr:ceph-fuse[43443]: starting ceph client 2024-09-19T07:43:40.864 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi146.stderr:ceph-fuse[43443]: starting fuse 2024-09-19T07:43:40.879 INFO:teuthology.orchestra.run.smithi146.stdout:42 2024-09-19T07:43:40.879 INFO:teuthology.orchestra.run.smithi146.stdout:44 2024-09-19T07:43:40.879 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 44] 2024-09-19T07:43:40.879 DEBUG:teuthology.orchestra.run.smithi146:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:43:40.879 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2024-09-19T07:43:40.879 DEBUG:teuthology.orchestra.run.smithi146:> import re 2024-09-19T07:43:40.879 DEBUG:teuthology.orchestra.run.smithi146:> import os 2024-09-19T07:43:40.879 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2024-09-19T07:43:40.880 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:43:40.880 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2024-09-19T07:43:40.880 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-09-19T07:43:40.880 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2024-09-19T07:43:40.880 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-09-19T07:43:40.880 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:43:40.880 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2024-09-19T07:43:40.880 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2024-09-19T07:43:40.880 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2024-09-19T07:43:40.880 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2024-09-19T07:43:40.880 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:43:40.880 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2024-09-19T07:43:40.880 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:43:40.880 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:43:40.880 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:43:40.880 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2024-09-19T07:43:40.881 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2024-09-19T07:43:40.881 DEBUG:teuthology.orchestra.run.smithi146:> return f 2024-09-19T07:43:40.881 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:43:40.881 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:43:40.881 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.3")) 2024-09-19T07:43:40.881 DEBUG:teuthology.orchestra.run.smithi146:> ' 2024-09-19T07:43:40.996 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.3.43443.asok 2024-09-19T07:43:40.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:43:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:43:41.007 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.43443.asok 2024-09-19T07:43:41.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:41.007 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.43443.asok status 2024-09-19T07:43:41.140 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-19T07:43:41.140 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2024-09-19T07:43:41.140 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:43:41.140 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:43:41.140 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "3", 2024-09-19T07:43:41.140 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2024-09-19T07:43:41.140 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-09-19T07:43:41.140 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "43443", 2024-09-19T07:43:41.140 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2024-09-19T07:43:41.141 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:43:41.141 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2024-09-19T07:43:41.141 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2024-09-19T07:43:41.141 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 4672, 2024-09-19T07:43:41.141 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2024-09-19T07:43:41.141 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2024-09-19T07:43:41.141 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2024-09-19T07:43:41.141 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 4672 2024-09-19T07:43:41.141 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:43:41.141 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:43:41.141 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:43:41.141 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.2:0", 2024-09-19T07:43:41.141 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 2180398340 2024-09-19T07:43:41.141 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-19T07:43:41.142 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:43:41.142 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:43:41.142 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:43:41.142 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.2:0", 2024-09-19T07:43:41.142 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 2180398340 2024-09-19T07:43:41.142 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:43:41.142 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.4672 192.168.0.2:0/2180398340", 2024-09-19T07:43:41.142 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.2:0/2180398340", 2024-09-19T07:43:41.142 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2024-09-19T07:43:41.142 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 8, 2024-09-19T07:43:41.142 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 26, 2024-09-19T07:43:41.142 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:43:41.142 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2024-09-19T07:43:41.142 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2024-09-19T07:43:41.143 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-19T07:43:41.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:41.148 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-19T07:43:41.149 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T07:43:41.169 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-19T07:43:41.169 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T07:43:41.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:41.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:43:41.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:41.241 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-19T07:43:41.241 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.2 2024-09-19T07:43:41.301 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-19T07:43:41.301 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-09-19T07:43:41.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:41.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-19T07:43:41.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:41.371 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2024-09-19T07:43:41.372 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T07:43:41.388 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2024-09-19T07:43:41.388 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T07:43:41.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:41.389 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:43:41.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:41.453 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2024-09-19T07:43:41.453 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.3 2024-09-19T07:43:41.509 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2024-09-19T07:43:41.509 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-09-19T07:43:41.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:41.509 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-19T07:43:41.574 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-19T07:43:41.617 INFO:tasks.cephfs_test_runner:Starting test: test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) Mostly readdir acquires caps faster than the mds recalls, so the cap 2024-09-19T07:43:41.617 INFO:tasks.cephfs_test_runner:test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-19T07:43:41.617 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:43:41.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.732+0000 7fcf6ffff640 1 Processor -- start 2024-09-19T07:43:41.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.732+0000 7fcf6ffff640 1 -- start start 2024-09-19T07:43:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.733+0000 7fcf6ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf700b4170 0x7fcf700d27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.733+0000 7fcf6ffff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf700d2d10 0x7fcf700cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.733+0000 7fcf6ffff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf700d24d0 con 0x7fcf7015e500 2024-09-19T07:43:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.733+0000 7fcf6ffff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fcf700b5210 con 0x7fcf700b4170 2024-09-19T07:43:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.733+0000 7fcf6ffff640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcf700b4600 con 0x7fcf700d2d10 2024-09-19T07:43:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.733+0000 7fcf6e7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf700b4170 0x7fcf700d27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.733+0000 7fcf6effd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcf7015e500 0x7fcf700d1dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38378/0 (socket says 172.21.15.16:38378) 2024-09-19T07:43:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.733+0000 7fcf6effd640 1 -- 172.21.15.16:0/2698352316 learned_addr learned my addr 172.21.15.16:0/2698352316 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.733+0000 7fcf6f7fe640 1 --2- 172.21.15.16:0/2698352316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf700d2d10 0x7fcf700cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.733+0000 7fcf6e7fc640 1 -- 172.21.15.16:0/2698352316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf700d2d10 msgr2=0x7fcf700cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.733+0000 7fcf6e7fc640 1 --2- 172.21.15.16:0/2698352316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf700d2d10 0x7fcf700cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.733+0000 7fcf6e7fc640 1 -- 172.21.15.16:0/2698352316 >> v1:172.21.15.16:6789/0 conn(0x7fcf7015e500 legacy=0x7fcf700d1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.733+0000 7fcf6e7fc640 1 -- 172.21.15.16:0/2698352316 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf700b48e0 con 0x7fcf700b4170 2024-09-19T07:43:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.733+0000 7fcf6dffb640 1 -- 172.21.15.16:0/2698352316 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3820521205 0 0) 0x7fcf700d24d0 con 0x7fcf7015e500 2024-09-19T07:43:41.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.734+0000 7fcf6e7fc640 1 --2- 172.21.15.16:0/2698352316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf700b4170 0x7fcf700d27d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcf640097a0 tx=0x7fcf6402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a1b2bc473657f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:41.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.734+0000 7fcf6dffb640 1 -- 172.21.15.16:0/2698352316 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64037070 con 0x7fcf700b4170 2024-09-19T07:43:41.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.734+0000 7fcf6dffb640 1 -- 172.21.15.16:0/2698352316 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf64032440 con 0x7fcf700b4170 2024-09-19T07:43:41.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.734+0000 7fcf6dffb640 1 -- 172.21.15.16:0/2698352316 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64032810 con 0x7fcf700b4170 2024-09-19T07:43:41.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.735+0000 7fcf6ffff640 1 -- 172.21.15.16:0/2698352316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf700b4170 msgr2=0x7fcf700d27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:41.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.735+0000 7fcf6ffff640 1 --2- 172.21.15.16:0/2698352316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf700b4170 0x7fcf700d27d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcf640097a0 tx=0x7fcf6402a210 comp rx=0 tx=0).stop 2024-09-19T07:43:41.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.735+0000 7fcf6ffff640 1 -- 172.21.15.16:0/2698352316 shutdown_connections 2024-09-19T07:43:41.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.735+0000 7fcf6ffff640 1 --2- 172.21.15.16:0/2698352316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf700d2d10 0x7fcf700cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:41.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.735+0000 7fcf6ffff640 1 --2- 172.21.15.16:0/2698352316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf700b4170 0x7fcf700d27d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:41.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.735+0000 7fcf6ffff640 1 -- 172.21.15.16:0/2698352316 >> 172.21.15.16:0/2698352316 conn(0x7fcf7008dc00 msgr2=0x7fcf7009d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:41.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.735+0000 7fcf6ffff640 1 -- 172.21.15.16:0/2698352316 shutdown_connections 2024-09-19T07:43:41.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.735+0000 7fcf6ffff640 1 -- 172.21.15.16:0/2698352316 wait complete. 2024-09-19T07:43:41.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.736+0000 7fcf6ffff640 1 Processor -- start 2024-09-19T07:43:41.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.736+0000 7fcf6ffff640 1 -- start start 2024-09-19T07:43:41.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.737+0000 7fcf6ffff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf700d2d10 0x7fcf700cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:41.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.737+0000 7fcf6ffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf7015e500 0x7fcf700b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:41.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.737+0000 7fcf6ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf700cc850 0x7fcf700b7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:41.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.737+0000 7fcf6ffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf700cf7d0 con 0x7fcf700cc850 2024-09-19T07:43:41.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.737+0000 7fcf6ffff640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcf700b4ea0 con 0x7fcf7015e500 2024-09-19T07:43:41.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.737+0000 7fcf6ffff640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcf700cf950 con 0x7fcf700d2d10 2024-09-19T07:43:41.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.737+0000 7fcf6f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf700cc850 0x7fcf700b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:41.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.737+0000 7fcf6f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf700cc850 0x7fcf700b7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56126/0 (socket says 172.21.15.16:56126) 2024-09-19T07:43:41.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.737+0000 7fcf6f7fe640 1 -- 172.21.15.16:0/176204334 learned_addr learned my addr 172.21.15.16:0/176204334 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:41.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.737+0000 7fcf6effd640 1 --2- 172.21.15.16:0/176204334 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf700d2d10 0x7fcf700cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:41.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.738+0000 7fcf6e7fc640 1 --2- 172.21.15.16:0/176204334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf7015e500 0x7fcf700b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:41.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.738+0000 7fcf6f7fe640 1 -- 172.21.15.16:0/176204334 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf700d2d10 msgr2=0x7fcf700cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:41.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.738+0000 7fcf6f7fe640 1 --2- 172.21.15.16:0/176204334 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf700d2d10 0x7fcf700cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:41.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.738+0000 7fcf6f7fe640 1 -- 172.21.15.16:0/176204334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf7015e500 msgr2=0x7fcf700b7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:41.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.738+0000 7fcf6f7fe640 1 --2- 172.21.15.16:0/176204334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf7015e500 0x7fcf700b7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:41.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.738+0000 7fcf6f7fe640 1 -- 172.21.15.16:0/176204334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf700ccd90 con 0x7fcf700cc850 2024-09-19T07:43:41.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.738+0000 7fcf6e7fc640 1 --2- 172.21.15.16:0/176204334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf7015e500 0x7fcf700b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:41.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.738+0000 7fcf6f7fe640 1 --2- 172.21.15.16:0/176204334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf700cc850 0x7fcf700b7580 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcf68007e40 tx=0x7fcf6800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:41.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.738+0000 7fcf577fe640 1 -- 172.21.15.16:0/176204334 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68014620 con 0x7fcf700cc850 2024-09-19T07:43:41.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.738+0000 7fcf577fe640 1 -- 172.21.15.16:0/176204334 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf68019070 con 0x7fcf700cc850 2024-09-19T07:43:41.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.738+0000 7fcf577fe640 1 -- 172.21.15.16:0/176204334 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68014a20 con 0x7fcf700cc850 2024-09-19T07:43:41.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.738+0000 7fcf6ffff640 1 -- 172.21.15.16:0/176204334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf700cd070 con 0x7fcf700cc850 2024-09-19T07:43:41.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.738+0000 7fcf6ffff640 1 -- 172.21.15.16:0/176204334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf700cd630 con 0x7fcf700cc850 2024-09-19T07:43:41.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.739+0000 7fcf6ffff640 1 -- 172.21.15.16:0/176204334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf70075710 con 0x7fcf700cc850 2024-09-19T07:43:41.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.740+0000 7fcf577fe640 1 -- 172.21.15.16:0/176204334 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcf68014bc0 con 0x7fcf700cc850 2024-09-19T07:43:41.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.740+0000 7fcf577fe640 1 --2- 172.21.15.16:0/176204334 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcf5c06c2b0 0x7fcf5c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:41.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.740+0000 7fcf577fe640 1 -- 172.21.15.16:0/176204334 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fcf6808bfe0 con 0x7fcf700cc850 2024-09-19T07:43:41.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.740+0000 7fcf6effd640 1 --2- 172.21.15.16:0/176204334 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcf5c06c2b0 0x7fcf5c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:41.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.741+0000 7fcf6effd640 1 --2- 172.21.15.16:0/176204334 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcf5c06c2b0 0x7fcf5c06e770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcf60007fd0 tx=0x7fcf60007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:41.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.744+0000 7fcf577fe640 1 -- 172.21.15.16:0/176204334 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fcf70075710 con 0x7fcf700cc850 2024-09-19T07:43:41.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.922+0000 7fcf6ffff640 1 -- 172.21.15.16:0/176204334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf700d10b0 con 0x7fcf700cc850 2024-09-19T07:43:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.923+0000 7fcf577fe640 1 -- 172.21.15.16:0/176204334 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18622 (secure 0 0 0) 0x7fcf6805a410 con 0x7fcf700cc850 2024-09-19T07:43:41.925 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:41.925 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:43:10.642082+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:43:03.485582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:43:04.603961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:43:08.299591+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:43:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.925+0000 7fcf6ffff640 1 -- 172.21.15.16:0/176204334 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcf5c06c2b0 msgr2=0x7fcf5c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.925+0000 7fcf6ffff640 1 --2- 172.21.15.16:0/176204334 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcf5c06c2b0 0x7fcf5c06e770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcf60007fd0 tx=0x7fcf60007de0 comp rx=0 tx=0).stop 2024-09-19T07:43:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.925+0000 7fcf6ffff640 1 -- 172.21.15.16:0/176204334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf700cc850 msgr2=0x7fcf700b7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.925+0000 7fcf6ffff640 1 --2- 172.21.15.16:0/176204334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf700cc850 0x7fcf700b7580 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcf68007e40 tx=0x7fcf6800e4b0 comp rx=0 tx=0).stop 2024-09-19T07:43:41.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.925+0000 7fcf6ffff640 1 -- 172.21.15.16:0/176204334 shutdown_connections 2024-09-19T07:43:41.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.925+0000 7fcf6ffff640 1 --2- 172.21.15.16:0/176204334 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcf5c06c2b0 0x7fcf5c06e770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:41.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.925+0000 7fcf6ffff640 1 --2- 172.21.15.16:0/176204334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf700cc850 0x7fcf700b7580 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:41.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.925+0000 7fcf6ffff640 1 --2- 172.21.15.16:0/176204334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf7015e500 0x7fcf700b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:41.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.925+0000 7fcf6ffff640 1 --2- 172.21.15.16:0/176204334 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf700d2d10 0x7fcf700cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:41.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.925+0000 7fcf6ffff640 1 -- 172.21.15.16:0/176204334 >> 172.21.15.16:0/176204334 conn(0x7fcf7008dc00 msgr2=0x7fcf700b0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:41.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.925+0000 7fcf6ffff640 1 -- 172.21.15.16:0/176204334 shutdown_connections 2024-09-19T07:43:41.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:41.925+0000 7fcf6ffff640 1 -- 172.21.15.16:0/176204334 wait complete. 2024-09-19T07:43:41.938 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T07:42:54.500338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T07:43:03.485582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T07:43:04.603961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-19T07:43:08.299591+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-19T07:43:41.939 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T07:43:42.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.043+0000 7fa1c8d3b640 1 Processor -- start 2024-09-19T07:43:42.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.043+0000 7fa1c8d3b640 1 -- start start 2024-09-19T07:43:42.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.044+0000 7fa1c8d3b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa1c415f0d0 0x7fa1c415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.044+0000 7fa1c8d3b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1c40d0070 0x7fa1c40d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.044+0000 7fa1c8d3b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1c40d1940 0x7fa1c40d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.044+0000 7fa1c8d3b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa1c40cf2b0 con 0x7fa1c40d0070 2024-09-19T07:43:42.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.044+0000 7fa1c8d3b640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa1c40cf590 con 0x7fa1c415f0d0 2024-09-19T07:43:42.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.044+0000 7fa1c8d3b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c40cf870 con 0x7fa1c40d1940 2024-09-19T07:43:42.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.044+0000 7fa1c1d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1c40d0070 0x7fa1c40d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.044+0000 7fa1c1d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1c40d0070 0x7fa1c40d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56132/0 (socket says 172.21.15.16:56132) 2024-09-19T07:43:42.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.044+0000 7fa1c1d9b640 1 -- 172.21.15.16:0/492161234 learned_addr learned my addr 172.21.15.16:0/492161234 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:42.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.044+0000 7fa1c2d9d640 1 --2- 172.21.15.16:0/492161234 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1c40d1940 0x7fa1c40d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.044+0000 7fa1c259c640 1 --2- 172.21.15.16:0/492161234 >> v2:172.21.15.146:3300/0 conn(0x7fa1c415f0d0 0x7fa1c415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.044+0000 7fa1c2d9d640 1 -- 172.21.15.16:0/492161234 >> v2:172.21.15.146:3300/0 conn(0x7fa1c415f0d0 msgr2=0x7fa1c415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:42.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.044+0000 7fa1c2d9d640 1 --2- 172.21.15.16:0/492161234 >> v2:172.21.15.146:3300/0 conn(0x7fa1c415f0d0 0x7fa1c415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.044+0000 7fa1c2d9d640 1 -- 172.21.15.16:0/492161234 >> v2:172.21.15.16:3300/0 conn(0x7fa1c40d0070 msgr2=0x7fa1c40d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:42.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.044+0000 7fa1c2d9d640 1 --2- 172.21.15.16:0/492161234 >> v2:172.21.15.16:3300/0 conn(0x7fa1c40d0070 0x7fa1c40d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.044+0000 7fa1c2d9d640 1 -- 172.21.15.16:0/492161234 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c40cfb50 con 0x7fa1c40d1940 2024-09-19T07:43:42.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.044+0000 7fa1c1d9b640 1 --2- 172.21.15.16:0/492161234 >> v2:172.21.15.16:3300/0 conn(0x7fa1c40d0070 0x7fa1c40d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:42.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.045+0000 7fa1c2d9d640 1 --2- 172.21.15.16:0/492161234 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1c40d1940 0x7fa1c40d4130 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40b6f40 tx=0x7fa1bc02a540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3be0aa53c61af900 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:42.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.045+0000 7fa1c159a640 1 -- 172.21.15.16:0/492161234 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc007740 con 0x7fa1c40d1940 2024-09-19T07:43:42.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.045+0000 7fa1c159a640 1 -- 172.21.15.16:0/492161234 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1bc0078e0 con 0x7fa1c40d1940 2024-09-19T07:43:42.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.045+0000 7fa1c159a640 1 -- 172.21.15.16:0/492161234 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc007cb0 con 0x7fa1c40d1940 2024-09-19T07:43:42.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.046+0000 7fa1c8d3b640 1 -- 172.21.15.16:0/492161234 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1c40d1940 msgr2=0x7fa1c40d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.046+0000 7fa1c8d3b640 1 --2- 172.21.15.16:0/492161234 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1c40d1940 0x7fa1c40d4130 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40b6f40 tx=0x7fa1bc02a540 comp rx=0 tx=0).stop 2024-09-19T07:43:42.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.046+0000 7fa1c8d3b640 1 -- 172.21.15.16:0/492161234 shutdown_connections 2024-09-19T07:43:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.046+0000 7fa1c8d3b640 1 --2- 172.21.15.16:0/492161234 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1c40d1940 0x7fa1c40d4130 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.046+0000 7fa1c8d3b640 1 --2- 172.21.15.16:0/492161234 >> v2:172.21.15.16:3300/0 conn(0x7fa1c40d0070 0x7fa1c40d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.046+0000 7fa1c8d3b640 1 --2- 172.21.15.16:0/492161234 >> v2:172.21.15.146:3300/0 conn(0x7fa1c415f0d0 0x7fa1c415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.046+0000 7fa1c8d3b640 1 -- 172.21.15.16:0/492161234 >> 172.21.15.16:0/492161234 conn(0x7fa1c409d760 msgr2=0x7fa1c408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.046+0000 7fa1c8d3b640 1 -- 172.21.15.16:0/492161234 shutdown_connections 2024-09-19T07:43:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.046+0000 7fa1c8d3b640 1 -- 172.21.15.16:0/492161234 wait complete. 2024-09-19T07:43:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.047+0000 7fa1c8d3b640 1 Processor -- start 2024-09-19T07:43:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.047+0000 7fa1c8d3b640 1 -- start start 2024-09-19T07:43:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.048+0000 7fa1c8d3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c40d0070 0x7fa1c41549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.048+0000 7fa1c8d3b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1c40d1940 0x7fa1c41f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.048+0000 7fa1c8d3b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1c415f0d0 0x7fa1c41f5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.048+0000 7fa1c8d3b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c40d47f0 con 0x7fa1c40d0070 2024-09-19T07:43:42.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.048+0000 7fa1c8d3b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c40d4670 con 0x7fa1c415f0d0 2024-09-19T07:43:42.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.048+0000 7fa1c8d3b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c40d4970 con 0x7fa1c40d1940 2024-09-19T07:43:42.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.048+0000 7fa1c259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c40d0070 0x7fa1c41549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.048+0000 7fa1c259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c40d0070 0x7fa1c41549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56146/0 (socket says 172.21.15.16:56146) 2024-09-19T07:43:42.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.048+0000 7fa1c259c640 1 -- 172.21.15.16:0/2218071057 learned_addr learned my addr 172.21.15.16:0/2218071057 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:42.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.048+0000 7fa1c2d9d640 1 --2- 172.21.15.16:0/2218071057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1c415f0d0 0x7fa1c41f5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.048+0000 7fa1c1d9b640 1 --2- 172.21.15.16:0/2218071057 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1c40d1940 0x7fa1c41f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.048+0000 7fa1c259c640 1 -- 172.21.15.16:0/2218071057 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1c40d1940 msgr2=0x7fa1c41f7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.048+0000 7fa1c259c640 1 --2- 172.21.15.16:0/2218071057 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1c40d1940 0x7fa1c41f7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.048+0000 7fa1c259c640 1 -- 172.21.15.16:0/2218071057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1c415f0d0 msgr2=0x7fa1c41f5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.048+0000 7fa1c259c640 1 --2- 172.21.15.16:0/2218071057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1c415f0d0 0x7fa1c41f5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.048+0000 7fa1c259c640 1 -- 172.21.15.16:0/2218071057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c41f5860 con 0x7fa1c40d0070 2024-09-19T07:43:42.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.048+0000 7fa1c2d9d640 1 --2- 172.21.15.16:0/2218071057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1c415f0d0 0x7fa1c41f5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:42.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.048+0000 7fa1c259c640 1 --2- 172.21.15.16:0/2218071057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c40d0070 0x7fa1c41549d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400e9b0 tx=0x7fa1b400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:42.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.049+0000 7fa1b2ffd640 1 -- 172.21.15.16:0/2218071057 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4017070 con 0x7fa1c40d0070 2024-09-19T07:43:42.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.049+0000 7fa1b2ffd640 1 -- 172.21.15.16:0/2218071057 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1b400c9a0 con 0x7fa1c40d0070 2024-09-19T07:43:42.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.049+0000 7fa1c8d3b640 1 -- 172.21.15.16:0/2218071057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c40b7340 con 0x7fa1c40d0070 2024-09-19T07:43:42.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.049+0000 7fa1b2ffd640 1 -- 172.21.15.16:0/2218071057 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b400cd90 con 0x7fa1c40d0070 2024-09-19T07:43:42.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.049+0000 7fa1c8d3b640 1 -- 172.21.15.16:0/2218071057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c40b78c0 con 0x7fa1c40d0070 2024-09-19T07:43:42.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.050+0000 7fa1c8d3b640 1 -- 172.21.15.16:0/2218071057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c40d1400 con 0x7fa1c40d0070 2024-09-19T07:43:42.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.050+0000 7fa1b2ffd640 1 -- 172.21.15.16:0/2218071057 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa1b401a070 con 0x7fa1c40d0070 2024-09-19T07:43:42.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.051+0000 7fa1b2ffd640 1 --2- 172.21.15.16:0/2218071057 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa1a406c2b0 0x7fa1a406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.051+0000 7fa1b2ffd640 1 -- 172.21.15.16:0/2218071057 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa1b408b720 con 0x7fa1c40d0070 2024-09-19T07:43:42.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.051+0000 7fa1c1d9b640 1 --2- 172.21.15.16:0/2218071057 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa1a406c2b0 0x7fa1a406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.051+0000 7fa1c1d9b640 1 --2- 172.21.15.16:0/2218071057 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa1a406c2b0 0x7fa1a406e770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40d3430 tx=0x7fa1b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:42.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.055+0000 7fa1b2ffd640 1 -- 172.21.15.16:0/2218071057 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa1b4059be0 con 0x7fa1c40d0070 2024-09-19T07:43:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.231+0000 7fa1c8d3b640 1 -- 172.21.15.16:0/2218071057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa1c406cf30 con 0x7fa1c40d0070 2024-09-19T07:43:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.231+0000 7fa1b2ffd640 1 -- 172.21.15.16:0/2218071057 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fa1b40594e0 con 0x7fa1c40d0070 2024-09-19T07:43:42.233 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-19T07:43:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.233+0000 7fa1c8d3b640 1 -- 172.21.15.16:0/2218071057 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa1a406c2b0 msgr2=0x7fa1a406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.233+0000 7fa1c8d3b640 1 --2- 172.21.15.16:0/2218071057 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa1a406c2b0 0x7fa1a406e770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40d3430 tx=0x7fa1b8008040 comp rx=0 tx=0).stop 2024-09-19T07:43:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.233+0000 7fa1c8d3b640 1 -- 172.21.15.16:0/2218071057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c40d0070 msgr2=0x7fa1c41549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.233+0000 7fa1c8d3b640 1 --2- 172.21.15.16:0/2218071057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c40d0070 0x7fa1c41549d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400e9b0 tx=0x7fa1b400ee70 comp rx=0 tx=0).stop 2024-09-19T07:43:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.233+0000 7fa1c8d3b640 1 -- 172.21.15.16:0/2218071057 shutdown_connections 2024-09-19T07:43:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.233+0000 7fa1c8d3b640 1 --2- 172.21.15.16:0/2218071057 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa1a406c2b0 0x7fa1a406e770 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.233+0000 7fa1c8d3b640 1 --2- 172.21.15.16:0/2218071057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1c415f0d0 0x7fa1c41f5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.233+0000 7fa1c8d3b640 1 --2- 172.21.15.16:0/2218071057 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1c40d1940 0x7fa1c41f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.233+0000 7fa1c8d3b640 1 --2- 172.21.15.16:0/2218071057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c40d0070 0x7fa1c41549d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.233+0000 7fa1c8d3b640 1 -- 172.21.15.16:0/2218071057 >> 172.21.15.16:0/2218071057 conn(0x7fa1c409d760 msgr2=0x7fa1c406dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.233+0000 7fa1c8d3b640 1 -- 172.21.15.16:0/2218071057 shutdown_connections 2024-09-19T07:43:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.233+0000 7fa1c8d3b640 1 -- 172.21.15.16:0/2218071057 wait complete. 2024-09-19T07:43:42.246 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T07:43:42.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.351+0000 7fd63dab7640 1 Processor -- start 2024-09-19T07:43:42.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.351+0000 7fd63dab7640 1 -- start start 2024-09-19T07:43:42.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.352+0000 7fd63dab7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6381619b0 0x7fd6380d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.352+0000 7fd63dab7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd638161d90 0x7fd6380d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.352+0000 7fd63dab7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd63806c990 con 0x7fd6380b0a80 2024-09-19T07:43:42.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.352+0000 7fd63dab7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fd6380d0ec0 con 0x7fd638161d90 2024-09-19T07:43:42.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.352+0000 7fd63dab7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd6380cec10 con 0x7fd6381619b0 2024-09-19T07:43:42.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.352+0000 7fd636ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd6380b0a80 0x7fd6380cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38384/0 (socket says 172.21.15.16:38384) 2024-09-19T07:43:42.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.352+0000 7fd636ffd640 1 -- 172.21.15.16:0/878711873 learned_addr learned my addr 172.21.15.16:0/878711873 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:42.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.352+0000 7fd637fff640 1 --2- 172.21.15.16:0/878711873 >> v2:172.21.15.146:3300/0 conn(0x7fd638161d90 0x7fd6380d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.352+0000 7fd6377fe640 1 --2- 172.21.15.16:0/878711873 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6381619b0 0x7fd6380d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.352+0000 7fd6367fc640 1 -- 172.21.15.16:0/878711873 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 515628540 0 0) 0x7fd63806c990 con 0x7fd6380b0a80 2024-09-19T07:43:42.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.352+0000 7fd6367fc640 1 -- 172.21.15.16:0/878711873 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd620003520 con 0x7fd6380b0a80 2024-09-19T07:43:42.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.353+0000 7fd637fff640 1 -- 172.21.15.16:0/878711873 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6381619b0 msgr2=0x7fd6380d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:42.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.353+0000 7fd637fff640 1 --2- 172.21.15.16:0/878711873 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6381619b0 0x7fd6380d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.353+0000 7fd6367fc640 1 -- 172.21.15.16:0/878711873 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 333089304 0 0) 0x7fd620003520 con 0x7fd6380b0a80 2024-09-19T07:43:42.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.353+0000 7fd637fff640 1 -- 172.21.15.16:0/878711873 >> v1:172.21.15.16:6789/0 conn(0x7fd6380b0a80 legacy=0x7fd6380cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.353+0000 7fd637fff640 1 -- 172.21.15.16:0/878711873 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6380ce930 con 0x7fd638161d90 2024-09-19T07:43:42.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.353+0000 7fd6377fe640 1 --2- 172.21.15.16:0/878711873 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6381619b0 0x7fd6380d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.353+0000 7fd637fff640 1 --2- 172.21.15.16:0/878711873 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd638161d90 0x7fd6380d0980 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd630009d90 tx=0x7fd63002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb69fdf2499a0190 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.354+0000 7fd6367fc640 1 -- 172.21.15.16:0/878711873 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c001670 con 0x7fd638161d90 2024-09-19T07:43:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.354+0000 7fd6367fc640 1 -- 172.21.15.16:0/878711873 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd630007540 con 0x7fd638161d90 2024-09-19T07:43:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.354+0000 7fd6367fc640 1 -- 172.21.15.16:0/878711873 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd630007910 con 0x7fd638161d90 2024-09-19T07:43:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.354+0000 7fd63dab7640 1 -- 172.21.15.16:0/878711873 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd638161d90 msgr2=0x7fd6380d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.354+0000 7fd63dab7640 1 --2- 172.21.15.16:0/878711873 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd638161d90 0x7fd6380d0980 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd630009d90 tx=0x7fd63002a5f0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.354+0000 7fd63dab7640 1 -- 172.21.15.16:0/878711873 shutdown_connections 2024-09-19T07:43:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.354+0000 7fd63dab7640 1 --2- 172.21.15.16:0/878711873 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd638161d90 0x7fd6380d0980 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.354+0000 7fd63dab7640 1 --2- 172.21.15.16:0/878711873 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6381619b0 0x7fd6380d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.354+0000 7fd63dab7640 1 -- 172.21.15.16:0/878711873 >> 172.21.15.16:0/878711873 conn(0x7fd63809a9f0 msgr2=0x7fd6380b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.354+0000 7fd63dab7640 1 -- 172.21.15.16:0/878711873 shutdown_connections 2024-09-19T07:43:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.354+0000 7fd63dab7640 1 -- 172.21.15.16:0/878711873 wait complete. 2024-09-19T07:43:42.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.355+0000 7fd63dab7640 1 Processor -- start 2024-09-19T07:43:42.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.356+0000 7fd63dab7640 1 -- start start 2024-09-19T07:43:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.356+0000 7fd63dab7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6380b0a80 0x7fd6380b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.356+0000 7fd63dab7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6381619b0 0x7fd6380b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.356+0000 7fd63dab7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd638161d90 0x7fd638156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.356+0000 7fd63dab7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd63806c620 con 0x7fd638161d90 2024-09-19T07:43:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.356+0000 7fd63dab7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd6380b0820 con 0x7fd6381619b0 2024-09-19T07:43:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.356+0000 7fd63dab7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd6380ccce0 con 0x7fd6380b0a80 2024-09-19T07:43:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.356+0000 7fd637fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd638161d90 0x7fd638156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.356+0000 7fd637fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd638161d90 0x7fd638156640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56162/0 (socket says 172.21.15.16:56162) 2024-09-19T07:43:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.356+0000 7fd637fff640 1 -- 172.21.15.16:0/1295163664 learned_addr learned my addr 172.21.15.16:0/1295163664 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.357+0000 7fd636ffd640 1 --2- 172.21.15.16:0/1295163664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6381619b0 0x7fd6380b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.357+0000 7fd6377fe640 1 --2- 172.21.15.16:0/1295163664 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6380b0a80 0x7fd6380b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.357+0000 7fd637fff640 1 -- 172.21.15.16:0/1295163664 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6380b0a80 msgr2=0x7fd6380b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.357+0000 7fd637fff640 1 --2- 172.21.15.16:0/1295163664 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6380b0a80 0x7fd6380b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.357+0000 7fd637fff640 1 -- 172.21.15.16:0/1295163664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6381619b0 msgr2=0x7fd6380b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.357+0000 7fd637fff640 1 --2- 172.21.15.16:0/1295163664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6381619b0 0x7fd6380b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.357+0000 7fd637fff640 1 -- 172.21.15.16:0/1295163664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6380b19d0 con 0x7fd638161d90 2024-09-19T07:43:42.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.357+0000 7fd637fff640 1 --2- 172.21.15.16:0/1295163664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd638161d90 0x7fd638156640 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd630009d90 tx=0x7fd630004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:42.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.357+0000 7fd61ffff640 1 -- 172.21.15.16:0/1295163664 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63003f070 con 0x7fd638161d90 2024-09-19T07:43:42.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.357+0000 7fd61ffff640 1 -- 172.21.15.16:0/1295163664 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd630031b40 con 0x7fd638161d90 2024-09-19T07:43:42.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.357+0000 7fd63dab7640 1 -- 172.21.15.16:0/1295163664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6380b1cb0 con 0x7fd638161d90 2024-09-19T07:43:42.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.357+0000 7fd61ffff640 1 -- 172.21.15.16:0/1295163664 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd630007e90 con 0x7fd638161d90 2024-09-19T07:43:42.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.357+0000 7fd63dab7640 1 -- 172.21.15.16:0/1295163664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6380b2270 con 0x7fd638161d90 2024-09-19T07:43:42.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.358+0000 7fd63dab7640 1 -- 172.21.15.16:0/1295163664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6380cd330 con 0x7fd638161d90 2024-09-19T07:43:42.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.359+0000 7fd61ffff640 1 -- 172.21.15.16:0/1295163664 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd63003b410 con 0x7fd638161d90 2024-09-19T07:43:42.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.359+0000 7fd61ffff640 1 --2- 172.21.15.16:0/1295163664 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd62406c2b0 0x7fd62406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.359+0000 7fd61ffff640 1 -- 172.21.15.16:0/1295163664 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd6300ac980 con 0x7fd638161d90 2024-09-19T07:43:42.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.359+0000 7fd6377fe640 1 --2- 172.21.15.16:0/1295163664 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd62406c2b0 0x7fd62406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.360+0000 7fd6377fe640 1 --2- 172.21.15.16:0/1295163664 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd62406c2b0 0x7fd62406e770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd628009560 tx=0x7fd628009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:42.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.363+0000 7fd61ffff640 1 -- 172.21.15.16:0/1295163664 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd63007a760 con 0x7fd638161d90 2024-09-19T07:43:42.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.541+0000 7fd63dab7640 1 -- 172.21.15.16:0/1295163664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd638160050 con 0x7fd638161d90 2024-09-19T07:43:42.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.541+0000 7fd61ffff640 1 -- 172.21.15.16:0/1295163664 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fd63007ea60 con 0x7fd638161d90 2024-09-19T07:43:42.543 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-19T07:43:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.543+0000 7fd63dab7640 1 -- 172.21.15.16:0/1295163664 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd62406c2b0 msgr2=0x7fd62406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.543+0000 7fd63dab7640 1 --2- 172.21.15.16:0/1295163664 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd62406c2b0 0x7fd62406e770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd628009560 tx=0x7fd628009190 comp rx=0 tx=0).stop 2024-09-19T07:43:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.543+0000 7fd63dab7640 1 -- 172.21.15.16:0/1295163664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd638161d90 msgr2=0x7fd638156640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.543+0000 7fd63dab7640 1 --2- 172.21.15.16:0/1295163664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd638161d90 0x7fd638156640 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd630009d90 tx=0x7fd630004a40 comp rx=0 tx=0).stop 2024-09-19T07:43:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.543+0000 7fd63dab7640 1 -- 172.21.15.16:0/1295163664 shutdown_connections 2024-09-19T07:43:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.543+0000 7fd63dab7640 1 --2- 172.21.15.16:0/1295163664 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd62406c2b0 0x7fd62406e770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.543+0000 7fd63dab7640 1 --2- 172.21.15.16:0/1295163664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd638161d90 0x7fd638156640 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.543+0000 7fd63dab7640 1 --2- 172.21.15.16:0/1295163664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6381619b0 0x7fd6380b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.543+0000 7fd63dab7640 1 --2- 172.21.15.16:0/1295163664 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6380b0a80 0x7fd6380b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.543+0000 7fd63dab7640 1 -- 172.21.15.16:0/1295163664 >> 172.21.15.16:0/1295163664 conn(0x7fd63809a9f0 msgr2=0x7fd6380d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:42.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.543+0000 7fd63dab7640 1 -- 172.21.15.16:0/1295163664 shutdown_connections 2024-09-19T07:43:42.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.543+0000 7fd63dab7640 1 -- 172.21.15.16:0/1295163664 wait complete. 2024-09-19T07:43:42.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T07:43:42.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.659+0000 7f9f2d823640 1 Processor -- start 2024-09-19T07:43:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.659+0000 7f9f2d823640 1 -- start start 2024-09-19T07:43:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.660+0000 7f9f2d823640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f280d2840 0x7f9f28141b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.660+0000 7f9f2d823640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f28161f10 con 0x7f9f28141470 2024-09-19T07:43:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.660+0000 7f9f2d823640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f28141850 con 0x7f9f2806d580 2024-09-19T07:43:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.660+0000 7f9f2d823640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9f281621b0 con 0x7f9f280d2840 2024-09-19T07:43:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.660+0000 7f9f277fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f280d2840 0x7f9f28141b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.660+0000 7f9f26ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9f28141470 0x7f9f2806ce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37722/0 (socket says 172.21.15.16:37722) 2024-09-19T07:43:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.660+0000 7f9f26ffd640 1 -- 172.21.15.16:0/2070779794 learned_addr learned my addr 172.21.15.16:0/2070779794 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.660+0000 7f9f277fe640 1 -- 172.21.15.16:0/2070779794 >> v1:172.21.15.146:6789/0 conn(0x7f9f2806d580 legacy=0x7f9f280d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.660+0000 7f9f277fe640 1 -- 172.21.15.16:0/2070779794 >> v1:172.21.15.16:6789/0 conn(0x7f9f28141470 legacy=0x7f9f2806ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.660+0000 7f9f277fe640 1 -- 172.21.15.16:0/2070779794 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f28162490 con 0x7f9f280d2840 2024-09-19T07:43:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.660+0000 7f9f25ffb640 1 -- 172.21.15.16:0/2070779794 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3083854465 0 0) 0x7f9f28161f10 con 0x7f9f28141470 2024-09-19T07:43:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.661+0000 7f9f277fe640 1 --2- 172.21.15.16:0/2070779794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f280d2840 0x7f9f28141b70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9f200097a0 tx=0x7f9f2002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467a3d816cd05ad3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.661+0000 7f9f25ffb640 1 -- 172.21.15.16:0/2070779794 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20037070 con 0x7f9f280d2840 2024-09-19T07:43:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.661+0000 7f9f25ffb640 1 -- 172.21.15.16:0/2070779794 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f20032440 con 0x7f9f280d2840 2024-09-19T07:43:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.661+0000 7f9f25ffb640 1 -- 172.21.15.16:0/2070779794 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20032810 con 0x7f9f280d2840 2024-09-19T07:43:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.662+0000 7f9f2d823640 1 -- 172.21.15.16:0/2070779794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f280d2840 msgr2=0x7f9f28141b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.662+0000 7f9f2d823640 1 --2- 172.21.15.16:0/2070779794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f280d2840 0x7f9f28141b70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9f200097a0 tx=0x7f9f2002a230 comp rx=0 tx=0).stop 2024-09-19T07:43:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.662+0000 7f9f2d823640 1 -- 172.21.15.16:0/2070779794 shutdown_connections 2024-09-19T07:43:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.662+0000 7f9f2d823640 1 --2- 172.21.15.16:0/2070779794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f280d2840 0x7f9f28141b70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.662+0000 7f9f2d823640 1 -- 172.21.15.16:0/2070779794 >> 172.21.15.16:0/2070779794 conn(0x7f9f280b4040 msgr2=0x7f9f280b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.662+0000 7f9f2d823640 1 -- 172.21.15.16:0/2070779794 shutdown_connections 2024-09-19T07:43:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.662+0000 7f9f2d823640 1 -- 172.21.15.16:0/2070779794 wait complete. 2024-09-19T07:43:42.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.663+0000 7f9f2d823640 1 Processor -- start 2024-09-19T07:43:42.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.663+0000 7f9f2d823640 1 -- start start 2024-09-19T07:43:42.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.664+0000 7f9f2d823640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f2806d580 0x7f9f2815dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.664+0000 7f9f2d823640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f280d2840 0x7f9f28160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.664+0000 7f9f2d823640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f28141470 0x7f9f28161050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.664+0000 7f9f2d823640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f28162d30 con 0x7f9f280d2840 2024-09-19T07:43:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.664+0000 7f9f2d823640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9f28162a50 con 0x7f9f2806d580 2024-09-19T07:43:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.664+0000 7f9f2d823640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2815ebd0 con 0x7f9f28141470 2024-09-19T07:43:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.664+0000 7f9f267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f280d2840 0x7f9f28160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.664+0000 7f9f267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f280d2840 0x7f9f28160b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35106/0 (socket says 172.21.15.16:35106) 2024-09-19T07:43:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.664+0000 7f9f267fc640 1 -- 172.21.15.16:0/324478945 learned_addr learned my addr 172.21.15.16:0/324478945 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.664+0000 7f9f26ffd640 1 --2- 172.21.15.16:0/324478945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f2806d580 0x7f9f2815dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.664+0000 7f9f277fe640 1 --2- 172.21.15.16:0/324478945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f28141470 0x7f9f28161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.664+0000 7f9f267fc640 1 -- 172.21.15.16:0/324478945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f28141470 msgr2=0x7f9f28161050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.664+0000 7f9f267fc640 1 --2- 172.21.15.16:0/324478945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f28141470 0x7f9f28161050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.664+0000 7f9f267fc640 1 -- 172.21.15.16:0/324478945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f2806d580 msgr2=0x7f9f2815dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.664+0000 7f9f267fc640 1 --2- 172.21.15.16:0/324478945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f2806d580 0x7f9f2815dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.664+0000 7f9f267fc640 1 -- 172.21.15.16:0/324478945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f280d1aa0 con 0x7f9f280d2840 2024-09-19T07:43:42.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.664+0000 7f9f26ffd640 1 --2- 172.21.15.16:0/324478945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f2806d580 0x7f9f2815dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:42.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.665+0000 7f9f267fc640 1 --2- 172.21.15.16:0/324478945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f280d2840 0x7f9f28160b10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000c430 tx=0x7f9f1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:42.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.665+0000 7f9f1ffff640 1 -- 172.21.15.16:0/324478945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10017070 con 0x7f9f280d2840 2024-09-19T07:43:42.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.665+0000 7f9f1ffff640 1 -- 172.21.15.16:0/324478945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f100052d0 con 0x7f9f280d2840 2024-09-19T07:43:42.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.665+0000 7f9f2d823640 1 -- 172.21.15.16:0/324478945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f280d1d80 con 0x7f9f280d2840 2024-09-19T07:43:42.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.665+0000 7f9f1ffff640 1 -- 172.21.15.16:0/324478945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10003830 con 0x7f9f280d2840 2024-09-19T07:43:42.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.665+0000 7f9f2d823640 1 -- 172.21.15.16:0/324478945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2815e270 con 0x7f9f280d2840 2024-09-19T07:43:42.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.666+0000 7f9f2d823640 1 -- 172.21.15.16:0/324478945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f28141b70 con 0x7f9f280d2840 2024-09-19T07:43:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.666+0000 7f9f1ffff640 1 -- 172.21.15.16:0/324478945 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9f1001a070 con 0x7f9f280d2840 2024-09-19T07:43:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.667+0000 7f9f1ffff640 1 --2- 172.21.15.16:0/324478945 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9f0806c2b0 0x7f9f0806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.667+0000 7f9f1ffff640 1 -- 172.21.15.16:0/324478945 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9f1008aa20 con 0x7f9f280d2840 2024-09-19T07:43:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.667+0000 7f9f26ffd640 1 --2- 172.21.15.16:0/324478945 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9f0806c2b0 0x7f9f0806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.668+0000 7f9f26ffd640 1 --2- 172.21.15.16:0/324478945 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9f0806c2b0 0x7f9f0806e770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9f180012e0 tx=0x7f9f18009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:42.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.671+0000 7f9f1ffff640 1 -- 172.21.15.16:0/324478945 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9f10058890 con 0x7f9f280d2840 2024-09-19T07:43:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.845+0000 7f9f2d823640 1 -- 172.21.15.16:0/324478945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9f280756d0 con 0x7f9f280d2840 2024-09-19T07:43:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.845+0000 7f9f1ffff640 1 -- 172.21.15.16:0/324478945 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f9f1005cb90 con 0x7f9f280d2840 2024-09-19T07:43:42.847 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-19T07:43:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.847+0000 7f9f2d823640 1 -- 172.21.15.16:0/324478945 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9f0806c2b0 msgr2=0x7f9f0806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.847+0000 7f9f2d823640 1 --2- 172.21.15.16:0/324478945 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9f0806c2b0 0x7f9f0806e770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9f180012e0 tx=0x7f9f18009040 comp rx=0 tx=0).stop 2024-09-19T07:43:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.847+0000 7f9f2d823640 1 -- 172.21.15.16:0/324478945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f280d2840 msgr2=0x7f9f28160b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.847+0000 7f9f2d823640 1 --2- 172.21.15.16:0/324478945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f280d2840 0x7f9f28160b10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000c430 tx=0x7f9f1000c8f0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.847+0000 7f9f2d823640 1 -- 172.21.15.16:0/324478945 shutdown_connections 2024-09-19T07:43:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.847+0000 7f9f2d823640 1 --2- 172.21.15.16:0/324478945 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9f0806c2b0 0x7f9f0806e770 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.847+0000 7f9f2d823640 1 --2- 172.21.15.16:0/324478945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f28141470 0x7f9f28161050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.847+0000 7f9f2d823640 1 --2- 172.21.15.16:0/324478945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f280d2840 0x7f9f28160b10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.847+0000 7f9f2d823640 1 --2- 172.21.15.16:0/324478945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f2806d580 0x7f9f2815dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.847+0000 7f9f2d823640 1 -- 172.21.15.16:0/324478945 >> 172.21.15.16:0/324478945 conn(0x7f9f280b4040 msgr2=0x7f9f2806ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.847+0000 7f9f2d823640 1 -- 172.21.15.16:0/324478945 shutdown_connections 2024-09-19T07:43:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.847+0000 7f9f2d823640 1 -- 172.21.15.16:0/324478945 wait complete. 2024-09-19T07:43:42.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-19T07:43:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.964+0000 7f63720d0640 1 Processor -- start 2024-09-19T07:43:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.964+0000 7f63720d0640 1 -- start start 2024-09-19T07:43:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.965+0000 7f63720d0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f636c15e680 0x7f636c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.965+0000 7f63720d0640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f636c0d0010 0x7f636c0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.965+0000 7f63720d0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f636c0d1880 con 0x7f636c15e2a0 2024-09-19T07:43:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.965+0000 7f63720d0640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f636c0d1b60 con 0x7f636c0d0010 2024-09-19T07:43:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.965+0000 7f63720d0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f636c0cf2d0 con 0x7f636c15e680 2024-09-19T07:43:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.965+0000 7f636b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f636c15e2a0 0x7f636c06d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37734/0 (socket says 172.21.15.16:37734) 2024-09-19T07:43:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.965+0000 7f636b7fe640 1 -- 172.21.15.16:0/1653641370 learned_addr learned my addr 172.21.15.16:0/1653641370 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.965+0000 7f636affd640 1 --2- 172.21.15.16:0/1653641370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f636c15e680 0x7f636c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.965+0000 7f636bfff640 1 --2- 172.21.15.16:0/1653641370 >> v2:172.21.15.146:3300/0 conn(0x7f636c0d0010 0x7f636c0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.965+0000 7f636a7fc640 1 -- 172.21.15.16:0/1653641370 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 25058533 0 0) 0x7f636c0d1880 con 0x7f636c15e2a0 2024-09-19T07:43:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.965+0000 7f636a7fc640 1 -- 172.21.15.16:0/1653641370 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6354003520 con 0x7f636c15e2a0 2024-09-19T07:43:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.965+0000 7f636a7fc640 1 -- 172.21.15.16:0/1653641370 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4270456010 0 0) 0x7f6354003520 con 0x7f636c15e2a0 2024-09-19T07:43:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.965+0000 7f636a7fc640 1 -- 172.21.15.16:0/1653641370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f636c15e680 msgr2=0x7f636c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.965+0000 7f636a7fc640 1 --2- 172.21.15.16:0/1653641370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f636c15e680 0x7f636c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.965+0000 7f636a7fc640 1 -- 172.21.15.16:0/1653641370 >> v2:172.21.15.146:3300/0 conn(0x7f636c0d0010 msgr2=0x7f636c0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.965+0000 7f636a7fc640 1 --2- 172.21.15.16:0/1653641370 >> v2:172.21.15.146:3300/0 conn(0x7f636c0d0010 0x7f636c0d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.965+0000 7f636a7fc640 1 -- 172.21.15.16:0/1653641370 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c0cf5b0 con 0x7f636c15e2a0 2024-09-19T07:43:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.965+0000 7f636a7fc640 1 -- 172.21.15.16:0/1653641370 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f635c001670 con 0x7f636c15e2a0 2024-09-19T07:43:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.965+0000 7f636a7fc640 1 -- 172.21.15.16:0/1653641370 >> v1:172.21.15.16:6789/0 conn(0x7f636c15e2a0 legacy=0x7f636c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.966+0000 7f636a7fc640 1 --2- 172.21.15.16:0/1653641370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f636c15e680 0x7f63540039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.966+0000 7f636a7fc640 1 --2- 172.21.15.16:0/1653641370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6354003f30 0x7f6354004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.966+0000 7f636a7fc640 1 --2- 172.21.15.16:0/1653641370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6354004d40 0x7f63540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.966+0000 7f636a7fc640 1 -- 172.21.15.16:0/1653641370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f636c0d1b60 con 0x7f6354004d40 2024-09-19T07:43:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.966+0000 7f636a7fc640 1 -- 172.21.15.16:0/1653641370 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f636c0cf2d0 con 0x7f6354003f30 2024-09-19T07:43:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.966+0000 7f636a7fc640 1 -- 172.21.15.16:0/1653641370 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f635400ba10 con 0x7f636c15e680 2024-09-19T07:43:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.966+0000 7f636bfff640 1 --2- 172.21.15.16:0/1653641370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6354004d40 0x7f63540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.966+0000 7f636b7fe640 1 --2- 172.21.15.16:0/1653641370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f636c15e680 0x7f63540039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.966+0000 7f636affd640 1 --2- 172.21.15.16:0/1653641370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6354003f30 0x7f6354004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.966+0000 7f636bfff640 1 -- 172.21.15.16:0/1653641370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f636c15e680 msgr2=0x7f63540039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.966+0000 7f636bfff640 1 --2- 172.21.15.16:0/1653641370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f636c15e680 0x7f63540039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.966+0000 7f636bfff640 1 -- 172.21.15.16:0/1653641370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6354003f30 msgr2=0x7f6354004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:42.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.966+0000 7f636bfff640 1 --2- 172.21.15.16:0/1653641370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6354003f30 0x7f6354004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.966+0000 7f636bfff640 1 -- 172.21.15.16:0/1653641370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6354003520 con 0x7f6354004d40 2024-09-19T07:43:42.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.966+0000 7f636affd640 1 --2- 172.21.15.16:0/1653641370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6354003f30 0x7f6354004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:42.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.967+0000 7f636bfff640 1 --2- 172.21.15.16:0/1653641370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6354004d40 0x7f63540051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f636400e170 tx=0x7f6354005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd8a7e4148d9684 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:42.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.967+0000 7f636a7fc640 1 -- 172.21.15.16:0/1653641370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364034020 con 0x7f6354004d40 2024-09-19T07:43:42.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.967+0000 7f636a7fc640 1 -- 172.21.15.16:0/1653641370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6364030430 con 0x7f6354004d40 2024-09-19T07:43:42.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.967+0000 7f636a7fc640 1 -- 172.21.15.16:0/1653641370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364009c10 con 0x7f6354004d40 2024-09-19T07:43:42.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.967+0000 7f63720d0640 1 -- 172.21.15.16:0/1653641370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6354004d40 msgr2=0x7f63540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.967+0000 7f63720d0640 1 --2- 172.21.15.16:0/1653641370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6354004d40 0x7f63540051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f636400e170 tx=0x7f6354005830 comp rx=0 tx=0).stop 2024-09-19T07:43:42.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.967+0000 7f636bfff640 1 -- 172.21.15.16:0/1653641370 reap_dead start 2024-09-19T07:43:42.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.967+0000 7f63720d0640 1 -- 172.21.15.16:0/1653641370 shutdown_connections 2024-09-19T07:43:42.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.967+0000 7f63720d0640 1 -- 172.21.15.16:0/1653641370 >> 172.21.15.16:0/1653641370 conn(0x7f636c09d760 msgr2=0x7f636c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:42.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.967+0000 7f63720d0640 1 -- 172.21.15.16:0/1653641370 shutdown_connections 2024-09-19T07:43:42.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.967+0000 7f63720d0640 1 -- 172.21.15.16:0/1653641370 wait complete. 2024-09-19T07:43:42.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.969+0000 7f63720d0640 1 Processor -- start 2024-09-19T07:43:42.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.969+0000 7f63720d0640 1 -- start start 2024-09-19T07:43:42.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.969+0000 7f63720d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f636c09f940 0x7f636c1f8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.969+0000 7f63720d0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f636c15f550 0x7f636c15f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.969+0000 7f63720d0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f636c150410 0x7f636c150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.969+0000 7f63720d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f636c06dcb0 con 0x7f636c09f940 2024-09-19T07:43:42.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.969+0000 7f63720d0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f636c06db30 con 0x7f636c150410 2024-09-19T07:43:42.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.969+0000 7f63720d0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f636c0cfad0 con 0x7f636c15f550 2024-09-19T07:43:42.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.970+0000 7f636b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f636c09f940 0x7f636c1f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.970+0000 7f636b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f636c09f940 0x7f636c1f8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35114/0 (socket says 172.21.15.16:35114) 2024-09-19T07:43:42.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.970+0000 7f636b7fe640 1 -- 172.21.15.16:0/3570568467 learned_addr learned my addr 172.21.15.16:0/3570568467 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:42.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.970+0000 7f636bfff640 1 --2- 172.21.15.16:0/3570568467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f636c150410 0x7f636c150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.970+0000 7f636affd640 1 --2- 172.21.15.16:0/3570568467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f636c15f550 0x7f636c15f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.970+0000 7f636b7fe640 1 -- 172.21.15.16:0/3570568467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f636c15f550 msgr2=0x7f636c15f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.970+0000 7f636b7fe640 1 --2- 172.21.15.16:0/3570568467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f636c15f550 0x7f636c15f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.970+0000 7f636b7fe640 1 -- 172.21.15.16:0/3570568467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f636c150410 msgr2=0x7f636c150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:42.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.970+0000 7f636b7fe640 1 --2- 172.21.15.16:0/3570568467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f636c150410 0x7f636c150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:42.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.970+0000 7f636b7fe640 1 -- 172.21.15.16:0/3570568467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c1f8a70 con 0x7f636c09f940 2024-09-19T07:43:42.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.970+0000 7f636b7fe640 1 --2- 172.21.15.16:0/3570568467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f636c09f940 0x7f636c1f8530 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f635c0072c0 tx=0x7f635c007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:42.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.970+0000 7f6353fff640 1 -- 172.21.15.16:0/3570568467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c01a070 con 0x7f636c09f940 2024-09-19T07:43:42.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.970+0000 7f6353fff640 1 -- 172.21.15.16:0/3570568467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f635c00fa60 con 0x7f636c09f940 2024-09-19T07:43:42.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.970+0000 7f63720d0640 1 -- 172.21.15.16:0/3570568467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636c0c9f20 con 0x7f636c09f940 2024-09-19T07:43:42.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.970+0000 7f6353fff640 1 -- 172.21.15.16:0/3570568467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c00fe50 con 0x7f636c09f940 2024-09-19T07:43:42.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.970+0000 7f63720d0640 1 -- 172.21.15.16:0/3570568467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f636c0ca4a0 con 0x7f636c09f940 2024-09-19T07:43:42.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.971+0000 7f63720d0640 1 -- 172.21.15.16:0/3570568467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f636c06d420 con 0x7f636c09f940 2024-09-19T07:43:42.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.972+0000 7f6353fff640 1 -- 172.21.15.16:0/3570568467 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f635c01d070 con 0x7f636c09f940 2024-09-19T07:43:42.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.972+0000 7f6353fff640 1 --2- 172.21.15.16:0/3570568467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f635806c2e0 0x7f635806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:42.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.973+0000 7f6353fff640 1 -- 172.21.15.16:0/3570568467 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f635c08b1e0 con 0x7f636c09f940 2024-09-19T07:43:42.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.973+0000 7f636affd640 1 --2- 172.21.15.16:0/3570568467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f635806c2e0 0x7f635806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:42.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.973+0000 7f636affd640 1 --2- 172.21.15.16:0/3570568467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f635806c2e0 0x7f635806e7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f636c0d3c90 tx=0x7f6360008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:42.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:42.977+0000 7f6353fff640 1 -- 172.21.15.16:0/3570568467 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f636c1f8a70 con 0x7f636c09f940 2024-09-19T07:43:43.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.152+0000 7f63720d0640 1 -- 172.21.15.16:0/3570568467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f636c0c7c90 con 0x7f636c09f940 2024-09-19T07:43:43.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.152+0000 7f6353fff640 1 -- 172.21.15.16:0/3570568467 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f635c058fc0 con 0x7f636c09f940 2024-09-19T07:43:43.154 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-19T07:43:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.154+0000 7f63720d0640 1 -- 172.21.15.16:0/3570568467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f635806c2e0 msgr2=0x7f635806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.154+0000 7f63720d0640 1 --2- 172.21.15.16:0/3570568467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f635806c2e0 0x7f635806e7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f636c0d3c90 tx=0x7f6360008040 comp rx=0 tx=0).stop 2024-09-19T07:43:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.154+0000 7f63720d0640 1 -- 172.21.15.16:0/3570568467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f636c09f940 msgr2=0x7f636c1f8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.154+0000 7f63720d0640 1 --2- 172.21.15.16:0/3570568467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f636c09f940 0x7f636c1f8530 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f635c0072c0 tx=0x7f635c007780 comp rx=0 tx=0).stop 2024-09-19T07:43:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.154+0000 7f63720d0640 1 -- 172.21.15.16:0/3570568467 shutdown_connections 2024-09-19T07:43:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.154+0000 7f63720d0640 1 --2- 172.21.15.16:0/3570568467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f635806c2e0 0x7f635806e7a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.154+0000 7f63720d0640 1 --2- 172.21.15.16:0/3570568467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f636c150410 0x7f636c150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.154+0000 7f63720d0640 1 --2- 172.21.15.16:0/3570568467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f636c15f550 0x7f636c15f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.154+0000 7f63720d0640 1 --2- 172.21.15.16:0/3570568467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f636c09f940 0x7f636c1f8530 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.154+0000 7f63720d0640 1 -- 172.21.15.16:0/3570568467 >> 172.21.15.16:0/3570568467 conn(0x7f636c09d760 msgr2=0x7f636c0b97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.154+0000 7f63720d0640 1 -- 172.21.15.16:0/3570568467 shutdown_connections 2024-09-19T07:43:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.154+0000 7f63720d0640 1 -- 172.21.15.16:0/3570568467 wait complete. 2024-09-19T07:43:43.167 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2024-09-19T07:43:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.267+0000 7f4316eba640 1 Processor -- start 2024-09-19T07:43:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.267+0000 7f4316eba640 1 -- start start 2024-09-19T07:43:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.268+0000 7f4316eba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43100d0070 0x7f431015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.268+0000 7f4316eba640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f43100d4130 0x7f43100d45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.268+0000 7f4316eba640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43100d4b00 con 0x7f431015f0d0 2024-09-19T07:43:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.268+0000 7f4316eba640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f43100d1b10 con 0x7f43100d4130 2024-09-19T07:43:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.268+0000 7f4316eba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f43100cf2b0 con 0x7f43100d0070 2024-09-19T07:43:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.268+0000 7f4314c56640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f431015f0d0 0x7f43100d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37750/0 (socket says 172.21.15.16:37750) 2024-09-19T07:43:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.268+0000 7f4314c56640 1 -- 172.21.15.16:0/2613395411 learned_addr learned my addr 172.21.15.16:0/2613395411 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.268+0000 7f4315457640 1 --2- 172.21.15.16:0/2613395411 >> v2:172.21.15.146:3300/0 conn(0x7f43100d4130 0x7f43100d45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.268+0000 7f430ffff640 1 --2- 172.21.15.16:0/2613395411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43100d0070 0x7f431015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f430f7fe640 1 -- 172.21.15.16:0/2613395411 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1012848500 0 0) 0x7f43100d4b00 con 0x7f431015f0d0 2024-09-19T07:43:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f430f7fe640 1 -- 172.21.15.16:0/2613395411 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4300003520 con 0x7f431015f0d0 2024-09-19T07:43:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f430f7fe640 1 -- 172.21.15.16:0/2613395411 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 23297713 0 0) 0x7f4300003520 con 0x7f431015f0d0 2024-09-19T07:43:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f430f7fe640 1 -- 172.21.15.16:0/2613395411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43100d0070 msgr2=0x7f431015f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f430f7fe640 1 --2- 172.21.15.16:0/2613395411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43100d0070 0x7f431015f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f430f7fe640 1 -- 172.21.15.16:0/2613395411 >> v2:172.21.15.146:3300/0 conn(0x7f43100d4130 msgr2=0x7f43100d45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f430f7fe640 1 --2- 172.21.15.16:0/2613395411 >> v2:172.21.15.146:3300/0 conn(0x7f43100d4130 0x7f43100d45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f430f7fe640 1 -- 172.21.15.16:0/2613395411 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43100cf590 con 0x7f431015f0d0 2024-09-19T07:43:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f430f7fe640 1 -- 172.21.15.16:0/2613395411 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f4304001670 con 0x7f431015f0d0 2024-09-19T07:43:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f430f7fe640 1 -- 172.21.15.16:0/2613395411 >> v1:172.21.15.16:6789/0 conn(0x7f431015f0d0 legacy=0x7f43100d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f430f7fe640 1 --2- 172.21.15.16:0/2613395411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43000039b0 0x7f4300003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f430f7fe640 1 --2- 172.21.15.16:0/2613395411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4300004800 0x7f4300004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f430f7fe640 1 --2- 172.21.15.16:0/2613395411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43000051c0 0x7f430000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f430f7fe640 1 -- 172.21.15.16:0/2613395411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43100d1b10 con 0x7f43000051c0 2024-09-19T07:43:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f430f7fe640 1 -- 172.21.15.16:0/2613395411 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f43100cf2b0 con 0x7f43000039b0 2024-09-19T07:43:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f430f7fe640 1 -- 172.21.15.16:0/2613395411 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f430000bf50 con 0x7f4300004800 2024-09-19T07:43:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f4315457640 1 --2- 172.21.15.16:0/2613395411 >> v2:172.21.15.146:3300/0 conn(0x7f43100d4130 0x7f43100d45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f430ffff640 1 --2- 172.21.15.16:0/2613395411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43100d0070 0x7f431015f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f4315457640 1 --2- 172.21.15.16:0/2613395411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43000051c0 0x7f430000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f4314c56640 1 --2- 172.21.15.16:0/2613395411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43000039b0 0x7f4300003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f430ffff640 1 --2- 172.21.15.16:0/2613395411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4300004800 0x7f4300004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f4315457640 1 -- 172.21.15.16:0/2613395411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4300004800 msgr2=0x7f4300004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:43.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f4315457640 1 --2- 172.21.15.16:0/2613395411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4300004800 0x7f4300004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f4315457640 1 -- 172.21.15.16:0/2613395411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43000039b0 msgr2=0x7f4300003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:43.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f4315457640 1 --2- 172.21.15.16:0/2613395411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43000039b0 0x7f4300003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.269+0000 7f4315457640 1 -- 172.21.15.16:0/2613395411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4300003520 con 0x7f43000051c0 2024-09-19T07:43:43.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.270+0000 7f4315457640 1 --2- 172.21.15.16:0/2613395411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43000051c0 0x7f430000b9c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f430800e380 tx=0x7f4300005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31a4dccfb6dcd49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:43.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.270+0000 7f430f7fe640 1 -- 172.21.15.16:0/2613395411 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308009dc0 con 0x7f43000051c0 2024-09-19T07:43:43.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.270+0000 7f430f7fe640 1 -- 172.21.15.16:0/2613395411 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4308003b90 con 0x7f43000051c0 2024-09-19T07:43:43.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.270+0000 7f430f7fe640 1 -- 172.21.15.16:0/2613395411 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308004020 con 0x7f43000051c0 2024-09-19T07:43:43.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.271+0000 7f4316eba640 1 -- 172.21.15.16:0/2613395411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43000051c0 msgr2=0x7f430000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:43.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.271+0000 7f4316eba640 1 --2- 172.21.15.16:0/2613395411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43000051c0 0x7f430000b9c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f430800e380 tx=0x7f4300005830 comp rx=0 tx=0).stop 2024-09-19T07:43:43.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.271+0000 7f4315457640 1 -- 172.21.15.16:0/2613395411 reap_dead start 2024-09-19T07:43:43.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.271+0000 7f4316eba640 1 -- 172.21.15.16:0/2613395411 shutdown_connections 2024-09-19T07:43:43.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.271+0000 7f4316eba640 1 -- 172.21.15.16:0/2613395411 >> 172.21.15.16:0/2613395411 conn(0x7f431009d760 msgr2=0x7f431008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:43.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.271+0000 7f4316eba640 1 -- 172.21.15.16:0/2613395411 shutdown_connections 2024-09-19T07:43:43.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.271+0000 7f4316eba640 1 -- 172.21.15.16:0/2613395411 wait complete. 2024-09-19T07:43:43.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.272+0000 7f4316eba640 1 Processor -- start 2024-09-19T07:43:43.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.272+0000 7f4316eba640 1 -- start start 2024-09-19T07:43:43.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.273+0000 7f4316eba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43000051c0 0x7f43101f16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.273+0000 7f4316eba640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43100cc570 0x7f43100b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.273+0000 7f4316eba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4310159500 0x7f4310159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.273+0000 7f4316eba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43100cef60 con 0x7f43000051c0 2024-09-19T07:43:43.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.273+0000 7f4316eba640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f43100cfb50 con 0x7f43100cc570 2024-09-19T07:43:43.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.273+0000 7f4316eba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f431015f4b0 con 0x7f4310159500 2024-09-19T07:43:43.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.273+0000 7f4314c56640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43000051c0 0x7f43101f16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.273+0000 7f4314c56640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43000051c0 0x7f43101f16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35132/0 (socket says 172.21.15.16:35132) 2024-09-19T07:43:43.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.273+0000 7f4314c56640 1 -- 172.21.15.16:0/2614768467 learned_addr learned my addr 172.21.15.16:0/2614768467 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:43.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.273+0000 7f4315457640 1 --2- 172.21.15.16:0/2614768467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4310159500 0x7f4310159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.273+0000 7f430ffff640 1 --2- 172.21.15.16:0/2614768467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43100cc570 0x7f43100b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.273+0000 7f4314c56640 1 -- 172.21.15.16:0/2614768467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4310159500 msgr2=0x7f4310159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.273+0000 7f4314c56640 1 --2- 172.21.15.16:0/2614768467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4310159500 0x7f4310159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.273+0000 7f4314c56640 1 -- 172.21.15.16:0/2614768467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43100cc570 msgr2=0x7f43100b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.273+0000 7f4314c56640 1 --2- 172.21.15.16:0/2614768467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43100cc570 0x7f43100b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.273+0000 7f4314c56640 1 -- 172.21.15.16:0/2614768467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43101f7860 con 0x7f43000051c0 2024-09-19T07:43:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.273+0000 7f430ffff640 1 --2- 172.21.15.16:0/2614768467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43100cc570 0x7f43100b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.274+0000 7f4314c56640 1 --2- 172.21.15.16:0/2614768467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43000051c0 0x7f43101f16e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4304007340 tx=0x7f4304007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.274+0000 7f430d7fa640 1 -- 172.21.15.16:0/2614768467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430400f8e0 con 0x7f43000051c0 2024-09-19T07:43:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.274+0000 7f430d7fa640 1 -- 172.21.15.16:0/2614768467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4304018070 con 0x7f43000051c0 2024-09-19T07:43:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.274+0000 7f4316eba640 1 -- 172.21.15.16:0/2614768467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43101f7b40 con 0x7f43000051c0 2024-09-19T07:43:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.274+0000 7f430d7fa640 1 -- 172.21.15.16:0/2614768467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430400fcc0 con 0x7f43000051c0 2024-09-19T07:43:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.274+0000 7f4316eba640 1 -- 172.21.15.16:0/2614768467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43101f8100 con 0x7f43000051c0 2024-09-19T07:43:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.275+0000 7f4316eba640 1 -- 172.21.15.16:0/2614768467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4310159ec0 con 0x7f43000051c0 2024-09-19T07:43:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.275+0000 7f430d7fa640 1 -- 172.21.15.16:0/2614768467 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f430401d460 con 0x7f43000051c0 2024-09-19T07:43:43.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.276+0000 7f430d7fa640 1 --2- 172.21.15.16:0/2614768467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f42f406c2b0 0x7f42f406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.276+0000 7f430d7fa640 1 -- 172.21.15.16:0/2614768467 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f430408b7a0 con 0x7f43000051c0 2024-09-19T07:43:43.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.276+0000 7f430ffff640 1 --2- 172.21.15.16:0/2614768467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f42f406c2b0 0x7f42f406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.276+0000 7f430ffff640 1 --2- 172.21.15.16:0/2614768467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f42f406c2b0 0x7f42f406e770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f43100b6b00 tx=0x7f42fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:43.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.280+0000 7f430d7fa640 1 -- 172.21.15.16:0/2614768467 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f430400fe60 con 0x7f43000051c0 2024-09-19T07:43:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:43.522+0000 7f4316eba640 1 -- 172.21.15.16:0/2614768467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) -- 0x7f431006e160 con 0x7f43000051c0 2024-09-19T07:43:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.390+0000 7f430d7fa640 1 -- 172.21.15.16:0/2614768467 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v59) ==== 168+0+0 (secure 0 0 0) 0x7f431006e160 con 0x7f43000051c0 2024-09-19T07:43:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.393+0000 7f4316eba640 1 -- 172.21.15.16:0/2614768467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f42f406c2b0 msgr2=0x7f42f406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.393+0000 7f4316eba640 1 --2- 172.21.15.16:0/2614768467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f42f406c2b0 0x7f42f406e770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f43100b6b00 tx=0x7f42fc008040 comp rx=0 tx=0).stop 2024-09-19T07:43:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.393+0000 7f4316eba640 1 -- 172.21.15.16:0/2614768467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43000051c0 msgr2=0x7f43101f16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.393+0000 7f4316eba640 1 --2- 172.21.15.16:0/2614768467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43000051c0 0x7f43101f16e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4304007340 tx=0x7f4304007800 comp rx=0 tx=0).stop 2024-09-19T07:43:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.393+0000 7f4316eba640 1 -- 172.21.15.16:0/2614768467 shutdown_connections 2024-09-19T07:43:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.393+0000 7f4316eba640 1 --2- 172.21.15.16:0/2614768467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f42f406c2b0 0x7f42f406e770 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.393+0000 7f4316eba640 1 --2- 172.21.15.16:0/2614768467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4310159500 0x7f4310159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.393+0000 7f4316eba640 1 --2- 172.21.15.16:0/2614768467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43100cc570 0x7f43100b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.393+0000 7f4316eba640 1 --2- 172.21.15.16:0/2614768467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43000051c0 0x7f43101f16e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.393+0000 7f4316eba640 1 -- 172.21.15.16:0/2614768467 >> 172.21.15.16:0/2614768467 conn(0x7f431009d760 msgr2=0x7f431015a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.393+0000 7f4316eba640 1 -- 172.21.15.16:0/2614768467 shutdown_connections 2024-09-19T07:43:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.393+0000 7f4316eba640 1 -- 172.21.15.16:0/2614768467 wait complete. 2024-09-19T07:43:44.408 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T07:43:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.519+0000 7f7d01142640 1 Processor -- start 2024-09-19T07:43:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.519+0000 7f7d01142640 1 -- start start 2024-09-19T07:43:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.520+0000 7f7d01142640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cfc15fcb0 0x7f7cfc0ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.520+0000 7f7d01142640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7cfc0cccd0 0x7f7cfc160090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.520+0000 7f7d01142640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cfc0d0560 con 0x7f7cfc0cd0f0 2024-09-19T07:43:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.520+0000 7f7d01142640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f7cfc0ceb10 con 0x7f7cfc0cccd0 2024-09-19T07:43:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.520+0000 7f7d01142640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7cfc0cffa0 con 0x7f7cfc15fcb0 2024-09-19T07:43:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.521+0000 7f7cfad9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cfc15fcb0 0x7f7cfc0ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.521+0000 7f7cfa59c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7cfc0cccd0 0x7f7cfc160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.521+0000 7f7cfb59e640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f7cfc0cd0f0 0x7f7cfc09b290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:32864/0 (socket says 172.21.15.16:32864) 2024-09-19T07:43:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.521+0000 7f7cfad9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cfc15fcb0 0x7f7cfc0ce5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:33708/0 (socket says 172.21.15.16:33708) 2024-09-19T07:43:44.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.521+0000 7f7cfb59e640 1 -- 172.21.15.16:0/4289464247 learned_addr learned my addr 172.21.15.16:0/4289464247 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.521+0000 7f7cfad9d640 1 -- 172.21.15.16:0/4289464247 >> v2:172.21.15.146:3300/0 conn(0x7f7cfc0cccd0 msgr2=0x7f7cfc160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.521+0000 7f7cfad9d640 1 --2- 172.21.15.16:0/4289464247 >> v2:172.21.15.146:3300/0 conn(0x7f7cfc0cccd0 0x7f7cfc160090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.521+0000 7f7cfad9d640 1 -- 172.21.15.16:0/4289464247 >> v1:172.21.15.146:6789/0 conn(0x7f7cfc0cd0f0 legacy=0x7f7cfc09b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.521+0000 7f7cfad9d640 1 -- 172.21.15.16:0/4289464247 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cfc0d0280 con 0x7f7cfc15fcb0 2024-09-19T07:43:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.523+0000 7f7cfad9d640 1 --2- 172.21.15.16:0/4289464247 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cfc15fcb0 0x7f7cfc0ce5d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7cec009d90 tx=0x7f7cec02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cb37623a154f180 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.523+0000 7f7cf9d9b640 1 -- 172.21.15.16:0/4289464247 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec007740 con 0x7f7cfc15fcb0 2024-09-19T07:43:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.523+0000 7f7cf9d9b640 1 -- 172.21.15.16:0/4289464247 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cec0078e0 con 0x7f7cfc15fcb0 2024-09-19T07:43:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.523+0000 7f7cf9d9b640 1 -- 172.21.15.16:0/4289464247 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec007cb0 con 0x7f7cfc15fcb0 2024-09-19T07:43:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.523+0000 7f7d01142640 1 -- 172.21.15.16:0/4289464247 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cfc15fcb0 msgr2=0x7f7cfc0ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:44.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.523+0000 7f7d01142640 1 --2- 172.21.15.16:0/4289464247 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cfc15fcb0 0x7f7cfc0ce5d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7cec009d90 tx=0x7f7cec02a660 comp rx=0 tx=0).stop 2024-09-19T07:43:44.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.523+0000 7f7d01142640 1 -- 172.21.15.16:0/4289464247 shutdown_connections 2024-09-19T07:43:44.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.523+0000 7f7d01142640 1 --2- 172.21.15.16:0/4289464247 >> v2:172.21.15.146:3300/0 conn(0x7f7cfc0cccd0 0x7f7cfc160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:44.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.523+0000 7f7d01142640 1 --2- 172.21.15.16:0/4289464247 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cfc15fcb0 0x7f7cfc0ce5d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:44.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.523+0000 7f7d01142640 1 -- 172.21.15.16:0/4289464247 >> 172.21.15.16:0/4289464247 conn(0x7f7cfc0d2230 msgr2=0x7f7cfc0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:44.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.523+0000 7f7d01142640 1 -- 172.21.15.16:0/4289464247 shutdown_connections 2024-09-19T07:43:44.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.523+0000 7f7d01142640 1 -- 172.21.15.16:0/4289464247 wait complete. 2024-09-19T07:43:44.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.524+0000 7f7d01142640 1 Processor -- start 2024-09-19T07:43:44.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.524+0000 7f7d01142640 1 -- start start 2024-09-19T07:43:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.525+0000 7f7d01142640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cfc0cccd0 0x7f7cfc0b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.525+0000 7f7d01142640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc0cd0f0 0x7f7cfc0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.525+0000 7f7d01142640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cfc15fcb0 0x7f7cfc150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.525+0000 7f7d01142640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc15d550 con 0x7f7cfc0cd0f0 2024-09-19T07:43:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.525+0000 7f7d01142640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc09ad70 con 0x7f7cfc15fcb0 2024-09-19T07:43:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.525+0000 7f7d01142640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7cfc09b9a0 con 0x7f7cfc0cccd0 2024-09-19T07:43:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.525+0000 7f7cfa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc0cd0f0 0x7f7cfc0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.525+0000 7f7cfa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc0cd0f0 0x7f7cfc0b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35138/0 (socket says 172.21.15.16:35138) 2024-09-19T07:43:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.525+0000 7f7cfa59c640 1 -- 172.21.15.16:0/282812049 learned_addr learned my addr 172.21.15.16:0/282812049 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.525+0000 7f7cfad9d640 1 --2- 172.21.15.16:0/282812049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cfc0cccd0 0x7f7cfc0b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.525+0000 7f7cfb59e640 1 --2- 172.21.15.16:0/282812049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cfc15fcb0 0x7f7cfc150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.525+0000 7f7cfa59c640 1 -- 172.21.15.16:0/282812049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cfc0cccd0 msgr2=0x7f7cfc0b7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.525+0000 7f7cfa59c640 1 --2- 172.21.15.16:0/282812049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cfc0cccd0 0x7f7cfc0b7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.525+0000 7f7cfa59c640 1 -- 172.21.15.16:0/282812049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cfc15fcb0 msgr2=0x7f7cfc150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.525+0000 7f7cfa59c640 1 --2- 172.21.15.16:0/282812049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cfc15fcb0 0x7f7cfc150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.525+0000 7f7cfa59c640 1 -- 172.21.15.16:0/282812049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cfc0b8000 con 0x7f7cfc0cd0f0 2024-09-19T07:43:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.525+0000 7f7cfb59e640 1 --2- 172.21.15.16:0/282812049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cfc15fcb0 0x7f7cfc150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.525+0000 7f7cfa59c640 1 --2- 172.21.15.16:0/282812049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc0cd0f0 0x7f7cfc0b5ab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400e980 tx=0x7f7ce400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.526+0000 7f7cf37fe640 1 -- 172.21.15.16:0/282812049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4017070 con 0x7f7cfc0cd0f0 2024-09-19T07:43:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.526+0000 7f7cf37fe640 1 -- 172.21.15.16:0/282812049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce400ca00 con 0x7f7cfc0cd0f0 2024-09-19T07:43:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.526+0000 7f7d01142640 1 -- 172.21.15.16:0/282812049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cfc0b6d00 con 0x7f7cfc0cd0f0 2024-09-19T07:43:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.526+0000 7f7cf37fe640 1 -- 172.21.15.16:0/282812049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce400cdd0 con 0x7f7cfc0cd0f0 2024-09-19T07:43:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.526+0000 7f7d01142640 1 -- 172.21.15.16:0/282812049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cfc0b72c0 con 0x7f7cfc0cd0f0 2024-09-19T07:43:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.527+0000 7f7d01142640 1 -- 172.21.15.16:0/282812049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cfc09bc70 con 0x7f7cfc0cd0f0 2024-09-19T07:43:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.527+0000 7f7cf37fe640 1 -- 172.21.15.16:0/282812049 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7ce401a070 con 0x7f7cfc0cd0f0 2024-09-19T07:43:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.528+0000 7f7cf37fe640 1 --2- 172.21.15.16:0/282812049 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7cdc06c2b0 0x7f7cdc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.528+0000 7f7cf37fe640 1 -- 172.21.15.16:0/282812049 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7ce408b830 con 0x7f7cfc0cd0f0 2024-09-19T07:43:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.528+0000 7f7cfad9d640 1 --2- 172.21.15.16:0/282812049 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7cdc06c2b0 0x7f7cdc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.529+0000 7f7cfad9d640 1 --2- 172.21.15.16:0/282812049 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7cdc06c2b0 0x7f7cdc06e770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7cec034000 tx=0x7f7cec02f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:44.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.533+0000 7f7cf37fe640 1 -- 172.21.15.16:0/282812049 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7ce40594e0 con 0x7f7cfc0cd0f0 2024-09-19T07:43:44.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.707+0000 7f7d01142640 1 -- 172.21.15.16:0/282812049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f7cfc0d2e90 con 0x7f7cfc0cd0f0 2024-09-19T07:43:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.710+0000 7f7cf37fe640 1 -- 172.21.15.16:0/282812049 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f7ce405d7e0 con 0x7f7cfc0cd0f0 2024-09-19T07:43:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.712+0000 7f7d01142640 1 -- 172.21.15.16:0/282812049 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7cdc06c2b0 msgr2=0x7f7cdc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.712+0000 7f7d01142640 1 --2- 172.21.15.16:0/282812049 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7cdc06c2b0 0x7f7cdc06e770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7cec034000 tx=0x7f7cec02f000 comp rx=0 tx=0).stop 2024-09-19T07:43:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.712+0000 7f7d01142640 1 -- 172.21.15.16:0/282812049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc0cd0f0 msgr2=0x7f7cfc0b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.712+0000 7f7d01142640 1 --2- 172.21.15.16:0/282812049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc0cd0f0 0x7f7cfc0b5ab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400e980 tx=0x7f7ce400ee40 comp rx=0 tx=0).stop 2024-09-19T07:43:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.712+0000 7f7d01142640 1 -- 172.21.15.16:0/282812049 shutdown_connections 2024-09-19T07:43:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.712+0000 7f7d01142640 1 --2- 172.21.15.16:0/282812049 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7cdc06c2b0 0x7f7cdc06e770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.712+0000 7f7d01142640 1 --2- 172.21.15.16:0/282812049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cfc15fcb0 0x7f7cfc150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.712+0000 7f7d01142640 1 --2- 172.21.15.16:0/282812049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc0cd0f0 0x7f7cfc0b5ab0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.712+0000 7f7d01142640 1 --2- 172.21.15.16:0/282812049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cfc0cccd0 0x7f7cfc0b7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.712+0000 7f7d01142640 1 -- 172.21.15.16:0/282812049 >> 172.21.15.16:0/282812049 conn(0x7f7cfc0d2230 msgr2=0x7f7cfc0d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.712+0000 7f7d01142640 1 -- 172.21.15.16:0/282812049 shutdown_connections 2024-09-19T07:43:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:44.712+0000 7f7d01142640 1 -- 172.21.15.16:0/282812049 wait complete. 2024-09-19T07:43:44.724 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:43:44.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:43:44.764 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:43:44.823 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:43:44.823 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:43:44.823 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:43:44.840 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:43:44.840 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:43:44.896 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:43:45.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.039+0000 7f3782b0c640 1 Processor -- start 2024-09-19T07:43:45.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.039+0000 7f3782b0c640 1 -- start start 2024-09-19T07:43:45.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.040+0000 7f3782b0c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f377c0d4840 0x7f377c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.040+0000 7f3782b0c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f377c0d1b10 con 0x7f377c0d0070 2024-09-19T07:43:45.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.040+0000 7f3782b0c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f377c0cf2b0 con 0x7f377c15f0d0 2024-09-19T07:43:45.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.040+0000 7f3782b0c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f377c0cf590 con 0x7f377c0d4840 2024-09-19T07:43:45.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.040+0000 7f37810a9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f377c0d4840 0x7f377c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.040+0000 7f377bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f377c0d0070 0x7f377c0d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37764/0 (socket says 172.21.15.16:37764) 2024-09-19T07:43:45.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.040+0000 7f37810a9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f377c0d4840 0x7f377c15f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35148/0 (socket says 172.21.15.16:35148) 2024-09-19T07:43:45.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.040+0000 7f377bfff640 1 -- 172.21.15.16:0/1614865948 learned_addr learned my addr 172.21.15.16:0/1614865948 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:45.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.040+0000 7f37810a9640 1 -- 172.21.15.16:0/1614865948 >> v1:172.21.15.146:6789/0 conn(0x7f377c15f0d0 legacy=0x7f377c0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:45.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.040+0000 7f37810a9640 1 -- 172.21.15.16:0/1614865948 >> v1:172.21.15.16:6789/0 conn(0x7f377c0d0070 legacy=0x7f377c0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.040+0000 7f37810a9640 1 -- 172.21.15.16:0/1614865948 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c0cf870 con 0x7f377c0d4840 2024-09-19T07:43:45.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.041+0000 7f37810a9640 1 --2- 172.21.15.16:0/1614865948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f377c0d4840 0x7f377c15f4b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3774009dc0 tx=0x7f377402a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664475e29820361 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:45.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.041+0000 7f377b7fe640 1 -- 172.21.15.16:0/1614865948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774007740 con 0x7f377c0d4840 2024-09-19T07:43:45.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.041+0000 7f377b7fe640 1 -- 172.21.15.16:0/1614865948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37740078e0 con 0x7f377c0d4840 2024-09-19T07:43:45.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.041+0000 7f377b7fe640 1 -- 172.21.15.16:0/1614865948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774007cb0 con 0x7f377c0d4840 2024-09-19T07:43:45.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.042+0000 7f3782b0c640 1 -- 172.21.15.16:0/1614865948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f377c0d4840 msgr2=0x7f377c15f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.042+0000 7f3782b0c640 1 --2- 172.21.15.16:0/1614865948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f377c0d4840 0x7f377c15f4b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3774009dc0 tx=0x7f377402a540 comp rx=0 tx=0).stop 2024-09-19T07:43:45.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.042+0000 7f3782b0c640 1 -- 172.21.15.16:0/1614865948 shutdown_connections 2024-09-19T07:43:45.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.042+0000 7f3782b0c640 1 --2- 172.21.15.16:0/1614865948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f377c0d4840 0x7f377c15f4b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.042+0000 7f3782b0c640 1 -- 172.21.15.16:0/1614865948 >> 172.21.15.16:0/1614865948 conn(0x7f377c09d760 msgr2=0x7f377c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:45.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.042+0000 7f3782b0c640 1 -- 172.21.15.16:0/1614865948 shutdown_connections 2024-09-19T07:43:45.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.042+0000 7f3782b0c640 1 -- 172.21.15.16:0/1614865948 wait complete. 2024-09-19T07:43:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.043+0000 7f3782b0c640 1 Processor -- start 2024-09-19T07:43:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.043+0000 7f3782b0c640 1 -- start start 2024-09-19T07:43:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.044+0000 7f3782b0c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f377c0d0070 0x7f377c1549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.044+0000 7f3782b0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f377c0d4840 0x7f377c160710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.044+0000 7f3782b0c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f377c15f0d0 0x7f377c160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.044+0000 7f3782b0c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f377c0b2f70 con 0x7f377c0d4840 2024-09-19T07:43:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.044+0000 7f3782b0c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f377c0cef60 con 0x7f377c0d0070 2024-09-19T07:43:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.044+0000 7f3782b0c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f377c0b30f0 con 0x7f377c15f0d0 2024-09-19T07:43:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.044+0000 7f377bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f377c0d4840 0x7f377c160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.044+0000 7f377bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f377c0d4840 0x7f377c160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35158/0 (socket says 172.21.15.16:35158) 2024-09-19T07:43:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.044+0000 7f377bfff640 1 -- 172.21.15.16:0/873372675 learned_addr learned my addr 172.21.15.16:0/873372675 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.044+0000 7f377bfff640 1 -- 172.21.15.16:0/873372675 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f377c15f0d0 msgr2=0x7f377c160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.044+0000 7f377bfff640 1 --2- 172.21.15.16:0/873372675 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f377c15f0d0 0x7f377c160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.044+0000 7f377bfff640 1 -- 172.21.15.16:0/873372675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f377c0d0070 msgr2=0x7f377c1549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.044+0000 7f37808a8640 1 --2- 172.21.15.16:0/873372675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f377c0d0070 0x7f377c1549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.044+0000 7f377bfff640 1 --2- 172.21.15.16:0/873372675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f377c0d0070 0x7f377c1549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.044+0000 7f377bfff640 1 -- 172.21.15.16:0/873372675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c0ccd20 con 0x7f377c0d4840 2024-09-19T07:43:45.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.044+0000 7f377bfff640 1 --2- 172.21.15.16:0/873372675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f377c0d4840 0x7f377c160710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f376800c780 tx=0x7f376800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:45.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.045+0000 7f37797fa640 1 -- 172.21.15.16:0/873372675 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768018070 con 0x7f377c0d4840 2024-09-19T07:43:45.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.045+0000 7f37797fa640 1 -- 172.21.15.16:0/873372675 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37680072f0 con 0x7f377c0d4840 2024-09-19T07:43:45.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.045+0000 7f3782b0c640 1 -- 172.21.15.16:0/873372675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377c0cd000 con 0x7f377c0d4840 2024-09-19T07:43:45.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.045+0000 7f37797fa640 1 -- 172.21.15.16:0/873372675 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37680047e0 con 0x7f377c0d4840 2024-09-19T07:43:45.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.045+0000 7f3782b0c640 1 -- 172.21.15.16:0/873372675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f377c0cd5c0 con 0x7f377c0d4840 2024-09-19T07:43:45.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.046+0000 7f3782b0c640 1 -- 172.21.15.16:0/873372675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f377c15f4b0 con 0x7f377c0d4840 2024-09-19T07:43:45.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.046+0000 7f37797fa640 1 -- 172.21.15.16:0/873372675 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f376801b070 con 0x7f377c0d4840 2024-09-19T07:43:45.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.047+0000 7f37797fa640 1 --2- 172.21.15.16:0/873372675 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f376006c2b0 0x7f376006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.047+0000 7f37797fa640 1 -- 172.21.15.16:0/873372675 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f376808abe0 con 0x7f377c0d4840 2024-09-19T07:43:45.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.047+0000 7f37808a8640 1 --2- 172.21.15.16:0/873372675 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f376006c2b0 0x7f376006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.048+0000 7f37808a8640 1 --2- 172.21.15.16:0/873372675 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f376006c2b0 0x7f376006e770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f377c1587f0 tx=0x7f3770008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:45.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.051+0000 7f37797fa640 1 -- 172.21.15.16:0/873372675 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3768059010 con 0x7f377c0d4840 2024-09-19T07:43:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.238+0000 7f3782b0c640 1 -- 172.21.15.16:0/873372675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f377c06cf30 con 0x7f377c0d4840 2024-09-19T07:43:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.239+0000 7f37797fa640 1 -- 172.21.15.16:0/873372675 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f3768058910 con 0x7f377c0d4840 2024-09-19T07:43:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:43:45.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.241+0000 7f3782b0c640 1 -- 172.21.15.16:0/873372675 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f376006c2b0 msgr2=0x7f376006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.241+0000 7f3782b0c640 1 --2- 172.21.15.16:0/873372675 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f376006c2b0 0x7f376006e770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f377c1587f0 tx=0x7f3770008040 comp rx=0 tx=0).stop 2024-09-19T07:43:45.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.241+0000 7f3782b0c640 1 -- 172.21.15.16:0/873372675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f377c0d4840 msgr2=0x7f377c160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.241+0000 7f3782b0c640 1 --2- 172.21.15.16:0/873372675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f377c0d4840 0x7f377c160710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f376800c780 tx=0x7f376800cc40 comp rx=0 tx=0).stop 2024-09-19T07:43:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.241+0000 7f3782b0c640 1 -- 172.21.15.16:0/873372675 shutdown_connections 2024-09-19T07:43:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.241+0000 7f3782b0c640 1 --2- 172.21.15.16:0/873372675 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f376006c2b0 0x7f376006e770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.241+0000 7f3782b0c640 1 --2- 172.21.15.16:0/873372675 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f377c15f0d0 0x7f377c160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.241+0000 7f3782b0c640 1 --2- 172.21.15.16:0/873372675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f377c0d4840 0x7f377c160710 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.241+0000 7f3782b0c640 1 --2- 172.21.15.16:0/873372675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f377c0d0070 0x7f377c1549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.241+0000 7f3782b0c640 1 -- 172.21.15.16:0/873372675 >> 172.21.15.16:0/873372675 conn(0x7f377c09d760 msgr2=0x7f377c0b24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.241+0000 7f3782b0c640 1 -- 172.21.15.16:0/873372675 shutdown_connections 2024-09-19T07:43:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.241+0000 7f3782b0c640 1 -- 172.21.15.16:0/873372675 wait complete. 2024-09-19T07:43:45.252 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:43:45.252 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:43:45.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:43:45.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.412+0000 7f16905e8640 1 Processor -- start 2024-09-19T07:43:45.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.412+0000 7f16905e8640 1 -- start start 2024-09-19T07:43:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.413+0000 7f16905e8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16880d0070 0x7f168815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.413+0000 7f16905e8640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f16880d4130 0x7f16880d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.413+0000 7f16905e8640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16880d1b10 con 0x7f168815f0d0 2024-09-19T07:43:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.413+0000 7f16905e8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f16880cf2b0 con 0x7f16880d0070 2024-09-19T07:43:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.413+0000 7f16905e8640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f16880cf590 con 0x7f16880d4130 2024-09-19T07:43:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.413+0000 7f168db83640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16880d0070 0x7f168815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.413+0000 7f168db83640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16880d0070 0x7f168815f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35170/0 (socket says 172.21.15.16:35170) 2024-09-19T07:43:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.413+0000 7f168db83640 1 -- 172.21.15.16:0/2881597538 learned_addr learned my addr 172.21.15.16:0/2881597538 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.413+0000 7f168eb85640 1 --2- 172.21.15.16:0/2881597538 >> v2:172.21.15.146:3300/0 conn(0x7f16880d4130 0x7f16880d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.413+0000 7f168db83640 1 -- 172.21.15.16:0/2881597538 >> v2:172.21.15.146:3300/0 conn(0x7f16880d4130 msgr2=0x7f16880d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.413+0000 7f168db83640 1 --2- 172.21.15.16:0/2881597538 >> v2:172.21.15.146:3300/0 conn(0x7f16880d4130 0x7f16880d4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.413+0000 7f168db83640 1 -- 172.21.15.16:0/2881597538 >> v1:172.21.15.146:6789/0 conn(0x7f168815f0d0 legacy=0x7f16880d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.413+0000 7f168db83640 1 -- 172.21.15.16:0/2881597538 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16880cf870 con 0x7f16880d0070 2024-09-19T07:43:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.413+0000 7f168eb85640 1 --2- 172.21.15.16:0/2881597538 >> v2:172.21.15.146:3300/0 conn(0x7f16880d4130 0x7f16880d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.414+0000 7f168db83640 1 --2- 172.21.15.16:0/2881597538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16880d0070 0x7f168815f4b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1688162fc0 tx=0x7f168402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395c51f0d29ec0cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.414+0000 7f168d382640 1 -- 172.21.15.16:0/2881597538 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1684037070 con 0x7f16880d0070 2024-09-19T07:43:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.414+0000 7f168d382640 1 -- 172.21.15.16:0/2881597538 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1684032440 con 0x7f16880d0070 2024-09-19T07:43:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.414+0000 7f168d382640 1 -- 172.21.15.16:0/2881597538 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1684032810 con 0x7f16880d0070 2024-09-19T07:43:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.415+0000 7f16905e8640 1 -- 172.21.15.16:0/2881597538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16880d0070 msgr2=0x7f168815f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.415+0000 7f16905e8640 1 --2- 172.21.15.16:0/2881597538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16880d0070 0x7f168815f4b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1688162fc0 tx=0x7f168402a160 comp rx=0 tx=0).stop 2024-09-19T07:43:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.415+0000 7f16905e8640 1 -- 172.21.15.16:0/2881597538 shutdown_connections 2024-09-19T07:43:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.415+0000 7f16905e8640 1 --2- 172.21.15.16:0/2881597538 >> v2:172.21.15.146:3300/0 conn(0x7f16880d4130 0x7f16880d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.415+0000 7f16905e8640 1 --2- 172.21.15.16:0/2881597538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16880d0070 0x7f168815f4b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.415+0000 7f16905e8640 1 -- 172.21.15.16:0/2881597538 >> 172.21.15.16:0/2881597538 conn(0x7f168809d760 msgr2=0x7f168808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.415+0000 7f16905e8640 1 -- 172.21.15.16:0/2881597538 shutdown_connections 2024-09-19T07:43:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.415+0000 7f16905e8640 1 -- 172.21.15.16:0/2881597538 wait complete. 2024-09-19T07:43:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.416+0000 7f16905e8640 1 Processor -- start 2024-09-19T07:43:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.416+0000 7f16905e8640 1 -- start start 2024-09-19T07:43:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.417+0000 7f16905e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16880d0070 0x7f168814dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.417+0000 7f16905e8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16880d4130 0x7f16880b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.417+0000 7f16905e8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f168815f0d0 0x7f1688161e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.417+0000 7f16905e8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16880cef60 con 0x7f16880d0070 2024-09-19T07:43:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.417+0000 7f16905e8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f16880d4aa0 con 0x7f16880d4130 2024-09-19T07:43:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.417+0000 7f16905e8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f16880b2f70 con 0x7f168815f0d0 2024-09-19T07:43:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.417+0000 7f168e384640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16880d0070 0x7f168814dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.417+0000 7f168e384640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16880d0070 0x7f168814dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35184/0 (socket says 172.21.15.16:35184) 2024-09-19T07:43:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.417+0000 7f168e384640 1 -- 172.21.15.16:0/3570105155 learned_addr learned my addr 172.21.15.16:0/3570105155 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.417+0000 7f168db83640 1 --2- 172.21.15.16:0/3570105155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16880d4130 0x7f16880b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.417+0000 7f168eb85640 1 --2- 172.21.15.16:0/3570105155 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f168815f0d0 0x7f1688161e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.417+0000 7f168e384640 1 -- 172.21.15.16:0/3570105155 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f168815f0d0 msgr2=0x7f1688161e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.417+0000 7f168e384640 1 --2- 172.21.15.16:0/3570105155 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f168815f0d0 0x7f1688161e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.417+0000 7f168e384640 1 -- 172.21.15.16:0/3570105155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16880d4130 msgr2=0x7f16880b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.417+0000 7f168e384640 1 --2- 172.21.15.16:0/3570105155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16880d4130 0x7f16880b7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.417+0000 7f168e384640 1 -- 172.21.15.16:0/3570105155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16881f77a0 con 0x7f16880d0070 2024-09-19T07:43:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.417+0000 7f168eb85640 1 --2- 172.21.15.16:0/3570105155 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f168815f0d0 0x7f1688161e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.417+0000 7f168e384640 1 --2- 172.21.15.16:0/3570105155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16880d0070 0x7f168814dc40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f167800b3f0 tx=0x7f167800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.417+0000 7f167effd640 1 -- 172.21.15.16:0/3570105155 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678015020 con 0x7f16880d0070 2024-09-19T07:43:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.418+0000 7f167effd640 1 -- 172.21.15.16:0/3570105155 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1678011830 con 0x7f16880d0070 2024-09-19T07:43:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.418+0000 7f16905e8640 1 -- 172.21.15.16:0/3570105155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16881f7a80 con 0x7f16880d0070 2024-09-19T07:43:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.418+0000 7f167effd640 1 -- 172.21.15.16:0/3570105155 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678011c20 con 0x7f16880d0070 2024-09-19T07:43:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.418+0000 7f16905e8640 1 -- 172.21.15.16:0/3570105155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16881f8040 con 0x7f16880d0070 2024-09-19T07:43:45.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.419+0000 7f16905e8640 1 -- 172.21.15.16:0/3570105155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16880d4560 con 0x7f16880d0070 2024-09-19T07:43:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.419+0000 7f167effd640 1 -- 172.21.15.16:0/3570105155 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1678018020 con 0x7f16880d0070 2024-09-19T07:43:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.420+0000 7f167effd640 1 --2- 172.21.15.16:0/3570105155 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f166c06c2b0 0x7f166c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.420+0000 7f167effd640 1 -- 172.21.15.16:0/3570105155 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f167808b390 con 0x7f16880d0070 2024-09-19T07:43:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.420+0000 7f168db83640 1 --2- 172.21.15.16:0/3570105155 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f166c06c2b0 0x7f166c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.421+0000 7f168db83640 1 --2- 172.21.15.16:0/3570105155 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f166c06c2b0 0x7f166c06e770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1684034000 tx=0x7f168402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.424+0000 7f167effd640 1 -- 172.21.15.16:0/3570105155 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f167801f030 con 0x7f16880d0070 2024-09-19T07:43:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.613+0000 7f16905e8640 1 -- 172.21.15.16:0/3570105155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f168806cf30 con 0x7f16880d0070 2024-09-19T07:43:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.613+0000 7f167effd640 1 -- 172.21.15.16:0/3570105155 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f16780597c0 con 0x7f16880d0070 2024-09-19T07:43:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:43:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.615+0000 7f16905e8640 1 -- 172.21.15.16:0/3570105155 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f166c06c2b0 msgr2=0x7f166c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.615+0000 7f16905e8640 1 --2- 172.21.15.16:0/3570105155 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f166c06c2b0 0x7f166c06e770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1684034000 tx=0x7f168402e000 comp rx=0 tx=0).stop 2024-09-19T07:43:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.615+0000 7f16905e8640 1 -- 172.21.15.16:0/3570105155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16880d0070 msgr2=0x7f168814dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.615+0000 7f16905e8640 1 --2- 172.21.15.16:0/3570105155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16880d0070 0x7f168814dc40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f167800b3f0 tx=0x7f167800b8b0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.615+0000 7f16905e8640 1 -- 172.21.15.16:0/3570105155 shutdown_connections 2024-09-19T07:43:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.615+0000 7f16905e8640 1 --2- 172.21.15.16:0/3570105155 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f166c06c2b0 0x7f166c06e770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.615+0000 7f16905e8640 1 --2- 172.21.15.16:0/3570105155 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f168815f0d0 0x7f1688161e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.615+0000 7f16905e8640 1 --2- 172.21.15.16:0/3570105155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16880d4130 0x7f16880b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.615+0000 7f16905e8640 1 --2- 172.21.15.16:0/3570105155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16880d0070 0x7f168814dc40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.615+0000 7f16905e8640 1 -- 172.21.15.16:0/3570105155 >> 172.21.15.16:0/3570105155 conn(0x7f168809d760 msgr2=0x7f16880b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.615+0000 7f16905e8640 1 -- 172.21.15.16:0/3570105155 shutdown_connections 2024-09-19T07:43:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.615+0000 7f16905e8640 1 -- 172.21.15.16:0/3570105155 wait complete. 2024-09-19T07:43:45.626 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-19T07:43:45.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:45.627 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T07:43:45.653 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:43:45.653 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:43:45.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:45.654 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:43:45.710 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:43:45.711 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:43:45.730 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:43:45.730 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:43:45.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:43:45.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.892+0000 7f9591b86640 1 Processor -- start 2024-09-19T07:43:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.892+0000 7f9591b86640 1 -- start start 2024-09-19T07:43:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.892+0000 7f9591b86640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f958c15e2a0 0x7f958c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.893+0000 7f9591b86640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f958c0d1880 con 0x7f958c15e680 2024-09-19T07:43:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.893+0000 7f9591b86640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f958c0d1b60 con 0x7f958c06d960 2024-09-19T07:43:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.893+0000 7f9591b86640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f958c0cf2d0 con 0x7f958c15e2a0 2024-09-19T07:43:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.893+0000 7f958b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f958c15e2a0 0x7f958c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.893+0000 7f958b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f958c15e2a0 0x7f958c06d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35188/0 (socket says 172.21.15.16:35188) 2024-09-19T07:43:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.893+0000 7f958affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f958c15e680 0x7f958c0cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37768/0 (socket says 172.21.15.16:37768) 2024-09-19T07:43:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.893+0000 7f958b7fe640 1 -- 172.21.15.16:0/3660655781 learned_addr learned my addr 172.21.15.16:0/3660655781 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.893+0000 7f958b7fe640 1 -- 172.21.15.16:0/3660655781 >> v1:172.21.15.146:6789/0 conn(0x7f958c06d960 legacy=0x7f958c0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.893+0000 7f958b7fe640 1 -- 172.21.15.16:0/3660655781 >> v1:172.21.15.16:6789/0 conn(0x7f958c15e680 legacy=0x7f958c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.893+0000 7f958b7fe640 1 -- 172.21.15.16:0/3660655781 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958c0cf5b0 con 0x7f958c15e2a0 2024-09-19T07:43:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.894+0000 7f958b7fe640 1 --2- 172.21.15.16:0/3660655781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f958c15e2a0 0x7f958c06d420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f957c009870 tx=0x7f957c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441bdeed7323459d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.894+0000 7f958a7fc640 1 -- 172.21.15.16:0/3660655781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c037070 con 0x7f958c15e2a0 2024-09-19T07:43:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.894+0000 7f958a7fc640 1 -- 172.21.15.16:0/3660655781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f957c032440 con 0x7f958c15e2a0 2024-09-19T07:43:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.894+0000 7f958a7fc640 1 -- 172.21.15.16:0/3660655781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c032810 con 0x7f958c15e2a0 2024-09-19T07:43:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.894+0000 7f9591b86640 1 -- 172.21.15.16:0/3660655781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f958c15e2a0 msgr2=0x7f958c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.894+0000 7f9591b86640 1 --2- 172.21.15.16:0/3660655781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f958c15e2a0 0x7f958c06d420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f957c009870 tx=0x7f957c02a2a0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.894+0000 7f9591b86640 1 -- 172.21.15.16:0/3660655781 shutdown_connections 2024-09-19T07:43:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.895+0000 7f9591b86640 1 --2- 172.21.15.16:0/3660655781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f958c15e2a0 0x7f958c06d420 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.895+0000 7f9591b86640 1 -- 172.21.15.16:0/3660655781 >> 172.21.15.16:0/3660655781 conn(0x7f958c09d760 msgr2=0x7f958c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.895+0000 7f9591b86640 1 -- 172.21.15.16:0/3660655781 shutdown_connections 2024-09-19T07:43:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.895+0000 7f9591b86640 1 -- 172.21.15.16:0/3660655781 wait complete. 2024-09-19T07:43:45.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.896+0000 7f9591b86640 1 Processor -- start 2024-09-19T07:43:45.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.896+0000 7f9591b86640 1 -- start start 2024-09-19T07:43:45.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.897+0000 7f9591b86640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f958c06d960 0x7f958c15c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.897+0000 7f9591b86640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f958c15e2a0 0x7f958c0cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.897+0000 7f9591b86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f958c15e680 0x7f958c0cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.897+0000 7f9591b86640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f958c0d0360 con 0x7f958c15e680 2024-09-19T07:43:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.897+0000 7f9591b86640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f958c0d01e0 con 0x7f958c15e2a0 2024-09-19T07:43:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.897+0000 7f9591b86640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f958c0d4a80 con 0x7f958c06d960 2024-09-19T07:43:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.897+0000 7f958bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f958c15e680 0x7f958c0cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.897+0000 7f958bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f958c15e680 0x7f958c0cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35202/0 (socket says 172.21.15.16:35202) 2024-09-19T07:43:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.897+0000 7f958bfff640 1 -- 172.21.15.16:0/1199109210 learned_addr learned my addr 172.21.15.16:0/1199109210 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.897+0000 7f958b7fe640 1 --2- 172.21.15.16:0/1199109210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f958c06d960 0x7f958c15c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.897+0000 7f958affd640 1 --2- 172.21.15.16:0/1199109210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f958c15e2a0 0x7f958c0cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.897+0000 7f958bfff640 1 -- 172.21.15.16:0/1199109210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f958c06d960 msgr2=0x7f958c15c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.897+0000 7f958bfff640 1 --2- 172.21.15.16:0/1199109210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f958c06d960 0x7f958c15c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.897+0000 7f958bfff640 1 -- 172.21.15.16:0/1199109210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f958c15e2a0 msgr2=0x7f958c0cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.897+0000 7f958bfff640 1 --2- 172.21.15.16:0/1199109210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f958c15e2a0 0x7f958c0cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.897+0000 7f958bfff640 1 -- 172.21.15.16:0/1199109210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958c0cdd10 con 0x7f958c15e680 2024-09-19T07:43:45.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.897+0000 7f958bfff640 1 --2- 172.21.15.16:0/1199109210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f958c15e680 0x7f958c0cd060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f958400c780 tx=0x7f958400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:45.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.898+0000 7f9583fff640 1 -- 172.21.15.16:0/1199109210 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584004b60 con 0x7f958c15e680 2024-09-19T07:43:45.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.898+0000 7f9583fff640 1 -- 172.21.15.16:0/1199109210 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9584004d00 con 0x7f958c15e680 2024-09-19T07:43:45.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.898+0000 7f9591b86640 1 -- 172.21.15.16:0/1199109210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f958c0cdff0 con 0x7f958c15e680 2024-09-19T07:43:45.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.898+0000 7f9583fff640 1 -- 172.21.15.16:0/1199109210 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958400aa30 con 0x7f958c15e680 2024-09-19T07:43:45.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.898+0000 7f9591b86640 1 -- 172.21.15.16:0/1199109210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f958c0cb080 con 0x7f958c15e680 2024-09-19T07:43:45.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.898+0000 7f9591b86640 1 -- 172.21.15.16:0/1199109210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f958c06d420 con 0x7f958c15e680 2024-09-19T07:43:45.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.899+0000 7f9583fff640 1 -- 172.21.15.16:0/1199109210 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f958400abd0 con 0x7f958c15e680 2024-09-19T07:43:45.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.900+0000 7f9583fff640 1 --2- 172.21.15.16:0/1199109210 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f956c06c2b0 0x7f956c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.900+0000 7f9583fff640 1 -- 172.21.15.16:0/1199109210 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f958408b800 con 0x7f958c15e680 2024-09-19T07:43:45.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.900+0000 7f958b7fe640 1 --2- 172.21.15.16:0/1199109210 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f956c06c2b0 0x7f956c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.900+0000 7f958b7fe640 1 --2- 172.21.15.16:0/1199109210 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f956c06c2b0 0x7f956c06e770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f957c034000 tx=0x7f957c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:45.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:45.904+0000 7f9583fff640 1 -- 172.21.15.16:0/1199109210 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9584020070 con 0x7f958c15e680 2024-09-19T07:43:46.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.091+0000 7f9591b86640 1 -- 172.21.15.16:0/1199109210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f958c06cf90 con 0x7f958c15e680 2024-09-19T07:43:46.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.092+0000 7f9583fff640 1 -- 172.21.15.16:0/1199109210 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f9584059670 con 0x7f958c15e680 2024-09-19T07:43:46.093 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:43:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.094+0000 7f9591b86640 1 -- 172.21.15.16:0/1199109210 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f956c06c2b0 msgr2=0x7f956c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.094+0000 7f9591b86640 1 --2- 172.21.15.16:0/1199109210 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f956c06c2b0 0x7f956c06e770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f957c034000 tx=0x7f957c02e000 comp rx=0 tx=0).stop 2024-09-19T07:43:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.094+0000 7f9591b86640 1 -- 172.21.15.16:0/1199109210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f958c15e680 msgr2=0x7f958c0cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.094+0000 7f9591b86640 1 --2- 172.21.15.16:0/1199109210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f958c15e680 0x7f958c0cd060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f958400c780 tx=0x7f958400cc40 comp rx=0 tx=0).stop 2024-09-19T07:43:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.094+0000 7f9591b86640 1 -- 172.21.15.16:0/1199109210 shutdown_connections 2024-09-19T07:43:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.094+0000 7f9591b86640 1 --2- 172.21.15.16:0/1199109210 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f956c06c2b0 0x7f956c06e770 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.094+0000 7f9591b86640 1 --2- 172.21.15.16:0/1199109210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f958c15e680 0x7f958c0cd060 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.094+0000 7f9591b86640 1 --2- 172.21.15.16:0/1199109210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f958c15e2a0 0x7f958c0cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.094+0000 7f9591b86640 1 --2- 172.21.15.16:0/1199109210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f958c06d960 0x7f958c15c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.094+0000 7f9591b86640 1 -- 172.21.15.16:0/1199109210 >> 172.21.15.16:0/1199109210 conn(0x7f958c09d760 msgr2=0x7f958c15eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.094+0000 7f9591b86640 1 -- 172.21.15.16:0/1199109210 shutdown_connections 2024-09-19T07:43:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.094+0000 7f9591b86640 1 -- 172.21.15.16:0/1199109210 wait complete. 2024-09-19T07:43:46.105 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:43:46.105 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:43:46.125 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:43:46.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.228+0000 7f4d8094b640 1 Processor -- start 2024-09-19T07:43:46.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.228+0000 7f4d8094b640 1 -- start start 2024-09-19T07:43:46.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.229+0000 7f4d8094b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d7c15e2a0 0x7f4d7c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:46.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.229+0000 7f4d8094b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4d7c06d960 0x7f4d7c0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:46.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.229+0000 7f4d8094b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d7c0d1880 con 0x7f4d7c15e680 2024-09-19T07:43:46.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.229+0000 7f4d8094b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f4d7c0d1b60 con 0x7f4d7c15e2a0 2024-09-19T07:43:46.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.229+0000 7f4d8094b640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4d7c0cf2d0 con 0x7f4d7c06d960 2024-09-19T07:43:46.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.229+0000 7f4d7a59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d7c15e2a0 0x7f4d7c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:46.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.229+0000 7f4d7a59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d7c15e2a0 0x7f4d7c06d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35210/0 (socket says 172.21.15.16:35210) 2024-09-19T07:43:46.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.229+0000 7f4d7a59c640 1 -- 172.21.15.16:0/1627585327 learned_addr learned my addr 172.21.15.16:0/1627585327 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:46.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.229+0000 7f4d73fff640 1 --1- 172.21.15.16:0/1627585327 >> v1:172.21.15.16:6789/0 conn(0x7f4d7c15e680 0x7f4d7c0cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37772/0 (socket says 172.21.15.16:37772) 2024-09-19T07:43:46.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.229+0000 7f4d7a59c640 1 -- 172.21.15.16:0/1627585327 >> v2:172.21.15.146:3300/0 conn(0x7f4d7c06d960 msgr2=0x7f4d7c0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:46.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.229+0000 7f4d7a59c640 1 --2- 172.21.15.16:0/1627585327 >> v2:172.21.15.146:3300/0 conn(0x7f4d7c06d960 0x7f4d7c0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.229+0000 7f4d7a59c640 1 -- 172.21.15.16:0/1627585327 >> v1:172.21.15.16:6789/0 conn(0x7f4d7c15e680 legacy=0x7f4d7c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:46.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.229+0000 7f4d7a59c640 1 -- 172.21.15.16:0/1627585327 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7c0cf5b0 con 0x7f4d7c15e2a0 2024-09-19T07:43:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.230+0000 7f4d7a59c640 1 --2- 172.21.15.16:0/1627585327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d7c15e2a0 0x7f4d7c06d420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c163050 tx=0x7f4d6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd19b3a61062ea6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.230+0000 7f4d79d9b640 1 -- 172.21.15.16:0/1627585327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c037070 con 0x7f4d7c15e2a0 2024-09-19T07:43:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.230+0000 7f4d79d9b640 1 -- 172.21.15.16:0/1627585327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d6c032440 con 0x7f4d7c15e2a0 2024-09-19T07:43:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.230+0000 7f4d79d9b640 1 -- 172.21.15.16:0/1627585327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c032810 con 0x7f4d7c15e2a0 2024-09-19T07:43:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.231+0000 7f4d8094b640 1 -- 172.21.15.16:0/1627585327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d7c15e2a0 msgr2=0x7f4d7c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.231+0000 7f4d8094b640 1 --2- 172.21.15.16:0/1627585327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d7c15e2a0 0x7f4d7c06d420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c163050 tx=0x7f4d6c02a270 comp rx=0 tx=0).stop 2024-09-19T07:43:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.231+0000 7f4d8094b640 1 -- 172.21.15.16:0/1627585327 shutdown_connections 2024-09-19T07:43:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.231+0000 7f4d8094b640 1 --2- 172.21.15.16:0/1627585327 >> v2:172.21.15.146:3300/0 conn(0x7f4d7c06d960 0x7f4d7c0d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.231+0000 7f4d8094b640 1 --2- 172.21.15.16:0/1627585327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d7c15e2a0 0x7f4d7c06d420 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.231+0000 7f4d8094b640 1 -- 172.21.15.16:0/1627585327 >> 172.21.15.16:0/1627585327 conn(0x7f4d7c09d760 msgr2=0x7f4d7c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.231+0000 7f4d8094b640 1 -- 172.21.15.16:0/1627585327 shutdown_connections 2024-09-19T07:43:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.231+0000 7f4d8094b640 1 -- 172.21.15.16:0/1627585327 wait complete. 2024-09-19T07:43:46.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.232+0000 7f4d8094b640 1 Processor -- start 2024-09-19T07:43:46.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.232+0000 7f4d8094b640 1 -- start start 2024-09-19T07:43:46.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.233+0000 7f4d8094b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d7c06d960 0x7f4d7c0b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:46.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.233+0000 7f4d8094b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d7c15e2a0 0x7f4d7c0b87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:46.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.233+0000 7f4d8094b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d7c15e680 0x7f4d7c15cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:46.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.233+0000 7f4d8094b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c0d0360 con 0x7f4d7c15e680 2024-09-19T07:43:46.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.233+0000 7f4d8094b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c0d01e0 con 0x7f4d7c15e2a0 2024-09-19T07:43:46.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.233+0000 7f4d8094b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4d7c0d48b0 con 0x7f4d7c06d960 2024-09-19T07:43:46.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.233+0000 7f4d7ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d7c15e680 0x7f4d7c15cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:46.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.233+0000 7f4d7ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d7c15e680 0x7f4d7c15cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35222/0 (socket says 172.21.15.16:35222) 2024-09-19T07:43:46.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.233+0000 7f4d7ad9d640 1 -- 172.21.15.16:0/302639615 learned_addr learned my addr 172.21.15.16:0/302639615 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:46.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.233+0000 7f4d73fff640 1 --2- 172.21.15.16:0/302639615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d7c15e2a0 0x7f4d7c0b87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:46.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.233+0000 7f4d7a59c640 1 --2- 172.21.15.16:0/302639615 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d7c06d960 0x7f4d7c0b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:46.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.233+0000 7f4d7ad9d640 1 -- 172.21.15.16:0/302639615 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d7c06d960 msgr2=0x7f4d7c0b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:46.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.233+0000 7f4d7ad9d640 1 --2- 172.21.15.16:0/302639615 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d7c06d960 0x7f4d7c0b8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.233+0000 7f4d7ad9d640 1 -- 172.21.15.16:0/302639615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d7c15e2a0 msgr2=0x7f4d7c0b87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:46.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.233+0000 7f4d7ad9d640 1 --2- 172.21.15.16:0/302639615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d7c15e2a0 0x7f4d7c0b87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.233+0000 7f4d7ad9d640 1 -- 172.21.15.16:0/302639615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7c0ce3b0 con 0x7f4d7c15e680 2024-09-19T07:43:46.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.233+0000 7f4d7ad9d640 1 --2- 172.21.15.16:0/302639615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d7c15e680 0x7f4d7c15cf70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400b9a0 tx=0x7f4d7400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:46.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.234+0000 7f4d72ffd640 1 -- 172.21.15.16:0/302639615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74019070 con 0x7f4d7c15e680 2024-09-19T07:43:46.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.234+0000 7f4d72ffd640 1 -- 172.21.15.16:0/302639615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7400cbc0 con 0x7f4d7c15e680 2024-09-19T07:43:46.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.234+0000 7f4d8094b640 1 -- 172.21.15.16:0/302639615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7c0ce650 con 0x7f4d7c15e680 2024-09-19T07:43:46.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.234+0000 7f4d72ffd640 1 -- 172.21.15.16:0/302639615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74014440 con 0x7f4d7c15e680 2024-09-19T07:43:46.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.234+0000 7f4d8094b640 1 -- 172.21.15.16:0/302639615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d7c0cb8c0 con 0x7f4d7c15e680 2024-09-19T07:43:46.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.235+0000 7f4d8094b640 1 -- 172.21.15.16:0/302639615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d7c0d4c70 con 0x7f4d7c15e680 2024-09-19T07:43:46.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.235+0000 7f4d72ffd640 1 -- 172.21.15.16:0/302639615 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4d7401c070 con 0x7f4d7c15e680 2024-09-19T07:43:46.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.236+0000 7f4d72ffd640 1 --2- 172.21.15.16:0/302639615 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4d5c06c2b0 0x7f4d5c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:46.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.236+0000 7f4d72ffd640 1 -- 172.21.15.16:0/302639615 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4d7408b430 con 0x7f4d7c15e680 2024-09-19T07:43:46.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.236+0000 7f4d7a59c640 1 --2- 172.21.15.16:0/302639615 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4d5c06c2b0 0x7f4d5c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:46.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.236+0000 7f4d7a59c640 1 --2- 172.21.15.16:0/302639615 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4d5c06c2b0 0x7f4d5c06e770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c034000 tx=0x7f4d6c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:46.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.240+0000 7f4d72ffd640 1 -- 172.21.15.16:0/302639615 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4d740598f0 con 0x7f4d7c15e680 2024-09-19T07:43:46.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.429+0000 7f4d8094b640 1 -- 172.21.15.16:0/302639615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4d7c161700 con 0x7f4d7c15e680 2024-09-19T07:43:46.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.429+0000 7f4d72ffd640 1 -- 172.21.15.16:0/302639615 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f4d740591f0 con 0x7f4d7c15e680 2024-09-19T07:43:46.430 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:43:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.431+0000 7f4d8094b640 1 -- 172.21.15.16:0/302639615 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4d5c06c2b0 msgr2=0x7f4d5c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.431+0000 7f4d8094b640 1 --2- 172.21.15.16:0/302639615 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4d5c06c2b0 0x7f4d5c06e770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c034000 tx=0x7f4d6c02e000 comp rx=0 tx=0).stop 2024-09-19T07:43:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.431+0000 7f4d8094b640 1 -- 172.21.15.16:0/302639615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d7c15e680 msgr2=0x7f4d7c15cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.431+0000 7f4d8094b640 1 --2- 172.21.15.16:0/302639615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d7c15e680 0x7f4d7c15cf70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400b9a0 tx=0x7f4d7400be60 comp rx=0 tx=0).stop 2024-09-19T07:43:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.431+0000 7f4d8094b640 1 -- 172.21.15.16:0/302639615 shutdown_connections 2024-09-19T07:43:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.431+0000 7f4d8094b640 1 --2- 172.21.15.16:0/302639615 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4d5c06c2b0 0x7f4d5c06e770 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.431+0000 7f4d8094b640 1 --2- 172.21.15.16:0/302639615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d7c15e680 0x7f4d7c15cf70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.431+0000 7f4d8094b640 1 --2- 172.21.15.16:0/302639615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d7c15e2a0 0x7f4d7c0b87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.431+0000 7f4d8094b640 1 --2- 172.21.15.16:0/302639615 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d7c06d960 0x7f4d7c0b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.431+0000 7f4d8094b640 1 -- 172.21.15.16:0/302639615 >> 172.21.15.16:0/302639615 conn(0x7f4d7c09d760 msgr2=0x7f4d7c0d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.431+0000 7f4d8094b640 1 -- 172.21.15.16:0/302639615 shutdown_connections 2024-09-19T07:43:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.431+0000 7f4d8094b640 1 -- 172.21.15.16:0/302639615 wait complete. 2024-09-19T07:43:46.443 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2024-09-19T07:43:46.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:46.443 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T07:43:46.477 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:43:46.477 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:43:46.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:46.478 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T07:43:46.545 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:43:46.546 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:43:46.562 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:43:46.562 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:43:46.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:43:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.761+0000 7fe345ae4640 1 Processor -- start 2024-09-19T07:43:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.761+0000 7fe345ae4640 1 -- start start 2024-09-19T07:43:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.762+0000 7fe345ae4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe34015e2a0 0x7fe34006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.762+0000 7fe345ae4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe34015e680 0x7fe3400cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.762+0000 7fe345ae4640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe34006d960 0x7fe3400d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.762+0000 7fe345ae4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fe3400d1880 con 0x7fe34015e2a0 2024-09-19T07:43:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.762+0000 7fe345ae4640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe3400d1b60 con 0x7fe34006d960 2024-09-19T07:43:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.762+0000 7fe345ae4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe3400cf2d0 con 0x7fe34015e680 2024-09-19T07:43:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.762+0000 7fe33f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe34015e2a0 0x7fe34006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.762+0000 7fe33f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe34015e2a0 0x7fe34006d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35238/0 (socket says 172.21.15.16:35238) 2024-09-19T07:43:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.762+0000 7fe33f7fe640 1 -- 172.21.15.16:0/1924120579 learned_addr learned my addr 172.21.15.16:0/1924120579 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.763+0000 7fe33f7fe640 1 -- 172.21.15.16:0/1924120579 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe34015e680 msgr2=0x7fe3400cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.763+0000 7fe33f7fe640 1 --2- 172.21.15.16:0/1924120579 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe34015e680 0x7fe3400cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.763+0000 7fe33f7fe640 1 -- 172.21.15.16:0/1924120579 >> v2:172.21.15.146:3300/0 conn(0x7fe34006d960 msgr2=0x7fe3400d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.763+0000 7fe33ffff640 1 --2- 172.21.15.16:0/1924120579 >> v2:172.21.15.146:3300/0 conn(0x7fe34006d960 0x7fe3400d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.763+0000 7fe33f7fe640 1 --2- 172.21.15.16:0/1924120579 >> v2:172.21.15.146:3300/0 conn(0x7fe34006d960 0x7fe3400d0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.763+0000 7fe33f7fe640 1 -- 172.21.15.16:0/1924120579 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3400cf5b0 con 0x7fe34015e2a0 2024-09-19T07:43:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.763+0000 7fe33f7fe640 1 --2- 172.21.15.16:0/1924120579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe34015e2a0 0x7fe34006d420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe34006cd80 tx=0x7fe33002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a49a594a9062d39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.764+0000 7fe33e7fc640 1 -- 172.21.15.16:0/1924120579 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330037070 con 0x7fe34015e2a0 2024-09-19T07:43:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.764+0000 7fe33e7fc640 1 -- 172.21.15.16:0/1924120579 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe330032440 con 0x7fe34015e2a0 2024-09-19T07:43:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.764+0000 7fe33e7fc640 1 -- 172.21.15.16:0/1924120579 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330032810 con 0x7fe34015e2a0 2024-09-19T07:43:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.764+0000 7fe345ae4640 1 -- 172.21.15.16:0/1924120579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe34015e2a0 msgr2=0x7fe34006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.764+0000 7fe345ae4640 1 --2- 172.21.15.16:0/1924120579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe34015e2a0 0x7fe34006d420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe34006cd80 tx=0x7fe33002a200 comp rx=0 tx=0).stop 2024-09-19T07:43:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.764+0000 7fe345ae4640 1 -- 172.21.15.16:0/1924120579 shutdown_connections 2024-09-19T07:43:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.764+0000 7fe345ae4640 1 --2- 172.21.15.16:0/1924120579 >> v2:172.21.15.146:3300/0 conn(0x7fe34006d960 0x7fe3400d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.764+0000 7fe345ae4640 1 --2- 172.21.15.16:0/1924120579 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe34015e680 0x7fe3400cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.764+0000 7fe345ae4640 1 --2- 172.21.15.16:0/1924120579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe34015e2a0 0x7fe34006d420 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.764+0000 7fe345ae4640 1 -- 172.21.15.16:0/1924120579 >> 172.21.15.16:0/1924120579 conn(0x7fe34009d760 msgr2=0x7fe34008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.764+0000 7fe345ae4640 1 -- 172.21.15.16:0/1924120579 shutdown_connections 2024-09-19T07:43:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.764+0000 7fe345ae4640 1 -- 172.21.15.16:0/1924120579 wait complete. 2024-09-19T07:43:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.765+0000 7fe345ae4640 1 Processor -- start 2024-09-19T07:43:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.766+0000 7fe345ae4640 1 -- start start 2024-09-19T07:43:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.766+0000 7fe345ae4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe34006d960 0x7fe3400cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.766+0000 7fe345ae4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe34015e2a0 0x7fe3400ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.766+0000 7fe345ae4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe34015e680 0x7fe340156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.766+0000 7fe345ae4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe3400d44f0 con 0x7fe34015e2a0 2024-09-19T07:43:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.766+0000 7fe345ae4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe3400d4370 con 0x7fe34006d960 2024-09-19T07:43:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.766+0000 7fe345ae4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe3400d4670 con 0x7fe34015e680 2024-09-19T07:43:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.766+0000 7fe33effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe34015e2a0 0x7fe3400ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.766+0000 7fe33effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe34015e2a0 0x7fe3400ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35242/0 (socket says 172.21.15.16:35242) 2024-09-19T07:43:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.766+0000 7fe33effd640 1 -- 172.21.15.16:0/1510344700 learned_addr learned my addr 172.21.15.16:0/1510344700 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.767+0000 7fe33ffff640 1 --2- 172.21.15.16:0/1510344700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe34015e680 0x7fe340156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.767+0000 7fe33f7fe640 1 --2- 172.21.15.16:0/1510344700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe34006d960 0x7fe3400cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.767+0000 7fe33effd640 1 -- 172.21.15.16:0/1510344700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe34015e680 msgr2=0x7fe340156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.767+0000 7fe33effd640 1 --2- 172.21.15.16:0/1510344700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe34015e680 0x7fe340156670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.767+0000 7fe33effd640 1 -- 172.21.15.16:0/1510344700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe34006d960 msgr2=0x7fe3400cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.767+0000 7fe33effd640 1 --2- 172.21.15.16:0/1510344700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe34006d960 0x7fe3400cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.767+0000 7fe33effd640 1 -- 172.21.15.16:0/1510344700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3400cd3a0 con 0x7fe34015e2a0 2024-09-19T07:43:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.767+0000 7fe33effd640 1 --2- 172.21.15.16:0/1510344700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe34015e2a0 0x7fe3400ce160 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe33400b390 tx=0x7fe33400b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.767+0000 7fe327fff640 1 -- 172.21.15.16:0/1510344700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334015020 con 0x7fe34015e2a0 2024-09-19T07:43:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.767+0000 7fe327fff640 1 -- 172.21.15.16:0/1510344700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe334011830 con 0x7fe34015e2a0 2024-09-19T07:43:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.767+0000 7fe345ae4640 1 -- 172.21.15.16:0/1510344700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3400cd680 con 0x7fe34015e2a0 2024-09-19T07:43:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.767+0000 7fe327fff640 1 -- 172.21.15.16:0/1510344700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334011c20 con 0x7fe34015e2a0 2024-09-19T07:43:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.767+0000 7fe345ae4640 1 -- 172.21.15.16:0/1510344700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3400cd0b0 con 0x7fe34015e2a0 2024-09-19T07:43:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.768+0000 7fe345ae4640 1 -- 172.21.15.16:0/1510344700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3400d4a40 con 0x7fe34015e2a0 2024-09-19T07:43:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.769+0000 7fe327fff640 1 -- 172.21.15.16:0/1510344700 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe334018020 con 0x7fe34015e2a0 2024-09-19T07:43:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.769+0000 7fe327fff640 1 --2- 172.21.15.16:0/1510344700 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe32c06c2b0 0x7fe32c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.769+0000 7fe327fff640 1 -- 172.21.15.16:0/1510344700 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe33408b620 con 0x7fe34015e2a0 2024-09-19T07:43:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.769+0000 7fe33f7fe640 1 --2- 172.21.15.16:0/1510344700 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe32c06c2b0 0x7fe32c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.770+0000 7fe33f7fe640 1 --2- 172.21.15.16:0/1510344700 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe32c06c2b0 0x7fe32c06e770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe330034000 tx=0x7fe33002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:46.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.773+0000 7fe327fff640 1 -- 172.21.15.16:0/1510344700 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe33401f020 con 0x7fe34015e2a0 2024-09-19T07:43:46.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.960+0000 7fe345ae4640 1 -- 172.21.15.16:0/1510344700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe34006cf90 con 0x7fe34015e2a0 2024-09-19T07:43:46.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.960+0000 7fe327fff640 1 -- 172.21.15.16:0/1510344700 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fe3340172c0 con 0x7fe34015e2a0 2024-09-19T07:43:46.960 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:43:46.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.962+0000 7fe345ae4640 1 -- 172.21.15.16:0/1510344700 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe32c06c2b0 msgr2=0x7fe32c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:46.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.962+0000 7fe345ae4640 1 --2- 172.21.15.16:0/1510344700 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe32c06c2b0 0x7fe32c06e770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe330034000 tx=0x7fe33002e000 comp rx=0 tx=0).stop 2024-09-19T07:43:46.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.962+0000 7fe345ae4640 1 -- 172.21.15.16:0/1510344700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe34015e2a0 msgr2=0x7fe3400ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:46.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.962+0000 7fe345ae4640 1 --2- 172.21.15.16:0/1510344700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe34015e2a0 0x7fe3400ce160 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe33400b390 tx=0x7fe33400b850 comp rx=0 tx=0).stop 2024-09-19T07:43:46.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.962+0000 7fe345ae4640 1 -- 172.21.15.16:0/1510344700 shutdown_connections 2024-09-19T07:43:46.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.962+0000 7fe345ae4640 1 --2- 172.21.15.16:0/1510344700 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe32c06c2b0 0x7fe32c06e770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.962+0000 7fe345ae4640 1 --2- 172.21.15.16:0/1510344700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe34015e680 0x7fe340156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.962+0000 7fe345ae4640 1 --2- 172.21.15.16:0/1510344700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe34015e2a0 0x7fe3400ce160 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.962+0000 7fe345ae4640 1 --2- 172.21.15.16:0/1510344700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe34006d960 0x7fe3400cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:46.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.962+0000 7fe345ae4640 1 -- 172.21.15.16:0/1510344700 >> 172.21.15.16:0/1510344700 conn(0x7fe34009d760 msgr2=0x7fe340160960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:46.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.963+0000 7fe345ae4640 1 -- 172.21.15.16:0/1510344700 shutdown_connections 2024-09-19T07:43:46.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:46.963+0000 7fe345ae4640 1 -- 172.21.15.16:0/1510344700 wait complete. 2024-09-19T07:43:46.973 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:43:46.974 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:43:46.990 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:43:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.132+0000 7f13b830a640 1 Processor -- start 2024-09-19T07:43:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.132+0000 7f13b830a640 1 -- start start 2024-09-19T07:43:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.133+0000 7f13b830a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b015f0d0 0x7f13b015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.133+0000 7f13b830a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13b00d1b10 con 0x7f13b00d4130 2024-09-19T07:43:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.133+0000 7f13b830a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13b00cf2b0 con 0x7f13b00d0070 2024-09-19T07:43:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.133+0000 7f13b830a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f13b00cf590 con 0x7f13b015f0d0 2024-09-19T07:43:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.133+0000 7f13b68a7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f13b00d4130 0x7f13b00d45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37788/0 (socket says 172.21.15.16:37788) 2024-09-19T07:43:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.133+0000 7f13b68a7640 1 -- 172.21.15.16:0/3182305160 learned_addr learned my addr 172.21.15.16:0/3182305160 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b60a6640 1 --2- 172.21.15.16:0/3182305160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b015f0d0 0x7f13b015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b50a4640 1 -- 172.21.15.16:0/3182305160 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3557028523 0 0) 0x7f13b00d1b10 con 0x7f13b00d4130 2024-09-19T07:43:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b50a4640 1 -- 172.21.15.16:0/3182305160 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1398003520 con 0x7f13b00d4130 2024-09-19T07:43:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b50a4640 1 -- 172.21.15.16:0/3182305160 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 780110286 0 0) 0x7f13b00cf2b0 con 0x7f13b00d0070 2024-09-19T07:43:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b50a4640 1 -- 172.21.15.16:0/3182305160 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13b00d1b10 con 0x7f13b00d0070 2024-09-19T07:43:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b50a4640 1 -- 172.21.15.16:0/3182305160 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1365159779 0 0) 0x7f1398003520 con 0x7f13b00d4130 2024-09-19T07:43:47.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b50a4640 1 -- 172.21.15.16:0/3182305160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b015f0d0 msgr2=0x7f13b015f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:47.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b50a4640 1 --2- 172.21.15.16:0/3182305160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b015f0d0 0x7f13b015f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b50a4640 1 -- 172.21.15.16:0/3182305160 >> v1:172.21.15.146:6789/0 conn(0x7f13b00d0070 legacy=0x7f13b00d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b50a4640 1 -- 172.21.15.16:0/3182305160 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b00cf870 con 0x7f13b00d4130 2024-09-19T07:43:47.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b50a4640 1 -- 172.21.15.16:0/3182305160 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f13a00019f0 con 0x7f13b00d4130 2024-09-19T07:43:47.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b60a6640 1 --2- 172.21.15.16:0/3182305160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b015f0d0 0x7f13b015f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:47.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b50a4640 1 -- 172.21.15.16:0/3182305160 >> v1:172.21.15.16:6789/0 conn(0x7f13b00d4130 legacy=0x7f13b00d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b50a4640 1 --2- 172.21.15.16:0/3182305160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398003b70 0x7f1398003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b50a4640 1 --2- 172.21.15.16:0/3182305160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b015f0d0 0x7f1398004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b50a4640 1 --2- 172.21.15.16:0/3182305160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1398004d40 0x7f13980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b50a4640 1 -- 172.21.15.16:0/3182305160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13b00cf590 con 0x7f1398003b70 2024-09-19T07:43:47.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b50a4640 1 -- 172.21.15.16:0/3182305160 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f139800baa0 con 0x7f1398004d40 2024-09-19T07:43:47.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b50a4640 1 -- 172.21.15.16:0/3182305160 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f139800bca0 con 0x7f13b015f0d0 2024-09-19T07:43:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b60a6640 1 --2- 172.21.15.16:0/3182305160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398003b70 0x7f1398003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b68a7640 1 --2- 172.21.15.16:0/3182305160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1398004d40 0x7f13980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b58a5640 1 --2- 172.21.15.16:0/3182305160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b015f0d0 0x7f1398004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b60a6640 1 -- 172.21.15.16:0/3182305160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b015f0d0 msgr2=0x7f1398004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b60a6640 1 --2- 172.21.15.16:0/3182305160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b015f0d0 0x7f1398004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b60a6640 1 -- 172.21.15.16:0/3182305160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1398004d40 msgr2=0x7f13980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b60a6640 1 --2- 172.21.15.16:0/3182305160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1398004d40 0x7f13980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b60a6640 1 -- 172.21.15.16:0/3182305160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1398003520 con 0x7f1398003b70 2024-09-19T07:43:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.134+0000 7f13b68a7640 1 --2- 172.21.15.16:0/3182305160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1398004d40 0x7f13980051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.135+0000 7f13b60a6640 1 --2- 172.21.15.16:0/3182305160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398003b70 0x7f1398003f70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009c90 tx=0x7f1398005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f715fce1ae8f47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.135+0000 7f13b50a4640 1 -- 172.21.15.16:0/3182305160 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8036070 con 0x7f1398003b70 2024-09-19T07:43:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.135+0000 7f13b50a4640 1 -- 172.21.15.16:0/3182305160 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13a8031400 con 0x7f1398003b70 2024-09-19T07:43:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.135+0000 7f13b50a4640 1 -- 172.21.15.16:0/3182305160 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8009860 con 0x7f1398003b70 2024-09-19T07:43:47.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.136+0000 7f13b830a640 1 -- 172.21.15.16:0/3182305160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398003b70 msgr2=0x7f1398003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.136+0000 7f13b830a640 1 --2- 172.21.15.16:0/3182305160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398003b70 0x7f1398003f70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009c90 tx=0x7f1398005830 comp rx=0 tx=0).stop 2024-09-19T07:43:47.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.136+0000 7f13b68a7640 1 -- 172.21.15.16:0/3182305160 reap_dead start 2024-09-19T07:43:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.136+0000 7f13b830a640 1 -- 172.21.15.16:0/3182305160 shutdown_connections 2024-09-19T07:43:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.136+0000 7f13b830a640 1 -- 172.21.15.16:0/3182305160 >> 172.21.15.16:0/3182305160 conn(0x7f13b009d760 msgr2=0x7f13b008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.136+0000 7f13b830a640 1 -- 172.21.15.16:0/3182305160 shutdown_connections 2024-09-19T07:43:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.136+0000 7f13b830a640 1 -- 172.21.15.16:0/3182305160 wait complete. 2024-09-19T07:43:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.137+0000 7f13b830a640 1 Processor -- start 2024-09-19T07:43:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.137+0000 7f13b830a640 1 -- start start 2024-09-19T07:43:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.138+0000 7f13b830a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398003b70 0x7f13b00b9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.138+0000 7f13b830a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b00ca800 0x7f13b00cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.138+0000 7f13b830a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13b01f7b70 0x7f13b01f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.138+0000 7f13b830a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13b015f4b0 con 0x7f1398003b70 2024-09-19T07:43:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.138+0000 7f13b830a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f13b00cef60 con 0x7f13b01f7b70 2024-09-19T07:43:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.138+0000 7f13b830a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f13b015f630 con 0x7f13b00ca800 2024-09-19T07:43:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.138+0000 7f13b60a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398003b70 0x7f13b00b9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.138+0000 7f13b60a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398003b70 0x7f13b00b9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35252/0 (socket says 172.21.15.16:35252) 2024-09-19T07:43:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.138+0000 7f13b60a6640 1 -- 172.21.15.16:0/357002529 learned_addr learned my addr 172.21.15.16:0/357002529 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.138+0000 7f13b58a5640 1 --2- 172.21.15.16:0/357002529 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b00ca800 0x7f13b00cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.138+0000 7f13b68a7640 1 --2- 172.21.15.16:0/357002529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13b01f7b70 0x7f13b01f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.138+0000 7f13b60a6640 1 -- 172.21.15.16:0/357002529 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b00ca800 msgr2=0x7f13b00cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.138+0000 7f13b60a6640 1 --2- 172.21.15.16:0/357002529 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b00ca800 0x7f13b00cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.138+0000 7f13b60a6640 1 -- 172.21.15.16:0/357002529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13b01f7b70 msgr2=0x7f13b01f7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.138+0000 7f13b60a6640 1 --2- 172.21.15.16:0/357002529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13b01f7b70 0x7f13b01f7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.138+0000 7f13b60a6640 1 -- 172.21.15.16:0/357002529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b00b85e0 con 0x7f1398003b70 2024-09-19T07:43:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.138+0000 7f13b58a5640 1 --2- 172.21.15.16:0/357002529 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b00ca800 0x7f13b00cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.139+0000 7f13b60a6640 1 --2- 172.21.15.16:0/357002529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398003b70 0x7f13b00b9680 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f13a8000c00 tx=0x7f13a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.139+0000 7f13aeffd640 1 -- 172.21.15.16:0/357002529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8040070 con 0x7f1398003b70 2024-09-19T07:43:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.139+0000 7f13aeffd640 1 -- 172.21.15.16:0/357002529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13a80093e0 con 0x7f1398003b70 2024-09-19T07:43:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.139+0000 7f13b830a640 1 -- 172.21.15.16:0/357002529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b00b88c0 con 0x7f1398003b70 2024-09-19T07:43:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.139+0000 7f13aeffd640 1 -- 172.21.15.16:0/357002529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8036040 con 0x7f1398003b70 2024-09-19T07:43:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.139+0000 7f13b830a640 1 -- 172.21.15.16:0/357002529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b00b8e80 con 0x7f1398003b70 2024-09-19T07:43:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.140+0000 7f13b830a640 1 -- 172.21.15.16:0/357002529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13b00d1400 con 0x7f1398003b70 2024-09-19T07:43:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.140+0000 7f13aeffd640 1 -- 172.21.15.16:0/357002529 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f13a8043070 con 0x7f1398003b70 2024-09-19T07:43:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.141+0000 7f13aeffd640 1 --2- 172.21.15.16:0/357002529 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f139c06c2e0 0x7f139c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.141+0000 7f13aeffd640 1 -- 172.21.15.16:0/357002529 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f13a80abe00 con 0x7f1398003b70 2024-09-19T07:43:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.141+0000 7f13b58a5640 1 --2- 172.21.15.16:0/357002529 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f139c06c2e0 0x7f139c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.142+0000 7f13b58a5640 1 --2- 172.21.15.16:0/357002529 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f139c06c2e0 0x7f139c06e7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f13a4007ae0 tx=0x7f13a4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:47.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.145+0000 7f13aeffd640 1 -- 172.21.15.16:0/357002529 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f13a8079b40 con 0x7f1398003b70 2024-09-19T07:43:47.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.333+0000 7f13b830a640 1 -- 172.21.15.16:0/357002529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f13b006cf30 con 0x7f1398003b70 2024-09-19T07:43:47.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.333+0000 7f13aeffd640 1 -- 172.21.15.16:0/357002529 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f13a807de40 con 0x7f1398003b70 2024-09-19T07:43:47.334 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:43:47.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.336+0000 7f13b830a640 1 -- 172.21.15.16:0/357002529 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f139c06c2e0 msgr2=0x7f139c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.336+0000 7f13b830a640 1 --2- 172.21.15.16:0/357002529 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f139c06c2e0 0x7f139c06e7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f13a4007ae0 tx=0x7f13a4009040 comp rx=0 tx=0).stop 2024-09-19T07:43:47.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.336+0000 7f13b830a640 1 -- 172.21.15.16:0/357002529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398003b70 msgr2=0x7f13b00b9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.336+0000 7f13b830a640 1 --2- 172.21.15.16:0/357002529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398003b70 0x7f13b00b9680 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f13a8000c00 tx=0x7f13a8004730 comp rx=0 tx=0).stop 2024-09-19T07:43:47.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.336+0000 7f13b830a640 1 -- 172.21.15.16:0/357002529 shutdown_connections 2024-09-19T07:43:47.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.336+0000 7f13b830a640 1 --2- 172.21.15.16:0/357002529 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f139c06c2e0 0x7f139c06e7a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.336+0000 7f13b830a640 1 --2- 172.21.15.16:0/357002529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13b01f7b70 0x7f13b01f7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.336+0000 7f13b830a640 1 --2- 172.21.15.16:0/357002529 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b00ca800 0x7f13b00cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.336+0000 7f13b830a640 1 --2- 172.21.15.16:0/357002529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398003b70 0x7f13b00b9680 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.336+0000 7f13b830a640 1 -- 172.21.15.16:0/357002529 >> 172.21.15.16:0/357002529 conn(0x7f13b009d760 msgr2=0x7f13b006e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:47.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.336+0000 7f13b830a640 1 -- 172.21.15.16:0/357002529 shutdown_connections 2024-09-19T07:43:47.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.336+0000 7f13b830a640 1 -- 172.21.15.16:0/357002529 wait complete. 2024-09-19T07:43:47.347 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-19T07:43:47.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:47.347 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-09-19T07:43:47.373 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:43:47.373 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:43:47.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:47.373 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T07:43:47.429 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:43:47.429 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:43:47.447 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:43:47.447 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:43:47.504 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:43:47.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.607+0000 7f767a14b640 1 Processor -- start 2024-09-19T07:43:47.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.607+0000 7f767a14b640 1 -- start start 2024-09-19T07:43:47.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.608+0000 7f767a14b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7674141680 0x7f76740cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.608+0000 7f767a14b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f76740d0370 0x7f7674141a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.608+0000 7f767a14b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f767415d710 con 0x7f7674141fa0 2024-09-19T07:43:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.608+0000 7f767a14b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f76740d07e0 con 0x7f7674141680 2024-09-19T07:43:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.608+0000 7f767a14b640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f767415c6f0 con 0x7f76740d0370 2024-09-19T07:43:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.608+0000 7f76737fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7674141680 0x7f76740cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.608+0000 7f7673fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7674141fa0 0x7f76740cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37798/0 (socket says 172.21.15.16:37798) 2024-09-19T07:43:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.608+0000 7f7673fff640 1 -- 172.21.15.16:0/1928116742 learned_addr learned my addr 172.21.15.16:0/1928116742 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.608+0000 7f7672ffd640 1 --2- 172.21.15.16:0/1928116742 >> v2:172.21.15.146:3300/0 conn(0x7f76740d0370 0x7f7674141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.608+0000 7f76737fe640 1 -- 172.21.15.16:0/1928116742 >> v2:172.21.15.146:3300/0 conn(0x7f76740d0370 msgr2=0x7f7674141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.608+0000 7f76737fe640 1 --2- 172.21.15.16:0/1928116742 >> v2:172.21.15.146:3300/0 conn(0x7f76740d0370 0x7f7674141a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.608+0000 7f76737fe640 1 -- 172.21.15.16:0/1928116742 >> v1:172.21.15.16:6789/0 conn(0x7f7674141fa0 legacy=0x7f76740cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.608+0000 7f76737fe640 1 -- 172.21.15.16:0/1928116742 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767415c950 con 0x7f7674141680 2024-09-19T07:43:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.608+0000 7f76727fc640 1 -- 172.21.15.16:0/1928116742 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2247450440 0 0) 0x7f767415d710 con 0x7f7674141fa0 2024-09-19T07:43:47.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.609+0000 7f76737fe640 1 --2- 172.21.15.16:0/1928116742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7674141680 0x7f76740cfe30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7674071fc0 tx=0x7f766402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2f928493bd88b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:47.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.609+0000 7f76727fc640 1 -- 172.21.15.16:0/1928116742 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664037070 con 0x7f7674141680 2024-09-19T07:43:47.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.609+0000 7f76727fc640 1 -- 172.21.15.16:0/1928116742 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7664032440 con 0x7f7674141680 2024-09-19T07:43:47.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.609+0000 7f76727fc640 1 -- 172.21.15.16:0/1928116742 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664032810 con 0x7f7674141680 2024-09-19T07:43:47.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.610+0000 7f767a14b640 1 -- 172.21.15.16:0/1928116742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7674141680 msgr2=0x7f76740cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.610+0000 7f767a14b640 1 --2- 172.21.15.16:0/1928116742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7674141680 0x7f76740cfe30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7674071fc0 tx=0x7f766402a1e0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.610+0000 7f767a14b640 1 -- 172.21.15.16:0/1928116742 shutdown_connections 2024-09-19T07:43:47.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.610+0000 7f767a14b640 1 --2- 172.21.15.16:0/1928116742 >> v2:172.21.15.146:3300/0 conn(0x7f76740d0370 0x7f7674141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.610+0000 7f767a14b640 1 --2- 172.21.15.16:0/1928116742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7674141680 0x7f76740cfe30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.610+0000 7f767a14b640 1 -- 172.21.15.16:0/1928116742 >> 172.21.15.16:0/1928116742 conn(0x7f76740b3ed0 msgr2=0x7f76740b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:47.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.610+0000 7f767a14b640 1 -- 172.21.15.16:0/1928116742 shutdown_connections 2024-09-19T07:43:47.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.610+0000 7f767a14b640 1 -- 172.21.15.16:0/1928116742 wait complete. 2024-09-19T07:43:47.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.611+0000 7f767a14b640 1 Processor -- start 2024-09-19T07:43:47.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.611+0000 7f767a14b640 1 -- start start 2024-09-19T07:43:47.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.612+0000 7f767a14b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76740d0370 0x7f76740b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.612+0000 7f767a14b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7674141680 0x7f76740b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.612+0000 7f767a14b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7674141fa0 0x7f7674154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.612+0000 7f767a14b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f767415d1f0 con 0x7f7674141fa0 2024-09-19T07:43:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.612+0000 7f767a14b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f767415cf10 con 0x7f7674141680 2024-09-19T07:43:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.612+0000 7f767a14b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f76741408c0 con 0x7f76740d0370 2024-09-19T07:43:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.612+0000 7f7673fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7674141fa0 0x7f7674154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.612+0000 7f7673fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7674141fa0 0x7f7674154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35256/0 (socket says 172.21.15.16:35256) 2024-09-19T07:43:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.612+0000 7f7673fff640 1 -- 172.21.15.16:0/1148722990 learned_addr learned my addr 172.21.15.16:0/1148722990 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.612+0000 7f76737fe640 1 --2- 172.21.15.16:0/1148722990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76740d0370 0x7f76740b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.612+0000 7f7672ffd640 1 --2- 172.21.15.16:0/1148722990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7674141680 0x7f76740b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.612+0000 7f7673fff640 1 -- 172.21.15.16:0/1148722990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76740d0370 msgr2=0x7f76740b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.612+0000 7f7673fff640 1 --2- 172.21.15.16:0/1148722990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76740d0370 0x7f76740b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.612+0000 7f7673fff640 1 -- 172.21.15.16:0/1148722990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7674141680 msgr2=0x7f76740b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.612+0000 7f7673fff640 1 --2- 172.21.15.16:0/1148722990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7674141680 0x7f76740b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.612+0000 7f7673fff640 1 -- 172.21.15.16:0/1148722990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76740b58d0 con 0x7f7674141fa0 2024-09-19T07:43:47.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.613+0000 7f7673fff640 1 --2- 172.21.15.16:0/1148722990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7674141fa0 0x7f7674154bf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f766c00c780 tx=0x7f766c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:47.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.613+0000 7f765bfff640 1 -- 172.21.15.16:0/1148722990 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c018070 con 0x7f7674141fa0 2024-09-19T07:43:47.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.613+0000 7f765bfff640 1 -- 172.21.15.16:0/1148722990 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f766c00a640 con 0x7f7674141fa0 2024-09-19T07:43:47.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.613+0000 7f767a14b640 1 -- 172.21.15.16:0/1148722990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76740b5bb0 con 0x7f7674141fa0 2024-09-19T07:43:47.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.613+0000 7f765bfff640 1 -- 172.21.15.16:0/1148722990 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c00aa30 con 0x7f7674141fa0 2024-09-19T07:43:47.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.613+0000 7f767a14b640 1 -- 172.21.15.16:0/1148722990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76740b6170 con 0x7f7674141fa0 2024-09-19T07:43:47.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.614+0000 7f767a14b640 1 -- 172.21.15.16:0/1148722990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76741423a0 con 0x7f7674141fa0 2024-09-19T07:43:47.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.614+0000 7f765bfff640 1 -- 172.21.15.16:0/1148722990 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f766c01b070 con 0x7f7674141fa0 2024-09-19T07:43:47.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.615+0000 7f765bfff640 1 --2- 172.21.15.16:0/1148722990 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f766006c2b0 0x7f766006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.615+0000 7f765bfff640 1 -- 172.21.15.16:0/1148722990 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f766c08ad80 con 0x7f7674141fa0 2024-09-19T07:43:47.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.615+0000 7f76737fe640 1 --2- 172.21.15.16:0/1148722990 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f766006c2b0 0x7f766006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.615+0000 7f76737fe640 1 --2- 172.21.15.16:0/1148722990 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f766006c2b0 0x7f766006e770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7664034000 tx=0x7f766402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:47.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.619+0000 7f765bfff640 1 -- 172.21.15.16:0/1148722990 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f766c058aa0 con 0x7f7674141fa0 2024-09-19T07:43:47.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.806+0000 7f767a14b640 1 -- 172.21.15.16:0/1148722990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f767413f590 con 0x7f7674141fa0 2024-09-19T07:43:47.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.806+0000 7f765bfff640 1 -- 172.21.15.16:0/1148722990 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f766c05cda0 con 0x7f7674141fa0 2024-09-19T07:43:47.807 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:43:47.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.808+0000 7f767a14b640 1 -- 172.21.15.16:0/1148722990 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f766006c2b0 msgr2=0x7f766006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.808+0000 7f767a14b640 1 --2- 172.21.15.16:0/1148722990 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f766006c2b0 0x7f766006e770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7664034000 tx=0x7f766402e000 comp rx=0 tx=0).stop 2024-09-19T07:43:47.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.808+0000 7f767a14b640 1 -- 172.21.15.16:0/1148722990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7674141fa0 msgr2=0x7f7674154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.808+0000 7f767a14b640 1 --2- 172.21.15.16:0/1148722990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7674141fa0 0x7f7674154bf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f766c00c780 tx=0x7f766c00cc40 comp rx=0 tx=0).stop 2024-09-19T07:43:47.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.808+0000 7f767a14b640 1 -- 172.21.15.16:0/1148722990 shutdown_connections 2024-09-19T07:43:47.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.808+0000 7f767a14b640 1 --2- 172.21.15.16:0/1148722990 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f766006c2b0 0x7f766006e770 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.808+0000 7f767a14b640 1 --2- 172.21.15.16:0/1148722990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7674141fa0 0x7f7674154bf0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.808+0000 7f767a14b640 1 --2- 172.21.15.16:0/1148722990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7674141680 0x7f76740b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.808+0000 7f767a14b640 1 --2- 172.21.15.16:0/1148722990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76740d0370 0x7f76740b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.808+0000 7f767a14b640 1 -- 172.21.15.16:0/1148722990 >> 172.21.15.16:0/1148722990 conn(0x7f76740b3ed0 msgr2=0x7f767415ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:47.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.808+0000 7f767a14b640 1 -- 172.21.15.16:0/1148722990 shutdown_connections 2024-09-19T07:43:47.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.808+0000 7f767a14b640 1 -- 172.21.15.16:0/1148722990 wait complete. 2024-09-19T07:43:47.820 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:43:47.820 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:43:47.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:43:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.942+0000 7f81376a4640 1 Processor -- start 2024-09-19T07:43:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.942+0000 7f81376a4640 1 -- start start 2024-09-19T07:43:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.943+0000 7f81376a4640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f81300d0070 0x7f813015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.943+0000 7f81376a4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81300d4130 0x7f81300d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.943+0000 7f81376a4640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81300d1b10 con 0x7f813015f0d0 2024-09-19T07:43:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.943+0000 7f81376a4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f81300cf2b0 con 0x7f81300d4130 2024-09-19T07:43:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.943+0000 7f81376a4640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f81300cf590 con 0x7f81300d0070 2024-09-19T07:43:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.943+0000 7f8135c41640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81300d4130 0x7f81300d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.943+0000 7f8135c41640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81300d4130 0x7f81300d4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35272/0 (socket says 172.21.15.16:35272) 2024-09-19T07:43:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.943+0000 7f8135c41640 1 -- 172.21.15.16:0/389911300 learned_addr learned my addr 172.21.15.16:0/389911300 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.943+0000 7f8134c3f640 1 --2- 172.21.15.16:0/389911300 >> v2:172.21.15.146:3300/0 conn(0x7f81300d0070 0x7f813015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.943+0000 7f8135c41640 1 -- 172.21.15.16:0/389911300 >> v2:172.21.15.146:3300/0 conn(0x7f81300d0070 msgr2=0x7f813015f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.943+0000 7f8135c41640 1 --2- 172.21.15.16:0/389911300 >> v2:172.21.15.146:3300/0 conn(0x7f81300d0070 0x7f813015f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.943+0000 7f8135c41640 1 -- 172.21.15.16:0/389911300 >> v1:172.21.15.146:6789/0 conn(0x7f813015f0d0 legacy=0x7f81300d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.943+0000 7f8135c41640 1 -- 172.21.15.16:0/389911300 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81300cf870 con 0x7f81300d4130 2024-09-19T07:43:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.944+0000 7f8135c41640 1 --2- 172.21.15.16:0/389911300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81300d4130 0x7f81300d4560 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8130162fc0 tx=0x7f812802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17eb08882f793cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.944+0000 7f8127fff640 1 -- 172.21.15.16:0/389911300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128036070 con 0x7f81300d4130 2024-09-19T07:43:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.944+0000 7f8127fff640 1 -- 172.21.15.16:0/389911300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8128031600 con 0x7f81300d4130 2024-09-19T07:43:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.944+0000 7f8127fff640 1 -- 172.21.15.16:0/389911300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81280319d0 con 0x7f81300d4130 2024-09-19T07:43:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.945+0000 7f81376a4640 1 -- 172.21.15.16:0/389911300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81300d4130 msgr2=0x7f81300d4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.945+0000 7f81376a4640 1 --2- 172.21.15.16:0/389911300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81300d4130 0x7f81300d4560 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8130162fc0 tx=0x7f812802a6a0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.945+0000 7f81376a4640 1 -- 172.21.15.16:0/389911300 shutdown_connections 2024-09-19T07:43:47.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.945+0000 7f81376a4640 1 --2- 172.21.15.16:0/389911300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81300d4130 0x7f81300d4560 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.945+0000 7f81376a4640 1 --2- 172.21.15.16:0/389911300 >> v2:172.21.15.146:3300/0 conn(0x7f81300d0070 0x7f813015f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.945+0000 7f81376a4640 1 -- 172.21.15.16:0/389911300 >> 172.21.15.16:0/389911300 conn(0x7f813009d760 msgr2=0x7f813008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:47.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.945+0000 7f81376a4640 1 -- 172.21.15.16:0/389911300 shutdown_connections 2024-09-19T07:43:47.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.945+0000 7f81376a4640 1 -- 172.21.15.16:0/389911300 wait complete. 2024-09-19T07:43:47.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.946+0000 7f81376a4640 1 Processor -- start 2024-09-19T07:43:47.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.946+0000 7f81376a4640 1 -- start start 2024-09-19T07:43:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.947+0000 7f81376a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81300d0070 0x7f81301e8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.947+0000 7f81376a4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81300d4130 0x7f81300b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.947+0000 7f81376a4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f813015f0d0 0x7f813015b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.947+0000 7f81376a4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81300cef60 con 0x7f81300d0070 2024-09-19T07:43:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.947+0000 7f81376a4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f81300d4aa0 con 0x7f81300d4130 2024-09-19T07:43:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.947+0000 7f81376a4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f81300b2f70 con 0x7f813015f0d0 2024-09-19T07:43:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.947+0000 7f8135440640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81300d0070 0x7f81301e8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.947+0000 7f8135440640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81300d0070 0x7f81301e8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35274/0 (socket says 172.21.15.16:35274) 2024-09-19T07:43:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.947+0000 7f8135440640 1 -- 172.21.15.16:0/4193549416 learned_addr learned my addr 172.21.15.16:0/4193549416 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.947+0000 7f8134c3f640 1 --2- 172.21.15.16:0/4193549416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81300d4130 0x7f81300b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.947+0000 7f8135c41640 1 --2- 172.21.15.16:0/4193549416 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f813015f0d0 0x7f813015b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.947+0000 7f8135440640 1 -- 172.21.15.16:0/4193549416 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f813015f0d0 msgr2=0x7f813015b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.947+0000 7f8135440640 1 --2- 172.21.15.16:0/4193549416 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f813015f0d0 0x7f813015b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.947+0000 7f8135440640 1 -- 172.21.15.16:0/4193549416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81300d4130 msgr2=0x7f81300b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.947+0000 7f8135440640 1 --2- 172.21.15.16:0/4193549416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81300d4130 0x7f81300b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.947+0000 7f8135440640 1 -- 172.21.15.16:0/4193549416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81300cbce0 con 0x7f81300d0070 2024-09-19T07:43:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.947+0000 7f8135440640 1 --2- 172.21.15.16:0/4193549416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81300d0070 0x7f81301e8000 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f812c00c340 tx=0x7f812c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.948+0000 7f8125ffb640 1 -- 172.21.15.16:0/4193549416 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c017070 con 0x7f81300d0070 2024-09-19T07:43:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.948+0000 7f8125ffb640 1 -- 172.21.15.16:0/4193549416 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f812c0052d0 con 0x7f81300d0070 2024-09-19T07:43:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.948+0000 7f81376a4640 1 -- 172.21.15.16:0/4193549416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81300cbfc0 con 0x7f81300d0070 2024-09-19T07:43:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.948+0000 7f8125ffb640 1 -- 172.21.15.16:0/4193549416 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c003850 con 0x7f81300d0070 2024-09-19T07:43:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.948+0000 7f81376a4640 1 -- 172.21.15.16:0/4193549416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81300cc580 con 0x7f81300d0070 2024-09-19T07:43:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.949+0000 7f81376a4640 1 -- 172.21.15.16:0/4193549416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81300d4560 con 0x7f81300d0070 2024-09-19T07:43:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.949+0000 7f8125ffb640 1 -- 172.21.15.16:0/4193549416 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f812c01a070 con 0x7f81300d0070 2024-09-19T07:43:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.950+0000 7f8125ffb640 1 --2- 172.21.15.16:0/4193549416 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f811406c2e0 0x7f811406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.950+0000 7f8125ffb640 1 -- 172.21.15.16:0/4193549416 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f812c08aa70 con 0x7f81300d0070 2024-09-19T07:43:47.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.950+0000 7f8134c3f640 1 --2- 172.21.15.16:0/4193549416 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f811406c2e0 0x7f811406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.950+0000 7f8134c3f640 1 --2- 172.21.15.16:0/4193549416 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f811406c2e0 0x7f811406e7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f81300b1300 tx=0x7f8120008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:47.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:47.954+0000 7f8125ffb640 1 -- 172.21.15.16:0/4193549416 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f812c058790 con 0x7f81300d0070 2024-09-19T07:43:48.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.142+0000 7f81376a4640 1 -- 172.21.15.16:0/4193549416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f813006cf30 con 0x7f81300d0070 2024-09-19T07:43:48.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.142+0000 7f8125ffb640 1 -- 172.21.15.16:0/4193549416 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f812c05ca90 con 0x7f81300d0070 2024-09-19T07:43:48.142 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:43:48.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.144+0000 7f81376a4640 1 -- 172.21.15.16:0/4193549416 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f811406c2e0 msgr2=0x7f811406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:48.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.144+0000 7f81376a4640 1 --2- 172.21.15.16:0/4193549416 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f811406c2e0 0x7f811406e7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f81300b1300 tx=0x7f8120008040 comp rx=0 tx=0).stop 2024-09-19T07:43:48.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.144+0000 7f81376a4640 1 -- 172.21.15.16:0/4193549416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81300d0070 msgr2=0x7f81301e8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:48.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.144+0000 7f81376a4640 1 --2- 172.21.15.16:0/4193549416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81300d0070 0x7f81301e8000 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f812c00c340 tx=0x7f812c00c800 comp rx=0 tx=0).stop 2024-09-19T07:43:48.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.144+0000 7f81376a4640 1 -- 172.21.15.16:0/4193549416 shutdown_connections 2024-09-19T07:43:48.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.144+0000 7f81376a4640 1 --2- 172.21.15.16:0/4193549416 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f811406c2e0 0x7f811406e7a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.144+0000 7f81376a4640 1 --2- 172.21.15.16:0/4193549416 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f813015f0d0 0x7f813015b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.144+0000 7f81376a4640 1 --2- 172.21.15.16:0/4193549416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81300d4130 0x7f81300b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.144+0000 7f81376a4640 1 --2- 172.21.15.16:0/4193549416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81300d0070 0x7f81301e8000 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.144+0000 7f81376a4640 1 -- 172.21.15.16:0/4193549416 >> 172.21.15.16:0/4193549416 conn(0x7f813009d760 msgr2=0x7f813006d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.144+0000 7f81376a4640 1 -- 172.21.15.16:0/4193549416 shutdown_connections 2024-09-19T07:43:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.144+0000 7f81376a4640 1 -- 172.21.15.16:0/4193549416 wait complete. 2024-09-19T07:43:48.155 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2024-09-19T07:43:48.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:48.156 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-09-19T07:43:48.186 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:43:48.186 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:43:48.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:43:48.186 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T07:43:48.245 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T07:43:48.246 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:43:48.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.347+0000 7f2864531640 1 Processor -- start 2024-09-19T07:43:48.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.347+0000 7f2864531640 1 -- start start 2024-09-19T07:43:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.348+0000 7f2864531640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f285c15bee0 0x7f285c0d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.348+0000 7f2864531640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f285c15c2c0 0x7f285c0d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.348+0000 7f2864531640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f285c0cf200 0x7f285c0cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.348+0000 7f2864531640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f285c0ceaf0 con 0x7f285c0cf200 2024-09-19T07:43:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.348+0000 7f2864531640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f285c0cdee0 con 0x7f285c15bee0 2024-09-19T07:43:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.348+0000 7f2864531640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f285c0ce1c0 con 0x7f285c15c2c0 2024-09-19T07:43:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.348+0000 7f2862ace640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f285c0cf200 0x7f285c0cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.348+0000 7f2862ace640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f285c0cf200 0x7f285c0cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35288/0 (socket says 172.21.15.16:35288) 2024-09-19T07:43:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.348+0000 7f2862ace640 1 -- 172.21.15.16:0/1268162295 learned_addr learned my addr 172.21.15.16:0/1268162295 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.348+0000 7f28622cd640 1 --2- 172.21.15.16:0/1268162295 >> v2:172.21.15.146:3300/0 conn(0x7f285c15bee0 0x7f285c0d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:48.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.348+0000 7f2861acc640 1 --2- 172.21.15.16:0/1268162295 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f285c15c2c0 0x7f285c0d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:48.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.348+0000 7f2862ace640 1 -- 172.21.15.16:0/1268162295 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f285c15c2c0 msgr2=0x7f285c0d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:48.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.348+0000 7f2862ace640 1 --2- 172.21.15.16:0/1268162295 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f285c15c2c0 0x7f285c0d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.348+0000 7f2862ace640 1 -- 172.21.15.16:0/1268162295 >> v2:172.21.15.146:3300/0 conn(0x7f285c15bee0 msgr2=0x7f285c0d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:48.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.348+0000 7f2862ace640 1 --2- 172.21.15.16:0/1268162295 >> v2:172.21.15.146:3300/0 conn(0x7f285c15bee0 0x7f285c0d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.348+0000 7f2862ace640 1 -- 172.21.15.16:0/1268162295 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285c0ce4a0 con 0x7f285c0cf200 2024-09-19T07:43:48.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.349+0000 7f2862ace640 1 --2- 172.21.15.16:0/1268162295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c0cf200 0x7f285c0cf6b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f285c0b2f40 tx=0x7f285402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a494a494a332466f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:48.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.349+0000 7f28612cb640 1 -- 172.21.15.16:0/1268162295 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2854036070 con 0x7f285c0cf200 2024-09-19T07:43:48.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.349+0000 7f28612cb640 1 -- 172.21.15.16:0/1268162295 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2854031600 con 0x7f285c0cf200 2024-09-19T07:43:48.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.349+0000 7f28612cb640 1 -- 172.21.15.16:0/1268162295 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28540319d0 con 0x7f285c0cf200 2024-09-19T07:43:48.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.350+0000 7f2864531640 1 -- 172.21.15.16:0/1268162295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c0cf200 msgr2=0x7f285c0cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:48.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.350+0000 7f2864531640 1 --2- 172.21.15.16:0/1268162295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c0cf200 0x7f285c0cf6b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f285c0b2f40 tx=0x7f285402a6a0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.350+0000 7f2864531640 1 -- 172.21.15.16:0/1268162295 shutdown_connections 2024-09-19T07:43:48.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.350+0000 7f2864531640 1 --2- 172.21.15.16:0/1268162295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c0cf200 0x7f285c0cf6b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.350+0000 7f2864531640 1 --2- 172.21.15.16:0/1268162295 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f285c15c2c0 0x7f285c0d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.350+0000 7f2864531640 1 --2- 172.21.15.16:0/1268162295 >> v2:172.21.15.146:3300/0 conn(0x7f285c15bee0 0x7f285c0d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.350+0000 7f2864531640 1 -- 172.21.15.16:0/1268162295 >> 172.21.15.16:0/1268162295 conn(0x7f285c06bdf0 msgr2=0x7f285c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:48.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.350+0000 7f2864531640 1 -- 172.21.15.16:0/1268162295 shutdown_connections 2024-09-19T07:43:48.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.350+0000 7f2864531640 1 -- 172.21.15.16:0/1268162295 wait complete. 2024-09-19T07:43:48.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.351+0000 7f2864531640 1 Processor -- start 2024-09-19T07:43:48.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.351+0000 7f2864531640 1 -- start start 2024-09-19T07:43:48.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.352+0000 7f2864531640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f285c0cf200 0x7f285c0c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.352+0000 7f2864531640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f285c15bee0 0x7f285c157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.352+0000 7f2864531640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c15c2c0 0x7f285c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.352+0000 7f2864531640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f285c15fd40 con 0x7f285c15c2c0 2024-09-19T07:43:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.352+0000 7f2864531640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f285c0cfbf0 con 0x7f285c15bee0 2024-09-19T07:43:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.352+0000 7f2864531640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f285c15fec0 con 0x7f285c0cf200 2024-09-19T07:43:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.352+0000 7f2862ace640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c15c2c0 0x7f285c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.352+0000 7f2862ace640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c15c2c0 0x7f285c073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35298/0 (socket says 172.21.15.16:35298) 2024-09-19T07:43:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.352+0000 7f2862ace640 1 -- 172.21.15.16:0/3669233256 learned_addr learned my addr 172.21.15.16:0/3669233256 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.352+0000 7f2861acc640 1 --2- 172.21.15.16:0/3669233256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f285c15bee0 0x7f285c157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.352+0000 7f28622cd640 1 --2- 172.21.15.16:0/3669233256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f285c0cf200 0x7f285c0c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.352+0000 7f2862ace640 1 -- 172.21.15.16:0/3669233256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f285c0cf200 msgr2=0x7f285c0c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.352+0000 7f2862ace640 1 --2- 172.21.15.16:0/3669233256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f285c0cf200 0x7f285c0c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.352+0000 7f2862ace640 1 -- 172.21.15.16:0/3669233256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f285c15bee0 msgr2=0x7f285c157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.352+0000 7f2862ace640 1 --2- 172.21.15.16:0/3669233256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f285c15bee0 0x7f285c157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.352+0000 7f2862ace640 1 -- 172.21.15.16:0/3669233256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285c1f2d50 con 0x7f285c15c2c0 2024-09-19T07:43:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.353+0000 7f2862ace640 1 --2- 172.21.15.16:0/3669233256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c15c2c0 0x7f285c073480 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f285402ac90 tx=0x7f2854004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.353+0000 7f2852ffd640 1 -- 172.21.15.16:0/3669233256 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285403f020 con 0x7f285c15c2c0 2024-09-19T07:43:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.353+0000 7f2852ffd640 1 -- 172.21.15.16:0/3669233256 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2854037040 con 0x7f285c15c2c0 2024-09-19T07:43:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.353+0000 7f2852ffd640 1 -- 172.21.15.16:0/3669233256 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2854036070 con 0x7f285c15c2c0 2024-09-19T07:43:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.353+0000 7f2864531640 1 -- 172.21.15.16:0/3669233256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285c0c7450 con 0x7f285c15c2c0 2024-09-19T07:43:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.353+0000 7f2864531640 1 -- 172.21.15.16:0/3669233256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f285c0c7a10 con 0x7f285c15c2c0 2024-09-19T07:43:48.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.354+0000 7f2864531640 1 -- 172.21.15.16:0/3669233256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f285c0cf6b0 con 0x7f285c15c2c0 2024-09-19T07:43:48.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.354+0000 7f2852ffd640 1 -- 172.21.15.16:0/3669233256 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2854042020 con 0x7f285c15c2c0 2024-09-19T07:43:48.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.355+0000 7f2852ffd640 1 --2- 172.21.15.16:0/3669233256 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f284006c2b0 0x7f284006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:48.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.355+0000 7f2852ffd640 1 -- 172.21.15.16:0/3669233256 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f28540abc60 con 0x7f285c15c2c0 2024-09-19T07:43:48.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.355+0000 7f28622cd640 1 --2- 172.21.15.16:0/3669233256 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f284006c2b0 0x7f284006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:48.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.356+0000 7f28622cd640 1 --2- 172.21.15.16:0/3669233256 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f284006c2b0 0x7f284006e770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f285c06f590 tx=0x7f2858008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:48.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.359+0000 7f2852ffd640 1 -- 172.21.15.16:0/3669233256 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2854049030 con 0x7f285c15c2c0 2024-09-19T07:43:48.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.579+0000 7f2864531640 1 -- 172.21.15.16:0/3669233256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f285c0b1620 con 0x7f285c15c2c0 2024-09-19T07:43:48.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.580+0000 7f2852ffd640 1 -- 172.21.15.16:0/3669233256 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5222 (secure 0 0 0) 0x7f28540ab090 con 0x7f285c15c2c0 2024-09-19T07:43:48.580 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:48.581 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-19T07:43:25:334589+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2772948330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2772948330},{"type":"v1","addr":"172.21.15.146:6835","nonce":2772948330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/340172279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":340172279},{"type":"v1","addr":"172.21.15.16:6833","nonce":340172279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1204811459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1204811459},{"type":"v1","addr":"172.21.15.16:6835","nonce":1204811459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:43:05.610038+0000","modified":"2024-09-19T07:43:24.691417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/1303703249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1303703249},{"type":"v1","addr":"172.21.15.146:6837","nonce":1303703249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-09-19T07:43:48.581 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-19T07:43:48.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.582+0000 7f2864531640 1 -- 172.21.15.16:0/3669233256 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f284006c2b0 msgr2=0x7f284006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:48.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.582+0000 7f2864531640 1 --2- 172.21.15.16:0/3669233256 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f284006c2b0 0x7f284006e770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f285c06f590 tx=0x7f2858008040 comp rx=0 tx=0).stop 2024-09-19T07:43:48.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.582+0000 7f2864531640 1 -- 172.21.15.16:0/3669233256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c15c2c0 msgr2=0x7f285c073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:48.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.582+0000 7f2864531640 1 --2- 172.21.15.16:0/3669233256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c15c2c0 0x7f285c073480 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f285402ac90 tx=0x7f2854004a40 comp rx=0 tx=0).stop 2024-09-19T07:43:48.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.582+0000 7f2864531640 1 -- 172.21.15.16:0/3669233256 shutdown_connections 2024-09-19T07:43:48.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.582+0000 7f2864531640 1 --2- 172.21.15.16:0/3669233256 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f284006c2b0 0x7f284006e770 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.582+0000 7f2864531640 1 --2- 172.21.15.16:0/3669233256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c15c2c0 0x7f285c073480 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.582+0000 7f2864531640 1 --2- 172.21.15.16:0/3669233256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f285c15bee0 0x7f285c157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.582+0000 7f2864531640 1 --2- 172.21.15.16:0/3669233256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f285c0cf200 0x7f285c0c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.582+0000 7f2864531640 1 -- 172.21.15.16:0/3669233256 >> 172.21.15.16:0/3669233256 conn(0x7f285c06bdf0 msgr2=0x7f285c06c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:48.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.582+0000 7f2864531640 1 -- 172.21.15.16:0/3669233256 shutdown_connections 2024-09-19T07:43:48.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.582+0000 7f2864531640 1 -- 172.21.15.16:0/3669233256 wait complete. 2024-09-19T07:43:48.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:43:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.696+0000 7f3b24f49640 1 Processor -- start 2024-09-19T07:43:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.696+0000 7f3b24f49640 1 -- start start 2024-09-19T07:43:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.697+0000 7f3b24f49640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3b2015e7f0 0x7f3b200cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.697+0000 7f3b24f49640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b200b3c10 con 0x7f3b200b03c0 2024-09-19T07:43:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.697+0000 7f3b24f49640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b200cb900 con 0x7f3b200cc680 2024-09-19T07:43:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.697+0000 7f3b24f49640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f3b200cc1f0 con 0x7f3b2015e7f0 2024-09-19T07:43:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.698+0000 7f3b1e59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3b2015e7f0 0x7f3b200cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.698+0000 7f3b1dd9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3b200b03c0 0x7f3b200adff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37808/0 (socket says 172.21.15.16:37808) 2024-09-19T07:43:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.698+0000 7f3b1dd9b640 1 -- 172.21.15.16:0/58086191 learned_addr learned my addr 172.21.15.16:0/58086191 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.698+0000 7f3b1e59c640 1 -- 172.21.15.16:0/58086191 >> v1:172.21.15.146:6789/0 conn(0x7f3b200cc680 legacy=0x7f3b2006bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.698+0000 7f3b1e59c640 1 -- 172.21.15.16:0/58086191 >> v1:172.21.15.16:6789/0 conn(0x7f3b200b03c0 legacy=0x7f3b200adff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.698+0000 7f3b1e59c640 1 -- 172.21.15.16:0/58086191 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2015ebd0 con 0x7f3b2015e7f0 2024-09-19T07:43:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.698+0000 7f3b1d59a640 1 -- 172.21.15.16:0/58086191 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3093340141 0 0) 0x7f3b200b3c10 con 0x7f3b200b03c0 2024-09-19T07:43:48.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.699+0000 7f3b1e59c640 1 --2- 172.21.15.16:0/58086191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b2015e7f0 0x7f3b200cd360 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3b180097a0 tx=0x7f3b1802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff65d942f1c27fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:48.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.699+0000 7f3b1d59a640 1 -- 172.21.15.16:0/58086191 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18037070 con 0x7f3b2015e7f0 2024-09-19T07:43:48.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.699+0000 7f3b1d59a640 1 -- 172.21.15.16:0/58086191 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b1802ae50 con 0x7f3b2015e7f0 2024-09-19T07:43:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.699+0000 7f3b1d59a640 1 -- 172.21.15.16:0/58086191 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18032660 con 0x7f3b2015e7f0 2024-09-19T07:43:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.699+0000 7f3b24f49640 1 -- 172.21.15.16:0/58086191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b2015e7f0 msgr2=0x7f3b200cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.699+0000 7f3b24f49640 1 --2- 172.21.15.16:0/58086191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b2015e7f0 0x7f3b200cd360 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3b180097a0 tx=0x7f3b1802a140 comp rx=0 tx=0).stop 2024-09-19T07:43:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.699+0000 7f3b24f49640 1 -- 172.21.15.16:0/58086191 shutdown_connections 2024-09-19T07:43:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.699+0000 7f3b24f49640 1 --2- 172.21.15.16:0/58086191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b2015e7f0 0x7f3b200cd360 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.699+0000 7f3b24f49640 1 -- 172.21.15.16:0/58086191 >> 172.21.15.16:0/58086191 conn(0x7f3b200b2d10 msgr2=0x7f3b200b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.699+0000 7f3b24f49640 1 -- 172.21.15.16:0/58086191 shutdown_connections 2024-09-19T07:43:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.699+0000 7f3b24f49640 1 -- 172.21.15.16:0/58086191 wait complete. 2024-09-19T07:43:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.701+0000 7f3b24f49640 1 Processor -- start 2024-09-19T07:43:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.701+0000 7f3b24f49640 1 -- start start 2024-09-19T07:43:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.701+0000 7f3b24f49640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b200b03c0 0x7f3b201618f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.701+0000 7f3b24f49640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3b200cc680 0x7f3b201547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.701+0000 7f3b24f49640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b2015e7f0 0x7f3b200c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.701+0000 7f3b24f49640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3b200ae700 con 0x7f3b200b03c0 2024-09-19T07:43:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.701+0000 7f3b24f49640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3b200cbe80 con 0x7f3b2015e7f0 2024-09-19T07:43:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.701+0000 7f3b24f49640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3b200b3830 con 0x7f3b200cc680 2024-09-19T07:43:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.702+0000 7f3b1e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b200b03c0 0x7f3b201618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.702+0000 7f3b1e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b200b03c0 0x7f3b201618f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35304/0 (socket says 172.21.15.16:35304) 2024-09-19T07:43:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.702+0000 7f3b1e59c640 1 -- 172.21.15.16:0/1190622049 learned_addr learned my addr 172.21.15.16:0/1190622049 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.702+0000 7f3b1dd9b640 1 --2- 172.21.15.16:0/1190622049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3b200cc680 0x7f3b201547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.702+0000 7f3b1ed9d640 1 --2- 172.21.15.16:0/1190622049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b2015e7f0 0x7f3b200c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.702+0000 7f3b1e59c640 1 -- 172.21.15.16:0/1190622049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3b200cc680 msgr2=0x7f3b201547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.702+0000 7f3b1e59c640 1 --2- 172.21.15.16:0/1190622049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3b200cc680 0x7f3b201547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.702+0000 7f3b1e59c640 1 -- 172.21.15.16:0/1190622049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b2015e7f0 msgr2=0x7f3b200c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.702+0000 7f3b1e59c640 1 --2- 172.21.15.16:0/1190622049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b2015e7f0 0x7f3b200c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.702+0000 7f3b1e59c640 1 -- 172.21.15.16:0/1190622049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2015f890 con 0x7f3b200b03c0 2024-09-19T07:43:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.702+0000 7f3b1ed9d640 1 --2- 172.21.15.16:0/1190622049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b2015e7f0 0x7f3b200c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.702+0000 7f3b1e59c640 1 --2- 172.21.15.16:0/1190622049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b200b03c0 0x7f3b201618f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3b18000c00 tx=0x7f3b180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.702+0000 7f3b16ffd640 1 -- 172.21.15.16:0/1190622049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18042070 con 0x7f3b200b03c0 2024-09-19T07:43:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.702+0000 7f3b16ffd640 1 -- 172.21.15.16:0/1190622049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b1803e670 con 0x7f3b200b03c0 2024-09-19T07:43:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.702+0000 7f3b24f49640 1 -- 172.21.15.16:0/1190622049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b2015fb00 con 0x7f3b200b03c0 2024-09-19T07:43:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.702+0000 7f3b16ffd640 1 -- 172.21.15.16:0/1190622049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18037040 con 0x7f3b200b03c0 2024-09-19T07:43:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.702+0000 7f3b24f49640 1 -- 172.21.15.16:0/1190622049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b201600c0 con 0x7f3b200b03c0 2024-09-19T07:43:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.703+0000 7f3b24f49640 1 -- 172.21.15.16:0/1190622049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b200cd360 con 0x7f3b200b03c0 2024-09-19T07:43:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.704+0000 7f3b16ffd640 1 -- 172.21.15.16:0/1190622049 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3b18045070 con 0x7f3b200b03c0 2024-09-19T07:43:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.704+0000 7f3b16ffd640 1 --2- 172.21.15.16:0/1190622049 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3b0006c2b0 0x7f3b0006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.704+0000 7f3b16ffd640 1 -- 172.21.15.16:0/1190622049 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f3b180abd70 con 0x7f3b200b03c0 2024-09-19T07:43:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.704+0000 7f3b1dd9b640 1 --2- 172.21.15.16:0/1190622049 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3b0006c2b0 0x7f3b0006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:48.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.705+0000 7f3b1dd9b640 1 --2- 172.21.15.16:0/1190622049 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3b0006c2b0 0x7f3b0006e770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3b200c6960 tx=0x7f3b08007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:48.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.708+0000 7f3b16ffd640 1 -- 172.21.15.16:0/1190622049 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3b18079c40 con 0x7f3b200b03c0 2024-09-19T07:43:48.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.925+0000 7f3b24f49640 1 -- 172.21.15.16:0/1190622049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b200d1bc0 con 0x7f3b200b03c0 2024-09-19T07:43:48.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.925+0000 7f3b16ffd640 1 -- 172.21.15.16:0/1190622049 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5222 (secure 0 0 0) 0x7f3b1807df40 con 0x7f3b200b03c0 2024-09-19T07:43:48.926 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:48.926 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-19T07:43:25:334589+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2772948330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2772948330},{"type":"v1","addr":"172.21.15.146:6835","nonce":2772948330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/340172279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":340172279},{"type":"v1","addr":"172.21.15.16:6833","nonce":340172279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1204811459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1204811459},{"type":"v1","addr":"172.21.15.16:6835","nonce":1204811459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:43:05.610038+0000","modified":"2024-09-19T07:43:24.691417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/1303703249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1303703249},{"type":"v1","addr":"172.21.15.146:6837","nonce":1303703249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-09-19T07:43:48.926 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-19T07:43:48.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.928+0000 7f3b24f49640 1 -- 172.21.15.16:0/1190622049 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3b0006c2b0 msgr2=0x7f3b0006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:48.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.928+0000 7f3b24f49640 1 --2- 172.21.15.16:0/1190622049 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3b0006c2b0 0x7f3b0006e770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3b200c6960 tx=0x7f3b08007de0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.928+0000 7f3b24f49640 1 -- 172.21.15.16:0/1190622049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b200b03c0 msgr2=0x7f3b201618f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:48.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.928+0000 7f3b24f49640 1 --2- 172.21.15.16:0/1190622049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b200b03c0 0x7f3b201618f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3b18000c00 tx=0x7f3b180046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.928+0000 7f3b24f49640 1 -- 172.21.15.16:0/1190622049 shutdown_connections 2024-09-19T07:43:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.928+0000 7f3b24f49640 1 --2- 172.21.15.16:0/1190622049 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3b0006c2b0 0x7f3b0006e770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.928+0000 7f3b24f49640 1 --2- 172.21.15.16:0/1190622049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b2015e7f0 0x7f3b200c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.928+0000 7f3b24f49640 1 --2- 172.21.15.16:0/1190622049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3b200cc680 0x7f3b201547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.928+0000 7f3b24f49640 1 --2- 172.21.15.16:0/1190622049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b200b03c0 0x7f3b201618f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.928+0000 7f3b24f49640 1 -- 172.21.15.16:0/1190622049 >> 172.21.15.16:0/1190622049 conn(0x7f3b200b2d10 msgr2=0x7f3b200705c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.928+0000 7f3b24f49640 1 -- 172.21.15.16:0/1190622049 shutdown_connections 2024-09-19T07:43:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:48.928+0000 7f3b24f49640 1 -- 172.21.15.16:0/1190622049 wait complete. 2024-09-19T07:43:48.939 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:43:49.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.040+0000 7fe3b0080640 1 Processor -- start 2024-09-19T07:43:49.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.041+0000 7fe3b0080640 1 -- start start 2024-09-19T07:43:49.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.041+0000 7fe3b0080640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a8161eb0 0x7fe3a81622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3b0080640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a8162b70 con 0x7fe3a806cd20 2024-09-19T07:43:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3b0080640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a8141b00 con 0x7fe3a81425e0 2024-09-19T07:43:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3b0080640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a8162e50 con 0x7fe3a8161eb0 2024-09-19T07:43:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ade1c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe3a806cd20 0x7fe3a806d100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37822/0 (socket says 172.21.15.16:37822) 2024-09-19T07:43:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ade1c640 1 -- 172.21.15.16:0/398934021 learned_addr learned my addr 172.21.15.16:0/398934021 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ace1a640 1 -- 172.21.15.16:0/398934021 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2102056710 0 0) 0x7fe3a8162b70 con 0x7fe3a806cd20 2024-09-19T07:43:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ace1a640 1 -- 172.21.15.16:0/398934021 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe394003520 con 0x7fe3a806cd20 2024-09-19T07:43:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ad61b640 1 --2- 172.21.15.16:0/398934021 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a8161eb0 0x7fe3a81622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ace1a640 1 -- 172.21.15.16:0/398934021 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1891156683 0 0) 0x7fe394003520 con 0x7fe3a806cd20 2024-09-19T07:43:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ace1a640 1 -- 172.21.15.16:0/398934021 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a8161eb0 msgr2=0x7fe3a81622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ace1a640 1 --2- 172.21.15.16:0/398934021 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a8161eb0 0x7fe3a81622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ace1a640 1 -- 172.21.15.16:0/398934021 >> v1:172.21.15.146:6789/0 conn(0x7fe3a81425e0 legacy=0x7fe3a8142a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ace1a640 1 -- 172.21.15.16:0/398934021 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a8141de0 con 0x7fe3a806cd20 2024-09-19T07:43:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ace1a640 1 -- 172.21.15.16:0/398934021 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fe3a4001670 con 0x7fe3a806cd20 2024-09-19T07:43:49.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ace1a640 1 -- 172.21.15.16:0/398934021 >> v1:172.21.15.16:6789/0 conn(0x7fe3a806cd20 legacy=0x7fe3a806d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ace1a640 1 --2- 172.21.15.16:0/398934021 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a8161eb0 0x7fe3940039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ace1a640 1 --2- 172.21.15.16:0/398934021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe394004010 0x7fe394004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ace1a640 1 --2- 172.21.15.16:0/398934021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe394004d40 0x7fe3940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ace1a640 1 -- 172.21.15.16:0/398934021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a8162e50 con 0x7fe394004010 2024-09-19T07:43:49.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ace1a640 1 -- 172.21.15.16:0/398934021 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe39400bc00 con 0x7fe394004d40 2024-09-19T07:43:49.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ace1a640 1 -- 172.21.15.16:0/398934021 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe39400be00 con 0x7fe3a8161eb0 2024-09-19T07:43:49.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ad61b640 1 --2- 172.21.15.16:0/398934021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe394004010 0x7fe394004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.042+0000 7fe3ade1c640 1 --2- 172.21.15.16:0/398934021 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a8161eb0 0x7fe3940039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.043+0000 7fe3ae61d640 1 --2- 172.21.15.16:0/398934021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe394004d40 0x7fe3940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.043+0000 7fe3ad61b640 1 -- 172.21.15.16:0/398934021 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a8161eb0 msgr2=0x7fe3940039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:49.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.043+0000 7fe3ad61b640 1 --2- 172.21.15.16:0/398934021 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a8161eb0 0x7fe3940039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.043+0000 7fe3ad61b640 1 -- 172.21.15.16:0/398934021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe394004d40 msgr2=0x7fe3940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.043+0000 7fe3ad61b640 1 --2- 172.21.15.16:0/398934021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe394004d40 0x7fe3940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.043+0000 7fe3ad61b640 1 -- 172.21.15.16:0/398934021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe394003520 con 0x7fe394004010 2024-09-19T07:43:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.043+0000 7fe3ae61d640 1 --2- 172.21.15.16:0/398934021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe394004d40 0x7fe3940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.043+0000 7fe3ad61b640 1 --2- 172.21.15.16:0/398934021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe394004010 0x7fe394004800 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe39800ba10 tx=0x7fe394005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5a724818a3fa86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.044+0000 7fe3ace1a640 1 -- 172.21.15.16:0/398934021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe398030070 con 0x7fe394004010 2024-09-19T07:43:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.044+0000 7fe3ace1a640 1 -- 172.21.15.16:0/398934021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe39802adb0 con 0x7fe394004010 2024-09-19T07:43:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.044+0000 7fe3ace1a640 1 -- 172.21.15.16:0/398934021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39800b610 con 0x7fe394004010 2024-09-19T07:43:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.044+0000 7fe3b0080640 1 -- 172.21.15.16:0/398934021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe394004010 msgr2=0x7fe394004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.044+0000 7fe3b0080640 1 --2- 172.21.15.16:0/398934021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe394004010 0x7fe394004800 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe39800ba10 tx=0x7fe394005830 comp rx=0 tx=0).stop 2024-09-19T07:43:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.044+0000 7fe3ae61d640 1 -- 172.21.15.16:0/398934021 reap_dead start 2024-09-19T07:43:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.044+0000 7fe3b0080640 1 -- 172.21.15.16:0/398934021 shutdown_connections 2024-09-19T07:43:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.044+0000 7fe3b0080640 1 -- 172.21.15.16:0/398934021 >> 172.21.15.16:0/398934021 conn(0x7fe3a80b3ed0 msgr2=0x7fe3a80b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.044+0000 7fe3b0080640 1 -- 172.21.15.16:0/398934021 shutdown_connections 2024-09-19T07:43:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.044+0000 7fe3b0080640 1 -- 172.21.15.16:0/398934021 wait complete. 2024-09-19T07:43:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.045+0000 7fe3b0080640 1 Processor -- start 2024-09-19T07:43:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.045+0000 7fe3b0080640 1 -- start start 2024-09-19T07:43:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.046+0000 7fe3b0080640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe394004010 0x7fe3a8140ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.046+0000 7fe3b0080640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a8147110 0x7fe3a8147590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.046+0000 7fe3b0080640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80c82c0 0x7fe3a80c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.046+0000 7fe3b0080640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81622f0 con 0x7fe3a80c82c0 2024-09-19T07:43:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.046+0000 7fe3b0080640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a8142150 con 0x7fe394004010 2024-09-19T07:43:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.046+0000 7fe3b0080640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a8162470 con 0x7fe3a8147110 2024-09-19T07:43:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.046+0000 7fe3ae61d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80c82c0 0x7fe3a80c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.046+0000 7fe3ae61d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80c82c0 0x7fe3a80c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35312/0 (socket says 172.21.15.16:35312) 2024-09-19T07:43:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.046+0000 7fe3ae61d640 1 -- 172.21.15.16:0/2588682423 learned_addr learned my addr 172.21.15.16:0/2588682423 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.046+0000 7fe3ade1c640 1 --2- 172.21.15.16:0/2588682423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe394004010 0x7fe3a8140ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.046+0000 7fe3ad61b640 1 --2- 172.21.15.16:0/2588682423 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a8147110 0x7fe3a8147590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.047+0000 7fe3ae61d640 1 -- 172.21.15.16:0/2588682423 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a8147110 msgr2=0x7fe3a8147590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.047+0000 7fe3ae61d640 1 --2- 172.21.15.16:0/2588682423 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a8147110 0x7fe3a8147590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.047+0000 7fe3ae61d640 1 -- 172.21.15.16:0/2588682423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe394004010 msgr2=0x7fe3a8140ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.047+0000 7fe3ae61d640 1 --2- 172.21.15.16:0/2588682423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe394004010 0x7fe3a8140ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.047+0000 7fe3ae61d640 1 -- 172.21.15.16:0/2588682423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a8144de0 con 0x7fe3a80c82c0 2024-09-19T07:43:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.047+0000 7fe3ad61b640 1 --2- 172.21.15.16:0/2588682423 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a8147110 0x7fe3a8147590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.047+0000 7fe3ae61d640 1 --2- 172.21.15.16:0/2588682423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80c82c0 0x7fe3a80c8740 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe3a806e960 tx=0x7fe3a000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.047+0000 7fe39e7fc640 1 -- 172.21.15.16:0/2588682423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0003970 con 0x7fe3a80c82c0 2024-09-19T07:43:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.047+0000 7fe39e7fc640 1 -- 172.21.15.16:0/2588682423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3a0018070 con 0x7fe3a80c82c0 2024-09-19T07:43:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.047+0000 7fe3b0080640 1 -- 172.21.15.16:0/2588682423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a81450c0 con 0x7fe3a80c82c0 2024-09-19T07:43:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.047+0000 7fe39e7fc640 1 -- 172.21.15.16:0/2588682423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0003d50 con 0x7fe3a80c82c0 2024-09-19T07:43:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.047+0000 7fe3b0080640 1 -- 172.21.15.16:0/2588682423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a8145680 con 0x7fe3a80c82c0 2024-09-19T07:43:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.048+0000 7fe3b0080640 1 -- 172.21.15.16:0/2588682423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a8162b70 con 0x7fe3a80c82c0 2024-09-19T07:43:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.048+0000 7fe39e7fc640 1 -- 172.21.15.16:0/2588682423 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe3a00050d0 con 0x7fe3a80c82c0 2024-09-19T07:43:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.049+0000 7fe39e7fc640 1 --2- 172.21.15.16:0/2588682423 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe38c06c2b0 0x7fe38c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.049+0000 7fe39e7fc640 1 -- 172.21.15.16:0/2588682423 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fe3a008b300 con 0x7fe3a80c82c0 2024-09-19T07:43:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.049+0000 7fe3ade1c640 1 --2- 172.21.15.16:0/2588682423 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe38c06c2b0 0x7fe38c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.050+0000 7fe3ade1c640 1 --2- 172.21.15.16:0/2588682423 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe38c06c2b0 0x7fe38c06e770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4001050 tx=0x7fe3a4000f80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.053+0000 7fe39e7fc640 1 -- 172.21.15.16:0/2588682423 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe3a0022020 con 0x7fe3a80c82c0 2024-09-19T07:43:49.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.234+0000 7fe3b0080640 1 -- 172.21.15.16:0/2588682423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3a80690c0 con 0x7fe3a80c82c0 2024-09-19T07:43:49.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.235+0000 7fe39e7fc640 1 -- 172.21.15.16:0/2588682423 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+19011 (secure 0 0 0) 0x7fe3a00595d0 con 0x7fe3a80c82c0 2024-09-19T07:43:49.236 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:49.236 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:43:48.401851+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:43:03.485582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:43:04.603961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:43:08.299591+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:43:49.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.237+0000 7fe3b0080640 1 -- 172.21.15.16:0/2588682423 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe38c06c2b0 msgr2=0x7fe38c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.237+0000 7fe3b0080640 1 --2- 172.21.15.16:0/2588682423 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe38c06c2b0 0x7fe38c06e770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4001050 tx=0x7fe3a4000f80 comp rx=0 tx=0).stop 2024-09-19T07:43:49.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.237+0000 7fe3b0080640 1 -- 172.21.15.16:0/2588682423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80c82c0 msgr2=0x7fe3a80c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.237+0000 7fe3b0080640 1 --2- 172.21.15.16:0/2588682423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80c82c0 0x7fe3a80c8740 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe3a806e960 tx=0x7fe3a000eb90 comp rx=0 tx=0).stop 2024-09-19T07:43:49.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.237+0000 7fe3b0080640 1 -- 172.21.15.16:0/2588682423 shutdown_connections 2024-09-19T07:43:49.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.237+0000 7fe3b0080640 1 --2- 172.21.15.16:0/2588682423 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe38c06c2b0 0x7fe38c06e770 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.237+0000 7fe3b0080640 1 --2- 172.21.15.16:0/2588682423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80c82c0 0x7fe3a80c8740 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.237+0000 7fe3b0080640 1 --2- 172.21.15.16:0/2588682423 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a8147110 0x7fe3a8147590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.237+0000 7fe3b0080640 1 --2- 172.21.15.16:0/2588682423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe394004010 0x7fe3a8140ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.237+0000 7fe3b0080640 1 -- 172.21.15.16:0/2588682423 >> 172.21.15.16:0/2588682423 conn(0x7fe3a80b3ed0 msgr2=0x7fe3a806d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:49.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.237+0000 7fe3b0080640 1 -- 172.21.15.16:0/2588682423 shutdown_connections 2024-09-19T07:43:49.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.237+0000 7fe3b0080640 1 -- 172.21.15.16:0/2588682423 wait complete. 2024-09-19T07:43:49.253 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T07:43:49.253 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:43:49.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.354+0000 7f26703fd640 1 Processor -- start 2024-09-19T07:43:49.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.354+0000 7f26703fd640 1 -- start start 2024-09-19T07:43:49.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.355+0000 7f26703fd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2668158970 0x7f2668070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.355+0000 7f26703fd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f266806d330 0x7f26680ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.355+0000 7f26703fd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2668158d50 con 0x7f26680ce870 2024-09-19T07:43:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.355+0000 7f26703fd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f26680cf2a0 con 0x7f266806d330 2024-09-19T07:43:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.355+0000 7f26703fd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f26680cf580 con 0x7f2668158970 2024-09-19T07:43:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.356+0000 7f266e99a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f266806d330 0x7f26680ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.356+0000 7f266e99a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f266806d330 0x7f26680ae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35326/0 (socket says 172.21.15.16:35326) 2024-09-19T07:43:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.356+0000 7f266e99a640 1 -- 172.21.15.16:0/2535325465 learned_addr learned my addr 172.21.15.16:0/2535325465 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.356+0000 7f266e199640 1 --2- 172.21.15.16:0/2535325465 >> v2:172.21.15.146:3300/0 conn(0x7f2668158970 0x7f2668070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.356+0000 7f266e99a640 1 -- 172.21.15.16:0/2535325465 >> v2:172.21.15.146:3300/0 conn(0x7f2668158970 msgr2=0x7f2668070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.356+0000 7f266e99a640 1 --2- 172.21.15.16:0/2535325465 >> v2:172.21.15.146:3300/0 conn(0x7f2668158970 0x7f2668070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.356+0000 7f266e99a640 1 -- 172.21.15.16:0/2535325465 >> v1:172.21.15.146:6789/0 conn(0x7f26680ce870 legacy=0x7f266806cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.356+0000 7f266e99a640 1 -- 172.21.15.16:0/2535325465 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266806fb70 con 0x7f266806d330 2024-09-19T07:43:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.357+0000 7f266e99a640 1 --2- 172.21.15.16:0/2535325465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f266806d330 0x7f26680ae650 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f26680679e0 tx=0x7f265802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760ad363a94f083c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.357+0000 7f266d197640 1 -- 172.21.15.16:0/2535325465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658036070 con 0x7f266806d330 2024-09-19T07:43:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.357+0000 7f266d197640 1 -- 172.21.15.16:0/2535325465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2658031600 con 0x7f266806d330 2024-09-19T07:43:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.357+0000 7f266d197640 1 -- 172.21.15.16:0/2535325465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26580319d0 con 0x7f266806d330 2024-09-19T07:43:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.357+0000 7f26703fd640 1 -- 172.21.15.16:0/2535325465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f266806d330 msgr2=0x7f26680ae650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.357+0000 7f26703fd640 1 --2- 172.21.15.16:0/2535325465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f266806d330 0x7f26680ae650 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f26680679e0 tx=0x7f265802a730 comp rx=0 tx=0).stop 2024-09-19T07:43:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.357+0000 7f26703fd640 1 -- 172.21.15.16:0/2535325465 shutdown_connections 2024-09-19T07:43:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.357+0000 7f26703fd640 1 --2- 172.21.15.16:0/2535325465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f266806d330 0x7f26680ae650 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.357+0000 7f26703fd640 1 --2- 172.21.15.16:0/2535325465 >> v2:172.21.15.146:3300/0 conn(0x7f2668158970 0x7f2668070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.357+0000 7f26703fd640 1 -- 172.21.15.16:0/2535325465 >> 172.21.15.16:0/2535325465 conn(0x7f2668072530 msgr2=0x7f2668072940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.358+0000 7f26703fd640 1 -- 172.21.15.16:0/2535325465 shutdown_connections 2024-09-19T07:43:49.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.358+0000 7f26703fd640 1 -- 172.21.15.16:0/2535325465 wait complete. 2024-09-19T07:43:49.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.359+0000 7f26703fd640 1 Processor -- start 2024-09-19T07:43:49.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.359+0000 7f26703fd640 1 -- start start 2024-09-19T07:43:49.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.359+0000 7f26703fd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f266806d330 0x7f2668143310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.359+0000 7f26703fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26680ce870 0x7f266815b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f26703fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2668158970 0x7f2668154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f26703fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26680d0500 con 0x7f26680ce870 2024-09-19T07:43:49.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f26703fd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f26680708b0 con 0x7f266806d330 2024-09-19T07:43:49.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f26703fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f26680d0680 con 0x7f2668158970 2024-09-19T07:43:49.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f266d998640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26680ce870 0x7f266815b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f266d998640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26680ce870 0x7f266815b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35340/0 (socket says 172.21.15.16:35340) 2024-09-19T07:43:49.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f266d998640 1 -- 172.21.15.16:0/1253661172 learned_addr learned my addr 172.21.15.16:0/1253661172 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:49.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f266e199640 1 --2- 172.21.15.16:0/1253661172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f266806d330 0x7f2668143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f266e99a640 1 --2- 172.21.15.16:0/1253661172 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2668158970 0x7f2668154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f266d998640 1 -- 172.21.15.16:0/1253661172 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2668158970 msgr2=0x7f2668154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f266d998640 1 --2- 172.21.15.16:0/1253661172 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2668158970 0x7f2668154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f266d998640 1 -- 172.21.15.16:0/1253661172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f266806d330 msgr2=0x7f2668143310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f266d998640 1 --2- 172.21.15.16:0/1253661172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f266806d330 0x7f2668143310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f266d998640 1 -- 172.21.15.16:0/1253661172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2668143850 con 0x7f26680ce870 2024-09-19T07:43:49.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f266d998640 1 --2- 172.21.15.16:0/1253661172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26680ce870 0x7f266815b1e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f265c00c430 tx=0x7f265c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:49.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f2666ffd640 1 -- 172.21.15.16:0/1253661172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c0052d0 con 0x7f26680ce870 2024-09-19T07:43:49.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f2666ffd640 1 -- 172.21.15.16:0/1253661172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f265c005490 con 0x7f26680ce870 2024-09-19T07:43:49.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f26703fd640 1 -- 172.21.15.16:0/1253661172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2668155100 con 0x7f26680ce870 2024-09-19T07:43:49.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f2666ffd640 1 -- 172.21.15.16:0/1253661172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c007260 con 0x7f26680ce870 2024-09-19T07:43:49.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.360+0000 7f26703fd640 1 -- 172.21.15.16:0/1253661172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26681556c0 con 0x7f26680ce870 2024-09-19T07:43:49.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.361+0000 7f26703fd640 1 -- 172.21.15.16:0/1253661172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26680cece0 con 0x7f26680ce870 2024-09-19T07:43:49.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.362+0000 7f2666ffd640 1 -- 172.21.15.16:0/1253661172 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f265c007400 con 0x7f26680ce870 2024-09-19T07:43:49.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.362+0000 7f2666ffd640 1 --2- 172.21.15.16:0/1253661172 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f265406c2b0 0x7f265406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.363+0000 7f2666ffd640 1 -- 172.21.15.16:0/1253661172 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f265c08ad40 con 0x7f26680ce870 2024-09-19T07:43:49.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.363+0000 7f266e199640 1 --2- 172.21.15.16:0/1253661172 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f265406c2b0 0x7f265406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.363+0000 7f266e199640 1 --2- 172.21.15.16:0/1253661172 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f265406c2b0 0x7f265406e770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2660009aa0 tx=0x7f2660009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:49.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.367+0000 7f2666ffd640 1 -- 172.21.15.16:0/1253661172 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2668143850 con 0x7f26680ce870 2024-09-19T07:43:49.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.577+0000 7f26703fd640 1 -- 172.21.15.16:0/1253661172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26680b0000 con 0x7f26680ce870 2024-09-19T07:43:49.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.577+0000 7f2666ffd640 1 -- 172.21.15.16:0/1253661172 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5222 (secure 0 0 0) 0x7f265c0589c0 con 0x7f26680ce870 2024-09-19T07:43:49.578 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:49.578 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-19T07:43:25:334589+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2772948330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2772948330},{"type":"v1","addr":"172.21.15.146:6835","nonce":2772948330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/340172279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":340172279},{"type":"v1","addr":"172.21.15.16:6833","nonce":340172279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1204811459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1204811459},{"type":"v1","addr":"172.21.15.16:6835","nonce":1204811459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:43:05.610038+0000","modified":"2024-09-19T07:43:24.691417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/1303703249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1303703249},{"type":"v1","addr":"172.21.15.146:6837","nonce":1303703249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-09-19T07:43:49.578 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-19T07:43:49.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.579+0000 7f26703fd640 1 -- 172.21.15.16:0/1253661172 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f265406c2b0 msgr2=0x7f265406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.579+0000 7f26703fd640 1 --2- 172.21.15.16:0/1253661172 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f265406c2b0 0x7f265406e770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2660009aa0 tx=0x7f2660009190 comp rx=0 tx=0).stop 2024-09-19T07:43:49.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.579+0000 7f26703fd640 1 -- 172.21.15.16:0/1253661172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26680ce870 msgr2=0x7f266815b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.579+0000 7f26703fd640 1 --2- 172.21.15.16:0/1253661172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26680ce870 0x7f266815b1e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f265c00c430 tx=0x7f265c00c8f0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.579+0000 7f26703fd640 1 -- 172.21.15.16:0/1253661172 shutdown_connections 2024-09-19T07:43:49.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.579+0000 7f26703fd640 1 --2- 172.21.15.16:0/1253661172 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f265406c2b0 0x7f265406e770 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.579+0000 7f26703fd640 1 --2- 172.21.15.16:0/1253661172 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2668158970 0x7f2668154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.579+0000 7f26703fd640 1 --2- 172.21.15.16:0/1253661172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26680ce870 0x7f266815b1e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.579+0000 7f26703fd640 1 --2- 172.21.15.16:0/1253661172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f266806d330 0x7f2668143310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.579+0000 7f26703fd640 1 -- 172.21.15.16:0/1253661172 >> 172.21.15.16:0/1253661172 conn(0x7f2668072530 msgr2=0x7f26680ae800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:49.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.579+0000 7f26703fd640 1 -- 172.21.15.16:0/1253661172 shutdown_connections 2024-09-19T07:43:49.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.579+0000 7f26703fd640 1 -- 172.21.15.16:0/1253661172 wait complete. 2024-09-19T07:43:49.591 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:43:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.692+0000 7f8203578640 1 Processor -- start 2024-09-19T07:43:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.692+0000 7f8203578640 1 -- start start 2024-09-19T07:43:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.693+0000 7f8203578640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81fc15e2a0 0x7f81fc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.693+0000 7f8203578640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f81fc15e680 0x7f81fc0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.693+0000 7f8203578640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81fc0d1880 con 0x7f81fc06d960 2024-09-19T07:43:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.693+0000 7f8203578640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f81fc0d1b60 con 0x7f81fc15e2a0 2024-09-19T07:43:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.693+0000 7f8203578640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f81fc0cf2d0 con 0x7f81fc15e680 2024-09-19T07:43:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.693+0000 7f8201314640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81fc15e2a0 0x7f81fc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.693+0000 7f8201b15640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f81fc06d960 0x7f81fc0d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37826/0 (socket says 172.21.15.16:37826) 2024-09-19T07:43:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.693+0000 7f8201b15640 1 -- 172.21.15.16:0/2014020643 learned_addr learned my addr 172.21.15.16:0/2014020643 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.694+0000 7f8200b13640 1 --2- 172.21.15.16:0/2014020643 >> v2:172.21.15.146:3300/0 conn(0x7f81fc15e680 0x7f81fc0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.694+0000 7f8201314640 1 -- 172.21.15.16:0/2014020643 >> v2:172.21.15.146:3300/0 conn(0x7f81fc15e680 msgr2=0x7f81fc0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.694+0000 7f8201314640 1 --2- 172.21.15.16:0/2014020643 >> v2:172.21.15.146:3300/0 conn(0x7f81fc15e680 0x7f81fc0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.694+0000 7f8201314640 1 -- 172.21.15.16:0/2014020643 >> v1:172.21.15.16:6789/0 conn(0x7f81fc06d960 legacy=0x7f81fc0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.694+0000 7f81fbfff640 1 -- 172.21.15.16:0/2014020643 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2894545638 0 0) 0x7f81fc0d1880 con 0x7f81fc06d960 2024-09-19T07:43:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.694+0000 7f8201314640 1 -- 172.21.15.16:0/2014020643 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc0cf5b0 con 0x7f81fc15e2a0 2024-09-19T07:43:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.694+0000 7f8201314640 1 --2- 172.21.15.16:0/2014020643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81fc15e2a0 0x7f81fc06d420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0a22a0 tx=0x7f81f402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22034eed0658bdb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.695+0000 7f81fbfff640 1 -- 172.21.15.16:0/2014020643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4037070 con 0x7f81fc15e2a0 2024-09-19T07:43:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.695+0000 7f81fbfff640 1 -- 172.21.15.16:0/2014020643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81f4032440 con 0x7f81fc15e2a0 2024-09-19T07:43:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.695+0000 7f81fbfff640 1 -- 172.21.15.16:0/2014020643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4032810 con 0x7f81fc15e2a0 2024-09-19T07:43:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.695+0000 7f8203578640 1 -- 172.21.15.16:0/2014020643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81fc15e2a0 msgr2=0x7f81fc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.695+0000 7f8203578640 1 --2- 172.21.15.16:0/2014020643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81fc15e2a0 0x7f81fc06d420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0a22a0 tx=0x7f81f402a1a0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.695+0000 7f8203578640 1 -- 172.21.15.16:0/2014020643 shutdown_connections 2024-09-19T07:43:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.695+0000 7f8203578640 1 --2- 172.21.15.16:0/2014020643 >> v2:172.21.15.146:3300/0 conn(0x7f81fc15e680 0x7f81fc0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.695+0000 7f8203578640 1 --2- 172.21.15.16:0/2014020643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81fc15e2a0 0x7f81fc06d420 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.695+0000 7f8203578640 1 -- 172.21.15.16:0/2014020643 >> 172.21.15.16:0/2014020643 conn(0x7f81fc09d760 msgr2=0x7f81fc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:49.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.695+0000 7f8203578640 1 -- 172.21.15.16:0/2014020643 shutdown_connections 2024-09-19T07:43:49.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.695+0000 7f8203578640 1 -- 172.21.15.16:0/2014020643 wait complete. 2024-09-19T07:43:49.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.696+0000 7f8203578640 1 Processor -- start 2024-09-19T07:43:49.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.697+0000 7f8203578640 1 -- start start 2024-09-19T07:43:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.697+0000 7f8203578640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc06d960 0x7f81fc15be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.697+0000 7f8203578640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81fc15e2a0 0x7f81fc0c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.697+0000 7f8203578640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81fc15e680 0x7f81fc0c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.697+0000 7f8203578640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc0d4c00 con 0x7f81fc15e680 2024-09-19T07:43:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.697+0000 7f8203578640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc0d4a80 con 0x7f81fc15e2a0 2024-09-19T07:43:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.697+0000 7f8203578640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f81fc0d0010 con 0x7f81fc06d960 2024-09-19T07:43:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.697+0000 7f8201b15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81fc15e680 0x7f81fc0c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.697+0000 7f8201b15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81fc15e680 0x7f81fc0c9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35360/0 (socket says 172.21.15.16:35360) 2024-09-19T07:43:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.697+0000 7f8201b15640 1 -- 172.21.15.16:0/677676626 learned_addr learned my addr 172.21.15.16:0/677676626 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.698+0000 7f8200b13640 1 --2- 172.21.15.16:0/677676626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81fc15e2a0 0x7f81fc0c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.698+0000 7f8201314640 1 --2- 172.21.15.16:0/677676626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc06d960 0x7f81fc15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.698+0000 7f8201b15640 1 -- 172.21.15.16:0/677676626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc06d960 msgr2=0x7f81fc15be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.698+0000 7f8201b15640 1 --2- 172.21.15.16:0/677676626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc06d960 0x7f81fc15be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.698+0000 7f8201b15640 1 -- 172.21.15.16:0/677676626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81fc15e2a0 msgr2=0x7f81fc0c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.698+0000 7f8201b15640 1 --2- 172.21.15.16:0/677676626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81fc15e2a0 0x7f81fc0c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.698+0000 7f8201b15640 1 -- 172.21.15.16:0/677676626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc0c8880 con 0x7f81fc15e680 2024-09-19T07:43:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.698+0000 7f8201314640 1 --2- 172.21.15.16:0/677676626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc06d960 0x7f81fc15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.698+0000 7f8201b15640 1 --2- 172.21.15.16:0/677676626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81fc15e680 0x7f81fc0c9920 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f81ec00c7a0 tx=0x7f81ec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.698+0000 7f81f9ffb640 1 -- 172.21.15.16:0/677676626 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec018070 con 0x7f81fc15e680 2024-09-19T07:43:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.698+0000 7f81f9ffb640 1 -- 172.21.15.16:0/677676626 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81ec00a640 con 0x7f81fc15e680 2024-09-19T07:43:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.698+0000 7f8203578640 1 -- 172.21.15.16:0/677676626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81fc0c8b60 con 0x7f81fc15e680 2024-09-19T07:43:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.698+0000 7f81f9ffb640 1 -- 172.21.15.16:0/677676626 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec00aab0 con 0x7f81fc15e680 2024-09-19T07:43:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.698+0000 7f8203578640 1 -- 172.21.15.16:0/677676626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81fc0c9120 con 0x7f81fc15e680 2024-09-19T07:43:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.699+0000 7f8203578640 1 -- 172.21.15.16:0/677676626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81fc06d420 con 0x7f81fc15e680 2024-09-19T07:43:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.700+0000 7f81f9ffb640 1 -- 172.21.15.16:0/677676626 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f81ec01b070 con 0x7f81fc15e680 2024-09-19T07:43:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.700+0000 7f81f9ffb640 1 --2- 172.21.15.16:0/677676626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f81e806c2b0 0x7f81e806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.700+0000 7f81f9ffb640 1 -- 172.21.15.16:0/677676626 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f81ec08adc0 con 0x7f81fc15e680 2024-09-19T07:43:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.700+0000 7f8201314640 1 --2- 172.21.15.16:0/677676626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f81e806c2b0 0x7f81e806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.701+0000 7f8201314640 1 --2- 172.21.15.16:0/677676626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f81e806c2b0 0x7f81e806e770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f81f4034000 tx=0x7f81f402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.704+0000 7f81f9ffb640 1 -- 172.21.15.16:0/677676626 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f81ec058ae0 con 0x7f81fc15e680 2024-09-19T07:43:49.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.917+0000 7f8203578640 1 -- 172.21.15.16:0/677676626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81fc06cf90 con 0x7f81fc15e680 2024-09-19T07:43:49.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.918+0000 7f81f9ffb640 1 -- 172.21.15.16:0/677676626 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5222 (secure 0 0 0) 0x7f81ec05cde0 con 0x7f81fc15e680 2024-09-19T07:43:49.919 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:49.919 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-19T07:43:25:334589+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2772948330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2772948330},{"type":"v1","addr":"172.21.15.146:6835","nonce":2772948330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/340172279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":340172279},{"type":"v1","addr":"172.21.15.16:6833","nonce":340172279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1204811459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1204811459},{"type":"v1","addr":"172.21.15.16:6835","nonce":1204811459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:43:05.610038+0000","modified":"2024-09-19T07:43:24.691417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/1303703249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1303703249},{"type":"v1","addr":"172.21.15.146:6837","nonce":1303703249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4278,"qdb_cluster":[4278]},"id":1}]} 2024-09-19T07:43:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-19T07:43:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.920+0000 7f8203578640 1 -- 172.21.15.16:0/677676626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f81e806c2b0 msgr2=0x7f81e806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.920+0000 7f8203578640 1 --2- 172.21.15.16:0/677676626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f81e806c2b0 0x7f81e806e770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f81f4034000 tx=0x7f81f402e000 comp rx=0 tx=0).stop 2024-09-19T07:43:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.920+0000 7f8203578640 1 -- 172.21.15.16:0/677676626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81fc15e680 msgr2=0x7f81fc0c9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.920+0000 7f8203578640 1 --2- 172.21.15.16:0/677676626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81fc15e680 0x7f81fc0c9920 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f81ec00c7a0 tx=0x7f81ec00cc60 comp rx=0 tx=0).stop 2024-09-19T07:43:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.920+0000 7f8203578640 1 -- 172.21.15.16:0/677676626 shutdown_connections 2024-09-19T07:43:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.920+0000 7f8203578640 1 --2- 172.21.15.16:0/677676626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f81e806c2b0 0x7f81e806e770 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.920+0000 7f8203578640 1 --2- 172.21.15.16:0/677676626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81fc15e680 0x7f81fc0c9920 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.920+0000 7f8203578640 1 --2- 172.21.15.16:0/677676626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81fc15e2a0 0x7f81fc0c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.920+0000 7f8203578640 1 --2- 172.21.15.16:0/677676626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc06d960 0x7f81fc15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.920+0000 7f8203578640 1 -- 172.21.15.16:0/677676626 >> 172.21.15.16:0/677676626 conn(0x7f81fc09d760 msgr2=0x7f81fc15f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.920+0000 7f8203578640 1 -- 172.21.15.16:0/677676626 shutdown_connections 2024-09-19T07:43:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:49.920+0000 7f8203578640 1 -- 172.21.15.16:0/677676626 wait complete. 2024-09-19T07:43:49.932 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:43:50.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.034+0000 7f0dc8265640 1 Processor -- start 2024-09-19T07:43:50.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.034+0000 7f0dc8265640 1 -- start start 2024-09-19T07:43:50.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.035+0000 7f0dc8265640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0dc0141680 0x7f0dc00cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:50.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.035+0000 7f0dc8265640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dc015d710 con 0x7f0dc00d0370 2024-09-19T07:43:50.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.035+0000 7f0dc8265640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dc0142600 con 0x7f0dc0142170 2024-09-19T07:43:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.035+0000 7f0dc8265640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc00d07b0 con 0x7f0dc0141680 2024-09-19T07:43:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.035+0000 7f0dc5800640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0dc00d0370 0x7f0dc0141a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37838/0 (socket says 172.21.15.16:37838) 2024-09-19T07:43:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.035+0000 7f0dc5800640 1 -- 172.21.15.16:0/4226446086 learned_addr learned my addr 172.21.15.16:0/4226446086 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.035+0000 7f0dc6001640 1 --2- 172.21.15.16:0/4226446086 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0dc0141680 0x7f0dc00cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.036+0000 7f0dc4fff640 1 -- 172.21.15.16:0/4226446086 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2827996229 0 0) 0x7f0dc015d710 con 0x7f0dc00d0370 2024-09-19T07:43:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.036+0000 7f0dc4fff640 1 -- 172.21.15.16:0/4226446086 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0dac003520 con 0x7f0dc00d0370 2024-09-19T07:43:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.036+0000 7f0dc4fff640 1 -- 172.21.15.16:0/4226446086 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 578482887 0 0) 0x7f0dc0142600 con 0x7f0dc0142170 2024-09-19T07:43:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.036+0000 7f0dc4fff640 1 -- 172.21.15.16:0/4226446086 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0dc015d710 con 0x7f0dc0142170 2024-09-19T07:43:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.036+0000 7f0dc4fff640 1 -- 172.21.15.16:0/4226446086 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 445677562 0 0) 0x7f0dac003520 con 0x7f0dc00d0370 2024-09-19T07:43:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.036+0000 7f0dc6001640 1 -- 172.21.15.16:0/4226446086 >> v1:172.21.15.146:6789/0 conn(0x7f0dc0142170 legacy=0x7f0dc013f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.036+0000 7f0dc6001640 1 -- 172.21.15.16:0/4226446086 >> v1:172.21.15.16:6789/0 conn(0x7f0dc00d0370 legacy=0x7f0dc0141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:50.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.036+0000 7f0dc6001640 1 -- 172.21.15.16:0/4226446086 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc015c950 con 0x7f0dc0141680 2024-09-19T07:43:50.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.037+0000 7f0dc6001640 1 --2- 172.21.15.16:0/4226446086 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0dc0141680 0x7f0dc00cfe30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00d0930 tx=0x7f0dbc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fca99fc2d5ccccac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:50.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.037+0000 7f0dc4fff640 1 -- 172.21.15.16:0/4226446086 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc037070 con 0x7f0dc0141680 2024-09-19T07:43:50.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.037+0000 7f0dc4fff640 1 -- 172.21.15.16:0/4226446086 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dbc02ac10 con 0x7f0dc0141680 2024-09-19T07:43:50.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.037+0000 7f0dc4fff640 1 -- 172.21.15.16:0/4226446086 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0001670 con 0x7f0dc0141680 2024-09-19T07:43:50.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.037+0000 7f0dc8265640 1 -- 172.21.15.16:0/4226446086 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0dc0141680 msgr2=0x7f0dc00cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:50.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.038+0000 7f0dc8265640 1 --2- 172.21.15.16:0/4226446086 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0dc0141680 0x7f0dc00cfe30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00d0930 tx=0x7f0dbc02a180 comp rx=0 tx=0).stop 2024-09-19T07:43:50.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.038+0000 7f0dc8265640 1 -- 172.21.15.16:0/4226446086 shutdown_connections 2024-09-19T07:43:50.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.038+0000 7f0dc8265640 1 --2- 172.21.15.16:0/4226446086 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0dc0141680 0x7f0dc00cfe30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.038+0000 7f0dc8265640 1 -- 172.21.15.16:0/4226446086 >> 172.21.15.16:0/4226446086 conn(0x7f0dc00b3ed0 msgr2=0x7f0dc00b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:50.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.038+0000 7f0dc8265640 1 -- 172.21.15.16:0/4226446086 shutdown_connections 2024-09-19T07:43:50.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.038+0000 7f0dc8265640 1 -- 172.21.15.16:0/4226446086 wait complete. 2024-09-19T07:43:50.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.039+0000 7f0dc8265640 1 Processor -- start 2024-09-19T07:43:50.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.039+0000 7f0dc8265640 1 -- start start 2024-09-19T07:43:50.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.040+0000 7f0dc8265640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0dc00d0370 0x7f0dc00b7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.040+0000 7f0dc8265640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dc0141680 0x7f0dc0155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.040+0000 7f0dc8265640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0dc0142170 0x7f0dc013e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.040+0000 7f0dc8265640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc015cf10 con 0x7f0dc0141680 2024-09-19T07:43:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.040+0000 7f0dc8265640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc015c6f0 con 0x7f0dc00d0370 2024-09-19T07:43:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.040+0000 7f0dc8265640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc015d1f0 con 0x7f0dc0142170 2024-09-19T07:43:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.040+0000 7f0dc5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dc0141680 0x7f0dc0155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.040+0000 7f0dc5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dc0141680 0x7f0dc0155dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35374/0 (socket says 172.21.15.16:35374) 2024-09-19T07:43:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.040+0000 7f0dc5800640 1 -- 172.21.15.16:0/564208079 learned_addr learned my addr 172.21.15.16:0/564208079 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.040+0000 7f0dc6001640 1 --2- 172.21.15.16:0/564208079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0dc00d0370 0x7f0dc00b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.040+0000 7f0dc6802640 1 --2- 172.21.15.16:0/564208079 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0dc0142170 0x7f0dc013e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.040+0000 7f0dc5800640 1 -- 172.21.15.16:0/564208079 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0dc0142170 msgr2=0x7f0dc013e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.040+0000 7f0dc5800640 1 --2- 172.21.15.16:0/564208079 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0dc0142170 0x7f0dc013e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.040+0000 7f0dc5800640 1 -- 172.21.15.16:0/564208079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0dc00d0370 msgr2=0x7f0dc00b7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:50.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.040+0000 7f0dc5800640 1 --2- 172.21.15.16:0/564208079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0dc00d0370 0x7f0dc00b7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.040+0000 7f0dc5800640 1 -- 172.21.15.16:0/564208079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc00c94d0 con 0x7f0dc0141680 2024-09-19T07:43:50.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.040+0000 7f0dc5800640 1 --2- 172.21.15.16:0/564208079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dc0141680 0x7f0dc0155dd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0db0000fd0 tx=0x7f0db000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:50.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.040+0000 7f0db67fc640 1 -- 172.21.15.16:0/564208079 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0017070 con 0x7f0dc0141680 2024-09-19T07:43:50.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.041+0000 7f0db67fc640 1 -- 172.21.15.16:0/564208079 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db0005ce0 con 0x7f0dc0141680 2024-09-19T07:43:50.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.041+0000 7f0dc8265640 1 -- 172.21.15.16:0/564208079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc00c97b0 con 0x7f0dc0141680 2024-09-19T07:43:50.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.041+0000 7f0db67fc640 1 -- 172.21.15.16:0/564208079 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db00072b0 con 0x7f0dc0141680 2024-09-19T07:43:50.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.041+0000 7f0dc8265640 1 -- 172.21.15.16:0/564208079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc00c9d70 con 0x7f0dc0141680 2024-09-19T07:43:50.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.042+0000 7f0dc8265640 1 -- 172.21.15.16:0/564208079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc00ce7d0 con 0x7f0dc0141680 2024-09-19T07:43:50.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.042+0000 7f0db67fc640 1 -- 172.21.15.16:0/564208079 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0db001a070 con 0x7f0dc0141680 2024-09-19T07:43:50.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.043+0000 7f0db67fc640 1 --2- 172.21.15.16:0/564208079 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0da406c2b0 0x7f0da406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:50.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.043+0000 7f0db67fc640 1 -- 172.21.15.16:0/564208079 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f0db008ab20 con 0x7f0dc0141680 2024-09-19T07:43:50.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.043+0000 7f0dc6001640 1 --2- 172.21.15.16:0/564208079 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0da406c2b0 0x7f0da406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:50.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.043+0000 7f0dc6001640 1 --2- 172.21.15.16:0/564208079 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0da406c2b0 0x7f0da406e770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc034000 tx=0x7f0dbc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:50.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.047+0000 7f0db67fc640 1 -- 172.21.15.16:0/564208079 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f0db0058840 con 0x7f0dc0141680 2024-09-19T07:43:50.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.221+0000 7f0dc8265640 1 -- 172.21.15.16:0/564208079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0dc00690c0 con 0x7f0dc0141680 2024-09-19T07:43:50.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.222+0000 7f0db67fc640 1 -- 172.21.15.16:0/564208079 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+19011 (secure 0 0 0) 0x7f0db005cb40 con 0x7f0dc0141680 2024-09-19T07:43:50.223 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:50.223 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:43:48.401851+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:43:03.485582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:43:04.603961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:43:08.299591+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:43:50.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.224+0000 7f0dc8265640 1 -- 172.21.15.16:0/564208079 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0da406c2b0 msgr2=0x7f0da406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:50.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.224+0000 7f0dc8265640 1 --2- 172.21.15.16:0/564208079 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0da406c2b0 0x7f0da406e770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc034000 tx=0x7f0dbc02e000 comp rx=0 tx=0).stop 2024-09-19T07:43:50.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.224+0000 7f0dc8265640 1 -- 172.21.15.16:0/564208079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dc0141680 msgr2=0x7f0dc0155dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:50.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.224+0000 7f0dc8265640 1 --2- 172.21.15.16:0/564208079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dc0141680 0x7f0dc0155dd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0db0000fd0 tx=0x7f0db000c960 comp rx=0 tx=0).stop 2024-09-19T07:43:50.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.224+0000 7f0dc8265640 1 -- 172.21.15.16:0/564208079 shutdown_connections 2024-09-19T07:43:50.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.224+0000 7f0dc8265640 1 --2- 172.21.15.16:0/564208079 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0da406c2b0 0x7f0da406e770 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.224+0000 7f0dc8265640 1 --2- 172.21.15.16:0/564208079 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0dc0142170 0x7f0dc013e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.224+0000 7f0dc8265640 1 --2- 172.21.15.16:0/564208079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dc0141680 0x7f0dc0155dd0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.224+0000 7f0dc8265640 1 --2- 172.21.15.16:0/564208079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0dc00d0370 0x7f0dc00b7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.224+0000 7f0dc8265640 1 -- 172.21.15.16:0/564208079 >> 172.21.15.16:0/564208079 conn(0x7f0dc00b3ed0 msgr2=0x7f0dc00b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:50.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.224+0000 7f0dc8265640 1 -- 172.21.15.16:0/564208079 shutdown_connections 2024-09-19T07:43:50.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.224+0000 7f0dc8265640 1 -- 172.21.15.16:0/564208079 wait complete. 2024-09-19T07:43:50.236 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T07:43:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.340+0000 7f3af2926640 1 Processor -- start 2024-09-19T07:43:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.340+0000 7f3af2926640 1 -- start start 2024-09-19T07:43:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.341+0000 7f3af2926640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3aec15f0d0 0x7f3aec15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.341+0000 7f3af2926640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3aec0d4130 0x7f3aec0d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.341+0000 7f3af2926640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aec0d4b20 con 0x7f3aec0d0070 2024-09-19T07:43:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.341+0000 7f3af2926640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f3aec0d1b10 con 0x7f3aec0d4130 2024-09-19T07:43:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.341+0000 7f3af2926640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3aec0cf2b0 con 0x7f3aec15f0d0 2024-09-19T07:43:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.341+0000 7f3af0ec3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3aec0d4130 0x7f3aec0d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.341+0000 7f3af0ec3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3aec0d4130 0x7f3aec0d45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35390/0 (socket says 172.21.15.16:35390) 2024-09-19T07:43:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.341+0000 7f3af0ec3640 1 -- 172.21.15.16:0/3422855333 learned_addr learned my addr 172.21.15.16:0/3422855333 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.342+0000 7f3aebfff640 1 --2- 172.21.15.16:0/3422855333 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3aec15f0d0 0x7f3aec15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:50.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.342+0000 7f3af0ec3640 1 -- 172.21.15.16:0/3422855333 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3aec15f0d0 msgr2=0x7f3aec15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:50.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.342+0000 7f3af0ec3640 1 --2- 172.21.15.16:0/3422855333 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3aec15f0d0 0x7f3aec15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.342+0000 7f3af0ec3640 1 -- 172.21.15.16:0/3422855333 >> v1:172.21.15.146:6789/0 conn(0x7f3aec0d0070 legacy=0x7f3aec0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:50.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.342+0000 7f3af0ec3640 1 -- 172.21.15.16:0/3422855333 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aec0cf590 con 0x7f3aec0d4130 2024-09-19T07:43:50.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.342+0000 7f3aebfff640 1 --2- 172.21.15.16:0/3422855333 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3aec15f0d0 0x7f3aec15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:50.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.342+0000 7f3af0ec3640 1 --2- 172.21.15.16:0/3422855333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0d4130 0x7f3aec0d45e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0d2fc0 tx=0x7f3ae402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59476a535a0ed8d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:50.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.343+0000 7f3aeaffd640 1 -- 172.21.15.16:0/3422855333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4036070 con 0x7f3aec0d4130 2024-09-19T07:43:50.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.343+0000 7f3aeaffd640 1 -- 172.21.15.16:0/3422855333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae4031600 con 0x7f3aec0d4130 2024-09-19T07:43:50.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.343+0000 7f3aeaffd640 1 -- 172.21.15.16:0/3422855333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae40319d0 con 0x7f3aec0d4130 2024-09-19T07:43:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.343+0000 7f3af2926640 1 -- 172.21.15.16:0/3422855333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0d4130 msgr2=0x7f3aec0d45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.343+0000 7f3af2926640 1 --2- 172.21.15.16:0/3422855333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0d4130 0x7f3aec0d45e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0d2fc0 tx=0x7f3ae402a680 comp rx=0 tx=0).stop 2024-09-19T07:43:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.343+0000 7f3af2926640 1 -- 172.21.15.16:0/3422855333 shutdown_connections 2024-09-19T07:43:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.343+0000 7f3af2926640 1 --2- 172.21.15.16:0/3422855333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0d4130 0x7f3aec0d45e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.343+0000 7f3af2926640 1 --2- 172.21.15.16:0/3422855333 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3aec15f0d0 0x7f3aec15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.343+0000 7f3af2926640 1 -- 172.21.15.16:0/3422855333 >> 172.21.15.16:0/3422855333 conn(0x7f3aec09d760 msgr2=0x7f3aec08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.343+0000 7f3af2926640 1 -- 172.21.15.16:0/3422855333 shutdown_connections 2024-09-19T07:43:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.343+0000 7f3af2926640 1 -- 172.21.15.16:0/3422855333 wait complete. 2024-09-19T07:43:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.344+0000 7f3af2926640 1 Processor -- start 2024-09-19T07:43:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.345+0000 7f3af2926640 1 -- start start 2024-09-19T07:43:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.345+0000 7f3af2926640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0d0070 0x7f3aec0cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.345+0000 7f3af2926640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3aec0d4130 0x7f3aec15b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.345+0000 7f3af2926640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3aec15f0d0 0x7f3aec0b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.345+0000 7f3af2926640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec0cef60 con 0x7f3aec0d0070 2024-09-19T07:43:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.345+0000 7f3af2926640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec0cfb50 con 0x7f3aec0d4130 2024-09-19T07:43:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.345+0000 7f3af2926640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3aec0b2f70 con 0x7f3aec15f0d0 2024-09-19T07:43:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.345+0000 7f3aebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0d0070 0x7f3aec0cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.345+0000 7f3aebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0d0070 0x7f3aec0cbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35402/0 (socket says 172.21.15.16:35402) 2024-09-19T07:43:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.345+0000 7f3aebfff640 1 -- 172.21.15.16:0/3117366265 learned_addr learned my addr 172.21.15.16:0/3117366265 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.346+0000 7f3aeb7fe640 1 --2- 172.21.15.16:0/3117366265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3aec0d4130 0x7f3aec15b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.346+0000 7f3af0ec3640 1 --2- 172.21.15.16:0/3117366265 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3aec15f0d0 0x7f3aec0b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.346+0000 7f3aebfff640 1 -- 172.21.15.16:0/3117366265 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3aec15f0d0 msgr2=0x7f3aec0b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.346+0000 7f3aebfff640 1 --2- 172.21.15.16:0/3117366265 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3aec15f0d0 0x7f3aec0b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.346+0000 7f3aebfff640 1 -- 172.21.15.16:0/3117366265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3aec0d4130 msgr2=0x7f3aec15b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.346+0000 7f3aebfff640 1 --2- 172.21.15.16:0/3117366265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3aec0d4130 0x7f3aec15b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.346+0000 7f3aebfff640 1 -- 172.21.15.16:0/3117366265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aec0cc690 con 0x7f3aec0d0070 2024-09-19T07:43:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.346+0000 7f3aebfff640 1 --2- 172.21.15.16:0/3117366265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0d0070 0x7f3aec0cbe60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00c7f0 tx=0x7f3adc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.346+0000 7f3ae8ff9640 1 -- 172.21.15.16:0/3117366265 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc015070 con 0x7f3aec0d0070 2024-09-19T07:43:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.346+0000 7f3ae8ff9640 1 -- 172.21.15.16:0/3117366265 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3adc007820 con 0x7f3aec0d0070 2024-09-19T07:43:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.346+0000 7f3af2926640 1 -- 172.21.15.16:0/3117366265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aec0cc970 con 0x7f3aec0d0070 2024-09-19T07:43:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.346+0000 7f3ae8ff9640 1 -- 172.21.15.16:0/3117366265 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc007bd0 con 0x7f3aec0d0070 2024-09-19T07:43:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.346+0000 7f3af2926640 1 -- 172.21.15.16:0/3117366265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aec0ccf30 con 0x7f3aec0d0070 2024-09-19T07:43:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.347+0000 7f3af2926640 1 -- 172.21.15.16:0/3117366265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aec0d45e0 con 0x7f3aec0d0070 2024-09-19T07:43:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.348+0000 7f3ae8ff9640 1 -- 172.21.15.16:0/3117366265 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3adc018070 con 0x7f3aec0d0070 2024-09-19T07:43:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.348+0000 7f3ae8ff9640 1 --2- 172.21.15.16:0/3117366265 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3ad806c2b0 0x7f3ad806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:50.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.348+0000 7f3ae8ff9640 1 -- 172.21.15.16:0/3117366265 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f3adc08b860 con 0x7f3aec0d0070 2024-09-19T07:43:50.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.348+0000 7f3aeb7fe640 1 --2- 172.21.15.16:0/3117366265 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3ad806c2b0 0x7f3ad806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:50.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.349+0000 7f3aeb7fe640 1 --2- 172.21.15.16:0/3117366265 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3ad806c2b0 0x7f3ad806e770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3aec1f8ad0 tx=0x7f3ae0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:50.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.352+0000 7f3ae8ff9640 1 -- 172.21.15.16:0/3117366265 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3adc059610 con 0x7f3aec0d0070 2024-09-19T07:43:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.567+0000 7f3af2926640 1 -- 172.21.15.16:0/3117366265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3aec0d0660 con 0x7f3aec0d0070 2024-09-19T07:43:50.567 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-19T07:43:50.567+0000 7f6950fea640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:43:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.569+0000 7f3ae8ff9640 1 -- 172.21.15.16:0/3117366265 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f3adc05d910 con 0x7f3aec0d0070 2024-09-19T07:43:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T07:43:50.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.572+0000 7f3af2926640 1 -- 172.21.15.16:0/3117366265 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3ad806c2b0 msgr2=0x7f3ad806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:50.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.572+0000 7f3af2926640 1 --2- 172.21.15.16:0/3117366265 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3ad806c2b0 0x7f3ad806e770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3aec1f8ad0 tx=0x7f3ae0009040 comp rx=0 tx=0).stop 2024-09-19T07:43:50.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.572+0000 7f3af2926640 1 -- 172.21.15.16:0/3117366265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0d0070 msgr2=0x7f3aec0cbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:50.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.572+0000 7f3af2926640 1 --2- 172.21.15.16:0/3117366265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0d0070 0x7f3aec0cbe60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00c7f0 tx=0x7f3adc00ccb0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.572+0000 7f3af2926640 1 -- 172.21.15.16:0/3117366265 shutdown_connections 2024-09-19T07:43:50.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.572+0000 7f3af2926640 1 --2- 172.21.15.16:0/3117366265 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3ad806c2b0 0x7f3ad806e770 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.572+0000 7f3af2926640 1 --2- 172.21.15.16:0/3117366265 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3aec15f0d0 0x7f3aec0b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.572+0000 7f3af2926640 1 --2- 172.21.15.16:0/3117366265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3aec0d4130 0x7f3aec15b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.572+0000 7f3af2926640 1 --2- 172.21.15.16:0/3117366265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3aec0d0070 0x7f3aec0cbe60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.572+0000 7f3af2926640 1 -- 172.21.15.16:0/3117366265 >> 172.21.15.16:0/3117366265 conn(0x7f3aec09d760 msgr2=0x7f3aec15d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:50.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.572+0000 7f3af2926640 1 -- 172.21.15.16:0/3117366265 shutdown_connections 2024-09-19T07:43:50.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.572+0000 7f3af2926640 1 -- 172.21.15.16:0/3117366265 wait complete. 2024-09-19T07:43:50.575 INFO:tasks.ceph.mds.d.smithi146.stderr: -3249> 2024-09-19T07:43:11.232+0000 7fa2cc324640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T07:43:50.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T07:43:50.606 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2024-09-19T07:43:50.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.688+0000 7fe9dda4c640 1 Processor -- start 2024-09-19T07:43:50.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.688+0000 7fe9dda4c640 1 -- start start 2024-09-19T07:43:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.689+0000 7fe9dda4c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9d815e2a0 0x7fe9d806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.689+0000 7fe9dda4c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe9d806d960 0x7fe9d80d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.689+0000 7fe9dda4c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9d80d1880 con 0x7fe9d815e680 2024-09-19T07:43:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.689+0000 7fe9dda4c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fe9d80d1b60 con 0x7fe9d806d960 2024-09-19T07:43:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.689+0000 7fe9dda4c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d80cf2d0 con 0x7fe9d815e2a0 2024-09-19T07:43:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.689+0000 7fe9d77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe9d806d960 0x7fe9d80d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:50.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.689+0000 7fe9d77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe9d806d960 0x7fe9d80d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35408/0 (socket says 172.21.15.16:35408) 2024-09-19T07:43:50.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.689+0000 7fe9d67fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe9d815e680 0x7fe9d80cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37846/0 (socket says 172.21.15.16:37846) 2024-09-19T07:43:50.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.689+0000 7fe9d77fe640 1 -- 172.21.15.16:0/1994316356 learned_addr learned my addr 172.21.15.16:0/1994316356 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:50.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.689+0000 7fe9d6ffd640 1 --2- 172.21.15.16:0/1994316356 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9d815e2a0 0x7fe9d806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:50.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.689+0000 7fe9d77fe640 1 -- 172.21.15.16:0/1994316356 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9d815e2a0 msgr2=0x7fe9d806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:50.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.689+0000 7fe9d77fe640 1 --2- 172.21.15.16:0/1994316356 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9d815e2a0 0x7fe9d806d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.689+0000 7fe9d77fe640 1 -- 172.21.15.16:0/1994316356 >> v1:172.21.15.16:6789/0 conn(0x7fe9d815e680 legacy=0x7fe9d80cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:50.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.689+0000 7fe9d5ffb640 1 -- 172.21.15.16:0/1994316356 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 811458334 0 0) 0x7fe9d80d1880 con 0x7fe9d815e680 2024-09-19T07:43:50.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.689+0000 7fe9d77fe640 1 -- 172.21.15.16:0/1994316356 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d80cf5b0 con 0x7fe9d806d960 2024-09-19T07:43:50.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.690+0000 7fe9d77fe640 1 --2- 172.21.15.16:0/1994316356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d806d960 0x7fe9d80d4370 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009e50 tx=0x7fe9d002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b598f385c2d6be33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.690+0000 7fe9d5ffb640 1 -- 172.21.15.16:0/1994316356 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0036070 con 0x7fe9d806d960 2024-09-19T07:43:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.690+0000 7fe9d5ffb640 1 -- 172.21.15.16:0/1994316356 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9d0031600 con 0x7fe9d806d960 2024-09-19T07:43:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.690+0000 7fe9d5ffb640 1 -- 172.21.15.16:0/1994316356 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d00319d0 con 0x7fe9d806d960 2024-09-19T07:43:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.691+0000 7fe9dda4c640 1 -- 172.21.15.16:0/1994316356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d806d960 msgr2=0x7fe9d80d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.691+0000 7fe9dda4c640 1 --2- 172.21.15.16:0/1994316356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d806d960 0x7fe9d80d4370 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009e50 tx=0x7fe9d002a730 comp rx=0 tx=0).stop 2024-09-19T07:43:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.691+0000 7fe9dda4c640 1 -- 172.21.15.16:0/1994316356 shutdown_connections 2024-09-19T07:43:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.691+0000 7fe9dda4c640 1 --2- 172.21.15.16:0/1994316356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d806d960 0x7fe9d80d4370 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.691+0000 7fe9dda4c640 1 --2- 172.21.15.16:0/1994316356 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9d815e2a0 0x7fe9d806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.691+0000 7fe9dda4c640 1 -- 172.21.15.16:0/1994316356 >> 172.21.15.16:0/1994316356 conn(0x7fe9d809d760 msgr2=0x7fe9d808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.691+0000 7fe9dda4c640 1 -- 172.21.15.16:0/1994316356 shutdown_connections 2024-09-19T07:43:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.691+0000 7fe9dda4c640 1 -- 172.21.15.16:0/1994316356 wait complete. 2024-09-19T07:43:50.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.692+0000 7fe9dda4c640 1 Processor -- start 2024-09-19T07:43:50.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.692+0000 7fe9dda4c640 1 -- start start 2024-09-19T07:43:50.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.693+0000 7fe9dda4c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9d806d960 0x7fe9d8155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:50.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.693+0000 7fe9dda4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d815e2a0 0x7fe9d80c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:50.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.693+0000 7fe9dda4c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9d815e680 0x7fe9d81592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:50.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.693+0000 7fe9dda4c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d80d0360 con 0x7fe9d815e2a0 2024-09-19T07:43:50.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.693+0000 7fe9dda4c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d80d01e0 con 0x7fe9d806d960 2024-09-19T07:43:50.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.693+0000 7fe9dda4c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d80d48b0 con 0x7fe9d815e680 2024-09-19T07:43:50.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.693+0000 7fe9d67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d815e2a0 0x7fe9d80c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:50.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.693+0000 7fe9d67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d815e2a0 0x7fe9d80c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35414/0 (socket says 172.21.15.16:35414) 2024-09-19T07:43:50.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.693+0000 7fe9d67fc640 1 -- 172.21.15.16:0/2481911450 learned_addr learned my addr 172.21.15.16:0/2481911450 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.693+0000 7fe9d67fc640 1 -- 172.21.15.16:0/2481911450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9d815e680 msgr2=0x7fe9d81592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.693+0000 7fe9d77fe640 1 --2- 172.21.15.16:0/2481911450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9d815e680 0x7fe9d81592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.693+0000 7fe9d6ffd640 1 --2- 172.21.15.16:0/2481911450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9d806d960 0x7fe9d8155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.693+0000 7fe9d67fc640 1 --2- 172.21.15.16:0/2481911450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9d815e680 0x7fe9d81592a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.693+0000 7fe9d67fc640 1 -- 172.21.15.16:0/2481911450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9d806d960 msgr2=0x7fe9d8155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.694+0000 7fe9d67fc640 1 --2- 172.21.15.16:0/2481911450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9d806d960 0x7fe9d8155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.694+0000 7fe9d67fc640 1 -- 172.21.15.16:0/2481911450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d81584e0 con 0x7fe9d815e2a0 2024-09-19T07:43:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.694+0000 7fe9d6ffd640 1 --2- 172.21.15.16:0/2481911450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9d806d960 0x7fe9d8155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.694+0000 7fe9d67fc640 1 --2- 172.21.15.16:0/2481911450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d815e2a0 0x7fe9d80c7f10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00c440 tx=0x7fe9cc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.694+0000 7fe9bffff640 1 -- 172.21.15.16:0/2481911450 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc0074e0 con 0x7fe9d815e2a0 2024-09-19T07:43:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.694+0000 7fe9bffff640 1 -- 172.21.15.16:0/2481911450 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9cc0047e0 con 0x7fe9d815e2a0 2024-09-19T07:43:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.694+0000 7fe9dda4c640 1 -- 172.21.15.16:0/2481911450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d81587c0 con 0x7fe9d815e2a0 2024-09-19T07:43:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.694+0000 7fe9bffff640 1 -- 172.21.15.16:0/2481911450 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc004bd0 con 0x7fe9d815e2a0 2024-09-19T07:43:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.694+0000 7fe9dda4c640 1 -- 172.21.15.16:0/2481911450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d81561a0 con 0x7fe9d815e2a0 2024-09-19T07:43:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.695+0000 7fe9dda4c640 1 -- 172.21.15.16:0/2481911450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d80d4370 con 0x7fe9d815e2a0 2024-09-19T07:43:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.695+0000 7fe9bffff640 1 -- 172.21.15.16:0/2481911450 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe9cc011070 con 0x7fe9d815e2a0 2024-09-19T07:43:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.696+0000 7fe9bffff640 1 --2- 172.21.15.16:0/2481911450 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe9c406c2e0 0x7fe9c406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.696+0000 7fe9bffff640 1 -- 172.21.15.16:0/2481911450 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6821+0+0 (secure 0 0 0) 0x7fe9cc08b460 con 0x7fe9d815e2a0 2024-09-19T07:43:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.696+0000 7fe9d6ffd640 1 --2- 172.21.15.16:0/2481911450 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe9c406c2e0 0x7fe9c406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.697+0000 7fe9d6ffd640 1 --2- 172.21.15.16:0/2481911450 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe9c406c2e0 0x7fe9c406e7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8006fd0 tx=0x7fe9c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.700+0000 7fe9bffff640 1 -- 172.21.15.16:0/2481911450 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe9cc059770 con 0x7fe9d815e2a0 2024-09-19T07:43:50.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:50.914+0000 7fe9dda4c640 1 -- 172.21.15.16:0/2481911450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe9d806cf90 con 0x7fe9d815e2a0 2024-09-19T07:43:51.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.570+0000 7fe9bffff640 1 -- 172.21.15.16:0/2481911450 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7fe9cc059070 con 0x7fe9d815e2a0 2024-09-19T07:43:51.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.573+0000 7fe9dda4c640 1 -- 172.21.15.16:0/2481911450 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe9c406c2e0 msgr2=0x7fe9c406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:51.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.573+0000 7fe9dda4c640 1 --2- 172.21.15.16:0/2481911450 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe9c406c2e0 0x7fe9c406e7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8006fd0 tx=0x7fe9c8008040 comp rx=0 tx=0).stop 2024-09-19T07:43:51.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.573+0000 7fe9dda4c640 1 -- 172.21.15.16:0/2481911450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d815e2a0 msgr2=0x7fe9d80c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:51.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.573+0000 7fe9dda4c640 1 --2- 172.21.15.16:0/2481911450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d815e2a0 0x7fe9d80c7f10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00c440 tx=0x7fe9cc001e70 comp rx=0 tx=0).stop 2024-09-19T07:43:51.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.573+0000 7fe9dda4c640 1 -- 172.21.15.16:0/2481911450 shutdown_connections 2024-09-19T07:43:51.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.573+0000 7fe9dda4c640 1 --2- 172.21.15.16:0/2481911450 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe9c406c2e0 0x7fe9c406e7a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:51.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.573+0000 7fe9dda4c640 1 --2- 172.21.15.16:0/2481911450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9d815e680 0x7fe9d81592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:51.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.573+0000 7fe9dda4c640 1 --2- 172.21.15.16:0/2481911450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9d815e2a0 0x7fe9d80c7f10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:51.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.573+0000 7fe9dda4c640 1 --2- 172.21.15.16:0/2481911450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9d806d960 0x7fe9d8155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:51.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.573+0000 7fe9dda4c640 1 -- 172.21.15.16:0/2481911450 >> 172.21.15.16:0/2481911450 conn(0x7fe9d809d760 msgr2=0x7fe9d81613f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:51.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.573+0000 7fe9dda4c640 1 -- 172.21.15.16:0/2481911450 shutdown_connections 2024-09-19T07:43:51.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.573+0000 7fe9dda4c640 1 -- 172.21.15.16:0/2481911450 wait complete. 2024-09-19T07:43:51.588 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T07:43:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.701+0000 7f0f89911640 1 Processor -- start 2024-09-19T07:43:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.701+0000 7f0f89911640 1 -- start start 2024-09-19T07:43:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.702+0000 7f0f89911640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f8406d430 0x7f0f841439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.702+0000 7f0f89911640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f84143f30 0x7f0f840d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.702+0000 7f0f89911640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f840c7140 con 0x7f0f840c8d20 2024-09-19T07:43:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.702+0000 7f0f89911640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f0f84141960 con 0x7f0f8406d430 2024-09-19T07:43:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.702+0000 7f0f89911640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0f84142200 con 0x7f0f84143f30 2024-09-19T07:43:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.702+0000 7f0f827fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f8406d430 0x7f0f841439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.702+0000 7f0f827fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f8406d430 0x7f0f841439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35416/0 (socket says 172.21.15.16:35416) 2024-09-19T07:43:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.702+0000 7f0f827fc640 1 -- 172.21.15.16:0/1089799115 learned_addr learned my addr 172.21.15.16:0/1089799115 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.702+0000 7f0f837fe640 1 --2- 172.21.15.16:0/1089799115 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f84143f30 0x7f0f840d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.702+0000 7f0f827fc640 1 -- 172.21.15.16:0/1089799115 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f84143f30 msgr2=0x7f0f840d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.702+0000 7f0f827fc640 1 --2- 172.21.15.16:0/1089799115 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f84143f30 0x7f0f840d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.703+0000 7f0f827fc640 1 -- 172.21.15.16:0/1089799115 >> v1:172.21.15.146:6789/0 conn(0x7f0f840c8d20 legacy=0x7f0f8406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.703+0000 7f0f827fc640 1 -- 172.21.15.16:0/1089799115 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f840c9100 con 0x7f0f8406d430 2024-09-19T07:43:51.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.703+0000 7f0f827fc640 1 --2- 172.21.15.16:0/1089799115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8406d430 0x7f0f841439f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009870 tx=0x7f0f6c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73484ed5533fc8b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:51.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.703+0000 7f0f81ffb640 1 -- 172.21.15.16:0/1089799115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c037070 con 0x7f0f8406d430 2024-09-19T07:43:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.703+0000 7f0f81ffb640 1 -- 172.21.15.16:0/1089799115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6c032440 con 0x7f0f8406d430 2024-09-19T07:43:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.703+0000 7f0f81ffb640 1 -- 172.21.15.16:0/1089799115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c032810 con 0x7f0f8406d430 2024-09-19T07:43:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.704+0000 7f0f89911640 1 -- 172.21.15.16:0/1089799115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8406d430 msgr2=0x7f0f841439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.704+0000 7f0f89911640 1 --2- 172.21.15.16:0/1089799115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8406d430 0x7f0f841439f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009870 tx=0x7f0f6c02a130 comp rx=0 tx=0).stop 2024-09-19T07:43:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.704+0000 7f0f89911640 1 -- 172.21.15.16:0/1089799115 shutdown_connections 2024-09-19T07:43:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.704+0000 7f0f89911640 1 --2- 172.21.15.16:0/1089799115 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f84143f30 0x7f0f840d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.704+0000 7f0f89911640 1 --2- 172.21.15.16:0/1089799115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8406d430 0x7f0f841439f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.704+0000 7f0f89911640 1 -- 172.21.15.16:0/1089799115 >> 172.21.15.16:0/1089799115 conn(0x7f0f840b3ed0 msgr2=0x7f0f840b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.704+0000 7f0f89911640 1 -- 172.21.15.16:0/1089799115 shutdown_connections 2024-09-19T07:43:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.704+0000 7f0f89911640 1 -- 172.21.15.16:0/1089799115 wait complete. 2024-09-19T07:43:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.705+0000 7f0f89911640 1 Processor -- start 2024-09-19T07:43:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.705+0000 7f0f89911640 1 -- start start 2024-09-19T07:43:51.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.706+0000 7f0f89911640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f8406d430 0x7f0f840b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.706+0000 7f0f89911640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f840c8d20 0x7f0f84140b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.706+0000 7f0f89911640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f84143f30 0x7f0f840d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.706+0000 7f0f89911640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84141e90 con 0x7f0f84143f30 2024-09-19T07:43:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.706+0000 7f0f89911640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84141bb0 con 0x7f0f840c8d20 2024-09-19T07:43:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.706+0000 7f0f89911640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0f84144350 con 0x7f0f8406d430 2024-09-19T07:43:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.706+0000 7f0f837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f84143f30 0x7f0f840d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.706+0000 7f0f837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f84143f30 0x7f0f840d1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35426/0 (socket says 172.21.15.16:35426) 2024-09-19T07:43:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.706+0000 7f0f837fe640 1 -- 172.21.15.16:0/323456358 learned_addr learned my addr 172.21.15.16:0/323456358 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.706+0000 7f0f82ffd640 1 --2- 172.21.15.16:0/323456358 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f8406d430 0x7f0f840b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.706+0000 7f0f827fc640 1 --2- 172.21.15.16:0/323456358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f840c8d20 0x7f0f84140b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.707+0000 7f0f837fe640 1 -- 172.21.15.16:0/323456358 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f8406d430 msgr2=0x7f0f840b6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.707+0000 7f0f837fe640 1 --2- 172.21.15.16:0/323456358 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f8406d430 0x7f0f840b6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.707+0000 7f0f837fe640 1 -- 172.21.15.16:0/323456358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f840c8d20 msgr2=0x7f0f84140b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.707+0000 7f0f837fe640 1 --2- 172.21.15.16:0/323456358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f840c8d20 0x7f0f84140b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.707+0000 7f0f837fe640 1 -- 172.21.15.16:0/323456358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f840b6a70 con 0x7f0f84143f30 2024-09-19T07:43:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.707+0000 7f0f827fc640 1 --2- 172.21.15.16:0/323456358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f840c8d20 0x7f0f84140b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.707+0000 7f0f837fe640 1 --2- 172.21.15.16:0/323456358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f84143f30 0x7f0f840d1e20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00d5d0 tx=0x7f0f7c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.707+0000 7f0f7bfff640 1 -- 172.21.15.16:0/323456358 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0125f0 con 0x7f0f84143f30 2024-09-19T07:43:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.707+0000 7f0f7bfff640 1 -- 172.21.15.16:0/323456358 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f7c019070 con 0x7f0f84143f30 2024-09-19T07:43:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.707+0000 7f0f7bfff640 1 -- 172.21.15.16:0/323456358 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0129f0 con 0x7f0f84143f30 2024-09-19T07:43:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.707+0000 7f0f89911640 1 -- 172.21.15.16:0/323456358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f840b6d50 con 0x7f0f84143f30 2024-09-19T07:43:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.707+0000 7f0f89911640 1 -- 172.21.15.16:0/323456358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f840b7310 con 0x7f0f84143f30 2024-09-19T07:43:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.708+0000 7f0f89911640 1 -- 172.21.15.16:0/323456358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f841439f0 con 0x7f0f84143f30 2024-09-19T07:43:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.708+0000 7f0f7bfff640 1 -- 172.21.15.16:0/323456358 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0f7c012b90 con 0x7f0f84143f30 2024-09-19T07:43:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.709+0000 7f0f7bfff640 1 --2- 172.21.15.16:0/323456358 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0f6406c2b0 0x7f0f6406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.709+0000 7f0f7bfff640 1 -- 172.21.15.16:0/323456358 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6821+0+0 (secure 0 0 0) 0x7f0f7c08bbf0 con 0x7f0f84143f30 2024-09-19T07:43:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.709+0000 7f0f82ffd640 1 --2- 172.21.15.16:0/323456358 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0f6406c2b0 0x7f0f6406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.710+0000 7f0f82ffd640 1 --2- 172.21.15.16:0/323456358 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0f6406c2b0 0x7f0f6406e770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0f74001e70 tx=0x7f0f74009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.713+0000 7f0f7bfff640 1 -- 172.21.15.16:0/323456358 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f0f7c059f10 con 0x7f0f84143f30 2024-09-19T07:43:51.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:51.892+0000 7f0f89911640 1 -- 172.21.15.16:0/323456358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0f841428a0 con 0x7f0f84143f30 2024-09-19T07:43:52.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.572+0000 7f0f7bfff640 1 -- 172.21.15.16:0/323456358 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) ==== 181+0+0 (secure 0 0 0) 0x7f0f7c059810 con 0x7f0f84143f30 2024-09-19T07:43:52.573 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-19T07:43:52.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.576+0000 7f0f89911640 1 -- 172.21.15.16:0/323456358 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0f6406c2b0 msgr2=0x7f0f6406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:52.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.576+0000 7f0f89911640 1 --2- 172.21.15.16:0/323456358 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0f6406c2b0 0x7f0f6406e770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0f74001e70 tx=0x7f0f74009040 comp rx=0 tx=0).stop 2024-09-19T07:43:52.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.576+0000 7f0f89911640 1 -- 172.21.15.16:0/323456358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f84143f30 msgr2=0x7f0f840d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:52.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.576+0000 7f0f89911640 1 --2- 172.21.15.16:0/323456358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f84143f30 0x7f0f840d1e20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00d5d0 tx=0x7f0f7c00da90 comp rx=0 tx=0).stop 2024-09-19T07:43:52.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.576+0000 7f0f89911640 1 -- 172.21.15.16:0/323456358 shutdown_connections 2024-09-19T07:43:52.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.576+0000 7f0f89911640 1 --2- 172.21.15.16:0/323456358 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0f6406c2b0 0x7f0f6406e770 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.576+0000 7f0f89911640 1 --2- 172.21.15.16:0/323456358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f84143f30 0x7f0f840d1e20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.576+0000 7f0f89911640 1 --2- 172.21.15.16:0/323456358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f840c8d20 0x7f0f84140b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.576+0000 7f0f89911640 1 --2- 172.21.15.16:0/323456358 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f8406d430 0x7f0f840b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.576+0000 7f0f89911640 1 -- 172.21.15.16:0/323456358 >> 172.21.15.16:0/323456358 conn(0x7f0f840b3ed0 msgr2=0x7f0f84142dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:52.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.577+0000 7f0f89911640 1 -- 172.21.15.16:0/323456358 shutdown_connections 2024-09-19T07:43:52.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.577+0000 7f0f89911640 1 -- 172.21.15.16:0/323456358 wait complete. 2024-09-19T07:43:52.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T07:43:52.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.709+0000 7f99240ae640 1 Processor -- start 2024-09-19T07:43:52.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.709+0000 7f99240ae640 1 -- start start 2024-09-19T07:43:52.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.710+0000 7f99240ae640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c0d0070 0x7f991c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.710+0000 7f99240ae640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f991c0d4130 0x7f991c0d45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.710+0000 7f99240ae640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f991c0d4b00 con 0x7f991c15f0d0 2024-09-19T07:43:52.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.710+0000 7f99240ae640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f991c0d1b10 con 0x7f991c0d4130 2024-09-19T07:43:52.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.710+0000 7f99240ae640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f991c0cf2b0 con 0x7f991c0d0070 2024-09-19T07:43:52.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.710+0000 7f9921e4a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f991c15f0d0 0x7f991c0d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56106/0 (socket says 172.21.15.16:56106) 2024-09-19T07:43:52.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.710+0000 7f9921e4a640 1 -- 172.21.15.16:0/3280974500 learned_addr learned my addr 172.21.15.16:0/3280974500 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:52.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.710+0000 7f992264b640 1 --2- 172.21.15.16:0/3280974500 >> v2:172.21.15.146:3300/0 conn(0x7f991c0d4130 0x7f991c0d45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.710+0000 7f9921649640 1 --2- 172.21.15.16:0/3280974500 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c0d0070 0x7f991c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.710+0000 7f9920e48640 1 -- 172.21.15.16:0/3280974500 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3451166346 0 0) 0x7f991c0d4b00 con 0x7f991c15f0d0 2024-09-19T07:43:52.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.710+0000 7f9920e48640 1 -- 172.21.15.16:0/3280974500 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9908003520 con 0x7f991c15f0d0 2024-09-19T07:43:52.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.710+0000 7f9921649640 1 -- 172.21.15.16:0/3280974500 >> v2:172.21.15.146:3300/0 conn(0x7f991c0d4130 msgr2=0x7f991c0d45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:52.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.710+0000 7f9921649640 1 --2- 172.21.15.16:0/3280974500 >> v2:172.21.15.146:3300/0 conn(0x7f991c0d4130 0x7f991c0d45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.710+0000 7f9921649640 1 -- 172.21.15.16:0/3280974500 >> v1:172.21.15.16:6789/0 conn(0x7f991c15f0d0 legacy=0x7f991c0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:52.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.710+0000 7f9921649640 1 -- 172.21.15.16:0/3280974500 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991c0cf590 con 0x7f991c0d0070 2024-09-19T07:43:52.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.710+0000 7f992264b640 1 --2- 172.21.15.16:0/3280974500 >> v2:172.21.15.146:3300/0 conn(0x7f991c0d4130 0x7f991c0d45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:52.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.711+0000 7f9921649640 1 --2- 172.21.15.16:0/3280974500 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c0d0070 0x7f991c15f4b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f991c0d4cd0 tx=0x7f990c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=757f813e36001856 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:52.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.711+0000 7f9920e48640 1 -- 172.21.15.16:0/3280974500 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c037070 con 0x7f991c0d0070 2024-09-19T07:43:52.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.711+0000 7f9920e48640 1 -- 172.21.15.16:0/3280974500 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f990c02ae50 con 0x7f991c0d0070 2024-09-19T07:43:52.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.711+0000 7f9920e48640 1 -- 172.21.15.16:0/3280974500 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c032640 con 0x7f991c0d0070 2024-09-19T07:43:52.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.712+0000 7f99240ae640 1 -- 172.21.15.16:0/3280974500 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c0d0070 msgr2=0x7f991c15f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:52.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.712+0000 7f99240ae640 1 --2- 172.21.15.16:0/3280974500 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c0d0070 0x7f991c15f4b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f991c0d4cd0 tx=0x7f990c02a100 comp rx=0 tx=0).stop 2024-09-19T07:43:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.712+0000 7f99240ae640 1 -- 172.21.15.16:0/3280974500 shutdown_connections 2024-09-19T07:43:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.712+0000 7f99240ae640 1 --2- 172.21.15.16:0/3280974500 >> v2:172.21.15.146:3300/0 conn(0x7f991c0d4130 0x7f991c0d45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.712+0000 7f99240ae640 1 --2- 172.21.15.16:0/3280974500 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c0d0070 0x7f991c15f4b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.712+0000 7f99240ae640 1 -- 172.21.15.16:0/3280974500 >> 172.21.15.16:0/3280974500 conn(0x7f991c09d760 msgr2=0x7f991c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.712+0000 7f99240ae640 1 -- 172.21.15.16:0/3280974500 shutdown_connections 2024-09-19T07:43:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.712+0000 7f99240ae640 1 -- 172.21.15.16:0/3280974500 wait complete. 2024-09-19T07:43:52.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.713+0000 7f99240ae640 1 Processor -- start 2024-09-19T07:43:52.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.713+0000 7f99240ae640 1 -- start start 2024-09-19T07:43:52.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.714+0000 7f99240ae640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f991c0d0070 0x7f991c14d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.714+0000 7f99240ae640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c0d4130 0x7f991c0b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.714+0000 7f99240ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f991c15f0d0 0x7f991c0c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.714+0000 7f99240ae640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f991c0cef60 con 0x7f991c15f0d0 2024-09-19T07:43:52.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.714+0000 7f99240ae640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f991c0cfb50 con 0x7f991c0d0070 2024-09-19T07:43:52.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.714+0000 7f99240ae640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f991c0b2f70 con 0x7f991c0d4130 2024-09-19T07:43:52.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.714+0000 7f992264b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f991c15f0d0 0x7f991c0c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.714+0000 7f992264b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f991c15f0d0 0x7f991c0c9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34424/0 (socket says 172.21.15.16:34424) 2024-09-19T07:43:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.714+0000 7f992264b640 1 -- 172.21.15.16:0/1926187164 learned_addr learned my addr 172.21.15.16:0/1926187164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.714+0000 7f9921e4a640 1 --2- 172.21.15.16:0/1926187164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f991c0d0070 0x7f991c14d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.714+0000 7f9921649640 1 --2- 172.21.15.16:0/1926187164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c0d4130 0x7f991c0b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.715+0000 7f992264b640 1 -- 172.21.15.16:0/1926187164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c0d4130 msgr2=0x7f991c0b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.715+0000 7f992264b640 1 --2- 172.21.15.16:0/1926187164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c0d4130 0x7f991c0b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.715+0000 7f992264b640 1 -- 172.21.15.16:0/1926187164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f991c0d0070 msgr2=0x7f991c14d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.715+0000 7f992264b640 1 --2- 172.21.15.16:0/1926187164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f991c0d0070 0x7f991c14d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.715+0000 7f992264b640 1 -- 172.21.15.16:0/1926187164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991c14dd90 con 0x7f991c15f0d0 2024-09-19T07:43:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.715+0000 7f9921e4a640 1 --2- 172.21.15.16:0/1926187164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f991c0d0070 0x7f991c14d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.715+0000 7f9921649640 1 --2- 172.21.15.16:0/1926187164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c0d4130 0x7f991c0b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.715+0000 7f992264b640 1 --2- 172.21.15.16:0/1926187164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f991c15f0d0 0x7f991c0c9d90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f991400e930 tx=0x7f991400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:52.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.715+0000 7f99127fc640 1 -- 172.21.15.16:0/1926187164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914017070 con 0x7f991c15f0d0 2024-09-19T07:43:52.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.715+0000 7f99127fc640 1 -- 172.21.15.16:0/1926187164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f991400ca00 con 0x7f991c15f0d0 2024-09-19T07:43:52.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.715+0000 7f99127fc640 1 -- 172.21.15.16:0/1926187164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991400cdd0 con 0x7f991c15f0d0 2024-09-19T07:43:52.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.715+0000 7f99240ae640 1 -- 172.21.15.16:0/1926187164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991c14e070 con 0x7f991c15f0d0 2024-09-19T07:43:52.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.715+0000 7f99240ae640 1 -- 172.21.15.16:0/1926187164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f991c0b8a40 con 0x7f991c15f0d0 2024-09-19T07:43:52.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.716+0000 7f99240ae640 1 -- 172.21.15.16:0/1926187164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f991c14e240 con 0x7f991c15f0d0 2024-09-19T07:43:52.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.717+0000 7f99127fc640 1 -- 172.21.15.16:0/1926187164 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9914013470 con 0x7f991c15f0d0 2024-09-19T07:43:52.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.717+0000 7f99127fc640 1 --2- 172.21.15.16:0/1926187164 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f990006c2b0 0x7f990006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.717+0000 7f99127fc640 1 -- 172.21.15.16:0/1926187164 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6383+0+0 (secure 0 0 0) 0x7f991408b670 con 0x7f991c15f0d0 2024-09-19T07:43:52.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.717+0000 7f9921e4a640 1 --2- 172.21.15.16:0/1926187164 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f990006c2b0 0x7f990006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.718+0000 7f9921e4a640 1 --2- 172.21.15.16:0/1926187164 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f990006c2b0 0x7f990006e770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9918005c00 tx=0x7f9918009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:52.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.722+0000 7f99127fc640 1 -- 172.21.15.16:0/1926187164 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9914059b70 con 0x7f991c15f0d0 2024-09-19T07:43:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:52.896+0000 7f99240ae640 1 -- 172.21.15.16:0/1926187164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f991c15f4b0 con 0x7f991c15f0d0 2024-09-19T07:43:53.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.581+0000 7f99127fc640 1 -- 172.21.15.16:0/1926187164 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) ==== 169+0+0 (secure 0 0 0) 0x7f9914059470 con 0x7f991c15f0d0 2024-09-19T07:43:53.582 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-19T07:43:53.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.585+0000 7f99240ae640 1 -- 172.21.15.16:0/1926187164 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f990006c2b0 msgr2=0x7f990006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:53.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.585+0000 7f99240ae640 1 --2- 172.21.15.16:0/1926187164 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f990006c2b0 0x7f990006e770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9918005c00 tx=0x7f9918009040 comp rx=0 tx=0).stop 2024-09-19T07:43:53.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.585+0000 7f99240ae640 1 -- 172.21.15.16:0/1926187164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f991c15f0d0 msgr2=0x7f991c0c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:53.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.585+0000 7f99240ae640 1 --2- 172.21.15.16:0/1926187164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f991c15f0d0 0x7f991c0c9d90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f991400e930 tx=0x7f991400edf0 comp rx=0 tx=0).stop 2024-09-19T07:43:53.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.585+0000 7f99240ae640 1 -- 172.21.15.16:0/1926187164 shutdown_connections 2024-09-19T07:43:53.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.585+0000 7f99240ae640 1 --2- 172.21.15.16:0/1926187164 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f990006c2b0 0x7f990006e770 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:53.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.585+0000 7f99240ae640 1 --2- 172.21.15.16:0/1926187164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f991c15f0d0 0x7f991c0c9d90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:53.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.585+0000 7f99240ae640 1 --2- 172.21.15.16:0/1926187164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f991c0d4130 0x7f991c0b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:53.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.585+0000 7f99240ae640 1 --2- 172.21.15.16:0/1926187164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f991c0d0070 0x7f991c14d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:53.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.585+0000 7f99240ae640 1 -- 172.21.15.16:0/1926187164 >> 172.21.15.16:0/1926187164 conn(0x7f991c09d760 msgr2=0x7f991c0d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:53.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.585+0000 7f99240ae640 1 -- 172.21.15.16:0/1926187164 shutdown_connections 2024-09-19T07:43:53.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.586+0000 7f99240ae640 1 -- 172.21.15.16:0/1926187164 wait complete. 2024-09-19T07:43:53.601 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-19T07:43:53.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.714+0000 7f4ec3242640 1 Processor -- start 2024-09-19T07:43:53.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.714+0000 7f4ec3242640 1 -- start start 2024-09-19T07:43:53.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.714+0000 7f4ec3242640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4ebc15e2a0 0x7f4ebc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:53.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.714+0000 7f4ec3242640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ebc15e680 0x7f4ebc0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:53.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.714+0000 7f4ec3242640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ebc06d960 0x7f4ebc0d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:53.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.714+0000 7f4ec3242640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f4ebc0d1880 con 0x7f4ebc15e680 2024-09-19T07:43:53.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.714+0000 7f4ec3242640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4ebc0d1b60 con 0x7f4ebc15e2a0 2024-09-19T07:43:53.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.714+0000 7f4ec3242640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4ebc0cf2d0 con 0x7f4ebc06d960 2024-09-19T07:43:53.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.715+0000 7f4ebbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ebc15e680 0x7f4ebc0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:53.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.715+0000 7f4ebbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ebc15e680 0x7f4ebc0cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34428/0 (socket says 172.21.15.16:34428) 2024-09-19T07:43:53.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.715+0000 7f4ebbfff640 1 -- 172.21.15.16:0/1298093742 learned_addr learned my addr 172.21.15.16:0/1298093742 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:53.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.715+0000 7f4ec0fde640 1 --2- 172.21.15.16:0/1298093742 >> v2:172.21.15.146:3300/0 conn(0x7f4ebc15e2a0 0x7f4ebc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:53.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.715+0000 7f4ec17df640 1 --2- 172.21.15.16:0/1298093742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ebc06d960 0x7f4ebc0d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:53.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.715+0000 7f4ebbfff640 1 -- 172.21.15.16:0/1298093742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ebc06d960 msgr2=0x7f4ebc0d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:53.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.715+0000 7f4ebbfff640 1 --2- 172.21.15.16:0/1298093742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ebc06d960 0x7f4ebc0d0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:53.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.715+0000 7f4ebbfff640 1 -- 172.21.15.16:0/1298093742 >> v2:172.21.15.146:3300/0 conn(0x7f4ebc15e2a0 msgr2=0x7f4ebc06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:53.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.715+0000 7f4ebbfff640 1 --2- 172.21.15.16:0/1298093742 >> v2:172.21.15.146:3300/0 conn(0x7f4ebc15e2a0 0x7f4ebc06d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:53.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.715+0000 7f4ebbfff640 1 -- 172.21.15.16:0/1298093742 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ebc0cf5b0 con 0x7f4ebc15e680 2024-09-19T07:43:53.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.715+0000 7f4ec0fde640 1 --2- 172.21.15.16:0/1298093742 >> v2:172.21.15.146:3300/0 conn(0x7f4ebc15e2a0 0x7f4ebc06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:53.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.716+0000 7f4ebbfff640 1 --2- 172.21.15.16:0/1298093742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ebc15e680 0x7f4ebc0cfad0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0b6fa0 tx=0x7f4ea802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26afe09b2eeff09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:53.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.716+0000 7f4ebb7fe640 1 -- 172.21.15.16:0/1298093742 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8037070 con 0x7f4ebc15e680 2024-09-19T07:43:53.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.716+0000 7f4ebb7fe640 1 -- 172.21.15.16:0/1298093742 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea8032440 con 0x7f4ebc15e680 2024-09-19T07:43:53.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.716+0000 7f4ebb7fe640 1 -- 172.21.15.16:0/1298093742 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8032810 con 0x7f4ebc15e680 2024-09-19T07:43:53.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.716+0000 7f4ec3242640 1 -- 172.21.15.16:0/1298093742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ebc15e680 msgr2=0x7f4ebc0cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:53.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.716+0000 7f4ec3242640 1 --2- 172.21.15.16:0/1298093742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ebc15e680 0x7f4ebc0cfad0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0b6fa0 tx=0x7f4ea802a1e0 comp rx=0 tx=0).stop 2024-09-19T07:43:53.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.716+0000 7f4ec3242640 1 -- 172.21.15.16:0/1298093742 shutdown_connections 2024-09-19T07:43:53.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.716+0000 7f4ec3242640 1 --2- 172.21.15.16:0/1298093742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ebc06d960 0x7f4ebc0d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:53.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.717+0000 7f4ec3242640 1 --2- 172.21.15.16:0/1298093742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ebc15e680 0x7f4ebc0cfad0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:53.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.717+0000 7f4ec3242640 1 --2- 172.21.15.16:0/1298093742 >> v2:172.21.15.146:3300/0 conn(0x7f4ebc15e2a0 0x7f4ebc06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:53.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.717+0000 7f4ec3242640 1 -- 172.21.15.16:0/1298093742 >> 172.21.15.16:0/1298093742 conn(0x7f4ebc09d760 msgr2=0x7f4ebc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:53.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.717+0000 7f4ec3242640 1 -- 172.21.15.16:0/1298093742 shutdown_connections 2024-09-19T07:43:53.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.717+0000 7f4ec3242640 1 -- 172.21.15.16:0/1298093742 wait complete. 2024-09-19T07:43:53.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.718+0000 7f4ec3242640 1 Processor -- start 2024-09-19T07:43:53.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.718+0000 7f4ec3242640 1 -- start start 2024-09-19T07:43:53.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ec3242640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ebc06d960 0x7f4ebc0cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:53.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ec3242640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ebc15e2a0 0x7f4ebc1f4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:53.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ec3242640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ebc15e680 0x7f4ebc0c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:53.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ec3242640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc0d44f0 con 0x7f4ebc06d960 2024-09-19T07:43:53.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ec3242640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc0d4370 con 0x7f4ebc15e2a0 2024-09-19T07:43:53.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ec3242640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4ebc0d4670 con 0x7f4ebc15e680 2024-09-19T07:43:53.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ec0fde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ebc06d960 0x7f4ebc0cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:53.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ec0fde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ebc06d960 0x7f4ebc0cdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34438/0 (socket says 172.21.15.16:34438) 2024-09-19T07:43:53.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ec0fde640 1 -- 172.21.15.16:0/3403314540 learned_addr learned my addr 172.21.15.16:0/3403314540 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ec0fde640 1 -- 172.21.15.16:0/3403314540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ebc15e680 msgr2=0x7f4ebc0c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ec17df640 1 --2- 172.21.15.16:0/3403314540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ebc15e680 0x7f4ebc0c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ebbfff640 1 --2- 172.21.15.16:0/3403314540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ebc15e2a0 0x7f4ebc1f4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ec0fde640 1 --2- 172.21.15.16:0/3403314540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ebc15e680 0x7f4ebc0c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ec0fde640 1 -- 172.21.15.16:0/3403314540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ebc15e2a0 msgr2=0x7f4ebc1f4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ec0fde640 1 --2- 172.21.15.16:0/3403314540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ebc15e2a0 0x7f4ebc1f4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ec0fde640 1 -- 172.21.15.16:0/3403314540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ebc0ce260 con 0x7f4ebc06d960 2024-09-19T07:43:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ebbfff640 1 --2- 172.21.15.16:0/3403314540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ebc15e2a0 0x7f4ebc1f4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ec17df640 1 --2- 172.21.15.16:0/3403314540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ebc15e680 0x7f4ebc0c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ec0fde640 1 --2- 172.21.15.16:0/3403314540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ebc06d960 0x7f4ebc0cdbc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000c7f0 tx=0x7f4eb000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4eb97fa640 1 -- 172.21.15.16:0/3403314540 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0015070 con 0x7f4ebc06d960 2024-09-19T07:43:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4eb97fa640 1 -- 172.21.15.16:0/3403314540 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb0007a30 con 0x7f4ebc06d960 2024-09-19T07:43:53.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4ec3242640 1 -- 172.21.15.16:0/3403314540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ebc0ce540 con 0x7f4ebc06d960 2024-09-19T07:43:53.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.719+0000 7f4eb97fa640 1 -- 172.21.15.16:0/3403314540 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0007e00 con 0x7f4ebc06d960 2024-09-19T07:43:53.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.720+0000 7f4ec3242640 1 -- 172.21.15.16:0/3403314540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ebc0c94a0 con 0x7f4ebc06d960 2024-09-19T07:43:53.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.721+0000 7f4ec3242640 1 -- 172.21.15.16:0/3403314540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ebc06d420 con 0x7f4ebc06d960 2024-09-19T07:43:53.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.721+0000 7f4eb97fa640 1 -- 172.21.15.16:0/3403314540 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4eb0018070 con 0x7f4ebc06d960 2024-09-19T07:43:53.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.721+0000 7f4eb97fa640 1 --2- 172.21.15.16:0/3403314540 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4ea006c2b0 0x7f4ea006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:53.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.722+0000 7f4eb97fa640 1 -- 172.21.15.16:0/3403314540 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5853+0+0 (secure 0 0 0) 0x7f4eb008b350 con 0x7f4ebc06d960 2024-09-19T07:43:53.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.722+0000 7f4ebbfff640 1 --2- 172.21.15.16:0/3403314540 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4ea006c2b0 0x7f4ea006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:53.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.722+0000 7f4ebbfff640 1 --2- 172.21.15.16:0/3403314540 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4ea006c2b0 0x7f4ea006e770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8034000 tx=0x7f4ea802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:53.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.726+0000 7f4eb97fa640 1 -- 172.21.15.16:0/3403314540 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4eb0059ac0 con 0x7f4ebc06d960 2024-09-19T07:43:53.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:53.900+0000 7f4ec3242640 1 -- 172.21.15.16:0/3403314540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4ebc06cf90 con 0x7f4ebc06d960 2024-09-19T07:43:54.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.587+0000 7f4eb97fa640 1 -- 172.21.15.16:0/3403314540 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) ==== 178+0+0 (secure 0 0 0) 0x7f4eb00593c0 con 0x7f4ebc06d960 2024-09-19T07:43:54.588 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' removed 2024-09-19T07:43:54.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.591+0000 7f4ec3242640 1 -- 172.21.15.16:0/3403314540 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4ea006c2b0 msgr2=0x7f4ea006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:54.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.591+0000 7f4ec3242640 1 --2- 172.21.15.16:0/3403314540 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4ea006c2b0 0x7f4ea006e770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8034000 tx=0x7f4ea802e000 comp rx=0 tx=0).stop 2024-09-19T07:43:54.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.591+0000 7f4ec3242640 1 -- 172.21.15.16:0/3403314540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ebc06d960 msgr2=0x7f4ebc0cdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:54.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.591+0000 7f4ec3242640 1 --2- 172.21.15.16:0/3403314540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ebc06d960 0x7f4ebc0cdbc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000c7f0 tx=0x7f4eb000cbf0 comp rx=0 tx=0).stop 2024-09-19T07:43:54.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.591+0000 7f4ec3242640 1 -- 172.21.15.16:0/3403314540 shutdown_connections 2024-09-19T07:43:54.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.591+0000 7f4ec3242640 1 --2- 172.21.15.16:0/3403314540 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4ea006c2b0 0x7f4ea006e770 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:54.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.591+0000 7f4ec3242640 1 --2- 172.21.15.16:0/3403314540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ebc15e680 0x7f4ebc0c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:54.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.591+0000 7f4ec3242640 1 --2- 172.21.15.16:0/3403314540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ebc15e2a0 0x7f4ebc1f4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:54.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.591+0000 7f4ec3242640 1 --2- 172.21.15.16:0/3403314540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ebc06d960 0x7f4ebc0cdbc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:54.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.591+0000 7f4ec3242640 1 -- 172.21.15.16:0/3403314540 >> 172.21.15.16:0/3403314540 conn(0x7f4ebc09d760 msgr2=0x7f4ebc15d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:54.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.591+0000 7f4ec3242640 1 -- 172.21.15.16:0/3403314540 shutdown_connections 2024-09-19T07:43:54.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.591+0000 7f4ec3242640 1 -- 172.21.15.16:0/3403314540 wait complete. 2024-09-19T07:43:54.607 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T07:43:54.607 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T07:43:54.607 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T07:43:54.607 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:43:54.607 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T07:43:54.607 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T07:43:54.608 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T07:43:54.608 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:43:54.608 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T07:43:54.608 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T07:43:54.608 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T07:43:54.608 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:43:54.608 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T07:43:54.609 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T07:43:54.609 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T07:43:54.609 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:43:54.618 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T07:43:54.618 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T07:43:54.619 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T07:43:54.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T07:43:54.620 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T07:43:54.620 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T07:43:54.621 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T07:43:54.621 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T07:43:54.661 INFO:tasks.ceph.mds.a:Started 2024-09-19T07:43:54.662 DEBUG:teuthology.parallel:result is None 2024-09-19T07:43:54.663 INFO:tasks.ceph.mds.c:Started 2024-09-19T07:43:54.663 DEBUG:teuthology.parallel:result is None 2024-09-19T07:43:54.664 INFO:tasks.ceph.mds.b:Started 2024-09-19T07:43:54.664 DEBUG:teuthology.parallel:result is None 2024-09-19T07:43:54.664 INFO:tasks.ceph.mds.d:Started 2024-09-19T07:43:54.665 DEBUG:teuthology.parallel:result is None 2024-09-19T07:43:54.665 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T07:43:54.744 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2024-09-19T07:43:54.745 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2024-09-19T07:43:54.745 INFO:tasks.ceph.mds.d.smithi146.stderr:2024-09-19T07:43:54.745+0000 7f5d4a913640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-19T07:43:54.746 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-19T07:43:54.746 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-19T07:43:54.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.778+0000 7f93d95fd640 1 Processor -- start 2024-09-19T07:43:54.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.778+0000 7f93d95fd640 1 -- start start 2024-09-19T07:43:54.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.779+0000 7f93d95fd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93d415f0d0 0x7f93d415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:54.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.779+0000 7f93d95fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93d40d0070 0x7f93d40d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:54.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.779+0000 7f93d95fd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93d40cf2b0 con 0x7f93d40d1940 2024-09-19T07:43:54.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.779+0000 7f93d95fd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f93d40cf590 con 0x7f93d415f0d0 2024-09-19T07:43:54.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.779+0000 7f93d95fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f93d40cf870 con 0x7f93d40d0070 2024-09-19T07:43:54.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.779+0000 7f93d2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93d415f0d0 0x7f93d415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:54.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.779+0000 7f93d2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93d415f0d0 0x7f93d415f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34474/0 (socket says 172.21.15.16:34474) 2024-09-19T07:43:54.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.779+0000 7f93d2ffd640 1 -- 172.21.15.16:0/714592814 learned_addr learned my addr 172.21.15.16:0/714592814 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:54.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.779+0000 7f93d27fc640 1 --2- 172.21.15.16:0/714592814 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93d40d0070 0x7f93d40d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:54.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.779+0000 7f93d2ffd640 1 -- 172.21.15.16:0/714592814 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93d40d0070 msgr2=0x7f93d40d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:54.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.779+0000 7f93d2ffd640 1 --2- 172.21.15.16:0/714592814 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93d40d0070 0x7f93d40d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:54.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.779+0000 7f93d2ffd640 1 -- 172.21.15.16:0/714592814 >> v1:172.21.15.146:6789/0 conn(0x7f93d40d1940 legacy=0x7f93d40d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:54.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.779+0000 7f93d2ffd640 1 -- 172.21.15.16:0/714592814 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d40cfb50 con 0x7f93d415f0d0 2024-09-19T07:43:54.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.780+0000 7f93d2ffd640 1 --2- 172.21.15.16:0/714592814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d415f0d0 0x7f93d415f4b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f93c4009870 tx=0x7f93c402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5608604f27ea765 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:54.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.780+0000 7f93d1ffb640 1 -- 172.21.15.16:0/714592814 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4037070 con 0x7f93d415f0d0 2024-09-19T07:43:54.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.780+0000 7f93d1ffb640 1 -- 172.21.15.16:0/714592814 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93c4032440 con 0x7f93d415f0d0 2024-09-19T07:43:54.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.780+0000 7f93d1ffb640 1 -- 172.21.15.16:0/714592814 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4032810 con 0x7f93d415f0d0 2024-09-19T07:43:54.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.781+0000 7f93d95fd640 1 -- 172.21.15.16:0/714592814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d415f0d0 msgr2=0x7f93d415f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:54.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.781+0000 7f93d95fd640 1 --2- 172.21.15.16:0/714592814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d415f0d0 0x7f93d415f4b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f93c4009870 tx=0x7f93c402a1c0 comp rx=0 tx=0).stop 2024-09-19T07:43:54.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.781+0000 7f93d95fd640 1 -- 172.21.15.16:0/714592814 shutdown_connections 2024-09-19T07:43:54.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.781+0000 7f93d95fd640 1 --2- 172.21.15.16:0/714592814 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93d40d0070 0x7f93d40d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:54.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.781+0000 7f93d95fd640 1 --2- 172.21.15.16:0/714592814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d415f0d0 0x7f93d415f4b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:54.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.781+0000 7f93d95fd640 1 -- 172.21.15.16:0/714592814 >> 172.21.15.16:0/714592814 conn(0x7f93d409d760 msgr2=0x7f93d408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:54.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.781+0000 7f93d95fd640 1 -- 172.21.15.16:0/714592814 shutdown_connections 2024-09-19T07:43:54.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.781+0000 7f93d95fd640 1 -- 172.21.15.16:0/714592814 wait complete. 2024-09-19T07:43:54.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.782+0000 7f93d95fd640 1 Processor -- start 2024-09-19T07:43:54.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.782+0000 7f93d95fd640 1 -- start start 2024-09-19T07:43:54.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.783+0000 7f93d95fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93d40d0070 0x7f93d40b9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:54.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.783+0000 7f93d95fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d40d1940 0x7f93d4157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:54.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.783+0000 7f93d95fd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93d415f0d0 0x7f93d40cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:54.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.783+0000 7f93d95fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f93d40d49c0 con 0x7f93d40d1940 2024-09-19T07:43:54.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.783+0000 7f93d95fd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f93d40d4840 con 0x7f93d415f0d0 2024-09-19T07:43:54.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.783+0000 7f93d95fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f93d40d4b40 con 0x7f93d40d0070 2024-09-19T07:43:54.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.783+0000 7f93d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d40d1940 0x7f93d4157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:54.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.783+0000 7f93d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d40d1940 0x7f93d4157820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34484/0 (socket says 172.21.15.16:34484) 2024-09-19T07:43:54.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.783+0000 7f93d27fc640 1 -- 172.21.15.16:0/1894619481 learned_addr learned my addr 172.21.15.16:0/1894619481 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:54.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.783+0000 7f93d2ffd640 1 --2- 172.21.15.16:0/1894619481 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93d40d0070 0x7f93d40b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:54.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.783+0000 7f93d37fe640 1 --2- 172.21.15.16:0/1894619481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93d415f0d0 0x7f93d40cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:54.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.783+0000 7f93d27fc640 1 -- 172.21.15.16:0/1894619481 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93d40d0070 msgr2=0x7f93d40b9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:54.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.783+0000 7f93d27fc640 1 --2- 172.21.15.16:0/1894619481 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93d40d0070 0x7f93d40b9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:54.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.783+0000 7f93d27fc640 1 -- 172.21.15.16:0/1894619481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93d415f0d0 msgr2=0x7f93d40cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:54.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.783+0000 7f93d27fc640 1 --2- 172.21.15.16:0/1894619481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93d415f0d0 0x7f93d40cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:54.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.783+0000 7f93d27fc640 1 -- 172.21.15.16:0/1894619481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d41f7770 con 0x7f93d40d1940 2024-09-19T07:43:54.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.783+0000 7f93d37fe640 1 --2- 172.21.15.16:0/1894619481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93d415f0d0 0x7f93d40cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:54.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.784+0000 7f93d27fc640 1 --2- 172.21.15.16:0/1894619481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d40d1940 0x7f93d4157820 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f93c800d600 tx=0x7f93c800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:54.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.784+0000 7f93bb7fe640 1 -- 172.21.15.16:0/1894619481 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c80125f0 con 0x7f93d40d1940 2024-09-19T07:43:54.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.784+0000 7f93bb7fe640 1 -- 172.21.15.16:0/1894619481 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93c8019070 con 0x7f93d40d1940 2024-09-19T07:43:54.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.784+0000 7f93d95fd640 1 -- 172.21.15.16:0/1894619481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d41f7a50 con 0x7f93d40d1940 2024-09-19T07:43:54.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.784+0000 7f93bb7fe640 1 -- 172.21.15.16:0/1894619481 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c80129f0 con 0x7f93d40d1940 2024-09-19T07:43:54.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.784+0000 7f93d95fd640 1 -- 172.21.15.16:0/1894619481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d41f8010 con 0x7f93d40d1940 2024-09-19T07:43:54.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.785+0000 7f93d95fd640 1 -- 172.21.15.16:0/1894619481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d406cf30 con 0x7f93d40d1940 2024-09-19T07:43:54.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.785+0000 7f93bb7fe640 1 -- 172.21.15.16:0/1894619481 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f93c8012b90 con 0x7f93d40d1940 2024-09-19T07:43:54.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.786+0000 7f93bb7fe640 1 --2- 172.21.15.16:0/1894619481 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f93c006c2b0 0x7f93c006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:54.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.786+0000 7f93bb7fe640 1 -- 172.21.15.16:0/1894619481 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5335+0+0 (secure 0 0 0) 0x7f93c808b6c0 con 0x7f93d40d1940 2024-09-19T07:43:54.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.786+0000 7f93d2ffd640 1 --2- 172.21.15.16:0/1894619481 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f93c006c2b0 0x7f93c006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:54.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.787+0000 7f93d2ffd640 1 --2- 172.21.15.16:0/1894619481 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f93c006c2b0 0x7f93c006e770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f93c4034000 tx=0x7f93c402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:54.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.790+0000 7f93bb7fe640 1 -- 172.21.15.16:0/1894619481 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f93c8059f00 con 0x7f93d40d1940 2024-09-19T07:43:54.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:54.980+0000 7f93d95fd640 1 -- 172.21.15.16:0/1894619481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f93d415b540 con 0x7f93d40d1940 2024-09-19T07:43:55.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.597+0000 7f93bb7fe640 1 -- 172.21.15.16:0/1894619481 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) ==== 97+0+0 (secure 0 0 0) 0x7f93c8059800 con 0x7f93d40d1940 2024-09-19T07:43:55.597 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-19T07:43:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.600+0000 7f93d95fd640 1 -- 172.21.15.16:0/1894619481 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f93c006c2b0 msgr2=0x7f93c006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.600+0000 7f93d95fd640 1 --2- 172.21.15.16:0/1894619481 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f93c006c2b0 0x7f93c006e770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f93c4034000 tx=0x7f93c402e000 comp rx=0 tx=0).stop 2024-09-19T07:43:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.600+0000 7f93d95fd640 1 -- 172.21.15.16:0/1894619481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d40d1940 msgr2=0x7f93d4157820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.600+0000 7f93d95fd640 1 --2- 172.21.15.16:0/1894619481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d40d1940 0x7f93d4157820 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f93c800d600 tx=0x7f93c800dac0 comp rx=0 tx=0).stop 2024-09-19T07:43:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.600+0000 7f93d95fd640 1 -- 172.21.15.16:0/1894619481 shutdown_connections 2024-09-19T07:43:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.600+0000 7f93d95fd640 1 --2- 172.21.15.16:0/1894619481 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f93c006c2b0 0x7f93c006e770 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.600+0000 7f93d95fd640 1 --2- 172.21.15.16:0/1894619481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93d415f0d0 0x7f93d40cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.600+0000 7f93d95fd640 1 --2- 172.21.15.16:0/1894619481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93d40d1940 0x7f93d4157820 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.600+0000 7f93d95fd640 1 --2- 172.21.15.16:0/1894619481 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93d40d0070 0x7f93d40b9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.600+0000 7f93d95fd640 1 -- 172.21.15.16:0/1894619481 >> 172.21.15.16:0/1894619481 conn(0x7f93d409d760 msgr2=0x7f93d41624b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.600+0000 7f93d95fd640 1 -- 172.21.15.16:0/1894619481 shutdown_connections 2024-09-19T07:43:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.600+0000 7f93d95fd640 1 -- 172.21.15.16:0/1894619481 wait complete. 2024-09-19T07:43:55.615 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T07:43:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.723+0000 7fb5485cf640 1 Processor -- start 2024-09-19T07:43:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.723+0000 7fb5485cf640 1 -- start start 2024-09-19T07:43:55.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.724+0000 7fb5485cf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb54013f0c0 0x7fb54015d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:55.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.724+0000 7fb5485cf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5401419d0 0x7fb540141e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:55.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.724+0000 7fb5485cf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb54015d6b0 con 0x7fb54015fce0 2024-09-19T07:43:55.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.724+0000 7fb5485cf640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb540072450 con 0x7fb54013f0c0 2024-09-19T07:43:55.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.724+0000 7fb5485cf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb540072730 con 0x7fb5401419d0 2024-09-19T07:43:55.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.724+0000 7fb54636b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb54013f0c0 0x7fb54015d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:55.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.724+0000 7fb546b6c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb54015fce0 0x7fb540160130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56120/0 (socket says 172.21.15.16:56120) 2024-09-19T07:43:55.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.724+0000 7fb54636b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb54013f0c0 0x7fb54015d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34488/0 (socket says 172.21.15.16:34488) 2024-09-19T07:43:55.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.724+0000 7fb546b6c640 1 -- 172.21.15.16:0/2387147116 learned_addr learned my addr 172.21.15.16:0/2387147116 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:55.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.724+0000 7fb545b6a640 1 --2- 172.21.15.16:0/2387147116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5401419d0 0x7fb540141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:55.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.724+0000 7fb54636b640 1 -- 172.21.15.16:0/2387147116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5401419d0 msgr2=0x7fb540141e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:55.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.724+0000 7fb54636b640 1 --2- 172.21.15.16:0/2387147116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5401419d0 0x7fb540141e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:55.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.724+0000 7fb54636b640 1 -- 172.21.15.16:0/2387147116 >> v1:172.21.15.16:6789/0 conn(0x7fb54015fce0 legacy=0x7fb540160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:55.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.724+0000 7fb545369640 1 -- 172.21.15.16:0/2387147116 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 795753670 0 0) 0x7fb530001490 con 0x7fb54015fce0 2024-09-19T07:43:55.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.724+0000 7fb54636b640 1 -- 172.21.15.16:0/2387147116 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb540072a10 con 0x7fb54013f0c0 2024-09-19T07:43:55.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.725+0000 7fb54636b640 1 --2- 172.21.15.16:0/2387147116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54013f0c0 0x7fb54015d170 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb5380097a0 tx=0x7fb53802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ffebc905a4bc9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:55.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.725+0000 7fb545369640 1 -- 172.21.15.16:0/2387147116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb538037070 con 0x7fb54013f0c0 2024-09-19T07:43:55.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.725+0000 7fb545369640 1 -- 172.21.15.16:0/2387147116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb538032440 con 0x7fb54013f0c0 2024-09-19T07:43:55.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.725+0000 7fb545369640 1 -- 172.21.15.16:0/2387147116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb538032810 con 0x7fb54013f0c0 2024-09-19T07:43:55.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.726+0000 7fb5485cf640 1 -- 172.21.15.16:0/2387147116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54013f0c0 msgr2=0x7fb54015d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:55.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.726+0000 7fb5485cf640 1 --2- 172.21.15.16:0/2387147116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54013f0c0 0x7fb54015d170 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb5380097a0 tx=0x7fb53802a180 comp rx=0 tx=0).stop 2024-09-19T07:43:55.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.726+0000 7fb5485cf640 1 -- 172.21.15.16:0/2387147116 shutdown_connections 2024-09-19T07:43:55.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.726+0000 7fb5485cf640 1 --2- 172.21.15.16:0/2387147116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5401419d0 0x7fb540141e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:55.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.726+0000 7fb5485cf640 1 --2- 172.21.15.16:0/2387147116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54013f0c0 0x7fb54015d170 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:55.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.726+0000 7fb5485cf640 1 -- 172.21.15.16:0/2387147116 >> 172.21.15.16:0/2387147116 conn(0x7fb540067d70 msgr2=0x7fb54006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:55.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.726+0000 7fb5485cf640 1 -- 172.21.15.16:0/2387147116 shutdown_connections 2024-09-19T07:43:55.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.726+0000 7fb5485cf640 1 -- 172.21.15.16:0/2387147116 wait complete. 2024-09-19T07:43:55.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.727+0000 7fb5485cf640 1 Processor -- start 2024-09-19T07:43:55.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.727+0000 7fb5485cf640 1 -- start start 2024-09-19T07:43:55.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.728+0000 7fb5485cf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb54013f0c0 0x7fb540151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:55.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.728+0000 7fb5485cf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5401419d0 0x7fb540151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:55.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.728+0000 7fb5485cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54015fce0 0x7fb5401588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:55.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.728+0000 7fb5485cf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5401427a0 con 0x7fb54015fce0 2024-09-19T07:43:55.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.728+0000 7fb5485cf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb540072fd0 con 0x7fb54013f0c0 2024-09-19T07:43:55.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.728+0000 7fb5485cf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb5400720b0 con 0x7fb5401419d0 2024-09-19T07:43:55.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.728+0000 7fb546b6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54015fce0 0x7fb5401588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:55.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.728+0000 7fb546b6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54015fce0 0x7fb5401588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34496/0 (socket says 172.21.15.16:34496) 2024-09-19T07:43:55.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.728+0000 7fb546b6c640 1 -- 172.21.15.16:0/1336678022 learned_addr learned my addr 172.21.15.16:0/1336678022 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:55.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.728+0000 7fb54636b640 1 --2- 172.21.15.16:0/1336678022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb54013f0c0 0x7fb540151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:55.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.728+0000 7fb545b6a640 1 --2- 172.21.15.16:0/1336678022 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5401419d0 0x7fb540151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.728+0000 7fb546b6c640 1 -- 172.21.15.16:0/1336678022 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5401419d0 msgr2=0x7fb540151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.728+0000 7fb546b6c640 1 --2- 172.21.15.16:0/1336678022 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5401419d0 0x7fb540151750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.728+0000 7fb546b6c640 1 -- 172.21.15.16:0/1336678022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb54013f0c0 msgr2=0x7fb540151210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.728+0000 7fb546b6c640 1 --2- 172.21.15.16:0/1336678022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb54013f0c0 0x7fb540151210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.728+0000 7fb546b6c640 1 -- 172.21.15.16:0/1336678022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb540154c20 con 0x7fb54015fce0 2024-09-19T07:43:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.729+0000 7fb546b6c640 1 --2- 172.21.15.16:0/1336678022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54015fce0 0x7fb5401588c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb53000c700 tx=0x7fb53000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.729+0000 7fb53effd640 1 -- 172.21.15.16:0/1336678022 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530018070 con 0x7fb54015fce0 2024-09-19T07:43:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.729+0000 7fb53effd640 1 -- 172.21.15.16:0/1336678022 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb53000ce10 con 0x7fb54015fce0 2024-09-19T07:43:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.729+0000 7fb53effd640 1 -- 172.21.15.16:0/1336678022 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53000a860 con 0x7fb54015fce0 2024-09-19T07:43:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.729+0000 7fb5485cf640 1 -- 172.21.15.16:0/1336678022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5401e5d00 con 0x7fb54015fce0 2024-09-19T07:43:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.729+0000 7fb5485cf640 1 -- 172.21.15.16:0/1336678022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb540151f10 con 0x7fb54015fce0 2024-09-19T07:43:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.730+0000 7fb5485cf640 1 -- 172.21.15.16:0/1336678022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5401e5b20 con 0x7fb54015fce0 2024-09-19T07:43:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.730+0000 7fb53effd640 1 -- 172.21.15.16:0/1336678022 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb53001b070 con 0x7fb54015fce0 2024-09-19T07:43:55.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.731+0000 7fb53effd640 1 --2- 172.21.15.16:0/1336678022 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb52c06c2b0 0x7fb52c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:55.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.731+0000 7fb53effd640 1 -- 172.21.15.16:0/1336678022 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fb53008a920 con 0x7fb54015fce0 2024-09-19T07:43:55.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.731+0000 7fb54636b640 1 --2- 172.21.15.16:0/1336678022 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb52c06c2b0 0x7fb52c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:55.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.732+0000 7fb54636b640 1 --2- 172.21.15.16:0/1336678022 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb52c06c2b0 0x7fb52c06e770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb538034000 tx=0x7fb53802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:55.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.736+0000 7fb53effd640 1 -- 172.21.15.16:0/1336678022 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb5401e5b20 con 0x7fb54015fce0 2024-09-19T07:43:55.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.975+0000 7fb5485cf640 1 -- 172.21.15.16:0/1336678022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb54006ff00 con 0x7fb54015fce0 2024-09-19T07:43:55.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.976+0000 7fb53effd640 1 -- 172.21.15.16:0/1336678022 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7fb54015d6b0 con 0x7fb54015fce0 2024-09-19T07:43:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:55.977 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:43:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-19T07:43:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-19T07:43:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etms80KDxAAqTrG88LTEJIj9TWJu0witQ==", 2024-09-19T07:43:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:43:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:43:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:43:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:43:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-19T07:43:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etm4T0KGxAAcfDOKrSdKzhJ4xvTFIyxcQ==", 2024-09-19T07:43:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:43:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:43:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:43:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:43:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-19T07:43:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNDBGFhAAtBioJJ1+NtIDLhd/Hg7aZA==", 2024-09-19T07:43:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:43:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:43:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:43:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:43:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-19T07:43:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmaHEiIhAACqqjCRMV+xlGh9WqYsj+MA==", 2024-09-19T07:43:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:43:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:43:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:43:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:43:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-19T07:43:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDT1etm09plCxAAGvJYF6JrKw49FtXb71ox3A==", 2024-09-19T07:43:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:43:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:43:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:43:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-19T07:43:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDW1etmGy8yHRAAlcDsHLnkjtS6sCCR8Olv2A==", 2024-09-19T07:43:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:43:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:43:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:43:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-19T07:43:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDZ1etmMhGTHxAATQaW2kgJhHKftutQQtUa6A==", 2024-09-19T07:43:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:43:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:43:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:43:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-19T07:43:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDc1etmD3sLIhAAIdY00buvb3bIFA5ZpgxxjA==", 2024-09-19T07:43:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:43:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:43:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:43:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-19T07:43:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDi1etmgutcLRAAHJEl5SfQ7N0T4YoYeBfrQg==", 2024-09-19T07:43:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:43:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:43:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:43:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-19T07:43:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDm1etmCV3xARAA+5yhQEmIpyNId3RL1Xsxhw==", 2024-09-19T07:43:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:43:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:43:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:43:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-19T07:43:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDp1etm9XC+AxAA+Lrs0ExfSKaixgOxfX8Nug==", 2024-09-19T07:43:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:43:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:43:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:43:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-19T07:43:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDs1etmhmvMBhAAhvW/udIE2gCd8IrJCSFuvw==", 2024-09-19T07:43:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:43:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:43:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:43:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-19T07:43:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA==", 2024-09-19T07:43:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:43:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:43:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:43:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:43:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-19T07:43:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmc05KLxAAPQl6wOHqFIf8LVxskiuLVg==", 2024-09-19T07:43:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:43:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:43:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:43:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:43:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-19T07:43:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmjFsCLBAANa7gCMvDw84Ovo7u6RRJ3Q==", 2024-09-19T07:43:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:43:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:43:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:43:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:43:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-19T07:43:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmdUbgNBAAfwstyOeeoZfetgVZu4z5xg==", 2024-09-19T07:43:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:43:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:43:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:43:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:43:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-19T07:43:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDQ1etm9/vTLhAA9PRfAOA8R4oXp3x0dU8l3g==", 2024-09-19T07:43:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:43:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:43:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:43:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:43:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mds", 2024-09-19T07:43:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDy1etm2ez4ERAAobi4k/8jW/gqnPJZOaB7fw==", 2024-09-19T07:43:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mds" 2024-09-19T07:43:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mgr", 2024-09-19T07:43:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDy1etmv/X4ERAAqcjV2gm/nIzTpnsv0oZfsA==", 2024-09-19T07:43:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-19T07:43:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-osd", 2024-09-19T07:43:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDy1etmh/34ERAAR6HJvtpH5hPyRNZO600QyA==", 2024-09-19T07:43:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-osd" 2024-09-19T07:43:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd", 2024-09-19T07:43:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDy1etm7gT5ERAAVr/sAnGKukPtpfTT9q45+w==", 2024-09-19T07:43:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-19T07:43:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-19T07:43:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDy1etmng35ERAAn4oE00dkRS0J6JNCOsDeXg==", 2024-09-19T07:43:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-19T07:43:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rgw", 2024-09-19T07:43:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDy1etmUhX5ERAAguk0zyjtm/BY2ge5gC/TcA==", 2024-09-19T07:43:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-19T07:43:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-19T07:43:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNQ68CxAANkK+D3u/oTEz8Su4iuZrCg==", 2024-09-19T07:43:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:43:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:43:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:43:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:43:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-19T07:43:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmp+gSCBAAGXzmKfFNd6K2fWeqYhlHAg==", 2024-09-19T07:43:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:43:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:43:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:43:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:43:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T07:43:55.993 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:43:55.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.978+0000 7fb5485cf640 1 -- 172.21.15.16:0/1336678022 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb52c06c2b0 msgr2=0x7fb52c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:55.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.978+0000 7fb5485cf640 1 --2- 172.21.15.16:0/1336678022 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb52c06c2b0 0x7fb52c06e770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb538034000 tx=0x7fb53802e000 comp rx=0 tx=0).stop 2024-09-19T07:43:55.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.978+0000 7fb5485cf640 1 -- 172.21.15.16:0/1336678022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54015fce0 msgr2=0x7fb5401588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:55.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.978+0000 7fb5485cf640 1 --2- 172.21.15.16:0/1336678022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54015fce0 0x7fb5401588c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb53000c700 tx=0x7fb53000cbc0 comp rx=0 tx=0).stop 2024-09-19T07:43:55.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.978+0000 7fb5485cf640 1 -- 172.21.15.16:0/1336678022 shutdown_connections 2024-09-19T07:43:55.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.978+0000 7fb5485cf640 1 --2- 172.21.15.16:0/1336678022 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb52c06c2b0 0x7fb52c06e770 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:55.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.978+0000 7fb5485cf640 1 --2- 172.21.15.16:0/1336678022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54015fce0 0x7fb5401588c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:55.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.978+0000 7fb5485cf640 1 --2- 172.21.15.16:0/1336678022 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5401419d0 0x7fb540151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:55.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.978+0000 7fb5485cf640 1 --2- 172.21.15.16:0/1336678022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb54013f0c0 0x7fb540151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:55.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.978+0000 7fb5485cf640 1 -- 172.21.15.16:0/1336678022 >> 172.21.15.16:0/1336678022 conn(0x7fb540067d70 msgr2=0x7fb5400cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:55.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.978+0000 7fb5485cf640 1 -- 172.21.15.16:0/1336678022 shutdown_connections 2024-09-19T07:43:55.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:55.978+0000 7fb5485cf640 1 -- 172.21.15.16:0/1336678022 wait complete. 2024-09-19T07:43:55.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-19T07:43:56.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.099+0000 7f088faf1640 1 Processor -- start 2024-09-19T07:43:56.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.099+0000 7f088faf1640 1 -- start start 2024-09-19T07:43:56.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.100+0000 7f088faf1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f088815b6e0 0x7f08880cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.100+0000 7f088faf1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08880cd910 con 0x7f088815b300 2024-09-19T07:43:56.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.100+0000 7f088faf1640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08880ccda0 con 0x7f08880ce090 2024-09-19T07:43:56.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.100+0000 7f088faf1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f08880ce4c0 con 0x7f088815b6e0 2024-09-19T07:43:56.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.100+0000 7f088d08c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f088815b6e0 0x7f08880cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.100+0000 7f088d88d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f088815b300 0x7f08880cc690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56130/0 (socket says 172.21.15.16:56130) 2024-09-19T07:43:56.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.100+0000 7f088d08c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f088815b6e0 0x7f08880cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34500/0 (socket says 172.21.15.16:34500) 2024-09-19T07:43:56.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.100+0000 7f088d88d640 1 -- 172.21.15.16:0/1330408564 learned_addr learned my addr 172.21.15.16:0/1330408564 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:56.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.100+0000 7f088d08c640 1 -- 172.21.15.16:0/1330408564 >> v1:172.21.15.146:6789/0 conn(0x7f08880ce090 legacy=0x7f08880b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.100+0000 7f088d08c640 1 -- 172.21.15.16:0/1330408564 >> v1:172.21.15.16:6789/0 conn(0x7f088815b300 legacy=0x7f08880cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.100+0000 7f088d08c640 1 -- 172.21.15.16:0/1330408564 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08880cd070 con 0x7f088815b6e0 2024-09-19T07:43:56.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.100+0000 7f088c88b640 1 -- 172.21.15.16:0/1330408564 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3157345544 0 0) 0x7f08880cd910 con 0x7f088815b300 2024-09-19T07:43:56.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.101+0000 7f088d08c640 1 --2- 172.21.15.16:0/1330408564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f088815b6e0 0x7f08880cdb50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f08840097a0 tx=0x7f088402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b16f7ab55622094 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:56.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.101+0000 7f088c88b640 1 -- 172.21.15.16:0/1330408564 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0884037070 con 0x7f088815b6e0 2024-09-19T07:43:56.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.101+0000 7f088c88b640 1 -- 172.21.15.16:0/1330408564 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0884032440 con 0x7f088815b6e0 2024-09-19T07:43:56.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.101+0000 7f088c88b640 1 -- 172.21.15.16:0/1330408564 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0884032810 con 0x7f088815b6e0 2024-09-19T07:43:56.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.102+0000 7f088faf1640 1 -- 172.21.15.16:0/1330408564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f088815b6e0 msgr2=0x7f08880cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.102+0000 7f088faf1640 1 --2- 172.21.15.16:0/1330408564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f088815b6e0 0x7f08880cdb50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f08840097a0 tx=0x7f088402a230 comp rx=0 tx=0).stop 2024-09-19T07:43:56.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.102+0000 7f088faf1640 1 -- 172.21.15.16:0/1330408564 shutdown_connections 2024-09-19T07:43:56.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.102+0000 7f088faf1640 1 --2- 172.21.15.16:0/1330408564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f088815b6e0 0x7f08880cdb50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.102+0000 7f088faf1640 1 -- 172.21.15.16:0/1330408564 >> 172.21.15.16:0/1330408564 conn(0x7f08880b2d10 msgr2=0x7f08880b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:56.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.102+0000 7f088faf1640 1 -- 172.21.15.16:0/1330408564 shutdown_connections 2024-09-19T07:43:56.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.102+0000 7f088faf1640 1 -- 172.21.15.16:0/1330408564 wait complete. 2024-09-19T07:43:56.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.103+0000 7f088faf1640 1 Processor -- start 2024-09-19T07:43:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.103+0000 7f088faf1640 1 -- start start 2024-09-19T07:43:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.104+0000 7f088faf1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08880ce090 0x7f088814acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.104+0000 7f088faf1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f088815b300 0x7f088814df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.104+0000 7f088faf1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f088815b6e0 0x7f088806c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.104+0000 7f088faf1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08880b0b50 con 0x7f088815b300 2024-09-19T07:43:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.104+0000 7f088faf1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f08880cd630 con 0x7f088815b6e0 2024-09-19T07:43:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.104+0000 7f088faf1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f08880b39f0 con 0x7f08880ce090 2024-09-19T07:43:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.104+0000 7f088d08c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f088815b300 0x7f088814df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.104+0000 7f088d08c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f088815b300 0x7f088814df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34506/0 (socket says 172.21.15.16:34506) 2024-09-19T07:43:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.104+0000 7f088d08c640 1 -- 172.21.15.16:0/2830752071 learned_addr learned my addr 172.21.15.16:0/2830752071 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.104+0000 7f088d88d640 1 --2- 172.21.15.16:0/2830752071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08880ce090 0x7f088814acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.104+0000 7f088e08e640 1 --2- 172.21.15.16:0/2830752071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f088815b6e0 0x7f088806c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.105+0000 7f088d08c640 1 -- 172.21.15.16:0/2830752071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08880ce090 msgr2=0x7f088814acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.105+0000 7f088d08c640 1 --2- 172.21.15.16:0/2830752071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08880ce090 0x7f088814acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.105+0000 7f088d08c640 1 -- 172.21.15.16:0/2830752071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f088815b6e0 msgr2=0x7f088806c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.105+0000 7f088d08c640 1 --2- 172.21.15.16:0/2830752071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f088815b6e0 0x7f088806c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.105+0000 7f088d08c640 1 -- 172.21.15.16:0/2830752071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088814b210 con 0x7f088815b300 2024-09-19T07:43:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.105+0000 7f088e08e640 1 --2- 172.21.15.16:0/2830752071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f088815b6e0 0x7f088806c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.105+0000 7f088d08c640 1 --2- 172.21.15.16:0/2830752071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f088815b300 0x7f088814df60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0884035040 tx=0x7f08840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.105+0000 7f087a7fc640 1 -- 172.21.15.16:0/2830752071 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0884037050 con 0x7f088815b300 2024-09-19T07:43:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.105+0000 7f087a7fc640 1 -- 172.21.15.16:0/2830752071 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0884038040 con 0x7f088815b300 2024-09-19T07:43:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.105+0000 7f087a7fc640 1 -- 172.21.15.16:0/2830752071 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0884004490 con 0x7f088815b300 2024-09-19T07:43:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.105+0000 7f088faf1640 1 -- 172.21.15.16:0/2830752071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08880c8250 con 0x7f088815b300 2024-09-19T07:43:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.105+0000 7f088faf1640 1 -- 172.21.15.16:0/2830752071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f088814bab0 con 0x7f088815b300 2024-09-19T07:43:56.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.106+0000 7f088faf1640 1 -- 172.21.15.16:0/2830752071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08880cdb50 con 0x7f088815b300 2024-09-19T07:43:56.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.107+0000 7f087a7fc640 1 -- 172.21.15.16:0/2830752071 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0884030900 con 0x7f088815b300 2024-09-19T07:43:56.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.107+0000 7f087a7fc640 1 --2- 172.21.15.16:0/2830752071 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f086c06c2e0 0x7f086c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.107+0000 7f087a7fc640 1 -- 172.21.15.16:0/2830752071 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f08840abd30 con 0x7f088815b300 2024-09-19T07:43:56.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.107+0000 7f088d88d640 1 --2- 172.21.15.16:0/2830752071 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f086c06c2e0 0x7f086c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.108+0000 7f088d88d640 1 --2- 172.21.15.16:0/2830752071 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f086c06c2e0 0x7f086c06e7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f087c007f70 tx=0x7f087c007f00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.111+0000 7f087a7fc640 1 -- 172.21.15.16:0/2830752071 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f088407a810 con 0x7f088815b300 2024-09-19T07:43:56.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.351+0000 7f088faf1640 1 -- 172.21.15.16:0/2830752071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f08880c7ff0 con 0x7f088815b300 2024-09-19T07:43:56.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.353+0000 7f087a7fc640 1 -- 172.21.15.16:0/2830752071 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f088407a110 con 0x7f088815b300 2024-09-19T07:43:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.355+0000 7f088faf1640 1 -- 172.21.15.16:0/2830752071 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f086c06c2e0 msgr2=0x7f086c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.355+0000 7f088faf1640 1 --2- 172.21.15.16:0/2830752071 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f086c06c2e0 0x7f086c06e7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f087c007f70 tx=0x7f087c007f00 comp rx=0 tx=0).stop 2024-09-19T07:43:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.355+0000 7f088faf1640 1 -- 172.21.15.16:0/2830752071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f088815b300 msgr2=0x7f088814df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.355+0000 7f088faf1640 1 --2- 172.21.15.16:0/2830752071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f088815b300 0x7f088814df60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0884035040 tx=0x7f08840046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.355+0000 7f088faf1640 1 -- 172.21.15.16:0/2830752071 shutdown_connections 2024-09-19T07:43:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.355+0000 7f088faf1640 1 --2- 172.21.15.16:0/2830752071 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f086c06c2e0 0x7f086c06e7a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.355+0000 7f088faf1640 1 --2- 172.21.15.16:0/2830752071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f088815b6e0 0x7f088806c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.355+0000 7f088faf1640 1 --2- 172.21.15.16:0/2830752071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f088815b300 0x7f088814df60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.355+0000 7f088faf1640 1 --2- 172.21.15.16:0/2830752071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08880ce090 0x7f088814acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.355+0000 7f088faf1640 1 -- 172.21.15.16:0/2830752071 >> 172.21.15.16:0/2830752071 conn(0x7f08880b2d10 msgr2=0x7f08880cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.355+0000 7f088faf1640 1 -- 172.21.15.16:0/2830752071 shutdown_connections 2024-09-19T07:43:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.355+0000 7f088faf1640 1 -- 172.21.15.16:0/2830752071 wait complete. 2024-09-19T07:43:56.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-19T07:43:56.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.471+0000 7f1f541f6640 1 Processor -- start 2024-09-19T07:43:56.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.471+0000 7f1f541f6640 1 -- start start 2024-09-19T07:43:56.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.472+0000 7f1f541f6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f4c0d0070 0x7f1f4c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.472+0000 7f1f541f6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f4c0d1b10 con 0x7f1f4c15f0d0 2024-09-19T07:43:56.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.472+0000 7f1f541f6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f4c0cf2b0 con 0x7f1f4c0d4130 2024-09-19T07:43:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.472+0000 7f1f541f6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4c0cf590 con 0x7f1f4c0d0070 2024-09-19T07:43:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.472+0000 7f1f51f92640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1f4c15f0d0 0x7f1f4c0d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56146/0 (socket says 172.21.15.16:56146) 2024-09-19T07:43:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.472+0000 7f1f51f92640 1 -- 172.21.15.16:0/2691824055 learned_addr learned my addr 172.21.15.16:0/2691824055 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.473+0000 7f1f51791640 1 --2- 172.21.15.16:0/2691824055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f4c0d0070 0x7f1f4c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.473+0000 7f1f50f90640 1 -- 172.21.15.16:0/2691824055 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2299492564 0 0) 0x7f1f4c0d1b10 con 0x7f1f4c15f0d0 2024-09-19T07:43:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.473+0000 7f1f50f90640 1 -- 172.21.15.16:0/2691824055 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f34003520 con 0x7f1f4c15f0d0 2024-09-19T07:43:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.473+0000 7f1f50f90640 1 -- 172.21.15.16:0/2691824055 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 980003290 0 0) 0x7f1f4c0cf2b0 con 0x7f1f4c0d4130 2024-09-19T07:43:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.473+0000 7f1f50f90640 1 -- 172.21.15.16:0/2691824055 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f4c0d1b10 con 0x7f1f4c0d4130 2024-09-19T07:43:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.473+0000 7f1f51791640 1 -- 172.21.15.16:0/2691824055 >> v1:172.21.15.146:6789/0 conn(0x7f1f4c0d4130 legacy=0x7f1f4c0d45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.473+0000 7f1f51791640 1 -- 172.21.15.16:0/2691824055 >> v1:172.21.15.16:6789/0 conn(0x7f1f4c15f0d0 legacy=0x7f1f4c0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.473+0000 7f1f51791640 1 -- 172.21.15.16:0/2691824055 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4c0cf870 con 0x7f1f4c0d0070 2024-09-19T07:43:56.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.474+0000 7f1f51791640 1 --2- 172.21.15.16:0/2691824055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f4c0d0070 0x7f1f4c15f4b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1f400097d0 tx=0x7f1f4002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2da547b8457a954e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.474+0000 7f1f50f90640 1 -- 172.21.15.16:0/2691824055 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40037070 con 0x7f1f4c0d0070 2024-09-19T07:43:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.474+0000 7f1f50f90640 1 -- 172.21.15.16:0/2691824055 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f40032440 con 0x7f1f4c0d0070 2024-09-19T07:43:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.474+0000 7f1f50f90640 1 -- 172.21.15.16:0/2691824055 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40032810 con 0x7f1f4c0d0070 2024-09-19T07:43:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.475+0000 7f1f541f6640 1 -- 172.21.15.16:0/2691824055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f4c0d0070 msgr2=0x7f1f4c15f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.475+0000 7f1f541f6640 1 --2- 172.21.15.16:0/2691824055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f4c0d0070 0x7f1f4c15f4b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1f400097d0 tx=0x7f1f4002a180 comp rx=0 tx=0).stop 2024-09-19T07:43:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.475+0000 7f1f541f6640 1 -- 172.21.15.16:0/2691824055 shutdown_connections 2024-09-19T07:43:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.475+0000 7f1f541f6640 1 --2- 172.21.15.16:0/2691824055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f4c0d0070 0x7f1f4c15f4b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.475+0000 7f1f541f6640 1 -- 172.21.15.16:0/2691824055 >> 172.21.15.16:0/2691824055 conn(0x7f1f4c09d760 msgr2=0x7f1f4c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.475+0000 7f1f541f6640 1 -- 172.21.15.16:0/2691824055 shutdown_connections 2024-09-19T07:43:56.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.475+0000 7f1f541f6640 1 -- 172.21.15.16:0/2691824055 wait complete. 2024-09-19T07:43:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.476+0000 7f1f541f6640 1 Processor -- start 2024-09-19T07:43:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.476+0000 7f1f541f6640 1 -- start start 2024-09-19T07:43:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.477+0000 7f1f541f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f4c0d0070 0x7f1f4c15aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.477+0000 7f1f541f6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f4c0d4130 0x7f1f4c0b7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.477+0000 7f1f541f6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f4c15f0d0 0x7f1f4c0b91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.477+0000 7f1f541f6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c0b2f70 con 0x7f1f4c0d0070 2024-09-19T07:43:56.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.477+0000 7f1f541f6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c0cef60 con 0x7f1f4c0d4130 2024-09-19T07:43:56.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.477+0000 7f1f541f6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4c0b30f0 con 0x7f1f4c15f0d0 2024-09-19T07:43:56.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.477+0000 7f1f51f92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f4c0d0070 0x7f1f4c15aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.477+0000 7f1f51f92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f4c0d0070 0x7f1f4c15aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34512/0 (socket says 172.21.15.16:34512) 2024-09-19T07:43:56.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.477+0000 7f1f51f92640 1 -- 172.21.15.16:0/3267468204 learned_addr learned my addr 172.21.15.16:0/3267468204 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:56.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.477+0000 7f1f52793640 1 --2- 172.21.15.16:0/3267468204 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f4c15f0d0 0x7f1f4c0b91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.477+0000 7f1f51791640 1 --2- 172.21.15.16:0/3267468204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f4c0d4130 0x7f1f4c0b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.477+0000 7f1f51f92640 1 -- 172.21.15.16:0/3267468204 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f4c15f0d0 msgr2=0x7f1f4c0b91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.477+0000 7f1f51f92640 1 --2- 172.21.15.16:0/3267468204 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f4c15f0d0 0x7f1f4c0b91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.477+0000 7f1f51f92640 1 -- 172.21.15.16:0/3267468204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f4c0d4130 msgr2=0x7f1f4c0b7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.477+0000 7f1f51f92640 1 --2- 172.21.15.16:0/3267468204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f4c0d4130 0x7f1f4c0b7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.477+0000 7f1f51f92640 1 -- 172.21.15.16:0/3267468204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4c15b1f0 con 0x7f1f4c0d0070 2024-09-19T07:43:56.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.478+0000 7f1f51f92640 1 --2- 172.21.15.16:0/3267468204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f4c0d0070 0x7f1f4c15aaf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00cb50 tx=0x7f1f3c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:56.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.478+0000 7f1f4a7fc640 1 -- 172.21.15.16:0/3267468204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c017070 con 0x7f1f4c0d0070 2024-09-19T07:43:56.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.478+0000 7f1f4a7fc640 1 -- 172.21.15.16:0/3267468204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f3c00ad40 con 0x7f1f4c0d0070 2024-09-19T07:43:56.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.478+0000 7f1f4a7fc640 1 -- 172.21.15.16:0/3267468204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c012620 con 0x7f1f4c0d0070 2024-09-19T07:43:56.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.478+0000 7f1f541f6640 1 -- 172.21.15.16:0/3267468204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4c15b4d0 con 0x7f1f4c0d0070 2024-09-19T07:43:56.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.478+0000 7f1f541f6640 1 -- 172.21.15.16:0/3267468204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f4c15ba90 con 0x7f1f4c0d0070 2024-09-19T07:43:56.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.479+0000 7f1f541f6640 1 -- 172.21.15.16:0/3267468204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f4c15f4b0 con 0x7f1f4c0d0070 2024-09-19T07:43:56.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.480+0000 7f1f4a7fc640 1 -- 172.21.15.16:0/3267468204 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1f3c01a070 con 0x7f1f4c0d0070 2024-09-19T07:43:56.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.480+0000 7f1f4a7fc640 1 --2- 172.21.15.16:0/3267468204 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1f3806c2b0 0x7f1f3806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.480+0000 7f1f4a7fc640 1 -- 172.21.15.16:0/3267468204 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f1f3c08b0a0 con 0x7f1f4c0d0070 2024-09-19T07:43:56.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.480+0000 7f1f51791640 1 --2- 172.21.15.16:0/3267468204 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1f3806c2b0 0x7f1f3806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.481+0000 7f1f51791640 1 --2- 172.21.15.16:0/3267468204 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1f3806c2b0 0x7f1f3806e770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1f40034000 tx=0x7f1f4002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:56.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.484+0000 7f1f4a7fc640 1 -- 172.21.15.16:0/3267468204 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1f3c0599d0 con 0x7f1f4c0d0070 2024-09-19T07:43:56.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.728+0000 7f1f541f6640 1 -- 172.21.15.16:0/3267468204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f1f4c06cf30 con 0x7f1f4c0d0070 2024-09-19T07:43:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.730+0000 7f1f4a7fc640 1 -- 172.21.15.16:0/3267468204 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f1f3c0592d0 con 0x7f1f4c0d0070 2024-09-19T07:43:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.732+0000 7f1f541f6640 1 -- 172.21.15.16:0/3267468204 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1f3806c2b0 msgr2=0x7f1f3806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.732+0000 7f1f541f6640 1 --2- 172.21.15.16:0/3267468204 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1f3806c2b0 0x7f1f3806e770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1f40034000 tx=0x7f1f4002e000 comp rx=0 tx=0).stop 2024-09-19T07:43:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.732+0000 7f1f541f6640 1 -- 172.21.15.16:0/3267468204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f4c0d0070 msgr2=0x7f1f4c15aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.732+0000 7f1f541f6640 1 --2- 172.21.15.16:0/3267468204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f4c0d0070 0x7f1f4c15aaf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00cb50 tx=0x7f1f3c00a430 comp rx=0 tx=0).stop 2024-09-19T07:43:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.732+0000 7f1f541f6640 1 -- 172.21.15.16:0/3267468204 shutdown_connections 2024-09-19T07:43:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.732+0000 7f1f541f6640 1 --2- 172.21.15.16:0/3267468204 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1f3806c2b0 0x7f1f3806e770 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.732+0000 7f1f541f6640 1 --2- 172.21.15.16:0/3267468204 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f4c15f0d0 0x7f1f4c0b91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.732+0000 7f1f541f6640 1 --2- 172.21.15.16:0/3267468204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f4c0d4130 0x7f1f4c0b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.732+0000 7f1f541f6640 1 --2- 172.21.15.16:0/3267468204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f4c0d0070 0x7f1f4c15aaf0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.732+0000 7f1f541f6640 1 -- 172.21.15.16:0/3267468204 >> 172.21.15.16:0/3267468204 conn(0x7f1f4c09d760 msgr2=0x7f1f4c0b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.732+0000 7f1f541f6640 1 -- 172.21.15.16:0/3267468204 shutdown_connections 2024-09-19T07:43:56.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.732+0000 7f1f541f6640 1 -- 172.21.15.16:0/3267468204 wait complete. 2024-09-19T07:43:56.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-19T07:43:56.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.849+0000 7f1962d8d640 1 Processor -- start 2024-09-19T07:43:56.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.849+0000 7f1962d8d640 1 -- start start 2024-09-19T07:43:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.850+0000 7f1962d8d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f195c15b120 0x7f195c06f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.850+0000 7f1962d8d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f195c06dec0 0x7f195c06e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.850+0000 7f1962d8d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f195c0d04b0 0x7f195c0d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.850+0000 7f1962d8d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f195c0d0e40 con 0x7f195c0d04b0 2024-09-19T07:43:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.850+0000 7f1962d8d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f195c15b500 con 0x7f195c15b120 2024-09-19T07:43:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.850+0000 7f1962d8d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f195c06e940 con 0x7f195c06dec0 2024-09-19T07:43:56.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.850+0000 7f196132a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f195c0d04b0 0x7f195c0d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.850+0000 7f196132a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f195c0d04b0 0x7f195c0d0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34514/0 (socket says 172.21.15.16:34514) 2024-09-19T07:43:56.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.850+0000 7f196132a640 1 -- 172.21.15.16:0/969616776 learned_addr learned my addr 172.21.15.16:0/969616776 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:56.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.850+0000 7f1960b29640 1 --2- 172.21.15.16:0/969616776 >> v2:172.21.15.146:3300/0 conn(0x7f195c15b120 0x7f195c06f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.850+0000 7f195bfff640 1 --2- 172.21.15.16:0/969616776 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f195c06dec0 0x7f195c06e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.850+0000 7f196132a640 1 -- 172.21.15.16:0/969616776 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f195c06dec0 msgr2=0x7f195c06e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:56.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.850+0000 7f196132a640 1 --2- 172.21.15.16:0/969616776 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f195c06dec0 0x7f195c06e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.850+0000 7f196132a640 1 -- 172.21.15.16:0/969616776 >> v2:172.21.15.146:3300/0 conn(0x7f195c15b120 msgr2=0x7f195c06f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:56.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.850+0000 7f196132a640 1 --2- 172.21.15.16:0/969616776 >> v2:172.21.15.146:3300/0 conn(0x7f195c15b120 0x7f195c06f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.850+0000 7f196132a640 1 -- 172.21.15.16:0/969616776 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195c06ec20 con 0x7f195c0d04b0 2024-09-19T07:43:56.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.851+0000 7f196132a640 1 --2- 172.21.15.16:0/969616776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f195c0d04b0 0x7f195c0d0900 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f195c06cb50 tx=0x7f195402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b657e161aed617 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.851+0000 7f195b7fe640 1 -- 172.21.15.16:0/969616776 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954036070 con 0x7f195c0d04b0 2024-09-19T07:43:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.851+0000 7f195b7fe640 1 -- 172.21.15.16:0/969616776 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1954031600 con 0x7f195c0d04b0 2024-09-19T07:43:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.851+0000 7f195b7fe640 1 -- 172.21.15.16:0/969616776 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19540319d0 con 0x7f195c0d04b0 2024-09-19T07:43:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.852+0000 7f1962d8d640 1 -- 172.21.15.16:0/969616776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f195c0d04b0 msgr2=0x7f195c0d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.852+0000 7f1962d8d640 1 --2- 172.21.15.16:0/969616776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f195c0d04b0 0x7f195c0d0900 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f195c06cb50 tx=0x7f195402a6a0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.852+0000 7f1962d8d640 1 -- 172.21.15.16:0/969616776 shutdown_connections 2024-09-19T07:43:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.852+0000 7f1962d8d640 1 --2- 172.21.15.16:0/969616776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f195c0d04b0 0x7f195c0d0900 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.852+0000 7f1962d8d640 1 --2- 172.21.15.16:0/969616776 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f195c06dec0 0x7f195c06e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.852+0000 7f1962d8d640 1 --2- 172.21.15.16:0/969616776 >> v2:172.21.15.146:3300/0 conn(0x7f195c15b120 0x7f195c06f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.852+0000 7f1962d8d640 1 -- 172.21.15.16:0/969616776 >> 172.21.15.16:0/969616776 conn(0x7f195c06bdf0 msgr2=0x7f195c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.852+0000 7f1962d8d640 1 -- 172.21.15.16:0/969616776 shutdown_connections 2024-09-19T07:43:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.852+0000 7f1962d8d640 1 -- 172.21.15.16:0/969616776 wait complete. 2024-09-19T07:43:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.853+0000 7f1962d8d640 1 Processor -- start 2024-09-19T07:43:56.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.853+0000 7f1962d8d640 1 -- start start 2024-09-19T07:43:56.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.854+0000 7f1962d8d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f195c06dec0 0x7f195c0b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.854+0000 7f1962d8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f195c0d04b0 0x7f195c146260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.854+0000 7f1962d8d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f195c15b120 0x7f195c1467a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.854+0000 7f1962d8d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f195c06f4c0 con 0x7f195c0d04b0 2024-09-19T07:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.854+0000 7f1962d8d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f195c06f1e0 con 0x7f195c15b120 2024-09-19T07:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.854+0000 7f1962d8d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f195c0ce060 con 0x7f195c06dec0 2024-09-19T07:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.854+0000 7f195bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f195c0d04b0 0x7f195c146260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.854+0000 7f195bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f195c0d04b0 0x7f195c146260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34526/0 (socket says 172.21.15.16:34526) 2024-09-19T07:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.854+0000 7f195bfff640 1 -- 172.21.15.16:0/1935764694 learned_addr learned my addr 172.21.15.16:0/1935764694 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.855+0000 7f196132a640 1 --2- 172.21.15.16:0/1935764694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f195c15b120 0x7f195c1467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.855+0000 7f1960b29640 1 --2- 172.21.15.16:0/1935764694 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f195c06dec0 0x7f195c0b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.855+0000 7f195bfff640 1 -- 172.21.15.16:0/1935764694 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f195c06dec0 msgr2=0x7f195c0b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.855+0000 7f195bfff640 1 --2- 172.21.15.16:0/1935764694 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f195c06dec0 0x7f195c0b0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.855+0000 7f195bfff640 1 -- 172.21.15.16:0/1935764694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f195c15b120 msgr2=0x7f195c1467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.855+0000 7f195bfff640 1 --2- 172.21.15.16:0/1935764694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f195c15b120 0x7f195c1467a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.855+0000 7f195bfff640 1 -- 172.21.15.16:0/1935764694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195c0c7700 con 0x7f195c0d04b0 2024-09-19T07:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.855+0000 7f195bfff640 1 --2- 172.21.15.16:0/1935764694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f195c0d04b0 0x7f195c146260 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f194800e8a0 tx=0x7f194800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.855+0000 7f19597fa640 1 -- 172.21.15.16:0/1935764694 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948017070 con 0x7f195c0d04b0 2024-09-19T07:43:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.855+0000 7f19597fa640 1 -- 172.21.15.16:0/1935764694 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f194800c940 con 0x7f195c0d04b0 2024-09-19T07:43:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.855+0000 7f1962d8d640 1 -- 172.21.15.16:0/1935764694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f195c0c79e0 con 0x7f195c0d04b0 2024-09-19T07:43:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.855+0000 7f19597fa640 1 -- 172.21.15.16:0/1935764694 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194800cd30 con 0x7f195c0d04b0 2024-09-19T07:43:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.855+0000 7f1962d8d640 1 -- 172.21.15.16:0/1935764694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f195c0c7fa0 con 0x7f195c0d04b0 2024-09-19T07:43:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.856+0000 7f1962d8d640 1 -- 172.21.15.16:0/1935764694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f195c075710 con 0x7f195c0d04b0 2024-09-19T07:43:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.857+0000 7f19597fa640 1 -- 172.21.15.16:0/1935764694 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f194801a070 con 0x7f195c0d04b0 2024-09-19T07:43:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.857+0000 7f19597fa640 1 --2- 172.21.15.16:0/1935764694 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f194006c2b0 0x7f194006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.857+0000 7f19597fa640 1 -- 172.21.15.16:0/1935764694 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f194808b4c0 con 0x7f195c0d04b0 2024-09-19T07:43:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.857+0000 7f1960b29640 1 --2- 172.21.15.16:0/1935764694 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f194006c2b0 0x7f194006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.858+0000 7f1960b29640 1 --2- 172.21.15.16:0/1935764694 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f194006c2b0 0x7f194006e770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1950009990 tx=0x7f1950008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:56.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:56.861+0000 7f19597fa640 1 -- 172.21.15.16:0/1935764694 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f195c075710 con 0x7f195c0d04b0 2024-09-19T07:43:57.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.107+0000 7f1962d8d640 1 -- 172.21.15.16:0/1935764694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f195c0cf4a0 con 0x7f195c0d04b0 2024-09-19T07:43:57.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.109+0000 7f19597fa640 1 -- 172.21.15.16:0/1935764694 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f1948059df0 con 0x7f195c0d04b0 2024-09-19T07:43:57.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.111+0000 7f1962d8d640 1 -- 172.21.15.16:0/1935764694 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f194006c2b0 msgr2=0x7f194006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:57.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.111+0000 7f1962d8d640 1 --2- 172.21.15.16:0/1935764694 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f194006c2b0 0x7f194006e770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1950009990 tx=0x7f1950008040 comp rx=0 tx=0).stop 2024-09-19T07:43:57.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.111+0000 7f1962d8d640 1 -- 172.21.15.16:0/1935764694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f195c0d04b0 msgr2=0x7f195c146260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:57.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.111+0000 7f1962d8d640 1 --2- 172.21.15.16:0/1935764694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f195c0d04b0 0x7f195c146260 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f194800e8a0 tx=0x7f194800ed60 comp rx=0 tx=0).stop 2024-09-19T07:43:57.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.111+0000 7f1962d8d640 1 -- 172.21.15.16:0/1935764694 shutdown_connections 2024-09-19T07:43:57.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.111+0000 7f1962d8d640 1 --2- 172.21.15.16:0/1935764694 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f194006c2b0 0x7f194006e770 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.111+0000 7f1962d8d640 1 --2- 172.21.15.16:0/1935764694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f195c15b120 0x7f195c1467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.111+0000 7f1962d8d640 1 --2- 172.21.15.16:0/1935764694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f195c0d04b0 0x7f195c146260 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.111+0000 7f1962d8d640 1 --2- 172.21.15.16:0/1935764694 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f195c06dec0 0x7f195c0b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.111+0000 7f1962d8d640 1 -- 172.21.15.16:0/1935764694 >> 172.21.15.16:0/1935764694 conn(0x7f195c06bdf0 msgr2=0x7f195c0cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:57.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.111+0000 7f1962d8d640 1 -- 172.21.15.16:0/1935764694 shutdown_connections 2024-09-19T07:43:57.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.111+0000 7f1962d8d640 1 -- 172.21.15.16:0/1935764694 wait complete. 2024-09-19T07:43:57.122 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-19T07:43:57.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.225+0000 7f3e16798640 1 Processor -- start 2024-09-19T07:43:57.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.225+0000 7f3e16798640 1 -- start start 2024-09-19T07:43:57.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.226+0000 7f3e16798640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e100b4d90 0x7f3e100b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:57.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.226+0000 7f3e16798640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3e100b45c0 0x7f3e1015ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:57.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.226+0000 7f3e16798640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e1015c610 con 0x7f3e100b5170 2024-09-19T07:43:57.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.226+0000 7f3e16798640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f3e1015d220 con 0x7f3e100b45c0 2024-09-19T07:43:57.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.226+0000 7f3e16798640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3e1015c8f0 con 0x7f3e100b4d90 2024-09-19T07:43:57.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.226+0000 7f3e07fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3e100b45c0 0x7f3e1015ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:57.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.226+0000 7f3e14d35640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3e100b5170 0x7f3e1015eda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56154/0 (socket says 172.21.15.16:56154) 2024-09-19T07:43:57.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.226+0000 7f3e07fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3e100b45c0 0x7f3e1015ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34532/0 (socket says 172.21.15.16:34532) 2024-09-19T07:43:57.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.226+0000 7f3e14d35640 1 -- 172.21.15.16:0/1748454552 learned_addr learned my addr 172.21.15.16:0/1748454552 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:57.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.227+0000 7f3e0ffff640 1 --2- 172.21.15.16:0/1748454552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e100b4d90 0x7f3e100b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:57.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.227+0000 7f3e07fff640 1 -- 172.21.15.16:0/1748454552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e100b4d90 msgr2=0x7f3e100b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:57.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.227+0000 7f3e07fff640 1 --2- 172.21.15.16:0/1748454552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e100b4d90 0x7f3e100b4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.227+0000 7f3e07fff640 1 -- 172.21.15.16:0/1748454552 >> v1:172.21.15.16:6789/0 conn(0x7f3e100b5170 legacy=0x7f3e1015eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:57.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.227+0000 7f3e07fff640 1 -- 172.21.15.16:0/1748454552 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1015cbd0 con 0x7f3e100b45c0 2024-09-19T07:43:57.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.227+0000 7f3e07fff640 1 --2- 172.21.15.16:0/1748454552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e100b45c0 0x7f3e1015ac50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3df80097a0 tx=0x7f3df802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab2487d92a2a0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:57.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.228+0000 7f3e0f7fe640 1 -- 172.21.15.16:0/1748454552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8037070 con 0x7f3e100b45c0 2024-09-19T07:43:57.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.228+0000 7f3e0f7fe640 1 -- 172.21.15.16:0/1748454552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df8032440 con 0x7f3e100b45c0 2024-09-19T07:43:57.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.228+0000 7f3e0f7fe640 1 -- 172.21.15.16:0/1748454552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8032810 con 0x7f3e100b45c0 2024-09-19T07:43:57.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.228+0000 7f3e16798640 1 -- 172.21.15.16:0/1748454552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e100b45c0 msgr2=0x7f3e1015ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:57.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.228+0000 7f3e16798640 1 --2- 172.21.15.16:0/1748454552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e100b45c0 0x7f3e1015ac50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3df80097a0 tx=0x7f3df802a210 comp rx=0 tx=0).stop 2024-09-19T07:43:57.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.228+0000 7f3e16798640 1 -- 172.21.15.16:0/1748454552 shutdown_connections 2024-09-19T07:43:57.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.228+0000 7f3e16798640 1 --2- 172.21.15.16:0/1748454552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e100b45c0 0x7f3e1015ac50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.228+0000 7f3e16798640 1 --2- 172.21.15.16:0/1748454552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e100b4d90 0x7f3e100b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.228+0000 7f3e16798640 1 -- 172.21.15.16:0/1748454552 >> 172.21.15.16:0/1748454552 conn(0x7f3e1009a9f0 msgr2=0x7f3e100b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:57.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.228+0000 7f3e16798640 1 -- 172.21.15.16:0/1748454552 shutdown_connections 2024-09-19T07:43:57.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.228+0000 7f3e16798640 1 -- 172.21.15.16:0/1748454552 wait complete. 2024-09-19T07:43:57.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.230+0000 7f3e16798640 1 Processor -- start 2024-09-19T07:43:57.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.230+0000 7f3e16798640 1 -- start start 2024-09-19T07:43:57.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.230+0000 7f3e16798640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e100b4d90 0x7f3e10142d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:57.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.230+0000 7f3e16798640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e100b5170 0x7f3e10153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:57.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.230+0000 7f3e16798640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e101436b0 0x7f3e10153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:57.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.230+0000 7f3e16798640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1015c3d0 con 0x7f3e100b4d90 2024-09-19T07:43:57.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.230+0000 7f3e16798640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1015c250 con 0x7f3e100b5170 2024-09-19T07:43:57.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.230+0000 7f3e16798640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3e1015e6c0 con 0x7f3e101436b0 2024-09-19T07:43:57.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.230+0000 7f3e0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e100b4d90 0x7f3e10142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:57.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.231+0000 7f3e0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e100b4d90 0x7f3e10142d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34538/0 (socket says 172.21.15.16:34538) 2024-09-19T07:43:57.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.231+0000 7f3e0ffff640 1 -- 172.21.15.16:0/1280161891 learned_addr learned my addr 172.21.15.16:0/1280161891 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:57.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.231+0000 7f3e07fff640 1 --2- 172.21.15.16:0/1280161891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e100b5170 0x7f3e10153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:57.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.231+0000 7f3e14d35640 1 --2- 172.21.15.16:0/1280161891 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e101436b0 0x7f3e10153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:57.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.231+0000 7f3e0ffff640 1 -- 172.21.15.16:0/1280161891 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e101436b0 msgr2=0x7f3e10153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:57.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.231+0000 7f3e0ffff640 1 --2- 172.21.15.16:0/1280161891 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e101436b0 0x7f3e10153da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.231+0000 7f3e0ffff640 1 -- 172.21.15.16:0/1280161891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e100b5170 msgr2=0x7f3e10153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:57.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.231+0000 7f3e0ffff640 1 --2- 172.21.15.16:0/1280161891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e100b5170 0x7f3e10153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.231+0000 7f3e0ffff640 1 -- 172.21.15.16:0/1280161891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e10143b70 con 0x7f3e100b4d90 2024-09-19T07:43:57.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.231+0000 7f3e14d35640 1 --2- 172.21.15.16:0/1280161891 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e101436b0 0x7f3e10153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:57.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.231+0000 7f3e0ffff640 1 --2- 172.21.15.16:0/1280161891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e100b4d90 0x7f3e10142d60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000d800 tx=0x7f3e0000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:57.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.231+0000 7f3e0d7fa640 1 -- 172.21.15.16:0/1280161891 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0000b660 con 0x7f3e100b4d90 2024-09-19T07:43:57.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.231+0000 7f3e0d7fa640 1 -- 172.21.15.16:0/1280161891 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e00019070 con 0x7f3e100b4d90 2024-09-19T07:43:57.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.231+0000 7f3e16798640 1 -- 172.21.15.16:0/1280161891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e10143e50 con 0x7f3e100b4d90 2024-09-19T07:43:57.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.231+0000 7f3e0d7fa640 1 -- 172.21.15.16:0/1280161891 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0000ba20 con 0x7f3e100b4d90 2024-09-19T07:43:57.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.231+0000 7f3e16798640 1 -- 172.21.15.16:0/1280161891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e10144410 con 0x7f3e100b4d90 2024-09-19T07:43:57.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.232+0000 7f3e16798640 1 -- 172.21.15.16:0/1280161891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e100b4080 con 0x7f3e100b4d90 2024-09-19T07:43:57.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.233+0000 7f3e0d7fa640 1 -- 172.21.15.16:0/1280161891 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3e0000bbc0 con 0x7f3e100b4d90 2024-09-19T07:43:57.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.233+0000 7f3e0d7fa640 1 --2- 172.21.15.16:0/1280161891 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3df006c2e0 0x7f3df006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:57.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.233+0000 7f3e0d7fa640 1 -- 172.21.15.16:0/1280161891 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f3e0008b520 con 0x7f3e100b4d90 2024-09-19T07:43:57.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.234+0000 7f3e07fff640 1 --2- 172.21.15.16:0/1280161891 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3df006c2e0 0x7f3df006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:57.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.234+0000 7f3e07fff640 1 --2- 172.21.15.16:0/1280161891 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3df006c2e0 0x7f3df006e7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3df8034000 tx=0x7f3df802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:57.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.238+0000 7f3e0d7fa640 1 -- 172.21.15.16:0/1280161891 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3e00059ee0 con 0x7f3e100b4d90 2024-09-19T07:43:57.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.481+0000 7f3e16798640 1 -- 172.21.15.16:0/1280161891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f3e1006f610 con 0x7f3e100b4d90 2024-09-19T07:43:57.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.483+0000 7f3e0d7fa640 1 -- 172.21.15.16:0/1280161891 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f3e000597e0 con 0x7f3e100b4d90 2024-09-19T07:43:57.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.486+0000 7f3e16798640 1 -- 172.21.15.16:0/1280161891 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3df006c2e0 msgr2=0x7f3df006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:57.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.486+0000 7f3e16798640 1 --2- 172.21.15.16:0/1280161891 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3df006c2e0 0x7f3df006e7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3df8034000 tx=0x7f3df802e000 comp rx=0 tx=0).stop 2024-09-19T07:43:57.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.486+0000 7f3e16798640 1 -- 172.21.15.16:0/1280161891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e100b4d90 msgr2=0x7f3e10142d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:57.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.486+0000 7f3e16798640 1 --2- 172.21.15.16:0/1280161891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e100b4d90 0x7f3e10142d60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000d800 tx=0x7f3e0000dcc0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.486+0000 7f3e16798640 1 -- 172.21.15.16:0/1280161891 shutdown_connections 2024-09-19T07:43:57.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.486+0000 7f3e16798640 1 --2- 172.21.15.16:0/1280161891 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3df006c2e0 0x7f3df006e7a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.486+0000 7f3e16798640 1 --2- 172.21.15.16:0/1280161891 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e101436b0 0x7f3e10153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.486+0000 7f3e16798640 1 --2- 172.21.15.16:0/1280161891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e100b5170 0x7f3e10153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.486+0000 7f3e16798640 1 --2- 172.21.15.16:0/1280161891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e100b4d90 0x7f3e10142d60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.486+0000 7f3e16798640 1 -- 172.21.15.16:0/1280161891 >> 172.21.15.16:0/1280161891 conn(0x7f3e1009a9f0 msgr2=0x7f3e10155d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:57.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.486+0000 7f3e16798640 1 -- 172.21.15.16:0/1280161891 shutdown_connections 2024-09-19T07:43:57.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.486+0000 7f3e16798640 1 -- 172.21.15.16:0/1280161891 wait complete. 2024-09-19T07:43:57.497 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-19T07:43:57.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.600+0000 7fdde6477640 1 Processor -- start 2024-09-19T07:43:57.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.600+0000 7fdde6477640 1 -- start start 2024-09-19T07:43:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.602+0000 7fdde6477640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdde00b47c0 0x7fdde0162c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.602+0000 7fdde6477640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdde009a9f0 0x7fdde009ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.602+0000 7fdde6477640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdde00d05e0 con 0x7fdde0161af0 2024-09-19T07:43:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.602+0000 7fdde6477640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fdde00d03b0 con 0x7fdde009a9f0 2024-09-19T07:43:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.602+0000 7fdde6477640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdde0163150 con 0x7fdde00b47c0 2024-09-19T07:43:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.602+0000 7fdddffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdde0161af0 0x7fdde00d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56164/0 (socket says 172.21.15.16:56164) 2024-09-19T07:43:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.602+0000 7fdde4a14640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdde009a9f0 0x7fdde009ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.602+0000 7fdddffff640 1 -- 172.21.15.16:0/3449186318 learned_addr learned my addr 172.21.15.16:0/3449186318 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.602+0000 7fdddf7fe640 1 --2- 172.21.15.16:0/3449186318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdde00b47c0 0x7fdde0162c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.602+0000 7fdde4a14640 1 -- 172.21.15.16:0/3449186318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdde00b47c0 msgr2=0x7fdde0162c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.602+0000 7fdde4a14640 1 --2- 172.21.15.16:0/3449186318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdde00b47c0 0x7fdde0162c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.602+0000 7fdde4a14640 1 -- 172.21.15.16:0/3449186318 >> v1:172.21.15.16:6789/0 conn(0x7fdde0161af0 legacy=0x7fdde00d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.602+0000 7fdde4a14640 1 -- 172.21.15.16:0/3449186318 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde00d08c0 con 0x7fdde009a9f0 2024-09-19T07:43:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.602+0000 7fdddeffd640 1 -- 172.21.15.16:0/3449186318 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1878968696 0 0) 0x7fdde00d05e0 con 0x7fdde0161af0 2024-09-19T07:43:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.603+0000 7fdde4a14640 1 --2- 172.21.15.16:0/3449186318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde009a9f0 0x7fdde009ae80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009dc0 tx=0x7fddd802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b928367f19891ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.603+0000 7fdddeffd640 1 -- 172.21.15.16:0/3449186318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8007740 con 0x7fdde009a9f0 2024-09-19T07:43:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.603+0000 7fdddeffd640 1 -- 172.21.15.16:0/3449186318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd80078e0 con 0x7fdde009a9f0 2024-09-19T07:43:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.603+0000 7fdddeffd640 1 -- 172.21.15.16:0/3449186318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8007cb0 con 0x7fdde009a9f0 2024-09-19T07:43:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.604+0000 7fdde6477640 1 -- 172.21.15.16:0/3449186318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde009a9f0 msgr2=0x7fdde009ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.604+0000 7fdde6477640 1 --2- 172.21.15.16:0/3449186318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde009a9f0 0x7fdde009ae80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009dc0 tx=0x7fddd802a710 comp rx=0 tx=0).stop 2024-09-19T07:43:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.604+0000 7fdde6477640 1 -- 172.21.15.16:0/3449186318 shutdown_connections 2024-09-19T07:43:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.604+0000 7fdde6477640 1 --2- 172.21.15.16:0/3449186318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde009a9f0 0x7fdde009ae80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.604+0000 7fdde6477640 1 --2- 172.21.15.16:0/3449186318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdde00b47c0 0x7fdde0162c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.604+0000 7fdde6477640 1 -- 172.21.15.16:0/3449186318 >> 172.21.15.16:0/3449186318 conn(0x7fdde00b3ca0 msgr2=0x7fdde00b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.604+0000 7fdde6477640 1 -- 172.21.15.16:0/3449186318 shutdown_connections 2024-09-19T07:43:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.604+0000 7fdde6477640 1 -- 172.21.15.16:0/3449186318 wait complete. 2024-09-19T07:43:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.605+0000 7fdde6477640 1 Processor -- start 2024-09-19T07:43:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.605+0000 7fdde6477640 1 -- start start 2024-09-19T07:43:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.606+0000 7fdde6477640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdde009a9f0 0x7fdde00b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.606+0000 7fdde6477640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdde00b47c0 0x7fdde01568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.606+0000 7fdde6477640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde0161af0 0x7fdde015af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.606+0000 7fdde6477640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdde00d1160 con 0x7fdde0161af0 2024-09-19T07:43:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.606+0000 7fdde6477640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdde00d0e80 con 0x7fdde00b47c0 2024-09-19T07:43:57.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.606+0000 7fdde6477640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdde0161ed0 con 0x7fdde009a9f0 2024-09-19T07:43:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.606+0000 7fdde4a14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde0161af0 0x7fdde015af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.606+0000 7fdde4a14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde0161af0 0x7fdde015af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34560/0 (socket says 172.21.15.16:34560) 2024-09-19T07:43:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.606+0000 7fdde4a14640 1 -- 172.21.15.16:0/1742536512 learned_addr learned my addr 172.21.15.16:0/1742536512 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.606+0000 7fdddffff640 1 --2- 172.21.15.16:0/1742536512 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdde009a9f0 0x7fdde00b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.606+0000 7fdddf7fe640 1 --2- 172.21.15.16:0/1742536512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdde00b47c0 0x7fdde01568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.606+0000 7fdde4a14640 1 -- 172.21.15.16:0/1742536512 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdde009a9f0 msgr2=0x7fdde00b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.606+0000 7fdde4a14640 1 --2- 172.21.15.16:0/1742536512 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdde009a9f0 0x7fdde00b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.606+0000 7fdde4a14640 1 -- 172.21.15.16:0/1742536512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdde00b47c0 msgr2=0x7fdde01568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.606+0000 7fdde4a14640 1 --2- 172.21.15.16:0/1742536512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdde00b47c0 0x7fdde01568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.606+0000 7fdde4a14640 1 -- 172.21.15.16:0/1742536512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde00b6300 con 0x7fdde0161af0 2024-09-19T07:43:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.606+0000 7fdddffff640 1 --2- 172.21.15.16:0/1742536512 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdde009a9f0 0x7fdde00b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:57.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.607+0000 7fdde4a14640 1 --2- 172.21.15.16:0/1742536512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde0161af0 0x7fdde015af90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fddd802ad00 tx=0x7fddd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.607+0000 7fdddcff9640 1 -- 172.21.15.16:0/1742536512 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd803f070 con 0x7fdde0161af0 2024-09-19T07:43:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.607+0000 7fdddcff9640 1 -- 172.21.15.16:0/1742536512 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd8031bc0 con 0x7fdde0161af0 2024-09-19T07:43:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.607+0000 7fdde6477640 1 -- 172.21.15.16:0/1742536512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde00b5810 con 0x7fdde0161af0 2024-09-19T07:43:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.607+0000 7fdddcff9640 1 -- 172.21.15.16:0/1742536512 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8044460 con 0x7fdde0161af0 2024-09-19T07:43:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.607+0000 7fdde6477640 1 -- 172.21.15.16:0/1742536512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde00b5dd0 con 0x7fdde0161af0 2024-09-19T07:43:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.608+0000 7fdde6477640 1 -- 172.21.15.16:0/1742536512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdde00d05e0 con 0x7fdde0161af0 2024-09-19T07:43:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.609+0000 7fdddcff9640 1 -- 172.21.15.16:0/1742536512 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fddd803b410 con 0x7fdde0161af0 2024-09-19T07:43:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.609+0000 7fdddcff9640 1 --2- 172.21.15.16:0/1742536512 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fddcc06c2b0 0x7fddcc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.609+0000 7fdddcff9640 1 -- 172.21.15.16:0/1742536512 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fddd80aca10 con 0x7fdde0161af0 2024-09-19T07:43:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.610+0000 7fdddffff640 1 --2- 172.21.15.16:0/1742536512 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fddcc06c2b0 0x7fddcc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.610+0000 7fdddffff640 1 --2- 172.21.15.16:0/1742536512 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fddcc06c2b0 0x7fddcc06e770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fddd0000f60 tx=0x7fddd0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.617+0000 7fdddcff9640 1 -- 172.21.15.16:0/1742536512 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fddd807b330 con 0x7fdde0161af0 2024-09-19T07:43:57.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.859+0000 7fdde6477640 1 -- 172.21.15.16:0/1742536512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fdde009b850 con 0x7fdde0161af0 2024-09-19T07:43:57.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.861+0000 7fdddcff9640 1 -- 172.21.15.16:0/1742536512 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7fddd807ac30 con 0x7fdde0161af0 2024-09-19T07:43:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.863+0000 7fdde6477640 1 -- 172.21.15.16:0/1742536512 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fddcc06c2b0 msgr2=0x7fddcc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.863+0000 7fdde6477640 1 --2- 172.21.15.16:0/1742536512 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fddcc06c2b0 0x7fddcc06e770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fddd0000f60 tx=0x7fddd0009040 comp rx=0 tx=0).stop 2024-09-19T07:43:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.863+0000 7fdde6477640 1 -- 172.21.15.16:0/1742536512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde0161af0 msgr2=0x7fdde015af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:57.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.863+0000 7fdde6477640 1 --2- 172.21.15.16:0/1742536512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde0161af0 0x7fdde015af90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fddd802ad00 tx=0x7fddd8004a40 comp rx=0 tx=0).stop 2024-09-19T07:43:57.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.864+0000 7fdde6477640 1 -- 172.21.15.16:0/1742536512 shutdown_connections 2024-09-19T07:43:57.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.864+0000 7fdde6477640 1 --2- 172.21.15.16:0/1742536512 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fddcc06c2b0 0x7fddcc06e770 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.864+0000 7fdde6477640 1 --2- 172.21.15.16:0/1742536512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde0161af0 0x7fdde015af90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.864+0000 7fdde6477640 1 --2- 172.21.15.16:0/1742536512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdde00b47c0 0x7fdde01568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.864+0000 7fdde6477640 1 --2- 172.21.15.16:0/1742536512 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdde009a9f0 0x7fdde00b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.864+0000 7fdde6477640 1 -- 172.21.15.16:0/1742536512 >> 172.21.15.16:0/1742536512 conn(0x7fdde00b3ca0 msgr2=0x7fdde0070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:57.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.864+0000 7fdde6477640 1 -- 172.21.15.16:0/1742536512 shutdown_connections 2024-09-19T07:43:57.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.864+0000 7fdde6477640 1 -- 172.21.15.16:0/1742536512 wait complete. 2024-09-19T07:43:57.875 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-19T07:43:57.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.978+0000 7fbab5e47640 1 Processor -- start 2024-09-19T07:43:57.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.978+0000 7fbab5e47640 1 -- start start 2024-09-19T07:43:57.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.979+0000 7fbab5e47640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbab015e2a0 0x7fbab006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:57.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.979+0000 7fbab5e47640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbab015e680 0x7fbab00cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:57.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.979+0000 7fbab5e47640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbab00d1880 con 0x7fbab006d960 2024-09-19T07:43:57.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.979+0000 7fbab5e47640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fbab00d1b60 con 0x7fbab015e680 2024-09-19T07:43:57.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.979+0000 7fbab5e47640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbab00cf2d0 con 0x7fbab015e2a0 2024-09-19T07:43:57.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.979+0000 7fbaaeffd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbab015e680 0x7fbab00cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:57.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.979+0000 7fbaaf7fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbab015e2a0 0x7fbab006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:57.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.979+0000 7fbaaffff640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fbab006d960 0x7fbab00d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:40082/0 (socket says 172.21.15.16:40082) 2024-09-19T07:43:57.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.979+0000 7fbaaeffd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbab015e680 0x7fbab00cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:57612/0 (socket says 172.21.15.16:57612) 2024-09-19T07:43:57.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.979+0000 7fbaaffff640 1 -- 172.21.15.16:0/415187710 learned_addr learned my addr 172.21.15.16:0/415187710 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:43:57.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.980+0000 7fbaaf7fe640 1 -- 172.21.15.16:0/415187710 >> v2:172.21.15.146:3300/0 conn(0x7fbab015e680 msgr2=0x7fbab00cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:57.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.980+0000 7fbaaf7fe640 1 --2- 172.21.15.16:0/415187710 >> v2:172.21.15.146:3300/0 conn(0x7fbab015e680 0x7fbab00cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.980+0000 7fbaaf7fe640 1 -- 172.21.15.16:0/415187710 >> v1:172.21.15.146:6789/0 conn(0x7fbab006d960 legacy=0x7fbab00d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:57.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.980+0000 7fbaaf7fe640 1 -- 172.21.15.16:0/415187710 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab00cf5b0 con 0x7fbab015e2a0 2024-09-19T07:43:57.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.980+0000 7fbaae7fc640 1 -- 172.21.15.16:0/415187710 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2890800981 0 0) 0x7fbab00d1880 con 0x7fbab006d960 2024-09-19T07:43:57.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.980+0000 7fbaaeffd640 1 --2- 172.21.15.16:0/415187710 >> v2:172.21.15.146:3300/0 conn(0x7fbab015e680 0x7fbab00cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.981+0000 7fbaaf7fe640 1 --2- 172.21.15.16:0/415187710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbab015e2a0 0x7fbab006d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00097a0 tx=0x7fbaa002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6802fee999852f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.981+0000 7fbaae7fc640 1 -- 172.21.15.16:0/415187710 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0037070 con 0x7fbab015e2a0 2024-09-19T07:43:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.981+0000 7fbaae7fc640 1 -- 172.21.15.16:0/415187710 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaa0032440 con 0x7fbab015e2a0 2024-09-19T07:43:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.981+0000 7fbaae7fc640 1 -- 172.21.15.16:0/415187710 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0032810 con 0x7fbab015e2a0 2024-09-19T07:43:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.982+0000 7fbab5e47640 1 -- 172.21.15.16:0/415187710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbab015e2a0 msgr2=0x7fbab006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.982+0000 7fbab5e47640 1 --2- 172.21.15.16:0/415187710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbab015e2a0 0x7fbab006d420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00097a0 tx=0x7fbaa002a180 comp rx=0 tx=0).stop 2024-09-19T07:43:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.982+0000 7fbab5e47640 1 -- 172.21.15.16:0/415187710 shutdown_connections 2024-09-19T07:43:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.982+0000 7fbab5e47640 1 --2- 172.21.15.16:0/415187710 >> v2:172.21.15.146:3300/0 conn(0x7fbab015e680 0x7fbab00cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.982+0000 7fbab5e47640 1 --2- 172.21.15.16:0/415187710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbab015e2a0 0x7fbab006d420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.982+0000 7fbab5e47640 1 -- 172.21.15.16:0/415187710 >> 172.21.15.16:0/415187710 conn(0x7fbab009d760 msgr2=0x7fbab008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:57.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.982+0000 7fbab5e47640 1 -- 172.21.15.16:0/415187710 shutdown_connections 2024-09-19T07:43:57.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.982+0000 7fbab5e47640 1 -- 172.21.15.16:0/415187710 wait complete. 2024-09-19T07:43:57.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.983+0000 7fbab5e47640 1 Processor -- start 2024-09-19T07:43:57.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.983+0000 7fbab5e47640 1 -- start start 2024-09-19T07:43:57.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.984+0000 7fbab5e47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab006d960 0x7fbab01f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:57.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.984+0000 7fbab5e47640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbab015e2a0 0x7fbab01f8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:57.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.984+0000 7fbab5e47640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbab015e680 0x7fbab01f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:57.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.984+0000 7fbab5e47640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbab00d4c00 con 0x7fbab006d960 2024-09-19T07:43:57.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.984+0000 7fbab5e47640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbab00d4a80 con 0x7fbab015e2a0 2024-09-19T07:43:57.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.984+0000 7fbab5e47640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbab00d0010 con 0x7fbab015e680 2024-09-19T07:43:57.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.984+0000 7fbaaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab006d960 0x7fbab01f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:57.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.984+0000 7fbaaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab006d960 0x7fbab01f4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34566/0 (socket says 172.21.15.16:34566) 2024-09-19T07:43:57.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.984+0000 7fbaaf7fe640 1 -- 172.21.15.16:0/3436722631 learned_addr learned my addr 172.21.15.16:0/3436722631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:57.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.984+0000 7fbaaeffd640 1 --2- 172.21.15.16:0/3436722631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbab015e2a0 0x7fbab01f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:57.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.984+0000 7fbaaffff640 1 --2- 172.21.15.16:0/3436722631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbab015e680 0x7fbab01f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:57.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.984+0000 7fbaaf7fe640 1 -- 172.21.15.16:0/3436722631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbab015e680 msgr2=0x7fbab01f86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:57.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.984+0000 7fbaaf7fe640 1 --2- 172.21.15.16:0/3436722631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbab015e680 0x7fbab01f86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.984+0000 7fbaaf7fe640 1 -- 172.21.15.16:0/3436722631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbab015e2a0 msgr2=0x7fbab01f8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:57.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.984+0000 7fbaaf7fe640 1 --2- 172.21.15.16:0/3436722631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbab015e2a0 0x7fbab01f8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:57.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.984+0000 7fbaaf7fe640 1 -- 172.21.15.16:0/3436722631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab01f4ac0 con 0x7fbab006d960 2024-09-19T07:43:57.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.984+0000 7fbaaeffd640 1 --2- 172.21.15.16:0/3436722631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbab015e2a0 0x7fbab01f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:57.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.985+0000 7fbaaf7fe640 1 --2- 172.21.15.16:0/3436722631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab006d960 0x7fbab01f4580 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00097a0 tx=0x7fbaa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:57.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.985+0000 7fba97fff640 1 -- 172.21.15.16:0/3436722631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0041070 con 0x7fbab006d960 2024-09-19T07:43:57.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.985+0000 7fba97fff640 1 -- 172.21.15.16:0/3436722631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaa003c870 con 0x7fbab006d960 2024-09-19T07:43:57.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.985+0000 7fba97fff640 1 -- 172.21.15.16:0/3436722631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0037040 con 0x7fbab006d960 2024-09-19T07:43:57.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.985+0000 7fbab5e47640 1 -- 172.21.15.16:0/3436722631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab01f9030 con 0x7fbab006d960 2024-09-19T07:43:57.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.985+0000 7fbab5e47640 1 -- 172.21.15.16:0/3436722631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab00b7590 con 0x7fbab006d960 2024-09-19T07:43:57.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.986+0000 7fbab5e47640 1 -- 172.21.15.16:0/3436722631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab006d420 con 0x7fbab006d960 2024-09-19T07:43:57.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.987+0000 7fba97fff640 1 -- 172.21.15.16:0/3436722631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbaa0044070 con 0x7fbab006d960 2024-09-19T07:43:57.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.987+0000 7fba97fff640 1 --2- 172.21.15.16:0/3436722631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fba9c06c2b0 0x7fba9c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:57.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.987+0000 7fba97fff640 1 -- 172.21.15.16:0/3436722631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fbaa00abdf0 con 0x7fbab006d960 2024-09-19T07:43:57.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.987+0000 7fbaaeffd640 1 --2- 172.21.15.16:0/3436722631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fba9c06c2b0 0x7fba9c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:57.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.988+0000 7fbaaeffd640 1 --2- 172.21.15.16:0/3436722631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fba9c06c2b0 0x7fba9c06e770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbab0161570 tx=0x7fbaa4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:57.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:57.991+0000 7fba97fff640 1 -- 172.21.15.16:0/3436722631 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fbaa007a280 con 0x7fbab006d960 2024-09-19T07:43:58.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.236+0000 7fbab5e47640 1 -- 172.21.15.16:0/3436722631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fbab0160380 con 0x7fbab006d960 2024-09-19T07:43:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.238+0000 7fba97fff640 1 -- 172.21.15.16:0/3436722631 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fbaa007e580 con 0x7fbab006d960 2024-09-19T07:43:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.240+0000 7fbab5e47640 1 -- 172.21.15.16:0/3436722631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fba9c06c2b0 msgr2=0x7fba9c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.240+0000 7fbab5e47640 1 --2- 172.21.15.16:0/3436722631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fba9c06c2b0 0x7fba9c06e770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbab0161570 tx=0x7fbaa4009190 comp rx=0 tx=0).stop 2024-09-19T07:43:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.240+0000 7fbab5e47640 1 -- 172.21.15.16:0/3436722631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab006d960 msgr2=0x7fbab01f4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.240+0000 7fbab5e47640 1 --2- 172.21.15.16:0/3436722631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab006d960 0x7fbab01f4580 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00097a0 tx=0x7fbaa00046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.240+0000 7fbab5e47640 1 -- 172.21.15.16:0/3436722631 shutdown_connections 2024-09-19T07:43:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.240+0000 7fbab5e47640 1 --2- 172.21.15.16:0/3436722631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fba9c06c2b0 0x7fba9c06e770 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.240+0000 7fbab5e47640 1 --2- 172.21.15.16:0/3436722631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbab015e680 0x7fbab01f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.240+0000 7fbab5e47640 1 --2- 172.21.15.16:0/3436722631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbab015e2a0 0x7fbab01f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.240+0000 7fbab5e47640 1 --2- 172.21.15.16:0/3436722631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab006d960 0x7fbab01f4580 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.240+0000 7fbab5e47640 1 -- 172.21.15.16:0/3436722631 >> 172.21.15.16:0/3436722631 conn(0x7fbab009d760 msgr2=0x7fbab015fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.240+0000 7fbab5e47640 1 -- 172.21.15.16:0/3436722631 shutdown_connections 2024-09-19T07:43:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.240+0000 7fbab5e47640 1 -- 172.21.15.16:0/3436722631 wait complete. 2024-09-19T07:43:58.253 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T07:43:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.354+0000 7fbb614a5640 1 Processor -- start 2024-09-19T07:43:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.354+0000 7fbb614a5640 1 -- start start 2024-09-19T07:43:58.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.355+0000 7fbb614a5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb5c15f0d0 0x7fbb5c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:58.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.355+0000 7fbb614a5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c0d0070 0x7fbb5c0d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.355+0000 7fbb614a5640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb5c0cf2b0 con 0x7fbb5c0d1940 2024-09-19T07:43:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.355+0000 7fbb614a5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fbb5c0cf590 con 0x7fbb5c15f0d0 2024-09-19T07:43:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.355+0000 7fbb614a5640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbb5c0cf870 con 0x7fbb5c0d0070 2024-09-19T07:43:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.355+0000 7fbb5affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb5c15f0d0 0x7fbb5c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.355+0000 7fbb5affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb5c15f0d0 0x7fbb5c15f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34574/0 (socket says 172.21.15.16:34574) 2024-09-19T07:43:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.355+0000 7fbb5affd640 1 -- 172.21.15.16:0/736320037 learned_addr learned my addr 172.21.15.16:0/736320037 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.355+0000 7fbb5a7fc640 1 --2- 172.21.15.16:0/736320037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c0d0070 0x7fbb5c0d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.355+0000 7fbb5affd640 1 -- 172.21.15.16:0/736320037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c0d0070 msgr2=0x7fbb5c0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.355+0000 7fbb5affd640 1 --2- 172.21.15.16:0/736320037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c0d0070 0x7fbb5c0d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.355+0000 7fbb5affd640 1 -- 172.21.15.16:0/736320037 >> v1:172.21.15.146:6789/0 conn(0x7fbb5c0d1940 legacy=0x7fbb5c0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.355+0000 7fbb5affd640 1 -- 172.21.15.16:0/736320037 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5c0cfb50 con 0x7fbb5c15f0d0 2024-09-19T07:43:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.355+0000 7fbb5a7fc640 1 --2- 172.21.15.16:0/736320037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c0d0070 0x7fbb5c0d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.356+0000 7fbb5affd640 1 --2- 172.21.15.16:0/736320037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb5c15f0d0 0x7fbb5c15f4b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0b6f40 tx=0x7fbb4c02a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88a5671a23d2408 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.356+0000 7fbb59ffb640 1 -- 172.21.15.16:0/736320037 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c037070 con 0x7fbb5c15f0d0 2024-09-19T07:43:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.356+0000 7fbb59ffb640 1 -- 172.21.15.16:0/736320037 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb4c032440 con 0x7fbb5c15f0d0 2024-09-19T07:43:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.356+0000 7fbb59ffb640 1 -- 172.21.15.16:0/736320037 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c032810 con 0x7fbb5c15f0d0 2024-09-19T07:43:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.357+0000 7fbb614a5640 1 -- 172.21.15.16:0/736320037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb5c15f0d0 msgr2=0x7fbb5c15f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.357+0000 7fbb614a5640 1 --2- 172.21.15.16:0/736320037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb5c15f0d0 0x7fbb5c15f4b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0b6f40 tx=0x7fbb4c02a160 comp rx=0 tx=0).stop 2024-09-19T07:43:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.357+0000 7fbb614a5640 1 -- 172.21.15.16:0/736320037 shutdown_connections 2024-09-19T07:43:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.357+0000 7fbb614a5640 1 --2- 172.21.15.16:0/736320037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c0d0070 0x7fbb5c0d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.357+0000 7fbb614a5640 1 --2- 172.21.15.16:0/736320037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb5c15f0d0 0x7fbb5c15f4b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.357+0000 7fbb614a5640 1 -- 172.21.15.16:0/736320037 >> 172.21.15.16:0/736320037 conn(0x7fbb5c09d760 msgr2=0x7fbb5c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.357+0000 7fbb614a5640 1 -- 172.21.15.16:0/736320037 shutdown_connections 2024-09-19T07:43:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.357+0000 7fbb614a5640 1 -- 172.21.15.16:0/736320037 wait complete. 2024-09-19T07:43:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.358+0000 7fbb614a5640 1 Processor -- start 2024-09-19T07:43:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.358+0000 7fbb614a5640 1 -- start start 2024-09-19T07:43:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.359+0000 7fbb614a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb5c0d0070 0x7fbb5c15b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.359+0000 7fbb614a5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c0d1940 0x7fbb5c0c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.359+0000 7fbb614a5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb5c15f0d0 0x7fbb5c0b9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.359+0000 7fbb614a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c0d49c0 con 0x7fbb5c0d0070 2024-09-19T07:43:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.359+0000 7fbb614a5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c0d4840 con 0x7fbb5c15f0d0 2024-09-19T07:43:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.359+0000 7fbb614a5640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbb5c0d4b40 con 0x7fbb5c0d1940 2024-09-19T07:43:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.359+0000 7fbb5affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb5c0d0070 0x7fbb5c15b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.359+0000 7fbb5affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb5c0d0070 0x7fbb5c15b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34580/0 (socket says 172.21.15.16:34580) 2024-09-19T07:43:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.359+0000 7fbb5affd640 1 -- 172.21.15.16:0/4237360199 learned_addr learned my addr 172.21.15.16:0/4237360199 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.359+0000 7fbb5b7fe640 1 --2- 172.21.15.16:0/4237360199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb5c15f0d0 0x7fbb5c0b9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.359+0000 7fbb5a7fc640 1 --2- 172.21.15.16:0/4237360199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c0d1940 0x7fbb5c0c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.359+0000 7fbb5affd640 1 -- 172.21.15.16:0/4237360199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c0d1940 msgr2=0x7fbb5c0c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.359+0000 7fbb5affd640 1 --2- 172.21.15.16:0/4237360199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c0d1940 0x7fbb5c0c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.359+0000 7fbb5affd640 1 -- 172.21.15.16:0/4237360199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb5c15f0d0 msgr2=0x7fbb5c0b9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.359+0000 7fbb5affd640 1 --2- 172.21.15.16:0/4237360199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb5c15f0d0 0x7fbb5c0b9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.359+0000 7fbb5affd640 1 -- 172.21.15.16:0/4237360199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5c1e8f00 con 0x7fbb5c0d0070 2024-09-19T07:43:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.359+0000 7fbb5a7fc640 1 --2- 172.21.15.16:0/4237360199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c0d1940 0x7fbb5c0c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.359+0000 7fbb5affd640 1 --2- 172.21.15.16:0/4237360199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb5c0d0070 0x7fbb5c15b2f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c000c00 tx=0x7fbb4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.360+0000 7fbb437fe640 1 -- 172.21.15.16:0/4237360199 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c041020 con 0x7fbb5c0d0070 2024-09-19T07:43:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.360+0000 7fbb437fe640 1 -- 172.21.15.16:0/4237360199 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb4c030970 con 0x7fbb5c0d0070 2024-09-19T07:43:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.360+0000 7fbb614a5640 1 -- 172.21.15.16:0/4237360199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5c1e91e0 con 0x7fbb5c0d0070 2024-09-19T07:43:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.360+0000 7fbb437fe640 1 -- 172.21.15.16:0/4237360199 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c037040 con 0x7fbb5c0d0070 2024-09-19T07:43:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.360+0000 7fbb614a5640 1 -- 172.21.15.16:0/4237360199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb5c1e97a0 con 0x7fbb5c0d0070 2024-09-19T07:43:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.361+0000 7fbb614a5640 1 -- 172.21.15.16:0/4237360199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb5c0d1400 con 0x7fbb5c0d0070 2024-09-19T07:43:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.361+0000 7fbb437fe640 1 -- 172.21.15.16:0/4237360199 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbb4c044020 con 0x7fbb5c0d0070 2024-09-19T07:43:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.362+0000 7fbb437fe640 1 --2- 172.21.15.16:0/4237360199 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbb4806c2b0 0x7fbb4806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.362+0000 7fbb437fe640 1 -- 172.21.15.16:0/4237360199 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fbb4c0abc20 con 0x7fbb5c0d0070 2024-09-19T07:43:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.362+0000 7fbb5a7fc640 1 --2- 172.21.15.16:0/4237360199 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbb4806c2b0 0x7fbb4806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.362+0000 7fbb5a7fc640 1 --2- 172.21.15.16:0/4237360199 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbb4806c2b0 0x7fbb4806e770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbb50006fd0 tx=0x7fbb50008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.366+0000 7fbb437fe640 1 -- 172.21.15.16:0/4237360199 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fbb4c04b030 con 0x7fbb5c0d0070 2024-09-19T07:43:58.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.544+0000 7fbb614a5640 1 -- 172.21.15.16:0/4237360199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fbb5c06cf30 con 0x7fbb5c0d0070 2024-09-19T07:43:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.544+0000 7fbb437fe640 1 -- 172.21.15.16:0/4237360199 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) ==== 84+0+65 (secure 0 0 0) 0x7fbb4c07a5e0 con 0x7fbb5c0d0070 2024-09-19T07:43:58.545 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:43:58.545 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-19T07:43:58.545 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:43:58.545 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-19T07:43:58.545 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-19T07:43:58.545 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:43:58.545 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-19T07:43:58.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.546+0000 7fbb614a5640 1 -- 172.21.15.16:0/4237360199 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbb4806c2b0 msgr2=0x7fbb4806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:58.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.546+0000 7fbb614a5640 1 --2- 172.21.15.16:0/4237360199 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbb4806c2b0 0x7fbb4806e770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbb50006fd0 tx=0x7fbb50008040 comp rx=0 tx=0).stop 2024-09-19T07:43:58.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.546+0000 7fbb614a5640 1 -- 172.21.15.16:0/4237360199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb5c0d0070 msgr2=0x7fbb5c15b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:58.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.546+0000 7fbb614a5640 1 --2- 172.21.15.16:0/4237360199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb5c0d0070 0x7fbb5c15b2f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c000c00 tx=0x7fbb4c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.546+0000 7fbb614a5640 1 -- 172.21.15.16:0/4237360199 shutdown_connections 2024-09-19T07:43:58.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.546+0000 7fbb614a5640 1 --2- 172.21.15.16:0/4237360199 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbb4806c2b0 0x7fbb4806e770 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.546+0000 7fbb614a5640 1 --2- 172.21.15.16:0/4237360199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb5c15f0d0 0x7fbb5c0b9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.546+0000 7fbb614a5640 1 --2- 172.21.15.16:0/4237360199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c0d1940 0x7fbb5c0c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.546+0000 7fbb614a5640 1 --2- 172.21.15.16:0/4237360199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb5c0d0070 0x7fbb5c15b2f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.546+0000 7fbb614a5640 1 -- 172.21.15.16:0/4237360199 >> 172.21.15.16:0/4237360199 conn(0x7fbb5c09d760 msgr2=0x7fbb5c15ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:58.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.547+0000 7fbb614a5640 1 -- 172.21.15.16:0/4237360199 shutdown_connections 2024-09-19T07:43:58.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.547+0000 7fbb614a5640 1 -- 172.21.15.16:0/4237360199 wait complete. 2024-09-19T07:43:58.558 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T07:43:58.558 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T07:43:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.664+0000 7f7553a15640 1 Processor -- start 2024-09-19T07:43:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.665+0000 7f7553a15640 1 -- start start 2024-09-19T07:43:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.665+0000 7f7553a15640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f754c15f0d0 0x7f754c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.665+0000 7f7553a15640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f754c0d0070 0x7f754c0d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.665+0000 7f7553a15640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f754c0cf2b0 con 0x7f754c0d1940 2024-09-19T07:43:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.665+0000 7f7553a15640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f754c0cf590 con 0x7f754c0d0070 2024-09-19T07:43:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.665+0000 7f7553a15640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f754c0cf870 con 0x7f754c15f0d0 2024-09-19T07:43:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.665+0000 7f7550fb0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f754c0d0070 0x7f754c0d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.666+0000 7f7550fb0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f754c0d0070 0x7f754c0d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34596/0 (socket says 172.21.15.16:34596) 2024-09-19T07:43:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.666+0000 7f7550fb0640 1 -- 172.21.15.16:0/3866843716 learned_addr learned my addr 172.21.15.16:0/3866843716 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.666+0000 7f75517b1640 1 --2- 172.21.15.16:0/3866843716 >> v2:172.21.15.146:3300/0 conn(0x7f754c15f0d0 0x7f754c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.666+0000 7f7550fb0640 1 -- 172.21.15.16:0/3866843716 >> v2:172.21.15.146:3300/0 conn(0x7f754c15f0d0 msgr2=0x7f754c15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.666+0000 7f7550fb0640 1 --2- 172.21.15.16:0/3866843716 >> v2:172.21.15.146:3300/0 conn(0x7f754c15f0d0 0x7f754c15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.666+0000 7f7550fb0640 1 -- 172.21.15.16:0/3866843716 >> v1:172.21.15.146:6789/0 conn(0x7f754c0d1940 legacy=0x7f754c0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.666+0000 7f7550fb0640 1 -- 172.21.15.16:0/3866843716 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c0cfb50 con 0x7f754c0d0070 2024-09-19T07:43:58.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.667+0000 7f7550fb0640 1 --2- 172.21.15.16:0/3866843716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c0d0070 0x7f754c0d1400 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f754c162fc0 tx=0x7f754002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7831fee9d3e7312d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:58.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.667+0000 7f754bfff640 1 -- 172.21.15.16:0/3866843716 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540037070 con 0x7f754c0d0070 2024-09-19T07:43:58.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.667+0000 7f754bfff640 1 -- 172.21.15.16:0/3866843716 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7540032440 con 0x7f754c0d0070 2024-09-19T07:43:58.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.667+0000 7f754bfff640 1 -- 172.21.15.16:0/3866843716 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540032810 con 0x7f754c0d0070 2024-09-19T07:43:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.667+0000 7f7553a15640 1 -- 172.21.15.16:0/3866843716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c0d0070 msgr2=0x7f754c0d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.667+0000 7f7553a15640 1 --2- 172.21.15.16:0/3866843716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c0d0070 0x7f754c0d1400 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f754c162fc0 tx=0x7f754002a180 comp rx=0 tx=0).stop 2024-09-19T07:43:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.667+0000 7f7553a15640 1 -- 172.21.15.16:0/3866843716 shutdown_connections 2024-09-19T07:43:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.667+0000 7f7553a15640 1 --2- 172.21.15.16:0/3866843716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c0d0070 0x7f754c0d1400 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.667+0000 7f7553a15640 1 --2- 172.21.15.16:0/3866843716 >> v2:172.21.15.146:3300/0 conn(0x7f754c15f0d0 0x7f754c15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.667+0000 7f7553a15640 1 -- 172.21.15.16:0/3866843716 >> 172.21.15.16:0/3866843716 conn(0x7f754c09d760 msgr2=0x7f754c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.667+0000 7f7553a15640 1 -- 172.21.15.16:0/3866843716 shutdown_connections 2024-09-19T07:43:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.667+0000 7f7553a15640 1 -- 172.21.15.16:0/3866843716 wait complete. 2024-09-19T07:43:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.669+0000 7f7553a15640 1 Processor -- start 2024-09-19T07:43:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.669+0000 7f7553a15640 1 -- start start 2024-09-19T07:43:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.669+0000 7f7553a15640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f754c0d0070 0x7f754c14fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.669+0000 7f7553a15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c0d1940 0x7f754c0cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.669+0000 7f7553a15640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f754c15f0d0 0x7f754c15c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.669+0000 7f7553a15640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f754c0d49c0 con 0x7f754c0d1940 2024-09-19T07:43:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.669+0000 7f7553a15640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f754c0d4840 con 0x7f754c15f0d0 2024-09-19T07:43:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.669+0000 7f7553a15640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f754c0d4b40 con 0x7f754c0d0070 2024-09-19T07:43:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.669+0000 7f7550fb0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c0d1940 0x7f754c0cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.669+0000 7f7550fb0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c0d1940 0x7f754c0cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34600/0 (socket says 172.21.15.16:34600) 2024-09-19T07:43:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.669+0000 7f7550fb0640 1 -- 172.21.15.16:0/1276380565 learned_addr learned my addr 172.21.15.16:0/1276380565 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.670+0000 7f75517b1640 1 --2- 172.21.15.16:0/1276380565 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f754c0d0070 0x7f754c14fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.670+0000 7f7551fb2640 1 --2- 172.21.15.16:0/1276380565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f754c15f0d0 0x7f754c15c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.670+0000 7f7550fb0640 1 -- 172.21.15.16:0/1276380565 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f754c0d0070 msgr2=0x7f754c14fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.670+0000 7f7550fb0640 1 --2- 172.21.15.16:0/1276380565 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f754c0d0070 0x7f754c14fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.670+0000 7f7550fb0640 1 -- 172.21.15.16:0/1276380565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f754c15f0d0 msgr2=0x7f754c15c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.670+0000 7f7550fb0640 1 --2- 172.21.15.16:0/1276380565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f754c15f0d0 0x7f754c15c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.670+0000 7f7550fb0640 1 -- 172.21.15.16:0/1276380565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c0cbdb0 con 0x7f754c0d1940 2024-09-19T07:43:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.670+0000 7f7550fb0640 1 --2- 172.21.15.16:0/1276380565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c0d1940 0x7f754c0cb4b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7540000c00 tx=0x7f75400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.670+0000 7f7549ffb640 1 -- 172.21.15.16:0/1276380565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540041070 con 0x7f754c0d1940 2024-09-19T07:43:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.670+0000 7f7549ffb640 1 -- 172.21.15.16:0/1276380565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f754003c870 con 0x7f754c0d1940 2024-09-19T07:43:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.670+0000 7f7553a15640 1 -- 172.21.15.16:0/1276380565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754c0cc090 con 0x7f754c0d1940 2024-09-19T07:43:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.670+0000 7f7549ffb640 1 -- 172.21.15.16:0/1276380565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540037040 con 0x7f754c0d1940 2024-09-19T07:43:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.670+0000 7f7553a15640 1 -- 172.21.15.16:0/1276380565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754c0cc650 con 0x7f754c0d1940 2024-09-19T07:43:58.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.671+0000 7f7553a15640 1 -- 172.21.15.16:0/1276380565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f754c0d1400 con 0x7f754c0d1940 2024-09-19T07:43:58.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.672+0000 7f7549ffb640 1 -- 172.21.15.16:0/1276380565 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7540044070 con 0x7f754c0d1940 2024-09-19T07:43:58.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.672+0000 7f7549ffb640 1 --2- 172.21.15.16:0/1276380565 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f753806c2b0 0x7f753806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:58.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.672+0000 7f7549ffb640 1 -- 172.21.15.16:0/1276380565 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f75400abd10 con 0x7f754c0d1940 2024-09-19T07:43:58.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.673+0000 7f75517b1640 1 --2- 172.21.15.16:0/1276380565 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f753806c2b0 0x7f753806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:58.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.673+0000 7f75517b1640 1 --2- 172.21.15.16:0/1276380565 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f753806c2b0 0x7f753806e770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f754c15c0e0 tx=0x7f7544008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:58.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.677+0000 7f7549ffb640 1 -- 172.21.15.16:0/1276380565 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f75400b1020 con 0x7f754c0d1940 2024-09-19T07:43:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:58.851+0000 7f7553a15640 1 -- 172.21.15.16:0/1276380565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f754c06cf30 con 0x7f754c0d1940 2024-09-19T07:43:59.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.602+0000 7f7549ffb640 1 -- 172.21.15.16:0/1276380565 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) ==== 138+0+0 (secure 0 0 0) 0x7f754007a070 con 0x7f754c0d1940 2024-09-19T07:43:59.603 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-19T07:43:59.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.605+0000 7f7553a15640 1 -- 172.21.15.16:0/1276380565 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f753806c2b0 msgr2=0x7f753806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:59.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.605+0000 7f7553a15640 1 --2- 172.21.15.16:0/1276380565 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f753806c2b0 0x7f753806e770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f754c15c0e0 tx=0x7f7544008040 comp rx=0 tx=0).stop 2024-09-19T07:43:59.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.605+0000 7f7553a15640 1 -- 172.21.15.16:0/1276380565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c0d1940 msgr2=0x7f754c0cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:59.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.605+0000 7f7553a15640 1 --2- 172.21.15.16:0/1276380565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c0d1940 0x7f754c0cb4b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7540000c00 tx=0x7f75400046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.606+0000 7f7553a15640 1 -- 172.21.15.16:0/1276380565 shutdown_connections 2024-09-19T07:43:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.606+0000 7f7553a15640 1 --2- 172.21.15.16:0/1276380565 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f753806c2b0 0x7f753806e770 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.606+0000 7f7553a15640 1 --2- 172.21.15.16:0/1276380565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f754c15f0d0 0x7f754c15c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.606+0000 7f7553a15640 1 --2- 172.21.15.16:0/1276380565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c0d1940 0x7f754c0cb4b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.606+0000 7f7553a15640 1 --2- 172.21.15.16:0/1276380565 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f754c0d0070 0x7f754c14fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.606+0000 7f7553a15640 1 -- 172.21.15.16:0/1276380565 >> 172.21.15.16:0/1276380565 conn(0x7f754c09d760 msgr2=0x7f754c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.606+0000 7f7553a15640 1 -- 172.21.15.16:0/1276380565 shutdown_connections 2024-09-19T07:43:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.606+0000 7f7553a15640 1 -- 172.21.15.16:0/1276380565 wait complete. 2024-09-19T07:43:59.623 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T07:43:59.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.739+0000 7f71475d6640 1 Processor -- start 2024-09-19T07:43:59.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.739+0000 7f71475d6640 1 -- start start 2024-09-19T07:43:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.740+0000 7f71475d6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71400d4af0 0x7f7140162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.740+0000 7f71475d6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71400b4b70 0x7f71400b5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.740+0000 7f71475d6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f714015f250 con 0x7f714015e0a0 2024-09-19T07:43:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.740+0000 7f71475d6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f71400b5560 con 0x7f71400b4b70 2024-09-19T07:43:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.740+0000 7f71475d6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f714015f530 con 0x7f71400d4af0 2024-09-19T07:43:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.740+0000 7f7145b73640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71400b4b70 0x7f71400b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.740+0000 7f7145b73640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71400b4b70 0x7f71400b5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34604/0 (socket says 172.21.15.16:34604) 2024-09-19T07:43:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.740+0000 7f7145b73640 1 -- 172.21.15.16:0/3762500667 learned_addr learned my addr 172.21.15.16:0/3762500667 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.740+0000 7f7144b71640 1 --2- 172.21.15.16:0/3762500667 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71400d4af0 0x7f7140162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.740+0000 7f7145b73640 1 -- 172.21.15.16:0/3762500667 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71400d4af0 msgr2=0x7f7140162a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.740+0000 7f7145b73640 1 --2- 172.21.15.16:0/3762500667 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71400d4af0 0x7f7140162a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.740+0000 7f7145b73640 1 -- 172.21.15.16:0/3762500667 >> v1:172.21.15.146:6789/0 conn(0x7f714015e0a0 legacy=0x7f71400d43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.740+0000 7f7145b73640 1 -- 172.21.15.16:0/3762500667 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714015f810 con 0x7f71400b4b70 2024-09-19T07:43:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.741+0000 7f7145b73640 1 --2- 172.21.15.16:0/3762500667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71400b4b70 0x7f71400b5020 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7138009e50 tx=0x7f713802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3cbfaa59a3c8e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.741+0000 7f7137fff640 1 -- 172.21.15.16:0/3762500667 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138036070 con 0x7f71400b4b70 2024-09-19T07:43:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.741+0000 7f7137fff640 1 -- 172.21.15.16:0/3762500667 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7138031600 con 0x7f71400b4b70 2024-09-19T07:43:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.741+0000 7f7137fff640 1 -- 172.21.15.16:0/3762500667 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71380319d0 con 0x7f71400b4b70 2024-09-19T07:43:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.742+0000 7f71475d6640 1 -- 172.21.15.16:0/3762500667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71400b4b70 msgr2=0x7f71400b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.742+0000 7f71475d6640 1 --2- 172.21.15.16:0/3762500667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71400b4b70 0x7f71400b5020 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7138009e50 tx=0x7f713802a710 comp rx=0 tx=0).stop 2024-09-19T07:43:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.742+0000 7f71475d6640 1 -- 172.21.15.16:0/3762500667 shutdown_connections 2024-09-19T07:43:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.742+0000 7f71475d6640 1 --2- 172.21.15.16:0/3762500667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71400b4b70 0x7f71400b5020 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.742+0000 7f71475d6640 1 --2- 172.21.15.16:0/3762500667 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71400d4af0 0x7f7140162a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.742+0000 7f71475d6640 1 -- 172.21.15.16:0/3762500667 >> 172.21.15.16:0/3762500667 conn(0x7f714009d770 msgr2=0x7f714008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.742+0000 7f71475d6640 1 -- 172.21.15.16:0/3762500667 shutdown_connections 2024-09-19T07:43:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.742+0000 7f71475d6640 1 -- 172.21.15.16:0/3762500667 wait complete. 2024-09-19T07:43:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.743+0000 7f71475d6640 1 Processor -- start 2024-09-19T07:43:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.743+0000 7f71475d6640 1 -- start start 2024-09-19T07:43:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.744+0000 7f71475d6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71400b4b70 0x7f71400b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.744+0000 7f71475d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71400d4af0 0x7f714014dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.744+0000 7f71475d6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f714015e0a0 0x7f71400cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.744+0000 7f71475d6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f714015e600 con 0x7f71400d4af0 2024-09-19T07:43:59.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.744+0000 7f71475d6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f714015e480 con 0x7f714015e0a0 2024-09-19T07:43:59.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.744+0000 7f71475d6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f714015e780 con 0x7f71400b4b70 2024-09-19T07:43:59.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.744+0000 7f7144b71640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71400d4af0 0x7f714014dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.744+0000 7f7144b71640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71400d4af0 0x7f714014dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34606/0 (socket says 172.21.15.16:34606) 2024-09-19T07:43:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.744+0000 7f7144b71640 1 -- 172.21.15.16:0/936096454 learned_addr learned my addr 172.21.15.16:0/936096454 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:43:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.744+0000 7f7145372640 1 --2- 172.21.15.16:0/936096454 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71400b4b70 0x7f71400b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.744+0000 7f7145b73640 1 --2- 172.21.15.16:0/936096454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f714015e0a0 0x7f71400cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.744+0000 7f7144b71640 1 -- 172.21.15.16:0/936096454 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71400b4b70 msgr2=0x7f71400b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.744+0000 7f7144b71640 1 --2- 172.21.15.16:0/936096454 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71400b4b70 0x7f71400b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.744+0000 7f7144b71640 1 -- 172.21.15.16:0/936096454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f714015e0a0 msgr2=0x7f71400cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.744+0000 7f7144b71640 1 --2- 172.21.15.16:0/936096454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f714015e0a0 0x7f71400cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.744+0000 7f7144b71640 1 -- 172.21.15.16:0/936096454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71400b7f50 con 0x7f71400d4af0 2024-09-19T07:43:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.745+0000 7f7144b71640 1 --2- 172.21.15.16:0/936096454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71400d4af0 0x7f714014dc10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f713000d800 tx=0x7f713000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.745+0000 7f7135ffb640 1 -- 172.21.15.16:0/936096454 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130005990 con 0x7f71400d4af0 2024-09-19T07:43:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.745+0000 7f7135ffb640 1 -- 172.21.15.16:0/936096454 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7130018070 con 0x7f71400d4af0 2024-09-19T07:43:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.745+0000 7f71475d6640 1 -- 172.21.15.16:0/936096454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71400b1a30 con 0x7f71400d4af0 2024-09-19T07:43:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.745+0000 7f7135ffb640 1 -- 172.21.15.16:0/936096454 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130005d50 con 0x7f71400d4af0 2024-09-19T07:43:59.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.745+0000 7f71475d6640 1 -- 172.21.15.16:0/936096454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71400b1ff0 con 0x7f71400d4af0 2024-09-19T07:43:59.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.746+0000 7f71475d6640 1 -- 172.21.15.16:0/936096454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71400b5020 con 0x7f71400d4af0 2024-09-19T07:43:59.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.747+0000 7f7135ffb640 1 -- 172.21.15.16:0/936096454 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f713000b6c0 con 0x7f71400d4af0 2024-09-19T07:43:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.747+0000 7f7135ffb640 1 --2- 172.21.15.16:0/936096454 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f712406c2b0 0x7f712406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.747+0000 7f7135ffb640 1 -- 172.21.15.16:0/936096454 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5619+0+0 (secure 0 0 0) 0x7f713008b400 con 0x7f71400d4af0 2024-09-19T07:43:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.748+0000 7f7145372640 1 --2- 172.21.15.16:0/936096454 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f712406c2b0 0x7f712406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.748+0000 7f7145372640 1 --2- 172.21.15.16:0/936096454 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f712406c2b0 0x7f712406e770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f713c0016e0 tx=0x7f713c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:59.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.752+0000 7f7135ffb640 1 -- 172.21.15.16:0/936096454 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f71300596c0 con 0x7f71400d4af0 2024-09-19T07:43:59.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:43:59.929+0000 7f71475d6640 1 -- 172.21.15.16:0/936096454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f71400ba3d0 con 0x7f71400d4af0 2024-09-19T07:44:00.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.607+0000 7f7135ffb640 1 -- 172.21.15.16:0/936096454 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) ==== 170+0+0 (secure 0 0 0) 0x7f713005d9c0 con 0x7f71400d4af0 2024-09-19T07:44:00.608 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-19T07:44:00.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.610+0000 7f71475d6640 1 -- 172.21.15.16:0/936096454 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f712406c2b0 msgr2=0x7f712406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:00.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.610+0000 7f71475d6640 1 --2- 172.21.15.16:0/936096454 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f712406c2b0 0x7f712406e770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f713c0016e0 tx=0x7f713c009040 comp rx=0 tx=0).stop 2024-09-19T07:44:00.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.610+0000 7f71475d6640 1 -- 172.21.15.16:0/936096454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71400d4af0 msgr2=0x7f714014dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:00.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.610+0000 7f71475d6640 1 --2- 172.21.15.16:0/936096454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71400d4af0 0x7f714014dc10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f713000d800 tx=0x7f713000dcc0 comp rx=0 tx=0).stop 2024-09-19T07:44:00.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.611+0000 7f71475d6640 1 -- 172.21.15.16:0/936096454 shutdown_connections 2024-09-19T07:44:00.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.611+0000 7f71475d6640 1 --2- 172.21.15.16:0/936096454 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f712406c2b0 0x7f712406e770 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:00.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.611+0000 7f71475d6640 1 --2- 172.21.15.16:0/936096454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f714015e0a0 0x7f71400cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:00.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.611+0000 7f71475d6640 1 --2- 172.21.15.16:0/936096454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71400d4af0 0x7f714014dc10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:00.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.611+0000 7f71475d6640 1 --2- 172.21.15.16:0/936096454 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71400b4b70 0x7f71400b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:00.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.611+0000 7f71475d6640 1 -- 172.21.15.16:0/936096454 >> 172.21.15.16:0/936096454 conn(0x7f714009d770 msgr2=0x7f714008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:00.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.611+0000 7f71475d6640 1 -- 172.21.15.16:0/936096454 shutdown_connections 2024-09-19T07:44:00.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.611+0000 7f71475d6640 1 -- 172.21.15.16:0/936096454 wait complete. 2024-09-19T07:44:00.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T07:44:00.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.746+0000 7fc17a296640 1 Processor -- start 2024-09-19T07:44:00.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.746+0000 7fc17a296640 1 -- start start 2024-09-19T07:44:00.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.747+0000 7fc17a296640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc17415e680 0x7fc1740cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:00.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.747+0000 7fc17a296640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1740d1880 con 0x7fc17415e2a0 2024-09-19T07:44:00.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.747+0000 7fc17a296640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1740d1b60 con 0x7fc1740d0010 2024-09-19T07:44:00.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.747+0000 7fc17a296640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc1740cf2d0 con 0x7fc17415e680 2024-09-19T07:44:00.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.747+0000 7fc173fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc17415e2a0 0x7fc17406d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56178/0 (socket says 172.21.15.16:56178) 2024-09-19T07:44:00.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.747+0000 7fc173fff640 1 -- 172.21.15.16:0/491024177 learned_addr learned my addr 172.21.15.16:0/491024177 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:44:00.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.747+0000 7fc172ffd640 1 -- 172.21.15.16:0/491024177 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1659953734 0 0) 0x7fc1740d1880 con 0x7fc17415e2a0 2024-09-19T07:44:00.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.747+0000 7fc1737fe640 1 --2- 172.21.15.16:0/491024177 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc17415e680 0x7fc1740cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:00.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.747+0000 7fc172ffd640 1 -- 172.21.15.16:0/491024177 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc15c003520 con 0x7fc17415e2a0 2024-09-19T07:44:00.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc172ffd640 1 -- 172.21.15.16:0/491024177 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2452361358 0 0) 0x7fc15c003520 con 0x7fc17415e2a0 2024-09-19T07:44:00.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc172ffd640 1 -- 172.21.15.16:0/491024177 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc17415e680 msgr2=0x7fc1740cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:00.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc172ffd640 1 --2- 172.21.15.16:0/491024177 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc17415e680 0x7fc1740cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:00.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc172ffd640 1 -- 172.21.15.16:0/491024177 >> v1:172.21.15.146:6789/0 conn(0x7fc1740d0010 legacy=0x7fc1740d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:00.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc172ffd640 1 -- 172.21.15.16:0/491024177 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1740cf5b0 con 0x7fc17415e2a0 2024-09-19T07:44:00.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc172ffd640 1 -- 172.21.15.16:0/491024177 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fc164001670 con 0x7fc17415e2a0 2024-09-19T07:44:00.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc172ffd640 1 -- 172.21.15.16:0/491024177 >> v1:172.21.15.16:6789/0 conn(0x7fc17415e2a0 legacy=0x7fc17406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:00.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc172ffd640 1 --2- 172.21.15.16:0/491024177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c0039e0 0x7fc15c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:00.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc172ffd640 1 --2- 172.21.15.16:0/491024177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc15c004390 0x7fc15c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:00.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc172ffd640 1 --2- 172.21.15.16:0/491024177 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc17415e680 0x7fc15c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:00.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc172ffd640 1 -- 172.21.15.16:0/491024177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc1740cf2d0 con 0x7fc15c0039e0 2024-09-19T07:44:00.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc172ffd640 1 -- 172.21.15.16:0/491024177 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c00baa0 con 0x7fc15c004390 2024-09-19T07:44:00.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc172ffd640 1 -- 172.21.15.16:0/491024177 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc15c00bca0 con 0x7fc17415e680 2024-09-19T07:44:00.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc173fff640 1 --2- 172.21.15.16:0/491024177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c0039e0 0x7fc15c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:00.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc1737fe640 1 --2- 172.21.15.16:0/491024177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc15c004390 0x7fc15c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:00.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc178833640 1 --2- 172.21.15.16:0/491024177 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc17415e680 0x7fc15c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:00.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc173fff640 1 -- 172.21.15.16:0/491024177 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc17415e680 msgr2=0x7fc15c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:00.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc173fff640 1 --2- 172.21.15.16:0/491024177 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc17415e680 0x7fc15c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:00.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc173fff640 1 -- 172.21.15.16:0/491024177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc15c004390 msgr2=0x7fc15c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:00.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc173fff640 1 --2- 172.21.15.16:0/491024177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc15c004390 0x7fc15c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:00.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.748+0000 7fc173fff640 1 -- 172.21.15.16:0/491024177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15c003520 con 0x7fc15c0039e0 2024-09-19T07:44:00.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.749+0000 7fc173fff640 1 --2- 172.21.15.16:0/491024177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c0039e0 0x7fc15c003dc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc16400c290 tx=0x7fc15c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec47367cbd8eb75 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:00.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.749+0000 7fc172ffd640 1 -- 172.21.15.16:0/491024177 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164007d50 con 0x7fc15c0039e0 2024-09-19T07:44:00.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.749+0000 7fc172ffd640 1 -- 172.21.15.16:0/491024177 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc164005af0 con 0x7fc15c0039e0 2024-09-19T07:44:00.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.749+0000 7fc172ffd640 1 -- 172.21.15.16:0/491024177 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164007070 con 0x7fc15c0039e0 2024-09-19T07:44:00.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.750+0000 7fc17a296640 1 -- 172.21.15.16:0/491024177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c0039e0 msgr2=0x7fc15c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:00.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.750+0000 7fc17a296640 1 --2- 172.21.15.16:0/491024177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c0039e0 0x7fc15c003dc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc16400c290 tx=0x7fc15c005830 comp rx=0 tx=0).stop 2024-09-19T07:44:00.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.750+0000 7fc178833640 1 -- 172.21.15.16:0/491024177 reap_dead start 2024-09-19T07:44:00.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.750+0000 7fc17a296640 1 -- 172.21.15.16:0/491024177 shutdown_connections 2024-09-19T07:44:00.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.750+0000 7fc17a296640 1 -- 172.21.15.16:0/491024177 >> 172.21.15.16:0/491024177 conn(0x7fc17409d760 msgr2=0x7fc17408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:00.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.750+0000 7fc17a296640 1 -- 172.21.15.16:0/491024177 shutdown_connections 2024-09-19T07:44:00.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.750+0000 7fc17a296640 1 -- 172.21.15.16:0/491024177 wait complete. 2024-09-19T07:44:00.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.751+0000 7fc17a296640 1 Processor -- start 2024-09-19T07:44:00.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.751+0000 7fc17a296640 1 -- start start 2024-09-19T07:44:00.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.752+0000 7fc17a296640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c0039e0 0x7fc1740b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:00.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.752+0000 7fc17a296640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1741f48a0 0x7fc1740d0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:00.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.752+0000 7fc17a296640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1740d0a90 0x7fc1740b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:00.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.752+0000 7fc17a296640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc17406dcb0 con 0x7fc15c0039e0 2024-09-19T07:44:00.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.752+0000 7fc17a296640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc17406db30 con 0x7fc1741f48a0 2024-09-19T07:44:00.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.752+0000 7fc17a296640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc1740d4a80 con 0x7fc1740d0a90 2024-09-19T07:44:00.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.752+0000 7fc173fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c0039e0 0x7fc1740b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:00.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.752+0000 7fc173fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c0039e0 0x7fc1740b5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34626/0 (socket says 172.21.15.16:34626) 2024-09-19T07:44:00.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.752+0000 7fc173fff640 1 -- 172.21.15.16:0/383108439 learned_addr learned my addr 172.21.15.16:0/383108439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:00.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.752+0000 7fc1737fe640 1 --2- 172.21.15.16:0/383108439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1741f48a0 0x7fc1740d0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:00.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.752+0000 7fc178833640 1 --2- 172.21.15.16:0/383108439 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1740d0a90 0x7fc1740b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:00.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.752+0000 7fc173fff640 1 -- 172.21.15.16:0/383108439 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1740d0a90 msgr2=0x7fc1740b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:00.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.752+0000 7fc173fff640 1 --2- 172.21.15.16:0/383108439 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1740d0a90 0x7fc1740b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:00.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.752+0000 7fc173fff640 1 -- 172.21.15.16:0/383108439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1741f48a0 msgr2=0x7fc1740d0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:00.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.752+0000 7fc173fff640 1 --2- 172.21.15.16:0/383108439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1741f48a0 0x7fc1740d0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:00.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.752+0000 7fc173fff640 1 -- 172.21.15.16:0/383108439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1741f4d20 con 0x7fc15c0039e0 2024-09-19T07:44:00.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.752+0000 7fc173fff640 1 --2- 172.21.15.16:0/383108439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c0039e0 0x7fc1740b5850 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc16402ae50 tx=0x7fc164001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:00.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.753+0000 7fc170ff9640 1 -- 172.21.15.16:0/383108439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164042070 con 0x7fc15c0039e0 2024-09-19T07:44:00.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.753+0000 7fc170ff9640 1 -- 172.21.15.16:0/383108439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc164007280 con 0x7fc15c0039e0 2024-09-19T07:44:00.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.753+0000 7fc17a296640 1 -- 172.21.15.16:0/383108439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1741f7740 con 0x7fc15c0039e0 2024-09-19T07:44:00.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.753+0000 7fc170ff9640 1 -- 172.21.15.16:0/383108439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164007d50 con 0x7fc15c0039e0 2024-09-19T07:44:00.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.753+0000 7fc17a296640 1 -- 172.21.15.16:0/383108439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1741f7d00 con 0x7fc15c0039e0 2024-09-19T07:44:00.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.754+0000 7fc17a296640 1 -- 172.21.15.16:0/383108439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1740cfad0 con 0x7fc15c0039e0 2024-09-19T07:44:00.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.754+0000 7fc170ff9640 1 -- 172.21.15.16:0/383108439 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc164045070 con 0x7fc15c0039e0 2024-09-19T07:44:00.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.755+0000 7fc170ff9640 1 --2- 172.21.15.16:0/383108439 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc16006c2e0 0x7fc16006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:00.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.755+0000 7fc170ff9640 1 -- 172.21.15.16:0/383108439 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6001+0+0 (secure 0 0 0) 0x7fc1640ac3a0 con 0x7fc15c0039e0 2024-09-19T07:44:00.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.755+0000 7fc1737fe640 1 --2- 172.21.15.16:0/383108439 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc16006c2e0 0x7fc16006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:00.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.756+0000 7fc1737fe640 1 --2- 172.21.15.16:0/383108439 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc16006c2e0 0x7fc16006e7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc1740b8df0 tx=0x7fc168009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:00.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.760+0000 7fc170ff9640 1 -- 172.21.15.16:0/383108439 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc164005ac0 con 0x7fc15c0039e0 2024-09-19T07:44:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:00.969+0000 7fc17a296640 1 -- 172.21.15.16:0/383108439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc17406cf90 con 0x7fc15c0039e0 2024-09-19T07:44:01.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.609+0000 7fc170ff9640 1 -- 172.21.15.16:0/383108439 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) ==== 172+0+0 (secure 0 0 0) 0x7fc16407a2f0 con 0x7fc15c0039e0 2024-09-19T07:44:01.609 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 5 and data pool 6 2024-09-19T07:44:01.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.614+0000 7fc17a296640 1 -- 172.21.15.16:0/383108439 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc16006c2e0 msgr2=0x7fc16006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.614+0000 7fc17a296640 1 --2- 172.21.15.16:0/383108439 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc16006c2e0 0x7fc16006e7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc1740b8df0 tx=0x7fc168009190 comp rx=0 tx=0).stop 2024-09-19T07:44:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.614+0000 7fc17a296640 1 -- 172.21.15.16:0/383108439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c0039e0 msgr2=0x7fc1740b5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.614+0000 7fc17a296640 1 --2- 172.21.15.16:0/383108439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c0039e0 0x7fc1740b5850 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc16402ae50 tx=0x7fc164001a40 comp rx=0 tx=0).stop 2024-09-19T07:44:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.614+0000 7fc17a296640 1 -- 172.21.15.16:0/383108439 shutdown_connections 2024-09-19T07:44:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.614+0000 7fc17a296640 1 --2- 172.21.15.16:0/383108439 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc16006c2e0 0x7fc16006e7a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.614+0000 7fc17a296640 1 --2- 172.21.15.16:0/383108439 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1740d0a90 0x7fc1740b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.614+0000 7fc17a296640 1 --2- 172.21.15.16:0/383108439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1741f48a0 0x7fc1740d0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.614+0000 7fc17a296640 1 --2- 172.21.15.16:0/383108439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c0039e0 0x7fc1740b5850 secure :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc16402ae50 tx=0x7fc164001a40 comp rx=0 tx=0).stop 2024-09-19T07:44:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.614+0000 7fc17a296640 1 -- 172.21.15.16:0/383108439 >> 172.21.15.16:0/383108439 conn(0x7fc17409d760 msgr2=0x7fc17415f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.615+0000 7fc17a296640 1 -- 172.21.15.16:0/383108439 shutdown_connections 2024-09-19T07:44:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.615+0000 7fc17a296640 1 -- 172.21.15.16:0/383108439 wait complete. 2024-09-19T07:44:01.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:44:01.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.755+0000 7f2e3b7c4640 1 Processor -- start 2024-09-19T07:44:01.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.755+0000 7f2e3b7c4640 1 -- start start 2024-09-19T07:44:01.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.756+0000 7f2e3b7c4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e340d0070 0x7f2e3415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:01.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.756+0000 7f2e3b7c4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e340d1b10 con 0x7f2e3415f0d0 2024-09-19T07:44:01.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.756+0000 7f2e3b7c4640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e340cf2b0 con 0x7f2e340d4130 2024-09-19T07:44:01.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.756+0000 7f2e3b7c4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2e340cf590 con 0x7f2e340d0070 2024-09-19T07:44:01.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.756+0000 7f2e39560640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2e3415f0d0 0x7f2e340d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56182/0 (socket says 172.21.15.16:56182) 2024-09-19T07:44:01.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.756+0000 7f2e39560640 1 -- 172.21.15.16:0/487020840 learned_addr learned my addr 172.21.15.16:0/487020840 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:44:01.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.757+0000 7f2e38d5f640 1 --2- 172.21.15.16:0/487020840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e340d0070 0x7f2e3415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:01.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.757+0000 7f2e33fff640 1 -- 172.21.15.16:0/487020840 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2164702499 0 0) 0x7f2e340d1b10 con 0x7f2e3415f0d0 2024-09-19T07:44:01.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.757+0000 7f2e33fff640 1 -- 172.21.15.16:0/487020840 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e1c003520 con 0x7f2e3415f0d0 2024-09-19T07:44:01.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.757+0000 7f2e33fff640 1 -- 172.21.15.16:0/487020840 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3745627117 0 0) 0x7f2e340cf2b0 con 0x7f2e340d4130 2024-09-19T07:44:01.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.757+0000 7f2e33fff640 1 -- 172.21.15.16:0/487020840 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e340d1b10 con 0x7f2e340d4130 2024-09-19T07:44:01.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.757+0000 7f2e38d5f640 1 -- 172.21.15.16:0/487020840 >> v1:172.21.15.146:6789/0 conn(0x7f2e340d4130 legacy=0x7f2e340d45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:01.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.757+0000 7f2e38d5f640 1 -- 172.21.15.16:0/487020840 >> v1:172.21.15.16:6789/0 conn(0x7f2e3415f0d0 legacy=0x7f2e340d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:01.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.757+0000 7f2e38d5f640 1 -- 172.21.15.16:0/487020840 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e340cf870 con 0x7f2e340d0070 2024-09-19T07:44:01.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.758+0000 7f2e38d5f640 1 --2- 172.21.15.16:0/487020840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e340d0070 0x7f2e3415f4b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2e280097d0 tx=0x7f2e2802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7bfec2f1359c061 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.758+0000 7f2e33fff640 1 -- 172.21.15.16:0/487020840 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28037070 con 0x7f2e340d0070 2024-09-19T07:44:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.758+0000 7f2e33fff640 1 -- 172.21.15.16:0/487020840 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e28032440 con 0x7f2e340d0070 2024-09-19T07:44:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.758+0000 7f2e33fff640 1 -- 172.21.15.16:0/487020840 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28032810 con 0x7f2e340d0070 2024-09-19T07:44:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.758+0000 7f2e3b7c4640 1 -- 172.21.15.16:0/487020840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e340d0070 msgr2=0x7f2e3415f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.758+0000 7f2e3b7c4640 1 --2- 172.21.15.16:0/487020840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e340d0070 0x7f2e3415f4b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2e280097d0 tx=0x7f2e2802a180 comp rx=0 tx=0).stop 2024-09-19T07:44:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.758+0000 7f2e3b7c4640 1 -- 172.21.15.16:0/487020840 shutdown_connections 2024-09-19T07:44:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.758+0000 7f2e3b7c4640 1 --2- 172.21.15.16:0/487020840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e340d0070 0x7f2e3415f4b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.758+0000 7f2e3b7c4640 1 -- 172.21.15.16:0/487020840 >> 172.21.15.16:0/487020840 conn(0x7f2e3409d760 msgr2=0x7f2e3408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:01.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.758+0000 7f2e3b7c4640 1 -- 172.21.15.16:0/487020840 shutdown_connections 2024-09-19T07:44:01.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.758+0000 7f2e3b7c4640 1 -- 172.21.15.16:0/487020840 wait complete. 2024-09-19T07:44:01.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.760+0000 7f2e3b7c4640 1 Processor -- start 2024-09-19T07:44:01.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.760+0000 7f2e3b7c4640 1 -- start start 2024-09-19T07:44:01.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.760+0000 7f2e3b7c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e340d0070 0x7f2e3415aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:01.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.760+0000 7f2e3b7c4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e340d4130 0x7f2e340b7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:01.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.760+0000 7f2e3b7c4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e3415f0d0 0x7f2e340b91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:01.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.760+0000 7f2e3b7c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e340b2f70 con 0x7f2e340d0070 2024-09-19T07:44:01.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.760+0000 7f2e3b7c4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2e340cef60 con 0x7f2e340d4130 2024-09-19T07:44:01.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.760+0000 7f2e3b7c4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2e340b30f0 con 0x7f2e3415f0d0 2024-09-19T07:44:01.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.760+0000 7f2e39560640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e340d0070 0x7f2e3415aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:01.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.761+0000 7f2e39560640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e340d0070 0x7f2e3415aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34640/0 (socket says 172.21.15.16:34640) 2024-09-19T07:44:01.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.761+0000 7f2e38d5f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e340d4130 0x7f2e340b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:01.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.761+0000 7f2e39560640 1 -- 172.21.15.16:0/4178722115 learned_addr learned my addr 172.21.15.16:0/4178722115 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:01.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.761+0000 7f2e39d61640 1 --2- 172.21.15.16:0/4178722115 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e3415f0d0 0x7f2e340b91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:01.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.761+0000 7f2e39560640 1 -- 172.21.15.16:0/4178722115 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e3415f0d0 msgr2=0x7f2e340b91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:01.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.761+0000 7f2e39560640 1 --2- 172.21.15.16:0/4178722115 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e3415f0d0 0x7f2e340b91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:01.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.761+0000 7f2e39560640 1 -- 172.21.15.16:0/4178722115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e340d4130 msgr2=0x7f2e340b7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:01.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.761+0000 7f2e39560640 1 --2- 172.21.15.16:0/4178722115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e340d4130 0x7f2e340b7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:01.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.761+0000 7f2e39560640 1 -- 172.21.15.16:0/4178722115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3415b1f0 con 0x7f2e340d0070 2024-09-19T07:44:01.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.761+0000 7f2e39560640 1 --2- 172.21.15.16:0/4178722115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e340d0070 0x7f2e3415aaf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2e2400cbf0 tx=0x7f2e2400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:01.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.761+0000 7f2e31ffb640 1 -- 172.21.15.16:0/4178722115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e24017070 con 0x7f2e340d0070 2024-09-19T07:44:01.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.761+0000 7f2e31ffb640 1 -- 172.21.15.16:0/4178722115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e2400ad40 con 0x7f2e340d0070 2024-09-19T07:44:01.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.761+0000 7f2e3b7c4640 1 -- 172.21.15.16:0/4178722115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3415b4d0 con 0x7f2e340d0070 2024-09-19T07:44:01.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.761+0000 7f2e31ffb640 1 -- 172.21.15.16:0/4178722115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e24012600 con 0x7f2e340d0070 2024-09-19T07:44:01.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.761+0000 7f2e3b7c4640 1 -- 172.21.15.16:0/4178722115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e3415ba90 con 0x7f2e340d0070 2024-09-19T07:44:01.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.763+0000 7f2e3b7c4640 1 -- 172.21.15.16:0/4178722115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e3415f4b0 con 0x7f2e340d0070 2024-09-19T07:44:01.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.763+0000 7f2e31ffb640 1 -- 172.21.15.16:0/4178722115 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2e2401a070 con 0x7f2e340d0070 2024-09-19T07:44:01.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.763+0000 7f2e31ffb640 1 --2- 172.21.15.16:0/4178722115 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2e2006c2b0 0x7f2e2006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:01.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.763+0000 7f2e31ffb640 1 -- 172.21.15.16:0/4178722115 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f2e2408b2e0 con 0x7f2e340d0070 2024-09-19T07:44:01.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.764+0000 7f2e38d5f640 1 --2- 172.21.15.16:0/4178722115 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2e2006c2b0 0x7f2e2006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:01.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.764+0000 7f2e38d5f640 1 --2- 172.21.15.16:0/4178722115 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2e2006c2b0 0x7f2e2006e770 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2e28034000 tx=0x7f2e2802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:01.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.768+0000 7f2e31ffb640 1 -- 172.21.15.16:0/4178722115 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2e24059270 con 0x7f2e340d0070 2024-09-19T07:44:01.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.944+0000 7f2e3b7c4640 1 -- 172.21.15.16:0/4178722115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e3406cf30 con 0x7f2e340d0070 2024-09-19T07:44:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.945+0000 7f2e31ffb640 1 -- 172.21.15.16:0/4178722115 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17039 (secure 0 0 0) 0x7f2e2405d570 con 0x7f2e340d0070 2024-09-19T07:44:01.946 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:44:01.946 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:44:00.970372+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:43:58.882707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-19T07:43:59.960376+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:44:01.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.947+0000 7f2e3b7c4640 1 -- 172.21.15.16:0/4178722115 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2e2006c2b0 msgr2=0x7f2e2006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:01.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.947+0000 7f2e3b7c4640 1 --2- 172.21.15.16:0/4178722115 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2e2006c2b0 0x7f2e2006e770 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2e28034000 tx=0x7f2e2802e000 comp rx=0 tx=0).stop 2024-09-19T07:44:01.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.947+0000 7f2e3b7c4640 1 -- 172.21.15.16:0/4178722115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e340d0070 msgr2=0x7f2e3415aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:01.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.947+0000 7f2e3b7c4640 1 --2- 172.21.15.16:0/4178722115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e340d0070 0x7f2e3415aaf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2e2400cbf0 tx=0x7f2e2400a430 comp rx=0 tx=0).stop 2024-09-19T07:44:01.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.947+0000 7f2e3b7c4640 1 -- 172.21.15.16:0/4178722115 shutdown_connections 2024-09-19T07:44:01.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.947+0000 7f2e3b7c4640 1 --2- 172.21.15.16:0/4178722115 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2e2006c2b0 0x7f2e2006e770 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:01.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.947+0000 7f2e3b7c4640 1 --2- 172.21.15.16:0/4178722115 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e3415f0d0 0x7f2e340b91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:01.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.947+0000 7f2e3b7c4640 1 --2- 172.21.15.16:0/4178722115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e340d4130 0x7f2e340b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:01.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.947+0000 7f2e3b7c4640 1 --2- 172.21.15.16:0/4178722115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e340d0070 0x7f2e3415aaf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:01.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.947+0000 7f2e3b7c4640 1 -- 172.21.15.16:0/4178722115 >> 172.21.15.16:0/4178722115 conn(0x7f2e3409d760 msgr2=0x7f2e340b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:01.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.947+0000 7f2e3b7c4640 1 -- 172.21.15.16:0/4178722115 shutdown_connections 2024-09-19T07:44:01.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:01.947+0000 7f2e3b7c4640 1 -- 172.21.15.16:0/4178722115 wait complete. 2024-09-19T07:44:01.959 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:44:02.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.060+0000 7f7784731640 1 Processor -- start 2024-09-19T07:44:02.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.060+0000 7f7784731640 1 -- start start 2024-09-19T07:44:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.061+0000 7f7784731640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f777c15e2a0 0x7f777c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.061+0000 7f7784731640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f777c15e680 0x7f777c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.061+0000 7f7784731640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f777c0d1880 con 0x7f777c06d960 2024-09-19T07:44:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.061+0000 7f7784731640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f777c0d1b60 con 0x7f777c15e680 2024-09-19T07:44:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.061+0000 7f7784731640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f777c0cf2d0 con 0x7f777c15e2a0 2024-09-19T07:44:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.061+0000 7f7781ccc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f777c15e680 0x7f777c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.061+0000 7f7781ccc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f777c15e680 0x7f777c0cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34650/0 (socket says 172.21.15.16:34650) 2024-09-19T07:44:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.061+0000 7f7781ccc640 1 -- 172.21.15.16:0/1474552141 learned_addr learned my addr 172.21.15.16:0/1474552141 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.061+0000 7f77824cd640 1 --2- 172.21.15.16:0/1474552141 >> v2:172.21.15.146:3300/0 conn(0x7f777c15e2a0 0x7f777c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.061+0000 7f7781ccc640 1 -- 172.21.15.16:0/1474552141 >> v2:172.21.15.146:3300/0 conn(0x7f777c15e2a0 msgr2=0x7f777c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.061+0000 7f7781ccc640 1 --2- 172.21.15.16:0/1474552141 >> v2:172.21.15.146:3300/0 conn(0x7f777c15e2a0 0x7f777c06d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.061+0000 7f7781ccc640 1 -- 172.21.15.16:0/1474552141 >> v1:172.21.15.146:6789/0 conn(0x7f777c06d960 legacy=0x7f777c0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.061+0000 7f7781ccc640 1 -- 172.21.15.16:0/1474552141 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777c0cf5b0 con 0x7f777c15e680 2024-09-19T07:44:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.061+0000 7f77824cd640 1 --2- 172.21.15.16:0/1474552141 >> v2:172.21.15.146:3300/0 conn(0x7f777c15e2a0 0x7f777c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.062+0000 7f7781ccc640 1 --2- 172.21.15.16:0/1474552141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c15e680 0x7f777c0cfad0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f777c0b6fa0 tx=0x7f776c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6331479263e01431 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.062+0000 7f77814cb640 1 -- 172.21.15.16:0/1474552141 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c037070 con 0x7f777c15e680 2024-09-19T07:44:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.062+0000 7f77814cb640 1 -- 172.21.15.16:0/1474552141 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f776c032440 con 0x7f777c15e680 2024-09-19T07:44:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.062+0000 7f77814cb640 1 -- 172.21.15.16:0/1474552141 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c032810 con 0x7f777c15e680 2024-09-19T07:44:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.063+0000 7f7784731640 1 -- 172.21.15.16:0/1474552141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c15e680 msgr2=0x7f777c0cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.063+0000 7f7784731640 1 --2- 172.21.15.16:0/1474552141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c15e680 0x7f777c0cfad0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f777c0b6fa0 tx=0x7f776c02a270 comp rx=0 tx=0).stop 2024-09-19T07:44:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.063+0000 7f7784731640 1 -- 172.21.15.16:0/1474552141 shutdown_connections 2024-09-19T07:44:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.063+0000 7f7784731640 1 --2- 172.21.15.16:0/1474552141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c15e680 0x7f777c0cfad0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.063+0000 7f7784731640 1 --2- 172.21.15.16:0/1474552141 >> v2:172.21.15.146:3300/0 conn(0x7f777c15e2a0 0x7f777c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.063+0000 7f7784731640 1 -- 172.21.15.16:0/1474552141 >> 172.21.15.16:0/1474552141 conn(0x7f777c09d760 msgr2=0x7f777c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.063+0000 7f7784731640 1 -- 172.21.15.16:0/1474552141 shutdown_connections 2024-09-19T07:44:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.063+0000 7f7784731640 1 -- 172.21.15.16:0/1474552141 wait complete. 2024-09-19T07:44:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.064+0000 7f7784731640 1 Processor -- start 2024-09-19T07:44:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.064+0000 7f7784731640 1 -- start start 2024-09-19T07:44:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.065+0000 7f7784731640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f777c06d960 0x7f777c0c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.065+0000 7f7784731640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f777c15e2a0 0x7f777c0caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.065+0000 7f7784731640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c15e680 0x7f777c0d2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.065+0000 7f7784731640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f777c0d4c00 con 0x7f777c15e680 2024-09-19T07:44:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.065+0000 7f7784731640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f777c0d4a80 con 0x7f777c06d960 2024-09-19T07:44:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.065+0000 7f7784731640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f777c0d0010 con 0x7f777c15e2a0 2024-09-19T07:44:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.065+0000 7f7782cce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c15e680 0x7f777c0d2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.065+0000 7f7782cce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c15e680 0x7f777c0d2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34652/0 (socket says 172.21.15.16:34652) 2024-09-19T07:44:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.065+0000 7f7782cce640 1 -- 172.21.15.16:0/4257317002 learned_addr learned my addr 172.21.15.16:0/4257317002 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.065+0000 7f77824cd640 1 --2- 172.21.15.16:0/4257317002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f777c06d960 0x7f777c0c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.065+0000 7f7781ccc640 1 --2- 172.21.15.16:0/4257317002 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f777c15e2a0 0x7f777c0caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.065+0000 7f7782cce640 1 -- 172.21.15.16:0/4257317002 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f777c15e2a0 msgr2=0x7f777c0caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.065+0000 7f7782cce640 1 --2- 172.21.15.16:0/4257317002 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f777c15e2a0 0x7f777c0caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.065+0000 7f7782cce640 1 -- 172.21.15.16:0/4257317002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f777c06d960 msgr2=0x7f777c0c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.065+0000 7f7782cce640 1 --2- 172.21.15.16:0/4257317002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f777c06d960 0x7f777c0c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:02.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.065+0000 7f7782cce640 1 -- 172.21.15.16:0/4257317002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777c0c99a0 con 0x7f777c15e680 2024-09-19T07:44:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.065+0000 7f7782cce640 1 --2- 172.21.15.16:0/4257317002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c15e680 0x7f777c0d2440 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f777400c7d0 tx=0x7f777400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.066+0000 7f7772ffd640 1 -- 172.21.15.16:0/4257317002 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7774003b70 con 0x7f777c15e680 2024-09-19T07:44:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.066+0000 7f7772ffd640 1 -- 172.21.15.16:0/4257317002 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7774003d30 con 0x7f777c15e680 2024-09-19T07:44:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.066+0000 7f7772ffd640 1 -- 172.21.15.16:0/4257317002 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777400aa90 con 0x7f777c15e680 2024-09-19T07:44:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.066+0000 7f7784731640 1 -- 172.21.15.16:0/4257317002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777c0c9c80 con 0x7f777c15e680 2024-09-19T07:44:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.066+0000 7f7784731640 1 -- 172.21.15.16:0/4257317002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f777c0ca240 con 0x7f777c15e680 2024-09-19T07:44:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.067+0000 7f7784731640 1 -- 172.21.15.16:0/4257317002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f777c06d420 con 0x7f777c15e680 2024-09-19T07:44:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.067+0000 7f7772ffd640 1 -- 172.21.15.16:0/4257317002 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f777400ac30 con 0x7f777c15e680 2024-09-19T07:44:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.068+0000 7f7772ffd640 1 --2- 172.21.15.16:0/4257317002 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f776006c2b0 0x7f776006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.068+0000 7f7772ffd640 1 -- 172.21.15.16:0/4257317002 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f777408b320 con 0x7f777c15e680 2024-09-19T07:44:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.068+0000 7f77824cd640 1 --2- 172.21.15.16:0/4257317002 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f776006c2b0 0x7f776006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.068+0000 7f77824cd640 1 --2- 172.21.15.16:0/4257317002 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f776006c2b0 0x7f776006e770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f777c0d0a50 tx=0x7f7778008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.072+0000 7f7772ffd640 1 -- 172.21.15.16:0/4257317002 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7774059340 con 0x7f777c15e680 2024-09-19T07:44:02.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.247+0000 7f7784731640 1 -- 172.21.15.16:0/4257317002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f777c06cf90 con 0x7f777c15e680 2024-09-19T07:44:02.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.248+0000 7f7772ffd640 1 -- 172.21.15.16:0/4257317002 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17039 (secure 0 0 0) 0x7f777405d640 con 0x7f777c15e680 2024-09-19T07:44:02.249 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:44:02.249 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:44:00.970372+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:43:58.882707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-19T07:43:59.960376+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:44:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.250+0000 7f7784731640 1 -- 172.21.15.16:0/4257317002 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f776006c2b0 msgr2=0x7f776006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.250+0000 7f7784731640 1 --2- 172.21.15.16:0/4257317002 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f776006c2b0 0x7f776006e770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f777c0d0a50 tx=0x7f7778008040 comp rx=0 tx=0).stop 2024-09-19T07:44:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.250+0000 7f7784731640 1 -- 172.21.15.16:0/4257317002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c15e680 msgr2=0x7f777c0d2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.250+0000 7f7784731640 1 --2- 172.21.15.16:0/4257317002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c15e680 0x7f777c0d2440 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f777400c7d0 tx=0x7f777400cc90 comp rx=0 tx=0).stop 2024-09-19T07:44:02.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.250+0000 7f7784731640 1 -- 172.21.15.16:0/4257317002 shutdown_connections 2024-09-19T07:44:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.250+0000 7f7784731640 1 --2- 172.21.15.16:0/4257317002 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f776006c2b0 0x7f776006e770 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.250+0000 7f7784731640 1 --2- 172.21.15.16:0/4257317002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f777c15e680 0x7f777c0d2440 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.250+0000 7f7784731640 1 --2- 172.21.15.16:0/4257317002 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f777c15e2a0 0x7f777c0caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.250+0000 7f7784731640 1 --2- 172.21.15.16:0/4257317002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f777c06d960 0x7f777c0c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.250+0000 7f7784731640 1 -- 172.21.15.16:0/4257317002 >> 172.21.15.16:0/4257317002 conn(0x7f777c09d760 msgr2=0x7f777c0d0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.251+0000 7f7784731640 1 -- 172.21.15.16:0/4257317002 shutdown_connections 2024-09-19T07:44:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.251+0000 7f7784731640 1 -- 172.21.15.16:0/4257317002 wait complete. 2024-09-19T07:44:02.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T07:44:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.367+0000 7fe17f998640 1 Processor -- start 2024-09-19T07:44:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.367+0000 7fe17f998640 1 -- start start 2024-09-19T07:44:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.368+0000 7fe17f998640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe17815e2a0 0x7fe17806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.368+0000 7fe17f998640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe17806d960 0x7fe1780d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.368+0000 7fe17f998640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1780d1880 con 0x7fe17815e680 2024-09-19T07:44:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.368+0000 7fe17f998640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fe1780d1b60 con 0x7fe17815e2a0 2024-09-19T07:44:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.368+0000 7fe17f998640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe1780cf2d0 con 0x7fe17806d960 2024-09-19T07:44:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.368+0000 7fe17d734640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe17815e2a0 0x7fe17806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.368+0000 7fe17d734640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe17815e2a0 0x7fe17806d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34656/0 (socket says 172.21.15.16:34656) 2024-09-19T07:44:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.368+0000 7fe17cf33640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe17815e680 0x7fe1780cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56196/0 (socket says 172.21.15.16:56196) 2024-09-19T07:44:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.368+0000 7fe17d734640 1 -- 172.21.15.16:0/2027720342 learned_addr learned my addr 172.21.15.16:0/2027720342 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.368+0000 7fe17df35640 1 --2- 172.21.15.16:0/2027720342 >> v2:172.21.15.146:3300/0 conn(0x7fe17806d960 0x7fe1780d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.368+0000 7fe17d734640 1 -- 172.21.15.16:0/2027720342 >> v2:172.21.15.146:3300/0 conn(0x7fe17806d960 msgr2=0x7fe1780d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.368+0000 7fe17d734640 1 --2- 172.21.15.16:0/2027720342 >> v2:172.21.15.146:3300/0 conn(0x7fe17806d960 0x7fe1780d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.368+0000 7fe17d734640 1 -- 172.21.15.16:0/2027720342 >> v1:172.21.15.16:6789/0 conn(0x7fe17815e680 legacy=0x7fe1780cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.368+0000 7fe16ffff640 1 -- 172.21.15.16:0/2027720342 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1928122120 0 0) 0x7fe1780d1880 con 0x7fe17815e680 2024-09-19T07:44:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.368+0000 7fe17d734640 1 -- 172.21.15.16:0/2027720342 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1780cf5b0 con 0x7fe17815e2a0 2024-09-19T07:44:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.369+0000 7fe17d734640 1 --2- 172.21.15.16:0/2027720342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17815e2a0 0x7fe17806d420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe178163050 tx=0x7fe17402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164c2fa9ee6a1daa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.369+0000 7fe16ffff640 1 -- 172.21.15.16:0/2027720342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe174037070 con 0x7fe17815e2a0 2024-09-19T07:44:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.369+0000 7fe16ffff640 1 -- 172.21.15.16:0/2027720342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe174032440 con 0x7fe17815e2a0 2024-09-19T07:44:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.369+0000 7fe16ffff640 1 -- 172.21.15.16:0/2027720342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe174032810 con 0x7fe17815e2a0 2024-09-19T07:44:02.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.370+0000 7fe17f998640 1 -- 172.21.15.16:0/2027720342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17815e2a0 msgr2=0x7fe17806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:02.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.370+0000 7fe17f998640 1 --2- 172.21.15.16:0/2027720342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17815e2a0 0x7fe17806d420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe178163050 tx=0x7fe17402a270 comp rx=0 tx=0).stop 2024-09-19T07:44:02.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.370+0000 7fe17f998640 1 -- 172.21.15.16:0/2027720342 shutdown_connections 2024-09-19T07:44:02.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.370+0000 7fe17f998640 1 --2- 172.21.15.16:0/2027720342 >> v2:172.21.15.146:3300/0 conn(0x7fe17806d960 0x7fe1780d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:02.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.370+0000 7fe17f998640 1 --2- 172.21.15.16:0/2027720342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17815e2a0 0x7fe17806d420 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:02.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.370+0000 7fe17f998640 1 -- 172.21.15.16:0/2027720342 >> 172.21.15.16:0/2027720342 conn(0x7fe17809d760 msgr2=0x7fe17808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:02.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.370+0000 7fe17f998640 1 -- 172.21.15.16:0/2027720342 shutdown_connections 2024-09-19T07:44:02.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.370+0000 7fe17f998640 1 -- 172.21.15.16:0/2027720342 wait complete. 2024-09-19T07:44:02.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.371+0000 7fe17f998640 1 Processor -- start 2024-09-19T07:44:02.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.371+0000 7fe17f998640 1 -- start start 2024-09-19T07:44:02.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.372+0000 7fe17f998640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17806d960 0x7fe1780cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:02.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.372+0000 7fe17f998640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe17815e2a0 0x7fe178153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:02.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.372+0000 7fe17f998640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17815e680 0x7fe17815d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:02.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.372+0000 7fe17f998640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1780d0360 con 0x7fe17806d960 2024-09-19T07:44:02.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.372+0000 7fe17f998640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe1780d01e0 con 0x7fe17815e2a0 2024-09-19T07:44:02.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.372+0000 7fe17f998640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe1780d48b0 con 0x7fe17815e680 2024-09-19T07:44:02.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.372+0000 7fe17d734640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17806d960 0x7fe1780cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:02.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.372+0000 7fe17d734640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17806d960 0x7fe1780cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34670/0 (socket says 172.21.15.16:34670) 2024-09-19T07:44:02.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.372+0000 7fe17d734640 1 -- 172.21.15.16:0/3923715311 learned_addr learned my addr 172.21.15.16:0/3923715311 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:02.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.372+0000 7fe17cf33640 1 --2- 172.21.15.16:0/3923715311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe17815e2a0 0x7fe178153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.372+0000 7fe17df35640 1 --2- 172.21.15.16:0/3923715311 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17815e680 0x7fe17815d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.372+0000 7fe17d734640 1 -- 172.21.15.16:0/3923715311 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17815e680 msgr2=0x7fe17815d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.372+0000 7fe17d734640 1 --2- 172.21.15.16:0/3923715311 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17815e680 0x7fe17815d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.372+0000 7fe17d734640 1 -- 172.21.15.16:0/3923715311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe17815e2a0 msgr2=0x7fe178153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.372+0000 7fe17d734640 1 --2- 172.21.15.16:0/3923715311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe17815e2a0 0x7fe178153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.372+0000 7fe17d734640 1 -- 172.21.15.16:0/3923715311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe178153590 con 0x7fe17806d960 2024-09-19T07:44:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.372+0000 7fe17cf33640 1 --2- 172.21.15.16:0/3923715311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe17815e2a0 0x7fe178153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.373+0000 7fe17d734640 1 --2- 172.21.15.16:0/3923715311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17806d960 0x7fe1780cb4b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe174000c00 tx=0x7fe1740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.373+0000 7fe16dffb640 1 -- 172.21.15.16:0/3923715311 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe174037040 con 0x7fe17806d960 2024-09-19T07:44:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.373+0000 7fe16dffb640 1 -- 172.21.15.16:0/3923715311 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe174004030 con 0x7fe17806d960 2024-09-19T07:44:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.373+0000 7fe17f998640 1 -- 172.21.15.16:0/3923715311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe178150870 con 0x7fe17806d960 2024-09-19T07:44:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.373+0000 7fe16dffb640 1 -- 172.21.15.16:0/3923715311 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe174004400 con 0x7fe17806d960 2024-09-19T07:44:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.373+0000 7fe17f998640 1 -- 172.21.15.16:0/3923715311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe178150e30 con 0x7fe17806d960 2024-09-19T07:44:02.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.374+0000 7fe17f998640 1 -- 172.21.15.16:0/3923715311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1780d4c70 con 0x7fe17806d960 2024-09-19T07:44:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.374+0000 7fe16dffb640 1 -- 172.21.15.16:0/3923715311 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe17403a070 con 0x7fe17806d960 2024-09-19T07:44:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.375+0000 7fe16dffb640 1 --2- 172.21.15.16:0/3923715311 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe15c06c2e0 0x7fe15c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.375+0000 7fe16dffb640 1 -- 172.21.15.16:0/3923715311 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fe1740abbf0 con 0x7fe17806d960 2024-09-19T07:44:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.375+0000 7fe17cf33640 1 --2- 172.21.15.16:0/3923715311 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe15c06c2e0 0x7fe15c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.376+0000 7fe17cf33640 1 --2- 172.21.15.16:0/3923715311 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe15c06c2e0 0x7fe15c06e7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe178075130 tx=0x7fe168007ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:02.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.379+0000 7fe16dffb640 1 -- 172.21.15.16:0/3923715311 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe174041020 con 0x7fe17806d960 2024-09-19T07:44:02.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:02.591+0000 7fe17f998640 1 -- 172.21.15.16:0/3923715311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe17806cf90 con 0x7fe17806d960 2024-09-19T07:44:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.107+0000 7fe16dffb640 1 -- 172.21.15.16:0/3923715311 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7fe17407a1d0 con 0x7fe17806d960 2024-09-19T07:44:03.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.110+0000 7fe17f998640 1 -- 172.21.15.16:0/3923715311 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe15c06c2e0 msgr2=0x7fe15c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:03.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.110+0000 7fe17f998640 1 --2- 172.21.15.16:0/3923715311 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe15c06c2e0 0x7fe15c06e7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe178075130 tx=0x7fe168007ca0 comp rx=0 tx=0).stop 2024-09-19T07:44:03.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.110+0000 7fe17f998640 1 -- 172.21.15.16:0/3923715311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17806d960 msgr2=0x7fe1780cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:03.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.110+0000 7fe17f998640 1 --2- 172.21.15.16:0/3923715311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17806d960 0x7fe1780cb4b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe174000c00 tx=0x7fe1740046c0 comp rx=0 tx=0).stop 2024-09-19T07:44:03.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.110+0000 7fe17f998640 1 -- 172.21.15.16:0/3923715311 shutdown_connections 2024-09-19T07:44:03.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.110+0000 7fe17f998640 1 --2- 172.21.15.16:0/3923715311 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe15c06c2e0 0x7fe15c06e7a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:03.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.110+0000 7fe17f998640 1 --2- 172.21.15.16:0/3923715311 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17815e680 0x7fe17815d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:03.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.110+0000 7fe17f998640 1 --2- 172.21.15.16:0/3923715311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe17815e2a0 0x7fe178153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:03.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.110+0000 7fe17f998640 1 --2- 172.21.15.16:0/3923715311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe17806d960 0x7fe1780cb4b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:03.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.110+0000 7fe17f998640 1 -- 172.21.15.16:0/3923715311 >> 172.21.15.16:0/3923715311 conn(0x7fe17809d760 msgr2=0x7fe178160740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:03.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.110+0000 7fe17f998640 1 -- 172.21.15.16:0/3923715311 shutdown_connections 2024-09-19T07:44:03.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.110+0000 7fe17f998640 1 -- 172.21.15.16:0/3923715311 wait complete. 2024-09-19T07:44:03.123 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T07:44:03.123 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T07:44:03.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.231+0000 7ff8c332c640 1 Processor -- start 2024-09-19T07:44:03.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.232+0000 7ff8c332c640 1 -- start start 2024-09-19T07:44:03.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.232+0000 7ff8c332c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff8bc072250 0x7ff8bc0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:03.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.232+0000 7ff8c332c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8bc0726c0 0x7ff8bc0d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.232+0000 7ff8c332c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8bc0d2b30 con 0x7ff8bc0d2750 2024-09-19T07:44:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.232+0000 7ff8c332c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff8bc145bd0 con 0x7ff8bc072250 2024-09-19T07:44:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.232+0000 7ff8c332c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff8bc1427e0 con 0x7ff8bc0726c0 2024-09-19T07:44:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.233+0000 7ff8c08c7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff8bc072250 0x7ff8bc0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.233+0000 7ff8c08c7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff8bc072250 0x7ff8bc0cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51646/0 (socket says 172.21.15.16:51646) 2024-09-19T07:44:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.233+0000 7ff8c08c7640 1 -- 172.21.15.16:0/176493640 learned_addr learned my addr 172.21.15.16:0/176493640 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.233+0000 7ff8c18c9640 1 --2- 172.21.15.16:0/176493640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8bc0726c0 0x7ff8bc0d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.233+0000 7ff8c08c7640 1 -- 172.21.15.16:0/176493640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8bc0726c0 msgr2=0x7ff8bc0d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.233+0000 7ff8c08c7640 1 --2- 172.21.15.16:0/176493640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8bc0726c0 0x7ff8bc0d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.233+0000 7ff8c08c7640 1 -- 172.21.15.16:0/176493640 >> v1:172.21.15.146:6789/0 conn(0x7ff8bc0d2750 legacy=0x7ff8bc13f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.233+0000 7ff8c08c7640 1 -- 172.21.15.16:0/176493640 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8bc141c60 con 0x7ff8bc072250 2024-09-19T07:44:03.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.234+0000 7ff8c08c7640 1 --2- 172.21.15.16:0/176493640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8bc072250 0x7ff8bc0cfdc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009870 tx=0x7ff8ac02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1445d4d4181857ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:03.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.234+0000 7ff8b3fff640 1 -- 172.21.15.16:0/176493640 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac037070 con 0x7ff8bc072250 2024-09-19T07:44:03.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.234+0000 7ff8b3fff640 1 -- 172.21.15.16:0/176493640 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8ac032440 con 0x7ff8bc072250 2024-09-19T07:44:03.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.234+0000 7ff8b3fff640 1 -- 172.21.15.16:0/176493640 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac032810 con 0x7ff8bc072250 2024-09-19T07:44:03.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.234+0000 7ff8c332c640 1 -- 172.21.15.16:0/176493640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8bc072250 msgr2=0x7ff8bc0cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:03.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.234+0000 7ff8c332c640 1 --2- 172.21.15.16:0/176493640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8bc072250 0x7ff8bc0cfdc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009870 tx=0x7ff8ac02a1c0 comp rx=0 tx=0).stop 2024-09-19T07:44:03.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.234+0000 7ff8c332c640 1 -- 172.21.15.16:0/176493640 shutdown_connections 2024-09-19T07:44:03.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.234+0000 7ff8c332c640 1 --2- 172.21.15.16:0/176493640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8bc0726c0 0x7ff8bc0d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:03.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.234+0000 7ff8c332c640 1 --2- 172.21.15.16:0/176493640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8bc072250 0x7ff8bc0cfdc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:03.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.234+0000 7ff8c332c640 1 -- 172.21.15.16:0/176493640 >> 172.21.15.16:0/176493640 conn(0x7ff8bc0b4040 msgr2=0x7ff8bc0b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:03.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.235+0000 7ff8c332c640 1 -- 172.21.15.16:0/176493640 shutdown_connections 2024-09-19T07:44:03.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.235+0000 7ff8c332c640 1 -- 172.21.15.16:0/176493640 wait complete. 2024-09-19T07:44:03.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.236+0000 7ff8c332c640 1 Processor -- start 2024-09-19T07:44:03.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.236+0000 7ff8c332c640 1 -- start start 2024-09-19T07:44:03.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.236+0000 7ff8c332c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8bc072250 0x7ff8bc15e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:03.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.236+0000 7ff8c332c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8bc0726c0 0x7ff8bc145170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:03.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.236+0000 7ff8c332c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8bc0d2750 0x7ff8bc06e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:03.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.236+0000 7ff8c332c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc142500 con 0x7ff8bc0726c0 2024-09-19T07:44:03.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.236+0000 7ff8c332c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc142220 con 0x7ff8bc0d2750 2024-09-19T07:44:03.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.236+0000 7ff8c332c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff8bc1418b0 con 0x7ff8bc072250 2024-09-19T07:44:03.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.237+0000 7ff8c08c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8bc0726c0 0x7ff8bc145170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:03.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.237+0000 7ff8c08c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8bc0726c0 0x7ff8bc145170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51650/0 (socket says 172.21.15.16:51650) 2024-09-19T07:44:03.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.237+0000 7ff8c08c7640 1 -- 172.21.15.16:0/2017663102 learned_addr learned my addr 172.21.15.16:0/2017663102 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.237+0000 7ff8c18c9640 1 --2- 172.21.15.16:0/2017663102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8bc0d2750 0x7ff8bc06e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.237+0000 7ff8c10c8640 1 --2- 172.21.15.16:0/2017663102 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8bc072250 0x7ff8bc15e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.237+0000 7ff8c08c7640 1 -- 172.21.15.16:0/2017663102 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8bc072250 msgr2=0x7ff8bc15e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.237+0000 7ff8c08c7640 1 --2- 172.21.15.16:0/2017663102 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8bc072250 0x7ff8bc15e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.237+0000 7ff8c08c7640 1 -- 172.21.15.16:0/2017663102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8bc0d2750 msgr2=0x7ff8bc06e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.237+0000 7ff8c08c7640 1 --2- 172.21.15.16:0/2017663102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8bc0d2750 0x7ff8bc06e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.237+0000 7ff8c08c7640 1 -- 172.21.15.16:0/2017663102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8bc06cd20 con 0x7ff8bc0726c0 2024-09-19T07:44:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.237+0000 7ff8c08c7640 1 --2- 172.21.15.16:0/2017663102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8bc0726c0 0x7ff8bc145170 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac000c00 tx=0x7ff8ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.237+0000 7ff8b1ffb640 1 -- 172.21.15.16:0/2017663102 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac037040 con 0x7ff8bc0726c0 2024-09-19T07:44:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.237+0000 7ff8b1ffb640 1 -- 172.21.15.16:0/2017663102 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8ac042070 con 0x7ff8bc0726c0 2024-09-19T07:44:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.237+0000 7ff8c332c640 1 -- 172.21.15.16:0/2017663102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8bc06cfc0 con 0x7ff8bc0726c0 2024-09-19T07:44:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.237+0000 7ff8b1ffb640 1 -- 172.21.15.16:0/2017663102 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac030880 con 0x7ff8bc0726c0 2024-09-19T07:44:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.238+0000 7ff8c332c640 1 -- 172.21.15.16:0/2017663102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8bc06d580 con 0x7ff8bc0726c0 2024-09-19T07:44:03.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.238+0000 7ff8c332c640 1 -- 172.21.15.16:0/2017663102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8bc13f860 con 0x7ff8bc0726c0 2024-09-19T07:44:03.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.239+0000 7ff8b1ffb640 1 -- 172.21.15.16:0/2017663102 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff8ac02ad10 con 0x7ff8bc0726c0 2024-09-19T07:44:03.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.240+0000 7ff8b1ffb640 1 --2- 172.21.15.16:0/2017663102 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff8a006c2b0 0x7ff8a006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:03.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.240+0000 7ff8b1ffb640 1 -- 172.21.15.16:0/2017663102 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7ff8ac0aba70 con 0x7ff8bc0726c0 2024-09-19T07:44:03.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.240+0000 7ff8c10c8640 1 --2- 172.21.15.16:0/2017663102 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff8a006c2b0 0x7ff8a006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:03.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.240+0000 7ff8c10c8640 1 --2- 172.21.15.16:0/2017663102 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff8a006c2b0 0x7ff8a006e770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80016e0 tx=0x7ff8b8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.244+0000 7ff8b1ffb640 1 -- 172.21.15.16:0/2017663102 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff8ac079960 con 0x7ff8bc0726c0 2024-09-19T07:44:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:03.454+0000 7ff8c332c640 1 -- 172.21.15.16:0/2017663102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff8bc0b3120 con 0x7ff8bc0726c0 2024-09-19T07:44:04.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.109+0000 7ff8b1ffb640 1 -- 172.21.15.16:0/2017663102 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7ff8ac07dc60 con 0x7ff8bc0726c0 2024-09-19T07:44:04.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.112+0000 7ff8c332c640 1 -- 172.21.15.16:0/2017663102 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff8a006c2b0 msgr2=0x7ff8a006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.112+0000 7ff8c332c640 1 --2- 172.21.15.16:0/2017663102 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff8a006c2b0 0x7ff8a006e770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80016e0 tx=0x7ff8b8009040 comp rx=0 tx=0).stop 2024-09-19T07:44:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.112+0000 7ff8c332c640 1 -- 172.21.15.16:0/2017663102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8bc0726c0 msgr2=0x7ff8bc145170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.112+0000 7ff8c332c640 1 --2- 172.21.15.16:0/2017663102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8bc0726c0 0x7ff8bc145170 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac000c00 tx=0x7ff8ac0046c0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.112+0000 7ff8c332c640 1 -- 172.21.15.16:0/2017663102 shutdown_connections 2024-09-19T07:44:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.112+0000 7ff8c332c640 1 --2- 172.21.15.16:0/2017663102 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff8a006c2b0 0x7ff8a006e770 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.112+0000 7ff8c332c640 1 --2- 172.21.15.16:0/2017663102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8bc0d2750 0x7ff8bc06e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.112+0000 7ff8c332c640 1 --2- 172.21.15.16:0/2017663102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8bc0726c0 0x7ff8bc145170 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.112+0000 7ff8c332c640 1 --2- 172.21.15.16:0/2017663102 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8bc072250 0x7ff8bc15e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.112+0000 7ff8c332c640 1 -- 172.21.15.16:0/2017663102 >> 172.21.15.16:0/2017663102 conn(0x7ff8bc0b4040 msgr2=0x7ff8bc0cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.112+0000 7ff8c332c640 1 -- 172.21.15.16:0/2017663102 shutdown_connections 2024-09-19T07:44:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.112+0000 7ff8c332c640 1 -- 172.21.15.16:0/2017663102 wait complete. 2024-09-19T07:44:04.126 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T07:44:04.126 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:44:04.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.234+0000 7f5905c4c640 1 Processor -- start 2024-09-19T07:44:04.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.234+0000 7f5905c4c640 1 -- start start 2024-09-19T07:44:04.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.235+0000 7f5905c4c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f590006d430 0x7f59001439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.235+0000 7f5905c4c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5900143f30 0x7f59000d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.235+0000 7f5905c4c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59000c7140 con 0x7f59000c8d20 2024-09-19T07:44:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.235+0000 7f5905c4c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f5900141960 con 0x7f5900143f30 2024-09-19T07:44:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.235+0000 7f5905c4c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5900142200 con 0x7f590006d430 2024-09-19T07:44:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.235+0000 7f58fffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5900143f30 0x7f59000d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.235+0000 7f58fffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5900143f30 0x7f59000d1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51662/0 (socket says 172.21.15.16:51662) 2024-09-19T07:44:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.235+0000 7f58fffff640 1 -- 172.21.15.16:0/2857925117 learned_addr learned my addr 172.21.15.16:0/2857925117 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.235+0000 7f58feffd640 1 --2- 172.21.15.16:0/2857925117 >> v2:172.21.15.146:3300/0 conn(0x7f590006d430 0x7f59001439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.235+0000 7f58fffff640 1 -- 172.21.15.16:0/2857925117 >> v2:172.21.15.146:3300/0 conn(0x7f590006d430 msgr2=0x7f59001439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.235+0000 7f58fffff640 1 --2- 172.21.15.16:0/2857925117 >> v2:172.21.15.146:3300/0 conn(0x7f590006d430 0x7f59001439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.235+0000 7f58fffff640 1 -- 172.21.15.16:0/2857925117 >> v1:172.21.15.146:6789/0 conn(0x7f59000c8d20 legacy=0x7f590006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.235+0000 7f58fffff640 1 -- 172.21.15.16:0/2857925117 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59000c9100 con 0x7f5900143f30 2024-09-19T07:44:04.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.236+0000 7f58fffff640 1 --2- 172.21.15.16:0/2857925117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5900143f30 0x7f59000d1e50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f590006eed0 tx=0x7f58f802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98baaefaad836dc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:04.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.236+0000 7f58fe7fc640 1 -- 172.21.15.16:0/2857925117 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8036070 con 0x7f5900143f30 2024-09-19T07:44:04.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.236+0000 7f58fe7fc640 1 -- 172.21.15.16:0/2857925117 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f8031600 con 0x7f5900143f30 2024-09-19T07:44:04.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.236+0000 7f58fe7fc640 1 -- 172.21.15.16:0/2857925117 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f80319d0 con 0x7f5900143f30 2024-09-19T07:44:04.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.237+0000 7f5905c4c640 1 -- 172.21.15.16:0/2857925117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5900143f30 msgr2=0x7f59000d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:04.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.237+0000 7f5905c4c640 1 --2- 172.21.15.16:0/2857925117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5900143f30 0x7f59000d1e50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f590006eed0 tx=0x7f58f802a610 comp rx=0 tx=0).stop 2024-09-19T07:44:04.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.237+0000 7f5905c4c640 1 -- 172.21.15.16:0/2857925117 shutdown_connections 2024-09-19T07:44:04.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.237+0000 7f5905c4c640 1 --2- 172.21.15.16:0/2857925117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5900143f30 0x7f59000d1e50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.237+0000 7f5905c4c640 1 --2- 172.21.15.16:0/2857925117 >> v2:172.21.15.146:3300/0 conn(0x7f590006d430 0x7f59001439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.237+0000 7f5905c4c640 1 -- 172.21.15.16:0/2857925117 >> 172.21.15.16:0/2857925117 conn(0x7f59000b3ed0 msgr2=0x7f59000b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:04.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.237+0000 7f5905c4c640 1 -- 172.21.15.16:0/2857925117 shutdown_connections 2024-09-19T07:44:04.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.237+0000 7f5905c4c640 1 -- 172.21.15.16:0/2857925117 wait complete. 2024-09-19T07:44:04.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.238+0000 7f5905c4c640 1 Processor -- start 2024-09-19T07:44:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.238+0000 7f5905c4c640 1 -- start start 2024-09-19T07:44:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.239+0000 7f5905c4c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f590006d430 0x7f5900134a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.239+0000 7f5905c4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59000c8d20 0x7f59000b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.239+0000 7f5905c4c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5900143f30 0x7f59000b7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.239+0000 7f5905c4c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5900141e90 con 0x7f59000c8d20 2024-09-19T07:44:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.239+0000 7f5905c4c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5900141bb0 con 0x7f5900143f30 2024-09-19T07:44:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.239+0000 7f5905c4c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5900144380 con 0x7f590006d430 2024-09-19T07:44:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.239+0000 7f58feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59000c8d20 0x7f59000b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.239+0000 7f58feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59000c8d20 0x7f59000b6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51668/0 (socket says 172.21.15.16:51668) 2024-09-19T07:44:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.239+0000 7f58feffd640 1 -- 172.21.15.16:0/609906426 learned_addr learned my addr 172.21.15.16:0/609906426 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.239+0000 7f58fffff640 1 --2- 172.21.15.16:0/609906426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5900143f30 0x7f59000b7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.239+0000 7f58ff7fe640 1 --2- 172.21.15.16:0/609906426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f590006d430 0x7f5900134a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.239+0000 7f58feffd640 1 -- 172.21.15.16:0/609906426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f590006d430 msgr2=0x7f5900134a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:04.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.239+0000 7f58feffd640 1 --2- 172.21.15.16:0/609906426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f590006d430 0x7f5900134a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.239+0000 7f58feffd640 1 -- 172.21.15.16:0/609906426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5900143f30 msgr2=0x7f59000b7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:04.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.239+0000 7f58feffd640 1 --2- 172.21.15.16:0/609906426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5900143f30 0x7f59000b7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.239+0000 7f58feffd640 1 -- 172.21.15.16:0/609906426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59000ccf00 con 0x7f59000c8d20 2024-09-19T07:44:04.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.240+0000 7f58feffd640 1 --2- 172.21.15.16:0/609906426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59000c8d20 0x7f59000b6fd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f58f400d5d0 tx=0x7f58f400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:04.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.240+0000 7f58e7fff640 1 -- 172.21.15.16:0/609906426 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4004240 con 0x7f59000c8d20 2024-09-19T07:44:04.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.240+0000 7f58e7fff640 1 -- 172.21.15.16:0/609906426 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f40043e0 con 0x7f59000c8d20 2024-09-19T07:44:04.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.240+0000 7f5905c4c640 1 -- 172.21.15.16:0/609906426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59000cd1e0 con 0x7f59000c8d20 2024-09-19T07:44:04.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.240+0000 7f58e7fff640 1 -- 172.21.15.16:0/609906426 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4003830 con 0x7f59000c8d20 2024-09-19T07:44:04.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.240+0000 7f5905c4c640 1 -- 172.21.15.16:0/609906426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59000cd7a0 con 0x7f59000c8d20 2024-09-19T07:44:04.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.241+0000 7f5905c4c640 1 -- 172.21.15.16:0/609906426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59001439f0 con 0x7f59000c8d20 2024-09-19T07:44:04.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.241+0000 7f58e7fff640 1 -- 172.21.15.16:0/609906426 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f58f40039d0 con 0x7f59000c8d20 2024-09-19T07:44:04.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.242+0000 7f58e7fff640 1 --2- 172.21.15.16:0/609906426 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58ec06c2b0 0x7f58ec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:04.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.242+0000 7f58e7fff640 1 -- 172.21.15.16:0/609906426 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f58f408b860 con 0x7f59000c8d20 2024-09-19T07:44:04.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.242+0000 7f58ff7fe640 1 --2- 172.21.15.16:0/609906426 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58ec06c2b0 0x7f58ec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:04.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.243+0000 7f58ff7fe640 1 --2- 172.21.15.16:0/609906426 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58ec06c2b0 0x7f58ec06e770 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f58f00016e0 tx=0x7f58f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:04.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.246+0000 7f58e7fff640 1 -- 172.21.15.16:0/609906426 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f58f40596c0 con 0x7f59000c8d20 2024-09-19T07:44:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.460+0000 7f5905c4c640 1 -- 172.21.15.16:0/609906426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59000690c0 con 0x7f59000c8d20 2024-09-19T07:44:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.460+0000 7f58e7fff640 1 -- 172.21.15.16:0/609906426 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5229 (secure 0 0 0) 0x7f58f405d9c0 con 0x7f59000c8d20 2024-09-19T07:44:04.461 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:44:04.461 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-19T07:44:04:108666+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4655,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/3238551814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3238551814},{"type":"v1","addr":"172.21.15.146:6836","nonce":3238551814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3450384245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3450384245},{"type":"v1","addr":"172.21.15.146:6837","nonce":3450384245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4944,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3663416804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3663416804},{"type":"v1","addr":"172.21.15.16:6834","nonce":3663416804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:44:00.970677+0000","modified":"2024-09-19T07:44:03.455740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4947":{"gid":4947,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2028210099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2028210099},{"type":"v1","addr":"172.21.15.16:6835","nonce":2028210099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4947,"qdb_cluster":[4947]},"id":2}]} 2024-09-19T07:44:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-19T07:44:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.462+0000 7f5905c4c640 1 -- 172.21.15.16:0/609906426 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58ec06c2b0 msgr2=0x7f58ec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.462+0000 7f5905c4c640 1 --2- 172.21.15.16:0/609906426 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58ec06c2b0 0x7f58ec06e770 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f58f00016e0 tx=0x7f58f0009040 comp rx=0 tx=0).stop 2024-09-19T07:44:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.462+0000 7f5905c4c640 1 -- 172.21.15.16:0/609906426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59000c8d20 msgr2=0x7f59000b6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.462+0000 7f5905c4c640 1 --2- 172.21.15.16:0/609906426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59000c8d20 0x7f59000b6fd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f58f400d5d0 tx=0x7f58f400da90 comp rx=0 tx=0).stop 2024-09-19T07:44:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.462+0000 7f5905c4c640 1 -- 172.21.15.16:0/609906426 shutdown_connections 2024-09-19T07:44:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.462+0000 7f5905c4c640 1 --2- 172.21.15.16:0/609906426 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58ec06c2b0 0x7f58ec06e770 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.462+0000 7f5905c4c640 1 --2- 172.21.15.16:0/609906426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5900143f30 0x7f59000b7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.462+0000 7f5905c4c640 1 --2- 172.21.15.16:0/609906426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59000c8d20 0x7f59000b6fd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.462+0000 7f5905c4c640 1 --2- 172.21.15.16:0/609906426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f590006d430 0x7f5900134a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.462+0000 7f5905c4c640 1 -- 172.21.15.16:0/609906426 >> 172.21.15.16:0/609906426 conn(0x7f59000b3ed0 msgr2=0x7f59000c7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.462+0000 7f5905c4c640 1 -- 172.21.15.16:0/609906426 shutdown_connections 2024-09-19T07:44:04.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.462+0000 7f5905c4c640 1 -- 172.21.15.16:0/609906426 wait complete. 2024-09-19T07:44:04.473 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:44:04.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.575+0000 7fb60bf11640 1 Processor -- start 2024-09-19T07:44:04.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.575+0000 7fb60bf11640 1 -- start start 2024-09-19T07:44:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.576+0000 7fb60bf11640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb60415f0d0 0x7fb60415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.576+0000 7fb60bf11640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6040d0070 0x7fb6040d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.576+0000 7fb60bf11640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6040cf2b0 con 0x7fb6040d1940 2024-09-19T07:44:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.576+0000 7fb60bf11640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb6040cf590 con 0x7fb6040d0070 2024-09-19T07:44:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.576+0000 7fb60bf11640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb6040cf870 con 0x7fb60415f0d0 2024-09-19T07:44:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.576+0000 7fb6094ac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6040d0070 0x7fb6040d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.576+0000 7fb60a4ae640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb6040d1940 0x7fb6040d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44464/0 (socket says 172.21.15.16:44464) 2024-09-19T07:44:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.576+0000 7fb60a4ae640 1 -- 172.21.15.16:0/2618666065 learned_addr learned my addr 172.21.15.16:0/2618666065 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:44:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.576+0000 7fb609cad640 1 --2- 172.21.15.16:0/2618666065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb60415f0d0 0x7fb60415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.576+0000 7fb6094ac640 1 -- 172.21.15.16:0/2618666065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb60415f0d0 msgr2=0x7fb60415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.576+0000 7fb6094ac640 1 --2- 172.21.15.16:0/2618666065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb60415f0d0 0x7fb60415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.576+0000 7fb6094ac640 1 -- 172.21.15.16:0/2618666065 >> v1:172.21.15.16:6789/0 conn(0x7fb6040d1940 legacy=0x7fb6040d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.576+0000 7fb6094ac640 1 -- 172.21.15.16:0/2618666065 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6040cfb50 con 0x7fb6040d0070 2024-09-19T07:44:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.576+0000 7fb608cab640 1 -- 172.21.15.16:0/2618666065 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 48661611 0 0) 0x7fb6040cf2b0 con 0x7fb6040d1940 2024-09-19T07:44:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.577+0000 7fb6094ac640 1 --2- 172.21.15.16:0/2618666065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6040d0070 0x7fb6040d1400 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40097a0 tx=0x7fb5f402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f647d04bf1d07e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.577+0000 7fb608cab640 1 -- 172.21.15.16:0/2618666065 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4037070 con 0x7fb6040d0070 2024-09-19T07:44:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.577+0000 7fb608cab640 1 -- 172.21.15.16:0/2618666065 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f402ae50 con 0x7fb6040d0070 2024-09-19T07:44:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.577+0000 7fb608cab640 1 -- 172.21.15.16:0/2618666065 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4032660 con 0x7fb6040d0070 2024-09-19T07:44:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.578+0000 7fb60bf11640 1 -- 172.21.15.16:0/2618666065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6040d0070 msgr2=0x7fb6040d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.578+0000 7fb60bf11640 1 --2- 172.21.15.16:0/2618666065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6040d0070 0x7fb6040d1400 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40097a0 tx=0x7fb5f402a100 comp rx=0 tx=0).stop 2024-09-19T07:44:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.578+0000 7fb60bf11640 1 -- 172.21.15.16:0/2618666065 shutdown_connections 2024-09-19T07:44:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.578+0000 7fb60bf11640 1 --2- 172.21.15.16:0/2618666065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6040d0070 0x7fb6040d1400 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.578+0000 7fb60bf11640 1 --2- 172.21.15.16:0/2618666065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb60415f0d0 0x7fb60415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.578+0000 7fb60bf11640 1 -- 172.21.15.16:0/2618666065 >> 172.21.15.16:0/2618666065 conn(0x7fb60409d760 msgr2=0x7fb60408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.578+0000 7fb60bf11640 1 -- 172.21.15.16:0/2618666065 shutdown_connections 2024-09-19T07:44:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.578+0000 7fb60bf11640 1 -- 172.21.15.16:0/2618666065 wait complete. 2024-09-19T07:44:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.579+0000 7fb60bf11640 1 Processor -- start 2024-09-19T07:44:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.579+0000 7fb60bf11640 1 -- start start 2024-09-19T07:44:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb60bf11640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6040d0070 0x7fb60415ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb60bf11640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6040d1940 0x7fb6040ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb60bf11640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb60415f0d0 0x7fb604160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb60bf11640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6040d49c0 con 0x7fb60415f0d0 2024-09-19T07:44:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb60bf11640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb6040d4840 con 0x7fb6040d1940 2024-09-19T07:44:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb60bf11640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb6040d4b40 con 0x7fb6040d0070 2024-09-19T07:44:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb60a4ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb60415f0d0 0x7fb604160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb60a4ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb60415f0d0 0x7fb604160b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51698/0 (socket says 172.21.15.16:51698) 2024-09-19T07:44:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb60a4ae640 1 -- 172.21.15.16:0/3061614387 learned_addr learned my addr 172.21.15.16:0/3061614387 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb60a4ae640 1 -- 172.21.15.16:0/3061614387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6040d0070 msgr2=0x7fb60415ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb60a4ae640 1 --2- 172.21.15.16:0/3061614387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6040d0070 0x7fb60415ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb60a4ae640 1 -- 172.21.15.16:0/3061614387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6040d1940 msgr2=0x7fb6040ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb60a4ae640 1 --2- 172.21.15.16:0/3061614387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6040d1940 0x7fb6040ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb60a4ae640 1 -- 172.21.15.16:0/3061614387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6040cd700 con 0x7fb60415f0d0 2024-09-19T07:44:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb60a4ae640 1 --2- 172.21.15.16:0/3061614387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb60415f0d0 0x7fb604160b20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00c7b0 tx=0x7fb5fc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb5fa7fc640 1 -- 172.21.15.16:0/3061614387 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc018070 con 0x7fb60415f0d0 2024-09-19T07:44:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb5fa7fc640 1 -- 172.21.15.16:0/3061614387 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5fc00a640 con 0x7fb60415f0d0 2024-09-19T07:44:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb60bf11640 1 -- 172.21.15.16:0/3061614387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6040cd9e0 con 0x7fb60415f0d0 2024-09-19T07:44:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb5fa7fc640 1 -- 172.21.15.16:0/3061614387 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc00aa70 con 0x7fb60415f0d0 2024-09-19T07:44:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.580+0000 7fb60bf11640 1 -- 172.21.15.16:0/3061614387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6040cdfa0 con 0x7fb60415f0d0 2024-09-19T07:44:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.581+0000 7fb60bf11640 1 -- 172.21.15.16:0/3061614387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6040d1400 con 0x7fb60415f0d0 2024-09-19T07:44:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.582+0000 7fb5fa7fc640 1 -- 172.21.15.16:0/3061614387 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb5fc01b070 con 0x7fb60415f0d0 2024-09-19T07:44:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.583+0000 7fb5fa7fc640 1 --2- 172.21.15.16:0/3061614387 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb5e806c2b0 0x7fb5e806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.583+0000 7fb5fa7fc640 1 -- 172.21.15.16:0/3061614387 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fb5fc08acb0 con 0x7fb60415f0d0 2024-09-19T07:44:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.583+0000 7fb609cad640 1 --2- 172.21.15.16:0/3061614387 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb5e806c2b0 0x7fb5e806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.583+0000 7fb609cad640 1 --2- 172.21.15.16:0/3061614387 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb5e806c2b0 0x7fb5e806e770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb600006fd0 tx=0x7fb600008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.587+0000 7fb5fa7fc640 1 -- 172.21.15.16:0/3061614387 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb5fc0591f0 con 0x7fb60415f0d0 2024-09-19T07:44:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.762+0000 7fb60bf11640 1 -- 172.21.15.16:0/3061614387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6040756d0 con 0x7fb60415f0d0 2024-09-19T07:44:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.763+0000 7fb5fa7fc640 1 -- 172.21.15.16:0/3061614387 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17026 (secure 0 0 0) 0x7fb5fc058af0 con 0x7fb60415f0d0 2024-09-19T07:44:04.764 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:44:04.764 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":40,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:44:02.619462+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:43:58.882707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-19T07:43:59.960376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:44:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.765+0000 7fb60bf11640 1 -- 172.21.15.16:0/3061614387 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb5e806c2b0 msgr2=0x7fb5e806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.765+0000 7fb60bf11640 1 --2- 172.21.15.16:0/3061614387 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb5e806c2b0 0x7fb5e806e770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb600006fd0 tx=0x7fb600008040 comp rx=0 tx=0).stop 2024-09-19T07:44:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.765+0000 7fb60bf11640 1 -- 172.21.15.16:0/3061614387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb60415f0d0 msgr2=0x7fb604160b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.765+0000 7fb60bf11640 1 --2- 172.21.15.16:0/3061614387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb60415f0d0 0x7fb604160b20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00c7b0 tx=0x7fb5fc00cc70 comp rx=0 tx=0).stop 2024-09-19T07:44:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.765+0000 7fb60bf11640 1 -- 172.21.15.16:0/3061614387 shutdown_connections 2024-09-19T07:44:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.765+0000 7fb60bf11640 1 --2- 172.21.15.16:0/3061614387 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb5e806c2b0 0x7fb5e806e770 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.765+0000 7fb60bf11640 1 --2- 172.21.15.16:0/3061614387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb60415f0d0 0x7fb604160b20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.765+0000 7fb60bf11640 1 --2- 172.21.15.16:0/3061614387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6040d1940 0x7fb6040ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.765+0000 7fb60bf11640 1 --2- 172.21.15.16:0/3061614387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6040d0070 0x7fb60415ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.765+0000 7fb60bf11640 1 -- 172.21.15.16:0/3061614387 >> 172.21.15.16:0/3061614387 conn(0x7fb60409d760 msgr2=0x7fb6040d1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:04.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.765+0000 7fb60bf11640 1 -- 172.21.15.16:0/3061614387 shutdown_connections 2024-09-19T07:44:04.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.765+0000 7fb60bf11640 1 -- 172.21.15.16:0/3061614387 wait complete. 2024-09-19T07:44:04.777 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T07:44:04.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:44:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.878+0000 7f824f9f1640 1 Processor -- start 2024-09-19T07:44:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.878+0000 7f824f9f1640 1 -- start start 2024-09-19T07:44:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.879+0000 7f824f9f1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82480d21f0 0x7f82480d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.879+0000 7f824f9f1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8248141ad0 0x7f8248141f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.879+0000 7f824f9f1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82480d2b10 con 0x7f8248072590 2024-09-19T07:44:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.879+0000 7f824f9f1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f8248141750 con 0x7f8248141ad0 2024-09-19T07:44:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.879+0000 7f824f9f1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f824806d7f0 con 0x7f82480d21f0 2024-09-19T07:44:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.879+0000 7f824df8e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8248141ad0 0x7f8248141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.879+0000 7f824df8e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8248141ad0 0x7f8248141f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51710/0 (socket says 172.21.15.16:51710) 2024-09-19T07:44:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.879+0000 7f824cf8c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8248072590 0x7f824806d0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44470/0 (socket says 172.21.15.16:44470) 2024-09-19T07:44:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.879+0000 7f824df8e640 1 -- 172.21.15.16:0/860044371 learned_addr learned my addr 172.21.15.16:0/860044371 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.880+0000 7f824d78d640 1 --2- 172.21.15.16:0/860044371 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82480d21f0 0x7f82480d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.880+0000 7f824df8e640 1 -- 172.21.15.16:0/860044371 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82480d21f0 msgr2=0x7f82480d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.880+0000 7f824df8e640 1 --2- 172.21.15.16:0/860044371 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82480d21f0 0x7f82480d25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.880+0000 7f824df8e640 1 -- 172.21.15.16:0/860044371 >> v1:172.21.15.16:6789/0 conn(0x7f8248072590 legacy=0x7f824806d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.880+0000 7f824df8e640 1 -- 172.21.15.16:0/860044371 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82481620b0 con 0x7f8248141ad0 2024-09-19T07:44:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.880+0000 7f823ffff640 1 -- 172.21.15.16:0/860044371 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 74905849 0 0) 0x7f82480d2b10 con 0x7f8248072590 2024-09-19T07:44:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.880+0000 7f824df8e640 1 --2- 172.21.15.16:0/860044371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8248141ad0 0x7f8248141f80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8240009e80 tx=0x7f824002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855914b32159e911 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.881+0000 7f823ffff640 1 -- 172.21.15.16:0/860044371 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240036070 con 0x7f8248141ad0 2024-09-19T07:44:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.881+0000 7f823ffff640 1 -- 172.21.15.16:0/860044371 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8240031600 con 0x7f8248141ad0 2024-09-19T07:44:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.881+0000 7f823ffff640 1 -- 172.21.15.16:0/860044371 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82400319d0 con 0x7f8248141ad0 2024-09-19T07:44:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.881+0000 7f824f9f1640 1 -- 172.21.15.16:0/860044371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8248141ad0 msgr2=0x7f8248141f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.881+0000 7f824f9f1640 1 --2- 172.21.15.16:0/860044371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8248141ad0 0x7f8248141f80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8240009e80 tx=0x7f824002a6d0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.881+0000 7f824f9f1640 1 -- 172.21.15.16:0/860044371 shutdown_connections 2024-09-19T07:44:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.881+0000 7f824f9f1640 1 --2- 172.21.15.16:0/860044371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8248141ad0 0x7f8248141f80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.881+0000 7f824f9f1640 1 --2- 172.21.15.16:0/860044371 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82480d21f0 0x7f82480d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.881+0000 7f824f9f1640 1 -- 172.21.15.16:0/860044371 >> 172.21.15.16:0/860044371 conn(0x7f82480b3ed0 msgr2=0x7f82480b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.881+0000 7f824f9f1640 1 -- 172.21.15.16:0/860044371 shutdown_connections 2024-09-19T07:44:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.881+0000 7f824f9f1640 1 -- 172.21.15.16:0/860044371 wait complete. 2024-09-19T07:44:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.882+0000 7f824f9f1640 1 Processor -- start 2024-09-19T07:44:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.882+0000 7f824f9f1640 1 -- start start 2024-09-19T07:44:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.883+0000 7f824f9f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8248072590 0x7f82480b6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.883+0000 7f824f9f1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82480d21f0 0x7f8248140840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.883+0000 7f824f9f1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8248141ad0 0x7f8248140d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.883+0000 7f824f9f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8248162930 con 0x7f8248072590 2024-09-19T07:44:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.883+0000 7f824f9f1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8248162650 con 0x7f82480d21f0 2024-09-19T07:44:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.883+0000 7f824f9f1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8248162c10 con 0x7f8248141ad0 2024-09-19T07:44:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.883+0000 7f824d78d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8248072590 0x7f82480b6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.883+0000 7f824d78d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8248072590 0x7f82480b6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51714/0 (socket says 172.21.15.16:51714) 2024-09-19T07:44:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.883+0000 7f824d78d640 1 -- 172.21.15.16:0/1160218950 learned_addr learned my addr 172.21.15.16:0/1160218950 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.883+0000 7f824cf8c640 1 --2- 172.21.15.16:0/1160218950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82480d21f0 0x7f8248140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.883+0000 7f824df8e640 1 --2- 172.21.15.16:0/1160218950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8248141ad0 0x7f8248140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.884+0000 7f824d78d640 1 -- 172.21.15.16:0/1160218950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8248141ad0 msgr2=0x7f8248140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.884+0000 7f824d78d640 1 --2- 172.21.15.16:0/1160218950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8248141ad0 0x7f8248140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.884+0000 7f824d78d640 1 -- 172.21.15.16:0/1160218950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82480d21f0 msgr2=0x7f8248140840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.884+0000 7f824d78d640 1 --2- 172.21.15.16:0/1160218950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82480d21f0 0x7f8248140840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.884+0000 7f824d78d640 1 -- 172.21.15.16:0/1160218950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8248159cc0 con 0x7f8248072590 2024-09-19T07:44:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.884+0000 7f824d78d640 1 --2- 172.21.15.16:0/1160218950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8248072590 0x7f82480b6a90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f824400d800 tx=0x7f824400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.884+0000 7f823dffb640 1 -- 172.21.15.16:0/1160218950 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824400b6c0 con 0x7f8248072590 2024-09-19T07:44:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.884+0000 7f823dffb640 1 -- 172.21.15.16:0/1160218950 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8244019070 con 0x7f8248072590 2024-09-19T07:44:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.884+0000 7f824f9f1640 1 -- 172.21.15.16:0/1160218950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82481517b0 con 0x7f8248072590 2024-09-19T07:44:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.884+0000 7f823dffb640 1 -- 172.21.15.16:0/1160218950 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824400ba80 con 0x7f8248072590 2024-09-19T07:44:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.884+0000 7f824f9f1640 1 -- 172.21.15.16:0/1160218950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8248151d70 con 0x7f8248072590 2024-09-19T07:44:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.885+0000 7f824f9f1640 1 -- 172.21.15.16:0/1160218950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82481521b0 con 0x7f8248072590 2024-09-19T07:44:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.886+0000 7f823dffb640 1 -- 172.21.15.16:0/1160218950 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f824400bc20 con 0x7f8248072590 2024-09-19T07:44:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.886+0000 7f823dffb640 1 --2- 172.21.15.16:0/1160218950 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f822c06c2e0 0x7f822c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.886+0000 7f823dffb640 1 -- 172.21.15.16:0/1160218950 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f824408b9a0 con 0x7f8248072590 2024-09-19T07:44:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.886+0000 7f824cf8c640 1 --2- 172.21.15.16:0/1160218950 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f822c06c2e0 0x7f822c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.887+0000 7f824cf8c640 1 --2- 172.21.15.16:0/1160218950 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f822c06c2e0 0x7f822c06e7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f82481f78f0 tx=0x7f8238007b70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:04.890+0000 7f823dffb640 1 -- 172.21.15.16:0/1160218950 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f824401c360 con 0x7f8248072590 2024-09-19T07:44:05.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.061+0000 7f824f9f1640 1 -- 172.21.15.16:0/1160218950 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8248158c30 con 0x7f822c06c2e0 2024-09-19T07:44:05.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.067+0000 7f823dffb640 1 -- 172.21.15.16:0/1160218950 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242129 (secure 0 0 0) 0x7f8248158c30 con 0x7f822c06c2e0 2024-09-19T07:44:05.071 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:44:05.072 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:44:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.070+0000 7f824f9f1640 1 -- 172.21.15.16:0/1160218950 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f822c06c2e0 msgr2=0x7f822c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.070+0000 7f824f9f1640 1 --2- 172.21.15.16:0/1160218950 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f822c06c2e0 0x7f822c06e7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f82481f78f0 tx=0x7f8238007b70 comp rx=0 tx=0).stop 2024-09-19T07:44:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.070+0000 7f824f9f1640 1 -- 172.21.15.16:0/1160218950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8248072590 msgr2=0x7f82480b6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.070+0000 7f824f9f1640 1 --2- 172.21.15.16:0/1160218950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8248072590 0x7f82480b6a90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f824400d800 tx=0x7f824400dcc0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.070+0000 7f824f9f1640 1 -- 172.21.15.16:0/1160218950 shutdown_connections 2024-09-19T07:44:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.070+0000 7f824f9f1640 1 --2- 172.21.15.16:0/1160218950 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f822c06c2e0 0x7f822c06e7a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.070+0000 7f824f9f1640 1 --2- 172.21.15.16:0/1160218950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8248141ad0 0x7f8248140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.070+0000 7f824f9f1640 1 --2- 172.21.15.16:0/1160218950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82480d21f0 0x7f8248140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.070+0000 7f824f9f1640 1 --2- 172.21.15.16:0/1160218950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8248072590 0x7f82480b6a90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.070+0000 7f824f9f1640 1 -- 172.21.15.16:0/1160218950 >> 172.21.15.16:0/1160218950 conn(0x7f82480b3ed0 msgr2=0x7f82480cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.070+0000 7f824f9f1640 1 -- 172.21.15.16:0/1160218950 shutdown_connections 2024-09-19T07:44:05.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.070+0000 7f824f9f1640 1 -- 172.21.15.16:0/1160218950 wait complete. 2024-09-19T07:44:05.081 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-09-19T07:44:04.472124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592014,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":237364,"kb_used_data":4456,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737356,"statfs":{"total":773094113280,"available":772851052544,"internally_reserved":0,"allocated":4562944,"data_stored":2598054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.865307"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633184+0000","last_change":"2024-09-19T07:44:01.635139+0000","last_active":"2024-09-19T07:44:02.633184+0000","last_peered":"2024-09-19T07:44:02.633184+0000","last_clean":"2024-09-19T07:44:02.633184+0000","last_became_active":"2024-09-19T07:44:01.634842+0000","last_became_peered":"2024-09-19T07:44:01.634842+0000","last_unstale":"2024-09-19T07:44:02.633184+0000","last_undegraded":"2024-09-19T07:44:02.633184+0000","last_fullsized":"2024-09-19T07:44:02.633184+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:07.759986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633030+0000","last_change":"2024-09-19T07:44:01.629931+0000","last_active":"2024-09-19T07:44:02.633030+0000","last_peered":"2024-09-19T07:44:02.633030+0000","last_clean":"2024-09-19T07:44:02.633030+0000","last_became_active":"2024-09-19T07:44:01.629634+0000","last_became_peered":"2024-09-19T07:44:01.629634+0000","last_unstale":"2024-09-19T07:44:02.633030+0000","last_undegraded":"2024-09-19T07:44:02.633030+0000","last_fullsized":"2024-09-19T07:44:02.633030+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:00.266417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.619008+0000","last_change":"2024-09-19T07:44:00.619008+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.619008+0000","last_undegraded":"2024-09-19T07:44:00.619008+0000","last_fullsized":"2024-09-19T07:44:00.619008+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.619651+0000","last_change":"2024-09-19T07:44:00.619651+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.619651+0000","last_undegraded":"2024-09-19T07:44:00.619651+0000","last_fullsized":"2024-09-19T07:44:00.619651+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880437+0000","last_change":"2024-09-19T07:44:01.635795+0000","last_active":"2024-09-19T07:44:02.880437+0000","last_peered":"2024-09-19T07:44:02.880437+0000","last_clean":"2024-09-19T07:44:02.880437+0000","last_became_active":"2024-09-19T07:44:01.634728+0000","last_became_peered":"2024-09-19T07:44:01.634728+0000","last_unstale":"2024-09-19T07:44:02.880437+0000","last_undegraded":"2024-09-19T07:44:02.880437+0000","last_fullsized":"2024-09-19T07:44:02.880437+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:20.583756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632612+0000","last_change":"2024-09-19T07:44:01.621511+0000","last_active":"2024-09-19T07:44:02.632612+0000","last_peered":"2024-09-19T07:44:02.632612+0000","last_clean":"2024-09-19T07:44:02.632612+0000","last_became_active":"2024-09-19T07:44:01.620511+0000","last_became_peered":"2024-09-19T07:44:01.620511+0000","last_unstale":"2024-09-19T07:44:02.632612+0000","last_undegraded":"2024-09-19T07:44:02.632612+0000","last_fullsized":"2024-09-19T07:44:02.632612+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:57.716510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.620031+0000","last_change":"2024-09-19T07:44:00.620031+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.620031+0000","last_undegraded":"2024-09-19T07:44:00.620031+0000","last_fullsized":"2024-09-19T07:44:00.620031+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.618996+0000","last_change":"2024-09-19T07:44:00.618996+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.618996+0000","last_undegraded":"2024-09-19T07:44:00.618996+0000","last_fullsized":"2024-09-19T07:44:00.618996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879547+0000","last_change":"2024-09-19T07:44:01.632403+0000","last_active":"2024-09-19T07:44:02.879547+0000","last_peered":"2024-09-19T07:44:02.879547+0000","last_clean":"2024-09-19T07:44:02.879547+0000","last_became_active":"2024-09-19T07:44:01.632105+0000","last_became_peered":"2024-09-19T07:44:01.632105+0000","last_unstale":"2024-09-19T07:44:02.879547+0000","last_undegraded":"2024-09-19T07:44:02.879547+0000","last_fullsized":"2024-09-19T07:44:02.879547+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:22:00.022599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880311+0000","last_change":"2024-09-19T07:44:01.633044+0000","last_active":"2024-09-19T07:44:02.880311+0000","last_peered":"2024-09-19T07:44:02.880311+0000","last_clean":"2024-09-19T07:44:02.880311+0000","last_became_active":"2024-09-19T07:44:01.632761+0000","last_became_peered":"2024-09-19T07:44:01.632761+0000","last_unstale":"2024-09-19T07:44:02.880311+0000","last_undegraded":"2024-09-19T07:44:02.880311+0000","last_fullsized":"2024-09-19T07:44:02.880311+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:10.758189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879930+0000","last_change":"2024-09-19T07:44:01.630912+0000","last_active":"2024-09-19T07:44:02.879930+0000","last_peered":"2024-09-19T07:44:02.879930+0000","last_clean":"2024-09-19T07:44:02.879930+0000","last_became_active":"2024-09-19T07:44:01.630625+0000","last_became_peered":"2024-09-19T07:44:01.630625+0000","last_unstale":"2024-09-19T07:44:02.879930+0000","last_undegraded":"2024-09-19T07:44:02.879930+0000","last_fullsized":"2024-09-19T07:44:02.879930+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:16.252351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880122+0000","last_change":"2024-09-19T07:44:00.624268+0000","last_active":"2024-09-19T07:44:02.880122+0000","last_peered":"2024-09-19T07:44:02.880122+0000","last_clean":"2024-09-19T07:44:02.880122+0000","last_became_active":"2024-09-19T07:44:00.623980+0000","last_became_peered":"2024-09-19T07:44:00.623980+0000","last_unstale":"2024-09-19T07:44:02.880122+0000","last_undegraded":"2024-09-19T07:44:02.880122+0000","last_fullsized":"2024-09-19T07:44:02.880122+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:42.166793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633166+0000","last_change":"2024-09-19T07:44:01.642857+0000","last_active":"2024-09-19T07:44:02.633166+0000","last_peered":"2024-09-19T07:44:02.633166+0000","last_clean":"2024-09-19T07:44:02.633166+0000","last_became_active":"2024-09-19T07:44:01.642470+0000","last_became_peered":"2024-09-19T07:44:01.642470+0000","last_unstale":"2024-09-19T07:44:02.633166+0000","last_undegraded":"2024-09-19T07:44:02.633166+0000","last_fullsized":"2024-09-19T07:44:02.633166+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:32.560026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879466+0000","last_change":"2024-09-19T07:44:00.631253+0000","last_active":"2024-09-19T07:44:02.879466+0000","last_peered":"2024-09-19T07:44:02.879466+0000","last_clean":"2024-09-19T07:44:02.879466+0000","last_became_active":"2024-09-19T07:44:00.630967+0000","last_became_peered":"2024-09-19T07:44:00.630967+0000","last_unstale":"2024-09-19T07:44:02.879466+0000","last_undegraded":"2024-09-19T07:44:02.879466+0000","last_fullsized":"2024-09-19T07:44:02.879466+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:07.746484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632826+0000","last_change":"2024-09-19T07:44:01.637345+0000","last_active":"2024-09-19T07:44:02.632826+0000","last_peered":"2024-09-19T07:44:02.632826+0000","last_clean":"2024-09-19T07:44:02.632826+0000","last_became_active":"2024-09-19T07:44:01.633385+0000","last_became_peered":"2024-09-19T07:44:01.633385+0000","last_unstale":"2024-09-19T07:44:02.632826+0000","last_undegraded":"2024-09-19T07:44:02.632826+0000","last_fullsized":"2024-09-19T07:44:02.632826+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:26.999322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.613312+0000","last_change":"2024-09-19T07:44:00.613312+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.613312+0000","last_undegraded":"2024-09-19T07:44:00.613312+0000","last_fullsized":"2024-09-19T07:44:00.613312+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633235+0000","last_change":"2024-09-19T07:44:00.621900+0000","last_active":"2024-09-19T07:44:02.633235+0000","last_peered":"2024-09-19T07:44:02.633235+0000","last_clean":"2024-09-19T07:44:02.633235+0000","last_became_active":"2024-09-19T07:44:00.621596+0000","last_became_peered":"2024-09-19T07:44:00.621596+0000","last_unstale":"2024-09-19T07:44:02.633235+0000","last_undegraded":"2024-09-19T07:44:02.633235+0000","last_fullsized":"2024-09-19T07:44:02.633235+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:19.578605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632895+0000","last_change":"2024-09-19T07:44:00.629007+0000","last_active":"2024-09-19T07:44:02.632895+0000","last_peered":"2024-09-19T07:44:02.632895+0000","last_clean":"2024-09-19T07:44:02.632895+0000","last_became_active":"2024-09-19T07:44:00.628690+0000","last_became_peered":"2024-09-19T07:44:00.628690+0000","last_unstale":"2024-09-19T07:44:02.632895+0000","last_undegraded":"2024-09-19T07:44:02.632895+0000","last_fullsized":"2024-09-19T07:44:02.632895+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:15.247200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632967+0000","last_change":"2024-09-19T07:44:00.621833+0000","last_active":"2024-09-19T07:44:02.632967+0000","last_peered":"2024-09-19T07:44:02.632967+0000","last_clean":"2024-09-19T07:44:02.632967+0000","last_became_active":"2024-09-19T07:44:00.621525+0000","last_became_peered":"2024-09-19T07:44:00.621525+0000","last_unstale":"2024-09-19T07:44:02.632967+0000","last_undegraded":"2024-09-19T07:44:02.632967+0000","last_fullsized":"2024-09-19T07:44:02.632967+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:09.753037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632713+0000","last_change":"2024-09-19T07:44:01.635000+0000","last_active":"2024-09-19T07:44:02.632713+0000","last_peered":"2024-09-19T07:44:02.632713+0000","last_clean":"2024-09-19T07:44:02.632713+0000","last_became_active":"2024-09-19T07:44:01.634733+0000","last_became_peered":"2024-09-19T07:44:01.634733+0000","last_unstale":"2024-09-19T07:44:02.632713+0000","last_undegraded":"2024-09-19T07:44:02.632713+0000","last_fullsized":"2024-09-19T07:44:02.632713+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:26.448577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880363+0000","last_change":"2024-09-19T07:44:00.630777+0000","last_active":"2024-09-19T07:44:02.880363+0000","last_peered":"2024-09-19T07:44:02.880363+0000","last_clean":"2024-09-19T07:44:02.880363+0000","last_became_active":"2024-09-19T07:44:00.630494+0000","last_became_peered":"2024-09-19T07:44:00.630494+0000","last_unstale":"2024-09-19T07:44:02.880363+0000","last_undegraded":"2024-09-19T07:44:02.880363+0000","last_fullsized":"2024-09-19T07:44:02.880363+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:06:32.865629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632815+0000","last_change":"2024-09-19T07:44:01.637576+0000","last_active":"2024-09-19T07:44:02.632815+0000","last_peered":"2024-09-19T07:44:02.632815+0000","last_clean":"2024-09-19T07:44:02.632815+0000","last_became_active":"2024-09-19T07:44:01.637026+0000","last_became_peered":"2024-09-19T07:44:01.637026+0000","last_unstale":"2024-09-19T07:44:02.632815+0000","last_undegraded":"2024-09-19T07:44:02.632815+0000","last_fullsized":"2024-09-19T07:44:02.632815+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:34.910477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633057+0000","last_change":"2024-09-19T07:44:01.643372+0000","last_active":"2024-09-19T07:44:02.633057+0000","last_peered":"2024-09-19T07:44:02.633057+0000","last_clean":"2024-09-19T07:44:02.633057+0000","last_became_active":"2024-09-19T07:44:01.642744+0000","last_became_peered":"2024-09-19T07:44:01.642744+0000","last_unstale":"2024-09-19T07:44:02.633057+0000","last_undegraded":"2024-09-19T07:44:02.633057+0000","last_fullsized":"2024-09-19T07:44:02.633057+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:34.596548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.632871+0000","last_change":"2024-09-19T07:44:00.631635+0000","last_active":"2024-09-19T07:44:00.632871+0000","last_peered":"2024-09-19T07:44:00.632871+0000","last_clean":"2024-09-19T07:44:00.632871+0000","last_became_active":"2024-09-19T07:44:00.631343+0000","last_became_peered":"2024-09-19T07:44:00.631343+0000","last_unstale":"2024-09-19T07:44:00.632871+0000","last_undegraded":"2024-09-19T07:44:00.632871+0000","last_fullsized":"2024-09-19T07:44:00.632871+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:33.000415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.613328+0000","last_change":"2024-09-19T07:44:00.613328+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.613328+0000","last_undegraded":"2024-09-19T07:44:00.613328+0000","last_fullsized":"2024-09-19T07:44:00.613328+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879303+0000","last_change":"2024-09-19T07:44:00.625009+0000","last_active":"2024-09-19T07:44:02.879303+0000","last_peered":"2024-09-19T07:44:02.879303+0000","last_clean":"2024-09-19T07:44:02.879303+0000","last_became_active":"2024-09-19T07:44:00.624610+0000","last_became_peered":"2024-09-19T07:44:00.624610+0000","last_unstale":"2024-09-19T07:44:02.879303+0000","last_undegraded":"2024-09-19T07:44:02.879303+0000","last_fullsized":"2024-09-19T07:44:02.879303+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:58.958404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.625246+0000","last_change":"2024-09-19T07:44:00.623041+0000","last_active":"2024-09-19T07:44:00.625246+0000","last_peered":"2024-09-19T07:44:00.625246+0000","last_clean":"2024-09-19T07:44:00.625246+0000","last_became_active":"2024-09-19T07:44:00.622748+0000","last_became_peered":"2024-09-19T07:44:00.622748+0000","last_unstale":"2024-09-19T07:44:00.625246+0000","last_undegraded":"2024-09-19T07:44:00.625246+0000","last_fullsized":"2024-09-19T07:44:00.625246+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:41.495537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.628081+0000","last_change":"2024-09-19T07:44:00.627049+0000","last_active":"2024-09-19T07:44:00.628081+0000","last_peered":"2024-09-19T07:44:00.628081+0000","last_clean":"2024-09-19T07:44:00.628081+0000","last_became_active":"2024-09-19T07:44:00.626768+0000","last_became_peered":"2024-09-19T07:44:00.626768+0000","last_unstale":"2024-09-19T07:44:00.628081+0000","last_undegraded":"2024-09-19T07:44:00.628081+0000","last_fullsized":"2024-09-19T07:44:00.628081+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:18.929914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632969+0000","last_change":"2024-09-19T07:44:01.629794+0000","last_active":"2024-09-19T07:44:02.632969+0000","last_peered":"2024-09-19T07:44:02.632969+0000","last_clean":"2024-09-19T07:44:02.632969+0000","last_became_active":"2024-09-19T07:44:01.629453+0000","last_became_peered":"2024-09-19T07:44:01.629453+0000","last_unstale":"2024-09-19T07:44:02.632969+0000","last_undegraded":"2024-09-19T07:44:02.632969+0000","last_fullsized":"2024-09-19T07:44:02.632969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:34.270347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.624497+0000","last_change":"2024-09-19T07:44:00.623038+0000","last_active":"2024-09-19T07:44:00.624497+0000","last_peered":"2024-09-19T07:44:00.624497+0000","last_clean":"2024-09-19T07:44:00.624497+0000","last_became_active":"2024-09-19T07:44:00.622704+0000","last_became_peered":"2024-09-19T07:44:00.622704+0000","last_unstale":"2024-09-19T07:44:00.624497+0000","last_undegraded":"2024-09-19T07:44:00.624497+0000","last_fullsized":"2024-09-19T07:44:00.624497+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:38.159118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879198+0000","last_change":"2024-09-19T07:44:01.633567+0000","last_active":"2024-09-19T07:44:02.879198+0000","last_peered":"2024-09-19T07:44:02.879198+0000","last_clean":"2024-09-19T07:44:02.879198+0000","last_became_active":"2024-09-19T07:44:01.633291+0000","last_became_peered":"2024-09-19T07:44:01.633291+0000","last_unstale":"2024-09-19T07:44:02.879198+0000","last_undegraded":"2024-09-19T07:44:02.879198+0000","last_fullsized":"2024-09-19T07:44:02.879198+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:18.456608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.629136+0000","last_change":"2024-09-19T07:44:00.627575+0000","last_active":"2024-09-19T07:44:00.629136+0000","last_peered":"2024-09-19T07:44:00.629136+0000","last_clean":"2024-09-19T07:44:00.629136+0000","last_became_active":"2024-09-19T07:44:00.627217+0000","last_became_peered":"2024-09-19T07:44:00.627217+0000","last_unstale":"2024-09-19T07:44:00.629136+0000","last_undegraded":"2024-09-19T07:44:00.629136+0000","last_fullsized":"2024-09-19T07:44:00.629136+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:30.570507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633094+0000","last_change":"2024-09-19T07:44:01.620156+0000","last_active":"2024-09-19T07:44:02.633094+0000","last_peered":"2024-09-19T07:44:02.633094+0000","last_clean":"2024-09-19T07:44:02.633094+0000","last_became_active":"2024-09-19T07:44:01.619829+0000","last_became_peered":"2024-09-19T07:44:01.619829+0000","last_unstale":"2024-09-19T07:44:02.633094+0000","last_undegraded":"2024-09-19T07:44:02.633094+0000","last_fullsized":"2024-09-19T07:44:02.633094+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:17.164807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879371+0000","last_change":"2024-09-19T07:44:01.637663+0000","last_active":"2024-09-19T07:44:02.879371+0000","last_peered":"2024-09-19T07:44:02.879371+0000","last_clean":"2024-09-19T07:44:02.879371+0000","last_became_active":"2024-09-19T07:44:01.637391+0000","last_became_peered":"2024-09-19T07:44:01.637391+0000","last_unstale":"2024-09-19T07:44:02.879371+0000","last_undegraded":"2024-09-19T07:44:02.879371+0000","last_fullsized":"2024-09-19T07:44:02.879371+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:11.328826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632629+0000","last_change":"2024-09-19T07:44:00.622888+0000","last_active":"2024-09-19T07:44:02.632629+0000","last_peered":"2024-09-19T07:44:02.632629+0000","last_clean":"2024-09-19T07:44:02.632629+0000","last_became_active":"2024-09-19T07:44:00.622610+0000","last_became_peered":"2024-09-19T07:44:00.622610+0000","last_unstale":"2024-09-19T07:44:02.632629+0000","last_undegraded":"2024-09-19T07:44:02.632629+0000","last_fullsized":"2024-09-19T07:44:02.632629+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:59.017448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":195,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-19T07:43:56.599936+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:43:56.599936+0000","last_peered":"2024-09-19T07:43:56.599936+0000","last_clean":"2024-09-19T07:43:56.599936+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:43:56.599936+0000","last_undegraded":"2024-09-19T07:43:56.599936+0000","last_fullsized":"2024-09-19T07:43:56.599936+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":44,"acting":44,"num_store_stats":3},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":26,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":999424,"data_stored":759859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":405504,"data_stored":169491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":405504,"data_stored":169491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":169491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":999424,"data_stored":759859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31652,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":471040,"data_stored":197872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":446464,"data_stored":189624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":430080,"data_stored":182367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:44:05.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:44:05.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.224+0000 7f1c45f8d640 1 Processor -- start 2024-09-19T07:44:05.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.224+0000 7f1c45f8d640 1 -- start start 2024-09-19T07:44:05.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.224+0000 7f1c45f8d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c400d4840 0x7f1c4015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:05.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.224+0000 7f1c45f8d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c400d1b10 con 0x7f1c400d0070 2024-09-19T07:44:05.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.224+0000 7f1c45f8d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c400cf2b0 con 0x7f1c4015f0d0 2024-09-19T07:44:05.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.224+0000 7f1c45f8d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1c400cf590 con 0x7f1c400d4840 2024-09-19T07:44:05.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.225+0000 7f1c3effd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1c400d0070 0x7f1c400d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44474/0 (socket says 172.21.15.16:44474) 2024-09-19T07:44:05.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.225+0000 7f1c3effd640 1 -- 172.21.15.16:0/4281620582 learned_addr learned my addr 172.21.15.16:0/4281620582 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:44:05.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.225+0000 7f1c3ffff640 1 --2- 172.21.15.16:0/4281620582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c400d4840 0x7f1c4015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:05.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.225+0000 7f1c3e7fc640 1 -- 172.21.15.16:0/4281620582 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 962862283 0 0) 0x7f1c400d1b10 con 0x7f1c400d0070 2024-09-19T07:44:05.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.225+0000 7f1c3e7fc640 1 -- 172.21.15.16:0/4281620582 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c28003520 con 0x7f1c400d0070 2024-09-19T07:44:05.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.225+0000 7f1c3e7fc640 1 -- 172.21.15.16:0/4281620582 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1279139769 0 0) 0x7f1c400cf2b0 con 0x7f1c4015f0d0 2024-09-19T07:44:05.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.225+0000 7f1c3e7fc640 1 -- 172.21.15.16:0/4281620582 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c400d1b10 con 0x7f1c4015f0d0 2024-09-19T07:44:05.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.225+0000 7f1c3ffff640 1 -- 172.21.15.16:0/4281620582 >> v1:172.21.15.146:6789/0 conn(0x7f1c4015f0d0 legacy=0x7f1c400d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.225+0000 7f1c3ffff640 1 -- 172.21.15.16:0/4281620582 >> v1:172.21.15.16:6789/0 conn(0x7f1c400d0070 legacy=0x7f1c400d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.225+0000 7f1c3ffff640 1 -- 172.21.15.16:0/4281620582 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c400cf870 con 0x7f1c400d4840 2024-09-19T07:44:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.226+0000 7f1c3ffff640 1 --2- 172.21.15.16:0/4281620582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c400d4840 0x7f1c4015f4b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1c38009df0 tx=0x7f1c3802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e20ec67be7da4c36 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.226+0000 7f1c3e7fc640 1 -- 172.21.15.16:0/4281620582 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3802ae60 con 0x7f1c400d4840 2024-09-19T07:44:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.226+0000 7f1c3e7fc640 1 -- 172.21.15.16:0/4281620582 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c38007740 con 0x7f1c400d4840 2024-09-19T07:44:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.226+0000 7f1c3e7fc640 1 -- 172.21.15.16:0/4281620582 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38007b10 con 0x7f1c400d4840 2024-09-19T07:44:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.227+0000 7f1c45f8d640 1 -- 172.21.15.16:0/4281620582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c400d4840 msgr2=0x7f1c4015f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.227+0000 7f1c45f8d640 1 --2- 172.21.15.16:0/4281620582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c400d4840 0x7f1c4015f4b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1c38009df0 tx=0x7f1c3802a680 comp rx=0 tx=0).stop 2024-09-19T07:44:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.227+0000 7f1c45f8d640 1 -- 172.21.15.16:0/4281620582 shutdown_connections 2024-09-19T07:44:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.227+0000 7f1c45f8d640 1 --2- 172.21.15.16:0/4281620582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c400d4840 0x7f1c4015f4b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.227+0000 7f1c45f8d640 1 -- 172.21.15.16:0/4281620582 >> 172.21.15.16:0/4281620582 conn(0x7f1c4009d760 msgr2=0x7f1c4008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:05.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.227+0000 7f1c45f8d640 1 -- 172.21.15.16:0/4281620582 shutdown_connections 2024-09-19T07:44:05.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.227+0000 7f1c45f8d640 1 -- 172.21.15.16:0/4281620582 wait complete. 2024-09-19T07:44:05.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.228+0000 7f1c45f8d640 1 Processor -- start 2024-09-19T07:44:05.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.228+0000 7f1c45f8d640 1 -- start start 2024-09-19T07:44:05.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.229+0000 7f1c45f8d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c400d0070 0x7f1c401545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:05.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.229+0000 7f1c45f8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c400d4840 0x7f1c4015a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:05.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.229+0000 7f1c45f8d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c4015f0d0 0x7f1c400b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:05.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.229+0000 7f1c45f8d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c400b2f70 con 0x7f1c400d4840 2024-09-19T07:44:05.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.229+0000 7f1c45f8d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1c400cef60 con 0x7f1c400d0070 2024-09-19T07:44:05.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.229+0000 7f1c45f8d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1c400b30f0 con 0x7f1c4015f0d0 2024-09-19T07:44:05.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.229+0000 7f1c3effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c400d4840 0x7f1c4015a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.229+0000 7f1c3effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c400d4840 0x7f1c4015a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51720/0 (socket says 172.21.15.16:51720) 2024-09-19T07:44:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.229+0000 7f1c3effd640 1 -- 172.21.15.16:0/1840578726 learned_addr learned my addr 172.21.15.16:0/1840578726 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.229+0000 7f1c3f7fe640 1 --2- 172.21.15.16:0/1840578726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c400d0070 0x7f1c401545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.229+0000 7f1c3effd640 1 -- 172.21.15.16:0/1840578726 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c4015f0d0 msgr2=0x7f1c400b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.229+0000 7f1c3ffff640 1 --2- 172.21.15.16:0/1840578726 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c4015f0d0 0x7f1c400b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.229+0000 7f1c3effd640 1 --2- 172.21.15.16:0/1840578726 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c4015f0d0 0x7f1c400b19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.229+0000 7f1c3effd640 1 -- 172.21.15.16:0/1840578726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c400d0070 msgr2=0x7f1c401545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.229+0000 7f1c3effd640 1 --2- 172.21.15.16:0/1840578726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c400d0070 0x7f1c401545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.229+0000 7f1c3effd640 1 -- 172.21.15.16:0/1840578726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4015f760 con 0x7f1c400d4840 2024-09-19T07:44:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.229+0000 7f1c3f7fe640 1 --2- 172.21.15.16:0/1840578726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c400d0070 0x7f1c401545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:44:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.229+0000 7f1c3effd640 1 --2- 172.21.15.16:0/1840578726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c400d4840 0x7f1c4015a2e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1c3400c500 tx=0x7f1c3400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.230+0000 7f1c27fff640 1 -- 172.21.15.16:0/1840578726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c34017070 con 0x7f1c400d4840 2024-09-19T07:44:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.230+0000 7f1c27fff640 1 -- 172.21.15.16:0/1840578726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c34005ce0 con 0x7f1c400d4840 2024-09-19T07:44:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.230+0000 7f1c27fff640 1 -- 172.21.15.16:0/1840578726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c340072f0 con 0x7f1c400d4840 2024-09-19T07:44:05.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.230+0000 7f1c45f8d640 1 -- 172.21.15.16:0/1840578726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c4015fa00 con 0x7f1c400d4840 2024-09-19T07:44:05.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.230+0000 7f1c45f8d640 1 -- 172.21.15.16:0/1840578726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c4015ffc0 con 0x7f1c400d4840 2024-09-19T07:44:05.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.231+0000 7f1c45f8d640 1 -- 172.21.15.16:0/1840578726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c40068ce0 con 0x7f1c400d4840 2024-09-19T07:44:05.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.231+0000 7f1c27fff640 1 -- 172.21.15.16:0/1840578726 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1c3401a070 con 0x7f1c400d4840 2024-09-19T07:44:05.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.232+0000 7f1c27fff640 1 --2- 172.21.15.16:0/1840578726 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1c2c06c2b0 0x7f1c2c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:05.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.232+0000 7f1c27fff640 1 -- 172.21.15.16:0/1840578726 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f1c3408a900 con 0x7f1c400d4840 2024-09-19T07:44:05.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.232+0000 7f1c3f7fe640 1 --2- 172.21.15.16:0/1840578726 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1c2c06c2b0 0x7f1c2c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:05.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.232+0000 7f1c3f7fe640 1 --2- 172.21.15.16:0/1840578726 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1c2c06c2b0 0x7f1c2c06e770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1c30001610 tx=0x7f1c30009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:05.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.236+0000 7f1c27fff640 1 -- 172.21.15.16:0/1840578726 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1c40068ce0 con 0x7f1c400d4840 2024-09-19T07:44:05.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.410+0000 7f1c45f8d640 1 -- 172.21.15.16:0/1840578726 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1c4015f4b0 con 0x7f1c2c06c2b0 2024-09-19T07:44:05.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.416+0000 7f1c27fff640 1 -- 172.21.15.16:0/1840578726 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242129 (secure 0 0 0) 0x7f1c4015f4b0 con 0x7f1c2c06c2b0 2024-09-19T07:44:05.419 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:44:05.420 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:44:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.419+0000 7f1c45f8d640 1 -- 172.21.15.16:0/1840578726 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1c2c06c2b0 msgr2=0x7f1c2c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.419+0000 7f1c45f8d640 1 --2- 172.21.15.16:0/1840578726 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1c2c06c2b0 0x7f1c2c06e770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1c30001610 tx=0x7f1c30009040 comp rx=0 tx=0).stop 2024-09-19T07:44:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.419+0000 7f1c45f8d640 1 -- 172.21.15.16:0/1840578726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c400d4840 msgr2=0x7f1c4015a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.419+0000 7f1c45f8d640 1 --2- 172.21.15.16:0/1840578726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c400d4840 0x7f1c4015a2e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1c3400c500 tx=0x7f1c3400c9c0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.419+0000 7f1c45f8d640 1 -- 172.21.15.16:0/1840578726 shutdown_connections 2024-09-19T07:44:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.419+0000 7f1c45f8d640 1 --2- 172.21.15.16:0/1840578726 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1c2c06c2b0 0x7f1c2c06e770 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.419+0000 7f1c45f8d640 1 --2- 172.21.15.16:0/1840578726 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c4015f0d0 0x7f1c400b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.419+0000 7f1c45f8d640 1 --2- 172.21.15.16:0/1840578726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c400d4840 0x7f1c4015a2e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.419+0000 7f1c45f8d640 1 --2- 172.21.15.16:0/1840578726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c400d0070 0x7f1c401545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.419+0000 7f1c45f8d640 1 -- 172.21.15.16:0/1840578726 >> 172.21.15.16:0/1840578726 conn(0x7f1c4009d760 msgr2=0x7f1c400b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:05.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.419+0000 7f1c45f8d640 1 -- 172.21.15.16:0/1840578726 shutdown_connections 2024-09-19T07:44:05.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.419+0000 7f1c45f8d640 1 -- 172.21.15.16:0/1840578726 wait complete. 2024-09-19T07:44:05.430 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-09-19T07:44:04.472124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592014,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":237364,"kb_used_data":4456,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737356,"statfs":{"total":773094113280,"available":772851052544,"internally_reserved":0,"allocated":4562944,"data_stored":2598054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.865307"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633184+0000","last_change":"2024-09-19T07:44:01.635139+0000","last_active":"2024-09-19T07:44:02.633184+0000","last_peered":"2024-09-19T07:44:02.633184+0000","last_clean":"2024-09-19T07:44:02.633184+0000","last_became_active":"2024-09-19T07:44:01.634842+0000","last_became_peered":"2024-09-19T07:44:01.634842+0000","last_unstale":"2024-09-19T07:44:02.633184+0000","last_undegraded":"2024-09-19T07:44:02.633184+0000","last_fullsized":"2024-09-19T07:44:02.633184+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:07.759986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633030+0000","last_change":"2024-09-19T07:44:01.629931+0000","last_active":"2024-09-19T07:44:02.633030+0000","last_peered":"2024-09-19T07:44:02.633030+0000","last_clean":"2024-09-19T07:44:02.633030+0000","last_became_active":"2024-09-19T07:44:01.629634+0000","last_became_peered":"2024-09-19T07:44:01.629634+0000","last_unstale":"2024-09-19T07:44:02.633030+0000","last_undegraded":"2024-09-19T07:44:02.633030+0000","last_fullsized":"2024-09-19T07:44:02.633030+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:00.266417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.619008+0000","last_change":"2024-09-19T07:44:00.619008+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.619008+0000","last_undegraded":"2024-09-19T07:44:00.619008+0000","last_fullsized":"2024-09-19T07:44:00.619008+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.619651+0000","last_change":"2024-09-19T07:44:00.619651+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.619651+0000","last_undegraded":"2024-09-19T07:44:00.619651+0000","last_fullsized":"2024-09-19T07:44:00.619651+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880437+0000","last_change":"2024-09-19T07:44:01.635795+0000","last_active":"2024-09-19T07:44:02.880437+0000","last_peered":"2024-09-19T07:44:02.880437+0000","last_clean":"2024-09-19T07:44:02.880437+0000","last_became_active":"2024-09-19T07:44:01.634728+0000","last_became_peered":"2024-09-19T07:44:01.634728+0000","last_unstale":"2024-09-19T07:44:02.880437+0000","last_undegraded":"2024-09-19T07:44:02.880437+0000","last_fullsized":"2024-09-19T07:44:02.880437+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:20.583756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632612+0000","last_change":"2024-09-19T07:44:01.621511+0000","last_active":"2024-09-19T07:44:02.632612+0000","last_peered":"2024-09-19T07:44:02.632612+0000","last_clean":"2024-09-19T07:44:02.632612+0000","last_became_active":"2024-09-19T07:44:01.620511+0000","last_became_peered":"2024-09-19T07:44:01.620511+0000","last_unstale":"2024-09-19T07:44:02.632612+0000","last_undegraded":"2024-09-19T07:44:02.632612+0000","last_fullsized":"2024-09-19T07:44:02.632612+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:57.716510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.620031+0000","last_change":"2024-09-19T07:44:00.620031+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.620031+0000","last_undegraded":"2024-09-19T07:44:00.620031+0000","last_fullsized":"2024-09-19T07:44:00.620031+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.618996+0000","last_change":"2024-09-19T07:44:00.618996+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.618996+0000","last_undegraded":"2024-09-19T07:44:00.618996+0000","last_fullsized":"2024-09-19T07:44:00.618996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879547+0000","last_change":"2024-09-19T07:44:01.632403+0000","last_active":"2024-09-19T07:44:02.879547+0000","last_peered":"2024-09-19T07:44:02.879547+0000","last_clean":"2024-09-19T07:44:02.879547+0000","last_became_active":"2024-09-19T07:44:01.632105+0000","last_became_peered":"2024-09-19T07:44:01.632105+0000","last_unstale":"2024-09-19T07:44:02.879547+0000","last_undegraded":"2024-09-19T07:44:02.879547+0000","last_fullsized":"2024-09-19T07:44:02.879547+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:22:00.022599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880311+0000","last_change":"2024-09-19T07:44:01.633044+0000","last_active":"2024-09-19T07:44:02.880311+0000","last_peered":"2024-09-19T07:44:02.880311+0000","last_clean":"2024-09-19T07:44:02.880311+0000","last_became_active":"2024-09-19T07:44:01.632761+0000","last_became_peered":"2024-09-19T07:44:01.632761+0000","last_unstale":"2024-09-19T07:44:02.880311+0000","last_undegraded":"2024-09-19T07:44:02.880311+0000","last_fullsized":"2024-09-19T07:44:02.880311+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:10.758189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879930+0000","last_change":"2024-09-19T07:44:01.630912+0000","last_active":"2024-09-19T07:44:02.879930+0000","last_peered":"2024-09-19T07:44:02.879930+0000","last_clean":"2024-09-19T07:44:02.879930+0000","last_became_active":"2024-09-19T07:44:01.630625+0000","last_became_peered":"2024-09-19T07:44:01.630625+0000","last_unstale":"2024-09-19T07:44:02.879930+0000","last_undegraded":"2024-09-19T07:44:02.879930+0000","last_fullsized":"2024-09-19T07:44:02.879930+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:16.252351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880122+0000","last_change":"2024-09-19T07:44:00.624268+0000","last_active":"2024-09-19T07:44:02.880122+0000","last_peered":"2024-09-19T07:44:02.880122+0000","last_clean":"2024-09-19T07:44:02.880122+0000","last_became_active":"2024-09-19T07:44:00.623980+0000","last_became_peered":"2024-09-19T07:44:00.623980+0000","last_unstale":"2024-09-19T07:44:02.880122+0000","last_undegraded":"2024-09-19T07:44:02.880122+0000","last_fullsized":"2024-09-19T07:44:02.880122+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:42.166793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633166+0000","last_change":"2024-09-19T07:44:01.642857+0000","last_active":"2024-09-19T07:44:02.633166+0000","last_peered":"2024-09-19T07:44:02.633166+0000","last_clean":"2024-09-19T07:44:02.633166+0000","last_became_active":"2024-09-19T07:44:01.642470+0000","last_became_peered":"2024-09-19T07:44:01.642470+0000","last_unstale":"2024-09-19T07:44:02.633166+0000","last_undegraded":"2024-09-19T07:44:02.633166+0000","last_fullsized":"2024-09-19T07:44:02.633166+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:32.560026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879466+0000","last_change":"2024-09-19T07:44:00.631253+0000","last_active":"2024-09-19T07:44:02.879466+0000","last_peered":"2024-09-19T07:44:02.879466+0000","last_clean":"2024-09-19T07:44:02.879466+0000","last_became_active":"2024-09-19T07:44:00.630967+0000","last_became_peered":"2024-09-19T07:44:00.630967+0000","last_unstale":"2024-09-19T07:44:02.879466+0000","last_undegraded":"2024-09-19T07:44:02.879466+0000","last_fullsized":"2024-09-19T07:44:02.879466+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:07.746484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632826+0000","last_change":"2024-09-19T07:44:01.637345+0000","last_active":"2024-09-19T07:44:02.632826+0000","last_peered":"2024-09-19T07:44:02.632826+0000","last_clean":"2024-09-19T07:44:02.632826+0000","last_became_active":"2024-09-19T07:44:01.633385+0000","last_became_peered":"2024-09-19T07:44:01.633385+0000","last_unstale":"2024-09-19T07:44:02.632826+0000","last_undegraded":"2024-09-19T07:44:02.632826+0000","last_fullsized":"2024-09-19T07:44:02.632826+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:26.999322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.613312+0000","last_change":"2024-09-19T07:44:00.613312+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.613312+0000","last_undegraded":"2024-09-19T07:44:00.613312+0000","last_fullsized":"2024-09-19T07:44:00.613312+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633235+0000","last_change":"2024-09-19T07:44:00.621900+0000","last_active":"2024-09-19T07:44:02.633235+0000","last_peered":"2024-09-19T07:44:02.633235+0000","last_clean":"2024-09-19T07:44:02.633235+0000","last_became_active":"2024-09-19T07:44:00.621596+0000","last_became_peered":"2024-09-19T07:44:00.621596+0000","last_unstale":"2024-09-19T07:44:02.633235+0000","last_undegraded":"2024-09-19T07:44:02.633235+0000","last_fullsized":"2024-09-19T07:44:02.633235+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:19.578605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632895+0000","last_change":"2024-09-19T07:44:00.629007+0000","last_active":"2024-09-19T07:44:02.632895+0000","last_peered":"2024-09-19T07:44:02.632895+0000","last_clean":"2024-09-19T07:44:02.632895+0000","last_became_active":"2024-09-19T07:44:00.628690+0000","last_became_peered":"2024-09-19T07:44:00.628690+0000","last_unstale":"2024-09-19T07:44:02.632895+0000","last_undegraded":"2024-09-19T07:44:02.632895+0000","last_fullsized":"2024-09-19T07:44:02.632895+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:15.247200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632967+0000","last_change":"2024-09-19T07:44:00.621833+0000","last_active":"2024-09-19T07:44:02.632967+0000","last_peered":"2024-09-19T07:44:02.632967+0000","last_clean":"2024-09-19T07:44:02.632967+0000","last_became_active":"2024-09-19T07:44:00.621525+0000","last_became_peered":"2024-09-19T07:44:00.621525+0000","last_unstale":"2024-09-19T07:44:02.632967+0000","last_undegraded":"2024-09-19T07:44:02.632967+0000","last_fullsized":"2024-09-19T07:44:02.632967+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:09.753037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632713+0000","last_change":"2024-09-19T07:44:01.635000+0000","last_active":"2024-09-19T07:44:02.632713+0000","last_peered":"2024-09-19T07:44:02.632713+0000","last_clean":"2024-09-19T07:44:02.632713+0000","last_became_active":"2024-09-19T07:44:01.634733+0000","last_became_peered":"2024-09-19T07:44:01.634733+0000","last_unstale":"2024-09-19T07:44:02.632713+0000","last_undegraded":"2024-09-19T07:44:02.632713+0000","last_fullsized":"2024-09-19T07:44:02.632713+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:26.448577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880363+0000","last_change":"2024-09-19T07:44:00.630777+0000","last_active":"2024-09-19T07:44:02.880363+0000","last_peered":"2024-09-19T07:44:02.880363+0000","last_clean":"2024-09-19T07:44:02.880363+0000","last_became_active":"2024-09-19T07:44:00.630494+0000","last_became_peered":"2024-09-19T07:44:00.630494+0000","last_unstale":"2024-09-19T07:44:02.880363+0000","last_undegraded":"2024-09-19T07:44:02.880363+0000","last_fullsized":"2024-09-19T07:44:02.880363+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:06:32.865629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632815+0000","last_change":"2024-09-19T07:44:01.637576+0000","last_active":"2024-09-19T07:44:02.632815+0000","last_peered":"2024-09-19T07:44:02.632815+0000","last_clean":"2024-09-19T07:44:02.632815+0000","last_became_active":"2024-09-19T07:44:01.637026+0000","last_became_peered":"2024-09-19T07:44:01.637026+0000","last_unstale":"2024-09-19T07:44:02.632815+0000","last_undegraded":"2024-09-19T07:44:02.632815+0000","last_fullsized":"2024-09-19T07:44:02.632815+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:34.910477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633057+0000","last_change":"2024-09-19T07:44:01.643372+0000","last_active":"2024-09-19T07:44:02.633057+0000","last_peered":"2024-09-19T07:44:02.633057+0000","last_clean":"2024-09-19T07:44:02.633057+0000","last_became_active":"2024-09-19T07:44:01.642744+0000","last_became_peered":"2024-09-19T07:44:01.642744+0000","last_unstale":"2024-09-19T07:44:02.633057+0000","last_undegraded":"2024-09-19T07:44:02.633057+0000","last_fullsized":"2024-09-19T07:44:02.633057+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:34.596548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.632871+0000","last_change":"2024-09-19T07:44:00.631635+0000","last_active":"2024-09-19T07:44:00.632871+0000","last_peered":"2024-09-19T07:44:00.632871+0000","last_clean":"2024-09-19T07:44:00.632871+0000","last_became_active":"2024-09-19T07:44:00.631343+0000","last_became_peered":"2024-09-19T07:44:00.631343+0000","last_unstale":"2024-09-19T07:44:00.632871+0000","last_undegraded":"2024-09-19T07:44:00.632871+0000","last_fullsized":"2024-09-19T07:44:00.632871+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:33.000415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.613328+0000","last_change":"2024-09-19T07:44:00.613328+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.613328+0000","last_undegraded":"2024-09-19T07:44:00.613328+0000","last_fullsized":"2024-09-19T07:44:00.613328+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879303+0000","last_change":"2024-09-19T07:44:00.625009+0000","last_active":"2024-09-19T07:44:02.879303+0000","last_peered":"2024-09-19T07:44:02.879303+0000","last_clean":"2024-09-19T07:44:02.879303+0000","last_became_active":"2024-09-19T07:44:00.624610+0000","last_became_peered":"2024-09-19T07:44:00.624610+0000","last_unstale":"2024-09-19T07:44:02.879303+0000","last_undegraded":"2024-09-19T07:44:02.879303+0000","last_fullsized":"2024-09-19T07:44:02.879303+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:58.958404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.625246+0000","last_change":"2024-09-19T07:44:00.623041+0000","last_active":"2024-09-19T07:44:00.625246+0000","last_peered":"2024-09-19T07:44:00.625246+0000","last_clean":"2024-09-19T07:44:00.625246+0000","last_became_active":"2024-09-19T07:44:00.622748+0000","last_became_peered":"2024-09-19T07:44:00.622748+0000","last_unstale":"2024-09-19T07:44:00.625246+0000","last_undegraded":"2024-09-19T07:44:00.625246+0000","last_fullsized":"2024-09-19T07:44:00.625246+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:41.495537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.628081+0000","last_change":"2024-09-19T07:44:00.627049+0000","last_active":"2024-09-19T07:44:00.628081+0000","last_peered":"2024-09-19T07:44:00.628081+0000","last_clean":"2024-09-19T07:44:00.628081+0000","last_became_active":"2024-09-19T07:44:00.626768+0000","last_became_peered":"2024-09-19T07:44:00.626768+0000","last_unstale":"2024-09-19T07:44:00.628081+0000","last_undegraded":"2024-09-19T07:44:00.628081+0000","last_fullsized":"2024-09-19T07:44:00.628081+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:18.929914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632969+0000","last_change":"2024-09-19T07:44:01.629794+0000","last_active":"2024-09-19T07:44:02.632969+0000","last_peered":"2024-09-19T07:44:02.632969+0000","last_clean":"2024-09-19T07:44:02.632969+0000","last_became_active":"2024-09-19T07:44:01.629453+0000","last_became_peered":"2024-09-19T07:44:01.629453+0000","last_unstale":"2024-09-19T07:44:02.632969+0000","last_undegraded":"2024-09-19T07:44:02.632969+0000","last_fullsized":"2024-09-19T07:44:02.632969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:34.270347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.624497+0000","last_change":"2024-09-19T07:44:00.623038+0000","last_active":"2024-09-19T07:44:00.624497+0000","last_peered":"2024-09-19T07:44:00.624497+0000","last_clean":"2024-09-19T07:44:00.624497+0000","last_became_active":"2024-09-19T07:44:00.622704+0000","last_became_peered":"2024-09-19T07:44:00.622704+0000","last_unstale":"2024-09-19T07:44:00.624497+0000","last_undegraded":"2024-09-19T07:44:00.624497+0000","last_fullsized":"2024-09-19T07:44:00.624497+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:38.159118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879198+0000","last_change":"2024-09-19T07:44:01.633567+0000","last_active":"2024-09-19T07:44:02.879198+0000","last_peered":"2024-09-19T07:44:02.879198+0000","last_clean":"2024-09-19T07:44:02.879198+0000","last_became_active":"2024-09-19T07:44:01.633291+0000","last_became_peered":"2024-09-19T07:44:01.633291+0000","last_unstale":"2024-09-19T07:44:02.879198+0000","last_undegraded":"2024-09-19T07:44:02.879198+0000","last_fullsized":"2024-09-19T07:44:02.879198+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:18.456608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.629136+0000","last_change":"2024-09-19T07:44:00.627575+0000","last_active":"2024-09-19T07:44:00.629136+0000","last_peered":"2024-09-19T07:44:00.629136+0000","last_clean":"2024-09-19T07:44:00.629136+0000","last_became_active":"2024-09-19T07:44:00.627217+0000","last_became_peered":"2024-09-19T07:44:00.627217+0000","last_unstale":"2024-09-19T07:44:00.629136+0000","last_undegraded":"2024-09-19T07:44:00.629136+0000","last_fullsized":"2024-09-19T07:44:00.629136+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:30.570507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633094+0000","last_change":"2024-09-19T07:44:01.620156+0000","last_active":"2024-09-19T07:44:02.633094+0000","last_peered":"2024-09-19T07:44:02.633094+0000","last_clean":"2024-09-19T07:44:02.633094+0000","last_became_active":"2024-09-19T07:44:01.619829+0000","last_became_peered":"2024-09-19T07:44:01.619829+0000","last_unstale":"2024-09-19T07:44:02.633094+0000","last_undegraded":"2024-09-19T07:44:02.633094+0000","last_fullsized":"2024-09-19T07:44:02.633094+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:17.164807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879371+0000","last_change":"2024-09-19T07:44:01.637663+0000","last_active":"2024-09-19T07:44:02.879371+0000","last_peered":"2024-09-19T07:44:02.879371+0000","last_clean":"2024-09-19T07:44:02.879371+0000","last_became_active":"2024-09-19T07:44:01.637391+0000","last_became_peered":"2024-09-19T07:44:01.637391+0000","last_unstale":"2024-09-19T07:44:02.879371+0000","last_undegraded":"2024-09-19T07:44:02.879371+0000","last_fullsized":"2024-09-19T07:44:02.879371+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:11.328826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632629+0000","last_change":"2024-09-19T07:44:00.622888+0000","last_active":"2024-09-19T07:44:02.632629+0000","last_peered":"2024-09-19T07:44:02.632629+0000","last_clean":"2024-09-19T07:44:02.632629+0000","last_became_active":"2024-09-19T07:44:00.622610+0000","last_became_peered":"2024-09-19T07:44:00.622610+0000","last_unstale":"2024-09-19T07:44:02.632629+0000","last_undegraded":"2024-09-19T07:44:02.632629+0000","last_fullsized":"2024-09-19T07:44:02.632629+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:59.017448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":195,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-19T07:43:56.599936+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:43:56.599936+0000","last_peered":"2024-09-19T07:43:56.599936+0000","last_clean":"2024-09-19T07:43:56.599936+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:43:56.599936+0000","last_undegraded":"2024-09-19T07:43:56.599936+0000","last_fullsized":"2024-09-19T07:43:56.599936+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":44,"acting":44,"num_store_stats":3},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":26,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":999424,"data_stored":759859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":405504,"data_stored":169491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":405504,"data_stored":169491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":169491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":999424,"data_stored":759859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31652,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":471040,"data_stored":197872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":446464,"data_stored":189624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":430080,"data_stored":182367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:44:05.434 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:44:05.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.571+0000 7efc81e27640 1 Processor -- start 2024-09-19T07:44:05.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.571+0000 7efc81e27640 1 -- start start 2024-09-19T07:44:05.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.572+0000 7efc81e27640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc7c0ce260 0x7efc7c166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:05.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.572+0000 7efc81e27640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc7c0cd910 con 0x7efc7c15b6e0 2024-09-19T07:44:05.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.572+0000 7efc81e27640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc7c0ccda0 con 0x7efc7c15b300 2024-09-19T07:44:05.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.572+0000 7efc81e27640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efc7c0cd070 con 0x7efc7c0ce260 2024-09-19T07:44:05.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.572+0000 7efc7affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efc7c15b6e0 0x7efc7c0cdb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44490/0 (socket says 172.21.15.16:44490) 2024-09-19T07:44:05.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.572+0000 7efc7affd640 1 -- 172.21.15.16:0/576976332 learned_addr learned my addr 172.21.15.16:0/576976332 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:44:05.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.573+0000 7efc7bfff640 1 --2- 172.21.15.16:0/576976332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc7c0ce260 0x7efc7c166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:05.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.573+0000 7efc7a7fc640 1 -- 172.21.15.16:0/576976332 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 71792044 0 0) 0x7efc7c0cd910 con 0x7efc7c15b6e0 2024-09-19T07:44:05.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.573+0000 7efc7a7fc640 1 -- 172.21.15.16:0/576976332 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc64003520 con 0x7efc7c15b6e0 2024-09-19T07:44:05.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.573+0000 7efc7a7fc640 1 -- 172.21.15.16:0/576976332 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3980320542 0 0) 0x7efc7c0ccda0 con 0x7efc7c15b300 2024-09-19T07:44:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.573+0000 7efc7a7fc640 1 -- 172.21.15.16:0/576976332 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc7c0cd910 con 0x7efc7c15b300 2024-09-19T07:44:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.573+0000 7efc7bfff640 1 -- 172.21.15.16:0/576976332 >> v1:172.21.15.146:6789/0 conn(0x7efc7c15b300 legacy=0x7efc7c0cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.573+0000 7efc7bfff640 1 -- 172.21.15.16:0/576976332 >> v1:172.21.15.16:6789/0 conn(0x7efc7c15b6e0 legacy=0x7efc7c0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.573+0000 7efc7bfff640 1 -- 172.21.15.16:0/576976332 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7c0cd350 con 0x7efc7c0ce260 2024-09-19T07:44:05.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.574+0000 7efc7bfff640 1 --2- 172.21.15.16:0/576976332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc7c0ce260 0x7efc7c166030 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efc74009dc0 tx=0x7efc7402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62dcf91ca75b4564 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.574+0000 7efc7a7fc640 1 -- 172.21.15.16:0/576976332 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74007740 con 0x7efc7c0ce260 2024-09-19T07:44:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.574+0000 7efc7a7fc640 1 -- 172.21.15.16:0/576976332 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc740078e0 con 0x7efc7c0ce260 2024-09-19T07:44:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.574+0000 7efc7a7fc640 1 -- 172.21.15.16:0/576976332 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74007c90 con 0x7efc7c0ce260 2024-09-19T07:44:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.574+0000 7efc81e27640 1 -- 172.21.15.16:0/576976332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc7c0ce260 msgr2=0x7efc7c166030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.574+0000 7efc81e27640 1 --2- 172.21.15.16:0/576976332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc7c0ce260 0x7efc7c166030 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efc74009dc0 tx=0x7efc7402a6d0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.574+0000 7efc81e27640 1 -- 172.21.15.16:0/576976332 shutdown_connections 2024-09-19T07:44:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.574+0000 7efc81e27640 1 --2- 172.21.15.16:0/576976332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc7c0ce260 0x7efc7c166030 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.574+0000 7efc81e27640 1 -- 172.21.15.16:0/576976332 >> 172.21.15.16:0/576976332 conn(0x7efc7c0b2d10 msgr2=0x7efc7c0b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.575+0000 7efc81e27640 1 -- 172.21.15.16:0/576976332 shutdown_connections 2024-09-19T07:44:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.575+0000 7efc81e27640 1 -- 172.21.15.16:0/576976332 wait complete. 2024-09-19T07:44:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.576+0000 7efc81e27640 1 Processor -- start 2024-09-19T07:44:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.576+0000 7efc81e27640 1 -- start start 2024-09-19T07:44:05.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc81e27640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc7c0ce260 0x7efc7c156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:05.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc81e27640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc7c15b300 0x7efc7c15e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:05.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc81e27640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc7c15b6e0 0x7efc7c0b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:05.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc81e27640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c0b39f0 con 0x7efc7c15b6e0 2024-09-19T07:44:05.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc81e27640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c0b0b50 con 0x7efc7c15b300 2024-09-19T07:44:05.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc81e27640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efc7c0b3b70 con 0x7efc7c0ce260 2024-09-19T07:44:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc7c15b6e0 0x7efc7c0b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc7c15b6e0 0x7efc7c0b6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51724/0 (socket says 172.21.15.16:51724) 2024-09-19T07:44:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc7bfff640 1 -- 172.21.15.16:0/3615750946 learned_addr learned my addr 172.21.15.16:0/3615750946 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc7b7fe640 1 --2- 172.21.15.16:0/3615750946 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc7c0ce260 0x7efc7c156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc7affd640 1 --2- 172.21.15.16:0/3615750946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc7c15b300 0x7efc7c15e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc7bfff640 1 -- 172.21.15.16:0/3615750946 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc7c0ce260 msgr2=0x7efc7c156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc7bfff640 1 --2- 172.21.15.16:0/3615750946 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc7c0ce260 0x7efc7c156890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc7bfff640 1 -- 172.21.15.16:0/3615750946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc7c15b300 msgr2=0x7efc7c15e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc7bfff640 1 --2- 172.21.15.16:0/3615750946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc7c15b300 0x7efc7c15e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc7bfff640 1 -- 172.21.15.16:0/3615750946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7c0b55c0 con 0x7efc7c15b6e0 2024-09-19T07:44:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc7bfff640 1 --2- 172.21.15.16:0/3615750946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc7c15b6e0 0x7efc7c0b6660 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efc7402acc0 tx=0x7efc74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc63fff640 1 -- 172.21.15.16:0/3615750946 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7403f070 con 0x7efc7c15b6e0 2024-09-19T07:44:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc63fff640 1 -- 172.21.15.16:0/3615750946 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc74031c40 con 0x7efc7c15b6e0 2024-09-19T07:44:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc81e27640 1 -- 172.21.15.16:0/3615750946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc7c0b58a0 con 0x7efc7c15b6e0 2024-09-19T07:44:05.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.577+0000 7efc63fff640 1 -- 172.21.15.16:0/3615750946 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74009450 con 0x7efc7c15b6e0 2024-09-19T07:44:05.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.578+0000 7efc81e27640 1 -- 172.21.15.16:0/3615750946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc7c0b5e60 con 0x7efc7c15b6e0 2024-09-19T07:44:05.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.579+0000 7efc81e27640 1 -- 172.21.15.16:0/3615750946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc7c0cdb50 con 0x7efc7c15b6e0 2024-09-19T07:44:05.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.579+0000 7efc63fff640 1 -- 172.21.15.16:0/3615750946 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7efc7403b410 con 0x7efc7c15b6e0 2024-09-19T07:44:05.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.579+0000 7efc63fff640 1 --2- 172.21.15.16:0/3615750946 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efc6806c2b0 0x7efc6806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:05.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.579+0000 7efc63fff640 1 -- 172.21.15.16:0/3615750946 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7efc740acb70 con 0x7efc7c15b6e0 2024-09-19T07:44:05.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.580+0000 7efc7b7fe640 1 --2- 172.21.15.16:0/3615750946 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efc6806c2b0 0x7efc6806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:05.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.580+0000 7efc7b7fe640 1 --2- 172.21.15.16:0/3615750946 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efc6806c2b0 0x7efc6806e770 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efc6c007fd0 tx=0x7efc6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:05.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.584+0000 7efc63fff640 1 -- 172.21.15.16:0/3615750946 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7efc7407ab00 con 0x7efc7c15b6e0 2024-09-19T07:44:05.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.760+0000 7efc81e27640 1 -- 172.21.15.16:0/3615750946 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efc7c159f90 con 0x7efc6806c2b0 2024-09-19T07:44:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.766+0000 7efc63fff640 1 -- 172.21.15.16:0/3615750946 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242129 (secure 0 0 0) 0x7efc7c159f90 con 0x7efc6806c2b0 2024-09-19T07:44:05.769 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:44:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:44:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.769+0000 7efc81e27640 1 -- 172.21.15.16:0/3615750946 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efc6806c2b0 msgr2=0x7efc6806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.769+0000 7efc81e27640 1 --2- 172.21.15.16:0/3615750946 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efc6806c2b0 0x7efc6806e770 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efc6c007fd0 tx=0x7efc6c009040 comp rx=0 tx=0).stop 2024-09-19T07:44:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.769+0000 7efc81e27640 1 -- 172.21.15.16:0/3615750946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc7c15b6e0 msgr2=0x7efc7c0b6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.769+0000 7efc81e27640 1 --2- 172.21.15.16:0/3615750946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc7c15b6e0 0x7efc7c0b6660 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efc7402acc0 tx=0x7efc74004a40 comp rx=0 tx=0).stop 2024-09-19T07:44:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.769+0000 7efc81e27640 1 -- 172.21.15.16:0/3615750946 shutdown_connections 2024-09-19T07:44:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.769+0000 7efc81e27640 1 --2- 172.21.15.16:0/3615750946 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efc6806c2b0 0x7efc6806e770 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.769+0000 7efc81e27640 1 --2- 172.21.15.16:0/3615750946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc7c15b6e0 0x7efc7c0b6660 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.769+0000 7efc81e27640 1 --2- 172.21.15.16:0/3615750946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc7c15b300 0x7efc7c15e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.769+0000 7efc81e27640 1 --2- 172.21.15.16:0/3615750946 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efc7c0ce260 0x7efc7c156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.769+0000 7efc81e27640 1 -- 172.21.15.16:0/3615750946 >> 172.21.15.16:0/3615750946 conn(0x7efc7c0b2d10 msgr2=0x7efc7c159400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:05.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.769+0000 7efc81e27640 1 -- 172.21.15.16:0/3615750946 shutdown_connections 2024-09-19T07:44:05.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.769+0000 7efc81e27640 1 -- 172.21.15.16:0/3615750946 wait complete. 2024-09-19T07:44:05.780 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-09-19T07:44:04.472124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592014,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":237364,"kb_used_data":4456,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737356,"statfs":{"total":773094113280,"available":772851052544,"internally_reserved":0,"allocated":4562944,"data_stored":2598054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.865307"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633184+0000","last_change":"2024-09-19T07:44:01.635139+0000","last_active":"2024-09-19T07:44:02.633184+0000","last_peered":"2024-09-19T07:44:02.633184+0000","last_clean":"2024-09-19T07:44:02.633184+0000","last_became_active":"2024-09-19T07:44:01.634842+0000","last_became_peered":"2024-09-19T07:44:01.634842+0000","last_unstale":"2024-09-19T07:44:02.633184+0000","last_undegraded":"2024-09-19T07:44:02.633184+0000","last_fullsized":"2024-09-19T07:44:02.633184+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:07.759986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633030+0000","last_change":"2024-09-19T07:44:01.629931+0000","last_active":"2024-09-19T07:44:02.633030+0000","last_peered":"2024-09-19T07:44:02.633030+0000","last_clean":"2024-09-19T07:44:02.633030+0000","last_became_active":"2024-09-19T07:44:01.629634+0000","last_became_peered":"2024-09-19T07:44:01.629634+0000","last_unstale":"2024-09-19T07:44:02.633030+0000","last_undegraded":"2024-09-19T07:44:02.633030+0000","last_fullsized":"2024-09-19T07:44:02.633030+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:00.266417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.619008+0000","last_change":"2024-09-19T07:44:00.619008+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.619008+0000","last_undegraded":"2024-09-19T07:44:00.619008+0000","last_fullsized":"2024-09-19T07:44:00.619008+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.619651+0000","last_change":"2024-09-19T07:44:00.619651+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.619651+0000","last_undegraded":"2024-09-19T07:44:00.619651+0000","last_fullsized":"2024-09-19T07:44:00.619651+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880437+0000","last_change":"2024-09-19T07:44:01.635795+0000","last_active":"2024-09-19T07:44:02.880437+0000","last_peered":"2024-09-19T07:44:02.880437+0000","last_clean":"2024-09-19T07:44:02.880437+0000","last_became_active":"2024-09-19T07:44:01.634728+0000","last_became_peered":"2024-09-19T07:44:01.634728+0000","last_unstale":"2024-09-19T07:44:02.880437+0000","last_undegraded":"2024-09-19T07:44:02.880437+0000","last_fullsized":"2024-09-19T07:44:02.880437+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:20.583756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632612+0000","last_change":"2024-09-19T07:44:01.621511+0000","last_active":"2024-09-19T07:44:02.632612+0000","last_peered":"2024-09-19T07:44:02.632612+0000","last_clean":"2024-09-19T07:44:02.632612+0000","last_became_active":"2024-09-19T07:44:01.620511+0000","last_became_peered":"2024-09-19T07:44:01.620511+0000","last_unstale":"2024-09-19T07:44:02.632612+0000","last_undegraded":"2024-09-19T07:44:02.632612+0000","last_fullsized":"2024-09-19T07:44:02.632612+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:57.716510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.620031+0000","last_change":"2024-09-19T07:44:00.620031+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.620031+0000","last_undegraded":"2024-09-19T07:44:00.620031+0000","last_fullsized":"2024-09-19T07:44:00.620031+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.618996+0000","last_change":"2024-09-19T07:44:00.618996+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.618996+0000","last_undegraded":"2024-09-19T07:44:00.618996+0000","last_fullsized":"2024-09-19T07:44:00.618996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879547+0000","last_change":"2024-09-19T07:44:01.632403+0000","last_active":"2024-09-19T07:44:02.879547+0000","last_peered":"2024-09-19T07:44:02.879547+0000","last_clean":"2024-09-19T07:44:02.879547+0000","last_became_active":"2024-09-19T07:44:01.632105+0000","last_became_peered":"2024-09-19T07:44:01.632105+0000","last_unstale":"2024-09-19T07:44:02.879547+0000","last_undegraded":"2024-09-19T07:44:02.879547+0000","last_fullsized":"2024-09-19T07:44:02.879547+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:22:00.022599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880311+0000","last_change":"2024-09-19T07:44:01.633044+0000","last_active":"2024-09-19T07:44:02.880311+0000","last_peered":"2024-09-19T07:44:02.880311+0000","last_clean":"2024-09-19T07:44:02.880311+0000","last_became_active":"2024-09-19T07:44:01.632761+0000","last_became_peered":"2024-09-19T07:44:01.632761+0000","last_unstale":"2024-09-19T07:44:02.880311+0000","last_undegraded":"2024-09-19T07:44:02.880311+0000","last_fullsized":"2024-09-19T07:44:02.880311+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:10.758189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879930+0000","last_change":"2024-09-19T07:44:01.630912+0000","last_active":"2024-09-19T07:44:02.879930+0000","last_peered":"2024-09-19T07:44:02.879930+0000","last_clean":"2024-09-19T07:44:02.879930+0000","last_became_active":"2024-09-19T07:44:01.630625+0000","last_became_peered":"2024-09-19T07:44:01.630625+0000","last_unstale":"2024-09-19T07:44:02.879930+0000","last_undegraded":"2024-09-19T07:44:02.879930+0000","last_fullsized":"2024-09-19T07:44:02.879930+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:16.252351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880122+0000","last_change":"2024-09-19T07:44:00.624268+0000","last_active":"2024-09-19T07:44:02.880122+0000","last_peered":"2024-09-19T07:44:02.880122+0000","last_clean":"2024-09-19T07:44:02.880122+0000","last_became_active":"2024-09-19T07:44:00.623980+0000","last_became_peered":"2024-09-19T07:44:00.623980+0000","last_unstale":"2024-09-19T07:44:02.880122+0000","last_undegraded":"2024-09-19T07:44:02.880122+0000","last_fullsized":"2024-09-19T07:44:02.880122+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:42.166793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633166+0000","last_change":"2024-09-19T07:44:01.642857+0000","last_active":"2024-09-19T07:44:02.633166+0000","last_peered":"2024-09-19T07:44:02.633166+0000","last_clean":"2024-09-19T07:44:02.633166+0000","last_became_active":"2024-09-19T07:44:01.642470+0000","last_became_peered":"2024-09-19T07:44:01.642470+0000","last_unstale":"2024-09-19T07:44:02.633166+0000","last_undegraded":"2024-09-19T07:44:02.633166+0000","last_fullsized":"2024-09-19T07:44:02.633166+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:32.560026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879466+0000","last_change":"2024-09-19T07:44:00.631253+0000","last_active":"2024-09-19T07:44:02.879466+0000","last_peered":"2024-09-19T07:44:02.879466+0000","last_clean":"2024-09-19T07:44:02.879466+0000","last_became_active":"2024-09-19T07:44:00.630967+0000","last_became_peered":"2024-09-19T07:44:00.630967+0000","last_unstale":"2024-09-19T07:44:02.879466+0000","last_undegraded":"2024-09-19T07:44:02.879466+0000","last_fullsized":"2024-09-19T07:44:02.879466+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:07.746484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632826+0000","last_change":"2024-09-19T07:44:01.637345+0000","last_active":"2024-09-19T07:44:02.632826+0000","last_peered":"2024-09-19T07:44:02.632826+0000","last_clean":"2024-09-19T07:44:02.632826+0000","last_became_active":"2024-09-19T07:44:01.633385+0000","last_became_peered":"2024-09-19T07:44:01.633385+0000","last_unstale":"2024-09-19T07:44:02.632826+0000","last_undegraded":"2024-09-19T07:44:02.632826+0000","last_fullsized":"2024-09-19T07:44:02.632826+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:26.999322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.613312+0000","last_change":"2024-09-19T07:44:00.613312+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.613312+0000","last_undegraded":"2024-09-19T07:44:00.613312+0000","last_fullsized":"2024-09-19T07:44:00.613312+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633235+0000","last_change":"2024-09-19T07:44:00.621900+0000","last_active":"2024-09-19T07:44:02.633235+0000","last_peered":"2024-09-19T07:44:02.633235+0000","last_clean":"2024-09-19T07:44:02.633235+0000","last_became_active":"2024-09-19T07:44:00.621596+0000","last_became_peered":"2024-09-19T07:44:00.621596+0000","last_unstale":"2024-09-19T07:44:02.633235+0000","last_undegraded":"2024-09-19T07:44:02.633235+0000","last_fullsized":"2024-09-19T07:44:02.633235+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:19.578605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632895+0000","last_change":"2024-09-19T07:44:00.629007+0000","last_active":"2024-09-19T07:44:02.632895+0000","last_peered":"2024-09-19T07:44:02.632895+0000","last_clean":"2024-09-19T07:44:02.632895+0000","last_became_active":"2024-09-19T07:44:00.628690+0000","last_became_peered":"2024-09-19T07:44:00.628690+0000","last_unstale":"2024-09-19T07:44:02.632895+0000","last_undegraded":"2024-09-19T07:44:02.632895+0000","last_fullsized":"2024-09-19T07:44:02.632895+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:15.247200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632967+0000","last_change":"2024-09-19T07:44:00.621833+0000","last_active":"2024-09-19T07:44:02.632967+0000","last_peered":"2024-09-19T07:44:02.632967+0000","last_clean":"2024-09-19T07:44:02.632967+0000","last_became_active":"2024-09-19T07:44:00.621525+0000","last_became_peered":"2024-09-19T07:44:00.621525+0000","last_unstale":"2024-09-19T07:44:02.632967+0000","last_undegraded":"2024-09-19T07:44:02.632967+0000","last_fullsized":"2024-09-19T07:44:02.632967+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:09.753037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632713+0000","last_change":"2024-09-19T07:44:01.635000+0000","last_active":"2024-09-19T07:44:02.632713+0000","last_peered":"2024-09-19T07:44:02.632713+0000","last_clean":"2024-09-19T07:44:02.632713+0000","last_became_active":"2024-09-19T07:44:01.634733+0000","last_became_peered":"2024-09-19T07:44:01.634733+0000","last_unstale":"2024-09-19T07:44:02.632713+0000","last_undegraded":"2024-09-19T07:44:02.632713+0000","last_fullsized":"2024-09-19T07:44:02.632713+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:26.448577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880363+0000","last_change":"2024-09-19T07:44:00.630777+0000","last_active":"2024-09-19T07:44:02.880363+0000","last_peered":"2024-09-19T07:44:02.880363+0000","last_clean":"2024-09-19T07:44:02.880363+0000","last_became_active":"2024-09-19T07:44:00.630494+0000","last_became_peered":"2024-09-19T07:44:00.630494+0000","last_unstale":"2024-09-19T07:44:02.880363+0000","last_undegraded":"2024-09-19T07:44:02.880363+0000","last_fullsized":"2024-09-19T07:44:02.880363+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:06:32.865629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632815+0000","last_change":"2024-09-19T07:44:01.637576+0000","last_active":"2024-09-19T07:44:02.632815+0000","last_peered":"2024-09-19T07:44:02.632815+0000","last_clean":"2024-09-19T07:44:02.632815+0000","last_became_active":"2024-09-19T07:44:01.637026+0000","last_became_peered":"2024-09-19T07:44:01.637026+0000","last_unstale":"2024-09-19T07:44:02.632815+0000","last_undegraded":"2024-09-19T07:44:02.632815+0000","last_fullsized":"2024-09-19T07:44:02.632815+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:34.910477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633057+0000","last_change":"2024-09-19T07:44:01.643372+0000","last_active":"2024-09-19T07:44:02.633057+0000","last_peered":"2024-09-19T07:44:02.633057+0000","last_clean":"2024-09-19T07:44:02.633057+0000","last_became_active":"2024-09-19T07:44:01.642744+0000","last_became_peered":"2024-09-19T07:44:01.642744+0000","last_unstale":"2024-09-19T07:44:02.633057+0000","last_undegraded":"2024-09-19T07:44:02.633057+0000","last_fullsized":"2024-09-19T07:44:02.633057+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:34.596548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.632871+0000","last_change":"2024-09-19T07:44:00.631635+0000","last_active":"2024-09-19T07:44:00.632871+0000","last_peered":"2024-09-19T07:44:00.632871+0000","last_clean":"2024-09-19T07:44:00.632871+0000","last_became_active":"2024-09-19T07:44:00.631343+0000","last_became_peered":"2024-09-19T07:44:00.631343+0000","last_unstale":"2024-09-19T07:44:00.632871+0000","last_undegraded":"2024-09-19T07:44:00.632871+0000","last_fullsized":"2024-09-19T07:44:00.632871+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:33.000415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.613328+0000","last_change":"2024-09-19T07:44:00.613328+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.613328+0000","last_undegraded":"2024-09-19T07:44:00.613328+0000","last_fullsized":"2024-09-19T07:44:00.613328+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879303+0000","last_change":"2024-09-19T07:44:00.625009+0000","last_active":"2024-09-19T07:44:02.879303+0000","last_peered":"2024-09-19T07:44:02.879303+0000","last_clean":"2024-09-19T07:44:02.879303+0000","last_became_active":"2024-09-19T07:44:00.624610+0000","last_became_peered":"2024-09-19T07:44:00.624610+0000","last_unstale":"2024-09-19T07:44:02.879303+0000","last_undegraded":"2024-09-19T07:44:02.879303+0000","last_fullsized":"2024-09-19T07:44:02.879303+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:58.958404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.625246+0000","last_change":"2024-09-19T07:44:00.623041+0000","last_active":"2024-09-19T07:44:00.625246+0000","last_peered":"2024-09-19T07:44:00.625246+0000","last_clean":"2024-09-19T07:44:00.625246+0000","last_became_active":"2024-09-19T07:44:00.622748+0000","last_became_peered":"2024-09-19T07:44:00.622748+0000","last_unstale":"2024-09-19T07:44:00.625246+0000","last_undegraded":"2024-09-19T07:44:00.625246+0000","last_fullsized":"2024-09-19T07:44:00.625246+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:41.495537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.628081+0000","last_change":"2024-09-19T07:44:00.627049+0000","last_active":"2024-09-19T07:44:00.628081+0000","last_peered":"2024-09-19T07:44:00.628081+0000","last_clean":"2024-09-19T07:44:00.628081+0000","last_became_active":"2024-09-19T07:44:00.626768+0000","last_became_peered":"2024-09-19T07:44:00.626768+0000","last_unstale":"2024-09-19T07:44:00.628081+0000","last_undegraded":"2024-09-19T07:44:00.628081+0000","last_fullsized":"2024-09-19T07:44:00.628081+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:18.929914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632969+0000","last_change":"2024-09-19T07:44:01.629794+0000","last_active":"2024-09-19T07:44:02.632969+0000","last_peered":"2024-09-19T07:44:02.632969+0000","last_clean":"2024-09-19T07:44:02.632969+0000","last_became_active":"2024-09-19T07:44:01.629453+0000","last_became_peered":"2024-09-19T07:44:01.629453+0000","last_unstale":"2024-09-19T07:44:02.632969+0000","last_undegraded":"2024-09-19T07:44:02.632969+0000","last_fullsized":"2024-09-19T07:44:02.632969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:34.270347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.624497+0000","last_change":"2024-09-19T07:44:00.623038+0000","last_active":"2024-09-19T07:44:00.624497+0000","last_peered":"2024-09-19T07:44:00.624497+0000","last_clean":"2024-09-19T07:44:00.624497+0000","last_became_active":"2024-09-19T07:44:00.622704+0000","last_became_peered":"2024-09-19T07:44:00.622704+0000","last_unstale":"2024-09-19T07:44:00.624497+0000","last_undegraded":"2024-09-19T07:44:00.624497+0000","last_fullsized":"2024-09-19T07:44:00.624497+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:38.159118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879198+0000","last_change":"2024-09-19T07:44:01.633567+0000","last_active":"2024-09-19T07:44:02.879198+0000","last_peered":"2024-09-19T07:44:02.879198+0000","last_clean":"2024-09-19T07:44:02.879198+0000","last_became_active":"2024-09-19T07:44:01.633291+0000","last_became_peered":"2024-09-19T07:44:01.633291+0000","last_unstale":"2024-09-19T07:44:02.879198+0000","last_undegraded":"2024-09-19T07:44:02.879198+0000","last_fullsized":"2024-09-19T07:44:02.879198+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:18.456608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.629136+0000","last_change":"2024-09-19T07:44:00.627575+0000","last_active":"2024-09-19T07:44:00.629136+0000","last_peered":"2024-09-19T07:44:00.629136+0000","last_clean":"2024-09-19T07:44:00.629136+0000","last_became_active":"2024-09-19T07:44:00.627217+0000","last_became_peered":"2024-09-19T07:44:00.627217+0000","last_unstale":"2024-09-19T07:44:00.629136+0000","last_undegraded":"2024-09-19T07:44:00.629136+0000","last_fullsized":"2024-09-19T07:44:00.629136+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:30.570507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633094+0000","last_change":"2024-09-19T07:44:01.620156+0000","last_active":"2024-09-19T07:44:02.633094+0000","last_peered":"2024-09-19T07:44:02.633094+0000","last_clean":"2024-09-19T07:44:02.633094+0000","last_became_active":"2024-09-19T07:44:01.619829+0000","last_became_peered":"2024-09-19T07:44:01.619829+0000","last_unstale":"2024-09-19T07:44:02.633094+0000","last_undegraded":"2024-09-19T07:44:02.633094+0000","last_fullsized":"2024-09-19T07:44:02.633094+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:17.164807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879371+0000","last_change":"2024-09-19T07:44:01.637663+0000","last_active":"2024-09-19T07:44:02.879371+0000","last_peered":"2024-09-19T07:44:02.879371+0000","last_clean":"2024-09-19T07:44:02.879371+0000","last_became_active":"2024-09-19T07:44:01.637391+0000","last_became_peered":"2024-09-19T07:44:01.637391+0000","last_unstale":"2024-09-19T07:44:02.879371+0000","last_undegraded":"2024-09-19T07:44:02.879371+0000","last_fullsized":"2024-09-19T07:44:02.879371+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:11.328826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632629+0000","last_change":"2024-09-19T07:44:00.622888+0000","last_active":"2024-09-19T07:44:02.632629+0000","last_peered":"2024-09-19T07:44:02.632629+0000","last_clean":"2024-09-19T07:44:02.632629+0000","last_became_active":"2024-09-19T07:44:00.622610+0000","last_became_peered":"2024-09-19T07:44:00.622610+0000","last_unstale":"2024-09-19T07:44:02.632629+0000","last_undegraded":"2024-09-19T07:44:02.632629+0000","last_fullsized":"2024-09-19T07:44:02.632629+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:59.017448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:44:00.603665+0000","last_change":"2024-09-19T07:44:00.603665+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.603665+0000","last_undegraded":"2024-09-19T07:44:00.603665+0000","last_fullsized":"2024-09-19T07:44:00.603665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:43:59.598514+0000","last_change":"2024-09-19T07:43:59.598514+0000","last_active":"2024-09-19T07:43:59.598514+0000","last_peered":"2024-09-19T07:43:59.598514+0000","last_clean":"2024-09-19T07:43:59.598514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:43:59.598514+0000","last_undegraded":"2024-09-19T07:43:59.598514+0000","last_fullsized":"2024-09-19T07:43:59.598514+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":195,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-19T07:43:56.599936+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:43:56.599936+0000","last_peered":"2024-09-19T07:43:56.599936+0000","last_clean":"2024-09-19T07:43:56.599936+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:43:56.599936+0000","last_undegraded":"2024-09-19T07:43:56.599936+0000","last_fullsized":"2024-09-19T07:43:56.599936+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":44,"acting":44,"num_store_stats":3},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":26,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":999424,"data_stored":759859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":405504,"data_stored":169491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":405504,"data_stored":169491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":169491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":999424,"data_stored":759859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31652,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":471040,"data_stored":197872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":446464,"data_stored":189624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":430080,"data_stored":182367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:44:05.783 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3d is not active+clean 2024-09-19T07:44:05.783 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.784 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3c is not active+clean 2024-09-19T07:44:05.784 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:44:00.619008+0000', 'last_change': '2024-09-19T07:44:00.619008+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.619008+0000', 'last_undegraded': '2024-09-19T07:44:00.619008+0000', 'last_fullsized': '2024-09-19T07:44:00.619008+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T07:44:05.784 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3b is not active+clean 2024-09-19T07:44:05.784 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:44:00.619651+0000', 'last_change': '2024-09-19T07:44:00.619651+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.619651+0000', 'last_undegraded': '2024-09-19T07:44:00.619651+0000', 'last_fullsized': '2024-09-19T07:44:00.619651+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T07:44:05.784 INFO:tasks.ceph.ceph_manager.ceph:PG 6.39 is not active+clean 2024-09-19T07:44:05.784 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.784 INFO:tasks.ceph.ceph_manager.ceph:PG 6.37 is not active+clean 2024-09-19T07:44:05.784 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.784 INFO:tasks.ceph.ceph_manager.ceph:PG 6.36 is not active+clean 2024-09-19T07:44:05.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.785 INFO:tasks.ceph.ceph_manager.ceph:PG 6.35 is not active+clean 2024-09-19T07:44:05.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.785 INFO:tasks.ceph.ceph_manager.ceph:PG 6.34 is not active+clean 2024-09-19T07:44:05.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.785 INFO:tasks.ceph.ceph_manager.ceph:PG 6.33 is not active+clean 2024-09-19T07:44:05.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.785 INFO:tasks.ceph.ceph_manager.ceph:PG 6.32 is not active+clean 2024-09-19T07:44:05.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.785 INFO:tasks.ceph.ceph_manager.ceph:PG 6.31 is not active+clean 2024-09-19T07:44:05.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.786 INFO:tasks.ceph.ceph_manager.ceph:PG 6.30 is not active+clean 2024-09-19T07:44:05.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.786 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2f is not active+clean 2024-09-19T07:44:05.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.786 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2e is not active+clean 2024-09-19T07:44:05.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.786 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2d is not active+clean 2024-09-19T07:44:05.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:44:00.620031+0000', 'last_change': '2024-09-19T07:44:00.620031+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.620031+0000', 'last_undegraded': '2024-09-19T07:44:00.620031+0000', 'last_fullsized': '2024-09-19T07:44:00.620031+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T07:44:05.786 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2c is not active+clean 2024-09-19T07:44:05.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.786 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2b is not active+clean 2024-09-19T07:44:05.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.787 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2a is not active+clean 2024-09-19T07:44:05.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.787 INFO:tasks.ceph.ceph_manager.ceph:PG 6.29 is not active+clean 2024-09-19T07:44:05.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.787 INFO:tasks.ceph.ceph_manager.ceph:PG 6.28 is not active+clean 2024-09-19T07:44:05.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:44:00.618996+0000', 'last_change': '2024-09-19T07:44:00.618996+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.618996+0000', 'last_undegraded': '2024-09-19T07:44:00.618996+0000', 'last_fullsized': '2024-09-19T07:44:00.618996+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T07:44:05.787 INFO:tasks.ceph.ceph_manager.ceph:PG 6.27 is not active+clean 2024-09-19T07:44:05.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.787 INFO:tasks.ceph.ceph_manager.ceph:PG 6.25 is not active+clean 2024-09-19T07:44:05.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.788 INFO:tasks.ceph.ceph_manager.ceph:PG 6.24 is not active+clean 2024-09-19T07:44:05.788 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.788 INFO:tasks.ceph.ceph_manager.ceph:PG 6.23 is not active+clean 2024-09-19T07:44:05.788 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.788 INFO:tasks.ceph.ceph_manager.ceph:PG 6.22 is not active+clean 2024-09-19T07:44:05.788 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.788 INFO:tasks.ceph.ceph_manager.ceph:PG 6.21 is not active+clean 2024-09-19T07:44:05.788 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.788 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1c is not active+clean 2024-09-19T07:44:05.788 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.788 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1f is not active+clean 2024-09-19T07:44:05.789 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:43:59.598514+0000', 'last_change': '2024-09-19T07:43:59.598514+0000', 'last_active': '2024-09-19T07:43:59.598514+0000', 'last_peered': '2024-09-19T07:43:59.598514+0000', 'last_clean': '2024-09-19T07:43:59.598514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:43:59.598514+0000', 'last_undegraded': '2024-09-19T07:43:59.598514+0000', 'last_fullsized': '2024-09-19T07:43:59.598514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_clean_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.789 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1e is not active+clean 2024-09-19T07:44:05.789 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:43:59.598514+0000', 'last_change': '2024-09-19T07:43:59.598514+0000', 'last_active': '2024-09-19T07:43:59.598514+0000', 'last_peered': '2024-09-19T07:43:59.598514+0000', 'last_clean': '2024-09-19T07:43:59.598514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:43:59.598514+0000', 'last_undegraded': '2024-09-19T07:43:59.598514+0000', 'last_fullsized': '2024-09-19T07:43:59.598514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_clean_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.789 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1e is not active+clean 2024-09-19T07:44:05.789 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.789 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1c is not active+clean 2024-09-19T07:44:05.789 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:43:59.598514+0000', 'last_change': '2024-09-19T07:43:59.598514+0000', 'last_active': '2024-09-19T07:43:59.598514+0000', 'last_peered': '2024-09-19T07:43:59.598514+0000', 'last_clean': '2024-09-19T07:43:59.598514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:43:59.598514+0000', 'last_undegraded': '2024-09-19T07:43:59.598514+0000', 'last_fullsized': '2024-09-19T07:43:59.598514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_clean_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.789 INFO:tasks.ceph.ceph_manager.ceph:PG 6.18 is not active+clean 2024-09-19T07:44:05.789 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.789 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1b is not active+clean 2024-09-19T07:44:05.789 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:43:59.598514+0000', 'last_change': '2024-09-19T07:43:59.598514+0000', 'last_active': '2024-09-19T07:43:59.598514+0000', 'last_peered': '2024-09-19T07:43:59.598514+0000', 'last_clean': '2024-09-19T07:43:59.598514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:43:59.598514+0000', 'last_undegraded': '2024-09-19T07:43:59.598514+0000', 'last_fullsized': '2024-09-19T07:43:59.598514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_clean_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.790 INFO:tasks.ceph.ceph_manager.ceph:PG 6.19 is not active+clean 2024-09-19T07:44:05.790 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.790 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1a is not active+clean 2024-09-19T07:44:05.790 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:43:59.598514+0000', 'last_change': '2024-09-19T07:43:59.598514+0000', 'last_active': '2024-09-19T07:43:59.598514+0000', 'last_peered': '2024-09-19T07:43:59.598514+0000', 'last_clean': '2024-09-19T07:43:59.598514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:43:59.598514+0000', 'last_undegraded': '2024-09-19T07:43:59.598514+0000', 'last_fullsized': '2024-09-19T07:43:59.598514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_clean_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.790 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1a is not active+clean 2024-09-19T07:44:05.790 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.790 INFO:tasks.ceph.ceph_manager.ceph:PG 5.18 is not active+clean 2024-09-19T07:44:05.790 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:43:59.598514+0000', 'last_change': '2024-09-19T07:43:59.598514+0000', 'last_active': '2024-09-19T07:43:59.598514+0000', 'last_peered': '2024-09-19T07:43:59.598514+0000', 'last_clean': '2024-09-19T07:43:59.598514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:43:59.598514+0000', 'last_undegraded': '2024-09-19T07:43:59.598514+0000', 'last_fullsized': '2024-09-19T07:43:59.598514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_clean_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.790 INFO:tasks.ceph.ceph_manager.ceph:PG 6.14 is not active+clean 2024-09-19T07:44:05.790 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:44:00.613312+0000', 'last_change': '2024-09-19T07:44:00.613312+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.613312+0000', 'last_undegraded': '2024-09-19T07:44:00.613312+0000', 'last_fullsized': '2024-09-19T07:44:00.613312+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T07:44:05.790 INFO:tasks.ceph.ceph_manager.ceph:PG 6.15 is not active+clean 2024-09-19T07:44:05.791 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.791 INFO:tasks.ceph.ceph_manager.ceph:PG 5.16 is not active+clean 2024-09-19T07:44:05.791 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:43:59.598514+0000', 'last_change': '2024-09-19T07:43:59.598514+0000', 'last_active': '2024-09-19T07:43:59.598514+0000', 'last_peered': '2024-09-19T07:43:59.598514+0000', 'last_clean': '2024-09-19T07:43:59.598514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:43:59.598514+0000', 'last_undegraded': '2024-09-19T07:43:59.598514+0000', 'last_fullsized': '2024-09-19T07:43:59.598514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_clean_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.791 INFO:tasks.ceph.ceph_manager.ceph:PG 6.16 is not active+clean 2024-09-19T07:44:05.791 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.791 INFO:tasks.ceph.ceph_manager.ceph:PG 5.15 is not active+clean 2024-09-19T07:44:05.791 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:43:59.598514+0000', 'last_change': '2024-09-19T07:43:59.598514+0000', 'last_active': '2024-09-19T07:43:59.598514+0000', 'last_peered': '2024-09-19T07:43:59.598514+0000', 'last_clean': '2024-09-19T07:43:59.598514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:43:59.598514+0000', 'last_undegraded': '2024-09-19T07:43:59.598514+0000', 'last_fullsized': '2024-09-19T07:43:59.598514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_clean_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.791 INFO:tasks.ceph.ceph_manager.ceph:PG 6.17 is not active+clean 2024-09-19T07:44:05.791 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.791 INFO:tasks.ceph.ceph_manager.ceph:PG 6.10 is not active+clean 2024-09-19T07:44:05.791 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.792 INFO:tasks.ceph.ceph_manager.ceph:PG 6.12 is not active+clean 2024-09-19T07:44:05.792 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.792 INFO:tasks.ceph.ceph_manager.ceph:PG 5.11 is not active+clean 2024-09-19T07:44:05.792 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:43:59.598514+0000', 'last_change': '2024-09-19T07:43:59.598514+0000', 'last_active': '2024-09-19T07:43:59.598514+0000', 'last_peered': '2024-09-19T07:43:59.598514+0000', 'last_clean': '2024-09-19T07:43:59.598514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:43:59.598514+0000', 'last_undegraded': '2024-09-19T07:43:59.598514+0000', 'last_fullsized': '2024-09-19T07:43:59.598514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_clean_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.792 INFO:tasks.ceph.ceph_manager.ceph:PG 5.10 is not active+clean 2024-09-19T07:44:05.792 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:43:59.598514+0000', 'last_change': '2024-09-19T07:43:59.598514+0000', 'last_active': '2024-09-19T07:43:59.598514+0000', 'last_peered': '2024-09-19T07:43:59.598514+0000', 'last_clean': '2024-09-19T07:43:59.598514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:43:59.598514+0000', 'last_undegraded': '2024-09-19T07:43:59.598514+0000', 'last_fullsized': '2024-09-19T07:43:59.598514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_clean_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.792 INFO:tasks.ceph.ceph_manager.ceph:PG 5.f is not active+clean 2024-09-19T07:44:05.792 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:43:59.598514+0000', 'last_change': '2024-09-19T07:43:59.598514+0000', 'last_active': '2024-09-19T07:43:59.598514+0000', 'last_peered': '2024-09-19T07:43:59.598514+0000', 'last_clean': '2024-09-19T07:43:59.598514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:43:59.598514+0000', 'last_undegraded': '2024-09-19T07:43:59.598514+0000', 'last_fullsized': '2024-09-19T07:43:59.598514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_clean_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.792 INFO:tasks.ceph.ceph_manager.ceph:PG 6.d is not active+clean 2024-09-19T07:44:05.792 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.792 INFO:tasks.ceph.ceph_manager.ceph:PG 5.e is not active+clean 2024-09-19T07:44:05.792 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:43:59.598514+0000', 'last_change': '2024-09-19T07:43:59.598514+0000', 'last_active': '2024-09-19T07:43:59.598514+0000', 'last_peered': '2024-09-19T07:43:59.598514+0000', 'last_clean': '2024-09-19T07:43:59.598514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:43:59.598514+0000', 'last_undegraded': '2024-09-19T07:43:59.598514+0000', 'last_fullsized': '2024-09-19T07:43:59.598514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_clean_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.793 INFO:tasks.ceph.ceph_manager.ceph:PG 6.e is not active+clean 2024-09-19T07:44:05.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.793 INFO:tasks.ceph.ceph_manager.ceph:PG 6.f is not active+clean 2024-09-19T07:44:05.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:44:00.613328+0000', 'last_change': '2024-09-19T07:44:00.613328+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.613328+0000', 'last_undegraded': '2024-09-19T07:44:00.613328+0000', 'last_fullsized': '2024-09-19T07:44:00.613328+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T07:44:05.793 INFO:tasks.ceph.ceph_manager.ceph:PG 6.8 is not active+clean 2024-09-19T07:44:05.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.793 INFO:tasks.ceph.ceph_manager.ceph:PG 6.9 is not active+clean 2024-09-19T07:44:05.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.793 INFO:tasks.ceph.ceph_manager.ceph:PG 6.a is not active+clean 2024-09-19T07:44:05.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.794 INFO:tasks.ceph.ceph_manager.ceph:PG 5.9 is not active+clean 2024-09-19T07:44:05.794 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:43:59.598514+0000', 'last_change': '2024-09-19T07:43:59.598514+0000', 'last_active': '2024-09-19T07:43:59.598514+0000', 'last_peered': '2024-09-19T07:43:59.598514+0000', 'last_clean': '2024-09-19T07:43:59.598514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:43:59.598514+0000', 'last_undegraded': '2024-09-19T07:43:59.598514+0000', 'last_fullsized': '2024-09-19T07:43:59.598514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_clean_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.794 INFO:tasks.ceph.ceph_manager.ceph:PG 5.7 is not active+clean 2024-09-19T07:44:05.794 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:43:59.598514+0000', 'last_change': '2024-09-19T07:43:59.598514+0000', 'last_active': '2024-09-19T07:43:59.598514+0000', 'last_peered': '2024-09-19T07:43:59.598514+0000', 'last_clean': '2024-09-19T07:43:59.598514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:43:59.598514+0000', 'last_undegraded': '2024-09-19T07:43:59.598514+0000', 'last_fullsized': '2024-09-19T07:43:59.598514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_clean_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.794 INFO:tasks.ceph.ceph_manager.ceph:PG 6.5 is not active+clean 2024-09-19T07:44:05.794 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.794 INFO:tasks.ceph.ceph_manager.ceph:PG 5.5 is not active+clean 2024-09-19T07:44:05.794 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:43:59.598514+0000', 'last_change': '2024-09-19T07:43:59.598514+0000', 'last_active': '2024-09-19T07:43:59.598514+0000', 'last_peered': '2024-09-19T07:43:59.598514+0000', 'last_clean': '2024-09-19T07:43:59.598514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:43:59.598514+0000', 'last_undegraded': '2024-09-19T07:43:59.598514+0000', 'last_fullsized': '2024-09-19T07:43:59.598514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_clean_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.794 INFO:tasks.ceph.ceph_manager.ceph:PG 6.0 is not active+clean 2024-09-19T07:44:05.794 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.794 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3 is not active+clean 2024-09-19T07:44:05.795 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:43:59.598514+0000', 'last_change': '2024-09-19T07:43:59.598514+0000', 'last_active': '2024-09-19T07:43:59.598514+0000', 'last_peered': '2024-09-19T07:43:59.598514+0000', 'last_clean': '2024-09-19T07:43:59.598514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:43:59.598514+0000', 'last_undegraded': '2024-09-19T07:43:59.598514+0000', 'last_fullsized': '2024-09-19T07:43:59.598514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_clean_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.795 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2 is not active+clean 2024-09-19T07:44:05.795 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:43:59.598514+0000', 'last_change': '2024-09-19T07:43:59.598514+0000', 'last_active': '2024-09-19T07:43:59.598514+0000', 'last_peered': '2024-09-19T07:43:59.598514+0000', 'last_clean': '2024-09-19T07:43:59.598514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:43:59.598514+0000', 'last_undegraded': '2024-09-19T07:43:59.598514+0000', 'last_fullsized': '2024-09-19T07:43:59.598514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_clean_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.795 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2 is not active+clean 2024-09-19T07:44:05.795 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.795 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1 is not active+clean 2024-09-19T07:44:05.795 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:43:59.598514+0000', 'last_change': '2024-09-19T07:43:59.598514+0000', 'last_active': '2024-09-19T07:43:59.598514+0000', 'last_peered': '2024-09-19T07:43:59.598514+0000', 'last_clean': '2024-09-19T07:43:59.598514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:43:59.598514+0000', 'last_undegraded': '2024-09-19T07:43:59.598514+0000', 'last_fullsized': '2024-09-19T07:43:59.598514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_clean_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.795 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3 is not active+clean 2024-09-19T07:44:05.795 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.795 INFO:tasks.ceph.ceph_manager.ceph:PG 6.7 is not active+clean 2024-09-19T07:44:05.795 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:44:00.603665+0000', 'last_change': '2024-09-19T07:44:00.603665+0000', 'last_active': '2024-09-19T07:44:00.603665+0000', 'last_peered': '2024-09-19T07:44:00.603665+0000', 'last_clean': '2024-09-19T07:44:00.603665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:44:00.603665+0000', 'last_undegraded': '2024-09-19T07:44:00.603665+0000', 'last_fullsized': '2024-09-19T07:44:00.603665+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'last_clean_scrub_stamp': '2024-09-19T07:44:00.603665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.796 INFO:tasks.ceph.ceph_manager.ceph:PG 5.4 is not active+clean 2024-09-19T07:44:05.796 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:43:59.598514+0000', 'last_change': '2024-09-19T07:43:59.598514+0000', 'last_active': '2024-09-19T07:43:59.598514+0000', 'last_peered': '2024-09-19T07:43:59.598514+0000', 'last_clean': '2024-09-19T07:43:59.598514+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:43:59.598514+0000', 'last_undegraded': '2024-09-19T07:43:59.598514+0000', 'last_fullsized': '2024-09-19T07:43:59.598514+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'last_clean_scrub_stamp': '2024-09-19T07:43:59.598514+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:44:05.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T07:44:05.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.921+0000 7f24a3853640 1 Processor -- start 2024-09-19T07:44:05.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.921+0000 7f24a3853640 1 -- start start 2024-09-19T07:44:05.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.922+0000 7f24a3853640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f249c15e2a0 0x7f249c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:05.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.922+0000 7f24a3853640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f249c06d960 0x7f249c0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:05.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.922+0000 7f24a3853640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f249c0d1880 con 0x7f249c15e680 2024-09-19T07:44:05.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.922+0000 7f24a3853640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f249c0d1b60 con 0x7f249c06d960 2024-09-19T07:44:05.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.922+0000 7f24a3853640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f249c0cf2d0 con 0x7f249c15e2a0 2024-09-19T07:44:05.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.922+0000 7f24a1df0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f249c06d960 0x7f249c0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:05.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.922+0000 7f24a0dee640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f249c15e680 0x7f249c0cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44504/0 (socket says 172.21.15.16:44504) 2024-09-19T07:44:05.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.922+0000 7f24a0dee640 1 -- 172.21.15.16:0/2889339696 learned_addr learned my addr 172.21.15.16:0/2889339696 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:44:05.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.922+0000 7f24a15ef640 1 --2- 172.21.15.16:0/2889339696 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f249c15e2a0 0x7f249c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:05.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.922+0000 7f24a1df0640 1 -- 172.21.15.16:0/2889339696 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f249c15e2a0 msgr2=0x7f249c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:05.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.922+0000 7f24a1df0640 1 --2- 172.21.15.16:0/2889339696 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f249c15e2a0 0x7f249c06d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.922+0000 7f24a1df0640 1 -- 172.21.15.16:0/2889339696 >> v1:172.21.15.16:6789/0 conn(0x7f249c15e680 legacy=0x7f249c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.922+0000 7f24a1df0640 1 -- 172.21.15.16:0/2889339696 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249c0cf5b0 con 0x7f249c06d960 2024-09-19T07:44:05.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.922+0000 7f249bfff640 1 -- 172.21.15.16:0/2889339696 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4206270893 0 0) 0x7f249c0d1880 con 0x7f249c15e680 2024-09-19T07:44:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.922+0000 7f24a15ef640 1 --2- 172.21.15.16:0/2889339696 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f249c15e2a0 0x7f249c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.923+0000 7f24a1df0640 1 --2- 172.21.15.16:0/2889339696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c06d960 0x7f249c0d4370 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f24940097a0 tx=0x7f249402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d26587f0c75c40 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.923+0000 7f249bfff640 1 -- 172.21.15.16:0/2889339696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2494037070 con 0x7f249c06d960 2024-09-19T07:44:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.923+0000 7f249bfff640 1 -- 172.21.15.16:0/2889339696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2494032440 con 0x7f249c06d960 2024-09-19T07:44:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.923+0000 7f249bfff640 1 -- 172.21.15.16:0/2889339696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2494032810 con 0x7f249c06d960 2024-09-19T07:44:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.924+0000 7f24a3853640 1 -- 172.21.15.16:0/2889339696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c06d960 msgr2=0x7f249c0d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.924+0000 7f24a3853640 1 --2- 172.21.15.16:0/2889339696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c06d960 0x7f249c0d4370 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f24940097a0 tx=0x7f249402a1a0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.924+0000 7f24a3853640 1 -- 172.21.15.16:0/2889339696 shutdown_connections 2024-09-19T07:44:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.924+0000 7f24a3853640 1 --2- 172.21.15.16:0/2889339696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c06d960 0x7f249c0d4370 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.924+0000 7f24a3853640 1 --2- 172.21.15.16:0/2889339696 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f249c15e2a0 0x7f249c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.924+0000 7f24a3853640 1 -- 172.21.15.16:0/2889339696 >> 172.21.15.16:0/2889339696 conn(0x7f249c09d760 msgr2=0x7f249c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.924+0000 7f24a3853640 1 -- 172.21.15.16:0/2889339696 shutdown_connections 2024-09-19T07:44:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.924+0000 7f24a3853640 1 -- 172.21.15.16:0/2889339696 wait complete. 2024-09-19T07:44:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.925+0000 7f24a3853640 1 Processor -- start 2024-09-19T07:44:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.925+0000 7f24a3853640 1 -- start start 2024-09-19T07:44:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.926+0000 7f24a3853640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f249c06d960 0x7f249c1f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.926+0000 7f24a3853640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c15e2a0 0x7f249c0ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.926+0000 7f24a3853640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f249c15e680 0x7f249c160f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.926+0000 7f24a3853640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f249c0d0360 con 0x7f249c15e2a0 2024-09-19T07:44:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.926+0000 7f24a3853640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f249c0d01e0 con 0x7f249c15e680 2024-09-19T07:44:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.926+0000 7f24a3853640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f249c0d48b0 con 0x7f249c06d960 2024-09-19T07:44:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.926+0000 7f24a0dee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c15e2a0 0x7f249c0ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.926+0000 7f24a0dee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c15e2a0 0x7f249c0ca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51748/0 (socket says 172.21.15.16:51748) 2024-09-19T07:44:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.926+0000 7f24a0dee640 1 -- 172.21.15.16:0/796557820 learned_addr learned my addr 172.21.15.16:0/796557820 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.926+0000 7f24a15ef640 1 --2- 172.21.15.16:0/796557820 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f249c06d960 0x7f249c1f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.926+0000 7f24a1df0640 1 --2- 172.21.15.16:0/796557820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f249c15e680 0x7f249c160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.926+0000 7f24a0dee640 1 -- 172.21.15.16:0/796557820 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f249c06d960 msgr2=0x7f249c1f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.926+0000 7f24a0dee640 1 --2- 172.21.15.16:0/796557820 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f249c06d960 0x7f249c1f49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.926+0000 7f24a0dee640 1 -- 172.21.15.16:0/796557820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f249c15e680 msgr2=0x7f249c160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.926+0000 7f24a0dee640 1 --2- 172.21.15.16:0/796557820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f249c15e680 0x7f249c160f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.926+0000 7f24a0dee640 1 -- 172.21.15.16:0/796557820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249c1f4f30 con 0x7f249c15e2a0 2024-09-19T07:44:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.926+0000 7f24a15ef640 1 --2- 172.21.15.16:0/796557820 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f249c06d960 0x7f249c1f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:44:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.926+0000 7f24a1df0640 1 --2- 172.21.15.16:0/796557820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f249c15e680 0x7f249c160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:44:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.926+0000 7f24a0dee640 1 --2- 172.21.15.16:0/796557820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c15e2a0 0x7f249c0ca5f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f249000c410 tx=0x7f2490001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.927+0000 7f2499ffb640 1 -- 172.21.15.16:0/796557820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24900074e0 con 0x7f249c15e2a0 2024-09-19T07:44:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.927+0000 7f2499ffb640 1 -- 172.21.15.16:0/796557820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24900047e0 con 0x7f249c15e2a0 2024-09-19T07:44:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.927+0000 7f24a3853640 1 -- 172.21.15.16:0/796557820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f249c14c150 con 0x7f249c15e2a0 2024-09-19T07:44:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.927+0000 7f2499ffb640 1 -- 172.21.15.16:0/796557820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490004bd0 con 0x7f249c15e2a0 2024-09-19T07:44:05.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.927+0000 7f24a3853640 1 -- 172.21.15.16:0/796557820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f249c0b84b0 con 0x7f249c15e2a0 2024-09-19T07:44:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.928+0000 7f24a3853640 1 -- 172.21.15.16:0/796557820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f249c0c7c90 con 0x7f249c15e2a0 2024-09-19T07:44:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.928+0000 7f2499ffb640 1 -- 172.21.15.16:0/796557820 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2490011070 con 0x7f249c15e2a0 2024-09-19T07:44:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.929+0000 7f2499ffb640 1 --2- 172.21.15.16:0/796557820 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f248806c2b0 0x7f248806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.929+0000 7f2499ffb640 1 -- 172.21.15.16:0/796557820 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f249008b190 con 0x7f249c15e2a0 2024-09-19T07:44:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.929+0000 7f24a15ef640 1 --2- 172.21.15.16:0/796557820 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f248806c2b0 0x7f248806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.929+0000 7f24a15ef640 1 --2- 172.21.15.16:0/796557820 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f248806c2b0 0x7f248806e770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f248c0094c0 tx=0x7f248c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:05.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:05.933+0000 7f2499ffb640 1 -- 172.21.15.16:0/796557820 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f249c0c7c90 con 0x7f249c15e2a0 2024-09-19T07:44:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.175+0000 7f24a3853640 1 -- 172.21.15.16:0/796557820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f249c0d4c70 con 0x7f249c15e2a0 2024-09-19T07:44:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.175+0000 7f2499ffb640 1 -- 172.21.15.16:0/796557820 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1763 (secure 0 0 0) 0x7f249c1f4f30 con 0x7f249c15e2a0 2024-09-19T07:44:06.176 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:44:06.176 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":83,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":40,"num_osds":8,"num_up_osds":8,"osd_up_since":1726731773,"num_in_osds":8,"osd_in_since":1726731764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":61},{"state_name":"active+clean","count":30},{"state_name":"creating+peering","count":6}],"num_pgs":97,"num_pools":3,"num_objects":9,"data_bytes":592014,"bytes_used":243060736,"bytes_avail":772851052544,"bytes_total":773094113280,"unknown_pgs_ratio":0.62886595726013184,"inactive_pgs_ratio":0.061855670064687729,"write_bytes_sec":2104,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":16,"btime":"2024-09-19T07:44:04:108666+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":4947}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-19T07:43:56.470692+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T07:44:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.177+0000 7f24a3853640 1 -- 172.21.15.16:0/796557820 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f248806c2b0 msgr2=0x7f248806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.177+0000 7f24a3853640 1 --2- 172.21.15.16:0/796557820 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f248806c2b0 0x7f248806e770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f248c0094c0 tx=0x7f248c00b040 comp rx=0 tx=0).stop 2024-09-19T07:44:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.177+0000 7f24a3853640 1 -- 172.21.15.16:0/796557820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c15e2a0 msgr2=0x7f249c0ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.177+0000 7f24a3853640 1 --2- 172.21.15.16:0/796557820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c15e2a0 0x7f249c0ca5f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f249000c410 tx=0x7f2490001e70 comp rx=0 tx=0).stop 2024-09-19T07:44:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.177+0000 7f24a3853640 1 -- 172.21.15.16:0/796557820 shutdown_connections 2024-09-19T07:44:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.177+0000 7f24a3853640 1 --2- 172.21.15.16:0/796557820 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f248806c2b0 0x7f248806e770 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.177+0000 7f24a3853640 1 --2- 172.21.15.16:0/796557820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f249c15e680 0x7f249c160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.177+0000 7f24a3853640 1 --2- 172.21.15.16:0/796557820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249c15e2a0 0x7f249c0ca5f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.177+0000 7f24a3853640 1 --2- 172.21.15.16:0/796557820 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f249c06d960 0x7f249c1f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.177+0000 7f24a3853640 1 -- 172.21.15.16:0/796557820 >> 172.21.15.16:0/796557820 conn(0x7f249c09d760 msgr2=0x7f249c15f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.177+0000 7f24a3853640 1 -- 172.21.15.16:0/796557820 shutdown_connections 2024-09-19T07:44:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.177+0000 7f24a3853640 1 -- 172.21.15.16:0/796557820 wait complete. 2024-09-19T07:44:06.188 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-19T07:44:06.188 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:44:06.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.290+0000 7fcac6e74640 1 Processor -- start 2024-09-19T07:44:06.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.290+0000 7fcac6e74640 1 -- start start 2024-09-19T07:44:06.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.291+0000 7fcac6e74640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcac00d0540 0x7fcac0141a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:06.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.291+0000 7fcac6e74640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcac0141fa0 0x7fcac013f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:06.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.291+0000 7fcac6e74640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcac015d710 con 0x7fcac0141680 2024-09-19T07:44:06.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.291+0000 7fcac6e74640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fcac015c6f0 con 0x7fcac0141fa0 2024-09-19T07:44:06.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.291+0000 7fcac6e74640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcac015c950 con 0x7fcac00d0540 2024-09-19T07:44:06.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.291+0000 7fcac5411640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcac0141fa0 0x7fcac013f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:06.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.291+0000 7fcac5411640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcac0141fa0 0x7fcac013f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51756/0 (socket says 172.21.15.16:51756) 2024-09-19T07:44:06.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.291+0000 7fcac5411640 1 -- 172.21.15.16:0/54781740 learned_addr learned my addr 172.21.15.16:0/54781740 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:06.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.291+0000 7fcabffff640 1 --2- 172.21.15.16:0/54781740 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcac00d0540 0x7fcac0141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:06.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.291+0000 7fcac5411640 1 -- 172.21.15.16:0/54781740 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcac00d0540 msgr2=0x7fcac0141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:06.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.291+0000 7fcac5411640 1 --2- 172.21.15.16:0/54781740 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcac00d0540 0x7fcac0141a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:06.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.291+0000 7fcac5411640 1 -- 172.21.15.16:0/54781740 >> v1:172.21.15.146:6789/0 conn(0x7fcac0141680 legacy=0x7fcac00cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:06.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.291+0000 7fcac5411640 1 -- 172.21.15.16:0/54781740 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac015cc30 con 0x7fcac0141fa0 2024-09-19T07:44:06.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.292+0000 7fcac5411640 1 --2- 172.21.15.16:0/54781740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcac0141fa0 0x7fcac013f8b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009e50 tx=0x7fcab802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e646d062cf38e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:06.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.292+0000 7fcabf7fe640 1 -- 172.21.15.16:0/54781740 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8036070 con 0x7fcac0141fa0 2024-09-19T07:44:06.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.292+0000 7fcabf7fe640 1 -- 172.21.15.16:0/54781740 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcab8031600 con 0x7fcac0141fa0 2024-09-19T07:44:06.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.292+0000 7fcabf7fe640 1 -- 172.21.15.16:0/54781740 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab80319d0 con 0x7fcac0141fa0 2024-09-19T07:44:06.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.292+0000 7fcac6e74640 1 -- 172.21.15.16:0/54781740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcac0141fa0 msgr2=0x7fcac013f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:06.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.292+0000 7fcac6e74640 1 --2- 172.21.15.16:0/54781740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcac0141fa0 0x7fcac013f8b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009e50 tx=0x7fcab802a710 comp rx=0 tx=0).stop 2024-09-19T07:44:06.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.293+0000 7fcac6e74640 1 -- 172.21.15.16:0/54781740 shutdown_connections 2024-09-19T07:44:06.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.293+0000 7fcac6e74640 1 --2- 172.21.15.16:0/54781740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcac0141fa0 0x7fcac013f8b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:06.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.293+0000 7fcac6e74640 1 --2- 172.21.15.16:0/54781740 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcac00d0540 0x7fcac0141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:06.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.293+0000 7fcac6e74640 1 -- 172.21.15.16:0/54781740 >> 172.21.15.16:0/54781740 conn(0x7fcac00b3ed0 msgr2=0x7fcac00b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:06.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.293+0000 7fcac6e74640 1 -- 172.21.15.16:0/54781740 shutdown_connections 2024-09-19T07:44:06.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.293+0000 7fcac6e74640 1 -- 172.21.15.16:0/54781740 wait complete. 2024-09-19T07:44:06.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.294+0000 7fcac6e74640 1 Processor -- start 2024-09-19T07:44:06.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.294+0000 7fcac6e74640 1 -- start start 2024-09-19T07:44:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.295+0000 7fcac6e74640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcac00d0540 0x7fcac00c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.295+0000 7fcac6e74640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcac0141680 0x7fcac006d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.295+0000 7fcac6e74640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac0141fa0 0x7fcac00cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.295+0000 7fcac6e74640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcac0142430 con 0x7fcac00d0540 2024-09-19T07:44:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.295+0000 7fcac6e74640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcac015d1f0 con 0x7fcac0141fa0 2024-09-19T07:44:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.295+0000 7fcac6e74640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcac01425b0 con 0x7fcac0141680 2024-09-19T07:44:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.295+0000 7fcac4c10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcac00d0540 0x7fcac00c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.295+0000 7fcac4c10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcac00d0540 0x7fcac00c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51758/0 (socket says 172.21.15.16:51758) 2024-09-19T07:44:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.295+0000 7fcac4c10640 1 -- 172.21.15.16:0/1652429841 learned_addr learned my addr 172.21.15.16:0/1652429841 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.295+0000 7fcac4c10640 1 -- 172.21.15.16:0/1652429841 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcac0141680 msgr2=0x7fcac006d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.295+0000 7fcac4c10640 1 --2- 172.21.15.16:0/1652429841 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcac0141680 0x7fcac006d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.295+0000 7fcac4c10640 1 -- 172.21.15.16:0/1652429841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac0141fa0 msgr2=0x7fcac00cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.295+0000 7fcac5411640 1 --2- 172.21.15.16:0/1652429841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac0141fa0 0x7fcac00cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.295+0000 7fcac4c10640 1 --2- 172.21.15.16:0/1652429841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac0141fa0 0x7fcac00cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.295+0000 7fcac4c10640 1 -- 172.21.15.16:0/1652429841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac00cbd60 con 0x7fcac00d0540 2024-09-19T07:44:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.295+0000 7fcac4c10640 1 --2- 172.21.15.16:0/1652429841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcac00d0540 0x7fcac00c98d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcab400c430 tx=0x7fcab400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.295+0000 7fcabd7fa640 1 -- 172.21.15.16:0/1652429841 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab40052d0 con 0x7fcac00d0540 2024-09-19T07:44:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.296+0000 7fcabd7fa640 1 -- 172.21.15.16:0/1652429841 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcab4005470 con 0x7fcac00d0540 2024-09-19T07:44:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.296+0000 7fcabd7fa640 1 -- 172.21.15.16:0/1652429841 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4007260 con 0x7fcac00d0540 2024-09-19T07:44:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.296+0000 7fcac6e74640 1 -- 172.21.15.16:0/1652429841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac00cc040 con 0x7fcac00d0540 2024-09-19T07:44:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.296+0000 7fcac6e74640 1 -- 172.21.15.16:0/1652429841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac006d210 con 0x7fcac00d0540 2024-09-19T07:44:06.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.297+0000 7fcac6e74640 1 -- 172.21.15.16:0/1652429841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcac006cd20 con 0x7fcac00d0540 2024-09-19T07:44:06.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.297+0000 7fcabd7fa640 1 -- 172.21.15.16:0/1652429841 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcab4007400 con 0x7fcac00d0540 2024-09-19T07:44:06.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.297+0000 7fcabd7fa640 1 --2- 172.21.15.16:0/1652429841 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcaa406c2b0 0x7fcaa406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:06.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.298+0000 7fcabd7fa640 1 -- 172.21.15.16:0/1652429841 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fcab408aa70 con 0x7fcac00d0540 2024-09-19T07:44:06.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.298+0000 7fcabffff640 1 --2- 172.21.15.16:0/1652429841 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcaa406c2b0 0x7fcaa406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:06.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.298+0000 7fcabffff640 1 --2- 172.21.15.16:0/1652429841 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcaa406c2b0 0x7fcaa406e770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcaac006fd0 tx=0x7fcaac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:06.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.302+0000 7fcabd7fa640 1 -- 172.21.15.16:0/1652429841 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fcab40590e0 con 0x7fcac00d0540 2024-09-19T07:44:06.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.476+0000 7fcac6e74640 1 -- 172.21.15.16:0/1652429841 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcac0141a60 con 0x7fcaa406c2b0 2024-09-19T07:44:06.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.483+0000 7fcabd7fa640 1 -- 172.21.15.16:0/1652429841 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251457 (secure 0 0 0) 0x7fcac0141a60 con 0x7fcaa406c2b0 2024-09-19T07:44:06.486 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:44:06.487 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:44:06.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.486+0000 7fcac6e74640 1 -- 172.21.15.16:0/1652429841 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcaa406c2b0 msgr2=0x7fcaa406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:06.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.486+0000 7fcac6e74640 1 --2- 172.21.15.16:0/1652429841 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcaa406c2b0 0x7fcaa406e770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcaac006fd0 tx=0x7fcaac008040 comp rx=0 tx=0).stop 2024-09-19T07:44:06.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.486+0000 7fcac6e74640 1 -- 172.21.15.16:0/1652429841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcac00d0540 msgr2=0x7fcac00c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:06.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.486+0000 7fcac6e74640 1 --2- 172.21.15.16:0/1652429841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcac00d0540 0x7fcac00c98d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcab400c430 tx=0x7fcab400c8f0 comp rx=0 tx=0).stop 2024-09-19T07:44:06.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.486+0000 7fcac6e74640 1 -- 172.21.15.16:0/1652429841 shutdown_connections 2024-09-19T07:44:06.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.486+0000 7fcac6e74640 1 --2- 172.21.15.16:0/1652429841 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcaa406c2b0 0x7fcaa406e770 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:06.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.486+0000 7fcac6e74640 1 --2- 172.21.15.16:0/1652429841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac0141fa0 0x7fcac00cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:06.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.486+0000 7fcac6e74640 1 --2- 172.21.15.16:0/1652429841 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcac0141680 0x7fcac006d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:06.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.486+0000 7fcac6e74640 1 --2- 172.21.15.16:0/1652429841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcac00d0540 0x7fcac00c98d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:06.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.486+0000 7fcac6e74640 1 -- 172.21.15.16:0/1652429841 >> 172.21.15.16:0/1652429841 conn(0x7fcac00b3ed0 msgr2=0x7fcac00b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:06.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.486+0000 7fcac6e74640 1 -- 172.21.15.16:0/1652429841 shutdown_connections 2024-09-19T07:44:06.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:06.486+0000 7fcac6e74640 1 -- 172.21.15.16:0/1652429841 wait complete. 2024-09-19T07:44:06.497 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-19T07:44:06.472576+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":201,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237616,"kb_used_data":4736,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737104,"statfs":{"total":773094113280,"available":772850794496,"internally_reserved":0,"allocated":4849664,"data_stored":2734743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001044"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633184+0000","last_change":"2024-09-19T07:44:01.635139+0000","last_active":"2024-09-19T07:44:02.633184+0000","last_peered":"2024-09-19T07:44:02.633184+0000","last_clean":"2024-09-19T07:44:02.633184+0000","last_became_active":"2024-09-19T07:44:01.634842+0000","last_became_peered":"2024-09-19T07:44:01.634842+0000","last_unstale":"2024-09-19T07:44:02.633184+0000","last_undegraded":"2024-09-19T07:44:02.633184+0000","last_fullsized":"2024-09-19T07:44:02.633184+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:07.759986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633030+0000","last_change":"2024-09-19T07:44:01.629931+0000","last_active":"2024-09-19T07:44:02.633030+0000","last_peered":"2024-09-19T07:44:02.633030+0000","last_clean":"2024-09-19T07:44:02.633030+0000","last_became_active":"2024-09-19T07:44:01.629634+0000","last_became_peered":"2024-09-19T07:44:01.629634+0000","last_unstale":"2024-09-19T07:44:02.633030+0000","last_undegraded":"2024-09-19T07:44:02.633030+0000","last_fullsized":"2024-09-19T07:44:02.633030+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:00.266417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879583+0000","last_change":"2024-09-19T07:44:01.635942+0000","last_active":"2024-09-19T07:44:02.879583+0000","last_peered":"2024-09-19T07:44:02.879583+0000","last_clean":"2024-09-19T07:44:02.879583+0000","last_became_active":"2024-09-19T07:44:01.635639+0000","last_became_peered":"2024-09-19T07:44:01.635639+0000","last_unstale":"2024-09-19T07:44:02.879583+0000","last_undegraded":"2024-09-19T07:44:02.879583+0000","last_fullsized":"2024-09-19T07:44:02.879583+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:18.456608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.619008+0000","last_change":"2024-09-19T07:44:00.619008+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.619008+0000","last_undegraded":"2024-09-19T07:44:00.619008+0000","last_fullsized":"2024-09-19T07:44:00.619008+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.619651+0000","last_change":"2024-09-19T07:44:00.619651+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.619651+0000","last_undegraded":"2024-09-19T07:44:00.619651+0000","last_fullsized":"2024-09-19T07:44:00.619651+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880437+0000","last_change":"2024-09-19T07:44:01.635795+0000","last_active":"2024-09-19T07:44:02.880437+0000","last_peered":"2024-09-19T07:44:02.880437+0000","last_clean":"2024-09-19T07:44:02.880437+0000","last_became_active":"2024-09-19T07:44:01.634728+0000","last_became_peered":"2024-09-19T07:44:01.634728+0000","last_unstale":"2024-09-19T07:44:02.880437+0000","last_undegraded":"2024-09-19T07:44:02.880437+0000","last_fullsized":"2024-09-19T07:44:02.880437+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:20.583756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636513+0000","last_change":"2024-09-19T07:44:01.641177+0000","last_active":"2024-09-19T07:44:02.636513+0000","last_peered":"2024-09-19T07:44:02.636513+0000","last_clean":"2024-09-19T07:44:02.636513+0000","last_became_active":"2024-09-19T07:44:01.637104+0000","last_became_peered":"2024-09-19T07:44:01.637104+0000","last_unstale":"2024-09-19T07:44:02.636513+0000","last_undegraded":"2024-09-19T07:44:02.636513+0000","last_fullsized":"2024-09-19T07:44:02.636513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:47:46.856860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632612+0000","last_change":"2024-09-19T07:44:01.621511+0000","last_active":"2024-09-19T07:44:02.632612+0000","last_peered":"2024-09-19T07:44:02.632612+0000","last_clean":"2024-09-19T07:44:02.632612+0000","last_became_active":"2024-09-19T07:44:01.620511+0000","last_became_peered":"2024-09-19T07:44:01.620511+0000","last_unstale":"2024-09-19T07:44:02.632612+0000","last_undegraded":"2024-09-19T07:44:02.632612+0000","last_fullsized":"2024-09-19T07:44:02.632612+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:57.716510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879610+0000","last_change":"2024-09-19T07:44:01.635157+0000","last_active":"2024-09-19T07:44:02.879610+0000","last_peered":"2024-09-19T07:44:02.879610+0000","last_clean":"2024-09-19T07:44:02.879610+0000","last_became_active":"2024-09-19T07:44:01.634821+0000","last_became_peered":"2024-09-19T07:44:01.634821+0000","last_unstale":"2024-09-19T07:44:02.879610+0000","last_undegraded":"2024-09-19T07:44:02.879610+0000","last_fullsized":"2024-09-19T07:44:02.879610+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:11.328826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637663+0000","last_change":"2024-09-19T07:44:01.631689+0000","last_active":"2024-09-19T07:44:02.637663+0000","last_peered":"2024-09-19T07:44:02.637663+0000","last_clean":"2024-09-19T07:44:02.637663+0000","last_became_active":"2024-09-19T07:44:01.631389+0000","last_became_peered":"2024-09-19T07:44:01.631389+0000","last_unstale":"2024-09-19T07:44:02.637663+0000","last_undegraded":"2024-09-19T07:44:02.637663+0000","last_fullsized":"2024-09-19T07:44:02.637663+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:34.270347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637601+0000","last_change":"2024-09-19T07:44:01.620732+0000","last_active":"2024-09-19T07:44:02.637601+0000","last_peered":"2024-09-19T07:44:02.637601+0000","last_clean":"2024-09-19T07:44:02.637601+0000","last_became_active":"2024-09-19T07:44:01.620458+0000","last_became_peered":"2024-09-19T07:44:01.620458+0000","last_unstale":"2024-09-19T07:44:02.637601+0000","last_undegraded":"2024-09-19T07:44:02.637601+0000","last_fullsized":"2024-09-19T07:44:02.637601+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:34.596548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879452+0000","last_change":"2024-09-19T07:44:01.635852+0000","last_active":"2024-09-19T07:44:02.879452+0000","last_peered":"2024-09-19T07:44:02.879452+0000","last_clean":"2024-09-19T07:44:02.879452+0000","last_became_active":"2024-09-19T07:44:01.635563+0000","last_became_peered":"2024-09-19T07:44:01.635563+0000","last_unstale":"2024-09-19T07:44:02.879452+0000","last_undegraded":"2024-09-19T07:44:02.879452+0000","last_fullsized":"2024-09-19T07:44:02.879452+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:42.118230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637139+0000","last_change":"2024-09-19T07:44:01.630966+0000","last_active":"2024-09-19T07:44:02.637139+0000","last_peered":"2024-09-19T07:44:02.637139+0000","last_clean":"2024-09-19T07:44:02.637139+0000","last_became_active":"2024-09-19T07:44:01.630693+0000","last_became_peered":"2024-09-19T07:44:01.630693+0000","last_unstale":"2024-09-19T07:44:02.637139+0000","last_undegraded":"2024-09-19T07:44:02.637139+0000","last_fullsized":"2024-09-19T07:44:02.637139+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:57.716510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636697+0000","last_change":"2024-09-19T07:44:01.636530+0000","last_active":"2024-09-19T07:44:02.636697+0000","last_peered":"2024-09-19T07:44:02.636697+0000","last_clean":"2024-09-19T07:44:02.636697+0000","last_became_active":"2024-09-19T07:44:01.636084+0000","last_became_peered":"2024-09-19T07:44:01.636084+0000","last_unstale":"2024-09-19T07:44:02.636697+0000","last_undegraded":"2024-09-19T07:44:02.636697+0000","last_fullsized":"2024-09-19T07:44:02.636697+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:00.787345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636671+0000","last_change":"2024-09-19T07:44:01.641990+0000","last_active":"2024-09-19T07:44:02.636671+0000","last_peered":"2024-09-19T07:44:02.636671+0000","last_clean":"2024-09-19T07:44:02.636671+0000","last_became_active":"2024-09-19T07:44:01.641212+0000","last_became_peered":"2024-09-19T07:44:01.641212+0000","last_unstale":"2024-09-19T07:44:02.636671+0000","last_undegraded":"2024-09-19T07:44:02.636671+0000","last_fullsized":"2024-09-19T07:44:02.636671+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:19.575604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879128+0000","last_change":"2024-09-19T07:44:01.630670+0000","last_active":"2024-09-19T07:44:02.879128+0000","last_peered":"2024-09-19T07:44:02.879128+0000","last_clean":"2024-09-19T07:44:02.879128+0000","last_became_active":"2024-09-19T07:44:01.630419+0000","last_became_peered":"2024-09-19T07:44:01.630419+0000","last_unstale":"2024-09-19T07:44:02.879128+0000","last_undegraded":"2024-09-19T07:44:02.879128+0000","last_fullsized":"2024-09-19T07:44:02.879128+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:15:37.153759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879517+0000","last_change":"2024-09-19T07:44:01.631309+0000","last_active":"2024-09-19T07:44:02.879517+0000","last_peered":"2024-09-19T07:44:02.879517+0000","last_clean":"2024-09-19T07:44:02.879517+0000","last_became_active":"2024-09-19T07:44:01.631048+0000","last_became_peered":"2024-09-19T07:44:01.631048+0000","last_unstale":"2024-09-19T07:44:02.879517+0000","last_undegraded":"2024-09-19T07:44:02.879517+0000","last_fullsized":"2024-09-19T07:44:02.879517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:19.575604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.858000+0000","last_change":"2024-09-19T07:44:01.635852+0000","last_active":"2024-09-19T07:44:02.858000+0000","last_peered":"2024-09-19T07:44:02.858000+0000","last_clean":"2024-09-19T07:44:02.858000+0000","last_became_active":"2024-09-19T07:44:01.635487+0000","last_became_peered":"2024-09-19T07:44:01.635487+0000","last_unstale":"2024-09-19T07:44:02.858000+0000","last_undegraded":"2024-09-19T07:44:02.858000+0000","last_fullsized":"2024-09-19T07:44:02.858000+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:08.097213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.620031+0000","last_change":"2024-09-19T07:44:00.620031+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.620031+0000","last_undegraded":"2024-09-19T07:44:00.620031+0000","last_fullsized":"2024-09-19T07:44:00.620031+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.857873+0000","last_change":"2024-09-19T07:44:01.635914+0000","last_active":"2024-09-19T07:44:02.857873+0000","last_peered":"2024-09-19T07:44:02.857873+0000","last_clean":"2024-09-19T07:44:02.857873+0000","last_became_active":"2024-09-19T07:44:01.635626+0000","last_became_peered":"2024-09-19T07:44:01.635626+0000","last_unstale":"2024-09-19T07:44:02.857873+0000","last_undegraded":"2024-09-19T07:44:02.857873+0000","last_fullsized":"2024-09-19T07:44:02.857873+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:58.893344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637194+0000","last_change":"2024-09-19T07:44:01.634487+0000","last_active":"2024-09-19T07:44:02.637194+0000","last_peered":"2024-09-19T07:44:02.637194+0000","last_clean":"2024-09-19T07:44:02.637194+0000","last_became_active":"2024-09-19T07:44:01.634225+0000","last_became_peered":"2024-09-19T07:44:01.634225+0000","last_unstale":"2024-09-19T07:44:02.637194+0000","last_undegraded":"2024-09-19T07:44:02.637194+0000","last_fullsized":"2024-09-19T07:44:02.637194+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:26.448577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637004+0000","last_change":"2024-09-19T07:44:01.631807+0000","last_active":"2024-09-19T07:44:02.637004+0000","last_peered":"2024-09-19T07:44:02.637004+0000","last_clean":"2024-09-19T07:44:02.637004+0000","last_became_active":"2024-09-19T07:44:01.631515+0000","last_became_peered":"2024-09-19T07:44:01.631515+0000","last_unstale":"2024-09-19T07:44:02.637004+0000","last_undegraded":"2024-09-19T07:44:02.637004+0000","last_fullsized":"2024-09-19T07:44:02.637004+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:42.118230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879177+0000","last_change":"2024-09-19T07:44:01.630667+0000","last_active":"2024-09-19T07:44:02.879177+0000","last_peered":"2024-09-19T07:44:02.879177+0000","last_clean":"2024-09-19T07:44:02.879177+0000","last_became_active":"2024-09-19T07:44:01.630388+0000","last_became_peered":"2024-09-19T07:44:01.630388+0000","last_unstale":"2024-09-19T07:44:02.879177+0000","last_undegraded":"2024-09-19T07:44:02.879177+0000","last_fullsized":"2024-09-19T07:44:02.879177+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:57:21.017364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.618996+0000","last_change":"2024-09-19T07:44:00.618996+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.618996+0000","last_undegraded":"2024-09-19T07:44:00.618996+0000","last_fullsized":"2024-09-19T07:44:00.618996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.858062+0000","last_change":"2024-09-19T07:44:01.638722+0000","last_active":"2024-09-19T07:44:02.858062+0000","last_peered":"2024-09-19T07:44:02.858062+0000","last_clean":"2024-09-19T07:44:02.858062+0000","last_became_active":"2024-09-19T07:44:01.638450+0000","last_became_peered":"2024-09-19T07:44:01.638450+0000","last_unstale":"2024-09-19T07:44:02.858062+0000","last_undegraded":"2024-09-19T07:44:02.858062+0000","last_fullsized":"2024-09-19T07:44:02.858062+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:33.573198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879547+0000","last_change":"2024-09-19T07:44:01.632403+0000","last_active":"2024-09-19T07:44:02.879547+0000","last_peered":"2024-09-19T07:44:02.879547+0000","last_clean":"2024-09-19T07:44:02.879547+0000","last_became_active":"2024-09-19T07:44:01.632105+0000","last_became_peered":"2024-09-19T07:44:01.632105+0000","last_unstale":"2024-09-19T07:44:02.879547+0000","last_undegraded":"2024-09-19T07:44:02.879547+0000","last_fullsized":"2024-09-19T07:44:02.879547+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:22:00.022599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.858031+0000","last_change":"2024-09-19T07:44:01.640185+0000","last_active":"2024-09-19T07:44:02.858031+0000","last_peered":"2024-09-19T07:44:02.858031+0000","last_clean":"2024-09-19T07:44:02.858031+0000","last_became_active":"2024-09-19T07:44:01.639498+0000","last_became_peered":"2024-09-19T07:44:01.639498+0000","last_unstale":"2024-09-19T07:44:02.858031+0000","last_undegraded":"2024-09-19T07:44:02.858031+0000","last_fullsized":"2024-09-19T07:44:02.858031+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:29:52.280755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637457+0000","last_change":"2024-09-19T07:44:01.623150+0000","last_active":"2024-09-19T07:44:02.637457+0000","last_peered":"2024-09-19T07:44:02.637457+0000","last_clean":"2024-09-19T07:44:02.637457+0000","last_became_active":"2024-09-19T07:44:01.622344+0000","last_became_peered":"2024-09-19T07:44:01.622344+0000","last_unstale":"2024-09-19T07:44:02.637457+0000","last_undegraded":"2024-09-19T07:44:02.637457+0000","last_fullsized":"2024-09-19T07:44:02.637457+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:34.910477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.857867+0000","last_change":"2024-09-19T07:44:01.639571+0000","last_active":"2024-09-19T07:44:02.857867+0000","last_peered":"2024-09-19T07:44:02.857867+0000","last_clean":"2024-09-19T07:44:02.857867+0000","last_became_active":"2024-09-19T07:44:01.639259+0000","last_became_peered":"2024-09-19T07:44:01.639259+0000","last_unstale":"2024-09-19T07:44:02.857867+0000","last_undegraded":"2024-09-19T07:44:02.857867+0000","last_fullsized":"2024-09-19T07:44:02.857867+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:43.171945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.857808+0000","last_change":"2024-09-19T07:44:01.640084+0000","last_active":"2024-09-19T07:44:02.857808+0000","last_peered":"2024-09-19T07:44:02.857808+0000","last_clean":"2024-09-19T07:44:02.857808+0000","last_became_active":"2024-09-19T07:44:01.639804+0000","last_became_peered":"2024-09-19T07:44:01.639804+0000","last_unstale":"2024-09-19T07:44:02.857808+0000","last_undegraded":"2024-09-19T07:44:02.857808+0000","last_fullsized":"2024-09-19T07:44:02.857808+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:06:33.870781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879323+0000","last_change":"2024-09-19T07:44:01.633165+0000","last_active":"2024-09-19T07:44:02.879323+0000","last_peered":"2024-09-19T07:44:02.879323+0000","last_clean":"2024-09-19T07:44:02.879323+0000","last_became_active":"2024-09-19T07:44:01.632905+0000","last_became_peered":"2024-09-19T07:44:01.632905+0000","last_unstale":"2024-09-19T07:44:02.879323+0000","last_undegraded":"2024-09-19T07:44:02.879323+0000","last_fullsized":"2024-09-19T07:44:02.879323+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:45.305611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880311+0000","last_change":"2024-09-19T07:44:01.633044+0000","last_active":"2024-09-19T07:44:02.880311+0000","last_peered":"2024-09-19T07:44:02.880311+0000","last_clean":"2024-09-19T07:44:02.880311+0000","last_became_active":"2024-09-19T07:44:01.632761+0000","last_became_peered":"2024-09-19T07:44:01.632761+0000","last_unstale":"2024-09-19T07:44:02.880311+0000","last_undegraded":"2024-09-19T07:44:02.880311+0000","last_fullsized":"2024-09-19T07:44:02.880311+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:10.758189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879515+0000","last_change":"2024-09-19T07:44:01.633367+0000","last_active":"2024-09-19T07:44:02.879515+0000","last_peered":"2024-09-19T07:44:02.879515+0000","last_clean":"2024-09-19T07:44:02.879515+0000","last_became_active":"2024-09-19T07:44:01.632924+0000","last_became_peered":"2024-09-19T07:44:01.632924+0000","last_unstale":"2024-09-19T07:44:02.879515+0000","last_undegraded":"2024-09-19T07:44:02.879515+0000","last_fullsized":"2024-09-19T07:44:02.879515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:47:46.856860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.858098+0000","last_change":"2024-09-19T07:44:00.631265+0000","last_active":"2024-09-19T07:44:02.858098+0000","last_peered":"2024-09-19T07:44:02.858098+0000","last_clean":"2024-09-19T07:44:02.858098+0000","last_became_active":"2024-09-19T07:44:00.631003+0000","last_became_peered":"2024-09-19T07:44:00.631003+0000","last_unstale":"2024-09-19T07:44:02.858098+0000","last_undegraded":"2024-09-19T07:44:02.858098+0000","last_fullsized":"2024-09-19T07:44:02.858098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:29.392209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879930+0000","last_change":"2024-09-19T07:44:01.630912+0000","last_active":"2024-09-19T07:44:02.879930+0000","last_peered":"2024-09-19T07:44:02.879930+0000","last_clean":"2024-09-19T07:44:02.879930+0000","last_became_active":"2024-09-19T07:44:01.630625+0000","last_became_peered":"2024-09-19T07:44:01.630625+0000","last_unstale":"2024-09-19T07:44:02.879930+0000","last_undegraded":"2024-09-19T07:44:02.879930+0000","last_fullsized":"2024-09-19T07:44:02.879930+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:16.252351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879734+0000","last_change":"2024-09-19T07:44:00.628877+0000","last_active":"2024-09-19T07:44:02.879734+0000","last_peered":"2024-09-19T07:44:02.879734+0000","last_clean":"2024-09-19T07:44:02.879734+0000","last_became_active":"2024-09-19T07:44:00.628595+0000","last_became_peered":"2024-09-19T07:44:00.628595+0000","last_unstale":"2024-09-19T07:44:02.879734+0000","last_undegraded":"2024-09-19T07:44:02.879734+0000","last_fullsized":"2024-09-19T07:44:02.879734+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:58.958404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636651+0000","last_change":"2024-09-19T07:44:01.636642+0000","last_active":"2024-09-19T07:44:02.636651+0000","last_peered":"2024-09-19T07:44:02.636651+0000","last_clean":"2024-09-19T07:44:02.636651+0000","last_became_active":"2024-09-19T07:44:01.636223+0000","last_became_peered":"2024-09-19T07:44:01.636223+0000","last_unstale":"2024-09-19T07:44:02.636651+0000","last_undegraded":"2024-09-19T07:44:02.636651+0000","last_fullsized":"2024-09-19T07:44:02.636651+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:33.573198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880122+0000","last_change":"2024-09-19T07:44:00.624268+0000","last_active":"2024-09-19T07:44:02.880122+0000","last_peered":"2024-09-19T07:44:02.880122+0000","last_clean":"2024-09-19T07:44:02.880122+0000","last_became_active":"2024-09-19T07:44:00.623980+0000","last_became_peered":"2024-09-19T07:44:00.623980+0000","last_unstale":"2024-09-19T07:44:02.880122+0000","last_undegraded":"2024-09-19T07:44:02.880122+0000","last_fullsized":"2024-09-19T07:44:02.880122+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:42.166793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633166+0000","last_change":"2024-09-19T07:44:01.642857+0000","last_active":"2024-09-19T07:44:02.633166+0000","last_peered":"2024-09-19T07:44:02.633166+0000","last_clean":"2024-09-19T07:44:02.633166+0000","last_became_active":"2024-09-19T07:44:01.642470+0000","last_became_peered":"2024-09-19T07:44:01.642470+0000","last_unstale":"2024-09-19T07:44:02.633166+0000","last_undegraded":"2024-09-19T07:44:02.633166+0000","last_fullsized":"2024-09-19T07:44:02.633166+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:32.560026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636670+0000","last_change":"2024-09-19T07:44:00.625579+0000","last_active":"2024-09-19T07:44:02.636670+0000","last_peered":"2024-09-19T07:44:02.636670+0000","last_clean":"2024-09-19T07:44:02.636670+0000","last_became_active":"2024-09-19T07:44:00.625308+0000","last_became_peered":"2024-09-19T07:44:00.625308+0000","last_unstale":"2024-09-19T07:44:02.636670+0000","last_undegraded":"2024-09-19T07:44:02.636670+0000","last_fullsized":"2024-09-19T07:44:02.636670+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:18.057432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879672+0000","last_change":"2024-09-19T07:44:01.634412+0000","last_active":"2024-09-19T07:44:02.879672+0000","last_peered":"2024-09-19T07:44:02.879672+0000","last_clean":"2024-09-19T07:44:02.879672+0000","last_became_active":"2024-09-19T07:44:01.633940+0000","last_became_peered":"2024-09-19T07:44:01.633940+0000","last_unstale":"2024-09-19T07:44:02.879672+0000","last_undegraded":"2024-09-19T07:44:02.879672+0000","last_fullsized":"2024-09-19T07:44:02.879672+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:16.252351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637355+0000","last_change":"2024-09-19T07:44:00.625126+0000","last_active":"2024-09-19T07:44:02.637355+0000","last_peered":"2024-09-19T07:44:02.637355+0000","last_clean":"2024-09-19T07:44:02.637355+0000","last_became_active":"2024-09-19T07:44:00.624639+0000","last_became_peered":"2024-09-19T07:44:00.624639+0000","last_unstale":"2024-09-19T07:44:02.637355+0000","last_undegraded":"2024-09-19T07:44:02.637355+0000","last_fullsized":"2024-09-19T07:44:02.637355+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:32.568047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637110+0000","last_change":"2024-09-19T07:44:01.626697+0000","last_active":"2024-09-19T07:44:02.637110+0000","last_peered":"2024-09-19T07:44:02.637110+0000","last_clean":"2024-09-19T07:44:02.637110+0000","last_became_active":"2024-09-19T07:44:01.626410+0000","last_became_peered":"2024-09-19T07:44:01.626410+0000","last_unstale":"2024-09-19T07:44:02.637110+0000","last_undegraded":"2024-09-19T07:44:02.637110+0000","last_fullsized":"2024-09-19T07:44:02.637110+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:18.230752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879284+0000","last_change":"2024-09-19T07:44:00.635497+0000","last_active":"2024-09-19T07:44:02.879284+0000","last_peered":"2024-09-19T07:44:02.879284+0000","last_clean":"2024-09-19T07:44:02.879284+0000","last_became_active":"2024-09-19T07:44:00.635231+0000","last_became_peered":"2024-09-19T07:44:00.635231+0000","last_unstale":"2024-09-19T07:44:02.879284+0000","last_undegraded":"2024-09-19T07:44:02.879284+0000","last_fullsized":"2024-09-19T07:44:02.879284+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:42.166793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637171+0000","last_change":"2024-09-19T07:44:01.628887+0000","last_active":"2024-09-19T07:44:02.637171+0000","last_peered":"2024-09-19T07:44:02.637171+0000","last_clean":"2024-09-19T07:44:02.637171+0000","last_became_active":"2024-09-19T07:44:01.628580+0000","last_became_peered":"2024-09-19T07:44:01.628580+0000","last_unstale":"2024-09-19T07:44:02.637171+0000","last_undegraded":"2024-09-19T07:44:02.637171+0000","last_fullsized":"2024-09-19T07:44:02.637171+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:18.456608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879466+0000","last_change":"2024-09-19T07:44:00.631253+0000","last_active":"2024-09-19T07:44:02.879466+0000","last_peered":"2024-09-19T07:44:02.879466+0000","last_clean":"2024-09-19T07:44:02.879466+0000","last_became_active":"2024-09-19T07:44:00.630967+0000","last_became_peered":"2024-09-19T07:44:00.630967+0000","last_unstale":"2024-09-19T07:44:02.879466+0000","last_undegraded":"2024-09-19T07:44:02.879466+0000","last_fullsized":"2024-09-19T07:44:02.879466+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:07.746484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632826+0000","last_change":"2024-09-19T07:44:01.637345+0000","last_active":"2024-09-19T07:44:02.632826+0000","last_peered":"2024-09-19T07:44:02.632826+0000","last_clean":"2024-09-19T07:44:02.632826+0000","last_became_active":"2024-09-19T07:44:01.633385+0000","last_became_peered":"2024-09-19T07:44:01.633385+0000","last_unstale":"2024-09-19T07:44:02.632826+0000","last_undegraded":"2024-09-19T07:44:02.632826+0000","last_fullsized":"2024-09-19T07:44:02.632826+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:26.999322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636289+0000","last_change":"2024-09-19T07:44:00.626219+0000","last_active":"2024-09-19T07:44:02.636289+0000","last_peered":"2024-09-19T07:44:02.636289+0000","last_clean":"2024-09-19T07:44:02.636289+0000","last_became_active":"2024-09-19T07:44:00.625897+0000","last_became_peered":"2024-09-19T07:44:00.625897+0000","last_unstale":"2024-09-19T07:44:02.636289+0000","last_undegraded":"2024-09-19T07:44:02.636289+0000","last_fullsized":"2024-09-19T07:44:02.636289+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:37.739420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.613312+0000","last_change":"2024-09-19T07:44:00.613312+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.613312+0000","last_undegraded":"2024-09-19T07:44:00.613312+0000","last_fullsized":"2024-09-19T07:44:00.613312+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633235+0000","last_change":"2024-09-19T07:44:00.621900+0000","last_active":"2024-09-19T07:44:02.633235+0000","last_peered":"2024-09-19T07:44:02.633235+0000","last_clean":"2024-09-19T07:44:02.633235+0000","last_became_active":"2024-09-19T07:44:00.621596+0000","last_became_peered":"2024-09-19T07:44:00.621596+0000","last_unstale":"2024-09-19T07:44:02.633235+0000","last_undegraded":"2024-09-19T07:44:02.633235+0000","last_fullsized":"2024-09-19T07:44:02.633235+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:19.578605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879600+0000","last_change":"2024-09-19T07:44:01.637539+0000","last_active":"2024-09-19T07:44:02.879600+0000","last_peered":"2024-09-19T07:44:02.879600+0000","last_clean":"2024-09-19T07:44:02.879600+0000","last_became_active":"2024-09-19T07:44:01.637285+0000","last_became_peered":"2024-09-19T07:44:01.637285+0000","last_unstale":"2024-09-19T07:44:02.879600+0000","last_undegraded":"2024-09-19T07:44:02.879600+0000","last_fullsized":"2024-09-19T07:44:02.879600+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:35.866583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637751+0000","last_change":"2024-09-19T07:44:00.627400+0000","last_active":"2024-09-19T07:44:02.637751+0000","last_peered":"2024-09-19T07:44:02.637751+0000","last_clean":"2024-09-19T07:44:02.637751+0000","last_became_active":"2024-09-19T07:44:00.627087+0000","last_became_peered":"2024-09-19T07:44:00.627087+0000","last_unstale":"2024-09-19T07:44:02.637751+0000","last_undegraded":"2024-09-19T07:44:02.637751+0000","last_fullsized":"2024-09-19T07:44:02.637751+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:17.451456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879620+0000","last_change":"2024-09-19T07:44:01.631643+0000","last_active":"2024-09-19T07:44:02.879620+0000","last_peered":"2024-09-19T07:44:02.879620+0000","last_clean":"2024-09-19T07:44:02.879620+0000","last_became_active":"2024-09-19T07:44:01.631352+0000","last_became_peered":"2024-09-19T07:44:01.631352+0000","last_unstale":"2024-09-19T07:44:02.879620+0000","last_undegraded":"2024-09-19T07:44:02.879620+0000","last_fullsized":"2024-09-19T07:44:02.879620+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:22:00.022599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637737+0000","last_change":"2024-09-19T07:44:00.632425+0000","last_active":"2024-09-19T07:44:02.637737+0000","last_peered":"2024-09-19T07:44:02.637737+0000","last_clean":"2024-09-19T07:44:02.637737+0000","last_became_active":"2024-09-19T07:44:00.631713+0000","last_became_peered":"2024-09-19T07:44:00.631713+0000","last_unstale":"2024-09-19T07:44:02.637737+0000","last_undegraded":"2024-09-19T07:44:02.637737+0000","last_fullsized":"2024-09-19T07:44:02.637737+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:59.782194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636568+0000","last_change":"2024-09-19T07:44:01.638031+0000","last_active":"2024-09-19T07:44:02.636568+0000","last_peered":"2024-09-19T07:44:02.636568+0000","last_clean":"2024-09-19T07:44:02.636568+0000","last_became_active":"2024-09-19T07:44:01.637692+0000","last_became_peered":"2024-09-19T07:44:01.637692+0000","last_unstale":"2024-09-19T07:44:02.636568+0000","last_undegraded":"2024-09-19T07:44:02.636568+0000","last_fullsized":"2024-09-19T07:44:02.636568+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:29:52.280755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632895+0000","last_change":"2024-09-19T07:44:00.629007+0000","last_active":"2024-09-19T07:44:02.632895+0000","last_peered":"2024-09-19T07:44:02.632895+0000","last_clean":"2024-09-19T07:44:02.632895+0000","last_became_active":"2024-09-19T07:44:00.628690+0000","last_became_peered":"2024-09-19T07:44:00.628690+0000","last_unstale":"2024-09-19T07:44:02.632895+0000","last_undegraded":"2024-09-19T07:44:02.632895+0000","last_fullsized":"2024-09-19T07:44:02.632895+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:15.247200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879821+0000","last_change":"2024-09-19T07:44:01.634198+0000","last_active":"2024-09-19T07:44:02.879821+0000","last_peered":"2024-09-19T07:44:02.879821+0000","last_clean":"2024-09-19T07:44:02.879821+0000","last_became_active":"2024-09-19T07:44:01.633557+0000","last_became_peered":"2024-09-19T07:44:01.633557+0000","last_unstale":"2024-09-19T07:44:02.879821+0000","last_undegraded":"2024-09-19T07:44:02.879821+0000","last_fullsized":"2024-09-19T07:44:02.879821+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:10.758189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632967+0000","last_change":"2024-09-19T07:44:00.621833+0000","last_active":"2024-09-19T07:44:02.632967+0000","last_peered":"2024-09-19T07:44:02.632967+0000","last_clean":"2024-09-19T07:44:02.632967+0000","last_became_active":"2024-09-19T07:44:00.621525+0000","last_became_peered":"2024-09-19T07:44:00.621525+0000","last_unstale":"2024-09-19T07:44:02.632967+0000","last_undegraded":"2024-09-19T07:44:02.632967+0000","last_fullsized":"2024-09-19T07:44:02.632967+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:09.753037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632713+0000","last_change":"2024-09-19T07:44:01.635000+0000","last_active":"2024-09-19T07:44:02.632713+0000","last_peered":"2024-09-19T07:44:02.632713+0000","last_clean":"2024-09-19T07:44:02.632713+0000","last_became_active":"2024-09-19T07:44:01.634733+0000","last_became_peered":"2024-09-19T07:44:01.634733+0000","last_unstale":"2024-09-19T07:44:02.632713+0000","last_undegraded":"2024-09-19T07:44:02.632713+0000","last_fullsized":"2024-09-19T07:44:02.632713+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:26.448577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880363+0000","last_change":"2024-09-19T07:44:00.630777+0000","last_active":"2024-09-19T07:44:02.880363+0000","last_peered":"2024-09-19T07:44:02.880363+0000","last_clean":"2024-09-19T07:44:02.880363+0000","last_became_active":"2024-09-19T07:44:00.630494+0000","last_became_peered":"2024-09-19T07:44:00.630494+0000","last_unstale":"2024-09-19T07:44:02.880363+0000","last_undegraded":"2024-09-19T07:44:02.880363+0000","last_fullsized":"2024-09-19T07:44:02.880363+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:06:32.865629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879425+0000","last_change":"2024-09-19T07:44:01.632590+0000","last_active":"2024-09-19T07:44:02.879425+0000","last_peered":"2024-09-19T07:44:02.879425+0000","last_clean":"2024-09-19T07:44:02.879425+0000","last_became_active":"2024-09-19T07:44:01.632284+0000","last_became_peered":"2024-09-19T07:44:01.632284+0000","last_unstale":"2024-09-19T07:44:02.879425+0000","last_undegraded":"2024-09-19T07:44:02.879425+0000","last_fullsized":"2024-09-19T07:44:02.879425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:49.781908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.857683+0000","last_change":"2024-09-19T07:44:00.630565+0000","last_active":"2024-09-19T07:44:02.857683+0000","last_peered":"2024-09-19T07:44:02.857683+0000","last_clean":"2024-09-19T07:44:02.857683+0000","last_became_active":"2024-09-19T07:44:00.630302+0000","last_became_peered":"2024-09-19T07:44:00.630302+0000","last_unstale":"2024-09-19T07:44:02.857683+0000","last_undegraded":"2024-09-19T07:44:02.857683+0000","last_fullsized":"2024-09-19T07:44:02.857683+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:13:09.025629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632815+0000","last_change":"2024-09-19T07:44:01.637576+0000","last_active":"2024-09-19T07:44:02.632815+0000","last_peered":"2024-09-19T07:44:02.632815+0000","last_clean":"2024-09-19T07:44:02.632815+0000","last_became_active":"2024-09-19T07:44:01.637026+0000","last_became_peered":"2024-09-19T07:44:01.637026+0000","last_unstale":"2024-09-19T07:44:02.632815+0000","last_undegraded":"2024-09-19T07:44:02.632815+0000","last_fullsized":"2024-09-19T07:44:02.632815+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:34.910477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879205+0000","last_change":"2024-09-19T07:44:00.635283+0000","last_active":"2024-09-19T07:44:02.879205+0000","last_peered":"2024-09-19T07:44:02.879205+0000","last_clean":"2024-09-19T07:44:02.879205+0000","last_became_active":"2024-09-19T07:44:00.635035+0000","last_became_peered":"2024-09-19T07:44:00.635035+0000","last_unstale":"2024-09-19T07:44:02.879205+0000","last_undegraded":"2024-09-19T07:44:02.879205+0000","last_fullsized":"2024-09-19T07:44:02.879205+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:06:32.865629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633057+0000","last_change":"2024-09-19T07:44:01.643372+0000","last_active":"2024-09-19T07:44:02.633057+0000","last_peered":"2024-09-19T07:44:02.633057+0000","last_clean":"2024-09-19T07:44:02.633057+0000","last_became_active":"2024-09-19T07:44:01.642744+0000","last_became_peered":"2024-09-19T07:44:01.642744+0000","last_unstale":"2024-09-19T07:44:02.633057+0000","last_undegraded":"2024-09-19T07:44:02.633057+0000","last_fullsized":"2024-09-19T07:44:02.633057+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:34.596548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637842+0000","last_change":"2024-09-19T07:44:00.627327+0000","last_active":"2024-09-19T07:44:02.637842+0000","last_peered":"2024-09-19T07:44:02.637842+0000","last_clean":"2024-09-19T07:44:02.637842+0000","last_became_active":"2024-09-19T07:44:00.627059+0000","last_became_peered":"2024-09-19T07:44:00.627059+0000","last_unstale":"2024-09-19T07:44:02.637842+0000","last_undegraded":"2024-09-19T07:44:02.637842+0000","last_fullsized":"2024-09-19T07:44:02.637842+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:09.150256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637600+0000","last_change":"2024-09-19T07:44:01.625997+0000","last_active":"2024-09-19T07:44:02.637600+0000","last_peered":"2024-09-19T07:44:02.637600+0000","last_clean":"2024-09-19T07:44:02.637600+0000","last_became_active":"2024-09-19T07:44:01.625686+0000","last_became_peered":"2024-09-19T07:44:01.625686+0000","last_unstale":"2024-09-19T07:44:02.637600+0000","last_undegraded":"2024-09-19T07:44:02.637600+0000","last_fullsized":"2024-09-19T07:44:02.637600+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:26.999322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636755+0000","last_change":"2024-09-19T07:44:00.628926+0000","last_active":"2024-09-19T07:44:02.636755+0000","last_peered":"2024-09-19T07:44:02.636755+0000","last_clean":"2024-09-19T07:44:02.636755+0000","last_became_active":"2024-09-19T07:44:00.628629+0000","last_became_peered":"2024-09-19T07:44:00.628629+0000","last_unstale":"2024-09-19T07:44:02.636755+0000","last_undegraded":"2024-09-19T07:44:02.636755+0000","last_fullsized":"2024-09-19T07:44:02.636755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:50.180508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636754+0000","last_change":"2024-09-19T07:44:01.626175+0000","last_active":"2024-09-19T07:44:02.636754+0000","last_peered":"2024-09-19T07:44:02.636754+0000","last_clean":"2024-09-19T07:44:02.636754+0000","last_became_active":"2024-09-19T07:44:01.625903+0000","last_became_peered":"2024-09-19T07:44:01.625903+0000","last_unstale":"2024-09-19T07:44:02.636754+0000","last_undegraded":"2024-09-19T07:44:02.636754+0000","last_fullsized":"2024-09-19T07:44:02.636754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:10.155408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.632871+0000","last_change":"2024-09-19T07:44:00.631635+0000","last_active":"2024-09-19T07:44:00.632871+0000","last_peered":"2024-09-19T07:44:00.632871+0000","last_clean":"2024-09-19T07:44:00.632871+0000","last_became_active":"2024-09-19T07:44:00.631343+0000","last_became_peered":"2024-09-19T07:44:00.631343+0000","last_unstale":"2024-09-19T07:44:00.632871+0000","last_undegraded":"2024-09-19T07:44:00.632871+0000","last_fullsized":"2024-09-19T07:44:00.632871+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:33.000415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-19T07:44:00.613328+0000","last_change":"2024-09-19T07:44:00.613328+0000","last_active":"2024-09-19T07:44:00.603665+0000","last_peered":"2024-09-19T07:44:00.603665+0000","last_clean":"2024-09-19T07:44:00.603665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:44:00.613328+0000","last_undegraded":"2024-09-19T07:44:00.613328+0000","last_fullsized":"2024-09-19T07:44:00.613328+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879303+0000","last_change":"2024-09-19T07:44:00.625009+0000","last_active":"2024-09-19T07:44:02.879303+0000","last_peered":"2024-09-19T07:44:02.879303+0000","last_clean":"2024-09-19T07:44:02.879303+0000","last_became_active":"2024-09-19T07:44:00.624610+0000","last_became_peered":"2024-09-19T07:44:00.624610+0000","last_unstale":"2024-09-19T07:44:02.879303+0000","last_undegraded":"2024-09-19T07:44:02.879303+0000","last_fullsized":"2024-09-19T07:44:02.879303+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:58.958404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879327+0000","last_change":"2024-09-19T07:44:01.634357+0000","last_active":"2024-09-19T07:44:02.879327+0000","last_peered":"2024-09-19T07:44:02.879327+0000","last_clean":"2024-09-19T07:44:02.879327+0000","last_became_active":"2024-09-19T07:44:01.634056+0000","last_became_peered":"2024-09-19T07:44:01.634056+0000","last_unstale":"2024-09-19T07:44:02.879327+0000","last_undegraded":"2024-09-19T07:44:02.879327+0000","last_fullsized":"2024-09-19T07:44:02.879327+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:37.429765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.625246+0000","last_change":"2024-09-19T07:44:00.623041+0000","last_active":"2024-09-19T07:44:00.625246+0000","last_peered":"2024-09-19T07:44:00.625246+0000","last_clean":"2024-09-19T07:44:00.625246+0000","last_became_active":"2024-09-19T07:44:00.622748+0000","last_became_peered":"2024-09-19T07:44:00.622748+0000","last_unstale":"2024-09-19T07:44:00.625246+0000","last_undegraded":"2024-09-19T07:44:00.625246+0000","last_fullsized":"2024-09-19T07:44:00.625246+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:41.495537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879836+0000","last_change":"2024-09-19T07:44:01.630563+0000","last_active":"2024-09-19T07:44:02.879836+0000","last_peered":"2024-09-19T07:44:02.879836+0000","last_clean":"2024-09-19T07:44:02.879836+0000","last_became_active":"2024-09-19T07:44:01.629355+0000","last_became_peered":"2024-09-19T07:44:01.629355+0000","last_unstale":"2024-09-19T07:44:02.879836+0000","last_undegraded":"2024-09-19T07:44:02.879836+0000","last_fullsized":"2024-09-19T07:44:02.879836+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:20.583756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.628081+0000","last_change":"2024-09-19T07:44:00.627049+0000","last_active":"2024-09-19T07:44:00.628081+0000","last_peered":"2024-09-19T07:44:00.628081+0000","last_clean":"2024-09-19T07:44:00.628081+0000","last_became_active":"2024-09-19T07:44:00.626768+0000","last_became_peered":"2024-09-19T07:44:00.626768+0000","last_unstale":"2024-09-19T07:44:00.628081+0000","last_undegraded":"2024-09-19T07:44:00.628081+0000","last_fullsized":"2024-09-19T07:44:00.628081+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:18.929914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636889+0000","last_change":"2024-09-19T07:44:01.641474+0000","last_active":"2024-09-19T07:44:02.636889+0000","last_peered":"2024-09-19T07:44:02.636889+0000","last_clean":"2024-09-19T07:44:02.636889+0000","last_became_active":"2024-09-19T07:44:01.641163+0000","last_became_peered":"2024-09-19T07:44:01.641163+0000","last_unstale":"2024-09-19T07:44:02.636889+0000","last_undegraded":"2024-09-19T07:44:02.636889+0000","last_fullsized":"2024-09-19T07:44:02.636889+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:35.866583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879473+0000","last_change":"2024-09-19T07:44:00.630434+0000","last_active":"2024-09-19T07:44:02.879473+0000","last_peered":"2024-09-19T07:44:02.879473+0000","last_clean":"2024-09-19T07:44:02.879473+0000","last_became_active":"2024-09-19T07:44:00.630177+0000","last_became_peered":"2024-09-19T07:44:00.630177+0000","last_unstale":"2024-09-19T07:44:02.879473+0000","last_undegraded":"2024-09-19T07:44:02.879473+0000","last_fullsized":"2024-09-19T07:44:02.879473+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:57.888193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632969+0000","last_change":"2024-09-19T07:44:01.629794+0000","last_active":"2024-09-19T07:44:02.632969+0000","last_peered":"2024-09-19T07:44:02.632969+0000","last_clean":"2024-09-19T07:44:02.632969+0000","last_became_active":"2024-09-19T07:44:01.629453+0000","last_became_peered":"2024-09-19T07:44:01.629453+0000","last_unstale":"2024-09-19T07:44:02.632969+0000","last_undegraded":"2024-09-19T07:44:02.632969+0000","last_fullsized":"2024-09-19T07:44:02.632969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:34.270347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.624497+0000","last_change":"2024-09-19T07:44:00.623038+0000","last_active":"2024-09-19T07:44:00.624497+0000","last_peered":"2024-09-19T07:44:00.624497+0000","last_clean":"2024-09-19T07:44:00.624497+0000","last_became_active":"2024-09-19T07:44:00.622704+0000","last_became_peered":"2024-09-19T07:44:00.622704+0000","last_unstale":"2024-09-19T07:44:00.624497+0000","last_undegraded":"2024-09-19T07:44:00.624497+0000","last_fullsized":"2024-09-19T07:44:00.624497+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:38.159118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879198+0000","last_change":"2024-09-19T07:44:01.633567+0000","last_active":"2024-09-19T07:44:02.879198+0000","last_peered":"2024-09-19T07:44:02.879198+0000","last_clean":"2024-09-19T07:44:02.879198+0000","last_became_active":"2024-09-19T07:44:01.633291+0000","last_became_peered":"2024-09-19T07:44:01.633291+0000","last_unstale":"2024-09-19T07:44:02.879198+0000","last_undegraded":"2024-09-19T07:44:02.879198+0000","last_fullsized":"2024-09-19T07:44:02.879198+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:18.456608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637949+0000","last_change":"2024-09-19T07:44:00.631958+0000","last_active":"2024-09-19T07:44:02.637949+0000","last_peered":"2024-09-19T07:44:02.637949+0000","last_clean":"2024-09-19T07:44:02.637949+0000","last_became_active":"2024-09-19T07:44:00.631630+0000","last_became_peered":"2024-09-19T07:44:00.631630+0000","last_unstale":"2024-09-19T07:44:02.637949+0000","last_undegraded":"2024-09-19T07:44:02.637949+0000","last_fullsized":"2024-09-19T07:44:02.637949+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:41.113078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879655+0000","last_change":"2024-09-19T07:44:01.638149+0000","last_active":"2024-09-19T07:44:02.879655+0000","last_peered":"2024-09-19T07:44:02.879655+0000","last_clean":"2024-09-19T07:44:02.879655+0000","last_became_active":"2024-09-19T07:44:01.637903+0000","last_became_peered":"2024-09-19T07:44:01.637903+0000","last_unstale":"2024-09-19T07:44:02.879655+0000","last_undegraded":"2024-09-19T07:44:02.879655+0000","last_fullsized":"2024-09-19T07:44:02.879655+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:18.230752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-19T07:44:00.629136+0000","last_change":"2024-09-19T07:44:00.627575+0000","last_active":"2024-09-19T07:44:00.629136+0000","last_peered":"2024-09-19T07:44:00.629136+0000","last_clean":"2024-09-19T07:44:00.629136+0000","last_became_active":"2024-09-19T07:44:00.627217+0000","last_became_peered":"2024-09-19T07:44:00.627217+0000","last_unstale":"2024-09-19T07:44:00.629136+0000","last_undegraded":"2024-09-19T07:44:00.629136+0000","last_fullsized":"2024-09-19T07:44:00.629136+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:30.570507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633094+0000","last_change":"2024-09-19T07:44:01.620156+0000","last_active":"2024-09-19T07:44:02.633094+0000","last_peered":"2024-09-19T07:44:02.633094+0000","last_clean":"2024-09-19T07:44:02.633094+0000","last_became_active":"2024-09-19T07:44:01.619829+0000","last_became_peered":"2024-09-19T07:44:01.619829+0000","last_unstale":"2024-09-19T07:44:02.633094+0000","last_undegraded":"2024-09-19T07:44:02.633094+0000","last_fullsized":"2024-09-19T07:44:02.633094+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:17.164807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879680+0000","last_change":"2024-09-19T07:44:00.632242+0000","last_active":"2024-09-19T07:44:02.879680+0000","last_peered":"2024-09-19T07:44:02.879680+0000","last_clean":"2024-09-19T07:44:02.879680+0000","last_became_active":"2024-09-19T07:44:00.631953+0000","last_became_peered":"2024-09-19T07:44:00.631953+0000","last_unstale":"2024-09-19T07:44:02.879680+0000","last_undegraded":"2024-09-19T07:44:02.879680+0000","last_fullsized":"2024-09-19T07:44:02.879680+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:28.881326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879972+0000","last_change":"2024-09-19T07:44:01.643566+0000","last_active":"2024-09-19T07:44:02.879972+0000","last_peered":"2024-09-19T07:44:02.879972+0000","last_clean":"2024-09-19T07:44:02.879972+0000","last_became_active":"2024-09-19T07:44:01.643264+0000","last_became_peered":"2024-09-19T07:44:01.643264+0000","last_unstale":"2024-09-19T07:44:02.879972+0000","last_undegraded":"2024-09-19T07:44:02.879972+0000","last_fullsized":"2024-09-19T07:44:02.879972+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:15:37.153759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880264+0000","last_change":"2024-09-19T07:44:00.629323+0000","last_active":"2024-09-19T07:44:02.880264+0000","last_peered":"2024-09-19T07:44:02.880264+0000","last_clean":"2024-09-19T07:44:02.880264+0000","last_became_active":"2024-09-19T07:44:00.629037+0000","last_became_peered":"2024-09-19T07:44:00.629037+0000","last_unstale":"2024-09-19T07:44:02.880264+0000","last_undegraded":"2024-09-19T07:44:02.880264+0000","last_fullsized":"2024-09-19T07:44:02.880264+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:07.746484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879371+0000","last_change":"2024-09-19T07:44:01.637663+0000","last_active":"2024-09-19T07:44:02.879371+0000","last_peered":"2024-09-19T07:44:02.879371+0000","last_clean":"2024-09-19T07:44:02.879371+0000","last_became_active":"2024-09-19T07:44:01.637391+0000","last_became_peered":"2024-09-19T07:44:01.637391+0000","last_unstale":"2024-09-19T07:44:02.879371+0000","last_undegraded":"2024-09-19T07:44:02.879371+0000","last_fullsized":"2024-09-19T07:44:02.879371+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:11.328826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.857755+0000","last_change":"2024-09-19T07:44:00.628801+0000","last_active":"2024-09-19T07:44:02.857755+0000","last_peered":"2024-09-19T07:44:02.857755+0000","last_clean":"2024-09-19T07:44:02.857755+0000","last_became_active":"2024-09-19T07:44:00.626609+0000","last_became_peered":"2024-09-19T07:44:00.626609+0000","last_unstale":"2024-09-19T07:44:02.857755+0000","last_undegraded":"2024-09-19T07:44:02.857755+0000","last_fullsized":"2024-09-19T07:44:02.857755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:31.700567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637203+0000","last_change":"2024-09-19T07:44:01.618078+0000","last_active":"2024-09-19T07:44:02.637203+0000","last_peered":"2024-09-19T07:44:02.637203+0000","last_clean":"2024-09-19T07:44:02.637203+0000","last_became_active":"2024-09-19T07:44:01.617847+0000","last_became_peered":"2024-09-19T07:44:01.617847+0000","last_unstale":"2024-09-19T07:44:02.637203+0000","last_undegraded":"2024-09-19T07:44:02.637203+0000","last_fullsized":"2024-09-19T07:44:02.637203+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:11.328826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636690+0000","last_change":"2024-09-19T07:44:00.630615+0000","last_active":"2024-09-19T07:44:02.636690+0000","last_peered":"2024-09-19T07:44:02.636690+0000","last_clean":"2024-09-19T07:44:02.636690+0000","last_became_active":"2024-09-19T07:44:00.630335+0000","last_became_peered":"2024-09-19T07:44:00.630335+0000","last_unstale":"2024-09-19T07:44:02.636690+0000","last_undegraded":"2024-09-19T07:44:02.636690+0000","last_fullsized":"2024-09-19T07:44:02.636690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:26:30.972385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879426+0000","last_change":"2024-09-19T07:44:01.643552+0000","last_active":"2024-09-19T07:44:02.879426+0000","last_peered":"2024-09-19T07:44:02.879426+0000","last_clean":"2024-09-19T07:44:02.879426+0000","last_became_active":"2024-09-19T07:44:01.642710+0000","last_became_peered":"2024-09-19T07:44:01.642710+0000","last_unstale":"2024-09-19T07:44:02.879426+0000","last_undegraded":"2024-09-19T07:44:02.879426+0000","last_fullsized":"2024-09-19T07:44:02.879426+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:17.407644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632629+0000","last_change":"2024-09-19T07:44:00.622888+0000","last_active":"2024-09-19T07:44:02.632629+0000","last_peered":"2024-09-19T07:44:02.632629+0000","last_clean":"2024-09-19T07:44:02.632629+0000","last_became_active":"2024-09-19T07:44:00.622610+0000","last_became_peered":"2024-09-19T07:44:00.622610+0000","last_unstale":"2024-09-19T07:44:02.632629+0000","last_undegraded":"2024-09-19T07:44:02.632629+0000","last_fullsized":"2024-09-19T07:44:02.632629+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:59.017448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637807+0000","last_change":"2024-09-19T07:44:01.636163+0000","last_active":"2024-09-19T07:44:02.637807+0000","last_peered":"2024-09-19T07:44:02.637807+0000","last_clean":"2024-09-19T07:44:02.637807+0000","last_became_active":"2024-09-19T07:44:01.635746+0000","last_became_peered":"2024-09-19T07:44:01.635746+0000","last_unstale":"2024-09-19T07:44:02.637807+0000","last_undegraded":"2024-09-19T07:44:02.637807+0000","last_fullsized":"2024-09-19T07:44:02.637807+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:00.266417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879621+0000","last_change":"2024-09-19T07:44:00.628940+0000","last_active":"2024-09-19T07:44:02.879621+0000","last_peered":"2024-09-19T07:44:02.879621+0000","last_clean":"2024-09-19T07:44:02.879621+0000","last_became_active":"2024-09-19T07:44:00.628668+0000","last_became_peered":"2024-09-19T07:44:00.628668+0000","last_unstale":"2024-09-19T07:44:02.879621+0000","last_undegraded":"2024-09-19T07:44:02.879621+0000","last_fullsized":"2024-09-19T07:44:02.879621+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:07.092062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":203,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879397+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:44:02.879397+0000","last_peered":"2024-09-19T07:44:02.879397+0000","last_clean":"2024-09-19T07:44:02.879397+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:44:02.879397+0000","last_undegraded":"2024-09-19T07:44:02.879397+0000","last_fullsized":"2024-09-19T07:44:02.879397+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1064960,"data_stored":788276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":196808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":462848,"data_stored":196338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":462848,"data_stored":196830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1052672,"data_stored":786628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31652,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":471040,"data_stored":197872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":446464,"data_stored":189624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":430080,"data_stored":182367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:44:06.601 INFO:tasks.ceph.mds.a.smithi016.stderr:2024-09-19T07:44:06.601+0000 7f1aa6ebd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T07:44:07.016 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:44:07.019 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:44:09.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:44:09.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.617+0000 7f34354e6640 1 Processor -- start 2024-09-19T07:44:09.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.617+0000 7f34354e6640 1 -- start start 2024-09-19T07:44:09.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.618+0000 7f34354e6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f343015c270 0x7f343006c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:09.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.618+0000 7f34354e6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34300cf680 0x7f34300cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:09.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.618+0000 7f34354e6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34300cfda0 con 0x7f34300ae270 2024-09-19T07:44:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.618+0000 7f34354e6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f34300ccf80 con 0x7f343015c270 2024-09-19T07:44:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.618+0000 7f34354e6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f34300cd260 con 0x7f34300cf680 2024-09-19T07:44:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.618+0000 7f342f7fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34300cf680 0x7f34300cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.618+0000 7f342effd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f343015c270 0x7f343006c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.618+0000 7f342e7fc640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f34300ae270 0x7f34300cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:55206/0 (socket says 172.21.15.16:55206) 2024-09-19T07:44:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.618+0000 7f342e7fc640 1 -- 172.21.15.16:0/1155268978 learned_addr learned my addr 172.21.15.16:0/1155268978 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:44:09.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.618+0000 7f342effd640 1 -- 172.21.15.16:0/1155268978 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34300cf680 msgr2=0x7f34300cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:09.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.618+0000 7f342effd640 1 --2- 172.21.15.16:0/1155268978 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34300cf680 0x7f34300cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:09.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.618+0000 7f342effd640 1 -- 172.21.15.16:0/1155268978 >> v1:172.21.15.146:6789/0 conn(0x7f34300ae270 legacy=0x7f34300cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:09.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.618+0000 7f342effd640 1 -- 172.21.15.16:0/1155268978 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34300cd540 con 0x7f343015c270 2024-09-19T07:44:09.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.619+0000 7f342effd640 1 --2- 172.21.15.16:0/1155268978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f343015c270 0x7f343006c620 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f34200097a0 tx=0x7f342002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32b2f7c53c8666e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:09.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.620+0000 7f342dffb640 1 -- 172.21.15.16:0/1155268978 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420037070 con 0x7f343015c270 2024-09-19T07:44:09.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.620+0000 7f342dffb640 1 -- 172.21.15.16:0/1155268978 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f342002ae50 con 0x7f343015c270 2024-09-19T07:44:09.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.620+0000 7f342dffb640 1 -- 172.21.15.16:0/1155268978 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420032660 con 0x7f343015c270 2024-09-19T07:44:09.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.620+0000 7f34354e6640 1 -- 172.21.15.16:0/1155268978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f343015c270 msgr2=0x7f343006c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:09.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.620+0000 7f34354e6640 1 --2- 172.21.15.16:0/1155268978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f343015c270 0x7f343006c620 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f34200097a0 tx=0x7f342002a100 comp rx=0 tx=0).stop 2024-09-19T07:44:09.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.620+0000 7f34354e6640 1 -- 172.21.15.16:0/1155268978 shutdown_connections 2024-09-19T07:44:09.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.620+0000 7f34354e6640 1 --2- 172.21.15.16:0/1155268978 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34300cf680 0x7f34300cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:09.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.620+0000 7f34354e6640 1 --2- 172.21.15.16:0/1155268978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f343015c270 0x7f343006c620 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:09.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.620+0000 7f34354e6640 1 -- 172.21.15.16:0/1155268978 >> 172.21.15.16:0/1155268978 conn(0x7f343009a9f0 msgr2=0x7f34300b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:09.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.620+0000 7f34354e6640 1 -- 172.21.15.16:0/1155268978 shutdown_connections 2024-09-19T07:44:09.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.620+0000 7f34354e6640 1 -- 172.21.15.16:0/1155268978 wait complete. 2024-09-19T07:44:09.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.622+0000 7f34354e6640 1 Processor -- start 2024-09-19T07:44:09.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.622+0000 7f34354e6640 1 -- start start 2024-09-19T07:44:09.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.622+0000 7f34354e6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34300ae270 0x7f34300c7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:09.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.622+0000 7f34354e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34300cf680 0x7f34301f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:09.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.622+0000 7f34354e6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f343015c270 0x7f343009c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:09.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.622+0000 7f34354e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f343015c7d0 con 0x7f34300cf680 2024-09-19T07:44:09.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.622+0000 7f34354e6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f343015c650 con 0x7f343015c270 2024-09-19T07:44:09.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.622+0000 7f34354e6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f34300ccba0 con 0x7f34300ae270 2024-09-19T07:44:09.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.623+0000 7f342e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34300cf680 0x7f34301f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:09.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.623+0000 7f342e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34300cf680 0x7f34301f3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51770/0 (socket says 172.21.15.16:51770) 2024-09-19T07:44:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.623+0000 7f342e7fc640 1 -- 172.21.15.16:0/1739429165 learned_addr learned my addr 172.21.15.16:0/1739429165 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.623+0000 7f342f7fe640 1 --2- 172.21.15.16:0/1739429165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f343015c270 0x7f343009c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.623+0000 7f342effd640 1 --2- 172.21.15.16:0/1739429165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34300ae270 0x7f34300c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.623+0000 7f342e7fc640 1 -- 172.21.15.16:0/1739429165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34300ae270 msgr2=0x7f34300c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.623+0000 7f342e7fc640 1 --2- 172.21.15.16:0/1739429165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34300ae270 0x7f34300c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.623+0000 7f342e7fc640 1 -- 172.21.15.16:0/1739429165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f343015c270 msgr2=0x7f343009c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.623+0000 7f342e7fc640 1 --2- 172.21.15.16:0/1739429165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f343015c270 0x7f343009c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.623+0000 7f342e7fc640 1 -- 172.21.15.16:0/1739429165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343009d060 con 0x7f34300cf680 2024-09-19T07:44:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.623+0000 7f342e7fc640 1 --2- 172.21.15.16:0/1739429165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34300cf680 0x7f34301f3e80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f342400c430 tx=0x7f342400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.623+0000 7f34177fe640 1 -- 172.21.15.16:0/1739429165 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424017070 con 0x7f34300cf680 2024-09-19T07:44:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.623+0000 7f34354e6640 1 -- 172.21.15.16:0/1739429165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f343006f010 con 0x7f34300cf680 2024-09-19T07:44:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.623+0000 7f34177fe640 1 -- 172.21.15.16:0/1739429165 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3424005ce0 con 0x7f34300cf680 2024-09-19T07:44:09.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.623+0000 7f34177fe640 1 -- 172.21.15.16:0/1739429165 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34240072f0 con 0x7f34300cf680 2024-09-19T07:44:09.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.623+0000 7f34354e6640 1 -- 172.21.15.16:0/1739429165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f343006f5d0 con 0x7f34300cf680 2024-09-19T07:44:09.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.625+0000 7f34354e6640 1 -- 172.21.15.16:0/1739429165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f343006c2d0 con 0x7f34300cf680 2024-09-19T07:44:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.625+0000 7f34177fe640 1 -- 172.21.15.16:0/1739429165 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f342401a070 con 0x7f34300cf680 2024-09-19T07:44:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.626+0000 7f34177fe640 1 --2- 172.21.15.16:0/1739429165 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f341c06c2b0 0x7f341c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.626+0000 7f34177fe640 1 -- 172.21.15.16:0/1739429165 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f342408a960 con 0x7f34300cf680 2024-09-19T07:44:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.626+0000 7f342effd640 1 --2- 172.21.15.16:0/1739429165 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f341c06c2b0 0x7f341c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.626+0000 7f342effd640 1 --2- 172.21.15.16:0/1739429165 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f341c06c2b0 0x7f341c06e770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3420034000 tx=0x7f342002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:09.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.630+0000 7f34177fe640 1 -- 172.21.15.16:0/1739429165 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f34300cfda0 con 0x7f34300cf680 2024-09-19T07:44:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.806+0000 7f34354e6640 1 -- 172.21.15.16:0/1739429165 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f34300cdd50 con 0x7f341c06c2b0 2024-09-19T07:44:09.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.813+0000 7f34177fe640 1 -- 172.21.15.16:0/1739429165 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251957 (secure 0 0 0) 0x7f34300cdd50 con 0x7f341c06c2b0 2024-09-19T07:44:09.816 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:44:09.817 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:44:09.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.816+0000 7f34354e6640 1 -- 172.21.15.16:0/1739429165 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f341c06c2b0 msgr2=0x7f341c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:09.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.816+0000 7f34354e6640 1 --2- 172.21.15.16:0/1739429165 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f341c06c2b0 0x7f341c06e770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3420034000 tx=0x7f342002e000 comp rx=0 tx=0).stop 2024-09-19T07:44:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.816+0000 7f34354e6640 1 -- 172.21.15.16:0/1739429165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34300cf680 msgr2=0x7f34301f3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.816+0000 7f34354e6640 1 --2- 172.21.15.16:0/1739429165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34300cf680 0x7f34301f3e80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f342400c430 tx=0x7f342400c8f0 comp rx=0 tx=0).stop 2024-09-19T07:44:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.816+0000 7f34354e6640 1 -- 172.21.15.16:0/1739429165 shutdown_connections 2024-09-19T07:44:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.816+0000 7f34354e6640 1 --2- 172.21.15.16:0/1739429165 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f341c06c2b0 0x7f341c06e770 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.816+0000 7f34354e6640 1 --2- 172.21.15.16:0/1739429165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f343015c270 0x7f343009c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.816+0000 7f34354e6640 1 --2- 172.21.15.16:0/1739429165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34300cf680 0x7f34301f3e80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.816+0000 7f34354e6640 1 --2- 172.21.15.16:0/1739429165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34300ae270 0x7f34300c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.816+0000 7f34354e6640 1 -- 172.21.15.16:0/1739429165 >> 172.21.15.16:0/1739429165 conn(0x7f343009a9f0 msgr2=0x7f34300ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.816+0000 7f34354e6640 1 -- 172.21.15.16:0/1739429165 shutdown_connections 2024-09-19T07:44:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.816+0000 7f34354e6640 1 -- 172.21.15.16:0/1739429165 wait complete. 2024-09-19T07:44:09.827 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-09-19T07:44:08.473170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237652,"kb_used_data":4764,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737068,"statfs":{"total":773094113280,"available":772850757632,"internally_reserved":0,"allocated":4878336,"data_stored":2748672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.859564"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633184+0000","last_change":"2024-09-19T07:44:01.635139+0000","last_active":"2024-09-19T07:44:02.633184+0000","last_peered":"2024-09-19T07:44:02.633184+0000","last_clean":"2024-09-19T07:44:02.633184+0000","last_became_active":"2024-09-19T07:44:01.634842+0000","last_became_peered":"2024-09-19T07:44:01.634842+0000","last_unstale":"2024-09-19T07:44:02.633184+0000","last_undegraded":"2024-09-19T07:44:02.633184+0000","last_fullsized":"2024-09-19T07:44:02.633184+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:07.759986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633030+0000","last_change":"2024-09-19T07:44:01.629931+0000","last_active":"2024-09-19T07:44:02.633030+0000","last_peered":"2024-09-19T07:44:02.633030+0000","last_clean":"2024-09-19T07:44:02.633030+0000","last_became_active":"2024-09-19T07:44:01.629634+0000","last_became_peered":"2024-09-19T07:44:01.629634+0000","last_unstale":"2024-09-19T07:44:02.633030+0000","last_undegraded":"2024-09-19T07:44:02.633030+0000","last_fullsized":"2024-09-19T07:44:02.633030+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:00.266417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879583+0000","last_change":"2024-09-19T07:44:01.635942+0000","last_active":"2024-09-19T07:44:02.879583+0000","last_peered":"2024-09-19T07:44:02.879583+0000","last_clean":"2024-09-19T07:44:02.879583+0000","last_became_active":"2024-09-19T07:44:01.635639+0000","last_became_peered":"2024-09-19T07:44:01.635639+0000","last_unstale":"2024-09-19T07:44:02.879583+0000","last_undegraded":"2024-09-19T07:44:02.879583+0000","last_fullsized":"2024-09-19T07:44:02.879583+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:18.456608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879350+0000","last_change":"2024-09-19T07:44:01.634356+0000","last_active":"2024-09-19T07:44:02.879350+0000","last_peered":"2024-09-19T07:44:02.879350+0000","last_clean":"2024-09-19T07:44:02.879350+0000","last_became_active":"2024-09-19T07:44:01.634076+0000","last_became_peered":"2024-09-19T07:44:01.634076+0000","last_unstale":"2024-09-19T07:44:02.879350+0000","last_undegraded":"2024-09-19T07:44:02.879350+0000","last_fullsized":"2024-09-19T07:44:02.879350+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:04:59.630093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879091+0000","last_change":"2024-09-19T07:44:01.630336+0000","last_active":"2024-09-19T07:44:02.879091+0000","last_peered":"2024-09-19T07:44:02.879091+0000","last_clean":"2024-09-19T07:44:02.879091+0000","last_became_active":"2024-09-19T07:44:01.629794+0000","last_became_peered":"2024-09-19T07:44:01.629794+0000","last_unstale":"2024-09-19T07:44:02.879091+0000","last_undegraded":"2024-09-19T07:44:02.879091+0000","last_fullsized":"2024-09-19T07:44:02.879091+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:20.561408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880437+0000","last_change":"2024-09-19T07:44:01.635795+0000","last_active":"2024-09-19T07:44:02.880437+0000","last_peered":"2024-09-19T07:44:02.880437+0000","last_clean":"2024-09-19T07:44:02.880437+0000","last_became_active":"2024-09-19T07:44:01.634728+0000","last_became_peered":"2024-09-19T07:44:01.634728+0000","last_unstale":"2024-09-19T07:44:02.880437+0000","last_undegraded":"2024-09-19T07:44:02.880437+0000","last_fullsized":"2024-09-19T07:44:02.880437+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:20.583756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636513+0000","last_change":"2024-09-19T07:44:01.641177+0000","last_active":"2024-09-19T07:44:02.636513+0000","last_peered":"2024-09-19T07:44:02.636513+0000","last_clean":"2024-09-19T07:44:02.636513+0000","last_became_active":"2024-09-19T07:44:01.637104+0000","last_became_peered":"2024-09-19T07:44:01.637104+0000","last_unstale":"2024-09-19T07:44:02.636513+0000","last_undegraded":"2024-09-19T07:44:02.636513+0000","last_fullsized":"2024-09-19T07:44:02.636513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:47:46.856860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632612+0000","last_change":"2024-09-19T07:44:01.621511+0000","last_active":"2024-09-19T07:44:02.632612+0000","last_peered":"2024-09-19T07:44:02.632612+0000","last_clean":"2024-09-19T07:44:02.632612+0000","last_became_active":"2024-09-19T07:44:01.620511+0000","last_became_peered":"2024-09-19T07:44:01.620511+0000","last_unstale":"2024-09-19T07:44:02.632612+0000","last_undegraded":"2024-09-19T07:44:02.632612+0000","last_fullsized":"2024-09-19T07:44:02.632612+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:57.716510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879610+0000","last_change":"2024-09-19T07:44:01.635157+0000","last_active":"2024-09-19T07:44:02.879610+0000","last_peered":"2024-09-19T07:44:02.879610+0000","last_clean":"2024-09-19T07:44:02.879610+0000","last_became_active":"2024-09-19T07:44:01.634821+0000","last_became_peered":"2024-09-19T07:44:01.634821+0000","last_unstale":"2024-09-19T07:44:02.879610+0000","last_undegraded":"2024-09-19T07:44:02.879610+0000","last_fullsized":"2024-09-19T07:44:02.879610+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:11.328826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637663+0000","last_change":"2024-09-19T07:44:01.631689+0000","last_active":"2024-09-19T07:44:02.637663+0000","last_peered":"2024-09-19T07:44:02.637663+0000","last_clean":"2024-09-19T07:44:02.637663+0000","last_became_active":"2024-09-19T07:44:01.631389+0000","last_became_peered":"2024-09-19T07:44:01.631389+0000","last_unstale":"2024-09-19T07:44:02.637663+0000","last_undegraded":"2024-09-19T07:44:02.637663+0000","last_fullsized":"2024-09-19T07:44:02.637663+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:34.270347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637601+0000","last_change":"2024-09-19T07:44:01.620732+0000","last_active":"2024-09-19T07:44:02.637601+0000","last_peered":"2024-09-19T07:44:02.637601+0000","last_clean":"2024-09-19T07:44:02.637601+0000","last_became_active":"2024-09-19T07:44:01.620458+0000","last_became_peered":"2024-09-19T07:44:01.620458+0000","last_unstale":"2024-09-19T07:44:02.637601+0000","last_undegraded":"2024-09-19T07:44:02.637601+0000","last_fullsized":"2024-09-19T07:44:02.637601+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:34.596548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879452+0000","last_change":"2024-09-19T07:44:01.635852+0000","last_active":"2024-09-19T07:44:02.879452+0000","last_peered":"2024-09-19T07:44:02.879452+0000","last_clean":"2024-09-19T07:44:02.879452+0000","last_became_active":"2024-09-19T07:44:01.635563+0000","last_became_peered":"2024-09-19T07:44:01.635563+0000","last_unstale":"2024-09-19T07:44:02.879452+0000","last_undegraded":"2024-09-19T07:44:02.879452+0000","last_fullsized":"2024-09-19T07:44:02.879452+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:42.118230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637139+0000","last_change":"2024-09-19T07:44:01.630966+0000","last_active":"2024-09-19T07:44:02.637139+0000","last_peered":"2024-09-19T07:44:02.637139+0000","last_clean":"2024-09-19T07:44:02.637139+0000","last_became_active":"2024-09-19T07:44:01.630693+0000","last_became_peered":"2024-09-19T07:44:01.630693+0000","last_unstale":"2024-09-19T07:44:02.637139+0000","last_undegraded":"2024-09-19T07:44:02.637139+0000","last_fullsized":"2024-09-19T07:44:02.637139+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:57.716510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636697+0000","last_change":"2024-09-19T07:44:01.636530+0000","last_active":"2024-09-19T07:44:02.636697+0000","last_peered":"2024-09-19T07:44:02.636697+0000","last_clean":"2024-09-19T07:44:02.636697+0000","last_became_active":"2024-09-19T07:44:01.636084+0000","last_became_peered":"2024-09-19T07:44:01.636084+0000","last_unstale":"2024-09-19T07:44:02.636697+0000","last_undegraded":"2024-09-19T07:44:02.636697+0000","last_fullsized":"2024-09-19T07:44:02.636697+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:00.787345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636671+0000","last_change":"2024-09-19T07:44:01.641990+0000","last_active":"2024-09-19T07:44:02.636671+0000","last_peered":"2024-09-19T07:44:02.636671+0000","last_clean":"2024-09-19T07:44:02.636671+0000","last_became_active":"2024-09-19T07:44:01.641212+0000","last_became_peered":"2024-09-19T07:44:01.641212+0000","last_unstale":"2024-09-19T07:44:02.636671+0000","last_undegraded":"2024-09-19T07:44:02.636671+0000","last_fullsized":"2024-09-19T07:44:02.636671+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:19.575604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879128+0000","last_change":"2024-09-19T07:44:01.630670+0000","last_active":"2024-09-19T07:44:02.879128+0000","last_peered":"2024-09-19T07:44:02.879128+0000","last_clean":"2024-09-19T07:44:02.879128+0000","last_became_active":"2024-09-19T07:44:01.630419+0000","last_became_peered":"2024-09-19T07:44:01.630419+0000","last_unstale":"2024-09-19T07:44:02.879128+0000","last_undegraded":"2024-09-19T07:44:02.879128+0000","last_fullsized":"2024-09-19T07:44:02.879128+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:15:37.153759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879517+0000","last_change":"2024-09-19T07:44:01.631309+0000","last_active":"2024-09-19T07:44:02.879517+0000","last_peered":"2024-09-19T07:44:02.879517+0000","last_clean":"2024-09-19T07:44:02.879517+0000","last_became_active":"2024-09-19T07:44:01.631048+0000","last_became_peered":"2024-09-19T07:44:01.631048+0000","last_unstale":"2024-09-19T07:44:02.879517+0000","last_undegraded":"2024-09-19T07:44:02.879517+0000","last_fullsized":"2024-09-19T07:44:02.879517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:19.575604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.858000+0000","last_change":"2024-09-19T07:44:01.635852+0000","last_active":"2024-09-19T07:44:02.858000+0000","last_peered":"2024-09-19T07:44:02.858000+0000","last_clean":"2024-09-19T07:44:02.858000+0000","last_became_active":"2024-09-19T07:44:01.635487+0000","last_became_peered":"2024-09-19T07:44:01.635487+0000","last_unstale":"2024-09-19T07:44:02.858000+0000","last_undegraded":"2024-09-19T07:44:02.858000+0000","last_fullsized":"2024-09-19T07:44:02.858000+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:08.097213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879884+0000","last_change":"2024-09-19T07:44:01.632032+0000","last_active":"2024-09-19T07:44:02.879884+0000","last_peered":"2024-09-19T07:44:02.879884+0000","last_clean":"2024-09-19T07:44:02.879884+0000","last_became_active":"2024-09-19T07:44:01.631376+0000","last_became_peered":"2024-09-19T07:44:01.631376+0000","last_unstale":"2024-09-19T07:44:02.879884+0000","last_undegraded":"2024-09-19T07:44:02.879884+0000","last_fullsized":"2024-09-19T07:44:02.879884+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:34:56.993843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.857873+0000","last_change":"2024-09-19T07:44:01.635914+0000","last_active":"2024-09-19T07:44:02.857873+0000","last_peered":"2024-09-19T07:44:02.857873+0000","last_clean":"2024-09-19T07:44:02.857873+0000","last_became_active":"2024-09-19T07:44:01.635626+0000","last_became_peered":"2024-09-19T07:44:01.635626+0000","last_unstale":"2024-09-19T07:44:02.857873+0000","last_undegraded":"2024-09-19T07:44:02.857873+0000","last_fullsized":"2024-09-19T07:44:02.857873+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:58.893344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637194+0000","last_change":"2024-09-19T07:44:01.634487+0000","last_active":"2024-09-19T07:44:02.637194+0000","last_peered":"2024-09-19T07:44:02.637194+0000","last_clean":"2024-09-19T07:44:02.637194+0000","last_became_active":"2024-09-19T07:44:01.634225+0000","last_became_peered":"2024-09-19T07:44:01.634225+0000","last_unstale":"2024-09-19T07:44:02.637194+0000","last_undegraded":"2024-09-19T07:44:02.637194+0000","last_fullsized":"2024-09-19T07:44:02.637194+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:26.448577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637004+0000","last_change":"2024-09-19T07:44:01.631807+0000","last_active":"2024-09-19T07:44:02.637004+0000","last_peered":"2024-09-19T07:44:02.637004+0000","last_clean":"2024-09-19T07:44:02.637004+0000","last_became_active":"2024-09-19T07:44:01.631515+0000","last_became_peered":"2024-09-19T07:44:01.631515+0000","last_unstale":"2024-09-19T07:44:02.637004+0000","last_undegraded":"2024-09-19T07:44:02.637004+0000","last_fullsized":"2024-09-19T07:44:02.637004+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:42.118230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879177+0000","last_change":"2024-09-19T07:44:01.630667+0000","last_active":"2024-09-19T07:44:02.879177+0000","last_peered":"2024-09-19T07:44:02.879177+0000","last_clean":"2024-09-19T07:44:02.879177+0000","last_became_active":"2024-09-19T07:44:01.630388+0000","last_became_peered":"2024-09-19T07:44:01.630388+0000","last_unstale":"2024-09-19T07:44:02.879177+0000","last_undegraded":"2024-09-19T07:44:02.879177+0000","last_fullsized":"2024-09-19T07:44:02.879177+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:57:21.017364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879019+0000","last_change":"2024-09-19T07:44:01.630272+0000","last_active":"2024-09-19T07:44:02.879019+0000","last_peered":"2024-09-19T07:44:02.879019+0000","last_clean":"2024-09-19T07:44:02.879019+0000","last_became_active":"2024-09-19T07:44:01.629944+0000","last_became_peered":"2024-09-19T07:44:01.629944+0000","last_unstale":"2024-09-19T07:44:02.879019+0000","last_undegraded":"2024-09-19T07:44:02.879019+0000","last_fullsized":"2024-09-19T07:44:02.879019+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:25.848047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.858062+0000","last_change":"2024-09-19T07:44:01.638722+0000","last_active":"2024-09-19T07:44:02.858062+0000","last_peered":"2024-09-19T07:44:02.858062+0000","last_clean":"2024-09-19T07:44:02.858062+0000","last_became_active":"2024-09-19T07:44:01.638450+0000","last_became_peered":"2024-09-19T07:44:01.638450+0000","last_unstale":"2024-09-19T07:44:02.858062+0000","last_undegraded":"2024-09-19T07:44:02.858062+0000","last_fullsized":"2024-09-19T07:44:02.858062+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:33.573198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879547+0000","last_change":"2024-09-19T07:44:01.632403+0000","last_active":"2024-09-19T07:44:02.879547+0000","last_peered":"2024-09-19T07:44:02.879547+0000","last_clean":"2024-09-19T07:44:02.879547+0000","last_became_active":"2024-09-19T07:44:01.632105+0000","last_became_peered":"2024-09-19T07:44:01.632105+0000","last_unstale":"2024-09-19T07:44:02.879547+0000","last_undegraded":"2024-09-19T07:44:02.879547+0000","last_fullsized":"2024-09-19T07:44:02.879547+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:22:00.022599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.858031+0000","last_change":"2024-09-19T07:44:01.640185+0000","last_active":"2024-09-19T07:44:02.858031+0000","last_peered":"2024-09-19T07:44:02.858031+0000","last_clean":"2024-09-19T07:44:02.858031+0000","last_became_active":"2024-09-19T07:44:01.639498+0000","last_became_peered":"2024-09-19T07:44:01.639498+0000","last_unstale":"2024-09-19T07:44:02.858031+0000","last_undegraded":"2024-09-19T07:44:02.858031+0000","last_fullsized":"2024-09-19T07:44:02.858031+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:29:52.280755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637457+0000","last_change":"2024-09-19T07:44:01.623150+0000","last_active":"2024-09-19T07:44:02.637457+0000","last_peered":"2024-09-19T07:44:02.637457+0000","last_clean":"2024-09-19T07:44:02.637457+0000","last_became_active":"2024-09-19T07:44:01.622344+0000","last_became_peered":"2024-09-19T07:44:01.622344+0000","last_unstale":"2024-09-19T07:44:02.637457+0000","last_undegraded":"2024-09-19T07:44:02.637457+0000","last_fullsized":"2024-09-19T07:44:02.637457+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:34.910477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.857867+0000","last_change":"2024-09-19T07:44:01.639571+0000","last_active":"2024-09-19T07:44:02.857867+0000","last_peered":"2024-09-19T07:44:02.857867+0000","last_clean":"2024-09-19T07:44:02.857867+0000","last_became_active":"2024-09-19T07:44:01.639259+0000","last_became_peered":"2024-09-19T07:44:01.639259+0000","last_unstale":"2024-09-19T07:44:02.857867+0000","last_undegraded":"2024-09-19T07:44:02.857867+0000","last_fullsized":"2024-09-19T07:44:02.857867+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:43.171945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.857808+0000","last_change":"2024-09-19T07:44:01.640084+0000","last_active":"2024-09-19T07:44:02.857808+0000","last_peered":"2024-09-19T07:44:02.857808+0000","last_clean":"2024-09-19T07:44:02.857808+0000","last_became_active":"2024-09-19T07:44:01.639804+0000","last_became_peered":"2024-09-19T07:44:01.639804+0000","last_unstale":"2024-09-19T07:44:02.857808+0000","last_undegraded":"2024-09-19T07:44:02.857808+0000","last_fullsized":"2024-09-19T07:44:02.857808+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:06:33.870781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879323+0000","last_change":"2024-09-19T07:44:01.633165+0000","last_active":"2024-09-19T07:44:02.879323+0000","last_peered":"2024-09-19T07:44:02.879323+0000","last_clean":"2024-09-19T07:44:02.879323+0000","last_became_active":"2024-09-19T07:44:01.632905+0000","last_became_peered":"2024-09-19T07:44:01.632905+0000","last_unstale":"2024-09-19T07:44:02.879323+0000","last_undegraded":"2024-09-19T07:44:02.879323+0000","last_fullsized":"2024-09-19T07:44:02.879323+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:45.305611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880311+0000","last_change":"2024-09-19T07:44:01.633044+0000","last_active":"2024-09-19T07:44:02.880311+0000","last_peered":"2024-09-19T07:44:02.880311+0000","last_clean":"2024-09-19T07:44:02.880311+0000","last_became_active":"2024-09-19T07:44:01.632761+0000","last_became_peered":"2024-09-19T07:44:01.632761+0000","last_unstale":"2024-09-19T07:44:02.880311+0000","last_undegraded":"2024-09-19T07:44:02.880311+0000","last_fullsized":"2024-09-19T07:44:02.880311+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:10.758189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879515+0000","last_change":"2024-09-19T07:44:01.633367+0000","last_active":"2024-09-19T07:44:02.879515+0000","last_peered":"2024-09-19T07:44:02.879515+0000","last_clean":"2024-09-19T07:44:02.879515+0000","last_became_active":"2024-09-19T07:44:01.632924+0000","last_became_peered":"2024-09-19T07:44:01.632924+0000","last_unstale":"2024-09-19T07:44:02.879515+0000","last_undegraded":"2024-09-19T07:44:02.879515+0000","last_fullsized":"2024-09-19T07:44:02.879515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:47:46.856860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.858098+0000","last_change":"2024-09-19T07:44:00.631265+0000","last_active":"2024-09-19T07:44:02.858098+0000","last_peered":"2024-09-19T07:44:02.858098+0000","last_clean":"2024-09-19T07:44:02.858098+0000","last_became_active":"2024-09-19T07:44:00.631003+0000","last_became_peered":"2024-09-19T07:44:00.631003+0000","last_unstale":"2024-09-19T07:44:02.858098+0000","last_undegraded":"2024-09-19T07:44:02.858098+0000","last_fullsized":"2024-09-19T07:44:02.858098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:29.392209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879930+0000","last_change":"2024-09-19T07:44:01.630912+0000","last_active":"2024-09-19T07:44:02.879930+0000","last_peered":"2024-09-19T07:44:02.879930+0000","last_clean":"2024-09-19T07:44:02.879930+0000","last_became_active":"2024-09-19T07:44:01.630625+0000","last_became_peered":"2024-09-19T07:44:01.630625+0000","last_unstale":"2024-09-19T07:44:02.879930+0000","last_undegraded":"2024-09-19T07:44:02.879930+0000","last_fullsized":"2024-09-19T07:44:02.879930+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:16.252351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879734+0000","last_change":"2024-09-19T07:44:00.628877+0000","last_active":"2024-09-19T07:44:02.879734+0000","last_peered":"2024-09-19T07:44:02.879734+0000","last_clean":"2024-09-19T07:44:02.879734+0000","last_became_active":"2024-09-19T07:44:00.628595+0000","last_became_peered":"2024-09-19T07:44:00.628595+0000","last_unstale":"2024-09-19T07:44:02.879734+0000","last_undegraded":"2024-09-19T07:44:02.879734+0000","last_fullsized":"2024-09-19T07:44:02.879734+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:58.958404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636651+0000","last_change":"2024-09-19T07:44:01.636642+0000","last_active":"2024-09-19T07:44:02.636651+0000","last_peered":"2024-09-19T07:44:02.636651+0000","last_clean":"2024-09-19T07:44:02.636651+0000","last_became_active":"2024-09-19T07:44:01.636223+0000","last_became_peered":"2024-09-19T07:44:01.636223+0000","last_unstale":"2024-09-19T07:44:02.636651+0000","last_undegraded":"2024-09-19T07:44:02.636651+0000","last_fullsized":"2024-09-19T07:44:02.636651+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:33.573198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880122+0000","last_change":"2024-09-19T07:44:00.624268+0000","last_active":"2024-09-19T07:44:02.880122+0000","last_peered":"2024-09-19T07:44:02.880122+0000","last_clean":"2024-09-19T07:44:02.880122+0000","last_became_active":"2024-09-19T07:44:00.623980+0000","last_became_peered":"2024-09-19T07:44:00.623980+0000","last_unstale":"2024-09-19T07:44:02.880122+0000","last_undegraded":"2024-09-19T07:44:02.880122+0000","last_fullsized":"2024-09-19T07:44:02.880122+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:42.166793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633166+0000","last_change":"2024-09-19T07:44:01.642857+0000","last_active":"2024-09-19T07:44:02.633166+0000","last_peered":"2024-09-19T07:44:02.633166+0000","last_clean":"2024-09-19T07:44:02.633166+0000","last_became_active":"2024-09-19T07:44:01.642470+0000","last_became_peered":"2024-09-19T07:44:01.642470+0000","last_unstale":"2024-09-19T07:44:02.633166+0000","last_undegraded":"2024-09-19T07:44:02.633166+0000","last_fullsized":"2024-09-19T07:44:02.633166+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:32.560026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636670+0000","last_change":"2024-09-19T07:44:00.625579+0000","last_active":"2024-09-19T07:44:02.636670+0000","last_peered":"2024-09-19T07:44:02.636670+0000","last_clean":"2024-09-19T07:44:02.636670+0000","last_became_active":"2024-09-19T07:44:00.625308+0000","last_became_peered":"2024-09-19T07:44:00.625308+0000","last_unstale":"2024-09-19T07:44:02.636670+0000","last_undegraded":"2024-09-19T07:44:02.636670+0000","last_fullsized":"2024-09-19T07:44:02.636670+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:18.057432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879672+0000","last_change":"2024-09-19T07:44:01.634412+0000","last_active":"2024-09-19T07:44:02.879672+0000","last_peered":"2024-09-19T07:44:02.879672+0000","last_clean":"2024-09-19T07:44:02.879672+0000","last_became_active":"2024-09-19T07:44:01.633940+0000","last_became_peered":"2024-09-19T07:44:01.633940+0000","last_unstale":"2024-09-19T07:44:02.879672+0000","last_undegraded":"2024-09-19T07:44:02.879672+0000","last_fullsized":"2024-09-19T07:44:02.879672+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:16.252351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637355+0000","last_change":"2024-09-19T07:44:00.625126+0000","last_active":"2024-09-19T07:44:02.637355+0000","last_peered":"2024-09-19T07:44:02.637355+0000","last_clean":"2024-09-19T07:44:02.637355+0000","last_became_active":"2024-09-19T07:44:00.624639+0000","last_became_peered":"2024-09-19T07:44:00.624639+0000","last_unstale":"2024-09-19T07:44:02.637355+0000","last_undegraded":"2024-09-19T07:44:02.637355+0000","last_fullsized":"2024-09-19T07:44:02.637355+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:32.568047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637110+0000","last_change":"2024-09-19T07:44:01.626697+0000","last_active":"2024-09-19T07:44:02.637110+0000","last_peered":"2024-09-19T07:44:02.637110+0000","last_clean":"2024-09-19T07:44:02.637110+0000","last_became_active":"2024-09-19T07:44:01.626410+0000","last_became_peered":"2024-09-19T07:44:01.626410+0000","last_unstale":"2024-09-19T07:44:02.637110+0000","last_undegraded":"2024-09-19T07:44:02.637110+0000","last_fullsized":"2024-09-19T07:44:02.637110+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:18.230752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879284+0000","last_change":"2024-09-19T07:44:00.635497+0000","last_active":"2024-09-19T07:44:02.879284+0000","last_peered":"2024-09-19T07:44:02.879284+0000","last_clean":"2024-09-19T07:44:02.879284+0000","last_became_active":"2024-09-19T07:44:00.635231+0000","last_became_peered":"2024-09-19T07:44:00.635231+0000","last_unstale":"2024-09-19T07:44:02.879284+0000","last_undegraded":"2024-09-19T07:44:02.879284+0000","last_fullsized":"2024-09-19T07:44:02.879284+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:42.166793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637171+0000","last_change":"2024-09-19T07:44:01.628887+0000","last_active":"2024-09-19T07:44:02.637171+0000","last_peered":"2024-09-19T07:44:02.637171+0000","last_clean":"2024-09-19T07:44:02.637171+0000","last_became_active":"2024-09-19T07:44:01.628580+0000","last_became_peered":"2024-09-19T07:44:01.628580+0000","last_unstale":"2024-09-19T07:44:02.637171+0000","last_undegraded":"2024-09-19T07:44:02.637171+0000","last_fullsized":"2024-09-19T07:44:02.637171+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:18.456608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879466+0000","last_change":"2024-09-19T07:44:00.631253+0000","last_active":"2024-09-19T07:44:02.879466+0000","last_peered":"2024-09-19T07:44:02.879466+0000","last_clean":"2024-09-19T07:44:02.879466+0000","last_became_active":"2024-09-19T07:44:00.630967+0000","last_became_peered":"2024-09-19T07:44:00.630967+0000","last_unstale":"2024-09-19T07:44:02.879466+0000","last_undegraded":"2024-09-19T07:44:02.879466+0000","last_fullsized":"2024-09-19T07:44:02.879466+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:07.746484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632826+0000","last_change":"2024-09-19T07:44:01.637345+0000","last_active":"2024-09-19T07:44:02.632826+0000","last_peered":"2024-09-19T07:44:02.632826+0000","last_clean":"2024-09-19T07:44:02.632826+0000","last_became_active":"2024-09-19T07:44:01.633385+0000","last_became_peered":"2024-09-19T07:44:01.633385+0000","last_unstale":"2024-09-19T07:44:02.632826+0000","last_undegraded":"2024-09-19T07:44:02.632826+0000","last_fullsized":"2024-09-19T07:44:02.632826+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:26.999322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636289+0000","last_change":"2024-09-19T07:44:00.626219+0000","last_active":"2024-09-19T07:44:02.636289+0000","last_peered":"2024-09-19T07:44:02.636289+0000","last_clean":"2024-09-19T07:44:02.636289+0000","last_became_active":"2024-09-19T07:44:00.625897+0000","last_became_peered":"2024-09-19T07:44:00.625897+0000","last_unstale":"2024-09-19T07:44:02.636289+0000","last_undegraded":"2024-09-19T07:44:02.636289+0000","last_fullsized":"2024-09-19T07:44:02.636289+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:37.739420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879482+0000","last_change":"2024-09-19T07:44:01.630749+0000","last_active":"2024-09-19T07:44:02.879482+0000","last_peered":"2024-09-19T07:44:02.879482+0000","last_clean":"2024-09-19T07:44:02.879482+0000","last_became_active":"2024-09-19T07:44:01.630033+0000","last_became_peered":"2024-09-19T07:44:01.630033+0000","last_unstale":"2024-09-19T07:44:02.879482+0000","last_undegraded":"2024-09-19T07:44:02.879482+0000","last_fullsized":"2024-09-19T07:44:02.879482+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:30.230864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633235+0000","last_change":"2024-09-19T07:44:00.621900+0000","last_active":"2024-09-19T07:44:02.633235+0000","last_peered":"2024-09-19T07:44:02.633235+0000","last_clean":"2024-09-19T07:44:02.633235+0000","last_became_active":"2024-09-19T07:44:00.621596+0000","last_became_peered":"2024-09-19T07:44:00.621596+0000","last_unstale":"2024-09-19T07:44:02.633235+0000","last_undegraded":"2024-09-19T07:44:02.633235+0000","last_fullsized":"2024-09-19T07:44:02.633235+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:19.578605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879600+0000","last_change":"2024-09-19T07:44:01.637539+0000","last_active":"2024-09-19T07:44:02.879600+0000","last_peered":"2024-09-19T07:44:02.879600+0000","last_clean":"2024-09-19T07:44:02.879600+0000","last_became_active":"2024-09-19T07:44:01.637285+0000","last_became_peered":"2024-09-19T07:44:01.637285+0000","last_unstale":"2024-09-19T07:44:02.879600+0000","last_undegraded":"2024-09-19T07:44:02.879600+0000","last_fullsized":"2024-09-19T07:44:02.879600+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:35.866583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637751+0000","last_change":"2024-09-19T07:44:00.627400+0000","last_active":"2024-09-19T07:44:02.637751+0000","last_peered":"2024-09-19T07:44:02.637751+0000","last_clean":"2024-09-19T07:44:02.637751+0000","last_became_active":"2024-09-19T07:44:00.627087+0000","last_became_peered":"2024-09-19T07:44:00.627087+0000","last_unstale":"2024-09-19T07:44:02.637751+0000","last_undegraded":"2024-09-19T07:44:02.637751+0000","last_fullsized":"2024-09-19T07:44:02.637751+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:17.451456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879620+0000","last_change":"2024-09-19T07:44:01.631643+0000","last_active":"2024-09-19T07:44:02.879620+0000","last_peered":"2024-09-19T07:44:02.879620+0000","last_clean":"2024-09-19T07:44:02.879620+0000","last_became_active":"2024-09-19T07:44:01.631352+0000","last_became_peered":"2024-09-19T07:44:01.631352+0000","last_unstale":"2024-09-19T07:44:02.879620+0000","last_undegraded":"2024-09-19T07:44:02.879620+0000","last_fullsized":"2024-09-19T07:44:02.879620+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:22:00.022599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637737+0000","last_change":"2024-09-19T07:44:00.632425+0000","last_active":"2024-09-19T07:44:02.637737+0000","last_peered":"2024-09-19T07:44:02.637737+0000","last_clean":"2024-09-19T07:44:02.637737+0000","last_became_active":"2024-09-19T07:44:00.631713+0000","last_became_peered":"2024-09-19T07:44:00.631713+0000","last_unstale":"2024-09-19T07:44:02.637737+0000","last_undegraded":"2024-09-19T07:44:02.637737+0000","last_fullsized":"2024-09-19T07:44:02.637737+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:59.782194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636568+0000","last_change":"2024-09-19T07:44:01.638031+0000","last_active":"2024-09-19T07:44:02.636568+0000","last_peered":"2024-09-19T07:44:02.636568+0000","last_clean":"2024-09-19T07:44:02.636568+0000","last_became_active":"2024-09-19T07:44:01.637692+0000","last_became_peered":"2024-09-19T07:44:01.637692+0000","last_unstale":"2024-09-19T07:44:02.636568+0000","last_undegraded":"2024-09-19T07:44:02.636568+0000","last_fullsized":"2024-09-19T07:44:02.636568+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:29:52.280755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632895+0000","last_change":"2024-09-19T07:44:00.629007+0000","last_active":"2024-09-19T07:44:02.632895+0000","last_peered":"2024-09-19T07:44:02.632895+0000","last_clean":"2024-09-19T07:44:02.632895+0000","last_became_active":"2024-09-19T07:44:00.628690+0000","last_became_peered":"2024-09-19T07:44:00.628690+0000","last_unstale":"2024-09-19T07:44:02.632895+0000","last_undegraded":"2024-09-19T07:44:02.632895+0000","last_fullsized":"2024-09-19T07:44:02.632895+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:15.247200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879821+0000","last_change":"2024-09-19T07:44:01.634198+0000","last_active":"2024-09-19T07:44:02.879821+0000","last_peered":"2024-09-19T07:44:02.879821+0000","last_clean":"2024-09-19T07:44:02.879821+0000","last_became_active":"2024-09-19T07:44:01.633557+0000","last_became_peered":"2024-09-19T07:44:01.633557+0000","last_unstale":"2024-09-19T07:44:02.879821+0000","last_undegraded":"2024-09-19T07:44:02.879821+0000","last_fullsized":"2024-09-19T07:44:02.879821+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:10.758189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632967+0000","last_change":"2024-09-19T07:44:00.621833+0000","last_active":"2024-09-19T07:44:02.632967+0000","last_peered":"2024-09-19T07:44:02.632967+0000","last_clean":"2024-09-19T07:44:02.632967+0000","last_became_active":"2024-09-19T07:44:00.621525+0000","last_became_peered":"2024-09-19T07:44:00.621525+0000","last_unstale":"2024-09-19T07:44:02.632967+0000","last_undegraded":"2024-09-19T07:44:02.632967+0000","last_fullsized":"2024-09-19T07:44:02.632967+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:09.753037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632713+0000","last_change":"2024-09-19T07:44:01.635000+0000","last_active":"2024-09-19T07:44:02.632713+0000","last_peered":"2024-09-19T07:44:02.632713+0000","last_clean":"2024-09-19T07:44:02.632713+0000","last_became_active":"2024-09-19T07:44:01.634733+0000","last_became_peered":"2024-09-19T07:44:01.634733+0000","last_unstale":"2024-09-19T07:44:02.632713+0000","last_undegraded":"2024-09-19T07:44:02.632713+0000","last_fullsized":"2024-09-19T07:44:02.632713+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:26.448577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880363+0000","last_change":"2024-09-19T07:44:00.630777+0000","last_active":"2024-09-19T07:44:02.880363+0000","last_peered":"2024-09-19T07:44:02.880363+0000","last_clean":"2024-09-19T07:44:02.880363+0000","last_became_active":"2024-09-19T07:44:00.630494+0000","last_became_peered":"2024-09-19T07:44:00.630494+0000","last_unstale":"2024-09-19T07:44:02.880363+0000","last_undegraded":"2024-09-19T07:44:02.880363+0000","last_fullsized":"2024-09-19T07:44:02.880363+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:06:32.865629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879425+0000","last_change":"2024-09-19T07:44:01.632590+0000","last_active":"2024-09-19T07:44:02.879425+0000","last_peered":"2024-09-19T07:44:02.879425+0000","last_clean":"2024-09-19T07:44:02.879425+0000","last_became_active":"2024-09-19T07:44:01.632284+0000","last_became_peered":"2024-09-19T07:44:01.632284+0000","last_unstale":"2024-09-19T07:44:02.879425+0000","last_undegraded":"2024-09-19T07:44:02.879425+0000","last_fullsized":"2024-09-19T07:44:02.879425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:49.781908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.857683+0000","last_change":"2024-09-19T07:44:00.630565+0000","last_active":"2024-09-19T07:44:02.857683+0000","last_peered":"2024-09-19T07:44:02.857683+0000","last_clean":"2024-09-19T07:44:02.857683+0000","last_became_active":"2024-09-19T07:44:00.630302+0000","last_became_peered":"2024-09-19T07:44:00.630302+0000","last_unstale":"2024-09-19T07:44:02.857683+0000","last_undegraded":"2024-09-19T07:44:02.857683+0000","last_fullsized":"2024-09-19T07:44:02.857683+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:13:09.025629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632815+0000","last_change":"2024-09-19T07:44:01.637576+0000","last_active":"2024-09-19T07:44:02.632815+0000","last_peered":"2024-09-19T07:44:02.632815+0000","last_clean":"2024-09-19T07:44:02.632815+0000","last_became_active":"2024-09-19T07:44:01.637026+0000","last_became_peered":"2024-09-19T07:44:01.637026+0000","last_unstale":"2024-09-19T07:44:02.632815+0000","last_undegraded":"2024-09-19T07:44:02.632815+0000","last_fullsized":"2024-09-19T07:44:02.632815+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:34.910477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879205+0000","last_change":"2024-09-19T07:44:00.635283+0000","last_active":"2024-09-19T07:44:02.879205+0000","last_peered":"2024-09-19T07:44:02.879205+0000","last_clean":"2024-09-19T07:44:02.879205+0000","last_became_active":"2024-09-19T07:44:00.635035+0000","last_became_peered":"2024-09-19T07:44:00.635035+0000","last_unstale":"2024-09-19T07:44:02.879205+0000","last_undegraded":"2024-09-19T07:44:02.879205+0000","last_fullsized":"2024-09-19T07:44:02.879205+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:06:32.865629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633057+0000","last_change":"2024-09-19T07:44:01.643372+0000","last_active":"2024-09-19T07:44:02.633057+0000","last_peered":"2024-09-19T07:44:02.633057+0000","last_clean":"2024-09-19T07:44:02.633057+0000","last_became_active":"2024-09-19T07:44:01.642744+0000","last_became_peered":"2024-09-19T07:44:01.642744+0000","last_unstale":"2024-09-19T07:44:02.633057+0000","last_undegraded":"2024-09-19T07:44:02.633057+0000","last_fullsized":"2024-09-19T07:44:02.633057+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:34.596548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637842+0000","last_change":"2024-09-19T07:44:00.627327+0000","last_active":"2024-09-19T07:44:02.637842+0000","last_peered":"2024-09-19T07:44:02.637842+0000","last_clean":"2024-09-19T07:44:02.637842+0000","last_became_active":"2024-09-19T07:44:00.627059+0000","last_became_peered":"2024-09-19T07:44:00.627059+0000","last_unstale":"2024-09-19T07:44:02.637842+0000","last_undegraded":"2024-09-19T07:44:02.637842+0000","last_fullsized":"2024-09-19T07:44:02.637842+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:09.150256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637600+0000","last_change":"2024-09-19T07:44:01.625997+0000","last_active":"2024-09-19T07:44:02.637600+0000","last_peered":"2024-09-19T07:44:02.637600+0000","last_clean":"2024-09-19T07:44:02.637600+0000","last_became_active":"2024-09-19T07:44:01.625686+0000","last_became_peered":"2024-09-19T07:44:01.625686+0000","last_unstale":"2024-09-19T07:44:02.637600+0000","last_undegraded":"2024-09-19T07:44:02.637600+0000","last_fullsized":"2024-09-19T07:44:02.637600+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:26.999322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636755+0000","last_change":"2024-09-19T07:44:00.628926+0000","last_active":"2024-09-19T07:44:02.636755+0000","last_peered":"2024-09-19T07:44:02.636755+0000","last_clean":"2024-09-19T07:44:02.636755+0000","last_became_active":"2024-09-19T07:44:00.628629+0000","last_became_peered":"2024-09-19T07:44:00.628629+0000","last_unstale":"2024-09-19T07:44:02.636755+0000","last_undegraded":"2024-09-19T07:44:02.636755+0000","last_fullsized":"2024-09-19T07:44:02.636755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:50.180508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636754+0000","last_change":"2024-09-19T07:44:01.626175+0000","last_active":"2024-09-19T07:44:02.636754+0000","last_peered":"2024-09-19T07:44:02.636754+0000","last_clean":"2024-09-19T07:44:02.636754+0000","last_became_active":"2024-09-19T07:44:01.625903+0000","last_became_peered":"2024-09-19T07:44:01.625903+0000","last_unstale":"2024-09-19T07:44:02.636754+0000","last_undegraded":"2024-09-19T07:44:02.636754+0000","last_fullsized":"2024-09-19T07:44:02.636754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:10.155408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879980+0000","last_change":"2024-09-19T07:44:00.631635+0000","last_active":"2024-09-19T07:44:02.879980+0000","last_peered":"2024-09-19T07:44:02.879980+0000","last_clean":"2024-09-19T07:44:02.879980+0000","last_became_active":"2024-09-19T07:44:00.631343+0000","last_became_peered":"2024-09-19T07:44:00.631343+0000","last_unstale":"2024-09-19T07:44:02.879980+0000","last_undegraded":"2024-09-19T07:44:02.879980+0000","last_fullsized":"2024-09-19T07:44:02.879980+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:39.277968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879823+0000","last_change":"2024-09-19T07:44:01.634995+0000","last_active":"2024-09-19T07:44:02.879823+0000","last_peered":"2024-09-19T07:44:02.879823+0000","last_clean":"2024-09-19T07:44:02.879823+0000","last_became_active":"2024-09-19T07:44:01.634684+0000","last_became_peered":"2024-09-19T07:44:01.634684+0000","last_unstale":"2024-09-19T07:44:02.879823+0000","last_undegraded":"2024-09-19T07:44:02.879823+0000","last_fullsized":"2024-09-19T07:44:02.879823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:59.912054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879303+0000","last_change":"2024-09-19T07:44:00.625009+0000","last_active":"2024-09-19T07:44:02.879303+0000","last_peered":"2024-09-19T07:44:02.879303+0000","last_clean":"2024-09-19T07:44:02.879303+0000","last_became_active":"2024-09-19T07:44:00.624610+0000","last_became_peered":"2024-09-19T07:44:00.624610+0000","last_unstale":"2024-09-19T07:44:02.879303+0000","last_undegraded":"2024-09-19T07:44:02.879303+0000","last_fullsized":"2024-09-19T07:44:02.879303+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:58.958404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879327+0000","last_change":"2024-09-19T07:44:01.634357+0000","last_active":"2024-09-19T07:44:02.879327+0000","last_peered":"2024-09-19T07:44:02.879327+0000","last_clean":"2024-09-19T07:44:02.879327+0000","last_became_active":"2024-09-19T07:44:01.634056+0000","last_became_peered":"2024-09-19T07:44:01.634056+0000","last_unstale":"2024-09-19T07:44:02.879327+0000","last_undegraded":"2024-09-19T07:44:02.879327+0000","last_fullsized":"2024-09-19T07:44:02.879327+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:37.429765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879137+0000","last_change":"2024-09-19T07:44:00.623041+0000","last_active":"2024-09-19T07:44:02.879137+0000","last_peered":"2024-09-19T07:44:02.879137+0000","last_clean":"2024-09-19T07:44:02.879137+0000","last_became_active":"2024-09-19T07:44:00.622748+0000","last_became_peered":"2024-09-19T07:44:00.622748+0000","last_unstale":"2024-09-19T07:44:02.879137+0000","last_undegraded":"2024-09-19T07:44:02.879137+0000","last_fullsized":"2024-09-19T07:44:02.879137+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:31.969200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879836+0000","last_change":"2024-09-19T07:44:01.630563+0000","last_active":"2024-09-19T07:44:02.879836+0000","last_peered":"2024-09-19T07:44:02.879836+0000","last_clean":"2024-09-19T07:44:02.879836+0000","last_became_active":"2024-09-19T07:44:01.629355+0000","last_became_peered":"2024-09-19T07:44:01.629355+0000","last_unstale":"2024-09-19T07:44:02.879836+0000","last_undegraded":"2024-09-19T07:44:02.879836+0000","last_fullsized":"2024-09-19T07:44:02.879836+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:20.583756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879075+0000","last_change":"2024-09-19T07:44:00.627049+0000","last_active":"2024-09-19T07:44:02.879075+0000","last_peered":"2024-09-19T07:44:02.879075+0000","last_clean":"2024-09-19T07:44:02.879075+0000","last_became_active":"2024-09-19T07:44:00.626768+0000","last_became_peered":"2024-09-19T07:44:00.626768+0000","last_unstale":"2024-09-19T07:44:02.879075+0000","last_undegraded":"2024-09-19T07:44:02.879075+0000","last_fullsized":"2024-09-19T07:44:02.879075+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:19:05.240775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636889+0000","last_change":"2024-09-19T07:44:01.641474+0000","last_active":"2024-09-19T07:44:02.636889+0000","last_peered":"2024-09-19T07:44:02.636889+0000","last_clean":"2024-09-19T07:44:02.636889+0000","last_became_active":"2024-09-19T07:44:01.641163+0000","last_became_peered":"2024-09-19T07:44:01.641163+0000","last_unstale":"2024-09-19T07:44:02.636889+0000","last_undegraded":"2024-09-19T07:44:02.636889+0000","last_fullsized":"2024-09-19T07:44:02.636889+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:35.866583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879473+0000","last_change":"2024-09-19T07:44:00.630434+0000","last_active":"2024-09-19T07:44:02.879473+0000","last_peered":"2024-09-19T07:44:02.879473+0000","last_clean":"2024-09-19T07:44:02.879473+0000","last_became_active":"2024-09-19T07:44:00.630177+0000","last_became_peered":"2024-09-19T07:44:00.630177+0000","last_unstale":"2024-09-19T07:44:02.879473+0000","last_undegraded":"2024-09-19T07:44:02.879473+0000","last_fullsized":"2024-09-19T07:44:02.879473+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:57.888193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632969+0000","last_change":"2024-09-19T07:44:01.629794+0000","last_active":"2024-09-19T07:44:02.632969+0000","last_peered":"2024-09-19T07:44:02.632969+0000","last_clean":"2024-09-19T07:44:02.632969+0000","last_became_active":"2024-09-19T07:44:01.629453+0000","last_became_peered":"2024-09-19T07:44:01.629453+0000","last_unstale":"2024-09-19T07:44:02.632969+0000","last_undegraded":"2024-09-19T07:44:02.632969+0000","last_fullsized":"2024-09-19T07:44:02.632969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:34.270347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879022+0000","last_change":"2024-09-19T07:44:00.623038+0000","last_active":"2024-09-19T07:44:02.879022+0000","last_peered":"2024-09-19T07:44:02.879022+0000","last_clean":"2024-09-19T07:44:02.879022+0000","last_became_active":"2024-09-19T07:44:00.622704+0000","last_became_peered":"2024-09-19T07:44:00.622704+0000","last_unstale":"2024-09-19T07:44:02.879022+0000","last_undegraded":"2024-09-19T07:44:02.879022+0000","last_fullsized":"2024-09-19T07:44:02.879022+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:47:36.429186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879198+0000","last_change":"2024-09-19T07:44:01.633567+0000","last_active":"2024-09-19T07:44:02.879198+0000","last_peered":"2024-09-19T07:44:02.879198+0000","last_clean":"2024-09-19T07:44:02.879198+0000","last_became_active":"2024-09-19T07:44:01.633291+0000","last_became_peered":"2024-09-19T07:44:01.633291+0000","last_unstale":"2024-09-19T07:44:02.879198+0000","last_undegraded":"2024-09-19T07:44:02.879198+0000","last_fullsized":"2024-09-19T07:44:02.879198+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:18.456608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637949+0000","last_change":"2024-09-19T07:44:00.631958+0000","last_active":"2024-09-19T07:44:02.637949+0000","last_peered":"2024-09-19T07:44:02.637949+0000","last_clean":"2024-09-19T07:44:02.637949+0000","last_became_active":"2024-09-19T07:44:00.631630+0000","last_became_peered":"2024-09-19T07:44:00.631630+0000","last_unstale":"2024-09-19T07:44:02.637949+0000","last_undegraded":"2024-09-19T07:44:02.637949+0000","last_fullsized":"2024-09-19T07:44:02.637949+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:41.113078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879655+0000","last_change":"2024-09-19T07:44:01.638149+0000","last_active":"2024-09-19T07:44:02.879655+0000","last_peered":"2024-09-19T07:44:02.879655+0000","last_clean":"2024-09-19T07:44:02.879655+0000","last_became_active":"2024-09-19T07:44:01.637903+0000","last_became_peered":"2024-09-19T07:44:01.637903+0000","last_unstale":"2024-09-19T07:44:02.879655+0000","last_undegraded":"2024-09-19T07:44:02.879655+0000","last_fullsized":"2024-09-19T07:44:02.879655+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:18.230752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879643+0000","last_change":"2024-09-19T07:44:00.627575+0000","last_active":"2024-09-19T07:44:02.879643+0000","last_peered":"2024-09-19T07:44:02.879643+0000","last_clean":"2024-09-19T07:44:02.879643+0000","last_became_active":"2024-09-19T07:44:00.627217+0000","last_became_peered":"2024-09-19T07:44:00.627217+0000","last_unstale":"2024-09-19T07:44:02.879643+0000","last_undegraded":"2024-09-19T07:44:02.879643+0000","last_fullsized":"2024-09-19T07:44:02.879643+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:57:42.145633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.633094+0000","last_change":"2024-09-19T07:44:01.620156+0000","last_active":"2024-09-19T07:44:02.633094+0000","last_peered":"2024-09-19T07:44:02.633094+0000","last_clean":"2024-09-19T07:44:02.633094+0000","last_became_active":"2024-09-19T07:44:01.619829+0000","last_became_peered":"2024-09-19T07:44:01.619829+0000","last_unstale":"2024-09-19T07:44:02.633094+0000","last_undegraded":"2024-09-19T07:44:02.633094+0000","last_fullsized":"2024-09-19T07:44:02.633094+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:17.164807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879680+0000","last_change":"2024-09-19T07:44:00.632242+0000","last_active":"2024-09-19T07:44:02.879680+0000","last_peered":"2024-09-19T07:44:02.879680+0000","last_clean":"2024-09-19T07:44:02.879680+0000","last_became_active":"2024-09-19T07:44:00.631953+0000","last_became_peered":"2024-09-19T07:44:00.631953+0000","last_unstale":"2024-09-19T07:44:02.879680+0000","last_undegraded":"2024-09-19T07:44:02.879680+0000","last_fullsized":"2024-09-19T07:44:02.879680+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:28.881326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879972+0000","last_change":"2024-09-19T07:44:01.643566+0000","last_active":"2024-09-19T07:44:02.879972+0000","last_peered":"2024-09-19T07:44:02.879972+0000","last_clean":"2024-09-19T07:44:02.879972+0000","last_became_active":"2024-09-19T07:44:01.643264+0000","last_became_peered":"2024-09-19T07:44:01.643264+0000","last_unstale":"2024-09-19T07:44:02.879972+0000","last_undegraded":"2024-09-19T07:44:02.879972+0000","last_fullsized":"2024-09-19T07:44:02.879972+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:15:37.153759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.880264+0000","last_change":"2024-09-19T07:44:00.629323+0000","last_active":"2024-09-19T07:44:02.880264+0000","last_peered":"2024-09-19T07:44:02.880264+0000","last_clean":"2024-09-19T07:44:02.880264+0000","last_became_active":"2024-09-19T07:44:00.629037+0000","last_became_peered":"2024-09-19T07:44:00.629037+0000","last_unstale":"2024-09-19T07:44:02.880264+0000","last_undegraded":"2024-09-19T07:44:02.880264+0000","last_fullsized":"2024-09-19T07:44:02.880264+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:07.746484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879371+0000","last_change":"2024-09-19T07:44:01.637663+0000","last_active":"2024-09-19T07:44:02.879371+0000","last_peered":"2024-09-19T07:44:02.879371+0000","last_clean":"2024-09-19T07:44:02.879371+0000","last_became_active":"2024-09-19T07:44:01.637391+0000","last_became_peered":"2024-09-19T07:44:01.637391+0000","last_unstale":"2024-09-19T07:44:02.879371+0000","last_undegraded":"2024-09-19T07:44:02.879371+0000","last_fullsized":"2024-09-19T07:44:02.879371+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:11.328826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.857755+0000","last_change":"2024-09-19T07:44:00.628801+0000","last_active":"2024-09-19T07:44:02.857755+0000","last_peered":"2024-09-19T07:44:02.857755+0000","last_clean":"2024-09-19T07:44:02.857755+0000","last_became_active":"2024-09-19T07:44:00.626609+0000","last_became_peered":"2024-09-19T07:44:00.626609+0000","last_unstale":"2024-09-19T07:44:02.857755+0000","last_undegraded":"2024-09-19T07:44:02.857755+0000","last_fullsized":"2024-09-19T07:44:02.857755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:31.700567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637203+0000","last_change":"2024-09-19T07:44:01.618078+0000","last_active":"2024-09-19T07:44:02.637203+0000","last_peered":"2024-09-19T07:44:02.637203+0000","last_clean":"2024-09-19T07:44:02.637203+0000","last_became_active":"2024-09-19T07:44:01.617847+0000","last_became_peered":"2024-09-19T07:44:01.617847+0000","last_unstale":"2024-09-19T07:44:02.637203+0000","last_undegraded":"2024-09-19T07:44:02.637203+0000","last_fullsized":"2024-09-19T07:44:02.637203+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:11.328826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.636690+0000","last_change":"2024-09-19T07:44:00.630615+0000","last_active":"2024-09-19T07:44:02.636690+0000","last_peered":"2024-09-19T07:44:02.636690+0000","last_clean":"2024-09-19T07:44:02.636690+0000","last_became_active":"2024-09-19T07:44:00.630335+0000","last_became_peered":"2024-09-19T07:44:00.630335+0000","last_unstale":"2024-09-19T07:44:02.636690+0000","last_undegraded":"2024-09-19T07:44:02.636690+0000","last_fullsized":"2024-09-19T07:44:02.636690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:26:30.972385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879426+0000","last_change":"2024-09-19T07:44:01.643552+0000","last_active":"2024-09-19T07:44:02.879426+0000","last_peered":"2024-09-19T07:44:02.879426+0000","last_clean":"2024-09-19T07:44:02.879426+0000","last_became_active":"2024-09-19T07:44:01.642710+0000","last_became_peered":"2024-09-19T07:44:01.642710+0000","last_unstale":"2024-09-19T07:44:02.879426+0000","last_undegraded":"2024-09-19T07:44:02.879426+0000","last_fullsized":"2024-09-19T07:44:02.879426+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:17.407644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.632629+0000","last_change":"2024-09-19T07:44:00.622888+0000","last_active":"2024-09-19T07:44:02.632629+0000","last_peered":"2024-09-19T07:44:02.632629+0000","last_clean":"2024-09-19T07:44:02.632629+0000","last_became_active":"2024-09-19T07:44:00.622610+0000","last_became_peered":"2024-09-19T07:44:00.622610+0000","last_unstale":"2024-09-19T07:44:02.632629+0000","last_undegraded":"2024-09-19T07:44:02.632629+0000","last_fullsized":"2024-09-19T07:44:02.632629+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:59.017448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.637807+0000","last_change":"2024-09-19T07:44:01.636163+0000","last_active":"2024-09-19T07:44:02.637807+0000","last_peered":"2024-09-19T07:44:02.637807+0000","last_clean":"2024-09-19T07:44:02.637807+0000","last_became_active":"2024-09-19T07:44:01.635746+0000","last_became_peered":"2024-09-19T07:44:01.635746+0000","last_unstale":"2024-09-19T07:44:02.637807+0000","last_undegraded":"2024-09-19T07:44:02.637807+0000","last_fullsized":"2024-09-19T07:44:02.637807+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:44:00.603665+0000","last_clean_scrub_stamp":"2024-09-19T07:44:00.603665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:00.266417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879621+0000","last_change":"2024-09-19T07:44:00.628940+0000","last_active":"2024-09-19T07:44:02.879621+0000","last_peered":"2024-09-19T07:44:02.879621+0000","last_clean":"2024-09-19T07:44:02.879621+0000","last_became_active":"2024-09-19T07:44:00.628668+0000","last_became_peered":"2024-09-19T07:44:00.628668+0000","last_unstale":"2024-09-19T07:44:02.879621+0000","last_undegraded":"2024-09-19T07:44:02.879621+0000","last_fullsized":"2024-09-19T07:44:02.879621+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:43:59.598514+0000","last_clean_scrub_stamp":"2024-09-19T07:43:59.598514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:07.092062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":203,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-19T07:44:02.879397+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:44:02.879397+0000","last_peered":"2024-09-19T07:44:02.879397+0000","last_clean":"2024-09-19T07:44:02.879397+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:44:02.879397+0000","last_undegraded":"2024-09-19T07:44:02.879397+0000","last_fullsized":"2024-09-19T07:44:02.879397+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1064960,"data_stored":788276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":196808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":462848,"data_stored":196338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":462848,"data_stored":196830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1052672,"data_stored":786628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31652,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":471040,"data_stored":197872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":446464,"data_stored":189624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":458752,"data_stored":196296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:44:09.832 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T07:44:09.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T07:44:09.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.978+0000 7ff7a5253640 1 Processor -- start 2024-09-19T07:44:09.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.978+0000 7ff7a5253640 1 -- start start 2024-09-19T07:44:09.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.979+0000 7ff7a5253640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff7a00d4af0 0x7ff7a0162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.979+0000 7ff7a5253640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7a00b4b70 0x7ff7a00b5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.979+0000 7ff7a5253640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7a015f250 con 0x7ff7a015e0a0 2024-09-19T07:44:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.979+0000 7ff7a5253640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff7a00b5560 con 0x7ff7a00d4af0 2024-09-19T07:44:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.979+0000 7ff7a5253640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff7a015f530 con 0x7ff7a00b4b70 2024-09-19T07:44:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.979+0000 7ff79ed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff7a015e0a0 0x7ff7a00d43e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44516/0 (socket says 172.21.15.16:44516) 2024-09-19T07:44:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.979+0000 7ff79ed9d640 1 -- 172.21.15.16:0/89393466 learned_addr learned my addr 172.21.15.16:0/89393466 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:44:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79e59c640 1 --2- 172.21.15.16:0/89393466 >> v2:172.21.15.146:3300/0 conn(0x7ff7a00d4af0 0x7ff7a0162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79dd9b640 1 -- 172.21.15.16:0/89393466 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3585666663 0 0) 0x7ff7a015f250 con 0x7ff7a015e0a0 2024-09-19T07:44:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79dd9b640 1 -- 172.21.15.16:0/89393466 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff788003520 con 0x7ff7a015e0a0 2024-09-19T07:44:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79f59e640 1 --2- 172.21.15.16:0/89393466 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7a00b4b70 0x7ff7a00b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79dd9b640 1 -- 172.21.15.16:0/89393466 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2721640727 0 0) 0x7ff788003520 con 0x7ff7a015e0a0 2024-09-19T07:44:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79dd9b640 1 -- 172.21.15.16:0/89393466 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7a00b4b70 msgr2=0x7ff7a00b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79dd9b640 1 --2- 172.21.15.16:0/89393466 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7a00b4b70 0x7ff7a00b5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79dd9b640 1 -- 172.21.15.16:0/89393466 >> v2:172.21.15.146:3300/0 conn(0x7ff7a00d4af0 msgr2=0x7ff7a0162a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79dd9b640 1 --2- 172.21.15.16:0/89393466 >> v2:172.21.15.146:3300/0 conn(0x7ff7a00d4af0 0x7ff7a0162a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79dd9b640 1 -- 172.21.15.16:0/89393466 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a015f810 con 0x7ff7a015e0a0 2024-09-19T07:44:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79dd9b640 1 -- 172.21.15.16:0/89393466 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7ff790001670 con 0x7ff7a015e0a0 2024-09-19T07:44:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79dd9b640 1 -- 172.21.15.16:0/89393466 >> v1:172.21.15.16:6789/0 conn(0x7ff7a015e0a0 legacy=0x7ff7a00d43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79dd9b640 1 --2- 172.21.15.16:0/89393466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7880039b0 0x7ff788003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79dd9b640 1 --2- 172.21.15.16:0/89393466 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7a00b4b70 0x7ff788004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79dd9b640 1 --2- 172.21.15.16:0/89393466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff788004d40 0x7ff7880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79dd9b640 1 -- 172.21.15.16:0/89393466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a00b5560 con 0x7ff788004d40 2024-09-19T07:44:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79dd9b640 1 -- 172.21.15.16:0/89393466 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a015f530 con 0x7ff7880039b0 2024-09-19T07:44:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79dd9b640 1 -- 172.21.15.16:0/89393466 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff78800b9c0 con 0x7ff7a00b4b70 2024-09-19T07:44:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79f59e640 1 --2- 172.21.15.16:0/89393466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff788004d40 0x7ff7880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79e59c640 1 --2- 172.21.15.16:0/89393466 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7a00b4b70 0x7ff788004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79ed9d640 1 --2- 172.21.15.16:0/89393466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7880039b0 0x7ff788003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:09.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79f59e640 1 -- 172.21.15.16:0/89393466 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7a00b4b70 msgr2=0x7ff788004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:09.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79f59e640 1 --2- 172.21.15.16:0/89393466 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7a00b4b70 0x7ff788004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:09.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79f59e640 1 -- 172.21.15.16:0/89393466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7880039b0 msgr2=0x7ff788003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:09.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.980+0000 7ff79f59e640 1 --2- 172.21.15.16:0/89393466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7880039b0 0x7ff788003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:09.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.981+0000 7ff79f59e640 1 -- 172.21.15.16:0/89393466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff788003520 con 0x7ff788004d40 2024-09-19T07:44:09.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.981+0000 7ff79f59e640 1 --2- 172.21.15.16:0/89393466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff788004d40 0x7ff7880051c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff79800e170 tx=0x7ff788005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187b941ad082fb92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:09.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.981+0000 7ff79dd9b640 1 -- 172.21.15.16:0/89393466 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff798034020 con 0x7ff788004d40 2024-09-19T07:44:09.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.981+0000 7ff79dd9b640 1 -- 172.21.15.16:0/89393466 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7980305f0 con 0x7ff788004d40 2024-09-19T07:44:09.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.981+0000 7ff79dd9b640 1 -- 172.21.15.16:0/89393466 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff798009c10 con 0x7ff788004d40 2024-09-19T07:44:09.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.982+0000 7ff7a5253640 1 -- 172.21.15.16:0/89393466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff788004d40 msgr2=0x7ff7880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:09.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.982+0000 7ff7a5253640 1 --2- 172.21.15.16:0/89393466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff788004d40 0x7ff7880051c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff79800e170 tx=0x7ff788005830 comp rx=0 tx=0).stop 2024-09-19T07:44:09.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.982+0000 7ff79f59e640 1 -- 172.21.15.16:0/89393466 reap_dead start 2024-09-19T07:44:09.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.982+0000 7ff7a5253640 1 -- 172.21.15.16:0/89393466 shutdown_connections 2024-09-19T07:44:09.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.982+0000 7ff7a5253640 1 -- 172.21.15.16:0/89393466 >> 172.21.15.16:0/89393466 conn(0x7ff7a009d770 msgr2=0x7ff7a008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:09.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.982+0000 7ff7a5253640 1 -- 172.21.15.16:0/89393466 shutdown_connections 2024-09-19T07:44:09.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.982+0000 7ff7a5253640 1 -- 172.21.15.16:0/89393466 wait complete. 2024-09-19T07:44:09.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.983+0000 7ff7a5253640 1 Processor -- start 2024-09-19T07:44:09.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.983+0000 7ff7a5253640 1 -- start start 2024-09-19T07:44:09.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.984+0000 7ff7a5253640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff788004d40 0x7ff7a00cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.984+0000 7ff7a5253640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7a014f6e0 0x7ff7a00b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.984+0000 7ff7a5253640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7a00b3160 0x7ff7a00d0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.984+0000 7ff7a5253640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a00b51a0 con 0x7ff788004d40 2024-09-19T07:44:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.984+0000 7ff7a5253640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a00b5020 con 0x7ff7a00b3160 2024-09-19T07:44:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.984+0000 7ff7a5253640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff7a00b5320 con 0x7ff7a014f6e0 2024-09-19T07:44:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.984+0000 7ff79ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff788004d40 0x7ff7a00cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.984+0000 7ff79ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff788004d40 0x7ff7a00cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51796/0 (socket says 172.21.15.16:51796) 2024-09-19T07:44:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.984+0000 7ff79ed9d640 1 -- 172.21.15.16:0/4034290375 learned_addr learned my addr 172.21.15.16:0/4034290375 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.984+0000 7ff79f59e640 1 --2- 172.21.15.16:0/4034290375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7a00b3160 0x7ff7a00d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.984+0000 7ff79e59c640 1 --2- 172.21.15.16:0/4034290375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7a014f6e0 0x7ff7a00b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.985+0000 7ff79ed9d640 1 -- 172.21.15.16:0/4034290375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7a014f6e0 msgr2=0x7ff7a00b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.985+0000 7ff79ed9d640 1 --2- 172.21.15.16:0/4034290375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7a014f6e0 0x7ff7a00b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.985+0000 7ff79ed9d640 1 -- 172.21.15.16:0/4034290375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7a00b3160 msgr2=0x7ff7a00d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.985+0000 7ff79ed9d640 1 --2- 172.21.15.16:0/4034290375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7a00b3160 0x7ff7a00d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.985+0000 7ff79ed9d640 1 -- 172.21.15.16:0/4034290375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a00b7320 con 0x7ff788004d40 2024-09-19T07:44:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.985+0000 7ff79ed9d640 1 --2- 172.21.15.16:0/4034290375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff788004d40 0x7ff7a00cb520 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff790007310 tx=0x7ff7900077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.985+0000 7ff7877fe640 1 -- 172.21.15.16:0/4034290375 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79000f960 con 0x7ff788004d40 2024-09-19T07:44:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.985+0000 7ff7877fe640 1 -- 172.21.15.16:0/4034290375 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff790018070 con 0x7ff788004d40 2024-09-19T07:44:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.985+0000 7ff7877fe640 1 -- 172.21.15.16:0/4034290375 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79000fd40 con 0x7ff788004d40 2024-09-19T07:44:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.985+0000 7ff7a5253640 1 -- 172.21.15.16:0/4034290375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a00b75c0 con 0x7ff788004d40 2024-09-19T07:44:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.985+0000 7ff7a5253640 1 -- 172.21.15.16:0/4034290375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a00b35e0 con 0x7ff788004d40 2024-09-19T07:44:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.986+0000 7ff7a5253640 1 -- 172.21.15.16:0/4034290375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7a0162a70 con 0x7ff788004d40 2024-09-19T07:44:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.987+0000 7ff7877fe640 1 -- 172.21.15.16:0/4034290375 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff79001d460 con 0x7ff788004d40 2024-09-19T07:44:09.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.987+0000 7ff7877fe640 1 --2- 172.21.15.16:0/4034290375 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff78c06c2b0 0x7ff78c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:09.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.987+0000 7ff7877fe640 1 -- 172.21.15.16:0/4034290375 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7ff79008b750 con 0x7ff788004d40 2024-09-19T07:44:09.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.987+0000 7ff79e59c640 1 --2- 172.21.15.16:0/4034290375 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff78c06c2b0 0x7ff78c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:09.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.988+0000 7ff79e59c640 1 --2- 172.21.15.16:0/4034290375 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff78c06c2b0 0x7ff78c06e770 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff7a014ff90 tx=0x7ff794008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:09.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:09.991+0000 7ff7877fe640 1 -- 172.21.15.16:0/4034290375 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff790059650 con 0x7ff788004d40 2024-09-19T07:44:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.235+0000 7ff7a5253640 1 -- 172.21.15.16:0/4034290375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff7a00ba3d0 con 0x7ff788004d40 2024-09-19T07:44:10.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.237+0000 7ff7877fe640 1 -- 172.21.15.16:0/4034290375 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7ff79005d950 con 0x7ff788004d40 2024-09-19T07:44:10.237 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-19T07:44:10.237 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA== 2024-09-19T07:44:10.237 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-19T07:44:10.238 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-19T07:44:10.238 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T07:44:10.238 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-19T07:44:10.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.239+0000 7ff7a5253640 1 -- 172.21.15.16:0/4034290375 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff78c06c2b0 msgr2=0x7ff78c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.239+0000 7ff7a5253640 1 --2- 172.21.15.16:0/4034290375 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff78c06c2b0 0x7ff78c06e770 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff7a014ff90 tx=0x7ff794008040 comp rx=0 tx=0).stop 2024-09-19T07:44:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.239+0000 7ff7a5253640 1 -- 172.21.15.16:0/4034290375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff788004d40 msgr2=0x7ff7a00cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.239+0000 7ff7a5253640 1 --2- 172.21.15.16:0/4034290375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff788004d40 0x7ff7a00cb520 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff790007310 tx=0x7ff7900077d0 comp rx=0 tx=0).stop 2024-09-19T07:44:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.239+0000 7ff7a5253640 1 -- 172.21.15.16:0/4034290375 shutdown_connections 2024-09-19T07:44:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.239+0000 7ff7a5253640 1 --2- 172.21.15.16:0/4034290375 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff78c06c2b0 0x7ff78c06e770 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.239+0000 7ff7a5253640 1 --2- 172.21.15.16:0/4034290375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7a00b3160 0x7ff7a00d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.239+0000 7ff7a5253640 1 --2- 172.21.15.16:0/4034290375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7a014f6e0 0x7ff7a00b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.239+0000 7ff7a5253640 1 --2- 172.21.15.16:0/4034290375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff788004d40 0x7ff7a00cb520 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.239+0000 7ff7a5253640 1 -- 172.21.15.16:0/4034290375 >> 172.21.15.16:0/4034290375 conn(0x7ff7a009d770 msgr2=0x7ff7a015e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.239+0000 7ff7a5253640 1 -- 172.21.15.16:0/4034290375 shutdown_connections 2024-09-19T07:44:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.239+0000 7ff7a5253640 1 -- 172.21.15.16:0/4034290375 wait complete. 2024-09-19T07:44:10.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:44:10.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.352+0000 7fbe65c9b640 1 Processor -- start 2024-09-19T07:44:10.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.352+0000 7fbe65c9b640 1 -- start start 2024-09-19T07:44:10.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.352+0000 7fbe65c9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbe6015e680 0x7fbe600cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:10.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.353+0000 7fbe65c9b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbe600d0010 0x7fbe600d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:10.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.353+0000 7fbe65c9b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe600d1880 con 0x7fbe6015e2a0 2024-09-19T07:44:10.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.353+0000 7fbe65c9b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fbe600d1b60 con 0x7fbe6015e680 2024-09-19T07:44:10.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.353+0000 7fbe65c9b640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fbe600cf2d0 con 0x7fbe600d0010 2024-09-19T07:44:10.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.353+0000 7fbe5effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbe6015e680 0x7fbe600cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:10.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.353+0000 7fbe5effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbe6015e680 0x7fbe600cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51806/0 (socket says 172.21.15.16:51806) 2024-09-19T07:44:10.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.353+0000 7fbe5effd640 1 -- 172.21.15.16:0/2760848923 learned_addr learned my addr 172.21.15.16:0/2760848923 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:10.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.353+0000 7fbe5effd640 1 -- 172.21.15.16:0/2760848923 >> v2:172.21.15.146:3300/0 conn(0x7fbe600d0010 msgr2=0x7fbe600d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:10.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.353+0000 7fbe5effd640 1 --2- 172.21.15.16:0/2760848923 >> v2:172.21.15.146:3300/0 conn(0x7fbe600d0010 0x7fbe600d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:10.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.353+0000 7fbe5effd640 1 -- 172.21.15.16:0/2760848923 >> v1:172.21.15.146:6789/0 conn(0x7fbe6015e2a0 legacy=0x7fbe6006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:10.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.353+0000 7fbe5effd640 1 -- 172.21.15.16:0/2760848923 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe600cf5b0 con 0x7fbe6015e680 2024-09-19T07:44:10.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.354+0000 7fbe5effd640 1 --2- 172.21.15.16:0/2760848923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6015e680 0x7fbe600cfad0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbe60163050 tx=0x7fbe5402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143dce047a4a3137 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:10.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.354+0000 7fbe5e7fc640 1 -- 172.21.15.16:0/2760848923 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54037070 con 0x7fbe6015e680 2024-09-19T07:44:10.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.354+0000 7fbe5e7fc640 1 -- 172.21.15.16:0/2760848923 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe54032440 con 0x7fbe6015e680 2024-09-19T07:44:10.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.354+0000 7fbe5e7fc640 1 -- 172.21.15.16:0/2760848923 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54032810 con 0x7fbe6015e680 2024-09-19T07:44:10.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.354+0000 7fbe65c9b640 1 -- 172.21.15.16:0/2760848923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6015e680 msgr2=0x7fbe600cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:10.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.354+0000 7fbe65c9b640 1 --2- 172.21.15.16:0/2760848923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6015e680 0x7fbe600cfad0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbe60163050 tx=0x7fbe5402a1a0 comp rx=0 tx=0).stop 2024-09-19T07:44:10.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.354+0000 7fbe65c9b640 1 -- 172.21.15.16:0/2760848923 shutdown_connections 2024-09-19T07:44:10.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.354+0000 7fbe65c9b640 1 --2- 172.21.15.16:0/2760848923 >> v2:172.21.15.146:3300/0 conn(0x7fbe600d0010 0x7fbe600d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:10.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.354+0000 7fbe65c9b640 1 --2- 172.21.15.16:0/2760848923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6015e680 0x7fbe600cfad0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:10.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.354+0000 7fbe65c9b640 1 -- 172.21.15.16:0/2760848923 >> 172.21.15.16:0/2760848923 conn(0x7fbe6009d760 msgr2=0x7fbe6008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:10.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.354+0000 7fbe65c9b640 1 -- 172.21.15.16:0/2760848923 shutdown_connections 2024-09-19T07:44:10.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.355+0000 7fbe65c9b640 1 -- 172.21.15.16:0/2760848923 wait complete. 2024-09-19T07:44:10.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.356+0000 7fbe65c9b640 1 Processor -- start 2024-09-19T07:44:10.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.356+0000 7fbe65c9b640 1 -- start start 2024-09-19T07:44:10.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.356+0000 7fbe65c9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe600d0010 0x7fbe601f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:10.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe65c9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6015e2a0 0x7fbe600ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:10.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe65c9b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbe6015e680 0x7fbe600b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:10.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe65c9b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6006dcb0 con 0x7fbe6015e2a0 2024-09-19T07:44:10.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe65c9b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6006db30 con 0x7fbe600d0010 2024-09-19T07:44:10.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe65c9b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbe600d48b0 con 0x7fbe6015e680 2024-09-19T07:44:10.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe5effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6015e2a0 0x7fbe600ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:10.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe5effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6015e2a0 0x7fbe600ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51812/0 (socket says 172.21.15.16:51812) 2024-09-19T07:44:10.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe5effd640 1 -- 172.21.15.16:0/3615432469 learned_addr learned my addr 172.21.15.16:0/3615432469 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:10.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe5f7fe640 1 --2- 172.21.15.16:0/3615432469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe600d0010 0x7fbe601f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:10.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe5ffff640 1 --2- 172.21.15.16:0/3615432469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbe6015e680 0x7fbe600b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:10.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe5effd640 1 -- 172.21.15.16:0/3615432469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbe6015e680 msgr2=0x7fbe600b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:10.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe5effd640 1 --2- 172.21.15.16:0/3615432469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbe6015e680 0x7fbe600b8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:10.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe5effd640 1 -- 172.21.15.16:0/3615432469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe600d0010 msgr2=0x7fbe601f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:10.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe5effd640 1 --2- 172.21.15.16:0/3615432469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe600d0010 0x7fbe601f4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:10.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe5effd640 1 -- 172.21.15.16:0/3615432469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe600c7f10 con 0x7fbe6015e2a0 2024-09-19T07:44:10.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe5effd640 1 --2- 172.21.15.16:0/3615432469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6015e2a0 0x7fbe600ca770 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbe5402a1e0 tx=0x7fbe540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:10.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe47fff640 1 -- 172.21.15.16:0/3615432469 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54041070 con 0x7fbe6015e2a0 2024-09-19T07:44:10.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe47fff640 1 -- 172.21.15.16:0/3615432469 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe5403c890 con 0x7fbe6015e2a0 2024-09-19T07:44:10.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe65c9b640 1 -- 172.21.15.16:0/3615432469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe600c81b0 con 0x7fbe6015e2a0 2024-09-19T07:44:10.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.357+0000 7fbe47fff640 1 -- 172.21.15.16:0/3615432469 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54037040 con 0x7fbe6015e2a0 2024-09-19T07:44:10.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.358+0000 7fbe65c9b640 1 -- 172.21.15.16:0/3615432469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe600c8770 con 0x7fbe6015e2a0 2024-09-19T07:44:10.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.359+0000 7fbe65c9b640 1 -- 172.21.15.16:0/3615432469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe600d2440 con 0x7fbe6015e2a0 2024-09-19T07:44:10.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.359+0000 7fbe47fff640 1 -- 172.21.15.16:0/3615432469 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbe54044070 con 0x7fbe6015e2a0 2024-09-19T07:44:10.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.359+0000 7fbe47fff640 1 --2- 172.21.15.16:0/3615432469 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbe4c06c2b0 0x7fbe4c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:10.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.360+0000 7fbe47fff640 1 -- 172.21.15.16:0/3615432469 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fbe540ac310 con 0x7fbe6015e2a0 2024-09-19T07:44:10.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.360+0000 7fbe5f7fe640 1 --2- 172.21.15.16:0/3615432469 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbe4c06c2b0 0x7fbe4c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:10.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.360+0000 7fbe5f7fe640 1 --2- 172.21.15.16:0/3615432469 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbe4c06c2b0 0x7fbe4c06e770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbe601e7fb0 tx=0x7fbe50009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:10.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.364+0000 7fbe47fff640 1 -- 172.21.15.16:0/3615432469 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fbe600d2440 con 0x7fbe6015e2a0 2024-09-19T07:44:10.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.576+0000 7fbe65c9b640 1 -- 172.21.15.16:0/3615432469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe600d4c70 con 0x7fbe6015e2a0 2024-09-19T07:44:10.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.576+0000 7fbe47fff640 1 -- 172.21.15.16:0/3615432469 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5229 (secure 0 0 0) 0x7fbe5407a3c0 con 0x7fbe6015e2a0 2024-09-19T07:44:10.577 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:44:10.577 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-19T07:44:04:108666+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4655,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/3238551814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3238551814},{"type":"v1","addr":"172.21.15.146:6836","nonce":3238551814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3450384245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3450384245},{"type":"v1","addr":"172.21.15.146:6837","nonce":3450384245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4944,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3663416804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3663416804},{"type":"v1","addr":"172.21.15.16:6834","nonce":3663416804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:44:00.970677+0000","modified":"2024-09-19T07:44:03.455740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4947":{"gid":4947,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2028210099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2028210099},{"type":"v1","addr":"172.21.15.16:6835","nonce":2028210099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4947,"qdb_cluster":[4947]},"id":2}]} 2024-09-19T07:44:10.577 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-19T07:44:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.578+0000 7fbe65c9b640 1 -- 172.21.15.16:0/3615432469 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbe4c06c2b0 msgr2=0x7fbe4c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.578+0000 7fbe65c9b640 1 --2- 172.21.15.16:0/3615432469 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbe4c06c2b0 0x7fbe4c06e770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbe601e7fb0 tx=0x7fbe50009040 comp rx=0 tx=0).stop 2024-09-19T07:44:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.578+0000 7fbe65c9b640 1 -- 172.21.15.16:0/3615432469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6015e2a0 msgr2=0x7fbe600ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:10.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.578+0000 7fbe65c9b640 1 --2- 172.21.15.16:0/3615432469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6015e2a0 0x7fbe600ca770 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbe5402a1e0 tx=0x7fbe540046c0 comp rx=0 tx=0).stop 2024-09-19T07:44:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.579+0000 7fbe65c9b640 1 -- 172.21.15.16:0/3615432469 shutdown_connections 2024-09-19T07:44:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.579+0000 7fbe65c9b640 1 --2- 172.21.15.16:0/3615432469 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbe4c06c2b0 0x7fbe4c06e770 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.579+0000 7fbe65c9b640 1 --2- 172.21.15.16:0/3615432469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbe6015e680 0x7fbe600b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.579+0000 7fbe65c9b640 1 --2- 172.21.15.16:0/3615432469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe6015e2a0 0x7fbe600ca770 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.579+0000 7fbe65c9b640 1 --2- 172.21.15.16:0/3615432469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe600d0010 0x7fbe601f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.579+0000 7fbe65c9b640 1 -- 172.21.15.16:0/3615432469 >> 172.21.15.16:0/3615432469 conn(0x7fbe6009d760 msgr2=0x7fbe6006d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.579+0000 7fbe65c9b640 1 -- 172.21.15.16:0/3615432469 shutdown_connections 2024-09-19T07:44:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:10.579+0000 7fbe65c9b640 1 -- 172.21.15.16:0/3615432469 wait complete. 2024-09-19T07:44:10.992 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T07:44:00.970677+0000', 'modified': '2024-09-19T07:44:03.455740+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4947}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4947': {'gid': 4947, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6835/2028210099', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6833', 'nonce': 2028210099}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 2028210099}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4947, 'qdb_cluster': [4947]} 2024-09-19T07:44:10.993 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T07:44:10.993 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T07:44:11.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.100+0000 7f2c7a7d6640 1 Processor -- start 2024-09-19T07:44:11.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.100+0000 7f2c7a7d6640 1 -- start start 2024-09-19T07:44:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.101+0000 7f2c7a7d6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c7415e680 0x7f2c740cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.101+0000 7f2c7a7d6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c740d0010 0x7f2c740d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.101+0000 7f2c7a7d6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c740d1880 con 0x7f2c7415e2a0 2024-09-19T07:44:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.101+0000 7f2c7a7d6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2c740d1b60 con 0x7f2c7415e680 2024-09-19T07:44:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.101+0000 7f2c7a7d6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2c740cf2d0 con 0x7f2c740d0010 2024-09-19T07:44:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.101+0000 7f2c737fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c7415e680 0x7f2c740cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.101+0000 7f2c737fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c7415e680 0x7f2c740cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51820/0 (socket says 172.21.15.16:51820) 2024-09-19T07:44:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.101+0000 7f2c737fe640 1 -- 172.21.15.16:0/1466688442 learned_addr learned my addr 172.21.15.16:0/1466688442 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.102+0000 7f2c78d73640 1 --2- 172.21.15.16:0/1466688442 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c740d0010 0x7f2c740d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.102+0000 7f2c737fe640 1 -- 172.21.15.16:0/1466688442 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c740d0010 msgr2=0x7f2c740d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.102+0000 7f2c737fe640 1 --2- 172.21.15.16:0/1466688442 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c740d0010 0x7f2c740d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.102+0000 7f2c737fe640 1 -- 172.21.15.16:0/1466688442 >> v1:172.21.15.146:6789/0 conn(0x7f2c7415e2a0 legacy=0x7f2c7406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.102+0000 7f2c737fe640 1 -- 172.21.15.16:0/1466688442 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c740cf5b0 con 0x7f2c7415e680 2024-09-19T07:44:11.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.102+0000 7f2c737fe640 1 --2- 172.21.15.16:0/1466688442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7415e680 0x7f2c740cfad0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009870 tx=0x7f2c6802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12dd135c94d1cd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:11.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.103+0000 7f2c72ffd640 1 -- 172.21.15.16:0/1466688442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68037070 con 0x7f2c7415e680 2024-09-19T07:44:11.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.103+0000 7f2c72ffd640 1 -- 172.21.15.16:0/1466688442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c68032440 con 0x7f2c7415e680 2024-09-19T07:44:11.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.103+0000 7f2c72ffd640 1 -- 172.21.15.16:0/1466688442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68032810 con 0x7f2c7415e680 2024-09-19T07:44:11.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.103+0000 7f2c7a7d6640 1 -- 172.21.15.16:0/1466688442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7415e680 msgr2=0x7f2c740cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:11.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.103+0000 7f2c7a7d6640 1 --2- 172.21.15.16:0/1466688442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7415e680 0x7f2c740cfad0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009870 tx=0x7f2c6802a180 comp rx=0 tx=0).stop 2024-09-19T07:44:11.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.103+0000 7f2c7a7d6640 1 -- 172.21.15.16:0/1466688442 shutdown_connections 2024-09-19T07:44:11.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.103+0000 7f2c7a7d6640 1 --2- 172.21.15.16:0/1466688442 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c740d0010 0x7f2c740d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.103+0000 7f2c7a7d6640 1 --2- 172.21.15.16:0/1466688442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7415e680 0x7f2c740cfad0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.103+0000 7f2c7a7d6640 1 -- 172.21.15.16:0/1466688442 >> 172.21.15.16:0/1466688442 conn(0x7f2c7409d760 msgr2=0x7f2c7408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:11.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.103+0000 7f2c7a7d6640 1 -- 172.21.15.16:0/1466688442 shutdown_connections 2024-09-19T07:44:11.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.103+0000 7f2c7a7d6640 1 -- 172.21.15.16:0/1466688442 wait complete. 2024-09-19T07:44:11.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.104+0000 7f2c7a7d6640 1 Processor -- start 2024-09-19T07:44:11.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.105+0000 7f2c7a7d6640 1 -- start start 2024-09-19T07:44:11.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.105+0000 7f2c7a7d6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c740d0010 0x7f2c740cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.105+0000 7f2c7a7d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7415e2a0 0x7f2c741e8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.105+0000 7f2c7a7d6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c7415e680 0x7f2c7415d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.105+0000 7f2c7a7d6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7406dcb0 con 0x7f2c7415e2a0 2024-09-19T07:44:11.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.105+0000 7f2c7a7d6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7406db30 con 0x7f2c7415e680 2024-09-19T07:44:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.105+0000 7f2c7a7d6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2c740d48b0 con 0x7f2c740d0010 2024-09-19T07:44:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.105+0000 7f2c737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7415e2a0 0x7f2c741e8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.105+0000 7f2c737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7415e2a0 0x7f2c741e8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51834/0 (socket says 172.21.15.16:51834) 2024-09-19T07:44:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.105+0000 7f2c737fe640 1 -- 172.21.15.16:0/2718565400 learned_addr learned my addr 172.21.15.16:0/2718565400 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.105+0000 7f2c73fff640 1 --2- 172.21.15.16:0/2718565400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c740d0010 0x7f2c740cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.105+0000 7f2c78d73640 1 --2- 172.21.15.16:0/2718565400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c7415e680 0x7f2c7415d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.106+0000 7f2c737fe640 1 -- 172.21.15.16:0/2718565400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c740d0010 msgr2=0x7f2c740cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.106+0000 7f2c737fe640 1 --2- 172.21.15.16:0/2718565400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c740d0010 0x7f2c740cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.106+0000 7f2c737fe640 1 -- 172.21.15.16:0/2718565400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c7415e680 msgr2=0x7f2c7415d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.106+0000 7f2c737fe640 1 --2- 172.21.15.16:0/2718565400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c7415e680 0x7f2c7415d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.106+0000 7f2c737fe640 1 -- 172.21.15.16:0/2718565400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c741e95a0 con 0x7f2c7415e2a0 2024-09-19T07:44:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.106+0000 7f2c78d73640 1 --2- 172.21.15.16:0/2718565400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c7415e680 0x7f2c7415d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.106+0000 7f2c737fe640 1 --2- 172.21.15.16:0/2718565400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7415e2a0 0x7f2c741e8e00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2c680090b0 tx=0x7f2c680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:11.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.106+0000 7f2c70ff9640 1 -- 172.21.15.16:0/2718565400 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68041020 con 0x7f2c7415e2a0 2024-09-19T07:44:11.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.106+0000 7f2c70ff9640 1 -- 172.21.15.16:0/2718565400 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c68030990 con 0x7f2c7415e2a0 2024-09-19T07:44:11.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.106+0000 7f2c7a7d6640 1 -- 172.21.15.16:0/2718565400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c741e9880 con 0x7f2c7415e2a0 2024-09-19T07:44:11.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.106+0000 7f2c70ff9640 1 -- 172.21.15.16:0/2718565400 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68037040 con 0x7f2c7415e2a0 2024-09-19T07:44:11.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.106+0000 7f2c7a7d6640 1 -- 172.21.15.16:0/2718565400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c741e9e40 con 0x7f2c7415e2a0 2024-09-19T07:44:11.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.108+0000 7f2c70ff9640 1 -- 172.21.15.16:0/2718565400 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2c68044020 con 0x7f2c7415e2a0 2024-09-19T07:44:11.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.108+0000 7f2c70ff9640 1 --2- 172.21.15.16:0/2718565400 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2c6006c2b0 0x7f2c6006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.108+0000 7f2c70ff9640 1 -- 172.21.15.16:0/2718565400 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f2c680abeb0 con 0x7f2c7415e2a0 2024-09-19T07:44:11.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.108+0000 7f2c73fff640 1 --2- 172.21.15.16:0/2718565400 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2c6006c2b0 0x7f2c6006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.109+0000 7f2c73fff640 1 --2- 172.21.15.16:0/2718565400 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2c6006c2b0 0x7f2c6006e770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2c64004f60 tx=0x7f2c64007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:11.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.109+0000 7f2c7d64e740 1 Processor -- start 2024-09-19T07:44:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.109+0000 7f2c7d64e740 1 -- start start 2024-09-19T07:44:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c7d64e740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36b2c0 0x5638cb3b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c7d64e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb368a80 0x5638cb36b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c7d64e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb36ca50 0x5638cb3658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c7d64e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5638cb2cd520 con 0x5638cb368a80 2024-09-19T07:44:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c7d64e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5638cb2753e0 con 0x5638cb36ca50 2024-09-19T07:44:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c7d64e740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x5638cb2c94e0 con 0x5638cb36b2c0 2024-09-19T07:44:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb368a80 0x5638cb36b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb368a80 0x5638cb36b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51836/0 (socket says 172.21.15.16:51836) 2024-09-19T07:44:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c737fe640 1 -- 172.21.15.16:0/1339434522 learned_addr learned my addr 172.21.15.16:0/1339434522 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c73fff640 1 --2- 172.21.15.16:0/1339434522 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36b2c0 0x5638cb3b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c78d73640 1 --2- 172.21.15.16:0/1339434522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb36ca50 0x5638cb3658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c737fe640 1 -- 172.21.15.16:0/1339434522 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36b2c0 msgr2=0x5638cb3b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:11.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c737fe640 1 --2- 172.21.15.16:0/1339434522 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36b2c0 0x5638cb3b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c737fe640 1 -- 172.21.15.16:0/1339434522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb36ca50 msgr2=0x5638cb3658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:11.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c737fe640 1 --2- 172.21.15.16:0/1339434522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb36ca50 0x5638cb3658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c737fe640 1 -- 172.21.15.16:0/1339434522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638cb2902f0 con 0x5638cb368a80 2024-09-19T07:44:11.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c73fff640 1 --2- 172.21.15.16:0/1339434522 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36b2c0 0x5638cb3b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:11.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c737fe640 1 --2- 172.21.15.16:0/1339434522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb368a80 0x5638cb36b6a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2c680aa280 tx=0x7f2c68079960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5d0238c1089cdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:11.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c59ffb640 1 -- 172.21.15.16:0/1339434522 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c680a8070 con 0x5638cb368a80 2024-09-19T07:44:11.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c59ffb640 1 -- 172.21.15.16:0/1339434522 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c68079e20 con 0x5638cb368a80 2024-09-19T07:44:11.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.110+0000 7f2c59ffb640 1 -- 172.21.15.16:0/1339434522 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6807e1d0 con 0x5638cb368a80 2024-09-19T07:44:11.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.111+0000 7f2c7d64e740 1 -- 172.21.15.16:0/1339434522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb368a80 msgr2=0x5638cb36b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:11.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.111+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/1339434522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb368a80 0x5638cb36b6a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2c680aa280 tx=0x7f2c68079960 comp rx=0 tx=0).stop 2024-09-19T07:44:11.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.111+0000 7f2c7d64e740 1 -- 172.21.15.16:0/1339434522 shutdown_connections 2024-09-19T07:44:11.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.111+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/1339434522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb36ca50 0x5638cb3658b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.111+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/1339434522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb368a80 0x5638cb36b6a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.111+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/1339434522 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36b2c0 0x5638cb3b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.111+0000 7f2c7d64e740 1 -- 172.21.15.16:0/1339434522 >> 172.21.15.16:0/1339434522 conn(0x5638cb367a80 msgr2=0x5638cb3aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:11.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.111+0000 7f2c7d64e740 1 -- 172.21.15.16:0/1339434522 shutdown_connections 2024-09-19T07:44:11.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.111+0000 7f2c7d64e740 1 -- 172.21.15.16:0/1339434522 wait complete. 2024-09-19T07:44:11.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.112+0000 7f2c7d64e740 1 -- start start 2024-09-19T07:44:11.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.112+0000 7f2c7d64e740 1 Processor -- start 2024-09-19T07:44:11.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c7d64e740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36b2c0 0x5638cb38ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c7d64e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb36ca50 0x5638cb38d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c7d64e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb3647d0 0x5638cb3863e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c7d64e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5638cb2cd520 con 0x5638cb3647d0 2024-09-19T07:44:11.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c7d64e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5638cb256450 con 0x5638cb36ca50 2024-09-19T07:44:11.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c7d64e740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x5638cb366f20 con 0x5638cb36b2c0 2024-09-19T07:44:11.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c7d64e740 10 client.0 _ll_register_callbacks cb 0x5638cafdfac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c78d73640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb3647d0 0x5638cb3863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c78d73640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb3647d0 0x5638cb3863e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51852/0 (socket says 172.21.15.16:51852) 2024-09-19T07:44:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c78d73640 1 -- 172.21.15.16:0/2968096615 learned_addr learned my addr 172.21.15.16:0/2968096615 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c73fff640 1 --2- 172.21.15.16:0/2968096615 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36b2c0 0x5638cb38ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c737fe640 1 --2- 172.21.15.16:0/2968096615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb36ca50 0x5638cb38d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c78d73640 1 -- 172.21.15.16:0/2968096615 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36b2c0 msgr2=0x5638cb38ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c78d73640 1 --2- 172.21.15.16:0/2968096615 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36b2c0 0x5638cb38ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c78d73640 1 -- 172.21.15.16:0/2968096615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb36ca50 msgr2=0x5638cb38d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c78d73640 1 --2- 172.21.15.16:0/2968096615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb36ca50 0x5638cb38d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c78d73640 1 -- 172.21.15.16:0/2968096615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638cafe02d0 con 0x5638cb3647d0 2024-09-19T07:44:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c78d73640 1 --2- 172.21.15.16:0/2968096615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb3647d0 0x5638cb3863e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c010c00 tx=0x7f2c6c00e760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa15686c1eb7b96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:44:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c52ffd640 1 -- 172.21.15.16:0/2968096615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c01d060 con 0x5638cb3647d0 2024-09-19T07:44:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c52ffd640 1 -- 172.21.15.16:0/2968096615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f2c6c00a790 con 0x5638cb3647d0 2024-09-19T07:44:11.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.113+0000 7f2c7d64e740 1 -- 172.21.15.16:0/2968096615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c68079e20 con 0x5638cb3647d0 2024-09-19T07:44:11.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.114+0000 7f2c52ffd640 1 -- 172.21.15.16:0/2968096615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f2c6c023050 con 0x5638cb3647d0 2024-09-19T07:44:11.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.114+0000 7f2c7d64e740 10 client.5133 fetch_fsmap learned FSMap version 16 2024-09-19T07:44:11.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.114+0000 7f2c7d64e740 1 -- 172.21.15.16:0/2968096615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x5638cb0aa6b0 con 0x5638cb3647d0 2024-09-19T07:44:11.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.114+0000 7f2c52ffd640 1 -- 172.21.15.16:0/2968096615 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c68079e20 con 0x5638cb3647d0 2024-09-19T07:44:11.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.114+0000 7f2c52ffd640 1 -- 172.21.15.16:0/2968096615 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 16) ==== 3502+0+0 (secure 0 0 0) 0x7f2c6c026020 con 0x5638cb3647d0 2024-09-19T07:44:11.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.114+0000 7f2c7d64e740 10 client.5133 fetch_fsmap finished waiting for FSMap version 16 2024-09-19T07:44:11.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.114+0000 7f2c7d64e740 10 client.5133 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T07:44:11.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.114+0000 7f2c7d64e740 20 client.5133 populate_metadata read hostname 'smithi016' 2024-09-19T07:44:11.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.114+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/2968096615 >> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] conn(0x5638cb393d00 0x5638cb3cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.114+0000 7f2c7d64e740 4 client.5133 mds_command: new command op to 4947 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:44:11.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.114+0000 7f2c7d64e740 1 -- 172.21.15.16:0/2968096615 --> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638cb2dac50 con 0x5638cb393d00 2024-09-19T07:44:11.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.114+0000 7f2c73fff640 1 --2- 172.21.15.16:0/2968096615 >> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] conn(0x5638cb393d00 0x5638cb3cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.114+0000 7f2c52ffd640 10 client.5133 ms_handle_connect on v2:172.21.15.16:6833/2028210099 2024-09-19T07:44:11.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.114+0000 7f2c73fff640 1 --2- 172.21.15.16:0/2968096615 >> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] conn(0x5638cb393d00 0x5638cb3cdd40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c85aedba7208519 server_cookie=d5c47a61556f757 in_seq=0 out_seq=0 2024-09-19T07:44:11.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.115+0000 7f2c52ffd640 1 -- 172.21.15.16:0/2968096615 <== mds.0 v2:172.21.15.16:6833/2028210099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638cb2dac50 con 0x5638cb393d00 2024-09-19T07:44:11.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.115+0000 7f2c52ffd640 10 client.5133 handle_command_reply: tid=0 2024-09-19T07:44:11.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.115+0000 7f2c7d64e740 1 client.5133 shutdown 2024-09-19T07:44:11.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.115+0000 7f2c7d64e740 1 -- 172.21.15.16:0/2968096615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb3647d0 msgr2=0x5638cb3863e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:11.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.115+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/2968096615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb3647d0 0x5638cb3863e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c010c00 tx=0x7f2c6c00e760 comp rx=0 tx=0).stop 2024-09-19T07:44:11.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.115+0000 7f2c7d64e740 1 -- 172.21.15.16:0/2968096615 shutdown_connections 2024-09-19T07:44:11.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.115+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/2968096615 >> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] conn(0x5638cb393d00 0x5638cb3cdd40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.115+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/2968096615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb3647d0 0x5638cb3863e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.115+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/2968096615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb36ca50 0x5638cb38d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.115+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/2968096615 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36b2c0 0x5638cb38ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.115+0000 7f2c7d64e740 1 -- 172.21.15.16:0/2968096615 >> 172.21.15.16:0/2968096615 conn(0x5638cb368a80 msgr2=0x5638cb3af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:11.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.115+0000 7f2c52ffd640 0 client.5133 ms_handle_reset on v2:172.21.15.16:6833/2028210099 2024-09-19T07:44:11.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.115+0000 7f2c7d64e740 1 -- 172.21.15.16:0/2968096615 shutdown_connections 2024-09-19T07:44:11.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.115+0000 7f2c7d64e740 1 -- 172.21.15.16:0/2968096615 wait complete. 2024-09-19T07:44:11.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.115+0000 7f2c7d64e740 20 client.5133 trim_cache size 0 max 16384 2024-09-19T07:44:11.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.134+0000 7f2c7d64e740 1 Processor -- start 2024-09-19T07:44:11.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.134+0000 7f2c7d64e740 1 -- start start 2024-09-19T07:44:11.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.134+0000 7f2c7d64e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb3647d0 0x5638cb38a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.134+0000 7f2c7d64e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb36b2c0 0x5638cb38ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.134+0000 7f2c7d64e740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36ca50 0x5638cb3b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.134+0000 7f2c7d64e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c74200110 con 0x5638cb36b2c0 2024-09-19T07:44:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.134+0000 7f2c7d64e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2c741ff5f0 con 0x5638cb3647d0 2024-09-19T07:44:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.134+0000 7f2c7d64e740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2c741ffaf0 con 0x5638cb36ca50 2024-09-19T07:44:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.135+0000 7f2c737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb36b2c0 0x5638cb38ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.135+0000 7f2c737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb36b2c0 0x5638cb38ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51866/0 (socket says 172.21.15.16:51866) 2024-09-19T07:44:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.135+0000 7f2c737fe640 1 -- 172.21.15.16:0/1083594451 learned_addr learned my addr 172.21.15.16:0/1083594451 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.135+0000 7f2c78d73640 1 --2- 172.21.15.16:0/1083594451 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36ca50 0x5638cb3b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.135+0000 7f2c73fff640 1 --2- 172.21.15.16:0/1083594451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb3647d0 0x5638cb38a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.135+0000 7f2c737fe640 1 -- 172.21.15.16:0/1083594451 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36ca50 msgr2=0x5638cb3b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.135+0000 7f2c737fe640 1 --2- 172.21.15.16:0/1083594451 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36ca50 0x5638cb3b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.135+0000 7f2c737fe640 1 -- 172.21.15.16:0/1083594451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb3647d0 msgr2=0x5638cb38a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.135+0000 7f2c737fe640 1 --2- 172.21.15.16:0/1083594451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb3647d0 0x5638cb38a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.135+0000 7f2c737fe640 1 -- 172.21.15.16:0/1083594451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638cb36c3f0 con 0x5638cb36b2c0 2024-09-19T07:44:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.135+0000 7f2c737fe640 1 --2- 172.21.15.16:0/1083594451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb36b2c0 0x5638cb38ce20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2c680aa330 tx=0x7f2c6808f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea63a6c6bf2c9f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.135+0000 7f2c597fa640 1 -- 172.21.15.16:0/1083594451 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c680a8040 con 0x5638cb36b2c0 2024-09-19T07:44:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.135+0000 7f2c597fa640 1 -- 172.21.15.16:0/1083594451 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c6807e1d0 con 0x5638cb36b2c0 2024-09-19T07:44:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.135+0000 7f2c597fa640 1 -- 172.21.15.16:0/1083594451 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68079e20 con 0x5638cb36b2c0 2024-09-19T07:44:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.136+0000 7f2c7d64e740 1 -- 172.21.15.16:0/1083594451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb36b2c0 msgr2=0x5638cb38ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.136+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/1083594451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb36b2c0 0x5638cb38ce20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2c680aa330 tx=0x7f2c6808f000 comp rx=0 tx=0).stop 2024-09-19T07:44:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.136+0000 7f2c7d64e740 1 -- 172.21.15.16:0/1083594451 shutdown_connections 2024-09-19T07:44:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.136+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/1083594451 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36ca50 0x5638cb3b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.136+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/1083594451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb36b2c0 0x5638cb38ce20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.136+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/1083594451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb3647d0 0x5638cb38a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.136+0000 7f2c7d64e740 1 -- 172.21.15.16:0/1083594451 >> 172.21.15.16:0/1083594451 conn(0x5638cb368a80 msgr2=0x5638cb394b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.136+0000 7f2c7d64e740 1 -- 172.21.15.16:0/1083594451 shutdown_connections 2024-09-19T07:44:11.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.136+0000 7f2c7d64e740 1 -- 172.21.15.16:0/1083594451 wait complete. 2024-09-19T07:44:11.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.136+0000 7f2c7d64e740 1 -- start start 2024-09-19T07:44:11.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.137+0000 7f2c7d64e740 1 Processor -- start 2024-09-19T07:44:11.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.137+0000 7f2c7d64e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb3647d0 0x5638cb364f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.137+0000 7f2c7d64e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb36b2c0 0x5638cb390580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.137+0000 7f2c7d64e740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36ca50 0x5638cb390ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.137+0000 7f2c7d64e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5638cb393b80 con 0x5638cb36b2c0 2024-09-19T07:44:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.137+0000 7f2c7d64e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5638cb3957c0 con 0x5638cb3647d0 2024-09-19T07:44:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.137+0000 7f2c7d64e740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x5638cb394140 con 0x5638cb36ca50 2024-09-19T07:44:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c7d64e740 10 client.0 _ll_register_callbacks cb 0x7f2c6c007750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb36b2c0 0x5638cb390580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb36b2c0 0x5638cb390580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51876/0 (socket says 172.21.15.16:51876) 2024-09-19T07:44:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c737fe640 1 -- 172.21.15.16:0/3646707778 learned_addr learned my addr 172.21.15.16:0/3646707778 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c73fff640 1 --2- 172.21.15.16:0/3646707778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb3647d0 0x5638cb364f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c78d73640 1 --2- 172.21.15.16:0/3646707778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36ca50 0x5638cb390ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c737fe640 1 -- 172.21.15.16:0/3646707778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36ca50 msgr2=0x5638cb390ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c737fe640 1 --2- 172.21.15.16:0/3646707778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36ca50 0x5638cb390ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c737fe640 1 -- 172.21.15.16:0/3646707778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb3647d0 msgr2=0x5638cb364f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c737fe640 1 --2- 172.21.15.16:0/3646707778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb3647d0 0x5638cb364f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c737fe640 1 -- 172.21.15.16:0/3646707778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638cb3af060 con 0x5638cb36b2c0 2024-09-19T07:44:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c737fe640 1 --2- 172.21.15.16:0/3646707778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb36b2c0 0x5638cb390580 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2c6802a770 tx=0x7f2c6802a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e2781ac4ede25f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:44:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c52ffd640 1 -- 172.21.15.16:0/3646707778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c680a8040 con 0x5638cb36b2c0 2024-09-19T07:44:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c52ffd640 1 -- 172.21.15.16:0/3646707778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f2c68030d80 con 0x5638cb36b2c0 2024-09-19T07:44:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c7d64e740 1 -- 172.21.15.16:0/3646707778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c6807e1d0 con 0x5638cb36b2c0 2024-09-19T07:44:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c52ffd640 1 -- 172.21.15.16:0/3646707778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f2c68089050 con 0x5638cb36b2c0 2024-09-19T07:44:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c7d64e740 10 client.5139 fetch_fsmap learned FSMap version 16 2024-09-19T07:44:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c7d64e740 1 -- 172.21.15.16:0/3646707778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x5638cb3af060 con 0x5638cb36b2c0 2024-09-19T07:44:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.138+0000 7f2c52ffd640 1 -- 172.21.15.16:0/3646707778 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c6807e1d0 con 0x5638cb36b2c0 2024-09-19T07:44:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.139+0000 7f2c52ffd640 1 -- 172.21.15.16:0/3646707778 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 16) ==== 3502+0+0 (secure 0 0 0) 0x7f2c6808c020 con 0x5638cb36b2c0 2024-09-19T07:44:11.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.139+0000 7f2c7d64e740 10 client.5139 fetch_fsmap finished waiting for FSMap version 16 2024-09-19T07:44:11.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.139+0000 7f2c7d64e740 10 client.5139 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T07:44:11.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.139+0000 7f2c7d64e740 20 client.5139 populate_metadata read hostname 'smithi016' 2024-09-19T07:44:11.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.139+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/3646707778 >> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] conn(0x5638cb3c8d10 0x5638cb3ac180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.139+0000 7f2c7d64e740 4 client.5139 mds_command: new command op to 4947 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:44:11.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.139+0000 7f2c7d64e740 1 -- 172.21.15.16:0/3646707778 --> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] -- command(tid 0: {"prefix": "status"}) -- 0x5638cb2d4fa0 con 0x5638cb3c8d10 2024-09-19T07:44:11.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.139+0000 7f2c73fff640 1 --2- 172.21.15.16:0/3646707778 >> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] conn(0x5638cb3c8d10 0x5638cb3ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.139+0000 7f2c52ffd640 10 client.5139 ms_handle_connect on v2:172.21.15.16:6833/2028210099 2024-09-19T07:44:11.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.139+0000 7f2c73fff640 1 --2- 172.21.15.16:0/3646707778 >> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] conn(0x5638cb3c8d10 0x5638cb3ac180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feef38b99b316b67 server_cookie=57a7bd842c79bb5 in_seq=0 out_seq=0 2024-09-19T07:44:11.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.139+0000 7f2c52ffd640 1 -- 172.21.15.16:0/3646707778 <== mds.0 v2:172.21.15.16:6833/2028210099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5638cb2d4fa0 con 0x5638cb3c8d10 2024-09-19T07:44:11.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.139+0000 7f2c52ffd640 10 client.5139 handle_command_reply: tid=0 2024-09-19T07:44:11.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.139+0000 7f2c7d64e740 1 client.5139 shutdown 2024-09-19T07:44:11.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.139+0000 7f2c7d64e740 1 -- 172.21.15.16:0/3646707778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb36b2c0 msgr2=0x5638cb390580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:11.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.139+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/3646707778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb36b2c0 0x5638cb390580 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2c6802a770 tx=0x7f2c6802a700 comp rx=0 tx=0).stop 2024-09-19T07:44:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7d64e740 1 -- 172.21.15.16:0/3646707778 shutdown_connections 2024-09-19T07:44:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/3646707778 >> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] conn(0x5638cb3c8d10 0x5638cb3ac180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/3646707778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5638cb36ca50 0x5638cb390ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/3646707778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638cb36b2c0 0x5638cb390580 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c52ffd640 0 client.5139 ms_handle_reset on v2:172.21.15.16:6833/2028210099 2024-09-19T07:44:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7d64e740 1 --2- 172.21.15.16:0/3646707778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5638cb3647d0 0x5638cb364f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7d64e740 1 -- 172.21.15.16:0/3646707778 >> 172.21.15.16:0/3646707778 conn(0x5638cb368a80 msgr2=0x5638cb38ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7d64e740 1 -- 172.21.15.16:0/3646707778 shutdown_connections 2024-09-19T07:44:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7d64e740 1 -- 172.21.15.16:0/3646707778 wait complete. 2024-09-19T07:44:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7d64e740 20 client.5139 trim_cache size 0 max 16384 2024-09-19T07:44:11.146 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:44:11.147 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb", 2024-09-19T07:44:11.147 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-19T07:44:11.147 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4947, 2024-09-19T07:44:11.147 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T07:44:11.147 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T07:44:11.147 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T07:44:11.147 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 9.5299185340000001, 2024-09-19T07:44:11.147 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 16, 2024-09-19T07:44:11.147 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 40, 2024-09-19T07:44:11.147 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 40, 2024-09-19T07:44:11.147 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 16.391102621000002 2024-09-19T07:44:11.147 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:44:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7a7d6640 1 -- 172.21.15.16:0/2718565400 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2c6006c2b0 msgr2=0x7f2c6006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7a7d6640 1 --2- 172.21.15.16:0/2718565400 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2c6006c2b0 0x7f2c6006e770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2c64004f60 tx=0x7f2c64007e30 comp rx=0 tx=0).stop 2024-09-19T07:44:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7a7d6640 1 -- 172.21.15.16:0/2718565400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7415e2a0 msgr2=0x7f2c741e8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7a7d6640 1 --2- 172.21.15.16:0/2718565400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7415e2a0 0x7f2c741e8e00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2c680090b0 tx=0x7f2c680046c0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7a7d6640 1 -- 172.21.15.16:0/2718565400 shutdown_connections 2024-09-19T07:44:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7a7d6640 1 --2- 172.21.15.16:0/2718565400 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2c6006c2b0 0x7f2c6006e770 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7a7d6640 1 --2- 172.21.15.16:0/2718565400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c7415e680 0x7f2c7415d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7a7d6640 1 --2- 172.21.15.16:0/2718565400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7415e2a0 0x7f2c741e8e00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7a7d6640 1 --2- 172.21.15.16:0/2718565400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c740d0010 0x7f2c740cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7a7d6640 1 -- 172.21.15.16:0/2718565400 >> 172.21.15.16:0/2718565400 conn(0x7f2c7409d760 msgr2=0x7f2c7415ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7a7d6640 1 -- 172.21.15.16:0/2718565400 shutdown_connections 2024-09-19T07:44:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.140+0000 7f2c7a7d6640 1 -- 172.21.15.16:0/2718565400 wait complete. 2024-09-19T07:44:11.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:44:11.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.253+0000 7f162be5e640 1 Processor -- start 2024-09-19T07:44:11.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.253+0000 7f162be5e640 1 -- start start 2024-09-19T07:44:11.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.254+0000 7f162be5e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16240d0070 0x7f162415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.254+0000 7f162be5e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16240d4130 0x7f16240d45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.254+0000 7f162be5e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16240d4b00 con 0x7f162415f0d0 2024-09-19T07:44:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.254+0000 7f162be5e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f16240d1b10 con 0x7f16240d4130 2024-09-19T07:44:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.254+0000 7f162be5e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f16240cf2b0 con 0x7f16240d0070 2024-09-19T07:44:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.254+0000 7f1629bfa640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f162415f0d0 0x7f16240d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44528/0 (socket says 172.21.15.16:44528) 2024-09-19T07:44:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.254+0000 7f1629bfa640 1 -- 172.21.15.16:0/1815044640 learned_addr learned my addr 172.21.15.16:0/1815044640 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:44:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.254+0000 7f162a3fb640 1 --2- 172.21.15.16:0/1815044640 >> v2:172.21.15.16:3300/0 conn(0x7f16240d4130 0x7f16240d45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.255+0000 7f1628bf8640 1 -- 172.21.15.16:0/1815044640 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 896524079 0 0) 0x7f16240d4b00 con 0x7f162415f0d0 2024-09-19T07:44:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.255+0000 7f1628bf8640 1 -- 172.21.15.16:0/1815044640 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f160c003520 con 0x7f162415f0d0 2024-09-19T07:44:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.255+0000 7f16293f9640 1 --2- 172.21.15.16:0/1815044640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16240d0070 0x7f162415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.255+0000 7f162a3fb640 1 -- 172.21.15.16:0/1815044640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16240d0070 msgr2=0x7f162415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.255+0000 7f162a3fb640 1 --2- 172.21.15.16:0/1815044640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16240d0070 0x7f162415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.255+0000 7f162a3fb640 1 -- 172.21.15.16:0/1815044640 >> v1:172.21.15.16:6789/0 conn(0x7f162415f0d0 legacy=0x7f16240d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.255+0000 7f162a3fb640 1 -- 172.21.15.16:0/1815044640 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16240cf590 con 0x7f16240d4130 2024-09-19T07:44:11.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.255+0000 7f16293f9640 1 --2- 172.21.15.16:0/1815044640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16240d0070 0x7f162415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:44:11.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.256+0000 7f162a3fb640 1 --2- 172.21.15.16:0/1815044640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16240d4130 0x7f16240d45c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f16240d4cd0 tx=0x7f161c02a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b82eccff3084897 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:11.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.256+0000 7f1628bf8640 1 -- 172.21.15.16:0/1815044640 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c007740 con 0x7f16240d4130 2024-09-19T07:44:11.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.256+0000 7f1628bf8640 1 -- 172.21.15.16:0/1815044640 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f161c0078e0 con 0x7f16240d4130 2024-09-19T07:44:11.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.256+0000 7f1628bf8640 1 -- 172.21.15.16:0/1815044640 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c007cb0 con 0x7f16240d4130 2024-09-19T07:44:11.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.256+0000 7f162be5e640 1 -- 172.21.15.16:0/1815044640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16240d4130 msgr2=0x7f16240d45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:11.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.256+0000 7f162be5e640 1 --2- 172.21.15.16:0/1815044640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16240d4130 0x7f16240d45c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f16240d4cd0 tx=0x7f161c02a5b0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.256+0000 7f162be5e640 1 -- 172.21.15.16:0/1815044640 shutdown_connections 2024-09-19T07:44:11.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.256+0000 7f162be5e640 1 --2- 172.21.15.16:0/1815044640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16240d4130 0x7f16240d45c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.256+0000 7f162be5e640 1 --2- 172.21.15.16:0/1815044640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16240d0070 0x7f162415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.256+0000 7f162be5e640 1 -- 172.21.15.16:0/1815044640 >> 172.21.15.16:0/1815044640 conn(0x7f162409d760 msgr2=0x7f162408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:11.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.256+0000 7f162be5e640 1 -- 172.21.15.16:0/1815044640 shutdown_connections 2024-09-19T07:44:11.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.256+0000 7f162be5e640 1 -- 172.21.15.16:0/1815044640 wait complete. 2024-09-19T07:44:11.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.258+0000 7f162be5e640 1 Processor -- start 2024-09-19T07:44:11.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.258+0000 7f162be5e640 1 -- start start 2024-09-19T07:44:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.258+0000 7f162be5e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16240d0070 0x7f1624155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.258+0000 7f162be5e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16240d4130 0x7f16240b98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.258+0000 7f162be5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f162415f0d0 0x7f16240b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.258+0000 7f162be5e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16240cef60 con 0x7f162415f0d0 2024-09-19T07:44:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.258+0000 7f162be5e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f16240cfb50 con 0x7f16240d4130 2024-09-19T07:44:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.258+0000 7f162be5e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f16240b2f70 con 0x7f16240d0070 2024-09-19T07:44:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.258+0000 7f162a3fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f162415f0d0 0x7f16240b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.258+0000 7f162a3fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f162415f0d0 0x7f16240b77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51904/0 (socket says 172.21.15.16:51904) 2024-09-19T07:44:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.258+0000 7f162a3fb640 1 -- 172.21.15.16:0/1586458118 learned_addr learned my addr 172.21.15.16:0/1586458118 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:11.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.259+0000 7f16293f9640 1 --2- 172.21.15.16:0/1586458118 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16240d4130 0x7f16240b98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.259+0000 7f1629bfa640 1 --2- 172.21.15.16:0/1586458118 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16240d0070 0x7f1624155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.259+0000 7f162a3fb640 1 -- 172.21.15.16:0/1586458118 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16240d0070 msgr2=0x7f1624155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.259+0000 7f162a3fb640 1 --2- 172.21.15.16:0/1586458118 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16240d0070 0x7f1624155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.259+0000 7f162a3fb640 1 -- 172.21.15.16:0/1586458118 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16240d4130 msgr2=0x7f16240b98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.259+0000 7f162a3fb640 1 --2- 172.21.15.16:0/1586458118 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16240d4130 0x7f16240b98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.259+0000 7f162a3fb640 1 -- 172.21.15.16:0/1586458118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1624160170 con 0x7f162415f0d0 2024-09-19T07:44:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.259+0000 7f16293f9640 1 --2- 172.21.15.16:0/1586458118 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16240d4130 0x7f16240b98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.259+0000 7f1629bfa640 1 --2- 172.21.15.16:0/1586458118 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16240d0070 0x7f1624155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.259+0000 7f162a3fb640 1 --2- 172.21.15.16:0/1586458118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f162415f0d0 0x7f16240b77d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f161c02aba0 tx=0x7f161c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.259+0000 7f16227fc640 1 -- 172.21.15.16:0/1586458118 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c040070 con 0x7f162415f0d0 2024-09-19T07:44:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.259+0000 7f16227fc640 1 -- 172.21.15.16:0/1586458118 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f161c031bc0 con 0x7f162415f0d0 2024-09-19T07:44:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.259+0000 7f162be5e640 1 -- 172.21.15.16:0/1586458118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1624160450 con 0x7f162415f0d0 2024-09-19T07:44:11.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.259+0000 7f16227fc640 1 -- 172.21.15.16:0/1586458118 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c009450 con 0x7f162415f0d0 2024-09-19T07:44:11.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.259+0000 7f162be5e640 1 -- 172.21.15.16:0/1586458118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1624160a10 con 0x7f162415f0d0 2024-09-19T07:44:11.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.260+0000 7f162be5e640 1 -- 172.21.15.16:0/1586458118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f162415aae0 con 0x7f162415f0d0 2024-09-19T07:44:11.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.261+0000 7f16227fc640 1 -- 172.21.15.16:0/1586458118 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f161c039040 con 0x7f162415f0d0 2024-09-19T07:44:11.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.261+0000 7f16227fc640 1 --2- 172.21.15.16:0/1586458118 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f161006c2b0 0x7f161006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:11.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.261+0000 7f16227fc640 1 -- 172.21.15.16:0/1586458118 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f161c0ace80 con 0x7f162415f0d0 2024-09-19T07:44:11.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.261+0000 7f1629bfa640 1 --2- 172.21.15.16:0/1586458118 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f161006c2b0 0x7f161006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:11.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.262+0000 7f1629bfa640 1 --2- 172.21.15.16:0/1586458118 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f161006c2b0 0x7f161006e770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1614007fd0 tx=0x7f1614009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.265+0000 7f16227fc640 1 -- 172.21.15.16:0/1586458118 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f162415aae0 con 0x7f162415f0d0 2024-09-19T07:44:11.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.477+0000 7f162be5e640 1 -- 172.21.15.16:0/1586458118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f162415f4b0 con 0x7f162415f0d0 2024-09-19T07:44:11.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.478+0000 7f16227fc640 1 -- 172.21.15.16:0/1586458118 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f161c07ae10 con 0x7f162415f0d0 2024-09-19T07:44:11.478 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:44:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.480+0000 7f162be5e640 1 -- 172.21.15.16:0/1586458118 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f161006c2b0 msgr2=0x7f161006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.480+0000 7f162be5e640 1 --2- 172.21.15.16:0/1586458118 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f161006c2b0 0x7f161006e770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1614007fd0 tx=0x7f1614009040 comp rx=0 tx=0).stop 2024-09-19T07:44:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.480+0000 7f162be5e640 1 -- 172.21.15.16:0/1586458118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f162415f0d0 msgr2=0x7f16240b77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.480+0000 7f162be5e640 1 --2- 172.21.15.16:0/1586458118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f162415f0d0 0x7f16240b77d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f161c02aba0 tx=0x7f161c004a40 comp rx=0 tx=0).stop 2024-09-19T07:44:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.480+0000 7f162be5e640 1 -- 172.21.15.16:0/1586458118 shutdown_connections 2024-09-19T07:44:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.480+0000 7f162be5e640 1 --2- 172.21.15.16:0/1586458118 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f161006c2b0 0x7f161006e770 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.480+0000 7f162be5e640 1 --2- 172.21.15.16:0/1586458118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f162415f0d0 0x7f16240b77d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.480+0000 7f162be5e640 1 --2- 172.21.15.16:0/1586458118 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16240d4130 0x7f16240b98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.480+0000 7f162be5e640 1 --2- 172.21.15.16:0/1586458118 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16240d0070 0x7f1624155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.480+0000 7f162be5e640 1 -- 172.21.15.16:0/1586458118 >> 172.21.15.16:0/1586458118 conn(0x7f162409d760 msgr2=0x7f16240d2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.480+0000 7f162be5e640 1 -- 172.21.15.16:0/1586458118 shutdown_connections 2024-09-19T07:44:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11.480+0000 7f162be5e640 1 -- 172.21.15.16:0/1586458118 wait complete. 2024-09-19T07:44:11.491 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:44:11.491 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:44:11.491 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T07:44:11.491 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T07:44:11.491 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:44:11.491 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:44:11.491 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:44:11.491 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:44:11.492 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:44:11.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:11.492 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T07:44:11.508 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:44:11.508 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:44:11.508 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:44:11.508 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:44:11.508 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:44:11.508 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:44:11.508 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:44:11.509 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:44:11.509 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T07:44:11.509 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:44:11.509 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:44:11.509 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T07:44:11.509 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:44:11.509 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:44:11.509 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T07:44:11.509 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T07:44:11.509 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42737sec preferred_lft 42737sec 2024-09-19T07:44:11.509 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T07:44:11.509 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:44:11.509 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:44:11.510 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:44:11.510 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T07:44:11.510 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:44:11.510 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:44:11.510 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:44:11.510 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:44:11.510 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::32:7bff:fe92:8895/64 scope link 2024-09-19T07:44:11.510 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:44:11.510 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:44:11.510 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether e6:fd:21:6a:43:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:44:11.510 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e4fd:21ff:fe6a:4369/64 scope link 2024-09-19T07:44:11.510 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:44:11.510 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:44:11.510 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:44:11.511 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::80f4:81ff:fe7d:b7f0/64 scope link 2024-09-19T07:44:11.511 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:44:11.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:11.511 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:44:11.572 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:44:11.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:11.574 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T07:44:11.631 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T07:44:11.631 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:44:11.632 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T07:44:11.632 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T07:44:11.632 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:44:11.632 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:44:11.632 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:44:11.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T07:44:11.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T07:44:11.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T07:44:11.632 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:44:11.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:44:11.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:44:11.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:11.762 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:44:11.778 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T07:44:11.778 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:44:11.778 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:44:11.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:11.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-19T07:44:11.844 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T07:44:11.844 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:44:11.844 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T07:44:11.900 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:44:11.900 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:44:11.900 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:44:11.956 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:44:12.022 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:44:12.080 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:44:12.080 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:44:12.080 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:44:12.080 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:44:12.080 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:44:12.080 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:44:12.081 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:44:12.081 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:44:12.081 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:44:12.081 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:44:12.081 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:44:12.081 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:44:12.081 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:44:12.081 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:44:12.081 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:44:12.081 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:44:12.081 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:44:12.081 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:44:12.081 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:44:12.082 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:44:12.082 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:44:12.082 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:44:12.082 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:44:12.082 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:44:12.082 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:44:12.082 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:44:12.082 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:44:12.082 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:44:12.082 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:44:12.082 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:44:12.082 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:44:12.082 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:44:12.083 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:44:12.083 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:44:12.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:12.083 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:44:12.135 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:44:12.136 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T07:44:12.178 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:44:12.204 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:44:12.261 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:44:12.261 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:44:12.261 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:44:12.261 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:44:12.261 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:44:12.261 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:44:12.261 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:44:12.261 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:44:12.262 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:44:12.262 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:44:12.262 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:44:12.262 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:44:12.262 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:44:12.262 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:44:12.262 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:44:12.262 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:44:12.262 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:44:12.262 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:44:12.262 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:44:12.262 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:44:12.262 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:44:12.263 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:44:12.263 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:44:12.263 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:44:12.263 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:44:12.263 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:44:12.263 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:44:12.263 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:44:12.263 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:44:12.263 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:44:12.263 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:44:12.263 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:44:12.263 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:44:12.263 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:44:12.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:12.264 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:44:12.265 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-19T07:44:12.262+0000 7efdd4ce9480 -1 init, newargv = 0x55d4d2432910 newargc=15 2024-09-19T07:44:12.265 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[48356]: starting ceph client 2024-09-19T07:44:12.267 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[48356]: starting fuse 2024-09-19T07:44:12.280 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:44:12.281 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-19T07:44:12.281 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:44:12.281 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:44:12.281 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:44:12.281 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:44:12.281 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:44:12.281 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:12.281 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:44:12.281 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:44:12.281 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:44:12.281 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:44:12.282 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:12.282 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:44:12.282 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:44:12.282 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:44:12.282 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:44:12.282 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:12.282 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:44:12.282 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:44:12.282 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:44:12.282 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:44:12.282 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:44:12.282 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:44:12.282 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:44:12.282 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:44:12.282 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:12.283 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:44:12.283 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:44:12.401 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.48356.asok 2024-09-19T07:44:12.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:44:12.412 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48356.asok 2024-09-19T07:44:12.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:12.412 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48356.asok status 2024-09-19T07:44:12.549 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:44:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:44:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:44:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:44:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:44:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:44:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:44:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "48356", 2024-09-19T07:44:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:44:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-19T07:44:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-19T07:44:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5151, 2024-09-19T07:44:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:44:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:44:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:44:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:44:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:44:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2625487776 2024-09-19T07:44:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:44:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:44:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:44:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2625487776 2024-09-19T07:44:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.5151 192.168.0.1:0/2625487776", 2024-09-19T07:44:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/2625487776", 2024-09-19T07:44:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-19T07:44:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 16, 2024-09-19T07:44:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 40, 2024-09-19T07:44:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:44:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:44:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:44:12.552 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:44:12.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:12.558 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-19T07:44:12.558 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T07:44:12.574 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-19T07:44:12.575 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T07:44:12.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:12.575 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:44:12.642 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:44:12.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.783+0000 7f01511f6640 1 Processor -- start 2024-09-19T07:44:12.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.783+0000 7f01511f6640 1 -- start start 2024-09-19T07:44:12.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.784+0000 7f01511f6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f014c0c8d20 0x7f014c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:12.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.784+0000 7f01511f6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f014c144100 0x7f014c0d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:12.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.784+0000 7f01511f6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f014c0c7140 con 0x7f014c06d260 2024-09-19T07:44:12.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.784+0000 7f01511f6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f014c06d6d0 con 0x7f014c0c8d20 2024-09-19T07:44:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.784+0000 7f01511f6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f014c141960 con 0x7f014c144100 2024-09-19T07:44:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.784+0000 7f014ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f014c0c8d20 0x7f014c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.784+0000 7f014ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f014c0c8d20 0x7f014c06cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41432/0 (socket says 172.21.15.16:41432) 2024-09-19T07:44:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.784+0000 7f014ad9d640 1 -- 172.21.15.16:0/3613215447 learned_addr learned my addr 172.21.15.16:0/3613215447 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.784+0000 7f014a59c640 1 --1- 172.21.15.16:0/3613215447 >> v1:172.21.15.16:6789/0 conn(0x7f014c06d260 0x7f014c1439f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54320/0 (socket says 172.21.15.16:54320) 2024-09-19T07:44:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.784+0000 7f014b59e640 1 --2- 172.21.15.16:0/3613215447 >> v2:172.21.15.146:3300/0 conn(0x7f014c144100 0x7f014c0d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.784+0000 7f014ad9d640 1 -- 172.21.15.16:0/3613215447 >> v2:172.21.15.146:3300/0 conn(0x7f014c144100 msgr2=0x7f014c0d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.784+0000 7f014ad9d640 1 --2- 172.21.15.16:0/3613215447 >> v2:172.21.15.146:3300/0 conn(0x7f014c144100 0x7f014c0d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.784+0000 7f014ad9d640 1 -- 172.21.15.16:0/3613215447 >> v1:172.21.15.16:6789/0 conn(0x7f014c06d260 legacy=0x7f014c1439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.784+0000 7f0149d9b640 1 -- 172.21.15.16:0/3613215447 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2699865904 0 0) 0x7f0134001110 con 0x7f014c06d260 2024-09-19T07:44:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.784+0000 7f014ad9d640 1 -- 172.21.15.16:0/3613215447 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014c0c9100 con 0x7f014c0c8d20 2024-09-19T07:44:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.784+0000 7f014b59e640 1 --2- 172.21.15.16:0/3613215447 >> v2:172.21.15.146:3300/0 conn(0x7f014c144100 0x7f014c0d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:44:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.785+0000 7f014ad9d640 1 --2- 172.21.15.16:0/3613215447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c0c8d20 0x7f014c06cd20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f014c06ef00 tx=0x7f013c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3838ea38604f5acd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:12.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.785+0000 7f0149d9b640 1 -- 172.21.15.16:0/3613215447 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c036070 con 0x7f014c0c8d20 2024-09-19T07:44:12.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.785+0000 7f0149d9b640 1 -- 172.21.15.16:0/3613215447 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f013c031600 con 0x7f014c0c8d20 2024-09-19T07:44:12.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.785+0000 7f0149d9b640 1 -- 172.21.15.16:0/3613215447 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c0319d0 con 0x7f014c0c8d20 2024-09-19T07:44:12.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.786+0000 7f01511f6640 1 -- 172.21.15.16:0/3613215447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c0c8d20 msgr2=0x7f014c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:12.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.786+0000 7f01511f6640 1 --2- 172.21.15.16:0/3613215447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c0c8d20 0x7f014c06cd20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f014c06ef00 tx=0x7f013c02a610 comp rx=0 tx=0).stop 2024-09-19T07:44:12.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.786+0000 7f01511f6640 1 -- 172.21.15.16:0/3613215447 shutdown_connections 2024-09-19T07:44:12.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.786+0000 7f01511f6640 1 --2- 172.21.15.16:0/3613215447 >> v2:172.21.15.146:3300/0 conn(0x7f014c144100 0x7f014c0d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:12.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.786+0000 7f01511f6640 1 --2- 172.21.15.16:0/3613215447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c0c8d20 0x7f014c06cd20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:12.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.786+0000 7f01511f6640 1 -- 172.21.15.16:0/3613215447 >> 172.21.15.16:0/3613215447 conn(0x7f014c0b3ed0 msgr2=0x7f014c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:12.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.786+0000 7f01511f6640 1 -- 172.21.15.16:0/3613215447 shutdown_connections 2024-09-19T07:44:12.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.786+0000 7f01511f6640 1 -- 172.21.15.16:0/3613215447 wait complete. 2024-09-19T07:44:12.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.787+0000 7f01511f6640 1 Processor -- start 2024-09-19T07:44:12.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.788+0000 7f01511f6640 1 -- start start 2024-09-19T07:44:12.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.788+0000 7f01511f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c06d260 0x7f014c13f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:12.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.788+0000 7f01511f6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f014c0c8d20 0x7f014c15dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:12.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.788+0000 7f01511f6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f014c144100 0x7f014c0b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:12.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.788+0000 7f01511f6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f014c144510 con 0x7f014c06d260 2024-09-19T07:44:12.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.788+0000 7f01511f6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f014c142200 con 0x7f014c0c8d20 2024-09-19T07:44:12.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.788+0000 7f01511f6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f014c141bb0 con 0x7f014c144100 2024-09-19T07:44:12.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.788+0000 7f014ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c06d260 0x7f014c13f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:12.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.788+0000 7f014ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c06d260 0x7f014c13f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41444/0 (socket says 172.21.15.16:41444) 2024-09-19T07:44:12.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.788+0000 7f014ad9d640 1 -- 172.21.15.16:0/932980246 learned_addr learned my addr 172.21.15.16:0/932980246 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:12.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.788+0000 7f014a59c640 1 --2- 172.21.15.16:0/932980246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f014c0c8d20 0x7f014c15dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:12.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.789+0000 7f014b59e640 1 --2- 172.21.15.16:0/932980246 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f014c144100 0x7f014c0b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:12.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.789+0000 7f014ad9d640 1 -- 172.21.15.16:0/932980246 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f014c144100 msgr2=0x7f014c0b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:12.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.789+0000 7f014ad9d640 1 --2- 172.21.15.16:0/932980246 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f014c144100 0x7f014c0b75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:12.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.789+0000 7f014ad9d640 1 -- 172.21.15.16:0/932980246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f014c0c8d20 msgr2=0x7f014c15dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:12.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.789+0000 7f014ad9d640 1 --2- 172.21.15.16:0/932980246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f014c0c8d20 0x7f014c15dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:12.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.789+0000 7f014ad9d640 1 -- 172.21.15.16:0/932980246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014c0b1150 con 0x7f014c06d260 2024-09-19T07:44:12.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.789+0000 7f014ad9d640 1 --2- 172.21.15.16:0/932980246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c06d260 0x7f014c13f930 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f013c000f80 tx=0x7f013c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:12.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.789+0000 7f01437fe640 1 -- 172.21.15.16:0/932980246 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c036050 con 0x7f014c06d260 2024-09-19T07:44:12.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.789+0000 7f01437fe640 1 -- 172.21.15.16:0/932980246 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f013c042070 con 0x7f014c06d260 2024-09-19T07:44:12.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.789+0000 7f01511f6640 1 -- 172.21.15.16:0/932980246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014c0b1430 con 0x7f014c06d260 2024-09-19T07:44:12.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.789+0000 7f01437fe640 1 -- 172.21.15.16:0/932980246 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c009510 con 0x7f014c06d260 2024-09-19T07:44:12.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.789+0000 7f01511f6640 1 -- 172.21.15.16:0/932980246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f014c15e000 con 0x7f014c06d260 2024-09-19T07:44:12.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.790+0000 7f01511f6640 1 -- 172.21.15.16:0/932980246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f014c06cd20 con 0x7f014c06d260 2024-09-19T07:44:12.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.791+0000 7f01437fe640 1 -- 172.21.15.16:0/932980246 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f013c007c70 con 0x7f014c06d260 2024-09-19T07:44:12.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.791+0000 7f01437fe640 1 --2- 172.21.15.16:0/932980246 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f012c06c2b0 0x7f012c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:12.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.791+0000 7f01437fe640 1 -- 172.21.15.16:0/932980246 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f013c0ac580 con 0x7f014c06d260 2024-09-19T07:44:12.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.791+0000 7f014a59c640 1 --2- 172.21.15.16:0/932980246 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f012c06c2b0 0x7f012c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:12.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.792+0000 7f014a59c640 1 --2- 172.21.15.16:0/932980246 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f012c06c2b0 0x7f012c06e770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f014c0b2be0 tx=0x7f0134007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:12.796+0000 7f01437fe640 1 -- 172.21.15.16:0/932980246 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f013c07ab50 con 0x7f014c06d260 2024-09-19T07:44:13.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:13.005+0000 7f01511f6640 1 -- 172.21.15.16:0/932980246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f014c0690c0 con 0x7f014c06d260 2024-09-19T07:44:13.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:13.005+0000 7f01437fe640 1 -- 172.21.15.16:0/932980246 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f013c07a450 con 0x7f014c06d260 2024-09-19T07:44:13.006 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:44:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:13.007+0000 7f01511f6640 1 -- 172.21.15.16:0/932980246 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f012c06c2b0 msgr2=0x7f012c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:13.007+0000 7f01511f6640 1 --2- 172.21.15.16:0/932980246 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f012c06c2b0 0x7f012c06e770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f014c0b2be0 tx=0x7f0134007c00 comp rx=0 tx=0).stop 2024-09-19T07:44:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:13.007+0000 7f01511f6640 1 -- 172.21.15.16:0/932980246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c06d260 msgr2=0x7f014c13f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:13.007+0000 7f01511f6640 1 --2- 172.21.15.16:0/932980246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c06d260 0x7f014c13f930 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f013c000f80 tx=0x7f013c004a40 comp rx=0 tx=0).stop 2024-09-19T07:44:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:13.007+0000 7f01511f6640 1 -- 172.21.15.16:0/932980246 shutdown_connections 2024-09-19T07:44:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:13.007+0000 7f01511f6640 1 --2- 172.21.15.16:0/932980246 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f012c06c2b0 0x7f012c06e770 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:13.007+0000 7f01511f6640 1 --2- 172.21.15.16:0/932980246 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f014c144100 0x7f014c0b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:13.007+0000 7f01511f6640 1 --2- 172.21.15.16:0/932980246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f014c0c8d20 0x7f014c15dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:13.007+0000 7f01511f6640 1 --2- 172.21.15.16:0/932980246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c06d260 0x7f014c13f930 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:13.007+0000 7f01511f6640 1 -- 172.21.15.16:0/932980246 >> 172.21.15.16:0/932980246 conn(0x7f014c0b3ed0 msgr2=0x7f014c0b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:13.007+0000 7f01511f6640 1 -- 172.21.15.16:0/932980246 shutdown_connections 2024-09-19T07:44:13.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:13.007+0000 7f01511f6640 1 -- 172.21.15.16:0/932980246 wait complete. 2024-09-19T07:44:13.018 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:44:13.018 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:44:13.018 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-09-19T07:44:13.018 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T07:44:13.019 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T07:44:13.019 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:44:13.019 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:44:13.019 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:44:13.019 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:44:13.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:13.019 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-09-19T07:44:13.039 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:44:13.039 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:44:13.039 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:44:13.039 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:44:13.039 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-09-19T07:44:13.039 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:44:13.039 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:44:13.040 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-09-19T07:44:13.040 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2024-09-19T07:44:13.040 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:44:13.040 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-09-19T07:44:13.040 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2024-09-19T07:44:13.040 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:44:13.040 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-09-19T07:44:13.040 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:44:13.040 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-09-19T07:44:13.040 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T07:44:13.040 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42703sec preferred_lft 42703sec 2024-09-19T07:44:13.040 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link noprefixroute 2024-09-19T07:44:13.040 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:44:13.040 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:44:13.041 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:44:13.041 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:44:13.041 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:44:13.041 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::e47f:32ff:fe68:31ca/64 scope link 2024-09-19T07:44:13.041 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:44:13.041 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:44:13.041 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether f6:1f:22:44:9d:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:44:13.041 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::f41f:22ff:fe44:9dcd/64 scope link 2024-09-19T07:44:13.041 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:44:13.041 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:44:13.041 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:44:13.041 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::8459:eeff:fe21:a113/64 scope link 2024-09-19T07:44:13.041 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:44:13.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:13.042 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:44:13.107 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-09-19T07:44:13.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:13.109 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-09-19T07:44:13.168 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-09-19T07:44:13.168 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:44:13.168 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T07:44:13.168 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T07:44:13.168 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:44:13.169 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:44:13.169 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-19T07:44:13.169 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T07:44:13.169 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T07:44:13.169 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T07:44:13.169 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-19T07:44:13.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:44:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:44:13.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:44:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:44:13.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:13.303 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-09-19T07:44:13.319 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T07:44:13.319 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T07:44:13.319 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:44:13.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:13.319 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2024-09-19T07:44:13.384 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T07:44:13.385 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:44:13.385 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T07:44:13.440 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:44:13.441 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:44:13.441 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:44:13.496 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:44:13.561 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:44:13.617 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:44:13.617 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:44:13.617 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:44:13.618 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:44:13.618 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:44:13.618 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:44:13.618 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:44:13.618 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:44:13.618 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:44:13.618 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:44:13.618 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:44:13.618 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:44:13.618 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:44:13.618 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:44:13.618 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:44:13.618 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:44:13.619 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:44:13.619 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:44:13.619 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:44:13.619 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:44:13.619 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:44:13.619 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:44:13.619 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:44:13.619 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:44:13.619 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:44:13.619 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:44:13.619 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:44:13.619 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:44:13.619 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:44:13.619 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:44:13.620 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:44:13.620 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:44:13.620 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:44:13.620 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:44:13.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:13.620 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:44:13.673 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:44:13.673 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T07:44:13.715 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:44:13.740 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:44:13.797 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:44:13.797 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:44:13.797 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:44:13.797 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:44:13.797 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:44:13.797 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:44:13.797 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:44:13.797 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:44:13.797 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:44:13.797 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:44:13.798 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:44:13.798 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:44:13.798 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:44:13.798 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:44:13.798 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:44:13.798 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:44:13.798 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:44:13.798 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:44:13.798 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:44:13.798 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:44:13.798 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:44:13.798 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:44:13.798 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:44:13.798 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:44:13.798 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:44:13.799 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:44:13.799 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:44:13.799 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:44:13.799 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:44:13.799 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:44:13.799 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:44:13.799 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:44:13.799 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:44:13.799 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:44:13.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:13.799 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:44:13.800 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2024-09-19T07:44:13.795+0000 7f5ef5dc4480 -1 init, newargv = 0x560d4cde5940 newargc=15 2024-09-19T07:44:13.800 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[44154]: starting ceph client 2024-09-19T07:44:13.801 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[44154]: starting fuse 2024-09-19T07:44:13.816 INFO:teuthology.orchestra.run.smithi146.stdout:42 2024-09-19T07:44:13.816 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-19T07:44:13.816 DEBUG:teuthology.orchestra.run.smithi146:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:44:13.817 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2024-09-19T07:44:13.817 DEBUG:teuthology.orchestra.run.smithi146:> import re 2024-09-19T07:44:13.817 DEBUG:teuthology.orchestra.run.smithi146:> import os 2024-09-19T07:44:13.817 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2024-09-19T07:44:13.817 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:44:13.817 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2024-09-19T07:44:13.817 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T07:44:13.817 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2024-09-19T07:44:13.817 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T07:44:13.817 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:44:13.817 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2024-09-19T07:44:13.817 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2024-09-19T07:44:13.817 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2024-09-19T07:44:13.817 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2024-09-19T07:44:13.818 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:44:13.818 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2024-09-19T07:44:13.818 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:44:13.818 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:44:13.818 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:44:13.818 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2024-09-19T07:44:13.818 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2024-09-19T07:44:13.818 DEBUG:teuthology.orchestra.run.smithi146:> return f 2024-09-19T07:44:13.818 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:44:13.818 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:44:13.818 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2024-09-19T07:44:13.818 DEBUG:teuthology.orchestra.run.smithi146:> ' 2024-09-19T07:44:13.933 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.44154.asok 2024-09-19T07:44:13.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:44:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:44:13.944 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.44154.asok 2024-09-19T07:44:13.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:13.944 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.44154.asok status 2024-09-19T07:44:14.077 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-19T07:44:14.077 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2024-09-19T07:44:14.077 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:44:14.077 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:44:14.077 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2024-09-19T07:44:14.077 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2024-09-19T07:44:14.078 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T07:44:14.078 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "44154", 2024-09-19T07:44:14.078 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2024-09-19T07:44:14.078 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:44:14.078 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2024-09-19T07:44:14.078 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2024-09-19T07:44:14.078 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 4975, 2024-09-19T07:44:14.078 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2024-09-19T07:44:14.078 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2024-09-19T07:44:14.078 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2024-09-19T07:44:14.078 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 4975 2024-09-19T07:44:14.078 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:44:14.078 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:44:14.079 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:44:14.079 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:44:14.079 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 613476257 2024-09-19T07:44:14.079 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-19T07:44:14.079 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:44:14.079 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:44:14.079 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:44:14.079 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:44:14.079 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 613476257 2024-09-19T07:44:14.079 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:44:14.079 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.4975 192.168.0.1:0/613476257", 2024-09-19T07:44:14.079 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/613476257", 2024-09-19T07:44:14.079 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2024-09-19T07:44:14.080 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 16, 2024-09-19T07:44:14.080 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 40, 2024-09-19T07:44:14.080 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:44:14.080 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2024-09-19T07:44:14.080 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2024-09-19T07:44:14.080 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-19T07:44:14.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:14.086 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2024-09-19T07:44:14.086 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T07:44:14.101 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2024-09-19T07:44:14.101 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T07:44:14.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:14.102 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:44:14.167 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_caps_per_client 1 2024-09-19T07:44:14.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.285+0000 7f03e2465640 1 Processor -- start 2024-09-19T07:44:14.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.285+0000 7f03e2465640 1 -- start start 2024-09-19T07:44:14.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.286+0000 7f03e2465640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f03dc15e2a0 0x7f03dc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:14.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.286+0000 7f03e2465640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f03dc15e680 0x7f03dc0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:14.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.286+0000 7f03e2465640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03dc06d960 0x7f03dc0d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:14.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.286+0000 7f03e2465640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f03dc0d1880 con 0x7f03dc15e680 2024-09-19T07:44:14.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.286+0000 7f03e2465640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f03dc0d1b60 con 0x7f03dc15e2a0 2024-09-19T07:44:14.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.286+0000 7f03e2465640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f03dc0cf2d0 con 0x7f03dc06d960 2024-09-19T07:44:14.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.286+0000 7f03db7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f03dc15e680 0x7f03dc0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:14.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.286+0000 7f03db7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f03dc15e680 0x7f03dc0cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41454/0 (socket says 172.21.15.16:41454) 2024-09-19T07:44:14.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.286+0000 7f03db7fe640 1 -- 172.21.15.16:0/1797785370 learned_addr learned my addr 172.21.15.16:0/1797785370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:14.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.286+0000 7f03dbfff640 1 --2- 172.21.15.16:0/1797785370 >> v2:172.21.15.146:3300/0 conn(0x7f03dc15e2a0 0x7f03dc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:14.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.286+0000 7f03e0a02640 1 --2- 172.21.15.16:0/1797785370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03dc06d960 0x7f03dc0d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:14.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.286+0000 7f03db7fe640 1 -- 172.21.15.16:0/1797785370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03dc06d960 msgr2=0x7f03dc0d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:14.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.286+0000 7f03db7fe640 1 --2- 172.21.15.16:0/1797785370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03dc06d960 0x7f03dc0d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.286+0000 7f03db7fe640 1 -- 172.21.15.16:0/1797785370 >> v2:172.21.15.146:3300/0 conn(0x7f03dc15e2a0 msgr2=0x7f03dc06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:14.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.287+0000 7f03db7fe640 1 --2- 172.21.15.16:0/1797785370 >> v2:172.21.15.146:3300/0 conn(0x7f03dc15e2a0 0x7f03dc06d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.287+0000 7f03db7fe640 1 -- 172.21.15.16:0/1797785370 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03dc0cf5b0 con 0x7f03dc15e680 2024-09-19T07:44:14.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.287+0000 7f03dbfff640 1 --2- 172.21.15.16:0/1797785370 >> v2:172.21.15.146:3300/0 conn(0x7f03dc15e2a0 0x7f03dc06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:14.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.287+0000 7f03db7fe640 1 --2- 172.21.15.16:0/1797785370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03dc15e680 0x7f03dc0cfad0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f03dc0b6fa0 tx=0x7f03d002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92132e018a187250 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:14.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.287+0000 7f03daffd640 1 -- 172.21.15.16:0/1797785370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0037070 con 0x7f03dc15e680 2024-09-19T07:44:14.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.287+0000 7f03daffd640 1 -- 172.21.15.16:0/1797785370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03d0032440 con 0x7f03dc15e680 2024-09-19T07:44:14.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.288+0000 7f03daffd640 1 -- 172.21.15.16:0/1797785370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0032810 con 0x7f03dc15e680 2024-09-19T07:44:14.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.288+0000 7f03e2465640 1 -- 172.21.15.16:0/1797785370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03dc15e680 msgr2=0x7f03dc0cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:14.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.288+0000 7f03e2465640 1 --2- 172.21.15.16:0/1797785370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03dc15e680 0x7f03dc0cfad0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f03dc0b6fa0 tx=0x7f03d002a1e0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.288+0000 7f03e2465640 1 -- 172.21.15.16:0/1797785370 shutdown_connections 2024-09-19T07:44:14.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.288+0000 7f03e2465640 1 --2- 172.21.15.16:0/1797785370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03dc06d960 0x7f03dc0d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.288+0000 7f03e2465640 1 --2- 172.21.15.16:0/1797785370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03dc15e680 0x7f03dc0cfad0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.288+0000 7f03e2465640 1 --2- 172.21.15.16:0/1797785370 >> v2:172.21.15.146:3300/0 conn(0x7f03dc15e2a0 0x7f03dc06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.288+0000 7f03e2465640 1 -- 172.21.15.16:0/1797785370 >> 172.21.15.16:0/1797785370 conn(0x7f03dc09d760 msgr2=0x7f03dc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:14.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.288+0000 7f03e2465640 1 -- 172.21.15.16:0/1797785370 shutdown_connections 2024-09-19T07:44:14.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.288+0000 7f03e2465640 1 -- 172.21.15.16:0/1797785370 wait complete. 2024-09-19T07:44:14.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.289+0000 7f03e2465640 1 Processor -- start 2024-09-19T07:44:14.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.290+0000 7f03e2465640 1 -- start start 2024-09-19T07:44:14.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.290+0000 7f03e2465640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03dc06d960 0x7f03dc1f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:14.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.290+0000 7f03e2465640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03dc15e2a0 0x7f03dc15b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:14.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.290+0000 7f03e2465640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03dc15e680 0x7f03dc0b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:14.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.290+0000 7f03e2465640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc0d44f0 con 0x7f03dc06d960 2024-09-19T07:44:14.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.290+0000 7f03e2465640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc0d4370 con 0x7f03dc15e680 2024-09-19T07:44:14.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.290+0000 7f03e2465640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f03dc0d4670 con 0x7f03dc15e2a0 2024-09-19T07:44:14.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.290+0000 7f03dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03dc06d960 0x7f03dc1f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:14.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.290+0000 7f03dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03dc06d960 0x7f03dc1f4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41458/0 (socket says 172.21.15.16:41458) 2024-09-19T07:44:14.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.291+0000 7f03dbfff640 1 -- 172.21.15.16:0/3059538173 learned_addr learned my addr 172.21.15.16:0/3059538173 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:14.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.291+0000 7f03db7fe640 1 --2- 172.21.15.16:0/3059538173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03dc15e2a0 0x7f03dc15b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:14.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.291+0000 7f03e0a02640 1 --2- 172.21.15.16:0/3059538173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03dc15e680 0x7f03dc0b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:14.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.291+0000 7f03dbfff640 1 -- 172.21.15.16:0/3059538173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03dc15e2a0 msgr2=0x7f03dc15b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:14.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.291+0000 7f03dbfff640 1 --2- 172.21.15.16:0/3059538173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03dc15e2a0 0x7f03dc15b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.291+0000 7f03dbfff640 1 -- 172.21.15.16:0/3059538173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03dc15e680 msgr2=0x7f03dc0b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:14.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.291+0000 7f03dbfff640 1 --2- 172.21.15.16:0/3059538173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03dc15e680 0x7f03dc0b5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.291+0000 7f03dbfff640 1 -- 172.21.15.16:0/3059538173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03dc0c82f0 con 0x7f03dc06d960 2024-09-19T07:44:14.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.291+0000 7f03dbfff640 1 --2- 172.21.15.16:0/3059538173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03dc06d960 0x7f03dc1f4480 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00e730 tx=0x7f03cc00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:14.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.291+0000 7f03d8ff9640 1 -- 172.21.15.16:0/3059538173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc00f040 con 0x7f03dc06d960 2024-09-19T07:44:14.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.291+0000 7f03d8ff9640 1 -- 172.21.15.16:0/3059538173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03cc004230 con 0x7f03dc06d960 2024-09-19T07:44:14.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.291+0000 7f03e2465640 1 -- 172.21.15.16:0/3059538173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03dc0c85d0 con 0x7f03dc06d960 2024-09-19T07:44:14.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.291+0000 7f03d8ff9640 1 -- 172.21.15.16:0/3059538173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc00ee40 con 0x7f03dc06d960 2024-09-19T07:44:14.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.291+0000 7f03e2465640 1 -- 172.21.15.16:0/3059538173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03dc0c8b90 con 0x7f03dc06d960 2024-09-19T07:44:14.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.292+0000 7f03e2465640 1 -- 172.21.15.16:0/3059538173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03dc06d420 con 0x7f03dc06d960 2024-09-19T07:44:14.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.293+0000 7f03d8ff9640 1 -- 172.21.15.16:0/3059538173 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f03cc00c6d0 con 0x7f03dc06d960 2024-09-19T07:44:14.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.293+0000 7f03d8ff9640 1 --2- 172.21.15.16:0/3059538173 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f03c806c2b0 0x7f03c806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:14.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.293+0000 7f03d8ff9640 1 -- 172.21.15.16:0/3059538173 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f03cc08b190 con 0x7f03dc06d960 2024-09-19T07:44:14.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.293+0000 7f03db7fe640 1 --2- 172.21.15.16:0/3059538173 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f03c806c2b0 0x7f03c806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:14.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.294+0000 7f03db7fe640 1 --2- 172.21.15.16:0/3059538173 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f03c806c2b0 0x7f03c806e770 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f03d0034000 tx=0x7f03d002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:14.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.297+0000 7f03d8ff9640 1 -- 172.21.15.16:0/3059538173 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f03cc017020 con 0x7f03dc06d960 2024-09-19T07:44:14.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.473+0000 7f03e2465640 1 -- 172.21.15.16:0/3059538173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_max_caps_per_client}] v 0) -- 0x7f03dc0c9f60 con 0x7f03dc06d960 2024-09-19T07:44:14.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.476+0000 7f03d8ff9640 1 -- 172.21.15.16:0/3059538173 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_caps_per_client}]=0 v11)=0 v11) ==== 121+0+0 (secure 0 0 0) 0x7f03cc059800 con 0x7f03dc06d960 2024-09-19T07:44:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.478+0000 7f03e2465640 1 -- 172.21.15.16:0/3059538173 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f03c806c2b0 msgr2=0x7f03c806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.478+0000 7f03e2465640 1 --2- 172.21.15.16:0/3059538173 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f03c806c2b0 0x7f03c806e770 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f03d0034000 tx=0x7f03d002e000 comp rx=0 tx=0).stop 2024-09-19T07:44:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.478+0000 7f03e2465640 1 -- 172.21.15.16:0/3059538173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03dc06d960 msgr2=0x7f03dc1f4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.478+0000 7f03e2465640 1 --2- 172.21.15.16:0/3059538173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03dc06d960 0x7f03dc1f4480 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00e730 tx=0x7f03cc00ebf0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.478+0000 7f03e2465640 1 -- 172.21.15.16:0/3059538173 shutdown_connections 2024-09-19T07:44:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.478+0000 7f03e2465640 1 --2- 172.21.15.16:0/3059538173 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f03c806c2b0 0x7f03c806e770 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.478+0000 7f03e2465640 1 --2- 172.21.15.16:0/3059538173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03dc15e680 0x7f03dc0b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.478+0000 7f03e2465640 1 --2- 172.21.15.16:0/3059538173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03dc15e2a0 0x7f03dc15b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.478+0000 7f03e2465640 1 --2- 172.21.15.16:0/3059538173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03dc06d960 0x7f03dc1f4480 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.478+0000 7f03e2465640 1 -- 172.21.15.16:0/3059538173 >> 172.21.15.16:0/3059538173 conn(0x7f03dc09d760 msgr2=0x7f03dc161540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.478+0000 7f03e2465640 1 -- 172.21.15.16:0/3059538173 shutdown_connections 2024-09-19T07:44:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.478+0000 7f03e2465640 1 -- 172.21.15.16:0/3059538173 wait complete. 2024-09-19T07:44:14.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_session_cap_acquisition_throttle 37 2024-09-19T07:44:14.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.595+0000 7fafe84f1640 1 Processor -- start 2024-09-19T07:44:14.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.595+0000 7fafe84f1640 1 -- start start 2024-09-19T07:44:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.596+0000 7fafe84f1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fafe015e680 0x7fafe00cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.596+0000 7fafe84f1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafe00d1880 con 0x7fafe015e2a0 2024-09-19T07:44:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.596+0000 7fafe84f1640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafe00d1b60 con 0x7fafe00d0010 2024-09-19T07:44:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.596+0000 7fafe84f1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fafe00cf2d0 con 0x7fafe015e680 2024-09-19T07:44:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.596+0000 7fafe5a8c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fafe015e680 0x7fafe00cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.596+0000 7fafe628d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fafe015e2a0 0x7fafe006d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54334/0 (socket says 172.21.15.16:54334) 2024-09-19T07:44:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.596+0000 7fafe628d640 1 -- 172.21.15.16:0/2576486165 learned_addr learned my addr 172.21.15.16:0/2576486165 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:44:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.596+0000 7fafe5a8c640 1 -- 172.21.15.16:0/2576486165 >> v1:172.21.15.146:6789/0 conn(0x7fafe00d0010 legacy=0x7fafe00d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:14.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.596+0000 7fafe5a8c640 1 -- 172.21.15.16:0/2576486165 >> v1:172.21.15.16:6789/0 conn(0x7fafe015e2a0 legacy=0x7fafe006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:14.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.596+0000 7fafe5a8c640 1 -- 172.21.15.16:0/2576486165 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe00cf5b0 con 0x7fafe015e680 2024-09-19T07:44:14.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.597+0000 7fafe5a8c640 1 --2- 172.21.15.16:0/2576486165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe015e680 0x7fafe00cfad0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fafd40097a0 tx=0x7fafd402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4045bf9c4564a1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:14.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.597+0000 7fafe528b640 1 -- 172.21.15.16:0/2576486165 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4037070 con 0x7fafe015e680 2024-09-19T07:44:14.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.597+0000 7fafe528b640 1 -- 172.21.15.16:0/2576486165 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd402ae50 con 0x7fafe015e680 2024-09-19T07:44:14.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.597+0000 7fafe528b640 1 -- 172.21.15.16:0/2576486165 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4032660 con 0x7fafe015e680 2024-09-19T07:44:14.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.598+0000 7fafe84f1640 1 -- 172.21.15.16:0/2576486165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe015e680 msgr2=0x7fafe00cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:14.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.598+0000 7fafe84f1640 1 --2- 172.21.15.16:0/2576486165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe015e680 0x7fafe00cfad0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fafd40097a0 tx=0x7fafd402a140 comp rx=0 tx=0).stop 2024-09-19T07:44:14.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.598+0000 7fafe84f1640 1 -- 172.21.15.16:0/2576486165 shutdown_connections 2024-09-19T07:44:14.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.598+0000 7fafe84f1640 1 --2- 172.21.15.16:0/2576486165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe015e680 0x7fafe00cfad0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.598+0000 7fafe84f1640 1 -- 172.21.15.16:0/2576486165 >> 172.21.15.16:0/2576486165 conn(0x7fafe009d760 msgr2=0x7fafe008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:14.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.598+0000 7fafe84f1640 1 -- 172.21.15.16:0/2576486165 shutdown_connections 2024-09-19T07:44:14.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.598+0000 7fafe84f1640 1 -- 172.21.15.16:0/2576486165 wait complete. 2024-09-19T07:44:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.599+0000 7fafe84f1640 1 Processor -- start 2024-09-19T07:44:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.599+0000 7fafe84f1640 1 -- start start 2024-09-19T07:44:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.600+0000 7fafe84f1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafe00d0010 0x7fafe00b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.600+0000 7fafe84f1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafe015e2a0 0x7fafe0150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.600+0000 7fafe84f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe015e680 0x7fafe0150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.600+0000 7fafe84f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fafe006dcb0 con 0x7fafe015e680 2024-09-19T07:44:14.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.600+0000 7fafe84f1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fafe006db30 con 0x7fafe015e2a0 2024-09-19T07:44:14.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.600+0000 7fafe84f1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fafe00d4a80 con 0x7fafe00d0010 2024-09-19T07:44:14.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.600+0000 7fafe6a8e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe015e680 0x7fafe0150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:14.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.600+0000 7fafe6a8e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe015e680 0x7fafe0150650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41488/0 (socket says 172.21.15.16:41488) 2024-09-19T07:44:14.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.600+0000 7fafe6a8e640 1 -- 172.21.15.16:0/1125736698 learned_addr learned my addr 172.21.15.16:0/1125736698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:14.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.600+0000 7fafe5a8c640 1 --2- 172.21.15.16:0/1125736698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafe015e2a0 0x7fafe0150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:14.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.600+0000 7fafe628d640 1 --2- 172.21.15.16:0/1125736698 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafe00d0010 0x7fafe00b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:14.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.600+0000 7fafe6a8e640 1 -- 172.21.15.16:0/1125736698 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafe00d0010 msgr2=0x7fafe00b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:14.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.600+0000 7fafe6a8e640 1 --2- 172.21.15.16:0/1125736698 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafe00d0010 0x7fafe00b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.600+0000 7fafe6a8e640 1 -- 172.21.15.16:0/1125736698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafe015e2a0 msgr2=0x7fafe0150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:14.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.600+0000 7fafe6a8e640 1 --2- 172.21.15.16:0/1125736698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafe015e2a0 0x7fafe0150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.600+0000 7fafe6a8e640 1 -- 172.21.15.16:0/1125736698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe014b2a0 con 0x7fafe015e680 2024-09-19T07:44:14.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.601+0000 7fafe6a8e640 1 --2- 172.21.15.16:0/1125736698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe015e680 0x7fafe0150650 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fafd800c780 tx=0x7fafd800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:14.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.601+0000 7fafd2ffd640 1 -- 172.21.15.16:0/1125736698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8004b60 con 0x7fafe015e680 2024-09-19T07:44:14.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.601+0000 7fafd2ffd640 1 -- 172.21.15.16:0/1125736698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd8004d00 con 0x7fafe015e680 2024-09-19T07:44:14.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.601+0000 7fafd2ffd640 1 -- 172.21.15.16:0/1125736698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd800aa90 con 0x7fafe015e680 2024-09-19T07:44:14.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.601+0000 7fafe84f1640 1 -- 172.21.15.16:0/1125736698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe014b580 con 0x7fafe015e680 2024-09-19T07:44:14.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.601+0000 7fafe84f1640 1 -- 172.21.15.16:0/1125736698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe014bb40 con 0x7fafe015e680 2024-09-19T07:44:14.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.602+0000 7fafe84f1640 1 -- 172.21.15.16:0/1125736698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafe00cfad0 con 0x7fafe015e680 2024-09-19T07:44:14.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.602+0000 7fafd2ffd640 1 -- 172.21.15.16:0/1125736698 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fafd800ac30 con 0x7fafe015e680 2024-09-19T07:44:14.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.603+0000 7fafd2ffd640 1 --2- 172.21.15.16:0/1125736698 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fafc406c2b0 0x7fafc406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:14.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.603+0000 7fafd2ffd640 1 -- 172.21.15.16:0/1125736698 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fafd808b7a0 con 0x7fafe015e680 2024-09-19T07:44:14.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.603+0000 7fafe628d640 1 --2- 172.21.15.16:0/1125736698 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fafc406c2b0 0x7fafc406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:14.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.603+0000 7fafe628d640 1 --2- 172.21.15.16:0/1125736698 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fafc406c2b0 0x7fafc406e770 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fafdc007fd0 tx=0x7fafdc007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:14.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.607+0000 7fafd2ffd640 1 -- 172.21.15.16:0/1125736698 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fafd8020070 con 0x7fafe015e680 2024-09-19T07:44:14.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.782+0000 7fafe84f1640 1 -- 172.21.15.16:0/1125736698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_session_cap_acquisition_throttle}] v 0) -- 0x7fafe006cf90 con 0x7fafe015e680 2024-09-19T07:44:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.785+0000 7fafd2ffd640 1 -- 172.21.15.16:0/1125736698 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_session_cap_acquisition_throttle}]=0 v12)=0 v12) ==== 135+0+0 (secure 0 0 0) 0x7fafd80596a0 con 0x7fafe015e680 2024-09-19T07:44:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.787+0000 7fafe84f1640 1 -- 172.21.15.16:0/1125736698 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fafc406c2b0 msgr2=0x7fafc406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.787+0000 7fafe84f1640 1 --2- 172.21.15.16:0/1125736698 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fafc406c2b0 0x7fafc406e770 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fafdc007fd0 tx=0x7fafdc007e80 comp rx=0 tx=0).stop 2024-09-19T07:44:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.787+0000 7fafe84f1640 1 -- 172.21.15.16:0/1125736698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe015e680 msgr2=0x7fafe0150650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.787+0000 7fafe84f1640 1 --2- 172.21.15.16:0/1125736698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe015e680 0x7fafe0150650 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fafd800c780 tx=0x7fafd800cc40 comp rx=0 tx=0).stop 2024-09-19T07:44:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.787+0000 7fafe84f1640 1 -- 172.21.15.16:0/1125736698 shutdown_connections 2024-09-19T07:44:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.787+0000 7fafe84f1640 1 --2- 172.21.15.16:0/1125736698 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fafc406c2b0 0x7fafc406e770 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.787+0000 7fafe84f1640 1 --2- 172.21.15.16:0/1125736698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe015e680 0x7fafe0150650 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.787+0000 7fafe84f1640 1 --2- 172.21.15.16:0/1125736698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafe015e2a0 0x7fafe0150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.787+0000 7fafe84f1640 1 --2- 172.21.15.16:0/1125736698 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafe00d0010 0x7fafe00b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.787+0000 7fafe84f1640 1 -- 172.21.15.16:0/1125736698 >> 172.21.15.16:0/1125736698 conn(0x7fafe009d760 msgr2=0x7fafe006d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.788+0000 7fafe84f1640 1 -- 172.21.15.16:0/1125736698 shutdown_connections 2024-09-19T07:44:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:14.788+0000 7fafe84f1640 1 -- 172.21.15.16:0/1125736698 wait complete. 2024-09-19T07:44:14.799 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:44:14.799 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:44:14.816 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-19T07:44:14.816 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:44:14.816 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-19T07:44:14.816 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:14.816 DEBUG:teuthology.orchestra.run.smithi016:> n = 25 2024-09-19T07:44:14.817 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-19T07:44:14.817 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-19T07:44:14.817 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-19T07:44:14.817 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/dir1/file" 2024-09-19T07:44:14.817 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:14.817 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-19T07:44:14.817 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-19T07:44:14.817 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-19T07:44:14.817 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:14.817 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-19T07:44:14.817 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-19T07:44:14.817 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:14.817 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-19T07:44:14.818 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-19T07:44:14.818 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-19T07:44:14.818 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-19T07:44:14.818 DEBUG:teuthology.orchestra.run.smithi016:> if True: 2024-09-19T07:44:14.818 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-19T07:44:14.818 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-19T07:44:14.818 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:44:14.818 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-19T07:44:14.818 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:44:14.818 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:44:14.818 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-19T07:44:14.818 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-19T07:44:14.818 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-19T07:44:14.818 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:44:14.819 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:44:14.819 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-19T07:44:14.819 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-19T07:44:14.819 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:44:15.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:44:15.045 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:44:15.045 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:44:15.102 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-19T07:44:15.102 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:44:15.102 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-19T07:44:15.102 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:15.102 DEBUG:teuthology.orchestra.run.smithi016:> n = 25 2024-09-19T07:44:15.102 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-19T07:44:15.102 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-19T07:44:15.102 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-19T07:44:15.102 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/dir2/file" 2024-09-19T07:44:15.102 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:15.103 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-19T07:44:15.103 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-19T07:44:15.103 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-19T07:44:15.103 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:15.103 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-19T07:44:15.103 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-19T07:44:15.103 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:15.103 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-19T07:44:15.103 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-19T07:44:15.103 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-19T07:44:15.103 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-19T07:44:15.103 DEBUG:teuthology.orchestra.run.smithi016:> if True: 2024-09-19T07:44:15.103 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-19T07:44:15.103 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-19T07:44:15.103 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:44:15.103 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-19T07:44:15.104 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:44:15.104 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:44:15.104 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-19T07:44:15.104 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-19T07:44:15.104 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-19T07:44:15.104 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:44:15.104 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:44:15.104 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-19T07:44:15.104 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-19T07:44:15.104 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:44:15.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:44:15.326 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:44:15.326 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:44:15.382 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-19T07:44:15.382 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:44:15.383 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-19T07:44:15.383 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:15.383 DEBUG:teuthology.orchestra.run.smithi016:> n = 25 2024-09-19T07:44:15.383 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-19T07:44:15.383 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-19T07:44:15.383 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-19T07:44:15.383 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/dir3/file" 2024-09-19T07:44:15.383 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:15.383 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-19T07:44:15.383 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-19T07:44:15.383 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-19T07:44:15.383 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:15.384 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-19T07:44:15.384 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-19T07:44:15.384 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:15.384 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-19T07:44:15.384 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-19T07:44:15.384 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-19T07:44:15.384 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-19T07:44:15.384 DEBUG:teuthology.orchestra.run.smithi016:> if True: 2024-09-19T07:44:15.384 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-19T07:44:15.384 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-19T07:44:15.384 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:44:15.384 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-19T07:44:15.384 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:44:15.384 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:44:15.385 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-19T07:44:15.385 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-19T07:44:15.385 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-19T07:44:15.385 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:44:15.385 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:44:15.385 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-19T07:44:15.385 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-19T07:44:15.385 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:44:15.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:44:15.619 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:44:15.619 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:44:15.676 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-19T07:44:15.676 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:44:15.676 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-19T07:44:15.676 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:15.676 DEBUG:teuthology.orchestra.run.smithi016:> n = 25 2024-09-19T07:44:15.676 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-19T07:44:15.676 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-19T07:44:15.676 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-19T07:44:15.676 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/dir4/file" 2024-09-19T07:44:15.677 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:15.677 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-19T07:44:15.677 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-19T07:44:15.677 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-19T07:44:15.677 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:15.677 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-19T07:44:15.677 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-19T07:44:15.677 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:15.677 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-19T07:44:15.677 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-19T07:44:15.677 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-19T07:44:15.677 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-19T07:44:15.677 DEBUG:teuthology.orchestra.run.smithi016:> if True: 2024-09-19T07:44:15.678 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-19T07:44:15.678 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-19T07:44:15.678 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:44:15.678 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-19T07:44:15.678 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:44:15.678 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:44:15.678 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-19T07:44:15.678 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-19T07:44:15.678 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-19T07:44:15.678 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:44:15.678 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:44:15.678 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-19T07:44:15.678 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-19T07:44:15.678 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:44:15.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:44:15.907 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:44:15.907 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:44:15.907 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:44:15.907 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:44:15.907 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:44:15.907 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:15.907 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:44:15.907 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:44:15.908 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:44:15.908 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:44:15.908 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:15.908 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:44:15.908 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:44:15.908 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:44:15.908 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:44:15.908 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:15.908 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:44:15.908 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:44:15.908 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:44:15.908 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:44:15.908 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:44:15.909 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:44:15.909 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:44:15.909 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:44:15.909 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:44:15.909 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:44:15.909 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:44:16.027 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.48356.asok 2024-09-19T07:44:16.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:44:16.038 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48356.asok 2024-09-19T07:44:16.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:16.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48356.asok mds_sessions 2024-09-19T07:44:16.175 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:44:16.175 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5151, 2024-09-19T07:44:16.175 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:44:16.175 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:44:16.176 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:16.176 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:16.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:16.176 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:44:16.176 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:44:16.176 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:44:16.176 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2625487776 2024-09-19T07:44:16.176 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:16.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:16.176 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.5151 v1:192.168.0.1:0/2625487776", 2024-09-19T07:44:16.176 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "v1:192.168.0.1:0/2625487776", 2024-09-19T07:44:16.176 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions": [ 2024-09-19T07:44:16.177 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:16.177 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": 0, 2024-09-19T07:44:16.177 INFO:teuthology.orchestra.run.smithi016.stdout: "addrs": { 2024-09-19T07:44:16.177 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-19T07:44:16.177 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:16.177 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-19T07:44:16.177 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6833", 2024-09-19T07:44:16.177 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2028210099 2024-09-19T07:44:16.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:16.177 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:16.177 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:44:16.178 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6835", 2024-09-19T07:44:16.178 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2028210099 2024-09-19T07:44:16.178 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:16.178 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T07:44:16.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:16.178 INFO:teuthology.orchestra.run.smithi016.stdout: "seq": 204, 2024-09-19T07:44:16.178 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_gen": 0, 2024-09-19T07:44:16.178 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_ttl": "2024-09-19T07:45:13.265125+0000", 2024-09-19T07:44:16.178 INFO:teuthology.orchestra.run.smithi016.stdout: "last_cap_renew_request": "2024-09-19T07:44:13.265125+0000", 2024-09-19T07:44:16.178 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_renew_seq": 2, 2024-09-19T07:44:16.178 INFO:teuthology.orchestra.run.smithi016.stdout: "num_caps": 105, 2024-09-19T07:44:16.178 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "open" 2024-09-19T07:44:16.178 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:16.179 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:16.179 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 16 2024-09-19T07:44:16.179 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:44:16.183 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'find . | wc') 2024-09-19T07:44:16.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T07:44:37.167 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:44:37.171 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:44:55.267 INFO:teuthology.orchestra.run.smithi016.stdout: 105 105 1490 2024-09-19T07:44:55.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:44:55.276 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:44:55.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.388+0000 7f9ab757f640 1 Processor -- start 2024-09-19T07:44:55.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.388+0000 7f9ab757f640 1 -- start start 2024-09-19T07:44:55.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.389+0000 7f9ab757f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ab015e2a0 0x7f9ab006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:55.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.389+0000 7f9ab757f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9ab015e680 0x7f9ab00cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:55.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.389+0000 7f9ab757f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ab00d1880 con 0x7f9ab006d960 2024-09-19T07:44:55.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.389+0000 7f9ab757f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f9ab00d1b60 con 0x7f9ab015e680 2024-09-19T07:44:55.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.389+0000 7f9ab757f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab00cf2d0 con 0x7f9ab015e2a0 2024-09-19T07:44:55.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.390+0000 7f9ab531b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ab015e2a0 0x7f9ab006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:55.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.390+0000 7f9ab4b1a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9ab015e680 0x7f9ab00cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:55.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.390+0000 7f9ab5b1c640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f9ab006d960 0x7f9ab00d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:38726/0 (socket says 172.21.15.16:38726) 2024-09-19T07:44:55.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.390+0000 7f9ab531b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ab015e2a0 0x7f9ab006d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:35764/0 (socket says 172.21.15.16:35764) 2024-09-19T07:44:55.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.390+0000 7f9ab5b1c640 1 -- 172.21.15.16:0/1288494423 learned_addr learned my addr 172.21.15.16:0/1288494423 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:44:55.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.390+0000 7f9aa7fff640 1 -- 172.21.15.16:0/1288494423 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2205311116 0 0) 0x7f9ab00d1880 con 0x7f9ab006d960 2024-09-19T07:44:55.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.390+0000 7f9aa7fff640 1 -- 172.21.15.16:0/1288494423 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a9c003520 con 0x7f9ab006d960 2024-09-19T07:44:55.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.390+0000 7f9ab4b1a640 1 -- 172.21.15.16:0/1288494423 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ab015e2a0 msgr2=0x7f9ab006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:55.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.390+0000 7f9ab4b1a640 1 --2- 172.21.15.16:0/1288494423 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ab015e2a0 0x7f9ab006d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:55.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.390+0000 7f9ab4b1a640 1 -- 172.21.15.16:0/1288494423 >> v1:172.21.15.146:6789/0 conn(0x7f9ab006d960 legacy=0x7f9ab00d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:55.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.390+0000 7f9ab4b1a640 1 -- 172.21.15.16:0/1288494423 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab00cf5b0 con 0x7f9ab015e680 2024-09-19T07:44:55.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.390+0000 7f9ab531b640 1 --2- 172.21.15.16:0/1288494423 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ab015e2a0 0x7f9ab006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:44:55.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.391+0000 7f9ab4b1a640 1 --2- 172.21.15.16:0/1288494423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ab015e680 0x7f9ab00cfad0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00097a0 tx=0x7f9aa002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a39deb0a1aeec3e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:55.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.391+0000 7f9aa7fff640 1 -- 172.21.15.16:0/1288494423 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0037070 con 0x7f9ab015e680 2024-09-19T07:44:55.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.391+0000 7f9aa7fff640 1 -- 172.21.15.16:0/1288494423 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa0032440 con 0x7f9ab015e680 2024-09-19T07:44:55.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.391+0000 7f9aa7fff640 1 -- 172.21.15.16:0/1288494423 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa00327f0 con 0x7f9ab015e680 2024-09-19T07:44:55.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.392+0000 7f9ab757f640 1 -- 172.21.15.16:0/1288494423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ab015e680 msgr2=0x7f9ab00cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:55.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.392+0000 7f9ab757f640 1 --2- 172.21.15.16:0/1288494423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ab015e680 0x7f9ab00cfad0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00097a0 tx=0x7f9aa002a180 comp rx=0 tx=0).stop 2024-09-19T07:44:55.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.392+0000 7f9ab757f640 1 -- 172.21.15.16:0/1288494423 shutdown_connections 2024-09-19T07:44:55.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.392+0000 7f9ab757f640 1 --2- 172.21.15.16:0/1288494423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ab015e680 0x7f9ab00cfad0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:55.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.392+0000 7f9ab757f640 1 --2- 172.21.15.16:0/1288494423 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ab015e2a0 0x7f9ab006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:55.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.392+0000 7f9ab757f640 1 -- 172.21.15.16:0/1288494423 >> 172.21.15.16:0/1288494423 conn(0x7f9ab009d760 msgr2=0x7f9ab008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:55.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.392+0000 7f9ab757f640 1 -- 172.21.15.16:0/1288494423 shutdown_connections 2024-09-19T07:44:55.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.392+0000 7f9ab757f640 1 -- 172.21.15.16:0/1288494423 wait complete. 2024-09-19T07:44:55.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.393+0000 7f9ab757f640 1 Processor -- start 2024-09-19T07:44:55.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.393+0000 7f9ab757f640 1 -- start start 2024-09-19T07:44:55.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.394+0000 7f9ab757f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ab006d960 0x7f9ab00caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:55.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.394+0000 7f9ab757f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ab015e2a0 0x7f9ab00b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:55.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.394+0000 7f9ab757f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab015e680 0x7f9ab00cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:55.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.394+0000 7f9ab757f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab00d4c00 con 0x7f9ab015e680 2024-09-19T07:44:55.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.394+0000 7f9ab757f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab00d4a80 con 0x7f9ab015e2a0 2024-09-19T07:44:55.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.394+0000 7f9ab757f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab00d0010 con 0x7f9ab006d960 2024-09-19T07:44:55.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.394+0000 7f9ab5b1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab015e680 0x7f9ab00cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:55.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.394+0000 7f9ab5b1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab015e680 0x7f9ab00cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60174/0 (socket says 172.21.15.16:60174) 2024-09-19T07:44:55.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.394+0000 7f9ab5b1c640 1 -- 172.21.15.16:0/3025033519 learned_addr learned my addr 172.21.15.16:0/3025033519 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:55.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.394+0000 7f9ab531b640 1 --2- 172.21.15.16:0/3025033519 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ab006d960 0x7f9ab00caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:55.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.394+0000 7f9ab4b1a640 1 --2- 172.21.15.16:0/3025033519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ab015e2a0 0x7f9ab00b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:55.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.394+0000 7f9ab5b1c640 1 -- 172.21.15.16:0/3025033519 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ab006d960 msgr2=0x7f9ab00caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:55.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.394+0000 7f9ab5b1c640 1 --2- 172.21.15.16:0/3025033519 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ab006d960 0x7f9ab00caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:55.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.394+0000 7f9ab5b1c640 1 -- 172.21.15.16:0/3025033519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ab015e2a0 msgr2=0x7f9ab00b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:55.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.394+0000 7f9ab5b1c640 1 --2- 172.21.15.16:0/3025033519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ab015e2a0 0x7f9ab00b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:55.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.394+0000 7f9ab5b1c640 1 -- 172.21.15.16:0/3025033519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab00b7a60 con 0x7f9ab015e680 2024-09-19T07:44:55.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.394+0000 7f9ab531b640 1 --2- 172.21.15.16:0/3025033519 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ab006d960 0x7f9ab00caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:55.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.395+0000 7f9ab5b1c640 1 --2- 172.21.15.16:0/3025033519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab015e680 0x7f9ab00cd730 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800c460 tx=0x7f9aa800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:55.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.395+0000 7f9aa5ffb640 1 -- 172.21.15.16:0/3025033519 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8017070 con 0x7f9ab015e680 2024-09-19T07:44:55.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.395+0000 7f9aa5ffb640 1 -- 172.21.15.16:0/3025033519 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa8005ce0 con 0x7f9ab015e680 2024-09-19T07:44:55.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.395+0000 7f9ab757f640 1 -- 172.21.15.16:0/3025033519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab00b7d40 con 0x7f9ab015e680 2024-09-19T07:44:55.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.395+0000 7f9aa5ffb640 1 -- 172.21.15.16:0/3025033519 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8007290 con 0x7f9ab015e680 2024-09-19T07:44:55.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.395+0000 7f9ab757f640 1 -- 172.21.15.16:0/3025033519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab00b8840 con 0x7f9ab015e680 2024-09-19T07:44:55.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.396+0000 7f9ab757f640 1 -- 172.21.15.16:0/3025033519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab006d420 con 0x7f9ab015e680 2024-09-19T07:44:55.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.397+0000 7f9aa5ffb640 1 -- 172.21.15.16:0/3025033519 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9aa801a070 con 0x7f9ab015e680 2024-09-19T07:44:55.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.397+0000 7f9aa5ffb640 1 --2- 172.21.15.16:0/3025033519 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9a9406c2b0 0x7f9a9406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:55.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.397+0000 7f9aa5ffb640 1 -- 172.21.15.16:0/3025033519 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f9aa808a790 con 0x7f9ab015e680 2024-09-19T07:44:55.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.397+0000 7f9ab531b640 1 --2- 172.21.15.16:0/3025033519 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9a9406c2b0 0x7f9a9406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:55.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.398+0000 7f9ab531b640 1 --2- 172.21.15.16:0/3025033519 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9a9406c2b0 0x7f9a9406e770 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9aac007520 tx=0x7f9aac00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:55.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.402+0000 7f9aa5ffb640 1 -- 172.21.15.16:0/3025033519 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9aa8058790 con 0x7f9ab015e680 2024-09-19T07:44:55.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.615+0000 7f9ab757f640 1 -- 172.21.15.16:0/3025033519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ab006cf90 con 0x7f9ab015e680 2024-09-19T07:44:55.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.615+0000 7f9aa5ffb640 1 -- 172.21.15.16:0/3025033519 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5229 (secure 0 0 0) 0x7f9aa805ca90 con 0x7f9ab015e680 2024-09-19T07:44:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:44:55.617 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-19T07:44:04:108666+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4655,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/3238551814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3238551814},{"type":"v1","addr":"172.21.15.146:6836","nonce":3238551814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3450384245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3450384245},{"type":"v1","addr":"172.21.15.146:6837","nonce":3450384245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4944,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3663416804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3663416804},{"type":"v1","addr":"172.21.15.16:6834","nonce":3663416804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:44:00.970677+0000","modified":"2024-09-19T07:44:03.455740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4947":{"gid":4947,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2028210099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2028210099},{"type":"v1","addr":"172.21.15.16:6835","nonce":2028210099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4947,"qdb_cluster":[4947]},"id":2}]} 2024-09-19T07:44:55.617 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-19T07:44:55.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.617+0000 7f9ab757f640 1 -- 172.21.15.16:0/3025033519 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9a9406c2b0 msgr2=0x7f9a9406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:55.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.617+0000 7f9ab757f640 1 --2- 172.21.15.16:0/3025033519 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9a9406c2b0 0x7f9a9406e770 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9aac007520 tx=0x7f9aac00b040 comp rx=0 tx=0).stop 2024-09-19T07:44:55.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.617+0000 7f9ab757f640 1 -- 172.21.15.16:0/3025033519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab015e680 msgr2=0x7f9ab00cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:55.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.617+0000 7f9ab757f640 1 --2- 172.21.15.16:0/3025033519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab015e680 0x7f9ab00cd730 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800c460 tx=0x7f9aa800c920 comp rx=0 tx=0).stop 2024-09-19T07:44:55.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.618+0000 7f9ab757f640 1 -- 172.21.15.16:0/3025033519 shutdown_connections 2024-09-19T07:44:55.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.618+0000 7f9ab757f640 1 --2- 172.21.15.16:0/3025033519 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9a9406c2b0 0x7f9a9406e770 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:55.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.618+0000 7f9ab757f640 1 --2- 172.21.15.16:0/3025033519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab015e680 0x7f9ab00cd730 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:55.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.618+0000 7f9ab757f640 1 --2- 172.21.15.16:0/3025033519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ab015e2a0 0x7f9ab00b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:55.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.618+0000 7f9ab757f640 1 --2- 172.21.15.16:0/3025033519 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ab006d960 0x7f9ab00caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:55.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.618+0000 7f9ab757f640 1 -- 172.21.15.16:0/3025033519 >> 172.21.15.16:0/3025033519 conn(0x7f9ab009d760 msgr2=0x7f9ab0074ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:55.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.618+0000 7f9ab757f640 1 -- 172.21.15.16:0/3025033519 shutdown_connections 2024-09-19T07:44:55.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.618+0000 7f9ab757f640 1 -- 172.21.15.16:0/3025033519 wait complete. 2024-09-19T07:44:55.630 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-09-19T07:44:55.749 INFO:teuthology.orchestra.run.smithi016.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":184,"msgr_send_messages":145,"msgr_recv_bytes":52875,"msgr_send_bytes":663678,"msgr_created_connections":8,"msgr_active_connections":4,"msgr_running_total_time":0.026577097,"msgr_running_send_time":0.005336035,"msgr_running_recv_time":0.013519300,"msgr_running_fast_dispatch_time":0.005352300,"msgr_send_messages_queue_lat":{"avgcount":143,"sum":0.117981764,"avgtime":0.000825047},"msgr_handle_ack_lat":{"avgcount":55,"sum":0.000010070,"avgtime":0.000000183},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":79,"msgr_send_messages":70,"msgr_recv_bytes":121656,"msgr_send_bytes":22622,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.021080613,"msgr_running_send_time":0.005216812,"msgr_running_recv_time":0.031193202,"msgr_running_fast_dispatch_time":0.000280378,"msgr_send_messages_queue_lat":{"avgcount":70,"sum":0.094470014,"avgtime":0.001349571},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":118944,"msgr_send_encrypted_bytes":17968},"AsyncMessenger::Worker-2":{"msgr_recv_messages":494,"msgr_send_messages":479,"msgr_recv_bytes":147705,"msgr_send_bytes":369981,"msgr_created_connections":8,"msgr_active_connections":7,"msgr_running_total_time":0.037763187,"msgr_running_send_time":0.014611203,"msgr_running_recv_time":0.027319036,"msgr_running_fast_dispatch_time":0.002095823,"msgr_send_messages_queue_lat":{"avgcount":478,"sum":0.150057150,"avgtime":0.000313927},"msgr_handle_ack_lat":{"avgcount":937,"sum":0.000122301,"avgtime":0.000000130},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":108432},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":262,"sum":0.072850369,"avgtime":0.000278054}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000008565,"avgtime":0.000004282}},"mds":{"request":131,"reply":131,"reply_latency":{"avgcount":131,"sum":39.089839258,"avgtime":0.298395719},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":114,"inodes_top":0,"inodes_bottom":114,"inodes_pin_tail":0,"inodes_pinned":114,"inodes_expired":0,"inodes_with_caps":100,"caps":101,"subtrees":2,"traverse":157,"traverse_hit":144,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":100,"root_rbytes":160,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":9,"ceph_cap_op_grant":104,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":204,"handle_client_caps_dirty":100,"handle_client_cap_release":1,"process_request_cap_release":105},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":313,"evex":0,"evtrm":0,"ev":313,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4796143,"rdpos":0,"jlat":{"avgcount":110,"sum":0.173860734,"avgtime":0.001580552},"replayed":0},"mds_mem":{"ino":117,"ino+":117,"ino-":0,"dir":16,"dir+":16,"dir-":0,"dn":114,"dn+":115,"dn-":1,"cap":101,"cap+":106,"cap-":5,"rss":54208,"heap":231788},"mds_server":{"dispatch_client_request":218,"dispatch_server_request":0,"handle_client_request":131,"handle_client_session":110,"handle_peer_request":0,"req_create_latency":{"avgcount":100,"sum":0.063693613,"avgtime":0.000636936},"req_getattr_latency":{"avgcount":17,"sum":0.003030258,"avgtime":0.000178250},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":4,"sum":0.000418795,"avgtime":0.000104698},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":4,"sum":0.004544636,"avgtime":0.001136159},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":5,"sum":39.017753099,"avgtime":7.803550619},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000398857,"avgtime":0.000398857},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":78,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":2,"session_add":2,"session_remove":0,"sessions_open":2,"sessions_stale":0,"total_load":83,"average_load":41,"avg_session_uptime":41,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":243491,"buffer_anon_items":95,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5520,"osdmap_items":78,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":392272,"mds_co_items":2313,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":154,"op_send_bytes":621135,"op_resend":31,"op_reply":154,"op_latency":{"avgcount":154,"sum":0.977986481,"avgtime":0.006350561},"op_inflight":0,"oplen_avg":{"avgcount":154,"sum":183},"op":154,"op_r":0,"op_w":154,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":110,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":42,"map_full":0,"map_inc":3,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":22,"omap_rd":0,"omap_del":4},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":2},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":752,"get_sum":249765,"get_or_fail_fail":0,"get_or_fail_success":752,"take":0,"take_sum":0,"put":752,"put_sum":249765,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":154,"take_sum":621135,"put":154,"put_sum":621135,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":154,"take_sum":154,"put":154,"put_sum":154,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x5618aef00500":{"val":0,"max":3758096384,"get_started":0,"get":313,"get_sum":601839,"get_or_fail_fail":0,"get_or_fail_success":313,"take":0,"take_sum":0,"put":110,"put_sum":601839,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-19T07:44:55.750 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 4, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 55, "avgtime": 1.83e-07, "sum": 1.007e-05 }, "msgr_recv_bytes": 52875, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 184, "msgr_running_fast_dispatch_time": 0.0053523, "msgr_running_recv_time": 0.0135193, "msgr_running_send_time": 0.005336035, "msgr_running_total_time": 0.026577097, "msgr_send_bytes": 663678, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 145, "msgr_send_messages_queue_lat": { "avgcount": 143, "avgtime": 0.000825047, "sum": 0.117981764 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 121656, "msgr_recv_encrypted_bytes": 118944, "msgr_recv_messages": 79, "msgr_running_fast_dispatch_time": 0.000280378, "msgr_running_recv_time": 0.031193202, "msgr_running_send_time": 0.005216812, "msgr_running_total_time": 0.021080613, "msgr_send_bytes": 22622, "msgr_send_encrypted_bytes": 17968, "msgr_send_messages": 70, "msgr_send_messages_queue_lat": { "avgcount": 70, "avgtime": 0.001349571, "sum": 0.094470014 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 7, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 937, "avgtime": 1.3e-07, "sum": 0.000122301 }, "msgr_recv_bytes": 147705, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 494, "msgr_running_fast_dispatch_time": 0.002095823, "msgr_running_recv_time": 0.027319036, "msgr_running_send_time": 0.014611203, "msgr_running_total_time": 0.037763187, "msgr_send_bytes": 369981, "msgr_send_encrypted_bytes": 108432, "msgr_send_messages": 479, "msgr_send_messages_queue_lat": { "avgcount": 478, "avgtime": 0.000313927, "sum": 0.15005715 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 262, "avgtime": 0.000278054, "sum": 0.072850369 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 4.282e-06, "sum": 8.565e-06 }, "queue_len": 0 }, "mds": { "caps": 101, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 104, "ceph_cap_op_revoke": 9, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 1, "handle_client_caps": 204, "handle_client_caps_dirty": 100, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 114, "inodes_bottom": 114, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 114, "inodes_top": 0, "inodes_with_caps": 100, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 105, "q": 0, "reply": 131, "reply_latency": { "avgcount": 131, "avgtime": 0.298395719, "sum": 39.089839258 }, "request": 131, "root_rbytes": 160, "root_rfiles": 100, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 157, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 144, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 313, "evadd": 313, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 110, "avgtime": 0.001580552, "sum": 0.173860734 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4796143 }, "mds_mem": { "cap": 101, "cap+": 106, "cap-": 5, "dir": 16, "dir+": 16, "dir-": 0, "dn": 114, "dn+": 115, "dn-": 1, "heap": 231788, "ino": 117, "ino+": 117, "ino-": 0, "rss": 54208 }, "mds_server": { "cap_acquisition_throttle": 78, "cap_revoke_eviction": 0, "dispatch_client_request": 218, "dispatch_server_request": 0, "handle_client_request": 131, "handle_client_session": 110, "handle_peer_request": 0, "req_create_latency": { "avgcount": 100, "avgtime": 0.000636936, "sum": 0.063693613 }, "req_getattr_latency": { "avgcount": 17, "avgtime": 0.00017825, "sum": 0.003030258 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 4, "avgtime": 0.000104698, "sum": 0.000418795 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 4, "avgtime": 0.001136159, "sum": 0.004544636 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 5, "avgtime": 7.803550619, "sum": 39.017753099 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000398857, "sum": 0.000398857 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 41, "avg_session_uptime": 41, "mdthresh_evicted": 0, "session_add": 2, "session_count": 2, "session_remove": 0, "sessions_open": 2, "sessions_stale": 0, "total_load": 83 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 243491, "buffer_anon_items": 95, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 392272, "mds_co_items": 2313, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5520, "osdmap_items": 78, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 42, "map_full": 0, "map_inc": 3, "omap_del": 4, "omap_rd": 0, "omap_wr": 22, "op": 154, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 154, "avgtime": 0.006350561, "sum": 0.977986481 }, "op_pg": 0, "op_r": 0, "op_reply": 154, "op_resend": 31, "op_rmw": 0, "op_send": 154, "op_send_bytes": 621135, "op_w": 154, "oplen_avg": { "avgcount": 154, "sum": 183 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 110, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 2 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 752, "get_or_fail_fail": 0, "get_or_fail_success": 752, "get_started": 0, "get_sum": 249765, "max": 104857600, "put": 752, "put_sum": 249765, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 154, "put_sum": 621135, "take": 154, "take_sum": 621135, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 154, "put_sum": 154, "take": 154, "take_sum": 154, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x5618aef00500": { "get": 313, "get_or_fail_fail": 0, "get_or_fail_success": 313, "get_started": 0, "get_sum": 601839, "max": 3758096384, "put": 110, "put_sum": 601839, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-19T07:44:55.751 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:44:55.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.851+0000 7f574c04c640 1 Processor -- start 2024-09-19T07:44:55.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.852+0000 7f574c04c640 1 -- start start 2024-09-19T07:44:55.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.852+0000 7f574c04c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f574415f0d0 0x7f574415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:55.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.852+0000 7f574c04c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f57440d4130 0x7f57440d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:55.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.852+0000 7f574c04c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57440d4b20 con 0x7f57440d0070 2024-09-19T07:44:55.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.852+0000 7f574c04c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f57440d1b10 con 0x7f57440d4130 2024-09-19T07:44:55.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.852+0000 7f574c04c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f57440cf2b0 con 0x7f574415f0d0 2024-09-19T07:44:55.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.853+0000 7f574a5e9640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f57440d4130 0x7f57440d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.853+0000 7f5749de8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f574415f0d0 0x7f574415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.853+0000 7f57495e7640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f57440d0070 0x7f57440d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:38736/0 (socket says 172.21.15.16:38736) 2024-09-19T07:44:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.853+0000 7f57495e7640 1 -- 172.21.15.16:0/3091268016 learned_addr learned my addr 172.21.15.16:0/3091268016 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:44:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.853+0000 7f574a5e9640 1 -- 172.21.15.16:0/3091268016 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f574415f0d0 msgr2=0x7f574415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.853+0000 7f574a5e9640 1 --2- 172.21.15.16:0/3091268016 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f574415f0d0 0x7f574415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.853+0000 7f574a5e9640 1 -- 172.21.15.16:0/3091268016 >> v1:172.21.15.146:6789/0 conn(0x7f57440d0070 legacy=0x7f57440d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:55.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.853+0000 7f574a5e9640 1 -- 172.21.15.16:0/3091268016 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57440cf590 con 0x7f57440d4130 2024-09-19T07:44:55.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.854+0000 7f574a5e9640 1 --2- 172.21.15.16:0/3091268016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57440d4130 0x7f57440d45e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f57440d2fc0 tx=0x7f573c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ce2caf7076bc1bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:55.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.854+0000 7f5748de6640 1 -- 172.21.15.16:0/3091268016 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c037070 con 0x7f57440d4130 2024-09-19T07:44:55.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.854+0000 7f5748de6640 1 -- 172.21.15.16:0/3091268016 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f573c02ae50 con 0x7f57440d4130 2024-09-19T07:44:55.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.854+0000 7f5748de6640 1 -- 172.21.15.16:0/3091268016 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c032660 con 0x7f57440d4130 2024-09-19T07:44:55.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.855+0000 7f574c04c640 1 -- 172.21.15.16:0/3091268016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57440d4130 msgr2=0x7f57440d45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:55.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.855+0000 7f574c04c640 1 --2- 172.21.15.16:0/3091268016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57440d4130 0x7f57440d45e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f57440d2fc0 tx=0x7f573c02a100 comp rx=0 tx=0).stop 2024-09-19T07:44:55.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.855+0000 7f574c04c640 1 -- 172.21.15.16:0/3091268016 shutdown_connections 2024-09-19T07:44:55.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.855+0000 7f574c04c640 1 --2- 172.21.15.16:0/3091268016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57440d4130 0x7f57440d45e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:55.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.855+0000 7f574c04c640 1 --2- 172.21.15.16:0/3091268016 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f574415f0d0 0x7f574415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:55.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.855+0000 7f574c04c640 1 -- 172.21.15.16:0/3091268016 >> 172.21.15.16:0/3091268016 conn(0x7f574409d760 msgr2=0x7f574408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:55.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.855+0000 7f574c04c640 1 -- 172.21.15.16:0/3091268016 shutdown_connections 2024-09-19T07:44:55.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.855+0000 7f574c04c640 1 -- 172.21.15.16:0/3091268016 wait complete. 2024-09-19T07:44:55.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.856+0000 7f574c04c640 1 Processor -- start 2024-09-19T07:44:55.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.856+0000 7f574c04c640 1 -- start start 2024-09-19T07:44:55.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.857+0000 7f574c04c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57440d0070 0x7f57440cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.857+0000 7f574c04c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57440d4130 0x7f57440ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.857+0000 7f574c04c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f574415f0d0 0x7f57440b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.857+0000 7f574c04c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57440cef60 con 0x7f57440d4130 2024-09-19T07:44:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.857+0000 7f574c04c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f57440cfb50 con 0x7f57440d0070 2024-09-19T07:44:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.857+0000 7f574c04c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f57440b2f70 con 0x7f574415f0d0 2024-09-19T07:44:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.857+0000 7f57495e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57440d4130 0x7f57440ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.857+0000 7f57495e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57440d4130 0x7f57440ca820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60176/0 (socket says 172.21.15.16:60176) 2024-09-19T07:44:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.857+0000 7f57495e7640 1 -- 172.21.15.16:0/4100024283 learned_addr learned my addr 172.21.15.16:0/4100024283 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.857+0000 7f5749de8640 1 --2- 172.21.15.16:0/4100024283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57440d0070 0x7f57440cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.857+0000 7f574a5e9640 1 --2- 172.21.15.16:0/4100024283 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f574415f0d0 0x7f57440b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.857+0000 7f57495e7640 1 -- 172.21.15.16:0/4100024283 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f574415f0d0 msgr2=0x7f57440b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:55.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.857+0000 7f57495e7640 1 --2- 172.21.15.16:0/4100024283 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f574415f0d0 0x7f57440b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:55.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.857+0000 7f57495e7640 1 -- 172.21.15.16:0/4100024283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57440d0070 msgr2=0x7f57440cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:55.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.857+0000 7f57495e7640 1 --2- 172.21.15.16:0/4100024283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57440d0070 0x7f57440cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:55.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.857+0000 7f57495e7640 1 -- 172.21.15.16:0/4100024283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57440c9a60 con 0x7f57440d4130 2024-09-19T07:44:55.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.857+0000 7f5749de8640 1 --2- 172.21.15.16:0/4100024283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57440d0070 0x7f57440cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:44:55.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.858+0000 7f57495e7640 1 --2- 172.21.15.16:0/4100024283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57440d4130 0x7f57440ca820 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f573800c440 tx=0x7f5738001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:55.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.858+0000 7f57427fc640 1 -- 172.21.15.16:0/4100024283 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738017070 con 0x7f57440d4130 2024-09-19T07:44:55.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.858+0000 7f57427fc640 1 -- 172.21.15.16:0/4100024283 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5738005ce0 con 0x7f57440d4130 2024-09-19T07:44:55.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.858+0000 7f57427fc640 1 -- 172.21.15.16:0/4100024283 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57380072f0 con 0x7f57440d4130 2024-09-19T07:44:55.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.858+0000 7f574c04c640 1 -- 172.21.15.16:0/4100024283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57440c9d40 con 0x7f57440d4130 2024-09-19T07:44:55.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.858+0000 7f574c04c640 1 -- 172.21.15.16:0/4100024283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57440ccd50 con 0x7f57440d4130 2024-09-19T07:44:55.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.859+0000 7f574c04c640 1 -- 172.21.15.16:0/4100024283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57440d45e0 con 0x7f57440d4130 2024-09-19T07:44:55.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.859+0000 7f57427fc640 1 -- 172.21.15.16:0/4100024283 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f573801a070 con 0x7f57440d4130 2024-09-19T07:44:55.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.860+0000 7f57427fc640 1 --2- 172.21.15.16:0/4100024283 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f573006c2e0 0x7f573006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:55.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.860+0000 7f57427fc640 1 -- 172.21.15.16:0/4100024283 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f573808a930 con 0x7f57440d4130 2024-09-19T07:44:55.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.860+0000 7f5749de8640 1 --2- 172.21.15.16:0/4100024283 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f573006c2e0 0x7f573006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:55.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.861+0000 7f5749de8640 1 --2- 172.21.15.16:0/4100024283 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f573006c2e0 0x7f573006e7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5734009fd0 tx=0x7f573400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:55.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:55.865+0000 7f57427fc640 1 -- 172.21.15.16:0/4100024283 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f57440d4b20 con 0x7f57440d4130 2024-09-19T07:44:56.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.076+0000 7f574c04c640 1 -- 172.21.15.16:0/4100024283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f574406cf30 con 0x7f57440d4130 2024-09-19T07:44:56.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.076+0000 7f57427fc640 1 -- 172.21.15.16:0/4100024283 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5229 (secure 0 0 0) 0x7f57380588a0 con 0x7f57440d4130 2024-09-19T07:44:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:44:56.078 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-19T07:44:04:108666+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4655,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/3238551814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3238551814},{"type":"v1","addr":"172.21.15.146:6836","nonce":3238551814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3450384245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3450384245},{"type":"v1","addr":"172.21.15.146:6837","nonce":3450384245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4944,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3663416804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3663416804},{"type":"v1","addr":"172.21.15.16:6834","nonce":3663416804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:44:00.970677+0000","modified":"2024-09-19T07:44:03.455740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4947":{"gid":4947,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2028210099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2028210099},{"type":"v1","addr":"172.21.15.16:6835","nonce":2028210099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4947,"qdb_cluster":[4947]},"id":2}]} 2024-09-19T07:44:56.078 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-19T07:44:56.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.078+0000 7f574c04c640 1 -- 172.21.15.16:0/4100024283 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f573006c2e0 msgr2=0x7f573006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:56.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.078+0000 7f574c04c640 1 --2- 172.21.15.16:0/4100024283 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f573006c2e0 0x7f573006e7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5734009fd0 tx=0x7f573400b040 comp rx=0 tx=0).stop 2024-09-19T07:44:56.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.078+0000 7f574c04c640 1 -- 172.21.15.16:0/4100024283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57440d4130 msgr2=0x7f57440ca820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:56.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.078+0000 7f574c04c640 1 --2- 172.21.15.16:0/4100024283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57440d4130 0x7f57440ca820 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f573800c440 tx=0x7f5738001e70 comp rx=0 tx=0).stop 2024-09-19T07:44:56.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.078+0000 7f574c04c640 1 -- 172.21.15.16:0/4100024283 shutdown_connections 2024-09-19T07:44:56.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.078+0000 7f574c04c640 1 --2- 172.21.15.16:0/4100024283 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f573006c2e0 0x7f573006e7a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.078+0000 7f574c04c640 1 --2- 172.21.15.16:0/4100024283 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f574415f0d0 0x7f57440b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.078+0000 7f574c04c640 1 --2- 172.21.15.16:0/4100024283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57440d4130 0x7f57440ca820 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.078+0000 7f574c04c640 1 --2- 172.21.15.16:0/4100024283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57440d0070 0x7f57440cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.078+0000 7f574c04c640 1 -- 172.21.15.16:0/4100024283 >> 172.21.15.16:0/4100024283 conn(0x7f574409d760 msgr2=0x7f574408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:56.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.078+0000 7f574c04c640 1 -- 172.21.15.16:0/4100024283 shutdown_connections 2024-09-19T07:44:56.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.078+0000 7f574c04c640 1 -- 172.21.15.16:0/4100024283 wait complete. 2024-09-19T07:44:56.091 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-19T07:44:56.212 INFO:teuthology.orchestra.run.smithi016.stdout:[{"id":4975,"entity":{"name":{"type":"client","num":4975},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":613476257}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":42.403414388000002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.7247719840189677,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.4975 192.168.0.1:0/613476257","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"44154","root":"/"}},{"id":5151,"entity":{"name":{"type":"client","num":5151},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2625487776}},"state":"open","num_leases":0,"num_caps":100,"request_load_avg":81,"uptime":43.937393202999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":3.1652149644810907,"halflife":60},"release_caps":{"value":3.2021413061594735,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":296.2188709187634,"halflife":300},"cap_acquisition":{"value":60.456084518156125,"halflife":30},"last_trim_completed_requests_tid":129,"last_trim_completed_flushes_tid":101,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.5151 192.168.0.1:0/2625487776","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000025c","length":899}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"0","hostname":"smithi016","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"48356","root":"/"}}] 2024-09-19T07:44:56.212 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "44154", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 613476257, "type": "any" }, "name": { "num": 4975, "type": "client" } }, "id": 4975, "inst": "client.4975 192.168.0.1:0/613476257", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.7247719840189677 }, "state": "open", "uptime": 42.403414388 }, { "cap_acquisition": { "halflife": 30, "value": 60.456084518156125 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "0", "hostname": "smithi016", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "48356", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2625487776, "type": "any" }, "name": { "num": 5151, "type": "client" } }, "id": 5151, "inst": "client.5151 192.168.0.1:0/2625487776", "last_trim_completed_flushes_tid": 101, "last_trim_completed_requests_tid": 129, "num_caps": 100, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 899, "start": "0x1000000025c" } ], "recall_caps": { "halflife": 60, "value": 3.1652149644810907 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 3.2021413061594735 }, "request_load_avg": 81, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 296.2188709187634 }, "state": "open", "uptime": 43.937393203 } ] 2024-09-19T07:44:56.213 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump_historic_ops 2024-09-19T07:44:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.314+0000 7f223c850640 1 Processor -- start 2024-09-19T07:44:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.314+0000 7f223c850640 1 -- start start 2024-09-19T07:44:56.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f223c850640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f22380c8d20 0x7f223806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f223c850640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f223806d260 0x7f22381439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f223c850640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2238143f30 0x7f22380d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f223c850640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f223806d6d0 con 0x7f223806d260 2024-09-19T07:44:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f223c850640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2238141960 con 0x7f22380c8d20 2024-09-19T07:44:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f223c850640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2238142200 con 0x7f2238143f30 2024-09-19T07:44:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f2235d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f223806d260 0x7f22381439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f2235d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f223806d260 0x7f22381439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60182/0 (socket says 172.21.15.16:60182) 2024-09-19T07:44:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f2235d9b640 1 -- 172.21.15.16:0/3000784392 learned_addr learned my addr 172.21.15.16:0/3000784392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f2235d9b640 1 -- 172.21.15.16:0/3000784392 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2238143f30 msgr2=0x7f22380d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f223659c640 1 --2- 172.21.15.16:0/3000784392 >> v2:172.21.15.146:3300/0 conn(0x7f22380c8d20 0x7f223806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f2236d9d640 1 --2- 172.21.15.16:0/3000784392 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2238143f30 0x7f22380d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f2235d9b640 1 --2- 172.21.15.16:0/3000784392 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2238143f30 0x7f22380d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f2235d9b640 1 -- 172.21.15.16:0/3000784392 >> v2:172.21.15.146:3300/0 conn(0x7f22380c8d20 msgr2=0x7f223806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f2235d9b640 1 --2- 172.21.15.16:0/3000784392 >> v2:172.21.15.146:3300/0 conn(0x7f22380c8d20 0x7f223806cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f2235d9b640 1 -- 172.21.15.16:0/3000784392 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22380c7140 con 0x7f223806d260 2024-09-19T07:44:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f223659c640 1 --2- 172.21.15.16:0/3000784392 >> v2:172.21.15.146:3300/0 conn(0x7f22380c8d20 0x7f223806cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:44:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f2236d9d640 1 --2- 172.21.15.16:0/3000784392 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2238143f30 0x7f22380d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:44:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f2235d9b640 1 --2- 172.21.15.16:0/3000784392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f223806d260 0x7f22381439f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f22380d2e50 tx=0x7f222002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf4c841f73d600f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f223559a640 1 -- 172.21.15.16:0/3000784392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220037070 con 0x7f223806d260 2024-09-19T07:44:56.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f223559a640 1 -- 172.21.15.16:0/3000784392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f222002ae00 con 0x7f223806d260 2024-09-19T07:44:56.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.316+0000 7f223559a640 1 -- 172.21.15.16:0/3000784392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220032660 con 0x7f223806d260 2024-09-19T07:44:56.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.317+0000 7f223c850640 1 -- 172.21.15.16:0/3000784392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f223806d260 msgr2=0x7f22381439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:56.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.317+0000 7f223c850640 1 --2- 172.21.15.16:0/3000784392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f223806d260 0x7f22381439f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f22380d2e50 tx=0x7f222002a0b0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.317+0000 7f223c850640 1 -- 172.21.15.16:0/3000784392 shutdown_connections 2024-09-19T07:44:56.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.317+0000 7f223c850640 1 --2- 172.21.15.16:0/3000784392 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2238143f30 0x7f22380d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.317+0000 7f223c850640 1 --2- 172.21.15.16:0/3000784392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f223806d260 0x7f22381439f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.317+0000 7f223c850640 1 --2- 172.21.15.16:0/3000784392 >> v2:172.21.15.146:3300/0 conn(0x7f22380c8d20 0x7f223806cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.317+0000 7f223c850640 1 -- 172.21.15.16:0/3000784392 >> 172.21.15.16:0/3000784392 conn(0x7f22380b3ed0 msgr2=0x7f22380b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:56.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.317+0000 7f223c850640 1 -- 172.21.15.16:0/3000784392 shutdown_connections 2024-09-19T07:44:56.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.317+0000 7f223c850640 1 -- 172.21.15.16:0/3000784392 wait complete. 2024-09-19T07:44:56.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.318+0000 7f223c850640 1 Processor -- start 2024-09-19T07:44:56.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.318+0000 7f223c850640 1 -- start start 2024-09-19T07:44:56.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.319+0000 7f223c850640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f223806d260 0x7f2238140d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.319+0000 7f223c850640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22380c8d20 0x7f22380c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.319+0000 7f223c850640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2238143f30 0x7f22380b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.319+0000 7f223c850640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2238141bb0 con 0x7f22380c8d20 2024-09-19T07:44:56.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.319+0000 7f223c850640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2238144760 con 0x7f223806d260 2024-09-19T07:44:56.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.319+0000 7f223c850640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2238141e90 con 0x7f2238143f30 2024-09-19T07:44:56.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.319+0000 7f2235d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22380c8d20 0x7f22380c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.319+0000 7f2235d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22380c8d20 0x7f22380c9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60196/0 (socket says 172.21.15.16:60196) 2024-09-19T07:44:56.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.319+0000 7f2235d9b640 1 -- 172.21.15.16:0/1667547635 learned_addr learned my addr 172.21.15.16:0/1667547635 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:56.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.319+0000 7f2236d9d640 1 --2- 172.21.15.16:0/1667547635 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2238143f30 0x7f22380b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.319+0000 7f223659c640 1 --2- 172.21.15.16:0/1667547635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f223806d260 0x7f2238140d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.320+0000 7f2235d9b640 1 -- 172.21.15.16:0/1667547635 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2238143f30 msgr2=0x7f22380b73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:56.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.320+0000 7f2235d9b640 1 --2- 172.21.15.16:0/1667547635 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2238143f30 0x7f22380b73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.320+0000 7f2235d9b640 1 -- 172.21.15.16:0/1667547635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f223806d260 msgr2=0x7f2238140d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:56.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.320+0000 7f2235d9b640 1 --2- 172.21.15.16:0/1667547635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f223806d260 0x7f2238140d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.320+0000 7f2235d9b640 1 -- 172.21.15.16:0/1667547635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22380d0a20 con 0x7f22380c8d20 2024-09-19T07:44:56.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.320+0000 7f223659c640 1 --2- 172.21.15.16:0/1667547635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f223806d260 0x7f2238140d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:56.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.320+0000 7f2235d9b640 1 --2- 172.21.15.16:0/1667547635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22380c8d20 0x7f22380c9330 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f222002a0f0 tx=0x7f22200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:56.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.320+0000 7f222effd640 1 -- 172.21.15.16:0/1667547635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220041030 con 0x7f22380c8d20 2024-09-19T07:44:56.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.320+0000 7f222effd640 1 -- 172.21.15.16:0/1667547635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f222003c670 con 0x7f22380c8d20 2024-09-19T07:44:56.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.320+0000 7f223c850640 1 -- 172.21.15.16:0/1667547635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22380d0d00 con 0x7f22380c8d20 2024-09-19T07:44:56.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.320+0000 7f222effd640 1 -- 172.21.15.16:0/1667547635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220037040 con 0x7f22380c8d20 2024-09-19T07:44:56.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.320+0000 7f223c850640 1 -- 172.21.15.16:0/1667547635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22380d17b0 con 0x7f22380c8d20 2024-09-19T07:44:56.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.322+0000 7f222effd640 1 -- 172.21.15.16:0/1667547635 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2220044070 con 0x7f22380c8d20 2024-09-19T07:44:56.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.323+0000 7f222effd640 1 --2- 172.21.15.16:0/1667547635 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f221806c2b0 0x7f221806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.323+0000 7f222effd640 1 -- 172.21.15.16:0/1667547635 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f22200ab9c0 con 0x7f22380c8d20 2024-09-19T07:44:56.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.323+0000 7f223659c640 1 --2- 172.21.15.16:0/1667547635 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f221806c2b0 0x7f221806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.323+0000 7f223f5e2740 1 Processor -- start 2024-09-19T07:44:56.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.323+0000 7f223659c640 1 --2- 172.21.15.16:0/1667547635 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f221806c2b0 0x7f221806e770 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f222800e490 tx=0x7f22280075f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:56.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.324+0000 7f223f5e2740 1 -- start start 2024-09-19T07:44:56.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.324+0000 7f223f5e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1d540 0x55a450c62ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.324+0000 7f223f5e2740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55a450c1ad00 0x55a450c1d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.324+0000 7f223f5e2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a450c1ecd0 0x55a450c17ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.324+0000 7f223f5e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a450b805e0 con 0x55a450c1d540 2024-09-19T07:44:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.324+0000 7f223f5e2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55a450b28400 con 0x55a450c1ecd0 2024-09-19T07:44:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.324+0000 7f223f5e2740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55a450b7c5a0 con 0x55a450c1ad00 2024-09-19T07:44:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.324+0000 7f223659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1d540 0x55a450c62ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.324+0000 7f223659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1d540 0x55a450c62ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60212/0 (socket says 172.21.15.16:60212) 2024-09-19T07:44:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.324+0000 7f223659c640 1 -- 172.21.15.16:0/406407796 learned_addr learned my addr 172.21.15.16:0/406407796 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.324+0000 7f2235d9b640 1 --2- 172.21.15.16:0/406407796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55a450c1ad00 0x55a450c1d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.324+0000 7f2236d9d640 1 --2- 172.21.15.16:0/406407796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a450c1ecd0 0x55a450c17ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.325+0000 7f223659c640 1 -- 172.21.15.16:0/406407796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55a450c1ad00 msgr2=0x55a450c1d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.325+0000 7f223659c640 1 --2- 172.21.15.16:0/406407796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55a450c1ad00 0x55a450c1d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.325+0000 7f223659c640 1 -- 172.21.15.16:0/406407796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a450c1ecd0 msgr2=0x55a450c17ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.325+0000 7f223659c640 1 --2- 172.21.15.16:0/406407796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a450c1ecd0 0x55a450c17ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.325+0000 7f223659c640 1 -- 172.21.15.16:0/406407796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a450b4de90 con 0x55a450c1d540 2024-09-19T07:44:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.325+0000 7f223659c640 1 --2- 172.21.15.16:0/406407796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1d540 0x55a450c62ce0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f222800f900 tx=0x7f2228011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf8542bf2dc3eae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.325+0000 7f2217fff640 1 -- 172.21.15.16:0/406407796 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222801acd0 con 0x55a450c1d540 2024-09-19T07:44:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.325+0000 7f2217fff640 1 -- 172.21.15.16:0/406407796 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f222801ae70 con 0x55a450c1d540 2024-09-19T07:44:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.325+0000 7f2217fff640 1 -- 172.21.15.16:0/406407796 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2228025680 con 0x55a450c1d540 2024-09-19T07:44:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.326+0000 7f223f5e2740 1 -- 172.21.15.16:0/406407796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1d540 msgr2=0x55a450c62ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.326+0000 7f223f5e2740 1 --2- 172.21.15.16:0/406407796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1d540 0x55a450c62ce0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f222800f900 tx=0x7f2228011000 comp rx=0 tx=0).stop 2024-09-19T07:44:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.326+0000 7f223f5e2740 1 -- 172.21.15.16:0/406407796 shutdown_connections 2024-09-19T07:44:56.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.326+0000 7f223f5e2740 1 --2- 172.21.15.16:0/406407796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a450c1ecd0 0x55a450c17ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.326+0000 7f223f5e2740 1 --2- 172.21.15.16:0/406407796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55a450c1ad00 0x55a450c1d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.326+0000 7f223f5e2740 1 --2- 172.21.15.16:0/406407796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1d540 0x55a450c62ce0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.326+0000 7f223f5e2740 1 -- 172.21.15.16:0/406407796 >> 172.21.15.16:0/406407796 conn(0x55a450c19d00 msgr2=0x55a450c621b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:56.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.326+0000 7f223f5e2740 1 -- 172.21.15.16:0/406407796 shutdown_connections 2024-09-19T07:44:56.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.326+0000 7f223f5e2740 1 -- 172.21.15.16:0/406407796 wait complete. 2024-09-19T07:44:56.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.326+0000 7f223f5e2740 1 -- start start 2024-09-19T07:44:56.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.326+0000 7f223f5e2740 1 Processor -- start 2024-09-19T07:44:56.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.327+0000 7f223f5e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1d540 0x55a450c5f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.327+0000 7f223f5e2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a450c1ecd0 0x55a450c3d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.327+0000 7f223f5e2740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55a450c46b80 0x55a450c72950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.327+0000 7f223f5e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a450c19150 con 0x55a450c1d540 2024-09-19T07:44:56.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.327+0000 7f223f5e2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55a450b09350 con 0x55a450c1ecd0 2024-09-19T07:44:56.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.327+0000 7f223f5e2740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55a450c63510 con 0x55a450c46b80 2024-09-19T07:44:56.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.327+0000 7f223f5e2740 10 client.0 _ll_register_callbacks cb 0x55a450b2d3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:56.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.327+0000 7f223659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1d540 0x55a450c5f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.327+0000 7f223659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1d540 0x55a450c5f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60220/0 (socket says 172.21.15.16:60220) 2024-09-19T07:44:56.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.327+0000 7f223659c640 1 -- 172.21.15.16:0/3948589867 learned_addr learned my addr 172.21.15.16:0/3948589867 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:56.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.328+0000 7f2236d9d640 1 --2- 172.21.15.16:0/3948589867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55a450c46b80 0x55a450c72950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.328+0000 7f2235d9b640 1 --2- 172.21.15.16:0/3948589867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a450c1ecd0 0x55a450c3d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.328+0000 7f223659c640 1 -- 172.21.15.16:0/3948589867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55a450c46b80 msgr2=0x55a450c72950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:56.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.328+0000 7f223659c640 1 --2- 172.21.15.16:0/3948589867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55a450c46b80 0x55a450c72950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.328+0000 7f223659c640 1 -- 172.21.15.16:0/3948589867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a450c1ecd0 msgr2=0x55a450c3d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:56.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.328+0000 7f223659c640 1 --2- 172.21.15.16:0/3948589867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a450c1ecd0 0x55a450c3d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.328+0000 7f223659c640 1 -- 172.21.15.16:0/3948589867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4508933f0 con 0x55a450c1d540 2024-09-19T07:44:56.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.328+0000 7f223659c640 1 --2- 172.21.15.16:0/3948589867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1d540 0x55a450c5f8c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f222800f870 tx=0x7f2228011070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f203ce51e7b7cd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:56.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.328+0000 7f22157fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:44:56.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.328+0000 7f22157fa640 1 -- 172.21.15.16:0/3948589867 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2228025950 con 0x55a450c1d540 2024-09-19T07:44:56.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.328+0000 7f22157fa640 1 -- 172.21.15.16:0/3948589867 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f2228025af0 con 0x55a450c1d540 2024-09-19T07:44:56.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.328+0000 7f223f5e2740 1 -- 172.21.15.16:0/3948589867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f222801ae70 con 0x55a450c1d540 2024-09-19T07:44:56.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.328+0000 7f22157fa640 1 -- 172.21.15.16:0/3948589867 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f2228025cf0 con 0x55a450c1d540 2024-09-19T07:44:56.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.328+0000 7f223f5e2740 10 client.5196 fetch_fsmap learned FSMap version 16 2024-09-19T07:44:56.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.328+0000 7f223f5e2740 1 -- 172.21.15.16:0/3948589867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55a45095b760 con 0x55a450c1d540 2024-09-19T07:44:56.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.328+0000 7f22157fa640 1 -- 172.21.15.16:0/3948589867 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f222801ae70 con 0x55a450c1d540 2024-09-19T07:44:56.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.329+0000 7f22157fa640 1 -- 172.21.15.16:0/3948589867 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 16) ==== 3502+0+0 (secure 0 0 0) 0x7f222800c050 con 0x55a450c1d540 2024-09-19T07:44:56.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.329+0000 7f223f5e2740 10 client.5196 fetch_fsmap finished waiting for FSMap version 16 2024-09-19T07:44:56.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.329+0000 7f223f5e2740 10 client.5196 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-09-19T07:44:56.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.329+0000 7f223f5e2740 20 client.5196 populate_metadata read hostname 'smithi016' 2024-09-19T07:44:56.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.329+0000 7f223f5e2740 1 --2- 172.21.15.16:0/3948589867 >> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] conn(0x55a450c72e90 0x55a450c43a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.329+0000 7f223f5e2740 4 client.5196 mds_command: new command op to 4947 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:44:56.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.329+0000 7f223f5e2740 1 -- 172.21.15.16:0/3948589867 --> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a450b8dd10 con 0x55a450c72e90 2024-09-19T07:44:56.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.329+0000 7f2235d9b640 1 --2- 172.21.15.16:0/3948589867 >> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] conn(0x55a450c72e90 0x55a450c43a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.329+0000 7f22157fa640 10 client.5196 ms_handle_connect on v2:172.21.15.16:6833/2028210099 2024-09-19T07:44:56.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.329+0000 7f2235d9b640 1 --2- 172.21.15.16:0/3948589867 >> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] conn(0x55a450c72e90 0x55a450c43a40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7300e85cb5631f4 server_cookie=42df2ecb0976b2df in_seq=0 out_seq=0 2024-09-19T07:44:56.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.330+0000 7f22157fa640 1 -- 172.21.15.16:0/3948589867 <== mds.0 v2:172.21.15.16:6833/2028210099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a450b8dd10 con 0x55a450c72e90 2024-09-19T07:44:56.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.330+0000 7f22157fa640 10 client.5196 handle_command_reply: tid=0 2024-09-19T07:44:56.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.330+0000 7f223f5e2740 1 client.5196 shutdown 2024-09-19T07:44:56.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.330+0000 7f223f5e2740 1 -- 172.21.15.16:0/3948589867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1d540 msgr2=0x55a450c5f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:56.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.330+0000 7f223f5e2740 1 --2- 172.21.15.16:0/3948589867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1d540 0x55a450c5f8c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f222800f870 tx=0x7f2228011070 comp rx=0 tx=0).stop 2024-09-19T07:44:56.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.330+0000 7f223f5e2740 1 -- 172.21.15.16:0/3948589867 shutdown_connections 2024-09-19T07:44:56.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.330+0000 7f223f5e2740 1 --2- 172.21.15.16:0/3948589867 >> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] conn(0x55a450c72e90 0x55a450c43a40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.330+0000 7f223f5e2740 1 --2- 172.21.15.16:0/3948589867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55a450c46b80 0x55a450c72950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.330+0000 7f223f5e2740 1 --2- 172.21.15.16:0/3948589867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a450c1ecd0 0x55a450c3d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.330+0000 7f223f5e2740 1 --2- 172.21.15.16:0/3948589867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1d540 0x55a450c5f8c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.330+0000 7f22157fa640 0 client.5196 ms_handle_reset on v2:172.21.15.16:6833/2028210099 2024-09-19T07:44:56.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.330+0000 7f223f5e2740 1 -- 172.21.15.16:0/3948589867 >> 172.21.15.16:0/3948589867 conn(0x55a450c1ad00 msgr2=0x55a450c617b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:56.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.330+0000 7f223f5e2740 1 -- 172.21.15.16:0/3948589867 shutdown_connections 2024-09-19T07:44:56.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.330+0000 7f223f5e2740 1 -- 172.21.15.16:0/3948589867 wait complete. 2024-09-19T07:44:56.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.330+0000 7f223f5e2740 20 client.5196 trim_cache size 0 max 16384 2024-09-19T07:44:56.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.344+0000 7f223f5e2740 1 Processor -- start 2024-09-19T07:44:56.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.344+0000 7f223f5e2740 1 -- start start 2024-09-19T07:44:56.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.345+0000 7f223f5e2740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55a450c1d540 0x55a450c86cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.345+0000 7f223f5e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1ecd0 0x55a450c64080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.345+0000 7f223f5e2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a450c46b80 0x55a450c620e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.345+0000 7f223f5e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2238200fe0 con 0x55a450c1ecd0 2024-09-19T07:44:56.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.345+0000 7f223f5e2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f22381ffff0 con 0x55a450c46b80 2024-09-19T07:44:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.345+0000 7f223f5e2740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f22381ff330 con 0x55a450c1d540 2024-09-19T07:44:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.345+0000 7f2235d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1ecd0 0x55a450c64080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.345+0000 7f2235d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1ecd0 0x55a450c64080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60226/0 (socket says 172.21.15.16:60226) 2024-09-19T07:44:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.345+0000 7f2235d9b640 1 -- 172.21.15.16:0/652203744 learned_addr learned my addr 172.21.15.16:0/652203744 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.345+0000 7f2235d9b640 1 -- 172.21.15.16:0/652203744 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55a450c1d540 msgr2=0x55a450c86cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.345+0000 7f2235d9b640 1 --2- 172.21.15.16:0/652203744 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55a450c1d540 0x55a450c86cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.345+0000 7f2235d9b640 1 -- 172.21.15.16:0/652203744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a450c46b80 msgr2=0x55a450c620e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.345+0000 7f2235d9b640 1 --2- 172.21.15.16:0/652203744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a450c46b80 0x55a450c620e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.345+0000 7f2235d9b640 1 -- 172.21.15.16:0/652203744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a450c466c0 con 0x55a450c1ecd0 2024-09-19T07:44:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.346+0000 7f2235d9b640 1 --2- 172.21.15.16:0/652203744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1ecd0 0x55a450c64080 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f22200a9d40 tx=0x7f2220080040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44dc3d509b0e0a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.346+0000 7f22177fe640 1 -- 172.21.15.16:0/652203744 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220091020 con 0x55a450c1ecd0 2024-09-19T07:44:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.346+0000 7f22177fe640 1 -- 172.21.15.16:0/652203744 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f222007dc30 con 0x55a450c1ecd0 2024-09-19T07:44:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.346+0000 7f22177fe640 1 -- 172.21.15.16:0/652203744 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220082460 con 0x55a450c1ecd0 2024-09-19T07:44:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.346+0000 7f223f5e2740 1 -- 172.21.15.16:0/652203744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1ecd0 msgr2=0x55a450c64080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.346+0000 7f223f5e2740 1 --2- 172.21.15.16:0/652203744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1ecd0 0x55a450c64080 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f22200a9d40 tx=0x7f2220080040 comp rx=0 tx=0).stop 2024-09-19T07:44:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.346+0000 7f223f5e2740 1 -- 172.21.15.16:0/652203744 shutdown_connections 2024-09-19T07:44:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.346+0000 7f223f5e2740 1 --2- 172.21.15.16:0/652203744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a450c46b80 0x55a450c620e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.346+0000 7f223f5e2740 1 --2- 172.21.15.16:0/652203744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c1ecd0 0x55a450c64080 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.346+0000 7f223f5e2740 1 --2- 172.21.15.16:0/652203744 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55a450c1d540 0x55a450c86cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.346+0000 7f223f5e2740 1 -- 172.21.15.16:0/652203744 >> 172.21.15.16:0/652203744 conn(0x55a450c1ad00 msgr2=0x55a450c13050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.346+0000 7f223f5e2740 1 -- 172.21.15.16:0/652203744 shutdown_connections 2024-09-19T07:44:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.346+0000 7f223f5e2740 1 -- 172.21.15.16:0/652203744 wait complete. 2024-09-19T07:44:56.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.347+0000 7f223f5e2740 1 -- start start 2024-09-19T07:44:56.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.347+0000 7f223f5e2740 1 Processor -- start 2024-09-19T07:44:56.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.348+0000 7f223f5e2740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55a450c1d540 0x55a450ce06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.348+0000 7f223f5e2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a450c1ecd0 0x55a450c853d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.348+0000 7f223f5e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c46b80 0x55a450c85910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.348+0000 7f223f5e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a450c5fe40 con 0x55a450c46b80 2024-09-19T07:44:56.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.348+0000 7f223f5e2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f22381ff980 con 0x55a450c1ecd0 2024-09-19T07:44:56.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.348+0000 7f223f5e2740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55a450c3c6e0 con 0x55a450c1d540 2024-09-19T07:44:56.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.348+0000 7f223f5e2740 10 client.0 _ll_register_callbacks cb 0x7f222800f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:56.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.348+0000 7f2236d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c46b80 0x55a450c85910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.348+0000 7f2236d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c46b80 0x55a450c85910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60240/0 (socket says 172.21.15.16:60240) 2024-09-19T07:44:56.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.348+0000 7f2236d9d640 1 -- 172.21.15.16:0/1691888852 learned_addr learned my addr 172.21.15.16:0/1691888852 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:56.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.348+0000 7f2235d9b640 1 --2- 172.21.15.16:0/1691888852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a450c1ecd0 0x55a450c853d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.348+0000 7f223659c640 1 --2- 172.21.15.16:0/1691888852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55a450c1d540 0x55a450ce06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.348+0000 7f2236d9d640 1 -- 172.21.15.16:0/1691888852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55a450c1d540 msgr2=0x55a450ce06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:56.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.348+0000 7f2236d9d640 1 --2- 172.21.15.16:0/1691888852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55a450c1d540 0x55a450ce06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.348+0000 7f2236d9d640 1 -- 172.21.15.16:0/1691888852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a450c1ecd0 msgr2=0x55a450c853d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:56.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.348+0000 7f2236d9d640 1 --2- 172.21.15.16:0/1691888852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a450c1ecd0 0x55a450c853d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.348+0000 7f2236d9d640 1 -- 172.21.15.16:0/1691888852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a450c3c120 con 0x55a450c46b80 2024-09-19T07:44:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.348+0000 7f2236d9d640 1 --2- 172.21.15.16:0/1691888852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c46b80 0x55a450c85910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f22300145d0 tx=0x7f2230014a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1fe46d871c70b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.348+0000 7f22157fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:44:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.349+0000 7f22157fa640 1 -- 172.21.15.16:0/1691888852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230016900 con 0x55a450c46b80 2024-09-19T07:44:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.349+0000 7f22157fa640 1 -- 172.21.15.16:0/1691888852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f2230016aa0 con 0x55a450c46b80 2024-09-19T07:44:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.349+0000 7f223f5e2740 1 -- 172.21.15.16:0/1691888852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f222007dc30 con 0x55a450c46b80 2024-09-19T07:44:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.349+0000 7f22157fa640 1 -- 172.21.15.16:0/1691888852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f2230027050 con 0x55a450c46b80 2024-09-19T07:44:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.349+0000 7f223f5e2740 10 client.5202 fetch_fsmap learned FSMap version 16 2024-09-19T07:44:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.349+0000 7f223f5e2740 1 -- 172.21.15.16:0/1691888852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55a450ce0c50 con 0x55a450c46b80 2024-09-19T07:44:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.349+0000 7f22157fa640 1 -- 172.21.15.16:0/1691888852 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f222007dc30 con 0x55a450c46b80 2024-09-19T07:44:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.349+0000 7f22157fa640 1 -- 172.21.15.16:0/1691888852 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 16) ==== 3502+0+0 (secure 0 0 0) 0x7f223002a020 con 0x55a450c46b80 2024-09-19T07:44:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.349+0000 7f223f5e2740 10 client.5202 fetch_fsmap finished waiting for FSMap version 16 2024-09-19T07:44:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.349+0000 7f223f5e2740 10 client.5202 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-09-19T07:44:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.349+0000 7f223f5e2740 20 client.5202 populate_metadata read hostname 'smithi016' 2024-09-19T07:44:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.349+0000 7f223f5e2740 1 --2- 172.21.15.16:0/1691888852 >> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] conn(0x55a450c85e50 0x55a450c42340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.349+0000 7f223f5e2740 4 client.5202 mds_command: new command op to 4947 tid=0 multi_id=0 [{"prefix": "dump_historic_ops"}] 2024-09-19T07:44:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.349+0000 7f223f5e2740 1 -- 172.21.15.16:0/1691888852 --> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] -- command(tid 0: {"prefix": "dump_historic_ops"}) -- 0x55a450b88060 con 0x55a450c85e50 2024-09-19T07:44:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.349+0000 7f223659c640 1 --2- 172.21.15.16:0/1691888852 >> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] conn(0x55a450c85e50 0x55a450c42340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.350+0000 7f223659c640 1 --2- 172.21.15.16:0/1691888852 >> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] conn(0x55a450c85e50 0x55a450c42340 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbb084be94f6e9ed server_cookie=4ca7d9438cb3ad3d in_seq=0 out_seq=0 2024-09-19T07:44:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.350+0000 7f22157fa640 10 client.5202 ms_handle_connect on v2:172.21.15.16:6833/2028210099 2024-09-19T07:44:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.351+0000 7f22157fa640 1 -- 172.21.15.16:0/1691888852 <== mds.0 v2:172.21.15.16:6833/2028210099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+71741 (crc 0 0 0) 0x55a450b88060 con 0x55a450c85e50 2024-09-19T07:44:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.351+0000 7f22157fa640 10 client.5202 handle_command_reply: tid=0 2024-09-19T07:44:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.351+0000 7f223f5e2740 1 client.5202 shutdown 2024-09-19T07:44:56.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.351+0000 7f223f5e2740 1 -- 172.21.15.16:0/1691888852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c46b80 msgr2=0x55a450c85910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:56.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.351+0000 7f223f5e2740 1 --2- 172.21.15.16:0/1691888852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c46b80 0x55a450c85910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f22300145d0 tx=0x7f2230014a90 comp rx=0 tx=0).stop 2024-09-19T07:44:56.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.351+0000 7f223f5e2740 1 -- 172.21.15.16:0/1691888852 shutdown_connections 2024-09-19T07:44:56.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.351+0000 7f223f5e2740 1 --2- 172.21.15.16:0/1691888852 >> [v2:172.21.15.16:6833/2028210099,v1:172.21.15.16:6835/2028210099] conn(0x55a450c85e50 0x55a450c42340 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.351+0000 7f223f5e2740 1 --2- 172.21.15.16:0/1691888852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a450c46b80 0x55a450c85910 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.351+0000 7f223f5e2740 1 --2- 172.21.15.16:0/1691888852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a450c1ecd0 0x55a450c853d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.351+0000 7f223f5e2740 1 --2- 172.21.15.16:0/1691888852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55a450c1d540 0x55a450ce06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.351+0000 7f22157fa640 0 client.5202 ms_handle_reset on v2:172.21.15.16:6833/2028210099 2024-09-19T07:44:56.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.351+0000 7f223f5e2740 1 -- 172.21.15.16:0/1691888852 >> 172.21.15.16:0/1691888852 conn(0x55a450c1ad00 msgr2=0x55a450c61ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:56.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.351+0000 7f223f5e2740 1 -- 172.21.15.16:0/1691888852 shutdown_connections 2024-09-19T07:44:56.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.351+0000 7f223f5e2740 1 -- 172.21.15.16:0/1691888852 wait complete. 2024-09-19T07:44:56.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.351+0000 7f223f5e2740 20 client.5202 trim_cache size 0 max 16384 2024-09-19T07:44:56.360 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:44:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "size": 20, 2024-09-19T07:44:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 600, 2024-09-19T07:44:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "ops": [ 2024-09-19T07:44:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:2 setattr mode=041777 #0x1 2024-09-19T07:44:12.640770+0000 caller_uid=0, caller_gid=0{0,})", 2024-09-19T07:44:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:12.640880+0000", 2024-09-19T07:44:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 43.711175793999999, 2024-09-19T07:44:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 1.1607073409999999, 2024-09-19T07:44:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 2 2024-09-19T07:44:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:12.640880+0000", 2024-09-19T07:44:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:12.640880+0000", 2024-09-19T07:44:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:12.640881+0000", 2024-09-19T07:44:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:12.640892+0000", 2024-09-19T07:44:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:12.640932+0000", 2024-09-19T07:44:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:12.641180+0000", 2024-09-19T07:44:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:12.641283+0000", 2024-09-19T07:44:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-19T07:44:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:12.641284+0000", 2024-09-19T07:44:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-19T07:44:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:13.801430+0000", 2024-09-19T07:44:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-19T07:44:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:13.801481+0000", 2024-09-19T07:44:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:13.801566+0000", 2024-09-19T07:44:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:13.801587+0000", 2024-09-19T07:44:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:13.801587+0000", 2024-09-19T07:44:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:5 create owner_uid=1000, owner_gid=1333 #0x10000000000/file_0 2024-09-19T07:44:14.933780+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})", 2024-09-19T07:44:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:14.933857+0000", 2024-09-19T07:44:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.418198306999997, 2024-09-19T07:44:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0054419179999999996, 2024-09-19T07:44:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 5 2024-09-19T07:44:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.933857+0000", 2024-09-19T07:44:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.933857+0000", 2024-09-19T07:44:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.933858+0000", 2024-09-19T07:44:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.933863+0000", 2024-09-19T07:44:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.933884+0000", 2024-09-19T07:44:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.934149+0000", 2024-09-19T07:44:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "failed to rdlock, waiting" 2024-09-19T07:44:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.934314+0000", 2024-09-19T07:44:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.934580+0000", 2024-09-19T07:44:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-19T07:44:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.934580+0000", 2024-09-19T07:44:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-19T07:44:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.938937+0000", 2024-09-19T07:44:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-19T07:44:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.939068+0000", 2024-09-19T07:44:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.939178+0000", 2024-09-19T07:44:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.939298+0000", 2024-09-19T07:44:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.939299+0000", 2024-09-19T07:44:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:16 create owner_uid=1000, owner_gid=1333 #0x10000000000/file_11 2024-09-19T07:44:14.980390+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})", 2024-09-19T07:44:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:14.980460+0000", 2024-09-19T07:44:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.371595943999999, 2024-09-19T07:44:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.005461568, 2024-09-19T07:44:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 16 2024-09-19T07:44:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.980460+0000", 2024-09-19T07:44:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.980460+0000", 2024-09-19T07:44:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.980460+0000", 2024-09-19T07:44:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.980465+0000", 2024-09-19T07:44:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.980484+0000", 2024-09-19T07:44:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.980870+0000", 2024-09-19T07:44:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.981449+0000", 2024-09-19T07:44:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-19T07:44:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.981450+0000", 2024-09-19T07:44:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-19T07:44:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.985554+0000", 2024-09-19T07:44:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-19T07:44:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.985689+0000", 2024-09-19T07:44:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.985792+0000", 2024-09-19T07:44:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.985918+0000", 2024-09-19T07:44:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:14.985921+0000", 2024-09-19T07:44:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:32 mkdir owner_uid=1000, owner_gid=1333 #0x1/dir2 2024-09-19T07:44:15.216682+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})", 2024-09-19T07:44:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:15.216756+0000", 2024-09-19T07:44:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.135299844999999, 2024-09-19T07:44:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.005166454, 2024-09-19T07:44:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 32 2024-09-19T07:44:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.216756+0000", 2024-09-19T07:44:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.216756+0000", 2024-09-19T07:44:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.216756+0000", 2024-09-19T07:44:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.216761+0000", 2024-09-19T07:44:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.216776+0000", 2024-09-19T07:44:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.217045+0000", 2024-09-19T07:44:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.217312+0000", 2024-09-19T07:44:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-19T07:44:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.217313+0000", 2024-09-19T07:44:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-19T07:44:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.221672+0000", 2024-09-19T07:44:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-19T07:44:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.221786+0000", 2024-09-19T07:44:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.221866+0000", 2024-09-19T07:44:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.221922+0000", 2024-09-19T07:44:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.221922+0000", 2024-09-19T07:44:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:33 create owner_uid=1000, owner_gid=1333 #0x1000000020e/file_0 2024-09-19T07:44:15.217970+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})", 2024-09-19T07:44:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:15.218043+0000", 2024-09-19T07:44:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.134012024999997, 2024-09-19T07:44:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0042401239999999996, 2024-09-19T07:44:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 33 2024-09-19T07:44:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.218043+0000", 2024-09-19T07:44:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.218043+0000", 2024-09-19T07:44:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.218044+0000", 2024-09-19T07:44:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.218049+0000", 2024-09-19T07:44:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.218067+0000", 2024-09-19T07:44:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.218295+0000", 2024-09-19T07:44:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "failed to rdlock, waiting" 2024-09-19T07:44:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.218453+0000", 2024-09-19T07:44:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.218745+0000", 2024-09-19T07:44:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-19T07:44:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.218745+0000", 2024-09-19T07:44:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-19T07:44:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.221925+0000", 2024-09-19T07:44:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-19T07:44:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.222073+0000", 2024-09-19T07:44:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.222172+0000", 2024-09-19T07:44:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.222282+0000", 2024-09-19T07:44:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.222284+0000", 2024-09-19T07:44:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:47 create owner_uid=1000, owner_gid=1333 #0x1000000020e/file_14 2024-09-19T07:44:15.273424+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})", 2024-09-19T07:44:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:15.273507+0000", 2024-09-19T07:44:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.078548820000002, 2024-09-19T07:44:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0055835420000000004, 2024-09-19T07:44:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 47 2024-09-19T07:44:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.273507+0000", 2024-09-19T07:44:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.273507+0000", 2024-09-19T07:44:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.273507+0000", 2024-09-19T07:44:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.273512+0000", 2024-09-19T07:44:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.273530+0000", 2024-09-19T07:44:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.273733+0000", 2024-09-19T07:44:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.274053+0000", 2024-09-19T07:44:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-19T07:44:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.274053+0000", 2024-09-19T07:44:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-19T07:44:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.278553+0000", 2024-09-19T07:44:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-19T07:44:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.278763+0000", 2024-09-19T07:44:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.278918+0000", 2024-09-19T07:44:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.279089+0000", 2024-09-19T07:44:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.279090+0000", 2024-09-19T07:44:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:60 mkdir owner_uid=1000, owner_gid=1333 #0x1/dir3 2024-09-19T07:44:15.496569+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})", 2024-09-19T07:44:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:15.496646+0000", 2024-09-19T07:44:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.855409113999997, 2024-09-19T07:44:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0048813449999999996, 2024-09-19T07:44:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 60 2024-09-19T07:44:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.496646+0000", 2024-09-19T07:44:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.496646+0000", 2024-09-19T07:44:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.496647+0000", 2024-09-19T07:44:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.496650+0000", 2024-09-19T07:44:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.496667+0000", 2024-09-19T07:44:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.496934+0000", 2024-09-19T07:44:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.497199+0000", 2024-09-19T07:44:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-19T07:44:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.497199+0000", 2024-09-19T07:44:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-19T07:44:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.501301+0000", 2024-09-19T07:44:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-19T07:44:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.501406+0000", 2024-09-19T07:44:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.501482+0000", 2024-09-19T07:44:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.501527+0000", 2024-09-19T07:44:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.501528+0000", 2024-09-19T07:44:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:64 create owner_uid=1000, owner_gid=1333 #0x10000000228/file_3 2024-09-19T07:44:15.509989+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})", 2024-09-19T07:44:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:15.510060+0000", 2024-09-19T07:44:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.841995216999997, 2024-09-19T07:44:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0055252000000000001, 2024-09-19T07:44:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 64 2024-09-19T07:44:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.510060+0000", 2024-09-19T07:44:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.510060+0000", 2024-09-19T07:44:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.510061+0000", 2024-09-19T07:44:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.510067+0000", 2024-09-19T07:44:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.510085+0000", 2024-09-19T07:44:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.510285+0000", 2024-09-19T07:44:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.510591+0000", 2024-09-19T07:44:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-19T07:44:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.510592+0000", 2024-09-19T07:44:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-19T07:44:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.515241+0000", 2024-09-19T07:44:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-19T07:44:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.515370+0000", 2024-09-19T07:44:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.515471+0000", 2024-09-19T07:44:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.515584+0000", 2024-09-19T07:44:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.515585+0000", 2024-09-19T07:44:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:74 create owner_uid=1000, owner_gid=1333 #0x10000000228/file_13 2024-09-19T07:44:15.552515+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})", 2024-09-19T07:44:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:15.552585+0000", 2024-09-19T07:44:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.799470395999997, 2024-09-19T07:44:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0085849280000000004, 2024-09-19T07:44:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 74 2024-09-19T07:44:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.552585+0000", 2024-09-19T07:44:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.552585+0000", 2024-09-19T07:44:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.552586+0000", 2024-09-19T07:44:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.552590+0000", 2024-09-19T07:44:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.552615+0000", 2024-09-19T07:44:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.552841+0000", 2024-09-19T07:44:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.553166+0000", 2024-09-19T07:44:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-19T07:44:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.553166+0000", 2024-09-19T07:44:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-19T07:44:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.560637+0000", 2024-09-19T07:44:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-19T07:44:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.560849+0000", 2024-09-19T07:44:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.561004+0000", 2024-09-19T07:44:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.561168+0000", 2024-09-19T07:44:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.561170+0000", 2024-09-19T07:44:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:75 create owner_uid=1000, owner_gid=1333 #0x10000000228/file_14 2024-09-19T07:44:15.561759+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})", 2024-09-19T07:44:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:15.561838+0000", 2024-09-19T07:44:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.790217058000003, 2024-09-19T07:44:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0039972599999999999, 2024-09-19T07:44:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 75 2024-09-19T07:44:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.561838+0000", 2024-09-19T07:44:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.561838+0000", 2024-09-19T07:44:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.561839+0000", 2024-09-19T07:44:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.561844+0000", 2024-09-19T07:44:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.561884+0000", 2024-09-19T07:44:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.562285+0000", 2024-09-19T07:44:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.562835+0000", 2024-09-19T07:44:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-19T07:44:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.562836+0000", 2024-09-19T07:44:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-19T07:44:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.565306+0000", 2024-09-19T07:44:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-19T07:44:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.565517+0000", 2024-09-19T07:44:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.565665+0000", 2024-09-19T07:44:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.565834+0000", 2024-09-19T07:44:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.565836+0000", 2024-09-19T07:44:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:83 create owner_uid=1000, owner_gid=1333 #0x10000000228/file_22 2024-09-19T07:44:15.597496+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})", 2024-09-19T07:44:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:15.597593+0000", 2024-09-19T07:44:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.754462109000002, 2024-09-19T07:44:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0040191369999999999, 2024-09-19T07:44:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 83 2024-09-19T07:44:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.597593+0000", 2024-09-19T07:44:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.597593+0000", 2024-09-19T07:44:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.597594+0000", 2024-09-19T07:44:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.597599+0000", 2024-09-19T07:44:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.597616+0000", 2024-09-19T07:44:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.598006+0000", 2024-09-19T07:44:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.598567+0000", 2024-09-19T07:44:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-19T07:44:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.598568+0000", 2024-09-19T07:44:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-19T07:44:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.601087+0000", 2024-09-19T07:44:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-19T07:44:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.601297+0000", 2024-09-19T07:44:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.601442+0000", 2024-09-19T07:44:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.601611+0000", 2024-09-19T07:44:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.601613+0000", 2024-09-19T07:44:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:84 create owner_uid=1000, owner_gid=1333 #0x10000000228/file_23 2024-09-19T07:44:15.602195+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})", 2024-09-19T07:44:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:15.602293+0000", 2024-09-19T07:44:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.749762570999998, 2024-09-19T07:44:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0040110099999999997, 2024-09-19T07:44:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 84 2024-09-19T07:44:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.602293+0000", 2024-09-19T07:44:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.602293+0000", 2024-09-19T07:44:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.602294+0000", 2024-09-19T07:44:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.602298+0000", 2024-09-19T07:44:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.602332+0000", 2024-09-19T07:44:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.602719+0000", 2024-09-19T07:44:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.603251+0000", 2024-09-19T07:44:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-19T07:44:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.603251+0000", 2024-09-19T07:44:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-19T07:44:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.605935+0000", 2024-09-19T07:44:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-19T07:44:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.606078+0000", 2024-09-19T07:44:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.606184+0000", 2024-09-19T07:44:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.606302+0000", 2024-09-19T07:44:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.606304+0000", 2024-09-19T07:44:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:88 mkdir owner_uid=1000, owner_gid=1333 #0x1/dir4 2024-09-19T07:44:15.790783+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})", 2024-09-19T07:44:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:15.790844+0000", 2024-09-19T07:44:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.561211616999998, 2024-09-19T07:44:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0045597299999999997, 2024-09-19T07:44:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 88 2024-09-19T07:44:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.790844+0000", 2024-09-19T07:44:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.790844+0000", 2024-09-19T07:44:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.790844+0000", 2024-09-19T07:44:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.790849+0000", 2024-09-19T07:44:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.790865+0000", 2024-09-19T07:44:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.791124+0000", 2024-09-19T07:44:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.791388+0000", 2024-09-19T07:44:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-19T07:44:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.791389+0000", 2024-09-19T07:44:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-19T07:44:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.795137+0000", 2024-09-19T07:44:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-19T07:44:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.795293+0000", 2024-09-19T07:44:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.795371+0000", 2024-09-19T07:44:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.795403+0000", 2024-09-19T07:44:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.795404+0000", 2024-09-19T07:44:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:90 create owner_uid=1000, owner_gid=1333 #0x10000000242/file_1 2024-09-19T07:44:15.796338+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})", 2024-09-19T07:44:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:15.796421+0000", 2024-09-19T07:44:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.555634742000002, 2024-09-19T07:44:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0060842969999999998, 2024-09-19T07:44:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 90 2024-09-19T07:44:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.796421+0000", 2024-09-19T07:44:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.796421+0000", 2024-09-19T07:44:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.796421+0000", 2024-09-19T07:44:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.796426+0000", 2024-09-19T07:44:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.796452+0000", 2024-09-19T07:44:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.796799+0000", 2024-09-19T07:44:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.797099+0000", 2024-09-19T07:44:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-19T07:44:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.797099+0000", 2024-09-19T07:44:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-19T07:44:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.802209+0000", 2024-09-19T07:44:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-19T07:44:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.802339+0000", 2024-09-19T07:44:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.802416+0000", 2024-09-19T07:44:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.802504+0000", 2024-09-19T07:44:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.802505+0000", 2024-09-19T07:44:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:99 create owner_uid=1000, owner_gid=1333 #0x10000000242/file_10 2024-09-19T07:44:15.833154+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})", 2024-09-19T07:44:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:15.833226+0000", 2024-09-19T07:44:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.518829988999997, 2024-09-19T07:44:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0054175780000000002, 2024-09-19T07:44:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 99 2024-09-19T07:44:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.833226+0000", 2024-09-19T07:44:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.833226+0000", 2024-09-19T07:44:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.833226+0000", 2024-09-19T07:44:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.833238+0000", 2024-09-19T07:44:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.833257+0000", 2024-09-19T07:44:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.833472+0000", 2024-09-19T07:44:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.833777+0000", 2024-09-19T07:44:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-19T07:44:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.833778+0000", 2024-09-19T07:44:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-19T07:44:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.838299+0000", 2024-09-19T07:44:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-19T07:44:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.838445+0000", 2024-09-19T07:44:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.838540+0000", 2024-09-19T07:44:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.838642+0000", 2024-09-19T07:44:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.838643+0000", 2024-09-19T07:44:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:102 create owner_uid=1000, owner_gid=1333 #0x10000000242/file_13 2024-09-19T07:44:15.846336+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})", 2024-09-19T07:44:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:15.846407+0000", 2024-09-19T07:44:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.505648903000001, 2024-09-19T07:44:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0050125880000000001, 2024-09-19T07:44:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 102 2024-09-19T07:44:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.846407+0000", 2024-09-19T07:44:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.846407+0000", 2024-09-19T07:44:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.846407+0000", 2024-09-19T07:44:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.846411+0000", 2024-09-19T07:44:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.846435+0000", 2024-09-19T07:44:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.846660+0000", 2024-09-19T07:44:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.846981+0000", 2024-09-19T07:44:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-19T07:44:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.846982+0000", 2024-09-19T07:44:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-19T07:44:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.851067+0000", 2024-09-19T07:44:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-19T07:44:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.851200+0000", 2024-09-19T07:44:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.851300+0000", 2024-09-19T07:44:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.851418+0000", 2024-09-19T07:44:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.851419+0000", 2024-09-19T07:44:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:106 create owner_uid=1000, owner_gid=1333 #0x10000000242/file_17 2024-09-19T07:44:15.863124+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})", 2024-09-19T07:44:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:15.863192+0000", 2024-09-19T07:44:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.488863207999998, 2024-09-19T07:44:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0051380829999999999, 2024-09-19T07:44:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 106 2024-09-19T07:44:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.863192+0000", 2024-09-19T07:44:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.863192+0000", 2024-09-19T07:44:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.863193+0000", 2024-09-19T07:44:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.863197+0000", 2024-09-19T07:44:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.863221+0000", 2024-09-19T07:44:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.863431+0000", 2024-09-19T07:44:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.863757+0000", 2024-09-19T07:44:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-19T07:44:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.863758+0000", 2024-09-19T07:44:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-19T07:44:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.867972+0000", 2024-09-19T07:44:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-19T07:44:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.868100+0000", 2024-09-19T07:44:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.868207+0000", 2024-09-19T07:44:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.868329+0000", 2024-09-19T07:44:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.868330+0000", 2024-09-19T07:44:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:107 create owner_uid=1000, owner_gid=1333 #0x10000000242/file_18 2024-09-19T07:44:15.868812+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})", 2024-09-19T07:44:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:15.868896+0000", 2024-09-19T07:44:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.483159633, 2024-09-19T07:44:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0059439059999999997, 2024-09-19T07:44:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 107 2024-09-19T07:44:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.868896+0000", 2024-09-19T07:44:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.868896+0000", 2024-09-19T07:44:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.868896+0000", 2024-09-19T07:44:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.868901+0000", 2024-09-19T07:44:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.868932+0000", 2024-09-19T07:44:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.869154+0000", 2024-09-19T07:44:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.869465+0000", 2024-09-19T07:44:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-19T07:44:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.869465+0000", 2024-09-19T07:44:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-19T07:44:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.874477+0000", 2024-09-19T07:44:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-19T07:44:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.874609+0000", 2024-09-19T07:44:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.874717+0000", 2024-09-19T07:44:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.874838+0000", 2024-09-19T07:44:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:15.874840+0000", 2024-09-19T07:44:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:127 readdir #0x10000000242 2024-09-19T07:44:16.246454+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})", 2024-09-19T07:44:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:16.246510+0000", 2024-09-19T07:44:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.105545837999998, 2024-09-19T07:44:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 16.506877736, 2024-09-19T07:44:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 127 2024-09-19T07:44:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:16.246510+0000", 2024-09-19T07:44:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:16.246510+0000", 2024-09-19T07:44:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:16.246510+0000", 2024-09-19T07:44:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:16.246513+0000", 2024-09-19T07:44:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:16.246533+0000", 2024-09-19T07:44:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:16.246584+0000", 2024-09-19T07:44:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:16.746774+0000", 2024-09-19T07:44:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:17.246955+0000", 2024-09-19T07:44:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:17.747087+0000", 2024-09-19T07:44:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:18.247202+0000", 2024-09-19T07:44:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:18.747296+0000", 2024-09-19T07:44:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:19.247481+0000", 2024-09-19T07:44:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:19.747646+0000", 2024-09-19T07:44:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:20.247836+0000", 2024-09-19T07:44:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:20.748003+0000", 2024-09-19T07:44:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:21.248188+0000", 2024-09-19T07:44:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:21.748308+0000", 2024-09-19T07:44:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:22.248493+0000", 2024-09-19T07:44:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:22.748663+0000", 2024-09-19T07:44:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:23.248824+0000", 2024-09-19T07:44:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:23.748999+0000", 2024-09-19T07:44:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:24.249190+0000", 2024-09-19T07:44:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:24.749364+0000", 2024-09-19T07:44:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:25.249534+0000", 2024-09-19T07:44:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:25.749703+0000", 2024-09-19T07:44:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:26.249843+0000", 2024-09-19T07:44:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:26.750014+0000", 2024-09-19T07:44:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:27.250199+0000", 2024-09-19T07:44:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:27.750389+0000", 2024-09-19T07:44:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:28.250574+0000", 2024-09-19T07:44:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:28.750764+0000", 2024-09-19T07:44:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:29.250969+0000", 2024-09-19T07:44:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:29.751132+0000", 2024-09-19T07:44:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:30.251299+0000", 2024-09-19T07:44:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:30.751400+0000", 2024-09-19T07:44:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:31.251596+0000", 2024-09-19T07:44:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:31.751770+0000", 2024-09-19T07:44:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:32.251959+0000", 2024-09-19T07:44:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:32.752287+0000", 2024-09-19T07:44:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "failed to rdlock, waiting" 2024-09-19T07:44:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:32.752770+0000", 2024-09-19T07:44:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:32.753205+0000", 2024-09-19T07:44:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:32.753255+0000", 2024-09-19T07:44:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:32.753385+0000", 2024-09-19T07:44:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:32.753387+0000", 2024-09-19T07:44:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5151:130 readdir #0x10000000228 2024-09-19T07:44:32.756300+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})", 2024-09-19T07:44:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-19T07:44:32.756382+0000", 2024-09-19T07:44:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 23.595673204000001, 2024-09-19T07:44:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 22.509342530000001, 2024-09-19T07:44:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-19T07:44:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-19T07:44:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-19T07:44:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-19T07:44:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-19T07:44:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-19T07:44:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:44:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5151 2024-09-19T07:44:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 130 2024-09-19T07:44:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-19T07:44:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-19T07:44:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:32.756382+0000", 2024-09-19T07:44:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-19T07:44:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:32.756382+0000", 2024-09-19T07:44:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-19T07:44:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:32.756383+0000", 2024-09-19T07:44:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-19T07:44:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:32.756386+0000", 2024-09-19T07:44:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-19T07:44:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:32.756414+0000", 2024-09-19T07:44:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-19T07:44:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:32.756481+0000", 2024-09-19T07:44:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:33.256676+0000", 2024-09-19T07:44:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:33.756849+0000", 2024-09-19T07:44:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:34.257048+0000", 2024-09-19T07:44:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:34.757191+0000", 2024-09-19T07:44:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:35.257397+0000", 2024-09-19T07:44:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:35.757549+0000", 2024-09-19T07:44:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:36.257732+0000", 2024-09-19T07:44:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:36.757936+0000", 2024-09-19T07:44:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:37.258079+0000", 2024-09-19T07:44:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:37.758297+0000", 2024-09-19T07:44:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:38.258496+0000", 2024-09-19T07:44:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:38.758700+0000", 2024-09-19T07:44:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:39.258863+0000", 2024-09-19T07:44:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:39.759060+0000", 2024-09-19T07:44:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:40.259225+0000", 2024-09-19T07:44:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:40.759432+0000", 2024-09-19T07:44:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:41.259618+0000", 2024-09-19T07:44:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:41.759806+0000", 2024-09-19T07:44:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:42.259992+0000", 2024-09-19T07:44:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:42.760157+0000", 2024-09-19T07:44:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:43.260304+0000", 2024-09-19T07:44:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:43.760493+0000", 2024-09-19T07:44:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:44.260624+0000", 2024-09-19T07:44:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:44.760762+0000", 2024-09-19T07:44:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:45.260954+0000", 2024-09-19T07:44:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:45.761048+0000", 2024-09-19T07:44:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:46.261187+0000", 2024-09-19T07:44:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:46.761354+0000", 2024-09-19T07:44:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:47.261547+0000", 2024-09-19T07:44:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:47.761722+0000", 2024-09-19T07:44:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:48.261924+0000", 2024-09-19T07:44:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:48.762123+0000", 2024-09-19T07:44:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:49.262251+0000", 2024-09-19T07:44:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:49.762420+0000", 2024-09-19T07:44:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:50.262609+0000", 2024-09-19T07:44:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:50.762805+0000", 2024-09-19T07:44:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:51.262994+0000", 2024-09-19T07:44:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:51.763164+0000", 2024-09-19T07:44:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:52.263339+0000", 2024-09-19T07:44:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:52.763499+0000", 2024-09-19T07:44:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:53.263623+0000", 2024-09-19T07:44:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:53.763793+0000", 2024-09-19T07:44:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:54.263988+0000", 2024-09-19T07:44:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:54.764143+0000", 2024-09-19T07:44:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-19T07:44:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:55.264587+0000", 2024-09-19T07:44:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "failed to rdlock, waiting" 2024-09-19T07:44:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:55.265117+0000", 2024-09-19T07:44:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-19T07:44:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:55.265538+0000", 2024-09-19T07:44:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-19T07:44:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:55.265583+0000", 2024-09-19T07:44:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-19T07:44:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:55.265723+0000", 2024-09-19T07:44:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-19T07:44:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:44:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:44:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-19T07:44:55.265725+0000", 2024-09-19T07:44:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-19T07:44:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:44:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-19T07:44:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:44:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T07:44:56.515 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:44:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.352+0000 7f223c850640 1 -- 172.21.15.16:0/1667547635 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f221806c2b0 msgr2=0x7f221806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.352+0000 7f223c850640 1 --2- 172.21.15.16:0/1667547635 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f221806c2b0 0x7f221806e770 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f222800e490 tx=0x7f22280075f0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.352+0000 7f223c850640 1 -- 172.21.15.16:0/1667547635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22380c8d20 msgr2=0x7f22380c9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.352+0000 7f223c850640 1 --2- 172.21.15.16:0/1667547635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22380c8d20 0x7f22380c9330 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f222002a0f0 tx=0x7f22200046c0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.352+0000 7f223c850640 1 -- 172.21.15.16:0/1667547635 shutdown_connections 2024-09-19T07:44:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.352+0000 7f223c850640 1 --2- 172.21.15.16:0/1667547635 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f221806c2b0 0x7f221806e770 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.352+0000 7f223c850640 1 --2- 172.21.15.16:0/1667547635 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2238143f30 0x7f22380b73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.352+0000 7f223c850640 1 --2- 172.21.15.16:0/1667547635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22380c8d20 0x7f22380c9330 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.352+0000 7f223c850640 1 --2- 172.21.15.16:0/1667547635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f223806d260 0x7f2238140d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.352+0000 7f223c850640 1 -- 172.21.15.16:0/1667547635 >> 172.21.15.16:0/1667547635 conn(0x7f22380b3ed0 msgr2=0x7f22380c7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.352+0000 7f223c850640 1 -- 172.21.15.16:0/1667547635 shutdown_connections 2024-09-19T07:44:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.352+0000 7f223c850640 1 -- 172.21.15.16:0/1667547635 wait complete. 2024-09-19T07:44:56.519 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:44:56.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:44:56.562 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:44:56.616 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:44:56.616 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:44:56.616 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:44:56.616 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:44:56.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:44:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.775+0000 7ff19d31f640 1 Processor -- start 2024-09-19T07:44:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.775+0000 7ff19d31f640 1 -- start start 2024-09-19T07:44:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.776+0000 7ff19d31f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff19815e680 0x7ff1980cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.776+0000 7ff19d31f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff1980d0010 0x7ff1980d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.776+0000 7ff19d31f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1980d1880 con 0x7ff19815e2a0 2024-09-19T07:44:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.776+0000 7ff19d31f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff1980d1b60 con 0x7ff1980d0010 2024-09-19T07:44:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.776+0000 7ff19d31f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff1980cf2d0 con 0x7ff19815e680 2024-09-19T07:44:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.776+0000 7ff1977fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff1980d0010 0x7ff1980d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.776+0000 7ff1977fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff1980d0010 0x7ff1980d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60250/0 (socket says 172.21.15.16:60250) 2024-09-19T07:44:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.776+0000 7ff1977fe640 1 -- 172.21.15.16:0/1250875060 learned_addr learned my addr 172.21.15.16:0/1250875060 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.776+0000 7ff1967fc640 1 --2- 172.21.15.16:0/1250875060 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff19815e680 0x7ff1980cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.776+0000 7ff1977fe640 1 -- 172.21.15.16:0/1250875060 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff19815e680 msgr2=0x7ff1980cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.776+0000 7ff1977fe640 1 --2- 172.21.15.16:0/1250875060 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff19815e680 0x7ff1980cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.776+0000 7ff1977fe640 1 -- 172.21.15.16:0/1250875060 >> v1:172.21.15.146:6789/0 conn(0x7ff19815e2a0 legacy=0x7ff19806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.776+0000 7ff1977fe640 1 -- 172.21.15.16:0/1250875060 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1980cf5b0 con 0x7ff1980d0010 2024-09-19T07:44:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.776+0000 7ff1967fc640 1 --2- 172.21.15.16:0/1250875060 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff19815e680 0x7ff1980cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.777+0000 7ff1977fe640 1 --2- 172.21.15.16:0/1250875060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1980d0010 0x7ff1980d4370 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff190009e50 tx=0x7ff19002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f1022a1bc6450f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:56.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.777+0000 7ff195ffb640 1 -- 172.21.15.16:0/1250875060 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190036070 con 0x7ff1980d0010 2024-09-19T07:44:56.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.777+0000 7ff195ffb640 1 -- 172.21.15.16:0/1250875060 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff190031600 con 0x7ff1980d0010 2024-09-19T07:44:56.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.777+0000 7ff195ffb640 1 -- 172.21.15.16:0/1250875060 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1900319d0 con 0x7ff1980d0010 2024-09-19T07:44:56.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.778+0000 7ff19d31f640 1 -- 172.21.15.16:0/1250875060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1980d0010 msgr2=0x7ff1980d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:56.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.778+0000 7ff19d31f640 1 --2- 172.21.15.16:0/1250875060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1980d0010 0x7ff1980d4370 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff190009e50 tx=0x7ff19002a680 comp rx=0 tx=0).stop 2024-09-19T07:44:56.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.778+0000 7ff19d31f640 1 -- 172.21.15.16:0/1250875060 shutdown_connections 2024-09-19T07:44:56.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.778+0000 7ff19d31f640 1 --2- 172.21.15.16:0/1250875060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1980d0010 0x7ff1980d4370 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.778+0000 7ff19d31f640 1 --2- 172.21.15.16:0/1250875060 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff19815e680 0x7ff1980cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.778+0000 7ff19d31f640 1 -- 172.21.15.16:0/1250875060 >> 172.21.15.16:0/1250875060 conn(0x7ff19809d760 msgr2=0x7ff19808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:56.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.778+0000 7ff19d31f640 1 -- 172.21.15.16:0/1250875060 shutdown_connections 2024-09-19T07:44:56.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.778+0000 7ff19d31f640 1 -- 172.21.15.16:0/1250875060 wait complete. 2024-09-19T07:44:56.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.779+0000 7ff19d31f640 1 Processor -- start 2024-09-19T07:44:56.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.779+0000 7ff19d31f640 1 -- start start 2024-09-19T07:44:56.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff19d31f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1980d0010 0x7ff1981f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff19d31f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff19815e2a0 0x7ff1980c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff19d31f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff19815e680 0x7ff1980c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff19d31f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff19806dcb0 con 0x7ff19815e680 2024-09-19T07:44:56.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff19d31f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff19806db30 con 0x7ff19815e2a0 2024-09-19T07:44:56.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff19d31f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff1980d48b0 con 0x7ff1980d0010 2024-09-19T07:44:56.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff1977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff19815e680 0x7ff1980c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff1977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff19815e680 0x7ff1980c8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60264/0 (socket says 172.21.15.16:60264) 2024-09-19T07:44:56.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff1977fe640 1 -- 172.21.15.16:0/1000039567 learned_addr learned my addr 172.21.15.16:0/1000039567 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:56.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff1967fc640 1 --2- 172.21.15.16:0/1000039567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff19815e2a0 0x7ff1980c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff196ffd640 1 --2- 172.21.15.16:0/1000039567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1980d0010 0x7ff1981f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff1977fe640 1 -- 172.21.15.16:0/1000039567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1980d0010 msgr2=0x7ff1981f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:56.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff1977fe640 1 --2- 172.21.15.16:0/1000039567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1980d0010 0x7ff1981f4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff1977fe640 1 -- 172.21.15.16:0/1000039567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff19815e2a0 msgr2=0x7ff1980c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:56.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff1977fe640 1 --2- 172.21.15.16:0/1000039567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff19815e2a0 0x7ff1980c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff1977fe640 1 -- 172.21.15.16:0/1000039567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1981e9f60 con 0x7ff19815e680 2024-09-19T07:44:56.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff1967fc640 1 --2- 172.21.15.16:0/1000039567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff19815e2a0 0x7ff1980c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:44:56.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff1977fe640 1 --2- 172.21.15.16:0/1000039567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff19815e680 0x7ff1980c8e60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff190034040 tx=0x7ff190004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:56.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff17f7fe640 1 -- 172.21.15.16:0/1000039567 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190041020 con 0x7ff19815e680 2024-09-19T07:44:56.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff17f7fe640 1 -- 172.21.15.16:0/1000039567 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1900077a0 con 0x7ff19815e680 2024-09-19T07:44:56.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff19d31f640 1 -- 172.21.15.16:0/1000039567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1981ea240 con 0x7ff19815e680 2024-09-19T07:44:56.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff17f7fe640 1 -- 172.21.15.16:0/1000039567 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190036040 con 0x7ff19815e680 2024-09-19T07:44:56.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff19d31f640 1 -- 172.21.15.16:0/1000039567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1981eaae0 con 0x7ff19815e680 2024-09-19T07:44:56.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.781+0000 7ff19d31f640 1 -- 172.21.15.16:0/1000039567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff19806dfe0 con 0x7ff19815e680 2024-09-19T07:44:56.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.782+0000 7ff17f7fe640 1 -- 172.21.15.16:0/1000039567 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff190044020 con 0x7ff19815e680 2024-09-19T07:44:56.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.782+0000 7ff17f7fe640 1 --2- 172.21.15.16:0/1000039567 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff18406c2b0 0x7ff18406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:56.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.782+0000 7ff17f7fe640 1 -- 172.21.15.16:0/1000039567 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7ff1900ac300 con 0x7ff19815e680 2024-09-19T07:44:56.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.783+0000 7ff196ffd640 1 --2- 172.21.15.16:0/1000039567 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff18406c2b0 0x7ff18406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:56.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.783+0000 7ff196ffd640 1 --2- 172.21.15.16:0/1000039567 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff18406c2b0 0x7ff18406e770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff1880015e0 tx=0x7ff188009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:56.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.788+0000 7ff17f7fe640 1 -- 172.21.15.16:0/1000039567 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff19004b030 con 0x7ff19815e680 2024-09-19T07:44:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.974+0000 7ff19d31f640 1 -- 172.21.15.16:0/1000039567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff1980d4370 con 0x7ff19815e680 2024-09-19T07:44:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.975+0000 7ff17f7fe640 1 -- 172.21.15.16:0/1000039567 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) ==== 80+0+0 (secure 0 0 0) 0x7ff19007a8c0 con 0x7ff19815e680 2024-09-19T07:44:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:44:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.978+0000 7ff19d31f640 1 -- 172.21.15.16:0/1000039567 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff18406c2b0 msgr2=0x7ff18406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.978+0000 7ff19d31f640 1 --2- 172.21.15.16:0/1000039567 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff18406c2b0 0x7ff18406e770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff1880015e0 tx=0x7ff188009040 comp rx=0 tx=0).stop 2024-09-19T07:44:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.978+0000 7ff19d31f640 1 -- 172.21.15.16:0/1000039567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff19815e680 msgr2=0x7ff1980c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.978+0000 7ff19d31f640 1 --2- 172.21.15.16:0/1000039567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff19815e680 0x7ff1980c8e60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff190034040 tx=0x7ff190004a40 comp rx=0 tx=0).stop 2024-09-19T07:44:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.978+0000 7ff19d31f640 1 -- 172.21.15.16:0/1000039567 shutdown_connections 2024-09-19T07:44:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.978+0000 7ff19d31f640 1 --2- 172.21.15.16:0/1000039567 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff18406c2b0 0x7ff18406e770 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.978+0000 7ff19d31f640 1 --2- 172.21.15.16:0/1000039567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff19815e680 0x7ff1980c8e60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.978+0000 7ff19d31f640 1 --2- 172.21.15.16:0/1000039567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff19815e2a0 0x7ff1980c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.978+0000 7ff19d31f640 1 --2- 172.21.15.16:0/1000039567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1980d0010 0x7ff1981f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.978+0000 7ff19d31f640 1 -- 172.21.15.16:0/1000039567 >> 172.21.15.16:0/1000039567 conn(0x7ff19809d760 msgr2=0x7ff19815f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:56.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.978+0000 7ff19d31f640 1 -- 172.21.15.16:0/1000039567 shutdown_connections 2024-09-19T07:44:56.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:56.978+0000 7ff19d31f640 1 -- 172.21.15.16:0/1000039567 wait complete. 2024-09-19T07:44:56.990 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-19T07:44:56.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:56.990 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T07:44:57.014 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[48356]: fuse finished with error 0 and tester_r 0 2024-09-19T07:44:57.016 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:44:57.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:57.016 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:44:57.113 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:44:57.113 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:44:57.113 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:44:57.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:44:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.256+0000 7fcde828d640 1 Processor -- start 2024-09-19T07:44:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.256+0000 7fcde828d640 1 -- start start 2024-09-19T07:44:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.257+0000 7fcde828d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcde015e680 0x7fcde00cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.257+0000 7fcde828d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcde00d0010 0x7fcde00d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.257+0000 7fcde828d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcde00d1880 con 0x7fcde015e2a0 2024-09-19T07:44:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.257+0000 7fcde828d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fcde00d1b60 con 0x7fcde00d0010 2024-09-19T07:44:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.257+0000 7fcde828d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fcde00cf2d0 con 0x7fcde015e680 2024-09-19T07:44:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.257+0000 7fcde682a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcde00d0010 0x7fcde00d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.257+0000 7fcde682a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcde00d0010 0x7fcde00d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60280/0 (socket says 172.21.15.16:60280) 2024-09-19T07:44:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.257+0000 7fcde682a640 1 -- 172.21.15.16:0/632855083 learned_addr learned my addr 172.21.15.16:0/632855083 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.257+0000 7fcde5828640 1 --2- 172.21.15.16:0/632855083 >> v2:172.21.15.146:3300/0 conn(0x7fcde015e680 0x7fcde00cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.257+0000 7fcde682a640 1 -- 172.21.15.16:0/632855083 >> v2:172.21.15.146:3300/0 conn(0x7fcde015e680 msgr2=0x7fcde00cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.257+0000 7fcde682a640 1 --2- 172.21.15.16:0/632855083 >> v2:172.21.15.146:3300/0 conn(0x7fcde015e680 0x7fcde00cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.257+0000 7fcde682a640 1 -- 172.21.15.16:0/632855083 >> v1:172.21.15.146:6789/0 conn(0x7fcde015e2a0 legacy=0x7fcde006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.257+0000 7fcde682a640 1 -- 172.21.15.16:0/632855083 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde00cf5b0 con 0x7fcde00d0010 2024-09-19T07:44:57.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.258+0000 7fcde682a640 1 --2- 172.21.15.16:0/632855083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcde00d0010 0x7fcde00d4370 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcde00a22a0 tx=0x7fcdd802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b427ed5f82054217 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:57.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.258+0000 7fcde5027640 1 -- 172.21.15.16:0/632855083 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8036070 con 0x7fcde00d0010 2024-09-19T07:44:57.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.258+0000 7fcde5027640 1 -- 172.21.15.16:0/632855083 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd8031600 con 0x7fcde00d0010 2024-09-19T07:44:57.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.258+0000 7fcde5027640 1 -- 172.21.15.16:0/632855083 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80319d0 con 0x7fcde00d0010 2024-09-19T07:44:57.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.259+0000 7fcde828d640 1 -- 172.21.15.16:0/632855083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcde00d0010 msgr2=0x7fcde00d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:57.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.259+0000 7fcde828d640 1 --2- 172.21.15.16:0/632855083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcde00d0010 0x7fcde00d4370 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcde00a22a0 tx=0x7fcdd802a6a0 comp rx=0 tx=0).stop 2024-09-19T07:44:57.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.259+0000 7fcde828d640 1 -- 172.21.15.16:0/632855083 shutdown_connections 2024-09-19T07:44:57.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.259+0000 7fcde828d640 1 --2- 172.21.15.16:0/632855083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcde00d0010 0x7fcde00d4370 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:57.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.259+0000 7fcde828d640 1 --2- 172.21.15.16:0/632855083 >> v2:172.21.15.146:3300/0 conn(0x7fcde015e680 0x7fcde00cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:57.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.259+0000 7fcde828d640 1 -- 172.21.15.16:0/632855083 >> 172.21.15.16:0/632855083 conn(0x7fcde009d760 msgr2=0x7fcde008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:57.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.259+0000 7fcde828d640 1 -- 172.21.15.16:0/632855083 shutdown_connections 2024-09-19T07:44:57.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.259+0000 7fcde828d640 1 -- 172.21.15.16:0/632855083 wait complete. 2024-09-19T07:44:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.260+0000 7fcde828d640 1 Processor -- start 2024-09-19T07:44:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.260+0000 7fcde828d640 1 -- start start 2024-09-19T07:44:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.261+0000 7fcde828d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcde00d0010 0x7fcde00c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.261+0000 7fcde828d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcde015e2a0 0x7fcde00c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.261+0000 7fcde828d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcde015e680 0x7fcde00c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.261+0000 7fcde828d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcde006dcb0 con 0x7fcde015e680 2024-09-19T07:44:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.261+0000 7fcde828d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcde006db30 con 0x7fcde015e2a0 2024-09-19T07:44:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.261+0000 7fcde828d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcde00d48b0 con 0x7fcde00d0010 2024-09-19T07:44:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.261+0000 7fcde682a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcde015e680 0x7fcde00c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.261+0000 7fcde682a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcde015e680 0x7fcde00c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60290/0 (socket says 172.21.15.16:60290) 2024-09-19T07:44:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.261+0000 7fcde682a640 1 -- 172.21.15.16:0/723543669 learned_addr learned my addr 172.21.15.16:0/723543669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.261+0000 7fcde5828640 1 --2- 172.21.15.16:0/723543669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcde015e2a0 0x7fcde00c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.261+0000 7fcde6029640 1 --2- 172.21.15.16:0/723543669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcde00d0010 0x7fcde00c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.261+0000 7fcde682a640 1 -- 172.21.15.16:0/723543669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcde00d0010 msgr2=0x7fcde00c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.261+0000 7fcde682a640 1 --2- 172.21.15.16:0/723543669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcde00d0010 0x7fcde00c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.261+0000 7fcde682a640 1 -- 172.21.15.16:0/723543669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcde015e2a0 msgr2=0x7fcde00c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.261+0000 7fcde682a640 1 --2- 172.21.15.16:0/723543669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcde015e2a0 0x7fcde00c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.261+0000 7fcde682a640 1 -- 172.21.15.16:0/723543669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde00ca670 con 0x7fcde015e680 2024-09-19T07:44:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.262+0000 7fcde682a640 1 --2- 172.21.15.16:0/723543669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcde015e680 0x7fcde00c8450 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8034040 tx=0x7fcdd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.262+0000 7fcdd6ffd640 1 -- 172.21.15.16:0/723543669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8036050 con 0x7fcde015e680 2024-09-19T07:44:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.262+0000 7fcdd6ffd640 1 -- 172.21.15.16:0/723543669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd8042070 con 0x7fcde015e680 2024-09-19T07:44:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.262+0000 7fcdd6ffd640 1 -- 172.21.15.16:0/723543669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8009510 con 0x7fcde015e680 2024-09-19T07:44:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.262+0000 7fcde828d640 1 -- 172.21.15.16:0/723543669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde00ca950 con 0x7fcde015e680 2024-09-19T07:44:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.262+0000 7fcde828d640 1 -- 172.21.15.16:0/723543669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde00caf10 con 0x7fcde015e680 2024-09-19T07:44:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.263+0000 7fcde828d640 1 -- 172.21.15.16:0/723543669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde00d4c70 con 0x7fcde015e680 2024-09-19T07:44:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.263+0000 7fcdd6ffd640 1 -- 172.21.15.16:0/723543669 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcdd8007bd0 con 0x7fcde015e680 2024-09-19T07:44:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.264+0000 7fcdd6ffd640 1 --2- 172.21.15.16:0/723543669 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcdc406c2b0 0x7fcdc406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.264+0000 7fcdd6ffd640 1 -- 172.21.15.16:0/723543669 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fcdd80ac270 con 0x7fcde015e680 2024-09-19T07:44:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.264+0000 7fcde6029640 1 --2- 172.21.15.16:0/723543669 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcdc406c2b0 0x7fcdc406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.265+0000 7fcde6029640 1 --2- 172.21.15.16:0/723543669 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcdc406c2b0 0x7fcdc406e770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcddc001780 tx=0x7fcddc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:57.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.268+0000 7fcdd6ffd640 1 -- 172.21.15.16:0/723543669 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fcdd807a270 con 0x7fcde015e680 2024-09-19T07:44:57.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.462+0000 7fcde828d640 1 -- 172.21.15.16:0/723543669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcde006cf90 con 0x7fcde015e680 2024-09-19T07:44:57.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.462+0000 7fcdd6ffd640 1 -- 172.21.15.16:0/723543669 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) ==== 80+0+0 (secure 0 0 0) 0x7fcdd807e570 con 0x7fcde015e680 2024-09-19T07:44:57.463 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:44:57.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.464+0000 7fcde828d640 1 -- 172.21.15.16:0/723543669 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcdc406c2b0 msgr2=0x7fcdc406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:57.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.464+0000 7fcde828d640 1 --2- 172.21.15.16:0/723543669 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcdc406c2b0 0x7fcdc406e770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcddc001780 tx=0x7fcddc009040 comp rx=0 tx=0).stop 2024-09-19T07:44:57.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.464+0000 7fcde828d640 1 -- 172.21.15.16:0/723543669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcde015e680 msgr2=0x7fcde00c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:57.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.464+0000 7fcde828d640 1 --2- 172.21.15.16:0/723543669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcde015e680 0x7fcde00c8450 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8034040 tx=0x7fcdd8004a40 comp rx=0 tx=0).stop 2024-09-19T07:44:57.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.464+0000 7fcde828d640 1 -- 172.21.15.16:0/723543669 shutdown_connections 2024-09-19T07:44:57.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.464+0000 7fcde828d640 1 --2- 172.21.15.16:0/723543669 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcdc406c2b0 0x7fcdc406e770 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:57.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.464+0000 7fcde828d640 1 --2- 172.21.15.16:0/723543669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcde015e680 0x7fcde00c8450 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:57.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.464+0000 7fcde828d640 1 --2- 172.21.15.16:0/723543669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcde015e2a0 0x7fcde00c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:57.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.464+0000 7fcde828d640 1 --2- 172.21.15.16:0/723543669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcde00d0010 0x7fcde00c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:57.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.464+0000 7fcde828d640 1 -- 172.21.15.16:0/723543669 >> 172.21.15.16:0/723543669 conn(0x7fcde009d760 msgr2=0x7fcde0160970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:57.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.464+0000 7fcde828d640 1 -- 172.21.15.16:0/723543669 shutdown_connections 2024-09-19T07:44:57.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.464+0000 7fcde828d640 1 -- 172.21.15.16:0/723543669 wait complete. 2024-09-19T07:44:57.475 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2024-09-19T07:44:57.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:57.476 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T07:44:57.503 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[44154]: fuse finished with error 0 and tester_r 0 2024-09-19T07:44:57.505 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:44:57.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:57.505 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T07:44:57.646 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:44:57.646 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:44:57.646 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:44:57.662 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:44:57.662 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:44:57.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:57.663 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T07:44:57.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:44:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T07:44:57.719 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:44:57.719 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:44:57.719 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:44:57.737 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:44:57.737 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:44:57.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:44:57.737 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T07:44:57.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:44:57.793 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T07:44:57.794 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:44:57.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.895+0000 7f1db42f1640 1 Processor -- start 2024-09-19T07:44:57.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.895+0000 7f1db42f1640 1 -- start start 2024-09-19T07:44:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.896+0000 7f1db42f1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dac15e2a0 0x7f1dac06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.896+0000 7f1db42f1640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1dac15e680 0x7f1dac0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.896+0000 7f1db42f1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dac0d1880 con 0x7f1dac06d960 2024-09-19T07:44:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.896+0000 7f1db42f1640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f1dac0d1b60 con 0x7f1dac15e680 2024-09-19T07:44:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.896+0000 7f1db42f1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1dac0cf2d0 con 0x7f1dac15e2a0 2024-09-19T07:44:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.896+0000 7f1db288e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1dac06d960 0x7f1dac0d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43990/0 (socket says 172.21.15.16:43990) 2024-09-19T07:44:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.896+0000 7f1db288e640 1 -- 172.21.15.16:0/2683008515 learned_addr learned my addr 172.21.15.16:0/2683008515 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:44:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db188c640 1 --2- 172.21.15.16:0/2683008515 >> v2:172.21.15.146:3300/0 conn(0x7f1dac15e680 0x7f1dac0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db208d640 1 --2- 172.21.15.16:0/2683008515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dac15e2a0 0x7f1dac06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db108b640 1 -- 172.21.15.16:0/2683008515 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 552067143 0 0) 0x7f1dac0d1880 con 0x7f1dac06d960 2024-09-19T07:44:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db108b640 1 -- 172.21.15.16:0/2683008515 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d98003520 con 0x7f1dac06d960 2024-09-19T07:44:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db108b640 1 -- 172.21.15.16:0/2683008515 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3862665346 0 0) 0x7f1d98003520 con 0x7f1dac06d960 2024-09-19T07:44:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db108b640 1 -- 172.21.15.16:0/2683008515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dac15e2a0 msgr2=0x7f1dac06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db108b640 1 --2- 172.21.15.16:0/2683008515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dac15e2a0 0x7f1dac06d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db108b640 1 -- 172.21.15.16:0/2683008515 >> v2:172.21.15.146:3300/0 conn(0x7f1dac15e680 msgr2=0x7f1dac0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db108b640 1 --2- 172.21.15.16:0/2683008515 >> v2:172.21.15.146:3300/0 conn(0x7f1dac15e680 0x7f1dac0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db108b640 1 -- 172.21.15.16:0/2683008515 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac0cf5b0 con 0x7f1dac06d960 2024-09-19T07:44:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db108b640 1 -- 172.21.15.16:0/2683008515 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f1da40019f0 con 0x7f1dac06d960 2024-09-19T07:44:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db108b640 1 -- 172.21.15.16:0/2683008515 >> v1:172.21.15.16:6789/0 conn(0x7f1dac06d960 legacy=0x7f1dac0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db108b640 1 --2- 172.21.15.16:0/2683008515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d980039b0 0x7f1d98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db108b640 1 --2- 172.21.15.16:0/2683008515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dac15e2a0 0x7f1d98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db108b640 1 --2- 172.21.15.16:0/2683008515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d98004d40 0x7f1d980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db108b640 1 -- 172.21.15.16:0/2683008515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac0d1b60 con 0x7f1d980039b0 2024-09-19T07:44:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db108b640 1 -- 172.21.15.16:0/2683008515 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac0cf2d0 con 0x7f1d98004d40 2024-09-19T07:44:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db108b640 1 -- 172.21.15.16:0/2683008515 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1d9800b9c0 con 0x7f1dac15e2a0 2024-09-19T07:44:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db208d640 1 --2- 172.21.15.16:0/2683008515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d980039b0 0x7f1d98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db188c640 1 --2- 172.21.15.16:0/2683008515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dac15e2a0 0x7f1d98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db288e640 1 --2- 172.21.15.16:0/2683008515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d98004d40 0x7f1d980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db208d640 1 -- 172.21.15.16:0/2683008515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dac15e2a0 msgr2=0x7f1d98004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db208d640 1 --2- 172.21.15.16:0/2683008515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dac15e2a0 0x7f1d98004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db208d640 1 -- 172.21.15.16:0/2683008515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d98004d40 msgr2=0x7f1d980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.897+0000 7f1db208d640 1 --2- 172.21.15.16:0/2683008515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d98004d40 0x7f1d980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.898+0000 7f1db208d640 1 -- 172.21.15.16:0/2683008515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d98003520 con 0x7f1d980039b0 2024-09-19T07:44:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.898+0000 7f1db208d640 1 --2- 172.21.15.16:0/2683008515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d980039b0 0x7f1d98003dd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009ac0 tx=0x7f1d98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe250315674abb8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.898+0000 7f1db108b640 1 -- 172.21.15.16:0/2683008515 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8036070 con 0x7f1d980039b0 2024-09-19T07:44:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.898+0000 7f1db108b640 1 -- 172.21.15.16:0/2683008515 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da8031400 con 0x7f1d980039b0 2024-09-19T07:44:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.898+0000 7f1db108b640 1 -- 172.21.15.16:0/2683008515 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8009690 con 0x7f1d980039b0 2024-09-19T07:44:57.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.899+0000 7f1db42f1640 1 -- 172.21.15.16:0/2683008515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d980039b0 msgr2=0x7f1d98003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.899+0000 7f1db42f1640 1 --2- 172.21.15.16:0/2683008515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d980039b0 0x7f1d98003dd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009ac0 tx=0x7f1d98005830 comp rx=0 tx=0).stop 2024-09-19T07:44:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.899+0000 7f1db288e640 1 -- 172.21.15.16:0/2683008515 reap_dead start 2024-09-19T07:44:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.899+0000 7f1db42f1640 1 -- 172.21.15.16:0/2683008515 shutdown_connections 2024-09-19T07:44:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.899+0000 7f1db42f1640 1 -- 172.21.15.16:0/2683008515 >> 172.21.15.16:0/2683008515 conn(0x7f1dac09d760 msgr2=0x7f1dac08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.899+0000 7f1db42f1640 1 -- 172.21.15.16:0/2683008515 shutdown_connections 2024-09-19T07:44:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.899+0000 7f1db42f1640 1 -- 172.21.15.16:0/2683008515 wait complete. 2024-09-19T07:44:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.900+0000 7f1db42f1640 1 Processor -- start 2024-09-19T07:44:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.900+0000 7f1db42f1640 1 -- start start 2024-09-19T07:44:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.901+0000 7f1db42f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d980039b0 0x7f1dac0cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.901+0000 7f1db42f1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dac0b9c20 0x7f1dac1e8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.901+0000 7f1db42f1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dac1e87b0 0x7f1dac158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.901+0000 7f1db42f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac0d4c00 con 0x7f1d980039b0 2024-09-19T07:44:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.901+0000 7f1db42f1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac0d4a80 con 0x7f1dac0b9c20 2024-09-19T07:44:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.901+0000 7f1db42f1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1dac06d420 con 0x7f1dac1e87b0 2024-09-19T07:44:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.901+0000 7f1db208d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d980039b0 0x7f1dac0cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.901+0000 7f1db208d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d980039b0 0x7f1dac0cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60314/0 (socket says 172.21.15.16:60314) 2024-09-19T07:44:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.901+0000 7f1db208d640 1 -- 172.21.15.16:0/824096992 learned_addr learned my addr 172.21.15.16:0/824096992 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.901+0000 7f1db188c640 1 --2- 172.21.15.16:0/824096992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dac0b9c20 0x7f1dac1e8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.901+0000 7f1db288e640 1 --2- 172.21.15.16:0/824096992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dac1e87b0 0x7f1dac158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.901+0000 7f1db208d640 1 -- 172.21.15.16:0/824096992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dac1e87b0 msgr2=0x7f1dac158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.901+0000 7f1db208d640 1 --2- 172.21.15.16:0/824096992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dac1e87b0 0x7f1dac158280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.901+0000 7f1db208d640 1 -- 172.21.15.16:0/824096992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dac0b9c20 msgr2=0x7f1dac1e8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.901+0000 7f1db208d640 1 --2- 172.21.15.16:0/824096992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dac0b9c20 0x7f1dac1e8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.901+0000 7f1db208d640 1 -- 172.21.15.16:0/824096992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac0ba0a0 con 0x7f1d980039b0 2024-09-19T07:44:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.902+0000 7f1db208d640 1 --2- 172.21.15.16:0/824096992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d980039b0 0x7f1dac0cda90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1da8000c00 tx=0x7f1da8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.902+0000 7f1da2ffd640 1 -- 172.21.15.16:0/824096992 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8040020 con 0x7f1d980039b0 2024-09-19T07:44:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.902+0000 7f1da2ffd640 1 -- 172.21.15.16:0/824096992 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da8003a40 con 0x7f1d980039b0 2024-09-19T07:44:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.902+0000 7f1db42f1640 1 -- 172.21.15.16:0/824096992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dac0b8310 con 0x7f1d980039b0 2024-09-19T07:44:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.902+0000 7f1da2ffd640 1 -- 172.21.15.16:0/824096992 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8036040 con 0x7f1d980039b0 2024-09-19T07:44:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.902+0000 7f1db42f1640 1 -- 172.21.15.16:0/824096992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dac0ccfb0 con 0x7f1d980039b0 2024-09-19T07:44:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.903+0000 7f1db42f1640 1 -- 172.21.15.16:0/824096992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dac0ce8a0 con 0x7f1d980039b0 2024-09-19T07:44:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.904+0000 7f1da2ffd640 1 -- 172.21.15.16:0/824096992 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1da8043020 con 0x7f1d980039b0 2024-09-19T07:44:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.904+0000 7f1da2ffd640 1 --2- 172.21.15.16:0/824096992 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1d9006c2e0 0x7f1d9006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.904+0000 7f1da2ffd640 1 -- 172.21.15.16:0/824096992 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f1da80ab5e0 con 0x7f1d980039b0 2024-09-19T07:44:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.904+0000 7f1db188c640 1 --2- 172.21.15.16:0/824096992 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1d9006c2e0 0x7f1d9006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.905+0000 7f1db188c640 1 --2- 172.21.15.16:0/824096992 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1d9006c2e0 0x7f1d9006e7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0b7570 tx=0x7f1d9c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:57.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:57.908+0000 7f1da2ffd640 1 -- 172.21.15.16:0/824096992 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1dac0ce8a0 con 0x7f1d980039b0 2024-09-19T07:44:58.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.119+0000 7f1db42f1640 1 -- 172.21.15.16:0/824096992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1dac0d4370 con 0x7f1d980039b0 2024-09-19T07:44:58.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.119+0000 7f1da2ffd640 1 -- 172.21.15.16:0/824096992 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5229 (secure 0 0 0) 0x7f1da804a070 con 0x7f1d980039b0 2024-09-19T07:44:58.120 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:44:58.120 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-19T07:44:04:108666+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4655,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/3238551814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3238551814},{"type":"v1","addr":"172.21.15.146:6836","nonce":3238551814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3450384245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3450384245},{"type":"v1","addr":"172.21.15.146:6837","nonce":3450384245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4944,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3663416804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3663416804},{"type":"v1","addr":"172.21.15.16:6834","nonce":3663416804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:44:00.970677+0000","modified":"2024-09-19T07:44:03.455740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4947":{"gid":4947,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2028210099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2028210099},{"type":"v1","addr":"172.21.15.16:6835","nonce":2028210099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4947,"qdb_cluster":[4947]},"id":2}]} 2024-09-19T07:44:58.120 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-19T07:44:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.121+0000 7f1db42f1640 1 -- 172.21.15.16:0/824096992 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1d9006c2e0 msgr2=0x7f1d9006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.121+0000 7f1db42f1640 1 --2- 172.21.15.16:0/824096992 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1d9006c2e0 0x7f1d9006e7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0b7570 tx=0x7f1d9c009190 comp rx=0 tx=0).stop 2024-09-19T07:44:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.121+0000 7f1db42f1640 1 -- 172.21.15.16:0/824096992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d980039b0 msgr2=0x7f1dac0cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.121+0000 7f1db42f1640 1 --2- 172.21.15.16:0/824096992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d980039b0 0x7f1dac0cda90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1da8000c00 tx=0x7f1da8004730 comp rx=0 tx=0).stop 2024-09-19T07:44:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.121+0000 7f1db42f1640 1 -- 172.21.15.16:0/824096992 shutdown_connections 2024-09-19T07:44:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.121+0000 7f1db42f1640 1 --2- 172.21.15.16:0/824096992 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1d9006c2e0 0x7f1d9006e7a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.121+0000 7f1db42f1640 1 --2- 172.21.15.16:0/824096992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dac1e87b0 0x7f1dac158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.121+0000 7f1db42f1640 1 --2- 172.21.15.16:0/824096992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dac0b9c20 0x7f1dac1e8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.121+0000 7f1db42f1640 1 --2- 172.21.15.16:0/824096992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d980039b0 0x7f1dac0cda90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.121+0000 7f1db42f1640 1 -- 172.21.15.16:0/824096992 >> 172.21.15.16:0/824096992 conn(0x7f1dac09d760 msgr2=0x7f1dac0cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.121+0000 7f1db42f1640 1 -- 172.21.15.16:0/824096992 shutdown_connections 2024-09-19T07:44:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.121+0000 7f1db42f1640 1 -- 172.21.15.16:0/824096992 wait complete. 2024-09-19T07:44:58.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:44:58.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.234+0000 7f4ad8711640 1 Processor -- start 2024-09-19T07:44:58.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.234+0000 7f4ad8711640 1 -- start start 2024-09-19T07:44:58.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.235+0000 7f4ad8711640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015e2a0 0x7f4ad006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:58.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.235+0000 7f4ad8711640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4ad015e680 0x7f4ad00cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:58.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.235+0000 7f4ad8711640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ad00d1880 con 0x7f4ad006d960 2024-09-19T07:44:58.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.235+0000 7f4ad8711640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4ad00d1b60 con 0x7f4ad015e680 2024-09-19T07:44:58.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.235+0000 7f4ad8711640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad00cf2d0 con 0x7f4ad015e2a0 2024-09-19T07:44:58.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.235+0000 7f4ad64ad640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015e2a0 0x7f4ad006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:58.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.235+0000 7f4ad5cac640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4ad015e680 0x7f4ad00cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:58.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.235+0000 7f4ad64ad640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015e2a0 0x7f4ad006d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:35912/0 (socket says 172.21.15.16:35912) 2024-09-19T07:44:58.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.235+0000 7f4ad6cae640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f4ad006d960 0x7f4ad00d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:38764/0 (socket says 172.21.15.16:38764) 2024-09-19T07:44:58.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.235+0000 7f4ad64ad640 1 -- 172.21.15.16:0/2236541412 learned_addr learned my addr 172.21.15.16:0/2236541412 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:58.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.235+0000 7f4ad5cac640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4ad015e680 0x7f4ad00cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:37906/0 (socket says 172.21.15.16:37906) 2024-09-19T07:44:58.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.235+0000 7f4ad5cac640 1 -- 172.21.15.16:0/2236541412 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015e2a0 msgr2=0x7f4ad006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.235+0000 7f4ad5cac640 1 --2- 172.21.15.16:0/2236541412 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015e2a0 0x7f4ad006d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.235+0000 7f4ad5cac640 1 -- 172.21.15.16:0/2236541412 >> v1:172.21.15.146:6789/0 conn(0x7f4ad006d960 legacy=0x7f4ad00d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.235+0000 7f4ad54ab640 1 -- 172.21.15.16:0/2236541412 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3722632871 0 0) 0x7f4ad00d1880 con 0x7f4ad006d960 2024-09-19T07:44:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.235+0000 7f4ad5cac640 1 -- 172.21.15.16:0/2236541412 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad00cf5b0 con 0x7f4ad015e680 2024-09-19T07:44:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.235+0000 7f4ad64ad640 1 --2- 172.21.15.16:0/2236541412 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015e2a0 0x7f4ad006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:44:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.236+0000 7f4ad5cac640 1 --2- 172.21.15.16:0/2236541412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ad015e680 0x7f4ad00cfad0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00097a0 tx=0x7f4ac002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa34b158c9ffa183 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.236+0000 7f4ad54ab640 1 -- 172.21.15.16:0/2236541412 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0037070 con 0x7f4ad015e680 2024-09-19T07:44:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.236+0000 7f4ad54ab640 1 -- 172.21.15.16:0/2236541412 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac0032440 con 0x7f4ad015e680 2024-09-19T07:44:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.236+0000 7f4ad54ab640 1 -- 172.21.15.16:0/2236541412 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0032810 con 0x7f4ad015e680 2024-09-19T07:44:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.237+0000 7f4ad8711640 1 -- 172.21.15.16:0/2236541412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ad015e680 msgr2=0x7f4ad00cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.237+0000 7f4ad8711640 1 --2- 172.21.15.16:0/2236541412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ad015e680 0x7f4ad00cfad0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00097a0 tx=0x7f4ac002a180 comp rx=0 tx=0).stop 2024-09-19T07:44:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.237+0000 7f4ad8711640 1 -- 172.21.15.16:0/2236541412 shutdown_connections 2024-09-19T07:44:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.237+0000 7f4ad8711640 1 --2- 172.21.15.16:0/2236541412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ad015e680 0x7f4ad00cfad0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.237+0000 7f4ad8711640 1 --2- 172.21.15.16:0/2236541412 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015e2a0 0x7f4ad006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.237+0000 7f4ad8711640 1 -- 172.21.15.16:0/2236541412 >> 172.21.15.16:0/2236541412 conn(0x7f4ad009d760 msgr2=0x7f4ad008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.237+0000 7f4ad8711640 1 -- 172.21.15.16:0/2236541412 shutdown_connections 2024-09-19T07:44:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.237+0000 7f4ad8711640 1 -- 172.21.15.16:0/2236541412 wait complete. 2024-09-19T07:44:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.238+0000 7f4ad8711640 1 Processor -- start 2024-09-19T07:44:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.238+0000 7f4ad8711640 1 -- start start 2024-09-19T07:44:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.239+0000 7f4ad8711640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ad006d960 0x7f4ad01ea610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.239+0000 7f4ad8711640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad015e2a0 0x7f4ad01496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.239+0000 7f4ad8711640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015e680 0x7f4ad0160f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.239+0000 7f4ad8711640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad00d4c00 con 0x7f4ad015e2a0 2024-09-19T07:44:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.239+0000 7f4ad8711640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad00d4a80 con 0x7f4ad006d960 2024-09-19T07:44:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.239+0000 7f4ad8711640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad00d0010 con 0x7f4ad015e680 2024-09-19T07:44:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.239+0000 7f4ad5cac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad015e2a0 0x7f4ad01496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.239+0000 7f4ad5cac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad015e2a0 0x7f4ad01496a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60328/0 (socket says 172.21.15.16:60328) 2024-09-19T07:44:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.239+0000 7f4ad5cac640 1 -- 172.21.15.16:0/4091248433 learned_addr learned my addr 172.21.15.16:0/4091248433 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.239+0000 7f4ad64ad640 1 --2- 172.21.15.16:0/4091248433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ad006d960 0x7f4ad01ea610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.239+0000 7f4ad6cae640 1 --2- 172.21.15.16:0/4091248433 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015e680 0x7f4ad0160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.239+0000 7f4ad5cac640 1 -- 172.21.15.16:0/4091248433 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015e680 msgr2=0x7f4ad0160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.239+0000 7f4ad5cac640 1 --2- 172.21.15.16:0/4091248433 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015e680 0x7f4ad0160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.239+0000 7f4ad5cac640 1 -- 172.21.15.16:0/4091248433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ad006d960 msgr2=0x7f4ad01ea610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.240+0000 7f4ad5cac640 1 --2- 172.21.15.16:0/4091248433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ad006d960 0x7f4ad01ea610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.240+0000 7f4ad5cac640 1 -- 172.21.15.16:0/4091248433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad00cd940 con 0x7f4ad015e2a0 2024-09-19T07:44:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.240+0000 7f4ad5cac640 1 --2- 172.21.15.16:0/4091248433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad015e2a0 0x7f4ad01496a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00097a0 tx=0x7f4ac00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.240+0000 7f4ac6ffd640 1 -- 172.21.15.16:0/4091248433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0037040 con 0x7f4ad015e2a0 2024-09-19T07:44:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.240+0000 7f4ac6ffd640 1 -- 172.21.15.16:0/4091248433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac0004230 con 0x7f4ad015e2a0 2024-09-19T07:44:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.240+0000 7f4ad8711640 1 -- 172.21.15.16:0/4091248433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad00cdc20 con 0x7f4ad015e2a0 2024-09-19T07:44:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.240+0000 7f4ac6ffd640 1 -- 172.21.15.16:0/4091248433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0003610 con 0x7f4ad015e2a0 2024-09-19T07:44:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.240+0000 7f4ad8711640 1 -- 172.21.15.16:0/4091248433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad00ce1e0 con 0x7f4ad015e2a0 2024-09-19T07:44:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.241+0000 7f4ad8711640 1 -- 172.21.15.16:0/4091248433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad006d420 con 0x7f4ad015e2a0 2024-09-19T07:44:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.241+0000 7f4ac6ffd640 1 -- 172.21.15.16:0/4091248433 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4ac00037b0 con 0x7f4ad015e2a0 2024-09-19T07:44:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.242+0000 7f4ac6ffd640 1 --2- 172.21.15.16:0/4091248433 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4ab406c2e0 0x7f4ab406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.242+0000 7f4ac6ffd640 1 -- 172.21.15.16:0/4091248433 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f4ac00ac810 con 0x7f4ad015e2a0 2024-09-19T07:44:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.242+0000 7f4ad64ad640 1 --2- 172.21.15.16:0/4091248433 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4ab406c2e0 0x7f4ab406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.243+0000 7f4ad64ad640 1 --2- 172.21.15.16:0/4091248433 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4ab406c2e0 0x7f4ab406e7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4acc006ca0 tx=0x7f4acc009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:58.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.246+0000 7f4ac6ffd640 1 -- 172.21.15.16:0/4091248433 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4ac007af00 con 0x7f4ad015e2a0 2024-09-19T07:44:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.457+0000 7f4ad8711640 1 -- 172.21.15.16:0/4091248433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ad00d3ab0 con 0x7f4ad015e2a0 2024-09-19T07:44:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.457+0000 7f4ac6ffd640 1 -- 172.21.15.16:0/4091248433 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5229 (secure 0 0 0) 0x7f4ac00810d0 con 0x7f4ad015e2a0 2024-09-19T07:44:58.458 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:44:58.458 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-19T07:44:04:108666+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4655,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/3238551814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3238551814},{"type":"v1","addr":"172.21.15.146:6836","nonce":3238551814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3450384245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3450384245},{"type":"v1","addr":"172.21.15.146:6837","nonce":3450384245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4944,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3663416804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3663416804},{"type":"v1","addr":"172.21.15.16:6834","nonce":3663416804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:44:00.970677+0000","modified":"2024-09-19T07:44:03.455740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4947":{"gid":4947,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2028210099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2028210099},{"type":"v1","addr":"172.21.15.16:6835","nonce":2028210099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4947,"qdb_cluster":[4947]},"id":2}]} 2024-09-19T07:44:58.458 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-19T07:44:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.459+0000 7f4ad8711640 1 -- 172.21.15.16:0/4091248433 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4ab406c2e0 msgr2=0x7f4ab406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.459+0000 7f4ad8711640 1 --2- 172.21.15.16:0/4091248433 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4ab406c2e0 0x7f4ab406e7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4acc006ca0 tx=0x7f4acc009240 comp rx=0 tx=0).stop 2024-09-19T07:44:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.459+0000 7f4ad8711640 1 -- 172.21.15.16:0/4091248433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad015e2a0 msgr2=0x7f4ad01496a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.459+0000 7f4ad8711640 1 --2- 172.21.15.16:0/4091248433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad015e2a0 0x7f4ad01496a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00097a0 tx=0x7f4ac00046c0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.459+0000 7f4ad8711640 1 -- 172.21.15.16:0/4091248433 shutdown_connections 2024-09-19T07:44:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.459+0000 7f4ad8711640 1 --2- 172.21.15.16:0/4091248433 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4ab406c2e0 0x7f4ab406e7a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.459+0000 7f4ad8711640 1 --2- 172.21.15.16:0/4091248433 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015e680 0x7f4ad0160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.459+0000 7f4ad8711640 1 --2- 172.21.15.16:0/4091248433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad015e2a0 0x7f4ad01496a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.459+0000 7f4ad8711640 1 --2- 172.21.15.16:0/4091248433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ad006d960 0x7f4ad01ea610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.459+0000 7f4ad8711640 1 -- 172.21.15.16:0/4091248433 >> 172.21.15.16:0/4091248433 conn(0x7f4ad009d760 msgr2=0x7f4ad015f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.459+0000 7f4ad8711640 1 -- 172.21.15.16:0/4091248433 shutdown_connections 2024-09-19T07:44:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.459+0000 7f4ad8711640 1 -- 172.21.15.16:0/4091248433 wait complete. 2024-09-19T07:44:58.471 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:44:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.573+0000 7fedb314b640 1 Processor -- start 2024-09-19T07:44:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.573+0000 7fedb314b640 1 -- start start 2024-09-19T07:44:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.574+0000 7fedb314b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fedac15e2a0 0x7fedac06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.574+0000 7fedb314b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fedac15e680 0x7fedac0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.574+0000 7fedb314b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fedac06d960 0x7fedac0d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.574+0000 7fedb314b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fedac0d1880 con 0x7fedac06d960 2024-09-19T07:44:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.574+0000 7fedb314b640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fedac0d1b60 con 0x7fedac15e2a0 2024-09-19T07:44:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.574+0000 7fedb314b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fedac0cf2d0 con 0x7fedac15e680 2024-09-19T07:44:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.574+0000 7fedb16e8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fedac06d960 0x7fedac0d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.574+0000 7fedb16e8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fedac06d960 0x7fedac0d0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60336/0 (socket says 172.21.15.16:60336) 2024-09-19T07:44:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.574+0000 7fedb16e8640 1 -- 172.21.15.16:0/936019807 learned_addr learned my addr 172.21.15.16:0/936019807 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.574+0000 7fedb0ee7640 1 --2- 172.21.15.16:0/936019807 >> v2:172.21.15.146:3300/0 conn(0x7fedac15e2a0 0x7fedac06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.575+0000 7fedabfff640 1 --2- 172.21.15.16:0/936019807 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fedac15e680 0x7fedac0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.575+0000 7fedb16e8640 1 -- 172.21.15.16:0/936019807 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fedac15e680 msgr2=0x7fedac0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.575+0000 7fedb16e8640 1 --2- 172.21.15.16:0/936019807 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fedac15e680 0x7fedac0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.575+0000 7fedb16e8640 1 -- 172.21.15.16:0/936019807 >> v2:172.21.15.146:3300/0 conn(0x7fedac15e2a0 msgr2=0x7fedac06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.575+0000 7fedb16e8640 1 --2- 172.21.15.16:0/936019807 >> v2:172.21.15.146:3300/0 conn(0x7fedac15e2a0 0x7fedac06d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.575+0000 7fedb16e8640 1 -- 172.21.15.16:0/936019807 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedac0cf5b0 con 0x7fedac06d960 2024-09-19T07:44:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.575+0000 7fedb16e8640 1 --2- 172.21.15.16:0/936019807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac06d960 0x7fedac0d0010 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fedac0b6fa0 tx=0x7feda402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677d793b1f4fee74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.576+0000 7fedab7fe640 1 -- 172.21.15.16:0/936019807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4037070 con 0x7fedac06d960 2024-09-19T07:44:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.576+0000 7fedab7fe640 1 -- 172.21.15.16:0/936019807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda4032440 con 0x7fedac06d960 2024-09-19T07:44:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.576+0000 7fedab7fe640 1 -- 172.21.15.16:0/936019807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4032810 con 0x7fedac06d960 2024-09-19T07:44:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.576+0000 7fedb314b640 1 -- 172.21.15.16:0/936019807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac06d960 msgr2=0x7fedac0d0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.576+0000 7fedb314b640 1 --2- 172.21.15.16:0/936019807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac06d960 0x7fedac0d0010 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fedac0b6fa0 tx=0x7feda402a1e0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.576+0000 7fedb314b640 1 -- 172.21.15.16:0/936019807 shutdown_connections 2024-09-19T07:44:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.576+0000 7fedb314b640 1 --2- 172.21.15.16:0/936019807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac06d960 0x7fedac0d0010 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.576+0000 7fedb314b640 1 --2- 172.21.15.16:0/936019807 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fedac15e680 0x7fedac0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.576+0000 7fedb314b640 1 --2- 172.21.15.16:0/936019807 >> v2:172.21.15.146:3300/0 conn(0x7fedac15e2a0 0x7fedac06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.576+0000 7fedb314b640 1 -- 172.21.15.16:0/936019807 >> 172.21.15.16:0/936019807 conn(0x7fedac09d760 msgr2=0x7fedac08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.576+0000 7fedb314b640 1 -- 172.21.15.16:0/936019807 shutdown_connections 2024-09-19T07:44:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.576+0000 7fedb314b640 1 -- 172.21.15.16:0/936019807 wait complete. 2024-09-19T07:44:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.577+0000 7fedb314b640 1 Processor -- start 2024-09-19T07:44:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.577+0000 7fedb314b640 1 -- start start 2024-09-19T07:44:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.578+0000 7fedb314b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac06d960 0x7fedac0ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.578+0000 7fedb314b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedac15e2a0 0x7fedac0b9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.578+0000 7fedb314b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fedac15e680 0x7fedac15f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.578+0000 7fedb314b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fedac0d44f0 con 0x7fedac06d960 2024-09-19T07:44:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.578+0000 7fedb314b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fedac0d4370 con 0x7fedac15e2a0 2024-09-19T07:44:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.578+0000 7fedb314b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fedac0d4670 con 0x7fedac15e680 2024-09-19T07:44:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.578+0000 7fedb0ee7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac06d960 0x7fedac0ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.578+0000 7fedb0ee7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac06d960 0x7fedac0ca0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60348/0 (socket says 172.21.15.16:60348) 2024-09-19T07:44:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.578+0000 7fedb0ee7640 1 -- 172.21.15.16:0/677608166 learned_addr learned my addr 172.21.15.16:0/677608166 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.578+0000 7fedabfff640 1 --2- 172.21.15.16:0/677608166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedac15e2a0 0x7fedac0b9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.578+0000 7fedb16e8640 1 --2- 172.21.15.16:0/677608166 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fedac15e680 0x7fedac15f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.579+0000 7fedb0ee7640 1 -- 172.21.15.16:0/677608166 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fedac15e680 msgr2=0x7fedac15f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.579+0000 7fedb0ee7640 1 --2- 172.21.15.16:0/677608166 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fedac15e680 0x7fedac15f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.579+0000 7fedb0ee7640 1 -- 172.21.15.16:0/677608166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedac15e2a0 msgr2=0x7fedac0b9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.579+0000 7fedb0ee7640 1 --2- 172.21.15.16:0/677608166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedac15e2a0 0x7fedac0b9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.579+0000 7fedb0ee7640 1 -- 172.21.15.16:0/677608166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedac0ca680 con 0x7fedac06d960 2024-09-19T07:44:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.579+0000 7fedb0ee7640 1 --2- 172.21.15.16:0/677608166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac06d960 0x7fedac0ca0a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00d830 tx=0x7fed9c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.579+0000 7feda97fa640 1 -- 172.21.15.16:0/677608166 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c00b6c0 con 0x7fedac06d960 2024-09-19T07:44:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.579+0000 7feda97fa640 1 -- 172.21.15.16:0/677608166 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed9c019070 con 0x7fedac06d960 2024-09-19T07:44:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.579+0000 7feda97fa640 1 -- 172.21.15.16:0/677608166 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c00ba80 con 0x7fedac06d960 2024-09-19T07:44:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.579+0000 7fedb314b640 1 -- 172.21.15.16:0/677608166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedac0ca960 con 0x7fedac06d960 2024-09-19T07:44:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.579+0000 7fedb314b640 1 -- 172.21.15.16:0/677608166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedac0ba270 con 0x7fedac06d960 2024-09-19T07:44:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.580+0000 7fedb314b640 1 -- 172.21.15.16:0/677608166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedac0d4a40 con 0x7fedac06d960 2024-09-19T07:44:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.581+0000 7feda97fa640 1 -- 172.21.15.16:0/677608166 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fed9c01e460 con 0x7fedac06d960 2024-09-19T07:44:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.581+0000 7feda97fa640 1 --2- 172.21.15.16:0/677608166 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fed8c06c2e0 0x7fed8c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.581+0000 7feda97fa640 1 -- 172.21.15.16:0/677608166 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fed9c08ba90 con 0x7fedac06d960 2024-09-19T07:44:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.581+0000 7fedabfff640 1 --2- 172.21.15.16:0/677608166 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fed8c06c2e0 0x7fed8c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.582+0000 7fedabfff640 1 --2- 172.21.15.16:0/677608166 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fed8c06c2e0 0x7fed8c06e7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fedac15a650 tx=0x7fed94009830 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.585+0000 7feda97fa640 1 -- 172.21.15.16:0/677608166 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fed9c059a90 con 0x7fedac06d960 2024-09-19T07:44:58.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.763+0000 7fedb314b640 1 -- 172.21.15.16:0/677608166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fedac0b98f0 con 0x7fedac06d960 2024-09-19T07:44:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.764+0000 7feda97fa640 1 -- 172.21.15.16:0/677608166 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+17075 (secure 0 0 0) 0x7fed9c05dd90 con 0x7fedac06d960 2024-09-19T07:44:58.765 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:44:58.765 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":42,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:44:48.525842+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:43:58.882707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-19T07:43:59.960376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:44:58.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.767+0000 7fedb314b640 1 -- 172.21.15.16:0/677608166 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fed8c06c2e0 msgr2=0x7fed8c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:58.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.767+0000 7fedb314b640 1 --2- 172.21.15.16:0/677608166 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fed8c06c2e0 0x7fed8c06e7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fedac15a650 tx=0x7fed94009830 comp rx=0 tx=0).stop 2024-09-19T07:44:58.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.767+0000 7fedb314b640 1 -- 172.21.15.16:0/677608166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac06d960 msgr2=0x7fedac0ca0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:58.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.767+0000 7fedb314b640 1 --2- 172.21.15.16:0/677608166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac06d960 0x7fedac0ca0a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00d830 tx=0x7fed9c00dcf0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.767+0000 7fedb314b640 1 -- 172.21.15.16:0/677608166 shutdown_connections 2024-09-19T07:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.767+0000 7fedb314b640 1 --2- 172.21.15.16:0/677608166 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fed8c06c2e0 0x7fed8c06e7a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.767+0000 7fedb314b640 1 --2- 172.21.15.16:0/677608166 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fedac15e680 0x7fedac15f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.767+0000 7fedb314b640 1 --2- 172.21.15.16:0/677608166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedac15e2a0 0x7fedac0b9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.767+0000 7fedb314b640 1 --2- 172.21.15.16:0/677608166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac06d960 0x7fedac0ca0a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.767+0000 7fedb314b640 1 -- 172.21.15.16:0/677608166 >> 172.21.15.16:0/677608166 conn(0x7fedac09d760 msgr2=0x7fedac15d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.767+0000 7fedb314b640 1 -- 172.21.15.16:0/677608166 shutdown_connections 2024-09-19T07:44:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.767+0000 7fedb314b640 1 -- 172.21.15.16:0/677608166 wait complete. 2024-09-19T07:44:58.778 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T07:44:58.778 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:44:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.880+0000 7f952254b640 1 Processor -- start 2024-09-19T07:44:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.880+0000 7f952254b640 1 -- start start 2024-09-19T07:44:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.881+0000 7f952254b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c0ae270 0x7f951c06c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:58.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.881+0000 7f952254b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f951c0cf680 0x7f951c0cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.881+0000 7f952254b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f951c0cfda0 con 0x7f951c15c270 2024-09-19T07:44:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.881+0000 7f952254b640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f951c0ccf80 con 0x7f951c0cf680 2024-09-19T07:44:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.881+0000 7f952254b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f951c0cd260 con 0x7f951c0ae270 2024-09-19T07:44:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.881+0000 7f9520ae8640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f951c0cf680 0x7f951c0cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.881+0000 7f951b7fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c0ae270 0x7f951c06c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.881+0000 7f951bfff640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f951c15c270 0x7f951c0cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:38766/0 (socket says 172.21.15.16:38766) 2024-09-19T07:44:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.881+0000 7f9520ae8640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f951c0cf680 0x7f951c0cdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:37916/0 (socket says 172.21.15.16:37916) 2024-09-19T07:44:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.881+0000 7f951bfff640 1 -- 172.21.15.16:0/3142738382 learned_addr learned my addr 172.21.15.16:0/3142738382 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:44:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.881+0000 7f951affd640 1 -- 172.21.15.16:0/3142738382 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2092586464 0 0) 0x7f950c001110 con 0x7f951c15c270 2024-09-19T07:44:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.881+0000 7f951b7fe640 1 -- 172.21.15.16:0/3142738382 >> v2:172.21.15.146:3300/0 conn(0x7f951c0cf680 msgr2=0x7f951c0cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.881+0000 7f951b7fe640 1 --2- 172.21.15.16:0/3142738382 >> v2:172.21.15.146:3300/0 conn(0x7f951c0cf680 0x7f951c0cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.881+0000 7f951b7fe640 1 -- 172.21.15.16:0/3142738382 >> v1:172.21.15.146:6789/0 conn(0x7f951c15c270 legacy=0x7f951c0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.881+0000 7f951b7fe640 1 -- 172.21.15.16:0/3142738382 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951c0cd540 con 0x7f951c0ae270 2024-09-19T07:44:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.882+0000 7f951b7fe640 1 --2- 172.21.15.16:0/3142738382 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c0ae270 0x7f951c06c620 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f95100097a0 tx=0x7f951002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27412094ac8013fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.882+0000 7f951affd640 1 -- 172.21.15.16:0/3142738382 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9510037070 con 0x7f951c0ae270 2024-09-19T07:44:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.882+0000 7f951affd640 1 -- 172.21.15.16:0/3142738382 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f951002ae50 con 0x7f951c0ae270 2024-09-19T07:44:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.882+0000 7f951affd640 1 -- 172.21.15.16:0/3142738382 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9510032660 con 0x7f951c0ae270 2024-09-19T07:44:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.883+0000 7f952254b640 1 -- 172.21.15.16:0/3142738382 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c0ae270 msgr2=0x7f951c06c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.883+0000 7f952254b640 1 --2- 172.21.15.16:0/3142738382 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c0ae270 0x7f951c06c620 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f95100097a0 tx=0x7f951002a100 comp rx=0 tx=0).stop 2024-09-19T07:44:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.883+0000 7f952254b640 1 -- 172.21.15.16:0/3142738382 shutdown_connections 2024-09-19T07:44:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.883+0000 7f952254b640 1 --2- 172.21.15.16:0/3142738382 >> v2:172.21.15.146:3300/0 conn(0x7f951c0cf680 0x7f951c0cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.883+0000 7f952254b640 1 --2- 172.21.15.16:0/3142738382 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c0ae270 0x7f951c06c620 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.883+0000 7f952254b640 1 -- 172.21.15.16:0/3142738382 >> 172.21.15.16:0/3142738382 conn(0x7f951c09a9f0 msgr2=0x7f951c0b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.883+0000 7f952254b640 1 -- 172.21.15.16:0/3142738382 shutdown_connections 2024-09-19T07:44:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.883+0000 7f952254b640 1 -- 172.21.15.16:0/3142738382 wait complete. 2024-09-19T07:44:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.884+0000 7f952254b640 1 Processor -- start 2024-09-19T07:44:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.885+0000 7f952254b640 1 -- start start 2024-09-19T07:44:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.885+0000 7f952254b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f951c0ae270 0x7f951c06f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.885+0000 7f952254b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f951c0cf680 0x7f951c155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.885+0000 7f952254b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c15c270 0x7f951c0d1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.885+0000 7f952254b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f951c15c7d0 con 0x7f951c0ae270 2024-09-19T07:44:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.885+0000 7f952254b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f951c15c650 con 0x7f951c0cf680 2024-09-19T07:44:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.885+0000 7f952254b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f951c0ccba0 con 0x7f951c15c270 2024-09-19T07:44:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.885+0000 7f951bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f951c0ae270 0x7f951c06f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.885+0000 7f951bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f951c0ae270 0x7f951c06f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60350/0 (socket says 172.21.15.16:60350) 2024-09-19T07:44:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.885+0000 7f951bfff640 1 -- 172.21.15.16:0/2420928988 learned_addr learned my addr 172.21.15.16:0/2420928988 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.886+0000 7f9520ae8640 1 --2- 172.21.15.16:0/2420928988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c15c270 0x7f951c0d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.886+0000 7f951b7fe640 1 --2- 172.21.15.16:0/2420928988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f951c0cf680 0x7f951c155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.886+0000 7f951bfff640 1 -- 172.21.15.16:0/2420928988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c15c270 msgr2=0x7f951c0d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.886+0000 7f951bfff640 1 --2- 172.21.15.16:0/2420928988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c15c270 0x7f951c0d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.886+0000 7f951bfff640 1 -- 172.21.15.16:0/2420928988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f951c0cf680 msgr2=0x7f951c155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.886+0000 7f951bfff640 1 --2- 172.21.15.16:0/2420928988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f951c0cf680 0x7f951c155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.886+0000 7f951bfff640 1 -- 172.21.15.16:0/2420928988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951c0c96d0 con 0x7f951c0ae270 2024-09-19T07:44:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.886+0000 7f9520ae8640 1 --2- 172.21.15.16:0/2420928988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c15c270 0x7f951c0d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.886+0000 7f951bfff640 1 --2- 172.21.15.16:0/2420928988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f951c0ae270 0x7f951c06f9c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f950c00c7b0 tx=0x7f950c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.886+0000 7f9518ff9640 1 -- 172.21.15.16:0/2420928988 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c018070 con 0x7f951c0ae270 2024-09-19T07:44:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.886+0000 7f9518ff9640 1 -- 172.21.15.16:0/2420928988 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f950c0072d0 con 0x7f951c0ae270 2024-09-19T07:44:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.886+0000 7f952254b640 1 -- 172.21.15.16:0/2420928988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f951c0c99b0 con 0x7f951c0ae270 2024-09-19T07:44:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.886+0000 7f9518ff9640 1 -- 172.21.15.16:0/2420928988 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c0047e0 con 0x7f951c0ae270 2024-09-19T07:44:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.886+0000 7f952254b640 1 -- 172.21.15.16:0/2420928988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f951c0c9f70 con 0x7f951c0ae270 2024-09-19T07:44:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.887+0000 7f952254b640 1 -- 172.21.15.16:0/2420928988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f951c073230 con 0x7f951c0ae270 2024-09-19T07:44:58.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.888+0000 7f9518ff9640 1 -- 172.21.15.16:0/2420928988 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f950c01b070 con 0x7f951c0ae270 2024-09-19T07:44:58.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.888+0000 7f9518ff9640 1 --2- 172.21.15.16:0/2420928988 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f950806c2b0 0x7f950806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:58.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.888+0000 7f9518ff9640 1 -- 172.21.15.16:0/2420928988 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f950c08a800 con 0x7f951c0ae270 2024-09-19T07:44:58.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.889+0000 7f951b7fe640 1 --2- 172.21.15.16:0/2420928988 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f950806c2b0 0x7f950806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:58.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.889+0000 7f951b7fe640 1 --2- 172.21.15.16:0/2420928988 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f950806c2b0 0x7f950806e770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9510034000 tx=0x7f951002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:58.893+0000 7f9518ff9640 1 -- 172.21.15.16:0/2420928988 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f951c073230 con 0x7f951c0ae270 2024-09-19T07:44:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.103+0000 7f952254b640 1 -- 172.21.15.16:0/2420928988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f951c06c7a0 con 0x7f951c0ae270 2024-09-19T07:44:59.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.103+0000 7f9518ff9640 1 -- 172.21.15.16:0/2420928988 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5229 (secure 0 0 0) 0x7f951c0cfda0 con 0x7f951c0ae270 2024-09-19T07:44:59.104 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:44:59.104 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-19T07:44:04:108666+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4655,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/3238551814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3238551814},{"type":"v1","addr":"172.21.15.146:6836","nonce":3238551814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3450384245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3450384245},{"type":"v1","addr":"172.21.15.146:6837","nonce":3450384245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4944,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3663416804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3663416804},{"type":"v1","addr":"172.21.15.16:6834","nonce":3663416804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:44:00.970677+0000","modified":"2024-09-19T07:44:03.455740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4947":{"gid":4947,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2028210099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2028210099},{"type":"v1","addr":"172.21.15.16:6835","nonce":2028210099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4947,"qdb_cluster":[4947]},"id":2}]} 2024-09-19T07:44:59.104 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-19T07:44:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.106+0000 7f952254b640 1 -- 172.21.15.16:0/2420928988 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f950806c2b0 msgr2=0x7f950806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.106+0000 7f952254b640 1 --2- 172.21.15.16:0/2420928988 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f950806c2b0 0x7f950806e770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9510034000 tx=0x7f951002e000 comp rx=0 tx=0).stop 2024-09-19T07:44:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.106+0000 7f952254b640 1 -- 172.21.15.16:0/2420928988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f951c0ae270 msgr2=0x7f951c06f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.106+0000 7f952254b640 1 --2- 172.21.15.16:0/2420928988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f951c0ae270 0x7f951c06f9c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f950c00c7b0 tx=0x7f950c00cc70 comp rx=0 tx=0).stop 2024-09-19T07:44:59.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.106+0000 7f952254b640 1 -- 172.21.15.16:0/2420928988 shutdown_connections 2024-09-19T07:44:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.106+0000 7f952254b640 1 --2- 172.21.15.16:0/2420928988 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f950806c2b0 0x7f950806e770 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.106+0000 7f952254b640 1 --2- 172.21.15.16:0/2420928988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f951c15c270 0x7f951c0d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.106+0000 7f952254b640 1 --2- 172.21.15.16:0/2420928988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f951c0cf680 0x7f951c155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.106+0000 7f952254b640 1 --2- 172.21.15.16:0/2420928988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f951c0ae270 0x7f951c06f9c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.106+0000 7f952254b640 1 -- 172.21.15.16:0/2420928988 >> 172.21.15.16:0/2420928988 conn(0x7f951c09a9f0 msgr2=0x7f951c0b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.106+0000 7f952254b640 1 -- 172.21.15.16:0/2420928988 shutdown_connections 2024-09-19T07:44:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.106+0000 7f952254b640 1 -- 172.21.15.16:0/2420928988 wait complete. 2024-09-19T07:44:59.117 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:44:59.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.219+0000 7f5a9159f640 1 Processor -- start 2024-09-19T07:44:59.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.219+0000 7f5a9159f640 1 -- start start 2024-09-19T07:44:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.219+0000 7f5a9159f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a8c15e2f0 0x7f5a8c0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.219+0000 7f5a9159f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5a8c15e6d0 0x7f5a8c160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.219+0000 7f5a9159f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a8c15ead0 0x7f5a8c1614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.219+0000 7f5a9159f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f5a8c0b5dd0 con 0x7f5a8c15ead0 2024-09-19T07:44:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.219+0000 7f5a9159f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5a8c0b6050 con 0x7f5a8c15e6d0 2024-09-19T07:44:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.219+0000 7f5a9159f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5a8c0b6330 con 0x7f5a8c15e2f0 2024-09-19T07:44:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.220+0000 7f5a90d9e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a8c15ead0 0x7f5a8c1614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.220+0000 7f5a90d9e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a8c15ead0 0x7f5a8c1614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60352/0 (socket says 172.21.15.16:60352) 2024-09-19T07:44:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.220+0000 7f5a90d9e640 1 -- 172.21.15.16:0/2876389167 learned_addr learned my addr 172.21.15.16:0/2876389167 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.220+0000 7f5a8bfff640 1 --2- 172.21.15.16:0/2876389167 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a8c15e2f0 0x7f5a8c0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.220+0000 7f5a8b7fe640 1 --2- 172.21.15.16:0/2876389167 >> v2:172.21.15.146:3300/0 conn(0x7f5a8c15e6d0 0x7f5a8c160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.220+0000 7f5a90d9e640 1 -- 172.21.15.16:0/2876389167 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a8c15e2f0 msgr2=0x7f5a8c0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.220+0000 7f5a90d9e640 1 --2- 172.21.15.16:0/2876389167 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a8c15e2f0 0x7f5a8c0b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.220+0000 7f5a90d9e640 1 -- 172.21.15.16:0/2876389167 >> v2:172.21.15.146:3300/0 conn(0x7f5a8c15e6d0 msgr2=0x7f5a8c160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.220+0000 7f5a90d9e640 1 --2- 172.21.15.16:0/2876389167 >> v2:172.21.15.146:3300/0 conn(0x7f5a8c15e6d0 0x7f5a8c160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.220+0000 7f5a90d9e640 1 -- 172.21.15.16:0/2876389167 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a8c0b6610 con 0x7f5a8c15ead0 2024-09-19T07:44:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.220+0000 7f5a8b7fe640 1 --2- 172.21.15.16:0/2876389167 >> v2:172.21.15.146:3300/0 conn(0x7f5a8c15e6d0 0x7f5a8c160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.220+0000 7f5a8bfff640 1 --2- 172.21.15.16:0/2876389167 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a8c15e2f0 0x7f5a8c0b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.221+0000 7f5a90d9e640 1 --2- 172.21.15.16:0/2876389167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a8c15ead0 0x7f5a8c1614a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c073730 tx=0x7f5a8402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e841db9cd7a520 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:59.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.221+0000 7f5a8affd640 1 -- 172.21.15.16:0/2876389167 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a84036070 con 0x7f5a8c15ead0 2024-09-19T07:44:59.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.221+0000 7f5a8affd640 1 -- 172.21.15.16:0/2876389167 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a84031600 con 0x7f5a8c15ead0 2024-09-19T07:44:59.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.221+0000 7f5a8affd640 1 -- 172.21.15.16:0/2876389167 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a840319d0 con 0x7f5a8c15ead0 2024-09-19T07:44:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.221+0000 7f5a9159f640 1 -- 172.21.15.16:0/2876389167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a8c15ead0 msgr2=0x7f5a8c1614a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.221+0000 7f5a9159f640 1 --2- 172.21.15.16:0/2876389167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a8c15ead0 0x7f5a8c1614a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c073730 tx=0x7f5a8402a730 comp rx=0 tx=0).stop 2024-09-19T07:44:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.221+0000 7f5a9159f640 1 -- 172.21.15.16:0/2876389167 shutdown_connections 2024-09-19T07:44:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.221+0000 7f5a9159f640 1 --2- 172.21.15.16:0/2876389167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a8c15ead0 0x7f5a8c1614a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.221+0000 7f5a9159f640 1 --2- 172.21.15.16:0/2876389167 >> v2:172.21.15.146:3300/0 conn(0x7f5a8c15e6d0 0x7f5a8c160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.221+0000 7f5a9159f640 1 --2- 172.21.15.16:0/2876389167 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a8c15e2f0 0x7f5a8c0b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.221+0000 7f5a9159f640 1 -- 172.21.15.16:0/2876389167 >> 172.21.15.16:0/2876389167 conn(0x7f5a8c0d2030 msgr2=0x7f5a8c0d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.221+0000 7f5a9159f640 1 -- 172.21.15.16:0/2876389167 shutdown_connections 2024-09-19T07:44:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.221+0000 7f5a9159f640 1 -- 172.21.15.16:0/2876389167 wait complete. 2024-09-19T07:44:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.223+0000 7f5a9159f640 1 Processor -- start 2024-09-19T07:44:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.223+0000 7f5a9159f640 1 -- start start 2024-09-19T07:44:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.223+0000 7f5a9159f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a8c15e2f0 0x7f5a8c0ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.223+0000 7f5a9159f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a8c15e6d0 0x7f5a8c0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a9159f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a8c15ead0 0x7f5a8c1513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a9159f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c1619e0 con 0x7f5a8c15ead0 2024-09-19T07:44:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a9159f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c0b6bd0 con 0x7f5a8c15e6d0 2024-09-19T07:44:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a9159f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5a8c161b60 con 0x7f5a8c15e2f0 2024-09-19T07:44:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a90d9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a8c15ead0 0x7f5a8c1513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a90d9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a8c15ead0 0x7f5a8c1513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60354/0 (socket says 172.21.15.16:60354) 2024-09-19T07:44:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a90d9e640 1 -- 172.21.15.16:0/342193336 learned_addr learned my addr 172.21.15.16:0/342193336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a8bfff640 1 --2- 172.21.15.16:0/342193336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a8c15e2f0 0x7f5a8c0ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a8b7fe640 1 --2- 172.21.15.16:0/342193336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a8c15e6d0 0x7f5a8c0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a90d9e640 1 -- 172.21.15.16:0/342193336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a8c15e2f0 msgr2=0x7f5a8c0ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a90d9e640 1 --2- 172.21.15.16:0/342193336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a8c15e2f0 0x7f5a8c0ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a90d9e640 1 -- 172.21.15.16:0/342193336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a8c15e6d0 msgr2=0x7f5a8c0cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a90d9e640 1 --2- 172.21.15.16:0/342193336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a8c15e6d0 0x7f5a8c0cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a90d9e640 1 -- 172.21.15.16:0/342193336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a8c0ce830 con 0x7f5a8c15ead0 2024-09-19T07:44:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a90d9e640 1 --2- 172.21.15.16:0/342193336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a8c15ead0 0x7f5a8c1513c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5a84034040 tx=0x7f5a84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a88ff9640 1 -- 172.21.15.16:0/342193336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a84041020 con 0x7f5a8c15ead0 2024-09-19T07:44:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a88ff9640 1 -- 172.21.15.16:0/342193336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a840077a0 con 0x7f5a8c15ead0 2024-09-19T07:44:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a9159f640 1 -- 172.21.15.16:0/342193336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a8c0ceb10 con 0x7f5a8c15ead0 2024-09-19T07:44:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a88ff9640 1 -- 172.21.15.16:0/342193336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a84036040 con 0x7f5a8c15ead0 2024-09-19T07:44:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.224+0000 7f5a9159f640 1 -- 172.21.15.16:0/342193336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a8c1f4050 con 0x7f5a8c15ead0 2024-09-19T07:44:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.225+0000 7f5a9159f640 1 -- 172.21.15.16:0/342193336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a8c0b5890 con 0x7f5a8c15ead0 2024-09-19T07:44:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.226+0000 7f5a88ff9640 1 -- 172.21.15.16:0/342193336 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5a84044020 con 0x7f5a8c15ead0 2024-09-19T07:44:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.226+0000 7f5a88ff9640 1 --2- 172.21.15.16:0/342193336 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a7806c2e0 0x7f5a7806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.226+0000 7f5a88ff9640 1 -- 172.21.15.16:0/342193336 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f5a840ac0e0 con 0x7f5a8c15ead0 2024-09-19T07:44:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.227+0000 7f5a8bfff640 1 --2- 172.21.15.16:0/342193336 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a7806c2e0 0x7f5a7806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.227+0000 7f5a8bfff640 1 --2- 172.21.15.16:0/342193336 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a7806c2e0 0x7f5a7806e7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c15d330 tx=0x7f5a7c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:59.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.231+0000 7f5a88ff9640 1 -- 172.21.15.16:0/342193336 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5a8404b030 con 0x7f5a8c15ead0 2024-09-19T07:44:59.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.441+0000 7f5a9159f640 1 -- 172.21.15.16:0/342193336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a8c0d0b70 con 0x7f5a8c15ead0 2024-09-19T07:44:59.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.441+0000 7f5a88ff9640 1 -- 172.21.15.16:0/342193336 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5229 (secure 0 0 0) 0x7f5a8407a160 con 0x7f5a8c15ead0 2024-09-19T07:44:59.442 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:44:59.442 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"btime":"2024-09-19T07:44:04:108666+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4655,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/3238551814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3238551814},{"type":"v1","addr":"172.21.15.146:6836","nonce":3238551814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3450384245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3450384245},{"type":"v1","addr":"172.21.15.146:6837","nonce":3450384245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4944,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3663416804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3663416804},{"type":"v1","addr":"172.21.15.16:6834","nonce":3663416804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:44:00.970677+0000","modified":"2024-09-19T07:44:03.455740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4947},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4947":{"gid":4947,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2028210099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2028210099},{"type":"v1","addr":"172.21.15.16:6835","nonce":2028210099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4947,"qdb_cluster":[4947]},"id":2}]} 2024-09-19T07:44:59.443 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-09-19T07:44:59.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.444+0000 7f5a9159f640 1 -- 172.21.15.16:0/342193336 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a7806c2e0 msgr2=0x7f5a7806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:59.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.444+0000 7f5a9159f640 1 --2- 172.21.15.16:0/342193336 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a7806c2e0 0x7f5a7806e7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c15d330 tx=0x7f5a7c008040 comp rx=0 tx=0).stop 2024-09-19T07:44:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.444+0000 7f5a9159f640 1 -- 172.21.15.16:0/342193336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a8c15ead0 msgr2=0x7f5a8c1513c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.444+0000 7f5a9159f640 1 --2- 172.21.15.16:0/342193336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a8c15ead0 0x7f5a8c1513c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5a84034040 tx=0x7f5a84004a40 comp rx=0 tx=0).stop 2024-09-19T07:44:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.444+0000 7f5a9159f640 1 -- 172.21.15.16:0/342193336 shutdown_connections 2024-09-19T07:44:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.444+0000 7f5a9159f640 1 --2- 172.21.15.16:0/342193336 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a7806c2e0 0x7f5a7806e7a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.444+0000 7f5a9159f640 1 --2- 172.21.15.16:0/342193336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a8c15ead0 0x7f5a8c1513c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.444+0000 7f5a9159f640 1 --2- 172.21.15.16:0/342193336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a8c15e6d0 0x7f5a8c0cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.444+0000 7f5a9159f640 1 --2- 172.21.15.16:0/342193336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a8c15e2f0 0x7f5a8c0ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.444+0000 7f5a9159f640 1 -- 172.21.15.16:0/342193336 >> 172.21.15.16:0/342193336 conn(0x7f5a8c0d2030 msgr2=0x7f5a8c070930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.444+0000 7f5a9159f640 1 -- 172.21.15.16:0/342193336 shutdown_connections 2024-09-19T07:44:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.444+0000 7f5a9159f640 1 -- 172.21.15.16:0/342193336 wait complete. 2024-09-19T07:44:59.456 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:44:59.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.557+0000 7f5a71bd6640 1 Processor -- start 2024-09-19T07:44:59.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.558+0000 7f5a71bd6640 1 -- start start 2024-09-19T07:44:59.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.558+0000 7f5a71bd6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a6c15f0d0 0x7f5a6c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.558+0000 7f5a71bd6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6c0d0070 0x7f5a6c0d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.558+0000 7f5a71bd6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5a6c0d1940 0x7f5a6c0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.558+0000 7f5a71bd6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f5a6c0cf2b0 con 0x7f5a6c15f0d0 2024-09-19T07:44:59.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.558+0000 7f5a71bd6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5a6c0cf590 con 0x7f5a6c0d1940 2024-09-19T07:44:59.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.558+0000 7f5a71bd6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5a6c0cf870 con 0x7f5a6c0d0070 2024-09-19T07:44:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.559+0000 7f5a6b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a6c15f0d0 0x7f5a6c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.559+0000 7f5a6b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a6c15f0d0 0x7f5a6c15f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60362/0 (socket says 172.21.15.16:60362) 2024-09-19T07:44:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.559+0000 7f5a6b7fe640 1 -- 172.21.15.16:0/1700796165 learned_addr learned my addr 172.21.15.16:0/1700796165 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.559+0000 7f5a6affd640 1 --2- 172.21.15.16:0/1700796165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6c0d0070 0x7f5a6c0d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.559+0000 7f5a6bfff640 1 --2- 172.21.15.16:0/1700796165 >> v2:172.21.15.146:3300/0 conn(0x7f5a6c0d1940 0x7f5a6c0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.559+0000 7f5a6b7fe640 1 -- 172.21.15.16:0/1700796165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6c0d0070 msgr2=0x7f5a6c0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.559+0000 7f5a6b7fe640 1 --2- 172.21.15.16:0/1700796165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6c0d0070 0x7f5a6c0d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.559+0000 7f5a6b7fe640 1 -- 172.21.15.16:0/1700796165 >> v2:172.21.15.146:3300/0 conn(0x7f5a6c0d1940 msgr2=0x7f5a6c0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.559+0000 7f5a6b7fe640 1 --2- 172.21.15.16:0/1700796165 >> v2:172.21.15.146:3300/0 conn(0x7f5a6c0d1940 0x7f5a6c0d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.559+0000 7f5a6b7fe640 1 -- 172.21.15.16:0/1700796165 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c0cfb50 con 0x7f5a6c15f0d0 2024-09-19T07:44:59.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.559+0000 7f5a6bfff640 1 --2- 172.21.15.16:0/1700796165 >> v2:172.21.15.146:3300/0 conn(0x7f5a6c0d1940 0x7f5a6c0d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:59.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.560+0000 7f5a6b7fe640 1 --2- 172.21.15.16:0/1700796165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c15f0d0 0x7f5a6c15f4b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c161e50 tx=0x7f5a5c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e728b2f4a55dab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:59.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.560+0000 7f5a6a7fc640 1 -- 172.21.15.16:0/1700796165 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c037070 con 0x7f5a6c15f0d0 2024-09-19T07:44:59.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.560+0000 7f5a6a7fc640 1 -- 172.21.15.16:0/1700796165 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a5c032440 con 0x7f5a6c15f0d0 2024-09-19T07:44:59.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.560+0000 7f5a6a7fc640 1 -- 172.21.15.16:0/1700796165 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c032810 con 0x7f5a6c15f0d0 2024-09-19T07:44:59.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.560+0000 7f5a71bd6640 1 -- 172.21.15.16:0/1700796165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c15f0d0 msgr2=0x7f5a6c15f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:59.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.560+0000 7f5a71bd6640 1 --2- 172.21.15.16:0/1700796165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c15f0d0 0x7f5a6c15f4b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c161e50 tx=0x7f5a5c02a140 comp rx=0 tx=0).stop 2024-09-19T07:44:59.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.560+0000 7f5a71bd6640 1 -- 172.21.15.16:0/1700796165 shutdown_connections 2024-09-19T07:44:59.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.560+0000 7f5a71bd6640 1 --2- 172.21.15.16:0/1700796165 >> v2:172.21.15.146:3300/0 conn(0x7f5a6c0d1940 0x7f5a6c0d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.560+0000 7f5a71bd6640 1 --2- 172.21.15.16:0/1700796165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6c0d0070 0x7f5a6c0d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.560+0000 7f5a71bd6640 1 --2- 172.21.15.16:0/1700796165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c15f0d0 0x7f5a6c15f4b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.560+0000 7f5a71bd6640 1 -- 172.21.15.16:0/1700796165 >> 172.21.15.16:0/1700796165 conn(0x7f5a6c09d760 msgr2=0x7f5a6c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:59.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.560+0000 7f5a71bd6640 1 -- 172.21.15.16:0/1700796165 shutdown_connections 2024-09-19T07:44:59.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.560+0000 7f5a71bd6640 1 -- 172.21.15.16:0/1700796165 wait complete. 2024-09-19T07:44:59.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.562+0000 7f5a71bd6640 1 Processor -- start 2024-09-19T07:44:59.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.562+0000 7f5a71bd6640 1 -- start start 2024-09-19T07:44:59.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.562+0000 7f5a71bd6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c0d0070 0x7f5a6c1f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.562+0000 7f5a71bd6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a6c0d1940 0x7f5a6c1f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a71bd6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6c15f0d0 0x7f5a6c160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a71bd6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c0d47f0 con 0x7f5a6c0d0070 2024-09-19T07:44:59.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a71bd6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c0d4670 con 0x7f5a6c0d1940 2024-09-19T07:44:59.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a71bd6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5a6c0d4970 con 0x7f5a6c15f0d0 2024-09-19T07:44:59.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c0d0070 0x7f5a6c1f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c0d0070 0x7f5a6c1f4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60378/0 (socket says 172.21.15.16:60378) 2024-09-19T07:44:59.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a6b7fe640 1 -- 172.21.15.16:0/3622479956 learned_addr learned my addr 172.21.15.16:0/3622479956 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:59.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a6affd640 1 --2- 172.21.15.16:0/3622479956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a6c0d1940 0x7f5a6c1f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a6bfff640 1 --2- 172.21.15.16:0/3622479956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6c15f0d0 0x7f5a6c160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a6b7fe640 1 -- 172.21.15.16:0/3622479956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6c15f0d0 msgr2=0x7f5a6c160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:59.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a6b7fe640 1 --2- 172.21.15.16:0/3622479956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6c15f0d0 0x7f5a6c160500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a6b7fe640 1 -- 172.21.15.16:0/3622479956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a6c0d1940 msgr2=0x7f5a6c1f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:59.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a6b7fe640 1 --2- 172.21.15.16:0/3622479956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a6c0d1940 0x7f5a6c1f4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a6b7fe640 1 -- 172.21.15.16:0/3622479956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c1f5070 con 0x7f5a6c0d0070 2024-09-19T07:44:59.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a6affd640 1 --2- 172.21.15.16:0/3622479956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a6c0d1940 0x7f5a6c1f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:59.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a6bfff640 1 --2- 172.21.15.16:0/3622479956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6c15f0d0 0x7f5a6c160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:59.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a6b7fe640 1 --2- 172.21.15.16:0/3622479956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c0d0070 0x7f5a6c1f4520 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c02a180 tx=0x7f5a5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:59.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a53fff640 1 -- 172.21.15.16:0/3622479956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c037040 con 0x7f5a6c0d0070 2024-09-19T07:44:59.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a53fff640 1 -- 172.21.15.16:0/3622479956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a5c042070 con 0x7f5a6c0d0070 2024-09-19T07:44:59.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a71bd6640 1 -- 172.21.15.16:0/3622479956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6c1f5350 con 0x7f5a6c0d0070 2024-09-19T07:44:59.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a53fff640 1 -- 172.21.15.16:0/3622479956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c030880 con 0x7f5a6c0d0070 2024-09-19T07:44:59.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.563+0000 7f5a71bd6640 1 -- 172.21.15.16:0/3622479956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6c0cdac0 con 0x7f5a6c0d0070 2024-09-19T07:44:59.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.564+0000 7f5a71bd6640 1 -- 172.21.15.16:0/3622479956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a6c1f5520 con 0x7f5a6c0d0070 2024-09-19T07:44:59.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.565+0000 7f5a53fff640 1 -- 172.21.15.16:0/3622479956 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5a5c030a20 con 0x7f5a6c0d0070 2024-09-19T07:44:59.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.565+0000 7f5a53fff640 1 --2- 172.21.15.16:0/3622479956 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a5806c2e0 0x7f5a5806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.565+0000 7f5a53fff640 1 -- 172.21.15.16:0/3622479956 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f5a5c0abe30 con 0x7f5a6c0d0070 2024-09-19T07:44:59.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.566+0000 7f5a6affd640 1 --2- 172.21.15.16:0/3622479956 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a5806c2e0 0x7f5a5806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.566+0000 7f5a6affd640 1 --2- 172.21.15.16:0/3622479956 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a5806c2e0 0x7f5a5806e7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c15dc50 tx=0x7f5a60008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:59.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.570+0000 7f5a53fff640 1 -- 172.21.15.16:0/3622479956 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5a5c07a3e0 con 0x7f5a6c0d0070 2024-09-19T07:44:59.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.747+0000 7f5a71bd6640 1 -- 172.21.15.16:0/3622479956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a6c06e0b0 con 0x7f5a6c0d0070 2024-09-19T07:44:59.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.748+0000 7f5a53fff640 1 -- 172.21.15.16:0/3622479956 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+17075 (secure 0 0 0) 0x7f5a5c079ce0 con 0x7f5a6c0d0070 2024-09-19T07:44:59.748 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:44:59.749 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":42,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:44:48.525842+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:43:58.882707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-19T07:43:59.960376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:44:59.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.750+0000 7f5a71bd6640 1 -- 172.21.15.16:0/3622479956 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a5806c2e0 msgr2=0x7f5a5806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.750+0000 7f5a71bd6640 1 --2- 172.21.15.16:0/3622479956 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a5806c2e0 0x7f5a5806e7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c15dc50 tx=0x7f5a60008040 comp rx=0 tx=0).stop 2024-09-19T07:44:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.750+0000 7f5a71bd6640 1 -- 172.21.15.16:0/3622479956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c0d0070 msgr2=0x7f5a6c1f4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.750+0000 7f5a71bd6640 1 --2- 172.21.15.16:0/3622479956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c0d0070 0x7f5a6c1f4520 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c02a180 tx=0x7f5a5c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.750+0000 7f5a71bd6640 1 -- 172.21.15.16:0/3622479956 shutdown_connections 2024-09-19T07:44:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.750+0000 7f5a71bd6640 1 --2- 172.21.15.16:0/3622479956 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a5806c2e0 0x7f5a5806e7a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.750+0000 7f5a71bd6640 1 --2- 172.21.15.16:0/3622479956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6c15f0d0 0x7f5a6c160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.750+0000 7f5a71bd6640 1 --2- 172.21.15.16:0/3622479956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a6c0d1940 0x7f5a6c1f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.750+0000 7f5a71bd6640 1 --2- 172.21.15.16:0/3622479956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c0d0070 0x7f5a6c1f4520 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.750+0000 7f5a71bd6640 1 -- 172.21.15.16:0/3622479956 >> 172.21.15.16:0/3622479956 conn(0x7f5a6c09d760 msgr2=0x7f5a6c15c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.750+0000 7f5a71bd6640 1 -- 172.21.15.16:0/3622479956 shutdown_connections 2024-09-19T07:44:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.750+0000 7f5a71bd6640 1 -- 172.21.15.16:0/3622479956 wait complete. 2024-09-19T07:44:59.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T07:44:59.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.864+0000 7f5f9fc77640 1 Processor -- start 2024-09-19T07:44:59.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.864+0000 7f5f9fc77640 1 -- start start 2024-09-19T07:44:59.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.865+0000 7f5f9fc77640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5f98072590 0x7f5f9806d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.865+0000 7f5f9fc77640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f98141ad0 0x7f5f98141f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.865+0000 7f5f9fc77640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f981620b0 con 0x7f5f980d21f0 2024-09-19T07:44:59.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.865+0000 7f5f9fc77640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5f98141750 con 0x7f5f98072590 2024-09-19T07:44:59.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.865+0000 7f5f9fc77640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5f98162370 con 0x7f5f98141ad0 2024-09-19T07:44:59.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.865+0000 7f5f9da13640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5f980d21f0 0x7f5f980d25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43996/0 (socket says 172.21.15.16:43996) 2024-09-19T07:44:59.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.865+0000 7f5f9da13640 1 -- 172.21.15.16:0/2644997813 learned_addr learned my addr 172.21.15.16:0/2644997813 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:44:59.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.865+0000 7f5f9d212640 1 --2- 172.21.15.16:0/2644997813 >> v2:172.21.15.146:3300/0 conn(0x7f5f98072590 0x7f5f9806d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.865+0000 7f5f9e214640 1 --2- 172.21.15.16:0/2644997813 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f98141ad0 0x7f5f98141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.865+0000 7f5f9ca11640 1 -- 172.21.15.16:0/2644997813 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3895296621 0 0) 0x7f5f981620b0 con 0x7f5f980d21f0 2024-09-19T07:44:59.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.865+0000 7f5f9ca11640 1 -- 172.21.15.16:0/2644997813 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f80003520 con 0x7f5f980d21f0 2024-09-19T07:44:59.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9ca11640 1 -- 172.21.15.16:0/2644997813 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1040536080 0 0) 0x7f5f80003520 con 0x7f5f980d21f0 2024-09-19T07:44:59.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9ca11640 1 -- 172.21.15.16:0/2644997813 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f98141ad0 msgr2=0x7f5f98141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:59.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9ca11640 1 --2- 172.21.15.16:0/2644997813 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f98141ad0 0x7f5f98141f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9ca11640 1 -- 172.21.15.16:0/2644997813 >> v2:172.21.15.146:3300/0 conn(0x7f5f98072590 msgr2=0x7f5f9806d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:59.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9ca11640 1 --2- 172.21.15.16:0/2644997813 >> v2:172.21.15.146:3300/0 conn(0x7f5f98072590 0x7f5f9806d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9ca11640 1 -- 172.21.15.16:0/2644997813 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f98162650 con 0x7f5f980d21f0 2024-09-19T07:44:59.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9ca11640 1 -- 172.21.15.16:0/2644997813 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f5f880019f0 con 0x7f5f980d21f0 2024-09-19T07:44:59.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9ca11640 1 -- 172.21.15.16:0/2644997813 >> v1:172.21.15.16:6789/0 conn(0x7f5f980d21f0 legacy=0x7f5f980d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:59.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9ca11640 1 --2- 172.21.15.16:0/2644997813 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f98141ad0 0x7f5f800039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9ca11640 1 --2- 172.21.15.16:0/2644997813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f80003f30 0x7f5f80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9ca11640 1 --2- 172.21.15.16:0/2644997813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f80004d40 0x7f5f800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9ca11640 1 -- 172.21.15.16:0/2644997813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f98141750 con 0x7f5f80004d40 2024-09-19T07:44:59.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9ca11640 1 -- 172.21.15.16:0/2644997813 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5f98162370 con 0x7f5f80003f30 2024-09-19T07:44:59.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9ca11640 1 -- 172.21.15.16:0/2644997813 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5f8000ba10 con 0x7f5f98141ad0 2024-09-19T07:44:59.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9e214640 1 --2- 172.21.15.16:0/2644997813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f80004d40 0x7f5f800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9da13640 1 --2- 172.21.15.16:0/2644997813 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f98141ad0 0x7f5f800039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9d212640 1 --2- 172.21.15.16:0/2644997813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f80003f30 0x7f5f80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9e214640 1 -- 172.21.15.16:0/2644997813 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f98141ad0 msgr2=0x7f5f800039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:59.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9e214640 1 --2- 172.21.15.16:0/2644997813 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f98141ad0 0x7f5f800039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9e214640 1 -- 172.21.15.16:0/2644997813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f80003f30 msgr2=0x7f5f80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:59.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9e214640 1 --2- 172.21.15.16:0/2644997813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f80003f30 0x7f5f80004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.866+0000 7f5f9e214640 1 -- 172.21.15.16:0/2644997813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f80003520 con 0x7f5f80004d40 2024-09-19T07:44:59.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.867+0000 7f5f9e214640 1 --2- 172.21.15.16:0/2644997813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f80004d40 0x7f5f800051c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009af0 tx=0x7f5f80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf655cdad200691 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:59.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.867+0000 7f5f9ca11640 1 -- 172.21.15.16:0/2644997813 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90036070 con 0x7f5f80004d40 2024-09-19T07:44:59.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.867+0000 7f5f9ca11640 1 -- 172.21.15.16:0/2644997813 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f90031400 con 0x7f5f80004d40 2024-09-19T07:44:59.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.867+0000 7f5f9ca11640 1 -- 172.21.15.16:0/2644997813 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90009690 con 0x7f5f80004d40 2024-09-19T07:44:59.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.868+0000 7f5f9fc77640 1 -- 172.21.15.16:0/2644997813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f80004d40 msgr2=0x7f5f800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:59.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.868+0000 7f5f9fc77640 1 --2- 172.21.15.16:0/2644997813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f80004d40 0x7f5f800051c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009af0 tx=0x7f5f80005830 comp rx=0 tx=0).stop 2024-09-19T07:44:59.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.868+0000 7f5f9e214640 1 -- 172.21.15.16:0/2644997813 reap_dead start 2024-09-19T07:44:59.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.868+0000 7f5f9fc77640 1 -- 172.21.15.16:0/2644997813 shutdown_connections 2024-09-19T07:44:59.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.868+0000 7f5f9fc77640 1 -- 172.21.15.16:0/2644997813 >> 172.21.15.16:0/2644997813 conn(0x7f5f980b3ed0 msgr2=0x7f5f980b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:59.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.868+0000 7f5f9fc77640 1 -- 172.21.15.16:0/2644997813 shutdown_connections 2024-09-19T07:44:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.868+0000 7f5f9fc77640 1 -- 172.21.15.16:0/2644997813 wait complete. 2024-09-19T07:44:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.869+0000 7f5f9fc77640 1 Processor -- start 2024-09-19T07:44:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.869+0000 7f5f9fc77640 1 -- start start 2024-09-19T07:44:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.870+0000 7f5f9fc77640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f80004d40 0x7f5f981f3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.870+0000 7f5f9fc77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f9815c9e0 0x7f5f98155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.870+0000 7f5f9fc77640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f9815ce60 0x7f5f98156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.870+0000 7f5f9fc77640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9806d620 con 0x7f5f9815c9e0 2024-09-19T07:44:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.870+0000 7f5f9fc77640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5f98162c10 con 0x7f5f80004d40 2024-09-19T07:44:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.870+0000 7f5f9fc77640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5f9806d7a0 con 0x7f5f9815ce60 2024-09-19T07:44:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.870+0000 7f5f9d212640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f9815c9e0 0x7f5f98155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.870+0000 7f5f9d212640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f9815c9e0 0x7f5f98155e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60390/0 (socket says 172.21.15.16:60390) 2024-09-19T07:44:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.870+0000 7f5f9d212640 1 -- 172.21.15.16:0/779415056 learned_addr learned my addr 172.21.15.16:0/779415056 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:44:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.870+0000 7f5f9da13640 1 --2- 172.21.15.16:0/779415056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f80004d40 0x7f5f981f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.870+0000 7f5f9e214640 1 --2- 172.21.15.16:0/779415056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f9815ce60 0x7f5f98156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.870+0000 7f5f9d212640 1 -- 172.21.15.16:0/779415056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f9815ce60 msgr2=0x7f5f98156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:59.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.870+0000 7f5f9d212640 1 --2- 172.21.15.16:0/779415056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f9815ce60 0x7f5f98156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.870+0000 7f5f9d212640 1 -- 172.21.15.16:0/779415056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f80004d40 msgr2=0x7f5f981f3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:59.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.870+0000 7f5f9d212640 1 --2- 172.21.15.16:0/779415056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f80004d40 0x7f5f981f3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:59.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.870+0000 7f5f9d212640 1 -- 172.21.15.16:0/779415056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9815bc60 con 0x7f5f9815c9e0 2024-09-19T07:44:59.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.870+0000 7f5f9d212640 1 --2- 172.21.15.16:0/779415056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f9815c9e0 0x7f5f98155e40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c011960 tx=0x7f5f8c011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:59.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.871+0000 7f5f967fc640 1 -- 172.21.15.16:0/779415056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c00f9d0 con 0x7f5f9815c9e0 2024-09-19T07:44:59.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.871+0000 7f5f967fc640 1 -- 172.21.15.16:0/779415056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f8c00fb70 con 0x7f5f9815c9e0 2024-09-19T07:44:59.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.871+0000 7f5f967fc640 1 -- 172.21.15.16:0/779415056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c014440 con 0x7f5f9815c9e0 2024-09-19T07:44:59.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.871+0000 7f5f9fc77640 1 -- 172.21.15.16:0/779415056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f98152120 con 0x7f5f9815c9e0 2024-09-19T07:44:59.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.871+0000 7f5f9fc77640 1 -- 172.21.15.16:0/779415056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f9815c1e0 con 0x7f5f9815c9e0 2024-09-19T07:44:59.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.872+0000 7f5f9fc77640 1 -- 172.21.15.16:0/779415056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f98068ca0 con 0x7f5f9815c9e0 2024-09-19T07:44:59.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.872+0000 7f5f967fc640 1 -- 172.21.15.16:0/779415056 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5f8c012040 con 0x7f5f9815c9e0 2024-09-19T07:44:59.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.873+0000 7f5f967fc640 1 --2- 172.21.15.16:0/779415056 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5f8406c2e0 0x7f5f8406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:59.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.873+0000 7f5f967fc640 1 -- 172.21.15.16:0/779415056 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f5f8c08bab0 con 0x7f5f9815c9e0 2024-09-19T07:44:59.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.873+0000 7f5f9da13640 1 --2- 172.21.15.16:0/779415056 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5f8406c2e0 0x7f5f8406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:59.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.874+0000 7f5f9da13640 1 --2- 172.21.15.16:0/779415056 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5f8406c2e0 0x7f5f8406e7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5f88002230 tx=0x7f5f88006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:59.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:44:59.877+0000 7f5f967fc640 1 -- 172.21.15.16:0/779415056 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5f8c019030 con 0x7f5f9815c9e0 2024-09-19T07:45:00.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.089+0000 7f5f9fc77640 1 -- 172.21.15.16:0/779415056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5f98072290 con 0x7f5f9815c9e0 2024-09-19T07:45:00.090 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-19T07:45:00.090+0000 7f6950fea640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:45:00.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.091+0000 7f5f967fc640 1 -- 172.21.15.16:0/779415056 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7f5f8c05a070 con 0x7f5f9815c9e0 2024-09-19T07:45:00.092 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T07:45:00.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.095+0000 7f5f9fc77640 1 -- 172.21.15.16:0/779415056 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5f8406c2e0 msgr2=0x7f5f8406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:00.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.095+0000 7f5f9fc77640 1 --2- 172.21.15.16:0/779415056 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5f8406c2e0 0x7f5f8406e7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5f88002230 tx=0x7f5f88006040 comp rx=0 tx=0).stop 2024-09-19T07:45:00.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.095+0000 7f5f9fc77640 1 -- 172.21.15.16:0/779415056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f9815c9e0 msgr2=0x7f5f98155e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:00.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.095+0000 7f5f9fc77640 1 --2- 172.21.15.16:0/779415056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f9815c9e0 0x7f5f98155e40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c011960 tx=0x7f5f8c011d60 comp rx=0 tx=0).stop 2024-09-19T07:45:00.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.095+0000 7f5f9fc77640 1 -- 172.21.15.16:0/779415056 shutdown_connections 2024-09-19T07:45:00.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.095+0000 7f5f9fc77640 1 --2- 172.21.15.16:0/779415056 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5f8406c2e0 0x7f5f8406e7a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:00.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.095+0000 7f5f9fc77640 1 --2- 172.21.15.16:0/779415056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f9815ce60 0x7f5f98156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:00.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.095+0000 7f5f9fc77640 1 --2- 172.21.15.16:0/779415056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f9815c9e0 0x7f5f98155e40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:00.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.095+0000 7f5f9fc77640 1 --2- 172.21.15.16:0/779415056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f80004d40 0x7f5f981f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:00.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.095+0000 7f5f9fc77640 1 -- 172.21.15.16:0/779415056 >> 172.21.15.16:0/779415056 conn(0x7f5f980b3ed0 msgr2=0x7f5f980c7140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:00.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.095+0000 7f5f9fc77640 1 -- 172.21.15.16:0/779415056 shutdown_connections 2024-09-19T07:45:00.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.095+0000 7f5f9fc77640 1 -- 172.21.15.16:0/779415056 wait complete. 2024-09-19T07:45:00.107 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T07:45:00.134 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-19T07:45:00.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.213+0000 7f878d3e4640 1 Processor -- start 2024-09-19T07:45:00.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.213+0000 7f878d3e4640 1 -- start start 2024-09-19T07:45:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.214+0000 7f878d3e4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87880d0070 0x7f878815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.214+0000 7f878d3e4640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f87880d4130 0x7f87880d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.214+0000 7f878d3e4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87880d1b10 con 0x7f878815f0d0 2024-09-19T07:45:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.214+0000 7f878d3e4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f87880cf2b0 con 0x7f87880d0070 2024-09-19T07:45:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.214+0000 7f878d3e4640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f87880cf590 con 0x7f87880d4130 2024-09-19T07:45:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.214+0000 7f877ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87880d0070 0x7f878815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.214+0000 7f8786ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f878815f0d0 0x7f87880d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44016/0 (socket says 172.21.15.16:44016) 2024-09-19T07:45:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.214+0000 7f8786ffd640 1 -- 172.21.15.16:0/1043774123 learned_addr learned my addr 172.21.15.16:0/1043774123 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.214+0000 7f87877fe640 1 --2- 172.21.15.16:0/1043774123 >> v2:172.21.15.146:3300/0 conn(0x7f87880d4130 0x7f87880d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.214+0000 7f877ffff640 1 -- 172.21.15.16:0/1043774123 >> v2:172.21.15.146:3300/0 conn(0x7f87880d4130 msgr2=0x7f87880d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.214+0000 7f877ffff640 1 --2- 172.21.15.16:0/1043774123 >> v2:172.21.15.146:3300/0 conn(0x7f87880d4130 0x7f87880d4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.214+0000 7f877ffff640 1 -- 172.21.15.16:0/1043774123 >> v1:172.21.15.16:6789/0 conn(0x7f878815f0d0 legacy=0x7f87880d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.214+0000 7f87867fc640 1 -- 172.21.15.16:0/1043774123 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2495789194 0 0) 0x7f87880d1b10 con 0x7f878815f0d0 2024-09-19T07:45:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.214+0000 7f877ffff640 1 -- 172.21.15.16:0/1043774123 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87880cf870 con 0x7f87880d0070 2024-09-19T07:45:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.214+0000 7f87877fe640 1 --2- 172.21.15.16:0/1043774123 >> v2:172.21.15.146:3300/0 conn(0x7f87880d4130 0x7f87880d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:00.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.215+0000 7f877ffff640 1 --2- 172.21.15.16:0/1043774123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87880d0070 0x7f878815f4b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f87880d04e0 tx=0x7f877002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2ec4dc92437787 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:00.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.215+0000 7f87867fc640 1 -- 172.21.15.16:0/1043774123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770037070 con 0x7f87880d0070 2024-09-19T07:45:00.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.215+0000 7f87867fc640 1 -- 172.21.15.16:0/1043774123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8770032440 con 0x7f87880d0070 2024-09-19T07:45:00.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.215+0000 7f87867fc640 1 -- 172.21.15.16:0/1043774123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877002ae60 con 0x7f87880d0070 2024-09-19T07:45:00.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.216+0000 7f878d3e4640 1 -- 172.21.15.16:0/1043774123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87880d0070 msgr2=0x7f878815f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:00.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.216+0000 7f878d3e4640 1 --2- 172.21.15.16:0/1043774123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87880d0070 0x7f878815f4b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f87880d04e0 tx=0x7f877002a110 comp rx=0 tx=0).stop 2024-09-19T07:45:00.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.216+0000 7f878d3e4640 1 -- 172.21.15.16:0/1043774123 shutdown_connections 2024-09-19T07:45:00.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.216+0000 7f878d3e4640 1 --2- 172.21.15.16:0/1043774123 >> v2:172.21.15.146:3300/0 conn(0x7f87880d4130 0x7f87880d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:00.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.216+0000 7f878d3e4640 1 --2- 172.21.15.16:0/1043774123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87880d0070 0x7f878815f4b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:00.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.216+0000 7f878d3e4640 1 -- 172.21.15.16:0/1043774123 >> 172.21.15.16:0/1043774123 conn(0x7f878809d760 msgr2=0x7f878808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:00.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.216+0000 7f878d3e4640 1 -- 172.21.15.16:0/1043774123 shutdown_connections 2024-09-19T07:45:00.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.216+0000 7f878d3e4640 1 -- 172.21.15.16:0/1043774123 wait complete. 2024-09-19T07:45:00.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.217+0000 7f878d3e4640 1 Processor -- start 2024-09-19T07:45:00.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.217+0000 7f878d3e4640 1 -- start start 2024-09-19T07:45:00.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.218+0000 7f878d3e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87880d0070 0x7f87880b9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:00.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.218+0000 7f878d3e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87880d4130 0x7f87880b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:00.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.218+0000 7f878d3e4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f878815f0d0 0x7f87881f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:00.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.218+0000 7f878d3e4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87880cef60 con 0x7f87880d4130 2024-09-19T07:45:00.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.218+0000 7f878d3e4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f87880d4aa0 con 0x7f87880d0070 2024-09-19T07:45:00.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.218+0000 7f878d3e4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f87880b2f70 con 0x7f878815f0d0 2024-09-19T07:45:00.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.218+0000 7f877ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87880d4130 0x7f87880b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:00.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.218+0000 7f877ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87880d4130 0x7f87880b2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60426/0 (socket says 172.21.15.16:60426) 2024-09-19T07:45:00.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.218+0000 7f877ffff640 1 -- 172.21.15.16:0/3715633402 learned_addr learned my addr 172.21.15.16:0/3715633402 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:00.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.218+0000 7f8786ffd640 1 --2- 172.21.15.16:0/3715633402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87880d0070 0x7f87880b9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:00.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.218+0000 7f87877fe640 1 --2- 172.21.15.16:0/3715633402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f878815f0d0 0x7f87881f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:00.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.218+0000 7f877ffff640 1 -- 172.21.15.16:0/3715633402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f878815f0d0 msgr2=0x7f87881f77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:00.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.218+0000 7f877ffff640 1 --2- 172.21.15.16:0/3715633402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f878815f0d0 0x7f87881f77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:00.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.218+0000 7f877ffff640 1 -- 172.21.15.16:0/3715633402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87880d0070 msgr2=0x7f87880b9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:00.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.218+0000 7f877ffff640 1 --2- 172.21.15.16:0/3715633402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87880d0070 0x7f87880b9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:00.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.218+0000 7f877ffff640 1 -- 172.21.15.16:0/3715633402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87881f8270 con 0x7f87880d4130 2024-09-19T07:45:00.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.219+0000 7f877ffff640 1 --2- 172.21.15.16:0/3715633402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87880d4130 0x7f87880b2610 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8770000c00 tx=0x7f87700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:00.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.219+0000 7f877f7fe640 1 -- 172.21.15.16:0/3715633402 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770037050 con 0x7f87880d4130 2024-09-19T07:45:00.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.219+0000 7f877f7fe640 1 -- 172.21.15.16:0/3715633402 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8770038040 con 0x7f87880d4130 2024-09-19T07:45:00.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.219+0000 7f878d3e4640 1 -- 172.21.15.16:0/3715633402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87881f8550 con 0x7f87880d4130 2024-09-19T07:45:00.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.219+0000 7f877f7fe640 1 -- 172.21.15.16:0/3715633402 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770004470 con 0x7f87880d4130 2024-09-19T07:45:00.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.219+0000 7f878d3e4640 1 -- 172.21.15.16:0/3715633402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87880b9200 con 0x7f87880d4130 2024-09-19T07:45:00.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.220+0000 7f878d3e4640 1 -- 172.21.15.16:0/3715633402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87880d4560 con 0x7f87880d4130 2024-09-19T07:45:00.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.220+0000 7f877f7fe640 1 -- 172.21.15.16:0/3715633402 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8770030900 con 0x7f87880d4130 2024-09-19T07:45:00.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.221+0000 7f877f7fe640 1 --2- 172.21.15.16:0/3715633402 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f876806c2b0 0x7f876806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:00.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.221+0000 7f877f7fe640 1 -- 172.21.15.16:0/3715633402 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6216+0+0 (secure 0 0 0) 0x7f87700ac1d0 con 0x7f87880d4130 2024-09-19T07:45:00.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.221+0000 7f8786ffd640 1 --2- 172.21.15.16:0/3715633402 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f876806c2b0 0x7f876806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:00.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.221+0000 7f8786ffd640 1 --2- 172.21.15.16:0/3715633402 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f876806c2b0 0x7f876806e770 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8778000f60 tx=0x7f8778009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:00.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.225+0000 7f877f7fe640 1 -- 172.21.15.16:0/3715633402 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f877007a8f0 con 0x7f87880d4130 2024-09-19T07:45:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:00.437+0000 7f878d3e4640 1 -- 172.21.15.16:0/3715633402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f878806cf30 con 0x7f87880d4130 2024-09-19T07:45:01.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.096+0000 7f877f7fe640 1 -- 172.21.15.16:0/3715633402 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7f877007a1f0 con 0x7f87880d4130 2024-09-19T07:45:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.099+0000 7f878d3e4640 1 -- 172.21.15.16:0/3715633402 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f876806c2b0 msgr2=0x7f876806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.099+0000 7f878d3e4640 1 --2- 172.21.15.16:0/3715633402 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f876806c2b0 0x7f876806e770 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8778000f60 tx=0x7f8778009040 comp rx=0 tx=0).stop 2024-09-19T07:45:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.099+0000 7f878d3e4640 1 -- 172.21.15.16:0/3715633402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87880d4130 msgr2=0x7f87880b2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.099+0000 7f878d3e4640 1 --2- 172.21.15.16:0/3715633402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87880d4130 0x7f87880b2610 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8770000c00 tx=0x7f87700046c0 comp rx=0 tx=0).stop 2024-09-19T07:45:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.099+0000 7f878d3e4640 1 -- 172.21.15.16:0/3715633402 shutdown_connections 2024-09-19T07:45:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.099+0000 7f878d3e4640 1 --2- 172.21.15.16:0/3715633402 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f876806c2b0 0x7f876806e770 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8778000f60 tx=0x7f8778009040 comp rx=0 tx=0).stop 2024-09-19T07:45:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.099+0000 7f878d3e4640 1 --2- 172.21.15.16:0/3715633402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f878815f0d0 0x7f87881f77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.099+0000 7f878d3e4640 1 --2- 172.21.15.16:0/3715633402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87880d4130 0x7f87880b2610 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.099+0000 7f878d3e4640 1 --2- 172.21.15.16:0/3715633402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87880d0070 0x7f87880b9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.099+0000 7f878d3e4640 1 -- 172.21.15.16:0/3715633402 >> 172.21.15.16:0/3715633402 conn(0x7f878809d760 msgr2=0x7f878806db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.099+0000 7f878d3e4640 1 -- 172.21.15.16:0/3715633402 shutdown_connections 2024-09-19T07:45:01.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.099+0000 7f878d3e4640 1 -- 172.21.15.16:0/3715633402 wait complete. 2024-09-19T07:45:01.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T07:45:01.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.269+0000 7f2f53caf640 1 Processor -- start 2024-09-19T07:45:01.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.269+0000 7f2f53caf640 1 -- start start 2024-09-19T07:45:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.270+0000 7f2f53caf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2f4c06f9e0 0x7f2f4c0d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.270+0000 7f2f53caf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f4c0d0e00 con 0x7f2f4c0d09f0 2024-09-19T07:45:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.270+0000 7f2f53caf640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f4c15b500 con 0x7f2f4c15b120 2024-09-19T07:45:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.270+0000 7f2f53caf640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2f4c06e940 con 0x7f2f4c06f9e0 2024-09-19T07:45:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.270+0000 7f2f5124a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2f4c06f9e0 0x7f2f4c0d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.270+0000 7f2f5224c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2f4c0d09f0 0x7f2f4c0c5160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44024/0 (socket says 172.21.15.16:44024) 2024-09-19T07:45:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.270+0000 7f2f5224c640 1 -- 172.21.15.16:0/3462594509 learned_addr learned my addr 172.21.15.16:0/3462594509 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.271+0000 7f2f5124a640 1 -- 172.21.15.16:0/3462594509 >> v1:172.21.15.146:6789/0 conn(0x7f2f4c15b120 legacy=0x7f2f4c06dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.271+0000 7f2f5124a640 1 -- 172.21.15.16:0/3462594509 >> v1:172.21.15.16:6789/0 conn(0x7f2f4c0d09f0 legacy=0x7f2f4c0c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.271+0000 7f2f50a49640 1 -- 172.21.15.16:0/3462594509 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1333950363 0 0) 0x7f2f44001490 con 0x7f2f4c0d09f0 2024-09-19T07:45:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.271+0000 7f2f5124a640 1 -- 172.21.15.16:0/3462594509 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4c06ec20 con 0x7f2f4c06f9e0 2024-09-19T07:45:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.271+0000 7f2f5124a640 1 --2- 172.21.15.16:0/3462594509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f4c06f9e0 0x7f2f4c0d04b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0097a0 tx=0x7f2f3c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22804df1a0fc953 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.271+0000 7f2f50a49640 1 -- 172.21.15.16:0/3462594509 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c037070 con 0x7f2f4c06f9e0 2024-09-19T07:45:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.272+0000 7f2f50a49640 1 -- 172.21.15.16:0/3462594509 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f3c032440 con 0x7f2f4c06f9e0 2024-09-19T07:45:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.272+0000 7f2f50a49640 1 -- 172.21.15.16:0/3462594509 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c032810 con 0x7f2f4c06f9e0 2024-09-19T07:45:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.272+0000 7f2f53caf640 1 -- 172.21.15.16:0/3462594509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f4c06f9e0 msgr2=0x7f2f4c0d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.272+0000 7f2f53caf640 1 --2- 172.21.15.16:0/3462594509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f4c06f9e0 0x7f2f4c0d04b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0097a0 tx=0x7f2f3c02a230 comp rx=0 tx=0).stop 2024-09-19T07:45:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.272+0000 7f2f53caf640 1 -- 172.21.15.16:0/3462594509 shutdown_connections 2024-09-19T07:45:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.272+0000 7f2f53caf640 1 --2- 172.21.15.16:0/3462594509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f4c06f9e0 0x7f2f4c0d04b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.272+0000 7f2f53caf640 1 -- 172.21.15.16:0/3462594509 >> 172.21.15.16:0/3462594509 conn(0x7f2f4c06bdf0 msgr2=0x7f2f4c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.272+0000 7f2f53caf640 1 -- 172.21.15.16:0/3462594509 shutdown_connections 2024-09-19T07:45:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.272+0000 7f2f53caf640 1 -- 172.21.15.16:0/3462594509 wait complete. 2024-09-19T07:45:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.273+0000 7f2f53caf640 1 Processor -- start 2024-09-19T07:45:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.273+0000 7f2f53caf640 1 -- start start 2024-09-19T07:45:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.274+0000 7f2f53caf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f4c06f9e0 0x7f2f4c14d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.274+0000 7f2f53caf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f4c0d09f0 0x7f2f4c0c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.274+0000 7f2f53caf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f4c15b120 0x7f2f4c1571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.274+0000 7f2f53caf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c06f4c0 con 0x7f2f4c15b120 2024-09-19T07:45:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.274+0000 7f2f53caf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c06f1e0 con 0x7f2f4c0d09f0 2024-09-19T07:45:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.274+0000 7f2f53caf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2f4c06e5d0 con 0x7f2f4c06f9e0 2024-09-19T07:45:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.274+0000 7f2f5224c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f4c15b120 0x7f2f4c1571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.274+0000 7f2f5224c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f4c15b120 0x7f2f4c1571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60432/0 (socket says 172.21.15.16:60432) 2024-09-19T07:45:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.274+0000 7f2f5224c640 1 -- 172.21.15.16:0/54166781 learned_addr learned my addr 172.21.15.16:0/54166781 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.274+0000 7f2f51a4b640 1 --2- 172.21.15.16:0/54166781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f4c06f9e0 0x7f2f4c14d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.274+0000 7f2f5124a640 1 --2- 172.21.15.16:0/54166781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f4c0d09f0 0x7f2f4c0c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.275+0000 7f2f5224c640 1 -- 172.21.15.16:0/54166781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f4c06f9e0 msgr2=0x7f2f4c14d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.275+0000 7f2f5224c640 1 --2- 172.21.15.16:0/54166781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f4c06f9e0 0x7f2f4c14d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.275+0000 7f2f5224c640 1 -- 172.21.15.16:0/54166781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f4c0d09f0 msgr2=0x7f2f4c0c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.275+0000 7f2f5224c640 1 --2- 172.21.15.16:0/54166781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f4c0d09f0 0x7f2f4c0c7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.275+0000 7f2f5224c640 1 -- 172.21.15.16:0/54166781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4c0b1090 con 0x7f2f4c15b120 2024-09-19T07:45:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.275+0000 7f2f5124a640 1 --2- 172.21.15.16:0/54166781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f4c0d09f0 0x7f2f4c0c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.275+0000 7f2f5224c640 1 --2- 172.21.15.16:0/54166781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f4c15b120 0x7f2f4c1571f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400c7f0 tx=0x7f2f4400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.275+0000 7f2f427fc640 1 -- 172.21.15.16:0/54166781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44018070 con 0x7f2f4c15b120 2024-09-19T07:45:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.275+0000 7f2f427fc640 1 -- 172.21.15.16:0/54166781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4400a5e0 con 0x7f2f4c15b120 2024-09-19T07:45:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.275+0000 7f2f53caf640 1 -- 172.21.15.16:0/54166781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4c0b1370 con 0x7f2f4c15b120 2024-09-19T07:45:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.275+0000 7f2f427fc640 1 -- 172.21.15.16:0/54166781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4400aa10 con 0x7f2f4c15b120 2024-09-19T07:45:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.275+0000 7f2f53caf640 1 -- 172.21.15.16:0/54166781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f4c0b1930 con 0x7f2f4c15b120 2024-09-19T07:45:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.276+0000 7f2f53caf640 1 -- 172.21.15.16:0/54166781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f4c0ce060 con 0x7f2f4c15b120 2024-09-19T07:45:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.277+0000 7f2f427fc640 1 -- 172.21.15.16:0/54166781 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2f4401b070 con 0x7f2f4c15b120 2024-09-19T07:45:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.277+0000 7f2f427fc640 1 --2- 172.21.15.16:0/54166781 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2f3006c2b0 0x7f2f3006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.277+0000 7f2f427fc640 1 -- 172.21.15.16:0/54166781 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6216+0+0 (secure 0 0 0) 0x7f2f4408abd0 con 0x7f2f4c15b120 2024-09-19T07:45:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.277+0000 7f2f51a4b640 1 --2- 172.21.15.16:0/54166781 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2f3006c2b0 0x7f2f3006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.278+0000 7f2f51a4b640 1 --2- 172.21.15.16:0/54166781 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2f3006c2b0 0x7f2f3006e770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2f48000f60 tx=0x7f2f48009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.281+0000 7f2f427fc640 1 -- 172.21.15.16:0/54166781 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2f4c0d0e00 con 0x7f2f4c15b120 2024-09-19T07:45:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:01.459+0000 7f2f53caf640 1 -- 172.21.15.16:0/54166781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2f4c06df40 con 0x7f2f4c15b120 2024-09-19T07:45:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.098+0000 7f2f427fc640 1 -- 172.21.15.16:0/54166781 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) ==== 181+0+0 (secure 0 0 0) 0x7f2f44058af0 con 0x7f2f4c15b120 2024-09-19T07:45:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-19T07:45:02.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.102+0000 7f2f53caf640 1 -- 172.21.15.16:0/54166781 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2f3006c2b0 msgr2=0x7f2f3006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:02.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.102+0000 7f2f53caf640 1 --2- 172.21.15.16:0/54166781 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2f3006c2b0 0x7f2f3006e770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2f48000f60 tx=0x7f2f48009040 comp rx=0 tx=0).stop 2024-09-19T07:45:02.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.102+0000 7f2f53caf640 1 -- 172.21.15.16:0/54166781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f4c15b120 msgr2=0x7f2f4c1571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:02.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.102+0000 7f2f53caf640 1 --2- 172.21.15.16:0/54166781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f4c15b120 0x7f2f4c1571f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400c7f0 tx=0x7f2f4400ccb0 comp rx=0 tx=0).stop 2024-09-19T07:45:02.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.103+0000 7f2f53caf640 1 -- 172.21.15.16:0/54166781 shutdown_connections 2024-09-19T07:45:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.103+0000 7f2f53caf640 1 --2- 172.21.15.16:0/54166781 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2f3006c2b0 0x7f2f3006e770 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2f48000f60 tx=0x7f2f48009040 comp rx=0 tx=0).stop 2024-09-19T07:45:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.103+0000 7f2f53caf640 1 --2- 172.21.15.16:0/54166781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f4c15b120 0x7f2f4c1571f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.103+0000 7f2f53caf640 1 --2- 172.21.15.16:0/54166781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f4c0d09f0 0x7f2f4c0c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.103+0000 7f2f53caf640 1 --2- 172.21.15.16:0/54166781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f4c06f9e0 0x7f2f4c14d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.103+0000 7f2f53caf640 1 -- 172.21.15.16:0/54166781 >> 172.21.15.16:0/54166781 conn(0x7f2f4c06bdf0 msgr2=0x7f2f4c06c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.103+0000 7f2f53caf640 1 -- 172.21.15.16:0/54166781 shutdown_connections 2024-09-19T07:45:02.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.103+0000 7f2f53caf640 1 -- 172.21.15.16:0/54166781 wait complete. 2024-09-19T07:45:02.118 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T07:45:02.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.229+0000 7f2d32c0a640 1 Processor -- start 2024-09-19T07:45:02.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.229+0000 7f2d32c0a640 1 -- start start 2024-09-19T07:45:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.229+0000 7f2d32c0a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d2c15f0d0 0x7f2d2c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.229+0000 7f2d32c0a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2d2c0d0070 0x7f2d2c0d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.229+0000 7f2d32c0a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d2c0d1940 0x7f2d2c0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.229+0000 7f2d32c0a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2d2c0cf2b0 con 0x7f2d2c0d1940 2024-09-19T07:45:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.229+0000 7f2d32c0a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2d2c0cf590 con 0x7f2d2c0d0070 2024-09-19T07:45:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.229+0000 7f2d32c0a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2c0cf870 con 0x7f2d2c15f0d0 2024-09-19T07:45:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.230+0000 7f2d311a7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d2c0d1940 0x7f2d2c0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.230+0000 7f2d311a7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d2c0d1940 0x7f2d2c0d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60446/0 (socket says 172.21.15.16:60446) 2024-09-19T07:45:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.230+0000 7f2d311a7640 1 -- 172.21.15.16:0/939356606 learned_addr learned my addr 172.21.15.16:0/939356606 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.230+0000 7f2d2bfff640 1 --2- 172.21.15.16:0/939356606 >> v2:172.21.15.146:3300/0 conn(0x7f2d2c0d0070 0x7f2d2c0d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.230+0000 7f2d309a6640 1 --2- 172.21.15.16:0/939356606 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d2c15f0d0 0x7f2d2c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.230+0000 7f2d311a7640 1 -- 172.21.15.16:0/939356606 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d2c15f0d0 msgr2=0x7f2d2c15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.230+0000 7f2d311a7640 1 --2- 172.21.15.16:0/939356606 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d2c15f0d0 0x7f2d2c15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.230+0000 7f2d311a7640 1 -- 172.21.15.16:0/939356606 >> v2:172.21.15.146:3300/0 conn(0x7f2d2c0d0070 msgr2=0x7f2d2c0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.230+0000 7f2d311a7640 1 --2- 172.21.15.16:0/939356606 >> v2:172.21.15.146:3300/0 conn(0x7f2d2c0d0070 0x7f2d2c0d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.230+0000 7f2d311a7640 1 -- 172.21.15.16:0/939356606 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2c0cfb50 con 0x7f2d2c0d1940 2024-09-19T07:45:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.230+0000 7f2d2bfff640 1 --2- 172.21.15.16:0/939356606 >> v2:172.21.15.146:3300/0 conn(0x7f2d2c0d0070 0x7f2d2c0d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:02.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.231+0000 7f2d311a7640 1 --2- 172.21.15.16:0/939356606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d2c0d1940 0x7f2d2c0d4130 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0d1f60 tx=0x7f2d2402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e234e37e73b6f696 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:02.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.231+0000 7f2d2b7fe640 1 -- 172.21.15.16:0/939356606 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24036070 con 0x7f2d2c0d1940 2024-09-19T07:45:02.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.231+0000 7f2d2b7fe640 1 -- 172.21.15.16:0/939356606 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d24031600 con 0x7f2d2c0d1940 2024-09-19T07:45:02.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.231+0000 7f2d2b7fe640 1 -- 172.21.15.16:0/939356606 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d240319d0 con 0x7f2d2c0d1940 2024-09-19T07:45:02.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.231+0000 7f2d32c0a640 1 -- 172.21.15.16:0/939356606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d2c0d1940 msgr2=0x7f2d2c0d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:02.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.231+0000 7f2d32c0a640 1 --2- 172.21.15.16:0/939356606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d2c0d1940 0x7f2d2c0d4130 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0d1f60 tx=0x7f2d2402a610 comp rx=0 tx=0).stop 2024-09-19T07:45:02.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.231+0000 7f2d32c0a640 1 -- 172.21.15.16:0/939356606 shutdown_connections 2024-09-19T07:45:02.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.231+0000 7f2d32c0a640 1 --2- 172.21.15.16:0/939356606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d2c0d1940 0x7f2d2c0d4130 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:02.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.231+0000 7f2d32c0a640 1 --2- 172.21.15.16:0/939356606 >> v2:172.21.15.146:3300/0 conn(0x7f2d2c0d0070 0x7f2d2c0d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:02.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.231+0000 7f2d32c0a640 1 --2- 172.21.15.16:0/939356606 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d2c15f0d0 0x7f2d2c15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:02.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.231+0000 7f2d32c0a640 1 -- 172.21.15.16:0/939356606 >> 172.21.15.16:0/939356606 conn(0x7f2d2c09d760 msgr2=0x7f2d2c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:02.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.231+0000 7f2d32c0a640 1 -- 172.21.15.16:0/939356606 shutdown_connections 2024-09-19T07:45:02.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.231+0000 7f2d32c0a640 1 -- 172.21.15.16:0/939356606 wait complete. 2024-09-19T07:45:02.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.233+0000 7f2d32c0a640 1 Processor -- start 2024-09-19T07:45:02.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.233+0000 7f2d32c0a640 1 -- start start 2024-09-19T07:45:02.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d32c0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d2c0d0070 0x7f2d2c1f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:02.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d32c0a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d2c0d1940 0x7f2d2c15a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:02.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d32c0a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d2c15f0d0 0x7f2d2c15b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:02.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d32c0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c0d47f0 con 0x7f2d2c0d0070 2024-09-19T07:45:02.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d32c0a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c0d4670 con 0x7f2d2c15f0d0 2024-09-19T07:45:02.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d32c0a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2c0d4970 con 0x7f2d2c0d1940 2024-09-19T07:45:02.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d309a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d2c0d0070 0x7f2d2c1f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:02.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d309a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d2c0d0070 0x7f2d2c1f4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60458/0 (socket says 172.21.15.16:60458) 2024-09-19T07:45:02.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d309a6640 1 -- 172.21.15.16:0/135402906 learned_addr learned my addr 172.21.15.16:0/135402906 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:02.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d311a7640 1 --2- 172.21.15.16:0/135402906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d2c15f0d0 0x7f2d2c15b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:02.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d2bfff640 1 --2- 172.21.15.16:0/135402906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d2c0d1940 0x7f2d2c15a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:02.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d309a6640 1 -- 172.21.15.16:0/135402906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d2c0d1940 msgr2=0x7f2d2c15a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:02.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d309a6640 1 --2- 172.21.15.16:0/135402906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d2c0d1940 0x7f2d2c15a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:02.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d309a6640 1 -- 172.21.15.16:0/135402906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d2c15f0d0 msgr2=0x7f2d2c15b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:02.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d309a6640 1 --2- 172.21.15.16:0/135402906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d2c15f0d0 0x7f2d2c15b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:02.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d309a6640 1 -- 172.21.15.16:0/135402906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2c15bee0 con 0x7f2d2c0d0070 2024-09-19T07:45:02.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d2bfff640 1 --2- 172.21.15.16:0/135402906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d2c0d1940 0x7f2d2c15a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:02.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d309a6640 1 --2- 172.21.15.16:0/135402906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d2c0d0070 0x7f2d2c1f4250 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000d5f0 tx=0x7f2d2000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:02.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d297fa640 1 -- 172.21.15.16:0/135402906 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d200125f0 con 0x7f2d2c0d0070 2024-09-19T07:45:02.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d297fa640 1 -- 172.21.15.16:0/135402906 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d20019070 con 0x7f2d2c0d0070 2024-09-19T07:45:02.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d32c0a640 1 -- 172.21.15.16:0/135402906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2c0cb5e0 con 0x7f2d2c0d0070 2024-09-19T07:45:02.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d297fa640 1 -- 172.21.15.16:0/135402906 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d200129d0 con 0x7f2d2c0d0070 2024-09-19T07:45:02.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.234+0000 7f2d32c0a640 1 -- 172.21.15.16:0/135402906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d2c0cbb60 con 0x7f2d2c0d0070 2024-09-19T07:45:02.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.235+0000 7f2d32c0a640 1 -- 172.21.15.16:0/135402906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2c0cc270 con 0x7f2d2c0d0070 2024-09-19T07:45:02.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.236+0000 7f2d297fa640 1 -- 172.21.15.16:0/135402906 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2d20012b70 con 0x7f2d2c0d0070 2024-09-19T07:45:02.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.236+0000 7f2d297fa640 1 --2- 172.21.15.16:0/135402906 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2d1006c2e0 0x7f2d1006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:02.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.236+0000 7f2d297fa640 1 -- 172.21.15.16:0/135402906 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5778+0+0 (secure 0 0 0) 0x7f2d2008b6e0 con 0x7f2d2c0d0070 2024-09-19T07:45:02.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.237+0000 7f2d2bfff640 1 --2- 172.21.15.16:0/135402906 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2d1006c2e0 0x7f2d1006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:02.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.237+0000 7f2d2bfff640 1 --2- 172.21.15.16:0/135402906 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2d1006c2e0 0x7f2d1006e7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2d18006fd0 tx=0x7f2d18008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:02.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.241+0000 7f2d297fa640 1 -- 172.21.15.16:0/135402906 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2d20059e90 con 0x7f2d2c0d0070 2024-09-19T07:45:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:02.423+0000 7f2d32c0a640 1 -- 172.21.15.16:0/135402906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2d2c06d6f0 con 0x7f2d2c0d0070 2024-09-19T07:45:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.103+0000 7f2d297fa640 1 -- 172.21.15.16:0/135402906 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) ==== 169+0+0 (secure 0 0 0) 0x7f2d20059790 con 0x7f2d2c0d0070 2024-09-19T07:45:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-19T07:45:03.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.107+0000 7f2d32c0a640 1 -- 172.21.15.16:0/135402906 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2d1006c2e0 msgr2=0x7f2d1006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:03.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.107+0000 7f2d32c0a640 1 --2- 172.21.15.16:0/135402906 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2d1006c2e0 0x7f2d1006e7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2d18006fd0 tx=0x7f2d18008040 comp rx=0 tx=0).stop 2024-09-19T07:45:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.107+0000 7f2d32c0a640 1 -- 172.21.15.16:0/135402906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d2c0d0070 msgr2=0x7f2d2c1f4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.107+0000 7f2d32c0a640 1 --2- 172.21.15.16:0/135402906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d2c0d0070 0x7f2d2c1f4250 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000d5f0 tx=0x7f2d2000dab0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.109+0000 7f2d32c0a640 1 -- 172.21.15.16:0/135402906 shutdown_connections 2024-09-19T07:45:03.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.109+0000 7f2d32c0a640 1 --2- 172.21.15.16:0/135402906 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2d1006c2e0 0x7f2d1006e7a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.109+0000 7f2d32c0a640 1 --2- 172.21.15.16:0/135402906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d2c15f0d0 0x7f2d2c15b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.109+0000 7f2d32c0a640 1 --2- 172.21.15.16:0/135402906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d2c0d1940 0x7f2d2c15a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.109+0000 7f2d32c0a640 1 --2- 172.21.15.16:0/135402906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d2c0d0070 0x7f2d2c1f4250 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.109+0000 7f2d32c0a640 1 -- 172.21.15.16:0/135402906 >> 172.21.15.16:0/135402906 conn(0x7f2d2c09d760 msgr2=0x7f2d2c0d1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:03.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.109+0000 7f2d32c0a640 1 -- 172.21.15.16:0/135402906 shutdown_connections 2024-09-19T07:45:03.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.110+0000 7f2d32c0a640 1 -- 172.21.15.16:0/135402906 wait complete. 2024-09-19T07:45:03.127 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_session_cap_acquisition_throttle 2024-09-19T07:45:03.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.235+0000 7f133c5fd640 1 Processor -- start 2024-09-19T07:45:03.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.235+0000 7f133c5fd640 1 -- start start 2024-09-19T07:45:03.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.236+0000 7f133c5fd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f133415e4d0 0x7f13340d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:03.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.236+0000 7f133c5fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13340d29c0 0x7f13340d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:03.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.236+0000 7f133c5fd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13340d2250 con 0x7f133415e8b0 2024-09-19T07:45:03.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.236+0000 7f133c5fd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f13340d1170 con 0x7f133415e4d0 2024-09-19T07:45:03.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.236+0000 7f133c5fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f13340d3340 con 0x7f13340d29c0 2024-09-19T07:45:03.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.237+0000 7f1339b98640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13340d29c0 0x7f13340d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:03.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.237+0000 7f133a399640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f133415e4d0 0x7f13340d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:03.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.237+0000 7f133a399640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f133415e4d0 0x7f13340d1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:40540/0 (socket says 172.21.15.16:40540) 2024-09-19T07:45:03.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.237+0000 7f133a399640 1 -- 172.21.15.16:0/1932520133 learned_addr learned my addr 172.21.15.16:0/1932520133 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:03.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.237+0000 7f133a399640 1 -- 172.21.15.16:0/1932520133 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13340d29c0 msgr2=0x7f13340d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:03.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.237+0000 7f133a399640 1 --2- 172.21.15.16:0/1932520133 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13340d29c0 0x7f13340d2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.237+0000 7f133a399640 1 -- 172.21.15.16:0/1932520133 >> v1:172.21.15.146:6789/0 conn(0x7f133415e8b0 legacy=0x7f133415ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:03.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.237+0000 7f1339397640 1 -- 172.21.15.16:0/1932520133 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 645411395 0 0) 0x7f13340d2250 con 0x7f133415e8b0 2024-09-19T07:45:03.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.237+0000 7f133a399640 1 -- 172.21.15.16:0/1932520133 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133415f520 con 0x7f133415e4d0 2024-09-19T07:45:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.238+0000 7f133a399640 1 --2- 172.21.15.16:0/1932520133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f133415e4d0 0x7f13340d1600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1324009870 tx=0x7f132402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4000949a4c370ac3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.238+0000 7f1339397640 1 -- 172.21.15.16:0/1932520133 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324037070 con 0x7f133415e4d0 2024-09-19T07:45:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.238+0000 7f1339397640 1 -- 172.21.15.16:0/1932520133 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1324032440 con 0x7f133415e4d0 2024-09-19T07:45:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.238+0000 7f1339397640 1 -- 172.21.15.16:0/1932520133 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324032810 con 0x7f133415e4d0 2024-09-19T07:45:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.239+0000 7f133c5fd640 1 -- 172.21.15.16:0/1932520133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f133415e4d0 msgr2=0x7f13340d1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.239+0000 7f133c5fd640 1 --2- 172.21.15.16:0/1932520133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f133415e4d0 0x7f13340d1600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1324009870 tx=0x7f132402a270 comp rx=0 tx=0).stop 2024-09-19T07:45:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.239+0000 7f133c5fd640 1 -- 172.21.15.16:0/1932520133 shutdown_connections 2024-09-19T07:45:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.239+0000 7f133c5fd640 1 --2- 172.21.15.16:0/1932520133 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13340d29c0 0x7f13340d2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.239+0000 7f133c5fd640 1 --2- 172.21.15.16:0/1932520133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f133415e4d0 0x7f13340d1600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.239+0000 7f133c5fd640 1 -- 172.21.15.16:0/1932520133 >> 172.21.15.16:0/1932520133 conn(0x7f133409d770 msgr2=0x7f133408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:03.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.239+0000 7f133c5fd640 1 -- 172.21.15.16:0/1932520133 shutdown_connections 2024-09-19T07:45:03.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.239+0000 7f133c5fd640 1 -- 172.21.15.16:0/1932520133 wait complete. 2024-09-19T07:45:03.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.240+0000 7f133c5fd640 1 Processor -- start 2024-09-19T07:45:03.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.240+0000 7f133c5fd640 1 -- start start 2024-09-19T07:45:03.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.241+0000 7f133c5fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13340d29c0 0x7f13340cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:03.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.241+0000 7f133c5fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f133415e4d0 0x7f13340b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:03.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.241+0000 7f133c5fd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f133415e8b0 0x7f133415b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:03.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.241+0000 7f133c5fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13340d0e00 con 0x7f133415e4d0 2024-09-19T07:45:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.241+0000 7f133c5fd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f13340d0b20 con 0x7f133415e8b0 2024-09-19T07:45:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.241+0000 7f133c5fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f13340d1b40 con 0x7f13340d29c0 2024-09-19T07:45:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.241+0000 7f1339b98640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f133415e4d0 0x7f13340b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.241+0000 7f1339b98640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f133415e4d0 0x7f13340b7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57696/0 (socket says 172.21.15.16:57696) 2024-09-19T07:45:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.241+0000 7f1339b98640 1 -- 172.21.15.16:0/4269730383 learned_addr learned my addr 172.21.15.16:0/4269730383 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.241+0000 7f133ab9a640 1 --2- 172.21.15.16:0/4269730383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f133415e8b0 0x7f133415b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.241+0000 7f133a399640 1 --2- 172.21.15.16:0/4269730383 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13340d29c0 0x7f13340cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.241+0000 7f1339b98640 1 -- 172.21.15.16:0/4269730383 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13340d29c0 msgr2=0x7f13340cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.241+0000 7f1339b98640 1 --2- 172.21.15.16:0/4269730383 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13340d29c0 0x7f13340cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.241+0000 7f1339b98640 1 -- 172.21.15.16:0/4269730383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f133415e8b0 msgr2=0x7f133415b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.241+0000 7f1339b98640 1 --2- 172.21.15.16:0/4269730383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f133415e8b0 0x7f133415b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.241+0000 7f1339b98640 1 -- 172.21.15.16:0/4269730383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13340cc110 con 0x7f133415e4d0 2024-09-19T07:45:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.241+0000 7f133ab9a640 1 --2- 172.21.15.16:0/4269730383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f133415e8b0 0x7f133415b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.242+0000 7f1339b98640 1 --2- 172.21.15.16:0/4269730383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f133415e4d0 0x7f13340b7210 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f132800e900 tx=0x7f132800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.242+0000 7f1332ffd640 1 -- 172.21.15.16:0/4269730383 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1328017070 con 0x7f133415e4d0 2024-09-19T07:45:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.242+0000 7f1332ffd640 1 -- 172.21.15.16:0/4269730383 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f132800ca00 con 0x7f133415e4d0 2024-09-19T07:45:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.242+0000 7f133c5fd640 1 -- 172.21.15.16:0/4269730383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13340cc3f0 con 0x7f133415e4d0 2024-09-19T07:45:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.242+0000 7f1332ffd640 1 -- 172.21.15.16:0/4269730383 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132800cdb0 con 0x7f133415e4d0 2024-09-19T07:45:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.242+0000 7f133c5fd640 1 -- 172.21.15.16:0/4269730383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13341546c0 con 0x7f133415e4d0 2024-09-19T07:45:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.243+0000 7f133c5fd640 1 -- 172.21.15.16:0/4269730383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13340cb270 con 0x7f133415e4d0 2024-09-19T07:45:03.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.243+0000 7f1332ffd640 1 -- 172.21.15.16:0/4269730383 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f132801a070 con 0x7f133415e4d0 2024-09-19T07:45:03.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.244+0000 7f1332ffd640 1 --2- 172.21.15.16:0/4269730383 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f132006c2b0 0x7f132006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:03.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.244+0000 7f1332ffd640 1 -- 172.21.15.16:0/4269730383 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f132808b5d0 con 0x7f133415e4d0 2024-09-19T07:45:03.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.244+0000 7f133a399640 1 --2- 172.21.15.16:0/4269730383 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f132006c2b0 0x7f132006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:03.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.245+0000 7f133a399640 1 --2- 172.21.15.16:0/4269730383 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f132006c2b0 0x7f132006e770 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1324034000 tx=0x7f132402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:03.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.248+0000 7f1332ffd640 1 -- 172.21.15.16:0/4269730383 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f13340cb270 con 0x7f133415e4d0 2024-09-19T07:45:03.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.427+0000 7f133c5fd640 1 -- 172.21.15.16:0/4269730383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_session_cap_acquisition_throttle"} v 0) -- 0x7f13340d2e00 con 0x7f133415e4d0 2024-09-19T07:45:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.430+0000 7f1332ffd640 1 -- 172.21.15.16:0/4269730383 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_session_cap_acquisition_throttle"}]=0 v13) ==== 119+0+0 (secure 0 0 0) 0x7f1328059860 con 0x7f133415e4d0 2024-09-19T07:45:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.432+0000 7f133c5fd640 1 -- 172.21.15.16:0/4269730383 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f132006c2b0 msgr2=0x7f132006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.432+0000 7f133c5fd640 1 --2- 172.21.15.16:0/4269730383 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f132006c2b0 0x7f132006e770 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1324034000 tx=0x7f132402e000 comp rx=0 tx=0).stop 2024-09-19T07:45:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.432+0000 7f133c5fd640 1 -- 172.21.15.16:0/4269730383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f133415e4d0 msgr2=0x7f13340b7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.432+0000 7f133c5fd640 1 --2- 172.21.15.16:0/4269730383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f133415e4d0 0x7f13340b7210 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f132800e900 tx=0x7f132800edc0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.432+0000 7f133c5fd640 1 -- 172.21.15.16:0/4269730383 shutdown_connections 2024-09-19T07:45:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.432+0000 7f133c5fd640 1 --2- 172.21.15.16:0/4269730383 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f132006c2b0 0x7f132006e770 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.432+0000 7f133c5fd640 1 --2- 172.21.15.16:0/4269730383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f133415e8b0 0x7f133415b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.432+0000 7f133c5fd640 1 --2- 172.21.15.16:0/4269730383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f133415e4d0 0x7f13340b7210 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.432+0000 7f133c5fd640 1 --2- 172.21.15.16:0/4269730383 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13340d29c0 0x7f13340cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.432+0000 7f133c5fd640 1 -- 172.21.15.16:0/4269730383 >> 172.21.15.16:0/4269730383 conn(0x7f133409d770 msgr2=0x7f13340b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.432+0000 7f133c5fd640 1 -- 172.21.15.16:0/4269730383 shutdown_connections 2024-09-19T07:45:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.432+0000 7f133c5fd640 1 -- 172.21.15.16:0/4269730383 wait complete. 2024-09-19T07:45:03.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_max_caps_per_client 2024-09-19T07:45:03.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.548+0000 7f2315d31640 1 Processor -- start 2024-09-19T07:45:03.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.548+0000 7f2315d31640 1 -- start start 2024-09-19T07:45:03.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.548+0000 7f2315d31640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f231015f0d0 0x7f231015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.548+0000 7f2315d31640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23100d0070 0x7f23100d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.548+0000 7f2315d31640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23100cf2b0 con 0x7f23100d1940 2024-09-19T07:45:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.548+0000 7f2315d31640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f23100cf590 con 0x7f231015f0d0 2024-09-19T07:45:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.548+0000 7f2315d31640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f23100cf870 con 0x7f23100d0070 2024-09-19T07:45:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.549+0000 7f230f7fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f231015f0d0 0x7f231015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.549+0000 7f2306ffd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23100d0070 0x7f23100d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.549+0000 7f230ffff640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f23100d1940 0x7f23100d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:37686/0 (socket says 172.21.15.16:37686) 2024-09-19T07:45:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.549+0000 7f230f7fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f231015f0d0 0x7f231015f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:40556/0 (socket says 172.21.15.16:40556) 2024-09-19T07:45:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.549+0000 7f230ffff640 1 -- 172.21.15.16:0/899304428 learned_addr learned my addr 172.21.15.16:0/899304428 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.549+0000 7f230f7fe640 1 -- 172.21.15.16:0/899304428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23100d0070 msgr2=0x7f23100d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.549+0000 7f230f7fe640 1 --2- 172.21.15.16:0/899304428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23100d0070 0x7f23100d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.549+0000 7f230f7fe640 1 -- 172.21.15.16:0/899304428 >> v1:172.21.15.146:6789/0 conn(0x7f23100d1940 legacy=0x7f23100d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.549+0000 7f230f7fe640 1 -- 172.21.15.16:0/899304428 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23100cfb50 con 0x7f231015f0d0 2024-09-19T07:45:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.550+0000 7f230f7fe640 1 --2- 172.21.15.16:0/899304428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f231015f0d0 0x7f231015f4b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f23000097a0 tx=0x7f230002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13258e77c0c2ebfb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.550+0000 7f230effd640 1 -- 172.21.15.16:0/899304428 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2300037070 con 0x7f231015f0d0 2024-09-19T07:45:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.550+0000 7f230effd640 1 -- 172.21.15.16:0/899304428 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f230002ae50 con 0x7f231015f0d0 2024-09-19T07:45:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.550+0000 7f230effd640 1 -- 172.21.15.16:0/899304428 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2300032660 con 0x7f231015f0d0 2024-09-19T07:45:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.551+0000 7f2315d31640 1 -- 172.21.15.16:0/899304428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f231015f0d0 msgr2=0x7f231015f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.551+0000 7f2315d31640 1 --2- 172.21.15.16:0/899304428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f231015f0d0 0x7f231015f4b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f23000097a0 tx=0x7f230002a100 comp rx=0 tx=0).stop 2024-09-19T07:45:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.551+0000 7f2315d31640 1 -- 172.21.15.16:0/899304428 shutdown_connections 2024-09-19T07:45:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.551+0000 7f2315d31640 1 --2- 172.21.15.16:0/899304428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23100d0070 0x7f23100d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.551+0000 7f2315d31640 1 --2- 172.21.15.16:0/899304428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f231015f0d0 0x7f231015f4b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.551+0000 7f2315d31640 1 -- 172.21.15.16:0/899304428 >> 172.21.15.16:0/899304428 conn(0x7f231009d760 msgr2=0x7f231008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.551+0000 7f2315d31640 1 -- 172.21.15.16:0/899304428 shutdown_connections 2024-09-19T07:45:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.551+0000 7f2315d31640 1 -- 172.21.15.16:0/899304428 wait complete. 2024-09-19T07:45:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.552+0000 7f2315d31640 1 Processor -- start 2024-09-19T07:45:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.552+0000 7f2315d31640 1 -- start start 2024-09-19T07:45:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.553+0000 7f2315d31640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23100d0070 0x7f23100b9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.553+0000 7f2315d31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23100d1940 0x7f23101eb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.553+0000 7f2315d31640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f231015f0d0 0x7f23101eb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.553+0000 7f2315d31640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23100d49c0 con 0x7f23100d1940 2024-09-19T07:45:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.553+0000 7f2315d31640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f23100d4840 con 0x7f231015f0d0 2024-09-19T07:45:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.553+0000 7f2315d31640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f23100d4b40 con 0x7f23100d0070 2024-09-19T07:45:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.553+0000 7f2306ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23100d1940 0x7f23101eb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.553+0000 7f2306ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23100d1940 0x7f23101eb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57710/0 (socket says 172.21.15.16:57710) 2024-09-19T07:45:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.553+0000 7f2306ffd640 1 -- 172.21.15.16:0/3224420115 learned_addr learned my addr 172.21.15.16:0/3224420115 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.553+0000 7f230ffff640 1 --2- 172.21.15.16:0/3224420115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f231015f0d0 0x7f23101eb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.553+0000 7f230f7fe640 1 --2- 172.21.15.16:0/3224420115 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23100d0070 0x7f23100b9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.553+0000 7f2306ffd640 1 -- 172.21.15.16:0/3224420115 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23100d0070 msgr2=0x7f23100b9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.553+0000 7f2306ffd640 1 --2- 172.21.15.16:0/3224420115 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23100d0070 0x7f23100b9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.553+0000 7f2306ffd640 1 -- 172.21.15.16:0/3224420115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f231015f0d0 msgr2=0x7f23101eb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.553+0000 7f2306ffd640 1 --2- 172.21.15.16:0/3224420115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f231015f0d0 0x7f23101eb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.553+0000 7f2306ffd640 1 -- 172.21.15.16:0/3224420115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23101599e0 con 0x7f23100d1940 2024-09-19T07:45:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.554+0000 7f2306ffd640 1 --2- 172.21.15.16:0/3224420115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23100d1940 0x7f23101eb480 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f22f800e870 tx=0x7f22f800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.554+0000 7f230cff9640 1 -- 172.21.15.16:0/3224420115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8017070 con 0x7f23100d1940 2024-09-19T07:45:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.554+0000 7f230cff9640 1 -- 172.21.15.16:0/3224420115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f800c940 con 0x7f23100d1940 2024-09-19T07:45:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.554+0000 7f2315d31640 1 -- 172.21.15.16:0/3224420115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23100cc090 con 0x7f23100d1940 2024-09-19T07:45:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.554+0000 7f230cff9640 1 -- 172.21.15.16:0/3224420115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f800cd30 con 0x7f23100d1940 2024-09-19T07:45:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.554+0000 7f2315d31640 1 -- 172.21.15.16:0/3224420115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23100c8cb0 con 0x7f23100d1940 2024-09-19T07:45:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.555+0000 7f2315d31640 1 -- 172.21.15.16:0/3224420115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2310075710 con 0x7f23100d1940 2024-09-19T07:45:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.555+0000 7f230cff9640 1 -- 172.21.15.16:0/3224420115 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f22f801a070 con 0x7f23100d1940 2024-09-19T07:45:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.556+0000 7f230cff9640 1 --2- 172.21.15.16:0/3224420115 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22f006c2b0 0x7f22f006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.556+0000 7f230cff9640 1 -- 172.21.15.16:0/3224420115 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f22f808b0e0 con 0x7f23100d1940 2024-09-19T07:45:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.556+0000 7f230f7fe640 1 --2- 172.21.15.16:0/3224420115 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22f006c2b0 0x7f22f006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.556+0000 7f230f7fe640 1 --2- 172.21.15.16:0/3224420115 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22f006c2b0 0x7f22f006e770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2300034000 tx=0x7f230002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.560+0000 7f230cff9640 1 -- 172.21.15.16:0/3224420115 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2310075710 con 0x7f23100d1940 2024-09-19T07:45:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.741+0000 7f2315d31640 1 -- 172.21.15.16:0/3224420115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"} v 0) -- 0x7f23100d1400 con 0x7f23100d1940 2024-09-19T07:45:03.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.744+0000 7f230cff9640 1 -- 172.21.15.16:0/3224420115 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"}]=0 v14) ==== 106+0+0 (secure 0 0 0) 0x7f22f8059a50 con 0x7f23100d1940 2024-09-19T07:45:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.746+0000 7f2315d31640 1 -- 172.21.15.16:0/3224420115 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22f006c2b0 msgr2=0x7f22f006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.746+0000 7f2315d31640 1 --2- 172.21.15.16:0/3224420115 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22f006c2b0 0x7f22f006e770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2300034000 tx=0x7f230002e000 comp rx=0 tx=0).stop 2024-09-19T07:45:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.746+0000 7f2315d31640 1 -- 172.21.15.16:0/3224420115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23100d1940 msgr2=0x7f23101eb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.746+0000 7f2315d31640 1 --2- 172.21.15.16:0/3224420115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23100d1940 0x7f23101eb480 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f22f800e870 tx=0x7f22f800ed30 comp rx=0 tx=0).stop 2024-09-19T07:45:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.746+0000 7f2315d31640 1 -- 172.21.15.16:0/3224420115 shutdown_connections 2024-09-19T07:45:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.746+0000 7f2315d31640 1 --2- 172.21.15.16:0/3224420115 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22f006c2b0 0x7f22f006e770 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.746+0000 7f2315d31640 1 --2- 172.21.15.16:0/3224420115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f231015f0d0 0x7f23101eb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.746+0000 7f2315d31640 1 --2- 172.21.15.16:0/3224420115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23100d1940 0x7f23101eb480 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.746+0000 7f2315d31640 1 --2- 172.21.15.16:0/3224420115 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23100d0070 0x7f23100b9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.746+0000 7f2315d31640 1 -- 172.21.15.16:0/3224420115 >> 172.21.15.16:0/3224420115 conn(0x7f231009d760 msgr2=0x7f231006dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.746+0000 7f2315d31640 1 -- 172.21.15.16:0/3224420115 shutdown_connections 2024-09-19T07:45:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.746+0000 7f2315d31640 1 -- 172.21.15.16:0/3224420115 wait complete. 2024-09-19T07:45:03.758 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T07:45:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.861+0000 7f5ea7b6d640 1 Processor -- start 2024-09-19T07:45:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.862+0000 7f5ea7b6d640 1 -- start start 2024-09-19T07:45:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.862+0000 7f5ea7b6d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5ea015e680 0x7f5ea00cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.862+0000 7f5ea7b6d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ea00d0010 0x7f5ea00d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.862+0000 7f5ea7b6d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ea00d1880 con 0x7f5ea015e2a0 2024-09-19T07:45:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.862+0000 7f5ea7b6d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5ea00d1b60 con 0x7f5ea015e680 2024-09-19T07:45:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.862+0000 7f5ea7b6d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea00cf2d0 con 0x7f5ea00d0010 2024-09-19T07:45:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.863+0000 7f5ea610a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ea00d0010 0x7f5ea00d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.863+0000 7f5ea5108640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5ea015e680 0x7f5ea00cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.863+0000 7f5ea5909640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f5ea015e2a0 0x7f5ea006d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:37696/0 (socket says 172.21.15.16:37696) 2024-09-19T07:45:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.863+0000 7f5ea5909640 1 -- 172.21.15.16:0/408885794 learned_addr learned my addr 172.21.15.16:0/408885794 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.863+0000 7f5ea5108640 1 -- 172.21.15.16:0/408885794 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ea00d0010 msgr2=0x7f5ea00d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.863+0000 7f5ea5108640 1 --2- 172.21.15.16:0/408885794 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ea00d0010 0x7f5ea00d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.863+0000 7f5ea5108640 1 -- 172.21.15.16:0/408885794 >> v1:172.21.15.146:6789/0 conn(0x7f5ea015e2a0 legacy=0x7f5ea006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.863+0000 7f5ea4907640 1 -- 172.21.15.16:0/408885794 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1646759462 0 0) 0x7f5ea00d1880 con 0x7f5ea015e2a0 2024-09-19T07:45:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.863+0000 7f5ea5108640 1 -- 172.21.15.16:0/408885794 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea00cf5b0 con 0x7f5ea015e680 2024-09-19T07:45:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.863+0000 7f5ea610a640 1 --2- 172.21.15.16:0/408885794 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ea00d0010 0x7f5ea00d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.864+0000 7f5ea5108640 1 --2- 172.21.15.16:0/408885794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ea015e680 0x7f5ea00cfad0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5e940097a0 tx=0x7f5e9402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3cb3b2578060ef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.864+0000 7f5ea4907640 1 -- 172.21.15.16:0/408885794 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e94037070 con 0x7f5ea015e680 2024-09-19T07:45:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.864+0000 7f5ea4907640 1 -- 172.21.15.16:0/408885794 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e9402ae50 con 0x7f5ea015e680 2024-09-19T07:45:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.864+0000 7f5ea4907640 1 -- 172.21.15.16:0/408885794 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e94032660 con 0x7f5ea015e680 2024-09-19T07:45:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.865+0000 7f5ea7b6d640 1 -- 172.21.15.16:0/408885794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ea015e680 msgr2=0x7f5ea00cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.865+0000 7f5ea7b6d640 1 --2- 172.21.15.16:0/408885794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ea015e680 0x7f5ea00cfad0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5e940097a0 tx=0x7f5e9402a100 comp rx=0 tx=0).stop 2024-09-19T07:45:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.865+0000 7f5ea7b6d640 1 -- 172.21.15.16:0/408885794 shutdown_connections 2024-09-19T07:45:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.865+0000 7f5ea7b6d640 1 --2- 172.21.15.16:0/408885794 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ea00d0010 0x7f5ea00d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.865+0000 7f5ea7b6d640 1 --2- 172.21.15.16:0/408885794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ea015e680 0x7f5ea00cfad0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.865+0000 7f5ea7b6d640 1 -- 172.21.15.16:0/408885794 >> 172.21.15.16:0/408885794 conn(0x7f5ea009d760 msgr2=0x7f5ea008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.865+0000 7f5ea7b6d640 1 -- 172.21.15.16:0/408885794 shutdown_connections 2024-09-19T07:45:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.865+0000 7f5ea7b6d640 1 -- 172.21.15.16:0/408885794 wait complete. 2024-09-19T07:45:03.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.866+0000 7f5ea7b6d640 1 Processor -- start 2024-09-19T07:45:03.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.866+0000 7f5ea7b6d640 1 -- start start 2024-09-19T07:45:03.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.867+0000 7f5ea7b6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ea00d0010 0x7f5ea00cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:03.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.867+0000 7f5ea7b6d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ea015e2a0 0x7f5ea0155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:03.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.867+0000 7f5ea7b6d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ea015e680 0x7f5ea01f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:03.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.867+0000 7f5ea7b6d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea006dcb0 con 0x7f5ea00d0010 2024-09-19T07:45:03.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.867+0000 7f5ea7b6d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea006db30 con 0x7f5ea015e2a0 2024-09-19T07:45:03.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.867+0000 7f5ea7b6d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea00d48b0 con 0x7f5ea015e680 2024-09-19T07:45:03.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.867+0000 7f5ea5909640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ea00d0010 0x7f5ea00cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:03.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.867+0000 7f5ea5909640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ea00d0010 0x7f5ea00cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57726/0 (socket says 172.21.15.16:57726) 2024-09-19T07:45:03.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.867+0000 7f5ea5909640 1 -- 172.21.15.16:0/2874010388 learned_addr learned my addr 172.21.15.16:0/2874010388 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:03.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.867+0000 7f5ea610a640 1 --2- 172.21.15.16:0/2874010388 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ea015e680 0x7f5ea01f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:03.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.867+0000 7f5ea5108640 1 --2- 172.21.15.16:0/2874010388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ea015e2a0 0x7f5ea0155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:03.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.867+0000 7f5ea5909640 1 -- 172.21.15.16:0/2874010388 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ea015e680 msgr2=0x7f5ea01f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:03.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.867+0000 7f5ea5909640 1 --2- 172.21.15.16:0/2874010388 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ea015e680 0x7f5ea01f7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.867+0000 7f5ea5909640 1 -- 172.21.15.16:0/2874010388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ea015e2a0 msgr2=0x7f5ea0155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.867+0000 7f5ea5909640 1 --2- 172.21.15.16:0/2874010388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ea015e2a0 0x7f5ea0155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.867+0000 7f5ea5909640 1 -- 172.21.15.16:0/2874010388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea0156180 con 0x7f5ea00d0010 2024-09-19T07:45:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.867+0000 7f5ea5108640 1 --2- 172.21.15.16:0/2874010388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ea015e2a0 0x7f5ea0155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.867+0000 7f5ea5909640 1 --2- 172.21.15.16:0/2874010388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ea00d0010 0x7f5ea00cb4b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000c8c0 tx=0x7f5e9000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.868+0000 7f5e9e7fc640 1 -- 172.21.15.16:0/2874010388 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e90018070 con 0x7f5ea00d0010 2024-09-19T07:45:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.868+0000 7f5e9e7fc640 1 -- 172.21.15.16:0/2874010388 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e9000a640 con 0x7f5ea00d0010 2024-09-19T07:45:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.868+0000 7f5e9e7fc640 1 -- 172.21.15.16:0/2874010388 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9000aa30 con 0x7f5ea00d0010 2024-09-19T07:45:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.868+0000 7f5ea7b6d640 1 -- 172.21.15.16:0/2874010388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea00b9d00 con 0x7f5ea00d0010 2024-09-19T07:45:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.868+0000 7f5ea7b6d640 1 -- 172.21.15.16:0/2874010388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea00ce4e0 con 0x7f5ea00d0010 2024-09-19T07:45:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.869+0000 7f5ea7b6d640 1 -- 172.21.15.16:0/2874010388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea00d4c70 con 0x7f5ea00d0010 2024-09-19T07:45:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.869+0000 7f5e9e7fc640 1 -- 172.21.15.16:0/2874010388 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5e9000abd0 con 0x7f5ea00d0010 2024-09-19T07:45:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.870+0000 7f5e9e7fc640 1 --2- 172.21.15.16:0/2874010388 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5e8c06c2b0 0x7f5e8c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.870+0000 7f5e9e7fc640 1 -- 172.21.15.16:0/2874010388 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f5e9008a970 con 0x7f5ea00d0010 2024-09-19T07:45:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.870+0000 7f5ea5108640 1 --2- 172.21.15.16:0/2874010388 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5e8c06c2b0 0x7f5e8c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.871+0000 7f5ea5108640 1 --2- 172.21.15.16:0/2874010388 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5e8c06c2b0 0x7f5e8c06e770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5e94034000 tx=0x7f5e9402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:03.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:03.874+0000 7f5e9e7fc640 1 -- 172.21.15.16:0/2874010388 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5e900592e0 con 0x7f5ea00d0010 2024-09-19T07:45:04.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.052+0000 7f5ea7b6d640 1 -- 172.21.15.16:0/2874010388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f5ea006cf90 con 0x7f5ea00d0010 2024-09-19T07:45:04.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.055+0000 7f5e9e7fc640 1 -- 172.21.15.16:0/2874010388 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f5e90058be0 con 0x7f5ea00d0010 2024-09-19T07:45:04.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.058+0000 7f5ea7b6d640 1 -- 172.21.15.16:0/2874010388 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5e8c06c2b0 msgr2=0x7f5e8c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:04.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.058+0000 7f5ea7b6d640 1 --2- 172.21.15.16:0/2874010388 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5e8c06c2b0 0x7f5e8c06e770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5e94034000 tx=0x7f5e9402e000 comp rx=0 tx=0).stop 2024-09-19T07:45:04.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.058+0000 7f5ea7b6d640 1 -- 172.21.15.16:0/2874010388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ea00d0010 msgr2=0x7f5ea00cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:04.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.058+0000 7f5ea7b6d640 1 --2- 172.21.15.16:0/2874010388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ea00d0010 0x7f5ea00cb4b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000c8c0 tx=0x7f5e9000cd80 comp rx=0 tx=0).stop 2024-09-19T07:45:04.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.058+0000 7f5ea7b6d640 1 -- 172.21.15.16:0/2874010388 shutdown_connections 2024-09-19T07:45:04.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.058+0000 7f5ea7b6d640 1 --2- 172.21.15.16:0/2874010388 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5e8c06c2b0 0x7f5e8c06e770 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:04.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.058+0000 7f5ea7b6d640 1 --2- 172.21.15.16:0/2874010388 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ea015e680 0x7f5ea01f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:04.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.058+0000 7f5ea7b6d640 1 --2- 172.21.15.16:0/2874010388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ea015e2a0 0x7f5ea0155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:04.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.058+0000 7f5ea7b6d640 1 --2- 172.21.15.16:0/2874010388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ea00d0010 0x7f5ea00cb4b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:04.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.058+0000 7f5ea7b6d640 1 -- 172.21.15.16:0/2874010388 >> 172.21.15.16:0/2874010388 conn(0x7f5ea009d760 msgr2=0x7f5ea015fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:04.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.058+0000 7f5ea7b6d640 1 -- 172.21.15.16:0/2874010388 shutdown_connections 2024-09-19T07:45:04.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.058+0000 7f5ea7b6d640 1 -- 172.21.15.16:0/2874010388 wait complete. 2024-09-19T07:45:04.069 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2024-09-19T07:45:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.171+0000 7f57b0b6d640 1 Processor -- start 2024-09-19T07:45:04.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.171+0000 7f57b0b6d640 1 -- start start 2024-09-19T07:45:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.172+0000 7f57b0b6d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57ac15b300 0x7f57ac0cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.172+0000 7f57b0b6d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f57ac15b6e0 0x7f57ac0b1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.172+0000 7f57b0b6d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57ac0cd910 con 0x7f57ac0ccbd0 2024-09-19T07:45:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.172+0000 7f57b0b6d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f57ac0cd070 con 0x7f57ac15b6e0 2024-09-19T07:45:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.172+0000 7f57b0b6d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f57ac0cd350 con 0x7f57ac15b300 2024-09-19T07:45:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.172+0000 7f57aad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f57ac15b6e0 0x7f57ac0b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.172+0000 7f57a9d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f57ac0ccbd0 0x7f57ac0cdb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58132/0 (socket says 172.21.15.16:58132) 2024-09-19T07:45:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.172+0000 7f57a9d9b640 1 -- 172.21.15.16:0/1573572624 learned_addr learned my addr 172.21.15.16:0/1573572624 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.172+0000 7f57aad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f57ac15b6e0 0x7f57ac0b1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57736/0 (socket says 172.21.15.16:57736) 2024-09-19T07:45:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.172+0000 7f57aa59c640 1 --2- 172.21.15.16:0/1573572624 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57ac15b300 0x7f57ac0cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.173+0000 7f57aad9d640 1 -- 172.21.15.16:0/1573572624 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57ac15b300 msgr2=0x7f57ac0cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.173+0000 7f57aad9d640 1 --2- 172.21.15.16:0/1573572624 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57ac15b300 0x7f57ac0cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.173+0000 7f57aad9d640 1 -- 172.21.15.16:0/1573572624 >> v1:172.21.15.16:6789/0 conn(0x7f57ac0ccbd0 legacy=0x7f57ac0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.173+0000 7f57aad9d640 1 -- 172.21.15.16:0/1573572624 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ac0cd630 con 0x7f57ac15b6e0 2024-09-19T07:45:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.173+0000 7f57a959a640 1 -- 172.21.15.16:0/1573572624 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 726917313 0 0) 0x7f57ac0cd910 con 0x7f57ac0ccbd0 2024-09-19T07:45:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.173+0000 7f57aad9d640 1 --2- 172.21.15.16:0/1573572624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac15b6e0 0x7f57ac0b1050 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f57a40097a0 tx=0x7f57a402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc38119f9956ad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.173+0000 7f57a959a640 1 -- 172.21.15.16:0/1573572624 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4037070 con 0x7f57ac15b6e0 2024-09-19T07:45:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.173+0000 7f57a959a640 1 -- 172.21.15.16:0/1573572624 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a4032440 con 0x7f57ac15b6e0 2024-09-19T07:45:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.173+0000 7f57a959a640 1 -- 172.21.15.16:0/1573572624 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4032810 con 0x7f57ac15b6e0 2024-09-19T07:45:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.174+0000 7f57b0b6d640 1 -- 172.21.15.16:0/1573572624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac15b6e0 msgr2=0x7f57ac0b1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.174+0000 7f57b0b6d640 1 --2- 172.21.15.16:0/1573572624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac15b6e0 0x7f57ac0b1050 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f57a40097a0 tx=0x7f57a402a180 comp rx=0 tx=0).stop 2024-09-19T07:45:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.174+0000 7f57b0b6d640 1 -- 172.21.15.16:0/1573572624 shutdown_connections 2024-09-19T07:45:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.174+0000 7f57b0b6d640 1 --2- 172.21.15.16:0/1573572624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac15b6e0 0x7f57ac0b1050 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.174+0000 7f57b0b6d640 1 --2- 172.21.15.16:0/1573572624 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57ac15b300 0x7f57ac0cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.174+0000 7f57b0b6d640 1 -- 172.21.15.16:0/1573572624 >> 172.21.15.16:0/1573572624 conn(0x7f57ac0b2d10 msgr2=0x7f57ac0b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.174+0000 7f57b0b6d640 1 -- 172.21.15.16:0/1573572624 shutdown_connections 2024-09-19T07:45:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.174+0000 7f57b0b6d640 1 -- 172.21.15.16:0/1573572624 wait complete. 2024-09-19T07:45:04.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.175+0000 7f57b0b6d640 1 Processor -- start 2024-09-19T07:45:04.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.175+0000 7f57b0b6d640 1 -- start start 2024-09-19T07:45:04.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.176+0000 7f57b0b6d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57ac0ccbd0 0x7f57ac15a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:04.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.176+0000 7f57b0b6d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57ac15b300 0x7f57ac09ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:04.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.176+0000 7f57b0b6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac15b6e0 0x7f57ac09cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:04.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.176+0000 7f57b0b6d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac0b3b70 con 0x7f57ac15b6e0 2024-09-19T07:45:04.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.176+0000 7f57b0b6d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac0b39f0 con 0x7f57ac15b300 2024-09-19T07:45:04.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.176+0000 7f57b0b6d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f57ac0ce260 con 0x7f57ac0ccbd0 2024-09-19T07:45:04.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.176+0000 7f57aad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac15b6e0 0x7f57ac09cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:04.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.176+0000 7f57aad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac15b6e0 0x7f57ac09cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57748/0 (socket says 172.21.15.16:57748) 2024-09-19T07:45:04.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.176+0000 7f57aad9d640 1 -- 172.21.15.16:0/2717783631 learned_addr learned my addr 172.21.15.16:0/2717783631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:04.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.176+0000 7f57a9d9b640 1 --2- 172.21.15.16:0/2717783631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57ac15b300 0x7f57ac09ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:04.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.176+0000 7f57aa59c640 1 --2- 172.21.15.16:0/2717783631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57ac0ccbd0 0x7f57ac15a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:04.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.177+0000 7f57aad9d640 1 -- 172.21.15.16:0/2717783631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57ac0ccbd0 msgr2=0x7f57ac15a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:04.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.177+0000 7f57aad9d640 1 --2- 172.21.15.16:0/2717783631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57ac0ccbd0 0x7f57ac15a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:04.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.177+0000 7f57aad9d640 1 -- 172.21.15.16:0/2717783631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57ac15b300 msgr2=0x7f57ac09ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:04.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.177+0000 7f57aad9d640 1 --2- 172.21.15.16:0/2717783631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57ac15b300 0x7f57ac09ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:04.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.177+0000 7f57aad9d640 1 -- 172.21.15.16:0/2717783631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ac09e600 con 0x7f57ac15b6e0 2024-09-19T07:45:04.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.177+0000 7f57a9d9b640 1 --2- 172.21.15.16:0/2717783631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57ac15b300 0x7f57ac09ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:04.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.177+0000 7f57aad9d640 1 --2- 172.21.15.16:0/2717783631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac15b6e0 0x7f57ac09cbd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f57a4035040 tx=0x7f57a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:04.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.177+0000 7f579affd640 1 -- 172.21.15.16:0/2717783631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4037040 con 0x7f57ac15b6e0 2024-09-19T07:45:04.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.177+0000 7f579affd640 1 -- 172.21.15.16:0/2717783631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a4042070 con 0x7f57ac15b6e0 2024-09-19T07:45:04.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.177+0000 7f57b0b6d640 1 -- 172.21.15.16:0/2717783631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57ac09e8e0 con 0x7f57ac15b6e0 2024-09-19T07:45:04.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.177+0000 7f579affd640 1 -- 172.21.15.16:0/2717783631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4030b50 con 0x7f57ac15b6e0 2024-09-19T07:45:04.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.177+0000 7f57b0b6d640 1 -- 172.21.15.16:0/2717783631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57ac09e310 con 0x7f57ac15b6e0 2024-09-19T07:45:04.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.178+0000 7f57b0b6d640 1 -- 172.21.15.16:0/2717783631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57ac0cc690 con 0x7f57ac15b6e0 2024-09-19T07:45:04.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.179+0000 7f579affd640 1 -- 172.21.15.16:0/2717783631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f57a4030cf0 con 0x7f57ac15b6e0 2024-09-19T07:45:04.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.179+0000 7f579affd640 1 --2- 172.21.15.16:0/2717783631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f578c06c2e0 0x7f578c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:04.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.179+0000 7f579affd640 1 -- 172.21.15.16:0/2717783631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f57a40abe10 con 0x7f57ac15b6e0 2024-09-19T07:45:04.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.179+0000 7f57aa59c640 1 --2- 172.21.15.16:0/2717783631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f578c06c2e0 0x7f578c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:04.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.180+0000 7f57aa59c640 1 --2- 172.21.15.16:0/2717783631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f578c06c2e0 0x7f578c06e7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f579c009a60 tx=0x7f579c003990 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:04.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.183+0000 7f579affd640 1 -- 172.21.15.16:0/2717783631 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f57a4032e20 con 0x7f57ac15b6e0 2024-09-19T07:45:04.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:04.424+0000 7f57b0b6d640 1 -- 172.21.15.16:0/2717783631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) -- 0x7f57ac0cb490 con 0x7f57ac15b6e0 2024-09-19T07:45:05.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.109+0000 7f579affd640 1 -- 172.21.15.16:0/2717783631 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v118) ==== 165+0+0 (secure 0 0 0) 0x7f57a4045360 con 0x7f57ac15b6e0 2024-09-19T07:45:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.112+0000 7f57b0b6d640 1 -- 172.21.15.16:0/2717783631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f578c06c2e0 msgr2=0x7f578c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.112+0000 7f57b0b6d640 1 --2- 172.21.15.16:0/2717783631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f578c06c2e0 0x7f578c06e7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f579c009a60 tx=0x7f579c003990 comp rx=0 tx=0).stop 2024-09-19T07:45:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.112+0000 7f57b0b6d640 1 -- 172.21.15.16:0/2717783631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac15b6e0 msgr2=0x7f57ac09cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.112+0000 7f57b0b6d640 1 --2- 172.21.15.16:0/2717783631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac15b6e0 0x7f57ac09cbd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f57a4035040 tx=0x7f57a40046c0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.112+0000 7f57b0b6d640 1 -- 172.21.15.16:0/2717783631 shutdown_connections 2024-09-19T07:45:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.112+0000 7f57b0b6d640 1 --2- 172.21.15.16:0/2717783631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f578c06c2e0 0x7f578c06e7a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.112+0000 7f57b0b6d640 1 --2- 172.21.15.16:0/2717783631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac15b6e0 0x7f57ac09cbd0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.112+0000 7f57b0b6d640 1 --2- 172.21.15.16:0/2717783631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57ac15b300 0x7f57ac09ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.112+0000 7f57b0b6d640 1 --2- 172.21.15.16:0/2717783631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57ac0ccbd0 0x7f57ac15a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.112+0000 7f57b0b6d640 1 -- 172.21.15.16:0/2717783631 >> 172.21.15.16:0/2717783631 conn(0x7f57ac0b2d10 msgr2=0x7f57ac0b6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.112+0000 7f57b0b6d640 1 -- 172.21.15.16:0/2717783631 shutdown_connections 2024-09-19T07:45:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.112+0000 7f57b0b6d640 1 -- 172.21.15.16:0/2717783631 wait complete. 2024-09-19T07:45:05.126 INFO:tasks.cephfs_test_runner:Mostly readdir acquires caps faster than the mds recalls, so the cap ... ok 2024-09-19T07:45:05.126 INFO:tasks.cephfs_test_runner:Starting test: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) that a client is blocklisted when its encoded session metadata exceeds the 2024-09-19T07:45:05.126 INFO:tasks.cephfs_test_runner:test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-19T07:45:05.127 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:45:05.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.236+0000 7facb6d5f640 1 Processor -- start 2024-09-19T07:45:05.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.236+0000 7facb6d5f640 1 -- start start 2024-09-19T07:45:05.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.236+0000 7facb6d5f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7facb015b120 0x7facb006f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.236+0000 7facb6d5f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7facb006dec0 0x7facb006e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.236+0000 7facb6d5f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7facb00d04b0 0x7facb00d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.237+0000 7facb6d5f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7facb00d0e60 con 0x7facb006dec0 2024-09-19T07:45:05.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.237+0000 7facb6d5f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7facb015b500 con 0x7facb00d04b0 2024-09-19T07:45:05.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.237+0000 7facb6d5f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7facb006e940 con 0x7facb015b120 2024-09-19T07:45:05.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.237+0000 7facaffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7facb006dec0 0x7facb006e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:05.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.237+0000 7facaffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7facb006dec0 0x7facb006e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57762/0 (socket says 172.21.15.16:57762) 2024-09-19T07:45:05.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.237+0000 7facaffff640 1 -- 172.21.15.16:0/4189564721 learned_addr learned my addr 172.21.15.16:0/4189564721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.237+0000 7facb52fc640 1 --2- 172.21.15.16:0/4189564721 >> v2:172.21.15.146:3300/0 conn(0x7facb00d04b0 0x7facb00d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.237+0000 7facb4afb640 1 --2- 172.21.15.16:0/4189564721 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7facb015b120 0x7facb006f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.237+0000 7facaffff640 1 -- 172.21.15.16:0/4189564721 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7facb015b120 msgr2=0x7facb006f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.237+0000 7facaffff640 1 --2- 172.21.15.16:0/4189564721 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7facb015b120 0x7facb006f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.237+0000 7facaffff640 1 -- 172.21.15.16:0/4189564721 >> v2:172.21.15.146:3300/0 conn(0x7facb00d04b0 msgr2=0x7facb00d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.237+0000 7facaffff640 1 --2- 172.21.15.16:0/4189564721 >> v2:172.21.15.146:3300/0 conn(0x7facb00d04b0 0x7facb00d0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.237+0000 7facaffff640 1 -- 172.21.15.16:0/4189564721 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb006ec20 con 0x7facb006dec0 2024-09-19T07:45:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.237+0000 7facb52fc640 1 --2- 172.21.15.16:0/4189564721 >> v2:172.21.15.146:3300/0 conn(0x7facb00d04b0 0x7facb00d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.238+0000 7facaffff640 1 --2- 172.21.15.16:0/4189564721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb006dec0 0x7facb006e300 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7facb00b4c90 tx=0x7fac9802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56355f073b0fadb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.238+0000 7facaf7fe640 1 -- 172.21.15.16:0/4189564721 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98037070 con 0x7facb006dec0 2024-09-19T07:45:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.238+0000 7facaf7fe640 1 -- 172.21.15.16:0/4189564721 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac98032440 con 0x7facb006dec0 2024-09-19T07:45:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.238+0000 7facaf7fe640 1 -- 172.21.15.16:0/4189564721 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98032810 con 0x7facb006dec0 2024-09-19T07:45:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.238+0000 7facb6d5f640 1 -- 172.21.15.16:0/4189564721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb006dec0 msgr2=0x7facb006e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:05.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.238+0000 7facb6d5f640 1 --2- 172.21.15.16:0/4189564721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb006dec0 0x7facb006e300 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7facb00b4c90 tx=0x7fac9802a1e0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.239+0000 7facb6d5f640 1 -- 172.21.15.16:0/4189564721 shutdown_connections 2024-09-19T07:45:05.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.239+0000 7facb6d5f640 1 --2- 172.21.15.16:0/4189564721 >> v2:172.21.15.146:3300/0 conn(0x7facb00d04b0 0x7facb00d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.239+0000 7facb6d5f640 1 --2- 172.21.15.16:0/4189564721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb006dec0 0x7facb006e300 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.239+0000 7facb6d5f640 1 --2- 172.21.15.16:0/4189564721 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7facb015b120 0x7facb006f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.239+0000 7facb6d5f640 1 -- 172.21.15.16:0/4189564721 >> 172.21.15.16:0/4189564721 conn(0x7facb006bdf0 msgr2=0x7facb006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:05.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.239+0000 7facb6d5f640 1 -- 172.21.15.16:0/4189564721 shutdown_connections 2024-09-19T07:45:05.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.239+0000 7facb6d5f640 1 -- 172.21.15.16:0/4189564721 wait complete. 2024-09-19T07:45:05.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.240+0000 7facb6d5f640 1 Processor -- start 2024-09-19T07:45:05.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.240+0000 7facb6d5f640 1 -- start start 2024-09-19T07:45:05.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.241+0000 7facb6d5f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facb006dec0 0x7facb00b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.241+0000 7facb6d5f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb00d04b0 0x7facb01473c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.241+0000 7facb6d5f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7facb015b120 0x7facb0073320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.241+0000 7facb6d5f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7facb006f4c0 con 0x7facb00d04b0 2024-09-19T07:45:05.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.241+0000 7facb6d5f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7facb006f1e0 con 0x7facb006dec0 2024-09-19T07:45:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.241+0000 7facb6d5f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7facb00ce060 con 0x7facb015b120 2024-09-19T07:45:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.241+0000 7facaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb00d04b0 0x7facb01473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.241+0000 7facaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb00d04b0 0x7facb01473c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57770/0 (socket says 172.21.15.16:57770) 2024-09-19T07:45:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.241+0000 7facaffff640 1 -- 172.21.15.16:0/3915521792 learned_addr learned my addr 172.21.15.16:0/3915521792 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.241+0000 7facaffff640 1 -- 172.21.15.16:0/3915521792 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7facb015b120 msgr2=0x7facb0073320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.241+0000 7facaffff640 1 --2- 172.21.15.16:0/3915521792 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7facb015b120 0x7facb0073320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.241+0000 7facaffff640 1 -- 172.21.15.16:0/3915521792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facb006dec0 msgr2=0x7facb00b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.241+0000 7facb4afb640 1 --2- 172.21.15.16:0/3915521792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facb006dec0 0x7facb00b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.241+0000 7facaffff640 1 --2- 172.21.15.16:0/3915521792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facb006dec0 0x7facb00b2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.241+0000 7facaffff640 1 -- 172.21.15.16:0/3915521792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb00b1560 con 0x7facb00d04b0 2024-09-19T07:45:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.241+0000 7facaffff640 1 --2- 172.21.15.16:0/3915521792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb00d04b0 0x7facb01473c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fac9802a220 tx=0x7fac980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.241+0000 7facad7fa640 1 -- 172.21.15.16:0/3915521792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98037040 con 0x7facb00d04b0 2024-09-19T07:45:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.241+0000 7facad7fa640 1 -- 172.21.15.16:0/3915521792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac98042070 con 0x7facb00d04b0 2024-09-19T07:45:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.241+0000 7facad7fa640 1 -- 172.21.15.16:0/3915521792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98030bb0 con 0x7facb00d04b0 2024-09-19T07:45:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.242+0000 7facb6d5f640 1 -- 172.21.15.16:0/3915521792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb00b1840 con 0x7facb00d04b0 2024-09-19T07:45:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.242+0000 7facb6d5f640 1 -- 172.21.15.16:0/3915521792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb00b1e00 con 0x7facb00d04b0 2024-09-19T07:45:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.243+0000 7facb6d5f640 1 -- 172.21.15.16:0/3915521792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb00d0920 con 0x7facb00d04b0 2024-09-19T07:45:05.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.243+0000 7facad7fa640 1 -- 172.21.15.16:0/3915521792 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fac98047460 con 0x7facb00d04b0 2024-09-19T07:45:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.243+0000 7facad7fa640 1 --2- 172.21.15.16:0/3915521792 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fac9006c2b0 0x7fac9006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.244+0000 7facad7fa640 1 -- 172.21.15.16:0/3915521792 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fac980abef0 con 0x7facb00d04b0 2024-09-19T07:45:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.244+0000 7facb4afb640 1 --2- 172.21.15.16:0/3915521792 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fac9006c2b0 0x7fac9006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.244+0000 7facb4afb640 1 --2- 172.21.15.16:0/3915521792 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fac9006c2b0 0x7fac9006e770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faca800b740 tx=0x7faca8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:05.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.248+0000 7facad7fa640 1 -- 172.21.15.16:0/3915521792 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fac98045360 con 0x7facb00d04b0 2024-09-19T07:45:05.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.427+0000 7facb6d5f640 1 -- 172.21.15.16:0/3915521792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7facb0075550 con 0x7facb00d04b0 2024-09-19T07:45:05.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.428+0000 7facad7fa640 1 -- 172.21.15.16:0/3915521792 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+13329 (secure 0 0 0) 0x7fac9807a7d0 con 0x7facb00d04b0 2024-09-19T07:45:05.429 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:45:05.429 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":45,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:45:03.099497+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/2028210099":"2024-09-20T07:45:00.090979+0000","172.21.15.16:6833/2028210099":"2024-09-20T07:45:00.090979+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:45:05.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.429+0000 7facb6d5f640 1 -- 172.21.15.16:0/3915521792 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fac9006c2b0 msgr2=0x7fac9006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:05.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.429+0000 7facb6d5f640 1 --2- 172.21.15.16:0/3915521792 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fac9006c2b0 0x7fac9006e770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faca800b740 tx=0x7faca8008040 comp rx=0 tx=0).stop 2024-09-19T07:45:05.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.429+0000 7facb6d5f640 1 -- 172.21.15.16:0/3915521792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb00d04b0 msgr2=0x7facb01473c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:05.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.429+0000 7facb6d5f640 1 --2- 172.21.15.16:0/3915521792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb00d04b0 0x7facb01473c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fac9802a220 tx=0x7fac980046c0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.430+0000 7facb6d5f640 1 -- 172.21.15.16:0/3915521792 shutdown_connections 2024-09-19T07:45:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.430+0000 7facb6d5f640 1 --2- 172.21.15.16:0/3915521792 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fac9006c2b0 0x7fac9006e770 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.430+0000 7facb6d5f640 1 --2- 172.21.15.16:0/3915521792 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7facb015b120 0x7facb0073320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.430+0000 7facb6d5f640 1 --2- 172.21.15.16:0/3915521792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb00d04b0 0x7facb01473c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.430+0000 7facb6d5f640 1 --2- 172.21.15.16:0/3915521792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facb006dec0 0x7facb00b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.430+0000 7facb6d5f640 1 -- 172.21.15.16:0/3915521792 >> 172.21.15.16:0/3915521792 conn(0x7facb006bdf0 msgr2=0x7facb00b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.430+0000 7facb6d5f640 1 -- 172.21.15.16:0/3915521792 shutdown_connections 2024-09-19T07:45:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.430+0000 7facb6d5f640 1 -- 172.21.15.16:0/3915521792 wait complete. 2024-09-19T07:45:05.441 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T07:42:54.500338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T07:45:05.441 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T07:45:05.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.546+0000 7f3bac27b640 1 Processor -- start 2024-09-19T07:45:05.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.546+0000 7f3bac27b640 1 -- start start 2024-09-19T07:45:05.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.547+0000 7f3bac27b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3ba415e2a0 0x7f3ba406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.547+0000 7f3bac27b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ba40d1880 con 0x7f3ba415e680 2024-09-19T07:45:05.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.547+0000 7f3bac27b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ba40d1b60 con 0x7f3ba406d960 2024-09-19T07:45:05.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.547+0000 7f3bac27b640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f3ba40cf2d0 con 0x7f3ba415e2a0 2024-09-19T07:45:05.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.547+0000 7f3ba9816640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3ba415e680 0x7f3ba40cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58148/0 (socket says 172.21.15.16:58148) 2024-09-19T07:45:05.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.547+0000 7f3ba9816640 1 -- 172.21.15.16:0/3761210462 learned_addr learned my addr 172.21.15.16:0/3761210462 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:05.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.547+0000 7f3baa017640 1 --2- 172.21.15.16:0/3761210462 >> v2:172.21.15.146:3300/0 conn(0x7f3ba415e2a0 0x7f3ba406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:05.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.547+0000 7f3ba9015640 1 -- 172.21.15.16:0/3761210462 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1623625912 0 0) 0x7f3ba40d1880 con 0x7f3ba415e680 2024-09-19T07:45:05.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.547+0000 7f3ba9015640 1 -- 172.21.15.16:0/3761210462 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b90003520 con 0x7f3ba415e680 2024-09-19T07:45:05.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3ba9015640 1 -- 172.21.15.16:0/3761210462 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1517193284 0 0) 0x7f3b90003520 con 0x7f3ba415e680 2024-09-19T07:45:05.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3ba9015640 1 -- 172.21.15.16:0/3761210462 >> v2:172.21.15.146:3300/0 conn(0x7f3ba415e2a0 msgr2=0x7f3ba406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:05.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3ba9015640 1 --2- 172.21.15.16:0/3761210462 >> v2:172.21.15.146:3300/0 conn(0x7f3ba415e2a0 0x7f3ba406d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3ba9015640 1 -- 172.21.15.16:0/3761210462 >> v1:172.21.15.146:6789/0 conn(0x7f3ba406d960 legacy=0x7f3ba40d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:05.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3ba9015640 1 -- 172.21.15.16:0/3761210462 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba40cf5b0 con 0x7f3ba415e680 2024-09-19T07:45:05.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3ba9015640 1 -- 172.21.15.16:0/3761210462 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f3b94001670 con 0x7f3ba415e680 2024-09-19T07:45:05.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3ba9015640 1 -- 172.21.15.16:0/3761210462 >> v1:172.21.15.16:6789/0 conn(0x7f3ba415e680 legacy=0x7f3ba40cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:05.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3ba9015640 1 --2- 172.21.15.16:0/3761210462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b900039b0 0x7f3b90003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3ba9015640 1 --2- 172.21.15.16:0/3761210462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b900043b0 0x7f3b90004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3ba9015640 1 --2- 172.21.15.16:0/3761210462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3b90005080 0x7f3b9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3ba9015640 1 -- 172.21.15.16:0/3761210462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba40cf2d0 con 0x7f3b900039b0 2024-09-19T07:45:05.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3ba9015640 1 -- 172.21.15.16:0/3761210462 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9000bfe0 con 0x7f3b900043b0 2024-09-19T07:45:05.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3ba9015640 1 -- 172.21.15.16:0/3761210462 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9000c1e0 con 0x7f3b90005080 2024-09-19T07:45:05.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3baa017640 1 --2- 172.21.15.16:0/3761210462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b900039b0 0x7f3b90003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:05.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3ba9816640 1 --2- 172.21.15.16:0/3761210462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b900043b0 0x7f3b90004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:05.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3baa818640 1 --2- 172.21.15.16:0/3761210462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3b90005080 0x7f3b9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:05.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3baa017640 1 -- 172.21.15.16:0/3761210462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3b90005080 msgr2=0x7f3b9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:05.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3baa017640 1 --2- 172.21.15.16:0/3761210462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3b90005080 0x7f3b9000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3baa017640 1 -- 172.21.15.16:0/3761210462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b900043b0 msgr2=0x7f3b90004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:05.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3baa017640 1 --2- 172.21.15.16:0/3761210462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b900043b0 0x7f3b90004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3baa017640 1 -- 172.21.15.16:0/3761210462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b90003520 con 0x7f3b900039b0 2024-09-19T07:45:05.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.548+0000 7f3baa818640 1 -- 172.21.15.16:0/3761210462 reap_dead start 2024-09-19T07:45:05.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.549+0000 7f3baa017640 1 --2- 172.21.15.16:0/3761210462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b900039b0 0x7f3b90003d90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009ac0 tx=0x7f3b90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc28e63bd1dcaeb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:05.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.549+0000 7f3ba9015640 1 -- 172.21.15.16:0/3761210462 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0036070 con 0x7f3b900039b0 2024-09-19T07:45:05.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.549+0000 7f3ba9015640 1 -- 172.21.15.16:0/3761210462 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba0031400 con 0x7f3b900039b0 2024-09-19T07:45:05.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.549+0000 7f3ba9015640 1 -- 172.21.15.16:0/3761210462 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0009690 con 0x7f3b900039b0 2024-09-19T07:45:05.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.550+0000 7f3bac27b640 1 -- 172.21.15.16:0/3761210462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b900039b0 msgr2=0x7f3b90003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:05.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.550+0000 7f3bac27b640 1 --2- 172.21.15.16:0/3761210462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b900039b0 0x7f3b90003d90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009ac0 tx=0x7f3b90005830 comp rx=0 tx=0).stop 2024-09-19T07:45:05.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.550+0000 7f3bac27b640 1 -- 172.21.15.16:0/3761210462 shutdown_connections 2024-09-19T07:45:05.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.550+0000 7f3bac27b640 1 --2- 172.21.15.16:0/3761210462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b900039b0 0x7f3b90003d90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.550+0000 7f3bac27b640 1 -- 172.21.15.16:0/3761210462 >> 172.21.15.16:0/3761210462 conn(0x7f3ba409d760 msgr2=0x7f3ba408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:05.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.550+0000 7f3bac27b640 1 -- 172.21.15.16:0/3761210462 shutdown_connections 2024-09-19T07:45:05.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.550+0000 7f3bac27b640 1 -- 172.21.15.16:0/3761210462 wait complete. 2024-09-19T07:45:05.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.551+0000 7f3bac27b640 1 Processor -- start 2024-09-19T07:45:05.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.551+0000 7f3bac27b640 1 -- start start 2024-09-19T07:45:05.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.552+0000 7f3bac27b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b900039b0 0x7f3ba4159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.552+0000 7f3bac27b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ba40cde20 0x7f3ba40c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.552+0000 7f3bac27b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ba40ce2a0 0x7f3ba40ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.552+0000 7f3bac27b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba406d5a0 con 0x7f3b900039b0 2024-09-19T07:45:05.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.552+0000 7f3bac27b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba406d420 con 0x7f3ba40cde20 2024-09-19T07:45:05.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.552+0000 7f3bac27b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3ba406d720 con 0x7f3ba40ce2a0 2024-09-19T07:45:05.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.552+0000 7f3baa017640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b900039b0 0x7f3ba4159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:05.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.552+0000 7f3baa017640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b900039b0 0x7f3ba4159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57798/0 (socket says 172.21.15.16:57798) 2024-09-19T07:45:05.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.552+0000 7f3baa017640 1 -- 172.21.15.16:0/1218024742 learned_addr learned my addr 172.21.15.16:0/1218024742 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:05.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.552+0000 7f3baa017640 1 -- 172.21.15.16:0/1218024742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ba40ce2a0 msgr2=0x7f3ba40ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:05.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.552+0000 7f3baa017640 1 --2- 172.21.15.16:0/1218024742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ba40ce2a0 0x7f3ba40ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.552+0000 7f3baa017640 1 -- 172.21.15.16:0/1218024742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ba40cde20 msgr2=0x7f3ba40c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:05.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.552+0000 7f3baa017640 1 --2- 172.21.15.16:0/1218024742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ba40cde20 0x7f3ba40c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.552+0000 7f3baa017640 1 -- 172.21.15.16:0/1218024742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba40cb4b0 con 0x7f3b900039b0 2024-09-19T07:45:05.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.552+0000 7f3baa017640 1 --2- 172.21.15.16:0/1218024742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b900039b0 0x7f3ba4159c60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3ba002abb0 tx=0x7f3ba0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:05.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.552+0000 7f3b9affd640 1 -- 172.21.15.16:0/1218024742 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba003f020 con 0x7f3b900039b0 2024-09-19T07:45:05.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.552+0000 7f3b9affd640 1 -- 172.21.15.16:0/1218024742 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba0037040 con 0x7f3b900039b0 2024-09-19T07:45:05.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.553+0000 7f3bac27b640 1 -- 172.21.15.16:0/1218024742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba40b9140 con 0x7f3b900039b0 2024-09-19T07:45:05.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.553+0000 7f3b9affd640 1 -- 172.21.15.16:0/1218024742 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0036070 con 0x7f3b900039b0 2024-09-19T07:45:05.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.553+0000 7f3bac27b640 1 -- 172.21.15.16:0/1218024742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba40b9700 con 0x7f3b900039b0 2024-09-19T07:45:05.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.554+0000 7f3bac27b640 1 -- 172.21.15.16:0/1218024742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ba40cfad0 con 0x7f3b900039b0 2024-09-19T07:45:05.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.554+0000 7f3b9affd640 1 -- 172.21.15.16:0/1218024742 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3ba0042020 con 0x7f3b900039b0 2024-09-19T07:45:05.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.555+0000 7f3b9affd640 1 --2- 172.21.15.16:0/1218024742 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3b8806c2e0 0x7f3b8806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.555+0000 7f3b9affd640 1 -- 172.21.15.16:0/1218024742 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f3ba00ab4f0 con 0x7f3b900039b0 2024-09-19T07:45:05.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.555+0000 7f3ba9816640 1 --2- 172.21.15.16:0/1218024742 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3b8806c2e0 0x7f3b8806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:05.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.555+0000 7f3ba9816640 1 --2- 172.21.15.16:0/1218024742 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3b8806c2e0 0x7f3b8806e7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3b94000f60 tx=0x7f3b94000ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:05.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.559+0000 7f3b9affd640 1 -- 172.21.15.16:0/1218024742 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3ba0049020 con 0x7f3b900039b0 2024-09-19T07:45:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.735+0000 7f3bac27b640 1 -- 172.21.15.16:0/1218024742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3ba406cf90 con 0x7f3b900039b0 2024-09-19T07:45:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.735+0000 7f3b9affd640 1 -- 172.21.15.16:0/1218024742 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) ==== 93+0+10 (secure 0 0 0) 0x7f3ba0079e80 con 0x7f3b900039b0 2024-09-19T07:45:05.736 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-19T07:45:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.737+0000 7f3bac27b640 1 -- 172.21.15.16:0/1218024742 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3b8806c2e0 msgr2=0x7f3b8806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.737+0000 7f3bac27b640 1 --2- 172.21.15.16:0/1218024742 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3b8806c2e0 0x7f3b8806e7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3b94000f60 tx=0x7f3b94000ed0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.737+0000 7f3bac27b640 1 -- 172.21.15.16:0/1218024742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b900039b0 msgr2=0x7f3ba4159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.737+0000 7f3bac27b640 1 --2- 172.21.15.16:0/1218024742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b900039b0 0x7f3ba4159c60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3ba002abb0 tx=0x7f3ba0004730 comp rx=0 tx=0).stop 2024-09-19T07:45:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.737+0000 7f3bac27b640 1 -- 172.21.15.16:0/1218024742 shutdown_connections 2024-09-19T07:45:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.737+0000 7f3bac27b640 1 --2- 172.21.15.16:0/1218024742 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3b8806c2e0 0x7f3b8806e7a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.737+0000 7f3bac27b640 1 --2- 172.21.15.16:0/1218024742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ba40ce2a0 0x7f3ba40ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.737+0000 7f3bac27b640 1 --2- 172.21.15.16:0/1218024742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ba40cde20 0x7f3ba40c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.737+0000 7f3bac27b640 1 --2- 172.21.15.16:0/1218024742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b900039b0 0x7f3ba4159c60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.737+0000 7f3bac27b640 1 -- 172.21.15.16:0/1218024742 >> 172.21.15.16:0/1218024742 conn(0x7f3ba409d760 msgr2=0x7f3ba415f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.737+0000 7f3bac27b640 1 -- 172.21.15.16:0/1218024742 shutdown_connections 2024-09-19T07:45:05.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.737+0000 7f3bac27b640 1 -- 172.21.15.16:0/1218024742 wait complete. 2024-09-19T07:45:05.749 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid' 2024-09-19T07:45:05.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.850+0000 7f9b30d31640 1 Processor -- start 2024-09-19T07:45:05.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.850+0000 7f9b30d31640 1 -- start start 2024-09-19T07:45:05.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.851+0000 7f9b30d31640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b2c0d0070 0x7f9b2c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.851+0000 7f9b30d31640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2c0d4130 0x7f9b2c0d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.851+0000 7f9b30d31640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b2c0d4ae0 con 0x7f9b2c15f0d0 2024-09-19T07:45:05.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.851+0000 7f9b30d31640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9b2c0d1b10 con 0x7f9b2c0d0070 2024-09-19T07:45:05.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.851+0000 7f9b30d31640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9b2c0cf2b0 con 0x7f9b2c0d4130 2024-09-19T07:45:05.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.851+0000 7f9b29d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b2c0d0070 0x7f9b2c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:05.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.851+0000 7f9b29d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b2c0d0070 0x7f9b2c15f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57808/0 (socket says 172.21.15.16:57808) 2024-09-19T07:45:05.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.851+0000 7f9b29d9b640 1 -- 172.21.15.16:0/2804768336 learned_addr learned my addr 172.21.15.16:0/2804768336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:05.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.852+0000 7f9b2ad9d640 1 --2- 172.21.15.16:0/2804768336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2c0d4130 0x7f9b2c0d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:05.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.852+0000 7f9b29d9b640 1 -- 172.21.15.16:0/2804768336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2c0d4130 msgr2=0x7f9b2c0d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:05.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.852+0000 7f9b29d9b640 1 --2- 172.21.15.16:0/2804768336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2c0d4130 0x7f9b2c0d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.852+0000 7f9b29d9b640 1 -- 172.21.15.16:0/2804768336 >> v1:172.21.15.16:6789/0 conn(0x7f9b2c15f0d0 legacy=0x7f9b2c0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:05.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.852+0000 7f9b29d9b640 1 -- 172.21.15.16:0/2804768336 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2c0cf590 con 0x7f9b2c0d0070 2024-09-19T07:45:05.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.852+0000 7f9b2959a640 1 -- 172.21.15.16:0/2804768336 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2306060932 0 0) 0x7f9b2c0d4ae0 con 0x7f9b2c15f0d0 2024-09-19T07:45:05.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.852+0000 7f9b29d9b640 1 --2- 172.21.15.16:0/2804768336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b2c0d0070 0x7f9b2c15f4b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009e50 tx=0x7f9b2002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ec6f7dbf6817b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:05.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.853+0000 7f9b2959a640 1 -- 172.21.15.16:0/2804768336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20036070 con 0x7f9b2c0d0070 2024-09-19T07:45:05.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.853+0000 7f9b2959a640 1 -- 172.21.15.16:0/2804768336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b20031600 con 0x7f9b2c0d0070 2024-09-19T07:45:05.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.853+0000 7f9b2959a640 1 -- 172.21.15.16:0/2804768336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b200319d0 con 0x7f9b2c0d0070 2024-09-19T07:45:05.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.853+0000 7f9b30d31640 1 -- 172.21.15.16:0/2804768336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b2c0d0070 msgr2=0x7f9b2c15f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:05.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.853+0000 7f9b30d31640 1 --2- 172.21.15.16:0/2804768336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b2c0d0070 0x7f9b2c15f4b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009e50 tx=0x7f9b2002a680 comp rx=0 tx=0).stop 2024-09-19T07:45:05.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.853+0000 7f9b30d31640 1 -- 172.21.15.16:0/2804768336 shutdown_connections 2024-09-19T07:45:05.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.853+0000 7f9b30d31640 1 --2- 172.21.15.16:0/2804768336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2c0d4130 0x7f9b2c0d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.853+0000 7f9b30d31640 1 --2- 172.21.15.16:0/2804768336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b2c0d0070 0x7f9b2c15f4b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.853+0000 7f9b30d31640 1 -- 172.21.15.16:0/2804768336 >> 172.21.15.16:0/2804768336 conn(0x7f9b2c09d760 msgr2=0x7f9b2c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:05.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.853+0000 7f9b30d31640 1 -- 172.21.15.16:0/2804768336 shutdown_connections 2024-09-19T07:45:05.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.853+0000 7f9b30d31640 1 -- 172.21.15.16:0/2804768336 wait complete. 2024-09-19T07:45:05.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.855+0000 7f9b30d31640 1 Processor -- start 2024-09-19T07:45:05.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.855+0000 7f9b30d31640 1 -- start start 2024-09-19T07:45:05.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.855+0000 7f9b30d31640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2c0d0070 0x7f9b2c0cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.855+0000 7f9b30d31640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b2c0d4130 0x7f9b2c161610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.855+0000 7f9b30d31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b2c15f0d0 0x7f9b2c14e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.855+0000 7f9b30d31640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c0cef60 con 0x7f9b2c15f0d0 2024-09-19T07:45:05.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.855+0000 7f9b30d31640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c0cfb50 con 0x7f9b2c0d4130 2024-09-19T07:45:05.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.855+0000 7f9b30d31640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9b2c0b2f70 con 0x7f9b2c0d0070 2024-09-19T07:45:05.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.856+0000 7f9b2ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b2c15f0d0 0x7f9b2c14e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:05.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.856+0000 7f9b2ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b2c15f0d0 0x7f9b2c14e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57822/0 (socket says 172.21.15.16:57822) 2024-09-19T07:45:05.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.856+0000 7f9b2ad9d640 1 -- 172.21.15.16:0/4115587895 learned_addr learned my addr 172.21.15.16:0/4115587895 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:05.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.856+0000 7f9b29d9b640 1 --2- 172.21.15.16:0/4115587895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b2c0d4130 0x7f9b2c161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:05.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.856+0000 7f9b2a59c640 1 --2- 172.21.15.16:0/4115587895 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2c0d0070 0x7f9b2c0cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:05.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.856+0000 7f9b2ad9d640 1 -- 172.21.15.16:0/4115587895 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2c0d0070 msgr2=0x7f9b2c0cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:05.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.856+0000 7f9b2ad9d640 1 --2- 172.21.15.16:0/4115587895 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2c0d0070 0x7f9b2c0cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.856+0000 7f9b2ad9d640 1 -- 172.21.15.16:0/4115587895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b2c0d4130 msgr2=0x7f9b2c161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:05.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.856+0000 7f9b2ad9d640 1 --2- 172.21.15.16:0/4115587895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b2c0d4130 0x7f9b2c161610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:05.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.856+0000 7f9b2ad9d640 1 -- 172.21.15.16:0/4115587895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2c14ec30 con 0x7f9b2c15f0d0 2024-09-19T07:45:05.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.856+0000 7f9b2ad9d640 1 --2- 172.21.15.16:0/4115587895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b2c15f0d0 0x7f9b2c14e6f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00d800 tx=0x7f9b1c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:05.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.856+0000 7f9b1affd640 1 -- 172.21.15.16:0/4115587895 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c00b6c0 con 0x7f9b2c15f0d0 2024-09-19T07:45:05.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.856+0000 7f9b1affd640 1 -- 172.21.15.16:0/4115587895 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b1c019070 con 0x7f9b2c15f0d0 2024-09-19T07:45:05.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.856+0000 7f9b30d31640 1 -- 172.21.15.16:0/4115587895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b2c15dd20 con 0x7f9b2c15f0d0 2024-09-19T07:45:05.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.856+0000 7f9b1affd640 1 -- 172.21.15.16:0/4115587895 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c00bac0 con 0x7f9b2c15f0d0 2024-09-19T07:45:05.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.856+0000 7f9b30d31640 1 -- 172.21.15.16:0/4115587895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b2c15e2e0 con 0x7f9b2c15f0d0 2024-09-19T07:45:05.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.857+0000 7f9b30d31640 1 -- 172.21.15.16:0/4115587895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b2c06d6f0 con 0x7f9b2c15f0d0 2024-09-19T07:45:05.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.858+0000 7f9b1affd640 1 -- 172.21.15.16:0/4115587895 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9b1c00bc60 con 0x7f9b2c15f0d0 2024-09-19T07:45:05.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.858+0000 7f9b1affd640 1 --2- 172.21.15.16:0/4115587895 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b0c06c2b0 0x7f9b0c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:05.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.858+0000 7f9b1affd640 1 -- 172.21.15.16:0/4115587895 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f9b1c08b7c0 con 0x7f9b2c15f0d0 2024-09-19T07:45:05.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.858+0000 7f9b2a59c640 1 --2- 172.21.15.16:0/4115587895 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b0c06c2b0 0x7f9b0c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:05.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.859+0000 7f9b2a59c640 1 --2- 172.21.15.16:0/4115587895 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b0c06c2b0 0x7f9b0c06e770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9b24007fd0 tx=0x7f9b24009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:05.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:05.863+0000 7f9b1affd640 1 -- 172.21.15.16:0/4115587895 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9b1c05a010 con 0x7f9b2c15f0d0 2024-09-19T07:45:06.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.106+0000 7f9b30d31640 1 -- 172.21.15.16:0/4115587895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]} v 0) -- 0x7f9b2c0d4740 con 0x7f9b2c15f0d0 2024-09-19T07:45:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.110+0000 7f9b1affd640 1 -- 172.21.15.16:0/4115587895 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]}]=0 v119) ==== 165+0+0 (secure 0 0 0) 0x7f9b1c059910 con 0x7f9b2c15f0d0 2024-09-19T07:45:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.112+0000 7f9b30d31640 1 -- 172.21.15.16:0/4115587895 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b0c06c2b0 msgr2=0x7f9b0c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.112+0000 7f9b30d31640 1 --2- 172.21.15.16:0/4115587895 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b0c06c2b0 0x7f9b0c06e770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9b24007fd0 tx=0x7f9b24009040 comp rx=0 tx=0).stop 2024-09-19T07:45:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.112+0000 7f9b30d31640 1 -- 172.21.15.16:0/4115587895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b2c15f0d0 msgr2=0x7f9b2c14e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.112+0000 7f9b30d31640 1 --2- 172.21.15.16:0/4115587895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b2c15f0d0 0x7f9b2c14e6f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00d800 tx=0x7f9b1c00dcc0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.112+0000 7f9b30d31640 1 -- 172.21.15.16:0/4115587895 shutdown_connections 2024-09-19T07:45:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.112+0000 7f9b30d31640 1 --2- 172.21.15.16:0/4115587895 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b0c06c2b0 0x7f9b0c06e770 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.112+0000 7f9b30d31640 1 --2- 172.21.15.16:0/4115587895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b2c15f0d0 0x7f9b2c14e6f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.112+0000 7f9b30d31640 1 --2- 172.21.15.16:0/4115587895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b2c0d4130 0x7f9b2c161610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.112+0000 7f9b30d31640 1 --2- 172.21.15.16:0/4115587895 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2c0d0070 0x7f9b2c0cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.112+0000 7f9b30d31640 1 -- 172.21.15.16:0/4115587895 >> 172.21.15.16:0/4115587895 conn(0x7f9b2c09d760 msgr2=0x7f9b2c0d1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:06.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.112+0000 7f9b30d31640 1 -- 172.21.15.16:0/4115587895 shutdown_connections 2024-09-19T07:45:06.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.112+0000 7f9b30d31640 1 -- 172.21.15.16:0/4115587895 wait complete. 2024-09-19T07:45:06.123 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T07:45:06.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.228+0000 7f2906250640 1 Processor -- start 2024-09-19T07:45:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.228+0000 7f2906250640 1 -- start start 2024-09-19T07:45:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.229+0000 7f2906250640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f29000c8d20 0x7f290006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.229+0000 7f2906250640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2900144100 0x7f29000d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.229+0000 7f2906250640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29000c7140 con 0x7f290006d260 2024-09-19T07:45:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.229+0000 7f2906250640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f290006d6d0 con 0x7f2900144100 2024-09-19T07:45:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.229+0000 7f2906250640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2900141960 con 0x7f29000c8d20 2024-09-19T07:45:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.229+0000 7f28fffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2900144100 0x7f29000d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.229+0000 7f28fffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2900144100 0x7f29000d1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57828/0 (socket says 172.21.15.16:57828) 2024-09-19T07:45:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.229+0000 7f28fffff640 1 -- 172.21.15.16:0/2952517225 learned_addr learned my addr 172.21.15.16:0/2952517225 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.229+0000 7f28ff7fe640 1 --2- 172.21.15.16:0/2952517225 >> v2:172.21.15.146:3300/0 conn(0x7f29000c8d20 0x7f290006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.229+0000 7f28fffff640 1 -- 172.21.15.16:0/2952517225 >> v2:172.21.15.146:3300/0 conn(0x7f29000c8d20 msgr2=0x7f290006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.229+0000 7f28fffff640 1 --2- 172.21.15.16:0/2952517225 >> v2:172.21.15.146:3300/0 conn(0x7f29000c8d20 0x7f290006cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.229+0000 7f28fffff640 1 -- 172.21.15.16:0/2952517225 >> v1:172.21.15.146:6789/0 conn(0x7f290006d260 legacy=0x7f29001439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.229+0000 7f28fffff640 1 -- 172.21.15.16:0/2952517225 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29000c9100 con 0x7f2900144100 2024-09-19T07:45:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.229+0000 7f28ff7fe640 1 --2- 172.21.15.16:0/2952517225 >> v2:172.21.15.146:3300/0 conn(0x7f29000c8d20 0x7f290006cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.230+0000 7f28fffff640 1 --2- 172.21.15.16:0/2952517225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2900144100 0x7f29000d1e50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f29000d2e50 tx=0x7f28f802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a5bab66673c37f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.230+0000 7f28feffd640 1 -- 172.21.15.16:0/2952517225 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8036070 con 0x7f2900144100 2024-09-19T07:45:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.230+0000 7f28feffd640 1 -- 172.21.15.16:0/2952517225 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f8031600 con 0x7f2900144100 2024-09-19T07:45:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.230+0000 7f28feffd640 1 -- 172.21.15.16:0/2952517225 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f80319d0 con 0x7f2900144100 2024-09-19T07:45:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.231+0000 7f2906250640 1 -- 172.21.15.16:0/2952517225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2900144100 msgr2=0x7f29000d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.231+0000 7f2906250640 1 --2- 172.21.15.16:0/2952517225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2900144100 0x7f29000d1e50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f29000d2e50 tx=0x7f28f802a5d0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.231+0000 7f2906250640 1 -- 172.21.15.16:0/2952517225 shutdown_connections 2024-09-19T07:45:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.231+0000 7f2906250640 1 --2- 172.21.15.16:0/2952517225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2900144100 0x7f29000d1e50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.231+0000 7f2906250640 1 --2- 172.21.15.16:0/2952517225 >> v2:172.21.15.146:3300/0 conn(0x7f29000c8d20 0x7f290006cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.231+0000 7f2906250640 1 -- 172.21.15.16:0/2952517225 >> 172.21.15.16:0/2952517225 conn(0x7f29000b3ed0 msgr2=0x7f29000b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.231+0000 7f2906250640 1 -- 172.21.15.16:0/2952517225 shutdown_connections 2024-09-19T07:45:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.231+0000 7f2906250640 1 -- 172.21.15.16:0/2952517225 wait complete. 2024-09-19T07:45:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.232+0000 7f2906250640 1 Processor -- start 2024-09-19T07:45:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.232+0000 7f2906250640 1 -- start start 2024-09-19T07:45:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.233+0000 7f2906250640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f290006d260 0x7f2900140bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.233+0000 7f2906250640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29000c8d20 0x7f29000cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.233+0000 7f2906250640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2900144100 0x7f29000cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.233+0000 7f2906250640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2900144510 con 0x7f29000c8d20 2024-09-19T07:45:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.233+0000 7f2906250640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2900142200 con 0x7f2900144100 2024-09-19T07:45:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.233+0000 7f2906250640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2900141bb0 con 0x7f290006d260 2024-09-19T07:45:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.233+0000 7f28f6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29000c8d20 0x7f29000cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.233+0000 7f28f6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29000c8d20 0x7f29000cd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57830/0 (socket says 172.21.15.16:57830) 2024-09-19T07:45:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.233+0000 7f28f6ffd640 1 -- 172.21.15.16:0/3383684193 learned_addr learned my addr 172.21.15.16:0/3383684193 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.233+0000 7f28fffff640 1 --2- 172.21.15.16:0/3383684193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2900144100 0x7f29000cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.233+0000 7f28ff7fe640 1 --2- 172.21.15.16:0/3383684193 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f290006d260 0x7f2900140bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.233+0000 7f28f6ffd640 1 -- 172.21.15.16:0/3383684193 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f290006d260 msgr2=0x7f2900140bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.233+0000 7f28f6ffd640 1 --2- 172.21.15.16:0/3383684193 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f290006d260 0x7f2900140bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.233+0000 7f28f6ffd640 1 -- 172.21.15.16:0/3383684193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2900144100 msgr2=0x7f29000cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.233+0000 7f28f6ffd640 1 --2- 172.21.15.16:0/3383684193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2900144100 0x7f29000cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.234+0000 7f28f6ffd640 1 -- 172.21.15.16:0/3383684193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29000ccb50 con 0x7f29000c8d20 2024-09-19T07:45:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.234+0000 7f28fffff640 1 --2- 172.21.15.16:0/3383684193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2900144100 0x7f29000cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.234+0000 7f28ff7fe640 1 --2- 172.21.15.16:0/3383684193 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f290006d260 0x7f2900140bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.234+0000 7f28f6ffd640 1 --2- 172.21.15.16:0/3383684193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29000c8d20 0x7f29000cd7d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f28e800b3f0 tx=0x7f28e800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.234+0000 7f28fcff9640 1 -- 172.21.15.16:0/3383684193 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8015020 con 0x7f29000c8d20 2024-09-19T07:45:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.234+0000 7f28fcff9640 1 -- 172.21.15.16:0/3383684193 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e8011830 con 0x7f29000c8d20 2024-09-19T07:45:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.234+0000 7f2906250640 1 -- 172.21.15.16:0/3383684193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29000ce340 con 0x7f29000c8d20 2024-09-19T07:45:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.234+0000 7f28fcff9640 1 -- 172.21.15.16:0/3383684193 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8011c20 con 0x7f29000c8d20 2024-09-19T07:45:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.234+0000 7f2906250640 1 -- 172.21.15.16:0/3383684193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29000ce900 con 0x7f29000c8d20 2024-09-19T07:45:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.235+0000 7f2906250640 1 -- 172.21.15.16:0/3383684193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f290006cd20 con 0x7f29000c8d20 2024-09-19T07:45:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.235+0000 7f28fcff9640 1 -- 172.21.15.16:0/3383684193 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f28e8018020 con 0x7f29000c8d20 2024-09-19T07:45:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.236+0000 7f28fcff9640 1 --2- 172.21.15.16:0/3383684193 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f28e006c2b0 0x7f28e006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.236+0000 7f28fcff9640 1 -- 172.21.15.16:0/3383684193 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f28e808b1a0 con 0x7f29000c8d20 2024-09-19T07:45:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.236+0000 7f28ff7fe640 1 --2- 172.21.15.16:0/3383684193 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f28e006c2b0 0x7f28e006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.237+0000 7f28ff7fe640 1 --2- 172.21.15.16:0/3383684193 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f28e006c2b0 0x7f28e006e770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f28f0006fd0 tx=0x7f28f0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.240+0000 7f28fcff9640 1 -- 172.21.15.16:0/3383684193 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f28e801f020 con 0x7f29000c8d20 2024-09-19T07:45:06.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.416+0000 7f2906250640 1 -- 172.21.15.16:0/3383684193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2900072520 con 0x7f29000c8d20 2024-09-19T07:45:06.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.419+0000 7f28fcff9640 1 -- 172.21.15.16:0/3383684193 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f28e80172c0 con 0x7f29000c8d20 2024-09-19T07:45:06.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.422+0000 7f2906250640 1 -- 172.21.15.16:0/3383684193 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f28e006c2b0 msgr2=0x7f28e006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:06.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.422+0000 7f2906250640 1 --2- 172.21.15.16:0/3383684193 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f28e006c2b0 0x7f28e006e770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f28f0006fd0 tx=0x7f28f0008040 comp rx=0 tx=0).stop 2024-09-19T07:45:06.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.422+0000 7f2906250640 1 -- 172.21.15.16:0/3383684193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29000c8d20 msgr2=0x7f29000cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:06.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.422+0000 7f2906250640 1 --2- 172.21.15.16:0/3383684193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29000c8d20 0x7f29000cd7d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f28e800b3f0 tx=0x7f28e800b8b0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.422+0000 7f2906250640 1 -- 172.21.15.16:0/3383684193 shutdown_connections 2024-09-19T07:45:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.422+0000 7f2906250640 1 --2- 172.21.15.16:0/3383684193 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f28e006c2b0 0x7f28e006e770 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.422+0000 7f2906250640 1 --2- 172.21.15.16:0/3383684193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2900144100 0x7f29000cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.422+0000 7f2906250640 1 --2- 172.21.15.16:0/3383684193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29000c8d20 0x7f29000cd7d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.422+0000 7f2906250640 1 --2- 172.21.15.16:0/3383684193 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f290006d260 0x7f2900140bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.422+0000 7f2906250640 1 -- 172.21.15.16:0/3383684193 >> 172.21.15.16:0/3383684193 conn(0x7f29000b3ed0 msgr2=0x7f290015fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.422+0000 7f2906250640 1 -- 172.21.15.16:0/3383684193 shutdown_connections 2024-09-19T07:45:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.422+0000 7f2906250640 1 -- 172.21.15.16:0/3383684193 wait complete. 2024-09-19T07:45:06.433 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:45:06.434 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:45:06.475 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:45:06.523 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:45:06.523 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:45:06.523 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:45:06.539 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:45:06.539 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:45:06.539 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:45:06.557 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:45:06.557 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:45:06.558 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:45:06.595 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:45:06.595 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:45:06.595 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:45:06.616 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:45:06.616 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T07:45:06.616 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:45:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.736+0000 7f17ae806640 1 Processor -- start 2024-09-19T07:45:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.737+0000 7f17ae806640 1 -- start start 2024-09-19T07:45:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.737+0000 7f17ae806640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17a815cb00 0x7f17a80d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.737+0000 7f17ae806640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f17a815cee0 0x7f17a815d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.737+0000 7f17ae806640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17a80d22d0 con 0x7f17a80cfdb0 2024-09-19T07:45:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.737+0000 7f17ae806640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f17a80b40b0 con 0x7f17a815cee0 2024-09-19T07:45:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.737+0000 7f17ae806640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f17a80d25b0 con 0x7f17a815cb00 2024-09-19T07:45:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.738+0000 7f179ffff640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f17a815cee0 0x7f17a815d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.738+0000 7f17a7fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17a815cb00 0x7f17a80d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.738+0000 7f179ffff640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f17a815cee0 0x7f17a815d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:40650/0 (socket says 172.21.15.16:40650) 2024-09-19T07:45:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.738+0000 7f179ffff640 1 -- 172.21.15.16:0/3992817482 learned_addr learned my addr 172.21.15.16:0/3992817482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.738+0000 7f179ffff640 1 -- 172.21.15.16:0/3992817482 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17a815cb00 msgr2=0x7f17a80d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:06.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.738+0000 7f179ffff640 1 --2- 172.21.15.16:0/3992817482 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17a815cb00 0x7f17a80d1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.738+0000 7f179ffff640 1 -- 172.21.15.16:0/3992817482 >> v1:172.21.15.146:6789/0 conn(0x7f17a80cfdb0 legacy=0x7f17a81562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:06.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.738+0000 7f179ffff640 1 -- 172.21.15.16:0/3992817482 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a80d2890 con 0x7f17a815cee0 2024-09-19T07:45:06.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.738+0000 7f17a7fff640 1 --2- 172.21.15.16:0/3992817482 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17a815cb00 0x7f17a80d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:06.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.739+0000 7f179ffff640 1 --2- 172.21.15.16:0/3992817482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17a815cee0 0x7f17a815d340 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1790009870 tx=0x7f179002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7060df4a0e782d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:06.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.739+0000 7f17a77fe640 1 -- 172.21.15.16:0/3992817482 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790037070 con 0x7f17a815cee0 2024-09-19T07:45:06.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.739+0000 7f17a77fe640 1 -- 172.21.15.16:0/3992817482 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1790032440 con 0x7f17a815cee0 2024-09-19T07:45:06.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.739+0000 7f17a77fe640 1 -- 172.21.15.16:0/3992817482 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790032810 con 0x7f17a815cee0 2024-09-19T07:45:06.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.740+0000 7f17ae806640 1 -- 172.21.15.16:0/3992817482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17a815cee0 msgr2=0x7f17a815d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:06.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.740+0000 7f17ae806640 1 --2- 172.21.15.16:0/3992817482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17a815cee0 0x7f17a815d340 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1790009870 tx=0x7f179002a1c0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.740+0000 7f17ae806640 1 -- 172.21.15.16:0/3992817482 shutdown_connections 2024-09-19T07:45:06.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.740+0000 7f17ae806640 1 --2- 172.21.15.16:0/3992817482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17a815cee0 0x7f17a815d340 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.740+0000 7f17ae806640 1 --2- 172.21.15.16:0/3992817482 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17a815cb00 0x7f17a80d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.740+0000 7f17ae806640 1 -- 172.21.15.16:0/3992817482 >> 172.21.15.16:0/3992817482 conn(0x7f17a8067fe0 msgr2=0x7f17a806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:06.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.740+0000 7f17ae806640 1 -- 172.21.15.16:0/3992817482 shutdown_connections 2024-09-19T07:45:06.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.740+0000 7f17ae806640 1 -- 172.21.15.16:0/3992817482 wait complete. 2024-09-19T07:45:06.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.741+0000 7f17ae806640 1 Processor -- start 2024-09-19T07:45:06.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.741+0000 7f17ae806640 1 -- start start 2024-09-19T07:45:06.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.742+0000 7f17ae806640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17a80cfdb0 0x7f17a806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:06.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.742+0000 7f17ae806640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17a815cb00 0x7f17a80b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:06.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.742+0000 7f17ae806640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17a815cee0 0x7f17a80cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:06.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.742+0000 7f17ae806640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17a8156b50 con 0x7f17a815cee0 2024-09-19T07:45:06.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.742+0000 7f17ae806640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f17a81569d0 con 0x7f17a80cfdb0 2024-09-19T07:45:06.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.742+0000 7f17ae806640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f17a8156cd0 con 0x7f17a815cb00 2024-09-19T07:45:06.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.742+0000 7f17acda3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17a815cee0 0x7f17a80cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:06.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.742+0000 7f17acda3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17a815cee0 0x7f17a80cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57834/0 (socket says 172.21.15.16:57834) 2024-09-19T07:45:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.742+0000 7f17acda3640 1 -- 172.21.15.16:0/2673799158 learned_addr learned my addr 172.21.15.16:0/2673799158 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.742+0000 7f179ffff640 1 --2- 172.21.15.16:0/2673799158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17a815cb00 0x7f17a80b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.742+0000 7f17a7fff640 1 --2- 172.21.15.16:0/2673799158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17a80cfdb0 0x7f17a806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.742+0000 7f17acda3640 1 -- 172.21.15.16:0/2673799158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17a815cb00 msgr2=0x7f17a80b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.742+0000 7f17acda3640 1 --2- 172.21.15.16:0/2673799158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17a815cb00 0x7f17a80b6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.742+0000 7f17acda3640 1 -- 172.21.15.16:0/2673799158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17a80cfdb0 msgr2=0x7f17a806e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.742+0000 7f17acda3640 1 --2- 172.21.15.16:0/2673799158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17a80cfdb0 0x7f17a806e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.742+0000 7f17acda3640 1 -- 172.21.15.16:0/2673799158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a815f0d0 con 0x7f17a815cee0 2024-09-19T07:45:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.743+0000 7f17acda3640 1 --2- 172.21.15.16:0/2673799158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17a815cee0 0x7f17a80cba80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f17a000c730 tx=0x7f17a000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.743+0000 7f17a57fa640 1 -- 172.21.15.16:0/2673799158 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0016020 con 0x7f17a815cee0 2024-09-19T07:45:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.743+0000 7f17a57fa640 1 -- 172.21.15.16:0/2673799158 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17a0006d50 con 0x7f17a815cee0 2024-09-19T07:45:06.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.743+0000 7f17ae806640 1 -- 172.21.15.16:0/2673799158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a806d860 con 0x7f17a815cee0 2024-09-19T07:45:06.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.743+0000 7f17a57fa640 1 -- 172.21.15.16:0/2673799158 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0003bb0 con 0x7f17a815cee0 2024-09-19T07:45:06.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.743+0000 7f17ae806640 1 -- 172.21.15.16:0/2673799158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a806de20 con 0x7f17a815cee0 2024-09-19T07:45:06.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.744+0000 7f17ae806640 1 -- 172.21.15.16:0/2673799158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a8141020 con 0x7f17a815cee0 2024-09-19T07:45:06.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.744+0000 7f17a57fa640 1 -- 172.21.15.16:0/2673799158 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f17a00050d0 con 0x7f17a815cee0 2024-09-19T07:45:06.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.745+0000 7f17a57fa640 1 --2- 172.21.15.16:0/2673799158 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f178806c2b0 0x7f178806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:06.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.745+0000 7f17a57fa640 1 -- 172.21.15.16:0/2673799158 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f17a008ad60 con 0x7f17a815cee0 2024-09-19T07:45:06.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.745+0000 7f17a7fff640 1 --2- 172.21.15.16:0/2673799158 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f178806c2b0 0x7f178806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:06.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.745+0000 7f17a7fff640 1 --2- 172.21.15.16:0/2673799158 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f178806c2b0 0x7f178806e770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1798009560 tx=0x7f1798009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:06.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.749+0000 7f17a57fa640 1 -- 172.21.15.16:0/2673799158 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f17a8141020 con 0x7f17a815cee0 2024-09-19T07:45:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.959+0000 7f17ae806640 1 -- 172.21.15.16:0/2673799158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17a80d1ca0 con 0x7f17a815cee0 2024-09-19T07:45:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.959+0000 7f17a57fa640 1 -- 172.21.15.16:0/2673799158 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+3917 (secure 0 0 0) 0x7f17a0020020 con 0x7f17a815cee0 2024-09-19T07:45:06.960 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:45:06.960 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":18,"btime":"2024-09-19T07:45:01:093114+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4655,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/3238551814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3238551814},{"type":"v1","addr":"172.21.15.146:6836","nonce":3238551814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4813,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3450384245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3450384245},{"type":"v1","addr":"172.21.15.146:6837","nonce":3450384245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4944,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3663416804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3663416804},{"type":"v1","addr":"172.21.15.16:6834","nonce":3663416804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5256,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2126397673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2126397673},{"type":"v1","addr":"172.21.15.16:6835","nonce":2126397673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[]} 2024-09-19T07:45:06.960 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 18 2024-09-19T07:45:06.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.961+0000 7f17ae806640 1 -- 172.21.15.16:0/2673799158 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f178806c2b0 msgr2=0x7f178806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:06.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.961+0000 7f17ae806640 1 --2- 172.21.15.16:0/2673799158 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f178806c2b0 0x7f178806e770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1798009560 tx=0x7f1798009190 comp rx=0 tx=0).stop 2024-09-19T07:45:06.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.961+0000 7f17ae806640 1 -- 172.21.15.16:0/2673799158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17a815cee0 msgr2=0x7f17a80cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:06.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.961+0000 7f17ae806640 1 --2- 172.21.15.16:0/2673799158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17a815cee0 0x7f17a80cba80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f17a000c730 tx=0x7f17a000cb30 comp rx=0 tx=0).stop 2024-09-19T07:45:06.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.961+0000 7f17ae806640 1 -- 172.21.15.16:0/2673799158 shutdown_connections 2024-09-19T07:45:06.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.961+0000 7f17ae806640 1 --2- 172.21.15.16:0/2673799158 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f178806c2b0 0x7f178806e770 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.961+0000 7f17ae806640 1 --2- 172.21.15.16:0/2673799158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17a815cee0 0x7f17a80cba80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.961+0000 7f17ae806640 1 --2- 172.21.15.16:0/2673799158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17a815cb00 0x7f17a80b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.961+0000 7f17ae806640 1 --2- 172.21.15.16:0/2673799158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17a80cfdb0 0x7f17a806e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:06.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.961+0000 7f17ae806640 1 -- 172.21.15.16:0/2673799158 >> 172.21.15.16:0/2673799158 conn(0x7f17a8067fe0 msgr2=0x7f17a80c89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:06.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.961+0000 7f17ae806640 1 -- 172.21.15.16:0/2673799158 shutdown_connections 2024-09-19T07:45:06.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:06.961+0000 7f17ae806640 1 -- 172.21.15.16:0/2673799158 wait complete. 2024-09-19T07:45:06.973 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T07:45:06.973 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T07:45:06.973 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T07:45:06.973 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:45:06.974 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T07:45:06.974 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T07:45:06.974 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T07:45:06.974 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:45:06.974 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T07:45:06.974 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T07:45:06.974 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T07:45:06.975 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:45:06.975 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T07:45:06.975 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T07:45:06.975 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T07:45:06.975 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:45:06.983 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T07:45:06.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T07:45:06.984 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T07:45:06.984 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T07:45:06.987 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T07:45:06.987 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T07:45:06.988 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T07:45:06.988 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T07:45:07.026 INFO:tasks.ceph.mds.a:Started 2024-09-19T07:45:07.026 DEBUG:teuthology.parallel:result is None 2024-09-19T07:45:07.027 INFO:tasks.ceph.mds.c:Started 2024-09-19T07:45:07.027 DEBUG:teuthology.parallel:result is None 2024-09-19T07:45:07.029 INFO:tasks.ceph.mds.d:Started 2024-09-19T07:45:07.029 DEBUG:teuthology.parallel:result is None 2024-09-19T07:45:07.030 INFO:tasks.ceph.mds.b:Started 2024-09-19T07:45:07.030 DEBUG:teuthology.parallel:result is None 2024-09-19T07:45:07.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T07:45:07.103 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-19T07:45:07.103 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-19T07:45:07.110 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2024-09-19T07:45:07.110 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2024-09-19T07:45:07.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.140+0000 7fd804d8d640 1 Processor -- start 2024-09-19T07:45:07.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.140+0000 7fd804d8d640 1 -- start start 2024-09-19T07:45:07.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.141+0000 7fd804d8d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd80015bee0 0x7fd8000d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.141+0000 7fd804d8d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8000cf910 0x7fd8000d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.141+0000 7fd804d8d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8000cdee0 con 0x7fd80015c2c0 2024-09-19T07:45:07.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.141+0000 7fd804d8d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fd8000ceaf0 con 0x7fd80015bee0 2024-09-19T07:45:07.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.141+0000 7fd804d8d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd8000ce1c0 con 0x7fd8000cf910 2024-09-19T07:45:07.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.142+0000 7fd7fe59c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd80015bee0 0x7fd8000d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:07.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.142+0000 7fd7fed9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8000cf910 0x7fd8000d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:07.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.142+0000 7fd7fdd9b640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fd80015c2c0 0x7fd8000cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:37744/0 (socket says 172.21.15.16:37744) 2024-09-19T07:45:07.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.142+0000 7fd7fe59c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd80015bee0 0x7fd8000d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:40704/0 (socket says 172.21.15.16:40704) 2024-09-19T07:45:07.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.142+0000 7fd7fed9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8000cf910 0x7fd8000d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:50940/0 (socket says 172.21.15.16:50940) 2024-09-19T07:45:07.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.142+0000 7fd7fdd9b640 1 -- 172.21.15.16:0/3018817116 learned_addr learned my addr 172.21.15.16:0/3018817116 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:07.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.142+0000 7fd7fed9d640 1 -- 172.21.15.16:0/3018817116 >> v2:172.21.15.146:3300/0 conn(0x7fd80015bee0 msgr2=0x7fd8000d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:07.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.142+0000 7fd7fed9d640 1 --2- 172.21.15.16:0/3018817116 >> v2:172.21.15.146:3300/0 conn(0x7fd80015bee0 0x7fd8000d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.142+0000 7fd7fed9d640 1 -- 172.21.15.16:0/3018817116 >> v1:172.21.15.146:6789/0 conn(0x7fd80015c2c0 legacy=0x7fd8000cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:07.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.142+0000 7fd7fed9d640 1 -- 172.21.15.16:0/3018817116 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8000ce4a0 con 0x7fd8000cf910 2024-09-19T07:45:07.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.142+0000 7fd7fd59a640 1 -- 172.21.15.16:0/3018817116 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2072357673 0 0) 0x7fd8000cdee0 con 0x7fd80015c2c0 2024-09-19T07:45:07.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.142+0000 7fd7fe59c640 1 --2- 172.21.15.16:0/3018817116 >> v2:172.21.15.146:3300/0 conn(0x7fd80015bee0 0x7fd8000d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.143+0000 7fd7fed9d640 1 --2- 172.21.15.16:0/3018817116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8000cf910 0x7fd8000d09f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009d90 tx=0x7fd7f802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=824c73564920aa0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.143+0000 7fd7fd59a640 1 -- 172.21.15.16:0/3018817116 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8007740 con 0x7fd8000cf910 2024-09-19T07:45:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.143+0000 7fd7fd59a640 1 -- 172.21.15.16:0/3018817116 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f80078e0 con 0x7fd8000cf910 2024-09-19T07:45:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.143+0000 7fd7fd59a640 1 -- 172.21.15.16:0/3018817116 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8007cb0 con 0x7fd8000cf910 2024-09-19T07:45:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.144+0000 7fd804d8d640 1 -- 172.21.15.16:0/3018817116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8000cf910 msgr2=0x7fd8000d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.144+0000 7fd804d8d640 1 --2- 172.21.15.16:0/3018817116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8000cf910 0x7fd8000d09f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009d90 tx=0x7fd7f802a660 comp rx=0 tx=0).stop 2024-09-19T07:45:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.144+0000 7fd804d8d640 1 -- 172.21.15.16:0/3018817116 shutdown_connections 2024-09-19T07:45:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.144+0000 7fd804d8d640 1 --2- 172.21.15.16:0/3018817116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8000cf910 0x7fd8000d09f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.144+0000 7fd804d8d640 1 --2- 172.21.15.16:0/3018817116 >> v2:172.21.15.146:3300/0 conn(0x7fd80015bee0 0x7fd8000d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.144+0000 7fd804d8d640 1 -- 172.21.15.16:0/3018817116 >> 172.21.15.16:0/3018817116 conn(0x7fd80006bdf0 msgr2=0x7fd80006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:07.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.144+0000 7fd804d8d640 1 -- 172.21.15.16:0/3018817116 shutdown_connections 2024-09-19T07:45:07.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.144+0000 7fd804d8d640 1 -- 172.21.15.16:0/3018817116 wait complete. 2024-09-19T07:45:07.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.145+0000 7fd804d8d640 1 Processor -- start 2024-09-19T07:45:07.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.145+0000 7fd804d8d640 1 -- start start 2024-09-19T07:45:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.146+0000 7fd804d8d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8000cf910 0x7fd8001f1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.146+0000 7fd804d8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd80015bee0 0x7fd8000adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.146+0000 7fd804d8d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd80015c2c0 0x7fd8000cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.146+0000 7fd804d8d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd80006e040 con 0x7fd80015bee0 2024-09-19T07:45:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.146+0000 7fd804d8d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd80006dec0 con 0x7fd8000cf910 2024-09-19T07:45:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.146+0000 7fd804d8d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd80006e1c0 con 0x7fd80015c2c0 2024-09-19T07:45:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.146+0000 7fd7fdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd80015bee0 0x7fd8000adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.146+0000 7fd7fdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd80015bee0 0x7fd8000adeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57872/0 (socket says 172.21.15.16:57872) 2024-09-19T07:45:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.146+0000 7fd7fdd9b640 1 -- 172.21.15.16:0/2826614691 learned_addr learned my addr 172.21.15.16:0/2826614691 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.146+0000 7fd7fe59c640 1 --2- 172.21.15.16:0/2826614691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8000cf910 0x7fd8001f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.146+0000 7fd7fed9d640 1 --2- 172.21.15.16:0/2826614691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd80015c2c0 0x7fd8000cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.146+0000 7fd7fdd9b640 1 -- 172.21.15.16:0/2826614691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd80015c2c0 msgr2=0x7fd8000cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.146+0000 7fd7fdd9b640 1 --2- 172.21.15.16:0/2826614691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd80015c2c0 0x7fd8000cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.146+0000 7fd7fdd9b640 1 -- 172.21.15.16:0/2826614691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8000cf910 msgr2=0x7fd8001f1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.146+0000 7fd7fdd9b640 1 --2- 172.21.15.16:0/2826614691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8000cf910 0x7fd8001f1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.146+0000 7fd7fdd9b640 1 -- 172.21.15.16:0/2826614691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8000ae900 con 0x7fd80015bee0 2024-09-19T07:45:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.147+0000 7fd7fdd9b640 1 --2- 172.21.15.16:0/2826614691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd80015bee0 0x7fd8000adeb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400c500 tx=0x7fd7f400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.147+0000 7fd7eeffd640 1 -- 172.21.15.16:0/2826614691 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4018070 con 0x7fd80015bee0 2024-09-19T07:45:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.147+0000 7fd7eeffd640 1 -- 172.21.15.16:0/2826614691 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f4007580 con 0x7fd80015bee0 2024-09-19T07:45:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.147+0000 7fd804d8d640 1 -- 172.21.15.16:0/2826614691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8000aebe0 con 0x7fd80015bee0 2024-09-19T07:45:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.147+0000 7fd7eeffd640 1 -- 172.21.15.16:0/2826614691 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4004a20 con 0x7fd80015bee0 2024-09-19T07:45:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.147+0000 7fd804d8d640 1 -- 172.21.15.16:0/2826614691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8000af1a0 con 0x7fd80015bee0 2024-09-19T07:45:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.148+0000 7fd804d8d640 1 -- 172.21.15.16:0/2826614691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8000cf200 con 0x7fd80015bee0 2024-09-19T07:45:07.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.148+0000 7fd7eeffd640 1 -- 172.21.15.16:0/2826614691 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd7f401b070 con 0x7fd80015bee0 2024-09-19T07:45:07.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.149+0000 7fd7eeffd640 1 --2- 172.21.15.16:0/2826614691 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd7e006c2b0 0x7fd7e006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.149+0000 7fd7eeffd640 1 -- 172.21.15.16:0/2826614691 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5335+0+0 (secure 0 0 0) 0x7fd7f408a5a0 con 0x7fd80015bee0 2024-09-19T07:45:07.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.149+0000 7fd7fe59c640 1 --2- 172.21.15.16:0/2826614691 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd7e006c2b0 0x7fd7e006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:07.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.150+0000 7fd7fe59c640 1 --2- 172.21.15.16:0/2826614691 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd7e006c2b0 0x7fd7e006e770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009560 tx=0x7fd7f0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:07.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.153+0000 7fd7eeffd640 1 -- 172.21.15.16:0/2826614691 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd7f4058f10 con 0x7fd80015bee0 2024-09-19T07:45:07.230 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:45:07.232 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:45:07.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.342+0000 7fd804d8d640 1 -- 172.21.15.16:0/2826614691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd80015b130 con 0x7fd80015bee0 2024-09-19T07:45:07.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.432+0000 7fd7eeffd640 1 -- 172.21.15.16:0/2826614691 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v49) ==== 97+0+0 (secure 0 0 0) 0x7fd7f4058810 con 0x7fd80015bee0 2024-09-19T07:45:07.432 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-19T07:45:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.434+0000 7fd804d8d640 1 -- 172.21.15.16:0/2826614691 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd7e006c2b0 msgr2=0x7fd7e006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.434+0000 7fd804d8d640 1 --2- 172.21.15.16:0/2826614691 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd7e006c2b0 0x7fd7e006e770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009560 tx=0x7fd7f0008040 comp rx=0 tx=0).stop 2024-09-19T07:45:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.434+0000 7fd804d8d640 1 -- 172.21.15.16:0/2826614691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd80015bee0 msgr2=0x7fd8000adeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.434+0000 7fd804d8d640 1 --2- 172.21.15.16:0/2826614691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd80015bee0 0x7fd8000adeb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400c500 tx=0x7fd7f400c9c0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.434+0000 7fd804d8d640 1 -- 172.21.15.16:0/2826614691 shutdown_connections 2024-09-19T07:45:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.434+0000 7fd804d8d640 1 --2- 172.21.15.16:0/2826614691 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd7e006c2b0 0x7fd7e006e770 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.434+0000 7fd804d8d640 1 --2- 172.21.15.16:0/2826614691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd80015c2c0 0x7fd8000cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.434+0000 7fd804d8d640 1 --2- 172.21.15.16:0/2826614691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd80015bee0 0x7fd8000adeb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.434+0000 7fd804d8d640 1 --2- 172.21.15.16:0/2826614691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8000cf910 0x7fd8001f1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.434+0000 7fd804d8d640 1 -- 172.21.15.16:0/2826614691 >> 172.21.15.16:0/2826614691 conn(0x7fd80006bdf0 msgr2=0x7fd8000ca400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.434+0000 7fd804d8d640 1 -- 172.21.15.16:0/2826614691 shutdown_connections 2024-09-19T07:45:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.434+0000 7fd804d8d640 1 -- 172.21.15.16:0/2826614691 wait complete. 2024-09-19T07:45:07.446 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T07:45:07.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.548+0000 7f35da79f640 1 Processor -- start 2024-09-19T07:45:07.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.548+0000 7f35da79f640 1 -- start start 2024-09-19T07:45:07.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.549+0000 7f35da79f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f35d40b4b70 0x7f35d40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.549+0000 7f35da79f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35d40731d0 0x7f35d40b5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.549+0000 7f35da79f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d415b6d0 con 0x7f35d40b3e30 2024-09-19T07:45:07.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.549+0000 7f35da79f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f35d415b9b0 con 0x7f35d40b4b70 2024-09-19T07:45:07.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.549+0000 7f35da79f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f35d415bc90 con 0x7f35d40731d0 2024-09-19T07:45:07.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.550+0000 7f35d9f9e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35d40731d0 0x7f35d40b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:07.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.550+0000 7f35d979d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f35d40b4b70 0x7f35d40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:07.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.550+0000 7f35d8f9c640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f35d40b3e30 0x7f35d4072ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:37752/0 (socket says 172.21.15.16:37752) 2024-09-19T07:45:07.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.550+0000 7f35d9f9e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35d40731d0 0x7f35d40b5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:50972/0 (socket says 172.21.15.16:50972) 2024-09-19T07:45:07.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.550+0000 7f35d8f9c640 1 -- 172.21.15.16:0/2222231117 learned_addr learned my addr 172.21.15.16:0/2222231117 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:07.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.550+0000 7f35d979d640 1 -- 172.21.15.16:0/2222231117 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35d40731d0 msgr2=0x7f35d40b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:07.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.550+0000 7f35d979d640 1 --2- 172.21.15.16:0/2222231117 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35d40731d0 0x7f35d40b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.550+0000 7f35d979d640 1 -- 172.21.15.16:0/2222231117 >> v1:172.21.15.146:6789/0 conn(0x7f35d40b3e30 legacy=0x7f35d4072ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:07.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.550+0000 7f35d979d640 1 -- 172.21.15.16:0/2222231117 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d415bf70 con 0x7f35d40b4b70 2024-09-19T07:45:07.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.550+0000 7f35c7fff640 1 -- 172.21.15.16:0/2222231117 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3508821157 0 0) 0x7f35d415b6d0 con 0x7f35d40b3e30 2024-09-19T07:45:07.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.550+0000 7f35d9f9e640 1 --2- 172.21.15.16:0/2222231117 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35d40731d0 0x7f35d40b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:07.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.551+0000 7f35d979d640 1 --2- 172.21.15.16:0/2222231117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35d40b4b70 0x7f35d40b5050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f35d00097a0 tx=0x7f35d002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4541fdec4a0477db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:07.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.551+0000 7f35c7fff640 1 -- 172.21.15.16:0/2222231117 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0037070 con 0x7f35d40b4b70 2024-09-19T07:45:07.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.551+0000 7f35c7fff640 1 -- 172.21.15.16:0/2222231117 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d002ae50 con 0x7f35d40b4b70 2024-09-19T07:45:07.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.551+0000 7f35c7fff640 1 -- 172.21.15.16:0/2222231117 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0032660 con 0x7f35d40b4b70 2024-09-19T07:45:07.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.551+0000 7f35da79f640 1 -- 172.21.15.16:0/2222231117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35d40b4b70 msgr2=0x7f35d40b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:07.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.551+0000 7f35da79f640 1 --2- 172.21.15.16:0/2222231117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35d40b4b70 0x7f35d40b5050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f35d00097a0 tx=0x7f35d002a100 comp rx=0 tx=0).stop 2024-09-19T07:45:07.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.551+0000 7f35da79f640 1 -- 172.21.15.16:0/2222231117 shutdown_connections 2024-09-19T07:45:07.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.551+0000 7f35da79f640 1 --2- 172.21.15.16:0/2222231117 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35d40731d0 0x7f35d40b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.551+0000 7f35da79f640 1 --2- 172.21.15.16:0/2222231117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35d40b4b70 0x7f35d40b5050 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.551+0000 7f35da79f640 1 -- 172.21.15.16:0/2222231117 >> 172.21.15.16:0/2222231117 conn(0x7f35d40d0ec0 msgr2=0x7f35d40d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:07.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.552+0000 7f35da79f640 1 -- 172.21.15.16:0/2222231117 shutdown_connections 2024-09-19T07:45:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.552+0000 7f35da79f640 1 -- 172.21.15.16:0/2222231117 wait complete. 2024-09-19T07:45:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.553+0000 7f35da79f640 1 Processor -- start 2024-09-19T07:45:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.553+0000 7f35da79f640 1 -- start start 2024-09-19T07:45:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.553+0000 7f35da79f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d40731d0 0x7f35d41673b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.553+0000 7f35da79f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35d40b3e30 0x7f35d414a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.553+0000 7f35da79f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35d40b4b70 0x7f35d4156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.553+0000 7f35da79f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35d415ed10 con 0x7f35d40731d0 2024-09-19T07:45:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.553+0000 7f35da79f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f35d415eb90 con 0x7f35d40b3e30 2024-09-19T07:45:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.553+0000 7f35da79f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f35d415ee90 con 0x7f35d40b4b70 2024-09-19T07:45:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.554+0000 7f35d979d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d40731d0 0x7f35d41673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.554+0000 7f35d979d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d40731d0 0x7f35d41673b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57874/0 (socket says 172.21.15.16:57874) 2024-09-19T07:45:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.554+0000 7f35d979d640 1 -- 172.21.15.16:0/3428587267 learned_addr learned my addr 172.21.15.16:0/3428587267 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.554+0000 7f35d9f9e640 1 --2- 172.21.15.16:0/3428587267 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35d40b4b70 0x7f35d4156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.554+0000 7f35d8f9c640 1 --2- 172.21.15.16:0/3428587267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35d40b3e30 0x7f35d414a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.554+0000 7f35d979d640 1 -- 172.21.15.16:0/3428587267 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35d40b4b70 msgr2=0x7f35d4156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.554+0000 7f35d979d640 1 --2- 172.21.15.16:0/3428587267 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35d40b4b70 0x7f35d4156700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.554+0000 7f35d979d640 1 -- 172.21.15.16:0/3428587267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35d40b3e30 msgr2=0x7f35d414a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.554+0000 7f35d979d640 1 --2- 172.21.15.16:0/3428587267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35d40b3e30 0x7f35d414a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.554+0000 7f35d979d640 1 -- 172.21.15.16:0/3428587267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d414ab80 con 0x7f35d40731d0 2024-09-19T07:45:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.554+0000 7f35d979d640 1 --2- 172.21.15.16:0/3428587267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d40731d0 0x7f35d41673b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f35d00098c0 tx=0x7f35d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.554+0000 7f35c5ffb640 1 -- 172.21.15.16:0/3428587267 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0041030 con 0x7f35d40731d0 2024-09-19T07:45:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.554+0000 7f35c5ffb640 1 -- 172.21.15.16:0/3428587267 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d002ac50 con 0x7f35d40731d0 2024-09-19T07:45:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.554+0000 7f35da79f640 1 -- 172.21.15.16:0/3428587267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d414c7a0 con 0x7f35d40731d0 2024-09-19T07:45:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.554+0000 7f35c5ffb640 1 -- 172.21.15.16:0/3428587267 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0037040 con 0x7f35d40731d0 2024-09-19T07:45:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.555+0000 7f35da79f640 1 -- 172.21.15.16:0/3428587267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d41668d0 con 0x7f35d40731d0 2024-09-19T07:45:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.555+0000 7f35da79f640 1 -- 172.21.15.16:0/3428587267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d4072ac0 con 0x7f35d40731d0 2024-09-19T07:45:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.556+0000 7f35c5ffb640 1 -- 172.21.15.16:0/3428587267 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f35d0044070 con 0x7f35d40731d0 2024-09-19T07:45:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.556+0000 7f35c5ffb640 1 --2- 172.21.15.16:0/3428587267 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f35b806c2b0 0x7f35b806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.556+0000 7f35c5ffb640 1 -- 172.21.15.16:0/3428587267 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5249+0+0 (secure 0 0 0) 0x7f35d00ab720 con 0x7f35d40731d0 2024-09-19T07:45:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.557+0000 7f35d8f9c640 1 --2- 172.21.15.16:0/3428587267 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f35b806c2b0 0x7f35b806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.557+0000 7f35d8f9c640 1 --2- 172.21.15.16:0/3428587267 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f35b806c2b0 0x7f35b806e770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f35d4167980 tx=0x7f35c8007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:07.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.561+0000 7f35c5ffb640 1 -- 172.21.15.16:0/3428587267 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f35d0079fc0 con 0x7f35d40731d0 2024-09-19T07:45:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.805+0000 7f35da79f640 1 -- 172.21.15.16:0/3428587267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f35d40aee80 con 0x7f35d40731d0 2024-09-19T07:45:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.806+0000 7f35c5ffb640 1 -- 172.21.15.16:0/3428587267 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f35d00798c0 con 0x7f35d40731d0 2024-09-19T07:45:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:45:07.807 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:45:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-19T07:45:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-19T07:45:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etms80KDxAAqTrG88LTEJIj9TWJu0witQ==", 2024-09-19T07:45:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:45:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:45:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:45:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:45:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:45:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-19T07:45:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etm4T0KGxAAcfDOKrSdKzhJ4xvTFIyxcQ==", 2024-09-19T07:45:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:45:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:45:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:45:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:45:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:45:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-19T07:45:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNDBGFhAAtBioJJ1+NtIDLhd/Hg7aZA==", 2024-09-19T07:45:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:45:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:45:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:45:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:45:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:45:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-19T07:45:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmaHEiIhAACqqjCRMV+xlGh9WqYsj+MA==", 2024-09-19T07:45:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:45:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:45:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:45:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:45:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:45:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-19T07:45:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDT1etm09plCxAAGvJYF6JrKw49FtXb71ox3A==", 2024-09-19T07:45:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:45:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:45:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:45:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:45:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-19T07:45:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDW1etmGy8yHRAAlcDsHLnkjtS6sCCR8Olv2A==", 2024-09-19T07:45:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:45:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:45:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:45:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:45:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-19T07:45:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDZ1etmMhGTHxAATQaW2kgJhHKftutQQtUa6A==", 2024-09-19T07:45:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:45:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:45:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:45:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:45:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-19T07:45:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDc1etmD3sLIhAAIdY00buvb3bIFA5ZpgxxjA==", 2024-09-19T07:45:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:45:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:45:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:45:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:45:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-19T07:45:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDi1etmgutcLRAAHJEl5SfQ7N0T4YoYeBfrQg==", 2024-09-19T07:45:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:45:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:45:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:45:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:45:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-19T07:45:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDm1etmCV3xARAA+5yhQEmIpyNId3RL1Xsxhw==", 2024-09-19T07:45:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:45:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:45:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:45:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:45:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-19T07:45:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDp1etm9XC+AxAA+Lrs0ExfSKaixgOxfX8Nug==", 2024-09-19T07:45:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:45:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:45:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:45:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:45:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-19T07:45:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDs1etmhmvMBhAAhvW/udIE2gCd8IrJCSFuvw==", 2024-09-19T07:45:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:45:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:45:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:45:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:45:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-19T07:45:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA==", 2024-09-19T07:45:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:45:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:45:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-19T07:45:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T07:45:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-19T07:45:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmc05KLxAAPQl6wOHqFIf8LVxskiuLVg==", 2024-09-19T07:45:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:45:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:45:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:45:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:45:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:45:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-19T07:45:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmjFsCLBAANa7gCMvDw84Ovo7u6RRJ3Q==", 2024-09-19T07:45:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:45:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:45:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:45:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:45:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:45:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-19T07:45:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmdUbgNBAAfwstyOeeoZfetgVZu4z5xg==", 2024-09-19T07:45:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:45:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:45:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:45:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:45:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:45:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-19T07:45:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDQ1etm9/vTLhAA9PRfAOA8R4oXp3x0dU8l3g==", 2024-09-19T07:45:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:45:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:45:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:45:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:45:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:45:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-19T07:45:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNQ68CxAANkK+D3u/oTEz8Su4iuZrCg==", 2024-09-19T07:45:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:45:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:45:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:45:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:45:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-19T07:45:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmp+gSCBAAGXzmKfFNd6K2fWeqYhlHAg==", 2024-09-19T07:45:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:45:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:45:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:45:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:45:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T07:45:07.821 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:45:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.808+0000 7f35da79f640 1 -- 172.21.15.16:0/3428587267 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f35b806c2b0 msgr2=0x7f35b806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.808+0000 7f35da79f640 1 --2- 172.21.15.16:0/3428587267 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f35b806c2b0 0x7f35b806e770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f35d4167980 tx=0x7f35c8007d50 comp rx=0 tx=0).stop 2024-09-19T07:45:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.808+0000 7f35da79f640 1 -- 172.21.15.16:0/3428587267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d40731d0 msgr2=0x7f35d41673b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.808+0000 7f35da79f640 1 --2- 172.21.15.16:0/3428587267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d40731d0 0x7f35d41673b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f35d00098c0 tx=0x7f35d00046c0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.808+0000 7f35da79f640 1 -- 172.21.15.16:0/3428587267 shutdown_connections 2024-09-19T07:45:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.808+0000 7f35da79f640 1 --2- 172.21.15.16:0/3428587267 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f35b806c2b0 0x7f35b806e770 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.808+0000 7f35da79f640 1 --2- 172.21.15.16:0/3428587267 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35d40b4b70 0x7f35d4156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.808+0000 7f35da79f640 1 --2- 172.21.15.16:0/3428587267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35d40b3e30 0x7f35d414a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.808+0000 7f35da79f640 1 --2- 172.21.15.16:0/3428587267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d40731d0 0x7f35d41673b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.808+0000 7f35da79f640 1 -- 172.21.15.16:0/3428587267 >> 172.21.15.16:0/3428587267 conn(0x7f35d40d0ec0 msgr2=0x7f35d415ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.808+0000 7f35da79f640 1 -- 172.21.15.16:0/3428587267 shutdown_connections 2024-09-19T07:45:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.808+0000 7f35da79f640 1 -- 172.21.15.16:0/3428587267 wait complete. 2024-09-19T07:45:07.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T07:45:07.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.926+0000 7f7042bf6640 1 Processor -- start 2024-09-19T07:45:07.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.926+0000 7f7042bf6640 1 -- start start 2024-09-19T07:45:07.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.927+0000 7f7042bf6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f703c15c2c0 0x7f703c0cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.927+0000 7f7042bf6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f703c0cdee0 con 0x7f703c0d0bc0 2024-09-19T07:45:07.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.927+0000 7f7042bf6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f703c0ce1c0 con 0x7f703c15bee0 2024-09-19T07:45:07.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.927+0000 7f7042bf6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f703c0ceaf0 con 0x7f703c15c2c0 2024-09-19T07:45:07.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.927+0000 7f703bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f703c0d0bc0 0x7f703c0cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58178/0 (socket says 172.21.15.16:58178) 2024-09-19T07:45:07.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.927+0000 7f703bfff640 1 -- 172.21.15.16:0/2845336990 learned_addr learned my addr 172.21.15.16:0/2845336990 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:07.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.927+0000 7f703b7fe640 1 -- 172.21.15.16:0/2845336990 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1884705708 0 0) 0x7f703c0cdee0 con 0x7f703c0d0bc0 2024-09-19T07:45:07.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.927+0000 7f703b7fe640 1 -- 172.21.15.16:0/2845336990 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f702c003520 con 0x7f703c0d0bc0 2024-09-19T07:45:07.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.927+0000 7f7041193640 1 --2- 172.21.15.16:0/2845336990 >> v2:172.21.15.146:3300/0 conn(0x7f703c15c2c0 0x7f703c0cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:07.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.927+0000 7f703b7fe640 1 -- 172.21.15.16:0/2845336990 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2761688320 0 0) 0x7f702c003520 con 0x7f703c0d0bc0 2024-09-19T07:45:07.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.927+0000 7f703b7fe640 1 -- 172.21.15.16:0/2845336990 >> v2:172.21.15.146:3300/0 conn(0x7f703c15c2c0 msgr2=0x7f703c0cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:07.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.927+0000 7f703b7fe640 1 --2- 172.21.15.16:0/2845336990 >> v2:172.21.15.146:3300/0 conn(0x7f703c15c2c0 0x7f703c0cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.927+0000 7f703b7fe640 1 -- 172.21.15.16:0/2845336990 >> v1:172.21.15.146:6789/0 conn(0x7f703c15bee0 legacy=0x7f703c0d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:07.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.927+0000 7f703b7fe640 1 -- 172.21.15.16:0/2845336990 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c0ce4a0 con 0x7f703c0d0bc0 2024-09-19T07:45:07.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.927+0000 7f703b7fe640 1 -- 172.21.15.16:0/2845336990 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f7028001670 con 0x7f703c0d0bc0 2024-09-19T07:45:07.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.927+0000 7f703b7fe640 1 -- 172.21.15.16:0/2845336990 >> v1:172.21.15.16:6789/0 conn(0x7f703c0d0bc0 legacy=0x7f703c0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:07.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.927+0000 7f703b7fe640 1 --2- 172.21.15.16:0/2845336990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702c0039b0 0x7f702c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.927+0000 7f703b7fe640 1 --2- 172.21.15.16:0/2845336990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f702c004800 0x7f702c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.928+0000 7f703b7fe640 1 --2- 172.21.15.16:0/2845336990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f702c0051c0 0x7f702c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.928+0000 7f703b7fe640 1 -- 172.21.15.16:0/2845336990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f703c0ceaf0 con 0x7f702c0039b0 2024-09-19T07:45:07.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.928+0000 7f703b7fe640 1 -- 172.21.15.16:0/2845336990 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f702c00c060 con 0x7f702c004800 2024-09-19T07:45:07.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.928+0000 7f703b7fe640 1 -- 172.21.15.16:0/2845336990 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f702c00c260 con 0x7f702c0051c0 2024-09-19T07:45:07.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.928+0000 7f7040992640 1 --2- 172.21.15.16:0/2845336990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702c0039b0 0x7f702c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:07.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.928+0000 7f703bfff640 1 --2- 172.21.15.16:0/2845336990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f702c004800 0x7f702c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:07.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.928+0000 7f7041193640 1 --2- 172.21.15.16:0/2845336990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f702c0051c0 0x7f702c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:07.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.928+0000 7f7040992640 1 -- 172.21.15.16:0/2845336990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f702c0051c0 msgr2=0x7f702c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:07.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.928+0000 7f7040992640 1 --2- 172.21.15.16:0/2845336990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f702c0051c0 0x7f702c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.928+0000 7f7040992640 1 -- 172.21.15.16:0/2845336990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f702c004800 msgr2=0x7f702c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:07.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.928+0000 7f7040992640 1 --2- 172.21.15.16:0/2845336990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f702c004800 0x7f702c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.928+0000 7f7040992640 1 -- 172.21.15.16:0/2845336990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702c003520 con 0x7f702c0039b0 2024-09-19T07:45:07.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.928+0000 7f7041193640 1 -- 172.21.15.16:0/2845336990 reap_dead start 2024-09-19T07:45:07.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.929+0000 7f7040992640 1 --2- 172.21.15.16:0/2845336990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702c0039b0 0x7f702c003dd0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7030001750 tx=0x7f702c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223fcbf18bce2ebd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:07.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.929+0000 7f703b7fe640 1 -- 172.21.15.16:0/2845336990 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030001370 con 0x7f702c0039b0 2024-09-19T07:45:07.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.929+0000 7f703b7fe640 1 -- 172.21.15.16:0/2845336990 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70300052d0 con 0x7f702c0039b0 2024-09-19T07:45:07.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.929+0000 7f703b7fe640 1 -- 172.21.15.16:0/2845336990 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703002abc0 con 0x7f702c0039b0 2024-09-19T07:45:07.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.929+0000 7f7042bf6640 1 -- 172.21.15.16:0/2845336990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702c0039b0 msgr2=0x7f702c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:07.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.929+0000 7f7042bf6640 1 --2- 172.21.15.16:0/2845336990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702c0039b0 0x7f702c003dd0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7030001750 tx=0x7f702c005830 comp rx=0 tx=0).stop 2024-09-19T07:45:07.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.929+0000 7f7042bf6640 1 -- 172.21.15.16:0/2845336990 shutdown_connections 2024-09-19T07:45:07.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.929+0000 7f7042bf6640 1 --2- 172.21.15.16:0/2845336990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702c0039b0 0x7f702c003dd0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.929+0000 7f7042bf6640 1 -- 172.21.15.16:0/2845336990 >> 172.21.15.16:0/2845336990 conn(0x7f703c06bdf0 msgr2=0x7f703c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:07.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.929+0000 7f7042bf6640 1 -- 172.21.15.16:0/2845336990 shutdown_connections 2024-09-19T07:45:07.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.929+0000 7f7042bf6640 1 -- 172.21.15.16:0/2845336990 wait complete. 2024-09-19T07:45:07.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.931+0000 7f7042bf6640 1 Processor -- start 2024-09-19T07:45:07.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.931+0000 7f7042bf6640 1 -- start start 2024-09-19T07:45:07.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f7042bf6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702c0039b0 0x7f703c0c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f7042bf6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f703c073cc0 0x7f703c15cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f7042bf6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f703c0ca090 0x7f703c15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f7042bf6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f703c0d0630 con 0x7f702c0039b0 2024-09-19T07:45:07.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f7042bf6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f703c0d04b0 con 0x7f703c073cc0 2024-09-19T07:45:07.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f7042bf6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f703c0d07b0 con 0x7f703c0ca090 2024-09-19T07:45:07.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f7040992640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702c0039b0 0x7f703c0c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:07.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f7040992640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702c0039b0 0x7f703c0c7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57890/0 (socket says 172.21.15.16:57890) 2024-09-19T07:45:07.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f7040992640 1 -- 172.21.15.16:0/3754480458 learned_addr learned my addr 172.21.15.16:0/3754480458 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:07.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f703bfff640 1 --2- 172.21.15.16:0/3754480458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f703c073cc0 0x7f703c15cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:07.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f7040992640 1 -- 172.21.15.16:0/3754480458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f703c0ca090 msgr2=0x7f703c15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:07.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f7040992640 1 --2- 172.21.15.16:0/3754480458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f703c0ca090 0x7f703c15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f7040992640 1 -- 172.21.15.16:0/3754480458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f703c073cc0 msgr2=0x7f703c15cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:07.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f7040992640 1 --2- 172.21.15.16:0/3754480458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f703c073cc0 0x7f703c15cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:07.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f7040992640 1 -- 172.21.15.16:0/3754480458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c071e90 con 0x7f702c0039b0 2024-09-19T07:45:07.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f7040992640 1 --2- 172.21.15.16:0/3754480458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702c0039b0 0x7f703c0c7340 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7030001870 tx=0x7f7030001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:07.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f70397fa640 1 -- 172.21.15.16:0/3754480458 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703003f020 con 0x7f702c0039b0 2024-09-19T07:45:07.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f70397fa640 1 -- 172.21.15.16:0/3754480458 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f703002be50 con 0x7f702c0039b0 2024-09-19T07:45:07.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f7042bf6640 1 -- 172.21.15.16:0/3754480458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703c072170 con 0x7f702c0039b0 2024-09-19T07:45:07.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f70397fa640 1 -- 172.21.15.16:0/3754480458 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030001370 con 0x7f702c0039b0 2024-09-19T07:45:07.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.932+0000 7f7042bf6640 1 -- 172.21.15.16:0/3754480458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f703c072730 con 0x7f702c0039b0 2024-09-19T07:45:07.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.933+0000 7f7042bf6640 1 -- 172.21.15.16:0/3754480458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f703c0ca560 con 0x7f702c0039b0 2024-09-19T07:45:07.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.934+0000 7f70397fa640 1 -- 172.21.15.16:0/3754480458 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7030042020 con 0x7f702c0039b0 2024-09-19T07:45:07.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.934+0000 7f70397fa640 1 --2- 172.21.15.16:0/3754480458 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f702006c2e0 0x7f702006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:07.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.935+0000 7f70397fa640 1 -- 172.21.15.16:0/3754480458 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5249+0+0 (secure 0 0 0) 0x7f70300aae60 con 0x7f702c0039b0 2024-09-19T07:45:07.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.935+0000 7f703bfff640 1 --2- 172.21.15.16:0/3754480458 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f702006c2e0 0x7f702006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:07.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.935+0000 7f703bfff640 1 --2- 172.21.15.16:0/3754480458 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f702006c2e0 0x7f702006e7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7028001050 tx=0x7f7028009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:07.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:07.939+0000 7f70397fa640 1 -- 172.21.15.16:0/3754480458 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7030049030 con 0x7f702c0039b0 2024-09-19T07:45:08.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.117+0000 7f7042bf6640 1 -- 172.21.15.16:0/3754480458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f703c06df40 con 0x7f702c0039b0 2024-09-19T07:45:08.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.117+0000 7f70397fa640 1 -- 172.21.15.16:0/3754480458 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v49) ==== 84+0+65 (secure 0 0 0) 0x7f70300796f0 con 0x7f702c0039b0 2024-09-19T07:45:08.118 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:45:08.118 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-19T07:45:08.118 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:45:08.118 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-19T07:45:08.118 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-19T07:45:08.118 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:08.118 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-19T07:45:08.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.119+0000 7f7042bf6640 1 -- 172.21.15.16:0/3754480458 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f702006c2e0 msgr2=0x7f702006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:08.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.119+0000 7f7042bf6640 1 --2- 172.21.15.16:0/3754480458 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f702006c2e0 0x7f702006e7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7028001050 tx=0x7f7028009040 comp rx=0 tx=0).stop 2024-09-19T07:45:08.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.119+0000 7f7042bf6640 1 -- 172.21.15.16:0/3754480458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702c0039b0 msgr2=0x7f703c0c7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:08.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.119+0000 7f7042bf6640 1 --2- 172.21.15.16:0/3754480458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702c0039b0 0x7f703c0c7340 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7030001870 tx=0x7f7030001120 comp rx=0 tx=0).stop 2024-09-19T07:45:08.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.119+0000 7f7042bf6640 1 -- 172.21.15.16:0/3754480458 shutdown_connections 2024-09-19T07:45:08.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.119+0000 7f7042bf6640 1 --2- 172.21.15.16:0/3754480458 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f702006c2e0 0x7f702006e7a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:08.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.119+0000 7f7042bf6640 1 --2- 172.21.15.16:0/3754480458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f703c0ca090 0x7f703c15ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:08.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.119+0000 7f7042bf6640 1 --2- 172.21.15.16:0/3754480458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f703c073cc0 0x7f703c15cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:08.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.119+0000 7f7042bf6640 1 --2- 172.21.15.16:0/3754480458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702c0039b0 0x7f703c0c7340 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:08.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.119+0000 7f7042bf6640 1 -- 172.21.15.16:0/3754480458 >> 172.21.15.16:0/3754480458 conn(0x7f703c06bdf0 msgr2=0x7f703c0afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:08.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.120+0000 7f7042bf6640 1 -- 172.21.15.16:0/3754480458 shutdown_connections 2024-09-19T07:45:08.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.120+0000 7f7042bf6640 1 -- 172.21.15.16:0/3754480458 wait complete. 2024-09-19T07:45:08.131 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T07:45:08.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T07:45:08.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.236+0000 7fb8cff11640 1 Processor -- start 2024-09-19T07:45:08.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.237+0000 7fb8cff11640 1 -- start start 2024-09-19T07:45:08.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.237+0000 7fb8cff11640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb8c815f0d0 0x7fb8c815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:08.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.237+0000 7fb8cff11640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8c80d0070 0x7fb8c80d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:08.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.238+0000 7fb8cff11640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8c80cf2b0 con 0x7fb8c80d1940 2024-09-19T07:45:08.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.238+0000 7fb8cff11640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c80cf590 con 0x7fb8c80d0070 2024-09-19T07:45:08.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.238+0000 7fb8cff11640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c80cf870 con 0x7fb8c815f0d0 2024-09-19T07:45:08.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.238+0000 7fb8cd4ac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8c80d0070 0x7fb8c80d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:08.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.238+0000 7fb8ce4ae640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb8c80d1940 0x7fb8c80d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58182/0 (socket says 172.21.15.16:58182) 2024-09-19T07:45:08.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.238+0000 7fb8ce4ae640 1 -- 172.21.15.16:0/1003556423 learned_addr learned my addr 172.21.15.16:0/1003556423 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:08.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.238+0000 7fb8cd4ac640 1 -- 172.21.15.16:0/1003556423 >> v2:172.21.15.146:3300/0 conn(0x7fb8c815f0d0 msgr2=0x7fb8c815f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:08.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.238+0000 7fb8cd4ac640 1 --2- 172.21.15.16:0/1003556423 >> v2:172.21.15.146:3300/0 conn(0x7fb8c815f0d0 0x7fb8c815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:08.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.238+0000 7fb8cd4ac640 1 -- 172.21.15.16:0/1003556423 >> v1:172.21.15.16:6789/0 conn(0x7fb8c80d1940 legacy=0x7fb8c80d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:08.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.238+0000 7fb8cd4ac640 1 -- 172.21.15.16:0/1003556423 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c80cfb50 con 0x7fb8c80d0070 2024-09-19T07:45:08.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.238+0000 7fb8cccab640 1 -- 172.21.15.16:0/1003556423 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1267893242 0 0) 0x7fb8c80cf2b0 con 0x7fb8c80d1940 2024-09-19T07:45:08.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.239+0000 7fb8cd4ac640 1 --2- 172.21.15.16:0/1003556423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c80d0070 0x7fb8c80d1400 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80b6f40 tx=0x7fb8bc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b04e6a951af455 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:08.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.239+0000 7fb8cccab640 1 -- 172.21.15.16:0/1003556423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc037070 con 0x7fb8c80d0070 2024-09-19T07:45:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.239+0000 7fb8cccab640 1 -- 172.21.15.16:0/1003556423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8bc032440 con 0x7fb8c80d0070 2024-09-19T07:45:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.239+0000 7fb8cccab640 1 -- 172.21.15.16:0/1003556423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc02ae60 con 0x7fb8c80d0070 2024-09-19T07:45:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.239+0000 7fb8cff11640 1 -- 172.21.15.16:0/1003556423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c80d0070 msgr2=0x7fb8c80d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.239+0000 7fb8cff11640 1 --2- 172.21.15.16:0/1003556423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c80d0070 0x7fb8c80d1400 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80b6f40 tx=0x7fb8bc02a110 comp rx=0 tx=0).stop 2024-09-19T07:45:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.239+0000 7fb8cff11640 1 -- 172.21.15.16:0/1003556423 shutdown_connections 2024-09-19T07:45:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.239+0000 7fb8cff11640 1 --2- 172.21.15.16:0/1003556423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c80d0070 0x7fb8c80d1400 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.239+0000 7fb8cff11640 1 --2- 172.21.15.16:0/1003556423 >> v2:172.21.15.146:3300/0 conn(0x7fb8c815f0d0 0x7fb8c815f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.239+0000 7fb8cff11640 1 -- 172.21.15.16:0/1003556423 >> 172.21.15.16:0/1003556423 conn(0x7fb8c809d760 msgr2=0x7fb8c808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.239+0000 7fb8cff11640 1 -- 172.21.15.16:0/1003556423 shutdown_connections 2024-09-19T07:45:08.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.240+0000 7fb8cff11640 1 -- 172.21.15.16:0/1003556423 wait complete. 2024-09-19T07:45:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.241+0000 7fb8cff11640 1 Processor -- start 2024-09-19T07:45:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.241+0000 7fb8cff11640 1 -- start start 2024-09-19T07:45:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.241+0000 7fb8cff11640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8c80d0070 0x7fb8c81f4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.241+0000 7fb8cff11640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8c80d1940 0x7fb8c815b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.241+0000 7fb8cff11640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c815f0d0 0x7fb8c815d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.241+0000 7fb8cff11640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c80d49c0 con 0x7fb8c815f0d0 2024-09-19T07:45:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.241+0000 7fb8cff11640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c80d4840 con 0x7fb8c80d1940 2024-09-19T07:45:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.241+0000 7fb8cff11640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c80d4b40 con 0x7fb8c80d0070 2024-09-19T07:45:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.242+0000 7fb8ce4ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c815f0d0 0x7fb8c815d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.242+0000 7fb8ce4ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c815f0d0 0x7fb8c815d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57906/0 (socket says 172.21.15.16:57906) 2024-09-19T07:45:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.242+0000 7fb8ce4ae640 1 -- 172.21.15.16:0/3310050027 learned_addr learned my addr 172.21.15.16:0/3310050027 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.242+0000 7fb8cdcad640 1 --2- 172.21.15.16:0/3310050027 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8c80d0070 0x7fb8c81f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.242+0000 7fb8cd4ac640 1 --2- 172.21.15.16:0/3310050027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8c80d1940 0x7fb8c815b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.242+0000 7fb8ce4ae640 1 -- 172.21.15.16:0/3310050027 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8c80d0070 msgr2=0x7fb8c81f4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.242+0000 7fb8ce4ae640 1 --2- 172.21.15.16:0/3310050027 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8c80d0070 0x7fb8c81f4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:08.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.242+0000 7fb8ce4ae640 1 -- 172.21.15.16:0/3310050027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8c80d1940 msgr2=0x7fb8c815b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:08.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.242+0000 7fb8ce4ae640 1 --2- 172.21.15.16:0/3310050027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8c80d1940 0x7fb8c815b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:08.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.242+0000 7fb8ce4ae640 1 -- 172.21.15.16:0/3310050027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c80b7210 con 0x7fb8c815f0d0 2024-09-19T07:45:08.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.242+0000 7fb8cd4ac640 1 --2- 172.21.15.16:0/3310050027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8c80d1940 0x7fb8c815b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:08.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.242+0000 7fb8ce4ae640 1 --2- 172.21.15.16:0/3310050027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c815f0d0 0x7fb8c815d5a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000c450 tx=0x7fb8c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:08.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.242+0000 7fb8c67fc640 1 -- 172.21.15.16:0/3310050027 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0017070 con 0x7fb8c815f0d0 2024-09-19T07:45:08.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.242+0000 7fb8c67fc640 1 -- 172.21.15.16:0/3310050027 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c0005d00 con 0x7fb8c815f0d0 2024-09-19T07:45:08.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.242+0000 7fb8cff11640 1 -- 172.21.15.16:0/3310050027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c80b74f0 con 0x7fb8c815f0d0 2024-09-19T07:45:08.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.242+0000 7fb8c67fc640 1 -- 172.21.15.16:0/3310050027 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0007310 con 0x7fb8c815f0d0 2024-09-19T07:45:08.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.242+0000 7fb8cff11640 1 -- 172.21.15.16:0/3310050027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c80b7ab0 con 0x7fb8c815f0d0 2024-09-19T07:45:08.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.243+0000 7fb8cff11640 1 -- 172.21.15.16:0/3310050027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c80d1400 con 0x7fb8c815f0d0 2024-09-19T07:45:08.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.244+0000 7fb8c67fc640 1 -- 172.21.15.16:0/3310050027 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb8c001a070 con 0x7fb8c815f0d0 2024-09-19T07:45:08.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.244+0000 7fb8c67fc640 1 --2- 172.21.15.16:0/3310050027 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb8b406c2b0 0x7fb8b406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:08.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.244+0000 7fb8c67fc640 1 -- 172.21.15.16:0/3310050027 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5249+0+0 (secure 0 0 0) 0x7fb8c008a5f0 con 0x7fb8c815f0d0 2024-09-19T07:45:08.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.245+0000 7fb8cdcad640 1 --2- 172.21.15.16:0/3310050027 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb8b406c2b0 0x7fb8b406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:08.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.245+0000 7fb8cdcad640 1 --2- 172.21.15.16:0/3310050027 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb8b406c2b0 0x7fb8b406e770 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb8c815e1b0 tx=0x7fb8b80094f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:08.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.249+0000 7fb8c67fc640 1 -- 172.21.15.16:0/3310050027 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb8c0058fb0 con 0x7fb8c815f0d0 2024-09-19T07:45:08.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:08.424+0000 7fb8cff11640 1 -- 172.21.15.16:0/3310050027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb8c806d2f0 con 0x7fb8c815f0d0 2024-09-19T07:45:09.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.120+0000 7fb8c67fc640 1 -- 172.21.15.16:0/3310050027 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v50) ==== 138+0+0 (secure 0 0 0) 0x7fb8c00588b0 con 0x7fb8c815f0d0 2024-09-19T07:45:09.121 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-19T07:45:09.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.123+0000 7fb8cff11640 1 -- 172.21.15.16:0/3310050027 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb8b406c2b0 msgr2=0x7fb8b406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.123+0000 7fb8cff11640 1 --2- 172.21.15.16:0/3310050027 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb8b406c2b0 0x7fb8b406e770 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb8c815e1b0 tx=0x7fb8b80094f0 comp rx=0 tx=0).stop 2024-09-19T07:45:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.123+0000 7fb8cff11640 1 -- 172.21.15.16:0/3310050027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c815f0d0 msgr2=0x7fb8c815d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.123+0000 7fb8cff11640 1 --2- 172.21.15.16:0/3310050027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c815f0d0 0x7fb8c815d5a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000c450 tx=0x7fb8c000c910 comp rx=0 tx=0).stop 2024-09-19T07:45:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.123+0000 7fb8cff11640 1 -- 172.21.15.16:0/3310050027 shutdown_connections 2024-09-19T07:45:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.123+0000 7fb8cff11640 1 --2- 172.21.15.16:0/3310050027 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb8b406c2b0 0x7fb8b406e770 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.123+0000 7fb8cff11640 1 --2- 172.21.15.16:0/3310050027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c815f0d0 0x7fb8c815d5a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.123+0000 7fb8cff11640 1 --2- 172.21.15.16:0/3310050027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8c80d1940 0x7fb8c815b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.123+0000 7fb8cff11640 1 --2- 172.21.15.16:0/3310050027 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8c80d0070 0x7fb8c81f4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.123+0000 7fb8cff11640 1 -- 172.21.15.16:0/3310050027 >> 172.21.15.16:0/3310050027 conn(0x7fb8c809d760 msgr2=0x7fb8c815c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.123+0000 7fb8cff11640 1 -- 172.21.15.16:0/3310050027 shutdown_connections 2024-09-19T07:45:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.124+0000 7fb8cff11640 1 -- 172.21.15.16:0/3310050027 wait complete. 2024-09-19T07:45:09.141 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T07:45:09.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.256+0000 7faa6b642640 1 Processor -- start 2024-09-19T07:45:09.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.256+0000 7faa6b642640 1 -- start start 2024-09-19T07:45:09.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.257+0000 7faa6b642640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6415e0a0 0x7faa64162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:09.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.257+0000 7faa6b642640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faa640b5280 0x7faa640d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:09.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.257+0000 7faa6b642640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa6415f250 con 0x7faa640d43e0 2024-09-19T07:45:09.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.257+0000 7faa6b642640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7faa640d4d60 con 0x7faa640b5280 2024-09-19T07:45:09.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.257+0000 7faa6b642640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faa6415f530 con 0x7faa6415e0a0 2024-09-19T07:45:09.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.257+0000 7faa693de640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6415e0a0 0x7faa64162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:09.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.257+0000 7faa69bdf640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faa640b5280 0x7faa640d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:09.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.258+0000 7faa68bdd640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7faa640d43e0 0x7faa640b4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:37768/0 (socket says 172.21.15.16:37768) 2024-09-19T07:45:09.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.258+0000 7faa68bdd640 1 -- 172.21.15.16:0/3166703453 learned_addr learned my addr 172.21.15.16:0/3166703453 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:09.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.258+0000 7faa63fff640 1 -- 172.21.15.16:0/3166703453 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1029156774 0 0) 0x7faa6415f250 con 0x7faa640d43e0 2024-09-19T07:45:09.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.258+0000 7faa63fff640 1 -- 172.21.15.16:0/3166703453 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa4c003520 con 0x7faa640d43e0 2024-09-19T07:45:09.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.258+0000 7faa693de640 1 -- 172.21.15.16:0/3166703453 >> v2:172.21.15.146:3300/0 conn(0x7faa640b5280 msgr2=0x7faa640d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:09.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.258+0000 7faa693de640 1 --2- 172.21.15.16:0/3166703453 >> v2:172.21.15.146:3300/0 conn(0x7faa640b5280 0x7faa640d4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:09.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.258+0000 7faa693de640 1 -- 172.21.15.16:0/3166703453 >> v1:172.21.15.146:6789/0 conn(0x7faa640d43e0 legacy=0x7faa640b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.258+0000 7faa693de640 1 -- 172.21.15.16:0/3166703453 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6415f810 con 0x7faa6415e0a0 2024-09-19T07:45:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.258+0000 7faa69bdf640 1 --2- 172.21.15.16:0/3166703453 >> v2:172.21.15.146:3300/0 conn(0x7faa640b5280 0x7faa640d4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.259+0000 7faa693de640 1 --2- 172.21.15.16:0/3166703453 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6415e0a0 0x7faa64162a70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0097a0 tx=0x7faa5c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b64d5854469c950 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.259+0000 7faa63fff640 1 -- 172.21.15.16:0/3166703453 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c037070 con 0x7faa6415e0a0 2024-09-19T07:45:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.259+0000 7faa63fff640 1 -- 172.21.15.16:0/3166703453 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa5c032440 con 0x7faa6415e0a0 2024-09-19T07:45:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.259+0000 7faa63fff640 1 -- 172.21.15.16:0/3166703453 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c0327f0 con 0x7faa6415e0a0 2024-09-19T07:45:09.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.260+0000 7faa6b642640 1 -- 172.21.15.16:0/3166703453 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6415e0a0 msgr2=0x7faa64162a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:09.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.260+0000 7faa6b642640 1 --2- 172.21.15.16:0/3166703453 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6415e0a0 0x7faa64162a70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0097a0 tx=0x7faa5c02a180 comp rx=0 tx=0).stop 2024-09-19T07:45:09.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.260+0000 7faa6b642640 1 -- 172.21.15.16:0/3166703453 shutdown_connections 2024-09-19T07:45:09.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.260+0000 7faa6b642640 1 --2- 172.21.15.16:0/3166703453 >> v2:172.21.15.146:3300/0 conn(0x7faa640b5280 0x7faa640d4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:09.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.260+0000 7faa6b642640 1 --2- 172.21.15.16:0/3166703453 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6415e0a0 0x7faa64162a70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:09.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.260+0000 7faa6b642640 1 -- 172.21.15.16:0/3166703453 >> 172.21.15.16:0/3166703453 conn(0x7faa6409d770 msgr2=0x7faa6408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:09.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.260+0000 7faa6b642640 1 -- 172.21.15.16:0/3166703453 shutdown_connections 2024-09-19T07:45:09.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.260+0000 7faa6b642640 1 -- 172.21.15.16:0/3166703453 wait complete. 2024-09-19T07:45:09.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.261+0000 7faa6b642640 1 Processor -- start 2024-09-19T07:45:09.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.261+0000 7faa6b642640 1 -- start start 2024-09-19T07:45:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.262+0000 7faa6b642640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa640b5280 0x7faa6415ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.262+0000 7faa6b642640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa640d43e0 0x7faa640b9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.262+0000 7faa6b642640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6415e0a0 0x7faa640cf940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.262+0000 7faa6b642640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa6415e600 con 0x7faa640b5280 2024-09-19T07:45:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.262+0000 7faa6b642640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faa6415e480 con 0x7faa640d43e0 2024-09-19T07:45:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.262+0000 7faa6b642640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faa6415e780 con 0x7faa6415e0a0 2024-09-19T07:45:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.262+0000 7faa693de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa640b5280 0x7faa6415ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.262+0000 7faa693de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa640b5280 0x7faa6415ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57910/0 (socket says 172.21.15.16:57910) 2024-09-19T07:45:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.262+0000 7faa693de640 1 -- 172.21.15.16:0/1215710743 learned_addr learned my addr 172.21.15.16:0/1215710743 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.262+0000 7faa69bdf640 1 --2- 172.21.15.16:0/1215710743 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6415e0a0 0x7faa640cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.262+0000 7faa68bdd640 1 --2- 172.21.15.16:0/1215710743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa640d43e0 0x7faa640b9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.262+0000 7faa693de640 1 -- 172.21.15.16:0/1215710743 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6415e0a0 msgr2=0x7faa640cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:09.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.262+0000 7faa693de640 1 --2- 172.21.15.16:0/1215710743 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6415e0a0 0x7faa640cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:09.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.262+0000 7faa693de640 1 -- 172.21.15.16:0/1215710743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa640d43e0 msgr2=0x7faa640b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:09.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.262+0000 7faa693de640 1 --2- 172.21.15.16:0/1215710743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa640d43e0 0x7faa640b9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:09.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.262+0000 7faa693de640 1 -- 172.21.15.16:0/1215710743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa640b8290 con 0x7faa640b5280 2024-09-19T07:45:09.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.262+0000 7faa68bdd640 1 --2- 172.21.15.16:0/1215710743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa640d43e0 0x7faa640b9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:09.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.262+0000 7faa693de640 1 --2- 172.21.15.16:0/1215710743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa640b5280 0x7faa6415ad70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7faa5c02a1c0 tx=0x7faa5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:09.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.263+0000 7faa61ffb640 1 -- 172.21.15.16:0/1215710743 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c037040 con 0x7faa640b5280 2024-09-19T07:45:09.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.263+0000 7faa61ffb640 1 -- 172.21.15.16:0/1215710743 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa5c042070 con 0x7faa640b5280 2024-09-19T07:45:09.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.263+0000 7faa61ffb640 1 -- 172.21.15.16:0/1215710743 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c032830 con 0x7faa640b5280 2024-09-19T07:45:09.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.263+0000 7faa6b642640 1 -- 172.21.15.16:0/1215710743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa640b7710 con 0x7faa640b5280 2024-09-19T07:45:09.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.263+0000 7faa6b642640 1 -- 172.21.15.16:0/1215710743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa640b7cd0 con 0x7faa640b5280 2024-09-19T07:45:09.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.264+0000 7faa6b642640 1 -- 172.21.15.16:0/1215710743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa640d4820 con 0x7faa640b5280 2024-09-19T07:45:09.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.264+0000 7faa61ffb640 1 -- 172.21.15.16:0/1215710743 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7faa5c0329d0 con 0x7faa640b5280 2024-09-19T07:45:09.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.265+0000 7faa61ffb640 1 --2- 172.21.15.16:0/1215710743 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faa5006c2b0 0x7faa5006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:09.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.265+0000 7faa61ffb640 1 -- 172.21.15.16:0/1215710743 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5619+0+0 (secure 0 0 0) 0x7faa5c0abc90 con 0x7faa640b5280 2024-09-19T07:45:09.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.265+0000 7faa68bdd640 1 --2- 172.21.15.16:0/1215710743 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faa5006c2b0 0x7faa5006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:09.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.266+0000 7faa68bdd640 1 --2- 172.21.15.16:0/1215710743 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faa5006c2b0 0x7faa5006e770 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faa640b3880 tx=0x7faa58009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:09.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.269+0000 7faa61ffb640 1 -- 172.21.15.16:0/1215710743 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7faa5c07a450 con 0x7faa640b5280 2024-09-19T07:45:09.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:09.445+0000 7faa6b642640 1 -- 172.21.15.16:0/1215710743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7faa640ba3d0 con 0x7faa640b5280 2024-09-19T07:45:10.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.122+0000 7faa61ffb640 1 -- 172.21.15.16:0/1215710743 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v51) ==== 170+0+0 (secure 0 0 0) 0x7faa5c079d50 con 0x7faa640b5280 2024-09-19T07:45:10.123 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-19T07:45:10.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.126+0000 7faa6b642640 1 -- 172.21.15.16:0/1215710743 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faa5006c2b0 msgr2=0x7faa5006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:10.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.126+0000 7faa6b642640 1 --2- 172.21.15.16:0/1215710743 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faa5006c2b0 0x7faa5006e770 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faa640b3880 tx=0x7faa58009040 comp rx=0 tx=0).stop 2024-09-19T07:45:10.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.126+0000 7faa6b642640 1 -- 172.21.15.16:0/1215710743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa640b5280 msgr2=0x7faa6415ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:10.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.126+0000 7faa6b642640 1 --2- 172.21.15.16:0/1215710743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa640b5280 0x7faa6415ad70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7faa5c02a1c0 tx=0x7faa5c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:45:10.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.127+0000 7faa6b642640 1 -- 172.21.15.16:0/1215710743 shutdown_connections 2024-09-19T07:45:10.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.127+0000 7faa6b642640 1 --2- 172.21.15.16:0/1215710743 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faa5006c2b0 0x7faa5006e770 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:10.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.127+0000 7faa6b642640 1 --2- 172.21.15.16:0/1215710743 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6415e0a0 0x7faa640cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:10.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.127+0000 7faa6b642640 1 --2- 172.21.15.16:0/1215710743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa640d43e0 0x7faa640b9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:10.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.127+0000 7faa6b642640 1 --2- 172.21.15.16:0/1215710743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa640b5280 0x7faa6415ad70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:10.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.127+0000 7faa6b642640 1 -- 172.21.15.16:0/1215710743 >> 172.21.15.16:0/1215710743 conn(0x7faa6409d770 msgr2=0x7faa640d2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:10.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.128+0000 7faa6b642640 1 -- 172.21.15.16:0/1215710743 shutdown_connections 2024-09-19T07:45:10.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.130+0000 7faa6b642640 1 -- 172.21.15.16:0/1215710743 wait complete. 2024-09-19T07:45:10.159 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T07:45:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.269+0000 7f7a48d78640 1 Processor -- start 2024-09-19T07:45:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.269+0000 7f7a48d78640 1 -- start start 2024-09-19T07:45:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.270+0000 7f7a48d78640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a4415e2a0 0x7f7a4406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.270+0000 7f7a48d78640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a440d1880 con 0x7f7a4415e680 2024-09-19T07:45:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.270+0000 7f7a48d78640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a440d1b60 con 0x7f7a4406d960 2024-09-19T07:45:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.270+0000 7f7a48d78640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f7a440cf2d0 con 0x7f7a4415e2a0 2024-09-19T07:45:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.271+0000 7f7a4259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a4415e2a0 0x7f7a4406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.271+0000 7f7a4259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a4415e2a0 0x7f7a4406d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57914/0 (socket says 172.21.15.16:57914) 2024-09-19T07:45:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.271+0000 7f7a4259c640 1 -- 172.21.15.16:0/250479098 learned_addr learned my addr 172.21.15.16:0/250479098 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.271+0000 7f7a41d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7a4415e680 0x7f7a440cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58198/0 (socket says 172.21.15.16:58198) 2024-09-19T07:45:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.271+0000 7f7a4259c640 1 -- 172.21.15.16:0/250479098 >> v1:172.21.15.146:6789/0 conn(0x7f7a4406d960 legacy=0x7f7a440d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.271+0000 7f7a4259c640 1 -- 172.21.15.16:0/250479098 >> v1:172.21.15.16:6789/0 conn(0x7f7a4415e680 legacy=0x7f7a440cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.271+0000 7f7a4259c640 1 -- 172.21.15.16:0/250479098 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a440cf5b0 con 0x7f7a4415e2a0 2024-09-19T07:45:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.272+0000 7f7a4259c640 1 --2- 172.21.15.16:0/250479098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4415e2a0 0x7f7a4406d420 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009870 tx=0x7f7a3c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bc2ffbacf654b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.272+0000 7f7a4159a640 1 -- 172.21.15.16:0/250479098 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c037070 con 0x7f7a4415e2a0 2024-09-19T07:45:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.272+0000 7f7a4159a640 1 -- 172.21.15.16:0/250479098 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a3c02ae50 con 0x7f7a4415e2a0 2024-09-19T07:45:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.272+0000 7f7a4159a640 1 -- 172.21.15.16:0/250479098 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c032860 con 0x7f7a4415e2a0 2024-09-19T07:45:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.272+0000 7f7a48d78640 1 -- 172.21.15.16:0/250479098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4415e2a0 msgr2=0x7f7a4406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:10.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.272+0000 7f7a48d78640 1 --2- 172.21.15.16:0/250479098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4415e2a0 0x7f7a4406d420 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009870 tx=0x7f7a3c02a300 comp rx=0 tx=0).stop 2024-09-19T07:45:10.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.272+0000 7f7a48d78640 1 -- 172.21.15.16:0/250479098 shutdown_connections 2024-09-19T07:45:10.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.272+0000 7f7a48d78640 1 --2- 172.21.15.16:0/250479098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4415e2a0 0x7f7a4406d420 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:10.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.272+0000 7f7a48d78640 1 -- 172.21.15.16:0/250479098 >> 172.21.15.16:0/250479098 conn(0x7f7a4409d760 msgr2=0x7f7a4408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:10.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.273+0000 7f7a48d78640 1 -- 172.21.15.16:0/250479098 shutdown_connections 2024-09-19T07:45:10.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.273+0000 7f7a48d78640 1 -- 172.21.15.16:0/250479098 wait complete. 2024-09-19T07:45:10.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.274+0000 7f7a48d78640 1 Processor -- start 2024-09-19T07:45:10.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.274+0000 7f7a48d78640 1 -- start start 2024-09-19T07:45:10.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a48d78640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a4406d960 0x7f7a441f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:10.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a48d78640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4415e2a0 0x7f7a441f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:10.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a48d78640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a4415e680 0x7f7a441f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:10.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a48d78640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a440d0360 con 0x7f7a4415e2a0 2024-09-19T07:45:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a48d78640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7a440d01e0 con 0x7f7a4406d960 2024-09-19T07:45:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a48d78640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7a440d4a80 con 0x7f7a4415e680 2024-09-19T07:45:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a41d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4415e2a0 0x7f7a441f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a41d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4415e2a0 0x7f7a441f45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57924/0 (socket says 172.21.15.16:57924) 2024-09-19T07:45:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a4259c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a4406d960 0x7f7a441f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a41d9b640 1 -- 172.21.15.16:0/3703425991 learned_addr learned my addr 172.21.15.16:0/3703425991 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a42d9d640 1 --2- 172.21.15.16:0/3703425991 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a4415e680 0x7f7a441f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a41d9b640 1 -- 172.21.15.16:0/3703425991 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a4415e680 msgr2=0x7f7a441f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a41d9b640 1 --2- 172.21.15.16:0/3703425991 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a4415e680 0x7f7a441f4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a41d9b640 1 -- 172.21.15.16:0/3703425991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a4406d960 msgr2=0x7f7a441f40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a41d9b640 1 --2- 172.21.15.16:0/3703425991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a4406d960 0x7f7a441f40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a41d9b640 1 -- 172.21.15.16:0/3703425991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a440ca9f0 con 0x7f7a4415e2a0 2024-09-19T07:45:10.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a41d9b640 1 --2- 172.21.15.16:0/3703425991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4415e2a0 0x7f7a441f45e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00c3c0 tx=0x7f7a2c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:10.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a3affd640 1 -- 172.21.15.16:0/3703425991 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c0074e0 con 0x7f7a4415e2a0 2024-09-19T07:45:10.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a3affd640 1 -- 172.21.15.16:0/3703425991 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a2c007680 con 0x7f7a4415e2a0 2024-09-19T07:45:10.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a48d78640 1 -- 172.21.15.16:0/3703425991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a440c9950 con 0x7f7a4415e2a0 2024-09-19T07:45:10.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a3affd640 1 -- 172.21.15.16:0/3703425991 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c0049e0 con 0x7f7a4415e2a0 2024-09-19T07:45:10.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.275+0000 7f7a48d78640 1 -- 172.21.15.16:0/3703425991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a440c9f10 con 0x7f7a4415e2a0 2024-09-19T07:45:10.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.277+0000 7f7a48d78640 1 -- 172.21.15.16:0/3703425991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a4406d420 con 0x7f7a4415e2a0 2024-09-19T07:45:10.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.277+0000 7f7a3affd640 1 -- 172.21.15.16:0/3703425991 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7a2c01a020 con 0x7f7a4415e2a0 2024-09-19T07:45:10.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.278+0000 7f7a3affd640 1 --2- 172.21.15.16:0/3703425991 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7a2406c2b0 0x7f7a2406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:10.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.278+0000 7f7a3affd640 1 -- 172.21.15.16:0/3703425991 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6001+0+0 (secure 0 0 0) 0x7f7a2c08ab50 con 0x7f7a4415e2a0 2024-09-19T07:45:10.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.278+0000 7f7a4259c640 1 --2- 172.21.15.16:0/3703425991 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7a2406c2b0 0x7f7a2406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:10.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.278+0000 7f7a4259c640 1 --2- 172.21.15.16:0/3703425991 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7a2406c2b0 0x7f7a2406e770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c034000 tx=0x7f7a3c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.282+0000 7f7a3affd640 1 -- 172.21.15.16:0/3703425991 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7a2c058bd0 con 0x7f7a4415e2a0 2024-09-19T07:45:10.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:10.493+0000 7f7a48d78640 1 -- 172.21.15.16:0/3703425991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7a4406bc10 con 0x7f7a4415e2a0 2024-09-19T07:45:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.144+0000 7f7a3affd640 1 -- 172.21.15.16:0/3703425991 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v21) ==== 172+0+0 (secure 0 0 0) 0x7f7a2c05ced0 con 0x7f7a4415e2a0 2024-09-19T07:45:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 7 and data pool 8 2024-09-19T07:45:11.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.156+0000 7f7a48d78640 1 -- 172.21.15.16:0/3703425991 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7a2406c2b0 msgr2=0x7f7a2406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:11.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.156+0000 7f7a48d78640 1 --2- 172.21.15.16:0/3703425991 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7a2406c2b0 0x7f7a2406e770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c034000 tx=0x7f7a3c02e000 comp rx=0 tx=0).stop 2024-09-19T07:45:11.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.156+0000 7f7a48d78640 1 -- 172.21.15.16:0/3703425991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4415e2a0 msgr2=0x7f7a441f45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:11.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.156+0000 7f7a48d78640 1 --2- 172.21.15.16:0/3703425991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4415e2a0 0x7f7a441f45e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00c3c0 tx=0x7f7a2c00c880 comp rx=0 tx=0).stop 2024-09-19T07:45:11.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.156+0000 7f7a48d78640 1 -- 172.21.15.16:0/3703425991 shutdown_connections 2024-09-19T07:45:11.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.156+0000 7f7a48d78640 1 --2- 172.21.15.16:0/3703425991 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7a2406c2b0 0x7f7a2406e770 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.156+0000 7f7a48d78640 1 --2- 172.21.15.16:0/3703425991 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a4415e680 0x7f7a441f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.156+0000 7f7a48d78640 1 --2- 172.21.15.16:0/3703425991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a4415e2a0 0x7f7a441f45e0 secure :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00c3c0 tx=0x7f7a2c00c880 comp rx=0 tx=0).stop 2024-09-19T07:45:11.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.156+0000 7f7a48d78640 1 --2- 172.21.15.16:0/3703425991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a4406d960 0x7f7a441f40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.156+0000 7f7a48d78640 1 -- 172.21.15.16:0/3703425991 >> 172.21.15.16:0/3703425991 conn(0x7f7a4409d760 msgr2=0x7f7a440d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:11.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.157+0000 7f7a48d78640 1 -- 172.21.15.16:0/3703425991 shutdown_connections 2024-09-19T07:45:11.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.157+0000 7f7a48d78640 1 -- 172.21.15.16:0/3703425991 wait complete. 2024-09-19T07:45:11.183 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:45:11.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.290+0000 7f879aa5e640 1 Processor -- start 2024-09-19T07:45:11.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.290+0000 7f879aa5e640 1 -- start start 2024-09-19T07:45:11.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f879aa5e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f879415e2a0 0x7f879406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f879aa5e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87940d1880 con 0x7f879415e680 2024-09-19T07:45:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f879aa5e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87940d1b60 con 0x7f879406d960 2024-09-19T07:45:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f879aa5e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f87940cf2d0 con 0x7f879415e2a0 2024-09-19T07:45:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f87937fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f879415e680 0x7f87940cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58214/0 (socket says 172.21.15.16:58214) 2024-09-19T07:45:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f87937fe640 1 -- 172.21.15.16:0/4294371800 learned_addr learned my addr 172.21.15.16:0/4294371800 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f8793fff640 1 --2- 172.21.15.16:0/4294371800 >> v2:172.21.15.146:3300/0 conn(0x7f879415e2a0 0x7f879406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f8792ffd640 1 -- 172.21.15.16:0/4294371800 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1938423468 0 0) 0x7f87940d1880 con 0x7f879415e680 2024-09-19T07:45:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f8792ffd640 1 -- 172.21.15.16:0/4294371800 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f877c003520 con 0x7f879415e680 2024-09-19T07:45:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f8792ffd640 1 -- 172.21.15.16:0/4294371800 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 188015564 0 0) 0x7f877c003520 con 0x7f879415e680 2024-09-19T07:45:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f8792ffd640 1 -- 172.21.15.16:0/4294371800 >> v2:172.21.15.146:3300/0 conn(0x7f879415e2a0 msgr2=0x7f879406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f8792ffd640 1 --2- 172.21.15.16:0/4294371800 >> v2:172.21.15.146:3300/0 conn(0x7f879415e2a0 0x7f879406d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f8792ffd640 1 -- 172.21.15.16:0/4294371800 >> v1:172.21.15.146:6789/0 conn(0x7f879406d960 legacy=0x7f87940d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f8792ffd640 1 -- 172.21.15.16:0/4294371800 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87940cf5b0 con 0x7f879415e680 2024-09-19T07:45:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f8792ffd640 1 -- 172.21.15.16:0/4294371800 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f8788001670 con 0x7f879415e680 2024-09-19T07:45:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f8792ffd640 1 -- 172.21.15.16:0/4294371800 >> v1:172.21.15.16:6789/0 conn(0x7f879415e680 legacy=0x7f87940cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f8792ffd640 1 --2- 172.21.15.16:0/4294371800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f877c0039b0 0x7f877c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f8792ffd640 1 --2- 172.21.15.16:0/4294371800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f877c004800 0x7f877c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f8792ffd640 1 --2- 172.21.15.16:0/4294371800 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f877c0051c0 0x7f877c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f8792ffd640 1 -- 172.21.15.16:0/4294371800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87940cf2d0 con 0x7f877c004800 2024-09-19T07:45:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f8792ffd640 1 -- 172.21.15.16:0/4294371800 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f877c00c060 con 0x7f877c0039b0 2024-09-19T07:45:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.291+0000 7f8792ffd640 1 -- 172.21.15.16:0/4294371800 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f877c00c260 con 0x7f877c0051c0 2024-09-19T07:45:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.292+0000 7f87937fe640 1 --2- 172.21.15.16:0/4294371800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f877c004800 0x7f877c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.292+0000 7f8798ffb640 1 --2- 172.21.15.16:0/4294371800 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f877c0051c0 0x7f877c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.292+0000 7f8793fff640 1 --2- 172.21.15.16:0/4294371800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f877c0039b0 0x7f877c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.292+0000 7f87937fe640 1 -- 172.21.15.16:0/4294371800 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f877c0051c0 msgr2=0x7f877c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:11.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.292+0000 7f87937fe640 1 --2- 172.21.15.16:0/4294371800 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f877c0051c0 0x7f877c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.292+0000 7f87937fe640 1 -- 172.21.15.16:0/4294371800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f877c0039b0 msgr2=0x7f877c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:11.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.292+0000 7f87937fe640 1 --2- 172.21.15.16:0/4294371800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f877c0039b0 0x7f877c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.292+0000 7f87937fe640 1 -- 172.21.15.16:0/4294371800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877c003520 con 0x7f877c004800 2024-09-19T07:45:11.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.292+0000 7f8798ffb640 1 -- 172.21.15.16:0/4294371800 reap_dead start 2024-09-19T07:45:11.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.293+0000 7f87937fe640 1 --2- 172.21.15.16:0/4294371800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f877c004800 0x7f877c004c80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f879415ea80 tx=0x7f877c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a847f0aff4f85cae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:11.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.293+0000 7f8792ffd640 1 -- 172.21.15.16:0/4294371800 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788007d30 con 0x7f877c004800 2024-09-19T07:45:11.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.293+0000 7f8792ffd640 1 -- 172.21.15.16:0/4294371800 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8788005ad0 con 0x7f877c004800 2024-09-19T07:45:11.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.293+0000 7f8792ffd640 1 -- 172.21.15.16:0/4294371800 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788007070 con 0x7f877c004800 2024-09-19T07:45:11.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.293+0000 7f879aa5e640 1 -- 172.21.15.16:0/4294371800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f877c004800 msgr2=0x7f877c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:11.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.293+0000 7f879aa5e640 1 --2- 172.21.15.16:0/4294371800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f877c004800 0x7f877c004c80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f879415ea80 tx=0x7f877c005830 comp rx=0 tx=0).stop 2024-09-19T07:45:11.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.293+0000 7f879aa5e640 1 -- 172.21.15.16:0/4294371800 shutdown_connections 2024-09-19T07:45:11.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.293+0000 7f879aa5e640 1 --2- 172.21.15.16:0/4294371800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f877c004800 0x7f877c004c80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.293+0000 7f879aa5e640 1 -- 172.21.15.16:0/4294371800 >> 172.21.15.16:0/4294371800 conn(0x7f879409d760 msgr2=0x7f879408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.293+0000 7f879aa5e640 1 -- 172.21.15.16:0/4294371800 shutdown_connections 2024-09-19T07:45:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.293+0000 7f879aa5e640 1 -- 172.21.15.16:0/4294371800 wait complete. 2024-09-19T07:45:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.295+0000 7f879aa5e640 1 Processor -- start 2024-09-19T07:45:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.295+0000 7f879aa5e640 1 -- start start 2024-09-19T07:45:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f879aa5e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f877c004800 0x7f87941493c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f879aa5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8794149900 0x7f87940b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f879aa5e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87940b87a0 0x7f87940b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f879aa5e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f879406d5a0 con 0x7f8794149900 2024-09-19T07:45:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f879aa5e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f879406d420 con 0x7f87940b87a0 2024-09-19T07:45:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f879aa5e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f879406d720 con 0x7f877c004800 2024-09-19T07:45:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f87937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8794149900 0x7f87940b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f87937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8794149900 0x7f87940b8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57940/0 (socket says 172.21.15.16:57940) 2024-09-19T07:45:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f87937fe640 1 -- 172.21.15.16:0/655894648 learned_addr learned my addr 172.21.15.16:0/655894648 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f8793fff640 1 --2- 172.21.15.16:0/655894648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f877c004800 0x7f87941493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f8798ffb640 1 --2- 172.21.15.16:0/655894648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87940b87a0 0x7f87940b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f87937fe640 1 -- 172.21.15.16:0/655894648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f877c004800 msgr2=0x7f87941493c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:11.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f87937fe640 1 --2- 172.21.15.16:0/655894648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f877c004800 0x7f87941493c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f87937fe640 1 -- 172.21.15.16:0/655894648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87940b87a0 msgr2=0x7f87940b8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:11.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f87937fe640 1 --2- 172.21.15.16:0/655894648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87940b87a0 0x7f87940b8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f87937fe640 1 -- 172.21.15.16:0/655894648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8794159f30 con 0x7f8794149900 2024-09-19T07:45:11.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f87937fe640 1 --2- 172.21.15.16:0/655894648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8794149900 0x7f87940b8260 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f878802adb0 tx=0x7f8788001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:11.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f8790ff9640 1 -- 172.21.15.16:0/655894648 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788040030 con 0x7f8794149900 2024-09-19T07:45:11.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f8790ff9640 1 -- 172.21.15.16:0/655894648 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8788007d10 con 0x7f8794149900 2024-09-19T07:45:11.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f879aa5e640 1 -- 172.21.15.16:0/655894648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87940caf30 con 0x7f8794149900 2024-09-19T07:45:11.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.296+0000 7f8790ff9640 1 -- 172.21.15.16:0/655894648 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788037040 con 0x7f8794149900 2024-09-19T07:45:11.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.297+0000 7f879aa5e640 1 -- 172.21.15.16:0/655894648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87941e8210 con 0x7f8794149900 2024-09-19T07:45:11.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.298+0000 7f879aa5e640 1 -- 172.21.15.16:0/655894648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87940d0c50 con 0x7f8794149900 2024-09-19T07:45:11.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.298+0000 7f8790ff9640 1 -- 172.21.15.16:0/655894648 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8788043070 con 0x7f8794149900 2024-09-19T07:45:11.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.298+0000 7f8790ff9640 1 --2- 172.21.15.16:0/655894648 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f878006c2e0 0x7f878006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.299+0000 7f8790ff9640 1 -- 172.21.15.16:0/655894648 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f87880aa880 con 0x7f8794149900 2024-09-19T07:45:11.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.299+0000 7f8793fff640 1 --2- 172.21.15.16:0/655894648 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f878006c2e0 0x7f878006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.299+0000 7f8793fff640 1 --2- 172.21.15.16:0/655894648 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f878006c2e0 0x7f878006e7a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8784009bc0 tx=0x7f8784009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:11.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.303+0000 7f8790ff9640 1 -- 172.21.15.16:0/655894648 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f878804a020 con 0x7f8794149900 2024-09-19T07:45:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.480+0000 7f879aa5e640 1 -- 172.21.15.16:0/655894648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87941628e0 con 0x7f8794149900 2024-09-19T07:45:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.482+0000 7f8790ff9640 1 -- 172.21.15.16:0/655894648 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+17039 (secure 0 0 0) 0x7f8794159f30 con 0x7f8794149900 2024-09-19T07:45:11.483 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:45:11.483 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":52,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:45:10.494205+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:45:08.455974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-19T07:45:09.476448+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:45:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.484+0000 7f879aa5e640 1 -- 172.21.15.16:0/655894648 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f878006c2e0 msgr2=0x7f878006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.484+0000 7f879aa5e640 1 --2- 172.21.15.16:0/655894648 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f878006c2e0 0x7f878006e7a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8784009bc0 tx=0x7f8784009190 comp rx=0 tx=0).stop 2024-09-19T07:45:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.484+0000 7f879aa5e640 1 -- 172.21.15.16:0/655894648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8794149900 msgr2=0x7f87940b8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.484+0000 7f879aa5e640 1 --2- 172.21.15.16:0/655894648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8794149900 0x7f87940b8260 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f878802adb0 tx=0x7f8788001a40 comp rx=0 tx=0).stop 2024-09-19T07:45:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.484+0000 7f879aa5e640 1 -- 172.21.15.16:0/655894648 shutdown_connections 2024-09-19T07:45:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.484+0000 7f879aa5e640 1 --2- 172.21.15.16:0/655894648 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f878006c2e0 0x7f878006e7a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.484+0000 7f879aa5e640 1 --2- 172.21.15.16:0/655894648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87940b87a0 0x7f87940b8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.484+0000 7f879aa5e640 1 --2- 172.21.15.16:0/655894648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8794149900 0x7f87940b8260 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.484+0000 7f879aa5e640 1 --2- 172.21.15.16:0/655894648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f877c004800 0x7f87941493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.484+0000 7f879aa5e640 1 -- 172.21.15.16:0/655894648 >> 172.21.15.16:0/655894648 conn(0x7f879409d760 msgr2=0x7f879415c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.484+0000 7f879aa5e640 1 -- 172.21.15.16:0/655894648 shutdown_connections 2024-09-19T07:45:11.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.484+0000 7f879aa5e640 1 -- 172.21.15.16:0/655894648 wait complete. 2024-09-19T07:45:11.496 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:45:11.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.598+0000 7fa63c22e640 1 Processor -- start 2024-09-19T07:45:11.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.598+0000 7fa63c22e640 1 -- start start 2024-09-19T07:45:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.599+0000 7fa63c22e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6340c8d20 0x7fa63406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.599+0000 7fa63c22e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa63406d260 0x7fa6341439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.599+0000 7fa63c22e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6340c7140 con 0x7fa634143f30 2024-09-19T07:45:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.599+0000 7fa63c22e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa634141960 con 0x7fa63406d260 2024-09-19T07:45:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.599+0000 7fa63c22e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa634142200 con 0x7fa6340c8d20 2024-09-19T07:45:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.599+0000 7fa6397c9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa63406d260 0x7fa6341439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.599+0000 7fa63a7cb640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa634143f30 0x7fa6340d1e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58222/0 (socket says 172.21.15.16:58222) 2024-09-19T07:45:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.599+0000 7fa6397c9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa63406d260 0x7fa6341439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57956/0 (socket says 172.21.15.16:57956) 2024-09-19T07:45:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.599+0000 7fa63a7cb640 1 -- 172.21.15.16:0/2071375775 learned_addr learned my addr 172.21.15.16:0/2071375775 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.599+0000 7fa639fca640 1 --2- 172.21.15.16:0/2071375775 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6340c8d20 0x7fa63406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.599+0000 7fa6397c9640 1 -- 172.21.15.16:0/2071375775 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6340c8d20 msgr2=0x7fa63406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:11.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.599+0000 7fa6397c9640 1 --2- 172.21.15.16:0/2071375775 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6340c8d20 0x7fa63406cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.599+0000 7fa6397c9640 1 -- 172.21.15.16:0/2071375775 >> v1:172.21.15.16:6789/0 conn(0x7fa634143f30 legacy=0x7fa6340d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:11.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.599+0000 7fa6397c9640 1 -- 172.21.15.16:0/2071375775 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6340c9100 con 0x7fa63406d260 2024-09-19T07:45:11.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.599+0000 7fa638fc8640 1 -- 172.21.15.16:0/2071375775 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2404004688 0 0) 0x7fa62c001110 con 0x7fa634143f30 2024-09-19T07:45:11.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.600+0000 7fa6397c9640 1 --2- 172.21.15.16:0/2071375775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa63406d260 0x7fa6341439f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa6240097a0 tx=0x7fa62402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3f0c8acda5d457 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:11.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.600+0000 7fa638fc8640 1 -- 172.21.15.16:0/2071375775 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624037070 con 0x7fa63406d260 2024-09-19T07:45:11.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.600+0000 7fa638fc8640 1 -- 172.21.15.16:0/2071375775 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa62402adb0 con 0x7fa63406d260 2024-09-19T07:45:11.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.600+0000 7fa638fc8640 1 -- 172.21.15.16:0/2071375775 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624032660 con 0x7fa63406d260 2024-09-19T07:45:11.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.601+0000 7fa63c22e640 1 -- 172.21.15.16:0/2071375775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa63406d260 msgr2=0x7fa6341439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:11.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.601+0000 7fa63c22e640 1 --2- 172.21.15.16:0/2071375775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa63406d260 0x7fa6341439f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa6240097a0 tx=0x7fa62402a060 comp rx=0 tx=0).stop 2024-09-19T07:45:11.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.601+0000 7fa63c22e640 1 -- 172.21.15.16:0/2071375775 shutdown_connections 2024-09-19T07:45:11.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.601+0000 7fa63c22e640 1 --2- 172.21.15.16:0/2071375775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa63406d260 0x7fa6341439f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.601+0000 7fa63c22e640 1 --2- 172.21.15.16:0/2071375775 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6340c8d20 0x7fa63406cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.601+0000 7fa63c22e640 1 -- 172.21.15.16:0/2071375775 >> 172.21.15.16:0/2071375775 conn(0x7fa6340b3ed0 msgr2=0x7fa6340b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:11.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.601+0000 7fa63c22e640 1 -- 172.21.15.16:0/2071375775 shutdown_connections 2024-09-19T07:45:11.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.601+0000 7fa63c22e640 1 -- 172.21.15.16:0/2071375775 wait complete. 2024-09-19T07:45:11.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.602+0000 7fa63c22e640 1 Processor -- start 2024-09-19T07:45:11.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.602+0000 7fa63c22e640 1 -- start start 2024-09-19T07:45:11.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.603+0000 7fa63c22e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa63406d260 0x7fa634140d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.603+0000 7fa63c22e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6340c8d20 0x7fa6340cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.603+0000 7fa63c22e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa634143f30 0x7fa6340cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.603+0000 7fa63c22e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa634141bb0 con 0x7fa634143f30 2024-09-19T07:45:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.603+0000 7fa63c22e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa63406d6a0 con 0x7fa6340c8d20 2024-09-19T07:45:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.603+0000 7fa63c22e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa634141e90 con 0x7fa63406d260 2024-09-19T07:45:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.603+0000 7fa63a7cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa634143f30 0x7fa6340cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.603+0000 7fa63a7cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa634143f30 0x7fa6340cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57970/0 (socket says 172.21.15.16:57970) 2024-09-19T07:45:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.603+0000 7fa63a7cb640 1 -- 172.21.15.16:0/2346421361 learned_addr learned my addr 172.21.15.16:0/2346421361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.603+0000 7fa6397c9640 1 --2- 172.21.15.16:0/2346421361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6340c8d20 0x7fa6340cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.603+0000 7fa639fca640 1 --2- 172.21.15.16:0/2346421361 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa63406d260 0x7fa634140d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.603+0000 7fa63a7cb640 1 -- 172.21.15.16:0/2346421361 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa63406d260 msgr2=0x7fa634140d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.603+0000 7fa63a7cb640 1 --2- 172.21.15.16:0/2346421361 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa63406d260 0x7fa634140d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.603+0000 7fa63a7cb640 1 -- 172.21.15.16:0/2346421361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6340c8d20 msgr2=0x7fa6340cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.603+0000 7fa63a7cb640 1 --2- 172.21.15.16:0/2346421361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6340c8d20 0x7fa6340cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.603+0000 7fa63a7cb640 1 -- 172.21.15.16:0/2346421361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6340d07b0 con 0x7fa634143f30 2024-09-19T07:45:11.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.603+0000 7fa639fca640 1 --2- 172.21.15.16:0/2346421361 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa63406d260 0x7fa634140d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:11.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.603+0000 7fa63a7cb640 1 --2- 172.21.15.16:0/2346421361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa634143f30 0x7fa6340cb500 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00c440 tx=0x7fa62c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:11.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.604+0000 7fa62a7fc640 1 -- 172.21.15.16:0/2346421361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c0074e0 con 0x7fa634143f30 2024-09-19T07:45:11.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.604+0000 7fa62a7fc640 1 -- 172.21.15.16:0/2346421361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa62c0047e0 con 0x7fa634143f30 2024-09-19T07:45:11.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.604+0000 7fa63c22e640 1 -- 172.21.15.16:0/2346421361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6340d0a90 con 0x7fa634143f30 2024-09-19T07:45:11.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.604+0000 7fa62a7fc640 1 -- 172.21.15.16:0/2346421361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c004bd0 con 0x7fa634143f30 2024-09-19T07:45:11.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.604+0000 7fa63c22e640 1 -- 172.21.15.16:0/2346421361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6340d1050 con 0x7fa634143f30 2024-09-19T07:45:11.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.605+0000 7fa63c22e640 1 -- 172.21.15.16:0/2346421361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa634144360 con 0x7fa634143f30 2024-09-19T07:45:11.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.605+0000 7fa62a7fc640 1 -- 172.21.15.16:0/2346421361 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa62c011070 con 0x7fa634143f30 2024-09-19T07:45:11.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.606+0000 7fa62a7fc640 1 --2- 172.21.15.16:0/2346421361 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa61806c2b0 0x7fa61806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.606+0000 7fa62a7fc640 1 -- 172.21.15.16:0/2346421361 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fa62c08b0b0 con 0x7fa634143f30 2024-09-19T07:45:11.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.606+0000 7fa639fca640 1 --2- 172.21.15.16:0/2346421361 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa61806c2b0 0x7fa61806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.607+0000 7fa639fca640 1 --2- 172.21.15.16:0/2346421361 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa61806c2b0 0x7fa61806e770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa630009fd0 tx=0x7fa63000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:11.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.610+0000 7fa62a7fc640 1 -- 172.21.15.16:0/2346421361 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa6340c7140 con 0x7fa634143f30 2024-09-19T07:45:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.784+0000 7fa63c22e640 1 -- 172.21.15.16:0/2346421361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6340690c0 con 0x7fa634143f30 2024-09-19T07:45:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.785+0000 7fa62a7fc640 1 -- 172.21.15.16:0/2346421361 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+17039 (secure 0 0 0) 0x7fa62c059690 con 0x7fa634143f30 2024-09-19T07:45:11.786 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:45:11.786 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":52,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:45:10.494205+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:45:08.455974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-19T07:45:09.476448+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:45:11.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.788+0000 7fa63c22e640 1 -- 172.21.15.16:0/2346421361 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa61806c2b0 msgr2=0x7fa61806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.788+0000 7fa63c22e640 1 --2- 172.21.15.16:0/2346421361 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa61806c2b0 0x7fa61806e770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa630009fd0 tx=0x7fa63000b040 comp rx=0 tx=0).stop 2024-09-19T07:45:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.788+0000 7fa63c22e640 1 -- 172.21.15.16:0/2346421361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa634143f30 msgr2=0x7fa6340cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.788+0000 7fa63c22e640 1 --2- 172.21.15.16:0/2346421361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa634143f30 0x7fa6340cb500 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00c440 tx=0x7fa62c00c900 comp rx=0 tx=0).stop 2024-09-19T07:45:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.788+0000 7fa63c22e640 1 -- 172.21.15.16:0/2346421361 shutdown_connections 2024-09-19T07:45:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.788+0000 7fa63c22e640 1 --2- 172.21.15.16:0/2346421361 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa61806c2b0 0x7fa61806e770 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.788+0000 7fa63c22e640 1 --2- 172.21.15.16:0/2346421361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa634143f30 0x7fa6340cb500 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.788+0000 7fa63c22e640 1 --2- 172.21.15.16:0/2346421361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6340c8d20 0x7fa6340cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.788+0000 7fa63c22e640 1 --2- 172.21.15.16:0/2346421361 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa63406d260 0x7fa634140d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.788+0000 7fa63c22e640 1 -- 172.21.15.16:0/2346421361 >> 172.21.15.16:0/2346421361 conn(0x7fa6340b3ed0 msgr2=0x7fa634143230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.788+0000 7fa63c22e640 1 -- 172.21.15.16:0/2346421361 shutdown_connections 2024-09-19T07:45:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.788+0000 7fa63c22e640 1 -- 172.21.15.16:0/2346421361 wait complete. 2024-09-19T07:45:11.799 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T07:45:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.904+0000 7f3ceb2e6640 1 Processor -- start 2024-09-19T07:45:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.904+0000 7f3ceb2e6640 1 -- start start 2024-09-19T07:45:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.905+0000 7f3ceb2e6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3ce41619b0 0x7f3ce40d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.905+0000 7f3ceb2e6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ce40b0a80 0x7f3ce40d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.905+0000 7f3ceb2e6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ce406c990 con 0x7f3ce4161d90 2024-09-19T07:45:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.905+0000 7f3ceb2e6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f3ce40d0ec0 con 0x7f3ce41619b0 2024-09-19T07:45:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.905+0000 7f3ceb2e6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3ce40cec10 con 0x7f3ce40b0a80 2024-09-19T07:45:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.905+0000 7f3ce8881640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ce40b0a80 0x7f3ce40d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.905+0000 7f3ce9082640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3ce41619b0 0x7f3ce40d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.905+0000 7f3ce8881640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ce40b0a80 0x7f3ce40d0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:51116/0 (socket says 172.21.15.16:51116) 2024-09-19T07:45:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.905+0000 7f3ce8881640 1 -- 172.21.15.16:0/2085358622 learned_addr learned my addr 172.21.15.16:0/2085358622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.905+0000 7f3ce9883640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f3ce4161d90 0x7f3ce40cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:37788/0 (socket says 172.21.15.16:37788) 2024-09-19T07:45:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.906+0000 7f3ce8881640 1 -- 172.21.15.16:0/2085358622 >> v2:172.21.15.146:3300/0 conn(0x7f3ce41619b0 msgr2=0x7f3ce40d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.906+0000 7f3ce8881640 1 --2- 172.21.15.16:0/2085358622 >> v2:172.21.15.146:3300/0 conn(0x7f3ce41619b0 0x7f3ce40d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.906+0000 7f3ce8881640 1 -- 172.21.15.16:0/2085358622 >> v1:172.21.15.146:6789/0 conn(0x7f3ce4161d90 legacy=0x7f3ce40cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.906+0000 7f3ce8881640 1 -- 172.21.15.16:0/2085358622 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce40ce930 con 0x7f3ce40b0a80 2024-09-19T07:45:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.906+0000 7f3cdbfff640 1 -- 172.21.15.16:0/2085358622 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1596527427 0 0) 0x7f3ce406c990 con 0x7f3ce4161d90 2024-09-19T07:45:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.906+0000 7f3ce9082640 1 --2- 172.21.15.16:0/2085358622 >> v2:172.21.15.146:3300/0 conn(0x7f3ce41619b0 0x7f3ce40d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.906+0000 7f3ce8881640 1 --2- 172.21.15.16:0/2085358622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ce40b0a80 0x7f3ce40d0980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4009870 tx=0x7f3cd402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81cb2bd77c2c4379 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.907+0000 7f3cdbfff640 1 -- 172.21.15.16:0/2085358622 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4037070 con 0x7f3ce40b0a80 2024-09-19T07:45:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.907+0000 7f3cdbfff640 1 -- 172.21.15.16:0/2085358622 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cd4032440 con 0x7f3ce40b0a80 2024-09-19T07:45:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.907+0000 7f3cdbfff640 1 -- 172.21.15.16:0/2085358622 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4032810 con 0x7f3ce40b0a80 2024-09-19T07:45:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.907+0000 7f3ceb2e6640 1 -- 172.21.15.16:0/2085358622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ce40b0a80 msgr2=0x7f3ce40d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.907+0000 7f3ceb2e6640 1 --2- 172.21.15.16:0/2085358622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ce40b0a80 0x7f3ce40d0980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4009870 tx=0x7f3cd402a250 comp rx=0 tx=0).stop 2024-09-19T07:45:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.907+0000 7f3ceb2e6640 1 -- 172.21.15.16:0/2085358622 shutdown_connections 2024-09-19T07:45:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.907+0000 7f3ceb2e6640 1 --2- 172.21.15.16:0/2085358622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ce40b0a80 0x7f3ce40d0980 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.907+0000 7f3ceb2e6640 1 --2- 172.21.15.16:0/2085358622 >> v2:172.21.15.146:3300/0 conn(0x7f3ce41619b0 0x7f3ce40d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.907+0000 7f3ceb2e6640 1 -- 172.21.15.16:0/2085358622 >> 172.21.15.16:0/2085358622 conn(0x7f3ce409a9f0 msgr2=0x7f3ce40b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.907+0000 7f3ceb2e6640 1 -- 172.21.15.16:0/2085358622 shutdown_connections 2024-09-19T07:45:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.907+0000 7f3ceb2e6640 1 -- 172.21.15.16:0/2085358622 wait complete. 2024-09-19T07:45:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.908+0000 7f3ceb2e6640 1 Processor -- start 2024-09-19T07:45:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.908+0000 7f3ceb2e6640 1 -- start start 2024-09-19T07:45:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.909+0000 7f3ceb2e6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ce40b0a80 0x7f3ce40c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.909+0000 7f3ceb2e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ce41619b0 0x7f3ce4157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.909+0000 7f3ceb2e6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ce4161d90 0x7f3ce40b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.909+0000 7f3ceb2e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce406c620 con 0x7f3ce41619b0 2024-09-19T07:45:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.909+0000 7f3ceb2e6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce40b0820 con 0x7f3ce4161d90 2024-09-19T07:45:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.909+0000 7f3ceb2e6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3ce40ccce0 con 0x7f3ce40b0a80 2024-09-19T07:45:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.909+0000 7f3ce8881640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ce41619b0 0x7f3ce4157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.909+0000 7f3ce8881640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ce41619b0 0x7f3ce4157830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57976/0 (socket says 172.21.15.16:57976) 2024-09-19T07:45:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.909+0000 7f3ce8881640 1 -- 172.21.15.16:0/4155098631 learned_addr learned my addr 172.21.15.16:0/4155098631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.910+0000 7f3ce9082640 1 --2- 172.21.15.16:0/4155098631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ce40b0a80 0x7f3ce40c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.910+0000 7f3ce9883640 1 --2- 172.21.15.16:0/4155098631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ce4161d90 0x7f3ce40b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.910+0000 7f3ce8881640 1 -- 172.21.15.16:0/4155098631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ce40b0a80 msgr2=0x7f3ce40c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.910+0000 7f3ce8881640 1 --2- 172.21.15.16:0/4155098631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ce40b0a80 0x7f3ce40c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.910+0000 7f3ce8881640 1 -- 172.21.15.16:0/4155098631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ce4161d90 msgr2=0x7f3ce40b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.910+0000 7f3ce8881640 1 --2- 172.21.15.16:0/4155098631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ce4161d90 0x7f3ce40b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.910+0000 7f3ce8881640 1 -- 172.21.15.16:0/4155098631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce40c9b00 con 0x7f3ce41619b0 2024-09-19T07:45:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.910+0000 7f3ce9883640 1 --2- 172.21.15.16:0/4155098631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ce4161d90 0x7f3ce40b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.910+0000 7f3ce9082640 1 --2- 172.21.15.16:0/4155098631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ce40b0a80 0x7f3ce40c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.910+0000 7f3ce8881640 1 --2- 172.21.15.16:0/4155098631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ce41619b0 0x7f3ce4157830 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4000c00 tx=0x7f3cd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.910+0000 7f3cd9ffb640 1 -- 172.21.15.16:0/4155098631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4037040 con 0x7f3ce41619b0 2024-09-19T07:45:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.910+0000 7f3cd9ffb640 1 -- 172.21.15.16:0/4155098631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cd4042070 con 0x7f3ce41619b0 2024-09-19T07:45:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.910+0000 7f3ceb2e6640 1 -- 172.21.15.16:0/4155098631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce40c9de0 con 0x7f3ce41619b0 2024-09-19T07:45:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.910+0000 7f3cd9ffb640 1 -- 172.21.15.16:0/4155098631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4030880 con 0x7f3ce41619b0 2024-09-19T07:45:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.910+0000 7f3ceb2e6640 1 -- 172.21.15.16:0/4155098631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce40ca3a0 con 0x7f3ce41619b0 2024-09-19T07:45:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.911+0000 7f3ceb2e6640 1 -- 172.21.15.16:0/4155098631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce40cd330 con 0x7f3ce41619b0 2024-09-19T07:45:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.912+0000 7f3cd9ffb640 1 -- 172.21.15.16:0/4155098631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3cd4030a20 con 0x7f3ce41619b0 2024-09-19T07:45:11.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.912+0000 7f3cd9ffb640 1 --2- 172.21.15.16:0/4155098631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3cc806c2b0 0x7f3cc806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:11.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.912+0000 7f3cd9ffb640 1 -- 172.21.15.16:0/4155098631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f3cd40abed0 con 0x7f3ce41619b0 2024-09-19T07:45:11.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.913+0000 7f3ce9082640 1 --2- 172.21.15.16:0/4155098631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3cc806c2b0 0x7f3cc806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:11.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.915+0000 7f3ce9082640 1 --2- 172.21.15.16:0/4155098631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3cc806c2b0 0x7f3cc806e770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009ac0 tx=0x7f3ce0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:11.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:11.917+0000 7f3cd9ffb640 1 -- 172.21.15.16:0/4155098631 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3cd407a390 con 0x7f3ce41619b0 2024-09-19T07:45:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:12.133+0000 7f3ceb2e6640 1 -- 172.21.15.16:0/4155098631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3ce4070720 con 0x7f3ce41619b0 2024-09-19T07:45:12.430 INFO:tasks.ceph.mds.c.smithi016.stderr:2024-09-19T07:45:12.430+0000 7efdd0b0b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T07:45:13.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.114+0000 7f3cd9ffb640 1 -- 172.21.15.16:0/4155098631 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) ==== 118+0+0 (secure 0 0 0) 0x7f3cd4079c90 con 0x7f3ce41619b0 2024-09-19T07:45:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.117+0000 7f3ceb2e6640 1 -- 172.21.15.16:0/4155098631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3cc806c2b0 msgr2=0x7f3cc806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.117+0000 7f3ceb2e6640 1 --2- 172.21.15.16:0/4155098631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3cc806c2b0 0x7f3cc806e770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009ac0 tx=0x7f3ce0008040 comp rx=0 tx=0).stop 2024-09-19T07:45:13.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.117+0000 7f3ceb2e6640 1 -- 172.21.15.16:0/4155098631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ce41619b0 msgr2=0x7f3ce4157830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:13.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.117+0000 7f3ceb2e6640 1 --2- 172.21.15.16:0/4155098631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ce41619b0 0x7f3ce4157830 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4000c00 tx=0x7f3cd40046c0 comp rx=0 tx=0).stop 2024-09-19T07:45:13.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.117+0000 7f3ceb2e6640 1 -- 172.21.15.16:0/4155098631 shutdown_connections 2024-09-19T07:45:13.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.117+0000 7f3ceb2e6640 1 --2- 172.21.15.16:0/4155098631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3cc806c2b0 0x7f3cc806e770 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:13.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.117+0000 7f3ceb2e6640 1 --2- 172.21.15.16:0/4155098631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ce4161d90 0x7f3ce40b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:13.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.117+0000 7f3ceb2e6640 1 --2- 172.21.15.16:0/4155098631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ce41619b0 0x7f3ce4157830 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:13.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.117+0000 7f3ceb2e6640 1 --2- 172.21.15.16:0/4155098631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ce40b0a80 0x7f3ce40c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:13.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.117+0000 7f3ceb2e6640 1 -- 172.21.15.16:0/4155098631 >> 172.21.15.16:0/4155098631 conn(0x7f3ce409a9f0 msgr2=0x7f3ce4160b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:13.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.117+0000 7f3ceb2e6640 1 -- 172.21.15.16:0/4155098631 shutdown_connections 2024-09-19T07:45:13.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.117+0000 7f3ceb2e6640 1 -- 172.21.15.16:0/4155098631 wait complete. 2024-09-19T07:45:13.130 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T07:45:13.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T07:45:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.242+0000 7fa0e67fd640 1 Processor -- start 2024-09-19T07:45:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.242+0000 7fa0e67fd640 1 -- start start 2024-09-19T07:45:13.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.243+0000 7fa0e67fd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa0e015f0d0 0x7fa0e015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:13.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.243+0000 7fa0e67fd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa0e00d0070 0x7fa0e00d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:13.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.243+0000 7fa0e67fd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0e00cf2b0 con 0x7fa0e00d1940 2024-09-19T07:45:13.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.243+0000 7fa0e67fd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa0e00cf590 con 0x7fa0e015f0d0 2024-09-19T07:45:13.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.243+0000 7fa0e67fd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa0e00cf870 con 0x7fa0e00d0070 2024-09-19T07:45:13.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.243+0000 7fa0dffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa0e015f0d0 0x7fa0e015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:13.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.243+0000 7fa0dffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa0e015f0d0 0x7fa0e015f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46878/0 (socket says 172.21.15.16:46878) 2024-09-19T07:45:13.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.243+0000 7fa0dffff640 1 -- 172.21.15.16:0/278969012 learned_addr learned my addr 172.21.15.16:0/278969012 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:13.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.243+0000 7fa0df7fe640 1 --2- 172.21.15.16:0/278969012 >> v2:172.21.15.146:3300/0 conn(0x7fa0e00d0070 0x7fa0e00d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:13.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.243+0000 7fa0dffff640 1 -- 172.21.15.16:0/278969012 >> v2:172.21.15.146:3300/0 conn(0x7fa0e00d0070 msgr2=0x7fa0e00d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:13.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.243+0000 7fa0dffff640 1 --2- 172.21.15.16:0/278969012 >> v2:172.21.15.146:3300/0 conn(0x7fa0e00d0070 0x7fa0e00d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:13.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.243+0000 7fa0dffff640 1 -- 172.21.15.16:0/278969012 >> v1:172.21.15.146:6789/0 conn(0x7fa0e00d1940 legacy=0x7fa0e00d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:13.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.243+0000 7fa0dffff640 1 -- 172.21.15.16:0/278969012 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e00cfb50 con 0x7fa0e015f0d0 2024-09-19T07:45:13.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.243+0000 7fa0df7fe640 1 --2- 172.21.15.16:0/278969012 >> v2:172.21.15.146:3300/0 conn(0x7fa0e00d0070 0x7fa0e00d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:13.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.244+0000 7fa0dffff640 1 --2- 172.21.15.16:0/278969012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0e015f0d0 0x7fa0e015f4b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00b6f40 tx=0x7fa0d002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779e9c943dfb035f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:13.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.244+0000 7fa0deffd640 1 -- 172.21.15.16:0/278969012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0037070 con 0x7fa0e015f0d0 2024-09-19T07:45:13.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.244+0000 7fa0deffd640 1 -- 172.21.15.16:0/278969012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0d0032440 con 0x7fa0e015f0d0 2024-09-19T07:45:13.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.244+0000 7fa0deffd640 1 -- 172.21.15.16:0/278969012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0032810 con 0x7fa0e015f0d0 2024-09-19T07:45:13.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.245+0000 7fa0e67fd640 1 -- 172.21.15.16:0/278969012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0e015f0d0 msgr2=0x7fa0e015f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.245+0000 7fa0e67fd640 1 --2- 172.21.15.16:0/278969012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0e015f0d0 0x7fa0e015f4b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00b6f40 tx=0x7fa0d002a180 comp rx=0 tx=0).stop 2024-09-19T07:45:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.245+0000 7fa0e67fd640 1 -- 172.21.15.16:0/278969012 shutdown_connections 2024-09-19T07:45:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.245+0000 7fa0e67fd640 1 --2- 172.21.15.16:0/278969012 >> v2:172.21.15.146:3300/0 conn(0x7fa0e00d0070 0x7fa0e00d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.245+0000 7fa0e67fd640 1 --2- 172.21.15.16:0/278969012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0e015f0d0 0x7fa0e015f4b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.245+0000 7fa0e67fd640 1 -- 172.21.15.16:0/278969012 >> 172.21.15.16:0/278969012 conn(0x7fa0e009d760 msgr2=0x7fa0e008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.245+0000 7fa0e67fd640 1 -- 172.21.15.16:0/278969012 shutdown_connections 2024-09-19T07:45:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.245+0000 7fa0e67fd640 1 -- 172.21.15.16:0/278969012 wait complete. 2024-09-19T07:45:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.246+0000 7fa0e67fd640 1 Processor -- start 2024-09-19T07:45:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.246+0000 7fa0e67fd640 1 -- start start 2024-09-19T07:45:13.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.247+0000 7fa0e67fd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0e00d0070 0x7fa0e00b9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:13.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.247+0000 7fa0e67fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0e00d1940 0x7fa0e015c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:13.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.247+0000 7fa0e67fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0e015f0d0 0x7fa0e00c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:13.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.247+0000 7fa0e67fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e00d49c0 con 0x7fa0e00d1940 2024-09-19T07:45:13.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.247+0000 7fa0e67fd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e00d4840 con 0x7fa0e00d0070 2024-09-19T07:45:13.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.247+0000 7fa0e67fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e00d4b40 con 0x7fa0e015f0d0 2024-09-19T07:45:13.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.247+0000 7fa0df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0e00d1940 0x7fa0e015c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:13.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.247+0000 7fa0df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0e00d1940 0x7fa0e015c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46890/0 (socket says 172.21.15.16:46890) 2024-09-19T07:45:13.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.247+0000 7fa0df7fe640 1 -- 172.21.15.16:0/798479316 learned_addr learned my addr 172.21.15.16:0/798479316 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:13.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.247+0000 7fa0dffff640 1 --2- 172.21.15.16:0/798479316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0e00d0070 0x7fa0e00b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:13.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.247+0000 7fa0e4d9a640 1 --2- 172.21.15.16:0/798479316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0e015f0d0 0x7fa0e00c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:13.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.247+0000 7fa0df7fe640 1 -- 172.21.15.16:0/798479316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0e015f0d0 msgr2=0x7fa0e00c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:13.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.247+0000 7fa0df7fe640 1 --2- 172.21.15.16:0/798479316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0e015f0d0 0x7fa0e00c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:13.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.247+0000 7fa0df7fe640 1 -- 172.21.15.16:0/798479316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0e00d0070 msgr2=0x7fa0e00b9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:13.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.247+0000 7fa0df7fe640 1 --2- 172.21.15.16:0/798479316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0e00d0070 0x7fa0e00b9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:13.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.247+0000 7fa0df7fe640 1 -- 172.21.15.16:0/798479316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e01f53c0 con 0x7fa0e00d1940 2024-09-19T07:45:13.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.247+0000 7fa0df7fe640 1 --2- 172.21.15.16:0/798479316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0e00d1940 0x7fa0e015c3a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400c7f0 tx=0x7fa0d400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:13.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.248+0000 7fa0dcff9640 1 -- 172.21.15.16:0/798479316 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4015070 con 0x7fa0e00d1940 2024-09-19T07:45:13.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.248+0000 7fa0dcff9640 1 -- 172.21.15.16:0/798479316 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0d40079d0 con 0x7fa0e00d1940 2024-09-19T07:45:13.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.248+0000 7fa0dcff9640 1 -- 172.21.15.16:0/798479316 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4007da0 con 0x7fa0e00d1940 2024-09-19T07:45:13.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.248+0000 7fa0e67fd640 1 -- 172.21.15.16:0/798479316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e00cde70 con 0x7fa0e00d1940 2024-09-19T07:45:13.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.248+0000 7fa0e67fd640 1 -- 172.21.15.16:0/798479316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e00ce430 con 0x7fa0e00d1940 2024-09-19T07:45:13.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.249+0000 7fa0e67fd640 1 -- 172.21.15.16:0/798479316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0e0075710 con 0x7fa0e00d1940 2024-09-19T07:45:13.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.249+0000 7fa0dcff9640 1 -- 172.21.15.16:0/798479316 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa0d4018070 con 0x7fa0e00d1940 2024-09-19T07:45:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.250+0000 7fa0dcff9640 1 --2- 172.21.15.16:0/798479316 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa0cc06c2e0 0x7fa0cc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.250+0000 7fa0dcff9640 1 -- 172.21.15.16:0/798479316 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fa0d408b410 con 0x7fa0e00d1940 2024-09-19T07:45:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.250+0000 7fa0dffff640 1 --2- 172.21.15.16:0/798479316 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa0cc06c2e0 0x7fa0cc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.251+0000 7fa0dffff640 1 --2- 172.21.15.16:0/798479316 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa0cc06c2e0 0x7fa0cc06e7a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0034000 tx=0x7fa0d002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:13.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.255+0000 7fa0dcff9640 1 -- 172.21.15.16:0/798479316 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa0e0075710 con 0x7fa0e00d1940 2024-09-19T07:45:13.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:13.466+0000 7fa0e67fd640 1 -- 172.21.15.16:0/798479316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa0e006cf30 con 0x7fa0e00d1940 2024-09-19T07:45:14.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.151+0000 7fa0dcff9640 1 -- 172.21.15.16:0/798479316 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7fa0d4059390 con 0x7fa0e00d1940 2024-09-19T07:45:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.153+0000 7fa0e67fd640 1 -- 172.21.15.16:0/798479316 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa0cc06c2e0 msgr2=0x7fa0cc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.153+0000 7fa0e67fd640 1 --2- 172.21.15.16:0/798479316 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa0cc06c2e0 0x7fa0cc06e7a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0034000 tx=0x7fa0d002e000 comp rx=0 tx=0).stop 2024-09-19T07:45:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.153+0000 7fa0e67fd640 1 -- 172.21.15.16:0/798479316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0e00d1940 msgr2=0x7fa0e015c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.153+0000 7fa0e67fd640 1 --2- 172.21.15.16:0/798479316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0e00d1940 0x7fa0e015c3a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400c7f0 tx=0x7fa0d400cbf0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.154+0000 7fa0e67fd640 1 -- 172.21.15.16:0/798479316 shutdown_connections 2024-09-19T07:45:14.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.154+0000 7fa0e67fd640 1 --2- 172.21.15.16:0/798479316 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa0cc06c2e0 0x7fa0cc06e7a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.154+0000 7fa0e67fd640 1 --2- 172.21.15.16:0/798479316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0e015f0d0 0x7fa0e00c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.154+0000 7fa0e67fd640 1 --2- 172.21.15.16:0/798479316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0e00d1940 0x7fa0e015c3a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.154+0000 7fa0e67fd640 1 --2- 172.21.15.16:0/798479316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0e00d0070 0x7fa0e00b9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.154+0000 7fa0e67fd640 1 -- 172.21.15.16:0/798479316 >> 172.21.15.16:0/798479316 conn(0x7fa0e009d760 msgr2=0x7fa0e006e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:14.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.154+0000 7fa0e67fd640 1 -- 172.21.15.16:0/798479316 shutdown_connections 2024-09-19T07:45:14.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.154+0000 7fa0e67fd640 1 -- 172.21.15.16:0/798479316 wait complete. 2024-09-19T07:45:14.167 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T07:45:14.168 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:45:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.276+0000 7f8e89c77640 1 Processor -- start 2024-09-19T07:45:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.276+0000 7f8e89c77640 1 -- start start 2024-09-19T07:45:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.277+0000 7f8e89c77640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8e840d21f0 0x7f8e840d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.277+0000 7f8e89c77640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8e84072590 0x7f8e8406d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.277+0000 7f8e89c77640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e840d2b10 con 0x7f8e84143a60 2024-09-19T07:45:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.277+0000 7f8e89c77640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f8e84141750 con 0x7f8e840d21f0 2024-09-19T07:45:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.277+0000 7f8e89c77640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f8e841620b0 con 0x7f8e84072590 2024-09-19T07:45:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.277+0000 7f8e837fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8e840d21f0 0x7f8e840d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.277+0000 7f8e837fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8e840d21f0 0x7f8e840d25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46896/0 (socket says 172.21.15.16:46896) 2024-09-19T07:45:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.277+0000 7f8e837fe640 1 -- 172.21.15.16:0/97489137 learned_addr learned my addr 172.21.15.16:0/97489137 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.277+0000 7f8e82ffd640 1 --2- 172.21.15.16:0/97489137 >> v2:172.21.15.146:3300/0 conn(0x7f8e84072590 0x7f8e8406d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.277+0000 7f8e837fe640 1 -- 172.21.15.16:0/97489137 >> v2:172.21.15.146:3300/0 conn(0x7f8e84072590 msgr2=0x7f8e8406d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:14.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.277+0000 7f8e837fe640 1 --2- 172.21.15.16:0/97489137 >> v2:172.21.15.146:3300/0 conn(0x7f8e84072590 0x7f8e8406d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.277+0000 7f8e837fe640 1 -- 172.21.15.16:0/97489137 >> v1:172.21.15.146:6789/0 conn(0x7f8e84143a60 legacy=0x7f8e84143f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:14.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.277+0000 7f8e837fe640 1 -- 172.21.15.16:0/97489137 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e84162370 con 0x7f8e840d21f0 2024-09-19T07:45:14.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.278+0000 7f8e837fe640 1 --2- 172.21.15.16:0/97489137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e840d21f0 0x7f8e840d25d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8e840b8490 tx=0x7f8e7402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c761b434a41e9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:14.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.278+0000 7f8e827fc640 1 -- 172.21.15.16:0/97489137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74037070 con 0x7f8e840d21f0 2024-09-19T07:45:14.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.278+0000 7f8e827fc640 1 -- 172.21.15.16:0/97489137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e74032440 con 0x7f8e840d21f0 2024-09-19T07:45:14.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.278+0000 7f8e827fc640 1 -- 172.21.15.16:0/97489137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74032810 con 0x7f8e840d21f0 2024-09-19T07:45:14.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.279+0000 7f8e89c77640 1 -- 172.21.15.16:0/97489137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e840d21f0 msgr2=0x7f8e840d25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:14.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.279+0000 7f8e89c77640 1 --2- 172.21.15.16:0/97489137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e840d21f0 0x7f8e840d25d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8e840b8490 tx=0x7f8e7402a1a0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.279+0000 7f8e89c77640 1 -- 172.21.15.16:0/97489137 shutdown_connections 2024-09-19T07:45:14.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.279+0000 7f8e89c77640 1 --2- 172.21.15.16:0/97489137 >> v2:172.21.15.146:3300/0 conn(0x7f8e84072590 0x7f8e8406d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.279+0000 7f8e89c77640 1 --2- 172.21.15.16:0/97489137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e840d21f0 0x7f8e840d25d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.279+0000 7f8e89c77640 1 -- 172.21.15.16:0/97489137 >> 172.21.15.16:0/97489137 conn(0x7f8e840b3ed0 msgr2=0x7f8e840b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:14.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.279+0000 7f8e89c77640 1 -- 172.21.15.16:0/97489137 shutdown_connections 2024-09-19T07:45:14.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.279+0000 7f8e89c77640 1 -- 172.21.15.16:0/97489137 wait complete. 2024-09-19T07:45:14.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.280+0000 7f8e89c77640 1 Processor -- start 2024-09-19T07:45:14.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.280+0000 7f8e89c77640 1 -- start start 2024-09-19T07:45:14.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.281+0000 7f8e89c77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e84072590 0x7f8e8413d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.281+0000 7f8e89c77640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e840d21f0 0x7f8e8413df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.281+0000 7f8e89c77640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e84143a60 0x7f8e840b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.281+0000 7f8e89c77640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8e84162c10 con 0x7f8e84072590 2024-09-19T07:45:14.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.281+0000 7f8e89c77640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8e84162930 con 0x7f8e840d21f0 2024-09-19T07:45:14.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.281+0000 7f8e89c77640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8e8406d620 con 0x7f8e84143a60 2024-09-19T07:45:14.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.281+0000 7f8e837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e84072590 0x7f8e8413d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.281+0000 7f8e837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e84072590 0x7f8e8413d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46904/0 (socket says 172.21.15.16:46904) 2024-09-19T07:45:14.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.281+0000 7f8e837fe640 1 -- 172.21.15.16:0/2137549661 learned_addr learned my addr 172.21.15.16:0/2137549661 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:14.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.281+0000 7f8e82ffd640 1 --2- 172.21.15.16:0/2137549661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e840d21f0 0x7f8e8413df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.281+0000 7f8e83fff640 1 --2- 172.21.15.16:0/2137549661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e84143a60 0x7f8e840b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.281+0000 7f8e837fe640 1 -- 172.21.15.16:0/2137549661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e84143a60 msgr2=0x7f8e840b6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:14.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.281+0000 7f8e837fe640 1 --2- 172.21.15.16:0/2137549661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e84143a60 0x7f8e840b6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.282+0000 7f8e837fe640 1 -- 172.21.15.16:0/2137549661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e840d21f0 msgr2=0x7f8e8413df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:14.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.282+0000 7f8e837fe640 1 --2- 172.21.15.16:0/2137549661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e840d21f0 0x7f8e8413df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.282+0000 7f8e837fe640 1 -- 172.21.15.16:0/2137549661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e840b7900 con 0x7f8e84072590 2024-09-19T07:45:14.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.282+0000 7f8e837fe640 1 --2- 172.21.15.16:0/2137549661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e84072590 0x7f8e8413d9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8e74000c00 tx=0x7f8e740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:14.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.282+0000 7f8e6bfff640 1 -- 172.21.15.16:0/2137549661 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74041020 con 0x7f8e84072590 2024-09-19T07:45:14.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.282+0000 7f8e6bfff640 1 -- 172.21.15.16:0/2137549661 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e74030970 con 0x7f8e84072590 2024-09-19T07:45:14.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.282+0000 7f8e89c77640 1 -- 172.21.15.16:0/2137549661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e840b7be0 con 0x7f8e84072590 2024-09-19T07:45:14.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.282+0000 7f8e6bfff640 1 -- 172.21.15.16:0/2137549661 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74037040 con 0x7f8e84072590 2024-09-19T07:45:14.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.282+0000 7f8e89c77640 1 -- 172.21.15.16:0/2137549661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8413b550 con 0x7f8e84072590 2024-09-19T07:45:14.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.283+0000 7f8e89c77640 1 -- 172.21.15.16:0/2137549661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e84158c30 con 0x7f8e84072590 2024-09-19T07:45:14.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.283+0000 7f8e6bfff640 1 -- 172.21.15.16:0/2137549661 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8e74044020 con 0x7f8e84072590 2024-09-19T07:45:14.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.284+0000 7f8e6bfff640 1 --2- 172.21.15.16:0/2137549661 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8e7006c2b0 0x7f8e7006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.284+0000 7f8e6bfff640 1 -- 172.21.15.16:0/2137549661 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f8e740abf70 con 0x7f8e84072590 2024-09-19T07:45:14.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.284+0000 7f8e82ffd640 1 --2- 172.21.15.16:0/2137549661 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8e7006c2b0 0x7f8e7006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.285+0000 7f8e82ffd640 1 --2- 172.21.15.16:0/2137549661 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8e7006c2b0 0x7f8e7006e770 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8e8415c3a0 tx=0x7f8e78008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:14.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.289+0000 7f8e6bfff640 1 -- 172.21.15.16:0/2137549661 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8e7404b030 con 0x7f8e84072590 2024-09-19T07:45:14.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.499+0000 7f8e89c77640 1 -- 172.21.15.16:0/2137549661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e84072290 con 0x7f8e84072590 2024-09-19T07:45:14.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.499+0000 7f8e6bfff640 1 -- 172.21.15.16:0/2137549661 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5223 (secure 0 0 0) 0x7f8e7407a5e0 con 0x7f8e84072590 2024-09-19T07:45:14.500 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:45:14.500 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-09-19T07:45:14:150420+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/3536366465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3536366465},{"type":"v1","addr":"172.21.15.146:6836","nonce":3536366465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3612374586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3612374586},{"type":"v1","addr":"172.21.15.146:6837","nonce":3612374586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5337,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/781070730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":781070730},{"type":"v1","addr":"172.21.15.16:6833","nonce":781070730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:45:10.494498+0000","modified":"2024-09-19T07:45:13.468183+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5340":{"gid":5340,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/642332950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":642332950},{"type":"v1","addr":"172.21.15.16:6835","nonce":642332950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5340,"qdb_cluster":[5340]},"id":3}]} 2024-09-19T07:45:14.500 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-09-19T07:45:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.501+0000 7f8e89c77640 1 -- 172.21.15.16:0/2137549661 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8e7006c2b0 msgr2=0x7f8e7006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.501+0000 7f8e89c77640 1 --2- 172.21.15.16:0/2137549661 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8e7006c2b0 0x7f8e7006e770 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8e8415c3a0 tx=0x7f8e78008040 comp rx=0 tx=0).stop 2024-09-19T07:45:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.501+0000 7f8e89c77640 1 -- 172.21.15.16:0/2137549661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e84072590 msgr2=0x7f8e8413d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.501+0000 7f8e89c77640 1 --2- 172.21.15.16:0/2137549661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e84072590 0x7f8e8413d9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8e74000c00 tx=0x7f8e740046c0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.501+0000 7f8e89c77640 1 -- 172.21.15.16:0/2137549661 shutdown_connections 2024-09-19T07:45:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.501+0000 7f8e89c77640 1 --2- 172.21.15.16:0/2137549661 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8e7006c2b0 0x7f8e7006e770 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.501+0000 7f8e89c77640 1 --2- 172.21.15.16:0/2137549661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e84143a60 0x7f8e840b6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.501+0000 7f8e89c77640 1 --2- 172.21.15.16:0/2137549661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e840d21f0 0x7f8e8413df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.501+0000 7f8e89c77640 1 --2- 172.21.15.16:0/2137549661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e84072590 0x7f8e8413d9c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.502+0000 7f8e89c77640 1 -- 172.21.15.16:0/2137549661 >> 172.21.15.16:0/2137549661 conn(0x7f8e840b3ed0 msgr2=0x7f8e840b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.502+0000 7f8e89c77640 1 -- 172.21.15.16:0/2137549661 shutdown_connections 2024-09-19T07:45:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.502+0000 7f8e89c77640 1 -- 172.21.15.16:0/2137549661 wait complete. 2024-09-19T07:45:14.513 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:45:14.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.614+0000 7fb6ea477640 1 Processor -- start 2024-09-19T07:45:14.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.614+0000 7fb6ea477640 1 -- start start 2024-09-19T07:45:14.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.615+0000 7fb6ea477640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6e415e2a0 0x7fb6e406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.615+0000 7fb6ea477640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb6e415e680 0x7fb6e40cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.615+0000 7fb6ea477640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6e406d960 0x7fb6e40d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.615+0000 7fb6ea477640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb6e40d1880 con 0x7fb6e415e2a0 2024-09-19T07:45:14.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.615+0000 7fb6ea477640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb6e40d1b60 con 0x7fb6e415e680 2024-09-19T07:45:14.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.615+0000 7fb6ea477640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb6e40cf2d0 con 0x7fb6e406d960 2024-09-19T07:45:14.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.615+0000 7fb6e3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6e415e2a0 0x7fb6e406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.615+0000 7fb6e3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6e415e2a0 0x7fb6e406d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46912/0 (socket says 172.21.15.16:46912) 2024-09-19T07:45:14.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.615+0000 7fb6e3fff640 1 -- 172.21.15.16:0/3014875873 learned_addr learned my addr 172.21.15.16:0/3014875873 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:14.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.616+0000 7fb6e8a14640 1 --2- 172.21.15.16:0/3014875873 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6e406d960 0x7fb6e40d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.616+0000 7fb6e37fe640 1 --2- 172.21.15.16:0/3014875873 >> v2:172.21.15.146:3300/0 conn(0x7fb6e415e680 0x7fb6e40cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.616+0000 7fb6e3fff640 1 -- 172.21.15.16:0/3014875873 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6e406d960 msgr2=0x7fb6e40d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:14.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.616+0000 7fb6e3fff640 1 --2- 172.21.15.16:0/3014875873 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6e406d960 0x7fb6e40d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.616+0000 7fb6e3fff640 1 -- 172.21.15.16:0/3014875873 >> v2:172.21.15.146:3300/0 conn(0x7fb6e415e680 msgr2=0x7fb6e40cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:14.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.616+0000 7fb6e3fff640 1 --2- 172.21.15.16:0/3014875873 >> v2:172.21.15.146:3300/0 conn(0x7fb6e415e680 0x7fb6e40cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.616+0000 7fb6e3fff640 1 -- 172.21.15.16:0/3014875873 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e40cf5b0 con 0x7fb6e415e2a0 2024-09-19T07:45:14.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.616+0000 7fb6e37fe640 1 --2- 172.21.15.16:0/3014875873 >> v2:172.21.15.146:3300/0 conn(0x7fb6e415e680 0x7fb6e40cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:14.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.616+0000 7fb6e3fff640 1 --2- 172.21.15.16:0/3014875873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6e415e2a0 0x7fb6e406d420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb6e40a22a0 tx=0x7fb6d402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fbb0ccb23d7c9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:14.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.617+0000 7fb6e2ffd640 1 -- 172.21.15.16:0/3014875873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4037070 con 0x7fb6e415e2a0 2024-09-19T07:45:14.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.617+0000 7fb6e2ffd640 1 -- 172.21.15.16:0/3014875873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d4032440 con 0x7fb6e415e2a0 2024-09-19T07:45:14.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.617+0000 7fb6e2ffd640 1 -- 172.21.15.16:0/3014875873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4032810 con 0x7fb6e415e2a0 2024-09-19T07:45:14.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.617+0000 7fb6ea477640 1 -- 172.21.15.16:0/3014875873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6e415e2a0 msgr2=0x7fb6e406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:14.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.617+0000 7fb6ea477640 1 --2- 172.21.15.16:0/3014875873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6e415e2a0 0x7fb6e406d420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb6e40a22a0 tx=0x7fb6d402a1e0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.617+0000 7fb6ea477640 1 -- 172.21.15.16:0/3014875873 shutdown_connections 2024-09-19T07:45:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.617+0000 7fb6ea477640 1 --2- 172.21.15.16:0/3014875873 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6e406d960 0x7fb6e40d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.617+0000 7fb6ea477640 1 --2- 172.21.15.16:0/3014875873 >> v2:172.21.15.146:3300/0 conn(0x7fb6e415e680 0x7fb6e40cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.617+0000 7fb6ea477640 1 --2- 172.21.15.16:0/3014875873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6e415e2a0 0x7fb6e406d420 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.617+0000 7fb6ea477640 1 -- 172.21.15.16:0/3014875873 >> 172.21.15.16:0/3014875873 conn(0x7fb6e409d760 msgr2=0x7fb6e408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.617+0000 7fb6ea477640 1 -- 172.21.15.16:0/3014875873 shutdown_connections 2024-09-19T07:45:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.617+0000 7fb6ea477640 1 -- 172.21.15.16:0/3014875873 wait complete. 2024-09-19T07:45:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.618+0000 7fb6ea477640 1 Processor -- start 2024-09-19T07:45:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.619+0000 7fb6ea477640 1 -- start start 2024-09-19T07:45:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.619+0000 7fb6ea477640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6e406d960 0x7fb6e415cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.619+0000 7fb6ea477640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6e415e2a0 0x7fb6e4156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.619+0000 7fb6ea477640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6e415e680 0x7fb6e40cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.619+0000 7fb6ea477640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e40d44f0 con 0x7fb6e415e2a0 2024-09-19T07:45:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.619+0000 7fb6ea477640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e40d4370 con 0x7fb6e415e680 2024-09-19T07:45:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.619+0000 7fb6ea477640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb6e40d4670 con 0x7fb6e406d960 2024-09-19T07:45:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.619+0000 7fb6e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6e415e2a0 0x7fb6e4156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.619+0000 7fb6e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6e415e2a0 0x7fb6e4156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46926/0 (socket says 172.21.15.16:46926) 2024-09-19T07:45:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.619+0000 7fb6e37fe640 1 -- 172.21.15.16:0/181545119 learned_addr learned my addr 172.21.15.16:0/181545119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.620+0000 7fb6e3fff640 1 --2- 172.21.15.16:0/181545119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6e406d960 0x7fb6e415cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.620+0000 7fb6e8a14640 1 --2- 172.21.15.16:0/181545119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6e415e680 0x7fb6e40cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.620+0000 7fb6e37fe640 1 -- 172.21.15.16:0/181545119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6e406d960 msgr2=0x7fb6e415cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.620+0000 7fb6e37fe640 1 --2- 172.21.15.16:0/181545119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6e406d960 0x7fb6e415cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.620+0000 7fb6e37fe640 1 -- 172.21.15.16:0/181545119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6e415e680 msgr2=0x7fb6e40cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:14.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.620+0000 7fb6e37fe640 1 --2- 172.21.15.16:0/181545119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6e415e680 0x7fb6e40cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.620+0000 7fb6e37fe640 1 -- 172.21.15.16:0/181545119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e415d440 con 0x7fb6e415e2a0 2024-09-19T07:45:14.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.620+0000 7fb6e37fe640 1 --2- 172.21.15.16:0/181545119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6e415e2a0 0x7fb6e4156830 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800d800 tx=0x7fb6d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:14.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.620+0000 7fb6e0ff9640 1 -- 172.21.15.16:0/181545119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8004030 con 0x7fb6e415e2a0 2024-09-19T07:45:14.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.620+0000 7fb6e0ff9640 1 -- 172.21.15.16:0/181545119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d80041d0 con 0x7fb6e415e2a0 2024-09-19T07:45:14.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.620+0000 7fb6ea477640 1 -- 172.21.15.16:0/181545119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e415d720 con 0x7fb6e415e2a0 2024-09-19T07:45:14.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.620+0000 7fb6e0ff9640 1 -- 172.21.15.16:0/181545119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d800b6c0 con 0x7fb6e415e2a0 2024-09-19T07:45:14.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.620+0000 7fb6ea477640 1 -- 172.21.15.16:0/181545119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e40c80d0 con 0x7fb6e415e2a0 2024-09-19T07:45:14.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.621+0000 7fb6ea477640 1 -- 172.21.15.16:0/181545119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6e406d420 con 0x7fb6e415e2a0 2024-09-19T07:45:14.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.622+0000 7fb6e0ff9640 1 -- 172.21.15.16:0/181545119 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb6d800b860 con 0x7fb6e415e2a0 2024-09-19T07:45:14.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.622+0000 7fb6e0ff9640 1 --2- 172.21.15.16:0/181545119 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb6d006c2b0 0x7fb6d006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.622+0000 7fb6e0ff9640 1 -- 172.21.15.16:0/181545119 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fb6d808b620 con 0x7fb6e415e2a0 2024-09-19T07:45:14.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.622+0000 7fb6e3fff640 1 --2- 172.21.15.16:0/181545119 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb6d006c2b0 0x7fb6d006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.623+0000 7fb6e3fff640 1 --2- 172.21.15.16:0/181545119 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb6d006c2b0 0x7fb6d006e770 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4034000 tx=0x7fb6d402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:14.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.627+0000 7fb6e0ff9640 1 -- 172.21.15.16:0/181545119 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb6d80595b0 con 0x7fb6e415e2a0 2024-09-19T07:45:14.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.804+0000 7fb6ea477640 1 -- 172.21.15.16:0/181545119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6e406cf90 con 0x7fb6e415e2a0 2024-09-19T07:45:14.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.805+0000 7fb6e0ff9640 1 -- 172.21.15.16:0/181545119 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+17026 (secure 0 0 0) 0x7fb6d805d8b0 con 0x7fb6e415e2a0 2024-09-19T07:45:14.806 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:45:14.806 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":53,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:45:12.136654+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:45:08.455974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-19T07:45:09.476448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:45:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.807+0000 7fb6ea477640 1 -- 172.21.15.16:0/181545119 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb6d006c2b0 msgr2=0x7fb6d006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.807+0000 7fb6ea477640 1 --2- 172.21.15.16:0/181545119 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb6d006c2b0 0x7fb6d006e770 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4034000 tx=0x7fb6d402e000 comp rx=0 tx=0).stop 2024-09-19T07:45:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.807+0000 7fb6ea477640 1 -- 172.21.15.16:0/181545119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6e415e2a0 msgr2=0x7fb6e4156830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.807+0000 7fb6ea477640 1 --2- 172.21.15.16:0/181545119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6e415e2a0 0x7fb6e4156830 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800d800 tx=0x7fb6d800dcc0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.808+0000 7fb6ea477640 1 -- 172.21.15.16:0/181545119 shutdown_connections 2024-09-19T07:45:14.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.808+0000 7fb6ea477640 1 --2- 172.21.15.16:0/181545119 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb6d006c2b0 0x7fb6d006e770 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.808+0000 7fb6ea477640 1 --2- 172.21.15.16:0/181545119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6e415e680 0x7fb6e40cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.808+0000 7fb6ea477640 1 --2- 172.21.15.16:0/181545119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6e415e2a0 0x7fb6e4156830 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.808+0000 7fb6ea477640 1 --2- 172.21.15.16:0/181545119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6e406d960 0x7fb6e415cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.808+0000 7fb6ea477640 1 -- 172.21.15.16:0/181545119 >> 172.21.15.16:0/181545119 conn(0x7fb6e409d760 msgr2=0x7fb6e4160910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:14.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.808+0000 7fb6ea477640 1 -- 172.21.15.16:0/181545119 shutdown_connections 2024-09-19T07:45:14.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.808+0000 7fb6ea477640 1 -- 172.21.15.16:0/181545119 wait complete. 2024-09-19T07:45:14.819 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T07:45:14.820 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:45:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.921+0000 7faa2925e640 1 Processor -- start 2024-09-19T07:45:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.922+0000 7faa2925e640 1 -- start start 2024-09-19T07:45:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.922+0000 7faa2925e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa2415e680 0x7faa240cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.922+0000 7faa2925e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa240d1880 con 0x7faa2415e2a0 2024-09-19T07:45:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.922+0000 7faa2925e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa240d1b60 con 0x7faa240d0010 2024-09-19T07:45:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.922+0000 7faa2925e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faa240cf2d0 con 0x7faa2415e680 2024-09-19T07:45:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa22d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7faa2415e2a0 0x7faa2406d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50342/0 (socket says 172.21.15.16:50342) 2024-09-19T07:45:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa22d9d640 1 -- 172.21.15.16:0/2919609260 learned_addr learned my addr 172.21.15.16:0/2919609260 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa2259c640 1 --2- 172.21.15.16:0/2919609260 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa2415e680 0x7faa240cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa21d9b640 1 -- 172.21.15.16:0/2919609260 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3487769291 0 0) 0x7faa240d1880 con 0x7faa2415e2a0 2024-09-19T07:45:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa21d9b640 1 -- 172.21.15.16:0/2919609260 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa0c003520 con 0x7faa2415e2a0 2024-09-19T07:45:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa21d9b640 1 -- 172.21.15.16:0/2919609260 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 387369055 0 0) 0x7faa240d1b60 con 0x7faa240d0010 2024-09-19T07:45:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa21d9b640 1 -- 172.21.15.16:0/2919609260 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa240d1880 con 0x7faa240d0010 2024-09-19T07:45:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa21d9b640 1 -- 172.21.15.16:0/2919609260 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2707883853 0 0) 0x7faa0c003520 con 0x7faa2415e2a0 2024-09-19T07:45:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa21d9b640 1 -- 172.21.15.16:0/2919609260 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa2415e680 msgr2=0x7faa240cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa21d9b640 1 --2- 172.21.15.16:0/2919609260 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa2415e680 0x7faa240cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa21d9b640 1 -- 172.21.15.16:0/2919609260 >> v1:172.21.15.146:6789/0 conn(0x7faa240d0010 legacy=0x7faa240d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:14.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa21d9b640 1 -- 172.21.15.16:0/2919609260 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa240cf5b0 con 0x7faa2415e2a0 2024-09-19T07:45:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa21d9b640 1 -- 172.21.15.16:0/2919609260 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7faa14001670 con 0x7faa2415e2a0 2024-09-19T07:45:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa2259c640 1 --2- 172.21.15.16:0/2919609260 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa2415e680 0x7faa240cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa21d9b640 1 -- 172.21.15.16:0/2919609260 >> v1:172.21.15.16:6789/0 conn(0x7faa2415e2a0 legacy=0x7faa2406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa21d9b640 1 --2- 172.21.15.16:0/2919609260 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa2415e680 0x7faa0c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa21d9b640 1 --2- 172.21.15.16:0/2919609260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa0c0040b0 0x7faa0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa21d9b640 1 --2- 172.21.15.16:0/2919609260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c004d40 0x7faa0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa21d9b640 1 -- 172.21.15.16:0/2919609260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa240cf2d0 con 0x7faa0c004d40 2024-09-19T07:45:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa21d9b640 1 -- 172.21.15.16:0/2919609260 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c00bb00 con 0x7faa0c0040b0 2024-09-19T07:45:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa21d9b640 1 -- 172.21.15.16:0/2919609260 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faa0c00bd00 con 0x7faa2415e680 2024-09-19T07:45:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa2359e640 1 --2- 172.21.15.16:0/2919609260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c004d40 0x7faa0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa22d9d640 1 --2- 172.21.15.16:0/2919609260 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa2415e680 0x7faa0c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.923+0000 7faa2259c640 1 --2- 172.21.15.16:0/2919609260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa0c0040b0 0x7faa0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.924+0000 7faa2359e640 1 -- 172.21.15.16:0/2919609260 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa2415e680 msgr2=0x7faa0c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.924+0000 7faa2359e640 1 --2- 172.21.15.16:0/2919609260 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa2415e680 0x7faa0c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.924+0000 7faa2359e640 1 -- 172.21.15.16:0/2919609260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa0c0040b0 msgr2=0x7faa0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.924+0000 7faa2359e640 1 --2- 172.21.15.16:0/2919609260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa0c0040b0 0x7faa0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.924+0000 7faa2359e640 1 -- 172.21.15.16:0/2919609260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c003520 con 0x7faa0c004d40 2024-09-19T07:45:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.924+0000 7faa2359e640 1 --2- 172.21.15.16:0/2919609260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c004d40 0x7faa0c0051c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7faa1c001fd0 tx=0x7faa0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195ae0ded557791f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.925+0000 7faa21d9b640 1 -- 172.21.15.16:0/2919609260 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c001bf0 con 0x7faa0c004d40 2024-09-19T07:45:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.925+0000 7faa21d9b640 1 -- 172.21.15.16:0/2919609260 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa1c003b70 con 0x7faa0c004d40 2024-09-19T07:45:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.925+0000 7faa21d9b640 1 -- 172.21.15.16:0/2919609260 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0050d0 con 0x7faa0c004d40 2024-09-19T07:45:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.925+0000 7faa2925e640 1 -- 172.21.15.16:0/2919609260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c004d40 msgr2=0x7faa0c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.925+0000 7faa2925e640 1 --2- 172.21.15.16:0/2919609260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c004d40 0x7faa0c0051c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7faa1c001fd0 tx=0x7faa0c005830 comp rx=0 tx=0).stop 2024-09-19T07:45:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.925+0000 7faa2359e640 1 -- 172.21.15.16:0/2919609260 reap_dead start 2024-09-19T07:45:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.925+0000 7faa2925e640 1 -- 172.21.15.16:0/2919609260 shutdown_connections 2024-09-19T07:45:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.925+0000 7faa2925e640 1 -- 172.21.15.16:0/2919609260 >> 172.21.15.16:0/2919609260 conn(0x7faa2409d760 msgr2=0x7faa2408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.925+0000 7faa2925e640 1 -- 172.21.15.16:0/2919609260 shutdown_connections 2024-09-19T07:45:14.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.925+0000 7faa2925e640 1 -- 172.21.15.16:0/2919609260 wait complete. 2024-09-19T07:45:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.927+0000 7faa2925e640 1 Processor -- start 2024-09-19T07:45:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.927+0000 7faa2925e640 1 -- start start 2024-09-19T07:45:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.927+0000 7faa2925e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c004d40 0x7faa240cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.927+0000 7faa2925e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa240cdb50 0x7faa24154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.927+0000 7faa2925e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa241ea1e0 0x7faa241ea660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.927+0000 7faa2925e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa2406dcb0 con 0x7faa0c004d40 2024-09-19T07:45:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.927+0000 7faa2925e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faa2406db30 con 0x7faa240cdb50 2024-09-19T07:45:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.927+0000 7faa2925e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faa240d4a80 con 0x7faa241ea1e0 2024-09-19T07:45:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.928+0000 7faa22d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c004d40 0x7faa240cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.928+0000 7faa22d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c004d40 0x7faa240cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46948/0 (socket says 172.21.15.16:46948) 2024-09-19T07:45:14.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.928+0000 7faa22d9d640 1 -- 172.21.15.16:0/637083854 learned_addr learned my addr 172.21.15.16:0/637083854 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:14.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.928+0000 7faa2259c640 1 --2- 172.21.15.16:0/637083854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa240cdb50 0x7faa24154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.928+0000 7faa2359e640 1 --2- 172.21.15.16:0/637083854 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa241ea1e0 0x7faa241ea660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.928+0000 7faa22d9d640 1 -- 172.21.15.16:0/637083854 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa241ea1e0 msgr2=0x7faa241ea660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:14.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.928+0000 7faa22d9d640 1 --2- 172.21.15.16:0/637083854 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa241ea1e0 0x7faa241ea660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.928+0000 7faa22d9d640 1 -- 172.21.15.16:0/637083854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa240cdb50 msgr2=0x7faa24154980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:14.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.928+0000 7faa22d9d640 1 --2- 172.21.15.16:0/637083854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa240cdb50 0x7faa24154980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:14.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.928+0000 7faa22d9d640 1 -- 172.21.15.16:0/637083854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa240cc7a0 con 0x7faa0c004d40 2024-09-19T07:45:14.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.928+0000 7faa2259c640 1 --2- 172.21.15.16:0/637083854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa240cdb50 0x7faa24154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:14.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.928+0000 7faa22d9d640 1 --2- 172.21.15.16:0/637083854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c004d40 0x7faa240cc130 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7faa14007340 tx=0x7faa14007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:14.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.928+0000 7faa0b7fe640 1 -- 172.21.15.16:0/637083854 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1400f940 con 0x7faa0c004d40 2024-09-19T07:45:14.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.928+0000 7faa0b7fe640 1 -- 172.21.15.16:0/637083854 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa14018070 con 0x7faa0c004d40 2024-09-19T07:45:14.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.928+0000 7faa2925e640 1 -- 172.21.15.16:0/637083854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa240cca80 con 0x7faa0c004d40 2024-09-19T07:45:14.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.928+0000 7faa0b7fe640 1 -- 172.21.15.16:0/637083854 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1400fd60 con 0x7faa0c004d40 2024-09-19T07:45:14.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.928+0000 7faa2925e640 1 -- 172.21.15.16:0/637083854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa240cd040 con 0x7faa0c004d40 2024-09-19T07:45:14.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.929+0000 7faa2925e640 1 -- 172.21.15.16:0/637083854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa240cfad0 con 0x7faa0c004d40 2024-09-19T07:45:14.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.930+0000 7faa0b7fe640 1 -- 172.21.15.16:0/637083854 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7faa1401d460 con 0x7faa0c004d40 2024-09-19T07:45:14.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.930+0000 7faa0b7fe640 1 --2- 172.21.15.16:0/637083854 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faa1006c2e0 0x7faa1006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:14.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.931+0000 7faa0b7fe640 1 -- 172.21.15.16:0/637083854 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7faa1408b4d0 con 0x7faa0c004d40 2024-09-19T07:45:14.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.931+0000 7faa2259c640 1 --2- 172.21.15.16:0/637083854 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faa1006c2e0 0x7faa1006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:14.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.931+0000 7faa2259c640 1 --2- 172.21.15.16:0/637083854 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faa1006c2e0 0x7faa1006e7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faa240ce1e0 tx=0x7faa18009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:14.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:14.935+0000 7faa0b7fe640 1 -- 172.21.15.16:0/637083854 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7faa14059450 con 0x7faa0c004d40 2024-09-19T07:45:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.110+0000 7faa2925e640 1 -- 172.21.15.16:0/637083854 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faa240ce8a0 con 0x7faa1006c2e0 2024-09-19T07:45:15.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.117+0000 7faa0b7fe640 1 -- 172.21.15.16:0/637083854 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251954 (secure 0 0 0) 0x7faa240ce8a0 con 0x7faa1006c2e0 2024-09-19T07:45:15.121 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:45:15.123 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:45:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.120+0000 7faa2925e640 1 -- 172.21.15.16:0/637083854 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faa1006c2e0 msgr2=0x7faa1006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.120+0000 7faa2925e640 1 --2- 172.21.15.16:0/637083854 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faa1006c2e0 0x7faa1006e7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faa240ce1e0 tx=0x7faa18009190 comp rx=0 tx=0).stop 2024-09-19T07:45:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.120+0000 7faa2925e640 1 -- 172.21.15.16:0/637083854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c004d40 msgr2=0x7faa240cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.120+0000 7faa2925e640 1 --2- 172.21.15.16:0/637083854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c004d40 0x7faa240cc130 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7faa14007340 tx=0x7faa14007800 comp rx=0 tx=0).stop 2024-09-19T07:45:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.120+0000 7faa2925e640 1 -- 172.21.15.16:0/637083854 shutdown_connections 2024-09-19T07:45:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.120+0000 7faa2925e640 1 --2- 172.21.15.16:0/637083854 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faa1006c2e0 0x7faa1006e7a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.120+0000 7faa2925e640 1 --2- 172.21.15.16:0/637083854 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa241ea1e0 0x7faa241ea660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.120+0000 7faa2925e640 1 --2- 172.21.15.16:0/637083854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa240cdb50 0x7faa24154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.120+0000 7faa2925e640 1 --2- 172.21.15.16:0/637083854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c004d40 0x7faa240cc130 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.120+0000 7faa2925e640 1 -- 172.21.15.16:0/637083854 >> 172.21.15.16:0/637083854 conn(0x7faa2409d760 msgr2=0x7faa2415f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.120+0000 7faa2925e640 1 -- 172.21.15.16:0/637083854 shutdown_connections 2024-09-19T07:45:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.120+0000 7faa2925e640 1 -- 172.21.15.16:0/637083854 wait complete. 2024-09-19T07:45:15.131 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-09-19T07:45:14.488917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243104,"kb_used_data":6052,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731616,"statfs":{"total":773094113280,"available":772845174784,"internally_reserved":0,"allocated":6197248,"data_stored":3383541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.362871"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139720+0000","last_change":"2024-09-19T07:45:11.145009+0000","last_active":"2024-09-19T07:45:12.139720+0000","last_peered":"2024-09-19T07:45:12.139720+0000","last_clean":"2024-09-19T07:45:12.139720+0000","last_became_active":"2024-09-19T07:45:11.144715+0000","last_became_peered":"2024-09-19T07:45:11.144715+0000","last_unstale":"2024-09-19T07:45:12.139720+0000","last_undegraded":"2024-09-19T07:45:12.139720+0000","last_fullsized":"2024-09-19T07:45:12.139720+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:14.467600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.168262+0000","last_change":"2024-09-19T07:45:11.158808+0000","last_active":"2024-09-19T07:45:12.168262+0000","last_peered":"2024-09-19T07:45:12.168262+0000","last_clean":"2024-09-19T07:45:12.168262+0000","last_became_active":"2024-09-19T07:45:11.158288+0000","last_became_peered":"2024-09-19T07:45:11.158288+0000","last_unstale":"2024-09-19T07:45:12.168262+0000","last_undegraded":"2024-09-19T07:45:12.168262+0000","last_fullsized":"2024-09-19T07:45:12.168262+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:20.215194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164977+0000","last_change":"2024-09-19T07:45:11.168324+0000","last_active":"2024-09-19T07:45:12.164977+0000","last_peered":"2024-09-19T07:45:12.164977+0000","last_clean":"2024-09-19T07:45:12.164977+0000","last_became_active":"2024-09-19T07:45:11.166933+0000","last_became_peered":"2024-09-19T07:45:11.166933+0000","last_unstale":"2024-09-19T07:45:12.164977+0000","last_undegraded":"2024-09-19T07:45:12.164977+0000","last_fullsized":"2024-09-19T07:45:12.164977+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:20.215194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.168114+0000","last_change":"2024-09-19T07:45:11.160049+0000","last_active":"2024-09-19T07:45:12.168114+0000","last_peered":"2024-09-19T07:45:12.168114+0000","last_clean":"2024-09-19T07:45:12.168114+0000","last_became_active":"2024-09-19T07:45:11.159220+0000","last_became_peered":"2024-09-19T07:45:11.159220+0000","last_unstale":"2024-09-19T07:45:12.168114+0000","last_undegraded":"2024-09-19T07:45:12.168114+0000","last_fullsized":"2024-09-19T07:45:12.168114+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:27:55.022291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.797030+0000","last_change":"2024-09-19T07:45:11.156372+0000","last_active":"2024-09-19T07:45:12.797030+0000","last_peered":"2024-09-19T07:45:12.797030+0000","last_clean":"2024-09-19T07:45:12.797030+0000","last_became_active":"2024-09-19T07:45:11.155909+0000","last_became_peered":"2024-09-19T07:45:11.155909+0000","last_unstale":"2024-09-19T07:45:12.797030+0000","last_undegraded":"2024-09-19T07:45:12.797030+0000","last_fullsized":"2024-09-19T07:45:12.797030+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:50.962826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139519+0000","last_change":"2024-09-19T07:45:11.162738+0000","last_active":"2024-09-19T07:45:12.139519+0000","last_peered":"2024-09-19T07:45:12.139519+0000","last_clean":"2024-09-19T07:45:12.139519+0000","last_became_active":"2024-09-19T07:45:11.161804+0000","last_became_peered":"2024-09-19T07:45:11.161804+0000","last_unstale":"2024-09-19T07:45:12.139519+0000","last_undegraded":"2024-09-19T07:45:12.139519+0000","last_fullsized":"2024-09-19T07:45:12.139519+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:50.592195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164801+0000","last_change":"2024-09-19T07:45:11.160689+0000","last_active":"2024-09-19T07:45:12.164801+0000","last_peered":"2024-09-19T07:45:12.164801+0000","last_clean":"2024-09-19T07:45:12.164801+0000","last_became_active":"2024-09-19T07:45:11.160400+0000","last_became_peered":"2024-09-19T07:45:11.160400+0000","last_unstale":"2024-09-19T07:45:12.164801+0000","last_undegraded":"2024-09-19T07:45:12.164801+0000","last_fullsized":"2024-09-19T07:45:12.164801+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:48.149096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226265+0000","last_change":"2024-09-19T07:45:11.151579+0000","last_active":"2024-09-19T07:45:12.226265+0000","last_peered":"2024-09-19T07:45:12.226265+0000","last_clean":"2024-09-19T07:45:12.226265+0000","last_became_active":"2024-09-19T07:45:11.150845+0000","last_became_peered":"2024-09-19T07:45:11.150845+0000","last_unstale":"2024-09-19T07:45:12.226265+0000","last_undegraded":"2024-09-19T07:45:12.226265+0000","last_fullsized":"2024-09-19T07:45:12.226265+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:51.235254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226274+0000","last_change":"2024-09-19T07:45:11.173152+0000","last_active":"2024-09-19T07:45:12.226274+0000","last_peered":"2024-09-19T07:45:12.226274+0000","last_clean":"2024-09-19T07:45:12.226274+0000","last_became_active":"2024-09-19T07:45:11.172874+0000","last_became_peered":"2024-09-19T07:45:11.172874+0000","last_unstale":"2024-09-19T07:45:12.226274+0000","last_undegraded":"2024-09-19T07:45:12.226274+0000","last_fullsized":"2024-09-19T07:45:12.226274+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:02.608766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139884+0000","last_change":"2024-09-19T07:45:11.151003+0000","last_active":"2024-09-19T07:45:12.139884+0000","last_peered":"2024-09-19T07:45:12.139884+0000","last_clean":"2024-09-19T07:45:12.139884+0000","last_became_active":"2024-09-19T07:45:11.150660+0000","last_became_peered":"2024-09-19T07:45:11.150660+0000","last_unstale":"2024-09-19T07:45:12.139884+0000","last_undegraded":"2024-09-19T07:45:12.139884+0000","last_fullsized":"2024-09-19T07:45:12.139884+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:42.526073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-19T07:45:11.161675+0000","last_change":"2024-09-19T07:45:11.160870+0000","last_active":"2024-09-19T07:45:11.161675+0000","last_peered":"2024-09-19T07:45:11.161675+0000","last_clean":"2024-09-19T07:45:11.161675+0000","last_became_active":"2024-09-19T07:45:11.160605+0000","last_became_peered":"2024-09-19T07:45:11.160605+0000","last_unstale":"2024-09-19T07:45:11.161675+0000","last_undegraded":"2024-09-19T07:45:11.161675+0000","last_fullsized":"2024-09-19T07:45:11.161675+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:47.872850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226222+0000","last_change":"2024-09-19T07:45:11.153833+0000","last_active":"2024-09-19T07:45:12.226222+0000","last_peered":"2024-09-19T07:45:12.226222+0000","last_clean":"2024-09-19T07:45:12.226222+0000","last_became_active":"2024-09-19T07:45:11.152953+0000","last_became_peered":"2024-09-19T07:45:11.152953+0000","last_unstale":"2024-09-19T07:45:12.226222+0000","last_undegraded":"2024-09-19T07:45:12.226222+0000","last_fullsized":"2024-09-19T07:45:12.226222+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:23.640090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139608+0000","last_change":"2024-09-19T07:45:11.152517+0000","last_active":"2024-09-19T07:45:12.139608+0000","last_peered":"2024-09-19T07:45:12.139608+0000","last_clean":"2024-09-19T07:45:12.139608+0000","last_became_active":"2024-09-19T07:45:11.152197+0000","last_became_peered":"2024-09-19T07:45:11.152197+0000","last_unstale":"2024-09-19T07:45:12.139608+0000","last_undegraded":"2024-09-19T07:45:12.139608+0000","last_fullsized":"2024-09-19T07:45:12.139608+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:17.354756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164882+0000","last_change":"2024-09-19T07:45:11.170619+0000","last_active":"2024-09-19T07:45:12.164882+0000","last_peered":"2024-09-19T07:45:12.164882+0000","last_clean":"2024-09-19T07:45:12.164882+0000","last_became_active":"2024-09-19T07:45:11.169189+0000","last_became_peered":"2024-09-19T07:45:11.169189+0000","last_unstale":"2024-09-19T07:45:12.164882+0000","last_undegraded":"2024-09-19T07:45:12.164882+0000","last_fullsized":"2024-09-19T07:45:12.164882+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:35.156705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.167938+0000","last_change":"2024-09-19T07:45:11.159503+0000","last_active":"2024-09-19T07:45:12.167938+0000","last_peered":"2024-09-19T07:45:12.167938+0000","last_clean":"2024-09-19T07:45:12.167938+0000","last_became_active":"2024-09-19T07:45:11.158846+0000","last_became_peered":"2024-09-19T07:45:11.158846+0000","last_unstale":"2024-09-19T07:45:12.167938+0000","last_undegraded":"2024-09-19T07:45:12.167938+0000","last_fullsized":"2024-09-19T07:45:12.167938+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:17.366751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139402+0000","last_change":"2024-09-19T07:45:11.151033+0000","last_active":"2024-09-19T07:45:12.139402+0000","last_peered":"2024-09-19T07:45:12.139402+0000","last_clean":"2024-09-19T07:45:12.139402+0000","last_became_active":"2024-09-19T07:45:11.150684+0000","last_became_peered":"2024-09-19T07:45:11.150684+0000","last_unstale":"2024-09-19T07:45:12.139402+0000","last_undegraded":"2024-09-19T07:45:12.139402+0000","last_fullsized":"2024-09-19T07:45:12.139402+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:26:58.382817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.165376+0000","last_change":"2024-09-19T07:45:11.173239+0000","last_active":"2024-09-19T07:45:12.165376+0000","last_peered":"2024-09-19T07:45:12.165376+0000","last_clean":"2024-09-19T07:45:12.165376+0000","last_became_active":"2024-09-19T07:45:11.171542+0000","last_became_peered":"2024-09-19T07:45:11.171542+0000","last_unstale":"2024-09-19T07:45:12.165376+0000","last_undegraded":"2024-09-19T07:45:12.165376+0000","last_fullsized":"2024-09-19T07:45:12.165376+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:31.214285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.168254+0000","last_change":"2024-09-19T07:45:11.158729+0000","last_active":"2024-09-19T07:45:12.168254+0000","last_peered":"2024-09-19T07:45:12.168254+0000","last_clean":"2024-09-19T07:45:12.168254+0000","last_became_active":"2024-09-19T07:45:11.158441+0000","last_became_peered":"2024-09-19T07:45:11.158441+0000","last_unstale":"2024-09-19T07:45:12.168254+0000","last_undegraded":"2024-09-19T07:45:12.168254+0000","last_fullsized":"2024-09-19T07:45:12.168254+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:08:39.515448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.165003+0000","last_change":"2024-09-19T07:45:11.168043+0000","last_active":"2024-09-19T07:45:12.165003+0000","last_peered":"2024-09-19T07:45:12.165003+0000","last_clean":"2024-09-19T07:45:12.165003+0000","last_became_active":"2024-09-19T07:45:11.165051+0000","last_became_peered":"2024-09-19T07:45:11.165051+0000","last_unstale":"2024-09-19T07:45:12.165003+0000","last_undegraded":"2024-09-19T07:45:12.165003+0000","last_fullsized":"2024-09-19T07:45:12.165003+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:03.671841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143791+0000","last_change":"2024-09-19T07:45:11.151488+0000","last_active":"2024-09-19T07:45:12.143791+0000","last_peered":"2024-09-19T07:45:12.143791+0000","last_clean":"2024-09-19T07:45:12.143791+0000","last_became_active":"2024-09-19T07:45:11.150944+0000","last_became_peered":"2024-09-19T07:45:11.150944+0000","last_unstale":"2024-09-19T07:45:12.143791+0000","last_undegraded":"2024-09-19T07:45:12.143791+0000","last_fullsized":"2024-09-19T07:45:12.143791+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:50.962826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143666+0000","last_change":"2024-09-19T07:45:11.156079+0000","last_active":"2024-09-19T07:45:12.143666+0000","last_peered":"2024-09-19T07:45:12.143666+0000","last_clean":"2024-09-19T07:45:12.143666+0000","last_became_active":"2024-09-19T07:45:11.155798+0000","last_became_peered":"2024-09-19T07:45:11.155798+0000","last_unstale":"2024-09-19T07:45:12.143666+0000","last_undegraded":"2024-09-19T07:45:12.143666+0000","last_fullsized":"2024-09-19T07:45:12.143666+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:55.980609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.225887+0000","last_change":"2024-09-19T07:45:11.176023+0000","last_active":"2024-09-19T07:45:12.225887+0000","last_peered":"2024-09-19T07:45:12.225887+0000","last_clean":"2024-09-19T07:45:12.225887+0000","last_became_active":"2024-09-19T07:45:11.175693+0000","last_became_peered":"2024-09-19T07:45:11.175693+0000","last_unstale":"2024-09-19T07:45:12.225887+0000","last_undegraded":"2024-09-19T07:45:12.225887+0000","last_fullsized":"2024-09-19T07:45:12.225887+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:29:29.101102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-19T07:45:11.161502+0000","last_change":"2024-09-19T07:45:11.160207+0000","last_active":"2024-09-19T07:45:11.161502+0000","last_peered":"2024-09-19T07:45:11.161502+0000","last_clean":"2024-09-19T07:45:11.161502+0000","last_became_active":"2024-09-19T07:45:11.159891+0000","last_became_peered":"2024-09-19T07:45:11.159891+0000","last_unstale":"2024-09-19T07:45:11.161502+0000","last_undegraded":"2024-09-19T07:45:11.161502+0000","last_fullsized":"2024-09-19T07:45:11.161502+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:41.004093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164543+0000","last_change":"2024-09-19T07:45:11.159414+0000","last_active":"2024-09-19T07:45:12.164543+0000","last_peered":"2024-09-19T07:45:12.164543+0000","last_clean":"2024-09-19T07:45:12.164543+0000","last_became_active":"2024-09-19T07:45:11.159046+0000","last_became_peered":"2024-09-19T07:45:11.159046+0000","last_unstale":"2024-09-19T07:45:12.164543+0000","last_undegraded":"2024-09-19T07:45:12.164543+0000","last_fullsized":"2024-09-19T07:45:12.164543+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:09.128499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.165440+0000","last_change":"2024-09-19T07:45:11.172493+0000","last_active":"2024-09-19T07:45:12.165440+0000","last_peered":"2024-09-19T07:45:12.165440+0000","last_clean":"2024-09-19T07:45:12.165440+0000","last_became_active":"2024-09-19T07:45:11.171240+0000","last_became_peered":"2024-09-19T07:45:11.171240+0000","last_unstale":"2024-09-19T07:45:12.165440+0000","last_undegraded":"2024-09-19T07:45:12.165440+0000","last_fullsized":"2024-09-19T07:45:12.165440+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:18.448468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226078+0000","last_change":"2024-09-19T07:45:11.147813+0000","last_active":"2024-09-19T07:45:12.226078+0000","last_peered":"2024-09-19T07:45:12.226078+0000","last_clean":"2024-09-19T07:45:12.226078+0000","last_became_active":"2024-09-19T07:45:11.147493+0000","last_became_peered":"2024-09-19T07:45:11.147493+0000","last_unstale":"2024-09-19T07:45:12.226078+0000","last_undegraded":"2024-09-19T07:45:12.226078+0000","last_fullsized":"2024-09-19T07:45:12.226078+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:11.709408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139984+0000","last_change":"2024-09-19T07:45:11.166695+0000","last_active":"2024-09-19T07:45:12.139984+0000","last_peered":"2024-09-19T07:45:12.139984+0000","last_clean":"2024-09-19T07:45:12.139984+0000","last_became_active":"2024-09-19T07:45:11.166318+0000","last_became_peered":"2024-09-19T07:45:11.166318+0000","last_unstale":"2024-09-19T07:45:12.139984+0000","last_undegraded":"2024-09-19T07:45:12.139984+0000","last_fullsized":"2024-09-19T07:45:12.139984+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:40.863237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143846+0000","last_change":"2024-09-19T07:45:11.147522+0000","last_active":"2024-09-19T07:45:12.143846+0000","last_peered":"2024-09-19T07:45:12.143846+0000","last_clean":"2024-09-19T07:45:12.143846+0000","last_became_active":"2024-09-19T07:45:11.147239+0000","last_became_peered":"2024-09-19T07:45:11.147239+0000","last_unstale":"2024-09-19T07:45:12.143846+0000","last_undegraded":"2024-09-19T07:45:12.143846+0000","last_fullsized":"2024-09-19T07:45:12.143846+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:15.960700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164943+0000","last_change":"2024-09-19T07:45:11.168884+0000","last_active":"2024-09-19T07:45:12.164943+0000","last_peered":"2024-09-19T07:45:12.164943+0000","last_clean":"2024-09-19T07:45:12.164943+0000","last_became_active":"2024-09-19T07:45:11.168583+0000","last_became_peered":"2024-09-19T07:45:11.168583+0000","last_unstale":"2024-09-19T07:45:12.164943+0000","last_undegraded":"2024-09-19T07:45:12.164943+0000","last_fullsized":"2024-09-19T07:45:12.164943+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:08:39.515448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.167889+0000","last_change":"2024-09-19T07:45:11.160283+0000","last_active":"2024-09-19T07:45:12.167889+0000","last_peered":"2024-09-19T07:45:12.167889+0000","last_clean":"2024-09-19T07:45:12.167889+0000","last_became_active":"2024-09-19T07:45:11.159399+0000","last_became_peered":"2024-09-19T07:45:11.159399+0000","last_unstale":"2024-09-19T07:45:12.167889+0000","last_undegraded":"2024-09-19T07:45:12.167889+0000","last_fullsized":"2024-09-19T07:45:12.167889+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:47.682310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164703+0000","last_change":"2024-09-19T07:45:11.162321+0000","last_active":"2024-09-19T07:45:12.164703+0000","last_peered":"2024-09-19T07:45:12.164703+0000","last_clean":"2024-09-19T07:45:12.164703+0000","last_became_active":"2024-09-19T07:45:11.159890+0000","last_became_peered":"2024-09-19T07:45:11.159890+0000","last_unstale":"2024-09-19T07:45:12.164703+0000","last_undegraded":"2024-09-19T07:45:12.164703+0000","last_fullsized":"2024-09-19T07:45:12.164703+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:47.682310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.144721+0000","last_change":"2024-09-19T07:45:11.154147+0000","last_active":"2024-09-19T07:45:12.144721+0000","last_peered":"2024-09-19T07:45:12.144721+0000","last_clean":"2024-09-19T07:45:12.144721+0000","last_became_active":"2024-09-19T07:45:11.153578+0000","last_became_peered":"2024-09-19T07:45:11.153578+0000","last_unstale":"2024-09-19T07:45:12.144721+0000","last_undegraded":"2024-09-19T07:45:12.144721+0000","last_fullsized":"2024-09-19T07:45:12.144721+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:08:39.515448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.167799+0000","last_change":"2024-09-19T07:45:11.158400+0000","last_active":"2024-09-19T07:45:12.167799+0000","last_peered":"2024-09-19T07:45:12.167799+0000","last_clean":"2024-09-19T07:45:12.167799+0000","last_became_active":"2024-09-19T07:45:11.158143+0000","last_became_peered":"2024-09-19T07:45:11.158143+0000","last_unstale":"2024-09-19T07:45:12.167799+0000","last_undegraded":"2024-09-19T07:45:12.167799+0000","last_fullsized":"2024-09-19T07:45:12.167799+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:48.149096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"52'4","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143707+0000","last_change":"2024-09-19T07:45:10.142735+0000","last_active":"2024-09-19T07:45:12.143707+0000","last_peered":"2024-09-19T07:45:12.143707+0000","last_clean":"2024-09-19T07:45:12.143707+0000","last_became_active":"2024-09-19T07:45:10.141359+0000","last_became_peered":"2024-09-19T07:45:10.141359+0000","last_unstale":"2024-09-19T07:45:12.143707+0000","last_undegraded":"2024-09-19T07:45:12.143707+0000","last_fullsized":"2024-09-19T07:45:12.143707+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:35.509895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.144790+0000","last_change":"2024-09-19T07:45:11.156059+0000","last_active":"2024-09-19T07:45:12.144790+0000","last_peered":"2024-09-19T07:45:12.144790+0000","last_clean":"2024-09-19T07:45:12.144790+0000","last_became_active":"2024-09-19T07:45:11.155772+0000","last_became_peered":"2024-09-19T07:45:11.155772+0000","last_unstale":"2024-09-19T07:45:12.144790+0000","last_undegraded":"2024-09-19T07:45:12.144790+0000","last_fullsized":"2024-09-19T07:45:12.144790+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:03.671841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139876+0000","last_change":"2024-09-19T07:45:10.145766+0000","last_active":"2024-09-19T07:45:12.139876+0000","last_peered":"2024-09-19T07:45:12.139876+0000","last_clean":"2024-09-19T07:45:12.139876+0000","last_became_active":"2024-09-19T07:45:10.142733+0000","last_became_peered":"2024-09-19T07:45:10.142733+0000","last_unstale":"2024-09-19T07:45:12.139876+0000","last_undegraded":"2024-09-19T07:45:12.139876+0000","last_fullsized":"2024-09-19T07:45:12.139876+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:16.454219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143644+0000","last_change":"2024-09-19T07:45:11.156103+0000","last_active":"2024-09-19T07:45:12.143644+0000","last_peered":"2024-09-19T07:45:12.143644+0000","last_clean":"2024-09-19T07:45:12.143644+0000","last_became_active":"2024-09-19T07:45:11.155821+0000","last_became_peered":"2024-09-19T07:45:11.155821+0000","last_unstale":"2024-09-19T07:45:12.143644+0000","last_undegraded":"2024-09-19T07:45:12.143644+0000","last_fullsized":"2024-09-19T07:45:12.143644+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:38.636372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139825+0000","last_change":"2024-09-19T07:45:10.147013+0000","last_active":"2024-09-19T07:45:12.139825+0000","last_peered":"2024-09-19T07:45:12.139825+0000","last_clean":"2024-09-19T07:45:12.139825+0000","last_became_active":"2024-09-19T07:45:10.146032+0000","last_became_peered":"2024-09-19T07:45:10.146032+0000","last_unstale":"2024-09-19T07:45:12.139825+0000","last_undegraded":"2024-09-19T07:45:12.139825+0000","last_fullsized":"2024-09-19T07:45:12.139825+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:26.904212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.225998+0000","last_change":"2024-09-19T07:45:11.148263+0000","last_active":"2024-09-19T07:45:12.225998+0000","last_peered":"2024-09-19T07:45:12.225998+0000","last_clean":"2024-09-19T07:45:12.225998+0000","last_became_active":"2024-09-19T07:45:11.147983+0000","last_became_peered":"2024-09-19T07:45:11.147983+0000","last_unstale":"2024-09-19T07:45:12.225998+0000","last_undegraded":"2024-09-19T07:45:12.225998+0000","last_fullsized":"2024-09-19T07:45:12.225998+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:53:53.137209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.144739+0000","last_change":"2024-09-19T07:45:10.133632+0000","last_active":"2024-09-19T07:45:12.144739+0000","last_peered":"2024-09-19T07:45:12.144739+0000","last_clean":"2024-09-19T07:45:12.144739+0000","last_became_active":"2024-09-19T07:45:10.133328+0000","last_became_peered":"2024-09-19T07:45:10.133328+0000","last_unstale":"2024-09-19T07:45:12.144739+0000","last_undegraded":"2024-09-19T07:45:12.144739+0000","last_fullsized":"2024-09-19T07:45:12.144739+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:41.519074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.797139+0000","last_change":"2024-09-19T07:45:11.171607+0000","last_active":"2024-09-19T07:45:12.797139+0000","last_peered":"2024-09-19T07:45:12.797139+0000","last_clean":"2024-09-19T07:45:12.797139+0000","last_became_active":"2024-09-19T07:45:11.171351+0000","last_became_peered":"2024-09-19T07:45:11.171351+0000","last_unstale":"2024-09-19T07:45:12.797139+0000","last_undegraded":"2024-09-19T07:45:12.797139+0000","last_fullsized":"2024-09-19T07:45:12.797139+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:15.960700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164726+0000","last_change":"2024-09-19T07:45:10.141444+0000","last_active":"2024-09-19T07:45:12.164726+0000","last_peered":"2024-09-19T07:45:12.164726+0000","last_clean":"2024-09-19T07:45:12.164726+0000","last_became_active":"2024-09-19T07:45:10.141168+0000","last_became_peered":"2024-09-19T07:45:10.141168+0000","last_unstale":"2024-09-19T07:45:12.164726+0000","last_undegraded":"2024-09-19T07:45:12.164726+0000","last_fullsized":"2024-09-19T07:45:12.164726+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:54.973611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.168090+0000","last_change":"2024-09-19T07:45:11.160433+0000","last_active":"2024-09-19T07:45:12.168090+0000","last_peered":"2024-09-19T07:45:12.168090+0000","last_clean":"2024-09-19T07:45:12.168090+0000","last_became_active":"2024-09-19T07:45:11.159020+0000","last_became_peered":"2024-09-19T07:45:11.159020+0000","last_unstale":"2024-09-19T07:45:12.168090+0000","last_undegraded":"2024-09-19T07:45:12.168090+0000","last_fullsized":"2024-09-19T07:45:12.168090+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:30:53.075010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.796815+0000","last_change":"2024-09-19T07:45:10.141409+0000","last_active":"2024-09-19T07:45:12.796815+0000","last_peered":"2024-09-19T07:45:12.796815+0000","last_clean":"2024-09-19T07:45:12.796815+0000","last_became_active":"2024-09-19T07:45:10.140876+0000","last_became_peered":"2024-09-19T07:45:10.140876+0000","last_unstale":"2024-09-19T07:45:12.796815+0000","last_undegraded":"2024-09-19T07:45:12.796815+0000","last_fullsized":"2024-09-19T07:45:12.796815+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:16.270559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226226+0000","last_change":"2024-09-19T07:45:11.147187+0000","last_active":"2024-09-19T07:45:12.226226+0000","last_peered":"2024-09-19T07:45:12.226226+0000","last_clean":"2024-09-19T07:45:12.226226+0000","last_became_active":"2024-09-19T07:45:11.146894+0000","last_became_peered":"2024-09-19T07:45:11.146894+0000","last_unstale":"2024-09-19T07:45:12.226226+0000","last_undegraded":"2024-09-19T07:45:12.226226+0000","last_fullsized":"2024-09-19T07:45:12.226226+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:12.008322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143494+0000","last_change":"2024-09-19T07:45:10.143967+0000","last_active":"2024-09-19T07:45:12.143494+0000","last_peered":"2024-09-19T07:45:12.143494+0000","last_clean":"2024-09-19T07:45:12.143494+0000","last_became_active":"2024-09-19T07:45:10.143694+0000","last_became_peered":"2024-09-19T07:45:10.143694+0000","last_unstale":"2024-09-19T07:45:12.143494+0000","last_undegraded":"2024-09-19T07:45:12.143494+0000","last_fullsized":"2024-09-19T07:45:12.143494+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:42.780920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.168236+0000","last_change":"2024-09-19T07:45:11.174590+0000","last_active":"2024-09-19T07:45:12.168236+0000","last_peered":"2024-09-19T07:45:12.168236+0000","last_clean":"2024-09-19T07:45:12.168236+0000","last_became_active":"2024-09-19T07:45:11.174317+0000","last_became_peered":"2024-09-19T07:45:11.174317+0000","last_unstale":"2024-09-19T07:45:12.168236+0000","last_undegraded":"2024-09-19T07:45:12.168236+0000","last_fullsized":"2024-09-19T07:45:12.168236+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:35.156705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164540+0000","last_change":"2024-09-19T07:45:10.146821+0000","last_active":"2024-09-19T07:45:12.164540+0000","last_peered":"2024-09-19T07:45:12.164540+0000","last_clean":"2024-09-19T07:45:12.164540+0000","last_became_active":"2024-09-19T07:45:10.145946+0000","last_became_peered":"2024-09-19T07:45:10.145946+0000","last_unstale":"2024-09-19T07:45:12.164540+0000","last_undegraded":"2024-09-19T07:45:12.164540+0000","last_fullsized":"2024-09-19T07:45:12.164540+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:37.629374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.796839+0000","last_change":"2024-09-19T07:45:11.149201+0000","last_active":"2024-09-19T07:45:12.796839+0000","last_peered":"2024-09-19T07:45:12.796839+0000","last_clean":"2024-09-19T07:45:12.796839+0000","last_became_active":"2024-09-19T07:45:11.148873+0000","last_became_peered":"2024-09-19T07:45:11.148873+0000","last_unstale":"2024-09-19T07:45:12.796839+0000","last_undegraded":"2024-09-19T07:45:12.796839+0000","last_fullsized":"2024-09-19T07:45:12.796839+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:38.636372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.168113+0000","last_change":"2024-09-19T07:45:10.142038+0000","last_active":"2024-09-19T07:45:12.168113+0000","last_peered":"2024-09-19T07:45:12.168113+0000","last_clean":"2024-09-19T07:45:12.168113+0000","last_became_active":"2024-09-19T07:45:10.141756+0000","last_became_peered":"2024-09-19T07:45:10.141756+0000","last_unstale":"2024-09-19T07:45:12.168113+0000","last_undegraded":"2024-09-19T07:45:12.168113+0000","last_fullsized":"2024-09-19T07:45:12.168113+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:26:57.375819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.144594+0000","last_change":"2024-09-19T07:45:11.150745+0000","last_active":"2024-09-19T07:45:12.144594+0000","last_peered":"2024-09-19T07:45:12.144594+0000","last_clean":"2024-09-19T07:45:12.144594+0000","last_became_active":"2024-09-19T07:45:11.150307+0000","last_became_peered":"2024-09-19T07:45:11.150307+0000","last_unstale":"2024-09-19T07:45:12.144594+0000","last_undegraded":"2024-09-19T07:45:12.144594+0000","last_fullsized":"2024-09-19T07:45:12.144594+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:30:53.075010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.796824+0000","last_change":"2024-09-19T07:45:10.144207+0000","last_active":"2024-09-19T07:45:12.796824+0000","last_peered":"2024-09-19T07:45:12.796824+0000","last_clean":"2024-09-19T07:45:12.796824+0000","last_became_active":"2024-09-19T07:45:10.143382+0000","last_became_peered":"2024-09-19T07:45:10.143382+0000","last_unstale":"2024-09-19T07:45:12.796824+0000","last_undegraded":"2024-09-19T07:45:12.796824+0000","last_fullsized":"2024-09-19T07:45:12.796824+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:41.070599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164845+0000","last_change":"2024-09-19T07:45:11.170934+0000","last_active":"2024-09-19T07:45:12.164845+0000","last_peered":"2024-09-19T07:45:12.164845+0000","last_clean":"2024-09-19T07:45:12.164845+0000","last_became_active":"2024-09-19T07:45:11.168811+0000","last_became_peered":"2024-09-19T07:45:11.168811+0000","last_unstale":"2024-09-19T07:45:12.164845+0000","last_undegraded":"2024-09-19T07:45:12.164845+0000","last_fullsized":"2024-09-19T07:45:12.164845+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:27:55.022291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"52'3","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.168155+0000","last_change":"2024-09-19T07:45:10.149035+0000","last_active":"2024-09-19T07:45:12.168155+0000","last_peered":"2024-09-19T07:45:12.168155+0000","last_clean":"2024-09-19T07:45:12.168155+0000","last_became_active":"2024-09-19T07:45:10.148748+0000","last_became_peered":"2024-09-19T07:45:10.148748+0000","last_unstale":"2024-09-19T07:45:12.168155+0000","last_undegraded":"2024-09-19T07:45:12.168155+0000","last_fullsized":"2024-09-19T07:45:12.168155+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:16.459646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.797086+0000","last_change":"2024-09-19T07:45:11.150255+0000","last_active":"2024-09-19T07:45:12.797086+0000","last_peered":"2024-09-19T07:45:12.797086+0000","last_clean":"2024-09-19T07:45:12.797086+0000","last_became_active":"2024-09-19T07:45:11.149834+0000","last_became_peered":"2024-09-19T07:45:11.149834+0000","last_unstale":"2024-09-19T07:45:12.797086+0000","last_undegraded":"2024-09-19T07:45:12.797086+0000","last_fullsized":"2024-09-19T07:45:12.797086+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:48.790896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"52'6","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226249+0000","last_change":"2024-09-19T07:45:10.141653+0000","last_active":"2024-09-19T07:45:12.226249+0000","last_peered":"2024-09-19T07:45:12.226249+0000","last_clean":"2024-09-19T07:45:12.226249+0000","last_became_active":"2024-09-19T07:45:10.141023+0000","last_became_peered":"2024-09-19T07:45:10.141023+0000","last_unstale":"2024-09-19T07:45:12.226249+0000","last_undegraded":"2024-09-19T07:45:12.226249+0000","last_fullsized":"2024-09-19T07:45:12.226249+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:46.675311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.144572+0000","last_change":"2024-09-19T07:45:11.160328+0000","last_active":"2024-09-19T07:45:12.144572+0000","last_peered":"2024-09-19T07:45:12.144572+0000","last_clean":"2024-09-19T07:45:12.144572+0000","last_became_active":"2024-09-19T07:45:11.160012+0000","last_became_peered":"2024-09-19T07:45:11.160012+0000","last_unstale":"2024-09-19T07:45:12.144572+0000","last_undegraded":"2024-09-19T07:45:12.144572+0000","last_fullsized":"2024-09-19T07:45:12.144572+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:27:55.022291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164861+0000","last_change":"2024-09-19T07:45:10.143597+0000","last_active":"2024-09-19T07:45:12.164861+0000","last_peered":"2024-09-19T07:45:12.164861+0000","last_clean":"2024-09-19T07:45:12.164861+0000","last_became_active":"2024-09-19T07:45:10.143337+0000","last_became_peered":"2024-09-19T07:45:10.143337+0000","last_unstale":"2024-09-19T07:45:12.164861+0000","last_undegraded":"2024-09-19T07:45:12.164861+0000","last_fullsized":"2024-09-19T07:45:12.164861+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:49.955828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.165116+0000","last_change":"2024-09-19T07:45:11.165460+0000","last_active":"2024-09-19T07:45:12.165116+0000","last_peered":"2024-09-19T07:45:12.165116+0000","last_clean":"2024-09-19T07:45:12.165116+0000","last_became_active":"2024-09-19T07:45:11.164722+0000","last_became_peered":"2024-09-19T07:45:11.164722+0000","last_unstale":"2024-09-19T07:45:12.165116+0000","last_undegraded":"2024-09-19T07:45:12.165116+0000","last_fullsized":"2024-09-19T07:45:12.165116+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:25.374821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143598+0000","last_change":"2024-09-19T07:45:10.149191+0000","last_active":"2024-09-19T07:45:12.143598+0000","last_peered":"2024-09-19T07:45:12.143598+0000","last_clean":"2024-09-19T07:45:12.143598+0000","last_became_active":"2024-09-19T07:45:10.148886+0000","last_became_peered":"2024-09-19T07:45:10.148886+0000","last_unstale":"2024-09-19T07:45:12.143598+0000","last_undegraded":"2024-09-19T07:45:12.143598+0000","last_fullsized":"2024-09-19T07:45:12.143598+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:25.675380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226125+0000","last_change":"2024-09-19T07:45:11.161322+0000","last_active":"2024-09-19T07:45:12.226125+0000","last_peered":"2024-09-19T07:45:12.226125+0000","last_clean":"2024-09-19T07:45:12.226125+0000","last_became_active":"2024-09-19T07:45:11.160672+0000","last_became_peered":"2024-09-19T07:45:11.160672+0000","last_unstale":"2024-09-19T07:45:12.226125+0000","last_undegraded":"2024-09-19T07:45:12.226125+0000","last_fullsized":"2024-09-19T07:45:12.226125+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:41.266322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.167959+0000","last_change":"2024-09-19T07:45:10.143903+0000","last_active":"2024-09-19T07:45:12.167959+0000","last_peered":"2024-09-19T07:45:12.167959+0000","last_clean":"2024-09-19T07:45:12.167959+0000","last_became_active":"2024-09-19T07:45:10.143552+0000","last_became_peered":"2024-09-19T07:45:10.143552+0000","last_unstale":"2024-09-19T07:45:12.167959+0000","last_undegraded":"2024-09-19T07:45:12.167959+0000","last_fullsized":"2024-09-19T07:45:12.167959+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:16.347757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.165305+0000","last_change":"2024-09-19T07:45:11.172882+0000","last_active":"2024-09-19T07:45:12.165305+0000","last_peered":"2024-09-19T07:45:12.165305+0000","last_clean":"2024-09-19T07:45:12.165305+0000","last_became_active":"2024-09-19T07:45:11.172243+0000","last_became_peered":"2024-09-19T07:45:11.172243+0000","last_unstale":"2024-09-19T07:45:12.165305+0000","last_undegraded":"2024-09-19T07:45:12.165305+0000","last_fullsized":"2024-09-19T07:45:12.165305+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:21:42.341500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"52'8","reported_seq":22,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226137+0000","last_change":"2024-09-19T07:45:10.141231+0000","last_active":"2024-09-19T07:45:12.226137+0000","last_peered":"2024-09-19T07:45:12.226137+0000","last_clean":"2024-09-19T07:45:12.226137+0000","last_became_active":"2024-09-19T07:45:10.140945+0000","last_became_peered":"2024-09-19T07:45:10.140945+0000","last_unstale":"2024-09-19T07:45:12.226137+0000","last_undegraded":"2024-09-19T07:45:12.226137+0000","last_fullsized":"2024-09-19T07:45:12.226137+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:34.149707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226029+0000","last_change":"2024-09-19T07:45:11.151822+0000","last_active":"2024-09-19T07:45:12.226029+0000","last_peered":"2024-09-19T07:45:12.226029+0000","last_clean":"2024-09-19T07:45:12.226029+0000","last_became_active":"2024-09-19T07:45:11.151224+0000","last_became_peered":"2024-09-19T07:45:11.151224+0000","last_unstale":"2024-09-19T07:45:12.226029+0000","last_undegraded":"2024-09-19T07:45:12.226029+0000","last_fullsized":"2024-09-19T07:45:12.226029+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:46:36.919958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.165425+0000","last_change":"2024-09-19T07:45:10.149077+0000","last_active":"2024-09-19T07:45:12.165425+0000","last_peered":"2024-09-19T07:45:12.165425+0000","last_clean":"2024-09-19T07:45:12.165425+0000","last_became_active":"2024-09-19T07:45:10.148745+0000","last_became_peered":"2024-09-19T07:45:10.148745+0000","last_unstale":"2024-09-19T07:45:12.165425+0000","last_undegraded":"2024-09-19T07:45:12.165425+0000","last_fullsized":"2024-09-19T07:45:12.165425+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:03.332021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139418+0000","last_change":"2024-09-19T07:45:11.156578+0000","last_active":"2024-09-19T07:45:12.139418+0000","last_peered":"2024-09-19T07:45:12.139418+0000","last_clean":"2024-09-19T07:45:12.139418+0000","last_became_active":"2024-09-19T07:45:11.156292+0000","last_became_peered":"2024-09-19T07:45:11.156292+0000","last_unstale":"2024-09-19T07:45:12.139418+0000","last_undegraded":"2024-09-19T07:45:12.139418+0000","last_fullsized":"2024-09-19T07:45:12.139418+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:17.466645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"52'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-19T07:45:11.177482+0000","last_change":"2024-09-19T07:45:10.133568+0000","last_active":"2024-09-19T07:45:11.177482+0000","last_peered":"2024-09-19T07:45:11.177482+0000","last_clean":"2024-09-19T07:45:11.177482+0000","last_became_active":"2024-09-19T07:45:10.133255+0000","last_became_peered":"2024-09-19T07:45:10.133255+0000","last_unstale":"2024-09-19T07:45:11.177482+0000","last_undegraded":"2024-09-19T07:45:11.177482+0000","last_fullsized":"2024-09-19T07:45:11.177482+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:54:35.081711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.796965+0000","last_change":"2024-09-19T07:45:11.171801+0000","last_active":"2024-09-19T07:45:12.796965+0000","last_peered":"2024-09-19T07:45:12.796965+0000","last_clean":"2024-09-19T07:45:12.796965+0000","last_became_active":"2024-09-19T07:45:11.171531+0000","last_became_peered":"2024-09-19T07:45:11.171531+0000","last_unstale":"2024-09-19T07:45:12.796965+0000","last_undegraded":"2024-09-19T07:45:12.796965+0000","last_fullsized":"2024-09-19T07:45:12.796965+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:55.980609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143914+0000","last_change":"2024-09-19T07:45:10.142185+0000","last_active":"2024-09-19T07:45:12.143914+0000","last_peered":"2024-09-19T07:45:12.143914+0000","last_clean":"2024-09-19T07:45:12.143914+0000","last_became_active":"2024-09-19T07:45:10.141823+0000","last_became_peered":"2024-09-19T07:45:10.141823+0000","last_unstale":"2024-09-19T07:45:12.143914+0000","last_undegraded":"2024-09-19T07:45:12.143914+0000","last_fullsized":"2024-09-19T07:45:12.143914+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:08.776990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139620+0000","last_change":"2024-09-19T07:45:11.144563+0000","last_active":"2024-09-19T07:45:12.139620+0000","last_peered":"2024-09-19T07:45:12.139620+0000","last_clean":"2024-09-19T07:45:12.139620+0000","last_became_active":"2024-09-19T07:45:11.144221+0000","last_became_peered":"2024-09-19T07:45:11.144221+0000","last_unstale":"2024-09-19T07:45:12.139620+0000","last_undegraded":"2024-09-19T07:45:12.139620+0000","last_fullsized":"2024-09-19T07:45:12.139620+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:52.729562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143855+0000","last_change":"2024-09-19T07:45:10.141010+0000","last_active":"2024-09-19T07:45:12.143855+0000","last_peered":"2024-09-19T07:45:12.143855+0000","last_clean":"2024-09-19T07:45:12.143855+0000","last_became_active":"2024-09-19T07:45:10.140747+0000","last_became_peered":"2024-09-19T07:45:10.140747+0000","last_unstale":"2024-09-19T07:45:12.143855+0000","last_undegraded":"2024-09-19T07:45:12.143855+0000","last_fullsized":"2024-09-19T07:45:12.143855+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:41.070599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.144791+0000","last_change":"2024-09-19T07:45:11.150879+0000","last_active":"2024-09-19T07:45:12.144791+0000","last_peered":"2024-09-19T07:45:12.144791+0000","last_clean":"2024-09-19T07:45:12.144791+0000","last_became_active":"2024-09-19T07:45:11.150562+0000","last_became_peered":"2024-09-19T07:45:11.150562+0000","last_unstale":"2024-09-19T07:45:12.144791+0000","last_undegraded":"2024-09-19T07:45:12.144791+0000","last_fullsized":"2024-09-19T07:45:12.144791+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:25.374821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139717+0000","last_change":"2024-09-19T07:45:10.140315+0000","last_active":"2024-09-19T07:45:12.139717+0000","last_peered":"2024-09-19T07:45:12.139717+0000","last_clean":"2024-09-19T07:45:12.139717+0000","last_became_active":"2024-09-19T07:45:10.140052+0000","last_became_peered":"2024-09-19T07:45:10.140052+0000","last_unstale":"2024-09-19T07:45:12.139717+0000","last_undegraded":"2024-09-19T07:45:12.139717+0000","last_fullsized":"2024-09-19T07:45:12.139717+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:55.554697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143851+0000","last_change":"2024-09-19T07:45:11.151218+0000","last_active":"2024-09-19T07:45:12.143851+0000","last_peered":"2024-09-19T07:45:12.143851+0000","last_clean":"2024-09-19T07:45:12.143851+0000","last_became_active":"2024-09-19T07:45:11.150938+0000","last_became_peered":"2024-09-19T07:45:11.150938+0000","last_unstale":"2024-09-19T07:45:12.143851+0000","last_undegraded":"2024-09-19T07:45:12.143851+0000","last_fullsized":"2024-09-19T07:45:12.143851+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:48.790896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.225818+0000","last_change":"2024-09-19T07:45:10.145203+0000","last_active":"2024-09-19T07:45:12.225818+0000","last_peered":"2024-09-19T07:45:12.225818+0000","last_clean":"2024-09-19T07:45:12.225818+0000","last_became_active":"2024-09-19T07:45:10.144489+0000","last_became_peered":"2024-09-19T07:45:10.144489+0000","last_unstale":"2024-09-19T07:45:12.225818+0000","last_undegraded":"2024-09-19T07:45:12.225818+0000","last_fullsized":"2024-09-19T07:45:12.225818+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:30:52.068011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.165188+0000","last_change":"2024-09-19T07:45:11.169319+0000","last_active":"2024-09-19T07:45:12.165188+0000","last_peered":"2024-09-19T07:45:12.165188+0000","last_clean":"2024-09-19T07:45:12.165188+0000","last_became_active":"2024-09-19T07:45:11.168698+0000","last_became_peered":"2024-09-19T07:45:11.168698+0000","last_unstale":"2024-09-19T07:45:12.165188+0000","last_undegraded":"2024-09-19T07:45:12.165188+0000","last_fullsized":"2024-09-19T07:45:12.165188+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:01.056784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164764+0000","last_change":"2024-09-19T07:45:10.142588+0000","last_active":"2024-09-19T07:45:12.164764+0000","last_peered":"2024-09-19T07:45:12.164764+0000","last_clean":"2024-09-19T07:45:12.164764+0000","last_became_active":"2024-09-19T07:45:10.142215+0000","last_became_peered":"2024-09-19T07:45:10.142215+0000","last_unstale":"2024-09-19T07:45:12.164764+0000","last_undegraded":"2024-09-19T07:45:12.164764+0000","last_fullsized":"2024-09-19T07:45:12.164764+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:08.197576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139881+0000","last_change":"2024-09-19T07:45:11.151059+0000","last_active":"2024-09-19T07:45:12.139881+0000","last_peered":"2024-09-19T07:45:12.139881+0000","last_clean":"2024-09-19T07:45:12.139881+0000","last_became_active":"2024-09-19T07:45:11.150738+0000","last_became_peered":"2024-09-19T07:45:11.150738+0000","last_unstale":"2024-09-19T07:45:12.139881+0000","last_undegraded":"2024-09-19T07:45:12.139881+0000","last_fullsized":"2024-09-19T07:45:12.139881+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:18.574474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-19T07:45:11.143869+0000","last_change":"2024-09-19T07:45:10.135637+0000","last_active":"2024-09-19T07:45:11.143869+0000","last_peered":"2024-09-19T07:45:11.143869+0000","last_clean":"2024-09-19T07:45:11.143869+0000","last_became_active":"2024-09-19T07:45:10.135359+0000","last_became_peered":"2024-09-19T07:45:10.135359+0000","last_unstale":"2024-09-19T07:45:11.143869+0000","last_undegraded":"2024-09-19T07:45:11.143869+0000","last_fullsized":"2024-09-19T07:45:11.143869+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:45.339997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.144729+0000","last_change":"2024-09-19T07:45:11.153930+0000","last_active":"2024-09-19T07:45:12.144729+0000","last_peered":"2024-09-19T07:45:12.144729+0000","last_clean":"2024-09-19T07:45:12.144729+0000","last_became_active":"2024-09-19T07:45:11.153222+0000","last_became_peered":"2024-09-19T07:45:11.153222+0000","last_unstale":"2024-09-19T07:45:12.144729+0000","last_undegraded":"2024-09-19T07:45:12.144729+0000","last_fullsized":"2024-09-19T07:45:12.144729+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:20.215194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143801+0000","last_change":"2024-09-19T07:45:10.149272+0000","last_active":"2024-09-19T07:45:12.143801+0000","last_peered":"2024-09-19T07:45:12.143801+0000","last_clean":"2024-09-19T07:45:12.143801+0000","last_became_active":"2024-09-19T07:45:10.148962+0000","last_became_peered":"2024-09-19T07:45:10.148962+0000","last_unstale":"2024-09-19T07:45:12.143801+0000","last_undegraded":"2024-09-19T07:45:12.143801+0000","last_fullsized":"2024-09-19T07:45:12.143801+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:26:43.107121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139623+0000","last_change":"2024-09-19T07:45:11.146344+0000","last_active":"2024-09-19T07:45:12.139623+0000","last_peered":"2024-09-19T07:45:12.139623+0000","last_clean":"2024-09-19T07:45:12.139623+0000","last_became_active":"2024-09-19T07:45:11.146039+0000","last_became_peered":"2024-09-19T07:45:11.146039+0000","last_unstale":"2024-09-19T07:45:12.139623+0000","last_undegraded":"2024-09-19T07:45:12.139623+0000","last_fullsized":"2024-09-19T07:45:12.139623+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:08.219281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.168182+0000","last_change":"2024-09-19T07:45:10.143810+0000","last_active":"2024-09-19T07:45:12.168182+0000","last_peered":"2024-09-19T07:45:12.168182+0000","last_clean":"2024-09-19T07:45:12.168182+0000","last_became_active":"2024-09-19T07:45:10.143462+0000","last_became_peered":"2024-09-19T07:45:10.143462+0000","last_unstale":"2024-09-19T07:45:12.168182+0000","last_undegraded":"2024-09-19T07:45:12.168182+0000","last_fullsized":"2024-09-19T07:45:12.168182+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:49.585196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226198+0000","last_change":"2024-09-19T07:45:11.151896+0000","last_active":"2024-09-19T07:45:12.226198+0000","last_peered":"2024-09-19T07:45:12.226198+0000","last_clean":"2024-09-19T07:45:12.226198+0000","last_became_active":"2024-09-19T07:45:11.151588+0000","last_became_peered":"2024-09-19T07:45:11.151588+0000","last_unstale":"2024-09-19T07:45:12.226198+0000","last_undegraded":"2024-09-19T07:45:12.226198+0000","last_fullsized":"2024-09-19T07:45:12.226198+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:39:20.162645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164887+0000","last_change":"2024-09-19T07:45:10.144986+0000","last_active":"2024-09-19T07:45:12.164887+0000","last_peered":"2024-09-19T07:45:12.164887+0000","last_clean":"2024-09-19T07:45:12.164887+0000","last_became_active":"2024-09-19T07:45:10.144710+0000","last_became_peered":"2024-09-19T07:45:10.144710+0000","last_unstale":"2024-09-19T07:45:12.164887+0000","last_undegraded":"2024-09-19T07:45:12.164887+0000","last_fullsized":"2024-09-19T07:45:12.164887+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:14.953702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164834+0000","last_change":"2024-09-19T07:45:11.158294+0000","last_active":"2024-09-19T07:45:12.164834+0000","last_peered":"2024-09-19T07:45:12.164834+0000","last_clean":"2024-09-19T07:45:12.164834+0000","last_became_active":"2024-09-19T07:45:11.158022+0000","last_became_peered":"2024-09-19T07:45:11.158022+0000","last_unstale":"2024-09-19T07:45:12.164834+0000","last_undegraded":"2024-09-19T07:45:12.164834+0000","last_fullsized":"2024-09-19T07:45:12.164834+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:17.366751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226102+0000","last_change":"2024-09-19T07:45:10.141582+0000","last_active":"2024-09-19T07:45:12.226102+0000","last_peered":"2024-09-19T07:45:12.226102+0000","last_clean":"2024-09-19T07:45:12.226102+0000","last_became_active":"2024-09-19T07:45:10.141072+0000","last_became_peered":"2024-09-19T07:45:10.141072+0000","last_unstale":"2024-09-19T07:45:12.226102+0000","last_undegraded":"2024-09-19T07:45:12.226102+0000","last_fullsized":"2024-09-19T07:45:12.226102+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:27:54.015292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.144675+0000","last_change":"2024-09-19T07:45:11.160091+0000","last_active":"2024-09-19T07:45:12.144675+0000","last_peered":"2024-09-19T07:45:12.144675+0000","last_clean":"2024-09-19T07:45:12.144675+0000","last_became_active":"2024-09-19T07:45:11.159811+0000","last_became_peered":"2024-09-19T07:45:11.159811+0000","last_unstale":"2024-09-19T07:45:12.144675+0000","last_undegraded":"2024-09-19T07:45:12.144675+0000","last_fullsized":"2024-09-19T07:45:12.144675+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:35.156705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.225947+0000","last_change":"2024-09-19T07:45:10.141689+0000","last_active":"2024-09-19T07:45:12.225947+0000","last_peered":"2024-09-19T07:45:12.225947+0000","last_clean":"2024-09-19T07:45:12.225947+0000","last_became_active":"2024-09-19T07:45:10.141392+0000","last_became_peered":"2024-09-19T07:45:10.141392+0000","last_unstale":"2024-09-19T07:45:12.225947+0000","last_undegraded":"2024-09-19T07:45:12.225947+0000","last_fullsized":"2024-09-19T07:45:12.225947+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:16.359752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164831+0000","last_change":"2024-09-19T07:45:11.169551+0000","last_active":"2024-09-19T07:45:12.164831+0000","last_peered":"2024-09-19T07:45:12.164831+0000","last_clean":"2024-09-19T07:45:12.164831+0000","last_became_active":"2024-09-19T07:45:11.169035+0000","last_became_peered":"2024-09-19T07:45:11.169035+0000","last_unstale":"2024-09-19T07:45:12.164831+0000","last_undegraded":"2024-09-19T07:45:12.164831+0000","last_fullsized":"2024-09-19T07:45:12.164831+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:30:53.075010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.796655+0000","last_change":"2024-09-19T07:45:10.134962+0000","last_active":"2024-09-19T07:45:12.796655+0000","last_peered":"2024-09-19T07:45:12.796655+0000","last_clean":"2024-09-19T07:45:12.796655+0000","last_became_active":"2024-09-19T07:45:10.134663+0000","last_became_peered":"2024-09-19T07:45:10.134663+0000","last_unstale":"2024-09-19T07:45:12.796655+0000","last_undegraded":"2024-09-19T07:45:12.796655+0000","last_fullsized":"2024-09-19T07:45:12.796655+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:14.040132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226305+0000","last_change":"2024-09-19T07:45:11.153489+0000","last_active":"2024-09-19T07:45:12.226305+0000","last_peered":"2024-09-19T07:45:12.226305+0000","last_clean":"2024-09-19T07:45:12.226305+0000","last_became_active":"2024-09-19T07:45:11.153190+0000","last_became_peered":"2024-09-19T07:45:11.153190+0000","last_unstale":"2024-09-19T07:45:12.226305+0000","last_undegraded":"2024-09-19T07:45:12.226305+0000","last_fullsized":"2024-09-19T07:45:12.226305+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:29.711588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.144797+0000","last_change":"2024-09-19T07:45:10.138425+0000","last_active":"2024-09-19T07:45:12.144797+0000","last_peered":"2024-09-19T07:45:12.144797+0000","last_clean":"2024-09-19T07:45:12.144797+0000","last_became_active":"2024-09-19T07:45:10.138176+0000","last_became_peered":"2024-09-19T07:45:10.138176+0000","last_unstale":"2024-09-19T07:45:12.144797+0000","last_undegraded":"2024-09-19T07:45:12.144797+0000","last_fullsized":"2024-09-19T07:45:12.144797+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:39.856239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143503+0000","last_change":"2024-09-19T07:45:11.147998+0000","last_active":"2024-09-19T07:45:12.143503+0000","last_peered":"2024-09-19T07:45:12.143503+0000","last_clean":"2024-09-19T07:45:12.143503+0000","last_became_active":"2024-09-19T07:45:11.147682+0000","last_became_peered":"2024-09-19T07:45:11.147682+0000","last_unstale":"2024-09-19T07:45:12.143503+0000","last_undegraded":"2024-09-19T07:45:12.143503+0000","last_fullsized":"2024-09-19T07:45:12.143503+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:35.314160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"53'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.165136+0000","last_change":"2024-09-19T07:45:10.144229+0000","last_active":"2024-09-19T07:45:12.165136+0000","last_peered":"2024-09-19T07:45:12.165136+0000","last_clean":"2024-09-19T07:45:12.165136+0000","last_became_active":"2024-09-19T07:45:10.143971+0000","last_became_peered":"2024-09-19T07:45:10.143971+0000","last_unstale":"2024-09-19T07:45:12.165136+0000","last_undegraded":"2024-09-19T07:45:12.165136+0000","last_fullsized":"2024-09-19T07:45:12.165136+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:47.783898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":228,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.167752+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:45:12.167752+0000","last_peered":"2024-09-19T07:45:12.167752+0000","last_clean":"2024-09-19T07:45:12.167752+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:45:12.167752+0000","last_undegraded":"2024-09-19T07:45:12.167752+0000","last_fullsized":"2024-09-19T07:45:12.167752+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1224704,"data_stored":867174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":277334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509471,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":622592,"data_stored":276682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":626688,"data_stored":277264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509470,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1224704,"data_stored":867106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542174,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":638976,"data_stored":278316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":630784,"data_stored":276842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31772,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340068,"statfs":{"total":96636764160,"available":96604229632,"internally_reserved":0,"allocated":598016,"data_stored":262823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:45:15.136 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:45:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.275+0000 7fc021549640 1 Processor -- start 2024-09-19T07:45:15.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.275+0000 7fc021549640 1 -- start start 2024-09-19T07:45:15.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.276+0000 7fc021549640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c15e680 0x7fc01c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:15.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.276+0000 7fc021549640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc01c0d0010 0x7fc01c0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:15.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.276+0000 7fc021549640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc01c0d1880 con 0x7fc01c15e2a0 2024-09-19T07:45:15.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.276+0000 7fc021549640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc01c0d1b60 con 0x7fc01c0d0010 2024-09-19T07:45:15.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.276+0000 7fc021549640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc01c0cf2d0 con 0x7fc01c15e680 2024-09-19T07:45:15.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.276+0000 7fc01b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc01c0d0010 0x7fc01c0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:15.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.276+0000 7fc01b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc01c0d0010 0x7fc01c0d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46960/0 (socket says 172.21.15.16:46960) 2024-09-19T07:45:15.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.276+0000 7fc01b7fe640 1 -- 172.21.15.16:0/1255997957 learned_addr learned my addr 172.21.15.16:0/1255997957 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:15.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.276+0000 7fc01affd640 1 --1- 172.21.15.16:0/1255997957 >> v1:172.21.15.16:6789/0 conn(0x7fc01c15e2a0 0x7fc01c06d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50356/0 (socket says 172.21.15.16:50356) 2024-09-19T07:45:15.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.276+0000 7fc013fff640 1 --2- 172.21.15.16:0/1255997957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c15e680 0x7fc01c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:15.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.276+0000 7fc01b7fe640 1 -- 172.21.15.16:0/1255997957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c15e680 msgr2=0x7fc01c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:15.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.276+0000 7fc01b7fe640 1 --2- 172.21.15.16:0/1255997957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c15e680 0x7fc01c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.276+0000 7fc01b7fe640 1 -- 172.21.15.16:0/1255997957 >> v1:172.21.15.16:6789/0 conn(0x7fc01c15e2a0 legacy=0x7fc01c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:15.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.276+0000 7fc01a7fc640 1 -- 172.21.15.16:0/1255997957 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2029682280 0 0) 0x7fc01c0d1880 con 0x7fc01c15e2a0 2024-09-19T07:45:15.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.276+0000 7fc01b7fe640 1 -- 172.21.15.16:0/1255997957 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c0cf5b0 con 0x7fc01c0d0010 2024-09-19T07:45:15.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.277+0000 7fc01b7fe640 1 --2- 172.21.15.16:0/1255997957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c0d0010 0x7fc01c0d4370 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc014009e50 tx=0x7fc01402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f5e6b60c2741a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:15.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.277+0000 7fc01a7fc640 1 -- 172.21.15.16:0/1255997957 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014036070 con 0x7fc01c0d0010 2024-09-19T07:45:15.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.277+0000 7fc01a7fc640 1 -- 172.21.15.16:0/1255997957 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc014031600 con 0x7fc01c0d0010 2024-09-19T07:45:15.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.277+0000 7fc01a7fc640 1 -- 172.21.15.16:0/1255997957 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0140319d0 con 0x7fc01c0d0010 2024-09-19T07:45:15.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.278+0000 7fc021549640 1 -- 172.21.15.16:0/1255997957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c0d0010 msgr2=0x7fc01c0d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:15.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.278+0000 7fc021549640 1 --2- 172.21.15.16:0/1255997957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c0d0010 0x7fc01c0d4370 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc014009e50 tx=0x7fc01402a680 comp rx=0 tx=0).stop 2024-09-19T07:45:15.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.278+0000 7fc021549640 1 -- 172.21.15.16:0/1255997957 shutdown_connections 2024-09-19T07:45:15.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.278+0000 7fc021549640 1 --2- 172.21.15.16:0/1255997957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c0d0010 0x7fc01c0d4370 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.278+0000 7fc021549640 1 --2- 172.21.15.16:0/1255997957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c15e680 0x7fc01c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.278+0000 7fc021549640 1 -- 172.21.15.16:0/1255997957 >> 172.21.15.16:0/1255997957 conn(0x7fc01c09d760 msgr2=0x7fc01c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:15.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.278+0000 7fc021549640 1 -- 172.21.15.16:0/1255997957 shutdown_connections 2024-09-19T07:45:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.278+0000 7fc021549640 1 -- 172.21.15.16:0/1255997957 wait complete. 2024-09-19T07:45:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.279+0000 7fc021549640 1 Processor -- start 2024-09-19T07:45:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.279+0000 7fc021549640 1 -- start start 2024-09-19T07:45:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.280+0000 7fc021549640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c0d0010 0x7fc01c156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.280+0000 7fc021549640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c15e2a0 0x7fc01c0b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.280+0000 7fc021549640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc01c15e680 0x7fc01c0b9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.280+0000 7fc021549640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c06dcb0 con 0x7fc01c0d0010 2024-09-19T07:45:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.280+0000 7fc021549640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c06db30 con 0x7fc01c15e680 2024-09-19T07:45:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.280+0000 7fc021549640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc01c0d48b0 con 0x7fc01c15e2a0 2024-09-19T07:45:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.280+0000 7fc01affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c0d0010 0x7fc01c156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.280+0000 7fc01affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c0d0010 0x7fc01c156a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46962/0 (socket says 172.21.15.16:46962) 2024-09-19T07:45:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.280+0000 7fc01affd640 1 -- 172.21.15.16:0/327317969 learned_addr learned my addr 172.21.15.16:0/327317969 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.280+0000 7fc013fff640 1 --2- 172.21.15.16:0/327317969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c15e2a0 0x7fc01c0b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:15.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.280+0000 7fc01b7fe640 1 --2- 172.21.15.16:0/327317969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc01c15e680 0x7fc01c0b9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.280+0000 7fc01affd640 1 -- 172.21.15.16:0/327317969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c15e2a0 msgr2=0x7fc01c0b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.280+0000 7fc01affd640 1 --2- 172.21.15.16:0/327317969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c15e2a0 0x7fc01c0b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.280+0000 7fc01affd640 1 -- 172.21.15.16:0/327317969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc01c15e680 msgr2=0x7fc01c0b9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.280+0000 7fc01affd640 1 --2- 172.21.15.16:0/327317969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc01c15e680 0x7fc01c0b9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.280+0000 7fc01affd640 1 -- 172.21.15.16:0/327317969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c0b7d20 con 0x7fc01c0d0010 2024-09-19T07:45:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.280+0000 7fc01affd640 1 --2- 172.21.15.16:0/327317969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c0d0010 0x7fc01c156a30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00c4d0 tx=0x7fc00c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.281+0000 7fc0137fe640 1 -- 172.21.15.16:0/327317969 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c018070 con 0x7fc01c0d0010 2024-09-19T07:45:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.281+0000 7fc0137fe640 1 -- 172.21.15.16:0/327317969 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc00c007500 con 0x7fc01c0d0010 2024-09-19T07:45:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.281+0000 7fc021549640 1 -- 172.21.15.16:0/327317969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01c0b8000 con 0x7fc01c0d0010 2024-09-19T07:45:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.281+0000 7fc0137fe640 1 -- 172.21.15.16:0/327317969 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c004a20 con 0x7fc01c0d0010 2024-09-19T07:45:15.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.281+0000 7fc021549640 1 -- 172.21.15.16:0/327317969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01c0cc370 con 0x7fc01c0d0010 2024-09-19T07:45:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.282+0000 7fc021549640 1 -- 172.21.15.16:0/327317969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01c0d4370 con 0x7fc01c0d0010 2024-09-19T07:45:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.282+0000 7fc0137fe640 1 -- 172.21.15.16:0/327317969 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc00c01b070 con 0x7fc01c0d0010 2024-09-19T07:45:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.283+0000 7fc0137fe640 1 --2- 172.21.15.16:0/327317969 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbffc06c2b0 0x7fbffc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.283+0000 7fc0137fe640 1 -- 172.21.15.16:0/327317969 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fc00c08a820 con 0x7fc01c0d0010 2024-09-19T07:45:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.283+0000 7fc013fff640 1 --2- 172.21.15.16:0/327317969 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbffc06c2b0 0x7fbffc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.283+0000 7fc013fff640 1 --2- 172.21.15.16:0/327317969 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbffc06c2b0 0x7fbffc06e770 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0c7b20 tx=0x7fc004008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.287+0000 7fc0137fe640 1 -- 172.21.15.16:0/327317969 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc00c058840 con 0x7fc01c0d0010 2024-09-19T07:45:15.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.461+0000 7fc021549640 1 -- 172.21.15.16:0/327317969 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc01c06cf90 con 0x7fbffc06c2b0 2024-09-19T07:45:15.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.468+0000 7fc0137fe640 1 -- 172.21.15.16:0/327317969 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251954 (secure 0 0 0) 0x7fc01c06cf90 con 0x7fbffc06c2b0 2024-09-19T07:45:15.471 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:45:15.473 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:45:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.471+0000 7fc021549640 1 -- 172.21.15.16:0/327317969 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbffc06c2b0 msgr2=0x7fbffc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.471+0000 7fc021549640 1 --2- 172.21.15.16:0/327317969 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbffc06c2b0 0x7fbffc06e770 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0c7b20 tx=0x7fc004008040 comp rx=0 tx=0).stop 2024-09-19T07:45:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.471+0000 7fc021549640 1 -- 172.21.15.16:0/327317969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c0d0010 msgr2=0x7fc01c156a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.471+0000 7fc021549640 1 --2- 172.21.15.16:0/327317969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c0d0010 0x7fc01c156a30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00c4d0 tx=0x7fc00c00c990 comp rx=0 tx=0).stop 2024-09-19T07:45:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.471+0000 7fc021549640 1 -- 172.21.15.16:0/327317969 shutdown_connections 2024-09-19T07:45:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.471+0000 7fc021549640 1 --2- 172.21.15.16:0/327317969 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbffc06c2b0 0x7fbffc06e770 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.471+0000 7fc021549640 1 --2- 172.21.15.16:0/327317969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc01c15e680 0x7fc01c0b9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.471+0000 7fc021549640 1 --2- 172.21.15.16:0/327317969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01c15e2a0 0x7fc01c0b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.471+0000 7fc021549640 1 --2- 172.21.15.16:0/327317969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c0d0010 0x7fc01c156a30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.471+0000 7fc021549640 1 -- 172.21.15.16:0/327317969 >> 172.21.15.16:0/327317969 conn(0x7fc01c09d760 msgr2=0x7fc01c15f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.471+0000 7fc021549640 1 -- 172.21.15.16:0/327317969 shutdown_connections 2024-09-19T07:45:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.471+0000 7fc021549640 1 -- 172.21.15.16:0/327317969 wait complete. 2024-09-19T07:45:15.483 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-09-19T07:45:14.488917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243104,"kb_used_data":6052,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731616,"statfs":{"total":773094113280,"available":772845174784,"internally_reserved":0,"allocated":6197248,"data_stored":3383541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.362871"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139720+0000","last_change":"2024-09-19T07:45:11.145009+0000","last_active":"2024-09-19T07:45:12.139720+0000","last_peered":"2024-09-19T07:45:12.139720+0000","last_clean":"2024-09-19T07:45:12.139720+0000","last_became_active":"2024-09-19T07:45:11.144715+0000","last_became_peered":"2024-09-19T07:45:11.144715+0000","last_unstale":"2024-09-19T07:45:12.139720+0000","last_undegraded":"2024-09-19T07:45:12.139720+0000","last_fullsized":"2024-09-19T07:45:12.139720+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:14.467600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.168262+0000","last_change":"2024-09-19T07:45:11.158808+0000","last_active":"2024-09-19T07:45:12.168262+0000","last_peered":"2024-09-19T07:45:12.168262+0000","last_clean":"2024-09-19T07:45:12.168262+0000","last_became_active":"2024-09-19T07:45:11.158288+0000","last_became_peered":"2024-09-19T07:45:11.158288+0000","last_unstale":"2024-09-19T07:45:12.168262+0000","last_undegraded":"2024-09-19T07:45:12.168262+0000","last_fullsized":"2024-09-19T07:45:12.168262+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:20.215194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164977+0000","last_change":"2024-09-19T07:45:11.168324+0000","last_active":"2024-09-19T07:45:12.164977+0000","last_peered":"2024-09-19T07:45:12.164977+0000","last_clean":"2024-09-19T07:45:12.164977+0000","last_became_active":"2024-09-19T07:45:11.166933+0000","last_became_peered":"2024-09-19T07:45:11.166933+0000","last_unstale":"2024-09-19T07:45:12.164977+0000","last_undegraded":"2024-09-19T07:45:12.164977+0000","last_fullsized":"2024-09-19T07:45:12.164977+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:20.215194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.168114+0000","last_change":"2024-09-19T07:45:11.160049+0000","last_active":"2024-09-19T07:45:12.168114+0000","last_peered":"2024-09-19T07:45:12.168114+0000","last_clean":"2024-09-19T07:45:12.168114+0000","last_became_active":"2024-09-19T07:45:11.159220+0000","last_became_peered":"2024-09-19T07:45:11.159220+0000","last_unstale":"2024-09-19T07:45:12.168114+0000","last_undegraded":"2024-09-19T07:45:12.168114+0000","last_fullsized":"2024-09-19T07:45:12.168114+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:27:55.022291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.797030+0000","last_change":"2024-09-19T07:45:11.156372+0000","last_active":"2024-09-19T07:45:12.797030+0000","last_peered":"2024-09-19T07:45:12.797030+0000","last_clean":"2024-09-19T07:45:12.797030+0000","last_became_active":"2024-09-19T07:45:11.155909+0000","last_became_peered":"2024-09-19T07:45:11.155909+0000","last_unstale":"2024-09-19T07:45:12.797030+0000","last_undegraded":"2024-09-19T07:45:12.797030+0000","last_fullsized":"2024-09-19T07:45:12.797030+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:50.962826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139519+0000","last_change":"2024-09-19T07:45:11.162738+0000","last_active":"2024-09-19T07:45:12.139519+0000","last_peered":"2024-09-19T07:45:12.139519+0000","last_clean":"2024-09-19T07:45:12.139519+0000","last_became_active":"2024-09-19T07:45:11.161804+0000","last_became_peered":"2024-09-19T07:45:11.161804+0000","last_unstale":"2024-09-19T07:45:12.139519+0000","last_undegraded":"2024-09-19T07:45:12.139519+0000","last_fullsized":"2024-09-19T07:45:12.139519+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:50.592195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164801+0000","last_change":"2024-09-19T07:45:11.160689+0000","last_active":"2024-09-19T07:45:12.164801+0000","last_peered":"2024-09-19T07:45:12.164801+0000","last_clean":"2024-09-19T07:45:12.164801+0000","last_became_active":"2024-09-19T07:45:11.160400+0000","last_became_peered":"2024-09-19T07:45:11.160400+0000","last_unstale":"2024-09-19T07:45:12.164801+0000","last_undegraded":"2024-09-19T07:45:12.164801+0000","last_fullsized":"2024-09-19T07:45:12.164801+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:48.149096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226265+0000","last_change":"2024-09-19T07:45:11.151579+0000","last_active":"2024-09-19T07:45:12.226265+0000","last_peered":"2024-09-19T07:45:12.226265+0000","last_clean":"2024-09-19T07:45:12.226265+0000","last_became_active":"2024-09-19T07:45:11.150845+0000","last_became_peered":"2024-09-19T07:45:11.150845+0000","last_unstale":"2024-09-19T07:45:12.226265+0000","last_undegraded":"2024-09-19T07:45:12.226265+0000","last_fullsized":"2024-09-19T07:45:12.226265+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:51.235254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226274+0000","last_change":"2024-09-19T07:45:11.173152+0000","last_active":"2024-09-19T07:45:12.226274+0000","last_peered":"2024-09-19T07:45:12.226274+0000","last_clean":"2024-09-19T07:45:12.226274+0000","last_became_active":"2024-09-19T07:45:11.172874+0000","last_became_peered":"2024-09-19T07:45:11.172874+0000","last_unstale":"2024-09-19T07:45:12.226274+0000","last_undegraded":"2024-09-19T07:45:12.226274+0000","last_fullsized":"2024-09-19T07:45:12.226274+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:02.608766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139884+0000","last_change":"2024-09-19T07:45:11.151003+0000","last_active":"2024-09-19T07:45:12.139884+0000","last_peered":"2024-09-19T07:45:12.139884+0000","last_clean":"2024-09-19T07:45:12.139884+0000","last_became_active":"2024-09-19T07:45:11.150660+0000","last_became_peered":"2024-09-19T07:45:11.150660+0000","last_unstale":"2024-09-19T07:45:12.139884+0000","last_undegraded":"2024-09-19T07:45:12.139884+0000","last_fullsized":"2024-09-19T07:45:12.139884+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:42.526073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-19T07:45:11.161675+0000","last_change":"2024-09-19T07:45:11.160870+0000","last_active":"2024-09-19T07:45:11.161675+0000","last_peered":"2024-09-19T07:45:11.161675+0000","last_clean":"2024-09-19T07:45:11.161675+0000","last_became_active":"2024-09-19T07:45:11.160605+0000","last_became_peered":"2024-09-19T07:45:11.160605+0000","last_unstale":"2024-09-19T07:45:11.161675+0000","last_undegraded":"2024-09-19T07:45:11.161675+0000","last_fullsized":"2024-09-19T07:45:11.161675+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:47.872850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226222+0000","last_change":"2024-09-19T07:45:11.153833+0000","last_active":"2024-09-19T07:45:12.226222+0000","last_peered":"2024-09-19T07:45:12.226222+0000","last_clean":"2024-09-19T07:45:12.226222+0000","last_became_active":"2024-09-19T07:45:11.152953+0000","last_became_peered":"2024-09-19T07:45:11.152953+0000","last_unstale":"2024-09-19T07:45:12.226222+0000","last_undegraded":"2024-09-19T07:45:12.226222+0000","last_fullsized":"2024-09-19T07:45:12.226222+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:23.640090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139608+0000","last_change":"2024-09-19T07:45:11.152517+0000","last_active":"2024-09-19T07:45:12.139608+0000","last_peered":"2024-09-19T07:45:12.139608+0000","last_clean":"2024-09-19T07:45:12.139608+0000","last_became_active":"2024-09-19T07:45:11.152197+0000","last_became_peered":"2024-09-19T07:45:11.152197+0000","last_unstale":"2024-09-19T07:45:12.139608+0000","last_undegraded":"2024-09-19T07:45:12.139608+0000","last_fullsized":"2024-09-19T07:45:12.139608+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:17.354756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164882+0000","last_change":"2024-09-19T07:45:11.170619+0000","last_active":"2024-09-19T07:45:12.164882+0000","last_peered":"2024-09-19T07:45:12.164882+0000","last_clean":"2024-09-19T07:45:12.164882+0000","last_became_active":"2024-09-19T07:45:11.169189+0000","last_became_peered":"2024-09-19T07:45:11.169189+0000","last_unstale":"2024-09-19T07:45:12.164882+0000","last_undegraded":"2024-09-19T07:45:12.164882+0000","last_fullsized":"2024-09-19T07:45:12.164882+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:35.156705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.167938+0000","last_change":"2024-09-19T07:45:11.159503+0000","last_active":"2024-09-19T07:45:12.167938+0000","last_peered":"2024-09-19T07:45:12.167938+0000","last_clean":"2024-09-19T07:45:12.167938+0000","last_became_active":"2024-09-19T07:45:11.158846+0000","last_became_peered":"2024-09-19T07:45:11.158846+0000","last_unstale":"2024-09-19T07:45:12.167938+0000","last_undegraded":"2024-09-19T07:45:12.167938+0000","last_fullsized":"2024-09-19T07:45:12.167938+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:17.366751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139402+0000","last_change":"2024-09-19T07:45:11.151033+0000","last_active":"2024-09-19T07:45:12.139402+0000","last_peered":"2024-09-19T07:45:12.139402+0000","last_clean":"2024-09-19T07:45:12.139402+0000","last_became_active":"2024-09-19T07:45:11.150684+0000","last_became_peered":"2024-09-19T07:45:11.150684+0000","last_unstale":"2024-09-19T07:45:12.139402+0000","last_undegraded":"2024-09-19T07:45:12.139402+0000","last_fullsized":"2024-09-19T07:45:12.139402+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:26:58.382817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.165376+0000","last_change":"2024-09-19T07:45:11.173239+0000","last_active":"2024-09-19T07:45:12.165376+0000","last_peered":"2024-09-19T07:45:12.165376+0000","last_clean":"2024-09-19T07:45:12.165376+0000","last_became_active":"2024-09-19T07:45:11.171542+0000","last_became_peered":"2024-09-19T07:45:11.171542+0000","last_unstale":"2024-09-19T07:45:12.165376+0000","last_undegraded":"2024-09-19T07:45:12.165376+0000","last_fullsized":"2024-09-19T07:45:12.165376+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:31.214285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.168254+0000","last_change":"2024-09-19T07:45:11.158729+0000","last_active":"2024-09-19T07:45:12.168254+0000","last_peered":"2024-09-19T07:45:12.168254+0000","last_clean":"2024-09-19T07:45:12.168254+0000","last_became_active":"2024-09-19T07:45:11.158441+0000","last_became_peered":"2024-09-19T07:45:11.158441+0000","last_unstale":"2024-09-19T07:45:12.168254+0000","last_undegraded":"2024-09-19T07:45:12.168254+0000","last_fullsized":"2024-09-19T07:45:12.168254+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:08:39.515448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.165003+0000","last_change":"2024-09-19T07:45:11.168043+0000","last_active":"2024-09-19T07:45:12.165003+0000","last_peered":"2024-09-19T07:45:12.165003+0000","last_clean":"2024-09-19T07:45:12.165003+0000","last_became_active":"2024-09-19T07:45:11.165051+0000","last_became_peered":"2024-09-19T07:45:11.165051+0000","last_unstale":"2024-09-19T07:45:12.165003+0000","last_undegraded":"2024-09-19T07:45:12.165003+0000","last_fullsized":"2024-09-19T07:45:12.165003+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:03.671841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143791+0000","last_change":"2024-09-19T07:45:11.151488+0000","last_active":"2024-09-19T07:45:12.143791+0000","last_peered":"2024-09-19T07:45:12.143791+0000","last_clean":"2024-09-19T07:45:12.143791+0000","last_became_active":"2024-09-19T07:45:11.150944+0000","last_became_peered":"2024-09-19T07:45:11.150944+0000","last_unstale":"2024-09-19T07:45:12.143791+0000","last_undegraded":"2024-09-19T07:45:12.143791+0000","last_fullsized":"2024-09-19T07:45:12.143791+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:50.962826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143666+0000","last_change":"2024-09-19T07:45:11.156079+0000","last_active":"2024-09-19T07:45:12.143666+0000","last_peered":"2024-09-19T07:45:12.143666+0000","last_clean":"2024-09-19T07:45:12.143666+0000","last_became_active":"2024-09-19T07:45:11.155798+0000","last_became_peered":"2024-09-19T07:45:11.155798+0000","last_unstale":"2024-09-19T07:45:12.143666+0000","last_undegraded":"2024-09-19T07:45:12.143666+0000","last_fullsized":"2024-09-19T07:45:12.143666+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:55.980609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.225887+0000","last_change":"2024-09-19T07:45:11.176023+0000","last_active":"2024-09-19T07:45:12.225887+0000","last_peered":"2024-09-19T07:45:12.225887+0000","last_clean":"2024-09-19T07:45:12.225887+0000","last_became_active":"2024-09-19T07:45:11.175693+0000","last_became_peered":"2024-09-19T07:45:11.175693+0000","last_unstale":"2024-09-19T07:45:12.225887+0000","last_undegraded":"2024-09-19T07:45:12.225887+0000","last_fullsized":"2024-09-19T07:45:12.225887+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:29:29.101102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-19T07:45:11.161502+0000","last_change":"2024-09-19T07:45:11.160207+0000","last_active":"2024-09-19T07:45:11.161502+0000","last_peered":"2024-09-19T07:45:11.161502+0000","last_clean":"2024-09-19T07:45:11.161502+0000","last_became_active":"2024-09-19T07:45:11.159891+0000","last_became_peered":"2024-09-19T07:45:11.159891+0000","last_unstale":"2024-09-19T07:45:11.161502+0000","last_undegraded":"2024-09-19T07:45:11.161502+0000","last_fullsized":"2024-09-19T07:45:11.161502+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:41.004093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164543+0000","last_change":"2024-09-19T07:45:11.159414+0000","last_active":"2024-09-19T07:45:12.164543+0000","last_peered":"2024-09-19T07:45:12.164543+0000","last_clean":"2024-09-19T07:45:12.164543+0000","last_became_active":"2024-09-19T07:45:11.159046+0000","last_became_peered":"2024-09-19T07:45:11.159046+0000","last_unstale":"2024-09-19T07:45:12.164543+0000","last_undegraded":"2024-09-19T07:45:12.164543+0000","last_fullsized":"2024-09-19T07:45:12.164543+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:09.128499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.165440+0000","last_change":"2024-09-19T07:45:11.172493+0000","last_active":"2024-09-19T07:45:12.165440+0000","last_peered":"2024-09-19T07:45:12.165440+0000","last_clean":"2024-09-19T07:45:12.165440+0000","last_became_active":"2024-09-19T07:45:11.171240+0000","last_became_peered":"2024-09-19T07:45:11.171240+0000","last_unstale":"2024-09-19T07:45:12.165440+0000","last_undegraded":"2024-09-19T07:45:12.165440+0000","last_fullsized":"2024-09-19T07:45:12.165440+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:18.448468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226078+0000","last_change":"2024-09-19T07:45:11.147813+0000","last_active":"2024-09-19T07:45:12.226078+0000","last_peered":"2024-09-19T07:45:12.226078+0000","last_clean":"2024-09-19T07:45:12.226078+0000","last_became_active":"2024-09-19T07:45:11.147493+0000","last_became_peered":"2024-09-19T07:45:11.147493+0000","last_unstale":"2024-09-19T07:45:12.226078+0000","last_undegraded":"2024-09-19T07:45:12.226078+0000","last_fullsized":"2024-09-19T07:45:12.226078+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:11.709408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139984+0000","last_change":"2024-09-19T07:45:11.166695+0000","last_active":"2024-09-19T07:45:12.139984+0000","last_peered":"2024-09-19T07:45:12.139984+0000","last_clean":"2024-09-19T07:45:12.139984+0000","last_became_active":"2024-09-19T07:45:11.166318+0000","last_became_peered":"2024-09-19T07:45:11.166318+0000","last_unstale":"2024-09-19T07:45:12.139984+0000","last_undegraded":"2024-09-19T07:45:12.139984+0000","last_fullsized":"2024-09-19T07:45:12.139984+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:40.863237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143846+0000","last_change":"2024-09-19T07:45:11.147522+0000","last_active":"2024-09-19T07:45:12.143846+0000","last_peered":"2024-09-19T07:45:12.143846+0000","last_clean":"2024-09-19T07:45:12.143846+0000","last_became_active":"2024-09-19T07:45:11.147239+0000","last_became_peered":"2024-09-19T07:45:11.147239+0000","last_unstale":"2024-09-19T07:45:12.143846+0000","last_undegraded":"2024-09-19T07:45:12.143846+0000","last_fullsized":"2024-09-19T07:45:12.143846+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:15.960700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164943+0000","last_change":"2024-09-19T07:45:11.168884+0000","last_active":"2024-09-19T07:45:12.164943+0000","last_peered":"2024-09-19T07:45:12.164943+0000","last_clean":"2024-09-19T07:45:12.164943+0000","last_became_active":"2024-09-19T07:45:11.168583+0000","last_became_peered":"2024-09-19T07:45:11.168583+0000","last_unstale":"2024-09-19T07:45:12.164943+0000","last_undegraded":"2024-09-19T07:45:12.164943+0000","last_fullsized":"2024-09-19T07:45:12.164943+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:08:39.515448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.167889+0000","last_change":"2024-09-19T07:45:11.160283+0000","last_active":"2024-09-19T07:45:12.167889+0000","last_peered":"2024-09-19T07:45:12.167889+0000","last_clean":"2024-09-19T07:45:12.167889+0000","last_became_active":"2024-09-19T07:45:11.159399+0000","last_became_peered":"2024-09-19T07:45:11.159399+0000","last_unstale":"2024-09-19T07:45:12.167889+0000","last_undegraded":"2024-09-19T07:45:12.167889+0000","last_fullsized":"2024-09-19T07:45:12.167889+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:47.682310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164703+0000","last_change":"2024-09-19T07:45:11.162321+0000","last_active":"2024-09-19T07:45:12.164703+0000","last_peered":"2024-09-19T07:45:12.164703+0000","last_clean":"2024-09-19T07:45:12.164703+0000","last_became_active":"2024-09-19T07:45:11.159890+0000","last_became_peered":"2024-09-19T07:45:11.159890+0000","last_unstale":"2024-09-19T07:45:12.164703+0000","last_undegraded":"2024-09-19T07:45:12.164703+0000","last_fullsized":"2024-09-19T07:45:12.164703+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:47.682310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.144721+0000","last_change":"2024-09-19T07:45:11.154147+0000","last_active":"2024-09-19T07:45:12.144721+0000","last_peered":"2024-09-19T07:45:12.144721+0000","last_clean":"2024-09-19T07:45:12.144721+0000","last_became_active":"2024-09-19T07:45:11.153578+0000","last_became_peered":"2024-09-19T07:45:11.153578+0000","last_unstale":"2024-09-19T07:45:12.144721+0000","last_undegraded":"2024-09-19T07:45:12.144721+0000","last_fullsized":"2024-09-19T07:45:12.144721+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:08:39.515448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.167799+0000","last_change":"2024-09-19T07:45:11.158400+0000","last_active":"2024-09-19T07:45:12.167799+0000","last_peered":"2024-09-19T07:45:12.167799+0000","last_clean":"2024-09-19T07:45:12.167799+0000","last_became_active":"2024-09-19T07:45:11.158143+0000","last_became_peered":"2024-09-19T07:45:11.158143+0000","last_unstale":"2024-09-19T07:45:12.167799+0000","last_undegraded":"2024-09-19T07:45:12.167799+0000","last_fullsized":"2024-09-19T07:45:12.167799+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:48.149096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"52'4","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143707+0000","last_change":"2024-09-19T07:45:10.142735+0000","last_active":"2024-09-19T07:45:12.143707+0000","last_peered":"2024-09-19T07:45:12.143707+0000","last_clean":"2024-09-19T07:45:12.143707+0000","last_became_active":"2024-09-19T07:45:10.141359+0000","last_became_peered":"2024-09-19T07:45:10.141359+0000","last_unstale":"2024-09-19T07:45:12.143707+0000","last_undegraded":"2024-09-19T07:45:12.143707+0000","last_fullsized":"2024-09-19T07:45:12.143707+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:35.509895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.144790+0000","last_change":"2024-09-19T07:45:11.156059+0000","last_active":"2024-09-19T07:45:12.144790+0000","last_peered":"2024-09-19T07:45:12.144790+0000","last_clean":"2024-09-19T07:45:12.144790+0000","last_became_active":"2024-09-19T07:45:11.155772+0000","last_became_peered":"2024-09-19T07:45:11.155772+0000","last_unstale":"2024-09-19T07:45:12.144790+0000","last_undegraded":"2024-09-19T07:45:12.144790+0000","last_fullsized":"2024-09-19T07:45:12.144790+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:03.671841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139876+0000","last_change":"2024-09-19T07:45:10.145766+0000","last_active":"2024-09-19T07:45:12.139876+0000","last_peered":"2024-09-19T07:45:12.139876+0000","last_clean":"2024-09-19T07:45:12.139876+0000","last_became_active":"2024-09-19T07:45:10.142733+0000","last_became_peered":"2024-09-19T07:45:10.142733+0000","last_unstale":"2024-09-19T07:45:12.139876+0000","last_undegraded":"2024-09-19T07:45:12.139876+0000","last_fullsized":"2024-09-19T07:45:12.139876+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:16.454219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143644+0000","last_change":"2024-09-19T07:45:11.156103+0000","last_active":"2024-09-19T07:45:12.143644+0000","last_peered":"2024-09-19T07:45:12.143644+0000","last_clean":"2024-09-19T07:45:12.143644+0000","last_became_active":"2024-09-19T07:45:11.155821+0000","last_became_peered":"2024-09-19T07:45:11.155821+0000","last_unstale":"2024-09-19T07:45:12.143644+0000","last_undegraded":"2024-09-19T07:45:12.143644+0000","last_fullsized":"2024-09-19T07:45:12.143644+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:38.636372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139825+0000","last_change":"2024-09-19T07:45:10.147013+0000","last_active":"2024-09-19T07:45:12.139825+0000","last_peered":"2024-09-19T07:45:12.139825+0000","last_clean":"2024-09-19T07:45:12.139825+0000","last_became_active":"2024-09-19T07:45:10.146032+0000","last_became_peered":"2024-09-19T07:45:10.146032+0000","last_unstale":"2024-09-19T07:45:12.139825+0000","last_undegraded":"2024-09-19T07:45:12.139825+0000","last_fullsized":"2024-09-19T07:45:12.139825+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:26.904212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.225998+0000","last_change":"2024-09-19T07:45:11.148263+0000","last_active":"2024-09-19T07:45:12.225998+0000","last_peered":"2024-09-19T07:45:12.225998+0000","last_clean":"2024-09-19T07:45:12.225998+0000","last_became_active":"2024-09-19T07:45:11.147983+0000","last_became_peered":"2024-09-19T07:45:11.147983+0000","last_unstale":"2024-09-19T07:45:12.225998+0000","last_undegraded":"2024-09-19T07:45:12.225998+0000","last_fullsized":"2024-09-19T07:45:12.225998+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:53:53.137209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.144739+0000","last_change":"2024-09-19T07:45:10.133632+0000","last_active":"2024-09-19T07:45:12.144739+0000","last_peered":"2024-09-19T07:45:12.144739+0000","last_clean":"2024-09-19T07:45:12.144739+0000","last_became_active":"2024-09-19T07:45:10.133328+0000","last_became_peered":"2024-09-19T07:45:10.133328+0000","last_unstale":"2024-09-19T07:45:12.144739+0000","last_undegraded":"2024-09-19T07:45:12.144739+0000","last_fullsized":"2024-09-19T07:45:12.144739+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:41.519074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.797139+0000","last_change":"2024-09-19T07:45:11.171607+0000","last_active":"2024-09-19T07:45:12.797139+0000","last_peered":"2024-09-19T07:45:12.797139+0000","last_clean":"2024-09-19T07:45:12.797139+0000","last_became_active":"2024-09-19T07:45:11.171351+0000","last_became_peered":"2024-09-19T07:45:11.171351+0000","last_unstale":"2024-09-19T07:45:12.797139+0000","last_undegraded":"2024-09-19T07:45:12.797139+0000","last_fullsized":"2024-09-19T07:45:12.797139+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:15.960700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164726+0000","last_change":"2024-09-19T07:45:10.141444+0000","last_active":"2024-09-19T07:45:12.164726+0000","last_peered":"2024-09-19T07:45:12.164726+0000","last_clean":"2024-09-19T07:45:12.164726+0000","last_became_active":"2024-09-19T07:45:10.141168+0000","last_became_peered":"2024-09-19T07:45:10.141168+0000","last_unstale":"2024-09-19T07:45:12.164726+0000","last_undegraded":"2024-09-19T07:45:12.164726+0000","last_fullsized":"2024-09-19T07:45:12.164726+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:54.973611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.168090+0000","last_change":"2024-09-19T07:45:11.160433+0000","last_active":"2024-09-19T07:45:12.168090+0000","last_peered":"2024-09-19T07:45:12.168090+0000","last_clean":"2024-09-19T07:45:12.168090+0000","last_became_active":"2024-09-19T07:45:11.159020+0000","last_became_peered":"2024-09-19T07:45:11.159020+0000","last_unstale":"2024-09-19T07:45:12.168090+0000","last_undegraded":"2024-09-19T07:45:12.168090+0000","last_fullsized":"2024-09-19T07:45:12.168090+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:30:53.075010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.796815+0000","last_change":"2024-09-19T07:45:10.141409+0000","last_active":"2024-09-19T07:45:12.796815+0000","last_peered":"2024-09-19T07:45:12.796815+0000","last_clean":"2024-09-19T07:45:12.796815+0000","last_became_active":"2024-09-19T07:45:10.140876+0000","last_became_peered":"2024-09-19T07:45:10.140876+0000","last_unstale":"2024-09-19T07:45:12.796815+0000","last_undegraded":"2024-09-19T07:45:12.796815+0000","last_fullsized":"2024-09-19T07:45:12.796815+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:16.270559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226226+0000","last_change":"2024-09-19T07:45:11.147187+0000","last_active":"2024-09-19T07:45:12.226226+0000","last_peered":"2024-09-19T07:45:12.226226+0000","last_clean":"2024-09-19T07:45:12.226226+0000","last_became_active":"2024-09-19T07:45:11.146894+0000","last_became_peered":"2024-09-19T07:45:11.146894+0000","last_unstale":"2024-09-19T07:45:12.226226+0000","last_undegraded":"2024-09-19T07:45:12.226226+0000","last_fullsized":"2024-09-19T07:45:12.226226+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:12.008322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143494+0000","last_change":"2024-09-19T07:45:10.143967+0000","last_active":"2024-09-19T07:45:12.143494+0000","last_peered":"2024-09-19T07:45:12.143494+0000","last_clean":"2024-09-19T07:45:12.143494+0000","last_became_active":"2024-09-19T07:45:10.143694+0000","last_became_peered":"2024-09-19T07:45:10.143694+0000","last_unstale":"2024-09-19T07:45:12.143494+0000","last_undegraded":"2024-09-19T07:45:12.143494+0000","last_fullsized":"2024-09-19T07:45:12.143494+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:42.780920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.168236+0000","last_change":"2024-09-19T07:45:11.174590+0000","last_active":"2024-09-19T07:45:12.168236+0000","last_peered":"2024-09-19T07:45:12.168236+0000","last_clean":"2024-09-19T07:45:12.168236+0000","last_became_active":"2024-09-19T07:45:11.174317+0000","last_became_peered":"2024-09-19T07:45:11.174317+0000","last_unstale":"2024-09-19T07:45:12.168236+0000","last_undegraded":"2024-09-19T07:45:12.168236+0000","last_fullsized":"2024-09-19T07:45:12.168236+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:35.156705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164540+0000","last_change":"2024-09-19T07:45:10.146821+0000","last_active":"2024-09-19T07:45:12.164540+0000","last_peered":"2024-09-19T07:45:12.164540+0000","last_clean":"2024-09-19T07:45:12.164540+0000","last_became_active":"2024-09-19T07:45:10.145946+0000","last_became_peered":"2024-09-19T07:45:10.145946+0000","last_unstale":"2024-09-19T07:45:12.164540+0000","last_undegraded":"2024-09-19T07:45:12.164540+0000","last_fullsized":"2024-09-19T07:45:12.164540+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:37.629374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.796839+0000","last_change":"2024-09-19T07:45:11.149201+0000","last_active":"2024-09-19T07:45:12.796839+0000","last_peered":"2024-09-19T07:45:12.796839+0000","last_clean":"2024-09-19T07:45:12.796839+0000","last_became_active":"2024-09-19T07:45:11.148873+0000","last_became_peered":"2024-09-19T07:45:11.148873+0000","last_unstale":"2024-09-19T07:45:12.796839+0000","last_undegraded":"2024-09-19T07:45:12.796839+0000","last_fullsized":"2024-09-19T07:45:12.796839+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:38.636372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.168113+0000","last_change":"2024-09-19T07:45:10.142038+0000","last_active":"2024-09-19T07:45:12.168113+0000","last_peered":"2024-09-19T07:45:12.168113+0000","last_clean":"2024-09-19T07:45:12.168113+0000","last_became_active":"2024-09-19T07:45:10.141756+0000","last_became_peered":"2024-09-19T07:45:10.141756+0000","last_unstale":"2024-09-19T07:45:12.168113+0000","last_undegraded":"2024-09-19T07:45:12.168113+0000","last_fullsized":"2024-09-19T07:45:12.168113+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:26:57.375819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.144594+0000","last_change":"2024-09-19T07:45:11.150745+0000","last_active":"2024-09-19T07:45:12.144594+0000","last_peered":"2024-09-19T07:45:12.144594+0000","last_clean":"2024-09-19T07:45:12.144594+0000","last_became_active":"2024-09-19T07:45:11.150307+0000","last_became_peered":"2024-09-19T07:45:11.150307+0000","last_unstale":"2024-09-19T07:45:12.144594+0000","last_undegraded":"2024-09-19T07:45:12.144594+0000","last_fullsized":"2024-09-19T07:45:12.144594+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:30:53.075010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.796824+0000","last_change":"2024-09-19T07:45:10.144207+0000","last_active":"2024-09-19T07:45:12.796824+0000","last_peered":"2024-09-19T07:45:12.796824+0000","last_clean":"2024-09-19T07:45:12.796824+0000","last_became_active":"2024-09-19T07:45:10.143382+0000","last_became_peered":"2024-09-19T07:45:10.143382+0000","last_unstale":"2024-09-19T07:45:12.796824+0000","last_undegraded":"2024-09-19T07:45:12.796824+0000","last_fullsized":"2024-09-19T07:45:12.796824+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:41.070599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164845+0000","last_change":"2024-09-19T07:45:11.170934+0000","last_active":"2024-09-19T07:45:12.164845+0000","last_peered":"2024-09-19T07:45:12.164845+0000","last_clean":"2024-09-19T07:45:12.164845+0000","last_became_active":"2024-09-19T07:45:11.168811+0000","last_became_peered":"2024-09-19T07:45:11.168811+0000","last_unstale":"2024-09-19T07:45:12.164845+0000","last_undegraded":"2024-09-19T07:45:12.164845+0000","last_fullsized":"2024-09-19T07:45:12.164845+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:27:55.022291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"52'3","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.168155+0000","last_change":"2024-09-19T07:45:10.149035+0000","last_active":"2024-09-19T07:45:12.168155+0000","last_peered":"2024-09-19T07:45:12.168155+0000","last_clean":"2024-09-19T07:45:12.168155+0000","last_became_active":"2024-09-19T07:45:10.148748+0000","last_became_peered":"2024-09-19T07:45:10.148748+0000","last_unstale":"2024-09-19T07:45:12.168155+0000","last_undegraded":"2024-09-19T07:45:12.168155+0000","last_fullsized":"2024-09-19T07:45:12.168155+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:16.459646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.797086+0000","last_change":"2024-09-19T07:45:11.150255+0000","last_active":"2024-09-19T07:45:12.797086+0000","last_peered":"2024-09-19T07:45:12.797086+0000","last_clean":"2024-09-19T07:45:12.797086+0000","last_became_active":"2024-09-19T07:45:11.149834+0000","last_became_peered":"2024-09-19T07:45:11.149834+0000","last_unstale":"2024-09-19T07:45:12.797086+0000","last_undegraded":"2024-09-19T07:45:12.797086+0000","last_fullsized":"2024-09-19T07:45:12.797086+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:48.790896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"52'6","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226249+0000","last_change":"2024-09-19T07:45:10.141653+0000","last_active":"2024-09-19T07:45:12.226249+0000","last_peered":"2024-09-19T07:45:12.226249+0000","last_clean":"2024-09-19T07:45:12.226249+0000","last_became_active":"2024-09-19T07:45:10.141023+0000","last_became_peered":"2024-09-19T07:45:10.141023+0000","last_unstale":"2024-09-19T07:45:12.226249+0000","last_undegraded":"2024-09-19T07:45:12.226249+0000","last_fullsized":"2024-09-19T07:45:12.226249+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:46.675311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.144572+0000","last_change":"2024-09-19T07:45:11.160328+0000","last_active":"2024-09-19T07:45:12.144572+0000","last_peered":"2024-09-19T07:45:12.144572+0000","last_clean":"2024-09-19T07:45:12.144572+0000","last_became_active":"2024-09-19T07:45:11.160012+0000","last_became_peered":"2024-09-19T07:45:11.160012+0000","last_unstale":"2024-09-19T07:45:12.144572+0000","last_undegraded":"2024-09-19T07:45:12.144572+0000","last_fullsized":"2024-09-19T07:45:12.144572+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:27:55.022291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164861+0000","last_change":"2024-09-19T07:45:10.143597+0000","last_active":"2024-09-19T07:45:12.164861+0000","last_peered":"2024-09-19T07:45:12.164861+0000","last_clean":"2024-09-19T07:45:12.164861+0000","last_became_active":"2024-09-19T07:45:10.143337+0000","last_became_peered":"2024-09-19T07:45:10.143337+0000","last_unstale":"2024-09-19T07:45:12.164861+0000","last_undegraded":"2024-09-19T07:45:12.164861+0000","last_fullsized":"2024-09-19T07:45:12.164861+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:49.955828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.165116+0000","last_change":"2024-09-19T07:45:11.165460+0000","last_active":"2024-09-19T07:45:12.165116+0000","last_peered":"2024-09-19T07:45:12.165116+0000","last_clean":"2024-09-19T07:45:12.165116+0000","last_became_active":"2024-09-19T07:45:11.164722+0000","last_became_peered":"2024-09-19T07:45:11.164722+0000","last_unstale":"2024-09-19T07:45:12.165116+0000","last_undegraded":"2024-09-19T07:45:12.165116+0000","last_fullsized":"2024-09-19T07:45:12.165116+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:25.374821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143598+0000","last_change":"2024-09-19T07:45:10.149191+0000","last_active":"2024-09-19T07:45:12.143598+0000","last_peered":"2024-09-19T07:45:12.143598+0000","last_clean":"2024-09-19T07:45:12.143598+0000","last_became_active":"2024-09-19T07:45:10.148886+0000","last_became_peered":"2024-09-19T07:45:10.148886+0000","last_unstale":"2024-09-19T07:45:12.143598+0000","last_undegraded":"2024-09-19T07:45:12.143598+0000","last_fullsized":"2024-09-19T07:45:12.143598+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:25.675380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226125+0000","last_change":"2024-09-19T07:45:11.161322+0000","last_active":"2024-09-19T07:45:12.226125+0000","last_peered":"2024-09-19T07:45:12.226125+0000","last_clean":"2024-09-19T07:45:12.226125+0000","last_became_active":"2024-09-19T07:45:11.160672+0000","last_became_peered":"2024-09-19T07:45:11.160672+0000","last_unstale":"2024-09-19T07:45:12.226125+0000","last_undegraded":"2024-09-19T07:45:12.226125+0000","last_fullsized":"2024-09-19T07:45:12.226125+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:41.266322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.167959+0000","last_change":"2024-09-19T07:45:10.143903+0000","last_active":"2024-09-19T07:45:12.167959+0000","last_peered":"2024-09-19T07:45:12.167959+0000","last_clean":"2024-09-19T07:45:12.167959+0000","last_became_active":"2024-09-19T07:45:10.143552+0000","last_became_peered":"2024-09-19T07:45:10.143552+0000","last_unstale":"2024-09-19T07:45:12.167959+0000","last_undegraded":"2024-09-19T07:45:12.167959+0000","last_fullsized":"2024-09-19T07:45:12.167959+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:16.347757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.165305+0000","last_change":"2024-09-19T07:45:11.172882+0000","last_active":"2024-09-19T07:45:12.165305+0000","last_peered":"2024-09-19T07:45:12.165305+0000","last_clean":"2024-09-19T07:45:12.165305+0000","last_became_active":"2024-09-19T07:45:11.172243+0000","last_became_peered":"2024-09-19T07:45:11.172243+0000","last_unstale":"2024-09-19T07:45:12.165305+0000","last_undegraded":"2024-09-19T07:45:12.165305+0000","last_fullsized":"2024-09-19T07:45:12.165305+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:21:42.341500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"52'8","reported_seq":22,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226137+0000","last_change":"2024-09-19T07:45:10.141231+0000","last_active":"2024-09-19T07:45:12.226137+0000","last_peered":"2024-09-19T07:45:12.226137+0000","last_clean":"2024-09-19T07:45:12.226137+0000","last_became_active":"2024-09-19T07:45:10.140945+0000","last_became_peered":"2024-09-19T07:45:10.140945+0000","last_unstale":"2024-09-19T07:45:12.226137+0000","last_undegraded":"2024-09-19T07:45:12.226137+0000","last_fullsized":"2024-09-19T07:45:12.226137+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:34.149707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226029+0000","last_change":"2024-09-19T07:45:11.151822+0000","last_active":"2024-09-19T07:45:12.226029+0000","last_peered":"2024-09-19T07:45:12.226029+0000","last_clean":"2024-09-19T07:45:12.226029+0000","last_became_active":"2024-09-19T07:45:11.151224+0000","last_became_peered":"2024-09-19T07:45:11.151224+0000","last_unstale":"2024-09-19T07:45:12.226029+0000","last_undegraded":"2024-09-19T07:45:12.226029+0000","last_fullsized":"2024-09-19T07:45:12.226029+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:46:36.919958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.165425+0000","last_change":"2024-09-19T07:45:10.149077+0000","last_active":"2024-09-19T07:45:12.165425+0000","last_peered":"2024-09-19T07:45:12.165425+0000","last_clean":"2024-09-19T07:45:12.165425+0000","last_became_active":"2024-09-19T07:45:10.148745+0000","last_became_peered":"2024-09-19T07:45:10.148745+0000","last_unstale":"2024-09-19T07:45:12.165425+0000","last_undegraded":"2024-09-19T07:45:12.165425+0000","last_fullsized":"2024-09-19T07:45:12.165425+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:03.332021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139418+0000","last_change":"2024-09-19T07:45:11.156578+0000","last_active":"2024-09-19T07:45:12.139418+0000","last_peered":"2024-09-19T07:45:12.139418+0000","last_clean":"2024-09-19T07:45:12.139418+0000","last_became_active":"2024-09-19T07:45:11.156292+0000","last_became_peered":"2024-09-19T07:45:11.156292+0000","last_unstale":"2024-09-19T07:45:12.139418+0000","last_undegraded":"2024-09-19T07:45:12.139418+0000","last_fullsized":"2024-09-19T07:45:12.139418+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:17.466645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"52'2","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-19T07:45:11.177482+0000","last_change":"2024-09-19T07:45:10.133568+0000","last_active":"2024-09-19T07:45:11.177482+0000","last_peered":"2024-09-19T07:45:11.177482+0000","last_clean":"2024-09-19T07:45:11.177482+0000","last_became_active":"2024-09-19T07:45:10.133255+0000","last_became_peered":"2024-09-19T07:45:10.133255+0000","last_unstale":"2024-09-19T07:45:11.177482+0000","last_undegraded":"2024-09-19T07:45:11.177482+0000","last_fullsized":"2024-09-19T07:45:11.177482+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:54:35.081711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.796965+0000","last_change":"2024-09-19T07:45:11.171801+0000","last_active":"2024-09-19T07:45:12.796965+0000","last_peered":"2024-09-19T07:45:12.796965+0000","last_clean":"2024-09-19T07:45:12.796965+0000","last_became_active":"2024-09-19T07:45:11.171531+0000","last_became_peered":"2024-09-19T07:45:11.171531+0000","last_unstale":"2024-09-19T07:45:12.796965+0000","last_undegraded":"2024-09-19T07:45:12.796965+0000","last_fullsized":"2024-09-19T07:45:12.796965+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:00:55.980609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143914+0000","last_change":"2024-09-19T07:45:10.142185+0000","last_active":"2024-09-19T07:45:12.143914+0000","last_peered":"2024-09-19T07:45:12.143914+0000","last_clean":"2024-09-19T07:45:12.143914+0000","last_became_active":"2024-09-19T07:45:10.141823+0000","last_became_peered":"2024-09-19T07:45:10.141823+0000","last_unstale":"2024-09-19T07:45:12.143914+0000","last_undegraded":"2024-09-19T07:45:12.143914+0000","last_fullsized":"2024-09-19T07:45:12.143914+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:08.776990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139620+0000","last_change":"2024-09-19T07:45:11.144563+0000","last_active":"2024-09-19T07:45:12.139620+0000","last_peered":"2024-09-19T07:45:12.139620+0000","last_clean":"2024-09-19T07:45:12.139620+0000","last_became_active":"2024-09-19T07:45:11.144221+0000","last_became_peered":"2024-09-19T07:45:11.144221+0000","last_unstale":"2024-09-19T07:45:12.139620+0000","last_undegraded":"2024-09-19T07:45:12.139620+0000","last_fullsized":"2024-09-19T07:45:12.139620+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:52.729562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143855+0000","last_change":"2024-09-19T07:45:10.141010+0000","last_active":"2024-09-19T07:45:12.143855+0000","last_peered":"2024-09-19T07:45:12.143855+0000","last_clean":"2024-09-19T07:45:12.143855+0000","last_became_active":"2024-09-19T07:45:10.140747+0000","last_became_peered":"2024-09-19T07:45:10.140747+0000","last_unstale":"2024-09-19T07:45:12.143855+0000","last_undegraded":"2024-09-19T07:45:12.143855+0000","last_fullsized":"2024-09-19T07:45:12.143855+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:41.070599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.144791+0000","last_change":"2024-09-19T07:45:11.150879+0000","last_active":"2024-09-19T07:45:12.144791+0000","last_peered":"2024-09-19T07:45:12.144791+0000","last_clean":"2024-09-19T07:45:12.144791+0000","last_became_active":"2024-09-19T07:45:11.150562+0000","last_became_peered":"2024-09-19T07:45:11.150562+0000","last_unstale":"2024-09-19T07:45:12.144791+0000","last_undegraded":"2024-09-19T07:45:12.144791+0000","last_fullsized":"2024-09-19T07:45:12.144791+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:25.374821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139717+0000","last_change":"2024-09-19T07:45:10.140315+0000","last_active":"2024-09-19T07:45:12.139717+0000","last_peered":"2024-09-19T07:45:12.139717+0000","last_clean":"2024-09-19T07:45:12.139717+0000","last_became_active":"2024-09-19T07:45:10.140052+0000","last_became_peered":"2024-09-19T07:45:10.140052+0000","last_unstale":"2024-09-19T07:45:12.139717+0000","last_undegraded":"2024-09-19T07:45:12.139717+0000","last_fullsized":"2024-09-19T07:45:12.139717+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:55.554697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143851+0000","last_change":"2024-09-19T07:45:11.151218+0000","last_active":"2024-09-19T07:45:12.143851+0000","last_peered":"2024-09-19T07:45:12.143851+0000","last_clean":"2024-09-19T07:45:12.143851+0000","last_became_active":"2024-09-19T07:45:11.150938+0000","last_became_peered":"2024-09-19T07:45:11.150938+0000","last_unstale":"2024-09-19T07:45:12.143851+0000","last_undegraded":"2024-09-19T07:45:12.143851+0000","last_fullsized":"2024-09-19T07:45:12.143851+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:48.790896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.225818+0000","last_change":"2024-09-19T07:45:10.145203+0000","last_active":"2024-09-19T07:45:12.225818+0000","last_peered":"2024-09-19T07:45:12.225818+0000","last_clean":"2024-09-19T07:45:12.225818+0000","last_became_active":"2024-09-19T07:45:10.144489+0000","last_became_peered":"2024-09-19T07:45:10.144489+0000","last_unstale":"2024-09-19T07:45:12.225818+0000","last_undegraded":"2024-09-19T07:45:12.225818+0000","last_fullsized":"2024-09-19T07:45:12.225818+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:30:52.068011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.165188+0000","last_change":"2024-09-19T07:45:11.169319+0000","last_active":"2024-09-19T07:45:12.165188+0000","last_peered":"2024-09-19T07:45:12.165188+0000","last_clean":"2024-09-19T07:45:12.165188+0000","last_became_active":"2024-09-19T07:45:11.168698+0000","last_became_peered":"2024-09-19T07:45:11.168698+0000","last_unstale":"2024-09-19T07:45:12.165188+0000","last_undegraded":"2024-09-19T07:45:12.165188+0000","last_fullsized":"2024-09-19T07:45:12.165188+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:01.056784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164764+0000","last_change":"2024-09-19T07:45:10.142588+0000","last_active":"2024-09-19T07:45:12.164764+0000","last_peered":"2024-09-19T07:45:12.164764+0000","last_clean":"2024-09-19T07:45:12.164764+0000","last_became_active":"2024-09-19T07:45:10.142215+0000","last_became_peered":"2024-09-19T07:45:10.142215+0000","last_unstale":"2024-09-19T07:45:12.164764+0000","last_undegraded":"2024-09-19T07:45:12.164764+0000","last_fullsized":"2024-09-19T07:45:12.164764+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:08.197576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139881+0000","last_change":"2024-09-19T07:45:11.151059+0000","last_active":"2024-09-19T07:45:12.139881+0000","last_peered":"2024-09-19T07:45:12.139881+0000","last_clean":"2024-09-19T07:45:12.139881+0000","last_became_active":"2024-09-19T07:45:11.150738+0000","last_became_peered":"2024-09-19T07:45:11.150738+0000","last_unstale":"2024-09-19T07:45:12.139881+0000","last_undegraded":"2024-09-19T07:45:12.139881+0000","last_fullsized":"2024-09-19T07:45:12.139881+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:18.574474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-19T07:45:11.143869+0000","last_change":"2024-09-19T07:45:10.135637+0000","last_active":"2024-09-19T07:45:11.143869+0000","last_peered":"2024-09-19T07:45:11.143869+0000","last_clean":"2024-09-19T07:45:11.143869+0000","last_became_active":"2024-09-19T07:45:10.135359+0000","last_became_peered":"2024-09-19T07:45:10.135359+0000","last_unstale":"2024-09-19T07:45:11.143869+0000","last_undegraded":"2024-09-19T07:45:11.143869+0000","last_fullsized":"2024-09-19T07:45:11.143869+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:45.339997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.144729+0000","last_change":"2024-09-19T07:45:11.153930+0000","last_active":"2024-09-19T07:45:12.144729+0000","last_peered":"2024-09-19T07:45:12.144729+0000","last_clean":"2024-09-19T07:45:12.144729+0000","last_became_active":"2024-09-19T07:45:11.153222+0000","last_became_peered":"2024-09-19T07:45:11.153222+0000","last_unstale":"2024-09-19T07:45:12.144729+0000","last_undegraded":"2024-09-19T07:45:12.144729+0000","last_fullsized":"2024-09-19T07:45:12.144729+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:20.215194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143801+0000","last_change":"2024-09-19T07:45:10.149272+0000","last_active":"2024-09-19T07:45:12.143801+0000","last_peered":"2024-09-19T07:45:12.143801+0000","last_clean":"2024-09-19T07:45:12.143801+0000","last_became_active":"2024-09-19T07:45:10.148962+0000","last_became_peered":"2024-09-19T07:45:10.148962+0000","last_unstale":"2024-09-19T07:45:12.143801+0000","last_undegraded":"2024-09-19T07:45:12.143801+0000","last_fullsized":"2024-09-19T07:45:12.143801+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:26:43.107121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.139623+0000","last_change":"2024-09-19T07:45:11.146344+0000","last_active":"2024-09-19T07:45:12.139623+0000","last_peered":"2024-09-19T07:45:12.139623+0000","last_clean":"2024-09-19T07:45:12.139623+0000","last_became_active":"2024-09-19T07:45:11.146039+0000","last_became_peered":"2024-09-19T07:45:11.146039+0000","last_unstale":"2024-09-19T07:45:12.139623+0000","last_undegraded":"2024-09-19T07:45:12.139623+0000","last_fullsized":"2024-09-19T07:45:12.139623+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:08.219281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.168182+0000","last_change":"2024-09-19T07:45:10.143810+0000","last_active":"2024-09-19T07:45:12.168182+0000","last_peered":"2024-09-19T07:45:12.168182+0000","last_clean":"2024-09-19T07:45:12.168182+0000","last_became_active":"2024-09-19T07:45:10.143462+0000","last_became_peered":"2024-09-19T07:45:10.143462+0000","last_unstale":"2024-09-19T07:45:12.168182+0000","last_undegraded":"2024-09-19T07:45:12.168182+0000","last_fullsized":"2024-09-19T07:45:12.168182+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:49.585196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226198+0000","last_change":"2024-09-19T07:45:11.151896+0000","last_active":"2024-09-19T07:45:12.226198+0000","last_peered":"2024-09-19T07:45:12.226198+0000","last_clean":"2024-09-19T07:45:12.226198+0000","last_became_active":"2024-09-19T07:45:11.151588+0000","last_became_peered":"2024-09-19T07:45:11.151588+0000","last_unstale":"2024-09-19T07:45:12.226198+0000","last_undegraded":"2024-09-19T07:45:12.226198+0000","last_fullsized":"2024-09-19T07:45:12.226198+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:39:20.162645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164887+0000","last_change":"2024-09-19T07:45:10.144986+0000","last_active":"2024-09-19T07:45:12.164887+0000","last_peered":"2024-09-19T07:45:12.164887+0000","last_clean":"2024-09-19T07:45:12.164887+0000","last_became_active":"2024-09-19T07:45:10.144710+0000","last_became_peered":"2024-09-19T07:45:10.144710+0000","last_unstale":"2024-09-19T07:45:12.164887+0000","last_undegraded":"2024-09-19T07:45:12.164887+0000","last_fullsized":"2024-09-19T07:45:12.164887+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:14.953702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164834+0000","last_change":"2024-09-19T07:45:11.158294+0000","last_active":"2024-09-19T07:45:12.164834+0000","last_peered":"2024-09-19T07:45:12.164834+0000","last_clean":"2024-09-19T07:45:12.164834+0000","last_became_active":"2024-09-19T07:45:11.158022+0000","last_became_peered":"2024-09-19T07:45:11.158022+0000","last_unstale":"2024-09-19T07:45:12.164834+0000","last_undegraded":"2024-09-19T07:45:12.164834+0000","last_fullsized":"2024-09-19T07:45:12.164834+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:17.366751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226102+0000","last_change":"2024-09-19T07:45:10.141582+0000","last_active":"2024-09-19T07:45:12.226102+0000","last_peered":"2024-09-19T07:45:12.226102+0000","last_clean":"2024-09-19T07:45:12.226102+0000","last_became_active":"2024-09-19T07:45:10.141072+0000","last_became_peered":"2024-09-19T07:45:10.141072+0000","last_unstale":"2024-09-19T07:45:12.226102+0000","last_undegraded":"2024-09-19T07:45:12.226102+0000","last_fullsized":"2024-09-19T07:45:12.226102+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:27:54.015292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.144675+0000","last_change":"2024-09-19T07:45:11.160091+0000","last_active":"2024-09-19T07:45:12.144675+0000","last_peered":"2024-09-19T07:45:12.144675+0000","last_clean":"2024-09-19T07:45:12.144675+0000","last_became_active":"2024-09-19T07:45:11.159811+0000","last_became_peered":"2024-09-19T07:45:11.159811+0000","last_unstale":"2024-09-19T07:45:12.144675+0000","last_undegraded":"2024-09-19T07:45:12.144675+0000","last_fullsized":"2024-09-19T07:45:12.144675+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:35.156705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.225947+0000","last_change":"2024-09-19T07:45:10.141689+0000","last_active":"2024-09-19T07:45:12.225947+0000","last_peered":"2024-09-19T07:45:12.225947+0000","last_clean":"2024-09-19T07:45:12.225947+0000","last_became_active":"2024-09-19T07:45:10.141392+0000","last_became_peered":"2024-09-19T07:45:10.141392+0000","last_unstale":"2024-09-19T07:45:12.225947+0000","last_undegraded":"2024-09-19T07:45:12.225947+0000","last_fullsized":"2024-09-19T07:45:12.225947+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:16.359752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.164831+0000","last_change":"2024-09-19T07:45:11.169551+0000","last_active":"2024-09-19T07:45:12.164831+0000","last_peered":"2024-09-19T07:45:12.164831+0000","last_clean":"2024-09-19T07:45:12.164831+0000","last_became_active":"2024-09-19T07:45:11.169035+0000","last_became_peered":"2024-09-19T07:45:11.169035+0000","last_unstale":"2024-09-19T07:45:12.164831+0000","last_undegraded":"2024-09-19T07:45:12.164831+0000","last_fullsized":"2024-09-19T07:45:12.164831+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:30:53.075010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.796655+0000","last_change":"2024-09-19T07:45:10.134962+0000","last_active":"2024-09-19T07:45:12.796655+0000","last_peered":"2024-09-19T07:45:12.796655+0000","last_clean":"2024-09-19T07:45:12.796655+0000","last_became_active":"2024-09-19T07:45:10.134663+0000","last_became_peered":"2024-09-19T07:45:10.134663+0000","last_unstale":"2024-09-19T07:45:12.796655+0000","last_undegraded":"2024-09-19T07:45:12.796655+0000","last_fullsized":"2024-09-19T07:45:12.796655+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:14.040132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.226305+0000","last_change":"2024-09-19T07:45:11.153489+0000","last_active":"2024-09-19T07:45:12.226305+0000","last_peered":"2024-09-19T07:45:12.226305+0000","last_clean":"2024-09-19T07:45:12.226305+0000","last_became_active":"2024-09-19T07:45:11.153190+0000","last_became_peered":"2024-09-19T07:45:11.153190+0000","last_unstale":"2024-09-19T07:45:12.226305+0000","last_undegraded":"2024-09-19T07:45:12.226305+0000","last_fullsized":"2024-09-19T07:45:12.226305+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:29.711588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.144797+0000","last_change":"2024-09-19T07:45:10.138425+0000","last_active":"2024-09-19T07:45:12.144797+0000","last_peered":"2024-09-19T07:45:12.144797+0000","last_clean":"2024-09-19T07:45:12.144797+0000","last_became_active":"2024-09-19T07:45:10.138176+0000","last_became_peered":"2024-09-19T07:45:10.138176+0000","last_unstale":"2024-09-19T07:45:12.144797+0000","last_undegraded":"2024-09-19T07:45:12.144797+0000","last_fullsized":"2024-09-19T07:45:12.144797+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:39.856239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.143503+0000","last_change":"2024-09-19T07:45:11.147998+0000","last_active":"2024-09-19T07:45:12.143503+0000","last_peered":"2024-09-19T07:45:12.143503+0000","last_clean":"2024-09-19T07:45:12.143503+0000","last_became_active":"2024-09-19T07:45:11.147682+0000","last_became_peered":"2024-09-19T07:45:11.147682+0000","last_unstale":"2024-09-19T07:45:12.143503+0000","last_undegraded":"2024-09-19T07:45:12.143503+0000","last_fullsized":"2024-09-19T07:45:12.143503+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:10.121237+0000","last_clean_scrub_stamp":"2024-09-19T07:45:10.121237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:35.314160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"53'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.165136+0000","last_change":"2024-09-19T07:45:10.144229+0000","last_active":"2024-09-19T07:45:12.165136+0000","last_peered":"2024-09-19T07:45:12.165136+0000","last_clean":"2024-09-19T07:45:12.165136+0000","last_became_active":"2024-09-19T07:45:10.143971+0000","last_became_peered":"2024-09-19T07:45:10.143971+0000","last_unstale":"2024-09-19T07:45:12.165136+0000","last_undegraded":"2024-09-19T07:45:12.165136+0000","last_fullsized":"2024-09-19T07:45:12.165136+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:45:09.114238+0000","last_clean_scrub_stamp":"2024-09-19T07:45:09.114238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:47.783898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":228,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T07:45:12.167752+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:45:12.167752+0000","last_peered":"2024-09-19T07:45:12.167752+0000","last_clean":"2024-09-19T07:45:12.167752+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:45:12.167752+0000","last_undegraded":"2024-09-19T07:45:12.167752+0000","last_fullsized":"2024-09-19T07:45:12.167752+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1224704,"data_stored":867174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":277334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509471,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":622592,"data_stored":276682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":626688,"data_stored":277264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509470,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1224704,"data_stored":867106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542174,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":638976,"data_stored":278316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":630784,"data_stored":276842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31772,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340068,"statfs":{"total":96636764160,"available":96604229632,"internally_reserved":0,"allocated":598016,"data_stored":262823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:45:15.486 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T07:45:15.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T07:45:15.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.632+0000 7fc7c10af640 1 Processor -- start 2024-09-19T07:45:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.632+0000 7fc7c10af640 1 -- start start 2024-09-19T07:45:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.633+0000 7fc7c10af640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7bc15e0a0 0x7fc7bc162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.633+0000 7fc7c10af640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc7bc0d43e0 0x7fc7bc0d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.633+0000 7fc7c10af640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7bc15f250 con 0x7fc7bc0b4b70 2024-09-19T07:45:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.633+0000 7fc7c10af640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc7bc0d4d60 con 0x7fc7bc0d43e0 2024-09-19T07:45:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.633+0000 7fc7c10af640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc7bc15f530 con 0x7fc7bc15e0a0 2024-09-19T07:45:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.633+0000 7fc7bad9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7bc15e0a0 0x7fc7bc162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.633+0000 7fc7ba59c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc7bc0d43e0 0x7fc7bc0d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.633+0000 7fc7ba59c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc7bc0d43e0 0x7fc7bc0d4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:33792/0 (socket says 172.21.15.16:33792) 2024-09-19T07:45:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.633+0000 7fc7ba59c640 1 -- 172.21.15.16:0/2114033119 learned_addr learned my addr 172.21.15.16:0/2114033119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.633+0000 7fc7bad9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7bc15e0a0 0x7fc7bc162a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:46728/0 (socket says 172.21.15.16:46728) 2024-09-19T07:45:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.634+0000 7fc7ba59c640 1 -- 172.21.15.16:0/2114033119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7bc15e0a0 msgr2=0x7fc7bc162a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.634+0000 7fc7ba59c640 1 --2- 172.21.15.16:0/2114033119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7bc15e0a0 0x7fc7bc162a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.634+0000 7fc7ba59c640 1 -- 172.21.15.16:0/2114033119 >> v1:172.21.15.146:6789/0 conn(0x7fc7bc0b4b70 legacy=0x7fc7bc15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.634+0000 7fc7ba59c640 1 -- 172.21.15.16:0/2114033119 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7bc15f810 con 0x7fc7bc0d43e0 2024-09-19T07:45:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.634+0000 7fc7bad9d640 1 --2- 172.21.15.16:0/2114033119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7bc15e0a0 0x7fc7bc162a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:15.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.634+0000 7fc7ba59c640 1 --2- 172.21.15.16:0/2114033119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7bc0d43e0 0x7fc7bc0d4820 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009850 tx=0x7fc7b002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edc63cf5ac83057f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:15.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.635+0000 7fc7b9d9b640 1 -- 172.21.15.16:0/2114033119 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0004030 con 0x7fc7bc0d43e0 2024-09-19T07:45:15.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.635+0000 7fc7b9d9b640 1 -- 172.21.15.16:0/2114033119 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b00041d0 con 0x7fc7bc0d43e0 2024-09-19T07:45:15.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.635+0000 7fc7b9d9b640 1 -- 172.21.15.16:0/2114033119 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0003610 con 0x7fc7bc0d43e0 2024-09-19T07:45:15.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.635+0000 7fc7c10af640 1 -- 172.21.15.16:0/2114033119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7bc0d43e0 msgr2=0x7fc7bc0d4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:15.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.635+0000 7fc7c10af640 1 --2- 172.21.15.16:0/2114033119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7bc0d43e0 0x7fc7bc0d4820 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009850 tx=0x7fc7b002a250 comp rx=0 tx=0).stop 2024-09-19T07:45:15.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.635+0000 7fc7c10af640 1 -- 172.21.15.16:0/2114033119 shutdown_connections 2024-09-19T07:45:15.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.635+0000 7fc7c10af640 1 --2- 172.21.15.16:0/2114033119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7bc0d43e0 0x7fc7bc0d4820 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.635+0000 7fc7c10af640 1 --2- 172.21.15.16:0/2114033119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7bc15e0a0 0x7fc7bc162a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.635+0000 7fc7c10af640 1 -- 172.21.15.16:0/2114033119 >> 172.21.15.16:0/2114033119 conn(0x7fc7bc09d770 msgr2=0x7fc7bc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:15.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.635+0000 7fc7c10af640 1 -- 172.21.15.16:0/2114033119 shutdown_connections 2024-09-19T07:45:15.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.635+0000 7fc7c10af640 1 -- 172.21.15.16:0/2114033119 wait complete. 2024-09-19T07:45:15.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.637+0000 7fc7c10af640 1 Processor -- start 2024-09-19T07:45:15.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.637+0000 7fc7c10af640 1 -- start start 2024-09-19T07:45:15.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.637+0000 7fc7c10af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7bc0b4b70 0x7fc7bc15d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:15.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.637+0000 7fc7c10af640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7bc0d43e0 0x7fc7bc0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:15.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.637+0000 7fc7c10af640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7bc15e0a0 0x7fc7bc0cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:15.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.637+0000 7fc7c10af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc15ed10 con 0x7fc7bc0b4b70 2024-09-19T07:45:15.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.637+0000 7fc7c10af640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc15eb90 con 0x7fc7bc0d43e0 2024-09-19T07:45:15.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.637+0000 7fc7c10af640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc7bc15ee90 con 0x7fc7bc15e0a0 2024-09-19T07:45:15.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.637+0000 7fc7bad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7bc0b4b70 0x7fc7bc15d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:15.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.638+0000 7fc7bad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7bc0b4b70 0x7fc7bc15d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46978/0 (socket says 172.21.15.16:46978) 2024-09-19T07:45:15.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.638+0000 7fc7bad9d640 1 -- 172.21.15.16:0/1457896999 learned_addr learned my addr 172.21.15.16:0/1457896999 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:15.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.638+0000 7fc7bb59e640 1 --2- 172.21.15.16:0/1457896999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7bc15e0a0 0x7fc7bc0cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:15.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.638+0000 7fc7ba59c640 1 --2- 172.21.15.16:0/1457896999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7bc0d43e0 0x7fc7bc0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:15.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.638+0000 7fc7bad9d640 1 -- 172.21.15.16:0/1457896999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7bc15e0a0 msgr2=0x7fc7bc0cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:15.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.638+0000 7fc7bad9d640 1 --2- 172.21.15.16:0/1457896999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7bc15e0a0 0x7fc7bc0cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.638+0000 7fc7bad9d640 1 -- 172.21.15.16:0/1457896999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7bc0d43e0 msgr2=0x7fc7bc0cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:15.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.638+0000 7fc7bad9d640 1 --2- 172.21.15.16:0/1457896999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7bc0d43e0 0x7fc7bc0cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.638+0000 7fc7bad9d640 1 -- 172.21.15.16:0/1457896999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7bc0cb770 con 0x7fc7bc0b4b70 2024-09-19T07:45:15.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.638+0000 7fc7bad9d640 1 --2- 172.21.15.16:0/1457896999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7bc0b4b70 0x7fc7bc15d9f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00e960 tx=0x7fc7ac00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.638+0000 7fc7a37fe640 1 -- 172.21.15.16:0/1457896999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac017070 con 0x7fc7bc0b4b70 2024-09-19T07:45:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.638+0000 7fc7a37fe640 1 -- 172.21.15.16:0/1457896999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7ac00ca00 con 0x7fc7bc0b4b70 2024-09-19T07:45:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.638+0000 7fc7a37fe640 1 -- 172.21.15.16:0/1457896999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac00cdd0 con 0x7fc7bc0b4b70 2024-09-19T07:45:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.638+0000 7fc7c10af640 1 -- 172.21.15.16:0/1457896999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7bc0cba50 con 0x7fc7bc0b4b70 2024-09-19T07:45:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.638+0000 7fc7c10af640 1 -- 172.21.15.16:0/1457896999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7bc0cc530 con 0x7fc7bc0b4b70 2024-09-19T07:45:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.639+0000 7fc7c10af640 1 -- 172.21.15.16:0/1457896999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7bc0cb500 con 0x7fc7bc0b4b70 2024-09-19T07:45:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.640+0000 7fc7a37fe640 1 -- 172.21.15.16:0/1457896999 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc7ac01a070 con 0x7fc7bc0b4b70 2024-09-19T07:45:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.640+0000 7fc7a37fe640 1 --2- 172.21.15.16:0/1457896999 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc7a806c2b0 0x7fc7a806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.641+0000 7fc7a37fe640 1 -- 172.21.15.16:0/1457896999 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fc7ac08b6f0 con 0x7fc7bc0b4b70 2024-09-19T07:45:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.641+0000 7fc7ba59c640 1 --2- 172.21.15.16:0/1457896999 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc7a806c2b0 0x7fc7a806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.641+0000 7fc7ba59c640 1 --2- 172.21.15.16:0/1457896999 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc7a806c2b0 0x7fc7a806e770 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0033000 tx=0x7fc7b002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.645+0000 7fc7a37fe640 1 -- 172.21.15.16:0/1457896999 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc7ac059710 con 0x7fc7bc0b4b70 2024-09-19T07:45:15.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.889+0000 7fc7c10af640 1 -- 172.21.15.16:0/1457896999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc7bc0d4820 con 0x7fc7bc0b4b70 2024-09-19T07:45:15.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.891+0000 7fc7a37fe640 1 -- 172.21.15.16:0/1457896999 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7fc7ac05da10 con 0x7fc7bc0b4b70 2024-09-19T07:45:15.891 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-19T07:45:15.892 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA== 2024-09-19T07:45:15.892 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-19T07:45:15.892 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-19T07:45:15.892 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T07:45:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-19T07:45:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.893+0000 7fc7c10af640 1 -- 172.21.15.16:0/1457896999 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc7a806c2b0 msgr2=0x7fc7a806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.893+0000 7fc7c10af640 1 --2- 172.21.15.16:0/1457896999 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc7a806c2b0 0x7fc7a806e770 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0033000 tx=0x7fc7b002e000 comp rx=0 tx=0).stop 2024-09-19T07:45:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.893+0000 7fc7c10af640 1 -- 172.21.15.16:0/1457896999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7bc0b4b70 msgr2=0x7fc7bc15d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.893+0000 7fc7c10af640 1 --2- 172.21.15.16:0/1457896999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7bc0b4b70 0x7fc7bc15d9f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00e960 tx=0x7fc7ac00ee20 comp rx=0 tx=0).stop 2024-09-19T07:45:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.893+0000 7fc7c10af640 1 -- 172.21.15.16:0/1457896999 shutdown_connections 2024-09-19T07:45:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.893+0000 7fc7c10af640 1 --2- 172.21.15.16:0/1457896999 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc7a806c2b0 0x7fc7a806e770 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.893+0000 7fc7c10af640 1 --2- 172.21.15.16:0/1457896999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7bc15e0a0 0x7fc7bc0cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.893+0000 7fc7c10af640 1 --2- 172.21.15.16:0/1457896999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7bc0d43e0 0x7fc7bc0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.893+0000 7fc7c10af640 1 --2- 172.21.15.16:0/1457896999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7bc0b4b70 0x7fc7bc15d9f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.893+0000 7fc7c10af640 1 -- 172.21.15.16:0/1457896999 >> 172.21.15.16:0/1457896999 conn(0x7fc7bc09d770 msgr2=0x7fc7bc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.893+0000 7fc7c10af640 1 -- 172.21.15.16:0/1457896999 shutdown_connections 2024-09-19T07:45:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:15.893+0000 7fc7c10af640 1 -- 172.21.15.16:0/1457896999 wait complete. 2024-09-19T07:45:15.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:45:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.006+0000 7fd37721e640 1 Processor -- start 2024-09-19T07:45:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.006+0000 7fd37721e640 1 -- start start 2024-09-19T07:45:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.007+0000 7fd37721e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd37015f0d0 0x7fd37015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.007+0000 7fd37721e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3700d0070 0x7fd3700d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.007+0000 7fd37721e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3700cf2b0 con 0x7fd3700d1940 2024-09-19T07:45:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.007+0000 7fd37721e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd3700cf590 con 0x7fd3700d0070 2024-09-19T07:45:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.007+0000 7fd37721e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd3700cf870 con 0x7fd37015f0d0 2024-09-19T07:45:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.007+0000 7fd36ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3700d0070 0x7fd3700d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.007+0000 7fd3757bb640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd3700d1940 0x7fd3700d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50364/0 (socket says 172.21.15.16:50364) 2024-09-19T07:45:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.007+0000 7fd3757bb640 1 -- 172.21.15.16:0/1560157679 learned_addr learned my addr 172.21.15.16:0/1560157679 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.008+0000 7fd374fba640 1 --2- 172.21.15.16:0/1560157679 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd37015f0d0 0x7fd37015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.008+0000 7fd36ffff640 1 -- 172.21.15.16:0/1560157679 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd37015f0d0 msgr2=0x7fd37015f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:16.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.008+0000 7fd36ffff640 1 --2- 172.21.15.16:0/1560157679 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd37015f0d0 0x7fd37015f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.008+0000 7fd36ffff640 1 -- 172.21.15.16:0/1560157679 >> v1:172.21.15.16:6789/0 conn(0x7fd3700d1940 legacy=0x7fd3700d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.008+0000 7fd36ffff640 1 -- 172.21.15.16:0/1560157679 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3700cfb50 con 0x7fd3700d0070 2024-09-19T07:45:16.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.008+0000 7fd36f7fe640 1 -- 172.21.15.16:0/1560157679 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 983845650 0 0) 0x7fd3700cf2b0 con 0x7fd3700d1940 2024-09-19T07:45:16.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.008+0000 7fd36ffff640 1 --2- 172.21.15.16:0/1560157679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3700d0070 0x7fd3700d1400 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd3640097a0 tx=0x7fd36402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef308725793308d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:16.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.009+0000 7fd36f7fe640 1 -- 172.21.15.16:0/1560157679 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364037070 con 0x7fd3700d0070 2024-09-19T07:45:16.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.009+0000 7fd36f7fe640 1 -- 172.21.15.16:0/1560157679 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd36402ae50 con 0x7fd3700d0070 2024-09-19T07:45:16.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.009+0000 7fd36f7fe640 1 -- 172.21.15.16:0/1560157679 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364032660 con 0x7fd3700d0070 2024-09-19T07:45:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.009+0000 7fd37721e640 1 -- 172.21.15.16:0/1560157679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3700d0070 msgr2=0x7fd3700d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.009+0000 7fd37721e640 1 --2- 172.21.15.16:0/1560157679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3700d0070 0x7fd3700d1400 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd3640097a0 tx=0x7fd36402a100 comp rx=0 tx=0).stop 2024-09-19T07:45:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.009+0000 7fd37721e640 1 -- 172.21.15.16:0/1560157679 shutdown_connections 2024-09-19T07:45:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.009+0000 7fd37721e640 1 --2- 172.21.15.16:0/1560157679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3700d0070 0x7fd3700d1400 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.009+0000 7fd37721e640 1 --2- 172.21.15.16:0/1560157679 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd37015f0d0 0x7fd37015f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.009+0000 7fd37721e640 1 -- 172.21.15.16:0/1560157679 >> 172.21.15.16:0/1560157679 conn(0x7fd37009d760 msgr2=0x7fd37008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.009+0000 7fd37721e640 1 -- 172.21.15.16:0/1560157679 shutdown_connections 2024-09-19T07:45:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.009+0000 7fd37721e640 1 -- 172.21.15.16:0/1560157679 wait complete. 2024-09-19T07:45:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.010+0000 7fd37721e640 1 Processor -- start 2024-09-19T07:45:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.010+0000 7fd37721e640 1 -- start start 2024-09-19T07:45:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.011+0000 7fd37721e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3700d0070 0x7fd37015ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.011+0000 7fd37721e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3700d1940 0x7fd3700ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.011+0000 7fd37721e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd37015f0d0 0x7fd370160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.011+0000 7fd37721e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3700d49c0 con 0x7fd37015f0d0 2024-09-19T07:45:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.011+0000 7fd37721e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd3700d4840 con 0x7fd3700d0070 2024-09-19T07:45:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.011+0000 7fd37721e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd3700d4b40 con 0x7fd3700d1940 2024-09-19T07:45:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.012+0000 7fd3757bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd37015f0d0 0x7fd370160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.012+0000 7fd3757bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd37015f0d0 0x7fd370160b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47004/0 (socket says 172.21.15.16:47004) 2024-09-19T07:45:16.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.012+0000 7fd3757bb640 1 -- 172.21.15.16:0/4065145824 learned_addr learned my addr 172.21.15.16:0/4065145824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:16.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.012+0000 7fd374fba640 1 --2- 172.21.15.16:0/4065145824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3700d0070 0x7fd37015ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.012+0000 7fd36ffff640 1 --2- 172.21.15.16:0/4065145824 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3700d1940 0x7fd3700ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.012+0000 7fd3757bb640 1 -- 172.21.15.16:0/4065145824 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3700d1940 msgr2=0x7fd3700ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.012+0000 7fd3757bb640 1 --2- 172.21.15.16:0/4065145824 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3700d1940 0x7fd3700ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.012+0000 7fd3757bb640 1 -- 172.21.15.16:0/4065145824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3700d0070 msgr2=0x7fd37015ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.012+0000 7fd3757bb640 1 --2- 172.21.15.16:0/4065145824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3700d0070 0x7fd37015ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.012+0000 7fd3757bb640 1 -- 172.21.15.16:0/4065145824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3700cd700 con 0x7fd37015f0d0 2024-09-19T07:45:16.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.012+0000 7fd3757bb640 1 --2- 172.21.15.16:0/4065145824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd37015f0d0 0x7fd370160b20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd36800c7b0 tx=0x7fd36800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:16.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.012+0000 7fd36d7fa640 1 -- 172.21.15.16:0/4065145824 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368018070 con 0x7fd37015f0d0 2024-09-19T07:45:16.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.012+0000 7fd36d7fa640 1 -- 172.21.15.16:0/4065145824 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd36800a640 con 0x7fd37015f0d0 2024-09-19T07:45:16.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.012+0000 7fd37721e640 1 -- 172.21.15.16:0/4065145824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3700cd9e0 con 0x7fd37015f0d0 2024-09-19T07:45:16.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.012+0000 7fd36d7fa640 1 -- 172.21.15.16:0/4065145824 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36800aa70 con 0x7fd37015f0d0 2024-09-19T07:45:16.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.012+0000 7fd37721e640 1 -- 172.21.15.16:0/4065145824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3700cdfa0 con 0x7fd37015f0d0 2024-09-19T07:45:16.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.013+0000 7fd37721e640 1 -- 172.21.15.16:0/4065145824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3700d1400 con 0x7fd37015f0d0 2024-09-19T07:45:16.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.014+0000 7fd36d7fa640 1 -- 172.21.15.16:0/4065145824 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd36801b070 con 0x7fd37015f0d0 2024-09-19T07:45:16.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.014+0000 7fd36d7fa640 1 --2- 172.21.15.16:0/4065145824 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd35c06c2b0 0x7fd35c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.015+0000 7fd36d7fa640 1 -- 172.21.15.16:0/4065145824 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fd36808ab60 con 0x7fd37015f0d0 2024-09-19T07:45:16.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.015+0000 7fd374fba640 1 --2- 172.21.15.16:0/4065145824 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd35c06c2b0 0x7fd35c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.015+0000 7fd374fba640 1 --2- 172.21.15.16:0/4065145824 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd35c06c2b0 0x7fd35c06e770 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd3600079a0 tx=0x7fd360008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:16.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.019+0000 7fd36d7fa640 1 -- 172.21.15.16:0/4065145824 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd368058af0 con 0x7fd37015f0d0 2024-09-19T07:45:16.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.231+0000 7fd37721e640 1 -- 172.21.15.16:0/4065145824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3700756d0 con 0x7fd37015f0d0 2024-09-19T07:45:16.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.232+0000 7fd36d7fa640 1 -- 172.21.15.16:0/4065145824 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5223 (secure 0 0 0) 0x7fd36805cdf0 con 0x7fd37015f0d0 2024-09-19T07:45:16.232 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:45:16.232 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"btime":"2024-09-19T07:45:14:150420+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/3536366465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3536366465},{"type":"v1","addr":"172.21.15.146:6836","nonce":3536366465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3612374586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3612374586},{"type":"v1","addr":"172.21.15.146:6837","nonce":3612374586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5337,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/781070730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":781070730},{"type":"v1","addr":"172.21.15.16:6833","nonce":781070730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:45:10.494498+0000","modified":"2024-09-19T07:45:13.468183+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5340":{"gid":5340,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/642332950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":642332950},{"type":"v1","addr":"172.21.15.16:6835","nonce":642332950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5340,"qdb_cluster":[5340]},"id":3}]} 2024-09-19T07:45:16.233 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-09-19T07:45:16.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.233+0000 7fd37721e640 1 -- 172.21.15.16:0/4065145824 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd35c06c2b0 msgr2=0x7fd35c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.233+0000 7fd37721e640 1 --2- 172.21.15.16:0/4065145824 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd35c06c2b0 0x7fd35c06e770 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd3600079a0 tx=0x7fd360008040 comp rx=0 tx=0).stop 2024-09-19T07:45:16.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.233+0000 7fd37721e640 1 -- 172.21.15.16:0/4065145824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd37015f0d0 msgr2=0x7fd370160b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.233+0000 7fd37721e640 1 --2- 172.21.15.16:0/4065145824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd37015f0d0 0x7fd370160b20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd36800c7b0 tx=0x7fd36800cc70 comp rx=0 tx=0).stop 2024-09-19T07:45:16.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.234+0000 7fd37721e640 1 -- 172.21.15.16:0/4065145824 shutdown_connections 2024-09-19T07:45:16.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.234+0000 7fd37721e640 1 --2- 172.21.15.16:0/4065145824 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd35c06c2b0 0x7fd35c06e770 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.234+0000 7fd37721e640 1 --2- 172.21.15.16:0/4065145824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd37015f0d0 0x7fd370160b20 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.234+0000 7fd37721e640 1 --2- 172.21.15.16:0/4065145824 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3700d1940 0x7fd3700ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.234+0000 7fd37721e640 1 --2- 172.21.15.16:0/4065145824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3700d0070 0x7fd37015ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.234+0000 7fd37721e640 1 -- 172.21.15.16:0/4065145824 >> 172.21.15.16:0/4065145824 conn(0x7fd37009d760 msgr2=0x7fd3700d1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:16.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.234+0000 7fd37721e640 1 -- 172.21.15.16:0/4065145824 shutdown_connections 2024-09-19T07:45:16.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.234+0000 7fd37721e640 1 -- 172.21.15.16:0/4065145824 wait complete. 2024-09-19T07:45:16.647 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T07:45:10.494498+0000', 'modified': '2024-09-19T07:45:13.468183+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5340}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5340': {'gid': 5340, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6835/642332950', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 642332950}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 642332950}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5340, 'qdb_cluster': [5340]} 2024-09-19T07:45:16.647 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T07:45:16.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-19T07:45:16.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.754+0000 7f7d922f1640 1 Processor -- start 2024-09-19T07:45:16.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.754+0000 7f7d922f1640 1 -- start start 2024-09-19T07:45:16.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.755+0000 7f7d922f1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d8c06cd20 0x7f7d8c06d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.755+0000 7f7d922f1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d8c06d640 con 0x7f7d8c161eb0 2024-09-19T07:45:16.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.755+0000 7f7d922f1640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d8c162b70 con 0x7f7d8c1425e0 2024-09-19T07:45:16.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.755+0000 7f7d922f1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f7d8c162e50 con 0x7f7d8c06cd20 2024-09-19T07:45:16.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.755+0000 7f7d8bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d8c06cd20 0x7f7d8c06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.755+0000 7f7d8b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7d8c161eb0 0x7f7d8c1622f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50368/0 (socket says 172.21.15.16:50368) 2024-09-19T07:45:16.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.755+0000 7f7d8b7fe640 1 -- 172.21.15.16:0/3570071669 learned_addr learned my addr 172.21.15.16:0/3570071669 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:16.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.755+0000 7f7d8affd640 1 -- 172.21.15.16:0/3570071669 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3834728836 0 0) 0x7f7d8c06d640 con 0x7f7d8c161eb0 2024-09-19T07:45:16.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.756+0000 7f7d8affd640 1 -- 172.21.15.16:0/3570071669 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d74003520 con 0x7f7d8c161eb0 2024-09-19T07:45:16.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.756+0000 7f7d8bfff640 1 -- 172.21.15.16:0/3570071669 >> v1:172.21.15.146:6789/0 conn(0x7f7d8c1425e0 legacy=0x7f7d8c142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:16.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.756+0000 7f7d8bfff640 1 -- 172.21.15.16:0/3570071669 >> v1:172.21.15.16:6789/0 conn(0x7f7d8c161eb0 legacy=0x7f7d8c1622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.756+0000 7f7d8bfff640 1 -- 172.21.15.16:0/3570071669 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c141b00 con 0x7f7d8c06cd20 2024-09-19T07:45:16.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.756+0000 7f7d8bfff640 1 --2- 172.21.15.16:0/3570071669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d8c06cd20 0x7f7d8c06d100 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c0097a0 tx=0x7f7d7c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da8b0aae52cffb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:16.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.756+0000 7f7d8affd640 1 -- 172.21.15.16:0/3570071669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c037070 con 0x7f7d8c06cd20 2024-09-19T07:45:16.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.757+0000 7f7d8affd640 1 -- 172.21.15.16:0/3570071669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d7c032440 con 0x7f7d8c06cd20 2024-09-19T07:45:16.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.757+0000 7f7d8affd640 1 -- 172.21.15.16:0/3570071669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c0327f0 con 0x7f7d8c06cd20 2024-09-19T07:45:16.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.757+0000 7f7d922f1640 1 -- 172.21.15.16:0/3570071669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d8c06cd20 msgr2=0x7f7d8c06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.757+0000 7f7d922f1640 1 --2- 172.21.15.16:0/3570071669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d8c06cd20 0x7f7d8c06d100 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c0097a0 tx=0x7f7d7c02a1a0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.757+0000 7f7d922f1640 1 -- 172.21.15.16:0/3570071669 shutdown_connections 2024-09-19T07:45:16.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.757+0000 7f7d922f1640 1 --2- 172.21.15.16:0/3570071669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d8c06cd20 0x7f7d8c06d100 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.757+0000 7f7d922f1640 1 -- 172.21.15.16:0/3570071669 >> 172.21.15.16:0/3570071669 conn(0x7f7d8c0b3ed0 msgr2=0x7f7d8c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:16.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.757+0000 7f7d922f1640 1 -- 172.21.15.16:0/3570071669 shutdown_connections 2024-09-19T07:45:16.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.757+0000 7f7d922f1640 1 -- 172.21.15.16:0/3570071669 wait complete. 2024-09-19T07:45:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.758+0000 7f7d922f1640 1 Processor -- start 2024-09-19T07:45:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.759+0000 7f7d922f1640 1 -- start start 2024-09-19T07:45:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.759+0000 7f7d922f1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d8c06cd20 0x7f7d8c1559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.759+0000 7f7d922f1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d8c1425e0 0x7f7d8c15b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.759+0000 7f7d922f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d8c161eb0 0x7f7d8c0d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.759+0000 7f7d922f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c141600 con 0x7f7d8c161eb0 2024-09-19T07:45:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.759+0000 7f7d922f1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c142150 con 0x7f7d8c06cd20 2024-09-19T07:45:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.759+0000 7f7d922f1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8c143180 con 0x7f7d8c1425e0 2024-09-19T07:45:16.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.759+0000 7f7d9088e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d8c161eb0 0x7f7d8c0d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.759+0000 7f7d9088e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d8c161eb0 0x7f7d8c0d0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47018/0 (socket says 172.21.15.16:47018) 2024-09-19T07:45:16.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.759+0000 7f7d9088e640 1 -- 172.21.15.16:0/417565984 learned_addr learned my addr 172.21.15.16:0/417565984 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:16.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.760+0000 7f7d8b7fe640 1 --2- 172.21.15.16:0/417565984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d8c1425e0 0x7f7d8c15b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.760+0000 7f7d8bfff640 1 --2- 172.21.15.16:0/417565984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d8c06cd20 0x7f7d8c1559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.760+0000 7f7d9088e640 1 -- 172.21.15.16:0/417565984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d8c1425e0 msgr2=0x7f7d8c15b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.760+0000 7f7d9088e640 1 --2- 172.21.15.16:0/417565984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d8c1425e0 0x7f7d8c15b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.760+0000 7f7d9088e640 1 -- 172.21.15.16:0/417565984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d8c06cd20 msgr2=0x7f7d8c1559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.760+0000 7f7d9088e640 1 --2- 172.21.15.16:0/417565984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d8c06cd20 0x7f7d8c1559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.760+0000 7f7d9088e640 1 -- 172.21.15.16:0/417565984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c1590a0 con 0x7f7d8c161eb0 2024-09-19T07:45:16.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.760+0000 7f7d8bfff640 1 --2- 172.21.15.16:0/417565984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d8c06cd20 0x7f7d8c1559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:16.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.760+0000 7f7d9088e640 1 --2- 172.21.15.16:0/417565984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d8c161eb0 0x7f7d8c0d0da0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400c7a0 tx=0x7f7d8400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:16.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.760+0000 7f7d88ff9640 1 -- 172.21.15.16:0/417565984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84003b70 con 0x7f7d8c161eb0 2024-09-19T07:45:16.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.760+0000 7f7d88ff9640 1 -- 172.21.15.16:0/417565984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d84003d30 con 0x7f7d8c161eb0 2024-09-19T07:45:16.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.760+0000 7f7d922f1640 1 -- 172.21.15.16:0/417565984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8c15bb90 con 0x7f7d8c161eb0 2024-09-19T07:45:16.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.760+0000 7f7d88ff9640 1 -- 172.21.15.16:0/417565984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8400aa90 con 0x7f7d8c161eb0 2024-09-19T07:45:16.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.760+0000 7f7d922f1640 1 -- 172.21.15.16:0/417565984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d8c0cdc00 con 0x7f7d8c161eb0 2024-09-19T07:45:16.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.762+0000 7f7d88ff9640 1 -- 172.21.15.16:0/417565984 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7d8400ac30 con 0x7f7d8c161eb0 2024-09-19T07:45:16.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.762+0000 7f7d88ff9640 1 --2- 172.21.15.16:0/417565984 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7d7806c2b0 0x7f7d7806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.762+0000 7f7d88ff9640 1 -- 172.21.15.16:0/417565984 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f7d8408b570 con 0x7f7d8c161eb0 2024-09-19T07:45:16.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.762+0000 7f7d8bfff640 1 --2- 172.21.15.16:0/417565984 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7d7806c2b0 0x7f7d7806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.763+0000 7f7d8bfff640 1 --2- 172.21.15.16:0/417565984 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7d7806c2b0 0x7f7d7806e770 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c034000 tx=0x7f7d7c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:16.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.763+0000 7f7d9508e740 1 Processor -- start 2024-09-19T07:45:16.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.763+0000 7f7d9508e740 1 -- start start 2024-09-19T07:45:16.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.764+0000 7f7d9508e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b062c0 0x55d089b4ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.764+0000 7f7d9508e740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b03a80 0x55d089b066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.764+0000 7f7d9508e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b07a50 0x55d089b008b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.764+0000 7f7d9508e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d089a68520 con 0x55d089b07a50 2024-09-19T07:45:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.764+0000 7f7d9508e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d089a103e0 con 0x55d089b062c0 2024-09-19T07:45:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.764+0000 7f7d9508e740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55d089a644e0 con 0x55d089b03a80 2024-09-19T07:45:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.764+0000 7f7d9088e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b07a50 0x55d089b008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.764+0000 7f7d9088e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b07a50 0x55d089b008b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47032/0 (socket says 172.21.15.16:47032) 2024-09-19T07:45:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.764+0000 7f7d9088e640 1 -- 172.21.15.16:0/3840531984 learned_addr learned my addr 172.21.15.16:0/3840531984 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.764+0000 7f7d8bfff640 1 --2- 172.21.15.16:0/3840531984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b062c0 0x55d089b4ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.764+0000 7f7d8b7fe640 1 --2- 172.21.15.16:0/3840531984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b03a80 0x55d089b066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.764+0000 7f7d9088e640 1 -- 172.21.15.16:0/3840531984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b03a80 msgr2=0x55d089b066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.764+0000 7f7d9088e640 1 --2- 172.21.15.16:0/3840531984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b03a80 0x55d089b066a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.764+0000 7f7d9088e640 1 -- 172.21.15.16:0/3840531984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b062c0 msgr2=0x55d089b4ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.764+0000 7f7d9088e640 1 --2- 172.21.15.16:0/3840531984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b062c0 0x55d089b4ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.764+0000 7f7d9088e640 1 -- 172.21.15.16:0/3840531984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d089a2b2f0 con 0x55d089b07a50 2024-09-19T07:45:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.764+0000 7f7d8bfff640 1 --2- 172.21.15.16:0/3840531984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b062c0 0x55d089b4ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.764+0000 7f7d9088e640 1 --2- 172.21.15.16:0/3840531984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b07a50 0x55d089b008b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7d84089880 tx=0x7f7d84058db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836abdfc1c153130 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.764+0000 7f7d71ffb640 1 -- 172.21.15.16:0/3840531984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8405d800 con 0x55d089b07a50 2024-09-19T07:45:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.765+0000 7f7d71ffb640 1 -- 172.21.15.16:0/3840531984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d84062030 con 0x55d089b07a50 2024-09-19T07:45:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.765+0000 7f7d71ffb640 1 -- 172.21.15.16:0/3840531984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84010dc0 con 0x55d089b07a50 2024-09-19T07:45:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.765+0000 7f7d9508e740 1 -- 172.21.15.16:0/3840531984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b07a50 msgr2=0x55d089b008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.765+0000 7f7d9508e740 1 --2- 172.21.15.16:0/3840531984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b07a50 0x55d089b008b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7d84089880 tx=0x7f7d84058db0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.765+0000 7f7d9508e740 1 -- 172.21.15.16:0/3840531984 shutdown_connections 2024-09-19T07:45:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.765+0000 7f7d9508e740 1 --2- 172.21.15.16:0/3840531984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b07a50 0x55d089b008b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.765+0000 7f7d9508e740 1 --2- 172.21.15.16:0/3840531984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b03a80 0x55d089b066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.765+0000 7f7d9508e740 1 --2- 172.21.15.16:0/3840531984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b062c0 0x55d089b4ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.765+0000 7f7d9508e740 1 -- 172.21.15.16:0/3840531984 >> 172.21.15.16:0/3840531984 conn(0x55d089b02a80 msgr2=0x55d089b4af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.765+0000 7f7d9508e740 1 -- 172.21.15.16:0/3840531984 shutdown_connections 2024-09-19T07:45:16.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.765+0000 7f7d9508e740 1 -- 172.21.15.16:0/3840531984 wait complete. 2024-09-19T07:45:16.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.766+0000 7f7d9508e740 1 -- start start 2024-09-19T07:45:16.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.766+0000 7f7d9508e740 1 Processor -- start 2024-09-19T07:45:16.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d9508e740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b062c0 0x55d089b48640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d9508e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b07a50 0x55d089b26200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d9508e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b2c310 0x55d089b2c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d9508e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d089b01f20 con 0x55d089b2c310 2024-09-19T07:45:16.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d9508e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d0899f1450 con 0x55d089b07a50 2024-09-19T07:45:16.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d9508e740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55d089b4c290 con 0x55d089b062c0 2024-09-19T07:45:16.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d9508e740 10 client.0 _ll_register_callbacks cb 0x55d08977aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:16.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d9088e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b2c310 0x55d089b2c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d9088e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b2c310 0x55d089b2c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47044/0 (socket says 172.21.15.16:47044) 2024-09-19T07:45:16.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d9088e640 1 -- 172.21.15.16:0/4040303532 learned_addr learned my addr 172.21.15.16:0/4040303532 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:16.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d8b7fe640 1 --2- 172.21.15.16:0/4040303532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b07a50 0x55d089b26200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d8bfff640 1 --2- 172.21.15.16:0/4040303532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b062c0 0x55d089b48640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d9088e640 1 -- 172.21.15.16:0/4040303532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b062c0 msgr2=0x55d089b48640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:16.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d9088e640 1 --2- 172.21.15.16:0/4040303532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b062c0 0x55d089b48640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d9088e640 1 -- 172.21.15.16:0/4040303532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b07a50 msgr2=0x55d089b26200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:16.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d9088e640 1 --2- 172.21.15.16:0/4040303532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b07a50 0x55d089b26200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d9088e640 1 -- 172.21.15.16:0/4040303532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d08977b2d0 con 0x55d089b2c310 2024-09-19T07:45:16.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d8bfff640 1 --2- 172.21.15.16:0/4040303532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b062c0 0x55d089b48640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:16.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d9088e640 1 --2- 172.21.15.16:0/4040303532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b2c310 0x55d089b2c790 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7d84061b00 tx=0x7f7d84001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f365cb5172e571 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:16.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d6affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:45:16.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d6affd640 1 -- 172.21.15.16:0/4040303532 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84078060 con 0x55d089b2c310 2024-09-19T07:45:16.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d6affd640 1 -- 172.21.15.16:0/4040303532 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f7d84071180 con 0x55d089b2c310 2024-09-19T07:45:16.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.767+0000 7f7d9508e740 1 -- 172.21.15.16:0/4040303532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d84062030 con 0x55d089b2c310 2024-09-19T07:45:16.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.768+0000 7f7d6affd640 1 -- 172.21.15.16:0/4040303532 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7d84057810 con 0x55d089b2c310 2024-09-19T07:45:16.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.768+0000 7f7d9508e740 10 client.5463 fetch_fsmap learned FSMap version 24 2024-09-19T07:45:16.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.768+0000 7f7d9508e740 1 -- 172.21.15.16:0/4040303532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55d0898456b0 con 0x55d089b2c310 2024-09-19T07:45:16.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.768+0000 7f7d6affd640 1 -- 172.21.15.16:0/4040303532 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d84062030 con 0x55d089b2c310 2024-09-19T07:45:16.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.768+0000 7f7d6affd640 1 -- 172.21.15.16:0/4040303532 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 3502+0+0 (secure 0 0 0) 0x7f7d84081020 con 0x55d089b2c310 2024-09-19T07:45:16.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.768+0000 7f7d9508e740 10 client.5463 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T07:45:16.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.768+0000 7f7d9508e740 10 client.5463 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T07:45:16.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.768+0000 7f7d9508e740 20 client.5463 populate_metadata read hostname 'smithi016' 2024-09-19T07:45:16.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.768+0000 7f7d9508e740 1 --2- 172.21.15.16:0/4040303532 >> [v2:172.21.15.16:6834/642332950,v1:172.21.15.16:6835/642332950] conn(0x55d089b5f6e0 0x55d089b2fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.768+0000 7f7d9508e740 4 client.5463 mds_command: new command op to 5340 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:45:16.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.768+0000 7f7d9508e740 1 -- 172.21.15.16:0/4040303532 --> [v2:172.21.15.16:6834/642332950,v1:172.21.15.16:6835/642332950] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d089a75c50 con 0x55d089b5f6e0 2024-09-19T07:45:16.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.768+0000 7f7d8bfff640 1 --2- 172.21.15.16:0/4040303532 >> [v2:172.21.15.16:6834/642332950,v1:172.21.15.16:6835/642332950] conn(0x55d089b5f6e0 0x55d089b2fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.768+0000 7f7d6affd640 10 client.5463 ms_handle_connect on v2:172.21.15.16:6834/642332950 2024-09-19T07:45:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.768+0000 7f7d8bfff640 1 --2- 172.21.15.16:0/4040303532 >> [v2:172.21.15.16:6834/642332950,v1:172.21.15.16:6835/642332950] conn(0x55d089b5f6e0 0x55d089b2fd90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e91427d4b052a70 server_cookie=cd6c1ee835150a43 in_seq=0 out_seq=0 2024-09-19T07:45:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.769+0000 7f7d6affd640 1 -- 172.21.15.16:0/4040303532 <== mds.0 v2:172.21.15.16:6834/642332950 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d089a75c50 con 0x55d089b5f6e0 2024-09-19T07:45:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.769+0000 7f7d6affd640 10 client.5463 handle_command_reply: tid=0 2024-09-19T07:45:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.769+0000 7f7d9508e740 1 client.5463 shutdown 2024-09-19T07:45:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.769+0000 7f7d9508e740 1 -- 172.21.15.16:0/4040303532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b2c310 msgr2=0x55d089b2c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.769+0000 7f7d9508e740 1 --2- 172.21.15.16:0/4040303532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b2c310 0x55d089b2c790 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7d84061b00 tx=0x7f7d84001250 comp rx=0 tx=0).stop 2024-09-19T07:45:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.769+0000 7f7d9508e740 1 -- 172.21.15.16:0/4040303532 shutdown_connections 2024-09-19T07:45:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.769+0000 7f7d9508e740 1 --2- 172.21.15.16:0/4040303532 >> [v2:172.21.15.16:6834/642332950,v1:172.21.15.16:6835/642332950] conn(0x55d089b5f6e0 0x55d089b2fd90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.769+0000 7f7d9508e740 1 --2- 172.21.15.16:0/4040303532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b2c310 0x55d089b2c790 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.769+0000 7f7d6affd640 0 client.5463 ms_handle_reset on v2:172.21.15.16:6834/642332950 2024-09-19T07:45:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.769+0000 7f7d9508e740 1 --2- 172.21.15.16:0/4040303532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b07a50 0x55d089b26200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.769+0000 7f7d9508e740 1 --2- 172.21.15.16:0/4040303532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b062c0 0x55d089b48640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.769+0000 7f7d9508e740 1 -- 172.21.15.16:0/4040303532 >> 172.21.15.16:0/4040303532 conn(0x55d089b03a80 msgr2=0x55d089b48ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:16.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.770+0000 7f7d9508e740 1 -- 172.21.15.16:0/4040303532 shutdown_connections 2024-09-19T07:45:16.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.770+0000 7f7d9508e740 1 -- 172.21.15.16:0/4040303532 wait complete. 2024-09-19T07:45:16.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.770+0000 7f7d9508e740 20 client.5463 trim_cache size 0 max 16384 2024-09-19T07:45:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.788+0000 7f7d9508e740 1 Processor -- start 2024-09-19T07:45:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.788+0000 7f7d9508e740 1 -- start start 2024-09-19T07:45:16.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.788+0000 7f7d9508e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b062c0 0x55d089b490d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.788+0000 7f7d9508e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b07a50 0x55d089b6f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.788+0000 7f7d9508e740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b2c310 0x55d089b6f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.788+0000 7f7d9508e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c1fff10 con 0x55d089b07a50 2024-09-19T07:45:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.788+0000 7f7d9508e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c0cd520 con 0x55d089b062c0 2024-09-19T07:45:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.788+0000 7f7d9508e740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8c1ff8c0 con 0x55d089b2c310 2024-09-19T07:45:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.788+0000 7f7d8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b07a50 0x55d089b6f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.788+0000 7f7d8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b07a50 0x55d089b6f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47058/0 (socket says 172.21.15.16:47058) 2024-09-19T07:45:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.788+0000 7f7d8b7fe640 1 -- 172.21.15.16:0/2555140969 learned_addr learned my addr 172.21.15.16:0/2555140969 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.789+0000 7f7d8b7fe640 1 -- 172.21.15.16:0/2555140969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b2c310 msgr2=0x55d089b6f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.789+0000 7f7d8b7fe640 1 --2- 172.21.15.16:0/2555140969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b2c310 0x55d089b6f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.789+0000 7f7d8b7fe640 1 -- 172.21.15.16:0/2555140969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b062c0 msgr2=0x55d089b490d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.789+0000 7f7d8b7fe640 1 --2- 172.21.15.16:0/2555140969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b062c0 0x55d089b490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.789+0000 7f7d8b7fe640 1 -- 172.21.15.16:0/2555140969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d089b27a00 con 0x55d089b07a50 2024-09-19T07:45:16.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.789+0000 7f7d8b7fe640 1 --2- 172.21.15.16:0/2555140969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b07a50 0x55d089b6f330 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7d800015e0 tx=0x7f7d80001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704ea62b35c5c4df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:16.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.789+0000 7f7d717fa640 1 -- 172.21.15.16:0/2555140969 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80019020 con 0x55d089b07a50 2024-09-19T07:45:16.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.789+0000 7f7d717fa640 1 -- 172.21.15.16:0/2555140969 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d80005a60 con 0x55d089b07a50 2024-09-19T07:45:16.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.789+0000 7f7d717fa640 1 -- 172.21.15.16:0/2555140969 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80005e30 con 0x55d089b07a50 2024-09-19T07:45:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.790+0000 7f7d9508e740 1 -- 172.21.15.16:0/2555140969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b07a50 msgr2=0x55d089b6f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.790+0000 7f7d9508e740 1 --2- 172.21.15.16:0/2555140969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b07a50 0x55d089b6f330 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7d800015e0 tx=0x7f7d80001aa0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.790+0000 7f7d9508e740 1 -- 172.21.15.16:0/2555140969 shutdown_connections 2024-09-19T07:45:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.790+0000 7f7d9508e740 1 --2- 172.21.15.16:0/2555140969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b2c310 0x55d089b6f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.790+0000 7f7d9508e740 1 --2- 172.21.15.16:0/2555140969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b07a50 0x55d089b6f330 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.790+0000 7f7d9508e740 1 --2- 172.21.15.16:0/2555140969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b062c0 0x55d089b490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.790+0000 7f7d9508e740 1 -- 172.21.15.16:0/2555140969 >> 172.21.15.16:0/2555140969 conn(0x55d089b03a80 msgr2=0x55d089b4cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.790+0000 7f7d9508e740 1 -- 172.21.15.16:0/2555140969 shutdown_connections 2024-09-19T07:45:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.790+0000 7f7d9508e740 1 -- 172.21.15.16:0/2555140969 wait complete. 2024-09-19T07:45:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.790+0000 7f7d9508e740 1 -- start start 2024-09-19T07:45:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.790+0000 7f7d9508e740 1 Processor -- start 2024-09-19T07:45:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.791+0000 7f7d9508e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b062c0 0x55d089b5e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.791+0000 7f7d9508e740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b07a50 0x55d089baca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.791+0000 7f7d9508e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b2c310 0x55d089b26750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.791+0000 7f7d9508e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d089b01f20 con 0x55d089b2c310 2024-09-19T07:45:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.791+0000 7f7d9508e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d0899f1450 con 0x55d089b062c0 2024-09-19T07:45:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.791+0000 7f7d9508e740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55d089b2e470 con 0x55d089b07a50 2024-09-19T07:45:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.791+0000 7f7d9508e740 10 client.0 _ll_register_callbacks cb 0x7f7d84059310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.791+0000 7f7d9088e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b2c310 0x55d089b26750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.791+0000 7f7d9088e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b2c310 0x55d089b26750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47066/0 (socket says 172.21.15.16:47066) 2024-09-19T07:45:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.791+0000 7f7d9088e640 1 -- 172.21.15.16:0/3963690354 learned_addr learned my addr 172.21.15.16:0/3963690354 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.791+0000 7f7d8b7fe640 1 --2- 172.21.15.16:0/3963690354 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b07a50 0x55d089baca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.791+0000 7f7d8bfff640 1 --2- 172.21.15.16:0/3963690354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b062c0 0x55d089b5e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d9088e640 1 -- 172.21.15.16:0/3963690354 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b07a50 msgr2=0x55d089baca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d9088e640 1 --2- 172.21.15.16:0/3963690354 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b07a50 0x55d089baca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d9088e640 1 -- 172.21.15.16:0/3963690354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b062c0 msgr2=0x55d089b5e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d9088e640 1 --2- 172.21.15.16:0/3963690354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b062c0 0x55d089b5e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d9088e640 1 -- 172.21.15.16:0/3963690354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d089b4e440 con 0x55d089b2c310 2024-09-19T07:45:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d8b7fe640 1 --2- 172.21.15.16:0/3963690354 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b07a50 0x55d089baca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d8bfff640 1 --2- 172.21.15.16:0/3963690354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b062c0 0x55d089b5e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d9088e640 1 --2- 172.21.15.16:0/3963690354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b2c310 0x55d089b26750 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7d84001330 tx=0x7f7d840014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fd8f24738c68c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d6affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:45:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d6affd640 1 -- 172.21.15.16:0/3963690354 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84059b50 con 0x55d089b2c310 2024-09-19T07:45:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d9508e740 1 -- 172.21.15.16:0/3963690354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d80005a60 con 0x55d089b2c310 2024-09-19T07:45:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d6affd640 1 -- 172.21.15.16:0/3963690354 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f7d84055ac0 con 0x55d089b2c310 2024-09-19T07:45:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d6affd640 1 -- 172.21.15.16:0/3963690354 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7d84062030 con 0x55d089b2c310 2024-09-19T07:45:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d9508e740 10 client.5469 fetch_fsmap learned FSMap version 24 2024-09-19T07:45:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d9508e740 1 -- 172.21.15.16:0/3963690354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55d089b2ee20 con 0x55d089b2c310 2024-09-19T07:45:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d6affd640 1 -- 172.21.15.16:0/3963690354 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d80005a60 con 0x55d089b2c310 2024-09-19T07:45:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d6affd640 1 -- 172.21.15.16:0/3963690354 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 3502+0+0 (secure 0 0 0) 0x7f7d84081070 con 0x55d089b2c310 2024-09-19T07:45:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d9508e740 10 client.5469 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T07:45:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d9508e740 10 client.5469 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T07:45:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d9508e740 20 client.5469 populate_metadata read hostname 'smithi016' 2024-09-19T07:45:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d9508e740 1 --2- 172.21.15.16:0/3963690354 >> [v2:172.21.15.16:6834/642332950,v1:172.21.15.16:6835/642332950] conn(0x55d089b04ea0 0x55d089b59510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d9508e740 4 client.5469 mds_command: new command op to 5340 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:45:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d9508e740 1 -- 172.21.15.16:0/3963690354 --> [v2:172.21.15.16:6834/642332950,v1:172.21.15.16:6835/642332950] -- command(tid 0: {"prefix": "status"}) -- 0x55d089a6ffa0 con 0x55d089b04ea0 2024-09-19T07:45:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.792+0000 7f7d8bfff640 1 --2- 172.21.15.16:0/3963690354 >> [v2:172.21.15.16:6834/642332950,v1:172.21.15.16:6835/642332950] conn(0x55d089b04ea0 0x55d089b59510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.793+0000 7f7d6affd640 10 client.5469 ms_handle_connect on v2:172.21.15.16:6834/642332950 2024-09-19T07:45:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.793+0000 7f7d8bfff640 1 --2- 172.21.15.16:0/3963690354 >> [v2:172.21.15.16:6834/642332950,v1:172.21.15.16:6835/642332950] conn(0x55d089b04ea0 0x55d089b59510 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aedaf95718e658b2 server_cookie=dc10500af5b03666 in_seq=0 out_seq=0 2024-09-19T07:45:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.793+0000 7f7d6affd640 1 -- 172.21.15.16:0/3963690354 <== mds.0 v2:172.21.15.16:6834/642332950 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55d089a6ffa0 con 0x55d089b04ea0 2024-09-19T07:45:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.793+0000 7f7d6affd640 10 client.5469 handle_command_reply: tid=0 2024-09-19T07:45:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.793+0000 7f7d9508e740 1 client.5469 shutdown 2024-09-19T07:45:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.793+0000 7f7d9508e740 1 -- 172.21.15.16:0/3963690354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b2c310 msgr2=0x55d089b26750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.793+0000 7f7d9508e740 1 --2- 172.21.15.16:0/3963690354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b2c310 0x55d089b26750 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7d84001330 tx=0x7f7d840014a0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.793+0000 7f7d9508e740 1 -- 172.21.15.16:0/3963690354 shutdown_connections 2024-09-19T07:45:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.793+0000 7f7d9508e740 1 --2- 172.21.15.16:0/3963690354 >> [v2:172.21.15.16:6834/642332950,v1:172.21.15.16:6835/642332950] conn(0x55d089b04ea0 0x55d089b59510 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.793+0000 7f7d9508e740 1 --2- 172.21.15.16:0/3963690354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d089b2c310 0x55d089b26750 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.793+0000 7f7d9508e740 1 --2- 172.21.15.16:0/3963690354 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55d089b07a50 0x55d089baca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.793+0000 7f7d9508e740 1 --2- 172.21.15.16:0/3963690354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d089b062c0 0x55d089b5e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.793+0000 7f7d9508e740 1 -- 172.21.15.16:0/3963690354 >> 172.21.15.16:0/3963690354 conn(0x55d089b03a80 msgr2=0x55d089b493f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.793+0000 7f7d6affd640 0 client.5469 ms_handle_reset on v2:172.21.15.16:6834/642332950 2024-09-19T07:45:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.793+0000 7f7d9508e740 1 -- 172.21.15.16:0/3963690354 shutdown_connections 2024-09-19T07:45:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.793+0000 7f7d9508e740 1 -- 172.21.15.16:0/3963690354 wait complete. 2024-09-19T07:45:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.793+0000 7f7d9508e740 20 client.5469 trim_cache size 0 max 16384 2024-09-19T07:45:16.801 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:45:16.801 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb", 2024-09-19T07:45:16.801 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-19T07:45:16.801 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5340, 2024-09-19T07:45:16.801 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T07:45:16.801 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T07:45:16.801 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T07:45:16.801 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 5.6479393910000004, 2024-09-19T07:45:16.801 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-09-19T07:45:16.801 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 53, 2024-09-19T07:45:16.801 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 53, 2024-09-19T07:45:16.801 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 9.6883202940000004 2024-09-19T07:45:16.801 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:45:16.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.794+0000 7f7d922f1640 1 -- 172.21.15.16:0/417565984 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7d7806c2b0 msgr2=0x7f7d7806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.794+0000 7f7d922f1640 1 --2- 172.21.15.16:0/417565984 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7d7806c2b0 0x7f7d7806e770 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c034000 tx=0x7f7d7c02e000 comp rx=0 tx=0).stop 2024-09-19T07:45:16.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.794+0000 7f7d922f1640 1 -- 172.21.15.16:0/417565984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d8c161eb0 msgr2=0x7f7d8c0d0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.794+0000 7f7d922f1640 1 --2- 172.21.15.16:0/417565984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d8c161eb0 0x7f7d8c0d0da0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400c7a0 tx=0x7f7d8400cc60 comp rx=0 tx=0).stop 2024-09-19T07:45:16.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.794+0000 7f7d922f1640 1 -- 172.21.15.16:0/417565984 shutdown_connections 2024-09-19T07:45:16.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.794+0000 7f7d922f1640 1 --2- 172.21.15.16:0/417565984 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7d7806c2b0 0x7f7d7806e770 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.794+0000 7f7d922f1640 1 --2- 172.21.15.16:0/417565984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d8c161eb0 0x7f7d8c0d0da0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.794+0000 7f7d922f1640 1 --2- 172.21.15.16:0/417565984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d8c1425e0 0x7f7d8c15b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.794+0000 7f7d922f1640 1 --2- 172.21.15.16:0/417565984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d8c06cd20 0x7f7d8c1559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.794+0000 7f7d922f1640 1 -- 172.21.15.16:0/417565984 >> 172.21.15.16:0/417565984 conn(0x7f7d8c0b3ed0 msgr2=0x7f7d8c0b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:16.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.794+0000 7f7d922f1640 1 -- 172.21.15.16:0/417565984 shutdown_connections 2024-09-19T07:45:16.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.794+0000 7f7d922f1640 1 -- 172.21.15.16:0/417565984 wait complete. 2024-09-19T07:45:16.805 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:45:16.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.907+0000 7f9dc259d640 1 Processor -- start 2024-09-19T07:45:16.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.907+0000 7f9dc259d640 1 -- start start 2024-09-19T07:45:16.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.908+0000 7f9dc259d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9dbc15e500 0x7f9dbc0d1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.908+0000 7f9dc259d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dbc0b5210 con 0x7f9dbc0d2ee0 2024-09-19T07:45:16.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.908+0000 7f9dc259d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dbc0b4600 con 0x7f9dbc0b4170 2024-09-19T07:45:16.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.908+0000 7f9dc259d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9dbc0b48e0 con 0x7f9dbc15e500 2024-09-19T07:45:16.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.908+0000 7f9dc159b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9dbc15e500 0x7f9dbc0d1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.908+0000 7f9dc1d9c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9dbc0d2ee0 0x7f9dbc0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50376/0 (socket says 172.21.15.16:50376) 2024-09-19T07:45:16.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.908+0000 7f9dc1d9c640 1 -- 172.21.15.16:0/3368879507 learned_addr learned my addr 172.21.15.16:0/3368879507 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:16.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.908+0000 7f9dc159b640 1 -- 172.21.15.16:0/3368879507 >> v1:172.21.15.146:6789/0 conn(0x7f9dbc0b4170 legacy=0x7f9dbc0d27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.908+0000 7f9dc159b640 1 -- 172.21.15.16:0/3368879507 >> v1:172.21.15.16:6789/0 conn(0x7f9dbc0d2ee0 legacy=0x7f9dbc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.908+0000 7f9dc159b640 1 -- 172.21.15.16:0/3368879507 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dbc0b4bc0 con 0x7f9dbc15e500 2024-09-19T07:45:16.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.909+0000 7f9dc159b640 1 --2- 172.21.15.16:0/3368879507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dbc15e500 0x7f9dbc0d1dc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9db80097a0 tx=0x7f9db802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab40b5b69db3a71b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:16.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.909+0000 7f9db3fff640 1 -- 172.21.15.16:0/3368879507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db8037070 con 0x7f9dbc15e500 2024-09-19T07:45:16.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.909+0000 7f9db3fff640 1 -- 172.21.15.16:0/3368879507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db802ae50 con 0x7f9dbc15e500 2024-09-19T07:45:16.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.909+0000 7f9db3fff640 1 -- 172.21.15.16:0/3368879507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db8032660 con 0x7f9dbc15e500 2024-09-19T07:45:16.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.910+0000 7f9dc259d640 1 -- 172.21.15.16:0/3368879507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dbc15e500 msgr2=0x7f9dbc0d1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.910+0000 7f9dc259d640 1 --2- 172.21.15.16:0/3368879507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dbc15e500 0x7f9dbc0d1dc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9db80097a0 tx=0x7f9db802a1c0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.910+0000 7f9dc259d640 1 -- 172.21.15.16:0/3368879507 shutdown_connections 2024-09-19T07:45:16.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.910+0000 7f9dc259d640 1 --2- 172.21.15.16:0/3368879507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dbc15e500 0x7f9dbc0d1dc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.910+0000 7f9dc259d640 1 -- 172.21.15.16:0/3368879507 >> 172.21.15.16:0/3368879507 conn(0x7f9dbc08dc00 msgr2=0x7f9dbc09d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:16.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.910+0000 7f9dc259d640 1 -- 172.21.15.16:0/3368879507 shutdown_connections 2024-09-19T07:45:16.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.910+0000 7f9dc259d640 1 -- 172.21.15.16:0/3368879507 wait complete. 2024-09-19T07:45:16.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.911+0000 7f9dc259d640 1 Processor -- start 2024-09-19T07:45:16.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.911+0000 7f9dc259d640 1 -- start start 2024-09-19T07:45:16.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.912+0000 7f9dc259d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dbc0d2ee0 0x7f9dbc154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.912+0000 7f9dc259d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dbc15e500 0x7f9dbc0b9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.912+0000 7f9dc259d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9dbc0b9d90 0x7f9dbc1f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.912+0000 7f9dc259d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc0d2480 con 0x7f9dbc15e500 2024-09-19T07:45:16.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.912+0000 7f9dc259d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc0d2300 con 0x7f9dbc0d2ee0 2024-09-19T07:45:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.912+0000 7f9dc259d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9dbc0cf9a0 con 0x7f9dbc0b9d90 2024-09-19T07:45:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.912+0000 7f9dc0d9a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dbc15e500 0x7f9dbc0b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.912+0000 7f9dc0d9a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dbc15e500 0x7f9dbc0b9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47084/0 (socket says 172.21.15.16:47084) 2024-09-19T07:45:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.912+0000 7f9dc0d9a640 1 -- 172.21.15.16:0/2123646295 learned_addr learned my addr 172.21.15.16:0/2123646295 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.912+0000 7f9dc159b640 1 --2- 172.21.15.16:0/2123646295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dbc0d2ee0 0x7f9dbc154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.912+0000 7f9dc0d9a640 1 -- 172.21.15.16:0/2123646295 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9dbc0b9d90 msgr2=0x7f9dbc1f7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.912+0000 7f9dc0d9a640 1 --2- 172.21.15.16:0/2123646295 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9dbc0b9d90 0x7f9dbc1f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.912+0000 7f9dc0d9a640 1 -- 172.21.15.16:0/2123646295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dbc0d2ee0 msgr2=0x7f9dbc154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.912+0000 7f9dc0d9a640 1 --2- 172.21.15.16:0/2123646295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dbc0d2ee0 0x7f9dbc154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.912+0000 7f9dc0d9a640 1 -- 172.21.15.16:0/2123646295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dbc0b83f0 con 0x7f9dbc15e500 2024-09-19T07:45:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.912+0000 7f9dc0d9a640 1 --2- 172.21.15.16:0/2123646295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dbc15e500 0x7f9dbc0b9850 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00c400 tx=0x7f9dac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.912+0000 7f9db1ffb640 1 -- 172.21.15.16:0/2123646295 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac0052d0 con 0x7f9dbc15e500 2024-09-19T07:45:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.912+0000 7f9db1ffb640 1 -- 172.21.15.16:0/2123646295 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dac005490 con 0x7f9dbc15e500 2024-09-19T07:45:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.912+0000 7f9db1ffb640 1 -- 172.21.15.16:0/2123646295 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac007260 con 0x7f9dbc15e500 2024-09-19T07:45:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.913+0000 7f9dc259d640 1 -- 172.21.15.16:0/2123646295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dbc0b7e30 con 0x7f9dbc15e500 2024-09-19T07:45:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.913+0000 7f9dc259d640 1 -- 172.21.15.16:0/2123646295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dbc155e50 con 0x7f9dbc15e500 2024-09-19T07:45:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.914+0000 7f9dc259d640 1 -- 172.21.15.16:0/2123646295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dbc0d1dc0 con 0x7f9dbc15e500 2024-09-19T07:45:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.914+0000 7f9db1ffb640 1 -- 172.21.15.16:0/2123646295 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9dac007400 con 0x7f9dbc15e500 2024-09-19T07:45:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.914+0000 7f9db1ffb640 1 --2- 172.21.15.16:0/2123646295 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9da006c2b0 0x7f9da006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.915+0000 7f9db1ffb640 1 -- 172.21.15.16:0/2123646295 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f9dac08ac30 con 0x7f9dbc15e500 2024-09-19T07:45:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.915+0000 7f9dc159b640 1 --2- 172.21.15.16:0/2123646295 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9da006c2b0 0x7f9da006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.915+0000 7f9dc159b640 1 --2- 172.21.15.16:0/2123646295 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9da006c2b0 0x7f9da006e770 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9db8034000 tx=0x7f9db802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:16.919+0000 7f9db1ffb640 1 -- 172.21.15.16:0/2123646295 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9dbc0b83f0 con 0x7f9dbc15e500 2024-09-19T07:45:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:17.136+0000 7f9dc259d640 1 -- 172.21.15.16:0/2123646295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9dbc06d2c0 con 0x7f9dbc15e500 2024-09-19T07:45:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:17.136+0000 7f9db1ffb640 1 -- 172.21.15.16:0/2123646295 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f9dac0590f0 con 0x7f9dbc15e500 2024-09-19T07:45:17.137 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:45:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:17.138+0000 7f9dc259d640 1 -- 172.21.15.16:0/2123646295 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9da006c2b0 msgr2=0x7f9da006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:17.138+0000 7f9dc259d640 1 --2- 172.21.15.16:0/2123646295 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9da006c2b0 0x7f9da006e770 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9db8034000 tx=0x7f9db802e000 comp rx=0 tx=0).stop 2024-09-19T07:45:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:17.138+0000 7f9dc259d640 1 -- 172.21.15.16:0/2123646295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dbc15e500 msgr2=0x7f9dbc0b9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:17.138+0000 7f9dc259d640 1 --2- 172.21.15.16:0/2123646295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dbc15e500 0x7f9dbc0b9850 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00c400 tx=0x7f9dac00c8c0 comp rx=0 tx=0).stop 2024-09-19T07:45:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:17.138+0000 7f9dc259d640 1 -- 172.21.15.16:0/2123646295 shutdown_connections 2024-09-19T07:45:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:17.138+0000 7f9dc259d640 1 --2- 172.21.15.16:0/2123646295 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9da006c2b0 0x7f9da006e770 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:17.138+0000 7f9dc259d640 1 --2- 172.21.15.16:0/2123646295 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9dbc0b9d90 0x7f9dbc1f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:17.138+0000 7f9dc259d640 1 --2- 172.21.15.16:0/2123646295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dbc15e500 0x7f9dbc0b9850 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:17.138+0000 7f9dc259d640 1 --2- 172.21.15.16:0/2123646295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dbc0d2ee0 0x7f9dbc154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:17.138+0000 7f9dc259d640 1 -- 172.21.15.16:0/2123646295 >> 172.21.15.16:0/2123646295 conn(0x7f9dbc08dc00 msgr2=0x7f9dbc0cfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:17.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:17.138+0000 7f9dc259d640 1 -- 172.21.15.16:0/2123646295 shutdown_connections 2024-09-19T07:45:17.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:17.139+0000 7f9dc259d640 1 -- 172.21.15.16:0/2123646295 wait complete. 2024-09-19T07:45:17.149 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:45:17.149 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:45:17.149 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T07:45:17.150 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T07:45:17.150 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:45:17.150 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:45:17.150 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:45:17.150 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:45:17.150 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:45:17.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:17.150 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T07:45:17.167 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:45:17.167 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:45:17.167 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:45:17.167 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:17.167 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:45:17.167 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:17.167 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:45:17.167 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:45:17.167 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T07:45:17.167 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:45:17.167 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:45:17.168 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T07:45:17.168 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:45:17.168 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:45:17.168 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T07:45:17.168 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T07:45:17.168 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42672sec preferred_lft 42672sec 2024-09-19T07:45:17.168 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T07:45:17.168 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:17.168 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:45:17.168 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:45:17.168 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T07:45:17.168 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:45:17.168 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:45:17.169 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:45:17.169 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:17.169 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::32:7bff:fe92:8895/64 scope link 2024-09-19T07:45:17.169 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:17.169 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:45:17.169 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether e6:fd:21:6a:43:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:45:17.169 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e4fd:21ff:fe6a:4369/64 scope link 2024-09-19T07:45:17.169 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:17.169 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:45:17.169 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:45:17.169 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::80f4:81ff:fe7d:b7f0/64 scope link 2024-09-19T07:45:17.169 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:17.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:17.170 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:45:17.231 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:45:17.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:17.233 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T07:45:17.290 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T07:45:17.290 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:45:17.290 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T07:45:17.290 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T07:45:17.290 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:45:17.290 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:45:17.290 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:45:17.290 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T07:45:17.290 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T07:45:17.291 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T07:45:17.291 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:45:17.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:45:17.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:45:17.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:17.419 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:45:17.435 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T07:45:17.435 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:45:17.435 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:45:17.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:17.436 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-19T07:45:17.501 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T07:45:17.501 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:45:17.501 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T07:45:17.557 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:45:17.557 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:45:17.557 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:45:17.613 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:45:17.680 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:45:17.736 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:45:17.736 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:45:17.736 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:45:17.736 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:45:17.736 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:45:17.737 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:45:17.737 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:45:17.737 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:45:17.737 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:45:17.737 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:45:17.737 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:45:17.737 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:45:17.737 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:45:17.737 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:45:17.737 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:45:17.737 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:45:17.737 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:45:17.737 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:45:17.738 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:45:17.738 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:45:17.738 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:45:17.738 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:45:17.738 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:45:17.738 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:45:17.738 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:45:17.738 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:45:17.738 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:45:17.738 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:45:17.738 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:45:17.738 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:45:17.738 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:45:17.738 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:45:17.739 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:45:17.739 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:45:17.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:17.740 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:45:17.796 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:45:17.796 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T07:45:17.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:45:17.864 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:45:17.922 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:45:17.922 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:45:17.922 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:45:17.922 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:45:17.922 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:45:17.922 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:45:17.923 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:45:17.923 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:45:17.923 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:45:17.923 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:45:17.923 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:45:17.923 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:45:17.923 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:45:17.923 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:45:17.923 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:45:17.923 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:45:17.923 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:45:17.923 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:45:17.923 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:45:17.924 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:45:17.924 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:45:17.924 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:45:17.924 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:45:17.924 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:45:17.924 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:45:17.924 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:45:17.924 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:45:17.924 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:45:17.924 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:45:17.924 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:45:17.925 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:45:17.925 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:45:17.925 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:45:17.925 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:45:17.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:17.925 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:45:17.926 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-19T07:45:17.924+0000 7f8dc6689480 -1 init, newargv = 0x55db415099c0 newargc=15 2024-09-19T07:45:17.926 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[51210]: starting ceph client 2024-09-19T07:45:17.928 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[51210]: starting fuse 2024-09-19T07:45:17.942 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:45:17.942 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-19T07:45:17.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:45:17.942 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:45:17.942 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:45:17.942 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:45:17.942 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:45:17.942 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:45:17.942 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:45:17.942 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:45:17.942 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:45:17.943 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:45:17.943 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:45:17.943 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:45:17.943 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:45:17.943 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:45:17.943 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:45:17.943 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:45:17.943 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:45:17.943 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:45:17.943 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:45:17.943 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:45:17.943 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:45:17.943 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:45:17.943 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:45:17.944 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:45:17.944 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:45:17.944 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:45:17.944 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:45:18.062 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.51210.asok 2024-09-19T07:45:18.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:45:18.072 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51210.asok 2024-09-19T07:45:18.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:18.073 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51210.asok status 2024-09-19T07:45:18.211 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:45:18.211 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:45:18.211 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:45:18.211 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:45:18.211 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:45:18.211 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:45:18.212 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:45:18.212 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "51210", 2024-09-19T07:45:18.212 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:45:18.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:18.212 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-19T07:45:18.212 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-19T07:45:18.212 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5484, 2024-09-19T07:45:18.212 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:45:18.212 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:45:18.212 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:45:18.212 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5484 2024-09-19T07:45:18.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:18.213 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:45:18.213 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:45:18.213 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:45:18.213 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2706133929 2024-09-19T07:45:18.213 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:18.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:18.213 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:45:18.213 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:45:18.213 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:45:18.213 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2706133929 2024-09-19T07:45:18.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:18.213 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.5484 192.168.0.1:0/2706133929", 2024-09-19T07:45:18.213 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/2706133929", 2024-09-19T07:45:18.213 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-19T07:45:18.214 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 24, 2024-09-19T07:45:18.214 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 53, 2024-09-19T07:45:18.214 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:45:18.214 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:45:18.214 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:45:18.214 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:45:18.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:18.220 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-19T07:45:18.220 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T07:45:18.236 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-19T07:45:18.236 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T07:45:18.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:18.236 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:45:18.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:45:18.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.445+0000 7fefd4217640 1 Processor -- start 2024-09-19T07:45:18.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.445+0000 7fefd4217640 1 -- start start 2024-09-19T07:45:18.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.446+0000 7fefd4217640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fefcc15e2a0 0x7fefcc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.446+0000 7fefd4217640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefcc06d960 0x7fefcc0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.446+0000 7fefd4217640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefcc0d1880 con 0x7fefcc15e680 2024-09-19T07:45:18.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.446+0000 7fefd4217640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fefcc0d1b60 con 0x7fefcc15e2a0 2024-09-19T07:45:18.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.446+0000 7fefd4217640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fefcc0cf2d0 con 0x7fefcc06d960 2024-09-19T07:45:18.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.446+0000 7fefd27b4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefcc06d960 0x7fefcc0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.446+0000 7fefd1fb3640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fefcc15e2a0 0x7fefcc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.446+0000 7fefd27b4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefcc06d960 0x7fefcc0d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:46826/0 (socket says 172.21.15.16:46826) 2024-09-19T07:45:18.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.446+0000 7fefd1fb3640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fefcc15e2a0 0x7fefcc06d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:33876/0 (socket says 172.21.15.16:33876) 2024-09-19T07:45:18.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.446+0000 7fefd17b2640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fefcc15e680 0x7fefcc0cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:54182/0 (socket says 172.21.15.16:54182) 2024-09-19T07:45:18.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.446+0000 7fefd27b4640 1 -- 172.21.15.16:0/1671039546 learned_addr learned my addr 172.21.15.16:0/1671039546 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:18.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.446+0000 7fefd0fb1640 1 -- 172.21.15.16:0/1671039546 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 297238150 0 0) 0x7fefcc0d1880 con 0x7fefcc15e680 2024-09-19T07:45:18.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.446+0000 7fefd27b4640 1 -- 172.21.15.16:0/1671039546 >> v2:172.21.15.146:3300/0 conn(0x7fefcc15e2a0 msgr2=0x7fefcc06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:18.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.446+0000 7fefd27b4640 1 --2- 172.21.15.16:0/1671039546 >> v2:172.21.15.146:3300/0 conn(0x7fefcc15e2a0 0x7fefcc06d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.446+0000 7fefd27b4640 1 -- 172.21.15.16:0/1671039546 >> v1:172.21.15.146:6789/0 conn(0x7fefcc15e680 legacy=0x7fefcc0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:18.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.446+0000 7fefd27b4640 1 -- 172.21.15.16:0/1671039546 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefcc0cf5b0 con 0x7fefcc06d960 2024-09-19T07:45:18.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.446+0000 7fefd1fb3640 1 --2- 172.21.15.16:0/1671039546 >> v2:172.21.15.146:3300/0 conn(0x7fefcc15e2a0 0x7fefcc06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:18.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.447+0000 7fefd27b4640 1 --2- 172.21.15.16:0/1671039546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefcc06d960 0x7fefcc0d4370 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009e50 tx=0x7fefc402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed7ca692ef49253b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:18.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.447+0000 7fefd0fb1640 1 -- 172.21.15.16:0/1671039546 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc4036070 con 0x7fefcc06d960 2024-09-19T07:45:18.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.447+0000 7fefd0fb1640 1 -- 172.21.15.16:0/1671039546 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc4031600 con 0x7fefcc06d960 2024-09-19T07:45:18.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.447+0000 7fefd0fb1640 1 -- 172.21.15.16:0/1671039546 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc40319d0 con 0x7fefcc06d960 2024-09-19T07:45:18.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.448+0000 7fefd4217640 1 -- 172.21.15.16:0/1671039546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefcc06d960 msgr2=0x7fefcc0d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:18.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.448+0000 7fefd4217640 1 --2- 172.21.15.16:0/1671039546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefcc06d960 0x7fefcc0d4370 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009e50 tx=0x7fefc402a710 comp rx=0 tx=0).stop 2024-09-19T07:45:18.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.448+0000 7fefd4217640 1 -- 172.21.15.16:0/1671039546 shutdown_connections 2024-09-19T07:45:18.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.448+0000 7fefd4217640 1 --2- 172.21.15.16:0/1671039546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefcc06d960 0x7fefcc0d4370 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.448+0000 7fefd4217640 1 --2- 172.21.15.16:0/1671039546 >> v2:172.21.15.146:3300/0 conn(0x7fefcc15e2a0 0x7fefcc06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.448+0000 7fefd4217640 1 -- 172.21.15.16:0/1671039546 >> 172.21.15.16:0/1671039546 conn(0x7fefcc09d760 msgr2=0x7fefcc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:18.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.448+0000 7fefd4217640 1 -- 172.21.15.16:0/1671039546 shutdown_connections 2024-09-19T07:45:18.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.448+0000 7fefd4217640 1 -- 172.21.15.16:0/1671039546 wait complete. 2024-09-19T07:45:18.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.449+0000 7fefd4217640 1 Processor -- start 2024-09-19T07:45:18.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.449+0000 7fefd4217640 1 -- start start 2024-09-19T07:45:18.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.450+0000 7fefd4217640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefcc06d960 0x7fefcc15c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.450+0000 7fefd4217640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefcc15e2a0 0x7fefcc15a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.450+0000 7fefd4217640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefcc15e680 0x7fefcc0b9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.450+0000 7fefd4217640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fefcc0d0360 con 0x7fefcc15e2a0 2024-09-19T07:45:18.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.450+0000 7fefd4217640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fefcc0d01e0 con 0x7fefcc15e680 2024-09-19T07:45:18.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.450+0000 7fefd4217640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fefcc0d48b0 con 0x7fefcc06d960 2024-09-19T07:45:18.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.450+0000 7fefd17b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefcc15e2a0 0x7fefcc15a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.450+0000 7fefd17b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefcc15e2a0 0x7fefcc15a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47090/0 (socket says 172.21.15.16:47090) 2024-09-19T07:45:18.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.450+0000 7fefd17b2640 1 -- 172.21.15.16:0/1670625105 learned_addr learned my addr 172.21.15.16:0/1670625105 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:18.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.450+0000 7fefd1fb3640 1 --2- 172.21.15.16:0/1670625105 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefcc06d960 0x7fefcc15c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.450+0000 7fefd27b4640 1 --2- 172.21.15.16:0/1670625105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefcc15e680 0x7fefcc0b9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.450+0000 7fefd17b2640 1 -- 172.21.15.16:0/1670625105 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefcc06d960 msgr2=0x7fefcc15c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:18.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.450+0000 7fefd17b2640 1 --2- 172.21.15.16:0/1670625105 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefcc06d960 0x7fefcc15c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.450+0000 7fefd17b2640 1 -- 172.21.15.16:0/1670625105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefcc15e680 msgr2=0x7fefcc0b9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:18.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.450+0000 7fefd17b2640 1 --2- 172.21.15.16:0/1670625105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefcc15e680 0x7fefcc0b9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.450+0000 7fefd17b2640 1 -- 172.21.15.16:0/1670625105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefcc15b230 con 0x7fefcc15e2a0 2024-09-19T07:45:18.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.451+0000 7fefd17b2640 1 --2- 172.21.15.16:0/1670625105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefcc15e2a0 0x7fefcc15a8e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00c410 tx=0x7fefbc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:18.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.451+0000 7fefc27fc640 1 -- 172.21.15.16:0/1670625105 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc0074e0 con 0x7fefcc15e2a0 2024-09-19T07:45:18.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.451+0000 7fefc27fc640 1 -- 172.21.15.16:0/1670625105 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefbc0047e0 con 0x7fefcc15e2a0 2024-09-19T07:45:18.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.451+0000 7fefd4217640 1 -- 172.21.15.16:0/1670625105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefcc15b510 con 0x7fefcc15e2a0 2024-09-19T07:45:18.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.451+0000 7fefc27fc640 1 -- 172.21.15.16:0/1670625105 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc004bd0 con 0x7fefcc15e2a0 2024-09-19T07:45:18.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.451+0000 7fefd4217640 1 -- 172.21.15.16:0/1670625105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefcc15bad0 con 0x7fefcc15e2a0 2024-09-19T07:45:18.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.452+0000 7fefd4217640 1 -- 172.21.15.16:0/1670625105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefcc0d4c50 con 0x7fefcc15e2a0 2024-09-19T07:45:18.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.452+0000 7fefc27fc640 1 -- 172.21.15.16:0/1670625105 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fefbc011070 con 0x7fefcc15e2a0 2024-09-19T07:45:18.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.453+0000 7fefc27fc640 1 --2- 172.21.15.16:0/1670625105 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fefb006c2e0 0x7fefb006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.453+0000 7fefc27fc640 1 -- 172.21.15.16:0/1670625105 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fefbc08b0a0 con 0x7fefcc15e2a0 2024-09-19T07:45:18.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.453+0000 7fefd1fb3640 1 --2- 172.21.15.16:0/1670625105 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fefb006c2e0 0x7fefb006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.454+0000 7fefd1fb3640 1 --2- 172.21.15.16:0/1670625105 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fefb006c2e0 0x7fefb006e7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0cb120 tx=0x7fefc8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:18.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.457+0000 7fefc27fc640 1 -- 172.21.15.16:0/1670625105 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fefbc059030 con 0x7fefcc15e2a0 2024-09-19T07:45:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.667+0000 7fefd4217640 1 -- 172.21.15.16:0/1670625105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fefcc06cf90 con 0x7fefcc15e2a0 2024-09-19T07:45:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.668+0000 7fefc27fc640 1 -- 172.21.15.16:0/1670625105 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7fefbc05d330 con 0x7fefcc15e2a0 2024-09-19T07:45:18.668 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:45:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.670+0000 7fefd4217640 1 -- 172.21.15.16:0/1670625105 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fefb006c2e0 msgr2=0x7fefb006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.670+0000 7fefd4217640 1 --2- 172.21.15.16:0/1670625105 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fefb006c2e0 0x7fefb006e7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0cb120 tx=0x7fefc8009190 comp rx=0 tx=0).stop 2024-09-19T07:45:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.670+0000 7fefd4217640 1 -- 172.21.15.16:0/1670625105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefcc15e2a0 msgr2=0x7fefcc15a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.670+0000 7fefd4217640 1 --2- 172.21.15.16:0/1670625105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefcc15e2a0 0x7fefcc15a8e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00c410 tx=0x7fefbc00c8d0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.670+0000 7fefd4217640 1 -- 172.21.15.16:0/1670625105 shutdown_connections 2024-09-19T07:45:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.670+0000 7fefd4217640 1 --2- 172.21.15.16:0/1670625105 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fefb006c2e0 0x7fefb006e7a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.670+0000 7fefd4217640 1 --2- 172.21.15.16:0/1670625105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefcc15e680 0x7fefcc0b9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.670+0000 7fefd4217640 1 --2- 172.21.15.16:0/1670625105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefcc15e2a0 0x7fefcc15a8e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.670+0000 7fefd4217640 1 --2- 172.21.15.16:0/1670625105 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefcc06d960 0x7fefcc15c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.670+0000 7fefd4217640 1 -- 172.21.15.16:0/1670625105 >> 172.21.15.16:0/1670625105 conn(0x7fefcc09d760 msgr2=0x7fefcc0d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.670+0000 7fefd4217640 1 -- 172.21.15.16:0/1670625105 shutdown_connections 2024-09-19T07:45:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:18.670+0000 7fefd4217640 1 -- 172.21.15.16:0/1670625105 wait complete. 2024-09-19T07:45:18.681 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:45:18.681 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:45:18.681 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-09-19T07:45:18.681 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T07:45:18.681 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T07:45:18.681 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:45:18.681 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:45:18.681 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:45:18.681 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:45:18.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:18.682 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-09-19T07:45:18.701 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:45:18.701 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:45:18.701 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:45:18.702 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:18.702 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-09-19T07:45:18.702 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:18.702 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:45:18.702 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-09-19T07:45:18.702 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2024-09-19T07:45:18.702 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:45:18.702 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-09-19T07:45:18.702 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2024-09-19T07:45:18.702 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:45:18.702 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-09-19T07:45:18.702 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:45:18.702 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-09-19T07:45:18.702 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T07:45:18.703 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42638sec preferred_lft 42638sec 2024-09-19T07:45:18.703 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link noprefixroute 2024-09-19T07:45:18.703 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:18.703 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:45:18.703 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:45:18.703 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:45:18.703 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:18.703 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::e47f:32ff:fe68:31ca/64 scope link 2024-09-19T07:45:18.703 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:18.703 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:45:18.703 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether f6:1f:22:44:9d:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:45:18.703 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::f41f:22ff:fe44:9dcd/64 scope link 2024-09-19T07:45:18.703 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:18.703 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:45:18.704 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:45:18.704 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::8459:eeff:fe21:a113/64 scope link 2024-09-19T07:45:18.704 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:18.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:18.704 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:45:18.770 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-09-19T07:45:18.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:18.772 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-09-19T07:45:18.830 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-09-19T07:45:18.830 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:45:18.830 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T07:45:18.830 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T07:45:18.831 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:45:18.831 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:45:18.831 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-19T07:45:18.831 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T07:45:18.831 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T07:45:18.831 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T07:45:18.831 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-19T07:45:18.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:45:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:45:18.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:45:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:45:18.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:18.963 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-09-19T07:45:18.978 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T07:45:18.978 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T07:45:18.979 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:45:18.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:18.979 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2024-09-19T07:45:19.044 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T07:45:19.044 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:45:19.044 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T07:45:19.099 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:45:19.099 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:45:19.100 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:45:19.155 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:45:19.220 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:45:19.277 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:45:19.277 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:45:19.277 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:45:19.277 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:45:19.277 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:45:19.278 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:45:19.278 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:45:19.278 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:45:19.278 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:45:19.278 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:45:19.278 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:45:19.278 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:45:19.278 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:45:19.278 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:45:19.278 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:45:19.278 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:45:19.278 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:45:19.278 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:45:19.278 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:45:19.279 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:45:19.279 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:45:19.279 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:45:19.279 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:45:19.279 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:45:19.279 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:45:19.279 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:45:19.279 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:45:19.279 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:45:19.279 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:45:19.279 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:45:19.279 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:45:19.280 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:45:19.280 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:45:19.280 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:45:19.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:19.280 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:45:19.333 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:45:19.333 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T07:45:19.375 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:45:19.399 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:45:19.455 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:45:19.455 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:45:19.455 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:45:19.455 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:45:19.456 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:45:19.456 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:45:19.456 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:45:19.456 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:45:19.456 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:45:19.456 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:45:19.456 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:45:19.456 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:45:19.456 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:45:19.456 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:45:19.456 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:45:19.456 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:45:19.456 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:45:19.456 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:45:19.456 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:45:19.457 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:45:19.457 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:45:19.457 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:45:19.457 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:45:19.457 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:45:19.457 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:45:19.457 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:45:19.457 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:45:19.457 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:45:19.457 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:45:19.457 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:45:19.457 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:45:19.457 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:45:19.457 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:45:19.458 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:45:19.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:19.458 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:45:19.459 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2024-09-19T07:45:19.456+0000 7f12d6477480 -1 init, newargv = 0x55794eaaf5c0 newargc=15 2024-09-19T07:45:19.459 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[44802]: starting ceph client 2024-09-19T07:45:19.462 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[44802]: starting fuse 2024-09-19T07:45:19.474 INFO:teuthology.orchestra.run.smithi146.stdout:42 2024-09-19T07:45:19.474 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-19T07:45:19.474 DEBUG:teuthology.orchestra.run.smithi146:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:45:19.474 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2024-09-19T07:45:19.474 DEBUG:teuthology.orchestra.run.smithi146:> import re 2024-09-19T07:45:19.475 DEBUG:teuthology.orchestra.run.smithi146:> import os 2024-09-19T07:45:19.475 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2024-09-19T07:45:19.475 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:45:19.475 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2024-09-19T07:45:19.475 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T07:45:19.475 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2024-09-19T07:45:19.475 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T07:45:19.475 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:45:19.475 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2024-09-19T07:45:19.475 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2024-09-19T07:45:19.475 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2024-09-19T07:45:19.475 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2024-09-19T07:45:19.475 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:45:19.475 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2024-09-19T07:45:19.475 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:45:19.475 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:45:19.476 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:45:19.476 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2024-09-19T07:45:19.476 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2024-09-19T07:45:19.476 DEBUG:teuthology.orchestra.run.smithi146:> return f 2024-09-19T07:45:19.476 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:45:19.476 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:45:19.476 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2024-09-19T07:45:19.476 DEBUG:teuthology.orchestra.run.smithi146:> ' 2024-09-19T07:45:19.591 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.44802.asok 2024-09-19T07:45:19.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:45:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:45:19.603 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.44802.asok 2024-09-19T07:45:19.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:19.603 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.44802.asok status 2024-09-19T07:45:19.736 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-19T07:45:19.736 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2024-09-19T07:45:19.736 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:45:19.736 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:45:19.737 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2024-09-19T07:45:19.737 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2024-09-19T07:45:19.737 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T07:45:19.737 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "44802", 2024-09-19T07:45:19.737 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2024-09-19T07:45:19.737 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:45:19.737 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2024-09-19T07:45:19.737 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2024-09-19T07:45:19.737 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 5045, 2024-09-19T07:45:19.737 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2024-09-19T07:45:19.737 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2024-09-19T07:45:19.737 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2024-09-19T07:45:19.737 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 5045 2024-09-19T07:45:19.738 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:45:19.738 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:45:19.738 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:45:19.738 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:45:19.738 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 659723888 2024-09-19T07:45:19.738 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-19T07:45:19.738 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:45:19.738 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:45:19.738 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:45:19.738 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:45:19.738 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 659723888 2024-09-19T07:45:19.738 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:45:19.739 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.5045 192.168.0.1:0/659723888", 2024-09-19T07:45:19.739 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/659723888", 2024-09-19T07:45:19.739 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2024-09-19T07:45:19.739 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 24, 2024-09-19T07:45:19.739 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 53, 2024-09-19T07:45:19.739 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:45:19.739 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2024-09-19T07:45:19.739 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2024-09-19T07:45:19.739 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-19T07:45:19.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:19.745 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2024-09-19T07:45:19.745 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T07:45:19.760 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2024-09-19T07:45:19.761 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T07:45:19.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:19.761 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:45:19.826 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set client client_inject_fixed_oldest_tid true 2024-09-19T07:45:19.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.944+0000 7f1a8574b640 1 Processor -- start 2024-09-19T07:45:19.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.945+0000 7f1a8574b640 1 -- start start 2024-09-19T07:45:19.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.945+0000 7f1a8574b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a80072250 0x7f1a800cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:19.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.945+0000 7f1a8574b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a800d2b30 con 0x7f1a80072690 2024-09-19T07:45:19.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.945+0000 7f1a8574b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a80141c60 con 0x7f1a800d2750 2024-09-19T07:45:19.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.945+0000 7f1a8574b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1a80145bd0 con 0x7f1a80072250 2024-09-19T07:45:19.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.946+0000 7f1a7f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1a80072690 0x7f1a80160010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50380/0 (socket says 172.21.15.16:50380) 2024-09-19T07:45:19.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.946+0000 7f1a7f7fe640 1 -- 172.21.15.16:0/2754256771 learned_addr learned my addr 172.21.15.16:0/2754256771 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:19.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.946+0000 7f1a7e7fc640 1 --2- 172.21.15.16:0/2754256771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a80072250 0x7f1a800cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:19.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.946+0000 7f1a7dffb640 1 -- 172.21.15.16:0/2754256771 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3629737336 0 0) 0x7f1a800d2b30 con 0x7f1a80072690 2024-09-19T07:45:19.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.946+0000 7f1a7dffb640 1 -- 172.21.15.16:0/2754256771 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a6c003520 con 0x7f1a80072690 2024-09-19T07:45:19.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.946+0000 7f1a7dffb640 1 -- 172.21.15.16:0/2754256771 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2991289283 0 0) 0x7f1a80141c60 con 0x7f1a800d2750 2024-09-19T07:45:19.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.946+0000 7f1a7dffb640 1 -- 172.21.15.16:0/2754256771 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a800d2b30 con 0x7f1a800d2750 2024-09-19T07:45:19.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.946+0000 7f1a7e7fc640 1 -- 172.21.15.16:0/2754256771 >> v1:172.21.15.146:6789/0 conn(0x7f1a800d2750 legacy=0x7f1a8013f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:19.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.946+0000 7f1a7e7fc640 1 -- 172.21.15.16:0/2754256771 >> v1:172.21.15.16:6789/0 conn(0x7f1a80072690 legacy=0x7f1a80160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:19.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.946+0000 7f1a7e7fc640 1 -- 172.21.15.16:0/2754256771 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a80141f40 con 0x7f1a80072250 2024-09-19T07:45:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.947+0000 7f1a7e7fc640 1 --2- 172.21.15.16:0/2754256771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a80072250 0x7f1a800cfdc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1a680097d0 tx=0x7f1a6802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6810cebdaea86e44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.947+0000 7f1a7dffb640 1 -- 172.21.15.16:0/2754256771 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a68037070 con 0x7f1a80072250 2024-09-19T07:45:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.947+0000 7f1a7dffb640 1 -- 172.21.15.16:0/2754256771 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a68032440 con 0x7f1a80072250 2024-09-19T07:45:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.947+0000 7f1a7dffb640 1 -- 172.21.15.16:0/2754256771 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a68032810 con 0x7f1a80072250 2024-09-19T07:45:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.948+0000 7f1a8574b640 1 -- 172.21.15.16:0/2754256771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a80072250 msgr2=0x7f1a800cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.948+0000 7f1a8574b640 1 --2- 172.21.15.16:0/2754256771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a80072250 0x7f1a800cfdc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1a680097d0 tx=0x7f1a6802a180 comp rx=0 tx=0).stop 2024-09-19T07:45:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.948+0000 7f1a8574b640 1 -- 172.21.15.16:0/2754256771 shutdown_connections 2024-09-19T07:45:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.948+0000 7f1a8574b640 1 --2- 172.21.15.16:0/2754256771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a80072250 0x7f1a800cfdc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.948+0000 7f1a8574b640 1 -- 172.21.15.16:0/2754256771 >> 172.21.15.16:0/2754256771 conn(0x7f1a800b4040 msgr2=0x7f1a800b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.948+0000 7f1a8574b640 1 -- 172.21.15.16:0/2754256771 shutdown_connections 2024-09-19T07:45:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.948+0000 7f1a8574b640 1 -- 172.21.15.16:0/2754256771 wait complete. 2024-09-19T07:45:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.949+0000 7f1a8574b640 1 Processor -- start 2024-09-19T07:45:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.949+0000 7f1a8574b640 1 -- start start 2024-09-19T07:45:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.950+0000 7f1a8574b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a80072250 0x7f1a8006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.950+0000 7f1a8574b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a80072690 0x7f1a800cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.950+0000 7f1a8574b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a800d2750 0x7f1a800b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.950+0000 7f1a8574b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a80142500 con 0x7f1a800d2750 2024-09-19T07:45:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.950+0000 7f1a8574b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801427e0 con 0x7f1a80072690 2024-09-19T07:45:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.950+0000 7f1a8574b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1a801418b0 con 0x7f1a80072250 2024-09-19T07:45:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.950+0000 7f1a7f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a800d2750 0x7f1a800b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.950+0000 7f1a7f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a800d2750 0x7f1a800b0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47100/0 (socket says 172.21.15.16:47100) 2024-09-19T07:45:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.950+0000 7f1a7f7fe640 1 -- 172.21.15.16:0/1127916440 learned_addr learned my addr 172.21.15.16:0/1127916440 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.950+0000 7f1a7effd640 1 --2- 172.21.15.16:0/1127916440 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a80072250 0x7f1a8006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.950+0000 7f1a7e7fc640 1 --2- 172.21.15.16:0/1127916440 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a80072690 0x7f1a800cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.950+0000 7f1a7f7fe640 1 -- 172.21.15.16:0/1127916440 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a80072250 msgr2=0x7f1a8006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.950+0000 7f1a7f7fe640 1 --2- 172.21.15.16:0/1127916440 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a80072250 0x7f1a8006cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.950+0000 7f1a7f7fe640 1 -- 172.21.15.16:0/1127916440 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a80072690 msgr2=0x7f1a800cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.950+0000 7f1a7f7fe640 1 --2- 172.21.15.16:0/1127916440 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a80072690 0x7f1a800cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.950+0000 7f1a7f7fe640 1 -- 172.21.15.16:0/1127916440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a80144ff0 con 0x7f1a800d2750 2024-09-19T07:45:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.950+0000 7f1a7e7fc640 1 --2- 172.21.15.16:0/1127916440 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a80072690 0x7f1a800cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.951+0000 7f1a7f7fe640 1 --2- 172.21.15.16:0/1127916440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a800d2750 0x7f1a800b0e10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000cae0 tx=0x7f1a7000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.951+0000 7f1a777fe640 1 -- 172.21.15.16:0/1127916440 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70017070 con 0x7f1a800d2750 2024-09-19T07:45:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.951+0000 7f1a777fe640 1 -- 172.21.15.16:0/1127916440 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a7000ac40 con 0x7f1a800d2750 2024-09-19T07:45:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.951+0000 7f1a777fe640 1 -- 172.21.15.16:0/1127916440 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70012400 con 0x7f1a800d2750 2024-09-19T07:45:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.951+0000 7f1a8574b640 1 -- 172.21.15.16:0/1127916440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a801452d0 con 0x7f1a800d2750 2024-09-19T07:45:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.951+0000 7f1a8574b640 1 -- 172.21.15.16:0/1127916440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a80145890 con 0x7f1a800d2750 2024-09-19T07:45:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.952+0000 7f1a8574b640 1 -- 172.21.15.16:0/1127916440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a8015f980 con 0x7f1a800d2750 2024-09-19T07:45:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.953+0000 7f1a777fe640 1 -- 172.21.15.16:0/1127916440 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1a7001a070 con 0x7f1a800d2750 2024-09-19T07:45:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.953+0000 7f1a777fe640 1 --2- 172.21.15.16:0/1127916440 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1a6006c2b0 0x7f1a6006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.953+0000 7f1a777fe640 1 -- 172.21.15.16:0/1127916440 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f1a7008b310 con 0x7f1a800d2750 2024-09-19T07:45:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.953+0000 7f1a7effd640 1 --2- 172.21.15.16:0/1127916440 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1a6006c2b0 0x7f1a6006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.954+0000 7f1a7effd640 1 --2- 172.21.15.16:0/1127916440 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1a6006c2b0 0x7f1a6006e770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1a78007fd0 tx=0x7f1a78009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:19.957+0000 7f1a777fe640 1 -- 172.21.15.16:0/1127916440 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1a700592a0 con 0x7f1a800d2750 2024-09-19T07:45:20.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.133+0000 7f1a8574b640 1 -- 172.21.15.16:0/1127916440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=client_inject_fixed_oldest_tid}] v 0) -- 0x7f1a800b2430 con 0x7f1a800d2750 2024-09-19T07:45:20.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.136+0000 7f1a777fe640 1 -- 172.21.15.16:0/1127916440 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_inject_fixed_oldest_tid}]=0 v17)=0 v17) ==== 134+0+0 (secure 0 0 0) 0x7f1a7005d5a0 con 0x7f1a800d2750 2024-09-19T07:45:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.137+0000 7f1a777fe640 1 -- 172.21.15.16:0/1127916440 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(1 keys) ==== 46+0+0 (secure 0 0 0) 0x7f1a70061dd0 con 0x7f1a800d2750 2024-09-19T07:45:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.138+0000 7f1a8574b640 1 -- 172.21.15.16:0/1127916440 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1a6006c2b0 msgr2=0x7f1a6006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.138+0000 7f1a8574b640 1 --2- 172.21.15.16:0/1127916440 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1a6006c2b0 0x7f1a6006e770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1a78007fd0 tx=0x7f1a78009040 comp rx=0 tx=0).stop 2024-09-19T07:45:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.138+0000 7f1a8574b640 1 -- 172.21.15.16:0/1127916440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a800d2750 msgr2=0x7f1a800b0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.138+0000 7f1a8574b640 1 --2- 172.21.15.16:0/1127916440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a800d2750 0x7f1a800b0e10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000cae0 tx=0x7f1a7000a060 comp rx=0 tx=0).stop 2024-09-19T07:45:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.138+0000 7f1a8574b640 1 -- 172.21.15.16:0/1127916440 shutdown_connections 2024-09-19T07:45:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.138+0000 7f1a8574b640 1 --2- 172.21.15.16:0/1127916440 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1a6006c2b0 0x7f1a6006e770 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.138+0000 7f1a8574b640 1 --2- 172.21.15.16:0/1127916440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a800d2750 0x7f1a800b0e10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.138+0000 7f1a8574b640 1 --2- 172.21.15.16:0/1127916440 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a80072690 0x7f1a800cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.138+0000 7f1a8574b640 1 --2- 172.21.15.16:0/1127916440 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a80072250 0x7f1a8006cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.138+0000 7f1a8574b640 1 -- 172.21.15.16:0/1127916440 >> 172.21.15.16:0/1127916440 conn(0x7f1a800b4040 msgr2=0x7f1a800cc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.138+0000 7f1a8574b640 1 -- 172.21.15.16:0/1127916440 shutdown_connections 2024-09-19T07:45:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.138+0000 7f1a8574b640 1 -- 172.21.15.16:0/1127916440 wait complete. 2024-09-19T07:45:20.150 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:45:20.150 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:45:20.150 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:45:20.167 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:45:20.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.310+0000 7f157f598640 1 Processor -- start 2024-09-19T07:45:20.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.311+0000 7f157f598640 1 -- start start 2024-09-19T07:45:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.311+0000 7f157f598640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f157815f0d0 0x7f157815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.311+0000 7f157f598640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f15780d4130 0x7f15780d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.311+0000 7f157f598640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15780d4b20 con 0x7f15780d0070 2024-09-19T07:45:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.311+0000 7f157f598640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f15780d1b10 con 0x7f15780d4130 2024-09-19T07:45:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.311+0000 7f157f598640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f15780cf2b0 con 0x7f157815f0d0 2024-09-19T07:45:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.311+0000 7f157db35640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f15780d4130 0x7f15780d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.312+0000 7f157db35640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f15780d4130 0x7f15780d45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47114/0 (socket says 172.21.15.16:47114) 2024-09-19T07:45:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.312+0000 7f157db35640 1 -- 172.21.15.16:0/1111360286 learned_addr learned my addr 172.21.15.16:0/1111360286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.312+0000 7f157d334640 1 --2- 172.21.15.16:0/1111360286 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f157815f0d0 0x7f157815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.312+0000 7f157db35640 1 -- 172.21.15.16:0/1111360286 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f157815f0d0 msgr2=0x7f157815f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.312+0000 7f157db35640 1 --2- 172.21.15.16:0/1111360286 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f157815f0d0 0x7f157815f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.312+0000 7f157db35640 1 -- 172.21.15.16:0/1111360286 >> v1:172.21.15.146:6789/0 conn(0x7f15780d0070 legacy=0x7f15780d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.312+0000 7f157db35640 1 -- 172.21.15.16:0/1111360286 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15780cf590 con 0x7f15780d4130 2024-09-19T07:45:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.313+0000 7f157db35640 1 --2- 172.21.15.16:0/1111360286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15780d4130 0x7f15780d45e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f15780d2fc0 tx=0x7f156802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec5e06467091b6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.313+0000 7f1577fff640 1 -- 172.21.15.16:0/1111360286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1568036070 con 0x7f15780d4130 2024-09-19T07:45:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.313+0000 7f1577fff640 1 -- 172.21.15.16:0/1111360286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 46+0+0 (secure 0 0 0) 0x7f1568031600 con 0x7f15780d4130 2024-09-19T07:45:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.313+0000 7f1577fff640 1 -- 172.21.15.16:0/1111360286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15680319d0 con 0x7f15780d4130 2024-09-19T07:45:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.313+0000 7f157f598640 1 -- 172.21.15.16:0/1111360286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15780d4130 msgr2=0x7f15780d45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.313+0000 7f157f598640 1 --2- 172.21.15.16:0/1111360286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15780d4130 0x7f15780d45e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f15780d2fc0 tx=0x7f156802a680 comp rx=0 tx=0).stop 2024-09-19T07:45:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.313+0000 7f157f598640 1 -- 172.21.15.16:0/1111360286 shutdown_connections 2024-09-19T07:45:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.313+0000 7f157f598640 1 --2- 172.21.15.16:0/1111360286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15780d4130 0x7f15780d45e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.313+0000 7f157f598640 1 --2- 172.21.15.16:0/1111360286 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f157815f0d0 0x7f157815f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.313+0000 7f157f598640 1 -- 172.21.15.16:0/1111360286 >> 172.21.15.16:0/1111360286 conn(0x7f157809d760 msgr2=0x7f157808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.313+0000 7f157f598640 1 -- 172.21.15.16:0/1111360286 shutdown_connections 2024-09-19T07:45:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.313+0000 7f157f598640 1 -- 172.21.15.16:0/1111360286 wait complete. 2024-09-19T07:45:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.314+0000 7f157f598640 1 Processor -- start 2024-09-19T07:45:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.315+0000 7f157f598640 1 -- start start 2024-09-19T07:45:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.315+0000 7f157f598640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15780d0070 0x7f15781e84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.315+0000 7f157f598640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15780d4130 0x7f15780b7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.315+0000 7f157f598640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f157815f0d0 0x7f157814af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.315+0000 7f157f598640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15780cef60 con 0x7f15780d4130 2024-09-19T07:45:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.315+0000 7f157f598640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f15780cfb50 con 0x7f157815f0d0 2024-09-19T07:45:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.315+0000 7f157f598640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f15780b2f70 con 0x7f15780d0070 2024-09-19T07:45:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.315+0000 7f157cb33640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15780d4130 0x7f15780b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.315+0000 7f157cb33640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15780d4130 0x7f15780b7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47118/0 (socket says 172.21.15.16:47118) 2024-09-19T07:45:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.315+0000 7f157cb33640 1 -- 172.21.15.16:0/905682017 learned_addr learned my addr 172.21.15.16:0/905682017 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.315+0000 7f157d334640 1 --2- 172.21.15.16:0/905682017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15780d0070 0x7f15781e84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.315+0000 7f157db35640 1 --2- 172.21.15.16:0/905682017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f157815f0d0 0x7f157814af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.316+0000 7f157cb33640 1 -- 172.21.15.16:0/905682017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15780d0070 msgr2=0x7f15781e84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.316+0000 7f157cb33640 1 --2- 172.21.15.16:0/905682017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15780d0070 0x7f15781e84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.316+0000 7f157cb33640 1 -- 172.21.15.16:0/905682017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f157815f0d0 msgr2=0x7f157814af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.316+0000 7f157cb33640 1 --2- 172.21.15.16:0/905682017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f157815f0d0 0x7f157814af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.316+0000 7f157cb33640 1 -- 172.21.15.16:0/905682017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157814b4d0 con 0x7f15780d4130 2024-09-19T07:45:20.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.316+0000 7f157cb33640 1 --2- 172.21.15.16:0/905682017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15780d4130 0x7f15780b7700 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f156c00c470 tx=0x7f156c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.316+0000 7f1575ffb640 1 -- 172.21.15.16:0/905682017 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c0052d0 con 0x7f15780d4130 2024-09-19T07:45:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.316+0000 7f1575ffb640 1 -- 172.21.15.16:0/905682017 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 46+0+0 (secure 0 0 0) 0x7f156c005470 con 0x7f15780d4130 2024-09-19T07:45:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.316+0000 7f157f598640 1 -- 172.21.15.16:0/905682017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1578148670 con 0x7f15780d4130 2024-09-19T07:45:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.316+0000 7f1575ffb640 1 -- 172.21.15.16:0/905682017 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c007260 con 0x7f15780d4130 2024-09-19T07:45:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.316+0000 7f157f598640 1 -- 172.21.15.16:0/905682017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1578148c30 con 0x7f15780d4130 2024-09-19T07:45:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.317+0000 7f157f598640 1 -- 172.21.15.16:0/905682017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15780d45e0 con 0x7f15780d4130 2024-09-19T07:45:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.318+0000 7f1575ffb640 1 -- 172.21.15.16:0/905682017 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f156c007400 con 0x7f15780d4130 2024-09-19T07:45:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.318+0000 7f1575ffb640 1 --2- 172.21.15.16:0/905682017 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f156406c2b0 0x7f156406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.319+0000 7f1575ffb640 1 -- 172.21.15.16:0/905682017 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f156c08aa60 con 0x7f15780d4130 2024-09-19T07:45:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.319+0000 7f157d334640 1 --2- 172.21.15.16:0/905682017 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f156406c2b0 0x7f156406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.319+0000 7f157d334640 1 --2- 172.21.15.16:0/905682017 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f156406c2b0 0x7f156406e770 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f15780a2230 tx=0x7f1570008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:20.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.323+0000 7f1575ffb640 1 -- 172.21.15.16:0/905682017 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f15780d4b20 con 0x7f15780d4130 2024-09-19T07:45:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.509+0000 7f157f598640 1 -- 172.21.15.16:0/905682017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f15780c7cb0 con 0x7f15780d4130 2024-09-19T07:45:20.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.509+0000 7f1575ffb640 1 -- 172.21.15.16:0/905682017 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) ==== 80+0+0 (secure 0 0 0) 0x7f156c0589f0 con 0x7f15780d4130 2024-09-19T07:45:20.510 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:45:20.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.512+0000 7f157f598640 1 -- 172.21.15.16:0/905682017 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f156406c2b0 msgr2=0x7f156406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:20.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.512+0000 7f157f598640 1 --2- 172.21.15.16:0/905682017 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f156406c2b0 0x7f156406e770 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f15780a2230 tx=0x7f1570008040 comp rx=0 tx=0).stop 2024-09-19T07:45:20.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.512+0000 7f157f598640 1 -- 172.21.15.16:0/905682017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15780d4130 msgr2=0x7f15780b7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:20.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.512+0000 7f157f598640 1 --2- 172.21.15.16:0/905682017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15780d4130 0x7f15780b7700 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f156c00c470 tx=0x7f156c001e70 comp rx=0 tx=0).stop 2024-09-19T07:45:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.512+0000 7f157f598640 1 -- 172.21.15.16:0/905682017 shutdown_connections 2024-09-19T07:45:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.512+0000 7f157f598640 1 --2- 172.21.15.16:0/905682017 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f156406c2b0 0x7f156406e770 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.512+0000 7f157f598640 1 --2- 172.21.15.16:0/905682017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f157815f0d0 0x7f157814af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.512+0000 7f157f598640 1 --2- 172.21.15.16:0/905682017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15780d4130 0x7f15780b7700 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.512+0000 7f157f598640 1 --2- 172.21.15.16:0/905682017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15780d0070 0x7f15781e84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.512+0000 7f157f598640 1 -- 172.21.15.16:0/905682017 >> 172.21.15.16:0/905682017 conn(0x7f157809d760 msgr2=0x7f15780b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.512+0000 7f157f598640 1 -- 172.21.15.16:0/905682017 shutdown_connections 2024-09-19T07:45:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.512+0000 7f157f598640 1 -- 172.21.15.16:0/905682017 wait complete. 2024-09-19T07:45:20.523 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-19T07:45:20.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:20.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T07:45:20.547 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[51210]: fuse finished with error 0 and tester_r 0 2024-09-19T07:45:20.549 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:45:20.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:20.549 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:45:20.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:45:20.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.856+0000 7f4f33853640 1 Processor -- start 2024-09-19T07:45:20.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.857+0000 7f4f33853640 1 -- start start 2024-09-19T07:45:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.857+0000 7f4f33853640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4f2c15f0d0 0x7f4f2c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.857+0000 7f4f33853640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f2c0d0070 0x7f4f2c0d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.857+0000 7f4f33853640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f2c0cf2b0 con 0x7f4f2c0d1940 2024-09-19T07:45:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.857+0000 7f4f33853640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4f2c0cf590 con 0x7f4f2c15f0d0 2024-09-19T07:45:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.857+0000 7f4f33853640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4f2c0cf870 con 0x7f4f2c0d0070 2024-09-19T07:45:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.858+0000 7f4f30dee640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f2c0d0070 0x7f4f2c0d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.858+0000 7f4f31df0640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f4f2c0d1940 0x7f4f2c0d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:54212/0 (socket says 172.21.15.16:54212) 2024-09-19T07:45:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.858+0000 7f4f31df0640 1 -- 172.21.15.16:0/87830791 learned_addr learned my addr 172.21.15.16:0/87830791 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.858+0000 7f4f315ef640 1 --2- 172.21.15.16:0/87830791 >> v2:172.21.15.146:3300/0 conn(0x7f4f2c15f0d0 0x7f4f2c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.858+0000 7f4f23fff640 1 -- 172.21.15.16:0/87830791 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2616375942 0 0) 0x7f4f2c0cf2b0 con 0x7f4f2c0d1940 2024-09-19T07:45:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.858+0000 7f4f23fff640 1 -- 172.21.15.16:0/87830791 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f18003520 con 0x7f4f2c0d1940 2024-09-19T07:45:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.858+0000 7f4f315ef640 1 -- 172.21.15.16:0/87830791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f2c0d0070 msgr2=0x7f4f2c0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.858+0000 7f4f315ef640 1 --2- 172.21.15.16:0/87830791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f2c0d0070 0x7f4f2c0d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.858+0000 7f4f315ef640 1 -- 172.21.15.16:0/87830791 >> v1:172.21.15.146:6789/0 conn(0x7f4f2c0d1940 legacy=0x7f4f2c0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.858+0000 7f4f315ef640 1 -- 172.21.15.16:0/87830791 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f2c0cfb50 con 0x7f4f2c15f0d0 2024-09-19T07:45:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.858+0000 7f4f30dee640 1 --2- 172.21.15.16:0/87830791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f2c0d0070 0x7f4f2c0d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.859+0000 7f4f315ef640 1 --2- 172.21.15.16:0/87830791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f2c15f0d0 0x7f4f2c15f4b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4f280097a0 tx=0x7f4f2802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=853f3d4360a35296 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.859+0000 7f4f23fff640 1 -- 172.21.15.16:0/87830791 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28037070 con 0x7f4f2c15f0d0 2024-09-19T07:45:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.859+0000 7f4f23fff640 1 -- 172.21.15.16:0/87830791 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(1 keys) ==== 46+0+0 (secure 0 0 0) 0x7f4f2802ae50 con 0x7f4f2c15f0d0 2024-09-19T07:45:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.859+0000 7f4f23fff640 1 -- 172.21.15.16:0/87830791 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28032660 con 0x7f4f2c15f0d0 2024-09-19T07:45:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.860+0000 7f4f33853640 1 -- 172.21.15.16:0/87830791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f2c15f0d0 msgr2=0x7f4f2c15f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.860+0000 7f4f33853640 1 --2- 172.21.15.16:0/87830791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f2c15f0d0 0x7f4f2c15f4b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4f280097a0 tx=0x7f4f2802a100 comp rx=0 tx=0).stop 2024-09-19T07:45:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.860+0000 7f4f33853640 1 -- 172.21.15.16:0/87830791 shutdown_connections 2024-09-19T07:45:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.860+0000 7f4f33853640 1 --2- 172.21.15.16:0/87830791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f2c0d0070 0x7f4f2c0d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.860+0000 7f4f33853640 1 --2- 172.21.15.16:0/87830791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f2c15f0d0 0x7f4f2c15f4b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.860+0000 7f4f33853640 1 -- 172.21.15.16:0/87830791 >> 172.21.15.16:0/87830791 conn(0x7f4f2c09d760 msgr2=0x7f4f2c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.860+0000 7f4f33853640 1 -- 172.21.15.16:0/87830791 shutdown_connections 2024-09-19T07:45:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.860+0000 7f4f33853640 1 -- 172.21.15.16:0/87830791 wait complete. 2024-09-19T07:45:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.861+0000 7f4f33853640 1 Processor -- start 2024-09-19T07:45:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.861+0000 7f4f33853640 1 -- start start 2024-09-19T07:45:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.862+0000 7f4f33853640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f2c0d0070 0x7f4f2c15b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.862+0000 7f4f33853640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f2c0d1940 0x7f4f2c0b9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.862+0000 7f4f33853640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f2c15f0d0 0x7f4f2c0ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.862+0000 7f4f33853640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c0d49c0 con 0x7f4f2c0d1940 2024-09-19T07:45:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.862+0000 7f4f33853640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c0d4840 con 0x7f4f2c0d0070 2024-09-19T07:45:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.862+0000 7f4f33853640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4f2c0d4b40 con 0x7f4f2c15f0d0 2024-09-19T07:45:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.862+0000 7f4f30dee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f2c0d1940 0x7f4f2c0b9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.862+0000 7f4f30dee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f2c0d1940 0x7f4f2c0b9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47120/0 (socket says 172.21.15.16:47120) 2024-09-19T07:45:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.862+0000 7f4f30dee640 1 -- 172.21.15.16:0/1632775863 learned_addr learned my addr 172.21.15.16:0/1632775863 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.862+0000 7f4f315ef640 1 --2- 172.21.15.16:0/1632775863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f2c0d0070 0x7f4f2c15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.862+0000 7f4f31df0640 1 --2- 172.21.15.16:0/1632775863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f2c15f0d0 0x7f4f2c0ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.862+0000 7f4f30dee640 1 -- 172.21.15.16:0/1632775863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f2c15f0d0 msgr2=0x7f4f2c0ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.862+0000 7f4f30dee640 1 --2- 172.21.15.16:0/1632775863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f2c15f0d0 0x7f4f2c0ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.862+0000 7f4f30dee640 1 -- 172.21.15.16:0/1632775863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f2c0d0070 msgr2=0x7f4f2c15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.862+0000 7f4f30dee640 1 --2- 172.21.15.16:0/1632775863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f2c0d0070 0x7f4f2c15b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.862+0000 7f4f30dee640 1 -- 172.21.15.16:0/1632775863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f2c0cba70 con 0x7f4f2c0d1940 2024-09-19T07:45:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.862+0000 7f4f30dee640 1 --2- 172.21.15.16:0/1632775863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f2c0d1940 0x7f4f2c0b9c30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00ec10 tx=0x7f4f1c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.863+0000 7f4f21ffb640 1 -- 172.21.15.16:0/1632775863 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c017070 con 0x7f4f2c0d1940 2024-09-19T07:45:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.863+0000 7f4f21ffb640 1 -- 172.21.15.16:0/1632775863 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 46+0+0 (secure 0 0 0) 0x7f4f1c00cc30 con 0x7f4f2c0d1940 2024-09-19T07:45:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.863+0000 7f4f33853640 1 -- 172.21.15.16:0/1632775863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f2c0cbd50 con 0x7f4f2c0d1940 2024-09-19T07:45:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.863+0000 7f4f21ffb640 1 -- 172.21.15.16:0/1632775863 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c013470 con 0x7f4f2c0d1940 2024-09-19T07:45:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.863+0000 7f4f33853640 1 -- 172.21.15.16:0/1632775863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f2c0cc310 con 0x7f4f2c0d1940 2024-09-19T07:45:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.864+0000 7f4f33853640 1 -- 172.21.15.16:0/1632775863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f2c068ce0 con 0x7f4f2c0d1940 2024-09-19T07:45:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.864+0000 7f4f21ffb640 1 -- 172.21.15.16:0/1632775863 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4f1c01a070 con 0x7f4f2c0d1940 2024-09-19T07:45:20.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.865+0000 7f4f21ffb640 1 --2- 172.21.15.16:0/1632775863 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4f1006c2e0 0x7f4f1006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:20.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.865+0000 7f4f21ffb640 1 -- 172.21.15.16:0/1632775863 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f4f1c08b920 con 0x7f4f2c0d1940 2024-09-19T07:45:20.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.865+0000 7f4f315ef640 1 --2- 172.21.15.16:0/1632775863 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4f1006c2e0 0x7f4f1006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:20.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.866+0000 7f4f315ef640 1 --2- 172.21.15.16:0/1632775863 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4f1006c2e0 0x7f4f1006e7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4f280097a0 tx=0x7f4f2802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:20.869+0000 7f4f21ffb640 1 -- 172.21.15.16:0/1632775863 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4f2c068ce0 con 0x7f4f2c0d1940 2024-09-19T07:45:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:21.084+0000 7f4f33853640 1 -- 172.21.15.16:0/1632775863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4f2c0cb270 con 0x7f4f2c0d1940 2024-09-19T07:45:21.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:21.084+0000 7f4f21ffb640 1 -- 172.21.15.16:0/1632775863 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f4f2c0cb270 con 0x7f4f2c0d1940 2024-09-19T07:45:21.085 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:45:21.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:21.086+0000 7f4f33853640 1 -- 172.21.15.16:0/1632775863 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4f1006c2e0 msgr2=0x7f4f1006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:21.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:21.086+0000 7f4f33853640 1 --2- 172.21.15.16:0/1632775863 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4f1006c2e0 0x7f4f1006e7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4f280097a0 tx=0x7f4f2802e000 comp rx=0 tx=0).stop 2024-09-19T07:45:21.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:21.086+0000 7f4f33853640 1 -- 172.21.15.16:0/1632775863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f2c0d1940 msgr2=0x7f4f2c0b9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:21.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:21.086+0000 7f4f33853640 1 --2- 172.21.15.16:0/1632775863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f2c0d1940 0x7f4f2c0b9c30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00ec10 tx=0x7f4f1c00c520 comp rx=0 tx=0).stop 2024-09-19T07:45:21.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:21.086+0000 7f4f33853640 1 -- 172.21.15.16:0/1632775863 shutdown_connections 2024-09-19T07:45:21.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:21.086+0000 7f4f33853640 1 --2- 172.21.15.16:0/1632775863 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4f1006c2e0 0x7f4f1006e7a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:21.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:21.086+0000 7f4f33853640 1 --2- 172.21.15.16:0/1632775863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f2c15f0d0 0x7f4f2c0ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:21.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:21.086+0000 7f4f33853640 1 --2- 172.21.15.16:0/1632775863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f2c0d1940 0x7f4f2c0b9c30 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:21.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:21.086+0000 7f4f33853640 1 --2- 172.21.15.16:0/1632775863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f2c0d0070 0x7f4f2c15b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:21.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:21.086+0000 7f4f33853640 1 -- 172.21.15.16:0/1632775863 >> 172.21.15.16:0/1632775863 conn(0x7f4f2c09d760 msgr2=0x7f4f2c0d1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:21.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:21.086+0000 7f4f33853640 1 -- 172.21.15.16:0/1632775863 shutdown_connections 2024-09-19T07:45:21.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:21.086+0000 7f4f33853640 1 -- 172.21.15.16:0/1632775863 wait complete. 2024-09-19T07:45:21.098 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:45:21.098 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:45:21.098 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T07:45:21.098 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T07:45:21.098 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:45:21.098 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:45:21.098 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:45:21.098 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:45:21.098 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:45:21.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:21.099 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T07:45:21.115 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:45:21.115 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:45:21.115 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:45:21.115 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:21.116 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:45:21.116 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:21.116 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:45:21.116 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:45:21.116 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T07:45:21.116 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:45:21.116 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:45:21.116 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T07:45:21.116 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:45:21.116 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:45:21.116 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T07:45:21.117 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T07:45:21.117 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42668sec preferred_lft 42668sec 2024-09-19T07:45:21.117 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T07:45:21.117 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:21.117 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:45:21.117 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:45:21.117 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T07:45:21.117 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:45:21.117 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:45:21.117 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:45:21.117 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:21.117 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::32:7bff:fe92:8895/64 scope link 2024-09-19T07:45:21.117 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:21.118 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:45:21.118 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether e6:fd:21:6a:43:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:45:21.118 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e4fd:21ff:fe6a:4369/64 scope link 2024-09-19T07:45:21.118 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:21.118 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:45:21.118 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:45:21.118 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::80f4:81ff:fe7d:b7f0/64 scope link 2024-09-19T07:45:21.118 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:45:21.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:21.118 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:45:21.180 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:45:21.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:21.181 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T07:45:21.239 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T07:45:21.239 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:45:21.239 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T07:45:21.239 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T07:45:21.240 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:45:21.240 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:45:21.240 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:45:21.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T07:45:21.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T07:45:21.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T07:45:21.240 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:45:21.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:45:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:45:21.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:21.369 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:45:21.385 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T07:45:21.385 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:45:21.385 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:45:21.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:21.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-19T07:45:21.451 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T07:45:21.451 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:45:21.451 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T07:45:21.507 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:45:21.507 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:45:21.507 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:45:21.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:45:21.629 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:45:21.686 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:45:21.686 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:45:21.686 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:45:21.686 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:45:21.686 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:45:21.687 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:45:21.687 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:45:21.687 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:45:21.687 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:45:21.688 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:45:21.688 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:45:21.688 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:45:21.688 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:45:21.688 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:45:21.688 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:45:21.688 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:45:21.688 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:45:21.688 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:45:21.689 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:45:21.689 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:45:21.689 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:45:21.689 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:45:21.689 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:45:21.689 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:45:21.689 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:45:21.689 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:45:21.689 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:45:21.689 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:45:21.689 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:45:21.689 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:45:21.690 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:45:21.690 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:45:21.690 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:45:21.690 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:45:21.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:21.690 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:45:21.742 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:45:21.742 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T07:45:21.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:45:21.810 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:45:21.867 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:45:21.867 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:45:21.867 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:45:21.867 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:45:21.867 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:45:21.867 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:45:21.867 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:45:21.867 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:45:21.868 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:45:21.868 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:45:21.868 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:45:21.868 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:45:21.868 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:45:21.868 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:45:21.868 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:45:21.868 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:45:21.868 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:45:21.868 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:45:21.868 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:45:21.868 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:45:21.869 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:45:21.869 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:45:21.869 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:45:21.869 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:45:21.869 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:45:21.869 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:45:21.869 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:45:21.869 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:45:21.869 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:45:21.869 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:45:21.869 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:45:21.869 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:45:21.870 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:45:21.870 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:45:21.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:21.870 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:45:21.871 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-19T07:45:21.869+0000 7f006c71d480 -1 init, newargv = 0x55cbbf0ba430 newargc=15 2024-09-19T07:45:21.871 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[51804]: starting ceph client 2024-09-19T07:45:21.873 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[51804]: starting fuse 2024-09-19T07:45:21.887 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:45:21.887 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-19T07:45:21.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:45:21.888 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:45:21.888 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:45:21.888 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:45:21.888 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:45:21.888 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:45:21.888 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:45:21.888 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:45:21.888 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:45:21.888 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:45:21.888 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:45:21.888 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:45:21.888 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:45:21.889 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:45:21.889 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:45:21.889 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:45:21.889 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:45:21.889 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:45:21.889 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:45:21.889 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:45:21.889 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:45:21.889 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:45:21.889 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:45:21.889 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:45:21.889 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:45:21.889 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:45:21.889 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:45:22.007 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.51804.asok 2024-09-19T07:45:22.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:45:22.017 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51804.asok 2024-09-19T07:45:22.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:22.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51804.asok status 2024-09-19T07:45:22.154 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:45:22.154 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:45:22.155 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:45:22.155 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:45:22.155 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:45:22.155 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:45:22.155 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:45:22.155 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "51804", 2024-09-19T07:45:22.155 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:45:22.155 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:22.155 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-19T07:45:22.155 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-19T07:45:22.155 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5514, 2024-09-19T07:45:22.155 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:45:22.156 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:45:22.156 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:45:22.156 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5514 2024-09-19T07:45:22.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:22.156 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:45:22.156 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:45:22.156 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:45:22.156 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2345385426 2024-09-19T07:45:22.156 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:45:22.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:22.156 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:45:22.156 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:45:22.156 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:45:22.157 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2345385426 2024-09-19T07:45:22.157 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:45:22.157 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.5514 192.168.0.1:0/2345385426", 2024-09-19T07:45:22.157 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/2345385426", 2024-09-19T07:45:22.157 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-19T07:45:22.157 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 24, 2024-09-19T07:45:22.157 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 53, 2024-09-19T07:45:22.157 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:45:22.157 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:45:22.157 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:45:22.157 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:45:22.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:22.163 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-19T07:45:22.163 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T07:45:22.179 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-19T07:45:22.179 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T07:45:22.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:45:22.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:45:22.245 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_completed_requests 10000 2024-09-19T07:45:22.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.392+0000 7f597a9fd640 1 Processor -- start 2024-09-19T07:45:22.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.392+0000 7f597a9fd640 1 -- start start 2024-09-19T07:45:22.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.392+0000 7f597a9fd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f59740d0070 0x7f597415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.392+0000 7f597a9fd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59740d1b10 con 0x7f59740d4130 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.392+0000 7f597a9fd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59740cf2b0 con 0x7f597415f0d0 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.392+0000 7f597a9fd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f59740cf590 con 0x7f59740d0070 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.393+0000 7f5978f9a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f59740d4130 0x7f59740d4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50390/0 (socket says 172.21.15.16:50390) 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.393+0000 7f5978f9a640 1 -- 172.21.15.16:0/852816635 learned_addr learned my addr 172.21.15.16:0/852816635 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.393+0000 7f596bfff640 1 --2- 172.21.15.16:0/852816635 >> v2:172.21.15.146:3300/0 conn(0x7f59740d0070 0x7f597415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.393+0000 7f59737fe640 1 -- 172.21.15.16:0/852816635 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4102410211 0 0) 0x7f59740d1b10 con 0x7f59740d4130 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.393+0000 7f59737fe640 1 -- 172.21.15.16:0/852816635 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5960003520 con 0x7f59740d4130 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.393+0000 7f59737fe640 1 -- 172.21.15.16:0/852816635 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1146306258 0 0) 0x7f5960003520 con 0x7f59740d4130 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.393+0000 7f59737fe640 1 -- 172.21.15.16:0/852816635 >> v2:172.21.15.146:3300/0 conn(0x7f59740d0070 msgr2=0x7f597415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.393+0000 7f59737fe640 1 --2- 172.21.15.16:0/852816635 >> v2:172.21.15.146:3300/0 conn(0x7f59740d0070 0x7f597415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.393+0000 7f59737fe640 1 -- 172.21.15.16:0/852816635 >> v1:172.21.15.146:6789/0 conn(0x7f597415f0d0 legacy=0x7f59740d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.393+0000 7f59737fe640 1 -- 172.21.15.16:0/852816635 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59740cf870 con 0x7f59740d4130 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.393+0000 7f59737fe640 1 -- 172.21.15.16:0/852816635 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f596c001a10 con 0x7f59740d4130 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.393+0000 7f59737fe640 1 -- 172.21.15.16:0/852816635 >> v1:172.21.15.16:6789/0 conn(0x7f59740d4130 legacy=0x7f59740d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.393+0000 7f59737fe640 1 --2- 172.21.15.16:0/852816635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59600039b0 0x7f5960003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.393+0000 7f59737fe640 1 --2- 172.21.15.16:0/852816635 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5960004800 0x7f5960004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.393+0000 7f59737fe640 1 --2- 172.21.15.16:0/852816635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59600052e0 0x7f596000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.393+0000 7f59737fe640 1 -- 172.21.15.16:0/852816635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f59740cf590 con 0x7f59600039b0 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.393+0000 7f59737fe640 1 -- 172.21.15.16:0/852816635 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f596000c1b0 con 0x7f59600052e0 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.393+0000 7f59737fe640 1 -- 172.21.15.16:0/852816635 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f596000c3b0 con 0x7f5960004800 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.393+0000 7f5973fff640 1 --2- 172.21.15.16:0/852816635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59600039b0 0x7f5960003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.394+0000 7f596bfff640 1 --2- 172.21.15.16:0/852816635 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5960004800 0x7f5960004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.394+0000 7f5978f9a640 1 --2- 172.21.15.16:0/852816635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59600052e0 0x7f596000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.394+0000 7f5973fff640 1 -- 172.21.15.16:0/852816635 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5960004800 msgr2=0x7f5960004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.394+0000 7f5973fff640 1 --2- 172.21.15.16:0/852816635 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5960004800 0x7f5960004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.394+0000 7f5973fff640 1 -- 172.21.15.16:0/852816635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59600052e0 msgr2=0x7f596000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.394+0000 7f5973fff640 1 --2- 172.21.15.16:0/852816635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59600052e0 0x7f596000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.394+0000 7f5978f9a640 1 -- 172.21.15.16:0/852816635 reap_dead start 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.394+0000 7f5973fff640 1 -- 172.21.15.16:0/852816635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5960003520 con 0x7f59600039b0 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.394+0000 7f5973fff640 1 --2- 172.21.15.16:0/852816635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59600039b0 0x7f5960003dd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5964001720 tx=0x7f5960005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9e332ecdef4d6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.395+0000 7f59737fe640 1 -- 172.21.15.16:0/852816635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964001370 con 0x7f59600039b0 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.395+0000 7f59737fe640 1 -- 172.21.15.16:0/852816635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 46+0+0 (secure 0 0 0) 0x7f59640052d0 con 0x7f59600039b0 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.395+0000 7f59737fe640 1 -- 172.21.15.16:0/852816635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596402ab90 con 0x7f59600039b0 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.395+0000 7f597a9fd640 1 -- 172.21.15.16:0/852816635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59600039b0 msgr2=0x7f5960003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.395+0000 7f597a9fd640 1 --2- 172.21.15.16:0/852816635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59600039b0 0x7f5960003dd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5964001720 tx=0x7f5960005830 comp rx=0 tx=0).stop 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.395+0000 7f597a9fd640 1 -- 172.21.15.16:0/852816635 shutdown_connections 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.395+0000 7f597a9fd640 1 --2- 172.21.15.16:0/852816635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59600039b0 0x7f5960003dd0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.395+0000 7f597a9fd640 1 -- 172.21.15.16:0/852816635 >> 172.21.15.16:0/852816635 conn(0x7f597409d760 msgr2=0x7f597408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.395+0000 7f597a9fd640 1 -- 172.21.15.16:0/852816635 shutdown_connections 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.395+0000 7f597a9fd640 1 -- 172.21.15.16:0/852816635 wait complete. 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.396+0000 7f597a9fd640 1 Processor -- start 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.396+0000 7f597a9fd640 1 -- start start 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.397+0000 7f597a9fd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59600039b0 0x7f59741f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.397+0000 7f597a9fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59741f4a30 0x7f59741f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.397+0000 7f597a9fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59740cd220 0x7f597415b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.397+0000 7f597a9fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f597415f4b0 con 0x7f59741f4a30 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.397+0000 7f597a9fd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f59740cef60 con 0x7f59600039b0 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.397+0000 7f597a9fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f597415f630 con 0x7f59740cd220 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.397+0000 7f596bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59741f4a30 0x7f59741f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.397+0000 7f596bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59741f4a30 0x7f59741f7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47138/0 (socket says 172.21.15.16:47138) 2024-09-19T07:45:22.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.397+0000 7f596bfff640 1 -- 172.21.15.16:0/1088460099 learned_addr learned my addr 172.21.15.16:0/1088460099 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:22.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.398+0000 7f596bfff640 1 -- 172.21.15.16:0/1088460099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59740cd220 msgr2=0x7f597415b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:22.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.398+0000 7f596bfff640 1 --2- 172.21.15.16:0/1088460099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59740cd220 0x7f597415b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.398+0000 7f596bfff640 1 -- 172.21.15.16:0/1088460099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59600039b0 msgr2=0x7f59741f5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:22.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.398+0000 7f596bfff640 1 --2- 172.21.15.16:0/1088460099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59600039b0 0x7f59741f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.398+0000 7f596bfff640 1 -- 172.21.15.16:0/1088460099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59740cdb20 con 0x7f59741f4a30 2024-09-19T07:45:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.398+0000 7f596bfff640 1 --2- 172.21.15.16:0/1088460099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59741f4a30 0x7f59741f7840 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f595c011960 tx=0x7f595c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.398+0000 7f59717fa640 1 -- 172.21.15.16:0/1088460099 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c00fa10 con 0x7f59741f4a30 2024-09-19T07:45:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.398+0000 7f59717fa640 1 -- 172.21.15.16:0/1088460099 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 46+0+0 (secure 0 0 0) 0x7f595c00fbb0 con 0x7f59741f4a30 2024-09-19T07:45:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.398+0000 7f597a9fd640 1 -- 172.21.15.16:0/1088460099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59740cde00 con 0x7f59741f4a30 2024-09-19T07:45:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.398+0000 7f59717fa640 1 -- 172.21.15.16:0/1088460099 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c014440 con 0x7f59741f4a30 2024-09-19T07:45:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.398+0000 7f597a9fd640 1 -- 172.21.15.16:0/1088460099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59740cb700 con 0x7f59741f4a30 2024-09-19T07:45:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.399+0000 7f597a9fd640 1 -- 172.21.15.16:0/1088460099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59740d4560 con 0x7f59741f4a30 2024-09-19T07:45:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.399+0000 7f59717fa640 1 -- 172.21.15.16:0/1088460099 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f595c012040 con 0x7f59741f4a30 2024-09-19T07:45:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.400+0000 7f59717fa640 1 --2- 172.21.15.16:0/1088460099 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f595406c2b0 0x7f595406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.400+0000 7f59717fa640 1 -- 172.21.15.16:0/1088460099 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f595c08bb00 con 0x7f59741f4a30 2024-09-19T07:45:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.400+0000 7f5973fff640 1 --2- 172.21.15.16:0/1088460099 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f595406c2b0 0x7f595406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.401+0000 7f5973fff640 1 --2- 172.21.15.16:0/1088460099 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f595406c2b0 0x7f595406e770 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5964001720 tx=0x7f596402c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:22.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.404+0000 7f59717fa640 1 -- 172.21.15.16:0/1088460099 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f595c019030 con 0x7f59741f4a30 2024-09-19T07:45:22.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.580+0000 7f597a9fd640 1 -- 172.21.15.16:0/1088460099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_max_completed_requests}] v 0) -- 0x7f59740ce320 con 0x7f59741f4a30 2024-09-19T07:45:22.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.583+0000 7f59717fa640 1 -- 172.21.15.16:0/1088460099 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_completed_requests}]=0 v18)=0 v18) ==== 128+0+0 (secure 0 0 0) 0x7f595c059a90 con 0x7f59741f4a30 2024-09-19T07:45:22.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.586+0000 7f597a9fd640 1 -- 172.21.15.16:0/1088460099 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f595406c2b0 msgr2=0x7f595406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:22.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.586+0000 7f597a9fd640 1 --2- 172.21.15.16:0/1088460099 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f595406c2b0 0x7f595406e770 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5964001720 tx=0x7f596402c000 comp rx=0 tx=0).stop 2024-09-19T07:45:22.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.586+0000 7f597a9fd640 1 -- 172.21.15.16:0/1088460099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59741f4a30 msgr2=0x7f59741f7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:22.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.586+0000 7f597a9fd640 1 --2- 172.21.15.16:0/1088460099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59741f4a30 0x7f59741f7840 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f595c011960 tx=0x7f595c011e20 comp rx=0 tx=0).stop 2024-09-19T07:45:22.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.586+0000 7f597a9fd640 1 -- 172.21.15.16:0/1088460099 shutdown_connections 2024-09-19T07:45:22.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.586+0000 7f597a9fd640 1 --2- 172.21.15.16:0/1088460099 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f595406c2b0 0x7f595406e770 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.586+0000 7f597a9fd640 1 --2- 172.21.15.16:0/1088460099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59740cd220 0x7f597415b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.586+0000 7f597a9fd640 1 --2- 172.21.15.16:0/1088460099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59741f4a30 0x7f59741f7840 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.586+0000 7f597a9fd640 1 --2- 172.21.15.16:0/1088460099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59600039b0 0x7f59741f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.586+0000 7f597a9fd640 1 -- 172.21.15.16:0/1088460099 >> 172.21.15.16:0/1088460099 conn(0x7f597409d760 msgr2=0x7f597406db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:22.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.586+0000 7f597a9fd640 1 -- 172.21.15.16:0/1088460099 shutdown_connections 2024-09-19T07:45:22.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:22.586+0000 7f597a9fd640 1 -- 172.21.15.16:0/1088460099 wait complete. 2024-09-19T07:45:22.597 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:45:22.597 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:45:22.615 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-19T07:45:22.615 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:45:22.615 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-19T07:45:22.615 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:45:22.615 DEBUG:teuthology.orchestra.run.smithi016:> n = 10100 2024-09-19T07:45:22.616 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-19T07:45:22.616 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-19T07:45:22.616 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-19T07:45:22.616 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file1" 2024-09-19T07:45:22.616 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:45:22.616 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-19T07:45:22.617 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-19T07:45:22.617 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-19T07:45:22.617 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:45:22.617 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-19T07:45:22.617 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-19T07:45:22.617 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:45:22.618 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-19T07:45:22.618 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-19T07:45:22.618 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-19T07:45:22.618 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-19T07:45:22.618 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:45:22.618 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-19T07:45:22.619 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-19T07:45:22.619 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:45:22.619 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-19T07:45:22.619 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:45:22.619 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:45:22.619 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-19T07:45:22.620 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-19T07:45:22.620 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-19T07:45:22.620 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:45:22.620 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:45:22.620 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-19T07:45:22.620 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-19T07:45:22.621 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:45:37.295 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:45:37.298 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:45:56.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:45:56.586 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:45:56.586 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:45:56.603 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-19T07:45:56.603 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:45:56.603 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-19T07:45:56.603 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:45:56.603 DEBUG:teuthology.orchestra.run.smithi016:> n = 5 2024-09-19T07:45:56.603 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-19T07:45:56.604 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-19T07:45:56.604 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-19T07:45:56.604 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file2" 2024-09-19T07:45:56.604 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:45:56.604 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-19T07:45:56.604 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-19T07:45:56.604 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-19T07:45:56.604 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:45:56.604 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-19T07:45:56.604 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-19T07:45:56.604 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:45:56.604 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-19T07:45:56.604 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-19T07:45:56.604 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-19T07:45:56.605 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-19T07:45:56.605 DEBUG:teuthology.orchestra.run.smithi016:> if True: 2024-09-19T07:45:56.605 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-19T07:45:56.605 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-19T07:45:56.605 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:45:56.605 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-19T07:45:56.605 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:45:56.605 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:45:56.605 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-19T07:45:56.605 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-19T07:45:56.606 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-19T07:45:56.606 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:45:56.606 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:45:56.606 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-19T07:45:56.606 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-19T07:45:56.606 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:45:56.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:45:56.807 INFO:tasks.ceph_test_case:waiting 1000s for health warning matching MDS_CLIENT_OLDEST_TID 2024-09-19T07:45:56.807 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health detail --format=json 2024-09-19T07:45:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.949+0000 7f65cb45e640 1 Processor -- start 2024-09-19T07:45:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.949+0000 7f65cb45e640 1 -- start start 2024-09-19T07:45:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.950+0000 7f65cb45e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f65c4142ae0 0x7f65c41692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.950+0000 7f65cb45e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c41620b0 con 0x7f65c40d21f0 2024-09-19T07:45:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.950+0000 7f65cb45e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c4162370 con 0x7f65c4072590 2024-09-19T07:45:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.950+0000 7f65cb45e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f65c4141750 con 0x7f65c4142ae0 2024-09-19T07:45:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.950+0000 7f65c99fb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f65c4142ae0 0x7f65c41692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.950+0000 7f65c91fa640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f65c40d21f0 0x7f65c40d25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33222/0 (socket says 172.21.15.16:33222) 2024-09-19T07:45:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.950+0000 7f65c91fa640 1 -- 172.21.15.16:0/3859959469 learned_addr learned my addr 172.21.15.16:0/3859959469 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:45:56.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.951+0000 7f65c99fb640 1 -- 172.21.15.16:0/3859959469 >> v1:172.21.15.146:6789/0 conn(0x7f65c4072590 legacy=0x7f65c406d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:56.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.951+0000 7f65c99fb640 1 -- 172.21.15.16:0/3859959469 >> v1:172.21.15.16:6789/0 conn(0x7f65c40d21f0 legacy=0x7f65c40d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:56.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.951+0000 7f65c99fb640 1 -- 172.21.15.16:0/3859959469 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c4162650 con 0x7f65c4142ae0 2024-09-19T07:45:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.951+0000 7f65c99fb640 1 --2- 172.21.15.16:0/3859959469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c4142ae0 0x7f65c41692c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0097a0 tx=0x7f65bc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8daa13c47b49a128 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.951+0000 7f65c3fff640 1 -- 172.21.15.16:0/3859959469 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc037070 con 0x7f65c4142ae0 2024-09-19T07:45:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.952+0000 7f65c3fff640 1 -- 172.21.15.16:0/3859959469 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 46+0+0 (secure 0 0 0) 0x7f65bc032440 con 0x7f65c4142ae0 2024-09-19T07:45:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.952+0000 7f65c3fff640 1 -- 172.21.15.16:0/3859959469 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc032810 con 0x7f65c4142ae0 2024-09-19T07:45:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.952+0000 7f65cb45e640 1 -- 172.21.15.16:0/3859959469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c4142ae0 msgr2=0x7f65c41692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.952+0000 7f65cb45e640 1 --2- 172.21.15.16:0/3859959469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c4142ae0 0x7f65c41692c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0097a0 tx=0x7f65bc02a1a0 comp rx=0 tx=0).stop 2024-09-19T07:45:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.952+0000 7f65cb45e640 1 -- 172.21.15.16:0/3859959469 shutdown_connections 2024-09-19T07:45:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.952+0000 7f65cb45e640 1 --2- 172.21.15.16:0/3859959469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c4142ae0 0x7f65c41692c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.952+0000 7f65cb45e640 1 -- 172.21.15.16:0/3859959469 >> 172.21.15.16:0/3859959469 conn(0x7f65c40b3ed0 msgr2=0x7f65c40b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.952+0000 7f65cb45e640 1 -- 172.21.15.16:0/3859959469 shutdown_connections 2024-09-19T07:45:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.952+0000 7f65cb45e640 1 -- 172.21.15.16:0/3859959469 wait complete. 2024-09-19T07:45:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.953+0000 7f65cb45e640 1 Processor -- start 2024-09-19T07:45:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.953+0000 7f65cb45e640 1 -- start start 2024-09-19T07:45:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.954+0000 7f65cb45e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65c4072590 0x7f65c41f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.954+0000 7f65cb45e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c40d21f0 0x7f65c413fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.954+0000 7f65cb45e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65c4142ae0 0x7f65c41f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.954+0000 7f65cb45e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4162c10 con 0x7f65c40d21f0 2024-09-19T07:45:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.954+0000 7f65cb45e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f65c406d7f0 con 0x7f65c4072590 2024-09-19T07:45:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.954+0000 7f65cb45e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f65c4158c30 con 0x7f65c4142ae0 2024-09-19T07:45:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.954+0000 7f65c89f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c40d21f0 0x7f65c413fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.954+0000 7f65c89f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c40d21f0 0x7f65c413fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40426/0 (socket says 172.21.15.16:40426) 2024-09-19T07:45:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.954+0000 7f65c89f9640 1 -- 172.21.15.16:0/4223452668 learned_addr learned my addr 172.21.15.16:0/4223452668 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:45:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.954+0000 7f65c91fa640 1 --2- 172.21.15.16:0/4223452668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65c4072590 0x7f65c41f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.954+0000 7f65c99fb640 1 --2- 172.21.15.16:0/4223452668 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65c4142ae0 0x7f65c41f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.955+0000 7f65c89f9640 1 -- 172.21.15.16:0/4223452668 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65c4142ae0 msgr2=0x7f65c41f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.955+0000 7f65c89f9640 1 --2- 172.21.15.16:0/4223452668 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65c4142ae0 0x7f65c41f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.955+0000 7f65c89f9640 1 -- 172.21.15.16:0/4223452668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65c4072590 msgr2=0x7f65c41f54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.955+0000 7f65c89f9640 1 --2- 172.21.15.16:0/4223452668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65c4072590 0x7f65c41f54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.955+0000 7f65c89f9640 1 -- 172.21.15.16:0/4223452668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c40b7780 con 0x7f65c40d21f0 2024-09-19T07:45:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.955+0000 7f65c89f9640 1 --2- 172.21.15.16:0/4223452668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c40d21f0 0x7f65c413fe80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f65b800c3f0 tx=0x7f65b800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.955+0000 7f65c1ffb640 1 -- 172.21.15.16:0/4223452668 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b80052d0 con 0x7f65c40d21f0 2024-09-19T07:45:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.955+0000 7f65c1ffb640 1 -- 172.21.15.16:0/4223452668 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 46+0+0 (secure 0 0 0) 0x7f65b8005490 con 0x7f65c40d21f0 2024-09-19T07:45:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.955+0000 7f65cb45e640 1 -- 172.21.15.16:0/4223452668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c40b7a60 con 0x7f65c40d21f0 2024-09-19T07:45:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.955+0000 7f65c1ffb640 1 -- 172.21.15.16:0/4223452668 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8007260 con 0x7f65c40d21f0 2024-09-19T07:45:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.955+0000 7f65cb45e640 1 -- 172.21.15.16:0/4223452668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c41f5080 con 0x7f65c40d21f0 2024-09-19T07:45:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.956+0000 7f65cb45e640 1 -- 172.21.15.16:0/4223452668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c406d0e0 con 0x7f65c40d21f0 2024-09-19T07:45:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.957+0000 7f65c1ffb640 1 -- 172.21.15.16:0/4223452668 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f65b8007400 con 0x7f65c40d21f0 2024-09-19T07:45:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.957+0000 7f65c1ffb640 1 --2- 172.21.15.16:0/4223452668 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f65b006c2b0 0x7f65b006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.957+0000 7f65c1ffb640 1 -- 172.21.15.16:0/4223452668 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6159+0+0 (secure 0 0 0) 0x7f65b808aaa0 con 0x7f65c40d21f0 2024-09-19T07:45:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.957+0000 7f65c91fa640 1 --2- 172.21.15.16:0/4223452668 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f65b006c2b0 0x7f65b006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.958+0000 7f65c91fa640 1 --2- 172.21.15.16:0/4223452668 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f65b006c2b0 0x7f65b006e770 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f65b4005c00 tx=0x7f65b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:56.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:56.961+0000 7f65c1ffb640 1 -- 172.21.15.16:0/4223452668 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f65b8058970 con 0x7f65c40d21f0 2024-09-19T07:45:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:57.206+0000 7f65cb45e640 1 -- 172.21.15.16:0/4223452668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail", "format": "json"} v 0) -- 0x7f65c40690c0 con 0x7f65c40d21f0 2024-09-19T07:45:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:57.206+0000 7f65c1ffb640 1 -- 172.21.15.16:0/4223452668 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail", "format": "json"}]=0 v0) ==== 92+0+46 (secure 0 0 0) 0x7f65b805cc70 con 0x7f65c40d21f0 2024-09-19T07:45:57.206 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:45:57.207 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T07:45:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:57.208+0000 7f65cb45e640 1 -- 172.21.15.16:0/4223452668 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f65b006c2b0 msgr2=0x7f65b006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:57.208+0000 7f65cb45e640 1 --2- 172.21.15.16:0/4223452668 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f65b006c2b0 0x7f65b006e770 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f65b4005c00 tx=0x7f65b4009040 comp rx=0 tx=0).stop 2024-09-19T07:45:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:57.208+0000 7f65cb45e640 1 -- 172.21.15.16:0/4223452668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c40d21f0 msgr2=0x7f65c413fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:57.208+0000 7f65cb45e640 1 --2- 172.21.15.16:0/4223452668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c40d21f0 0x7f65c413fe80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f65b800c3f0 tx=0x7f65b800c8b0 comp rx=0 tx=0).stop 2024-09-19T07:45:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:57.208+0000 7f65cb45e640 1 -- 172.21.15.16:0/4223452668 shutdown_connections 2024-09-19T07:45:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:57.208+0000 7f65cb45e640 1 --2- 172.21.15.16:0/4223452668 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f65b006c2b0 0x7f65b006e770 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:57.208+0000 7f65cb45e640 1 --2- 172.21.15.16:0/4223452668 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f65c4142ae0 0x7f65c41f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:57.208+0000 7f65cb45e640 1 --2- 172.21.15.16:0/4223452668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c40d21f0 0x7f65c413fe80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:57.208+0000 7f65cb45e640 1 --2- 172.21.15.16:0/4223452668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f65c4072590 0x7f65c41f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:57.208+0000 7f65cb45e640 1 -- 172.21.15.16:0/4223452668 >> 172.21.15.16:0/4223452668 conn(0x7f65c40b3ed0 msgr2=0x7f65c41419d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:57.208+0000 7f65cb45e640 1 -- 172.21.15.16:0/4223452668 shutdown_connections 2024-09-19T07:45:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:45:57.208+0000 7f65cb45e640 1 -- 172.21.15.16:0/4223452668 wait complete. 2024-09-19T07:45:57.219 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-19T07:45:57.219 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=1000 retry_count=0)... 2024-09-19T07:46:02.220 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health detail --format=json 2024-09-19T07:46:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.336+0000 7f9575986640 1 Processor -- start 2024-09-19T07:46:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.336+0000 7f9575986640 1 -- start start 2024-09-19T07:46:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.337+0000 7f9575986640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f957015b120 0x7f957006f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.337+0000 7f9575986640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f95700d0bc0 0x7f957006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.337+0000 7f9575986640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f957015b500 con 0x7f957006dec0 2024-09-19T07:46:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.337+0000 7f9575986640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f957006e940 con 0x7f957015b120 2024-09-19T07:46:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.337+0000 7f9575986640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f957006ec20 con 0x7f95700d0bc0 2024-09-19T07:46:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.337+0000 7f956effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f957015b120 0x7f957006f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.337+0000 7f956effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f957015b120 0x7f957006f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40430/0 (socket says 172.21.15.16:40430) 2024-09-19T07:46:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.337+0000 7f956effd640 1 -- 172.21.15.16:0/4110027217 learned_addr learned my addr 172.21.15.16:0/4110027217 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:46:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.338+0000 7f956effd640 1 -- 172.21.15.16:0/4110027217 >> v2:172.21.15.146:3300/0 conn(0x7f95700d0bc0 msgr2=0x7f957006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:02.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.338+0000 7f956f7fe640 1 --2- 172.21.15.16:0/4110027217 >> v2:172.21.15.146:3300/0 conn(0x7f95700d0bc0 0x7f957006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:02.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.338+0000 7f956effd640 1 --2- 172.21.15.16:0/4110027217 >> v2:172.21.15.146:3300/0 conn(0x7f95700d0bc0 0x7f957006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:02.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.338+0000 7f956effd640 1 -- 172.21.15.16:0/4110027217 >> v1:172.21.15.146:6789/0 conn(0x7f957006dec0 legacy=0x7f95700d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:02.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.338+0000 7f956effd640 1 -- 172.21.15.16:0/4110027217 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957006ef00 con 0x7f957015b120 2024-09-19T07:46:02.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.338+0000 7f956f7fe640 1 --2- 172.21.15.16:0/4110027217 >> v2:172.21.15.146:3300/0 conn(0x7f95700d0bc0 0x7f957006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:46:02.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.338+0000 7f956effd640 1 --2- 172.21.15.16:0/4110027217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f957015b120 0x7f957006f9e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f957015eb60 tx=0x7f956002c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f86e47183e67c13 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:02.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.339+0000 7f956dffb640 1 -- 172.21.15.16:0/4110027217 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956000b2d0 con 0x7f957015b120 2024-09-19T07:46:02.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.339+0000 7f956dffb640 1 -- 172.21.15.16:0/4110027217 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 46+0+0 (secure 0 0 0) 0x7f956000b470 con 0x7f957015b120 2024-09-19T07:46:02.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.339+0000 7f956dffb640 1 -- 172.21.15.16:0/4110027217 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560009820 con 0x7f957015b120 2024-09-19T07:46:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.339+0000 7f9575986640 1 -- 172.21.15.16:0/4110027217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f957015b120 msgr2=0x7f957006f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.339+0000 7f9575986640 1 --2- 172.21.15.16:0/4110027217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f957015b120 0x7f957006f9e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f957015eb60 tx=0x7f956002c8f0 comp rx=0 tx=0).stop 2024-09-19T07:46:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.339+0000 7f9575986640 1 -- 172.21.15.16:0/4110027217 shutdown_connections 2024-09-19T07:46:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.339+0000 7f9575986640 1 --2- 172.21.15.16:0/4110027217 >> v2:172.21.15.146:3300/0 conn(0x7f95700d0bc0 0x7f957006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.339+0000 7f9575986640 1 --2- 172.21.15.16:0/4110027217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f957015b120 0x7f957006f9e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.339+0000 7f9575986640 1 -- 172.21.15.16:0/4110027217 >> 172.21.15.16:0/4110027217 conn(0x7f957006bdf0 msgr2=0x7f957006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.339+0000 7f9575986640 1 -- 172.21.15.16:0/4110027217 shutdown_connections 2024-09-19T07:46:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.339+0000 7f9575986640 1 -- 172.21.15.16:0/4110027217 wait complete. 2024-09-19T07:46:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.340+0000 7f9575986640 1 Processor -- start 2024-09-19T07:46:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.341+0000 7f9575986640 1 -- start start 2024-09-19T07:46:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.341+0000 7f9575986640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f957006dec0 0x7f9570144cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.341+0000 7f9575986640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95700d0bc0 0x7f95700998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.341+0000 7f9575986640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f957015b120 0x7f957015de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.341+0000 7f9575986640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f95700ce060 con 0x7f95700d0bc0 2024-09-19T07:46:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.341+0000 7f9575986640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f957006f4c0 con 0x7f957006dec0 2024-09-19T07:46:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.341+0000 7f9575986640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f95700c5160 con 0x7f957015b120 2024-09-19T07:46:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.342+0000 7f956e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95700d0bc0 0x7f95700998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.342+0000 7f956e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95700d0bc0 0x7f95700998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40436/0 (socket says 172.21.15.16:40436) 2024-09-19T07:46:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.342+0000 7f956e7fc640 1 -- 172.21.15.16:0/568706884 learned_addr learned my addr 172.21.15.16:0/568706884 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:46:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.342+0000 7f956effd640 1 --2- 172.21.15.16:0/568706884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f957006dec0 0x7f9570144cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.342+0000 7f956f7fe640 1 --2- 172.21.15.16:0/568706884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f957015b120 0x7f957015de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.342+0000 7f956e7fc640 1 -- 172.21.15.16:0/568706884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f957015b120 msgr2=0x7f957015de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.342+0000 7f956e7fc640 1 --2- 172.21.15.16:0/568706884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f957015b120 0x7f957015de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.342+0000 7f956e7fc640 1 -- 172.21.15.16:0/568706884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f957006dec0 msgr2=0x7f9570144cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:02.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.342+0000 7f956e7fc640 1 --2- 172.21.15.16:0/568706884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f957006dec0 0x7f9570144cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:02.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.342+0000 7f956e7fc640 1 -- 172.21.15.16:0/568706884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9570145200 con 0x7f95700d0bc0 2024-09-19T07:46:02.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.342+0000 7f956e7fc640 1 --2- 172.21.15.16:0/568706884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95700d0bc0 0x7f95700998d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f955800c460 tx=0x7f955800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:02.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.342+0000 7f9567fff640 1 -- 172.21.15.16:0/568706884 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558016020 con 0x7f95700d0bc0 2024-09-19T07:46:02.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.342+0000 7f9567fff640 1 -- 172.21.15.16:0/568706884 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 46+0+0 (secure 0 0 0) 0x7f9558012870 con 0x7f95700d0bc0 2024-09-19T07:46:02.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.342+0000 7f9575986640 1 -- 172.21.15.16:0/568706884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957015cdf0 con 0x7f95700d0bc0 2024-09-19T07:46:02.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.342+0000 7f9567fff640 1 -- 172.21.15.16:0/568706884 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558012c60 con 0x7f95700d0bc0 2024-09-19T07:46:02.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.342+0000 7f9575986640 1 -- 172.21.15.16:0/568706884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f957015d3b0 con 0x7f95700d0bc0 2024-09-19T07:46:02.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.343+0000 7f9575986640 1 -- 172.21.15.16:0/568706884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f957006e330 con 0x7f95700d0bc0 2024-09-19T07:46:02.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.344+0000 7f9567fff640 1 -- 172.21.15.16:0/568706884 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9558019020 con 0x7f95700d0bc0 2024-09-19T07:46:02.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.344+0000 7f9567fff640 1 --2- 172.21.15.16:0/568706884 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f955006c2b0 0x7f955006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:02.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.344+0000 7f9567fff640 1 -- 172.21.15.16:0/568706884 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6159+0+0 (secure 0 0 0) 0x7f955808b310 con 0x7f95700d0bc0 2024-09-19T07:46:02.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.345+0000 7f956effd640 1 --2- 172.21.15.16:0/568706884 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f955006c2b0 0x7f955006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:02.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.345+0000 7f956effd640 1 --2- 172.21.15.16:0/568706884 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f955006c2b0 0x7f955006e770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f957015eb60 tx=0x7f956000b000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:02.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.349+0000 7f9567fff640 1 -- 172.21.15.16:0/568706884 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9558020030 con 0x7f95700d0bc0 2024-09-19T07:46:02.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.591+0000 7f9575986640 1 -- 172.21.15.16:0/568706884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail", "format": "json"} v 0) -- 0x7f957015e940 con 0x7f95700d0bc0 2024-09-19T07:46:02.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.592+0000 7f9567fff640 1 -- 172.21.15.16:0/568706884 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail", "format": "json"}]=0 v0) ==== 92+0+322 (secure 0 0 0) 0x7f9570145200 con 0x7f95700d0bc0 2024-09-19T07:46:02.593 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:46:02.593 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CLIENT_OLDEST_TID":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to advance oldest client/flush tid","count":1},"detail":[{"message":"mds.c(mds.0): Client smithi016:0 failing to advance its oldest client/flush tid. client_id: 5514"}],"muted":false}},"mutes":[]} 2024-09-19T07:46:02.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.593+0000 7f9575986640 1 -- 172.21.15.16:0/568706884 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f955006c2b0 msgr2=0x7f955006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:02.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.593+0000 7f9575986640 1 --2- 172.21.15.16:0/568706884 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f955006c2b0 0x7f955006e770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f957015eb60 tx=0x7f956000b000 comp rx=0 tx=0).stop 2024-09-19T07:46:02.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.593+0000 7f9575986640 1 -- 172.21.15.16:0/568706884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95700d0bc0 msgr2=0x7f95700998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:02.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.593+0000 7f9575986640 1 --2- 172.21.15.16:0/568706884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95700d0bc0 0x7f95700998d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f955800c460 tx=0x7f955800c920 comp rx=0 tx=0).stop 2024-09-19T07:46:02.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.594+0000 7f9575986640 1 -- 172.21.15.16:0/568706884 shutdown_connections 2024-09-19T07:46:02.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.594+0000 7f9575986640 1 --2- 172.21.15.16:0/568706884 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f955006c2b0 0x7f955006e770 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:02.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.594+0000 7f9575986640 1 --2- 172.21.15.16:0/568706884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f957015b120 0x7f957015de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:02.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.594+0000 7f9575986640 1 --2- 172.21.15.16:0/568706884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95700d0bc0 0x7f95700998d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:02.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.594+0000 7f9575986640 1 --2- 172.21.15.16:0/568706884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f957006dec0 0x7f9570144cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:02.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.594+0000 7f9575986640 1 -- 172.21.15.16:0/568706884 >> 172.21.15.16:0/568706884 conn(0x7f957006bdf0 msgr2=0x7f95700b3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:02.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.594+0000 7f9575986640 1 -- 172.21.15.16:0/568706884 shutdown_connections 2024-09-19T07:46:02.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.594+0000 7f9575986640 1 -- 172.21.15.16:0/568706884 wait complete. 2024-09-19T07:46:02.606 DEBUG:tasks.ceph_test_case:detail_strings: ['mds.c(mds.0): Client smithi016:0 failing to advance its oldest client/flush tid. client_id: 5514'] 2024-09-19T07:46:02.606 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-19T07:46:02.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_session_metadata_threshold 5000 2024-09-19T07:46:02.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.711+0000 7f9e1aba2640 1 Processor -- start 2024-09-19T07:46:02.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.711+0000 7f9e1aba2640 1 -- start start 2024-09-19T07:46:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.712+0000 7f9e1aba2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e14162b00 0x7f9e140d2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.712+0000 7f9e1aba2640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e140d1de0 con 0x7f9e1415e300 2024-09-19T07:46:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.712+0000 7f9e1aba2640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e140d20c0 con 0x7f9e140d41a0 2024-09-19T07:46:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.712+0000 7f9e1aba2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9e140d29f0 con 0x7f9e14162b00 2024-09-19T07:46:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.712+0000 7f9e1939f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e14162b00 0x7f9e140d2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.712+0000 7f9e19ba0640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9e1415e300 0x7f9e140b4bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54230/0 (socket says 172.21.15.16:54230) 2024-09-19T07:46:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.712+0000 7f9e19ba0640 1 -- 172.21.15.16:0/395765091 learned_addr learned my addr 172.21.15.16:0/395765091 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:46:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.712+0000 7f9e1939f640 1 -- 172.21.15.16:0/395765091 >> v1:172.21.15.146:6789/0 conn(0x7f9e140d41a0 legacy=0x7f9e140cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.712+0000 7f9e1939f640 1 -- 172.21.15.16:0/395765091 >> v1:172.21.15.16:6789/0 conn(0x7f9e1415e300 legacy=0x7f9e140b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:02.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.712+0000 7f9e1939f640 1 -- 172.21.15.16:0/395765091 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e140d23a0 con 0x7f9e14162b00 2024-09-19T07:46:02.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.712+0000 7f9e18b9e640 1 -- 172.21.15.16:0/395765091 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1760545571 0 0) 0x7f9e140d1de0 con 0x7f9e1415e300 2024-09-19T07:46:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.713+0000 7f9e1939f640 1 --2- 172.21.15.16:0/395765091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e14162b00 0x7f9e140d2e80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9e10009790 tx=0x7f9e1002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ee59b6c6d20c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.713+0000 7f9e18b9e640 1 -- 172.21.15.16:0/395765091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e10037070 con 0x7f9e14162b00 2024-09-19T07:46:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.713+0000 7f9e18b9e640 1 -- 172.21.15.16:0/395765091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 46+0+0 (secure 0 0 0) 0x7f9e10032440 con 0x7f9e14162b00 2024-09-19T07:46:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.713+0000 7f9e18b9e640 1 -- 172.21.15.16:0/395765091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e10032810 con 0x7f9e14162b00 2024-09-19T07:46:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.714+0000 7f9e1aba2640 1 -- 172.21.15.16:0/395765091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e14162b00 msgr2=0x7f9e140d2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.714+0000 7f9e1aba2640 1 --2- 172.21.15.16:0/395765091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e14162b00 0x7f9e140d2e80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9e10009790 tx=0x7f9e1002a1c0 comp rx=0 tx=0).stop 2024-09-19T07:46:02.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.714+0000 7f9e1aba2640 1 -- 172.21.15.16:0/395765091 shutdown_connections 2024-09-19T07:46:02.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.714+0000 7f9e1aba2640 1 --2- 172.21.15.16:0/395765091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e14162b00 0x7f9e140d2e80 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:02.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.714+0000 7f9e1aba2640 1 -- 172.21.15.16:0/395765091 >> 172.21.15.16:0/395765091 conn(0x7f9e1408bf80 msgr2=0x7f9e1409d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:02.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.714+0000 7f9e1aba2640 1 -- 172.21.15.16:0/395765091 shutdown_connections 2024-09-19T07:46:02.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.714+0000 7f9e1aba2640 1 -- 172.21.15.16:0/395765091 wait complete. 2024-09-19T07:46:02.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.715+0000 7f9e1aba2640 1 Processor -- start 2024-09-19T07:46:02.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.715+0000 7f9e1aba2640 1 -- start start 2024-09-19T07:46:02.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.716+0000 7f9e1aba2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e140d41a0 0x7f9e14154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:02.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.716+0000 7f9e1aba2640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e1415e300 0x7f9e140c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:02.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.716+0000 7f9e1aba2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e14162b00 0x7f9e140c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:02.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.716+0000 7f9e1aba2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1415e860 con 0x7f9e140d41a0 2024-09-19T07:46:02.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.716+0000 7f9e1aba2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1415e6e0 con 0x7f9e14162b00 2024-09-19T07:46:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.716+0000 7f9e1aba2640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9e140d4630 con 0x7f9e1415e300 2024-09-19T07:46:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.716+0000 7f9e19ba0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e140d41a0 0x7f9e14154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.716+0000 7f9e19ba0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e140d41a0 0x7f9e14154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48396/0 (socket says 172.21.15.16:48396) 2024-09-19T07:46:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.716+0000 7f9e19ba0640 1 -- 172.21.15.16:0/1213263228 learned_addr learned my addr 172.21.15.16:0/1213263228 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:46:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.716+0000 7f9e1a3a1640 1 --2- 172.21.15.16:0/1213263228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e14162b00 0x7f9e140c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.716+0000 7f9e1939f640 1 --2- 172.21.15.16:0/1213263228 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e1415e300 0x7f9e140c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.716+0000 7f9e19ba0640 1 -- 172.21.15.16:0/1213263228 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e1415e300 msgr2=0x7f9e140c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.716+0000 7f9e19ba0640 1 --2- 172.21.15.16:0/1213263228 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e1415e300 0x7f9e140c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.716+0000 7f9e19ba0640 1 -- 172.21.15.16:0/1213263228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e14162b00 msgr2=0x7f9e140c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.716+0000 7f9e19ba0640 1 --2- 172.21.15.16:0/1213263228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e14162b00 0x7f9e140c99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.716+0000 7f9e19ba0640 1 -- 172.21.15.16:0/1213263228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e140c87c0 con 0x7f9e140d41a0 2024-09-19T07:46:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.716+0000 7f9e1a3a1640 1 --2- 172.21.15.16:0/1213263228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e14162b00 0x7f9e140c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:46:02.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.716+0000 7f9e19ba0640 1 --2- 172.21.15.16:0/1213263228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e140d41a0 0x7f9e14154580 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400c500 tx=0x7f9e0400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.716+0000 7f9e0a7fc640 1 -- 172.21.15.16:0/1213263228 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e04018070 con 0x7f9e140d41a0 2024-09-19T07:46:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.717+0000 7f9e0a7fc640 1 -- 172.21.15.16:0/1213263228 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 46+0+0 (secure 0 0 0) 0x7f9e04007580 con 0x7f9e140d41a0 2024-09-19T07:46:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.717+0000 7f9e1aba2640 1 -- 172.21.15.16:0/1213263228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e140c8aa0 con 0x7f9e140d41a0 2024-09-19T07:46:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.717+0000 7f9e0a7fc640 1 -- 172.21.15.16:0/1213263228 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e04004a20 con 0x7f9e140d41a0 2024-09-19T07:46:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.717+0000 7f9e1aba2640 1 -- 172.21.15.16:0/1213263228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e140c9060 con 0x7f9e140d41a0 2024-09-19T07:46:02.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.718+0000 7f9e1aba2640 1 -- 172.21.15.16:0/1213263228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e14069110 con 0x7f9e140d41a0 2024-09-19T07:46:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.718+0000 7f9e0a7fc640 1 -- 172.21.15.16:0/1213263228 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9e0401b070 con 0x7f9e140d41a0 2024-09-19T07:46:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.719+0000 7f9e0a7fc640 1 --2- 172.21.15.16:0/1213263228 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9df806c2b0 0x7f9df806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.719+0000 7f9e0a7fc640 1 -- 172.21.15.16:0/1213263228 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6159+0+0 (secure 0 0 0) 0x7f9e0408a920 con 0x7f9e140d41a0 2024-09-19T07:46:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.719+0000 7f9e1939f640 1 --2- 172.21.15.16:0/1213263228 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9df806c2b0 0x7f9df806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:02.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.719+0000 7f9e1939f640 1 --2- 172.21.15.16:0/1213263228 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9df806c2b0 0x7f9df806e770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9e10009790 tx=0x7f9e1002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:02.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.723+0000 7f9e0a7fc640 1 -- 172.21.15.16:0/1213263228 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9e04058f60 con 0x7f9e140d41a0 2024-09-19T07:46:02.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.897+0000 7f9e1aba2640 1 -- 172.21.15.16:0/1213263228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_session_metadata_threshold}] v 0) -- 0x7f9e14075710 con 0x7f9e140d41a0 2024-09-19T07:46:02.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.900+0000 7f9e0a7fc640 1 -- 172.21.15.16:0/1213263228 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_session_metadata_threshold}]=0 v19)=0 v19) ==== 131+0+0 (secure 0 0 0) 0x7f9e14075710 con 0x7f9e140d41a0 2024-09-19T07:46:02.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.902+0000 7f9e1aba2640 1 -- 172.21.15.16:0/1213263228 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9df806c2b0 msgr2=0x7f9df806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:02.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.902+0000 7f9e1aba2640 1 --2- 172.21.15.16:0/1213263228 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9df806c2b0 0x7f9df806e770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9e10009790 tx=0x7f9e1002e000 comp rx=0 tx=0).stop 2024-09-19T07:46:02.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.902+0000 7f9e1aba2640 1 -- 172.21.15.16:0/1213263228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e140d41a0 msgr2=0x7f9e14154580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:02.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.902+0000 7f9e1aba2640 1 --2- 172.21.15.16:0/1213263228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e140d41a0 0x7f9e14154580 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400c500 tx=0x7f9e0400c9c0 comp rx=0 tx=0).stop 2024-09-19T07:46:02.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.902+0000 7f9e1aba2640 1 -- 172.21.15.16:0/1213263228 shutdown_connections 2024-09-19T07:46:02.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.902+0000 7f9e1aba2640 1 --2- 172.21.15.16:0/1213263228 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9df806c2b0 0x7f9df806e770 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:02.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.902+0000 7f9e1aba2640 1 --2- 172.21.15.16:0/1213263228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e14162b00 0x7f9e140c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:02.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.902+0000 7f9e1aba2640 1 --2- 172.21.15.16:0/1213263228 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e1415e300 0x7f9e140c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:02.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.902+0000 7f9e1aba2640 1 --2- 172.21.15.16:0/1213263228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e140d41a0 0x7f9e14154580 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:02.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.902+0000 7f9e1aba2640 1 -- 172.21.15.16:0/1213263228 >> 172.21.15.16:0/1213263228 conn(0x7f9e1408bf80 msgr2=0x7f9e1406de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:02.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.902+0000 7f9e1aba2640 1 -- 172.21.15.16:0/1213263228 shutdown_connections 2024-09-19T07:46:02.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:46:02.902+0000 7f9e1aba2640 1 -- 172.21.15.16:0/1213263228 wait complete. 2024-09-19T07:46:02.914 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:46:02.915 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:46:02.931 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-19T07:46:02.931 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:46:02.931 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-19T07:46:02.932 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:46:02.932 DEBUG:teuthology.orchestra.run.smithi016:> n = 100000 2024-09-19T07:46:02.932 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-19T07:46:02.932 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-19T07:46:02.932 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-19T07:46:02.932 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file2" 2024-09-19T07:46:02.932 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:46:02.932 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-19T07:46:02.932 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-19T07:46:02.932 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-19T07:46:02.932 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:46:02.932 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-19T07:46:02.932 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-19T07:46:02.933 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:46:02.933 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-19T07:46:02.933 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-19T07:46:02.933 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-19T07:46:02.933 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-19T07:46:02.933 DEBUG:teuthology.orchestra.run.smithi016:> if True: 2024-09-19T07:46:02.933 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-19T07:46:02.933 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-19T07:46:02.933 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:46:02.933 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-19T07:46:02.933 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:46:02.933 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:46:02.933 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-19T07:46:02.934 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-19T07:46:02.934 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-19T07:46:02.934 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:46:02.934 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:46:02.934 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-19T07:46:02.934 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-19T07:46:02.934 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:46:07.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:46:07.360 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:46:37.474 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:46:37.477 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:47:07.846 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:47:07.852 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:47:37.911 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:47:37.915 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:48:08.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:48:08.010 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:48:30.371 INFO:tasks.ceph.mds.c.smithi016.stderr:2024-09-19T07:48:30.362+0000 7efdceb07640 -1 mds.0.sessionmap save: session (client.5514) exceeds sesion metadata threshold - blocklisting 2024-09-19T07:48:30.372 INFO:tasks.ceph.mds.c.smithi016.stderr:2024-09-19T07:48:30.366+0000 7efdceb07640 -1 mds.0.sessionmap save_if_dirty: session (client.5514) exceeds sesion metadata threshold - blocklisting 2024-09-19T07:48:30.430 INFO:tasks.ceph.mds.c.smithi016.stderr:2024-09-19T07:48:30.427+0000 7efdceb07640 -1 mds.0.sessionmap save_if_dirty: session (client.5514) exceeds sesion metadata threshold - blocklisting 2024-09-19T07:48:31.322 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-19T07:48:31.321+0000 7f0051ffb640 -1 client.5514 I was blocklisted at osd epoch 59 2024-09-19T07:48:31.753 INFO:teuthology.orchestra.run.smithi016.stderr:Traceback (most recent call last): 2024-09-19T07:48:31.753 INFO:teuthology.orchestra.run.smithi016.stderr: File "", line 20, in 2024-09-19T07:48:31.753 INFO:teuthology.orchestra.run.smithi016.stderr:BrokenPipeError: [Errno 108] Cannot send after transport endpoint shutdown: '/home/ubuntu/cephtest/mnt.0/testdir/file2_24400' 2024-09-19T07:48:31.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31 stdin-killer INFO: command exited with status 1: exiting normally with same code! 2024-09-19T07:48:31.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:48:31.766 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:48:31.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.887+0000 7f5a527fd640 1 Processor -- start 2024-09-19T07:48:31.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.887+0000 7f5a527fd640 1 -- start start 2024-09-19T07:48:31.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.888+0000 7f5a527fd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a4c0d0070 0x7f5a4c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:31.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.888+0000 7f5a527fd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5a4c0d4130 0x7f5a4c0d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:31.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.888+0000 7f5a527fd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a4c0d1b10 con 0x7f5a4c15f0d0 2024-09-19T07:48:31.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.888+0000 7f5a527fd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f5a4c0cf2b0 con 0x7f5a4c0d0070 2024-09-19T07:48:31.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.888+0000 7f5a527fd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5a4c0cf590 con 0x7f5a4c0d4130 2024-09-19T07:48:31.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.888+0000 7f5a4b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a4c0d0070 0x7f5a4c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:31.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.888+0000 7f5a4b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a4c0d0070 0x7f5a4c15f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57702/0 (socket says 172.21.15.16:57702) 2024-09-19T07:48:31.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.888+0000 7f5a4b7fe640 1 -- 172.21.15.16:0/2369422231 learned_addr learned my addr 172.21.15.16:0/2369422231 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:31.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.888+0000 7f5a4bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5a4c15f0d0 0x7f5a4c0d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40966/0 (socket says 172.21.15.16:40966) 2024-09-19T07:48:31.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.888+0000 7f5a50d9a640 1 --2- 172.21.15.16:0/2369422231 >> v2:172.21.15.146:3300/0 conn(0x7f5a4c0d4130 0x7f5a4c0d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:31.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.888+0000 7f5a4affd640 1 -- 172.21.15.16:0/2369422231 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3576728520 0 0) 0x7f5a4c0d1b10 con 0x7f5a4c15f0d0 2024-09-19T07:48:31.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.888+0000 7f5a4affd640 1 -- 172.21.15.16:0/2369422231 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a34003520 con 0x7f5a4c15f0d0 2024-09-19T07:48:31.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.888+0000 7f5a4b7fe640 1 -- 172.21.15.16:0/2369422231 >> v2:172.21.15.146:3300/0 conn(0x7f5a4c0d4130 msgr2=0x7f5a4c0d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:31.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.888+0000 7f5a4b7fe640 1 --2- 172.21.15.16:0/2369422231 >> v2:172.21.15.146:3300/0 conn(0x7f5a4c0d4130 0x7f5a4c0d4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:31.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.888+0000 7f5a4b7fe640 1 -- 172.21.15.16:0/2369422231 >> v1:172.21.15.16:6789/0 conn(0x7f5a4c15f0d0 legacy=0x7f5a4c0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:31.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.888+0000 7f5a4b7fe640 1 -- 172.21.15.16:0/2369422231 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4c0cf870 con 0x7f5a4c0d0070 2024-09-19T07:48:31.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.889+0000 7f5a4b7fe640 1 --2- 172.21.15.16:0/2369422231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4c0d0070 0x7f5a4c15f4b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0d04e0 tx=0x7f5a4002a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459caa4a4a6e6564 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:31.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.890+0000 7f5a4affd640 1 -- 172.21.15.16:0/2369422231 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a40037070 con 0x7f5a4c0d0070 2024-09-19T07:48:31.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.890+0000 7f5a4affd640 1 -- 172.21.15.16:0/2369422231 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 46+0+0 (secure 0 0 0) 0x7f5a40032440 con 0x7f5a4c0d0070 2024-09-19T07:48:31.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.890+0000 7f5a4affd640 1 -- 172.21.15.16:0/2369422231 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a40032810 con 0x7f5a4c0d0070 2024-09-19T07:48:31.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.890+0000 7f5a527fd640 1 -- 172.21.15.16:0/2369422231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4c0d0070 msgr2=0x7f5a4c15f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:31.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.890+0000 7f5a527fd640 1 --2- 172.21.15.16:0/2369422231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4c0d0070 0x7f5a4c15f4b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0d04e0 tx=0x7f5a4002a1f0 comp rx=0 tx=0).stop 2024-09-19T07:48:31.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.890+0000 7f5a527fd640 1 -- 172.21.15.16:0/2369422231 shutdown_connections 2024-09-19T07:48:31.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.890+0000 7f5a527fd640 1 --2- 172.21.15.16:0/2369422231 >> v2:172.21.15.146:3300/0 conn(0x7f5a4c0d4130 0x7f5a4c0d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:31.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.890+0000 7f5a527fd640 1 --2- 172.21.15.16:0/2369422231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4c0d0070 0x7f5a4c15f4b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:31.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.890+0000 7f5a527fd640 1 -- 172.21.15.16:0/2369422231 >> 172.21.15.16:0/2369422231 conn(0x7f5a4c09d760 msgr2=0x7f5a4c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:31.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.891+0000 7f5a527fd640 1 -- 172.21.15.16:0/2369422231 shutdown_connections 2024-09-19T07:48:31.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.891+0000 7f5a527fd640 1 -- 172.21.15.16:0/2369422231 wait complete. 2024-09-19T07:48:31.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.892+0000 7f5a527fd640 1 Processor -- start 2024-09-19T07:48:31.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.893+0000 7f5a527fd640 1 -- start start 2024-09-19T07:48:31.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.894+0000 7f5a527fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a4c0d0070 0x7f5a4c0cd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:31.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.894+0000 7f5a527fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4c0d4130 0x7f5a4c1513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:31.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.894+0000 7f5a4b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4c0d4130 0x7f5a4c1513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:31.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.894+0000 7f5a4b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4c0d4130 0x7f5a4c1513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57718/0 (socket says 172.21.15.16:57718) 2024-09-19T07:48:31.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.894+0000 7f5a4b7fe640 1 -- 172.21.15.16:0/1583119607 learned_addr learned my addr 172.21.15.16:0/1583119607 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:31.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.894+0000 7f5a527fd640 1 --2- 172.21.15.16:0/1583119607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a4c15f0d0 0x7f5a4c1f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:31.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.894+0000 7f5a4bfff640 1 --2- 172.21.15.16:0/1583119607 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a4c0d0070 0x7f5a4c0cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:31.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.894+0000 7f5a527fd640 1 -- 172.21.15.16:0/1583119607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c0cef60 con 0x7f5a4c0d4130 2024-09-19T07:48:31.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.894+0000 7f5a527fd640 1 -- 172.21.15.16:0/1583119607 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c0d4aa0 con 0x7f5a4c15f0d0 2024-09-19T07:48:31.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.894+0000 7f5a527fd640 1 -- 172.21.15.16:0/1583119607 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5a4c0b2f70 con 0x7f5a4c0d0070 2024-09-19T07:48:31.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.895+0000 7f5a50d9a640 1 --2- 172.21.15.16:0/1583119607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a4c15f0d0 0x7f5a4c1f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:31.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.895+0000 7f5a4b7fe640 1 -- 172.21.15.16:0/1583119607 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a4c0d0070 msgr2=0x7f5a4c0cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:31.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.895+0000 7f5a4b7fe640 1 --2- 172.21.15.16:0/1583119607 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a4c0d0070 0x7f5a4c0cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:31.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.895+0000 7f5a4b7fe640 1 -- 172.21.15.16:0/1583119607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a4c15f0d0 msgr2=0x7f5a4c1f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:31.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.895+0000 7f5a4b7fe640 1 --2- 172.21.15.16:0/1583119607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a4c15f0d0 0x7f5a4c1f3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:31.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.895+0000 7f5a4b7fe640 1 -- 172.21.15.16:0/1583119607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4c0cdd60 con 0x7f5a4c0d4130 2024-09-19T07:48:31.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.895+0000 7f5a4b7fe640 1 --2- 172.21.15.16:0/1583119607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4c0d4130 0x7f5a4c1513f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5a4002a230 tx=0x7f5a400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:31.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.895+0000 7f5a48ff9640 1 -- 172.21.15.16:0/1583119607 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a40041020 con 0x7f5a4c0d4130 2024-09-19T07:48:31.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.895+0000 7f5a48ff9640 1 -- 172.21.15.16:0/1583119607 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 46+0+0 (secure 0 0 0) 0x7f5a4003d620 con 0x7f5a4c0d4130 2024-09-19T07:48:31.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.895+0000 7f5a48ff9640 1 -- 172.21.15.16:0/1583119607 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a40037040 con 0x7f5a4c0d4130 2024-09-19T07:48:31.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.895+0000 7f5a527fd640 1 -- 172.21.15.16:0/1583119607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4c0cbc10 con 0x7f5a4c0d4130 2024-09-19T07:48:31.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.896+0000 7f5a527fd640 1 -- 172.21.15.16:0/1583119607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a4c0cb930 con 0x7f5a4c0d4130 2024-09-19T07:48:31.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.897+0000 7f5a527fd640 1 -- 172.21.15.16:0/1583119607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a4c0d4560 con 0x7f5a4c0d4130 2024-09-19T07:48:31.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.898+0000 7f5a48ff9640 1 -- 172.21.15.16:0/1583119607 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5a40044020 con 0x7f5a4c0d4130 2024-09-19T07:48:31.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.898+0000 7f5a48ff9640 1 --2- 172.21.15.16:0/1583119607 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a3806c300 0x7f5a3806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:31.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.899+0000 7f5a48ff9640 1 -- 172.21.15.16:0/1583119607 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6202+0+0 (secure 0 0 0) 0x7f5a400abc70 con 0x7f5a4c0d4130 2024-09-19T07:48:31.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.899+0000 7f5a4bfff640 1 --2- 172.21.15.16:0/1583119607 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a3806c300 0x7f5a3806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:31.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.899+0000 7f5a4bfff640 1 --2- 172.21.15.16:0/1583119607 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a3806c300 0x7f5a3806e7c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c007fd0 tx=0x7f5a3c007cf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:31.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:31.907+0000 7f5a48ff9640 1 -- 172.21.15.16:0/1583119607 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5a4004b030 con 0x7f5a4c0d4130 2024-09-19T07:48:32.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.207+0000 7f5a527fd640 1 -- 172.21.15.16:0/1583119607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a4c06cf30 con 0x7f5a4c0d4130 2024-09-19T07:48:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.208+0000 7f5a48ff9640 1 -- 172.21.15.16:0/1583119607 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+17182 (secure 0 0 0) 0x7f5a40079ba0 con 0x7f5a4c0d4130 2024-09-19T07:48:32.209 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:32.210 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":59,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:48:31.046831+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:45:08.455974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-19T07:45:09.476448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.28","mappings":[{"from":0,"to":6}]},{"pgid":"8.2f","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/2345385426":"2024-09-19T08:48:30.430325+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:48:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.211+0000 7f5a527fd640 1 -- 172.21.15.16:0/1583119607 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a3806c300 msgr2=0x7f5a3806e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.211+0000 7f5a527fd640 1 --2- 172.21.15.16:0/1583119607 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a3806c300 0x7f5a3806e7c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c007fd0 tx=0x7f5a3c007cf0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.211+0000 7f5a527fd640 1 -- 172.21.15.16:0/1583119607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4c0d4130 msgr2=0x7f5a4c1513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.211+0000 7f5a527fd640 1 --2- 172.21.15.16:0/1583119607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4c0d4130 0x7f5a4c1513f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5a4002a230 tx=0x7f5a400046c0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.212+0000 7f5a527fd640 1 -- 172.21.15.16:0/1583119607 shutdown_connections 2024-09-19T07:48:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.212+0000 7f5a527fd640 1 --2- 172.21.15.16:0/1583119607 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a3806c300 0x7f5a3806e7c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.212+0000 7f5a527fd640 1 --2- 172.21.15.16:0/1583119607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a4c15f0d0 0x7f5a4c1f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.212+0000 7f5a527fd640 1 --2- 172.21.15.16:0/1583119607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4c0d4130 0x7f5a4c1513f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.212+0000 7f5a527fd640 1 --2- 172.21.15.16:0/1583119607 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a4c0d0070 0x7f5a4c0cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.212+0000 7f5a527fd640 1 -- 172.21.15.16:0/1583119607 >> 172.21.15.16:0/1583119607 conn(0x7f5a4c09d760 msgr2=0x7f5a4c06d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.212+0000 7f5a527fd640 1 -- 172.21.15.16:0/1583119607 shutdown_connections 2024-09-19T07:48:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.212+0000 7f5a527fd640 1 -- 172.21.15.16:0/1583119607 wait complete. 2024-09-19T07:48:32.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:48:32.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.328+0000 7ff222f4b640 1 Processor -- start 2024-09-19T07:48:32.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.328+0000 7ff222f4b640 1 -- start start 2024-09-19T07:48:32.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.330+0000 7ff222f4b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff21c0c5800 0x7ff21c0ad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:32.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.330+0000 7ff222f4b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff21c06fac0 0x7ff21c16eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:32.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.330+0000 7ff222f4b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff21c0c6f50 con 0x7ff21c16e7e0 2024-09-19T07:48:32.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.330+0000 7ff222f4b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff21c06ff80 con 0x7ff21c0c5800 2024-09-19T07:48:32.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.330+0000 7ff222f4b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff21c06e450 con 0x7ff21c06fac0 2024-09-19T07:48:32.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.330+0000 7ff21bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff21c0c5800 0x7ff21c0ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:32.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.330+0000 7ff220ce7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff21c16e7e0 0x7ff21c0c50f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40982/0 (socket says 172.21.15.16:40982) 2024-09-19T07:48:32.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.330+0000 7ff220ce7640 1 -- 172.21.15.16:0/3092035252 learned_addr learned my addr 172.21.15.16:0/3092035252 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:32.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.330+0000 7ff2214e8640 1 --2- 172.21.15.16:0/3092035252 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff21c06fac0 0x7ff21c16eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:32.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.330+0000 7ff21b7fe640 1 -- 172.21.15.16:0/3092035252 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3033194705 0 0) 0x7ff21c0c6f50 con 0x7ff21c16e7e0 2024-09-19T07:48:32.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.341+0000 7ff21b7fe640 1 -- 172.21.15.16:0/3092035252 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff208003500 con 0x7ff21c16e7e0 2024-09-19T07:48:32.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.341+0000 7ff21bfff640 1 -- 172.21.15.16:0/3092035252 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff21c06fac0 msgr2=0x7ff21c16eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:32.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.341+0000 7ff21bfff640 1 --2- 172.21.15.16:0/3092035252 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff21c06fac0 0x7ff21c16eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.341+0000 7ff21bfff640 1 -- 172.21.15.16:0/3092035252 >> v1:172.21.15.16:6789/0 conn(0x7ff21c16e7e0 legacy=0x7ff21c0c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:32.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.341+0000 7ff21b7fe640 1 -- 172.21.15.16:0/3092035252 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2627490908 0 0) 0x7ff208003500 con 0x7ff21c16e7e0 2024-09-19T07:48:32.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.341+0000 7ff21bfff640 1 -- 172.21.15.16:0/3092035252 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21c06e0e0 con 0x7ff21c0c5800 2024-09-19T07:48:32.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.342+0000 7ff21bfff640 1 --2- 172.21.15.16:0/3092035252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21c0c5800 0x7ff21c0ad160 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff20400be30 tx=0x7ff2040092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58186987f5107a46 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:32.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.342+0000 7ff21b7fe640 1 -- 172.21.15.16:0/3092035252 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204036070 con 0x7ff21c0c5800 2024-09-19T07:48:32.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.342+0000 7ff21b7fe640 1 -- 172.21.15.16:0/3092035252 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 46+0+0 (secure 0 0 0) 0x7ff20400b2e0 con 0x7ff21c0c5800 2024-09-19T07:48:32.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.342+0000 7ff21b7fe640 1 -- 172.21.15.16:0/3092035252 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c001670 con 0x7ff21c0c5800 2024-09-19T07:48:32.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.343+0000 7ff222f4b640 1 -- 172.21.15.16:0/3092035252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21c0c5800 msgr2=0x7ff21c0ad160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:32.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.343+0000 7ff222f4b640 1 --2- 172.21.15.16:0/3092035252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21c0c5800 0x7ff21c0ad160 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff20400be30 tx=0x7ff2040092f0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.343+0000 7ff222f4b640 1 -- 172.21.15.16:0/3092035252 shutdown_connections 2024-09-19T07:48:32.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.343+0000 7ff222f4b640 1 --2- 172.21.15.16:0/3092035252 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff21c06fac0 0x7ff21c16eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.343+0000 7ff222f4b640 1 --2- 172.21.15.16:0/3092035252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21c0c5800 0x7ff21c0ad160 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.343+0000 7ff222f4b640 1 -- 172.21.15.16:0/3092035252 >> 172.21.15.16:0/3092035252 conn(0x7ff21c0b1e80 msgr2=0x7ff21c0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:32.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.343+0000 7ff222f4b640 1 -- 172.21.15.16:0/3092035252 shutdown_connections 2024-09-19T07:48:32.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.343+0000 7ff222f4b640 1 -- 172.21.15.16:0/3092035252 wait complete. 2024-09-19T07:48:32.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.344+0000 7ff222f4b640 1 Processor -- start 2024-09-19T07:48:32.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.344+0000 7ff222f4b640 1 -- start start 2024-09-19T07:48:32.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.345+0000 7ff222f4b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff21c06fac0 0x7ff21c200fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:32.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.345+0000 7ff222f4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21c0c5800 0x7ff21c0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:32.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.345+0000 7ff222f4b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff21c16e7e0 0x7ff21c0b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:32.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.345+0000 7ff222f4b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c0996a0 con 0x7ff21c0c5800 2024-09-19T07:48:32.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.345+0000 7ff222f4b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c0afbc0 con 0x7ff21c06fac0 2024-09-19T07:48:32.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.345+0000 7ff222f4b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff21c099820 con 0x7ff21c16e7e0 2024-09-19T07:48:32.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.345+0000 7ff21bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21c0c5800 0x7ff21c0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:32.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.345+0000 7ff21bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21c0c5800 0x7ff21c0cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57736/0 (socket says 172.21.15.16:57736) 2024-09-19T07:48:32.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.345+0000 7ff21bfff640 1 -- 172.21.15.16:0/1152390202 learned_addr learned my addr 172.21.15.16:0/1152390202 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:32.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.345+0000 7ff220ce7640 1 --2- 172.21.15.16:0/1152390202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff21c06fac0 0x7ff21c200fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:32.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.345+0000 7ff2214e8640 1 --2- 172.21.15.16:0/1152390202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff21c16e7e0 0x7ff21c0b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:32.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.345+0000 7ff21bfff640 1 -- 172.21.15.16:0/1152390202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff21c16e7e0 msgr2=0x7ff21c0b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:32.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.345+0000 7ff21bfff640 1 --2- 172.21.15.16:0/1152390202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff21c16e7e0 0x7ff21c0b53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.345+0000 7ff21bfff640 1 -- 172.21.15.16:0/1152390202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff21c06fac0 msgr2=0x7ff21c200fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:32.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.345+0000 7ff21bfff640 1 --2- 172.21.15.16:0/1152390202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff21c06fac0 0x7ff21c200fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.345+0000 7ff21bfff640 1 -- 172.21.15.16:0/1152390202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21c0c7550 con 0x7ff21c0c5800 2024-09-19T07:48:32.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.346+0000 7ff21bfff640 1 --2- 172.21.15.16:0/1152390202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21c0c5800 0x7ff21c0cc6b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff20402abd0 tx=0x7ff204004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:32.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.346+0000 7ff2197fa640 1 -- 172.21.15.16:0/1152390202 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204036040 con 0x7ff21c0c5800 2024-09-19T07:48:32.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.346+0000 7ff2197fa640 1 -- 172.21.15.16:0/1152390202 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 46+0+0 (secure 0 0 0) 0x7ff204003bb0 con 0x7ff21c0c5800 2024-09-19T07:48:32.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.346+0000 7ff2197fa640 1 -- 172.21.15.16:0/1152390202 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20400b660 con 0x7ff21c0c5800 2024-09-19T07:48:32.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.346+0000 7ff222f4b640 1 -- 172.21.15.16:0/1152390202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21c0c7830 con 0x7ff21c0c5800 2024-09-19T07:48:32.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.346+0000 7ff222f4b640 1 -- 172.21.15.16:0/1152390202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff21c0c7df0 con 0x7ff21c0c5800 2024-09-19T07:48:32.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.347+0000 7ff222f4b640 1 -- 172.21.15.16:0/1152390202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff21c0ad160 con 0x7ff21c0c5800 2024-09-19T07:48:32.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.347+0000 7ff2197fa640 1 -- 172.21.15.16:0/1152390202 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff204004030 con 0x7ff21c0c5800 2024-09-19T07:48:32.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.348+0000 7ff2197fa640 1 --2- 172.21.15.16:0/1152390202 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff1fc06c2b0 0x7ff1fc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:32.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.348+0000 7ff2197fa640 1 -- 172.21.15.16:0/1152390202 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6202+0+0 (secure 0 0 0) 0x7ff2040ac870 con 0x7ff21c0c5800 2024-09-19T07:48:32.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.348+0000 7ff220ce7640 1 --2- 172.21.15.16:0/1152390202 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff1fc06c2b0 0x7ff1fc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:32.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.349+0000 7ff220ce7640 1 --2- 172.21.15.16:0/1152390202 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff1fc06c2b0 0x7ff1fc06e770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff20c005880 tx=0x7ff20c0057f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:32.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.353+0000 7ff2197fa640 1 -- 172.21.15.16:0/1152390202 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff20407a710 con 0x7ff21c0c5800 2024-09-19T07:48:32.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.575+0000 7ff222f4b640 1 -- 172.21.15.16:0/1152390202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff21c06ce20 con 0x7ff21c0c5800 2024-09-19T07:48:32.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.576+0000 7ff2197fa640 1 -- 172.21.15.16:0/1152390202 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5224 (secure 0 0 0) 0x7ff20407ea10 con 0x7ff21c0c5800 2024-09-19T07:48:32.578 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:32.578 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-19T07:45:59:637528+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/3536366465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3536366465},{"type":"v1","addr":"172.21.15.146:6836","nonce":3536366465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3612374586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3612374586},{"type":"v1","addr":"172.21.15.146:6837","nonce":3612374586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5337,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/781070730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":781070730},{"type":"v1","addr":"172.21.15.16:6833","nonce":781070730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:45:10.494498+0000","modified":"2024-09-19T07:45:59.181557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5340":{"gid":5340,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.16:6835/642332950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":642332950},{"type":"v1","addr":"172.21.15.16:6835","nonce":642332950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5340,"qdb_cluster":[5340]},"id":3}]} 2024-09-19T07:48:32.578 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-19T07:48:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.578+0000 7ff222f4b640 1 -- 172.21.15.16:0/1152390202 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff1fc06c2b0 msgr2=0x7ff1fc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.578+0000 7ff222f4b640 1 --2- 172.21.15.16:0/1152390202 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff1fc06c2b0 0x7ff1fc06e770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff20c005880 tx=0x7ff20c0057f0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.578+0000 7ff222f4b640 1 -- 172.21.15.16:0/1152390202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21c0c5800 msgr2=0x7ff21c0cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.578+0000 7ff222f4b640 1 --2- 172.21.15.16:0/1152390202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21c0c5800 0x7ff21c0cc6b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff20402abd0 tx=0x7ff204004a40 comp rx=0 tx=0).stop 2024-09-19T07:48:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.578+0000 7ff222f4b640 1 -- 172.21.15.16:0/1152390202 shutdown_connections 2024-09-19T07:48:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.578+0000 7ff222f4b640 1 --2- 172.21.15.16:0/1152390202 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff1fc06c2b0 0x7ff1fc06e770 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.578+0000 7ff222f4b640 1 --2- 172.21.15.16:0/1152390202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff21c16e7e0 0x7ff21c0b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.578+0000 7ff222f4b640 1 --2- 172.21.15.16:0/1152390202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21c0c5800 0x7ff21c0cc6b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.578+0000 7ff222f4b640 1 --2- 172.21.15.16:0/1152390202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff21c06fac0 0x7ff21c200fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.578+0000 7ff222f4b640 1 -- 172.21.15.16:0/1152390202 >> 172.21.15.16:0/1152390202 conn(0x7ff21c0b1e80 msgr2=0x7ff21c0b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.578+0000 7ff222f4b640 1 -- 172.21.15.16:0/1152390202 shutdown_connections 2024-09-19T07:48:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.578+0000 7ff222f4b640 1 -- 172.21.15.16:0/1152390202 wait complete. 2024-09-19T07:48:32.591 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-19T07:48:32.710 INFO:teuthology.orchestra.run.smithi016.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":10950,"msgr_send_messages":10757,"msgr_recv_bytes":2797816,"msgr_send_bytes":69705848,"msgr_created_connections":8,"msgr_active_connections":3,"msgr_running_total_time":0.801247427,"msgr_running_send_time":0.357595549,"msgr_running_recv_time":0.148153826,"msgr_running_fast_dispatch_time":0.242453589,"msgr_send_messages_queue_lat":{"avgcount":10756,"sum":1.692809362,"avgtime":0.000157382},"msgr_handle_ack_lat":{"avgcount":221,"sum":0.000027198,"avgtime":0.000000123},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":11111,"msgr_send_messages":11104,"msgr_recv_bytes":2950060,"msgr_send_bytes":91120911,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.865370343,"msgr_running_send_time":0.416387296,"msgr_running_recv_time":0.151726092,"msgr_running_fast_dispatch_time":0.249318127,"msgr_send_messages_queue_lat":{"avgcount":11103,"sum":2.025980862,"avgtime":0.000182471},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":133216,"msgr_send_encrypted_bytes":49424},"AsyncMessenger::Worker-2":{"msgr_recv_messages":160015,"msgr_send_messages":180829,"msgr_recv_bytes":47751764,"msgr_send_bytes":103695235,"msgr_created_connections":10,"msgr_active_connections":4,"msgr_running_total_time":9.384393898,"msgr_running_send_time":4.509163842,"msgr_running_recv_time":6.784093231,"msgr_running_fast_dispatch_time":0.187992642,"msgr_send_messages_queue_lat":{"avgcount":180828,"sum":3.471506443,"avgtime":0.000019197},"msgr_handle_ack_lat":{"avgcount":327315,"sum":0.040611047,"avgtime":0.000000124},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":166256},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":49358,"sum":29.085262880,"avgtime":0.000589271}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000030245,"avgtime":0.000015122}},"mds":{"request":70746,"reply":70745,"reply_latency":{"avgcount":70745,"sum":64.648460994,"avgtime":0.000913823},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":21,"dir_split":1,"dir_merge":0,"inodes":34511,"inodes_top":0,"inodes_bottom":34511,"inodes_pin_tail":0,"inodes_pinned":32471,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":106985,"traverse_hit":34519,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":34450,"root_rbytes":150030,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":7,"ceph_cap_op_grant":34507,"ceph_cap_op_trunc":5,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":58160,"handle_client_caps_dirty":34505,"handle_client_cap_release":0,"process_request_cap_release":34502},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":757,"recovery_completed":757,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":1,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":93604,"evex":2143,"evtrm":1,"ev":93603,"evexg":0,"evexd":2142,"segadd":132,"segex":4,"segtrm":1,"seg":131,"segmjr":8,"segexg":0,"segexd":3,"expos":4194352,"wrpos":187354659,"rdpos":0,"jlat":{"avgcount":24543,"sum":33.636879075,"avgtime":0.001370528},"replayed":0},"mds_mem":{"ino":34514,"ino+":34514,"ino-":0,"dir":20,"dir+":21,"dir-":1,"dn":34511,"dn+":34511,"dn-":0,"cap":1,"cap+":34504,"cap-":34503,"rss":234884,"heap":231788},"mds_server":{"dispatch_client_request":70753,"dispatch_server_request":0,"handle_client_request":70746,"handle_client_session":24428,"handle_peer_request":0,"req_create_latency":{"avgcount":34500,"sum":51.922976713,"avgtime":0.001505013},"req_getattr_latency":{"avgcount":4,"sum":0.000935905,"avgtime":0.000233976},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":36234,"sum":12.711301073,"avgtime":0.000350811},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":1,"sum":0.002838002,"avgtime":0.002838002},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":5,"sum":0.010010025,"avgtime":0.002002005},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000399276,"avgtime":0.000399276},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":3,"session_remove":2,"sessions_open":1,"sessions_stale":0,"total_load":31875,"average_load":15937,"avg_session_uptime":192,"mdthresh_evicted":3},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":8623807,"buffer_anon_items":2181,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5784,"osdmap_items":94,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":100336264,"mds_co_items":662097,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":28236,"op_send_bytes":197267834,"op_resend":31,"op_reply":28236,"op_latency":{"avgcount":28236,"sum":313.937638303,"avgtime":0.011118346},"op_inflight":0,"oplen_avg":{"avgcount":28236,"sum":32907},"op":28236,"op_r":757,"op_w":27479,"op_rmw":0,"op_pg":0,"osdop_stat":757,"osdop_create":2146,"osdop_read":0,"osdop_write":24543,"osdop_writefull":21,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":1,"osdop_delete":549,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":4292,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":5,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":59,"map_full":0,"map_inc":7,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":410,"omap_rd":0,"omap_del":183},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":182071,"get_sum":37105906,"get_or_fail_fail":0,"get_or_fail_success":182071,"take":0,"take_sum":0,"put":182071,"put_sum":37105906,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":28236,"take_sum":197267834,"put":28236,"put_sum":197267834,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":28236,"take_sum":28236,"put":28236,"put_sum":28236,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x563acbe2e500":{"val":0,"max":3758096384,"get_started":0,"get":93604,"get_sum":183160355,"get_or_fail_fail":0,"get_or_fail_success":93604,"take":0,"take_sum":0,"put":24543,"put_sum":183160355,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-19T07:48:32.712 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 3, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 221, "avgtime": 1.23e-07, "sum": 2.7198e-05 }, "msgr_recv_bytes": 2797816, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 10950, "msgr_running_fast_dispatch_time": 0.242453589, "msgr_running_recv_time": 0.148153826, "msgr_running_send_time": 0.357595549, "msgr_running_total_time": 0.801247427, "msgr_send_bytes": 69705848, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 10757, "msgr_send_messages_queue_lat": { "avgcount": 10756, "avgtime": 0.000157382, "sum": 1.692809362 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 2950060, "msgr_recv_encrypted_bytes": 133216, "msgr_recv_messages": 11111, "msgr_running_fast_dispatch_time": 0.249318127, "msgr_running_recv_time": 0.151726092, "msgr_running_send_time": 0.416387296, "msgr_running_total_time": 0.865370343, "msgr_send_bytes": 91120911, "msgr_send_encrypted_bytes": 49424, "msgr_send_messages": 11104, "msgr_send_messages_queue_lat": { "avgcount": 11103, "avgtime": 0.000182471, "sum": 2.025980862 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 4, "msgr_created_connections": 10, "msgr_handle_ack_lat": { "avgcount": 327315, "avgtime": 1.24e-07, "sum": 0.040611047 }, "msgr_recv_bytes": 47751764, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 160015, "msgr_running_fast_dispatch_time": 0.187992642, "msgr_running_recv_time": 6.784093231, "msgr_running_send_time": 4.509163842, "msgr_running_total_time": 9.384393898, "msgr_send_bytes": 103695235, "msgr_send_encrypted_bytes": 166256, "msgr_send_messages": 180829, "msgr_send_messages_queue_lat": { "avgcount": 180828, "avgtime": 1.9197e-05, "sum": 3.471506443 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 49358, "avgtime": 0.000589271, "sum": 29.08526288 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 1.5122e-05, "sum": 3.0245e-05 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 34507, "ceph_cap_op_revoke": 7, "ceph_cap_op_trunc": 5, "dir_commit": 21, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 1, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 58160, "handle_client_caps_dirty": 34505, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 34511, "inodes_bottom": 34511, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 32471, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 34502, "q": 0, "reply": 70745, "reply_latency": { "avgcount": 70745, "avgtime": 0.000913823, "sum": 64.648460994 }, "request": 70746, "root_rbytes": 150030, "root_rfiles": 34450, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 106985, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 34519, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 1, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 757, "recovery_started": 757, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 93603, "evadd": 93604, "evex": 2143, "evexd": 2142, "evexg": 0, "evlrg": 0, "evtrm": 1, "expos": 4194352, "jlat": { "avgcount": 24543, "avgtime": 0.001370528, "sum": 33.636879075 }, "rdpos": 0, "replayed": 0, "seg": 131, "segadd": 132, "segex": 4, "segexd": 3, "segexg": 0, "segmjr": 8, "segtrm": 1, "wrpos": 187354659 }, "mds_mem": { "cap": 1, "cap+": 34504, "cap-": 34503, "dir": 20, "dir+": 21, "dir-": 1, "dn": 34511, "dn+": 34511, "dn-": 0, "heap": 231788, "ino": 34514, "ino+": 34514, "ino-": 0, "rss": 234884 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 70753, "dispatch_server_request": 0, "handle_client_request": 70746, "handle_client_session": 24428, "handle_peer_request": 0, "req_create_latency": { "avgcount": 34500, "avgtime": 0.001505013, "sum": 51.922976713 }, "req_getattr_latency": { "avgcount": 4, "avgtime": 0.000233976, "sum": 0.000935905 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 36234, "avgtime": 0.000350811, "sum": 12.711301073 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 1, "avgtime": 0.002838002, "sum": 0.002838002 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 5, "avgtime": 0.002002005, "sum": 0.010010025 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000399276, "sum": 0.000399276 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 15937, "avg_session_uptime": 192, "mdthresh_evicted": 3, "session_add": 3, "session_count": 1, "session_remove": 2, "sessions_open": 1, "sessions_stale": 0, "total_load": 31875 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 8623807, "buffer_anon_items": 2181, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 100336264, "mds_co_items": 662097, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5784, "osdmap_items": 94, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 59, "map_full": 0, "map_inc": 7, "omap_del": 183, "omap_rd": 0, "omap_wr": 410, "op": 28236, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 28236, "avgtime": 0.011118346, "sum": 313.937638303 }, "op_pg": 0, "op_r": 757, "op_reply": 28236, "op_resend": 31, "op_rmw": 0, "op_send": 28236, "op_send_bytes": 197267834, "op_w": 27479, "oplen_avg": { "avgcount": 28236, "sum": 32907 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 2146, "osdop_delete": 549, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 5, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 4292, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 757, "osdop_truncate": 1, "osdop_watch": 0, "osdop_write": 24543, "osdop_writefull": 21, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 182071, "get_or_fail_fail": 0, "get_or_fail_success": 182071, "get_started": 0, "get_sum": 37105906, "max": 104857600, "put": 182071, "put_sum": 37105906, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 28236, "put_sum": 197267834, "take": 28236, "take_sum": 197267834, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 28236, "put_sum": 28236, "take": 28236, "take_sum": 28236, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x563acbe2e500": { "get": 93604, "get_or_fail_fail": 0, "get_or_fail_success": 93604, "get_started": 0, "get_sum": 183160355, "max": 3758096384, "put": 24543, "put_sum": 183160355, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-19T07:48:32.712 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set client client_inject_fixed_oldest_tid false 2024-09-19T07:48:32.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f8745142640 1 Processor -- start 2024-09-19T07:48:32.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f8745142640 1 -- start start 2024-09-19T07:48:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f8745142640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f87400d4370 0x7f87400d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f8745142640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87400d1880 con 0x7f874015e2a0 2024-09-19T07:48:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f8745142640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87400d1b60 con 0x7f874015e680 2024-09-19T07:48:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f8745142640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f87400cf2d0 con 0x7f87400d4370 2024-09-19T07:48:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f873ed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f874015e2a0 0x7f874006d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56488/0 (socket says 172.21.15.16:56488) 2024-09-19T07:48:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f873ed9d640 1 -- 172.21.15.16:0/2208609188 learned_addr learned my addr 172.21.15.16:0/2208609188 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f873dd9b640 1 -- 172.21.15.16:0/2208609188 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3935574178 0 0) 0x7f87400d1880 con 0x7f874015e2a0 2024-09-19T07:48:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f873dd9b640 1 -- 172.21.15.16:0/2208609188 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8728003520 con 0x7f874015e2a0 2024-09-19T07:48:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f873f59e640 1 --2- 172.21.15.16:0/2208609188 >> v2:172.21.15.146:3300/0 conn(0x7f87400d4370 0x7f87400d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f873dd9b640 1 -- 172.21.15.16:0/2208609188 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1673833388 0 0) 0x7f8728003520 con 0x7f874015e2a0 2024-09-19T07:48:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f873dd9b640 1 -- 172.21.15.16:0/2208609188 >> v2:172.21.15.146:3300/0 conn(0x7f87400d4370 msgr2=0x7f87400d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f873dd9b640 1 --2- 172.21.15.16:0/2208609188 >> v2:172.21.15.146:3300/0 conn(0x7f87400d4370 0x7f87400d4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f873dd9b640 1 -- 172.21.15.16:0/2208609188 >> v1:172.21.15.146:6789/0 conn(0x7f874015e680 legacy=0x7f87400cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f873dd9b640 1 -- 172.21.15.16:0/2208609188 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87400cf5b0 con 0x7f874015e2a0 2024-09-19T07:48:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f873dd9b640 1 -- 172.21.15.16:0/2208609188 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f8730001670 con 0x7f874015e2a0 2024-09-19T07:48:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f873dd9b640 1 -- 172.21.15.16:0/2208609188 >> v1:172.21.15.16:6789/0 conn(0x7f874015e2a0 legacy=0x7f874006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f873dd9b640 1 --2- 172.21.15.16:0/2208609188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87280039b0 0x7f8728003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f873dd9b640 1 --2- 172.21.15.16:0/2208609188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8728004800 0x7f8728004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f873dd9b640 1 --2- 172.21.15.16:0/2208609188 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87280051c0 0x7f872800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f873dd9b640 1 -- 172.21.15.16:0/2208609188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87400cf2d0 con 0x7f8728004800 2024-09-19T07:48:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f873dd9b640 1 -- 172.21.15.16:0/2208609188 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f872800c060 con 0x7f87280039b0 2024-09-19T07:48:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.818+0000 7f873dd9b640 1 -- 172.21.15.16:0/2208609188 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f872800c260 con 0x7f87280051c0 2024-09-19T07:48:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.819+0000 7f873e59c640 1 --2- 172.21.15.16:0/2208609188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8728004800 0x7f8728004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.819+0000 7f873ed9d640 1 --2- 172.21.15.16:0/2208609188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87280039b0 0x7f8728003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.819+0000 7f873f59e640 1 --2- 172.21.15.16:0/2208609188 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87280051c0 0x7f872800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.819+0000 7f873e59c640 1 -- 172.21.15.16:0/2208609188 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87280051c0 msgr2=0x7f872800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.819+0000 7f873e59c640 1 --2- 172.21.15.16:0/2208609188 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87280051c0 0x7f872800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.819+0000 7f873e59c640 1 -- 172.21.15.16:0/2208609188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87280039b0 msgr2=0x7f8728003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.819+0000 7f873e59c640 1 --2- 172.21.15.16:0/2208609188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87280039b0 0x7f8728003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.819+0000 7f873f59e640 1 -- 172.21.15.16:0/2208609188 reap_dead start 2024-09-19T07:48:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.819+0000 7f873e59c640 1 -- 172.21.15.16:0/2208609188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8728003520 con 0x7f8728004800 2024-09-19T07:48:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.820+0000 7f873e59c640 1 --2- 172.21.15.16:0/2208609188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8728004800 0x7f8728004c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8734001720 tx=0x7f8728005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866e13eeb00723e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.820+0000 7f873dd9b640 1 -- 172.21.15.16:0/2208609188 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734001370 con 0x7f8728004800 2024-09-19T07:48:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.820+0000 7f873dd9b640 1 -- 172.21.15.16:0/2208609188 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 46+0+0 (secure 0 0 0) 0x7f87340052d0 con 0x7f8728004800 2024-09-19T07:48:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.820+0000 7f873dd9b640 1 -- 172.21.15.16:0/2208609188 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873402abc0 con 0x7f8728004800 2024-09-19T07:48:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.820+0000 7f8745142640 1 -- 172.21.15.16:0/2208609188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8728004800 msgr2=0x7f8728004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.820+0000 7f8745142640 1 --2- 172.21.15.16:0/2208609188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8728004800 0x7f8728004c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8734001720 tx=0x7f8728005830 comp rx=0 tx=0).stop 2024-09-19T07:48:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.820+0000 7f8745142640 1 -- 172.21.15.16:0/2208609188 shutdown_connections 2024-09-19T07:48:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.820+0000 7f8745142640 1 --2- 172.21.15.16:0/2208609188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8728004800 0x7f8728004c80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.820+0000 7f8745142640 1 -- 172.21.15.16:0/2208609188 >> 172.21.15.16:0/2208609188 conn(0x7f874009d760 msgr2=0x7f874008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.820+0000 7f8745142640 1 -- 172.21.15.16:0/2208609188 shutdown_connections 2024-09-19T07:48:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.820+0000 7f8745142640 1 -- 172.21.15.16:0/2208609188 wait complete. 2024-09-19T07:48:32.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f8745142640 1 Processor -- start 2024-09-19T07:48:32.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f8745142640 1 -- start start 2024-09-19T07:48:32.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f8745142640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8728004800 0x7f87400c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:32.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f8745142640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87400cd960 0x7f87400cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f8745142640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87401f3c40 0x7f87401f40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f8745142640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87400d49a0 con 0x7f87400cd960 2024-09-19T07:48:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f8745142640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f87400d4820 con 0x7f8728004800 2024-09-19T07:48:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f8745142640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f87400d4b20 con 0x7f87401f3c40 2024-09-19T07:48:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f873e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87400cd960 0x7f87400cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f873e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87400cd960 0x7f87400cba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36044/0 (socket says 172.21.15.16:36044) 2024-09-19T07:48:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f873e59c640 1 -- 172.21.15.16:0/1744644622 learned_addr learned my addr 172.21.15.16:0/1744644622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f873ed9d640 1 --2- 172.21.15.16:0/1744644622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8728004800 0x7f87400c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f873f59e640 1 --2- 172.21.15.16:0/1744644622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87401f3c40 0x7f87401f40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f873e59c640 1 -- 172.21.15.16:0/1744644622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87401f3c40 msgr2=0x7f87401f40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f873e59c640 1 --2- 172.21.15.16:0/1744644622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87401f3c40 0x7f87401f40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f873e59c640 1 -- 172.21.15.16:0/1744644622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8728004800 msgr2=0x7f87400c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f873e59c640 1 --2- 172.21.15.16:0/1744644622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8728004800 0x7f87400c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f873e59c640 1 -- 172.21.15.16:0/1744644622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87400c9790 con 0x7f87400cd960 2024-09-19T07:48:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f873ed9d640 1 --2- 172.21.15.16:0/1744644622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8728004800 0x7f87400c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f873e59c640 1 --2- 172.21.15.16:0/1744644622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87400cd960 0x7f87400cba30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8734001840 tx=0x7f8734001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f87277fe640 1 -- 172.21.15.16:0/1744644622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873403f020 con 0x7f87400cd960 2024-09-19T07:48:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f87277fe640 1 -- 172.21.15.16:0/1744644622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 46+0+0 (secure 0 0 0) 0x7f873402be50 con 0x7f87400cd960 2024-09-19T07:48:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f8745142640 1 -- 172.21.15.16:0/1744644622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87400c9a70 con 0x7f87400cd960 2024-09-19T07:48:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f87277fe640 1 -- 172.21.15.16:0/1744644622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734001370 con 0x7f87400cd960 2024-09-19T07:48:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.823+0000 7f8745142640 1 -- 172.21.15.16:0/1744644622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87400ca030 con 0x7f87400cd960 2024-09-19T07:48:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.824+0000 7f8745142640 1 -- 172.21.15.16:0/1744644622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f874006d420 con 0x7f87400cd960 2024-09-19T07:48:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.825+0000 7f87277fe640 1 -- 172.21.15.16:0/1744644622 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8734042020 con 0x7f87400cd960 2024-09-19T07:48:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.825+0000 7f87277fe640 1 --2- 172.21.15.16:0/1744644622 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f872c06c2e0 0x7f872c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.825+0000 7f87277fe640 1 -- 172.21.15.16:0/1744644622 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6202+0+0 (secure 0 0 0) 0x7f87340ab590 con 0x7f87400cd960 2024-09-19T07:48:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.826+0000 7f873ed9d640 1 --2- 172.21.15.16:0/1744644622 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f872c06c2e0 0x7f872c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.826+0000 7f873ed9d640 1 --2- 172.21.15.16:0/1744644622 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f872c06c2e0 0x7f872c06e7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8740051220 tx=0x7f8730009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:32.831+0000 7f87277fe640 1 -- 172.21.15.16:0/1744644622 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8734049020 con 0x7f87400cd960 2024-09-19T07:48:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.006+0000 7f8745142640 1 -- 172.21.15.16:0/1744644622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=client_inject_fixed_oldest_tid}] v 0) -- 0x7f87400a2010 con 0x7f87400cd960 2024-09-19T07:48:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.009+0000 7f87277fe640 1 -- 172.21.15.16:0/1744644622 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_inject_fixed_oldest_tid}]=0 v20)=0 v20) ==== 135+0+0 (secure 0 0 0) 0x7f8734079a80 con 0x7f87400cd960 2024-09-19T07:48:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.009+0000 7f87277fe640 1 -- 172.21.15.16:0/1744644622 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7f8734079380 con 0x7f87400cd960 2024-09-19T07:48:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.012+0000 7f8745142640 1 -- 172.21.15.16:0/1744644622 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f872c06c2e0 msgr2=0x7f872c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.012+0000 7f8745142640 1 --2- 172.21.15.16:0/1744644622 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f872c06c2e0 0x7f872c06e7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8740051220 tx=0x7f8730009040 comp rx=0 tx=0).stop 2024-09-19T07:48:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.012+0000 7f8745142640 1 -- 172.21.15.16:0/1744644622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87400cd960 msgr2=0x7f87400cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.012+0000 7f8745142640 1 --2- 172.21.15.16:0/1744644622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87400cd960 0x7f87400cba30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8734001840 tx=0x7f8734001120 comp rx=0 tx=0).stop 2024-09-19T07:48:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.012+0000 7f8745142640 1 -- 172.21.15.16:0/1744644622 shutdown_connections 2024-09-19T07:48:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.012+0000 7f8745142640 1 --2- 172.21.15.16:0/1744644622 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f872c06c2e0 0x7f872c06e7a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.012+0000 7f8745142640 1 --2- 172.21.15.16:0/1744644622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87401f3c40 0x7f87401f40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.012+0000 7f8745142640 1 --2- 172.21.15.16:0/1744644622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87400cd960 0x7f87400cba30 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.012+0000 7f8745142640 1 --2- 172.21.15.16:0/1744644622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8728004800 0x7f87400c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.012+0000 7f8745142640 1 -- 172.21.15.16:0/1744644622 >> 172.21.15.16:0/1744644622 conn(0x7f874009d760 msgr2=0x7f874015fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.012+0000 7f8745142640 1 -- 172.21.15.16:0/1744644622 shutdown_connections 2024-09-19T07:48:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.012+0000 7f8745142640 1 -- 172.21.15.16:0/1744644622 wait complete. 2024-09-19T07:48:33.023 INFO:tasks.cephfs.mount:Cleaning up killed connection on ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:48:33.024 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:48:33.024 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:48:33.040 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:48:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.185+0000 7fa5eb227640 1 Processor -- start 2024-09-19T07:48:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.185+0000 7fa5eb227640 1 -- start start 2024-09-19T07:48:33.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.186+0000 7fa5eb227640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa5e415a7b0 0x7fa5e40c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:33.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.186+0000 7fa5eb227640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e40cde40 0x7fa5e40cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:33.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.186+0000 7fa5eb227640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa5e40cce70 0x7fa5e40c6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:33.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.186+0000 7fa5eb227640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa5e40b4870 con 0x7fa5e40cce70 2024-09-19T07:48:33.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.186+0000 7fa5eb227640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa5e40b4540 con 0x7fa5e415a7b0 2024-09-19T07:48:33.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.186+0000 7fa5eb227640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e40cf9b0 con 0x7fa5e40cde40 2024-09-19T07:48:33.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.186+0000 7fa5e97c4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa5e40cce70 0x7fa5e40c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:33.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.186+0000 7fa5e97c4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa5e40cce70 0x7fa5e40c6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36060/0 (socket says 172.21.15.16:36060) 2024-09-19T07:48:33.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.186+0000 7fa5e97c4640 1 -- 172.21.15.16:0/3570035955 learned_addr learned my addr 172.21.15.16:0/3570035955 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:33.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.187+0000 7fa5e8fc3640 1 --2- 172.21.15.16:0/3570035955 >> v2:172.21.15.146:3300/0 conn(0x7fa5e415a7b0 0x7fa5e40c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:33.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.187+0000 7fa5e3fff640 1 --2- 172.21.15.16:0/3570035955 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e40cde40 0x7fa5e40cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:33.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.187+0000 7fa5e97c4640 1 -- 172.21.15.16:0/3570035955 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e40cde40 msgr2=0x7fa5e40cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:33.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.187+0000 7fa5e97c4640 1 --2- 172.21.15.16:0/3570035955 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e40cde40 0x7fa5e40cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.187+0000 7fa5e97c4640 1 -- 172.21.15.16:0/3570035955 >> v2:172.21.15.146:3300/0 conn(0x7fa5e415a7b0 msgr2=0x7fa5e40c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:33.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.187+0000 7fa5e97c4640 1 --2- 172.21.15.16:0/3570035955 >> v2:172.21.15.146:3300/0 conn(0x7fa5e415a7b0 0x7fa5e40c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.187+0000 7fa5e97c4640 1 -- 172.21.15.16:0/3570035955 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e40cfc90 con 0x7fa5e40cce70 2024-09-19T07:48:33.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.188+0000 7fa5e97c4640 1 --2- 172.21.15.16:0/3570035955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e40cce70 0x7fa5e40c6310 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40cdcd0 tx=0x7fa5dc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273c3706a790e6ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:33.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.188+0000 7fa5e37fe640 1 -- 172.21.15.16:0/3570035955 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc036070 con 0x7fa5e40cce70 2024-09-19T07:48:33.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.188+0000 7fa5e37fe640 1 -- 172.21.15.16:0/3570035955 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7fa5dc031600 con 0x7fa5e40cce70 2024-09-19T07:48:33.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.188+0000 7fa5e37fe640 1 -- 172.21.15.16:0/3570035955 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc0319d0 con 0x7fa5e40cce70 2024-09-19T07:48:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.188+0000 7fa5eb227640 1 -- 172.21.15.16:0/3570035955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e40cce70 msgr2=0x7fa5e40c6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.188+0000 7fa5eb227640 1 --2- 172.21.15.16:0/3570035955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e40cce70 0x7fa5e40c6310 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40cdcd0 tx=0x7fa5dc02a6a0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.188+0000 7fa5eb227640 1 -- 172.21.15.16:0/3570035955 shutdown_connections 2024-09-19T07:48:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.188+0000 7fa5eb227640 1 --2- 172.21.15.16:0/3570035955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e40cce70 0x7fa5e40c6310 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.188+0000 7fa5eb227640 1 --2- 172.21.15.16:0/3570035955 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e40cde40 0x7fa5e40cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.188+0000 7fa5eb227640 1 --2- 172.21.15.16:0/3570035955 >> v2:172.21.15.146:3300/0 conn(0x7fa5e415a7b0 0x7fa5e40c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.188+0000 7fa5eb227640 1 -- 172.21.15.16:0/3570035955 >> 172.21.15.16:0/3570035955 conn(0x7fa5e406bdf0 msgr2=0x7fa5e406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.188+0000 7fa5eb227640 1 -- 172.21.15.16:0/3570035955 shutdown_connections 2024-09-19T07:48:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.188+0000 7fa5eb227640 1 -- 172.21.15.16:0/3570035955 wait complete. 2024-09-19T07:48:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.189+0000 7fa5eb227640 1 Processor -- start 2024-09-19T07:48:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.190+0000 7fa5eb227640 1 -- start start 2024-09-19T07:48:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.190+0000 7fa5eb227640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e40cce70 0x7fa5e41f74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.190+0000 7fa5eb227640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5e40cde40 0x7fa5e41f79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.190+0000 7fa5eb227640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e415a7b0 0x7fa5e4154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.190+0000 7fa5eb227640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e40ca8c0 con 0x7fa5e40cce70 2024-09-19T07:48:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.190+0000 7fa5eb227640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e415ab90 con 0x7fa5e40cde40 2024-09-19T07:48:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.190+0000 7fa5eb227640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e40ce280 con 0x7fa5e415a7b0 2024-09-19T07:48:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.191+0000 7fa5e8fc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e40cce70 0x7fa5e41f74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.191+0000 7fa5e8fc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e40cce70 0x7fa5e41f74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36070/0 (socket says 172.21.15.16:36070) 2024-09-19T07:48:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.191+0000 7fa5e8fc3640 1 -- 172.21.15.16:0/6886682 learned_addr learned my addr 172.21.15.16:0/6886682 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.191+0000 7fa5e3fff640 1 --2- 172.21.15.16:0/6886682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5e40cde40 0x7fa5e41f79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.191+0000 7fa5e97c4640 1 --2- 172.21.15.16:0/6886682 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e415a7b0 0x7fa5e4154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.191+0000 7fa5e8fc3640 1 -- 172.21.15.16:0/6886682 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e415a7b0 msgr2=0x7fa5e4154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.191+0000 7fa5e8fc3640 1 --2- 172.21.15.16:0/6886682 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e415a7b0 0x7fa5e4154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.191+0000 7fa5e8fc3640 1 -- 172.21.15.16:0/6886682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5e40cde40 msgr2=0x7fa5e41f79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.191+0000 7fa5e8fc3640 1 --2- 172.21.15.16:0/6886682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5e40cde40 0x7fa5e41f79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.191+0000 7fa5e8fc3640 1 -- 172.21.15.16:0/6886682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e40af960 con 0x7fa5e40cce70 2024-09-19T07:48:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.191+0000 7fa5e3fff640 1 --2- 172.21.15.16:0/6886682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5e40cde40 0x7fa5e41f79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.191+0000 7fa5e8fc3640 1 --2- 172.21.15.16:0/6886682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e40cce70 0x7fa5e41f74b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400e9b0 tx=0x7fa5d400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.191+0000 7fa5e17fa640 1 -- 172.21.15.16:0/6886682 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4017070 con 0x7fa5e40cce70 2024-09-19T07:48:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.191+0000 7fa5e17fa640 1 -- 172.21.15.16:0/6886682 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7fa5d400c9a0 con 0x7fa5e40cce70 2024-09-19T07:48:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.191+0000 7fa5e17fa640 1 -- 172.21.15.16:0/6886682 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d400cd90 con 0x7fa5e40cce70 2024-09-19T07:48:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.191+0000 7fa5eb227640 1 -- 172.21.15.16:0/6886682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e4155900 con 0x7fa5e40cce70 2024-09-19T07:48:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.191+0000 7fa5eb227640 1 -- 172.21.15.16:0/6886682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e4156140 con 0x7fa5e40cce70 2024-09-19T07:48:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.192+0000 7fa5eb227640 1 -- 172.21.15.16:0/6886682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e40cc930 con 0x7fa5e40cce70 2024-09-19T07:48:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.193+0000 7fa5e17fa640 1 -- 172.21.15.16:0/6886682 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa5d401a070 con 0x7fa5e40cce70 2024-09-19T07:48:33.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.193+0000 7fa5e17fa640 1 --2- 172.21.15.16:0/6886682 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa5d006c2b0 0x7fa5d006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:33.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.194+0000 7fa5e17fa640 1 -- 172.21.15.16:0/6886682 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6202+0+0 (secure 0 0 0) 0x7fa5d408b530 con 0x7fa5e40cce70 2024-09-19T07:48:33.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.194+0000 7fa5e3fff640 1 --2- 172.21.15.16:0/6886682 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa5d006c2b0 0x7fa5d006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:33.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.194+0000 7fa5e3fff640 1 --2- 172.21.15.16:0/6886682 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa5d006c2b0 0x7fa5d006e770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa5e41f1ab0 tx=0x7fa5d8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:33.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.198+0000 7fa5e17fa640 1 -- 172.21.15.16:0/6886682 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa5e4155900 con 0x7fa5e40cce70 2024-09-19T07:48:33.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.386+0000 7fa5eb227640 1 -- 172.21.15.16:0/6886682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa5e406dea0 con 0x7fa5e40cce70 2024-09-19T07:48:33.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.386+0000 7fa5e17fa640 1 -- 172.21.15.16:0/6886682 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 1 entries v59) ==== 80+0+57 (secure 0 0 0) 0x7fa5d4059460 con 0x7fa5e40cce70 2024-09-19T07:48:33.387 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.1:0/2345385426 2024-09-19T08:48:30.430325+0000 2024-09-19T07:48:33.387 INFO:teuthology.orchestra.run.smithi016.stderr:listed 1 entries 2024-09-19T07:48:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.388+0000 7fa5eb227640 1 -- 172.21.15.16:0/6886682 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa5d006c2b0 msgr2=0x7fa5d006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.388+0000 7fa5eb227640 1 --2- 172.21.15.16:0/6886682 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa5d006c2b0 0x7fa5d006e770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa5e41f1ab0 tx=0x7fa5d8008040 comp rx=0 tx=0).stop 2024-09-19T07:48:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.388+0000 7fa5eb227640 1 -- 172.21.15.16:0/6886682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e40cce70 msgr2=0x7fa5e41f74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.388+0000 7fa5eb227640 1 --2- 172.21.15.16:0/6886682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e40cce70 0x7fa5e41f74b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400e9b0 tx=0x7fa5d400ee70 comp rx=0 tx=0).stop 2024-09-19T07:48:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.388+0000 7fa5eb227640 1 -- 172.21.15.16:0/6886682 shutdown_connections 2024-09-19T07:48:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.388+0000 7fa5eb227640 1 --2- 172.21.15.16:0/6886682 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa5d006c2b0 0x7fa5d006e770 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.388+0000 7fa5eb227640 1 --2- 172.21.15.16:0/6886682 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e415a7b0 0x7fa5e4154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.388+0000 7fa5eb227640 1 --2- 172.21.15.16:0/6886682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5e40cde40 0x7fa5e41f79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.388+0000 7fa5eb227640 1 --2- 172.21.15.16:0/6886682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e40cce70 0x7fa5e41f74b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.388+0000 7fa5eb227640 1 -- 172.21.15.16:0/6886682 >> 172.21.15.16:0/6886682 conn(0x7fa5e406bdf0 msgr2=0x7fa5e40cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.388+0000 7fa5eb227640 1 -- 172.21.15.16:0/6886682 shutdown_connections 2024-09-19T07:48:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.388+0000 7fa5eb227640 1 -- 172.21.15.16:0/6886682 wait complete. 2024-09-19T07:48:33.400 DEBUG:tasks.cephfs.mount:Force/lazy unmounting on client.0 2024-09-19T07:48:33.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:48:33.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount --lazy --force /home/ubuntu/cephtest/mnt.0 2024-09-19T07:48:33.465 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:48:33.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:48:33.466 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:48:33.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:48:33.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.628+0000 7f6a7ecc0640 1 Processor -- start 2024-09-19T07:48:33.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.628+0000 7f6a7ecc0640 1 -- start start 2024-09-19T07:48:33.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.629+0000 7f6a7ecc0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6a78141470 0x7f6a78141b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:33.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.629+0000 7f6a7ecc0640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6a7806ce70 0x7f6a7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:33.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.629+0000 7f6a7ecc0640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a78161f10 con 0x7f6a780d2130 2024-09-19T07:48:33.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.629+0000 7f6a7ecc0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f6a7806d840 con 0x7f6a78141470 2024-09-19T07:48:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.629+0000 7f6a7ecc0640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f6a78141850 con 0x7f6a7806ce70 2024-09-19T07:48:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.629+0000 7f6a7ca5c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6a78141470 0x7f6a78141b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.629+0000 7f6a7ca5c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6a78141470 0x7f6a78141b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36084/0 (socket says 172.21.15.16:36084) 2024-09-19T07:48:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.629+0000 7f6a7ca5c640 1 -- 172.21.15.16:0/1559261691 learned_addr learned my addr 172.21.15.16:0/1559261691 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.630+0000 7f6a77fff640 1 --2- 172.21.15.16:0/1559261691 >> v2:172.21.15.146:3300/0 conn(0x7f6a7806ce70 0x7f6a7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.630+0000 7f6a7ca5c640 1 -- 172.21.15.16:0/1559261691 >> v2:172.21.15.146:3300/0 conn(0x7f6a7806ce70 msgr2=0x7f6a7806d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.630+0000 7f6a7ca5c640 1 --2- 172.21.15.16:0/1559261691 >> v2:172.21.15.146:3300/0 conn(0x7f6a7806ce70 0x7f6a7806d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.630+0000 7f6a7ca5c640 1 -- 172.21.15.16:0/1559261691 >> v1:172.21.15.146:6789/0 conn(0x7f6a780d2130 legacy=0x7f6a780d2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.630+0000 7f6a7ca5c640 1 -- 172.21.15.16:0/1559261691 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a781621b0 con 0x7f6a78141470 2024-09-19T07:48:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.631+0000 7f6a7ca5c640 1 --2- 172.21.15.16:0/1559261691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a78141470 0x7f6a78141b70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6a780b8400 tx=0x7f6a6802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2e84d2896b7010 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.631+0000 7f6a777fe640 1 -- 172.21.15.16:0/1559261691 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a68037070 con 0x7f6a78141470 2024-09-19T07:48:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.631+0000 7f6a777fe640 1 -- 172.21.15.16:0/1559261691 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7f6a68032440 con 0x7f6a78141470 2024-09-19T07:48:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.631+0000 7f6a777fe640 1 -- 172.21.15.16:0/1559261691 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a68032810 con 0x7f6a78141470 2024-09-19T07:48:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.631+0000 7f6a7ecc0640 1 -- 172.21.15.16:0/1559261691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a78141470 msgr2=0x7f6a78141b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:33.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.631+0000 7f6a7ecc0640 1 --2- 172.21.15.16:0/1559261691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a78141470 0x7f6a78141b70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6a780b8400 tx=0x7f6a6802a1e0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.631+0000 7f6a7ecc0640 1 -- 172.21.15.16:0/1559261691 shutdown_connections 2024-09-19T07:48:33.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.631+0000 7f6a7ecc0640 1 --2- 172.21.15.16:0/1559261691 >> v2:172.21.15.146:3300/0 conn(0x7f6a7806ce70 0x7f6a7806d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.631+0000 7f6a7ecc0640 1 --2- 172.21.15.16:0/1559261691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a78141470 0x7f6a78141b70 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.631+0000 7f6a7ecc0640 1 -- 172.21.15.16:0/1559261691 >> 172.21.15.16:0/1559261691 conn(0x7f6a780b4040 msgr2=0x7f6a780b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:33.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.631+0000 7f6a7ecc0640 1 -- 172.21.15.16:0/1559261691 shutdown_connections 2024-09-19T07:48:33.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.631+0000 7f6a7ecc0640 1 -- 172.21.15.16:0/1559261691 wait complete. 2024-09-19T07:48:33.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.632+0000 7f6a7ecc0640 1 Processor -- start 2024-09-19T07:48:33.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.633+0000 7f6a7ecc0640 1 -- start start 2024-09-19T07:48:33.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.633+0000 7f6a7ecc0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a7806ce70 0x7f6a780b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.633+0000 7f6a7ecc0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a780d2130 0x7f6a7815db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.633+0000 7f6a7ecc0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a78141470 0x7f6a78160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.633+0000 7f6a7ecc0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a78162a50 con 0x7f6a780d2130 2024-09-19T07:48:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.633+0000 7f6a7ecc0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6a78162770 con 0x7f6a78141470 2024-09-19T07:48:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.633+0000 7f6a7ecc0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6a78162d30 con 0x7f6a7806ce70 2024-09-19T07:48:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.633+0000 7f6a77fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a780d2130 0x7f6a7815db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.634+0000 7f6a77fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a780d2130 0x7f6a7815db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36098/0 (socket says 172.21.15.16:36098) 2024-09-19T07:48:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.634+0000 7f6a77fff640 1 -- 172.21.15.16:0/316993024 learned_addr learned my addr 172.21.15.16:0/316993024 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.634+0000 7f6a7d25d640 1 --2- 172.21.15.16:0/316993024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a78141470 0x7f6a78160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.634+0000 7f6a7ca5c640 1 --2- 172.21.15.16:0/316993024 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a7806ce70 0x7f6a780b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.634+0000 7f6a77fff640 1 -- 172.21.15.16:0/316993024 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a7806ce70 msgr2=0x7f6a780b5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:33.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.634+0000 7f6a77fff640 1 --2- 172.21.15.16:0/316993024 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a7806ce70 0x7f6a780b5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.634+0000 7f6a77fff640 1 -- 172.21.15.16:0/316993024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a78141470 msgr2=0x7f6a78160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:33.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.634+0000 7f6a77fff640 1 --2- 172.21.15.16:0/316993024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a78141470 0x7f6a78160b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.634+0000 7f6a77fff640 1 -- 172.21.15.16:0/316993024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a780b60f0 con 0x7f6a780d2130 2024-09-19T07:48:33.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.634+0000 7f6a77fff640 1 --2- 172.21.15.16:0/316993024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a780d2130 0x7f6a7815db00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c007f20 tx=0x7f6a6c00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:33.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.634+0000 7f6a757fa640 1 -- 172.21.15.16:0/316993024 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c014620 con 0x7f6a780d2130 2024-09-19T07:48:33.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.634+0000 7f6a757fa640 1 -- 172.21.15.16:0/316993024 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7f6a6c019070 con 0x7f6a780d2130 2024-09-19T07:48:33.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.634+0000 7f6a7ecc0640 1 -- 172.21.15.16:0/316993024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a780b6ff0 con 0x7f6a780d2130 2024-09-19T07:48:33.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.634+0000 7f6a757fa640 1 -- 172.21.15.16:0/316993024 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c014a00 con 0x7f6a780d2130 2024-09-19T07:48:33.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.634+0000 7f6a7ecc0640 1 -- 172.21.15.16:0/316993024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a780b53b0 con 0x7f6a780d2130 2024-09-19T07:48:33.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.635+0000 7f6a7ecc0640 1 -- 172.21.15.16:0/316993024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a780b51c0 con 0x7f6a780d2130 2024-09-19T07:48:33.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.636+0000 7f6a757fa640 1 -- 172.21.15.16:0/316993024 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6a6c014ba0 con 0x7f6a780d2130 2024-09-19T07:48:33.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.636+0000 7f6a757fa640 1 --2- 172.21.15.16:0/316993024 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6a6406c2b0 0x7f6a6406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:33.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.636+0000 7f6a757fa640 1 -- 172.21.15.16:0/316993024 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6202+0+0 (secure 0 0 0) 0x7f6a6c08be20 con 0x7f6a780d2130 2024-09-19T07:48:33.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.636+0000 7f6a7ca5c640 1 --2- 172.21.15.16:0/316993024 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6a6406c2b0 0x7f6a6406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:33.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.637+0000 7f6a7ca5c640 1 --2- 172.21.15.16:0/316993024 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6a6406c2b0 0x7f6a6406e770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6a780b8400 tx=0x7f6a6802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:33.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.640+0000 7f6a757fa640 1 -- 172.21.15.16:0/316993024 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6a6c059d50 con 0x7f6a780d2130 2024-09-19T07:48:33.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.852+0000 7f6a7ecc0640 1 -- 172.21.15.16:0/316993024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6a781434d0 con 0x7f6a780d2130 2024-09-19T07:48:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.852+0000 7f6a757fa640 1 -- 172.21.15.16:0/316993024 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f6a6c05e050 con 0x7f6a780d2130 2024-09-19T07:48:33.853 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:48:33.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.854+0000 7f6a7ecc0640 1 -- 172.21.15.16:0/316993024 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6a6406c2b0 msgr2=0x7f6a6406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.854+0000 7f6a7ecc0640 1 --2- 172.21.15.16:0/316993024 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6a6406c2b0 0x7f6a6406e770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6a780b8400 tx=0x7f6a6802e000 comp rx=0 tx=0).stop 2024-09-19T07:48:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.854+0000 7f6a7ecc0640 1 -- 172.21.15.16:0/316993024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a780d2130 msgr2=0x7f6a7815db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.854+0000 7f6a7ecc0640 1 --2- 172.21.15.16:0/316993024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a780d2130 0x7f6a7815db00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c007f20 tx=0x7f6a6c00e570 comp rx=0 tx=0).stop 2024-09-19T07:48:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.854+0000 7f6a7ecc0640 1 -- 172.21.15.16:0/316993024 shutdown_connections 2024-09-19T07:48:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.854+0000 7f6a7ecc0640 1 --2- 172.21.15.16:0/316993024 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6a6406c2b0 0x7f6a6406e770 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.854+0000 7f6a7ecc0640 1 --2- 172.21.15.16:0/316993024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a78141470 0x7f6a78160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.854+0000 7f6a7ecc0640 1 --2- 172.21.15.16:0/316993024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a780d2130 0x7f6a7815db00 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.854+0000 7f6a7ecc0640 1 --2- 172.21.15.16:0/316993024 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a7806ce70 0x7f6a780b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.854+0000 7f6a7ecc0640 1 -- 172.21.15.16:0/316993024 >> 172.21.15.16:0/316993024 conn(0x7f6a780b4040 msgr2=0x7f6a7815f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.854+0000 7f6a7ecc0640 1 -- 172.21.15.16:0/316993024 shutdown_connections 2024-09-19T07:48:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:33.854+0000 7f6a7ecc0640 1 -- 172.21.15.16:0/316993024 wait complete. 2024-09-19T07:48:33.865 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:48:33.865 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:48:33.866 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T07:48:33.866 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T07:48:33.866 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:48:33.866 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:48:33.866 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:48:33.866 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:48:33.866 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:48:33.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:48:33.866 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T07:48:33.882 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:48:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:48:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:48:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:48:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:48:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:48:33.883 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:48:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:48:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T07:48:33.883 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:48:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:48:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T07:48:33.884 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:48:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:48:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T07:48:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T07:48:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42475sec preferred_lft 42475sec 2024-09-19T07:48:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T07:48:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:48:33.884 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:48:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:48:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T07:48:33.884 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:48:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:48:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:48:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:48:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::32:7bff:fe92:8895/64 scope link 2024-09-19T07:48:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:48:33.885 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:48:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether e6:fd:21:6a:43:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:48:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e4fd:21ff:fe6a:4369/64 scope link 2024-09-19T07:48:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:48:33.885 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:48:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:48:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::80f4:81ff:fe7d:b7f0/64 scope link 2024-09-19T07:48:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:48:33.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:48:33.886 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:48:33.948 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:48:33.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:48:33.950 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T07:48:34.008 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T07:48:34.008 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:48:34.008 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T07:48:34.008 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T07:48:34.008 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:48:34.008 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:48:34.008 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:48:34.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T07:48:34.009 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T07:48:34.009 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T07:48:34.009 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:48:34.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:48:34.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:48:34.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:48:34.139 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:48:34.155 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T07:48:34.155 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:48:34.156 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:48:34.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:48:34.156 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-19T07:48:34.222 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T07:48:34.222 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:48:34.222 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T07:48:34.278 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:48:34.278 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:48:34.278 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:48:34.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:48:34.400 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:48:34.457 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:48:34.457 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:48:34.457 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:48:34.457 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:48:34.457 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:48:34.457 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:48:34.457 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:48:34.457 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:48:34.458 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:48:34.458 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:48:34.458 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:48:34.458 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:48:34.458 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:48:34.458 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:48:34.458 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:48:34.458 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:48:34.458 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:48:34.458 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:48:34.458 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:48:34.458 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:48:34.458 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:48:34.458 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:48:34.459 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:48:34.459 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:48:34.459 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:48:34.459 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:48:34.459 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:48:34.459 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:48:34.459 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:48:34.459 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:48:34.459 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:48:34.459 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:48:34.459 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:48:34.459 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:48:34.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:48:34.460 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:48:34.513 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:48:34.513 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T07:48:34.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:48:34.581 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:48:34.638 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:48:34.638 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:48:34.638 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:48:34.638 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:48:34.639 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:48:34.639 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:48:34.639 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:48:34.639 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:48:34.639 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:48:34.639 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:48:34.639 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:48:34.639 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:48:34.639 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:48:34.639 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:48:34.639 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:48:34.639 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:48:34.639 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:48:34.640 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:48:34.640 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:48:34.640 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:48:34.640 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:48:34.640 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:48:34.640 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:48:34.640 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:48:34.640 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:48:34.640 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:48:34.640 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:48:34.640 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:48:34.640 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:48:34.640 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:48:34.641 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:48:34.641 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:48:34.641 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:48:34.641 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:48:34.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:48:34.641 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:48:34.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-19T07:48:34.639+0000 7ff5a25b9480 -1 init, newargv = 0x55bdf000b010 newargc=15 2024-09-19T07:48:34.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[52841]: starting ceph client 2024-09-19T07:48:34.644 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[52841]: starting fuse 2024-09-19T07:48:34.658 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:48:34.658 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-19T07:48:34.658 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:48:34.659 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:48:34.659 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:48:34.659 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:48:34.659 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:48:34.659 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:48:34.659 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:48:34.659 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:48:34.659 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:48:34.659 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:48:34.659 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:48:34.659 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:48:34.659 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:48:34.659 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:48:34.659 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:48:34.660 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:48:34.660 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:48:34.660 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:48:34.660 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:48:34.660 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:48:34.660 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:48:34.660 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:48:34.660 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:48:34.660 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:48:34.660 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:48:34.660 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:48:34.660 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:48:34.778 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.52841.asok 2024-09-19T07:48:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:48:34.789 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52841.asok 2024-09-19T07:48:34.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:48:34.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52841.asok status 2024-09-19T07:48:34.926 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:48:34.926 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:48:34.926 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:48:34.926 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:48:34.926 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:48:34.926 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:48:34.926 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:48:34.926 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "52841", 2024-09-19T07:48:34.926 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:48:34.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:34.926 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-19T07:48:34.927 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-19T07:48:34.927 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5589, 2024-09-19T07:48:34.927 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:48:34.927 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:48:34.927 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:48:34.927 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5589 2024-09-19T07:48:34.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:34.927 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:48:34.927 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:48:34.927 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:48:34.927 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1839221875 2024-09-19T07:48:34.927 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:34.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:34.928 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:48:34.928 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:48:34.928 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:48:34.928 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1839221875 2024-09-19T07:48:34.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:34.928 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.5589 192.168.0.1:0/1839221875", 2024-09-19T07:48:34.928 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/1839221875", 2024-09-19T07:48:34.928 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-19T07:48:34.928 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 25, 2024-09-19T07:48:34.928 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 59, 2024-09-19T07:48:34.928 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:48:34.928 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:48:34.928 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:48:34.929 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:48:34.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:48:34.934 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-19T07:48:34.934 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T07:48:34.951 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-19T07:48:34.951 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T07:48:34.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:48:34.951 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:48:35.021 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:48:35.022 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:48:35.104 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:48:35.156 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:48:35.156 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:48:35.156 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:48:35.156 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:48:35.172 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:48:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.315+0000 7fd6a901e640 1 Processor -- start 2024-09-19T07:48:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.315+0000 7fd6a901e640 1 -- start start 2024-09-19T07:48:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.315+0000 7fd6a901e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6a415b120 0x7fd6a406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.315+0000 7fd6a901e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd6a40d0bc0 0x7fd6a406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.315+0000 7fd6a901e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6a415b500 con 0x7fd6a406dec0 2024-09-19T07:48:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.315+0000 7fd6a901e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd6a406e940 con 0x7fd6a415b120 2024-09-19T07:48:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.315+0000 7fd6a901e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fd6a406ec20 con 0x7fd6a40d0bc0 2024-09-19T07:48:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.316+0000 7fd6a2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6a415b120 0x7fd6a406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.316+0000 7fd6a2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6a415b120 0x7fd6a406f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36114/0 (socket says 172.21.15.16:36114) 2024-09-19T07:48:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.316+0000 7fd6a2d9d640 1 -- 172.21.15.16:0/647882393 learned_addr learned my addr 172.21.15.16:0/647882393 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.316+0000 7fd6a359e640 1 --2- 172.21.15.16:0/647882393 >> v2:172.21.15.146:3300/0 conn(0x7fd6a40d0bc0 0x7fd6a406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.316+0000 7fd6a2d9d640 1 -- 172.21.15.16:0/647882393 >> v2:172.21.15.146:3300/0 conn(0x7fd6a40d0bc0 msgr2=0x7fd6a406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.316+0000 7fd6a2d9d640 1 --2- 172.21.15.16:0/647882393 >> v2:172.21.15.146:3300/0 conn(0x7fd6a40d0bc0 0x7fd6a406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.316+0000 7fd6a2d9d640 1 -- 172.21.15.16:0/647882393 >> v1:172.21.15.146:6789/0 conn(0x7fd6a406dec0 legacy=0x7fd6a40d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.316+0000 7fd6a2d9d640 1 -- 172.21.15.16:0/647882393 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a406ef00 con 0x7fd6a415b120 2024-09-19T07:48:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.316+0000 7fd6a359e640 1 --2- 172.21.15.16:0/647882393 >> v2:172.21.15.146:3300/0 conn(0x7fd6a40d0bc0 0x7fd6a406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.317+0000 7fd6a2d9d640 1 --2- 172.21.15.16:0/647882393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a415b120 0x7fd6a406f9e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd6a415eb60 tx=0x7fd69402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdcba7b493b2e932 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.317+0000 7fd6a1d9b640 1 -- 172.21.15.16:0/647882393 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd694037070 con 0x7fd6a415b120 2024-09-19T07:48:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.317+0000 7fd6a1d9b640 1 -- 172.21.15.16:0/647882393 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7fd694032440 con 0x7fd6a415b120 2024-09-19T07:48:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.317+0000 7fd6a1d9b640 1 -- 172.21.15.16:0/647882393 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd694032810 con 0x7fd6a415b120 2024-09-19T07:48:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.317+0000 7fd6a901e640 1 -- 172.21.15.16:0/647882393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a415b120 msgr2=0x7fd6a406f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.317+0000 7fd6a901e640 1 --2- 172.21.15.16:0/647882393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a415b120 0x7fd6a406f9e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd6a415eb60 tx=0x7fd69402a230 comp rx=0 tx=0).stop 2024-09-19T07:48:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.317+0000 7fd6a901e640 1 -- 172.21.15.16:0/647882393 shutdown_connections 2024-09-19T07:48:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.317+0000 7fd6a901e640 1 --2- 172.21.15.16:0/647882393 >> v2:172.21.15.146:3300/0 conn(0x7fd6a40d0bc0 0x7fd6a406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.317+0000 7fd6a901e640 1 --2- 172.21.15.16:0/647882393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a415b120 0x7fd6a406f9e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.317+0000 7fd6a901e640 1 -- 172.21.15.16:0/647882393 >> 172.21.15.16:0/647882393 conn(0x7fd6a406bdf0 msgr2=0x7fd6a406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.317+0000 7fd6a901e640 1 -- 172.21.15.16:0/647882393 shutdown_connections 2024-09-19T07:48:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.317+0000 7fd6a901e640 1 -- 172.21.15.16:0/647882393 wait complete. 2024-09-19T07:48:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.319+0000 7fd6a901e640 1 Processor -- start 2024-09-19T07:48:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.319+0000 7fd6a901e640 1 -- start start 2024-09-19T07:48:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.320+0000 7fd6a901e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6a406dec0 0x7fd6a40b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.320+0000 7fd6a901e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6a40d0bc0 0x7fd6a40b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.320+0000 7fd6a901e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a415b120 0x7fd6a41460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.320+0000 7fd6a901e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a40ce060 con 0x7fd6a415b120 2024-09-19T07:48:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.320+0000 7fd6a901e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a406f4c0 con 0x7fd6a406dec0 2024-09-19T07:48:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.320+0000 7fd6a901e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a40c5160 con 0x7fd6a40d0bc0 2024-09-19T07:48:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.320+0000 7fd6a359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a415b120 0x7fd6a41460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.320+0000 7fd6a359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a415b120 0x7fd6a41460d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36120/0 (socket says 172.21.15.16:36120) 2024-09-19T07:48:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.320+0000 7fd6a359e640 1 -- 172.21.15.16:0/3808824695 learned_addr learned my addr 172.21.15.16:0/3808824695 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.320+0000 7fd6a2d9d640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6a406dec0 0x7fd6a40b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.320+0000 7fd6a259c640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6a40d0bc0 0x7fd6a40b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.320+0000 7fd6a359e640 1 -- 172.21.15.16:0/3808824695 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6a40d0bc0 msgr2=0x7fd6a40b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.320+0000 7fd6a359e640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6a40d0bc0 0x7fd6a40b2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.320+0000 7fd6a359e640 1 -- 172.21.15.16:0/3808824695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6a406dec0 msgr2=0x7fd6a40b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.320+0000 7fd6a359e640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6a406dec0 0x7fd6a40b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.320+0000 7fd6a359e640 1 -- 172.21.15.16:0/3808824695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a40b2a40 con 0x7fd6a415b120 2024-09-19T07:48:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.320+0000 7fd6a259c640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6a40d0bc0 0x7fd6a40b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.320+0000 7fd6a359e640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a415b120 0x7fd6a41460d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00cea0 tx=0x7fd69c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.320+0000 7fd69b7fe640 1 -- 172.21.15.16:0/3808824695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c003b90 con 0x7fd6a415b120 2024-09-19T07:48:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.321+0000 7fd69b7fe640 1 -- 172.21.15.16:0/3808824695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7fd69c003d50 con 0x7fd6a415b120 2024-09-19T07:48:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.321+0000 7fd69b7fe640 1 -- 172.21.15.16:0/3808824695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c004260 con 0x7fd6a415b120 2024-09-19T07:48:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.321+0000 7fd6a901e640 1 -- 172.21.15.16:0/3808824695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a40afef0 con 0x7fd6a415b120 2024-09-19T07:48:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.321+0000 7fd6a901e640 1 -- 172.21.15.16:0/3808824695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a40b04b0 con 0x7fd6a415b120 2024-09-19T07:48:35.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.322+0000 7fd6a901e640 1 -- 172.21.15.16:0/3808824695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a406e310 con 0x7fd6a415b120 2024-09-19T07:48:35.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.322+0000 7fd69b7fe640 1 -- 172.21.15.16:0/3808824695 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd69c004400 con 0x7fd6a415b120 2024-09-19T07:48:35.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.323+0000 7fd69b7fe640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd68406c2b0 0x7fd68406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:35.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.323+0000 7fd69b7fe640 1 -- 172.21.15.16:0/3808824695 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6202+0+0 (secure 0 0 0) 0x7fd69c08c010 con 0x7fd6a415b120 2024-09-19T07:48:35.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.323+0000 7fd6a2d9d640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd68406c2b0 0x7fd68406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:35.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.323+0000 7fd6a2d9d640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd68406c2b0 0x7fd68406e770 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd6a415eb60 tx=0x7fd69402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:35.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.327+0000 7fd69b7fe640 1 -- 172.21.15.16:0/3808824695 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd69c059eb0 con 0x7fd6a415b120 2024-09-19T07:48:35.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.514+0000 7fd6a901e640 1 -- 172.21.15.16:0/3808824695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd6a40ad160 con 0x7fd6a415b120 2024-09-19T07:48:35.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.515+0000 7fd69b7fe640 1 -- 172.21.15.16:0/3808824695 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 1 entries v59) ==== 80+0+57 (secure 0 0 0) 0x7fd69c05e1b0 con 0x7fd6a415b120 2024-09-19T07:48:35.516 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.1:0/2345385426 2024-09-19T08:48:30.430325+0000 2024-09-19T07:48:35.516 INFO:teuthology.orchestra.run.smithi016.stderr:listed 1 entries 2024-09-19T07:48:35.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.516+0000 7fd6a901e640 1 -- 172.21.15.16:0/3808824695 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd68406c2b0 msgr2=0x7fd68406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:35.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.516+0000 7fd6a901e640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd68406c2b0 0x7fd68406e770 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd6a415eb60 tx=0x7fd69402e000 comp rx=0 tx=0).stop 2024-09-19T07:48:35.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.516+0000 7fd6a901e640 1 -- 172.21.15.16:0/3808824695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a415b120 msgr2=0x7fd6a41460d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:35.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.516+0000 7fd6a901e640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a415b120 0x7fd6a41460d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00cea0 tx=0x7fd69c00cf90 comp rx=0 tx=0).stop 2024-09-19T07:48:35.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.517+0000 7fd6a901e640 1 -- 172.21.15.16:0/3808824695 shutdown_connections 2024-09-19T07:48:35.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.517+0000 7fd6a901e640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd68406c2b0 0x7fd68406e770 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.517+0000 7fd6a901e640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a415b120 0x7fd6a41460d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.517+0000 7fd6a901e640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6a40d0bc0 0x7fd6a40b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.517+0000 7fd6a901e640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6a406dec0 0x7fd6a40b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.517+0000 7fd6a901e640 1 -- 172.21.15.16:0/3808824695 >> 172.21.15.16:0/3808824695 conn(0x7fd6a406bdf0 msgr2=0x7fd6a40cbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:35.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.517+0000 7fd6a901e640 1 -- 172.21.15.16:0/3808824695 shutdown_connections 2024-09-19T07:48:35.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.517+0000 7fd6a901e640 1 -- 172.21.15.16:0/3808824695 wait complete. 2024-09-19T07:48:35.528 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-19T07:48:35.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:48:35.528 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T07:48:35.552 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[52841]: fuse finished with error 0 and tester_r 0 2024-09-19T07:48:35.554 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:48:35.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:48:35.555 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:48:35.628 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:48:35.628 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:48:35.628 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:48:35.648 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:48:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.771+0000 7f6c44074640 1 Processor -- start 2024-09-19T07:48:35.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.771+0000 7f6c44074640 1 -- start start 2024-09-19T07:48:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.772+0000 7f6c44074640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6c3c0d4370 0x7f6c3c0d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.772+0000 7f6c44074640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c3c0d1880 con 0x7f6c3c15e2a0 2024-09-19T07:48:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.772+0000 7f6c44074640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c3c0d1b60 con 0x7f6c3c15e680 2024-09-19T07:48:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.772+0000 7f6c44074640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f6c3c0cf2d0 con 0x7f6c3c0d4370 2024-09-19T07:48:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.772+0000 7f6c41e10640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6c3c15e2a0 0x7f6c3c06d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56502/0 (socket says 172.21.15.16:56502) 2024-09-19T07:48:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.772+0000 7f6c41e10640 1 -- 172.21.15.16:0/3905417506 learned_addr learned my addr 172.21.15.16:0/3905417506 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.772+0000 7f6c40e0e640 1 -- 172.21.15.16:0/3905417506 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3501236849 0 0) 0x7f6c3c0d1880 con 0x7f6c3c15e2a0 2024-09-19T07:48:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.772+0000 7f6c40e0e640 1 -- 172.21.15.16:0/3905417506 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c28003520 con 0x7f6c3c15e2a0 2024-09-19T07:48:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.772+0000 7f6c42611640 1 --2- 172.21.15.16:0/3905417506 >> v2:172.21.15.146:3300/0 conn(0x7f6c3c0d4370 0x7f6c3c0d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.772+0000 7f6c40e0e640 1 -- 172.21.15.16:0/3905417506 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1533393580 0 0) 0x7f6c28003520 con 0x7f6c3c15e2a0 2024-09-19T07:48:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.772+0000 7f6c40e0e640 1 -- 172.21.15.16:0/3905417506 >> v2:172.21.15.146:3300/0 conn(0x7f6c3c0d4370 msgr2=0x7f6c3c0d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.772+0000 7f6c40e0e640 1 --2- 172.21.15.16:0/3905417506 >> v2:172.21.15.146:3300/0 conn(0x7f6c3c0d4370 0x7f6c3c0d4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.772+0000 7f6c40e0e640 1 -- 172.21.15.16:0/3905417506 >> v1:172.21.15.146:6789/0 conn(0x7f6c3c15e680 legacy=0x7f6c3c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.772+0000 7f6c40e0e640 1 -- 172.21.15.16:0/3905417506 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3c0cf5b0 con 0x7f6c3c15e2a0 2024-09-19T07:48:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.772+0000 7f6c40e0e640 1 -- 172.21.15.16:0/3905417506 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f6c2c001670 con 0x7f6c3c15e2a0 2024-09-19T07:48:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.773+0000 7f6c40e0e640 1 -- 172.21.15.16:0/3905417506 >> v1:172.21.15.16:6789/0 conn(0x7f6c3c15e2a0 legacy=0x7f6c3c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.773+0000 7f6c40e0e640 1 --2- 172.21.15.16:0/3905417506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c280039b0 0x7f6c28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.773+0000 7f6c40e0e640 1 --2- 172.21.15.16:0/3905417506 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c28004800 0x7f6c28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.773+0000 7f6c40e0e640 1 --2- 172.21.15.16:0/3905417506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c280051c0 0x7f6c2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.773+0000 7f6c40e0e640 1 -- 172.21.15.16:0/3905417506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c0cf2d0 con 0x7f6c280039b0 2024-09-19T07:48:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.773+0000 7f6c40e0e640 1 -- 172.21.15.16:0/3905417506 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2800c060 con 0x7f6c280051c0 2024-09-19T07:48:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.773+0000 7f6c40e0e640 1 -- 172.21.15.16:0/3905417506 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2800c260 con 0x7f6c28004800 2024-09-19T07:48:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.773+0000 7f6c41e10640 1 --2- 172.21.15.16:0/3905417506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c280039b0 0x7f6c28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.773+0000 7f6c42611640 1 --2- 172.21.15.16:0/3905417506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c280051c0 0x7f6c2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.773+0000 7f6c4160f640 1 --2- 172.21.15.16:0/3905417506 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c28004800 0x7f6c28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.773+0000 7f6c41e10640 1 -- 172.21.15.16:0/3905417506 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c28004800 msgr2=0x7f6c28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.773+0000 7f6c41e10640 1 --2- 172.21.15.16:0/3905417506 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c28004800 0x7f6c28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.773+0000 7f6c41e10640 1 -- 172.21.15.16:0/3905417506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c280051c0 msgr2=0x7f6c2800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.773+0000 7f6c41e10640 1 --2- 172.21.15.16:0/3905417506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c280051c0 0x7f6c2800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.773+0000 7f6c41e10640 1 -- 172.21.15.16:0/3905417506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c28003520 con 0x7f6c280039b0 2024-09-19T07:48:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.773+0000 7f6c42611640 1 -- 172.21.15.16:0/3905417506 reap_dead start 2024-09-19T07:48:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.774+0000 7f6c41e10640 1 --2- 172.21.15.16:0/3905417506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c280039b0 0x7f6c28003dd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00c210 tx=0x7f6c28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd621dda33a6e43c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.774+0000 7f6c40e0e640 1 -- 172.21.15.16:0/3905417506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c007d30 con 0x7f6c280039b0 2024-09-19T07:48:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.774+0000 7f6c40e0e640 1 -- 172.21.15.16:0/3905417506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7f6c2c005ad0 con 0x7f6c280039b0 2024-09-19T07:48:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.774+0000 7f6c40e0e640 1 -- 172.21.15.16:0/3905417506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c007070 con 0x7f6c280039b0 2024-09-19T07:48:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.774+0000 7f6c44074640 1 -- 172.21.15.16:0/3905417506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c280039b0 msgr2=0x7f6c28003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.774+0000 7f6c44074640 1 --2- 172.21.15.16:0/3905417506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c280039b0 0x7f6c28003dd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00c210 tx=0x7f6c28005830 comp rx=0 tx=0).stop 2024-09-19T07:48:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.774+0000 7f6c44074640 1 -- 172.21.15.16:0/3905417506 shutdown_connections 2024-09-19T07:48:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.774+0000 7f6c44074640 1 --2- 172.21.15.16:0/3905417506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c280039b0 0x7f6c28003dd0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.774+0000 7f6c44074640 1 -- 172.21.15.16:0/3905417506 >> 172.21.15.16:0/3905417506 conn(0x7f6c3c09d760 msgr2=0x7f6c3c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.775+0000 7f6c44074640 1 -- 172.21.15.16:0/3905417506 shutdown_connections 2024-09-19T07:48:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.775+0000 7f6c44074640 1 -- 172.21.15.16:0/3905417506 wait complete. 2024-09-19T07:48:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.776+0000 7f6c44074640 1 Processor -- start 2024-09-19T07:48:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.776+0000 7f6c44074640 1 -- start start 2024-09-19T07:48:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.776+0000 7f6c44074640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c280039b0 0x7f6c3c15cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.776+0000 7f6c44074640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3c0ccfc0 0x7f6c3c0c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c44074640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c3c0cd440 0x7f6c3c15f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c44074640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c0d49a0 con 0x7f6c3c0ccfc0 2024-09-19T07:48:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c44074640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c0d4820 con 0x7f6c3c0cd440 2024-09-19T07:48:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c44074640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6c3c0d4b20 con 0x7f6c280039b0 2024-09-19T07:48:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c4160f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3c0ccfc0 0x7f6c3c0c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c4160f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3c0ccfc0 0x7f6c3c0c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36138/0 (socket says 172.21.15.16:36138) 2024-09-19T07:48:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c4160f640 1 -- 172.21.15.16:0/4090595205 learned_addr learned my addr 172.21.15.16:0/4090595205 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c41e10640 1 --2- 172.21.15.16:0/4090595205 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c280039b0 0x7f6c3c15cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c42611640 1 --2- 172.21.15.16:0/4090595205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c3c0cd440 0x7f6c3c15f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c4160f640 1 -- 172.21.15.16:0/4090595205 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c280039b0 msgr2=0x7f6c3c15cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c4160f640 1 --2- 172.21.15.16:0/4090595205 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c280039b0 0x7f6c3c15cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c4160f640 1 -- 172.21.15.16:0/4090595205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c3c0cd440 msgr2=0x7f6c3c15f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c4160f640 1 --2- 172.21.15.16:0/4090595205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c3c0cd440 0x7f6c3c15f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c4160f640 1 -- 172.21.15.16:0/4090595205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3c0cbf60 con 0x7f6c3c0ccfc0 2024-09-19T07:48:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c4160f640 1 --2- 172.21.15.16:0/4090595205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3c0ccfc0 0x7f6c3c0c9bd0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800e410 tx=0x7f6c3800e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c327fc640 1 -- 172.21.15.16:0/4090595205 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38019070 con 0x7f6c3c0ccfc0 2024-09-19T07:48:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c327fc640 1 -- 172.21.15.16:0/4090595205 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7f6c380052e0 con 0x7f6c3c0ccfc0 2024-09-19T07:48:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c44074640 1 -- 172.21.15.16:0/4090595205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c3c0cc200 con 0x7f6c3c0ccfc0 2024-09-19T07:48:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c327fc640 1 -- 172.21.15.16:0/4090595205 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38003810 con 0x7f6c3c0ccfc0 2024-09-19T07:48:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.777+0000 7f6c44074640 1 -- 172.21.15.16:0/4090595205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c3c0cc7c0 con 0x7f6c3c0ccfc0 2024-09-19T07:48:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.778+0000 7f6c44074640 1 -- 172.21.15.16:0/4090595205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c3c06d420 con 0x7f6c3c0ccfc0 2024-09-19T07:48:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.779+0000 7f6c327fc640 1 -- 172.21.15.16:0/4090595205 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6c3801c070 con 0x7f6c3c0ccfc0 2024-09-19T07:48:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.779+0000 7f6c327fc640 1 --2- 172.21.15.16:0/4090595205 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6c2006c2b0 0x7f6c2006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.779+0000 7f6c327fc640 1 -- 172.21.15.16:0/4090595205 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6202+0+0 (secure 0 0 0) 0x7f6c3808ac30 con 0x7f6c3c0ccfc0 2024-09-19T07:48:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.780+0000 7f6c41e10640 1 --2- 172.21.15.16:0/4090595205 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6c2006c2b0 0x7f6c2006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.780+0000 7f6c41e10640 1 --2- 172.21.15.16:0/4090595205 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6c2006c2b0 0x7f6c2006e770 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00c210 tx=0x7f6c2c02d000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.784+0000 7f6c327fc640 1 -- 172.21.15.16:0/4090595205 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6c38058ad0 con 0x7f6c3c0ccfc0 2024-09-19T07:48:35.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.971+0000 7f6c44074640 1 -- 172.21.15.16:0/4090595205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6c3c0d05f0 con 0x7f6c3c0ccfc0 2024-09-19T07:48:35.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.972+0000 7f6c327fc640 1 -- 172.21.15.16:0/4090595205 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 1 entries v59) ==== 80+0+57 (secure 0 0 0) 0x7f6c3805cdd0 con 0x7f6c3c0ccfc0 2024-09-19T07:48:35.972 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.1:0/2345385426 2024-09-19T08:48:30.430325+0000 2024-09-19T07:48:35.972 INFO:teuthology.orchestra.run.smithi016.stderr:listed 1 entries 2024-09-19T07:48:35.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.974+0000 7f6c44074640 1 -- 172.21.15.16:0/4090595205 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6c2006c2b0 msgr2=0x7f6c2006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:35.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.974+0000 7f6c44074640 1 --2- 172.21.15.16:0/4090595205 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6c2006c2b0 0x7f6c2006e770 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00c210 tx=0x7f6c2c02d000 comp rx=0 tx=0).stop 2024-09-19T07:48:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.974+0000 7f6c44074640 1 -- 172.21.15.16:0/4090595205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3c0ccfc0 msgr2=0x7f6c3c0c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.974+0000 7f6c44074640 1 --2- 172.21.15.16:0/4090595205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3c0ccfc0 0x7f6c3c0c9bd0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800e410 tx=0x7f6c3800e8d0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.974+0000 7f6c44074640 1 -- 172.21.15.16:0/4090595205 shutdown_connections 2024-09-19T07:48:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.974+0000 7f6c44074640 1 --2- 172.21.15.16:0/4090595205 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6c2006c2b0 0x7f6c2006e770 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.974+0000 7f6c44074640 1 --2- 172.21.15.16:0/4090595205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c3c0cd440 0x7f6c3c15f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.974+0000 7f6c44074640 1 --2- 172.21.15.16:0/4090595205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3c0ccfc0 0x7f6c3c0c9bd0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.974+0000 7f6c44074640 1 --2- 172.21.15.16:0/4090595205 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c280039b0 0x7f6c3c15cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.974+0000 7f6c44074640 1 -- 172.21.15.16:0/4090595205 >> 172.21.15.16:0/4090595205 conn(0x7f6c3c09d760 msgr2=0x7f6c3c0b9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.974+0000 7f6c44074640 1 -- 172.21.15.16:0/4090595205 shutdown_connections 2024-09-19T07:48:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:35.974+0000 7f6c44074640 1 -- 172.21.15.16:0/4090595205 wait complete. 2024-09-19T07:48:35.985 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2024-09-19T07:48:35.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:48:35.985 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T07:48:36.014 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[44802]: fuse finished with error 0 and tester_r 0 2024-09-19T07:48:36.016 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:48:36.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:48:36.016 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T07:48:36.089 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:48:36.089 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:48:36.089 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:48:36.106 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:48:36.106 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:48:36.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:48:36.106 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T07:48:36.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:48:36.162 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T07:48:36.162 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:48:36.162 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:48:36.163 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:48:36.181 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:48:36.181 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:48:36.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:48:36.181 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T07:48:36.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:48:36.239 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T07:48:36.239 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:48:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.341+0000 7fef80c3c640 1 Processor -- start 2024-09-19T07:48:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.341+0000 7fef80c3c640 1 -- start start 2024-09-19T07:48:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.342+0000 7fef80c3c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef7c0d0bc0 0x7fef7c0cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.342+0000 7fef80c3c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef7c15c2c0 0x7fef7c0cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.342+0000 7fef80c3c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef7c0cfc80 con 0x7fef7c15bee0 2024-09-19T07:48:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.342+0000 7fef80c3c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fef7c0ceaf0 con 0x7fef7c0d0bc0 2024-09-19T07:48:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.342+0000 7fef80c3c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fef7c0cdee0 con 0x7fef7c15c2c0 2024-09-19T07:48:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.342+0000 7fef79d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef7c0d0bc0 0x7fef7c0cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.342+0000 7fef79d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef7c0d0bc0 0x7fef7c0cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36142/0 (socket says 172.21.15.16:36142) 2024-09-19T07:48:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.342+0000 7fef79d9b640 1 -- 172.21.15.16:0/660959835 learned_addr learned my addr 172.21.15.16:0/660959835 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.342+0000 7fef7ad9d640 1 --2- 172.21.15.16:0/660959835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef7c15c2c0 0x7fef7c0cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.342+0000 7fef79d9b640 1 -- 172.21.15.16:0/660959835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef7c15c2c0 msgr2=0x7fef7c0cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.342+0000 7fef79d9b640 1 --2- 172.21.15.16:0/660959835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef7c15c2c0 0x7fef7c0cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.342+0000 7fef79d9b640 1 -- 172.21.15.16:0/660959835 >> v1:172.21.15.146:6789/0 conn(0x7fef7c15bee0 legacy=0x7fef7c0d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.342+0000 7fef79d9b640 1 -- 172.21.15.16:0/660959835 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c0ce1c0 con 0x7fef7c0d0bc0 2024-09-19T07:48:36.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.343+0000 7fef79d9b640 1 --2- 172.21.15.16:0/660959835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef7c0d0bc0 0x7fef7c0cf200 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fef70009870 tx=0x7fef7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e444adeea0df0eb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:36.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.343+0000 7fef7959a640 1 -- 172.21.15.16:0/660959835 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70037070 con 0x7fef7c0d0bc0 2024-09-19T07:48:36.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.343+0000 7fef7959a640 1 -- 172.21.15.16:0/660959835 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7fef70032440 con 0x7fef7c0d0bc0 2024-09-19T07:48:36.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.343+0000 7fef7959a640 1 -- 172.21.15.16:0/660959835 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70032810 con 0x7fef7c0d0bc0 2024-09-19T07:48:36.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.344+0000 7fef80c3c640 1 -- 172.21.15.16:0/660959835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef7c0d0bc0 msgr2=0x7fef7c0cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:36.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.344+0000 7fef80c3c640 1 --2- 172.21.15.16:0/660959835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef7c0d0bc0 0x7fef7c0cf200 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fef70009870 tx=0x7fef7002a270 comp rx=0 tx=0).stop 2024-09-19T07:48:36.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.344+0000 7fef80c3c640 1 -- 172.21.15.16:0/660959835 shutdown_connections 2024-09-19T07:48:36.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.344+0000 7fef80c3c640 1 --2- 172.21.15.16:0/660959835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef7c15c2c0 0x7fef7c0cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.344+0000 7fef80c3c640 1 --2- 172.21.15.16:0/660959835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef7c0d0bc0 0x7fef7c0cf200 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.344+0000 7fef80c3c640 1 -- 172.21.15.16:0/660959835 >> 172.21.15.16:0/660959835 conn(0x7fef7c06bdf0 msgr2=0x7fef7c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:36.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.344+0000 7fef80c3c640 1 -- 172.21.15.16:0/660959835 shutdown_connections 2024-09-19T07:48:36.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.344+0000 7fef80c3c640 1 -- 172.21.15.16:0/660959835 wait complete. 2024-09-19T07:48:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.345+0000 7fef80c3c640 1 Processor -- start 2024-09-19T07:48:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.345+0000 7fef80c3c640 1 -- start start 2024-09-19T07:48:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.346+0000 7fef80c3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef7c0d0bc0 0x7fef7c154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.346+0000 7fef80c3c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef7c15bee0 0x7fef7c15d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.346+0000 7fef80c3c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef7c15c2c0 0x7fef7c1502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.346+0000 7fef80c3c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c06dec0 con 0x7fef7c0d0bc0 2024-09-19T07:48:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.346+0000 7fef80c3c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c0ce780 con 0x7fef7c15c2c0 2024-09-19T07:48:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.346+0000 7fef80c3c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fef7c06e040 con 0x7fef7c15bee0 2024-09-19T07:48:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.346+0000 7fef7a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef7c0d0bc0 0x7fef7c154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.346+0000 7fef7a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef7c0d0bc0 0x7fef7c154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36158/0 (socket says 172.21.15.16:36158) 2024-09-19T07:48:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.346+0000 7fef7a59c640 1 -- 172.21.15.16:0/1656767439 learned_addr learned my addr 172.21.15.16:0/1656767439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.346+0000 7fef79d9b640 1 --2- 172.21.15.16:0/1656767439 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef7c15bee0 0x7fef7c15d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.346+0000 7fef7ad9d640 1 --2- 172.21.15.16:0/1656767439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef7c15c2c0 0x7fef7c1502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.346+0000 7fef7a59c640 1 -- 172.21.15.16:0/1656767439 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef7c15bee0 msgr2=0x7fef7c15d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.346+0000 7fef7a59c640 1 --2- 172.21.15.16:0/1656767439 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef7c15bee0 0x7fef7c15d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.346+0000 7fef7a59c640 1 -- 172.21.15.16:0/1656767439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef7c15c2c0 msgr2=0x7fef7c1502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.346+0000 7fef7a59c640 1 --2- 172.21.15.16:0/1656767439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef7c15c2c0 0x7fef7c1502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.347+0000 7fef7a59c640 1 -- 172.21.15.16:0/1656767439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c0c85c0 con 0x7fef7c0d0bc0 2024-09-19T07:48:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.347+0000 7fef7a59c640 1 --2- 172.21.15.16:0/1656767439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef7c0d0bc0 0x7fef7c154be0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00c430 tx=0x7fef6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.347+0000 7fef6affd640 1 -- 172.21.15.16:0/1656767439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c0052d0 con 0x7fef7c0d0bc0 2024-09-19T07:48:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.347+0000 7fef6affd640 1 -- 172.21.15.16:0/1656767439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7fef6c005490 con 0x7fef7c0d0bc0 2024-09-19T07:48:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.347+0000 7fef6affd640 1 -- 172.21.15.16:0/1656767439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c007260 con 0x7fef7c0d0bc0 2024-09-19T07:48:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.347+0000 7fef80c3c640 1 -- 172.21.15.16:0/1656767439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7c0c88a0 con 0x7fef7c0d0bc0 2024-09-19T07:48:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.347+0000 7fef80c3c640 1 -- 172.21.15.16:0/1656767439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7c15d900 con 0x7fef7c0d0bc0 2024-09-19T07:48:36.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.348+0000 7fef80c3c640 1 -- 172.21.15.16:0/1656767439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef7c0d04b0 con 0x7fef7c0d0bc0 2024-09-19T07:48:36.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.349+0000 7fef6affd640 1 -- 172.21.15.16:0/1656767439 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fef6c007400 con 0x7fef7c0d0bc0 2024-09-19T07:48:36.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.349+0000 7fef6affd640 1 --2- 172.21.15.16:0/1656767439 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fef5c06c2b0 0x7fef5c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:36.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.349+0000 7fef6affd640 1 -- 172.21.15.16:0/1656767439 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6202+0+0 (secure 0 0 0) 0x7fef6c08ab50 con 0x7fef7c0d0bc0 2024-09-19T07:48:36.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.349+0000 7fef79d9b640 1 --2- 172.21.15.16:0/1656767439 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fef5c06c2b0 0x7fef5c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:36.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.350+0000 7fef79d9b640 1 --2- 172.21.15.16:0/1656767439 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fef5c06c2b0 0x7fef5c06e770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fef70009870 tx=0x7fef7002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.353+0000 7fef6affd640 1 -- 172.21.15.16:0/1656767439 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fef6c058a90 con 0x7fef7c0d0bc0 2024-09-19T07:48:36.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.567+0000 7fef80c3c640 1 -- 172.21.15.16:0/1656767439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef7c0cffe0 con 0x7fef7c0d0bc0 2024-09-19T07:48:36.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.568+0000 7fef6affd640 1 -- 172.21.15.16:0/1656767439 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5224 (secure 0 0 0) 0x7fef6c05cd90 con 0x7fef7c0d0bc0 2024-09-19T07:48:36.568 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:36.569 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-19T07:48:36:069113+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/3536366465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3536366465},{"type":"v1","addr":"172.21.15.146:6836","nonce":3536366465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3612374586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3612374586},{"type":"v1","addr":"172.21.15.146:6837","nonce":3612374586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5337,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/781070730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":781070730},{"type":"v1","addr":"172.21.15.16:6833","nonce":781070730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:45:10.494498+0000","modified":"2024-09-19T07:48:35.187029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5340":{"gid":5340,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":54,"addr":"172.21.15.16:6835/642332950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":642332950},{"type":"v1","addr":"172.21.15.16:6835","nonce":642332950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5340,"qdb_cluster":[5340]},"id":3}]} 2024-09-19T07:48:36.569 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-19T07:48:36.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.570+0000 7fef80c3c640 1 -- 172.21.15.16:0/1656767439 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fef5c06c2b0 msgr2=0x7fef5c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:36.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.570+0000 7fef80c3c640 1 --2- 172.21.15.16:0/1656767439 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fef5c06c2b0 0x7fef5c06e770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fef70009870 tx=0x7fef7002e000 comp rx=0 tx=0).stop 2024-09-19T07:48:36.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.570+0000 7fef80c3c640 1 -- 172.21.15.16:0/1656767439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef7c0d0bc0 msgr2=0x7fef7c154be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:36.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.570+0000 7fef80c3c640 1 --2- 172.21.15.16:0/1656767439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef7c0d0bc0 0x7fef7c154be0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00c430 tx=0x7fef6c00c8f0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.570+0000 7fef80c3c640 1 -- 172.21.15.16:0/1656767439 shutdown_connections 2024-09-19T07:48:36.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.570+0000 7fef80c3c640 1 --2- 172.21.15.16:0/1656767439 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fef5c06c2b0 0x7fef5c06e770 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.570+0000 7fef80c3c640 1 --2- 172.21.15.16:0/1656767439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef7c15c2c0 0x7fef7c1502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.570+0000 7fef80c3c640 1 --2- 172.21.15.16:0/1656767439 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef7c15bee0 0x7fef7c15d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.570+0000 7fef80c3c640 1 --2- 172.21.15.16:0/1656767439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef7c0d0bc0 0x7fef7c154be0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.570+0000 7fef80c3c640 1 -- 172.21.15.16:0/1656767439 >> 172.21.15.16:0/1656767439 conn(0x7fef7c06bdf0 msgr2=0x7fef7c0cd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:36.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.570+0000 7fef80c3c640 1 -- 172.21.15.16:0/1656767439 shutdown_connections 2024-09-19T07:48:36.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.570+0000 7fef80c3c640 1 -- 172.21.15.16:0/1656767439 wait complete. 2024-09-19T07:48:36.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:48:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.683+0000 7fec42a23640 1 Processor -- start 2024-09-19T07:48:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.683+0000 7fec42a23640 1 -- start start 2024-09-19T07:48:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.684+0000 7fec42a23640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec3c06d430 0x7fec3c1439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.684+0000 7fec42a23640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fec3c143f30 0x7fec3c0d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.684+0000 7fec42a23640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec3c0c7140 con 0x7fec3c0c8d20 2024-09-19T07:48:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.684+0000 7fec42a23640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fec3c141960 con 0x7fec3c143f30 2024-09-19T07:48:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.684+0000 7fec42a23640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fec3c142200 con 0x7fec3c06d430 2024-09-19T07:48:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.684+0000 7fec40fc0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fec3c143f30 0x7fec3c0d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.684+0000 7fec40fc0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fec3c143f30 0x7fec3c0d1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36160/0 (socket says 172.21.15.16:36160) 2024-09-19T07:48:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.684+0000 7fec40fc0640 1 -- 172.21.15.16:0/2461886354 learned_addr learned my addr 172.21.15.16:0/2461886354 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.684+0000 7fec3b7fe640 1 --2- 172.21.15.16:0/2461886354 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec3c06d430 0x7fec3c1439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.685+0000 7fec40fc0640 1 -- 172.21.15.16:0/2461886354 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec3c06d430 msgr2=0x7fec3c1439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:36.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.685+0000 7fec40fc0640 1 --2- 172.21.15.16:0/2461886354 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec3c06d430 0x7fec3c1439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.685+0000 7fec40fc0640 1 -- 172.21.15.16:0/2461886354 >> v1:172.21.15.146:6789/0 conn(0x7fec3c0c8d20 legacy=0x7fec3c06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:36.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.685+0000 7fec40fc0640 1 -- 172.21.15.16:0/2461886354 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c0c9100 con 0x7fec3c143f30 2024-09-19T07:48:36.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.685+0000 7fec3b7fe640 1 --2- 172.21.15.16:0/2461886354 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec3c06d430 0x7fec3c1439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.685+0000 7fec40fc0640 1 --2- 172.21.15.16:0/2461886354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec3c143f30 0x7fec3c0d1e50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fec34009e50 tx=0x7fec3402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118b9adf829b76c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.685+0000 7fec3affd640 1 -- 172.21.15.16:0/2461886354 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34036070 con 0x7fec3c143f30 2024-09-19T07:48:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.686+0000 7fec3affd640 1 -- 172.21.15.16:0/2461886354 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7fec34031600 con 0x7fec3c143f30 2024-09-19T07:48:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.686+0000 7fec3affd640 1 -- 172.21.15.16:0/2461886354 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec340319d0 con 0x7fec3c143f30 2024-09-19T07:48:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.686+0000 7fec42a23640 1 -- 172.21.15.16:0/2461886354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec3c143f30 msgr2=0x7fec3c0d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.686+0000 7fec42a23640 1 --2- 172.21.15.16:0/2461886354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec3c143f30 0x7fec3c0d1e50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fec34009e50 tx=0x7fec3402a5f0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.686+0000 7fec42a23640 1 -- 172.21.15.16:0/2461886354 shutdown_connections 2024-09-19T07:48:36.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.686+0000 7fec42a23640 1 --2- 172.21.15.16:0/2461886354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec3c143f30 0x7fec3c0d1e50 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.686+0000 7fec42a23640 1 --2- 172.21.15.16:0/2461886354 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec3c06d430 0x7fec3c1439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.686+0000 7fec42a23640 1 -- 172.21.15.16:0/2461886354 >> 172.21.15.16:0/2461886354 conn(0x7fec3c0b3ed0 msgr2=0x7fec3c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:36.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.686+0000 7fec42a23640 1 -- 172.21.15.16:0/2461886354 shutdown_connections 2024-09-19T07:48:36.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.686+0000 7fec42a23640 1 -- 172.21.15.16:0/2461886354 wait complete. 2024-09-19T07:48:36.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.687+0000 7fec42a23640 1 Processor -- start 2024-09-19T07:48:36.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.688+0000 7fec42a23640 1 -- start start 2024-09-19T07:48:36.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.688+0000 7fec42a23640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec3c06d430 0x7fec3c0cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:36.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.688+0000 7fec42a23640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec3c0c8d20 0x7fec3c13fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:36.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.688+0000 7fec42a23640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec3c143f30 0x7fec3c0cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:36.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.688+0000 7fec42a23640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c141e90 con 0x7fec3c06d430 2024-09-19T07:48:36.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.688+0000 7fec42a23640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c141bb0 con 0x7fec3c0c8d20 2024-09-19T07:48:36.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.688+0000 7fec42a23640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fec3c144380 con 0x7fec3c143f30 2024-09-19T07:48:36.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.688+0000 7fec3bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec3c06d430 0x7fec3c0cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:36.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.688+0000 7fec3bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec3c06d430 0x7fec3c0cbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36162/0 (socket says 172.21.15.16:36162) 2024-09-19T07:48:36.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.688+0000 7fec3bfff640 1 -- 172.21.15.16:0/355778793 learned_addr learned my addr 172.21.15.16:0/355778793 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:36.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.689+0000 7fec40fc0640 1 --2- 172.21.15.16:0/355778793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec3c143f30 0x7fec3c0cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:36.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.689+0000 7fec3b7fe640 1 --2- 172.21.15.16:0/355778793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec3c0c8d20 0x7fec3c13fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:36.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.689+0000 7fec3bfff640 1 -- 172.21.15.16:0/355778793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec3c143f30 msgr2=0x7fec3c0cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:36.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.689+0000 7fec3bfff640 1 --2- 172.21.15.16:0/355778793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec3c143f30 0x7fec3c0cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.689+0000 7fec3bfff640 1 -- 172.21.15.16:0/355778793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec3c0c8d20 msgr2=0x7fec3c13fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:36.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.689+0000 7fec3bfff640 1 --2- 172.21.15.16:0/355778793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec3c0c8d20 0x7fec3c13fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.689+0000 7fec3bfff640 1 -- 172.21.15.16:0/355778793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c0ca2b0 con 0x7fec3c06d430 2024-09-19T07:48:36.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.689+0000 7fec40fc0640 1 --2- 172.21.15.16:0/355778793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec3c143f30 0x7fec3c0cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:36.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.689+0000 7fec3bfff640 1 --2- 172.21.15.16:0/355778793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec3c06d430 0x7fec3c0cbca0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fec2c00c3c0 tx=0x7fec2c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:36.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.689+0000 7fec38ff9640 1 -- 172.21.15.16:0/355778793 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0052d0 con 0x7fec3c06d430 2024-09-19T07:48:36.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.689+0000 7fec38ff9640 1 -- 172.21.15.16:0/355778793 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7fec2c005490 con 0x7fec3c06d430 2024-09-19T07:48:36.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.689+0000 7fec42a23640 1 -- 172.21.15.16:0/355778793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3c0ca590 con 0x7fec3c06d430 2024-09-19T07:48:36.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.689+0000 7fec38ff9640 1 -- 172.21.15.16:0/355778793 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c007260 con 0x7fec3c06d430 2024-09-19T07:48:36.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.689+0000 7fec42a23640 1 -- 172.21.15.16:0/355778793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec3c0cae30 con 0x7fec3c06d430 2024-09-19T07:48:36.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.690+0000 7fec42a23640 1 -- 172.21.15.16:0/355778793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec3c1439f0 con 0x7fec3c06d430 2024-09-19T07:48:36.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.691+0000 7fec38ff9640 1 -- 172.21.15.16:0/355778793 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fec2c007400 con 0x7fec3c06d430 2024-09-19T07:48:36.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.691+0000 7fec38ff9640 1 --2- 172.21.15.16:0/355778793 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fec2806c2e0 0x7fec2806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:36.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.691+0000 7fec38ff9640 1 -- 172.21.15.16:0/355778793 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6202+0+0 (secure 0 0 0) 0x7fec2c08ab70 con 0x7fec3c06d430 2024-09-19T07:48:36.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.691+0000 7fec3b7fe640 1 --2- 172.21.15.16:0/355778793 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fec2806c2e0 0x7fec2806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:36.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.692+0000 7fec3b7fe640 1 --2- 172.21.15.16:0/355778793 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fec2806c2e0 0x7fec2806e7a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec3c1e9690 tx=0x7fec30008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:36.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.695+0000 7fec38ff9640 1 -- 172.21.15.16:0/355778793 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fec3c0c7140 con 0x7fec3c06d430 2024-09-19T07:48:36.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.905+0000 7fec42a23640 1 -- 172.21.15.16:0/355778793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec3c0690c0 con 0x7fec3c06d430 2024-09-19T07:48:36.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.906+0000 7fec38ff9640 1 -- 172.21.15.16:0/355778793 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5224 (secure 0 0 0) 0x7fec2c059060 con 0x7fec3c06d430 2024-09-19T07:48:36.906 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:36.907 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-19T07:48:36:069113+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/3536366465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3536366465},{"type":"v1","addr":"172.21.15.146:6836","nonce":3536366465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3612374586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3612374586},{"type":"v1","addr":"172.21.15.146:6837","nonce":3612374586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5337,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/781070730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":781070730},{"type":"v1","addr":"172.21.15.16:6833","nonce":781070730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:45:10.494498+0000","modified":"2024-09-19T07:48:35.187029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5340":{"gid":5340,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":54,"addr":"172.21.15.16:6835/642332950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":642332950},{"type":"v1","addr":"172.21.15.16:6835","nonce":642332950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5340,"qdb_cluster":[5340]},"id":3}]} 2024-09-19T07:48:36.907 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-19T07:48:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.908+0000 7fec42a23640 1 -- 172.21.15.16:0/355778793 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fec2806c2e0 msgr2=0x7fec2806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.908+0000 7fec42a23640 1 --2- 172.21.15.16:0/355778793 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fec2806c2e0 0x7fec2806e7a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec3c1e9690 tx=0x7fec30008040 comp rx=0 tx=0).stop 2024-09-19T07:48:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.908+0000 7fec42a23640 1 -- 172.21.15.16:0/355778793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec3c06d430 msgr2=0x7fec3c0cbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.908+0000 7fec42a23640 1 --2- 172.21.15.16:0/355778793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec3c06d430 0x7fec3c0cbca0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fec2c00c3c0 tx=0x7fec2c00c880 comp rx=0 tx=0).stop 2024-09-19T07:48:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.908+0000 7fec42a23640 1 -- 172.21.15.16:0/355778793 shutdown_connections 2024-09-19T07:48:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.908+0000 7fec42a23640 1 --2- 172.21.15.16:0/355778793 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fec2806c2e0 0x7fec2806e7a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.908+0000 7fec42a23640 1 --2- 172.21.15.16:0/355778793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec3c143f30 0x7fec3c0cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.908+0000 7fec42a23640 1 --2- 172.21.15.16:0/355778793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec3c0c8d20 0x7fec3c13fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.908+0000 7fec42a23640 1 --2- 172.21.15.16:0/355778793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec3c06d430 0x7fec3c0cbca0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.908+0000 7fec42a23640 1 -- 172.21.15.16:0/355778793 >> 172.21.15.16:0/355778793 conn(0x7fec3c0b3ed0 msgr2=0x7fec3c15e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.908+0000 7fec42a23640 1 -- 172.21.15.16:0/355778793 shutdown_connections 2024-09-19T07:48:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:36.908+0000 7fec42a23640 1 -- 172.21.15.16:0/355778793 wait complete. 2024-09-19T07:48:36.920 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:48:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.021+0000 7f9a9eae6640 1 Processor -- start 2024-09-19T07:48:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.021+0000 7f9a9eae6640 1 -- start start 2024-09-19T07:48:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.022+0000 7f9a9eae6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a9815e2a0 0x7f9a9806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.022+0000 7f9a9eae6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9a9815e680 0x7f9a980cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.022+0000 7f9a9eae6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a9806d960 0x7f9a980d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.022+0000 7f9a9eae6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9a980d1880 con 0x7f9a9815e2a0 2024-09-19T07:48:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.022+0000 7f9a9eae6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f9a980d1b60 con 0x7f9a9815e680 2024-09-19T07:48:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.022+0000 7f9a9eae6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9a980cf2d0 con 0x7f9a9806d960 2024-09-19T07:48:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.022+0000 7f9a9c882640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a9815e2a0 0x7f9a9806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.022+0000 7f9a9c882640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a9815e2a0 0x7f9a9806d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36168/0 (socket says 172.21.15.16:36168) 2024-09-19T07:48:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.022+0000 7f9a9c882640 1 -- 172.21.15.16:0/2365718756 learned_addr learned my addr 172.21.15.16:0/2365718756 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.022+0000 7f9a97fff640 1 --2- 172.21.15.16:0/2365718756 >> v2:172.21.15.146:3300/0 conn(0x7f9a9815e680 0x7f9a980cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.022+0000 7f9a9d083640 1 --2- 172.21.15.16:0/2365718756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a9806d960 0x7f9a980d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.022+0000 7f9a9c882640 1 -- 172.21.15.16:0/2365718756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a9806d960 msgr2=0x7f9a980d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.022+0000 7f9a9c882640 1 --2- 172.21.15.16:0/2365718756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a9806d960 0x7f9a980d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.022+0000 7f9a9c882640 1 -- 172.21.15.16:0/2365718756 >> v2:172.21.15.146:3300/0 conn(0x7f9a9815e680 msgr2=0x7f9a980cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.022+0000 7f9a9c882640 1 --2- 172.21.15.16:0/2365718756 >> v2:172.21.15.146:3300/0 conn(0x7f9a9815e680 0x7f9a980cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.022+0000 7f9a9c882640 1 -- 172.21.15.16:0/2365718756 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a980cf5b0 con 0x7f9a9815e2a0 2024-09-19T07:48:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.022+0000 7f9a9d083640 1 --2- 172.21.15.16:0/2365718756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a9806d960 0x7f9a980d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.022+0000 7f9a97fff640 1 --2- 172.21.15.16:0/2365718756 >> v2:172.21.15.146:3300/0 conn(0x7f9a9815e680 0x7f9a980cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.023+0000 7f9a9c882640 1 --2- 172.21.15.16:0/2365718756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a9815e2a0 0x7f9a9806d420 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9a980a22a0 tx=0x7f9a8c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d848a324307e4465 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.023+0000 7f9a977fe640 1 -- 172.21.15.16:0/2365718756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c037070 con 0x7f9a9815e2a0 2024-09-19T07:48:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.023+0000 7f9a977fe640 1 -- 172.21.15.16:0/2365718756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7f9a8c032440 con 0x7f9a9815e2a0 2024-09-19T07:48:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.023+0000 7f9a977fe640 1 -- 172.21.15.16:0/2365718756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c032810 con 0x7f9a9815e2a0 2024-09-19T07:48:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.024+0000 7f9a9eae6640 1 -- 172.21.15.16:0/2365718756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a9815e2a0 msgr2=0x7f9a9806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.024+0000 7f9a9eae6640 1 --2- 172.21.15.16:0/2365718756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a9815e2a0 0x7f9a9806d420 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9a980a22a0 tx=0x7f9a8c02a1e0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.024+0000 7f9a9eae6640 1 -- 172.21.15.16:0/2365718756 shutdown_connections 2024-09-19T07:48:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.024+0000 7f9a9eae6640 1 --2- 172.21.15.16:0/2365718756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a9806d960 0x7f9a980d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.024+0000 7f9a9eae6640 1 --2- 172.21.15.16:0/2365718756 >> v2:172.21.15.146:3300/0 conn(0x7f9a9815e680 0x7f9a980cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.024+0000 7f9a9eae6640 1 --2- 172.21.15.16:0/2365718756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a9815e2a0 0x7f9a9806d420 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.024+0000 7f9a9eae6640 1 -- 172.21.15.16:0/2365718756 >> 172.21.15.16:0/2365718756 conn(0x7f9a9809d760 msgr2=0x7f9a9808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.024+0000 7f9a9eae6640 1 -- 172.21.15.16:0/2365718756 shutdown_connections 2024-09-19T07:48:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.024+0000 7f9a9eae6640 1 -- 172.21.15.16:0/2365718756 wait complete. 2024-09-19T07:48:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.025+0000 7f9a9eae6640 1 Processor -- start 2024-09-19T07:48:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.025+0000 7f9a9eae6640 1 -- start start 2024-09-19T07:48:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.026+0000 7f9a9eae6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a9806d960 0x7f9a980c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.026+0000 7f9a9eae6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a9815e2a0 0x7f9a980ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.026+0000 7f9a9eae6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a9815e680 0x7f9a9814d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.026+0000 7f9a9eae6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a980d44f0 con 0x7f9a9815e680 2024-09-19T07:48:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.026+0000 7f9a9eae6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9a980d4370 con 0x7f9a9806d960 2024-09-19T07:48:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.026+0000 7f9a9eae6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9a980d4670 con 0x7f9a9815e2a0 2024-09-19T07:48:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.026+0000 7f9a9d083640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a9815e680 0x7f9a9814d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.026+0000 7f9a9d083640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a9815e680 0x7f9a9814d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36184/0 (socket says 172.21.15.16:36184) 2024-09-19T07:48:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.026+0000 7f9a9d083640 1 -- 172.21.15.16:0/728002444 learned_addr learned my addr 172.21.15.16:0/728002444 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.026+0000 7f9a97fff640 1 --2- 172.21.15.16:0/728002444 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a9815e2a0 0x7f9a980ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.026+0000 7f9a9c882640 1 --2- 172.21.15.16:0/728002444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a9806d960 0x7f9a980c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.026+0000 7f9a9d083640 1 -- 172.21.15.16:0/728002444 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a9815e2a0 msgr2=0x7f9a980ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.026+0000 7f9a9d083640 1 --2- 172.21.15.16:0/728002444 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a9815e2a0 0x7f9a980ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.026+0000 7f9a9d083640 1 -- 172.21.15.16:0/728002444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a9806d960 msgr2=0x7f9a980c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.026+0000 7f9a9d083640 1 --2- 172.21.15.16:0/728002444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a9806d960 0x7f9a980c8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.026+0000 7f9a9d083640 1 -- 172.21.15.16:0/728002444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a980c9300 con 0x7f9a9815e680 2024-09-19T07:48:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.027+0000 7f9a9d083640 1 --2- 172.21.15.16:0/728002444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a9815e680 0x7f9a9814d8a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000cef0 tx=0x7f9a9000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.027+0000 7f9a957fa640 1 -- 172.21.15.16:0/728002444 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9001a070 con 0x7f9a9815e680 2024-09-19T07:48:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.027+0000 7f9a957fa640 1 -- 172.21.15.16:0/728002444 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7f9a9000d860 con 0x7f9a9815e680 2024-09-19T07:48:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.027+0000 7f9a9eae6640 1 -- 172.21.15.16:0/728002444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a980c9020 con 0x7f9a9815e680 2024-09-19T07:48:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.027+0000 7f9a957fa640 1 -- 172.21.15.16:0/728002444 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9000dc50 con 0x7f9a9815e680 2024-09-19T07:48:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.027+0000 7f9a9eae6640 1 -- 172.21.15.16:0/728002444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a980cb830 con 0x7f9a9815e680 2024-09-19T07:48:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.028+0000 7f9a9eae6640 1 -- 172.21.15.16:0/728002444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a980d4a60 con 0x7f9a9815e680 2024-09-19T07:48:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.028+0000 7f9a957fa640 1 -- 172.21.15.16:0/728002444 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9a9001d070 con 0x7f9a9815e680 2024-09-19T07:48:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.029+0000 7f9a957fa640 1 --2- 172.21.15.16:0/728002444 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9a7c06c2b0 0x7f9a7c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.029+0000 7f9a957fa640 1 -- 172.21.15.16:0/728002444 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6202+0+0 (secure 0 0 0) 0x7f9a9008bf70 con 0x7f9a9815e680 2024-09-19T07:48:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.029+0000 7f9a9c882640 1 --2- 172.21.15.16:0/728002444 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9a7c06c2b0 0x7f9a7c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.030+0000 7f9a9c882640 1 --2- 172.21.15.16:0/728002444 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9a7c06c2b0 0x7f9a7c06e770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9a980a22a0 tx=0x7f9a8c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.033+0000 7f9a957fa640 1 -- 172.21.15.16:0/728002444 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9a9005a520 con 0x7f9a9815e680 2024-09-19T07:48:37.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.210+0000 7f9a9eae6640 1 -- 172.21.15.16:0/728002444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a9806cf90 con 0x7f9a9815e680 2024-09-19T07:48:37.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.211+0000 7f9a957fa640 1 -- 172.21.15.16:0/728002444 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+17182 (secure 0 0 0) 0x7f9a90059e20 con 0x7f9a9815e680 2024-09-19T07:48:37.212 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:37.212 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":59,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:48:31.046831+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:45:08.455974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-19T07:45:09.476448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.28","mappings":[{"from":0,"to":6}]},{"pgid":"8.2f","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/2345385426":"2024-09-19T08:48:30.430325+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:48:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.213+0000 7f9a9eae6640 1 -- 172.21.15.16:0/728002444 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9a7c06c2b0 msgr2=0x7f9a7c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.213+0000 7f9a9eae6640 1 --2- 172.21.15.16:0/728002444 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9a7c06c2b0 0x7f9a7c06e770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9a980a22a0 tx=0x7f9a8c02e000 comp rx=0 tx=0).stop 2024-09-19T07:48:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.213+0000 7f9a9eae6640 1 -- 172.21.15.16:0/728002444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a9815e680 msgr2=0x7f9a9814d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.213+0000 7f9a9eae6640 1 --2- 172.21.15.16:0/728002444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a9815e680 0x7f9a9814d8a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000cef0 tx=0x7f9a9000a710 comp rx=0 tx=0).stop 2024-09-19T07:48:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.213+0000 7f9a9eae6640 1 -- 172.21.15.16:0/728002444 shutdown_connections 2024-09-19T07:48:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.213+0000 7f9a9eae6640 1 --2- 172.21.15.16:0/728002444 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9a7c06c2b0 0x7f9a7c06e770 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.213+0000 7f9a9eae6640 1 --2- 172.21.15.16:0/728002444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a9815e680 0x7f9a9814d8a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.213+0000 7f9a9eae6640 1 --2- 172.21.15.16:0/728002444 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a9815e2a0 0x7f9a980ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.213+0000 7f9a9eae6640 1 --2- 172.21.15.16:0/728002444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a9806d960 0x7f9a980c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.213+0000 7f9a9eae6640 1 -- 172.21.15.16:0/728002444 >> 172.21.15.16:0/728002444 conn(0x7f9a9809d760 msgr2=0x7f9a980d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:37.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.213+0000 7f9a9eae6640 1 -- 172.21.15.16:0/728002444 shutdown_connections 2024-09-19T07:48:37.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.213+0000 7f9a9eae6640 1 -- 172.21.15.16:0/728002444 wait complete. 2024-09-19T07:48:37.225 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T07:48:37.225 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:48:37.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.327+0000 7f6a1ad4b640 1 Processor -- start 2024-09-19T07:48:37.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.327+0000 7f6a1ad4b640 1 -- start start 2024-09-19T07:48:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.328+0000 7f6a1ad4b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a14072590 0x7f6a1406d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.328+0000 7f6a1ad4b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6a14141ad0 0x7f6a14141f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.328+0000 7f6a1ad4b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a141620b0 con 0x7f6a140d21f0 2024-09-19T07:48:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.328+0000 7f6a1ad4b640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f6a14141750 con 0x7f6a14141ad0 2024-09-19T07:48:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.328+0000 7f6a1ad4b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6a14162370 con 0x7f6a14072590 2024-09-19T07:48:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.328+0000 7f6a18ae7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6a140d21f0 0x7f6a140d25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56516/0 (socket says 172.21.15.16:56516) 2024-09-19T07:48:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.328+0000 7f6a18ae7640 1 -- 172.21.15.16:0/3174234392 learned_addr learned my addr 172.21.15.16:0/3174234392 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.328+0000 7f6a13fff640 1 --2- 172.21.15.16:0/3174234392 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a14072590 0x7f6a1406d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.328+0000 7f6a192e8640 1 --2- 172.21.15.16:0/3174234392 >> v2:172.21.15.146:3300/0 conn(0x7f6a14141ad0 0x7f6a14141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.328+0000 7f6a137fe640 1 -- 172.21.15.16:0/3174234392 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2325670084 0 0) 0x7f6a141620b0 con 0x7f6a140d21f0 2024-09-19T07:48:37.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.328+0000 7f6a137fe640 1 -- 172.21.15.16:0/3174234392 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a00003520 con 0x7f6a140d21f0 2024-09-19T07:48:37.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.328+0000 7f6a13fff640 1 -- 172.21.15.16:0/3174234392 >> v2:172.21.15.146:3300/0 conn(0x7f6a14141ad0 msgr2=0x7f6a14141f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:37.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.328+0000 7f6a13fff640 1 --2- 172.21.15.16:0/3174234392 >> v2:172.21.15.146:3300/0 conn(0x7f6a14141ad0 0x7f6a14141f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.328+0000 7f6a13fff640 1 -- 172.21.15.16:0/3174234392 >> v1:172.21.15.16:6789/0 conn(0x7f6a140d21f0 legacy=0x7f6a140d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:37.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.328+0000 7f6a13fff640 1 -- 172.21.15.16:0/3174234392 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a14162650 con 0x7f6a14072590 2024-09-19T07:48:37.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.329+0000 7f6a13fff640 1 --2- 172.21.15.16:0/3174234392 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a14072590 0x7f6a1406d0e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0097a0 tx=0x7f69fc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18e796e5b52a70b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:37.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.329+0000 7f6a137fe640 1 -- 172.21.15.16:0/3174234392 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc037070 con 0x7f6a14072590 2024-09-19T07:48:37.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.329+0000 7f6a137fe640 1 -- 172.21.15.16:0/3174234392 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7f69fc02ae50 con 0x7f6a14072590 2024-09-19T07:48:37.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.329+0000 7f6a137fe640 1 -- 172.21.15.16:0/3174234392 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc032640 con 0x7f6a14072590 2024-09-19T07:48:37.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.330+0000 7f6a1ad4b640 1 -- 172.21.15.16:0/3174234392 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a14072590 msgr2=0x7f6a1406d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:37.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.330+0000 7f6a1ad4b640 1 --2- 172.21.15.16:0/3174234392 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a14072590 0x7f6a1406d0e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0097a0 tx=0x7f69fc02a100 comp rx=0 tx=0).stop 2024-09-19T07:48:37.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.330+0000 7f6a1ad4b640 1 -- 172.21.15.16:0/3174234392 shutdown_connections 2024-09-19T07:48:37.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.330+0000 7f6a1ad4b640 1 --2- 172.21.15.16:0/3174234392 >> v2:172.21.15.146:3300/0 conn(0x7f6a14141ad0 0x7f6a14141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.330+0000 7f6a1ad4b640 1 --2- 172.21.15.16:0/3174234392 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a14072590 0x7f6a1406d0e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.330+0000 7f6a1ad4b640 1 -- 172.21.15.16:0/3174234392 >> 172.21.15.16:0/3174234392 conn(0x7f6a140b3ed0 msgr2=0x7f6a140b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:37.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.330+0000 7f6a1ad4b640 1 -- 172.21.15.16:0/3174234392 shutdown_connections 2024-09-19T07:48:37.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.330+0000 7f6a1ad4b640 1 -- 172.21.15.16:0/3174234392 wait complete. 2024-09-19T07:48:37.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.331+0000 7f6a1ad4b640 1 Processor -- start 2024-09-19T07:48:37.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.331+0000 7f6a1ad4b640 1 -- start start 2024-09-19T07:48:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.332+0000 7f6a1ad4b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a14072590 0x7f6a140b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.332+0000 7f6a1ad4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a140d21f0 0x7f6a1415bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.332+0000 7f6a1ad4b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a14141ad0 0x7f6a141404d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.332+0000 7f6a1ad4b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1406d620 con 0x7f6a140d21f0 2024-09-19T07:48:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.332+0000 7f6a1ad4b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6a14162c10 con 0x7f6a14141ad0 2024-09-19T07:48:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.332+0000 7f6a1ad4b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6a1406d7a0 con 0x7f6a14072590 2024-09-19T07:48:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.332+0000 7f6a13fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a140d21f0 0x7f6a1415bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.332+0000 7f6a13fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a140d21f0 0x7f6a1415bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36198/0 (socket says 172.21.15.16:36198) 2024-09-19T07:48:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.332+0000 7f6a13fff640 1 -- 172.21.15.16:0/907207773 learned_addr learned my addr 172.21.15.16:0/907207773 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.332+0000 7f6a18ae7640 1 --2- 172.21.15.16:0/907207773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a14072590 0x7f6a140b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:37.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.332+0000 7f6a192e8640 1 --2- 172.21.15.16:0/907207773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a14141ad0 0x7f6a141404d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:37.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.332+0000 7f6a13fff640 1 -- 172.21.15.16:0/907207773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a14072590 msgr2=0x7f6a140b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:37.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.332+0000 7f6a13fff640 1 --2- 172.21.15.16:0/907207773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a14072590 0x7f6a140b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.332+0000 7f6a13fff640 1 -- 172.21.15.16:0/907207773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a14141ad0 msgr2=0x7f6a141404d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:37.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.332+0000 7f6a13fff640 1 --2- 172.21.15.16:0/907207773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a14141ad0 0x7f6a141404d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.333+0000 7f6a13fff640 1 -- 172.21.15.16:0/907207773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a140b6080 con 0x7f6a140d21f0 2024-09-19T07:48:37.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.333+0000 7f6a13fff640 1 --2- 172.21.15.16:0/907207773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a140d21f0 0x7f6a1415bbb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f69fc02a140 tx=0x7f69fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:37.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.333+0000 7f6a117fa640 1 -- 172.21.15.16:0/907207773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc041070 con 0x7f6a140d21f0 2024-09-19T07:48:37.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.333+0000 7f6a117fa640 1 -- 172.21.15.16:0/907207773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7f69fc03c460 con 0x7f6a140d21f0 2024-09-19T07:48:37.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.333+0000 7f6a1ad4b640 1 -- 172.21.15.16:0/907207773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a140b6640 con 0x7f6a140d21f0 2024-09-19T07:48:37.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.333+0000 7f6a117fa640 1 -- 172.21.15.16:0/907207773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc037040 con 0x7f6a140d21f0 2024-09-19T07:48:37.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.333+0000 7f6a1ad4b640 1 -- 172.21.15.16:0/907207773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a140b6c00 con 0x7f6a140d21f0 2024-09-19T07:48:37.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.334+0000 7f6a1ad4b640 1 -- 172.21.15.16:0/907207773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a140b6430 con 0x7f6a140d21f0 2024-09-19T07:48:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.334+0000 7f6a117fa640 1 -- 172.21.15.16:0/907207773 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f69fc03a070 con 0x7f6a140d21f0 2024-09-19T07:48:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.335+0000 7f6a117fa640 1 --2- 172.21.15.16:0/907207773 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f69f406c2b0 0x7f69f406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.335+0000 7f6a117fa640 1 -- 172.21.15.16:0/907207773 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6202+0+0 (secure 0 0 0) 0x7f69fc0aba50 con 0x7f6a140d21f0 2024-09-19T07:48:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.335+0000 7f6a18ae7640 1 --2- 172.21.15.16:0/907207773 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f69f406c2b0 0x7f69f406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.336+0000 7f6a18ae7640 1 --2- 172.21.15.16:0/907207773 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f69f406c2b0 0x7f69f406e770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c005880 tx=0x7f6a0c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.339+0000 7f6a117fa640 1 -- 172.21.15.16:0/907207773 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f69fc0797d0 con 0x7f6a140d21f0 2024-09-19T07:48:37.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.552+0000 7f6a1ad4b640 1 -- 172.21.15.16:0/907207773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a140690c0 con 0x7f6a140d21f0 2024-09-19T07:48:37.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.553+0000 7f6a117fa640 1 -- 172.21.15.16:0/907207773 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5224 (secure 0 0 0) 0x7f69fc07dad0 con 0x7f6a140d21f0 2024-09-19T07:48:37.554 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:37.554 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-19T07:48:36:069113+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/3536366465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3536366465},{"type":"v1","addr":"172.21.15.146:6836","nonce":3536366465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3612374586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3612374586},{"type":"v1","addr":"172.21.15.146:6837","nonce":3612374586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5337,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/781070730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":781070730},{"type":"v1","addr":"172.21.15.16:6833","nonce":781070730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:45:10.494498+0000","modified":"2024-09-19T07:48:35.187029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5340":{"gid":5340,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":54,"addr":"172.21.15.16:6835/642332950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":642332950},{"type":"v1","addr":"172.21.15.16:6835","nonce":642332950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5340,"qdb_cluster":[5340]},"id":3}]} 2024-09-19T07:48:37.555 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-19T07:48:37.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.555+0000 7f6a1ad4b640 1 -- 172.21.15.16:0/907207773 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f69f406c2b0 msgr2=0x7f69f406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:37.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.555+0000 7f6a1ad4b640 1 --2- 172.21.15.16:0/907207773 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f69f406c2b0 0x7f69f406e770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c005880 tx=0x7f6a0c009040 comp rx=0 tx=0).stop 2024-09-19T07:48:37.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.555+0000 7f6a1ad4b640 1 -- 172.21.15.16:0/907207773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a140d21f0 msgr2=0x7f6a1415bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:37.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.555+0000 7f6a1ad4b640 1 --2- 172.21.15.16:0/907207773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a140d21f0 0x7f6a1415bbb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f69fc02a140 tx=0x7f69fc0046c0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.555+0000 7f6a1ad4b640 1 -- 172.21.15.16:0/907207773 shutdown_connections 2024-09-19T07:48:37.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.555+0000 7f6a1ad4b640 1 --2- 172.21.15.16:0/907207773 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f69f406c2b0 0x7f69f406e770 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.555+0000 7f6a1ad4b640 1 --2- 172.21.15.16:0/907207773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a14141ad0 0x7f6a141404d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.555+0000 7f6a1ad4b640 1 --2- 172.21.15.16:0/907207773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a140d21f0 0x7f6a1415bbb0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.555+0000 7f6a1ad4b640 1 --2- 172.21.15.16:0/907207773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a14072590 0x7f6a140b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.555+0000 7f6a1ad4b640 1 -- 172.21.15.16:0/907207773 >> 172.21.15.16:0/907207773 conn(0x7f6a140b3ed0 msgr2=0x7f6a140b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:37.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.555+0000 7f6a1ad4b640 1 -- 172.21.15.16:0/907207773 shutdown_connections 2024-09-19T07:48:37.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.555+0000 7f6a1ad4b640 1 -- 172.21.15.16:0/907207773 wait complete. 2024-09-19T07:48:37.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:48:37.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.669+0000 7f72dc84c640 1 Processor -- start 2024-09-19T07:48:37.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.670+0000 7f72dc84c640 1 -- start start 2024-09-19T07:48:37.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.670+0000 7f72dc84c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f72d815a7b0 0x7f72d80d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:37.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.670+0000 7f72dc84c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f72d80cc930 0x7f72d80ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:37.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.670+0000 7f72dc84c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72d80b4540 con 0x7f72d80c5b00 2024-09-19T07:48:37.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.670+0000 7f72dc84c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f72d80b4870 con 0x7f72d80cc930 2024-09-19T07:48:37.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.670+0000 7f72dc84c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f72d80c5ee0 con 0x7f72d815a7b0 2024-09-19T07:48:37.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.671+0000 7f72d6d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f72d80cc930 0x7f72d80ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:37.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.671+0000 7f72d6d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f72d80cc930 0x7f72d80ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36214/0 (socket says 172.21.15.16:36214) 2024-09-19T07:48:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.671+0000 7f72d6d9d640 1 -- 172.21.15.16:0/3517815803 learned_addr learned my addr 172.21.15.16:0/3517815803 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.671+0000 7f72d5d9b640 1 --2- 172.21.15.16:0/3517815803 >> v2:172.21.15.146:3300/0 conn(0x7f72d815a7b0 0x7f72d80d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.671+0000 7f72d6d9d640 1 -- 172.21.15.16:0/3517815803 >> v2:172.21.15.146:3300/0 conn(0x7f72d815a7b0 msgr2=0x7f72d80d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.671+0000 7f72d6d9d640 1 --2- 172.21.15.16:0/3517815803 >> v2:172.21.15.146:3300/0 conn(0x7f72d815a7b0 0x7f72d80d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.671+0000 7f72d6d9d640 1 -- 172.21.15.16:0/3517815803 >> v1:172.21.15.146:6789/0 conn(0x7f72d80c5b00 legacy=0x7f72d80cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.671+0000 7f72d6d9d640 1 -- 172.21.15.16:0/3517815803 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d815abf0 con 0x7f72d80cc930 2024-09-19T07:48:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.672+0000 7f72d6d9d640 1 --2- 172.21.15.16:0/3517815803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d80cc930 0x7f72d80ccdc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f72d80cdbf0 tx=0x7f72d002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f443e0bcd7a37a53 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.672+0000 7f72d559a640 1 -- 172.21.15.16:0/3517815803 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d0036070 con 0x7f72d80cc930 2024-09-19T07:48:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.672+0000 7f72d559a640 1 -- 172.21.15.16:0/3517815803 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7f72d0031600 con 0x7f72d80cc930 2024-09-19T07:48:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.672+0000 7f72d559a640 1 -- 172.21.15.16:0/3517815803 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d00319d0 con 0x7f72d80cc930 2024-09-19T07:48:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.672+0000 7f72dc84c640 1 -- 172.21.15.16:0/3517815803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d80cc930 msgr2=0x7f72d80ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.672+0000 7f72dc84c640 1 --2- 172.21.15.16:0/3517815803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d80cc930 0x7f72d80ccdc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f72d80cdbf0 tx=0x7f72d002a6f0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.672+0000 7f72dc84c640 1 -- 172.21.15.16:0/3517815803 shutdown_connections 2024-09-19T07:48:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.672+0000 7f72dc84c640 1 --2- 172.21.15.16:0/3517815803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d80cc930 0x7f72d80ccdc0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.672+0000 7f72dc84c640 1 --2- 172.21.15.16:0/3517815803 >> v2:172.21.15.146:3300/0 conn(0x7f72d815a7b0 0x7f72d80d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.672+0000 7f72dc84c640 1 -- 172.21.15.16:0/3517815803 >> 172.21.15.16:0/3517815803 conn(0x7f72d806bdf0 msgr2=0x7f72d806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.672+0000 7f72dc84c640 1 -- 172.21.15.16:0/3517815803 shutdown_connections 2024-09-19T07:48:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.672+0000 7f72dc84c640 1 -- 172.21.15.16:0/3517815803 wait complete. 2024-09-19T07:48:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.674+0000 7f72dc84c640 1 Processor -- start 2024-09-19T07:48:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.674+0000 7f72dc84c640 1 -- start start 2024-09-19T07:48:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.674+0000 7f72dc84c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d80c5b00 0x7f72d80af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72dc84c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72d80cc930 0x7f72d8150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72dc84c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72d815a7b0 0x7f72d8167d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72dc84c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72d80cff70 con 0x7f72d80c5b00 2024-09-19T07:48:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72dc84c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f72d80cfc90 con 0x7f72d80cc930 2024-09-19T07:48:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72dc84c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f72d80ca8c0 con 0x7f72d815a7b0 2024-09-19T07:48:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72d659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d80c5b00 0x7f72d80af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72d659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d80c5b00 0x7f72d80af290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36220/0 (socket says 172.21.15.16:36220) 2024-09-19T07:48:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72d659c640 1 -- 172.21.15.16:0/1963905370 learned_addr learned my addr 172.21.15.16:0/1963905370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72d5d9b640 1 --2- 172.21.15.16:0/1963905370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72d80cc930 0x7f72d8150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72d6d9d640 1 --2- 172.21.15.16:0/1963905370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72d815a7b0 0x7f72d8167d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72d659c640 1 -- 172.21.15.16:0/1963905370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72d815a7b0 msgr2=0x7f72d8167d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72d659c640 1 --2- 172.21.15.16:0/1963905370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72d815a7b0 0x7f72d8167d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72d659c640 1 -- 172.21.15.16:0/1963905370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72d80cc930 msgr2=0x7f72d8150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72d659c640 1 --2- 172.21.15.16:0/1963905370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72d80cc930 0x7f72d8150210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72d659c640 1 -- 172.21.15.16:0/1963905370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d80ae1f0 con 0x7f72d80c5b00 2024-09-19T07:48:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72d5d9b640 1 --2- 172.21.15.16:0/1963905370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72d80cc930 0x7f72d8150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72d659c640 1 --2- 172.21.15.16:0/1963905370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d80c5b00 0x7f72d80af290 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f72c800c430 tx=0x7f72c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72c6ffd640 1 -- 172.21.15.16:0/1963905370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8011790 con 0x7f72d80c5b00 2024-09-19T07:48:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72c6ffd640 1 -- 172.21.15.16:0/1963905370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7f72c8017070 con 0x7f72d80c5b00 2024-09-19T07:48:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72dc84c640 1 -- 172.21.15.16:0/1963905370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d80ae4d0 con 0x7f72d80c5b00 2024-09-19T07:48:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72c6ffd640 1 -- 172.21.15.16:0/1963905370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8011bb0 con 0x7f72d80c5b00 2024-09-19T07:48:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.675+0000 7f72dc84c640 1 -- 172.21.15.16:0/1963905370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d80aea90 con 0x7f72d80c5b00 2024-09-19T07:48:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.676+0000 7f72dc84c640 1 -- 172.21.15.16:0/1963905370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72d80ca6a0 con 0x7f72d80c5b00 2024-09-19T07:48:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.677+0000 7f72c6ffd640 1 -- 172.21.15.16:0/1963905370 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f72c801c460 con 0x7f72d80c5b00 2024-09-19T07:48:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.677+0000 7f72c6ffd640 1 --2- 172.21.15.16:0/1963905370 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f72b806c2e0 0x7f72b806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.678+0000 7f72c6ffd640 1 -- 172.21.15.16:0/1963905370 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6202+0+0 (secure 0 0 0) 0x7f72c808b560 con 0x7f72d80c5b00 2024-09-19T07:48:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.678+0000 7f72d5d9b640 1 --2- 172.21.15.16:0/1963905370 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f72b806c2e0 0x7f72b806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.678+0000 7f72d5d9b640 1 --2- 172.21.15.16:0/1963905370 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f72b806c2e0 0x7f72b806e7a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f72cc006fd0 tx=0x7f72cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:37.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.682+0000 7f72c6ffd640 1 -- 172.21.15.16:0/1963905370 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f72d80ca6a0 con 0x7f72d80c5b00 2024-09-19T07:48:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.890+0000 7f72dc84c640 1 -- 172.21.15.16:0/1963905370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72d80ccdc0 con 0x7f72d80c5b00 2024-09-19T07:48:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.891+0000 7f72c6ffd640 1 -- 172.21.15.16:0/1963905370 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5224 (secure 0 0 0) 0x7f72c8059a50 con 0x7f72d80c5b00 2024-09-19T07:48:37.892 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:37.892 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-19T07:48:36:069113+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/3536366465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3536366465},{"type":"v1","addr":"172.21.15.146:6836","nonce":3536366465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3612374586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3612374586},{"type":"v1","addr":"172.21.15.146:6837","nonce":3612374586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5337,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/781070730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":781070730},{"type":"v1","addr":"172.21.15.16:6833","nonce":781070730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:45:10.494498+0000","modified":"2024-09-19T07:48:35.187029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5340":{"gid":5340,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":54,"addr":"172.21.15.16:6835/642332950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":642332950},{"type":"v1","addr":"172.21.15.16:6835","nonce":642332950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5340,"qdb_cluster":[5340]},"id":3}]} 2024-09-19T07:48:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-19T07:48:37.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.893+0000 7f72dc84c640 1 -- 172.21.15.16:0/1963905370 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f72b806c2e0 msgr2=0x7f72b806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:37.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.893+0000 7f72dc84c640 1 --2- 172.21.15.16:0/1963905370 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f72b806c2e0 0x7f72b806e7a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f72cc006fd0 tx=0x7f72cc008040 comp rx=0 tx=0).stop 2024-09-19T07:48:37.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.893+0000 7f72dc84c640 1 -- 172.21.15.16:0/1963905370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d80c5b00 msgr2=0x7f72d80af290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:37.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.893+0000 7f72dc84c640 1 --2- 172.21.15.16:0/1963905370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d80c5b00 0x7f72d80af290 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f72c800c430 tx=0x7f72c800c8f0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.893+0000 7f72dc84c640 1 -- 172.21.15.16:0/1963905370 shutdown_connections 2024-09-19T07:48:37.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.893+0000 7f72dc84c640 1 --2- 172.21.15.16:0/1963905370 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f72b806c2e0 0x7f72b806e7a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.893+0000 7f72dc84c640 1 --2- 172.21.15.16:0/1963905370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72d815a7b0 0x7f72d8167d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.893+0000 7f72dc84c640 1 --2- 172.21.15.16:0/1963905370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72d80cc930 0x7f72d8150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.893+0000 7f72dc84c640 1 --2- 172.21.15.16:0/1963905370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d80c5b00 0x7f72d80af290 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.893+0000 7f72dc84c640 1 -- 172.21.15.16:0/1963905370 >> 172.21.15.16:0/1963905370 conn(0x7f72d806bdf0 msgr2=0x7f72d80cab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.893+0000 7f72dc84c640 1 -- 172.21.15.16:0/1963905370 shutdown_connections 2024-09-19T07:48:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:37.893+0000 7f72dc84c640 1 -- 172.21.15.16:0/1963905370 wait complete. 2024-09-19T07:48:37.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:48:38.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.009+0000 7fc0f5ed9640 1 Processor -- start 2024-09-19T07:48:38.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.009+0000 7fc0f5ed9640 1 -- start start 2024-09-19T07:48:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.010+0000 7fc0f5ed9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0f015e2a0 0x7fc0f006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.010+0000 7fc0f5ed9640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc0f015e680 0x7fc0f00cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.010+0000 7fc0f5ed9640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0f00d1880 con 0x7fc0f006d960 2024-09-19T07:48:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.010+0000 7fc0f5ed9640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc0f00d1b60 con 0x7fc0f015e680 2024-09-19T07:48:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.010+0000 7fc0f5ed9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc0f00cf2d0 con 0x7fc0f015e2a0 2024-09-19T07:48:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.010+0000 7fc0eeffd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc0f015e680 0x7fc0f00cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.010+0000 7fc0ef7fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0f015e2a0 0x7fc0f006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.010+0000 7fc0effff640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fc0f006d960 0x7fc0f00d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:47092/0 (socket says 172.21.15.16:47092) 2024-09-19T07:48:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.010+0000 7fc0effff640 1 -- 172.21.15.16:0/649367826 learned_addr learned my addr 172.21.15.16:0/649367826 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.010+0000 7fc0ef7fe640 1 -- 172.21.15.16:0/649367826 >> v2:172.21.15.146:3300/0 conn(0x7fc0f015e680 msgr2=0x7fc0f00cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.010+0000 7fc0ef7fe640 1 --2- 172.21.15.16:0/649367826 >> v2:172.21.15.146:3300/0 conn(0x7fc0f015e680 0x7fc0f00cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.010+0000 7fc0ef7fe640 1 -- 172.21.15.16:0/649367826 >> v1:172.21.15.146:6789/0 conn(0x7fc0f006d960 legacy=0x7fc0f00d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.010+0000 7fc0ee7fc640 1 -- 172.21.15.16:0/649367826 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 718837457 0 0) 0x7fc0f00d1880 con 0x7fc0f006d960 2024-09-19T07:48:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.010+0000 7fc0ef7fe640 1 -- 172.21.15.16:0/649367826 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f00cf5b0 con 0x7fc0f015e2a0 2024-09-19T07:48:38.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.011+0000 7fc0ef7fe640 1 --2- 172.21.15.16:0/649367826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0f015e2a0 0x7fc0f006d420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00097a0 tx=0x7fc0e002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67d3dead6e32d966 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:38.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.011+0000 7fc0ee7fc640 1 -- 172.21.15.16:0/649367826 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0037070 con 0x7fc0f015e2a0 2024-09-19T07:48:38.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.011+0000 7fc0ee7fc640 1 -- 172.21.15.16:0/649367826 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7fc0e0032440 con 0x7fc0f015e2a0 2024-09-19T07:48:38.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.011+0000 7fc0ee7fc640 1 -- 172.21.15.16:0/649367826 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0032810 con 0x7fc0f015e2a0 2024-09-19T07:48:38.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.012+0000 7fc0f5ed9640 1 -- 172.21.15.16:0/649367826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0f015e2a0 msgr2=0x7fc0f006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:38.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.012+0000 7fc0f5ed9640 1 --2- 172.21.15.16:0/649367826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0f015e2a0 0x7fc0f006d420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00097a0 tx=0x7fc0e002a180 comp rx=0 tx=0).stop 2024-09-19T07:48:38.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.012+0000 7fc0f5ed9640 1 -- 172.21.15.16:0/649367826 shutdown_connections 2024-09-19T07:48:38.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.012+0000 7fc0f5ed9640 1 --2- 172.21.15.16:0/649367826 >> v2:172.21.15.146:3300/0 conn(0x7fc0f015e680 0x7fc0f00cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.012+0000 7fc0f5ed9640 1 --2- 172.21.15.16:0/649367826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0f015e2a0 0x7fc0f006d420 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.012+0000 7fc0f5ed9640 1 -- 172.21.15.16:0/649367826 >> 172.21.15.16:0/649367826 conn(0x7fc0f009d760 msgr2=0x7fc0f008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:38.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.012+0000 7fc0f5ed9640 1 -- 172.21.15.16:0/649367826 shutdown_connections 2024-09-19T07:48:38.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.012+0000 7fc0f5ed9640 1 -- 172.21.15.16:0/649367826 wait complete. 2024-09-19T07:48:38.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.013+0000 7fc0f5ed9640 1 Processor -- start 2024-09-19T07:48:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.013+0000 7fc0f5ed9640 1 -- start start 2024-09-19T07:48:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.014+0000 7fc0f5ed9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0f006d960 0x7fc0f01f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.014+0000 7fc0f5ed9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0f015e2a0 0x7fc0f00c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.014+0000 7fc0f5ed9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0f015e680 0x7fc0f0156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.014+0000 7fc0f5ed9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f00d4c00 con 0x7fc0f006d960 2024-09-19T07:48:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.014+0000 7fc0f5ed9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f00d4a80 con 0x7fc0f015e2a0 2024-09-19T07:48:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.014+0000 7fc0f5ed9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc0f00d0010 con 0x7fc0f015e680 2024-09-19T07:48:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.014+0000 7fc0ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0f006d960 0x7fc0f01f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.014+0000 7fc0ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0f006d960 0x7fc0f01f43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36234/0 (socket says 172.21.15.16:36234) 2024-09-19T07:48:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.014+0000 7fc0ef7fe640 1 -- 172.21.15.16:0/2995082183 learned_addr learned my addr 172.21.15.16:0/2995082183 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:38.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.014+0000 7fc0effff640 1 --2- 172.21.15.16:0/2995082183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0f015e680 0x7fc0f0156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.014+0000 7fc0eeffd640 1 --2- 172.21.15.16:0/2995082183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0f015e2a0 0x7fc0f00c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.015+0000 7fc0ef7fe640 1 -- 172.21.15.16:0/2995082183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0f015e680 msgr2=0x7fc0f0156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.015+0000 7fc0ef7fe640 1 --2- 172.21.15.16:0/2995082183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0f015e680 0x7fc0f0156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.015+0000 7fc0ef7fe640 1 -- 172.21.15.16:0/2995082183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0f015e2a0 msgr2=0x7fc0f00c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.015+0000 7fc0ef7fe640 1 --2- 172.21.15.16:0/2995082183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0f015e2a0 0x7fc0f00c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.015+0000 7fc0ef7fe640 1 -- 172.21.15.16:0/2995082183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f00c85f0 con 0x7fc0f006d960 2024-09-19T07:48:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.015+0000 7fc0effff640 1 --2- 172.21.15.16:0/2995082183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0f015e680 0x7fc0f0156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.015+0000 7fc0ef7fe640 1 --2- 172.21.15.16:0/2995082183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0f006d960 0x7fc0f01f43b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0035040 tx=0x7fc0e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.015+0000 7fc0e7fff640 1 -- 172.21.15.16:0/2995082183 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0037040 con 0x7fc0f006d960 2024-09-19T07:48:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.015+0000 7fc0e7fff640 1 -- 172.21.15.16:0/2995082183 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7fc0e0004230 con 0x7fc0f006d960 2024-09-19T07:48:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.015+0000 7fc0f5ed9640 1 -- 172.21.15.16:0/2995082183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f00c88d0 con 0x7fc0f006d960 2024-09-19T07:48:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.015+0000 7fc0e7fff640 1 -- 172.21.15.16:0/2995082183 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0003610 con 0x7fc0f006d960 2024-09-19T07:48:38.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.015+0000 7fc0f5ed9640 1 -- 172.21.15.16:0/2995082183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f00c8e90 con 0x7fc0f006d960 2024-09-19T07:48:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.016+0000 7fc0f5ed9640 1 -- 172.21.15.16:0/2995082183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f006d420 con 0x7fc0f006d960 2024-09-19T07:48:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.017+0000 7fc0e7fff640 1 -- 172.21.15.16:0/2995082183 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc0e00037b0 con 0x7fc0f006d960 2024-09-19T07:48:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.017+0000 7fc0e7fff640 1 --2- 172.21.15.16:0/2995082183 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc0d006c2b0 0x7fc0d006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:38.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.017+0000 7fc0e7fff640 1 -- 172.21.15.16:0/2995082183 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6202+0+0 (secure 0 0 0) 0x7fc0e00ac920 con 0x7fc0f006d960 2024-09-19T07:48:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.017+0000 7fc0eeffd640 1 --2- 172.21.15.16:0/2995082183 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc0d006c2b0 0x7fc0d006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.018+0000 7fc0eeffd640 1 --2- 172.21.15.16:0/2995082183 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc0d006c2b0 0x7fc0d006e770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00d2e60 tx=0x7fc0d8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:38.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.021+0000 7fc0e7fff640 1 -- 172.21.15.16:0/2995082183 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc0e007af30 con 0x7fc0f006d960 2024-09-19T07:48:38.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:48:38.068 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:48:38.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.202+0000 7fc0f5ed9640 1 -- 172.21.15.16:0/2995082183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0f015da10 con 0x7fc0f006d960 2024-09-19T07:48:38.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.204+0000 7fc0e7fff640 1 -- 172.21.15.16:0/2995082183 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+17182 (secure 0 0 0) 0x7fc0e007a830 con 0x7fc0f006d960 2024-09-19T07:48:38.204 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:38.204 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":59,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:48:31.046831+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:45:08.455974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-19T07:45:09.476448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.28","mappings":[{"from":0,"to":6}]},{"pgid":"8.2f","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/2345385426":"2024-09-19T08:48:30.430325+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:48:38.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.206+0000 7fc0f5ed9640 1 -- 172.21.15.16:0/2995082183 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc0d006c2b0 msgr2=0x7fc0d006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:38.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.206+0000 7fc0f5ed9640 1 --2- 172.21.15.16:0/2995082183 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc0d006c2b0 0x7fc0d006e770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00d2e60 tx=0x7fc0d8009190 comp rx=0 tx=0).stop 2024-09-19T07:48:38.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.206+0000 7fc0f5ed9640 1 -- 172.21.15.16:0/2995082183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0f006d960 msgr2=0x7fc0f01f43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.206+0000 7fc0f5ed9640 1 --2- 172.21.15.16:0/2995082183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0f006d960 0x7fc0f01f43b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0035040 tx=0x7fc0e00046c0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.206+0000 7fc0f5ed9640 1 -- 172.21.15.16:0/2995082183 shutdown_connections 2024-09-19T07:48:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.206+0000 7fc0f5ed9640 1 --2- 172.21.15.16:0/2995082183 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc0d006c2b0 0x7fc0d006e770 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.206+0000 7fc0f5ed9640 1 --2- 172.21.15.16:0/2995082183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0f015e680 0x7fc0f0156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.206+0000 7fc0f5ed9640 1 --2- 172.21.15.16:0/2995082183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0f015e2a0 0x7fc0f00c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.206+0000 7fc0f5ed9640 1 --2- 172.21.15.16:0/2995082183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0f006d960 0x7fc0f01f43b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.206+0000 7fc0f5ed9640 1 -- 172.21.15.16:0/2995082183 >> 172.21.15.16:0/2995082183 conn(0x7fc0f009d760 msgr2=0x7fc0f015cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.206+0000 7fc0f5ed9640 1 -- 172.21.15.16:0/2995082183 shutdown_connections 2024-09-19T07:48:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.206+0000 7fc0f5ed9640 1 -- 172.21.15.16:0/2995082183 wait complete. 2024-09-19T07:48:38.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T07:48:38.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.321+0000 7f9f42c0e640 1 Processor -- start 2024-09-19T07:48:38.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.321+0000 7f9f42c0e640 1 -- start start 2024-09-19T07:48:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.321+0000 7f9f42c0e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f3c0d0070 0x7f9f3c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.322+0000 7f9f42c0e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f3c0d4130 0x7f9f3c0d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.322+0000 7f9f42c0e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f3c0d4ae0 con 0x7f9f3c15f0d0 2024-09-19T07:48:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.322+0000 7f9f42c0e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9f3c0d1b10 con 0x7f9f3c0d0070 2024-09-19T07:48:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.322+0000 7f9f42c0e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3c0cf2b0 con 0x7f9f3c0d4130 2024-09-19T07:48:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.322+0000 7f9f3bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f3c0d0070 0x7f9f3c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.322+0000 7f9f3bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f3c0d0070 0x7f9f3c15f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36240/0 (socket says 172.21.15.16:36240) 2024-09-19T07:48:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.322+0000 7f9f3bfff640 1 -- 172.21.15.16:0/3601337287 learned_addr learned my addr 172.21.15.16:0/3601337287 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.322+0000 7f9f3bfff640 1 -- 172.21.15.16:0/3601337287 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f3c0d4130 msgr2=0x7f9f3c0d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.322+0000 7f9f411ab640 1 --2- 172.21.15.16:0/3601337287 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f3c0d4130 0x7f9f3c0d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.322+0000 7f9f3bfff640 1 --2- 172.21.15.16:0/3601337287 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f3c0d4130 0x7f9f3c0d45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.322+0000 7f9f3bfff640 1 -- 172.21.15.16:0/3601337287 >> v1:172.21.15.146:6789/0 conn(0x7f9f3c15f0d0 legacy=0x7f9f3c0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.322+0000 7f9f3bfff640 1 -- 172.21.15.16:0/3601337287 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c0cf590 con 0x7f9f3c0d0070 2024-09-19T07:48:38.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.323+0000 7f9f3bfff640 1 --2- 172.21.15.16:0/3601337287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f3c0d0070 0x7f9f3c15f4b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009870 tx=0x7f9f2c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b195a9f0ed521f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:38.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.323+0000 7f9f3b7fe640 1 -- 172.21.15.16:0/3601337287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c037070 con 0x7f9f3c0d0070 2024-09-19T07:48:38.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.323+0000 7f9f3b7fe640 1 -- 172.21.15.16:0/3601337287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7f9f2c032440 con 0x7f9f3c0d0070 2024-09-19T07:48:38.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.323+0000 7f9f3b7fe640 1 -- 172.21.15.16:0/3601337287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c032810 con 0x7f9f3c0d0070 2024-09-19T07:48:38.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.323+0000 7f9f42c0e640 1 -- 172.21.15.16:0/3601337287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f3c0d0070 msgr2=0x7f9f3c15f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:38.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.323+0000 7f9f42c0e640 1 --2- 172.21.15.16:0/3601337287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f3c0d0070 0x7f9f3c15f4b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009870 tx=0x7f9f2c02a180 comp rx=0 tx=0).stop 2024-09-19T07:48:38.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.323+0000 7f9f42c0e640 1 -- 172.21.15.16:0/3601337287 shutdown_connections 2024-09-19T07:48:38.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.323+0000 7f9f42c0e640 1 --2- 172.21.15.16:0/3601337287 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f3c0d4130 0x7f9f3c0d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.323+0000 7f9f42c0e640 1 --2- 172.21.15.16:0/3601337287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f3c0d0070 0x7f9f3c15f4b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.323+0000 7f9f42c0e640 1 -- 172.21.15.16:0/3601337287 >> 172.21.15.16:0/3601337287 conn(0x7f9f3c09d760 msgr2=0x7f9f3c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:38.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.324+0000 7f9f42c0e640 1 -- 172.21.15.16:0/3601337287 shutdown_connections 2024-09-19T07:48:38.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.324+0000 7f9f42c0e640 1 -- 172.21.15.16:0/3601337287 wait complete. 2024-09-19T07:48:38.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.325+0000 7f9f42c0e640 1 Processor -- start 2024-09-19T07:48:38.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.325+0000 7f9f42c0e640 1 -- start start 2024-09-19T07:48:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f42c0e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f3c0d0070 0x7f9f3c15ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f42c0e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f3c0d4130 0x7f9f3c160f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f42c0e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f3c15f0d0 0x7f9f3c1f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f42c0e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c0cef60 con 0x7f9f3c0d0070 2024-09-19T07:48:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f42c0e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c0cfb50 con 0x7f9f3c0d4130 2024-09-19T07:48:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f42c0e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3c0b2f70 con 0x7f9f3c15f0d0 2024-09-19T07:48:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f409aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f3c0d0070 0x7f9f3c15ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f409aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f3c0d0070 0x7f9f3c15ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36246/0 (socket says 172.21.15.16:36246) 2024-09-19T07:48:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f409aa640 1 -- 172.21.15.16:0/20096113 learned_addr learned my addr 172.21.15.16:0/20096113 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f3bfff640 1 --2- 172.21.15.16:0/20096113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f3c0d4130 0x7f9f3c160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f411ab640 1 --2- 172.21.15.16:0/20096113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f3c15f0d0 0x7f9f3c1f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f409aa640 1 -- 172.21.15.16:0/20096113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f3c15f0d0 msgr2=0x7f9f3c1f7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f409aa640 1 --2- 172.21.15.16:0/20096113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f3c15f0d0 0x7f9f3c1f7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f409aa640 1 -- 172.21.15.16:0/20096113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f3c0d4130 msgr2=0x7f9f3c160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f409aa640 1 --2- 172.21.15.16:0/20096113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f3c0d4130 0x7f9f3c160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f409aa640 1 -- 172.21.15.16:0/20096113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c1f83c0 con 0x7f9f3c0d0070 2024-09-19T07:48:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f3bfff640 1 --2- 172.21.15.16:0/20096113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f3c0d4130 0x7f9f3c160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f411ab640 1 --2- 172.21.15.16:0/20096113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f3c15f0d0 0x7f9f3c1f7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f409aa640 1 --2- 172.21.15.16:0/20096113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f3c0d0070 0x7f9f3c15ba80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400b3f0 tx=0x7f9f2400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f397fa640 1 -- 172.21.15.16:0/20096113 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24015020 con 0x7f9f3c0d0070 2024-09-19T07:48:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f397fa640 1 -- 172.21.15.16:0/20096113 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7f9f24011830 con 0x7f9f3c0d0070 2024-09-19T07:48:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f42c0e640 1 -- 172.21.15.16:0/20096113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3c1f86a0 con 0x7f9f3c0d0070 2024-09-19T07:48:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f397fa640 1 -- 172.21.15.16:0/20096113 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24011c20 con 0x7f9f3c0d0070 2024-09-19T07:48:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.326+0000 7f9f42c0e640 1 -- 172.21.15.16:0/20096113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f3c1614a0 con 0x7f9f3c0d0070 2024-09-19T07:48:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.327+0000 7f9f42c0e640 1 -- 172.21.15.16:0/20096113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f3c0d45a0 con 0x7f9f3c0d0070 2024-09-19T07:48:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.328+0000 7f9f397fa640 1 -- 172.21.15.16:0/20096113 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9f24018020 con 0x7f9f3c0d0070 2024-09-19T07:48:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.328+0000 7f9f397fa640 1 --2- 172.21.15.16:0/20096113 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9f1c06c2b0 0x7f9f1c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.328+0000 7f9f397fa640 1 -- 172.21.15.16:0/20096113 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6202+0+0 (secure 0 0 0) 0x7f9f2408b210 con 0x7f9f3c0d0070 2024-09-19T07:48:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.329+0000 7f9f3bfff640 1 --2- 172.21.15.16:0/20096113 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9f1c06c2b0 0x7f9f1c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.329+0000 7f9f3bfff640 1 --2- 172.21.15.16:0/20096113 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9f1c06c2b0 0x7f9f1c06e770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009870 tx=0x7f9f2c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.333+0000 7f9f397fa640 1 -- 172.21.15.16:0/20096113 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9f2401f030 con 0x7f9f3c0d0070 2024-09-19T07:48:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.544+0000 7f9f42c0e640 1 -- 172.21.15.16:0/20096113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9f3c08c020 con 0x7f9f3c0d0070 2024-09-19T07:48:38.545 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-19T07:48:38.544+0000 7f6950fea640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:48:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.546+0000 7f9f397fa640 1 -- 172.21.15.16:0/20096113 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f9f240172c0 con 0x7f9f3c0d0070 2024-09-19T07:48:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T07:48:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.549+0000 7f9f42c0e640 1 -- 172.21.15.16:0/20096113 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9f1c06c2b0 msgr2=0x7f9f1c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.549+0000 7f9f42c0e640 1 --2- 172.21.15.16:0/20096113 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9f1c06c2b0 0x7f9f1c06e770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009870 tx=0x7f9f2c02e000 comp rx=0 tx=0).stop 2024-09-19T07:48:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.549+0000 7f9f42c0e640 1 -- 172.21.15.16:0/20096113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f3c0d0070 msgr2=0x7f9f3c15ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.549+0000 7f9f42c0e640 1 --2- 172.21.15.16:0/20096113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f3c0d0070 0x7f9f3c15ba80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400b3f0 tx=0x7f9f2400b8b0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.550+0000 7f9f42c0e640 1 -- 172.21.15.16:0/20096113 shutdown_connections 2024-09-19T07:48:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.550+0000 7f9f42c0e640 1 --2- 172.21.15.16:0/20096113 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9f1c06c2b0 0x7f9f1c06e770 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.550+0000 7f9f42c0e640 1 --2- 172.21.15.16:0/20096113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f3c15f0d0 0x7f9f3c1f7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.550+0000 7f9f42c0e640 1 --2- 172.21.15.16:0/20096113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f3c0d4130 0x7f9f3c160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.550+0000 7f9f42c0e640 1 --2- 172.21.15.16:0/20096113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f3c0d0070 0x7f9f3c15ba80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.550+0000 7f9f42c0e640 1 -- 172.21.15.16:0/20096113 >> 172.21.15.16:0/20096113 conn(0x7f9f3c09d760 msgr2=0x7f9f3c15dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.550+0000 7f9f42c0e640 1 -- 172.21.15.16:0/20096113 shutdown_connections 2024-09-19T07:48:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.550+0000 7f9f42c0e640 1 -- 172.21.15.16:0/20096113 wait complete. 2024-09-19T07:48:38.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T07:48:38.595 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-19T07:48:38.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.668+0000 7fa971406640 1 Processor -- start 2024-09-19T07:48:38.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.668+0000 7fa971406640 1 -- start start 2024-09-19T07:48:38.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.669+0000 7fa971406640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa96c15e680 0x7fa96c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:38.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.669+0000 7fa971406640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa96c0d0010 0x7fa96c0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:38.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.669+0000 7fa971406640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa96c0d1880 con 0x7fa96c15e2a0 2024-09-19T07:48:38.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.669+0000 7fa971406640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa96c0d1b60 con 0x7fa96c0d0010 2024-09-19T07:48:38.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.669+0000 7fa971406640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa96c0cf2d0 con 0x7fa96c15e680 2024-09-19T07:48:38.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.669+0000 7fa96b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa96c0d0010 0x7fa96c0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:38.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.669+0000 7fa96b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa96c0d0010 0x7fa96c0d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36278/0 (socket says 172.21.15.16:36278) 2024-09-19T07:48:38.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.669+0000 7fa96b7fe640 1 -- 172.21.15.16:0/2671897040 learned_addr learned my addr 172.21.15.16:0/2671897040 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:38.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.669+0000 7fa96a7fc640 1 --2- 172.21.15.16:0/2671897040 >> v2:172.21.15.146:3300/0 conn(0x7fa96c15e680 0x7fa96c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:38.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.669+0000 7fa96b7fe640 1 -- 172.21.15.16:0/2671897040 >> v2:172.21.15.146:3300/0 conn(0x7fa96c15e680 msgr2=0x7fa96c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:38.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.669+0000 7fa96b7fe640 1 --2- 172.21.15.16:0/2671897040 >> v2:172.21.15.146:3300/0 conn(0x7fa96c15e680 0x7fa96c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.669+0000 7fa96b7fe640 1 -- 172.21.15.16:0/2671897040 >> v1:172.21.15.16:6789/0 conn(0x7fa96c15e2a0 legacy=0x7fa96c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:38.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.669+0000 7fa96b7fe640 1 -- 172.21.15.16:0/2671897040 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96c0cf5b0 con 0x7fa96c0d0010 2024-09-19T07:48:38.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.669+0000 7fa969ffb640 1 -- 172.21.15.16:0/2671897040 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2940376959 0 0) 0x7fa96c0d1880 con 0x7fa96c15e2a0 2024-09-19T07:48:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.670+0000 7fa96b7fe640 1 --2- 172.21.15.16:0/2671897040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c0d0010 0x7fa96c0d4370 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0a22a0 tx=0x7fa96402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c591865a13d43f29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.670+0000 7fa969ffb640 1 -- 172.21.15.16:0/2671897040 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964036070 con 0x7fa96c0d0010 2024-09-19T07:48:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.670+0000 7fa969ffb640 1 -- 172.21.15.16:0/2671897040 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7fa964031600 con 0x7fa96c0d0010 2024-09-19T07:48:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.670+0000 7fa969ffb640 1 -- 172.21.15.16:0/2671897040 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9640319d0 con 0x7fa96c0d0010 2024-09-19T07:48:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.671+0000 7fa971406640 1 -- 172.21.15.16:0/2671897040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c0d0010 msgr2=0x7fa96c0d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.671+0000 7fa971406640 1 --2- 172.21.15.16:0/2671897040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c0d0010 0x7fa96c0d4370 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0a22a0 tx=0x7fa96402a6a0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.671+0000 7fa971406640 1 -- 172.21.15.16:0/2671897040 shutdown_connections 2024-09-19T07:48:38.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.671+0000 7fa971406640 1 --2- 172.21.15.16:0/2671897040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c0d0010 0x7fa96c0d4370 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.671+0000 7fa971406640 1 --2- 172.21.15.16:0/2671897040 >> v2:172.21.15.146:3300/0 conn(0x7fa96c15e680 0x7fa96c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.671+0000 7fa971406640 1 -- 172.21.15.16:0/2671897040 >> 172.21.15.16:0/2671897040 conn(0x7fa96c09d760 msgr2=0x7fa96c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:38.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.671+0000 7fa971406640 1 -- 172.21.15.16:0/2671897040 shutdown_connections 2024-09-19T07:48:38.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.671+0000 7fa971406640 1 -- 172.21.15.16:0/2671897040 wait complete. 2024-09-19T07:48:38.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.672+0000 7fa971406640 1 Processor -- start 2024-09-19T07:48:38.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.672+0000 7fa971406640 1 -- start start 2024-09-19T07:48:38.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.673+0000 7fa971406640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa96c0d0010 0x7fa96c14ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:38.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.673+0000 7fa971406640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa96c15e2a0 0x7fa96c0c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:38.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.673+0000 7fa971406640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c15e680 0x7fa96c0c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:38.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.673+0000 7fa971406640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c06dcb0 con 0x7fa96c15e680 2024-09-19T07:48:38.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.673+0000 7fa971406640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c06db30 con 0x7fa96c0d0010 2024-09-19T07:48:38.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.673+0000 7fa971406640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa96c0d48b0 con 0x7fa96c15e2a0 2024-09-19T07:48:38.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.673+0000 7fa96b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c15e680 0x7fa96c0c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:38.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.673+0000 7fa96b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c15e680 0x7fa96c0c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36286/0 (socket says 172.21.15.16:36286) 2024-09-19T07:48:38.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.673+0000 7fa96b7fe640 1 -- 172.21.15.16:0/3632405096 learned_addr learned my addr 172.21.15.16:0/3632405096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:38.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.673+0000 7fa96a7fc640 1 --2- 172.21.15.16:0/3632405096 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa96c15e2a0 0x7fa96c0c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:38.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.673+0000 7fa96affd640 1 --2- 172.21.15.16:0/3632405096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa96c0d0010 0x7fa96c14ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:38.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.673+0000 7fa96b7fe640 1 -- 172.21.15.16:0/3632405096 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa96c15e2a0 msgr2=0x7fa96c0c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:38.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.673+0000 7fa96b7fe640 1 --2- 172.21.15.16:0/3632405096 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa96c15e2a0 0x7fa96c0c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.673+0000 7fa96b7fe640 1 -- 172.21.15.16:0/3632405096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa96c0d0010 msgr2=0x7fa96c14ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:38.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.673+0000 7fa96b7fe640 1 --2- 172.21.15.16:0/3632405096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa96c0d0010 0x7fa96c14ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:38.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.673+0000 7fa96b7fe640 1 -- 172.21.15.16:0/3632405096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96c0c89d0 con 0x7fa96c15e680 2024-09-19T07:48:38.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.673+0000 7fa96a7fc640 1 --2- 172.21.15.16:0/3632405096 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa96c15e2a0 0x7fa96c0c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:38.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.673+0000 7fa96affd640 1 --2- 172.21.15.16:0/3632405096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa96c0d0010 0x7fa96c14ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:38.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.674+0000 7fa96b7fe640 1 --2- 172.21.15.16:0/3632405096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c15e680 0x7fa96c0c7f10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa96402a6e0 tx=0x7fa964004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:38.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.674+0000 7fa9537fe640 1 -- 172.21.15.16:0/3632405096 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964036050 con 0x7fa96c15e680 2024-09-19T07:48:38.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.674+0000 7fa9537fe640 1 -- 172.21.15.16:0/3632405096 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7fa964042070 con 0x7fa96c15e680 2024-09-19T07:48:38.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.674+0000 7fa971406640 1 -- 172.21.15.16:0/3632405096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96c0c8cb0 con 0x7fa96c15e680 2024-09-19T07:48:38.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.674+0000 7fa9537fe640 1 -- 172.21.15.16:0/3632405096 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964009510 con 0x7fa96c15e680 2024-09-19T07:48:38.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.674+0000 7fa971406640 1 -- 172.21.15.16:0/3632405096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96c0c9270 con 0x7fa96c15e680 2024-09-19T07:48:38.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.675+0000 7fa971406640 1 -- 172.21.15.16:0/3632405096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa96c0d4370 con 0x7fa96c15e680 2024-09-19T07:48:38.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.675+0000 7fa9537fe640 1 -- 172.21.15.16:0/3632405096 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa964007bd0 con 0x7fa96c15e680 2024-09-19T07:48:38.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.676+0000 7fa9537fe640 1 --2- 172.21.15.16:0/3632405096 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa95806c2b0 0x7fa95806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:38.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.676+0000 7fa9537fe640 1 -- 172.21.15.16:0/3632405096 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6288+0+0 (secure 0 0 0) 0x7fa9640ac680 con 0x7fa96c15e680 2024-09-19T07:48:38.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.676+0000 7fa96affd640 1 --2- 172.21.15.16:0/3632405096 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa95806c2b0 0x7fa95806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:38.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.677+0000 7fa96affd640 1 --2- 172.21.15.16:0/3632405096 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa95806c2b0 0x7fa95806e770 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa95c000f60 tx=0x7fa95c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:38.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.680+0000 7fa9537fe640 1 -- 172.21.15.16:0/3632405096 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa96407ab20 con 0x7fa96c15e680 2024-09-19T07:48:38.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:38.891+0000 7fa971406640 1 -- 172.21.15.16:0/3632405096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa96c06cf90 con 0x7fa96c15e680 2024-09-19T07:48:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.548+0000 7fa9537fe640 1 -- 172.21.15.16:0/3632405096 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7fa96407a420 con 0x7fa96c15e680 2024-09-19T07:48:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.550+0000 7fa971406640 1 -- 172.21.15.16:0/3632405096 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa95806c2b0 msgr2=0x7fa95806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.550+0000 7fa971406640 1 --2- 172.21.15.16:0/3632405096 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa95806c2b0 0x7fa95806e770 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa95c000f60 tx=0x7fa95c009040 comp rx=0 tx=0).stop 2024-09-19T07:48:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.550+0000 7fa971406640 1 -- 172.21.15.16:0/3632405096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c15e680 msgr2=0x7fa96c0c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.550+0000 7fa971406640 1 --2- 172.21.15.16:0/3632405096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c15e680 0x7fa96c0c7f10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa96402a6e0 tx=0x7fa964004a40 comp rx=0 tx=0).stop 2024-09-19T07:48:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.550+0000 7fa971406640 1 -- 172.21.15.16:0/3632405096 shutdown_connections 2024-09-19T07:48:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.550+0000 7fa971406640 1 --2- 172.21.15.16:0/3632405096 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa95806c2b0 0x7fa95806e770 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.550+0000 7fa971406640 1 --2- 172.21.15.16:0/3632405096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c15e680 0x7fa96c0c7f10 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.550+0000 7fa971406640 1 --2- 172.21.15.16:0/3632405096 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa96c15e2a0 0x7fa96c0c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.550+0000 7fa971406640 1 --2- 172.21.15.16:0/3632405096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa96c0d0010 0x7fa96c14ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.550+0000 7fa971406640 1 -- 172.21.15.16:0/3632405096 >> 172.21.15.16:0/3632405096 conn(0x7fa96c09d760 msgr2=0x7fa96c0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.550+0000 7fa971406640 1 -- 172.21.15.16:0/3632405096 shutdown_connections 2024-09-19T07:48:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.550+0000 7fa971406640 1 -- 172.21.15.16:0/3632405096 wait complete. 2024-09-19T07:48:39.564 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T07:48:39.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.716+0000 7f4d5abed640 1 Processor -- start 2024-09-19T07:48:39.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.716+0000 7f4d5abed640 1 -- start start 2024-09-19T07:48:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.717+0000 7f4d5abed640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d5415e2a0 0x7f4d5406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.717+0000 7f4d5abed640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4d5406d960 0x7f4d540d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.717+0000 7f4d5abed640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d540d1880 con 0x7f4d5415e680 2024-09-19T07:48:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.717+0000 7f4d5abed640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f4d540d1b60 con 0x7f4d5415e2a0 2024-09-19T07:48:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.717+0000 7f4d5abed640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4d540cf2d0 con 0x7f4d5406d960 2024-09-19T07:48:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.717+0000 7f4d58989640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d5415e2a0 0x7f4d5406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.717+0000 7f4d53fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4d5415e680 0x7f4d540cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56540/0 (socket says 172.21.15.16:56540) 2024-09-19T07:48:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.717+0000 7f4d53fff640 1 -- 172.21.15.16:0/3522827279 learned_addr learned my addr 172.21.15.16:0/3522827279 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.717+0000 7f4d58989640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d5415e2a0 0x7f4d5406d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36290/0 (socket says 172.21.15.16:36290) 2024-09-19T07:48:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.717+0000 7f4d5918a640 1 --2- 172.21.15.16:0/3522827279 >> v2:172.21.15.146:3300/0 conn(0x7f4d5406d960 0x7f4d540d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.717+0000 7f4d58989640 1 -- 172.21.15.16:0/3522827279 >> v2:172.21.15.146:3300/0 conn(0x7f4d5406d960 msgr2=0x7f4d540d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.717+0000 7f4d58989640 1 --2- 172.21.15.16:0/3522827279 >> v2:172.21.15.146:3300/0 conn(0x7f4d5406d960 0x7f4d540d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.717+0000 7f4d58989640 1 -- 172.21.15.16:0/3522827279 >> v1:172.21.15.16:6789/0 conn(0x7f4d5415e680 legacy=0x7f4d540cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:39.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.717+0000 7f4d58989640 1 -- 172.21.15.16:0/3522827279 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d540cf5b0 con 0x7f4d5415e2a0 2024-09-19T07:48:39.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.717+0000 7f4d5918a640 1 --2- 172.21.15.16:0/3522827279 >> v2:172.21.15.146:3300/0 conn(0x7f4d5406d960 0x7f4d540d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:39.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.718+0000 7f4d58989640 1 --2- 172.21.15.16:0/3522827279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5415e2a0 0x7f4d5406d420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4d54163050 tx=0x7f4d4802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f18eaffdda17b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:39.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.718+0000 7f4d537fe640 1 -- 172.21.15.16:0/3522827279 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48037070 con 0x7f4d5415e2a0 2024-09-19T07:48:39.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.718+0000 7f4d537fe640 1 -- 172.21.15.16:0/3522827279 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7f4d48032440 con 0x7f4d5415e2a0 2024-09-19T07:48:39.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.718+0000 7f4d537fe640 1 -- 172.21.15.16:0/3522827279 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48032810 con 0x7f4d5415e2a0 2024-09-19T07:48:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.719+0000 7f4d5abed640 1 -- 172.21.15.16:0/3522827279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5415e2a0 msgr2=0x7f4d5406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.719+0000 7f4d5abed640 1 --2- 172.21.15.16:0/3522827279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5415e2a0 0x7f4d5406d420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4d54163050 tx=0x7f4d4802a1a0 comp rx=0 tx=0).stop 2024-09-19T07:48:39.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.719+0000 7f4d5abed640 1 -- 172.21.15.16:0/3522827279 shutdown_connections 2024-09-19T07:48:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.719+0000 7f4d5abed640 1 --2- 172.21.15.16:0/3522827279 >> v2:172.21.15.146:3300/0 conn(0x7f4d5406d960 0x7f4d540d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.719+0000 7f4d5abed640 1 --2- 172.21.15.16:0/3522827279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5415e2a0 0x7f4d5406d420 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.719+0000 7f4d5abed640 1 -- 172.21.15.16:0/3522827279 >> 172.21.15.16:0/3522827279 conn(0x7f4d5409d760 msgr2=0x7f4d5408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.719+0000 7f4d5abed640 1 -- 172.21.15.16:0/3522827279 shutdown_connections 2024-09-19T07:48:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.719+0000 7f4d5abed640 1 -- 172.21.15.16:0/3522827279 wait complete. 2024-09-19T07:48:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.720+0000 7f4d5abed640 1 Processor -- start 2024-09-19T07:48:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.720+0000 7f4d5abed640 1 -- start start 2024-09-19T07:48:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.721+0000 7f4d5abed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5406d960 0x7f4d540cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.721+0000 7f4d5abed640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d5415e2a0 0x7f4d540b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.721+0000 7f4d5abed640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d5415e680 0x7f4d5415aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.721+0000 7f4d5abed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d540d0360 con 0x7f4d5406d960 2024-09-19T07:48:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.721+0000 7f4d5abed640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4d540d01e0 con 0x7f4d5415e2a0 2024-09-19T07:48:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.721+0000 7f4d5abed640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4d540d48b0 con 0x7f4d5415e680 2024-09-19T07:48:39.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.721+0000 7f4d58989640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5406d960 0x7f4d540cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.721+0000 7f4d58989640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5406d960 0x7f4d540cbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36296/0 (socket says 172.21.15.16:36296) 2024-09-19T07:48:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.721+0000 7f4d58989640 1 -- 172.21.15.16:0/255514797 learned_addr learned my addr 172.21.15.16:0/255514797 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.721+0000 7f4d53fff640 1 --2- 172.21.15.16:0/255514797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d5415e2a0 0x7f4d540b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.721+0000 7f4d5918a640 1 --2- 172.21.15.16:0/255514797 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d5415e680 0x7f4d5415aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.721+0000 7f4d58989640 1 -- 172.21.15.16:0/255514797 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d5415e680 msgr2=0x7f4d5415aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.721+0000 7f4d58989640 1 --2- 172.21.15.16:0/255514797 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d5415e680 0x7f4d5415aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.721+0000 7f4d58989640 1 -- 172.21.15.16:0/255514797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d5415e2a0 msgr2=0x7f4d540b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.721+0000 7f4d58989640 1 --2- 172.21.15.16:0/255514797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d5415e2a0 0x7f4d540b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.721+0000 7f4d58989640 1 -- 172.21.15.16:0/255514797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d540b56e0 con 0x7f4d5406d960 2024-09-19T07:48:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.722+0000 7f4d58989640 1 --2- 172.21.15.16:0/255514797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5406d960 0x7f4d540cbf30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4d48035040 tx=0x7f4d480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.722+0000 7f4d517fa640 1 -- 172.21.15.16:0/255514797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48037040 con 0x7f4d5406d960 2024-09-19T07:48:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.722+0000 7f4d517fa640 1 -- 172.21.15.16:0/255514797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7f4d48030830 con 0x7f4d5406d960 2024-09-19T07:48:39.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.722+0000 7f4d5abed640 1 -- 172.21.15.16:0/255514797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d540cd730 con 0x7f4d5406d960 2024-09-19T07:48:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.722+0000 7f4d517fa640 1 -- 172.21.15.16:0/255514797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48030c20 con 0x7f4d5406d960 2024-09-19T07:48:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.722+0000 7f4d5abed640 1 -- 172.21.15.16:0/255514797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d540cdcf0 con 0x7f4d5406d960 2024-09-19T07:48:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.723+0000 7f4d5abed640 1 -- 172.21.15.16:0/255514797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d540d4370 con 0x7f4d5406d960 2024-09-19T07:48:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.723+0000 7f4d517fa640 1 -- 172.21.15.16:0/255514797 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4d48044020 con 0x7f4d5406d960 2024-09-19T07:48:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.724+0000 7f4d517fa640 1 --2- 172.21.15.16:0/255514797 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4d3806c2b0 0x7f4d3806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.724+0000 7f4d517fa640 1 -- 172.21.15.16:0/255514797 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6288+0+0 (secure 0 0 0) 0x7f4d480abc60 con 0x7f4d5406d960 2024-09-19T07:48:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.724+0000 7f4d53fff640 1 --2- 172.21.15.16:0/255514797 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4d3806c2b0 0x7f4d3806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.725+0000 7f4d53fff640 1 --2- 172.21.15.16:0/255514797 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4d3806c2b0 0x7f4d3806e770 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4d540b9c60 tx=0x7f4d40007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.728+0000 7f4d517fa640 1 -- 172.21.15.16:0/255514797 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4d4807a100 con 0x7f4d5406d960 2024-09-19T07:48:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:39.905+0000 7f4d5abed640 1 -- 172.21.15.16:0/255514797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4d5406cf90 con 0x7f4d5406d960 2024-09-19T07:48:40.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.549+0000 7f4d517fa640 1 -- 172.21.15.16:0/255514797 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v61) ==== 181+0+0 (secure 0 0 0) 0x7f4d48079a00 con 0x7f4d5406d960 2024-09-19T07:48:40.550 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-19T07:48:40.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.554+0000 7f4d5abed640 1 -- 172.21.15.16:0/255514797 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4d3806c2b0 msgr2=0x7f4d3806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:40.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.554+0000 7f4d5abed640 1 --2- 172.21.15.16:0/255514797 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4d3806c2b0 0x7f4d3806e770 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4d540b9c60 tx=0x7f4d40007e80 comp rx=0 tx=0).stop 2024-09-19T07:48:40.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.554+0000 7f4d5abed640 1 -- 172.21.15.16:0/255514797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5406d960 msgr2=0x7f4d540cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:40.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.554+0000 7f4d5abed640 1 --2- 172.21.15.16:0/255514797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5406d960 0x7f4d540cbf30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4d48035040 tx=0x7f4d480046c0 comp rx=0 tx=0).stop 2024-09-19T07:48:40.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.555+0000 7f4d5abed640 1 -- 172.21.15.16:0/255514797 shutdown_connections 2024-09-19T07:48:40.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.555+0000 7f4d5abed640 1 --2- 172.21.15.16:0/255514797 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4d3806c2b0 0x7f4d3806e770 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:40.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.555+0000 7f4d5abed640 1 --2- 172.21.15.16:0/255514797 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d5415e680 0x7f4d5415aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:40.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.555+0000 7f4d5abed640 1 --2- 172.21.15.16:0/255514797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d5415e2a0 0x7f4d540b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:40.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.555+0000 7f4d5abed640 1 --2- 172.21.15.16:0/255514797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5406d960 0x7f4d540cbf30 secure :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4d48035040 tx=0x7f4d480046c0 comp rx=0 tx=0).stop 2024-09-19T07:48:40.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.555+0000 7f4d5abed640 1 -- 172.21.15.16:0/255514797 >> 172.21.15.16:0/255514797 conn(0x7f4d5409d760 msgr2=0x7f4d5415d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.556+0000 7f4d5abed640 1 -- 172.21.15.16:0/255514797 shutdown_connections 2024-09-19T07:48:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.558+0000 7f4d5abed640 1 -- 172.21.15.16:0/255514797 wait complete. 2024-09-19T07:48:40.579 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T07:48:40.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.692+0000 7f2fd8b86640 1 Processor -- start 2024-09-19T07:48:40.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.692+0000 7f2fd8b86640 1 -- start start 2024-09-19T07:48:40.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.693+0000 7f2fd8b86640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fd40b3830 0x7f2fd406bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:40.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.693+0000 7f2fd8b86640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2fd406c150 0x7f2fd40c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:40.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.693+0000 7f2fd8b86640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fd40ce000 con 0x7f2fd415f5d0 2024-09-19T07:48:40.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.693+0000 7f2fd8b86640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2fd40c8250 con 0x7f2fd406c150 2024-09-19T07:48:40.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.693+0000 7f2fd8b86640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd40ce2e0 con 0x7f2fd40b3830 2024-09-19T07:48:40.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.693+0000 7f2fd2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2fd406c150 0x7f2fd40c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:40.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.693+0000 7f2fd2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2fd406c150 0x7f2fd40c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36308/0 (socket says 172.21.15.16:36308) 2024-09-19T07:48:40.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.693+0000 7f2fd2d9d640 1 -- 172.21.15.16:0/2163614465 learned_addr learned my addr 172.21.15.16:0/2163614465 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:40.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.693+0000 7f2fd259c640 1 --1- 172.21.15.16:0/2163614465 >> v1:172.21.15.16:6789/0 conn(0x7f2fd415f5d0 0x7f2fd40cd7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56552/0 (socket says 172.21.15.16:56552) 2024-09-19T07:48:40.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.693+0000 7f2fd2d9d640 1 -- 172.21.15.16:0/2163614465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fd40b3830 msgr2=0x7f2fd406bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:40.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.693+0000 7f2fd1d9b640 1 --2- 172.21.15.16:0/2163614465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fd40b3830 0x7f2fd406bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:40.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.693+0000 7f2fd2d9d640 1 --2- 172.21.15.16:0/2163614465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fd40b3830 0x7f2fd406bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:40.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.693+0000 7f2fd2d9d640 1 -- 172.21.15.16:0/2163614465 >> v1:172.21.15.16:6789/0 conn(0x7f2fd415f5d0 legacy=0x7f2fd40cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:40.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.693+0000 7f2fd159a640 1 -- 172.21.15.16:0/2163614465 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1963165915 0 0) 0x7f2fd40ce000 con 0x7f2fd415f5d0 2024-09-19T07:48:40.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.693+0000 7f2fd2d9d640 1 -- 172.21.15.16:0/2163614465 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd40ce5c0 con 0x7f2fd406c150 2024-09-19T07:48:40.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.694+0000 7f2fd2d9d640 1 --2- 172.21.15.16:0/2163614465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd406c150 0x7f2fd40c86e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009870 tx=0x7f2fcc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d776379a46a38fa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:40.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.694+0000 7f2fd159a640 1 -- 172.21.15.16:0/2163614465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc037070 con 0x7f2fd406c150 2024-09-19T07:48:40.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.694+0000 7f2fd159a640 1 -- 172.21.15.16:0/2163614465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7f2fcc032440 con 0x7f2fd406c150 2024-09-19T07:48:40.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.694+0000 7f2fd159a640 1 -- 172.21.15.16:0/2163614465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc032810 con 0x7f2fd406c150 2024-09-19T07:48:40.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.694+0000 7f2fd8b86640 1 -- 172.21.15.16:0/2163614465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd406c150 msgr2=0x7f2fd40c86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:40.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.694+0000 7f2fd8b86640 1 --2- 172.21.15.16:0/2163614465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd406c150 0x7f2fd40c86e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009870 tx=0x7f2fcc02a1c0 comp rx=0 tx=0).stop 2024-09-19T07:48:40.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.695+0000 7f2fd8b86640 1 -- 172.21.15.16:0/2163614465 shutdown_connections 2024-09-19T07:48:40.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.695+0000 7f2fd8b86640 1 --2- 172.21.15.16:0/2163614465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd406c150 0x7f2fd40c86e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:40.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.695+0000 7f2fd8b86640 1 --2- 172.21.15.16:0/2163614465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fd40b3830 0x7f2fd406bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:40.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.695+0000 7f2fd8b86640 1 -- 172.21.15.16:0/2163614465 >> 172.21.15.16:0/2163614465 conn(0x7f2fd40b2d10 msgr2=0x7f2fd40b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:40.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.695+0000 7f2fd8b86640 1 -- 172.21.15.16:0/2163614465 shutdown_connections 2024-09-19T07:48:40.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.695+0000 7f2fd8b86640 1 -- 172.21.15.16:0/2163614465 wait complete. 2024-09-19T07:48:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.696+0000 7f2fd8b86640 1 Processor -- start 2024-09-19T07:48:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.696+0000 7f2fd8b86640 1 -- start start 2024-09-19T07:48:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.697+0000 7f2fd8b86640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fd406c150 0x7f2fd414e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.697+0000 7f2fd8b86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd40b3830 0x7f2fd415a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.697+0000 7f2fd8b86640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fd415f5d0 0x7f2fd415a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.697+0000 7f2fd8b86640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd415fb30 con 0x7f2fd40b3830 2024-09-19T07:48:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.697+0000 7f2fd8b86640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd415f9b0 con 0x7f2fd415f5d0 2024-09-19T07:48:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.697+0000 7f2fd8b86640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd40c8c20 con 0x7f2fd406c150 2024-09-19T07:48:40.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.697+0000 7f2fd1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd40b3830 0x7f2fd415a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:40.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.697+0000 7f2fd1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd40b3830 0x7f2fd415a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36310/0 (socket says 172.21.15.16:36310) 2024-09-19T07:48:40.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.697+0000 7f2fd1d9b640 1 -- 172.21.15.16:0/1284366186 learned_addr learned my addr 172.21.15.16:0/1284366186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:40.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.697+0000 7f2fd259c640 1 --2- 172.21.15.16:0/1284366186 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fd406c150 0x7f2fd414e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:40.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.697+0000 7f2fd2d9d640 1 --2- 172.21.15.16:0/1284366186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fd415f5d0 0x7f2fd415a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:40.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.697+0000 7f2fd1d9b640 1 -- 172.21.15.16:0/1284366186 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fd406c150 msgr2=0x7f2fd414e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:40.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.697+0000 7f2fd1d9b640 1 --2- 172.21.15.16:0/1284366186 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fd406c150 0x7f2fd414e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:40.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.697+0000 7f2fd1d9b640 1 -- 172.21.15.16:0/1284366186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fd415f5d0 msgr2=0x7f2fd415a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:40.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.697+0000 7f2fd1d9b640 1 --2- 172.21.15.16:0/1284366186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fd415f5d0 0x7f2fd415a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:40.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.697+0000 7f2fd1d9b640 1 -- 172.21.15.16:0/1284366186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd40b62f0 con 0x7f2fd40b3830 2024-09-19T07:48:40.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.697+0000 7f2fd2d9d640 1 --2- 172.21.15.16:0/1284366186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fd415f5d0 0x7f2fd415a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:40.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.697+0000 7f2fd1d9b640 1 --2- 172.21.15.16:0/1284366186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd40b3830 0x7f2fd415a290 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00c7f0 tx=0x7f2fbc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:40.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.697+0000 7f2fcaffd640 1 -- 172.21.15.16:0/1284366186 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc015070 con 0x7f2fd40b3830 2024-09-19T07:48:40.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.698+0000 7f2fcaffd640 1 -- 172.21.15.16:0/1284366186 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7f2fbc004240 con 0x7f2fd40b3830 2024-09-19T07:48:40.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.698+0000 7f2fcaffd640 1 -- 172.21.15.16:0/1284366186 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc007620 con 0x7f2fd40b3830 2024-09-19T07:48:40.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.698+0000 7f2fd8b86640 1 -- 172.21.15.16:0/1284366186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd40c54c0 con 0x7f2fd40b3830 2024-09-19T07:48:40.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.698+0000 7f2fd8b86640 1 -- 172.21.15.16:0/1284366186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd40c5a80 con 0x7f2fd40b3830 2024-09-19T07:48:40.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.699+0000 7f2fd8b86640 1 -- 172.21.15.16:0/1284366186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd40cb760 con 0x7f2fd40b3830 2024-09-19T07:48:40.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.699+0000 7f2fcaffd640 1 -- 172.21.15.16:0/1284366186 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2fbc0079d0 con 0x7f2fd40b3830 2024-09-19T07:48:40.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.699+0000 7f2fcaffd640 1 --2- 172.21.15.16:0/1284366186 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2fb406c2b0 0x7f2fb406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:40.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.700+0000 7f2fcaffd640 1 -- 172.21.15.16:0/1284366186 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5850+0+0 (secure 0 0 0) 0x7f2fbc08ba00 con 0x7f2fd40b3830 2024-09-19T07:48:40.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.700+0000 7f2fd259c640 1 --2- 172.21.15.16:0/1284366186 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2fb406c2b0 0x7f2fb406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:40.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.700+0000 7f2fd259c640 1 --2- 172.21.15.16:0/1284366186 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2fb406c2b0 0x7f2fb406e770 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4007ed0 tx=0x7f2fc4007e60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:40.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.704+0000 7f2fcaffd640 1 -- 172.21.15.16:0/1284366186 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2fbc05a040 con 0x7f2fd40b3830 2024-09-19T07:48:40.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:40.882+0000 7f2fd8b86640 1 -- 172.21.15.16:0/1284366186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2fd40c86c0 con 0x7f2fd40b3830 2024-09-19T07:48:41.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.555+0000 7f2fcaffd640 1 -- 172.21.15.16:0/1284366186 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v62) ==== 169+0+0 (secure 0 0 0) 0x7f2fbc059940 con 0x7f2fd40b3830 2024-09-19T07:48:41.556 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-19T07:48:41.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.561+0000 7f2fd8b86640 1 -- 172.21.15.16:0/1284366186 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2fb406c2b0 msgr2=0x7f2fb406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:41.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.561+0000 7f2fd8b86640 1 --2- 172.21.15.16:0/1284366186 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2fb406c2b0 0x7f2fb406e770 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4007ed0 tx=0x7f2fc4007e60 comp rx=0 tx=0).stop 2024-09-19T07:48:41.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.561+0000 7f2fd8b86640 1 -- 172.21.15.16:0/1284366186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd40b3830 msgr2=0x7f2fd415a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:41.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.561+0000 7f2fd8b86640 1 --2- 172.21.15.16:0/1284366186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd40b3830 0x7f2fd415a290 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00c7f0 tx=0x7f2fbc00ccb0 comp rx=0 tx=0).stop 2024-09-19T07:48:41.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.562+0000 7f2fd8b86640 1 -- 172.21.15.16:0/1284366186 shutdown_connections 2024-09-19T07:48:41.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.562+0000 7f2fd8b86640 1 --2- 172.21.15.16:0/1284366186 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2fb406c2b0 0x7f2fb406e770 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:41.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.562+0000 7f2fd8b86640 1 --2- 172.21.15.16:0/1284366186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fd415f5d0 0x7f2fd415a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:41.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.562+0000 7f2fd8b86640 1 --2- 172.21.15.16:0/1284366186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fd40b3830 0x7f2fd415a290 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:41.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.562+0000 7f2fd8b86640 1 --2- 172.21.15.16:0/1284366186 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fd406c150 0x7f2fd414e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:41.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.562+0000 7f2fd8b86640 1 -- 172.21.15.16:0/1284366186 >> 172.21.15.16:0/1284366186 conn(0x7f2fd40b2d10 msgr2=0x7f2fd40cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:41.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.562+0000 7f2fd8b86640 1 -- 172.21.15.16:0/1284366186 shutdown_connections 2024-09-19T07:48:41.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.562+0000 7f2fd8b86640 1 -- 172.21.15.16:0/1284366186 wait complete. 2024-09-19T07:48:41.580 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_session_metadata_threshold 2024-09-19T07:48:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.723+0000 7f2dfd7ff640 1 Processor -- start 2024-09-19T07:48:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.723+0000 7f2dfd7ff640 1 -- start start 2024-09-19T07:48:41.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.724+0000 7f2dfd7ff640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2df8162920 0x7f2df80b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:41.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.724+0000 7f2dfd7ff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df80afaa0 0x7f2df80b5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:41.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.724+0000 7f2dfd7ff640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2df80b51b0 con 0x7f2df80aff00 2024-09-19T07:48:41.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.724+0000 7f2dfd7ff640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2df80b3bd0 con 0x7f2df8162920 2024-09-19T07:48:41.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.724+0000 7f2dfd7ff640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2df80b3eb0 con 0x7f2df80afaa0 2024-09-19T07:48:41.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.724+0000 7f2deffff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df80afaa0 0x7f2df80b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:41.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.724+0000 7f2deffff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df80afaa0 0x7f2df80b5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:33918/0 (socket says 172.21.15.16:33918) 2024-09-19T07:48:41.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.724+0000 7f2deffff640 1 -- 172.21.15.16:0/2946825103 learned_addr learned my addr 172.21.15.16:0/2946825103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:41.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.724+0000 7f2df6ffd640 1 --2- 172.21.15.16:0/2946825103 >> v2:172.21.15.146:3300/0 conn(0x7f2df8162920 0x7f2df80b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.725+0000 7f2deffff640 1 -- 172.21.15.16:0/2946825103 >> v2:172.21.15.146:3300/0 conn(0x7f2df8162920 msgr2=0x7f2df80b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.725+0000 7f2deffff640 1 --2- 172.21.15.16:0/2946825103 >> v2:172.21.15.146:3300/0 conn(0x7f2df8162920 0x7f2df80b4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.725+0000 7f2deffff640 1 -- 172.21.15.16:0/2946825103 >> v1:172.21.15.146:6789/0 conn(0x7f2df80aff00 legacy=0x7f2df80ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.725+0000 7f2deffff640 1 -- 172.21.15.16:0/2946825103 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df80b4190 con 0x7f2df80afaa0 2024-09-19T07:48:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.726+0000 7f2deffff640 1 --2- 172.21.15.16:0/2946825103 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df80afaa0 0x7f2df80b5750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009870 tx=0x7f2de002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7005b824cee3e4d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.726+0000 7f2df67fc640 1 -- 172.21.15.16:0/2946825103 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0037070 con 0x7f2df80afaa0 2024-09-19T07:48:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.726+0000 7f2df67fc640 1 -- 172.21.15.16:0/2946825103 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7f2de0032440 con 0x7f2df80afaa0 2024-09-19T07:48:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.726+0000 7f2df67fc640 1 -- 172.21.15.16:0/2946825103 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0032810 con 0x7f2df80afaa0 2024-09-19T07:48:41.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.727+0000 7f2dfd7ff640 1 -- 172.21.15.16:0/2946825103 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df80afaa0 msgr2=0x7f2df80b5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:41.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.727+0000 7f2dfd7ff640 1 --2- 172.21.15.16:0/2946825103 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df80afaa0 0x7f2df80b5750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009870 tx=0x7f2de002a270 comp rx=0 tx=0).stop 2024-09-19T07:48:41.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.727+0000 7f2dfd7ff640 1 -- 172.21.15.16:0/2946825103 shutdown_connections 2024-09-19T07:48:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.727+0000 7f2dfd7ff640 1 --2- 172.21.15.16:0/2946825103 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df80afaa0 0x7f2df80b5750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.727+0000 7f2dfd7ff640 1 --2- 172.21.15.16:0/2946825103 >> v2:172.21.15.146:3300/0 conn(0x7f2df8162920 0x7f2df80b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.727+0000 7f2dfd7ff640 1 -- 172.21.15.16:0/2946825103 >> 172.21.15.16:0/2946825103 conn(0x7f2df8067d70 msgr2=0x7f2df806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.727+0000 7f2dfd7ff640 1 -- 172.21.15.16:0/2946825103 shutdown_connections 2024-09-19T07:48:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.727+0000 7f2dfd7ff640 1 -- 172.21.15.16:0/2946825103 wait complete. 2024-09-19T07:48:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.729+0000 7f2dfd7ff640 1 Processor -- start 2024-09-19T07:48:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.729+0000 7f2dfd7ff640 1 -- start start 2024-09-19T07:48:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.730+0000 7f2dfd7ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df80afaa0 0x7f2df815ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.730+0000 7f2dfd7ff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df80aff00 0x7f2df80b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.730+0000 7f2dfd7ff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2df8162920 0x7f2df8164f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.730+0000 7f2dfd7ff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2df80b3860 con 0x7f2df80afaa0 2024-09-19T07:48:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.730+0000 7f2dfd7ff640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2df80b4750 con 0x7f2df8162920 2024-09-19T07:48:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.730+0000 7f2dfd7ff640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2df80b5c90 con 0x7f2df80aff00 2024-09-19T07:48:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.731+0000 7f2df6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df80afaa0 0x7f2df815ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.731+0000 7f2df6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df80afaa0 0x7f2df815ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36324/0 (socket says 172.21.15.16:36324) 2024-09-19T07:48:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.731+0000 7f2df6ffd640 1 -- 172.21.15.16:0/4018146017 learned_addr learned my addr 172.21.15.16:0/4018146017 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.731+0000 7f2df77fe640 1 --2- 172.21.15.16:0/4018146017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2df8162920 0x7f2df8164f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.731+0000 7f2deffff640 1 --2- 172.21.15.16:0/4018146017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df80aff00 0x7f2df80b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.731+0000 7f2df6ffd640 1 -- 172.21.15.16:0/4018146017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df80aff00 msgr2=0x7f2df80b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.731+0000 7f2df6ffd640 1 --2- 172.21.15.16:0/4018146017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df80aff00 0x7f2df80b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.731+0000 7f2df6ffd640 1 -- 172.21.15.16:0/4018146017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2df8162920 msgr2=0x7f2df8164f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.731+0000 7f2df6ffd640 1 --2- 172.21.15.16:0/4018146017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2df8162920 0x7f2df8164f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.731+0000 7f2df6ffd640 1 -- 172.21.15.16:0/4018146017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df80b1260 con 0x7f2df80afaa0 2024-09-19T07:48:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.731+0000 7f2df6ffd640 1 --2- 172.21.15.16:0/4018146017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df80afaa0 0x7f2df815ab50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2de800e870 tx=0x7f2de800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.732+0000 7f2def7fe640 1 -- 172.21.15.16:0/4018146017 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8017070 con 0x7f2df80afaa0 2024-09-19T07:48:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.732+0000 7f2def7fe640 1 -- 172.21.15.16:0/4018146017 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7f2de800c940 con 0x7f2df80afaa0 2024-09-19T07:48:41.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.732+0000 7f2dfd7ff640 1 -- 172.21.15.16:0/4018146017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df80b1ee0 con 0x7f2df80afaa0 2024-09-19T07:48:41.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.732+0000 7f2def7fe640 1 -- 172.21.15.16:0/4018146017 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de800cd30 con 0x7f2df80afaa0 2024-09-19T07:48:41.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.732+0000 7f2dfd7ff640 1 -- 172.21.15.16:0/4018146017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df80b24a0 con 0x7f2df80afaa0 2024-09-19T07:48:41.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.733+0000 7f2dfd7ff640 1 -- 172.21.15.16:0/4018146017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df80b5750 con 0x7f2df80afaa0 2024-09-19T07:48:41.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.733+0000 7f2def7fe640 1 -- 172.21.15.16:0/4018146017 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2de801a070 con 0x7f2df80afaa0 2024-09-19T07:48:41.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.734+0000 7f2def7fe640 1 --2- 172.21.15.16:0/4018146017 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2dd806c2b0 0x7f2dd806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:41.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.734+0000 7f2def7fe640 1 -- 172.21.15.16:0/4018146017 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5378+0+0 (secure 0 0 0) 0x7f2de808b3f0 con 0x7f2df80afaa0 2024-09-19T07:48:41.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.735+0000 7f2deffff640 1 --2- 172.21.15.16:0/4018146017 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2dd806c2b0 0x7f2dd806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:41.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.736+0000 7f2deffff640 1 --2- 172.21.15.16:0/4018146017 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2dd806c2b0 0x7f2dd806e770 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009870 tx=0x7f2de002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:41.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.739+0000 7f2def7fe640 1 -- 172.21.15.16:0/4018146017 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2de80596e0 con 0x7f2df80afaa0 2024-09-19T07:48:41.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.935+0000 7f2dfd7ff640 1 -- 172.21.15.16:0/4018146017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_session_metadata_threshold"} v 0) -- 0x7f2df8161dd0 con 0x7f2df80afaa0 2024-09-19T07:48:41.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.938+0000 7f2def7fe640 1 -- 172.21.15.16:0/4018146017 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_session_metadata_threshold"}]=0 v21) ==== 113+0+0 (secure 0 0 0) 0x7f2de805d9e0 con 0x7f2df80afaa0 2024-09-19T07:48:41.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.940+0000 7f2dfd7ff640 1 -- 172.21.15.16:0/4018146017 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2dd806c2b0 msgr2=0x7f2dd806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:41.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.940+0000 7f2dfd7ff640 1 --2- 172.21.15.16:0/4018146017 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2dd806c2b0 0x7f2dd806e770 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009870 tx=0x7f2de002e000 comp rx=0 tx=0).stop 2024-09-19T07:48:41.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.940+0000 7f2dfd7ff640 1 -- 172.21.15.16:0/4018146017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df80afaa0 msgr2=0x7f2df815ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:41.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.940+0000 7f2dfd7ff640 1 --2- 172.21.15.16:0/4018146017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df80afaa0 0x7f2df815ab50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2de800e870 tx=0x7f2de800ed30 comp rx=0 tx=0).stop 2024-09-19T07:48:41.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.940+0000 7f2dfd7ff640 1 -- 172.21.15.16:0/4018146017 shutdown_connections 2024-09-19T07:48:41.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.940+0000 7f2dfd7ff640 1 --2- 172.21.15.16:0/4018146017 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2dd806c2b0 0x7f2dd806e770 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:41.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.940+0000 7f2dfd7ff640 1 --2- 172.21.15.16:0/4018146017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2df8162920 0x7f2df8164f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:41.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.940+0000 7f2dfd7ff640 1 --2- 172.21.15.16:0/4018146017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df80aff00 0x7f2df80b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:41.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.940+0000 7f2dfd7ff640 1 --2- 172.21.15.16:0/4018146017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df80afaa0 0x7f2df815ab50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:41.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.940+0000 7f2dfd7ff640 1 -- 172.21.15.16:0/4018146017 >> 172.21.15.16:0/4018146017 conn(0x7f2df8067d70 msgr2=0x7f2df8160030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:41.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.940+0000 7f2dfd7ff640 1 -- 172.21.15.16:0/4018146017 shutdown_connections 2024-09-19T07:48:41.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:41.940+0000 7f2dfd7ff640 1 -- 172.21.15.16:0/4018146017 wait complete. 2024-09-19T07:48:41.952 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm client client_inject_fixed_oldest_tid 2024-09-19T07:48:42.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.056+0000 7fb85ed6d640 1 Processor -- start 2024-09-19T07:48:42.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.056+0000 7fb85ed6d640 1 -- start start 2024-09-19T07:48:42.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.057+0000 7fb85ed6d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb85815e680 0x7fb8580cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:42.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.057+0000 7fb85ed6d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8580d1880 con 0x7fb8580d0010 2024-09-19T07:48:42.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.057+0000 7fb85ed6d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8580d1b60 con 0x7fb85815e2a0 2024-09-19T07:48:42.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.057+0000 7fb85ed6d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb8580cf2d0 con 0x7fb85815e680 2024-09-19T07:48:42.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.057+0000 7fb85d30a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb8580d0010 0x7fb8580d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56564/0 (socket says 172.21.15.16:56564) 2024-09-19T07:48:42.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.057+0000 7fb85d30a640 1 -- 172.21.15.16:0/1436625063 learned_addr learned my addr 172.21.15.16:0/1436625063 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:42.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.057+0000 7fb857fff640 1 --2- 172.21.15.16:0/1436625063 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb85815e680 0x7fb8580cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:42.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.057+0000 7fb8577fe640 1 -- 172.21.15.16:0/1436625063 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 997131385 0 0) 0x7fb8580d1880 con 0x7fb8580d0010 2024-09-19T07:48:42.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.058+0000 7fb8577fe640 1 -- 172.21.15.16:0/1436625063 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb844003520 con 0x7fb8580d0010 2024-09-19T07:48:42.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.058+0000 7fb8577fe640 1 -- 172.21.15.16:0/1436625063 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1931864110 0 0) 0x7fb8580d1b60 con 0x7fb85815e2a0 2024-09-19T07:48:42.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.058+0000 7fb8577fe640 1 -- 172.21.15.16:0/1436625063 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8580d1880 con 0x7fb85815e2a0 2024-09-19T07:48:42.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.058+0000 7fb857fff640 1 -- 172.21.15.16:0/1436625063 >> v1:172.21.15.146:6789/0 conn(0x7fb85815e2a0 legacy=0x7fb85806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:42.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.058+0000 7fb857fff640 1 -- 172.21.15.16:0/1436625063 >> v1:172.21.15.16:6789/0 conn(0x7fb8580d0010 legacy=0x7fb8580d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:42.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.058+0000 7fb857fff640 1 -- 172.21.15.16:0/1436625063 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8580cf5b0 con 0x7fb85815e680 2024-09-19T07:48:42.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.058+0000 7fb857fff640 1 --2- 172.21.15.16:0/1436625063 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb85815e680 0x7fb8580cfad0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb8400097d0 tx=0x7fb84002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f0c2e99955f5c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.059+0000 7fb8577fe640 1 -- 172.21.15.16:0/1436625063 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840037070 con 0x7fb85815e680 2024-09-19T07:48:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.059+0000 7fb8577fe640 1 -- 172.21.15.16:0/1436625063 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7fb840032440 con 0x7fb85815e680 2024-09-19T07:48:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.059+0000 7fb8577fe640 1 -- 172.21.15.16:0/1436625063 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840032810 con 0x7fb85815e680 2024-09-19T07:48:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.059+0000 7fb85ed6d640 1 -- 172.21.15.16:0/1436625063 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb85815e680 msgr2=0x7fb8580cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.059+0000 7fb85ed6d640 1 --2- 172.21.15.16:0/1436625063 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb85815e680 0x7fb8580cfad0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb8400097d0 tx=0x7fb84002a180 comp rx=0 tx=0).stop 2024-09-19T07:48:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.059+0000 7fb85ed6d640 1 -- 172.21.15.16:0/1436625063 shutdown_connections 2024-09-19T07:48:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.059+0000 7fb85ed6d640 1 --2- 172.21.15.16:0/1436625063 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb85815e680 0x7fb8580cfad0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.059+0000 7fb85ed6d640 1 -- 172.21.15.16:0/1436625063 >> 172.21.15.16:0/1436625063 conn(0x7fb85809d760 msgr2=0x7fb85808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.059+0000 7fb85ed6d640 1 -- 172.21.15.16:0/1436625063 shutdown_connections 2024-09-19T07:48:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.059+0000 7fb85ed6d640 1 -- 172.21.15.16:0/1436625063 wait complete. 2024-09-19T07:48:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.060+0000 7fb85ed6d640 1 Processor -- start 2024-09-19T07:48:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.060+0000 7fb85ed6d640 1 -- start start 2024-09-19T07:48:42.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.061+0000 7fb85ed6d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8580d0010 0x7fb8581507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:42.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.061+0000 7fb85ed6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb85815e2a0 0x7fb8580cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:42.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.061+0000 7fb85ed6d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb85815e680 0x7fb8581609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:42.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.061+0000 7fb85ed6d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb85806dcb0 con 0x7fb85815e2a0 2024-09-19T07:48:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.061+0000 7fb85ed6d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb85806db30 con 0x7fb8580d0010 2024-09-19T07:48:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.061+0000 7fb85ed6d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb8580d4a80 con 0x7fb85815e680 2024-09-19T07:48:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.062+0000 7fb85cb09640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8580d0010 0x7fb8581507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.062+0000 7fb857fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb85815e2a0 0x7fb8580cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.062+0000 7fb85cb09640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8580d0010 0x7fb8581507b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:59300/0 (socket says 172.21.15.16:59300) 2024-09-19T07:48:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.062+0000 7fb85d30a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb85815e680 0x7fb8581609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.062+0000 7fb85cb09640 1 -- 172.21.15.16:0/1720407156 learned_addr learned my addr 172.21.15.16:0/1720407156 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.062+0000 7fb857fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb85815e2a0 0x7fb8580cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36332/0 (socket says 172.21.15.16:36332) 2024-09-19T07:48:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.062+0000 7fb857fff640 1 -- 172.21.15.16:0/1720407156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb85815e680 msgr2=0x7fb8581609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.062+0000 7fb857fff640 1 --2- 172.21.15.16:0/1720407156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb85815e680 0x7fb8581609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.062+0000 7fb857fff640 1 -- 172.21.15.16:0/1720407156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8580d0010 msgr2=0x7fb8581507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.062+0000 7fb857fff640 1 --2- 172.21.15.16:0/1720407156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8580d0010 0x7fb8581507b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.062+0000 7fb857fff640 1 -- 172.21.15.16:0/1720407156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8580cb9f0 con 0x7fb85815e2a0 2024-09-19T07:48:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.062+0000 7fb857fff640 1 --2- 172.21.15.16:0/1720407156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb85815e2a0 0x7fb8580cb4b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb8580d1ee0 tx=0x7fb8400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.062+0000 7fb8557fa640 1 -- 172.21.15.16:0/1720407156 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840041030 con 0x7fb85815e2a0 2024-09-19T07:48:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.062+0000 7fb8557fa640 1 -- 172.21.15.16:0/1720407156 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 47+0+0 (secure 0 0 0) 0x7fb84003d4e0 con 0x7fb85815e2a0 2024-09-19T07:48:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.062+0000 7fb8557fa640 1 -- 172.21.15.16:0/1720407156 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840037040 con 0x7fb85815e2a0 2024-09-19T07:48:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.062+0000 7fb85ed6d640 1 -- 172.21.15.16:0/1720407156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8580cdfb0 con 0x7fb85815e2a0 2024-09-19T07:48:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.062+0000 7fb85ed6d640 1 -- 172.21.15.16:0/1720407156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8580ce570 con 0x7fb85815e2a0 2024-09-19T07:48:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.063+0000 7fb85ed6d640 1 -- 172.21.15.16:0/1720407156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8580d2f40 con 0x7fb85815e2a0 2024-09-19T07:48:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.064+0000 7fb8557fa640 1 -- 172.21.15.16:0/1720407156 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb840044070 con 0x7fb85815e2a0 2024-09-19T07:48:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.064+0000 7fb8557fa640 1 --2- 172.21.15.16:0/1720407156 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb83806c2b0 0x7fb83806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.064+0000 7fb8557fa640 1 -- 172.21.15.16:0/1720407156 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5378+0+0 (secure 0 0 0) 0x7fb8400ab960 con 0x7fb85815e2a0 2024-09-19T07:48:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.064+0000 7fb85cb09640 1 --2- 172.21.15.16:0/1720407156 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb83806c2b0 0x7fb83806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.065+0000 7fb85cb09640 1 --2- 172.21.15.16:0/1720407156 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb83806c2b0 0x7fb83806e770 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb848007fd0 tx=0x7fb848009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:42.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.069+0000 7fb8557fa640 1 -- 172.21.15.16:0/1720407156 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb840079b30 con 0x7fb85815e2a0 2024-09-19T07:48:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.244+0000 7fb85ed6d640 1 -- 172.21.15.16:0/1720407156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "client", "name": "client_inject_fixed_oldest_tid"} v 0) -- 0x7fb85806cf90 con 0x7fb85815e2a0 2024-09-19T07:48:42.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.247+0000 7fb8557fa640 1 -- 172.21.15.16:0/1720407156 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "client", "name": "client_inject_fixed_oldest_tid"}]=0 v22) ==== 116+0+0 (secure 0 0 0) 0x7fb84007de30 con 0x7fb85815e2a0 2024-09-19T07:48:42.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.247+0000 7fb8557fa640 1 -- 172.21.15.16:0/1720407156 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8400826f0 con 0x7fb85815e2a0 2024-09-19T07:48:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.249+0000 7fb85ed6d640 1 -- 172.21.15.16:0/1720407156 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb83806c2b0 msgr2=0x7fb83806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.249+0000 7fb85ed6d640 1 --2- 172.21.15.16:0/1720407156 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb83806c2b0 0x7fb83806e770 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb848007fd0 tx=0x7fb848009040 comp rx=0 tx=0).stop 2024-09-19T07:48:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.249+0000 7fb85ed6d640 1 -- 172.21.15.16:0/1720407156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb85815e2a0 msgr2=0x7fb8580cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.249+0000 7fb85ed6d640 1 --2- 172.21.15.16:0/1720407156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb85815e2a0 0x7fb8580cb4b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb8580d1ee0 tx=0x7fb8400046c0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.249+0000 7fb85ed6d640 1 -- 172.21.15.16:0/1720407156 shutdown_connections 2024-09-19T07:48:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.249+0000 7fb85ed6d640 1 --2- 172.21.15.16:0/1720407156 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb83806c2b0 0x7fb83806e770 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.249+0000 7fb85ed6d640 1 --2- 172.21.15.16:0/1720407156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb85815e680 0x7fb8581609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.249+0000 7fb85ed6d640 1 --2- 172.21.15.16:0/1720407156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb85815e2a0 0x7fb8580cb4b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.249+0000 7fb85ed6d640 1 --2- 172.21.15.16:0/1720407156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8580d0010 0x7fb8581507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.249+0000 7fb85ed6d640 1 -- 172.21.15.16:0/1720407156 >> 172.21.15.16:0/1720407156 conn(0x7fb85809d760 msgr2=0x7fb85806d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.249+0000 7fb85ed6d640 1 -- 172.21.15.16:0/1720407156 shutdown_connections 2024-09-19T07:48:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.249+0000 7fb85ed6d640 1 -- 172.21.15.16:0/1720407156 wait complete. 2024-09-19T07:48:42.261 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T07:48:42.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.394+0000 7fb31ef54640 1 Processor -- start 2024-09-19T07:48:42.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.394+0000 7fb31ef54640 1 -- start start 2024-09-19T07:48:42.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.396+0000 7fb31ef54640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb31806ff50 0x7fb31815a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:42.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.396+0000 7fb31ef54640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3180b49a0 con 0x7fb31806efa0 2024-09-19T07:48:42.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.396+0000 7fb31ef54640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3180b4c80 con 0x7fb3180b44a0 2024-09-19T07:48:42.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.396+0000 7fb31ef54640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb3180b4f60 con 0x7fb31806ff50 2024-09-19T07:48:42.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.396+0000 7fb317fff640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fb3180b44a0 0x7fb3180cf900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:47132/0 (socket says 172.21.15.16:47132) 2024-09-19T07:48:42.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.396+0000 7fb317fff640 1 -- 172.21.15.16:0/2958574534 learned_addr learned my addr 172.21.15.16:0/2958574534 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:42.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.398+0000 7fb31ccf0640 1 --2- 172.21.15.16:0/2958574534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb31806ff50 0x7fb31815a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:42.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.398+0000 7fb3177fe640 1 -- 172.21.15.16:0/2958574534 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3139093179 0 0) 0x7fb3180b49a0 con 0x7fb31806efa0 2024-09-19T07:48:42.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.398+0000 7fb3177fe640 1 -- 172.21.15.16:0/2958574534 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb300003520 con 0x7fb31806efa0 2024-09-19T07:48:42.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.398+0000 7fb3177fe640 1 -- 172.21.15.16:0/2958574534 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 127286210 0 0) 0x7fb300003520 con 0x7fb31806efa0 2024-09-19T07:48:42.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.398+0000 7fb3177fe640 1 -- 172.21.15.16:0/2958574534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb31806ff50 msgr2=0x7fb31815a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:42.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.398+0000 7fb3177fe640 1 --2- 172.21.15.16:0/2958574534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb31806ff50 0x7fb31815a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.398+0000 7fb3177fe640 1 -- 172.21.15.16:0/2958574534 >> v1:172.21.15.146:6789/0 conn(0x7fb3180b44a0 legacy=0x7fb3180cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:42.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.398+0000 7fb3177fe640 1 -- 172.21.15.16:0/2958574534 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31806ea80 con 0x7fb31806efa0 2024-09-19T07:48:42.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.398+0000 7fb3177fe640 1 -- 172.21.15.16:0/2958574534 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fb30c001670 con 0x7fb31806efa0 2024-09-19T07:48:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.398+0000 7fb3177fe640 1 -- 172.21.15.16:0/2958574534 >> v1:172.21.15.16:6789/0 conn(0x7fb31806efa0 legacy=0x7fb3180cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.398+0000 7fb3177fe640 1 --2- 172.21.15.16:0/2958574534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3000039e0 0x7fb300003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.398+0000 7fb3177fe640 1 --2- 172.21.15.16:0/2958574534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb31806ff50 0x7fb300004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.398+0000 7fb3177fe640 1 --2- 172.21.15.16:0/2958574534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb300004d40 0x7fb3000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.398+0000 7fb3177fe640 1 -- 172.21.15.16:0/2958574534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3180b4f60 con 0x7fb300004d40 2024-09-19T07:48:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.398+0000 7fb3177fe640 1 -- 172.21.15.16:0/2958574534 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb30000ba70 con 0x7fb3000039e0 2024-09-19T07:48:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.398+0000 7fb3177fe640 1 -- 172.21.15.16:0/2958574534 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb30000bc70 con 0x7fb31806ff50 2024-09-19T07:48:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.399+0000 7fb317fff640 1 --2- 172.21.15.16:0/2958574534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb31806ff50 0x7fb300004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.399+0000 7fb31ccf0640 1 --2- 172.21.15.16:0/2958574534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3000039e0 0x7fb300003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.399+0000 7fb31d4f1640 1 --2- 172.21.15.16:0/2958574534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb300004d40 0x7fb3000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.399+0000 7fb317fff640 1 -- 172.21.15.16:0/2958574534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3000039e0 msgr2=0x7fb300003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.399+0000 7fb317fff640 1 --2- 172.21.15.16:0/2958574534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3000039e0 0x7fb300003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.399+0000 7fb317fff640 1 -- 172.21.15.16:0/2958574534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb300004d40 msgr2=0x7fb3000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:42.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.399+0000 7fb317fff640 1 --2- 172.21.15.16:0/2958574534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb300004d40 0x7fb3000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.399+0000 7fb317fff640 1 -- 172.21.15.16:0/2958574534 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31806ea80 con 0x7fb31806ff50 2024-09-19T07:48:42.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.399+0000 7fb31d4f1640 1 --2- 172.21.15.16:0/2958574534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb300004d40 0x7fb3000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:48:42.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.402+0000 7fb317fff640 1 --2- 172.21.15.16:0/2958574534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb31806ff50 0x7fb300004800 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb3180b48d0 tx=0x7fb300005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f012a351eef9f418 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:42.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.403+0000 7fb3177fe640 1 -- 172.21.15.16:0/2958574534 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310001f80 con 0x7fb31806ff50 2024-09-19T07:48:42.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.403+0000 7fb3177fe640 1 -- 172.21.15.16:0/2958574534 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb310032630 con 0x7fb31806ff50 2024-09-19T07:48:42.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.403+0000 7fb3177fe640 1 -- 172.21.15.16:0/2958574534 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310032a20 con 0x7fb31806ff50 2024-09-19T07:48:42.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.405+0000 7fb31ef54640 1 -- 172.21.15.16:0/2958574534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb31806ff50 msgr2=0x7fb300004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:42.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.405+0000 7fb31ef54640 1 --2- 172.21.15.16:0/2958574534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb31806ff50 0x7fb300004800 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb3180b48d0 tx=0x7fb300005830 comp rx=0 tx=0).stop 2024-09-19T07:48:42.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.406+0000 7fb31d4f1640 1 -- 172.21.15.16:0/2958574534 reap_dead start 2024-09-19T07:48:42.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.406+0000 7fb31ef54640 1 -- 172.21.15.16:0/2958574534 shutdown_connections 2024-09-19T07:48:42.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.406+0000 7fb31ef54640 1 -- 172.21.15.16:0/2958574534 >> 172.21.15.16:0/2958574534 conn(0x7fb31806bdf0 msgr2=0x7fb31806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:42.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.406+0000 7fb31ef54640 1 -- 172.21.15.16:0/2958574534 shutdown_connections 2024-09-19T07:48:42.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.406+0000 7fb31ef54640 1 -- 172.21.15.16:0/2958574534 wait complete. 2024-09-19T07:48:42.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.408+0000 7fb31ef54640 1 Processor -- start 2024-09-19T07:48:42.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.409+0000 7fb31ef54640 1 -- start start 2024-09-19T07:48:42.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.410+0000 7fb31ef54640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb31806ff50 0x7fb3180cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:42.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.410+0000 7fb31ef54640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3180ca310 0x7fb3180ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:42.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.410+0000 7fb31ef54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3180c7340 0x7fb3180b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:42.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.410+0000 7fb31ef54640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb31806f3b0 con 0x7fb3180c7340 2024-09-19T07:48:42.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.410+0000 7fb31ef54640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb3180ce7a0 con 0x7fb3180ca310 2024-09-19T07:48:42.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.410+0000 7fb31ef54640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb31806f530 con 0x7fb31806ff50 2024-09-19T07:48:42.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.411+0000 7fb31d4f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3180c7340 0x7fb3180b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:42.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.411+0000 7fb31d4f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3180c7340 0x7fb3180b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36358/0 (socket says 172.21.15.16:36358) 2024-09-19T07:48:42.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.411+0000 7fb31d4f1640 1 -- 172.21.15.16:0/3151342834 learned_addr learned my addr 172.21.15.16:0/3151342834 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:42.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.411+0000 7fb317fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3180ca310 0x7fb3180ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:42.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.411+0000 7fb31ccf0640 1 --2- 172.21.15.16:0/3151342834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb31806ff50 0x7fb3180cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:42.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.411+0000 7fb31d4f1640 1 -- 172.21.15.16:0/3151342834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb31806ff50 msgr2=0x7fb3180cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:42.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.411+0000 7fb31d4f1640 1 --2- 172.21.15.16:0/3151342834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb31806ff50 0x7fb3180cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.411+0000 7fb31d4f1640 1 -- 172.21.15.16:0/3151342834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3180ca310 msgr2=0x7fb3180ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:42.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.411+0000 7fb31d4f1640 1 --2- 172.21.15.16:0/3151342834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3180ca310 0x7fb3180ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.411+0000 7fb31d4f1640 1 -- 172.21.15.16:0/3151342834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3180c8280 con 0x7fb3180c7340 2024-09-19T07:48:42.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.412+0000 7fb317fff640 1 --2- 172.21.15.16:0/3151342834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3180ca310 0x7fb3180ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:48:42.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.412+0000 7fb31d4f1640 1 --2- 172.21.15.16:0/3151342834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3180c7340 0x7fb3180b3180 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb30c001eb0 tx=0x7fb30c00ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:42.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.412+0000 7fb3157fa640 1 -- 172.21.15.16:0/3151342834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c007d30 con 0x7fb3180c7340 2024-09-19T07:48:42.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.412+0000 7fb3157fa640 1 -- 172.21.15.16:0/3151342834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb30c01c070 con 0x7fb3180c7340 2024-09-19T07:48:42.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.412+0000 7fb3157fa640 1 -- 172.21.15.16:0/3151342834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c018bc0 con 0x7fb3180c7340 2024-09-19T07:48:42.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.413+0000 7fb31ef54640 1 -- 172.21.15.16:0/3151342834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31815d290 con 0x7fb3180c7340 2024-09-19T07:48:42.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.413+0000 7fb31ef54640 1 -- 172.21.15.16:0/3151342834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3180c7bc0 con 0x7fb3180c7340 2024-09-19T07:48:42.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.415+0000 7fb31ef54640 1 -- 172.21.15.16:0/3151342834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3180cde40 con 0x7fb3180c7340 2024-09-19T07:48:42.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.416+0000 7fb3157fa640 1 -- 172.21.15.16:0/3151342834 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb30c00d720 con 0x7fb3180c7340 2024-09-19T07:48:42.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.416+0000 7fb3157fa640 1 --2- 172.21.15.16:0/3151342834 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb30406c380 0x7fb30406e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:42.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.416+0000 7fb3157fa640 1 -- 172.21.15.16:0/3151342834 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5378+0+0 (secure 0 0 0) 0x7fb30c093750 con 0x7fb3180c7340 2024-09-19T07:48:42.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.417+0000 7fb31ccf0640 1 --2- 172.21.15.16:0/3151342834 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb30406c380 0x7fb30406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:42.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.420+0000 7fb31ccf0640 1 --2- 172.21.15.16:0/3151342834 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb30406c380 0x7fb30406e840 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb31815c920 tx=0x7fb308008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:42.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.425+0000 7fb3157fa640 1 -- 172.21.15.16:0/3151342834 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb30c00e040 con 0x7fb3180c7340 2024-09-19T07:48:42.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.762+0000 7fb31ef54640 1 -- 172.21.15.16:0/3151342834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb3180cfde0 con 0x7fb3180c7340 2024-09-19T07:48:42.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.766+0000 7fb3157fa640 1 -- 172.21.15.16:0/3151342834 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7fb30c061990 con 0x7fb3180c7340 2024-09-19T07:48:42.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.769+0000 7fb31ef54640 1 -- 172.21.15.16:0/3151342834 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb30406c380 msgr2=0x7fb30406e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:42.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.769+0000 7fb31ef54640 1 --2- 172.21.15.16:0/3151342834 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb30406c380 0x7fb30406e840 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb31815c920 tx=0x7fb308008040 comp rx=0 tx=0).stop 2024-09-19T07:48:42.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.769+0000 7fb31ef54640 1 -- 172.21.15.16:0/3151342834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3180c7340 msgr2=0x7fb3180b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:42.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.769+0000 7fb31ef54640 1 --2- 172.21.15.16:0/3151342834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3180c7340 0x7fb3180b3180 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb30c001eb0 tx=0x7fb30c00ef80 comp rx=0 tx=0).stop 2024-09-19T07:48:42.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.769+0000 7fb31ef54640 1 -- 172.21.15.16:0/3151342834 shutdown_connections 2024-09-19T07:48:42.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.769+0000 7fb31ef54640 1 --2- 172.21.15.16:0/3151342834 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb30406c380 0x7fb30406e840 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.769+0000 7fb31ef54640 1 --2- 172.21.15.16:0/3151342834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3180c7340 0x7fb3180b3180 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.769+0000 7fb31ef54640 1 --2- 172.21.15.16:0/3151342834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3180ca310 0x7fb3180ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.769+0000 7fb31ef54640 1 --2- 172.21.15.16:0/3151342834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb31806ff50 0x7fb3180cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.769+0000 7fb31ef54640 1 -- 172.21.15.16:0/3151342834 >> 172.21.15.16:0/3151342834 conn(0x7fb31806bdf0 msgr2=0x7fb31806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:42.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.769+0000 7fb31ef54640 1 -- 172.21.15.16:0/3151342834 shutdown_connections 2024-09-19T07:48:42.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.769+0000 7fb31ef54640 1 -- 172.21.15.16:0/3151342834 wait complete. 2024-09-19T07:48:42.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_max_completed_requests 2024-09-19T07:48:42.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.892+0000 7fc64c531640 1 Processor -- start 2024-09-19T07:48:42.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.892+0000 7fc64c531640 1 -- start start 2024-09-19T07:48:42.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.893+0000 7fc64c531640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc6440d4370 0x7fc6440d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:42.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.893+0000 7fc64c531640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6440d1880 con 0x7fc64415e680 2024-09-19T07:48:42.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.893+0000 7fc64c531640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6440d1b60 con 0x7fc64415e2a0 2024-09-19T07:48:42.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.893+0000 7fc64c531640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc6440cf2d0 con 0x7fc6440d4370 2024-09-19T07:48:42.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.893+0000 7fc649acc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc64415e680 0x7fc6440cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38504/0 (socket says 172.21.15.16:38504) 2024-09-19T07:48:42.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.893+0000 7fc649acc640 1 -- 172.21.15.16:0/637735392 learned_addr learned my addr 172.21.15.16:0/637735392 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:42.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.893+0000 7fc64aace640 1 --2- 172.21.15.16:0/637735392 >> v2:172.21.15.146:3300/0 conn(0x7fc6440d4370 0x7fc6440d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:42.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.893+0000 7fc6492cb640 1 -- 172.21.15.16:0/637735392 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1111001629 0 0) 0x7fc6440d1880 con 0x7fc64415e680 2024-09-19T07:48:42.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.893+0000 7fc6492cb640 1 -- 172.21.15.16:0/637735392 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc630003520 con 0x7fc64415e680 2024-09-19T07:48:42.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.893+0000 7fc6492cb640 1 -- 172.21.15.16:0/637735392 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1169298631 0 0) 0x7fc6440d1b60 con 0x7fc64415e2a0 2024-09-19T07:48:42.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.893+0000 7fc6492cb640 1 -- 172.21.15.16:0/637735392 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6440d1880 con 0x7fc64415e2a0 2024-09-19T07:48:42.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.893+0000 7fc64aace640 1 -- 172.21.15.16:0/637735392 >> v1:172.21.15.146:6789/0 conn(0x7fc64415e2a0 legacy=0x7fc64406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:42.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.893+0000 7fc64aace640 1 -- 172.21.15.16:0/637735392 >> v1:172.21.15.16:6789/0 conn(0x7fc64415e680 legacy=0x7fc6440cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:42.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.894+0000 7fc6492cb640 1 -- 172.21.15.16:0/637735392 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1803268480 0 0) 0x7fc630003520 con 0x7fc64415e680 2024-09-19T07:48:42.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.894+0000 7fc64aace640 1 -- 172.21.15.16:0/637735392 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6440cf5b0 con 0x7fc6440d4370 2024-09-19T07:48:42.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.894+0000 7fc64aace640 1 --2- 172.21.15.16:0/637735392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6440d4370 0x7fc6440d4820 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc63c009d90 tx=0x7fc63c02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cb1501a27402d40 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:42.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.895+0000 7fc6492cb640 1 -- 172.21.15.16:0/637735392 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638001670 con 0x7fc6440d4370 2024-09-19T07:48:42.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.895+0000 7fc6492cb640 1 -- 172.21.15.16:0/637735392 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63c007540 con 0x7fc6440d4370 2024-09-19T07:48:42.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.895+0000 7fc6492cb640 1 -- 172.21.15.16:0/637735392 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c0078f0 con 0x7fc6440d4370 2024-09-19T07:48:42.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.895+0000 7fc64c531640 1 -- 172.21.15.16:0/637735392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6440d4370 msgr2=0x7fc6440d4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:42.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.895+0000 7fc64c531640 1 --2- 172.21.15.16:0/637735392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6440d4370 0x7fc6440d4820 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc63c009d90 tx=0x7fc63c02a680 comp rx=0 tx=0).stop 2024-09-19T07:48:42.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.895+0000 7fc64c531640 1 -- 172.21.15.16:0/637735392 shutdown_connections 2024-09-19T07:48:42.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.895+0000 7fc64c531640 1 --2- 172.21.15.16:0/637735392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6440d4370 0x7fc6440d4820 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.895+0000 7fc64c531640 1 -- 172.21.15.16:0/637735392 >> 172.21.15.16:0/637735392 conn(0x7fc64409d760 msgr2=0x7fc64408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:42.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.895+0000 7fc64c531640 1 -- 172.21.15.16:0/637735392 shutdown_connections 2024-09-19T07:48:42.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.895+0000 7fc64c531640 1 -- 172.21.15.16:0/637735392 wait complete. 2024-09-19T07:48:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.896+0000 7fc64c531640 1 Processor -- start 2024-09-19T07:48:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.896+0000 7fc64c531640 1 -- start start 2024-09-19T07:48:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.897+0000 7fc64c531640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6440d4370 0x7fc64415d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.897+0000 7fc64c531640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc64415e2a0 0x7fc6440a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.897+0000 7fc64c531640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc64415e680 0x7fc6440a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.897+0000 7fc64c531640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc64406dcb0 con 0x7fc64415e2a0 2024-09-19T07:48:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.897+0000 7fc64c531640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc64406db30 con 0x7fc64415e680 2024-09-19T07:48:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.897+0000 7fc64c531640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc6440d01e0 con 0x7fc6440d4370 2024-09-19T07:48:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.897+0000 7fc649acc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc64415e2a0 0x7fc6440a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.897+0000 7fc649acc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc64415e2a0 0x7fc6440a14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59966/0 (socket says 172.21.15.16:59966) 2024-09-19T07:48:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.897+0000 7fc649acc640 1 -- 172.21.15.16:0/172915073 learned_addr learned my addr 172.21.15.16:0/172915073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.897+0000 7fc64a2cd640 1 --2- 172.21.15.16:0/172915073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6440d4370 0x7fc64415d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.897+0000 7fc64aace640 1 --2- 172.21.15.16:0/172915073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc64415e680 0x7fc6440a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.898+0000 7fc649acc640 1 -- 172.21.15.16:0/172915073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6440d4370 msgr2=0x7fc64415d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.898+0000 7fc649acc640 1 --2- 172.21.15.16:0/172915073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6440d4370 0x7fc64415d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.898+0000 7fc649acc640 1 -- 172.21.15.16:0/172915073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc64415e680 msgr2=0x7fc6440a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.898+0000 7fc649acc640 1 --2- 172.21.15.16:0/172915073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc64415e680 0x7fc6440a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.898+0000 7fc649acc640 1 -- 172.21.15.16:0/172915073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc64415de50 con 0x7fc64415e2a0 2024-09-19T07:48:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.898+0000 7fc64aace640 1 --2- 172.21.15.16:0/172915073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc64415e680 0x7fc6440a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.898+0000 7fc64a2cd640 1 --2- 172.21.15.16:0/172915073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6440d4370 0x7fc64415d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.898+0000 7fc649acc640 1 --2- 172.21.15.16:0/172915073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc64415e2a0 0x7fc6440a14a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc638000fd0 tx=0x7fc63800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.898+0000 7fc636ffd640 1 -- 172.21.15.16:0/172915073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638017070 con 0x7fc64415e2a0 2024-09-19T07:48:42.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.898+0000 7fc636ffd640 1 -- 172.21.15.16:0/172915073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc638005ad0 con 0x7fc64415e2a0 2024-09-19T07:48:42.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.898+0000 7fc64c531640 1 -- 172.21.15.16:0/172915073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6441eacf0 con 0x7fc64415e2a0 2024-09-19T07:48:42.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.898+0000 7fc636ffd640 1 -- 172.21.15.16:0/172915073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638007070 con 0x7fc64415e2a0 2024-09-19T07:48:42.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.898+0000 7fc64c531640 1 -- 172.21.15.16:0/172915073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6441ea210 con 0x7fc64415e2a0 2024-09-19T07:48:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.899+0000 7fc64c531640 1 -- 172.21.15.16:0/172915073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6440d4820 con 0x7fc64415e2a0 2024-09-19T07:48:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.900+0000 7fc636ffd640 1 -- 172.21.15.16:0/172915073 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc63801a070 con 0x7fc64415e2a0 2024-09-19T07:48:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.900+0000 7fc636ffd640 1 --2- 172.21.15.16:0/172915073 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc62806c2b0 0x7fc62806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.900+0000 7fc636ffd640 1 -- 172.21.15.16:0/172915073 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5378+0+0 (secure 0 0 0) 0x7fc63808a5f0 con 0x7fc64415e2a0 2024-09-19T07:48:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.900+0000 7fc64a2cd640 1 --2- 172.21.15.16:0/172915073 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc62806c2b0 0x7fc62806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.901+0000 7fc64a2cd640 1 --2- 172.21.15.16:0/172915073 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc62806c2b0 0x7fc62806e770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc6400011e0 tx=0x7fc640009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:42.904+0000 7fc636ffd640 1 -- 172.21.15.16:0/172915073 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc64415de50 con 0x7fc64415e2a0 2024-09-19T07:48:43.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.079+0000 7fc64c531640 1 -- 172.21.15.16:0/172915073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_max_completed_requests"} v 0) -- 0x7fc64406cf90 con 0x7fc64415e2a0 2024-09-19T07:48:43.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.082+0000 7fc636ffd640 1 -- 172.21.15.16:0/172915073 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_max_completed_requests"}]=0 v24) ==== 109+0+0 (secure 0 0 0) 0x7fc638058850 con 0x7fc64415e2a0 2024-09-19T07:48:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.084+0000 7fc64c531640 1 -- 172.21.15.16:0/172915073 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc62806c2b0 msgr2=0x7fc62806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.084+0000 7fc64c531640 1 --2- 172.21.15.16:0/172915073 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc62806c2b0 0x7fc62806e770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc6400011e0 tx=0x7fc640009040 comp rx=0 tx=0).stop 2024-09-19T07:48:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.084+0000 7fc64c531640 1 -- 172.21.15.16:0/172915073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc64415e2a0 msgr2=0x7fc6440a14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.084+0000 7fc64c531640 1 --2- 172.21.15.16:0/172915073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc64415e2a0 0x7fc6440a14a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc638000fd0 tx=0x7fc63800cbe0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.084+0000 7fc64c531640 1 -- 172.21.15.16:0/172915073 shutdown_connections 2024-09-19T07:48:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.084+0000 7fc64c531640 1 --2- 172.21.15.16:0/172915073 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc62806c2b0 0x7fc62806e770 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.084+0000 7fc64c531640 1 --2- 172.21.15.16:0/172915073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc64415e680 0x7fc6440a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.084+0000 7fc64c531640 1 --2- 172.21.15.16:0/172915073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc64415e2a0 0x7fc6440a14a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.084+0000 7fc64c531640 1 --2- 172.21.15.16:0/172915073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6440d4370 0x7fc64415d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.084+0000 7fc64c531640 1 -- 172.21.15.16:0/172915073 >> 172.21.15.16:0/172915073 conn(0x7fc64409d760 msgr2=0x7fc6440b9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:43.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.084+0000 7fc64c531640 1 -- 172.21.15.16:0/172915073 shutdown_connections 2024-09-19T07:48:43.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.084+0000 7fc64c531640 1 -- 172.21.15.16:0/172915073 wait complete. 2024-09-19T07:48:43.096 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid' 2024-09-19T07:48:43.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.198+0000 7fd9e54e6640 1 Processor -- start 2024-09-19T07:48:43.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.198+0000 7fd9e54e6640 1 -- start start 2024-09-19T07:48:43.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.199+0000 7fd9e54e6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd9e015e680 0x7fd9e00cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.199+0000 7fd9e54e6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9e00d1880 con 0x7fd9e015e2a0 2024-09-19T07:48:43.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.199+0000 7fd9e54e6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9e00d1b60 con 0x7fd9e00d0010 2024-09-19T07:48:43.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.199+0000 7fd9e54e6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd9e00cf2d0 con 0x7fd9e015e680 2024-09-19T07:48:43.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.199+0000 7fd9de7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd9e015e680 0x7fd9e00cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.199+0000 7fd9deffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd9e015e2a0 0x7fd9e006d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38508/0 (socket says 172.21.15.16:38508) 2024-09-19T07:48:43.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.199+0000 7fd9de7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd9e015e680 0x7fd9e00cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59968/0 (socket says 172.21.15.16:59968) 2024-09-19T07:48:43.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.199+0000 7fd9deffd640 1 -- 172.21.15.16:0/3760935542 learned_addr learned my addr 172.21.15.16:0/3760935542 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:43.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.199+0000 7fd9de7fc640 1 -- 172.21.15.16:0/3760935542 >> v1:172.21.15.146:6789/0 conn(0x7fd9e00d0010 legacy=0x7fd9e00d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:43.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.199+0000 7fd9de7fc640 1 -- 172.21.15.16:0/3760935542 >> v1:172.21.15.16:6789/0 conn(0x7fd9e015e2a0 legacy=0x7fd9e006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.199+0000 7fd9de7fc640 1 -- 172.21.15.16:0/3760935542 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e00cf5b0 con 0x7fd9e015e680 2024-09-19T07:48:43.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.199+0000 7fd9ddffb640 1 -- 172.21.15.16:0/3760935542 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2622894439 0 0) 0x7fd9e00d1880 con 0x7fd9e015e2a0 2024-09-19T07:48:43.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.200+0000 7fd9de7fc640 1 --2- 172.21.15.16:0/3760935542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e015e680 0x7fd9e00cfad0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80097a0 tx=0x7fd9c802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a491d0902fe5c37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:43.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.200+0000 7fd9ddffb640 1 -- 172.21.15.16:0/3760935542 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8037070 con 0x7fd9e015e680 2024-09-19T07:48:43.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.200+0000 7fd9ddffb640 1 -- 172.21.15.16:0/3760935542 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c802ae50 con 0x7fd9e015e680 2024-09-19T07:48:43.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.200+0000 7fd9ddffb640 1 -- 172.21.15.16:0/3760935542 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8032660 con 0x7fd9e015e680 2024-09-19T07:48:43.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.201+0000 7fd9e54e6640 1 -- 172.21.15.16:0/3760935542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e015e680 msgr2=0x7fd9e00cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.201+0000 7fd9e54e6640 1 --2- 172.21.15.16:0/3760935542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e015e680 0x7fd9e00cfad0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80097a0 tx=0x7fd9c802a140 comp rx=0 tx=0).stop 2024-09-19T07:48:43.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.201+0000 7fd9e54e6640 1 -- 172.21.15.16:0/3760935542 shutdown_connections 2024-09-19T07:48:43.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.201+0000 7fd9e54e6640 1 --2- 172.21.15.16:0/3760935542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e015e680 0x7fd9e00cfad0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.201+0000 7fd9e54e6640 1 -- 172.21.15.16:0/3760935542 >> 172.21.15.16:0/3760935542 conn(0x7fd9e009d760 msgr2=0x7fd9e008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:43.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.201+0000 7fd9e54e6640 1 -- 172.21.15.16:0/3760935542 shutdown_connections 2024-09-19T07:48:43.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.201+0000 7fd9e54e6640 1 -- 172.21.15.16:0/3760935542 wait complete. 2024-09-19T07:48:43.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.202+0000 7fd9e54e6640 1 Processor -- start 2024-09-19T07:48:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.202+0000 7fd9e54e6640 1 -- start start 2024-09-19T07:48:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.203+0000 7fd9e54e6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9e00d0010 0x7fd9e0156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.203+0000 7fd9e54e6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd9e015e2a0 0x7fd9e00cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.203+0000 7fd9e54e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e015e680 0x7fd9e00cde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.203+0000 7fd9e54e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e006dcb0 con 0x7fd9e015e680 2024-09-19T07:48:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.203+0000 7fd9e54e6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e006db30 con 0x7fd9e00d0010 2024-09-19T07:48:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.203+0000 7fd9e54e6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd9e00d4a80 con 0x7fd9e015e2a0 2024-09-19T07:48:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.203+0000 7fd9df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e015e680 0x7fd9e00cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.203+0000 7fd9df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e015e680 0x7fd9e00cde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59974/0 (socket says 172.21.15.16:59974) 2024-09-19T07:48:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.203+0000 7fd9df7fe640 1 -- 172.21.15.16:0/4177372460 learned_addr learned my addr 172.21.15.16:0/4177372460 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.203+0000 7fd9deffd640 1 --2- 172.21.15.16:0/4177372460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9e00d0010 0x7fd9e0156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.203+0000 7fd9de7fc640 1 --2- 172.21.15.16:0/4177372460 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd9e015e2a0 0x7fd9e00cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.203+0000 7fd9df7fe640 1 -- 172.21.15.16:0/4177372460 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd9e015e2a0 msgr2=0x7fd9e00cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.203+0000 7fd9df7fe640 1 --2- 172.21.15.16:0/4177372460 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd9e015e2a0 0x7fd9e00cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.203+0000 7fd9df7fe640 1 -- 172.21.15.16:0/4177372460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9e00d0010 msgr2=0x7fd9e0156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.203+0000 7fd9df7fe640 1 --2- 172.21.15.16:0/4177372460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9e00d0010 0x7fd9e0156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.203+0000 7fd9df7fe640 1 -- 172.21.15.16:0/4177372460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e00cd720 con 0x7fd9e015e680 2024-09-19T07:48:43.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.203+0000 7fd9de7fc640 1 --2- 172.21.15.16:0/4177372460 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd9e015e2a0 0x7fd9e00cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:43.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.204+0000 7fd9df7fe640 1 --2- 172.21.15.16:0/4177372460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e015e680 0x7fd9e00cde50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800c440 tx=0x7fd9d800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:43.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.204+0000 7fd9d77fe640 1 -- 172.21.15.16:0/4177372460 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8011790 con 0x7fd9e015e680 2024-09-19T07:48:43.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.204+0000 7fd9d77fe640 1 -- 172.21.15.16:0/4177372460 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d8017070 con 0x7fd9e015e680 2024-09-19T07:48:43.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.204+0000 7fd9e54e6640 1 -- 172.21.15.16:0/4177372460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e00cb850 con 0x7fd9e015e680 2024-09-19T07:48:43.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.204+0000 7fd9d77fe640 1 -- 172.21.15.16:0/4177372460 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8011b70 con 0x7fd9e015e680 2024-09-19T07:48:43.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.204+0000 7fd9e54e6640 1 -- 172.21.15.16:0/4177372460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e00cbe10 con 0x7fd9e015e680 2024-09-19T07:48:43.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.205+0000 7fd9e54e6640 1 -- 172.21.15.16:0/4177372460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9e00cfad0 con 0x7fd9e015e680 2024-09-19T07:48:43.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.205+0000 7fd9d77fe640 1 -- 172.21.15.16:0/4177372460 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd9d8011d10 con 0x7fd9e015e680 2024-09-19T07:48:43.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.206+0000 7fd9d77fe640 1 --2- 172.21.15.16:0/4177372460 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd9c006c2b0 0x7fd9c006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.206+0000 7fd9d77fe640 1 -- 172.21.15.16:0/4177372460 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5378+0+0 (secure 0 0 0) 0x7fd9d808b160 con 0x7fd9e015e680 2024-09-19T07:48:43.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.206+0000 7fd9deffd640 1 --2- 172.21.15.16:0/4177372460 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd9c006c2b0 0x7fd9c006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.207+0000 7fd9deffd640 1 --2- 172.21.15.16:0/4177372460 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd9c006c2b0 0x7fd9c006e770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00012e0 tx=0x7fd9d0007ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:43.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.210+0000 7fd9d77fe640 1 -- 172.21.15.16:0/4177372460 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd9d8059a10 con 0x7fd9e015e680 2024-09-19T07:48:43.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.451+0000 7fd9e54e6640 1 -- 172.21.15.16:0/4177372460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]} v 0) -- 0x7fd9e015d4f0 con 0x7fd9e015e680 2024-09-19T07:48:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.558+0000 7fd9d77fe640 1 -- 172.21.15.16:0/4177372460 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]}]=0 v246) ==== 162+0+0 (secure 0 0 0) 0x7fd9d8059310 con 0x7fd9e015e680 2024-09-19T07:48:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.560+0000 7fd9e54e6640 1 -- 172.21.15.16:0/4177372460 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd9c006c2b0 msgr2=0x7fd9c006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.560+0000 7fd9e54e6640 1 --2- 172.21.15.16:0/4177372460 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd9c006c2b0 0x7fd9c006e770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00012e0 tx=0x7fd9d0007ed0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.560+0000 7fd9e54e6640 1 -- 172.21.15.16:0/4177372460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e015e680 msgr2=0x7fd9e00cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.560+0000 7fd9e54e6640 1 --2- 172.21.15.16:0/4177372460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e015e680 0x7fd9e00cde50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800c440 tx=0x7fd9d800c900 comp rx=0 tx=0).stop 2024-09-19T07:48:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.560+0000 7fd9e54e6640 1 -- 172.21.15.16:0/4177372460 shutdown_connections 2024-09-19T07:48:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.560+0000 7fd9e54e6640 1 --2- 172.21.15.16:0/4177372460 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd9c006c2b0 0x7fd9c006e770 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.560+0000 7fd9e54e6640 1 --2- 172.21.15.16:0/4177372460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e015e680 0x7fd9e00cde50 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.560+0000 7fd9e54e6640 1 --2- 172.21.15.16:0/4177372460 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd9e015e2a0 0x7fd9e00cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.560+0000 7fd9e54e6640 1 --2- 172.21.15.16:0/4177372460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9e00d0010 0x7fd9e0156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.560+0000 7fd9e54e6640 1 -- 172.21.15.16:0/4177372460 >> 172.21.15.16:0/4177372460 conn(0x7fd9e009d760 msgr2=0x7fd9e0074ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.560+0000 7fd9e54e6640 1 -- 172.21.15.16:0/4177372460 shutdown_connections 2024-09-19T07:48:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.560+0000 7fd9e54e6640 1 -- 172.21.15.16:0/4177372460 wait complete. 2024-09-19T07:48:43.584 INFO:tasks.cephfs_test_runner:that a client is blocklisted when its encoded session metadata exceeds the ... ok 2024-09-19T07:48:43.585 INFO:tasks.cephfs_test_runner:Starting test: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-19T07:48:43.585 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:48:43.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.687+0000 7f2fc02d0640 1 Processor -- start 2024-09-19T07:48:43.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.687+0000 7f2fc02d0640 1 -- start start 2024-09-19T07:48:43.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.688+0000 7f2fc02d0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fb8141680 0x7f2fb80cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.688+0000 7f2fc02d0640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2fb80d0370 0x7f2fb8141a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.688+0000 7f2fc02d0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fb815d710 con 0x7f2fb8141fa0 2024-09-19T07:48:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.688+0000 7f2fc02d0640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2fb80d07b0 con 0x7f2fb80d0370 2024-09-19T07:48:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.688+0000 7f2fc02d0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb815c6f0 con 0x7f2fb8141680 2024-09-19T07:48:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.688+0000 7f2fbe86d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2fb8141fa0 0x7f2fb80cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38518/0 (socket says 172.21.15.16:38518) 2024-09-19T07:48:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.688+0000 7f2fbe86d640 1 -- 172.21.15.16:0/3735719215 learned_addr learned my addr 172.21.15.16:0/3735719215 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:43.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbd06a640 1 -- 172.21.15.16:0/3735719215 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3124872149 0 0) 0x7f2fb815d710 con 0x7f2fb8141fa0 2024-09-19T07:48:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbe06c640 1 --2- 172.21.15.16:0/3735719215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fb8141680 0x7f2fb80cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbd06a640 1 -- 172.21.15.16:0/3735719215 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fa0003520 con 0x7f2fb8141fa0 2024-09-19T07:48:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbd86b640 1 --2- 172.21.15.16:0/3735719215 >> v2:172.21.15.146:3300/0 conn(0x7f2fb80d0370 0x7f2fb8141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbd06a640 1 -- 172.21.15.16:0/3735719215 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 898476630 0 0) 0x7f2fa0003520 con 0x7f2fb8141fa0 2024-09-19T07:48:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbd06a640 1 -- 172.21.15.16:0/3735719215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fb8141680 msgr2=0x7f2fb80cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbd06a640 1 --2- 172.21.15.16:0/3735719215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fb8141680 0x7f2fb80cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbd06a640 1 -- 172.21.15.16:0/3735719215 >> v2:172.21.15.146:3300/0 conn(0x7f2fb80d0370 msgr2=0x7f2fb8141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbd06a640 1 --2- 172.21.15.16:0/3735719215 >> v2:172.21.15.146:3300/0 conn(0x7f2fb80d0370 0x7f2fb8141a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbd06a640 1 -- 172.21.15.16:0/3735719215 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb815c950 con 0x7f2fb8141fa0 2024-09-19T07:48:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbd06a640 1 -- 172.21.15.16:0/3735719215 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f2fb0001670 con 0x7f2fb8141fa0 2024-09-19T07:48:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbd06a640 1 -- 172.21.15.16:0/3735719215 >> v1:172.21.15.16:6789/0 conn(0x7f2fb8141fa0 legacy=0x7f2fb80cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbd06a640 1 --2- 172.21.15.16:0/3735719215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa00039b0 0x7f2fa0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbd06a640 1 --2- 172.21.15.16:0/3735719215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fa0004800 0x7f2fa0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbd06a640 1 --2- 172.21.15.16:0/3735719215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fb8141680 0x7f2fa00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbd06a640 1 -- 172.21.15.16:0/3735719215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb80d07b0 con 0x7f2fa00039b0 2024-09-19T07:48:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbd06a640 1 -- 172.21.15.16:0/3735719215 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb815c6f0 con 0x7f2fa0004800 2024-09-19T07:48:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbd06a640 1 -- 172.21.15.16:0/3735719215 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa000b9c0 con 0x7f2fb8141680 2024-09-19T07:48:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbe06c640 1 --2- 172.21.15.16:0/3735719215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa00039b0 0x7f2fa0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbd86b640 1 --2- 172.21.15.16:0/3735719215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fa0004800 0x7f2fa0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbe86d640 1 --2- 172.21.15.16:0/3735719215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fb8141680 0x7f2fa00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbe06c640 1 -- 172.21.15.16:0/3735719215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fb8141680 msgr2=0x7f2fa00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbe06c640 1 --2- 172.21.15.16:0/3735719215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fb8141680 0x7f2fa00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbe06c640 1 -- 172.21.15.16:0/3735719215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fa0004800 msgr2=0x7f2fa0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbe06c640 1 --2- 172.21.15.16:0/3735719215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fa0004800 0x7f2fa0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.689+0000 7f2fbe06c640 1 -- 172.21.15.16:0/3735719215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa0003520 con 0x7f2fa00039b0 2024-09-19T07:48:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.690+0000 7f2fbe06c640 1 --2- 172.21.15.16:0/3735719215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa00039b0 0x7f2fa0003dd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009fc0 tx=0x7f2fa0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06d826829663fef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.690+0000 7f2fbd06a640 1 -- 172.21.15.16:0/3735719215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8009c10 con 0x7f2fa00039b0 2024-09-19T07:48:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.690+0000 7f2fbd06a640 1 -- 172.21.15.16:0/3735719215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fa8036070 con 0x7f2fa00039b0 2024-09-19T07:48:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.690+0000 7f2fbd06a640 1 -- 172.21.15.16:0/3735719215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8031880 con 0x7f2fa00039b0 2024-09-19T07:48:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.691+0000 7f2fc02d0640 1 -- 172.21.15.16:0/3735719215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa00039b0 msgr2=0x7f2fa0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.691+0000 7f2fc02d0640 1 --2- 172.21.15.16:0/3735719215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa00039b0 0x7f2fa0003dd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009fc0 tx=0x7f2fa0005830 comp rx=0 tx=0).stop 2024-09-19T07:48:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.691+0000 7f2fbe86d640 1 -- 172.21.15.16:0/3735719215 reap_dead start 2024-09-19T07:48:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.691+0000 7f2fc02d0640 1 -- 172.21.15.16:0/3735719215 shutdown_connections 2024-09-19T07:48:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.691+0000 7f2fc02d0640 1 -- 172.21.15.16:0/3735719215 >> 172.21.15.16:0/3735719215 conn(0x7f2fb80b3ed0 msgr2=0x7f2fb80b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.691+0000 7f2fc02d0640 1 -- 172.21.15.16:0/3735719215 shutdown_connections 2024-09-19T07:48:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.691+0000 7f2fc02d0640 1 -- 172.21.15.16:0/3735719215 wait complete. 2024-09-19T07:48:43.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.692+0000 7f2fc02d0640 1 Processor -- start 2024-09-19T07:48:43.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.692+0000 7f2fc02d0640 1 -- start start 2024-09-19T07:48:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.693+0000 7f2fc02d0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fa00039b0 0x7f2fb80b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.693+0000 7f2fc02d0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fb806d730 0x7f2fb81582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.693+0000 7f2fc02d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb80b75b0 0x7f2fb80b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.693+0000 7f2fc02d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb815d1f0 con 0x7f2fb80b75b0 2024-09-19T07:48:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.693+0000 7f2fc02d0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb815cf10 con 0x7f2fa00039b0 2024-09-19T07:48:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.693+0000 7f2fc02d0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb81408c0 con 0x7f2fb806d730 2024-09-19T07:48:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.693+0000 7f2fbe86d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb80b75b0 0x7f2fb80b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.693+0000 7f2fbe86d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb80b75b0 0x7f2fb80b7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59986/0 (socket says 172.21.15.16:59986) 2024-09-19T07:48:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.693+0000 7f2fbe86d640 1 -- 172.21.15.16:0/2223523967 learned_addr learned my addr 172.21.15.16:0/2223523967 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.693+0000 7f2fbd86b640 1 --2- 172.21.15.16:0/2223523967 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fb806d730 0x7f2fb81582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.693+0000 7f2fbe06c640 1 --2- 172.21.15.16:0/2223523967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fa00039b0 0x7f2fb80b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.693+0000 7f2fbe86d640 1 -- 172.21.15.16:0/2223523967 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fb806d730 msgr2=0x7f2fb81582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.693+0000 7f2fbe86d640 1 --2- 172.21.15.16:0/2223523967 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fb806d730 0x7f2fb81582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.693+0000 7f2fbe86d640 1 -- 172.21.15.16:0/2223523967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fa00039b0 msgr2=0x7f2fb80b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.693+0000 7f2fbe86d640 1 --2- 172.21.15.16:0/2223523967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fa00039b0 0x7f2fb80b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.693+0000 7f2fbe86d640 1 -- 172.21.15.16:0/2223523967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb806dbc0 con 0x7f2fb80b75b0 2024-09-19T07:48:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.694+0000 7f2fbe86d640 1 --2- 172.21.15.16:0/2223523967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb80b75b0 0x7f2fb80b7a30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8142380 tx=0x7f2fb0007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.694+0000 7f2fb6ffd640 1 -- 172.21.15.16:0/2223523967 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0001640 con 0x7f2fb80b75b0 2024-09-19T07:48:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.694+0000 7f2fb6ffd640 1 -- 172.21.15.16:0/2223523967 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb0018070 con 0x7f2fb80b75b0 2024-09-19T07:48:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.694+0000 7f2fc02d0640 1 -- 172.21.15.16:0/2223523967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb806dea0 con 0x7f2fb80b75b0 2024-09-19T07:48:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.694+0000 7f2fb6ffd640 1 -- 172.21.15.16:0/2223523967 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb000fb50 con 0x7f2fb80b75b0 2024-09-19T07:48:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.694+0000 7f2fc02d0640 1 -- 172.21.15.16:0/2223523967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb806e460 con 0x7f2fb80b75b0 2024-09-19T07:48:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.695+0000 7f2fc02d0640 1 -- 172.21.15.16:0/2223523967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb8142430 con 0x7f2fb80b75b0 2024-09-19T07:48:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.695+0000 7f2fb6ffd640 1 -- 172.21.15.16:0/2223523967 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2fb000fcf0 con 0x7f2fb80b75b0 2024-09-19T07:48:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.696+0000 7f2fb6ffd640 1 --2- 172.21.15.16:0/2223523967 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2fa406c2b0 0x7f2fa406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.696+0000 7f2fb6ffd640 1 -- 172.21.15.16:0/2223523967 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5378+0+0 (secure 0 0 0) 0x7f2fb008afc0 con 0x7f2fb80b75b0 2024-09-19T07:48:43.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.696+0000 7f2fbe06c640 1 --2- 172.21.15.16:0/2223523967 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2fa406c2b0 0x7f2fa406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.697+0000 7f2fbe06c640 1 --2- 172.21.15.16:0/2223523967 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2fa406c2b0 0x7f2fa406e770 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8035000 tx=0x7f2fa802f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:43.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.700+0000 7f2fb6ffd640 1 -- 172.21.15.16:0/2223523967 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2fb00592b0 con 0x7f2fb80b75b0 2024-09-19T07:48:43.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.879+0000 7f2fc02d0640 1 -- 172.21.15.16:0/2223523967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2fb80690c0 con 0x7f2fb80b75b0 2024-09-19T07:48:43.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.880+0000 7f2fb6ffd640 1 -- 172.21.15.16:0/2223523967 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+13388 (secure 0 0 0) 0x7f2fb005d5b0 con 0x7f2fb80b75b0 2024-09-19T07:48:43.880 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:43.880 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":62,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:48:41.550882+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/642332950":"2024-09-20T07:48:38.545519+0000","172.21.15.16:6834/642332950":"2024-09-20T07:48:38.545519+0000","192.168.0.1:0/2345385426":"2024-09-19T08:48:30.430325+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:48:43.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.882+0000 7f2fc02d0640 1 -- 172.21.15.16:0/2223523967 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2fa406c2b0 msgr2=0x7f2fa406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.882+0000 7f2fc02d0640 1 --2- 172.21.15.16:0/2223523967 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2fa406c2b0 0x7f2fa406e770 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8035000 tx=0x7f2fa802f000 comp rx=0 tx=0).stop 2024-09-19T07:48:43.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.882+0000 7f2fc02d0640 1 -- 172.21.15.16:0/2223523967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb80b75b0 msgr2=0x7f2fb80b7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.882+0000 7f2fc02d0640 1 --2- 172.21.15.16:0/2223523967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb80b75b0 0x7f2fb80b7a30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8142380 tx=0x7f2fb0007720 comp rx=0 tx=0).stop 2024-09-19T07:48:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.882+0000 7f2fc02d0640 1 -- 172.21.15.16:0/2223523967 shutdown_connections 2024-09-19T07:48:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.882+0000 7f2fc02d0640 1 --2- 172.21.15.16:0/2223523967 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2fa406c2b0 0x7f2fa406e770 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.882+0000 7f2fc02d0640 1 --2- 172.21.15.16:0/2223523967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb80b75b0 0x7f2fb80b7a30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.882+0000 7f2fc02d0640 1 --2- 172.21.15.16:0/2223523967 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fb806d730 0x7f2fb81582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.882+0000 7f2fc02d0640 1 --2- 172.21.15.16:0/2223523967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fa00039b0 0x7f2fb80b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.882+0000 7f2fc02d0640 1 -- 172.21.15.16:0/2223523967 >> 172.21.15.16:0/2223523967 conn(0x7f2fb80b3ed0 msgr2=0x7f2fb80cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.882+0000 7f2fc02d0640 1 -- 172.21.15.16:0/2223523967 shutdown_connections 2024-09-19T07:48:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.882+0000 7f2fc02d0640 1 -- 172.21.15.16:0/2223523967 wait complete. 2024-09-19T07:48:43.894 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T07:42:54.500338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T07:48:43.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T07:48:43.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.999+0000 7fe6680e4640 1 Processor -- start 2024-09-19T07:48:43.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:43.999+0000 7fe6680e4640 1 -- start start 2024-09-19T07:48:44.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.000+0000 7fe6680e4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe66015e2a0 0x7fe66006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:44.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.000+0000 7fe6680e4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6600d1880 con 0x7fe66006d960 2024-09-19T07:48:44.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.000+0000 7fe6680e4640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6600d1b60 con 0x7fe66015e680 2024-09-19T07:48:44.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.000+0000 7fe6680e4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fe6600cf2d0 con 0x7fe66015e2a0 2024-09-19T07:48:44.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.000+0000 7fe665e80640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe66015e2a0 0x7fe66006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:44.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.000+0000 7fe666681640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe66006d960 0x7fe6600d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38520/0 (socket says 172.21.15.16:38520) 2024-09-19T07:48:44.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.000+0000 7fe665e80640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe66015e2a0 0x7fe66006d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59994/0 (socket says 172.21.15.16:59994) 2024-09-19T07:48:44.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.000+0000 7fe666681640 1 -- 172.21.15.16:0/101124593 learned_addr learned my addr 172.21.15.16:0/101124593 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:44.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.000+0000 7fe665e80640 1 -- 172.21.15.16:0/101124593 >> v1:172.21.15.146:6789/0 conn(0x7fe66015e680 legacy=0x7fe6600cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:44.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.000+0000 7fe665e80640 1 -- 172.21.15.16:0/101124593 >> v1:172.21.15.16:6789/0 conn(0x7fe66006d960 legacy=0x7fe6600d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.000+0000 7fe665e80640 1 -- 172.21.15.16:0/101124593 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6600cf5b0 con 0x7fe66015e2a0 2024-09-19T07:48:44.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.000+0000 7fe664e7e640 1 -- 172.21.15.16:0/101124593 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1238034316 0 0) 0x7fe6600d1880 con 0x7fe66006d960 2024-09-19T07:48:44.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.001+0000 7fe665e80640 1 --2- 172.21.15.16:0/101124593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66015e2a0 0x7fe66006d420 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0097a0 tx=0x7fe65c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c7c0256f111148 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:44.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.001+0000 7fe664e7e640 1 -- 172.21.15.16:0/101124593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c037070 con 0x7fe66015e2a0 2024-09-19T07:48:44.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.001+0000 7fe664e7e640 1 -- 172.21.15.16:0/101124593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe65c032440 con 0x7fe66015e2a0 2024-09-19T07:48:44.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.001+0000 7fe664e7e640 1 -- 172.21.15.16:0/101124593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c032810 con 0x7fe66015e2a0 2024-09-19T07:48:44.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.001+0000 7fe6680e4640 1 -- 172.21.15.16:0/101124593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66015e2a0 msgr2=0x7fe66006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.001+0000 7fe6680e4640 1 --2- 172.21.15.16:0/101124593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66015e2a0 0x7fe66006d420 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0097a0 tx=0x7fe65c02a1e0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.002+0000 7fe6680e4640 1 -- 172.21.15.16:0/101124593 shutdown_connections 2024-09-19T07:48:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.002+0000 7fe6680e4640 1 --2- 172.21.15.16:0/101124593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66015e2a0 0x7fe66006d420 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.002+0000 7fe6680e4640 1 -- 172.21.15.16:0/101124593 >> 172.21.15.16:0/101124593 conn(0x7fe66009d760 msgr2=0x7fe66008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.002+0000 7fe6680e4640 1 -- 172.21.15.16:0/101124593 shutdown_connections 2024-09-19T07:48:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.002+0000 7fe6680e4640 1 -- 172.21.15.16:0/101124593 wait complete. 2024-09-19T07:48:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.003+0000 7fe6680e4640 1 Processor -- start 2024-09-19T07:48:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.003+0000 7fe6680e4640 1 -- start start 2024-09-19T07:48:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.003+0000 7fe6680e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe66006d960 0x7fe6601597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe6680e4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe66015e2a0 0x7fe6600d3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe6680e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66015e680 0x7fe6600d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe6680e4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6600d0360 con 0x7fe66015e680 2024-09-19T07:48:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe6680e4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe6600d01e0 con 0x7fe66006d960 2024-09-19T07:48:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe6680e4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe6600d4a80 con 0x7fe66015e2a0 2024-09-19T07:48:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe666681640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66015e680 0x7fe6600d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe666681640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66015e680 0x7fe6600d36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59996/0 (socket says 172.21.15.16:59996) 2024-09-19T07:48:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe666681640 1 -- 172.21.15.16:0/353037636 learned_addr learned my addr 172.21.15.16:0/353037636 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe665e80640 1 --2- 172.21.15.16:0/353037636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe66006d960 0x7fe6601597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe66567f640 1 --2- 172.21.15.16:0/353037636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe66015e2a0 0x7fe6600d3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe666681640 1 -- 172.21.15.16:0/353037636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe66015e2a0 msgr2=0x7fe6600d3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe666681640 1 --2- 172.21.15.16:0/353037636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe66015e2a0 0x7fe6600d3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe666681640 1 -- 172.21.15.16:0/353037636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe66006d960 msgr2=0x7fe6601597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe666681640 1 --2- 172.21.15.16:0/353037636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe66006d960 0x7fe6601597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe666681640 1 -- 172.21.15.16:0/353037636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6600cdd00 con 0x7fe66015e680 2024-09-19T07:48:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe66567f640 1 --2- 172.21.15.16:0/353037636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe66015e2a0 0x7fe6600d3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe666681640 1 --2- 172.21.15.16:0/353037636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66015e680 0x7fe6600d36b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe65800c7a0 tx=0x7fe65800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe6567fc640 1 -- 172.21.15.16:0/353037636 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe658018070 con 0x7fe66015e680 2024-09-19T07:48:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe6567fc640 1 -- 172.21.15.16:0/353037636 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe65800a640 con 0x7fe66015e680 2024-09-19T07:48:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe6680e4640 1 -- 172.21.15.16:0/353037636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6600cdfe0 con 0x7fe66015e680 2024-09-19T07:48:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe6567fc640 1 -- 172.21.15.16:0/353037636 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65800aab0 con 0x7fe66015e680 2024-09-19T07:48:44.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.004+0000 7fe6680e4640 1 -- 172.21.15.16:0/353037636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6600ce5a0 con 0x7fe66015e680 2024-09-19T07:48:44.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.005+0000 7fe6680e4640 1 -- 172.21.15.16:0/353037636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6600ce8e0 con 0x7fe66015e680 2024-09-19T07:48:44.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.006+0000 7fe6567fc640 1 -- 172.21.15.16:0/353037636 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe65801b070 con 0x7fe66015e680 2024-09-19T07:48:44.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.006+0000 7fe6567fc640 1 --2- 172.21.15.16:0/353037636 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe64406c2b0 0x7fe64406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:44.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.007+0000 7fe6567fc640 1 -- 172.21.15.16:0/353037636 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5378+0+0 (secure 0 0 0) 0x7fe65808a840 con 0x7fe66015e680 2024-09-19T07:48:44.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.007+0000 7fe665e80640 1 --2- 172.21.15.16:0/353037636 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe64406c2b0 0x7fe64406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:44.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.008+0000 7fe665e80640 1 --2- 172.21.15.16:0/353037636 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe64406c2b0 0x7fe64406e770 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe65c034000 tx=0x7fe65c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:44.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.011+0000 7fe6567fc640 1 -- 172.21.15.16:0/353037636 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe6600ce8e0 con 0x7fe66015e680 2024-09-19T07:48:44.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.186+0000 7fe6680e4640 1 -- 172.21.15.16:0/353037636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe66006d420 con 0x7fe66015e680 2024-09-19T07:48:44.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.186+0000 7fe6567fc640 1 -- 172.21.15.16:0/353037636 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) ==== 93+0+10 (secure 0 0 0) 0x7fe658058aa0 con 0x7fe66015e680 2024-09-19T07:48:44.187 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-19T07:48:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.188+0000 7fe6680e4640 1 -- 172.21.15.16:0/353037636 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe64406c2b0 msgr2=0x7fe64406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.188+0000 7fe6680e4640 1 --2- 172.21.15.16:0/353037636 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe64406c2b0 0x7fe64406e770 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe65c034000 tx=0x7fe65c02e000 comp rx=0 tx=0).stop 2024-09-19T07:48:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.188+0000 7fe6680e4640 1 -- 172.21.15.16:0/353037636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66015e680 msgr2=0x7fe6600d36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.188+0000 7fe6680e4640 1 --2- 172.21.15.16:0/353037636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66015e680 0x7fe6600d36b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe65800c7a0 tx=0x7fe65800cc60 comp rx=0 tx=0).stop 2024-09-19T07:48:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.188+0000 7fe6680e4640 1 -- 172.21.15.16:0/353037636 shutdown_connections 2024-09-19T07:48:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.188+0000 7fe6680e4640 1 --2- 172.21.15.16:0/353037636 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe64406c2b0 0x7fe64406e770 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.188+0000 7fe6680e4640 1 --2- 172.21.15.16:0/353037636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66015e680 0x7fe6600d36b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.188+0000 7fe6680e4640 1 --2- 172.21.15.16:0/353037636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe66015e2a0 0x7fe6600d3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.188+0000 7fe6680e4640 1 --2- 172.21.15.16:0/353037636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe66006d960 0x7fe6601597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.189+0000 7fe6680e4640 1 -- 172.21.15.16:0/353037636 >> 172.21.15.16:0/353037636 conn(0x7fe66009d760 msgr2=0x7fe66015fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:44.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.189+0000 7fe6680e4640 1 -- 172.21.15.16:0/353037636 shutdown_connections 2024-09-19T07:48:44.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.189+0000 7fe6680e4640 1 -- 172.21.15.16:0/353037636 wait complete. 2024-09-19T07:48:44.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size' 2024-09-19T07:48:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.302+0000 7f14ca6e4640 1 Processor -- start 2024-09-19T07:48:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.302+0000 7f14ca6e4640 1 -- start start 2024-09-19T07:48:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.303+0000 7f14ca6e4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14c40d4840 0x7f14c415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.303+0000 7f14ca6e4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14c40d1b10 con 0x7f14c415f0d0 2024-09-19T07:48:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.303+0000 7f14ca6e4640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14c40cf2b0 con 0x7f14c40d0070 2024-09-19T07:48:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.303+0000 7f14ca6e4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f14c40cf590 con 0x7f14c40d4840 2024-09-19T07:48:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.303+0000 7f14c8c81640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14c40d4840 0x7f14c415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.303+0000 7f14c3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f14c415f0d0 0x7f14c40d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38532/0 (socket says 172.21.15.16:38532) 2024-09-19T07:48:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.303+0000 7f14c3fff640 1 -- 172.21.15.16:0/1825082732 learned_addr learned my addr 172.21.15.16:0/1825082732 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.303+0000 7f14c8c81640 1 --2- 172.21.15.16:0/1825082732 >> v2:172.21.15.16:3300/0 conn(0x7f14c40d4840 0x7f14c415f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60002/0 (socket says 172.21.15.16:60002) 2024-09-19T07:48:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.304+0000 7f14c8c81640 1 -- 172.21.15.16:0/1825082732 >> v1:172.21.15.146:6789/0 conn(0x7f14c40d0070 legacy=0x7f14c40d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.304+0000 7f14c8c81640 1 -- 172.21.15.16:0/1825082732 >> v1:172.21.15.16:6789/0 conn(0x7f14c415f0d0 legacy=0x7f14c40d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.304+0000 7f14c8c81640 1 -- 172.21.15.16:0/1825082732 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c40cf870 con 0x7f14c40d4840 2024-09-19T07:48:44.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.304+0000 7f14c2ffd640 1 -- 172.21.15.16:0/1825082732 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 741426756 0 0) 0x7f14c40d1b10 con 0x7f14c415f0d0 2024-09-19T07:48:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.304+0000 7f14c8c81640 1 --2- 172.21.15.16:0/1825082732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c40d4840 0x7f14c415f4b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009dc0 tx=0x7f14bc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cf15c1e7fee506 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.304+0000 7f14c2ffd640 1 -- 172.21.15.16:0/1825082732 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc007740 con 0x7f14c40d4840 2024-09-19T07:48:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.305+0000 7f14c2ffd640 1 -- 172.21.15.16:0/1825082732 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14bc0078e0 con 0x7f14c40d4840 2024-09-19T07:48:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.305+0000 7f14c2ffd640 1 -- 172.21.15.16:0/1825082732 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc007cb0 con 0x7f14c40d4840 2024-09-19T07:48:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.305+0000 7f14ca6e4640 1 -- 172.21.15.16:0/1825082732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c40d4840 msgr2=0x7f14c415f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.305+0000 7f14ca6e4640 1 --2- 172.21.15.16:0/1825082732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c40d4840 0x7f14c415f4b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009dc0 tx=0x7f14bc02a6a0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.305+0000 7f14ca6e4640 1 -- 172.21.15.16:0/1825082732 shutdown_connections 2024-09-19T07:48:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.305+0000 7f14ca6e4640 1 --2- 172.21.15.16:0/1825082732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c40d4840 0x7f14c415f4b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.305+0000 7f14ca6e4640 1 -- 172.21.15.16:0/1825082732 >> 172.21.15.16:0/1825082732 conn(0x7f14c409d760 msgr2=0x7f14c408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.305+0000 7f14ca6e4640 1 -- 172.21.15.16:0/1825082732 shutdown_connections 2024-09-19T07:48:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.305+0000 7f14ca6e4640 1 -- 172.21.15.16:0/1825082732 wait complete. 2024-09-19T07:48:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.306+0000 7f14ca6e4640 1 Processor -- start 2024-09-19T07:48:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.306+0000 7f14ca6e4640 1 -- start start 2024-09-19T07:48:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.307+0000 7f14ca6e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c40d0070 0x7f14c4151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.307+0000 7f14ca6e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14c40d4840 0x7f14c40cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.307+0000 7f14ca6e4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14c415f0d0 0x7f14c41597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.307+0000 7f14ca6e4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f14c40b2f70 con 0x7f14c40d0070 2024-09-19T07:48:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.307+0000 7f14ca6e4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f14c40cef60 con 0x7f14c40d4840 2024-09-19T07:48:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.307+0000 7f14ca6e4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f14c40b30f0 con 0x7f14c415f0d0 2024-09-19T07:48:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.307+0000 7f14c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c40d0070 0x7f14c4151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.307+0000 7f14c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c40d0070 0x7f14c4151400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60006/0 (socket says 172.21.15.16:60006) 2024-09-19T07:48:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.307+0000 7f14c3fff640 1 -- 172.21.15.16:0/237855153 learned_addr learned my addr 172.21.15.16:0/237855153 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.308+0000 7f14c37fe640 1 --2- 172.21.15.16:0/237855153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14c40d4840 0x7f14c40cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.308+0000 7f14c8c81640 1 --2- 172.21.15.16:0/237855153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14c415f0d0 0x7f14c41597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.308+0000 7f14c3fff640 1 -- 172.21.15.16:0/237855153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14c415f0d0 msgr2=0x7f14c41597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.308+0000 7f14c3fff640 1 --2- 172.21.15.16:0/237855153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14c415f0d0 0x7f14c41597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.308+0000 7f14c3fff640 1 -- 172.21.15.16:0/237855153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14c40d4840 msgr2=0x7f14c40cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.308+0000 7f14c3fff640 1 --2- 172.21.15.16:0/237855153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14c40d4840 0x7f14c40cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.308+0000 7f14c3fff640 1 -- 172.21.15.16:0/237855153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c40ccff0 con 0x7f14c40d0070 2024-09-19T07:48:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.308+0000 7f14c3fff640 1 --2- 172.21.15.16:0/237855153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c40d0070 0x7f14c4151400 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f14b400c420 tx=0x7f14b400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.308+0000 7f14c0ff9640 1 -- 172.21.15.16:0/237855153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b40074e0 con 0x7f14c40d0070 2024-09-19T07:48:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.308+0000 7f14c0ff9640 1 -- 172.21.15.16:0/237855153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b40047e0 con 0x7f14c40d0070 2024-09-19T07:48:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.308+0000 7f14c0ff9640 1 -- 172.21.15.16:0/237855153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4004bd0 con 0x7f14c40d0070 2024-09-19T07:48:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.308+0000 7f14ca6e4640 1 -- 172.21.15.16:0/237855153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c40cd2d0 con 0x7f14c40d0070 2024-09-19T07:48:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.308+0000 7f14ca6e4640 1 -- 172.21.15.16:0/237855153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c40cd890 con 0x7f14c40d0070 2024-09-19T07:48:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.309+0000 7f14ca6e4640 1 -- 172.21.15.16:0/237855153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c406cf30 con 0x7f14c40d0070 2024-09-19T07:48:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.310+0000 7f14c0ff9640 1 -- 172.21.15.16:0/237855153 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f14b4011070 con 0x7f14c40d0070 2024-09-19T07:48:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.310+0000 7f14c0ff9640 1 --2- 172.21.15.16:0/237855153 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f14b006c2b0 0x7f14b006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.310+0000 7f14c0ff9640 1 -- 172.21.15.16:0/237855153 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5378+0+0 (secure 0 0 0) 0x7f14b408af20 con 0x7f14c40d0070 2024-09-19T07:48:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.310+0000 7f14c37fe640 1 --2- 172.21.15.16:0/237855153 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f14b006c2b0 0x7f14b006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.311+0000 7f14c37fe640 1 --2- 172.21.15.16:0/237855153 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f14b006c2b0 0x7f14b006e770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f14c4151940 tx=0x7f14b8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.314+0000 7f14c0ff9640 1 -- 172.21.15.16:0/237855153 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f14b400e280 con 0x7f14c40d0070 2024-09-19T07:48:44.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.558+0000 7f14ca6e4640 1 -- 172.21.15.16:0/237855153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]} v 0) -- 0x7f14c40d18e0 con 0x7f14c40d0070 2024-09-19T07:48:44.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.562+0000 7f14c0ff9640 1 -- 172.21.15.16:0/237855153 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]}]=0 v247) ==== 153+0+0 (secure 0 0 0) 0x7f14b40597d0 con 0x7f14c40d0070 2024-09-19T07:48:44.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.565+0000 7f14ca6e4640 1 -- 172.21.15.16:0/237855153 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f14b006c2b0 msgr2=0x7f14b006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.565+0000 7f14ca6e4640 1 --2- 172.21.15.16:0/237855153 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f14b006c2b0 0x7f14b006e770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f14c4151940 tx=0x7f14b8009040 comp rx=0 tx=0).stop 2024-09-19T07:48:44.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.565+0000 7f14ca6e4640 1 -- 172.21.15.16:0/237855153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c40d0070 msgr2=0x7f14c4151400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.565+0000 7f14ca6e4640 1 --2- 172.21.15.16:0/237855153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c40d0070 0x7f14c4151400 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f14b400c420 tx=0x7f14b400c8e0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.565+0000 7f14ca6e4640 1 -- 172.21.15.16:0/237855153 shutdown_connections 2024-09-19T07:48:44.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.565+0000 7f14ca6e4640 1 --2- 172.21.15.16:0/237855153 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f14b006c2b0 0x7f14b006e770 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.565+0000 7f14ca6e4640 1 --2- 172.21.15.16:0/237855153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14c415f0d0 0x7f14c41597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.565+0000 7f14ca6e4640 1 --2- 172.21.15.16:0/237855153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14c40d4840 0x7f14c40cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.565+0000 7f14ca6e4640 1 --2- 172.21.15.16:0/237855153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c40d0070 0x7f14c4151400 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.565+0000 7f14ca6e4640 1 -- 172.21.15.16:0/237855153 >> 172.21.15.16:0/237855153 conn(0x7f14c409d760 msgr2=0x7f14c415b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:44.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.565+0000 7f14ca6e4640 1 -- 172.21.15.16:0/237855153 shutdown_connections 2024-09-19T07:48:44.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.565+0000 7f14ca6e4640 1 -- 172.21.15.16:0/237855153 wait complete. 2024-09-19T07:48:44.576 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T07:48:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.680+0000 7fb2f339f640 1 Processor -- start 2024-09-19T07:48:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.681+0000 7fb2f339f640 1 -- start start 2024-09-19T07:48:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.681+0000 7fb2f339f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb2ec071350 0x7fb2ec15c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.681+0000 7fb2f339f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2ec072fe0 con 0x7fb2ec06f6e0 2024-09-19T07:48:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f339f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2ec0732c0 con 0x7fb2ec0cecd0 2024-09-19T07:48:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f339f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb2ec073bf0 con 0x7fb2ec071350 2024-09-19T07:48:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f2b9e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb2ec06f6e0 0x7fb2ec15cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38534/0 (socket says 172.21.15.16:38534) 2024-09-19T07:48:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f2b9e640 1 -- 172.21.15.16:0/2171397135 learned_addr learned my addr 172.21.15.16:0/2171397135 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f1b9c640 1 --2- 172.21.15.16:0/2171397135 >> v2:172.21.15.146:3300/0 conn(0x7fb2ec071350 0x7fb2ec15c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f139b640 1 -- 172.21.15.16:0/2171397135 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1017798980 0 0) 0x7fb2ec072fe0 con 0x7fb2ec06f6e0 2024-09-19T07:48:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f139b640 1 -- 172.21.15.16:0/2171397135 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2d8003520 con 0x7fb2ec06f6e0 2024-09-19T07:48:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f139b640 1 -- 172.21.15.16:0/2171397135 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3860565198 0 0) 0x7fb2d8003520 con 0x7fb2ec06f6e0 2024-09-19T07:48:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f139b640 1 -- 172.21.15.16:0/2171397135 >> v2:172.21.15.146:3300/0 conn(0x7fb2ec071350 msgr2=0x7fb2ec15c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f139b640 1 --2- 172.21.15.16:0/2171397135 >> v2:172.21.15.146:3300/0 conn(0x7fb2ec071350 0x7fb2ec15c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f139b640 1 -- 172.21.15.16:0/2171397135 >> v1:172.21.15.146:6789/0 conn(0x7fb2ec0cecd0 legacy=0x7fb2ec0cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f139b640 1 -- 172.21.15.16:0/2171397135 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2ec0735a0 con 0x7fb2ec06f6e0 2024-09-19T07:48:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f139b640 1 -- 172.21.15.16:0/2171397135 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fb2e40019f0 con 0x7fb2ec06f6e0 2024-09-19T07:48:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f139b640 1 -- 172.21.15.16:0/2171397135 >> v1:172.21.15.16:6789/0 conn(0x7fb2ec06f6e0 legacy=0x7fb2ec15cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f139b640 1 --2- 172.21.15.16:0/2171397135 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2d80039b0 0x7fb2d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f139b640 1 --2- 172.21.15.16:0/2171397135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d8004800 0x7fb2d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f139b640 1 --2- 172.21.15.16:0/2171397135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2d80051c0 0x7fb2d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f139b640 1 -- 172.21.15.16:0/2171397135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec073bf0 con 0x7fb2d8004800 2024-09-19T07:48:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f139b640 1 -- 172.21.15.16:0/2171397135 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d800c160 con 0x7fb2d80051c0 2024-09-19T07:48:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f139b640 1 -- 172.21.15.16:0/2171397135 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d800c360 con 0x7fb2d80039b0 2024-09-19T07:48:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f1b9c640 1 --2- 172.21.15.16:0/2171397135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d8004800 0x7fb2d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.682+0000 7fb2f239d640 1 --2- 172.21.15.16:0/2171397135 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2d80039b0 0x7fb2d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.683+0000 7fb2f2b9e640 1 --2- 172.21.15.16:0/2171397135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2d80051c0 0x7fb2d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.683+0000 7fb2f1b9c640 1 -- 172.21.15.16:0/2171397135 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2d80039b0 msgr2=0x7fb2d8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.683+0000 7fb2f1b9c640 1 --2- 172.21.15.16:0/2171397135 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2d80039b0 0x7fb2d8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.683+0000 7fb2f1b9c640 1 -- 172.21.15.16:0/2171397135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2d80051c0 msgr2=0x7fb2d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.683+0000 7fb2f1b9c640 1 --2- 172.21.15.16:0/2171397135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2d80051c0 0x7fb2d800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.683+0000 7fb2f1b9c640 1 -- 172.21.15.16:0/2171397135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d8003520 con 0x7fb2d8004800 2024-09-19T07:48:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.683+0000 7fb2f2b9e640 1 -- 172.21.15.16:0/2171397135 reap_dead start 2024-09-19T07:48:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.683+0000 7fb2f1b9c640 1 --2- 172.21.15.16:0/2171397135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d8004800 0x7fb2d8004c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009ac0 tx=0x7fb2d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd37b63e34607f71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.684+0000 7fb2f139b640 1 -- 172.21.15.16:0/2171397135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc036070 con 0x7fb2d8004800 2024-09-19T07:48:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.684+0000 7fb2f139b640 1 -- 172.21.15.16:0/2171397135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2dc031400 con 0x7fb2d8004800 2024-09-19T07:48:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.684+0000 7fb2f139b640 1 -- 172.21.15.16:0/2171397135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc009690 con 0x7fb2d8004800 2024-09-19T07:48:44.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.684+0000 7fb2f339f640 1 -- 172.21.15.16:0/2171397135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d8004800 msgr2=0x7fb2d8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.684+0000 7fb2f339f640 1 --2- 172.21.15.16:0/2171397135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d8004800 0x7fb2d8004c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009ac0 tx=0x7fb2d8005830 comp rx=0 tx=0).stop 2024-09-19T07:48:44.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.684+0000 7fb2f339f640 1 -- 172.21.15.16:0/2171397135 shutdown_connections 2024-09-19T07:48:44.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.684+0000 7fb2f339f640 1 --2- 172.21.15.16:0/2171397135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d8004800 0x7fb2d8004c80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.684+0000 7fb2f339f640 1 -- 172.21.15.16:0/2171397135 >> 172.21.15.16:0/2171397135 conn(0x7fb2ec06ebf0 msgr2=0x7fb2ec06efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.684+0000 7fb2f339f640 1 -- 172.21.15.16:0/2171397135 shutdown_connections 2024-09-19T07:48:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.684+0000 7fb2f339f640 1 -- 172.21.15.16:0/2171397135 wait complete. 2024-09-19T07:48:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.685+0000 7fb2f339f640 1 Processor -- start 2024-09-19T07:48:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.686+0000 7fb2f339f640 1 -- start start 2024-09-19T07:48:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.686+0000 7fb2f339f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2d8004800 0x7fb2ec0cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.686+0000 7fb2f339f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2ec0cccb0 0x7fb2ec1375e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.686+0000 7fb2f339f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2ec144970 0x7fb2ec0ab740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.686+0000 7fb2f339f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec145a50 con 0x7fb2ec144970 2024-09-19T07:48:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.686+0000 7fb2f339f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec072c70 con 0x7fb2d8004800 2024-09-19T07:48:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.686+0000 7fb2f339f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb2ec145bd0 con 0x7fb2ec0cccb0 2024-09-19T07:48:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.686+0000 7fb2f2b9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2ec144970 0x7fb2ec0ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.686+0000 7fb2f2b9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2ec144970 0x7fb2ec0ab740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60034/0 (socket says 172.21.15.16:60034) 2024-09-19T07:48:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.686+0000 7fb2f2b9e640 1 -- 172.21.15.16:0/582859550 learned_addr learned my addr 172.21.15.16:0/582859550 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.687+0000 7fb2f239d640 1 --2- 172.21.15.16:0/582859550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2d8004800 0x7fb2ec0cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.687+0000 7fb2f1b9c640 1 --2- 172.21.15.16:0/582859550 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2ec0cccb0 0x7fb2ec1375e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.687+0000 7fb2f2b9e640 1 -- 172.21.15.16:0/582859550 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2ec0cccb0 msgr2=0x7fb2ec1375e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.687+0000 7fb2f2b9e640 1 --2- 172.21.15.16:0/582859550 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2ec0cccb0 0x7fb2ec1375e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.687+0000 7fb2f2b9e640 1 -- 172.21.15.16:0/582859550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2d8004800 msgr2=0x7fb2ec0cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.687+0000 7fb2f2b9e640 1 --2- 172.21.15.16:0/582859550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2d8004800 0x7fb2ec0cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.687+0000 7fb2f2b9e640 1 -- 172.21.15.16:0/582859550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2ec0ca950 con 0x7fb2ec144970 2024-09-19T07:48:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.687+0000 7fb2f2b9e640 1 --2- 172.21.15.16:0/582859550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2ec144970 0x7fb2ec0ab740 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec072c00 tx=0x7fb2e4007bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.687+0000 7fb2e2ffd640 1 -- 172.21.15.16:0/582859550 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e400ab70 con 0x7fb2ec144970 2024-09-19T07:48:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.687+0000 7fb2e2ffd640 1 -- 172.21.15.16:0/582859550 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2e4018070 con 0x7fb2ec144970 2024-09-19T07:48:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.687+0000 7fb2f339f640 1 -- 172.21.15.16:0/582859550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2ec0aac40 con 0x7fb2ec144970 2024-09-19T07:48:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.687+0000 7fb2e2ffd640 1 -- 172.21.15.16:0/582859550 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40133f0 con 0x7fb2ec144970 2024-09-19T07:48:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.687+0000 7fb2f339f640 1 -- 172.21.15.16:0/582859550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2ec0caed0 con 0x7fb2ec144970 2024-09-19T07:48:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.688+0000 7fb2f339f640 1 -- 172.21.15.16:0/582859550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2ec070f30 con 0x7fb2ec144970 2024-09-19T07:48:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.689+0000 7fb2e2ffd640 1 -- 172.21.15.16:0/582859550 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb2e4013590 con 0x7fb2ec144970 2024-09-19T07:48:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.689+0000 7fb2e2ffd640 1 --2- 172.21.15.16:0/582859550 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb2d006c2b0 0x7fb2d006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.689+0000 7fb2e2ffd640 1 -- 172.21.15.16:0/582859550 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5378+0+0 (secure 0 0 0) 0x7fb2e408b8b0 con 0x7fb2ec144970 2024-09-19T07:48:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.689+0000 7fb2f239d640 1 --2- 172.21.15.16:0/582859550 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb2d006c2b0 0x7fb2d006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.690+0000 7fb2f239d640 1 --2- 172.21.15.16:0/582859550 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb2d006c2b0 0x7fb2d006e770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8000c00 tx=0x7fb2e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.693+0000 7fb2e2ffd640 1 -- 172.21.15.16:0/582859550 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb2ec070f30 con 0x7fb2ec144970 2024-09-19T07:48:44.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.869+0000 7fb2f339f640 1 -- 172.21.15.16:0/582859550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb2ec072e80 con 0x7fb2ec144970 2024-09-19T07:48:44.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.872+0000 7fb2e2ffd640 1 -- 172.21.15.16:0/582859550 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v25)=0 v25) ==== 122+0+0 (secure 0 0 0) 0x7fb2e405a160 con 0x7fb2ec144970 2024-09-19T07:48:44.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.874+0000 7fb2f339f640 1 -- 172.21.15.16:0/582859550 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb2d006c2b0 msgr2=0x7fb2d006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.874+0000 7fb2f339f640 1 --2- 172.21.15.16:0/582859550 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb2d006c2b0 0x7fb2d006e770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8000c00 tx=0x7fb2e8009040 comp rx=0 tx=0).stop 2024-09-19T07:48:44.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.874+0000 7fb2f339f640 1 -- 172.21.15.16:0/582859550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2ec144970 msgr2=0x7fb2ec0ab740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:44.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.874+0000 7fb2f339f640 1 --2- 172.21.15.16:0/582859550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2ec144970 0x7fb2ec0ab740 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec072c00 tx=0x7fb2e4007bb0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.874+0000 7fb2f339f640 1 -- 172.21.15.16:0/582859550 shutdown_connections 2024-09-19T07:48:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.874+0000 7fb2f339f640 1 --2- 172.21.15.16:0/582859550 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb2d006c2b0 0x7fb2d006e770 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.874+0000 7fb2f339f640 1 --2- 172.21.15.16:0/582859550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2ec144970 0x7fb2ec0ab740 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.874+0000 7fb2f339f640 1 --2- 172.21.15.16:0/582859550 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2ec0cccb0 0x7fb2ec1375e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.874+0000 7fb2f339f640 1 --2- 172.21.15.16:0/582859550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2d8004800 0x7fb2ec0cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.874+0000 7fb2f339f640 1 -- 172.21.15.16:0/582859550 >> 172.21.15.16:0/582859550 conn(0x7fb2ec06ebf0 msgr2=0x7fb2ec1e4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.874+0000 7fb2f339f640 1 -- 172.21.15.16:0/582859550 shutdown_connections 2024-09-19T07:48:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:44.874+0000 7fb2f339f640 1 -- 172.21.15.16:0/582859550 wait complete. 2024-09-19T07:48:44.886 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:48:44.886 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:48:44.926 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:48:44.976 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:48:44.976 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:48:44.976 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:48:44.993 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:48:44.993 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:48:44.993 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:48:45.011 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:48:45.011 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:48:45.011 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:48:45.050 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:48:45.050 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:48:45.050 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:48:45.068 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:48:45.068 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T07:48:45.068 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:48:45.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.193+0000 7f11cb42e640 1 Processor -- start 2024-09-19T07:48:45.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.193+0000 7f11cb42e640 1 -- start start 2024-09-19T07:48:45.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.194+0000 7f11cb42e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f11c406f9e0 0x7f11c40d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:45.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.194+0000 7f11cb42e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11c40d09f0 0x7f11c40c5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:45.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.194+0000 7f11cb42e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11c415b500 con 0x7f11c415b120 2024-09-19T07:48:45.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.194+0000 7f11cb42e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f11c40d0e40 con 0x7f11c406f9e0 2024-09-19T07:48:45.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.194+0000 7f11cb42e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f11c406e940 con 0x7f11c40d09f0 2024-09-19T07:48:45.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.194+0000 7f11c99cb640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11c40d09f0 0x7f11c40c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:45.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.194+0000 7f11c99cb640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11c40d09f0 0x7f11c40c5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:58446/0 (socket says 172.21.15.16:58446) 2024-09-19T07:48:45.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.194+0000 7f11c99cb640 1 -- 172.21.15.16:0/2566891838 learned_addr learned my addr 172.21.15.16:0/2566891838 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:45.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.194+0000 7f11c89c9640 1 --2- 172.21.15.16:0/2566891838 >> v2:172.21.15.146:3300/0 conn(0x7f11c406f9e0 0x7f11c40d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:45.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.195+0000 7f11bbfff640 1 -- 172.21.15.16:0/2566891838 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1689013339 0 0) 0x7f11c415b500 con 0x7f11c415b120 2024-09-19T07:48:45.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.195+0000 7f11c99cb640 1 -- 172.21.15.16:0/2566891838 >> v2:172.21.15.146:3300/0 conn(0x7f11c406f9e0 msgr2=0x7f11c40d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:45.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.195+0000 7f11c99cb640 1 --2- 172.21.15.16:0/2566891838 >> v2:172.21.15.146:3300/0 conn(0x7f11c406f9e0 0x7f11c40d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.195+0000 7f11c99cb640 1 -- 172.21.15.16:0/2566891838 >> v1:172.21.15.146:6789/0 conn(0x7f11c415b120 legacy=0x7f11c406dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:45.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.195+0000 7f11c99cb640 1 -- 172.21.15.16:0/2566891838 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c406ec20 con 0x7f11c40d09f0 2024-09-19T07:48:45.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.195+0000 7f11c99cb640 1 --2- 172.21.15.16:0/2566891838 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11c40d09f0 0x7f11c40c5160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f11bc009e50 tx=0x7f11bc02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1c268203d7d2c5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:45.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.196+0000 7f11bbfff640 1 -- 172.21.15.16:0/2566891838 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc036070 con 0x7f11c40d09f0 2024-09-19T07:48:45.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.196+0000 7f11bbfff640 1 -- 172.21.15.16:0/2566891838 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11bc031600 con 0x7f11c40d09f0 2024-09-19T07:48:45.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.196+0000 7f11bbfff640 1 -- 172.21.15.16:0/2566891838 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc0319d0 con 0x7f11c40d09f0 2024-09-19T07:48:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.196+0000 7f11cb42e640 1 -- 172.21.15.16:0/2566891838 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11c40d09f0 msgr2=0x7f11c40c5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.196+0000 7f11cb42e640 1 --2- 172.21.15.16:0/2566891838 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11c40d09f0 0x7f11c40c5160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f11bc009e50 tx=0x7f11bc02a710 comp rx=0 tx=0).stop 2024-09-19T07:48:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.196+0000 7f11cb42e640 1 -- 172.21.15.16:0/2566891838 shutdown_connections 2024-09-19T07:48:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.196+0000 7f11cb42e640 1 --2- 172.21.15.16:0/2566891838 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11c40d09f0 0x7f11c40c5160 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.196+0000 7f11cb42e640 1 --2- 172.21.15.16:0/2566891838 >> v2:172.21.15.146:3300/0 conn(0x7f11c406f9e0 0x7f11c40d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.196+0000 7f11cb42e640 1 -- 172.21.15.16:0/2566891838 >> 172.21.15.16:0/2566891838 conn(0x7f11c406bdf0 msgr2=0x7f11c406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.196+0000 7f11cb42e640 1 -- 172.21.15.16:0/2566891838 shutdown_connections 2024-09-19T07:48:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.196+0000 7f11cb42e640 1 -- 172.21.15.16:0/2566891838 wait complete. 2024-09-19T07:48:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.197+0000 7f11cb42e640 1 Processor -- start 2024-09-19T07:48:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.198+0000 7f11cb42e640 1 -- start start 2024-09-19T07:48:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.198+0000 7f11cb42e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11c406f9e0 0x7f11c415b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.198+0000 7f11cb42e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c40d09f0 0x7f11c40ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.198+0000 7f11cb42e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11c415b120 0x7f11c40c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.198+0000 7f11cb42e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11c406f4c0 con 0x7f11c40d09f0 2024-09-19T07:48:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.198+0000 7f11cb42e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f11c406f1e0 con 0x7f11c415b120 2024-09-19T07:48:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.198+0000 7f11cb42e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f11c406e5d0 con 0x7f11c406f9e0 2024-09-19T07:48:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.198+0000 7f11c89c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c40d09f0 0x7f11c40ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.198+0000 7f11c89c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c40d09f0 0x7f11c40ae420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60044/0 (socket says 172.21.15.16:60044) 2024-09-19T07:48:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.198+0000 7f11c89c9640 1 -- 172.21.15.16:0/3020791818 learned_addr learned my addr 172.21.15.16:0/3020791818 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.198+0000 7f11c91ca640 1 --2- 172.21.15.16:0/3020791818 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11c406f9e0 0x7f11c415b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.198+0000 7f11c99cb640 1 --2- 172.21.15.16:0/3020791818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11c415b120 0x7f11c40c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.199+0000 7f11c89c9640 1 -- 172.21.15.16:0/3020791818 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11c406f9e0 msgr2=0x7f11c415b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.199+0000 7f11c89c9640 1 --2- 172.21.15.16:0/3020791818 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11c406f9e0 0x7f11c415b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.199+0000 7f11c89c9640 1 -- 172.21.15.16:0/3020791818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11c415b120 msgr2=0x7f11c40c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.199+0000 7f11c89c9640 1 --2- 172.21.15.16:0/3020791818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11c415b120 0x7f11c40c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.199+0000 7f11c89c9640 1 -- 172.21.15.16:0/3020791818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c40afef0 con 0x7f11c40d09f0 2024-09-19T07:48:45.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.199+0000 7f11c99cb640 1 --2- 172.21.15.16:0/3020791818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11c415b120 0x7f11c40c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:48:45.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.199+0000 7f11c91ca640 1 --2- 172.21.15.16:0/3020791818 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11c406f9e0 0x7f11c415b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:48:45.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.199+0000 7f11c89c9640 1 --2- 172.21.15.16:0/3020791818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c40d09f0 0x7f11c40ae420 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f11b400e9b0 tx=0x7f11b400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:45.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.199+0000 7f11b9ffb640 1 -- 172.21.15.16:0/3020791818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4017070 con 0x7f11c40d09f0 2024-09-19T07:48:45.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.199+0000 7f11b9ffb640 1 -- 172.21.15.16:0/3020791818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b400c9a0 con 0x7f11c40d09f0 2024-09-19T07:48:45.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.199+0000 7f11b9ffb640 1 -- 172.21.15.16:0/3020791818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b400cd90 con 0x7f11c40d09f0 2024-09-19T07:48:45.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.199+0000 7f11cb42e640 1 -- 172.21.15.16:0/3020791818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c40b01d0 con 0x7f11c40d09f0 2024-09-19T07:48:45.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.199+0000 7f11cb42e640 1 -- 172.21.15.16:0/3020791818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c40b0790 con 0x7f11c40d09f0 2024-09-19T07:48:45.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.200+0000 7f11cb42e640 1 -- 172.21.15.16:0/3020791818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11c40d0500 con 0x7f11c40d09f0 2024-09-19T07:48:45.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.201+0000 7f11b9ffb640 1 -- 172.21.15.16:0/3020791818 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f11b401a070 con 0x7f11c40d09f0 2024-09-19T07:48:45.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.201+0000 7f11b9ffb640 1 --2- 172.21.15.16:0/3020791818 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f11a806c2b0 0x7f11a806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:45.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.201+0000 7f11b9ffb640 1 -- 172.21.15.16:0/3020791818 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5378+0+0 (secure 0 0 0) 0x7f11b408b5b0 con 0x7f11c40d09f0 2024-09-19T07:48:45.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.202+0000 7f11c91ca640 1 --2- 172.21.15.16:0/3020791818 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f11a806c2b0 0x7f11a806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:45.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.202+0000 7f11c91ca640 1 --2- 172.21.15.16:0/3020791818 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f11a806c2b0 0x7f11a806e770 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f11c0000f60 tx=0x7f11c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:45.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.206+0000 7f11b9ffb640 1 -- 172.21.15.16:0/3020791818 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f11c40d0500 con 0x7f11c40d09f0 2024-09-19T07:48:45.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.416+0000 7f11cb42e640 1 -- 172.21.15.16:0/3020791818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11c406c420 con 0x7f11c40d09f0 2024-09-19T07:48:45.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.417+0000 7f11b9ffb640 1 -- 172.21.15.16:0/3020791818 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+3914 (secure 0 0 0) 0x7f11b40598a0 con 0x7f11c40d09f0 2024-09-19T07:48:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:45.417 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-19T07:48:39:547476+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/3536366465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3536366465},{"type":"v1","addr":"172.21.15.146:6836","nonce":3536366465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3612374586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3612374586},{"type":"v1","addr":"172.21.15.146:6837","nonce":3612374586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5337,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/781070730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":781070730},{"type":"v1","addr":"172.21.15.16:6833","nonce":781070730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1223772520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1223772520},{"type":"v1","addr":"172.21.15.16:6835","nonce":1223772520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[]} 2024-09-19T07:48:45.418 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-19T07:48:45.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.419+0000 7f11cb42e640 1 -- 172.21.15.16:0/3020791818 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f11a806c2b0 msgr2=0x7f11a806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.419+0000 7f11cb42e640 1 --2- 172.21.15.16:0/3020791818 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f11a806c2b0 0x7f11a806e770 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f11c0000f60 tx=0x7f11c0009040 comp rx=0 tx=0).stop 2024-09-19T07:48:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.419+0000 7f11cb42e640 1 -- 172.21.15.16:0/3020791818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c40d09f0 msgr2=0x7f11c40ae420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.419+0000 7f11cb42e640 1 --2- 172.21.15.16:0/3020791818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c40d09f0 0x7f11c40ae420 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f11b400e9b0 tx=0x7f11b400ee70 comp rx=0 tx=0).stop 2024-09-19T07:48:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.419+0000 7f11cb42e640 1 -- 172.21.15.16:0/3020791818 shutdown_connections 2024-09-19T07:48:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.419+0000 7f11cb42e640 1 --2- 172.21.15.16:0/3020791818 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f11a806c2b0 0x7f11a806e770 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.419+0000 7f11cb42e640 1 --2- 172.21.15.16:0/3020791818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11c415b120 0x7f11c40c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.419+0000 7f11cb42e640 1 --2- 172.21.15.16:0/3020791818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c40d09f0 0x7f11c40ae420 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.419+0000 7f11cb42e640 1 --2- 172.21.15.16:0/3020791818 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11c406f9e0 0x7f11c415b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.419+0000 7f11cb42e640 1 -- 172.21.15.16:0/3020791818 >> 172.21.15.16:0/3020791818 conn(0x7f11c406bdf0 msgr2=0x7f11c40cc030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.419+0000 7f11cb42e640 1 -- 172.21.15.16:0/3020791818 shutdown_connections 2024-09-19T07:48:45.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.419+0000 7f11cb42e640 1 -- 172.21.15.16:0/3020791818 wait complete. 2024-09-19T07:48:45.431 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T07:48:45.432 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T07:48:45.432 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T07:48:45.432 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:48:45.432 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T07:48:45.432 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T07:48:45.432 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T07:48:45.432 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:48:45.433 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T07:48:45.433 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T07:48:45.433 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T07:48:45.433 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:48:45.433 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T07:48:45.433 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T07:48:45.433 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T07:48:45.434 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:48:45.441 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T07:48:45.442 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T07:48:45.442 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T07:48:45.442 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T07:48:45.444 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T07:48:45.445 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T07:48:45.445 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T07:48:45.445 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T07:48:45.484 INFO:tasks.ceph.mds.a:Started 2024-09-19T07:48:45.485 DEBUG:teuthology.parallel:result is None 2024-09-19T07:48:45.485 INFO:tasks.ceph.mds.c:Started 2024-09-19T07:48:45.485 DEBUG:teuthology.parallel:result is None 2024-09-19T07:48:45.487 INFO:tasks.ceph.mds.b:Started 2024-09-19T07:48:45.488 DEBUG:teuthology.parallel:result is None 2024-09-19T07:48:45.488 INFO:tasks.ceph.mds.d:Started 2024-09-19T07:48:45.489 DEBUG:teuthology.parallel:result is None 2024-09-19T07:48:45.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T07:48:45.564 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-19T07:48:45.565 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-19T07:48:45.570 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2024-09-19T07:48:45.570 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2024-09-19T07:48:45.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.596+0000 7f58188e4640 1 Processor -- start 2024-09-19T07:48:45.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.596+0000 7f58188e4640 1 -- start start 2024-09-19T07:48:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.597+0000 7f58188e4640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5814072250 0x7f58140cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.597+0000 7f58188e4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58140726c0 0x7f58140d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.597+0000 7f58188e4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58140d2b30 con 0x7f58140d2750 2024-09-19T07:48:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.597+0000 7f58188e4640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5814145bd0 con 0x7f5814072250 2024-09-19T07:48:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.597+0000 7f58188e4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f58141427e0 con 0x7f58140726c0 2024-09-19T07:48:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.598+0000 7f581259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f58140d2750 0x7f581413f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38564/0 (socket says 172.21.15.16:38564) 2024-09-19T07:48:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.598+0000 7f581259c640 1 -- 172.21.15.16:0/1818523053 learned_addr learned my addr 172.21.15.16:0/1818523053 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.598+0000 7f5812d9d640 1 --2- 172.21.15.16:0/1818523053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58140726c0 0x7f58140d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.598+0000 7f5811d9b640 1 --2- 172.21.15.16:0/1818523053 >> v2:172.21.15.146:3300/0 conn(0x7f5814072250 0x7f58140cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.598+0000 7f5812d9d640 1 -- 172.21.15.16:0/1818523053 >> v2:172.21.15.146:3300/0 conn(0x7f5814072250 msgr2=0x7f58140cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.598+0000 7f581159a640 1 -- 172.21.15.16:0/1818523053 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4225292155 0 0) 0x7f58140d2b30 con 0x7f58140d2750 2024-09-19T07:48:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.598+0000 7f5812d9d640 1 --2- 172.21.15.16:0/1818523053 >> v2:172.21.15.146:3300/0 conn(0x7f5814072250 0x7f58140cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.598+0000 7f5812d9d640 1 -- 172.21.15.16:0/1818523053 >> v1:172.21.15.16:6789/0 conn(0x7f58140d2750 legacy=0x7f581413f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.598+0000 7f5812d9d640 1 -- 172.21.15.16:0/1818523053 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5814141c60 con 0x7f58140726c0 2024-09-19T07:48:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.599+0000 7f5812d9d640 1 --2- 172.21.15.16:0/1818523053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58140726c0 0x7f58140d0300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f580c009d90 tx=0x7f580c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce76ece6899dd266 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.599+0000 7f581159a640 1 -- 172.21.15.16:0/1818523053 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c031600 con 0x7f58140726c0 2024-09-19T07:48:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.599+0000 7f581159a640 1 -- 172.21.15.16:0/1818523053 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f580c0317a0 con 0x7f58140726c0 2024-09-19T07:48:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.599+0000 7f581159a640 1 -- 172.21.15.16:0/1818523053 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c031b70 con 0x7f58140726c0 2024-09-19T07:48:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.599+0000 7f58188e4640 1 -- 172.21.15.16:0/1818523053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58140726c0 msgr2=0x7f58140d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.599+0000 7f58188e4640 1 --2- 172.21.15.16:0/1818523053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58140726c0 0x7f58140d0300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f580c009d90 tx=0x7f580c02a5d0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.599+0000 7f58188e4640 1 -- 172.21.15.16:0/1818523053 shutdown_connections 2024-09-19T07:48:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.599+0000 7f58188e4640 1 --2- 172.21.15.16:0/1818523053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58140726c0 0x7f58140d0300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.599+0000 7f58188e4640 1 --2- 172.21.15.16:0/1818523053 >> v2:172.21.15.146:3300/0 conn(0x7f5814072250 0x7f58140cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.600+0000 7f58188e4640 1 -- 172.21.15.16:0/1818523053 >> 172.21.15.16:0/1818523053 conn(0x7f58140b4040 msgr2=0x7f58140b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.600+0000 7f58188e4640 1 -- 172.21.15.16:0/1818523053 shutdown_connections 2024-09-19T07:48:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.600+0000 7f58188e4640 1 -- 172.21.15.16:0/1818523053 wait complete. 2024-09-19T07:48:45.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.601+0000 7f58188e4640 1 Processor -- start 2024-09-19T07:48:45.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.601+0000 7f58188e4640 1 -- start start 2024-09-19T07:48:45.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.602+0000 7f58188e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5814072250 0x7f581413ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.602+0000 7f58188e4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58140726c0 0x7f58140b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.602+0000 7f58188e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58140d2750 0x7f58141ef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.602+0000 7f58188e4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5814142500 con 0x7f58140d2750 2024-09-19T07:48:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.602+0000 7f58188e4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5814142220 con 0x7f5814072250 2024-09-19T07:48:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.602+0000 7f58188e4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f58141418b0 con 0x7f58140726c0 2024-09-19T07:48:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.602+0000 7f5812d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58140d2750 0x7f58141ef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.602+0000 7f5812d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58140d2750 0x7f58141ef8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60082/0 (socket says 172.21.15.16:60082) 2024-09-19T07:48:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.602+0000 7f5812d9d640 1 -- 172.21.15.16:0/1771904369 learned_addr learned my addr 172.21.15.16:0/1771904369 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.602+0000 7f581259c640 1 --2- 172.21.15.16:0/1771904369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5814072250 0x7f581413ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.602+0000 7f5811d9b640 1 --2- 172.21.15.16:0/1771904369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58140726c0 0x7f58140b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.602+0000 7f5812d9d640 1 -- 172.21.15.16:0/1771904369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58140726c0 msgr2=0x7f58140b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.602+0000 7f5812d9d640 1 --2- 172.21.15.16:0/1771904369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58140726c0 0x7f58140b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.602+0000 7f5812d9d640 1 -- 172.21.15.16:0/1771904369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5814072250 msgr2=0x7f581413ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.602+0000 7f5812d9d640 1 --2- 172.21.15.16:0/1771904369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5814072250 0x7f581413ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.602+0000 7f5812d9d640 1 -- 172.21.15.16:0/1771904369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5814144590 con 0x7f58140d2750 2024-09-19T07:48:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.603+0000 7f5812d9d640 1 --2- 172.21.15.16:0/1771904369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58140d2750 0x7f58141ef8d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f580c009d90 tx=0x7f580c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.603+0000 7f5802ffd640 1 -- 172.21.15.16:0/1771904369 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c040070 con 0x7f58140d2750 2024-09-19T07:48:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.603+0000 7f5802ffd640 1 -- 172.21.15.16:0/1771904369 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f580c030be0 con 0x7f58140d2750 2024-09-19T07:48:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.603+0000 7f58188e4640 1 -- 172.21.15.16:0/1771904369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5814144810 con 0x7f58140d2750 2024-09-19T07:48:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.603+0000 7f5802ffd640 1 -- 172.21.15.16:0/1771904369 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c009450 con 0x7f58140d2750 2024-09-19T07:48:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.603+0000 7f58188e4640 1 -- 172.21.15.16:0/1771904369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58141f06a0 con 0x7f58140d2750 2024-09-19T07:48:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.604+0000 7f58188e4640 1 -- 172.21.15.16:0/1771904369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58140d1dc0 con 0x7f58140d2750 2024-09-19T07:48:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.604+0000 7f5802ffd640 1 -- 172.21.15.16:0/1771904369 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f580c039040 con 0x7f58140d2750 2024-09-19T07:48:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.605+0000 7f5802ffd640 1 --2- 172.21.15.16:0/1771904369 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f57f406c2e0 0x7f57f406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.605+0000 7f5802ffd640 1 -- 172.21.15.16:0/1771904369 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5378+0+0 (secure 0 0 0) 0x7f580c0abea0 con 0x7f58140d2750 2024-09-19T07:48:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.605+0000 7f581259c640 1 --2- 172.21.15.16:0/1771904369 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f57f406c2e0 0x7f57f406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.605+0000 7f581259c640 1 --2- 172.21.15.16:0/1771904369 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f57f406c2e0 0x7f57f406e7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5804007fd0 tx=0x7f5804007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.609+0000 7f5802ffd640 1 -- 172.21.15.16:0/1771904369 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5814144590 con 0x7f58140d2750 2024-09-19T07:48:45.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:45.799+0000 7f58188e4640 1 -- 172.21.15.16:0/1771904369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f581406ef30 con 0x7f58140d2750 2024-09-19T07:48:46.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.574+0000 7f5802ffd640 1 -- 172.21.15.16:0/1771904369 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v64) ==== 97+0+0 (secure 0 0 0) 0x7f580c07a110 con 0x7f58140d2750 2024-09-19T07:48:46.574 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-19T07:48:46.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.578+0000 7f58188e4640 1 -- 172.21.15.16:0/1771904369 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f57f406c2e0 msgr2=0x7f57f406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:46.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.578+0000 7f58188e4640 1 --2- 172.21.15.16:0/1771904369 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f57f406c2e0 0x7f57f406e7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5804007fd0 tx=0x7f5804007de0 comp rx=0 tx=0).stop 2024-09-19T07:48:46.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.578+0000 7f58188e4640 1 -- 172.21.15.16:0/1771904369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58140d2750 msgr2=0x7f58141ef8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:46.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.578+0000 7f58188e4640 1 --2- 172.21.15.16:0/1771904369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58140d2750 0x7f58141ef8d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f580c009d90 tx=0x7f580c004a40 comp rx=0 tx=0).stop 2024-09-19T07:48:46.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.578+0000 7f58188e4640 1 -- 172.21.15.16:0/1771904369 shutdown_connections 2024-09-19T07:48:46.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.578+0000 7f58188e4640 1 --2- 172.21.15.16:0/1771904369 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f57f406c2e0 0x7f57f406e7a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:46.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.578+0000 7f58188e4640 1 --2- 172.21.15.16:0/1771904369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58140d2750 0x7f58141ef8d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:46.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.578+0000 7f58188e4640 1 --2- 172.21.15.16:0/1771904369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58140726c0 0x7f58140b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:46.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.578+0000 7f58188e4640 1 --2- 172.21.15.16:0/1771904369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5814072250 0x7f581413ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:46.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.578+0000 7f58188e4640 1 -- 172.21.15.16:0/1771904369 >> 172.21.15.16:0/1771904369 conn(0x7f58140b4040 msgr2=0x7f581415bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:46.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.578+0000 7f58188e4640 1 -- 172.21.15.16:0/1771904369 shutdown_connections 2024-09-19T07:48:46.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.578+0000 7f58188e4640 1 -- 172.21.15.16:0/1771904369 wait complete. 2024-09-19T07:48:46.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T07:48:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.703+0000 7f729539d640 1 Processor -- start 2024-09-19T07:48:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.703+0000 7f729539d640 1 -- start start 2024-09-19T07:48:46.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.704+0000 7f729539d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72901616f0 0x7f72900d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:46.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.704+0000 7f729539d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f72900b12c0 0x7f72900d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:46.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.704+0000 7f729539d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72900d2480 con 0x7f72900b0e80 2024-09-19T07:48:46.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.704+0000 7f729539d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f72900cf730 con 0x7f72900b12c0 2024-09-19T07:48:46.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.704+0000 7f729539d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f72900d21a0 con 0x7f72901616f0 2024-09-19T07:48:46.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.704+0000 7f7294b9c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f72900b12c0 0x7f72900d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:46.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.704+0000 7f7294b9c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f72900b12c0 0x7f72900d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60098/0 (socket says 172.21.15.16:60098) 2024-09-19T07:48:46.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.704+0000 7f728f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f72900b0e80 0x7f72900cfbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38568/0 (socket says 172.21.15.16:38568) 2024-09-19T07:48:46.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.704+0000 7f7294b9c640 1 -- 172.21.15.16:0/513428999 learned_addr learned my addr 172.21.15.16:0/513428999 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:46.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.704+0000 7f728ffff640 1 --2- 172.21.15.16:0/513428999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72901616f0 0x7f72900d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:46.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.704+0000 7f7294b9c640 1 -- 172.21.15.16:0/513428999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72901616f0 msgr2=0x7f72900d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:46.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.704+0000 7f7294b9c640 1 --2- 172.21.15.16:0/513428999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72901616f0 0x7f72900d12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:46.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.704+0000 7f7294b9c640 1 -- 172.21.15.16:0/513428999 >> v1:172.21.15.16:6789/0 conn(0x7f72900b0e80 legacy=0x7f72900cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:46.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.705+0000 7f728effd640 1 -- 172.21.15.16:0/513428999 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2640774559 0 0) 0x7f72900d2480 con 0x7f72900b0e80 2024-09-19T07:48:46.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.705+0000 7f7294b9c640 1 -- 172.21.15.16:0/513428999 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72900cf0e0 con 0x7f72900b12c0 2024-09-19T07:48:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.705+0000 7f7294b9c640 1 --2- 172.21.15.16:0/513428999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72900b12c0 0x7f72900d26a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7288009e50 tx=0x7f728802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831eccd6a9be5f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.705+0000 7f728effd640 1 -- 172.21.15.16:0/513428999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7288036070 con 0x7f72900b12c0 2024-09-19T07:48:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.705+0000 7f728effd640 1 -- 172.21.15.16:0/513428999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7288031600 con 0x7f72900b12c0 2024-09-19T07:48:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.705+0000 7f728effd640 1 -- 172.21.15.16:0/513428999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72880319d0 con 0x7f72900b12c0 2024-09-19T07:48:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.706+0000 7f729539d640 1 -- 172.21.15.16:0/513428999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72900b12c0 msgr2=0x7f72900d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.706+0000 7f729539d640 1 --2- 172.21.15.16:0/513428999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72900b12c0 0x7f72900d26a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7288009e50 tx=0x7f728802a7a0 comp rx=0 tx=0).stop 2024-09-19T07:48:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.706+0000 7f729539d640 1 -- 172.21.15.16:0/513428999 shutdown_connections 2024-09-19T07:48:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.706+0000 7f729539d640 1 --2- 172.21.15.16:0/513428999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72900b12c0 0x7f72900d26a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.706+0000 7f729539d640 1 --2- 172.21.15.16:0/513428999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72901616f0 0x7f72900d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.706+0000 7f729539d640 1 -- 172.21.15.16:0/513428999 >> 172.21.15.16:0/513428999 conn(0x7f729009d520 msgr2=0x7f72900d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.706+0000 7f729539d640 1 -- 172.21.15.16:0/513428999 shutdown_connections 2024-09-19T07:48:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.706+0000 7f729539d640 1 -- 172.21.15.16:0/513428999 wait complete. 2024-09-19T07:48:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.707+0000 7f729539d640 1 Processor -- start 2024-09-19T07:48:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.707+0000 7f729539d640 1 -- start start 2024-09-19T07:48:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.708+0000 7f729539d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72900b0e80 0x7f72900b9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.708+0000 7f729539d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72900b12c0 0x7f7290150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.708+0000 7f729539d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72901616f0 0x7f729014e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.708+0000 7f729539d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7290161ad0 con 0x7f72900b12c0 2024-09-19T07:48:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.708+0000 7f729539d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f72900d1800 con 0x7f72901616f0 2024-09-19T07:48:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.708+0000 7f729539d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f72901511e0 con 0x7f72900b0e80 2024-09-19T07:48:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.708+0000 7f728f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72900b12c0 0x7f7290150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.708+0000 7f728f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72900b12c0 0x7f7290150ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60106/0 (socket says 172.21.15.16:60106) 2024-09-19T07:48:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.708+0000 7f728f7fe640 1 -- 172.21.15.16:0/2312524255 learned_addr learned my addr 172.21.15.16:0/2312524255 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.709+0000 7f728ffff640 1 --2- 172.21.15.16:0/2312524255 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72900b0e80 0x7f72900b9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.709+0000 7f7294b9c640 1 --2- 172.21.15.16:0/2312524255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72901616f0 0x7f729014e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.709+0000 7f728f7fe640 1 -- 172.21.15.16:0/2312524255 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72900b0e80 msgr2=0x7f72900b9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.709+0000 7f728f7fe640 1 --2- 172.21.15.16:0/2312524255 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72900b0e80 0x7f72900b9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.709+0000 7f728f7fe640 1 -- 172.21.15.16:0/2312524255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72901616f0 msgr2=0x7f729014e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.709+0000 7f728f7fe640 1 --2- 172.21.15.16:0/2312524255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72901616f0 0x7f729014e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.709+0000 7f728f7fe640 1 -- 172.21.15.16:0/2312524255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729014e940 con 0x7f72900b12c0 2024-09-19T07:48:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.709+0000 7f7294b9c640 1 --2- 172.21.15.16:0/2312524255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72901616f0 0x7f729014e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.709+0000 7f728f7fe640 1 --2- 172.21.15.16:0/2312524255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72900b12c0 0x7f7290150ca0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f728400c4d0 tx=0x7f728400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.709+0000 7f728cff9640 1 -- 172.21.15.16:0/2312524255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284018070 con 0x7f72900b12c0 2024-09-19T07:48:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.709+0000 7f728cff9640 1 -- 172.21.15.16:0/2312524255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7284007500 con 0x7f72900b12c0 2024-09-19T07:48:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.709+0000 7f728cff9640 1 -- 172.21.15.16:0/2312524255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284004a20 con 0x7f72900b12c0 2024-09-19T07:48:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.709+0000 7f729539d640 1 -- 172.21.15.16:0/2312524255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7290151480 con 0x7f72900b12c0 2024-09-19T07:48:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.709+0000 7f729539d640 1 -- 172.21.15.16:0/2312524255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7290151a40 con 0x7f72900b12c0 2024-09-19T07:48:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.710+0000 7f729539d640 1 -- 172.21.15.16:0/2312524255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72900d1dc0 con 0x7f72900b12c0 2024-09-19T07:48:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.711+0000 7f728cff9640 1 -- 172.21.15.16:0/2312524255 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f728401b070 con 0x7f72900b12c0 2024-09-19T07:48:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.711+0000 7f728cff9640 1 --2- 172.21.15.16:0/2312524255 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f727c06c2b0 0x7f727c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.711+0000 7f728cff9640 1 -- 172.21.15.16:0/2312524255 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5249+0+0 (secure 0 0 0) 0x7f728408a530 con 0x7f72900b12c0 2024-09-19T07:48:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.711+0000 7f728ffff640 1 --2- 172.21.15.16:0/2312524255 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f727c06c2b0 0x7f727c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.712+0000 7f728ffff640 1 --2- 172.21.15.16:0/2312524255 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f727c06c2b0 0x7f727c06e770 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7280006fd0 tx=0x7f7280008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.715+0000 7f728cff9640 1 -- 172.21.15.16:0/2312524255 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7284058ef0 con 0x7f72900b12c0 2024-09-19T07:48:46.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.956+0000 7f729539d640 1 -- 172.21.15.16:0/2312524255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f729008e2d0 con 0x7f72900b12c0 2024-09-19T07:48:46.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.957+0000 7f728cff9640 1 -- 172.21.15.16:0/2312524255 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7f728405f0c0 con 0x7f72900b12c0 2024-09-19T07:48:46.957 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:46.958 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:48:46.958 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-19T07:48:46.958 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:46.958 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-19T07:48:46.958 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etms80KDxAAqTrG88LTEJIj9TWJu0witQ==", 2024-09-19T07:48:46.958 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:48:46.958 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:48:46.958 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:48:46.958 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:48:46.958 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:48:46.958 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.958 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:46.958 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:46.959 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-19T07:48:46.959 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etm4T0KGxAAcfDOKrSdKzhJ4xvTFIyxcQ==", 2024-09-19T07:48:46.959 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:48:46.959 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:48:46.959 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:48:46.959 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:48:46.959 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:48:46.959 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:46.959 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:46.959 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-19T07:48:46.959 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNDBGFhAAtBioJJ1+NtIDLhd/Hg7aZA==", 2024-09-19T07:48:46.960 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:48:46.960 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:48:46.960 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:48:46.960 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:48:46.960 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:48:46.960 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.960 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:46.960 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:46.960 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-19T07:48:46.960 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmaHEiIhAACqqjCRMV+xlGh9WqYsj+MA==", 2024-09-19T07:48:46.960 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:48:46.961 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:48:46.961 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:48:46.961 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:48:46.961 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:48:46.961 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.961 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:46.961 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:46.961 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-19T07:48:46.961 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDT1etm09plCxAAGvJYF6JrKw49FtXb71ox3A==", 2024-09-19T07:48:46.961 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:48:46.961 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:48:46.961 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:48:46.961 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:48:46.962 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.962 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:46.962 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:46.962 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-19T07:48:46.962 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDW1etmGy8yHRAAlcDsHLnkjtS6sCCR8Olv2A==", 2024-09-19T07:48:46.962 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:48:46.962 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:48:46.962 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:48:46.962 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:48:46.962 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.962 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:46.962 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:46.962 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-19T07:48:46.963 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDZ1etmMhGTHxAATQaW2kgJhHKftutQQtUa6A==", 2024-09-19T07:48:46.963 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:48:46.963 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:48:46.963 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:48:46.963 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:48:46.963 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.963 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:46.963 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:46.963 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-19T07:48:46.963 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDc1etmD3sLIhAAIdY00buvb3bIFA5ZpgxxjA==", 2024-09-19T07:48:46.963 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:48:46.963 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:48:46.963 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:48:46.964 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:48:46.964 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.964 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:46.964 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:46.964 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-19T07:48:46.964 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDi1etmgutcLRAAHJEl5SfQ7N0T4YoYeBfrQg==", 2024-09-19T07:48:46.964 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:48:46.964 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:48:46.964 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:48:46.964 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:48:46.964 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.964 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:46.964 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:46.964 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-19T07:48:46.965 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDm1etmCV3xARAA+5yhQEmIpyNId3RL1Xsxhw==", 2024-09-19T07:48:46.965 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:48:46.965 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:48:46.965 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:48:46.965 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:48:46.965 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:46.965 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:46.965 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-19T07:48:46.965 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDp1etm9XC+AxAA+Lrs0ExfSKaixgOxfX8Nug==", 2024-09-19T07:48:46.965 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:48:46.965 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:48:46.965 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:48:46.966 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:48:46.966 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.966 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:46.966 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:46.966 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-19T07:48:46.966 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDs1etmhmvMBhAAhvW/udIE2gCd8IrJCSFuvw==", 2024-09-19T07:48:46.966 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:48:46.966 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:48:46.966 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:48:46.966 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:48:46.966 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.966 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:46.966 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:46.967 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-19T07:48:46.967 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA==", 2024-09-19T07:48:46.967 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:48:46.967 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:48:46.967 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-19T07:48:46.967 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T07:48:46.967 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:46.967 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:46.967 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-19T07:48:46.967 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmc05KLxAAPQl6wOHqFIf8LVxskiuLVg==", 2024-09-19T07:48:46.967 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:48:46.967 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:48:46.967 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:48:46.968 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:48:46.968 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:48:46.968 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.968 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:46.968 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:46.968 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-19T07:48:46.968 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmjFsCLBAANa7gCMvDw84Ovo7u6RRJ3Q==", 2024-09-19T07:48:46.968 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:48:46.968 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:48:46.968 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:48:46.968 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:48:46.968 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:48:46.968 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:46.969 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:46.969 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-19T07:48:46.969 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmdUbgNBAAfwstyOeeoZfetgVZu4z5xg==", 2024-09-19T07:48:46.969 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:48:46.969 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:48:46.969 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:48:46.969 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:48:46.969 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:48:46.969 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:46.969 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:46.969 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-19T07:48:46.970 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDQ1etm9/vTLhAA9PRfAOA8R4oXp3x0dU8l3g==", 2024-09-19T07:48:46.970 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:48:46.970 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:48:46.970 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:48:46.970 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:48:46.970 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:48:46.970 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:46.970 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:46.970 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-19T07:48:46.970 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNQ68CxAANkK+D3u/oTEz8Su4iuZrCg==", 2024-09-19T07:48:46.970 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:48:46.970 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:48:46.971 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:48:46.971 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:48:46.971 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:48:46.971 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:46.971 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-19T07:48:46.971 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmp+gSCBAAGXzmKfFNd6K2fWeqYhlHAg==", 2024-09-19T07:48:46.971 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:48:46.971 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:48:46.971 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:48:46.971 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:48:46.971 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.971 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:46.971 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T07:48:46.972 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:48:46.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.958+0000 7f729539d640 1 -- 172.21.15.16:0/2312524255 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f727c06c2b0 msgr2=0x7f727c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:46.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.958+0000 7f729539d640 1 --2- 172.21.15.16:0/2312524255 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f727c06c2b0 0x7f727c06e770 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7280006fd0 tx=0x7f7280008040 comp rx=0 tx=0).stop 2024-09-19T07:48:46.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.958+0000 7f729539d640 1 -- 172.21.15.16:0/2312524255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72900b12c0 msgr2=0x7f7290150ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:46.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.958+0000 7f729539d640 1 --2- 172.21.15.16:0/2312524255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72900b12c0 0x7f7290150ca0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f728400c4d0 tx=0x7f728400c990 comp rx=0 tx=0).stop 2024-09-19T07:48:46.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.959+0000 7f729539d640 1 -- 172.21.15.16:0/2312524255 shutdown_connections 2024-09-19T07:48:46.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.959+0000 7f729539d640 1 --2- 172.21.15.16:0/2312524255 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f727c06c2b0 0x7f727c06e770 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:46.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.959+0000 7f729539d640 1 --2- 172.21.15.16:0/2312524255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72901616f0 0x7f729014e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:46.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.959+0000 7f729539d640 1 --2- 172.21.15.16:0/2312524255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72900b12c0 0x7f7290150ca0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:46.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.959+0000 7f729539d640 1 --2- 172.21.15.16:0/2312524255 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72900b0e80 0x7f72900b9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:46.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.959+0000 7f729539d640 1 -- 172.21.15.16:0/2312524255 >> 172.21.15.16:0/2312524255 conn(0x7f729009d520 msgr2=0x7f72900b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:46.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.959+0000 7f729539d640 1 -- 172.21.15.16:0/2312524255 shutdown_connections 2024-09-19T07:48:46.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:46.959+0000 7f729539d640 1 -- 172.21.15.16:0/2312524255 wait complete. 2024-09-19T07:48:46.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T07:48:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.077+0000 7f961d59d640 1 Processor -- start 2024-09-19T07:48:47.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.077+0000 7f961d59d640 1 -- start start 2024-09-19T07:48:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.078+0000 7f961d59d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f961815e500 0x7f96180d1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.078+0000 7f961d59d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96180b4170 0x7f96180d27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.078+0000 7f961d59d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96180d2d10 0x7f96180d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.078+0000 7f961d59d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f96180b5210 con 0x7f961815e500 2024-09-19T07:48:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.078+0000 7f961d59d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f96180b4600 con 0x7f96180b4170 2024-09-19T07:48:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.078+0000 7f961d59d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f96180b48e0 con 0x7f96180d2d10 2024-09-19T07:48:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.078+0000 7f9617fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f961815e500 0x7f96180d1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.078+0000 7f9617fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f961815e500 0x7f96180d1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60114/0 (socket says 172.21.15.16:60114) 2024-09-19T07:48:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.078+0000 7f9617fff640 1 -- 172.21.15.16:0/3028385275 learned_addr learned my addr 172.21.15.16:0/3028385275 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.078+0000 7f9617fff640 1 -- 172.21.15.16:0/3028385275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96180d2d10 msgr2=0x7f96180d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.078+0000 7f9617fff640 1 --2- 172.21.15.16:0/3028385275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96180d2d10 0x7f96180d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.078+0000 7f9617fff640 1 -- 172.21.15.16:0/3028385275 >> v2:172.21.15.146:3300/0 conn(0x7f96180b4170 msgr2=0x7f96180d27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.078+0000 7f9617fff640 1 --2- 172.21.15.16:0/3028385275 >> v2:172.21.15.146:3300/0 conn(0x7f96180b4170 0x7f96180d27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.078+0000 7f9617fff640 1 -- 172.21.15.16:0/3028385275 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96180b4bc0 con 0x7f961815e500 2024-09-19T07:48:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.079+0000 7f9617fff640 1 --2- 172.21.15.16:0/3028385275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961815e500 0x7f96180d1dc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f96180d0960 tx=0x7f960802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba03a3b9a1efe57 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.079+0000 7f9616ffd640 1 -- 172.21.15.16:0/3028385275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608037070 con 0x7f961815e500 2024-09-19T07:48:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.079+0000 7f9616ffd640 1 -- 172.21.15.16:0/3028385275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9608032440 con 0x7f961815e500 2024-09-19T07:48:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.079+0000 7f9616ffd640 1 -- 172.21.15.16:0/3028385275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608032810 con 0x7f961815e500 2024-09-19T07:48:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.080+0000 7f961d59d640 1 -- 172.21.15.16:0/3028385275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961815e500 msgr2=0x7f96180d1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.080+0000 7f961d59d640 1 --2- 172.21.15.16:0/3028385275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961815e500 0x7f96180d1dc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f96180d0960 tx=0x7f960802a270 comp rx=0 tx=0).stop 2024-09-19T07:48:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.080+0000 7f961d59d640 1 -- 172.21.15.16:0/3028385275 shutdown_connections 2024-09-19T07:48:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.080+0000 7f961d59d640 1 --2- 172.21.15.16:0/3028385275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96180d2d10 0x7f96180d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.080+0000 7f961d59d640 1 --2- 172.21.15.16:0/3028385275 >> v2:172.21.15.146:3300/0 conn(0x7f96180b4170 0x7f96180d27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.080+0000 7f961d59d640 1 --2- 172.21.15.16:0/3028385275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961815e500 0x7f96180d1dc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.080+0000 7f961d59d640 1 -- 172.21.15.16:0/3028385275 >> 172.21.15.16:0/3028385275 conn(0x7f961808dc00 msgr2=0x7f961809d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.080+0000 7f961d59d640 1 -- 172.21.15.16:0/3028385275 shutdown_connections 2024-09-19T07:48:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.080+0000 7f961d59d640 1 -- 172.21.15.16:0/3028385275 wait complete. 2024-09-19T07:48:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.081+0000 7f961d59d640 1 Processor -- start 2024-09-19T07:48:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.081+0000 7f961d59d640 1 -- start start 2024-09-19T07:48:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.082+0000 7f961d59d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96180d2d10 0x7f961815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.082+0000 7f961d59d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961815e500 0x7f96180b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.082+0000 7f961d59d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96180b8700 0x7f96180cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.082+0000 7f961d59d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96180d2480 con 0x7f961815e500 2024-09-19T07:48:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.082+0000 7f961d59d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f96180d2300 con 0x7f96180b8700 2024-09-19T07:48:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.082+0000 7f961d59d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f96180d1120 con 0x7f96180d2d10 2024-09-19T07:48:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.082+0000 7f96177fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961815e500 0x7f96180b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.082+0000 7f9617fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96180d2d10 0x7f961815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.082+0000 7f9617fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96180d2d10 0x7f961815c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:58552/0 (socket says 172.21.15.16:58552) 2024-09-19T07:48:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.082+0000 7f9617fff640 1 -- 172.21.15.16:0/290792405 learned_addr learned my addr 172.21.15.16:0/290792405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.082+0000 7f961cd9c640 1 --2- 172.21.15.16:0/290792405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96180b8700 0x7f96180cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.083+0000 7f96177fe640 1 -- 172.21.15.16:0/290792405 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96180d2d10 msgr2=0x7f961815c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.083+0000 7f96177fe640 1 --2- 172.21.15.16:0/290792405 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96180d2d10 0x7f961815c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.083+0000 7f96177fe640 1 -- 172.21.15.16:0/290792405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96180b8700 msgr2=0x7f96180cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.083+0000 7f96177fe640 1 --2- 172.21.15.16:0/290792405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96180b8700 0x7f96180cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.083+0000 7f96177fe640 1 -- 172.21.15.16:0/290792405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96180b9550 con 0x7f961815e500 2024-09-19T07:48:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.083+0000 7f9617fff640 1 --2- 172.21.15.16:0/290792405 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96180d2d10 0x7f961815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:48:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.083+0000 7f96177fe640 1 --2- 172.21.15.16:0/290792405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961815e500 0x7f96180b81c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f960c00b3f0 tx=0x7f960c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.083+0000 7f9614ff9640 1 -- 172.21.15.16:0/290792405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c015020 con 0x7f961815e500 2024-09-19T07:48:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.083+0000 7f9614ff9640 1 -- 172.21.15.16:0/290792405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f960c011830 con 0x7f961815e500 2024-09-19T07:48:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.083+0000 7f9614ff9640 1 -- 172.21.15.16:0/290792405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c011c20 con 0x7f961815e500 2024-09-19T07:48:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.083+0000 7f961d59d640 1 -- 172.21.15.16:0/290792405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96180b9830 con 0x7f961815e500 2024-09-19T07:48:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.083+0000 7f961d59d640 1 -- 172.21.15.16:0/290792405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9618155c80 con 0x7f961815e500 2024-09-19T07:48:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.084+0000 7f961d59d640 1 -- 172.21.15.16:0/290792405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96180d0be0 con 0x7f961815e500 2024-09-19T07:48:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.085+0000 7f9614ff9640 1 -- 172.21.15.16:0/290792405 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f960c018020 con 0x7f961815e500 2024-09-19T07:48:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.085+0000 7f9614ff9640 1 --2- 172.21.15.16:0/290792405 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f960406c2b0 0x7f960406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.085+0000 7f9614ff9640 1 -- 172.21.15.16:0/290792405 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5249+0+0 (secure 0 0 0) 0x7f960c08af30 con 0x7f961815e500 2024-09-19T07:48:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.086+0000 7f9617fff640 1 --2- 172.21.15.16:0/290792405 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f960406c2b0 0x7f960406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.086+0000 7f9617fff640 1 --2- 172.21.15.16:0/290792405 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f960406c2b0 0x7f960406e770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f96080090b0 tx=0x7f960802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:47.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.089+0000 7f9614ff9640 1 -- 172.21.15.16:0/290792405 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f960c01f020 con 0x7f961815e500 2024-09-19T07:48:47.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.267+0000 7f961d59d640 1 -- 172.21.15.16:0/290792405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f96180b7a50 con 0x7f961815e500 2024-09-19T07:48:47.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.267+0000 7f9614ff9640 1 -- 172.21.15.16:0/290792405 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v66) ==== 84+0+65 (secure 0 0 0) 0x7f960c0592a0 con 0x7f961815e500 2024-09-19T07:48:47.271 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:47.271 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-19T07:48:47.271 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:48:47.271 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-19T07:48:47.272 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-19T07:48:47.272 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:48:47.272 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-19T07:48:47.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.269+0000 7f961d59d640 1 -- 172.21.15.16:0/290792405 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f960406c2b0 msgr2=0x7f960406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:47.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.269+0000 7f961d59d640 1 --2- 172.21.15.16:0/290792405 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f960406c2b0 0x7f960406e770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f96080090b0 tx=0x7f960802e000 comp rx=0 tx=0).stop 2024-09-19T07:48:47.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.269+0000 7f961d59d640 1 -- 172.21.15.16:0/290792405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961815e500 msgr2=0x7f96180b81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:47.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.269+0000 7f961d59d640 1 --2- 172.21.15.16:0/290792405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961815e500 0x7f96180b81c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f960c00b3f0 tx=0x7f960c00b8b0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.269+0000 7f961d59d640 1 -- 172.21.15.16:0/290792405 shutdown_connections 2024-09-19T07:48:47.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.269+0000 7f961d59d640 1 --2- 172.21.15.16:0/290792405 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f960406c2b0 0x7f960406e770 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.269+0000 7f961d59d640 1 --2- 172.21.15.16:0/290792405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96180b8700 0x7f96180cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.269+0000 7f961d59d640 1 --2- 172.21.15.16:0/290792405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961815e500 0x7f96180b81c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.269+0000 7f961d59d640 1 --2- 172.21.15.16:0/290792405 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96180d2d10 0x7f961815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.269+0000 7f961d59d640 1 -- 172.21.15.16:0/290792405 >> 172.21.15.16:0/290792405 conn(0x7f961808dc00 msgr2=0x7f961806dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:47.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.269+0000 7f961d59d640 1 -- 172.21.15.16:0/290792405 shutdown_connections 2024-09-19T07:48:47.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.269+0000 7f961d59d640 1 -- 172.21.15.16:0/290792405 wait complete. 2024-09-19T07:48:47.281 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T07:48:47.281 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T07:48:47.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.388+0000 7f851498d640 1 Processor -- start 2024-09-19T07:48:47.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.388+0000 7f851498d640 1 -- start start 2024-09-19T07:48:47.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.389+0000 7f851498d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85100d2750 0x7f851013f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.389+0000 7f851498d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8510072250 0x7f85100cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.389+0000 7f851498d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8510072690 0x7f85100d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.389+0000 7f851498d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f8510145bd0 con 0x7f85100d2750 2024-09-19T07:48:47.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.389+0000 7f851498d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f85101427e0 con 0x7f8510072690 2024-09-19T07:48:47.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.389+0000 7f851498d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f851013fda0 con 0x7f8510072250 2024-09-19T07:48:47.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.389+0000 7f850e59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85100d2750 0x7f851013f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.389+0000 7f850e59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85100d2750 0x7f851013f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60128/0 (socket says 172.21.15.16:60128) 2024-09-19T07:48:47.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.389+0000 7f850e59c640 1 -- 172.21.15.16:0/2519005581 learned_addr learned my addr 172.21.15.16:0/2519005581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:47.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.389+0000 7f850dd9b640 1 --2- 172.21.15.16:0/2519005581 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8510072250 0x7f85100cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.389+0000 7f850ed9d640 1 --2- 172.21.15.16:0/2519005581 >> v2:172.21.15.146:3300/0 conn(0x7f8510072690 0x7f85100d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.389+0000 7f850e59c640 1 -- 172.21.15.16:0/2519005581 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8510072250 msgr2=0x7f85100cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:47.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.389+0000 7f850e59c640 1 --2- 172.21.15.16:0/2519005581 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8510072250 0x7f85100cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.389+0000 7f850e59c640 1 -- 172.21.15.16:0/2519005581 >> v2:172.21.15.146:3300/0 conn(0x7f8510072690 msgr2=0x7f85100d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:47.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.389+0000 7f850e59c640 1 --2- 172.21.15.16:0/2519005581 >> v2:172.21.15.146:3300/0 conn(0x7f8510072690 0x7f85100d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.389+0000 7f850e59c640 1 -- 172.21.15.16:0/2519005581 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85100d2b30 con 0x7f85100d2750 2024-09-19T07:48:47.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.389+0000 7f850ed9d640 1 --2- 172.21.15.16:0/2519005581 >> v2:172.21.15.146:3300/0 conn(0x7f8510072690 0x7f85100d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:47.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.390+0000 7f850e59c640 1 --2- 172.21.15.16:0/2519005581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85100d2750 0x7f851013f860 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8510145e80 tx=0x7f850002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04f2caf9ac40efb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:47.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.390+0000 7f850d59a640 1 -- 172.21.15.16:0/2519005581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500037070 con 0x7f85100d2750 2024-09-19T07:48:47.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.390+0000 7f850d59a640 1 -- 172.21.15.16:0/2519005581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8500032440 con 0x7f85100d2750 2024-09-19T07:48:47.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.390+0000 7f850d59a640 1 -- 172.21.15.16:0/2519005581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500032810 con 0x7f85100d2750 2024-09-19T07:48:47.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.390+0000 7f851498d640 1 -- 172.21.15.16:0/2519005581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85100d2750 msgr2=0x7f851013f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:47.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.390+0000 7f851498d640 1 --2- 172.21.15.16:0/2519005581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85100d2750 0x7f851013f860 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8510145e80 tx=0x7f850002a140 comp rx=0 tx=0).stop 2024-09-19T07:48:47.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.391+0000 7f851498d640 1 -- 172.21.15.16:0/2519005581 shutdown_connections 2024-09-19T07:48:47.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.391+0000 7f851498d640 1 --2- 172.21.15.16:0/2519005581 >> v2:172.21.15.146:3300/0 conn(0x7f8510072690 0x7f85100d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.391+0000 7f851498d640 1 --2- 172.21.15.16:0/2519005581 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8510072250 0x7f85100cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.391+0000 7f851498d640 1 --2- 172.21.15.16:0/2519005581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85100d2750 0x7f851013f860 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.391+0000 7f851498d640 1 -- 172.21.15.16:0/2519005581 >> 172.21.15.16:0/2519005581 conn(0x7f85100b4040 msgr2=0x7f85100b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.391+0000 7f851498d640 1 -- 172.21.15.16:0/2519005581 shutdown_connections 2024-09-19T07:48:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.391+0000 7f851498d640 1 -- 172.21.15.16:0/2519005581 wait complete. 2024-09-19T07:48:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.392+0000 7f851498d640 1 Processor -- start 2024-09-19T07:48:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.392+0000 7f851498d640 1 -- start start 2024-09-19T07:48:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f851498d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8510072250 0x7f85101f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f851498d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8510072690 0x7f851015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f851498d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85100d2750 0x7f85100b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f851498d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8510142220 con 0x7f85100d2750 2024-09-19T07:48:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f851498d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8510141f40 con 0x7f8510072250 2024-09-19T07:48:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f851498d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8510142500 con 0x7f8510072690 2024-09-19T07:48:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f850ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85100d2750 0x7f85100b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f850ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85100d2750 0x7f85100b6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60138/0 (socket says 172.21.15.16:60138) 2024-09-19T07:48:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f850ed9d640 1 -- 172.21.15.16:0/2906306909 learned_addr learned my addr 172.21.15.16:0/2906306909 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f850dd9b640 1 --2- 172.21.15.16:0/2906306909 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8510072690 0x7f851015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f850e59c640 1 --2- 172.21.15.16:0/2906306909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8510072250 0x7f85101f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f850ed9d640 1 -- 172.21.15.16:0/2906306909 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8510072690 msgr2=0x7f851015b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f850ed9d640 1 --2- 172.21.15.16:0/2906306909 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8510072690 0x7f851015b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f850ed9d640 1 -- 172.21.15.16:0/2906306909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8510072250 msgr2=0x7f85101f2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f850ed9d640 1 --2- 172.21.15.16:0/2906306909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8510072250 0x7f85101f2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f850ed9d640 1 -- 172.21.15.16:0/2906306909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85100b6090 con 0x7f85100d2750 2024-09-19T07:48:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f850ed9d640 1 --2- 172.21.15.16:0/2906306909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85100d2750 0x7f85100b6d20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f850800cef0 tx=0x7f850800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f84feffd640 1 -- 172.21.15.16:0/2906306909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850801a070 con 0x7f85100d2750 2024-09-19T07:48:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f84feffd640 1 -- 172.21.15.16:0/2906306909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f850800d860 con 0x7f85100d2750 2024-09-19T07:48:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.393+0000 7f851498d640 1 -- 172.21.15.16:0/2906306909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85100c92c0 con 0x7f85100d2750 2024-09-19T07:48:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.394+0000 7f84feffd640 1 -- 172.21.15.16:0/2906306909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850800dc50 con 0x7f85100d2750 2024-09-19T07:48:47.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.394+0000 7f851498d640 1 -- 172.21.15.16:0/2906306909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85100c9880 con 0x7f85100d2750 2024-09-19T07:48:47.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.395+0000 7f851498d640 1 -- 172.21.15.16:0/2906306909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85101418b0 con 0x7f85100d2750 2024-09-19T07:48:47.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.395+0000 7f84feffd640 1 -- 172.21.15.16:0/2906306909 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f850801d070 con 0x7f85100d2750 2024-09-19T07:48:47.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.395+0000 7f84feffd640 1 --2- 172.21.15.16:0/2906306909 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f84f006c2e0 0x7f84f006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.395+0000 7f84feffd640 1 -- 172.21.15.16:0/2906306909 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5249+0+0 (secure 0 0 0) 0x7f850808b8f0 con 0x7f85100d2750 2024-09-19T07:48:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.396+0000 7f850e59c640 1 --2- 172.21.15.16:0/2906306909 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f84f006c2e0 0x7f84f006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.396+0000 7f850e59c640 1 --2- 172.21.15.16:0/2906306909 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f84f006c2e0 0x7f84f006e7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8500034000 tx=0x7f850002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.400+0000 7f84feffd640 1 -- 172.21.15.16:0/2906306909 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8508059bd0 con 0x7f85100d2750 2024-09-19T07:48:47.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:47.580+0000 7f851498d640 1 -- 172.21.15.16:0/2906306909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f851006ef30 con 0x7f85100d2750 2024-09-19T07:48:48.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.576+0000 7f84feffd640 1 -- 172.21.15.16:0/2906306909 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v67) ==== 138+0+0 (secure 0 0 0) 0x7f850805ded0 con 0x7f85100d2750 2024-09-19T07:48:48.577 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-19T07:48:48.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.579+0000 7f851498d640 1 -- 172.21.15.16:0/2906306909 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f84f006c2e0 msgr2=0x7f84f006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:48.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.579+0000 7f851498d640 1 --2- 172.21.15.16:0/2906306909 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f84f006c2e0 0x7f84f006e7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8500034000 tx=0x7f850002e000 comp rx=0 tx=0).stop 2024-09-19T07:48:48.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.579+0000 7f851498d640 1 -- 172.21.15.16:0/2906306909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85100d2750 msgr2=0x7f85100b6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:48.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.579+0000 7f851498d640 1 --2- 172.21.15.16:0/2906306909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85100d2750 0x7f85100b6d20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f850800cef0 tx=0x7f850800a710 comp rx=0 tx=0).stop 2024-09-19T07:48:48.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.579+0000 7f851498d640 1 -- 172.21.15.16:0/2906306909 shutdown_connections 2024-09-19T07:48:48.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.579+0000 7f851498d640 1 --2- 172.21.15.16:0/2906306909 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f84f006c2e0 0x7f84f006e7a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:48.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.579+0000 7f851498d640 1 --2- 172.21.15.16:0/2906306909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85100d2750 0x7f85100b6d20 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:48.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.579+0000 7f851498d640 1 --2- 172.21.15.16:0/2906306909 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8510072690 0x7f851015b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:48.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.579+0000 7f851498d640 1 --2- 172.21.15.16:0/2906306909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8510072250 0x7f85101f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:48.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.579+0000 7f851498d640 1 -- 172.21.15.16:0/2906306909 >> 172.21.15.16:0/2906306909 conn(0x7f85100b4040 msgr2=0x7f85100cc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:48.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.580+0000 7f851498d640 1 -- 172.21.15.16:0/2906306909 shutdown_connections 2024-09-19T07:48:48.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.580+0000 7f851498d640 1 -- 172.21.15.16:0/2906306909 wait complete. 2024-09-19T07:48:48.598 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T07:48:48.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.715+0000 7f7f28191640 1 Processor -- start 2024-09-19T07:48:48.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.716+0000 7f7f28191640 1 -- start start 2024-09-19T07:48:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f28191640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7f2006f590 0x7f7f200ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f28191640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f200ad830 con 0x7f7f2015b320 2024-09-19T07:48:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f28191640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f200adb10 con 0x7f7f200af6d0 2024-09-19T07:48:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f28191640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f7f200addf0 con 0x7f7f2006f590 2024-09-19T07:48:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f25f2d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7f2015b320 0x7f7f200acf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38576/0 (socket says 172.21.15.16:38576) 2024-09-19T07:48:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f25f2d640 1 -- 172.21.15.16:0/2381947613 learned_addr learned my addr 172.21.15.16:0/2381947613 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f2672e640 1 --2- 172.21.15.16:0/2381947613 >> v2:172.21.15.146:3300/0 conn(0x7f7f2006f590 0x7f7f200ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f24f2b640 1 -- 172.21.15.16:0/2381947613 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4272514278 0 0) 0x7f7f200ad830 con 0x7f7f2015b320 2024-09-19T07:48:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f24f2b640 1 -- 172.21.15.16:0/2381947613 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f0c003520 con 0x7f7f2015b320 2024-09-19T07:48:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f24f2b640 1 -- 172.21.15.16:0/2381947613 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1955576761 0 0) 0x7f7f200adb10 con 0x7f7f200af6d0 2024-09-19T07:48:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f24f2b640 1 -- 172.21.15.16:0/2381947613 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f200ad830 con 0x7f7f200af6d0 2024-09-19T07:48:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f24f2b640 1 -- 172.21.15.16:0/2381947613 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 345513544 0 0) 0x7f7f0c003520 con 0x7f7f2015b320 2024-09-19T07:48:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f24f2b640 1 -- 172.21.15.16:0/2381947613 >> v2:172.21.15.146:3300/0 conn(0x7f7f2006f590 msgr2=0x7f7f200ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f24f2b640 1 --2- 172.21.15.16:0/2381947613 >> v2:172.21.15.146:3300/0 conn(0x7f7f2006f590 0x7f7f200ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f24f2b640 1 -- 172.21.15.16:0/2381947613 >> v1:172.21.15.146:6789/0 conn(0x7f7f200af6d0 legacy=0x7f7f2006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f24f2b640 1 -- 172.21.15.16:0/2381947613 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f200ae0d0 con 0x7f7f2015b320 2024-09-19T07:48:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f24f2b640 1 -- 172.21.15.16:0/2381947613 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f7f1c001970 con 0x7f7f2015b320 2024-09-19T07:48:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f24f2b640 1 -- 172.21.15.16:0/2381947613 >> v1:172.21.15.16:6789/0 conn(0x7f7f2015b320 legacy=0x7f7f200acf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f24f2b640 1 --2- 172.21.15.16:0/2381947613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c003b90 0x7f7f0c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f24f2b640 1 --2- 172.21.15.16:0/2381947613 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f0c004800 0x7f7f0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f24f2b640 1 --2- 172.21.15.16:0/2381947613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f0c005210 0x7f7f0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f24f2b640 1 -- 172.21.15.16:0/2381947613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7f200addf0 con 0x7f7f0c003b90 2024-09-19T07:48:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f24f2b640 1 -- 172.21.15.16:0/2381947613 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0c00c140 con 0x7f7f0c005210 2024-09-19T07:48:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.717+0000 7f7f24f2b640 1 -- 172.21.15.16:0/2381947613 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7f0c00c340 con 0x7f7f0c004800 2024-09-19T07:48:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.718+0000 7f7f2572c640 1 --2- 172.21.15.16:0/2381947613 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f0c004800 0x7f7f0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.718+0000 7f7f25f2d640 1 --2- 172.21.15.16:0/2381947613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c003b90 0x7f7f0c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.718+0000 7f7f2672e640 1 --2- 172.21.15.16:0/2381947613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f0c005210 0x7f7f0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.718+0000 7f7f25f2d640 1 -- 172.21.15.16:0/2381947613 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f0c004800 msgr2=0x7f7f0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.718+0000 7f7f25f2d640 1 --2- 172.21.15.16:0/2381947613 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f0c004800 0x7f7f0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.718+0000 7f7f25f2d640 1 -- 172.21.15.16:0/2381947613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f0c005210 msgr2=0x7f7f0c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.718+0000 7f7f25f2d640 1 --2- 172.21.15.16:0/2381947613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f0c005210 0x7f7f0c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.718+0000 7f7f25f2d640 1 -- 172.21.15.16:0/2381947613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0c003520 con 0x7f7f0c003b90 2024-09-19T07:48:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.718+0000 7f7f2672e640 1 -- 172.21.15.16:0/2381947613 reap_dead start 2024-09-19T07:48:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.719+0000 7f7f25f2d640 1 --2- 172.21.15.16:0/2381947613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c003b90 0x7f7f0c003f90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00a6b0 tx=0x7f7f0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13662f5465c2417a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.719+0000 7f7f24f2b640 1 -- 172.21.15.16:0/2381947613 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c00a2d0 con 0x7f7f0c003b90 2024-09-19T07:48:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.719+0000 7f7f24f2b640 1 -- 172.21.15.16:0/2381947613 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f1c02f070 con 0x7f7f0c003b90 2024-09-19T07:48:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.719+0000 7f7f24f2b640 1 -- 172.21.15.16:0/2381947613 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c02bc10 con 0x7f7f0c003b90 2024-09-19T07:48:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.719+0000 7f7f28191640 1 -- 172.21.15.16:0/2381947613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c003b90 msgr2=0x7f7f0c003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.719+0000 7f7f28191640 1 --2- 172.21.15.16:0/2381947613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c003b90 0x7f7f0c003f90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00a6b0 tx=0x7f7f0c005830 comp rx=0 tx=0).stop 2024-09-19T07:48:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.719+0000 7f7f28191640 1 -- 172.21.15.16:0/2381947613 shutdown_connections 2024-09-19T07:48:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.719+0000 7f7f28191640 1 --2- 172.21.15.16:0/2381947613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c003b90 0x7f7f0c003f90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.719+0000 7f7f28191640 1 -- 172.21.15.16:0/2381947613 >> 172.21.15.16:0/2381947613 conn(0x7f7f200b1c70 msgr2=0x7f7f200b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.720+0000 7f7f28191640 1 -- 172.21.15.16:0/2381947613 shutdown_connections 2024-09-19T07:48:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.720+0000 7f7f28191640 1 -- 172.21.15.16:0/2381947613 wait complete. 2024-09-19T07:48:48.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.721+0000 7f7f28191640 1 Processor -- start 2024-09-19T07:48:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.721+0000 7f7f28191640 1 -- start start 2024-09-19T07:48:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.721+0000 7f7f28191640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c003b90 0x7f7f201531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.721+0000 7f7f28191640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f201553e0 0x7f7f20155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f28191640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f200cecf0 0x7f7f200cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f28191640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7f200ae8d0 con 0x7f7f0c003b90 2024-09-19T07:48:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f28191640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2015b700 con 0x7f7f201553e0 2024-09-19T07:48:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f28191640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7f200aea50 con 0x7f7f200cecf0 2024-09-19T07:48:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f25f2d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c003b90 0x7f7f201531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f25f2d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c003b90 0x7f7f201531f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60150/0 (socket says 172.21.15.16:60150) 2024-09-19T07:48:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f25f2d640 1 -- 172.21.15.16:0/2707755205 learned_addr learned my addr 172.21.15.16:0/2707755205 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f2672e640 1 --2- 172.21.15.16:0/2707755205 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f200cecf0 0x7f7f200cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f2572c640 1 --2- 172.21.15.16:0/2707755205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f201553e0 0x7f7f20155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f25f2d640 1 -- 172.21.15.16:0/2707755205 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f200cecf0 msgr2=0x7f7f200cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:48.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f25f2d640 1 --2- 172.21.15.16:0/2707755205 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f200cecf0 0x7f7f200cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f25f2d640 1 -- 172.21.15.16:0/2707755205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f201553e0 msgr2=0x7f7f20155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f25f2d640 1 --2- 172.21.15.16:0/2707755205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f201553e0 0x7f7f20155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f25f2d640 1 -- 172.21.15.16:0/2707755205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2015bbd0 con 0x7f7f0c003b90 2024-09-19T07:48:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f25f2d640 1 --2- 172.21.15.16:0/2707755205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c003b90 0x7f7f201531f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c02b380 tx=0x7f7f1c02d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f167fc640 1 -- 172.21.15.16:0/2707755205 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c00a2d0 con 0x7f7f0c003b90 2024-09-19T07:48:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f167fc640 1 -- 172.21.15.16:0/2707755205 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f1c034470 con 0x7f7f0c003b90 2024-09-19T07:48:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f28191640 1 -- 172.21.15.16:0/2707755205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2015beb0 con 0x7f7f0c003b90 2024-09-19T07:48:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f167fc640 1 -- 172.21.15.16:0/2707755205 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c034820 con 0x7f7f0c003b90 2024-09-19T07:48:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.722+0000 7f7f28191640 1 -- 172.21.15.16:0/2707755205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f20153a80 con 0x7f7f0c003b90 2024-09-19T07:48:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.723+0000 7f7f28191640 1 -- 172.21.15.16:0/2707755205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f200ce120 con 0x7f7f0c003b90 2024-09-19T07:48:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.724+0000 7f7f167fc640 1 -- 172.21.15.16:0/2707755205 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7f1c0349c0 con 0x7f7f0c003b90 2024-09-19T07:48:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.724+0000 7f7f167fc640 1 --2- 172.21.15.16:0/2707755205 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7f0406c2e0 0x7f7f0406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.724+0000 7f7f167fc640 1 -- 172.21.15.16:0/2707755205 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5619+0+0 (secure 0 0 0) 0x7f7f1c0ab890 con 0x7f7f0c003b90 2024-09-19T07:48:48.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.725+0000 7f7f2572c640 1 --2- 172.21.15.16:0/2707755205 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7f0406c2e0 0x7f7f0406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:48.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.725+0000 7f7f2572c640 1 --2- 172.21.15.16:0/2707755205 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7f0406c2e0 0x7f7f0406e7a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7f201f73d0 tx=0x7f7f10009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:48.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.729+0000 7f7f167fc640 1 -- 172.21.15.16:0/2707755205 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7f1c0799f0 con 0x7f7f0c003b90 2024-09-19T07:48:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:48.903+0000 7f7f28191640 1 -- 172.21.15.16:0/2707755205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7f20075510 con 0x7f7f0c003b90 2024-09-19T07:48:49.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.579+0000 7f7f167fc640 1 -- 172.21.15.16:0/2707755205 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v68) ==== 170+0+0 (secure 0 0 0) 0x7f7f1c07dcf0 con 0x7f7f0c003b90 2024-09-19T07:48:49.579 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-19T07:48:49.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.583+0000 7f7f28191640 1 -- 172.21.15.16:0/2707755205 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7f0406c2e0 msgr2=0x7f7f0406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:49.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.583+0000 7f7f28191640 1 --2- 172.21.15.16:0/2707755205 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7f0406c2e0 0x7f7f0406e7a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7f201f73d0 tx=0x7f7f10009040 comp rx=0 tx=0).stop 2024-09-19T07:48:49.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.583+0000 7f7f28191640 1 -- 172.21.15.16:0/2707755205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c003b90 msgr2=0x7f7f201531f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:49.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.583+0000 7f7f28191640 1 --2- 172.21.15.16:0/2707755205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c003b90 0x7f7f201531f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c02b380 tx=0x7f7f1c02d070 comp rx=0 tx=0).stop 2024-09-19T07:48:49.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.584+0000 7f7f28191640 1 -- 172.21.15.16:0/2707755205 shutdown_connections 2024-09-19T07:48:49.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.584+0000 7f7f28191640 1 --2- 172.21.15.16:0/2707755205 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7f0406c2e0 0x7f7f0406e7a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.584+0000 7f7f28191640 1 --2- 172.21.15.16:0/2707755205 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f200cecf0 0x7f7f200cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.584+0000 7f7f28191640 1 --2- 172.21.15.16:0/2707755205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f201553e0 0x7f7f20155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.584+0000 7f7f28191640 1 --2- 172.21.15.16:0/2707755205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c003b90 0x7f7f201531f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.584+0000 7f7f28191640 1 -- 172.21.15.16:0/2707755205 >> 172.21.15.16:0/2707755205 conn(0x7f7f200b1c70 msgr2=0x7f7f200b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:49.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.594+0000 7f7f28191640 1 -- 172.21.15.16:0/2707755205 shutdown_connections 2024-09-19T07:48:49.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.595+0000 7f7f28191640 1 -- 172.21.15.16:0/2707755205 wait complete. 2024-09-19T07:48:49.614 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T07:48:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.724+0000 7f55a6766640 1 Processor -- start 2024-09-19T07:48:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.724+0000 7f55a6766640 1 -- start start 2024-09-19T07:48:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.725+0000 7f55a6766640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f55a015e680 0x7f55a00cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.725+0000 7f55a6766640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55a00d0010 0x7f55a00d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.725+0000 7f55a6766640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55a00d1880 con 0x7f55a015e2a0 2024-09-19T07:48:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.725+0000 7f55a6766640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f55a00d1b60 con 0x7f55a00d0010 2024-09-19T07:48:49.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.725+0000 7f55a6766640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f55a00cf2d0 con 0x7f55a015e680 2024-09-19T07:48:49.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.725+0000 7f55a4d03640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55a00d0010 0x7f55a00d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.725+0000 7f55a4d03640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55a00d0010 0x7f55a00d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60158/0 (socket says 172.21.15.16:60158) 2024-09-19T07:48:49.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.725+0000 7f55a4d03640 1 -- 172.21.15.16:0/3423343803 learned_addr learned my addr 172.21.15.16:0/3423343803 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:49.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.726+0000 7f559f7fe640 1 --2- 172.21.15.16:0/3423343803 >> v2:172.21.15.146:3300/0 conn(0x7f55a015e680 0x7f55a00cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.726+0000 7f55a4d03640 1 -- 172.21.15.16:0/3423343803 >> v2:172.21.15.146:3300/0 conn(0x7f55a015e680 msgr2=0x7f55a00cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.726+0000 7f55a4d03640 1 --2- 172.21.15.16:0/3423343803 >> v2:172.21.15.146:3300/0 conn(0x7f55a015e680 0x7f55a00cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.726+0000 7f55a4d03640 1 -- 172.21.15.16:0/3423343803 >> v1:172.21.15.146:6789/0 conn(0x7f55a015e2a0 legacy=0x7f55a006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.726+0000 7f55a4d03640 1 -- 172.21.15.16:0/3423343803 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a00cf5b0 con 0x7f55a00d0010 2024-09-19T07:48:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.726+0000 7f55a4d03640 1 --2- 172.21.15.16:0/3423343803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55a00d0010 0x7f55a00d4370 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f55a00a22a0 tx=0x7f559802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1df87912cff1625 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.727+0000 7f559effd640 1 -- 172.21.15.16:0/3423343803 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598036070 con 0x7f55a00d0010 2024-09-19T07:48:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.727+0000 7f559effd640 1 -- 172.21.15.16:0/3423343803 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5598031600 con 0x7f55a00d0010 2024-09-19T07:48:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.727+0000 7f559effd640 1 -- 172.21.15.16:0/3423343803 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55980319d0 con 0x7f55a00d0010 2024-09-19T07:48:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.727+0000 7f55a6766640 1 -- 172.21.15.16:0/3423343803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55a00d0010 msgr2=0x7f55a00d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.727+0000 7f55a6766640 1 --2- 172.21.15.16:0/3423343803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55a00d0010 0x7f55a00d4370 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f55a00a22a0 tx=0x7f559802a6a0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.727+0000 7f55a6766640 1 -- 172.21.15.16:0/3423343803 shutdown_connections 2024-09-19T07:48:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.727+0000 7f55a6766640 1 --2- 172.21.15.16:0/3423343803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55a00d0010 0x7f55a00d4370 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.727+0000 7f55a6766640 1 --2- 172.21.15.16:0/3423343803 >> v2:172.21.15.146:3300/0 conn(0x7f55a015e680 0x7f55a00cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.727+0000 7f55a6766640 1 -- 172.21.15.16:0/3423343803 >> 172.21.15.16:0/3423343803 conn(0x7f55a009d760 msgr2=0x7f55a008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.727+0000 7f55a6766640 1 -- 172.21.15.16:0/3423343803 shutdown_connections 2024-09-19T07:48:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.727+0000 7f55a6766640 1 -- 172.21.15.16:0/3423343803 wait complete. 2024-09-19T07:48:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.728+0000 7f55a6766640 1 Processor -- start 2024-09-19T07:48:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.729+0000 7f55a6766640 1 -- start start 2024-09-19T07:48:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.729+0000 7f55a6766640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55a00d0010 0x7f55a009ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.729+0000 7f55a6766640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55a015e2a0 0x7f55a00c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.729+0000 7f55a6766640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55a015e680 0x7f55a0154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.729+0000 7f55a6766640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55a006dcb0 con 0x7f55a015e2a0 2024-09-19T07:48:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.729+0000 7f55a6766640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f55a006db30 con 0x7f55a00d0010 2024-09-19T07:48:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.729+0000 7f55a6766640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f55a00d48b0 con 0x7f55a015e680 2024-09-19T07:48:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.729+0000 7f559f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55a015e2a0 0x7f55a00c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.729+0000 7f559f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55a015e2a0 0x7f55a00c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60166/0 (socket says 172.21.15.16:60166) 2024-09-19T07:48:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.729+0000 7f559f7fe640 1 -- 172.21.15.16:0/81770921 learned_addr learned my addr 172.21.15.16:0/81770921 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.730+0000 7f559ffff640 1 --2- 172.21.15.16:0/81770921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55a00d0010 0x7f55a009ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.730+0000 7f55a4d03640 1 --2- 172.21.15.16:0/81770921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55a015e680 0x7f55a0154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.730+0000 7f559f7fe640 1 -- 172.21.15.16:0/81770921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55a015e680 msgr2=0x7f55a0154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:49.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.730+0000 7f559f7fe640 1 --2- 172.21.15.16:0/81770921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55a015e680 0x7f55a0154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.730+0000 7f559f7fe640 1 -- 172.21.15.16:0/81770921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55a00d0010 msgr2=0x7f55a009ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:49.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.730+0000 7f559f7fe640 1 --2- 172.21.15.16:0/81770921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55a00d0010 0x7f55a009ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.730+0000 7f559f7fe640 1 -- 172.21.15.16:0/81770921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a00ca430 con 0x7f55a015e2a0 2024-09-19T07:48:49.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.730+0000 7f559f7fe640 1 --2- 172.21.15.16:0/81770921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55a015e2a0 0x7f55a00c9ef0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f559400d5a0 tx=0x7f559400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:49.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.730+0000 7f559cff9640 1 -- 172.21.15.16:0/81770921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55940125f0 con 0x7f55a015e2a0 2024-09-19T07:48:49.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.730+0000 7f559cff9640 1 -- 172.21.15.16:0/81770921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5594019070 con 0x7f55a015e2a0 2024-09-19T07:48:49.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.730+0000 7f55a6766640 1 -- 172.21.15.16:0/81770921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a00caf50 con 0x7f55a015e2a0 2024-09-19T07:48:49.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.730+0000 7f559cff9640 1 -- 172.21.15.16:0/81770921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55940129f0 con 0x7f55a015e2a0 2024-09-19T07:48:49.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.730+0000 7f55a6766640 1 -- 172.21.15.16:0/81770921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a00ca6b0 con 0x7f55a015e2a0 2024-09-19T07:48:49.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.731+0000 7f55a6766640 1 -- 172.21.15.16:0/81770921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a0073710 con 0x7f55a015e2a0 2024-09-19T07:48:49.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.732+0000 7f559cff9640 1 -- 172.21.15.16:0/81770921 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5594012b90 con 0x7f55a015e2a0 2024-09-19T07:48:49.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.732+0000 7f559cff9640 1 --2- 172.21.15.16:0/81770921 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f558c06c2b0 0x7f558c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.732+0000 7f559cff9640 1 -- 172.21.15.16:0/81770921 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6001+0+0 (secure 0 0 0) 0x7f559408b760 con 0x7f55a015e2a0 2024-09-19T07:48:49.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.733+0000 7f559ffff640 1 --2- 172.21.15.16:0/81770921 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f558c06c2b0 0x7f558c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.733+0000 7f559ffff640 1 --2- 172.21.15.16:0/81770921 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f558c06c2b0 0x7f558c06e770 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5590001780 tx=0x7f5590009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:49.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.737+0000 7f559cff9640 1 -- 172.21.15.16:0/81770921 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f55a0073710 con 0x7f55a015e2a0 2024-09-19T07:48:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:49.953+0000 7f55a6766640 1 -- 172.21.15.16:0/81770921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f55a00d4c70 con 0x7f55a015e2a0 2024-09-19T07:48:50.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.609+0000 7f559cff9640 1 -- 172.21.15.16:0/81770921 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v31) ==== 173+0+0 (secure 0 0 0) 0x7f55a00ca430 con 0x7f55a015e2a0 2024-09-19T07:48:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 9 and data pool 10 2024-09-19T07:48:50.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.622+0000 7f55a6766640 1 -- 172.21.15.16:0/81770921 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f558c06c2b0 msgr2=0x7f558c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:50.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.622+0000 7f55a6766640 1 --2- 172.21.15.16:0/81770921 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f558c06c2b0 0x7f558c06e770 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5590001780 tx=0x7f5590009040 comp rx=0 tx=0).stop 2024-09-19T07:48:50.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.622+0000 7f55a6766640 1 -- 172.21.15.16:0/81770921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55a015e2a0 msgr2=0x7f55a00c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:50.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.622+0000 7f55a6766640 1 --2- 172.21.15.16:0/81770921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55a015e2a0 0x7f55a00c9ef0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f559400d5a0 tx=0x7f559400da60 comp rx=0 tx=0).stop 2024-09-19T07:48:50.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.622+0000 7f55a6766640 1 -- 172.21.15.16:0/81770921 shutdown_connections 2024-09-19T07:48:50.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.622+0000 7f55a6766640 1 --2- 172.21.15.16:0/81770921 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f558c06c2b0 0x7f558c06e770 secure :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5590001780 tx=0x7f5590009040 comp rx=0 tx=0).stop 2024-09-19T07:48:50.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.622+0000 7f55a6766640 1 --2- 172.21.15.16:0/81770921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55a015e680 0x7f55a0154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.622+0000 7f55a6766640 1 --2- 172.21.15.16:0/81770921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55a015e2a0 0x7f55a00c9ef0 secure :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f559400d5a0 tx=0x7f559400da60 comp rx=0 tx=0).stop 2024-09-19T07:48:50.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.622+0000 7f55a6766640 1 --2- 172.21.15.16:0/81770921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55a00d0010 0x7f55a009ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.622+0000 7f55a6766640 1 -- 172.21.15.16:0/81770921 >> 172.21.15.16:0/81770921 conn(0x7f55a009d760 msgr2=0x7f55a00b5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:50.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.625+0000 7f55a6766640 1 -- 172.21.15.16:0/81770921 shutdown_connections 2024-09-19T07:48:50.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.626+0000 7f55a6766640 1 -- 172.21.15.16:0/81770921 wait complete. 2024-09-19T07:48:50.643 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:48:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.751+0000 7f76a9d78640 1 Processor -- start 2024-09-19T07:48:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.751+0000 7f76a9d78640 1 -- start start 2024-09-19T07:48:50.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.752+0000 7f76a9d78640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76a40d0070 0x7f76a415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.752+0000 7f76a9d78640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76a40d1b10 con 0x7f76a40d4130 2024-09-19T07:48:50.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.752+0000 7f76a9d78640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76a40cf2b0 con 0x7f76a415f0d0 2024-09-19T07:48:50.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.752+0000 7f76a9d78640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f76a40cf590 con 0x7f76a40d0070 2024-09-19T07:48:50.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.752+0000 7f76a3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f76a40d4130 0x7f76a40d45c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38586/0 (socket says 172.21.15.16:38586) 2024-09-19T07:48:50.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.752+0000 7f76a3fff640 1 -- 172.21.15.16:0/43987368 learned_addr learned my addr 172.21.15.16:0/43987368 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:50.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.752+0000 7f76a2ffd640 1 --2- 172.21.15.16:0/43987368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76a40d0070 0x7f76a415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.752+0000 7f76a27fc640 1 -- 172.21.15.16:0/43987368 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3719326373 0 0) 0x7f76a40d1b10 con 0x7f76a40d4130 2024-09-19T07:48:50.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.752+0000 7f76a27fc640 1 -- 172.21.15.16:0/43987368 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f768c003520 con 0x7f76a40d4130 2024-09-19T07:48:50.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.752+0000 7f76a27fc640 1 -- 172.21.15.16:0/43987368 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3990883423 0 0) 0x7f76a40cf2b0 con 0x7f76a415f0d0 2024-09-19T07:48:50.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.752+0000 7f76a27fc640 1 -- 172.21.15.16:0/43987368 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76a40d1b10 con 0x7f76a415f0d0 2024-09-19T07:48:50.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.752+0000 7f76a2ffd640 1 -- 172.21.15.16:0/43987368 >> v1:172.21.15.146:6789/0 conn(0x7f76a415f0d0 legacy=0x7f76a40d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:50.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.752+0000 7f76a2ffd640 1 -- 172.21.15.16:0/43987368 >> v1:172.21.15.16:6789/0 conn(0x7f76a40d4130 legacy=0x7f76a40d45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:50.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.752+0000 7f76a2ffd640 1 -- 172.21.15.16:0/43987368 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a40cf870 con 0x7f76a40d0070 2024-09-19T07:48:50.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.753+0000 7f76a2ffd640 1 --2- 172.21.15.16:0/43987368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76a40d0070 0x7f76a415f4b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f76980097d0 tx=0x7f769802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=606c53e9ce426925 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:50.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.753+0000 7f76a27fc640 1 -- 172.21.15.16:0/43987368 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698037070 con 0x7f76a40d0070 2024-09-19T07:48:50.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.753+0000 7f76a27fc640 1 -- 172.21.15.16:0/43987368 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7698032440 con 0x7f76a40d0070 2024-09-19T07:48:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.753+0000 7f76a27fc640 1 -- 172.21.15.16:0/43987368 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76980327f0 con 0x7f76a40d0070 2024-09-19T07:48:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.754+0000 7f76a9d78640 1 -- 172.21.15.16:0/43987368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76a40d0070 msgr2=0x7f76a415f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.754+0000 7f76a9d78640 1 --2- 172.21.15.16:0/43987368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76a40d0070 0x7f76a415f4b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f76980097d0 tx=0x7f769802a180 comp rx=0 tx=0).stop 2024-09-19T07:48:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.754+0000 7f76a9d78640 1 -- 172.21.15.16:0/43987368 shutdown_connections 2024-09-19T07:48:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.754+0000 7f76a9d78640 1 --2- 172.21.15.16:0/43987368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76a40d0070 0x7f76a415f4b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.754+0000 7f76a9d78640 1 -- 172.21.15.16:0/43987368 >> 172.21.15.16:0/43987368 conn(0x7f76a409d760 msgr2=0x7f76a408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.754+0000 7f76a9d78640 1 -- 172.21.15.16:0/43987368 shutdown_connections 2024-09-19T07:48:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.754+0000 7f76a9d78640 1 -- 172.21.15.16:0/43987368 wait complete. 2024-09-19T07:48:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.755+0000 7f76a9d78640 1 Processor -- start 2024-09-19T07:48:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.755+0000 7f76a9d78640 1 -- start start 2024-09-19T07:48:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.756+0000 7f76a9d78640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76a40d0070 0x7f76a415b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.756+0000 7f76a9d78640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a40d4130 0x7f76a41601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.756+0000 7f76a9d78640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76a415f0d0 0x7f76a41f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.756+0000 7f76a9d78640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76a40b2f70 con 0x7f76a40d4130 2024-09-19T07:48:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.756+0000 7f76a9d78640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f76a40cef60 con 0x7f76a415f0d0 2024-09-19T07:48:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.756+0000 7f76a9d78640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f76a40b30f0 con 0x7f76a40d0070 2024-09-19T07:48:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.756+0000 7f76a2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a40d4130 0x7f76a41601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.756+0000 7f76a2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a40d4130 0x7f76a41601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60182/0 (socket says 172.21.15.16:60182) 2024-09-19T07:48:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.756+0000 7f76a2ffd640 1 -- 172.21.15.16:0/3467211514 learned_addr learned my addr 172.21.15.16:0/3467211514 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.756+0000 7f76a37fe640 1 --2- 172.21.15.16:0/3467211514 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76a40d0070 0x7f76a415b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.756+0000 7f76a3fff640 1 --2- 172.21.15.16:0/3467211514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76a415f0d0 0x7f76a41f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.756+0000 7f76a2ffd640 1 -- 172.21.15.16:0/3467211514 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76a40d0070 msgr2=0x7f76a415b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.756+0000 7f76a2ffd640 1 --2- 172.21.15.16:0/3467211514 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76a40d0070 0x7f76a415b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.756+0000 7f76a2ffd640 1 -- 172.21.15.16:0/3467211514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76a415f0d0 msgr2=0x7f76a41f7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:50.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.756+0000 7f76a2ffd640 1 --2- 172.21.15.16:0/3467211514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76a415f0d0 0x7f76a41f7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.756+0000 7f76a2ffd640 1 -- 172.21.15.16:0/3467211514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a4160df0 con 0x7f76a40d4130 2024-09-19T07:48:50.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.756+0000 7f76a3fff640 1 --2- 172.21.15.16:0/3467211514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76a415f0d0 0x7f76a41f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:50.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.756+0000 7f76a2ffd640 1 --2- 172.21.15.16:0/3467211514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a40d4130 0x7f76a41601a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7698035040 tx=0x7f76980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:50.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.757+0000 7f768bfff640 1 -- 172.21.15.16:0/3467211514 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698037040 con 0x7f76a40d4130 2024-09-19T07:48:50.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.757+0000 7f768bfff640 1 -- 172.21.15.16:0/3467211514 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76980044c0 con 0x7f76a40d4130 2024-09-19T07:48:50.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.757+0000 7f76a9d78640 1 -- 172.21.15.16:0/3467211514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a41f8870 con 0x7f76a40d4130 2024-09-19T07:48:50.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.757+0000 7f768bfff640 1 -- 172.21.15.16:0/3467211514 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76980325f0 con 0x7f76a40d4130 2024-09-19T07:48:50.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.757+0000 7f76a9d78640 1 -- 172.21.15.16:0/3467211514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a415f6c0 con 0x7f76a40d4130 2024-09-19T07:48:50.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.758+0000 7f76a9d78640 1 -- 172.21.15.16:0/3467211514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a415f4b0 con 0x7f76a40d4130 2024-09-19T07:48:50.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.758+0000 7f768bfff640 1 -- 172.21.15.16:0/3467211514 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7698044020 con 0x7f76a40d4130 2024-09-19T07:48:50.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.759+0000 7f768bfff640 1 --2- 172.21.15.16:0/3467211514 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f769006c2e0 0x7f769006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.759+0000 7f768bfff640 1 -- 172.21.15.16:0/3467211514 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6101+0+0 (secure 0 0 0) 0x7f76980ab760 con 0x7f76a40d4130 2024-09-19T07:48:50.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.759+0000 7f76a37fe640 1 --2- 172.21.15.16:0/3467211514 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f769006c2e0 0x7f769006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.760+0000 7f76a37fe640 1 --2- 172.21.15.16:0/3467211514 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f769006c2e0 0x7f769006e7a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f76940015b0 tx=0x7f7694009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:50.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.763+0000 7f768bfff640 1 -- 172.21.15.16:0/3467211514 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7698079dd0 con 0x7f76a40d4130 2024-09-19T07:48:50.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.939+0000 7f76a9d78640 1 -- 172.21.15.16:0/3467211514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76a415e370 con 0x7f76a40d4130 2024-09-19T07:48:50.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.940+0000 7f768bfff640 1 -- 172.21.15.16:0/3467211514 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16981 (secure 0 0 0) 0x7f76980796d0 con 0x7f76a40d4130 2024-09-19T07:48:50.941 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:50.941 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":69,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:48:49.954397+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:48:47.611316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-19T07:48:48.934687+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:48:50.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.942+0000 7f76a9d78640 1 -- 172.21.15.16:0/3467211514 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f769006c2e0 msgr2=0x7f769006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:50.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.942+0000 7f76a9d78640 1 --2- 172.21.15.16:0/3467211514 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f769006c2e0 0x7f769006e7a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f76940015b0 tx=0x7f7694009040 comp rx=0 tx=0).stop 2024-09-19T07:48:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.942+0000 7f76a9d78640 1 -- 172.21.15.16:0/3467211514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a40d4130 msgr2=0x7f76a41601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.942+0000 7f76a9d78640 1 --2- 172.21.15.16:0/3467211514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a40d4130 0x7f76a41601a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7698035040 tx=0x7f76980046c0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.942+0000 7f76a9d78640 1 -- 172.21.15.16:0/3467211514 shutdown_connections 2024-09-19T07:48:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.942+0000 7f76a9d78640 1 --2- 172.21.15.16:0/3467211514 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f769006c2e0 0x7f769006e7a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.942+0000 7f76a9d78640 1 --2- 172.21.15.16:0/3467211514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76a415f0d0 0x7f76a41f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.942+0000 7f76a9d78640 1 --2- 172.21.15.16:0/3467211514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a40d4130 0x7f76a41601a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.942+0000 7f76a9d78640 1 --2- 172.21.15.16:0/3467211514 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76a40d0070 0x7f76a415b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.942+0000 7f76a9d78640 1 -- 172.21.15.16:0/3467211514 >> 172.21.15.16:0/3467211514 conn(0x7f76a409d760 msgr2=0x7f76a40b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.942+0000 7f76a9d78640 1 -- 172.21.15.16:0/3467211514 shutdown_connections 2024-09-19T07:48:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:50.942+0000 7f76a9d78640 1 -- 172.21.15.16:0/3467211514 wait complete. 2024-09-19T07:48:50.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:48:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.057+0000 7f593ff31640 1 Processor -- start 2024-09-19T07:48:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.057+0000 7f593ff31640 1 -- start start 2024-09-19T07:48:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.058+0000 7f593ff31640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f593815f0d0 0x7f593815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.058+0000 7f593ff31640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59380d0070 0x7f59380d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.058+0000 7f593ff31640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59380cf2b0 con 0x7f59380d1940 2024-09-19T07:48:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.058+0000 7f593ff31640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f59380cf590 con 0x7f593815f0d0 2024-09-19T07:48:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.058+0000 7f593ff31640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f59380cf870 con 0x7f59380d0070 2024-09-19T07:48:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.059+0000 7f593dccd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f593815f0d0 0x7f593815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.059+0000 7f593e4ce640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f59380d1940 0x7f59380d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:40752/0 (socket says 172.21.15.16:40752) 2024-09-19T07:48:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.059+0000 7f593d4cc640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59380d0070 0x7f59380d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.059+0000 7f593e4ce640 1 -- 172.21.15.16:0/2176042699 learned_addr learned my addr 172.21.15.16:0/2176042699 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.059+0000 7f593dccd640 1 -- 172.21.15.16:0/2176042699 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59380d0070 msgr2=0x7f59380d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.059+0000 7f593dccd640 1 --2- 172.21.15.16:0/2176042699 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59380d0070 0x7f59380d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.059+0000 7f593dccd640 1 -- 172.21.15.16:0/2176042699 >> v1:172.21.15.146:6789/0 conn(0x7f59380d1940 legacy=0x7f59380d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.059+0000 7f593dccd640 1 -- 172.21.15.16:0/2176042699 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59380cfb50 con 0x7f593815f0d0 2024-09-19T07:48:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.060+0000 7f593dccd640 1 --2- 172.21.15.16:0/2176042699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f593815f0d0 0x7f593815f4b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f59340097a0 tx=0x7f593402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29e8d6e14343ee53 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.060+0000 7f593cccb640 1 -- 172.21.15.16:0/2176042699 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934037070 con 0x7f593815f0d0 2024-09-19T07:48:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.060+0000 7f593cccb640 1 -- 172.21.15.16:0/2176042699 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f593402ae50 con 0x7f593815f0d0 2024-09-19T07:48:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.060+0000 7f593cccb640 1 -- 172.21.15.16:0/2176042699 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934032660 con 0x7f593815f0d0 2024-09-19T07:48:51.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.061+0000 7f593ff31640 1 -- 172.21.15.16:0/2176042699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f593815f0d0 msgr2=0x7f593815f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:51.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.061+0000 7f593ff31640 1 --2- 172.21.15.16:0/2176042699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f593815f0d0 0x7f593815f4b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f59340097a0 tx=0x7f593402a100 comp rx=0 tx=0).stop 2024-09-19T07:48:51.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.061+0000 7f593ff31640 1 -- 172.21.15.16:0/2176042699 shutdown_connections 2024-09-19T07:48:51.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.061+0000 7f593ff31640 1 --2- 172.21.15.16:0/2176042699 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59380d0070 0x7f59380d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.061+0000 7f593ff31640 1 --2- 172.21.15.16:0/2176042699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f593815f0d0 0x7f593815f4b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.061+0000 7f593ff31640 1 -- 172.21.15.16:0/2176042699 >> 172.21.15.16:0/2176042699 conn(0x7f593809d760 msgr2=0x7f593808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:51.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.061+0000 7f593ff31640 1 -- 172.21.15.16:0/2176042699 shutdown_connections 2024-09-19T07:48:51.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.061+0000 7f593ff31640 1 -- 172.21.15.16:0/2176042699 wait complete. 2024-09-19T07:48:51.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.062+0000 7f593ff31640 1 Processor -- start 2024-09-19T07:48:51.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.062+0000 7f593ff31640 1 -- start start 2024-09-19T07:48:51.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.063+0000 7f593ff31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59380d0070 0x7f593814b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:51.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.063+0000 7f593ff31640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59380d1940 0x7f59380cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:51.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.063+0000 7f593ff31640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f593815f0d0 0x7f59380b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:51.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.063+0000 7f593ff31640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f59380d49c0 con 0x7f59380d0070 2024-09-19T07:48:51.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.063+0000 7f593ff31640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f59380d4840 con 0x7f593815f0d0 2024-09-19T07:48:51.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.063+0000 7f593ff31640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f59380d4b40 con 0x7f59380d1940 2024-09-19T07:48:51.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.063+0000 7f593dccd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59380d0070 0x7f593814b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:51.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.063+0000 7f593dccd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59380d0070 0x7f593814b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60198/0 (socket says 172.21.15.16:60198) 2024-09-19T07:48:51.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.063+0000 7f593dccd640 1 -- 172.21.15.16:0/527512019 learned_addr learned my addr 172.21.15.16:0/527512019 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:51.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.063+0000 7f593e4ce640 1 --2- 172.21.15.16:0/527512019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f593815f0d0 0x7f59380b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:51.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.063+0000 7f593d4cc640 1 --2- 172.21.15.16:0/527512019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59380d1940 0x7f59380cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:51.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.063+0000 7f593dccd640 1 -- 172.21.15.16:0/527512019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59380d1940 msgr2=0x7f59380cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:51.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.063+0000 7f593dccd640 1 --2- 172.21.15.16:0/527512019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59380d1940 0x7f59380cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.063+0000 7f593dccd640 1 -- 172.21.15.16:0/527512019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f593815f0d0 msgr2=0x7f59380b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:51.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.063+0000 7f593dccd640 1 --2- 172.21.15.16:0/527512019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f593815f0d0 0x7f59380b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.063+0000 7f593dccd640 1 -- 172.21.15.16:0/527512019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59380ccd40 con 0x7f59380d0070 2024-09-19T07:48:51.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.064+0000 7f593dccd640 1 --2- 172.21.15.16:0/527512019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59380d0070 0x7f593814b860 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5934035040 tx=0x7f59340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:51.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.064+0000 7f592e7fc640 1 -- 172.21.15.16:0/527512019 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934041070 con 0x7f59380d0070 2024-09-19T07:48:51.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.064+0000 7f592e7fc640 1 -- 172.21.15.16:0/527512019 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f593403c460 con 0x7f59380d0070 2024-09-19T07:48:51.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.064+0000 7f592e7fc640 1 -- 172.21.15.16:0/527512019 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934037040 con 0x7f59380d0070 2024-09-19T07:48:51.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.064+0000 7f593ff31640 1 -- 172.21.15.16:0/527512019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59380cd020 con 0x7f59380d0070 2024-09-19T07:48:51.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.064+0000 7f593ff31640 1 -- 172.21.15.16:0/527512019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59380cd5e0 con 0x7f59380d0070 2024-09-19T07:48:51.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.065+0000 7f593ff31640 1 -- 172.21.15.16:0/527512019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59380d1400 con 0x7f59380d0070 2024-09-19T07:48:51.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.065+0000 7f592e7fc640 1 -- 172.21.15.16:0/527512019 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f593403a070 con 0x7f59380d0070 2024-09-19T07:48:51.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.066+0000 7f592e7fc640 1 --2- 172.21.15.16:0/527512019 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f591c06c2b0 0x7f591c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:51.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.066+0000 7f592e7fc640 1 -- 172.21.15.16:0/527512019 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6101+0+0 (secure 0 0 0) 0x7f59340ab940 con 0x7f59380d0070 2024-09-19T07:48:51.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.066+0000 7f593d4cc640 1 --2- 172.21.15.16:0/527512019 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f591c06c2b0 0x7f591c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:51.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.067+0000 7f593d4cc640 1 --2- 172.21.15.16:0/527512019 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f591c06c2b0 0x7f591c06e770 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f59380cb460 tx=0x7f5928008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:51.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.070+0000 7f592e7fc640 1 -- 172.21.15.16:0/527512019 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5934079e80 con 0x7f59380d0070 2024-09-19T07:48:51.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.247+0000 7f593ff31640 1 -- 172.21.15.16:0/527512019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f593806cf30 con 0x7f59380d0070 2024-09-19T07:48:51.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.248+0000 7f592e7fc640 1 -- 172.21.15.16:0/527512019 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16981 (secure 0 0 0) 0x7f5934079780 con 0x7f59380d0070 2024-09-19T07:48:51.248 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:51.249 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":69,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:48:49.954397+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:48:47.611316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-19T07:48:48.934687+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:48:51.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.250+0000 7f593ff31640 1 -- 172.21.15.16:0/527512019 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f591c06c2b0 msgr2=0x7f591c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:51.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.250+0000 7f593ff31640 1 --2- 172.21.15.16:0/527512019 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f591c06c2b0 0x7f591c06e770 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f59380cb460 tx=0x7f5928008040 comp rx=0 tx=0).stop 2024-09-19T07:48:51.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.250+0000 7f593ff31640 1 -- 172.21.15.16:0/527512019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59380d0070 msgr2=0x7f593814b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:51.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.250+0000 7f593ff31640 1 --2- 172.21.15.16:0/527512019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59380d0070 0x7f593814b860 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5934035040 tx=0x7f59340046c0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.250+0000 7f593ff31640 1 -- 172.21.15.16:0/527512019 shutdown_connections 2024-09-19T07:48:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.250+0000 7f593ff31640 1 --2- 172.21.15.16:0/527512019 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f591c06c2b0 0x7f591c06e770 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.250+0000 7f593ff31640 1 --2- 172.21.15.16:0/527512019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f593815f0d0 0x7f59380b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.250+0000 7f593ff31640 1 --2- 172.21.15.16:0/527512019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59380d1940 0x7f59380cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.250+0000 7f593ff31640 1 --2- 172.21.15.16:0/527512019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59380d0070 0x7f593814b860 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.250+0000 7f593ff31640 1 -- 172.21.15.16:0/527512019 >> 172.21.15.16:0/527512019 conn(0x7f593809d760 msgr2=0x7f59380d1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.250+0000 7f593ff31640 1 -- 172.21.15.16:0/527512019 shutdown_connections 2024-09-19T07:48:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.250+0000 7f593ff31640 1 -- 172.21.15.16:0/527512019 wait complete. 2024-09-19T07:48:51.261 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T07:48:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.367+0000 7f2e93ba6640 1 Processor -- start 2024-09-19T07:48:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.367+0000 7f2e93ba6640 1 -- start start 2024-09-19T07:48:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.368+0000 7f2e93ba6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2e8c15e680 0x7f2e8c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.368+0000 7f2e93ba6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2e8c0d0010 0x7f2e8c0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.368+0000 7f2e93ba6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e8c0d1880 con 0x7f2e8c15e2a0 2024-09-19T07:48:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.368+0000 7f2e93ba6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2e8c0d1b60 con 0x7f2e8c0d0010 2024-09-19T07:48:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.368+0000 7f2e93ba6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2e8c0cf2d0 con 0x7f2e8c15e680 2024-09-19T07:48:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.368+0000 7f2e92143640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2e8c0d0010 0x7f2e8c0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.368+0000 7f2e92143640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2e8c0d0010 0x7f2e8c0d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60208/0 (socket says 172.21.15.16:60208) 2024-09-19T07:48:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.368+0000 7f2e92143640 1 -- 172.21.15.16:0/761309088 learned_addr learned my addr 172.21.15.16:0/761309088 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.368+0000 7f2e91141640 1 --2- 172.21.15.16:0/761309088 >> v2:172.21.15.146:3300/0 conn(0x7f2e8c15e680 0x7f2e8c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.369+0000 7f2e92143640 1 -- 172.21.15.16:0/761309088 >> v2:172.21.15.146:3300/0 conn(0x7f2e8c15e680 msgr2=0x7f2e8c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.369+0000 7f2e92143640 1 --2- 172.21.15.16:0/761309088 >> v2:172.21.15.146:3300/0 conn(0x7f2e8c15e680 0x7f2e8c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.369+0000 7f2e92143640 1 -- 172.21.15.16:0/761309088 >> v1:172.21.15.146:6789/0 conn(0x7f2e8c15e2a0 legacy=0x7f2e8c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.369+0000 7f2e92143640 1 -- 172.21.15.16:0/761309088 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8c0cf5b0 con 0x7f2e8c0d0010 2024-09-19T07:48:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.369+0000 7f2e92143640 1 --2- 172.21.15.16:0/761309088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e8c0d0010 0x7f2e8c0d4370 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0a22a0 tx=0x7f2e8402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4a7c7eeb3fb7a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.370+0000 7f2e90940640 1 -- 172.21.15.16:0/761309088 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84037070 con 0x7f2e8c0d0010 2024-09-19T07:48:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.370+0000 7f2e90940640 1 -- 172.21.15.16:0/761309088 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e84032440 con 0x7f2e8c0d0010 2024-09-19T07:48:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.370+0000 7f2e90940640 1 -- 172.21.15.16:0/761309088 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84032810 con 0x7f2e8c0d0010 2024-09-19T07:48:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.370+0000 7f2e93ba6640 1 -- 172.21.15.16:0/761309088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e8c0d0010 msgr2=0x7f2e8c0d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.370+0000 7f2e93ba6640 1 --2- 172.21.15.16:0/761309088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e8c0d0010 0x7f2e8c0d4370 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0a22a0 tx=0x7f2e8402a1e0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.370+0000 7f2e93ba6640 1 -- 172.21.15.16:0/761309088 shutdown_connections 2024-09-19T07:48:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.370+0000 7f2e93ba6640 1 --2- 172.21.15.16:0/761309088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e8c0d0010 0x7f2e8c0d4370 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.370+0000 7f2e93ba6640 1 --2- 172.21.15.16:0/761309088 >> v2:172.21.15.146:3300/0 conn(0x7f2e8c15e680 0x7f2e8c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.370+0000 7f2e93ba6640 1 -- 172.21.15.16:0/761309088 >> 172.21.15.16:0/761309088 conn(0x7f2e8c09d760 msgr2=0x7f2e8c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.370+0000 7f2e93ba6640 1 -- 172.21.15.16:0/761309088 shutdown_connections 2024-09-19T07:48:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.370+0000 7f2e93ba6640 1 -- 172.21.15.16:0/761309088 wait complete. 2024-09-19T07:48:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.371+0000 7f2e93ba6640 1 Processor -- start 2024-09-19T07:48:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.371+0000 7f2e93ba6640 1 -- start start 2024-09-19T07:48:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.372+0000 7f2e93ba6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e8c0d0010 0x7f2e8c1528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.372+0000 7f2e93ba6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e8c15e2a0 0x7f2e8c0c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.372+0000 7f2e93ba6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e8c15e680 0x7f2e8c15fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.372+0000 7f2e93ba6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c06dcb0 con 0x7f2e8c15e680 2024-09-19T07:48:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.372+0000 7f2e93ba6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c06db30 con 0x7f2e8c0d0010 2024-09-19T07:48:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.372+0000 7f2e93ba6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2e8c0d48b0 con 0x7f2e8c15e2a0 2024-09-19T07:48:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.372+0000 7f2e92143640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e8c15e680 0x7f2e8c15fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.372+0000 7f2e92143640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e8c15e680 0x7f2e8c15fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60224/0 (socket says 172.21.15.16:60224) 2024-09-19T07:48:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.372+0000 7f2e92143640 1 -- 172.21.15.16:0/1992433193 learned_addr learned my addr 172.21.15.16:0/1992433193 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.373+0000 7f2e92143640 1 -- 172.21.15.16:0/1992433193 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e8c15e2a0 msgr2=0x7f2e8c0c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.373+0000 7f2e92143640 1 --2- 172.21.15.16:0/1992433193 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e8c15e2a0 0x7f2e8c0c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.373+0000 7f2e92143640 1 -- 172.21.15.16:0/1992433193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e8c0d0010 msgr2=0x7f2e8c1528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.373+0000 7f2e91942640 1 --2- 172.21.15.16:0/1992433193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e8c0d0010 0x7f2e8c1528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.373+0000 7f2e92143640 1 --2- 172.21.15.16:0/1992433193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e8c0d0010 0x7f2e8c1528f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.373+0000 7f2e92143640 1 -- 172.21.15.16:0/1992433193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8c0ca510 con 0x7f2e8c15e680 2024-09-19T07:48:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.373+0000 7f2e92143640 1 --2- 172.21.15.16:0/1992433193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e8c15e680 0x7f2e8c15fed0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2e840090b0 tx=0x7f2e840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.373+0000 7f2e8a7fc640 1 -- 172.21.15.16:0/1992433193 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84041020 con 0x7f2e8c15e680 2024-09-19T07:48:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.373+0000 7f2e8a7fc640 1 -- 172.21.15.16:0/1992433193 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e8403d640 con 0x7f2e8c15e680 2024-09-19T07:48:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.373+0000 7f2e93ba6640 1 -- 172.21.15.16:0/1992433193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e8c0ca7f0 con 0x7f2e8c15e680 2024-09-19T07:48:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.373+0000 7f2e8a7fc640 1 -- 172.21.15.16:0/1992433193 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84037040 con 0x7f2e8c15e680 2024-09-19T07:48:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.373+0000 7f2e93ba6640 1 -- 172.21.15.16:0/1992433193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e8c0cadb0 con 0x7f2e8c15e680 2024-09-19T07:48:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.374+0000 7f2e93ba6640 1 -- 172.21.15.16:0/1992433193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e8c0d4c70 con 0x7f2e8c15e680 2024-09-19T07:48:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.375+0000 7f2e8a7fc640 1 -- 172.21.15.16:0/1992433193 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2e84044020 con 0x7f2e8c15e680 2024-09-19T07:48:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.375+0000 7f2e8a7fc640 1 --2- 172.21.15.16:0/1992433193 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2e7806c2b0 0x7f2e7806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.375+0000 7f2e8a7fc640 1 -- 172.21.15.16:0/1992433193 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6101+0+0 (secure 0 0 0) 0x7f2e840abcd0 con 0x7f2e8c15e680 2024-09-19T07:48:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.376+0000 7f2e91942640 1 --2- 172.21.15.16:0/1992433193 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2e7806c2b0 0x7f2e7806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.376+0000 7f2e91942640 1 --2- 172.21.15.16:0/1992433193 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2e7806c2b0 0x7f2e7806e770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c007fd0 tx=0x7f2e7c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.379+0000 7f2e8a7fc640 1 -- 172.21.15.16:0/1992433193 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2e8404b030 con 0x7f2e8c15e680 2024-09-19T07:48:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.594+0000 7f2e93ba6640 1 -- 172.21.15.16:0/1992433193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2e8c06cf90 con 0x7f2e8c15e680 2024-09-19T07:48:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.606+0000 7f2e8a7fc640 1 -- 172.21.15.16:0/1992433193 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) ==== 118+0+0 (secure 0 0 0) 0x7f2e8407a2c0 con 0x7f2e8c15e680 2024-09-19T07:48:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.608+0000 7f2e93ba6640 1 -- 172.21.15.16:0/1992433193 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2e7806c2b0 msgr2=0x7f2e7806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.608+0000 7f2e93ba6640 1 --2- 172.21.15.16:0/1992433193 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2e7806c2b0 0x7f2e7806e770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c007fd0 tx=0x7f2e7c009040 comp rx=0 tx=0).stop 2024-09-19T07:48:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.608+0000 7f2e93ba6640 1 -- 172.21.15.16:0/1992433193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e8c15e680 msgr2=0x7f2e8c15fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.608+0000 7f2e93ba6640 1 --2- 172.21.15.16:0/1992433193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e8c15e680 0x7f2e8c15fed0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2e840090b0 tx=0x7f2e840046c0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.608+0000 7f2e93ba6640 1 -- 172.21.15.16:0/1992433193 shutdown_connections 2024-09-19T07:48:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.608+0000 7f2e93ba6640 1 --2- 172.21.15.16:0/1992433193 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2e7806c2b0 0x7f2e7806e770 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.608+0000 7f2e93ba6640 1 --2- 172.21.15.16:0/1992433193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e8c15e680 0x7f2e8c15fed0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.608+0000 7f2e93ba6640 1 --2- 172.21.15.16:0/1992433193 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e8c15e2a0 0x7f2e8c0c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.608+0000 7f2e93ba6640 1 --2- 172.21.15.16:0/1992433193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e8c0d0010 0x7f2e8c1528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.608+0000 7f2e93ba6640 1 -- 172.21.15.16:0/1992433193 >> 172.21.15.16:0/1992433193 conn(0x7f2e8c09d760 msgr2=0x7f2e8c161020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.608+0000 7f2e93ba6640 1 -- 172.21.15.16:0/1992433193 shutdown_connections 2024-09-19T07:48:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.608+0000 7f2e93ba6640 1 -- 172.21.15.16:0/1992433193 wait complete. 2024-09-19T07:48:51.621 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T07:48:51.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T07:48:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.728+0000 7fee8bf91640 1 Processor -- start 2024-09-19T07:48:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.728+0000 7fee8bf91640 1 -- start start 2024-09-19T07:48:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.729+0000 7fee8bf91640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fee8415f0d0 0x7fee8415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.729+0000 7fee8bf91640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee840d0070 0x7fee840d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.729+0000 7fee8bf91640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee840cf2b0 con 0x7fee840d1940 2024-09-19T07:48:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.729+0000 7fee8bf91640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fee840cf590 con 0x7fee8415f0d0 2024-09-19T07:48:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.729+0000 7fee8bf91640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fee840cf870 con 0x7fee840d0070 2024-09-19T07:48:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.729+0000 7fee8a52e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fee840d1940 0x7fee840d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38602/0 (socket says 172.21.15.16:38602) 2024-09-19T07:48:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.729+0000 7fee8a52e640 1 -- 172.21.15.16:0/2492906187 learned_addr learned my addr 172.21.15.16:0/2492906187 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.729+0000 7fee8952c640 1 --2- 172.21.15.16:0/2492906187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee840d0070 0x7fee840d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.729+0000 7fee89d2d640 1 --2- 172.21.15.16:0/2492906187 >> v2:172.21.15.146:3300/0 conn(0x7fee8415f0d0 0x7fee8415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:51.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.729+0000 7fee88d2b640 1 -- 172.21.15.16:0/2492906187 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2473576566 0 0) 0x7fee840cf2b0 con 0x7fee840d1940 2024-09-19T07:48:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.729+0000 7fee88d2b640 1 -- 172.21.15.16:0/2492906187 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee70003520 con 0x7fee840d1940 2024-09-19T07:48:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.730+0000 7fee8952c640 1 -- 172.21.15.16:0/2492906187 >> v2:172.21.15.146:3300/0 conn(0x7fee8415f0d0 msgr2=0x7fee8415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.730+0000 7fee8952c640 1 --2- 172.21.15.16:0/2492906187 >> v2:172.21.15.146:3300/0 conn(0x7fee8415f0d0 0x7fee8415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.730+0000 7fee8952c640 1 -- 172.21.15.16:0/2492906187 >> v1:172.21.15.16:6789/0 conn(0x7fee840d1940 legacy=0x7fee840d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.730+0000 7fee8952c640 1 -- 172.21.15.16:0/2492906187 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee840cfb50 con 0x7fee840d0070 2024-09-19T07:48:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.730+0000 7fee89d2d640 1 --2- 172.21.15.16:0/2492906187 >> v2:172.21.15.146:3300/0 conn(0x7fee8415f0d0 0x7fee8415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:48:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.730+0000 7fee8952c640 1 --2- 172.21.15.16:0/2492906187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee840d0070 0x7fee840d1400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fee740097a0 tx=0x7fee7402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9131f8e40ad0fe62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.731+0000 7fee88d2b640 1 -- 172.21.15.16:0/2492906187 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74037070 con 0x7fee840d0070 2024-09-19T07:48:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.731+0000 7fee88d2b640 1 -- 172.21.15.16:0/2492906187 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee7402ae50 con 0x7fee840d0070 2024-09-19T07:48:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.731+0000 7fee88d2b640 1 -- 172.21.15.16:0/2492906187 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74032640 con 0x7fee840d0070 2024-09-19T07:48:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.731+0000 7fee8bf91640 1 -- 172.21.15.16:0/2492906187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee840d0070 msgr2=0x7fee840d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.731+0000 7fee8bf91640 1 --2- 172.21.15.16:0/2492906187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee840d0070 0x7fee840d1400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fee740097a0 tx=0x7fee7402a100 comp rx=0 tx=0).stop 2024-09-19T07:48:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.731+0000 7fee8bf91640 1 -- 172.21.15.16:0/2492906187 shutdown_connections 2024-09-19T07:48:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.731+0000 7fee8bf91640 1 --2- 172.21.15.16:0/2492906187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee840d0070 0x7fee840d1400 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.731+0000 7fee8bf91640 1 --2- 172.21.15.16:0/2492906187 >> v2:172.21.15.146:3300/0 conn(0x7fee8415f0d0 0x7fee8415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.731+0000 7fee8bf91640 1 -- 172.21.15.16:0/2492906187 >> 172.21.15.16:0/2492906187 conn(0x7fee8409d760 msgr2=0x7fee8408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.731+0000 7fee8bf91640 1 -- 172.21.15.16:0/2492906187 shutdown_connections 2024-09-19T07:48:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.731+0000 7fee8bf91640 1 -- 172.21.15.16:0/2492906187 wait complete. 2024-09-19T07:48:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.732+0000 7fee8bf91640 1 Processor -- start 2024-09-19T07:48:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.733+0000 7fee8bf91640 1 -- start start 2024-09-19T07:48:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.733+0000 7fee8bf91640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee840d0070 0x7fee8409eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.733+0000 7fee8bf91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee840d1940 0x7fee840b9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.733+0000 7fee8bf91640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee8415f0d0 0x7fee84158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.733+0000 7fee8bf91640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee840d49c0 con 0x7fee840d1940 2024-09-19T07:48:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.733+0000 7fee8bf91640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fee840d4840 con 0x7fee8415f0d0 2024-09-19T07:48:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.733+0000 7fee8bf91640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fee840d4b40 con 0x7fee840d0070 2024-09-19T07:48:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.733+0000 7fee8952c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee840d1940 0x7fee840b9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.733+0000 7fee8952c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee840d1940 0x7fee840b9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60230/0 (socket says 172.21.15.16:60230) 2024-09-19T07:48:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.733+0000 7fee8952c640 1 -- 172.21.15.16:0/2402854056 learned_addr learned my addr 172.21.15.16:0/2402854056 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.734+0000 7fee89d2d640 1 --2- 172.21.15.16:0/2402854056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee840d0070 0x7fee8409eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.734+0000 7fee8a52e640 1 --2- 172.21.15.16:0/2402854056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee8415f0d0 0x7fee84158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.734+0000 7fee8952c640 1 -- 172.21.15.16:0/2402854056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee840d0070 msgr2=0x7fee8409eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.734+0000 7fee8952c640 1 --2- 172.21.15.16:0/2402854056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee840d0070 0x7fee8409eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.734+0000 7fee8952c640 1 -- 172.21.15.16:0/2402854056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee8415f0d0 msgr2=0x7fee84158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.734+0000 7fee8952c640 1 --2- 172.21.15.16:0/2402854056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee8415f0d0 0x7fee84158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.734+0000 7fee8952c640 1 -- 172.21.15.16:0/2402854056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8415f4b0 con 0x7fee840d1940 2024-09-19T07:48:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.734+0000 7fee89d2d640 1 --2- 172.21.15.16:0/2402854056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee840d0070 0x7fee8409eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.734+0000 7fee8a52e640 1 --2- 172.21.15.16:0/2402854056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee8415f0d0 0x7fee84158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.734+0000 7fee8952c640 1 --2- 172.21.15.16:0/2402854056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee840d1940 0x7fee840b9e50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fee7402a140 tx=0x7fee740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.734+0000 7fee7a7fc640 1 -- 172.21.15.16:0/2402854056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74041070 con 0x7fee840d1940 2024-09-19T07:48:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.734+0000 7fee7a7fc640 1 -- 172.21.15.16:0/2402854056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee7403c460 con 0x7fee840d1940 2024-09-19T07:48:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.734+0000 7fee8bf91640 1 -- 172.21.15.16:0/2402854056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8409f0c0 con 0x7fee840d1940 2024-09-19T07:48:51.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.734+0000 7fee7a7fc640 1 -- 172.21.15.16:0/2402854056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74037040 con 0x7fee840d1940 2024-09-19T07:48:51.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.734+0000 7fee8bf91640 1 -- 172.21.15.16:0/2402854056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee8415fa50 con 0x7fee840d1940 2024-09-19T07:48:51.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.735+0000 7fee8bf91640 1 -- 172.21.15.16:0/2402854056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee840d1400 con 0x7fee840d1940 2024-09-19T07:48:51.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.736+0000 7fee7a7fc640 1 -- 172.21.15.16:0/2402854056 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fee74044070 con 0x7fee840d1940 2024-09-19T07:48:51.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.737+0000 7fee7a7fc640 1 --2- 172.21.15.16:0/2402854056 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fee6806c2b0 0x7fee6806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:51.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.737+0000 7fee7a7fc640 1 -- 172.21.15.16:0/2402854056 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7fee740ab8e0 con 0x7fee840d1940 2024-09-19T07:48:51.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.738+0000 7fee89d2d640 1 --2- 172.21.15.16:0/2402854056 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fee6806c2b0 0x7fee6806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:51.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.739+0000 7fee89d2d640 1 --2- 172.21.15.16:0/2402854056 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fee6806c2b0 0x7fee6806e770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fee80009ba0 tx=0x7fee80008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.741+0000 7fee7a7fc640 1 -- 172.21.15.16:0/2402854056 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fee74079f50 con 0x7fee840d1940 2024-09-19T07:48:51.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:51.956+0000 7fee8bf91640 1 -- 172.21.15.16:0/2402854056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fee8406cf30 con 0x7fee840d1940 2024-09-19T07:48:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.612+0000 7fee7a7fc640 1 -- 172.21.15.16:0/2402854056 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) ==== 122+0+0 (secure 0 0 0) 0x7fee74079850 con 0x7fee840d1940 2024-09-19T07:48:52.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.615+0000 7fee8bf91640 1 -- 172.21.15.16:0/2402854056 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fee6806c2b0 msgr2=0x7fee6806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:52.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.615+0000 7fee8bf91640 1 --2- 172.21.15.16:0/2402854056 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fee6806c2b0 0x7fee6806e770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fee80009ba0 tx=0x7fee80008040 comp rx=0 tx=0).stop 2024-09-19T07:48:52.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.615+0000 7fee8bf91640 1 -- 172.21.15.16:0/2402854056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee840d1940 msgr2=0x7fee840b9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:52.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.615+0000 7fee8bf91640 1 --2- 172.21.15.16:0/2402854056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee840d1940 0x7fee840b9e50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fee7402a140 tx=0x7fee740046c0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.615+0000 7fee8bf91640 1 -- 172.21.15.16:0/2402854056 shutdown_connections 2024-09-19T07:48:52.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.615+0000 7fee8bf91640 1 --2- 172.21.15.16:0/2402854056 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fee6806c2b0 0x7fee6806e770 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.615+0000 7fee8bf91640 1 --2- 172.21.15.16:0/2402854056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee8415f0d0 0x7fee84158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.615+0000 7fee8bf91640 1 --2- 172.21.15.16:0/2402854056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee840d1940 0x7fee840b9e50 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.615+0000 7fee8bf91640 1 --2- 172.21.15.16:0/2402854056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee840d0070 0x7fee8409eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.615+0000 7fee8bf91640 1 -- 172.21.15.16:0/2402854056 >> 172.21.15.16:0/2402854056 conn(0x7fee8409d760 msgr2=0x7fee8408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:52.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.615+0000 7fee8bf91640 1 -- 172.21.15.16:0/2402854056 shutdown_connections 2024-09-19T07:48:52.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.616+0000 7fee8bf91640 1 -- 172.21.15.16:0/2402854056 wait complete. 2024-09-19T07:48:52.631 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T07:48:52.631 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:48:52.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.740+0000 7f0e0bba6640 1 Processor -- start 2024-09-19T07:48:52.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.740+0000 7f0e0bba6640 1 -- start start 2024-09-19T07:48:52.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.741+0000 7f0e0bba6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e0415bee0 0x7f0e040d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.741+0000 7f0e0bba6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0e040cf910 0x7f0e040d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.741+0000 7f0e0bba6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e040cdee0 con 0x7f0e0415c2c0 2024-09-19T07:48:52.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.741+0000 7f0e0bba6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f0e040ceaf0 con 0x7f0e040cf910 2024-09-19T07:48:52.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.741+0000 7f0e0bba6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0e040ce1c0 con 0x7f0e0415bee0 2024-09-19T07:48:52.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.741+0000 7f0e09942640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e0415bee0 0x7f0e040d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.741+0000 7f0e0a143640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0e040cf910 0x7f0e040d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.741+0000 7f0e09141640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f0e0415c2c0 0x7f0e040cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:53074/0 (socket says 172.21.15.16:53074) 2024-09-19T07:48:52.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.741+0000 7f0e09141640 1 -- 172.21.15.16:0/4099591394 learned_addr learned my addr 172.21.15.16:0/4099591394 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:52.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.742+0000 7f0e09942640 1 -- 172.21.15.16:0/4099591394 >> v2:172.21.15.146:3300/0 conn(0x7f0e040cf910 msgr2=0x7f0e040d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:52.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.742+0000 7f0e09942640 1 --2- 172.21.15.16:0/4099591394 >> v2:172.21.15.146:3300/0 conn(0x7f0e040cf910 0x7f0e040d09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.742+0000 7f0e09942640 1 -- 172.21.15.16:0/4099591394 >> v1:172.21.15.146:6789/0 conn(0x7f0e0415c2c0 legacy=0x7f0e040cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:52.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.742+0000 7f0e09942640 1 -- 172.21.15.16:0/4099591394 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e040ce4a0 con 0x7f0e0415bee0 2024-09-19T07:48:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.743+0000 7f0e09942640 1 --2- 172.21.15.16:0/4099591394 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e0415bee0 0x7f0e040d04b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0e000097a0 tx=0x7f0e0002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1967c349711674 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.743+0000 7f0e08940640 1 -- 172.21.15.16:0/4099591394 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00037070 con 0x7f0e0415bee0 2024-09-19T07:48:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.743+0000 7f0e08940640 1 -- 172.21.15.16:0/4099591394 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e00032440 con 0x7f0e0415bee0 2024-09-19T07:48:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.743+0000 7f0e08940640 1 -- 172.21.15.16:0/4099591394 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00032810 con 0x7f0e0415bee0 2024-09-19T07:48:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.743+0000 7f0e0bba6640 1 -- 172.21.15.16:0/4099591394 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e0415bee0 msgr2=0x7f0e040d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.743+0000 7f0e0bba6640 1 --2- 172.21.15.16:0/4099591394 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e0415bee0 0x7f0e040d04b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0e000097a0 tx=0x7f0e0002a180 comp rx=0 tx=0).stop 2024-09-19T07:48:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.743+0000 7f0e0bba6640 1 -- 172.21.15.16:0/4099591394 shutdown_connections 2024-09-19T07:48:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.743+0000 7f0e0bba6640 1 --2- 172.21.15.16:0/4099591394 >> v2:172.21.15.146:3300/0 conn(0x7f0e040cf910 0x7f0e040d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.743+0000 7f0e0bba6640 1 --2- 172.21.15.16:0/4099591394 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e0415bee0 0x7f0e040d04b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.743+0000 7f0e0bba6640 1 -- 172.21.15.16:0/4099591394 >> 172.21.15.16:0/4099591394 conn(0x7f0e0406bdf0 msgr2=0x7f0e0406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:52.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.743+0000 7f0e0bba6640 1 -- 172.21.15.16:0/4099591394 shutdown_connections 2024-09-19T07:48:52.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.743+0000 7f0e0bba6640 1 -- 172.21.15.16:0/4099591394 wait complete. 2024-09-19T07:48:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.745+0000 7f0e0bba6640 1 Processor -- start 2024-09-19T07:48:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.745+0000 7f0e0bba6640 1 -- start start 2024-09-19T07:48:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.745+0000 7f0e0bba6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e040cf910 0x7f0e04158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.745+0000 7f0e0bba6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e0415bee0 0x7f0e040cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.745+0000 7f0e0bba6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e0415c2c0 0x7f0e040cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.745+0000 7f0e0bba6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0406e040 con 0x7f0e040cf910 2024-09-19T07:48:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.745+0000 7f0e0bba6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0406dec0 con 0x7f0e0415c2c0 2024-09-19T07:48:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.745+0000 7f0e0bba6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0406e1c0 con 0x7f0e0415bee0 2024-09-19T07:48:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.745+0000 7f0e09942640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e040cf910 0x7f0e04158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.746+0000 7f0e09141640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e0415bee0 0x7f0e040cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.746+0000 7f0e09942640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e040cf910 0x7f0e04158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59324/0 (socket says 172.21.15.16:59324) 2024-09-19T07:48:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.746+0000 7f0e09942640 1 -- 172.21.15.16:0/1187302700 learned_addr learned my addr 172.21.15.16:0/1187302700 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.746+0000 7f0e0a143640 1 --2- 172.21.15.16:0/1187302700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e0415c2c0 0x7f0e040cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.746+0000 7f0e09942640 1 -- 172.21.15.16:0/1187302700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e0415bee0 msgr2=0x7f0e040cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.746+0000 7f0e09942640 1 --2- 172.21.15.16:0/1187302700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e0415bee0 0x7f0e040cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.746+0000 7f0e09942640 1 -- 172.21.15.16:0/1187302700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e0415c2c0 msgr2=0x7f0e040cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.746+0000 7f0e09942640 1 --2- 172.21.15.16:0/1187302700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e0415c2c0 0x7f0e040cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.746+0000 7f0e09942640 1 -- 172.21.15.16:0/1187302700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e04159310 con 0x7f0e040cf910 2024-09-19T07:48:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.746+0000 7f0e09141640 1 --2- 172.21.15.16:0/1187302700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e0415bee0 0x7f0e040cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:48:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.746+0000 7f0e09942640 1 --2- 172.21.15.16:0/1187302700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e040cf910 0x7f0e04158930 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0e00035040 tx=0x7f0e000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.746+0000 7f0dfa7fc640 1 -- 172.21.15.16:0/1187302700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00037040 con 0x7f0e040cf910 2024-09-19T07:48:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.746+0000 7f0dfa7fc640 1 -- 172.21.15.16:0/1187302700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e00030500 con 0x7f0e040cf910 2024-09-19T07:48:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.746+0000 7f0e0bba6640 1 -- 172.21.15.16:0/1187302700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e040c74e0 con 0x7f0e040cf910 2024-09-19T07:48:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.746+0000 7f0dfa7fc640 1 -- 172.21.15.16:0/1187302700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00030910 con 0x7f0e040cf910 2024-09-19T07:48:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.746+0000 7f0e0bba6640 1 -- 172.21.15.16:0/1187302700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e040cd880 con 0x7f0e040cf910 2024-09-19T07:48:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.747+0000 7f0e0bba6640 1 -- 172.21.15.16:0/1187302700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e040cf200 con 0x7f0e040cf910 2024-09-19T07:48:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.748+0000 7f0dfa7fc640 1 -- 172.21.15.16:0/1187302700 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0e00044020 con 0x7f0e040cf910 2024-09-19T07:48:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.748+0000 7f0dfa7fc640 1 --2- 172.21.15.16:0/1187302700 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0de806c310 0x7f0de806e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.748+0000 7f0dfa7fc640 1 -- 172.21.15.16:0/1187302700 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f0e000ab680 con 0x7f0e040cf910 2024-09-19T07:48:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.749+0000 7f0e09141640 1 --2- 172.21.15.16:0/1187302700 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0de806c310 0x7f0de806e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.749+0000 7f0e09141640 1 --2- 172.21.15.16:0/1187302700 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0de806c310 0x7f0de806e7d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0e04073e20 tx=0x7f0df4007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.753+0000 7f0dfa7fc640 1 -- 172.21.15.16:0/1187302700 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f0e00079600 con 0x7f0e040cf910 2024-09-19T07:48:52.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.962+0000 7f0e0bba6640 1 -- 172.21.15.16:0/1187302700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e040b4b30 con 0x7f0e040cf910 2024-09-19T07:48:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.963+0000 7f0dfa7fc640 1 -- 172.21.15.16:0/1187302700 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5227 (secure 0 0 0) 0x7f0e0007d900 con 0x7f0e040cf910 2024-09-19T07:48:52.963 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:52.963 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":33,"btime":"2024-09-19T07:48:52:612055+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2104039348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2104039348},{"type":"v1","addr":"172.21.15.16:6835","nonce":2104039348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/1109733621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1109733621},{"type":"v1","addr":"172.21.15.146:6836","nonce":1109733621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3602092462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3602092462},{"type":"v1","addr":"172.21.15.146:6837","nonce":3602092462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:48:49.954706+0000","modified":"2024-09-19T07:48:51.957927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5751},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5751":{"gid":5751,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6834/418904597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":418904597},{"type":"v1","addr":"172.21.15.16:6834","nonce":418904597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5751,"qdb_cluster":[5751]},"id":4}]} 2024-09-19T07:48:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 33 2024-09-19T07:48:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.964+0000 7f0e0bba6640 1 -- 172.21.15.16:0/1187302700 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0de806c310 msgr2=0x7f0de806e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.964+0000 7f0e0bba6640 1 --2- 172.21.15.16:0/1187302700 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0de806c310 0x7f0de806e7d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0e04073e20 tx=0x7f0df4007e80 comp rx=0 tx=0).stop 2024-09-19T07:48:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.964+0000 7f0e0bba6640 1 -- 172.21.15.16:0/1187302700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e040cf910 msgr2=0x7f0e04158930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.964+0000 7f0e0bba6640 1 --2- 172.21.15.16:0/1187302700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e040cf910 0x7f0e04158930 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0e00035040 tx=0x7f0e000046c0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.965+0000 7f0e0bba6640 1 -- 172.21.15.16:0/1187302700 shutdown_connections 2024-09-19T07:48:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.965+0000 7f0e0bba6640 1 --2- 172.21.15.16:0/1187302700 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0de806c310 0x7f0de806e7d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.965+0000 7f0e0bba6640 1 --2- 172.21.15.16:0/1187302700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e0415c2c0 0x7f0e040cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.965+0000 7f0e0bba6640 1 --2- 172.21.15.16:0/1187302700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e0415bee0 0x7f0e040cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.965+0000 7f0e0bba6640 1 --2- 172.21.15.16:0/1187302700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e040cf910 0x7f0e04158930 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.965+0000 7f0e0bba6640 1 -- 172.21.15.16:0/1187302700 >> 172.21.15.16:0/1187302700 conn(0x7f0e0406bdf0 msgr2=0x7f0e0415a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.965+0000 7f0e0bba6640 1 -- 172.21.15.16:0/1187302700 shutdown_connections 2024-09-19T07:48:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:52.965+0000 7f0e0bba6640 1 -- 172.21.15.16:0/1187302700 wait complete. 2024-09-19T07:48:52.976 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:48:53.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.078+0000 7fa4fbbb0640 1 Processor -- start 2024-09-19T07:48:53.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.078+0000 7fa4fbbb0640 1 -- start start 2024-09-19T07:48:53.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.079+0000 7fa4fbbb0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa4f4144100 0x7fa4f40d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.079+0000 7fa4fbbb0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4f40c7140 con 0x7fa4f406d430 2024-09-19T07:48:53.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.079+0000 7fa4fbbb0640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4f40c9100 con 0x7fa4f40c8d20 2024-09-19T07:48:53.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.079+0000 7fa4fbbb0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f4141960 con 0x7fa4f4144100 2024-09-19T07:48:53.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.079+0000 7fa4fa14d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa4f4144100 0x7fa4f40d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.079+0000 7fa4f914b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa4f406d430 0x7fa4f41439f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51198/0 (socket says 172.21.15.16:51198) 2024-09-19T07:48:53.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.079+0000 7fa4fa14d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa4f4144100 0x7fa4f40d1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59338/0 (socket says 172.21.15.16:59338) 2024-09-19T07:48:53.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.079+0000 7fa4f914b640 1 -- 172.21.15.16:0/3335950977 learned_addr learned my addr 172.21.15.16:0/3335950977 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:53.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.079+0000 7fa4fa14d640 1 -- 172.21.15.16:0/3335950977 >> v1:172.21.15.146:6789/0 conn(0x7fa4f40c8d20 legacy=0x7fa4f406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:53.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.079+0000 7fa4fa14d640 1 -- 172.21.15.16:0/3335950977 >> v1:172.21.15.16:6789/0 conn(0x7fa4f406d430 legacy=0x7fa4f41439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.079+0000 7fa4f894a640 1 -- 172.21.15.16:0/3335950977 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1320965817 0 0) 0x7fa4e8001110 con 0x7fa4f406d430 2024-09-19T07:48:53.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.080+0000 7fa4fa14d640 1 -- 172.21.15.16:0/3335950977 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f4144760 con 0x7fa4f4144100 2024-09-19T07:48:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.080+0000 7fa4fa14d640 1 --2- 172.21.15.16:0/3335950977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4144100 0x7fa4f40d1e50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4141ae0 tx=0x7fa4ec02a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f55545918ee24a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.080+0000 7fa4f894a640 1 -- 172.21.15.16:0/3335950977 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec037070 con 0x7fa4f4144100 2024-09-19T07:48:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.080+0000 7fa4f894a640 1 -- 172.21.15.16:0/3335950977 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ec02adb0 con 0x7fa4f4144100 2024-09-19T07:48:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.080+0000 7fa4f894a640 1 -- 172.21.15.16:0/3335950977 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec032660 con 0x7fa4f4144100 2024-09-19T07:48:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.081+0000 7fa4fbbb0640 1 -- 172.21.15.16:0/3335950977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4144100 msgr2=0x7fa4f40d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.081+0000 7fa4fbbb0640 1 --2- 172.21.15.16:0/3335950977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4144100 0x7fa4f40d1e50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4141ae0 tx=0x7fa4ec02a0a0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.081+0000 7fa4fbbb0640 1 -- 172.21.15.16:0/3335950977 shutdown_connections 2024-09-19T07:48:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.081+0000 7fa4fbbb0640 1 --2- 172.21.15.16:0/3335950977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4144100 0x7fa4f40d1e50 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.081+0000 7fa4fbbb0640 1 -- 172.21.15.16:0/3335950977 >> 172.21.15.16:0/3335950977 conn(0x7fa4f40b3ed0 msgr2=0x7fa4f40b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.081+0000 7fa4fbbb0640 1 -- 172.21.15.16:0/3335950977 shutdown_connections 2024-09-19T07:48:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.081+0000 7fa4fbbb0640 1 -- 172.21.15.16:0/3335950977 wait complete. 2024-09-19T07:48:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.082+0000 7fa4fbbb0640 1 Processor -- start 2024-09-19T07:48:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.082+0000 7fa4fbbb0640 1 -- start start 2024-09-19T07:48:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.083+0000 7fa4fbbb0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4f406d430 0x7fa4f40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.083+0000 7fa4fbbb0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4f40c8d20 0x7fa4f4140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.083+0000 7fa4fbbb0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4144100 0x7fa4f41408c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.083+0000 7fa4fbbb0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f4141e90 con 0x7fa4f4144100 2024-09-19T07:48:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.083+0000 7fa4fbbb0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f4142200 con 0x7fa4f406d430 2024-09-19T07:48:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.083+0000 7fa4fbbb0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f41436a0 con 0x7fa4f40c8d20 2024-09-19T07:48:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.083+0000 7fa4fa14d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4144100 0x7fa4f41408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.083+0000 7fa4fa14d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4144100 0x7fa4f41408c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59350/0 (socket says 172.21.15.16:59350) 2024-09-19T07:48:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.083+0000 7fa4fa14d640 1 -- 172.21.15.16:0/3679145702 learned_addr learned my addr 172.21.15.16:0/3679145702 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.083+0000 7fa4f914b640 1 --2- 172.21.15.16:0/3679145702 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4f40c8d20 0x7fa4f4140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.083+0000 7fa4f994c640 1 --2- 172.21.15.16:0/3679145702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4f406d430 0x7fa4f40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.083+0000 7fa4fa14d640 1 -- 172.21.15.16:0/3679145702 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4f40c8d20 msgr2=0x7fa4f4140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.083+0000 7fa4fa14d640 1 --2- 172.21.15.16:0/3679145702 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4f40c8d20 0x7fa4f4140380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.083+0000 7fa4fa14d640 1 -- 172.21.15.16:0/3679145702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4f406d430 msgr2=0x7fa4f40b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.083+0000 7fa4fa14d640 1 --2- 172.21.15.16:0/3679145702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4f406d430 0x7fa4f40b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.083+0000 7fa4fa14d640 1 -- 172.21.15.16:0/3679145702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f40d1e10 con 0x7fa4f4144100 2024-09-19T07:48:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.084+0000 7fa4fa14d640 1 --2- 172.21.15.16:0/3679145702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4144100 0x7fa4f41408c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009890 tx=0x7fa4ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.084+0000 7fa4f27fc640 1 -- 172.21.15.16:0/3679145702 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec042070 con 0x7fa4f4144100 2024-09-19T07:48:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.084+0000 7fa4f27fc640 1 -- 172.21.15.16:0/3679145702 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ec032e10 con 0x7fa4f4144100 2024-09-19T07:48:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.084+0000 7fa4fbbb0640 1 -- 172.21.15.16:0/3679145702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f40d20b0 con 0x7fa4f4144100 2024-09-19T07:48:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.084+0000 7fa4f27fc640 1 -- 172.21.15.16:0/3679145702 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec037040 con 0x7fa4f4144100 2024-09-19T07:48:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.084+0000 7fa4fbbb0640 1 -- 172.21.15.16:0/3679145702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f4140e00 con 0x7fa4f4144100 2024-09-19T07:48:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.085+0000 7fa4fbbb0640 1 -- 172.21.15.16:0/3679145702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f41439f0 con 0x7fa4f4144100 2024-09-19T07:48:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.085+0000 7fa4f27fc640 1 -- 172.21.15.16:0/3679145702 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa4ec045070 con 0x7fa4f4144100 2024-09-19T07:48:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.086+0000 7fa4f27fc640 1 --2- 172.21.15.16:0/3679145702 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa4e006c2b0 0x7fa4e006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.086+0000 7fa4f27fc640 1 -- 172.21.15.16:0/3679145702 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7fa4ec0abdc0 con 0x7fa4f4144100 2024-09-19T07:48:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.087+0000 7fa4f994c640 1 --2- 172.21.15.16:0/3679145702 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa4e006c2b0 0x7fa4e006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.087+0000 7fa4f994c640 1 --2- 172.21.15.16:0/3679145702 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa4e006c2b0 0x7fa4e006e770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400b450 tx=0x7fa4e400d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:53.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.090+0000 7fa4f27fc640 1 -- 172.21.15.16:0/3679145702 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa4ec079de0 con 0x7fa4f4144100 2024-09-19T07:48:53.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.267+0000 7fa4fbbb0640 1 -- 172.21.15.16:0/3679145702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4f40690c0 con 0x7fa4f4144100 2024-09-19T07:48:53.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.269+0000 7fa4f27fc640 1 -- 172.21.15.16:0/3679145702 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16968 (secure 0 0 0) 0x7fa4ec07e0e0 con 0x7fa4f4144100 2024-09-19T07:48:53.269 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:53.269 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":70,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:48:51.594413+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:48:47.611316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-19T07:48:48.934687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:48:53.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.271+0000 7fa4fbbb0640 1 -- 172.21.15.16:0/3679145702 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa4e006c2b0 msgr2=0x7fa4e006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.271+0000 7fa4fbbb0640 1 --2- 172.21.15.16:0/3679145702 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa4e006c2b0 0x7fa4e006e770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400b450 tx=0x7fa4e400d040 comp rx=0 tx=0).stop 2024-09-19T07:48:53.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.271+0000 7fa4fbbb0640 1 -- 172.21.15.16:0/3679145702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4144100 msgr2=0x7fa4f41408c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.271+0000 7fa4fbbb0640 1 --2- 172.21.15.16:0/3679145702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4144100 0x7fa4f41408c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009890 tx=0x7fa4ec0046c0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.271+0000 7fa4fbbb0640 1 -- 172.21.15.16:0/3679145702 shutdown_connections 2024-09-19T07:48:53.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.271+0000 7fa4fbbb0640 1 --2- 172.21.15.16:0/3679145702 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa4e006c2b0 0x7fa4e006e770 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.271+0000 7fa4fbbb0640 1 --2- 172.21.15.16:0/3679145702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4144100 0x7fa4f41408c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.271+0000 7fa4fbbb0640 1 --2- 172.21.15.16:0/3679145702 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4f40c8d20 0x7fa4f4140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.271+0000 7fa4fbbb0640 1 --2- 172.21.15.16:0/3679145702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4f406d430 0x7fa4f40b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.271+0000 7fa4fbbb0640 1 -- 172.21.15.16:0/3679145702 >> 172.21.15.16:0/3679145702 conn(0x7fa4f40b3ed0 msgr2=0x7fa4f41628a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:53.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.271+0000 7fa4fbbb0640 1 -- 172.21.15.16:0/3679145702 shutdown_connections 2024-09-19T07:48:53.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.271+0000 7fa4fbbb0640 1 -- 172.21.15.16:0/3679145702 wait complete. 2024-09-19T07:48:53.284 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T07:48:53.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:48:53.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.386+0000 7f98e4d2a640 1 Processor -- start 2024-09-19T07:48:53.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.386+0000 7f98e4d2a640 1 -- start start 2024-09-19T07:48:53.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.387+0000 7f98e4d2a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98e015b120 0x7f98e006f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.387+0000 7f98e4d2a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98e00d0bc0 0x7f98e006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.387+0000 7f98e4d2a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98e015b500 con 0x7f98e006dec0 2024-09-19T07:48:53.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.387+0000 7f98e4d2a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f98e006e940 con 0x7f98e015b120 2024-09-19T07:48:53.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.387+0000 7f98e4d2a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f98e006ec20 con 0x7f98e00d0bc0 2024-09-19T07:48:53.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.387+0000 7f98de59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98e015b120 0x7f98e006f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.387+0000 7f98de59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98e015b120 0x7f98e006f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59358/0 (socket says 172.21.15.16:59358) 2024-09-19T07:48:53.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.387+0000 7f98de59c640 1 -- 172.21.15.16:0/1557310533 learned_addr learned my addr 172.21.15.16:0/1557310533 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:53.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.387+0000 7f98de59c640 1 -- 172.21.15.16:0/1557310533 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98e00d0bc0 msgr2=0x7f98e006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:53.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.387+0000 7f98de59c640 1 --2- 172.21.15.16:0/1557310533 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98e00d0bc0 0x7f98e006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.387+0000 7f98de59c640 1 -- 172.21.15.16:0/1557310533 >> v1:172.21.15.146:6789/0 conn(0x7f98e006dec0 legacy=0x7f98e00d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:53.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.387+0000 7f98de59c640 1 -- 172.21.15.16:0/1557310533 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e006ef00 con 0x7f98e015b120 2024-09-19T07:48:53.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.388+0000 7f98de59c640 1 --2- 172.21.15.16:0/1557310533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e015b120 0x7f98e006f9e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009870 tx=0x7f98d002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd81ac72a2df19d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:53.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.388+0000 7f98dd59a640 1 -- 172.21.15.16:0/1557310533 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0037070 con 0x7f98e015b120 2024-09-19T07:48:53.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.388+0000 7f98dd59a640 1 -- 172.21.15.16:0/1557310533 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d0032440 con 0x7f98e015b120 2024-09-19T07:48:53.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.388+0000 7f98dd59a640 1 -- 172.21.15.16:0/1557310533 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0032810 con 0x7f98e015b120 2024-09-19T07:48:53.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.389+0000 7f98e4d2a640 1 -- 172.21.15.16:0/1557310533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e015b120 msgr2=0x7f98e006f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.389+0000 7f98e4d2a640 1 --2- 172.21.15.16:0/1557310533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e015b120 0x7f98e006f9e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009870 tx=0x7f98d002a230 comp rx=0 tx=0).stop 2024-09-19T07:48:53.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.389+0000 7f98e4d2a640 1 -- 172.21.15.16:0/1557310533 shutdown_connections 2024-09-19T07:48:53.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.389+0000 7f98e4d2a640 1 --2- 172.21.15.16:0/1557310533 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98e00d0bc0 0x7f98e006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.389+0000 7f98e4d2a640 1 --2- 172.21.15.16:0/1557310533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e015b120 0x7f98e006f9e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.389+0000 7f98e4d2a640 1 -- 172.21.15.16:0/1557310533 >> 172.21.15.16:0/1557310533 conn(0x7f98e006bdf0 msgr2=0x7f98e006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:53.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.389+0000 7f98e4d2a640 1 -- 172.21.15.16:0/1557310533 shutdown_connections 2024-09-19T07:48:53.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.389+0000 7f98e4d2a640 1 -- 172.21.15.16:0/1557310533 wait complete. 2024-09-19T07:48:53.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.390+0000 7f98e4d2a640 1 Processor -- start 2024-09-19T07:48:53.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.390+0000 7f98e4d2a640 1 -- start start 2024-09-19T07:48:53.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.391+0000 7f98e4d2a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98e006dec0 0x7f98e015a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.391+0000 7f98e4d2a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e00d0bc0 0x7f98e00b0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.391+0000 7f98e4d2a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98e015b120 0x7f98e01587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.391+0000 7f98e4d2a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98e00ce060 con 0x7f98e00d0bc0 2024-09-19T07:48:53.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.391+0000 7f98e4d2a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f98e006f4c0 con 0x7f98e006dec0 2024-09-19T07:48:53.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.391+0000 7f98e4d2a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f98e00c5160 con 0x7f98e015b120 2024-09-19T07:48:53.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.391+0000 7f98ddd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e00d0bc0 0x7f98e00b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.391+0000 7f98ddd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e00d0bc0 0x7f98e00b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59370/0 (socket says 172.21.15.16:59370) 2024-09-19T07:48:53.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.391+0000 7f98ddd9b640 1 -- 172.21.15.16:0/1050400188 learned_addr learned my addr 172.21.15.16:0/1050400188 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:53.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.391+0000 7f98de59c640 1 --2- 172.21.15.16:0/1050400188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98e006dec0 0x7f98e015a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.391+0000 7f98ded9d640 1 --2- 172.21.15.16:0/1050400188 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98e015b120 0x7f98e01587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.391+0000 7f98ddd9b640 1 -- 172.21.15.16:0/1050400188 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98e015b120 msgr2=0x7f98e01587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.391+0000 7f98ddd9b640 1 --2- 172.21.15.16:0/1050400188 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98e015b120 0x7f98e01587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.391+0000 7f98ddd9b640 1 -- 172.21.15.16:0/1050400188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98e006dec0 msgr2=0x7f98e015a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.391+0000 7f98ddd9b640 1 --2- 172.21.15.16:0/1050400188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98e006dec0 0x7f98e015a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.391+0000 7f98ddd9b640 1 -- 172.21.15.16:0/1050400188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e01591a0 con 0x7f98e00d0bc0 2024-09-19T07:48:53.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.392+0000 7f98ddd9b640 1 --2- 172.21.15.16:0/1050400188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e00d0bc0 0x7f98e00b0720 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f98d400c410 tx=0x7f98d400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:53.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.392+0000 7f98ceffd640 1 -- 172.21.15.16:0/1050400188 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4017070 con 0x7f98e00d0bc0 2024-09-19T07:48:53.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.392+0000 7f98ceffd640 1 -- 172.21.15.16:0/1050400188 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d40035f0 con 0x7f98e00d0bc0 2024-09-19T07:48:53.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.392+0000 7f98ceffd640 1 -- 172.21.15.16:0/1050400188 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4003a20 con 0x7f98e00d0bc0 2024-09-19T07:48:53.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.392+0000 7f98e4d2a640 1 -- 172.21.15.16:0/1050400188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e0159480 con 0x7f98e00d0bc0 2024-09-19T07:48:53.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.392+0000 7f98e4d2a640 1 -- 172.21.15.16:0/1050400188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e0159d20 con 0x7f98e00d0bc0 2024-09-19T07:48:53.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.393+0000 7f98e4d2a640 1 -- 172.21.15.16:0/1050400188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98e0159650 con 0x7f98e00d0bc0 2024-09-19T07:48:53.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.393+0000 7f98ceffd640 1 -- 172.21.15.16:0/1050400188 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f98d401a070 con 0x7f98e00d0bc0 2024-09-19T07:48:53.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.394+0000 7f98ceffd640 1 --2- 172.21.15.16:0/1050400188 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f98c006c2b0 0x7f98c006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.394+0000 7f98ceffd640 1 -- 172.21.15.16:0/1050400188 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f98d408a9b0 con 0x7f98e00d0bc0 2024-09-19T07:48:53.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.394+0000 7f98de59c640 1 --2- 172.21.15.16:0/1050400188 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f98c006c2b0 0x7f98c006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.395+0000 7f98de59c640 1 --2- 172.21.15.16:0/1050400188 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f98c006c2b0 0x7f98c006e770 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f98d0034000 tx=0x7f98d002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:53.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.398+0000 7f98ceffd640 1 -- 172.21.15.16:0/1050400188 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f98d4058940 con 0x7f98e00d0bc0 2024-09-19T07:48:53.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.569+0000 7f98e4d2a640 1 -- 172.21.15.16:0/1050400188 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f98e006e330 con 0x7f98c006c2b0 2024-09-19T07:48:53.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.576+0000 7f98ceffd640 1 -- 172.21.15.16:0/1050400188 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240077 (secure 0 0 0) 0x7f98e006e330 con 0x7f98c006c2b0 2024-09-19T07:48:53.580 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:53.582 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:48:53.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.579+0000 7f98e4d2a640 1 -- 172.21.15.16:0/1050400188 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f98c006c2b0 msgr2=0x7f98c006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.579+0000 7f98e4d2a640 1 --2- 172.21.15.16:0/1050400188 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f98c006c2b0 0x7f98c006e770 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f98d0034000 tx=0x7f98d002e000 comp rx=0 tx=0).stop 2024-09-19T07:48:53.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.579+0000 7f98e4d2a640 1 -- 172.21.15.16:0/1050400188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e00d0bc0 msgr2=0x7f98e00b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.579+0000 7f98e4d2a640 1 --2- 172.21.15.16:0/1050400188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e00d0bc0 0x7f98e00b0720 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f98d400c410 tx=0x7f98d400c8d0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.580+0000 7f98e4d2a640 1 -- 172.21.15.16:0/1050400188 shutdown_connections 2024-09-19T07:48:53.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.580+0000 7f98e4d2a640 1 --2- 172.21.15.16:0/1050400188 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f98c006c2b0 0x7f98c006e770 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.580+0000 7f98e4d2a640 1 --2- 172.21.15.16:0/1050400188 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98e015b120 0x7f98e01587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.580+0000 7f98e4d2a640 1 --2- 172.21.15.16:0/1050400188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e00d0bc0 0x7f98e00b0720 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.580+0000 7f98e4d2a640 1 --2- 172.21.15.16:0/1050400188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98e006dec0 0x7f98e015a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.580+0000 7f98e4d2a640 1 -- 172.21.15.16:0/1050400188 >> 172.21.15.16:0/1050400188 conn(0x7f98e006bdf0 msgr2=0x7f98e00ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:53.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.580+0000 7f98e4d2a640 1 -- 172.21.15.16:0/1050400188 shutdown_connections 2024-09-19T07:48:53.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.580+0000 7f98e4d2a640 1 -- 172.21.15.16:0/1050400188 wait complete. 2024-09-19T07:48:53.591 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-09-19T07:48:52.534136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":54,"acting":54,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":749484,"kb_used_data":7420,"kb_used_omap":12,"kb_used_meta":741875,"kb_avail":754225236,"statfs":{"total":773094113280,"available":772326641664,"internally_reserved":0,"allocated":7598080,"data_stored":4075545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":759680560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001045"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.610050+0000","last_change":"2024-09-19T07:48:50.609214+0000","last_active":"2024-09-19T07:48:50.610050+0000","last_peered":"2024-09-19T07:48:50.610050+0000","last_clean":"2024-09-19T07:48:50.610050+0000","last_became_active":"2024-09-19T07:48:50.608804+0000","last_became_peered":"2024-09-19T07:48:50.608804+0000","last_unstale":"2024-09-19T07:48:50.610050+0000","last_undegraded":"2024-09-19T07:48:50.610050+0000","last_fullsized":"2024-09-19T07:48:50.610050+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:31:31.683080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.607757+0000","last_change":"2024-09-19T07:48:50.604348+0000","last_active":"2024-09-19T07:48:50.607757+0000","last_peered":"2024-09-19T07:48:50.607757+0000","last_clean":"2024-09-19T07:48:50.607757+0000","last_became_active":"2024-09-19T07:48:50.604037+0000","last_became_peered":"2024-09-19T07:48:50.604037+0000","last_unstale":"2024-09-19T07:48:50.607757+0000","last_undegraded":"2024-09-19T07:48:50.607757+0000","last_fullsized":"2024-09-19T07:48:50.607757+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:16.200749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.591934+0000","last_change":"2024-09-19T07:48:48.591934+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.591934+0000","last_undegraded":"2024-09-19T07:48:48.591934+0000","last_fullsized":"2024-09-19T07:48:48.591934+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.592157+0000","last_change":"2024-09-19T07:48:48.592157+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.592157+0000","last_undegraded":"2024-09-19T07:48:48.592157+0000","last_fullsized":"2024-09-19T07:48:48.592157+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.591664+0000","last_change":"2024-09-19T07:48:48.591664+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.591664+0000","last_undegraded":"2024-09-19T07:48:48.591664+0000","last_fullsized":"2024-09-19T07:48:48.591664+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.609656+0000","last_change":"2024-09-19T07:48:50.607600+0000","last_active":"2024-09-19T07:48:50.609656+0000","last_peered":"2024-09-19T07:48:50.609656+0000","last_clean":"2024-09-19T07:48:50.609656+0000","last_became_active":"2024-09-19T07:48:50.606735+0000","last_became_peered":"2024-09-19T07:48:50.606735+0000","last_unstale":"2024-09-19T07:48:50.609656+0000","last_undegraded":"2024-09-19T07:48:50.609656+0000","last_fullsized":"2024-09-19T07:48:50.609656+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:11.159734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585139+0000","last_change":"2024-09-19T07:48:48.585139+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585139+0000","last_undegraded":"2024-09-19T07:48:48.585139+0000","last_fullsized":"2024-09-19T07:48:48.585139+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585324+0000","last_change":"2024-09-19T07:48:48.585324+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585324+0000","last_undegraded":"2024-09-19T07:48:48.585324+0000","last_fullsized":"2024-09-19T07:48:48.585324+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.583145+0000","last_change":"2024-09-19T07:48:48.583145+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.583145+0000","last_undegraded":"2024-09-19T07:48:48.583145+0000","last_fullsized":"2024-09-19T07:48:48.583145+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.587478+0000","last_change":"2024-09-19T07:48:48.587478+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.587478+0000","last_undegraded":"2024-09-19T07:48:48.587478+0000","last_fullsized":"2024-09-19T07:48:48.587478+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.587663+0000","last_change":"2024-09-19T07:48:48.587663+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.587663+0000","last_undegraded":"2024-09-19T07:48:48.587663+0000","last_fullsized":"2024-09-19T07:48:48.587663+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585431+0000","last_change":"2024-09-19T07:48:48.585431+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585431+0000","last_undegraded":"2024-09-19T07:48:48.585431+0000","last_fullsized":"2024-09-19T07:48:48.585431+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.586411+0000","last_change":"2024-09-19T07:48:48.586411+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.586411+0000","last_undegraded":"2024-09-19T07:48:48.586411+0000","last_fullsized":"2024-09-19T07:48:48.586411+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.583018+0000","last_change":"2024-09-19T07:48:48.583018+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.583018+0000","last_undegraded":"2024-09-19T07:48:48.583018+0000","last_fullsized":"2024-09-19T07:48:48.583018+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585897+0000","last_change":"2024-09-19T07:48:48.585897+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585897+0000","last_undegraded":"2024-09-19T07:48:48.585897+0000","last_fullsized":"2024-09-19T07:48:48.585897+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.587418+0000","last_change":"2024-09-19T07:48:48.587418+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.587418+0000","last_undegraded":"2024-09-19T07:48:48.587418+0000","last_fullsized":"2024-09-19T07:48:48.587418+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585058+0000","last_change":"2024-09-19T07:48:48.585058+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585058+0000","last_undegraded":"2024-09-19T07:48:48.585058+0000","last_fullsized":"2024-09-19T07:48:48.585058+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.591562+0000","last_change":"2024-09-19T07:48:48.591562+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.591562+0000","last_undegraded":"2024-09-19T07:48:48.591562+0000","last_fullsized":"2024-09-19T07:48:48.591562+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.584778+0000","last_change":"2024-09-19T07:48:48.584778+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.584778+0000","last_undegraded":"2024-09-19T07:48:48.584778+0000","last_fullsized":"2024-09-19T07:48:48.584778+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.591509+0000","last_change":"2024-09-19T07:48:48.591509+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.591509+0000","last_undegraded":"2024-09-19T07:48:48.591509+0000","last_fullsized":"2024-09-19T07:48:48.591509+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585145+0000","last_change":"2024-09-19T07:48:48.585145+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585145+0000","last_undegraded":"2024-09-19T07:48:48.585145+0000","last_fullsized":"2024-09-19T07:48:48.585145+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.622119+0000","last_change":"2024-09-19T07:48:50.613584+0000","last_active":"2024-09-19T07:48:50.622119+0000","last_peered":"2024-09-19T07:48:50.622119+0000","last_clean":"2024-09-19T07:48:50.622119+0000","last_became_active":"2024-09-19T07:48:50.612698+0000","last_became_peered":"2024-09-19T07:48:50.612698+0000","last_unstale":"2024-09-19T07:48:50.622119+0000","last_undegraded":"2024-09-19T07:48:50.622119+0000","last_fullsized":"2024-09-19T07:48:50.622119+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:51.388918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585906+0000","last_change":"2024-09-19T07:48:48.585906+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585906+0000","last_undegraded":"2024-09-19T07:48:48.585906+0000","last_fullsized":"2024-09-19T07:48:48.585906+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.591640+0000","last_change":"2024-09-19T07:48:48.591640+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.591640+0000","last_undegraded":"2024-09-19T07:48:48.591640+0000","last_fullsized":"2024-09-19T07:48:48.591640+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585154+0000","last_change":"2024-09-19T07:48:48.585154+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585154+0000","last_undegraded":"2024-09-19T07:48:48.585154+0000","last_fullsized":"2024-09-19T07:48:48.585154+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.608930+0000","last_change":"2024-09-19T07:48:50.604249+0000","last_active":"2024-09-19T07:48:50.608930+0000","last_peered":"2024-09-19T07:48:50.608930+0000","last_clean":"2024-09-19T07:48:50.608930+0000","last_became_active":"2024-09-19T07:48:50.603870+0000","last_became_peered":"2024-09-19T07:48:50.603870+0000","last_unstale":"2024-09-19T07:48:50.608930+0000","last_undegraded":"2024-09-19T07:48:50.608930+0000","last_fullsized":"2024-09-19T07:48:50.608930+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:17.668476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":253,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T07:48:48.588335+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:48:48.588335+0000","last_peered":"2024-09-19T07:48:48.588335+0000","last_clean":"2024-09-19T07:48:48.588335+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:48:48.588335+0000","last_undegraded":"2024-09-19T07:48:48.588335+0000","last_fullsized":"2024-09-19T07:48:48.588335+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":42,"acting":42,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509514,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94952,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":93566,"kb_avail":94276888,"statfs":{"total":96636764160,"available":96539533312,"internally_reserved":0,"allocated":1396736,"data_stored":952939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":95812042},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509513,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":112484,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94259356,"statfs":{"total":96636764160,"available":96521580544,"internally_reserved":0,"allocated":802816,"data_stored":362571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":114358726},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509514,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50020,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321820,"statfs":{"total":96636764160,"available":96585543680,"internally_reserved":0,"allocated":802816,"data_stored":362571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509514,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62500,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309340,"statfs":{"total":96636764160,"available":96572764160,"internally_reserved":0,"allocated":802816,"data_stored":362571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509513,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100524,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271316,"statfs":{"total":96636764160,"available":96533827584,"internally_reserved":0,"allocated":1396736,"data_stored":952939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542217,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":130460,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94241380,"statfs":{"total":96636764160,"available":96503173120,"internally_reserved":0,"allocated":790528,"data_stored":356362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542217,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":94428,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94277412,"statfs":{"total":96636764160,"available":96540069888,"internally_reserved":0,"allocated":790528,"data_stored":356362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104116,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267724,"statfs":{"total":96636764160,"available":96530149376,"internally_reserved":0,"allocated":815104,"data_stored":369230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:48:53.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:48:53.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.733+0000 7ff85f59f640 1 Processor -- start 2024-09-19T07:48:53.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.734+0000 7ff85f59f640 1 -- start start 2024-09-19T07:48:53.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.734+0000 7ff85f59f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff8580b03c0 0x7ff8580cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.734+0000 7ff85f59f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8580b3c10 con 0x7ff8580cc680 2024-09-19T07:48:53.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.734+0000 7ff85f59f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8580cb900 con 0x7ff85815e7f0 2024-09-19T07:48:53.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.734+0000 7ff85f59f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff8580cc1f0 con 0x7ff8580b03c0 2024-09-19T07:48:53.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff85db3c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff8580cc680 0x7ff85806bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51212/0 (socket says 172.21.15.16:51212) 2024-09-19T07:48:53.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff85db3c640 1 -- 172.21.15.16:0/1924778231 learned_addr learned my addr 172.21.15.16:0/1924778231 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:53.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff857fff640 1 -- 172.21.15.16:0/1924778231 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1637387135 0 0) 0x7ff8580b3c10 con 0x7ff8580cc680 2024-09-19T07:48:53.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff85cb3a640 1 --2- 172.21.15.16:0/1924778231 >> v2:172.21.15.146:3300/0 conn(0x7ff8580b03c0 0x7ff8580cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff857fff640 1 -- 172.21.15.16:0/1924778231 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff840003520 con 0x7ff8580cc680 2024-09-19T07:48:53.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff857fff640 1 -- 172.21.15.16:0/1924778231 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3566075432 0 0) 0x7ff840003520 con 0x7ff8580cc680 2024-09-19T07:48:53.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff857fff640 1 -- 172.21.15.16:0/1924778231 >> v2:172.21.15.146:3300/0 conn(0x7ff8580b03c0 msgr2=0x7ff8580cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:53.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff857fff640 1 --2- 172.21.15.16:0/1924778231 >> v2:172.21.15.146:3300/0 conn(0x7ff8580b03c0 0x7ff8580cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff857fff640 1 -- 172.21.15.16:0/1924778231 >> v1:172.21.15.146:6789/0 conn(0x7ff85815e7f0 legacy=0x7ff8580adff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff857fff640 1 -- 172.21.15.16:0/1924778231 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85815ebd0 con 0x7ff8580cc680 2024-09-19T07:48:53.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff857fff640 1 -- 172.21.15.16:0/1924778231 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7ff850001670 con 0x7ff8580cc680 2024-09-19T07:48:53.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff857fff640 1 -- 172.21.15.16:0/1924778231 >> v1:172.21.15.16:6789/0 conn(0x7ff8580cc680 legacy=0x7ff85806bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff857fff640 1 --2- 172.21.15.16:0/1924778231 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8400039b0 0x7ff840003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff857fff640 1 --2- 172.21.15.16:0/1924778231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff840004800 0x7ff840004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff857fff640 1 --2- 172.21.15.16:0/1924778231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8400051c0 0x7ff84000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff857fff640 1 -- 172.21.15.16:0/1924778231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8580cc1f0 con 0x7ff8400051c0 2024-09-19T07:48:53.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff857fff640 1 -- 172.21.15.16:0/1924778231 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff84000c160 con 0x7ff840004800 2024-09-19T07:48:53.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff857fff640 1 -- 172.21.15.16:0/1924778231 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff84000c360 con 0x7ff8400039b0 2024-09-19T07:48:53.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff85db3c640 1 --2- 172.21.15.16:0/1924778231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8400051c0 0x7ff84000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff85cb3a640 1 --2- 172.21.15.16:0/1924778231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff840004800 0x7ff840004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.735+0000 7ff85d33b640 1 --2- 172.21.15.16:0/1924778231 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8400039b0 0x7ff840003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.736+0000 7ff85db3c640 1 -- 172.21.15.16:0/1924778231 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8400039b0 msgr2=0x7ff840003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.736+0000 7ff85db3c640 1 --2- 172.21.15.16:0/1924778231 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8400039b0 0x7ff840003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.736+0000 7ff85db3c640 1 -- 172.21.15.16:0/1924778231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff840004800 msgr2=0x7ff840004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.736+0000 7ff85db3c640 1 --2- 172.21.15.16:0/1924778231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff840004800 0x7ff840004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.736+0000 7ff85db3c640 1 -- 172.21.15.16:0/1924778231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff840003520 con 0x7ff8400051c0 2024-09-19T07:48:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.736+0000 7ff85d33b640 1 --2- 172.21.15.16:0/1924778231 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8400039b0 0x7ff840003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.736+0000 7ff85db3c640 1 -- 172.21.15.16:0/1924778231 reap_dead start 2024-09-19T07:48:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.736+0000 7ff85db3c640 1 --2- 172.21.15.16:0/1924778231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8400051c0 0x7ff84000b9c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff85000c210 tx=0x7ff840005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=767edf0d2cd22432 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.736+0000 7ff857fff640 1 -- 172.21.15.16:0/1924778231 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850007d30 con 0x7ff8400051c0 2024-09-19T07:48:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.736+0000 7ff857fff640 1 -- 172.21.15.16:0/1924778231 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff850005af0 con 0x7ff8400051c0 2024-09-19T07:48:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.736+0000 7ff857fff640 1 -- 172.21.15.16:0/1924778231 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850007070 con 0x7ff8400051c0 2024-09-19T07:48:53.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.737+0000 7ff85f59f640 1 -- 172.21.15.16:0/1924778231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8400051c0 msgr2=0x7ff84000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.737+0000 7ff85f59f640 1 --2- 172.21.15.16:0/1924778231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8400051c0 0x7ff84000b9c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff85000c210 tx=0x7ff840005830 comp rx=0 tx=0).stop 2024-09-19T07:48:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.737+0000 7ff85f59f640 1 -- 172.21.15.16:0/1924778231 shutdown_connections 2024-09-19T07:48:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.737+0000 7ff85f59f640 1 --2- 172.21.15.16:0/1924778231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8400051c0 0x7ff84000b9c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.737+0000 7ff85f59f640 1 -- 172.21.15.16:0/1924778231 >> 172.21.15.16:0/1924778231 conn(0x7ff8580b2d10 msgr2=0x7ff8580b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.737+0000 7ff85f59f640 1 -- 172.21.15.16:0/1924778231 shutdown_connections 2024-09-19T07:48:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.737+0000 7ff85f59f640 1 -- 172.21.15.16:0/1924778231 wait complete. 2024-09-19T07:48:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.738+0000 7ff85f59f640 1 Processor -- start 2024-09-19T07:48:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.738+0000 7ff85f59f640 1 -- start start 2024-09-19T07:48:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.739+0000 7ff85f59f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8400051c0 0x7ff8581545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.739+0000 7ff85f59f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8580c9650 0x7ff8580c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.739+0000 7ff85f59f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8580c6920 0x7ff8580b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.739+0000 7ff85f59f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8580b3830 con 0x7ff8400051c0 2024-09-19T07:48:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.739+0000 7ff85f59f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff8580cbe80 con 0x7ff8580c6920 2024-09-19T07:48:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.739+0000 7ff85f59f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff8580b39b0 con 0x7ff8580c9650 2024-09-19T07:48:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.739+0000 7ff85d33b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8400051c0 0x7ff8581545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.739+0000 7ff85d33b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8400051c0 0x7ff8581545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59386/0 (socket says 172.21.15.16:59386) 2024-09-19T07:48:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.739+0000 7ff85d33b640 1 -- 172.21.15.16:0/4031965945 learned_addr learned my addr 172.21.15.16:0/4031965945 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.739+0000 7ff85db3c640 1 --2- 172.21.15.16:0/4031965945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8580c6920 0x7ff8580b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.739+0000 7ff85cb3a640 1 --2- 172.21.15.16:0/4031965945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8580c9650 0x7ff8580c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.739+0000 7ff85d33b640 1 -- 172.21.15.16:0/4031965945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8580c9650 msgr2=0x7ff8580c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.739+0000 7ff85d33b640 1 --2- 172.21.15.16:0/4031965945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8580c9650 0x7ff8580c63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.740+0000 7ff85d33b640 1 -- 172.21.15.16:0/4031965945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8580c6920 msgr2=0x7ff8580b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.740+0000 7ff85d33b640 1 --2- 172.21.15.16:0/4031965945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8580c6920 0x7ff8580b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.740+0000 7ff85d33b640 1 -- 172.21.15.16:0/4031965945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff858154c40 con 0x7ff8400051c0 2024-09-19T07:48:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.740+0000 7ff85db3c640 1 --2- 172.21.15.16:0/4031965945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8580c6920 0x7ff8580b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.740+0000 7ff85d33b640 1 --2- 172.21.15.16:0/4031965945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8400051c0 0x7ff8581545e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff84800e7f0 tx=0x7ff84800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.740+0000 7ff855ffb640 1 -- 172.21.15.16:0/4031965945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848003b80 con 0x7ff8400051c0 2024-09-19T07:48:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.740+0000 7ff855ffb640 1 -- 172.21.15.16:0/4031965945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff848018070 con 0x7ff8400051c0 2024-09-19T07:48:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.740+0000 7ff85f59f640 1 -- 172.21.15.16:0/4031965945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8580c9d00 con 0x7ff8400051c0 2024-09-19T07:48:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.740+0000 7ff855ffb640 1 -- 172.21.15.16:0/4031965945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8480050d0 con 0x7ff8400051c0 2024-09-19T07:48:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.740+0000 7ff85f59f640 1 -- 172.21.15.16:0/4031965945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8580ca2c0 con 0x7ff8400051c0 2024-09-19T07:48:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.741+0000 7ff85f59f640 1 -- 172.21.15.16:0/4031965945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8580cd360 con 0x7ff8400051c0 2024-09-19T07:48:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.741+0000 7ff855ffb640 1 -- 172.21.15.16:0/4031965945 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff848005270 con 0x7ff8400051c0 2024-09-19T07:48:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.742+0000 7ff855ffb640 1 --2- 172.21.15.16:0/4031965945 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff84406c2e0 0x7ff84406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.742+0000 7ff855ffb640 1 -- 172.21.15.16:0/4031965945 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7ff84808b540 con 0x7ff8400051c0 2024-09-19T07:48:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.742+0000 7ff85cb3a640 1 --2- 172.21.15.16:0/4031965945 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff84406c2e0 0x7ff84406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.743+0000 7ff85cb3a640 1 --2- 172.21.15.16:0/4031965945 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff84406c2e0 0x7ff84406e7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff8580b64f0 tx=0x7ff84c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:53.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.746+0000 7ff855ffb640 1 -- 172.21.15.16:0/4031965945 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff84801a310 con 0x7ff8400051c0 2024-09-19T07:48:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.920+0000 7ff85f59f640 1 -- 172.21.15.16:0/4031965945 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff8580c6f80 con 0x7ff84406c2e0 2024-09-19T07:48:53.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.926+0000 7ff855ffb640 1 -- 172.21.15.16:0/4031965945 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240077 (secure 0 0 0) 0x7ff8580c6f80 con 0x7ff84406c2e0 2024-09-19T07:48:53.929 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:53.930 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:48:53.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.930+0000 7ff85f59f640 1 -- 172.21.15.16:0/4031965945 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff84406c2e0 msgr2=0x7ff84406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.930+0000 7ff85f59f640 1 --2- 172.21.15.16:0/4031965945 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff84406c2e0 0x7ff84406e7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff8580b64f0 tx=0x7ff84c008040 comp rx=0 tx=0).stop 2024-09-19T07:48:53.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.930+0000 7ff85f59f640 1 -- 172.21.15.16:0/4031965945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8400051c0 msgr2=0x7ff8581545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.930+0000 7ff85f59f640 1 --2- 172.21.15.16:0/4031965945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8400051c0 0x7ff8581545e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff84800e7f0 tx=0x7ff84800ecb0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.930+0000 7ff85f59f640 1 -- 172.21.15.16:0/4031965945 shutdown_connections 2024-09-19T07:48:53.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.930+0000 7ff85f59f640 1 --2- 172.21.15.16:0/4031965945 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff84406c2e0 0x7ff84406e7a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.930+0000 7ff85f59f640 1 --2- 172.21.15.16:0/4031965945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8580c6920 0x7ff8580b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.930+0000 7ff85f59f640 1 --2- 172.21.15.16:0/4031965945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8580c9650 0x7ff8580c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.930+0000 7ff85f59f640 1 --2- 172.21.15.16:0/4031965945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8400051c0 0x7ff8581545e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.930+0000 7ff85f59f640 1 -- 172.21.15.16:0/4031965945 >> 172.21.15.16:0/4031965945 conn(0x7ff8580b2d10 msgr2=0x7ff85806ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.930+0000 7ff85f59f640 1 -- 172.21.15.16:0/4031965945 shutdown_connections 2024-09-19T07:48:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:53.930+0000 7ff85f59f640 1 -- 172.21.15.16:0/4031965945 wait complete. 2024-09-19T07:48:53.941 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-09-19T07:48:52.534136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":54,"acting":54,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":749484,"kb_used_data":7420,"kb_used_omap":12,"kb_used_meta":741875,"kb_avail":754225236,"statfs":{"total":773094113280,"available":772326641664,"internally_reserved":0,"allocated":7598080,"data_stored":4075545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":759680560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001045"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.610050+0000","last_change":"2024-09-19T07:48:50.609214+0000","last_active":"2024-09-19T07:48:50.610050+0000","last_peered":"2024-09-19T07:48:50.610050+0000","last_clean":"2024-09-19T07:48:50.610050+0000","last_became_active":"2024-09-19T07:48:50.608804+0000","last_became_peered":"2024-09-19T07:48:50.608804+0000","last_unstale":"2024-09-19T07:48:50.610050+0000","last_undegraded":"2024-09-19T07:48:50.610050+0000","last_fullsized":"2024-09-19T07:48:50.610050+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:31:31.683080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.607757+0000","last_change":"2024-09-19T07:48:50.604348+0000","last_active":"2024-09-19T07:48:50.607757+0000","last_peered":"2024-09-19T07:48:50.607757+0000","last_clean":"2024-09-19T07:48:50.607757+0000","last_became_active":"2024-09-19T07:48:50.604037+0000","last_became_peered":"2024-09-19T07:48:50.604037+0000","last_unstale":"2024-09-19T07:48:50.607757+0000","last_undegraded":"2024-09-19T07:48:50.607757+0000","last_fullsized":"2024-09-19T07:48:50.607757+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:16.200749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.591934+0000","last_change":"2024-09-19T07:48:48.591934+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.591934+0000","last_undegraded":"2024-09-19T07:48:48.591934+0000","last_fullsized":"2024-09-19T07:48:48.591934+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.592157+0000","last_change":"2024-09-19T07:48:48.592157+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.592157+0000","last_undegraded":"2024-09-19T07:48:48.592157+0000","last_fullsized":"2024-09-19T07:48:48.592157+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.591664+0000","last_change":"2024-09-19T07:48:48.591664+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.591664+0000","last_undegraded":"2024-09-19T07:48:48.591664+0000","last_fullsized":"2024-09-19T07:48:48.591664+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.609656+0000","last_change":"2024-09-19T07:48:50.607600+0000","last_active":"2024-09-19T07:48:50.609656+0000","last_peered":"2024-09-19T07:48:50.609656+0000","last_clean":"2024-09-19T07:48:50.609656+0000","last_became_active":"2024-09-19T07:48:50.606735+0000","last_became_peered":"2024-09-19T07:48:50.606735+0000","last_unstale":"2024-09-19T07:48:50.609656+0000","last_undegraded":"2024-09-19T07:48:50.609656+0000","last_fullsized":"2024-09-19T07:48:50.609656+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:11.159734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585139+0000","last_change":"2024-09-19T07:48:48.585139+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585139+0000","last_undegraded":"2024-09-19T07:48:48.585139+0000","last_fullsized":"2024-09-19T07:48:48.585139+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585324+0000","last_change":"2024-09-19T07:48:48.585324+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585324+0000","last_undegraded":"2024-09-19T07:48:48.585324+0000","last_fullsized":"2024-09-19T07:48:48.585324+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.583145+0000","last_change":"2024-09-19T07:48:48.583145+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.583145+0000","last_undegraded":"2024-09-19T07:48:48.583145+0000","last_fullsized":"2024-09-19T07:48:48.583145+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.587478+0000","last_change":"2024-09-19T07:48:48.587478+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.587478+0000","last_undegraded":"2024-09-19T07:48:48.587478+0000","last_fullsized":"2024-09-19T07:48:48.587478+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.587663+0000","last_change":"2024-09-19T07:48:48.587663+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.587663+0000","last_undegraded":"2024-09-19T07:48:48.587663+0000","last_fullsized":"2024-09-19T07:48:48.587663+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585431+0000","last_change":"2024-09-19T07:48:48.585431+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585431+0000","last_undegraded":"2024-09-19T07:48:48.585431+0000","last_fullsized":"2024-09-19T07:48:48.585431+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.586411+0000","last_change":"2024-09-19T07:48:48.586411+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.586411+0000","last_undegraded":"2024-09-19T07:48:48.586411+0000","last_fullsized":"2024-09-19T07:48:48.586411+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.583018+0000","last_change":"2024-09-19T07:48:48.583018+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.583018+0000","last_undegraded":"2024-09-19T07:48:48.583018+0000","last_fullsized":"2024-09-19T07:48:48.583018+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585897+0000","last_change":"2024-09-19T07:48:48.585897+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585897+0000","last_undegraded":"2024-09-19T07:48:48.585897+0000","last_fullsized":"2024-09-19T07:48:48.585897+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.587418+0000","last_change":"2024-09-19T07:48:48.587418+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.587418+0000","last_undegraded":"2024-09-19T07:48:48.587418+0000","last_fullsized":"2024-09-19T07:48:48.587418+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585058+0000","last_change":"2024-09-19T07:48:48.585058+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585058+0000","last_undegraded":"2024-09-19T07:48:48.585058+0000","last_fullsized":"2024-09-19T07:48:48.585058+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.591562+0000","last_change":"2024-09-19T07:48:48.591562+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.591562+0000","last_undegraded":"2024-09-19T07:48:48.591562+0000","last_fullsized":"2024-09-19T07:48:48.591562+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.584778+0000","last_change":"2024-09-19T07:48:48.584778+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.584778+0000","last_undegraded":"2024-09-19T07:48:48.584778+0000","last_fullsized":"2024-09-19T07:48:48.584778+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.591509+0000","last_change":"2024-09-19T07:48:48.591509+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.591509+0000","last_undegraded":"2024-09-19T07:48:48.591509+0000","last_fullsized":"2024-09-19T07:48:48.591509+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585145+0000","last_change":"2024-09-19T07:48:48.585145+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585145+0000","last_undegraded":"2024-09-19T07:48:48.585145+0000","last_fullsized":"2024-09-19T07:48:48.585145+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.622119+0000","last_change":"2024-09-19T07:48:50.613584+0000","last_active":"2024-09-19T07:48:50.622119+0000","last_peered":"2024-09-19T07:48:50.622119+0000","last_clean":"2024-09-19T07:48:50.622119+0000","last_became_active":"2024-09-19T07:48:50.612698+0000","last_became_peered":"2024-09-19T07:48:50.612698+0000","last_unstale":"2024-09-19T07:48:50.622119+0000","last_undegraded":"2024-09-19T07:48:50.622119+0000","last_fullsized":"2024-09-19T07:48:50.622119+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:51.388918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585906+0000","last_change":"2024-09-19T07:48:48.585906+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585906+0000","last_undegraded":"2024-09-19T07:48:48.585906+0000","last_fullsized":"2024-09-19T07:48:48.585906+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.591640+0000","last_change":"2024-09-19T07:48:48.591640+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.591640+0000","last_undegraded":"2024-09-19T07:48:48.591640+0000","last_fullsized":"2024-09-19T07:48:48.591640+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585154+0000","last_change":"2024-09-19T07:48:48.585154+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585154+0000","last_undegraded":"2024-09-19T07:48:48.585154+0000","last_fullsized":"2024-09-19T07:48:48.585154+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.608930+0000","last_change":"2024-09-19T07:48:50.604249+0000","last_active":"2024-09-19T07:48:50.608930+0000","last_peered":"2024-09-19T07:48:50.608930+0000","last_clean":"2024-09-19T07:48:50.608930+0000","last_became_active":"2024-09-19T07:48:50.603870+0000","last_became_peered":"2024-09-19T07:48:50.603870+0000","last_unstale":"2024-09-19T07:48:50.608930+0000","last_undegraded":"2024-09-19T07:48:50.608930+0000","last_fullsized":"2024-09-19T07:48:50.608930+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:17.668476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":253,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T07:48:48.588335+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:48:48.588335+0000","last_peered":"2024-09-19T07:48:48.588335+0000","last_clean":"2024-09-19T07:48:48.588335+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:48:48.588335+0000","last_undegraded":"2024-09-19T07:48:48.588335+0000","last_fullsized":"2024-09-19T07:48:48.588335+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":42,"acting":42,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509514,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94952,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":93566,"kb_avail":94276888,"statfs":{"total":96636764160,"available":96539533312,"internally_reserved":0,"allocated":1396736,"data_stored":952939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":95812042},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509513,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":112484,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94259356,"statfs":{"total":96636764160,"available":96521580544,"internally_reserved":0,"allocated":802816,"data_stored":362571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":114358726},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509514,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50020,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321820,"statfs":{"total":96636764160,"available":96585543680,"internally_reserved":0,"allocated":802816,"data_stored":362571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509514,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62500,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309340,"statfs":{"total":96636764160,"available":96572764160,"internally_reserved":0,"allocated":802816,"data_stored":362571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509513,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100524,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271316,"statfs":{"total":96636764160,"available":96533827584,"internally_reserved":0,"allocated":1396736,"data_stored":952939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542217,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":130460,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94241380,"statfs":{"total":96636764160,"available":96503173120,"internally_reserved":0,"allocated":790528,"data_stored":356362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542217,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":94428,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94277412,"statfs":{"total":96636764160,"available":96540069888,"internally_reserved":0,"allocated":790528,"data_stored":356362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104116,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267724,"statfs":{"total":96636764160,"available":96530149376,"internally_reserved":0,"allocated":815104,"data_stored":369230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:48:53.945 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:48:54.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.083+0000 7ff744723640 1 Processor -- start 2024-09-19T07:48:54.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.083+0000 7ff744723640 1 -- start start 2024-09-19T07:48:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.084+0000 7ff744723640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff73c15e680 0x7ff73c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.084+0000 7ff744723640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff73c0d0010 0x7ff73c0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.084+0000 7ff744723640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff73c0d1880 con 0x7ff73c15e2a0 2024-09-19T07:48:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.084+0000 7ff744723640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff73c0d1b60 con 0x7ff73c0d0010 2024-09-19T07:48:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.084+0000 7ff744723640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff73c0cf2d0 con 0x7ff73c15e680 2024-09-19T07:48:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.084+0000 7ff741cbe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff73c15e680 0x7ff73c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.084+0000 7ff742cc0640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff73c0d0010 0x7ff73c0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.084+0000 7ff7424bf640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ff73c15e2a0 0x7ff73c06d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:53120/0 (socket says 172.21.15.16:53120) 2024-09-19T07:48:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.084+0000 7ff7424bf640 1 -- 172.21.15.16:0/121827076 learned_addr learned my addr 172.21.15.16:0/121827076 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.085+0000 7ff742cc0640 1 -- 172.21.15.16:0/121827076 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff73c15e680 msgr2=0x7ff73c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.085+0000 7ff742cc0640 1 --2- 172.21.15.16:0/121827076 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff73c15e680 0x7ff73c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.085+0000 7ff742cc0640 1 -- 172.21.15.16:0/121827076 >> v1:172.21.15.146:6789/0 conn(0x7ff73c15e2a0 legacy=0x7ff73c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.085+0000 7ff742cc0640 1 -- 172.21.15.16:0/121827076 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73c0cf5b0 con 0x7ff73c0d0010 2024-09-19T07:48:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.085+0000 7ff741cbe640 1 --2- 172.21.15.16:0/121827076 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff73c15e680 0x7ff73c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:48:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.085+0000 7ff742cc0640 1 --2- 172.21.15.16:0/121827076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff73c0d0010 0x7ff73c0d4370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff734009d90 tx=0x7ff73402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e382b106a4613ae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.086+0000 7ff7414bd640 1 -- 172.21.15.16:0/121827076 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734031600 con 0x7ff73c0d0010 2024-09-19T07:48:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.086+0000 7ff7414bd640 1 -- 172.21.15.16:0/121827076 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7340317a0 con 0x7ff73c0d0010 2024-09-19T07:48:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.086+0000 7ff7414bd640 1 -- 172.21.15.16:0/121827076 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734031b70 con 0x7ff73c0d0010 2024-09-19T07:48:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.086+0000 7ff744723640 1 -- 172.21.15.16:0/121827076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff73c0d0010 msgr2=0x7ff73c0d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:54.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.086+0000 7ff744723640 1 --2- 172.21.15.16:0/121827076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff73c0d0010 0x7ff73c0d4370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff734009d90 tx=0x7ff73402a5d0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.086+0000 7ff744723640 1 -- 172.21.15.16:0/121827076 shutdown_connections 2024-09-19T07:48:54.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.086+0000 7ff744723640 1 --2- 172.21.15.16:0/121827076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff73c0d0010 0x7ff73c0d4370 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.086+0000 7ff744723640 1 --2- 172.21.15.16:0/121827076 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff73c15e680 0x7ff73c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.086+0000 7ff744723640 1 -- 172.21.15.16:0/121827076 >> 172.21.15.16:0/121827076 conn(0x7ff73c09d760 msgr2=0x7ff73c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:54.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.086+0000 7ff744723640 1 -- 172.21.15.16:0/121827076 shutdown_connections 2024-09-19T07:48:54.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.086+0000 7ff744723640 1 -- 172.21.15.16:0/121827076 wait complete. 2024-09-19T07:48:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.087+0000 7ff744723640 1 Processor -- start 2024-09-19T07:48:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.088+0000 7ff744723640 1 -- start start 2024-09-19T07:48:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.088+0000 7ff744723640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff73c0d0010 0x7ff73c1563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.088+0000 7ff744723640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff73c15e2a0 0x7ff73c0b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.088+0000 7ff744723640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff73c15e680 0x7ff73c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.088+0000 7ff744723640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c06dcb0 con 0x7ff73c15e680 2024-09-19T07:48:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.088+0000 7ff744723640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c06db30 con 0x7ff73c15e2a0 2024-09-19T07:48:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.088+0000 7ff744723640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff73c0d48b0 con 0x7ff73c0d0010 2024-09-19T07:48:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.088+0000 7ff742cc0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff73c15e680 0x7ff73c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.088+0000 7ff742cc0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff73c15e680 0x7ff73c0b8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59392/0 (socket says 172.21.15.16:59392) 2024-09-19T07:48:54.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.088+0000 7ff742cc0640 1 -- 172.21.15.16:0/2700360783 learned_addr learned my addr 172.21.15.16:0/2700360783 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:54.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.089+0000 7ff7424bf640 1 --2- 172.21.15.16:0/2700360783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff73c0d0010 0x7ff73c1563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:54.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.089+0000 7ff741cbe640 1 --2- 172.21.15.16:0/2700360783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff73c15e2a0 0x7ff73c0b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:54.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.089+0000 7ff742cc0640 1 -- 172.21.15.16:0/2700360783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff73c0d0010 msgr2=0x7ff73c1563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:54.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.089+0000 7ff742cc0640 1 --2- 172.21.15.16:0/2700360783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff73c0d0010 0x7ff73c1563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.089+0000 7ff742cc0640 1 -- 172.21.15.16:0/2700360783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff73c15e2a0 msgr2=0x7ff73c0b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:54.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.089+0000 7ff742cc0640 1 --2- 172.21.15.16:0/2700360783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff73c15e2a0 0x7ff73c0b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.089+0000 7ff742cc0640 1 -- 172.21.15.16:0/2700360783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73c1568e0 con 0x7ff73c15e680 2024-09-19T07:48:54.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.089+0000 7ff741cbe640 1 --2- 172.21.15.16:0/2700360783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff73c15e2a0 0x7ff73c0b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:54.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.089+0000 7ff742cc0640 1 --2- 172.21.15.16:0/2700360783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff73c15e680 0x7ff73c0b8a70 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff734009eb0 tx=0x7ff734004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:54.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.089+0000 7ff732ffd640 1 -- 172.21.15.16:0/2700360783 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734040070 con 0x7ff73c15e680 2024-09-19T07:48:54.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.089+0000 7ff732ffd640 1 -- 172.21.15.16:0/2700360783 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff734030b60 con 0x7ff73c15e680 2024-09-19T07:48:54.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.089+0000 7ff744723640 1 -- 172.21.15.16:0/2700360783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff73c1e7f50 con 0x7ff73c15e680 2024-09-19T07:48:54.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.089+0000 7ff732ffd640 1 -- 172.21.15.16:0/2700360783 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734009450 con 0x7ff73c15e680 2024-09-19T07:48:54.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.089+0000 7ff744723640 1 -- 172.21.15.16:0/2700360783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff73c1e8510 con 0x7ff73c15e680 2024-09-19T07:48:54.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.090+0000 7ff744723640 1 -- 172.21.15.16:0/2700360783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff73c0d4c70 con 0x7ff73c15e680 2024-09-19T07:48:54.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.091+0000 7ff732ffd640 1 -- 172.21.15.16:0/2700360783 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff734039040 con 0x7ff73c15e680 2024-09-19T07:48:54.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.091+0000 7ff732ffd640 1 --2- 172.21.15.16:0/2700360783 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff72006c2b0 0x7ff72006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:54.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.091+0000 7ff732ffd640 1 -- 172.21.15.16:0/2700360783 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7ff7340acbd0 con 0x7ff73c15e680 2024-09-19T07:48:54.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.091+0000 7ff7424bf640 1 --2- 172.21.15.16:0/2700360783 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff72006c2b0 0x7ff72006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:54.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.092+0000 7ff7424bf640 1 --2- 172.21.15.16:0/2700360783 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff72006c2b0 0x7ff72006e770 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff73c15a3a0 tx=0x7ff738007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:54.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.096+0000 7ff732ffd640 1 -- 172.21.15.16:0/2700360783 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff73407b240 con 0x7ff73c15e680 2024-09-19T07:48:54.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.269+0000 7ff744723640 1 -- 172.21.15.16:0/2700360783 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff73c0b8fb0 con 0x7ff72006c2b0 2024-09-19T07:48:54.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.275+0000 7ff732ffd640 1 -- 172.21.15.16:0/2700360783 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240077 (secure 0 0 0) 0x7ff73c0b8fb0 con 0x7ff72006c2b0 2024-09-19T07:48:54.278 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:54.280 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:48:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.279+0000 7ff744723640 1 -- 172.21.15.16:0/2700360783 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff72006c2b0 msgr2=0x7ff72006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.279+0000 7ff744723640 1 --2- 172.21.15.16:0/2700360783 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff72006c2b0 0x7ff72006e770 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff73c15a3a0 tx=0x7ff738007e80 comp rx=0 tx=0).stop 2024-09-19T07:48:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.279+0000 7ff744723640 1 -- 172.21.15.16:0/2700360783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff73c15e680 msgr2=0x7ff73c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.279+0000 7ff744723640 1 --2- 172.21.15.16:0/2700360783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff73c15e680 0x7ff73c0b8a70 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff734009eb0 tx=0x7ff734004a40 comp rx=0 tx=0).stop 2024-09-19T07:48:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.279+0000 7ff744723640 1 -- 172.21.15.16:0/2700360783 shutdown_connections 2024-09-19T07:48:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.279+0000 7ff744723640 1 --2- 172.21.15.16:0/2700360783 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff72006c2b0 0x7ff72006e770 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.279+0000 7ff744723640 1 --2- 172.21.15.16:0/2700360783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff73c15e680 0x7ff73c0b8a70 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.279+0000 7ff744723640 1 --2- 172.21.15.16:0/2700360783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff73c15e2a0 0x7ff73c0b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.279+0000 7ff744723640 1 --2- 172.21.15.16:0/2700360783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff73c0d0010 0x7ff73c1563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.279+0000 7ff744723640 1 -- 172.21.15.16:0/2700360783 >> 172.21.15.16:0/2700360783 conn(0x7ff73c09d760 msgr2=0x7ff73c0d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.279+0000 7ff744723640 1 -- 172.21.15.16:0/2700360783 shutdown_connections 2024-09-19T07:48:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.279+0000 7ff744723640 1 -- 172.21.15.16:0/2700360783 wait complete. 2024-09-19T07:48:54.290 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-09-19T07:48:52.534136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":54,"acting":54,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":749484,"kb_used_data":7420,"kb_used_omap":12,"kb_used_meta":741875,"kb_avail":754225236,"statfs":{"total":773094113280,"available":772326641664,"internally_reserved":0,"allocated":7598080,"data_stored":4075545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":759680560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001045"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.610050+0000","last_change":"2024-09-19T07:48:50.609214+0000","last_active":"2024-09-19T07:48:50.610050+0000","last_peered":"2024-09-19T07:48:50.610050+0000","last_clean":"2024-09-19T07:48:50.610050+0000","last_became_active":"2024-09-19T07:48:50.608804+0000","last_became_peered":"2024-09-19T07:48:50.608804+0000","last_unstale":"2024-09-19T07:48:50.610050+0000","last_undegraded":"2024-09-19T07:48:50.610050+0000","last_fullsized":"2024-09-19T07:48:50.610050+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:31:31.683080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.607757+0000","last_change":"2024-09-19T07:48:50.604348+0000","last_active":"2024-09-19T07:48:50.607757+0000","last_peered":"2024-09-19T07:48:50.607757+0000","last_clean":"2024-09-19T07:48:50.607757+0000","last_became_active":"2024-09-19T07:48:50.604037+0000","last_became_peered":"2024-09-19T07:48:50.604037+0000","last_unstale":"2024-09-19T07:48:50.607757+0000","last_undegraded":"2024-09-19T07:48:50.607757+0000","last_fullsized":"2024-09-19T07:48:50.607757+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:16.200749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.591934+0000","last_change":"2024-09-19T07:48:48.591934+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.591934+0000","last_undegraded":"2024-09-19T07:48:48.591934+0000","last_fullsized":"2024-09-19T07:48:48.591934+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.592157+0000","last_change":"2024-09-19T07:48:48.592157+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.592157+0000","last_undegraded":"2024-09-19T07:48:48.592157+0000","last_fullsized":"2024-09-19T07:48:48.592157+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.591664+0000","last_change":"2024-09-19T07:48:48.591664+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.591664+0000","last_undegraded":"2024-09-19T07:48:48.591664+0000","last_fullsized":"2024-09-19T07:48:48.591664+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.609656+0000","last_change":"2024-09-19T07:48:50.607600+0000","last_active":"2024-09-19T07:48:50.609656+0000","last_peered":"2024-09-19T07:48:50.609656+0000","last_clean":"2024-09-19T07:48:50.609656+0000","last_became_active":"2024-09-19T07:48:50.606735+0000","last_became_peered":"2024-09-19T07:48:50.606735+0000","last_unstale":"2024-09-19T07:48:50.609656+0000","last_undegraded":"2024-09-19T07:48:50.609656+0000","last_fullsized":"2024-09-19T07:48:50.609656+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:11.159734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585139+0000","last_change":"2024-09-19T07:48:48.585139+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585139+0000","last_undegraded":"2024-09-19T07:48:48.585139+0000","last_fullsized":"2024-09-19T07:48:48.585139+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585324+0000","last_change":"2024-09-19T07:48:48.585324+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585324+0000","last_undegraded":"2024-09-19T07:48:48.585324+0000","last_fullsized":"2024-09-19T07:48:48.585324+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.583145+0000","last_change":"2024-09-19T07:48:48.583145+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.583145+0000","last_undegraded":"2024-09-19T07:48:48.583145+0000","last_fullsized":"2024-09-19T07:48:48.583145+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.587478+0000","last_change":"2024-09-19T07:48:48.587478+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.587478+0000","last_undegraded":"2024-09-19T07:48:48.587478+0000","last_fullsized":"2024-09-19T07:48:48.587478+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.587663+0000","last_change":"2024-09-19T07:48:48.587663+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.587663+0000","last_undegraded":"2024-09-19T07:48:48.587663+0000","last_fullsized":"2024-09-19T07:48:48.587663+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585431+0000","last_change":"2024-09-19T07:48:48.585431+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585431+0000","last_undegraded":"2024-09-19T07:48:48.585431+0000","last_fullsized":"2024-09-19T07:48:48.585431+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.586411+0000","last_change":"2024-09-19T07:48:48.586411+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.586411+0000","last_undegraded":"2024-09-19T07:48:48.586411+0000","last_fullsized":"2024-09-19T07:48:48.586411+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.583018+0000","last_change":"2024-09-19T07:48:48.583018+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.583018+0000","last_undegraded":"2024-09-19T07:48:48.583018+0000","last_fullsized":"2024-09-19T07:48:48.583018+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585897+0000","last_change":"2024-09-19T07:48:48.585897+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585897+0000","last_undegraded":"2024-09-19T07:48:48.585897+0000","last_fullsized":"2024-09-19T07:48:48.585897+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.587418+0000","last_change":"2024-09-19T07:48:48.587418+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.587418+0000","last_undegraded":"2024-09-19T07:48:48.587418+0000","last_fullsized":"2024-09-19T07:48:48.587418+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585058+0000","last_change":"2024-09-19T07:48:48.585058+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585058+0000","last_undegraded":"2024-09-19T07:48:48.585058+0000","last_fullsized":"2024-09-19T07:48:48.585058+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.591562+0000","last_change":"2024-09-19T07:48:48.591562+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.591562+0000","last_undegraded":"2024-09-19T07:48:48.591562+0000","last_fullsized":"2024-09-19T07:48:48.591562+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.584778+0000","last_change":"2024-09-19T07:48:48.584778+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.584778+0000","last_undegraded":"2024-09-19T07:48:48.584778+0000","last_fullsized":"2024-09-19T07:48:48.584778+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.591509+0000","last_change":"2024-09-19T07:48:48.591509+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.591509+0000","last_undegraded":"2024-09-19T07:48:48.591509+0000","last_fullsized":"2024-09-19T07:48:48.591509+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585145+0000","last_change":"2024-09-19T07:48:48.585145+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585145+0000","last_undegraded":"2024-09-19T07:48:48.585145+0000","last_fullsized":"2024-09-19T07:48:48.585145+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.622119+0000","last_change":"2024-09-19T07:48:50.613584+0000","last_active":"2024-09-19T07:48:50.622119+0000","last_peered":"2024-09-19T07:48:50.622119+0000","last_clean":"2024-09-19T07:48:50.622119+0000","last_became_active":"2024-09-19T07:48:50.612698+0000","last_became_peered":"2024-09-19T07:48:50.612698+0000","last_unstale":"2024-09-19T07:48:50.622119+0000","last_undegraded":"2024-09-19T07:48:50.622119+0000","last_fullsized":"2024-09-19T07:48:50.622119+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:51.388918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585906+0000","last_change":"2024-09-19T07:48:48.585906+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585906+0000","last_undegraded":"2024-09-19T07:48:48.585906+0000","last_fullsized":"2024-09-19T07:48:48.585906+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.591640+0000","last_change":"2024-09-19T07:48:48.591640+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.591640+0000","last_undegraded":"2024-09-19T07:48:48.591640+0000","last_fullsized":"2024-09-19T07:48:48.591640+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-19T07:48:48.585154+0000","last_change":"2024-09-19T07:48:48.585154+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.585154+0000","last_undegraded":"2024-09-19T07:48:48.585154+0000","last_fullsized":"2024-09-19T07:48:48.585154+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.608930+0000","last_change":"2024-09-19T07:48:50.604249+0000","last_active":"2024-09-19T07:48:50.608930+0000","last_peered":"2024-09-19T07:48:50.608930+0000","last_clean":"2024-09-19T07:48:50.608930+0000","last_became_active":"2024-09-19T07:48:50.603870+0000","last_became_peered":"2024-09-19T07:48:50.603870+0000","last_unstale":"2024-09-19T07:48:50.608930+0000","last_undegraded":"2024-09-19T07:48:50.608930+0000","last_fullsized":"2024-09-19T07:48:50.608930+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:17.668476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:49.577543+0000","last_change":"2024-09-19T07:48:49.577543+0000","last_active":"2024-09-19T07:48:49.577543+0000","last_peered":"2024-09-19T07:48:49.577543+0000","last_clean":"2024-09-19T07:48:49.577543+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:49.577543+0000","last_undegraded":"2024-09-19T07:48:49.577543+0000","last_fullsized":"2024-09-19T07:48:49.577543+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:48:48.573172+0000","last_change":"2024-09-19T07:48:48.573172+0000","last_active":"2024-09-19T07:48:48.573172+0000","last_peered":"2024-09-19T07:48:48.573172+0000","last_clean":"2024-09-19T07:48:48.573172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:48:48.573172+0000","last_undegraded":"2024-09-19T07:48:48.573172+0000","last_fullsized":"2024-09-19T07:48:48.573172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":253,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T07:48:48.588335+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:48:48.588335+0000","last_peered":"2024-09-19T07:48:48.588335+0000","last_clean":"2024-09-19T07:48:48.588335+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:48:48.588335+0000","last_undegraded":"2024-09-19T07:48:48.588335+0000","last_fullsized":"2024-09-19T07:48:48.588335+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":42,"acting":42,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509514,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94952,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":93566,"kb_avail":94276888,"statfs":{"total":96636764160,"available":96539533312,"internally_reserved":0,"allocated":1396736,"data_stored":952939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":95812042},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509513,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":112484,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94259356,"statfs":{"total":96636764160,"available":96521580544,"internally_reserved":0,"allocated":802816,"data_stored":362571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":114358726},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509514,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50020,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321820,"statfs":{"total":96636764160,"available":96585543680,"internally_reserved":0,"allocated":802816,"data_stored":362571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509514,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62500,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309340,"statfs":{"total":96636764160,"available":96572764160,"internally_reserved":0,"allocated":802816,"data_stored":362571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509513,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100524,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271316,"statfs":{"total":96636764160,"available":96533827584,"internally_reserved":0,"allocated":1396736,"data_stored":952939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542217,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":130460,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94241380,"statfs":{"total":96636764160,"available":96503173120,"internally_reserved":0,"allocated":790528,"data_stored":356362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542217,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":94428,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94277412,"statfs":{"total":96636764160,"available":96540069888,"internally_reserved":0,"allocated":790528,"data_stored":356362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104116,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267724,"statfs":{"total":96636764160,"available":96530149376,"internally_reserved":0,"allocated":815104,"data_stored":369230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:48:54.294 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3f is not active+clean 2024-09-19T07:48:54.294 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.294 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3e is not active+clean 2024-09-19T07:48:54.294 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.294 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3d is not active+clean 2024-09-19T07:48:54.294 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.294 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3c is not active+clean 2024-09-19T07:48:54.294 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.295 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3b is not active+clean 2024-09-19T07:48:54.295 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.295 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3a is not active+clean 2024-09-19T07:48:54.295 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.295 INFO:tasks.ceph.ceph_manager.ceph:PG 10.38 is not active+clean 2024-09-19T07:48:54.295 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.295 INFO:tasks.ceph.ceph_manager.ceph:PG 10.37 is not active+clean 2024-09-19T07:48:54.295 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.295 INFO:tasks.ceph.ceph_manager.ceph:PG 10.36 is not active+clean 2024-09-19T07:48:54.295 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.295 INFO:tasks.ceph.ceph_manager.ceph:PG 10.35 is not active+clean 2024-09-19T07:48:54.296 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.296 INFO:tasks.ceph.ceph_manager.ceph:PG 10.34 is not active+clean 2024-09-19T07:48:54.296 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.296 INFO:tasks.ceph.ceph_manager.ceph:PG 10.33 is not active+clean 2024-09-19T07:48:54.296 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.296 INFO:tasks.ceph.ceph_manager.ceph:PG 10.32 is not active+clean 2024-09-19T07:48:54.296 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.296 INFO:tasks.ceph.ceph_manager.ceph:PG 10.31 is not active+clean 2024-09-19T07:48:54.296 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.296 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2f is not active+clean 2024-09-19T07:48:54.297 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.297 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2e is not active+clean 2024-09-19T07:48:54.297 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.297 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2d is not active+clean 2024-09-19T07:48:54.297 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.297 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2c is not active+clean 2024-09-19T07:48:54.297 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.297 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2b is not active+clean 2024-09-19T07:48:54.297 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.297 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2a is not active+clean 2024-09-19T07:48:54.297 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.298 INFO:tasks.ceph.ceph_manager.ceph:PG 10.29 is not active+clean 2024-09-19T07:48:54.298 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.298 INFO:tasks.ceph.ceph_manager.ceph:PG 10.28 is not active+clean 2024-09-19T07:48:54.298 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.298 INFO:tasks.ceph.ceph_manager.ceph:PG 10.27 is not active+clean 2024-09-19T07:48:54.298 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.298 INFO:tasks.ceph.ceph_manager.ceph:PG 10.26 is not active+clean 2024-09-19T07:48:54.298 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.298 INFO:tasks.ceph.ceph_manager.ceph:PG 10.25 is not active+clean 2024-09-19T07:48:54.298 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.298 INFO:tasks.ceph.ceph_manager.ceph:PG 10.24 is not active+clean 2024-09-19T07:48:54.299 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.299 INFO:tasks.ceph.ceph_manager.ceph:PG 10.23 is not active+clean 2024-09-19T07:48:54.299 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.299 INFO:tasks.ceph.ceph_manager.ceph:PG 10.22 is not active+clean 2024-09-19T07:48:54.299 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.299 INFO:tasks.ceph.ceph_manager.ceph:PG 10.21 is not active+clean 2024-09-19T07:48:54.299 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.299 INFO:tasks.ceph.ceph_manager.ceph:PG 10.20 is not active+clean 2024-09-19T07:48:54.299 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.299 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1c is not active+clean 2024-09-19T07:48:54.299 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.300 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1f is not active+clean 2024-09-19T07:48:54.300 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:48.573172+0000', 'last_change': '2024-09-19T07:48:48.573172+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.573172+0000', 'last_undegraded': '2024-09-19T07:48:48.573172+0000', 'last_fullsized': '2024-09-19T07:48:48.573172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.300 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1d is not active+clean 2024-09-19T07:48:54.300 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.300 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1e is not active+clean 2024-09-19T07:48:54.300 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.591934+0000', 'last_change': '2024-09-19T07:48:48.591934+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.591934+0000', 'last_undegraded': '2024-09-19T07:48:48.591934+0000', 'last_fullsized': '2024-09-19T07:48:48.591934+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T07:48:54.300 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1e is not active+clean 2024-09-19T07:48:54.300 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.300 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1d is not active+clean 2024-09-19T07:48:54.300 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.592157+0000', 'last_change': '2024-09-19T07:48:48.592157+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.592157+0000', 'last_undegraded': '2024-09-19T07:48:48.592157+0000', 'last_fullsized': '2024-09-19T07:48:48.592157+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T07:48:54.300 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1f is not active+clean 2024-09-19T07:48:54.301 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.301 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1c is not active+clean 2024-09-19T07:48:54.301 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.591664+0000', 'last_change': '2024-09-19T07:48:48.591664+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.591664+0000', 'last_undegraded': '2024-09-19T07:48:48.591664+0000', 'last_fullsized': '2024-09-19T07:48:48.591664+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T07:48:54.301 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1b is not active+clean 2024-09-19T07:48:54.301 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:48.573172+0000', 'last_change': '2024-09-19T07:48:48.573172+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.573172+0000', 'last_undegraded': '2024-09-19T07:48:48.573172+0000', 'last_fullsized': '2024-09-19T07:48:48.573172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.301 INFO:tasks.ceph.ceph_manager.ceph:PG 10.19 is not active+clean 2024-09-19T07:48:54.301 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.301 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1a is not active+clean 2024-09-19T07:48:54.301 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:48.573172+0000', 'last_change': '2024-09-19T07:48:48.573172+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.573172+0000', 'last_undegraded': '2024-09-19T07:48:48.573172+0000', 'last_fullsized': '2024-09-19T07:48:48.573172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.301 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1a is not active+clean 2024-09-19T07:48:54.301 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.302 INFO:tasks.ceph.ceph_manager.ceph:PG 9.19 is not active+clean 2024-09-19T07:48:54.302 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:48.573172+0000', 'last_change': '2024-09-19T07:48:48.573172+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.573172+0000', 'last_undegraded': '2024-09-19T07:48:48.573172+0000', 'last_fullsized': '2024-09-19T07:48:48.573172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.302 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1b is not active+clean 2024-09-19T07:48:54.302 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.302 INFO:tasks.ceph.ceph_manager.ceph:PG 9.18 is not active+clean 2024-09-19T07:48:54.302 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.585139+0000', 'last_change': '2024-09-19T07:48:48.585139+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.585139+0000', 'last_undegraded': '2024-09-19T07:48:48.585139+0000', 'last_fullsized': '2024-09-19T07:48:48.585139+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T07:48:54.302 INFO:tasks.ceph.ceph_manager.ceph:PG 10.14 is not active+clean 2024-09-19T07:48:54.302 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.302 INFO:tasks.ceph.ceph_manager.ceph:PG 9.17 is not active+clean 2024-09-19T07:48:54.302 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.585324+0000', 'last_change': '2024-09-19T07:48:48.585324+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.585324+0000', 'last_undegraded': '2024-09-19T07:48:48.585324+0000', 'last_fullsized': '2024-09-19T07:48:48.585324+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T07:48:54.303 INFO:tasks.ceph.ceph_manager.ceph:PG 10.15 is not active+clean 2024-09-19T07:48:54.303 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.303 INFO:tasks.ceph.ceph_manager.ceph:PG 9.16 is not active+clean 2024-09-19T07:48:54.303 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.583145+0000', 'last_change': '2024-09-19T07:48:48.583145+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.583145+0000', 'last_undegraded': '2024-09-19T07:48:48.583145+0000', 'last_fullsized': '2024-09-19T07:48:48.583145+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T07:48:54.303 INFO:tasks.ceph.ceph_manager.ceph:PG 10.16 is not active+clean 2024-09-19T07:48:54.303 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.303 INFO:tasks.ceph.ceph_manager.ceph:PG 9.15 is not active+clean 2024-09-19T07:48:54.303 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.587478+0000', 'last_change': '2024-09-19T07:48:48.587478+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.587478+0000', 'last_undegraded': '2024-09-19T07:48:48.587478+0000', 'last_fullsized': '2024-09-19T07:48:48.587478+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T07:48:54.303 INFO:tasks.ceph.ceph_manager.ceph:PG 10.17 is not active+clean 2024-09-19T07:48:54.303 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.303 INFO:tasks.ceph.ceph_manager.ceph:PG 9.14 is not active+clean 2024-09-19T07:48:54.303 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:48.573172+0000', 'last_change': '2024-09-19T07:48:48.573172+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.573172+0000', 'last_undegraded': '2024-09-19T07:48:48.573172+0000', 'last_fullsized': '2024-09-19T07:48:48.573172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.304 INFO:tasks.ceph.ceph_manager.ceph:PG 10.10 is not active+clean 2024-09-19T07:48:54.304 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.304 INFO:tasks.ceph.ceph_manager.ceph:PG 9.13 is not active+clean 2024-09-19T07:48:54.304 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.587663+0000', 'last_change': '2024-09-19T07:48:48.587663+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.587663+0000', 'last_undegraded': '2024-09-19T07:48:48.587663+0000', 'last_fullsized': '2024-09-19T07:48:48.587663+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T07:48:54.304 INFO:tasks.ceph.ceph_manager.ceph:PG 10.11 is not active+clean 2024-09-19T07:48:54.304 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.304 INFO:tasks.ceph.ceph_manager.ceph:PG 9.12 is not active+clean 2024-09-19T07:48:54.304 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.585431+0000', 'last_change': '2024-09-19T07:48:48.585431+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.585431+0000', 'last_undegraded': '2024-09-19T07:48:48.585431+0000', 'last_fullsized': '2024-09-19T07:48:48.585431+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T07:48:54.304 INFO:tasks.ceph.ceph_manager.ceph:PG 10.12 is not active+clean 2024-09-19T07:48:54.304 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.305 INFO:tasks.ceph.ceph_manager.ceph:PG 9.11 is not active+clean 2024-09-19T07:48:54.305 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.586411+0000', 'last_change': '2024-09-19T07:48:48.586411+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.586411+0000', 'last_undegraded': '2024-09-19T07:48:48.586411+0000', 'last_fullsized': '2024-09-19T07:48:48.586411+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T07:48:54.305 INFO:tasks.ceph.ceph_manager.ceph:PG 10.13 is not active+clean 2024-09-19T07:48:54.305 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.305 INFO:tasks.ceph.ceph_manager.ceph:PG 9.10 is not active+clean 2024-09-19T07:48:54.305 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.583018+0000', 'last_change': '2024-09-19T07:48:48.583018+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.583018+0000', 'last_undegraded': '2024-09-19T07:48:48.583018+0000', 'last_fullsized': '2024-09-19T07:48:48.583018+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T07:48:54.305 INFO:tasks.ceph.ceph_manager.ceph:PG 10.c is not active+clean 2024-09-19T07:48:54.305 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.305 INFO:tasks.ceph.ceph_manager.ceph:PG 9.f is not active+clean 2024-09-19T07:48:54.305 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.585897+0000', 'last_change': '2024-09-19T07:48:48.585897+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.585897+0000', 'last_undegraded': '2024-09-19T07:48:48.585897+0000', 'last_fullsized': '2024-09-19T07:48:48.585897+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T07:48:54.305 INFO:tasks.ceph.ceph_manager.ceph:PG 10.d is not active+clean 2024-09-19T07:48:54.306 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.306 INFO:tasks.ceph.ceph_manager.ceph:PG 9.e is not active+clean 2024-09-19T07:48:54.306 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.587418+0000', 'last_change': '2024-09-19T07:48:48.587418+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.587418+0000', 'last_undegraded': '2024-09-19T07:48:48.587418+0000', 'last_fullsized': '2024-09-19T07:48:48.587418+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T07:48:54.306 INFO:tasks.ceph.ceph_manager.ceph:PG 10.e is not active+clean 2024-09-19T07:48:54.306 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.306 INFO:tasks.ceph.ceph_manager.ceph:PG 9.d is not active+clean 2024-09-19T07:48:54.306 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.585058+0000', 'last_change': '2024-09-19T07:48:48.585058+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.585058+0000', 'last_undegraded': '2024-09-19T07:48:48.585058+0000', 'last_fullsized': '2024-09-19T07:48:48.585058+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T07:48:54.306 INFO:tasks.ceph.ceph_manager.ceph:PG 10.f is not active+clean 2024-09-19T07:48:54.306 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.306 INFO:tasks.ceph.ceph_manager.ceph:PG 9.c is not active+clean 2024-09-19T07:48:54.306 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:48.573172+0000', 'last_change': '2024-09-19T07:48:48.573172+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.573172+0000', 'last_undegraded': '2024-09-19T07:48:48.573172+0000', 'last_fullsized': '2024-09-19T07:48:48.573172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.307 INFO:tasks.ceph.ceph_manager.ceph:PG 10.8 is not active+clean 2024-09-19T07:48:54.307 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.307 INFO:tasks.ceph.ceph_manager.ceph:PG 9.b is not active+clean 2024-09-19T07:48:54.307 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.591562+0000', 'last_change': '2024-09-19T07:48:48.591562+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.591562+0000', 'last_undegraded': '2024-09-19T07:48:48.591562+0000', 'last_fullsized': '2024-09-19T07:48:48.591562+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T07:48:54.307 INFO:tasks.ceph.ceph_manager.ceph:PG 10.9 is not active+clean 2024-09-19T07:48:54.307 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.307 INFO:tasks.ceph.ceph_manager.ceph:PG 9.a is not active+clean 2024-09-19T07:48:54.307 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.584778+0000', 'last_change': '2024-09-19T07:48:48.584778+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.584778+0000', 'last_undegraded': '2024-09-19T07:48:48.584778+0000', 'last_fullsized': '2024-09-19T07:48:48.584778+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T07:48:54.307 INFO:tasks.ceph.ceph_manager.ceph:PG 10.a is not active+clean 2024-09-19T07:48:54.307 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.307 INFO:tasks.ceph.ceph_manager.ceph:PG 9.9 is not active+clean 2024-09-19T07:48:54.307 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.591509+0000', 'last_change': '2024-09-19T07:48:48.591509+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.591509+0000', 'last_undegraded': '2024-09-19T07:48:48.591509+0000', 'last_fullsized': '2024-09-19T07:48:48.591509+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T07:48:54.308 INFO:tasks.ceph.ceph_manager.ceph:PG 10.4 is not active+clean 2024-09-19T07:48:54.308 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.308 INFO:tasks.ceph.ceph_manager.ceph:PG 9.7 is not active+clean 2024-09-19T07:48:54.308 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.585145+0000', 'last_change': '2024-09-19T07:48:48.585145+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.585145+0000', 'last_undegraded': '2024-09-19T07:48:48.585145+0000', 'last_fullsized': '2024-09-19T07:48:48.585145+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T07:48:54.308 INFO:tasks.ceph.ceph_manager.ceph:PG 9.6 is not active+clean 2024-09-19T07:48:54.308 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.585906+0000', 'last_change': '2024-09-19T07:48:48.585906+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.585906+0000', 'last_undegraded': '2024-09-19T07:48:48.585906+0000', 'last_fullsized': '2024-09-19T07:48:48.585906+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T07:48:54.308 INFO:tasks.ceph.ceph_manager.ceph:PG 10.6 is not active+clean 2024-09-19T07:48:54.308 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.308 INFO:tasks.ceph.ceph_manager.ceph:PG 9.5 is not active+clean 2024-09-19T07:48:54.308 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.591640+0000', 'last_change': '2024-09-19T07:48:48.591640+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.591640+0000', 'last_undegraded': '2024-09-19T07:48:48.591640+0000', 'last_fullsized': '2024-09-19T07:48:48.591640+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T07:48:54.309 INFO:tasks.ceph.ceph_manager.ceph:PG 10.7 is not active+clean 2024-09-19T07:48:54.309 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.309 INFO:tasks.ceph.ceph_manager.ceph:PG 9.4 is not active+clean 2024-09-19T07:48:54.309 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:48.573172+0000', 'last_change': '2024-09-19T07:48:48.573172+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.573172+0000', 'last_undegraded': '2024-09-19T07:48:48.573172+0000', 'last_fullsized': '2024-09-19T07:48:48.573172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.309 INFO:tasks.ceph.ceph_manager.ceph:PG 10.0 is not active+clean 2024-09-19T07:48:54.309 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.309 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3 is not active+clean 2024-09-19T07:48:54.309 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:48:48.585154+0000', 'last_change': '2024-09-19T07:48:48.585154+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.585154+0000', 'last_undegraded': '2024-09-19T07:48:48.585154+0000', 'last_fullsized': '2024-09-19T07:48:48.585154+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T07:48:54.309 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1 is not active+clean 2024-09-19T07:48:54.309 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.309 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2 is not active+clean 2024-09-19T07:48:54.310 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:48.573172+0000', 'last_change': '2024-09-19T07:48:48.573172+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.573172+0000', 'last_undegraded': '2024-09-19T07:48:48.573172+0000', 'last_fullsized': '2024-09-19T07:48:48.573172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.310 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2 is not active+clean 2024-09-19T07:48:54.310 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.310 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1 is not active+clean 2024-09-19T07:48:54.310 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:48.573172+0000', 'last_change': '2024-09-19T07:48:48.573172+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.573172+0000', 'last_undegraded': '2024-09-19T07:48:48.573172+0000', 'last_fullsized': '2024-09-19T07:48:48.573172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.310 INFO:tasks.ceph.ceph_manager.ceph:PG 9.0 is not active+clean 2024-09-19T07:48:54.310 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:48.573172+0000', 'last_change': '2024-09-19T07:48:48.573172+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.573172+0000', 'last_undegraded': '2024-09-19T07:48:48.573172+0000', 'last_fullsized': '2024-09-19T07:48:48.573172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.310 INFO:tasks.ceph.ceph_manager.ceph:PG 10.b is not active+clean 2024-09-19T07:48:54.310 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:49.577543+0000', 'last_change': '2024-09-19T07:48:49.577543+0000', 'last_active': '2024-09-19T07:48:49.577543+0000', 'last_peered': '2024-09-19T07:48:49.577543+0000', 'last_clean': '2024-09-19T07:48:49.577543+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:49.577543+0000', 'last_undegraded': '2024-09-19T07:48:49.577543+0000', 'last_fullsized': '2024-09-19T07:48:49.577543+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:49.577543+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.310 INFO:tasks.ceph.ceph_manager.ceph:PG 9.8 is not active+clean 2024-09-19T07:48:54.310 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:48:48.573172+0000', 'last_change': '2024-09-19T07:48:48.573172+0000', 'last_active': '2024-09-19T07:48:48.573172+0000', 'last_peered': '2024-09-19T07:48:48.573172+0000', 'last_clean': '2024-09-19T07:48:48.573172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:48:48.573172+0000', 'last_undegraded': '2024-09-19T07:48:48.573172+0000', 'last_fullsized': '2024-09-19T07:48:48.573172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'last_clean_scrub_stamp': '2024-09-19T07:48:48.573172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:48:54.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T07:48:54.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.431+0000 7f80cff6d640 1 Processor -- start 2024-09-19T07:48:54.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.431+0000 7f80cff6d640 1 -- start start 2024-09-19T07:48:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.432+0000 7f80cff6d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80c815f0d0 0x7f80c815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.432+0000 7f80cff6d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f80c80d4130 0x7f80c80d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.432+0000 7f80cff6d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80c80d1b10 con 0x7f80c80d0070 2024-09-19T07:48:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.432+0000 7f80cff6d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f80c80cf2b0 con 0x7f80c815f0d0 2024-09-19T07:48:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.432+0000 7f80cff6d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f80c80cf590 con 0x7f80c80d4130 2024-09-19T07:48:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.432+0000 7f80cdd09640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80c815f0d0 0x7f80c815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.432+0000 7f80cdd09640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80c815f0d0 0x7f80c815f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59394/0 (socket says 172.21.15.16:59394) 2024-09-19T07:48:54.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.432+0000 7f80cd508640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f80c80d0070 0x7f80c80d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51224/0 (socket says 172.21.15.16:51224) 2024-09-19T07:48:54.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.432+0000 7f80cdd09640 1 -- 172.21.15.16:0/346233103 learned_addr learned my addr 172.21.15.16:0/346233103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:54.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.433+0000 7f80ce50a640 1 --2- 172.21.15.16:0/346233103 >> v2:172.21.15.146:3300/0 conn(0x7f80c80d4130 0x7f80c80d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:54.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.433+0000 7f80cdd09640 1 -- 172.21.15.16:0/346233103 >> v2:172.21.15.146:3300/0 conn(0x7f80c80d4130 msgr2=0x7f80c80d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:54.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.433+0000 7f80cdd09640 1 --2- 172.21.15.16:0/346233103 >> v2:172.21.15.146:3300/0 conn(0x7f80c80d4130 0x7f80c80d4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.433+0000 7f80cdd09640 1 -- 172.21.15.16:0/346233103 >> v1:172.21.15.16:6789/0 conn(0x7f80c80d0070 legacy=0x7f80c80d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:54.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.433+0000 7f80cdd09640 1 -- 172.21.15.16:0/346233103 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c80cf870 con 0x7f80c815f0d0 2024-09-19T07:48:54.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.433+0000 7f80cdd09640 1 --2- 172.21.15.16:0/346233103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c815f0d0 0x7f80c815f4b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f80c80d04e0 tx=0x7f80c402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7748015c53b63cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:54.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.433+0000 7f80ccd07640 1 -- 172.21.15.16:0/346233103 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4036070 con 0x7f80c815f0d0 2024-09-19T07:48:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.434+0000 7f80ccd07640 1 -- 172.21.15.16:0/346233103 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c4031600 con 0x7f80c815f0d0 2024-09-19T07:48:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.434+0000 7f80ccd07640 1 -- 172.21.15.16:0/346233103 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c40319d0 con 0x7f80c815f0d0 2024-09-19T07:48:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.434+0000 7f80cff6d640 1 -- 172.21.15.16:0/346233103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c815f0d0 msgr2=0x7f80c815f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.434+0000 7f80cff6d640 1 --2- 172.21.15.16:0/346233103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c815f0d0 0x7f80c815f4b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f80c80d04e0 tx=0x7f80c402a6a0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.434+0000 7f80cff6d640 1 -- 172.21.15.16:0/346233103 shutdown_connections 2024-09-19T07:48:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.434+0000 7f80cff6d640 1 --2- 172.21.15.16:0/346233103 >> v2:172.21.15.146:3300/0 conn(0x7f80c80d4130 0x7f80c80d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.434+0000 7f80cff6d640 1 --2- 172.21.15.16:0/346233103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c815f0d0 0x7f80c815f4b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.434+0000 7f80cff6d640 1 -- 172.21.15.16:0/346233103 >> 172.21.15.16:0/346233103 conn(0x7f80c809d760 msgr2=0x7f80c808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.434+0000 7f80cff6d640 1 -- 172.21.15.16:0/346233103 shutdown_connections 2024-09-19T07:48:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.434+0000 7f80cff6d640 1 -- 172.21.15.16:0/346233103 wait complete. 2024-09-19T07:48:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.435+0000 7f80cff6d640 1 Processor -- start 2024-09-19T07:48:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.435+0000 7f80cff6d640 1 -- start start 2024-09-19T07:48:54.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.436+0000 7f80cff6d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80c80d0070 0x7f80c814b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:54.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.436+0000 7f80cff6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c80d4130 0x7f80c80c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:54.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.436+0000 7f80cff6d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80c815f0d0 0x7f80c81f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:54.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.436+0000 7f80cff6d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80c80cef60 con 0x7f80c80d4130 2024-09-19T07:48:54.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.436+0000 7f80cff6d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f80c80d4aa0 con 0x7f80c80d0070 2024-09-19T07:48:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.436+0000 7f80cff6d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f80c80b2f70 con 0x7f80c815f0d0 2024-09-19T07:48:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.436+0000 7f80cd508640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c80d4130 0x7f80c80c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.436+0000 7f80cd508640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c80d4130 0x7f80c80c7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59396/0 (socket says 172.21.15.16:59396) 2024-09-19T07:48:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.436+0000 7f80cd508640 1 -- 172.21.15.16:0/1269798083 learned_addr learned my addr 172.21.15.16:0/1269798083 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.436+0000 7f80ce50a640 1 --2- 172.21.15.16:0/1269798083 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80c815f0d0 0x7f80c81f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.436+0000 7f80cdd09640 1 --2- 172.21.15.16:0/1269798083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80c80d0070 0x7f80c814b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.437+0000 7f80cd508640 1 -- 172.21.15.16:0/1269798083 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80c815f0d0 msgr2=0x7f80c81f7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.437+0000 7f80cd508640 1 --2- 172.21.15.16:0/1269798083 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80c815f0d0 0x7f80c81f7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.437+0000 7f80cd508640 1 -- 172.21.15.16:0/1269798083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80c80d0070 msgr2=0x7f80c814b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.437+0000 7f80cd508640 1 --2- 172.21.15.16:0/1269798083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80c80d0070 0x7f80c814b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.437+0000 7f80cd508640 1 -- 172.21.15.16:0/1269798083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c8151990 con 0x7f80c80d4130 2024-09-19T07:48:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.437+0000 7f80cd508640 1 --2- 172.21.15.16:0/1269798083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c80d4130 0x7f80c80c7ef0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f80b800c400 tx=0x7f80b800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.437+0000 7f80be7fc640 1 -- 172.21.15.16:0/1269798083 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b80074e0 con 0x7f80c80d4130 2024-09-19T07:48:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.437+0000 7f80be7fc640 1 -- 172.21.15.16:0/1269798083 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b80047e0 con 0x7f80c80d4130 2024-09-19T07:48:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.437+0000 7f80cff6d640 1 -- 172.21.15.16:0/1269798083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c80ca6e0 con 0x7f80c80d4130 2024-09-19T07:48:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.437+0000 7f80be7fc640 1 -- 172.21.15.16:0/1269798083 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8004bd0 con 0x7f80c80d4130 2024-09-19T07:48:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.437+0000 7f80cff6d640 1 -- 172.21.15.16:0/1269798083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c80caca0 con 0x7f80c80d4130 2024-09-19T07:48:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.438+0000 7f80cff6d640 1 -- 172.21.15.16:0/1269798083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c80d4560 con 0x7f80c80d4130 2024-09-19T07:48:54.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.439+0000 7f80be7fc640 1 -- 172.21.15.16:0/1269798083 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f80b801a020 con 0x7f80c80d4130 2024-09-19T07:48:54.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.439+0000 7f80be7fc640 1 --2- 172.21.15.16:0/1269798083 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f80ac06c2b0 0x7f80ac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:54.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.439+0000 7f80be7fc640 1 -- 172.21.15.16:0/1269798083 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f80b808ad00 con 0x7f80c80d4130 2024-09-19T07:48:54.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.440+0000 7f80cdd09640 1 --2- 172.21.15.16:0/1269798083 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f80ac06c2b0 0x7f80ac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:54.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.440+0000 7f80cdd09640 1 --2- 172.21.15.16:0/1269798083 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f80ac06c2b0 0x7f80ac06e770 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f80c4033000 tx=0x7f80c4009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.443+0000 7f80be7fc640 1 -- 172.21.15.16:0/1269798083 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f80b8058bf0 con 0x7f80c80d4130 2024-09-19T07:48:54.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.686+0000 7f80cff6d640 1 -- 172.21.15.16:0/1269798083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f80c806cf30 con 0x7f80c80d4130 2024-09-19T07:48:54.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.686+0000 7f80be7fc640 1 -- 172.21.15.16:0/1269798083 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1056 (secure 0 0 0) 0x7f80b805cef0 con 0x7f80c80d4130 2024-09-19T07:48:54.686 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:54.687 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":372,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":70,"num_osds":8,"num_up_osds":8,"osd_up_since":1726731773,"num_in_osds":8,"osd_in_since":1726731764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":767823872,"bytes_avail":772326289408,"bytes_total":773094113280,"write_bytes_sec":2408,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":33,"btime":"2024-09-19T07:48:52:612055+0000","id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":5751}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-19T07:46:18.501829+0000","services":{}},"progress_events":{}} 2024-09-19T07:48:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.688+0000 7f80cff6d640 1 -- 172.21.15.16:0/1269798083 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f80ac06c2b0 msgr2=0x7f80ac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.688+0000 7f80cff6d640 1 --2- 172.21.15.16:0/1269798083 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f80ac06c2b0 0x7f80ac06e770 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f80c4033000 tx=0x7f80c4009000 comp rx=0 tx=0).stop 2024-09-19T07:48:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.688+0000 7f80cff6d640 1 -- 172.21.15.16:0/1269798083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c80d4130 msgr2=0x7f80c80c7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.688+0000 7f80cff6d640 1 --2- 172.21.15.16:0/1269798083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c80d4130 0x7f80c80c7ef0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f80b800c400 tx=0x7f80b800c8c0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.688+0000 7f80cff6d640 1 -- 172.21.15.16:0/1269798083 shutdown_connections 2024-09-19T07:48:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.688+0000 7f80cff6d640 1 --2- 172.21.15.16:0/1269798083 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f80ac06c2b0 0x7f80ac06e770 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.688+0000 7f80cff6d640 1 --2- 172.21.15.16:0/1269798083 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80c815f0d0 0x7f80c81f7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.688+0000 7f80cff6d640 1 --2- 172.21.15.16:0/1269798083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c80d4130 0x7f80c80c7ef0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.688+0000 7f80cff6d640 1 --2- 172.21.15.16:0/1269798083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80c80d0070 0x7f80c814b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.688+0000 7f80cff6d640 1 -- 172.21.15.16:0/1269798083 >> 172.21.15.16:0/1269798083 conn(0x7f80c809d760 msgr2=0x7f80c80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.688+0000 7f80cff6d640 1 -- 172.21.15.16:0/1269798083 shutdown_connections 2024-09-19T07:48:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.688+0000 7f80cff6d640 1 -- 172.21.15.16:0/1269798083 wait complete. 2024-09-19T07:48:54.699 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-19T07:48:54.700 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:48:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.801+0000 7fe7706e6640 1 Processor -- start 2024-09-19T07:48:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.801+0000 7fe7706e6640 1 -- start start 2024-09-19T07:48:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.802+0000 7fe7706e6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe76815f0d0 0x7fe76815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.802+0000 7fe7706e6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7680d4130 0x7fe7680d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.802+0000 7fe7706e6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7680d4ae0 con 0x7fe7680d0070 2024-09-19T07:48:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.802+0000 7fe7706e6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe7680d1b10 con 0x7fe76815f0d0 2024-09-19T07:48:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.802+0000 7fe7706e6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe7680cf2b0 con 0x7fe7680d4130 2024-09-19T07:48:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.803+0000 7fe76dc81640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe7680d0070 0x7fe7680d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51234/0 (socket says 172.21.15.16:51234) 2024-09-19T07:48:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.803+0000 7fe76dc81640 1 -- 172.21.15.16:0/649072680 learned_addr learned my addr 172.21.15.16:0/649072680 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.803+0000 7fe76ec83640 1 --2- 172.21.15.16:0/649072680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7680d4130 0x7fe7680d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.803+0000 7fe76e482640 1 --2- 172.21.15.16:0/649072680 >> v2:172.21.15.146:3300/0 conn(0x7fe76815f0d0 0x7fe76815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.803+0000 7fe76d480640 1 -- 172.21.15.16:0/649072680 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2949979653 0 0) 0x7fe7680d4ae0 con 0x7fe7680d0070 2024-09-19T07:48:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.803+0000 7fe76d480640 1 -- 172.21.15.16:0/649072680 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe750003520 con 0x7fe7680d0070 2024-09-19T07:48:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.803+0000 7fe76d480640 1 -- 172.21.15.16:0/649072680 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2373710843 0 0) 0x7fe750003520 con 0x7fe7680d0070 2024-09-19T07:48:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.803+0000 7fe76ec83640 1 -- 172.21.15.16:0/649072680 >> v2:172.21.15.146:3300/0 conn(0x7fe76815f0d0 msgr2=0x7fe76815f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.803+0000 7fe76ec83640 1 --2- 172.21.15.16:0/649072680 >> v2:172.21.15.146:3300/0 conn(0x7fe76815f0d0 0x7fe76815f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.803+0000 7fe76ec83640 1 -- 172.21.15.16:0/649072680 >> v1:172.21.15.16:6789/0 conn(0x7fe7680d0070 legacy=0x7fe7680d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.803+0000 7fe76ec83640 1 -- 172.21.15.16:0/649072680 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7680cf590 con 0x7fe7680d4130 2024-09-19T07:48:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.803+0000 7fe76e482640 1 --2- 172.21.15.16:0/649072680 >> v2:172.21.15.146:3300/0 conn(0x7fe76815f0d0 0x7fe76815f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:48:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.804+0000 7fe76ec83640 1 --2- 172.21.15.16:0/649072680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7680d4130 0x7fe7680d45a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe7600097a0 tx=0x7fe76002a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe796ff9b36b236e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.804+0000 7fe76d480640 1 -- 172.21.15.16:0/649072680 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760037070 con 0x7fe7680d4130 2024-09-19T07:48:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.804+0000 7fe76d480640 1 -- 172.21.15.16:0/649072680 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe76002ab10 con 0x7fe7680d4130 2024-09-19T07:48:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.804+0000 7fe76d480640 1 -- 172.21.15.16:0/649072680 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c001670 con 0x7fe7680d4130 2024-09-19T07:48:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.805+0000 7fe7706e6640 1 -- 172.21.15.16:0/649072680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7680d4130 msgr2=0x7fe7680d45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.805+0000 7fe7706e6640 1 --2- 172.21.15.16:0/649072680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7680d4130 0x7fe7680d45a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe7600097a0 tx=0x7fe76002a080 comp rx=0 tx=0).stop 2024-09-19T07:48:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.805+0000 7fe7706e6640 1 -- 172.21.15.16:0/649072680 shutdown_connections 2024-09-19T07:48:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.805+0000 7fe7706e6640 1 --2- 172.21.15.16:0/649072680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7680d4130 0x7fe7680d45a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.805+0000 7fe7706e6640 1 --2- 172.21.15.16:0/649072680 >> v2:172.21.15.146:3300/0 conn(0x7fe76815f0d0 0x7fe76815f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.805+0000 7fe7706e6640 1 -- 172.21.15.16:0/649072680 >> 172.21.15.16:0/649072680 conn(0x7fe76809d760 msgr2=0x7fe76808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.805+0000 7fe7706e6640 1 -- 172.21.15.16:0/649072680 shutdown_connections 2024-09-19T07:48:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.805+0000 7fe7706e6640 1 -- 172.21.15.16:0/649072680 wait complete. 2024-09-19T07:48:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.806+0000 7fe7706e6640 1 Processor -- start 2024-09-19T07:48:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.806+0000 7fe7706e6640 1 -- start start 2024-09-19T07:48:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.807+0000 7fe7706e6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7680d0070 0x7fe7680cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.807+0000 7fe7706e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7680d4130 0x7fe768161120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.807+0000 7fe7706e6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe76815f0d0 0x7fe76815b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.807+0000 7fe7706e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe7680cef60 con 0x7fe7680d4130 2024-09-19T07:48:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.807+0000 7fe7706e6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe7680cfb50 con 0x7fe76815f0d0 2024-09-19T07:48:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.807+0000 7fe7706e6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe7680b2f70 con 0x7fe7680d0070 2024-09-19T07:48:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.807+0000 7fe76dc81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7680d4130 0x7fe768161120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.807+0000 7fe76dc81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7680d4130 0x7fe768161120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59400/0 (socket says 172.21.15.16:59400) 2024-09-19T07:48:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.807+0000 7fe76dc81640 1 -- 172.21.15.16:0/2025808259 learned_addr learned my addr 172.21.15.16:0/2025808259 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.807+0000 7fe76e482640 1 --2- 172.21.15.16:0/2025808259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7680d0070 0x7fe7680cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.807+0000 7fe76ec83640 1 --2- 172.21.15.16:0/2025808259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe76815f0d0 0x7fe76815b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.807+0000 7fe76dc81640 1 -- 172.21.15.16:0/2025808259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7680d0070 msgr2=0x7fe7680cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.807+0000 7fe76dc81640 1 --2- 172.21.15.16:0/2025808259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7680d0070 0x7fe7680cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.807+0000 7fe76dc81640 1 -- 172.21.15.16:0/2025808259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe76815f0d0 msgr2=0x7fe76815b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.807+0000 7fe76dc81640 1 --2- 172.21.15.16:0/2025808259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe76815f0d0 0x7fe76815b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.807+0000 7fe76dc81640 1 -- 172.21.15.16:0/2025808259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7680ce270 con 0x7fe7680d4130 2024-09-19T07:48:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.808+0000 7fe76dc81640 1 --2- 172.21.15.16:0/2025808259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7680d4130 0x7fe768161120 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00c780 tx=0x7fe75c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.808+0000 7fe766ffd640 1 -- 172.21.15.16:0/2025808259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c017070 con 0x7fe7680d4130 2024-09-19T07:48:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.808+0000 7fe766ffd640 1 -- 172.21.15.16:0/2025808259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe75c005ad0 con 0x7fe7680d4130 2024-09-19T07:48:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.808+0000 7fe7706e6640 1 -- 172.21.15.16:0/2025808259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7680c9870 con 0x7fe7680d4130 2024-09-19T07:48:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.808+0000 7fe766ffd640 1 -- 172.21.15.16:0/2025808259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c007070 con 0x7fe7680d4130 2024-09-19T07:48:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.808+0000 7fe7706e6640 1 -- 172.21.15.16:0/2025808259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7680c9e30 con 0x7fe7680d4130 2024-09-19T07:48:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.809+0000 7fe7706e6640 1 -- 172.21.15.16:0/2025808259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe76815f4b0 con 0x7fe7680d4130 2024-09-19T07:48:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.809+0000 7fe766ffd640 1 -- 172.21.15.16:0/2025808259 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe75c01a070 con 0x7fe7680d4130 2024-09-19T07:48:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.810+0000 7fe766ffd640 1 --2- 172.21.15.16:0/2025808259 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe75406c2b0 0x7fe75406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.810+0000 7fe766ffd640 1 -- 172.21.15.16:0/2025808259 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7fe75c08aa10 con 0x7fe7680d4130 2024-09-19T07:48:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.810+0000 7fe76e482640 1 --2- 172.21.15.16:0/2025808259 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe75406c2b0 0x7fe75406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.811+0000 7fe76e482640 1 --2- 172.21.15.16:0/2025808259 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe75406c2b0 0x7fe75406e770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe758009bd0 tx=0x7fe75800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:54.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.814+0000 7fe766ffd640 1 -- 172.21.15.16:0/2025808259 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe75c0589a0 con 0x7fe7680d4130 2024-09-19T07:48:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.989+0000 7fe7706e6640 1 -- 172.21.15.16:0/2025808259 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe76806cf30 con 0x7fe75406c2b0 2024-09-19T07:48:54.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.995+0000 7fe766ffd640 1 -- 172.21.15.16:0/2025808259 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252025 (secure 0 0 0) 0x7fe76806cf30 con 0x7fe75406c2b0 2024-09-19T07:48:54.998 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:48:55.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.999+0000 7fe7706e6640 1 -- 172.21.15.16:0/2025808259 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe75406c2b0 msgr2=0x7fe75406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:55.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.999+0000 7fe7706e6640 1 --2- 172.21.15.16:0/2025808259 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe75406c2b0 0x7fe75406e770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe758009bd0 tx=0x7fe75800b040 comp rx=0 tx=0).stop 2024-09-19T07:48:55.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.999+0000 7fe7706e6640 1 -- 172.21.15.16:0/2025808259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7680d4130 msgr2=0x7fe768161120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:55.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.999+0000 7fe7706e6640 1 --2- 172.21.15.16:0/2025808259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7680d4130 0x7fe768161120 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00c780 tx=0x7fe75c00cc40 comp rx=0 tx=0).stop 2024-09-19T07:48:55.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.999+0000 7fe7706e6640 1 -- 172.21.15.16:0/2025808259 shutdown_connections 2024-09-19T07:48:55.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.999+0000 7fe7706e6640 1 --2- 172.21.15.16:0/2025808259 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe75406c2b0 0x7fe75406e770 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:55.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.999+0000 7fe7706e6640 1 --2- 172.21.15.16:0/2025808259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe76815f0d0 0x7fe76815b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:55.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.999+0000 7fe7706e6640 1 --2- 172.21.15.16:0/2025808259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7680d4130 0x7fe768161120 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:55.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.999+0000 7fe7706e6640 1 --2- 172.21.15.16:0/2025808259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7680d0070 0x7fe7680cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:55.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.999+0000 7fe7706e6640 1 -- 172.21.15.16:0/2025808259 >> 172.21.15.16:0/2025808259 conn(0x7fe76809d760 msgr2=0x7fe7680b2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:55.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.999+0000 7fe7706e6640 1 -- 172.21.15.16:0/2025808259 shutdown_connections 2024-09-19T07:48:55.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:54.999+0000 7fe7706e6640 1 -- 172.21.15.16:0/2025808259 wait complete. 2024-09-19T07:48:55.010 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-19T07:48:54.534572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":749828,"kb_used_data":7760,"kb_used_omap":12,"kb_used_meta":741875,"kb_avail":754224892,"statfs":{"total":773094113280,"available":772326289408,"internally_reserved":0,"allocated":7946240,"data_stored":4236409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":759680560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.953113"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597587+0000","last_change":"2024-09-19T07:48:50.617279+0000","last_active":"2024-09-19T07:48:51.597587+0000","last_peered":"2024-09-19T07:48:51.597587+0000","last_clean":"2024-09-19T07:48:51.597587+0000","last_became_active":"2024-09-19T07:48:50.616175+0000","last_became_peered":"2024-09-19T07:48:50.616175+0000","last_unstale":"2024-09-19T07:48:51.597587+0000","last_undegraded":"2024-09-19T07:48:51.597587+0000","last_fullsized":"2024-09-19T07:48:51.597587+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:07.791953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597582+0000","last_change":"2024-09-19T07:48:50.618861+0000","last_active":"2024-09-19T07:48:51.597582+0000","last_peered":"2024-09-19T07:48:51.597582+0000","last_clean":"2024-09-19T07:48:51.597582+0000","last_became_active":"2024-09-19T07:48:50.618594+0000","last_became_peered":"2024-09-19T07:48:50.618594+0000","last_unstale":"2024-09-19T07:48:51.597582+0000","last_undegraded":"2024-09-19T07:48:51.597582+0000","last_fullsized":"2024-09-19T07:48:51.597582+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:32.771059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723962+0000","last_change":"2024-09-19T07:48:50.619839+0000","last_active":"2024-09-19T07:48:51.723962+0000","last_peered":"2024-09-19T07:48:51.723962+0000","last_clean":"2024-09-19T07:48:51.723962+0000","last_became_active":"2024-09-19T07:48:50.619499+0000","last_became_peered":"2024-09-19T07:48:50.619499+0000","last_unstale":"2024-09-19T07:48:51.723962+0000","last_undegraded":"2024-09-19T07:48:51.723962+0000","last_fullsized":"2024-09-19T07:48:51.723962+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:28.691031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601667+0000","last_change":"2024-09-19T07:48:50.618256+0000","last_active":"2024-09-19T07:48:51.601667+0000","last_peered":"2024-09-19T07:48:51.601667+0000","last_clean":"2024-09-19T07:48:51.601667+0000","last_became_active":"2024-09-19T07:48:50.617581+0000","last_became_peered":"2024-09-19T07:48:50.617581+0000","last_unstale":"2024-09-19T07:48:51.601667+0000","last_undegraded":"2024-09-19T07:48:51.601667+0000","last_fullsized":"2024-09-19T07:48:51.601667+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:23:24.632533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723687+0000","last_change":"2024-09-19T07:48:50.613364+0000","last_active":"2024-09-19T07:48:51.723687+0000","last_peered":"2024-09-19T07:48:51.723687+0000","last_clean":"2024-09-19T07:48:51.723687+0000","last_became_active":"2024-09-19T07:48:50.611676+0000","last_became_peered":"2024-09-19T07:48:50.611676+0000","last_unstale":"2024-09-19T07:48:51.723687+0000","last_undegraded":"2024-09-19T07:48:51.723687+0000","last_fullsized":"2024-09-19T07:48:51.723687+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:31.662628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723769+0000","last_change":"2024-09-19T07:48:50.622683+0000","last_active":"2024-09-19T07:48:51.723769+0000","last_peered":"2024-09-19T07:48:51.723769+0000","last_clean":"2024-09-19T07:48:51.723769+0000","last_became_active":"2024-09-19T07:48:50.622075+0000","last_became_peered":"2024-09-19T07:48:50.622075+0000","last_unstale":"2024-09-19T07:48:51.723769+0000","last_undegraded":"2024-09-19T07:48:51.723769+0000","last_fullsized":"2024-09-19T07:48:51.723769+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:28.691031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.610050+0000","last_change":"2024-09-19T07:48:50.609214+0000","last_active":"2024-09-19T07:48:50.610050+0000","last_peered":"2024-09-19T07:48:50.610050+0000","last_clean":"2024-09-19T07:48:50.610050+0000","last_became_active":"2024-09-19T07:48:50.608804+0000","last_became_peered":"2024-09-19T07:48:50.608804+0000","last_unstale":"2024-09-19T07:48:50.610050+0000","last_undegraded":"2024-09-19T07:48:50.610050+0000","last_fullsized":"2024-09-19T07:48:50.610050+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:31:31.683080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601391+0000","last_change":"2024-09-19T07:48:50.614939+0000","last_active":"2024-09-19T07:48:51.601391+0000","last_peered":"2024-09-19T07:48:51.601391+0000","last_clean":"2024-09-19T07:48:51.601391+0000","last_became_active":"2024-09-19T07:48:50.614661+0000","last_became_peered":"2024-09-19T07:48:50.614661+0000","last_unstale":"2024-09-19T07:48:51.601391+0000","last_undegraded":"2024-09-19T07:48:51.601391+0000","last_fullsized":"2024-09-19T07:48:51.601391+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:10.375253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724172+0000","last_change":"2024-09-19T07:48:50.611473+0000","last_active":"2024-09-19T07:48:51.724172+0000","last_peered":"2024-09-19T07:48:51.724172+0000","last_clean":"2024-09-19T07:48:51.724172+0000","last_became_active":"2024-09-19T07:48:50.611205+0000","last_became_peered":"2024-09-19T07:48:50.611205+0000","last_unstale":"2024-09-19T07:48:51.724172+0000","last_undegraded":"2024-09-19T07:48:51.724172+0000","last_fullsized":"2024-09-19T07:48:51.724172+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:33.519772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724403+0000","last_change":"2024-09-19T07:48:50.617206+0000","last_active":"2024-09-19T07:48:51.724403+0000","last_peered":"2024-09-19T07:48:51.724403+0000","last_clean":"2024-09-19T07:48:51.724403+0000","last_became_active":"2024-09-19T07:48:50.616917+0000","last_became_peered":"2024-09-19T07:48:50.616917+0000","last_unstale":"2024-09-19T07:48:51.724403+0000","last_undegraded":"2024-09-19T07:48:51.724403+0000","last_fullsized":"2024-09-19T07:48:51.724403+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:39.970307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.600909+0000","last_change":"2024-09-19T07:48:50.606655+0000","last_active":"2024-09-19T07:48:51.600909+0000","last_peered":"2024-09-19T07:48:51.600909+0000","last_clean":"2024-09-19T07:48:51.600909+0000","last_became_active":"2024-09-19T07:48:50.606376+0000","last_became_peered":"2024-09-19T07:48:50.606376+0000","last_unstale":"2024-09-19T07:48:51.600909+0000","last_undegraded":"2024-09-19T07:48:51.600909+0000","last_fullsized":"2024-09-19T07:48:51.600909+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:02.115678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.600960+0000","last_change":"2024-09-19T07:48:50.614911+0000","last_active":"2024-09-19T07:48:51.600960+0000","last_peered":"2024-09-19T07:48:51.600960+0000","last_clean":"2024-09-19T07:48:51.600960+0000","last_became_active":"2024-09-19T07:48:50.614623+0000","last_became_peered":"2024-09-19T07:48:50.614623+0000","last_unstale":"2024-09-19T07:48:51.600960+0000","last_undegraded":"2024-09-19T07:48:51.600960+0000","last_fullsized":"2024-09-19T07:48:51.600960+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:11.128746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.600828+0000","last_change":"2024-09-19T07:48:50.603843+0000","last_active":"2024-09-19T07:48:51.600828+0000","last_peered":"2024-09-19T07:48:51.600828+0000","last_clean":"2024-09-19T07:48:51.600828+0000","last_became_active":"2024-09-19T07:48:50.603521+0000","last_became_peered":"2024-09-19T07:48:50.603521+0000","last_unstale":"2024-09-19T07:48:51.600828+0000","last_undegraded":"2024-09-19T07:48:51.600828+0000","last_fullsized":"2024-09-19T07:48:51.600828+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:16.152040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724418+0000","last_change":"2024-09-19T07:48:50.621010+0000","last_active":"2024-09-19T07:48:51.724418+0000","last_peered":"2024-09-19T07:48:51.724418+0000","last_clean":"2024-09-19T07:48:51.724418+0000","last_became_active":"2024-09-19T07:48:50.619664+0000","last_became_peered":"2024-09-19T07:48:50.619664+0000","last_unstale":"2024-09-19T07:48:51.724418+0000","last_undegraded":"2024-09-19T07:48:51.724418+0000","last_fullsized":"2024-09-19T07:48:51.724418+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:10.375253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724468+0000","last_change":"2024-09-19T07:48:50.616922+0000","last_active":"2024-09-19T07:48:51.724468+0000","last_peered":"2024-09-19T07:48:51.724468+0000","last_clean":"2024-09-19T07:48:51.724468+0000","last_became_active":"2024-09-19T07:48:50.616630+0000","last_became_peered":"2024-09-19T07:48:50.616630+0000","last_unstale":"2024-09-19T07:48:51.724468+0000","last_undegraded":"2024-09-19T07:48:51.724468+0000","last_fullsized":"2024-09-19T07:48:51.724468+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:50.664743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.607757+0000","last_change":"2024-09-19T07:48:50.604348+0000","last_active":"2024-09-19T07:48:50.607757+0000","last_peered":"2024-09-19T07:48:50.607757+0000","last_clean":"2024-09-19T07:48:50.607757+0000","last_became_active":"2024-09-19T07:48:50.604037+0000","last_became_peered":"2024-09-19T07:48:50.604037+0000","last_unstale":"2024-09-19T07:48:50.607757+0000","last_undegraded":"2024-09-19T07:48:50.607757+0000","last_fullsized":"2024-09-19T07:48:50.607757+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:16.200749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597616+0000","last_change":"2024-09-19T07:48:50.606625+0000","last_active":"2024-09-19T07:48:51.597616+0000","last_peered":"2024-09-19T07:48:51.597616+0000","last_clean":"2024-09-19T07:48:51.597616+0000","last_became_active":"2024-09-19T07:48:50.606339+0000","last_became_peered":"2024-09-19T07:48:50.606339+0000","last_unstale":"2024-09-19T07:48:51.597616+0000","last_undegraded":"2024-09-19T07:48:51.597616+0000","last_fullsized":"2024-09-19T07:48:51.597616+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:03:39.985164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601479+0000","last_change":"2024-09-19T07:48:50.616872+0000","last_active":"2024-09-19T07:48:51.601479+0000","last_peered":"2024-09-19T07:48:51.601479+0000","last_clean":"2024-09-19T07:48:51.601479+0000","last_became_active":"2024-09-19T07:48:50.616363+0000","last_became_peered":"2024-09-19T07:48:50.616363+0000","last_unstale":"2024-09-19T07:48:51.601479+0000","last_undegraded":"2024-09-19T07:48:51.601479+0000","last_fullsized":"2024-09-19T07:48:51.601479+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:31.662628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723657+0000","last_change":"2024-09-19T07:48:50.623011+0000","last_active":"2024-09-19T07:48:51.723657+0000","last_peered":"2024-09-19T07:48:51.723657+0000","last_clean":"2024-09-19T07:48:51.723657+0000","last_became_active":"2024-09-19T07:48:50.622611+0000","last_became_peered":"2024-09-19T07:48:50.622611+0000","last_unstale":"2024-09-19T07:48:51.723657+0000","last_undegraded":"2024-09-19T07:48:51.723657+0000","last_fullsized":"2024-09-19T07:48:51.723657+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:00.856819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723831+0000","last_change":"2024-09-19T07:48:50.613355+0000","last_active":"2024-09-19T07:48:51.723831+0000","last_peered":"2024-09-19T07:48:51.723831+0000","last_clean":"2024-09-19T07:48:51.723831+0000","last_became_active":"2024-09-19T07:48:50.612994+0000","last_became_peered":"2024-09-19T07:48:50.612994+0000","last_unstale":"2024-09-19T07:48:51.723831+0000","last_undegraded":"2024-09-19T07:48:51.723831+0000","last_fullsized":"2024-09-19T07:48:51.723831+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:28.078297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601169+0000","last_change":"2024-09-19T07:48:50.608752+0000","last_active":"2024-09-19T07:48:51.601169+0000","last_peered":"2024-09-19T07:48:51.601169+0000","last_clean":"2024-09-19T07:48:51.601169+0000","last_became_active":"2024-09-19T07:48:50.606884+0000","last_became_peered":"2024-09-19T07:48:50.606884+0000","last_unstale":"2024-09-19T07:48:51.601169+0000","last_undegraded":"2024-09-19T07:48:51.601169+0000","last_fullsized":"2024-09-19T07:48:51.601169+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:33.519772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723555+0000","last_change":"2024-09-19T07:48:50.608708+0000","last_active":"2024-09-19T07:48:51.723555+0000","last_peered":"2024-09-19T07:48:51.723555+0000","last_clean":"2024-09-19T07:48:51.723555+0000","last_became_active":"2024-09-19T07:48:50.608398+0000","last_became_peered":"2024-09-19T07:48:50.608398+0000","last_unstale":"2024-09-19T07:48:51.723555+0000","last_undegraded":"2024-09-19T07:48:51.723555+0000","last_fullsized":"2024-09-19T07:48:51.723555+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:39:39.078202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723612+0000","last_change":"2024-09-19T07:48:50.608395+0000","last_active":"2024-09-19T07:48:51.723612+0000","last_peered":"2024-09-19T07:48:51.723612+0000","last_clean":"2024-09-19T07:48:51.723612+0000","last_became_active":"2024-09-19T07:48:50.607050+0000","last_became_peered":"2024-09-19T07:48:50.607050+0000","last_unstale":"2024-09-19T07:48:51.723612+0000","last_undegraded":"2024-09-19T07:48:51.723612+0000","last_fullsized":"2024-09-19T07:48:51.723612+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:25.713782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723647+0000","last_change":"2024-09-19T07:48:50.619593+0000","last_active":"2024-09-19T07:48:51.723647+0000","last_peered":"2024-09-19T07:48:51.723647+0000","last_clean":"2024-09-19T07:48:51.723647+0000","last_became_active":"2024-09-19T07:48:50.619318+0000","last_became_peered":"2024-09-19T07:48:50.619318+0000","last_unstale":"2024-09-19T07:48:51.723647+0000","last_undegraded":"2024-09-19T07:48:51.723647+0000","last_fullsized":"2024-09-19T07:48:51.723647+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:16.152040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601542+0000","last_change":"2024-09-19T07:48:50.605776+0000","last_active":"2024-09-19T07:48:51.601542+0000","last_peered":"2024-09-19T07:48:51.601542+0000","last_clean":"2024-09-19T07:48:51.601542+0000","last_became_active":"2024-09-19T07:48:50.605472+0000","last_became_peered":"2024-09-19T07:48:50.605472+0000","last_unstale":"2024-09-19T07:48:51.601542+0000","last_undegraded":"2024-09-19T07:48:51.601542+0000","last_fullsized":"2024-09-19T07:48:51.601542+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:50.664743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601221+0000","last_change":"2024-09-19T07:48:50.606522+0000","last_active":"2024-09-19T07:48:51.601221+0000","last_peered":"2024-09-19T07:48:51.601221+0000","last_clean":"2024-09-19T07:48:51.601221+0000","last_became_active":"2024-09-19T07:48:50.606251+0000","last_became_peered":"2024-09-19T07:48:50.606251+0000","last_unstale":"2024-09-19T07:48:51.601221+0000","last_undegraded":"2024-09-19T07:48:51.601221+0000","last_fullsized":"2024-09-19T07:48:51.601221+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:39.970307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601058+0000","last_change":"2024-09-19T07:48:50.605074+0000","last_active":"2024-09-19T07:48:51.601058+0000","last_peered":"2024-09-19T07:48:51.601058+0000","last_clean":"2024-09-19T07:48:51.601058+0000","last_became_active":"2024-09-19T07:48:50.604749+0000","last_became_peered":"2024-09-19T07:48:50.604749+0000","last_unstale":"2024-09-19T07:48:51.601058+0000","last_undegraded":"2024-09-19T07:48:51.601058+0000","last_fullsized":"2024-09-19T07:48:51.601058+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:02.116554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597433+0000","last_change":"2024-09-19T07:48:50.621418+0000","last_active":"2024-09-19T07:48:51.597433+0000","last_peered":"2024-09-19T07:48:51.597433+0000","last_clean":"2024-09-19T07:48:51.597433+0000","last_became_active":"2024-09-19T07:48:50.620678+0000","last_became_peered":"2024-09-19T07:48:50.620678+0000","last_unstale":"2024-09-19T07:48:51.597433+0000","last_undegraded":"2024-09-19T07:48:51.597433+0000","last_fullsized":"2024-09-19T07:48:51.597433+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:18.120305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723984+0000","last_change":"2024-09-19T07:48:50.615969+0000","last_active":"2024-09-19T07:48:51.723984+0000","last_peered":"2024-09-19T07:48:51.723984+0000","last_clean":"2024-09-19T07:48:51.723984+0000","last_became_active":"2024-09-19T07:48:50.613857+0000","last_became_peered":"2024-09-19T07:48:50.613857+0000","last_unstale":"2024-09-19T07:48:51.723984+0000","last_undegraded":"2024-09-19T07:48:51.723984+0000","last_fullsized":"2024-09-19T07:48:51.723984+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:28.290169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601058+0000","last_change":"2024-09-19T07:48:50.618585+0000","last_active":"2024-09-19T07:48:51.601058+0000","last_peered":"2024-09-19T07:48:51.601058+0000","last_clean":"2024-09-19T07:48:51.601058+0000","last_became_active":"2024-09-19T07:48:50.618277+0000","last_became_peered":"2024-09-19T07:48:50.618277+0000","last_unstale":"2024-09-19T07:48:51.601058+0000","last_undegraded":"2024-09-19T07:48:51.601058+0000","last_fullsized":"2024-09-19T07:48:51.601058+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:28.691031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601172+0000","last_change":"2024-09-19T07:48:50.601676+0000","last_active":"2024-09-19T07:48:51.601172+0000","last_peered":"2024-09-19T07:48:51.601172+0000","last_clean":"2024-09-19T07:48:51.601172+0000","last_became_active":"2024-09-19T07:48:50.601372+0000","last_became_peered":"2024-09-19T07:48:50.601372+0000","last_unstale":"2024-09-19T07:48:51.601172+0000","last_undegraded":"2024-09-19T07:48:51.601172+0000","last_fullsized":"2024-09-19T07:48:51.601172+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:11.968589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723186+0000","last_change":"2024-09-19T07:48:50.612616+0000","last_active":"2024-09-19T07:48:51.723186+0000","last_peered":"2024-09-19T07:48:51.723186+0000","last_clean":"2024-09-19T07:48:51.723186+0000","last_became_active":"2024-09-19T07:48:50.611466+0000","last_became_peered":"2024-09-19T07:48:50.611466+0000","last_unstale":"2024-09-19T07:48:51.723186+0000","last_undegraded":"2024-09-19T07:48:51.723186+0000","last_fullsized":"2024-09-19T07:48:51.723186+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:58.590631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723931+0000","last_change":"2024-09-19T07:48:50.623832+0000","last_active":"2024-09-19T07:48:51.723931+0000","last_peered":"2024-09-19T07:48:51.723931+0000","last_clean":"2024-09-19T07:48:51.723931+0000","last_became_active":"2024-09-19T07:48:50.623431+0000","last_became_peered":"2024-09-19T07:48:50.623431+0000","last_unstale":"2024-09-19T07:48:51.723931+0000","last_undegraded":"2024-09-19T07:48:51.723931+0000","last_fullsized":"2024-09-19T07:48:51.723931+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:41.958871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"69'4","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597696+0000","last_change":"2024-09-19T07:48:49.602061+0000","last_active":"2024-09-19T07:48:51.597696+0000","last_peered":"2024-09-19T07:48:51.597696+0000","last_clean":"2024-09-19T07:48:51.597696+0000","last_became_active":"2024-09-19T07:48:49.601684+0000","last_became_peered":"2024-09-19T07:48:49.601684+0000","last_unstale":"2024-09-19T07:48:51.597696+0000","last_undegraded":"2024-09-19T07:48:51.597696+0000","last_fullsized":"2024-09-19T07:48:51.597696+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:10.964218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601383+0000","last_change":"2024-09-19T07:48:50.619058+0000","last_active":"2024-09-19T07:48:51.601383+0000","last_peered":"2024-09-19T07:48:51.601383+0000","last_clean":"2024-09-19T07:48:51.601383+0000","last_became_active":"2024-09-19T07:48:50.618776+0000","last_became_peered":"2024-09-19T07:48:50.618776+0000","last_unstale":"2024-09-19T07:48:51.601383+0000","last_undegraded":"2024-09-19T07:48:51.601383+0000","last_fullsized":"2024-09-19T07:48:51.601383+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:00.856819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723565+0000","last_change":"2024-09-19T07:48:49.603413+0000","last_active":"2024-09-19T07:48:51.723565+0000","last_peered":"2024-09-19T07:48:51.723565+0000","last_clean":"2024-09-19T07:48:51.723565+0000","last_became_active":"2024-09-19T07:48:49.603043+0000","last_became_peered":"2024-09-19T07:48:49.603043+0000","last_unstale":"2024-09-19T07:48:51.723565+0000","last_undegraded":"2024-09-19T07:48:51.723565+0000","last_fullsized":"2024-09-19T07:48:51.723565+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:03:38.980793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723514+0000","last_change":"2024-09-19T07:48:50.616434+0000","last_active":"2024-09-19T07:48:51.723514+0000","last_peered":"2024-09-19T07:48:51.723514+0000","last_clean":"2024-09-19T07:48:51.723514+0000","last_became_active":"2024-09-19T07:48:50.615413+0000","last_became_peered":"2024-09-19T07:48:50.615413+0000","last_unstale":"2024-09-19T07:48:51.723514+0000","last_undegraded":"2024-09-19T07:48:51.723514+0000","last_fullsized":"2024-09-19T07:48:51.723514+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:11.128746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723784+0000","last_change":"2024-09-19T07:48:49.600450+0000","last_active":"2024-09-19T07:48:51.723784+0000","last_peered":"2024-09-19T07:48:51.723784+0000","last_clean":"2024-09-19T07:48:51.723784+0000","last_became_active":"2024-09-19T07:48:49.600094+0000","last_became_peered":"2024-09-19T07:48:49.600094+0000","last_unstale":"2024-09-19T07:48:51.723784+0000","last_undegraded":"2024-09-19T07:48:51.723784+0000","last_fullsized":"2024-09-19T07:48:51.723784+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:07.031415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723918+0000","last_change":"2024-09-19T07:48:50.622627+0000","last_active":"2024-09-19T07:48:51.723918+0000","last_peered":"2024-09-19T07:48:51.723918+0000","last_clean":"2024-09-19T07:48:51.723918+0000","last_became_active":"2024-09-19T07:48:50.622335+0000","last_became_peered":"2024-09-19T07:48:50.622335+0000","last_unstale":"2024-09-19T07:48:51.723918+0000","last_undegraded":"2024-09-19T07:48:51.723918+0000","last_fullsized":"2024-09-19T07:48:51.723918+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:11.128746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723325+0000","last_change":"2024-09-19T07:48:49.596140+0000","last_active":"2024-09-19T07:48:51.723325+0000","last_peered":"2024-09-19T07:48:51.723325+0000","last_clean":"2024-09-19T07:48:51.723325+0000","last_became_active":"2024-09-19T07:48:49.595802+0000","last_became_peered":"2024-09-19T07:48:49.595802+0000","last_unstale":"2024-09-19T07:48:51.723325+0000","last_undegraded":"2024-09-19T07:48:51.723325+0000","last_fullsized":"2024-09-19T07:48:51.723325+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:17.115934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.609656+0000","last_change":"2024-09-19T07:48:50.607600+0000","last_active":"2024-09-19T07:48:50.609656+0000","last_peered":"2024-09-19T07:48:50.609656+0000","last_clean":"2024-09-19T07:48:50.609656+0000","last_became_active":"2024-09-19T07:48:50.606735+0000","last_became_peered":"2024-09-19T07:48:50.606735+0000","last_unstale":"2024-09-19T07:48:50.609656+0000","last_undegraded":"2024-09-19T07:48:50.609656+0000","last_fullsized":"2024-09-19T07:48:50.609656+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:11.159734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723767+0000","last_change":"2024-09-19T07:48:49.597306+0000","last_active":"2024-09-19T07:48:51.723767+0000","last_peered":"2024-09-19T07:48:51.723767+0000","last_clean":"2024-09-19T07:48:51.723767+0000","last_became_active":"2024-09-19T07:48:49.597002+0000","last_became_peered":"2024-09-19T07:48:49.597002+0000","last_unstale":"2024-09-19T07:48:51.723767+0000","last_undegraded":"2024-09-19T07:48:51.723767+0000","last_fullsized":"2024-09-19T07:48:51.723767+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:29.429632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597211+0000","last_change":"2024-09-19T07:48:50.619874+0000","last_active":"2024-09-19T07:48:51.597211+0000","last_peered":"2024-09-19T07:48:51.597211+0000","last_clean":"2024-09-19T07:48:51.597211+0000","last_became_active":"2024-09-19T07:48:50.619587+0000","last_became_peered":"2024-09-19T07:48:50.619587+0000","last_unstale":"2024-09-19T07:48:51.597211+0000","last_undegraded":"2024-09-19T07:48:51.597211+0000","last_fullsized":"2024-09-19T07:48:51.597211+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:11.464590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723654+0000","last_change":"2024-09-19T07:48:49.591854+0000","last_active":"2024-09-19T07:48:51.723654+0000","last_peered":"2024-09-19T07:48:51.723654+0000","last_clean":"2024-09-19T07:48:51.723654+0000","last_became_active":"2024-09-19T07:48:49.591532+0000","last_became_peered":"2024-09-19T07:48:49.591532+0000","last_unstale":"2024-09-19T07:48:51.723654+0000","last_undegraded":"2024-09-19T07:48:51.723654+0000","last_fullsized":"2024-09-19T07:48:51.723654+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:06.402253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723576+0000","last_change":"2024-09-19T07:48:50.620827+0000","last_active":"2024-09-19T07:48:51.723576+0000","last_peered":"2024-09-19T07:48:51.723576+0000","last_clean":"2024-09-19T07:48:51.723576+0000","last_became_active":"2024-09-19T07:48:50.620190+0000","last_became_peered":"2024-09-19T07:48:50.620190+0000","last_unstale":"2024-09-19T07:48:51.723576+0000","last_undegraded":"2024-09-19T07:48:51.723576+0000","last_fullsized":"2024-09-19T07:48:51.723576+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:51.090644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723756+0000","last_change":"2024-09-19T07:48:49.595497+0000","last_active":"2024-09-19T07:48:51.723756+0000","last_peered":"2024-09-19T07:48:51.723756+0000","last_clean":"2024-09-19T07:48:51.723756+0000","last_became_active":"2024-09-19T07:48:49.594777+0000","last_became_peered":"2024-09-19T07:48:49.594777+0000","last_unstale":"2024-09-19T07:48:51.723756+0000","last_undegraded":"2024-09-19T07:48:51.723756+0000","last_fullsized":"2024-09-19T07:48:51.723756+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:11.653266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723658+0000","last_change":"2024-09-19T07:48:50.606445+0000","last_active":"2024-09-19T07:48:51.723658+0000","last_peered":"2024-09-19T07:48:51.723658+0000","last_clean":"2024-09-19T07:48:51.723658+0000","last_became_active":"2024-09-19T07:48:50.606155+0000","last_became_peered":"2024-09-19T07:48:50.606155+0000","last_unstale":"2024-09-19T07:48:51.723658+0000","last_undegraded":"2024-09-19T07:48:51.723658+0000","last_fullsized":"2024-09-19T07:48:51.723658+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:20.244263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723358+0000","last_change":"2024-09-19T07:48:49.596857+0000","last_active":"2024-09-19T07:48:51.723358+0000","last_peered":"2024-09-19T07:48:51.723358+0000","last_clean":"2024-09-19T07:48:51.723358+0000","last_became_active":"2024-09-19T07:48:49.596597+0000","last_became_peered":"2024-09-19T07:48:49.596597+0000","last_unstale":"2024-09-19T07:48:51.723358+0000","last_undegraded":"2024-09-19T07:48:51.723358+0000","last_fullsized":"2024-09-19T07:48:51.723358+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:28.988556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597587+0000","last_change":"2024-09-19T07:48:50.610040+0000","last_active":"2024-09-19T07:48:51.597587+0000","last_peered":"2024-09-19T07:48:51.597587+0000","last_clean":"2024-09-19T07:48:51.597587+0000","last_became_active":"2024-09-19T07:48:50.609745+0000","last_became_peered":"2024-09-19T07:48:50.609745+0000","last_unstale":"2024-09-19T07:48:51.597587+0000","last_undegraded":"2024-09-19T07:48:51.597587+0000","last_fullsized":"2024-09-19T07:48:51.597587+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:08.035787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724114+0000","last_change":"2024-09-19T07:48:49.603689+0000","last_active":"2024-09-19T07:48:51.724114+0000","last_peered":"2024-09-19T07:48:51.724114+0000","last_clean":"2024-09-19T07:48:51.724114+0000","last_became_active":"2024-09-19T07:48:49.602991+0000","last_became_peered":"2024-09-19T07:48:49.602991+0000","last_unstale":"2024-09-19T07:48:51.724114+0000","last_undegraded":"2024-09-19T07:48:51.724114+0000","last_fullsized":"2024-09-19T07:48:51.724114+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:50.104285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723504+0000","last_change":"2024-09-19T07:48:50.608683+0000","last_active":"2024-09-19T07:48:51.723504+0000","last_peered":"2024-09-19T07:48:51.723504+0000","last_clean":"2024-09-19T07:48:51.723504+0000","last_became_active":"2024-09-19T07:48:50.608336+0000","last_became_peered":"2024-09-19T07:48:50.608336+0000","last_unstale":"2024-09-19T07:48:51.723504+0000","last_undegraded":"2024-09-19T07:48:51.723504+0000","last_fullsized":"2024-09-19T07:48:51.723504+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:43.270718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601589+0000","last_change":"2024-09-19T07:48:49.597480+0000","last_active":"2024-09-19T07:48:51.601589+0000","last_peered":"2024-09-19T07:48:51.601589+0000","last_clean":"2024-09-19T07:48:51.601589+0000","last_became_active":"2024-09-19T07:48:49.597175+0000","last_became_peered":"2024-09-19T07:48:49.597175+0000","last_unstale":"2024-09-19T07:48:51.601589+0000","last_undegraded":"2024-09-19T07:48:51.601589+0000","last_fullsized":"2024-09-19T07:48:51.601589+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:33:05.500303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723753+0000","last_change":"2024-09-19T07:48:50.617419+0000","last_active":"2024-09-19T07:48:51.723753+0000","last_peered":"2024-09-19T07:48:51.723753+0000","last_clean":"2024-09-19T07:48:51.723753+0000","last_became_active":"2024-09-19T07:48:50.617165+0000","last_became_peered":"2024-09-19T07:48:50.617165+0000","last_unstale":"2024-09-19T07:48:51.723753+0000","last_undegraded":"2024-09-19T07:48:51.723753+0000","last_fullsized":"2024-09-19T07:48:51.723753+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:06.722534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"69'3","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724022+0000","last_change":"2024-09-19T07:48:49.604058+0000","last_active":"2024-09-19T07:48:51.724022+0000","last_peered":"2024-09-19T07:48:51.724022+0000","last_clean":"2024-09-19T07:48:51.724022+0000","last_became_active":"2024-09-19T07:48:49.602835+0000","last_became_peered":"2024-09-19T07:48:49.602835+0000","last_unstale":"2024-09-19T07:48:51.724022+0000","last_undegraded":"2024-09-19T07:48:51.724022+0000","last_fullsized":"2024-09-19T07:48:51.724022+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:52.157019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601888+0000","last_change":"2024-09-19T07:48:50.605331+0000","last_active":"2024-09-19T07:48:51.601888+0000","last_peered":"2024-09-19T07:48:51.601888+0000","last_clean":"2024-09-19T07:48:51.601888+0000","last_became_active":"2024-09-19T07:48:50.605066+0000","last_became_peered":"2024-09-19T07:48:50.605066+0000","last_unstale":"2024-09-19T07:48:51.601888+0000","last_undegraded":"2024-09-19T07:48:51.601888+0000","last_fullsized":"2024-09-19T07:48:51.601888+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:11.570438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"69'6","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597581+0000","last_change":"2024-09-19T07:48:49.599753+0000","last_active":"2024-09-19T07:48:51.597581+0000","last_peered":"2024-09-19T07:48:51.597581+0000","last_clean":"2024-09-19T07:48:51.597581+0000","last_became_active":"2024-09-19T07:48:49.599452+0000","last_became_peered":"2024-09-19T07:48:49.599452+0000","last_unstale":"2024-09-19T07:48:51.597581+0000","last_undegraded":"2024-09-19T07:48:51.597581+0000","last_fullsized":"2024-09-19T07:48:51.597581+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:30.658256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601585+0000","last_change":"2024-09-19T07:48:50.610147+0000","last_active":"2024-09-19T07:48:51.601585+0000","last_peered":"2024-09-19T07:48:51.601585+0000","last_clean":"2024-09-19T07:48:51.601585+0000","last_became_active":"2024-09-19T07:48:50.609675+0000","last_became_peered":"2024-09-19T07:48:50.609675+0000","last_unstale":"2024-09-19T07:48:51.601585+0000","last_undegraded":"2024-09-19T07:48:51.601585+0000","last_fullsized":"2024-09-19T07:48:51.601585+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:06.722534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724117+0000","last_change":"2024-09-19T07:48:49.602673+0000","last_active":"2024-09-19T07:48:51.724117+0000","last_peered":"2024-09-19T07:48:51.724117+0000","last_clean":"2024-09-19T07:48:51.724117+0000","last_became_active":"2024-09-19T07:48:49.601601+0000","last_became_peered":"2024-09-19T07:48:49.601601+0000","last_unstale":"2024-09-19T07:48:51.724117+0000","last_undegraded":"2024-09-19T07:48:51.724117+0000","last_fullsized":"2024-09-19T07:48:51.724117+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:17.115934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.600815+0000","last_change":"2024-09-19T07:48:50.612160+0000","last_active":"2024-09-19T07:48:51.600815+0000","last_peered":"2024-09-19T07:48:51.600815+0000","last_clean":"2024-09-19T07:48:51.600815+0000","last_became_active":"2024-09-19T07:48:50.611833+0000","last_became_peered":"2024-09-19T07:48:50.611833+0000","last_unstale":"2024-09-19T07:48:51.600815+0000","last_undegraded":"2024-09-19T07:48:51.600815+0000","last_fullsized":"2024-09-19T07:48:51.600815+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:58.590631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601128+0000","last_change":"2024-09-19T07:48:49.592362+0000","last_active":"2024-09-19T07:48:51.601128+0000","last_peered":"2024-09-19T07:48:51.601128+0000","last_clean":"2024-09-19T07:48:51.601128+0000","last_became_active":"2024-09-19T07:48:49.591829+0000","last_became_peered":"2024-09-19T07:48:49.591829+0000","last_unstale":"2024-09-19T07:48:51.601128+0000","last_undegraded":"2024-09-19T07:48:51.601128+0000","last_fullsized":"2024-09-19T07:48:51.601128+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:08:30.501080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723592+0000","last_change":"2024-09-19T07:48:50.613080+0000","last_active":"2024-09-19T07:48:51.723592+0000","last_peered":"2024-09-19T07:48:51.723592+0000","last_clean":"2024-09-19T07:48:51.723592+0000","last_became_active":"2024-09-19T07:48:50.612625+0000","last_became_peered":"2024-09-19T07:48:50.612625+0000","last_unstale":"2024-09-19T07:48:51.723592+0000","last_undegraded":"2024-09-19T07:48:51.723592+0000","last_fullsized":"2024-09-19T07:48:51.723592+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:11.968589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724498+0000","last_change":"2024-09-19T07:48:49.603802+0000","last_active":"2024-09-19T07:48:51.724498+0000","last_peered":"2024-09-19T07:48:51.724498+0000","last_clean":"2024-09-19T07:48:51.724498+0000","last_became_active":"2024-09-19T07:48:49.602459+0000","last_became_peered":"2024-09-19T07:48:49.602459+0000","last_unstale":"2024-09-19T07:48:51.724498+0000","last_undegraded":"2024-09-19T07:48:51.724498+0000","last_fullsized":"2024-09-19T07:48:51.724498+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:08:30.501080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597396+0000","last_change":"2024-09-19T07:48:50.617463+0000","last_active":"2024-09-19T07:48:51.597396+0000","last_peered":"2024-09-19T07:48:51.597396+0000","last_clean":"2024-09-19T07:48:51.597396+0000","last_became_active":"2024-09-19T07:48:50.617128+0000","last_became_peered":"2024-09-19T07:48:50.617128+0000","last_unstale":"2024-09-19T07:48:51.597396+0000","last_undegraded":"2024-09-19T07:48:51.597396+0000","last_fullsized":"2024-09-19T07:48:51.597396+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:51.108656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"69'8","reported_seq":22,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601801+0000","last_change":"2024-09-19T07:48:49.601117+0000","last_active":"2024-09-19T07:48:51.601801+0000","last_peered":"2024-09-19T07:48:51.601801+0000","last_clean":"2024-09-19T07:48:51.601801+0000","last_became_active":"2024-09-19T07:48:49.600793+0000","last_became_peered":"2024-09-19T07:48:49.600793+0000","last_unstale":"2024-09-19T07:48:51.601801+0000","last_undegraded":"2024-09-19T07:48:51.601801+0000","last_fullsized":"2024-09-19T07:48:51.601801+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:28.988556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723714+0000","last_change":"2024-09-19T07:48:50.610704+0000","last_active":"2024-09-19T07:48:51.723714+0000","last_peered":"2024-09-19T07:48:51.723714+0000","last_clean":"2024-09-19T07:48:51.723714+0000","last_became_active":"2024-09-19T07:48:50.610197+0000","last_became_peered":"2024-09-19T07:48:50.610197+0000","last_unstale":"2024-09-19T07:48:51.723714+0000","last_undegraded":"2024-09-19T07:48:51.723714+0000","last_fullsized":"2024-09-19T07:48:51.723714+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:26.777068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601596+0000","last_change":"2024-09-19T07:48:49.595123+0000","last_active":"2024-09-19T07:48:51.601596+0000","last_peered":"2024-09-19T07:48:51.601596+0000","last_clean":"2024-09-19T07:48:51.601596+0000","last_became_active":"2024-09-19T07:48:49.594733+0000","last_became_peered":"2024-09-19T07:48:49.594733+0000","last_unstale":"2024-09-19T07:48:51.601596+0000","last_undegraded":"2024-09-19T07:48:51.601596+0000","last_fullsized":"2024-09-19T07:48:51.601596+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:51.351283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723827+0000","last_change":"2024-09-19T07:48:50.614642+0000","last_active":"2024-09-19T07:48:51.723827+0000","last_peered":"2024-09-19T07:48:51.723827+0000","last_clean":"2024-09-19T07:48:51.723827+0000","last_became_active":"2024-09-19T07:48:50.614370+0000","last_became_peered":"2024-09-19T07:48:50.614370+0000","last_unstale":"2024-09-19T07:48:51.723827+0000","last_undegraded":"2024-09-19T07:48:51.723827+0000","last_fullsized":"2024-09-19T07:48:51.723827+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:11.570438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724128+0000","last_change":"2024-09-19T07:48:49.604264+0000","last_active":"2024-09-19T07:48:51.724128+0000","last_peered":"2024-09-19T07:48:51.724128+0000","last_clean":"2024-09-19T07:48:51.724128+0000","last_became_active":"2024-09-19T07:48:49.603395+0000","last_became_peered":"2024-09-19T07:48:49.603395+0000","last_unstale":"2024-09-19T07:48:51.724128+0000","last_undegraded":"2024-09-19T07:48:51.724128+0000","last_fullsized":"2024-09-19T07:48:51.724128+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:31.766687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601811+0000","last_change":"2024-09-19T07:48:50.616633+0000","last_active":"2024-09-19T07:48:51.601811+0000","last_peered":"2024-09-19T07:48:51.601811+0000","last_clean":"2024-09-19T07:48:51.601811+0000","last_became_active":"2024-09-19T07:48:50.615974+0000","last_became_peered":"2024-09-19T07:48:50.615974+0000","last_unstale":"2024-09-19T07:48:51.601811+0000","last_undegraded":"2024-09-19T07:48:51.601811+0000","last_fullsized":"2024-09-19T07:48:51.601811+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:41.628823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724085+0000","last_change":"2024-09-19T07:48:49.604469+0000","last_active":"2024-09-19T07:48:51.724085+0000","last_peered":"2024-09-19T07:48:51.724085+0000","last_clean":"2024-09-19T07:48:51.724085+0000","last_became_active":"2024-09-19T07:48:49.604187+0000","last_became_peered":"2024-09-19T07:48:49.604187+0000","last_unstale":"2024-09-19T07:48:51.724085+0000","last_undegraded":"2024-09-19T07:48:51.724085+0000","last_fullsized":"2024-09-19T07:48:51.724085+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:10.460218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724253+0000","last_change":"2024-09-19T07:48:50.618311+0000","last_active":"2024-09-19T07:48:51.724253+0000","last_peered":"2024-09-19T07:48:51.724253+0000","last_clean":"2024-09-19T07:48:51.724253+0000","last_became_active":"2024-09-19T07:48:50.617821+0000","last_became_peered":"2024-09-19T07:48:50.617821+0000","last_unstale":"2024-09-19T07:48:51.724253+0000","last_undegraded":"2024-09-19T07:48:51.724253+0000","last_fullsized":"2024-09-19T07:48:51.724253+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:06.848583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723935+0000","last_change":"2024-09-19T07:48:49.598870+0000","last_active":"2024-09-19T07:48:51.723935+0000","last_peered":"2024-09-19T07:48:51.723935+0000","last_clean":"2024-09-19T07:48:51.723935+0000","last_became_active":"2024-09-19T07:48:49.598519+0000","last_became_peered":"2024-09-19T07:48:49.598519+0000","last_unstale":"2024-09-19T07:48:51.723935+0000","last_undegraded":"2024-09-19T07:48:51.723935+0000","last_fullsized":"2024-09-19T07:48:51.723935+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:18.693182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723262+0000","last_change":"2024-09-19T07:48:50.612364+0000","last_active":"2024-09-19T07:48:51.723262+0000","last_peered":"2024-09-19T07:48:51.723262+0000","last_clean":"2024-09-19T07:48:51.723262+0000","last_became_active":"2024-09-19T07:48:50.611087+0000","last_became_peered":"2024-09-19T07:48:50.611087+0000","last_unstale":"2024-09-19T07:48:51.723262+0000","last_undegraded":"2024-09-19T07:48:51.723262+0000","last_fullsized":"2024-09-19T07:48:51.723262+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:16.152040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723397+0000","last_change":"2024-09-19T07:48:49.595532+0000","last_active":"2024-09-19T07:48:51.723397+0000","last_peered":"2024-09-19T07:48:51.723397+0000","last_clean":"2024-09-19T07:48:51.723397+0000","last_became_active":"2024-09-19T07:48:49.595176+0000","last_became_peered":"2024-09-19T07:48:49.595176+0000","last_unstale":"2024-09-19T07:48:51.723397+0000","last_undegraded":"2024-09-19T07:48:51.723397+0000","last_fullsized":"2024-09-19T07:48:51.723397+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:50.104285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.600867+0000","last_change":"2024-09-19T07:48:50.604743+0000","last_active":"2024-09-19T07:48:51.600867+0000","last_peered":"2024-09-19T07:48:51.600867+0000","last_clean":"2024-09-19T07:48:51.600867+0000","last_became_active":"2024-09-19T07:48:50.604459+0000","last_became_peered":"2024-09-19T07:48:50.604459+0000","last_unstale":"2024-09-19T07:48:51.600867+0000","last_undegraded":"2024-09-19T07:48:51.600867+0000","last_fullsized":"2024-09-19T07:48:51.600867+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:26.777068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601223+0000","last_change":"2024-09-19T07:48:49.592097+0000","last_active":"2024-09-19T07:48:51.601223+0000","last_peered":"2024-09-19T07:48:51.601223+0000","last_clean":"2024-09-19T07:48:51.601223+0000","last_became_active":"2024-09-19T07:48:49.591470+0000","last_became_peered":"2024-09-19T07:48:49.591470+0000","last_unstale":"2024-09-19T07:48:51.601223+0000","last_undegraded":"2024-09-19T07:48:51.601223+0000","last_fullsized":"2024-09-19T07:48:51.601223+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:52.157019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723873+0000","last_change":"2024-09-19T07:48:50.620944+0000","last_active":"2024-09-19T07:48:51.723873+0000","last_peered":"2024-09-19T07:48:51.723873+0000","last_clean":"2024-09-19T07:48:51.723873+0000","last_became_active":"2024-09-19T07:48:50.620554+0000","last_became_peered":"2024-09-19T07:48:50.620554+0000","last_unstale":"2024-09-19T07:48:51.723873+0000","last_undegraded":"2024-09-19T07:48:51.723873+0000","last_fullsized":"2024-09-19T07:48:51.723873+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:23:24.632533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723207+0000","last_change":"2024-09-19T07:48:49.602003+0000","last_active":"2024-09-19T07:48:51.723207+0000","last_peered":"2024-09-19T07:48:51.723207+0000","last_clean":"2024-09-19T07:48:51.723207+0000","last_became_active":"2024-09-19T07:48:49.601652+0000","last_became_peered":"2024-09-19T07:48:49.601652+0000","last_unstale":"2024-09-19T07:48:51.723207+0000","last_undegraded":"2024-09-19T07:48:51.723207+0000","last_fullsized":"2024-09-19T07:48:51.723207+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:31.766687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723505+0000","last_change":"2024-09-19T07:48:50.621641+0000","last_active":"2024-09-19T07:48:51.723505+0000","last_peered":"2024-09-19T07:48:51.723505+0000","last_clean":"2024-09-19T07:48:51.723505+0000","last_became_active":"2024-09-19T07:48:50.621021+0000","last_became_peered":"2024-09-19T07:48:50.621021+0000","last_unstale":"2024-09-19T07:48:51.723505+0000","last_undegraded":"2024-09-19T07:48:51.723505+0000","last_fullsized":"2024-09-19T07:48:51.723505+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:26.777068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601706+0000","last_change":"2024-09-19T07:48:49.602206+0000","last_active":"2024-09-19T07:48:51.601706+0000","last_peered":"2024-09-19T07:48:51.601706+0000","last_clean":"2024-09-19T07:48:51.601706+0000","last_became_active":"2024-09-19T07:48:49.601925+0000","last_became_peered":"2024-09-19T07:48:49.601925+0000","last_unstale":"2024-09-19T07:48:51.601706+0000","last_undegraded":"2024-09-19T07:48:51.601706+0000","last_fullsized":"2024-09-19T07:48:51.601706+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:50.104285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.622119+0000","last_change":"2024-09-19T07:48:50.613584+0000","last_active":"2024-09-19T07:48:50.622119+0000","last_peered":"2024-09-19T07:48:50.622119+0000","last_clean":"2024-09-19T07:48:50.622119+0000","last_became_active":"2024-09-19T07:48:50.612698+0000","last_became_peered":"2024-09-19T07:48:50.612698+0000","last_unstale":"2024-09-19T07:48:50.622119+0000","last_undegraded":"2024-09-19T07:48:50.622119+0000","last_fullsized":"2024-09-19T07:48:50.622119+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:51.388918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"70'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601388+0000","last_change":"2024-09-19T07:48:49.588889+0000","last_active":"2024-09-19T07:48:51.601388+0000","last_peered":"2024-09-19T07:48:51.601388+0000","last_clean":"2024-09-19T07:48:51.601388+0000","last_became_active":"2024-09-19T07:48:49.588216+0000","last_became_peered":"2024-09-19T07:48:49.588216+0000","last_unstale":"2024-09-19T07:48:51.601388+0000","last_undegraded":"2024-09-19T07:48:51.601388+0000","last_fullsized":"2024-09-19T07:48:51.601388+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:10.460218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601121+0000","last_change":"2024-09-19T07:48:50.618857+0000","last_active":"2024-09-19T07:48:51.601121+0000","last_peered":"2024-09-19T07:48:51.601121+0000","last_clean":"2024-09-19T07:48:51.601121+0000","last_became_active":"2024-09-19T07:48:50.618565+0000","last_became_peered":"2024-09-19T07:48:50.618565+0000","last_unstale":"2024-09-19T07:48:51.601121+0000","last_undegraded":"2024-09-19T07:48:51.601121+0000","last_fullsized":"2024-09-19T07:48:51.601121+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:28.290169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723842+0000","last_change":"2024-09-19T07:48:49.597252+0000","last_active":"2024-09-19T07:48:51.723842+0000","last_peered":"2024-09-19T07:48:51.723842+0000","last_clean":"2024-09-19T07:48:51.723842+0000","last_became_active":"2024-09-19T07:48:49.596983+0000","last_became_peered":"2024-09-19T07:48:49.596983+0000","last_unstale":"2024-09-19T07:48:51.723842+0000","last_undegraded":"2024-09-19T07:48:51.723842+0000","last_fullsized":"2024-09-19T07:48:51.723842+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:06.787582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724421+0000","last_change":"2024-09-19T07:48:50.618096+0000","last_active":"2024-09-19T07:48:51.724421+0000","last_peered":"2024-09-19T07:48:51.724421+0000","last_clean":"2024-09-19T07:48:51.724421+0000","last_became_active":"2024-09-19T07:48:50.617456+0000","last_became_peered":"2024-09-19T07:48:50.617456+0000","last_unstale":"2024-09-19T07:48:51.724421+0000","last_undegraded":"2024-09-19T07:48:51.724421+0000","last_fullsized":"2024-09-19T07:48:51.724421+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:07.353888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597678+0000","last_change":"2024-09-19T07:48:49.599857+0000","last_active":"2024-09-19T07:48:51.597678+0000","last_peered":"2024-09-19T07:48:51.597678+0000","last_clean":"2024-09-19T07:48:51.597678+0000","last_became_active":"2024-09-19T07:48:49.599550+0000","last_became_peered":"2024-09-19T07:48:49.599550+0000","last_unstale":"2024-09-19T07:48:51.597678+0000","last_undegraded":"2024-09-19T07:48:51.597678+0000","last_fullsized":"2024-09-19T07:48:51.597678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:05.718163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723372+0000","last_change":"2024-09-19T07:48:50.623756+0000","last_active":"2024-09-19T07:48:51.723372+0000","last_peered":"2024-09-19T07:48:51.723372+0000","last_clean":"2024-09-19T07:48:51.723372+0000","last_became_active":"2024-09-19T07:48:50.623440+0000","last_became_peered":"2024-09-19T07:48:50.623440+0000","last_unstale":"2024-09-19T07:48:51.723372+0000","last_undegraded":"2024-09-19T07:48:51.723372+0000","last_fullsized":"2024-09-19T07:48:51.723372+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:02.116554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724246+0000","last_change":"2024-09-19T07:48:49.603929+0000","last_active":"2024-09-19T07:48:51.724246+0000","last_peered":"2024-09-19T07:48:51.724246+0000","last_clean":"2024-09-19T07:48:51.724246+0000","last_became_active":"2024-09-19T07:48:49.603208+0000","last_became_peered":"2024-09-19T07:48:49.603208+0000","last_unstale":"2024-09-19T07:48:51.724246+0000","last_undegraded":"2024-09-19T07:48:51.724246+0000","last_fullsized":"2024-09-19T07:48:51.724246+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:51.351283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601099+0000","last_change":"2024-09-19T07:48:50.608905+0000","last_active":"2024-09-19T07:48:51.601099+0000","last_peered":"2024-09-19T07:48:51.601099+0000","last_clean":"2024-09-19T07:48:51.601099+0000","last_became_active":"2024-09-19T07:48:50.608632+0000","last_became_peered":"2024-09-19T07:48:50.608632+0000","last_unstale":"2024-09-19T07:48:51.601099+0000","last_undegraded":"2024-09-19T07:48:51.601099+0000","last_fullsized":"2024-09-19T07:48:51.601099+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:51.090644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724018+0000","last_change":"2024-09-19T07:48:49.593753+0000","last_active":"2024-09-19T07:48:51.724018+0000","last_peered":"2024-09-19T07:48:51.724018+0000","last_clean":"2024-09-19T07:48:51.724018+0000","last_became_active":"2024-09-19T07:48:49.592814+0000","last_became_peered":"2024-09-19T07:48:49.592814+0000","last_unstale":"2024-09-19T07:48:51.724018+0000","last_undegraded":"2024-09-19T07:48:51.724018+0000","last_fullsized":"2024-09-19T07:48:51.724018+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:48.330162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597219+0000","last_change":"2024-09-19T07:48:50.619443+0000","last_active":"2024-09-19T07:48:51.597219+0000","last_peered":"2024-09-19T07:48:51.597219+0000","last_clean":"2024-09-19T07:48:51.597219+0000","last_became_active":"2024-09-19T07:48:50.619073+0000","last_became_peered":"2024-09-19T07:48:50.619073+0000","last_unstale":"2024-09-19T07:48:51.597219+0000","last_undegraded":"2024-09-19T07:48:51.597219+0000","last_fullsized":"2024-09-19T07:48:51.597219+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:52.355654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723700+0000","last_change":"2024-09-19T07:48:49.594762+0000","last_active":"2024-09-19T07:48:51.723700+0000","last_peered":"2024-09-19T07:48:51.723700+0000","last_clean":"2024-09-19T07:48:51.723700+0000","last_became_active":"2024-09-19T07:48:49.594420+0000","last_became_peered":"2024-09-19T07:48:49.594420+0000","last_unstale":"2024-09-19T07:48:51.723700+0000","last_undegraded":"2024-09-19T07:48:51.723700+0000","last_fullsized":"2024-09-19T07:48:51.723700+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:03:01.422142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-19T07:48:50.608930+0000","last_change":"2024-09-19T07:48:50.604249+0000","last_active":"2024-09-19T07:48:50.608930+0000","last_peered":"2024-09-19T07:48:50.608930+0000","last_clean":"2024-09-19T07:48:50.608930+0000","last_became_active":"2024-09-19T07:48:50.603870+0000","last_became_peered":"2024-09-19T07:48:50.603870+0000","last_unstale":"2024-09-19T07:48:50.608930+0000","last_undegraded":"2024-09-19T07:48:50.608930+0000","last_fullsized":"2024-09-19T07:48:50.608930+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:17.668476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597187+0000","last_change":"2024-09-19T07:48:49.597016+0000","last_active":"2024-09-19T07:48:51.597187+0000","last_peered":"2024-09-19T07:48:51.597187+0000","last_clean":"2024-09-19T07:48:51.597187+0000","last_became_active":"2024-09-19T07:48:49.596725+0000","last_became_peered":"2024-09-19T07:48:49.596725+0000","last_unstale":"2024-09-19T07:48:51.597187+0000","last_undegraded":"2024-09-19T07:48:51.597187+0000","last_fullsized":"2024-09-19T07:48:51.597187+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:59.852448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723831+0000","last_change":"2024-09-19T07:48:50.623507+0000","last_active":"2024-09-19T07:48:51.723831+0000","last_peered":"2024-09-19T07:48:51.723831+0000","last_clean":"2024-09-19T07:48:51.723831+0000","last_became_active":"2024-09-19T07:48:50.623221+0000","last_became_peered":"2024-09-19T07:48:50.623221+0000","last_unstale":"2024-09-19T07:48:51.723831+0000","last_undegraded":"2024-09-19T07:48:51.723831+0000","last_fullsized":"2024-09-19T07:48:51.723831+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:41.628823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723423+0000","last_change":"2024-09-19T07:48:49.598225+0000","last_active":"2024-09-19T07:48:51.723423+0000","last_peered":"2024-09-19T07:48:51.723423+0000","last_clean":"2024-09-19T07:48:51.723423+0000","last_became_active":"2024-09-19T07:48:49.597938+0000","last_became_peered":"2024-09-19T07:48:49.597938+0000","last_unstale":"2024-09-19T07:48:51.723423+0000","last_undegraded":"2024-09-19T07:48:51.723423+0000","last_fullsized":"2024-09-19T07:48:51.723423+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:02:14.020905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":259,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723642+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:48:51.723642+0000","last_peered":"2024-09-19T07:48:51.723642+0000","last_clean":"2024-09-19T07:48:51.723642+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:48:51.723642+0000","last_undegraded":"2024-09-19T07:48:51.723642+0000","last_fullsized":"2024-09-19T07:48:51.723642+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509515,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95008,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":93566,"kb_avail":94276832,"statfs":{"total":96636764160,"available":96539475968,"internally_reserved":0,"allocated":1449984,"data_stored":974233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":95812042},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":112524,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94259316,"statfs":{"total":96636764160,"available":96521539584,"internally_reserved":0,"allocated":843776,"data_stored":383123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":114358726},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509515,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50060,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321780,"statfs":{"total":96636764160,"available":96585502720,"internally_reserved":0,"allocated":843776,"data_stored":383111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509515,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62548,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309292,"statfs":{"total":96636764160,"available":96572715008,"internally_reserved":0,"allocated":851968,"data_stored":384701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509514,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100564,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271276,"statfs":{"total":96636764160,"available":96533786624,"internally_reserved":0,"allocated":1433600,"data_stored":973457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542218,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130504,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94241336,"statfs":{"total":96636764160,"available":96503128064,"internally_reserved":0,"allocated":839680,"data_stored":383089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542218,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94504,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94277336,"statfs":{"total":96636764160,"available":96539992064,"internally_reserved":0,"allocated":868352,"data_stored":385465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104116,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267724,"statfs":{"total":96636764160,"available":96530149376,"internally_reserved":0,"allocated":815104,"data_stored":369230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:48:56.578 INFO:tasks.ceph.mds.a.smithi016.stderr:2024-09-19T07:48:56.578+0000 7f15350c9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T07:48:58.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:48:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.131+0000 7f5a584e4640 1 Processor -- start 2024-09-19T07:48:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.131+0000 7f5a584e4640 1 -- start start 2024-09-19T07:48:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.132+0000 7f5a584e4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a5015e2a0 0x7f5a5006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.132+0000 7f5a584e4640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5a5015e680 0x7f5a500cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.132+0000 7f5a584e4640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a500d1880 con 0x7f5a5006d960 2024-09-19T07:48:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.132+0000 7f5a584e4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f5a500d1b60 con 0x7f5a5015e2a0 2024-09-19T07:48:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.132+0000 7f5a584e4640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5a500cf2d0 con 0x7f5a5015e680 2024-09-19T07:48:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.132+0000 7f5a56280640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a5015e2a0 0x7f5a5006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.132+0000 7f5a56280640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a5015e2a0 0x7f5a5006d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59402/0 (socket says 172.21.15.16:59402) 2024-09-19T07:48:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.132+0000 7f5a56280640 1 -- 172.21.15.16:0/4258128452 learned_addr learned my addr 172.21.15.16:0/4258128452 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.132+0000 7f5a55a7f640 1 --2- 172.21.15.16:0/4258128452 >> v2:172.21.15.146:3300/0 conn(0x7f5a5015e680 0x7f5a500cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.132+0000 7f5a56280640 1 -- 172.21.15.16:0/4258128452 >> v2:172.21.15.146:3300/0 conn(0x7f5a5015e680 msgr2=0x7f5a500cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.132+0000 7f5a56280640 1 --2- 172.21.15.16:0/4258128452 >> v2:172.21.15.146:3300/0 conn(0x7f5a5015e680 0x7f5a500cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.132+0000 7f5a56280640 1 -- 172.21.15.16:0/4258128452 >> v1:172.21.15.146:6789/0 conn(0x7f5a5006d960 legacy=0x7f5a500d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.132+0000 7f5a56280640 1 -- 172.21.15.16:0/4258128452 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a500cf5b0 con 0x7f5a5015e2a0 2024-09-19T07:48:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.132+0000 7f5a55a7f640 1 --2- 172.21.15.16:0/4258128452 >> v2:172.21.15.146:3300/0 conn(0x7f5a5015e680 0x7f5a500cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.133+0000 7f5a56280640 1 --2- 172.21.15.16:0/4258128452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a5015e2a0 0x7f5a5006d420 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5a500a22a0 tx=0x7f5a4c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb20d33dc3ac45a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.133+0000 7f5a5527e640 1 -- 172.21.15.16:0/4258128452 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c037070 con 0x7f5a5015e2a0 2024-09-19T07:48:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.133+0000 7f5a5527e640 1 -- 172.21.15.16:0/4258128452 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a4c032440 con 0x7f5a5015e2a0 2024-09-19T07:48:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.133+0000 7f5a5527e640 1 -- 172.21.15.16:0/4258128452 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c032810 con 0x7f5a5015e2a0 2024-09-19T07:48:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.134+0000 7f5a584e4640 1 -- 172.21.15.16:0/4258128452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a5015e2a0 msgr2=0x7f5a5006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.134+0000 7f5a584e4640 1 --2- 172.21.15.16:0/4258128452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a5015e2a0 0x7f5a5006d420 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5a500a22a0 tx=0x7f5a4c02a270 comp rx=0 tx=0).stop 2024-09-19T07:48:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.134+0000 7f5a584e4640 1 -- 172.21.15.16:0/4258128452 shutdown_connections 2024-09-19T07:48:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.134+0000 7f5a584e4640 1 --2- 172.21.15.16:0/4258128452 >> v2:172.21.15.146:3300/0 conn(0x7f5a5015e680 0x7f5a500cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.134+0000 7f5a584e4640 1 --2- 172.21.15.16:0/4258128452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a5015e2a0 0x7f5a5006d420 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.134+0000 7f5a584e4640 1 -- 172.21.15.16:0/4258128452 >> 172.21.15.16:0/4258128452 conn(0x7f5a5009d760 msgr2=0x7f5a5008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.134+0000 7f5a584e4640 1 -- 172.21.15.16:0/4258128452 shutdown_connections 2024-09-19T07:48:58.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.134+0000 7f5a584e4640 1 -- 172.21.15.16:0/4258128452 wait complete. 2024-09-19T07:48:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.135+0000 7f5a584e4640 1 Processor -- start 2024-09-19T07:48:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.135+0000 7f5a584e4640 1 -- start start 2024-09-19T07:48:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.136+0000 7f5a584e4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a5006d960 0x7f5a5015c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.136+0000 7f5a584e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a5015e2a0 0x7f5a500a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.136+0000 7f5a584e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a5015e680 0x7f5a50160e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.136+0000 7f5a584e4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a500d4c00 con 0x7f5a5015e680 2024-09-19T07:48:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.136+0000 7f5a584e4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5a500d4a80 con 0x7f5a5015e2a0 2024-09-19T07:48:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.136+0000 7f5a584e4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5a500d0010 con 0x7f5a5006d960 2024-09-19T07:48:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.136+0000 7f5a56a81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a5015e680 0x7f5a50160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.136+0000 7f5a56280640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a5006d960 0x7f5a5015c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.136+0000 7f5a56a81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a5015e680 0x7f5a50160e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59408/0 (socket says 172.21.15.16:59408) 2024-09-19T07:48:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.136+0000 7f5a56a81640 1 -- 172.21.15.16:0/3933358807 learned_addr learned my addr 172.21.15.16:0/3933358807 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.136+0000 7f5a55a7f640 1 --2- 172.21.15.16:0/3933358807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a5015e2a0 0x7f5a500a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.136+0000 7f5a56a81640 1 -- 172.21.15.16:0/3933358807 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a5006d960 msgr2=0x7f5a5015c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.136+0000 7f5a56a81640 1 --2- 172.21.15.16:0/3933358807 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a5006d960 0x7f5a5015c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.136+0000 7f5a56a81640 1 -- 172.21.15.16:0/3933358807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a5015e2a0 msgr2=0x7f5a500a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.136+0000 7f5a56a81640 1 --2- 172.21.15.16:0/3933358807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a5015e2a0 0x7f5a500a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.136+0000 7f5a56a81640 1 -- 172.21.15.16:0/3933358807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a500a10b0 con 0x7f5a5015e680 2024-09-19T07:48:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.137+0000 7f5a56a81640 1 --2- 172.21.15.16:0/3933358807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a5015e680 0x7f5a50160e70 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5a4800c7a0 tx=0x7f5a4800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.137+0000 7f5a46ffd640 1 -- 172.21.15.16:0/3933358807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a48003b70 con 0x7f5a5015e680 2024-09-19T07:48:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.137+0000 7f5a46ffd640 1 -- 172.21.15.16:0/3933358807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a48003d30 con 0x7f5a5015e680 2024-09-19T07:48:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.137+0000 7f5a584e4640 1 -- 172.21.15.16:0/3933358807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a500a1390 con 0x7f5a5015e680 2024-09-19T07:48:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.137+0000 7f5a46ffd640 1 -- 172.21.15.16:0/3933358807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4800aa90 con 0x7f5a5015e680 2024-09-19T07:48:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.137+0000 7f5a584e4640 1 -- 172.21.15.16:0/3933358807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a500a1950 con 0x7f5a5015e680 2024-09-19T07:48:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.138+0000 7f5a584e4640 1 -- 172.21.15.16:0/3933358807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a500b8390 con 0x7f5a5015e680 2024-09-19T07:48:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.138+0000 7f5a46ffd640 1 -- 172.21.15.16:0/3933358807 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5a4800ac30 con 0x7f5a5015e680 2024-09-19T07:48:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.139+0000 7f5a46ffd640 1 --2- 172.21.15.16:0/3933358807 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a3406c2b0 0x7f5a3406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.139+0000 7f5a46ffd640 1 -- 172.21.15.16:0/3933358807 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f5a4808b3d0 con 0x7f5a5015e680 2024-09-19T07:48:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.139+0000 7f5a56280640 1 --2- 172.21.15.16:0/3933358807 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a3406c2b0 0x7f5a3406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.140+0000 7f5a56280640 1 --2- 172.21.15.16:0/3933358807 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a3406c2b0 0x7f5a3406e770 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c034000 tx=0x7f5a4c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.143+0000 7f5a46ffd640 1 -- 172.21.15.16:0/3933358807 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5a48059360 con 0x7f5a5015e680 2024-09-19T07:48:58.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.316+0000 7f5a584e4640 1 -- 172.21.15.16:0/3933358807 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a5006d420 con 0x7f5a3406c2b0 2024-09-19T07:48:58.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.322+0000 7f5a46ffd640 1 -- 172.21.15.16:0/3933358807 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252019 (secure 0 0 0) 0x7f5a5006d420 con 0x7f5a3406c2b0 2024-09-19T07:48:58.325 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:48:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.325+0000 7f5a584e4640 1 -- 172.21.15.16:0/3933358807 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a3406c2b0 msgr2=0x7f5a3406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.325+0000 7f5a584e4640 1 --2- 172.21.15.16:0/3933358807 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a3406c2b0 0x7f5a3406e770 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c034000 tx=0x7f5a4c02e000 comp rx=0 tx=0).stop 2024-09-19T07:48:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.325+0000 7f5a584e4640 1 -- 172.21.15.16:0/3933358807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a5015e680 msgr2=0x7f5a50160e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.325+0000 7f5a584e4640 1 --2- 172.21.15.16:0/3933358807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a5015e680 0x7f5a50160e70 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5a4800c7a0 tx=0x7f5a4800cc60 comp rx=0 tx=0).stop 2024-09-19T07:48:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.326+0000 7f5a584e4640 1 -- 172.21.15.16:0/3933358807 shutdown_connections 2024-09-19T07:48:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.326+0000 7f5a584e4640 1 --2- 172.21.15.16:0/3933358807 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a3406c2b0 0x7f5a3406e770 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.326+0000 7f5a584e4640 1 --2- 172.21.15.16:0/3933358807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a5015e680 0x7f5a50160e70 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.326+0000 7f5a584e4640 1 --2- 172.21.15.16:0/3933358807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a5015e2a0 0x7f5a500a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.326+0000 7f5a584e4640 1 --2- 172.21.15.16:0/3933358807 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a5006d960 0x7f5a5015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.326+0000 7f5a584e4640 1 -- 172.21.15.16:0/3933358807 >> 172.21.15.16:0/3933358807 conn(0x7f5a5009d760 msgr2=0x7f5a500d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.326+0000 7f5a584e4640 1 -- 172.21.15.16:0/3933358807 shutdown_connections 2024-09-19T07:48:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.326+0000 7f5a584e4640 1 -- 172.21.15.16:0/3933358807 wait complete. 2024-09-19T07:48:58.337 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-19T07:48:56.535208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":749852,"kb_used_data":7784,"kb_used_omap":12,"kb_used_meta":741875,"kb_avail":754224868,"statfs":{"total":773094113280,"available":772326264832,"internally_reserved":0,"allocated":7970816,"data_stored":4250268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":759680560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.953918"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597587+0000","last_change":"2024-09-19T07:48:50.617279+0000","last_active":"2024-09-19T07:48:51.597587+0000","last_peered":"2024-09-19T07:48:51.597587+0000","last_clean":"2024-09-19T07:48:51.597587+0000","last_became_active":"2024-09-19T07:48:50.616175+0000","last_became_peered":"2024-09-19T07:48:50.616175+0000","last_unstale":"2024-09-19T07:48:51.597587+0000","last_undegraded":"2024-09-19T07:48:51.597587+0000","last_fullsized":"2024-09-19T07:48:51.597587+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:07.791953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597582+0000","last_change":"2024-09-19T07:48:50.618861+0000","last_active":"2024-09-19T07:48:51.597582+0000","last_peered":"2024-09-19T07:48:51.597582+0000","last_clean":"2024-09-19T07:48:51.597582+0000","last_became_active":"2024-09-19T07:48:50.618594+0000","last_became_peered":"2024-09-19T07:48:50.618594+0000","last_unstale":"2024-09-19T07:48:51.597582+0000","last_undegraded":"2024-09-19T07:48:51.597582+0000","last_fullsized":"2024-09-19T07:48:51.597582+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:32.771059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723962+0000","last_change":"2024-09-19T07:48:50.619839+0000","last_active":"2024-09-19T07:48:51.723962+0000","last_peered":"2024-09-19T07:48:51.723962+0000","last_clean":"2024-09-19T07:48:51.723962+0000","last_became_active":"2024-09-19T07:48:50.619499+0000","last_became_peered":"2024-09-19T07:48:50.619499+0000","last_unstale":"2024-09-19T07:48:51.723962+0000","last_undegraded":"2024-09-19T07:48:51.723962+0000","last_fullsized":"2024-09-19T07:48:51.723962+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:28.691031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601667+0000","last_change":"2024-09-19T07:48:50.618256+0000","last_active":"2024-09-19T07:48:51.601667+0000","last_peered":"2024-09-19T07:48:51.601667+0000","last_clean":"2024-09-19T07:48:51.601667+0000","last_became_active":"2024-09-19T07:48:50.617581+0000","last_became_peered":"2024-09-19T07:48:50.617581+0000","last_unstale":"2024-09-19T07:48:51.601667+0000","last_undegraded":"2024-09-19T07:48:51.601667+0000","last_fullsized":"2024-09-19T07:48:51.601667+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:23:24.632533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723687+0000","last_change":"2024-09-19T07:48:50.613364+0000","last_active":"2024-09-19T07:48:51.723687+0000","last_peered":"2024-09-19T07:48:51.723687+0000","last_clean":"2024-09-19T07:48:51.723687+0000","last_became_active":"2024-09-19T07:48:50.611676+0000","last_became_peered":"2024-09-19T07:48:50.611676+0000","last_unstale":"2024-09-19T07:48:51.723687+0000","last_undegraded":"2024-09-19T07:48:51.723687+0000","last_fullsized":"2024-09-19T07:48:51.723687+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:31.662628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723769+0000","last_change":"2024-09-19T07:48:50.622683+0000","last_active":"2024-09-19T07:48:51.723769+0000","last_peered":"2024-09-19T07:48:51.723769+0000","last_clean":"2024-09-19T07:48:51.723769+0000","last_became_active":"2024-09-19T07:48:50.622075+0000","last_became_peered":"2024-09-19T07:48:50.622075+0000","last_unstale":"2024-09-19T07:48:51.723769+0000","last_undegraded":"2024-09-19T07:48:51.723769+0000","last_fullsized":"2024-09-19T07:48:51.723769+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:28.691031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.618153+0000","last_change":"2024-09-19T07:48:50.609214+0000","last_active":"2024-09-19T07:48:51.618153+0000","last_peered":"2024-09-19T07:48:51.618153+0000","last_clean":"2024-09-19T07:48:51.618153+0000","last_became_active":"2024-09-19T07:48:50.608804+0000","last_became_peered":"2024-09-19T07:48:50.608804+0000","last_unstale":"2024-09-19T07:48:51.618153+0000","last_undegraded":"2024-09-19T07:48:51.618153+0000","last_fullsized":"2024-09-19T07:48:51.618153+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:59.745266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601391+0000","last_change":"2024-09-19T07:48:50.614939+0000","last_active":"2024-09-19T07:48:51.601391+0000","last_peered":"2024-09-19T07:48:51.601391+0000","last_clean":"2024-09-19T07:48:51.601391+0000","last_became_active":"2024-09-19T07:48:50.614661+0000","last_became_peered":"2024-09-19T07:48:50.614661+0000","last_unstale":"2024-09-19T07:48:51.601391+0000","last_undegraded":"2024-09-19T07:48:51.601391+0000","last_fullsized":"2024-09-19T07:48:51.601391+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:10.375253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724172+0000","last_change":"2024-09-19T07:48:50.611473+0000","last_active":"2024-09-19T07:48:51.724172+0000","last_peered":"2024-09-19T07:48:51.724172+0000","last_clean":"2024-09-19T07:48:51.724172+0000","last_became_active":"2024-09-19T07:48:50.611205+0000","last_became_peered":"2024-09-19T07:48:50.611205+0000","last_unstale":"2024-09-19T07:48:51.724172+0000","last_undegraded":"2024-09-19T07:48:51.724172+0000","last_fullsized":"2024-09-19T07:48:51.724172+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:33.519772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724403+0000","last_change":"2024-09-19T07:48:50.617206+0000","last_active":"2024-09-19T07:48:51.724403+0000","last_peered":"2024-09-19T07:48:51.724403+0000","last_clean":"2024-09-19T07:48:51.724403+0000","last_became_active":"2024-09-19T07:48:50.616917+0000","last_became_peered":"2024-09-19T07:48:50.616917+0000","last_unstale":"2024-09-19T07:48:51.724403+0000","last_undegraded":"2024-09-19T07:48:51.724403+0000","last_fullsized":"2024-09-19T07:48:51.724403+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:39.970307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.600909+0000","last_change":"2024-09-19T07:48:50.606655+0000","last_active":"2024-09-19T07:48:51.600909+0000","last_peered":"2024-09-19T07:48:51.600909+0000","last_clean":"2024-09-19T07:48:51.600909+0000","last_became_active":"2024-09-19T07:48:50.606376+0000","last_became_peered":"2024-09-19T07:48:50.606376+0000","last_unstale":"2024-09-19T07:48:51.600909+0000","last_undegraded":"2024-09-19T07:48:51.600909+0000","last_fullsized":"2024-09-19T07:48:51.600909+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:02.115678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.600960+0000","last_change":"2024-09-19T07:48:50.614911+0000","last_active":"2024-09-19T07:48:51.600960+0000","last_peered":"2024-09-19T07:48:51.600960+0000","last_clean":"2024-09-19T07:48:51.600960+0000","last_became_active":"2024-09-19T07:48:50.614623+0000","last_became_peered":"2024-09-19T07:48:50.614623+0000","last_unstale":"2024-09-19T07:48:51.600960+0000","last_undegraded":"2024-09-19T07:48:51.600960+0000","last_fullsized":"2024-09-19T07:48:51.600960+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:11.128746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.600828+0000","last_change":"2024-09-19T07:48:50.603843+0000","last_active":"2024-09-19T07:48:51.600828+0000","last_peered":"2024-09-19T07:48:51.600828+0000","last_clean":"2024-09-19T07:48:51.600828+0000","last_became_active":"2024-09-19T07:48:50.603521+0000","last_became_peered":"2024-09-19T07:48:50.603521+0000","last_unstale":"2024-09-19T07:48:51.600828+0000","last_undegraded":"2024-09-19T07:48:51.600828+0000","last_fullsized":"2024-09-19T07:48:51.600828+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:16.152040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724418+0000","last_change":"2024-09-19T07:48:50.621010+0000","last_active":"2024-09-19T07:48:51.724418+0000","last_peered":"2024-09-19T07:48:51.724418+0000","last_clean":"2024-09-19T07:48:51.724418+0000","last_became_active":"2024-09-19T07:48:50.619664+0000","last_became_peered":"2024-09-19T07:48:50.619664+0000","last_unstale":"2024-09-19T07:48:51.724418+0000","last_undegraded":"2024-09-19T07:48:51.724418+0000","last_fullsized":"2024-09-19T07:48:51.724418+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:10.375253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724468+0000","last_change":"2024-09-19T07:48:50.616922+0000","last_active":"2024-09-19T07:48:51.724468+0000","last_peered":"2024-09-19T07:48:51.724468+0000","last_clean":"2024-09-19T07:48:51.724468+0000","last_became_active":"2024-09-19T07:48:50.616630+0000","last_became_peered":"2024-09-19T07:48:50.616630+0000","last_unstale":"2024-09-19T07:48:51.724468+0000","last_undegraded":"2024-09-19T07:48:51.724468+0000","last_fullsized":"2024-09-19T07:48:51.724468+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:50.664743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.618132+0000","last_change":"2024-09-19T07:48:50.604348+0000","last_active":"2024-09-19T07:48:51.618132+0000","last_peered":"2024-09-19T07:48:51.618132+0000","last_clean":"2024-09-19T07:48:51.618132+0000","last_became_active":"2024-09-19T07:48:50.604037+0000","last_became_peered":"2024-09-19T07:48:50.604037+0000","last_unstale":"2024-09-19T07:48:51.618132+0000","last_undegraded":"2024-09-19T07:48:51.618132+0000","last_fullsized":"2024-09-19T07:48:51.618132+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:35.846671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597616+0000","last_change":"2024-09-19T07:48:50.606625+0000","last_active":"2024-09-19T07:48:51.597616+0000","last_peered":"2024-09-19T07:48:51.597616+0000","last_clean":"2024-09-19T07:48:51.597616+0000","last_became_active":"2024-09-19T07:48:50.606339+0000","last_became_peered":"2024-09-19T07:48:50.606339+0000","last_unstale":"2024-09-19T07:48:51.597616+0000","last_undegraded":"2024-09-19T07:48:51.597616+0000","last_fullsized":"2024-09-19T07:48:51.597616+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:03:39.985164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601479+0000","last_change":"2024-09-19T07:48:50.616872+0000","last_active":"2024-09-19T07:48:51.601479+0000","last_peered":"2024-09-19T07:48:51.601479+0000","last_clean":"2024-09-19T07:48:51.601479+0000","last_became_active":"2024-09-19T07:48:50.616363+0000","last_became_peered":"2024-09-19T07:48:50.616363+0000","last_unstale":"2024-09-19T07:48:51.601479+0000","last_undegraded":"2024-09-19T07:48:51.601479+0000","last_fullsized":"2024-09-19T07:48:51.601479+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:31.662628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723657+0000","last_change":"2024-09-19T07:48:50.623011+0000","last_active":"2024-09-19T07:48:51.723657+0000","last_peered":"2024-09-19T07:48:51.723657+0000","last_clean":"2024-09-19T07:48:51.723657+0000","last_became_active":"2024-09-19T07:48:50.622611+0000","last_became_peered":"2024-09-19T07:48:50.622611+0000","last_unstale":"2024-09-19T07:48:51.723657+0000","last_undegraded":"2024-09-19T07:48:51.723657+0000","last_fullsized":"2024-09-19T07:48:51.723657+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:00.856819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723831+0000","last_change":"2024-09-19T07:48:50.613355+0000","last_active":"2024-09-19T07:48:51.723831+0000","last_peered":"2024-09-19T07:48:51.723831+0000","last_clean":"2024-09-19T07:48:51.723831+0000","last_became_active":"2024-09-19T07:48:50.612994+0000","last_became_peered":"2024-09-19T07:48:50.612994+0000","last_unstale":"2024-09-19T07:48:51.723831+0000","last_undegraded":"2024-09-19T07:48:51.723831+0000","last_fullsized":"2024-09-19T07:48:51.723831+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:36:28.078297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601169+0000","last_change":"2024-09-19T07:48:50.608752+0000","last_active":"2024-09-19T07:48:51.601169+0000","last_peered":"2024-09-19T07:48:51.601169+0000","last_clean":"2024-09-19T07:48:51.601169+0000","last_became_active":"2024-09-19T07:48:50.606884+0000","last_became_peered":"2024-09-19T07:48:50.606884+0000","last_unstale":"2024-09-19T07:48:51.601169+0000","last_undegraded":"2024-09-19T07:48:51.601169+0000","last_fullsized":"2024-09-19T07:48:51.601169+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:33.519772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723555+0000","last_change":"2024-09-19T07:48:50.608708+0000","last_active":"2024-09-19T07:48:51.723555+0000","last_peered":"2024-09-19T07:48:51.723555+0000","last_clean":"2024-09-19T07:48:51.723555+0000","last_became_active":"2024-09-19T07:48:50.608398+0000","last_became_peered":"2024-09-19T07:48:50.608398+0000","last_unstale":"2024-09-19T07:48:51.723555+0000","last_undegraded":"2024-09-19T07:48:51.723555+0000","last_fullsized":"2024-09-19T07:48:51.723555+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:39:39.078202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723612+0000","last_change":"2024-09-19T07:48:50.608395+0000","last_active":"2024-09-19T07:48:51.723612+0000","last_peered":"2024-09-19T07:48:51.723612+0000","last_clean":"2024-09-19T07:48:51.723612+0000","last_became_active":"2024-09-19T07:48:50.607050+0000","last_became_peered":"2024-09-19T07:48:50.607050+0000","last_unstale":"2024-09-19T07:48:51.723612+0000","last_undegraded":"2024-09-19T07:48:51.723612+0000","last_fullsized":"2024-09-19T07:48:51.723612+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:25.713782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723647+0000","last_change":"2024-09-19T07:48:50.619593+0000","last_active":"2024-09-19T07:48:51.723647+0000","last_peered":"2024-09-19T07:48:51.723647+0000","last_clean":"2024-09-19T07:48:51.723647+0000","last_became_active":"2024-09-19T07:48:50.619318+0000","last_became_peered":"2024-09-19T07:48:50.619318+0000","last_unstale":"2024-09-19T07:48:51.723647+0000","last_undegraded":"2024-09-19T07:48:51.723647+0000","last_fullsized":"2024-09-19T07:48:51.723647+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:16.152040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601542+0000","last_change":"2024-09-19T07:48:50.605776+0000","last_active":"2024-09-19T07:48:51.601542+0000","last_peered":"2024-09-19T07:48:51.601542+0000","last_clean":"2024-09-19T07:48:51.601542+0000","last_became_active":"2024-09-19T07:48:50.605472+0000","last_became_peered":"2024-09-19T07:48:50.605472+0000","last_unstale":"2024-09-19T07:48:51.601542+0000","last_undegraded":"2024-09-19T07:48:51.601542+0000","last_fullsized":"2024-09-19T07:48:51.601542+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:50.664743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601221+0000","last_change":"2024-09-19T07:48:50.606522+0000","last_active":"2024-09-19T07:48:51.601221+0000","last_peered":"2024-09-19T07:48:51.601221+0000","last_clean":"2024-09-19T07:48:51.601221+0000","last_became_active":"2024-09-19T07:48:50.606251+0000","last_became_peered":"2024-09-19T07:48:50.606251+0000","last_unstale":"2024-09-19T07:48:51.601221+0000","last_undegraded":"2024-09-19T07:48:51.601221+0000","last_fullsized":"2024-09-19T07:48:51.601221+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:39.970307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601058+0000","last_change":"2024-09-19T07:48:50.605074+0000","last_active":"2024-09-19T07:48:51.601058+0000","last_peered":"2024-09-19T07:48:51.601058+0000","last_clean":"2024-09-19T07:48:51.601058+0000","last_became_active":"2024-09-19T07:48:50.604749+0000","last_became_peered":"2024-09-19T07:48:50.604749+0000","last_unstale":"2024-09-19T07:48:51.601058+0000","last_undegraded":"2024-09-19T07:48:51.601058+0000","last_fullsized":"2024-09-19T07:48:51.601058+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:02.116554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597433+0000","last_change":"2024-09-19T07:48:50.621418+0000","last_active":"2024-09-19T07:48:51.597433+0000","last_peered":"2024-09-19T07:48:51.597433+0000","last_clean":"2024-09-19T07:48:51.597433+0000","last_became_active":"2024-09-19T07:48:50.620678+0000","last_became_peered":"2024-09-19T07:48:50.620678+0000","last_unstale":"2024-09-19T07:48:51.597433+0000","last_undegraded":"2024-09-19T07:48:51.597433+0000","last_fullsized":"2024-09-19T07:48:51.597433+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:18.120305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723984+0000","last_change":"2024-09-19T07:48:50.615969+0000","last_active":"2024-09-19T07:48:51.723984+0000","last_peered":"2024-09-19T07:48:51.723984+0000","last_clean":"2024-09-19T07:48:51.723984+0000","last_became_active":"2024-09-19T07:48:50.613857+0000","last_became_peered":"2024-09-19T07:48:50.613857+0000","last_unstale":"2024-09-19T07:48:51.723984+0000","last_undegraded":"2024-09-19T07:48:51.723984+0000","last_fullsized":"2024-09-19T07:48:51.723984+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:28.290169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601058+0000","last_change":"2024-09-19T07:48:50.618585+0000","last_active":"2024-09-19T07:48:51.601058+0000","last_peered":"2024-09-19T07:48:51.601058+0000","last_clean":"2024-09-19T07:48:51.601058+0000","last_became_active":"2024-09-19T07:48:50.618277+0000","last_became_peered":"2024-09-19T07:48:50.618277+0000","last_unstale":"2024-09-19T07:48:51.601058+0000","last_undegraded":"2024-09-19T07:48:51.601058+0000","last_fullsized":"2024-09-19T07:48:51.601058+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:28.691031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601172+0000","last_change":"2024-09-19T07:48:50.601676+0000","last_active":"2024-09-19T07:48:51.601172+0000","last_peered":"2024-09-19T07:48:51.601172+0000","last_clean":"2024-09-19T07:48:51.601172+0000","last_became_active":"2024-09-19T07:48:50.601372+0000","last_became_peered":"2024-09-19T07:48:50.601372+0000","last_unstale":"2024-09-19T07:48:51.601172+0000","last_undegraded":"2024-09-19T07:48:51.601172+0000","last_fullsized":"2024-09-19T07:48:51.601172+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:11.968589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723186+0000","last_change":"2024-09-19T07:48:50.612616+0000","last_active":"2024-09-19T07:48:51.723186+0000","last_peered":"2024-09-19T07:48:51.723186+0000","last_clean":"2024-09-19T07:48:51.723186+0000","last_became_active":"2024-09-19T07:48:50.611466+0000","last_became_peered":"2024-09-19T07:48:50.611466+0000","last_unstale":"2024-09-19T07:48:51.723186+0000","last_undegraded":"2024-09-19T07:48:51.723186+0000","last_fullsized":"2024-09-19T07:48:51.723186+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:58.590631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723931+0000","last_change":"2024-09-19T07:48:50.623832+0000","last_active":"2024-09-19T07:48:51.723931+0000","last_peered":"2024-09-19T07:48:51.723931+0000","last_clean":"2024-09-19T07:48:51.723931+0000","last_became_active":"2024-09-19T07:48:50.623431+0000","last_became_peered":"2024-09-19T07:48:50.623431+0000","last_unstale":"2024-09-19T07:48:51.723931+0000","last_undegraded":"2024-09-19T07:48:51.723931+0000","last_fullsized":"2024-09-19T07:48:51.723931+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:41.958871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"69'4","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597696+0000","last_change":"2024-09-19T07:48:49.602061+0000","last_active":"2024-09-19T07:48:51.597696+0000","last_peered":"2024-09-19T07:48:51.597696+0000","last_clean":"2024-09-19T07:48:51.597696+0000","last_became_active":"2024-09-19T07:48:49.601684+0000","last_became_peered":"2024-09-19T07:48:49.601684+0000","last_unstale":"2024-09-19T07:48:51.597696+0000","last_undegraded":"2024-09-19T07:48:51.597696+0000","last_fullsized":"2024-09-19T07:48:51.597696+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:10.964218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601383+0000","last_change":"2024-09-19T07:48:50.619058+0000","last_active":"2024-09-19T07:48:51.601383+0000","last_peered":"2024-09-19T07:48:51.601383+0000","last_clean":"2024-09-19T07:48:51.601383+0000","last_became_active":"2024-09-19T07:48:50.618776+0000","last_became_peered":"2024-09-19T07:48:50.618776+0000","last_unstale":"2024-09-19T07:48:51.601383+0000","last_undegraded":"2024-09-19T07:48:51.601383+0000","last_fullsized":"2024-09-19T07:48:51.601383+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:00.856819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723565+0000","last_change":"2024-09-19T07:48:49.603413+0000","last_active":"2024-09-19T07:48:51.723565+0000","last_peered":"2024-09-19T07:48:51.723565+0000","last_clean":"2024-09-19T07:48:51.723565+0000","last_became_active":"2024-09-19T07:48:49.603043+0000","last_became_peered":"2024-09-19T07:48:49.603043+0000","last_unstale":"2024-09-19T07:48:51.723565+0000","last_undegraded":"2024-09-19T07:48:51.723565+0000","last_fullsized":"2024-09-19T07:48:51.723565+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:03:38.980793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723514+0000","last_change":"2024-09-19T07:48:50.616434+0000","last_active":"2024-09-19T07:48:51.723514+0000","last_peered":"2024-09-19T07:48:51.723514+0000","last_clean":"2024-09-19T07:48:51.723514+0000","last_became_active":"2024-09-19T07:48:50.615413+0000","last_became_peered":"2024-09-19T07:48:50.615413+0000","last_unstale":"2024-09-19T07:48:51.723514+0000","last_undegraded":"2024-09-19T07:48:51.723514+0000","last_fullsized":"2024-09-19T07:48:51.723514+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:11.128746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723784+0000","last_change":"2024-09-19T07:48:49.600450+0000","last_active":"2024-09-19T07:48:51.723784+0000","last_peered":"2024-09-19T07:48:51.723784+0000","last_clean":"2024-09-19T07:48:51.723784+0000","last_became_active":"2024-09-19T07:48:49.600094+0000","last_became_peered":"2024-09-19T07:48:49.600094+0000","last_unstale":"2024-09-19T07:48:51.723784+0000","last_undegraded":"2024-09-19T07:48:51.723784+0000","last_fullsized":"2024-09-19T07:48:51.723784+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:07.031415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723918+0000","last_change":"2024-09-19T07:48:50.622627+0000","last_active":"2024-09-19T07:48:51.723918+0000","last_peered":"2024-09-19T07:48:51.723918+0000","last_clean":"2024-09-19T07:48:51.723918+0000","last_became_active":"2024-09-19T07:48:50.622335+0000","last_became_peered":"2024-09-19T07:48:50.622335+0000","last_unstale":"2024-09-19T07:48:51.723918+0000","last_undegraded":"2024-09-19T07:48:51.723918+0000","last_fullsized":"2024-09-19T07:48:51.723918+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:11.128746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723325+0000","last_change":"2024-09-19T07:48:49.596140+0000","last_active":"2024-09-19T07:48:51.723325+0000","last_peered":"2024-09-19T07:48:51.723325+0000","last_clean":"2024-09-19T07:48:51.723325+0000","last_became_active":"2024-09-19T07:48:49.595802+0000","last_became_peered":"2024-09-19T07:48:49.595802+0000","last_unstale":"2024-09-19T07:48:51.723325+0000","last_undegraded":"2024-09-19T07:48:51.723325+0000","last_fullsized":"2024-09-19T07:48:51.723325+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:17.115934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.618315+0000","last_change":"2024-09-19T07:48:50.607600+0000","last_active":"2024-09-19T07:48:51.618315+0000","last_peered":"2024-09-19T07:48:51.618315+0000","last_clean":"2024-09-19T07:48:51.618315+0000","last_became_active":"2024-09-19T07:48:50.606735+0000","last_became_peered":"2024-09-19T07:48:50.606735+0000","last_unstale":"2024-09-19T07:48:51.618315+0000","last_undegraded":"2024-09-19T07:48:51.618315+0000","last_fullsized":"2024-09-19T07:48:51.618315+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:12.676635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723767+0000","last_change":"2024-09-19T07:48:49.597306+0000","last_active":"2024-09-19T07:48:51.723767+0000","last_peered":"2024-09-19T07:48:51.723767+0000","last_clean":"2024-09-19T07:48:51.723767+0000","last_became_active":"2024-09-19T07:48:49.597002+0000","last_became_peered":"2024-09-19T07:48:49.597002+0000","last_unstale":"2024-09-19T07:48:51.723767+0000","last_undegraded":"2024-09-19T07:48:51.723767+0000","last_fullsized":"2024-09-19T07:48:51.723767+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:29.429632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597211+0000","last_change":"2024-09-19T07:48:50.619874+0000","last_active":"2024-09-19T07:48:51.597211+0000","last_peered":"2024-09-19T07:48:51.597211+0000","last_clean":"2024-09-19T07:48:51.597211+0000","last_became_active":"2024-09-19T07:48:50.619587+0000","last_became_peered":"2024-09-19T07:48:50.619587+0000","last_unstale":"2024-09-19T07:48:51.597211+0000","last_undegraded":"2024-09-19T07:48:51.597211+0000","last_fullsized":"2024-09-19T07:48:51.597211+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:11.464590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723654+0000","last_change":"2024-09-19T07:48:49.591854+0000","last_active":"2024-09-19T07:48:51.723654+0000","last_peered":"2024-09-19T07:48:51.723654+0000","last_clean":"2024-09-19T07:48:51.723654+0000","last_became_active":"2024-09-19T07:48:49.591532+0000","last_became_peered":"2024-09-19T07:48:49.591532+0000","last_unstale":"2024-09-19T07:48:51.723654+0000","last_undegraded":"2024-09-19T07:48:51.723654+0000","last_fullsized":"2024-09-19T07:48:51.723654+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:06.402253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723576+0000","last_change":"2024-09-19T07:48:50.620827+0000","last_active":"2024-09-19T07:48:51.723576+0000","last_peered":"2024-09-19T07:48:51.723576+0000","last_clean":"2024-09-19T07:48:51.723576+0000","last_became_active":"2024-09-19T07:48:50.620190+0000","last_became_peered":"2024-09-19T07:48:50.620190+0000","last_unstale":"2024-09-19T07:48:51.723576+0000","last_undegraded":"2024-09-19T07:48:51.723576+0000","last_fullsized":"2024-09-19T07:48:51.723576+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:51.090644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723756+0000","last_change":"2024-09-19T07:48:49.595497+0000","last_active":"2024-09-19T07:48:51.723756+0000","last_peered":"2024-09-19T07:48:51.723756+0000","last_clean":"2024-09-19T07:48:51.723756+0000","last_became_active":"2024-09-19T07:48:49.594777+0000","last_became_peered":"2024-09-19T07:48:49.594777+0000","last_unstale":"2024-09-19T07:48:51.723756+0000","last_undegraded":"2024-09-19T07:48:51.723756+0000","last_fullsized":"2024-09-19T07:48:51.723756+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:11.653266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723658+0000","last_change":"2024-09-19T07:48:50.606445+0000","last_active":"2024-09-19T07:48:51.723658+0000","last_peered":"2024-09-19T07:48:51.723658+0000","last_clean":"2024-09-19T07:48:51.723658+0000","last_became_active":"2024-09-19T07:48:50.606155+0000","last_became_peered":"2024-09-19T07:48:50.606155+0000","last_unstale":"2024-09-19T07:48:51.723658+0000","last_undegraded":"2024-09-19T07:48:51.723658+0000","last_fullsized":"2024-09-19T07:48:51.723658+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:20.244263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723358+0000","last_change":"2024-09-19T07:48:49.596857+0000","last_active":"2024-09-19T07:48:51.723358+0000","last_peered":"2024-09-19T07:48:51.723358+0000","last_clean":"2024-09-19T07:48:51.723358+0000","last_became_active":"2024-09-19T07:48:49.596597+0000","last_became_peered":"2024-09-19T07:48:49.596597+0000","last_unstale":"2024-09-19T07:48:51.723358+0000","last_undegraded":"2024-09-19T07:48:51.723358+0000","last_fullsized":"2024-09-19T07:48:51.723358+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:28.988556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597587+0000","last_change":"2024-09-19T07:48:50.610040+0000","last_active":"2024-09-19T07:48:51.597587+0000","last_peered":"2024-09-19T07:48:51.597587+0000","last_clean":"2024-09-19T07:48:51.597587+0000","last_became_active":"2024-09-19T07:48:50.609745+0000","last_became_peered":"2024-09-19T07:48:50.609745+0000","last_unstale":"2024-09-19T07:48:51.597587+0000","last_undegraded":"2024-09-19T07:48:51.597587+0000","last_fullsized":"2024-09-19T07:48:51.597587+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:08.035787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724114+0000","last_change":"2024-09-19T07:48:49.603689+0000","last_active":"2024-09-19T07:48:51.724114+0000","last_peered":"2024-09-19T07:48:51.724114+0000","last_clean":"2024-09-19T07:48:51.724114+0000","last_became_active":"2024-09-19T07:48:49.602991+0000","last_became_peered":"2024-09-19T07:48:49.602991+0000","last_unstale":"2024-09-19T07:48:51.724114+0000","last_undegraded":"2024-09-19T07:48:51.724114+0000","last_fullsized":"2024-09-19T07:48:51.724114+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:50.104285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723504+0000","last_change":"2024-09-19T07:48:50.608683+0000","last_active":"2024-09-19T07:48:51.723504+0000","last_peered":"2024-09-19T07:48:51.723504+0000","last_clean":"2024-09-19T07:48:51.723504+0000","last_became_active":"2024-09-19T07:48:50.608336+0000","last_became_peered":"2024-09-19T07:48:50.608336+0000","last_unstale":"2024-09-19T07:48:51.723504+0000","last_undegraded":"2024-09-19T07:48:51.723504+0000","last_fullsized":"2024-09-19T07:48:51.723504+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:43.270718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601589+0000","last_change":"2024-09-19T07:48:49.597480+0000","last_active":"2024-09-19T07:48:51.601589+0000","last_peered":"2024-09-19T07:48:51.601589+0000","last_clean":"2024-09-19T07:48:51.601589+0000","last_became_active":"2024-09-19T07:48:49.597175+0000","last_became_peered":"2024-09-19T07:48:49.597175+0000","last_unstale":"2024-09-19T07:48:51.601589+0000","last_undegraded":"2024-09-19T07:48:51.601589+0000","last_fullsized":"2024-09-19T07:48:51.601589+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:33:05.500303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723753+0000","last_change":"2024-09-19T07:48:50.617419+0000","last_active":"2024-09-19T07:48:51.723753+0000","last_peered":"2024-09-19T07:48:51.723753+0000","last_clean":"2024-09-19T07:48:51.723753+0000","last_became_active":"2024-09-19T07:48:50.617165+0000","last_became_peered":"2024-09-19T07:48:50.617165+0000","last_unstale":"2024-09-19T07:48:51.723753+0000","last_undegraded":"2024-09-19T07:48:51.723753+0000","last_fullsized":"2024-09-19T07:48:51.723753+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:06.722534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"69'3","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724022+0000","last_change":"2024-09-19T07:48:49.604058+0000","last_active":"2024-09-19T07:48:51.724022+0000","last_peered":"2024-09-19T07:48:51.724022+0000","last_clean":"2024-09-19T07:48:51.724022+0000","last_became_active":"2024-09-19T07:48:49.602835+0000","last_became_peered":"2024-09-19T07:48:49.602835+0000","last_unstale":"2024-09-19T07:48:51.724022+0000","last_undegraded":"2024-09-19T07:48:51.724022+0000","last_fullsized":"2024-09-19T07:48:51.724022+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:52.157019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601888+0000","last_change":"2024-09-19T07:48:50.605331+0000","last_active":"2024-09-19T07:48:51.601888+0000","last_peered":"2024-09-19T07:48:51.601888+0000","last_clean":"2024-09-19T07:48:51.601888+0000","last_became_active":"2024-09-19T07:48:50.605066+0000","last_became_peered":"2024-09-19T07:48:50.605066+0000","last_unstale":"2024-09-19T07:48:51.601888+0000","last_undegraded":"2024-09-19T07:48:51.601888+0000","last_fullsized":"2024-09-19T07:48:51.601888+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:11.570438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"69'6","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597581+0000","last_change":"2024-09-19T07:48:49.599753+0000","last_active":"2024-09-19T07:48:51.597581+0000","last_peered":"2024-09-19T07:48:51.597581+0000","last_clean":"2024-09-19T07:48:51.597581+0000","last_became_active":"2024-09-19T07:48:49.599452+0000","last_became_peered":"2024-09-19T07:48:49.599452+0000","last_unstale":"2024-09-19T07:48:51.597581+0000","last_undegraded":"2024-09-19T07:48:51.597581+0000","last_fullsized":"2024-09-19T07:48:51.597581+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:30.658256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601585+0000","last_change":"2024-09-19T07:48:50.610147+0000","last_active":"2024-09-19T07:48:51.601585+0000","last_peered":"2024-09-19T07:48:51.601585+0000","last_clean":"2024-09-19T07:48:51.601585+0000","last_became_active":"2024-09-19T07:48:50.609675+0000","last_became_peered":"2024-09-19T07:48:50.609675+0000","last_unstale":"2024-09-19T07:48:51.601585+0000","last_undegraded":"2024-09-19T07:48:51.601585+0000","last_fullsized":"2024-09-19T07:48:51.601585+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:06.722534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724117+0000","last_change":"2024-09-19T07:48:49.602673+0000","last_active":"2024-09-19T07:48:51.724117+0000","last_peered":"2024-09-19T07:48:51.724117+0000","last_clean":"2024-09-19T07:48:51.724117+0000","last_became_active":"2024-09-19T07:48:49.601601+0000","last_became_peered":"2024-09-19T07:48:49.601601+0000","last_unstale":"2024-09-19T07:48:51.724117+0000","last_undegraded":"2024-09-19T07:48:51.724117+0000","last_fullsized":"2024-09-19T07:48:51.724117+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:17.115934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.600815+0000","last_change":"2024-09-19T07:48:50.612160+0000","last_active":"2024-09-19T07:48:51.600815+0000","last_peered":"2024-09-19T07:48:51.600815+0000","last_clean":"2024-09-19T07:48:51.600815+0000","last_became_active":"2024-09-19T07:48:50.611833+0000","last_became_peered":"2024-09-19T07:48:50.611833+0000","last_unstale":"2024-09-19T07:48:51.600815+0000","last_undegraded":"2024-09-19T07:48:51.600815+0000","last_fullsized":"2024-09-19T07:48:51.600815+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:58.590631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601128+0000","last_change":"2024-09-19T07:48:49.592362+0000","last_active":"2024-09-19T07:48:51.601128+0000","last_peered":"2024-09-19T07:48:51.601128+0000","last_clean":"2024-09-19T07:48:51.601128+0000","last_became_active":"2024-09-19T07:48:49.591829+0000","last_became_peered":"2024-09-19T07:48:49.591829+0000","last_unstale":"2024-09-19T07:48:51.601128+0000","last_undegraded":"2024-09-19T07:48:51.601128+0000","last_fullsized":"2024-09-19T07:48:51.601128+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:08:30.501080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723592+0000","last_change":"2024-09-19T07:48:50.613080+0000","last_active":"2024-09-19T07:48:51.723592+0000","last_peered":"2024-09-19T07:48:51.723592+0000","last_clean":"2024-09-19T07:48:51.723592+0000","last_became_active":"2024-09-19T07:48:50.612625+0000","last_became_peered":"2024-09-19T07:48:50.612625+0000","last_unstale":"2024-09-19T07:48:51.723592+0000","last_undegraded":"2024-09-19T07:48:51.723592+0000","last_fullsized":"2024-09-19T07:48:51.723592+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:11.968589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724498+0000","last_change":"2024-09-19T07:48:49.603802+0000","last_active":"2024-09-19T07:48:51.724498+0000","last_peered":"2024-09-19T07:48:51.724498+0000","last_clean":"2024-09-19T07:48:51.724498+0000","last_became_active":"2024-09-19T07:48:49.602459+0000","last_became_peered":"2024-09-19T07:48:49.602459+0000","last_unstale":"2024-09-19T07:48:51.724498+0000","last_undegraded":"2024-09-19T07:48:51.724498+0000","last_fullsized":"2024-09-19T07:48:51.724498+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:08:30.501080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597396+0000","last_change":"2024-09-19T07:48:50.617463+0000","last_active":"2024-09-19T07:48:51.597396+0000","last_peered":"2024-09-19T07:48:51.597396+0000","last_clean":"2024-09-19T07:48:51.597396+0000","last_became_active":"2024-09-19T07:48:50.617128+0000","last_became_peered":"2024-09-19T07:48:50.617128+0000","last_unstale":"2024-09-19T07:48:51.597396+0000","last_undegraded":"2024-09-19T07:48:51.597396+0000","last_fullsized":"2024-09-19T07:48:51.597396+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:51.108656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"69'8","reported_seq":22,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601801+0000","last_change":"2024-09-19T07:48:49.601117+0000","last_active":"2024-09-19T07:48:51.601801+0000","last_peered":"2024-09-19T07:48:51.601801+0000","last_clean":"2024-09-19T07:48:51.601801+0000","last_became_active":"2024-09-19T07:48:49.600793+0000","last_became_peered":"2024-09-19T07:48:49.600793+0000","last_unstale":"2024-09-19T07:48:51.601801+0000","last_undegraded":"2024-09-19T07:48:51.601801+0000","last_fullsized":"2024-09-19T07:48:51.601801+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:28.988556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723714+0000","last_change":"2024-09-19T07:48:50.610704+0000","last_active":"2024-09-19T07:48:51.723714+0000","last_peered":"2024-09-19T07:48:51.723714+0000","last_clean":"2024-09-19T07:48:51.723714+0000","last_became_active":"2024-09-19T07:48:50.610197+0000","last_became_peered":"2024-09-19T07:48:50.610197+0000","last_unstale":"2024-09-19T07:48:51.723714+0000","last_undegraded":"2024-09-19T07:48:51.723714+0000","last_fullsized":"2024-09-19T07:48:51.723714+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:26.777068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601596+0000","last_change":"2024-09-19T07:48:49.595123+0000","last_active":"2024-09-19T07:48:51.601596+0000","last_peered":"2024-09-19T07:48:51.601596+0000","last_clean":"2024-09-19T07:48:51.601596+0000","last_became_active":"2024-09-19T07:48:49.594733+0000","last_became_peered":"2024-09-19T07:48:49.594733+0000","last_unstale":"2024-09-19T07:48:51.601596+0000","last_undegraded":"2024-09-19T07:48:51.601596+0000","last_fullsized":"2024-09-19T07:48:51.601596+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:51.351283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723827+0000","last_change":"2024-09-19T07:48:50.614642+0000","last_active":"2024-09-19T07:48:51.723827+0000","last_peered":"2024-09-19T07:48:51.723827+0000","last_clean":"2024-09-19T07:48:51.723827+0000","last_became_active":"2024-09-19T07:48:50.614370+0000","last_became_peered":"2024-09-19T07:48:50.614370+0000","last_unstale":"2024-09-19T07:48:51.723827+0000","last_undegraded":"2024-09-19T07:48:51.723827+0000","last_fullsized":"2024-09-19T07:48:51.723827+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:11.570438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724128+0000","last_change":"2024-09-19T07:48:49.604264+0000","last_active":"2024-09-19T07:48:51.724128+0000","last_peered":"2024-09-19T07:48:51.724128+0000","last_clean":"2024-09-19T07:48:51.724128+0000","last_became_active":"2024-09-19T07:48:49.603395+0000","last_became_peered":"2024-09-19T07:48:49.603395+0000","last_unstale":"2024-09-19T07:48:51.724128+0000","last_undegraded":"2024-09-19T07:48:51.724128+0000","last_fullsized":"2024-09-19T07:48:51.724128+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:31.766687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601811+0000","last_change":"2024-09-19T07:48:50.616633+0000","last_active":"2024-09-19T07:48:51.601811+0000","last_peered":"2024-09-19T07:48:51.601811+0000","last_clean":"2024-09-19T07:48:51.601811+0000","last_became_active":"2024-09-19T07:48:50.615974+0000","last_became_peered":"2024-09-19T07:48:50.615974+0000","last_unstale":"2024-09-19T07:48:51.601811+0000","last_undegraded":"2024-09-19T07:48:51.601811+0000","last_fullsized":"2024-09-19T07:48:51.601811+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:41.628823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724085+0000","last_change":"2024-09-19T07:48:49.604469+0000","last_active":"2024-09-19T07:48:51.724085+0000","last_peered":"2024-09-19T07:48:51.724085+0000","last_clean":"2024-09-19T07:48:51.724085+0000","last_became_active":"2024-09-19T07:48:49.604187+0000","last_became_peered":"2024-09-19T07:48:49.604187+0000","last_unstale":"2024-09-19T07:48:51.724085+0000","last_undegraded":"2024-09-19T07:48:51.724085+0000","last_fullsized":"2024-09-19T07:48:51.724085+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:10.460218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724253+0000","last_change":"2024-09-19T07:48:50.618311+0000","last_active":"2024-09-19T07:48:51.724253+0000","last_peered":"2024-09-19T07:48:51.724253+0000","last_clean":"2024-09-19T07:48:51.724253+0000","last_became_active":"2024-09-19T07:48:50.617821+0000","last_became_peered":"2024-09-19T07:48:50.617821+0000","last_unstale":"2024-09-19T07:48:51.724253+0000","last_undegraded":"2024-09-19T07:48:51.724253+0000","last_fullsized":"2024-09-19T07:48:51.724253+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:06.848583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723935+0000","last_change":"2024-09-19T07:48:49.598870+0000","last_active":"2024-09-19T07:48:51.723935+0000","last_peered":"2024-09-19T07:48:51.723935+0000","last_clean":"2024-09-19T07:48:51.723935+0000","last_became_active":"2024-09-19T07:48:49.598519+0000","last_became_peered":"2024-09-19T07:48:49.598519+0000","last_unstale":"2024-09-19T07:48:51.723935+0000","last_undegraded":"2024-09-19T07:48:51.723935+0000","last_fullsized":"2024-09-19T07:48:51.723935+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:18.693182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723262+0000","last_change":"2024-09-19T07:48:50.612364+0000","last_active":"2024-09-19T07:48:51.723262+0000","last_peered":"2024-09-19T07:48:51.723262+0000","last_clean":"2024-09-19T07:48:51.723262+0000","last_became_active":"2024-09-19T07:48:50.611087+0000","last_became_peered":"2024-09-19T07:48:50.611087+0000","last_unstale":"2024-09-19T07:48:51.723262+0000","last_undegraded":"2024-09-19T07:48:51.723262+0000","last_fullsized":"2024-09-19T07:48:51.723262+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:16.152040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723397+0000","last_change":"2024-09-19T07:48:49.595532+0000","last_active":"2024-09-19T07:48:51.723397+0000","last_peered":"2024-09-19T07:48:51.723397+0000","last_clean":"2024-09-19T07:48:51.723397+0000","last_became_active":"2024-09-19T07:48:49.595176+0000","last_became_peered":"2024-09-19T07:48:49.595176+0000","last_unstale":"2024-09-19T07:48:51.723397+0000","last_undegraded":"2024-09-19T07:48:51.723397+0000","last_fullsized":"2024-09-19T07:48:51.723397+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:50.104285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.600867+0000","last_change":"2024-09-19T07:48:50.604743+0000","last_active":"2024-09-19T07:48:51.600867+0000","last_peered":"2024-09-19T07:48:51.600867+0000","last_clean":"2024-09-19T07:48:51.600867+0000","last_became_active":"2024-09-19T07:48:50.604459+0000","last_became_peered":"2024-09-19T07:48:50.604459+0000","last_unstale":"2024-09-19T07:48:51.600867+0000","last_undegraded":"2024-09-19T07:48:51.600867+0000","last_fullsized":"2024-09-19T07:48:51.600867+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:26.777068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601223+0000","last_change":"2024-09-19T07:48:49.592097+0000","last_active":"2024-09-19T07:48:51.601223+0000","last_peered":"2024-09-19T07:48:51.601223+0000","last_clean":"2024-09-19T07:48:51.601223+0000","last_became_active":"2024-09-19T07:48:49.591470+0000","last_became_peered":"2024-09-19T07:48:49.591470+0000","last_unstale":"2024-09-19T07:48:51.601223+0000","last_undegraded":"2024-09-19T07:48:51.601223+0000","last_fullsized":"2024-09-19T07:48:51.601223+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:52.157019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723873+0000","last_change":"2024-09-19T07:48:50.620944+0000","last_active":"2024-09-19T07:48:51.723873+0000","last_peered":"2024-09-19T07:48:51.723873+0000","last_clean":"2024-09-19T07:48:51.723873+0000","last_became_active":"2024-09-19T07:48:50.620554+0000","last_became_peered":"2024-09-19T07:48:50.620554+0000","last_unstale":"2024-09-19T07:48:51.723873+0000","last_undegraded":"2024-09-19T07:48:51.723873+0000","last_fullsized":"2024-09-19T07:48:51.723873+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:23:24.632533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723207+0000","last_change":"2024-09-19T07:48:49.602003+0000","last_active":"2024-09-19T07:48:51.723207+0000","last_peered":"2024-09-19T07:48:51.723207+0000","last_clean":"2024-09-19T07:48:51.723207+0000","last_became_active":"2024-09-19T07:48:49.601652+0000","last_became_peered":"2024-09-19T07:48:49.601652+0000","last_unstale":"2024-09-19T07:48:51.723207+0000","last_undegraded":"2024-09-19T07:48:51.723207+0000","last_fullsized":"2024-09-19T07:48:51.723207+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:31.766687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723505+0000","last_change":"2024-09-19T07:48:50.621641+0000","last_active":"2024-09-19T07:48:51.723505+0000","last_peered":"2024-09-19T07:48:51.723505+0000","last_clean":"2024-09-19T07:48:51.723505+0000","last_became_active":"2024-09-19T07:48:50.621021+0000","last_became_peered":"2024-09-19T07:48:50.621021+0000","last_unstale":"2024-09-19T07:48:51.723505+0000","last_undegraded":"2024-09-19T07:48:51.723505+0000","last_fullsized":"2024-09-19T07:48:51.723505+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:26.777068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601706+0000","last_change":"2024-09-19T07:48:49.602206+0000","last_active":"2024-09-19T07:48:51.601706+0000","last_peered":"2024-09-19T07:48:51.601706+0000","last_clean":"2024-09-19T07:48:51.601706+0000","last_became_active":"2024-09-19T07:48:49.601925+0000","last_became_peered":"2024-09-19T07:48:49.601925+0000","last_unstale":"2024-09-19T07:48:51.601706+0000","last_undegraded":"2024-09-19T07:48:51.601706+0000","last_fullsized":"2024-09-19T07:48:51.601706+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:50.104285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.618219+0000","last_change":"2024-09-19T07:48:50.613584+0000","last_active":"2024-09-19T07:48:51.618219+0000","last_peered":"2024-09-19T07:48:51.618219+0000","last_clean":"2024-09-19T07:48:51.618219+0000","last_became_active":"2024-09-19T07:48:50.612698+0000","last_became_peered":"2024-09-19T07:48:50.612698+0000","last_unstale":"2024-09-19T07:48:51.618219+0000","last_undegraded":"2024-09-19T07:48:51.618219+0000","last_fullsized":"2024-09-19T07:48:51.618219+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:22:22.942559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"70'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601388+0000","last_change":"2024-09-19T07:48:49.588889+0000","last_active":"2024-09-19T07:48:51.601388+0000","last_peered":"2024-09-19T07:48:51.601388+0000","last_clean":"2024-09-19T07:48:51.601388+0000","last_became_active":"2024-09-19T07:48:49.588216+0000","last_became_peered":"2024-09-19T07:48:49.588216+0000","last_unstale":"2024-09-19T07:48:51.601388+0000","last_undegraded":"2024-09-19T07:48:51.601388+0000","last_fullsized":"2024-09-19T07:48:51.601388+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:10.460218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601121+0000","last_change":"2024-09-19T07:48:50.618857+0000","last_active":"2024-09-19T07:48:51.601121+0000","last_peered":"2024-09-19T07:48:51.601121+0000","last_clean":"2024-09-19T07:48:51.601121+0000","last_became_active":"2024-09-19T07:48:50.618565+0000","last_became_peered":"2024-09-19T07:48:50.618565+0000","last_unstale":"2024-09-19T07:48:51.601121+0000","last_undegraded":"2024-09-19T07:48:51.601121+0000","last_fullsized":"2024-09-19T07:48:51.601121+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:28.290169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723842+0000","last_change":"2024-09-19T07:48:49.597252+0000","last_active":"2024-09-19T07:48:51.723842+0000","last_peered":"2024-09-19T07:48:51.723842+0000","last_clean":"2024-09-19T07:48:51.723842+0000","last_became_active":"2024-09-19T07:48:49.596983+0000","last_became_peered":"2024-09-19T07:48:49.596983+0000","last_unstale":"2024-09-19T07:48:51.723842+0000","last_undegraded":"2024-09-19T07:48:51.723842+0000","last_fullsized":"2024-09-19T07:48:51.723842+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:06.787582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724421+0000","last_change":"2024-09-19T07:48:50.618096+0000","last_active":"2024-09-19T07:48:51.724421+0000","last_peered":"2024-09-19T07:48:51.724421+0000","last_clean":"2024-09-19T07:48:51.724421+0000","last_became_active":"2024-09-19T07:48:50.617456+0000","last_became_peered":"2024-09-19T07:48:50.617456+0000","last_unstale":"2024-09-19T07:48:51.724421+0000","last_undegraded":"2024-09-19T07:48:51.724421+0000","last_fullsized":"2024-09-19T07:48:51.724421+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:07.353888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597678+0000","last_change":"2024-09-19T07:48:49.599857+0000","last_active":"2024-09-19T07:48:51.597678+0000","last_peered":"2024-09-19T07:48:51.597678+0000","last_clean":"2024-09-19T07:48:51.597678+0000","last_became_active":"2024-09-19T07:48:49.599550+0000","last_became_peered":"2024-09-19T07:48:49.599550+0000","last_unstale":"2024-09-19T07:48:51.597678+0000","last_undegraded":"2024-09-19T07:48:51.597678+0000","last_fullsized":"2024-09-19T07:48:51.597678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:05.718163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723372+0000","last_change":"2024-09-19T07:48:50.623756+0000","last_active":"2024-09-19T07:48:51.723372+0000","last_peered":"2024-09-19T07:48:51.723372+0000","last_clean":"2024-09-19T07:48:51.723372+0000","last_became_active":"2024-09-19T07:48:50.623440+0000","last_became_peered":"2024-09-19T07:48:50.623440+0000","last_unstale":"2024-09-19T07:48:51.723372+0000","last_undegraded":"2024-09-19T07:48:51.723372+0000","last_fullsized":"2024-09-19T07:48:51.723372+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:02.116554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724246+0000","last_change":"2024-09-19T07:48:49.603929+0000","last_active":"2024-09-19T07:48:51.724246+0000","last_peered":"2024-09-19T07:48:51.724246+0000","last_clean":"2024-09-19T07:48:51.724246+0000","last_became_active":"2024-09-19T07:48:49.603208+0000","last_became_peered":"2024-09-19T07:48:49.603208+0000","last_unstale":"2024-09-19T07:48:51.724246+0000","last_undegraded":"2024-09-19T07:48:51.724246+0000","last_fullsized":"2024-09-19T07:48:51.724246+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:51.351283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.601099+0000","last_change":"2024-09-19T07:48:50.608905+0000","last_active":"2024-09-19T07:48:51.601099+0000","last_peered":"2024-09-19T07:48:51.601099+0000","last_clean":"2024-09-19T07:48:51.601099+0000","last_became_active":"2024-09-19T07:48:50.608632+0000","last_became_peered":"2024-09-19T07:48:50.608632+0000","last_unstale":"2024-09-19T07:48:51.601099+0000","last_undegraded":"2024-09-19T07:48:51.601099+0000","last_fullsized":"2024-09-19T07:48:51.601099+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:51.090644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.724018+0000","last_change":"2024-09-19T07:48:49.593753+0000","last_active":"2024-09-19T07:48:51.724018+0000","last_peered":"2024-09-19T07:48:51.724018+0000","last_clean":"2024-09-19T07:48:51.724018+0000","last_became_active":"2024-09-19T07:48:49.592814+0000","last_became_peered":"2024-09-19T07:48:49.592814+0000","last_unstale":"2024-09-19T07:48:51.724018+0000","last_undegraded":"2024-09-19T07:48:51.724018+0000","last_fullsized":"2024-09-19T07:48:51.724018+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:48.330162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597219+0000","last_change":"2024-09-19T07:48:50.619443+0000","last_active":"2024-09-19T07:48:51.597219+0000","last_peered":"2024-09-19T07:48:51.597219+0000","last_clean":"2024-09-19T07:48:51.597219+0000","last_became_active":"2024-09-19T07:48:50.619073+0000","last_became_peered":"2024-09-19T07:48:50.619073+0000","last_unstale":"2024-09-19T07:48:51.597219+0000","last_undegraded":"2024-09-19T07:48:51.597219+0000","last_fullsized":"2024-09-19T07:48:51.597219+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:52.355654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723700+0000","last_change":"2024-09-19T07:48:49.594762+0000","last_active":"2024-09-19T07:48:51.723700+0000","last_peered":"2024-09-19T07:48:51.723700+0000","last_clean":"2024-09-19T07:48:51.723700+0000","last_became_active":"2024-09-19T07:48:49.594420+0000","last_became_peered":"2024-09-19T07:48:49.594420+0000","last_unstale":"2024-09-19T07:48:51.723700+0000","last_undegraded":"2024-09-19T07:48:51.723700+0000","last_fullsized":"2024-09-19T07:48:51.723700+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:03:01.422142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.618418+0000","last_change":"2024-09-19T07:48:50.604249+0000","last_active":"2024-09-19T07:48:51.618418+0000","last_peered":"2024-09-19T07:48:51.618418+0000","last_clean":"2024-09-19T07:48:51.618418+0000","last_became_active":"2024-09-19T07:48:50.603870+0000","last_became_peered":"2024-09-19T07:48:50.603870+0000","last_unstale":"2024-09-19T07:48:51.618418+0000","last_undegraded":"2024-09-19T07:48:51.618418+0000","last_fullsized":"2024-09-19T07:48:51.618418+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:33.907102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.597187+0000","last_change":"2024-09-19T07:48:49.597016+0000","last_active":"2024-09-19T07:48:51.597187+0000","last_peered":"2024-09-19T07:48:51.597187+0000","last_clean":"2024-09-19T07:48:51.597187+0000","last_became_active":"2024-09-19T07:48:49.596725+0000","last_became_peered":"2024-09-19T07:48:49.596725+0000","last_unstale":"2024-09-19T07:48:51.597187+0000","last_undegraded":"2024-09-19T07:48:51.597187+0000","last_fullsized":"2024-09-19T07:48:51.597187+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:59.852448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723831+0000","last_change":"2024-09-19T07:48:50.623507+0000","last_active":"2024-09-19T07:48:51.723831+0000","last_peered":"2024-09-19T07:48:51.723831+0000","last_clean":"2024-09-19T07:48:51.723831+0000","last_became_active":"2024-09-19T07:48:50.623221+0000","last_became_peered":"2024-09-19T07:48:50.623221+0000","last_unstale":"2024-09-19T07:48:51.723831+0000","last_undegraded":"2024-09-19T07:48:51.723831+0000","last_fullsized":"2024-09-19T07:48:51.723831+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:49.577543+0000","last_clean_scrub_stamp":"2024-09-19T07:48:49.577543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:41.628823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723423+0000","last_change":"2024-09-19T07:48:49.598225+0000","last_active":"2024-09-19T07:48:51.723423+0000","last_peered":"2024-09-19T07:48:51.723423+0000","last_clean":"2024-09-19T07:48:51.723423+0000","last_became_active":"2024-09-19T07:48:49.597938+0000","last_became_peered":"2024-09-19T07:48:49.597938+0000","last_unstale":"2024-09-19T07:48:51.723423+0000","last_undegraded":"2024-09-19T07:48:51.723423+0000","last_fullsized":"2024-09-19T07:48:51.723423+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:48:48.573172+0000","last_clean_scrub_stamp":"2024-09-19T07:48:48.573172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:02:14.020905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":259,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-19T07:48:51.723642+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:48:51.723642+0000","last_peered":"2024-09-19T07:48:51.723642+0000","last_clean":"2024-09-19T07:48:51.723642+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:48:51.723642+0000","last_undegraded":"2024-09-19T07:48:51.723642+0000","last_fullsized":"2024-09-19T07:48:51.723642+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509515,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95008,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":93566,"kb_avail":94276832,"statfs":{"total":96636764160,"available":96539475968,"internally_reserved":0,"allocated":1449984,"data_stored":974233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":95812042},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":112524,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94259316,"statfs":{"total":96636764160,"available":96521539584,"internally_reserved":0,"allocated":843776,"data_stored":383123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":114358726},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509515,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50060,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321780,"statfs":{"total":96636764160,"available":96585502720,"internally_reserved":0,"allocated":843776,"data_stored":383111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509515,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62548,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309292,"statfs":{"total":96636764160,"available":96572715008,"internally_reserved":0,"allocated":851968,"data_stored":384701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509514,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100564,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271276,"statfs":{"total":96636764160,"available":96533786624,"internally_reserved":0,"allocated":1433600,"data_stored":973457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542218,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130504,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94241336,"statfs":{"total":96636764160,"available":96503128064,"internally_reserved":0,"allocated":839680,"data_stored":383089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542218,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94504,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94277336,"statfs":{"total":96636764160,"available":96539992064,"internally_reserved":0,"allocated":868352,"data_stored":385465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104140,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267700,"statfs":{"total":96636764160,"available":96530124800,"internally_reserved":0,"allocated":839680,"data_stored":383089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:48:58.343 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T07:48:58.343 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T07:48:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.486+0000 7f046570a640 1 Processor -- start 2024-09-19T07:48:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.487+0000 7f046570a640 1 -- start start 2024-09-19T07:48:58.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.487+0000 7f046570a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f04600d2460 0x7f04600d28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:58.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.487+0000 7f046570a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0460072050 con 0x7f04600cbea0 2024-09-19T07:48:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.487+0000 7f046570a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0460072330 con 0x7f046015e2e0 2024-09-19T07:48:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.487+0000 7f046570a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f04600ccf80 con 0x7f04600d2460 2024-09-19T07:48:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.488+0000 7f045e7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f04600cbea0 0x7f04600cc870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51250/0 (socket says 172.21.15.16:51250) 2024-09-19T07:48:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.488+0000 7f045e7fc640 1 -- 172.21.15.16:0/814013731 learned_addr learned my addr 172.21.15.16:0/814013731 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:48:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.488+0000 7f045f7fe640 1 --2- 172.21.15.16:0/814013731 >> v2:172.21.15.146:3300/0 conn(0x7f04600d2460 0x7f04600d28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.488+0000 7f045dffb640 1 -- 172.21.15.16:0/814013731 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3190306521 0 0) 0x7f0460072050 con 0x7f04600cbea0 2024-09-19T07:48:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.488+0000 7f045dffb640 1 -- 172.21.15.16:0/814013731 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0448003520 con 0x7f04600cbea0 2024-09-19T07:48:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.488+0000 7f045dffb640 1 -- 172.21.15.16:0/814013731 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2742306049 0 0) 0x7f0448003520 con 0x7f04600cbea0 2024-09-19T07:48:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.488+0000 7f045f7fe640 1 -- 172.21.15.16:0/814013731 >> v1:172.21.15.146:6789/0 conn(0x7f046015e2e0 legacy=0x7f0460071750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.488+0000 7f045f7fe640 1 -- 172.21.15.16:0/814013731 >> v1:172.21.15.16:6789/0 conn(0x7f04600cbea0 legacy=0x7f04600cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.488+0000 7f045f7fe640 1 -- 172.21.15.16:0/814013731 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0460072610 con 0x7f04600d2460 2024-09-19T07:48:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.489+0000 7f045f7fe640 1 --2- 172.21.15.16:0/814013731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04600d2460 0x7f04600d28d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0458009d90 tx=0x7f045802a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3ce1b58119548c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.489+0000 7f045dffb640 1 -- 172.21.15.16:0/814013731 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458035020 con 0x7f04600d2460 2024-09-19T07:48:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.489+0000 7f045dffb640 1 -- 172.21.15.16:0/814013731 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0458007540 con 0x7f04600d2460 2024-09-19T07:48:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.489+0000 7f045dffb640 1 -- 172.21.15.16:0/814013731 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454001670 con 0x7f04600d2460 2024-09-19T07:48:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.490+0000 7f046570a640 1 -- 172.21.15.16:0/814013731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04600d2460 msgr2=0x7f04600d28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.490+0000 7f046570a640 1 --2- 172.21.15.16:0/814013731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04600d2460 0x7f04600d28d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0458009d90 tx=0x7f045802a5b0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.490+0000 7f046570a640 1 -- 172.21.15.16:0/814013731 shutdown_connections 2024-09-19T07:48:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.490+0000 7f046570a640 1 --2- 172.21.15.16:0/814013731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04600d2460 0x7f04600d28d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.490+0000 7f046570a640 1 -- 172.21.15.16:0/814013731 >> 172.21.15.16:0/814013731 conn(0x7f04600701d0 msgr2=0x7f04600705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.490+0000 7f046570a640 1 -- 172.21.15.16:0/814013731 shutdown_connections 2024-09-19T07:48:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.490+0000 7f046570a640 1 -- 172.21.15.16:0/814013731 wait complete. 2024-09-19T07:48:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.491+0000 7f046570a640 1 Processor -- start 2024-09-19T07:48:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.491+0000 7f046570a640 1 -- start start 2024-09-19T07:48:58.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.492+0000 7f046570a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04600cbea0 0x7f046014bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:58.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.492+0000 7f046570a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04600d2460 0x7f046006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:58.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.492+0000 7f046570a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f046015e2e0 0x7f046006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:58.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.492+0000 7f046570a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f046015e6c0 con 0x7f04600d2460 2024-09-19T07:48:58.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.492+0000 7f046570a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f04600d2e10 con 0x7f04600cbea0 2024-09-19T07:48:58.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.492+0000 7f046570a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f04600d0df0 con 0x7f046015e2e0 2024-09-19T07:48:58.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.492+0000 7f045e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04600d2460 0x7f046006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:58.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.492+0000 7f045e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04600d2460 0x7f046006cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59422/0 (socket says 172.21.15.16:59422) 2024-09-19T07:48:58.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.492+0000 7f045e7fc640 1 -- 172.21.15.16:0/2087838293 learned_addr learned my addr 172.21.15.16:0/2087838293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:58.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.492+0000 7f045f7fe640 1 --2- 172.21.15.16:0/2087838293 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f046015e2e0 0x7f046006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:58.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.492+0000 7f045effd640 1 --2- 172.21.15.16:0/2087838293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04600cbea0 0x7f046014bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.492+0000 7f045e7fc640 1 -- 172.21.15.16:0/2087838293 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f046015e2e0 msgr2=0x7f046006d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.492+0000 7f045e7fc640 1 --2- 172.21.15.16:0/2087838293 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f046015e2e0 0x7f046006d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.492+0000 7f045e7fc640 1 -- 172.21.15.16:0/2087838293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04600cbea0 msgr2=0x7f046014bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.492+0000 7f045e7fc640 1 --2- 172.21.15.16:0/2087838293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04600cbea0 0x7f046014bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.492+0000 7f045e7fc640 1 -- 172.21.15.16:0/2087838293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04601f8220 con 0x7f04600d2460 2024-09-19T07:48:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.492+0000 7f045effd640 1 --2- 172.21.15.16:0/2087838293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04600cbea0 0x7f046014bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.493+0000 7f045e7fc640 1 --2- 172.21.15.16:0/2087838293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04600d2460 0x7f046006cd20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f045400c4d0 tx=0x7f045400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.493+0000 7f04477fe640 1 -- 172.21.15.16:0/2087838293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454017070 con 0x7f04600d2460 2024-09-19T07:48:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.493+0000 7f04477fe640 1 -- 172.21.15.16:0/2087838293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0454005ce0 con 0x7f04600d2460 2024-09-19T07:48:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.493+0000 7f046570a640 1 -- 172.21.15.16:0/2087838293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04601581d0 con 0x7f04600d2460 2024-09-19T07:48:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.493+0000 7f04477fe640 1 -- 172.21.15.16:0/2087838293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454007290 con 0x7f04600d2460 2024-09-19T07:48:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.493+0000 7f046570a640 1 -- 172.21.15.16:0/2087838293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f046015e950 con 0x7f04600d2460 2024-09-19T07:48:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.494+0000 7f046570a640 1 -- 172.21.15.16:0/2087838293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04600d28d0 con 0x7f04600d2460 2024-09-19T07:48:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.494+0000 7f04477fe640 1 -- 172.21.15.16:0/2087838293 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f045401a070 con 0x7f04600d2460 2024-09-19T07:48:58.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.495+0000 7f04477fe640 1 --2- 172.21.15.16:0/2087838293 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f044c06c2b0 0x7f044c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:58.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.495+0000 7f04477fe640 1 -- 172.21.15.16:0/2087838293 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f045408a780 con 0x7f04600d2460 2024-09-19T07:48:58.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.495+0000 7f045effd640 1 --2- 172.21.15.16:0/2087838293 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f044c06c2b0 0x7f044c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:58.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.496+0000 7f045effd640 1 --2- 172.21.15.16:0/2087838293 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f044c06c2b0 0x7f044c06e770 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0450007fd0 tx=0x7f0450009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:58.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.499+0000 7f04477fe640 1 -- 172.21.15.16:0/2087838293 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f04601581d0 con 0x7f04600d2460 2024-09-19T07:48:58.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.745+0000 7f046570a640 1 -- 172.21.15.16:0/2087838293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f0460157c80 con 0x7f04600d2460 2024-09-19T07:48:58.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.747+0000 7f04477fe640 1 -- 172.21.15.16:0/2087838293 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f04540587a0 con 0x7f04600d2460 2024-09-19T07:48:58.748 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-19T07:48:58.748 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA== 2024-09-19T07:48:58.748 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-19T07:48:58.748 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-19T07:48:58.748 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T07:48:58.748 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-19T07:48:58.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.749+0000 7f046570a640 1 -- 172.21.15.16:0/2087838293 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f044c06c2b0 msgr2=0x7f044c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:58.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.749+0000 7f046570a640 1 --2- 172.21.15.16:0/2087838293 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f044c06c2b0 0x7f044c06e770 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0450007fd0 tx=0x7f0450009040 comp rx=0 tx=0).stop 2024-09-19T07:48:58.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.749+0000 7f046570a640 1 -- 172.21.15.16:0/2087838293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04600d2460 msgr2=0x7f046006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:58.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.749+0000 7f046570a640 1 --2- 172.21.15.16:0/2087838293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04600d2460 0x7f046006cd20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f045400c4d0 tx=0x7f045400c990 comp rx=0 tx=0).stop 2024-09-19T07:48:58.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.750+0000 7f046570a640 1 -- 172.21.15.16:0/2087838293 shutdown_connections 2024-09-19T07:48:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.750+0000 7f046570a640 1 --2- 172.21.15.16:0/2087838293 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f044c06c2b0 0x7f044c06e770 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.750+0000 7f046570a640 1 --2- 172.21.15.16:0/2087838293 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f046015e2e0 0x7f046006d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.750+0000 7f046570a640 1 --2- 172.21.15.16:0/2087838293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04600d2460 0x7f046006cd20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.750+0000 7f046570a640 1 --2- 172.21.15.16:0/2087838293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04600cbea0 0x7f046014bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.750+0000 7f046570a640 1 -- 172.21.15.16:0/2087838293 >> 172.21.15.16:0/2087838293 conn(0x7f04600701d0 msgr2=0x7f046015a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.750+0000 7f046570a640 1 -- 172.21.15.16:0/2087838293 shutdown_connections 2024-09-19T07:48:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.750+0000 7f046570a640 1 -- 172.21.15.16:0/2087838293 wait complete. 2024-09-19T07:48:58.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:48:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.862+0000 7f48d0f68640 1 Processor -- start 2024-09-19T07:48:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.862+0000 7f48d0f68640 1 -- start start 2024-09-19T07:48:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.863+0000 7f48d0f68640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f48cc0d1ca0 0x7f48cc0cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.863+0000 7f48d0f68640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48cc0d06d0 con 0x7f48cc15cb00 2024-09-19T07:48:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.863+0000 7f48d0f68640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48cc0d22d0 con 0x7f48cc0d02f0 2024-09-19T07:48:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.863+0000 7f48d0f68640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f48cc0b40b0 con 0x7f48cc0d1ca0 2024-09-19T07:48:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.864+0000 7f48c9d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f48cc0d1ca0 0x7f48cc0cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.864+0000 7f48c9d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f48cc0d1ca0 0x7f48cc0cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59432/0 (socket says 172.21.15.16:59432) 2024-09-19T07:48:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.864+0000 7f48c9d9b640 1 -- 172.21.15.16:0/4021192181 learned_addr learned my addr 172.21.15.16:0/4021192181 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.864+0000 7f48c9d9b640 1 -- 172.21.15.16:0/4021192181 >> v1:172.21.15.146:6789/0 conn(0x7f48cc0d02f0 legacy=0x7f48cc1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.864+0000 7f48c9d9b640 1 -- 172.21.15.16:0/4021192181 >> v1:172.21.15.16:6789/0 conn(0x7f48cc15cb00 legacy=0x7f48cc15cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.864+0000 7f48c9d9b640 1 -- 172.21.15.16:0/4021192181 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc0d25b0 con 0x7f48cc0d1ca0 2024-09-19T07:48:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.864+0000 7f48c9d9b640 1 --2- 172.21.15.16:0/4021192181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48cc0d1ca0 0x7f48cc0cfdb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009870 tx=0x7f48b402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece7a63a8509c18d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.865+0000 7f48c959a640 1 -- 172.21.15.16:0/4021192181 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b4037070 con 0x7f48cc0d1ca0 2024-09-19T07:48:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.865+0000 7f48c959a640 1 -- 172.21.15.16:0/4021192181 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b4032440 con 0x7f48cc0d1ca0 2024-09-19T07:48:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.865+0000 7f48c959a640 1 -- 172.21.15.16:0/4021192181 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b4032810 con 0x7f48cc0d1ca0 2024-09-19T07:48:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.865+0000 7f48d0f68640 1 -- 172.21.15.16:0/4021192181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48cc0d1ca0 msgr2=0x7f48cc0cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.865+0000 7f48d0f68640 1 --2- 172.21.15.16:0/4021192181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48cc0d1ca0 0x7f48cc0cfdb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009870 tx=0x7f48b402a210 comp rx=0 tx=0).stop 2024-09-19T07:48:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.865+0000 7f48d0f68640 1 -- 172.21.15.16:0/4021192181 shutdown_connections 2024-09-19T07:48:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.865+0000 7f48d0f68640 1 --2- 172.21.15.16:0/4021192181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48cc0d1ca0 0x7f48cc0cfdb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.865+0000 7f48d0f68640 1 -- 172.21.15.16:0/4021192181 >> 172.21.15.16:0/4021192181 conn(0x7f48cc067fe0 msgr2=0x7f48cc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.865+0000 7f48d0f68640 1 -- 172.21.15.16:0/4021192181 shutdown_connections 2024-09-19T07:48:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.865+0000 7f48d0f68640 1 -- 172.21.15.16:0/4021192181 wait complete. 2024-09-19T07:48:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.867+0000 7f48d0f68640 1 Processor -- start 2024-09-19T07:48:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.867+0000 7f48d0f68640 1 -- start start 2024-09-19T07:48:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.867+0000 7f48d0f68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48cc0d02f0 0x7f48cc0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.867+0000 7f48d0f68640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48cc0d1ca0 0x7f48cc0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.867+0000 7f48d0f68640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48cc15cb00 0x7f48cc13e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.867+0000 7f48d0f68640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc15d5f0 con 0x7f48cc0d02f0 2024-09-19T07:48:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.867+0000 7f48d0f68640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc0d2b70 con 0x7f48cc15cb00 2024-09-19T07:48:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.867+0000 7f48d0f68640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f48cc15d770 con 0x7f48cc0d1ca0 2024-09-19T07:48:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.867+0000 7f48ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48cc0d02f0 0x7f48cc0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.868+0000 7f48ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48cc0d02f0 0x7f48cc0b7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59438/0 (socket says 172.21.15.16:59438) 2024-09-19T07:48:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.868+0000 7f48ca59c640 1 -- 172.21.15.16:0/951267007 learned_addr learned my addr 172.21.15.16:0/951267007 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.868+0000 7f48cad9d640 1 --2- 172.21.15.16:0/951267007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48cc15cb00 0x7f48cc13e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.868+0000 7f48c9d9b640 1 --2- 172.21.15.16:0/951267007 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48cc0d1ca0 0x7f48cc0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.868+0000 7f48ca59c640 1 -- 172.21.15.16:0/951267007 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48cc0d1ca0 msgr2=0x7f48cc0c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.868+0000 7f48ca59c640 1 --2- 172.21.15.16:0/951267007 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48cc0d1ca0 0x7f48cc0c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.868+0000 7f48ca59c640 1 -- 172.21.15.16:0/951267007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48cc15cb00 msgr2=0x7f48cc13e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.868+0000 7f48ca59c640 1 --2- 172.21.15.16:0/951267007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48cc15cb00 0x7f48cc13e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.868+0000 7f48ca59c640 1 -- 172.21.15.16:0/951267007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc0ca190 con 0x7f48cc0d02f0 2024-09-19T07:48:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.868+0000 7f48cad9d640 1 --2- 172.21.15.16:0/951267007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48cc15cb00 0x7f48cc13e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.868+0000 7f48c9d9b640 1 --2- 172.21.15.16:0/951267007 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48cc0d1ca0 0x7f48cc0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.868+0000 7f48ca59c640 1 --2- 172.21.15.16:0/951267007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48cc0d02f0 0x7f48cc0b7a40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00c7c0 tx=0x7f48bc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.868+0000 7f48c2ffd640 1 -- 172.21.15.16:0/951267007 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc016020 con 0x7f48cc0d02f0 2024-09-19T07:48:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.868+0000 7f48c2ffd640 1 -- 172.21.15.16:0/951267007 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48bc006d50 con 0x7f48cc0d02f0 2024-09-19T07:48:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.868+0000 7f48d0f68640 1 -- 172.21.15.16:0/951267007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48cc0b6b80 con 0x7f48cc0d02f0 2024-09-19T07:48:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.868+0000 7f48c2ffd640 1 -- 172.21.15.16:0/951267007 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc003bb0 con 0x7f48cc0d02f0 2024-09-19T07:48:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.868+0000 7f48d0f68640 1 -- 172.21.15.16:0/951267007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48cc0c9450 con 0x7f48cc0d02f0 2024-09-19T07:48:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.869+0000 7f48d0f68640 1 -- 172.21.15.16:0/951267007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48cc0b70a0 con 0x7f48cc0d02f0 2024-09-19T07:48:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.870+0000 7f48c2ffd640 1 -- 172.21.15.16:0/951267007 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f48bc00a850 con 0x7f48cc0d02f0 2024-09-19T07:48:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.870+0000 7f48c2ffd640 1 --2- 172.21.15.16:0/951267007 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f48ac06c2b0 0x7f48ac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.871+0000 7f48c2ffd640 1 -- 172.21.15.16:0/951267007 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f48bc08b260 con 0x7f48cc0d02f0 2024-09-19T07:48:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.871+0000 7f48c9d9b640 1 --2- 172.21.15.16:0/951267007 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f48ac06c2b0 0x7f48ac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.871+0000 7f48c9d9b640 1 --2- 172.21.15.16:0/951267007 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f48ac06c2b0 0x7f48ac06e770 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f48b4034000 tx=0x7f48b402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:58.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:58.875+0000 7f48c2ffd640 1 -- 172.21.15.16:0/951267007 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f48cc0b70a0 con 0x7f48cc0d02f0 2024-09-19T07:48:59.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.085+0000 7f48d0f68640 1 -- 172.21.15.16:0/951267007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48cc0cfdb0 con 0x7f48cc0d02f0 2024-09-19T07:48:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.085+0000 7f48c2ffd640 1 -- 172.21.15.16:0/951267007 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5227 (secure 0 0 0) 0x7f48bc020020 con 0x7f48cc0d02f0 2024-09-19T07:48:59.086 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:48:59.086 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":33,"btime":"2024-09-19T07:48:52:612055+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2104039348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2104039348},{"type":"v1","addr":"172.21.15.16:6835","nonce":2104039348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/1109733621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1109733621},{"type":"v1","addr":"172.21.15.146:6836","nonce":1109733621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3602092462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3602092462},{"type":"v1","addr":"172.21.15.146:6837","nonce":3602092462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:48:49.954706+0000","modified":"2024-09-19T07:48:51.957927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5751},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5751":{"gid":5751,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6834/418904597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":418904597},{"type":"v1","addr":"172.21.15.16:6834","nonce":418904597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5751,"qdb_cluster":[5751]},"id":4}]} 2024-09-19T07:48:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 33 2024-09-19T07:48:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.087+0000 7f48d0f68640 1 -- 172.21.15.16:0/951267007 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f48ac06c2b0 msgr2=0x7f48ac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.087+0000 7f48d0f68640 1 --2- 172.21.15.16:0/951267007 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f48ac06c2b0 0x7f48ac06e770 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f48b4034000 tx=0x7f48b402e000 comp rx=0 tx=0).stop 2024-09-19T07:48:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.087+0000 7f48d0f68640 1 -- 172.21.15.16:0/951267007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48cc0d02f0 msgr2=0x7f48cc0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.087+0000 7f48d0f68640 1 --2- 172.21.15.16:0/951267007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48cc0d02f0 0x7f48cc0b7a40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00c7c0 tx=0x7f48bc00cc80 comp rx=0 tx=0).stop 2024-09-19T07:48:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.087+0000 7f48d0f68640 1 -- 172.21.15.16:0/951267007 shutdown_connections 2024-09-19T07:48:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.087+0000 7f48d0f68640 1 --2- 172.21.15.16:0/951267007 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f48ac06c2b0 0x7f48ac06e770 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.087+0000 7f48d0f68640 1 --2- 172.21.15.16:0/951267007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48cc15cb00 0x7f48cc13e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.087+0000 7f48d0f68640 1 --2- 172.21.15.16:0/951267007 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48cc0d1ca0 0x7f48cc0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.087+0000 7f48d0f68640 1 --2- 172.21.15.16:0/951267007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48cc0d02f0 0x7f48cc0b7a40 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.087+0000 7f48d0f68640 1 -- 172.21.15.16:0/951267007 >> 172.21.15.16:0/951267007 conn(0x7f48cc067fe0 msgr2=0x7f48cc15db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:59.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.087+0000 7f48d0f68640 1 -- 172.21.15.16:0/951267007 shutdown_connections 2024-09-19T07:48:59.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.087+0000 7f48d0f68640 1 -- 172.21.15.16:0/951267007 wait complete. 2024-09-19T07:48:59.501 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T07:48:49.954706+0000', 'modified': '2024-09-19T07:48:51.957927+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5751}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5751': {'gid': 5751, 'name': 'a', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6834/418904597', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6832', 'nonce': 418904597}, {'type': 'v1', 'addr': '172.21.15.16:6834', 'nonce': 418904597}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5751, 'qdb_cluster': [5751]} 2024-09-19T07:48:59.501 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T07:48:59.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T07:48:59.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.610+0000 7f621a2a5640 1 Processor -- start 2024-09-19T07:48:59.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.610+0000 7f621a2a5640 1 -- start start 2024-09-19T07:48:59.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.611+0000 7f621a2a5640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f621415bee0 0x7f62140d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.611+0000 7f621a2a5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f621415c2c0 0x7f62140d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.611+0000 7f621a2a5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f62140cf200 0x7f62140cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.611+0000 7f621a2a5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f62140ceaf0 con 0x7f621415c2c0 2024-09-19T07:48:59.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.611+0000 7f621a2a5640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f62140cdee0 con 0x7f621415bee0 2024-09-19T07:48:59.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.611+0000 7f621a2a5640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f62140ce1c0 con 0x7f62140cf200 2024-09-19T07:48:59.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.611+0000 7f62137fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f621415c2c0 0x7f62140d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.611+0000 7f62137fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f621415c2c0 0x7f62140d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59442/0 (socket says 172.21.15.16:59442) 2024-09-19T07:48:59.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.611+0000 7f62137fe640 1 -- 172.21.15.16:0/2349102774 learned_addr learned my addr 172.21.15.16:0/2349102774 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:59.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.611+0000 7f6213fff640 1 --2- 172.21.15.16:0/2349102774 >> v2:172.21.15.146:3300/0 conn(0x7f621415bee0 0x7f62140d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.611+0000 7f6218842640 1 --2- 172.21.15.16:0/2349102774 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f62140cf200 0x7f62140cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.611+0000 7f62137fe640 1 -- 172.21.15.16:0/2349102774 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f62140cf200 msgr2=0x7f62140cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:59.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.611+0000 7f62137fe640 1 --2- 172.21.15.16:0/2349102774 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f62140cf200 0x7f62140cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.611+0000 7f62137fe640 1 -- 172.21.15.16:0/2349102774 >> v2:172.21.15.146:3300/0 conn(0x7f621415bee0 msgr2=0x7f62140d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:59.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.611+0000 7f62137fe640 1 --2- 172.21.15.16:0/2349102774 >> v2:172.21.15.146:3300/0 conn(0x7f621415bee0 0x7f62140d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.611+0000 7f62137fe640 1 -- 172.21.15.16:0/2349102774 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62140ce4a0 con 0x7f621415c2c0 2024-09-19T07:48:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.612+0000 7f62137fe640 1 --2- 172.21.15.16:0/2349102774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f621415c2c0 0x7f62140d09f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f62140b2f40 tx=0x7f620802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a19db61e6bd3ed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.612+0000 7f6212ffd640 1 -- 172.21.15.16:0/2349102774 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208037070 con 0x7f621415c2c0 2024-09-19T07:48:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.612+0000 7f6212ffd640 1 -- 172.21.15.16:0/2349102774 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6208032440 con 0x7f621415c2c0 2024-09-19T07:48:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.612+0000 7f6212ffd640 1 -- 172.21.15.16:0/2349102774 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208032810 con 0x7f621415c2c0 2024-09-19T07:48:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.613+0000 7f621a2a5640 1 -- 172.21.15.16:0/2349102774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f621415c2c0 msgr2=0x7f62140d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.613+0000 7f621a2a5640 1 --2- 172.21.15.16:0/2349102774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f621415c2c0 0x7f62140d09f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f62140b2f40 tx=0x7f620802a1e0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.613+0000 7f621a2a5640 1 -- 172.21.15.16:0/2349102774 shutdown_connections 2024-09-19T07:48:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.613+0000 7f621a2a5640 1 --2- 172.21.15.16:0/2349102774 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f62140cf200 0x7f62140cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.613+0000 7f621a2a5640 1 --2- 172.21.15.16:0/2349102774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f621415c2c0 0x7f62140d09f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.613+0000 7f621a2a5640 1 --2- 172.21.15.16:0/2349102774 >> v2:172.21.15.146:3300/0 conn(0x7f621415bee0 0x7f62140d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.613+0000 7f621a2a5640 1 -- 172.21.15.16:0/2349102774 >> 172.21.15.16:0/2349102774 conn(0x7f621406bdf0 msgr2=0x7f621406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:59.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.613+0000 7f621a2a5640 1 -- 172.21.15.16:0/2349102774 shutdown_connections 2024-09-19T07:48:59.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.613+0000 7f621a2a5640 1 -- 172.21.15.16:0/2349102774 wait complete. 2024-09-19T07:48:59.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.614+0000 7f621a2a5640 1 Processor -- start 2024-09-19T07:48:59.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.614+0000 7f621a2a5640 1 -- start start 2024-09-19T07:48:59.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.615+0000 7f621a2a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62140cf200 0x7f62140cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.615+0000 7f621a2a5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f621415bee0 0x7f62140ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.615+0000 7f621a2a5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f621415c2c0 0x7f6214159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.615+0000 7f621a2a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f621415fd40 con 0x7f62140cf200 2024-09-19T07:48:59.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.615+0000 7f621a2a5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f62140cfbd0 con 0x7f621415c2c0 2024-09-19T07:48:59.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.615+0000 7f621a2a5640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f621415fec0 con 0x7f621415bee0 2024-09-19T07:48:59.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.615+0000 7f6213fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62140cf200 0x7f62140cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.615+0000 7f6213fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62140cf200 0x7f62140cc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59452/0 (socket says 172.21.15.16:59452) 2024-09-19T07:48:59.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.615+0000 7f6213fff640 1 -- 172.21.15.16:0/2899645073 learned_addr learned my addr 172.21.15.16:0/2899645073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:59.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.615+0000 7f62137fe640 1 --2- 172.21.15.16:0/2899645073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f621415bee0 0x7f62140ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.615+0000 7f6218842640 1 --2- 172.21.15.16:0/2899645073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f621415c2c0 0x7f6214159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.615+0000 7f6213fff640 1 -- 172.21.15.16:0/2899645073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f621415bee0 msgr2=0x7f62140ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.615+0000 7f6213fff640 1 --2- 172.21.15.16:0/2899645073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f621415bee0 0x7f62140ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.615+0000 7f6213fff640 1 -- 172.21.15.16:0/2899645073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f621415c2c0 msgr2=0x7f6214159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.615+0000 7f6213fff640 1 --2- 172.21.15.16:0/2899645073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f621415c2c0 0x7f6214159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.615+0000 7f6213fff640 1 -- 172.21.15.16:0/2899645073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6214159f50 con 0x7f62140cf200 2024-09-19T07:48:59.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.615+0000 7f6213fff640 1 --2- 172.21.15.16:0/2899645073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62140cf200 0x7f62140cc2d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f620400d800 tx=0x7f620400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:59.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.616+0000 7f6210ff9640 1 -- 172.21.15.16:0/2899645073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620400b6c0 con 0x7f62140cf200 2024-09-19T07:48:59.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.616+0000 7f6210ff9640 1 -- 172.21.15.16:0/2899645073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6204019070 con 0x7f62140cf200 2024-09-19T07:48:59.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.616+0000 7f6210ff9640 1 -- 172.21.15.16:0/2899645073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620400ba80 con 0x7f62140cf200 2024-09-19T07:48:59.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.616+0000 7f621a2a5640 1 -- 172.21.15.16:0/2899645073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621415a230 con 0x7f62140cf200 2024-09-19T07:48:59.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.616+0000 7f621a2a5640 1 -- 172.21.15.16:0/2899645073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6214158c30 con 0x7f62140cf200 2024-09-19T07:48:59.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.617+0000 7f6210ff9640 1 -- 172.21.15.16:0/2899645073 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f620400bc20 con 0x7f62140cf200 2024-09-19T07:48:59.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.618+0000 7f6210ff9640 1 --2- 172.21.15.16:0/2899645073 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f620006c2b0 0x7f620006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.618+0000 7f6210ff9640 1 -- 172.21.15.16:0/2899645073 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f620408b9d0 con 0x7f62140cf200 2024-09-19T07:48:59.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.618+0000 7f62137fe640 1 --2- 172.21.15.16:0/2899645073 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f620006c2b0 0x7f620006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.619+0000 7f62137fe640 1 --2- 172.21.15.16:0/2899645073 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f620006c2b0 0x7f620006e770 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6208034000 tx=0x7f620802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:59.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.619+0000 7f621d044740 1 Processor -- start 2024-09-19T07:48:59.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.619+0000 7f621d044740 1 -- start start 2024-09-19T07:48:59.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.619+0000 7f621d044740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b4c2c0 0x55e622b91a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.619+0000 7f621d044740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b49a80 0x55e622b4c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.619+0000 7f621d044740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b4da50 0x55e622b468b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.619+0000 7f621d044740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e622aae520 con 0x55e622b4da50 2024-09-19T07:48:59.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.619+0000 7f621d044740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55e622a563e0 con 0x55e622b49a80 2024-09-19T07:48:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.619+0000 7f621d044740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55e622aaa4e0 con 0x55e622b4c2c0 2024-09-19T07:48:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.620+0000 7f6218842640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b4da50 0x55e622b468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.620+0000 7f6218842640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b4da50 0x55e622b468b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59468/0 (socket says 172.21.15.16:59468) 2024-09-19T07:48:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.620+0000 7f6218842640 1 -- 172.21.15.16:0/3978476996 learned_addr learned my addr 172.21.15.16:0/3978476996 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.620+0000 7f62137fe640 1 --2- 172.21.15.16:0/3978476996 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b49a80 0x55e622b4c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.620+0000 7f6213fff640 1 --2- 172.21.15.16:0/3978476996 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b4c2c0 0x55e622b91a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.620+0000 7f6218842640 1 -- 172.21.15.16:0/3978476996 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b4c2c0 msgr2=0x55e622b91a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.620+0000 7f6218842640 1 --2- 172.21.15.16:0/3978476996 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b4c2c0 0x55e622b91a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.620+0000 7f6218842640 1 -- 172.21.15.16:0/3978476996 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b49a80 msgr2=0x55e622b4c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.620+0000 7f6218842640 1 --2- 172.21.15.16:0/3978476996 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b49a80 0x55e622b4c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.620+0000 7f6218842640 1 -- 172.21.15.16:0/3978476996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e622a712f0 con 0x55e622b4da50 2024-09-19T07:48:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.620+0000 7f6218842640 1 --2- 172.21.15.16:0/3978476996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b4da50 0x55e622b468b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f620c007410 tx=0x7f620c0078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bf2f4494767de4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.620+0000 7f61f9ffb640 1 -- 172.21.15.16:0/3978476996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c0147e0 con 0x55e622b4da50 2024-09-19T07:48:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.620+0000 7f61f9ffb640 1 -- 172.21.15.16:0/3978476996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f620c01a070 con 0x55e622b4da50 2024-09-19T07:48:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.620+0000 7f61f9ffb640 1 -- 172.21.15.16:0/3978476996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c014b80 con 0x55e622b4da50 2024-09-19T07:48:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.621+0000 7f621d044740 1 -- 172.21.15.16:0/3978476996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b4da50 msgr2=0x55e622b468b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.621+0000 7f621d044740 1 --2- 172.21.15.16:0/3978476996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b4da50 0x55e622b468b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f620c007410 tx=0x7f620c0078d0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.621+0000 7f621d044740 1 -- 172.21.15.16:0/3978476996 shutdown_connections 2024-09-19T07:48:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.621+0000 7f621d044740 1 --2- 172.21.15.16:0/3978476996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b4da50 0x55e622b468b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.621+0000 7f621d044740 1 --2- 172.21.15.16:0/3978476996 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b49a80 0x55e622b4c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.621+0000 7f621d044740 1 --2- 172.21.15.16:0/3978476996 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b4c2c0 0x55e622b91a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.621+0000 7f621d044740 1 -- 172.21.15.16:0/3978476996 >> 172.21.15.16:0/3978476996 conn(0x55e622b48a80 msgr2=0x55e622b90f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.621+0000 7f621d044740 1 -- 172.21.15.16:0/3978476996 shutdown_connections 2024-09-19T07:48:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.621+0000 7f621d044740 1 -- 172.21.15.16:0/3978476996 wait complete. 2024-09-19T07:48:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.621+0000 7f621d044740 1 -- start start 2024-09-19T07:48:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.622+0000 7f621d044740 1 Processor -- start 2024-09-19T07:48:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.622+0000 7f621d044740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b4c2c0 0x55e622b8e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.622+0000 7f621d044740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b4da50 0x55e622b73ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.622+0000 7f621d044740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b61be0 0x55e622b741e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.622+0000 7f621d044740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e622b47f20 con 0x55e622b61be0 2024-09-19T07:48:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.622+0000 7f621d044740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55e622a37450 con 0x55e622b4da50 2024-09-19T07:48:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.622+0000 7f621d044740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55e622b92290 con 0x55e622b4c2c0 2024-09-19T07:48:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.623+0000 7f621d044740 10 client.0 _ll_register_callbacks cb 0x55e6227c0ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.623+0000 7f6218842640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b61be0 0x55e622b741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.623+0000 7f6218842640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b61be0 0x55e622b741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59472/0 (socket says 172.21.15.16:59472) 2024-09-19T07:48:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.623+0000 7f6218842640 1 -- 172.21.15.16:0/2450905817 learned_addr learned my addr 172.21.15.16:0/2450905817 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.623+0000 7f6213fff640 1 --2- 172.21.15.16:0/2450905817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b4c2c0 0x55e622b8e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.623+0000 7f62137fe640 1 --2- 172.21.15.16:0/2450905817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b4da50 0x55e622b73ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.623+0000 7f6218842640 1 -- 172.21.15.16:0/2450905817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b4c2c0 msgr2=0x55e622b8e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.623+0000 7f6218842640 1 --2- 172.21.15.16:0/2450905817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b4c2c0 0x55e622b8e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.623+0000 7f6218842640 1 -- 172.21.15.16:0/2450905817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b4da50 msgr2=0x55e622b73ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.623+0000 7f6218842640 1 --2- 172.21.15.16:0/2450905817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b4da50 0x55e622b73ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.623+0000 7f6218842640 1 -- 172.21.15.16:0/2450905817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6227c12d0 con 0x55e622b61be0 2024-09-19T07:48:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.623+0000 7f6218842640 1 --2- 172.21.15.16:0/2450905817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b61be0 0x55e622b741e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f620c000f80 tx=0x7f620c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e72ddd174e1fa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.623+0000 7f61f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:48:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.623+0000 7f61f2ffd640 1 -- 172.21.15.16:0/2450905817 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c014d20 con 0x55e622b61be0 2024-09-19T07:48:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.623+0000 7f61f2ffd640 1 -- 172.21.15.16:0/2450905817 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f620c02a480 con 0x55e622b61be0 2024-09-19T07:48:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.623+0000 7f621d044740 1 -- 172.21.15.16:0/2450905817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f620c01a070 con 0x55e622b61be0 2024-09-19T07:48:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.623+0000 7f61f2ffd640 1 -- 172.21.15.16:0/2450905817 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f620c032050 con 0x55e622b61be0 2024-09-19T07:48:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.623+0000 7f621d044740 10 client.5895 fetch_fsmap learned FSMap version 33 2024-09-19T07:48:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.623+0000 7f621d044740 1 -- 172.21.15.16:0/2450905817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55e62288b6b0 con 0x55e622b61be0 2024-09-19T07:48:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.624+0000 7f61f2ffd640 1 -- 172.21.15.16:0/2450905817 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f620c01a070 con 0x55e622b61be0 2024-09-19T07:48:59.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.624+0000 7f61f2ffd640 1 -- 172.21.15.16:0/2450905817 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 33) ==== 3502+0+0 (secure 0 0 0) 0x7f620c035020 con 0x55e622b61be0 2024-09-19T07:48:59.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.624+0000 7f621d044740 10 client.5895 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T07:48:59.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.624+0000 7f621d044740 10 client.5895 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T07:48:59.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.624+0000 7f621d044740 20 client.5895 populate_metadata read hostname 'smithi016' 2024-09-19T07:48:59.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.624+0000 7f621d044740 1 --2- 172.21.15.16:0/2450905817 >> [v2:172.21.15.16:6832/418904597,v1:172.21.15.16:6834/418904597] conn(0x55e622b95780 0x55e622b970c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.624+0000 7f621d044740 4 client.5895 mds_command: new command op to 5751 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:48:59.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.624+0000 7f621d044740 1 -- 172.21.15.16:0/2450905817 --> [v2:172.21.15.16:6832/418904597,v1:172.21.15.16:6834/418904597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e622abbc50 con 0x55e622b95780 2024-09-19T07:48:59.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.624+0000 7f6213fff640 1 --2- 172.21.15.16:0/2450905817 >> [v2:172.21.15.16:6832/418904597,v1:172.21.15.16:6834/418904597] conn(0x55e622b95780 0x55e622b970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.624+0000 7f6213fff640 1 --2- 172.21.15.16:0/2450905817 >> [v2:172.21.15.16:6832/418904597,v1:172.21.15.16:6834/418904597] conn(0x55e622b95780 0x55e622b970c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4c981013f468a1a server_cookie=888b2f5831780f0a in_seq=0 out_seq=0 2024-09-19T07:48:59.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.624+0000 7f61f2ffd640 10 client.5895 ms_handle_connect on v2:172.21.15.16:6832/418904597 2024-09-19T07:48:59.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.625+0000 7f61f2ffd640 1 -- 172.21.15.16:0/2450905817 <== mds.0 v2:172.21.15.16:6832/418904597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e622abbc50 con 0x55e622b95780 2024-09-19T07:48:59.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.625+0000 7f61f2ffd640 10 client.5895 handle_command_reply: tid=0 2024-09-19T07:48:59.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.625+0000 7f621d044740 1 client.5895 shutdown 2024-09-19T07:48:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.625+0000 7f621d044740 1 -- 172.21.15.16:0/2450905817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b61be0 msgr2=0x55e622b741e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.625+0000 7f621d044740 1 --2- 172.21.15.16:0/2450905817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b61be0 0x55e622b741e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f620c000f80 tx=0x7f620c004ab0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.625+0000 7f621d044740 1 -- 172.21.15.16:0/2450905817 shutdown_connections 2024-09-19T07:48:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.625+0000 7f621d044740 1 --2- 172.21.15.16:0/2450905817 >> [v2:172.21.15.16:6832/418904597,v1:172.21.15.16:6834/418904597] conn(0x55e622b95780 0x55e622b970c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.625+0000 7f621d044740 1 --2- 172.21.15.16:0/2450905817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b61be0 0x55e622b741e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.625+0000 7f621d044740 1 --2- 172.21.15.16:0/2450905817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b4da50 0x55e622b73ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.625+0000 7f621d044740 1 --2- 172.21.15.16:0/2450905817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b4c2c0 0x55e622b8e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.625+0000 7f61f2ffd640 0 client.5895 ms_handle_reset on v2:172.21.15.16:6832/418904597 2024-09-19T07:48:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.625+0000 7f621d044740 1 -- 172.21.15.16:0/2450905817 >> 172.21.15.16:0/2450905817 conn(0x55e622b49a80 msgr2=0x55e622b94440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.625+0000 7f621d044740 1 -- 172.21.15.16:0/2450905817 shutdown_connections 2024-09-19T07:48:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.625+0000 7f621d044740 1 -- 172.21.15.16:0/2450905817 wait complete. 2024-09-19T07:48:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.625+0000 7f621d044740 20 client.5895 trim_cache size 0 max 16384 2024-09-19T07:48:59.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.644+0000 7f621d044740 1 Processor -- start 2024-09-19T07:48:59.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.644+0000 7f621d044740 1 -- start start 2024-09-19T07:48:59.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.644+0000 7f621d044740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b4c2c0 0x55e622b97040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.644+0000 7f621d044740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b4da50 0x55e622b92a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.644+0000 7f621d044740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b61be0 0x55e622b92fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.644+0000 7f621d044740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62141fff60 con 0x55e622b61be0 2024-09-19T07:48:59.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.644+0000 7f621d044740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f62140c5c40 con 0x55e622b4c2c0 2024-09-19T07:48:59.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.644+0000 7f621d044740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f62141ff910 con 0x55e622b4da50 2024-09-19T07:48:59.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.645+0000 7f6218842640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b61be0 0x55e622b92fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.645+0000 7f6218842640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b61be0 0x55e622b92fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59478/0 (socket says 172.21.15.16:59478) 2024-09-19T07:48:59.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.645+0000 7f6218842640 1 -- 172.21.15.16:0/2603410759 learned_addr learned my addr 172.21.15.16:0/2603410759 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:59.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.645+0000 7f6213fff640 1 --2- 172.21.15.16:0/2603410759 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b4c2c0 0x55e622b97040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.645+0000 7f62137fe640 1 --2- 172.21.15.16:0/2603410759 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b4da50 0x55e622b92a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.645+0000 7f6218842640 1 -- 172.21.15.16:0/2603410759 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b4da50 msgr2=0x55e622b92a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:59.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.645+0000 7f6218842640 1 --2- 172.21.15.16:0/2603410759 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b4da50 0x55e622b92a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.645+0000 7f6218842640 1 -- 172.21.15.16:0/2603410759 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b4c2c0 msgr2=0x55e622b97040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:59.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.645+0000 7f6218842640 1 --2- 172.21.15.16:0/2603410759 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b4c2c0 0x55e622b97040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.645+0000 7f6218842640 1 -- 172.21.15.16:0/2603410759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e622b70830 con 0x55e622b61be0 2024-09-19T07:48:59.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.645+0000 7f62137fe640 1 --2- 172.21.15.16:0/2603410759 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b4da50 0x55e622b92a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:59.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.645+0000 7f6218842640 1 --2- 172.21.15.16:0/2603410759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b61be0 0x55e622b92fd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f620c014870 tx=0x7f620c021070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88fb3db82939a62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:59.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.645+0000 7f61f97fa640 1 -- 172.21.15.16:0/2603410759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c014d20 con 0x55e622b61be0 2024-09-19T07:48:59.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.645+0000 7f61f97fa640 1 -- 172.21.15.16:0/2603410759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f620c01a040 con 0x55e622b61be0 2024-09-19T07:48:59.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.645+0000 7f61f97fa640 1 -- 172.21.15.16:0/2603410759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c015ab0 con 0x55e622b61be0 2024-09-19T07:48:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.646+0000 7f621d044740 1 -- 172.21.15.16:0/2603410759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b61be0 msgr2=0x55e622b92fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.646+0000 7f621d044740 1 --2- 172.21.15.16:0/2603410759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b61be0 0x55e622b92fd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f620c014870 tx=0x7f620c021070 comp rx=0 tx=0).stop 2024-09-19T07:48:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.646+0000 7f621d044740 1 -- 172.21.15.16:0/2603410759 shutdown_connections 2024-09-19T07:48:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.646+0000 7f621d044740 1 --2- 172.21.15.16:0/2603410759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b61be0 0x55e622b92fd0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.646+0000 7f621d044740 1 --2- 172.21.15.16:0/2603410759 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b4da50 0x55e622b92a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.646+0000 7f621d044740 1 --2- 172.21.15.16:0/2603410759 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b4c2c0 0x55e622b97040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.646+0000 7f621d044740 1 -- 172.21.15.16:0/2603410759 >> 172.21.15.16:0/2603410759 conn(0x55e622b49a80 msgr2=0x55e622b755e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.646+0000 7f621d044740 1 -- 172.21.15.16:0/2603410759 shutdown_connections 2024-09-19T07:48:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.646+0000 7f621d044740 1 -- 172.21.15.16:0/2603410759 wait complete. 2024-09-19T07:48:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.646+0000 7f621d044740 1 -- start start 2024-09-19T07:48:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.647+0000 7f621d044740 1 Processor -- start 2024-09-19T07:48:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.647+0000 7f621d044740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b4c2c0 0x55e622b457d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.647+0000 7f621d044740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b4da50 0x55e622bb44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.647+0000 7f621d044740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b61be0 0x55e622b9f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.647+0000 7f621d044740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e622b95600 con 0x55e622b4da50 2024-09-19T07:48:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.647+0000 7f621d044740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55e622b70a00 con 0x55e622b4c2c0 2024-09-19T07:48:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.647+0000 7f621d044740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55e622b95c50 con 0x55e622b61be0 2024-09-19T07:48:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.647+0000 7f621d044740 10 client.0 _ll_register_callbacks cb 0x7f62080096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.648+0000 7f62137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b4da50 0x55e622bb44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.648+0000 7f62137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b4da50 0x55e622bb44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59480/0 (socket says 172.21.15.16:59480) 2024-09-19T07:48:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.648+0000 7f62137fe640 1 -- 172.21.15.16:0/104265619 learned_addr learned my addr 172.21.15.16:0/104265619 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.648+0000 7f6213fff640 1 --2- 172.21.15.16:0/104265619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b4c2c0 0x55e622b457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.648+0000 7f6218842640 1 --2- 172.21.15.16:0/104265619 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b61be0 0x55e622b9f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.648+0000 7f62137fe640 1 -- 172.21.15.16:0/104265619 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b61be0 msgr2=0x55e622b9f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.648+0000 7f62137fe640 1 --2- 172.21.15.16:0/104265619 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b61be0 0x55e622b9f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.648+0000 7f62137fe640 1 -- 172.21.15.16:0/104265619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b4c2c0 msgr2=0x55e622b457d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.648+0000 7f62137fe640 1 --2- 172.21.15.16:0/104265619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b4c2c0 0x55e622b457d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.648+0000 7f62137fe640 1 -- 172.21.15.16:0/104265619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e622b932e0 con 0x55e622b4da50 2024-09-19T07:48:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.648+0000 7f62137fe640 1 --2- 172.21.15.16:0/104265619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b4da50 0x55e622bb44b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f620802a220 tx=0x7f6208004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e225db8ac204f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.648+0000 7f61f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:48:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.648+0000 7f61f2ffd640 1 -- 172.21.15.16:0/104265619 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208037040 con 0x55e622b4da50 2024-09-19T07:48:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.648+0000 7f621d044740 1 -- 172.21.15.16:0/104265619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f620c01a040 con 0x55e622b4da50 2024-09-19T07:48:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.648+0000 7f61f2ffd640 1 -- 172.21.15.16:0/104265619 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f62080489d0 con 0x55e622b4da50 2024-09-19T07:48:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.648+0000 7f61f2ffd640 1 -- 172.21.15.16:0/104265619 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6208048bd0 con 0x55e622b4da50 2024-09-19T07:48:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.648+0000 7f621d044740 10 client.5901 fetch_fsmap learned FSMap version 33 2024-09-19T07:48:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.648+0000 7f621d044740 1 -- 172.21.15.16:0/104265619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55e622b6f800 con 0x55e622b4da50 2024-09-19T07:48:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.649+0000 7f61f2ffd640 1 -- 172.21.15.16:0/104265619 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f620c01a040 con 0x55e622b4da50 2024-09-19T07:48:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.649+0000 7f61f2ffd640 1 -- 172.21.15.16:0/104265619 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 33) ==== 3502+0+0 (secure 0 0 0) 0x7f6208046040 con 0x55e622b4da50 2024-09-19T07:48:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.649+0000 7f621d044740 10 client.5901 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T07:48:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.649+0000 7f621d044740 10 client.5901 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T07:48:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.649+0000 7f621d044740 20 client.5901 populate_metadata read hostname 'smithi016' 2024-09-19T07:48:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.649+0000 7f621d044740 1 --2- 172.21.15.16:0/104265619 >> [v2:172.21.15.16:6832/418904597,v1:172.21.15.16:6834/418904597] conn(0x55e622c11fc0 0x55e622b6e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.649+0000 7f621d044740 4 client.5901 mds_command: new command op to 5751 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:48:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.649+0000 7f621d044740 1 -- 172.21.15.16:0/104265619 --> [v2:172.21.15.16:6832/418904597,v1:172.21.15.16:6834/418904597] -- command(tid 0: {"prefix": "status"}) -- 0x55e622b76100 con 0x55e622c11fc0 2024-09-19T07:48:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.649+0000 7f6213fff640 1 --2- 172.21.15.16:0/104265619 >> [v2:172.21.15.16:6832/418904597,v1:172.21.15.16:6834/418904597] conn(0x55e622c11fc0 0x55e622b6e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.649+0000 7f61f2ffd640 10 client.5901 ms_handle_connect on v2:172.21.15.16:6832/418904597 2024-09-19T07:48:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.649+0000 7f6213fff640 1 --2- 172.21.15.16:0/104265619 >> [v2:172.21.15.16:6832/418904597,v1:172.21.15.16:6834/418904597] conn(0x55e622c11fc0 0x55e622b6e020 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=230283e8e83ac1ea server_cookie=bccaef246b0b9ce5 in_seq=0 out_seq=0 2024-09-19T07:48:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.649+0000 7f61f2ffd640 1 -- 172.21.15.16:0/104265619 <== mds.0 v2:172.21.15.16:6832/418904597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55e622b76100 con 0x55e622c11fc0 2024-09-19T07:48:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.649+0000 7f61f2ffd640 10 client.5901 handle_command_reply: tid=0 2024-09-19T07:48:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.649+0000 7f621d044740 1 client.5901 shutdown 2024-09-19T07:48:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621d044740 1 -- 172.21.15.16:0/104265619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b4da50 msgr2=0x55e622bb44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621d044740 1 --2- 172.21.15.16:0/104265619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b4da50 0x55e622bb44b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f620802a220 tx=0x7f6208004730 comp rx=0 tx=0).stop 2024-09-19T07:48:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621d044740 1 -- 172.21.15.16:0/104265619 shutdown_connections 2024-09-19T07:48:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621d044740 1 --2- 172.21.15.16:0/104265619 >> [v2:172.21.15.16:6832/418904597,v1:172.21.15.16:6834/418904597] conn(0x55e622c11fc0 0x55e622b6e020 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621d044740 1 --2- 172.21.15.16:0/104265619 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55e622b61be0 0x55e622b9f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621d044740 1 --2- 172.21.15.16:0/104265619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e622b4da50 0x55e622bb44b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f61f2ffd640 0 client.5901 ms_handle_reset on v2:172.21.15.16:6832/418904597 2024-09-19T07:48:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621d044740 1 --2- 172.21.15.16:0/104265619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e622b4c2c0 0x55e622b457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621d044740 1 -- 172.21.15.16:0/104265619 >> 172.21.15.16:0/104265619 conn(0x55e622b49a80 msgr2=0x55e622b90ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621d044740 1 -- 172.21.15.16:0/104265619 shutdown_connections 2024-09-19T07:48:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621d044740 1 -- 172.21.15.16:0/104265619 wait complete. 2024-09-19T07:48:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621d044740 20 client.5901 trim_cache size 0 max 16384 2024-09-19T07:48:59.656 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:48:59.656 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb", 2024-09-19T07:48:59.657 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-19T07:48:59.657 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5751, 2024-09-19T07:48:59.657 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T07:48:59.657 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T07:48:59.657 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T07:48:59.657 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 9.0430102310000002, 2024-09-19T07:48:59.657 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 33, 2024-09-19T07:48:59.657 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 70, 2024-09-19T07:48:59.657 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 70, 2024-09-19T07:48:59.657 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 14.083482217 2024-09-19T07:48:59.657 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:48:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621a2a5640 1 -- 172.21.15.16:0/2899645073 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f620006c2b0 msgr2=0x7f620006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621a2a5640 1 --2- 172.21.15.16:0/2899645073 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f620006c2b0 0x7f620006e770 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6208034000 tx=0x7f620802e000 comp rx=0 tx=0).stop 2024-09-19T07:48:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621a2a5640 1 -- 172.21.15.16:0/2899645073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62140cf200 msgr2=0x7f62140cc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621a2a5640 1 --2- 172.21.15.16:0/2899645073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62140cf200 0x7f62140cc2d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f620400d800 tx=0x7f620400dcc0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621a2a5640 1 -- 172.21.15.16:0/2899645073 shutdown_connections 2024-09-19T07:48:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621a2a5640 1 --2- 172.21.15.16:0/2899645073 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f620006c2b0 0x7f620006e770 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621a2a5640 1 --2- 172.21.15.16:0/2899645073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f621415c2c0 0x7f6214159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621a2a5640 1 --2- 172.21.15.16:0/2899645073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f621415bee0 0x7f62140ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621a2a5640 1 --2- 172.21.15.16:0/2899645073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62140cf200 0x7f62140cc2d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621a2a5640 1 -- 172.21.15.16:0/2899645073 >> 172.21.15.16:0/2899645073 conn(0x7f621406bdf0 msgr2=0x7f6214150ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.650+0000 7f621a2a5640 1 -- 172.21.15.16:0/2899645073 shutdown_connections 2024-09-19T07:48:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.651+0000 7f621a2a5640 1 -- 172.21.15.16:0/2899645073 wait complete. 2024-09-19T07:48:59.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:48:59.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.763+0000 7f689c1b7640 1 Processor -- start 2024-09-19T07:48:59.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.764+0000 7f689c1b7640 1 -- start start 2024-09-19T07:48:59.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.765+0000 7f689c1b7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f68940d0070 0x7f689415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.765+0000 7f689c1b7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f68940d4130 0x7f68940d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.765+0000 7f689c1b7640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68940d1b10 con 0x7f689415f0d0 2024-09-19T07:48:59.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.765+0000 7f689c1b7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f68940cf2b0 con 0x7f68940d0070 2024-09-19T07:48:59.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.765+0000 7f689c1b7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f68940cf590 con 0x7f68940d4130 2024-09-19T07:48:59.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.765+0000 7f6899752640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f68940d0070 0x7f689415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.765+0000 7f6899752640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f68940d0070 0x7f689415f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59482/0 (socket says 172.21.15.16:59482) 2024-09-19T07:48:59.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.765+0000 7f6899752640 1 -- 172.21.15.16:0/2971339610 learned_addr learned my addr 172.21.15.16:0/2971339610 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:59.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.765+0000 7f689a754640 1 --2- 172.21.15.16:0/2971339610 >> v2:172.21.15.146:3300/0 conn(0x7f68940d4130 0x7f68940d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.765+0000 7f6899752640 1 -- 172.21.15.16:0/2971339610 >> v2:172.21.15.146:3300/0 conn(0x7f68940d4130 msgr2=0x7f68940d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:59.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.765+0000 7f6899752640 1 --2- 172.21.15.16:0/2971339610 >> v2:172.21.15.146:3300/0 conn(0x7f68940d4130 0x7f68940d4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.765+0000 7f6899752640 1 -- 172.21.15.16:0/2971339610 >> v1:172.21.15.146:6789/0 conn(0x7f689415f0d0 legacy=0x7f68940d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.765+0000 7f6899752640 1 -- 172.21.15.16:0/2971339610 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68940cf870 con 0x7f68940d0070 2024-09-19T07:48:59.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.766+0000 7f6899752640 1 --2- 172.21.15.16:0/2971339610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68940d0070 0x7f689415f4b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f68940d04e0 tx=0x7f688402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=714b0fa928b7be4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:59.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.766+0000 7f6898f51640 1 -- 172.21.15.16:0/2971339610 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884037070 con 0x7f68940d0070 2024-09-19T07:48:59.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.766+0000 7f6898f51640 1 -- 172.21.15.16:0/2971339610 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6884032440 con 0x7f68940d0070 2024-09-19T07:48:59.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.766+0000 7f6898f51640 1 -- 172.21.15.16:0/2971339610 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884032810 con 0x7f68940d0070 2024-09-19T07:48:59.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.766+0000 7f689c1b7640 1 -- 172.21.15.16:0/2971339610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68940d0070 msgr2=0x7f689415f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.766+0000 7f689c1b7640 1 --2- 172.21.15.16:0/2971339610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68940d0070 0x7f689415f4b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f68940d04e0 tx=0x7f688402a1e0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.767+0000 7f689c1b7640 1 -- 172.21.15.16:0/2971339610 shutdown_connections 2024-09-19T07:48:59.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.767+0000 7f689c1b7640 1 --2- 172.21.15.16:0/2971339610 >> v2:172.21.15.146:3300/0 conn(0x7f68940d4130 0x7f68940d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.767+0000 7f689c1b7640 1 --2- 172.21.15.16:0/2971339610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68940d0070 0x7f689415f4b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.767+0000 7f689c1b7640 1 -- 172.21.15.16:0/2971339610 >> 172.21.15.16:0/2971339610 conn(0x7f689409d760 msgr2=0x7f689408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:59.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.767+0000 7f689c1b7640 1 -- 172.21.15.16:0/2971339610 shutdown_connections 2024-09-19T07:48:59.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.767+0000 7f689c1b7640 1 -- 172.21.15.16:0/2971339610 wait complete. 2024-09-19T07:48:59.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.768+0000 7f689c1b7640 1 Processor -- start 2024-09-19T07:48:59.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.768+0000 7f689c1b7640 1 -- start start 2024-09-19T07:48:59.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f689c1b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68940d0070 0x7f68941f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f689c1b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68940d4130 0x7f68940cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f689c1b7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f689415f0d0 0x7f68941e8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f689c1b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68940cef60 con 0x7f68940d4130 2024-09-19T07:48:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f689c1b7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f68940d4aa0 con 0x7f689415f0d0 2024-09-19T07:48:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f689c1b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f68940b2f70 con 0x7f68940d0070 2024-09-19T07:48:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f6899752640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68940d4130 0x7f68940cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f6899752640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68940d4130 0x7f68940cde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59498/0 (socket says 172.21.15.16:59498) 2024-09-19T07:48:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f6899752640 1 -- 172.21.15.16:0/210664739 learned_addr learned my addr 172.21.15.16:0/210664739 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:48:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f689a754640 1 --2- 172.21.15.16:0/210664739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f689415f0d0 0x7f68941e8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f6899f53640 1 --2- 172.21.15.16:0/210664739 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68940d0070 0x7f68941f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f6899752640 1 -- 172.21.15.16:0/210664739 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68940d0070 msgr2=0x7f68941f4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f6899752640 1 --2- 172.21.15.16:0/210664739 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68940d0070 0x7f68941f4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f6899752640 1 -- 172.21.15.16:0/210664739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f689415f0d0 msgr2=0x7f68941e8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f6899752640 1 --2- 172.21.15.16:0/210664739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f689415f0d0 0x7f68941e8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f6899752640 1 -- 172.21.15.16:0/210664739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68940cb940 con 0x7f68940d4130 2024-09-19T07:48:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f6899752640 1 --2- 172.21.15.16:0/210664739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68940d4130 0x7f68940cde40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6884000c00 tx=0x7f68840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f688a7fc640 1 -- 172.21.15.16:0/210664739 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884041020 con 0x7f68940d4130 2024-09-19T07:48:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f688a7fc640 1 -- 172.21.15.16:0/210664739 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68840306c0 con 0x7f68940d4130 2024-09-19T07:48:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.769+0000 7f689c1b7640 1 -- 172.21.15.16:0/210664739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68940cd360 con 0x7f68940d4130 2024-09-19T07:48:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.770+0000 7f688a7fc640 1 -- 172.21.15.16:0/210664739 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884037040 con 0x7f68940d4130 2024-09-19T07:48:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.770+0000 7f689c1b7640 1 -- 172.21.15.16:0/210664739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68941e95e0 con 0x7f68940d4130 2024-09-19T07:48:59.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.770+0000 7f689c1b7640 1 -- 172.21.15.16:0/210664739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68940d4560 con 0x7f68940d4130 2024-09-19T07:48:59.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.771+0000 7f688a7fc640 1 -- 172.21.15.16:0/210664739 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6884044020 con 0x7f68940d4130 2024-09-19T07:48:59.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.772+0000 7f688a7fc640 1 --2- 172.21.15.16:0/210664739 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f687806c2e0 0x7f687806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.772+0000 7f688a7fc640 1 -- 172.21.15.16:0/210664739 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f68840abc60 con 0x7f68940d4130 2024-09-19T07:48:59.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.772+0000 7f6899f53640 1 --2- 172.21.15.16:0/210664739 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f687806c2e0 0x7f687806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.772+0000 7f6899f53640 1 --2- 172.21.15.16:0/210664739 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f687806c2e0 0x7f687806e7a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6890000f60 tx=0x7f6890009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:59.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.776+0000 7f688a7fc640 1 -- 172.21.15.16:0/210664739 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f688404b030 con 0x7f68940d4130 2024-09-19T07:48:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.985+0000 7f689c1b7640 1 -- 172.21.15.16:0/210664739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f689406cf30 con 0x7f68940d4130 2024-09-19T07:48:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.985+0000 7f688a7fc640 1 -- 172.21.15.16:0/210664739 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) ==== 53+0+73 (secure 0 0 0) 0x7f6884079bf0 con 0x7f68940d4130 2024-09-19T07:48:59.986 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:48:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.988+0000 7f689c1b7640 1 -- 172.21.15.16:0/210664739 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f687806c2e0 msgr2=0x7f687806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.988+0000 7f689c1b7640 1 --2- 172.21.15.16:0/210664739 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f687806c2e0 0x7f687806e7a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6890000f60 tx=0x7f6890009040 comp rx=0 tx=0).stop 2024-09-19T07:48:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.988+0000 7f689c1b7640 1 -- 172.21.15.16:0/210664739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68940d4130 msgr2=0x7f68940cde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.988+0000 7f689c1b7640 1 --2- 172.21.15.16:0/210664739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68940d4130 0x7f68940cde40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6884000c00 tx=0x7f68840046c0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.988+0000 7f689c1b7640 1 -- 172.21.15.16:0/210664739 shutdown_connections 2024-09-19T07:48:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.988+0000 7f689c1b7640 1 --2- 172.21.15.16:0/210664739 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f687806c2e0 0x7f687806e7a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.988+0000 7f689c1b7640 1 --2- 172.21.15.16:0/210664739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f689415f0d0 0x7f68941e8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.988+0000 7f689c1b7640 1 --2- 172.21.15.16:0/210664739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68940d4130 0x7f68940cde40 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.988+0000 7f689c1b7640 1 --2- 172.21.15.16:0/210664739 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68940d0070 0x7f68941f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.988+0000 7f689c1b7640 1 -- 172.21.15.16:0/210664739 >> 172.21.15.16:0/210664739 conn(0x7f689409d760 msgr2=0x7f689415dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.988+0000 7f689c1b7640 1 -- 172.21.15.16:0/210664739 shutdown_connections 2024-09-19T07:48:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:48:59.988+0000 7f689c1b7640 1 -- 172.21.15.16:0/210664739 wait complete. 2024-09-19T07:48:59.999 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:49:00.000 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:49:00.000 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T07:49:00.000 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T07:49:00.000 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:00.001 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:49:00.001 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:49:00.001 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:49:00.001 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:49:00.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:00.001 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T07:49:00.019 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:49:00.019 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:49:00.019 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:49:00.019 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:00.019 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:49:00.019 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:00.019 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:49:00.019 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:00.019 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T07:49:00.019 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:49:00.019 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:00.019 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T07:49:00.020 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:49:00.020 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:00.020 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T07:49:00.020 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T07:49:00.020 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42449sec preferred_lft 42449sec 2024-09-19T07:49:00.020 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T07:49:00.020 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:00.020 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:49:00.020 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:00.020 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T07:49:00.020 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:49:00.020 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:00.021 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:49:00.021 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:00.021 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::32:7bff:fe92:8895/64 scope link 2024-09-19T07:49:00.021 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:00.021 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:49:00.021 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether e6:fd:21:6a:43:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:49:00.021 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e4fd:21ff:fe6a:4369/64 scope link 2024-09-19T07:49:00.021 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:00.021 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:49:00.021 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:49:00.021 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::80f4:81ff:fe7d:b7f0/64 scope link 2024-09-19T07:49:00.021 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:00.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:00.022 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:49:00.083 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:49:00.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:00.085 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T07:49:00.142 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T07:49:00.142 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:49:00.142 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T07:49:00.142 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T07:49:00.142 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:49:00.142 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:49:00.143 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:49:00.143 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T07:49:00.143 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T07:49:00.143 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T07:49:00.143 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:49:00.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:49:00.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:00.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:00.271 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:49:00.287 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T07:49:00.287 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:49:00.287 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:49:00.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:00.287 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-19T07:49:00.353 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T07:49:00.353 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:49:00.353 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:00.409 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:49:00.409 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:49:00.409 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:00.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:49:00.531 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:49:00.588 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:49:00.588 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:49:00.588 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:49:00.588 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:49:00.589 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:49:00.589 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:49:00.589 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:49:00.589 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:49:00.589 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:49:00.589 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:49:00.589 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:49:00.589 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:49:00.589 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:49:00.589 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:49:00.589 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:49:00.590 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:49:00.590 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:49:00.590 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:49:00.590 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:49:00.590 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:49:00.590 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:49:00.590 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:49:00.590 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:49:00.590 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:49:00.591 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:49:00.591 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:49:00.591 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:49:00.591 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:49:00.591 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:49:00.591 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:49:00.591 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:49:00.591 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:49:00.591 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:49:00.591 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:49:00.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:00.592 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:49:00.644 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:49:00.644 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T07:49:00.687 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:49:00.712 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:49:00.770 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:49:00.770 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:49:00.770 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:49:00.770 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:49:00.771 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:49:00.771 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:49:00.771 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:49:00.771 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:49:00.771 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:49:00.771 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:49:00.771 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:49:00.771 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:49:00.771 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:49:00.771 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:49:00.771 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:49:00.771 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:49:00.771 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:49:00.772 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:49:00.772 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:49:00.772 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:49:00.772 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:49:00.772 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:49:00.772 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:49:00.772 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:49:00.772 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:49:00.772 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:49:00.772 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:49:00.772 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:49:00.772 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:49:00.772 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:49:00.773 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:49:00.773 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:49:00.773 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:49:00.773 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:49:00.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:00.773 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:49:00.774 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-19T07:49:00.770+0000 7f5060f22480 -1 init, newargv = 0x564bc9a74b90 newargc=15 2024-09-19T07:49:00.774 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[55377]: starting ceph client 2024-09-19T07:49:00.776 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[55377]: starting fuse 2024-09-19T07:49:00.791 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:49:00.792 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-19T07:49:00.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:49:00.792 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:49:00.792 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:49:00.792 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:49:00.792 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:49:00.792 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:00.792 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:49:00.792 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:49:00.792 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:49:00.792 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:49:00.793 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:00.793 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:49:00.793 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:49:00.793 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:49:00.793 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:49:00.793 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:00.793 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:49:00.793 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:49:00.793 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:49:00.793 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:49:00.793 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:49:00.793 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:49:00.793 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:49:00.793 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:49:00.793 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:00.794 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:49:00.794 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:49:00.912 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.55377.asok 2024-09-19T07:49:00.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:00.923 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.55377.asok 2024-09-19T07:49:00.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:00.924 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.55377.asok status 2024-09-19T07:49:01.059 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:49:01.060 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:49:01.060 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:49:01.060 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:49:01.060 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:49:01.060 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:49:01.060 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:49:01.060 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "55377", 2024-09-19T07:49:01.060 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:49:01.060 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:01.060 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-19T07:49:01.060 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-19T07:49:01.061 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5913, 2024-09-19T07:49:01.061 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:49:01.061 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:49:01.061 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:49:01.061 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5913 2024-09-19T07:49:01.061 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:01.061 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:49:01.061 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:01.061 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:01.061 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2972178987 2024-09-19T07:49:01.061 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:01.061 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:01.062 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:49:01.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:01.062 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:01.062 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2972178987 2024-09-19T07:49:01.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:01.062 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.5913 192.168.0.1:0/2972178987", 2024-09-19T07:49:01.062 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/2972178987", 2024-09-19T07:49:01.062 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-19T07:49:01.062 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 33, 2024-09-19T07:49:01.062 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 70, 2024-09-19T07:49:01.062 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:49:01.062 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:49:01.062 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:49:01.062 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:49:01.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:01.068 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-19T07:49:01.068 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:01.084 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-19T07:49:01.085 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:01.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:01.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:01.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:49:01.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.295+0000 7f5034949640 1 Processor -- start 2024-09-19T07:49:01.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.295+0000 7f5034949640 1 -- start start 2024-09-19T07:49:01.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.295+0000 7f5034949640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f503006f9e0 0x7f50300d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.295+0000 7f5034949640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f50300d09f0 0x7f50300c5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.296+0000 7f5034949640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50300d0e00 con 0x7f503015b120 2024-09-19T07:49:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.296+0000 7f5034949640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f503015b500 con 0x7f50300d09f0 2024-09-19T07:49:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.296+0000 7f5034949640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f503006e940 con 0x7f503006f9e0 2024-09-19T07:49:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.296+0000 7f502ed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f50300d09f0 0x7f50300c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.296+0000 7f502ed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f50300d09f0 0x7f50300c5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59502/0 (socket says 172.21.15.16:59502) 2024-09-19T07:49:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.296+0000 7f502ed9d640 1 -- 172.21.15.16:0/3740671469 learned_addr learned my addr 172.21.15.16:0/3740671469 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.296+0000 7f502dd9b640 1 --2- 172.21.15.16:0/3740671469 >> v2:172.21.15.146:3300/0 conn(0x7f503006f9e0 0x7f50300d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.296+0000 7f502ed9d640 1 -- 172.21.15.16:0/3740671469 >> v2:172.21.15.146:3300/0 conn(0x7f503006f9e0 msgr2=0x7f50300d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.296+0000 7f502ed9d640 1 --2- 172.21.15.16:0/3740671469 >> v2:172.21.15.146:3300/0 conn(0x7f503006f9e0 0x7f50300d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.296+0000 7f502ed9d640 1 -- 172.21.15.16:0/3740671469 >> v1:172.21.15.146:6789/0 conn(0x7f503015b120 legacy=0x7f503006dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.296+0000 7f502ed9d640 1 -- 172.21.15.16:0/3740671469 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503006ec20 con 0x7f50300d09f0 2024-09-19T07:49:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.296+0000 7f502dd9b640 1 --2- 172.21.15.16:0/3740671469 >> v2:172.21.15.146:3300/0 conn(0x7f503006f9e0 0x7f50300d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:01.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.297+0000 7f502ed9d640 1 --2- 172.21.15.16:0/3740671469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50300d09f0 0x7f50300c5160 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f503006dcd0 tx=0x7f502802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29dde84ca83da63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.297+0000 7f502d59a640 1 -- 172.21.15.16:0/3740671469 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028036070 con 0x7f50300d09f0 2024-09-19T07:49:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.297+0000 7f502d59a640 1 -- 172.21.15.16:0/3740671469 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5028031600 con 0x7f50300d09f0 2024-09-19T07:49:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.297+0000 7f502d59a640 1 -- 172.21.15.16:0/3740671469 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50280319d0 con 0x7f50300d09f0 2024-09-19T07:49:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.297+0000 7f5034949640 1 -- 172.21.15.16:0/3740671469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50300d09f0 msgr2=0x7f50300c5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.297+0000 7f5034949640 1 --2- 172.21.15.16:0/3740671469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50300d09f0 0x7f50300c5160 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f503006dcd0 tx=0x7f502802a6f0 comp rx=0 tx=0).stop 2024-09-19T07:49:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.298+0000 7f5034949640 1 -- 172.21.15.16:0/3740671469 shutdown_connections 2024-09-19T07:49:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.298+0000 7f5034949640 1 --2- 172.21.15.16:0/3740671469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50300d09f0 0x7f50300c5160 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.298+0000 7f5034949640 1 --2- 172.21.15.16:0/3740671469 >> v2:172.21.15.146:3300/0 conn(0x7f503006f9e0 0x7f50300d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.298+0000 7f5034949640 1 -- 172.21.15.16:0/3740671469 >> 172.21.15.16:0/3740671469 conn(0x7f503006bdf0 msgr2=0x7f503006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.298+0000 7f5034949640 1 -- 172.21.15.16:0/3740671469 shutdown_connections 2024-09-19T07:49:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.298+0000 7f5034949640 1 -- 172.21.15.16:0/3740671469 wait complete. 2024-09-19T07:49:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.299+0000 7f5034949640 1 Processor -- start 2024-09-19T07:49:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.299+0000 7f5034949640 1 -- start start 2024-09-19T07:49:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f5034949640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f503006f9e0 0x7f50300cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f5034949640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50300d09f0 0x7f50300c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f5034949640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f503015b120 0x7f50300994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f5034949640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f503006f4c0 con 0x7f503006f9e0 2024-09-19T07:49:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f5034949640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f503006f1e0 con 0x7f50300d09f0 2024-09-19T07:49:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f5034949640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f503006e5d0 con 0x7f503015b120 2024-09-19T07:49:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f502e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f503006f9e0 0x7f50300cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f502e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f503006f9e0 0x7f50300cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59512/0 (socket says 172.21.15.16:59512) 2024-09-19T07:49:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f502e59c640 1 -- 172.21.15.16:0/878422545 learned_addr learned my addr 172.21.15.16:0/878422545 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f502dd9b640 1 --2- 172.21.15.16:0/878422545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50300d09f0 0x7f50300c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f502ed9d640 1 --2- 172.21.15.16:0/878422545 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f503015b120 0x7f50300994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f502e59c640 1 -- 172.21.15.16:0/878422545 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f503015b120 msgr2=0x7f50300994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:01.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f502e59c640 1 --2- 172.21.15.16:0/878422545 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f503015b120 0x7f50300994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:01.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f502e59c640 1 -- 172.21.15.16:0/878422545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50300d09f0 msgr2=0x7f50300c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:01.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f502e59c640 1 --2- 172.21.15.16:0/878422545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50300d09f0 0x7f50300c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:01.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f502e59c640 1 -- 172.21.15.16:0/878422545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503014a430 con 0x7f503006f9e0 2024-09-19T07:49:01.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f502e59c640 1 --2- 172.21.15.16:0/878422545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f503006f9e0 0x7f50300cd4f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f502000b420 tx=0x7f502000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:01.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f501effd640 1 -- 172.21.15.16:0/878422545 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5020011830 con 0x7f503006f9e0 2024-09-19T07:49:01.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f501effd640 1 -- 172.21.15.16:0/878422545 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5020016070 con 0x7f503006f9e0 2024-09-19T07:49:01.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f5034949640 1 -- 172.21.15.16:0/878422545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f503014a710 con 0x7f503006f9e0 2024-09-19T07:49:01.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f501effd640 1 -- 172.21.15.16:0/878422545 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5020011c10 con 0x7f503006f9e0 2024-09-19T07:49:01.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.300+0000 7f5034949640 1 -- 172.21.15.16:0/878422545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50300c5e40 con 0x7f503006f9e0 2024-09-19T07:49:01.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.302+0000 7f5034949640 1 -- 172.21.15.16:0/878422545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50300ce060 con 0x7f503006f9e0 2024-09-19T07:49:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.302+0000 7f501effd640 1 -- 172.21.15.16:0/878422545 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f502001b460 con 0x7f503006f9e0 2024-09-19T07:49:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.302+0000 7f501effd640 1 --2- 172.21.15.16:0/878422545 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f501006c2b0 0x7f501006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.303+0000 7f501effd640 1 -- 172.21.15.16:0/878422545 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f502008b260 con 0x7f503006f9e0 2024-09-19T07:49:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.303+0000 7f502dd9b640 1 --2- 172.21.15.16:0/878422545 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f501006c2b0 0x7f501006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.303+0000 7f502dd9b640 1 --2- 172.21.15.16:0/878422545 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f501006c2b0 0x7f501006e770 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f50301f5570 tx=0x7f5024008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.307+0000 7f501effd640 1 -- 172.21.15.16:0/878422545 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f50200598d0 con 0x7f503006f9e0 2024-09-19T07:49:01.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.517+0000 7f5034949640 1 -- 172.21.15.16:0/878422545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f503014a8e0 con 0x7f503006f9e0 2024-09-19T07:49:01.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.518+0000 7f501effd640 1 -- 172.21.15.16:0/878422545 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) ==== 53+0+73 (secure 0 0 0) 0x7f50200591d0 con 0x7f503006f9e0 2024-09-19T07:49:01.518 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:49:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.520+0000 7f5034949640 1 -- 172.21.15.16:0/878422545 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f501006c2b0 msgr2=0x7f501006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.520+0000 7f5034949640 1 --2- 172.21.15.16:0/878422545 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f501006c2b0 0x7f501006e770 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f50301f5570 tx=0x7f5024008040 comp rx=0 tx=0).stop 2024-09-19T07:49:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.520+0000 7f5034949640 1 -- 172.21.15.16:0/878422545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f503006f9e0 msgr2=0x7f50300cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.520+0000 7f5034949640 1 --2- 172.21.15.16:0/878422545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f503006f9e0 0x7f50300cd4f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f502000b420 tx=0x7f502000b8e0 comp rx=0 tx=0).stop 2024-09-19T07:49:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.520+0000 7f5034949640 1 -- 172.21.15.16:0/878422545 shutdown_connections 2024-09-19T07:49:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.520+0000 7f5034949640 1 --2- 172.21.15.16:0/878422545 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f501006c2b0 0x7f501006e770 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:01.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.520+0000 7f5034949640 1 --2- 172.21.15.16:0/878422545 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f503015b120 0x7f50300994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:01.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.520+0000 7f5034949640 1 --2- 172.21.15.16:0/878422545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50300d09f0 0x7f50300c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:01.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.520+0000 7f5034949640 1 --2- 172.21.15.16:0/878422545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f503006f9e0 0x7f50300cd4f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:01.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.520+0000 7f5034949640 1 -- 172.21.15.16:0/878422545 >> 172.21.15.16:0/878422545 conn(0x7f503006bdf0 msgr2=0x7f5030159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:01.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.520+0000 7f5034949640 1 -- 172.21.15.16:0/878422545 shutdown_connections 2024-09-19T07:49:01.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:01.520+0000 7f5034949640 1 -- 172.21.15.16:0/878422545 wait complete. 2024-09-19T07:49:01.531 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:49:01.531 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:49:01.531 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-09-19T07:49:01.531 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T07:49:01.531 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T07:49:01.532 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:49:01.532 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:49:01.532 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:49:01.532 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:49:01.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:01.532 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-09-19T07:49:01.552 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:49:01.552 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:49:01.552 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:49:01.552 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:01.552 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-09-19T07:49:01.552 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:01.553 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:49:01.553 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:01.553 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2024-09-19T07:49:01.553 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:49:01.553 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:01.553 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2024-09-19T07:49:01.553 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:49:01.553 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:01.553 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:49:01.553 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:01.553 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T07:49:01.554 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42415sec preferred_lft 42415sec 2024-09-19T07:49:01.554 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link noprefixroute 2024-09-19T07:49:01.554 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:01.554 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:49:01.554 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:01.554 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:49:01.554 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:01.554 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::e47f:32ff:fe68:31ca/64 scope link 2024-09-19T07:49:01.555 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:01.555 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:49:01.555 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether f6:1f:22:44:9d:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:49:01.555 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::f41f:22ff:fe44:9dcd/64 scope link 2024-09-19T07:49:01.555 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:01.555 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:49:01.555 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:49:01.555 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::8459:eeff:fe21:a113/64 scope link 2024-09-19T07:49:01.555 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:01.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:01.555 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:49:01.621 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-09-19T07:49:01.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:01.623 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-09-19T07:49:01.682 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-09-19T07:49:01.682 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:49:01.682 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T07:49:01.682 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T07:49:01.682 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:49:01.682 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:49:01.682 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-19T07:49:01.683 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T07:49:01.683 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T07:49:01.683 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T07:49:01.683 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-19T07:49:01.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:49:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:49:01.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:49:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:01.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:01.815 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-09-19T07:49:01.832 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T07:49:01.832 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T07:49:01.832 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:49:01.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:01.832 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2024-09-19T07:49:01.897 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T07:49:01.897 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:49:01.897 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T07:49:01.953 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:49:01.953 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:49:01.953 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:49:02.008 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:49:02.074 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:49:02.132 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:49:02.132 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:49:02.132 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:49:02.132 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:49:02.132 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:49:02.132 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:49:02.132 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:49:02.132 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:49:02.132 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:49:02.132 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:49:02.132 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:49:02.133 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:49:02.133 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:49:02.133 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:49:02.133 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:49:02.133 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:49:02.133 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:49:02.133 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:49:02.133 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:49:02.133 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:49:02.134 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:49:02.134 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:49:02.134 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:49:02.134 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:49:02.134 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:49:02.134 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:49:02.134 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:49:02.134 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:49:02.134 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:49:02.134 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:49:02.134 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:49:02.134 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:49:02.135 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:49:02.135 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:49:02.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:02.135 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:49:02.188 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:49:02.188 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T07:49:02.231 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:49:02.257 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:49:02.311 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2024-09-19T07:49:02.311+0000 7fdff2e8d480 -1 init, newargv = 0x55aebf88fb90 newargc=15 2024-09-19T07:49:02.312 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[45545]: starting ceph client 2024-09-19T07:49:02.314 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:49:02.314 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:49:02.314 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:49:02.314 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:49:02.314 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:49:02.315 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:49:02.315 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:49:02.315 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:49:02.315 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:49:02.315 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:49:02.315 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:49:02.316 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:49:02.316 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:49:02.316 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:49:02.316 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:49:02.316 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:49:02.316 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:49:02.317 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:49:02.317 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:49:02.317 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:49:02.317 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:49:02.317 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:49:02.317 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:49:02.318 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:49:02.318 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:49:02.318 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:49:02.318 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:49:02.318 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:49:02.318 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:49:02.319 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:49:02.319 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:49:02.319 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:49:02.319 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:49:02.319 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:49:02.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:02.320 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:49:02.321 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[45545]: starting fuse 2024-09-19T07:49:02.336 INFO:teuthology.orchestra.run.smithi146.stdout:42 2024-09-19T07:49:02.336 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-19T07:49:02.337 DEBUG:teuthology.orchestra.run.smithi146:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:49:02.337 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2024-09-19T07:49:02.337 DEBUG:teuthology.orchestra.run.smithi146:> import re 2024-09-19T07:49:02.337 DEBUG:teuthology.orchestra.run.smithi146:> import os 2024-09-19T07:49:02.337 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2024-09-19T07:49:02.337 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:49:02.337 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2024-09-19T07:49:02.337 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T07:49:02.338 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2024-09-19T07:49:02.338 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T07:49:02.338 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:49:02.338 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2024-09-19T07:49:02.338 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2024-09-19T07:49:02.338 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2024-09-19T07:49:02.338 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2024-09-19T07:49:02.338 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:49:02.338 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2024-09-19T07:49:02.338 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:49:02.339 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:49:02.339 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:49:02.339 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2024-09-19T07:49:02.339 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2024-09-19T07:49:02.339 DEBUG:teuthology.orchestra.run.smithi146:> return f 2024-09-19T07:49:02.339 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:49:02.339 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:49:02.339 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2024-09-19T07:49:02.339 DEBUG:teuthology.orchestra.run.smithi146:> ' 2024-09-19T07:49:02.454 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.45545.asok 2024-09-19T07:49:02.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:49:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:02.464 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.45545.asok 2024-09-19T07:49:02.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:02.464 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.45545.asok status 2024-09-19T07:49:02.598 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-19T07:49:02.598 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2024-09-19T07:49:02.598 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:49:02.598 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:49:02.598 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2024-09-19T07:49:02.598 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2024-09-19T07:49:02.598 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T07:49:02.598 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "45545", 2024-09-19T07:49:02.598 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2024-09-19T07:49:02.599 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:49:02.599 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2024-09-19T07:49:02.599 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2024-09-19T07:49:02.599 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 5659, 2024-09-19T07:49:02.599 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2024-09-19T07:49:02.599 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2024-09-19T07:49:02.599 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2024-09-19T07:49:02.599 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 5659 2024-09-19T07:49:02.599 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:49:02.599 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:49:02.599 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:49:02.600 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:02.600 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 1321164363 2024-09-19T07:49:02.600 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-19T07:49:02.600 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:49:02.600 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:49:02.600 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:49:02.600 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:02.600 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 1321164363 2024-09-19T07:49:02.600 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:49:02.600 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.5659 192.168.0.1:0/1321164363", 2024-09-19T07:49:02.600 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/1321164363", 2024-09-19T07:49:02.600 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2024-09-19T07:49:02.601 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 33, 2024-09-19T07:49:02.601 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 70, 2024-09-19T07:49:02.601 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:49:02.601 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2024-09-19T07:49:02.601 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2024-09-19T07:49:02.601 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-19T07:49:02.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:02.606 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2024-09-19T07:49:02.606 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T07:49:02.622 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2024-09-19T07:49:02.622 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T07:49:02.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:02.622 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:49:02.688 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-19T07:49:02.689 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:49:02.689 DEBUG:teuthology.orchestra.run.smithi016:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir") 2024-09-19T07:49:02.689 DEBUG:teuthology.orchestra.run.smithi016:> for n in range(0, 1000): 2024-09-19T07:49:02.689 DEBUG:teuthology.orchestra.run.smithi016:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir/dir{0}".format(n)) 2024-09-19T07:49:02.689 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:49:03.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:03.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:03.857 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync) 2024-09-19T07:49:04.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:49:04.556 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:49:04.556 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:49:04.556 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:49:04.556 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:49:04.556 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:04.557 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:49:04.557 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:49:04.557 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:49:04.557 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:49:04.557 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:04.557 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:49:04.557 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:49:04.557 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:49:04.557 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:49:04.557 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:04.557 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:49:04.557 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:49:04.557 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:49:04.557 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:49:04.557 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:49:04.558 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:49:04.558 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:49:04.558 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:49:04.558 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:04.558 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:49:04.558 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:49:04.650 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.55377.asok 2024-09-19T07:49:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:04.662 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.55377.asok 2024-09-19T07:49:04.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:04.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.55377.asok status 2024-09-19T07:49:04.802 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:49:04.802 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:49:04.802 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:49:04.802 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:49:04.802 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:49:04.802 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:49:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:49:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "55377", 2024-09-19T07:49:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:49:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 1001, 2024-09-19T07:49:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 1001, 2024-09-19T07:49:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5913, 2024-09-19T07:49:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:49:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:49:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:49:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5913 2024-09-19T07:49:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:49:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2972178987 2024-09-19T07:49:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:49:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2972178987 2024-09-19T07:49:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.5913 192.168.0.1:0/2972178987", 2024-09-19T07:49:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/2972178987", 2024-09-19T07:49:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1002, 2024-09-19T07:49:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 33, 2024-09-19T07:49:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 70, 2024-09-19T07:49:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 70, 2024-09-19T07:49:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:49:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:49:04.805 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:49:04.811 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:49:04.811 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:49:04.811 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:49:04.811 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:49:04.811 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:49:04.811 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:04.811 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:49:04.811 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:49:04.811 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:49:04.811 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:49:04.811 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:04.811 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:49:04.811 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:49:04.812 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:49:04.812 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:49:04.812 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:04.812 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:49:04.812 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:49:04.812 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:49:04.812 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:49:04.812 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:49:04.812 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:49:04.812 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:49:04.812 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:49:04.812 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:04.812 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:49:04.812 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:49:04.892 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.55377.asok 2024-09-19T07:49:04.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:04.903 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.55377.asok 2024-09-19T07:49:04.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:04.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.55377.asok config set client_cache_size 100 2024-09-19T07:49:05.039 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:49:05.039 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "client_cache_size = '' " 2024-09-19T07:49:05.039 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:49:05.047 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:49:05.047 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:49:05.048 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:49:05.048 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:49:05.048 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:49:05.048 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:05.048 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:49:05.048 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:49:05.048 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:49:05.048 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:49:05.048 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:05.048 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:49:05.048 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:49:05.048 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:49:05.048 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:49:05.048 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:05.049 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:49:05.049 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:49:05.049 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:49:05.049 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:49:05.049 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:49:05.049 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:49:05.049 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:49:05.049 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:49:05.049 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:05.049 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:49:05.049 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:49:05.129 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.55377.asok 2024-09-19T07:49:05.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:05.139 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.55377.asok 2024-09-19T07:49:05.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:05.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.55377.asok status 2024-09-19T07:49:05.276 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:49:05.276 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:49:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:49:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:49:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:49:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:49:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:49:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "55377", 2024-09-19T07:49:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:49:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 1001, 2024-09-19T07:49:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 1001, 2024-09-19T07:49:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5913, 2024-09-19T07:49:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:49:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:49:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:49:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5913 2024-09-19T07:49:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:49:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2972178987 2024-09-19T07:49:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:49:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2972178987 2024-09-19T07:49:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.5913 192.168.0.1:0/2972178987", 2024-09-19T07:49:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/2972178987", 2024-09-19T07:49:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1002, 2024-09-19T07:49:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 33, 2024-09-19T07:49:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 70, 2024-09-19T07:49:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 70, 2024-09-19T07:49:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:49:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:49:05.279 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:49:05.285 INFO:tasks.cephfs.test_client_limits:waiting, dentry_count, dentry_pinned_count: 1001, 1001 2024-09-19T07:49:05.285 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-19T07:49:08.126 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:49:08.129 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:49:10.286 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:49:10.286 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:49:10.286 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:49:10.287 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:49:10.287 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:49:10.287 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:10.287 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:49:10.287 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:49:10.287 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:49:10.287 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:49:10.287 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:10.287 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:49:10.287 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:49:10.287 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:49:10.287 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:49:10.287 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:10.287 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:49:10.288 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:49:10.288 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:49:10.288 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:49:10.288 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:49:10.288 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:49:10.288 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:49:10.288 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:49:10.288 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:10.288 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:49:10.288 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:49:10.381 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.55377.asok 2024-09-19T07:49:10.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:10.393 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.55377.asok 2024-09-19T07:49:10.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:10.394 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.55377.asok status 2024-09-19T07:49:10.535 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:49:10.535 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:49:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:49:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:49:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:49:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:49:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:49:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "55377", 2024-09-19T07:49:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:49:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 100, 2024-09-19T07:49:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 1, 2024-09-19T07:49:10.536 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5913, 2024-09-19T07:49:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:49:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:49:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:49:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5913 2024-09-19T07:49:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:49:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2972178987 2024-09-19T07:49:10.537 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:49:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2972178987 2024-09-19T07:49:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.5913 192.168.0.1:0/2972178987", 2024-09-19T07:49:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/2972178987", 2024-09-19T07:49:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 101, 2024-09-19T07:49:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 33, 2024-09-19T07:49:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 70, 2024-09-19T07:49:10.538 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 70, 2024-09-19T07:49:10.539 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:49:10.539 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:49:10.539 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:49:10.544 INFO:tasks.cephfs.test_client_limits:waiting, dentry_count, dentry_pinned_count: 100, 1 2024-09-19T07:49:10.544 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-19T07:49:10.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:10.545 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2024-09-19T07:49:10.564 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:49:10.564 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:49:10.621 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:49:10.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.764+0000 7f4758053640 1 Processor -- start 2024-09-19T07:49:10.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.764+0000 7f4758053640 1 -- start start 2024-09-19T07:49:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.765+0000 7f4758053640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4750141680 0x7f47500cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.765+0000 7f4758053640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f47500d0370 0x7f4750141a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.765+0000 7f4758053640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4750141fa0 0x7f47500cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.765+0000 7f4758053640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f47500d07b0 con 0x7f47500d0370 2024-09-19T07:49:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.765+0000 7f4758053640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f475015c6f0 con 0x7f4750141fa0 2024-09-19T07:49:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.765+0000 7f4758053640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f475015d710 con 0x7f4750141680 2024-09-19T07:49:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.765+0000 7f47555ee640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f47500d0370 0x7f4750141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.765+0000 7f47555ee640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f47500d0370 0x7f4750141a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33784/0 (socket says 172.21.15.16:33784) 2024-09-19T07:49:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.765+0000 7f47555ee640 1 -- 172.21.15.16:0/3188347747 learned_addr learned my addr 172.21.15.16:0/3188347747 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.765+0000 7f47565f0640 1 --2- 172.21.15.16:0/3188347747 >> v2:172.21.15.146:3300/0 conn(0x7f4750141fa0 0x7f47500cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.765+0000 7f4755def640 1 --2- 172.21.15.16:0/3188347747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4750141680 0x7f47500cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.765+0000 7f47555ee640 1 -- 172.21.15.16:0/3188347747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4750141680 msgr2=0x7f47500cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:10.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.765+0000 7f47555ee640 1 --2- 172.21.15.16:0/3188347747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4750141680 0x7f47500cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:10.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.765+0000 7f47555ee640 1 -- 172.21.15.16:0/3188347747 >> v2:172.21.15.146:3300/0 conn(0x7f4750141fa0 msgr2=0x7f47500cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:10.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.765+0000 7f47555ee640 1 --2- 172.21.15.16:0/3188347747 >> v2:172.21.15.146:3300/0 conn(0x7f4750141fa0 0x7f47500cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:10.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.765+0000 7f47555ee640 1 -- 172.21.15.16:0/3188347747 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475015c950 con 0x7f47500d0370 2024-09-19T07:49:10.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.765+0000 7f4755def640 1 --2- 172.21.15.16:0/3188347747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4750141680 0x7f47500cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:10.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.766+0000 7f47555ee640 1 --2- 172.21.15.16:0/3188347747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47500d0370 0x7f4750141a60 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4750073ce0 tx=0x7f474402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41de2dcd3baa6e6b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:10.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.766+0000 7f4754ded640 1 -- 172.21.15.16:0/3188347747 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744037070 con 0x7f47500d0370 2024-09-19T07:49:10.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.766+0000 7f4754ded640 1 -- 172.21.15.16:0/3188347747 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4744032440 con 0x7f47500d0370 2024-09-19T07:49:10.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.766+0000 7f4754ded640 1 -- 172.21.15.16:0/3188347747 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744032810 con 0x7f47500d0370 2024-09-19T07:49:10.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.767+0000 7f4758053640 1 -- 172.21.15.16:0/3188347747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47500d0370 msgr2=0x7f4750141a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:10.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.767+0000 7f4758053640 1 --2- 172.21.15.16:0/3188347747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47500d0370 0x7f4750141a60 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4750073ce0 tx=0x7f474402a210 comp rx=0 tx=0).stop 2024-09-19T07:49:10.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.767+0000 7f4758053640 1 -- 172.21.15.16:0/3188347747 shutdown_connections 2024-09-19T07:49:10.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.767+0000 7f4758053640 1 --2- 172.21.15.16:0/3188347747 >> v2:172.21.15.146:3300/0 conn(0x7f4750141fa0 0x7f47500cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:10.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.767+0000 7f4758053640 1 --2- 172.21.15.16:0/3188347747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47500d0370 0x7f4750141a60 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:10.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.767+0000 7f4758053640 1 --2- 172.21.15.16:0/3188347747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4750141680 0x7f47500cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:10.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.767+0000 7f4758053640 1 -- 172.21.15.16:0/3188347747 >> 172.21.15.16:0/3188347747 conn(0x7f47500b3ed0 msgr2=0x7f47500b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:10.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.767+0000 7f4758053640 1 -- 172.21.15.16:0/3188347747 shutdown_connections 2024-09-19T07:49:10.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.767+0000 7f4758053640 1 -- 172.21.15.16:0/3188347747 wait complete. 2024-09-19T07:49:10.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.768+0000 7f4758053640 1 Processor -- start 2024-09-19T07:49:10.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.768+0000 7f4758053640 1 -- start start 2024-09-19T07:49:10.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.769+0000 7f4758053640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47500d0370 0x7f475013e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:10.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.769+0000 7f4758053640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4750141680 0x7f47501e57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:10.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.769+0000 7f4758053640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4750141fa0 0x7f47501e5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:10.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.769+0000 7f4758053640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f475015d1f0 con 0x7f47500d0370 2024-09-19T07:49:10.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.769+0000 7f4758053640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f475015cf10 con 0x7f4750141680 2024-09-19T07:49:10.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.769+0000 7f4758053640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4750142430 con 0x7f4750141fa0 2024-09-19T07:49:10.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.769+0000 7f4755def640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47500d0370 0x7f475013e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:10.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.769+0000 7f4755def640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47500d0370 0x7f475013e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33796/0 (socket says 172.21.15.16:33796) 2024-09-19T07:49:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.769+0000 7f4755def640 1 -- 172.21.15.16:0/2331501670 learned_addr learned my addr 172.21.15.16:0/2331501670 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.769+0000 7f47555ee640 1 --2- 172.21.15.16:0/2331501670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4750141680 0x7f47501e57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.769+0000 7f4755def640 1 -- 172.21.15.16:0/2331501670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4750141fa0 msgr2=0x7f47501e5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.769+0000 7f47565f0640 1 --2- 172.21.15.16:0/2331501670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4750141fa0 0x7f47501e5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.769+0000 7f4755def640 1 --2- 172.21.15.16:0/2331501670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4750141fa0 0x7f47501e5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.769+0000 7f4755def640 1 -- 172.21.15.16:0/2331501670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4750141680 msgr2=0x7f47501e57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.769+0000 7f4755def640 1 --2- 172.21.15.16:0/2331501670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4750141680 0x7f47501e57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.769+0000 7f4755def640 1 -- 172.21.15.16:0/2331501670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47501e3c80 con 0x7f47500d0370 2024-09-19T07:49:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.770+0000 7f47565f0640 1 --2- 172.21.15.16:0/2331501670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4750141fa0 0x7f47501e5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.770+0000 7f47555ee640 1 --2- 172.21.15.16:0/2331501670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4750141680 0x7f47501e57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.770+0000 7f4755def640 1 --2- 172.21.15.16:0/2331501670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47500d0370 0x7f475013e5d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f474000e810 tx=0x7f474000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.770+0000 7f474e7fc640 1 -- 172.21.15.16:0/2331501670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474000f040 con 0x7f47500d0370 2024-09-19T07:49:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.770+0000 7f474e7fc640 1 -- 172.21.15.16:0/2331501670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4740004030 con 0x7f47500d0370 2024-09-19T07:49:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.770+0000 7f4758053640 1 -- 172.21.15.16:0/2331501670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47501e3f60 con 0x7f47500d0370 2024-09-19T07:49:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.770+0000 7f474e7fc640 1 -- 172.21.15.16:0/2331501670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740004400 con 0x7f47500d0370 2024-09-19T07:49:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.770+0000 7f4758053640 1 -- 172.21.15.16:0/2331501670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47501e4520 con 0x7f47500d0370 2024-09-19T07:49:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.771+0000 7f4758053640 1 -- 172.21.15.16:0/2331501670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4750141a60 con 0x7f47500d0370 2024-09-19T07:49:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.772+0000 7f474e7fc640 1 -- 172.21.15.16:0/2331501670 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f474000c6d0 con 0x7f47500d0370 2024-09-19T07:49:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.772+0000 7f474e7fc640 1 --2- 172.21.15.16:0/2331501670 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f473c06c2b0 0x7f473c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.772+0000 7f474e7fc640 1 -- 172.21.15.16:0/2331501670 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f474008b290 con 0x7f47500d0370 2024-09-19T07:49:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.772+0000 7f47555ee640 1 --2- 172.21.15.16:0/2331501670 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f473c06c2b0 0x7f473c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.773+0000 7f47555ee640 1 --2- 172.21.15.16:0/2331501670 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f473c06c2b0 0x7f473c06e770 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4744034000 tx=0x7f474402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.776+0000 7f474e7fc640 1 -- 172.21.15.16:0/2331501670 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4740017020 con 0x7f47500d0370 2024-09-19T07:49:10.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.965+0000 7f4758053640 1 -- 172.21.15.16:0/2331501670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f47500690c0 con 0x7f47500d0370 2024-09-19T07:49:10.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.965+0000 7f474e7fc640 1 -- 172.21.15.16:0/2331501670 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) ==== 80+0+0 (secure 0 0 0) 0x7f4740059870 con 0x7f47500d0370 2024-09-19T07:49:10.965 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:49:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.967+0000 7f4758053640 1 -- 172.21.15.16:0/2331501670 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f473c06c2b0 msgr2=0x7f473c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.967+0000 7f4758053640 1 --2- 172.21.15.16:0/2331501670 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f473c06c2b0 0x7f473c06e770 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4744034000 tx=0x7f474402e000 comp rx=0 tx=0).stop 2024-09-19T07:49:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.967+0000 7f4758053640 1 -- 172.21.15.16:0/2331501670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47500d0370 msgr2=0x7f475013e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.967+0000 7f4758053640 1 --2- 172.21.15.16:0/2331501670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47500d0370 0x7f475013e5d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f474000e810 tx=0x7f474000ecd0 comp rx=0 tx=0).stop 2024-09-19T07:49:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.967+0000 7f4758053640 1 -- 172.21.15.16:0/2331501670 shutdown_connections 2024-09-19T07:49:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.967+0000 7f4758053640 1 --2- 172.21.15.16:0/2331501670 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f473c06c2b0 0x7f473c06e770 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.967+0000 7f4758053640 1 --2- 172.21.15.16:0/2331501670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4750141fa0 0x7f47501e5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.967+0000 7f4758053640 1 --2- 172.21.15.16:0/2331501670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4750141680 0x7f47501e57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.967+0000 7f4758053640 1 --2- 172.21.15.16:0/2331501670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47500d0370 0x7f475013e5d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.967+0000 7f4758053640 1 -- 172.21.15.16:0/2331501670 >> 172.21.15.16:0/2331501670 conn(0x7f47500b3ed0 msgr2=0x7f4750155f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.967+0000 7f4758053640 1 -- 172.21.15.16:0/2331501670 shutdown_connections 2024-09-19T07:49:10.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:10.967+0000 7f4758053640 1 -- 172.21.15.16:0/2331501670 wait complete. 2024-09-19T07:49:10.981 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:49:10.981 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:49:10.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:49:11.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.142+0000 7fdf1c111640 1 Processor -- start 2024-09-19T07:49:11.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.142+0000 7fdf1c111640 1 -- start start 2024-09-19T07:49:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.143+0000 7fdf1c111640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf140d4370 0x7fdf140d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.143+0000 7fdf1c111640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf140d1880 con 0x7fdf1415e680 2024-09-19T07:49:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.143+0000 7fdf1c111640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf140d1b60 con 0x7fdf1415e2a0 2024-09-19T07:49:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.143+0000 7fdf1c111640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdf140cf2d0 con 0x7fdf140d4370 2024-09-19T07:49:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.143+0000 7fdf196ac640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdf1415e680 0x7fdf140cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51818/0 (socket says 172.21.15.16:51818) 2024-09-19T07:49:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.143+0000 7fdf196ac640 1 -- 172.21.15.16:0/488032259 learned_addr learned my addr 172.21.15.16:0/488032259 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.143+0000 7fdf1a6ae640 1 --2- 172.21.15.16:0/488032259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf140d4370 0x7fdf140d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.143+0000 7fdf18eab640 1 -- 172.21.15.16:0/488032259 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 456127760 0 0) 0x7fdf140d1880 con 0x7fdf1415e680 2024-09-19T07:49:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.143+0000 7fdf18eab640 1 -- 172.21.15.16:0/488032259 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf00003520 con 0x7fdf1415e680 2024-09-19T07:49:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.143+0000 7fdf18eab640 1 -- 172.21.15.16:0/488032259 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2426051859 0 0) 0x7fdf140d1b60 con 0x7fdf1415e2a0 2024-09-19T07:49:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.143+0000 7fdf18eab640 1 -- 172.21.15.16:0/488032259 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf140d1880 con 0x7fdf1415e2a0 2024-09-19T07:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.143+0000 7fdf18eab640 1 -- 172.21.15.16:0/488032259 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2233418126 0 0) 0x7fdf00003520 con 0x7fdf1415e680 2024-09-19T07:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.143+0000 7fdf18eab640 1 -- 172.21.15.16:0/488032259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf140d4370 msgr2=0x7fdf140d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.143+0000 7fdf18eab640 1 --2- 172.21.15.16:0/488032259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf140d4370 0x7fdf140d4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.143+0000 7fdf18eab640 1 -- 172.21.15.16:0/488032259 >> v1:172.21.15.146:6789/0 conn(0x7fdf1415e2a0 legacy=0x7fdf1406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.143+0000 7fdf18eab640 1 -- 172.21.15.16:0/488032259 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf140cf5b0 con 0x7fdf1415e680 2024-09-19T07:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.143+0000 7fdf18eab640 1 -- 172.21.15.16:0/488032259 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fdf04001670 con 0x7fdf1415e680 2024-09-19T07:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.143+0000 7fdf18eab640 1 -- 172.21.15.16:0/488032259 >> v1:172.21.15.16:6789/0 conn(0x7fdf1415e680 legacy=0x7fdf140cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.143+0000 7fdf18eab640 1 --2- 172.21.15.16:0/488032259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf00003b70 0x7fdf00003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.144+0000 7fdf18eab640 1 --2- 172.21.15.16:0/488032259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf140d4370 0x7fdf00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.144+0000 7fdf18eab640 1 --2- 172.21.15.16:0/488032259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf00004d40 0x7fdf000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.144+0000 7fdf18eab640 1 -- 172.21.15.16:0/488032259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdf140cf2d0 con 0x7fdf00004d40 2024-09-19T07:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.144+0000 7fdf18eab640 1 -- 172.21.15.16:0/488032259 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0000ba70 con 0x7fdf00003b70 2024-09-19T07:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.144+0000 7fdf18eab640 1 -- 172.21.15.16:0/488032259 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdf0000bc70 con 0x7fdf140d4370 2024-09-19T07:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.144+0000 7fdf1a6ae640 1 --2- 172.21.15.16:0/488032259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf00004d40 0x7fdf000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.144+0000 7fdf19ead640 1 --2- 172.21.15.16:0/488032259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf00003b70 0x7fdf00003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.144+0000 7fdf196ac640 1 --2- 172.21.15.16:0/488032259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf140d4370 0x7fdf00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.144+0000 7fdf1a6ae640 1 -- 172.21.15.16:0/488032259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf140d4370 msgr2=0x7fdf00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.144+0000 7fdf1a6ae640 1 --2- 172.21.15.16:0/488032259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf140d4370 0x7fdf00004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.144+0000 7fdf1a6ae640 1 -- 172.21.15.16:0/488032259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf00003b70 msgr2=0x7fdf00003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.144+0000 7fdf1a6ae640 1 --2- 172.21.15.16:0/488032259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf00003b70 0x7fdf00003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.144+0000 7fdf1a6ae640 1 -- 172.21.15.16:0/488032259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf00003520 con 0x7fdf00004d40 2024-09-19T07:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.145+0000 7fdf1a6ae640 1 --2- 172.21.15.16:0/488032259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf00004d40 0x7fdf000051c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009fc0 tx=0x7fdf00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc67f44a482e12e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.145+0000 7fdf18eab640 1 -- 172.21.15.16:0/488032259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c009c10 con 0x7fdf00004d40 2024-09-19T07:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.145+0000 7fdf18eab640 1 -- 172.21.15.16:0/488032259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf0c036070 con 0x7fdf00004d40 2024-09-19T07:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.145+0000 7fdf18eab640 1 -- 172.21.15.16:0/488032259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c031880 con 0x7fdf00004d40 2024-09-19T07:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.145+0000 7fdf1c111640 1 -- 172.21.15.16:0/488032259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf00004d40 msgr2=0x7fdf000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.145+0000 7fdf1c111640 1 --2- 172.21.15.16:0/488032259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf00004d40 0x7fdf000051c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009fc0 tx=0x7fdf00005830 comp rx=0 tx=0).stop 2024-09-19T07:49:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.145+0000 7fdf1a6ae640 1 -- 172.21.15.16:0/488032259 reap_dead start 2024-09-19T07:49:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.145+0000 7fdf1c111640 1 -- 172.21.15.16:0/488032259 shutdown_connections 2024-09-19T07:49:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.145+0000 7fdf1c111640 1 -- 172.21.15.16:0/488032259 >> 172.21.15.16:0/488032259 conn(0x7fdf1409d760 msgr2=0x7fdf1408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.145+0000 7fdf1c111640 1 -- 172.21.15.16:0/488032259 shutdown_connections 2024-09-19T07:49:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.145+0000 7fdf1c111640 1 -- 172.21.15.16:0/488032259 wait complete. 2024-09-19T07:49:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.147+0000 7fdf1c111640 1 Processor -- start 2024-09-19T07:49:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.147+0000 7fdf1c111640 1 -- start start 2024-09-19T07:49:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.147+0000 7fdf1c111640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf14155bb0 0x7fdf14159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.147+0000 7fdf1c111640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf140cb4b0 0x7fdf14156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.147+0000 7fdf1c111640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf140ca0e0 0x7fdf140ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.147+0000 7fdf1c111640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1406dcb0 con 0x7fdf140ca0e0 2024-09-19T07:49:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.147+0000 7fdf1c111640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1406db30 con 0x7fdf14155bb0 2024-09-19T07:49:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.147+0000 7fdf1c111640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdf140d01e0 con 0x7fdf140cb4b0 2024-09-19T07:49:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.148+0000 7fdf1a6ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf140ca0e0 0x7fdf140ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.148+0000 7fdf1a6ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf140ca0e0 0x7fdf140ca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33820/0 (socket says 172.21.15.16:33820) 2024-09-19T07:49:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.148+0000 7fdf1a6ae640 1 -- 172.21.15.16:0/3848407232 learned_addr learned my addr 172.21.15.16:0/3848407232 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:11.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.148+0000 7fdf196ac640 1 --2- 172.21.15.16:0/3848407232 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf140cb4b0 0x7fdf14156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.148+0000 7fdf19ead640 1 --2- 172.21.15.16:0/3848407232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf14155bb0 0x7fdf14159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.148+0000 7fdf1a6ae640 1 -- 172.21.15.16:0/3848407232 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf140cb4b0 msgr2=0x7fdf14156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.148+0000 7fdf1a6ae640 1 --2- 172.21.15.16:0/3848407232 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf140cb4b0 0x7fdf14156610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.148+0000 7fdf1a6ae640 1 -- 172.21.15.16:0/3848407232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf14155bb0 msgr2=0x7fdf14159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.148+0000 7fdf1a6ae640 1 --2- 172.21.15.16:0/3848407232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf14155bb0 0x7fdf14159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.148+0000 7fdf1a6ae640 1 -- 172.21.15.16:0/3848407232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf140cb930 con 0x7fdf140ca0e0 2024-09-19T07:49:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.148+0000 7fdf1a6ae640 1 --2- 172.21.15.16:0/3848407232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf140ca0e0 0x7fdf140ca560 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00e230 tx=0x7fdf0c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.148+0000 7fdf0a7fc640 1 -- 172.21.15.16:0/3848407232 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c009c10 con 0x7fdf140ca0e0 2024-09-19T07:49:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.148+0000 7fdf0a7fc640 1 -- 172.21.15.16:0/3848407232 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf0c031df0 con 0x7fdf140ca0e0 2024-09-19T07:49:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.148+0000 7fdf0a7fc640 1 -- 172.21.15.16:0/3848407232 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c003bd0 con 0x7fdf140ca0e0 2024-09-19T07:49:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.148+0000 7fdf1c111640 1 -- 172.21.15.16:0/3848407232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1415b290 con 0x7fdf140ca0e0 2024-09-19T07:49:11.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.148+0000 7fdf1c111640 1 -- 172.21.15.16:0/3848407232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1415b850 con 0x7fdf140ca0e0 2024-09-19T07:49:11.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.149+0000 7fdf1c111640 1 -- 172.21.15.16:0/3848407232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf140d4820 con 0x7fdf140ca0e0 2024-09-19T07:49:11.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.150+0000 7fdf0a7fc640 1 -- 172.21.15.16:0/3848407232 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdf0c005d10 con 0x7fdf140ca0e0 2024-09-19T07:49:11.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.150+0000 7fdf0a7fc640 1 --2- 172.21.15.16:0/3848407232 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdef806c2e0 0x7fdef806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:11.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.150+0000 7fdf0a7fc640 1 -- 172.21.15.16:0/3848407232 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7fdf0c0ac5c0 con 0x7fdf140ca0e0 2024-09-19T07:49:11.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.151+0000 7fdf19ead640 1 --2- 172.21.15.16:0/3848407232 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdef806c2e0 0x7fdef806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:11.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.152+0000 7fdf19ead640 1 --2- 172.21.15.16:0/3848407232 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdef806c2e0 0x7fdef806e7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdf140a22a0 tx=0x7fdf10009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:11.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.154+0000 7fdf0a7fc640 1 -- 172.21.15.16:0/3848407232 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fdf0c04a060 con 0x7fdf140ca0e0 2024-09-19T07:49:11.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.342+0000 7fdf1c111640 1 -- 172.21.15.16:0/3848407232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdf1406cf90 con 0x7fdf140ca0e0 2024-09-19T07:49:11.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.342+0000 7fdf0a7fc640 1 -- 172.21.15.16:0/3848407232 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) ==== 80+0+0 (secure 0 0 0) 0x7fdf140cb930 con 0x7fdf140ca0e0 2024-09-19T07:49:11.343 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:49:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.344+0000 7fdf1c111640 1 -- 172.21.15.16:0/3848407232 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdef806c2e0 msgr2=0x7fdef806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.344+0000 7fdf1c111640 1 --2- 172.21.15.16:0/3848407232 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdef806c2e0 0x7fdef806e7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdf140a22a0 tx=0x7fdf10009040 comp rx=0 tx=0).stop 2024-09-19T07:49:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.344+0000 7fdf1c111640 1 -- 172.21.15.16:0/3848407232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf140ca0e0 msgr2=0x7fdf140ca560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.344+0000 7fdf1c111640 1 --2- 172.21.15.16:0/3848407232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf140ca0e0 0x7fdf140ca560 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00e230 tx=0x7fdf0c009750 comp rx=0 tx=0).stop 2024-09-19T07:49:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.344+0000 7fdf1c111640 1 -- 172.21.15.16:0/3848407232 shutdown_connections 2024-09-19T07:49:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.344+0000 7fdf1c111640 1 --2- 172.21.15.16:0/3848407232 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdef806c2e0 0x7fdef806e7a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.344+0000 7fdf1c111640 1 --2- 172.21.15.16:0/3848407232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf140ca0e0 0x7fdf140ca560 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.344+0000 7fdf1c111640 1 --2- 172.21.15.16:0/3848407232 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf140cb4b0 0x7fdf14156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.344+0000 7fdf1c111640 1 --2- 172.21.15.16:0/3848407232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf14155bb0 0x7fdf14159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.345+0000 7fdf1c111640 1 -- 172.21.15.16:0/3848407232 >> 172.21.15.16:0/3848407232 conn(0x7fdf1409d760 msgr2=0x7fdf140b7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.345+0000 7fdf1c111640 1 -- 172.21.15.16:0/3848407232 shutdown_connections 2024-09-19T07:49:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:11.345+0000 7fdf1c111640 1 -- 172.21.15.16:0/3848407232 wait complete. 2024-09-19T07:49:11.355 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-19T07:49:11.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:11.356 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:11.380 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[55377]: fuse finished with error 0 and tester_r 0 2024-09-19T07:49:11.382 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:49:17.484 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:49:17.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:17.485 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:49:17.506 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T07:49:17.548 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T07:49:17.599 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-09-19T07:49:17.599 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2024-09-19T07:49:17.599 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T07:49:17.599 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2024-09-19T07:49:17.599 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.599 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2024-09-19T07:49:17.599 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.599 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2024-09-19T07:49:17.599 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.600 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2024-09-19T07:49:17.600 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2024-09-19T07:49:17.600 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.600 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2024-09-19T07:49:17.600 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2024-09-19T07:49:17.600 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.600 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T07:49:17.600 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2024-09-19T07:49:17.600 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.600 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T07:49:17.600 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T07:49:17.600 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.600 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2024-09-19T07:49:17.601 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.601 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2024-09-19T07:49:17.601 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2024-09-19T07:49:17.601 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2024-09-19T07:49:17.601 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2024-09-19T07:49:17.601 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2024-09-19T07:49:17.601 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2024-09-19T07:49:17.601 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2024-09-19T07:49:17.601 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T07:49:17.601 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T07:49:17.601 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2024-09-19T07:49:17.601 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.601 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T07:49:17.601 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.602 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2024-09-19T07:49:17.602 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.602 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2024-09-19T07:49:17.602 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.602 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2024-09-19T07:49:17.602 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2024-09-19T07:49:17.602 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2024-09-19T07:49:17.602 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-09-19T07:49:17.602 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb 2024-09-19T07:49:17.602 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.602 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2024-09-19T07:49:17.602 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2024-09-19T07:49:17.602 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.602 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2024-09-19T07:49:17.603 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2024-09-19T07:49:17.603 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2024-09-19T07:49:17.603 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.603 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2024-09-19T07:49:17.603 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2024-09-19T07:49:17.603 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.603 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2024-09-19T07:49:17.603 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2024-09-19T07:49:17.603 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2024-09-19T07:49:17.603 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2024-09-19T07:49:17.603 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.603 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2024-09-19T07:49:17.603 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2024-09-19T07:49:17.603 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.604 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2024-09-19T07:49:17.604 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.604 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2024-09-19T07:49:17.604 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.604 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2024-09-19T07:49:17.604 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.604 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2024-09-19T07:49:17.604 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2024-09-19T07:49:17.604 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.604 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2024-09-19T07:49:17.604 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2024-09-19T07:49:17.604 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2024-09-19T07:49:17.604 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2024-09-19T07:49:17.604 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2024-09-19T07:49:17.604 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2024-09-19T07:49:17.605 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:49:17.605 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2024-09-19T07:49:17.605 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2024-09-19T07:49:17.605 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T07:49:17.605 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2024-09-19T07:49:17.605 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2024-09-19T07:49:17.605 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2024-09-19T07:49:17.605 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2024-09-19T07:49:17.605 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-09-19T07:49:17.605 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.605 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2024-09-19T07:49:17.605 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:49:17.605 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2024-09-19T07:49:17.606 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-19T07:49:17.606 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-19T07:49:17.606 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:49:17.606 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:49:17.606 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2024-09-19T07:49:17.606 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-09-19T07:49:17.606 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.606 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2024-09-19T07:49:17.606 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:49:17.606 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-19T07:49:17.606 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2024-09-19T07:49:17.606 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-19T07:49:17.606 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2024-09-19T07:49:17.606 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2024-09-19T07:49:17.607 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:49:17.607 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2024-09-19T07:49:17.607 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2024-09-19T07:49:17.607 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:49:17.607 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.607 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T07:49:17.607 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T07:49:17.607 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T07:49:17.607 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.607 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T07:49:17.607 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T07:49:17.607 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T07:49:17.607 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.607 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2024-09-19T07:49:17.608 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2024-09-19T07:49:17.608 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.608 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2024-09-19T07:49:17.608 INFO:teuthology.orchestra.run.smithi016.stdout: mon down mkfs grace = 300 2024-09-19T07:49:17.608 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2024-09-19T07:49:17.608 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on pool no app = False 2024-09-19T07:49:17.608 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:17.608 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2024-09-19T07:49:17.608 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2024-09-19T07:49:17.608 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2024-09-19T07:49:17.608 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2024-09-19T07:49:17.608 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T07:49:17.609 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T07:49:17.609 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2024-09-19T07:49:17.609 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-09-19T07:49:17.609 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:49:17.609 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2024-09-19T07:49:17.609 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-09-19T07:49:17.609 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-09-19T07:49:17.609 INFO:teuthology.orchestra.run.smithi016.stdout: client mountpoint = /subdir 2024-09-19T07:49:17.609 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2024-09-19T07:49:17.609 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2024-09-19T07:49:17.609 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2024-09-19T07:49:17.609 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-09-19T07:49:17.609 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-09-19T07:49:17.610 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-09-19T07:49:17.610 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-09-19T07:49:17.610 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2024-09-19T07:49:17.610 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2024-09-19T07:49:17.610 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2024-09-19T07:49:17.610 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:49:17.610 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2024-09-19T07:49:17.610 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2024-09-19T07:49:17.610 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2024-09-19T07:49:17.610 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = True 2024-09-19T07:49:17.610 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-09-19T07:49:17.610 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-09-19T07:49:17.610 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-09-19T07:49:17.611 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2024-09-19T07:49:17.611 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2024-09-19T07:49:17.611 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T07:49:17.611 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2024-09-19T07:49:17.611 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.611 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2024-09-19T07:49:17.611 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.611 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2024-09-19T07:49:17.611 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.611 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2024-09-19T07:49:17.612 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2024-09-19T07:49:17.612 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.612 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2024-09-19T07:49:17.612 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2024-09-19T07:49:17.612 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.612 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T07:49:17.612 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2024-09-19T07:49:17.612 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.612 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T07:49:17.612 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T07:49:17.612 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.612 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2024-09-19T07:49:17.612 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.613 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2024-09-19T07:49:17.613 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2024-09-19T07:49:17.613 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2024-09-19T07:49:17.613 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2024-09-19T07:49:17.613 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2024-09-19T07:49:17.613 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2024-09-19T07:49:17.613 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2024-09-19T07:49:17.613 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T07:49:17.613 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T07:49:17.613 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2024-09-19T07:49:17.613 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.613 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T07:49:17.613 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.614 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2024-09-19T07:49:17.614 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.614 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2024-09-19T07:49:17.614 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.614 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2024-09-19T07:49:17.614 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2024-09-19T07:49:17.614 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2024-09-19T07:49:17.614 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.16,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-09-19T07:49:17.614 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = 6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb 2024-09-19T07:49:17.614 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.614 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2024-09-19T07:49:17.614 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2024-09-19T07:49:17.614 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.614 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2024-09-19T07:49:17.615 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2024-09-19T07:49:17.615 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2024-09-19T07:49:17.615 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.615 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2024-09-19T07:49:17.615 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2024-09-19T07:49:17.615 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.615 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2024-09-19T07:49:17.615 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2024-09-19T07:49:17.615 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2024-09-19T07:49:17.615 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug trim objects = true 2024-09-19T07:49:17.615 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.615 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2024-09-19T07:49:17.615 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2024-09-19T07:49:17.615 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.616 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2024-09-19T07:49:17.616 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.616 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2024-09-19T07:49:17.616 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.616 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2024-09-19T07:49:17.616 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.616 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2024-09-19T07:49:17.616 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2024-09-19T07:49:17.616 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.616 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2024-09-19T07:49:17.616 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2024-09-19T07:49:17.616 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2024-09-19T07:49:17.616 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2024-09-19T07:49:17.617 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 20 2024-09-19T07:49:17.617 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 20 2024-09-19T07:49:17.617 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:49:17.617 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2024-09-19T07:49:17.617 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 10 2024-09-19T07:49:17.617 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T07:49:17.617 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2024-09-19T07:49:17.617 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2024-09-19T07:49:17.617 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2024-09-19T07:49:17.617 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2024-09-19T07:49:17.617 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-09-19T07:49:17.617 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.617 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2024-09-19T07:49:17.617 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:49:17.618 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2024-09-19T07:49:17.618 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-19T07:49:17.618 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-19T07:49:17.618 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:49:17.618 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:49:17.618 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2024-09-19T07:49:17.618 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-09-19T07:49:17.618 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.618 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2024-09-19T07:49:17.618 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:49:17.618 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-19T07:49:17.618 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2024-09-19T07:49:17.618 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-19T07:49:17.618 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2024-09-19T07:49:17.619 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2024-09-19T07:49:17.619 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:49:17.619 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2024-09-19T07:49:17.619 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2024-09-19T07:49:17.619 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:49:17.619 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.619 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T07:49:17.619 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T07:49:17.619 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T07:49:17.619 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.619 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T07:49:17.619 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T07:49:17.619 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T07:49:17.619 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.619 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2024-09-19T07:49:17.620 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2024-09-19T07:49:17.620 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.620 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2024-09-19T07:49:17.620 INFO:teuthology.orchestra.run.smithi146.stdout: mon down mkfs grace = 300 2024-09-19T07:49:17.620 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2024-09-19T07:49:17.620 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on pool no app = False 2024-09-19T07:49:17.620 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:17.620 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2024-09-19T07:49:17.620 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2024-09-19T07:49:17.620 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2024-09-19T07:49:17.620 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2024-09-19T07:49:17.620 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T07:49:17.620 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T07:49:17.621 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2024-09-19T07:49:17.621 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-09-19T07:49:17.621 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:49:17.621 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = "" 2024-09-19T07:49:17.621 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-09-19T07:49:17.621 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-09-19T07:49:17.621 INFO:teuthology.orchestra.run.smithi146.stdout: client mountpoint = /subdir 2024-09-19T07:49:17.621 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2024-09-19T07:49:17.621 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2024-09-19T07:49:17.621 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2024-09-19T07:49:17.621 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2024-09-19T07:49:17.621 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2024-09-19T07:49:17.621 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2024-09-19T07:49:17.622 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2024-09-19T07:49:17.622 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2024-09-19T07:49:17.622 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2024-09-19T07:49:17.622 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2024-09-19T07:49:17.622 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:49:17.622 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2024-09-19T07:49:17.622 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2024-09-19T07:49:17.622 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2024-09-19T07:49:17.622 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2024-09-19T07:49:17.622 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-09-19T07:49:17.622 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-09-19T07:49:17.622 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-09-19T07:49:17.624 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:49:17.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.731+0000 7f8885ce6640 1 Processor -- start 2024-09-19T07:49:17.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.731+0000 7f8885ce6640 1 -- start start 2024-09-19T07:49:17.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.731+0000 7f8885ce6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88800cdcc0 0x7f88800ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.731+0000 7f8885ce6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f88800b3b90 0x7f88800b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.731+0000 7f8885ce6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f888006ea40 con 0x7f8880155700 2024-09-19T07:49:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.731+0000 7f8885ce6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f88800b3140 con 0x7f88800cdcc0 2024-09-19T07:49:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.731+0000 7f8885ce6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f88800b3700 con 0x7f88800b3b90 2024-09-19T07:49:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.732+0000 7f887effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88800cdcc0 0x7f88800ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.732+0000 7f887effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88800cdcc0 0x7f88800ce150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35720/0 (socket says 172.21.15.16:35720) 2024-09-19T07:49:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.732+0000 7f887effd640 1 -- 172.21.15.16:0/3310128935 learned_addr learned my addr 172.21.15.16:0/3310128935 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.732+0000 7f887ffff640 1 --2- 172.21.15.16:0/3310128935 >> v2:172.21.15.146:3300/0 conn(0x7f88800b3b90 0x7f88800b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.732+0000 7f887effd640 1 -- 172.21.15.16:0/3310128935 >> v2:172.21.15.146:3300/0 conn(0x7f88800b3b90 msgr2=0x7f88800b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.732+0000 7f887effd640 1 --2- 172.21.15.16:0/3310128935 >> v2:172.21.15.146:3300/0 conn(0x7f88800b3b90 0x7f88800b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.732+0000 7f887effd640 1 -- 172.21.15.16:0/3310128935 >> v1:172.21.15.146:6789/0 conn(0x7f8880155700 legacy=0x7f888006df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.732+0000 7f887effd640 1 -- 172.21.15.16:0/3310128935 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888006ed20 con 0x7f88800cdcc0 2024-09-19T07:49:17.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.733+0000 7f887effd640 1 --2- 172.21.15.16:0/3310128935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88800cdcc0 0x7f88800ce150 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8874009870 tx=0x7f887402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942b071cd916075 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:17.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.733+0000 7f887e7fc640 1 -- 172.21.15.16:0/3310128935 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874037070 con 0x7f88800cdcc0 2024-09-19T07:49:17.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.733+0000 7f887e7fc640 1 -- 172.21.15.16:0/3310128935 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8874032440 con 0x7f88800cdcc0 2024-09-19T07:49:17.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.733+0000 7f887e7fc640 1 -- 172.21.15.16:0/3310128935 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874032810 con 0x7f88800cdcc0 2024-09-19T07:49:17.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.733+0000 7f8885ce6640 1 -- 172.21.15.16:0/3310128935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88800cdcc0 msgr2=0x7f88800ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:17.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.733+0000 7f8885ce6640 1 --2- 172.21.15.16:0/3310128935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88800cdcc0 0x7f88800ce150 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8874009870 tx=0x7f887402a210 comp rx=0 tx=0).stop 2024-09-19T07:49:17.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.733+0000 7f8885ce6640 1 -- 172.21.15.16:0/3310128935 shutdown_connections 2024-09-19T07:49:17.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.733+0000 7f8885ce6640 1 --2- 172.21.15.16:0/3310128935 >> v2:172.21.15.146:3300/0 conn(0x7f88800b3b90 0x7f88800b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:17.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.733+0000 7f8885ce6640 1 --2- 172.21.15.16:0/3310128935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88800cdcc0 0x7f88800ce150 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:17.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.733+0000 7f8885ce6640 1 -- 172.21.15.16:0/3310128935 >> 172.21.15.16:0/3310128935 conn(0x7f888006bbd0 msgr2=0x7f888006bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:17.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.734+0000 7f8885ce6640 1 -- 172.21.15.16:0/3310128935 shutdown_connections 2024-09-19T07:49:17.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.734+0000 7f8885ce6640 1 -- 172.21.15.16:0/3310128935 wait complete. 2024-09-19T07:49:17.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.735+0000 7f8885ce6640 1 Processor -- start 2024-09-19T07:49:17.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.735+0000 7f8885ce6640 1 -- start start 2024-09-19T07:49:17.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f8885ce6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88800b3b90 0x7f88801f9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:17.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f8885ce6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88800cdcc0 0x7f88801f8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:17.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f8885ce6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8880155700 0x7f88801fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:17.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f8885ce6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8880155ae0 con 0x7f88800cdcc0 2024-09-19T07:49:17.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f8885ce6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f888015b9a0 con 0x7f8880155700 2024-09-19T07:49:17.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f8885ce6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f888006e690 con 0x7f88800b3b90 2024-09-19T07:49:17.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f887effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88800cdcc0 0x7f88801f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:17.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f887effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88800cdcc0 0x7f88801f8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35724/0 (socket says 172.21.15.16:35724) 2024-09-19T07:49:17.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f887effd640 1 -- 172.21.15.16:0/3992822132 learned_addr learned my addr 172.21.15.16:0/3992822132 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:17.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f887ffff640 1 --2- 172.21.15.16:0/3992822132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8880155700 0x7f88801fbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:17.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f887f7fe640 1 --2- 172.21.15.16:0/3992822132 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88800b3b90 0x7f88801f9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:17.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f887effd640 1 -- 172.21.15.16:0/3992822132 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88800b3b90 msgr2=0x7f88801f9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:17.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f887effd640 1 --2- 172.21.15.16:0/3992822132 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88800b3b90 0x7f88801f9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:17.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f887effd640 1 -- 172.21.15.16:0/3992822132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8880155700 msgr2=0x7f88801fbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:17.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f887effd640 1 --2- 172.21.15.16:0/3992822132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8880155700 0x7f88801fbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:17.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f887effd640 1 -- 172.21.15.16:0/3992822132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88801fc200 con 0x7f88800cdcc0 2024-09-19T07:49:17.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f887effd640 1 --2- 172.21.15.16:0/3992822132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88800cdcc0 0x7f88801f8190 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8874000c00 tx=0x7f88740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:17.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f8867fff640 1 -- 172.21.15.16:0/3992822132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874037040 con 0x7f88800cdcc0 2024-09-19T07:49:17.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f8867fff640 1 -- 172.21.15.16:0/3992822132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8874042070 con 0x7f88800cdcc0 2024-09-19T07:49:17.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f8885ce6640 1 -- 172.21.15.16:0/3992822132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88801fc4d0 con 0x7f88800cdcc0 2024-09-19T07:49:17.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.736+0000 7f8867fff640 1 -- 172.21.15.16:0/3992822132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874030880 con 0x7f88800cdcc0 2024-09-19T07:49:17.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.737+0000 7f8885ce6640 1 -- 172.21.15.16:0/3992822132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88801fca90 con 0x7f88800cdcc0 2024-09-19T07:49:17.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.737+0000 7f8885ce6640 1 -- 172.21.15.16:0/3992822132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88800ce150 con 0x7f88800cdcc0 2024-09-19T07:49:17.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.738+0000 7f8867fff640 1 -- 172.21.15.16:0/3992822132 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8874030a20 con 0x7f88800cdcc0 2024-09-19T07:49:17.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.738+0000 7f8867fff640 1 --2- 172.21.15.16:0/3992822132 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f886c06c2b0 0x7f886c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:17.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.739+0000 7f8867fff640 1 -- 172.21.15.16:0/3992822132 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f88740abc80 con 0x7f88800cdcc0 2024-09-19T07:49:17.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.739+0000 7f887f7fe640 1 --2- 172.21.15.16:0/3992822132 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f886c06c2b0 0x7f886c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:17.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.739+0000 7f887f7fe640 1 --2- 172.21.15.16:0/3992822132 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f886c06c2b0 0x7f886c06e770 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f88700016e0 tx=0x7f8870009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:17.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.743+0000 7f8867fff640 1 -- 172.21.15.16:0/3992822132 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8874079c00 con 0x7f88800cdcc0 2024-09-19T07:49:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.953+0000 7f8885ce6640 1 -- 172.21.15.16:0/3992822132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f888014e7a0 con 0x7f88800cdcc0 2024-09-19T07:49:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.954+0000 7f8867fff640 1 -- 172.21.15.16:0/3992822132 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) ==== 53+0+73 (secure 0 0 0) 0x7f887407df00 con 0x7f88800cdcc0 2024-09-19T07:49:17.954 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:49:17.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.956+0000 7f8885ce6640 1 -- 172.21.15.16:0/3992822132 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f886c06c2b0 msgr2=0x7f886c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:17.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.956+0000 7f8885ce6640 1 --2- 172.21.15.16:0/3992822132 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f886c06c2b0 0x7f886c06e770 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f88700016e0 tx=0x7f8870009040 comp rx=0 tx=0).stop 2024-09-19T07:49:17.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.956+0000 7f8885ce6640 1 -- 172.21.15.16:0/3992822132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88800cdcc0 msgr2=0x7f88801f8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:17.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.956+0000 7f8885ce6640 1 --2- 172.21.15.16:0/3992822132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88800cdcc0 0x7f88801f8190 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8874000c00 tx=0x7f88740046c0 comp rx=0 tx=0).stop 2024-09-19T07:49:17.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.956+0000 7f8885ce6640 1 -- 172.21.15.16:0/3992822132 shutdown_connections 2024-09-19T07:49:17.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.956+0000 7f8885ce6640 1 --2- 172.21.15.16:0/3992822132 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f886c06c2b0 0x7f886c06e770 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:17.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.956+0000 7f8885ce6640 1 --2- 172.21.15.16:0/3992822132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8880155700 0x7f88801fbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:17.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.956+0000 7f8885ce6640 1 --2- 172.21.15.16:0/3992822132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88800cdcc0 0x7f88801f8190 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:17.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.956+0000 7f8885ce6640 1 --2- 172.21.15.16:0/3992822132 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88800b3b90 0x7f88801f9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:17.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.956+0000 7f8885ce6640 1 -- 172.21.15.16:0/3992822132 >> 172.21.15.16:0/3992822132 conn(0x7f888006bbd0 msgr2=0x7f88800b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:17.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.956+0000 7f8885ce6640 1 -- 172.21.15.16:0/3992822132 shutdown_connections 2024-09-19T07:49:17.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:17.956+0000 7f8885ce6640 1 -- 172.21.15.16:0/3992822132 wait complete. 2024-09-19T07:49:17.967 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:49:17.967 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:49:17.967 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T07:49:17.967 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T07:49:17.967 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:17.968 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:49:17.968 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:49:17.968 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:49:17.968 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:49:17.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:17.968 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T07:49:17.984 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:49:17.984 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:49:17.985 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:49:17.985 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:17.985 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:49:17.985 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:17.985 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:49:17.985 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:17.985 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T07:49:17.985 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:49:17.985 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:17.985 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T07:49:17.985 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:49:17.985 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:17.985 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T07:49:17.986 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T07:49:17.986 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42431sec preferred_lft 42431sec 2024-09-19T07:49:17.986 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T07:49:17.986 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:17.986 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:49:17.986 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:17.986 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T07:49:17.986 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:49:17.986 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:17.986 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:49:17.986 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:17.986 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::32:7bff:fe92:8895/64 scope link 2024-09-19T07:49:17.986 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:17.986 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:49:17.986 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether e6:fd:21:6a:43:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:49:17.987 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e4fd:21ff:fe6a:4369/64 scope link 2024-09-19T07:49:17.987 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:17.987 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:49:17.987 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:49:17.987 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::80f4:81ff:fe7d:b7f0/64 scope link 2024-09-19T07:49:17.987 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:17.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:17.987 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:49:18.049 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:49:18.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:18.051 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T07:49:18.107 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T07:49:18.108 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:49:18.108 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T07:49:18.108 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T07:49:18.108 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:49:18.108 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:49:18.108 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:49:18.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T07:49:18.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T07:49:18.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T07:49:18.109 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:49:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:49:18.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:18.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:18.237 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:49:18.253 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T07:49:18.253 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:49:18.253 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:49:18.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:18.253 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-19T07:49:18.319 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T07:49:18.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:49:18.319 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:18.375 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:49:18.375 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:49:18.375 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:18.430 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:49:18.497 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:49:18.555 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:49:18.555 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:49:18.555 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:49:18.555 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:49:18.555 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:49:18.555 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:49:18.555 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:49:18.555 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:49:18.555 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:49:18.556 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:49:18.556 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:49:18.556 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:49:18.556 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:49:18.556 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:49:18.556 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:49:18.556 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:49:18.556 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:49:18.556 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:49:18.556 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:49:18.556 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:49:18.556 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:49:18.556 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:49:18.557 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:49:18.557 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:49:18.557 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:49:18.557 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:49:18.557 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:49:18.557 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:49:18.557 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:49:18.557 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:49:18.557 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:49:18.557 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:49:18.557 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:49:18.557 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:49:18.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:18.558 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:49:18.611 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:49:18.611 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T07:49:18.653 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:49:18.680 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:49:18.736 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:49:18.736 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:49:18.737 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:49:18.737 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:49:18.737 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:49:18.737 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:49:18.737 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:49:18.737 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:49:18.737 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:49:18.737 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:49:18.737 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:49:18.737 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:49:18.738 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:49:18.738 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:49:18.738 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:49:18.738 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:49:18.738 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:49:18.739 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:49:18.739 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:49:18.739 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:49:18.739 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:49:18.739 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:49:18.739 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:49:18.739 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:49:18.739 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:49:18.739 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:49:18.739 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:49:18.739 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:49:18.739 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:49:18.739 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:49:18.739 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:49:18.740 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:49:18.740 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:49:18.740 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:49:18.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:18.741 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:49:18.742 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-19T07:49:18.739+0000 7f11a4469480 -1 init, newargv = 0x55cb81b627d0 newargc=15 2024-09-19T07:49:18.742 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[56235]: starting ceph client 2024-09-19T07:49:18.743 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[56235]: starting fuse 2024-09-19T07:49:18.758 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:49:18.758 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-19T07:49:18.759 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:49:18.759 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:49:18.759 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:49:18.759 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:49:18.759 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:49:18.759 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:18.759 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:49:18.759 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:49:18.759 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:49:18.759 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:49:18.759 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:18.760 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:49:18.760 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:49:18.760 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:49:18.760 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:49:18.760 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:18.760 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:49:18.760 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:49:18.761 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:49:18.761 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:49:18.761 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:49:18.761 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:49:18.761 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:49:18.761 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:49:18.761 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:18.761 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:49:18.761 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:49:18.878 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.56235.asok 2024-09-19T07:49:18.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:18.889 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.56235.asok 2024-09-19T07:49:18.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:18.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.56235.asok status 2024-09-19T07:49:19.026 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:49:19.026 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:49:19.027 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:49:19.027 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:49:19.027 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:49:19.027 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:49:19.027 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:49:19.027 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "56235", 2024-09-19T07:49:19.027 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/subdir" 2024-09-19T07:49:19.027 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:19.027 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-19T07:49:19.027 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-19T07:49:19.027 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5949, 2024-09-19T07:49:19.027 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:49:19.028 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:49:19.028 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:49:19.028 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5949 2024-09-19T07:49:19.028 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:19.028 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:49:19.028 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:19.028 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:19.028 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1025819991 2024-09-19T07:49:19.028 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:19.028 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:19.028 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:49:19.028 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:19.028 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:19.029 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1025819991 2024-09-19T07:49:19.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:19.029 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.5949 192.168.0.1:0/1025819991", 2024-09-19T07:49:19.029 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/1025819991", 2024-09-19T07:49:19.029 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 2, 2024-09-19T07:49:19.029 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 33, 2024-09-19T07:49:19.029 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 70, 2024-09-19T07:49:19.029 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:49:19.029 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:49:19.030 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:49:19.030 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:49:19.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:19.035 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-19T07:49:19.036 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:19.052 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-19T07:49:19.053 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:19.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:19.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:19.120 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-19T07:49:19.120 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:49:19.120 DEBUG:teuthology.orchestra.run.smithi016:> import stat 2024-09-19T07:49:19.120 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:19.120 DEBUG:teuthology.orchestra.run.smithi016:> print(os.stat("/home/ubuntu/cephtest/mnt.0/.").st_ino) 2024-09-19T07:49:19.120 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:49:19.227 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:49:19.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:19.236 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-19T07:49:19.236 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:49:19.236 DEBUG:teuthology.orchestra.run.smithi016:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir") 2024-09-19T07:49:19.236 DEBUG:teuthology.orchestra.run.smithi016:> for n in range(0, 1000): 2024-09-19T07:49:19.237 DEBUG:teuthology.orchestra.run.smithi016:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir/dir{0}".format(n)) 2024-09-19T07:49:19.237 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:49:20.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:20.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:20.461 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync) 2024-09-19T07:49:21.326 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:49:21.326 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:49:21.326 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:49:21.326 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:49:21.326 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:49:21.326 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:21.326 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:49:21.327 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:49:21.327 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:49:21.327 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:49:21.327 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:21.327 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:49:21.327 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:49:21.327 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:49:21.327 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:49:21.327 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:21.327 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:49:21.327 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:49:21.327 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:49:21.327 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:49:21.327 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:49:21.327 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:49:21.328 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:49:21.328 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:49:21.328 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:21.328 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:49:21.328 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:49:21.421 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.56235.asok 2024-09-19T07:49:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:21.431 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.56235.asok 2024-09-19T07:49:21.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:21.431 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.56235.asok status 2024-09-19T07:49:21.569 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:49:21.569 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:49:21.570 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:49:21.570 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:49:21.570 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:49:21.570 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:49:21.570 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:49:21.570 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "56235", 2024-09-19T07:49:21.570 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/subdir" 2024-09-19T07:49:21.570 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:21.570 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 1001, 2024-09-19T07:49:21.570 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 1001, 2024-09-19T07:49:21.570 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5949, 2024-09-19T07:49:21.570 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:49:21.570 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:49:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:49:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5949 2024-09-19T07:49:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:49:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1025819991 2024-09-19T07:49:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:49:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1025819991 2024-09-19T07:49:21.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:21.572 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.5949 192.168.0.1:0/1025819991", 2024-09-19T07:49:21.572 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/1025819991", 2024-09-19T07:49:21.572 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1003, 2024-09-19T07:49:21.572 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 33, 2024-09-19T07:49:21.572 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 70, 2024-09-19T07:49:21.572 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 70, 2024-09-19T07:49:21.572 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:49:21.572 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:49:21.572 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:49:21.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:49:21.578 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:49:21.578 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:49:21.578 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:49:21.578 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:49:21.578 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:21.578 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:49:21.578 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:49:21.579 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:49:21.579 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:49:21.579 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:21.579 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:49:21.579 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:49:21.579 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:49:21.579 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:49:21.579 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:21.579 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:49:21.579 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:49:21.579 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:49:21.579 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:49:21.579 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:49:21.579 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:49:21.579 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:49:21.580 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:49:21.580 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:21.580 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:49:21.580 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:49:21.660 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.56235.asok 2024-09-19T07:49:21.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:21.671 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.56235.asok 2024-09-19T07:49:21.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:21.672 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.56235.asok config set client_cache_size 100 2024-09-19T07:49:21.808 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:49:21.808 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "client_cache_size = '' " 2024-09-19T07:49:21.808 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:49:21.817 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:49:21.817 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:49:21.817 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:49:21.817 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:49:21.817 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:49:21.817 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:21.817 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:49:21.817 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:49:21.817 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:49:21.817 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:49:21.817 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:21.817 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:49:21.817 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:49:21.818 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:49:21.818 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:49:21.818 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:21.818 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:49:21.818 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:49:21.818 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:49:21.818 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:49:21.818 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:49:21.818 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:49:21.818 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:49:21.818 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:49:21.818 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:21.818 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:49:21.818 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:49:21.899 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.56235.asok 2024-09-19T07:49:21.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:21.908 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.56235.asok 2024-09-19T07:49:21.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:21.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.56235.asok status 2024-09-19T07:49:22.045 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:49:22.045 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:49:22.045 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:49:22.046 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:49:22.046 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:49:22.046 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:49:22.046 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:49:22.046 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "56235", 2024-09-19T07:49:22.046 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/subdir" 2024-09-19T07:49:22.046 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:22.046 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 1001, 2024-09-19T07:49:22.046 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 1001, 2024-09-19T07:49:22.046 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5949, 2024-09-19T07:49:22.046 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:49:22.046 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:49:22.046 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:49:22.047 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5949 2024-09-19T07:49:22.047 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:22.047 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:49:22.047 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:22.047 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:22.047 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1025819991 2024-09-19T07:49:22.047 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:22.047 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:22.047 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:49:22.047 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:22.047 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:22.047 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1025819991 2024-09-19T07:49:22.047 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:22.047 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.5949 192.168.0.1:0/1025819991", 2024-09-19T07:49:22.048 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/1025819991", 2024-09-19T07:49:22.048 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1003, 2024-09-19T07:49:22.048 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 33, 2024-09-19T07:49:22.048 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 70, 2024-09-19T07:49:22.048 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 70, 2024-09-19T07:49:22.048 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:49:22.048 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:49:22.048 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:49:22.054 INFO:tasks.cephfs.test_client_limits:waiting, dentry_count, dentry_pinned_count: 1001, 1001 2024-09-19T07:49:22.054 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-19T07:49:27.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:49:27.055 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:49:27.055 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:49:27.055 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:49:27.055 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:49:27.055 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:27.055 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:49:27.055 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:49:27.055 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:49:27.055 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:49:27.056 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:27.056 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:49:27.056 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:49:27.056 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:49:27.056 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:49:27.056 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:27.056 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:49:27.056 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:49:27.056 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:49:27.056 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:49:27.056 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:49:27.056 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:49:27.056 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:49:27.056 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:49:27.056 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:27.057 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:49:27.057 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:49:27.152 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.56235.asok 2024-09-19T07:49:27.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:27.161 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.56235.asok 2024-09-19T07:49:27.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:27.162 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.56235.asok status 2024-09-19T07:49:27.302 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:49:27.302 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:49:27.302 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:49:27.302 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:49:27.302 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:49:27.302 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:49:27.302 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:49:27.302 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "56235", 2024-09-19T07:49:27.302 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/subdir" 2024-09-19T07:49:27.303 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:27.303 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 100, 2024-09-19T07:49:27.303 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 1, 2024-09-19T07:49:27.303 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5949, 2024-09-19T07:49:27.303 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:49:27.303 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:49:27.304 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:49:27.304 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5949 2024-09-19T07:49:27.304 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:27.304 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:49:27.304 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:27.304 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:27.304 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1025819991 2024-09-19T07:49:27.304 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:27.304 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:27.304 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:49:27.304 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:27.304 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:27.304 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1025819991 2024-09-19T07:49:27.305 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:27.305 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.5949 192.168.0.1:0/1025819991", 2024-09-19T07:49:27.305 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/1025819991", 2024-09-19T07:49:27.305 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 102, 2024-09-19T07:49:27.305 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 33, 2024-09-19T07:49:27.305 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 70, 2024-09-19T07:49:27.305 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 70, 2024-09-19T07:49:27.305 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:49:27.305 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:49:27.305 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:49:27.310 INFO:tasks.cephfs.test_client_limits:waiting, dentry_count, dentry_pinned_count: 100, 1 2024-09-19T07:49:27.310 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-19T07:49:27.310 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:49:27.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:49:27.362 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:49:27.414 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:49:27.414 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:49:27.414 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:49:27.414 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:49:27.431 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:49:27.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.575+0000 7f8bdeb8d640 1 Processor -- start 2024-09-19T07:49:27.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.575+0000 7f8bdeb8d640 1 -- start start 2024-09-19T07:49:27.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.576+0000 7f8bdeb8d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8bd80b6a90 0x7f8bd8169760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:27.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.576+0000 7f8bdeb8d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bd80b5bf0 con 0x7f8bd8161cb0 2024-09-19T07:49:27.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.576+0000 7f8bdeb8d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bd80d53a0 con 0x7f8bd80d4f60 2024-09-19T07:49:27.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.576+0000 7f8bdeb8d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f8bd80b4400 con 0x7f8bd80b6a90 2024-09-19T07:49:27.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.576+0000 7f8bdc929640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8bd8161cb0 0x7f8bd80b3cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56190/0 (socket says 172.21.15.16:56190) 2024-09-19T07:49:27.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.576+0000 7f8bdc929640 1 -- 172.21.15.16:0/1499294221 learned_addr learned my addr 172.21.15.16:0/1499294221 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:27.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.576+0000 7f8bdd12a640 1 --2- 172.21.15.16:0/1499294221 >> v2:172.21.15.16:3300/0 conn(0x7f8bd80b6a90 0x7f8bd8169760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:27.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.576+0000 7f8bdd12a640 1 -- 172.21.15.16:0/1499294221 >> v1:172.21.15.146:6789/0 conn(0x7f8bd80d4f60 legacy=0x7f8bd80b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:27.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.576+0000 7f8bdd12a640 1 -- 172.21.15.16:0/1499294221 >> v1:172.21.15.16:6789/0 conn(0x7f8bd8161cb0 legacy=0x7f8bd80b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:27.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.576+0000 7f8bdd12a640 1 -- 172.21.15.16:0/1499294221 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd80cff40 con 0x7f8bd80b6a90 2024-09-19T07:49:27.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.577+0000 7f8bdd12a640 1 --2- 172.21.15.16:0/1499294221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd80b6a90 0x7f8bd8169760 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009d90 tx=0x7f8bd002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae09bca94a5f4d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:27.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.577+0000 7f8bd77fe640 1 -- 172.21.15.16:0/1499294221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0031600 con 0x7f8bd80b6a90 2024-09-19T07:49:27.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.577+0000 7f8bd77fe640 1 -- 172.21.15.16:0/1499294221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd00317a0 con 0x7f8bd80b6a90 2024-09-19T07:49:27.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.577+0000 7f8bd77fe640 1 -- 172.21.15.16:0/1499294221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0031b70 con 0x7f8bd80b6a90 2024-09-19T07:49:27.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.578+0000 7f8bdeb8d640 1 -- 172.21.15.16:0/1499294221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd80b6a90 msgr2=0x7f8bd8169760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:27.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.578+0000 7f8bdeb8d640 1 --2- 172.21.15.16:0/1499294221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd80b6a90 0x7f8bd8169760 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009d90 tx=0x7f8bd002a610 comp rx=0 tx=0).stop 2024-09-19T07:49:27.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.578+0000 7f8bdeb8d640 1 -- 172.21.15.16:0/1499294221 shutdown_connections 2024-09-19T07:49:27.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.578+0000 7f8bdeb8d640 1 --2- 172.21.15.16:0/1499294221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd80b6a90 0x7f8bd8169760 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:27.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.578+0000 7f8bdeb8d640 1 -- 172.21.15.16:0/1499294221 >> 172.21.15.16:0/1499294221 conn(0x7f8bd808bfb0 msgr2=0x7f8bd80d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:27.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.578+0000 7f8bdeb8d640 1 -- 172.21.15.16:0/1499294221 shutdown_connections 2024-09-19T07:49:27.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.578+0000 7f8bdeb8d640 1 -- 172.21.15.16:0/1499294221 wait complete. 2024-09-19T07:49:27.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.579+0000 7f8bdeb8d640 1 Processor -- start 2024-09-19T07:49:27.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.579+0000 7f8bdeb8d640 1 -- start start 2024-09-19T07:49:27.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.580+0000 7f8bdeb8d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8bd80b6a90 0x7f8bd81577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:27.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.580+0000 7f8bdeb8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd80d4f60 0x7f8bd81f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:27.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.580+0000 7f8bdeb8d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bd8161cb0 0x7f8bd81586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:27.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.580+0000 7f8bdeb8d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd80b5e60 con 0x7f8bd80d4f60 2024-09-19T07:49:27.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.580+0000 7f8bdeb8d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd80cfcf0 con 0x7f8bd8161cb0 2024-09-19T07:49:27.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.580+0000 7f8bdeb8d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8bd80d0500 con 0x7f8bd80b6a90 2024-09-19T07:49:27.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.580+0000 7f8bd7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd80d4f60 0x7f8bd81f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:27.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.580+0000 7f8bd7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd80d4f60 0x7f8bd81f4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55826/0 (socket says 172.21.15.16:55826) 2024-09-19T07:49:27.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.580+0000 7f8bd7fff640 1 -- 172.21.15.16:0/3510317784 learned_addr learned my addr 172.21.15.16:0/3510317784 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:27.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.580+0000 7f8bdd12a640 1 --2- 172.21.15.16:0/3510317784 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bd8161cb0 0x7f8bd81586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.580+0000 7f8bdc929640 1 --2- 172.21.15.16:0/3510317784 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8bd80b6a90 0x7f8bd81577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.580+0000 7f8bd7fff640 1 -- 172.21.15.16:0/3510317784 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8bd80b6a90 msgr2=0x7f8bd81577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.580+0000 7f8bd7fff640 1 --2- 172.21.15.16:0/3510317784 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8bd80b6a90 0x7f8bd81577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.581+0000 7f8bd7fff640 1 -- 172.21.15.16:0/3510317784 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bd8161cb0 msgr2=0x7f8bd81586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.581+0000 7f8bd7fff640 1 --2- 172.21.15.16:0/3510317784 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bd8161cb0 0x7f8bd81586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.581+0000 7f8bd7fff640 1 -- 172.21.15.16:0/3510317784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd8157d00 con 0x7f8bd80d4f60 2024-09-19T07:49:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.581+0000 7f8bd7fff640 1 --2- 172.21.15.16:0/3510317784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd80d4f60 0x7f8bd81f4660 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00c430 tx=0x7f8bcc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.581+0000 7f8bd57fa640 1 -- 172.21.15.16:0/3510317784 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc011790 con 0x7f8bd80d4f60 2024-09-19T07:49:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.581+0000 7f8bd57fa640 1 -- 172.21.15.16:0/3510317784 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bcc017070 con 0x7f8bd80d4f60 2024-09-19T07:49:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.581+0000 7f8bd57fa640 1 -- 172.21.15.16:0/3510317784 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc011bb0 con 0x7f8bd80d4f60 2024-09-19T07:49:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.581+0000 7f8bdeb8d640 1 -- 172.21.15.16:0/3510317784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd80c8ec0 con 0x7f8bd80d4f60 2024-09-19T07:49:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.581+0000 7f8bdeb8d640 1 -- 172.21.15.16:0/3510317784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd80c9480 con 0x7f8bd80d4f60 2024-09-19T07:49:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.582+0000 7f8bdeb8d640 1 -- 172.21.15.16:0/3510317784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd8162090 con 0x7f8bd80d4f60 2024-09-19T07:49:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.583+0000 7f8bd57fa640 1 -- 172.21.15.16:0/3510317784 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8bcc01c460 con 0x7f8bd80d4f60 2024-09-19T07:49:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.583+0000 7f8bd57fa640 1 --2- 172.21.15.16:0/3510317784 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8bc406c2b0 0x7f8bc406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.583+0000 7f8bd57fa640 1 -- 172.21.15.16:0/3510317784 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f8bcc08b4b0 con 0x7f8bd80d4f60 2024-09-19T07:49:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.583+0000 7f8bdc929640 1 --2- 172.21.15.16:0/3510317784 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8bc406c2b0 0x7f8bc406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.584+0000 7f8bdc929640 1 --2- 172.21.15.16:0/3510317784 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8bc406c2b0 0x7f8bc406e770 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8bd80cfab0 tx=0x7f8bc8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.588+0000 7f8bd57fa640 1 -- 172.21.15.16:0/3510317784 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8bcc059a80 con 0x7f8bd80d4f60 2024-09-19T07:49:27.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.777+0000 7f8bdeb8d640 1 -- 172.21.15.16:0/3510317784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8bd808c6d0 con 0x7f8bd80d4f60 2024-09-19T07:49:27.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.778+0000 7f8bd57fa640 1 -- 172.21.15.16:0/3510317784 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) ==== 80+0+0 (secure 0 0 0) 0x7f8bcc059380 con 0x7f8bd80d4f60 2024-09-19T07:49:27.778 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:49:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.780+0000 7f8bdeb8d640 1 -- 172.21.15.16:0/3510317784 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8bc406c2b0 msgr2=0x7f8bc406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.780+0000 7f8bdeb8d640 1 --2- 172.21.15.16:0/3510317784 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8bc406c2b0 0x7f8bc406e770 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8bd80cfab0 tx=0x7f8bc8009040 comp rx=0 tx=0).stop 2024-09-19T07:49:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.780+0000 7f8bdeb8d640 1 -- 172.21.15.16:0/3510317784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd80d4f60 msgr2=0x7f8bd81f4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.780+0000 7f8bdeb8d640 1 --2- 172.21.15.16:0/3510317784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd80d4f60 0x7f8bd81f4660 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00c430 tx=0x7f8bcc00c8f0 comp rx=0 tx=0).stop 2024-09-19T07:49:27.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.780+0000 7f8bdeb8d640 1 -- 172.21.15.16:0/3510317784 shutdown_connections 2024-09-19T07:49:27.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.780+0000 7f8bdeb8d640 1 --2- 172.21.15.16:0/3510317784 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8bc406c2b0 0x7f8bc406e770 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:27.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.780+0000 7f8bdeb8d640 1 --2- 172.21.15.16:0/3510317784 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bd8161cb0 0x7f8bd81586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:27.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.780+0000 7f8bdeb8d640 1 --2- 172.21.15.16:0/3510317784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd80d4f60 0x7f8bd81f4660 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:27.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.780+0000 7f8bdeb8d640 1 --2- 172.21.15.16:0/3510317784 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8bd80b6a90 0x7f8bd81577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:27.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.780+0000 7f8bdeb8d640 1 -- 172.21.15.16:0/3510317784 >> 172.21.15.16:0/3510317784 conn(0x7f8bd808bfb0 msgr2=0x7f8bd80d0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:27.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.780+0000 7f8bdeb8d640 1 -- 172.21.15.16:0/3510317784 shutdown_connections 2024-09-19T07:49:27.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:27.780+0000 7f8bdeb8d640 1 -- 172.21.15.16:0/3510317784 wait complete. 2024-09-19T07:49:27.791 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-19T07:49:27.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:27.791 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:27.815 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[56235]: fuse finished with error 0 and tester_r 0 2024-09-19T07:49:27.818 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:49:27.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:27.819 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:49:27.988 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:49:27.988 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:49:27.988 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:49:28.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:49:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.131+0000 7f4e7fbbd640 1 Processor -- start 2024-09-19T07:49:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.131+0000 7f4e7fbbd640 1 -- start start 2024-09-19T07:49:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.132+0000 7f4e7fbbd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e78161cb0 0x7f4e780d4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.132+0000 7f4e7fbbd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e780b4130 0x7f4e780d2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.132+0000 7f4e7fbbd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e780b5bf0 con 0x7f4e780b3cf0 2024-09-19T07:49:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.132+0000 7f4e7fbbd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f4e780cfcf0 con 0x7f4e78161cb0 2024-09-19T07:49:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.132+0000 7f4e7fbbd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4e780cff40 con 0x7f4e780b4130 2024-09-19T07:49:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.132+0000 7f4e7d959640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e78161cb0 0x7f4e780d4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.132+0000 7f4e7d959640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e78161cb0 0x7f4e780d4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55834/0 (socket says 172.21.15.16:55834) 2024-09-19T07:49:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.132+0000 7f4e7d959640 1 -- 172.21.15.16:0/2896555076 learned_addr learned my addr 172.21.15.16:0/2896555076 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.133+0000 7f4e7e15a640 1 --2- 172.21.15.16:0/2896555076 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e780b4130 0x7f4e780d2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.133+0000 7f4e7d959640 1 -- 172.21.15.16:0/2896555076 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e780b4130 msgr2=0x7f4e780d2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.133+0000 7f4e7d959640 1 --2- 172.21.15.16:0/2896555076 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e780b4130 0x7f4e780d2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.133+0000 7f4e7d959640 1 -- 172.21.15.16:0/2896555076 >> v1:172.21.15.146:6789/0 conn(0x7f4e780b3cf0 legacy=0x7f4e780b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.133+0000 7f4e7d959640 1 -- 172.21.15.16:0/2896555076 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e780d0220 con 0x7f4e78161cb0 2024-09-19T07:49:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.133+0000 7f4e7d959640 1 --2- 172.21.15.16:0/2896555076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e78161cb0 0x7f4e780d4f60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009e50 tx=0x7f4e6802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b80afc4cd3bce9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.133+0000 7f4e7c957640 1 -- 172.21.15.16:0/2896555076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68036070 con 0x7f4e78161cb0 2024-09-19T07:49:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.134+0000 7f4e7c957640 1 -- 172.21.15.16:0/2896555076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e68031600 con 0x7f4e78161cb0 2024-09-19T07:49:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.134+0000 7f4e7c957640 1 -- 172.21.15.16:0/2896555076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e680319d0 con 0x7f4e78161cb0 2024-09-19T07:49:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.134+0000 7f4e7fbbd640 1 -- 172.21.15.16:0/2896555076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e78161cb0 msgr2=0x7f4e780d4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.134+0000 7f4e7fbbd640 1 --2- 172.21.15.16:0/2896555076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e78161cb0 0x7f4e780d4f60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009e50 tx=0x7f4e6802a680 comp rx=0 tx=0).stop 2024-09-19T07:49:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.134+0000 7f4e7fbbd640 1 -- 172.21.15.16:0/2896555076 shutdown_connections 2024-09-19T07:49:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.134+0000 7f4e7fbbd640 1 --2- 172.21.15.16:0/2896555076 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e780b4130 0x7f4e780d2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.134+0000 7f4e7fbbd640 1 --2- 172.21.15.16:0/2896555076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e78161cb0 0x7f4e780d4f60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.134+0000 7f4e7fbbd640 1 -- 172.21.15.16:0/2896555076 >> 172.21.15.16:0/2896555076 conn(0x7f4e7808bfb0 msgr2=0x7f4e780d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.134+0000 7f4e7fbbd640 1 -- 172.21.15.16:0/2896555076 shutdown_connections 2024-09-19T07:49:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.134+0000 7f4e7fbbd640 1 -- 172.21.15.16:0/2896555076 wait complete. 2024-09-19T07:49:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.135+0000 7f4e7fbbd640 1 Processor -- start 2024-09-19T07:49:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.135+0000 7f4e7fbbd640 1 -- start start 2024-09-19T07:49:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.136+0000 7f4e7fbbd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e780b3cf0 0x7f4e781579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.136+0000 7f4e7fbbd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e780b4130 0x7f4e780c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.136+0000 7f4e7fbbd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e78161cb0 0x7f4e7815b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.136+0000 7f4e7fbbd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e78162090 con 0x7f4e780b3cf0 2024-09-19T07:49:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.136+0000 7f4e7fbbd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4e780d0500 con 0x7f4e780b4130 2024-09-19T07:49:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.136+0000 7f4e7fbbd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4e78162210 con 0x7f4e78161cb0 2024-09-19T07:49:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.136+0000 7f4e7d959640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e780b3cf0 0x7f4e781579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.136+0000 7f4e7d959640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e780b3cf0 0x7f4e781579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55838/0 (socket says 172.21.15.16:55838) 2024-09-19T07:49:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.136+0000 7f4e7d959640 1 -- 172.21.15.16:0/2410677367 learned_addr learned my addr 172.21.15.16:0/2410677367 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.136+0000 7f4e7e15a640 1 --2- 172.21.15.16:0/2410677367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e78161cb0 0x7f4e7815b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.136+0000 7f4e7d158640 1 --2- 172.21.15.16:0/2410677367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e780b4130 0x7f4e780c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.137+0000 7f4e7d959640 1 -- 172.21.15.16:0/2410677367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e78161cb0 msgr2=0x7f4e7815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.137+0000 7f4e7d959640 1 --2- 172.21.15.16:0/2410677367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e78161cb0 0x7f4e7815b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.137+0000 7f4e7d959640 1 -- 172.21.15.16:0/2410677367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e780b4130 msgr2=0x7f4e780c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.137+0000 7f4e7d959640 1 --2- 172.21.15.16:0/2410677367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e780b4130 0x7f4e780c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.137+0000 7f4e7d959640 1 -- 172.21.15.16:0/2410677367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e780c9b40 con 0x7f4e780b3cf0 2024-09-19T07:49:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.137+0000 7f4e7d959640 1 --2- 172.21.15.16:0/2410677367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e780b3cf0 0x7f4e781579d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4e68034040 tx=0x7f4e68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.137+0000 7f4e767fc640 1 -- 172.21.15.16:0/2410677367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6803f020 con 0x7f4e780b3cf0 2024-09-19T07:49:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.137+0000 7f4e767fc640 1 -- 172.21.15.16:0/2410677367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e68037040 con 0x7f4e780b3cf0 2024-09-19T07:49:28.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.137+0000 7f4e7fbbd640 1 -- 172.21.15.16:0/2410677367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e780c9e20 con 0x7f4e780b3cf0 2024-09-19T07:49:28.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.137+0000 7f4e767fc640 1 -- 172.21.15.16:0/2410677367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68036070 con 0x7f4e780b3cf0 2024-09-19T07:49:28.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.137+0000 7f4e7fbbd640 1 -- 172.21.15.16:0/2410677367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e780ca3e0 con 0x7f4e780b3cf0 2024-09-19T07:49:28.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.138+0000 7f4e7fbbd640 1 -- 172.21.15.16:0/2410677367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e780d4f60 con 0x7f4e780b3cf0 2024-09-19T07:49:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.138+0000 7f4e767fc640 1 -- 172.21.15.16:0/2410677367 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4e68042020 con 0x7f4e780b3cf0 2024-09-19T07:49:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.139+0000 7f4e767fc640 1 --2- 172.21.15.16:0/2410677367 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4e6406c2b0 0x7f4e6406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.139+0000 7f4e767fc640 1 -- 172.21.15.16:0/2410677367 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f4e680ab8f0 con 0x7f4e780b3cf0 2024-09-19T07:49:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.139+0000 7f4e7d158640 1 --2- 172.21.15.16:0/2410677367 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4e6406c2b0 0x7f4e6406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.140+0000 7f4e7d158640 1 --2- 172.21.15.16:0/2410677367 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4e6406c2b0 0x7f4e6406e770 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4e7815ad80 tx=0x7f4e6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.143+0000 7f4e767fc640 1 -- 172.21.15.16:0/2410677367 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4e68049030 con 0x7f4e780b3cf0 2024-09-19T07:49:28.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.330+0000 7f4e7fbbd640 1 -- 172.21.15.16:0/2410677367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4e780b1c90 con 0x7f4e780b3cf0 2024-09-19T07:49:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.330+0000 7f4e767fc640 1 -- 172.21.15.16:0/2410677367 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) ==== 80+0+0 (secure 0 0 0) 0x7f4e68079910 con 0x7f4e780b3cf0 2024-09-19T07:49:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:49:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.333+0000 7f4e7fbbd640 1 -- 172.21.15.16:0/2410677367 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4e6406c2b0 msgr2=0x7f4e6406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.333+0000 7f4e7fbbd640 1 --2- 172.21.15.16:0/2410677367 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4e6406c2b0 0x7f4e6406e770 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4e7815ad80 tx=0x7f4e6c009040 comp rx=0 tx=0).stop 2024-09-19T07:49:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.333+0000 7f4e7fbbd640 1 -- 172.21.15.16:0/2410677367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e780b3cf0 msgr2=0x7f4e781579d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.333+0000 7f4e7fbbd640 1 --2- 172.21.15.16:0/2410677367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e780b3cf0 0x7f4e781579d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4e68034040 tx=0x7f4e68004a40 comp rx=0 tx=0).stop 2024-09-19T07:49:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.333+0000 7f4e7fbbd640 1 -- 172.21.15.16:0/2410677367 shutdown_connections 2024-09-19T07:49:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.333+0000 7f4e7fbbd640 1 --2- 172.21.15.16:0/2410677367 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4e6406c2b0 0x7f4e6406e770 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.333+0000 7f4e7fbbd640 1 --2- 172.21.15.16:0/2410677367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e78161cb0 0x7f4e7815b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.333+0000 7f4e7fbbd640 1 --2- 172.21.15.16:0/2410677367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e780b4130 0x7f4e780c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.333+0000 7f4e7fbbd640 1 --2- 172.21.15.16:0/2410677367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e780b3cf0 0x7f4e781579d0 secure :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4e68034040 tx=0x7f4e68004a40 comp rx=0 tx=0).stop 2024-09-19T07:49:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.333+0000 7f4e7fbbd640 1 -- 172.21.15.16:0/2410677367 >> 172.21.15.16:0/2410677367 conn(0x7f4e7808bfb0 msgr2=0x7f4e7806e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.333+0000 7f4e7fbbd640 1 -- 172.21.15.16:0/2410677367 shutdown_connections 2024-09-19T07:49:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.333+0000 7f4e7fbbd640 1 -- 172.21.15.16:0/2410677367 wait complete. 2024-09-19T07:49:28.344 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2024-09-19T07:49:28.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:28.344 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T07:49:28.373 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[45545]: fuse finished with error 0 and tester_r 0 2024-09-19T07:49:28.375 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:49:28.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:28.375 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T07:49:28.538 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:49:28.538 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:49:28.538 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:49:28.555 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:49:28.555 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:49:28.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:28.555 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T07:49:28.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:49:28.613 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T07:49:28.614 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:49:28.614 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:49:28.614 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:49:28.633 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:49:28.633 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:49:28.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:28.634 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T07:49:28.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:49:28.693 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T07:49:28.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:49:28.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.796+0000 7faf6b5e8640 1 Processor -- start 2024-09-19T07:49:28.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.796+0000 7faf6b5e8640 1 -- start start 2024-09-19T07:49:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.797+0000 7faf6b5e8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf640d3650 0x7faf641691a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.797+0000 7faf6b5e8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf640cf970 con 0x7faf640b7350 2024-09-19T07:49:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.797+0000 7faf6b5e8640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf640b1d30 con 0x7faf6415e5f0 2024-09-19T07:49:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.797+0000 7faf6b5e8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faf640b1ae0 con 0x7faf640d3650 2024-09-19T07:49:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf68b83640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7faf640b7350 0x7faf640cf260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56200/0 (socket says 172.21.15.16:56200) 2024-09-19T07:49:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf68b83640 1 -- 172.21.15.16:0/424884073 learned_addr learned my addr 172.21.15.16:0/424884073 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf69b85640 1 --2- 172.21.15.16:0/424884073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf640d3650 0x7faf641691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf63fff640 1 -- 172.21.15.16:0/424884073 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2189956679 0 0) 0x7faf640cf970 con 0x7faf640b7350 2024-09-19T07:49:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf63fff640 1 -- 172.21.15.16:0/424884073 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf4c003520 con 0x7faf640b7350 2024-09-19T07:49:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf63fff640 1 -- 172.21.15.16:0/424884073 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1207195212 0 0) 0x7faf4c003520 con 0x7faf640b7350 2024-09-19T07:49:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf63fff640 1 -- 172.21.15.16:0/424884073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf640d3650 msgr2=0x7faf641691a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf63fff640 1 --2- 172.21.15.16:0/424884073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf640d3650 0x7faf641691a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf63fff640 1 -- 172.21.15.16:0/424884073 >> v1:172.21.15.146:6789/0 conn(0x7faf6415e5f0 legacy=0x7faf640ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf63fff640 1 -- 172.21.15.16:0/424884073 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf640b2010 con 0x7faf640b7350 2024-09-19T07:49:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf63fff640 1 -- 172.21.15.16:0/424884073 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7faf580017f0 con 0x7faf640b7350 2024-09-19T07:49:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf63fff640 1 -- 172.21.15.16:0/424884073 >> v1:172.21.15.16:6789/0 conn(0x7faf640b7350 legacy=0x7faf640cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf63fff640 1 --2- 172.21.15.16:0/424884073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf4c0039e0 0x7faf4c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf63fff640 1 --2- 172.21.15.16:0/424884073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf640d3650 0x7faf4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf63fff640 1 --2- 172.21.15.16:0/424884073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf4c004d40 0x7faf4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf63fff640 1 -- 172.21.15.16:0/424884073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf640b1ae0 con 0x7faf4c0039e0 2024-09-19T07:49:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf63fff640 1 -- 172.21.15.16:0/424884073 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c00ba70 con 0x7faf4c004d40 2024-09-19T07:49:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf63fff640 1 -- 172.21.15.16:0/424884073 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faf4c00bc70 con 0x7faf640d3650 2024-09-19T07:49:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf69384640 1 --2- 172.21.15.16:0/424884073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf4c0039e0 0x7faf4c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf68b83640 1 --2- 172.21.15.16:0/424884073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf640d3650 0x7faf4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.798+0000 7faf69b85640 1 --2- 172.21.15.16:0/424884073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf4c004d40 0x7faf4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.799+0000 7faf69384640 1 -- 172.21.15.16:0/424884073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf640d3650 msgr2=0x7faf4c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.799+0000 7faf69384640 1 --2- 172.21.15.16:0/424884073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf640d3650 0x7faf4c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.799+0000 7faf69384640 1 -- 172.21.15.16:0/424884073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf4c004d40 msgr2=0x7faf4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.799+0000 7faf69384640 1 --2- 172.21.15.16:0/424884073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf4c004d40 0x7faf4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.799+0000 7faf69384640 1 -- 172.21.15.16:0/424884073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4c003520 con 0x7faf4c0039e0 2024-09-19T07:49:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.799+0000 7faf69384640 1 --2- 172.21.15.16:0/424884073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf4c0039e0 0x7faf4c003e00 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7faf54001720 tx=0x7faf4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690460e318d9de95 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.799+0000 7faf63fff640 1 -- 172.21.15.16:0/424884073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54001370 con 0x7faf4c0039e0 2024-09-19T07:49:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.799+0000 7faf63fff640 1 -- 172.21.15.16:0/424884073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf540052d0 con 0x7faf4c0039e0 2024-09-19T07:49:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.799+0000 7faf63fff640 1 -- 172.21.15.16:0/424884073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5402abc0 con 0x7faf4c0039e0 2024-09-19T07:49:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.800+0000 7faf6b5e8640 1 -- 172.21.15.16:0/424884073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf4c0039e0 msgr2=0x7faf4c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.800+0000 7faf6b5e8640 1 --2- 172.21.15.16:0/424884073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf4c0039e0 0x7faf4c003e00 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7faf54001720 tx=0x7faf4c005830 comp rx=0 tx=0).stop 2024-09-19T07:49:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.800+0000 7faf69b85640 1 -- 172.21.15.16:0/424884073 reap_dead start 2024-09-19T07:49:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.800+0000 7faf6b5e8640 1 -- 172.21.15.16:0/424884073 shutdown_connections 2024-09-19T07:49:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.800+0000 7faf6b5e8640 1 -- 172.21.15.16:0/424884073 >> 172.21.15.16:0/424884073 conn(0x7faf6408bfb0 msgr2=0x7faf640d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.800+0000 7faf6b5e8640 1 -- 172.21.15.16:0/424884073 shutdown_connections 2024-09-19T07:49:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.800+0000 7faf6b5e8640 1 -- 172.21.15.16:0/424884073 wait complete. 2024-09-19T07:49:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.801+0000 7faf6b5e8640 1 Processor -- start 2024-09-19T07:49:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.801+0000 7faf6b5e8640 1 -- start start 2024-09-19T07:49:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.802+0000 7faf6b5e8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf4c0039e0 0x7faf640cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.802+0000 7faf6b5e8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf641557d0 0x7faf64153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.802+0000 7faf6b5e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf641606b0 0x7faf64160b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.802+0000 7faf6b5e8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf6415e9d0 con 0x7faf641606b0 2024-09-19T07:49:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.802+0000 7faf6b5e8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faf640b2660 con 0x7faf4c0039e0 2024-09-19T07:49:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.802+0000 7faf6b5e8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faf640d5190 con 0x7faf641557d0 2024-09-19T07:49:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.802+0000 7faf69b85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf641606b0 0x7faf64160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:28.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.802+0000 7faf69b85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf641606b0 0x7faf64160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55860/0 (socket says 172.21.15.16:55860) 2024-09-19T07:49:28.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.802+0000 7faf69b85640 1 -- 172.21.15.16:0/636145890 learned_addr learned my addr 172.21.15.16:0/636145890 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:28.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.802+0000 7faf68b83640 1 --2- 172.21.15.16:0/636145890 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf641557d0 0x7faf64153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:28.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.802+0000 7faf69384640 1 --2- 172.21.15.16:0/636145890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf4c0039e0 0x7faf640cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:28.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.802+0000 7faf69b85640 1 -- 172.21.15.16:0/636145890 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf641557d0 msgr2=0x7faf64153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:28.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.802+0000 7faf69b85640 1 --2- 172.21.15.16:0/636145890 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf641557d0 0x7faf64153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:28.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.803+0000 7faf69b85640 1 -- 172.21.15.16:0/636145890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf4c0039e0 msgr2=0x7faf640cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:28.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.803+0000 7faf69b85640 1 --2- 172.21.15.16:0/636145890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf4c0039e0 0x7faf640cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:28.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.803+0000 7faf69b85640 1 -- 172.21.15.16:0/636145890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf640cc4f0 con 0x7faf641606b0 2024-09-19T07:49:28.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.803+0000 7faf69b85640 1 --2- 172.21.15.16:0/636145890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf641606b0 0x7faf64160b30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7faf641620c0 tx=0x7faf5c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:28.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.803+0000 7faf61ffb640 1 -- 172.21.15.16:0/636145890 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c00fe10 con 0x7faf641606b0 2024-09-19T07:49:28.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.803+0000 7faf61ffb640 1 -- 172.21.15.16:0/636145890 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf5c015430 con 0x7faf641606b0 2024-09-19T07:49:28.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.803+0000 7faf6b5e8640 1 -- 172.21.15.16:0/636145890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf641568f0 con 0x7faf641606b0 2024-09-19T07:49:28.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.803+0000 7faf61ffb640 1 -- 172.21.15.16:0/636145890 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c015820 con 0x7faf641606b0 2024-09-19T07:49:28.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.803+0000 7faf6b5e8640 1 -- 172.21.15.16:0/636145890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf64156050 con 0x7faf641606b0 2024-09-19T07:49:28.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.804+0000 7faf6b5e8640 1 -- 172.21.15.16:0/636145890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf640ceb50 con 0x7faf641606b0 2024-09-19T07:49:28.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.804+0000 7faf61ffb640 1 -- 172.21.15.16:0/636145890 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7faf5c0159c0 con 0x7faf641606b0 2024-09-19T07:49:28.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.805+0000 7faf61ffb640 1 --2- 172.21.15.16:0/636145890 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faf5006c2e0 0x7faf5006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:28.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.805+0000 7faf61ffb640 1 -- 172.21.15.16:0/636145890 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7faf5c08c130 con 0x7faf641606b0 2024-09-19T07:49:28.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.805+0000 7faf69384640 1 --2- 172.21.15.16:0/636145890 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faf5006c2e0 0x7faf5006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:28.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.806+0000 7faf69384640 1 --2- 172.21.15.16:0/636145890 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faf5006c2e0 0x7faf5006e7a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faf54032000 tx=0x7faf5402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:28.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:28.809+0000 7faf61ffb640 1 -- 172.21.15.16:0/636145890 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7faf5c019030 con 0x7faf641606b0 2024-09-19T07:49:29.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.019+0000 7faf6b5e8640 1 -- 172.21.15.16:0/636145890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf6406dc30 con 0x7faf641606b0 2024-09-19T07:49:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.020+0000 7faf61ffb640 1 -- 172.21.15.16:0/636145890 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5227 (secure 0 0 0) 0x7faf5c010040 con 0x7faf641606b0 2024-09-19T07:49:29.020 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:29.020 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":33,"btime":"2024-09-19T07:48:52:612055+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2104039348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2104039348},{"type":"v1","addr":"172.21.15.16:6835","nonce":2104039348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/1109733621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1109733621},{"type":"v1","addr":"172.21.15.146:6836","nonce":1109733621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3602092462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3602092462},{"type":"v1","addr":"172.21.15.146:6837","nonce":3602092462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:48:49.954706+0000","modified":"2024-09-19T07:48:51.957927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5751},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5751":{"gid":5751,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6834/418904597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":418904597},{"type":"v1","addr":"172.21.15.16:6834","nonce":418904597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5751,"qdb_cluster":[5751]},"id":4}]} 2024-09-19T07:49:29.021 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 33 2024-09-19T07:49:29.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.022+0000 7faf6b5e8640 1 -- 172.21.15.16:0/636145890 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faf5006c2e0 msgr2=0x7faf5006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.022+0000 7faf6b5e8640 1 --2- 172.21.15.16:0/636145890 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faf5006c2e0 0x7faf5006e7a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faf54032000 tx=0x7faf5402e000 comp rx=0 tx=0).stop 2024-09-19T07:49:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.022+0000 7faf6b5e8640 1 -- 172.21.15.16:0/636145890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf641606b0 msgr2=0x7faf64160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.022+0000 7faf6b5e8640 1 --2- 172.21.15.16:0/636145890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf641606b0 0x7faf64160b30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7faf641620c0 tx=0x7faf5c00f580 comp rx=0 tx=0).stop 2024-09-19T07:49:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.022+0000 7faf6b5e8640 1 -- 172.21.15.16:0/636145890 shutdown_connections 2024-09-19T07:49:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.022+0000 7faf6b5e8640 1 --2- 172.21.15.16:0/636145890 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faf5006c2e0 0x7faf5006e7a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.022+0000 7faf6b5e8640 1 --2- 172.21.15.16:0/636145890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf641606b0 0x7faf64160b30 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.022+0000 7faf6b5e8640 1 --2- 172.21.15.16:0/636145890 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf641557d0 0x7faf64153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.022+0000 7faf6b5e8640 1 --2- 172.21.15.16:0/636145890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf4c0039e0 0x7faf640cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.022+0000 7faf6b5e8640 1 -- 172.21.15.16:0/636145890 >> 172.21.15.16:0/636145890 conn(0x7faf6408bfb0 msgr2=0x7faf640b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.022+0000 7faf6b5e8640 1 -- 172.21.15.16:0/636145890 shutdown_connections 2024-09-19T07:49:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.022+0000 7faf6b5e8640 1 -- 172.21.15.16:0/636145890 wait complete. 2024-09-19T07:49:29.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:49:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.135+0000 7f8f25b0a640 1 Processor -- start 2024-09-19T07:49:29.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.135+0000 7f8f25b0a640 1 -- start start 2024-09-19T07:49:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.136+0000 7f8f25b0a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f200cdcc0 0x7f8f200ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.136+0000 7f8f25b0a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8f200b3b90 0x7f8f200b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.136+0000 7f8f25b0a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f2006ea40 con 0x7f8f20155700 2024-09-19T07:49:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.136+0000 7f8f25b0a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f8f200ce660 con 0x7f8f200b3b90 2024-09-19T07:49:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.136+0000 7f8f25b0a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8f200b3140 con 0x7f8f200cdcc0 2024-09-19T07:49:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.136+0000 7f8f1ffff640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8f200b3b90 0x7f8f200b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.136+0000 7f8f1effd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f200cdcc0 0x7f8f200ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.136+0000 7f8f1effd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f200cdcc0 0x7f8f200ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:60404/0 (socket says 172.21.15.16:60404) 2024-09-19T07:49:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.136+0000 7f8f1ffff640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8f200b3b90 0x7f8f200b4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:39648/0 (socket says 172.21.15.16:39648) 2024-09-19T07:49:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.136+0000 7f8f1effd640 1 -- 172.21.15.16:0/1072844384 learned_addr learned my addr 172.21.15.16:0/1072844384 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:29.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.136+0000 7f8f1f7fe640 1 --1- 172.21.15.16:0/1072844384 >> v1:172.21.15.146:6789/0 conn(0x7f8f20155700 0x7f8f2006df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:53516/0 (socket says 172.21.15.16:53516) 2024-09-19T07:49:29.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.137+0000 7f8f1effd640 1 -- 172.21.15.16:0/1072844384 >> v2:172.21.15.146:3300/0 conn(0x7f8f200b3b90 msgr2=0x7f8f200b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:29.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.137+0000 7f8f1effd640 1 --2- 172.21.15.16:0/1072844384 >> v2:172.21.15.146:3300/0 conn(0x7f8f200b3b90 0x7f8f200b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.137+0000 7f8f1effd640 1 -- 172.21.15.16:0/1072844384 >> v1:172.21.15.146:6789/0 conn(0x7f8f20155700 legacy=0x7f8f2006df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:29.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.137+0000 7f8f1effd640 1 -- 172.21.15.16:0/1072844384 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2006ed20 con 0x7f8f200cdcc0 2024-09-19T07:49:29.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.137+0000 7f8f1effd640 1 --2- 172.21.15.16:0/1072844384 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f200cdcc0 0x7f8f200ce120 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8f140098a0 tx=0x7f8f1402a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af8eabf4c13e9371 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:29.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.138+0000 7f8f1e7fc640 1 -- 172.21.15.16:0/1072844384 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14037070 con 0x7f8f200cdcc0 2024-09-19T07:49:29.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.138+0000 7f8f1e7fc640 1 -- 172.21.15.16:0/1072844384 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f14032440 con 0x7f8f200cdcc0 2024-09-19T07:49:29.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.138+0000 7f8f1e7fc640 1 -- 172.21.15.16:0/1072844384 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14032810 con 0x7f8f200cdcc0 2024-09-19T07:49:29.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.138+0000 7f8f25b0a640 1 -- 172.21.15.16:0/1072844384 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f200cdcc0 msgr2=0x7f8f200ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:29.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.138+0000 7f8f25b0a640 1 --2- 172.21.15.16:0/1072844384 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f200cdcc0 0x7f8f200ce120 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8f140098a0 tx=0x7f8f1402a200 comp rx=0 tx=0).stop 2024-09-19T07:49:29.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.138+0000 7f8f25b0a640 1 -- 172.21.15.16:0/1072844384 shutdown_connections 2024-09-19T07:49:29.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.138+0000 7f8f25b0a640 1 --2- 172.21.15.16:0/1072844384 >> v2:172.21.15.146:3300/0 conn(0x7f8f200b3b90 0x7f8f200b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.138+0000 7f8f25b0a640 1 --2- 172.21.15.16:0/1072844384 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f200cdcc0 0x7f8f200ce120 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.138+0000 7f8f25b0a640 1 -- 172.21.15.16:0/1072844384 >> 172.21.15.16:0/1072844384 conn(0x7f8f2006bbd0 msgr2=0x7f8f2006bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:29.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.138+0000 7f8f25b0a640 1 -- 172.21.15.16:0/1072844384 shutdown_connections 2024-09-19T07:49:29.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.138+0000 7f8f25b0a640 1 -- 172.21.15.16:0/1072844384 wait complete. 2024-09-19T07:49:29.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.139+0000 7f8f25b0a640 1 Processor -- start 2024-09-19T07:49:29.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.140+0000 7f8f25b0a640 1 -- start start 2024-09-19T07:49:29.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.140+0000 7f8f25b0a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f200b3b90 0x7f8f201f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:29.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.140+0000 7f8f25b0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f200cdcc0 0x7f8f2015cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:29.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.140+0000 7f8f25b0a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f20155700 0x7f8f2015aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:29.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.140+0000 7f8f25b0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2015b9a0 con 0x7f8f200cdcc0 2024-09-19T07:49:29.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.140+0000 7f8f25b0a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8f200b3700 con 0x7f8f20155700 2024-09-19T07:49:29.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.140+0000 7f8f25b0a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8f20155ae0 con 0x7f8f200b3b90 2024-09-19T07:49:29.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.140+0000 7f8f1effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f200cdcc0 0x7f8f2015cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:29.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.140+0000 7f8f1effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f200cdcc0 0x7f8f2015cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55874/0 (socket says 172.21.15.16:55874) 2024-09-19T07:49:29.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.140+0000 7f8f1effd640 1 -- 172.21.15.16:0/2835375039 learned_addr learned my addr 172.21.15.16:0/2835375039 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:29.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.141+0000 7f8f1f7fe640 1 --2- 172.21.15.16:0/2835375039 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f200b3b90 0x7f8f201f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:29.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.141+0000 7f8f1ffff640 1 --2- 172.21.15.16:0/2835375039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f20155700 0x7f8f2015aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:29.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.141+0000 7f8f1effd640 1 -- 172.21.15.16:0/2835375039 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f200b3b90 msgr2=0x7f8f201f44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:29.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.141+0000 7f8f1effd640 1 --2- 172.21.15.16:0/2835375039 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f200b3b90 0x7f8f201f44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.141+0000 7f8f1effd640 1 -- 172.21.15.16:0/2835375039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f20155700 msgr2=0x7f8f2015aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:29.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.141+0000 7f8f1effd640 1 --2- 172.21.15.16:0/2835375039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f20155700 0x7f8f2015aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.141+0000 7f8f1effd640 1 -- 172.21.15.16:0/2835375039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f20073080 con 0x7f8f200cdcc0 2024-09-19T07:49:29.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.141+0000 7f8f1f7fe640 1 --2- 172.21.15.16:0/2835375039 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f200b3b90 0x7f8f201f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:29.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.141+0000 7f8f1effd640 1 --2- 172.21.15.16:0/2835375039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f200cdcc0 0x7f8f2015cb10 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8f14000c00 tx=0x7f8f140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:29.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.141+0000 7f8f07fff640 1 -- 172.21.15.16:0/2835375039 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14037040 con 0x7f8f200cdcc0 2024-09-19T07:49:29.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.141+0000 7f8f07fff640 1 -- 172.21.15.16:0/2835375039 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f14042070 con 0x7f8f200cdcc0 2024-09-19T07:49:29.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.141+0000 7f8f25b0a640 1 -- 172.21.15.16:0/2835375039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f20073360 con 0x7f8f200cdcc0 2024-09-19T07:49:29.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.141+0000 7f8f07fff640 1 -- 172.21.15.16:0/2835375039 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14030880 con 0x7f8f200cdcc0 2024-09-19T07:49:29.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.141+0000 7f8f25b0a640 1 -- 172.21.15.16:0/2835375039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f20073920 con 0x7f8f200cdcc0 2024-09-19T07:49:29.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.142+0000 7f8f25b0a640 1 -- 172.21.15.16:0/2835375039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f200ce120 con 0x7f8f200cdcc0 2024-09-19T07:49:29.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.143+0000 7f8f07fff640 1 -- 172.21.15.16:0/2835375039 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8f1402ad10 con 0x7f8f200cdcc0 2024-09-19T07:49:29.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.143+0000 7f8f07fff640 1 --2- 172.21.15.16:0/2835375039 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8f0c06c2b0 0x7f8f0c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:29.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.143+0000 7f8f07fff640 1 -- 172.21.15.16:0/2835375039 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f8f140aba50 con 0x7f8f200cdcc0 2024-09-19T07:49:29.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.143+0000 7f8f1f7fe640 1 --2- 172.21.15.16:0/2835375039 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8f0c06c2b0 0x7f8f0c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:29.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.144+0000 7f8f1f7fe640 1 --2- 172.21.15.16:0/2835375039 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8f0c06c2b0 0x7f8f0c06e770 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8f10007fd0 tx=0x7f8f10007d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:29.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.147+0000 7f8f07fff640 1 -- 172.21.15.16:0/2835375039 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8f1407a030 con 0x7f8f200cdcc0 2024-09-19T07:49:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.357+0000 7f8f25b0a640 1 -- 172.21.15.16:0/2835375039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f2006f7c0 con 0x7f8f200cdcc0 2024-09-19T07:49:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.358+0000 7f8f07fff640 1 -- 172.21.15.16:0/2835375039 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5227 (secure 0 0 0) 0x7f8f14079930 con 0x7f8f200cdcc0 2024-09-19T07:49:29.359 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:29.359 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":33,"btime":"2024-09-19T07:48:52:612055+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2104039348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2104039348},{"type":"v1","addr":"172.21.15.16:6835","nonce":2104039348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/1109733621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1109733621},{"type":"v1","addr":"172.21.15.146:6836","nonce":1109733621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3602092462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3602092462},{"type":"v1","addr":"172.21.15.146:6837","nonce":3602092462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:48:49.954706+0000","modified":"2024-09-19T07:48:51.957927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5751},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5751":{"gid":5751,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6834/418904597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":418904597},{"type":"v1","addr":"172.21.15.16:6834","nonce":418904597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5751,"qdb_cluster":[5751]},"id":4}]} 2024-09-19T07:49:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 33 2024-09-19T07:49:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.360+0000 7f8f25b0a640 1 -- 172.21.15.16:0/2835375039 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8f0c06c2b0 msgr2=0x7f8f0c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.360+0000 7f8f25b0a640 1 --2- 172.21.15.16:0/2835375039 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8f0c06c2b0 0x7f8f0c06e770 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8f10007fd0 tx=0x7f8f10007d60 comp rx=0 tx=0).stop 2024-09-19T07:49:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.360+0000 7f8f25b0a640 1 -- 172.21.15.16:0/2835375039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f200cdcc0 msgr2=0x7f8f2015cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.360+0000 7f8f25b0a640 1 --2- 172.21.15.16:0/2835375039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f200cdcc0 0x7f8f2015cb10 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8f14000c00 tx=0x7f8f140046c0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.360+0000 7f8f25b0a640 1 -- 172.21.15.16:0/2835375039 shutdown_connections 2024-09-19T07:49:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.360+0000 7f8f25b0a640 1 --2- 172.21.15.16:0/2835375039 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8f0c06c2b0 0x7f8f0c06e770 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.360+0000 7f8f25b0a640 1 --2- 172.21.15.16:0/2835375039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f20155700 0x7f8f2015aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.360+0000 7f8f25b0a640 1 --2- 172.21.15.16:0/2835375039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f200cdcc0 0x7f8f2015cb10 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.360+0000 7f8f25b0a640 1 --2- 172.21.15.16:0/2835375039 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f200b3b90 0x7f8f201f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.360+0000 7f8f25b0a640 1 -- 172.21.15.16:0/2835375039 >> 172.21.15.16:0/2835375039 conn(0x7f8f2006bbd0 msgr2=0x7f8f200b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:29.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.360+0000 7f8f25b0a640 1 -- 172.21.15.16:0/2835375039 shutdown_connections 2024-09-19T07:49:29.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.360+0000 7f8f25b0a640 1 -- 172.21.15.16:0/2835375039 wait complete. 2024-09-19T07:49:29.371 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:49:29.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.473+0000 7fd626d18640 1 Processor -- start 2024-09-19T07:49:29.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.473+0000 7fd626d18640 1 -- start start 2024-09-19T07:49:29.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.474+0000 7fd626d18640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6200cdcc0 0x7fd6200ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:29.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.474+0000 7fd626d18640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6200b3b90 0x7fd6200b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:29.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.474+0000 7fd626d18640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd62006ea40 con 0x7fd620155700 2024-09-19T07:49:29.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.474+0000 7fd626d18640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd6200ce660 con 0x7fd6200b3b90 2024-09-19T07:49:29.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.474+0000 7fd626d18640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd6200b3140 con 0x7fd6200cdcc0 2024-09-19T07:49:29.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.474+0000 7fd6252b5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6200b3b90 0x7fd6200b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:29.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.474+0000 7fd6252b5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6200b3b90 0x7fd6200b4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55890/0 (socket says 172.21.15.16:55890) 2024-09-19T07:49:29.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.474+0000 7fd6252b5640 1 -- 172.21.15.16:0/3065902984 learned_addr learned my addr 172.21.15.16:0/3065902984 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:29.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.474+0000 7fd61ffff640 1 --2- 172.21.15.16:0/3065902984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6200cdcc0 0x7fd6200ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:29.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.474+0000 7fd6252b5640 1 -- 172.21.15.16:0/3065902984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6200cdcc0 msgr2=0x7fd6200ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:29.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.474+0000 7fd6252b5640 1 --2- 172.21.15.16:0/3065902984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6200cdcc0 0x7fd6200ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.474+0000 7fd6252b5640 1 -- 172.21.15.16:0/3065902984 >> v1:172.21.15.146:6789/0 conn(0x7fd620155700 legacy=0x7fd62006df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:29.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.475+0000 7fd6252b5640 1 -- 172.21.15.16:0/3065902984 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62006ed20 con 0x7fd6200b3b90 2024-09-19T07:49:29.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.475+0000 7fd61ffff640 1 --2- 172.21.15.16:0/3065902984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6200cdcc0 0x7fd6200ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.475+0000 7fd6252b5640 1 --2- 172.21.15.16:0/3065902984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6200b3b90 0x7fd6200b4020 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd618009e80 tx=0x7fd61802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a35c920b978737d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.475+0000 7fd61f7fe640 1 -- 172.21.15.16:0/3065902984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618036070 con 0x7fd6200b3b90 2024-09-19T07:49:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.475+0000 7fd61f7fe640 1 -- 172.21.15.16:0/3065902984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd618031600 con 0x7fd6200b3b90 2024-09-19T07:49:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.475+0000 7fd61f7fe640 1 -- 172.21.15.16:0/3065902984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6180319d0 con 0x7fd6200b3b90 2024-09-19T07:49:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.476+0000 7fd626d18640 1 -- 172.21.15.16:0/3065902984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6200b3b90 msgr2=0x7fd6200b4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.476+0000 7fd626d18640 1 --2- 172.21.15.16:0/3065902984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6200b3b90 0x7fd6200b4020 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd618009e80 tx=0x7fd61802a660 comp rx=0 tx=0).stop 2024-09-19T07:49:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.476+0000 7fd626d18640 1 -- 172.21.15.16:0/3065902984 shutdown_connections 2024-09-19T07:49:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.476+0000 7fd626d18640 1 --2- 172.21.15.16:0/3065902984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6200b3b90 0x7fd6200b4020 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.476+0000 7fd626d18640 1 --2- 172.21.15.16:0/3065902984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6200cdcc0 0x7fd6200ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.476+0000 7fd626d18640 1 -- 172.21.15.16:0/3065902984 >> 172.21.15.16:0/3065902984 conn(0x7fd62006bbd0 msgr2=0x7fd62006bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.476+0000 7fd626d18640 1 -- 172.21.15.16:0/3065902984 shutdown_connections 2024-09-19T07:49:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.476+0000 7fd626d18640 1 -- 172.21.15.16:0/3065902984 wait complete. 2024-09-19T07:49:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.477+0000 7fd626d18640 1 Processor -- start 2024-09-19T07:49:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.477+0000 7fd626d18640 1 -- start start 2024-09-19T07:49:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.478+0000 7fd626d18640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6200b3b90 0x7fd620154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.478+0000 7fd626d18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6200cdcc0 0x7fd620156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.478+0000 7fd626d18640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd620155700 0x7fd6201e5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.478+0000 7fd626d18640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd62015b9a0 con 0x7fd6200cdcc0 2024-09-19T07:49:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.478+0000 7fd626d18640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd6200b3700 con 0x7fd6200b3b90 2024-09-19T07:49:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.478+0000 7fd626d18640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd620155ae0 con 0x7fd620155700 2024-09-19T07:49:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.478+0000 7fd61ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6200cdcc0 0x7fd620156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.478+0000 7fd61ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6200cdcc0 0x7fd620156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55898/0 (socket says 172.21.15.16:55898) 2024-09-19T07:49:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.478+0000 7fd61ffff640 1 -- 172.21.15.16:0/1075169321 learned_addr learned my addr 172.21.15.16:0/1075169321 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.478+0000 7fd624ab4640 1 --2- 172.21.15.16:0/1075169321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6200b3b90 0x7fd620154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.479+0000 7fd6252b5640 1 --2- 172.21.15.16:0/1075169321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd620155700 0x7fd6201e5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.479+0000 7fd61ffff640 1 -- 172.21.15.16:0/1075169321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd620155700 msgr2=0x7fd6201e5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.479+0000 7fd61ffff640 1 --2- 172.21.15.16:0/1075169321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd620155700 0x7fd6201e5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.479+0000 7fd61ffff640 1 -- 172.21.15.16:0/1075169321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6200b3b90 msgr2=0x7fd620154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.479+0000 7fd61ffff640 1 --2- 172.21.15.16:0/1075169321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6200b3b90 0x7fd620154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.479+0000 7fd61ffff640 1 -- 172.21.15.16:0/1075169321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd620156cc0 con 0x7fd6200cdcc0 2024-09-19T07:49:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.479+0000 7fd6252b5640 1 --2- 172.21.15.16:0/1075169321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd620155700 0x7fd6201e5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.479+0000 7fd61ffff640 1 --2- 172.21.15.16:0/1075169321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6200cdcc0 0x7fd620156780 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd61400c7f0 tx=0x7fd61400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.479+0000 7fd61d7fa640 1 -- 172.21.15.16:0/1075169321 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614015070 con 0x7fd6200cdcc0 2024-09-19T07:49:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.479+0000 7fd61d7fa640 1 -- 172.21.15.16:0/1075169321 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd614007a30 con 0x7fd6200cdcc0 2024-09-19T07:49:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.479+0000 7fd61d7fa640 1 -- 172.21.15.16:0/1075169321 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614007e00 con 0x7fd6200cdcc0 2024-09-19T07:49:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.479+0000 7fd626d18640 1 -- 172.21.15.16:0/1075169321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd62015ad90 con 0x7fd6200cdcc0 2024-09-19T07:49:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.479+0000 7fd626d18640 1 -- 172.21.15.16:0/1075169321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6201f2c30 con 0x7fd6200cdcc0 2024-09-19T07:49:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.480+0000 7fd626d18640 1 -- 172.21.15.16:0/1075169321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6200ce120 con 0x7fd6200cdcc0 2024-09-19T07:49:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.480+0000 7fd61d7fa640 1 -- 172.21.15.16:0/1075169321 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd614018070 con 0x7fd6200cdcc0 2024-09-19T07:49:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.481+0000 7fd61d7fa640 1 --2- 172.21.15.16:0/1075169321 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd60c06c2b0 0x7fd60c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.481+0000 7fd61d7fa640 1 -- 172.21.15.16:0/1075169321 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7fd61408b5e0 con 0x7fd6200cdcc0 2024-09-19T07:49:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.481+0000 7fd624ab4640 1 --2- 172.21.15.16:0/1075169321 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd60c06c2b0 0x7fd60c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.482+0000 7fd624ab4640 1 --2- 172.21.15.16:0/1075169321 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd60c06c2b0 0x7fd60c06e770 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd620073be0 tx=0x7fd610009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:29.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.486+0000 7fd61d7fa640 1 -- 172.21.15.16:0/1075169321 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd614059b20 con 0x7fd6200cdcc0 2024-09-19T07:49:29.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.663+0000 7fd626d18640 1 -- 172.21.15.16:0/1075169321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6200b1660 con 0x7fd6200cdcc0 2024-09-19T07:49:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.664+0000 7fd61d7fa640 1 -- 172.21.15.16:0/1075169321 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16968 (secure 0 0 0) 0x7fd614059420 con 0x7fd6200cdcc0 2024-09-19T07:49:29.665 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:29.665 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":70,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:48:51.594413+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:48:47.611316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-19T07:48:48.934687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:49:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.666+0000 7fd626d18640 1 -- 172.21.15.16:0/1075169321 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd60c06c2b0 msgr2=0x7fd60c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.666+0000 7fd626d18640 1 --2- 172.21.15.16:0/1075169321 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd60c06c2b0 0x7fd60c06e770 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd620073be0 tx=0x7fd610009040 comp rx=0 tx=0).stop 2024-09-19T07:49:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.666+0000 7fd626d18640 1 -- 172.21.15.16:0/1075169321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6200cdcc0 msgr2=0x7fd620156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.666+0000 7fd626d18640 1 --2- 172.21.15.16:0/1075169321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6200cdcc0 0x7fd620156780 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd61400c7f0 tx=0x7fd61400cbf0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.666+0000 7fd626d18640 1 -- 172.21.15.16:0/1075169321 shutdown_connections 2024-09-19T07:49:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.666+0000 7fd626d18640 1 --2- 172.21.15.16:0/1075169321 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd60c06c2b0 0x7fd60c06e770 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.666+0000 7fd626d18640 1 --2- 172.21.15.16:0/1075169321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd620155700 0x7fd6201e5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.666+0000 7fd626d18640 1 --2- 172.21.15.16:0/1075169321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6200cdcc0 0x7fd620156780 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.666+0000 7fd626d18640 1 --2- 172.21.15.16:0/1075169321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6200b3b90 0x7fd620154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.666+0000 7fd626d18640 1 -- 172.21.15.16:0/1075169321 >> 172.21.15.16:0/1075169321 conn(0x7fd62006bbd0 msgr2=0x7fd62015ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.666+0000 7fd626d18640 1 -- 172.21.15.16:0/1075169321 shutdown_connections 2024-09-19T07:49:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.666+0000 7fd626d18640 1 -- 172.21.15.16:0/1075169321 wait complete. 2024-09-19T07:49:29.678 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T07:49:29.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:49:29.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.781+0000 7f3f7d082640 1 Processor -- start 2024-09-19T07:49:29.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.781+0000 7f3f7d082640 1 -- start start 2024-09-19T07:49:29.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.781+0000 7f3f7d082640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f78161cb0 0x7f3f780d4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:29.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.781+0000 7f3f7d082640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f780b3cf0 0x7f3f780b6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:29.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.781+0000 7f3f7d082640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f780b5bf0 con 0x7f3f780b4130 2024-09-19T07:49:29.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.781+0000 7f3f7d082640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f3f780cfcf0 con 0x7f3f78161cb0 2024-09-19T07:49:29.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.781+0000 7f3f7d082640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3f780cff40 con 0x7f3f780b3cf0 2024-09-19T07:49:29.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.782+0000 7f3f76d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f78161cb0 0x7f3f780d4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:29.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.782+0000 7f3f76d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f78161cb0 0x7f3f780d4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55910/0 (socket says 172.21.15.16:55910) 2024-09-19T07:49:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.782+0000 7f3f76d9d640 1 -- 172.21.15.16:0/2008215808 learned_addr learned my addr 172.21.15.16:0/2008215808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.782+0000 7f3f6ffff640 1 --2- 172.21.15.16:0/2008215808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f780b3cf0 0x7f3f780b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.782+0000 7f3f76d9d640 1 -- 172.21.15.16:0/2008215808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f780b3cf0 msgr2=0x7f3f780b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.782+0000 7f3f76d9d640 1 --2- 172.21.15.16:0/2008215808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f780b3cf0 0x7f3f780b6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.782+0000 7f3f76d9d640 1 -- 172.21.15.16:0/2008215808 >> v1:172.21.15.146:6789/0 conn(0x7f3f780b4130 legacy=0x7f3f78162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.782+0000 7f3f76d9d640 1 -- 172.21.15.16:0/2008215808 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f780d0220 con 0x7f3f78161cb0 2024-09-19T07:49:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.783+0000 7f3f76d9d640 1 --2- 172.21.15.16:0/2008215808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f78161cb0 0x7f3f780d4f60 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009870 tx=0x7f3f6802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37abc7a2ff0b526 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.783+0000 7f3f7659c640 1 -- 172.21.15.16:0/2008215808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68037070 con 0x7f3f78161cb0 2024-09-19T07:49:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.783+0000 7f3f7659c640 1 -- 172.21.15.16:0/2008215808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f68032440 con 0x7f3f78161cb0 2024-09-19T07:49:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.783+0000 7f3f7659c640 1 -- 172.21.15.16:0/2008215808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68032810 con 0x7f3f78161cb0 2024-09-19T07:49:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.783+0000 7f3f7d082640 1 -- 172.21.15.16:0/2008215808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f78161cb0 msgr2=0x7f3f780d4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.783+0000 7f3f7d082640 1 --2- 172.21.15.16:0/2008215808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f78161cb0 0x7f3f780d4f60 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009870 tx=0x7f3f6802a1c0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.783+0000 7f3f7d082640 1 -- 172.21.15.16:0/2008215808 shutdown_connections 2024-09-19T07:49:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.783+0000 7f3f7d082640 1 --2- 172.21.15.16:0/2008215808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f780b3cf0 0x7f3f780b6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.783+0000 7f3f7d082640 1 --2- 172.21.15.16:0/2008215808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f78161cb0 0x7f3f780d4f60 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.783+0000 7f3f7d082640 1 -- 172.21.15.16:0/2008215808 >> 172.21.15.16:0/2008215808 conn(0x7f3f7808bfb0 msgr2=0x7f3f780d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.783+0000 7f3f7d082640 1 -- 172.21.15.16:0/2008215808 shutdown_connections 2024-09-19T07:49:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.784+0000 7f3f7d082640 1 -- 172.21.15.16:0/2008215808 wait complete. 2024-09-19T07:49:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.785+0000 7f3f7d082640 1 Processor -- start 2024-09-19T07:49:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.785+0000 7f3f7d082640 1 -- start start 2024-09-19T07:49:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f7d082640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f780b3cf0 0x7f3f78154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f7d082640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f780b4130 0x7f3f780c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f7d082640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f78161cb0 0x7f3f780c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f7d082640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3f78162090 con 0x7f3f78161cb0 2024-09-19T07:49:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f7d082640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3f780d0500 con 0x7f3f780b4130 2024-09-19T07:49:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f7d082640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3f78162210 con 0x7f3f780b3cf0 2024-09-19T07:49:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f7759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f78161cb0 0x7f3f780c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f7759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f78161cb0 0x7f3f780c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55912/0 (socket says 172.21.15.16:55912) 2024-09-19T07:49:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f7759e640 1 -- 172.21.15.16:0/707253227 learned_addr learned my addr 172.21.15.16:0/707253227 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f76d9d640 1 --2- 172.21.15.16:0/707253227 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f780b3cf0 0x7f3f78154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f6ffff640 1 --2- 172.21.15.16:0/707253227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f780b4130 0x7f3f780c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f7759e640 1 -- 172.21.15.16:0/707253227 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f780b3cf0 msgr2=0x7f3f78154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f7759e640 1 --2- 172.21.15.16:0/707253227 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f780b3cf0 0x7f3f78154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f7759e640 1 -- 172.21.15.16:0/707253227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f780b4130 msgr2=0x7f3f780c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f7759e640 1 --2- 172.21.15.16:0/707253227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f780b4130 0x7f3f780c9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f7759e640 1 -- 172.21.15.16:0/707253227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f780b74f0 con 0x7f3f78161cb0 2024-09-19T07:49:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f7759e640 1 --2- 172.21.15.16:0/707253227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f78161cb0 0x7f3f780c9b80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000c7d0 tx=0x7f3f7000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f6f7fe640 1 -- 172.21.15.16:0/707253227 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70003b70 con 0x7f3f78161cb0 2024-09-19T07:49:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f6f7fe640 1 -- 172.21.15.16:0/707253227 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f70003d30 con 0x7f3f78161cb0 2024-09-19T07:49:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f7d082640 1 -- 172.21.15.16:0/707253227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f780b77d0 con 0x7f3f78161cb0 2024-09-19T07:49:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.786+0000 7f3f6f7fe640 1 -- 172.21.15.16:0/707253227 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7000aa90 con 0x7f3f78161cb0 2024-09-19T07:49:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.787+0000 7f3f7d082640 1 -- 172.21.15.16:0/707253227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f780b7d90 con 0x7f3f78161cb0 2024-09-19T07:49:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.788+0000 7f3f7d082640 1 -- 172.21.15.16:0/707253227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f780d4f60 con 0x7f3f78161cb0 2024-09-19T07:49:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.788+0000 7f3f6f7fe640 1 -- 172.21.15.16:0/707253227 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3f7000ac30 con 0x7f3f78161cb0 2024-09-19T07:49:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.788+0000 7f3f6f7fe640 1 --2- 172.21.15.16:0/707253227 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3f5806c2b0 0x7f3f5806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.789+0000 7f3f6f7fe640 1 -- 172.21.15.16:0/707253227 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f3f7008b320 con 0x7f3f78161cb0 2024-09-19T07:49:29.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.789+0000 7f3f76d9d640 1 --2- 172.21.15.16:0/707253227 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3f5806c2b0 0x7f3f5806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:29.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.789+0000 7f3f76d9d640 1 --2- 172.21.15.16:0/707253227 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3f5806c2b0 0x7f3f5806e770 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3f68034000 tx=0x7f3f6802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:29.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:29.793+0000 7f3f6f7fe640 1 -- 172.21.15.16:0/707253227 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3f70059340 con 0x7f3f78161cb0 2024-09-19T07:49:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.003+0000 7f3f7d082640 1 -- 172.21.15.16:0/707253227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f780b1b50 con 0x7f3f78161cb0 2024-09-19T07:49:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.003+0000 7f3f6f7fe640 1 -- 172.21.15.16:0/707253227 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5227 (secure 0 0 0) 0x7f3f7005d640 con 0x7f3f78161cb0 2024-09-19T07:49:30.004 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:30.004 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":33,"btime":"2024-09-19T07:48:52:612055+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2104039348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2104039348},{"type":"v1","addr":"172.21.15.16:6835","nonce":2104039348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/1109733621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1109733621},{"type":"v1","addr":"172.21.15.146:6836","nonce":1109733621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3602092462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3602092462},{"type":"v1","addr":"172.21.15.146:6837","nonce":3602092462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:48:49.954706+0000","modified":"2024-09-19T07:48:51.957927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5751},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5751":{"gid":5751,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6834/418904597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":418904597},{"type":"v1","addr":"172.21.15.16:6834","nonce":418904597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5751,"qdb_cluster":[5751]},"id":4}]} 2024-09-19T07:49:30.004 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 33 2024-09-19T07:49:30.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.005+0000 7f3f7d082640 1 -- 172.21.15.16:0/707253227 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3f5806c2b0 msgr2=0x7f3f5806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.005+0000 7f3f7d082640 1 --2- 172.21.15.16:0/707253227 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3f5806c2b0 0x7f3f5806e770 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3f68034000 tx=0x7f3f6802e000 comp rx=0 tx=0).stop 2024-09-19T07:49:30.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.005+0000 7f3f7d082640 1 -- 172.21.15.16:0/707253227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f78161cb0 msgr2=0x7f3f780c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.005+0000 7f3f7d082640 1 --2- 172.21.15.16:0/707253227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f78161cb0 0x7f3f780c9b80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000c7d0 tx=0x7f3f7000cc90 comp rx=0 tx=0).stop 2024-09-19T07:49:30.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.005+0000 7f3f7d082640 1 -- 172.21.15.16:0/707253227 shutdown_connections 2024-09-19T07:49:30.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.005+0000 7f3f7d082640 1 --2- 172.21.15.16:0/707253227 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3f5806c2b0 0x7f3f5806e770 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.005+0000 7f3f7d082640 1 --2- 172.21.15.16:0/707253227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f78161cb0 0x7f3f780c9b80 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.005+0000 7f3f7d082640 1 --2- 172.21.15.16:0/707253227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f780b4130 0x7f3f780c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.005+0000 7f3f7d082640 1 --2- 172.21.15.16:0/707253227 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f780b3cf0 0x7f3f78154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.005+0000 7f3f7d082640 1 -- 172.21.15.16:0/707253227 >> 172.21.15.16:0/707253227 conn(0x7f3f7808bfb0 msgr2=0x7f3f7806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:30.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.005+0000 7f3f7d082640 1 -- 172.21.15.16:0/707253227 shutdown_connections 2024-09-19T07:49:30.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.005+0000 7f3f7d082640 1 -- 172.21.15.16:0/707253227 wait complete. 2024-09-19T07:49:30.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:49:30.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.122+0000 7f8a9d98d640 1 Processor -- start 2024-09-19T07:49:30.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.122+0000 7f8a9d98d640 1 -- start start 2024-09-19T07:49:30.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.123+0000 7f8a9d98d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8a980d2ee0 0x7f8a980b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.123+0000 7f8a9d98d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a980d0b40 con 0x7f8a9809b130 2024-09-19T07:49:30.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.123+0000 7f8a9d98d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a980d0e20 con 0x7f8a9815fc50 2024-09-19T07:49:30.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.123+0000 7f8a9d98d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f8a980b3bf0 con 0x7f8a980d2ee0 2024-09-19T07:49:30.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.123+0000 7f8a967fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8a9809b130 0x7f8a980cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56208/0 (socket says 172.21.15.16:56208) 2024-09-19T07:49:30.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.123+0000 7f8a967fc640 1 -- 172.21.15.16:0/1407840345 learned_addr learned my addr 172.21.15.16:0/1407840345 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:30.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.123+0000 7f8a977fe640 1 --2- 172.21.15.16:0/1407840345 >> v2:172.21.15.146:3300/0 conn(0x7f8a980d2ee0 0x7f8a980b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.123+0000 7f8a95ffb640 1 -- 172.21.15.16:0/1407840345 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1538342416 0 0) 0x7f8a980d0b40 con 0x7f8a9809b130 2024-09-19T07:49:30.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.123+0000 7f8a95ffb640 1 -- 172.21.15.16:0/1407840345 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a84003520 con 0x7f8a9809b130 2024-09-19T07:49:30.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a95ffb640 1 -- 172.21.15.16:0/1407840345 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 81050786 0 0) 0x7f8a84003520 con 0x7f8a9809b130 2024-09-19T07:49:30.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a95ffb640 1 -- 172.21.15.16:0/1407840345 >> v2:172.21.15.146:3300/0 conn(0x7f8a980d2ee0 msgr2=0x7f8a980b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:30.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a95ffb640 1 --2- 172.21.15.16:0/1407840345 >> v2:172.21.15.146:3300/0 conn(0x7f8a980d2ee0 0x7f8a980b36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a95ffb640 1 -- 172.21.15.16:0/1407840345 >> v1:172.21.15.146:6789/0 conn(0x7f8a9815fc50 legacy=0x7f8a9809aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a95ffb640 1 -- 172.21.15.16:0/1407840345 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a980cced0 con 0x7f8a9809b130 2024-09-19T07:49:30.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a95ffb640 1 -- 172.21.15.16:0/1407840345 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f8a80001670 con 0x7f8a9809b130 2024-09-19T07:49:30.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a95ffb640 1 -- 172.21.15.16:0/1407840345 >> v1:172.21.15.16:6789/0 conn(0x7f8a9809b130 legacy=0x7f8a980cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a95ffb640 1 --2- 172.21.15.16:0/1407840345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a840039b0 0x7f8a84003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a95ffb640 1 --2- 172.21.15.16:0/1407840345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a840043b0 0x7f8a84004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a95ffb640 1 --2- 172.21.15.16:0/1407840345 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a84005080 0x7f8a8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a95ffb640 1 -- 172.21.15.16:0/1407840345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8400bfb0 con 0x7f8a840039b0 2024-09-19T07:49:30.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a95ffb640 1 -- 172.21.15.16:0/1407840345 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8400c160 con 0x7f8a840043b0 2024-09-19T07:49:30.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a95ffb640 1 -- 172.21.15.16:0/1407840345 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8a8400c360 con 0x7f8a84005080 2024-09-19T07:49:30.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a96ffd640 1 --2- 172.21.15.16:0/1407840345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a840039b0 0x7f8a84003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a967fc640 1 --2- 172.21.15.16:0/1407840345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a840043b0 0x7f8a84004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a977fe640 1 --2- 172.21.15.16:0/1407840345 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a84005080 0x7f8a8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a96ffd640 1 -- 172.21.15.16:0/1407840345 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a84005080 msgr2=0x7f8a8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:30.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a96ffd640 1 --2- 172.21.15.16:0/1407840345 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a84005080 0x7f8a8400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a96ffd640 1 -- 172.21.15.16:0/1407840345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a840043b0 msgr2=0x7f8a84004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:30.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a96ffd640 1 --2- 172.21.15.16:0/1407840345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a840043b0 0x7f8a84004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a96ffd640 1 -- 172.21.15.16:0/1407840345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a84003520 con 0x7f8a840039b0 2024-09-19T07:49:30.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.124+0000 7f8a977fe640 1 -- 172.21.15.16:0/1407840345 reap_dead start 2024-09-19T07:49:30.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.125+0000 7f8a96ffd640 1 --2- 172.21.15.16:0/1407840345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a840039b0 0x7f8a84003d90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8a90001860 tx=0x7f8a84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df5c909eeec9b46 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:30.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.125+0000 7f8a95ffb640 1 -- 172.21.15.16:0/1407840345 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90037070 con 0x7f8a840039b0 2024-09-19T07:49:30.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.125+0000 7f8a95ffb640 1 -- 172.21.15.16:0/1407840345 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a9002ac30 con 0x7f8a840039b0 2024-09-19T07:49:30.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.125+0000 7f8a95ffb640 1 -- 172.21.15.16:0/1407840345 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90001430 con 0x7f8a840039b0 2024-09-19T07:49:30.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.126+0000 7f8a9d98d640 1 -- 172.21.15.16:0/1407840345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a840039b0 msgr2=0x7f8a84003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.126+0000 7f8a9d98d640 1 --2- 172.21.15.16:0/1407840345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a840039b0 0x7f8a84003d90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8a90001860 tx=0x7f8a84005830 comp rx=0 tx=0).stop 2024-09-19T07:49:30.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.126+0000 7f8a9d98d640 1 -- 172.21.15.16:0/1407840345 shutdown_connections 2024-09-19T07:49:30.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.126+0000 7f8a9d98d640 1 --2- 172.21.15.16:0/1407840345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a840039b0 0x7f8a84003d90 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.126+0000 7f8a9d98d640 1 -- 172.21.15.16:0/1407840345 >> 172.21.15.16:0/1407840345 conn(0x7f8a980d23c0 msgr2=0x7f8a980d27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:30.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.126+0000 7f8a9d98d640 1 -- 172.21.15.16:0/1407840345 shutdown_connections 2024-09-19T07:49:30.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.126+0000 7f8a9d98d640 1 -- 172.21.15.16:0/1407840345 wait complete. 2024-09-19T07:49:30.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.127+0000 7f8a9d98d640 1 Processor -- start 2024-09-19T07:49:30.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.127+0000 7f8a9d98d640 1 -- start start 2024-09-19T07:49:30.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.128+0000 7f8a9d98d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a840039b0 0x7f8a9815ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.128+0000 7f8a9d98d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a980cb3d0 0x7f8a980cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.128+0000 7f8a9d98d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a9814f400 0x7f8a9814f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.128+0000 7f8a9d98d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a980b3830 con 0x7f8a980cb3d0 2024-09-19T07:49:30.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.128+0000 7f8a9d98d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8a980b36b0 con 0x7f8a9814f400 2024-09-19T07:49:30.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.128+0000 7f8a9d98d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8a980b39b0 con 0x7f8a840039b0 2024-09-19T07:49:30.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.128+0000 7f8a967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a980cb3d0 0x7f8a980cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.128+0000 7f8a967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a980cb3d0 0x7f8a980cf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55920/0 (socket says 172.21.15.16:55920) 2024-09-19T07:49:30.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.128+0000 7f8a967fc640 1 -- 172.21.15.16:0/4234152913 learned_addr learned my addr 172.21.15.16:0/4234152913 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:30.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.128+0000 7f8a96ffd640 1 --2- 172.21.15.16:0/4234152913 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a840039b0 0x7f8a9815ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.128+0000 7f8a977fe640 1 --2- 172.21.15.16:0/4234152913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a9814f400 0x7f8a9814f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.128+0000 7f8a967fc640 1 -- 172.21.15.16:0/4234152913 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a840039b0 msgr2=0x7f8a9815ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.128+0000 7f8a967fc640 1 --2- 172.21.15.16:0/4234152913 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a840039b0 0x7f8a9815ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.128+0000 7f8a967fc640 1 -- 172.21.15.16:0/4234152913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a9814f400 msgr2=0x7f8a9814f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.128+0000 7f8a967fc640 1 --2- 172.21.15.16:0/4234152913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a9814f400 0x7f8a9814f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.128+0000 7f8a967fc640 1 -- 172.21.15.16:0/4234152913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a980cbbd0 con 0x7f8a980cb3d0 2024-09-19T07:49:30.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.128+0000 7f8a977fe640 1 --2- 172.21.15.16:0/4234152913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a9814f400 0x7f8a9814f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:30.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.128+0000 7f8a967fc640 1 --2- 172.21.15.16:0/4234152913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a980cb3d0 0x7f8a980cf6f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8a80004fa0 tx=0x7f8a800074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:30.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.129+0000 7f8a8ffff640 1 -- 172.21.15.16:0/4234152913 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80007740 con 0x7f8a980cb3d0 2024-09-19T07:49:30.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.129+0000 7f8a8ffff640 1 -- 172.21.15.16:0/4234152913 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a80018070 con 0x7f8a980cb3d0 2024-09-19T07:49:30.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.129+0000 7f8a9d98d640 1 -- 172.21.15.16:0/4234152913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a980cbeb0 con 0x7f8a980cb3d0 2024-09-19T07:49:30.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.129+0000 7f8a8ffff640 1 -- 172.21.15.16:0/4234152913 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8000fbd0 con 0x7f8a980cb3d0 2024-09-19T07:49:30.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.129+0000 7f8a9d98d640 1 -- 172.21.15.16:0/4234152913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a980cc470 con 0x7f8a980cb3d0 2024-09-19T07:49:30.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.130+0000 7f8a9d98d640 1 -- 172.21.15.16:0/4234152913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a9809aa20 con 0x7f8a980cb3d0 2024-09-19T07:49:30.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.130+0000 7f8a8ffff640 1 -- 172.21.15.16:0/4234152913 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8a8001d460 con 0x7f8a980cb3d0 2024-09-19T07:49:30.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.131+0000 7f8a8ffff640 1 --2- 172.21.15.16:0/4234152913 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8a7806c450 0x7f8a7806e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.131+0000 7f8a8ffff640 1 -- 172.21.15.16:0/4234152913 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f8a8008b5d0 con 0x7f8a980cb3d0 2024-09-19T07:49:30.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.131+0000 7f8a96ffd640 1 --2- 172.21.15.16:0/4234152913 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8a7806c450 0x7f8a7806e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.131+0000 7f8a96ffd640 1 --2- 172.21.15.16:0/4234152913 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8a7806c450 0x7f8a7806e910 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8a90034000 tx=0x7f8a90009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:30.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.135+0000 7f8a8ffff640 1 -- 172.21.15.16:0/4234152913 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8a80059bb0 con 0x7f8a980cb3d0 2024-09-19T07:49:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.345+0000 7f8a9d98d640 1 -- 172.21.15.16:0/4234152913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a980d0780 con 0x7f8a980cb3d0 2024-09-19T07:49:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.346+0000 7f8a8ffff640 1 -- 172.21.15.16:0/4234152913 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5227 (secure 0 0 0) 0x7f8a800594b0 con 0x7f8a980cb3d0 2024-09-19T07:49:30.346 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:30.346 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":33,"btime":"2024-09-19T07:48:52:612055+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2104039348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2104039348},{"type":"v1","addr":"172.21.15.16:6835","nonce":2104039348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/1109733621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1109733621},{"type":"v1","addr":"172.21.15.146:6836","nonce":1109733621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3602092462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3602092462},{"type":"v1","addr":"172.21.15.146:6837","nonce":3602092462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:48:49.954706+0000","modified":"2024-09-19T07:48:51.957927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5751},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5751":{"gid":5751,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6834/418904597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":418904597},{"type":"v1","addr":"172.21.15.16:6834","nonce":418904597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5751,"qdb_cluster":[5751]},"id":4}]} 2024-09-19T07:49:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 33 2024-09-19T07:49:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.347+0000 7f8a9d98d640 1 -- 172.21.15.16:0/4234152913 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8a7806c450 msgr2=0x7f8a7806e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.347+0000 7f8a9d98d640 1 --2- 172.21.15.16:0/4234152913 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8a7806c450 0x7f8a7806e910 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8a90034000 tx=0x7f8a90009030 comp rx=0 tx=0).stop 2024-09-19T07:49:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.347+0000 7f8a9d98d640 1 -- 172.21.15.16:0/4234152913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a980cb3d0 msgr2=0x7f8a980cf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.347+0000 7f8a9d98d640 1 --2- 172.21.15.16:0/4234152913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a980cb3d0 0x7f8a980cf6f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8a80004fa0 tx=0x7f8a800074f0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.347+0000 7f8a9d98d640 1 -- 172.21.15.16:0/4234152913 shutdown_connections 2024-09-19T07:49:30.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.348+0000 7f8a9d98d640 1 --2- 172.21.15.16:0/4234152913 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8a7806c450 0x7f8a7806e910 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.348+0000 7f8a9d98d640 1 --2- 172.21.15.16:0/4234152913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a9814f400 0x7f8a9814f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.348+0000 7f8a9d98d640 1 --2- 172.21.15.16:0/4234152913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a980cb3d0 0x7f8a980cf6f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.348+0000 7f8a9d98d640 1 --2- 172.21.15.16:0/4234152913 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a840039b0 0x7f8a9815ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.348+0000 7f8a9d98d640 1 -- 172.21.15.16:0/4234152913 >> 172.21.15.16:0/4234152913 conn(0x7f8a980d23c0 msgr2=0x7f8a980d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.348+0000 7f8a9d98d640 1 -- 172.21.15.16:0/4234152913 shutdown_connections 2024-09-19T07:49:30.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.348+0000 7f8a9d98d640 1 -- 172.21.15.16:0/4234152913 wait complete. 2024-09-19T07:49:30.359 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:49:30.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.461+0000 7f7398d9f640 1 Processor -- start 2024-09-19T07:49:30.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.461+0000 7f7398d9f640 1 -- start start 2024-09-19T07:49:30.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.462+0000 7f7398d9f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f739415e6a0 0x7f73940b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.462+0000 7f7398d9f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73940cf920 0x7f73940d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.462+0000 7f7398d9f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f73940d2a60 0x7f73940d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.462+0000 7f7398d9f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f73940d2ea0 con 0x7f739415e6a0 2024-09-19T07:49:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.462+0000 7f7398d9f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f739415ea80 con 0x7f73940d2a60 2024-09-19T07:49:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.462+0000 7f7398d9f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f73940b39b0 con 0x7f73940cf920 2024-09-19T07:49:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.462+0000 7f73937fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f739415e6a0 0x7f73940b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.462+0000 7f73937fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f739415e6a0 0x7f73940b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55924/0 (socket says 172.21.15.16:55924) 2024-09-19T07:49:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.462+0000 7f73937fe640 1 -- 172.21.15.16:0/4130333897 learned_addr learned my addr 172.21.15.16:0/4130333897 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.462+0000 7f7392ffd640 1 --2- 172.21.15.16:0/4130333897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73940cf920 0x7f73940d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.462+0000 7f7393fff640 1 --2- 172.21.15.16:0/4130333897 >> v2:172.21.15.146:3300/0 conn(0x7f73940d2a60 0x7f73940d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.462+0000 7f73937fe640 1 -- 172.21.15.16:0/4130333897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73940cf920 msgr2=0x7f73940d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.462+0000 7f73937fe640 1 --2- 172.21.15.16:0/4130333897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73940cf920 0x7f73940d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.462+0000 7f73937fe640 1 -- 172.21.15.16:0/4130333897 >> v2:172.21.15.146:3300/0 conn(0x7f73940d2a60 msgr2=0x7f73940d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.462+0000 7f73937fe640 1 --2- 172.21.15.16:0/4130333897 >> v2:172.21.15.146:3300/0 conn(0x7f73940d2a60 0x7f73940d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.462+0000 7f73937fe640 1 -- 172.21.15.16:0/4130333897 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73940cfee0 con 0x7f739415e6a0 2024-09-19T07:49:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.462+0000 7f7392ffd640 1 --2- 172.21.15.16:0/4130333897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73940cf920 0x7f73940d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.463+0000 7f73937fe640 1 --2- 172.21.15.16:0/4130333897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f739415e6a0 0x7f73940b3470 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f73940ba420 tx=0x7f738402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ef748393d9b1f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.463+0000 7f73927fc640 1 -- 172.21.15.16:0/4130333897 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384037070 con 0x7f739415e6a0 2024-09-19T07:49:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.463+0000 7f73927fc640 1 -- 172.21.15.16:0/4130333897 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7384032440 con 0x7f739415e6a0 2024-09-19T07:49:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.463+0000 7f73927fc640 1 -- 172.21.15.16:0/4130333897 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384032810 con 0x7f739415e6a0 2024-09-19T07:49:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.464+0000 7f7398d9f640 1 -- 172.21.15.16:0/4130333897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f739415e6a0 msgr2=0x7f73940b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.464+0000 7f7398d9f640 1 --2- 172.21.15.16:0/4130333897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f739415e6a0 0x7f73940b3470 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f73940ba420 tx=0x7f738402a1e0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.464+0000 7f7398d9f640 1 -- 172.21.15.16:0/4130333897 shutdown_connections 2024-09-19T07:49:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.464+0000 7f7398d9f640 1 --2- 172.21.15.16:0/4130333897 >> v2:172.21.15.146:3300/0 conn(0x7f73940d2a60 0x7f73940d0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.464+0000 7f7398d9f640 1 --2- 172.21.15.16:0/4130333897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73940cf920 0x7f73940d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.464+0000 7f7398d9f640 1 --2- 172.21.15.16:0/4130333897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f739415e6a0 0x7f73940b3470 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.464+0000 7f7398d9f640 1 -- 172.21.15.16:0/4130333897 >> 172.21.15.16:0/4130333897 conn(0x7f739408bfb0 msgr2=0x7f73940d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.464+0000 7f7398d9f640 1 -- 172.21.15.16:0/4130333897 shutdown_connections 2024-09-19T07:49:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.464+0000 7f7398d9f640 1 -- 172.21.15.16:0/4130333897 wait complete. 2024-09-19T07:49:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.465+0000 7f7398d9f640 1 Processor -- start 2024-09-19T07:49:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.465+0000 7f7398d9f640 1 -- start start 2024-09-19T07:49:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.466+0000 7f7398d9f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73940cf920 0x7f739415b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.466+0000 7f7398d9f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73940d2a60 0x7f73940b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.466+0000 7f7398d9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f739415e6a0 0x7f7394160290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.466+0000 7f7398d9f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73940d0780 con 0x7f739415e6a0 2024-09-19T07:49:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.466+0000 7f7398d9f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f73940d04a0 con 0x7f73940cf920 2024-09-19T07:49:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.466+0000 7f7398d9f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f73940d0a60 con 0x7f73940d2a60 2024-09-19T07:49:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.466+0000 7f7393fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f739415e6a0 0x7f7394160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.466+0000 7f7393fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f739415e6a0 0x7f7394160290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55936/0 (socket says 172.21.15.16:55936) 2024-09-19T07:49:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.466+0000 7f7393fff640 1 -- 172.21.15.16:0/2085012511 learned_addr learned my addr 172.21.15.16:0/2085012511 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.466+0000 7f73937fe640 1 --2- 172.21.15.16:0/2085012511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73940cf920 0x7f739415b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.466+0000 7f7392ffd640 1 --2- 172.21.15.16:0/2085012511 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73940d2a60 0x7f73940b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.466+0000 7f7393fff640 1 -- 172.21.15.16:0/2085012511 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73940d2a60 msgr2=0x7f73940b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.466+0000 7f7393fff640 1 --2- 172.21.15.16:0/2085012511 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73940d2a60 0x7f73940b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.466+0000 7f7393fff640 1 -- 172.21.15.16:0/2085012511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73940cf920 msgr2=0x7f739415b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.466+0000 7f7393fff640 1 --2- 172.21.15.16:0/2085012511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73940cf920 0x7f739415b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.466+0000 7f7393fff640 1 -- 172.21.15.16:0/2085012511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739415ba20 con 0x7f739415e6a0 2024-09-19T07:49:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.467+0000 7f7393fff640 1 --2- 172.21.15.16:0/2085012511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f739415e6a0 0x7f7394160290 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f738c00cd20 tx=0x7f738c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.467+0000 7f737bfff640 1 -- 172.21.15.16:0/2085012511 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c003b90 con 0x7f739415e6a0 2024-09-19T07:49:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.467+0000 7f737bfff640 1 -- 172.21.15.16:0/2085012511 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f738c003d30 con 0x7f739415e6a0 2024-09-19T07:49:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.467+0000 7f737bfff640 1 -- 172.21.15.16:0/2085012511 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c004240 con 0x7f739415e6a0 2024-09-19T07:49:30.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.467+0000 7f7398d9f640 1 -- 172.21.15.16:0/2085012511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73940b5b50 con 0x7f739415e6a0 2024-09-19T07:49:30.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.467+0000 7f7398d9f640 1 -- 172.21.15.16:0/2085012511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73940b6110 con 0x7f739415e6a0 2024-09-19T07:49:30.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.468+0000 7f7398d9f640 1 -- 172.21.15.16:0/2085012511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f739406e4c0 con 0x7f739415e6a0 2024-09-19T07:49:30.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.468+0000 7f737bfff640 1 -- 172.21.15.16:0/2085012511 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f738c0043e0 con 0x7f739415e6a0 2024-09-19T07:49:30.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.469+0000 7f737bfff640 1 --2- 172.21.15.16:0/2085012511 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f738006c2b0 0x7f738006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.469+0000 7f737bfff640 1 -- 172.21.15.16:0/2085012511 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f738c08bf20 con 0x7f739415e6a0 2024-09-19T07:49:30.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.469+0000 7f73937fe640 1 --2- 172.21.15.16:0/2085012511 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f738006c2b0 0x7f738006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.470+0000 7f73937fe640 1 --2- 172.21.15.16:0/2085012511 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f738006c2b0 0x7f738006e770 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7384034000 tx=0x7f738402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:30.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.473+0000 7f737bfff640 1 -- 172.21.15.16:0/2085012511 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f738c059e20 con 0x7f739415e6a0 2024-09-19T07:49:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.652+0000 7f7398d9f640 1 -- 172.21.15.16:0/2085012511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73940d2520 con 0x7f739415e6a0 2024-09-19T07:49:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.653+0000 7f737bfff640 1 -- 172.21.15.16:0/2085012511 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16968 (secure 0 0 0) 0x7f738c05e120 con 0x7f739415e6a0 2024-09-19T07:49:30.656 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:30.656 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":70,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:48:51.594413+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:48:47.611316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-19T07:48:48.934687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:49:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.655+0000 7f7398d9f640 1 -- 172.21.15.16:0/2085012511 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f738006c2b0 msgr2=0x7f738006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.655+0000 7f7398d9f640 1 --2- 172.21.15.16:0/2085012511 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f738006c2b0 0x7f738006e770 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7384034000 tx=0x7f738402e000 comp rx=0 tx=0).stop 2024-09-19T07:49:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.655+0000 7f7398d9f640 1 -- 172.21.15.16:0/2085012511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f739415e6a0 msgr2=0x7f7394160290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.655+0000 7f7398d9f640 1 --2- 172.21.15.16:0/2085012511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f739415e6a0 0x7f7394160290 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f738c00cd20 tx=0x7f738c00a660 comp rx=0 tx=0).stop 2024-09-19T07:49:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.655+0000 7f7398d9f640 1 -- 172.21.15.16:0/2085012511 shutdown_connections 2024-09-19T07:49:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.655+0000 7f7398d9f640 1 --2- 172.21.15.16:0/2085012511 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f738006c2b0 0x7f738006e770 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.655+0000 7f7398d9f640 1 --2- 172.21.15.16:0/2085012511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f739415e6a0 0x7f7394160290 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.655+0000 7f7398d9f640 1 --2- 172.21.15.16:0/2085012511 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73940d2a60 0x7f73940b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.655+0000 7f7398d9f640 1 --2- 172.21.15.16:0/2085012511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73940cf920 0x7f739415b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.655+0000 7f7398d9f640 1 -- 172.21.15.16:0/2085012511 >> 172.21.15.16:0/2085012511 conn(0x7f739408bfb0 msgr2=0x7f739406d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.655+0000 7f7398d9f640 1 -- 172.21.15.16:0/2085012511 shutdown_connections 2024-09-19T07:49:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.655+0000 7f7398d9f640 1 -- 172.21.15.16:0/2085012511 wait complete. 2024-09-19T07:49:30.667 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T07:49:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.771+0000 7fd32a0ba640 1 Processor -- start 2024-09-19T07:49:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.771+0000 7fd32a0ba640 1 -- start start 2024-09-19T07:49:30.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.771+0000 7fd32a0ba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3240cfc80 0x7fd3240cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.771+0000 7fd32a0ba640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3240d39b0 con 0x7fd32415e600 2024-09-19T07:49:30.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.771+0000 7fd32a0ba640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3240d3c90 con 0x7fd3240d27f0 2024-09-19T07:49:30.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.771+0000 7fd32a0ba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd3240cb710 con 0x7fd3240cfc80 2024-09-19T07:49:30.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd3237fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd32415e600 0x7fd3240cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56218/0 (socket says 172.21.15.16:56218) 2024-09-19T07:49:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd3237fe640 1 -- 172.21.15.16:0/671814496 learned_addr learned my addr 172.21.15.16:0/671814496 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd322ffd640 1 --2- 172.21.15.16:0/671814496 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3240cfc80 0x7fd3240cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd3227fc640 1 -- 172.21.15.16:0/671814496 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2885242184 0 0) 0x7fd3240d39b0 con 0x7fd32415e600 2024-09-19T07:49:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd3227fc640 1 -- 172.21.15.16:0/671814496 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd30c003520 con 0x7fd32415e600 2024-09-19T07:49:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd3227fc640 1 -- 172.21.15.16:0/671814496 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 580830666 0 0) 0x7fd30c003520 con 0x7fd32415e600 2024-09-19T07:49:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd3227fc640 1 -- 172.21.15.16:0/671814496 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3240cfc80 msgr2=0x7fd3240cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd3227fc640 1 --2- 172.21.15.16:0/671814496 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3240cfc80 0x7fd3240cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd3227fc640 1 -- 172.21.15.16:0/671814496 >> v1:172.21.15.146:6789/0 conn(0x7fd3240d27f0 legacy=0x7fd3240cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd3227fc640 1 -- 172.21.15.16:0/671814496 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3240d3f70 con 0x7fd32415e600 2024-09-19T07:49:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd3227fc640 1 -- 172.21.15.16:0/671814496 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fd314001690 con 0x7fd32415e600 2024-09-19T07:49:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd322ffd640 1 --2- 172.21.15.16:0/671814496 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3240cfc80 0x7fd3240cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:49:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd3227fc640 1 -- 172.21.15.16:0/671814496 >> v1:172.21.15.16:6789/0 conn(0x7fd32415e600 legacy=0x7fd3240cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd3227fc640 1 --2- 172.21.15.16:0/671814496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd30c0039e0 0x7fd30c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd3227fc640 1 --2- 172.21.15.16:0/671814496 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd30c004390 0x7fd30c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd3227fc640 1 --2- 172.21.15.16:0/671814496 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3240cfc80 0x7fd30c005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd3227fc640 1 -- 172.21.15.16:0/671814496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3240cb710 con 0x7fd30c0039e0 2024-09-19T07:49:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd3227fc640 1 -- 172.21.15.16:0/671814496 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c00b9c0 con 0x7fd30c004390 2024-09-19T07:49:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd3227fc640 1 -- 172.21.15.16:0/671814496 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd30c00bbc0 con 0x7fd3240cfc80 2024-09-19T07:49:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd3237fe640 1 --2- 172.21.15.16:0/671814496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd30c0039e0 0x7fd30c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd322ffd640 1 --2- 172.21.15.16:0/671814496 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd30c004390 0x7fd30c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.772+0000 7fd323fff640 1 --2- 172.21.15.16:0/671814496 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3240cfc80 0x7fd30c005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.773+0000 7fd3237fe640 1 -- 172.21.15.16:0/671814496 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3240cfc80 msgr2=0x7fd30c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.773+0000 7fd3237fe640 1 --2- 172.21.15.16:0/671814496 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3240cfc80 0x7fd30c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.773+0000 7fd3237fe640 1 -- 172.21.15.16:0/671814496 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd30c004390 msgr2=0x7fd30c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.773+0000 7fd3237fe640 1 --2- 172.21.15.16:0/671814496 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd30c004390 0x7fd30c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.773+0000 7fd3237fe640 1 -- 172.21.15.16:0/671814496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd30c003520 con 0x7fd30c0039e0 2024-09-19T07:49:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.773+0000 7fd323fff640 1 --2- 172.21.15.16:0/671814496 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3240cfc80 0x7fd30c005080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.773+0000 7fd3237fe640 1 --2- 172.21.15.16:0/671814496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd30c0039e0 0x7fd30c003dc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd32408c5f0 tx=0x7fd30c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1600488a8169036 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.773+0000 7fd3227fc640 1 -- 172.21.15.16:0/671814496 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314007ce0 con 0x7fd30c0039e0 2024-09-19T07:49:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.773+0000 7fd3227fc640 1 -- 172.21.15.16:0/671814496 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd314005ad0 con 0x7fd30c0039e0 2024-09-19T07:49:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.774+0000 7fd3227fc640 1 -- 172.21.15.16:0/671814496 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314007070 con 0x7fd30c0039e0 2024-09-19T07:49:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.774+0000 7fd32a0ba640 1 -- 172.21.15.16:0/671814496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd30c0039e0 msgr2=0x7fd30c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.774+0000 7fd32a0ba640 1 --2- 172.21.15.16:0/671814496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd30c0039e0 0x7fd30c003dc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd32408c5f0 tx=0x7fd30c005830 comp rx=0 tx=0).stop 2024-09-19T07:49:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.774+0000 7fd323fff640 1 -- 172.21.15.16:0/671814496 reap_dead start 2024-09-19T07:49:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.774+0000 7fd32a0ba640 1 -- 172.21.15.16:0/671814496 shutdown_connections 2024-09-19T07:49:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.774+0000 7fd32a0ba640 1 -- 172.21.15.16:0/671814496 >> 172.21.15.16:0/671814496 conn(0x7fd32408bfb0 msgr2=0x7fd3240d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.774+0000 7fd32a0ba640 1 -- 172.21.15.16:0/671814496 shutdown_connections 2024-09-19T07:49:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.774+0000 7fd32a0ba640 1 -- 172.21.15.16:0/671814496 wait complete. 2024-09-19T07:49:30.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.775+0000 7fd32a0ba640 1 Processor -- start 2024-09-19T07:49:30.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.775+0000 7fd32a0ba640 1 -- start start 2024-09-19T07:49:30.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.776+0000 7fd32a0ba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd30c0039e0 0x7fd324153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.776+0000 7fd32a0ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32415d800 0x7fd32415b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.776+0000 7fd32a0ba640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3241f4740 0x7fd3241f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.776+0000 7fd32a0ba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3240cece0 con 0x7fd32415d800 2024-09-19T07:49:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.776+0000 7fd32a0ba640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd3240ceb60 con 0x7fd3241f4740 2024-09-19T07:49:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.776+0000 7fd32a0ba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd3240cee60 con 0x7fd30c0039e0 2024-09-19T07:49:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.776+0000 7fd322ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32415d800 0x7fd32415b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.776+0000 7fd322ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32415d800 0x7fd32415b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55940/0 (socket says 172.21.15.16:55940) 2024-09-19T07:49:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.776+0000 7fd322ffd640 1 -- 172.21.15.16:0/3057013037 learned_addr learned my addr 172.21.15.16:0/3057013037 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.776+0000 7fd3237fe640 1 --2- 172.21.15.16:0/3057013037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd30c0039e0 0x7fd324153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.776+0000 7fd323fff640 1 --2- 172.21.15.16:0/3057013037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3241f4740 0x7fd3241f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.777+0000 7fd322ffd640 1 -- 172.21.15.16:0/3057013037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd30c0039e0 msgr2=0x7fd324153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.777+0000 7fd322ffd640 1 --2- 172.21.15.16:0/3057013037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd30c0039e0 0x7fd324153550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.777+0000 7fd322ffd640 1 -- 172.21.15.16:0/3057013037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3241f4740 msgr2=0x7fd3241f4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.777+0000 7fd322ffd640 1 --2- 172.21.15.16:0/3057013037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3241f4740 0x7fd3241f4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.777+0000 7fd322ffd640 1 -- 172.21.15.16:0/3057013037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd324160540 con 0x7fd32415d800 2024-09-19T07:49:30.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.777+0000 7fd3237fe640 1 --2- 172.21.15.16:0/3057013037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd30c0039e0 0x7fd324153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:30.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.777+0000 7fd323fff640 1 --2- 172.21.15.16:0/3057013037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3241f4740 0x7fd3241f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:30.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.777+0000 7fd322ffd640 1 --2- 172.21.15.16:0/3057013037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32415d800 0x7fd32415b420 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd318011910 tx=0x7fd318011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:30.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.777+0000 7fd30bfff640 1 -- 172.21.15.16:0/3057013037 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31800fa90 con 0x7fd32415d800 2024-09-19T07:49:30.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.777+0000 7fd30bfff640 1 -- 172.21.15.16:0/3057013037 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31800fc30 con 0x7fd32415d800 2024-09-19T07:49:30.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.777+0000 7fd32a0ba640 1 -- 172.21.15.16:0/3057013037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd324160820 con 0x7fd32415d800 2024-09-19T07:49:30.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.777+0000 7fd30bfff640 1 -- 172.21.15.16:0/3057013037 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318004030 con 0x7fd32415d800 2024-09-19T07:49:30.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.777+0000 7fd32a0ba640 1 -- 172.21.15.16:0/3057013037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd324160de0 con 0x7fd32415d800 2024-09-19T07:49:30.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.778+0000 7fd32a0ba640 1 -- 172.21.15.16:0/3057013037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3240cf030 con 0x7fd32415d800 2024-09-19T07:49:30.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.778+0000 7fd30bfff640 1 -- 172.21.15.16:0/3057013037 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd318012040 con 0x7fd32415d800 2024-09-19T07:49:30.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.779+0000 7fd30bfff640 1 --2- 172.21.15.16:0/3057013037 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd31006c2e0 0x7fd31006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.779+0000 7fd30bfff640 1 -- 172.21.15.16:0/3057013037 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7fd31808b790 con 0x7fd32415d800 2024-09-19T07:49:30.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.779+0000 7fd3237fe640 1 --2- 172.21.15.16:0/3057013037 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd31006c2e0 0x7fd31006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.780+0000 7fd3237fe640 1 --2- 172.21.15.16:0/3057013037 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd31006c2e0 0x7fd31006e7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd314032000 tx=0x7fd314009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:30.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.783+0000 7fd30bfff640 1 -- 172.21.15.16:0/3057013037 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd318019070 con 0x7fd32415d800 2024-09-19T07:49:30.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.994+0000 7fd32a0ba640 1 -- 172.21.15.16:0/3057013037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd3240b6c30 con 0x7fd32415d800 2024-09-19T07:49:30.995 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-19T07:49:30.995+0000 7f6950fea640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:49:30.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.997+0000 7fd30bfff640 1 -- 172.21.15.16:0/3057013037 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) ==== 161+0+0 (secure 0 0 0) 0x7fd318059e00 con 0x7fd32415d800 2024-09-19T07:49:30.997 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T07:49:31.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.999+0000 7fd32a0ba640 1 -- 172.21.15.16:0/3057013037 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd31006c2e0 msgr2=0x7fd31006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:31.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.999+0000 7fd32a0ba640 1 --2- 172.21.15.16:0/3057013037 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd31006c2e0 0x7fd31006e7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd314032000 tx=0x7fd314009030 comp rx=0 tx=0).stop 2024-09-19T07:49:31.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.999+0000 7fd32a0ba640 1 -- 172.21.15.16:0/3057013037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32415d800 msgr2=0x7fd32415b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:31.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.999+0000 7fd32a0ba640 1 --2- 172.21.15.16:0/3057013037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32415d800 0x7fd32415b420 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd318011910 tx=0x7fd318011dd0 comp rx=0 tx=0).stop 2024-09-19T07:49:31.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.999+0000 7fd32a0ba640 1 -- 172.21.15.16:0/3057013037 shutdown_connections 2024-09-19T07:49:31.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.999+0000 7fd32a0ba640 1 --2- 172.21.15.16:0/3057013037 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd31006c2e0 0x7fd31006e7a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:31.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.999+0000 7fd32a0ba640 1 --2- 172.21.15.16:0/3057013037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3241f4740 0x7fd3241f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:31.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.999+0000 7fd32a0ba640 1 --2- 172.21.15.16:0/3057013037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32415d800 0x7fd32415b420 secure :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd318011910 tx=0x7fd318011dd0 comp rx=0 tx=0).stop 2024-09-19T07:49:31.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.999+0000 7fd32a0ba640 1 --2- 172.21.15.16:0/3057013037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd30c0039e0 0x7fd324153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:31.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:30.999+0000 7fd32a0ba640 1 -- 172.21.15.16:0/3057013037 >> 172.21.15.16:0/3057013037 conn(0x7fd32408bfb0 msgr2=0x7fd3240b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:31.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.000+0000 7fd32a0ba640 1 -- 172.21.15.16:0/3057013037 shutdown_connections 2024-09-19T07:49:31.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.000+0000 7fd32a0ba640 1 -- 172.21.15.16:0/3057013037 wait complete. 2024-09-19T07:49:31.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T07:49:31.038 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-19T07:49:31.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.116+0000 7f5663928640 1 Processor -- start 2024-09-19T07:49:31.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.116+0000 7f5663928640 1 -- start start 2024-09-19T07:49:31.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.117+0000 7f5663928640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f565c15e6d0 0x7f565c0d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:31.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.117+0000 7f5663928640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f565c09aa20 0x7f565c09aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:31.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.117+0000 7f5663928640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f565c15eab0 con 0x7f565c0702d0 2024-09-19T07:49:31.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.117+0000 7f5663928640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f565c0d15b0 con 0x7f565c15e6d0 2024-09-19T07:49:31.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.117+0000 7f5663928640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f565c0d09a0 con 0x7f565c09aa20 2024-09-19T07:49:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.117+0000 7f56616c4640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f565c15e6d0 0x7f565c0d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.117+0000 7f5661ec5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f565c09aa20 0x7f565c09aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.117+0000 7f5660ec3640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f565c0702d0 0x7f565c071130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:53564/0 (socket says 172.21.15.16:53564) 2024-09-19T07:49:31.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.117+0000 7f5660ec3640 1 -- 172.21.15.16:0/3417571440 learned_addr learned my addr 172.21.15.16:0/3417571440 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:31.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.118+0000 7f5661ec5640 1 -- 172.21.15.16:0/3417571440 >> v2:172.21.15.146:3300/0 conn(0x7f565c15e6d0 msgr2=0x7f565c0d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:31.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.118+0000 7f5661ec5640 1 --2- 172.21.15.16:0/3417571440 >> v2:172.21.15.146:3300/0 conn(0x7f565c15e6d0 0x7f565c0d1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:31.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.118+0000 7f5661ec5640 1 -- 172.21.15.16:0/3417571440 >> v1:172.21.15.146:6789/0 conn(0x7f565c0702d0 legacy=0x7f565c071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:31.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.118+0000 7f5653fff640 1 -- 172.21.15.16:0/3417571440 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2021800943 0 0) 0x7f565c15eab0 con 0x7f565c0702d0 2024-09-19T07:49:31.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.118+0000 7f5661ec5640 1 -- 172.21.15.16:0/3417571440 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565c0d0c80 con 0x7f565c09aa20 2024-09-19T07:49:31.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.118+0000 7f56616c4640 1 --2- 172.21.15.16:0/3417571440 >> v2:172.21.15.146:3300/0 conn(0x7f565c15e6d0 0x7f565c0d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:49:31.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.119+0000 7f5661ec5640 1 --2- 172.21.15.16:0/3417571440 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f565c09aa20 0x7f565c09aeb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5654009d90 tx=0x7f565402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a4efa3284dd8d54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:31.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.119+0000 7f5653fff640 1 -- 172.21.15.16:0/3417571440 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654031600 con 0x7f565c09aa20 2024-09-19T07:49:31.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.119+0000 7f5653fff640 1 -- 172.21.15.16:0/3417571440 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56540317a0 con 0x7f565c09aa20 2024-09-19T07:49:31.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.119+0000 7f5653fff640 1 -- 172.21.15.16:0/3417571440 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654031b70 con 0x7f565c09aa20 2024-09-19T07:49:31.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.119+0000 7f5663928640 1 -- 172.21.15.16:0/3417571440 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f565c09aa20 msgr2=0x7f565c09aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.119+0000 7f5663928640 1 --2- 172.21.15.16:0/3417571440 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f565c09aa20 0x7f565c09aeb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5654009d90 tx=0x7f565402a5d0 comp rx=0 tx=0).stop 2024-09-19T07:49:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.119+0000 7f5663928640 1 -- 172.21.15.16:0/3417571440 shutdown_connections 2024-09-19T07:49:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.119+0000 7f5663928640 1 --2- 172.21.15.16:0/3417571440 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f565c09aa20 0x7f565c09aeb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.119+0000 7f5663928640 1 --2- 172.21.15.16:0/3417571440 >> v2:172.21.15.146:3300/0 conn(0x7f565c15e6d0 0x7f565c0d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.119+0000 7f5663928640 1 -- 172.21.15.16:0/3417571440 >> 172.21.15.16:0/3417571440 conn(0x7f565c0d23a0 msgr2=0x7f565c0d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.119+0000 7f5663928640 1 -- 172.21.15.16:0/3417571440 shutdown_connections 2024-09-19T07:49:31.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.119+0000 7f5663928640 1 -- 172.21.15.16:0/3417571440 wait complete. 2024-09-19T07:49:31.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.121+0000 7f5663928640 1 Processor -- start 2024-09-19T07:49:31.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.121+0000 7f5663928640 1 -- start start 2024-09-19T07:49:31.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.121+0000 7f5663928640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f565c0702d0 0x7f565c0cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:31.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.121+0000 7f5663928640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f565c09aa20 0x7f565c0c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:31.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.121+0000 7f5663928640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f565c15e6d0 0x7f565c0b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:31.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.121+0000 7f5663928640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f565c0d0670 con 0x7f565c09aa20 2024-09-19T07:49:31.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.121+0000 7f5663928640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f565c0d1240 con 0x7f565c15e6d0 2024-09-19T07:49:31.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.121+0000 7f5663928640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f565c06cce0 con 0x7f565c0702d0 2024-09-19T07:49:31.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.121+0000 7f5660ec3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f565c09aa20 0x7f565c0c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:31.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.121+0000 7f56616c4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f565c0702d0 0x7f565c0cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:31.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.122+0000 7f5661ec5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f565c15e6d0 0x7f565c0b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:31.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.122+0000 7f5660ec3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f565c09aa20 0x7f565c0c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55962/0 (socket says 172.21.15.16:55962) 2024-09-19T07:49:31.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.122+0000 7f56616c4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f565c0702d0 0x7f565c0cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:60550/0 (socket says 172.21.15.16:60550) 2024-09-19T07:49:31.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.122+0000 7f5660ec3640 1 -- 172.21.15.16:0/2197568076 learned_addr learned my addr 172.21.15.16:0/2197568076 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:31.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.122+0000 7f5660ec3640 1 -- 172.21.15.16:0/2197568076 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f565c0702d0 msgr2=0x7f565c0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:31.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.122+0000 7f5660ec3640 1 --2- 172.21.15.16:0/2197568076 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f565c0702d0 0x7f565c0cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:31.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.122+0000 7f5660ec3640 1 -- 172.21.15.16:0/2197568076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f565c15e6d0 msgr2=0x7f565c0b7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:31.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.122+0000 7f5660ec3640 1 --2- 172.21.15.16:0/2197568076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f565c15e6d0 0x7f565c0b7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:31.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.122+0000 7f5660ec3640 1 -- 172.21.15.16:0/2197568076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565c0b6c90 con 0x7f565c09aa20 2024-09-19T07:49:31.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.122+0000 7f5660ec3640 1 --2- 172.21.15.16:0/2197568076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f565c09aa20 0x7f565c0c9fd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f564c00c460 tx=0x7f564c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:31.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.122+0000 7f5651ffb640 1 -- 172.21.15.16:0/2197568076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c0074e0 con 0x7f565c09aa20 2024-09-19T07:49:31.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.122+0000 7f5651ffb640 1 -- 172.21.15.16:0/2197568076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f564c0047e0 con 0x7f565c09aa20 2024-09-19T07:49:31.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.122+0000 7f5663928640 1 -- 172.21.15.16:0/2197568076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f565c0c9160 con 0x7f565c09aa20 2024-09-19T07:49:31.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.122+0000 7f5651ffb640 1 -- 172.21.15.16:0/2197568076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c004bd0 con 0x7f565c09aa20 2024-09-19T07:49:31.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.123+0000 7f5663928640 1 -- 172.21.15.16:0/2197568076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f565c0c6170 con 0x7f565c09aa20 2024-09-19T07:49:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.123+0000 7f5663928640 1 -- 172.21.15.16:0/2197568076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f565c069580 con 0x7f565c09aa20 2024-09-19T07:49:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.124+0000 7f5651ffb640 1 -- 172.21.15.16:0/2197568076 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f564c011070 con 0x7f565c09aa20 2024-09-19T07:49:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.124+0000 7f5651ffb640 1 --2- 172.21.15.16:0/2197568076 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f564006c2b0 0x7f564006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.124+0000 7f5651ffb640 1 -- 172.21.15.16:0/2197568076 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6187+0+0 (secure 0 0 0) 0x7f564c08b090 con 0x7f565c09aa20 2024-09-19T07:49:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.125+0000 7f56616c4640 1 --2- 172.21.15.16:0/2197568076 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f564006c2b0 0x7f564006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.125+0000 7f56616c4640 1 --2- 172.21.15.16:0/2197568076 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f564006c2b0 0x7f564006e770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5658009560 tx=0x7f5658009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:31.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.129+0000 7f5651ffb640 1 -- 172.21.15.16:0/2197568076 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f564c058fd0 con 0x7f565c09aa20 2024-09-19T07:49:31.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.343+0000 7f5663928640 1 -- 172.21.15.16:0/2197568076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f565c0d31d0 con 0x7f565c09aa20 2024-09-19T07:49:31.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:31.998+0000 7f5651ffb640 1 -- 172.21.15.16:0/2197568076 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) ==== 104+0+0 (secure 0 0 0) 0x7f564c05d2d0 con 0x7f565c09aa20 2024-09-19T07:49:32.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.001+0000 7f5663928640 1 -- 172.21.15.16:0/2197568076 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f564006c2b0 msgr2=0x7f564006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:32.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.001+0000 7f5663928640 1 --2- 172.21.15.16:0/2197568076 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f564006c2b0 0x7f564006e770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5658009560 tx=0x7f5658009190 comp rx=0 tx=0).stop 2024-09-19T07:49:32.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.001+0000 7f5663928640 1 -- 172.21.15.16:0/2197568076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f565c09aa20 msgr2=0x7f565c0c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:32.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.001+0000 7f5663928640 1 --2- 172.21.15.16:0/2197568076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f565c09aa20 0x7f565c0c9fd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f564c00c460 tx=0x7f564c00c920 comp rx=0 tx=0).stop 2024-09-19T07:49:32.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.001+0000 7f5663928640 1 -- 172.21.15.16:0/2197568076 shutdown_connections 2024-09-19T07:49:32.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.001+0000 7f5663928640 1 --2- 172.21.15.16:0/2197568076 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f564006c2b0 0x7f564006e770 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:32.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.001+0000 7f5663928640 1 --2- 172.21.15.16:0/2197568076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f565c15e6d0 0x7f565c0b7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:32.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.001+0000 7f5663928640 1 --2- 172.21.15.16:0/2197568076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f565c09aa20 0x7f565c0c9fd0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:32.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.001+0000 7f5663928640 1 --2- 172.21.15.16:0/2197568076 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f565c0702d0 0x7f565c0cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:32.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.001+0000 7f5663928640 1 -- 172.21.15.16:0/2197568076 >> 172.21.15.16:0/2197568076 conn(0x7f565c0d23a0 msgr2=0x7f565c0cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:32.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.001+0000 7f5663928640 1 -- 172.21.15.16:0/2197568076 shutdown_connections 2024-09-19T07:49:32.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.001+0000 7f5663928640 1 -- 172.21.15.16:0/2197568076 wait complete. 2024-09-19T07:49:32.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T07:49:32.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.168+0000 7f45f756d640 1 Processor -- start 2024-09-19T07:49:32.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.168+0000 7f45f756d640 1 -- start start 2024-09-19T07:49:32.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.169+0000 7f45f756d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45f015e630 0x7f45f00d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:32.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.169+0000 7f45f756d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f45f00d5200 0x7f45f00b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:32.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.169+0000 7f45f756d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45f00d35a0 con 0x7f45f00b6500 2024-09-19T07:49:32.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.169+0000 7f45f756d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f45f006d3a0 con 0x7f45f00d5200 2024-09-19T07:49:32.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.169+0000 7f45f756d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f45f00d3880 con 0x7f45f015e630 2024-09-19T07:49:32.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.169+0000 7f45f5b0a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f45f00b6500 0x7f45f00cefe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56220/0 (socket says 172.21.15.16:56220) 2024-09-19T07:49:32.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.169+0000 7f45f5b0a640 1 -- 172.21.15.16:0/3167966368 learned_addr learned my addr 172.21.15.16:0/3167966368 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:32.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45f5309640 1 --2- 172.21.15.16:0/3167966368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45f015e630 0x7f45f00d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:32.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45f4b08640 1 --2- 172.21.15.16:0/3167966368 >> v2:172.21.15.146:3300/0 conn(0x7f45f00d5200 0x7f45f00b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:32.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45effff640 1 -- 172.21.15.16:0/3167966368 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1706779810 0 0) 0x7f45f00d35a0 con 0x7f45f00b6500 2024-09-19T07:49:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45effff640 1 -- 172.21.15.16:0/3167966368 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45d8003520 con 0x7f45f00b6500 2024-09-19T07:49:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45effff640 1 -- 172.21.15.16:0/3167966368 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4049891825 0 0) 0x7f45d8003520 con 0x7f45f00b6500 2024-09-19T07:49:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45effff640 1 -- 172.21.15.16:0/3167966368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45f015e630 msgr2=0x7f45f00d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45effff640 1 --2- 172.21.15.16:0/3167966368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45f015e630 0x7f45f00d4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45effff640 1 -- 172.21.15.16:0/3167966368 >> v2:172.21.15.146:3300/0 conn(0x7f45f00d5200 msgr2=0x7f45f00b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45effff640 1 --2- 172.21.15.16:0/3167966368 >> v2:172.21.15.146:3300/0 conn(0x7f45f00d5200 0x7f45f00b5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45effff640 1 -- 172.21.15.16:0/3167966368 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f006d030 con 0x7f45f00b6500 2024-09-19T07:49:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45effff640 1 -- 172.21.15.16:0/3167966368 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f45e8001670 con 0x7f45f00b6500 2024-09-19T07:49:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45effff640 1 -- 172.21.15.16:0/3167966368 >> v1:172.21.15.16:6789/0 conn(0x7f45f00b6500 legacy=0x7f45f00cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45effff640 1 --2- 172.21.15.16:0/3167966368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45f015e630 0x7f45d80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45effff640 1 --2- 172.21.15.16:0/3167966368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d8003f30 0x7f45d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45effff640 1 --2- 172.21.15.16:0/3167966368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45d8004d40 0x7f45d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:32.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45effff640 1 -- 172.21.15.16:0/3167966368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f45f00d3880 con 0x7f45d8003f30 2024-09-19T07:49:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45effff640 1 -- 172.21.15.16:0/3167966368 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f45d800b9c0 con 0x7f45d8004d40 2024-09-19T07:49:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45effff640 1 -- 172.21.15.16:0/3167966368 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f45d800bb70 con 0x7f45f015e630 2024-09-19T07:49:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45f4b08640 1 --2- 172.21.15.16:0/3167966368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d8003f30 0x7f45d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45f5309640 1 --2- 172.21.15.16:0/3167966368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45f015e630 0x7f45d80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45f5b0a640 1 --2- 172.21.15.16:0/3167966368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45d8004d40 0x7f45d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45f4b08640 1 -- 172.21.15.16:0/3167966368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45f015e630 msgr2=0x7f45d80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45f4b08640 1 --2- 172.21.15.16:0/3167966368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45f015e630 0x7f45d80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45f4b08640 1 -- 172.21.15.16:0/3167966368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45d8004d40 msgr2=0x7f45d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45f4b08640 1 --2- 172.21.15.16:0/3167966368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45d8004d40 0x7f45d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.170+0000 7f45f4b08640 1 -- 172.21.15.16:0/3167966368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d8003520 con 0x7f45d8003f30 2024-09-19T07:49:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.171+0000 7f45f4b08640 1 --2- 172.21.15.16:0/3167966368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d8003f30 0x7f45d8004800 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f45f00d4510 tx=0x7f45d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2c8bade45dce45 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.171+0000 7f45effff640 1 -- 172.21.15.16:0/3167966368 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4036070 con 0x7f45d8003f30 2024-09-19T07:49:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.171+0000 7f45effff640 1 -- 172.21.15.16:0/3167966368 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e4031400 con 0x7f45d8003f30 2024-09-19T07:49:32.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.171+0000 7f45effff640 1 -- 172.21.15.16:0/3167966368 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4009690 con 0x7f45d8003f30 2024-09-19T07:49:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.172+0000 7f45f756d640 1 -- 172.21.15.16:0/3167966368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d8003f30 msgr2=0x7f45d8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.172+0000 7f45f756d640 1 --2- 172.21.15.16:0/3167966368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d8003f30 0x7f45d8004800 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f45f00d4510 tx=0x7f45d8005830 comp rx=0 tx=0).stop 2024-09-19T07:49:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.172+0000 7f45f5b0a640 1 -- 172.21.15.16:0/3167966368 reap_dead start 2024-09-19T07:49:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.172+0000 7f45f756d640 1 -- 172.21.15.16:0/3167966368 shutdown_connections 2024-09-19T07:49:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.172+0000 7f45f756d640 1 -- 172.21.15.16:0/3167966368 >> 172.21.15.16:0/3167966368 conn(0x7f45f008bfb0 msgr2=0x7f45f00d4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.172+0000 7f45f756d640 1 -- 172.21.15.16:0/3167966368 shutdown_connections 2024-09-19T07:49:32.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.172+0000 7f45f756d640 1 -- 172.21.15.16:0/3167966368 wait complete. 2024-09-19T07:49:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.173+0000 7f45f756d640 1 Processor -- start 2024-09-19T07:49:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.173+0000 7f45f756d640 1 -- start start 2024-09-19T07:49:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.174+0000 7f45f756d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45d8003f30 0x7f45f0151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.174+0000 7f45f756d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45f00a1aa0 0x7f45f00a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:32.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.174+0000 7f45f756d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45f00b84f0 0x7f45f00b8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.174+0000 7f45f756d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f45f00d32d0 con 0x7f45f00a1aa0 2024-09-19T07:49:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.174+0000 7f45f756d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f45f00d3150 con 0x7f45f00b84f0 2024-09-19T07:49:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.174+0000 7f45f756d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f45f006e840 con 0x7f45d8003f30 2024-09-19T07:49:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.174+0000 7f45f4b08640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45f00a1aa0 0x7f45f00a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.174+0000 7f45f4b08640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45f00a1aa0 0x7f45f00a1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55974/0 (socket says 172.21.15.16:55974) 2024-09-19T07:49:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.174+0000 7f45f4b08640 1 -- 172.21.15.16:0/227599009 learned_addr learned my addr 172.21.15.16:0/227599009 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.174+0000 7f45f5309640 1 --2- 172.21.15.16:0/227599009 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45d8003f30 0x7f45f0151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.174+0000 7f45f5b0a640 1 --2- 172.21.15.16:0/227599009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45f00b84f0 0x7f45f00b8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.174+0000 7f45f4b08640 1 -- 172.21.15.16:0/227599009 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45d8003f30 msgr2=0x7f45f0151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.174+0000 7f45f4b08640 1 --2- 172.21.15.16:0/227599009 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45d8003f30 0x7f45f0151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.174+0000 7f45f4b08640 1 -- 172.21.15.16:0/227599009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45f00b84f0 msgr2=0x7f45f00b8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.174+0000 7f45f4b08640 1 --2- 172.21.15.16:0/227599009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45f00b84f0 0x7f45f00b8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.174+0000 7f45f4b08640 1 -- 172.21.15.16:0/227599009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f009f270 con 0x7f45f00a1aa0 2024-09-19T07:49:32.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.174+0000 7f45f5309640 1 --2- 172.21.15.16:0/227599009 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45d8003f30 0x7f45f0151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.175+0000 7f45f4b08640 1 --2- 172.21.15.16:0/227599009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45f00a1aa0 0x7f45f00a1f00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f45e4034040 tx=0x7f45e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.175+0000 7f45edffb640 1 -- 172.21.15.16:0/227599009 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4040020 con 0x7f45f00a1aa0 2024-09-19T07:49:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.175+0000 7f45edffb640 1 -- 172.21.15.16:0/227599009 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e403b6a0 con 0x7f45f00a1aa0 2024-09-19T07:49:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.175+0000 7f45f756d640 1 -- 172.21.15.16:0/227599009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f009f550 con 0x7f45f00a1aa0 2024-09-19T07:49:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.175+0000 7f45edffb640 1 -- 172.21.15.16:0/227599009 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4036040 con 0x7f45f00a1aa0 2024-09-19T07:49:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.175+0000 7f45f756d640 1 -- 172.21.15.16:0/227599009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f01605a0 con 0x7f45f00a1aa0 2024-09-19T07:49:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.176+0000 7f45f756d640 1 -- 172.21.15.16:0/227599009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45f009fa70 con 0x7f45f00a1aa0 2024-09-19T07:49:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.176+0000 7f45edffb640 1 -- 172.21.15.16:0/227599009 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f45e4043020 con 0x7f45f00a1aa0 2024-09-19T07:49:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.177+0000 7f45edffb640 1 --2- 172.21.15.16:0/227599009 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f45dc06c2b0 0x7f45dc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.177+0000 7f45edffb640 1 -- 172.21.15.16:0/227599009 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6187+0+0 (secure 0 0 0) 0x7f45e40abde0 con 0x7f45f00a1aa0 2024-09-19T07:49:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.177+0000 7f45f5309640 1 --2- 172.21.15.16:0/227599009 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f45dc06c2b0 0x7f45dc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.178+0000 7f45f5309640 1 --2- 172.21.15.16:0/227599009 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f45dc06c2b0 0x7f45dc06e770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f45e0013370 tx=0x7f45e000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:32.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.181+0000 7f45edffb640 1 -- 172.21.15.16:0/227599009 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f45e404a030 con 0x7f45f00a1aa0 2024-09-19T07:49:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:32.361+0000 7f45f756d640 1 -- 172.21.15.16:0/227599009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f45f0160390 con 0x7f45f00a1aa0 2024-09-19T07:49:33.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.002+0000 7f45edffb640 1 -- 172.21.15.16:0/227599009 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v72) ==== 181+0+0 (secure 0 0 0) 0x7f45e4080020 con 0x7f45f00a1aa0 2024-09-19T07:49:33.002 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-19T07:49:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.006+0000 7f45f756d640 1 -- 172.21.15.16:0/227599009 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f45dc06c2b0 msgr2=0x7f45dc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.006+0000 7f45f756d640 1 --2- 172.21.15.16:0/227599009 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f45dc06c2b0 0x7f45dc06e770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f45e0013370 tx=0x7f45e000b040 comp rx=0 tx=0).stop 2024-09-19T07:49:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.006+0000 7f45f756d640 1 -- 172.21.15.16:0/227599009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45f00a1aa0 msgr2=0x7f45f00a1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.006+0000 7f45f756d640 1 --2- 172.21.15.16:0/227599009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45f00a1aa0 0x7f45f00a1f00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f45e4034040 tx=0x7f45e4004730 comp rx=0 tx=0).stop 2024-09-19T07:49:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.006+0000 7f45f756d640 1 -- 172.21.15.16:0/227599009 shutdown_connections 2024-09-19T07:49:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.006+0000 7f45f756d640 1 --2- 172.21.15.16:0/227599009 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f45dc06c2b0 0x7f45dc06e770 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.006+0000 7f45f756d640 1 --2- 172.21.15.16:0/227599009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f45f00b84f0 0x7f45f00b8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.006+0000 7f45f756d640 1 --2- 172.21.15.16:0/227599009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45f00a1aa0 0x7f45f00a1f00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.006+0000 7f45f756d640 1 --2- 172.21.15.16:0/227599009 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f45d8003f30 0x7f45f0151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.006+0000 7f45f756d640 1 -- 172.21.15.16:0/227599009 >> 172.21.15.16:0/227599009 conn(0x7f45f008bfb0 msgr2=0x7f45f00b9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.007+0000 7f45f756d640 1 -- 172.21.15.16:0/227599009 shutdown_connections 2024-09-19T07:49:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.007+0000 7f45f756d640 1 -- 172.21.15.16:0/227599009 wait complete. 2024-09-19T07:49:33.022 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T07:49:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.133+0000 7f4a08742640 1 Processor -- start 2024-09-19T07:49:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.133+0000 7f4a08742640 1 -- start start 2024-09-19T07:49:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.134+0000 7f4a08742640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4a0015e770 0x7f4a000d3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.134+0000 7f4a08742640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a000b6680 0x7f4a000d3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.134+0000 7f4a08742640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a000cecc0 con 0x7f4a000cfd60 2024-09-19T07:49:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.134+0000 7f4a08742640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f4a000cf8d0 con 0x7f4a000b6680 2024-09-19T07:49:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.134+0000 7f4a08742640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4a000cefa0 con 0x7f4a0015e770 2024-09-19T07:49:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.134+0000 7f4a06cdf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a000b6680 0x7f4a000d3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.134+0000 7f4a06cdf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a000b6680 0x7f4a000d3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53328/0 (socket says 172.21.15.16:53328) 2024-09-19T07:49:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.134+0000 7f4a06cdf640 1 -- 172.21.15.16:0/2855761706 learned_addr learned my addr 172.21.15.16:0/2855761706 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.134+0000 7f4a06cdf640 1 -- 172.21.15.16:0/2855761706 >> v2:172.21.15.146:3300/0 conn(0x7f4a0015e770 msgr2=0x7f4a000d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.134+0000 7f4a06cdf640 1 --2- 172.21.15.16:0/2855761706 >> v2:172.21.15.146:3300/0 conn(0x7f4a0015e770 0x7f4a000d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.134+0000 7f4a06cdf640 1 -- 172.21.15.16:0/2855761706 >> v1:172.21.15.146:6789/0 conn(0x7f4a000cfd60 legacy=0x7f4a000b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.134+0000 7f4a06cdf640 1 -- 172.21.15.16:0/2855761706 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a000cf280 con 0x7f4a000b6680 2024-09-19T07:49:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.135+0000 7f4a06cdf640 1 --2- 172.21.15.16:0/2855761706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a000b6680 0x7f4a000d3660 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4a000d1fc0 tx=0x7f49f802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7a87a9bad94d15 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.135+0000 7f4a054dc640 1 -- 172.21.15.16:0/2855761706 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f8036070 con 0x7f4a000b6680 2024-09-19T07:49:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.135+0000 7f4a054dc640 1 -- 172.21.15.16:0/2855761706 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49f8031600 con 0x7f4a000b6680 2024-09-19T07:49:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.135+0000 7f4a054dc640 1 -- 172.21.15.16:0/2855761706 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f80319d0 con 0x7f4a000b6680 2024-09-19T07:49:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.136+0000 7f4a08742640 1 -- 172.21.15.16:0/2855761706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a000b6680 msgr2=0x7f4a000d3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.136+0000 7f4a08742640 1 --2- 172.21.15.16:0/2855761706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a000b6680 0x7f4a000d3660 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4a000d1fc0 tx=0x7f49f802a6f0 comp rx=0 tx=0).stop 2024-09-19T07:49:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.136+0000 7f4a08742640 1 -- 172.21.15.16:0/2855761706 shutdown_connections 2024-09-19T07:49:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.136+0000 7f4a08742640 1 --2- 172.21.15.16:0/2855761706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a000b6680 0x7f4a000d3660 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.136+0000 7f4a08742640 1 --2- 172.21.15.16:0/2855761706 >> v2:172.21.15.146:3300/0 conn(0x7f4a0015e770 0x7f4a000d3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.136+0000 7f4a08742640 1 -- 172.21.15.16:0/2855761706 >> 172.21.15.16:0/2855761706 conn(0x7f4a0008bfb0 msgr2=0x7f4a000d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.136+0000 7f4a08742640 1 -- 172.21.15.16:0/2855761706 shutdown_connections 2024-09-19T07:49:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.136+0000 7f4a08742640 1 -- 172.21.15.16:0/2855761706 wait complete. 2024-09-19T07:49:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.137+0000 7f4a08742640 1 Processor -- start 2024-09-19T07:49:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.137+0000 7f4a08742640 1 -- start start 2024-09-19T07:49:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.138+0000 7f4a08742640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a000b6680 0x7f4a000b7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.138+0000 7f4a08742640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a000cfd60 0x7f4a00157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.138+0000 7f4a08742640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a0015e770 0x7f4a00158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.138+0000 7f4a08742640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4a00168270 con 0x7f4a000b6680 2024-09-19T07:49:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.138+0000 7f4a08742640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4a001680f0 con 0x7f4a0015e770 2024-09-19T07:49:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.138+0000 7f4a08742640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4a001683f0 con 0x7f4a000cfd60 2024-09-19T07:49:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.138+0000 7f4a064de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a000b6680 0x7f4a000b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.138+0000 7f4a064de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a000b6680 0x7f4a000b7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53338/0 (socket says 172.21.15.16:53338) 2024-09-19T07:49:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.138+0000 7f4a064de640 1 -- 172.21.15.16:0/1375007846 learned_addr learned my addr 172.21.15.16:0/1375007846 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.138+0000 7f4a06cdf640 1 --2- 172.21.15.16:0/1375007846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a0015e770 0x7f4a00158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.138+0000 7f4a05cdd640 1 --2- 172.21.15.16:0/1375007846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a000cfd60 0x7f4a00157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.138+0000 7f4a064de640 1 -- 172.21.15.16:0/1375007846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a000cfd60 msgr2=0x7f4a00157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.138+0000 7f4a064de640 1 --2- 172.21.15.16:0/1375007846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a000cfd60 0x7f4a00157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.138+0000 7f4a064de640 1 -- 172.21.15.16:0/1375007846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a0015e770 msgr2=0x7f4a00158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.138+0000 7f4a064de640 1 --2- 172.21.15.16:0/1375007846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a0015e770 0x7f4a00158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.138+0000 7f4a064de640 1 -- 172.21.15.16:0/1375007846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a001557a0 con 0x7f4a000b6680 2024-09-19T07:49:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.139+0000 7f4a064de640 1 --2- 172.21.15.16:0/1375007846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a000b6680 0x7f4a000b7550 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00b410 tx=0x7f49fc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.139+0000 7f49f6ffd640 1 -- 172.21.15.16:0/1375007846 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc015020 con 0x7f4a000b6680 2024-09-19T07:49:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.139+0000 7f49f6ffd640 1 -- 172.21.15.16:0/1375007846 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49fc011850 con 0x7f4a000b6680 2024-09-19T07:49:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.139+0000 7f4a08742640 1 -- 172.21.15.16:0/1375007846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a000b8280 con 0x7f4a000b6680 2024-09-19T07:49:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.139+0000 7f49f6ffd640 1 -- 172.21.15.16:0/1375007846 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc011c40 con 0x7f4a000b6680 2024-09-19T07:49:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.139+0000 7f4a08742640 1 -- 172.21.15.16:0/1375007846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a00155a40 con 0x7f4a000b6680 2024-09-19T07:49:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.140+0000 7f4a08742640 1 -- 172.21.15.16:0/1375007846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a000b6aa0 con 0x7f4a000b6680 2024-09-19T07:49:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.140+0000 7f49f6ffd640 1 -- 172.21.15.16:0/1375007846 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f49fc018020 con 0x7f4a000b6680 2024-09-19T07:49:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.141+0000 7f49f6ffd640 1 --2- 172.21.15.16:0/1375007846 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f49e406c2b0 0x7f49e406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.141+0000 7f49f6ffd640 1 -- 172.21.15.16:0/1375007846 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5749+0+0 (secure 0 0 0) 0x7f49fc08b310 con 0x7f4a000b6680 2024-09-19T07:49:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.141+0000 7f4a05cdd640 1 --2- 172.21.15.16:0/1375007846 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f49e406c2b0 0x7f49e406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.142+0000 7f4a05cdd640 1 --2- 172.21.15.16:0/1375007846 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f49e406c2b0 0x7f49e406e770 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f49f0007fd0 tx=0x7f49f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:33.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.145+0000 7f49f6ffd640 1 -- 172.21.15.16:0/1375007846 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4a000b6aa0 con 0x7f4a000b6680 2024-09-19T07:49:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:33.320+0000 7f4a08742640 1 -- 172.21.15.16:0/1375007846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4a000b5f70 con 0x7f4a000b6680 2024-09-19T07:49:34.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.009+0000 7f49f6ffd640 1 -- 172.21.15.16:0/1375007846 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v73) ==== 169+0+0 (secure 0 0 0) 0x7f49fc01f020 con 0x7f4a000b6680 2024-09-19T07:49:34.010 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-19T07:49:34.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.013+0000 7f4a08742640 1 -- 172.21.15.16:0/1375007846 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f49e406c2b0 msgr2=0x7f49e406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:34.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.013+0000 7f4a08742640 1 --2- 172.21.15.16:0/1375007846 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f49e406c2b0 0x7f49e406e770 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f49f0007fd0 tx=0x7f49f0009040 comp rx=0 tx=0).stop 2024-09-19T07:49:34.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.013+0000 7f4a08742640 1 -- 172.21.15.16:0/1375007846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a000b6680 msgr2=0x7f4a000b7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:34.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.013+0000 7f4a08742640 1 --2- 172.21.15.16:0/1375007846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a000b6680 0x7f4a000b7550 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00b410 tx=0x7f49fc00b8d0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.013+0000 7f4a08742640 1 -- 172.21.15.16:0/1375007846 shutdown_connections 2024-09-19T07:49:34.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.013+0000 7f4a08742640 1 --2- 172.21.15.16:0/1375007846 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f49e406c2b0 0x7f49e406e770 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.013+0000 7f4a08742640 1 --2- 172.21.15.16:0/1375007846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a0015e770 0x7f4a00158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.013+0000 7f4a08742640 1 --2- 172.21.15.16:0/1375007846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a000cfd60 0x7f4a00157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.013+0000 7f4a08742640 1 --2- 172.21.15.16:0/1375007846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a000b6680 0x7f4a000b7550 secure :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00b410 tx=0x7f49fc00b8d0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.013+0000 7f4a08742640 1 -- 172.21.15.16:0/1375007846 >> 172.21.15.16:0/1375007846 conn(0x7f4a0008bfb0 msgr2=0x7f4a0006e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:34.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.014+0000 7f4a08742640 1 -- 172.21.15.16:0/1375007846 shutdown_connections 2024-09-19T07:49:34.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.014+0000 7f4a08742640 1 -- 172.21.15.16:0/1375007846 wait complete. 2024-09-19T07:49:34.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T07:49:34.033 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T07:49:34.079 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-09-19T07:49:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2024-09-19T07:49:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T07:49:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2024-09-19T07:49:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2024-09-19T07:49:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2024-09-19T07:49:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2024-09-19T07:49:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2024-09-19T07:49:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2024-09-19T07:49:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2024-09-19T07:49:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T07:49:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2024-09-19T07:49:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T07:49:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T07:49:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2024-09-19T07:49:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2024-09-19T07:49:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2024-09-19T07:49:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2024-09-19T07:49:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2024-09-19T07:49:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2024-09-19T07:49:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2024-09-19T07:49:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2024-09-19T07:49:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T07:49:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T07:49:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2024-09-19T07:49:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T07:49:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2024-09-19T07:49:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2024-09-19T07:49:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2024-09-19T07:49:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2024-09-19T07:49:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2024-09-19T07:49:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-09-19T07:49:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb 2024-09-19T07:49:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.083 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2024-09-19T07:49:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2024-09-19T07:49:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2024-09-19T07:49:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2024-09-19T07:49:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2024-09-19T07:49:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2024-09-19T07:49:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2024-09-19T07:49:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2024-09-19T07:49:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2024-09-19T07:49:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2024-09-19T07:49:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2024-09-19T07:49:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2024-09-19T07:49:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2024-09-19T07:49:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2024-09-19T07:49:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2024-09-19T07:49:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2024-09-19T07:49:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2024-09-19T07:49:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2024-09-19T07:49:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2024-09-19T07:49:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2024-09-19T07:49:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2024-09-19T07:49:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2024-09-19T07:49:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2024-09-19T07:49:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2024-09-19T07:49:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:49:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2024-09-19T07:49:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2024-09-19T07:49:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T07:49:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2024-09-19T07:49:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2024-09-19T07:49:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2024-09-19T07:49:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2024-09-19T07:49:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-09-19T07:49:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.086 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2024-09-19T07:49:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:49:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2024-09-19T07:49:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-19T07:49:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-19T07:49:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:49:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:49:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2024-09-19T07:49:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-09-19T07:49:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.087 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2024-09-19T07:49:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:49:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-19T07:49:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2024-09-19T07:49:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-19T07:49:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2024-09-19T07:49:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2024-09-19T07:49:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:49:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2024-09-19T07:49:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2024-09-19T07:49:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:49:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T07:49:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T07:49:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T07:49:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T07:49:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T07:49:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T07:49:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2024-09-19T07:49:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2024-09-19T07:49:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2024-09-19T07:49:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: mon down mkfs grace = 300 2024-09-19T07:49:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2024-09-19T07:49:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on pool no app = False 2024-09-19T07:49:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:34.089 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2024-09-19T07:49:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2024-09-19T07:49:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2024-09-19T07:49:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2024-09-19T07:49:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T07:49:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T07:49:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2024-09-19T07:49:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-09-19T07:49:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:49:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2024-09-19T07:49:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-09-19T07:49:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-09-19T07:49:34.090 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2024-09-19T07:49:34.090 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2024-09-19T07:49:34.090 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2024-09-19T07:49:34.090 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-09-19T07:49:34.090 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-09-19T07:49:34.090 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-09-19T07:49:34.090 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-09-19T07:49:34.090 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2024-09-19T07:49:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2024-09-19T07:49:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2024-09-19T07:49:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:49:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2024-09-19T07:49:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2024-09-19T07:49:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2024-09-19T07:49:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = True 2024-09-19T07:49:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-09-19T07:49:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-09-19T07:49:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-09-19T07:49:34.092 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2024-09-19T07:49:34.092 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2024-09-19T07:49:34.092 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T07:49:34.092 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2024-09-19T07:49:34.092 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.092 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2024-09-19T07:49:34.092 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.092 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2024-09-19T07:49:34.092 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.092 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2024-09-19T07:49:34.092 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2024-09-19T07:49:34.092 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.093 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2024-09-19T07:49:34.093 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2024-09-19T07:49:34.093 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.093 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T07:49:34.093 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2024-09-19T07:49:34.093 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.093 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T07:49:34.093 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T07:49:34.093 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.094 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2024-09-19T07:49:34.094 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.094 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2024-09-19T07:49:34.094 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2024-09-19T07:49:34.094 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2024-09-19T07:49:34.094 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2024-09-19T07:49:34.094 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2024-09-19T07:49:34.094 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2024-09-19T07:49:34.094 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2024-09-19T07:49:34.094 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T07:49:34.094 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T07:49:34.094 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2024-09-19T07:49:34.094 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.095 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T07:49:34.095 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.095 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2024-09-19T07:49:34.095 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.095 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2024-09-19T07:49:34.095 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.095 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2024-09-19T07:49:34.095 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2024-09-19T07:49:34.095 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2024-09-19T07:49:34.095 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.16,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-09-19T07:49:34.095 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = 6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb 2024-09-19T07:49:34.095 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.095 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2024-09-19T07:49:34.095 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2024-09-19T07:49:34.096 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.096 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2024-09-19T07:49:34.096 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2024-09-19T07:49:34.096 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2024-09-19T07:49:34.096 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.096 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2024-09-19T07:49:34.096 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2024-09-19T07:49:34.096 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.096 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2024-09-19T07:49:34.096 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2024-09-19T07:49:34.096 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2024-09-19T07:49:34.096 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug trim objects = true 2024-09-19T07:49:34.096 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.096 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2024-09-19T07:49:34.097 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2024-09-19T07:49:34.097 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.097 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2024-09-19T07:49:34.097 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.097 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2024-09-19T07:49:34.097 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.097 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2024-09-19T07:49:34.097 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.097 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2024-09-19T07:49:34.097 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2024-09-19T07:49:34.097 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.097 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2024-09-19T07:49:34.097 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2024-09-19T07:49:34.097 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2024-09-19T07:49:34.097 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2024-09-19T07:49:34.098 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 20 2024-09-19T07:49:34.098 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 20 2024-09-19T07:49:34.098 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:49:34.098 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2024-09-19T07:49:34.098 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 10 2024-09-19T07:49:34.098 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T07:49:34.098 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2024-09-19T07:49:34.098 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2024-09-19T07:49:34.098 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2024-09-19T07:49:34.098 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2024-09-19T07:49:34.098 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-09-19T07:49:34.098 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.098 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2024-09-19T07:49:34.098 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:49:34.098 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2024-09-19T07:49:34.099 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-19T07:49:34.099 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-19T07:49:34.099 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:49:34.099 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:49:34.099 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2024-09-19T07:49:34.099 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-09-19T07:49:34.099 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.099 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2024-09-19T07:49:34.099 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:49:34.099 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-19T07:49:34.099 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2024-09-19T07:49:34.099 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-19T07:49:34.099 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2024-09-19T07:49:34.099 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2024-09-19T07:49:34.100 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:49:34.100 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2024-09-19T07:49:34.100 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2024-09-19T07:49:34.100 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:49:34.100 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.100 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T07:49:34.100 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T07:49:34.100 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T07:49:34.100 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.100 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T07:49:34.100 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T07:49:34.100 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T07:49:34.100 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.100 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2024-09-19T07:49:34.100 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2024-09-19T07:49:34.101 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.101 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2024-09-19T07:49:34.101 INFO:teuthology.orchestra.run.smithi146.stdout: mon down mkfs grace = 300 2024-09-19T07:49:34.101 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2024-09-19T07:49:34.101 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on pool no app = False 2024-09-19T07:49:34.101 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:49:34.101 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2024-09-19T07:49:34.101 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2024-09-19T07:49:34.101 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2024-09-19T07:49:34.101 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2024-09-19T07:49:34.101 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T07:49:34.101 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T07:49:34.101 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2024-09-19T07:49:34.101 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-09-19T07:49:34.102 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:49:34.102 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = "" 2024-09-19T07:49:34.102 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-09-19T07:49:34.102 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-09-19T07:49:34.102 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2024-09-19T07:49:34.102 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2024-09-19T07:49:34.102 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2024-09-19T07:49:34.102 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2024-09-19T07:49:34.102 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2024-09-19T07:49:34.102 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2024-09-19T07:49:34.102 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2024-09-19T07:49:34.102 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2024-09-19T07:49:34.102 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2024-09-19T07:49:34.102 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2024-09-19T07:49:34.103 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:49:34.103 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2024-09-19T07:49:34.103 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2024-09-19T07:49:34.103 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2024-09-19T07:49:34.103 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2024-09-19T07:49:34.103 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-09-19T07:49:34.103 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-09-19T07:49:34.103 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-09-19T07:49:34.105 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T07:49:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.210+0000 7f6922823640 1 Processor -- start 2024-09-19T07:49:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.210+0000 7f6922823640 1 -- start start 2024-09-19T07:49:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f6922823640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f691c141680 0x7f691c0cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f6922823640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f691c142170 0x7f691c13f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f6922823640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f691c15d710 con 0x7f691c0d0370 2024-09-19T07:49:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f6922823640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f691c0d07b0 con 0x7f691c142170 2024-09-19T07:49:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f6922823640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f691c15c6f0 con 0x7f691c141680 2024-09-19T07:49:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f691b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f691c0d0370 0x7f691c141a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58956/0 (socket says 172.21.15.16:58956) 2024-09-19T07:49:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f691b7fe640 1 -- 172.21.15.16:0/3662561115 learned_addr learned my addr 172.21.15.16:0/3662561115 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f691bfff640 1 --2- 172.21.15.16:0/3662561115 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f691c141680 0x7f691c0cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f6920dc0640 1 --2- 172.21.15.16:0/3662561115 >> v2:172.21.15.146:3300/0 conn(0x7f691c142170 0x7f691c13f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f691affd640 1 -- 172.21.15.16:0/3662561115 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2601545340 0 0) 0x7f691c15d710 con 0x7f691c0d0370 2024-09-19T07:49:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f691affd640 1 -- 172.21.15.16:0/3662561115 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6904003520 con 0x7f691c0d0370 2024-09-19T07:49:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f691affd640 1 -- 172.21.15.16:0/3662561115 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 732638299 0 0) 0x7f6904003520 con 0x7f691c0d0370 2024-09-19T07:49:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f691affd640 1 -- 172.21.15.16:0/3662561115 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f691c141680 msgr2=0x7f691c0cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f691affd640 1 --2- 172.21.15.16:0/3662561115 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f691c141680 0x7f691c0cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f691affd640 1 -- 172.21.15.16:0/3662561115 >> v2:172.21.15.146:3300/0 conn(0x7f691c142170 msgr2=0x7f691c13f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f691affd640 1 --2- 172.21.15.16:0/3662561115 >> v2:172.21.15.146:3300/0 conn(0x7f691c142170 0x7f691c13f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f691affd640 1 -- 172.21.15.16:0/3662561115 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691c142600 con 0x7f691c0d0370 2024-09-19T07:49:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f691affd640 1 -- 172.21.15.16:0/3662561115 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f6910001670 con 0x7f691c0d0370 2024-09-19T07:49:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f691affd640 1 -- 172.21.15.16:0/3662561115 >> v1:172.21.15.16:6789/0 conn(0x7f691c0d0370 legacy=0x7f691c141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f691affd640 1 --2- 172.21.15.16:0/3662561115 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f691c141680 0x7f69040039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f691affd640 1 --2- 172.21.15.16:0/3662561115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6904003f30 0x7f6904004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f691affd640 1 --2- 172.21.15.16:0/3662561115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6904004d40 0x7f69040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f691affd640 1 -- 172.21.15.16:0/3662561115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f691c0d07b0 con 0x7f6904003f30 2024-09-19T07:49:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f691affd640 1 -- 172.21.15.16:0/3662561115 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f691c15c6f0 con 0x7f6904004d40 2024-09-19T07:49:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.211+0000 7f691affd640 1 -- 172.21.15.16:0/3662561115 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f690400b9c0 con 0x7f691c141680 2024-09-19T07:49:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.212+0000 7f691b7fe640 1 --2- 172.21.15.16:0/3662561115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6904003f30 0x7f6904004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.212+0000 7f691bfff640 1 --2- 172.21.15.16:0/3662561115 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f691c141680 0x7f69040039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.212+0000 7f6920dc0640 1 --2- 172.21.15.16:0/3662561115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6904004d40 0x7f69040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.212+0000 7f691b7fe640 1 -- 172.21.15.16:0/3662561115 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f691c141680 msgr2=0x7f69040039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.212+0000 7f691b7fe640 1 --2- 172.21.15.16:0/3662561115 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f691c141680 0x7f69040039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.212+0000 7f691b7fe640 1 -- 172.21.15.16:0/3662561115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6904004d40 msgr2=0x7f69040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.212+0000 7f691b7fe640 1 --2- 172.21.15.16:0/3662561115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6904004d40 0x7f69040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.212+0000 7f691b7fe640 1 -- 172.21.15.16:0/3662561115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6904003520 con 0x7f6904003f30 2024-09-19T07:49:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.213+0000 7f691b7fe640 1 --2- 172.21.15.16:0/3662561115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6904003f30 0x7f6904004800 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6910007fd0 tx=0x7f6904005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12dd0c36915ca11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.213+0000 7f691affd640 1 -- 172.21.15.16:0/3662561115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910007d50 con 0x7f6904003f30 2024-09-19T07:49:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.213+0000 7f691affd640 1 -- 172.21.15.16:0/3662561115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6910005af0 con 0x7f6904003f30 2024-09-19T07:49:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.213+0000 7f691affd640 1 -- 172.21.15.16:0/3662561115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910007070 con 0x7f6904003f30 2024-09-19T07:49:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.213+0000 7f6922823640 1 -- 172.21.15.16:0/3662561115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6904003f30 msgr2=0x7f6904004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.213+0000 7f6922823640 1 --2- 172.21.15.16:0/3662561115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6904003f30 0x7f6904004800 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6910007fd0 tx=0x7f6904005830 comp rx=0 tx=0).stop 2024-09-19T07:49:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.213+0000 7f6920dc0640 1 -- 172.21.15.16:0/3662561115 reap_dead start 2024-09-19T07:49:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.213+0000 7f6922823640 1 -- 172.21.15.16:0/3662561115 shutdown_connections 2024-09-19T07:49:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.213+0000 7f6922823640 1 -- 172.21.15.16:0/3662561115 >> 172.21.15.16:0/3662561115 conn(0x7f691c0b3ed0 msgr2=0x7f691c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.213+0000 7f6922823640 1 -- 172.21.15.16:0/3662561115 shutdown_connections 2024-09-19T07:49:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.213+0000 7f6922823640 1 -- 172.21.15.16:0/3662561115 wait complete. 2024-09-19T07:49:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.214+0000 7f6922823640 1 Processor -- start 2024-09-19T07:49:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.215+0000 7f6922823640 1 -- start start 2024-09-19T07:49:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.215+0000 7f6922823640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6904003f30 0x7f691c13eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.215+0000 7f6922823640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f691c06e9d0 0x7f691c1e4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.215+0000 7f6922823640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f691c15b210 0x7f691c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.215+0000 7f6922823640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f691c15cf10 con 0x7f6904003f30 2024-09-19T07:49:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.215+0000 7f6922823640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f691c15cc30 con 0x7f691c06e9d0 2024-09-19T07:49:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.215+0000 7f6922823640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f691c15d1f0 con 0x7f691c15b210 2024-09-19T07:49:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.215+0000 7f691bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6904003f30 0x7f691c13eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.216+0000 7f691bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6904003f30 0x7f691c13eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53350/0 (socket says 172.21.15.16:53350) 2024-09-19T07:49:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.216+0000 7f691bfff640 1 -- 172.21.15.16:0/2344534983 learned_addr learned my addr 172.21.15.16:0/2344534983 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.216+0000 7f6920dc0640 1 --2- 172.21.15.16:0/2344534983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f691c15b210 0x7f691c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.216+0000 7f691b7fe640 1 --2- 172.21.15.16:0/2344534983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f691c06e9d0 0x7f691c1e4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.216+0000 7f691bfff640 1 -- 172.21.15.16:0/2344534983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f691c15b210 msgr2=0x7f691c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.216+0000 7f691bfff640 1 --2- 172.21.15.16:0/2344534983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f691c15b210 0x7f691c15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.216+0000 7f691bfff640 1 -- 172.21.15.16:0/2344534983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f691c06e9d0 msgr2=0x7f691c1e4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.216+0000 7f691bfff640 1 --2- 172.21.15.16:0/2344534983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f691c06e9d0 0x7f691c1e4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.216+0000 7f691bfff640 1 -- 172.21.15.16:0/2344534983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691c13f090 con 0x7f6904003f30 2024-09-19T07:49:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.216+0000 7f691bfff640 1 --2- 172.21.15.16:0/2344534983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6904003f30 0x7f691c13eb50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f690c0118e0 tx=0x7f690c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.216+0000 7f6918ff9640 1 -- 172.21.15.16:0/2344534983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c00fa90 con 0x7f6904003f30 2024-09-19T07:49:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.216+0000 7f6918ff9640 1 -- 172.21.15.16:0/2344534983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f690c00fc30 con 0x7f6904003f30 2024-09-19T07:49:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.216+0000 7f6918ff9640 1 -- 172.21.15.16:0/2344534983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c014440 con 0x7f6904003f30 2024-09-19T07:49:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.216+0000 7f6922823640 1 -- 172.21.15.16:0/2344534983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691c1e4f30 con 0x7f6904003f30 2024-09-19T07:49:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.216+0000 7f6922823640 1 -- 172.21.15.16:0/2344534983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691c1e54f0 con 0x7f6904003f30 2024-09-19T07:49:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.217+0000 7f6922823640 1 -- 172.21.15.16:0/2344534983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f691c0cfe30 con 0x7f6904003f30 2024-09-19T07:49:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.218+0000 7f6918ff9640 1 -- 172.21.15.16:0/2344534983 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f690c012040 con 0x7f6904003f30 2024-09-19T07:49:34.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.218+0000 7f6918ff9640 1 --2- 172.21.15.16:0/2344534983 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f690806c2e0 0x7f690806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:34.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.218+0000 7f6918ff9640 1 -- 172.21.15.16:0/2344534983 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5335+0+0 (secure 0 0 0) 0x7f690c08b770 con 0x7f6904003f30 2024-09-19T07:49:34.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.219+0000 7f691b7fe640 1 --2- 172.21.15.16:0/2344534983 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f690806c2e0 0x7f690806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:34.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.219+0000 7f691b7fe640 1 --2- 172.21.15.16:0/2344534983 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f690806c2e0 0x7f690806e7a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6910033000 tx=0x7f6910009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.223+0000 7f6918ff9640 1 -- 172.21.15.16:0/2344534983 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f690c019030 con 0x7f6904003f30 2024-09-19T07:49:34.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.405+0000 7f6922823640 1 -- 172.21.15.16:0/2344534983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f691c0690c0 con 0x7f6904003f30 2024-09-19T07:49:34.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.408+0000 7f6918ff9640 1 -- 172.21.15.16:0/2344534983 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v26) ==== 104+0+0 (secure 0 0 0) 0x7f690c059960 con 0x7f6904003f30 2024-09-19T07:49:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.410+0000 7f6922823640 1 -- 172.21.15.16:0/2344534983 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f690806c2e0 msgr2=0x7f690806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.410+0000 7f6922823640 1 --2- 172.21.15.16:0/2344534983 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f690806c2e0 0x7f690806e7a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6910033000 tx=0x7f6910009030 comp rx=0 tx=0).stop 2024-09-19T07:49:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.410+0000 7f6922823640 1 -- 172.21.15.16:0/2344534983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6904003f30 msgr2=0x7f691c13eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.410+0000 7f6922823640 1 --2- 172.21.15.16:0/2344534983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6904003f30 0x7f691c13eb50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f690c0118e0 tx=0x7f690c011da0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.410+0000 7f6922823640 1 -- 172.21.15.16:0/2344534983 shutdown_connections 2024-09-19T07:49:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.410+0000 7f6922823640 1 --2- 172.21.15.16:0/2344534983 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f690806c2e0 0x7f690806e7a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.410+0000 7f6922823640 1 --2- 172.21.15.16:0/2344534983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f691c15b210 0x7f691c15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.410+0000 7f6922823640 1 --2- 172.21.15.16:0/2344534983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f691c06e9d0 0x7f691c1e4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.410+0000 7f6922823640 1 --2- 172.21.15.16:0/2344534983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6904003f30 0x7f691c13eb50 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.410+0000 7f6922823640 1 -- 172.21.15.16:0/2344534983 >> 172.21.15.16:0/2344534983 conn(0x7f691c0b3ed0 msgr2=0x7f691c140ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.411+0000 7f6922823640 1 -- 172.21.15.16:0/2344534983 shutdown_connections 2024-09-19T07:49:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.411+0000 7f6922823640 1 -- 172.21.15.16:0/2344534983 wait complete. 2024-09-19T07:49:34.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size' 2024-09-19T07:49:34.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.524+0000 7f115ef6d640 1 Processor -- start 2024-09-19T07:49:34.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.524+0000 7f115ef6d640 1 -- start start 2024-09-19T07:49:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.525+0000 7f115ef6d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f115815e680 0x7f11580cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.525+0000 7f115ef6d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11580d0010 0x7f11580d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.525+0000 7f115ef6d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11580d1880 con 0x7f115815e2a0 2024-09-19T07:49:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.525+0000 7f115ef6d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f11580d1b60 con 0x7f115815e680 2024-09-19T07:49:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.525+0000 7f115ef6d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f11580cf2d0 con 0x7f11580d0010 2024-09-19T07:49:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.525+0000 7f1157fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f115815e680 0x7f11580cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.525+0000 7f1157fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f115815e680 0x7f11580cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53354/0 (socket says 172.21.15.16:53354) 2024-09-19T07:49:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.525+0000 7f1157fff640 1 -- 172.21.15.16:0/3200202026 learned_addr learned my addr 172.21.15.16:0/3200202026 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.525+0000 7f115d50a640 1 --2- 172.21.15.16:0/3200202026 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11580d0010 0x7f11580d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.525+0000 7f1157fff640 1 -- 172.21.15.16:0/3200202026 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11580d0010 msgr2=0x7f11580d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.525+0000 7f1157fff640 1 --2- 172.21.15.16:0/3200202026 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11580d0010 0x7f11580d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.525+0000 7f1157fff640 1 -- 172.21.15.16:0/3200202026 >> v1:172.21.15.146:6789/0 conn(0x7f115815e2a0 legacy=0x7f115806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.525+0000 7f1157fff640 1 -- 172.21.15.16:0/3200202026 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11580cf5b0 con 0x7f115815e680 2024-09-19T07:49:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.525+0000 7f115d50a640 1 --2- 172.21.15.16:0/3200202026 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11580d0010 0x7f11580d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.526+0000 7f1157fff640 1 --2- 172.21.15.16:0/3200202026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115815e680 0x7f11580cfad0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1144009870 tx=0x7f114402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8848546c681e4433 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.526+0000 7f11577fe640 1 -- 172.21.15.16:0/3200202026 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144037070 con 0x7f115815e680 2024-09-19T07:49:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.526+0000 7f11577fe640 1 -- 172.21.15.16:0/3200202026 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1144032440 con 0x7f115815e680 2024-09-19T07:49:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.526+0000 7f11577fe640 1 -- 172.21.15.16:0/3200202026 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144032810 con 0x7f115815e680 2024-09-19T07:49:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.527+0000 7f115ef6d640 1 -- 172.21.15.16:0/3200202026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115815e680 msgr2=0x7f11580cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.527+0000 7f115ef6d640 1 --2- 172.21.15.16:0/3200202026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115815e680 0x7f11580cfad0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1144009870 tx=0x7f114402a180 comp rx=0 tx=0).stop 2024-09-19T07:49:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.527+0000 7f115ef6d640 1 -- 172.21.15.16:0/3200202026 shutdown_connections 2024-09-19T07:49:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.527+0000 7f115ef6d640 1 --2- 172.21.15.16:0/3200202026 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11580d0010 0x7f11580d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.527+0000 7f115ef6d640 1 --2- 172.21.15.16:0/3200202026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115815e680 0x7f11580cfad0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.527+0000 7f115ef6d640 1 -- 172.21.15.16:0/3200202026 >> 172.21.15.16:0/3200202026 conn(0x7f115809d760 msgr2=0x7f115808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.527+0000 7f115ef6d640 1 -- 172.21.15.16:0/3200202026 shutdown_connections 2024-09-19T07:49:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.527+0000 7f115ef6d640 1 -- 172.21.15.16:0/3200202026 wait complete. 2024-09-19T07:49:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.528+0000 7f115ef6d640 1 Processor -- start 2024-09-19T07:49:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.528+0000 7f115ef6d640 1 -- start start 2024-09-19T07:49:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f115ef6d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11580d0010 0x7f115815d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f115ef6d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f115815e2a0 0x7f1158161f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f115ef6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115815e680 0x7f1158162480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f115ef6d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f115806dcb0 con 0x7f115815e680 2024-09-19T07:49:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f115ef6d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f115806db30 con 0x7f115815e2a0 2024-09-19T07:49:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f115ef6d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f11580d48b0 con 0x7f11580d0010 2024-09-19T07:49:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f115d50a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115815e680 0x7f1158162480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f115d50a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115815e680 0x7f1158162480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53362/0 (socket says 172.21.15.16:53362) 2024-09-19T07:49:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f115d50a640 1 -- 172.21.15.16:0/3555569023 learned_addr learned my addr 172.21.15.16:0/3555569023 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f115cd09640 1 --2- 172.21.15.16:0/3555569023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11580d0010 0x7f115815d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f1157fff640 1 --2- 172.21.15.16:0/3555569023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f115815e2a0 0x7f1158161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f115d50a640 1 -- 172.21.15.16:0/3555569023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11580d0010 msgr2=0x7f115815d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f115d50a640 1 --2- 172.21.15.16:0/3555569023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11580d0010 0x7f115815d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f115d50a640 1 -- 172.21.15.16:0/3555569023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f115815e2a0 msgr2=0x7f1158161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f115d50a640 1 --2- 172.21.15.16:0/3555569023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f115815e2a0 0x7f1158161f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f115d50a640 1 -- 172.21.15.16:0/3555569023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115815d850 con 0x7f115815e680 2024-09-19T07:49:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f115d50a640 1 --2- 172.21.15.16:0/3555569023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115815e680 0x7f1158162480 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f115000cef0 tx=0x7f115000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f11557fa640 1 -- 172.21.15.16:0/3555569023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115001a070 con 0x7f115815e680 2024-09-19T07:49:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f11557fa640 1 -- 172.21.15.16:0/3555569023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f115000d860 con 0x7f115815e680 2024-09-19T07:49:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.529+0000 7f11557fa640 1 -- 172.21.15.16:0/3555569023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115000dc50 con 0x7f115815e680 2024-09-19T07:49:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.530+0000 7f115ef6d640 1 -- 172.21.15.16:0/3555569023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11580cb550 con 0x7f115815e680 2024-09-19T07:49:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.530+0000 7f115ef6d640 1 -- 172.21.15.16:0/3555569023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11580cbb10 con 0x7f115815e680 2024-09-19T07:49:34.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.531+0000 7f115ef6d640 1 -- 172.21.15.16:0/3555569023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11580d4c70 con 0x7f115815e680 2024-09-19T07:49:34.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.531+0000 7f11557fa640 1 -- 172.21.15.16:0/3555569023 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f115001d070 con 0x7f115815e680 2024-09-19T07:49:34.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.532+0000 7f11557fa640 1 --2- 172.21.15.16:0/3555569023 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f113c06c2b0 0x7f113c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:34.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.532+0000 7f11557fa640 1 -- 172.21.15.16:0/3555569023 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5335+0+0 (secure 0 0 0) 0x7f115008baf0 con 0x7f115815e680 2024-09-19T07:49:34.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.532+0000 7f115cd09640 1 --2- 172.21.15.16:0/3555569023 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f113c06c2b0 0x7f113c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:34.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.532+0000 7f115cd09640 1 --2- 172.21.15.16:0/3555569023 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f113c06c2b0 0x7f113c06e770 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f114c004f60 tx=0x7f114c007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:34.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.536+0000 7f11557fa640 1 -- 172.21.15.16:0/3555569023 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1150059e40 con 0x7f115815e680 2024-09-19T07:49:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:34.779+0000 7f115ef6d640 1 -- 172.21.15.16:0/3555569023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]} v 0) -- 0x7f115815eb90 con 0x7f115815e680 2024-09-19T07:49:35.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.010+0000 7f11557fa640 1 -- 172.21.15.16:0/3555569023 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]}]=0 v284) ==== 150+0+0 (secure 0 0 0) 0x7f115005e140 con 0x7f115815e680 2024-09-19T07:49:35.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.012+0000 7f115ef6d640 1 -- 172.21.15.16:0/3555569023 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f113c06c2b0 msgr2=0x7f113c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.012+0000 7f115ef6d640 1 --2- 172.21.15.16:0/3555569023 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f113c06c2b0 0x7f113c06e770 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f114c004f60 tx=0x7f114c007e30 comp rx=0 tx=0).stop 2024-09-19T07:49:35.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.012+0000 7f115ef6d640 1 -- 172.21.15.16:0/3555569023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115815e680 msgr2=0x7f1158162480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.012+0000 7f115ef6d640 1 --2- 172.21.15.16:0/3555569023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115815e680 0x7f1158162480 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f115000cef0 tx=0x7f115000a710 comp rx=0 tx=0).stop 2024-09-19T07:49:35.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.012+0000 7f115ef6d640 1 -- 172.21.15.16:0/3555569023 shutdown_connections 2024-09-19T07:49:35.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.012+0000 7f115ef6d640 1 --2- 172.21.15.16:0/3555569023 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f113c06c2b0 0x7f113c06e770 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.012+0000 7f115ef6d640 1 --2- 172.21.15.16:0/3555569023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115815e680 0x7f1158162480 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.012+0000 7f115ef6d640 1 --2- 172.21.15.16:0/3555569023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f115815e2a0 0x7f1158161f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.012+0000 7f115ef6d640 1 --2- 172.21.15.16:0/3555569023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11580d0010 0x7f115815d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.013+0000 7f115ef6d640 1 -- 172.21.15.16:0/3555569023 >> 172.21.15.16:0/3555569023 conn(0x7f115809d760 msgr2=0x7f11580d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:35.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.013+0000 7f115ef6d640 1 -- 172.21.15.16:0/3555569023 shutdown_connections 2024-09-19T07:49:35.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.013+0000 7f115ef6d640 1 -- 172.21.15.16:0/3555569023 wait complete. 2024-09-19T07:49:35.024 INFO:tasks.cephfs_test_runner:test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2024-09-19T07:49:35.025 INFO:tasks.cephfs_test_runner:Starting test: test_client_max_caps (tasks.cephfs.test_client_limits.TestClientLimits) That the MDS will not let a client sit above mds_max_caps_per_client caps. 2024-09-19T07:49:35.025 INFO:tasks.cephfs_test_runner:test_client_max_caps (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-19T07:49:35.025 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:49:35.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.128+0000 7f99cdb03640 1 Processor -- start 2024-09-19T07:49:35.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.128+0000 7f99cdb03640 1 -- start start 2024-09-19T07:49:35.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.128+0000 7f99cdb03640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99c80d1ca0 0x7f99c80cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.128+0000 7f99cdb03640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f99c80d02f0 0x7f99c81562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.128+0000 7f99cdb03640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99c80d06d0 con 0x7f99c815cb00 2024-09-19T07:49:35.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.128+0000 7f99cdb03640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f99c80b40b0 con 0x7f99c80d1ca0 2024-09-19T07:49:35.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.128+0000 7f99cdb03640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f99c80d22d0 con 0x7f99c80d02f0 2024-09-19T07:49:35.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.129+0000 7f99c6ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99c80d1ca0 0x7f99c80cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.129+0000 7f99c77fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f99c815cb00 0x7f99c815cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58972/0 (socket says 172.21.15.16:58972) 2024-09-19T07:49:35.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.129+0000 7f99c77fe640 1 -- 172.21.15.16:0/3569967916 learned_addr learned my addr 172.21.15.16:0/3569967916 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:35.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.129+0000 7f99c7fff640 1 --2- 172.21.15.16:0/3569967916 >> v2:172.21.15.146:3300/0 conn(0x7f99c80d02f0 0x7f99c81562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.129+0000 7f99c6ffd640 1 -- 172.21.15.16:0/3569967916 >> v2:172.21.15.146:3300/0 conn(0x7f99c80d02f0 msgr2=0x7f99c81562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:35.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.129+0000 7f99c6ffd640 1 --2- 172.21.15.16:0/3569967916 >> v2:172.21.15.146:3300/0 conn(0x7f99c80d02f0 0x7f99c81562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.129+0000 7f99c6ffd640 1 -- 172.21.15.16:0/3569967916 >> v1:172.21.15.16:6789/0 conn(0x7f99c815cb00 legacy=0x7f99c815cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.129+0000 7f99c6ffd640 1 -- 172.21.15.16:0/3569967916 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c80d25b0 con 0x7f99c80d1ca0 2024-09-19T07:49:35.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.130+0000 7f99c6ffd640 1 --2- 172.21.15.16:0/3569967916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c80d1ca0 0x7f99c80cfdb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f99c80c9230 tx=0x7f99b002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da91f844604041fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:35.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.130+0000 7f99c67fc640 1 -- 172.21.15.16:0/3569967916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0037070 con 0x7f99c80d1ca0 2024-09-19T07:49:35.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.130+0000 7f99c67fc640 1 -- 172.21.15.16:0/3569967916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b0032440 con 0x7f99c80d1ca0 2024-09-19T07:49:35.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.130+0000 7f99c67fc640 1 -- 172.21.15.16:0/3569967916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b002ae60 con 0x7f99c80d1ca0 2024-09-19T07:49:35.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.130+0000 7f99cdb03640 1 -- 172.21.15.16:0/3569967916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c80d1ca0 msgr2=0x7f99c80cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.130+0000 7f99cdb03640 1 --2- 172.21.15.16:0/3569967916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c80d1ca0 0x7f99c80cfdb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f99c80c9230 tx=0x7f99b002a110 comp rx=0 tx=0).stop 2024-09-19T07:49:35.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.130+0000 7f99cdb03640 1 -- 172.21.15.16:0/3569967916 shutdown_connections 2024-09-19T07:49:35.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.130+0000 7f99cdb03640 1 --2- 172.21.15.16:0/3569967916 >> v2:172.21.15.146:3300/0 conn(0x7f99c80d02f0 0x7f99c81562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.130+0000 7f99cdb03640 1 --2- 172.21.15.16:0/3569967916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c80d1ca0 0x7f99c80cfdb0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.130+0000 7f99cdb03640 1 -- 172.21.15.16:0/3569967916 >> 172.21.15.16:0/3569967916 conn(0x7f99c8067fe0 msgr2=0x7f99c806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:35.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.130+0000 7f99cdb03640 1 -- 172.21.15.16:0/3569967916 shutdown_connections 2024-09-19T07:49:35.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.130+0000 7f99cdb03640 1 -- 172.21.15.16:0/3569967916 wait complete. 2024-09-19T07:49:35.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.132+0000 7f99cdb03640 1 Processor -- start 2024-09-19T07:49:35.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.132+0000 7f99cdb03640 1 -- start start 2024-09-19T07:49:35.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.132+0000 7f99cdb03640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99c80d02f0 0x7f99c80b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.132+0000 7f99cdb03640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99c80d1ca0 0x7f99c8157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.132+0000 7f99cdb03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c815cb00 0x7f99c80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.132+0000 7f99cdb03640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f99c815d5f0 con 0x7f99c815cb00 2024-09-19T07:49:35.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.132+0000 7f99cdb03640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f99c80d2b70 con 0x7f99c80d02f0 2024-09-19T07:49:35.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.132+0000 7f99cdb03640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f99c815d770 con 0x7f99c80d1ca0 2024-09-19T07:49:35.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.133+0000 7f99c7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c815cb00 0x7f99c80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.133+0000 7f99c7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c815cb00 0x7f99c80b77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53378/0 (socket says 172.21.15.16:53378) 2024-09-19T07:49:35.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.133+0000 7f99c7fff640 1 -- 172.21.15.16:0/2326697123 learned_addr learned my addr 172.21.15.16:0/2326697123 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:35.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.133+0000 7f99c77fe640 1 --2- 172.21.15.16:0/2326697123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99c80d02f0 0x7f99c80b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.133+0000 7f99c6ffd640 1 --2- 172.21.15.16:0/2326697123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99c80d1ca0 0x7f99c8157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.133+0000 7f99c7fff640 1 -- 172.21.15.16:0/2326697123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99c80d1ca0 msgr2=0x7f99c8157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.133+0000 7f99c7fff640 1 --2- 172.21.15.16:0/2326697123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99c80d1ca0 0x7f99c8157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.133+0000 7f99c7fff640 1 -- 172.21.15.16:0/2326697123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99c80d02f0 msgr2=0x7f99c80b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.133+0000 7f99c7fff640 1 --2- 172.21.15.16:0/2326697123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99c80d02f0 0x7f99c80b50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.133+0000 7f99c7fff640 1 -- 172.21.15.16:0/2326697123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c80b5940 con 0x7f99c815cb00 2024-09-19T07:49:35.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.133+0000 7f99c7fff640 1 --2- 172.21.15.16:0/2326697123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c815cb00 0x7f99c80b77a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f99c0007e60 tx=0x7f99c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:35.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.133+0000 7f99bffff640 1 -- 172.21.15.16:0/2326697123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c0014420 con 0x7f99c815cb00 2024-09-19T07:49:35.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.133+0000 7f99bffff640 1 -- 172.21.15.16:0/2326697123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99c0019070 con 0x7f99c815cb00 2024-09-19T07:49:35.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.133+0000 7f99cdb03640 1 -- 172.21.15.16:0/2326697123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c80b5c20 con 0x7f99c815cb00 2024-09-19T07:49:35.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.133+0000 7f99bffff640 1 -- 172.21.15.16:0/2326697123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c0014820 con 0x7f99c815cb00 2024-09-19T07:49:35.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.133+0000 7f99cdb03640 1 -- 172.21.15.16:0/2326697123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c80b61e0 con 0x7f99c815cb00 2024-09-19T07:49:35.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.134+0000 7f99cdb03640 1 -- 172.21.15.16:0/2326697123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c80cfdb0 con 0x7f99c815cb00 2024-09-19T07:49:35.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.135+0000 7f99bffff640 1 -- 172.21.15.16:0/2326697123 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f99c00149c0 con 0x7f99c815cb00 2024-09-19T07:49:35.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.135+0000 7f99bffff640 1 --2- 172.21.15.16:0/2326697123 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f99a806c2b0 0x7f99a806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.135+0000 7f99bffff640 1 -- 172.21.15.16:0/2326697123 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5335+0+0 (secure 0 0 0) 0x7f99c008ba70 con 0x7f99c815cb00 2024-09-19T07:49:35.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.136+0000 7f99c77fe640 1 --2- 172.21.15.16:0/2326697123 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f99a806c2b0 0x7f99a806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.136+0000 7f99c77fe640 1 --2- 172.21.15.16:0/2326697123 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f99a806c2b0 0x7f99a806e770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f99b8007fd0 tx=0x7f99b8007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:35.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.140+0000 7f99bffff640 1 -- 172.21.15.16:0/2326697123 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f99c0059c60 con 0x7f99c815cb00 2024-09-19T07:49:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.315+0000 7f99cdb03640 1 -- 172.21.15.16:0/2326697123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99c815a5e0 con 0x7f99c815cb00 2024-09-19T07:49:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.316+0000 7f99bffff640 1 -- 172.21.15.16:0/2326697123 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+13328 (secure 0 0 0) 0x7f99c005df60 con 0x7f99c815cb00 2024-09-19T07:49:35.316 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:35.316 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:49:34.003505+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6834/418904597":"2024-09-20T07:49:30.996178+0000","172.21.15.16:6832/418904597":"2024-09-20T07:49:30.996178+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:49:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.318+0000 7f99cdb03640 1 -- 172.21.15.16:0/2326697123 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f99a806c2b0 msgr2=0x7f99a806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.318+0000 7f99cdb03640 1 --2- 172.21.15.16:0/2326697123 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f99a806c2b0 0x7f99a806e770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f99b8007fd0 tx=0x7f99b8007e80 comp rx=0 tx=0).stop 2024-09-19T07:49:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.318+0000 7f99cdb03640 1 -- 172.21.15.16:0/2326697123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c815cb00 msgr2=0x7f99c80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.318+0000 7f99cdb03640 1 --2- 172.21.15.16:0/2326697123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c815cb00 0x7f99c80b77a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f99c0007e60 tx=0x7f99c000e4b0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.318+0000 7f99cdb03640 1 -- 172.21.15.16:0/2326697123 shutdown_connections 2024-09-19T07:49:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.318+0000 7f99cdb03640 1 --2- 172.21.15.16:0/2326697123 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f99a806c2b0 0x7f99a806e770 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.318+0000 7f99cdb03640 1 --2- 172.21.15.16:0/2326697123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c815cb00 0x7f99c80b77a0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.318+0000 7f99cdb03640 1 --2- 172.21.15.16:0/2326697123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99c80d1ca0 0x7f99c8157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.318+0000 7f99cdb03640 1 --2- 172.21.15.16:0/2326697123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99c80d02f0 0x7f99c80b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.318+0000 7f99cdb03640 1 -- 172.21.15.16:0/2326697123 >> 172.21.15.16:0/2326697123 conn(0x7f99c8067fe0 msgr2=0x7f99c806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.318+0000 7f99cdb03640 1 -- 172.21.15.16:0/2326697123 shutdown_connections 2024-09-19T07:49:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.318+0000 7f99cdb03640 1 -- 172.21.15.16:0/2326697123 wait complete. 2024-09-19T07:49:35.330 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T07:42:54.500338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T07:49:35.330 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T07:49:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.435+0000 7f6b112c9640 1 Processor -- start 2024-09-19T07:49:35.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.435+0000 7f6b112c9640 1 -- start start 2024-09-19T07:49:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.436+0000 7f6b112c9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b0c0d4370 0x7f6b0c0d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.436+0000 7f6b112c9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b0c0d1880 con 0x7f6b0c15e680 2024-09-19T07:49:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.436+0000 7f6b112c9640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b0c0d1b60 con 0x7f6b0c15e2a0 2024-09-19T07:49:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.436+0000 7f6b112c9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6b0c0cf2d0 con 0x7f6b0c0d4370 2024-09-19T07:49:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.436+0000 7f6b0a7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6b0c15e680 0x7f6b0c0cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58974/0 (socket says 172.21.15.16:58974) 2024-09-19T07:49:35.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.436+0000 7f6b0a7fc640 1 -- 172.21.15.16:0/24271127 learned_addr learned my addr 172.21.15.16:0/24271127 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.436+0000 7f6b09ffb640 1 -- 172.21.15.16:0/24271127 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 650985776 0 0) 0x7f6b0c0d1880 con 0x7f6b0c15e680 2024-09-19T07:49:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.436+0000 7f6b09ffb640 1 -- 172.21.15.16:0/24271127 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6af8003520 con 0x7f6b0c15e680 2024-09-19T07:49:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.436+0000 7f6b0b7fe640 1 --2- 172.21.15.16:0/24271127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b0c0d4370 0x7f6b0c0d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b09ffb640 1 -- 172.21.15.16:0/24271127 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1202858309 0 0) 0x7f6af8003520 con 0x7f6b0c15e680 2024-09-19T07:49:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b09ffb640 1 -- 172.21.15.16:0/24271127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b0c0d4370 msgr2=0x7f6b0c0d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b09ffb640 1 --2- 172.21.15.16:0/24271127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b0c0d4370 0x7f6b0c0d4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b09ffb640 1 -- 172.21.15.16:0/24271127 >> v1:172.21.15.146:6789/0 conn(0x7f6b0c15e2a0 legacy=0x7f6b0c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b09ffb640 1 -- 172.21.15.16:0/24271127 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b0c0cf5b0 con 0x7f6b0c15e680 2024-09-19T07:49:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b09ffb640 1 -- 172.21.15.16:0/24271127 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f6af4001670 con 0x7f6b0c15e680 2024-09-19T07:49:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b09ffb640 1 -- 172.21.15.16:0/24271127 >> v1:172.21.15.16:6789/0 conn(0x7f6b0c15e680 legacy=0x7f6b0c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b09ffb640 1 --2- 172.21.15.16:0/24271127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6af80039e0 0x7f6af8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b09ffb640 1 --2- 172.21.15.16:0/24271127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b0c0d4370 0x7f6af8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b09ffb640 1 --2- 172.21.15.16:0/24271127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6af8004d40 0x7f6af80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b09ffb640 1 -- 172.21.15.16:0/24271127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c0cf2d0 con 0x7f6af80039e0 2024-09-19T07:49:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b09ffb640 1 -- 172.21.15.16:0/24271127 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6af800baa0 con 0x7f6af8004d40 2024-09-19T07:49:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b09ffb640 1 -- 172.21.15.16:0/24271127 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6af800bca0 con 0x7f6b0c0d4370 2024-09-19T07:49:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b0affd640 1 --2- 172.21.15.16:0/24271127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6af80039e0 0x7f6af8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b0b7fe640 1 --2- 172.21.15.16:0/24271127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6af8004d40 0x7f6af80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b0a7fc640 1 --2- 172.21.15.16:0/24271127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b0c0d4370 0x7f6af8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b0affd640 1 -- 172.21.15.16:0/24271127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b0c0d4370 msgr2=0x7f6af8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:35.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b0affd640 1 --2- 172.21.15.16:0/24271127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b0c0d4370 0x7f6af8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b0affd640 1 -- 172.21.15.16:0/24271127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6af8004d40 msgr2=0x7f6af80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b0affd640 1 --2- 172.21.15.16:0/24271127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6af8004d40 0x7f6af80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.437+0000 7f6b0affd640 1 -- 172.21.15.16:0/24271127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af8003520 con 0x7f6af80039e0 2024-09-19T07:49:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.438+0000 7f6b0affd640 1 --2- 172.21.15.16:0/24271127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6af80039e0 0x7f6af8003e00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6afc001e20 tx=0x7f6af8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5c1bbe1ed4ed86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.438+0000 7f6b09ffb640 1 -- 172.21.15.16:0/24271127 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc001a70 con 0x7f6af80039e0 2024-09-19T07:49:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.438+0000 7f6b09ffb640 1 -- 172.21.15.16:0/24271127 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6afc02ae40 con 0x7f6af80039e0 2024-09-19T07:49:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.438+0000 7f6b09ffb640 1 -- 172.21.15.16:0/24271127 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc003bb0 con 0x7f6af80039e0 2024-09-19T07:49:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.438+0000 7f6b112c9640 1 -- 172.21.15.16:0/24271127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6af80039e0 msgr2=0x7f6af8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.438+0000 7f6b112c9640 1 --2- 172.21.15.16:0/24271127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6af80039e0 0x7f6af8003e00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6afc001e20 tx=0x7f6af8005830 comp rx=0 tx=0).stop 2024-09-19T07:49:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.439+0000 7f6b0b7fe640 1 -- 172.21.15.16:0/24271127 reap_dead start 2024-09-19T07:49:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.439+0000 7f6b112c9640 1 -- 172.21.15.16:0/24271127 shutdown_connections 2024-09-19T07:49:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.439+0000 7f6b112c9640 1 -- 172.21.15.16:0/24271127 >> 172.21.15.16:0/24271127 conn(0x7f6b0c09d760 msgr2=0x7f6b0c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:35.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.439+0000 7f6b112c9640 1 -- 172.21.15.16:0/24271127 shutdown_connections 2024-09-19T07:49:35.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.439+0000 7f6b112c9640 1 -- 172.21.15.16:0/24271127 wait complete. 2024-09-19T07:49:35.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.440+0000 7f6b112c9640 1 Processor -- start 2024-09-19T07:49:35.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.440+0000 7f6b112c9640 1 -- start start 2024-09-19T07:49:35.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b112c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b0c14c180 0x7f6b0c14f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b112c9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b0c0ce6d0 0x7f6b0c15fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b112c9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b0c160000 0x7f6b0c0ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b112c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c06dcb0 con 0x7f6b0c14c180 2024-09-19T07:49:35.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b112c9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c06db30 con 0x7f6b0c160000 2024-09-19T07:49:35.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b112c9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6b0c0d01e0 con 0x7f6b0c0ce6d0 2024-09-19T07:49:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b0affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b0c14c180 0x7f6b0c14f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b0affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b0c14c180 0x7f6b0c14f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53410/0 (socket says 172.21.15.16:53410) 2024-09-19T07:49:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b0affd640 1 -- 172.21.15.16:0/1962805183 learned_addr learned my addr 172.21.15.16:0/1962805183 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b0a7fc640 1 --2- 172.21.15.16:0/1962805183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b0c0ce6d0 0x7f6b0c15fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b0b7fe640 1 --2- 172.21.15.16:0/1962805183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b0c160000 0x7f6b0c0ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b0affd640 1 -- 172.21.15.16:0/1962805183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b0c0ce6d0 msgr2=0x7f6b0c15fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b0affd640 1 --2- 172.21.15.16:0/1962805183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b0c0ce6d0 0x7f6b0c15fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b0affd640 1 -- 172.21.15.16:0/1962805183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b0c160000 msgr2=0x7f6b0c0ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b0affd640 1 --2- 172.21.15.16:0/1962805183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b0c160000 0x7f6b0c0ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b0affd640 1 -- 172.21.15.16:0/1962805183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b0c09e0d0 con 0x7f6b0c14c180 2024-09-19T07:49:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b0b7fe640 1 --2- 172.21.15.16:0/1962805183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b0c160000 0x7f6b0c0ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b0affd640 1 --2- 172.21.15.16:0/1962805183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b0c14c180 0x7f6b0c14f110 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6afc02a8f0 tx=0x7f6afc0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b037fe640 1 -- 172.21.15.16:0/1962805183 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc037040 con 0x7f6b0c14c180 2024-09-19T07:49:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b037fe640 1 -- 172.21.15.16:0/1962805183 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6afc041070 con 0x7f6b0c14c180 2024-09-19T07:49:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.441+0000 7f6b037fe640 1 -- 172.21.15.16:0/1962805183 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc007db0 con 0x7f6b0c14c180 2024-09-19T07:49:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.442+0000 7f6b112c9640 1 -- 172.21.15.16:0/1962805183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b0c09e650 con 0x7f6b0c14c180 2024-09-19T07:49:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.442+0000 7f6b112c9640 1 -- 172.21.15.16:0/1962805183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b0c14c8c0 con 0x7f6b0c14c180 2024-09-19T07:49:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.443+0000 7f6b112c9640 1 -- 172.21.15.16:0/1962805183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b0c160410 con 0x7f6b0c14c180 2024-09-19T07:49:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.443+0000 7f6b037fe640 1 -- 172.21.15.16:0/1962805183 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6afc005480 con 0x7f6b0c14c180 2024-09-19T07:49:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.443+0000 7f6b037fe640 1 --2- 172.21.15.16:0/1962805183 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6aec06c2e0 0x7f6aec06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.444+0000 7f6b037fe640 1 -- 172.21.15.16:0/1962805183 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5335+0+0 (secure 0 0 0) 0x7f6afc0ab380 con 0x7f6b0c14c180 2024-09-19T07:49:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.444+0000 7f6b0a7fc640 1 --2- 172.21.15.16:0/1962805183 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6aec06c2e0 0x7f6aec06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.444+0000 7f6b0a7fc640 1 --2- 172.21.15.16:0/1962805183 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6aec06c2e0 0x7f6aec06e7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0a22a0 tx=0x7f6af4007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.448+0000 7f6b037fe640 1 -- 172.21.15.16:0/1962805183 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6b0c160410 con 0x7f6b0c14c180 2024-09-19T07:49:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.622+0000 7f6b112c9640 1 -- 172.21.15.16:0/1962805183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6b0c0d4820 con 0x7f6b0c14c180 2024-09-19T07:49:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.622+0000 7f6b037fe640 1 -- 172.21.15.16:0/1962805183 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f6afc079bd0 con 0x7f6b0c14c180 2024-09-19T07:49:35.623 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-19T07:49:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.624+0000 7f6b112c9640 1 -- 172.21.15.16:0/1962805183 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6aec06c2e0 msgr2=0x7f6aec06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.624+0000 7f6b112c9640 1 --2- 172.21.15.16:0/1962805183 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6aec06c2e0 0x7f6aec06e7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0a22a0 tx=0x7f6af4007040 comp rx=0 tx=0).stop 2024-09-19T07:49:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.624+0000 7f6b112c9640 1 -- 172.21.15.16:0/1962805183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b0c14c180 msgr2=0x7f6b0c14f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.624+0000 7f6b112c9640 1 --2- 172.21.15.16:0/1962805183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b0c14c180 0x7f6b0c14f110 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6afc02a8f0 tx=0x7f6afc0014a0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.624+0000 7f6b112c9640 1 -- 172.21.15.16:0/1962805183 shutdown_connections 2024-09-19T07:49:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.624+0000 7f6b112c9640 1 --2- 172.21.15.16:0/1962805183 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6aec06c2e0 0x7f6aec06e7a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.624+0000 7f6b112c9640 1 --2- 172.21.15.16:0/1962805183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b0c160000 0x7f6b0c0ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.624+0000 7f6b112c9640 1 --2- 172.21.15.16:0/1962805183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b0c0ce6d0 0x7f6b0c15fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.624+0000 7f6b112c9640 1 --2- 172.21.15.16:0/1962805183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b0c14c180 0x7f6b0c14f110 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.624+0000 7f6b112c9640 1 -- 172.21.15.16:0/1962805183 >> 172.21.15.16:0/1962805183 conn(0x7f6b0c09d760 msgr2=0x7f6b0c0b9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.624+0000 7f6b112c9640 1 -- 172.21.15.16:0/1962805183 shutdown_connections 2024-09-19T07:49:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.624+0000 7f6b112c9640 1 -- 172.21.15.16:0/1962805183 wait complete. 2024-09-19T07:49:35.636 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps' 2024-09-19T07:49:35.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.738+0000 7f8f1d8ba640 1 Processor -- start 2024-09-19T07:49:35.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.738+0000 7f8f1d8ba640 1 -- start start 2024-09-19T07:49:35.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.739+0000 7f8f1d8ba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8f1815e2a0 0x7f8f1806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.739+0000 7f8f1d8ba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f1806d960 0x7f8f180d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.739+0000 7f8f1d8ba640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f180d1880 con 0x7f8f1815e680 2024-09-19T07:49:35.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.739+0000 7f8f1d8ba640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f8f180d1b60 con 0x7f8f1815e2a0 2024-09-19T07:49:35.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.739+0000 7f8f1d8ba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8f180cf2d0 con 0x7f8f1806d960 2024-09-19T07:49:35.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.739+0000 7f8f16ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8f1815e2a0 0x7f8f1806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.739+0000 7f8f167fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8f1815e680 0x7f8f180cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58984/0 (socket says 172.21.15.16:58984) 2024-09-19T07:49:35.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.739+0000 7f8f167fc640 1 -- 172.21.15.16:0/2029894332 learned_addr learned my addr 172.21.15.16:0/2029894332 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:35.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.740+0000 7f8f177fe640 1 --2- 172.21.15.16:0/2029894332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f1806d960 0x7f8f180d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.740+0000 7f8f16ffd640 1 -- 172.21.15.16:0/2029894332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f1806d960 msgr2=0x7f8f180d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:35.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.740+0000 7f8f16ffd640 1 --2- 172.21.15.16:0/2029894332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f1806d960 0x7f8f180d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.740+0000 7f8f16ffd640 1 -- 172.21.15.16:0/2029894332 >> v1:172.21.15.16:6789/0 conn(0x7f8f1815e680 legacy=0x7f8f180cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.740+0000 7f8f16ffd640 1 -- 172.21.15.16:0/2029894332 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f180cf5b0 con 0x7f8f1815e2a0 2024-09-19T07:49:35.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.740+0000 7f8f16ffd640 1 --2- 172.21.15.16:0/2029894332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f1815e2a0 0x7f8f1806d420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009d90 tx=0x7f8f0802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254cb4370864e675 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:35.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.741+0000 7f8f15ffb640 1 -- 172.21.15.16:0/2029894332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08007740 con 0x7f8f1815e2a0 2024-09-19T07:49:35.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.741+0000 7f8f15ffb640 1 -- 172.21.15.16:0/2029894332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f080078e0 con 0x7f8f1815e2a0 2024-09-19T07:49:35.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.741+0000 7f8f15ffb640 1 -- 172.21.15.16:0/2029894332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08007cb0 con 0x7f8f1815e2a0 2024-09-19T07:49:35.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.741+0000 7f8f1d8ba640 1 -- 172.21.15.16:0/2029894332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f1815e2a0 msgr2=0x7f8f1806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.741+0000 7f8f1d8ba640 1 --2- 172.21.15.16:0/2029894332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f1815e2a0 0x7f8f1806d420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009d90 tx=0x7f8f0802a660 comp rx=0 tx=0).stop 2024-09-19T07:49:35.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.741+0000 7f8f1d8ba640 1 -- 172.21.15.16:0/2029894332 shutdown_connections 2024-09-19T07:49:35.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.741+0000 7f8f1d8ba640 1 --2- 172.21.15.16:0/2029894332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f1806d960 0x7f8f180d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.741+0000 7f8f1d8ba640 1 --2- 172.21.15.16:0/2029894332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f1815e2a0 0x7f8f1806d420 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.741+0000 7f8f1d8ba640 1 -- 172.21.15.16:0/2029894332 >> 172.21.15.16:0/2029894332 conn(0x7f8f1809d760 msgr2=0x7f8f1808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:35.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.741+0000 7f8f1d8ba640 1 -- 172.21.15.16:0/2029894332 shutdown_connections 2024-09-19T07:49:35.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.741+0000 7f8f1d8ba640 1 -- 172.21.15.16:0/2029894332 wait complete. 2024-09-19T07:49:35.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.742+0000 7f8f1d8ba640 1 Processor -- start 2024-09-19T07:49:35.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.743+0000 7f8f1d8ba640 1 -- start start 2024-09-19T07:49:35.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.743+0000 7f8f1d8ba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f1806d960 0x7f8f1815d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.743+0000 7f8f1d8ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f1815e2a0 0x7f8f180cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.743+0000 7f8f1d8ba640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f1815e680 0x7f8f180ce220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.743+0000 7f8f1d8ba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f180d0360 con 0x7f8f1815e2a0 2024-09-19T07:49:35.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.743+0000 7f8f1d8ba640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8f180d01e0 con 0x7f8f1815e680 2024-09-19T07:49:35.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.743+0000 7f8f1d8ba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8f180d48b0 con 0x7f8f1806d960 2024-09-19T07:49:35.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.743+0000 7f8f167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f1815e2a0 0x7f8f180cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.744+0000 7f8f167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f1815e2a0 0x7f8f180cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53432/0 (socket says 172.21.15.16:53432) 2024-09-19T07:49:35.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.744+0000 7f8f167fc640 1 -- 172.21.15.16:0/422363929 learned_addr learned my addr 172.21.15.16:0/422363929 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:35.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.744+0000 7f8f16ffd640 1 --2- 172.21.15.16:0/422363929 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f1806d960 0x7f8f1815d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.744+0000 7f8f177fe640 1 --2- 172.21.15.16:0/422363929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f1815e680 0x7f8f180ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.744+0000 7f8f167fc640 1 -- 172.21.15.16:0/422363929 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f1806d960 msgr2=0x7f8f1815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.744+0000 7f8f167fc640 1 --2- 172.21.15.16:0/422363929 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f1806d960 0x7f8f1815d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.744+0000 7f8f167fc640 1 -- 172.21.15.16:0/422363929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f1815e680 msgr2=0x7f8f180ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.744+0000 7f8f167fc640 1 --2- 172.21.15.16:0/422363929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f1815e680 0x7f8f180ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.744+0000 7f8f167fc640 1 -- 172.21.15.16:0/422363929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f181f7ef0 con 0x7f8f1815e2a0 2024-09-19T07:49:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.744+0000 7f8f167fc640 1 --2- 172.21.15.16:0/422363929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f1815e2a0 0x7f8f180cdce0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000c440 tx=0x7f8f0000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.744+0000 7f8f0ffff640 1 -- 172.21.15.16:0/422363929 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00017070 con 0x7f8f1815e2a0 2024-09-19T07:49:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.744+0000 7f8f0ffff640 1 -- 172.21.15.16:0/422363929 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f00005ce0 con 0x7f8f1815e2a0 2024-09-19T07:49:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.744+0000 7f8f1d8ba640 1 -- 172.21.15.16:0/422363929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f181f81d0 con 0x7f8f1815e2a0 2024-09-19T07:49:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.744+0000 7f8f0ffff640 1 -- 172.21.15.16:0/422363929 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f000072f0 con 0x7f8f1815e2a0 2024-09-19T07:49:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.744+0000 7f8f1d8ba640 1 -- 172.21.15.16:0/422363929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f181f8790 con 0x7f8f1815e2a0 2024-09-19T07:49:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.745+0000 7f8f1d8ba640 1 -- 172.21.15.16:0/422363929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f181f8c80 con 0x7f8f1815e2a0 2024-09-19T07:49:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.746+0000 7f8f0ffff640 1 -- 172.21.15.16:0/422363929 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8f0001a070 con 0x7f8f1815e2a0 2024-09-19T07:49:35.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.746+0000 7f8f0ffff640 1 --2- 172.21.15.16:0/422363929 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8ef806c2b0 0x7f8ef806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.746+0000 7f8f0ffff640 1 -- 172.21.15.16:0/422363929 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5335+0+0 (secure 0 0 0) 0x7f8f0008a550 con 0x7f8f1815e2a0 2024-09-19T07:49:35.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.747+0000 7f8f16ffd640 1 --2- 172.21.15.16:0/422363929 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8ef806c2b0 0x7f8ef806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.747+0000 7f8f16ffd640 1 --2- 172.21.15.16:0/422363929 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8ef806c2b0 0x7f8ef806e770 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8f08034000 tx=0x7f8f0802f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:35.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.750+0000 7f8f0ffff640 1 -- 172.21.15.16:0/422363929 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8f000587e0 con 0x7f8f1815e2a0 2024-09-19T07:49:35.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:35.992+0000 7f8f1d8ba640 1 -- 172.21.15.16:0/422363929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]} v 0) -- 0x7f8f180d4600 con 0x7f8f1815e2a0 2024-09-19T07:49:36.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.012+0000 7f8f0ffff640 1 -- 172.21.15.16:0/422363929 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]}]=0 v285) ==== 151+0+0 (secure 0 0 0) 0x7f8f0005cae0 con 0x7f8f1815e2a0 2024-09-19T07:49:36.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.014+0000 7f8f1d8ba640 1 -- 172.21.15.16:0/422363929 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8ef806c2b0 msgr2=0x7f8ef806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:36.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.014+0000 7f8f1d8ba640 1 --2- 172.21.15.16:0/422363929 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8ef806c2b0 0x7f8ef806e770 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8f08034000 tx=0x7f8f0802f000 comp rx=0 tx=0).stop 2024-09-19T07:49:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.014+0000 7f8f1d8ba640 1 -- 172.21.15.16:0/422363929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f1815e2a0 msgr2=0x7f8f180cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.014+0000 7f8f1d8ba640 1 --2- 172.21.15.16:0/422363929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f1815e2a0 0x7f8f180cdce0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000c440 tx=0x7f8f0000c900 comp rx=0 tx=0).stop 2024-09-19T07:49:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.014+0000 7f8f1d8ba640 1 -- 172.21.15.16:0/422363929 shutdown_connections 2024-09-19T07:49:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.014+0000 7f8f1d8ba640 1 --2- 172.21.15.16:0/422363929 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8ef806c2b0 0x7f8ef806e770 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.014+0000 7f8f1d8ba640 1 --2- 172.21.15.16:0/422363929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8f1815e680 0x7f8f180ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.014+0000 7f8f1d8ba640 1 --2- 172.21.15.16:0/422363929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f1815e2a0 0x7f8f180cdce0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.014+0000 7f8f1d8ba640 1 --2- 172.21.15.16:0/422363929 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8f1806d960 0x7f8f1815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.014+0000 7f8f1d8ba640 1 -- 172.21.15.16:0/422363929 >> 172.21.15.16:0/422363929 conn(0x7f8f1809d760 msgr2=0x7f8f18161440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.014+0000 7f8f1d8ba640 1 -- 172.21.15.16:0/422363929 shutdown_connections 2024-09-19T07:49:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.014+0000 7f8f1d8ba640 1 -- 172.21.15.16:0/422363929 wait complete. 2024-09-19T07:49:36.026 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T07:49:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.130+0000 7f7e92eaf640 1 Processor -- start 2024-09-19T07:49:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.130+0000 7f7e92eaf640 1 -- start start 2024-09-19T07:49:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.131+0000 7f7e92eaf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e8c141680 0x7f7e8c0cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.131+0000 7f7e92eaf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c142170 0x7f7e8c13f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.131+0000 7f7e92eaf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e8c15d710 con 0x7f7e8c0d0370 2024-09-19T07:49:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.131+0000 7f7e92eaf640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f7e8c0d07e0 con 0x7f7e8c141680 2024-09-19T07:49:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.131+0000 7f7e92eaf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7e8c15c6f0 con 0x7f7e8c142170 2024-09-19T07:49:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.131+0000 7f7e90c4b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e8c141680 0x7f7e8c0cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.131+0000 7f7e8bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7e8c0d0370 0x7f7e8c141a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59000/0 (socket says 172.21.15.16:59000) 2024-09-19T07:49:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.131+0000 7f7e8bfff640 1 -- 172.21.15.16:0/2353598595 learned_addr learned my addr 172.21.15.16:0/2353598595 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.131+0000 7f7e9144c640 1 --2- 172.21.15.16:0/2353598595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c142170 0x7f7e8c13f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.131+0000 7f7e90c4b640 1 -- 172.21.15.16:0/2353598595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c142170 msgr2=0x7f7e8c13f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.131+0000 7f7e90c4b640 1 --2- 172.21.15.16:0/2353598595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c142170 0x7f7e8c13f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.131+0000 7f7e90c4b640 1 -- 172.21.15.16:0/2353598595 >> v1:172.21.15.16:6789/0 conn(0x7f7e8c0d0370 legacy=0x7f7e8c141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.131+0000 7f7e90c4b640 1 -- 172.21.15.16:0/2353598595 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8c15c950 con 0x7f7e8c141680 2024-09-19T07:49:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.131+0000 7f7e8b7fe640 1 -- 172.21.15.16:0/2353598595 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 207440761 0 0) 0x7f7e8c15d710 con 0x7f7e8c0d0370 2024-09-19T07:49:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.132+0000 7f7e90c4b640 1 --2- 172.21.15.16:0/2353598595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e8c141680 0x7f7e8c0cfe30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009dc0 tx=0x7f7e8402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca2dabd9d04e123 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.132+0000 7f7e8b7fe640 1 -- 172.21.15.16:0/2353598595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8402ae60 con 0x7f7e8c141680 2024-09-19T07:49:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.132+0000 7f7e8b7fe640 1 -- 172.21.15.16:0/2353598595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e84007740 con 0x7f7e8c141680 2024-09-19T07:49:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.132+0000 7f7e8b7fe640 1 -- 172.21.15.16:0/2353598595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84007b10 con 0x7f7e8c141680 2024-09-19T07:49:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.133+0000 7f7e92eaf640 1 -- 172.21.15.16:0/2353598595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e8c141680 msgr2=0x7f7e8c0cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.133+0000 7f7e92eaf640 1 --2- 172.21.15.16:0/2353598595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e8c141680 0x7f7e8c0cfe30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009dc0 tx=0x7f7e8402a680 comp rx=0 tx=0).stop 2024-09-19T07:49:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.133+0000 7f7e92eaf640 1 -- 172.21.15.16:0/2353598595 shutdown_connections 2024-09-19T07:49:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.133+0000 7f7e92eaf640 1 --2- 172.21.15.16:0/2353598595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c142170 0x7f7e8c13f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.133+0000 7f7e92eaf640 1 --2- 172.21.15.16:0/2353598595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e8c141680 0x7f7e8c0cfe30 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.133+0000 7f7e92eaf640 1 -- 172.21.15.16:0/2353598595 >> 172.21.15.16:0/2353598595 conn(0x7f7e8c0b3ed0 msgr2=0x7f7e8c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:36.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.133+0000 7f7e92eaf640 1 -- 172.21.15.16:0/2353598595 shutdown_connections 2024-09-19T07:49:36.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.133+0000 7f7e92eaf640 1 -- 172.21.15.16:0/2353598595 wait complete. 2024-09-19T07:49:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.134+0000 7f7e92eaf640 1 Processor -- start 2024-09-19T07:49:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.134+0000 7f7e92eaf640 1 -- start start 2024-09-19T07:49:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.135+0000 7f7e92eaf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c0d0370 0x7f7e8c0b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.135+0000 7f7e92eaf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e8c141680 0x7f7e8c1f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.135+0000 7f7e92eaf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c142170 0x7f7e8c0b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.135+0000 7f7e92eaf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c15d1f0 con 0x7f7e8c141680 2024-09-19T07:49:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.135+0000 7f7e92eaf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c15cf10 con 0x7f7e8c142170 2024-09-19T07:49:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.135+0000 7f7e92eaf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7e8c1425c0 con 0x7f7e8c0d0370 2024-09-19T07:49:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.135+0000 7f7e8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e8c141680 0x7f7e8c1f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.135+0000 7f7e8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e8c141680 0x7f7e8c1f4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53464/0 (socket says 172.21.15.16:53464) 2024-09-19T07:49:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.135+0000 7f7e8bfff640 1 -- 172.21.15.16:0/3062175436 learned_addr learned my addr 172.21.15.16:0/3062175436 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.135+0000 7f7e8bfff640 1 -- 172.21.15.16:0/3062175436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c0d0370 msgr2=0x7f7e8c0b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.135+0000 7f7e8bfff640 1 --2- 172.21.15.16:0/3062175436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c0d0370 0x7f7e8c0b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.136+0000 7f7e8bfff640 1 -- 172.21.15.16:0/3062175436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c142170 msgr2=0x7f7e8c0b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.136+0000 7f7e8bfff640 1 --2- 172.21.15.16:0/3062175436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c142170 0x7f7e8c0b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.136+0000 7f7e8bfff640 1 -- 172.21.15.16:0/3062175436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8c1f7d30 con 0x7f7e8c141680 2024-09-19T07:49:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.136+0000 7f7e8bfff640 1 --2- 172.21.15.16:0/3062175436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e8c141680 0x7f7e8c1f4bf0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000c4a0 tx=0x7f7e8000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.136+0000 7f7e897fa640 1 -- 172.21.15.16:0/3062175436 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80017070 con 0x7f7e8c141680 2024-09-19T07:49:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.136+0000 7f7e897fa640 1 -- 172.21.15.16:0/3062175436 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e80005ce0 con 0x7f7e8c141680 2024-09-19T07:49:36.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.136+0000 7f7e92eaf640 1 -- 172.21.15.16:0/3062175436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8c1f8010 con 0x7f7e8c141680 2024-09-19T07:49:36.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.136+0000 7f7e897fa640 1 -- 172.21.15.16:0/3062175436 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e800072b0 con 0x7f7e8c141680 2024-09-19T07:49:36.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.136+0000 7f7e92eaf640 1 -- 172.21.15.16:0/3062175436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e8c1f85d0 con 0x7f7e8c141680 2024-09-19T07:49:36.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.137+0000 7f7e92eaf640 1 -- 172.21.15.16:0/3062175436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e8c0cfe30 con 0x7f7e8c141680 2024-09-19T07:49:36.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.137+0000 7f7e897fa640 1 -- 172.21.15.16:0/3062175436 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7e8001a070 con 0x7f7e8c141680 2024-09-19T07:49:36.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.138+0000 7f7e897fa640 1 --2- 172.21.15.16:0/3062175436 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7e7806c2b0 0x7f7e7806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:36.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.138+0000 7f7e897fa640 1 -- 172.21.15.16:0/3062175436 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5335+0+0 (secure 0 0 0) 0x7f7e8008a6c0 con 0x7f7e8c141680 2024-09-19T07:49:36.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.138+0000 7f7e90c4b640 1 --2- 172.21.15.16:0/3062175436 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7e7806c2b0 0x7f7e7806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:36.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.139+0000 7f7e90c4b640 1 --2- 172.21.15.16:0/3062175436 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7e7806c2b0 0x7f7e7806e770 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7e84034000 tx=0x7f7e8402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:36.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.142+0000 7f7e897fa640 1 -- 172.21.15.16:0/3062175436 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7e80058fa0 con 0x7f7e8c141680 2024-09-19T07:49:36.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.317+0000 7f7e92eaf640 1 -- 172.21.15.16:0/3062175436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f7e8c0690c0 con 0x7f7e8c141680 2024-09-19T07:49:36.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.320+0000 7f7e897fa640 1 -- 172.21.15.16:0/3062175436 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v27)=0 v27) ==== 122+0+0 (secure 0 0 0) 0x7f7e800588a0 con 0x7f7e8c141680 2024-09-19T07:49:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.322+0000 7f7e92eaf640 1 -- 172.21.15.16:0/3062175436 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7e7806c2b0 msgr2=0x7f7e7806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.322+0000 7f7e92eaf640 1 --2- 172.21.15.16:0/3062175436 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7e7806c2b0 0x7f7e7806e770 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7e84034000 tx=0x7f7e8402e000 comp rx=0 tx=0).stop 2024-09-19T07:49:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.322+0000 7f7e92eaf640 1 -- 172.21.15.16:0/3062175436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e8c141680 msgr2=0x7f7e8c1f4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.322+0000 7f7e92eaf640 1 --2- 172.21.15.16:0/3062175436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e8c141680 0x7f7e8c1f4bf0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000c4a0 tx=0x7f7e8000c960 comp rx=0 tx=0).stop 2024-09-19T07:49:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.322+0000 7f7e92eaf640 1 -- 172.21.15.16:0/3062175436 shutdown_connections 2024-09-19T07:49:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.322+0000 7f7e92eaf640 1 --2- 172.21.15.16:0/3062175436 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7e7806c2b0 0x7f7e7806e770 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.322+0000 7f7e92eaf640 1 --2- 172.21.15.16:0/3062175436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c142170 0x7f7e8c0b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.322+0000 7f7e92eaf640 1 --2- 172.21.15.16:0/3062175436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e8c141680 0x7f7e8c1f4bf0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.322+0000 7f7e92eaf640 1 --2- 172.21.15.16:0/3062175436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c0d0370 0x7f7e8c0b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.322+0000 7f7e92eaf640 1 -- 172.21.15.16:0/3062175436 >> 172.21.15.16:0/3062175436 conn(0x7f7e8c0b3ed0 msgr2=0x7f7e8c0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.322+0000 7f7e92eaf640 1 -- 172.21.15.16:0/3062175436 shutdown_connections 2024-09-19T07:49:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.322+0000 7f7e92eaf640 1 -- 172.21.15.16:0/3062175436 wait complete. 2024-09-19T07:49:36.334 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:49:36.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:49:36.373 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:49:36.420 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:49:36.420 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:49:36.420 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:49:36.436 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:49:36.436 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:49:36.436 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:49:36.455 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:49:36.455 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:49:36.455 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:49:36.492 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:49:36.492 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:49:36.492 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:49:36.513 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:49:36.513 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T07:49:36.514 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:49:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.634+0000 7f6f2a877640 1 Processor -- start 2024-09-19T07:49:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.634+0000 7f6f2a877640 1 -- start start 2024-09-19T07:49:36.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.635+0000 7f6f2a877640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2415e2a0 0x7f6f2406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.635+0000 7f6f2a877640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6f2415e680 0x7f6f240cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.635+0000 7f6f2a877640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f240d1880 con 0x7f6f2406d960 2024-09-19T07:49:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.635+0000 7f6f2a877640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f6f240d1b60 con 0x7f6f2415e680 2024-09-19T07:49:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.635+0000 7f6f2a877640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6f240cf2d0 con 0x7f6f2415e2a0 2024-09-19T07:49:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.635+0000 7f6f23fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2415e2a0 0x7f6f2406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.635+0000 7f6f237fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6f2415e680 0x7f6f240cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.635+0000 7f6f28e14640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f6f2406d960 0x7f6f240d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:55888/0 (socket says 172.21.15.16:55888) 2024-09-19T07:49:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.635+0000 7f6f237fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6f2415e680 0x7f6f240cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:52862/0 (socket says 172.21.15.16:52862) 2024-09-19T07:49:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.635+0000 7f6f28e14640 1 -- 172.21.15.16:0/3669361310 learned_addr learned my addr 172.21.15.16:0/3669361310 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.635+0000 7f6f23fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2415e2a0 0x7f6f2406d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:49342/0 (socket says 172.21.15.16:49342) 2024-09-19T07:49:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.635+0000 7f6f23fff640 1 -- 172.21.15.16:0/3669361310 >> v2:172.21.15.146:3300/0 conn(0x7f6f2415e680 msgr2=0x7f6f240cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.635+0000 7f6f23fff640 1 --2- 172.21.15.16:0/3669361310 >> v2:172.21.15.146:3300/0 conn(0x7f6f2415e680 0x7f6f240cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.635+0000 7f6f23fff640 1 -- 172.21.15.16:0/3669361310 >> v1:172.21.15.146:6789/0 conn(0x7f6f2406d960 legacy=0x7f6f240d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.635+0000 7f6f23fff640 1 -- 172.21.15.16:0/3669361310 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f240cf5b0 con 0x7f6f2415e2a0 2024-09-19T07:49:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.635+0000 7f6f237fe640 1 --2- 172.21.15.16:0/3669361310 >> v2:172.21.15.146:3300/0 conn(0x7f6f2415e680 0x7f6f240cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:49:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.636+0000 7f6f23fff640 1 --2- 172.21.15.16:0/3669361310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2415e2a0 0x7f6f2406d420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6f140097a0 tx=0x7f6f1402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c80e1d5f8d1fb9cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.636+0000 7f6f22ffd640 1 -- 172.21.15.16:0/3669361310 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f14037070 con 0x7f6f2415e2a0 2024-09-19T07:49:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.636+0000 7f6f22ffd640 1 -- 172.21.15.16:0/3669361310 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f14032440 con 0x7f6f2415e2a0 2024-09-19T07:49:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.636+0000 7f6f22ffd640 1 -- 172.21.15.16:0/3669361310 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f14032810 con 0x7f6f2415e2a0 2024-09-19T07:49:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.637+0000 7f6f2a877640 1 -- 172.21.15.16:0/3669361310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2415e2a0 msgr2=0x7f6f2406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.637+0000 7f6f2a877640 1 --2- 172.21.15.16:0/3669361310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2415e2a0 0x7f6f2406d420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6f140097a0 tx=0x7f6f1402a180 comp rx=0 tx=0).stop 2024-09-19T07:49:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.637+0000 7f6f2a877640 1 -- 172.21.15.16:0/3669361310 shutdown_connections 2024-09-19T07:49:36.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.637+0000 7f6f2a877640 1 --2- 172.21.15.16:0/3669361310 >> v2:172.21.15.146:3300/0 conn(0x7f6f2415e680 0x7f6f240cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.637+0000 7f6f2a877640 1 --2- 172.21.15.16:0/3669361310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2415e2a0 0x7f6f2406d420 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.637+0000 7f6f2a877640 1 -- 172.21.15.16:0/3669361310 >> 172.21.15.16:0/3669361310 conn(0x7f6f2409d760 msgr2=0x7f6f2408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:36.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.637+0000 7f6f2a877640 1 -- 172.21.15.16:0/3669361310 shutdown_connections 2024-09-19T07:49:36.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.637+0000 7f6f2a877640 1 -- 172.21.15.16:0/3669361310 wait complete. 2024-09-19T07:49:36.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.638+0000 7f6f2a877640 1 Processor -- start 2024-09-19T07:49:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.638+0000 7f6f2a877640 1 -- start start 2024-09-19T07:49:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.639+0000 7f6f2a877640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2406d960 0x7f6f240cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.639+0000 7f6f2a877640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f2415e2a0 0x7f6f24158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.639+0000 7f6f2a877640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f2415e680 0x7f6f24159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.639+0000 7f6f2a877640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f240d4c00 con 0x7f6f2415e2a0 2024-09-19T07:49:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.639+0000 7f6f2a877640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6f240d4a80 con 0x7f6f2415e680 2024-09-19T07:49:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.639+0000 7f6f2a877640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6f240d0010 con 0x7f6f2406d960 2024-09-19T07:49:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.639+0000 7f6f237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f2415e2a0 0x7f6f24158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.639+0000 7f6f237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f2415e2a0 0x7f6f24158f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53478/0 (socket says 172.21.15.16:53478) 2024-09-19T07:49:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.639+0000 7f6f237fe640 1 -- 172.21.15.16:0/3394915367 learned_addr learned my addr 172.21.15.16:0/3394915367 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.639+0000 7f6f28e14640 1 --2- 172.21.15.16:0/3394915367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f2415e680 0x7f6f24159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.639+0000 7f6f23fff640 1 --2- 172.21.15.16:0/3394915367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2406d960 0x7f6f240cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.640+0000 7f6f237fe640 1 -- 172.21.15.16:0/3394915367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2406d960 msgr2=0x7f6f240cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.640+0000 7f6f237fe640 1 --2- 172.21.15.16:0/3394915367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2406d960 0x7f6f240cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.640+0000 7f6f237fe640 1 -- 172.21.15.16:0/3394915367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f2415e680 msgr2=0x7f6f24159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.640+0000 7f6f237fe640 1 --2- 172.21.15.16:0/3394915367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f2415e680 0x7f6f24159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.640+0000 7f6f237fe640 1 -- 172.21.15.16:0/3394915367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f240b9df0 con 0x7f6f2415e2a0 2024-09-19T07:49:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.640+0000 7f6f237fe640 1 --2- 172.21.15.16:0/3394915367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f2415e2a0 0x7f6f24158f50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800e8e0 tx=0x7f6f1800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.640+0000 7f6f20ff9640 1 -- 172.21.15.16:0/3394915367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18017070 con 0x7f6f2415e2a0 2024-09-19T07:49:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.640+0000 7f6f20ff9640 1 -- 172.21.15.16:0/3394915367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f1800c8e0 con 0x7f6f2415e2a0 2024-09-19T07:49:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.640+0000 7f6f20ff9640 1 -- 172.21.15.16:0/3394915367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1800ccb0 con 0x7f6f2415e2a0 2024-09-19T07:49:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.640+0000 7f6f2a877640 1 -- 172.21.15.16:0/3394915367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f240ba0d0 con 0x7f6f2415e2a0 2024-09-19T07:49:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.640+0000 7f6f2a877640 1 -- 172.21.15.16:0/3394915367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f240c88f0 con 0x7f6f2415e2a0 2024-09-19T07:49:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.641+0000 7f6f2a877640 1 -- 172.21.15.16:0/3394915367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f2406d420 con 0x7f6f2415e2a0 2024-09-19T07:49:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.642+0000 7f6f20ff9640 1 -- 172.21.15.16:0/3394915367 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6f1801a070 con 0x7f6f2415e2a0 2024-09-19T07:49:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.642+0000 7f6f20ff9640 1 --2- 172.21.15.16:0/3394915367 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6f1006c2b0 0x7f6f1006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.642+0000 7f6f20ff9640 1 -- 172.21.15.16:0/3394915367 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5335+0+0 (secure 0 0 0) 0x7f6f1808b4d0 con 0x7f6f2415e2a0 2024-09-19T07:49:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.642+0000 7f6f23fff640 1 --2- 172.21.15.16:0/3394915367 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6f1006c2b0 0x7f6f1006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.643+0000 7f6f23fff640 1 --2- 172.21.15.16:0/3394915367 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6f1006c2b0 0x7f6f1006e770 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6f14034000 tx=0x7f6f1402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:36.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.646+0000 7f6f20ff9640 1 -- 172.21.15.16:0/3394915367 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6f180596d0 con 0x7f6f2415e2a0 2024-09-19T07:49:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.858+0000 7f6f2a877640 1 -- 172.21.15.16:0/3394915367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f2406cf90 con 0x7f6f2415e2a0 2024-09-19T07:49:36.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.858+0000 7f6f20ff9640 1 -- 172.21.15.16:0/3394915367 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+3914 (secure 0 0 0) 0x7f6f1805d9d0 con 0x7f6f2415e2a0 2024-09-19T07:49:36.859 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:36.859 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":35,"btime":"2024-09-19T07:49:31:998077+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2104039348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2104039348},{"type":"v1","addr":"172.21.15.16:6835","nonce":2104039348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/1109733621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1109733621},{"type":"v1","addr":"172.21.15.146:6836","nonce":1109733621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3602092462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3602092462},{"type":"v1","addr":"172.21.15.146:6837","nonce":3602092462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/532806980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":532806980},{"type":"v1","addr":"172.21.15.16:6834","nonce":532806980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[]} 2024-09-19T07:49:36.859 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 35 2024-09-19T07:49:36.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.860+0000 7f6f2a877640 1 -- 172.21.15.16:0/3394915367 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6f1006c2b0 msgr2=0x7f6f1006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:36.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.860+0000 7f6f2a877640 1 --2- 172.21.15.16:0/3394915367 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6f1006c2b0 0x7f6f1006e770 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6f14034000 tx=0x7f6f1402e000 comp rx=0 tx=0).stop 2024-09-19T07:49:36.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.860+0000 7f6f2a877640 1 -- 172.21.15.16:0/3394915367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f2415e2a0 msgr2=0x7f6f24158f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:36.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.860+0000 7f6f2a877640 1 --2- 172.21.15.16:0/3394915367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f2415e2a0 0x7f6f24158f50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800e8e0 tx=0x7f6f1800eda0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.860+0000 7f6f2a877640 1 -- 172.21.15.16:0/3394915367 shutdown_connections 2024-09-19T07:49:36.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.860+0000 7f6f2a877640 1 --2- 172.21.15.16:0/3394915367 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6f1006c2b0 0x7f6f1006e770 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.860+0000 7f6f2a877640 1 --2- 172.21.15.16:0/3394915367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f2415e680 0x7f6f24159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.860+0000 7f6f2a877640 1 --2- 172.21.15.16:0/3394915367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f2415e2a0 0x7f6f24158f50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.860+0000 7f6f2a877640 1 --2- 172.21.15.16:0/3394915367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f2406d960 0x7f6f240cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:36.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.860+0000 7f6f2a877640 1 -- 172.21.15.16:0/3394915367 >> 172.21.15.16:0/3394915367 conn(0x7f6f2409d760 msgr2=0x7f6f240d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:36.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.861+0000 7f6f2a877640 1 -- 172.21.15.16:0/3394915367 shutdown_connections 2024-09-19T07:49:36.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:36.861+0000 7f6f2a877640 1 -- 172.21.15.16:0/3394915367 wait complete. 2024-09-19T07:49:36.872 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T07:49:36.872 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T07:49:36.873 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T07:49:36.873 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:49:36.873 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T07:49:36.873 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T07:49:36.873 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T07:49:36.873 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:49:36.873 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T07:49:36.874 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T07:49:36.874 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T07:49:36.874 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:49:36.874 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T07:49:36.874 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T07:49:36.874 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T07:49:36.874 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:49:36.882 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T07:49:36.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T07:49:36.883 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T07:49:36.883 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T07:49:36.886 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T07:49:36.886 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T07:49:36.887 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T07:49:36.887 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T07:49:36.925 INFO:tasks.ceph.mds.a:Started 2024-09-19T07:49:36.925 DEBUG:teuthology.parallel:result is None 2024-09-19T07:49:36.926 INFO:tasks.ceph.mds.c:Started 2024-09-19T07:49:36.926 DEBUG:teuthology.parallel:result is None 2024-09-19T07:49:36.929 INFO:tasks.ceph.mds.b:Started 2024-09-19T07:49:36.930 DEBUG:teuthology.parallel:result is None 2024-09-19T07:49:36.930 INFO:tasks.ceph.mds.d:Started 2024-09-19T07:49:36.930 DEBUG:teuthology.parallel:result is None 2024-09-19T07:49:36.931 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T07:49:37.002 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-19T07:49:37.003 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-19T07:49:37.012 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2024-09-19T07:49:37.012 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2024-09-19T07:49:37.012 INFO:tasks.ceph.mds.d.smithi146.stderr:2024-09-19T07:49:37.013+0000 7f7f45f7c640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-19T07:49:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.037+0000 7f56c4398640 1 Processor -- start 2024-09-19T07:49:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.037+0000 7f56c4398640 1 -- start start 2024-09-19T07:49:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.038+0000 7f56c4398640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56bc15f0d0 0x7f56bc15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.038+0000 7f56c4398640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f56bc0d4130 0x7f56bc0d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.038+0000 7f56c4398640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56bc0d4b20 con 0x7f56bc0d0070 2024-09-19T07:49:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.038+0000 7f56c4398640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f56bc0d1b10 con 0x7f56bc0d4130 2024-09-19T07:49:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.038+0000 7f56c4398640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f56bc0cf2b0 con 0x7f56bc15f0d0 2024-09-19T07:49:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.038+0000 7f56c2935640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f56bc0d4130 0x7f56bc0d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.038+0000 7f56c1933640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f56bc0d0070 0x7f56bc0d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59016/0 (socket says 172.21.15.16:59016) 2024-09-19T07:49:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.038+0000 7f56c1933640 1 -- 172.21.15.16:0/3563983364 learned_addr learned my addr 172.21.15.16:0/3563983364 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.039+0000 7f56c2134640 1 --2- 172.21.15.16:0/3563983364 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56bc15f0d0 0x7f56bc15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.039+0000 7f56c2935640 1 -- 172.21.15.16:0/3563983364 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56bc15f0d0 msgr2=0x7f56bc15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:37.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.039+0000 7f56c2935640 1 --2- 172.21.15.16:0/3563983364 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56bc15f0d0 0x7f56bc15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.039+0000 7f56c2935640 1 -- 172.21.15.16:0/3563983364 >> v1:172.21.15.16:6789/0 conn(0x7f56bc0d0070 legacy=0x7f56bc0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:37.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.039+0000 7f56c1132640 1 -- 172.21.15.16:0/3563983364 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3389072642 0 0) 0x7f56ac001110 con 0x7f56bc0d0070 2024-09-19T07:49:37.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.039+0000 7f56c2935640 1 -- 172.21.15.16:0/3563983364 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc0cf590 con 0x7f56bc0d4130 2024-09-19T07:49:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.039+0000 7f56c2935640 1 --2- 172.21.15.16:0/3563983364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56bc0d4130 0x7f56bc0d45e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0d2fc0 tx=0x7f56b402a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2fc8a16d7c6cdd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.040+0000 7f56c1132640 1 -- 172.21.15.16:0/3563983364 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4007740 con 0x7f56bc0d4130 2024-09-19T07:49:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.040+0000 7f56c1132640 1 -- 172.21.15.16:0/3563983364 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56b40078e0 con 0x7f56bc0d4130 2024-09-19T07:49:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.040+0000 7f56c1132640 1 -- 172.21.15.16:0/3563983364 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4007cb0 con 0x7f56bc0d4130 2024-09-19T07:49:37.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.040+0000 7f56c4398640 1 -- 172.21.15.16:0/3563983364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56bc0d4130 msgr2=0x7f56bc0d45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:37.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.040+0000 7f56c4398640 1 --2- 172.21.15.16:0/3563983364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56bc0d4130 0x7f56bc0d45e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0d2fc0 tx=0x7f56b402a5b0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.040+0000 7f56c4398640 1 -- 172.21.15.16:0/3563983364 shutdown_connections 2024-09-19T07:49:37.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.040+0000 7f56c4398640 1 --2- 172.21.15.16:0/3563983364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56bc0d4130 0x7f56bc0d45e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.040+0000 7f56c4398640 1 --2- 172.21.15.16:0/3563983364 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56bc15f0d0 0x7f56bc15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.040+0000 7f56c4398640 1 -- 172.21.15.16:0/3563983364 >> 172.21.15.16:0/3563983364 conn(0x7f56bc09d760 msgr2=0x7f56bc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:37.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.040+0000 7f56c4398640 1 -- 172.21.15.16:0/3563983364 shutdown_connections 2024-09-19T07:49:37.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.040+0000 7f56c4398640 1 -- 172.21.15.16:0/3563983364 wait complete. 2024-09-19T07:49:37.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.042+0000 7f56c4398640 1 Processor -- start 2024-09-19T07:49:37.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.042+0000 7f56c4398640 1 -- start start 2024-09-19T07:49:37.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.042+0000 7f56c4398640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56bc0d0070 0x7f56bc0cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.042+0000 7f56c4398640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56bc0d4130 0x7f56bc15b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.042+0000 7f56c4398640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56bc15f0d0 0x7f56bc0b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.042+0000 7f56c4398640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc0cef60 con 0x7f56bc0d0070 2024-09-19T07:49:37.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.042+0000 7f56c4398640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc0cfb50 con 0x7f56bc0d4130 2024-09-19T07:49:37.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.042+0000 7f56c2134640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56bc0d0070 0x7f56bc0cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.042+0000 7f56c4398640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f56bc0b2f70 con 0x7f56bc15f0d0 2024-09-19T07:49:37.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.042+0000 7f56c2134640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56bc0d0070 0x7f56bc0cd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53512/0 (socket says 172.21.15.16:53512) 2024-09-19T07:49:37.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.042+0000 7f56c2134640 1 -- 172.21.15.16:0/2116762874 learned_addr learned my addr 172.21.15.16:0/2116762874 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:37.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.043+0000 7f56c2935640 1 --2- 172.21.15.16:0/2116762874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56bc15f0d0 0x7f56bc0b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.043+0000 7f56c1933640 1 --2- 172.21.15.16:0/2116762874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56bc0d4130 0x7f56bc15b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.043+0000 7f56c2134640 1 -- 172.21.15.16:0/2116762874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56bc15f0d0 msgr2=0x7f56bc0b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:37.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.043+0000 7f56c2134640 1 --2- 172.21.15.16:0/2116762874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56bc15f0d0 0x7f56bc0b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.043+0000 7f56c2134640 1 -- 172.21.15.16:0/2116762874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56bc0d4130 msgr2=0x7f56bc15b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:37.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.043+0000 7f56c2134640 1 --2- 172.21.15.16:0/2116762874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56bc0d4130 0x7f56bc15b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.043+0000 7f56c2134640 1 -- 172.21.15.16:0/2116762874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc0cdba0 con 0x7f56bc0d0070 2024-09-19T07:49:37.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.043+0000 7f56c2134640 1 --2- 172.21.15.16:0/2116762874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56bc0d0070 0x7f56bc0cd660 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f56b800d5f0 tx=0x7f56b800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:37.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.043+0000 7f56b2ffd640 1 -- 172.21.15.16:0/2116762874 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b80125f0 con 0x7f56bc0d0070 2024-09-19T07:49:37.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.043+0000 7f56b2ffd640 1 -- 172.21.15.16:0/2116762874 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56b8019070 con 0x7f56bc0d0070 2024-09-19T07:49:37.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.043+0000 7f56c4398640 1 -- 172.21.15.16:0/2116762874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56bc15bb70 con 0x7f56bc0d0070 2024-09-19T07:49:37.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.043+0000 7f56b2ffd640 1 -- 172.21.15.16:0/2116762874 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b80129d0 con 0x7f56bc0d0070 2024-09-19T07:49:37.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.043+0000 7f56c4398640 1 -- 172.21.15.16:0/2116762874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56bc0ce140 con 0x7f56bc0d0070 2024-09-19T07:49:37.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.044+0000 7f56c4398640 1 -- 172.21.15.16:0/2116762874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56bc0d45e0 con 0x7f56bc0d0070 2024-09-19T07:49:37.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.045+0000 7f56b2ffd640 1 -- 172.21.15.16:0/2116762874 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f56b8012b70 con 0x7f56bc0d0070 2024-09-19T07:49:37.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.045+0000 7f56b2ffd640 1 --2- 172.21.15.16:0/2116762874 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f56a006c2b0 0x7f56a006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.045+0000 7f56b2ffd640 1 -- 172.21.15.16:0/2116762874 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5335+0+0 (secure 0 0 0) 0x7f56b808c020 con 0x7f56bc0d0070 2024-09-19T07:49:37.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.046+0000 7f56c1933640 1 --2- 172.21.15.16:0/2116762874 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f56a006c2b0 0x7f56a006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.046+0000 7f56c1933640 1 --2- 172.21.15.16:0/2116762874 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f56a006c2b0 0x7f56a006e770 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0cb3b0 tx=0x7f56ac007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:37.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.049+0000 7f56b2ffd640 1 -- 172.21.15.16:0/2116762874 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f56b805a320 con 0x7f56bc0d0070 2024-09-19T07:49:37.089 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-19T07:49:37.089 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-19T07:49:37.089 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-19T07:49:37.089 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-19T07:49:37.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.235+0000 7f56c4398640 1 -- 172.21.15.16:0/2116762874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f56bc06cf30 con 0x7f56bc0d0070 2024-09-19T07:49:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.328+0000 7f56b2ffd640 1 -- 172.21.15.16:0/2116762874 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v77) ==== 97+0+0 (secure 0 0 0) 0x7f56b805e620 con 0x7f56bc0d0070 2024-09-19T07:49:37.329 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-19T07:49:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.332+0000 7f56c4398640 1 -- 172.21.15.16:0/2116762874 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f56a006c2b0 msgr2=0x7f56a006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.332+0000 7f56c4398640 1 --2- 172.21.15.16:0/2116762874 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f56a006c2b0 0x7f56a006e770 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0cb3b0 tx=0x7f56ac007b50 comp rx=0 tx=0).stop 2024-09-19T07:49:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.332+0000 7f56c4398640 1 -- 172.21.15.16:0/2116762874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56bc0d0070 msgr2=0x7f56bc0cd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.332+0000 7f56c4398640 1 --2- 172.21.15.16:0/2116762874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56bc0d0070 0x7f56bc0cd660 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f56b800d5f0 tx=0x7f56b800dab0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.332+0000 7f56c4398640 1 -- 172.21.15.16:0/2116762874 shutdown_connections 2024-09-19T07:49:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.332+0000 7f56c4398640 1 --2- 172.21.15.16:0/2116762874 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f56a006c2b0 0x7f56a006e770 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.332+0000 7f56c4398640 1 --2- 172.21.15.16:0/2116762874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56bc15f0d0 0x7f56bc0b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.332+0000 7f56c4398640 1 --2- 172.21.15.16:0/2116762874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56bc0d4130 0x7f56bc15b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.332+0000 7f56c4398640 1 --2- 172.21.15.16:0/2116762874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56bc0d0070 0x7f56bc0cd660 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.332+0000 7f56c4398640 1 -- 172.21.15.16:0/2116762874 >> 172.21.15.16:0/2116762874 conn(0x7f56bc09d760 msgr2=0x7f56bc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.332+0000 7f56c4398640 1 -- 172.21.15.16:0/2116762874 shutdown_connections 2024-09-19T07:49:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.332+0000 7f56c4398640 1 -- 172.21.15.16:0/2116762874 wait complete. 2024-09-19T07:49:37.343 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T07:49:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.447+0000 7f0f652e6640 1 Processor -- start 2024-09-19T07:49:37.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.447+0000 7f0f652e6640 1 -- start start 2024-09-19T07:49:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.448+0000 7f0f652e6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0f6006cd20 0x7f0f6006d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.448+0000 7f0f652e6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f60161eb0 0x7f0f601622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.448+0000 7f0f652e6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f60143a20 0x7f0f60143ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.448+0000 7f0f652e6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f0f60162e50 con 0x7f0f60143a20 2024-09-19T07:49:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.448+0000 7f0f652e6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f0f60142150 con 0x7f0f6006cd20 2024-09-19T07:49:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.448+0000 7f0f652e6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6006d640 con 0x7f0f60161eb0 2024-09-19T07:49:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.448+0000 7f0f5f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f60143a20 0x7f0f60143ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.448+0000 7f0f5f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f60143a20 0x7f0f60143ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53514/0 (socket says 172.21.15.16:53514) 2024-09-19T07:49:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.448+0000 7f0f5f7fe640 1 -- 172.21.15.16:0/1223005033 learned_addr learned my addr 172.21.15.16:0/1223005033 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.448+0000 7f0f5e7fc640 1 --2- 172.21.15.16:0/1223005033 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f60161eb0 0x7f0f601622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.448+0000 7f0f5effd640 1 --2- 172.21.15.16:0/1223005033 >> v2:172.21.15.146:3300/0 conn(0x7f0f6006cd20 0x7f0f6006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.448+0000 7f0f5f7fe640 1 -- 172.21.15.16:0/1223005033 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f60161eb0 msgr2=0x7f0f601622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.448+0000 7f0f5f7fe640 1 --2- 172.21.15.16:0/1223005033 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f60161eb0 0x7f0f601622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.448+0000 7f0f5f7fe640 1 -- 172.21.15.16:0/1223005033 >> v2:172.21.15.146:3300/0 conn(0x7f0f6006cd20 msgr2=0x7f0f6006d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.448+0000 7f0f5f7fe640 1 --2- 172.21.15.16:0/1223005033 >> v2:172.21.15.146:3300/0 conn(0x7f0f6006cd20 0x7f0f6006d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.448+0000 7f0f5f7fe640 1 -- 172.21.15.16:0/1223005033 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f60162b70 con 0x7f0f60143a20 2024-09-19T07:49:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.448+0000 7f0f5e7fc640 1 --2- 172.21.15.16:0/1223005033 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f60161eb0 0x7f0f601622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.448+0000 7f0f5effd640 1 --2- 172.21.15.16:0/1223005033 >> v2:172.21.15.146:3300/0 conn(0x7f0f6006cd20 0x7f0f6006d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:37.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.449+0000 7f0f5f7fe640 1 --2- 172.21.15.16:0/1223005033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f60143a20 0x7f0f60143ed0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0f60070ff0 tx=0x7f0f5002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a5d806d37f8520 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:37.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.449+0000 7f0f5dffb640 1 -- 172.21.15.16:0/1223005033 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f50036070 con 0x7f0f60143a20 2024-09-19T07:49:37.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.449+0000 7f0f5dffb640 1 -- 172.21.15.16:0/1223005033 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f50031600 con 0x7f0f60143a20 2024-09-19T07:49:37.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.449+0000 7f0f5dffb640 1 -- 172.21.15.16:0/1223005033 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f500319d0 con 0x7f0f60143a20 2024-09-19T07:49:37.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.450+0000 7f0f652e6640 1 -- 172.21.15.16:0/1223005033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f60143a20 msgr2=0x7f0f60143ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:37.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.450+0000 7f0f652e6640 1 --2- 172.21.15.16:0/1223005033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f60143a20 0x7f0f60143ed0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0f60070ff0 tx=0x7f0f5002a610 comp rx=0 tx=0).stop 2024-09-19T07:49:37.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.450+0000 7f0f652e6640 1 -- 172.21.15.16:0/1223005033 shutdown_connections 2024-09-19T07:49:37.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.450+0000 7f0f652e6640 1 --2- 172.21.15.16:0/1223005033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f60143a20 0x7f0f60143ed0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.450+0000 7f0f652e6640 1 --2- 172.21.15.16:0/1223005033 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f60161eb0 0x7f0f601622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.450+0000 7f0f652e6640 1 --2- 172.21.15.16:0/1223005033 >> v2:172.21.15.146:3300/0 conn(0x7f0f6006cd20 0x7f0f6006d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.450+0000 7f0f652e6640 1 -- 172.21.15.16:0/1223005033 >> 172.21.15.16:0/1223005033 conn(0x7f0f600b3ed0 msgr2=0x7f0f600b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:37.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.450+0000 7f0f652e6640 1 -- 172.21.15.16:0/1223005033 shutdown_connections 2024-09-19T07:49:37.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.450+0000 7f0f652e6640 1 -- 172.21.15.16:0/1223005033 wait complete. 2024-09-19T07:49:37.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.451+0000 7f0f652e6640 1 Processor -- start 2024-09-19T07:49:37.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.451+0000 7f0f652e6640 1 -- start start 2024-09-19T07:49:37.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.452+0000 7f0f652e6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f6006cd20 0x7f0f601f7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.452+0000 7f0f652e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f60143a20 0x7f0f600c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.452+0000 7f0f652e6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f60161eb0 0x7f0f600c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.452+0000 7f0f652e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f60162830 con 0x7f0f60143a20 2024-09-19T07:49:37.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.452+0000 7f0f652e6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0f60141de0 con 0x7f0f60161eb0 2024-09-19T07:49:37.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.452+0000 7f0f652e6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0f60141600 con 0x7f0f6006cd20 2024-09-19T07:49:37.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.452+0000 7f0f5e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f60143a20 0x7f0f600c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.452+0000 7f0f5e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f60143a20 0x7f0f600c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53528/0 (socket says 172.21.15.16:53528) 2024-09-19T07:49:37.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.452+0000 7f0f5e7fc640 1 -- 172.21.15.16:0/3102934515 learned_addr learned my addr 172.21.15.16:0/3102934515 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:37.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.452+0000 7f0f5effd640 1 --2- 172.21.15.16:0/3102934515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f6006cd20 0x7f0f601f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.452+0000 7f0f5f7fe640 1 --2- 172.21.15.16:0/3102934515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f60161eb0 0x7f0f600c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.452+0000 7f0f5e7fc640 1 -- 172.21.15.16:0/3102934515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f6006cd20 msgr2=0x7f0f601f7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:37.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.452+0000 7f0f5e7fc640 1 --2- 172.21.15.16:0/3102934515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f6006cd20 0x7f0f601f7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.452+0000 7f0f5e7fc640 1 -- 172.21.15.16:0/3102934515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f60161eb0 msgr2=0x7f0f600c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:37.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.452+0000 7f0f5e7fc640 1 --2- 172.21.15.16:0/3102934515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f60161eb0 0x7f0f600c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.452+0000 7f0f5e7fc640 1 -- 172.21.15.16:0/3102934515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f600b5960 con 0x7f0f60143a20 2024-09-19T07:49:37.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.452+0000 7f0f5e7fc640 1 --2- 172.21.15.16:0/3102934515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f60143a20 0x7f0f600c8840 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800e810 tx=0x7f0f4800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:37.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.453+0000 7f0f577fe640 1 -- 172.21.15.16:0/3102934515 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48017070 con 0x7f0f60143a20 2024-09-19T07:49:37.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.453+0000 7f0f577fe640 1 -- 172.21.15.16:0/3102934515 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f4800c8d0 con 0x7f0f60143a20 2024-09-19T07:49:37.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.453+0000 7f0f577fe640 1 -- 172.21.15.16:0/3102934515 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4800f040 con 0x7f0f60143a20 2024-09-19T07:49:37.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.453+0000 7f0f652e6640 1 -- 172.21.15.16:0/3102934515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f600b5c40 con 0x7f0f60143a20 2024-09-19T07:49:37.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.453+0000 7f0f652e6640 1 -- 172.21.15.16:0/3102934515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f600b6200 con 0x7f0f60143a20 2024-09-19T07:49:37.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.454+0000 7f0f652e6640 1 -- 172.21.15.16:0/3102934515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f6006d100 con 0x7f0f60143a20 2024-09-19T07:49:37.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.454+0000 7f0f577fe640 1 -- 172.21.15.16:0/3102934515 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0f4801a070 con 0x7f0f60143a20 2024-09-19T07:49:37.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.455+0000 7f0f577fe640 1 --2- 172.21.15.16:0/3102934515 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0f4006c2b0 0x7f0f4006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.455+0000 7f0f577fe640 1 -- 172.21.15.16:0/3102934515 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5249+0+0 (secure 0 0 0) 0x7f0f4808ae70 con 0x7f0f60143a20 2024-09-19T07:49:37.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.455+0000 7f0f5effd640 1 --2- 172.21.15.16:0/3102934515 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0f4006c2b0 0x7f0f4006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.455+0000 7f0f5effd640 1 --2- 172.21.15.16:0/3102934515 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0f4006c2b0 0x7f0f4006e770 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0f58006fd0 tx=0x7f0f58008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:37.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.459+0000 7f0f577fe640 1 -- 172.21.15.16:0/3102934515 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f0f480597a0 con 0x7f0f60143a20 2024-09-19T07:49:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.700+0000 7f0f652e6640 1 -- 172.21.15.16:0/3102934515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0f600690f0 con 0x7f0f60143a20 2024-09-19T07:49:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.700+0000 7f0f577fe640 1 -- 172.21.15.16:0/3102934515 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f0f480590a0 con 0x7f0f60143a20 2024-09-19T07:49:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:37.701 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:49:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-19T07:49:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-19T07:49:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etms80KDxAAqTrG88LTEJIj9TWJu0witQ==", 2024-09-19T07:49:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:49:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:49:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:49:37.701 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:49:37.702 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:49:37.702 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:37.702 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:37.702 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-19T07:49:37.702 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etm4T0KGxAAcfDOKrSdKzhJ4xvTFIyxcQ==", 2024-09-19T07:49:37.702 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:49:37.702 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:49:37.702 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:49:37.702 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:49:37.702 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:49:37.702 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:37.703 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:37.703 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-19T07:49:37.703 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNDBGFhAAtBioJJ1+NtIDLhd/Hg7aZA==", 2024-09-19T07:49:37.703 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:49:37.703 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:49:37.703 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:49:37.703 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:49:37.703 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:49:37.703 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:37.703 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:37.703 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-19T07:49:37.703 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmaHEiIhAACqqjCRMV+xlGh9WqYsj+MA==", 2024-09-19T07:49:37.703 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:49:37.703 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:49:37.704 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:49:37.704 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:49:37.704 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:49:37.704 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:37.704 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:37.704 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-19T07:49:37.704 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDT1etm09plCxAAGvJYF6JrKw49FtXb71ox3A==", 2024-09-19T07:49:37.704 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:49:37.704 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:49:37.704 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:49:37.704 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:49:37.704 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:37.704 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:37.705 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-19T07:49:37.705 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDW1etmGy8yHRAAlcDsHLnkjtS6sCCR8Olv2A==", 2024-09-19T07:49:37.705 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:49:37.705 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:49:37.705 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:49:37.705 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:49:37.705 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:37.705 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:37.705 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-19T07:49:37.705 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDZ1etmMhGTHxAATQaW2kgJhHKftutQQtUa6A==", 2024-09-19T07:49:37.705 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:49:37.706 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:49:37.706 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:49:37.706 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:49:37.706 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:37.706 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:37.706 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-19T07:49:37.706 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDc1etmD3sLIhAAIdY00buvb3bIFA5ZpgxxjA==", 2024-09-19T07:49:37.706 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:49:37.706 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:49:37.706 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:49:37.706 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:49:37.706 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:37.706 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:37.707 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-19T07:49:37.707 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDi1etmgutcLRAAHJEl5SfQ7N0T4YoYeBfrQg==", 2024-09-19T07:49:37.707 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:49:37.707 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:49:37.707 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:49:37.707 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:49:37.707 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:37.707 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:37.707 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-19T07:49:37.707 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDm1etmCV3xARAA+5yhQEmIpyNId3RL1Xsxhw==", 2024-09-19T07:49:37.707 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:49:37.707 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:49:37.708 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:49:37.708 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:49:37.708 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:37.708 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:37.708 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-19T07:49:37.708 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDp1etm9XC+AxAA+Lrs0ExfSKaixgOxfX8Nug==", 2024-09-19T07:49:37.708 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:49:37.708 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:49:37.708 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:49:37.708 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:49:37.708 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:37.709 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:37.709 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-19T07:49:37.709 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDs1etmhmvMBhAAhvW/udIE2gCd8IrJCSFuvw==", 2024-09-19T07:49:37.709 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:49:37.709 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:49:37.709 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:49:37.709 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:49:37.709 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.709 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:37.709 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:37.709 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-19T07:49:37.709 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA==", 2024-09-19T07:49:37.709 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:49:37.709 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:49:37.709 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-19T07:49:37.710 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T07:49:37.710 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:37.710 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:37.710 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-19T07:49:37.710 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmc05KLxAAPQl6wOHqFIf8LVxskiuLVg==", 2024-09-19T07:49:37.710 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:49:37.710 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:49:37.710 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:49:37.710 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:49:37.710 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:49:37.710 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:37.710 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:37.711 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-19T07:49:37.711 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmjFsCLBAANa7gCMvDw84Ovo7u6RRJ3Q==", 2024-09-19T07:49:37.711 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:49:37.711 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:49:37.711 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:49:37.711 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:49:37.711 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:49:37.711 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:37.711 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:37.711 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-19T07:49:37.711 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmdUbgNBAAfwstyOeeoZfetgVZu4z5xg==", 2024-09-19T07:49:37.711 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:49:37.711 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:49:37.712 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:49:37.712 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:49:37.712 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:49:37.712 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:37.712 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:37.712 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-19T07:49:37.712 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDQ1etm9/vTLhAA9PRfAOA8R4oXp3x0dU8l3g==", 2024-09-19T07:49:37.712 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:49:37.712 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:49:37.712 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:49:37.712 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:49:37.713 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:49:37.713 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.713 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:37.713 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:37.713 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-19T07:49:37.713 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNQ68CxAANkK+D3u/oTEz8Su4iuZrCg==", 2024-09-19T07:49:37.713 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:49:37.713 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:49:37.713 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:49:37.713 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:49:37.713 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.713 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:37.713 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:37.714 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-19T07:49:37.714 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmp+gSCBAAGXzmKfFNd6K2fWeqYhlHAg==", 2024-09-19T07:49:37.714 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:49:37.714 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:49:37.714 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:49:37.714 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:49:37.714 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.714 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:37.714 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T07:49:37.714 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:49:37.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.702+0000 7f0f652e6640 1 -- 172.21.15.16:0/3102934515 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0f4006c2b0 msgr2=0x7f0f4006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:37.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.702+0000 7f0f652e6640 1 --2- 172.21.15.16:0/3102934515 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0f4006c2b0 0x7f0f4006e770 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0f58006fd0 tx=0x7f0f58008040 comp rx=0 tx=0).stop 2024-09-19T07:49:37.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.702+0000 7f0f652e6640 1 -- 172.21.15.16:0/3102934515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f60143a20 msgr2=0x7f0f600c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:37.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.702+0000 7f0f652e6640 1 --2- 172.21.15.16:0/3102934515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f60143a20 0x7f0f600c8840 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800e810 tx=0x7f0f4800ecd0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.702+0000 7f0f652e6640 1 -- 172.21.15.16:0/3102934515 shutdown_connections 2024-09-19T07:49:37.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.702+0000 7f0f652e6640 1 --2- 172.21.15.16:0/3102934515 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0f4006c2b0 0x7f0f4006e770 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.702+0000 7f0f652e6640 1 --2- 172.21.15.16:0/3102934515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f60161eb0 0x7f0f600c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.702+0000 7f0f652e6640 1 --2- 172.21.15.16:0/3102934515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f60143a20 0x7f0f600c8840 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.702+0000 7f0f652e6640 1 --2- 172.21.15.16:0/3102934515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f6006cd20 0x7f0f601f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.702+0000 7f0f652e6640 1 -- 172.21.15.16:0/3102934515 >> 172.21.15.16:0/3102934515 conn(0x7f0f600b3ed0 msgr2=0x7f0f60160030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:37.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.702+0000 7f0f652e6640 1 -- 172.21.15.16:0/3102934515 shutdown_connections 2024-09-19T07:49:37.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.702+0000 7f0f652e6640 1 -- 172.21.15.16:0/3102934515 wait complete. 2024-09-19T07:49:37.717 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T07:49:37.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.819+0000 7fdda7f5f640 1 Processor -- start 2024-09-19T07:49:37.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.819+0000 7fdda7f5f640 1 -- start start 2024-09-19T07:49:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.820+0000 7fdda7f5f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdda00d4370 0x7fdda00d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.820+0000 7fdda7f5f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdda00d1880 con 0x7fdda015e2a0 2024-09-19T07:49:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.820+0000 7fdda7f5f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdda00d1b60 con 0x7fdda015e680 2024-09-19T07:49:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.820+0000 7fdda7f5f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fdda00cf2d0 con 0x7fdda00d4370 2024-09-19T07:49:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.820+0000 7fdda5cfb640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdda015e2a0 0x7fdda006d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59024/0 (socket says 172.21.15.16:59024) 2024-09-19T07:49:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.820+0000 7fdda5cfb640 1 -- 172.21.15.16:0/1307807731 learned_addr learned my addr 172.21.15.16:0/1307807731 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.820+0000 7fdda4cf9640 1 -- 172.21.15.16:0/1307807731 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2836690296 0 0) 0x7fdda00d1880 con 0x7fdda015e2a0 2024-09-19T07:49:37.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.820+0000 7fdda4cf9640 1 -- 172.21.15.16:0/1307807731 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd8c003520 con 0x7fdda015e2a0 2024-09-19T07:49:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.820+0000 7fdda64fc640 1 --2- 172.21.15.16:0/1307807731 >> v2:172.21.15.146:3300/0 conn(0x7fdda00d4370 0x7fdda00d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda4cf9640 1 -- 172.21.15.16:0/1307807731 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2724904426 0 0) 0x7fdd8c003520 con 0x7fdda015e2a0 2024-09-19T07:49:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda4cf9640 1 -- 172.21.15.16:0/1307807731 >> v2:172.21.15.146:3300/0 conn(0x7fdda00d4370 msgr2=0x7fdda00d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda4cf9640 1 --2- 172.21.15.16:0/1307807731 >> v2:172.21.15.146:3300/0 conn(0x7fdda00d4370 0x7fdda00d4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda4cf9640 1 -- 172.21.15.16:0/1307807731 >> v1:172.21.15.146:6789/0 conn(0x7fdda015e680 legacy=0x7fdda00cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda4cf9640 1 -- 172.21.15.16:0/1307807731 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda00cf5b0 con 0x7fdda015e2a0 2024-09-19T07:49:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda4cf9640 1 -- 172.21.15.16:0/1307807731 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fdd9c001670 con 0x7fdda015e2a0 2024-09-19T07:49:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda4cf9640 1 -- 172.21.15.16:0/1307807731 >> v1:172.21.15.16:6789/0 conn(0x7fdda015e2a0 legacy=0x7fdda006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda4cf9640 1 --2- 172.21.15.16:0/1307807731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd8c0039b0 0x7fdd8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda4cf9640 1 --2- 172.21.15.16:0/1307807731 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd8c004800 0x7fdd8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda4cf9640 1 --2- 172.21.15.16:0/1307807731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd8c0051c0 0x7fdd8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda4cf9640 1 -- 172.21.15.16:0/1307807731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdda00cf2d0 con 0x7fdd8c0039b0 2024-09-19T07:49:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda4cf9640 1 -- 172.21.15.16:0/1307807731 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c00c060 con 0x7fdd8c0051c0 2024-09-19T07:49:37.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda4cf9640 1 -- 172.21.15.16:0/1307807731 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdd8c00c260 con 0x7fdd8c004800 2024-09-19T07:49:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda5cfb640 1 --2- 172.21.15.16:0/1307807731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd8c0039b0 0x7fdd8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda64fc640 1 --2- 172.21.15.16:0/1307807731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd8c0051c0 0x7fdd8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda54fa640 1 --2- 172.21.15.16:0/1307807731 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd8c004800 0x7fdd8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda5cfb640 1 -- 172.21.15.16:0/1307807731 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd8c004800 msgr2=0x7fdd8c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda5cfb640 1 --2- 172.21.15.16:0/1307807731 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd8c004800 0x7fdd8c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda5cfb640 1 -- 172.21.15.16:0/1307807731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd8c0051c0 msgr2=0x7fdd8c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda5cfb640 1 --2- 172.21.15.16:0/1307807731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd8c0051c0 0x7fdd8c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda5cfb640 1 -- 172.21.15.16:0/1307807731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8c003520 con 0x7fdd8c0039b0 2024-09-19T07:49:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda64fc640 1 --2- 172.21.15.16:0/1307807731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd8c0051c0 0x7fdd8c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.821+0000 7fdda64fc640 1 -- 172.21.15.16:0/1307807731 reap_dead start 2024-09-19T07:49:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.822+0000 7fdda5cfb640 1 --2- 172.21.15.16:0/1307807731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd8c0039b0 0x7fdd8c003dd0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00c210 tx=0x7fdd8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec7b63317fa7a65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.822+0000 7fdda4cf9640 1 -- 172.21.15.16:0/1307807731 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c007d30 con 0x7fdd8c0039b0 2024-09-19T07:49:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.822+0000 7fdda4cf9640 1 -- 172.21.15.16:0/1307807731 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd9c0058d0 con 0x7fdd8c0039b0 2024-09-19T07:49:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.822+0000 7fdda4cf9640 1 -- 172.21.15.16:0/1307807731 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c005d20 con 0x7fdd8c0039b0 2024-09-19T07:49:37.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.822+0000 7fdda7f5f640 1 -- 172.21.15.16:0/1307807731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd8c0039b0 msgr2=0x7fdd8c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:37.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.822+0000 7fdda7f5f640 1 --2- 172.21.15.16:0/1307807731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd8c0039b0 0x7fdd8c003dd0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00c210 tx=0x7fdd8c005830 comp rx=0 tx=0).stop 2024-09-19T07:49:37.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.823+0000 7fdda7f5f640 1 -- 172.21.15.16:0/1307807731 shutdown_connections 2024-09-19T07:49:37.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.823+0000 7fdda7f5f640 1 --2- 172.21.15.16:0/1307807731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd8c0039b0 0x7fdd8c003dd0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.823+0000 7fdda7f5f640 1 -- 172.21.15.16:0/1307807731 >> 172.21.15.16:0/1307807731 conn(0x7fdda009d760 msgr2=0x7fdda008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:37.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.823+0000 7fdda7f5f640 1 -- 172.21.15.16:0/1307807731 shutdown_connections 2024-09-19T07:49:37.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.823+0000 7fdda7f5f640 1 -- 172.21.15.16:0/1307807731 wait complete. 2024-09-19T07:49:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.824+0000 7fdda7f5f640 1 Processor -- start 2024-09-19T07:49:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.824+0000 7fdda7f5f640 1 -- start start 2024-09-19T07:49:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.824+0000 7fdda7f5f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd8c0039b0 0x7fdda01598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdda7f5f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdda01f5740 0x7fdda00cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdda7f5f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdda00cdd00 0x7fdda00cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdda7f5f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdda00d49a0 con 0x7fdda00cdd00 2024-09-19T07:49:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdda7f5f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdda00d4820 con 0x7fdd8c0039b0 2024-09-19T07:49:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdda7f5f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdda00d4b20 con 0x7fdda01f5740 2024-09-19T07:49:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdda64fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdda00cdd00 0x7fdda00cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdda64fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdda00cdd00 0x7fdda00cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53546/0 (socket says 172.21.15.16:53546) 2024-09-19T07:49:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdda64fc640 1 -- 172.21.15.16:0/2948357254 learned_addr learned my addr 172.21.15.16:0/2948357254 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdda54fa640 1 --2- 172.21.15.16:0/2948357254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdda01f5740 0x7fdda00cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdda5cfb640 1 --2- 172.21.15.16:0/2948357254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd8c0039b0 0x7fdda01598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdda64fc640 1 -- 172.21.15.16:0/2948357254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdda01f5740 msgr2=0x7fdda00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:37.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdda64fc640 1 --2- 172.21.15.16:0/2948357254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdda01f5740 0x7fdda00cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdda64fc640 1 -- 172.21.15.16:0/2948357254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd8c0039b0 msgr2=0x7fdda01598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:37.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdda64fc640 1 --2- 172.21.15.16:0/2948357254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd8c0039b0 0x7fdda01598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:37.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdda64fc640 1 -- 172.21.15.16:0/2948357254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda01e7fe0 con 0x7fdda00cdd00 2024-09-19T07:49:37.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdda64fc640 1 --2- 172.21.15.16:0/2948357254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdda00cdd00 0x7fdda00cb4f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdda006eda0 tx=0x7fdd98010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:37.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdd967fc640 1 -- 172.21.15.16:0/2948357254 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98010da0 con 0x7fdda00cdd00 2024-09-19T07:49:37.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdd967fc640 1 -- 172.21.15.16:0/2948357254 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd98003990 con 0x7fdda00cdd00 2024-09-19T07:49:37.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdd967fc640 1 -- 172.21.15.16:0/2948357254 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98003d80 con 0x7fdda00cdd00 2024-09-19T07:49:37.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdda7f5f640 1 -- 172.21.15.16:0/2948357254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda00b81f0 con 0x7fdda00cdd00 2024-09-19T07:49:37.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.825+0000 7fdda7f5f640 1 -- 172.21.15.16:0/2948357254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda00b87b0 con 0x7fdda00cdd00 2024-09-19T07:49:37.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.826+0000 7fdda7f5f640 1 -- 172.21.15.16:0/2948357254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdda006d420 con 0x7fdda00cdd00 2024-09-19T07:49:37.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.827+0000 7fdd967fc640 1 -- 172.21.15.16:0/2948357254 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdd98005d10 con 0x7fdda00cdd00 2024-09-19T07:49:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.827+0000 7fdd967fc640 1 --2- 172.21.15.16:0/2948357254 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdd8406c2e0 0x7fdd8406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.827+0000 7fdd967fc640 1 -- 172.21.15.16:0/2948357254 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5249+0+0 (secure 0 0 0) 0x7fdd9808b880 con 0x7fdda00cdd00 2024-09-19T07:49:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.828+0000 7fdda5cfb640 1 --2- 172.21.15.16:0/2948357254 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdd8406c2e0 0x7fdd8406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.828+0000 7fdda5cfb640 1 --2- 172.21.15.16:0/2948357254 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdd8406c2e0 0x7fdd8406e7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c033000 tx=0x7fdd9c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:37.832+0000 7fdd967fc640 1 -- 172.21.15.16:0/2948357254 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fdd98025020 con 0x7fdda00cdd00 2024-09-19T07:49:38.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.008+0000 7fdda7f5f640 1 -- 172.21.15.16:0/2948357254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fdda006cf90 con 0x7fdda00cdd00 2024-09-19T07:49:38.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.009+0000 7fdd967fc640 1 -- 172.21.15.16:0/2948357254 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7fdd9805a110 con 0x7fdda00cdd00 2024-09-19T07:49:38.010 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:38.010 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-19T07:49:38.010 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:38.010 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-19T07:49:38.010 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-19T07:49:38.011 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:38.011 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-19T07:49:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.010+0000 7fdda7f5f640 1 -- 172.21.15.16:0/2948357254 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdd8406c2e0 msgr2=0x7fdd8406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.010+0000 7fdda7f5f640 1 --2- 172.21.15.16:0/2948357254 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdd8406c2e0 0x7fdd8406e7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c033000 tx=0x7fdd9c02e000 comp rx=0 tx=0).stop 2024-09-19T07:49:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.010+0000 7fdda7f5f640 1 -- 172.21.15.16:0/2948357254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdda00cdd00 msgr2=0x7fdda00cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.010+0000 7fdda7f5f640 1 --2- 172.21.15.16:0/2948357254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdda00cdd00 0x7fdda00cb4f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdda006eda0 tx=0x7fdd98010590 comp rx=0 tx=0).stop 2024-09-19T07:49:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.011+0000 7fdda7f5f640 1 -- 172.21.15.16:0/2948357254 shutdown_connections 2024-09-19T07:49:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.011+0000 7fdda7f5f640 1 --2- 172.21.15.16:0/2948357254 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdd8406c2e0 0x7fdd8406e7a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.011+0000 7fdda7f5f640 1 --2- 172.21.15.16:0/2948357254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdda00cdd00 0x7fdda00cb4f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.011+0000 7fdda7f5f640 1 --2- 172.21.15.16:0/2948357254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdda01f5740 0x7fdda00cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.011+0000 7fdda7f5f640 1 --2- 172.21.15.16:0/2948357254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd8c0039b0 0x7fdda01598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.011+0000 7fdda7f5f640 1 -- 172.21.15.16:0/2948357254 >> 172.21.15.16:0/2948357254 conn(0x7fdda009d760 msgr2=0x7fdda00d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:38.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.011+0000 7fdda7f5f640 1 -- 172.21.15.16:0/2948357254 shutdown_connections 2024-09-19T07:49:38.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.011+0000 7fdda7f5f640 1 -- 172.21.15.16:0/2948357254 wait complete. 2024-09-19T07:49:38.022 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T07:49:38.022 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T07:49:38.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.129+0000 7f3f4a511640 1 Processor -- start 2024-09-19T07:49:38.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.129+0000 7f3f4a511640 1 -- start start 2024-09-19T07:49:38.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.129+0000 7f3f4a511640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f4415e2a0 0x7f3f4406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.129+0000 7f3f4a511640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3f4406d960 0x7f3f440d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.129+0000 7f3f4a511640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f440d1880 con 0x7f3f4415e680 2024-09-19T07:49:38.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.129+0000 7f3f4a511640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f3f440d1b60 con 0x7f3f4406d960 2024-09-19T07:49:38.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.129+0000 7f3f4a511640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3f440cf2d0 con 0x7f3f4415e2a0 2024-09-19T07:49:38.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f437fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3f4415e680 0x7f3f440cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59036/0 (socket says 172.21.15.16:59036) 2024-09-19T07:49:38.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f437fe640 1 -- 172.21.15.16:0/1104010969 learned_addr learned my addr 172.21.15.16:0/1104010969 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:38.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f43fff640 1 --2- 172.21.15.16:0/1104010969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f4415e2a0 0x7f3f4406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f48aae640 1 --2- 172.21.15.16:0/1104010969 >> v2:172.21.15.146:3300/0 conn(0x7f3f4406d960 0x7f3f440d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f42ffd640 1 -- 172.21.15.16:0/1104010969 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4168688500 0 0) 0x7f3f440d1880 con 0x7f3f4415e680 2024-09-19T07:49:38.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f42ffd640 1 -- 172.21.15.16:0/1104010969 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f2c003520 con 0x7f3f4415e680 2024-09-19T07:49:38.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f42ffd640 1 -- 172.21.15.16:0/1104010969 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3984091256 0 0) 0x7f3f2c003520 con 0x7f3f4415e680 2024-09-19T07:49:38.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f42ffd640 1 -- 172.21.15.16:0/1104010969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f4415e2a0 msgr2=0x7f3f4406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:38.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f42ffd640 1 --2- 172.21.15.16:0/1104010969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f4415e2a0 0x7f3f4406d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f42ffd640 1 -- 172.21.15.16:0/1104010969 >> v2:172.21.15.146:3300/0 conn(0x7f3f4406d960 msgr2=0x7f3f440d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:38.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f42ffd640 1 --2- 172.21.15.16:0/1104010969 >> v2:172.21.15.146:3300/0 conn(0x7f3f4406d960 0x7f3f440d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f42ffd640 1 -- 172.21.15.16:0/1104010969 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f440cf5b0 con 0x7f3f4415e680 2024-09-19T07:49:38.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f42ffd640 1 -- 172.21.15.16:0/1104010969 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f3f38001670 con 0x7f3f4415e680 2024-09-19T07:49:38.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f42ffd640 1 -- 172.21.15.16:0/1104010969 >> v1:172.21.15.16:6789/0 conn(0x7f3f4415e680 legacy=0x7f3f440cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:38.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f42ffd640 1 --2- 172.21.15.16:0/1104010969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f4415e2a0 0x7f3f2c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f42ffd640 1 --2- 172.21.15.16:0/1104010969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f2c003f30 0x7f3f2c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f42ffd640 1 --2- 172.21.15.16:0/1104010969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f2c004d40 0x7f3f2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f42ffd640 1 -- 172.21.15.16:0/1104010969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3f440d1b60 con 0x7f3f2c003f30 2024-09-19T07:49:38.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f42ffd640 1 -- 172.21.15.16:0/1104010969 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3f440cf2d0 con 0x7f3f2c004d40 2024-09-19T07:49:38.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f42ffd640 1 -- 172.21.15.16:0/1104010969 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3f2c00b9c0 con 0x7f3f4415e2a0 2024-09-19T07:49:38.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f437fe640 1 --2- 172.21.15.16:0/1104010969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f2c003f30 0x7f3f2c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.130+0000 7f3f43fff640 1 --2- 172.21.15.16:0/1104010969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f4415e2a0 0x7f3f2c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.131+0000 7f3f48aae640 1 --2- 172.21.15.16:0/1104010969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f2c004d40 0x7f3f2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.131+0000 7f3f437fe640 1 -- 172.21.15.16:0/1104010969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f4415e2a0 msgr2=0x7f3f2c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:38.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.131+0000 7f3f437fe640 1 --2- 172.21.15.16:0/1104010969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f4415e2a0 0x7f3f2c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.131+0000 7f3f437fe640 1 -- 172.21.15.16:0/1104010969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f2c004d40 msgr2=0x7f3f2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:38.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.131+0000 7f3f437fe640 1 --2- 172.21.15.16:0/1104010969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f2c004d40 0x7f3f2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.131+0000 7f3f437fe640 1 -- 172.21.15.16:0/1104010969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2c003520 con 0x7f3f2c003f30 2024-09-19T07:49:38.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.131+0000 7f3f48aae640 1 --2- 172.21.15.16:0/1104010969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f2c004d40 0x7f3f2c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:38.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.131+0000 7f3f437fe640 1 --2- 172.21.15.16:0/1104010969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f2c003f30 0x7f3f2c004800 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3f440a22a0 tx=0x7f3f2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a5560366d276a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:38.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.132+0000 7f3f42ffd640 1 -- 172.21.15.16:0/1104010969 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38007d70 con 0x7f3f2c003f30 2024-09-19T07:49:38.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.132+0000 7f3f42ffd640 1 -- 172.21.15.16:0/1104010969 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f380058d0 con 0x7f3f2c003f30 2024-09-19T07:49:38.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.132+0000 7f3f42ffd640 1 -- 172.21.15.16:0/1104010969 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38005d00 con 0x7f3f2c003f30 2024-09-19T07:49:38.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.132+0000 7f3f4a511640 1 -- 172.21.15.16:0/1104010969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f2c003f30 msgr2=0x7f3f2c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:38.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.132+0000 7f3f4a511640 1 --2- 172.21.15.16:0/1104010969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f2c003f30 0x7f3f2c004800 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3f440a22a0 tx=0x7f3f2c005830 comp rx=0 tx=0).stop 2024-09-19T07:49:38.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.132+0000 7f3f48aae640 1 -- 172.21.15.16:0/1104010969 reap_dead start 2024-09-19T07:49:38.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.132+0000 7f3f4a511640 1 -- 172.21.15.16:0/1104010969 shutdown_connections 2024-09-19T07:49:38.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.132+0000 7f3f4a511640 1 -- 172.21.15.16:0/1104010969 >> 172.21.15.16:0/1104010969 conn(0x7f3f4409d760 msgr2=0x7f3f4408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:38.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.132+0000 7f3f4a511640 1 -- 172.21.15.16:0/1104010969 shutdown_connections 2024-09-19T07:49:38.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.132+0000 7f3f4a511640 1 -- 172.21.15.16:0/1104010969 wait complete. 2024-09-19T07:49:38.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.133+0000 7f3f4a511640 1 Processor -- start 2024-09-19T07:49:38.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.134+0000 7f3f4a511640 1 -- start start 2024-09-19T07:49:38.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.134+0000 7f3f4a511640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f2c003f30 0x7f3f441f4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.134+0000 7f3f4a511640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f440c9330 0x7f3f441e9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.134+0000 7f3f4a511640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f44161f40 0x7f3f441623f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.134+0000 7f3f4a511640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3f440d0360 con 0x7f3f440c9330 2024-09-19T07:49:38.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.134+0000 7f3f4a511640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3f440d01e0 con 0x7f3f2c003f30 2024-09-19T07:49:38.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.134+0000 7f3f4a511640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3f4406d420 con 0x7f3f44161f40 2024-09-19T07:49:38.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.134+0000 7f3f437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f440c9330 0x7f3f441e9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.134+0000 7f3f437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f440c9330 0x7f3f441e9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53566/0 (socket says 172.21.15.16:53566) 2024-09-19T07:49:38.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.134+0000 7f3f437fe640 1 -- 172.21.15.16:0/3971585110 learned_addr learned my addr 172.21.15.16:0/3971585110 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:38.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.135+0000 7f3f43fff640 1 --2- 172.21.15.16:0/3971585110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f2c003f30 0x7f3f441f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.135+0000 7f3f48aae640 1 --2- 172.21.15.16:0/3971585110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f44161f40 0x7f3f441623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.135+0000 7f3f437fe640 1 -- 172.21.15.16:0/3971585110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f44161f40 msgr2=0x7f3f441623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:38.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.135+0000 7f3f437fe640 1 --2- 172.21.15.16:0/3971585110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f44161f40 0x7f3f441623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.135+0000 7f3f437fe640 1 -- 172.21.15.16:0/3971585110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f2c003f30 msgr2=0x7f3f441f4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:38.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.135+0000 7f3f437fe640 1 --2- 172.21.15.16:0/3971585110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f2c003f30 0x7f3f441f4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.135+0000 7f3f437fe640 1 -- 172.21.15.16:0/3971585110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f441e9dd0 con 0x7f3f440c9330 2024-09-19T07:49:38.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.135+0000 7f3f48aae640 1 --2- 172.21.15.16:0/3971585110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f44161f40 0x7f3f441623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:38.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.135+0000 7f3f437fe640 1 --2- 172.21.15.16:0/3971585110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f440c9330 0x7f3f441e9890 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800c2d0 tx=0x7f3f38001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:38.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.135+0000 7f3f40ff9640 1 -- 172.21.15.16:0/3971585110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38040020 con 0x7f3f440c9330 2024-09-19T07:49:38.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.135+0000 7f3f40ff9640 1 -- 172.21.15.16:0/3971585110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f38007070 con 0x7f3f440c9330 2024-09-19T07:49:38.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.135+0000 7f3f4a511640 1 -- 172.21.15.16:0/3971585110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f440cb1f0 con 0x7f3f440c9330 2024-09-19T07:49:38.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.135+0000 7f3f40ff9640 1 -- 172.21.15.16:0/3971585110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38007260 con 0x7f3f440c9330 2024-09-19T07:49:38.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.135+0000 7f3f4a511640 1 -- 172.21.15.16:0/3971585110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f440c9b80 con 0x7f3f440c9330 2024-09-19T07:49:38.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.136+0000 7f3f4a511640 1 -- 172.21.15.16:0/3971585110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f440cfad0 con 0x7f3f440c9330 2024-09-19T07:49:38.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.137+0000 7f3f40ff9640 1 -- 172.21.15.16:0/3971585110 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3f38043020 con 0x7f3f440c9330 2024-09-19T07:49:38.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.137+0000 7f3f40ff9640 1 --2- 172.21.15.16:0/3971585110 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3f3006c2b0 0x7f3f3006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.137+0000 7f3f40ff9640 1 -- 172.21.15.16:0/3971585110 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5249+0+0 (secure 0 0 0) 0x7f3f380aade0 con 0x7f3f440c9330 2024-09-19T07:49:38.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.137+0000 7f3f43fff640 1 --2- 172.21.15.16:0/3971585110 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3f3006c2b0 0x7f3f3006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.138+0000 7f3f43fff640 1 --2- 172.21.15.16:0/3971585110 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3f3006c2b0 0x7f3f3006e770 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3f44162b60 tx=0x7f3f34009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:38.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.141+0000 7f3f40ff9640 1 -- 172.21.15.16:0/3971585110 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3f441e9dd0 con 0x7f3f440c9330 2024-09-19T07:49:38.191 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:49:38.193 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:49:38.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:38.317+0000 7f3f4a511640 1 -- 172.21.15.16:0/3971585110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3f4406cf90 con 0x7f3f440c9330 2024-09-19T07:49:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.018+0000 7f3f40ff9640 1 -- 172.21.15.16:0/3971585110 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7f3f38079700 con 0x7f3f440c9330 2024-09-19T07:49:39.020 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-19T07:49:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.021+0000 7f3f4a511640 1 -- 172.21.15.16:0/3971585110 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3f3006c2b0 msgr2=0x7f3f3006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.021+0000 7f3f4a511640 1 --2- 172.21.15.16:0/3971585110 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3f3006c2b0 0x7f3f3006e770 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3f44162b60 tx=0x7f3f34009190 comp rx=0 tx=0).stop 2024-09-19T07:49:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.021+0000 7f3f4a511640 1 -- 172.21.15.16:0/3971585110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f440c9330 msgr2=0x7f3f441e9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.021+0000 7f3f4a511640 1 --2- 172.21.15.16:0/3971585110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f440c9330 0x7f3f441e9890 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800c2d0 tx=0x7f3f38001a40 comp rx=0 tx=0).stop 2024-09-19T07:49:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.022+0000 7f3f4a511640 1 -- 172.21.15.16:0/3971585110 shutdown_connections 2024-09-19T07:49:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.022+0000 7f3f4a511640 1 --2- 172.21.15.16:0/3971585110 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3f3006c2b0 0x7f3f3006e770 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.022+0000 7f3f4a511640 1 --2- 172.21.15.16:0/3971585110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f44161f40 0x7f3f441623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.022+0000 7f3f4a511640 1 --2- 172.21.15.16:0/3971585110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f440c9330 0x7f3f441e9890 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.022+0000 7f3f4a511640 1 --2- 172.21.15.16:0/3971585110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f2c003f30 0x7f3f441f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.022+0000 7f3f4a511640 1 -- 172.21.15.16:0/3971585110 >> 172.21.15.16:0/3971585110 conn(0x7f3f4409d760 msgr2=0x7f3f4408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.022+0000 7f3f4a511640 1 -- 172.21.15.16:0/3971585110 shutdown_connections 2024-09-19T07:49:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.022+0000 7f3f4a511640 1 -- 172.21.15.16:0/3971585110 wait complete. 2024-09-19T07:49:39.041 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T07:49:39.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.156+0000 7f8836b86640 1 Processor -- start 2024-09-19T07:49:39.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.156+0000 7f8836b86640 1 -- start start 2024-09-19T07:49:39.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.157+0000 7f8836b86640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f883015e0a0 0x7f8830162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:39.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.157+0000 7f8836b86640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88300b5280 0x7f88300d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:39.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.157+0000 7f8836b86640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f883015f250 con 0x7f88300d43e0 2024-09-19T07:49:39.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.157+0000 7f8836b86640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f88300d4d60 con 0x7f883015e0a0 2024-09-19T07:49:39.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.157+0000 7f8836b86640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f883015f530 con 0x7f88300b5280 2024-09-19T07:49:39.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.157+0000 7f8834922640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f883015e0a0 0x7f8830162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:39.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.157+0000 7f8834922640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f883015e0a0 0x7f8830162a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53582/0 (socket says 172.21.15.16:53582) 2024-09-19T07:49:39.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.157+0000 7f8834922640 1 -- 172.21.15.16:0/3626745834 learned_addr learned my addr 172.21.15.16:0/3626745834 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:39.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.158+0000 7f8835123640 1 --2- 172.21.15.16:0/3626745834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88300b5280 0x7f88300d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:39.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.158+0000 7f8834922640 1 -- 172.21.15.16:0/3626745834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88300b5280 msgr2=0x7f88300d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:39.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.158+0000 7f8834922640 1 --2- 172.21.15.16:0/3626745834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88300b5280 0x7f88300d4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:39.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.158+0000 7f8834922640 1 -- 172.21.15.16:0/3626745834 >> v1:172.21.15.146:6789/0 conn(0x7f88300d43e0 legacy=0x7f88300b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:39.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.158+0000 7f8834922640 1 -- 172.21.15.16:0/3626745834 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883015f810 con 0x7f883015e0a0 2024-09-19T07:49:39.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.158+0000 7f8835123640 1 --2- 172.21.15.16:0/3626745834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88300b5280 0x7f88300d4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:39.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.158+0000 7f8834922640 1 --2- 172.21.15.16:0/3626745834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883015e0a0 0x7f8830162a70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8828009e50 tx=0x7f882802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49bcc64eea38aa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:39.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.159+0000 7f882f7fe640 1 -- 172.21.15.16:0/3626745834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828036070 con 0x7f883015e0a0 2024-09-19T07:49:39.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.159+0000 7f882f7fe640 1 -- 172.21.15.16:0/3626745834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8828031600 con 0x7f883015e0a0 2024-09-19T07:49:39.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.159+0000 7f882f7fe640 1 -- 172.21.15.16:0/3626745834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88280319d0 con 0x7f883015e0a0 2024-09-19T07:49:39.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.159+0000 7f8836b86640 1 -- 172.21.15.16:0/3626745834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883015e0a0 msgr2=0x7f8830162a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:39.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.159+0000 7f8836b86640 1 --2- 172.21.15.16:0/3626745834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883015e0a0 0x7f8830162a70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8828009e50 tx=0x7f882802a710 comp rx=0 tx=0).stop 2024-09-19T07:49:39.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.159+0000 7f8836b86640 1 -- 172.21.15.16:0/3626745834 shutdown_connections 2024-09-19T07:49:39.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.159+0000 7f8836b86640 1 --2- 172.21.15.16:0/3626745834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88300b5280 0x7f88300d4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:39.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.159+0000 7f8836b86640 1 --2- 172.21.15.16:0/3626745834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883015e0a0 0x7f8830162a70 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:39.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.159+0000 7f8836b86640 1 -- 172.21.15.16:0/3626745834 >> 172.21.15.16:0/3626745834 conn(0x7f883009d770 msgr2=0x7f883008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:39.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.159+0000 7f8836b86640 1 -- 172.21.15.16:0/3626745834 shutdown_connections 2024-09-19T07:49:39.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.159+0000 7f8836b86640 1 -- 172.21.15.16:0/3626745834 wait complete. 2024-09-19T07:49:39.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.161+0000 7f8836b86640 1 Processor -- start 2024-09-19T07:49:39.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.161+0000 7f8836b86640 1 -- start start 2024-09-19T07:49:39.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.161+0000 7f8836b86640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88300b5280 0x7f8830157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:39.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.161+0000 7f8836b86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88300d43e0 0x7f88300cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:39.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.161+0000 7f8836b86640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f883015e0a0 0x7f88300d0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:39.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.162+0000 7f8836b86640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f883015e600 con 0x7f88300d43e0 2024-09-19T07:49:39.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.162+0000 7f8836b86640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f883015e480 con 0x7f883015e0a0 2024-09-19T07:49:39.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.162+0000 7f8836b86640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f883015e780 con 0x7f88300b5280 2024-09-19T07:49:39.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.162+0000 7f882ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88300d43e0 0x7f88300cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:39.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.162+0000 7f882ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88300d43e0 0x7f88300cd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53596/0 (socket says 172.21.15.16:53596) 2024-09-19T07:49:39.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.162+0000 7f882ffff640 1 -- 172.21.15.16:0/1853302593 learned_addr learned my addr 172.21.15.16:0/1853302593 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:39.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.162+0000 7f8834922640 1 --2- 172.21.15.16:0/1853302593 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88300b5280 0x7f8830157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:39.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.162+0000 7f8835123640 1 --2- 172.21.15.16:0/1853302593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f883015e0a0 0x7f88300d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:39.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.162+0000 7f882ffff640 1 -- 172.21.15.16:0/1853302593 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88300b5280 msgr2=0x7f8830157c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:39.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.162+0000 7f882ffff640 1 --2- 172.21.15.16:0/1853302593 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88300b5280 0x7f8830157c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:39.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.162+0000 7f882ffff640 1 -- 172.21.15.16:0/1853302593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f883015e0a0 msgr2=0x7f88300d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:39.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.162+0000 7f882ffff640 1 --2- 172.21.15.16:0/1853302593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f883015e0a0 0x7f88300d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:39.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.162+0000 7f882ffff640 1 -- 172.21.15.16:0/1853302593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88300cdda0 con 0x7f88300d43e0 2024-09-19T07:49:39.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.162+0000 7f882ffff640 1 --2- 172.21.15.16:0/1853302593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88300d43e0 0x7f88300cd0e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f882400c400 tx=0x7f882400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:39.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.162+0000 7f882d7fa640 1 -- 172.21.15.16:0/1853302593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88240052d0 con 0x7f88300d43e0 2024-09-19T07:49:39.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.162+0000 7f882d7fa640 1 -- 172.21.15.16:0/1853302593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88240054b0 con 0x7f88300d43e0 2024-09-19T07:49:39.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.162+0000 7f882d7fa640 1 -- 172.21.15.16:0/1853302593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824006a40 con 0x7f88300d43e0 2024-09-19T07:49:39.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.162+0000 7f8836b86640 1 -- 172.21.15.16:0/1853302593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88300cdac0 con 0x7f88300d43e0 2024-09-19T07:49:39.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.162+0000 7f8836b86640 1 -- 172.21.15.16:0/1853302593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88301f7a90 con 0x7f88300d43e0 2024-09-19T07:49:39.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.163+0000 7f8836b86640 1 -- 172.21.15.16:0/1853302593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88300d4820 con 0x7f88300d43e0 2024-09-19T07:49:39.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.164+0000 7f882d7fa640 1 -- 172.21.15.16:0/1853302593 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8824006be0 con 0x7f88300d43e0 2024-09-19T07:49:39.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.165+0000 7f882d7fa640 1 --2- 172.21.15.16:0/1853302593 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f881c06c2b0 0x7f881c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:39.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.165+0000 7f882d7fa640 1 -- 172.21.15.16:0/1853302593 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5619+0+0 (secure 0 0 0) 0x7f882408a7b0 con 0x7f88300d43e0 2024-09-19T07:49:39.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.165+0000 7f8834922640 1 --2- 172.21.15.16:0/1853302593 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f881c06c2b0 0x7f881c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:39.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.165+0000 7f8834922640 1 --2- 172.21.15.16:0/1853302593 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f881c06c2b0 0x7f881c06e770 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8828033000 tx=0x7f8828009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:39.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.169+0000 7f882d7fa640 1 -- 172.21.15.16:0/1853302593 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8824058920 con 0x7f88300d43e0 2024-09-19T07:49:39.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:39.345+0000 7f8836b86640 1 -- 172.21.15.16:0/1853302593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f88300ba3d0 con 0x7f88300d43e0 2024-09-19T07:49:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.020+0000 7f882d7fa640 1 -- 172.21.15.16:0/1853302593 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) ==== 170+0+0 (secure 0 0 0) 0x7f882405cc20 con 0x7f88300d43e0 2024-09-19T07:49:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-19T07:49:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.024+0000 7f8836b86640 1 -- 172.21.15.16:0/1853302593 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f881c06c2b0 msgr2=0x7f881c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.024+0000 7f8836b86640 1 --2- 172.21.15.16:0/1853302593 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f881c06c2b0 0x7f881c06e770 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8828033000 tx=0x7f8828009000 comp rx=0 tx=0).stop 2024-09-19T07:49:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.024+0000 7f8836b86640 1 -- 172.21.15.16:0/1853302593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88300d43e0 msgr2=0x7f88300cd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.024+0000 7f8836b86640 1 --2- 172.21.15.16:0/1853302593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88300d43e0 0x7f88300cd0e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f882400c400 tx=0x7f882400c8c0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.024+0000 7f8836b86640 1 -- 172.21.15.16:0/1853302593 shutdown_connections 2024-09-19T07:49:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.024+0000 7f8836b86640 1 --2- 172.21.15.16:0/1853302593 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f881c06c2b0 0x7f881c06e770 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.024+0000 7f8836b86640 1 --2- 172.21.15.16:0/1853302593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f883015e0a0 0x7f88300d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.024+0000 7f8836b86640 1 --2- 172.21.15.16:0/1853302593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88300d43e0 0x7f88300cd0e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.024+0000 7f8836b86640 1 --2- 172.21.15.16:0/1853302593 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88300b5280 0x7f8830157c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.024+0000 7f8836b86640 1 -- 172.21.15.16:0/1853302593 >> 172.21.15.16:0/1853302593 conn(0x7f883009d770 msgr2=0x7f883006e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.026+0000 7f8836b86640 1 -- 172.21.15.16:0/1853302593 shutdown_connections 2024-09-19T07:49:40.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.026+0000 7f8836b86640 1 -- 172.21.15.16:0/1853302593 wait complete. 2024-09-19T07:49:40.062 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T07:49:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.172+0000 7ff5a1f6d640 1 Processor -- start 2024-09-19T07:49:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.172+0000 7ff5a1f6d640 1 -- start start 2024-09-19T07:49:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.172+0000 7ff5a1f6d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff59c15f0d0 0x7ff59c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.173+0000 7ff5a1f6d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff59c0d0070 0x7ff59c0d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.173+0000 7ff5a1f6d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff59c0d1940 0x7ff59c0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.173+0000 7ff5a1f6d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff59c0cf2b0 con 0x7ff59c0d0070 2024-09-19T07:49:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.173+0000 7ff5a1f6d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff59c0cf590 con 0x7ff59c0d1940 2024-09-19T07:49:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.173+0000 7ff5a1f6d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff59c0cf870 con 0x7ff59c15f0d0 2024-09-19T07:49:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.173+0000 7ff59affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff59c0d0070 0x7ff59c0d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.173+0000 7ff59affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff59c0d0070 0x7ff59c0d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53600/0 (socket says 172.21.15.16:53600) 2024-09-19T07:49:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.173+0000 7ff59affd640 1 -- 172.21.15.16:0/2693299483 learned_addr learned my addr 172.21.15.16:0/2693299483 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.173+0000 7ff59b7fe640 1 --2- 172.21.15.16:0/2693299483 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff59c15f0d0 0x7ff59c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.173+0000 7ff59bfff640 1 --2- 172.21.15.16:0/2693299483 >> v2:172.21.15.146:3300/0 conn(0x7ff59c0d1940 0x7ff59c0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.173+0000 7ff59affd640 1 -- 172.21.15.16:0/2693299483 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff59c15f0d0 msgr2=0x7ff59c15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.173+0000 7ff59affd640 1 --2- 172.21.15.16:0/2693299483 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff59c15f0d0 0x7ff59c15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.173+0000 7ff59affd640 1 -- 172.21.15.16:0/2693299483 >> v2:172.21.15.146:3300/0 conn(0x7ff59c0d1940 msgr2=0x7ff59c0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.173+0000 7ff59affd640 1 --2- 172.21.15.16:0/2693299483 >> v2:172.21.15.146:3300/0 conn(0x7ff59c0d1940 0x7ff59c0d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.173+0000 7ff59affd640 1 -- 172.21.15.16:0/2693299483 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59c0cfb50 con 0x7ff59c0d0070 2024-09-19T07:49:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.174+0000 7ff59affd640 1 --2- 172.21.15.16:0/2693299483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c0d0070 0x7ff59c0d1400 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff59c161e80 tx=0x7ff58402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c9c712d9355c2a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.174+0000 7ff59a7fc640 1 -- 172.21.15.16:0/2693299483 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584037070 con 0x7ff59c0d0070 2024-09-19T07:49:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.174+0000 7ff59a7fc640 1 -- 172.21.15.16:0/2693299483 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff584032440 con 0x7ff59c0d0070 2024-09-19T07:49:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.174+0000 7ff59a7fc640 1 -- 172.21.15.16:0/2693299483 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584032810 con 0x7ff59c0d0070 2024-09-19T07:49:40.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.175+0000 7ff5a1f6d640 1 -- 172.21.15.16:0/2693299483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c0d0070 msgr2=0x7ff59c0d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.175+0000 7ff5a1f6d640 1 --2- 172.21.15.16:0/2693299483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c0d0070 0x7ff59c0d1400 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff59c161e80 tx=0x7ff58402a140 comp rx=0 tx=0).stop 2024-09-19T07:49:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.175+0000 7ff5a1f6d640 1 -- 172.21.15.16:0/2693299483 shutdown_connections 2024-09-19T07:49:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.175+0000 7ff5a1f6d640 1 --2- 172.21.15.16:0/2693299483 >> v2:172.21.15.146:3300/0 conn(0x7ff59c0d1940 0x7ff59c0d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.175+0000 7ff5a1f6d640 1 --2- 172.21.15.16:0/2693299483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c0d0070 0x7ff59c0d1400 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.175+0000 7ff5a1f6d640 1 --2- 172.21.15.16:0/2693299483 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff59c15f0d0 0x7ff59c15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.175+0000 7ff5a1f6d640 1 -- 172.21.15.16:0/2693299483 >> 172.21.15.16:0/2693299483 conn(0x7ff59c09d760 msgr2=0x7ff59c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.175+0000 7ff5a1f6d640 1 -- 172.21.15.16:0/2693299483 shutdown_connections 2024-09-19T07:49:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.175+0000 7ff5a1f6d640 1 -- 172.21.15.16:0/2693299483 wait complete. 2024-09-19T07:49:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.176+0000 7ff5a1f6d640 1 Processor -- start 2024-09-19T07:49:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.176+0000 7ff5a1f6d640 1 -- start start 2024-09-19T07:49:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.177+0000 7ff5a1f6d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff59c0d0070 0x7ff59c0c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.177+0000 7ff5a1f6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c0d1940 0x7ff59c0cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.177+0000 7ff5a1f6d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff59c15f0d0 0x7ff59c0b93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.177+0000 7ff5a1f6d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff59c0d47f0 con 0x7ff59c0d1940 2024-09-19T07:49:40.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.177+0000 7ff5a1f6d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff59c0d4670 con 0x7ff59c0d0070 2024-09-19T07:49:40.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.177+0000 7ff5a1f6d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff59c0d4970 con 0x7ff59c15f0d0 2024-09-19T07:49:40.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.177+0000 7ff59affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c0d1940 0x7ff59c0cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.177+0000 7ff59affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c0d1940 0x7ff59c0cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53604/0 (socket says 172.21.15.16:53604) 2024-09-19T07:49:40.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.177+0000 7ff59affd640 1 -- 172.21.15.16:0/2761345258 learned_addr learned my addr 172.21.15.16:0/2761345258 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:40.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.177+0000 7ff59b7fe640 1 --2- 172.21.15.16:0/2761345258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff59c0d0070 0x7ff59c0c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.177+0000 7ff59bfff640 1 --2- 172.21.15.16:0/2761345258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff59c15f0d0 0x7ff59c0b93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.177+0000 7ff59affd640 1 -- 172.21.15.16:0/2761345258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff59c15f0d0 msgr2=0x7ff59c0b93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:40.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.177+0000 7ff59affd640 1 --2- 172.21.15.16:0/2761345258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff59c15f0d0 0x7ff59c0b93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.177+0000 7ff59affd640 1 -- 172.21.15.16:0/2761345258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff59c0d0070 msgr2=0x7ff59c0c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:40.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.177+0000 7ff59affd640 1 --2- 172.21.15.16:0/2761345258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff59c0d0070 0x7ff59c0c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.177+0000 7ff59affd640 1 -- 172.21.15.16:0/2761345258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59c0b49b0 con 0x7ff59c0d1940 2024-09-19T07:49:40.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.177+0000 7ff59bfff640 1 --2- 172.21.15.16:0/2761345258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff59c15f0d0 0x7ff59c0b93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:40.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.177+0000 7ff59affd640 1 --2- 172.21.15.16:0/2761345258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c0d1940 0x7ff59c0cab40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff584000c00 tx=0x7ff5840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:40.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.178+0000 7ff593fff640 1 -- 172.21.15.16:0/2761345258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584041020 con 0x7ff59c0d1940 2024-09-19T07:49:40.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.178+0000 7ff593fff640 1 -- 172.21.15.16:0/2761345258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff58403d620 con 0x7ff59c0d1940 2024-09-19T07:49:40.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.178+0000 7ff5a1f6d640 1 -- 172.21.15.16:0/2761345258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff59c0b4c90 con 0x7ff59c0d1940 2024-09-19T07:49:40.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.178+0000 7ff593fff640 1 -- 172.21.15.16:0/2761345258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584037040 con 0x7ff59c0d1940 2024-09-19T07:49:40.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.178+0000 7ff5a1f6d640 1 -- 172.21.15.16:0/2761345258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff59c0ca620 con 0x7ff59c0d1940 2024-09-19T07:49:40.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.179+0000 7ff5a1f6d640 1 -- 172.21.15.16:0/2761345258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff59c0d1400 con 0x7ff59c0d1940 2024-09-19T07:49:40.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.179+0000 7ff593fff640 1 -- 172.21.15.16:0/2761345258 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff584044020 con 0x7ff59c0d1940 2024-09-19T07:49:40.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.180+0000 7ff593fff640 1 --2- 172.21.15.16:0/2761345258 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff57c06c2b0 0x7ff57c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.180+0000 7ff593fff640 1 -- 172.21.15.16:0/2761345258 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6001+0+0 (secure 0 0 0) 0x7ff5840abd00 con 0x7ff59c0d1940 2024-09-19T07:49:40.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.180+0000 7ff59b7fe640 1 --2- 172.21.15.16:0/2761345258 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff57c06c2b0 0x7ff57c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.181+0000 7ff59b7fe640 1 --2- 172.21.15.16:0/2761345258 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff57c06c2b0 0x7ff57c06e770 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0073f0 tx=0x7ff58c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:40.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.184+0000 7ff593fff640 1 -- 172.21.15.16:0/2761345258 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff58404b030 con 0x7ff59c0d1940 2024-09-19T07:49:40.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:40.401+0000 7ff5a1f6d640 1 -- 172.21.15.16:0/2761345258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff59c1618b0 con 0x7ff59c0d1940 2024-09-19T07:49:41.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.038+0000 7ff593fff640 1 -- 172.21.15.16:0/2761345258 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v38) ==== 174+0+0 (secure 0 0 0) 0x7ff58407a2a0 con 0x7ff59c0d1940 2024-09-19T07:49:41.040 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 11 and data pool 12 2024-09-19T07:49:41.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.050+0000 7ff5a1f6d640 1 -- 172.21.15.16:0/2761345258 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff57c06c2b0 msgr2=0x7ff57c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.050+0000 7ff5a1f6d640 1 --2- 172.21.15.16:0/2761345258 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff57c06c2b0 0x7ff57c06e770 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0073f0 tx=0x7ff58c00b040 comp rx=0 tx=0).stop 2024-09-19T07:49:41.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.050+0000 7ff5a1f6d640 1 -- 172.21.15.16:0/2761345258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c0d1940 msgr2=0x7ff59c0cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.050+0000 7ff5a1f6d640 1 --2- 172.21.15.16:0/2761345258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c0d1940 0x7ff59c0cab40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff584000c00 tx=0x7ff5840046c0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.050+0000 7ff5a1f6d640 1 -- 172.21.15.16:0/2761345258 shutdown_connections 2024-09-19T07:49:41.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.050+0000 7ff5a1f6d640 1 --2- 172.21.15.16:0/2761345258 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff57c06c2b0 0x7ff57c06e770 secure :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0073f0 tx=0x7ff58c00b040 comp rx=0 tx=0).stop 2024-09-19T07:49:41.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.050+0000 7ff5a1f6d640 1 --2- 172.21.15.16:0/2761345258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff59c15f0d0 0x7ff59c0b93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.050+0000 7ff5a1f6d640 1 --2- 172.21.15.16:0/2761345258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c0d1940 0x7ff59c0cab40 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.050+0000 7ff5a1f6d640 1 --2- 172.21.15.16:0/2761345258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff59c0d0070 0x7ff59c0c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.051+0000 7ff5a1f6d640 1 -- 172.21.15.16:0/2761345258 >> 172.21.15.16:0/2761345258 conn(0x7ff59c09d760 msgr2=0x7ff59c0d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:41.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.051+0000 7ff5a1f6d640 1 -- 172.21.15.16:0/2761345258 shutdown_connections 2024-09-19T07:49:41.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.052+0000 7ff5a1f6d640 1 -- 172.21.15.16:0/2761345258 wait complete. 2024-09-19T07:49:41.079 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:49:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.184+0000 7f3109f31640 1 Processor -- start 2024-09-19T07:49:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.185+0000 7f3109f31640 1 -- start start 2024-09-19T07:49:41.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.185+0000 7f3109f31640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31040d4840 0x7f310415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.185+0000 7f3109f31640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31040d1b10 con 0x7f31040d0070 2024-09-19T07:49:41.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.185+0000 7f3109f31640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31040cf2b0 con 0x7f310415f0d0 2024-09-19T07:49:41.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.185+0000 7f3109f31640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f31040cf590 con 0x7f31040d4840 2024-09-19T07:49:41.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f3102ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f31040d0070 0x7f31040d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59050/0 (socket says 172.21.15.16:59050) 2024-09-19T07:49:41.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f3102ffd640 1 -- 172.21.15.16:0/3786392367 learned_addr learned my addr 172.21.15.16:0/3786392367 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:41.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f3103fff640 1 --2- 172.21.15.16:0/3786392367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31040d4840 0x7f310415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f31027fc640 1 -- 172.21.15.16:0/3786392367 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 655059312 0 0) 0x7f31040d1b10 con 0x7f31040d0070 2024-09-19T07:49:41.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f31027fc640 1 -- 172.21.15.16:0/3786392367 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30ec003520 con 0x7f31040d0070 2024-09-19T07:49:41.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f31027fc640 1 -- 172.21.15.16:0/3786392367 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3425411934 0 0) 0x7f30ec003520 con 0x7f31040d0070 2024-09-19T07:49:41.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f31027fc640 1 -- 172.21.15.16:0/3786392367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31040d4840 msgr2=0x7f310415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:41.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f31027fc640 1 --2- 172.21.15.16:0/3786392367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31040d4840 0x7f310415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f31027fc640 1 -- 172.21.15.16:0/3786392367 >> v1:172.21.15.146:6789/0 conn(0x7f310415f0d0 legacy=0x7f31040d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f31027fc640 1 -- 172.21.15.16:0/3786392367 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31040cf870 con 0x7f31040d0070 2024-09-19T07:49:41.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f31027fc640 1 -- 172.21.15.16:0/3786392367 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f30f8001670 con 0x7f31040d0070 2024-09-19T07:49:41.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f31027fc640 1 -- 172.21.15.16:0/3786392367 >> v1:172.21.15.16:6789/0 conn(0x7f31040d0070 legacy=0x7f31040d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f31027fc640 1 --2- 172.21.15.16:0/3786392367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31040d4840 0x7f30ec0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f31027fc640 1 --2- 172.21.15.16:0/3786392367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec003f20 0x7f30ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f31027fc640 1 --2- 172.21.15.16:0/3786392367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30ec004d40 0x7f30ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f31027fc640 1 -- 172.21.15.16:0/3786392367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f31040cf590 con 0x7f30ec003f20 2024-09-19T07:49:41.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f31027fc640 1 -- 172.21.15.16:0/3786392367 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec00bad0 con 0x7f30ec004d40 2024-09-19T07:49:41.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f31027fc640 1 -- 172.21.15.16:0/3786392367 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f30ec00bcd0 con 0x7f31040d4840 2024-09-19T07:49:41.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f3102ffd640 1 --2- 172.21.15.16:0/3786392367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec003f20 0x7f30ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f31037fe640 1 --2- 172.21.15.16:0/3786392367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31040d4840 0x7f30ec0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.186+0000 7f3103fff640 1 --2- 172.21.15.16:0/3786392367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30ec004d40 0x7f30ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.187+0000 7f3102ffd640 1 -- 172.21.15.16:0/3786392367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31040d4840 msgr2=0x7f30ec0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:41.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.187+0000 7f3102ffd640 1 --2- 172.21.15.16:0/3786392367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31040d4840 0x7f30ec0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.187+0000 7f3102ffd640 1 -- 172.21.15.16:0/3786392367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30ec004d40 msgr2=0x7f30ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:41.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.187+0000 7f3102ffd640 1 --2- 172.21.15.16:0/3786392367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30ec004d40 0x7f30ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.187+0000 7f3102ffd640 1 -- 172.21.15.16:0/3786392367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ec003520 con 0x7f30ec003f20 2024-09-19T07:49:41.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.187+0000 7f3102ffd640 1 --2- 172.21.15.16:0/3786392367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec003f20 0x7f30ec004800 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f30f800c290 tx=0x7f30ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27618fd9ced08d10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:41.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.188+0000 7f31027fc640 1 -- 172.21.15.16:0/3786392367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8007d50 con 0x7f30ec003f20 2024-09-19T07:49:41.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.188+0000 7f31027fc640 1 -- 172.21.15.16:0/3786392367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30f8005af0 con 0x7f30ec003f20 2024-09-19T07:49:41.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.188+0000 7f31027fc640 1 -- 172.21.15.16:0/3786392367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8007070 con 0x7f30ec003f20 2024-09-19T07:49:41.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.188+0000 7f3109f31640 1 -- 172.21.15.16:0/3786392367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec003f20 msgr2=0x7f30ec004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.188+0000 7f3109f31640 1 --2- 172.21.15.16:0/3786392367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec003f20 0x7f30ec004800 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f30f800c290 tx=0x7f30ec005830 comp rx=0 tx=0).stop 2024-09-19T07:49:41.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.188+0000 7f3103fff640 1 -- 172.21.15.16:0/3786392367 reap_dead start 2024-09-19T07:49:41.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.188+0000 7f3109f31640 1 -- 172.21.15.16:0/3786392367 shutdown_connections 2024-09-19T07:49:41.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.188+0000 7f3109f31640 1 -- 172.21.15.16:0/3786392367 >> 172.21.15.16:0/3786392367 conn(0x7f310409d760 msgr2=0x7f310408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:41.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.188+0000 7f3109f31640 1 -- 172.21.15.16:0/3786392367 shutdown_connections 2024-09-19T07:49:41.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.188+0000 7f3109f31640 1 -- 172.21.15.16:0/3786392367 wait complete. 2024-09-19T07:49:41.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.189+0000 7f3109f31640 1 Processor -- start 2024-09-19T07:49:41.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.190+0000 7f3109f31640 1 -- start start 2024-09-19T07:49:41.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.190+0000 7f3109f31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec003f20 0x7f31040b9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.190+0000 7f3109f31640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3104159510 0x7f3104159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.190+0000 7f3109f31640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31040b7210 0x7f31040b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.190+0000 7f3109f31640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f310415f4b0 con 0x7f30ec003f20 2024-09-19T07:49:41.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.190+0000 7f3109f31640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f31040cef60 con 0x7f31040b7210 2024-09-19T07:49:41.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.190+0000 7f3109f31640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f310415f630 con 0x7f3104159510 2024-09-19T07:49:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.190+0000 7f31037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec003f20 0x7f31040b9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.190+0000 7f31037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec003f20 0x7f31040b9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53616/0 (socket says 172.21.15.16:53616) 2024-09-19T07:49:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.190+0000 7f31037fe640 1 -- 172.21.15.16:0/2484710596 learned_addr learned my addr 172.21.15.16:0/2484710596 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.191+0000 7f3103fff640 1 --2- 172.21.15.16:0/2484710596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31040b7210 0x7f31040b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.191+0000 7f3102ffd640 1 --2- 172.21.15.16:0/2484710596 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3104159510 0x7f3104159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.191+0000 7f31037fe640 1 -- 172.21.15.16:0/2484710596 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3104159510 msgr2=0x7f3104159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.191+0000 7f31037fe640 1 --2- 172.21.15.16:0/2484710596 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3104159510 0x7f3104159990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.191+0000 7f31037fe640 1 -- 172.21.15.16:0/2484710596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31040b7210 msgr2=0x7f31040b76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.191+0000 7f31037fe640 1 --2- 172.21.15.16:0/2484710596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31040b7210 0x7f31040b76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.191+0000 7f31037fe640 1 -- 172.21.15.16:0/2484710596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31040cc0b0 con 0x7f30ec003f20 2024-09-19T07:49:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.191+0000 7f3102ffd640 1 --2- 172.21.15.16:0/2484710596 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3104159510 0x7f3104159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.191+0000 7f31037fe640 1 --2- 172.21.15.16:0/2484710596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec003f20 0x7f31040b9a10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f30f400fb90 tx=0x7f30f400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.191+0000 7f30ebfff640 1 -- 172.21.15.16:0/2484710596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f400dc30 con 0x7f30ec003f20 2024-09-19T07:49:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.191+0000 7f30ebfff640 1 -- 172.21.15.16:0/2484710596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30f4018070 con 0x7f30ec003f20 2024-09-19T07:49:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.191+0000 7f30ebfff640 1 -- 172.21.15.16:0/2484710596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f4003970 con 0x7f30ec003f20 2024-09-19T07:49:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.191+0000 7f3109f31640 1 -- 172.21.15.16:0/2484710596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31040cc350 con 0x7f30ec003f20 2024-09-19T07:49:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.191+0000 7f3109f31640 1 -- 172.21.15.16:0/2484710596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31041588c0 con 0x7f30ec003f20 2024-09-19T07:49:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.192+0000 7f3109f31640 1 -- 172.21.15.16:0/2484710596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31040cf2b0 con 0x7f30ec003f20 2024-09-19T07:49:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.193+0000 7f30ebfff640 1 -- 172.21.15.16:0/2484710596 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f30f4003b30 con 0x7f30ec003f20 2024-09-19T07:49:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.193+0000 7f30ebfff640 1 --2- 172.21.15.16:0/2484710596 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f30f006c2b0 0x7f30f006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.193+0000 7f30ebfff640 1 -- 172.21.15.16:0/2484710596 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f30f408b930 con 0x7f30ec003f20 2024-09-19T07:49:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.193+0000 7f3102ffd640 1 --2- 172.21.15.16:0/2484710596 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f30f006c2b0 0x7f30f006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.194+0000 7f3102ffd640 1 --2- 172.21.15.16:0/2484710596 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f30f006c2b0 0x7f30f006e770 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f30f8033000 tx=0x7f30f802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:41.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.198+0000 7f30ebfff640 1 -- 172.21.15.16:0/2484710596 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f30f4003e20 con 0x7f30ec003f20 2024-09-19T07:49:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.373+0000 7f3109f31640 1 -- 172.21.15.16:0/2484710596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f310415be80 con 0x7f30ec003f20 2024-09-19T07:49:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.375+0000 7f30ebfff640 1 -- 172.21.15.16:0/2484710596 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+17042 (secure 0 0 0) 0x7f30f4059f10 con 0x7f30ec003f20 2024-09-19T07:49:41.377 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:41.377 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":80,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:49:40.403575+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:49:38.349338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-19T07:49:39.377498+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:49:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.377+0000 7f3109f31640 1 -- 172.21.15.16:0/2484710596 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f30f006c2b0 msgr2=0x7f30f006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.377+0000 7f3109f31640 1 --2- 172.21.15.16:0/2484710596 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f30f006c2b0 0x7f30f006e770 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f30f8033000 tx=0x7f30f802e000 comp rx=0 tx=0).stop 2024-09-19T07:49:41.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.377+0000 7f3109f31640 1 -- 172.21.15.16:0/2484710596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec003f20 msgr2=0x7f31040b9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.377+0000 7f3109f31640 1 --2- 172.21.15.16:0/2484710596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec003f20 0x7f31040b9a10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f30f400fb90 tx=0x7f30f400d520 comp rx=0 tx=0).stop 2024-09-19T07:49:41.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.377+0000 7f3109f31640 1 -- 172.21.15.16:0/2484710596 shutdown_connections 2024-09-19T07:49:41.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.377+0000 7f3109f31640 1 --2- 172.21.15.16:0/2484710596 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f30f006c2b0 0x7f30f006e770 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.377+0000 7f3109f31640 1 --2- 172.21.15.16:0/2484710596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31040b7210 0x7f31040b76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.377+0000 7f3109f31640 1 --2- 172.21.15.16:0/2484710596 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3104159510 0x7f3104159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.377+0000 7f3109f31640 1 --2- 172.21.15.16:0/2484710596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec003f20 0x7f31040b9a10 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.377+0000 7f3109f31640 1 -- 172.21.15.16:0/2484710596 >> 172.21.15.16:0/2484710596 conn(0x7f310409d760 msgr2=0x7f31040cad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:41.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.377+0000 7f3109f31640 1 -- 172.21.15.16:0/2484710596 shutdown_connections 2024-09-19T07:49:41.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.377+0000 7f3109f31640 1 -- 172.21.15.16:0/2484710596 wait complete. 2024-09-19T07:49:41.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:49:41.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.490+0000 7f42016d0640 1 Processor -- start 2024-09-19T07:49:41.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.490+0000 7f42016d0640 1 -- start start 2024-09-19T07:49:41.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.491+0000 7f42016d0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41fc0d4840 0x7f41fc15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.491+0000 7f42016d0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41fc0d1b10 con 0x7f41fc0d0070 2024-09-19T07:49:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.491+0000 7f42016d0640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41fc0cf2b0 con 0x7f41fc15f0d0 2024-09-19T07:49:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.491+0000 7f42016d0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f41fc0cf590 con 0x7f41fc0d4840 2024-09-19T07:49:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.491+0000 7f41fa7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f41fc0d0070 0x7f41fc0d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59052/0 (socket says 172.21.15.16:59052) 2024-09-19T07:49:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.491+0000 7f41fa7fc640 1 -- 172.21.15.16:0/3718727779 learned_addr learned my addr 172.21.15.16:0/3718727779 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41fb7fe640 1 --2- 172.21.15.16:0/3718727779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41fc0d4840 0x7f41fc15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41f9ffb640 1 -- 172.21.15.16:0/3718727779 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2885031507 0 0) 0x7f41fc0d1b10 con 0x7f41fc0d0070 2024-09-19T07:49:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41f9ffb640 1 -- 172.21.15.16:0/3718727779 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41e8003520 con 0x7f41fc0d0070 2024-09-19T07:49:41.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41f9ffb640 1 -- 172.21.15.16:0/3718727779 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2451574321 0 0) 0x7f41e8003520 con 0x7f41fc0d0070 2024-09-19T07:49:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41f9ffb640 1 -- 172.21.15.16:0/3718727779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41fc0d4840 msgr2=0x7f41fc15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41f9ffb640 1 --2- 172.21.15.16:0/3718727779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41fc0d4840 0x7f41fc15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41f9ffb640 1 -- 172.21.15.16:0/3718727779 >> v1:172.21.15.146:6789/0 conn(0x7f41fc15f0d0 legacy=0x7f41fc0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41f9ffb640 1 -- 172.21.15.16:0/3718727779 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41fc0cf870 con 0x7f41fc0d0070 2024-09-19T07:49:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41f9ffb640 1 -- 172.21.15.16:0/3718727779 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f41e4001670 con 0x7f41fc0d0070 2024-09-19T07:49:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41fb7fe640 1 --2- 172.21.15.16:0/3718727779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41fc0d4840 0x7f41fc15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:49:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41f9ffb640 1 -- 172.21.15.16:0/3718727779 >> v1:172.21.15.16:6789/0 conn(0x7f41fc0d0070 legacy=0x7f41fc0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41f9ffb640 1 --2- 172.21.15.16:0/3718727779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e80039e0 0x7f41e8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41f9ffb640 1 --2- 172.21.15.16:0/3718727779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e8004390 0x7f41e8004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41f9ffb640 1 --2- 172.21.15.16:0/3718727779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41fc0d4840 0x7f41e8005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41f9ffb640 1 -- 172.21.15.16:0/3718727779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc0cf590 con 0x7f41e8004390 2024-09-19T07:49:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41f9ffb640 1 -- 172.21.15.16:0/3718727779 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f41e800ba10 con 0x7f41e80039e0 2024-09-19T07:49:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41f9ffb640 1 -- 172.21.15.16:0/3718727779 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f41e800bc10 con 0x7f41fc0d4840 2024-09-19T07:49:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41fa7fc640 1 --2- 172.21.15.16:0/3718727779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e8004390 0x7f41e8004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41faffd640 1 --2- 172.21.15.16:0/3718727779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e80039e0 0x7f41e8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41fb7fe640 1 --2- 172.21.15.16:0/3718727779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41fc0d4840 0x7f41e8005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41fa7fc640 1 -- 172.21.15.16:0/3718727779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41fc0d4840 msgr2=0x7f41e8005120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41fa7fc640 1 --2- 172.21.15.16:0/3718727779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41fc0d4840 0x7f41e8005120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41fa7fc640 1 -- 172.21.15.16:0/3718727779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e80039e0 msgr2=0x7f41e8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41fa7fc640 1 --2- 172.21.15.16:0/3718727779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e80039e0 0x7f41e8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.492+0000 7f41fa7fc640 1 -- 172.21.15.16:0/3718727779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e8003520 con 0x7f41e8004390 2024-09-19T07:49:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.493+0000 7f41fa7fc640 1 --2- 172.21.15.16:0/3718727779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e8004390 0x7f41e8004860 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f41e400c290 tx=0x7f41e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b36783d3d48a508 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:41.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.493+0000 7f41f9ffb640 1 -- 172.21.15.16:0/3718727779 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4007d50 con 0x7f41e8004390 2024-09-19T07:49:41.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.493+0000 7f41f9ffb640 1 -- 172.21.15.16:0/3718727779 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e4005af0 con 0x7f41e8004390 2024-09-19T07:49:41.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.493+0000 7f41f9ffb640 1 -- 172.21.15.16:0/3718727779 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4007070 con 0x7f41e8004390 2024-09-19T07:49:41.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.494+0000 7f42016d0640 1 -- 172.21.15.16:0/3718727779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e8004390 msgr2=0x7f41e8004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.494+0000 7f42016d0640 1 --2- 172.21.15.16:0/3718727779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e8004390 0x7f41e8004860 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f41e400c290 tx=0x7f41e8005830 comp rx=0 tx=0).stop 2024-09-19T07:49:41.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.494+0000 7f41fb7fe640 1 -- 172.21.15.16:0/3718727779 reap_dead start 2024-09-19T07:49:41.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.494+0000 7f42016d0640 1 -- 172.21.15.16:0/3718727779 shutdown_connections 2024-09-19T07:49:41.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.494+0000 7f42016d0640 1 -- 172.21.15.16:0/3718727779 >> 172.21.15.16:0/3718727779 conn(0x7f41fc09d760 msgr2=0x7f41fc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:41.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.494+0000 7f42016d0640 1 -- 172.21.15.16:0/3718727779 shutdown_connections 2024-09-19T07:49:41.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.494+0000 7f42016d0640 1 -- 172.21.15.16:0/3718727779 wait complete. 2024-09-19T07:49:41.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.495+0000 7f42016d0640 1 Processor -- start 2024-09-19T07:49:41.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.495+0000 7f42016d0640 1 -- start start 2024-09-19T07:49:41.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.496+0000 7f42016d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e8004390 0x7f41fc157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.496+0000 7f42016d0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41fc0b9330 0x7f41fc1e85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.496+0000 7f42016d0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41fc1e8b00 0x7f41fc0d2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.496+0000 7f42016d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc15f4b0 con 0x7f41e8004390 2024-09-19T07:49:41.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.496+0000 7f42016d0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc0cef60 con 0x7f41fc1e8b00 2024-09-19T07:49:41.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.496+0000 7f42016d0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f41fc15f630 con 0x7f41fc0b9330 2024-09-19T07:49:41.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.496+0000 7f41faffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e8004390 0x7f41fc157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.496+0000 7f41faffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e8004390 0x7f41fc157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53626/0 (socket says 172.21.15.16:53626) 2024-09-19T07:49:41.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.496+0000 7f41faffd640 1 -- 172.21.15.16:0/1158892673 learned_addr learned my addr 172.21.15.16:0/1158892673 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:41.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.496+0000 7f41fa7fc640 1 --2- 172.21.15.16:0/1158892673 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41fc0b9330 0x7f41fc1e85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.496+0000 7f41fb7fe640 1 --2- 172.21.15.16:0/1158892673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41fc1e8b00 0x7f41fc0d2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.497+0000 7f41faffd640 1 -- 172.21.15.16:0/1158892673 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41fc0b9330 msgr2=0x7f41fc1e85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.497+0000 7f41faffd640 1 --2- 172.21.15.16:0/1158892673 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41fc0b9330 0x7f41fc1e85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.497+0000 7f41faffd640 1 -- 172.21.15.16:0/1158892673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41fc1e8b00 msgr2=0x7f41fc0d2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.497+0000 7f41faffd640 1 --2- 172.21.15.16:0/1158892673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41fc1e8b00 0x7f41fc0d2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.497+0000 7f41faffd640 1 -- 172.21.15.16:0/1158892673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41fc0b97c0 con 0x7f41e8004390 2024-09-19T07:49:41.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.497+0000 7f41faffd640 1 --2- 172.21.15.16:0/1158892673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e8004390 0x7f41fc157a40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f41ec00fb60 tx=0x7f41ec00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:41.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.497+0000 7f41f37fe640 1 -- 172.21.15.16:0/1158892673 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec00dc30 con 0x7f41e8004390 2024-09-19T07:49:41.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.497+0000 7f41f37fe640 1 -- 172.21.15.16:0/1158892673 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ec018070 con 0x7f41e8004390 2024-09-19T07:49:41.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.497+0000 7f41f37fe640 1 -- 172.21.15.16:0/1158892673 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec003970 con 0x7f41e8004390 2024-09-19T07:49:41.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.497+0000 7f42016d0640 1 -- 172.21.15.16:0/1158892673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41fc0b9aa0 con 0x7f41e8004390 2024-09-19T07:49:41.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.497+0000 7f42016d0640 1 -- 172.21.15.16:0/1158892673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41fc0ba060 con 0x7f41e8004390 2024-09-19T07:49:41.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.498+0000 7f42016d0640 1 -- 172.21.15.16:0/1158892673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41fc0cf2b0 con 0x7f41e8004390 2024-09-19T07:49:41.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.498+0000 7f41f37fe640 1 -- 172.21.15.16:0/1158892673 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f41ec003b30 con 0x7f41e8004390 2024-09-19T07:49:41.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.499+0000 7f41f37fe640 1 --2- 172.21.15.16:0/1158892673 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f41dc06c2e0 0x7f41dc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.499+0000 7f41f37fe640 1 -- 172.21.15.16:0/1158892673 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f41ec08b870 con 0x7f41e8004390 2024-09-19T07:49:41.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.499+0000 7f41fa7fc640 1 --2- 172.21.15.16:0/1158892673 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f41dc06c2e0 0x7f41dc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.500+0000 7f41fa7fc640 1 --2- 172.21.15.16:0/1158892673 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f41dc06c2e0 0x7f41dc06e7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f41e4033000 tx=0x7f41e402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:41.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.503+0000 7f41f37fe640 1 -- 172.21.15.16:0/1158892673 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f41ec003e20 con 0x7f41e8004390 2024-09-19T07:49:41.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.683+0000 7f42016d0640 1 -- 172.21.15.16:0/1158892673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41fc15be80 con 0x7f41e8004390 2024-09-19T07:49:41.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.685+0000 7f41f37fe640 1 -- 172.21.15.16:0/1158892673 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+17042 (secure 0 0 0) 0x7f41ec059ee0 con 0x7f41e8004390 2024-09-19T07:49:41.686 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:41.686 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":80,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:49:40.403575+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:49:38.349338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-19T07:49:39.377498+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:49:41.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.687+0000 7f42016d0640 1 -- 172.21.15.16:0/1158892673 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f41dc06c2e0 msgr2=0x7f41dc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.687+0000 7f42016d0640 1 --2- 172.21.15.16:0/1158892673 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f41dc06c2e0 0x7f41dc06e7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f41e4033000 tx=0x7f41e402e000 comp rx=0 tx=0).stop 2024-09-19T07:49:41.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.687+0000 7f42016d0640 1 -- 172.21.15.16:0/1158892673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e8004390 msgr2=0x7f41fc157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.687+0000 7f42016d0640 1 --2- 172.21.15.16:0/1158892673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e8004390 0x7f41fc157a40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f41ec00fb60 tx=0x7f41ec00d520 comp rx=0 tx=0).stop 2024-09-19T07:49:41.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.687+0000 7f42016d0640 1 -- 172.21.15.16:0/1158892673 shutdown_connections 2024-09-19T07:49:41.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.687+0000 7f42016d0640 1 --2- 172.21.15.16:0/1158892673 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f41dc06c2e0 0x7f41dc06e7a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.687+0000 7f42016d0640 1 --2- 172.21.15.16:0/1158892673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41fc1e8b00 0x7f41fc0d2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.687+0000 7f42016d0640 1 --2- 172.21.15.16:0/1158892673 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41fc0b9330 0x7f41fc1e85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.687+0000 7f42016d0640 1 --2- 172.21.15.16:0/1158892673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41e8004390 0x7f41fc157a40 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.687+0000 7f42016d0640 1 -- 172.21.15.16:0/1158892673 >> 172.21.15.16:0/1158892673 conn(0x7f41fc09d760 msgr2=0x7f41fc15dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:41.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.687+0000 7f42016d0640 1 -- 172.21.15.16:0/1158892673 shutdown_connections 2024-09-19T07:49:41.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.687+0000 7f42016d0640 1 -- 172.21.15.16:0/1158892673 wait complete. 2024-09-19T07:49:41.700 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T07:49:41.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.809+0000 7f7c48e0e640 1 Processor -- start 2024-09-19T07:49:41.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.809+0000 7f7c48e0e640 1 -- start start 2024-09-19T07:49:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.810+0000 7f7c48e0e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7c4415e2a0 0x7f7c4406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.810+0000 7f7c48e0e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c4406d960 0x7f7c440d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.810+0000 7f7c48e0e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c440d1880 con 0x7f7c4415e680 2024-09-19T07:49:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.810+0000 7f7c48e0e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f7c440d1b60 con 0x7f7c4415e2a0 2024-09-19T07:49:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.810+0000 7f7c48e0e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7c440cf2d0 con 0x7f7c4406d960 2024-09-19T07:49:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.810+0000 7f7c4259c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7c4415e2a0 0x7f7c4406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.810+0000 7f7c42d9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c4406d960 0x7f7c440d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.810+0000 7f7c41d9b640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f7c4415e680 0x7f7c440cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:55942/0 (socket says 172.21.15.16:55942) 2024-09-19T07:49:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.810+0000 7f7c42d9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c4406d960 0x7f7c440d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:49592/0 (socket says 172.21.15.16:49592) 2024-09-19T07:49:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.810+0000 7f7c41d9b640 1 -- 172.21.15.16:0/1966717165 learned_addr learned my addr 172.21.15.16:0/1966717165 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.811+0000 7f7c4159a640 1 -- 172.21.15.16:0/1966717165 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2421445536 0 0) 0x7f7c440d1880 con 0x7f7c4415e680 2024-09-19T07:49:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.811+0000 7f7c4159a640 1 -- 172.21.15.16:0/1966717165 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c2c003520 con 0x7f7c4415e680 2024-09-19T07:49:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.811+0000 7f7c4259c640 1 -- 172.21.15.16:0/1966717165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c4406d960 msgr2=0x7f7c440d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.811+0000 7f7c4259c640 1 --2- 172.21.15.16:0/1966717165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c4406d960 0x7f7c440d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.811+0000 7f7c4259c640 1 -- 172.21.15.16:0/1966717165 >> v1:172.21.15.146:6789/0 conn(0x7f7c4415e680 legacy=0x7f7c440cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.811+0000 7f7c4259c640 1 -- 172.21.15.16:0/1966717165 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c440cf5b0 con 0x7f7c4415e2a0 2024-09-19T07:49:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.811+0000 7f7c42d9d640 1 --2- 172.21.15.16:0/1966717165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c4406d960 0x7f7c440d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:49:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.812+0000 7f7c4259c640 1 --2- 172.21.15.16:0/1966717165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c4415e2a0 0x7f7c4406d420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7c340097a0 tx=0x7f7c3402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eccf74549ca7906 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.812+0000 7f7c4159a640 1 -- 172.21.15.16:0/1966717165 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34037070 con 0x7f7c4415e2a0 2024-09-19T07:49:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.812+0000 7f7c4159a640 1 -- 172.21.15.16:0/1966717165 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c34032440 con 0x7f7c4415e2a0 2024-09-19T07:49:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.812+0000 7f7c4159a640 1 -- 172.21.15.16:0/1966717165 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c340327f0 con 0x7f7c4415e2a0 2024-09-19T07:49:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.812+0000 7f7c48e0e640 1 -- 172.21.15.16:0/1966717165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c4415e2a0 msgr2=0x7f7c4406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.812+0000 7f7c48e0e640 1 --2- 172.21.15.16:0/1966717165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c4415e2a0 0x7f7c4406d420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7c340097a0 tx=0x7f7c3402a180 comp rx=0 tx=0).stop 2024-09-19T07:49:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.812+0000 7f7c48e0e640 1 -- 172.21.15.16:0/1966717165 shutdown_connections 2024-09-19T07:49:41.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.812+0000 7f7c48e0e640 1 --2- 172.21.15.16:0/1966717165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c4406d960 0x7f7c440d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.812+0000 7f7c48e0e640 1 --2- 172.21.15.16:0/1966717165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c4415e2a0 0x7f7c4406d420 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.812+0000 7f7c48e0e640 1 -- 172.21.15.16:0/1966717165 >> 172.21.15.16:0/1966717165 conn(0x7f7c4409d760 msgr2=0x7f7c4408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:41.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.812+0000 7f7c48e0e640 1 -- 172.21.15.16:0/1966717165 shutdown_connections 2024-09-19T07:49:41.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.812+0000 7f7c48e0e640 1 -- 172.21.15.16:0/1966717165 wait complete. 2024-09-19T07:49:41.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.814+0000 7f7c48e0e640 1 Processor -- start 2024-09-19T07:49:41.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.814+0000 7f7c48e0e640 1 -- start start 2024-09-19T07:49:41.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c48e0e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c4406d960 0x7f7c4415d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c48e0e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c4415e2a0 0x7f7c440c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c48e0e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c4415e680 0x7f7c440c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c48e0e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c440d0360 con 0x7f7c4415e2a0 2024-09-19T07:49:41.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c48e0e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7c440d01e0 con 0x7f7c4406d960 2024-09-19T07:49:41.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c48e0e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7c440d48b0 con 0x7f7c4415e680 2024-09-19T07:49:41.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c41d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c4415e2a0 0x7f7c440c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c41d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c4415e2a0 0x7f7c440c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53632/0 (socket says 172.21.15.16:53632) 2024-09-19T07:49:41.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c41d9b640 1 -- 172.21.15.16:0/3468436669 learned_addr learned my addr 172.21.15.16:0/3468436669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:41.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c42d9d640 1 --2- 172.21.15.16:0/3468436669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c4415e680 0x7f7c440c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c4259c640 1 --2- 172.21.15.16:0/3468436669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c4406d960 0x7f7c4415d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c41d9b640 1 -- 172.21.15.16:0/3468436669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c4415e680 msgr2=0x7f7c440c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c41d9b640 1 --2- 172.21.15.16:0/3468436669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c4415e680 0x7f7c440c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c41d9b640 1 -- 172.21.15.16:0/3468436669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c4406d960 msgr2=0x7f7c4415d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c41d9b640 1 --2- 172.21.15.16:0/3468436669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c4406d960 0x7f7c4415d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c41d9b640 1 -- 172.21.15.16:0/3468436669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4409f8a0 con 0x7f7c4415e2a0 2024-09-19T07:49:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c41d9b640 1 --2- 172.21.15.16:0/3468436669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c4415e2a0 0x7f7c440c99b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7c38001eb0 tx=0x7f7c3800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c32ffd640 1 -- 172.21.15.16:0/3468436669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38017070 con 0x7f7c4415e2a0 2024-09-19T07:49:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c32ffd640 1 -- 172.21.15.16:0/3468436669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c38005d00 con 0x7f7c4415e2a0 2024-09-19T07:49:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c48e0e640 1 -- 172.21.15.16:0/3468436669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c440cc410 con 0x7f7c4415e2a0 2024-09-19T07:49:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.815+0000 7f7c32ffd640 1 -- 172.21.15.16:0/3468436669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38007310 con 0x7f7c4415e2a0 2024-09-19T07:49:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.816+0000 7f7c48e0e640 1 -- 172.21.15.16:0/3468436669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c440cb930 con 0x7f7c4415e2a0 2024-09-19T07:49:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.817+0000 7f7c48e0e640 1 -- 172.21.15.16:0/3468436669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c440d4c50 con 0x7f7c4415e2a0 2024-09-19T07:49:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.817+0000 7f7c32ffd640 1 -- 172.21.15.16:0/3468436669 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7c3801a070 con 0x7f7c4415e2a0 2024-09-19T07:49:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.818+0000 7f7c32ffd640 1 --2- 172.21.15.16:0/3468436669 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7c2406c2b0 0x7f7c2406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.818+0000 7f7c32ffd640 1 -- 172.21.15.16:0/3468436669 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f7c3808a950 con 0x7f7c4415e2a0 2024-09-19T07:49:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.818+0000 7f7c4259c640 1 --2- 172.21.15.16:0/3468436669 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7c2406c2b0 0x7f7c2406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.818+0000 7f7c4259c640 1 --2- 172.21.15.16:0/3468436669 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7c2406c2b0 0x7f7c2406e770 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7c34034000 tx=0x7f7c3402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:41.822+0000 7f7c32ffd640 1 -- 172.21.15.16:0/3468436669 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7c4409f8a0 con 0x7f7c4415e2a0 2024-09-19T07:49:42.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:42.037+0000 7f7c48e0e640 1 -- 172.21.15.16:0/3468436669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7c440cb380 con 0x7f7c4415e2a0 2024-09-19T07:49:42.332 INFO:tasks.ceph.mds.d.smithi146.stderr:2024-09-19T07:49:42.331+0000 7f7f3ff70640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T07:49:43.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.050+0000 7f7c32ffd640 1 -- 172.21.15.16:0/3468436669 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v40) ==== 118+0+0 (secure 0 0 0) 0x7f7c38058fc0 con 0x7f7c4415e2a0 2024-09-19T07:49:43.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.053+0000 7f7c48e0e640 1 -- 172.21.15.16:0/3468436669 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7c2406c2b0 msgr2=0x7f7c2406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:43.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.053+0000 7f7c48e0e640 1 --2- 172.21.15.16:0/3468436669 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7c2406c2b0 0x7f7c2406e770 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7c34034000 tx=0x7f7c3402e000 comp rx=0 tx=0).stop 2024-09-19T07:49:43.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.053+0000 7f7c48e0e640 1 -- 172.21.15.16:0/3468436669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c4415e2a0 msgr2=0x7f7c440c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:43.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.053+0000 7f7c48e0e640 1 --2- 172.21.15.16:0/3468436669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c4415e2a0 0x7f7c440c99b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7c38001eb0 tx=0x7f7c3800c820 comp rx=0 tx=0).stop 2024-09-19T07:49:43.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.054+0000 7f7c48e0e640 1 -- 172.21.15.16:0/3468436669 shutdown_connections 2024-09-19T07:49:43.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.054+0000 7f7c48e0e640 1 --2- 172.21.15.16:0/3468436669 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7c2406c2b0 0x7f7c2406e770 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:43.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.054+0000 7f7c48e0e640 1 --2- 172.21.15.16:0/3468436669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c4415e680 0x7f7c440c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:43.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.054+0000 7f7c48e0e640 1 --2- 172.21.15.16:0/3468436669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c4415e2a0 0x7f7c440c99b0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:43.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.054+0000 7f7c48e0e640 1 --2- 172.21.15.16:0/3468436669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c4406d960 0x7f7c4415d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:43.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.054+0000 7f7c48e0e640 1 -- 172.21.15.16:0/3468436669 >> 172.21.15.16:0/3468436669 conn(0x7f7c4409d760 msgr2=0x7f7c440b9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:43.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.054+0000 7f7c48e0e640 1 -- 172.21.15.16:0/3468436669 shutdown_connections 2024-09-19T07:49:43.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.054+0000 7f7c48e0e640 1 -- 172.21.15.16:0/3468436669 wait complete. 2024-09-19T07:49:43.069 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T07:49:43.069 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T07:49:43.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.179+0000 7f71a30ac640 1 Processor -- start 2024-09-19T07:49:43.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.179+0000 7f71a30ac640 1 -- start start 2024-09-19T07:49:43.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.180+0000 7f71a30ac640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f719c15e680 0x7f719c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:43.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.180+0000 7f71a30ac640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f719c0d0010 0x7f719c0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:43.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.180+0000 7f71a30ac640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f719c0d1880 con 0x7f719c15e2a0 2024-09-19T07:49:43.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.180+0000 7f71a30ac640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f719c0d1b60 con 0x7f719c15e680 2024-09-19T07:49:43.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.180+0000 7f71a30ac640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f719c0cf2d0 con 0x7f719c0d0010 2024-09-19T07:49:43.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.180+0000 7f71a1649640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f719c0d0010 0x7f719c0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:43.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.180+0000 7f71a1649640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f719c0d0010 0x7f719c0d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:53344/0 (socket says 172.21.15.16:53344) 2024-09-19T07:49:43.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.180+0000 7f71a1649640 1 -- 172.21.15.16:0/1619084794 learned_addr learned my addr 172.21.15.16:0/1619084794 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:43.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.180+0000 7f719bfff640 1 --2- 172.21.15.16:0/1619084794 >> v2:172.21.15.146:3300/0 conn(0x7f719c15e680 0x7f719c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:43.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.181+0000 7f71a1649640 1 -- 172.21.15.16:0/1619084794 >> v2:172.21.15.146:3300/0 conn(0x7f719c15e680 msgr2=0x7f719c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:43.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.181+0000 7f71a1649640 1 --2- 172.21.15.16:0/1619084794 >> v2:172.21.15.146:3300/0 conn(0x7f719c15e680 0x7f719c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:43.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.181+0000 7f71a1649640 1 -- 172.21.15.16:0/1619084794 >> v1:172.21.15.146:6789/0 conn(0x7f719c15e2a0 legacy=0x7f719c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:43.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.181+0000 7f71a1649640 1 -- 172.21.15.16:0/1619084794 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719c0cf5b0 con 0x7f719c0d0010 2024-09-19T07:49:43.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.181+0000 7f71a1649640 1 --2- 172.21.15.16:0/1619084794 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f719c0d0010 0x7f719c0d4370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7194009e50 tx=0x7f719402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=554311df7789e729 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:43.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.182+0000 7f719b7fe640 1 -- 172.21.15.16:0/1619084794 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7194036070 con 0x7f719c0d0010 2024-09-19T07:49:43.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.182+0000 7f719b7fe640 1 -- 172.21.15.16:0/1619084794 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7194031600 con 0x7f719c0d0010 2024-09-19T07:49:43.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.182+0000 7f719b7fe640 1 -- 172.21.15.16:0/1619084794 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71940319d0 con 0x7f719c0d0010 2024-09-19T07:49:43.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.182+0000 7f71a30ac640 1 -- 172.21.15.16:0/1619084794 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f719c0d0010 msgr2=0x7f719c0d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.182+0000 7f71a30ac640 1 --2- 172.21.15.16:0/1619084794 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f719c0d0010 0x7f719c0d4370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7194009e50 tx=0x7f719402a680 comp rx=0 tx=0).stop 2024-09-19T07:49:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.182+0000 7f71a30ac640 1 -- 172.21.15.16:0/1619084794 shutdown_connections 2024-09-19T07:49:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.182+0000 7f71a30ac640 1 --2- 172.21.15.16:0/1619084794 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f719c0d0010 0x7f719c0d4370 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.182+0000 7f71a30ac640 1 --2- 172.21.15.16:0/1619084794 >> v2:172.21.15.146:3300/0 conn(0x7f719c15e680 0x7f719c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.182+0000 7f71a30ac640 1 -- 172.21.15.16:0/1619084794 >> 172.21.15.16:0/1619084794 conn(0x7f719c09d760 msgr2=0x7f719c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.182+0000 7f71a30ac640 1 -- 172.21.15.16:0/1619084794 shutdown_connections 2024-09-19T07:49:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.182+0000 7f71a30ac640 1 -- 172.21.15.16:0/1619084794 wait complete. 2024-09-19T07:49:43.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.183+0000 7f71a30ac640 1 Processor -- start 2024-09-19T07:49:43.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.184+0000 7f71a30ac640 1 -- start start 2024-09-19T07:49:43.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.184+0000 7f71a30ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f719c0d0010 0x7f719c0a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:43.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.184+0000 7f71a30ac640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f719c15e2a0 0x7f719c0a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.184+0000 7f71a30ac640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f719c15e680 0x7f719c0a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.184+0000 7f71a30ac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f719c06dcb0 con 0x7f719c0d0010 2024-09-19T07:49:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.184+0000 7f71a30ac640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f719c06db30 con 0x7f719c15e680 2024-09-19T07:49:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.184+0000 7f71a30ac640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f719c0d48b0 con 0x7f719c15e2a0 2024-09-19T07:49:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.185+0000 7f71a0e48640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f719c0d0010 0x7f719c0a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.185+0000 7f71a0e48640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f719c0d0010 0x7f719c0a1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60832/0 (socket says 172.21.15.16:60832) 2024-09-19T07:49:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.185+0000 7f71a0e48640 1 -- 172.21.15.16:0/1831748779 learned_addr learned my addr 172.21.15.16:0/1831748779 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.185+0000 7f719bfff640 1 --2- 172.21.15.16:0/1831748779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f719c15e2a0 0x7f719c0a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.185+0000 7f71a1649640 1 --2- 172.21.15.16:0/1831748779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f719c15e680 0x7f719c0a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.185+0000 7f71a0e48640 1 -- 172.21.15.16:0/1831748779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f719c15e2a0 msgr2=0x7f719c0a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.185+0000 7f71a0e48640 1 --2- 172.21.15.16:0/1831748779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f719c15e2a0 0x7f719c0a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.185+0000 7f71a0e48640 1 -- 172.21.15.16:0/1831748779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f719c15e680 msgr2=0x7f719c0a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.185+0000 7f71a0e48640 1 --2- 172.21.15.16:0/1831748779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f719c15e680 0x7f719c0a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.185+0000 7f71a0e48640 1 -- 172.21.15.16:0/1831748779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719c0a0000 con 0x7f719c0d0010 2024-09-19T07:49:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.185+0000 7f71a0e48640 1 --2- 172.21.15.16:0/1831748779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f719c0d0010 0x7f719c0a1060 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f718c00c400 tx=0x7f718c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.185+0000 7f71997fa640 1 -- 172.21.15.16:0/1831748779 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c0052d0 con 0x7f719c0d0010 2024-09-19T07:49:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.185+0000 7f71997fa640 1 -- 172.21.15.16:0/1831748779 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f718c005490 con 0x7f719c0d0010 2024-09-19T07:49:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.185+0000 7f71a30ac640 1 -- 172.21.15.16:0/1831748779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719c0a02a0 con 0x7f719c0d0010 2024-09-19T07:49:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.185+0000 7f71997fa640 1 -- 172.21.15.16:0/1831748779 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c007260 con 0x7f719c0d0010 2024-09-19T07:49:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.185+0000 7f71a30ac640 1 -- 172.21.15.16:0/1831748779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f719c0a0860 con 0x7f719c0d0010 2024-09-19T07:49:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.186+0000 7f71a30ac640 1 -- 172.21.15.16:0/1831748779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f719c0d4c10 con 0x7f719c0d0010 2024-09-19T07:49:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.187+0000 7f71997fa640 1 -- 172.21.15.16:0/1831748779 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f718c007400 con 0x7f719c0d0010 2024-09-19T07:49:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.187+0000 7f71997fa640 1 --2- 172.21.15.16:0/1831748779 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f718806c2b0 0x7f718806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.188+0000 7f71997fa640 1 -- 172.21.15.16:0/1831748779 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f718c08aaf0 con 0x7f719c0d0010 2024-09-19T07:49:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.188+0000 7f719bfff640 1 --2- 172.21.15.16:0/1831748779 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f718806c2b0 0x7f718806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.188+0000 7f719bfff640 1 --2- 172.21.15.16:0/1831748779 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f718806c2b0 0x7f718806e770 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f719c0cb640 tx=0x7f7190009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:43.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.192+0000 7f71997fa640 1 -- 172.21.15.16:0/1831748779 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f718c0589e0 con 0x7f719c0d0010 2024-09-19T07:49:43.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:43.405+0000 7f71a30ac640 1 -- 172.21.15.16:0/1831748779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f719c06cf90 con 0x7f719c0d0010 2024-09-19T07:49:43.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-19T07:49:43.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-19T07:49:43.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-19T07:49:43.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-19T07:49:44.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.056+0000 7f71997fa640 1 -- 172.21.15.16:0/1831748779 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7f718c05cce0 con 0x7f719c0d0010 2024-09-19T07:49:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.059+0000 7f71a30ac640 1 -- 172.21.15.16:0/1831748779 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f718806c2b0 msgr2=0x7f718806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.059+0000 7f71a30ac640 1 --2- 172.21.15.16:0/1831748779 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f718806c2b0 0x7f718806e770 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f719c0cb640 tx=0x7f7190009190 comp rx=0 tx=0).stop 2024-09-19T07:49:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.059+0000 7f71a30ac640 1 -- 172.21.15.16:0/1831748779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f719c0d0010 msgr2=0x7f719c0a1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.059+0000 7f71a30ac640 1 --2- 172.21.15.16:0/1831748779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f719c0d0010 0x7f719c0a1060 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f718c00c400 tx=0x7f718c00c8c0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.059+0000 7f71a30ac640 1 -- 172.21.15.16:0/1831748779 shutdown_connections 2024-09-19T07:49:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.059+0000 7f71a30ac640 1 --2- 172.21.15.16:0/1831748779 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f718806c2b0 0x7f718806e770 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.059+0000 7f71a30ac640 1 --2- 172.21.15.16:0/1831748779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f719c15e680 0x7f719c0a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.059+0000 7f71a30ac640 1 --2- 172.21.15.16:0/1831748779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f719c15e2a0 0x7f719c0a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.059+0000 7f71a30ac640 1 --2- 172.21.15.16:0/1831748779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f719c0d0010 0x7f719c0a1060 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.059+0000 7f71a30ac640 1 -- 172.21.15.16:0/1831748779 >> 172.21.15.16:0/1831748779 conn(0x7f719c09d760 msgr2=0x7f719c15d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.059+0000 7f71a30ac640 1 -- 172.21.15.16:0/1831748779 shutdown_connections 2024-09-19T07:49:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.059+0000 7f71a30ac640 1 -- 172.21.15.16:0/1831748779 wait complete. 2024-09-19T07:49:44.075 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T07:49:44.075 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:49:44.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.183+0000 7eff45ffd640 1 Processor -- start 2024-09-19T07:49:44.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.183+0000 7eff45ffd640 1 -- start start 2024-09-19T07:49:44.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.183+0000 7eff45ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff400d0070 0x7eff4015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.183+0000 7eff45ffd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff400d1b10 con 0x7eff4015f0d0 2024-09-19T07:49:44.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.183+0000 7eff45ffd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff400cf2b0 con 0x7eff400d4130 2024-09-19T07:49:44.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.183+0000 7eff45ffd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7eff400cf590 con 0x7eff400d0070 2024-09-19T07:49:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.184+0000 7eff3effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff400d0070 0x7eff4015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.184+0000 7eff3effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff400d0070 0x7eff4015f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60844/0 (socket says 172.21.15.16:60844) 2024-09-19T07:49:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.184+0000 7eff3f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7eff4015f0d0 0x7eff400d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36078/0 (socket says 172.21.15.16:36078) 2024-09-19T07:49:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.184+0000 7eff3effd640 1 -- 172.21.15.16:0/3291335956 learned_addr learned my addr 172.21.15.16:0/3291335956 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.184+0000 7eff3effd640 1 -- 172.21.15.16:0/3291335956 >> v1:172.21.15.146:6789/0 conn(0x7eff400d4130 legacy=0x7eff400d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.184+0000 7eff3effd640 1 -- 172.21.15.16:0/3291335956 >> v1:172.21.15.16:6789/0 conn(0x7eff4015f0d0 legacy=0x7eff400d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.184+0000 7eff3effd640 1 -- 172.21.15.16:0/3291335956 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff400cf870 con 0x7eff400d0070 2024-09-19T07:49:44.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.185+0000 7eff3effd640 1 --2- 172.21.15.16:0/3291335956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff400d0070 0x7eff4015f4b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7eff34009870 tx=0x7eff3402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76a42a0e4d37576 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:44.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.185+0000 7eff3e7fc640 1 -- 172.21.15.16:0/3291335956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff34037070 con 0x7eff400d0070 2024-09-19T07:49:44.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.185+0000 7eff3e7fc640 1 -- 172.21.15.16:0/3291335956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff34032440 con 0x7eff400d0070 2024-09-19T07:49:44.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.185+0000 7eff3e7fc640 1 -- 172.21.15.16:0/3291335956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff34032810 con 0x7eff400d0070 2024-09-19T07:49:44.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.185+0000 7eff45ffd640 1 -- 172.21.15.16:0/3291335956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff400d0070 msgr2=0x7eff4015f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.185+0000 7eff45ffd640 1 --2- 172.21.15.16:0/3291335956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff400d0070 0x7eff4015f4b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7eff34009870 tx=0x7eff3402a180 comp rx=0 tx=0).stop 2024-09-19T07:49:44.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.185+0000 7eff45ffd640 1 -- 172.21.15.16:0/3291335956 shutdown_connections 2024-09-19T07:49:44.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.185+0000 7eff45ffd640 1 --2- 172.21.15.16:0/3291335956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff400d0070 0x7eff4015f4b0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.185+0000 7eff45ffd640 1 -- 172.21.15.16:0/3291335956 >> 172.21.15.16:0/3291335956 conn(0x7eff4009d760 msgr2=0x7eff4008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:44.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.185+0000 7eff45ffd640 1 -- 172.21.15.16:0/3291335956 shutdown_connections 2024-09-19T07:49:44.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.185+0000 7eff45ffd640 1 -- 172.21.15.16:0/3291335956 wait complete. 2024-09-19T07:49:44.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.187+0000 7eff45ffd640 1 Processor -- start 2024-09-19T07:49:44.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.187+0000 7eff45ffd640 1 -- start start 2024-09-19T07:49:44.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff45ffd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff400d0070 0x7eff400cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff45ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff400d4130 0x7eff400b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff45ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff4015f0d0 0x7eff40160980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff45ffd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff400b2f70 con 0x7eff4015f0d0 2024-09-19T07:49:44.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff45ffd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7eff400cef60 con 0x7eff400d4130 2024-09-19T07:49:44.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff45ffd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7eff400b30f0 con 0x7eff400d0070 2024-09-19T07:49:44.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff4015f0d0 0x7eff40160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff4015f0d0 0x7eff40160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60846/0 (socket says 172.21.15.16:60846) 2024-09-19T07:49:44.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff3ffff640 1 -- 172.21.15.16:0/2617056170 learned_addr learned my addr 172.21.15.16:0/2617056170 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:44.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff3f7fe640 1 --2- 172.21.15.16:0/2617056170 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff400d0070 0x7eff400cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff3effd640 1 --2- 172.21.15.16:0/2617056170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff400d4130 0x7eff400b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff3ffff640 1 -- 172.21.15.16:0/2617056170 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff400d0070 msgr2=0x7eff400cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff3ffff640 1 --2- 172.21.15.16:0/2617056170 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff400d0070 0x7eff400cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff3ffff640 1 -- 172.21.15.16:0/2617056170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff400d4130 msgr2=0x7eff400b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff3ffff640 1 --2- 172.21.15.16:0/2617056170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff400d4130 0x7eff400b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff3ffff640 1 -- 172.21.15.16:0/2617056170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff400cc5f0 con 0x7eff4015f0d0 2024-09-19T07:49:44.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff3effd640 1 --2- 172.21.15.16:0/2617056170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff400d4130 0x7eff400b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:44.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff3ffff640 1 --2- 172.21.15.16:0/2617056170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff4015f0d0 0x7eff40160980 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7eff3800c7b0 tx=0x7eff3800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:44.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff27fff640 1 -- 172.21.15.16:0/2617056170 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38017070 con 0x7eff4015f0d0 2024-09-19T07:49:44.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff27fff640 1 -- 172.21.15.16:0/2617056170 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff38003da0 con 0x7eff4015f0d0 2024-09-19T07:49:44.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff27fff640 1 -- 172.21.15.16:0/2617056170 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff380052b0 con 0x7eff4015f0d0 2024-09-19T07:49:44.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff45ffd640 1 -- 172.21.15.16:0/2617056170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff400cc8d0 con 0x7eff4015f0d0 2024-09-19T07:49:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.188+0000 7eff45ffd640 1 -- 172.21.15.16:0/2617056170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff400cce90 con 0x7eff4015f0d0 2024-09-19T07:49:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.190+0000 7eff45ffd640 1 -- 172.21.15.16:0/2617056170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff4015f4b0 con 0x7eff4015f0d0 2024-09-19T07:49:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.190+0000 7eff27fff640 1 -- 172.21.15.16:0/2617056170 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7eff38005450 con 0x7eff4015f0d0 2024-09-19T07:49:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.190+0000 7eff27fff640 1 --2- 172.21.15.16:0/2617056170 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7eff2c06c2b0 0x7eff2c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.190+0000 7eff27fff640 1 -- 172.21.15.16:0/2617056170 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7eff3808ab70 con 0x7eff4015f0d0 2024-09-19T07:49:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.191+0000 7eff3f7fe640 1 --2- 172.21.15.16:0/2617056170 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7eff2c06c2b0 0x7eff2c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.191+0000 7eff3f7fe640 1 --2- 172.21.15.16:0/2617056170 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7eff2c06c2b0 0x7eff2c06e770 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7eff30007fd0 tx=0x7eff30007c40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:44.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.195+0000 7eff27fff640 1 -- 172.21.15.16:0/2617056170 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7eff38059150 con 0x7eff4015f0d0 2024-09-19T07:49:44.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.410+0000 7eff45ffd640 1 -- 172.21.15.16:0/2617056170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff400d0c60 con 0x7eff4015f0d0 2024-09-19T07:49:44.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.411+0000 7eff27fff640 1 -- 172.21.15.16:0/2617056170 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5222 (secure 0 0 0) 0x7eff38058a50 con 0x7eff4015f0d0 2024-09-19T07:49:44.412 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:44.412 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"btime":"2024-09-19T07:49:44:053013+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/945862061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":945862061},{"type":"v1","addr":"172.21.15.146:6836","nonce":945862061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/42753795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":42753795},{"type":"v1","addr":"172.21.15.16:6834","nonce":42753795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1389787480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1389787480},{"type":"v1","addr":"172.21.15.16:6835","nonce":1389787480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:49:40.403885+0000","modified":"2024-09-19T07:49:43.407266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6096":{"gid":6096,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/3642987189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3642987189},{"type":"v1","addr":"172.21.15.146:6837","nonce":3642987189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6096,"qdb_cluster":[6096]},"id":5}]} 2024-09-19T07:49:44.413 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 41 2024-09-19T07:49:44.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.414+0000 7eff45ffd640 1 -- 172.21.15.16:0/2617056170 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7eff2c06c2b0 msgr2=0x7eff2c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.414+0000 7eff45ffd640 1 --2- 172.21.15.16:0/2617056170 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7eff2c06c2b0 0x7eff2c06e770 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7eff30007fd0 tx=0x7eff30007c40 comp rx=0 tx=0).stop 2024-09-19T07:49:44.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.414+0000 7eff45ffd640 1 -- 172.21.15.16:0/2617056170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff4015f0d0 msgr2=0x7eff40160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.414+0000 7eff45ffd640 1 --2- 172.21.15.16:0/2617056170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff4015f0d0 0x7eff40160980 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7eff3800c7b0 tx=0x7eff3800cc70 comp rx=0 tx=0).stop 2024-09-19T07:49:44.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.414+0000 7eff45ffd640 1 -- 172.21.15.16:0/2617056170 shutdown_connections 2024-09-19T07:49:44.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.414+0000 7eff45ffd640 1 --2- 172.21.15.16:0/2617056170 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7eff2c06c2b0 0x7eff2c06e770 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.414+0000 7eff45ffd640 1 --2- 172.21.15.16:0/2617056170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff4015f0d0 0x7eff40160980 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.414+0000 7eff45ffd640 1 --2- 172.21.15.16:0/2617056170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff400d4130 0x7eff400b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.414+0000 7eff45ffd640 1 --2- 172.21.15.16:0/2617056170 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff400d0070 0x7eff400cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.414+0000 7eff45ffd640 1 -- 172.21.15.16:0/2617056170 >> 172.21.15.16:0/2617056170 conn(0x7eff4009d760 msgr2=0x7eff400b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:44.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.414+0000 7eff45ffd640 1 -- 172.21.15.16:0/2617056170 shutdown_connections 2024-09-19T07:49:44.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.414+0000 7eff45ffd640 1 -- 172.21.15.16:0/2617056170 wait complete. 2024-09-19T07:49:44.426 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:49:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.527+0000 7f4c9d5c6640 1 Processor -- start 2024-09-19T07:49:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.527+0000 7f4c9d5c6640 1 -- start start 2024-09-19T07:49:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.528+0000 7f4c9d5c6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c980d0070 0x7f4c9815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.528+0000 7f4c9d5c6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c980d4130 0x7f4c980d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.528+0000 7f4c9d5c6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c980d4ae0 con 0x7f4c9815f0d0 2024-09-19T07:49:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.528+0000 7f4c9d5c6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f4c980d1b10 con 0x7f4c980d0070 2024-09-19T07:49:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.528+0000 7f4c9d5c6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4c980cf2b0 con 0x7f4c980d4130 2024-09-19T07:49:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.528+0000 7f4c967fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c980d0070 0x7f4c9815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.528+0000 7f4c967fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c980d0070 0x7f4c9815f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60860/0 (socket says 172.21.15.16:60860) 2024-09-19T07:49:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.528+0000 7f4c967fc640 1 -- 172.21.15.16:0/2088318659 learned_addr learned my addr 172.21.15.16:0/2088318659 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.528+0000 7f4c977fe640 1 --2- 172.21.15.16:0/2088318659 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c980d4130 0x7f4c980d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.529+0000 7f4c967fc640 1 -- 172.21.15.16:0/2088318659 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c980d4130 msgr2=0x7f4c980d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.529+0000 7f4c967fc640 1 --2- 172.21.15.16:0/2088318659 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c980d4130 0x7f4c980d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.529+0000 7f4c967fc640 1 -- 172.21.15.16:0/2088318659 >> v1:172.21.15.146:6789/0 conn(0x7f4c9815f0d0 legacy=0x7f4c980d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:44.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.529+0000 7f4c967fc640 1 -- 172.21.15.16:0/2088318659 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c980cf590 con 0x7f4c980d0070 2024-09-19T07:49:44.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.529+0000 7f4c967fc640 1 --2- 172.21.15.16:0/2088318659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980d0070 0x7f4c9815f4b0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009870 tx=0x7f4c8002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1b320d747a11f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:44.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.530+0000 7f4c95ffb640 1 -- 172.21.15.16:0/2088318659 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80037070 con 0x7f4c980d0070 2024-09-19T07:49:44.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.530+0000 7f4c95ffb640 1 -- 172.21.15.16:0/2088318659 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c80032440 con 0x7f4c980d0070 2024-09-19T07:49:44.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.530+0000 7f4c95ffb640 1 -- 172.21.15.16:0/2088318659 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80032810 con 0x7f4c980d0070 2024-09-19T07:49:44.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.530+0000 7f4c9d5c6640 1 -- 172.21.15.16:0/2088318659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980d0070 msgr2=0x7f4c9815f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.530+0000 7f4c9d5c6640 1 --2- 172.21.15.16:0/2088318659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980d0070 0x7f4c9815f4b0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009870 tx=0x7f4c8002a1c0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.530+0000 7f4c9d5c6640 1 -- 172.21.15.16:0/2088318659 shutdown_connections 2024-09-19T07:49:44.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.530+0000 7f4c9d5c6640 1 --2- 172.21.15.16:0/2088318659 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c980d4130 0x7f4c980d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.530+0000 7f4c9d5c6640 1 --2- 172.21.15.16:0/2088318659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980d0070 0x7f4c9815f4b0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.530+0000 7f4c9d5c6640 1 -- 172.21.15.16:0/2088318659 >> 172.21.15.16:0/2088318659 conn(0x7f4c9809d760 msgr2=0x7f4c9808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:44.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.530+0000 7f4c9d5c6640 1 -- 172.21.15.16:0/2088318659 shutdown_connections 2024-09-19T07:49:44.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.530+0000 7f4c9d5c6640 1 -- 172.21.15.16:0/2088318659 wait complete. 2024-09-19T07:49:44.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.531+0000 7f4c9d5c6640 1 Processor -- start 2024-09-19T07:49:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.532+0000 7f4c9d5c6640 1 -- start start 2024-09-19T07:49:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.532+0000 7f4c9d5c6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c980d0070 0x7f4c9814a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.532+0000 7f4c9d5c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980d4130 0x7f4c9814abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.532+0000 7f4c9d5c6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c9815f0d0 0x7f4c9815b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.532+0000 7f4c9d5c6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c980cef60 con 0x7f4c980d4130 2024-09-19T07:49:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.532+0000 7f4c9d5c6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4c980cfb50 con 0x7f4c980d0070 2024-09-19T07:49:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.532+0000 7f4c9d5c6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4c980b2f70 con 0x7f4c9815f0d0 2024-09-19T07:49:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.532+0000 7f4c967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980d4130 0x7f4c9814abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.532+0000 7f4c967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980d4130 0x7f4c9814abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60862/0 (socket says 172.21.15.16:60862) 2024-09-19T07:49:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.532+0000 7f4c967fc640 1 -- 172.21.15.16:0/4059511758 learned_addr learned my addr 172.21.15.16:0/4059511758 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.532+0000 7f4c96ffd640 1 --2- 172.21.15.16:0/4059511758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c980d0070 0x7f4c9814a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.532+0000 7f4c977fe640 1 --2- 172.21.15.16:0/4059511758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c9815f0d0 0x7f4c9815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.533+0000 7f4c967fc640 1 -- 172.21.15.16:0/4059511758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c9815f0d0 msgr2=0x7f4c9815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.533+0000 7f4c967fc640 1 --2- 172.21.15.16:0/4059511758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c9815f0d0 0x7f4c9815b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.533+0000 7f4c967fc640 1 -- 172.21.15.16:0/4059511758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c980d0070 msgr2=0x7f4c9814a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.533+0000 7f4c967fc640 1 --2- 172.21.15.16:0/4059511758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c980d0070 0x7f4c9814a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.533+0000 7f4c967fc640 1 -- 172.21.15.16:0/4059511758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c980b40a0 con 0x7f4c980d4130 2024-09-19T07:49:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.533+0000 7f4c967fc640 1 --2- 172.21.15.16:0/4059511758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980d4130 0x7f4c9814abe0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f4c80000c00 tx=0x7f4c800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.533+0000 7f4c8f7fe640 1 -- 172.21.15.16:0/4059511758 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80041020 con 0x7f4c980d4130 2024-09-19T07:49:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.533+0000 7f4c8f7fe640 1 -- 172.21.15.16:0/4059511758 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c800306c0 con 0x7f4c980d4130 2024-09-19T07:49:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.533+0000 7f4c9d5c6640 1 -- 172.21.15.16:0/4059511758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c980b4380 con 0x7f4c980d4130 2024-09-19T07:49:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.533+0000 7f4c8f7fe640 1 -- 172.21.15.16:0/4059511758 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80037040 con 0x7f4c980d4130 2024-09-19T07:49:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.533+0000 7f4c9d5c6640 1 -- 172.21.15.16:0/4059511758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c980b4940 con 0x7f4c980d4130 2024-09-19T07:49:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.534+0000 7f4c9d5c6640 1 -- 172.21.15.16:0/4059511758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c980d45a0 con 0x7f4c980d4130 2024-09-19T07:49:44.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.535+0000 7f4c8f7fe640 1 -- 172.21.15.16:0/4059511758 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4c80044020 con 0x7f4c980d4130 2024-09-19T07:49:44.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.535+0000 7f4c8f7fe640 1 --2- 172.21.15.16:0/4059511758 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4c7806c2e0 0x7f4c7806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.535+0000 7f4c8f7fe640 1 -- 172.21.15.16:0/4059511758 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f4c800abcb0 con 0x7f4c980d4130 2024-09-19T07:49:44.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.535+0000 7f4c96ffd640 1 --2- 172.21.15.16:0/4059511758 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4c7806c2e0 0x7f4c7806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.536+0000 7f4c96ffd640 1 --2- 172.21.15.16:0/4059511758 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4c7806c2e0 0x7f4c7806e7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4c900016e0 tx=0x7f4c90009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:44.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.539+0000 7f4c8f7fe640 1 -- 172.21.15.16:0/4059511758 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4c8004b030 con 0x7f4c980d4130 2024-09-19T07:49:44.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.718+0000 7f4c9d5c6640 1 -- 172.21.15.16:0/4059511758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c9806cf30 con 0x7f4c980d4130 2024-09-19T07:49:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.719+0000 7f4c8f7fe640 1 -- 172.21.15.16:0/4059511758 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+17029 (secure 0 0 0) 0x7f4c80079c40 con 0x7f4c980d4130 2024-09-19T07:49:44.721 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:44.721 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:49:42.039882+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:49:38.349338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-19T07:49:39.377498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:49:44.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.722+0000 7f4c9d5c6640 1 -- 172.21.15.16:0/4059511758 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4c7806c2e0 msgr2=0x7f4c7806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.722+0000 7f4c9d5c6640 1 --2- 172.21.15.16:0/4059511758 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4c7806c2e0 0x7f4c7806e7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4c900016e0 tx=0x7f4c90009040 comp rx=0 tx=0).stop 2024-09-19T07:49:44.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.722+0000 7f4c9d5c6640 1 -- 172.21.15.16:0/4059511758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980d4130 msgr2=0x7f4c9814abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.722+0000 7f4c9d5c6640 1 --2- 172.21.15.16:0/4059511758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980d4130 0x7f4c9814abe0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f4c80000c00 tx=0x7f4c800046c0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.722+0000 7f4c9d5c6640 1 -- 172.21.15.16:0/4059511758 shutdown_connections 2024-09-19T07:49:44.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.722+0000 7f4c9d5c6640 1 --2- 172.21.15.16:0/4059511758 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4c7806c2e0 0x7f4c7806e7a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.722+0000 7f4c9d5c6640 1 --2- 172.21.15.16:0/4059511758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c9815f0d0 0x7f4c9815b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.722+0000 7f4c9d5c6640 1 --2- 172.21.15.16:0/4059511758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c980d4130 0x7f4c9814abe0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.722+0000 7f4c9d5c6640 1 --2- 172.21.15.16:0/4059511758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c980d0070 0x7f4c9814a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.722+0000 7f4c9d5c6640 1 -- 172.21.15.16:0/4059511758 >> 172.21.15.16:0/4059511758 conn(0x7f4c9809d760 msgr2=0x7f4c980b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:44.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.722+0000 7f4c9d5c6640 1 -- 172.21.15.16:0/4059511758 shutdown_connections 2024-09-19T07:49:44.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.722+0000 7f4c9d5c6640 1 -- 172.21.15.16:0/4059511758 wait complete. 2024-09-19T07:49:44.734 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T07:49:44.735 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:49:44.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.835+0000 7f6cc345e640 1 Processor -- start 2024-09-19T07:49:44.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.835+0000 7f6cc345e640 1 -- start start 2024-09-19T07:49:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.836+0000 7f6cc345e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6cbc15f0d0 0x7f6cbc15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.836+0000 7f6cc345e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cbc0d1b10 con 0x7f6cbc0d0070 2024-09-19T07:49:44.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.836+0000 7f6cc345e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cbc0cf2b0 con 0x7f6cbc0d4130 2024-09-19T07:49:44.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.836+0000 7f6cc345e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f6cbc0cf590 con 0x7f6cbc15f0d0 2024-09-19T07:49:44.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.836+0000 7f6cc09f9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6cbc0d0070 0x7f6cbc0d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36080/0 (socket says 172.21.15.16:36080) 2024-09-19T07:49:44.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.836+0000 7f6cc09f9640 1 -- 172.21.15.16:0/3351879820 learned_addr learned my addr 172.21.15.16:0/3351879820 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:44.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.837+0000 7f6cc11fa640 1 --2- 172.21.15.16:0/3351879820 >> v2:172.21.15.146:3300/0 conn(0x7f6cbc15f0d0 0x7f6cbc15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.837+0000 7f6cb3fff640 1 -- 172.21.15.16:0/3351879820 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3082950364 0 0) 0x7f6cbc0d1b10 con 0x7f6cbc0d0070 2024-09-19T07:49:44.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.837+0000 7f6cb3fff640 1 -- 172.21.15.16:0/3351879820 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ca8003520 con 0x7f6cbc0d0070 2024-09-19T07:49:44.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.837+0000 7f6cc11fa640 1 -- 172.21.15.16:0/3351879820 >> v1:172.21.15.146:6789/0 conn(0x7f6cbc0d4130 legacy=0x7f6cbc0d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.837+0000 7f6cb3fff640 1 -- 172.21.15.16:0/3351879820 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2122980250 0 0) 0x7f6ca8003520 con 0x7f6cbc0d0070 2024-09-19T07:49:44.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.837+0000 7f6cc11fa640 1 -- 172.21.15.16:0/3351879820 >> v1:172.21.15.16:6789/0 conn(0x7f6cbc0d0070 legacy=0x7f6cbc0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.837+0000 7f6cc11fa640 1 -- 172.21.15.16:0/3351879820 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc0cf870 con 0x7f6cbc15f0d0 2024-09-19T07:49:44.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.838+0000 7f6cc11fa640 1 --2- 172.21.15.16:0/3351879820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cbc15f0d0 0x7f6cbc15f4b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80097a0 tx=0x7f6cb8029fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d59f32ca3724c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:44.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.838+0000 7f6cb3fff640 1 -- 172.21.15.16:0/3351879820 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8037070 con 0x7f6cbc15f0d0 2024-09-19T07:49:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.838+0000 7f6cb3fff640 1 -- 172.21.15.16:0/3351879820 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb802aa40 con 0x7f6cbc15f0d0 2024-09-19T07:49:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.838+0000 7f6cb3fff640 1 -- 172.21.15.16:0/3351879820 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac001690 con 0x7f6cbc15f0d0 2024-09-19T07:49:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.840+0000 7f6cc345e640 1 -- 172.21.15.16:0/3351879820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cbc15f0d0 msgr2=0x7f6cbc15f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.840+0000 7f6cc345e640 1 --2- 172.21.15.16:0/3351879820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cbc15f0d0 0x7f6cbc15f4b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80097a0 tx=0x7f6cb8029fb0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.840+0000 7f6cc345e640 1 -- 172.21.15.16:0/3351879820 shutdown_connections 2024-09-19T07:49:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.840+0000 7f6cc345e640 1 --2- 172.21.15.16:0/3351879820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cbc15f0d0 0x7f6cbc15f4b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.840+0000 7f6cc345e640 1 -- 172.21.15.16:0/3351879820 >> 172.21.15.16:0/3351879820 conn(0x7f6cbc09d760 msgr2=0x7f6cbc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.840+0000 7f6cc345e640 1 -- 172.21.15.16:0/3351879820 shutdown_connections 2024-09-19T07:49:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.840+0000 7f6cc345e640 1 -- 172.21.15.16:0/3351879820 wait complete. 2024-09-19T07:49:44.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.840+0000 7f6cc345e640 1 Processor -- start 2024-09-19T07:49:44.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.840+0000 7f6cc345e640 1 -- start start 2024-09-19T07:49:44.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cc345e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6cbc0d0070 0x7f6cbc0b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cc345e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cbc0d4130 0x7f6cbc14f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cc345e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cbc15f0d0 0x7f6cbc0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cc345e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc0b2f70 con 0x7f6cbc15f0d0 2024-09-19T07:49:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cc345e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc0cef60 con 0x7f6cbc0d4130 2024-09-19T07:49:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cc345e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6cbc0b30f0 con 0x7f6cbc0d0070 2024-09-19T07:49:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cc19fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cbc15f0d0 0x7f6cbc0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cc19fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cbc15f0d0 0x7f6cbc0cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60876/0 (socket says 172.21.15.16:60876) 2024-09-19T07:49:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cc19fb640 1 -- 172.21.15.16:0/655599716 learned_addr learned my addr 172.21.15.16:0/655599716 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cc11fa640 1 --2- 172.21.15.16:0/655599716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6cbc0d0070 0x7f6cbc0b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cc09f9640 1 --2- 172.21.15.16:0/655599716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cbc0d4130 0x7f6cbc14f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cc19fb640 1 -- 172.21.15.16:0/655599716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6cbc0d0070 msgr2=0x7f6cbc0b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cc19fb640 1 --2- 172.21.15.16:0/655599716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6cbc0d0070 0x7f6cbc0b5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cc19fb640 1 -- 172.21.15.16:0/655599716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cbc0d4130 msgr2=0x7f6cbc14f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cc19fb640 1 --2- 172.21.15.16:0/655599716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cbc0d4130 0x7f6cbc14f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cc19fb640 1 -- 172.21.15.16:0/655599716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc0b4560 con 0x7f6cbc15f0d0 2024-09-19T07:49:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cc19fb640 1 --2- 172.21.15.16:0/655599716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cbc15f0d0 0x7f6cbc0cd9e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400cb20 tx=0x7f6cb4002090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cb1ffb640 1 -- 172.21.15.16:0/655599716 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4018070 con 0x7f6cbc15f0d0 2024-09-19T07:49:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cb1ffb640 1 -- 172.21.15.16:0/655599716 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb4003da0 con 0x7f6cbc15f0d0 2024-09-19T07:49:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.841+0000 7f6cc345e640 1 -- 172.21.15.16:0/655599716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cbc0b4840 con 0x7f6cbc15f0d0 2024-09-19T07:49:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.842+0000 7f6cb1ffb640 1 -- 172.21.15.16:0/655599716 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4006d70 con 0x7f6cbc15f0d0 2024-09-19T07:49:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.842+0000 7f6cc345e640 1 -- 172.21.15.16:0/655599716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cbc0b4e00 con 0x7f6cbc15f0d0 2024-09-19T07:49:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.843+0000 7f6cc345e640 1 -- 172.21.15.16:0/655599716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cbc15f4b0 con 0x7f6cbc15f0d0 2024-09-19T07:49:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.843+0000 7f6cb1ffb640 1 -- 172.21.15.16:0/655599716 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6cb401b070 con 0x7f6cbc15f0d0 2024-09-19T07:49:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.844+0000 7f6cb1ffb640 1 --2- 172.21.15.16:0/655599716 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6ca006c2b0 0x7f6ca006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.844+0000 7f6cb1ffb640 1 -- 172.21.15.16:0/655599716 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f6cb408acc0 con 0x7f6cbc15f0d0 2024-09-19T07:49:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.844+0000 7f6cc11fa640 1 --2- 172.21.15.16:0/655599716 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6ca006c2b0 0x7f6ca006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.845+0000 7f6cc11fa640 1 --2- 172.21.15.16:0/655599716 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6ca006c2b0 0x7f6ca006e770 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8034000 tx=0x7f6cb802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:44.848+0000 7f6cb1ffb640 1 -- 172.21.15.16:0/655599716 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6cb40592a0 con 0x7f6cbc15f0d0 2024-09-19T07:49:45.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.020+0000 7f6cc345e640 1 -- 172.21.15.16:0/655599716 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6cbc06cf30 con 0x7f6ca006c2b0 2024-09-19T07:49:45.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.027+0000 7f6cb1ffb640 1 -- 172.21.15.16:0/655599716 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252077 (secure 0 0 0) 0x7f6cbc06cf30 con 0x7f6ca006c2b0 2024-09-19T07:49:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:45.033 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:49:45.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.030+0000 7f6cc345e640 1 -- 172.21.15.16:0/655599716 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6ca006c2b0 msgr2=0x7f6ca006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:45.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.030+0000 7f6cc345e640 1 --2- 172.21.15.16:0/655599716 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6ca006c2b0 0x7f6ca006e770 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8034000 tx=0x7f6cb802e000 comp rx=0 tx=0).stop 2024-09-19T07:49:45.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.030+0000 7f6cc345e640 1 -- 172.21.15.16:0/655599716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cbc15f0d0 msgr2=0x7f6cbc0cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:45.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.030+0000 7f6cc345e640 1 --2- 172.21.15.16:0/655599716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cbc15f0d0 0x7f6cbc0cd9e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400cb20 tx=0x7f6cb4002090 comp rx=0 tx=0).stop 2024-09-19T07:49:45.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.030+0000 7f6cc345e640 1 -- 172.21.15.16:0/655599716 shutdown_connections 2024-09-19T07:49:45.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.030+0000 7f6cc345e640 1 --2- 172.21.15.16:0/655599716 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6ca006c2b0 0x7f6ca006e770 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.030+0000 7f6cc345e640 1 --2- 172.21.15.16:0/655599716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cbc15f0d0 0x7f6cbc0cd9e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.030+0000 7f6cc345e640 1 --2- 172.21.15.16:0/655599716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cbc0d4130 0x7f6cbc14f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.030+0000 7f6cc345e640 1 --2- 172.21.15.16:0/655599716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6cbc0d0070 0x7f6cbc0b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.030+0000 7f6cc345e640 1 -- 172.21.15.16:0/655599716 >> 172.21.15.16:0/655599716 conn(0x7f6cbc09d760 msgr2=0x7f6cbc161eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:45.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.030+0000 7f6cc345e640 1 -- 172.21.15.16:0/655599716 shutdown_connections 2024-09-19T07:49:45.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.030+0000 7f6cc345e640 1 -- 172.21.15.16:0/655599716 wait complete. 2024-09-19T07:49:45.042 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-09-19T07:49:44.547732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":738480,"kb_used_data":8880,"kb_used_omap":12,"kb_used_meta":729395,"kb_avail":754236240,"statfs":{"total":773094113280,"available":772337909760,"internally_reserved":0,"allocated":9093120,"data_stored":4778356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":746901040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.526570"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824591+0000","last_change":"2024-09-19T07:49:41.055660+0000","last_active":"2024-09-19T07:49:42.824591+0000","last_peered":"2024-09-19T07:49:42.824591+0000","last_clean":"2024-09-19T07:49:42.824591+0000","last_became_active":"2024-09-19T07:49:41.055401+0000","last_became_peered":"2024-09-19T07:49:41.055401+0000","last_unstale":"2024-09-19T07:49:42.824591+0000","last_undegraded":"2024-09-19T07:49:42.824591+0000","last_fullsized":"2024-09-19T07:49:42.824591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:57.866953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697358+0000","last_change":"2024-09-19T07:49:41.065827+0000","last_active":"2024-09-19T07:49:42.697358+0000","last_peered":"2024-09-19T07:49:42.697358+0000","last_clean":"2024-09-19T07:49:42.697358+0000","last_became_active":"2024-09-19T07:49:41.064467+0000","last_became_peered":"2024-09-19T07:49:41.064467+0000","last_unstale":"2024-09-19T07:49:42.697358+0000","last_undegraded":"2024-09-19T07:49:42.697358+0000","last_fullsized":"2024-09-19T07:49:42.697358+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:26:38.483340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.043081+0000","last_change":"2024-09-19T07:49:41.061515+0000","last_active":"2024-09-19T07:49:42.043081+0000","last_peered":"2024-09-19T07:49:42.043081+0000","last_clean":"2024-09-19T07:49:42.043081+0000","last_became_active":"2024-09-19T07:49:41.061216+0000","last_became_peered":"2024-09-19T07:49:41.061216+0000","last_unstale":"2024-09-19T07:49:42.043081+0000","last_undegraded":"2024-09-19T07:49:42.043081+0000","last_fullsized":"2024-09-19T07:49:42.043081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:18.521095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044348+0000","last_change":"2024-09-19T07:49:41.048005+0000","last_active":"2024-09-19T07:49:42.044348+0000","last_peered":"2024-09-19T07:49:42.044348+0000","last_clean":"2024-09-19T07:49:42.044348+0000","last_became_active":"2024-09-19T07:49:41.047728+0000","last_became_peered":"2024-09-19T07:49:41.047728+0000","last_unstale":"2024-09-19T07:49:42.044348+0000","last_undegraded":"2024-09-19T07:49:42.044348+0000","last_fullsized":"2024-09-19T07:49:42.044348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:37.745819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.049521+0000","last_change":"2024-09-19T07:49:41.045867+0000","last_active":"2024-09-19T07:49:41.049521+0000","last_peered":"2024-09-19T07:49:41.049521+0000","last_clean":"2024-09-19T07:49:41.049521+0000","last_became_active":"2024-09-19T07:49:41.045562+0000","last_became_peered":"2024-09-19T07:49:41.045562+0000","last_unstale":"2024-09-19T07:49:41.049521+0000","last_undegraded":"2024-09-19T07:49:41.049521+0000","last_fullsized":"2024-09-19T07:49:41.049521+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:18.302335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031751+0000","last_change":"2024-09-19T07:49:41.043246+0000","last_active":"2024-09-19T07:49:43.031751+0000","last_peered":"2024-09-19T07:49:43.031751+0000","last_clean":"2024-09-19T07:49:43.031751+0000","last_became_active":"2024-09-19T07:49:41.042651+0000","last_became_peered":"2024-09-19T07:49:41.042651+0000","last_unstale":"2024-09-19T07:49:43.031751+0000","last_undegraded":"2024-09-19T07:49:43.031751+0000","last_fullsized":"2024-09-19T07:49:43.031751+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:21.000170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031054+0000","last_change":"2024-09-19T07:49:41.058873+0000","last_active":"2024-09-19T07:49:43.031054+0000","last_peered":"2024-09-19T07:49:43.031054+0000","last_clean":"2024-09-19T07:49:43.031054+0000","last_became_active":"2024-09-19T07:49:41.058564+0000","last_became_peered":"2024-09-19T07:49:41.058564+0000","last_unstale":"2024-09-19T07:49:43.031054+0000","last_undegraded":"2024-09-19T07:49:43.031054+0000","last_fullsized":"2024-09-19T07:49:43.031054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:11.306569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.053306+0000","last_change":"2024-09-19T07:49:41.046577+0000","last_active":"2024-09-19T07:49:41.053306+0000","last_peered":"2024-09-19T07:49:41.053306+0000","last_clean":"2024-09-19T07:49:41.053306+0000","last_became_active":"2024-09-19T07:49:41.046302+0000","last_became_peered":"2024-09-19T07:49:41.046302+0000","last_unstale":"2024-09-19T07:49:41.053306+0000","last_undegraded":"2024-09-19T07:49:41.053306+0000","last_fullsized":"2024-09-19T07:49:41.053306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:24.122562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044172+0000","last_change":"2024-09-19T07:49:41.048380+0000","last_active":"2024-09-19T07:49:42.044172+0000","last_peered":"2024-09-19T07:49:42.044172+0000","last_clean":"2024-09-19T07:49:42.044172+0000","last_became_active":"2024-09-19T07:49:41.048063+0000","last_became_peered":"2024-09-19T07:49:41.048063+0000","last_unstale":"2024-09-19T07:49:42.044172+0000","last_undegraded":"2024-09-19T07:49:42.044172+0000","last_fullsized":"2024-09-19T07:49:42.044172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:03.100435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044256+0000","last_change":"2024-09-19T07:49:41.050046+0000","last_active":"2024-09-19T07:49:42.044256+0000","last_peered":"2024-09-19T07:49:42.044256+0000","last_clean":"2024-09-19T07:49:42.044256+0000","last_became_active":"2024-09-19T07:49:41.049748+0000","last_became_peered":"2024-09-19T07:49:41.049748+0000","last_unstale":"2024-09-19T07:49:42.044256+0000","last_undegraded":"2024-09-19T07:49:42.044256+0000","last_fullsized":"2024-09-19T07:49:42.044256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:00.517111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697310+0000","last_change":"2024-09-19T07:49:41.059816+0000","last_active":"2024-09-19T07:49:42.697310+0000","last_peered":"2024-09-19T07:49:42.697310+0000","last_clean":"2024-09-19T07:49:42.697310+0000","last_became_active":"2024-09-19T07:49:41.058730+0000","last_became_peered":"2024-09-19T07:49:41.058730+0000","last_unstale":"2024-09-19T07:49:42.697310+0000","last_undegraded":"2024-09-19T07:49:42.697310+0000","last_fullsized":"2024-09-19T07:49:42.697310+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:57.866953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044472+0000","last_change":"2024-09-19T07:49:41.063887+0000","last_active":"2024-09-19T07:49:42.044472+0000","last_peered":"2024-09-19T07:49:42.044472+0000","last_clean":"2024-09-19T07:49:42.044472+0000","last_became_active":"2024-09-19T07:49:41.063602+0000","last_became_peered":"2024-09-19T07:49:41.063602+0000","last_unstale":"2024-09-19T07:49:42.044472+0000","last_undegraded":"2024-09-19T07:49:42.044472+0000","last_fullsized":"2024-09-19T07:49:42.044472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:23.597186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.044463+0000","last_change":"2024-09-19T07:49:41.042785+0000","last_active":"2024-09-19T07:49:41.044463+0000","last_peered":"2024-09-19T07:49:41.044463+0000","last_clean":"2024-09-19T07:49:41.044463+0000","last_became_active":"2024-09-19T07:49:41.042461+0000","last_became_peered":"2024-09-19T07:49:41.042461+0000","last_unstale":"2024-09-19T07:49:41.044463+0000","last_undegraded":"2024-09-19T07:49:41.044463+0000","last_fullsized":"2024-09-19T07:49:41.044463+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:52.492481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031842+0000","last_change":"2024-09-19T07:49:41.043227+0000","last_active":"2024-09-19T07:49:43.031842+0000","last_peered":"2024-09-19T07:49:43.031842+0000","last_clean":"2024-09-19T07:49:43.031842+0000","last_became_active":"2024-09-19T07:49:41.042613+0000","last_became_peered":"2024-09-19T07:49:41.042613+0000","last_unstale":"2024-09-19T07:49:43.031842+0000","last_undegraded":"2024-09-19T07:49:43.031842+0000","last_fullsized":"2024-09-19T07:49:43.031842+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:18.290122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.696880+0000","last_change":"2024-09-19T07:49:41.059455+0000","last_active":"2024-09-19T07:49:42.696880+0000","last_peered":"2024-09-19T07:49:42.696880+0000","last_clean":"2024-09-19T07:49:42.696880+0000","last_became_active":"2024-09-19T07:49:41.056688+0000","last_became_peered":"2024-09-19T07:49:41.056688+0000","last_unstale":"2024-09-19T07:49:42.696880+0000","last_undegraded":"2024-09-19T07:49:42.696880+0000","last_fullsized":"2024-09-19T07:49:42.696880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.574881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824275+0000","last_change":"2024-09-19T07:49:41.052008+0000","last_active":"2024-09-19T07:49:42.824275+0000","last_peered":"2024-09-19T07:49:42.824275+0000","last_clean":"2024-09-19T07:49:42.824275+0000","last_became_active":"2024-09-19T07:49:41.051736+0000","last_became_peered":"2024-09-19T07:49:41.051736+0000","last_unstale":"2024-09-19T07:49:42.824275+0000","last_undegraded":"2024-09-19T07:49:42.824275+0000","last_fullsized":"2024-09-19T07:49:42.824275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:11.504530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.053425+0000","last_change":"2024-09-19T07:49:41.050945+0000","last_active":"2024-09-19T07:49:41.053425+0000","last_peered":"2024-09-19T07:49:41.053425+0000","last_clean":"2024-09-19T07:49:41.053425+0000","last_became_active":"2024-09-19T07:49:41.050582+0000","last_became_peered":"2024-09-19T07:49:41.050582+0000","last_unstale":"2024-09-19T07:49:41.053425+0000","last_undegraded":"2024-09-19T07:49:41.053425+0000","last_fullsized":"2024-09-19T07:49:41.053425+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:08.228920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.042978+0000","last_change":"2024-09-19T07:49:41.047145+0000","last_active":"2024-09-19T07:49:42.042978+0000","last_peered":"2024-09-19T07:49:42.042978+0000","last_clean":"2024-09-19T07:49:42.042978+0000","last_became_active":"2024-09-19T07:49:41.046856+0000","last_became_peered":"2024-09-19T07:49:41.046856+0000","last_unstale":"2024-09-19T07:49:42.042978+0000","last_undegraded":"2024-09-19T07:49:42.042978+0000","last_fullsized":"2024-09-19T07:49:42.042978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:29.520999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031420+0000","last_change":"2024-09-19T07:49:41.040335+0000","last_active":"2024-09-19T07:49:43.031420+0000","last_peered":"2024-09-19T07:49:43.031420+0000","last_clean":"2024-09-19T07:49:43.031420+0000","last_became_active":"2024-09-19T07:49:41.040005+0000","last_became_peered":"2024-09-19T07:49:41.040005+0000","last_unstale":"2024-09-19T07:49:43.031420+0000","last_undegraded":"2024-09-19T07:49:43.031420+0000","last_fullsized":"2024-09-19T07:49:43.031420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:06.484754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.045010+0000","last_change":"2024-09-19T07:49:41.043861+0000","last_active":"2024-09-19T07:49:41.045010+0000","last_peered":"2024-09-19T07:49:41.045010+0000","last_clean":"2024-09-19T07:49:41.045010+0000","last_became_active":"2024-09-19T07:49:41.043556+0000","last_became_peered":"2024-09-19T07:49:41.043556+0000","last_unstale":"2024-09-19T07:49:41.045010+0000","last_undegraded":"2024-09-19T07:49:41.045010+0000","last_fullsized":"2024-09-19T07:49:41.045010+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:58:22.896865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044704+0000","last_change":"2024-09-19T07:49:41.057841+0000","last_active":"2024-09-19T07:49:42.044704+0000","last_peered":"2024-09-19T07:49:42.044704+0000","last_clean":"2024-09-19T07:49:42.044704+0000","last_became_active":"2024-09-19T07:49:41.057559+0000","last_became_peered":"2024-09-19T07:49:41.057559+0000","last_unstale":"2024-09-19T07:49:42.044704+0000","last_undegraded":"2024-09-19T07:49:42.044704+0000","last_fullsized":"2024-09-19T07:49:42.044704+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:39.777331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.046639+0000","last_change":"2024-09-19T07:49:41.045056+0000","last_active":"2024-09-19T07:49:41.046639+0000","last_peered":"2024-09-19T07:49:41.046639+0000","last_clean":"2024-09-19T07:49:41.046639+0000","last_became_active":"2024-09-19T07:49:41.044546+0000","last_became_peered":"2024-09-19T07:49:41.044546+0000","last_unstale":"2024-09-19T07:49:41.046639+0000","last_undegraded":"2024-09-19T07:49:41.046639+0000","last_fullsized":"2024-09-19T07:49:41.046639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:12.145979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044362+0000","last_change":"2024-09-19T07:49:41.047175+0000","last_active":"2024-09-19T07:49:42.044362+0000","last_peered":"2024-09-19T07:49:42.044362+0000","last_clean":"2024-09-19T07:49:42.044362+0000","last_became_active":"2024-09-19T07:49:41.046748+0000","last_became_peered":"2024-09-19T07:49:41.046748+0000","last_unstale":"2024-09-19T07:49:42.044362+0000","last_undegraded":"2024-09-19T07:49:42.044362+0000","last_fullsized":"2024-09-19T07:49:42.044362+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:57.849422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.696809+0000","last_change":"2024-09-19T07:49:41.056104+0000","last_active":"2024-09-19T07:49:42.696809+0000","last_peered":"2024-09-19T07:49:42.696809+0000","last_clean":"2024-09-19T07:49:42.696809+0000","last_became_active":"2024-09-19T07:49:41.055815+0000","last_became_peered":"2024-09-19T07:49:41.055815+0000","last_unstale":"2024-09-19T07:49:42.696809+0000","last_undegraded":"2024-09-19T07:49:42.696809+0000","last_fullsized":"2024-09-19T07:49:42.696809+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:41.384540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697238+0000","last_change":"2024-09-19T07:49:41.056003+0000","last_active":"2024-09-19T07:49:42.697238+0000","last_peered":"2024-09-19T07:49:42.697238+0000","last_clean":"2024-09-19T07:49:42.697238+0000","last_became_active":"2024-09-19T07:49:41.055680+0000","last_became_peered":"2024-09-19T07:49:41.055680+0000","last_unstale":"2024-09-19T07:49:42.697238+0000","last_undegraded":"2024-09-19T07:49:42.697238+0000","last_fullsized":"2024-09-19T07:49:42.697238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:55.331851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044459+0000","last_change":"2024-09-19T07:49:41.062788+0000","last_active":"2024-09-19T07:49:42.044459+0000","last_peered":"2024-09-19T07:49:42.044459+0000","last_clean":"2024-09-19T07:49:42.044459+0000","last_became_active":"2024-09-19T07:49:41.062477+0000","last_became_peered":"2024-09-19T07:49:41.062477+0000","last_unstale":"2024-09-19T07:49:42.044459+0000","last_undegraded":"2024-09-19T07:49:42.044459+0000","last_fullsized":"2024-09-19T07:49:42.044459+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:41.384540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824381+0000","last_change":"2024-09-19T07:49:41.050573+0000","last_active":"2024-09-19T07:49:42.824381+0000","last_peered":"2024-09-19T07:49:42.824381+0000","last_clean":"2024-09-19T07:49:42.824381+0000","last_became_active":"2024-09-19T07:49:41.050277+0000","last_became_peered":"2024-09-19T07:49:41.050277+0000","last_unstale":"2024-09-19T07:49:42.824381+0000","last_undegraded":"2024-09-19T07:49:42.824381+0000","last_fullsized":"2024-09-19T07:49:42.824381+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:45.208388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.053494+0000","last_change":"2024-09-19T07:49:41.049862+0000","last_active":"2024-09-19T07:49:41.053494+0000","last_peered":"2024-09-19T07:49:41.053494+0000","last_clean":"2024-09-19T07:49:41.053494+0000","last_became_active":"2024-09-19T07:49:41.047151+0000","last_became_peered":"2024-09-19T07:49:41.047151+0000","last_unstale":"2024-09-19T07:49:41.053494+0000","last_undegraded":"2024-09-19T07:49:41.053494+0000","last_fullsized":"2024-09-19T07:49:41.053494+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:21.022897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697050+0000","last_change":"2024-09-19T07:49:41.067158+0000","last_active":"2024-09-19T07:49:42.697050+0000","last_peered":"2024-09-19T07:49:42.697050+0000","last_clean":"2024-09-19T07:49:42.697050+0000","last_became_active":"2024-09-19T07:49:41.066475+0000","last_became_peered":"2024-09-19T07:49:41.066475+0000","last_unstale":"2024-09-19T07:49:42.697050+0000","last_undegraded":"2024-09-19T07:49:42.697050+0000","last_fullsized":"2024-09-19T07:49:42.697050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:11.504530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824345+0000","last_change":"2024-09-19T07:49:41.067574+0000","last_active":"2024-09-19T07:49:42.824345+0000","last_peered":"2024-09-19T07:49:42.824345+0000","last_clean":"2024-09-19T07:49:42.824345+0000","last_became_active":"2024-09-19T07:49:41.067305+0000","last_became_peered":"2024-09-19T07:49:41.067305+0000","last_unstale":"2024-09-19T07:49:42.824345+0000","last_undegraded":"2024-09-19T07:49:42.824345+0000","last_fullsized":"2024-09-19T07:49:42.824345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:55.331851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.042974+0000","last_change":"2024-09-19T07:49:41.052142+0000","last_active":"2024-09-19T07:49:42.042974+0000","last_peered":"2024-09-19T07:49:42.042974+0000","last_clean":"2024-09-19T07:49:42.042974+0000","last_became_active":"2024-09-19T07:49:41.051817+0000","last_became_peered":"2024-09-19T07:49:41.051817+0000","last_unstale":"2024-09-19T07:49:42.042974+0000","last_undegraded":"2024-09-19T07:49:42.042974+0000","last_fullsized":"2024-09-19T07:49:42.042974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:25.748958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824297+0000","last_change":"2024-09-19T07:49:41.059595+0000","last_active":"2024-09-19T07:49:42.824297+0000","last_peered":"2024-09-19T07:49:42.824297+0000","last_clean":"2024-09-19T07:49:42.824297+0000","last_became_active":"2024-09-19T07:49:41.058118+0000","last_became_peered":"2024-09-19T07:49:41.058118+0000","last_unstale":"2024-09-19T07:49:42.824297+0000","last_undegraded":"2024-09-19T07:49:42.824297+0000","last_fullsized":"2024-09-19T07:49:42.824297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:40:04.453626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.042975+0000","last_change":"2024-09-19T07:49:41.046439+0000","last_active":"2024-09-19T07:49:42.042975+0000","last_peered":"2024-09-19T07:49:42.042975+0000","last_clean":"2024-09-19T07:49:42.042975+0000","last_became_active":"2024-09-19T07:49:41.046121+0000","last_became_peered":"2024-09-19T07:49:41.046121+0000","last_unstale":"2024-09-19T07:49:42.042975+0000","last_undegraded":"2024-09-19T07:49:42.042975+0000","last_fullsized":"2024-09-19T07:49:42.042975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:33.713515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.065291+0000","last_change":"2024-09-19T07:49:40.044943+0000","last_active":"2024-09-19T07:49:41.065291+0000","last_peered":"2024-09-19T07:49:41.065291+0000","last_clean":"2024-09-19T07:49:41.065291+0000","last_became_active":"2024-09-19T07:49:40.044658+0000","last_became_peered":"2024-09-19T07:49:40.044658+0000","last_unstale":"2024-09-19T07:49:41.065291+0000","last_undegraded":"2024-09-19T07:49:41.065291+0000","last_fullsized":"2024-09-19T07:49:41.065291+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:16.643861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.696880+0000","last_change":"2024-09-19T07:49:41.056987+0000","last_active":"2024-09-19T07:49:42.696880+0000","last_peered":"2024-09-19T07:49:42.696880+0000","last_clean":"2024-09-19T07:49:42.696880+0000","last_became_active":"2024-09-19T07:49:41.056334+0000","last_became_peered":"2024-09-19T07:49:41.056334+0000","last_unstale":"2024-09-19T07:49:42.696880+0000","last_undegraded":"2024-09-19T07:49:42.696880+0000","last_fullsized":"2024-09-19T07:49:42.696880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:23.597186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824466+0000","last_change":"2024-09-19T07:49:40.040869+0000","last_active":"2024-09-19T07:49:42.824466+0000","last_peered":"2024-09-19T07:49:42.824466+0000","last_clean":"2024-09-19T07:49:42.824466+0000","last_became_active":"2024-09-19T07:49:40.040288+0000","last_became_peered":"2024-09-19T07:49:40.040288+0000","last_unstale":"2024-09-19T07:49:42.824466+0000","last_undegraded":"2024-09-19T07:49:42.824466+0000","last_fullsized":"2024-09-19T07:49:42.824466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:06.733002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044371+0000","last_change":"2024-09-19T07:49:41.050647+0000","last_active":"2024-09-19T07:49:42.044371+0000","last_peered":"2024-09-19T07:49:42.044371+0000","last_clean":"2024-09-19T07:49:42.044371+0000","last_became_active":"2024-09-19T07:49:41.050381+0000","last_became_peered":"2024-09-19T07:49:41.050381+0000","last_unstale":"2024-09-19T07:49:42.044371+0000","last_undegraded":"2024-09-19T07:49:42.044371+0000","last_fullsized":"2024-09-19T07:49:42.044371+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:10.140351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824359+0000","last_change":"2024-09-19T07:49:40.044368+0000","last_active":"2024-09-19T07:49:42.824359+0000","last_peered":"2024-09-19T07:49:42.824359+0000","last_clean":"2024-09-19T07:49:42.824359+0000","last_became_active":"2024-09-19T07:49:40.044097+0000","last_became_peered":"2024-09-19T07:49:40.044097+0000","last_unstale":"2024-09-19T07:49:42.824359+0000","last_undegraded":"2024-09-19T07:49:42.824359+0000","last_fullsized":"2024-09-19T07:49:42.824359+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:52.931623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044900+0000","last_change":"2024-09-19T07:49:41.058801+0000","last_active":"2024-09-19T07:49:42.044900+0000","last_peered":"2024-09-19T07:49:42.044900+0000","last_clean":"2024-09-19T07:49:42.044900+0000","last_became_active":"2024-09-19T07:49:41.057967+0000","last_became_peered":"2024-09-19T07:49:41.057967+0000","last_unstale":"2024-09-19T07:49:42.044900+0000","last_undegraded":"2024-09-19T07:49:42.044900+0000","last_fullsized":"2024-09-19T07:49:42.044900+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:13.724199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.030964+0000","last_change":"2024-09-19T07:49:40.036859+0000","last_active":"2024-09-19T07:49:43.030964+0000","last_peered":"2024-09-19T07:49:43.030964+0000","last_clean":"2024-09-19T07:49:43.030964+0000","last_became_active":"2024-09-19T07:49:40.036562+0000","last_became_peered":"2024-09-19T07:49:40.036562+0000","last_unstale":"2024-09-19T07:49:43.030964+0000","last_undegraded":"2024-09-19T07:49:43.030964+0000","last_fullsized":"2024-09-19T07:49:43.030964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:22.957733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697063+0000","last_change":"2024-09-19T07:49:41.064382+0000","last_active":"2024-09-19T07:49:42.697063+0000","last_peered":"2024-09-19T07:49:42.697063+0000","last_clean":"2024-09-19T07:49:42.697063+0000","last_became_active":"2024-09-19T07:49:41.063213+0000","last_became_peered":"2024-09-19T07:49:41.063213+0000","last_unstale":"2024-09-19T07:49:42.697063+0000","last_undegraded":"2024-09-19T07:49:42.697063+0000","last_fullsized":"2024-09-19T07:49:42.697063+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:40:04.453626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044934+0000","last_change":"2024-09-19T07:49:40.036204+0000","last_active":"2024-09-19T07:49:42.044934+0000","last_peered":"2024-09-19T07:49:42.044934+0000","last_clean":"2024-09-19T07:49:42.044934+0000","last_became_active":"2024-09-19T07:49:40.035901+0000","last_became_peered":"2024-09-19T07:49:40.035901+0000","last_unstale":"2024-09-19T07:49:42.044934+0000","last_undegraded":"2024-09-19T07:49:42.044934+0000","last_fullsized":"2024-09-19T07:49:42.044934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:53:05.590001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697349+0000","last_change":"2024-09-19T07:49:41.063471+0000","last_active":"2024-09-19T07:49:42.697349+0000","last_peered":"2024-09-19T07:49:42.697349+0000","last_clean":"2024-09-19T07:49:42.697349+0000","last_became_active":"2024-09-19T07:49:41.063182+0000","last_became_peered":"2024-09-19T07:49:41.063182+0000","last_unstale":"2024-09-19T07:49:42.697349+0000","last_undegraded":"2024-09-19T07:49:42.697349+0000","last_fullsized":"2024-09-19T07:49:42.697349+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:20.443319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.043998+0000","last_change":"2024-09-19T07:49:40.042284+0000","last_active":"2024-09-19T07:49:42.043998+0000","last_peered":"2024-09-19T07:49:42.043998+0000","last_clean":"2024-09-19T07:49:42.043998+0000","last_became_active":"2024-09-19T07:49:40.041940+0000","last_became_peered":"2024-09-19T07:49:40.041940+0000","last_unstale":"2024-09-19T07:49:42.043998+0000","last_undegraded":"2024-09-19T07:49:42.043998+0000","last_fullsized":"2024-09-19T07:49:42.043998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:50.820893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044943+0000","last_change":"2024-09-19T07:49:41.043055+0000","last_active":"2024-09-19T07:49:42.044943+0000","last_peered":"2024-09-19T07:49:42.044943+0000","last_clean":"2024-09-19T07:49:42.044943+0000","last_became_active":"2024-09-19T07:49:41.042761+0000","last_became_peered":"2024-09-19T07:49:41.042761+0000","last_unstale":"2024-09-19T07:49:42.044943+0000","last_undegraded":"2024-09-19T07:49:42.044943+0000","last_fullsized":"2024-09-19T07:49:42.044943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:03:42.763827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.696995+0000","last_change":"2024-09-19T07:49:40.040534+0000","last_active":"2024-09-19T07:49:42.696995+0000","last_peered":"2024-09-19T07:49:42.696995+0000","last_clean":"2024-09-19T07:49:42.696995+0000","last_became_active":"2024-09-19T07:49:40.040264+0000","last_became_peered":"2024-09-19T07:49:40.040264+0000","last_unstale":"2024-09-19T07:49:42.696995+0000","last_undegraded":"2024-09-19T07:49:42.696995+0000","last_fullsized":"2024-09-19T07:49:42.696995+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:15.151743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031287+0000","last_change":"2024-09-19T07:49:41.050953+0000","last_active":"2024-09-19T07:49:43.031287+0000","last_peered":"2024-09-19T07:49:43.031287+0000","last_clean":"2024-09-19T07:49:43.031287+0000","last_became_active":"2024-09-19T07:49:41.050694+0000","last_became_peered":"2024-09-19T07:49:41.050694+0000","last_unstale":"2024-09-19T07:49:43.031287+0000","last_undegraded":"2024-09-19T07:49:43.031287+0000","last_fullsized":"2024-09-19T07:49:43.031287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:16.156579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.038542+0000","last_change":"2024-09-19T07:49:40.036295+0000","last_active":"2024-09-19T07:49:41.038542+0000","last_peered":"2024-09-19T07:49:41.038542+0000","last_clean":"2024-09-19T07:49:41.038542+0000","last_became_active":"2024-09-19T07:49:40.035972+0000","last_became_peered":"2024-09-19T07:49:40.035972+0000","last_unstale":"2024-09-19T07:49:41.038542+0000","last_undegraded":"2024-09-19T07:49:41.038542+0000","last_fullsized":"2024-09-19T07:49:41.038542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:26.811584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.696808+0000","last_change":"2024-09-19T07:49:41.056640+0000","last_active":"2024-09-19T07:49:42.696808+0000","last_peered":"2024-09-19T07:49:42.696808+0000","last_clean":"2024-09-19T07:49:42.696808+0000","last_became_active":"2024-09-19T07:49:41.056359+0000","last_became_peered":"2024-09-19T07:49:41.056359+0000","last_unstale":"2024-09-19T07:49:42.696808+0000","last_undegraded":"2024-09-19T07:49:42.696808+0000","last_fullsized":"2024-09-19T07:49:42.696808+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:37.745819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824560+0000","last_change":"2024-09-19T07:49:40.040903+0000","last_active":"2024-09-19T07:49:42.824560+0000","last_peered":"2024-09-19T07:49:42.824560+0000","last_clean":"2024-09-19T07:49:42.824560+0000","last_became_active":"2024-09-19T07:49:40.040392+0000","last_became_peered":"2024-09-19T07:49:40.040392+0000","last_unstale":"2024-09-19T07:49:42.824560+0000","last_undegraded":"2024-09-19T07:49:42.824560+0000","last_fullsized":"2024-09-19T07:49:42.824560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:14.482826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044231+0000","last_change":"2024-09-19T07:49:41.049199+0000","last_active":"2024-09-19T07:49:42.044231+0000","last_peered":"2024-09-19T07:49:42.044231+0000","last_clean":"2024-09-19T07:49:42.044231+0000","last_became_active":"2024-09-19T07:49:41.048933+0000","last_became_peered":"2024-09-19T07:49:41.048933+0000","last_unstale":"2024-09-19T07:49:42.044231+0000","last_undegraded":"2024-09-19T07:49:42.044231+0000","last_fullsized":"2024-09-19T07:49:42.044231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:28.535549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044330+0000","last_change":"2024-09-19T07:49:40.041111+0000","last_active":"2024-09-19T07:49:42.044330+0000","last_peered":"2024-09-19T07:49:42.044330+0000","last_clean":"2024-09-19T07:49:42.044330+0000","last_became_active":"2024-09-19T07:49:40.040827+0000","last_became_peered":"2024-09-19T07:49:40.040827+0000","last_unstale":"2024-09-19T07:49:42.044330+0000","last_undegraded":"2024-09-19T07:49:42.044330+0000","last_fullsized":"2024-09-19T07:49:42.044330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:52.931623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.032146+0000","last_change":"2024-09-19T07:49:41.043019+0000","last_active":"2024-09-19T07:49:43.032146+0000","last_peered":"2024-09-19T07:49:43.032146+0000","last_clean":"2024-09-19T07:49:43.032146+0000","last_became_active":"2024-09-19T07:49:41.042246+0000","last_became_peered":"2024-09-19T07:49:41.042246+0000","last_unstale":"2024-09-19T07:49:43.032146+0000","last_undegraded":"2024-09-19T07:49:43.032146+0000","last_fullsized":"2024-09-19T07:49:43.032146+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:38.550958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.066504+0000","last_change":"2024-09-19T07:49:40.040472+0000","last_active":"2024-09-19T07:49:41.066504+0000","last_peered":"2024-09-19T07:49:41.066504+0000","last_clean":"2024-09-19T07:49:41.066504+0000","last_became_active":"2024-09-19T07:49:40.040227+0000","last_became_peered":"2024-09-19T07:49:40.040227+0000","last_unstale":"2024-09-19T07:49:41.066504+0000","last_undegraded":"2024-09-19T07:49:41.066504+0000","last_fullsized":"2024-09-19T07:49:41.066504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:32.130002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.043090+0000","last_change":"2024-09-19T07:49:41.067649+0000","last_active":"2024-09-19T07:49:42.043090+0000","last_peered":"2024-09-19T07:49:42.043090+0000","last_clean":"2024-09-19T07:49:42.043090+0000","last_became_active":"2024-09-19T07:49:41.067355+0000","last_became_peered":"2024-09-19T07:49:41.067355+0000","last_unstale":"2024-09-19T07:49:42.043090+0000","last_undegraded":"2024-09-19T07:49:42.043090+0000","last_fullsized":"2024-09-19T07:49:42.043090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:44.923391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"80'6","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031152+0000","last_change":"2024-09-19T07:49:40.034442+0000","last_active":"2024-09-19T07:49:43.031152+0000","last_peered":"2024-09-19T07:49:43.031152+0000","last_clean":"2024-09-19T07:49:43.031152+0000","last_became_active":"2024-09-19T07:49:40.034168+0000","last_became_peered":"2024-09-19T07:49:40.034168+0000","last_unstale":"2024-09-19T07:49:43.031152+0000","last_undegraded":"2024-09-19T07:49:43.031152+0000","last_fullsized":"2024-09-19T07:49:43.031152+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:29.408268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044627+0000","last_change":"2024-09-19T07:49:41.046137+0000","last_active":"2024-09-19T07:49:42.044627+0000","last_peered":"2024-09-19T07:49:42.044627+0000","last_clean":"2024-09-19T07:49:42.044627+0000","last_became_active":"2024-09-19T07:49:41.045826+0000","last_became_peered":"2024-09-19T07:49:41.045826+0000","last_unstale":"2024-09-19T07:49:42.044627+0000","last_undegraded":"2024-09-19T07:49:42.044627+0000","last_fullsized":"2024-09-19T07:49:42.044627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.574881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044414+0000","last_change":"2024-09-19T07:49:40.039449+0000","last_active":"2024-09-19T07:49:42.044414+0000","last_peered":"2024-09-19T07:49:42.044414+0000","last_clean":"2024-09-19T07:49:42.044414+0000","last_became_active":"2024-09-19T07:49:40.039127+0000","last_became_peered":"2024-09-19T07:49:40.039127+0000","last_unstale":"2024-09-19T07:49:42.044414+0000","last_undegraded":"2024-09-19T07:49:42.044414+0000","last_fullsized":"2024-09-19T07:49:42.044414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:41.786978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031061+0000","last_change":"2024-09-19T07:49:41.052856+0000","last_active":"2024-09-19T07:49:43.031061+0000","last_peered":"2024-09-19T07:49:43.031061+0000","last_clean":"2024-09-19T07:49:43.031061+0000","last_became_active":"2024-09-19T07:49:41.052591+0000","last_became_peered":"2024-09-19T07:49:41.052591+0000","last_unstale":"2024-09-19T07:49:43.031061+0000","last_undegraded":"2024-09-19T07:49:43.031061+0000","last_fullsized":"2024-09-19T07:49:43.031061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:25.748958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824123+0000","last_change":"2024-09-19T07:49:40.040152+0000","last_active":"2024-09-19T07:49:42.824123+0000","last_peered":"2024-09-19T07:49:42.824123+0000","last_clean":"2024-09-19T07:49:42.824123+0000","last_became_active":"2024-09-19T07:49:40.039848+0000","last_became_peered":"2024-09-19T07:49:40.039848+0000","last_unstale":"2024-09-19T07:49:42.824123+0000","last_undegraded":"2024-09-19T07:49:42.824123+0000","last_fullsized":"2024-09-19T07:49:42.824123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:58.425054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031311+0000","last_change":"2024-09-19T07:49:41.054387+0000","last_active":"2024-09-19T07:49:43.031311+0000","last_peered":"2024-09-19T07:49:43.031311+0000","last_clean":"2024-09-19T07:49:43.031311+0000","last_became_active":"2024-09-19T07:49:41.054125+0000","last_became_peered":"2024-09-19T07:49:41.054125+0000","last_unstale":"2024-09-19T07:49:43.031311+0000","last_undegraded":"2024-09-19T07:49:43.031311+0000","last_fullsized":"2024-09-19T07:49:43.031311+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:10.687061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.038042+0000","last_change":"2024-09-19T07:49:40.043587+0000","last_active":"2024-09-19T07:49:41.038042+0000","last_peered":"2024-09-19T07:49:41.038042+0000","last_clean":"2024-09-19T07:49:41.038042+0000","last_became_active":"2024-09-19T07:49:40.043339+0000","last_became_peered":"2024-09-19T07:49:40.043339+0000","last_unstale":"2024-09-19T07:49:41.038042+0000","last_undegraded":"2024-09-19T07:49:41.038042+0000","last_fullsized":"2024-09-19T07:49:41.038042+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:18.122629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031366+0000","last_change":"2024-09-19T07:49:41.062023+0000","last_active":"2024-09-19T07:49:43.031366+0000","last_peered":"2024-09-19T07:49:43.031366+0000","last_clean":"2024-09-19T07:49:43.031366+0000","last_became_active":"2024-09-19T07:49:41.059136+0000","last_became_peered":"2024-09-19T07:49:41.059136+0000","last_unstale":"2024-09-19T07:49:43.031366+0000","last_undegraded":"2024-09-19T07:49:43.031366+0000","last_fullsized":"2024-09-19T07:49:43.031366+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:18.521095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"80'8","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.696914+0000","last_change":"2024-09-19T07:49:40.033351+0000","last_active":"2024-09-19T07:49:42.696914+0000","last_peered":"2024-09-19T07:49:42.696914+0000","last_clean":"2024-09-19T07:49:42.696914+0000","last_became_active":"2024-09-19T07:49:40.033076+0000","last_became_peered":"2024-09-19T07:49:40.033076+0000","last_unstale":"2024-09-19T07:49:42.696914+0000","last_undegraded":"2024-09-19T07:49:42.696914+0000","last_fullsized":"2024-09-19T07:49:42.696914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:28.516163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031569+0000","last_change":"2024-09-19T07:49:41.044278+0000","last_active":"2024-09-19T07:49:43.031569+0000","last_peered":"2024-09-19T07:49:43.031569+0000","last_clean":"2024-09-19T07:49:43.031569+0000","last_became_active":"2024-09-19T07:49:41.043995+0000","last_became_peered":"2024-09-19T07:49:41.043995+0000","last_unstale":"2024-09-19T07:49:43.031569+0000","last_undegraded":"2024-09-19T07:49:43.031569+0000","last_fullsized":"2024-09-19T07:49:43.031569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:24.634859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824655+0000","last_change":"2024-09-19T07:49:40.044469+0000","last_active":"2024-09-19T07:49:42.824655+0000","last_peered":"2024-09-19T07:49:42.824655+0000","last_clean":"2024-09-19T07:49:42.824655+0000","last_became_active":"2024-09-19T07:49:40.043722+0000","last_became_peered":"2024-09-19T07:49:40.043722+0000","last_unstale":"2024-09-19T07:49:42.824655+0000","last_undegraded":"2024-09-19T07:49:42.824655+0000","last_fullsized":"2024-09-19T07:49:42.824655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:29:20.900485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.054002+0000","last_change":"2024-09-19T07:49:41.052193+0000","last_active":"2024-09-19T07:49:41.054002+0000","last_peered":"2024-09-19T07:49:41.054002+0000","last_clean":"2024-09-19T07:49:41.054002+0000","last_became_active":"2024-09-19T07:49:41.051891+0000","last_became_peered":"2024-09-19T07:49:41.051891+0000","last_unstale":"2024-09-19T07:49:41.054002+0000","last_undegraded":"2024-09-19T07:49:41.054002+0000","last_fullsized":"2024-09-19T07:49:41.054002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:17.335309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044792+0000","last_change":"2024-09-19T07:49:40.033040+0000","last_active":"2024-09-19T07:49:42.044792+0000","last_peered":"2024-09-19T07:49:42.044792+0000","last_clean":"2024-09-19T07:49:42.044792+0000","last_became_active":"2024-09-19T07:49:40.032755+0000","last_became_peered":"2024-09-19T07:49:40.032755+0000","last_unstale":"2024-09-19T07:49:42.044792+0000","last_undegraded":"2024-09-19T07:49:42.044792+0000","last_fullsized":"2024-09-19T07:49:42.044792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:16.215029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.061945+0000","last_change":"2024-09-19T07:49:41.057688+0000","last_active":"2024-09-19T07:49:41.061945+0000","last_peered":"2024-09-19T07:49:41.061945+0000","last_clean":"2024-09-19T07:49:41.061945+0000","last_became_active":"2024-09-19T07:49:41.057382+0000","last_became_peered":"2024-09-19T07:49:41.057382+0000","last_unstale":"2024-09-19T07:49:41.061945+0000","last_undegraded":"2024-09-19T07:49:41.061945+0000","last_fullsized":"2024-09-19T07:49:41.061945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:47.973441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.064816+0000","last_change":"2024-09-19T07:49:40.040575+0000","last_active":"2024-09-19T07:49:41.064816+0000","last_peered":"2024-09-19T07:49:41.064816+0000","last_clean":"2024-09-19T07:49:41.064816+0000","last_became_active":"2024-09-19T07:49:40.040186+0000","last_became_peered":"2024-09-19T07:49:40.040186+0000","last_unstale":"2024-09-19T07:49:41.064816+0000","last_undegraded":"2024-09-19T07:49:41.064816+0000","last_fullsized":"2024-09-19T07:49:41.064816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:12.523848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.032167+0000","last_change":"2024-09-19T07:49:41.045026+0000","last_active":"2024-09-19T07:49:43.032167+0000","last_peered":"2024-09-19T07:49:43.032167+0000","last_clean":"2024-09-19T07:49:43.032167+0000","last_became_active":"2024-09-19T07:49:41.044585+0000","last_became_peered":"2024-09-19T07:49:41.044585+0000","last_unstale":"2024-09-19T07:49:43.032167+0000","last_undegraded":"2024-09-19T07:49:43.032167+0000","last_fullsized":"2024-09-19T07:49:43.032167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:23.771644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.042814+0000","last_change":"2024-09-19T07:49:40.036123+0000","last_active":"2024-09-19T07:49:42.042814+0000","last_peered":"2024-09-19T07:49:42.042814+0000","last_clean":"2024-09-19T07:49:42.042814+0000","last_became_active":"2024-09-19T07:49:40.035838+0000","last_became_peered":"2024-09-19T07:49:40.035838+0000","last_unstale":"2024-09-19T07:49:42.042814+0000","last_undegraded":"2024-09-19T07:49:42.042814+0000","last_fullsized":"2024-09-19T07:49:42.042814+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:09.135514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697285+0000","last_change":"2024-09-19T07:49:41.060216+0000","last_active":"2024-09-19T07:49:42.697285+0000","last_peered":"2024-09-19T07:49:42.697285+0000","last_clean":"2024-09-19T07:49:42.697285+0000","last_became_active":"2024-09-19T07:49:41.059811+0000","last_became_peered":"2024-09-19T07:49:41.059811+0000","last_unstale":"2024-09-19T07:49:42.697285+0000","last_undegraded":"2024-09-19T07:49:42.697285+0000","last_fullsized":"2024-09-19T07:49:42.697285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:07.197132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031963+0000","last_change":"2024-09-19T07:49:40.041426+0000","last_active":"2024-09-19T07:49:43.031963+0000","last_peered":"2024-09-19T07:49:43.031963+0000","last_clean":"2024-09-19T07:49:43.031963+0000","last_became_active":"2024-09-19T07:49:40.041174+0000","last_became_peered":"2024-09-19T07:49:40.041174+0000","last_unstale":"2024-09-19T07:49:43.031963+0000","last_undegraded":"2024-09-19T07:49:43.031963+0000","last_fullsized":"2024-09-19T07:49:43.031963+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:44.203551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697463+0000","last_change":"2024-09-19T07:49:41.060569+0000","last_active":"2024-09-19T07:49:42.697463+0000","last_peered":"2024-09-19T07:49:42.697463+0000","last_clean":"2024-09-19T07:49:42.697463+0000","last_became_active":"2024-09-19T07:49:41.060285+0000","last_became_peered":"2024-09-19T07:49:41.060285+0000","last_unstale":"2024-09-19T07:49:42.697463+0000","last_undegraded":"2024-09-19T07:49:42.697463+0000","last_fullsized":"2024-09-19T07:49:42.697463+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:43.595570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031493+0000","last_change":"2024-09-19T07:49:40.047109+0000","last_active":"2024-09-19T07:49:43.031493+0000","last_peered":"2024-09-19T07:49:43.031493+0000","last_clean":"2024-09-19T07:49:43.031493+0000","last_became_active":"2024-09-19T07:49:40.046800+0000","last_became_peered":"2024-09-19T07:49:40.046800+0000","last_unstale":"2024-09-19T07:49:43.031493+0000","last_undegraded":"2024-09-19T07:49:43.031493+0000","last_fullsized":"2024-09-19T07:49:43.031493+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:56.862116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031389+0000","last_change":"2024-09-19T07:49:41.050201+0000","last_active":"2024-09-19T07:49:43.031389+0000","last_peered":"2024-09-19T07:49:43.031389+0000","last_clean":"2024-09-19T07:49:43.031389+0000","last_became_active":"2024-09-19T07:49:41.049920+0000","last_became_peered":"2024-09-19T07:49:41.049920+0000","last_unstale":"2024-09-19T07:49:43.031389+0000","last_undegraded":"2024-09-19T07:49:43.031389+0000","last_fullsized":"2024-09-19T07:49:43.031389+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:44.923391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031633+0000","last_change":"2024-09-19T07:49:40.040948+0000","last_active":"2024-09-19T07:49:43.031633+0000","last_peered":"2024-09-19T07:49:43.031633+0000","last_clean":"2024-09-19T07:49:43.031633+0000","last_became_active":"2024-09-19T07:49:40.040460+0000","last_became_peered":"2024-09-19T07:49:40.040460+0000","last_unstale":"2024-09-19T07:49:43.031633+0000","last_undegraded":"2024-09-19T07:49:43.031633+0000","last_fullsized":"2024-09-19T07:49:43.031633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:06.192295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.043972+0000","last_change":"2024-09-19T07:49:41.055770+0000","last_active":"2024-09-19T07:49:42.043972+0000","last_peered":"2024-09-19T07:49:42.043972+0000","last_clean":"2024-09-19T07:49:42.043972+0000","last_became_active":"2024-09-19T07:49:41.055506+0000","last_became_peered":"2024-09-19T07:49:41.055506+0000","last_unstale":"2024-09-19T07:49:42.043972+0000","last_undegraded":"2024-09-19T07:49:42.043972+0000","last_fullsized":"2024-09-19T07:49:42.043972+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:03:52.869311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.043939+0000","last_change":"2024-09-19T07:49:40.037347+0000","last_active":"2024-09-19T07:49:42.043939+0000","last_peered":"2024-09-19T07:49:42.043939+0000","last_clean":"2024-09-19T07:49:42.043939+0000","last_became_active":"2024-09-19T07:49:40.037043+0000","last_became_peered":"2024-09-19T07:49:40.037043+0000","last_unstale":"2024-09-19T07:49:42.043939+0000","last_undegraded":"2024-09-19T07:49:42.043939+0000","last_fullsized":"2024-09-19T07:49:42.043939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:26:15.212481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031094+0000","last_change":"2024-09-19T07:49:41.060272+0000","last_active":"2024-09-19T07:49:43.031094+0000","last_peered":"2024-09-19T07:49:43.031094+0000","last_clean":"2024-09-19T07:49:43.031094+0000","last_became_active":"2024-09-19T07:49:41.060004+0000","last_became_peered":"2024-09-19T07:49:41.060004+0000","last_unstale":"2024-09-19T07:49:43.031094+0000","last_undegraded":"2024-09-19T07:49:43.031094+0000","last_fullsized":"2024-09-19T07:49:43.031094+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:15.255291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697421+0000","last_change":"2024-09-19T07:49:40.043434+0000","last_active":"2024-09-19T07:49:42.697421+0000","last_peered":"2024-09-19T07:49:42.697421+0000","last_clean":"2024-09-19T07:49:42.697421+0000","last_became_active":"2024-09-19T07:49:40.042987+0000","last_became_peered":"2024-09-19T07:49:40.042987+0000","last_unstale":"2024-09-19T07:49:42.697421+0000","last_undegraded":"2024-09-19T07:49:42.697421+0000","last_fullsized":"2024-09-19T07:49:42.697421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:09.682224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044532+0000","last_change":"2024-09-19T07:49:41.049676+0000","last_active":"2024-09-19T07:49:42.044532+0000","last_peered":"2024-09-19T07:49:42.044532+0000","last_clean":"2024-09-19T07:49:42.044532+0000","last_became_active":"2024-09-19T07:49:41.049413+0000","last_became_peered":"2024-09-19T07:49:41.049413+0000","last_unstale":"2024-09-19T07:49:42.044532+0000","last_undegraded":"2024-09-19T07:49:42.044532+0000","last_fullsized":"2024-09-19T07:49:42.044532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:36:20.876800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824499+0000","last_change":"2024-09-19T07:49:40.040556+0000","last_active":"2024-09-19T07:49:42.824499+0000","last_peered":"2024-09-19T07:49:42.824499+0000","last_clean":"2024-09-19T07:49:42.824499+0000","last_became_active":"2024-09-19T07:49:40.040134+0000","last_became_peered":"2024-09-19T07:49:40.040134+0000","last_unstale":"2024-09-19T07:49:42.824499+0000","last_undegraded":"2024-09-19T07:49:42.824499+0000","last_fullsized":"2024-09-19T07:49:42.824499+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:56:28.020397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697243+0000","last_change":"2024-09-19T07:49:41.066024+0000","last_active":"2024-09-19T07:49:42.697243+0000","last_peered":"2024-09-19T07:49:42.697243+0000","last_clean":"2024-09-19T07:49:42.697243+0000","last_became_active":"2024-09-19T07:49:41.064768+0000","last_became_peered":"2024-09-19T07:49:41.064768+0000","last_unstale":"2024-09-19T07:49:42.697243+0000","last_undegraded":"2024-09-19T07:49:42.697243+0000","last_fullsized":"2024-09-19T07:49:42.697243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:45.208388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824335+0000","last_change":"2024-09-19T07:49:40.042468+0000","last_active":"2024-09-19T07:49:42.824335+0000","last_peered":"2024-09-19T07:49:42.824335+0000","last_clean":"2024-09-19T07:49:42.824335+0000","last_became_active":"2024-09-19T07:49:40.040583+0000","last_became_peered":"2024-09-19T07:49:40.040583+0000","last_unstale":"2024-09-19T07:49:42.824335+0000","last_undegraded":"2024-09-19T07:49:42.824335+0000","last_fullsized":"2024-09-19T07:49:42.824335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:41.786978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031075+0000","last_change":"2024-09-19T07:49:41.059536+0000","last_active":"2024-09-19T07:49:43.031075+0000","last_peered":"2024-09-19T07:49:43.031075+0000","last_clean":"2024-09-19T07:49:43.031075+0000","last_became_active":"2024-09-19T07:49:41.059206+0000","last_became_peered":"2024-09-19T07:49:41.059206+0000","last_unstale":"2024-09-19T07:49:43.031075+0000","last_undegraded":"2024-09-19T07:49:43.031075+0000","last_fullsized":"2024-09-19T07:49:43.031075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:33.713515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.030838+0000","last_change":"2024-09-19T07:49:40.037515+0000","last_active":"2024-09-19T07:49:43.030838+0000","last_peered":"2024-09-19T07:49:43.030838+0000","last_clean":"2024-09-19T07:49:43.030838+0000","last_became_active":"2024-09-19T07:49:40.037233+0000","last_became_peered":"2024-09-19T07:49:40.037233+0000","last_unstale":"2024-09-19T07:49:43.030838+0000","last_undegraded":"2024-09-19T07:49:43.030838+0000","last_fullsized":"2024-09-19T07:49:43.030838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:19:17.728130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031121+0000","last_change":"2024-09-19T07:49:41.046993+0000","last_active":"2024-09-19T07:49:43.031121+0000","last_peered":"2024-09-19T07:49:43.031121+0000","last_clean":"2024-09-19T07:49:43.031121+0000","last_became_active":"2024-09-19T07:49:41.046699+0000","last_became_peered":"2024-09-19T07:49:41.046699+0000","last_unstale":"2024-09-19T07:49:43.031121+0000","last_undegraded":"2024-09-19T07:49:43.031121+0000","last_fullsized":"2024-09-19T07:49:43.031121+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:29.520999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.038395+0000","last_change":"2024-09-19T07:49:40.042739+0000","last_active":"2024-09-19T07:49:41.038395+0000","last_peered":"2024-09-19T07:49:41.038395+0000","last_clean":"2024-09-19T07:49:41.038395+0000","last_became_active":"2024-09-19T07:49:40.042465+0000","last_became_peered":"2024-09-19T07:49:40.042465+0000","last_unstale":"2024-09-19T07:49:41.038395+0000","last_undegraded":"2024-09-19T07:49:41.038395+0000","last_fullsized":"2024-09-19T07:49:41.038395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:58.792956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.043061+0000","last_change":"2024-09-19T07:49:41.061934+0000","last_active":"2024-09-19T07:49:42.043061+0000","last_peered":"2024-09-19T07:49:42.043061+0000","last_clean":"2024-09-19T07:49:42.043061+0000","last_became_active":"2024-09-19T07:49:41.061636+0000","last_became_peered":"2024-09-19T07:49:41.061636+0000","last_unstale":"2024-09-19T07:49:42.043061+0000","last_undegraded":"2024-09-19T07:49:42.043061+0000","last_fullsized":"2024-09-19T07:49:42.043061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:10.687061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031118+0000","last_change":"2024-09-19T07:49:40.043622+0000","last_active":"2024-09-19T07:49:43.031118+0000","last_peered":"2024-09-19T07:49:43.031118+0000","last_clean":"2024-09-19T07:49:43.031118+0000","last_became_active":"2024-09-19T07:49:40.043000+0000","last_became_peered":"2024-09-19T07:49:40.043000+0000","last_unstale":"2024-09-19T07:49:43.031118+0000","last_undegraded":"2024-09-19T07:49:43.031118+0000","last_fullsized":"2024-09-19T07:49:43.031118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:59.813214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.043042+0000","last_change":"2024-09-19T07:49:41.044413+0000","last_active":"2024-09-19T07:49:42.043042+0000","last_peered":"2024-09-19T07:49:42.043042+0000","last_clean":"2024-09-19T07:49:42.043042+0000","last_became_active":"2024-09-19T07:49:41.044111+0000","last_became_peered":"2024-09-19T07:49:41.044111+0000","last_unstale":"2024-09-19T07:49:42.043042+0000","last_undegraded":"2024-09-19T07:49:42.043042+0000","last_fullsized":"2024-09-19T07:49:42.043042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:16.156579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.042653+0000","last_change":"2024-09-19T07:49:40.043335+0000","last_active":"2024-09-19T07:49:42.042653+0000","last_peered":"2024-09-19T07:49:42.042653+0000","last_clean":"2024-09-19T07:49:42.042653+0000","last_became_active":"2024-09-19T07:49:40.043042+0000","last_became_peered":"2024-09-19T07:49:40.043042+0000","last_unstale":"2024-09-19T07:49:42.042653+0000","last_undegraded":"2024-09-19T07:49:42.042653+0000","last_fullsized":"2024-09-19T07:49:42.042653+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:56.844586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824593+0000","last_change":"2024-09-19T07:49:41.055598+0000","last_active":"2024-09-19T07:49:42.824593+0000","last_peered":"2024-09-19T07:49:42.824593+0000","last_clean":"2024-09-19T07:49:42.824593+0000","last_became_active":"2024-09-19T07:49:41.055292+0000","last_became_peered":"2024-09-19T07:49:41.055292+0000","last_unstale":"2024-09-19T07:49:42.824593+0000","last_undegraded":"2024-09-19T07:49:42.824593+0000","last_fullsized":"2024-09-19T07:49:42.824593+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:07.197132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.032029+0000","last_change":"2024-09-19T07:49:40.046770+0000","last_active":"2024-09-19T07:49:43.032029+0000","last_peered":"2024-09-19T07:49:43.032029+0000","last_clean":"2024-09-19T07:49:43.032029+0000","last_became_active":"2024-09-19T07:49:40.046469+0000","last_became_peered":"2024-09-19T07:49:40.046469+0000","last_unstale":"2024-09-19T07:49:43.032029+0000","last_undegraded":"2024-09-19T07:49:43.032029+0000","last_fullsized":"2024-09-19T07:49:43.032029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:19.438482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":279,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824353+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:49:42.824353+0000","last_peered":"2024-09-19T07:49:42.824353+0000","last_clean":"2024-09-19T07:49:42.824353+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:49:42.824353+0000","last_undegraded":"2024-09-19T07:49:42.824353+0000","last_fullsized":"2024-09-19T07:49:42.824353+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99292,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":97726,"kb_avail":94272548,"statfs":{"total":96636764160,"available":96535089152,"internally_reserved":0,"allocated":1581056,"data_stored":1041163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":100071882},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509524,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":112672,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94259168,"statfs":{"total":96636764160,"available":96521388032,"internally_reserved":0,"allocated":995328,"data_stored":452385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":114358726},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509525,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50200,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321640,"statfs":{"total":96636764160,"available":96585359360,"internally_reserved":0,"allocated":987136,"data_stored":450807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509525,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46064,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325776,"statfs":{"total":96636764160,"available":96589594624,"internally_reserved":0,"allocated":1003520,"data_stored":452489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509524,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100704,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271136,"statfs":{"total":96636764160,"available":96533643264,"internally_reserved":0,"allocated":1576960,"data_stored":1041141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542228,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130644,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94241196,"statfs":{"total":96636764160,"available":96502984704,"internally_reserved":0,"allocated":974848,"data_stored":444663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542228,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94620,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94277220,"statfs":{"total":96636764160,"available":96539873280,"internally_reserved":0,"allocated":991232,"data_stored":450885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104284,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267556,"statfs":{"total":96636764160,"available":96529977344,"internally_reserved":0,"allocated":983040,"data_stored":444823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:49:45.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:49:45.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.185+0000 7f3a74e53640 1 Processor -- start 2024-09-19T07:49:45.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.185+0000 7f3a74e53640 1 -- start start 2024-09-19T07:49:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.186+0000 7f3a74e53640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3a7015b120 0x7f3a7006f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.186+0000 7f3a74e53640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a7006dec0 0x7f3a7006e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.186+0000 7f3a74e53640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a700d04b0 0x7f3a700d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.186+0000 7f3a74e53640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f3a700d0e40 con 0x7f3a700d04b0 2024-09-19T07:49:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.186+0000 7f3a74e53640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f3a7015b500 con 0x7f3a7015b120 2024-09-19T07:49:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.186+0000 7f3a74e53640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3a7006e940 con 0x7f3a7006dec0 2024-09-19T07:49:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.186+0000 7f3a6ed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a700d04b0 0x7f3a700d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.186+0000 7f3a6ed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a700d04b0 0x7f3a700d0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60878/0 (socket says 172.21.15.16:60878) 2024-09-19T07:49:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.186+0000 7f3a6ed9d640 1 -- 172.21.15.16:0/472670747 learned_addr learned my addr 172.21.15.16:0/472670747 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.186+0000 7f3a6dd9b640 1 --2- 172.21.15.16:0/472670747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a7006dec0 0x7f3a7006e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.186+0000 7f3a6e59c640 1 --2- 172.21.15.16:0/472670747 >> v2:172.21.15.146:3300/0 conn(0x7f3a7015b120 0x7f3a7006f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.186+0000 7f3a6ed9d640 1 -- 172.21.15.16:0/472670747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a7006dec0 msgr2=0x7f3a7006e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:45.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.186+0000 7f3a6ed9d640 1 --2- 172.21.15.16:0/472670747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a7006dec0 0x7f3a7006e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.186+0000 7f3a6ed9d640 1 -- 172.21.15.16:0/472670747 >> v2:172.21.15.146:3300/0 conn(0x7f3a7015b120 msgr2=0x7f3a7006f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:45.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.186+0000 7f3a6ed9d640 1 --2- 172.21.15.16:0/472670747 >> v2:172.21.15.146:3300/0 conn(0x7f3a7015b120 0x7f3a7006f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.186+0000 7f3a6ed9d640 1 -- 172.21.15.16:0/472670747 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7006ec20 con 0x7f3a700d04b0 2024-09-19T07:49:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.186+0000 7f3a6e59c640 1 --2- 172.21.15.16:0/472670747 >> v2:172.21.15.146:3300/0 conn(0x7f3a7015b120 0x7f3a7006f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.187+0000 7f3a6ed9d640 1 --2- 172.21.15.16:0/472670747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a700d04b0 0x7f3a700d0900 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3a7006cb50 tx=0x7f3a6802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9811f8d6278a1909 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.187+0000 7f3a6d59a640 1 -- 172.21.15.16:0/472670747 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68036070 con 0x7f3a700d04b0 2024-09-19T07:49:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.187+0000 7f3a6d59a640 1 -- 172.21.15.16:0/472670747 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a68031600 con 0x7f3a700d04b0 2024-09-19T07:49:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.187+0000 7f3a6d59a640 1 -- 172.21.15.16:0/472670747 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a680319d0 con 0x7f3a700d04b0 2024-09-19T07:49:45.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.187+0000 7f3a74e53640 1 -- 172.21.15.16:0/472670747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a700d04b0 msgr2=0x7f3a700d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:45.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.187+0000 7f3a74e53640 1 --2- 172.21.15.16:0/472670747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a700d04b0 0x7f3a700d0900 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3a7006cb50 tx=0x7f3a6802a6a0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.188+0000 7f3a74e53640 1 -- 172.21.15.16:0/472670747 shutdown_connections 2024-09-19T07:49:45.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.188+0000 7f3a74e53640 1 --2- 172.21.15.16:0/472670747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a700d04b0 0x7f3a700d0900 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.188+0000 7f3a74e53640 1 --2- 172.21.15.16:0/472670747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a7006dec0 0x7f3a7006e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.188+0000 7f3a74e53640 1 --2- 172.21.15.16:0/472670747 >> v2:172.21.15.146:3300/0 conn(0x7f3a7015b120 0x7f3a7006f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.188+0000 7f3a74e53640 1 -- 172.21.15.16:0/472670747 >> 172.21.15.16:0/472670747 conn(0x7f3a7006bdf0 msgr2=0x7f3a7006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.188+0000 7f3a74e53640 1 -- 172.21.15.16:0/472670747 shutdown_connections 2024-09-19T07:49:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.188+0000 7f3a74e53640 1 -- 172.21.15.16:0/472670747 wait complete. 2024-09-19T07:49:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.189+0000 7f3a74e53640 1 Processor -- start 2024-09-19T07:49:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.189+0000 7f3a74e53640 1 -- start start 2024-09-19T07:49:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.190+0000 7f3a74e53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a7006dec0 0x7f3a70157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.190+0000 7f3a74e53640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a700d04b0 0x7f3a700cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.190+0000 7f3a74e53640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a7015b120 0x7f3a700cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.190+0000 7f3a74e53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7006f4c0 con 0x7f3a7006dec0 2024-09-19T07:49:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.190+0000 7f3a74e53640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7006f1e0 con 0x7f3a7015b120 2024-09-19T07:49:45.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.190+0000 7f3a74e53640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3a700ce060 con 0x7f3a700d04b0 2024-09-19T07:49:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.190+0000 7f3a6e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a7006dec0 0x7f3a70157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.190+0000 7f3a6e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a7006dec0 0x7f3a70157d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60890/0 (socket says 172.21.15.16:60890) 2024-09-19T07:49:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.190+0000 7f3a6e59c640 1 -- 172.21.15.16:0/2742013335 learned_addr learned my addr 172.21.15.16:0/2742013335 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.190+0000 7f3a6e59c640 1 -- 172.21.15.16:0/2742013335 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a700d04b0 msgr2=0x7f3a700cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.190+0000 7f3a6dd9b640 1 --2- 172.21.15.16:0/2742013335 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a700d04b0 0x7f3a700cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.190+0000 7f3a6e59c640 1 --2- 172.21.15.16:0/2742013335 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a700d04b0 0x7f3a700cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.190+0000 7f3a6e59c640 1 -- 172.21.15.16:0/2742013335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a7015b120 msgr2=0x7f3a700cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.190+0000 7f3a6e59c640 1 --2- 172.21.15.16:0/2742013335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a7015b120 0x7f3a700cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.190+0000 7f3a6e59c640 1 -- 172.21.15.16:0/2742013335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a700cb1c0 con 0x7f3a7006dec0 2024-09-19T07:49:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.190+0000 7f3a6e59c640 1 --2- 172.21.15.16:0/2742013335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a7006dec0 0x7f3a70157d10 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000c7f0 tx=0x7f3a6000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.190+0000 7f3a5effd640 1 -- 172.21.15.16:0/2742013335 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60015070 con 0x7f3a7006dec0 2024-09-19T07:49:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.191+0000 7f3a5effd640 1 -- 172.21.15.16:0/2742013335 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a60007820 con 0x7f3a7006dec0 2024-09-19T07:49:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.191+0000 7f3a74e53640 1 -- 172.21.15.16:0/2742013335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a701471e0 con 0x7f3a7006dec0 2024-09-19T07:49:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.191+0000 7f3a5effd640 1 -- 172.21.15.16:0/2742013335 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60007bd0 con 0x7f3a7006dec0 2024-09-19T07:49:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.191+0000 7f3a74e53640 1 -- 172.21.15.16:0/2742013335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a70148430 con 0x7f3a7006dec0 2024-09-19T07:49:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.192+0000 7f3a74e53640 1 -- 172.21.15.16:0/2742013335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a70068da0 con 0x7f3a7006dec0 2024-09-19T07:49:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.192+0000 7f3a5effd640 1 -- 172.21.15.16:0/2742013335 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3a60018070 con 0x7f3a7006dec0 2024-09-19T07:49:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.193+0000 7f3a5effd640 1 --2- 172.21.15.16:0/2742013335 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3a5006c2b0 0x7f3a5006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.193+0000 7f3a5effd640 1 -- 172.21.15.16:0/2742013335 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f3a6008b410 con 0x7f3a7006dec0 2024-09-19T07:49:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.193+0000 7f3a6dd9b640 1 --2- 172.21.15.16:0/2742013335 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3a5006c2b0 0x7f3a5006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.194+0000 7f3a6dd9b640 1 --2- 172.21.15.16:0/2742013335 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3a5006c2b0 0x7f3a5006e770 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3a64006fd0 tx=0x7f3a64008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.197+0000 7f3a5effd640 1 -- 172.21.15.16:0/2742013335 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3a70068da0 con 0x7f3a7006dec0 2024-09-19T07:49:45.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.369+0000 7f3a74e53640 1 -- 172.21.15.16:0/2742013335 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3a7014bff0 con 0x7f3a5006c2b0 2024-09-19T07:49:45.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.375+0000 7f3a5effd640 1 -- 172.21.15.16:0/2742013335 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252077 (secure 0 0 0) 0x7f3a7014bff0 con 0x7f3a5006c2b0 2024-09-19T07:49:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:49:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.378+0000 7f3a74e53640 1 -- 172.21.15.16:0/2742013335 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3a5006c2b0 msgr2=0x7f3a5006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.378+0000 7f3a74e53640 1 --2- 172.21.15.16:0/2742013335 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3a5006c2b0 0x7f3a5006e770 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3a64006fd0 tx=0x7f3a64008040 comp rx=0 tx=0).stop 2024-09-19T07:49:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.378+0000 7f3a74e53640 1 -- 172.21.15.16:0/2742013335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a7006dec0 msgr2=0x7f3a70157d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.378+0000 7f3a74e53640 1 --2- 172.21.15.16:0/2742013335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a7006dec0 0x7f3a70157d10 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000c7f0 tx=0x7f3a6000ccb0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.378+0000 7f3a74e53640 1 -- 172.21.15.16:0/2742013335 shutdown_connections 2024-09-19T07:49:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.378+0000 7f3a74e53640 1 --2- 172.21.15.16:0/2742013335 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3a5006c2b0 0x7f3a5006e770 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.378+0000 7f3a74e53640 1 --2- 172.21.15.16:0/2742013335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a7015b120 0x7f3a700cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.378+0000 7f3a74e53640 1 --2- 172.21.15.16:0/2742013335 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a700d04b0 0x7f3a700cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.378+0000 7f3a74e53640 1 --2- 172.21.15.16:0/2742013335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a7006dec0 0x7f3a70157d10 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.378+0000 7f3a74e53640 1 -- 172.21.15.16:0/2742013335 >> 172.21.15.16:0/2742013335 conn(0x7f3a7006bdf0 msgr2=0x7f3a700cccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.378+0000 7f3a74e53640 1 -- 172.21.15.16:0/2742013335 shutdown_connections 2024-09-19T07:49:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.378+0000 7f3a74e53640 1 -- 172.21.15.16:0/2742013335 wait complete. 2024-09-19T07:49:45.390 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-09-19T07:49:44.547732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":738480,"kb_used_data":8880,"kb_used_omap":12,"kb_used_meta":729395,"kb_avail":754236240,"statfs":{"total":773094113280,"available":772337909760,"internally_reserved":0,"allocated":9093120,"data_stored":4778356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":746901040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.526570"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824591+0000","last_change":"2024-09-19T07:49:41.055660+0000","last_active":"2024-09-19T07:49:42.824591+0000","last_peered":"2024-09-19T07:49:42.824591+0000","last_clean":"2024-09-19T07:49:42.824591+0000","last_became_active":"2024-09-19T07:49:41.055401+0000","last_became_peered":"2024-09-19T07:49:41.055401+0000","last_unstale":"2024-09-19T07:49:42.824591+0000","last_undegraded":"2024-09-19T07:49:42.824591+0000","last_fullsized":"2024-09-19T07:49:42.824591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:57.866953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697358+0000","last_change":"2024-09-19T07:49:41.065827+0000","last_active":"2024-09-19T07:49:42.697358+0000","last_peered":"2024-09-19T07:49:42.697358+0000","last_clean":"2024-09-19T07:49:42.697358+0000","last_became_active":"2024-09-19T07:49:41.064467+0000","last_became_peered":"2024-09-19T07:49:41.064467+0000","last_unstale":"2024-09-19T07:49:42.697358+0000","last_undegraded":"2024-09-19T07:49:42.697358+0000","last_fullsized":"2024-09-19T07:49:42.697358+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:26:38.483340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.043081+0000","last_change":"2024-09-19T07:49:41.061515+0000","last_active":"2024-09-19T07:49:42.043081+0000","last_peered":"2024-09-19T07:49:42.043081+0000","last_clean":"2024-09-19T07:49:42.043081+0000","last_became_active":"2024-09-19T07:49:41.061216+0000","last_became_peered":"2024-09-19T07:49:41.061216+0000","last_unstale":"2024-09-19T07:49:42.043081+0000","last_undegraded":"2024-09-19T07:49:42.043081+0000","last_fullsized":"2024-09-19T07:49:42.043081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:18.521095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044348+0000","last_change":"2024-09-19T07:49:41.048005+0000","last_active":"2024-09-19T07:49:42.044348+0000","last_peered":"2024-09-19T07:49:42.044348+0000","last_clean":"2024-09-19T07:49:42.044348+0000","last_became_active":"2024-09-19T07:49:41.047728+0000","last_became_peered":"2024-09-19T07:49:41.047728+0000","last_unstale":"2024-09-19T07:49:42.044348+0000","last_undegraded":"2024-09-19T07:49:42.044348+0000","last_fullsized":"2024-09-19T07:49:42.044348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:37.745819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.049521+0000","last_change":"2024-09-19T07:49:41.045867+0000","last_active":"2024-09-19T07:49:41.049521+0000","last_peered":"2024-09-19T07:49:41.049521+0000","last_clean":"2024-09-19T07:49:41.049521+0000","last_became_active":"2024-09-19T07:49:41.045562+0000","last_became_peered":"2024-09-19T07:49:41.045562+0000","last_unstale":"2024-09-19T07:49:41.049521+0000","last_undegraded":"2024-09-19T07:49:41.049521+0000","last_fullsized":"2024-09-19T07:49:41.049521+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:18.302335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031751+0000","last_change":"2024-09-19T07:49:41.043246+0000","last_active":"2024-09-19T07:49:43.031751+0000","last_peered":"2024-09-19T07:49:43.031751+0000","last_clean":"2024-09-19T07:49:43.031751+0000","last_became_active":"2024-09-19T07:49:41.042651+0000","last_became_peered":"2024-09-19T07:49:41.042651+0000","last_unstale":"2024-09-19T07:49:43.031751+0000","last_undegraded":"2024-09-19T07:49:43.031751+0000","last_fullsized":"2024-09-19T07:49:43.031751+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:21.000170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031054+0000","last_change":"2024-09-19T07:49:41.058873+0000","last_active":"2024-09-19T07:49:43.031054+0000","last_peered":"2024-09-19T07:49:43.031054+0000","last_clean":"2024-09-19T07:49:43.031054+0000","last_became_active":"2024-09-19T07:49:41.058564+0000","last_became_peered":"2024-09-19T07:49:41.058564+0000","last_unstale":"2024-09-19T07:49:43.031054+0000","last_undegraded":"2024-09-19T07:49:43.031054+0000","last_fullsized":"2024-09-19T07:49:43.031054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:11.306569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.053306+0000","last_change":"2024-09-19T07:49:41.046577+0000","last_active":"2024-09-19T07:49:41.053306+0000","last_peered":"2024-09-19T07:49:41.053306+0000","last_clean":"2024-09-19T07:49:41.053306+0000","last_became_active":"2024-09-19T07:49:41.046302+0000","last_became_peered":"2024-09-19T07:49:41.046302+0000","last_unstale":"2024-09-19T07:49:41.053306+0000","last_undegraded":"2024-09-19T07:49:41.053306+0000","last_fullsized":"2024-09-19T07:49:41.053306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:24.122562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044172+0000","last_change":"2024-09-19T07:49:41.048380+0000","last_active":"2024-09-19T07:49:42.044172+0000","last_peered":"2024-09-19T07:49:42.044172+0000","last_clean":"2024-09-19T07:49:42.044172+0000","last_became_active":"2024-09-19T07:49:41.048063+0000","last_became_peered":"2024-09-19T07:49:41.048063+0000","last_unstale":"2024-09-19T07:49:42.044172+0000","last_undegraded":"2024-09-19T07:49:42.044172+0000","last_fullsized":"2024-09-19T07:49:42.044172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:03.100435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044256+0000","last_change":"2024-09-19T07:49:41.050046+0000","last_active":"2024-09-19T07:49:42.044256+0000","last_peered":"2024-09-19T07:49:42.044256+0000","last_clean":"2024-09-19T07:49:42.044256+0000","last_became_active":"2024-09-19T07:49:41.049748+0000","last_became_peered":"2024-09-19T07:49:41.049748+0000","last_unstale":"2024-09-19T07:49:42.044256+0000","last_undegraded":"2024-09-19T07:49:42.044256+0000","last_fullsized":"2024-09-19T07:49:42.044256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:00.517111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697310+0000","last_change":"2024-09-19T07:49:41.059816+0000","last_active":"2024-09-19T07:49:42.697310+0000","last_peered":"2024-09-19T07:49:42.697310+0000","last_clean":"2024-09-19T07:49:42.697310+0000","last_became_active":"2024-09-19T07:49:41.058730+0000","last_became_peered":"2024-09-19T07:49:41.058730+0000","last_unstale":"2024-09-19T07:49:42.697310+0000","last_undegraded":"2024-09-19T07:49:42.697310+0000","last_fullsized":"2024-09-19T07:49:42.697310+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:57.866953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044472+0000","last_change":"2024-09-19T07:49:41.063887+0000","last_active":"2024-09-19T07:49:42.044472+0000","last_peered":"2024-09-19T07:49:42.044472+0000","last_clean":"2024-09-19T07:49:42.044472+0000","last_became_active":"2024-09-19T07:49:41.063602+0000","last_became_peered":"2024-09-19T07:49:41.063602+0000","last_unstale":"2024-09-19T07:49:42.044472+0000","last_undegraded":"2024-09-19T07:49:42.044472+0000","last_fullsized":"2024-09-19T07:49:42.044472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:23.597186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.044463+0000","last_change":"2024-09-19T07:49:41.042785+0000","last_active":"2024-09-19T07:49:41.044463+0000","last_peered":"2024-09-19T07:49:41.044463+0000","last_clean":"2024-09-19T07:49:41.044463+0000","last_became_active":"2024-09-19T07:49:41.042461+0000","last_became_peered":"2024-09-19T07:49:41.042461+0000","last_unstale":"2024-09-19T07:49:41.044463+0000","last_undegraded":"2024-09-19T07:49:41.044463+0000","last_fullsized":"2024-09-19T07:49:41.044463+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:52.492481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031842+0000","last_change":"2024-09-19T07:49:41.043227+0000","last_active":"2024-09-19T07:49:43.031842+0000","last_peered":"2024-09-19T07:49:43.031842+0000","last_clean":"2024-09-19T07:49:43.031842+0000","last_became_active":"2024-09-19T07:49:41.042613+0000","last_became_peered":"2024-09-19T07:49:41.042613+0000","last_unstale":"2024-09-19T07:49:43.031842+0000","last_undegraded":"2024-09-19T07:49:43.031842+0000","last_fullsized":"2024-09-19T07:49:43.031842+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:18.290122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.696880+0000","last_change":"2024-09-19T07:49:41.059455+0000","last_active":"2024-09-19T07:49:42.696880+0000","last_peered":"2024-09-19T07:49:42.696880+0000","last_clean":"2024-09-19T07:49:42.696880+0000","last_became_active":"2024-09-19T07:49:41.056688+0000","last_became_peered":"2024-09-19T07:49:41.056688+0000","last_unstale":"2024-09-19T07:49:42.696880+0000","last_undegraded":"2024-09-19T07:49:42.696880+0000","last_fullsized":"2024-09-19T07:49:42.696880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.574881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824275+0000","last_change":"2024-09-19T07:49:41.052008+0000","last_active":"2024-09-19T07:49:42.824275+0000","last_peered":"2024-09-19T07:49:42.824275+0000","last_clean":"2024-09-19T07:49:42.824275+0000","last_became_active":"2024-09-19T07:49:41.051736+0000","last_became_peered":"2024-09-19T07:49:41.051736+0000","last_unstale":"2024-09-19T07:49:42.824275+0000","last_undegraded":"2024-09-19T07:49:42.824275+0000","last_fullsized":"2024-09-19T07:49:42.824275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:11.504530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.053425+0000","last_change":"2024-09-19T07:49:41.050945+0000","last_active":"2024-09-19T07:49:41.053425+0000","last_peered":"2024-09-19T07:49:41.053425+0000","last_clean":"2024-09-19T07:49:41.053425+0000","last_became_active":"2024-09-19T07:49:41.050582+0000","last_became_peered":"2024-09-19T07:49:41.050582+0000","last_unstale":"2024-09-19T07:49:41.053425+0000","last_undegraded":"2024-09-19T07:49:41.053425+0000","last_fullsized":"2024-09-19T07:49:41.053425+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:08.228920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.042978+0000","last_change":"2024-09-19T07:49:41.047145+0000","last_active":"2024-09-19T07:49:42.042978+0000","last_peered":"2024-09-19T07:49:42.042978+0000","last_clean":"2024-09-19T07:49:42.042978+0000","last_became_active":"2024-09-19T07:49:41.046856+0000","last_became_peered":"2024-09-19T07:49:41.046856+0000","last_unstale":"2024-09-19T07:49:42.042978+0000","last_undegraded":"2024-09-19T07:49:42.042978+0000","last_fullsized":"2024-09-19T07:49:42.042978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:29.520999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031420+0000","last_change":"2024-09-19T07:49:41.040335+0000","last_active":"2024-09-19T07:49:43.031420+0000","last_peered":"2024-09-19T07:49:43.031420+0000","last_clean":"2024-09-19T07:49:43.031420+0000","last_became_active":"2024-09-19T07:49:41.040005+0000","last_became_peered":"2024-09-19T07:49:41.040005+0000","last_unstale":"2024-09-19T07:49:43.031420+0000","last_undegraded":"2024-09-19T07:49:43.031420+0000","last_fullsized":"2024-09-19T07:49:43.031420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:06.484754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.045010+0000","last_change":"2024-09-19T07:49:41.043861+0000","last_active":"2024-09-19T07:49:41.045010+0000","last_peered":"2024-09-19T07:49:41.045010+0000","last_clean":"2024-09-19T07:49:41.045010+0000","last_became_active":"2024-09-19T07:49:41.043556+0000","last_became_peered":"2024-09-19T07:49:41.043556+0000","last_unstale":"2024-09-19T07:49:41.045010+0000","last_undegraded":"2024-09-19T07:49:41.045010+0000","last_fullsized":"2024-09-19T07:49:41.045010+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:58:22.896865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044704+0000","last_change":"2024-09-19T07:49:41.057841+0000","last_active":"2024-09-19T07:49:42.044704+0000","last_peered":"2024-09-19T07:49:42.044704+0000","last_clean":"2024-09-19T07:49:42.044704+0000","last_became_active":"2024-09-19T07:49:41.057559+0000","last_became_peered":"2024-09-19T07:49:41.057559+0000","last_unstale":"2024-09-19T07:49:42.044704+0000","last_undegraded":"2024-09-19T07:49:42.044704+0000","last_fullsized":"2024-09-19T07:49:42.044704+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:39.777331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.046639+0000","last_change":"2024-09-19T07:49:41.045056+0000","last_active":"2024-09-19T07:49:41.046639+0000","last_peered":"2024-09-19T07:49:41.046639+0000","last_clean":"2024-09-19T07:49:41.046639+0000","last_became_active":"2024-09-19T07:49:41.044546+0000","last_became_peered":"2024-09-19T07:49:41.044546+0000","last_unstale":"2024-09-19T07:49:41.046639+0000","last_undegraded":"2024-09-19T07:49:41.046639+0000","last_fullsized":"2024-09-19T07:49:41.046639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:12.145979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044362+0000","last_change":"2024-09-19T07:49:41.047175+0000","last_active":"2024-09-19T07:49:42.044362+0000","last_peered":"2024-09-19T07:49:42.044362+0000","last_clean":"2024-09-19T07:49:42.044362+0000","last_became_active":"2024-09-19T07:49:41.046748+0000","last_became_peered":"2024-09-19T07:49:41.046748+0000","last_unstale":"2024-09-19T07:49:42.044362+0000","last_undegraded":"2024-09-19T07:49:42.044362+0000","last_fullsized":"2024-09-19T07:49:42.044362+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:57.849422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.696809+0000","last_change":"2024-09-19T07:49:41.056104+0000","last_active":"2024-09-19T07:49:42.696809+0000","last_peered":"2024-09-19T07:49:42.696809+0000","last_clean":"2024-09-19T07:49:42.696809+0000","last_became_active":"2024-09-19T07:49:41.055815+0000","last_became_peered":"2024-09-19T07:49:41.055815+0000","last_unstale":"2024-09-19T07:49:42.696809+0000","last_undegraded":"2024-09-19T07:49:42.696809+0000","last_fullsized":"2024-09-19T07:49:42.696809+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:41.384540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697238+0000","last_change":"2024-09-19T07:49:41.056003+0000","last_active":"2024-09-19T07:49:42.697238+0000","last_peered":"2024-09-19T07:49:42.697238+0000","last_clean":"2024-09-19T07:49:42.697238+0000","last_became_active":"2024-09-19T07:49:41.055680+0000","last_became_peered":"2024-09-19T07:49:41.055680+0000","last_unstale":"2024-09-19T07:49:42.697238+0000","last_undegraded":"2024-09-19T07:49:42.697238+0000","last_fullsized":"2024-09-19T07:49:42.697238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:55.331851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044459+0000","last_change":"2024-09-19T07:49:41.062788+0000","last_active":"2024-09-19T07:49:42.044459+0000","last_peered":"2024-09-19T07:49:42.044459+0000","last_clean":"2024-09-19T07:49:42.044459+0000","last_became_active":"2024-09-19T07:49:41.062477+0000","last_became_peered":"2024-09-19T07:49:41.062477+0000","last_unstale":"2024-09-19T07:49:42.044459+0000","last_undegraded":"2024-09-19T07:49:42.044459+0000","last_fullsized":"2024-09-19T07:49:42.044459+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:41.384540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824381+0000","last_change":"2024-09-19T07:49:41.050573+0000","last_active":"2024-09-19T07:49:42.824381+0000","last_peered":"2024-09-19T07:49:42.824381+0000","last_clean":"2024-09-19T07:49:42.824381+0000","last_became_active":"2024-09-19T07:49:41.050277+0000","last_became_peered":"2024-09-19T07:49:41.050277+0000","last_unstale":"2024-09-19T07:49:42.824381+0000","last_undegraded":"2024-09-19T07:49:42.824381+0000","last_fullsized":"2024-09-19T07:49:42.824381+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:45.208388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.053494+0000","last_change":"2024-09-19T07:49:41.049862+0000","last_active":"2024-09-19T07:49:41.053494+0000","last_peered":"2024-09-19T07:49:41.053494+0000","last_clean":"2024-09-19T07:49:41.053494+0000","last_became_active":"2024-09-19T07:49:41.047151+0000","last_became_peered":"2024-09-19T07:49:41.047151+0000","last_unstale":"2024-09-19T07:49:41.053494+0000","last_undegraded":"2024-09-19T07:49:41.053494+0000","last_fullsized":"2024-09-19T07:49:41.053494+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:21.022897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697050+0000","last_change":"2024-09-19T07:49:41.067158+0000","last_active":"2024-09-19T07:49:42.697050+0000","last_peered":"2024-09-19T07:49:42.697050+0000","last_clean":"2024-09-19T07:49:42.697050+0000","last_became_active":"2024-09-19T07:49:41.066475+0000","last_became_peered":"2024-09-19T07:49:41.066475+0000","last_unstale":"2024-09-19T07:49:42.697050+0000","last_undegraded":"2024-09-19T07:49:42.697050+0000","last_fullsized":"2024-09-19T07:49:42.697050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:11.504530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824345+0000","last_change":"2024-09-19T07:49:41.067574+0000","last_active":"2024-09-19T07:49:42.824345+0000","last_peered":"2024-09-19T07:49:42.824345+0000","last_clean":"2024-09-19T07:49:42.824345+0000","last_became_active":"2024-09-19T07:49:41.067305+0000","last_became_peered":"2024-09-19T07:49:41.067305+0000","last_unstale":"2024-09-19T07:49:42.824345+0000","last_undegraded":"2024-09-19T07:49:42.824345+0000","last_fullsized":"2024-09-19T07:49:42.824345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:55.331851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.042974+0000","last_change":"2024-09-19T07:49:41.052142+0000","last_active":"2024-09-19T07:49:42.042974+0000","last_peered":"2024-09-19T07:49:42.042974+0000","last_clean":"2024-09-19T07:49:42.042974+0000","last_became_active":"2024-09-19T07:49:41.051817+0000","last_became_peered":"2024-09-19T07:49:41.051817+0000","last_unstale":"2024-09-19T07:49:42.042974+0000","last_undegraded":"2024-09-19T07:49:42.042974+0000","last_fullsized":"2024-09-19T07:49:42.042974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:25.748958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824297+0000","last_change":"2024-09-19T07:49:41.059595+0000","last_active":"2024-09-19T07:49:42.824297+0000","last_peered":"2024-09-19T07:49:42.824297+0000","last_clean":"2024-09-19T07:49:42.824297+0000","last_became_active":"2024-09-19T07:49:41.058118+0000","last_became_peered":"2024-09-19T07:49:41.058118+0000","last_unstale":"2024-09-19T07:49:42.824297+0000","last_undegraded":"2024-09-19T07:49:42.824297+0000","last_fullsized":"2024-09-19T07:49:42.824297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:40:04.453626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.042975+0000","last_change":"2024-09-19T07:49:41.046439+0000","last_active":"2024-09-19T07:49:42.042975+0000","last_peered":"2024-09-19T07:49:42.042975+0000","last_clean":"2024-09-19T07:49:42.042975+0000","last_became_active":"2024-09-19T07:49:41.046121+0000","last_became_peered":"2024-09-19T07:49:41.046121+0000","last_unstale":"2024-09-19T07:49:42.042975+0000","last_undegraded":"2024-09-19T07:49:42.042975+0000","last_fullsized":"2024-09-19T07:49:42.042975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:33.713515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.065291+0000","last_change":"2024-09-19T07:49:40.044943+0000","last_active":"2024-09-19T07:49:41.065291+0000","last_peered":"2024-09-19T07:49:41.065291+0000","last_clean":"2024-09-19T07:49:41.065291+0000","last_became_active":"2024-09-19T07:49:40.044658+0000","last_became_peered":"2024-09-19T07:49:40.044658+0000","last_unstale":"2024-09-19T07:49:41.065291+0000","last_undegraded":"2024-09-19T07:49:41.065291+0000","last_fullsized":"2024-09-19T07:49:41.065291+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:16.643861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.696880+0000","last_change":"2024-09-19T07:49:41.056987+0000","last_active":"2024-09-19T07:49:42.696880+0000","last_peered":"2024-09-19T07:49:42.696880+0000","last_clean":"2024-09-19T07:49:42.696880+0000","last_became_active":"2024-09-19T07:49:41.056334+0000","last_became_peered":"2024-09-19T07:49:41.056334+0000","last_unstale":"2024-09-19T07:49:42.696880+0000","last_undegraded":"2024-09-19T07:49:42.696880+0000","last_fullsized":"2024-09-19T07:49:42.696880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:23.597186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824466+0000","last_change":"2024-09-19T07:49:40.040869+0000","last_active":"2024-09-19T07:49:42.824466+0000","last_peered":"2024-09-19T07:49:42.824466+0000","last_clean":"2024-09-19T07:49:42.824466+0000","last_became_active":"2024-09-19T07:49:40.040288+0000","last_became_peered":"2024-09-19T07:49:40.040288+0000","last_unstale":"2024-09-19T07:49:42.824466+0000","last_undegraded":"2024-09-19T07:49:42.824466+0000","last_fullsized":"2024-09-19T07:49:42.824466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:06.733002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044371+0000","last_change":"2024-09-19T07:49:41.050647+0000","last_active":"2024-09-19T07:49:42.044371+0000","last_peered":"2024-09-19T07:49:42.044371+0000","last_clean":"2024-09-19T07:49:42.044371+0000","last_became_active":"2024-09-19T07:49:41.050381+0000","last_became_peered":"2024-09-19T07:49:41.050381+0000","last_unstale":"2024-09-19T07:49:42.044371+0000","last_undegraded":"2024-09-19T07:49:42.044371+0000","last_fullsized":"2024-09-19T07:49:42.044371+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:10.140351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824359+0000","last_change":"2024-09-19T07:49:40.044368+0000","last_active":"2024-09-19T07:49:42.824359+0000","last_peered":"2024-09-19T07:49:42.824359+0000","last_clean":"2024-09-19T07:49:42.824359+0000","last_became_active":"2024-09-19T07:49:40.044097+0000","last_became_peered":"2024-09-19T07:49:40.044097+0000","last_unstale":"2024-09-19T07:49:42.824359+0000","last_undegraded":"2024-09-19T07:49:42.824359+0000","last_fullsized":"2024-09-19T07:49:42.824359+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:52.931623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044900+0000","last_change":"2024-09-19T07:49:41.058801+0000","last_active":"2024-09-19T07:49:42.044900+0000","last_peered":"2024-09-19T07:49:42.044900+0000","last_clean":"2024-09-19T07:49:42.044900+0000","last_became_active":"2024-09-19T07:49:41.057967+0000","last_became_peered":"2024-09-19T07:49:41.057967+0000","last_unstale":"2024-09-19T07:49:42.044900+0000","last_undegraded":"2024-09-19T07:49:42.044900+0000","last_fullsized":"2024-09-19T07:49:42.044900+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:13.724199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.030964+0000","last_change":"2024-09-19T07:49:40.036859+0000","last_active":"2024-09-19T07:49:43.030964+0000","last_peered":"2024-09-19T07:49:43.030964+0000","last_clean":"2024-09-19T07:49:43.030964+0000","last_became_active":"2024-09-19T07:49:40.036562+0000","last_became_peered":"2024-09-19T07:49:40.036562+0000","last_unstale":"2024-09-19T07:49:43.030964+0000","last_undegraded":"2024-09-19T07:49:43.030964+0000","last_fullsized":"2024-09-19T07:49:43.030964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:22.957733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697063+0000","last_change":"2024-09-19T07:49:41.064382+0000","last_active":"2024-09-19T07:49:42.697063+0000","last_peered":"2024-09-19T07:49:42.697063+0000","last_clean":"2024-09-19T07:49:42.697063+0000","last_became_active":"2024-09-19T07:49:41.063213+0000","last_became_peered":"2024-09-19T07:49:41.063213+0000","last_unstale":"2024-09-19T07:49:42.697063+0000","last_undegraded":"2024-09-19T07:49:42.697063+0000","last_fullsized":"2024-09-19T07:49:42.697063+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:40:04.453626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044934+0000","last_change":"2024-09-19T07:49:40.036204+0000","last_active":"2024-09-19T07:49:42.044934+0000","last_peered":"2024-09-19T07:49:42.044934+0000","last_clean":"2024-09-19T07:49:42.044934+0000","last_became_active":"2024-09-19T07:49:40.035901+0000","last_became_peered":"2024-09-19T07:49:40.035901+0000","last_unstale":"2024-09-19T07:49:42.044934+0000","last_undegraded":"2024-09-19T07:49:42.044934+0000","last_fullsized":"2024-09-19T07:49:42.044934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:53:05.590001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697349+0000","last_change":"2024-09-19T07:49:41.063471+0000","last_active":"2024-09-19T07:49:42.697349+0000","last_peered":"2024-09-19T07:49:42.697349+0000","last_clean":"2024-09-19T07:49:42.697349+0000","last_became_active":"2024-09-19T07:49:41.063182+0000","last_became_peered":"2024-09-19T07:49:41.063182+0000","last_unstale":"2024-09-19T07:49:42.697349+0000","last_undegraded":"2024-09-19T07:49:42.697349+0000","last_fullsized":"2024-09-19T07:49:42.697349+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:20.443319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.043998+0000","last_change":"2024-09-19T07:49:40.042284+0000","last_active":"2024-09-19T07:49:42.043998+0000","last_peered":"2024-09-19T07:49:42.043998+0000","last_clean":"2024-09-19T07:49:42.043998+0000","last_became_active":"2024-09-19T07:49:40.041940+0000","last_became_peered":"2024-09-19T07:49:40.041940+0000","last_unstale":"2024-09-19T07:49:42.043998+0000","last_undegraded":"2024-09-19T07:49:42.043998+0000","last_fullsized":"2024-09-19T07:49:42.043998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:50.820893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044943+0000","last_change":"2024-09-19T07:49:41.043055+0000","last_active":"2024-09-19T07:49:42.044943+0000","last_peered":"2024-09-19T07:49:42.044943+0000","last_clean":"2024-09-19T07:49:42.044943+0000","last_became_active":"2024-09-19T07:49:41.042761+0000","last_became_peered":"2024-09-19T07:49:41.042761+0000","last_unstale":"2024-09-19T07:49:42.044943+0000","last_undegraded":"2024-09-19T07:49:42.044943+0000","last_fullsized":"2024-09-19T07:49:42.044943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:03:42.763827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.696995+0000","last_change":"2024-09-19T07:49:40.040534+0000","last_active":"2024-09-19T07:49:42.696995+0000","last_peered":"2024-09-19T07:49:42.696995+0000","last_clean":"2024-09-19T07:49:42.696995+0000","last_became_active":"2024-09-19T07:49:40.040264+0000","last_became_peered":"2024-09-19T07:49:40.040264+0000","last_unstale":"2024-09-19T07:49:42.696995+0000","last_undegraded":"2024-09-19T07:49:42.696995+0000","last_fullsized":"2024-09-19T07:49:42.696995+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:15.151743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031287+0000","last_change":"2024-09-19T07:49:41.050953+0000","last_active":"2024-09-19T07:49:43.031287+0000","last_peered":"2024-09-19T07:49:43.031287+0000","last_clean":"2024-09-19T07:49:43.031287+0000","last_became_active":"2024-09-19T07:49:41.050694+0000","last_became_peered":"2024-09-19T07:49:41.050694+0000","last_unstale":"2024-09-19T07:49:43.031287+0000","last_undegraded":"2024-09-19T07:49:43.031287+0000","last_fullsized":"2024-09-19T07:49:43.031287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:16.156579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.038542+0000","last_change":"2024-09-19T07:49:40.036295+0000","last_active":"2024-09-19T07:49:41.038542+0000","last_peered":"2024-09-19T07:49:41.038542+0000","last_clean":"2024-09-19T07:49:41.038542+0000","last_became_active":"2024-09-19T07:49:40.035972+0000","last_became_peered":"2024-09-19T07:49:40.035972+0000","last_unstale":"2024-09-19T07:49:41.038542+0000","last_undegraded":"2024-09-19T07:49:41.038542+0000","last_fullsized":"2024-09-19T07:49:41.038542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:26.811584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.696808+0000","last_change":"2024-09-19T07:49:41.056640+0000","last_active":"2024-09-19T07:49:42.696808+0000","last_peered":"2024-09-19T07:49:42.696808+0000","last_clean":"2024-09-19T07:49:42.696808+0000","last_became_active":"2024-09-19T07:49:41.056359+0000","last_became_peered":"2024-09-19T07:49:41.056359+0000","last_unstale":"2024-09-19T07:49:42.696808+0000","last_undegraded":"2024-09-19T07:49:42.696808+0000","last_fullsized":"2024-09-19T07:49:42.696808+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:37.745819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824560+0000","last_change":"2024-09-19T07:49:40.040903+0000","last_active":"2024-09-19T07:49:42.824560+0000","last_peered":"2024-09-19T07:49:42.824560+0000","last_clean":"2024-09-19T07:49:42.824560+0000","last_became_active":"2024-09-19T07:49:40.040392+0000","last_became_peered":"2024-09-19T07:49:40.040392+0000","last_unstale":"2024-09-19T07:49:42.824560+0000","last_undegraded":"2024-09-19T07:49:42.824560+0000","last_fullsized":"2024-09-19T07:49:42.824560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:14.482826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044231+0000","last_change":"2024-09-19T07:49:41.049199+0000","last_active":"2024-09-19T07:49:42.044231+0000","last_peered":"2024-09-19T07:49:42.044231+0000","last_clean":"2024-09-19T07:49:42.044231+0000","last_became_active":"2024-09-19T07:49:41.048933+0000","last_became_peered":"2024-09-19T07:49:41.048933+0000","last_unstale":"2024-09-19T07:49:42.044231+0000","last_undegraded":"2024-09-19T07:49:42.044231+0000","last_fullsized":"2024-09-19T07:49:42.044231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:28.535549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044330+0000","last_change":"2024-09-19T07:49:40.041111+0000","last_active":"2024-09-19T07:49:42.044330+0000","last_peered":"2024-09-19T07:49:42.044330+0000","last_clean":"2024-09-19T07:49:42.044330+0000","last_became_active":"2024-09-19T07:49:40.040827+0000","last_became_peered":"2024-09-19T07:49:40.040827+0000","last_unstale":"2024-09-19T07:49:42.044330+0000","last_undegraded":"2024-09-19T07:49:42.044330+0000","last_fullsized":"2024-09-19T07:49:42.044330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:52.931623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.032146+0000","last_change":"2024-09-19T07:49:41.043019+0000","last_active":"2024-09-19T07:49:43.032146+0000","last_peered":"2024-09-19T07:49:43.032146+0000","last_clean":"2024-09-19T07:49:43.032146+0000","last_became_active":"2024-09-19T07:49:41.042246+0000","last_became_peered":"2024-09-19T07:49:41.042246+0000","last_unstale":"2024-09-19T07:49:43.032146+0000","last_undegraded":"2024-09-19T07:49:43.032146+0000","last_fullsized":"2024-09-19T07:49:43.032146+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:38.550958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.066504+0000","last_change":"2024-09-19T07:49:40.040472+0000","last_active":"2024-09-19T07:49:41.066504+0000","last_peered":"2024-09-19T07:49:41.066504+0000","last_clean":"2024-09-19T07:49:41.066504+0000","last_became_active":"2024-09-19T07:49:40.040227+0000","last_became_peered":"2024-09-19T07:49:40.040227+0000","last_unstale":"2024-09-19T07:49:41.066504+0000","last_undegraded":"2024-09-19T07:49:41.066504+0000","last_fullsized":"2024-09-19T07:49:41.066504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:32.130002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.043090+0000","last_change":"2024-09-19T07:49:41.067649+0000","last_active":"2024-09-19T07:49:42.043090+0000","last_peered":"2024-09-19T07:49:42.043090+0000","last_clean":"2024-09-19T07:49:42.043090+0000","last_became_active":"2024-09-19T07:49:41.067355+0000","last_became_peered":"2024-09-19T07:49:41.067355+0000","last_unstale":"2024-09-19T07:49:42.043090+0000","last_undegraded":"2024-09-19T07:49:42.043090+0000","last_fullsized":"2024-09-19T07:49:42.043090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:44.923391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"80'6","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031152+0000","last_change":"2024-09-19T07:49:40.034442+0000","last_active":"2024-09-19T07:49:43.031152+0000","last_peered":"2024-09-19T07:49:43.031152+0000","last_clean":"2024-09-19T07:49:43.031152+0000","last_became_active":"2024-09-19T07:49:40.034168+0000","last_became_peered":"2024-09-19T07:49:40.034168+0000","last_unstale":"2024-09-19T07:49:43.031152+0000","last_undegraded":"2024-09-19T07:49:43.031152+0000","last_fullsized":"2024-09-19T07:49:43.031152+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:29.408268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044627+0000","last_change":"2024-09-19T07:49:41.046137+0000","last_active":"2024-09-19T07:49:42.044627+0000","last_peered":"2024-09-19T07:49:42.044627+0000","last_clean":"2024-09-19T07:49:42.044627+0000","last_became_active":"2024-09-19T07:49:41.045826+0000","last_became_peered":"2024-09-19T07:49:41.045826+0000","last_unstale":"2024-09-19T07:49:42.044627+0000","last_undegraded":"2024-09-19T07:49:42.044627+0000","last_fullsized":"2024-09-19T07:49:42.044627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.574881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044414+0000","last_change":"2024-09-19T07:49:40.039449+0000","last_active":"2024-09-19T07:49:42.044414+0000","last_peered":"2024-09-19T07:49:42.044414+0000","last_clean":"2024-09-19T07:49:42.044414+0000","last_became_active":"2024-09-19T07:49:40.039127+0000","last_became_peered":"2024-09-19T07:49:40.039127+0000","last_unstale":"2024-09-19T07:49:42.044414+0000","last_undegraded":"2024-09-19T07:49:42.044414+0000","last_fullsized":"2024-09-19T07:49:42.044414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:41.786978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031061+0000","last_change":"2024-09-19T07:49:41.052856+0000","last_active":"2024-09-19T07:49:43.031061+0000","last_peered":"2024-09-19T07:49:43.031061+0000","last_clean":"2024-09-19T07:49:43.031061+0000","last_became_active":"2024-09-19T07:49:41.052591+0000","last_became_peered":"2024-09-19T07:49:41.052591+0000","last_unstale":"2024-09-19T07:49:43.031061+0000","last_undegraded":"2024-09-19T07:49:43.031061+0000","last_fullsized":"2024-09-19T07:49:43.031061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:25.748958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824123+0000","last_change":"2024-09-19T07:49:40.040152+0000","last_active":"2024-09-19T07:49:42.824123+0000","last_peered":"2024-09-19T07:49:42.824123+0000","last_clean":"2024-09-19T07:49:42.824123+0000","last_became_active":"2024-09-19T07:49:40.039848+0000","last_became_peered":"2024-09-19T07:49:40.039848+0000","last_unstale":"2024-09-19T07:49:42.824123+0000","last_undegraded":"2024-09-19T07:49:42.824123+0000","last_fullsized":"2024-09-19T07:49:42.824123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:58.425054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031311+0000","last_change":"2024-09-19T07:49:41.054387+0000","last_active":"2024-09-19T07:49:43.031311+0000","last_peered":"2024-09-19T07:49:43.031311+0000","last_clean":"2024-09-19T07:49:43.031311+0000","last_became_active":"2024-09-19T07:49:41.054125+0000","last_became_peered":"2024-09-19T07:49:41.054125+0000","last_unstale":"2024-09-19T07:49:43.031311+0000","last_undegraded":"2024-09-19T07:49:43.031311+0000","last_fullsized":"2024-09-19T07:49:43.031311+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:10.687061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.038042+0000","last_change":"2024-09-19T07:49:40.043587+0000","last_active":"2024-09-19T07:49:41.038042+0000","last_peered":"2024-09-19T07:49:41.038042+0000","last_clean":"2024-09-19T07:49:41.038042+0000","last_became_active":"2024-09-19T07:49:40.043339+0000","last_became_peered":"2024-09-19T07:49:40.043339+0000","last_unstale":"2024-09-19T07:49:41.038042+0000","last_undegraded":"2024-09-19T07:49:41.038042+0000","last_fullsized":"2024-09-19T07:49:41.038042+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:18.122629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031366+0000","last_change":"2024-09-19T07:49:41.062023+0000","last_active":"2024-09-19T07:49:43.031366+0000","last_peered":"2024-09-19T07:49:43.031366+0000","last_clean":"2024-09-19T07:49:43.031366+0000","last_became_active":"2024-09-19T07:49:41.059136+0000","last_became_peered":"2024-09-19T07:49:41.059136+0000","last_unstale":"2024-09-19T07:49:43.031366+0000","last_undegraded":"2024-09-19T07:49:43.031366+0000","last_fullsized":"2024-09-19T07:49:43.031366+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:18.521095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"80'8","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.696914+0000","last_change":"2024-09-19T07:49:40.033351+0000","last_active":"2024-09-19T07:49:42.696914+0000","last_peered":"2024-09-19T07:49:42.696914+0000","last_clean":"2024-09-19T07:49:42.696914+0000","last_became_active":"2024-09-19T07:49:40.033076+0000","last_became_peered":"2024-09-19T07:49:40.033076+0000","last_unstale":"2024-09-19T07:49:42.696914+0000","last_undegraded":"2024-09-19T07:49:42.696914+0000","last_fullsized":"2024-09-19T07:49:42.696914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:28.516163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031569+0000","last_change":"2024-09-19T07:49:41.044278+0000","last_active":"2024-09-19T07:49:43.031569+0000","last_peered":"2024-09-19T07:49:43.031569+0000","last_clean":"2024-09-19T07:49:43.031569+0000","last_became_active":"2024-09-19T07:49:41.043995+0000","last_became_peered":"2024-09-19T07:49:41.043995+0000","last_unstale":"2024-09-19T07:49:43.031569+0000","last_undegraded":"2024-09-19T07:49:43.031569+0000","last_fullsized":"2024-09-19T07:49:43.031569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:24.634859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824655+0000","last_change":"2024-09-19T07:49:40.044469+0000","last_active":"2024-09-19T07:49:42.824655+0000","last_peered":"2024-09-19T07:49:42.824655+0000","last_clean":"2024-09-19T07:49:42.824655+0000","last_became_active":"2024-09-19T07:49:40.043722+0000","last_became_peered":"2024-09-19T07:49:40.043722+0000","last_unstale":"2024-09-19T07:49:42.824655+0000","last_undegraded":"2024-09-19T07:49:42.824655+0000","last_fullsized":"2024-09-19T07:49:42.824655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:29:20.900485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.054002+0000","last_change":"2024-09-19T07:49:41.052193+0000","last_active":"2024-09-19T07:49:41.054002+0000","last_peered":"2024-09-19T07:49:41.054002+0000","last_clean":"2024-09-19T07:49:41.054002+0000","last_became_active":"2024-09-19T07:49:41.051891+0000","last_became_peered":"2024-09-19T07:49:41.051891+0000","last_unstale":"2024-09-19T07:49:41.054002+0000","last_undegraded":"2024-09-19T07:49:41.054002+0000","last_fullsized":"2024-09-19T07:49:41.054002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:17.335309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044792+0000","last_change":"2024-09-19T07:49:40.033040+0000","last_active":"2024-09-19T07:49:42.044792+0000","last_peered":"2024-09-19T07:49:42.044792+0000","last_clean":"2024-09-19T07:49:42.044792+0000","last_became_active":"2024-09-19T07:49:40.032755+0000","last_became_peered":"2024-09-19T07:49:40.032755+0000","last_unstale":"2024-09-19T07:49:42.044792+0000","last_undegraded":"2024-09-19T07:49:42.044792+0000","last_fullsized":"2024-09-19T07:49:42.044792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:16.215029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.061945+0000","last_change":"2024-09-19T07:49:41.057688+0000","last_active":"2024-09-19T07:49:41.061945+0000","last_peered":"2024-09-19T07:49:41.061945+0000","last_clean":"2024-09-19T07:49:41.061945+0000","last_became_active":"2024-09-19T07:49:41.057382+0000","last_became_peered":"2024-09-19T07:49:41.057382+0000","last_unstale":"2024-09-19T07:49:41.061945+0000","last_undegraded":"2024-09-19T07:49:41.061945+0000","last_fullsized":"2024-09-19T07:49:41.061945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:47.973441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.064816+0000","last_change":"2024-09-19T07:49:40.040575+0000","last_active":"2024-09-19T07:49:41.064816+0000","last_peered":"2024-09-19T07:49:41.064816+0000","last_clean":"2024-09-19T07:49:41.064816+0000","last_became_active":"2024-09-19T07:49:40.040186+0000","last_became_peered":"2024-09-19T07:49:40.040186+0000","last_unstale":"2024-09-19T07:49:41.064816+0000","last_undegraded":"2024-09-19T07:49:41.064816+0000","last_fullsized":"2024-09-19T07:49:41.064816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:12.523848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.032167+0000","last_change":"2024-09-19T07:49:41.045026+0000","last_active":"2024-09-19T07:49:43.032167+0000","last_peered":"2024-09-19T07:49:43.032167+0000","last_clean":"2024-09-19T07:49:43.032167+0000","last_became_active":"2024-09-19T07:49:41.044585+0000","last_became_peered":"2024-09-19T07:49:41.044585+0000","last_unstale":"2024-09-19T07:49:43.032167+0000","last_undegraded":"2024-09-19T07:49:43.032167+0000","last_fullsized":"2024-09-19T07:49:43.032167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:23.771644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.042814+0000","last_change":"2024-09-19T07:49:40.036123+0000","last_active":"2024-09-19T07:49:42.042814+0000","last_peered":"2024-09-19T07:49:42.042814+0000","last_clean":"2024-09-19T07:49:42.042814+0000","last_became_active":"2024-09-19T07:49:40.035838+0000","last_became_peered":"2024-09-19T07:49:40.035838+0000","last_unstale":"2024-09-19T07:49:42.042814+0000","last_undegraded":"2024-09-19T07:49:42.042814+0000","last_fullsized":"2024-09-19T07:49:42.042814+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:09.135514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697285+0000","last_change":"2024-09-19T07:49:41.060216+0000","last_active":"2024-09-19T07:49:42.697285+0000","last_peered":"2024-09-19T07:49:42.697285+0000","last_clean":"2024-09-19T07:49:42.697285+0000","last_became_active":"2024-09-19T07:49:41.059811+0000","last_became_peered":"2024-09-19T07:49:41.059811+0000","last_unstale":"2024-09-19T07:49:42.697285+0000","last_undegraded":"2024-09-19T07:49:42.697285+0000","last_fullsized":"2024-09-19T07:49:42.697285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:07.197132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031963+0000","last_change":"2024-09-19T07:49:40.041426+0000","last_active":"2024-09-19T07:49:43.031963+0000","last_peered":"2024-09-19T07:49:43.031963+0000","last_clean":"2024-09-19T07:49:43.031963+0000","last_became_active":"2024-09-19T07:49:40.041174+0000","last_became_peered":"2024-09-19T07:49:40.041174+0000","last_unstale":"2024-09-19T07:49:43.031963+0000","last_undegraded":"2024-09-19T07:49:43.031963+0000","last_fullsized":"2024-09-19T07:49:43.031963+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:44.203551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697463+0000","last_change":"2024-09-19T07:49:41.060569+0000","last_active":"2024-09-19T07:49:42.697463+0000","last_peered":"2024-09-19T07:49:42.697463+0000","last_clean":"2024-09-19T07:49:42.697463+0000","last_became_active":"2024-09-19T07:49:41.060285+0000","last_became_peered":"2024-09-19T07:49:41.060285+0000","last_unstale":"2024-09-19T07:49:42.697463+0000","last_undegraded":"2024-09-19T07:49:42.697463+0000","last_fullsized":"2024-09-19T07:49:42.697463+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:43.595570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031493+0000","last_change":"2024-09-19T07:49:40.047109+0000","last_active":"2024-09-19T07:49:43.031493+0000","last_peered":"2024-09-19T07:49:43.031493+0000","last_clean":"2024-09-19T07:49:43.031493+0000","last_became_active":"2024-09-19T07:49:40.046800+0000","last_became_peered":"2024-09-19T07:49:40.046800+0000","last_unstale":"2024-09-19T07:49:43.031493+0000","last_undegraded":"2024-09-19T07:49:43.031493+0000","last_fullsized":"2024-09-19T07:49:43.031493+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:56.862116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031389+0000","last_change":"2024-09-19T07:49:41.050201+0000","last_active":"2024-09-19T07:49:43.031389+0000","last_peered":"2024-09-19T07:49:43.031389+0000","last_clean":"2024-09-19T07:49:43.031389+0000","last_became_active":"2024-09-19T07:49:41.049920+0000","last_became_peered":"2024-09-19T07:49:41.049920+0000","last_unstale":"2024-09-19T07:49:43.031389+0000","last_undegraded":"2024-09-19T07:49:43.031389+0000","last_fullsized":"2024-09-19T07:49:43.031389+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:44.923391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031633+0000","last_change":"2024-09-19T07:49:40.040948+0000","last_active":"2024-09-19T07:49:43.031633+0000","last_peered":"2024-09-19T07:49:43.031633+0000","last_clean":"2024-09-19T07:49:43.031633+0000","last_became_active":"2024-09-19T07:49:40.040460+0000","last_became_peered":"2024-09-19T07:49:40.040460+0000","last_unstale":"2024-09-19T07:49:43.031633+0000","last_undegraded":"2024-09-19T07:49:43.031633+0000","last_fullsized":"2024-09-19T07:49:43.031633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:06.192295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.043972+0000","last_change":"2024-09-19T07:49:41.055770+0000","last_active":"2024-09-19T07:49:42.043972+0000","last_peered":"2024-09-19T07:49:42.043972+0000","last_clean":"2024-09-19T07:49:42.043972+0000","last_became_active":"2024-09-19T07:49:41.055506+0000","last_became_peered":"2024-09-19T07:49:41.055506+0000","last_unstale":"2024-09-19T07:49:42.043972+0000","last_undegraded":"2024-09-19T07:49:42.043972+0000","last_fullsized":"2024-09-19T07:49:42.043972+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:03:52.869311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.043939+0000","last_change":"2024-09-19T07:49:40.037347+0000","last_active":"2024-09-19T07:49:42.043939+0000","last_peered":"2024-09-19T07:49:42.043939+0000","last_clean":"2024-09-19T07:49:42.043939+0000","last_became_active":"2024-09-19T07:49:40.037043+0000","last_became_peered":"2024-09-19T07:49:40.037043+0000","last_unstale":"2024-09-19T07:49:42.043939+0000","last_undegraded":"2024-09-19T07:49:42.043939+0000","last_fullsized":"2024-09-19T07:49:42.043939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:26:15.212481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031094+0000","last_change":"2024-09-19T07:49:41.060272+0000","last_active":"2024-09-19T07:49:43.031094+0000","last_peered":"2024-09-19T07:49:43.031094+0000","last_clean":"2024-09-19T07:49:43.031094+0000","last_became_active":"2024-09-19T07:49:41.060004+0000","last_became_peered":"2024-09-19T07:49:41.060004+0000","last_unstale":"2024-09-19T07:49:43.031094+0000","last_undegraded":"2024-09-19T07:49:43.031094+0000","last_fullsized":"2024-09-19T07:49:43.031094+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:15.255291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697421+0000","last_change":"2024-09-19T07:49:40.043434+0000","last_active":"2024-09-19T07:49:42.697421+0000","last_peered":"2024-09-19T07:49:42.697421+0000","last_clean":"2024-09-19T07:49:42.697421+0000","last_became_active":"2024-09-19T07:49:40.042987+0000","last_became_peered":"2024-09-19T07:49:40.042987+0000","last_unstale":"2024-09-19T07:49:42.697421+0000","last_undegraded":"2024-09-19T07:49:42.697421+0000","last_fullsized":"2024-09-19T07:49:42.697421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:09.682224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.044532+0000","last_change":"2024-09-19T07:49:41.049676+0000","last_active":"2024-09-19T07:49:42.044532+0000","last_peered":"2024-09-19T07:49:42.044532+0000","last_clean":"2024-09-19T07:49:42.044532+0000","last_became_active":"2024-09-19T07:49:41.049413+0000","last_became_peered":"2024-09-19T07:49:41.049413+0000","last_unstale":"2024-09-19T07:49:42.044532+0000","last_undegraded":"2024-09-19T07:49:42.044532+0000","last_fullsized":"2024-09-19T07:49:42.044532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:36:20.876800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824499+0000","last_change":"2024-09-19T07:49:40.040556+0000","last_active":"2024-09-19T07:49:42.824499+0000","last_peered":"2024-09-19T07:49:42.824499+0000","last_clean":"2024-09-19T07:49:42.824499+0000","last_became_active":"2024-09-19T07:49:40.040134+0000","last_became_peered":"2024-09-19T07:49:40.040134+0000","last_unstale":"2024-09-19T07:49:42.824499+0000","last_undegraded":"2024-09-19T07:49:42.824499+0000","last_fullsized":"2024-09-19T07:49:42.824499+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:56:28.020397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.697243+0000","last_change":"2024-09-19T07:49:41.066024+0000","last_active":"2024-09-19T07:49:42.697243+0000","last_peered":"2024-09-19T07:49:42.697243+0000","last_clean":"2024-09-19T07:49:42.697243+0000","last_became_active":"2024-09-19T07:49:41.064768+0000","last_became_peered":"2024-09-19T07:49:41.064768+0000","last_unstale":"2024-09-19T07:49:42.697243+0000","last_undegraded":"2024-09-19T07:49:42.697243+0000","last_fullsized":"2024-09-19T07:49:42.697243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:45.208388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824335+0000","last_change":"2024-09-19T07:49:40.042468+0000","last_active":"2024-09-19T07:49:42.824335+0000","last_peered":"2024-09-19T07:49:42.824335+0000","last_clean":"2024-09-19T07:49:42.824335+0000","last_became_active":"2024-09-19T07:49:40.040583+0000","last_became_peered":"2024-09-19T07:49:40.040583+0000","last_unstale":"2024-09-19T07:49:42.824335+0000","last_undegraded":"2024-09-19T07:49:42.824335+0000","last_fullsized":"2024-09-19T07:49:42.824335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:41.786978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031075+0000","last_change":"2024-09-19T07:49:41.059536+0000","last_active":"2024-09-19T07:49:43.031075+0000","last_peered":"2024-09-19T07:49:43.031075+0000","last_clean":"2024-09-19T07:49:43.031075+0000","last_became_active":"2024-09-19T07:49:41.059206+0000","last_became_peered":"2024-09-19T07:49:41.059206+0000","last_unstale":"2024-09-19T07:49:43.031075+0000","last_undegraded":"2024-09-19T07:49:43.031075+0000","last_fullsized":"2024-09-19T07:49:43.031075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:33.713515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.030838+0000","last_change":"2024-09-19T07:49:40.037515+0000","last_active":"2024-09-19T07:49:43.030838+0000","last_peered":"2024-09-19T07:49:43.030838+0000","last_clean":"2024-09-19T07:49:43.030838+0000","last_became_active":"2024-09-19T07:49:40.037233+0000","last_became_peered":"2024-09-19T07:49:40.037233+0000","last_unstale":"2024-09-19T07:49:43.030838+0000","last_undegraded":"2024-09-19T07:49:43.030838+0000","last_fullsized":"2024-09-19T07:49:43.030838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:19:17.728130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031121+0000","last_change":"2024-09-19T07:49:41.046993+0000","last_active":"2024-09-19T07:49:43.031121+0000","last_peered":"2024-09-19T07:49:43.031121+0000","last_clean":"2024-09-19T07:49:43.031121+0000","last_became_active":"2024-09-19T07:49:41.046699+0000","last_became_peered":"2024-09-19T07:49:41.046699+0000","last_unstale":"2024-09-19T07:49:43.031121+0000","last_undegraded":"2024-09-19T07:49:43.031121+0000","last_fullsized":"2024-09-19T07:49:43.031121+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:29.520999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-19T07:49:41.038395+0000","last_change":"2024-09-19T07:49:40.042739+0000","last_active":"2024-09-19T07:49:41.038395+0000","last_peered":"2024-09-19T07:49:41.038395+0000","last_clean":"2024-09-19T07:49:41.038395+0000","last_became_active":"2024-09-19T07:49:40.042465+0000","last_became_peered":"2024-09-19T07:49:40.042465+0000","last_unstale":"2024-09-19T07:49:41.038395+0000","last_undegraded":"2024-09-19T07:49:41.038395+0000","last_fullsized":"2024-09-19T07:49:41.038395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:58.792956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.043061+0000","last_change":"2024-09-19T07:49:41.061934+0000","last_active":"2024-09-19T07:49:42.043061+0000","last_peered":"2024-09-19T07:49:42.043061+0000","last_clean":"2024-09-19T07:49:42.043061+0000","last_became_active":"2024-09-19T07:49:41.061636+0000","last_became_peered":"2024-09-19T07:49:41.061636+0000","last_unstale":"2024-09-19T07:49:42.043061+0000","last_undegraded":"2024-09-19T07:49:42.043061+0000","last_fullsized":"2024-09-19T07:49:42.043061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:10.687061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.031118+0000","last_change":"2024-09-19T07:49:40.043622+0000","last_active":"2024-09-19T07:49:43.031118+0000","last_peered":"2024-09-19T07:49:43.031118+0000","last_clean":"2024-09-19T07:49:43.031118+0000","last_became_active":"2024-09-19T07:49:40.043000+0000","last_became_peered":"2024-09-19T07:49:40.043000+0000","last_unstale":"2024-09-19T07:49:43.031118+0000","last_undegraded":"2024-09-19T07:49:43.031118+0000","last_fullsized":"2024-09-19T07:49:43.031118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:59.813214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.043042+0000","last_change":"2024-09-19T07:49:41.044413+0000","last_active":"2024-09-19T07:49:42.043042+0000","last_peered":"2024-09-19T07:49:42.043042+0000","last_clean":"2024-09-19T07:49:42.043042+0000","last_became_active":"2024-09-19T07:49:41.044111+0000","last_became_peered":"2024-09-19T07:49:41.044111+0000","last_unstale":"2024-09-19T07:49:42.043042+0000","last_undegraded":"2024-09-19T07:49:42.043042+0000","last_fullsized":"2024-09-19T07:49:42.043042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:16.156579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.042653+0000","last_change":"2024-09-19T07:49:40.043335+0000","last_active":"2024-09-19T07:49:42.042653+0000","last_peered":"2024-09-19T07:49:42.042653+0000","last_clean":"2024-09-19T07:49:42.042653+0000","last_became_active":"2024-09-19T07:49:40.043042+0000","last_became_peered":"2024-09-19T07:49:40.043042+0000","last_unstale":"2024-09-19T07:49:42.042653+0000","last_undegraded":"2024-09-19T07:49:42.042653+0000","last_fullsized":"2024-09-19T07:49:42.042653+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:56.844586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824593+0000","last_change":"2024-09-19T07:49:41.055598+0000","last_active":"2024-09-19T07:49:42.824593+0000","last_peered":"2024-09-19T07:49:42.824593+0000","last_clean":"2024-09-19T07:49:42.824593+0000","last_became_active":"2024-09-19T07:49:41.055292+0000","last_became_peered":"2024-09-19T07:49:41.055292+0000","last_unstale":"2024-09-19T07:49:42.824593+0000","last_undegraded":"2024-09-19T07:49:42.824593+0000","last_fullsized":"2024-09-19T07:49:42.824593+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:40.020341+0000","last_clean_scrub_stamp":"2024-09-19T07:49:40.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:07.197132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:43.032029+0000","last_change":"2024-09-19T07:49:40.046770+0000","last_active":"2024-09-19T07:49:43.032029+0000","last_peered":"2024-09-19T07:49:43.032029+0000","last_clean":"2024-09-19T07:49:43.032029+0000","last_became_active":"2024-09-19T07:49:40.046469+0000","last_became_peered":"2024-09-19T07:49:40.046469+0000","last_unstale":"2024-09-19T07:49:43.032029+0000","last_undegraded":"2024-09-19T07:49:43.032029+0000","last_fullsized":"2024-09-19T07:49:43.032029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:49:39.015504+0000","last_clean_scrub_stamp":"2024-09-19T07:49:39.015504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:19.438482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":279,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T07:49:42.824353+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:49:42.824353+0000","last_peered":"2024-09-19T07:49:42.824353+0000","last_clean":"2024-09-19T07:49:42.824353+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:49:42.824353+0000","last_undegraded":"2024-09-19T07:49:42.824353+0000","last_fullsized":"2024-09-19T07:49:42.824353+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99292,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":97726,"kb_avail":94272548,"statfs":{"total":96636764160,"available":96535089152,"internally_reserved":0,"allocated":1581056,"data_stored":1041163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":100071882},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509524,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":112672,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94259168,"statfs":{"total":96636764160,"available":96521388032,"internally_reserved":0,"allocated":995328,"data_stored":452385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":114358726},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509525,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50200,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321640,"statfs":{"total":96636764160,"available":96585359360,"internally_reserved":0,"allocated":987136,"data_stored":450807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509525,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46064,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325776,"statfs":{"total":96636764160,"available":96589594624,"internally_reserved":0,"allocated":1003520,"data_stored":452489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509524,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100704,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271136,"statfs":{"total":96636764160,"available":96533643264,"internally_reserved":0,"allocated":1576960,"data_stored":1041141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542228,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130644,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94241196,"statfs":{"total":96636764160,"available":96502984704,"internally_reserved":0,"allocated":974848,"data_stored":444663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542228,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94620,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94277220,"statfs":{"total":96636764160,"available":96539873280,"internally_reserved":0,"allocated":991232,"data_stored":450885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104284,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267556,"statfs":{"total":96636764160,"available":96529977344,"internally_reserved":0,"allocated":983040,"data_stored":444823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:49:45.396 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T07:49:45.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T07:49:45.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.539+0000 7ffb92077640 1 Processor -- start 2024-09-19T07:49:45.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.539+0000 7ffb92077640 1 -- start start 2024-09-19T07:49:45.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.540+0000 7ffb92077640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffb8c0b5280 0x7ffb8c162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.540+0000 7ffb92077640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb8c15f250 con 0x7ffb8c0d4af0 2024-09-19T07:49:45.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.540+0000 7ffb92077640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb8c15f530 con 0x7ffb8c15e0a0 2024-09-19T07:49:45.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.540+0000 7ffb92077640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ffb8c15f810 con 0x7ffb8c0b5280 2024-09-19T07:49:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.540+0000 7ffb8affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffb8c0d4af0 0x7ffb8c0b4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36082/0 (socket says 172.21.15.16:36082) 2024-09-19T07:49:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.540+0000 7ffb8affd640 1 -- 172.21.15.16:0/830621422 learned_addr learned my addr 172.21.15.16:0/830621422 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8a7fc640 1 -- 172.21.15.16:0/830621422 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2677596160 0 0) 0x7ffb8c15f250 con 0x7ffb8c0d4af0 2024-09-19T07:49:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8a7fc640 1 -- 172.21.15.16:0/830621422 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb74003520 con 0x7ffb8c0d4af0 2024-09-19T07:49:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8bfff640 1 --2- 172.21.15.16:0/830621422 >> v2:172.21.15.146:3300/0 conn(0x7ffb8c0b5280 0x7ffb8c162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8a7fc640 1 -- 172.21.15.16:0/830621422 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 719549005 0 0) 0x7ffb74003520 con 0x7ffb8c0d4af0 2024-09-19T07:49:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8a7fc640 1 -- 172.21.15.16:0/830621422 >> v2:172.21.15.146:3300/0 conn(0x7ffb8c0b5280 msgr2=0x7ffb8c162a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8a7fc640 1 --2- 172.21.15.16:0/830621422 >> v2:172.21.15.146:3300/0 conn(0x7ffb8c0b5280 0x7ffb8c162a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8a7fc640 1 -- 172.21.15.16:0/830621422 >> v1:172.21.15.146:6789/0 conn(0x7ffb8c15e0a0 legacy=0x7ffb8c0d43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8a7fc640 1 -- 172.21.15.16:0/830621422 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c0b4650 con 0x7ffb8c0d4af0 2024-09-19T07:49:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8a7fc640 1 -- 172.21.15.16:0/830621422 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7ffb80001810 con 0x7ffb8c0d4af0 2024-09-19T07:49:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8a7fc640 1 -- 172.21.15.16:0/830621422 >> v1:172.21.15.16:6789/0 conn(0x7ffb8c0d4af0 legacy=0x7ffb8c0b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8a7fc640 1 --2- 172.21.15.16:0/830621422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb740039b0 0x7ffb74003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8a7fc640 1 --2- 172.21.15.16:0/830621422 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb74004800 0x7ffb74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8a7fc640 1 --2- 172.21.15.16:0/830621422 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb740051c0 0x7ffb7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8a7fc640 1 -- 172.21.15.16:0/830621422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c15f810 con 0x7ffb740039b0 2024-09-19T07:49:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8a7fc640 1 -- 172.21.15.16:0/830621422 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7400c0f0 con 0x7ffb74004800 2024-09-19T07:49:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8a7fc640 1 -- 172.21.15.16:0/830621422 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ffb7400c2f0 con 0x7ffb740051c0 2024-09-19T07:49:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8b7fe640 1 --2- 172.21.15.16:0/830621422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb740039b0 0x7ffb74003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8affd640 1 --2- 172.21.15.16:0/830621422 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb74004800 0x7ffb74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8b7fe640 1 -- 172.21.15.16:0/830621422 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb740051c0 msgr2=0x7ffb7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8b7fe640 1 --2- 172.21.15.16:0/830621422 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb740051c0 0x7ffb7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8b7fe640 1 -- 172.21.15.16:0/830621422 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb74004800 msgr2=0x7ffb74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8b7fe640 1 --2- 172.21.15.16:0/830621422 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb74004800 0x7ffb74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8b7fe640 1 -- 172.21.15.16:0/830621422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb74003520 con 0x7ffb740039b0 2024-09-19T07:49:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.541+0000 7ffb8bfff640 1 -- 172.21.15.16:0/830621422 reap_dead start 2024-09-19T07:49:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.542+0000 7ffb8b7fe640 1 --2- 172.21.15.16:0/830621422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb740039b0 0x7ffb74003db0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c003d80 tx=0x7ffb74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85450225560de1f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.542+0000 7ffb8a7fc640 1 -- 172.21.15.16:0/830621422 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c037070 con 0x7ffb740039b0 2024-09-19T07:49:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.542+0000 7ffb8a7fc640 1 -- 172.21.15.16:0/830621422 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7c02a9c0 con 0x7ffb740039b0 2024-09-19T07:49:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.542+0000 7ffb8a7fc640 1 -- 172.21.15.16:0/830621422 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c003980 con 0x7ffb740039b0 2024-09-19T07:49:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.543+0000 7ffb92077640 1 -- 172.21.15.16:0/830621422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb740039b0 msgr2=0x7ffb74003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.543+0000 7ffb92077640 1 --2- 172.21.15.16:0/830621422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb740039b0 0x7ffb74003db0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c003d80 tx=0x7ffb74005830 comp rx=0 tx=0).stop 2024-09-19T07:49:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.543+0000 7ffb92077640 1 -- 172.21.15.16:0/830621422 shutdown_connections 2024-09-19T07:49:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.543+0000 7ffb92077640 1 --2- 172.21.15.16:0/830621422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb740039b0 0x7ffb74003db0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.543+0000 7ffb92077640 1 -- 172.21.15.16:0/830621422 >> 172.21.15.16:0/830621422 conn(0x7ffb8c09d770 msgr2=0x7ffb8c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.543+0000 7ffb92077640 1 -- 172.21.15.16:0/830621422 shutdown_connections 2024-09-19T07:49:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.543+0000 7ffb92077640 1 -- 172.21.15.16:0/830621422 wait complete. 2024-09-19T07:49:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.544+0000 7ffb92077640 1 Processor -- start 2024-09-19T07:49:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.544+0000 7ffb92077640 1 -- start start 2024-09-19T07:49:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.545+0000 7ffb92077640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb740039b0 0x7ffb8c0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.545+0000 7ffb92077640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb8c1f46f0 0x7ffb8c0cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.545+0000 7ffb92077640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb8c1f4b70 0x7ffb8c0cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.545+0000 7ffb92077640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c162e00 con 0x7ffb740039b0 2024-09-19T07:49:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.545+0000 7ffb92077640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c162c80 con 0x7ffb8c1f4b70 2024-09-19T07:49:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.545+0000 7ffb92077640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ffb8c0b4b70 con 0x7ffb8c1f46f0 2024-09-19T07:49:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.545+0000 7ffb8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb740039b0 0x7ffb8c0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.545+0000 7ffb8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb740039b0 0x7ffb8c0ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60918/0 (socket says 172.21.15.16:60918) 2024-09-19T07:49:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.545+0000 7ffb8b7fe640 1 -- 172.21.15.16:0/3465892913 learned_addr learned my addr 172.21.15.16:0/3465892913 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.545+0000 7ffb8bfff640 1 --2- 172.21.15.16:0/3465892913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb8c1f4b70 0x7ffb8c0cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.545+0000 7ffb8affd640 1 --2- 172.21.15.16:0/3465892913 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb8c1f46f0 0x7ffb8c0cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.545+0000 7ffb8b7fe640 1 -- 172.21.15.16:0/3465892913 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb8c1f46f0 msgr2=0x7ffb8c0cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.545+0000 7ffb8b7fe640 1 --2- 172.21.15.16:0/3465892913 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb8c1f46f0 0x7ffb8c0cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.545+0000 7ffb8b7fe640 1 -- 172.21.15.16:0/3465892913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb8c1f4b70 msgr2=0x7ffb8c0cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.545+0000 7ffb8b7fe640 1 --2- 172.21.15.16:0/3465892913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb8c1f4b70 0x7ffb8c0cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.545+0000 7ffb8b7fe640 1 -- 172.21.15.16:0/3465892913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c0cd760 con 0x7ffb740039b0 2024-09-19T07:49:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.546+0000 7ffb8b7fe640 1 --2- 172.21.15.16:0/3465892913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb740039b0 0x7ffb8c0ce520 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c035040 tx=0x7ffb7c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.546+0000 7ffb73fff640 1 -- 172.21.15.16:0/3465892913 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c042020 con 0x7ffb740039b0 2024-09-19T07:49:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.546+0000 7ffb73fff640 1 -- 172.21.15.16:0/3465892913 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7c038040 con 0x7ffb740039b0 2024-09-19T07:49:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.546+0000 7ffb92077640 1 -- 172.21.15.16:0/3465892913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb8c0cda40 con 0x7ffb740039b0 2024-09-19T07:49:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.546+0000 7ffb73fff640 1 -- 172.21.15.16:0/3465892913 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c037050 con 0x7ffb740039b0 2024-09-19T07:49:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.546+0000 7ffb92077640 1 -- 172.21.15.16:0/3465892913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb8c0cd430 con 0x7ffb740039b0 2024-09-19T07:49:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.547+0000 7ffb92077640 1 -- 172.21.15.16:0/3465892913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb8c0b4f00 con 0x7ffb740039b0 2024-09-19T07:49:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.547+0000 7ffb73fff640 1 -- 172.21.15.16:0/3465892913 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ffb7c007450 con 0x7ffb740039b0 2024-09-19T07:49:45.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.548+0000 7ffb73fff640 1 --2- 172.21.15.16:0/3465892913 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffb7806c2e0 0x7ffb7806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.548+0000 7ffb73fff640 1 -- 172.21.15.16:0/3465892913 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7ffb7c0ab500 con 0x7ffb740039b0 2024-09-19T07:49:45.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.548+0000 7ffb8affd640 1 --2- 172.21.15.16:0/3465892913 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffb7806c2e0 0x7ffb7806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.548+0000 7ffb8affd640 1 --2- 172.21.15.16:0/3465892913 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffb7806c2e0 0x7ffb7806e7a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0ba3a0 tx=0x7ffb80009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:45.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.552+0000 7ffb73fff640 1 -- 172.21.15.16:0/3465892913 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ffb7c04c020 con 0x7ffb740039b0 2024-09-19T07:49:45.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.797+0000 7ffb92077640 1 -- 172.21.15.16:0/3465892913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ffb8c0ba3d0 con 0x7ffb740039b0 2024-09-19T07:49:45.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.799+0000 7ffb73fff640 1 -- 172.21.15.16:0/3465892913 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7ffb7c079b70 con 0x7ffb740039b0 2024-09-19T07:49:45.800 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-19T07:49:45.800 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA== 2024-09-19T07:49:45.800 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-19T07:49:45.800 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-19T07:49:45.800 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T07:49:45.800 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-19T07:49:45.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.802+0000 7ffb92077640 1 -- 172.21.15.16:0/3465892913 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffb7806c2e0 msgr2=0x7ffb7806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:45.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.802+0000 7ffb92077640 1 --2- 172.21.15.16:0/3465892913 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffb7806c2e0 0x7ffb7806e7a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0ba3a0 tx=0x7ffb80009040 comp rx=0 tx=0).stop 2024-09-19T07:49:45.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.802+0000 7ffb92077640 1 -- 172.21.15.16:0/3465892913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb740039b0 msgr2=0x7ffb8c0ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:45.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.802+0000 7ffb92077640 1 --2- 172.21.15.16:0/3465892913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb740039b0 0x7ffb8c0ce520 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c035040 tx=0x7ffb7c0036f0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.802+0000 7ffb92077640 1 -- 172.21.15.16:0/3465892913 shutdown_connections 2024-09-19T07:49:45.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.802+0000 7ffb92077640 1 --2- 172.21.15.16:0/3465892913 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffb7806c2e0 0x7ffb7806e7a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.802+0000 7ffb92077640 1 --2- 172.21.15.16:0/3465892913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb8c1f4b70 0x7ffb8c0cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.802+0000 7ffb92077640 1 --2- 172.21.15.16:0/3465892913 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb8c1f46f0 0x7ffb8c0cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.802+0000 7ffb92077640 1 --2- 172.21.15.16:0/3465892913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb740039b0 0x7ffb8c0ce520 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.802+0000 7ffb92077640 1 -- 172.21.15.16:0/3465892913 >> 172.21.15.16:0/3465892913 conn(0x7ffb8c09d770 msgr2=0x7ffb8c0d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:45.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.802+0000 7ffb92077640 1 -- 172.21.15.16:0/3465892913 shutdown_connections 2024-09-19T07:49:45.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.802+0000 7ffb92077640 1 -- 172.21.15.16:0/3465892913 wait complete. 2024-09-19T07:49:45.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:49:45.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.915+0000 7f8a07d52640 1 Processor -- start 2024-09-19T07:49:45.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.915+0000 7f8a07d52640 1 -- start start 2024-09-19T07:49:45.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.916+0000 7f8a07d52640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a0015f0d0 0x7f8a0015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.916+0000 7f8a07d52640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a000d1b10 con 0x7f8a000d0070 2024-09-19T07:49:45.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.916+0000 7f8a07d52640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a000cf2b0 con 0x7f8a000d4130 2024-09-19T07:49:45.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.916+0000 7f8a07d52640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8a000cf590 con 0x7f8a0015f0d0 2024-09-19T07:49:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.916+0000 7f8a052ed640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8a000d0070 0x7f8a000d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36084/0 (socket says 172.21.15.16:36084) 2024-09-19T07:49:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.916+0000 7f8a052ed640 1 -- 172.21.15.16:0/3299278780 learned_addr learned my addr 172.21.15.16:0/3299278780 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a05aee640 1 --2- 172.21.15.16:0/3299278780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a0015f0d0 0x7f8a0015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a04aec640 1 -- 172.21.15.16:0/3299278780 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 203388923 0 0) 0x7f8a000d1b10 con 0x7f8a000d0070 2024-09-19T07:49:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a04aec640 1 -- 172.21.15.16:0/3299278780 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89e8003520 con 0x7f8a000d0070 2024-09-19T07:49:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a04aec640 1 -- 172.21.15.16:0/3299278780 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3616299426 0 0) 0x7f89e8003520 con 0x7f8a000d0070 2024-09-19T07:49:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a04aec640 1 -- 172.21.15.16:0/3299278780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a0015f0d0 msgr2=0x7f8a0015f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a04aec640 1 --2- 172.21.15.16:0/3299278780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a0015f0d0 0x7f8a0015f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a04aec640 1 -- 172.21.15.16:0/3299278780 >> v1:172.21.15.146:6789/0 conn(0x7f8a000d4130 legacy=0x7f8a000d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a04aec640 1 -- 172.21.15.16:0/3299278780 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a000cf870 con 0x7f8a000d0070 2024-09-19T07:49:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a04aec640 1 -- 172.21.15.16:0/3299278780 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f89f4001670 con 0x7f8a000d0070 2024-09-19T07:49:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a04aec640 1 -- 172.21.15.16:0/3299278780 >> v1:172.21.15.16:6789/0 conn(0x7f8a000d0070 legacy=0x7f8a000d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a04aec640 1 --2- 172.21.15.16:0/3299278780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e80039e0 0x7f89e8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a04aec640 1 --2- 172.21.15.16:0/3299278780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89e8004390 0x7f89e8004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a04aec640 1 --2- 172.21.15.16:0/3299278780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a0015f0d0 0x7f89e8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a04aec640 1 -- 172.21.15.16:0/3299278780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a000cf590 con 0x7f89e80039e0 2024-09-19T07:49:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a04aec640 1 -- 172.21.15.16:0/3299278780 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f89e800ba70 con 0x7f89e8004390 2024-09-19T07:49:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a04aec640 1 -- 172.21.15.16:0/3299278780 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f89e800bc70 con 0x7f8a0015f0d0 2024-09-19T07:49:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a05aee640 1 --2- 172.21.15.16:0/3299278780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e80039e0 0x7f89e8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a052ed640 1 --2- 172.21.15.16:0/3299278780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89e8004390 0x7f89e8004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a062ef640 1 --2- 172.21.15.16:0/3299278780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a0015f0d0 0x7f89e8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a05aee640 1 -- 172.21.15.16:0/3299278780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a0015f0d0 msgr2=0x7f89e8005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a05aee640 1 --2- 172.21.15.16:0/3299278780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a0015f0d0 0x7f89e8005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a05aee640 1 -- 172.21.15.16:0/3299278780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89e8004390 msgr2=0x7f89e8004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a05aee640 1 --2- 172.21.15.16:0/3299278780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89e8004390 0x7f89e8004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a05aee640 1 -- 172.21.15.16:0/3299278780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e8003520 con 0x7f89e80039e0 2024-09-19T07:49:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.917+0000 7f8a062ef640 1 --2- 172.21.15.16:0/3299278780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a0015f0d0 0x7f89e8005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.918+0000 7f8a05aee640 1 --2- 172.21.15.16:0/3299278780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e80039e0 0x7f89e8003dc0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009ac0 tx=0x7f89e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade8a34550ce822b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.918+0000 7f8a04aec640 1 -- 172.21.15.16:0/3299278780 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8036070 con 0x7f89e80039e0 2024-09-19T07:49:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.918+0000 7f8a04aec640 1 -- 172.21.15.16:0/3299278780 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f8031400 con 0x7f89e80039e0 2024-09-19T07:49:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.918+0000 7f8a04aec640 1 -- 172.21.15.16:0/3299278780 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8009690 con 0x7f89e80039e0 2024-09-19T07:49:45.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.919+0000 7f8a07d52640 1 -- 172.21.15.16:0/3299278780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e80039e0 msgr2=0x7f89e8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:45.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.919+0000 7f8a07d52640 1 --2- 172.21.15.16:0/3299278780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e80039e0 0x7f89e8003dc0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009ac0 tx=0x7f89e8005830 comp rx=0 tx=0).stop 2024-09-19T07:49:45.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.919+0000 7f8a062ef640 1 -- 172.21.15.16:0/3299278780 reap_dead start 2024-09-19T07:49:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.919+0000 7f8a07d52640 1 -- 172.21.15.16:0/3299278780 shutdown_connections 2024-09-19T07:49:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.919+0000 7f8a07d52640 1 -- 172.21.15.16:0/3299278780 >> 172.21.15.16:0/3299278780 conn(0x7f8a0009d760 msgr2=0x7f8a0008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.919+0000 7f8a07d52640 1 -- 172.21.15.16:0/3299278780 shutdown_connections 2024-09-19T07:49:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.919+0000 7f8a07d52640 1 -- 172.21.15.16:0/3299278780 wait complete. 2024-09-19T07:49:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.920+0000 7f8a07d52640 1 Processor -- start 2024-09-19T07:49:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.920+0000 7f8a07d52640 1 -- start start 2024-09-19T07:49:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.921+0000 7f8a07d52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e80039e0 0x7f8a00154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.921+0000 7f8a07d52640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a000c9080 0x7f8a000c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.921+0000 7f8a07d52640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a000c9a40 0x7f8a0015d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.921+0000 7f8a07d52640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0015f4b0 con 0x7f89e80039e0 2024-09-19T07:49:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.921+0000 7f8a07d52640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8a000cef60 con 0x7f8a000c9a40 2024-09-19T07:49:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.921+0000 7f8a07d52640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0015f630 con 0x7f8a000c9080 2024-09-19T07:49:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.921+0000 7f8a05aee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e80039e0 0x7f8a00154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.921+0000 7f8a05aee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e80039e0 0x7f8a00154880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60940/0 (socket says 172.21.15.16:60940) 2024-09-19T07:49:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.921+0000 7f8a05aee640 1 -- 172.21.15.16:0/3607740704 learned_addr learned my addr 172.21.15.16:0/3607740704 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.921+0000 7f8a052ed640 1 --2- 172.21.15.16:0/3607740704 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a000c9080 0x7f8a000c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.921+0000 7f8a062ef640 1 --2- 172.21.15.16:0/3607740704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a000c9a40 0x7f8a0015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.921+0000 7f8a05aee640 1 -- 172.21.15.16:0/3607740704 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a000c9080 msgr2=0x7f8a000c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.921+0000 7f8a05aee640 1 --2- 172.21.15.16:0/3607740704 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a000c9080 0x7f8a000c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.921+0000 7f8a05aee640 1 -- 172.21.15.16:0/3607740704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a000c9a40 msgr2=0x7f8a0015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.921+0000 7f8a05aee640 1 --2- 172.21.15.16:0/3607740704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a000c9a40 0x7f8a0015d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.921+0000 7f8a05aee640 1 -- 172.21.15.16:0/3607740704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a000ca9c0 con 0x7f89e80039e0 2024-09-19T07:49:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.922+0000 7f8a05aee640 1 --2- 172.21.15.16:0/3607740704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e80039e0 0x7f8a00154880 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009ac0 tx=0x7f89f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.922+0000 7f89fe7fc640 1 -- 172.21.15.16:0/3607740704 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8040020 con 0x7f89e80039e0 2024-09-19T07:49:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.922+0000 7f89fe7fc640 1 -- 172.21.15.16:0/3607740704 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f8003bc0 con 0x7f89e80039e0 2024-09-19T07:49:45.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.922+0000 7f8a07d52640 1 -- 172.21.15.16:0/3607740704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0015c8b0 con 0x7f89e80039e0 2024-09-19T07:49:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.922+0000 7f89fe7fc640 1 -- 172.21.15.16:0/3607740704 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8036040 con 0x7f89e80039e0 2024-09-19T07:49:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.922+0000 7f8a07d52640 1 -- 172.21.15.16:0/3607740704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a000ca6e0 con 0x7f89e80039e0 2024-09-19T07:49:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.923+0000 7f8a07d52640 1 -- 172.21.15.16:0/3607740704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a000d45e0 con 0x7f89e80039e0 2024-09-19T07:49:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.923+0000 7f89fe7fc640 1 -- 172.21.15.16:0/3607740704 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f89f8043020 con 0x7f89e80039e0 2024-09-19T07:49:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.924+0000 7f89fe7fc640 1 --2- 172.21.15.16:0/3607740704 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f89ec06c2b0 0x7f89ec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.924+0000 7f89fe7fc640 1 -- 172.21.15.16:0/3607740704 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f89f80ab480 con 0x7f89e80039e0 2024-09-19T07:49:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.924+0000 7f8a052ed640 1 --2- 172.21.15.16:0/3607740704 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f89ec06c2b0 0x7f89ec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.924+0000 7f8a052ed640 1 --2- 172.21.15.16:0/3607740704 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f89ec06c2b0 0x7f89ec06e770 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f89f40011e0 tx=0x7f89f4001090 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:45.928+0000 7f89fe7fc640 1 -- 172.21.15.16:0/3607740704 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f89f804a070 con 0x7f89e80039e0 2024-09-19T07:49:46.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.140+0000 7f8a07d52640 1 -- 172.21.15.16:0/3607740704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a000d3f10 con 0x7f89e80039e0 2024-09-19T07:49:46.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.140+0000 7f89fe7fc640 1 -- 172.21.15.16:0/3607740704 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5222 (secure 0 0 0) 0x7f89f80044e0 con 0x7f89e80039e0 2024-09-19T07:49:46.142 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:46.142 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"btime":"2024-09-19T07:49:44:053013+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/945862061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":945862061},{"type":"v1","addr":"172.21.15.146:6836","nonce":945862061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/42753795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":42753795},{"type":"v1","addr":"172.21.15.16:6834","nonce":42753795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1389787480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1389787480},{"type":"v1","addr":"172.21.15.16:6835","nonce":1389787480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:49:40.403885+0000","modified":"2024-09-19T07:49:43.407266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6096":{"gid":6096,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/3642987189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3642987189},{"type":"v1","addr":"172.21.15.146:6837","nonce":3642987189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6096,"qdb_cluster":[6096]},"id":5}]} 2024-09-19T07:49:46.143 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 41 2024-09-19T07:49:46.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.142+0000 7f8a07d52640 1 -- 172.21.15.16:0/3607740704 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f89ec06c2b0 msgr2=0x7f89ec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:46.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.142+0000 7f8a07d52640 1 --2- 172.21.15.16:0/3607740704 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f89ec06c2b0 0x7f89ec06e770 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f89f40011e0 tx=0x7f89f4001090 comp rx=0 tx=0).stop 2024-09-19T07:49:46.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.142+0000 7f8a07d52640 1 -- 172.21.15.16:0/3607740704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e80039e0 msgr2=0x7f8a00154880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:46.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.142+0000 7f8a07d52640 1 --2- 172.21.15.16:0/3607740704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e80039e0 0x7f8a00154880 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009ac0 tx=0x7f89f8004730 comp rx=0 tx=0).stop 2024-09-19T07:49:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.142+0000 7f8a07d52640 1 -- 172.21.15.16:0/3607740704 shutdown_connections 2024-09-19T07:49:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.142+0000 7f8a07d52640 1 --2- 172.21.15.16:0/3607740704 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f89ec06c2b0 0x7f89ec06e770 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.142+0000 7f8a07d52640 1 --2- 172.21.15.16:0/3607740704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a000c9a40 0x7f8a0015d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.142+0000 7f8a07d52640 1 --2- 172.21.15.16:0/3607740704 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8a000c9080 0x7f8a000c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.142+0000 7f8a07d52640 1 --2- 172.21.15.16:0/3607740704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e80039e0 0x7f8a00154880 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.143+0000 7f8a07d52640 1 -- 172.21.15.16:0/3607740704 >> 172.21.15.16:0/3607740704 conn(0x7f8a0009d760 msgr2=0x7f8a000b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.143+0000 7f8a07d52640 1 -- 172.21.15.16:0/3607740704 shutdown_connections 2024-09-19T07:49:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.143+0000 7f8a07d52640 1 -- 172.21.15.16:0/3607740704 wait complete. 2024-09-19T07:49:46.556 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T07:49:40.403885+0000', 'modified': '2024-09-19T07:49:43.407266+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6096}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6096': {'gid': 6096, 'name': 'd', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.146:6837/3642987189', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6835', 'nonce': 3642987189}, {'type': 'v1', 'addr': '172.21.15.146:6837', 'nonce': 3642987189}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6096, 'qdb_cluster': [6096]} 2024-09-19T07:49:46.556 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T07:49:46.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-19T07:49:46.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.663+0000 7ff204d86640 1 Processor -- start 2024-09-19T07:49:46.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.663+0000 7ff204d86640 1 -- start start 2024-09-19T07:49:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.664+0000 7ff204d86640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff200141680 0x7ff2000cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.664+0000 7ff204d86640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff20015d710 con 0x7ff2000d0370 2024-09-19T07:49:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.664+0000 7ff204d86640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff20015c950 con 0x7ff200142170 2024-09-19T07:49:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.664+0000 7ff204d86640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff2000d07e0 con 0x7ff200141680 2024-09-19T07:49:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.664+0000 7ff1fdd9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff2000d0370 0x7ff200141a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36092/0 (socket says 172.21.15.16:36092) 2024-09-19T07:49:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.664+0000 7ff1fdd9b640 1 -- 172.21.15.16:0/1197537945 learned_addr learned my addr 172.21.15.16:0/1197537945 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.665+0000 7ff1fe59c640 1 --2- 172.21.15.16:0/1197537945 >> v2:172.21.15.146:3300/0 conn(0x7ff200141680 0x7ff2000cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.665+0000 7ff1fd59a640 1 -- 172.21.15.16:0/1197537945 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2123584210 0 0) 0x7ff20015d710 con 0x7ff2000d0370 2024-09-19T07:49:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.665+0000 7ff1fd59a640 1 -- 172.21.15.16:0/1197537945 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1ec003520 con 0x7ff2000d0370 2024-09-19T07:49:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.665+0000 7ff1fd59a640 1 -- 172.21.15.16:0/1197537945 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1615386441 0 0) 0x7ff1ec003520 con 0x7ff2000d0370 2024-09-19T07:49:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.665+0000 7ff1fd59a640 1 -- 172.21.15.16:0/1197537945 >> v2:172.21.15.146:3300/0 conn(0x7ff200141680 msgr2=0x7ff2000cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.665+0000 7ff1fd59a640 1 --2- 172.21.15.16:0/1197537945 >> v2:172.21.15.146:3300/0 conn(0x7ff200141680 0x7ff2000cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.665+0000 7ff1fd59a640 1 -- 172.21.15.16:0/1197537945 >> v1:172.21.15.146:6789/0 conn(0x7ff200142170 legacy=0x7ff20013f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.665+0000 7ff1fd59a640 1 -- 172.21.15.16:0/1197537945 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20015cc30 con 0x7ff2000d0370 2024-09-19T07:49:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.665+0000 7ff1fd59a640 1 -- 172.21.15.16:0/1197537945 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7ff1e8001670 con 0x7ff2000d0370 2024-09-19T07:49:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.665+0000 7ff1fd59a640 1 -- 172.21.15.16:0/1197537945 >> v1:172.21.15.16:6789/0 conn(0x7ff2000d0370 legacy=0x7ff200141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.665+0000 7ff1fd59a640 1 --2- 172.21.15.16:0/1197537945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1ec0039b0 0x7ff1ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.665+0000 7ff1fd59a640 1 --2- 172.21.15.16:0/1197537945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1ec004800 0x7ff1ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.665+0000 7ff1fd59a640 1 --2- 172.21.15.16:0/1197537945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1ec0051c0 0x7ff1ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.665+0000 7ff1fd59a640 1 -- 172.21.15.16:0/1197537945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2000d07e0 con 0x7ff1ec0051c0 2024-09-19T07:49:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.665+0000 7ff1fd59a640 1 -- 172.21.15.16:0/1197537945 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec00c060 con 0x7ff1ec004800 2024-09-19T07:49:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.665+0000 7ff1fd59a640 1 -- 172.21.15.16:0/1197537945 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff1ec00c260 con 0x7ff1ec0039b0 2024-09-19T07:49:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.665+0000 7ff1fed9d640 1 --2- 172.21.15.16:0/1197537945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1ec0051c0 0x7ff1ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.665+0000 7ff1fe59c640 1 --2- 172.21.15.16:0/1197537945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1ec0039b0 0x7ff1ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.665+0000 7ff1fdd9b640 1 --2- 172.21.15.16:0/1197537945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1ec004800 0x7ff1ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.666+0000 7ff1fed9d640 1 -- 172.21.15.16:0/1197537945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1ec0039b0 msgr2=0x7ff1ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.666+0000 7ff1fed9d640 1 --2- 172.21.15.16:0/1197537945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1ec0039b0 0x7ff1ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.666+0000 7ff1fed9d640 1 -- 172.21.15.16:0/1197537945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1ec004800 msgr2=0x7ff1ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.666+0000 7ff1fed9d640 1 --2- 172.21.15.16:0/1197537945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1ec004800 0x7ff1ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.666+0000 7ff1fed9d640 1 -- 172.21.15.16:0/1197537945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ec003520 con 0x7ff1ec0051c0 2024-09-19T07:49:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.666+0000 7ff1fed9d640 1 -- 172.21.15.16:0/1197537945 reap_dead start 2024-09-19T07:49:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.666+0000 7ff1fed9d640 1 --2- 172.21.15.16:0/1197537945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1ec0051c0 0x7ff1ec00b9c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0001e50 tx=0x7ff1ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9bdb5acc4d1dd51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.666+0000 7ff1fd59a640 1 -- 172.21.15.16:0/1197537945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0001a70 con 0x7ff1ec0051c0 2024-09-19T07:49:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.666+0000 7ff1fd59a640 1 -- 172.21.15.16:0/1197537945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f0003970 con 0x7ff1ec0051c0 2024-09-19T07:49:46.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.666+0000 7ff1fd59a640 1 -- 172.21.15.16:0/1197537945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0003d60 con 0x7ff1ec0051c0 2024-09-19T07:49:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.667+0000 7ff204d86640 1 -- 172.21.15.16:0/1197537945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1ec0051c0 msgr2=0x7ff1ec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.667+0000 7ff204d86640 1 --2- 172.21.15.16:0/1197537945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1ec0051c0 0x7ff1ec00b9c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0001e50 tx=0x7ff1ec005830 comp rx=0 tx=0).stop 2024-09-19T07:49:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.667+0000 7ff204d86640 1 -- 172.21.15.16:0/1197537945 shutdown_connections 2024-09-19T07:49:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.667+0000 7ff204d86640 1 --2- 172.21.15.16:0/1197537945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1ec0051c0 0x7ff1ec00b9c0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.667+0000 7ff204d86640 1 -- 172.21.15.16:0/1197537945 >> 172.21.15.16:0/1197537945 conn(0x7ff2000b3ed0 msgr2=0x7ff2000b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.667+0000 7ff204d86640 1 -- 172.21.15.16:0/1197537945 shutdown_connections 2024-09-19T07:49:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.667+0000 7ff204d86640 1 -- 172.21.15.16:0/1197537945 wait complete. 2024-09-19T07:49:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.669+0000 7ff204d86640 1 Processor -- start 2024-09-19T07:49:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.669+0000 7ff204d86640 1 -- start start 2024-09-19T07:49:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.669+0000 7ff204d86640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1ec0051c0 0x7ff2001f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.669+0000 7ff204d86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff200156e00 0x7ff2000b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.669+0000 7ff204d86640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff200157260 0x7ff2000b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.669+0000 7ff204d86640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff20015d1f0 con 0x7ff200156e00 2024-09-19T07:49:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.669+0000 7ff204d86640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff20015c6f0 con 0x7ff200157260 2024-09-19T07:49:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.669+0000 7ff204d86640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff200142570 con 0x7ff1ec0051c0 2024-09-19T07:49:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.669+0000 7ff1fdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff200156e00 0x7ff2000b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.669+0000 7ff1fdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff200156e00 0x7ff2000b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60958/0 (socket says 172.21.15.16:60958) 2024-09-19T07:49:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.669+0000 7ff1fdd9b640 1 -- 172.21.15.16:0/416672975 learned_addr learned my addr 172.21.15.16:0/416672975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.670+0000 7ff1fe59c640 1 --2- 172.21.15.16:0/416672975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1ec0051c0 0x7ff2001f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.670+0000 7ff1fed9d640 1 --2- 172.21.15.16:0/416672975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff200157260 0x7ff2000b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.670+0000 7ff1fdd9b640 1 -- 172.21.15.16:0/416672975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1ec0051c0 msgr2=0x7ff2001f1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.670+0000 7ff1fdd9b640 1 --2- 172.21.15.16:0/416672975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1ec0051c0 0x7ff2001f1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.670+0000 7ff1fdd9b640 1 -- 172.21.15.16:0/416672975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff200157260 msgr2=0x7ff2000b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.670+0000 7ff1fdd9b640 1 --2- 172.21.15.16:0/416672975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff200157260 0x7ff2000b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.670+0000 7ff1fdd9b640 1 -- 172.21.15.16:0/416672975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2000b7d80 con 0x7ff200156e00 2024-09-19T07:49:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.670+0000 7ff1fed9d640 1 --2- 172.21.15.16:0/416672975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff200157260 0x7ff2000b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.670+0000 7ff1fdd9b640 1 --2- 172.21.15.16:0/416672975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff200156e00 0x7ff2000b2a60 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80072f0 tx=0x7ff1e80077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.670+0000 7ff1f6ffd640 1 -- 172.21.15.16:0/416672975 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e801a070 con 0x7ff200156e00 2024-09-19T07:49:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.670+0000 7ff1f6ffd640 1 -- 172.21.15.16:0/416672975 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e800f8e0 con 0x7ff200156e00 2024-09-19T07:49:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.670+0000 7ff204d86640 1 -- 172.21.15.16:0/416672975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2000b8060 con 0x7ff200156e00 2024-09-19T07:49:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.670+0000 7ff1f6ffd640 1 -- 172.21.15.16:0/416672975 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e800fcb0 con 0x7ff200156e00 2024-09-19T07:49:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.670+0000 7ff204d86640 1 -- 172.21.15.16:0/416672975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff200159ea0 con 0x7ff200156e00 2024-09-19T07:49:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.672+0000 7ff1f6ffd640 1 -- 172.21.15.16:0/416672975 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff1e801d070 con 0x7ff200156e00 2024-09-19T07:49:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.672+0000 7ff1f6ffd640 1 --2- 172.21.15.16:0/416672975 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff1e006c2b0 0x7ff1e006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.672+0000 7ff1f6ffd640 1 -- 172.21.15.16:0/416672975 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7ff1e808b080 con 0x7ff200156e00 2024-09-19T07:49:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.672+0000 7ff1fe59c640 1 --2- 172.21.15.16:0/416672975 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff1e006c2b0 0x7ff1e006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.673+0000 7ff207bf6740 1 Processor -- start 2024-09-19T07:49:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.673+0000 7ff1fe59c640 1 --2- 172.21.15.16:0/416672975 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff1e006c2b0 0x7ff1e006e770 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009a80 tx=0x7ff1f8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.673+0000 7ff207bf6740 1 -- start start 2024-09-19T07:49:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.674+0000 7ff207bf6740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd12c0 0x564bc9016a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.674+0000 7ff207bf6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8fcea80 0x564bc8fd16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.674+0000 7ff207bf6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd2a50 0x564bc8fcb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.674+0000 7ff207bf6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564bc8f33520 con 0x564bc8fd2a50 2024-09-19T07:49:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.674+0000 7ff207bf6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564bc8edb3e0 con 0x564bc8fcea80 2024-09-19T07:49:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.674+0000 7ff207bf6740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x564bc8f2f4e0 con 0x564bc8fd12c0 2024-09-19T07:49:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.674+0000 7ff1fed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd2a50 0x564bc8fcb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.674+0000 7ff1fed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd2a50 0x564bc8fcb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60964/0 (socket says 172.21.15.16:60964) 2024-09-19T07:49:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.674+0000 7ff1fed9d640 1 -- 172.21.15.16:0/2807034000 learned_addr learned my addr 172.21.15.16:0/2807034000 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.674+0000 7ff1fe59c640 1 --2- 172.21.15.16:0/2807034000 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd12c0 0x564bc9016a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.674+0000 7ff1fdd9b640 1 --2- 172.21.15.16:0/2807034000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8fcea80 0x564bc8fd16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.674+0000 7ff1fed9d640 1 -- 172.21.15.16:0/2807034000 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd12c0 msgr2=0x564bc9016a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.674+0000 7ff1fed9d640 1 --2- 172.21.15.16:0/2807034000 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd12c0 0x564bc9016a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.674+0000 7ff1fed9d640 1 -- 172.21.15.16:0/2807034000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8fcea80 msgr2=0x564bc8fd16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.674+0000 7ff1fed9d640 1 --2- 172.21.15.16:0/2807034000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8fcea80 0x564bc8fd16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.674+0000 7ff1fed9d640 1 -- 172.21.15.16:0/2807034000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bc8ef62f0 con 0x564bc8fd2a50 2024-09-19T07:49:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.674+0000 7ff1fed9d640 1 --2- 172.21.15.16:0/2807034000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd2a50 0x564bc8fcb8b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0001f40 tx=0x7ff1f002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7cdd3f054f5ccb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.674+0000 7ff1dffff640 1 -- 172.21.15.16:0/2807034000 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0006b50 con 0x564bc8fd2a50 2024-09-19T07:49:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.675+0000 7ff1dffff640 1 -- 172.21.15.16:0/2807034000 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f0006cf0 con 0x564bc8fd2a50 2024-09-19T07:49:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.675+0000 7ff1dffff640 1 -- 172.21.15.16:0/2807034000 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0003b70 con 0x564bc8fd2a50 2024-09-19T07:49:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.675+0000 7ff207bf6740 1 -- 172.21.15.16:0/2807034000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd2a50 msgr2=0x564bc8fcb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.675+0000 7ff207bf6740 1 --2- 172.21.15.16:0/2807034000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd2a50 0x564bc8fcb8b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0001f40 tx=0x7ff1f002c000 comp rx=0 tx=0).stop 2024-09-19T07:49:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.675+0000 7ff207bf6740 1 -- 172.21.15.16:0/2807034000 shutdown_connections 2024-09-19T07:49:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.675+0000 7ff207bf6740 1 --2- 172.21.15.16:0/2807034000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd2a50 0x564bc8fcb8b0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.675+0000 7ff207bf6740 1 --2- 172.21.15.16:0/2807034000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8fcea80 0x564bc8fd16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.675+0000 7ff207bf6740 1 --2- 172.21.15.16:0/2807034000 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd12c0 0x564bc9016a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.675+0000 7ff207bf6740 1 -- 172.21.15.16:0/2807034000 >> 172.21.15.16:0/2807034000 conn(0x564bc8fcda80 msgr2=0x564bc9015f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.675+0000 7ff207bf6740 1 -- 172.21.15.16:0/2807034000 shutdown_connections 2024-09-19T07:49:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.675+0000 7ff207bf6740 1 -- 172.21.15.16:0/2807034000 wait complete. 2024-09-19T07:49:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.676+0000 7ff207bf6740 1 -- start start 2024-09-19T07:49:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.676+0000 7ff207bf6740 1 Processor -- start 2024-09-19T07:49:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff207bf6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd12c0 0x564bc8ff9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff207bf6740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd2a50 0x564bc8ff47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff207bf6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8ff4d00 0x564bc8ff3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff207bf6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564bc8fccf20 con 0x564bc8fd12c0 2024-09-19T07:49:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff207bf6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564bc8ebc450 con 0x564bc8ff4d00 2024-09-19T07:49:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff207bf6740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x564bc9017290 con 0x564bc8fd2a50 2024-09-19T07:49:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff207bf6740 10 client.0 _ll_register_callbacks cb 0x564bc8c45ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff1fe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd12c0 0x564bc8ff9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff1fe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd12c0 0x564bc8ff9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60968/0 (socket says 172.21.15.16:60968) 2024-09-19T07:49:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff1fe59c640 1 -- 172.21.15.16:0/3684216370 learned_addr learned my addr 172.21.15.16:0/3684216370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff1fdd9b640 1 --2- 172.21.15.16:0/3684216370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd2a50 0x564bc8ff47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff1fed9d640 1 --2- 172.21.15.16:0/3684216370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8ff4d00 0x564bc8ff3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff1fe59c640 1 -- 172.21.15.16:0/3684216370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd2a50 msgr2=0x564bc8ff47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff1fe59c640 1 --2- 172.21.15.16:0/3684216370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd2a50 0x564bc8ff47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff1fe59c640 1 -- 172.21.15.16:0/3684216370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8ff4d00 msgr2=0x564bc8ff3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff1fe59c640 1 --2- 172.21.15.16:0/3684216370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8ff4d00 0x564bc8ff3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff1fe59c640 1 -- 172.21.15.16:0/3684216370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bc8c462d0 con 0x564bc8fd12c0 2024-09-19T07:49:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff1fed9d640 1 --2- 172.21.15.16:0/3684216370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8ff4d00 0x564bc8ff3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff1fe59c640 1 --2- 172.21.15.16:0/3684216370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd12c0 0x564bc8ff9b90 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8003640 tx=0x7ff1f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9256bb28af13a62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.677+0000 7ff1dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:49:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.678+0000 7ff1dd7fa640 1 -- 172.21.15.16:0/3684216370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8029070 con 0x564bc8fd12c0 2024-09-19T07:49:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.678+0000 7ff207bf6740 1 -- 172.21.15.16:0/3684216370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1f0006cf0 con 0x564bc8fd12c0 2024-09-19T07:49:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.678+0000 7ff1dd7fa640 1 -- 172.21.15.16:0/3684216370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7ff1f8016700 con 0x564bc8fd12c0 2024-09-19T07:49:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.678+0000 7ff1dd7fa640 1 -- 172.21.15.16:0/3684216370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff1f8034050 con 0x564bc8fd12c0 2024-09-19T07:49:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.678+0000 7ff207bf6740 10 client.6204 fetch_fsmap learned FSMap version 41 2024-09-19T07:49:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.678+0000 7ff207bf6740 1 -- 172.21.15.16:0/3684216370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) -- 0x564bc8d106b0 con 0x564bc8fd12c0 2024-09-19T07:49:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.678+0000 7ff1dd7fa640 1 -- 172.21.15.16:0/3684216370 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f0006cf0 con 0x564bc8fd12c0 2024-09-19T07:49:46.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.678+0000 7ff1dd7fa640 1 -- 172.21.15.16:0/3684216370 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 41) ==== 3502+0+0 (secure 0 0 0) 0x7ff1f8037020 con 0x564bc8fd12c0 2024-09-19T07:49:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.678+0000 7ff207bf6740 10 client.6204 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T07:49:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.678+0000 7ff207bf6740 10 client.6204 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-19T07:49:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.678+0000 7ff207bf6740 20 client.6204 populate_metadata read hostname 'smithi016' 2024-09-19T07:49:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.678+0000 7ff207bf6740 1 --2- 172.21.15.16:0/3684216370 >> [v2:172.21.15.146:6835/3642987189,v1:172.21.15.146:6837/3642987189] conn(0x564bc8ffb150 0x564bc9011bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.678+0000 7ff207bf6740 4 client.6204 mds_command: new command op to 6096 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:49:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.678+0000 7ff207bf6740 1 -- 172.21.15.16:0/3684216370 --> [v2:172.21.15.146:6835/3642987189,v1:172.21.15.146:6837/3642987189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bc8f40c50 con 0x564bc8ffb150 2024-09-19T07:49:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.678+0000 7ff1fdd9b640 1 --2- 172.21.15.16:0/3684216370 >> [v2:172.21.15.146:6835/3642987189,v1:172.21.15.146:6837/3642987189] conn(0x564bc8ffb150 0x564bc9011bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.679+0000 7ff1fdd9b640 1 --2- 172.21.15.16:0/3684216370 >> [v2:172.21.15.146:6835/3642987189,v1:172.21.15.146:6837/3642987189] conn(0x564bc8ffb150 0x564bc9011bf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d87a0cb487cb7837 server_cookie=60c6fcb9ccdbb2e in_seq=0 out_seq=0 2024-09-19T07:49:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.679+0000 7ff1dd7fa640 10 client.6204 ms_handle_connect on v2:172.21.15.146:6835/3642987189 2024-09-19T07:49:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.680+0000 7ff1dd7fa640 1 -- 172.21.15.16:0/3684216370 <== mds.0 v2:172.21.15.146:6835/3642987189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564bc8f40c50 con 0x564bc8ffb150 2024-09-19T07:49:46.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.680+0000 7ff1dd7fa640 10 client.6204 handle_command_reply: tid=0 2024-09-19T07:49:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.680+0000 7ff207bf6740 1 client.6204 shutdown 2024-09-19T07:49:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.680+0000 7ff207bf6740 1 -- 172.21.15.16:0/3684216370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd12c0 msgr2=0x564bc8ff9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.680+0000 7ff207bf6740 1 --2- 172.21.15.16:0/3684216370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd12c0 0x564bc8ff9b90 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8003640 tx=0x7ff1f8004730 comp rx=0 tx=0).stop 2024-09-19T07:49:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.680+0000 7ff207bf6740 1 -- 172.21.15.16:0/3684216370 shutdown_connections 2024-09-19T07:49:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.680+0000 7ff207bf6740 1 --2- 172.21.15.16:0/3684216370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8ff4d00 0x564bc8ff3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.680+0000 7ff207bf6740 1 --2- 172.21.15.16:0/3684216370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd2a50 0x564bc8ff47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.680+0000 7ff207bf6740 1 --2- 172.21.15.16:0/3684216370 >> [v2:172.21.15.146:6835/3642987189,v1:172.21.15.146:6837/3642987189] conn(0x564bc8ffb150 0x564bc9011bf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.680+0000 7ff207bf6740 1 --2- 172.21.15.16:0/3684216370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd12c0 0x564bc8ff9b90 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.680+0000 7ff1dd7fa640 0 client.6204 ms_handle_reset on v2:172.21.15.146:6835/3642987189 2024-09-19T07:49:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.680+0000 7ff207bf6740 1 -- 172.21.15.16:0/3684216370 >> 172.21.15.16:0/3684216370 conn(0x564bc8fcea80 msgr2=0x564bc9013ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.680+0000 7ff207bf6740 1 -- 172.21.15.16:0/3684216370 shutdown_connections 2024-09-19T07:49:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.680+0000 7ff207bf6740 1 -- 172.21.15.16:0/3684216370 wait complete. 2024-09-19T07:49:46.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.680+0000 7ff207bf6740 20 client.6204 trim_cache size 0 max 16384 2024-09-19T07:49:46.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.698+0000 7ff207bf6740 1 Processor -- start 2024-09-19T07:49:46.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.698+0000 7ff207bf6740 1 -- start start 2024-09-19T07:49:46.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.699+0000 7ff207bf6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8fd12c0 0x564bc90140d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.699+0000 7ff207bf6740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd2a50 0x564bc9012de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.699+0000 7ff207bf6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8ff4d00 0x564bc9013320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.699+0000 7ff207bf6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2001fe370 con 0x564bc8ff4d00 2024-09-19T07:49:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.699+0000 7ff207bf6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff2000b6320 con 0x564bc8fd12c0 2024-09-19T07:49:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.699+0000 7ff207bf6740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff2001fdd00 con 0x564bc8fd2a50 2024-09-19T07:49:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.699+0000 7ff1fed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8ff4d00 0x564bc9013320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.699+0000 7ff1fed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8ff4d00 0x564bc9013320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60978/0 (socket says 172.21.15.16:60978) 2024-09-19T07:49:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.699+0000 7ff1fed9d640 1 -- 172.21.15.16:0/4248597126 learned_addr learned my addr 172.21.15.16:0/4248597126 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.699+0000 7ff1fdd9b640 1 --2- 172.21.15.16:0/4248597126 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd2a50 0x564bc9012de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.699+0000 7ff1fe59c640 1 --2- 172.21.15.16:0/4248597126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8fd12c0 0x564bc90140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.700+0000 7ff1fed9d640 1 -- 172.21.15.16:0/4248597126 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd2a50 msgr2=0x564bc9012de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.700+0000 7ff1fed9d640 1 --2- 172.21.15.16:0/4248597126 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd2a50 0x564bc9012de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.700+0000 7ff1fed9d640 1 -- 172.21.15.16:0/4248597126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8fd12c0 msgr2=0x564bc90140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.700+0000 7ff1fed9d640 1 --2- 172.21.15.16:0/4248597126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8fd12c0 0x564bc90140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.700+0000 7ff1fed9d640 1 -- 172.21.15.16:0/4248597126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bc9028330 con 0x564bc8ff4d00 2024-09-19T07:49:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.700+0000 7ff1fdd9b640 1 --2- 172.21.15.16:0/4248597126 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd2a50 0x564bc9012de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.700+0000 7ff1fed9d640 1 --2- 172.21.15.16:0/4248597126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8ff4d00 0x564bc9013320 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0007f00 tx=0x7ff1f0009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc161711ac4b470 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.700+0000 7ff1df7fe640 1 -- 172.21.15.16:0/4248597126 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0006c10 con 0x564bc8ff4d00 2024-09-19T07:49:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.700+0000 7ff1df7fe640 1 -- 172.21.15.16:0/4248597126 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f0006db0 con 0x564bc8ff4d00 2024-09-19T07:49:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.700+0000 7ff1df7fe640 1 -- 172.21.15.16:0/4248597126 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0003b90 con 0x564bc8ff4d00 2024-09-19T07:49:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.701+0000 7ff207bf6740 1 -- 172.21.15.16:0/4248597126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8ff4d00 msgr2=0x564bc9013320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.701+0000 7ff207bf6740 1 --2- 172.21.15.16:0/4248597126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8ff4d00 0x564bc9013320 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0007f00 tx=0x7ff1f0009030 comp rx=0 tx=0).stop 2024-09-19T07:49:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.701+0000 7ff207bf6740 1 -- 172.21.15.16:0/4248597126 shutdown_connections 2024-09-19T07:49:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.701+0000 7ff207bf6740 1 --2- 172.21.15.16:0/4248597126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8ff4d00 0x564bc9013320 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.701+0000 7ff207bf6740 1 --2- 172.21.15.16:0/4248597126 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd2a50 0x564bc9012de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.701+0000 7ff207bf6740 1 --2- 172.21.15.16:0/4248597126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8fd12c0 0x564bc90140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.701+0000 7ff207bf6740 1 -- 172.21.15.16:0/4248597126 >> 172.21.15.16:0/4248597126 conn(0x564bc8fcea80 msgr2=0x564bc8ff4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.701+0000 7ff207bf6740 1 -- 172.21.15.16:0/4248597126 shutdown_connections 2024-09-19T07:49:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.701+0000 7ff207bf6740 1 -- 172.21.15.16:0/4248597126 wait complete. 2024-09-19T07:49:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.701+0000 7ff207bf6740 1 -- start start 2024-09-19T07:49:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.701+0000 7ff207bf6740 1 Processor -- start 2024-09-19T07:49:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.702+0000 7ff207bf6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd12c0 0x564bc9081d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.702+0000 7ff207bf6740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd2a50 0x564bc8ff75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.702+0000 7ff207bf6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8ff4d00 0x564bc8ff7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.702+0000 7ff207bf6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564bc8ffafd0 con 0x564bc8fd12c0 2024-09-19T07:49:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.702+0000 7ff207bf6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564bc8ffa0d0 con 0x564bc8ff4d00 2024-09-19T07:49:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.702+0000 7ff207bf6740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x564bc8ffb5b0 con 0x564bc8fd2a50 2024-09-19T07:49:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.702+0000 7ff207bf6740 10 client.0 _ll_register_callbacks cb 0x7ff1f8014a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.702+0000 7ff1fe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd12c0 0x564bc9081d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.702+0000 7ff1fe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd12c0 0x564bc9081d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60984/0 (socket says 172.21.15.16:60984) 2024-09-19T07:49:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.702+0000 7ff1fe59c640 1 -- 172.21.15.16:0/479489310 learned_addr learned my addr 172.21.15.16:0/479489310 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.702+0000 7ff1fdd9b640 1 --2- 172.21.15.16:0/479489310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd2a50 0x564bc8ff75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.702+0000 7ff1fed9d640 1 --2- 172.21.15.16:0/479489310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8ff4d00 0x564bc8ff7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff1fe59c640 1 -- 172.21.15.16:0/479489310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd2a50 msgr2=0x564bc8ff75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff1fe59c640 1 --2- 172.21.15.16:0/479489310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd2a50 0x564bc8ff75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff1fe59c640 1 -- 172.21.15.16:0/479489310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8ff4d00 msgr2=0x564bc8ff7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff1fe59c640 1 --2- 172.21.15.16:0/479489310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8ff4d00 0x564bc8ff7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff1fe59c640 1 -- 172.21.15.16:0/479489310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bc9028500 con 0x564bc8fd12c0 2024-09-19T07:49:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff1fdd9b640 1 --2- 172.21.15.16:0/479489310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd2a50 0x564bc8ff75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff1fe59c640 1 --2- 172.21.15.16:0/479489310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd12c0 0x564bc9081d70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8003610 tx=0x7ff1f8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c53838cec7799a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff1dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:49:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff1dd7fa640 1 -- 172.21.15.16:0/479489310 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8029040 con 0x564bc8fd12c0 2024-09-19T07:49:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff1dd7fa640 1 -- 172.21.15.16:0/479489310 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7ff1f8016ac0 con 0x564bc8fd12c0 2024-09-19T07:49:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff207bf6740 1 -- 172.21.15.16:0/479489310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1f0006db0 con 0x564bc8fd12c0 2024-09-19T07:49:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff1dd7fa640 1 -- 172.21.15.16:0/479489310 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff1f8046050 con 0x564bc8fd12c0 2024-09-19T07:49:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff207bf6740 10 client.6210 fetch_fsmap learned FSMap version 41 2024-09-19T07:49:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff207bf6740 1 -- 172.21.15.16:0/479489310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) -- 0x564bc8fcfdb0 con 0x564bc8fd12c0 2024-09-19T07:49:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff1dd7fa640 1 -- 172.21.15.16:0/479489310 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f0006db0 con 0x564bc8fd12c0 2024-09-19T07:49:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff1dd7fa640 1 -- 172.21.15.16:0/479489310 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 41) ==== 3502+0+0 (secure 0 0 0) 0x7ff1f8049020 con 0x564bc8fd12c0 2024-09-19T07:49:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff207bf6740 10 client.6210 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T07:49:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff207bf6740 10 client.6210 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-19T07:49:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff207bf6740 20 client.6210 populate_metadata read hostname 'smithi016' 2024-09-19T07:49:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff207bf6740 1 --2- 172.21.15.16:0/479489310 >> [v2:172.21.15.146:6835/3642987189,v1:172.21.15.146:6837/3642987189] conn(0x564bc902fa70 0x564bc9017a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff207bf6740 4 client.6210 mds_command: new command op to 6096 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:49:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.703+0000 7ff207bf6740 1 -- 172.21.15.16:0/479489310 --> [v2:172.21.15.146:6835/3642987189,v1:172.21.15.146:6837/3642987189] -- command(tid 0: {"prefix": "status"}) -- 0x564bc8f3afa0 con 0x564bc902fa70 2024-09-19T07:49:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.704+0000 7ff1fdd9b640 1 --2- 172.21.15.16:0/479489310 >> [v2:172.21.15.146:6835/3642987189,v1:172.21.15.146:6837/3642987189] conn(0x564bc902fa70 0x564bc9017a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.704+0000 7ff1fdd9b640 1 --2- 172.21.15.16:0/479489310 >> [v2:172.21.15.146:6835/3642987189,v1:172.21.15.146:6837/3642987189] conn(0x564bc902fa70 0x564bc9017a90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2e179a9b67304e server_cookie=4f808e684409fbdb in_seq=0 out_seq=0 2024-09-19T07:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.704+0000 7ff1dd7fa640 10 client.6210 ms_handle_connect on v2:172.21.15.146:6835/3642987189 2024-09-19T07:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.704+0000 7ff1dd7fa640 1 -- 172.21.15.16:0/479489310 <== mds.0 v2:172.21.15.146:6835/3642987189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x564bc8f3afa0 con 0x564bc902fa70 2024-09-19T07:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.704+0000 7ff1dd7fa640 10 client.6210 handle_command_reply: tid=0 2024-09-19T07:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.704+0000 7ff207bf6740 1 client.6210 shutdown 2024-09-19T07:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.705+0000 7ff207bf6740 1 -- 172.21.15.16:0/479489310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd12c0 msgr2=0x564bc9081d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.705+0000 7ff207bf6740 1 --2- 172.21.15.16:0/479489310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd12c0 0x564bc9081d70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8003610 tx=0x7ff1f8018000 comp rx=0 tx=0).stop 2024-09-19T07:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.705+0000 7ff207bf6740 1 -- 172.21.15.16:0/479489310 shutdown_connections 2024-09-19T07:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.705+0000 7ff207bf6740 1 --2- 172.21.15.16:0/479489310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8ff4d00 0x564bc8ff7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.705+0000 7ff207bf6740 1 --2- 172.21.15.16:0/479489310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x564bc8fd2a50 0x564bc8ff75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.705+0000 7ff207bf6740 1 --2- 172.21.15.16:0/479489310 >> [v2:172.21.15.146:6835/3642987189,v1:172.21.15.146:6837/3642987189] conn(0x564bc902fa70 0x564bc9017a90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.705+0000 7ff207bf6740 1 --2- 172.21.15.16:0/479489310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bc8fd12c0 0x564bc9081d70 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.705+0000 7ff207bf6740 1 -- 172.21.15.16:0/479489310 >> 172.21.15.16:0/479489310 conn(0x564bc8fcea80 msgr2=0x564bc8ff4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.705+0000 7ff1dd7fa640 0 client.6210 ms_handle_reset on v2:172.21.15.146:6835/3642987189 2024-09-19T07:49:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.705+0000 7ff207bf6740 1 -- 172.21.15.16:0/479489310 shutdown_connections 2024-09-19T07:49:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.705+0000 7ff207bf6740 1 -- 172.21.15.16:0/479489310 wait complete. 2024-09-19T07:49:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.705+0000 7ff207bf6740 20 client.6210 trim_cache size 0 max 16384 2024-09-19T07:49:46.713 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:49:46.713 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb", 2024-09-19T07:49:46.713 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-19T07:49:46.713 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 6096, 2024-09-19T07:49:46.713 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T07:49:46.713 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T07:49:46.713 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T07:49:46.714 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 5.6642350290000003, 2024-09-19T07:49:46.714 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 41, 2024-09-19T07:49:46.714 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 81, 2024-09-19T07:49:46.714 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T07:49:46.714 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 9.6896882580000003 2024-09-19T07:49:46.714 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:49:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.705+0000 7ff204d86640 1 -- 172.21.15.16:0/416672975 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff1e006c2b0 msgr2=0x7ff1e006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.705+0000 7ff204d86640 1 --2- 172.21.15.16:0/416672975 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff1e006c2b0 0x7ff1e006e770 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009a80 tx=0x7ff1f8008040 comp rx=0 tx=0).stop 2024-09-19T07:49:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.705+0000 7ff204d86640 1 -- 172.21.15.16:0/416672975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff200156e00 msgr2=0x7ff2000b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.705+0000 7ff204d86640 1 --2- 172.21.15.16:0/416672975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff200156e00 0x7ff2000b2a60 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80072f0 tx=0x7ff1e80077b0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.705+0000 7ff204d86640 1 -- 172.21.15.16:0/416672975 shutdown_connections 2024-09-19T07:49:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.705+0000 7ff204d86640 1 --2- 172.21.15.16:0/416672975 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff1e006c2b0 0x7ff1e006e770 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.705+0000 7ff204d86640 1 --2- 172.21.15.16:0/416672975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff200157260 0x7ff2000b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.706+0000 7ff204d86640 1 --2- 172.21.15.16:0/416672975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff200156e00 0x7ff2000b2a60 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.706+0000 7ff204d86640 1 --2- 172.21.15.16:0/416672975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1ec0051c0 0x7ff2001f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.706+0000 7ff204d86640 1 -- 172.21.15.16:0/416672975 >> 172.21.15.16:0/416672975 conn(0x7ff2000b3ed0 msgr2=0x7ff2000b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.706+0000 7ff204d86640 1 -- 172.21.15.16:0/416672975 shutdown_connections 2024-09-19T07:49:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.706+0000 7ff204d86640 1 -- 172.21.15.16:0/416672975 wait complete. 2024-09-19T07:49:46.717 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:49:46.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.818+0000 7f27ea942640 1 Processor -- start 2024-09-19T07:49:46.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.819+0000 7f27ea942640 1 -- start start 2024-09-19T07:49:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.819+0000 7f27ea942640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f27e415f0d0 0x7f27e415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.819+0000 7f27ea942640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27e40d0070 0x7f27e40d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.819+0000 7f27ea942640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f27e40d1940 0x7f27e40d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.819+0000 7f27ea942640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f27e40cf2b0 con 0x7f27e40d1940 2024-09-19T07:49:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.819+0000 7f27ea942640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f27e40cf590 con 0x7f27e415f0d0 2024-09-19T07:49:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.819+0000 7f27ea942640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f27e40cf870 con 0x7f27e40d0070 2024-09-19T07:49:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.820+0000 7f27e8edf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f27e40d1940 0x7f27e40d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.820+0000 7f27e8edf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f27e40d1940 0x7f27e40d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60996/0 (socket says 172.21.15.16:60996) 2024-09-19T07:49:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.820+0000 7f27e8edf640 1 -- 172.21.15.16:0/653371923 learned_addr learned my addr 172.21.15.16:0/653371923 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.820+0000 7f27e3fff640 1 --2- 172.21.15.16:0/653371923 >> v2:172.21.15.146:3300/0 conn(0x7f27e415f0d0 0x7f27e415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.820+0000 7f27e37fe640 1 --2- 172.21.15.16:0/653371923 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27e40d0070 0x7f27e40d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.820+0000 7f27e8edf640 1 -- 172.21.15.16:0/653371923 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27e40d0070 msgr2=0x7f27e40d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.820+0000 7f27e8edf640 1 --2- 172.21.15.16:0/653371923 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27e40d0070 0x7f27e40d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.820+0000 7f27e8edf640 1 -- 172.21.15.16:0/653371923 >> v2:172.21.15.146:3300/0 conn(0x7f27e415f0d0 msgr2=0x7f27e415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.820+0000 7f27e8edf640 1 --2- 172.21.15.16:0/653371923 >> v2:172.21.15.146:3300/0 conn(0x7f27e415f0d0 0x7f27e415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.820+0000 7f27e8edf640 1 -- 172.21.15.16:0/653371923 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e40cfb50 con 0x7f27e40d1940 2024-09-19T07:49:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.820+0000 7f27e3fff640 1 --2- 172.21.15.16:0/653371923 >> v2:172.21.15.146:3300/0 conn(0x7f27e415f0d0 0x7f27e415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.820+0000 7f27e37fe640 1 --2- 172.21.15.16:0/653371923 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27e40d0070 0x7f27e40d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.821+0000 7f27e8edf640 1 --2- 172.21.15.16:0/653371923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e40d1940 0x7f27e40d4130 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f27e40b6f40 tx=0x7f27dc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63a7b1139e5a86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.821+0000 7f27e2ffd640 1 -- 172.21.15.16:0/653371923 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc036070 con 0x7f27e40d1940 2024-09-19T07:49:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.821+0000 7f27e2ffd640 1 -- 172.21.15.16:0/653371923 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27dc031600 con 0x7f27e40d1940 2024-09-19T07:49:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.821+0000 7f27e2ffd640 1 -- 172.21.15.16:0/653371923 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc0319d0 con 0x7f27e40d1940 2024-09-19T07:49:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.821+0000 7f27ea942640 1 -- 172.21.15.16:0/653371923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e40d1940 msgr2=0x7f27e40d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.821+0000 7f27ea942640 1 --2- 172.21.15.16:0/653371923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e40d1940 0x7f27e40d4130 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f27e40b6f40 tx=0x7f27dc02a610 comp rx=0 tx=0).stop 2024-09-19T07:49:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.821+0000 7f27ea942640 1 -- 172.21.15.16:0/653371923 shutdown_connections 2024-09-19T07:49:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.821+0000 7f27ea942640 1 --2- 172.21.15.16:0/653371923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e40d1940 0x7f27e40d4130 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.821+0000 7f27ea942640 1 --2- 172.21.15.16:0/653371923 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27e40d0070 0x7f27e40d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.821+0000 7f27ea942640 1 --2- 172.21.15.16:0/653371923 >> v2:172.21.15.146:3300/0 conn(0x7f27e415f0d0 0x7f27e415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.821+0000 7f27ea942640 1 -- 172.21.15.16:0/653371923 >> 172.21.15.16:0/653371923 conn(0x7f27e409d760 msgr2=0x7f27e408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.821+0000 7f27ea942640 1 -- 172.21.15.16:0/653371923 shutdown_connections 2024-09-19T07:49:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.821+0000 7f27ea942640 1 -- 172.21.15.16:0/653371923 wait complete. 2024-09-19T07:49:46.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.822+0000 7f27ea942640 1 Processor -- start 2024-09-19T07:49:46.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.823+0000 7f27ea942640 1 -- start start 2024-09-19T07:49:46.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.823+0000 7f27ea942640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27e40d0070 0x7f27e40b9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.823+0000 7f27ea942640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e40d1940 0x7f27e41ea1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.823+0000 7f27ea942640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27e415f0d0 0x7f27e40b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.823+0000 7f27ea942640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27e40d47f0 con 0x7f27e40d1940 2024-09-19T07:49:46.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.823+0000 7f27ea942640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f27e40d4670 con 0x7f27e415f0d0 2024-09-19T07:49:46.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.823+0000 7f27ea942640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f27e40d4970 con 0x7f27e40d0070 2024-09-19T07:49:46.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.824+0000 7f27e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e40d1940 0x7f27e41ea1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.824+0000 7f27e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e40d1940 0x7f27e41ea1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32770/0 (socket says 172.21.15.16:32770) 2024-09-19T07:49:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.824+0000 7f27e37fe640 1 -- 172.21.15.16:0/3893664734 learned_addr learned my addr 172.21.15.16:0/3893664734 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.824+0000 7f27e3fff640 1 --2- 172.21.15.16:0/3893664734 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27e40d0070 0x7f27e40b9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.824+0000 7f27e8edf640 1 --2- 172.21.15.16:0/3893664734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27e415f0d0 0x7f27e40b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.824+0000 7f27e37fe640 1 -- 172.21.15.16:0/3893664734 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27e40d0070 msgr2=0x7f27e40b9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.824+0000 7f27e37fe640 1 --2- 172.21.15.16:0/3893664734 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27e40d0070 0x7f27e40b9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.824+0000 7f27e37fe640 1 -- 172.21.15.16:0/3893664734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27e415f0d0 msgr2=0x7f27e40b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.824+0000 7f27e37fe640 1 --2- 172.21.15.16:0/3893664734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27e415f0d0 0x7f27e40b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.824+0000 7f27e37fe640 1 -- 172.21.15.16:0/3893664734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e40ba170 con 0x7f27e40d1940 2024-09-19T07:49:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.824+0000 7f27e37fe640 1 --2- 172.21.15.16:0/3893664734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e40d1940 0x7f27e41ea1a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f27d800c820 tx=0x7f27d800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.824+0000 7f27e0ff9640 1 -- 172.21.15.16:0/3893664734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8015070 con 0x7f27e40d1940 2024-09-19T07:49:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.824+0000 7f27e0ff9640 1 -- 172.21.15.16:0/3893664734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d8007a30 con 0x7f27e40d1940 2024-09-19T07:49:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.824+0000 7f27e0ff9640 1 -- 172.21.15.16:0/3893664734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8007e20 con 0x7f27e40d1940 2024-09-19T07:49:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.824+0000 7f27ea942640 1 -- 172.21.15.16:0/3893664734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e40cb4b0 con 0x7f27e40d1940 2024-09-19T07:49:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.824+0000 7f27ea942640 1 -- 172.21.15.16:0/3893664734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e40cba70 con 0x7f27e40d1940 2024-09-19T07:49:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.825+0000 7f27ea942640 1 -- 172.21.15.16:0/3893664734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e40d1400 con 0x7f27e40d1940 2024-09-19T07:49:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.826+0000 7f27e0ff9640 1 -- 172.21.15.16:0/3893664734 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f27d8018070 con 0x7f27e40d1940 2024-09-19T07:49:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.826+0000 7f27e0ff9640 1 --2- 172.21.15.16:0/3893664734 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f27d006c2b0 0x7f27d006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.827+0000 7f27e0ff9640 1 -- 172.21.15.16:0/3893664734 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f27d808b470 con 0x7f27e40d1940 2024-09-19T07:49:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.827+0000 7f27e3fff640 1 --2- 172.21.15.16:0/3893664734 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f27d006c2b0 0x7f27d006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.827+0000 7f27e3fff640 1 --2- 172.21.15.16:0/3893664734 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f27d006c2b0 0x7f27d006e770 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f27d400ab60 tx=0x7f27d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:46.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:46.831+0000 7f27e0ff9640 1 -- 172.21.15.16:0/3893664734 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f27d8059380 con 0x7f27e40d1940 2024-09-19T07:49:47.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:47.045+0000 7f27ea942640 1 -- 172.21.15.16:0/3893664734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f27e406cf30 con 0x7f27e40d1940 2024-09-19T07:49:47.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:47.046+0000 7f27e0ff9640 1 -- 172.21.15.16:0/3893664734 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) ==== 53+0+73 (secure 0 0 0) 0x7f27d805d680 con 0x7f27e40d1940 2024-09-19T07:49:47.046 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:49:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:47.048+0000 7f27ea942640 1 -- 172.21.15.16:0/3893664734 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f27d006c2b0 msgr2=0x7f27d006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:47.048+0000 7f27ea942640 1 --2- 172.21.15.16:0/3893664734 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f27d006c2b0 0x7f27d006e770 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f27d400ab60 tx=0x7f27d4008040 comp rx=0 tx=0).stop 2024-09-19T07:49:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:47.048+0000 7f27ea942640 1 -- 172.21.15.16:0/3893664734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e40d1940 msgr2=0x7f27e41ea1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:47.048+0000 7f27ea942640 1 --2- 172.21.15.16:0/3893664734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e40d1940 0x7f27e41ea1a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f27d800c820 tx=0x7f27d800cce0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:47.048+0000 7f27ea942640 1 -- 172.21.15.16:0/3893664734 shutdown_connections 2024-09-19T07:49:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:47.048+0000 7f27ea942640 1 --2- 172.21.15.16:0/3893664734 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f27d006c2b0 0x7f27d006e770 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:47.048+0000 7f27ea942640 1 --2- 172.21.15.16:0/3893664734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27e415f0d0 0x7f27e40b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:47.048+0000 7f27ea942640 1 --2- 172.21.15.16:0/3893664734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e40d1940 0x7f27e41ea1a0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:47.048+0000 7f27ea942640 1 --2- 172.21.15.16:0/3893664734 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27e40d0070 0x7f27e40b9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:47.048+0000 7f27ea942640 1 -- 172.21.15.16:0/3893664734 >> 172.21.15.16:0/3893664734 conn(0x7f27e409d760 msgr2=0x7f27e40d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:47.048+0000 7f27ea942640 1 -- 172.21.15.16:0/3893664734 shutdown_connections 2024-09-19T07:49:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:47.048+0000 7f27ea942640 1 -- 172.21.15.16:0/3893664734 wait complete. 2024-09-19T07:49:47.060 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:49:47.060 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:49:47.060 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T07:49:47.060 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T07:49:47.060 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:47.060 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:49:47.060 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:49:47.060 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:49:47.060 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:49:47.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:47.061 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T07:49:47.077 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:49:47.078 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:49:47.078 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:49:47.078 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:47.078 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:49:47.078 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:47.078 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:49:47.078 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:47.078 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T07:49:47.078 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:49:47.078 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:47.078 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T07:49:47.078 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:49:47.079 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:47.079 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T07:49:47.079 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T07:49:47.079 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42402sec preferred_lft 42402sec 2024-09-19T07:49:47.079 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T07:49:47.079 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:47.079 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:49:47.079 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:47.079 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T07:49:47.079 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:49:47.079 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:47.079 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:49:47.079 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:47.079 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::32:7bff:fe92:8895/64 scope link 2024-09-19T07:49:47.080 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:47.080 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:49:47.080 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether e6:fd:21:6a:43:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:49:47.080 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e4fd:21ff:fe6a:4369/64 scope link 2024-09-19T07:49:47.080 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:47.080 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:49:47.080 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:49:47.080 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::80f4:81ff:fe7d:b7f0/64 scope link 2024-09-19T07:49:47.080 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:47.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:47.081 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:49:47.143 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:49:47.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:47.144 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T07:49:47.201 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T07:49:47.201 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:49:47.201 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T07:49:47.201 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T07:49:47.201 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:49:47.202 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:49:47.202 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:49:47.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T07:49:47.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T07:49:47.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T07:49:47.202 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:49:47.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:49:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:47.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:47.330 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:49:47.346 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T07:49:47.346 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:49:47.346 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:49:47.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:47.346 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-19T07:49:47.412 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T07:49:47.412 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:49:47.412 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:47.468 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:49:47.469 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:49:47.469 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:47.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:49:47.591 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:49:47.647 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:49:47.647 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:49:47.648 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:49:47.648 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:49:47.648 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:49:47.648 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:49:47.648 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:49:47.648 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:49:47.648 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:49:47.648 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:49:47.648 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:49:47.648 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:49:47.648 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:49:47.649 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:49:47.649 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:49:47.649 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:49:47.649 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:49:47.649 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:49:47.649 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:49:47.649 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:49:47.649 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:49:47.649 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:49:47.649 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:49:47.649 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:49:47.649 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:49:47.649 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:49:47.650 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:49:47.650 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:49:47.650 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:49:47.650 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:49:47.650 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:49:47.650 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:49:47.650 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:49:47.650 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:49:47.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:47.651 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:49:47.707 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:49:47.707 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T07:49:47.750 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:49:47.775 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:49:47.832 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:49:47.832 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:49:47.832 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:49:47.832 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:49:47.832 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:49:47.833 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:49:47.833 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:49:47.833 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:49:47.833 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:49:47.833 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:49:47.833 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:49:47.833 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:49:47.833 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:49:47.833 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:49:47.833 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:49:47.833 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:49:47.833 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:49:47.834 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:49:47.834 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:49:47.834 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:49:47.834 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:49:47.834 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:49:47.834 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:49:47.834 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:49:47.834 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:49:47.834 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:49:47.834 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:49:47.834 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:49:47.834 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:49:47.834 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:49:47.834 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:49:47.835 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:49:47.835 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:49:47.835 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:49:47.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:47.835 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:49:47.836 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-19T07:49:47.833+0000 7f2ae676e480 -1 init, newargv = 0x55c1169288f0 newargc=15 2024-09-19T07:49:47.836 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[58695]: starting ceph client 2024-09-19T07:49:47.838 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[58695]: starting fuse 2024-09-19T07:49:47.851 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:49:47.852 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-19T07:49:47.852 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:49:47.852 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:49:47.852 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:49:47.852 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:49:47.852 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:49:47.852 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:47.852 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:49:47.852 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:49:47.853 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:49:47.853 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:49:47.853 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:47.853 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:49:47.853 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:49:47.853 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:49:47.853 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:49:47.853 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:47.853 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:49:47.853 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:49:47.853 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:49:47.853 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:49:47.853 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:49:47.853 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:49:47.853 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:49:47.854 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:49:47.854 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:47.854 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:49:47.854 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:49:47.973 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.58695.asok 2024-09-19T07:49:47.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:47.984 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.58695.asok 2024-09-19T07:49:47.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:47.984 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.58695.asok status 2024-09-19T07:49:48.122 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:49:48.122 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:49:48.122 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:49:48.122 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:49:48.122 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:49:48.123 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:49:48.123 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:49:48.123 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "58695", 2024-09-19T07:49:48.123 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:49:48.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:48.123 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-19T07:49:48.123 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-19T07:49:48.123 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 6222, 2024-09-19T07:49:48.123 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:49:48.123 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:49:48.123 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:49:48.123 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 6222 2024-09-19T07:49:48.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:48.124 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:49:48.124 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:48.124 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:48.124 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2034930876 2024-09-19T07:49:48.124 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:48.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:48.124 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:49:48.124 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:48.124 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:48.124 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2034930876 2024-09-19T07:49:48.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:48.124 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.6222 192.168.0.1:0/2034930876", 2024-09-19T07:49:48.124 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/2034930876", 2024-09-19T07:49:48.124 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-19T07:49:48.125 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 41, 2024-09-19T07:49:48.125 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 81, 2024-09-19T07:49:48.125 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:49:48.125 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:49:48.125 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:49:48.125 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:49:48.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:48.131 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-19T07:49:48.131 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:48.147 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-19T07:49:48.147 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:48.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:48.147 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:48.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:49:48.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.356+0000 7f28b7b52640 1 Processor -- start 2024-09-19T07:49:48.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.356+0000 7f28b7b52640 1 -- start start 2024-09-19T07:49:48.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.357+0000 7f28b7b52640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28b00c8d20 0x7f28b006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:48.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.357+0000 7f28b7b52640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f28b006d260 0x7f28b01439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:48.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.357+0000 7f28b7b52640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28b00c7140 con 0x7f28b0143f30 2024-09-19T07:49:48.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.357+0000 7f28b7b52640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f28b0141960 con 0x7f28b006d260 2024-09-19T07:49:48.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.357+0000 7f28b7b52640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f28b0142200 con 0x7f28b00c8d20 2024-09-19T07:49:48.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.357+0000 7f28b60ef640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f28b0143f30 0x7f28b00d1e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36100/0 (socket says 172.21.15.16:36100) 2024-09-19T07:49:48.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.357+0000 7f28b60ef640 1 -- 172.21.15.16:0/505363707 learned_addr learned my addr 172.21.15.16:0/505363707 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:48.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.357+0000 7f28b58ee640 1 --2- 172.21.15.16:0/505363707 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28b00c8d20 0x7f28b006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:48.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.357+0000 7f28b50ed640 1 --2- 172.21.15.16:0/505363707 >> v2:172.21.15.146:3300/0 conn(0x7f28b006d260 0x7f28b01439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:48.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.357+0000 7f28b48ec640 1 -- 172.21.15.16:0/505363707 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1175399233 0 0) 0x7f28a0001490 con 0x7f28b0143f30 2024-09-19T07:49:48.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.358+0000 7f28b48ec640 1 -- 172.21.15.16:0/505363707 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2898003520 con 0x7f28b0143f30 2024-09-19T07:49:48.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.358+0000 7f28b58ee640 1 -- 172.21.15.16:0/505363707 >> v2:172.21.15.146:3300/0 conn(0x7f28b006d260 msgr2=0x7f28b01439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:48.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.358+0000 7f28b58ee640 1 --2- 172.21.15.16:0/505363707 >> v2:172.21.15.146:3300/0 conn(0x7f28b006d260 0x7f28b01439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:48.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.358+0000 7f28b58ee640 1 -- 172.21.15.16:0/505363707 >> v1:172.21.15.16:6789/0 conn(0x7f28b0143f30 legacy=0x7f28b00d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:48.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.358+0000 7f28b58ee640 1 -- 172.21.15.16:0/505363707 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b00c9100 con 0x7f28b00c8d20 2024-09-19T07:49:48.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.358+0000 7f28b58ee640 1 --2- 172.21.15.16:0/505363707 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28b00c8d20 0x7f28b006cd20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f28a80097a0 tx=0x7f28a802a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76cc32fc551fb497 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:48.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.359+0000 7f28b48ec640 1 -- 172.21.15.16:0/505363707 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8037070 con 0x7f28b00c8d20 2024-09-19T07:49:48.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.359+0000 7f28b48ec640 1 -- 172.21.15.16:0/505363707 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a802adb0 con 0x7f28b00c8d20 2024-09-19T07:49:48.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.359+0000 7f28b48ec640 1 -- 172.21.15.16:0/505363707 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8032640 con 0x7f28b00c8d20 2024-09-19T07:49:48.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.359+0000 7f28b7b52640 1 -- 172.21.15.16:0/505363707 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28b00c8d20 msgr2=0x7f28b006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:48.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.359+0000 7f28b7b52640 1 --2- 172.21.15.16:0/505363707 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28b00c8d20 0x7f28b006cd20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f28a80097a0 tx=0x7f28a802a060 comp rx=0 tx=0).stop 2024-09-19T07:49:48.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.359+0000 7f28b7b52640 1 -- 172.21.15.16:0/505363707 shutdown_connections 2024-09-19T07:49:48.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.359+0000 7f28b7b52640 1 --2- 172.21.15.16:0/505363707 >> v2:172.21.15.146:3300/0 conn(0x7f28b006d260 0x7f28b01439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:48.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.359+0000 7f28b7b52640 1 --2- 172.21.15.16:0/505363707 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28b00c8d20 0x7f28b006cd20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:48.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.359+0000 7f28b7b52640 1 -- 172.21.15.16:0/505363707 >> 172.21.15.16:0/505363707 conn(0x7f28b00b3ed0 msgr2=0x7f28b00b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:48.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.359+0000 7f28b7b52640 1 -- 172.21.15.16:0/505363707 shutdown_connections 2024-09-19T07:49:48.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.359+0000 7f28b7b52640 1 -- 172.21.15.16:0/505363707 wait complete. 2024-09-19T07:49:48.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.360+0000 7f28b7b52640 1 Processor -- start 2024-09-19T07:49:48.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.361+0000 7f28b7b52640 1 -- start start 2024-09-19T07:49:48.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.361+0000 7f28b7b52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28b006d260 0x7f28b0140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:48.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.361+0000 7f28b7b52640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28b00c8d20 0x7f28b00d0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:48.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.361+0000 7f28b7b52640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28b0143f30 0x7f28b00d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:48.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.361+0000 7f28b7b52640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28b0141bb0 con 0x7f28b006d260 2024-09-19T07:49:48.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.361+0000 7f28b7b52640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f28b006d6a0 con 0x7f28b00c8d20 2024-09-19T07:49:48.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.361+0000 7f28b7b52640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f28b0141e90 con 0x7f28b0143f30 2024-09-19T07:49:48.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.361+0000 7f28b58ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28b006d260 0x7f28b0140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:48.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.361+0000 7f28b58ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28b006d260 0x7f28b0140190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32780/0 (socket says 172.21.15.16:32780) 2024-09-19T07:49:48.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.361+0000 7f28b58ee640 1 -- 172.21.15.16:0/1879874455 learned_addr learned my addr 172.21.15.16:0/1879874455 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:48.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.361+0000 7f28b60ef640 1 --2- 172.21.15.16:0/1879874455 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28b0143f30 0x7f28b00d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.361+0000 7f28b50ed640 1 --2- 172.21.15.16:0/1879874455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28b00c8d20 0x7f28b00d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.362+0000 7f28b58ee640 1 -- 172.21.15.16:0/1879874455 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28b0143f30 msgr2=0x7f28b00d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.362+0000 7f28b58ee640 1 --2- 172.21.15.16:0/1879874455 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28b0143f30 0x7f28b00d0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.362+0000 7f28b58ee640 1 -- 172.21.15.16:0/1879874455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28b00c8d20 msgr2=0x7f28b00d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.362+0000 7f28b58ee640 1 --2- 172.21.15.16:0/1879874455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28b00c8d20 0x7f28b00d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.362+0000 7f28b58ee640 1 -- 172.21.15.16:0/1879874455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b01e8bb0 con 0x7f28b006d260 2024-09-19T07:49:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.362+0000 7f28b58ee640 1 --2- 172.21.15.16:0/1879874455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28b006d260 0x7f28b0140190 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f28a8035040 tx=0x7f28a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.362+0000 7f28ae7fc640 1 -- 172.21.15.16:0/1879874455 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8037050 con 0x7f28b006d260 2024-09-19T07:49:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.362+0000 7f28ae7fc640 1 -- 172.21.15.16:0/1879874455 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a8038040 con 0x7f28b006d260 2024-09-19T07:49:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.362+0000 7f28ae7fc640 1 -- 172.21.15.16:0/1879874455 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a80044c0 con 0x7f28b006d260 2024-09-19T07:49:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.362+0000 7f28b7b52640 1 -- 172.21.15.16:0/1879874455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b01e88d0 con 0x7f28b006d260 2024-09-19T07:49:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.362+0000 7f28b7b52640 1 -- 172.21.15.16:0/1879874455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b01572c0 con 0x7f28b006d260 2024-09-19T07:49:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.363+0000 7f28b7b52640 1 -- 172.21.15.16:0/1879874455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b0144360 con 0x7f28b006d260 2024-09-19T07:49:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.364+0000 7f28ae7fc640 1 -- 172.21.15.16:0/1879874455 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f28a8045020 con 0x7f28b006d260 2024-09-19T07:49:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.364+0000 7f28ae7fc640 1 --2- 172.21.15.16:0/1879874455 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f289c06c2b0 0x7f289c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:48.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.365+0000 7f28ae7fc640 1 -- 172.21.15.16:0/1879874455 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f28a80ab660 con 0x7f28b006d260 2024-09-19T07:49:48.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.365+0000 7f28b50ed640 1 --2- 172.21.15.16:0/1879874455 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f289c06c2b0 0x7f289c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:48.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.365+0000 7f28b50ed640 1 --2- 172.21.15.16:0/1879874455 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f289c06c2b0 0x7f289c06e770 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f28b00721b0 tx=0x7f28a4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:48.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.369+0000 7f28ae7fc640 1 -- 172.21.15.16:0/1879874455 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f28a8079550 con 0x7f28b006d260 2024-09-19T07:49:48.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.583+0000 7f28b7b52640 1 -- 172.21.15.16:0/1879874455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f28b00690c0 con 0x7f28b006d260 2024-09-19T07:49:48.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.584+0000 7f28ae7fc640 1 -- 172.21.15.16:0/1879874455 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) ==== 53+0+73 (secure 0 0 0) 0x7f28a807d850 con 0x7f28b006d260 2024-09-19T07:49:48.584 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:49:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.586+0000 7f28b7b52640 1 -- 172.21.15.16:0/1879874455 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f289c06c2b0 msgr2=0x7f289c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.586+0000 7f28b7b52640 1 --2- 172.21.15.16:0/1879874455 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f289c06c2b0 0x7f289c06e770 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f28b00721b0 tx=0x7f28a4009190 comp rx=0 tx=0).stop 2024-09-19T07:49:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.586+0000 7f28b7b52640 1 -- 172.21.15.16:0/1879874455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28b006d260 msgr2=0x7f28b0140190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.586+0000 7f28b7b52640 1 --2- 172.21.15.16:0/1879874455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28b006d260 0x7f28b0140190 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f28a8035040 tx=0x7f28a80046c0 comp rx=0 tx=0).stop 2024-09-19T07:49:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.586+0000 7f28b7b52640 1 -- 172.21.15.16:0/1879874455 shutdown_connections 2024-09-19T07:49:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.586+0000 7f28b7b52640 1 --2- 172.21.15.16:0/1879874455 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f289c06c2b0 0x7f289c06e770 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.586+0000 7f28b7b52640 1 --2- 172.21.15.16:0/1879874455 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28b0143f30 0x7f28b00d0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.586+0000 7f28b7b52640 1 --2- 172.21.15.16:0/1879874455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28b00c8d20 0x7f28b00d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.586+0000 7f28b7b52640 1 --2- 172.21.15.16:0/1879874455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28b006d260 0x7f28b0140190 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.586+0000 7f28b7b52640 1 -- 172.21.15.16:0/1879874455 >> 172.21.15.16:0/1879874455 conn(0x7f28b00b3ed0 msgr2=0x7f28b0161520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:48.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.586+0000 7f28b7b52640 1 -- 172.21.15.16:0/1879874455 shutdown_connections 2024-09-19T07:49:48.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:48.586+0000 7f28b7b52640 1 -- 172.21.15.16:0/1879874455 wait complete. 2024-09-19T07:49:48.597 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:49:48.597 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:49:48.597 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-09-19T07:49:48.597 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T07:49:48.597 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T07:49:48.597 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:49:48.597 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:49:48.598 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:49:48.598 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:49:48.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:48.598 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-09-19T07:49:48.618 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:49:48.618 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:49:48.618 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:49:48.618 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:48.618 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-09-19T07:49:48.618 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:48.618 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:49:48.618 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:48.618 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2024-09-19T07:49:48.618 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:49:48.618 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:48.619 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2024-09-19T07:49:48.619 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:49:48.619 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:48.619 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:49:48.619 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:48.619 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T07:49:48.619 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42368sec preferred_lft 42368sec 2024-09-19T07:49:48.619 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link noprefixroute 2024-09-19T07:49:48.619 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:48.619 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:49:48.619 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:49:48.619 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:49:48.619 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:48.620 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::e47f:32ff:fe68:31ca/64 scope link 2024-09-19T07:49:48.620 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:48.620 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:49:48.620 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether f6:1f:22:44:9d:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:49:48.620 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::f41f:22ff:fe44:9dcd/64 scope link 2024-09-19T07:49:48.620 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:48.620 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:49:48.620 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:49:48.620 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::8459:eeff:fe21:a113/64 scope link 2024-09-19T07:49:48.620 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:49:48.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:48.621 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:49:48.686 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-09-19T07:49:48.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:48.688 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-09-19T07:49:48.747 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-09-19T07:49:48.747 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:49:48.747 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T07:49:48.747 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T07:49:48.747 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:49:48.748 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:49:48.748 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-19T07:49:48.748 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T07:49:48.748 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T07:49:48.748 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T07:49:48.748 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-19T07:49:48.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:49:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:49:48.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:49:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:48.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:48.881 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-09-19T07:49:48.897 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T07:49:48.897 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T07:49:48.897 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:49:48.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:48.897 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2024-09-19T07:49:48.962 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T07:49:48.962 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:49:48.962 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T07:49:49.017 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:49:49.017 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:49:49.017 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:49:49.074 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:49:49.139 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:49:49.196 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:49:49.196 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:49:49.197 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:49:49.197 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:49:49.197 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:49:49.197 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:49:49.197 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:49:49.197 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:49:49.197 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:49:49.197 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:49:49.197 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:49:49.197 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:49:49.197 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:49:49.198 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:49:49.198 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:49:49.198 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:49:49.198 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:49:49.198 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:49:49.198 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:49:49.198 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:49:49.198 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:49:49.198 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:49:49.198 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:49:49.198 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:49:49.198 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:49:49.198 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:49:49.199 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:49:49.199 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:49:49.199 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:49:49.199 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:49:49.199 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:49:49.199 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:49:49.199 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:49:49.199 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:49:49.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:49.199 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:49:49.252 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:49:49.252 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T07:49:49.294 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:49:49.319 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:49:49.375 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:49:49.375 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:49:49.375 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:49:49.375 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:49:49.375 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:49:49.375 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:49:49.376 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:49:49.376 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:49:49.376 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:49:49.376 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:49:49.376 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:49:49.376 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:49:49.376 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:49:49.376 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:49:49.376 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:49:49.376 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:49:49.376 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:49:49.376 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:49:49.376 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:49:49.377 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:49:49.377 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:49:49.377 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:49:49.377 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:49:49.377 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:49:49.377 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:49:49.377 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:49:49.377 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:49:49.377 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:49:49.377 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:49:49.378 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:49:49.378 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:49:49.378 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:49:49.378 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:49:49.378 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:49:49.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:49.378 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:49:49.379 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2024-09-19T07:49:49.374+0000 7f19ce931480 -1 init, newargv = 0x55f3bb8d0830 newargc=15 2024-09-19T07:49:49.379 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[46263]: starting ceph client 2024-09-19T07:49:49.379 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[46263]: starting fuse 2024-09-19T07:49:49.394 INFO:teuthology.orchestra.run.smithi146.stdout:42 2024-09-19T07:49:49.394 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-19T07:49:49.394 DEBUG:teuthology.orchestra.run.smithi146:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:49:49.394 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2024-09-19T07:49:49.394 DEBUG:teuthology.orchestra.run.smithi146:> import re 2024-09-19T07:49:49.394 DEBUG:teuthology.orchestra.run.smithi146:> import os 2024-09-19T07:49:49.394 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2024-09-19T07:49:49.394 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:49:49.394 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2024-09-19T07:49:49.394 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T07:49:49.395 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2024-09-19T07:49:49.395 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T07:49:49.395 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:49:49.395 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2024-09-19T07:49:49.395 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2024-09-19T07:49:49.395 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2024-09-19T07:49:49.395 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2024-09-19T07:49:49.395 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:49:49.395 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2024-09-19T07:49:49.395 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:49:49.395 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:49:49.395 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:49:49.395 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2024-09-19T07:49:49.395 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2024-09-19T07:49:49.395 DEBUG:teuthology.orchestra.run.smithi146:> return f 2024-09-19T07:49:49.396 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:49:49.396 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:49:49.396 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2024-09-19T07:49:49.396 DEBUG:teuthology.orchestra.run.smithi146:> ' 2024-09-19T07:49:49.511 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.46263.asok 2024-09-19T07:49:49.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:49:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:49.522 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.46263.asok 2024-09-19T07:49:49.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:49.522 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.46263.asok status 2024-09-19T07:49:49.656 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-19T07:49:49.656 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2024-09-19T07:49:49.657 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:49:49.657 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:49:49.657 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2024-09-19T07:49:49.657 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2024-09-19T07:49:49.657 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T07:49:49.657 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "46263", 2024-09-19T07:49:49.657 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2024-09-19T07:49:49.657 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:49:49.657 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2024-09-19T07:49:49.657 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2024-09-19T07:49:49.657 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 5932, 2024-09-19T07:49:49.657 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2024-09-19T07:49:49.657 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2024-09-19T07:49:49.658 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2024-09-19T07:49:49.658 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 5932 2024-09-19T07:49:49.658 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:49:49.658 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:49:49.658 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:49:49.658 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:49.658 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 2223430545 2024-09-19T07:49:49.658 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-19T07:49:49.658 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:49:49.658 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:49:49.658 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:49:49.658 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:49.658 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 2223430545 2024-09-19T07:49:49.658 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:49:49.659 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.5932 192.168.0.1:0/2223430545", 2024-09-19T07:49:49.659 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/2223430545", 2024-09-19T07:49:49.659 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2024-09-19T07:49:49.659 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 41, 2024-09-19T07:49:49.659 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 81, 2024-09-19T07:49:49.659 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:49:49.659 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2024-09-19T07:49:49.659 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2024-09-19T07:49:49.659 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-19T07:49:49.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:49.665 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2024-09-19T07:49:49.665 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T07:49:49.681 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2024-09-19T07:49:49.681 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T07:49:49.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:49.681 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:49:49.745 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-09-19T07:49:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.863+0000 7f53a241e640 1 Processor -- start 2024-09-19T07:49:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.863+0000 7f53a241e640 1 -- start start 2024-09-19T07:49:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.864+0000 7f53a241e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f539c142ae0 0x7f539c1692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.864+0000 7f53a241e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f539c1620b0 con 0x7f539c0d21f0 2024-09-19T07:49:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.864+0000 7f53a241e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f539c162370 con 0x7f539c072590 2024-09-19T07:49:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.864+0000 7f53a241e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f539c141750 con 0x7f539c142ae0 2024-09-19T07:49:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.864+0000 7f53a09bb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f539c142ae0 0x7f539c1692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.864+0000 7f539bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f539c0d21f0 0x7f539c0d25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36104/0 (socket says 172.21.15.16:36104) 2024-09-19T07:49:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.864+0000 7f539bfff640 1 -- 172.21.15.16:0/3452006983 learned_addr learned my addr 172.21.15.16:0/3452006983 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.864+0000 7f53a09bb640 1 -- 172.21.15.16:0/3452006983 >> v1:172.21.15.146:6789/0 conn(0x7f539c072590 legacy=0x7f539c06d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.864+0000 7f53a09bb640 1 -- 172.21.15.16:0/3452006983 >> v1:172.21.15.16:6789/0 conn(0x7f539c0d21f0 legacy=0x7f539c0d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.864+0000 7f53a09bb640 1 -- 172.21.15.16:0/3452006983 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c162650 con 0x7f539c142ae0 2024-09-19T07:49:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.865+0000 7f53a09bb640 1 --2- 172.21.15.16:0/3452006983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f539c142ae0 0x7f539c1692c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5394009d90 tx=0x7f539402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def34ae5f1aacdaa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.865+0000 7f539affd640 1 -- 172.21.15.16:0/3452006983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394007740 con 0x7f539c142ae0 2024-09-19T07:49:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.865+0000 7f539affd640 1 -- 172.21.15.16:0/3452006983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53940078e0 con 0x7f539c142ae0 2024-09-19T07:49:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.865+0000 7f539affd640 1 -- 172.21.15.16:0/3452006983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394007cb0 con 0x7f539c142ae0 2024-09-19T07:49:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.866+0000 7f53a241e640 1 -- 172.21.15.16:0/3452006983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f539c142ae0 msgr2=0x7f539c1692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.866+0000 7f53a241e640 1 --2- 172.21.15.16:0/3452006983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f539c142ae0 0x7f539c1692c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5394009d90 tx=0x7f539402a660 comp rx=0 tx=0).stop 2024-09-19T07:49:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.866+0000 7f53a241e640 1 -- 172.21.15.16:0/3452006983 shutdown_connections 2024-09-19T07:49:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.866+0000 7f53a241e640 1 --2- 172.21.15.16:0/3452006983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f539c142ae0 0x7f539c1692c0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.866+0000 7f53a241e640 1 -- 172.21.15.16:0/3452006983 >> 172.21.15.16:0/3452006983 conn(0x7f539c0b3ed0 msgr2=0x7f539c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.866+0000 7f53a241e640 1 -- 172.21.15.16:0/3452006983 shutdown_connections 2024-09-19T07:49:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.866+0000 7f53a241e640 1 -- 172.21.15.16:0/3452006983 wait complete. 2024-09-19T07:49:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.867+0000 7f53a241e640 1 Processor -- start 2024-09-19T07:49:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.868+0000 7f53a241e640 1 -- start start 2024-09-19T07:49:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.868+0000 7f53a241e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f539c072590 0x7f539c159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.868+0000 7f53a241e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f539c0d21f0 0x7f539c15a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.868+0000 7f53a241e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f539c142ae0 0x7f539c157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.868+0000 7f53a241e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f539c162c10 con 0x7f539c0d21f0 2024-09-19T07:49:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.868+0000 7f53a241e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f539c06d7f0 con 0x7f539c072590 2024-09-19T07:49:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.868+0000 7f53a241e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f539c158c30 con 0x7f539c142ae0 2024-09-19T07:49:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.868+0000 7f539b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f539c0d21f0 0x7f539c15a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.868+0000 7f539b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f539c0d21f0 0x7f539c15a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32796/0 (socket says 172.21.15.16:32796) 2024-09-19T07:49:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.868+0000 7f539b7fe640 1 -- 172.21.15.16:0/735306189 learned_addr learned my addr 172.21.15.16:0/735306189 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.868+0000 7f539bfff640 1 --2- 172.21.15.16:0/735306189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f539c072590 0x7f539c159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.869+0000 7f53a09bb640 1 --2- 172.21.15.16:0/735306189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f539c142ae0 0x7f539c157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.869+0000 7f539b7fe640 1 -- 172.21.15.16:0/735306189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f539c142ae0 msgr2=0x7f539c157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.869+0000 7f539b7fe640 1 --2- 172.21.15.16:0/735306189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f539c142ae0 0x7f539c157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.869+0000 7f539b7fe640 1 -- 172.21.15.16:0/735306189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f539c072590 msgr2=0x7f539c159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.869+0000 7f539b7fe640 1 --2- 172.21.15.16:0/735306189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f539c072590 0x7f539c159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.869+0000 7f539b7fe640 1 -- 172.21.15.16:0/735306189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c15a7b0 con 0x7f539c0d21f0 2024-09-19T07:49:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.869+0000 7f539b7fe640 1 --2- 172.21.15.16:0/735306189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f539c0d21f0 0x7f539c15a160 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f539000c400 tx=0x7f539000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.869+0000 7f5398ff9640 1 -- 172.21.15.16:0/735306189 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53900052d0 con 0x7f539c0d21f0 2024-09-19T07:49:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.869+0000 7f5398ff9640 1 -- 172.21.15.16:0/735306189 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5390005490 con 0x7f539c0d21f0 2024-09-19T07:49:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.869+0000 7f53a241e640 1 -- 172.21.15.16:0/735306189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539c1590f0 con 0x7f539c0d21f0 2024-09-19T07:49:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.869+0000 7f5398ff9640 1 -- 172.21.15.16:0/735306189 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390007260 con 0x7f539c0d21f0 2024-09-19T07:49:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.869+0000 7f53a241e640 1 -- 172.21.15.16:0/735306189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539c15aa90 con 0x7f539c0d21f0 2024-09-19T07:49:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.870+0000 7f53a241e640 1 -- 172.21.15.16:0/735306189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f539c06d0e0 con 0x7f539c0d21f0 2024-09-19T07:49:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.871+0000 7f5398ff9640 1 -- 172.21.15.16:0/735306189 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5390007400 con 0x7f539c0d21f0 2024-09-19T07:49:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.871+0000 7f5398ff9640 1 --2- 172.21.15.16:0/735306189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f538806c2b0 0x7f538806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.871+0000 7f5398ff9640 1 -- 172.21.15.16:0/735306189 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f539008aa10 con 0x7f539c0d21f0 2024-09-19T07:49:49.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.872+0000 7f539bfff640 1 --2- 172.21.15.16:0/735306189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f538806c2b0 0x7f538806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.872+0000 7f539bfff640 1 --2- 172.21.15.16:0/735306189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f538806c2b0 0x7f538806e770 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f538c005880 tx=0x7f538c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:49.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:49.876+0000 7f5398ff9640 1 -- 172.21.15.16:0/735306189 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5390058a30 con 0x7f539c0d21f0 2024-09-19T07:49:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.050+0000 7f53a241e640 1 -- 172.21.15.16:0/735306189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7f539c0690c0 con 0x7f539c0d21f0 2024-09-19T07:49:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.051+0000 7f5398ff9640 1 -- 172.21.15.16:0/735306189 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v27) ==== 106+0+4 (secure 0 0 0) 0x7f539005cd30 con 0x7f539c0d21f0 2024-09-19T07:49:50.051 INFO:teuthology.orchestra.run.smithi016.stdout:100 2024-09-19T07:49:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.053+0000 7f53a241e640 1 -- 172.21.15.16:0/735306189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f538806c2b0 msgr2=0x7f538806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.053+0000 7f53a241e640 1 --2- 172.21.15.16:0/735306189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f538806c2b0 0x7f538806e770 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f538c005880 tx=0x7f538c009040 comp rx=0 tx=0).stop 2024-09-19T07:49:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.053+0000 7f53a241e640 1 -- 172.21.15.16:0/735306189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f539c0d21f0 msgr2=0x7f539c15a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:50.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.053+0000 7f53a241e640 1 --2- 172.21.15.16:0/735306189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f539c0d21f0 0x7f539c15a160 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f539000c400 tx=0x7f539000c8c0 comp rx=0 tx=0).stop 2024-09-19T07:49:50.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.053+0000 7f53a241e640 1 -- 172.21.15.16:0/735306189 shutdown_connections 2024-09-19T07:49:50.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.053+0000 7f53a241e640 1 --2- 172.21.15.16:0/735306189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f538806c2b0 0x7f538806e770 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:50.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.053+0000 7f53a241e640 1 --2- 172.21.15.16:0/735306189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f539c142ae0 0x7f539c157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:50.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.053+0000 7f53a241e640 1 --2- 172.21.15.16:0/735306189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f539c0d21f0 0x7f539c15a160 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:50.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.053+0000 7f53a241e640 1 --2- 172.21.15.16:0/735306189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f539c072590 0x7f539c159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:50.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.053+0000 7f53a241e640 1 -- 172.21.15.16:0/735306189 >> 172.21.15.16:0/735306189 conn(0x7f539c0b3ed0 msgr2=0x7f539c0d25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:50.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.053+0000 7f53a241e640 1 -- 172.21.15.16:0/735306189 shutdown_connections 2024-09-19T07:49:50.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.053+0000 7f53a241e640 1 -- 172.21.15.16:0/735306189 wait complete. 2024-09-19T07:49:50.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_caps_per_client 200 2024-09-19T07:49:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.171+0000 7f22d0a96640 1 Processor -- start 2024-09-19T07:49:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.171+0000 7f22d0a96640 1 -- start start 2024-09-19T07:49:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.172+0000 7f22d0a96640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f22cc0c8d20 0x7f22cc06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.172+0000 7f22d0a96640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22cc0c7140 con 0x7f22cc06d260 2024-09-19T07:49:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.172+0000 7f22d0a96640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22cc0c9100 con 0x7f22cc144100 2024-09-19T07:49:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.172+0000 7f22d0a96640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f22cc06d6d0 con 0x7f22cc0c8d20 2024-09-19T07:49:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.172+0000 7f22ca59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f22cc0c8d20 0x7f22cc06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.172+0000 7f22c9d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f22cc06d260 0x7f22cc1439f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36112/0 (socket says 172.21.15.16:36112) 2024-09-19T07:49:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.172+0000 7f22c9d9b640 1 -- 172.21.15.16:0/1415946937 learned_addr learned my addr 172.21.15.16:0/1415946937 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.172+0000 7f22ca59c640 1 -- 172.21.15.16:0/1415946937 >> v1:172.21.15.146:6789/0 conn(0x7f22cc144100 legacy=0x7f22cc0d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.172+0000 7f22ca59c640 1 -- 172.21.15.16:0/1415946937 >> v1:172.21.15.16:6789/0 conn(0x7f22cc06d260 legacy=0x7f22cc1439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.172+0000 7f22ca59c640 1 -- 172.21.15.16:0/1415946937 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22cc144760 con 0x7f22cc0c8d20 2024-09-19T07:49:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.172+0000 7f22c959a640 1 -- 172.21.15.16:0/1415946937 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2212134580 0 0) 0x7f22c0001110 con 0x7f22cc06d260 2024-09-19T07:49:50.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.173+0000 7f22ca59c640 1 --2- 172.21.15.16:0/1415946937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22cc0c8d20 0x7f22cc06cd20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0097a0 tx=0x7f22bc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8c055f273b341b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.173+0000 7f22c959a640 1 -- 172.21.15.16:0/1415946937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc037070 con 0x7f22cc0c8d20 2024-09-19T07:49:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.173+0000 7f22c959a640 1 -- 172.21.15.16:0/1415946937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22bc032440 con 0x7f22cc0c8d20 2024-09-19T07:49:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.173+0000 7f22c959a640 1 -- 172.21.15.16:0/1415946937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc02ae60 con 0x7f22cc0c8d20 2024-09-19T07:49:50.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.174+0000 7f22d0a96640 1 -- 172.21.15.16:0/1415946937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22cc0c8d20 msgr2=0x7f22cc06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.174+0000 7f22d0a96640 1 --2- 172.21.15.16:0/1415946937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22cc0c8d20 0x7f22cc06cd20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0097a0 tx=0x7f22bc02a110 comp rx=0 tx=0).stop 2024-09-19T07:49:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.174+0000 7f22d0a96640 1 -- 172.21.15.16:0/1415946937 shutdown_connections 2024-09-19T07:49:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.174+0000 7f22d0a96640 1 --2- 172.21.15.16:0/1415946937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22cc0c8d20 0x7f22cc06cd20 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.174+0000 7f22d0a96640 1 -- 172.21.15.16:0/1415946937 >> 172.21.15.16:0/1415946937 conn(0x7f22cc0b3ed0 msgr2=0x7f22cc0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.174+0000 7f22d0a96640 1 -- 172.21.15.16:0/1415946937 shutdown_connections 2024-09-19T07:49:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.174+0000 7f22d0a96640 1 -- 172.21.15.16:0/1415946937 wait complete. 2024-09-19T07:49:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.175+0000 7f22d0a96640 1 Processor -- start 2024-09-19T07:49:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.175+0000 7f22d0a96640 1 -- start start 2024-09-19T07:49:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.176+0000 7f22d0a96640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22cc06d260 0x7f22cc140b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.176+0000 7f22d0a96640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22cc0c8d20 0x7f22cc0d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.176+0000 7f22d0a96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22cc144100 0x7f22cc0af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.176+0000 7f22d0a96640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc142200 con 0x7f22cc144100 2024-09-19T07:49:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.176+0000 7f22d0a96640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc141960 con 0x7f22cc06d260 2024-09-19T07:49:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.176+0000 7f22d0a96640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f22cc144510 con 0x7f22cc0c8d20 2024-09-19T07:49:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.176+0000 7f22cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22cc144100 0x7f22cc0af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.176+0000 7f22cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22cc144100 0x7f22cc0af7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32822/0 (socket says 172.21.15.16:32822) 2024-09-19T07:49:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.176+0000 7f22cad9d640 1 -- 172.21.15.16:0/187417781 learned_addr learned my addr 172.21.15.16:0/187417781 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.176+0000 7f22ca59c640 1 --2- 172.21.15.16:0/187417781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22cc06d260 0x7f22cc140b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.176+0000 7f22c9d9b640 1 --2- 172.21.15.16:0/187417781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22cc0c8d20 0x7f22cc0d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.176+0000 7f22cad9d640 1 -- 172.21.15.16:0/187417781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22cc0c8d20 msgr2=0x7f22cc0d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.176+0000 7f22cad9d640 1 --2- 172.21.15.16:0/187417781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22cc0c8d20 0x7f22cc0d25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.176+0000 7f22cad9d640 1 -- 172.21.15.16:0/187417781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22cc06d260 msgr2=0x7f22cc140b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.176+0000 7f22cad9d640 1 --2- 172.21.15.16:0/187417781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22cc06d260 0x7f22cc140b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.176+0000 7f22cad9d640 1 -- 172.21.15.16:0/187417781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22cc0b1be0 con 0x7f22cc144100 2024-09-19T07:49:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.177+0000 7f22cad9d640 1 --2- 172.21.15.16:0/187417781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22cc144100 0x7f22cc0af7a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f22c400c7d0 tx=0x7f22c400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.177+0000 7f22baffd640 1 -- 172.21.15.16:0/187417781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4003b70 con 0x7f22cc144100 2024-09-19T07:49:50.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.177+0000 7f22baffd640 1 -- 172.21.15.16:0/187417781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c4003d30 con 0x7f22cc144100 2024-09-19T07:49:50.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.177+0000 7f22d0a96640 1 -- 172.21.15.16:0/187417781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22cc0aee40 con 0x7f22cc144100 2024-09-19T07:49:50.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.177+0000 7f22baffd640 1 -- 172.21.15.16:0/187417781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c400aa90 con 0x7f22cc144100 2024-09-19T07:49:50.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.177+0000 7f22d0a96640 1 -- 172.21.15.16:0/187417781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22cc0aeb60 con 0x7f22cc144100 2024-09-19T07:49:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.178+0000 7f22d0a96640 1 -- 172.21.15.16:0/187417781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22cc06cd20 con 0x7f22cc144100 2024-09-19T07:49:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.178+0000 7f22baffd640 1 -- 172.21.15.16:0/187417781 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f22c400ac30 con 0x7f22cc144100 2024-09-19T07:49:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.179+0000 7f22baffd640 1 --2- 172.21.15.16:0/187417781 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22ac06c2b0 0x7f22ac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.179+0000 7f22baffd640 1 -- 172.21.15.16:0/187417781 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f22c408b320 con 0x7f22cc144100 2024-09-19T07:49:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.179+0000 7f22ca59c640 1 --2- 172.21.15.16:0/187417781 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22ac06c2b0 0x7f22ac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.179+0000 7f22ca59c640 1 --2- 172.21.15.16:0/187417781 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22ac06c2b0 0x7f22ac06e770 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f22bc034000 tx=0x7f22bc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:50.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.183+0000 7f22baffd640 1 -- 172.21.15.16:0/187417781 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f22cc0aee40 con 0x7f22cc144100 2024-09-19T07:49:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.358+0000 7f22d0a96640 1 -- 172.21.15.16:0/187417781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_max_caps_per_client}] v 0) -- 0x7f22cc0b2d60 con 0x7f22cc144100 2024-09-19T07:49:50.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.360+0000 7f22baffd640 1 -- 172.21.15.16:0/187417781 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_caps_per_client}]=0 v28)=0 v28) ==== 123+0+0 (secure 0 0 0) 0x7f22cc0b1be0 con 0x7f22cc144100 2024-09-19T07:49:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.362+0000 7f22d0a96640 1 -- 172.21.15.16:0/187417781 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22ac06c2b0 msgr2=0x7f22ac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.362+0000 7f22d0a96640 1 --2- 172.21.15.16:0/187417781 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22ac06c2b0 0x7f22ac06e770 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f22bc034000 tx=0x7f22bc02e000 comp rx=0 tx=0).stop 2024-09-19T07:49:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.362+0000 7f22d0a96640 1 -- 172.21.15.16:0/187417781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22cc144100 msgr2=0x7f22cc0af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.362+0000 7f22d0a96640 1 --2- 172.21.15.16:0/187417781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22cc144100 0x7f22cc0af7a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f22c400c7d0 tx=0x7f22c400cc90 comp rx=0 tx=0).stop 2024-09-19T07:49:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.363+0000 7f22d0a96640 1 -- 172.21.15.16:0/187417781 shutdown_connections 2024-09-19T07:49:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.363+0000 7f22d0a96640 1 --2- 172.21.15.16:0/187417781 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22ac06c2b0 0x7f22ac06e770 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.363+0000 7f22d0a96640 1 --2- 172.21.15.16:0/187417781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22cc144100 0x7f22cc0af7a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.363+0000 7f22d0a96640 1 --2- 172.21.15.16:0/187417781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22cc0c8d20 0x7f22cc0d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.363+0000 7f22d0a96640 1 --2- 172.21.15.16:0/187417781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22cc06d260 0x7f22cc140b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.363+0000 7f22d0a96640 1 -- 172.21.15.16:0/187417781 >> 172.21.15.16:0/187417781 conn(0x7f22cc0b3ed0 msgr2=0x7f22cc0b1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.363+0000 7f22d0a96640 1 -- 172.21.15.16:0/187417781 shutdown_connections 2024-09-19T07:49:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:50.363+0000 7f22d0a96640 1 -- 172.21.15.16:0/187417781 wait complete. 2024-09-19T07:49:50.374 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:49:50.374 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:49:50.391 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-19T07:49:50.391 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:49:50.391 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-19T07:49:50.391 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:50.391 DEBUG:teuthology.orchestra.run.smithi016:> n = 600 2024-09-19T07:49:50.391 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-19T07:49:50.391 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-19T07:49:50.391 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-19T07:49:50.391 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/foo/" 2024-09-19T07:49:50.391 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:50.391 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-19T07:49:50.391 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-19T07:49:50.391 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-19T07:49:50.392 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:50.392 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-19T07:49:50.392 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-19T07:49:50.392 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:50.392 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-19T07:49:50.392 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-19T07:49:50.392 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-19T07:49:50.392 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-19T07:49:50.392 DEBUG:teuthology.orchestra.run.smithi016:> if True: 2024-09-19T07:49:50.392 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-19T07:49:50.392 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-19T07:49:50.392 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:49:50.392 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-19T07:49:50.393 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:49:50.393 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:49:50.393 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-19T07:49:50.393 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-19T07:49:50.393 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-19T07:49:50.393 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:49:50.393 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:49:50.393 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-19T07:49:50.393 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-19T07:49:50.393 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:49:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:52.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:49:52.860 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:49:52.860 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:49:52.860 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:49:52.860 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:49:52.860 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:52.860 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:49:52.861 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:49:52.861 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:49:52.861 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:49:52.861 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:52.861 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:49:52.861 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:49:52.861 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:49:52.861 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:49:52.861 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:52.861 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:49:52.861 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:49:52.861 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:49:52.861 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:49:52.861 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:49:52.861 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:49:52.862 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:49:52.862 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:49:52.862 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:49:52.862 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:49:52.862 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:49:52.980 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.58695.asok 2024-09-19T07:49:52.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:49:52.993 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.58695.asok 2024-09-19T07:49:52.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:52.993 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.58695.asok mds_sessions 2024-09-19T07:49:53.129 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:49:53.129 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 6222, 2024-09-19T07:49:53.129 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:49:53.129 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:49:53.130 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:49:53.130 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 6222 2024-09-19T07:49:53.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:53.130 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:49:53.130 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:53.130 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:49:53.130 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2034930876 2024-09-19T07:49:53.130 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:53.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:53.130 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.6222 v1:192.168.0.1:0/2034930876", 2024-09-19T07:49:53.130 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "v1:192.168.0.1:0/2034930876", 2024-09-19T07:49:53.130 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions": [ 2024-09-19T07:49:53.130 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:53.130 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": 0, 2024-09-19T07:49:53.131 INFO:teuthology.orchestra.run.smithi016.stdout: "addrs": { 2024-09-19T07:49:53.131 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-19T07:49:53.131 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:53.131 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-19T07:49:53.131 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.146:6835", 2024-09-19T07:49:53.131 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3642987189 2024-09-19T07:49:53.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:53.131 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:49:53.131 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:49:53.131 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.146:6837", 2024-09-19T07:49:53.131 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3642987189 2024-09-19T07:49:53.131 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:53.131 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T07:49:53.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:49:53.132 INFO:teuthology.orchestra.run.smithi016.stdout: "seq": 1201, 2024-09-19T07:49:53.132 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_gen": 0, 2024-09-19T07:49:53.132 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_ttl": "2024-09-19T07:50:48.836138+0000", 2024-09-19T07:49:53.132 INFO:teuthology.orchestra.run.smithi016.stdout: "last_cap_renew_request": "2024-09-19T07:49:48.836138+0000", 2024-09-19T07:49:53.132 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_renew_seq": 2, 2024-09-19T07:49:53.132 INFO:teuthology.orchestra.run.smithi016.stdout: "num_caps": 282, 2024-09-19T07:49:53.132 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "open" 2024-09-19T07:49:53.132 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:49:53.132 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:49:53.132 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 41 2024-09-19T07:49:53.132 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:49:53.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:49:53.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.239+0000 7f3092be4640 1 Processor -- start 2024-09-19T07:49:53.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.239+0000 7f3092be4640 1 -- start start 2024-09-19T07:49:53.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.240+0000 7f3092be4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f308c06f9e0 0x7f308c0d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:53.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.240+0000 7f3092be4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f308c0d0e00 con 0x7f308c15b120 2024-09-19T07:49:53.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.240+0000 7f3092be4640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f308c15b500 con 0x7f308c0d09f0 2024-09-19T07:49:53.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.240+0000 7f3092be4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f308c06e940 con 0x7f308c06f9e0 2024-09-19T07:49:53.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.240+0000 7f3083fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f308c06f9e0 0x7f308c0d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:53.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.240+0000 7f3090980640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f308c15b120 0x7f308c06dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55288/0 (socket says 172.21.15.16:55288) 2024-09-19T07:49:53.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.240+0000 7f3090980640 1 -- 172.21.15.16:0/3218600301 learned_addr learned my addr 172.21.15.16:0/3218600301 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:53.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.240+0000 7f3083fff640 1 -- 172.21.15.16:0/3218600301 >> v1:172.21.15.146:6789/0 conn(0x7f308c0d09f0 legacy=0x7f308c0c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:53.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.240+0000 7f3083fff640 1 -- 172.21.15.16:0/3218600301 >> v1:172.21.15.16:6789/0 conn(0x7f308c15b120 legacy=0x7f308c06dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:53.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.240+0000 7f30837fe640 1 -- 172.21.15.16:0/3218600301 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4124778445 0 0) 0x7f3088001110 con 0x7f308c15b120 2024-09-19T07:49:53.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.240+0000 7f3083fff640 1 -- 172.21.15.16:0/3218600301 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308c06ec20 con 0x7f308c06f9e0 2024-09-19T07:49:53.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.241+0000 7f3083fff640 1 --2- 172.21.15.16:0/3218600301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308c06f9e0 0x7f308c0d04b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f307c0097a0 tx=0x7f307c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34c808ad9e6bc3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:53.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.241+0000 7f30837fe640 1 -- 172.21.15.16:0/3218600301 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c037070 con 0x7f308c06f9e0 2024-09-19T07:49:53.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.241+0000 7f30837fe640 1 -- 172.21.15.16:0/3218600301 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f307c032440 con 0x7f308c06f9e0 2024-09-19T07:49:53.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.241+0000 7f30837fe640 1 -- 172.21.15.16:0/3218600301 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c032810 con 0x7f308c06f9e0 2024-09-19T07:49:53.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.242+0000 7f3092be4640 1 -- 172.21.15.16:0/3218600301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308c06f9e0 msgr2=0x7f308c0d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:53.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.242+0000 7f3092be4640 1 --2- 172.21.15.16:0/3218600301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308c06f9e0 0x7f308c0d04b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f307c0097a0 tx=0x7f307c02a1e0 comp rx=0 tx=0).stop 2024-09-19T07:49:53.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.242+0000 7f3092be4640 1 -- 172.21.15.16:0/3218600301 shutdown_connections 2024-09-19T07:49:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.242+0000 7f3092be4640 1 --2- 172.21.15.16:0/3218600301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308c06f9e0 0x7f308c0d04b0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.242+0000 7f3092be4640 1 -- 172.21.15.16:0/3218600301 >> 172.21.15.16:0/3218600301 conn(0x7f308c06bdf0 msgr2=0x7f308c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.242+0000 7f3092be4640 1 -- 172.21.15.16:0/3218600301 shutdown_connections 2024-09-19T07:49:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.242+0000 7f3092be4640 1 -- 172.21.15.16:0/3218600301 wait complete. 2024-09-19T07:49:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.243+0000 7f3092be4640 1 Processor -- start 2024-09-19T07:49:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.243+0000 7f3092be4640 1 -- start start 2024-09-19T07:49:53.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.244+0000 7f3092be4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308c06f9e0 0x7f308c1459f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:53.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.244+0000 7f3092be4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f308c0d09f0 0x7f308c0731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:53.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.244+0000 7f3092be4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f308c15b120 0x7f308c1462f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:53.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.244+0000 7f3092be4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f308c06f4c0 con 0x7f308c06f9e0 2024-09-19T07:49:53.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.244+0000 7f3092be4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f308c06f1e0 con 0x7f308c15b120 2024-09-19T07:49:53.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.244+0000 7f3092be4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f308c06e5d0 con 0x7f308c0d09f0 2024-09-19T07:49:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.244+0000 7f3090980640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308c06f9e0 0x7f308c1459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.244+0000 7f3090980640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308c06f9e0 0x7f308c1459f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45720/0 (socket says 172.21.15.16:45720) 2024-09-19T07:49:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.244+0000 7f3090980640 1 -- 172.21.15.16:0/213555658 learned_addr learned my addr 172.21.15.16:0/213555658 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.244+0000 7f3091181640 1 --2- 172.21.15.16:0/213555658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f308c15b120 0x7f308c1462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.244+0000 7f3083fff640 1 --2- 172.21.15.16:0/213555658 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f308c0d09f0 0x7f308c0731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.244+0000 7f3090980640 1 -- 172.21.15.16:0/213555658 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f308c0d09f0 msgr2=0x7f308c0731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.244+0000 7f3090980640 1 --2- 172.21.15.16:0/213555658 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f308c0d09f0 0x7f308c0731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.244+0000 7f3090980640 1 -- 172.21.15.16:0/213555658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f308c15b120 msgr2=0x7f308c1462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.244+0000 7f3090980640 1 --2- 172.21.15.16:0/213555658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f308c15b120 0x7f308c1462f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.244+0000 7f3090980640 1 -- 172.21.15.16:0/213555658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308c071e40 con 0x7f308c06f9e0 2024-09-19T07:49:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.245+0000 7f3090980640 1 --2- 172.21.15.16:0/213555658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308c06f9e0 0x7f308c1459f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f308800c430 tx=0x7f308800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.245+0000 7f30817fa640 1 -- 172.21.15.16:0/213555658 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30880074e0 con 0x7f308c06f9e0 2024-09-19T07:49:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.245+0000 7f30817fa640 1 -- 172.21.15.16:0/213555658 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30880047e0 con 0x7f308c06f9e0 2024-09-19T07:49:53.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.245+0000 7f30817fa640 1 -- 172.21.15.16:0/213555658 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088004bd0 con 0x7f308c06f9e0 2024-09-19T07:49:53.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.245+0000 7f3092be4640 1 -- 172.21.15.16:0/213555658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f308c072120 con 0x7f308c06f9e0 2024-09-19T07:49:53.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.245+0000 7f3092be4640 1 -- 172.21.15.16:0/213555658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f308c0726e0 con 0x7f308c06f9e0 2024-09-19T07:49:53.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.246+0000 7f3092be4640 1 -- 172.21.15.16:0/213555658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f308c0ce060 con 0x7f308c06f9e0 2024-09-19T07:49:53.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.246+0000 7f30817fa640 1 -- 172.21.15.16:0/213555658 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3088011070 con 0x7f308c06f9e0 2024-09-19T07:49:53.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.247+0000 7f30817fa640 1 --2- 172.21.15.16:0/213555658 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f307806c2b0 0x7f307806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.247+0000 7f30817fa640 1 -- 172.21.15.16:0/213555658 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f308808b260 con 0x7f308c06f9e0 2024-09-19T07:49:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.247+0000 7f3083fff640 1 --2- 172.21.15.16:0/213555658 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f307806c2b0 0x7f307806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.248+0000 7f3083fff640 1 --2- 172.21.15.16:0/213555658 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f307806c2b0 0x7f307806e770 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f307c034000 tx=0x7f307c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:53.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.252+0000 7f30817fa640 1 -- 172.21.15.16:0/213555658 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f308c0d0e00 con 0x7f308c06f9e0 2024-09-19T07:49:53.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.463+0000 7f3092be4640 1 -- 172.21.15.16:0/213555658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f308c0af170 con 0x7f308c06f9e0 2024-09-19T07:49:53.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.464+0000 7f30817fa640 1 -- 172.21.15.16:0/213555658 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5222 (secure 0 0 0) 0x7f3088059840 con 0x7f308c06f9e0 2024-09-19T07:49:53.464 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:53.464 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"btime":"2024-09-19T07:49:44:053013+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/945862061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":945862061},{"type":"v1","addr":"172.21.15.146:6836","nonce":945862061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/42753795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":42753795},{"type":"v1","addr":"172.21.15.16:6834","nonce":42753795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1389787480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1389787480},{"type":"v1","addr":"172.21.15.16:6835","nonce":1389787480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:49:40.403885+0000","modified":"2024-09-19T07:49:43.407266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6096":{"gid":6096,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/3642987189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3642987189},{"type":"v1","addr":"172.21.15.146:6837","nonce":3642987189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6096,"qdb_cluster":[6096]},"id":5}]} 2024-09-19T07:49:53.464 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 41 2024-09-19T07:49:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.465+0000 7f3092be4640 1 -- 172.21.15.16:0/213555658 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f307806c2b0 msgr2=0x7f307806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.465+0000 7f3092be4640 1 --2- 172.21.15.16:0/213555658 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f307806c2b0 0x7f307806e770 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f307c034000 tx=0x7f307c02e000 comp rx=0 tx=0).stop 2024-09-19T07:49:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.465+0000 7f3092be4640 1 -- 172.21.15.16:0/213555658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308c06f9e0 msgr2=0x7f308c1459f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.465+0000 7f3092be4640 1 --2- 172.21.15.16:0/213555658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308c06f9e0 0x7f308c1459f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f308800c430 tx=0x7f308800c8f0 comp rx=0 tx=0).stop 2024-09-19T07:49:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.466+0000 7f3092be4640 1 -- 172.21.15.16:0/213555658 shutdown_connections 2024-09-19T07:49:53.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.466+0000 7f3092be4640 1 --2- 172.21.15.16:0/213555658 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f307806c2b0 0x7f307806e770 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:53.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.466+0000 7f3092be4640 1 --2- 172.21.15.16:0/213555658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f308c15b120 0x7f308c1462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:53.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.466+0000 7f3092be4640 1 --2- 172.21.15.16:0/213555658 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f308c0d09f0 0x7f308c0731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:53.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.466+0000 7f3092be4640 1 --2- 172.21.15.16:0/213555658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308c06f9e0 0x7f308c1459f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:53.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.466+0000 7f3092be4640 1 -- 172.21.15.16:0/213555658 >> 172.21.15.16:0/213555658 conn(0x7f308c06bdf0 msgr2=0x7f308c0b3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:53.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.466+0000 7f3092be4640 1 -- 172.21.15.16:0/213555658 shutdown_connections 2024-09-19T07:49:53.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:53.466+0000 7f3092be4640 1 -- 172.21.15.16:0/213555658 wait complete. 2024-09-19T07:49:53.477 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-19T07:49:53.595 INFO:teuthology.orchestra.run.smithi146.stdout:[{"id":5932,"entity":{"name":{"type":"client","num":5932},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2223430545}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":4.2099370970000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9761359393447773,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5932 192.168.0.1:0/2223430545","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"46263","root":"/"}},{"id":6222,"entity":{"name":{"type":"client","num":6222},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2034930876}},"state":"open","num_leases":1,"num_caps":282,"request_load_avg":950,"uptime":5.7509140710000004,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":494.77584065113984,"halflife":60},"release_caps":{"value":318.48424532162619,"halflife":60},"recall_caps_throttle":{"value":548.90131985855055,"halflife":1.5},"recall_caps_throttle2o":{"value":276.40132973146956,"halflife":0.5},"session_cache_liveness":{"value":1200.5220728452209,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":968,"last_trim_completed_flushes_tid":601,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6222 192.168.0.1:0/2034930876","completed_requests":[{"tid":969,"created_ino":"0x1000000044c"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000044d","length":903}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"0","hostname":"smithi016","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"58695","root":"/"}}] 2024-09-19T07:49:53.595 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "46263", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2223430545, "type": "any" }, "name": { "num": 5932, "type": "client" } }, "id": 5932, "inst": "client.5932 192.168.0.1:0/2223430545", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9761359393447773 }, "state": "open", "uptime": 4.209937097 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "0", "hostname": "smithi016", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "58695", "root": "/" }, "completed_requests": [ { "created_ino": "0x1000000044c", "tid": 969 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2034930876, "type": "any" }, "name": { "num": 6222, "type": "client" } }, "id": 6222, "inst": "client.6222 192.168.0.1:0/2034930876", "last_trim_completed_flushes_tid": 601, "last_trim_completed_requests_tid": 968, "num_caps": 282, "num_completed_flushes": 1, "num_completed_requests": 1, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 903, "start": "0x1000000044d" } ], "recall_caps": { "halflife": 60, "value": 494.77584065113984 }, "recall_caps_throttle": { "halflife": 1.5, "value": 548.9013198585506 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 276.40132973146956 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 318.4842453216262 }, "request_load_avg": 950, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1200.522072845221 }, "state": "open", "uptime": 5.750914071 } ] 2024-09-19T07:49:53.596 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-19T07:49:58.597 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:49:58.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.712+0000 7fd3d4e9b640 1 Processor -- start 2024-09-19T07:49:58.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.712+0000 7fd3d4e9b640 1 -- start start 2024-09-19T07:49:58.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.713+0000 7fd3d4e9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3d006f9e0 0x7fd3d00d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:58.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.713+0000 7fd3d4e9b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3d00d09f0 0x7fd3d00c5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:58.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.713+0000 7fd3d4e9b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3d015b500 con 0x7fd3d015b120 2024-09-19T07:49:58.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.713+0000 7fd3d4e9b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd3d00d0e40 con 0x7fd3d006f9e0 2024-09-19T07:49:58.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.713+0000 7fd3d4e9b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd3d006e940 con 0x7fd3d00d09f0 2024-09-19T07:49:58.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.713+0000 7fd3cdd9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3d006f9e0 0x7fd3d00d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:58.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.713+0000 7fd3ce59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd3d015b120 0x7fd3d006dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55294/0 (socket says 172.21.15.16:55294) 2024-09-19T07:49:58.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.714+0000 7fd3cdd9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3d006f9e0 0x7fd3d00d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45724/0 (socket says 172.21.15.16:45724) 2024-09-19T07:49:58.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.714+0000 7fd3ce59c640 1 -- 172.21.15.16:0/712535791 learned_addr learned my addr 172.21.15.16:0/712535791 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:49:58.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.714+0000 7fd3ced9d640 1 --2- 172.21.15.16:0/712535791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3d00d09f0 0x7fd3d00c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:58.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.714+0000 7fd3cdd9b640 1 -- 172.21.15.16:0/712535791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3d00d09f0 msgr2=0x7fd3d00c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:58.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.714+0000 7fd3cdd9b640 1 --2- 172.21.15.16:0/712535791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3d00d09f0 0x7fd3d00c5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:58.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.714+0000 7fd3cdd9b640 1 -- 172.21.15.16:0/712535791 >> v1:172.21.15.16:6789/0 conn(0x7fd3d015b120 legacy=0x7fd3d006dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:58.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.714+0000 7fd3cdd9b640 1 -- 172.21.15.16:0/712535791 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d006ec20 con 0x7fd3d006f9e0 2024-09-19T07:49:58.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.715+0000 7fd3cdd9b640 1 --2- 172.21.15.16:0/712535791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d006f9e0 0x7fd3d00d04b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800ba40 tx=0x7fd3b802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7115a82bc552dcf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:58.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.715+0000 7fd3cd59a640 1 -- 172.21.15.16:0/712535791 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8032750 con 0x7fd3d006f9e0 2024-09-19T07:49:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.715+0000 7fd3cd59a640 1 -- 172.21.15.16:0/712535791 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b8030050 con 0x7fd3d006f9e0 2024-09-19T07:49:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.715+0000 7fd3cd59a640 1 -- 172.21.15.16:0/712535791 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8032b10 con 0x7fd3d006f9e0 2024-09-19T07:49:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.715+0000 7fd3d4e9b640 1 -- 172.21.15.16:0/712535791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d006f9e0 msgr2=0x7fd3d00d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.715+0000 7fd3d4e9b640 1 --2- 172.21.15.16:0/712535791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d006f9e0 0x7fd3d00d04b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800ba40 tx=0x7fd3b802c850 comp rx=0 tx=0).stop 2024-09-19T07:49:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.715+0000 7fd3d4e9b640 1 -- 172.21.15.16:0/712535791 shutdown_connections 2024-09-19T07:49:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.715+0000 7fd3d4e9b640 1 --2- 172.21.15.16:0/712535791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3d00d09f0 0x7fd3d00c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.715+0000 7fd3d4e9b640 1 --2- 172.21.15.16:0/712535791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d006f9e0 0x7fd3d00d04b0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.715+0000 7fd3d4e9b640 1 -- 172.21.15.16:0/712535791 >> 172.21.15.16:0/712535791 conn(0x7fd3d006bdf0 msgr2=0x7fd3d006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.715+0000 7fd3d4e9b640 1 -- 172.21.15.16:0/712535791 shutdown_connections 2024-09-19T07:49:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.715+0000 7fd3d4e9b640 1 -- 172.21.15.16:0/712535791 wait complete. 2024-09-19T07:49:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.717+0000 7fd3d4e9b640 1 Processor -- start 2024-09-19T07:49:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.717+0000 7fd3d4e9b640 1 -- start start 2024-09-19T07:49:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3d4e9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d006f9e0 0x7fd3d0157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3d4e9b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3d00d09f0 0x7fd3d01526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3d4e9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3d015b120 0x7fd3d00ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3d4e9b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d006f4c0 con 0x7fd3d006f9e0 2024-09-19T07:49:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3d4e9b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d006f1e0 con 0x7fd3d015b120 2024-09-19T07:49:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3d4e9b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd3d006e5d0 con 0x7fd3d00d09f0 2024-09-19T07:49:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3ce59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d006f9e0 0x7fd3d0157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3ce59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d006f9e0 0x7fd3d0157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45730/0 (socket says 172.21.15.16:45730) 2024-09-19T07:49:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3ce59c640 1 -- 172.21.15.16:0/2911071600 learned_addr learned my addr 172.21.15.16:0/2911071600 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3cdd9b640 1 --2- 172.21.15.16:0/2911071600 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3d00d09f0 0x7fd3d01526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3ced9d640 1 --2- 172.21.15.16:0/2911071600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3d015b120 0x7fd3d00ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3ce59c640 1 -- 172.21.15.16:0/2911071600 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3d00d09f0 msgr2=0x7fd3d01526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3ce59c640 1 --2- 172.21.15.16:0/2911071600 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3d00d09f0 0x7fd3d01526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3ce59c640 1 -- 172.21.15.16:0/2911071600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3d015b120 msgr2=0x7fd3d00ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3ce59c640 1 --2- 172.21.15.16:0/2911071600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3d015b120 0x7fd3d00ad5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:58.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3ce59c640 1 -- 172.21.15.16:0/2911071600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d01e3b10 con 0x7fd3d006f9e0 2024-09-19T07:49:58.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3cdd9b640 1 --2- 172.21.15.16:0/2911071600 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3d00d09f0 0x7fd3d01526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:58.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3ce59c640 1 --2- 172.21.15.16:0/2911071600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d006f9e0 0x7fd3d0157330 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000cb10 tx=0x7fd3c000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:58.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3c6ffd640 1 -- 172.21.15.16:0/2911071600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0017070 con 0x7fd3d006f9e0 2024-09-19T07:49:58.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3c6ffd640 1 -- 172.21.15.16:0/2911071600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c000ac70 con 0x7fd3d006f9e0 2024-09-19T07:49:58.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3c6ffd640 1 -- 172.21.15.16:0/2911071600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0012620 con 0x7fd3d006f9e0 2024-09-19T07:49:58.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.718+0000 7fd3d4e9b640 1 -- 172.21.15.16:0/2911071600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d01e3df0 con 0x7fd3d006f9e0 2024-09-19T07:49:58.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.719+0000 7fd3d4e9b640 1 -- 172.21.15.16:0/2911071600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d0155610 con 0x7fd3d006f9e0 2024-09-19T07:49:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.720+0000 7fd3d4e9b640 1 -- 172.21.15.16:0/2911071600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d0075710 con 0x7fd3d006f9e0 2024-09-19T07:49:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.720+0000 7fd3c6ffd640 1 -- 172.21.15.16:0/2911071600 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd3c001a070 con 0x7fd3d006f9e0 2024-09-19T07:49:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.721+0000 7fd3c6ffd640 1 --2- 172.21.15.16:0/2911071600 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd3b006c2e0 0x7fd3b006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.721+0000 7fd3c6ffd640 1 -- 172.21.15.16:0/2911071600 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7fd3c008b340 con 0x7fd3d006f9e0 2024-09-19T07:49:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.721+0000 7fd3cdd9b640 1 --2- 172.21.15.16:0/2911071600 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd3b006c2e0 0x7fd3b006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.722+0000 7fd3cdd9b640 1 --2- 172.21.15.16:0/2911071600 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd3b006c2e0 0x7fd3b006e7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8037000 tx=0x7fd3b8036040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:58.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.725+0000 7fd3c6ffd640 1 -- 172.21.15.16:0/2911071600 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd3d0075710 con 0x7fd3d006f9e0 2024-09-19T07:49:58.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.938+0000 7fd3d4e9b640 1 -- 172.21.15.16:0/2911071600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3d00ce060 con 0x7fd3d006f9e0 2024-09-19T07:49:58.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.938+0000 7fd3c6ffd640 1 -- 172.21.15.16:0/2911071600 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5222 (secure 0 0 0) 0x7fd3c0059920 con 0x7fd3d006f9e0 2024-09-19T07:49:58.939 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:49:58.939 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"btime":"2024-09-19T07:49:44:053013+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/945862061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":945862061},{"type":"v1","addr":"172.21.15.146:6836","nonce":945862061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/42753795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":42753795},{"type":"v1","addr":"172.21.15.16:6834","nonce":42753795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1389787480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1389787480},{"type":"v1","addr":"172.21.15.16:6835","nonce":1389787480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:49:40.403885+0000","modified":"2024-09-19T07:49:43.407266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6096":{"gid":6096,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/3642987189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3642987189},{"type":"v1","addr":"172.21.15.146:6837","nonce":3642987189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6096,"qdb_cluster":[6096]},"id":5}]} 2024-09-19T07:49:58.939 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 41 2024-09-19T07:49:58.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.940+0000 7fd3d4e9b640 1 -- 172.21.15.16:0/2911071600 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd3b006c2e0 msgr2=0x7fd3b006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:58.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.940+0000 7fd3d4e9b640 1 --2- 172.21.15.16:0/2911071600 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd3b006c2e0 0x7fd3b006e7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8037000 tx=0x7fd3b8036040 comp rx=0 tx=0).stop 2024-09-19T07:49:58.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.940+0000 7fd3d4e9b640 1 -- 172.21.15.16:0/2911071600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d006f9e0 msgr2=0x7fd3d0157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:58.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.940+0000 7fd3d4e9b640 1 --2- 172.21.15.16:0/2911071600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d006f9e0 0x7fd3d0157330 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000cb10 tx=0x7fd3c000a060 comp rx=0 tx=0).stop 2024-09-19T07:49:58.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.940+0000 7fd3d4e9b640 1 -- 172.21.15.16:0/2911071600 shutdown_connections 2024-09-19T07:49:58.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.940+0000 7fd3d4e9b640 1 --2- 172.21.15.16:0/2911071600 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd3b006c2e0 0x7fd3b006e7a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:58.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.940+0000 7fd3d4e9b640 1 --2- 172.21.15.16:0/2911071600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3d015b120 0x7fd3d00ad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:58.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.940+0000 7fd3d4e9b640 1 --2- 172.21.15.16:0/2911071600 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3d00d09f0 0x7fd3d01526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:58.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.940+0000 7fd3d4e9b640 1 --2- 172.21.15.16:0/2911071600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d006f9e0 0x7fd3d0157330 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:58.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.940+0000 7fd3d4e9b640 1 -- 172.21.15.16:0/2911071600 >> 172.21.15.16:0/2911071600 conn(0x7fd3d006bdf0 msgr2=0x7fd3d00ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:58.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.940+0000 7fd3d4e9b640 1 -- 172.21.15.16:0/2911071600 shutdown_connections 2024-09-19T07:49:58.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:58.940+0000 7fd3d4e9b640 1 -- 172.21.15.16:0/2911071600 wait complete. 2024-09-19T07:49:58.952 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-19T07:49:59.082 INFO:teuthology.orchestra.run.smithi146.stdout:[{"id":5932,"entity":{"name":{"type":"client","num":5932},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2223430545}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":9.6958551269999997,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9386509997287407,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5932 192.168.0.1:0/2223430545","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"46263","root":"/"}},{"id":6222,"entity":{"name":{"type":"client","num":6222},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2034930876}},"state":"open","num_leases":1,"num_caps":100,"request_load_avg":892,"uptime":11.236832100999999,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":464.28283537129306,"halflife":60},"release_caps":{"value":470.77711745877281,"halflife":60},"recall_caps_throttle":{"value":61.380121813959668,"halflife":1.5},"recall_caps_throttle2o":{"value":0.3100356669307972,"halflife":0.5},"session_cache_liveness":{"value":1283.1315770576916,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":968,"last_trim_completed_flushes_tid":601,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6222 192.168.0.1:0/2034930876","completed_requests":[{"tid":969,"created_ino":"0x1000000044c"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000044d","length":903}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"0","hostname":"smithi016","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"58695","root":"/"}}] 2024-09-19T07:49:59.082 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "46263", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2223430545, "type": "any" }, "name": { "num": 5932, "type": "client" } }, "id": 5932, "inst": "client.5932 192.168.0.1:0/2223430545", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9386509997287407 }, "state": "open", "uptime": 9.695855127 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "0", "hostname": "smithi016", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "58695", "root": "/" }, "completed_requests": [ { "created_ino": "0x1000000044c", "tid": 969 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2034930876, "type": "any" }, "name": { "num": 6222, "type": "client" } }, "id": 6222, "inst": "client.6222 192.168.0.1:0/2034930876", "last_trim_completed_flushes_tid": 601, "last_trim_completed_requests_tid": 968, "num_caps": 100, "num_completed_flushes": 1, "num_completed_requests": 1, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 903, "start": "0x1000000044d" } ], "recall_caps": { "halflife": 60, "value": 464.28283537129306 }, "recall_caps_throttle": { "halflife": 1.5, "value": 61.38012181395967 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0.3100356669307972 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 470.7771174587728 }, "request_load_avg": 892, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1283.1315770576916 }, "state": "open", "uptime": 11.236832101 } ] 2024-09-19T07:49:59.082 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-19T07:49:59.083 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:49:59.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:49:59.123 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:49:59.158 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:49:59.158 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:49:59.158 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:49:59.158 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:49:59.180 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:49:59.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.323+0000 7f23db677640 1 Processor -- start 2024-09-19T07:49:59.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.323+0000 7f23db677640 1 -- start start 2024-09-19T07:49:59.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.324+0000 7f23db677640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f23d406cd20 0x7f23d406d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:59.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.324+0000 7f23db677640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23d4161eb0 0x7f23d41622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:59.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.324+0000 7f23db677640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f23d4143a20 0x7f23d4143ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:59.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.324+0000 7f23db677640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f23d4162e50 con 0x7f23d4143a20 2024-09-19T07:49:59.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.324+0000 7f23db677640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f23d4142150 con 0x7f23d406cd20 2024-09-19T07:49:59.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.324+0000 7f23db677640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f23d406d640 con 0x7f23d4161eb0 2024-09-19T07:49:59.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.324+0000 7f23d9c14640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f23d4143a20 0x7f23d4143ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.324+0000 7f23d9c14640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f23d4143a20 0x7f23d4143ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45738/0 (socket says 172.21.15.16:45738) 2024-09-19T07:49:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.324+0000 7f23d9c14640 1 -- 172.21.15.16:0/3617343369 learned_addr learned my addr 172.21.15.16:0/3617343369 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.324+0000 7f23d8c12640 1 --2- 172.21.15.16:0/3617343369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23d4161eb0 0x7f23d41622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.324+0000 7f23d9413640 1 --2- 172.21.15.16:0/3617343369 >> v2:172.21.15.146:3300/0 conn(0x7f23d406cd20 0x7f23d406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.325+0000 7f23d9c14640 1 -- 172.21.15.16:0/3617343369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23d4161eb0 msgr2=0x7f23d41622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.325+0000 7f23d9c14640 1 --2- 172.21.15.16:0/3617343369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23d4161eb0 0x7f23d41622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.325+0000 7f23d9c14640 1 -- 172.21.15.16:0/3617343369 >> v2:172.21.15.146:3300/0 conn(0x7f23d406cd20 msgr2=0x7f23d406d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.325+0000 7f23d9c14640 1 --2- 172.21.15.16:0/3617343369 >> v2:172.21.15.146:3300/0 conn(0x7f23d406cd20 0x7f23d406d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.325+0000 7f23d9c14640 1 -- 172.21.15.16:0/3617343369 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d4162b70 con 0x7f23d4143a20 2024-09-19T07:49:59.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.325+0000 7f23d9c14640 1 --2- 172.21.15.16:0/3617343369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23d4143a20 0x7f23d4143ed0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f23d4070ff0 tx=0x7f23cc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4bdb8f294b0768 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.325+0000 7f23d3fff640 1 -- 172.21.15.16:0/3617343369 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc037070 con 0x7f23d4143a20 2024-09-19T07:49:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.325+0000 7f23d3fff640 1 -- 172.21.15.16:0/3617343369 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23cc032440 con 0x7f23d4143a20 2024-09-19T07:49:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.325+0000 7f23d3fff640 1 -- 172.21.15.16:0/3617343369 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc032810 con 0x7f23d4143a20 2024-09-19T07:49:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.326+0000 7f23db677640 1 -- 172.21.15.16:0/3617343369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23d4143a20 msgr2=0x7f23d4143ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.326+0000 7f23db677640 1 --2- 172.21.15.16:0/3617343369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23d4143a20 0x7f23d4143ed0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f23d4070ff0 tx=0x7f23cc02a140 comp rx=0 tx=0).stop 2024-09-19T07:49:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.326+0000 7f23db677640 1 -- 172.21.15.16:0/3617343369 shutdown_connections 2024-09-19T07:49:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.326+0000 7f23db677640 1 --2- 172.21.15.16:0/3617343369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23d4143a20 0x7f23d4143ed0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.326+0000 7f23db677640 1 --2- 172.21.15.16:0/3617343369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23d4161eb0 0x7f23d41622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.326+0000 7f23db677640 1 --2- 172.21.15.16:0/3617343369 >> v2:172.21.15.146:3300/0 conn(0x7f23d406cd20 0x7f23d406d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.326+0000 7f23db677640 1 -- 172.21.15.16:0/3617343369 >> 172.21.15.16:0/3617343369 conn(0x7f23d40b3ed0 msgr2=0x7f23d40b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.326+0000 7f23db677640 1 -- 172.21.15.16:0/3617343369 shutdown_connections 2024-09-19T07:49:59.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.326+0000 7f23db677640 1 -- 172.21.15.16:0/3617343369 wait complete. 2024-09-19T07:49:59.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.327+0000 7f23db677640 1 Processor -- start 2024-09-19T07:49:59.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.327+0000 7f23db677640 1 -- start start 2024-09-19T07:49:59.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.328+0000 7f23db677640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23d406cd20 0x7f23d4158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:59.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.328+0000 7f23db677640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23d4143a20 0x7f23d40c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:59.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.328+0000 7f23db677640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23d4161eb0 0x7f23d4140a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:59.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.328+0000 7f23db677640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23d4162830 con 0x7f23d406cd20 2024-09-19T07:49:59.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.328+0000 7f23db677640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f23d4141de0 con 0x7f23d4143a20 2024-09-19T07:49:59.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.328+0000 7f23db677640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f23d4141600 con 0x7f23d4161eb0 2024-09-19T07:49:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.328+0000 7f23d9413640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23d406cd20 0x7f23d4158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.328+0000 7f23d9413640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23d406cd20 0x7f23d4158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45746/0 (socket says 172.21.15.16:45746) 2024-09-19T07:49:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.328+0000 7f23d9413640 1 -- 172.21.15.16:0/3294400375 learned_addr learned my addr 172.21.15.16:0/3294400375 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.328+0000 7f23d9c14640 1 --2- 172.21.15.16:0/3294400375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23d4161eb0 0x7f23d4140a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.328+0000 7f23d8c12640 1 --2- 172.21.15.16:0/3294400375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23d4143a20 0x7f23d40c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.328+0000 7f23d9413640 1 -- 172.21.15.16:0/3294400375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23d4161eb0 msgr2=0x7f23d4140a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.329+0000 7f23d9413640 1 --2- 172.21.15.16:0/3294400375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23d4161eb0 0x7f23d4140a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.329+0000 7f23d9413640 1 -- 172.21.15.16:0/3294400375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23d4143a20 msgr2=0x7f23d40c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.329+0000 7f23d9413640 1 --2- 172.21.15.16:0/3294400375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23d4143a20 0x7f23d40c95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.329+0000 7f23d9413640 1 -- 172.21.15.16:0/3294400375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d40ca320 con 0x7f23d406cd20 2024-09-19T07:49:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.329+0000 7f23d8c12640 1 --2- 172.21.15.16:0/3294400375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23d4143a20 0x7f23d40c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.329+0000 7f23d9413640 1 --2- 172.21.15.16:0/3294400375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23d406cd20 0x7f23d4158f80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f23c4007e40 tx=0x7f23c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.329+0000 7f23d1ffb640 1 -- 172.21.15.16:0/3294400375 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c4014620 con 0x7f23d406cd20 2024-09-19T07:49:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.329+0000 7f23d1ffb640 1 -- 172.21.15.16:0/3294400375 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c4019070 con 0x7f23d406cd20 2024-09-19T07:49:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.329+0000 7f23db677640 1 -- 172.21.15.16:0/3294400375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d40ca600 con 0x7f23d406cd20 2024-09-19T07:49:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.329+0000 7f23d1ffb640 1 -- 172.21.15.16:0/3294400375 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c4014a00 con 0x7f23d406cd20 2024-09-19T07:49:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.329+0000 7f23db677640 1 -- 172.21.15.16:0/3294400375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d40c9ae0 con 0x7f23d406cd20 2024-09-19T07:49:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.330+0000 7f23db677640 1 -- 172.21.15.16:0/3294400375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d406d100 con 0x7f23d406cd20 2024-09-19T07:49:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.330+0000 7f23d1ffb640 1 -- 172.21.15.16:0/3294400375 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f23c4014ba0 con 0x7f23d406cd20 2024-09-19T07:49:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.331+0000 7f23d1ffb640 1 --2- 172.21.15.16:0/3294400375 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f23c006c2b0 0x7f23c006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.331+0000 7f23d1ffb640 1 -- 172.21.15.16:0/3294400375 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f23c408bd80 con 0x7f23d406cd20 2024-09-19T07:49:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.331+0000 7f23d8c12640 1 --2- 172.21.15.16:0/3294400375 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f23c006c2b0 0x7f23c006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.332+0000 7f23d8c12640 1 --2- 172.21.15.16:0/3294400375 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f23c006c2b0 0x7f23c006e770 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f23d40cfcc0 tx=0x7f23c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.335+0000 7f23d1ffb640 1 -- 172.21.15.16:0/3294400375 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f23c405a360 con 0x7f23d406cd20 2024-09-19T07:49:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.525+0000 7f23db677640 1 -- 172.21.15.16:0/3294400375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f23d40690c0 con 0x7f23d406cd20 2024-09-19T07:49:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.526+0000 7f23d1ffb640 1 -- 172.21.15.16:0/3294400375 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v81) ==== 80+0+0 (secure 0 0 0) 0x7f23c4059c60 con 0x7f23d406cd20 2024-09-19T07:49:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:49:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.527+0000 7f23db677640 1 -- 172.21.15.16:0/3294400375 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f23c006c2b0 msgr2=0x7f23c006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.527+0000 7f23db677640 1 --2- 172.21.15.16:0/3294400375 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f23c006c2b0 0x7f23c006e770 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f23d40cfcc0 tx=0x7f23c8008040 comp rx=0 tx=0).stop 2024-09-19T07:49:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.527+0000 7f23db677640 1 -- 172.21.15.16:0/3294400375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23d406cd20 msgr2=0x7f23d4158f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.527+0000 7f23db677640 1 --2- 172.21.15.16:0/3294400375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23d406cd20 0x7f23d4158f80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f23c4007e40 tx=0x7f23c400e4b0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.528+0000 7f23db677640 1 -- 172.21.15.16:0/3294400375 shutdown_connections 2024-09-19T07:49:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.528+0000 7f23db677640 1 --2- 172.21.15.16:0/3294400375 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f23c006c2b0 0x7f23c006e770 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.528+0000 7f23db677640 1 --2- 172.21.15.16:0/3294400375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23d4161eb0 0x7f23d4140a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.528+0000 7f23db677640 1 --2- 172.21.15.16:0/3294400375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23d4143a20 0x7f23d40c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.528+0000 7f23db677640 1 --2- 172.21.15.16:0/3294400375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23d406cd20 0x7f23d4158f80 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.528+0000 7f23db677640 1 -- 172.21.15.16:0/3294400375 >> 172.21.15.16:0/3294400375 conn(0x7f23d40b3ed0 msgr2=0x7f23d4161250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.528+0000 7f23db677640 1 -- 172.21.15.16:0/3294400375 shutdown_connections 2024-09-19T07:49:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.528+0000 7f23db677640 1 -- 172.21.15.16:0/3294400375 wait complete. 2024-09-19T07:49:59.540 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-19T07:49:59.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:59.540 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T07:49:59.565 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[58695]: fuse finished with error 0 and tester_r 0 2024-09-19T07:49:59.567 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:49:59.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:59.567 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:49:59.638 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:49:59.638 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:49:59.638 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:49:59.655 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:49:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.781+0000 7fda05623640 1 Processor -- start 2024-09-19T07:49:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.781+0000 7fda05623640 1 -- start start 2024-09-19T07:49:59.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.782+0000 7fda05623640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda000d2750 0x7fda0013f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:59.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.782+0000 7fda05623640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fda000d04d0 0x7fda00160010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:59.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.782+0000 7fda05623640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda0013fda0 con 0x7fda00072250 2024-09-19T07:49:59.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.782+0000 7fda05623640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fda00145bd0 con 0x7fda000d04d0 2024-09-19T07:49:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.782+0000 7fda05623640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fda001427e0 con 0x7fda000d2750 2024-09-19T07:49:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.782+0000 7fd9ff7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fda000d04d0 0x7fda00160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.782+0000 7fd9ff7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fda000d04d0 0x7fda00160010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45760/0 (socket says 172.21.15.16:45760) 2024-09-19T07:49:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.782+0000 7fd9ff7fe640 1 -- 172.21.15.16:0/2593780084 learned_addr learned my addr 172.21.15.16:0/2593780084 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.783+0000 7fd9feffd640 1 --2- 172.21.15.16:0/2593780084 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda000d2750 0x7fda0013f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.783+0000 7fd9ff7fe640 1 -- 172.21.15.16:0/2593780084 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda000d2750 msgr2=0x7fda0013f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.783+0000 7fd9ff7fe640 1 --2- 172.21.15.16:0/2593780084 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda000d2750 0x7fda0013f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.783+0000 7fd9ff7fe640 1 -- 172.21.15.16:0/2593780084 >> v1:172.21.15.146:6789/0 conn(0x7fda00072250 legacy=0x7fda000cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.783+0000 7fd9ff7fe640 1 -- 172.21.15.16:0/2593780084 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda000d2b30 con 0x7fda000d04d0 2024-09-19T07:49:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.784+0000 7fd9ff7fe640 1 --2- 172.21.15.16:0/2593780084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda000d04d0 0x7fda00160010 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009e50 tx=0x7fd9f002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd70916e9d23591 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.784+0000 7fd9fdffb640 1 -- 172.21.15.16:0/2593780084 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0036070 con 0x7fda000d04d0 2024-09-19T07:49:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.784+0000 7fd9fdffb640 1 -- 172.21.15.16:0/2593780084 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f0031600 con 0x7fda000d04d0 2024-09-19T07:49:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.784+0000 7fd9fdffb640 1 -- 172.21.15.16:0/2593780084 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f00319d0 con 0x7fda000d04d0 2024-09-19T07:49:59.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.784+0000 7fda05623640 1 -- 172.21.15.16:0/2593780084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda000d04d0 msgr2=0x7fda00160010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:59.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.784+0000 7fda05623640 1 --2- 172.21.15.16:0/2593780084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda000d04d0 0x7fda00160010 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009e50 tx=0x7fd9f002a680 comp rx=0 tx=0).stop 2024-09-19T07:49:59.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.784+0000 7fda05623640 1 -- 172.21.15.16:0/2593780084 shutdown_connections 2024-09-19T07:49:59.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.784+0000 7fda05623640 1 --2- 172.21.15.16:0/2593780084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda000d04d0 0x7fda00160010 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.784+0000 7fda05623640 1 --2- 172.21.15.16:0/2593780084 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda000d2750 0x7fda0013f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.784+0000 7fda05623640 1 -- 172.21.15.16:0/2593780084 >> 172.21.15.16:0/2593780084 conn(0x7fda000b4040 msgr2=0x7fda000b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:59.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.784+0000 7fda05623640 1 -- 172.21.15.16:0/2593780084 shutdown_connections 2024-09-19T07:49:59.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.784+0000 7fda05623640 1 -- 172.21.15.16:0/2593780084 wait complete. 2024-09-19T07:49:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.786+0000 7fda05623640 1 Processor -- start 2024-09-19T07:49:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.786+0000 7fda05623640 1 -- start start 2024-09-19T07:49:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.786+0000 7fda05623640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda00072250 0x7fda000c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.786+0000 7fda05623640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda000d04d0 0x7fda0015b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fda05623640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda000d2750 0x7fda0015ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fda05623640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda00142220 con 0x7fda00072250 2024-09-19T07:49:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fda05623640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fda00141f40 con 0x7fda000d04d0 2024-09-19T07:49:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fda05623640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fda00142500 con 0x7fda000d2750 2024-09-19T07:49:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fd9feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda00072250 0x7fda000c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fd9feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda00072250 0x7fda000c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45772/0 (socket says 172.21.15.16:45772) 2024-09-19T07:49:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fd9feffd640 1 -- 172.21.15.16:0/1063875949 learned_addr learned my addr 172.21.15.16:0/1063875949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:49:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fd9fe7fc640 1 --2- 172.21.15.16:0/1063875949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda000d04d0 0x7fda0015b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fd9ff7fe640 1 --2- 172.21.15.16:0/1063875949 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda000d2750 0x7fda0015ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fd9feffd640 1 -- 172.21.15.16:0/1063875949 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda000d2750 msgr2=0x7fda0015ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fd9feffd640 1 --2- 172.21.15.16:0/1063875949 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda000d2750 0x7fda0015ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fd9feffd640 1 -- 172.21.15.16:0/1063875949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda000d04d0 msgr2=0x7fda0015b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fd9feffd640 1 --2- 172.21.15.16:0/1063875949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda000d04d0 0x7fda0015b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fd9feffd640 1 -- 172.21.15.16:0/1063875949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda000b5300 con 0x7fda00072250 2024-09-19T07:49:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fd9feffd640 1 --2- 172.21.15.16:0/1063875949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda00072250 0x7fda000c9cd0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800d5f0 tx=0x7fd9f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fd9e77fe640 1 -- 172.21.15.16:0/1063875949 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80125f0 con 0x7fda00072250 2024-09-19T07:49:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fd9e77fe640 1 -- 172.21.15.16:0/1063875949 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f8019070 con 0x7fda00072250 2024-09-19T07:49:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fda05623640 1 -- 172.21.15.16:0/1063875949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda000b55e0 con 0x7fda00072250 2024-09-19T07:49:59.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fd9e77fe640 1 -- 172.21.15.16:0/1063875949 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80129d0 con 0x7fda00072250 2024-09-19T07:49:59.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.787+0000 7fda05623640 1 -- 172.21.15.16:0/1063875949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda000b5ba0 con 0x7fda00072250 2024-09-19T07:49:59.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.788+0000 7fda05623640 1 -- 172.21.15.16:0/1063875949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda001418b0 con 0x7fda00072250 2024-09-19T07:49:59.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.789+0000 7fd9e77fe640 1 -- 172.21.15.16:0/1063875949 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd9f8012b70 con 0x7fda00072250 2024-09-19T07:49:59.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.789+0000 7fd9e77fe640 1 --2- 172.21.15.16:0/1063875949 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd9ec06c2b0 0x7fd9ec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:59.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.790+0000 7fd9e77fe640 1 -- 172.21.15.16:0/1063875949 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7fd9f808b860 con 0x7fda00072250 2024-09-19T07:49:59.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.790+0000 7fd9fe7fc640 1 --2- 172.21.15.16:0/1063875949 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd9ec06c2b0 0x7fd9ec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.790+0000 7fd9fe7fc640 1 --2- 172.21.15.16:0/1063875949 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd9ec06c2b0 0x7fd9ec06e770 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fda00070f80 tx=0x7fd9f4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.794+0000 7fd9e77fe640 1 -- 172.21.15.16:0/1063875949 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd9f80597e0 con 0x7fda00072250 2024-09-19T07:49:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.980+0000 7fda05623640 1 -- 172.21.15.16:0/1063875949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fda00070bf0 con 0x7fda00072250 2024-09-19T07:49:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.981+0000 7fd9e77fe640 1 -- 172.21.15.16:0/1063875949 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v81) ==== 80+0+0 (secure 0 0 0) 0x7fd9f805dae0 con 0x7fda00072250 2024-09-19T07:49:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:49:59.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.982+0000 7fda05623640 1 -- 172.21.15.16:0/1063875949 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd9ec06c2b0 msgr2=0x7fd9ec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:59.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.982+0000 7fda05623640 1 --2- 172.21.15.16:0/1063875949 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd9ec06c2b0 0x7fd9ec06e770 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fda00070f80 tx=0x7fd9f4009040 comp rx=0 tx=0).stop 2024-09-19T07:49:59.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.982+0000 7fda05623640 1 -- 172.21.15.16:0/1063875949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda00072250 msgr2=0x7fda000c9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:59.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.982+0000 7fda05623640 1 --2- 172.21.15.16:0/1063875949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda00072250 0x7fda000c9cd0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800d5f0 tx=0x7fd9f800dab0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.982+0000 7fda05623640 1 -- 172.21.15.16:0/1063875949 shutdown_connections 2024-09-19T07:49:59.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.982+0000 7fda05623640 1 --2- 172.21.15.16:0/1063875949 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd9ec06c2b0 0x7fd9ec06e770 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.983+0000 7fda05623640 1 --2- 172.21.15.16:0/1063875949 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda000d2750 0x7fda0015ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.983+0000 7fda05623640 1 --2- 172.21.15.16:0/1063875949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda000d04d0 0x7fda0015b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.983+0000 7fda05623640 1 --2- 172.21.15.16:0/1063875949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda00072250 0x7fda000c9cd0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.983+0000 7fda05623640 1 -- 172.21.15.16:0/1063875949 >> 172.21.15.16:0/1063875949 conn(0x7fda000b4040 msgr2=0x7fda00140c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.983+0000 7fda05623640 1 -- 172.21.15.16:0/1063875949 shutdown_connections 2024-09-19T07:49:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:49:59.983+0000 7fda05623640 1 -- 172.21.15.16:0/1063875949 wait complete. 2024-09-19T07:49:59.994 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2024-09-19T07:49:59.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:49:59.994 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T07:50:00.020 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[46263]: fuse finished with error 0 and tester_r 0 2024-09-19T07:50:00.022 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:50:00.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:00.022 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T07:50:00.177 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:50:00.177 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:50:00.178 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:50:00.194 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:50:00.194 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:50:00.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:00.195 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T07:50:00.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:50:00.251 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T07:50:00.251 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:50:00.251 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:50:00.251 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:50:00.268 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:50:00.268 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:50:00.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:00.268 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T07:50:00.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:50:00.324 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T07:50:00.324 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:50:00.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.428+0000 7f2bb9311640 1 Processor -- start 2024-09-19T07:50:00.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.428+0000 7f2bb9311640 1 -- start start 2024-09-19T07:50:00.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.429+0000 7f2bb9311640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bb415bee0 0x7f2bb40d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.429+0000 7f2bb9311640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2bb40cf910 0x7f2bb40d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.429+0000 7f2bb9311640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bb40cdee0 con 0x7f2bb415c2c0 2024-09-19T07:50:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.429+0000 7f2bb9311640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2bb40ceaf0 con 0x7f2bb40cf910 2024-09-19T07:50:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.429+0000 7f2bb9311640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2bb40ce1c0 con 0x7f2bb415bee0 2024-09-19T07:50:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.429+0000 7f2bb2ffd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bb415bee0 0x7f2bb40d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.429+0000 7f2bb37fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2bb40cf910 0x7f2bb40d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.429+0000 7f2bb2ffd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bb415bee0 0x7f2bb40d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:33810/0 (socket says 172.21.15.16:33810) 2024-09-19T07:50:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.429+0000 7f2bb2ffd640 1 -- 172.21.15.16:0/1161987492 learned_addr learned my addr 172.21.15.16:0/1161987492 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.429+0000 7f2bb2ffd640 1 -- 172.21.15.16:0/1161987492 >> v2:172.21.15.146:3300/0 conn(0x7f2bb40cf910 msgr2=0x7f2bb40d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.429+0000 7f2bb2ffd640 1 --2- 172.21.15.16:0/1161987492 >> v2:172.21.15.146:3300/0 conn(0x7f2bb40cf910 0x7f2bb40d09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.429+0000 7f2bb2ffd640 1 -- 172.21.15.16:0/1161987492 >> v1:172.21.15.146:6789/0 conn(0x7f2bb415c2c0 legacy=0x7f2bb40cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.429+0000 7f2bb2ffd640 1 -- 172.21.15.16:0/1161987492 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb40ce4a0 con 0x7f2bb415bee0 2024-09-19T07:50:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.430+0000 7f2bb2ffd640 1 --2- 172.21.15.16:0/1161987492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bb415bee0 0x7f2bb40d04b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009870 tx=0x7f2ba402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80682a48ebc1a1b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.430+0000 7f2bb27fc640 1 -- 172.21.15.16:0/1161987492 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4037070 con 0x7f2bb415bee0 2024-09-19T07:50:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.430+0000 7f2bb27fc640 1 -- 172.21.15.16:0/1161987492 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ba4032440 con 0x7f2bb415bee0 2024-09-19T07:50:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.430+0000 7f2bb27fc640 1 -- 172.21.15.16:0/1161987492 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4032810 con 0x7f2bb415bee0 2024-09-19T07:50:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.431+0000 7f2bb9311640 1 -- 172.21.15.16:0/1161987492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bb415bee0 msgr2=0x7f2bb40d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.431+0000 7f2bb9311640 1 --2- 172.21.15.16:0/1161987492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bb415bee0 0x7f2bb40d04b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009870 tx=0x7f2ba402a270 comp rx=0 tx=0).stop 2024-09-19T07:50:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.431+0000 7f2bb9311640 1 -- 172.21.15.16:0/1161987492 shutdown_connections 2024-09-19T07:50:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.431+0000 7f2bb9311640 1 --2- 172.21.15.16:0/1161987492 >> v2:172.21.15.146:3300/0 conn(0x7f2bb40cf910 0x7f2bb40d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.431+0000 7f2bb9311640 1 --2- 172.21.15.16:0/1161987492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bb415bee0 0x7f2bb40d04b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.431+0000 7f2bb9311640 1 -- 172.21.15.16:0/1161987492 >> 172.21.15.16:0/1161987492 conn(0x7f2bb406bdf0 msgr2=0x7f2bb406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.431+0000 7f2bb9311640 1 -- 172.21.15.16:0/1161987492 shutdown_connections 2024-09-19T07:50:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.431+0000 7f2bb9311640 1 -- 172.21.15.16:0/1161987492 wait complete. 2024-09-19T07:50:00.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.432+0000 7f2bb9311640 1 Processor -- start 2024-09-19T07:50:00.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.432+0000 7f2bb9311640 1 -- start start 2024-09-19T07:50:00.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.433+0000 7f2bb9311640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bb40cf910 0x7f2bb4158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:00.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.433+0000 7f2bb9311640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb415bee0 0x7f2bb40cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.433+0000 7f2bb9311640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bb415c2c0 0x7f2bb40cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.433+0000 7f2bb9311640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb406e040 con 0x7f2bb415bee0 2024-09-19T07:50:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.433+0000 7f2bb9311640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb406dec0 con 0x7f2bb40cf910 2024-09-19T07:50:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.433+0000 7f2bb9311640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2bb406e1c0 con 0x7f2bb415c2c0 2024-09-19T07:50:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.433+0000 7f2baa7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb415bee0 0x7f2bb40cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.433+0000 7f2bb2ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bb40cf910 0x7f2bb4158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.433+0000 7f2baa7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb415bee0 0x7f2bb40cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45780/0 (socket says 172.21.15.16:45780) 2024-09-19T07:50:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.433+0000 7f2bb2ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bb40cf910 0x7f2bb4158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:34598/0 (socket says 172.21.15.16:34598) 2024-09-19T07:50:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.433+0000 7f2baa7fc640 1 -- 172.21.15.16:0/1601221065 learned_addr learned my addr 172.21.15.16:0/1601221065 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.433+0000 7f2bb37fe640 1 --2- 172.21.15.16:0/1601221065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bb415c2c0 0x7f2bb40cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.434+0000 7f2baa7fc640 1 -- 172.21.15.16:0/1601221065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bb415c2c0 msgr2=0x7f2bb40cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.434+0000 7f2baa7fc640 1 --2- 172.21.15.16:0/1601221065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bb415c2c0 0x7f2bb40cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.434+0000 7f2baa7fc640 1 -- 172.21.15.16:0/1601221065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bb40cf910 msgr2=0x7f2bb4158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.434+0000 7f2baa7fc640 1 --2- 172.21.15.16:0/1601221065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bb40cf910 0x7f2bb4158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.434+0000 7f2baa7fc640 1 -- 172.21.15.16:0/1601221065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb4159310 con 0x7f2bb415bee0 2024-09-19T07:50:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.434+0000 7f2bb2ffd640 1 --2- 172.21.15.16:0/1601221065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bb40cf910 0x7f2bb4158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.434+0000 7f2baa7fc640 1 --2- 172.21.15.16:0/1601221065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb415bee0 0x7f2bb40cc910 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00c440 tx=0x7f2b9c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.434+0000 7f2babfff640 1 -- 172.21.15.16:0/1601221065 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c0052d0 con 0x7f2bb415bee0 2024-09-19T07:50:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.434+0000 7f2babfff640 1 -- 172.21.15.16:0/1601221065 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b9c005470 con 0x7f2bb415bee0 2024-09-19T07:50:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.434+0000 7f2babfff640 1 -- 172.21.15.16:0/1601221065 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c007260 con 0x7f2bb415bee0 2024-09-19T07:50:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.434+0000 7f2bb9311640 1 -- 172.21.15.16:0/1601221065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb40c74e0 con 0x7f2bb415bee0 2024-09-19T07:50:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.434+0000 7f2bb9311640 1 -- 172.21.15.16:0/1601221065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb40cd880 con 0x7f2bb415bee0 2024-09-19T07:50:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.435+0000 7f2bb9311640 1 -- 172.21.15.16:0/1601221065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb40cf200 con 0x7f2bb415bee0 2024-09-19T07:50:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.436+0000 7f2babfff640 1 -- 172.21.15.16:0/1601221065 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2b9c007400 con 0x7f2bb415bee0 2024-09-19T07:50:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.436+0000 7f2babfff640 1 --2- 172.21.15.16:0/1601221065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2b9406c310 0x7f2b9406e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.436+0000 7f2babfff640 1 -- 172.21.15.16:0/1601221065 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f2b9c08a970 con 0x7f2bb415bee0 2024-09-19T07:50:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.436+0000 7f2bb2ffd640 1 --2- 172.21.15.16:0/1601221065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2b9406c310 0x7f2b9406e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.437+0000 7f2bb2ffd640 1 --2- 172.21.15.16:0/1601221065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2b9406c310 0x7f2b9406e7d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4034000 tx=0x7f2ba402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.440+0000 7f2babfff640 1 -- 172.21.15.16:0/1601221065 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2b9c058990 con 0x7f2bb415bee0 2024-09-19T07:50:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.653+0000 7f2bb9311640 1 -- 172.21.15.16:0/1601221065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bb40b4b30 con 0x7f2bb415bee0 2024-09-19T07:50:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.654+0000 7f2babfff640 1 -- 172.21.15.16:0/1601221065 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5222 (secure 0 0 0) 0x7f2b9c05de20 con 0x7f2bb415bee0 2024-09-19T07:50:00.654 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:00.654 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"btime":"2024-09-19T07:49:44:053013+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/945862061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":945862061},{"type":"v1","addr":"172.21.15.146:6836","nonce":945862061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/42753795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":42753795},{"type":"v1","addr":"172.21.15.16:6834","nonce":42753795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1389787480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1389787480},{"type":"v1","addr":"172.21.15.16:6835","nonce":1389787480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:49:40.403885+0000","modified":"2024-09-19T07:49:43.407266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6096":{"gid":6096,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/3642987189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3642987189},{"type":"v1","addr":"172.21.15.146:6837","nonce":3642987189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6096,"qdb_cluster":[6096]},"id":5}]} 2024-09-19T07:50:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 41 2024-09-19T07:50:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.656+0000 7f2bb9311640 1 -- 172.21.15.16:0/1601221065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2b9406c310 msgr2=0x7f2b9406e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.656+0000 7f2bb9311640 1 --2- 172.21.15.16:0/1601221065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2b9406c310 0x7f2b9406e7d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4034000 tx=0x7f2ba402e000 comp rx=0 tx=0).stop 2024-09-19T07:50:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.656+0000 7f2bb9311640 1 -- 172.21.15.16:0/1601221065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb415bee0 msgr2=0x7f2bb40cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.656+0000 7f2bb9311640 1 --2- 172.21.15.16:0/1601221065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb415bee0 0x7f2bb40cc910 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00c440 tx=0x7f2b9c001e70 comp rx=0 tx=0).stop 2024-09-19T07:50:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.656+0000 7f2bb9311640 1 -- 172.21.15.16:0/1601221065 shutdown_connections 2024-09-19T07:50:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.656+0000 7f2bb9311640 1 --2- 172.21.15.16:0/1601221065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2b9406c310 0x7f2b9406e7d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.656+0000 7f2bb9311640 1 --2- 172.21.15.16:0/1601221065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bb415c2c0 0x7f2bb40cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.656+0000 7f2bb9311640 1 --2- 172.21.15.16:0/1601221065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bb415bee0 0x7f2bb40cc910 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.656+0000 7f2bb9311640 1 --2- 172.21.15.16:0/1601221065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bb40cf910 0x7f2bb4158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.656+0000 7f2bb9311640 1 -- 172.21.15.16:0/1601221065 >> 172.21.15.16:0/1601221065 conn(0x7f2bb406bdf0 msgr2=0x7f2bb415a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.656+0000 7f2bb9311640 1 -- 172.21.15.16:0/1601221065 shutdown_connections 2024-09-19T07:50:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.656+0000 7f2bb9311640 1 -- 172.21.15.16:0/1601221065 wait complete. 2024-09-19T07:50:00.667 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:50:00.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.771+0000 7f7087c47640 1 Processor -- start 2024-09-19T07:50:00.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.771+0000 7f7087c47640 1 -- start start 2024-09-19T07:50:00.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.772+0000 7f7087c47640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f708015f0d0 0x7f708015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:00.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.772+0000 7f7087c47640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70800d4130 0x7f70800d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:00.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.772+0000 7f7087c47640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70800d4ae0 con 0x7f70800d0070 2024-09-19T07:50:00.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.772+0000 7f7087c47640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f70800d1b10 con 0x7f708015f0d0 2024-09-19T07:50:00.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.772+0000 7f7087c47640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f70800cf2b0 con 0x7f70800d4130 2024-09-19T07:50:00.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.772+0000 7f70859e3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f708015f0d0 0x7f708015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:00.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.772+0000 7f70859e3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f708015f0d0 0x7f708015f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45790/0 (socket says 172.21.15.16:45790) 2024-09-19T07:50:00.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.772+0000 7f70859e3640 1 -- 172.21.15.16:0/2430030976 learned_addr learned my addr 172.21.15.16:0/2430030976 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.772+0000 7f70861e4640 1 --2- 172.21.15.16:0/2430030976 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70800d4130 0x7f70800d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.772+0000 7f70859e3640 1 -- 172.21.15.16:0/2430030976 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70800d4130 msgr2=0x7f70800d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.772+0000 7f70859e3640 1 --2- 172.21.15.16:0/2430030976 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70800d4130 0x7f70800d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.772+0000 7f70859e3640 1 -- 172.21.15.16:0/2430030976 >> v1:172.21.15.146:6789/0 conn(0x7f70800d0070 legacy=0x7f70800d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.772+0000 7f70859e3640 1 -- 172.21.15.16:0/2430030976 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70800cf590 con 0x7f708015f0d0 2024-09-19T07:50:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.773+0000 7f70859e3640 1 --2- 172.21.15.16:0/2430030976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708015f0d0 0x7f708015f4b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7078009870 tx=0x7f707802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2859724972e4ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.773+0000 7f70849e1640 1 -- 172.21.15.16:0/2430030976 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7078037070 con 0x7f708015f0d0 2024-09-19T07:50:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.773+0000 7f70849e1640 1 -- 172.21.15.16:0/2430030976 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7078032440 con 0x7f708015f0d0 2024-09-19T07:50:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.773+0000 7f70849e1640 1 -- 172.21.15.16:0/2430030976 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7078032810 con 0x7f708015f0d0 2024-09-19T07:50:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.774+0000 7f7087c47640 1 -- 172.21.15.16:0/2430030976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708015f0d0 msgr2=0x7f708015f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.774+0000 7f7087c47640 1 --2- 172.21.15.16:0/2430030976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708015f0d0 0x7f708015f4b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7078009870 tx=0x7f707802a1c0 comp rx=0 tx=0).stop 2024-09-19T07:50:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.774+0000 7f7087c47640 1 -- 172.21.15.16:0/2430030976 shutdown_connections 2024-09-19T07:50:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.774+0000 7f7087c47640 1 --2- 172.21.15.16:0/2430030976 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70800d4130 0x7f70800d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:00.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.774+0000 7f7087c47640 1 --2- 172.21.15.16:0/2430030976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708015f0d0 0x7f708015f4b0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:00.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.774+0000 7f7087c47640 1 -- 172.21.15.16:0/2430030976 >> 172.21.15.16:0/2430030976 conn(0x7f708009d760 msgr2=0x7f708008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:00.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.774+0000 7f7087c47640 1 -- 172.21.15.16:0/2430030976 shutdown_connections 2024-09-19T07:50:00.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.774+0000 7f7087c47640 1 -- 172.21.15.16:0/2430030976 wait complete. 2024-09-19T07:50:00.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.775+0000 7f7087c47640 1 Processor -- start 2024-09-19T07:50:00.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.775+0000 7f7087c47640 1 -- start start 2024-09-19T07:50:00.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f7087c47640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70800d0070 0x7f70800b7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:00.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f7087c47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70800d4130 0x7f708015b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:00.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f7087c47640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f708015f0d0 0x7f70800c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:00.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f7087c47640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f70800cef60 con 0x7f70800d4130 2024-09-19T07:50:00.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f7087c47640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f70800cfb50 con 0x7f708015f0d0 2024-09-19T07:50:00.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f7087c47640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f70800b2f70 con 0x7f70800d0070 2024-09-19T07:50:00.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f70851e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70800d4130 0x7f708015b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:00.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f70851e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70800d4130 0x7f708015b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45792/0 (socket says 172.21.15.16:45792) 2024-09-19T07:50:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f70851e2640 1 -- 172.21.15.16:0/699909668 learned_addr learned my addr 172.21.15.16:0/699909668 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f70859e3640 1 --2- 172.21.15.16:0/699909668 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70800d0070 0x7f70800b7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f70861e4640 1 --2- 172.21.15.16:0/699909668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f708015f0d0 0x7f70800c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f70851e2640 1 -- 172.21.15.16:0/699909668 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70800d0070 msgr2=0x7f70800b7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f70851e2640 1 --2- 172.21.15.16:0/699909668 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70800d0070 0x7f70800b7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f70851e2640 1 -- 172.21.15.16:0/699909668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f708015f0d0 msgr2=0x7f70800c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f70851e2640 1 --2- 172.21.15.16:0/699909668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f708015f0d0 0x7f70800c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f70851e2640 1 -- 172.21.15.16:0/699909668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70800ccaa0 con 0x7f70800d4130 2024-09-19T07:50:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f70851e2640 1 --2- 172.21.15.16:0/699909668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70800d4130 0x7f708015b7c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f707400c400 tx=0x7f707400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f707e7fc640 1 -- 172.21.15.16:0/699909668 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70740052d0 con 0x7f70800d4130 2024-09-19T07:50:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f707e7fc640 1 -- 172.21.15.16:0/699909668 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7074005490 con 0x7f70800d4130 2024-09-19T07:50:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f7087c47640 1 -- 172.21.15.16:0/699909668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70800ccd80 con 0x7f70800d4130 2024-09-19T07:50:00.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.776+0000 7f707e7fc640 1 -- 172.21.15.16:0/699909668 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074007260 con 0x7f70800d4130 2024-09-19T07:50:00.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.777+0000 7f7087c47640 1 -- 172.21.15.16:0/699909668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70800cd340 con 0x7f70800d4130 2024-09-19T07:50:00.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.778+0000 7f7087c47640 1 -- 172.21.15.16:0/699909668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70800d45a0 con 0x7f70800d4130 2024-09-19T07:50:00.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.778+0000 7f707e7fc640 1 -- 172.21.15.16:0/699909668 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7074007400 con 0x7f70800d4130 2024-09-19T07:50:00.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.779+0000 7f707e7fc640 1 --2- 172.21.15.16:0/699909668 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f706c06c2b0 0x7f706c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:00.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.779+0000 7f707e7fc640 1 -- 172.21.15.16:0/699909668 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f707408aa50 con 0x7f70800d4130 2024-09-19T07:50:00.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.779+0000 7f70859e3640 1 --2- 172.21.15.16:0/699909668 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f706c06c2b0 0x7f706c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:00.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.780+0000 7f70859e3640 1 --2- 172.21.15.16:0/699909668 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f706c06c2b0 0x7f706c06e770 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7078034000 tx=0x7f707802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:00.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.783+0000 7f707e7fc640 1 -- 172.21.15.16:0/699909668 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f70740589e0 con 0x7f70800d4130 2024-09-19T07:50:00.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.996+0000 7f7087c47640 1 -- 172.21.15.16:0/699909668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f708006da90 con 0x7f70800d4130 2024-09-19T07:50:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.997+0000 7f707e7fc640 1 -- 172.21.15.16:0/699909668 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5222 (secure 0 0 0) 0x7f707405cce0 con 0x7f70800d4130 2024-09-19T07:50:00.997 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:00.997 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"btime":"2024-09-19T07:49:44:053013+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/945862061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":945862061},{"type":"v1","addr":"172.21.15.146:6836","nonce":945862061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/42753795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":42753795},{"type":"v1","addr":"172.21.15.16:6834","nonce":42753795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1389787480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1389787480},{"type":"v1","addr":"172.21.15.16:6835","nonce":1389787480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:49:40.403885+0000","modified":"2024-09-19T07:49:43.407266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6096":{"gid":6096,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/3642987189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3642987189},{"type":"v1","addr":"172.21.15.146:6837","nonce":3642987189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6096,"qdb_cluster":[6096]},"id":5}]} 2024-09-19T07:50:00.998 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 41 2024-09-19T07:50:00.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.999+0000 7f7087c47640 1 -- 172.21.15.16:0/699909668 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f706c06c2b0 msgr2=0x7f706c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:00.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.999+0000 7f7087c47640 1 --2- 172.21.15.16:0/699909668 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f706c06c2b0 0x7f706c06e770 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7078034000 tx=0x7f707802e000 comp rx=0 tx=0).stop 2024-09-19T07:50:01.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.999+0000 7f7087c47640 1 -- 172.21.15.16:0/699909668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70800d4130 msgr2=0x7f708015b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:01.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.999+0000 7f7087c47640 1 --2- 172.21.15.16:0/699909668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70800d4130 0x7f708015b7c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f707400c400 tx=0x7f707400c8c0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.999+0000 7f7087c47640 1 -- 172.21.15.16:0/699909668 shutdown_connections 2024-09-19T07:50:01.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.999+0000 7f7087c47640 1 --2- 172.21.15.16:0/699909668 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f706c06c2b0 0x7f706c06e770 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.999+0000 7f7087c47640 1 --2- 172.21.15.16:0/699909668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f708015f0d0 0x7f70800c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.999+0000 7f7087c47640 1 --2- 172.21.15.16:0/699909668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70800d4130 0x7f708015b7c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.999+0000 7f7087c47640 1 --2- 172.21.15.16:0/699909668 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70800d0070 0x7f70800b7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.999+0000 7f7087c47640 1 -- 172.21.15.16:0/699909668 >> 172.21.15.16:0/699909668 conn(0x7f708009d760 msgr2=0x7f708015dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:01.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.999+0000 7f7087c47640 1 -- 172.21.15.16:0/699909668 shutdown_connections 2024-09-19T07:50:01.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:00.999+0000 7f7087c47640 1 -- 172.21.15.16:0/699909668 wait complete. 2024-09-19T07:50:01.010 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:50:01.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.112+0000 7ff4da26e640 1 Processor -- start 2024-09-19T07:50:01.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.112+0000 7ff4da26e640 1 -- start start 2024-09-19T07:50:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.113+0000 7ff4da26e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff4d415e680 0x7ff4d40cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.113+0000 7ff4da26e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4d40d0010 0x7ff4d40d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.113+0000 7ff4da26e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4d40d1880 con 0x7ff4d415e2a0 2024-09-19T07:50:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.113+0000 7ff4da26e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff4d40d1b60 con 0x7ff4d40d0010 2024-09-19T07:50:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.113+0000 7ff4da26e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff4d40cf2d0 con 0x7ff4d415e680 2024-09-19T07:50:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.113+0000 7ff4d880b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4d40d0010 0x7ff4d40d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.113+0000 7ff4d880b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4d40d0010 0x7ff4d40d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45798/0 (socket says 172.21.15.16:45798) 2024-09-19T07:50:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.113+0000 7ff4d880b640 1 -- 172.21.15.16:0/2988805676 learned_addr learned my addr 172.21.15.16:0/2988805676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.113+0000 7ff4d37fe640 1 --2- 172.21.15.16:0/2988805676 >> v2:172.21.15.146:3300/0 conn(0x7ff4d415e680 0x7ff4d40cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.113+0000 7ff4d880b640 1 -- 172.21.15.16:0/2988805676 >> v2:172.21.15.146:3300/0 conn(0x7ff4d415e680 msgr2=0x7ff4d40cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.113+0000 7ff4d880b640 1 --2- 172.21.15.16:0/2988805676 >> v2:172.21.15.146:3300/0 conn(0x7ff4d415e680 0x7ff4d40cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.113+0000 7ff4d880b640 1 -- 172.21.15.16:0/2988805676 >> v1:172.21.15.146:6789/0 conn(0x7ff4d415e2a0 legacy=0x7ff4d406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.113+0000 7ff4d880b640 1 -- 172.21.15.16:0/2988805676 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d40cf5b0 con 0x7ff4d40d0010 2024-09-19T07:50:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.114+0000 7ff4d880b640 1 --2- 172.21.15.16:0/2988805676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d40d0010 0x7ff4d40d4370 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40a22a0 tx=0x7ff4cc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54393de7233a76c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.114+0000 7ff4d2ffd640 1 -- 172.21.15.16:0/2988805676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc036070 con 0x7ff4d40d0010 2024-09-19T07:50:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.114+0000 7ff4d2ffd640 1 -- 172.21.15.16:0/2988805676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4cc031600 con 0x7ff4d40d0010 2024-09-19T07:50:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.114+0000 7ff4d2ffd640 1 -- 172.21.15.16:0/2988805676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc0319d0 con 0x7ff4d40d0010 2024-09-19T07:50:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.115+0000 7ff4da26e640 1 -- 172.21.15.16:0/2988805676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d40d0010 msgr2=0x7ff4d40d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.115+0000 7ff4da26e640 1 --2- 172.21.15.16:0/2988805676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d40d0010 0x7ff4d40d4370 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40a22a0 tx=0x7ff4cc02a660 comp rx=0 tx=0).stop 2024-09-19T07:50:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.115+0000 7ff4da26e640 1 -- 172.21.15.16:0/2988805676 shutdown_connections 2024-09-19T07:50:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.115+0000 7ff4da26e640 1 --2- 172.21.15.16:0/2988805676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d40d0010 0x7ff4d40d4370 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.115+0000 7ff4da26e640 1 --2- 172.21.15.16:0/2988805676 >> v2:172.21.15.146:3300/0 conn(0x7ff4d415e680 0x7ff4d40cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.115+0000 7ff4da26e640 1 -- 172.21.15.16:0/2988805676 >> 172.21.15.16:0/2988805676 conn(0x7ff4d409d760 msgr2=0x7ff4d408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.115+0000 7ff4da26e640 1 -- 172.21.15.16:0/2988805676 shutdown_connections 2024-09-19T07:50:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.115+0000 7ff4da26e640 1 -- 172.21.15.16:0/2988805676 wait complete. 2024-09-19T07:50:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.116+0000 7ff4da26e640 1 Processor -- start 2024-09-19T07:50:01.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.116+0000 7ff4da26e640 1 -- start start 2024-09-19T07:50:01.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.117+0000 7ff4da26e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4d40d0010 0x7ff4d415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.117+0000 7ff4da26e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d415e2a0 0x7ff4d40cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.117+0000 7ff4da26e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4d415e680 0x7ff4d40cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.117+0000 7ff4da26e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d406dcb0 con 0x7ff4d415e2a0 2024-09-19T07:50:01.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.117+0000 7ff4da26e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d406db30 con 0x7ff4d415e680 2024-09-19T07:50:01.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.117+0000 7ff4da26e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff4d40d48b0 con 0x7ff4d40d0010 2024-09-19T07:50:01.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.117+0000 7ff4d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d415e2a0 0x7ff4d40cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.117+0000 7ff4d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d415e2a0 0x7ff4d40cd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45802/0 (socket says 172.21.15.16:45802) 2024-09-19T07:50:01.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.117+0000 7ff4d37fe640 1 -- 172.21.15.16:0/2654871480 learned_addr learned my addr 172.21.15.16:0/2654871480 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:01.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.117+0000 7ff4d3fff640 1 --2- 172.21.15.16:0/2654871480 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4d40d0010 0x7ff4d415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.117+0000 7ff4d880b640 1 --2- 172.21.15.16:0/2654871480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4d415e680 0x7ff4d40cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.118+0000 7ff4d37fe640 1 -- 172.21.15.16:0/2654871480 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4d40d0010 msgr2=0x7ff4d415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.118+0000 7ff4d37fe640 1 --2- 172.21.15.16:0/2654871480 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4d40d0010 0x7ff4d415c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.118+0000 7ff4d37fe640 1 -- 172.21.15.16:0/2654871480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4d415e680 msgr2=0x7ff4d40cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.118+0000 7ff4d37fe640 1 --2- 172.21.15.16:0/2654871480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4d415e680 0x7ff4d40cdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.118+0000 7ff4d37fe640 1 -- 172.21.15.16:0/2654871480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d40ce740 con 0x7ff4d415e2a0 2024-09-19T07:50:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.118+0000 7ff4d3fff640 1 --2- 172.21.15.16:0/2654871480 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4d40d0010 0x7ff4d415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.118+0000 7ff4d880b640 1 --2- 172.21.15.16:0/2654871480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4d415e680 0x7ff4d40cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.118+0000 7ff4d37fe640 1 --2- 172.21.15.16:0/2654871480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d415e2a0 0x7ff4d40cd5c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800d5c0 tx=0x7ff4c800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.118+0000 7ff4d0ff9640 1 -- 172.21.15.16:0/2654871480 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c80125f0 con 0x7ff4d415e2a0 2024-09-19T07:50:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.118+0000 7ff4d0ff9640 1 -- 172.21.15.16:0/2654871480 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c8019070 con 0x7ff4d415e2a0 2024-09-19T07:50:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.118+0000 7ff4da26e640 1 -- 172.21.15.16:0/2654871480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d40ce180 con 0x7ff4d415e2a0 2024-09-19T07:50:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.118+0000 7ff4d0ff9640 1 -- 172.21.15.16:0/2654871480 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c80129d0 con 0x7ff4d415e2a0 2024-09-19T07:50:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.118+0000 7ff4da26e640 1 -- 172.21.15.16:0/2654871480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d40c9370 con 0x7ff4d415e2a0 2024-09-19T07:50:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.119+0000 7ff4da26e640 1 -- 172.21.15.16:0/2654871480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d40d4c70 con 0x7ff4d415e2a0 2024-09-19T07:50:01.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.120+0000 7ff4d0ff9640 1 -- 172.21.15.16:0/2654871480 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff4c8012b70 con 0x7ff4d415e2a0 2024-09-19T07:50:01.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.120+0000 7ff4d0ff9640 1 --2- 172.21.15.16:0/2654871480 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff4c006c2b0 0x7ff4c006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.120+0000 7ff4d0ff9640 1 -- 172.21.15.16:0/2654871480 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7ff4c808b930 con 0x7ff4d415e2a0 2024-09-19T07:50:01.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.120+0000 7ff4d3fff640 1 --2- 172.21.15.16:0/2654871480 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff4c006c2b0 0x7ff4c006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.121+0000 7ff4d3fff640 1 --2- 172.21.15.16:0/2654871480 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff4c006c2b0 0x7ff4c006e770 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4003eb0 tx=0x7ff4c4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:01.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.125+0000 7ff4d0ff9640 1 -- 172.21.15.16:0/2654871480 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff4c8059fa0 con 0x7ff4d415e2a0 2024-09-19T07:50:01.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.300+0000 7ff4da26e640 1 -- 172.21.15.16:0/2654871480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4d406cf90 con 0x7ff4d415e2a0 2024-09-19T07:50:01.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.301+0000 7ff4d0ff9640 1 -- 172.21.15.16:0/2654871480 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+17029 (secure 0 0 0) 0x7ff4c80598a0 con 0x7ff4d415e2a0 2024-09-19T07:50:01.305 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:01.305 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:49:42.039882+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:49:38.349338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-19T07:49:39.377498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:50:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.303+0000 7ff4da26e640 1 -- 172.21.15.16:0/2654871480 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff4c006c2b0 msgr2=0x7ff4c006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.303+0000 7ff4da26e640 1 --2- 172.21.15.16:0/2654871480 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff4c006c2b0 0x7ff4c006e770 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4003eb0 tx=0x7ff4c4009040 comp rx=0 tx=0).stop 2024-09-19T07:50:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.303+0000 7ff4da26e640 1 -- 172.21.15.16:0/2654871480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d415e2a0 msgr2=0x7ff4d40cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.303+0000 7ff4da26e640 1 --2- 172.21.15.16:0/2654871480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d415e2a0 0x7ff4d40cd5c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800d5c0 tx=0x7ff4c800da80 comp rx=0 tx=0).stop 2024-09-19T07:50:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.303+0000 7ff4da26e640 1 -- 172.21.15.16:0/2654871480 shutdown_connections 2024-09-19T07:50:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.303+0000 7ff4da26e640 1 --2- 172.21.15.16:0/2654871480 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff4c006c2b0 0x7ff4c006e770 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.303+0000 7ff4da26e640 1 --2- 172.21.15.16:0/2654871480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4d415e680 0x7ff4d40cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.303+0000 7ff4da26e640 1 --2- 172.21.15.16:0/2654871480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d415e2a0 0x7ff4d40cd5c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.303+0000 7ff4da26e640 1 --2- 172.21.15.16:0/2654871480 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4d40d0010 0x7ff4d415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.303+0000 7ff4da26e640 1 -- 172.21.15.16:0/2654871480 >> 172.21.15.16:0/2654871480 conn(0x7ff4d409d760 msgr2=0x7ff4d4162320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.303+0000 7ff4da26e640 1 -- 172.21.15.16:0/2654871480 shutdown_connections 2024-09-19T07:50:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.303+0000 7ff4da26e640 1 -- 172.21.15.16:0/2654871480 wait complete. 2024-09-19T07:50:01.318 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T07:50:01.319 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:50:01.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.429+0000 7f97dc0d0640 1 Processor -- start 2024-09-19T07:50:01.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.429+0000 7f97dc0d0640 1 -- start start 2024-09-19T07:50:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.430+0000 7f97dc0d0640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f97d40d0bc0 0x7f97d40c5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.430+0000 7f97dc0d0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97d415b500 con 0x7f97d406f9e0 2024-09-19T07:50:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.430+0000 7f97dc0d0640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97d406e940 con 0x7f97d415b120 2024-09-19T07:50:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.430+0000 7f97dc0d0640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f97d406ec20 con 0x7f97d40d0bc0 2024-09-19T07:50:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.430+0000 7f97d966b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f97d406f9e0 0x7f97d40d04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55304/0 (socket says 172.21.15.16:55304) 2024-09-19T07:50:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.430+0000 7f97d966b640 1 -- 172.21.15.16:0/402201322 learned_addr learned my addr 172.21.15.16:0/402201322 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.430+0000 7f97da66d640 1 --2- 172.21.15.16:0/402201322 >> v2:172.21.15.146:3300/0 conn(0x7f97d40d0bc0 0x7f97d40c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.430+0000 7f97d8e6a640 1 -- 172.21.15.16:0/402201322 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3526609158 0 0) 0x7f97d415b500 con 0x7f97d406f9e0 2024-09-19T07:50:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97d8e6a640 1 -- 172.21.15.16:0/402201322 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97c0003520 con 0x7f97d406f9e0 2024-09-19T07:50:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97d8e6a640 1 -- 172.21.15.16:0/402201322 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1093537499 0 0) 0x7f97c0003520 con 0x7f97d406f9e0 2024-09-19T07:50:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97d8e6a640 1 -- 172.21.15.16:0/402201322 >> v2:172.21.15.146:3300/0 conn(0x7f97d40d0bc0 msgr2=0x7f97d40c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97d8e6a640 1 --2- 172.21.15.16:0/402201322 >> v2:172.21.15.146:3300/0 conn(0x7f97d40d0bc0 0x7f97d40c5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97d8e6a640 1 -- 172.21.15.16:0/402201322 >> v1:172.21.15.146:6789/0 conn(0x7f97d415b120 legacy=0x7f97d406dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97d8e6a640 1 -- 172.21.15.16:0/402201322 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d406ef00 con 0x7f97d406f9e0 2024-09-19T07:50:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97d8e6a640 1 -- 172.21.15.16:0/402201322 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f97c4001670 con 0x7f97d406f9e0 2024-09-19T07:50:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97d8e6a640 1 -- 172.21.15.16:0/402201322 >> v1:172.21.15.16:6789/0 conn(0x7f97d406f9e0 legacy=0x7f97d40d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97d8e6a640 1 --2- 172.21.15.16:0/402201322 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97c00039b0 0x7f97c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97d8e6a640 1 --2- 172.21.15.16:0/402201322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97c0004800 0x7f97c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97d8e6a640 1 --2- 172.21.15.16:0/402201322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c00051c0 0x7f97c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97d8e6a640 1 -- 172.21.15.16:0/402201322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97d406ec20 con 0x7f97c00051c0 2024-09-19T07:50:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97d8e6a640 1 -- 172.21.15.16:0/402201322 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f97c000c060 con 0x7f97c0004800 2024-09-19T07:50:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97d8e6a640 1 -- 172.21.15.16:0/402201322 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f97c000c260 con 0x7f97c00039b0 2024-09-19T07:50:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97da66d640 1 --2- 172.21.15.16:0/402201322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c00051c0 0x7f97c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97d966b640 1 --2- 172.21.15.16:0/402201322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97c0004800 0x7f97c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97d9e6c640 1 --2- 172.21.15.16:0/402201322 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97c00039b0 0x7f97c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97da66d640 1 -- 172.21.15.16:0/402201322 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97c00039b0 msgr2=0x7f97c0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97da66d640 1 --2- 172.21.15.16:0/402201322 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97c00039b0 0x7f97c0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97da66d640 1 -- 172.21.15.16:0/402201322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97c0004800 msgr2=0x7f97c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97da66d640 1 --2- 172.21.15.16:0/402201322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97c0004800 0x7f97c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.431+0000 7f97da66d640 1 -- 172.21.15.16:0/402201322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c0003520 con 0x7f97c00051c0 2024-09-19T07:50:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.432+0000 7f97da66d640 1 -- 172.21.15.16:0/402201322 reap_dead start 2024-09-19T07:50:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.432+0000 7f97da66d640 1 --2- 172.21.15.16:0/402201322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c00051c0 0x7f97c000b9c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009fc0 tx=0x7f97c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdc5a3cb9e2d015 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.432+0000 7f97d8e6a640 1 -- 172.21.15.16:0/402201322 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc009c10 con 0x7f97c00051c0 2024-09-19T07:50:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.432+0000 7f97d8e6a640 1 -- 172.21.15.16:0/402201322 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97cc003bd0 con 0x7f97c00051c0 2024-09-19T07:50:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.432+0000 7f97d8e6a640 1 -- 172.21.15.16:0/402201322 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc004020 con 0x7f97c00051c0 2024-09-19T07:50:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.433+0000 7f97dc0d0640 1 -- 172.21.15.16:0/402201322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c00051c0 msgr2=0x7f97c000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.433+0000 7f97dc0d0640 1 --2- 172.21.15.16:0/402201322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c00051c0 0x7f97c000b9c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009fc0 tx=0x7f97c0005830 comp rx=0 tx=0).stop 2024-09-19T07:50:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.433+0000 7f97dc0d0640 1 -- 172.21.15.16:0/402201322 shutdown_connections 2024-09-19T07:50:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.433+0000 7f97dc0d0640 1 --2- 172.21.15.16:0/402201322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c00051c0 0x7f97c000b9c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.433+0000 7f97dc0d0640 1 -- 172.21.15.16:0/402201322 >> 172.21.15.16:0/402201322 conn(0x7f97d406bdf0 msgr2=0x7f97d406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.433+0000 7f97dc0d0640 1 -- 172.21.15.16:0/402201322 shutdown_connections 2024-09-19T07:50:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.433+0000 7f97dc0d0640 1 -- 172.21.15.16:0/402201322 wait complete. 2024-09-19T07:50:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.434+0000 7f97dc0d0640 1 Processor -- start 2024-09-19T07:50:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.434+0000 7f97dc0d0640 1 -- start start 2024-09-19T07:50:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.435+0000 7f97dc0d0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97c00051c0 0x7f97d40c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.435+0000 7f97dc0d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d40ad160 0x7f97d41f2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.435+0000 7f97dc0d0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97d41f2a90 0x7f97d40c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.435+0000 7f97dc0d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97d40ce060 con 0x7f97d40ad160 2024-09-19T07:50:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.435+0000 7f97dc0d0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f97d406f4c0 con 0x7f97c00051c0 2024-09-19T07:50:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.435+0000 7f97dc0d0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f97d40d04b0 con 0x7f97d41f2a90 2024-09-19T07:50:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.435+0000 7f97d966b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d40ad160 0x7f97d41f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.435+0000 7f97d966b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d40ad160 0x7f97d41f2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45830/0 (socket says 172.21.15.16:45830) 2024-09-19T07:50:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.435+0000 7f97d966b640 1 -- 172.21.15.16:0/1544151767 learned_addr learned my addr 172.21.15.16:0/1544151767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.435+0000 7f97d9e6c640 1 --2- 172.21.15.16:0/1544151767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97c00051c0 0x7f97d40c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.435+0000 7f97da66d640 1 --2- 172.21.15.16:0/1544151767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97d41f2a90 0x7f97d40c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.435+0000 7f97d966b640 1 -- 172.21.15.16:0/1544151767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97d41f2a90 msgr2=0x7f97d40c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.435+0000 7f97d966b640 1 --2- 172.21.15.16:0/1544151767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97d41f2a90 0x7f97d40c7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.435+0000 7f97d966b640 1 -- 172.21.15.16:0/1544151767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97c00051c0 msgr2=0x7f97d40c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.435+0000 7f97d966b640 1 --2- 172.21.15.16:0/1544151767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97c00051c0 0x7f97d40c9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.435+0000 7f97d966b640 1 -- 172.21.15.16:0/1544151767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d40aea70 con 0x7f97d40ad160 2024-09-19T07:50:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.435+0000 7f97da66d640 1 --2- 172.21.15.16:0/1544151767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97d41f2a90 0x7f97d40c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.435+0000 7f97d966b640 1 --2- 172.21.15.16:0/1544151767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d40ad160 0x7f97d41f2550 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f97c40072f0 tx=0x7f97c40077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.436+0000 7f97ca7fc640 1 -- 172.21.15.16:0/1544151767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c401a070 con 0x7f97d40ad160 2024-09-19T07:50:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.436+0000 7f97ca7fc640 1 -- 172.21.15.16:0/1544151767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c400f960 con 0x7f97d40ad160 2024-09-19T07:50:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.436+0000 7f97dc0d0640 1 -- 172.21.15.16:0/1544151767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d40ad9f0 con 0x7f97d40ad160 2024-09-19T07:50:01.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.436+0000 7f97ca7fc640 1 -- 172.21.15.16:0/1544151767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c400fd10 con 0x7f97d40ad160 2024-09-19T07:50:01.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.436+0000 7f97dc0d0640 1 -- 172.21.15.16:0/1544151767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d40ae290 con 0x7f97d40ad160 2024-09-19T07:50:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.437+0000 7f97dc0d0640 1 -- 172.21.15.16:0/1544151767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d40d07e0 con 0x7f97d40ad160 2024-09-19T07:50:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.437+0000 7f97ca7fc640 1 -- 172.21.15.16:0/1544151767 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f97c401d070 con 0x7f97d40ad160 2024-09-19T07:50:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.438+0000 7f97ca7fc640 1 --2- 172.21.15.16:0/1544151767 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f97b806c2b0 0x7f97b806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.438+0000 7f97ca7fc640 1 -- 172.21.15.16:0/1544151767 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f97c408af20 con 0x7f97d40ad160 2024-09-19T07:50:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.438+0000 7f97d9e6c640 1 --2- 172.21.15.16:0/1544151767 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f97b806c2b0 0x7f97b806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.438+0000 7f97d9e6c640 1 --2- 172.21.15.16:0/1544151767 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f97b806c2b0 0x7f97b806e770 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f97d0000c00 tx=0x7f97d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.442+0000 7f97ca7fc640 1 -- 172.21.15.16:0/1544151767 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f97c4058eb0 con 0x7f97d40ad160 2024-09-19T07:50:01.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.654+0000 7f97dc0d0640 1 -- 172.21.15.16:0/1544151767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97d40b4320 con 0x7f97d40ad160 2024-09-19T07:50:01.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.655+0000 7f97ca7fc640 1 -- 172.21.15.16:0/1544151767 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5222 (secure 0 0 0) 0x7f97c405d1b0 con 0x7f97d40ad160 2024-09-19T07:50:01.656 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:01.656 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"btime":"2024-09-19T07:49:44:053013+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/945862061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":945862061},{"type":"v1","addr":"172.21.15.146:6836","nonce":945862061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/42753795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":42753795},{"type":"v1","addr":"172.21.15.16:6834","nonce":42753795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1389787480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1389787480},{"type":"v1","addr":"172.21.15.16:6835","nonce":1389787480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:49:40.403885+0000","modified":"2024-09-19T07:49:43.407266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6096":{"gid":6096,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/3642987189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3642987189},{"type":"v1","addr":"172.21.15.146:6837","nonce":3642987189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6096,"qdb_cluster":[6096]},"id":5}]} 2024-09-19T07:50:01.656 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 41 2024-09-19T07:50:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.657+0000 7f97dc0d0640 1 -- 172.21.15.16:0/1544151767 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f97b806c2b0 msgr2=0x7f97b806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.657+0000 7f97dc0d0640 1 --2- 172.21.15.16:0/1544151767 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f97b806c2b0 0x7f97b806e770 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f97d0000c00 tx=0x7f97d0009040 comp rx=0 tx=0).stop 2024-09-19T07:50:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.657+0000 7f97dc0d0640 1 -- 172.21.15.16:0/1544151767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d40ad160 msgr2=0x7f97d41f2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.657+0000 7f97dc0d0640 1 --2- 172.21.15.16:0/1544151767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d40ad160 0x7f97d41f2550 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f97c40072f0 tx=0x7f97c40077b0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.657+0000 7f97dc0d0640 1 -- 172.21.15.16:0/1544151767 shutdown_connections 2024-09-19T07:50:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.657+0000 7f97dc0d0640 1 --2- 172.21.15.16:0/1544151767 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f97b806c2b0 0x7f97b806e770 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.657+0000 7f97dc0d0640 1 --2- 172.21.15.16:0/1544151767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97d41f2a90 0x7f97d40c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.657+0000 7f97dc0d0640 1 --2- 172.21.15.16:0/1544151767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d40ad160 0x7f97d41f2550 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.657+0000 7f97dc0d0640 1 --2- 172.21.15.16:0/1544151767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97c00051c0 0x7f97d40c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.657+0000 7f97dc0d0640 1 -- 172.21.15.16:0/1544151767 >> 172.21.15.16:0/1544151767 conn(0x7f97d406bdf0 msgr2=0x7f97d40c62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.657+0000 7f97dc0d0640 1 -- 172.21.15.16:0/1544151767 shutdown_connections 2024-09-19T07:50:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.657+0000 7f97dc0d0640 1 -- 172.21.15.16:0/1544151767 wait complete. 2024-09-19T07:50:01.668 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:50:01.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.770+0000 7fce12c7b640 1 Processor -- start 2024-09-19T07:50:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.770+0000 7fce12c7b640 1 -- start start 2024-09-19T07:50:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.771+0000 7fce12c7b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce0c15e2a0 0x7fce0c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.771+0000 7fce12c7b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce0c15e680 0x7fce0c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.771+0000 7fce12c7b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce0c0d1880 con 0x7fce0c06d960 2024-09-19T07:50:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.771+0000 7fce12c7b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fce0c0d1b60 con 0x7fce0c15e680 2024-09-19T07:50:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.771+0000 7fce12c7b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fce0c0cf2d0 con 0x7fce0c15e2a0 2024-09-19T07:50:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.771+0000 7fce0bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce0c15e680 0x7fce0c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.771+0000 7fce0bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce0c15e680 0x7fce0c0cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45834/0 (socket says 172.21.15.16:45834) 2024-09-19T07:50:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.771+0000 7fce0bfff640 1 -- 172.21.15.16:0/1876386286 learned_addr learned my addr 172.21.15.16:0/1876386286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.771+0000 7fce10a17640 1 --2- 172.21.15.16:0/1876386286 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce0c15e2a0 0x7fce0c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.771+0000 7fce0bfff640 1 -- 172.21.15.16:0/1876386286 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce0c15e2a0 msgr2=0x7fce0c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.771+0000 7fce0bfff640 1 --2- 172.21.15.16:0/1876386286 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce0c15e2a0 0x7fce0c06d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.771+0000 7fce0bfff640 1 -- 172.21.15.16:0/1876386286 >> v1:172.21.15.146:6789/0 conn(0x7fce0c06d960 legacy=0x7fce0c0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.771+0000 7fce0bfff640 1 -- 172.21.15.16:0/1876386286 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0c0cf5b0 con 0x7fce0c15e680 2024-09-19T07:50:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.772+0000 7fce0bfff640 1 --2- 172.21.15.16:0/1876386286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce0c15e680 0x7fce0c0cfad0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009870 tx=0x7fcdf802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f1f9dee2cf61a9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.772+0000 7fce0b7fe640 1 -- 172.21.15.16:0/1876386286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8037070 con 0x7fce0c15e680 2024-09-19T07:50:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.772+0000 7fce0b7fe640 1 -- 172.21.15.16:0/1876386286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf8032440 con 0x7fce0c15e680 2024-09-19T07:50:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.772+0000 7fce0b7fe640 1 -- 172.21.15.16:0/1876386286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8032810 con 0x7fce0c15e680 2024-09-19T07:50:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.773+0000 7fce12c7b640 1 -- 172.21.15.16:0/1876386286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce0c15e680 msgr2=0x7fce0c0cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.773+0000 7fce12c7b640 1 --2- 172.21.15.16:0/1876386286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce0c15e680 0x7fce0c0cfad0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009870 tx=0x7fcdf802a270 comp rx=0 tx=0).stop 2024-09-19T07:50:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.773+0000 7fce12c7b640 1 -- 172.21.15.16:0/1876386286 shutdown_connections 2024-09-19T07:50:01.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.773+0000 7fce12c7b640 1 --2- 172.21.15.16:0/1876386286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce0c15e680 0x7fce0c0cfad0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.773+0000 7fce12c7b640 1 --2- 172.21.15.16:0/1876386286 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce0c15e2a0 0x7fce0c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.773+0000 7fce12c7b640 1 -- 172.21.15.16:0/1876386286 >> 172.21.15.16:0/1876386286 conn(0x7fce0c09d760 msgr2=0x7fce0c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:01.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.773+0000 7fce12c7b640 1 -- 172.21.15.16:0/1876386286 shutdown_connections 2024-09-19T07:50:01.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.773+0000 7fce12c7b640 1 -- 172.21.15.16:0/1876386286 wait complete. 2024-09-19T07:50:01.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.774+0000 7fce12c7b640 1 Processor -- start 2024-09-19T07:50:01.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.774+0000 7fce12c7b640 1 -- start start 2024-09-19T07:50:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.775+0000 7fce12c7b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce0c06d960 0x7fce0c161f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.775+0000 7fce12c7b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce0c15e2a0 0x7fce0c162480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.775+0000 7fce12c7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce0c15e680 0x7fce0c0b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.775+0000 7fce12c7b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c0d4c00 con 0x7fce0c15e680 2024-09-19T07:50:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.775+0000 7fce12c7b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c0d4a80 con 0x7fce0c06d960 2024-09-19T07:50:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.775+0000 7fce12c7b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fce0c0d0010 con 0x7fce0c15e2a0 2024-09-19T07:50:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.775+0000 7fce11218640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce0c15e680 0x7fce0c0b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.775+0000 7fce11218640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce0c15e680 0x7fce0c0b8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45836/0 (socket says 172.21.15.16:45836) 2024-09-19T07:50:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.775+0000 7fce11218640 1 -- 172.21.15.16:0/2225588438 learned_addr learned my addr 172.21.15.16:0/2225588438 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.775+0000 7fce0bfff640 1 --2- 172.21.15.16:0/2225588438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce0c15e2a0 0x7fce0c162480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.775+0000 7fce10a17640 1 --2- 172.21.15.16:0/2225588438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce0c06d960 0x7fce0c161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.775+0000 7fce11218640 1 -- 172.21.15.16:0/2225588438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce0c15e2a0 msgr2=0x7fce0c162480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.775+0000 7fce11218640 1 --2- 172.21.15.16:0/2225588438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce0c15e2a0 0x7fce0c162480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.775+0000 7fce11218640 1 -- 172.21.15.16:0/2225588438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce0c06d960 msgr2=0x7fce0c161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.775+0000 7fce11218640 1 --2- 172.21.15.16:0/2225588438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce0c06d960 0x7fce0c161f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.775+0000 7fce11218640 1 -- 172.21.15.16:0/2225588438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0c0ca0e0 con 0x7fce0c15e680 2024-09-19T07:50:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.775+0000 7fce10a17640 1 --2- 172.21.15.16:0/2225588438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce0c06d960 0x7fce0c161f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.775+0000 7fce0bfff640 1 --2- 172.21.15.16:0/2225588438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce0c15e2a0 0x7fce0c162480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.775+0000 7fce11218640 1 --2- 172.21.15.16:0/2225588438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce0c15e680 0x7fce0c0b8260 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fce0400c7a0 tx=0x7fce0400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.776+0000 7fce097fa640 1 -- 172.21.15.16:0/2225588438 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04003b70 con 0x7fce0c15e680 2024-09-19T07:50:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.776+0000 7fce097fa640 1 -- 172.21.15.16:0/2225588438 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce04003d30 con 0x7fce0c15e680 2024-09-19T07:50:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.776+0000 7fce12c7b640 1 -- 172.21.15.16:0/2225588438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0c0ca3c0 con 0x7fce0c15e680 2024-09-19T07:50:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.776+0000 7fce097fa640 1 -- 172.21.15.16:0/2225588438 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0400aa90 con 0x7fce0c15e680 2024-09-19T07:50:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.776+0000 7fce12c7b640 1 -- 172.21.15.16:0/2225588438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce0c0ca980 con 0x7fce0c15e680 2024-09-19T07:50:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.777+0000 7fce12c7b640 1 -- 172.21.15.16:0/2225588438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce0c06d420 con 0x7fce0c15e680 2024-09-19T07:50:01.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.777+0000 7fce097fa640 1 -- 172.21.15.16:0/2225588438 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fce0400ac30 con 0x7fce0c15e680 2024-09-19T07:50:01.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.778+0000 7fce097fa640 1 --2- 172.21.15.16:0/2225588438 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcdf006c2b0 0x7fcdf006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:01.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.778+0000 7fce097fa640 1 -- 172.21.15.16:0/2225588438 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7fce0408b480 con 0x7fce0c15e680 2024-09-19T07:50:01.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.778+0000 7fce10a17640 1 --2- 172.21.15.16:0/2225588438 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcdf006c2b0 0x7fcdf006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:01.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.778+0000 7fce10a17640 1 --2- 172.21.15.16:0/2225588438 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcdf006c2b0 0x7fcdf006e770 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fce00006fd0 tx=0x7fce00008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:01.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.782+0000 7fce097fa640 1 -- 172.21.15.16:0/2225588438 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fce04059af0 con 0x7fce0c15e680 2024-09-19T07:50:01.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.994+0000 7fce12c7b640 1 -- 172.21.15.16:0/2225588438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce0c06cf90 con 0x7fce0c15e680 2024-09-19T07:50:01.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.994+0000 7fce097fa640 1 -- 172.21.15.16:0/2225588438 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5222 (secure 0 0 0) 0x7fce040593f0 con 0x7fce0c15e680 2024-09-19T07:50:01.995 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:01.995 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":41,"btime":"2024-09-19T07:49:44:053013+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/945862061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":945862061},{"type":"v1","addr":"172.21.15.146:6836","nonce":945862061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/42753795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":42753795},{"type":"v1","addr":"172.21.15.16:6834","nonce":42753795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1389787480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1389787480},{"type":"v1","addr":"172.21.15.16:6835","nonce":1389787480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:49:40.403885+0000","modified":"2024-09-19T07:49:43.407266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6096},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6096":{"gid":6096,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/3642987189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3642987189},{"type":"v1","addr":"172.21.15.146:6837","nonce":3642987189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6096,"qdb_cluster":[6096]},"id":5}]} 2024-09-19T07:50:01.995 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 41 2024-09-19T07:50:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.996+0000 7fce12c7b640 1 -- 172.21.15.16:0/2225588438 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcdf006c2b0 msgr2=0x7fcdf006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.996+0000 7fce12c7b640 1 --2- 172.21.15.16:0/2225588438 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcdf006c2b0 0x7fcdf006e770 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fce00006fd0 tx=0x7fce00008040 comp rx=0 tx=0).stop 2024-09-19T07:50:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.996+0000 7fce12c7b640 1 -- 172.21.15.16:0/2225588438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce0c15e680 msgr2=0x7fce0c0b8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.996+0000 7fce12c7b640 1 --2- 172.21.15.16:0/2225588438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce0c15e680 0x7fce0c0b8260 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fce0400c7a0 tx=0x7fce0400cc60 comp rx=0 tx=0).stop 2024-09-19T07:50:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.996+0000 7fce12c7b640 1 -- 172.21.15.16:0/2225588438 shutdown_connections 2024-09-19T07:50:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.996+0000 7fce12c7b640 1 --2- 172.21.15.16:0/2225588438 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcdf006c2b0 0x7fcdf006e770 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.996+0000 7fce12c7b640 1 --2- 172.21.15.16:0/2225588438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce0c15e680 0x7fce0c0b8260 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.996+0000 7fce12c7b640 1 --2- 172.21.15.16:0/2225588438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce0c15e2a0 0x7fce0c162480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.996+0000 7fce12c7b640 1 --2- 172.21.15.16:0/2225588438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce0c06d960 0x7fce0c161f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.996+0000 7fce12c7b640 1 -- 172.21.15.16:0/2225588438 >> 172.21.15.16:0/2225588438 conn(0x7fce0c09d760 msgr2=0x7fce0c06ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.997+0000 7fce12c7b640 1 -- 172.21.15.16:0/2225588438 shutdown_connections 2024-09-19T07:50:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:01.997+0000 7fce12c7b640 1 -- 172.21.15.16:0/2225588438 wait complete. 2024-09-19T07:50:02.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:50:02.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.109+0000 7f00eb4a5640 1 Processor -- start 2024-09-19T07:50:02.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.110+0000 7f00eb4a5640 1 -- start start 2024-09-19T07:50:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.110+0000 7f00eb4a5640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f00e415f0d0 0x7f00e415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.110+0000 7f00eb4a5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f00e40d0070 0x7f00e40d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.110+0000 7f00eb4a5640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e40cf2b0 con 0x7f00e40d1940 2024-09-19T07:50:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.111+0000 7f00eb4a5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f00e40cf590 con 0x7f00e40d0070 2024-09-19T07:50:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.111+0000 7f00eb4a5640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f00e40cf870 con 0x7f00e415f0d0 2024-09-19T07:50:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.111+0000 7f00e8a40640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f00e40d0070 0x7f00e40d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.111+0000 7f00e8a40640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f00e40d0070 0x7f00e40d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45846/0 (socket says 172.21.15.16:45846) 2024-09-19T07:50:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.111+0000 7f00e8a40640 1 -- 172.21.15.16:0/3638951316 learned_addr learned my addr 172.21.15.16:0/3638951316 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.111+0000 7f00e9241640 1 --2- 172.21.15.16:0/3638951316 >> v2:172.21.15.146:3300/0 conn(0x7f00e415f0d0 0x7f00e415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.111+0000 7f00e8a40640 1 -- 172.21.15.16:0/3638951316 >> v2:172.21.15.146:3300/0 conn(0x7f00e415f0d0 msgr2=0x7f00e415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.111+0000 7f00e8a40640 1 --2- 172.21.15.16:0/3638951316 >> v2:172.21.15.146:3300/0 conn(0x7f00e415f0d0 0x7f00e415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.111+0000 7f00e8a40640 1 -- 172.21.15.16:0/3638951316 >> v1:172.21.15.146:6789/0 conn(0x7f00e40d1940 legacy=0x7f00e40d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.111+0000 7f00e8a40640 1 -- 172.21.15.16:0/3638951316 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e40cfb50 con 0x7f00e40d0070 2024-09-19T07:50:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.111+0000 7f00e9241640 1 --2- 172.21.15.16:0/3638951316 >> v2:172.21.15.146:3300/0 conn(0x7f00e415f0d0 0x7f00e415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:02.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.112+0000 7f00e8a40640 1 --2- 172.21.15.16:0/3638951316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e40d0070 0x7f00e40d1400 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f00e40b6f40 tx=0x7f00d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72a093174ddc5e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:02.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.112+0000 7f00e3fff640 1 -- 172.21.15.16:0/3638951316 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8037070 con 0x7f00e40d0070 2024-09-19T07:50:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.112+0000 7f00e3fff640 1 -- 172.21.15.16:0/3638951316 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d8032440 con 0x7f00e40d0070 2024-09-19T07:50:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.112+0000 7f00e3fff640 1 -- 172.21.15.16:0/3638951316 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8032810 con 0x7f00e40d0070 2024-09-19T07:50:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.112+0000 7f00eb4a5640 1 -- 172.21.15.16:0/3638951316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e40d0070 msgr2=0x7f00e40d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.112+0000 7f00eb4a5640 1 --2- 172.21.15.16:0/3638951316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e40d0070 0x7f00e40d1400 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f00e40b6f40 tx=0x7f00d802a1e0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.112+0000 7f00eb4a5640 1 -- 172.21.15.16:0/3638951316 shutdown_connections 2024-09-19T07:50:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.112+0000 7f00eb4a5640 1 --2- 172.21.15.16:0/3638951316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e40d0070 0x7f00e40d1400 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.112+0000 7f00eb4a5640 1 --2- 172.21.15.16:0/3638951316 >> v2:172.21.15.146:3300/0 conn(0x7f00e415f0d0 0x7f00e415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.112+0000 7f00eb4a5640 1 -- 172.21.15.16:0/3638951316 >> 172.21.15.16:0/3638951316 conn(0x7f00e409d760 msgr2=0x7f00e408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.113+0000 7f00eb4a5640 1 -- 172.21.15.16:0/3638951316 shutdown_connections 2024-09-19T07:50:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.113+0000 7f00eb4a5640 1 -- 172.21.15.16:0/3638951316 wait complete. 2024-09-19T07:50:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.114+0000 7f00eb4a5640 1 Processor -- start 2024-09-19T07:50:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.114+0000 7f00eb4a5640 1 -- start start 2024-09-19T07:50:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.114+0000 7f00eb4a5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00e40d0070 0x7f00e40b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00eb4a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e40d1940 0x7f00e40cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00eb4a5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00e415f0d0 0x7f00e4154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00eb4a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00e40d49c0 con 0x7f00e40d1940 2024-09-19T07:50:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00eb4a5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f00e40d4840 con 0x7f00e40d0070 2024-09-19T07:50:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00eb4a5640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f00e40d4b40 con 0x7f00e415f0d0 2024-09-19T07:50:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00e8a40640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e40d1940 0x7f00e40cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00e8a40640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e40d1940 0x7f00e40cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45848/0 (socket says 172.21.15.16:45848) 2024-09-19T07:50:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00e8a40640 1 -- 172.21.15.16:0/2671771038 learned_addr learned my addr 172.21.15.16:0/2671771038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00e9241640 1 --2- 172.21.15.16:0/2671771038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00e40d0070 0x7f00e40b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00e9a42640 1 --2- 172.21.15.16:0/2671771038 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00e415f0d0 0x7f00e4154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00e8a40640 1 -- 172.21.15.16:0/2671771038 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00e415f0d0 msgr2=0x7f00e4154aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00e8a40640 1 --2- 172.21.15.16:0/2671771038 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00e415f0d0 0x7f00e4154aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00e8a40640 1 -- 172.21.15.16:0/2671771038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00e40d0070 msgr2=0x7f00e40b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00e8a40640 1 --2- 172.21.15.16:0/2671771038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00e40d0070 0x7f00e40b82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00e8a40640 1 -- 172.21.15.16:0/2671771038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e40cbce0 con 0x7f00e40d1940 2024-09-19T07:50:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00e9a42640 1 --2- 172.21.15.16:0/2671771038 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00e415f0d0 0x7f00e4154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00e8a40640 1 --2- 172.21.15.16:0/2671771038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e40d1940 0x7f00e40cb4b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f00d802a220 tx=0x7f00d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00e1ffb640 1 -- 172.21.15.16:0/2671771038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8041020 con 0x7f00e40d1940 2024-09-19T07:50:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00e1ffb640 1 -- 172.21.15.16:0/2671771038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d80306c0 con 0x7f00e40d1940 2024-09-19T07:50:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00eb4a5640 1 -- 172.21.15.16:0/2671771038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e40cbfc0 con 0x7f00e40d1940 2024-09-19T07:50:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.115+0000 7f00e1ffb640 1 -- 172.21.15.16:0/2671771038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8037040 con 0x7f00e40d1940 2024-09-19T07:50:02.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.116+0000 7f00eb4a5640 1 -- 172.21.15.16:0/2671771038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e40cc870 con 0x7f00e40d1940 2024-09-19T07:50:02.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.117+0000 7f00eb4a5640 1 -- 172.21.15.16:0/2671771038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e40d1400 con 0x7f00e40d1940 2024-09-19T07:50:02.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.117+0000 7f00e1ffb640 1 -- 172.21.15.16:0/2671771038 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f00d8044020 con 0x7f00e40d1940 2024-09-19T07:50:02.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.118+0000 7f00e1ffb640 1 --2- 172.21.15.16:0/2671771038 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f00d006c2e0 0x7f00d006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.118+0000 7f00e1ffb640 1 -- 172.21.15.16:0/2671771038 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f00d80abd30 con 0x7f00e40d1940 2024-09-19T07:50:02.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.118+0000 7f00e9241640 1 --2- 172.21.15.16:0/2671771038 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f00d006c2e0 0x7f00d006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.119+0000 7f00e9241640 1 --2- 172.21.15.16:0/2671771038 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f00d006c2e0 0x7f00d006e7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f00dc006fd0 tx=0x7f00dc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:02.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.122+0000 7f00e1ffb640 1 -- 172.21.15.16:0/2671771038 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f00d804b030 con 0x7f00e40d1940 2024-09-19T07:50:02.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.299+0000 7f00eb4a5640 1 -- 172.21.15.16:0/2671771038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00e406cf30 con 0x7f00e40d1940 2024-09-19T07:50:02.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.300+0000 7f00e1ffb640 1 -- 172.21.15.16:0/2671771038 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+17029 (secure 0 0 0) 0x7f00d807a430 con 0x7f00e40d1940 2024-09-19T07:50:02.301 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:02.301 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:49:42.039882+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:49:38.349338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-19T07:49:39.377498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:50:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.302+0000 7f00eb4a5640 1 -- 172.21.15.16:0/2671771038 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f00d006c2e0 msgr2=0x7f00d006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.302+0000 7f00eb4a5640 1 --2- 172.21.15.16:0/2671771038 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f00d006c2e0 0x7f00d006e7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f00dc006fd0 tx=0x7f00dc008040 comp rx=0 tx=0).stop 2024-09-19T07:50:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.302+0000 7f00eb4a5640 1 -- 172.21.15.16:0/2671771038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e40d1940 msgr2=0x7f00e40cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.302+0000 7f00eb4a5640 1 --2- 172.21.15.16:0/2671771038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e40d1940 0x7f00e40cb4b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f00d802a220 tx=0x7f00d80046c0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.302+0000 7f00eb4a5640 1 -- 172.21.15.16:0/2671771038 shutdown_connections 2024-09-19T07:50:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.302+0000 7f00eb4a5640 1 --2- 172.21.15.16:0/2671771038 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f00d006c2e0 0x7f00d006e7a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.302+0000 7f00eb4a5640 1 --2- 172.21.15.16:0/2671771038 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00e415f0d0 0x7f00e4154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.302+0000 7f00eb4a5640 1 --2- 172.21.15.16:0/2671771038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e40d1940 0x7f00e40cb4b0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.302+0000 7f00eb4a5640 1 --2- 172.21.15.16:0/2671771038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00e40d0070 0x7f00e40b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.302+0000 7f00eb4a5640 1 -- 172.21.15.16:0/2671771038 >> 172.21.15.16:0/2671771038 conn(0x7f00e409d760 msgr2=0x7f00e415c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:02.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.302+0000 7f00eb4a5640 1 -- 172.21.15.16:0/2671771038 shutdown_connections 2024-09-19T07:50:02.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.302+0000 7f00eb4a5640 1 -- 172.21.15.16:0/2671771038 wait complete. 2024-09-19T07:50:02.315 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T07:50:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.418+0000 7f02e3cc3640 1 Processor -- start 2024-09-19T07:50:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.418+0000 7f02e3cc3640 1 -- start start 2024-09-19T07:50:02.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.419+0000 7f02e3cc3640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f02dc142ae0 0x7f02dc1692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.419+0000 7f02e3cc3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02dc1620b0 con 0x7f02dc0d21f0 2024-09-19T07:50:02.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.419+0000 7f02e3cc3640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02dc162370 con 0x7f02dc072590 2024-09-19T07:50:02.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.419+0000 7f02e3cc3640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f02dc141750 con 0x7f02dc142ae0 2024-09-19T07:50:02.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.419+0000 7f02e1a5f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f02dc0d21f0 0x7f02dc0d25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55310/0 (socket says 172.21.15.16:55310) 2024-09-19T07:50:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.419+0000 7f02e1a5f640 1 -- 172.21.15.16:0/1816599485 learned_addr learned my addr 172.21.15.16:0/1816599485 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.419+0000 7f02e2260640 1 --2- 172.21.15.16:0/1816599485 >> v2:172.21.15.146:3300/0 conn(0x7f02dc142ae0 0x7f02dc1692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.419+0000 7f02e0a5d640 1 -- 172.21.15.16:0/1816599485 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4170437273 0 0) 0x7f02dc1620b0 con 0x7f02dc0d21f0 2024-09-19T07:50:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.419+0000 7f02e0a5d640 1 -- 172.21.15.16:0/1816599485 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02c4003520 con 0x7f02dc0d21f0 2024-09-19T07:50:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e0a5d640 1 -- 172.21.15.16:0/1816599485 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2555337649 0 0) 0x7f02c4003520 con 0x7f02dc0d21f0 2024-09-19T07:50:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e0a5d640 1 -- 172.21.15.16:0/1816599485 >> v2:172.21.15.146:3300/0 conn(0x7f02dc142ae0 msgr2=0x7f02dc1692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e0a5d640 1 --2- 172.21.15.16:0/1816599485 >> v2:172.21.15.146:3300/0 conn(0x7f02dc142ae0 0x7f02dc1692c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e0a5d640 1 -- 172.21.15.16:0/1816599485 >> v1:172.21.15.146:6789/0 conn(0x7f02dc072590 legacy=0x7f02dc06d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e0a5d640 1 -- 172.21.15.16:0/1816599485 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02dc162650 con 0x7f02dc0d21f0 2024-09-19T07:50:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e0a5d640 1 -- 172.21.15.16:0/1816599485 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f02d4001670 con 0x7f02dc0d21f0 2024-09-19T07:50:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e0a5d640 1 -- 172.21.15.16:0/1816599485 >> v1:172.21.15.16:6789/0 conn(0x7f02dc0d21f0 legacy=0x7f02dc0d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e0a5d640 1 --2- 172.21.15.16:0/1816599485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c4003a00 0x7f02c4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e0a5d640 1 --2- 172.21.15.16:0/1816599485 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02c4004800 0x7f02c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e0a5d640 1 --2- 172.21.15.16:0/1816599485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02c4005210 0x7f02c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e0a5d640 1 -- 172.21.15.16:0/1816599485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc141750 con 0x7f02c4003a00 2024-09-19T07:50:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e0a5d640 1 -- 172.21.15.16:0/1816599485 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f02c400c160 con 0x7f02c4005210 2024-09-19T07:50:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e0a5d640 1 -- 172.21.15.16:0/1816599485 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f02c400c360 con 0x7f02c4004800 2024-09-19T07:50:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e1a5f640 1 --2- 172.21.15.16:0/1816599485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c4003a00 0x7f02c4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e2260640 1 --2- 172.21.15.16:0/1816599485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02c4005210 0x7f02c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e125e640 1 --2- 172.21.15.16:0/1816599485 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02c4004800 0x7f02c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e1a5f640 1 -- 172.21.15.16:0/1816599485 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02c4004800 msgr2=0x7f02c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e1a5f640 1 --2- 172.21.15.16:0/1816599485 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02c4004800 0x7f02c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e1a5f640 1 -- 172.21.15.16:0/1816599485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02c4005210 msgr2=0x7f02c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e1a5f640 1 --2- 172.21.15.16:0/1816599485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02c4005210 0x7f02c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e1a5f640 1 -- 172.21.15.16:0/1816599485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c4003520 con 0x7f02c4003a00 2024-09-19T07:50:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e2260640 1 --2- 172.21.15.16:0/1816599485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02c4005210 0x7f02c400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.420+0000 7f02e2260640 1 -- 172.21.15.16:0/1816599485 reap_dead start 2024-09-19T07:50:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.421+0000 7f02e1a5f640 1 --2- 172.21.15.16:0/1816599485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c4003a00 0x7f02c4003e20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f02d400c160 tx=0x7f02c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7868c116e9185967 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.421+0000 7f02e0a5d640 1 -- 172.21.15.16:0/1816599485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d4007c60 con 0x7f02c4003a00 2024-09-19T07:50:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.421+0000 7f02e0a5d640 1 -- 172.21.15.16:0/1816599485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d4005af0 con 0x7f02c4003a00 2024-09-19T07:50:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.421+0000 7f02e0a5d640 1 -- 172.21.15.16:0/1816599485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d4007070 con 0x7f02c4003a00 2024-09-19T07:50:02.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.422+0000 7f02e3cc3640 1 -- 172.21.15.16:0/1816599485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c4003a00 msgr2=0x7f02c4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.422+0000 7f02e3cc3640 1 --2- 172.21.15.16:0/1816599485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c4003a00 0x7f02c4003e20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f02d400c160 tx=0x7f02c4005830 comp rx=0 tx=0).stop 2024-09-19T07:50:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.422+0000 7f02e3cc3640 1 -- 172.21.15.16:0/1816599485 shutdown_connections 2024-09-19T07:50:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.422+0000 7f02e3cc3640 1 --2- 172.21.15.16:0/1816599485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c4003a00 0x7f02c4003e20 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.422+0000 7f02e3cc3640 1 -- 172.21.15.16:0/1816599485 >> 172.21.15.16:0/1816599485 conn(0x7f02dc0b3ed0 msgr2=0x7f02dc0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.422+0000 7f02e3cc3640 1 -- 172.21.15.16:0/1816599485 shutdown_connections 2024-09-19T07:50:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.422+0000 7f02e3cc3640 1 -- 172.21.15.16:0/1816599485 wait complete. 2024-09-19T07:50:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.423+0000 7f02e3cc3640 1 Processor -- start 2024-09-19T07:50:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.423+0000 7f02e3cc3640 1 -- start start 2024-09-19T07:50:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.424+0000 7f02e3cc3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02c4003a00 0x7f02dc1f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.424+0000 7f02e3cc3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02dc0b51d0 0x7f02dc0d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.424+0000 7f02e3cc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02dc15c980 0x7f02dc15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.424+0000 7f02e3cc3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc162c10 con 0x7f02dc15c980 2024-09-19T07:50:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.424+0000 7f02e3cc3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc06d7f0 con 0x7f02c4003a00 2024-09-19T07:50:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.424+0000 7f02e3cc3640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f02dc158c30 con 0x7f02dc0b51d0 2024-09-19T07:50:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.424+0000 7f02e2260640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02dc15c980 0x7f02dc15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.424+0000 7f02e2260640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02dc15c980 0x7f02dc15ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45862/0 (socket says 172.21.15.16:45862) 2024-09-19T07:50:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.424+0000 7f02e2260640 1 -- 172.21.15.16:0/90806059 learned_addr learned my addr 172.21.15.16:0/90806059 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.424+0000 7f02e125e640 1 --2- 172.21.15.16:0/90806059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02dc0b51d0 0x7f02dc0d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.424+0000 7f02e1a5f640 1 --2- 172.21.15.16:0/90806059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02c4003a00 0x7f02dc1f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.424+0000 7f02e2260640 1 -- 172.21.15.16:0/90806059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02dc0b51d0 msgr2=0x7f02dc0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.424+0000 7f02e2260640 1 --2- 172.21.15.16:0/90806059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02dc0b51d0 0x7f02dc0d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.424+0000 7f02e2260640 1 -- 172.21.15.16:0/90806059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02c4003a00 msgr2=0x7f02dc1f16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.424+0000 7f02e2260640 1 --2- 172.21.15.16:0/90806059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02c4003a00 0x7f02dc1f16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.424+0000 7f02e2260640 1 -- 172.21.15.16:0/90806059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02dc0b5740 con 0x7f02dc15c980 2024-09-19T07:50:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.424+0000 7f02e2260640 1 --2- 172.21.15.16:0/90806059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02dc15c980 0x7f02dc15ce00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f02dc06db00 tx=0x7f02cc010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:02.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.424+0000 7f02da7fc640 1 -- 172.21.15.16:0/90806059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc010da0 con 0x7f02dc15c980 2024-09-19T07:50:02.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.425+0000 7f02da7fc640 1 -- 172.21.15.16:0/90806059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02cc003990 con 0x7f02dc15c980 2024-09-19T07:50:02.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.425+0000 7f02da7fc640 1 -- 172.21.15.16:0/90806059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc003d80 con 0x7f02dc15c980 2024-09-19T07:50:02.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.425+0000 7f02e3cc3640 1 -- 172.21.15.16:0/90806059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02dc0b5a20 con 0x7f02dc15c980 2024-09-19T07:50:02.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.425+0000 7f02e3cc3640 1 -- 172.21.15.16:0/90806059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02dc0b5fe0 con 0x7f02dc15c980 2024-09-19T07:50:02.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.426+0000 7f02e3cc3640 1 -- 172.21.15.16:0/90806059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02dc068ca0 con 0x7f02dc15c980 2024-09-19T07:50:02.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.426+0000 7f02da7fc640 1 -- 172.21.15.16:0/90806059 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f02cc01e020 con 0x7f02dc15c980 2024-09-19T07:50:02.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.427+0000 7f02da7fc640 1 --2- 172.21.15.16:0/90806059 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f02c806c2e0 0x7f02c806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.427+0000 7f02da7fc640 1 -- 172.21.15.16:0/90806059 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f02cc08b9f0 con 0x7f02dc15c980 2024-09-19T07:50:02.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.427+0000 7f02e1a5f640 1 --2- 172.21.15.16:0/90806059 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f02c806c2e0 0x7f02c806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.428+0000 7f02e1a5f640 1 --2- 172.21.15.16:0/90806059 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f02c806c2e0 0x7f02c806e7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f02c4004780 tx=0x7f02d402d000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:02.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.431+0000 7f02da7fc640 1 -- 172.21.15.16:0/90806059 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f02cc059a10 con 0x7f02dc15c980 2024-09-19T07:50:02.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.643+0000 7f02e3cc3640 1 -- 172.21.15.16:0/90806059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f02dc0690c0 con 0x7f02dc15c980 2024-09-19T07:50:02.644 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-19T07:50:02.644+0000 7f6950fea640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:50:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.645+0000 7f02da7fc640 1 -- 172.21.15.16:0/90806059 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) ==== 161+0+0 (secure 0 0 0) 0x7f02cc08b020 con 0x7f02dc15c980 2024-09-19T07:50:02.646 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T07:50:02.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.648+0000 7f02e3cc3640 1 -- 172.21.15.16:0/90806059 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f02c806c2e0 msgr2=0x7f02c806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:02.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.648+0000 7f02e3cc3640 1 --2- 172.21.15.16:0/90806059 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f02c806c2e0 0x7f02c806e7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f02c4004780 tx=0x7f02d402d000 comp rx=0 tx=0).stop 2024-09-19T07:50:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.648+0000 7f02e3cc3640 1 -- 172.21.15.16:0/90806059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02dc15c980 msgr2=0x7f02dc15ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.648+0000 7f02e3cc3640 1 --2- 172.21.15.16:0/90806059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02dc15c980 0x7f02dc15ce00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f02dc06db00 tx=0x7f02cc010590 comp rx=0 tx=0).stop 2024-09-19T07:50:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.648+0000 7f02e3cc3640 1 -- 172.21.15.16:0/90806059 shutdown_connections 2024-09-19T07:50:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.648+0000 7f02e3cc3640 1 --2- 172.21.15.16:0/90806059 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f02c806c2e0 0x7f02c806e7a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.648+0000 7f02e3cc3640 1 --2- 172.21.15.16:0/90806059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02dc15c980 0x7f02dc15ce00 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.648+0000 7f02e3cc3640 1 --2- 172.21.15.16:0/90806059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02dc0b51d0 0x7f02dc0d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.648+0000 7f02e3cc3640 1 --2- 172.21.15.16:0/90806059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02c4003a00 0x7f02dc1f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.648+0000 7f02e3cc3640 1 -- 172.21.15.16:0/90806059 >> 172.21.15.16:0/90806059 conn(0x7f02dc0b3ed0 msgr2=0x7f02dc158160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.648+0000 7f02e3cc3640 1 -- 172.21.15.16:0/90806059 shutdown_connections 2024-09-19T07:50:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.648+0000 7f02e3cc3640 1 -- 172.21.15.16:0/90806059 wait complete. 2024-09-19T07:50:02.660 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T07:50:02.687 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2024-09-19T07:50:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.764+0000 7fa418845640 1 Processor -- start 2024-09-19T07:50:02.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.764+0000 7fa418845640 1 -- start start 2024-09-19T07:50:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.765+0000 7fa418845640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa41406d430 0x7fa4141439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.765+0000 7fa418845640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa414143f30 0x7fa4140d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.765+0000 7fa418845640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4140c7140 con 0x7fa4140c8d20 2024-09-19T07:50:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.765+0000 7fa418845640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa414141960 con 0x7fa41406d430 2024-09-19T07:50:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.765+0000 7fa418845640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa414142200 con 0x7fa414143f30 2024-09-19T07:50:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.765+0000 7fa411d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa41406d430 0x7fa4141439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.765+0000 7fa411d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa41406d430 0x7fa4141439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38840/0 (socket says 172.21.15.16:38840) 2024-09-19T07:50:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.765+0000 7fa411d9b640 1 -- 172.21.15.16:0/3525228017 learned_addr learned my addr 172.21.15.16:0/3525228017 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.765+0000 7fa412d9d640 1 --2- 172.21.15.16:0/3525228017 >> v2:172.21.15.146:3300/0 conn(0x7fa414143f30 0x7fa4140d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.765+0000 7fa411d9b640 1 -- 172.21.15.16:0/3525228017 >> v2:172.21.15.146:3300/0 conn(0x7fa414143f30 msgr2=0x7fa4140d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.765+0000 7fa411d9b640 1 --2- 172.21.15.16:0/3525228017 >> v2:172.21.15.146:3300/0 conn(0x7fa414143f30 0x7fa4140d1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.765+0000 7fa411d9b640 1 -- 172.21.15.16:0/3525228017 >> v1:172.21.15.146:6789/0 conn(0x7fa4140c8d20 legacy=0x7fa41406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.765+0000 7fa411d9b640 1 -- 172.21.15.16:0/3525228017 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4140c9100 con 0x7fa41406d430 2024-09-19T07:50:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.766+0000 7fa411d9b640 1 --2- 172.21.15.16:0/3525228017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa41406d430 0x7fa4141439f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa414158ed0 tx=0x7fa3fc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4e11ca6dae1d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.766+0000 7fa41159a640 1 -- 172.21.15.16:0/3525228017 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc037070 con 0x7fa41406d430 2024-09-19T07:50:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.766+0000 7fa41159a640 1 -- 172.21.15.16:0/3525228017 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3fc02ae20 con 0x7fa41406d430 2024-09-19T07:50:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.766+0000 7fa41159a640 1 -- 172.21.15.16:0/3525228017 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc032660 con 0x7fa41406d430 2024-09-19T07:50:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.767+0000 7fa418845640 1 -- 172.21.15.16:0/3525228017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa41406d430 msgr2=0x7fa4141439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.767+0000 7fa418845640 1 --2- 172.21.15.16:0/3525228017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa41406d430 0x7fa4141439f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa414158ed0 tx=0x7fa3fc02a110 comp rx=0 tx=0).stop 2024-09-19T07:50:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.767+0000 7fa418845640 1 -- 172.21.15.16:0/3525228017 shutdown_connections 2024-09-19T07:50:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.767+0000 7fa418845640 1 --2- 172.21.15.16:0/3525228017 >> v2:172.21.15.146:3300/0 conn(0x7fa414143f30 0x7fa4140d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.767+0000 7fa418845640 1 --2- 172.21.15.16:0/3525228017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa41406d430 0x7fa4141439f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.767+0000 7fa418845640 1 -- 172.21.15.16:0/3525228017 >> 172.21.15.16:0/3525228017 conn(0x7fa4140b3ed0 msgr2=0x7fa4140b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:02.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.767+0000 7fa418845640 1 -- 172.21.15.16:0/3525228017 shutdown_connections 2024-09-19T07:50:02.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.767+0000 7fa418845640 1 -- 172.21.15.16:0/3525228017 wait complete. 2024-09-19T07:50:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.768+0000 7fa418845640 1 Processor -- start 2024-09-19T07:50:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.768+0000 7fa418845640 1 -- start start 2024-09-19T07:50:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.769+0000 7fa418845640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa41406d430 0x7fa4141f8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.769+0000 7fa418845640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4140c8d20 0x7fa4140b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.769+0000 7fa418845640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa414143f30 0x7fa414140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.769+0000 7fa418845640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa414141e90 con 0x7fa414143f30 2024-09-19T07:50:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.769+0000 7fa418845640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa414141bb0 con 0x7fa4140c8d20 2024-09-19T07:50:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.769+0000 7fa418845640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa414144380 con 0x7fa41406d430 2024-09-19T07:50:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.769+0000 7fa412d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa414143f30 0x7fa414140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.769+0000 7fa412d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa414143f30 0x7fa414140380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38844/0 (socket says 172.21.15.16:38844) 2024-09-19T07:50:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.769+0000 7fa412d9d640 1 -- 172.21.15.16:0/3379756551 learned_addr learned my addr 172.21.15.16:0/3379756551 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.769+0000 7fa411d9b640 1 --2- 172.21.15.16:0/3379756551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4140c8d20 0x7fa4140b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.769+0000 7fa41259c640 1 --2- 172.21.15.16:0/3379756551 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa41406d430 0x7fa4141f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.770+0000 7fa412d9d640 1 -- 172.21.15.16:0/3379756551 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa41406d430 msgr2=0x7fa4141f8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.770+0000 7fa412d9d640 1 --2- 172.21.15.16:0/3379756551 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa41406d430 0x7fa4141f8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.770+0000 7fa412d9d640 1 -- 172.21.15.16:0/3379756551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4140c8d20 msgr2=0x7fa4140b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.770+0000 7fa412d9d640 1 --2- 172.21.15.16:0/3379756551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4140c8d20 0x7fa4140b60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.770+0000 7fa412d9d640 1 -- 172.21.15.16:0/3379756551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4140cf830 con 0x7fa414143f30 2024-09-19T07:50:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.770+0000 7fa412d9d640 1 --2- 172.21.15.16:0/3379756551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa414143f30 0x7fa414140380 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00c7f0 tx=0x7fa40c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.770+0000 7fa40affd640 1 -- 172.21.15.16:0/3379756551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c015070 con 0x7fa414143f30 2024-09-19T07:50:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.770+0000 7fa40affd640 1 -- 172.21.15.16:0/3379756551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40c004220 con 0x7fa414143f30 2024-09-19T07:50:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.770+0000 7fa418845640 1 -- 172.21.15.16:0/3379756551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4140cf270 con 0x7fa414143f30 2024-09-19T07:50:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.770+0000 7fa40affd640 1 -- 172.21.15.16:0/3379756551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c007620 con 0x7fa414143f30 2024-09-19T07:50:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.770+0000 7fa418845640 1 -- 172.21.15.16:0/3379756551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4141f7d20 con 0x7fa414143f30 2024-09-19T07:50:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.771+0000 7fa418845640 1 -- 172.21.15.16:0/3379756551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4141439f0 con 0x7fa414143f30 2024-09-19T07:50:02.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.772+0000 7fa40affd640 1 -- 172.21.15.16:0/3379756551 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa40c0077c0 con 0x7fa414143f30 2024-09-19T07:50:02.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.772+0000 7fa40affd640 1 --2- 172.21.15.16:0/3379756551 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa3f406c2b0 0x7fa3f406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:02.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.772+0000 7fa40affd640 1 -- 172.21.15.16:0/3379756551 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7fa40c08b3c0 con 0x7fa414143f30 2024-09-19T07:50:02.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.772+0000 7fa41259c640 1 --2- 172.21.15.16:0/3379756551 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa3f406c2b0 0x7fa3f406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:02.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.773+0000 7fa41259c640 1 --2- 172.21.15.16:0/3379756551 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa3f406c2b0 0x7fa3f406e770 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa4140cefe0 tx=0x7fa40400d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:02.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.776+0000 7fa40affd640 1 -- 172.21.15.16:0/3379756551 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa40c059300 con 0x7fa414143f30 2024-09-19T07:50:02.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:02.991+0000 7fa418845640 1 -- 172.21.15.16:0/3379756551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa4140690c0 con 0x7fa414143f30 2024-09-19T07:50:03.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.647+0000 7fa40affd640 1 -- 172.21.15.16:0/3379756551 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) ==== 104+0+0 (secure 0 0 0) 0x7fa40c05d600 con 0x7fa414143f30 2024-09-19T07:50:03.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.650+0000 7fa418845640 1 -- 172.21.15.16:0/3379756551 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa3f406c2b0 msgr2=0x7fa3f406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:03.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.650+0000 7fa418845640 1 --2- 172.21.15.16:0/3379756551 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa3f406c2b0 0x7fa3f406e770 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa4140cefe0 tx=0x7fa40400d040 comp rx=0 tx=0).stop 2024-09-19T07:50:03.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.650+0000 7fa418845640 1 -- 172.21.15.16:0/3379756551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa414143f30 msgr2=0x7fa414140380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:03.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.650+0000 7fa418845640 1 --2- 172.21.15.16:0/3379756551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa414143f30 0x7fa414140380 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00c7f0 tx=0x7fa40c00ccb0 comp rx=0 tx=0).stop 2024-09-19T07:50:03.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.650+0000 7fa418845640 1 -- 172.21.15.16:0/3379756551 shutdown_connections 2024-09-19T07:50:03.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.650+0000 7fa418845640 1 --2- 172.21.15.16:0/3379756551 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa3f406c2b0 0x7fa3f406e770 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:03.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.650+0000 7fa418845640 1 --2- 172.21.15.16:0/3379756551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa414143f30 0x7fa414140380 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:03.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.650+0000 7fa418845640 1 --2- 172.21.15.16:0/3379756551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4140c8d20 0x7fa4140b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:03.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.650+0000 7fa418845640 1 --2- 172.21.15.16:0/3379756551 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa41406d430 0x7fa4141f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:03.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.650+0000 7fa418845640 1 -- 172.21.15.16:0/3379756551 >> 172.21.15.16:0/3379756551 conn(0x7fa4140b3ed0 msgr2=0x7fa4140b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:03.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.650+0000 7fa418845640 1 -- 172.21.15.16:0/3379756551 shutdown_connections 2024-09-19T07:50:03.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.650+0000 7fa418845640 1 -- 172.21.15.16:0/3379756551 wait complete. 2024-09-19T07:50:03.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T07:50:03.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.776+0000 7f334bfe8640 1 Processor -- start 2024-09-19T07:50:03.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.776+0000 7f334bfe8640 1 -- start start 2024-09-19T07:50:03.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.777+0000 7f334bfe8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f334415e2a0 0x7f334406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.777+0000 7f334bfe8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f334406d960 0x7f33440d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.777+0000 7f334bfe8640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33440d1880 con 0x7f334415e680 2024-09-19T07:50:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.777+0000 7f334bfe8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f33440d1b60 con 0x7f334415e2a0 2024-09-19T07:50:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.777+0000 7f334bfe8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f33440cf2d0 con 0x7f334406d960 2024-09-19T07:50:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.777+0000 7f3349d84640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f334415e2a0 0x7f334406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.777+0000 7f3349d84640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f334415e2a0 0x7f334406d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38846/0 (socket says 172.21.15.16:38846) 2024-09-19T07:50:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.777+0000 7f3349d84640 1 -- 172.21.15.16:0/3597836149 learned_addr learned my addr 172.21.15.16:0/3597836149 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.777+0000 7f3349d84640 1 -- 172.21.15.16:0/3597836149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f334406d960 msgr2=0x7f33440d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.777+0000 7f3349d84640 1 --2- 172.21.15.16:0/3597836149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f334406d960 0x7f33440d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.777+0000 7f3349d84640 1 -- 172.21.15.16:0/3597836149 >> v1:172.21.15.146:6789/0 conn(0x7f334415e680 legacy=0x7f33440cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.777+0000 7f3349d84640 1 -- 172.21.15.16:0/3597836149 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33440cf5b0 con 0x7f334415e2a0 2024-09-19T07:50:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.778+0000 7f3349d84640 1 --2- 172.21.15.16:0/3597836149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334415e2a0 0x7f334406d420 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3340009e50 tx=0x7f334002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3dbc546f37e27cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.778+0000 7f3348d82640 1 -- 172.21.15.16:0/3597836149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3340036070 con 0x7f334415e2a0 2024-09-19T07:50:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.778+0000 7f3348d82640 1 -- 172.21.15.16:0/3597836149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3340031600 con 0x7f334415e2a0 2024-09-19T07:50:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.778+0000 7f3348d82640 1 -- 172.21.15.16:0/3597836149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33400319d0 con 0x7f334415e2a0 2024-09-19T07:50:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.779+0000 7f334bfe8640 1 -- 172.21.15.16:0/3597836149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334415e2a0 msgr2=0x7f334406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.779+0000 7f334bfe8640 1 --2- 172.21.15.16:0/3597836149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334415e2a0 0x7f334406d420 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3340009e50 tx=0x7f334002a6c0 comp rx=0 tx=0).stop 2024-09-19T07:50:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.779+0000 7f334bfe8640 1 -- 172.21.15.16:0/3597836149 shutdown_connections 2024-09-19T07:50:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.779+0000 7f334bfe8640 1 --2- 172.21.15.16:0/3597836149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f334406d960 0x7f33440d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.779+0000 7f334bfe8640 1 --2- 172.21.15.16:0/3597836149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334415e2a0 0x7f334406d420 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.779+0000 7f334bfe8640 1 -- 172.21.15.16:0/3597836149 >> 172.21.15.16:0/3597836149 conn(0x7f334409d760 msgr2=0x7f334408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.779+0000 7f334bfe8640 1 -- 172.21.15.16:0/3597836149 shutdown_connections 2024-09-19T07:50:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.779+0000 7f334bfe8640 1 -- 172.21.15.16:0/3597836149 wait complete. 2024-09-19T07:50:03.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.780+0000 7f334bfe8640 1 Processor -- start 2024-09-19T07:50:03.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.780+0000 7f334bfe8640 1 -- start start 2024-09-19T07:50:03.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.781+0000 7f334bfe8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f334406d960 0x7f334415ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:03.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.781+0000 7f334bfe8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334415e2a0 0x7f334415c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:03.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.781+0000 7f334bfe8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f334415e680 0x7f33440c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:03.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.781+0000 7f334bfe8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33440d0360 con 0x7f334415e2a0 2024-09-19T07:50:03.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.781+0000 7f334bfe8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f33440d01e0 con 0x7f334415e680 2024-09-19T07:50:03.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.781+0000 7f334bfe8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f33440d48b0 con 0x7f334406d960 2024-09-19T07:50:03.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.781+0000 7f3349583640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334415e2a0 0x7f334415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:03.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.781+0000 7f3349583640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334415e2a0 0x7f334415c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38860/0 (socket says 172.21.15.16:38860) 2024-09-19T07:50:03.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.781+0000 7f3349583640 1 -- 172.21.15.16:0/3275427955 learned_addr learned my addr 172.21.15.16:0/3275427955 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:03.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.781+0000 7f334a585640 1 --2- 172.21.15.16:0/3275427955 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f334415e680 0x7f33440c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:03.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.781+0000 7f3349d84640 1 --2- 172.21.15.16:0/3275427955 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f334406d960 0x7f334415ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:03.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.781+0000 7f3349583640 1 -- 172.21.15.16:0/3275427955 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f334406d960 msgr2=0x7f334415ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:03.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.781+0000 7f3349583640 1 --2- 172.21.15.16:0/3275427955 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f334406d960 0x7f334415ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:03.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.781+0000 7f3349583640 1 -- 172.21.15.16:0/3275427955 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f334415e680 msgr2=0x7f33440c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:03.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.781+0000 7f3349583640 1 --2- 172.21.15.16:0/3275427955 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f334415e680 0x7f33440c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:03.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.781+0000 7f3349583640 1 -- 172.21.15.16:0/3275427955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33440ce4a0 con 0x7f334415e2a0 2024-09-19T07:50:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.782+0000 7f3349583640 1 --2- 172.21.15.16:0/3275427955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334415e2a0 0x7f334415c280 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f333400b3c0 tx=0x7f333400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.782+0000 7f333a7fc640 1 -- 172.21.15.16:0/3275427955 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334011830 con 0x7f334415e2a0 2024-09-19T07:50:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.782+0000 7f333a7fc640 1 -- 172.21.15.16:0/3275427955 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3334016070 con 0x7f334415e2a0 2024-09-19T07:50:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.782+0000 7f334bfe8640 1 -- 172.21.15.16:0/3275427955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33440c8560 con 0x7f334415e2a0 2024-09-19T07:50:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.782+0000 7f333a7fc640 1 -- 172.21.15.16:0/3275427955 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334011c30 con 0x7f334415e2a0 2024-09-19T07:50:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.782+0000 7f334bfe8640 1 -- 172.21.15.16:0/3275427955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f334415c7c0 con 0x7f334415e2a0 2024-09-19T07:50:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.783+0000 7f334bfe8640 1 -- 172.21.15.16:0/3275427955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33440d4c90 con 0x7f334415e2a0 2024-09-19T07:50:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.783+0000 7f333a7fc640 1 -- 172.21.15.16:0/3275427955 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f333401b460 con 0x7f334415e2a0 2024-09-19T07:50:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.784+0000 7f333a7fc640 1 --2- 172.21.15.16:0/3275427955 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f332806c2b0 0x7f332806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.784+0000 7f333a7fc640 1 -- 172.21.15.16:0/3275427955 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6187+0+0 (secure 0 0 0) 0x7f333408b170 con 0x7f334415e2a0 2024-09-19T07:50:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.784+0000 7f3349d84640 1 --2- 172.21.15.16:0/3275427955 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f332806c2b0 0x7f332806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.785+0000 7f3349d84640 1 --2- 172.21.15.16:0/3275427955 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f332806c2b0 0x7f332806e770 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3340033000 tx=0x7f3340009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:03.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.788+0000 7f333a7fc640 1 -- 172.21.15.16:0/3275427955 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3334059140 con 0x7f334415e2a0 2024-09-19T07:50:03.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:03.964+0000 7f334bfe8640 1 -- 172.21.15.16:0/3275427955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f334406cf90 con 0x7f334415e2a0 2024-09-19T07:50:04.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.648+0000 7f333a7fc640 1 -- 172.21.15.16:0/3275427955 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v83) ==== 181+0+0 (secure 0 0 0) 0x7f333405d440 con 0x7f334415e2a0 2024-09-19T07:50:04.649 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-19T07:50:04.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.652+0000 7f334bfe8640 1 -- 172.21.15.16:0/3275427955 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f332806c2b0 msgr2=0x7f332806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:04.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.652+0000 7f334bfe8640 1 --2- 172.21.15.16:0/3275427955 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f332806c2b0 0x7f332806e770 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3340033000 tx=0x7f3340009000 comp rx=0 tx=0).stop 2024-09-19T07:50:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.652+0000 7f334bfe8640 1 -- 172.21.15.16:0/3275427955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334415e2a0 msgr2=0x7f334415c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.652+0000 7f334bfe8640 1 --2- 172.21.15.16:0/3275427955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334415e2a0 0x7f334415c280 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f333400b3c0 tx=0x7f333400b880 comp rx=0 tx=0).stop 2024-09-19T07:50:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.652+0000 7f334bfe8640 1 -- 172.21.15.16:0/3275427955 shutdown_connections 2024-09-19T07:50:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.652+0000 7f334bfe8640 1 --2- 172.21.15.16:0/3275427955 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f332806c2b0 0x7f332806e770 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.652+0000 7f334bfe8640 1 --2- 172.21.15.16:0/3275427955 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f334415e680 0x7f33440c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.652+0000 7f334bfe8640 1 --2- 172.21.15.16:0/3275427955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334415e2a0 0x7f334415c280 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.652+0000 7f334bfe8640 1 --2- 172.21.15.16:0/3275427955 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f334406d960 0x7f334415ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.652+0000 7f334bfe8640 1 -- 172.21.15.16:0/3275427955 >> 172.21.15.16:0/3275427955 conn(0x7f334409d760 msgr2=0x7f33440d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.652+0000 7f334bfe8640 1 -- 172.21.15.16:0/3275427955 shutdown_connections 2024-09-19T07:50:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.652+0000 7f334bfe8640 1 -- 172.21.15.16:0/3275427955 wait complete. 2024-09-19T07:50:04.667 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T07:50:04.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.778+0000 7f117bb31640 1 Processor -- start 2024-09-19T07:50:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.778+0000 7f117bb31640 1 -- start start 2024-09-19T07:50:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.779+0000 7f117bb31640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f117415e2a0 0x7f117406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.779+0000 7f117bb31640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11740d1880 con 0x7f117415e680 2024-09-19T07:50:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.779+0000 7f117bb31640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11740d1b60 con 0x7f117406d960 2024-09-19T07:50:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.779+0000 7f117bb31640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f11740cf2d0 con 0x7f117415e2a0 2024-09-19T07:50:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.779+0000 7f11798cd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f117415e2a0 0x7f117406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.779+0000 7f11798cd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f117415e2a0 0x7f117406d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38864/0 (socket says 172.21.15.16:38864) 2024-09-19T07:50:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.779+0000 7f11798cd640 1 -- 172.21.15.16:0/858354717 learned_addr learned my addr 172.21.15.16:0/858354717 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.780+0000 7f11798cd640 1 -- 172.21.15.16:0/858354717 >> v1:172.21.15.146:6789/0 conn(0x7f117406d960 legacy=0x7f11740d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.780+0000 7f11798cd640 1 -- 172.21.15.16:0/858354717 >> v1:172.21.15.16:6789/0 conn(0x7f117415e680 legacy=0x7f11740cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.780+0000 7f11798cd640 1 -- 172.21.15.16:0/858354717 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11740cf5b0 con 0x7f117415e2a0 2024-09-19T07:50:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.780+0000 7f11788cb640 1 -- 172.21.15.16:0/858354717 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4201406319 0 0) 0x7f11740d1880 con 0x7f117415e680 2024-09-19T07:50:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.780+0000 7f11798cd640 1 --2- 172.21.15.16:0/858354717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117415e2a0 0x7f117406d420 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1170009870 tx=0x7f117002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7d416dd2b0e7cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.780+0000 7f11788cb640 1 -- 172.21.15.16:0/858354717 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170037070 con 0x7f117415e2a0 2024-09-19T07:50:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.781+0000 7f11788cb640 1 -- 172.21.15.16:0/858354717 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f117002ae50 con 0x7f117415e2a0 2024-09-19T07:50:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.781+0000 7f11788cb640 1 -- 172.21.15.16:0/858354717 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170032860 con 0x7f117415e2a0 2024-09-19T07:50:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.781+0000 7f117bb31640 1 -- 172.21.15.16:0/858354717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117415e2a0 msgr2=0x7f117406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.781+0000 7f117bb31640 1 --2- 172.21.15.16:0/858354717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117415e2a0 0x7f117406d420 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1170009870 tx=0x7f117002a300 comp rx=0 tx=0).stop 2024-09-19T07:50:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.781+0000 7f117bb31640 1 -- 172.21.15.16:0/858354717 shutdown_connections 2024-09-19T07:50:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.781+0000 7f117bb31640 1 --2- 172.21.15.16:0/858354717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117415e2a0 0x7f117406d420 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.781+0000 7f117bb31640 1 -- 172.21.15.16:0/858354717 >> 172.21.15.16:0/858354717 conn(0x7f117409d760 msgr2=0x7f117408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.781+0000 7f117bb31640 1 -- 172.21.15.16:0/858354717 shutdown_connections 2024-09-19T07:50:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.781+0000 7f117bb31640 1 -- 172.21.15.16:0/858354717 wait complete. 2024-09-19T07:50:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.782+0000 7f117bb31640 1 Processor -- start 2024-09-19T07:50:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.782+0000 7f117bb31640 1 -- start start 2024-09-19T07:50:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.783+0000 7f117bb31640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f117406d960 0x7f1174156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.783+0000 7f117bb31640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f117415e2a0 0x7f11741ea630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.783+0000 7f117bb31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117415e680 0x7f11740ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.783+0000 7f117bb31640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11740d0360 con 0x7f117415e680 2024-09-19T07:50:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.783+0000 7f117bb31640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f11740d01e0 con 0x7f117406d960 2024-09-19T07:50:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.783+0000 7f117bb31640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f11740d4a80 con 0x7f117415e2a0 2024-09-19T07:50:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.783+0000 7f117a0ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117415e680 0x7f11740ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.783+0000 7f117a0ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117415e680 0x7f11740ccb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38880/0 (socket says 172.21.15.16:38880) 2024-09-19T07:50:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.783+0000 7f117a0ce640 1 -- 172.21.15.16:0/3389876433 learned_addr learned my addr 172.21.15.16:0/3389876433 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.783+0000 7f11798cd640 1 --2- 172.21.15.16:0/3389876433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f117406d960 0x7f1174156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.783+0000 7f11790cc640 1 --2- 172.21.15.16:0/3389876433 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f117415e2a0 0x7f11741ea630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.784+0000 7f117a0ce640 1 -- 172.21.15.16:0/3389876433 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f117415e2a0 msgr2=0x7f11741ea630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.784+0000 7f117a0ce640 1 --2- 172.21.15.16:0/3389876433 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f117415e2a0 0x7f11741ea630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.784+0000 7f117a0ce640 1 -- 172.21.15.16:0/3389876433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f117406d960 msgr2=0x7f1174156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.784+0000 7f117a0ce640 1 --2- 172.21.15.16:0/3389876433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f117406d960 0x7f1174156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.784+0000 7f117a0ce640 1 -- 172.21.15.16:0/3389876433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1174156cb0 con 0x7f117415e680 2024-09-19T07:50:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.784+0000 7f117a0ce640 1 --2- 172.21.15.16:0/3389876433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117415e680 0x7f11740ccb20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f116c00c7d0 tx=0x7f116c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.784+0000 7f116a7fc640 1 -- 172.21.15.16:0/3389876433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c003b70 con 0x7f117415e680 2024-09-19T07:50:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.784+0000 7f116a7fc640 1 -- 172.21.15.16:0/3389876433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f116c003d30 con 0x7f117415e680 2024-09-19T07:50:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.784+0000 7f116a7fc640 1 -- 172.21.15.16:0/3389876433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c00aa90 con 0x7f117415e680 2024-09-19T07:50:04.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.784+0000 7f117bb31640 1 -- 172.21.15.16:0/3389876433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11741e98b0 con 0x7f117415e680 2024-09-19T07:50:04.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.784+0000 7f117bb31640 1 -- 172.21.15.16:0/3389876433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11741e9e30 con 0x7f117415e680 2024-09-19T07:50:04.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.785+0000 7f117bb31640 1 -- 172.21.15.16:0/3389876433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f117406d420 con 0x7f117415e680 2024-09-19T07:50:04.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.786+0000 7f116a7fc640 1 -- 172.21.15.16:0/3389876433 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f116c00ac30 con 0x7f117415e680 2024-09-19T07:50:04.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.786+0000 7f116a7fc640 1 --2- 172.21.15.16:0/3389876433 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f115806c2b0 0x7f115806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:04.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.786+0000 7f116a7fc640 1 -- 172.21.15.16:0/3389876433 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5749+0+0 (secure 0 0 0) 0x7f116c08b1c0 con 0x7f117415e680 2024-09-19T07:50:04.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.787+0000 7f11798cd640 1 --2- 172.21.15.16:0/3389876433 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f115806c2b0 0x7f115806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:04.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.787+0000 7f11798cd640 1 --2- 172.21.15.16:0/3389876433 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f115806c2b0 0x7f115806e770 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1170034000 tx=0x7f117002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.790+0000 7f116a7fc640 1 -- 172.21.15.16:0/3389876433 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f116c059340 con 0x7f117415e680 2024-09-19T07:50:04.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:04.966+0000 7f117bb31640 1 -- 172.21.15.16:0/3389876433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f117406cf90 con 0x7f117415e680 2024-09-19T07:50:05.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.650+0000 7f116a7fc640 1 -- 172.21.15.16:0/3389876433 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v84) ==== 169+0+0 (secure 0 0 0) 0x7f116c05d640 con 0x7f117415e680 2024-09-19T07:50:05.650 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-19T07:50:05.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.655+0000 7f117bb31640 1 -- 172.21.15.16:0/3389876433 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f115806c2b0 msgr2=0x7f115806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:05.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.655+0000 7f117bb31640 1 --2- 172.21.15.16:0/3389876433 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f115806c2b0 0x7f115806e770 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1170034000 tx=0x7f117002e000 comp rx=0 tx=0).stop 2024-09-19T07:50:05.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.655+0000 7f117bb31640 1 -- 172.21.15.16:0/3389876433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117415e680 msgr2=0x7f11740ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:05.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.655+0000 7f117bb31640 1 --2- 172.21.15.16:0/3389876433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117415e680 0x7f11740ccb20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f116c00c7d0 tx=0x7f116c00cc90 comp rx=0 tx=0).stop 2024-09-19T07:50:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.657+0000 7f117bb31640 1 -- 172.21.15.16:0/3389876433 shutdown_connections 2024-09-19T07:50:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.657+0000 7f117bb31640 1 --2- 172.21.15.16:0/3389876433 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f115806c2b0 0x7f115806e770 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.657+0000 7f117bb31640 1 --2- 172.21.15.16:0/3389876433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117415e680 0x7f11740ccb20 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.657+0000 7f117bb31640 1 --2- 172.21.15.16:0/3389876433 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f117415e2a0 0x7f11741ea630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.657+0000 7f117bb31640 1 --2- 172.21.15.16:0/3389876433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f117406d960 0x7f1174156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.657+0000 7f117bb31640 1 -- 172.21.15.16:0/3389876433 >> 172.21.15.16:0/3389876433 conn(0x7f117409d760 msgr2=0x7f11740d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.659+0000 7f117bb31640 1 -- 172.21.15.16:0/3389876433 shutdown_connections 2024-09-19T07:50:05.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.659+0000 7f117bb31640 1 -- 172.21.15.16:0/3389876433 wait complete. 2024-09-19T07:50:05.678 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_max_caps_per_client 2024-09-19T07:50:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.786+0000 7fed05152640 1 Processor -- start 2024-09-19T07:50:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.786+0000 7fed05152640 1 -- start start 2024-09-19T07:50:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.787+0000 7fed05152640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed0015e2a0 0x7fed0006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.787+0000 7fed05152640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fed0006d960 0x7fed000d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.787+0000 7fed05152640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed000d1880 con 0x7fed0015e680 2024-09-19T07:50:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.787+0000 7fed05152640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fed000d1b60 con 0x7fed0006d960 2024-09-19T07:50:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.787+0000 7fed05152640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fed000cf2d0 con 0x7fed0015e2a0 2024-09-19T07:50:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.787+0000 7fecff59e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fed0006d960 0x7fed000d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.787+0000 7fecfed9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed0015e2a0 0x7fed0006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.787+0000 7fecfe59c640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fed0015e680 0x7fed000cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:37716/0 (socket says 172.21.15.16:37716) 2024-09-19T07:50:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.787+0000 7fecfe59c640 1 -- 172.21.15.16:0/442586299 learned_addr learned my addr 172.21.15.16:0/442586299 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.788+0000 7fecff59e640 1 -- 172.21.15.16:0/442586299 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed0015e2a0 msgr2=0x7fed0006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.788+0000 7fecff59e640 1 --2- 172.21.15.16:0/442586299 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed0015e2a0 0x7fed0006d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.788+0000 7fecff59e640 1 -- 172.21.15.16:0/442586299 >> v1:172.21.15.146:6789/0 conn(0x7fed0015e680 legacy=0x7fed000cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.788+0000 7fecff59e640 1 -- 172.21.15.16:0/442586299 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed000cf5b0 con 0x7fed0006d960 2024-09-19T07:50:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.788+0000 7fecfed9d640 1 --2- 172.21.15.16:0/442586299 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed0015e2a0 0x7fed0006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.789+0000 7fecff59e640 1 --2- 172.21.15.16:0/442586299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed0006d960 0x7fed000d4370 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009d90 tx=0x7fecf802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f34c1c4e46929ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.789+0000 7fecfdd9b640 1 -- 172.21.15.16:0/442586299 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8007740 con 0x7fed0006d960 2024-09-19T07:50:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.789+0000 7fecfdd9b640 1 -- 172.21.15.16:0/442586299 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecf80078e0 con 0x7fed0006d960 2024-09-19T07:50:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.789+0000 7fecfdd9b640 1 -- 172.21.15.16:0/442586299 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8007cb0 con 0x7fed0006d960 2024-09-19T07:50:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.789+0000 7fed05152640 1 -- 172.21.15.16:0/442586299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed0006d960 msgr2=0x7fed000d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.789+0000 7fed05152640 1 --2- 172.21.15.16:0/442586299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed0006d960 0x7fed000d4370 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009d90 tx=0x7fecf802a660 comp rx=0 tx=0).stop 2024-09-19T07:50:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.789+0000 7fed05152640 1 -- 172.21.15.16:0/442586299 shutdown_connections 2024-09-19T07:50:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.789+0000 7fed05152640 1 --2- 172.21.15.16:0/442586299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed0006d960 0x7fed000d4370 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.789+0000 7fed05152640 1 --2- 172.21.15.16:0/442586299 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed0015e2a0 0x7fed0006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.789+0000 7fed05152640 1 -- 172.21.15.16:0/442586299 >> 172.21.15.16:0/442586299 conn(0x7fed0009d760 msgr2=0x7fed0008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.789+0000 7fed05152640 1 -- 172.21.15.16:0/442586299 shutdown_connections 2024-09-19T07:50:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.790+0000 7fed05152640 1 -- 172.21.15.16:0/442586299 wait complete. 2024-09-19T07:50:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.791+0000 7fed05152640 1 Processor -- start 2024-09-19T07:50:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.791+0000 7fed05152640 1 -- start start 2024-09-19T07:50:05.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.792+0000 7fed05152640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed0006d960 0x7fed000b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:05.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.792+0000 7fed05152640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed0015e2a0 0x7fed000b9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:05.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.792+0000 7fed05152640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed0015e680 0x7fed000b8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:05.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.792+0000 7fed05152640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed000d0360 con 0x7fed0006d960 2024-09-19T07:50:05.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.792+0000 7fed05152640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fed000d01e0 con 0x7fed0015e2a0 2024-09-19T07:50:05.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.792+0000 7fed05152640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fed000d48b0 con 0x7fed0015e680 2024-09-19T07:50:05.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.792+0000 7fecfed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed0006d960 0x7fed000b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:05.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.792+0000 7fecfed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed0006d960 0x7fed000b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38886/0 (socket says 172.21.15.16:38886) 2024-09-19T07:50:05.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.792+0000 7fecfed9d640 1 -- 172.21.15.16:0/594800798 learned_addr learned my addr 172.21.15.16:0/594800798 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:05.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.792+0000 7fecff59e640 1 --2- 172.21.15.16:0/594800798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed0015e680 0x7fed000b8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:05.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.792+0000 7fecfe59c640 1 --2- 172.21.15.16:0/594800798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed0015e2a0 0x7fed000b9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:05.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.792+0000 7fecfed9d640 1 -- 172.21.15.16:0/594800798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed0015e680 msgr2=0x7fed000b8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:05.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.792+0000 7fecfed9d640 1 --2- 172.21.15.16:0/594800798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed0015e680 0x7fed000b8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:05.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.792+0000 7fecfed9d640 1 -- 172.21.15.16:0/594800798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed0015e2a0 msgr2=0x7fed000b9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:05.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.792+0000 7fecfed9d640 1 --2- 172.21.15.16:0/594800798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed0015e2a0 0x7fed000b9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:05.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.792+0000 7fecfed9d640 1 -- 172.21.15.16:0/594800798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed001f5570 con 0x7fed0006d960 2024-09-19T07:50:05.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.792+0000 7fecff59e640 1 --2- 172.21.15.16:0/594800798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed0015e680 0x7fed000b8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:05.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.792+0000 7fecfed9d640 1 --2- 172.21.15.16:0/594800798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed0006d960 0x7fed000b5670 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fecf000ebe0 tx=0x7fecf000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:05.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.793+0000 7fece77fe640 1 -- 172.21.15.16:0/594800798 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0017070 con 0x7fed0006d960 2024-09-19T07:50:05.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.793+0000 7fece77fe640 1 -- 172.21.15.16:0/594800798 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecf000cc30 con 0x7fed0006d960 2024-09-19T07:50:05.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.793+0000 7fed05152640 1 -- 172.21.15.16:0/594800798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed001f5850 con 0x7fed0006d960 2024-09-19T07:50:05.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.793+0000 7fece77fe640 1 -- 172.21.15.16:0/594800798 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0013470 con 0x7fed0006d960 2024-09-19T07:50:05.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.793+0000 7fed05152640 1 -- 172.21.15.16:0/594800798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed00158070 con 0x7fed0006d960 2024-09-19T07:50:05.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.794+0000 7fed05152640 1 -- 172.21.15.16:0/594800798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed000d0000 con 0x7fed0006d960 2024-09-19T07:50:05.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.794+0000 7fece77fe640 1 -- 172.21.15.16:0/594800798 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fecf001a070 con 0x7fed0006d960 2024-09-19T07:50:05.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.795+0000 7fece77fe640 1 --2- 172.21.15.16:0/594800798 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fecec06c2e0 0x7fecec06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:05.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.795+0000 7fece77fe640 1 -- 172.21.15.16:0/594800798 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5335+0+0 (secure 0 0 0) 0x7fecf008b560 con 0x7fed0006d960 2024-09-19T07:50:05.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.795+0000 7fecfe59c640 1 --2- 172.21.15.16:0/594800798 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fecec06c2e0 0x7fecec06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:05.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.795+0000 7fecfe59c640 1 --2- 172.21.15.16:0/594800798 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fecec06c2e0 0x7fecec06e7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fed001f2540 tx=0x7fecf4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:05.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.799+0000 7fece77fe640 1 -- 172.21.15.16:0/594800798 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fed000d0000 con 0x7fed0006d960 2024-09-19T07:50:05.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.977+0000 7fed05152640 1 -- 172.21.15.16:0/594800798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"} v 0) -- 0x7fed000d4370 con 0x7fed0006d960 2024-09-19T07:50:05.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.981+0000 7fece77fe640 1 -- 172.21.15.16:0/594800798 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"}]=0 v29) ==== 106+0+0 (secure 0 0 0) 0x7fecf0059e40 con 0x7fed0006d960 2024-09-19T07:50:05.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.983+0000 7fed05152640 1 -- 172.21.15.16:0/594800798 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fecec06c2e0 msgr2=0x7fecec06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:05.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.983+0000 7fed05152640 1 --2- 172.21.15.16:0/594800798 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fecec06c2e0 0x7fecec06e7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fed001f2540 tx=0x7fecf4009040 comp rx=0 tx=0).stop 2024-09-19T07:50:05.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.983+0000 7fed05152640 1 -- 172.21.15.16:0/594800798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed0006d960 msgr2=0x7fed000b5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:05.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.983+0000 7fed05152640 1 --2- 172.21.15.16:0/594800798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed0006d960 0x7fed000b5670 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fecf000ebe0 tx=0x7fecf000c520 comp rx=0 tx=0).stop 2024-09-19T07:50:05.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.983+0000 7fed05152640 1 -- 172.21.15.16:0/594800798 shutdown_connections 2024-09-19T07:50:05.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.983+0000 7fed05152640 1 --2- 172.21.15.16:0/594800798 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fecec06c2e0 0x7fecec06e7a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:05.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.983+0000 7fed05152640 1 --2- 172.21.15.16:0/594800798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed0015e680 0x7fed000b8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:05.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.983+0000 7fed05152640 1 --2- 172.21.15.16:0/594800798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed0015e2a0 0x7fed000b9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:05.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.983+0000 7fed05152640 1 --2- 172.21.15.16:0/594800798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed0006d960 0x7fed000b5670 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:05.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.983+0000 7fed05152640 1 -- 172.21.15.16:0/594800798 >> 172.21.15.16:0/594800798 conn(0x7fed0009d760 msgr2=0x7fed00162740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:05.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.983+0000 7fed05152640 1 -- 172.21.15.16:0/594800798 shutdown_connections 2024-09-19T07:50:05.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:05.983+0000 7fed05152640 1 -- 172.21.15.16:0/594800798 wait complete. 2024-09-19T07:50:05.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T07:50:06.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.103+0000 7f1f91703640 1 Processor -- start 2024-09-19T07:50:06.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.103+0000 7f1f91703640 1 -- start start 2024-09-19T07:50:06.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.104+0000 7f1f91703640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f8c161eb0 0x7f1f8c1622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:06.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.104+0000 7f1f91703640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8c162b70 con 0x7f1f8c06cd20 2024-09-19T07:50:06.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.104+0000 7f1f91703640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8c141b00 con 0x7f1f8c1425e0 2024-09-19T07:50:06.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.104+0000 7f1f91703640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8c162e50 con 0x7f1f8c161eb0 2024-09-19T07:50:06.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.104+0000 7f1f8affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1f8c06cd20 0x7f1f8c06d100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35652/0 (socket says 172.21.15.16:35652) 2024-09-19T07:50:06.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.104+0000 7f1f8affd640 1 -- 172.21.15.16:0/1468380 learned_addr learned my addr 172.21.15.16:0/1468380 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:06.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.104+0000 7f1f89ffb640 1 -- 172.21.15.16:0/1468380 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3625959866 0 0) 0x7f1f8c162b70 con 0x7f1f8c06cd20 2024-09-19T07:50:06.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.104+0000 7f1f89ffb640 1 -- 172.21.15.16:0/1468380 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f78003520 con 0x7f1f8c06cd20 2024-09-19T07:50:06.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.104+0000 7f1f8a7fc640 1 --2- 172.21.15.16:0/1468380 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f8c161eb0 0x7f1f8c1622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:06.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.104+0000 7f1f89ffb640 1 -- 172.21.15.16:0/1468380 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1576226143 0 0) 0x7f1f78003520 con 0x7f1f8c06cd20 2024-09-19T07:50:06.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.104+0000 7f1f89ffb640 1 -- 172.21.15.16:0/1468380 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f8c161eb0 msgr2=0x7f1f8c1622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:06.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.105+0000 7f1f89ffb640 1 --2- 172.21.15.16:0/1468380 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f8c161eb0 0x7f1f8c1622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:06.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.105+0000 7f1f89ffb640 1 -- 172.21.15.16:0/1468380 >> v1:172.21.15.146:6789/0 conn(0x7f1f8c1425e0 legacy=0x7f1f8c142a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.105+0000 7f1f89ffb640 1 -- 172.21.15.16:0/1468380 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c141de0 con 0x7f1f8c06cd20 2024-09-19T07:50:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.105+0000 7f1f89ffb640 1 -- 172.21.15.16:0/1468380 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f1f84001670 con 0x7f1f8c06cd20 2024-09-19T07:50:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.105+0000 7f1f89ffb640 1 -- 172.21.15.16:0/1468380 >> v1:172.21.15.16:6789/0 conn(0x7f1f8c06cd20 legacy=0x7f1f8c06d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.105+0000 7f1f89ffb640 1 --2- 172.21.15.16:0/1468380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f780039e0 0x7f1f78003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.105+0000 7f1f89ffb640 1 --2- 172.21.15.16:0/1468380 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f8c161eb0 0x7f1f78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.105+0000 7f1f89ffb640 1 --2- 172.21.15.16:0/1468380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f78004d40 0x7f1f780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.105+0000 7f1f89ffb640 1 -- 172.21.15.16:0/1468380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c162e50 con 0x7f1f78004d40 2024-09-19T07:50:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.105+0000 7f1f89ffb640 1 -- 172.21.15.16:0/1468380 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7800bba0 con 0x7f1f780039e0 2024-09-19T07:50:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.105+0000 7f1f89ffb640 1 -- 172.21.15.16:0/1468380 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1f7800bda0 con 0x7f1f8c161eb0 2024-09-19T07:50:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.105+0000 7f1f8b7fe640 1 --2- 172.21.15.16:0/1468380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f78004d40 0x7f1f780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.105+0000 7f1f8affd640 1 --2- 172.21.15.16:0/1468380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f780039e0 0x7f1f78003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.105+0000 7f1f8a7fc640 1 --2- 172.21.15.16:0/1468380 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f8c161eb0 0x7f1f78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.105+0000 7f1f8b7fe640 1 -- 172.21.15.16:0/1468380 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f8c161eb0 msgr2=0x7f1f78004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.105+0000 7f1f8b7fe640 1 --2- 172.21.15.16:0/1468380 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f8c161eb0 0x7f1f78004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:06.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.105+0000 7f1f8b7fe640 1 -- 172.21.15.16:0/1468380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f780039e0 msgr2=0x7f1f78003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:06.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.105+0000 7f1f8b7fe640 1 --2- 172.21.15.16:0/1468380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f780039e0 0x7f1f78003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:06.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.105+0000 7f1f8b7fe640 1 -- 172.21.15.16:0/1468380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f78003520 con 0x7f1f78004d40 2024-09-19T07:50:06.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.106+0000 7f1f8b7fe640 1 --2- 172.21.15.16:0/1468380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f78004d40 0x7f1f780051c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c001e50 tx=0x7f1f78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4184e616f2294fce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:06.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.106+0000 7f1f89ffb640 1 -- 172.21.15.16:0/1468380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c001a70 con 0x7f1f78004d40 2024-09-19T07:50:06.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.106+0000 7f1f89ffb640 1 -- 172.21.15.16:0/1468380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f7c003970 con 0x7f1f78004d40 2024-09-19T07:50:06.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.106+0000 7f1f89ffb640 1 -- 172.21.15.16:0/1468380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c003d60 con 0x7f1f78004d40 2024-09-19T07:50:06.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.106+0000 7f1f91703640 1 -- 172.21.15.16:0/1468380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f78004d40 msgr2=0x7f1f780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:06.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.106+0000 7f1f91703640 1 --2- 172.21.15.16:0/1468380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f78004d40 0x7f1f780051c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c001e50 tx=0x7f1f78005830 comp rx=0 tx=0).stop 2024-09-19T07:50:06.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.106+0000 7f1f8b7fe640 1 -- 172.21.15.16:0/1468380 reap_dead start 2024-09-19T07:50:06.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.106+0000 7f1f91703640 1 -- 172.21.15.16:0/1468380 shutdown_connections 2024-09-19T07:50:06.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.106+0000 7f1f91703640 1 -- 172.21.15.16:0/1468380 >> 172.21.15.16:0/1468380 conn(0x7f1f8c0b3ed0 msgr2=0x7f1f8c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.106+0000 7f1f91703640 1 -- 172.21.15.16:0/1468380 shutdown_connections 2024-09-19T07:50:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.107+0000 7f1f91703640 1 -- 172.21.15.16:0/1468380 wait complete. 2024-09-19T07:50:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.108+0000 7f1f91703640 1 Processor -- start 2024-09-19T07:50:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.108+0000 7f1f91703640 1 -- start start 2024-09-19T07:50:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f91703640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f78004d40 0x7f1f8c138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f91703640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f8c1543c0 0x7f1f8c0c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f91703640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f8c153de0 0x7f1f8c0c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f91703640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1622f0 con 0x7f1f8c153de0 2024-09-19T07:50:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f91703640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c142150 con 0x7f1f8c1543c0 2024-09-19T07:50:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f91703640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8c162470 con 0x7f1f78004d40 2024-09-19T07:50:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f8c153de0 0x7f1f8c0c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f8c153de0 0x7f1f8c0c7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38902/0 (socket says 172.21.15.16:38902) 2024-09-19T07:50:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f8b7fe640 1 -- 172.21.15.16:0/1975976067 learned_addr learned my addr 172.21.15.16:0/1975976067 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f8affd640 1 --2- 172.21.15.16:0/1975976067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f78004d40 0x7f1f8c138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f8a7fc640 1 --2- 172.21.15.16:0/1975976067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f8c1543c0 0x7f1f8c0c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f8b7fe640 1 -- 172.21.15.16:0/1975976067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f78004d40 msgr2=0x7f1f8c138d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f8b7fe640 1 --2- 172.21.15.16:0/1975976067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f78004d40 0x7f1f8c138d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f8b7fe640 1 -- 172.21.15.16:0/1975976067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f8c1543c0 msgr2=0x7f1f8c0c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f8b7fe640 1 --2- 172.21.15.16:0/1975976067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f8c1543c0 0x7f1f8c0c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f8b7fe640 1 -- 172.21.15.16:0/1975976067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c13e320 con 0x7f1f8c153de0 2024-09-19T07:50:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f8a7fc640 1 --2- 172.21.15.16:0/1975976067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f8c1543c0 0x7f1f8c0c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f8b7fe640 1 --2- 172.21.15.16:0/1975976067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f8c153de0 0x7f1f8c0c7b10 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c005040 tx=0x7f1f7c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f837fe640 1 -- 172.21.15.16:0/1975976067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c007db0 con 0x7f1f8c153de0 2024-09-19T07:50:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f837fe640 1 -- 172.21.15.16:0/1975976067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f7c041070 con 0x7f1f8c153de0 2024-09-19T07:50:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f91703640 1 -- 172.21.15.16:0/1975976067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8c13e600 con 0x7f1f8c153de0 2024-09-19T07:50:06.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f837fe640 1 -- 172.21.15.16:0/1975976067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0050d0 con 0x7f1f8c153de0 2024-09-19T07:50:06.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.109+0000 7f1f91703640 1 -- 172.21.15.16:0/1975976067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8c13ebc0 con 0x7f1f8c153de0 2024-09-19T07:50:06.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.110+0000 7f1f91703640 1 -- 172.21.15.16:0/1975976067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f8c162b70 con 0x7f1f8c153de0 2024-09-19T07:50:06.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.111+0000 7f1f837fe640 1 -- 172.21.15.16:0/1975976067 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1f7c03a070 con 0x7f1f8c153de0 2024-09-19T07:50:06.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.111+0000 7f1f837fe640 1 --2- 172.21.15.16:0/1975976067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1f6c06c2e0 0x7f1f6c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:06.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.111+0000 7f1f837fe640 1 -- 172.21.15.16:0/1975976067 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5335+0+0 (secure 0 0 0) 0x7f1f7c0aaa50 con 0x7f1f8c153de0 2024-09-19T07:50:06.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.112+0000 7f1f8affd640 1 --2- 172.21.15.16:0/1975976067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1f6c06c2e0 0x7f1f6c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:06.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.112+0000 7f1f8affd640 1 --2- 172.21.15.16:0/1975976067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1f6c06c2e0 0x7f1f6c06e7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1f84001cc0 tx=0x7f1f84000f70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:06.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.116+0000 7f1f837fe640 1 -- 172.21.15.16:0/1975976067 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1f7c079330 con 0x7f1f8c153de0 2024-09-19T07:50:06.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.292+0000 7f1f91703640 1 -- 172.21.15.16:0/1975976067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f1f8c072220 con 0x7f1f8c153de0 2024-09-19T07:50:06.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.295+0000 7f1f837fe640 1 -- 172.21.15.16:0/1975976067 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7f1f7c078c30 con 0x7f1f8c153de0 2024-09-19T07:50:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.297+0000 7f1f91703640 1 -- 172.21.15.16:0/1975976067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1f6c06c2e0 msgr2=0x7f1f6c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.297+0000 7f1f91703640 1 --2- 172.21.15.16:0/1975976067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1f6c06c2e0 0x7f1f6c06e7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1f84001cc0 tx=0x7f1f84000f70 comp rx=0 tx=0).stop 2024-09-19T07:50:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.297+0000 7f1f91703640 1 -- 172.21.15.16:0/1975976067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f8c153de0 msgr2=0x7f1f8c0c7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.297+0000 7f1f91703640 1 --2- 172.21.15.16:0/1975976067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f8c153de0 0x7f1f8c0c7b10 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c005040 tx=0x7f1f7c0014a0 comp rx=0 tx=0).stop 2024-09-19T07:50:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.297+0000 7f1f91703640 1 -- 172.21.15.16:0/1975976067 shutdown_connections 2024-09-19T07:50:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.297+0000 7f1f91703640 1 --2- 172.21.15.16:0/1975976067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1f6c06c2e0 0x7f1f6c06e7a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.297+0000 7f1f91703640 1 --2- 172.21.15.16:0/1975976067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f8c153de0 0x7f1f8c0c7b10 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.297+0000 7f1f91703640 1 --2- 172.21.15.16:0/1975976067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1f8c1543c0 0x7f1f8c0c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.297+0000 7f1f91703640 1 --2- 172.21.15.16:0/1975976067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f78004d40 0x7f1f8c138d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.297+0000 7f1f91703640 1 -- 172.21.15.16:0/1975976067 >> 172.21.15.16:0/1975976067 conn(0x7f1f8c0b3ed0 msgr2=0x7f1f8c0b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.297+0000 7f1f91703640 1 -- 172.21.15.16:0/1975976067 shutdown_connections 2024-09-19T07:50:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.297+0000 7f1f91703640 1 -- 172.21.15.16:0/1975976067 wait complete. 2024-09-19T07:50:06.308 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps' 2024-09-19T07:50:06.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.410+0000 7f83e0247640 1 Processor -- start 2024-09-19T07:50:06.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.410+0000 7f83e0247640 1 -- start start 2024-09-19T07:50:06.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.411+0000 7f83e0247640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83d815e680 0x7f83d80cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:06.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.411+0000 7f83e0247640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f83d80d0010 0x7f83d80d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:06.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.411+0000 7f83e0247640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d80d1880 con 0x7f83d815e2a0 2024-09-19T07:50:06.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.411+0000 7f83e0247640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f83d80d1b60 con 0x7f83d80d0010 2024-09-19T07:50:06.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.411+0000 7f83e0247640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f83d80cf2d0 con 0x7f83d815e680 2024-09-19T07:50:06.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.411+0000 7f83dd7e2640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83d815e680 0x7f83d80cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:06.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.411+0000 7f83de7e4640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f83d80d0010 0x7f83d80d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.411+0000 7f83dd7e2640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83d815e680 0x7f83d80cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:34522/0 (socket says 172.21.15.16:34522) 2024-09-19T07:50:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.411+0000 7f83de7e4640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f83d80d0010 0x7f83d80d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:51640/0 (socket says 172.21.15.16:51640) 2024-09-19T07:50:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.411+0000 7f83dd7e2640 1 -- 172.21.15.16:0/3297425552 learned_addr learned my addr 172.21.15.16:0/3297425552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.411+0000 7f83ddfe3640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f83d815e2a0 0x7f83d806d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:37740/0 (socket says 172.21.15.16:37740) 2024-09-19T07:50:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.412+0000 7f83dd7e2640 1 -- 172.21.15.16:0/3297425552 >> v2:172.21.15.146:3300/0 conn(0x7f83d80d0010 msgr2=0x7f83d80d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.412+0000 7f83dd7e2640 1 --2- 172.21.15.16:0/3297425552 >> v2:172.21.15.146:3300/0 conn(0x7f83d80d0010 0x7f83d80d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.412+0000 7f83dd7e2640 1 -- 172.21.15.16:0/3297425552 >> v1:172.21.15.146:6789/0 conn(0x7f83d815e2a0 legacy=0x7f83d806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.412+0000 7f83dd7e2640 1 -- 172.21.15.16:0/3297425552 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d80cf5b0 con 0x7f83d815e680 2024-09-19T07:50:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.412+0000 7f83dd7e2640 1 --2- 172.21.15.16:0/3297425552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83d815e680 0x7f83d80cfad0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009870 tx=0x7f83c802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b0dc2557c6c9cad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.413+0000 7f83dcfe1640 1 -- 172.21.15.16:0/3297425552 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8037070 con 0x7f83d815e680 2024-09-19T07:50:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.413+0000 7f83dcfe1640 1 -- 172.21.15.16:0/3297425552 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c8032440 con 0x7f83d815e680 2024-09-19T07:50:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.413+0000 7f83dcfe1640 1 -- 172.21.15.16:0/3297425552 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8032810 con 0x7f83d815e680 2024-09-19T07:50:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.413+0000 7f83e0247640 1 -- 172.21.15.16:0/3297425552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83d815e680 msgr2=0x7f83d80cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.413+0000 7f83e0247640 1 --2- 172.21.15.16:0/3297425552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83d815e680 0x7f83d80cfad0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009870 tx=0x7f83c802a1c0 comp rx=0 tx=0).stop 2024-09-19T07:50:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.413+0000 7f83e0247640 1 -- 172.21.15.16:0/3297425552 shutdown_connections 2024-09-19T07:50:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.413+0000 7f83e0247640 1 --2- 172.21.15.16:0/3297425552 >> v2:172.21.15.146:3300/0 conn(0x7f83d80d0010 0x7f83d80d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.413+0000 7f83e0247640 1 --2- 172.21.15.16:0/3297425552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83d815e680 0x7f83d80cfad0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.413+0000 7f83e0247640 1 -- 172.21.15.16:0/3297425552 >> 172.21.15.16:0/3297425552 conn(0x7f83d809d760 msgr2=0x7f83d808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.413+0000 7f83e0247640 1 -- 172.21.15.16:0/3297425552 shutdown_connections 2024-09-19T07:50:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.413+0000 7f83e0247640 1 -- 172.21.15.16:0/3297425552 wait complete. 2024-09-19T07:50:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.415+0000 7f83e0247640 1 Processor -- start 2024-09-19T07:50:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.415+0000 7f83e0247640 1 -- start start 2024-09-19T07:50:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.415+0000 7f83e0247640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83d80d0010 0x7f83d8152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.415+0000 7f83e0247640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83d815e2a0 0x7f83d80b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.415+0000 7f83e0247640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d815e680 0x7f83d8160d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.415+0000 7f83e0247640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83d806dcb0 con 0x7f83d815e680 2024-09-19T07:50:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.415+0000 7f83e0247640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f83d806db30 con 0x7f83d815e2a0 2024-09-19T07:50:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.415+0000 7f83e0247640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f83d80d48b0 con 0x7f83d80d0010 2024-09-19T07:50:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.416+0000 7f83de7e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d815e680 0x7f83d8160d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.416+0000 7f83ddfe3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83d80d0010 0x7f83d8152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.416+0000 7f83ddfe3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83d80d0010 0x7f83d8152980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:34530/0 (socket says 172.21.15.16:34530) 2024-09-19T07:50:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.416+0000 7f83ddfe3640 1 -- 172.21.15.16:0/109301632 learned_addr learned my addr 172.21.15.16:0/109301632 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.416+0000 7f83de7e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d815e680 0x7f83d8160d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38918/0 (socket says 172.21.15.16:38918) 2024-09-19T07:50:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.416+0000 7f83dd7e2640 1 --2- 172.21.15.16:0/109301632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83d815e2a0 0x7f83d80b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.416+0000 7f83ddfe3640 1 -- 172.21.15.16:0/109301632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83d815e2a0 msgr2=0x7f83d80b8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.416+0000 7f83ddfe3640 1 --2- 172.21.15.16:0/109301632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83d815e2a0 0x7f83d80b8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.416+0000 7f83ddfe3640 1 -- 172.21.15.16:0/109301632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d815e680 msgr2=0x7f83d8160d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.416+0000 7f83ddfe3640 1 --2- 172.21.15.16:0/109301632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d815e680 0x7f83d8160d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.416+0000 7f83ddfe3640 1 -- 172.21.15.16:0/109301632 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d80b94a0 con 0x7f83d80d0010 2024-09-19T07:50:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.416+0000 7f83de7e4640 1 --2- 172.21.15.16:0/109301632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d815e680 0x7f83d8160d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.416+0000 7f83ddfe3640 1 --2- 172.21.15.16:0/109301632 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83d80d0010 0x7f83d8152980 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f83d400c400 tx=0x7f83d400c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.416+0000 7f83ce7fc640 1 -- 172.21.15.16:0/109301632 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d40074e0 con 0x7f83d80d0010 2024-09-19T07:50:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.416+0000 7f83ce7fc640 1 -- 172.21.15.16:0/109301632 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83d4007680 con 0x7f83d80d0010 2024-09-19T07:50:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.416+0000 7f83ce7fc640 1 -- 172.21.15.16:0/109301632 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4004a00 con 0x7f83d80d0010 2024-09-19T07:50:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.416+0000 7f83e0247640 1 -- 172.21.15.16:0/109301632 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d80cc630 con 0x7f83d80d0010 2024-09-19T07:50:06.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.416+0000 7f83e0247640 1 -- 172.21.15.16:0/109301632 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d80ccbf0 con 0x7f83d80d0010 2024-09-19T07:50:06.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.417+0000 7f83e0247640 1 -- 172.21.15.16:0/109301632 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d80d4370 con 0x7f83d80d0010 2024-09-19T07:50:06.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.418+0000 7f83ce7fc640 1 -- 172.21.15.16:0/109301632 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f83d401a020 con 0x7f83d80d0010 2024-09-19T07:50:06.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.418+0000 7f83ce7fc640 1 --2- 172.21.15.16:0/109301632 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f83bc06c2b0 0x7f83bc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:06.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.419+0000 7f83ce7fc640 1 -- 172.21.15.16:0/109301632 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(84..84 src has 1..84) ==== 5335+0+0 (secure 0 0 0) 0x7f83d408aa10 con 0x7f83d80d0010 2024-09-19T07:50:06.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.419+0000 7f83dd7e2640 1 --2- 172.21.15.16:0/109301632 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f83bc06c2b0 0x7f83bc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:06.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.419+0000 7f83dd7e2640 1 --2- 172.21.15.16:0/109301632 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f83bc06c2b0 0x7f83bc06e770 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f83c8034000 tx=0x7f83c802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:06.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.424+0000 7f83ce7fc640 1 -- 172.21.15.16:0/109301632 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f83d4058c20 con 0x7f83d80d0010 2024-09-19T07:50:06.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:06.669+0000 7f83e0247640 1 -- 172.21.15.16:0/109301632 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]} v 0) -- 0x7f83d806cf90 con 0x7f83d80d0010 2024-09-19T07:50:07.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.663+0000 7f83ce7fc640 1 -- 172.21.15.16:0/109301632 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]}]=0 v313) ==== 148+0+0 (secure 0 0 0) 0x7f83d405cf20 con 0x7f83d80d0010 2024-09-19T07:50:07.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.666+0000 7f83e0247640 1 -- 172.21.15.16:0/109301632 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f83bc06c2b0 msgr2=0x7f83bc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:07.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.666+0000 7f83e0247640 1 --2- 172.21.15.16:0/109301632 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f83bc06c2b0 0x7f83bc06e770 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f83c8034000 tx=0x7f83c802e000 comp rx=0 tx=0).stop 2024-09-19T07:50:07.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.666+0000 7f83e0247640 1 -- 172.21.15.16:0/109301632 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83d80d0010 msgr2=0x7f83d8152980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:07.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.666+0000 7f83e0247640 1 --2- 172.21.15.16:0/109301632 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83d80d0010 0x7f83d8152980 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f83d400c400 tx=0x7f83d400c8c0 comp rx=0 tx=0).stop 2024-09-19T07:50:07.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.666+0000 7f83e0247640 1 -- 172.21.15.16:0/109301632 shutdown_connections 2024-09-19T07:50:07.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.666+0000 7f83e0247640 1 --2- 172.21.15.16:0/109301632 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f83bc06c2b0 0x7f83bc06e770 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:07.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.666+0000 7f83e0247640 1 --2- 172.21.15.16:0/109301632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d815e680 0x7f83d8160d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:07.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.666+0000 7f83e0247640 1 --2- 172.21.15.16:0/109301632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83d815e2a0 0x7f83d80b8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:07.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.666+0000 7f83e0247640 1 --2- 172.21.15.16:0/109301632 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83d80d0010 0x7f83d8152980 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:07.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.666+0000 7f83e0247640 1 -- 172.21.15.16:0/109301632 >> 172.21.15.16:0/109301632 conn(0x7f83d809d760 msgr2=0x7f83d80d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:07.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.666+0000 7f83e0247640 1 -- 172.21.15.16:0/109301632 shutdown_connections 2024-09-19T07:50:07.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.666+0000 7f83e0247640 1 -- 172.21.15.16:0/109301632 wait complete. 2024-09-19T07:50:07.680 INFO:tasks.cephfs_test_runner:That the MDS will not let a client sit above mds_max_caps_per_client caps. ... ok 2024-09-19T07:50:07.681 INFO:tasks.cephfs_test_runner:Starting test: test_client_min_caps_working_set (tasks.cephfs.test_client_limits.TestClientLimits) When a client has inodes pinned in its cache (open files), that the MDS 2024-09-19T07:50:07.681 INFO:tasks.cephfs_test_runner:test_client_min_caps_working_set (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-19T07:50:07.681 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:50:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.789+0000 7f33806d0640 1 Processor -- start 2024-09-19T07:50:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.789+0000 7f33806d0640 1 -- start start 2024-09-19T07:50:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.790+0000 7f33806d0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f337815f0d0 0x7f337815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.790+0000 7f33806d0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33780d1b10 con 0x7f33780d4130 2024-09-19T07:50:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.790+0000 7f33806d0640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33780cf2b0 con 0x7f33780d0070 2024-09-19T07:50:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.790+0000 7f33806d0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f33780cf590 con 0x7f337815f0d0 2024-09-19T07:50:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.791+0000 7f337e46c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f337815f0d0 0x7f337815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.791+0000 7f337e46c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f337815f0d0 0x7f337815f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38930/0 (socket says 172.21.15.16:38930) 2024-09-19T07:50:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.791+0000 7f337e46c640 1 -- 172.21.15.16:0/1319249363 learned_addr learned my addr 172.21.15.16:0/1319249363 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.791+0000 7f337ec6d640 1 --1- 172.21.15.16:0/1319249363 >> v1:172.21.15.16:6789/0 conn(0x7f33780d4130 0x7f33780d4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35658/0 (socket says 172.21.15.16:35658) 2024-09-19T07:50:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.791+0000 7f337e46c640 1 -- 172.21.15.16:0/1319249363 >> v1:172.21.15.146:6789/0 conn(0x7f33780d0070 legacy=0x7f33780d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.791+0000 7f337e46c640 1 -- 172.21.15.16:0/1319249363 >> v1:172.21.15.16:6789/0 conn(0x7f33780d4130 legacy=0x7f33780d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.791+0000 7f337e46c640 1 -- 172.21.15.16:0/1319249363 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33780cf870 con 0x7f337815f0d0 2024-09-19T07:50:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.792+0000 7f337e46c640 1 --2- 172.21.15.16:0/1319249363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f337815f0d0 0x7f337815f4b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3374009870 tx=0x7f337402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b077f90c67222551 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.792+0000 7f337d46a640 1 -- 172.21.15.16:0/1319249363 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3374037070 con 0x7f337815f0d0 2024-09-19T07:50:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.792+0000 7f337d46a640 1 -- 172.21.15.16:0/1319249363 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3374032440 con 0x7f337815f0d0 2024-09-19T07:50:07.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.792+0000 7f337d46a640 1 -- 172.21.15.16:0/1319249363 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3374032810 con 0x7f337815f0d0 2024-09-19T07:50:07.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.792+0000 7f33806d0640 1 -- 172.21.15.16:0/1319249363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f337815f0d0 msgr2=0x7f337815f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:07.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.792+0000 7f33806d0640 1 --2- 172.21.15.16:0/1319249363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f337815f0d0 0x7f337815f4b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3374009870 tx=0x7f337402a180 comp rx=0 tx=0).stop 2024-09-19T07:50:07.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.792+0000 7f33806d0640 1 -- 172.21.15.16:0/1319249363 shutdown_connections 2024-09-19T07:50:07.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.792+0000 7f33806d0640 1 --2- 172.21.15.16:0/1319249363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f337815f0d0 0x7f337815f4b0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:07.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.792+0000 7f33806d0640 1 -- 172.21.15.16:0/1319249363 >> 172.21.15.16:0/1319249363 conn(0x7f337809d760 msgr2=0x7f337808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:07.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.793+0000 7f33806d0640 1 -- 172.21.15.16:0/1319249363 shutdown_connections 2024-09-19T07:50:07.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.793+0000 7f33806d0640 1 -- 172.21.15.16:0/1319249363 wait complete. 2024-09-19T07:50:07.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.794+0000 7f33806d0640 1 Processor -- start 2024-09-19T07:50:07.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.794+0000 7f33806d0640 1 -- start start 2024-09-19T07:50:07.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.794+0000 7f33806d0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33780d0070 0x7f3378161720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:07.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.794+0000 7f33806d0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33780d4130 0x7f33781f8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.794+0000 7f33806d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f337815f0d0 0x7f3378151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.794+0000 7f33806d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33780b2f70 con 0x7f337815f0d0 2024-09-19T07:50:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.794+0000 7f33806d0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f33780cef60 con 0x7f33780d0070 2024-09-19T07:50:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.794+0000 7f33806d0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f33780b30f0 con 0x7f33780d4130 2024-09-19T07:50:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.795+0000 7f337ec6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f337815f0d0 0x7f3378151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.795+0000 7f337ec6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f337815f0d0 0x7f3378151f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38938/0 (socket says 172.21.15.16:38938) 2024-09-19T07:50:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.795+0000 7f337ec6d640 1 -- 172.21.15.16:0/3699758719 learned_addr learned my addr 172.21.15.16:0/3699758719 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.795+0000 7f337dc6b640 1 --2- 172.21.15.16:0/3699758719 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33780d4130 0x7f33781f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.795+0000 7f337e46c640 1 --2- 172.21.15.16:0/3699758719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33780d0070 0x7f3378161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.795+0000 7f337ec6d640 1 -- 172.21.15.16:0/3699758719 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33780d4130 msgr2=0x7f33781f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.795+0000 7f337ec6d640 1 --2- 172.21.15.16:0/3699758719 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33780d4130 0x7f33781f8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.795+0000 7f337ec6d640 1 -- 172.21.15.16:0/3699758719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33780d0070 msgr2=0x7f3378161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.795+0000 7f337ec6d640 1 --2- 172.21.15.16:0/3699758719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33780d0070 0x7f3378161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.795+0000 7f337ec6d640 1 -- 172.21.15.16:0/3699758719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33780cd220 con 0x7f337815f0d0 2024-09-19T07:50:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.795+0000 7f337e46c640 1 --2- 172.21.15.16:0/3699758719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33780d0070 0x7f3378161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.795+0000 7f337ec6d640 1 --2- 172.21.15.16:0/3699758719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f337815f0d0 0x7f3378151f70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f337000c7a0 tx=0x7f337000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.795+0000 7f336effd640 1 -- 172.21.15.16:0/3699758719 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370017070 con 0x7f337815f0d0 2024-09-19T07:50:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.795+0000 7f336effd640 1 -- 172.21.15.16:0/3699758719 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f337000a640 con 0x7f337815f0d0 2024-09-19T07:50:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.795+0000 7f33806d0640 1 -- 172.21.15.16:0/3699758719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33781f8820 con 0x7f337815f0d0 2024-09-19T07:50:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.795+0000 7f336effd640 1 -- 172.21.15.16:0/3699758719 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337000aa30 con 0x7f337815f0d0 2024-09-19T07:50:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.795+0000 7f33806d0640 1 -- 172.21.15.16:0/3699758719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33781f8de0 con 0x7f337815f0d0 2024-09-19T07:50:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.796+0000 7f33806d0640 1 -- 172.21.15.16:0/3699758719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3378068ce0 con 0x7f337815f0d0 2024-09-19T07:50:07.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.797+0000 7f336effd640 1 -- 172.21.15.16:0/3699758719 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f337001a070 con 0x7f337815f0d0 2024-09-19T07:50:07.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.797+0000 7f336effd640 1 --2- 172.21.15.16:0/3699758719 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f335c06c2b0 0x7f335c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:07.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.797+0000 7f336effd640 1 -- 172.21.15.16:0/3699758719 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5335+0+0 (secure 0 0 0) 0x7f337008a9a0 con 0x7f337815f0d0 2024-09-19T07:50:07.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.798+0000 7f337e46c640 1 --2- 172.21.15.16:0/3699758719 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f335c06c2b0 0x7f335c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:07.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.798+0000 7f337e46c640 1 --2- 172.21.15.16:0/3699758719 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f335c06c2b0 0x7f335c06e770 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3374034000 tx=0x7f337402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:07.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.802+0000 7f336effd640 1 -- 172.21.15.16:0/3699758719 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3378068ce0 con 0x7f337815f0d0 2024-09-19T07:50:07.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.981+0000 7f33806d0640 1 -- 172.21.15.16:0/3699758719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33780c7c60 con 0x7f337815f0d0 2024-09-19T07:50:07.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.982+0000 7f336effd640 1 -- 172.21.15.16:0/3699758719 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+13332 (secure 0 0 0) 0x7f33700591e0 con 0x7f337815f0d0 2024-09-19T07:50:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:07.982 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:50:05.649739+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.146:6837/3642987189":"2024-09-20T07:50:02.644889+0000","172.21.15.146:6835/3642987189":"2024-09-20T07:50:02.644889+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:50:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.984+0000 7f33806d0640 1 -- 172.21.15.16:0/3699758719 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f335c06c2b0 msgr2=0x7f335c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.984+0000 7f33806d0640 1 --2- 172.21.15.16:0/3699758719 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f335c06c2b0 0x7f335c06e770 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3374034000 tx=0x7f337402e000 comp rx=0 tx=0).stop 2024-09-19T07:50:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.984+0000 7f33806d0640 1 -- 172.21.15.16:0/3699758719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f337815f0d0 msgr2=0x7f3378151f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.984+0000 7f33806d0640 1 --2- 172.21.15.16:0/3699758719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f337815f0d0 0x7f3378151f70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f337000c7a0 tx=0x7f337000cc60 comp rx=0 tx=0).stop 2024-09-19T07:50:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.984+0000 7f33806d0640 1 -- 172.21.15.16:0/3699758719 shutdown_connections 2024-09-19T07:50:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.984+0000 7f33806d0640 1 --2- 172.21.15.16:0/3699758719 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f335c06c2b0 0x7f335c06e770 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.984+0000 7f33806d0640 1 --2- 172.21.15.16:0/3699758719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f337815f0d0 0x7f3378151f70 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.984+0000 7f33806d0640 1 --2- 172.21.15.16:0/3699758719 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33780d4130 0x7f33781f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.984+0000 7f33806d0640 1 --2- 172.21.15.16:0/3699758719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33780d0070 0x7f3378161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.984+0000 7f33806d0640 1 -- 172.21.15.16:0/3699758719 >> 172.21.15.16:0/3699758719 conn(0x7f337809d760 msgr2=0x7f33780d4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.984+0000 7f33806d0640 1 -- 172.21.15.16:0/3699758719 shutdown_connections 2024-09-19T07:50:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:07.984+0000 7f33806d0640 1 -- 172.21.15.16:0/3699758719 wait complete. 2024-09-19T07:50:07.996 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T07:42:54.500338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T07:50:07.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T07:50:08.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.101+0000 7f121454b640 1 Processor -- start 2024-09-19T07:50:08.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.101+0000 7f121454b640 1 -- start start 2024-09-19T07:50:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.102+0000 7f121454b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f120c15e680 0x7f120c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.102+0000 7f121454b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f120c0d0010 0x7f120c0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.102+0000 7f121454b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f120c0d1880 con 0x7f120c15e2a0 2024-09-19T07:50:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.102+0000 7f121454b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f120c0d1b60 con 0x7f120c15e680 2024-09-19T07:50:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.102+0000 7f121454b640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f120c0cf2d0 con 0x7f120c0d0010 2024-09-19T07:50:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.102+0000 7f12122e7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f120c15e2a0 0x7f120c06d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35666/0 (socket says 172.21.15.16:35666) 2024-09-19T07:50:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.102+0000 7f12122e7640 1 -- 172.21.15.16:0/1685280748 learned_addr learned my addr 172.21.15.16:0/1685280748 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.102+0000 7f1211ae6640 1 --2- 172.21.15.16:0/1685280748 >> v2:172.21.15.16:3300/0 conn(0x7f120c15e680 0x7f120c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:08.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.103+0000 7f1212ae8640 1 --2- 172.21.15.16:0/1685280748 >> v2:172.21.15.146:3300/0 conn(0x7f120c0d0010 0x7f120c0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:08.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.103+0000 7f1211ae6640 1 -- 172.21.15.16:0/1685280748 >> v2:172.21.15.146:3300/0 conn(0x7f120c0d0010 msgr2=0x7f120c0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:08.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.103+0000 7f1211ae6640 1 --2- 172.21.15.16:0/1685280748 >> v2:172.21.15.146:3300/0 conn(0x7f120c0d0010 0x7f120c0d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:08.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.103+0000 7f1211ae6640 1 -- 172.21.15.16:0/1685280748 >> v1:172.21.15.16:6789/0 conn(0x7f120c15e2a0 legacy=0x7f120c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:08.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.103+0000 7f12112e5640 1 -- 172.21.15.16:0/1685280748 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2258299570 0 0) 0x7f120c0d1880 con 0x7f120c15e2a0 2024-09-19T07:50:08.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.103+0000 7f1211ae6640 1 -- 172.21.15.16:0/1685280748 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120c0cf5b0 con 0x7f120c15e680 2024-09-19T07:50:08.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.103+0000 7f1211ae6640 1 --2- 172.21.15.16:0/1685280748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120c15e680 0x7f120c0cfad0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f120c163050 tx=0x7f11fc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f1d1986028ff5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:08.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.103+0000 7f12112e5640 1 -- 172.21.15.16:0/1685280748 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc037070 con 0x7f120c15e680 2024-09-19T07:50:08.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.103+0000 7f12112e5640 1 -- 172.21.15.16:0/1685280748 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11fc032440 con 0x7f120c15e680 2024-09-19T07:50:08.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.104+0000 7f12112e5640 1 -- 172.21.15.16:0/1685280748 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc02ae60 con 0x7f120c15e680 2024-09-19T07:50:08.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.104+0000 7f121454b640 1 -- 172.21.15.16:0/1685280748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120c15e680 msgr2=0x7f120c0cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:08.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.104+0000 7f121454b640 1 --2- 172.21.15.16:0/1685280748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120c15e680 0x7f120c0cfad0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f120c163050 tx=0x7f11fc02a110 comp rx=0 tx=0).stop 2024-09-19T07:50:08.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.104+0000 7f121454b640 1 -- 172.21.15.16:0/1685280748 shutdown_connections 2024-09-19T07:50:08.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.104+0000 7f121454b640 1 --2- 172.21.15.16:0/1685280748 >> v2:172.21.15.146:3300/0 conn(0x7f120c0d0010 0x7f120c0d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:08.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.104+0000 7f121454b640 1 --2- 172.21.15.16:0/1685280748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120c15e680 0x7f120c0cfad0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:08.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.104+0000 7f121454b640 1 -- 172.21.15.16:0/1685280748 >> 172.21.15.16:0/1685280748 conn(0x7f120c09d760 msgr2=0x7f120c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:08.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.104+0000 7f121454b640 1 -- 172.21.15.16:0/1685280748 shutdown_connections 2024-09-19T07:50:08.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.104+0000 7f121454b640 1 -- 172.21.15.16:0/1685280748 wait complete. 2024-09-19T07:50:08.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.105+0000 7f121454b640 1 Processor -- start 2024-09-19T07:50:08.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.105+0000 7f121454b640 1 -- start start 2024-09-19T07:50:08.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.106+0000 7f121454b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f120c0d0010 0x7f120c157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:08.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.106+0000 7f121454b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120c15e2a0 0x7f120c0ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:08.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.106+0000 7f121454b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f120c15e680 0x7f120c0ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:08.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.106+0000 7f121454b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f120c06dcb0 con 0x7f120c15e2a0 2024-09-19T07:50:08.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.106+0000 7f121454b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f120c06db30 con 0x7f120c0d0010 2024-09-19T07:50:08.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.106+0000 7f121454b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f120c0d48b0 con 0x7f120c15e680 2024-09-19T07:50:08.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.106+0000 7f1211ae6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120c15e2a0 0x7f120c0ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:08.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.106+0000 7f1211ae6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120c15e2a0 0x7f120c0ca0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38954/0 (socket says 172.21.15.16:38954) 2024-09-19T07:50:08.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.106+0000 7f1211ae6640 1 -- 172.21.15.16:0/103067774 learned_addr learned my addr 172.21.15.16:0/103067774 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:08.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.106+0000 7f1212ae8640 1 --2- 172.21.15.16:0/103067774 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f120c15e680 0x7f120c0ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:08.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.106+0000 7f12122e7640 1 --2- 172.21.15.16:0/103067774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f120c0d0010 0x7f120c157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:08.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.106+0000 7f1211ae6640 1 -- 172.21.15.16:0/103067774 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f120c15e680 msgr2=0x7f120c0ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:08.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.106+0000 7f1211ae6640 1 --2- 172.21.15.16:0/103067774 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f120c15e680 0x7f120c0ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:08.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.106+0000 7f1211ae6640 1 -- 172.21.15.16:0/103067774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f120c0d0010 msgr2=0x7f120c157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:08.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.106+0000 7f1211ae6640 1 --2- 172.21.15.16:0/103067774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f120c0d0010 0x7f120c157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:08.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.106+0000 7f1211ae6640 1 -- 172.21.15.16:0/103067774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120c0b3c10 con 0x7f120c15e2a0 2024-09-19T07:50:08.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.106+0000 7f12122e7640 1 --2- 172.21.15.16:0/103067774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f120c0d0010 0x7f120c157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:08.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.107+0000 7f1211ae6640 1 --2- 172.21.15.16:0/103067774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120c15e2a0 0x7f120c0ca0e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f11fc000c00 tx=0x7f11fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:08.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.107+0000 7f1202ffd640 1 -- 172.21.15.16:0/103067774 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc037040 con 0x7f120c15e2a0 2024-09-19T07:50:08.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.107+0000 7f1202ffd640 1 -- 172.21.15.16:0/103067774 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11fc042070 con 0x7f120c15e2a0 2024-09-19T07:50:08.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.107+0000 7f121454b640 1 -- 172.21.15.16:0/103067774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120c0b3eb0 con 0x7f120c15e2a0 2024-09-19T07:50:08.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.107+0000 7f1202ffd640 1 -- 172.21.15.16:0/103067774 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc030bb0 con 0x7f120c15e2a0 2024-09-19T07:50:08.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.107+0000 7f121454b640 1 -- 172.21.15.16:0/103067774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f120c0b4470 con 0x7f120c15e2a0 2024-09-19T07:50:08.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.108+0000 7f121454b640 1 -- 172.21.15.16:0/103067774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f120c0d4c70 con 0x7f120c15e2a0 2024-09-19T07:50:08.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.108+0000 7f1202ffd640 1 -- 172.21.15.16:0/103067774 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f11fc047460 con 0x7f120c15e2a0 2024-09-19T07:50:08.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.109+0000 7f1202ffd640 1 --2- 172.21.15.16:0/103067774 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f11f006c2b0 0x7f11f006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:08.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.109+0000 7f1202ffd640 1 -- 172.21.15.16:0/103067774 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5335+0+0 (secure 0 0 0) 0x7f11fc0abfa0 con 0x7f120c15e2a0 2024-09-19T07:50:08.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.109+0000 7f12122e7640 1 --2- 172.21.15.16:0/103067774 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f11f006c2b0 0x7f11f006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:08.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.110+0000 7f12122e7640 1 --2- 172.21.15.16:0/103067774 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f11f006c2b0 0x7f11f006e770 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1208000f60 tx=0x7f1208009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:08.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.113+0000 7f1202ffd640 1 -- 172.21.15.16:0/103067774 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f11fc045360 con 0x7f120c15e2a0 2024-09-19T07:50:08.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:50:08.258 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:50:08.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.288+0000 7f121454b640 1 -- 172.21.15.16:0/103067774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f120c06cf90 con 0x7f120c15e2a0 2024-09-19T07:50:08.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.289+0000 7f1202ffd640 1 -- 172.21.15.16:0/103067774 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f11fc07a880 con 0x7f120c15e2a0 2024-09-19T07:50:08.289 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-19T07:50:08.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.291+0000 7f121454b640 1 -- 172.21.15.16:0/103067774 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f11f006c2b0 msgr2=0x7f11f006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:08.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.291+0000 7f121454b640 1 --2- 172.21.15.16:0/103067774 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f11f006c2b0 0x7f11f006e770 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1208000f60 tx=0x7f1208009040 comp rx=0 tx=0).stop 2024-09-19T07:50:08.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.291+0000 7f121454b640 1 -- 172.21.15.16:0/103067774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120c15e2a0 msgr2=0x7f120c0ca0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:08.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.291+0000 7f121454b640 1 --2- 172.21.15.16:0/103067774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120c15e2a0 0x7f120c0ca0e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f11fc000c00 tx=0x7f11fc0046c0 comp rx=0 tx=0).stop 2024-09-19T07:50:08.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.291+0000 7f121454b640 1 -- 172.21.15.16:0/103067774 shutdown_connections 2024-09-19T07:50:08.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.291+0000 7f121454b640 1 --2- 172.21.15.16:0/103067774 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f11f006c2b0 0x7f11f006e770 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:08.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.291+0000 7f121454b640 1 --2- 172.21.15.16:0/103067774 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f120c15e680 0x7f120c0ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:08.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.291+0000 7f121454b640 1 --2- 172.21.15.16:0/103067774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f120c15e2a0 0x7f120c0ca0e0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:08.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.291+0000 7f121454b640 1 --2- 172.21.15.16:0/103067774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f120c0d0010 0x7f120c157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:08.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.291+0000 7f121454b640 1 -- 172.21.15.16:0/103067774 >> 172.21.15.16:0/103067774 conn(0x7f120c09d760 msgr2=0x7f120c0d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:08.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.291+0000 7f121454b640 1 -- 172.21.15.16:0/103067774 shutdown_connections 2024-09-19T07:50:08.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.291+0000 7f121454b640 1 -- 172.21.15.16:0/103067774 wait complete. 2024-09-19T07:50:08.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set' 2024-09-19T07:50:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.451+0000 7fd400f0c640 1 Processor -- start 2024-09-19T07:50:08.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.451+0000 7fd400f0c640 1 -- start start 2024-09-19T07:50:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.452+0000 7fd400f0c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3fc15ea80 0x7fd3fc0c7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.452+0000 7fd400f0c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3fc0c75a0 0x7fd3fc162580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.452+0000 7fd400f0c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3fc143b40 con 0x7fd3fc1621a0 2024-09-19T07:50:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.452+0000 7fd400f0c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd3fc1421a0 con 0x7fd3fc15ea80 2024-09-19T07:50:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.452+0000 7fd400f0c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd3fc15efb0 con 0x7fd3fc0c75a0 2024-09-19T07:50:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.452+0000 7fd3f9d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3fc15ea80 0x7fd3fc0c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.452+0000 7fd3f9d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3fc15ea80 0x7fd3fc0c7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38968/0 (socket says 172.21.15.16:38968) 2024-09-19T07:50:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.452+0000 7fd3f9d9b640 1 -- 172.21.15.16:0/4121913714 learned_addr learned my addr 172.21.15.16:0/4121913714 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.452+0000 7fd3fad9d640 1 --2- 172.21.15.16:0/4121913714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3fc0c75a0 0x7fd3fc162580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.452+0000 7fd3f9d9b640 1 -- 172.21.15.16:0/4121913714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3fc0c75a0 msgr2=0x7fd3fc162580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.452+0000 7fd3f9d9b640 1 --2- 172.21.15.16:0/4121913714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3fc0c75a0 0x7fd3fc162580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.452+0000 7fd3f9d9b640 1 -- 172.21.15.16:0/4121913714 >> v1:172.21.15.146:6789/0 conn(0x7fd3fc1621a0 legacy=0x7fd3fc141a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.452+0000 7fd3f9d9b640 1 -- 172.21.15.16:0/4121913714 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3fc15f290 con 0x7fd3fc15ea80 2024-09-19T07:50:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.454+0000 7fd3f9d9b640 1 --2- 172.21.15.16:0/4121913714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc15ea80 0x7fd3fc0c7060 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0009870 tx=0x7fd3f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a4efc0ca5f72c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.454+0000 7fd3f959a640 1 -- 172.21.15.16:0/4121913714 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0037070 con 0x7fd3fc15ea80 2024-09-19T07:50:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.454+0000 7fd3f959a640 1 -- 172.21.15.16:0/4121913714 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f0032440 con 0x7fd3fc15ea80 2024-09-19T07:50:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.454+0000 7fd3f959a640 1 -- 172.21.15.16:0/4121913714 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0032810 con 0x7fd3fc15ea80 2024-09-19T07:50:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.454+0000 7fd400f0c640 1 -- 172.21.15.16:0/4121913714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc15ea80 msgr2=0x7fd3fc0c7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.454+0000 7fd400f0c640 1 --2- 172.21.15.16:0/4121913714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc15ea80 0x7fd3fc0c7060 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0009870 tx=0x7fd3f002a270 comp rx=0 tx=0).stop 2024-09-19T07:50:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.454+0000 7fd400f0c640 1 -- 172.21.15.16:0/4121913714 shutdown_connections 2024-09-19T07:50:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.454+0000 7fd400f0c640 1 --2- 172.21.15.16:0/4121913714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3fc0c75a0 0x7fd3fc162580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.454+0000 7fd400f0c640 1 --2- 172.21.15.16:0/4121913714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc15ea80 0x7fd3fc0c7060 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.454+0000 7fd400f0c640 1 -- 172.21.15.16:0/4121913714 >> 172.21.15.16:0/4121913714 conn(0x7fd3fc067fe0 msgr2=0x7fd3fc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.454+0000 7fd400f0c640 1 -- 172.21.15.16:0/4121913714 shutdown_connections 2024-09-19T07:50:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.454+0000 7fd400f0c640 1 -- 172.21.15.16:0/4121913714 wait complete. 2024-09-19T07:50:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.456+0000 7fd400f0c640 1 Processor -- start 2024-09-19T07:50:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.456+0000 7fd400f0c640 1 -- start start 2024-09-19T07:50:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.456+0000 7fd400f0c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3fc0c75a0 0x7fd3fc15fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.456+0000 7fd400f0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc15ea80 0x7fd3fc140430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd400f0c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3fc1621a0 0x7fd3fc15ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd400f0c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc0cfa50 con 0x7fd3fc15ea80 2024-09-19T07:50:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd400f0c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc15f850 con 0x7fd3fc1621a0 2024-09-19T07:50:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd400f0c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd3fc1601c0 con 0x7fd3fc0c75a0 2024-09-19T07:50:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd3f9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc15ea80 0x7fd3fc140430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd3f9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc15ea80 0x7fd3fc140430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38976/0 (socket says 172.21.15.16:38976) 2024-09-19T07:50:08.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd3f9d9b640 1 -- 172.21.15.16:0/574678705 learned_addr learned my addr 172.21.15.16:0/574678705 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd3fa59c640 1 --2- 172.21.15.16:0/574678705 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3fc0c75a0 0x7fd3fc15fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd3fad9d640 1 --2- 172.21.15.16:0/574678705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3fc1621a0 0x7fd3fc15ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd3f9d9b640 1 -- 172.21.15.16:0/574678705 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3fc0c75a0 msgr2=0x7fd3fc15fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd3f9d9b640 1 --2- 172.21.15.16:0/574678705 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3fc0c75a0 0x7fd3fc15fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd3f9d9b640 1 -- 172.21.15.16:0/574678705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3fc1621a0 msgr2=0x7fd3fc15ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd3f9d9b640 1 --2- 172.21.15.16:0/574678705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3fc1621a0 0x7fd3fc15ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd3f9d9b640 1 -- 172.21.15.16:0/574678705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3fc1604a0 con 0x7fd3fc15ea80 2024-09-19T07:50:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd3f9d9b640 1 --2- 172.21.15.16:0/574678705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc15ea80 0x7fd3fc140430 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0000c00 tx=0x7fd3f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd3eaffd640 1 -- 172.21.15.16:0/574678705 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0037040 con 0x7fd3fc15ea80 2024-09-19T07:50:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd3eaffd640 1 -- 172.21.15.16:0/574678705 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f0042070 con 0x7fd3fc15ea80 2024-09-19T07:50:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd400f0c640 1 -- 172.21.15.16:0/574678705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3fc160780 con 0x7fd3fc15ea80 2024-09-19T07:50:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd3eaffd640 1 -- 172.21.15.16:0/574678705 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0030860 con 0x7fd3fc15ea80 2024-09-19T07:50:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.457+0000 7fd400f0c640 1 -- 172.21.15.16:0/574678705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3fc161260 con 0x7fd3fc15ea80 2024-09-19T07:50:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.458+0000 7fd400f0c640 1 -- 172.21.15.16:0/574678705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3fc0c7060 con 0x7fd3fc15ea80 2024-09-19T07:50:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.459+0000 7fd3eaffd640 1 -- 172.21.15.16:0/574678705 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd3f0030a00 con 0x7fd3fc15ea80 2024-09-19T07:50:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.459+0000 7fd3eaffd640 1 --2- 172.21.15.16:0/574678705 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd3dc06c2b0 0x7fd3dc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.460+0000 7fd3eaffd640 1 -- 172.21.15.16:0/574678705 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5335+0+0 (secure 0 0 0) 0x7fd3f00abaf0 con 0x7fd3fc15ea80 2024-09-19T07:50:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.460+0000 7fd3fa59c640 1 --2- 172.21.15.16:0/574678705 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd3dc06c2b0 0x7fd3dc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.460+0000 7fd3fa59c640 1 --2- 172.21.15.16:0/574678705 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd3dc06c2b0 0x7fd3dc06e770 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec007fd0 tx=0x7fd3ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:08.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.468+0000 7fd3eaffd640 1 -- 172.21.15.16:0/574678705 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd3f007a460 con 0x7fd3fc15ea80 2024-09-19T07:50:08.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:08.714+0000 7fd400f0c640 1 -- 172.21.15.16:0/574678705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]} v 0) -- 0x7fd3fc0cf600 con 0x7fd3fc15ea80 2024-09-19T07:50:09.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.665+0000 7fd3eaffd640 1 -- 172.21.15.16:0/574678705 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]}]=0 v315) ==== 163+0+0 (secure 0 0 0) 0x7fd3f0079d60 con 0x7fd3fc15ea80 2024-09-19T07:50:09.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.667+0000 7fd400f0c640 1 -- 172.21.15.16:0/574678705 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd3dc06c2b0 msgr2=0x7fd3dc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.667+0000 7fd400f0c640 1 --2- 172.21.15.16:0/574678705 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd3dc06c2b0 0x7fd3dc06e770 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec007fd0 tx=0x7fd3ec009040 comp rx=0 tx=0).stop 2024-09-19T07:50:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.668+0000 7fd400f0c640 1 -- 172.21.15.16:0/574678705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc15ea80 msgr2=0x7fd3fc140430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.668+0000 7fd400f0c640 1 --2- 172.21.15.16:0/574678705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc15ea80 0x7fd3fc140430 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0000c00 tx=0x7fd3f00046c0 comp rx=0 tx=0).stop 2024-09-19T07:50:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.668+0000 7fd400f0c640 1 -- 172.21.15.16:0/574678705 shutdown_connections 2024-09-19T07:50:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.668+0000 7fd400f0c640 1 --2- 172.21.15.16:0/574678705 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd3dc06c2b0 0x7fd3dc06e770 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.668+0000 7fd400f0c640 1 --2- 172.21.15.16:0/574678705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3fc1621a0 0x7fd3fc15ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.668+0000 7fd400f0c640 1 --2- 172.21.15.16:0/574678705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc15ea80 0x7fd3fc140430 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.668+0000 7fd400f0c640 1 --2- 172.21.15.16:0/574678705 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3fc0c75a0 0x7fd3fc15fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.668+0000 7fd400f0c640 1 -- 172.21.15.16:0/574678705 >> 172.21.15.16:0/574678705 conn(0x7fd3fc067fe0 msgr2=0x7fd3fc162580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.668+0000 7fd400f0c640 1 -- 172.21.15.16:0/574678705 shutdown_connections 2024-09-19T07:50:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.668+0000 7fd400f0c640 1 -- 172.21.15.16:0/574678705 wait complete. 2024-09-19T07:50:09.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T07:50:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.784+0000 7f484139d640 1 Processor -- start 2024-09-19T07:50:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.784+0000 7f484139d640 1 -- start start 2024-09-19T07:50:09.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.785+0000 7f484139d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f483c145a60 0x7f483c15c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:09.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.785+0000 7f484139d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f483c0b4040 0x7f483c0b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:09.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.785+0000 7f484139d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f483c0b6be0 con 0x7f483c06bc10 2024-09-19T07:50:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.785+0000 7f484139d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f483c0705b0 con 0x7f483c0b4040 2024-09-19T07:50:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.785+0000 7f484139d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f483c0b6ec0 con 0x7f483c145a60 2024-09-19T07:50:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.786+0000 7f483b7fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f483c0b4040 0x7f483c0b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.786+0000 7f483bfff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f483c145a60 0x7f483c15c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.786+0000 7f4840b9c640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f483c06bc10 0x7f483c070a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:37766/0 (socket says 172.21.15.16:37766) 2024-09-19T07:50:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.786+0000 7f4840b9c640 1 -- 172.21.15.16:0/234582902 learned_addr learned my addr 172.21.15.16:0/234582902 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.786+0000 7f483affd640 1 -- 172.21.15.16:0/234582902 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3737921105 0 0) 0x7f483c0b6be0 con 0x7f483c06bc10 2024-09-19T07:50:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.786+0000 7f483affd640 1 -- 172.21.15.16:0/234582902 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4824003520 con 0x7f483c06bc10 2024-09-19T07:50:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.786+0000 7f483bfff640 1 -- 172.21.15.16:0/234582902 >> v2:172.21.15.146:3300/0 conn(0x7f483c0b4040 msgr2=0x7f483c0b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.786+0000 7f483bfff640 1 --2- 172.21.15.16:0/234582902 >> v2:172.21.15.146:3300/0 conn(0x7f483c0b4040 0x7f483c0b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:09.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.786+0000 7f483bfff640 1 -- 172.21.15.16:0/234582902 >> v1:172.21.15.146:6789/0 conn(0x7f483c06bc10 legacy=0x7f483c070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:09.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.786+0000 7f483bfff640 1 -- 172.21.15.16:0/234582902 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483c070240 con 0x7f483c145a60 2024-09-19T07:50:09.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.786+0000 7f483b7fe640 1 --2- 172.21.15.16:0/234582902 >> v2:172.21.15.146:3300/0 conn(0x7f483c0b4040 0x7f483c0b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:09.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.787+0000 7f483bfff640 1 --2- 172.21.15.16:0/234582902 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f483c145a60 0x7f483c15c4d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f482c0097a0 tx=0x7f482c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=510ad2ea77105103 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:09.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.787+0000 7f483affd640 1 -- 172.21.15.16:0/234582902 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c037070 con 0x7f483c145a60 2024-09-19T07:50:09.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.787+0000 7f483affd640 1 -- 172.21.15.16:0/234582902 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f482c032440 con 0x7f483c145a60 2024-09-19T07:50:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.787+0000 7f483affd640 1 -- 172.21.15.16:0/234582902 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c0327f0 con 0x7f483c145a60 2024-09-19T07:50:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.788+0000 7f484139d640 1 -- 172.21.15.16:0/234582902 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f483c145a60 msgr2=0x7f483c15c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.788+0000 7f484139d640 1 --2- 172.21.15.16:0/234582902 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f483c145a60 0x7f483c15c4d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f482c0097a0 tx=0x7f482c02a180 comp rx=0 tx=0).stop 2024-09-19T07:50:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.788+0000 7f484139d640 1 -- 172.21.15.16:0/234582902 shutdown_connections 2024-09-19T07:50:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.788+0000 7f484139d640 1 --2- 172.21.15.16:0/234582902 >> v2:172.21.15.146:3300/0 conn(0x7f483c0b4040 0x7f483c0b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.788+0000 7f484139d640 1 --2- 172.21.15.16:0/234582902 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f483c145a60 0x7f483c15c4d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.788+0000 7f484139d640 1 -- 172.21.15.16:0/234582902 >> 172.21.15.16:0/234582902 conn(0x7f483c0b2c60 msgr2=0x7f483c0b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.788+0000 7f484139d640 1 -- 172.21.15.16:0/234582902 shutdown_connections 2024-09-19T07:50:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.788+0000 7f484139d640 1 -- 172.21.15.16:0/234582902 wait complete. 2024-09-19T07:50:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.789+0000 7f484139d640 1 Processor -- start 2024-09-19T07:50:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.789+0000 7f484139d640 1 -- start start 2024-09-19T07:50:09.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.790+0000 7f484139d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f483c06bc10 0x7f483c1e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.790+0000 7f484139d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f483c0b4040 0x7f483c144450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.790+0000 7f484139d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f483c145a60 0x7f483c144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.790+0000 7f484139d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f483c0b68f0 con 0x7f483c06bc10 2024-09-19T07:50:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.790+0000 7f484139d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f483c0b6770 con 0x7f483c145a60 2024-09-19T07:50:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.790+0000 7f484139d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f483c071150 con 0x7f483c0b4040 2024-09-19T07:50:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.790+0000 7f483bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f483c06bc10 0x7f483c1e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.790+0000 7f483bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f483c06bc10 0x7f483c1e5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38990/0 (socket says 172.21.15.16:38990) 2024-09-19T07:50:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.790+0000 7f483bfff640 1 -- 172.21.15.16:0/1435982373 learned_addr learned my addr 172.21.15.16:0/1435982373 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.790+0000 7f483b7fe640 1 --2- 172.21.15.16:0/1435982373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f483c0b4040 0x7f483c144450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.790+0000 7f4840b9c640 1 --2- 172.21.15.16:0/1435982373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f483c145a60 0x7f483c144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.790+0000 7f483bfff640 1 -- 172.21.15.16:0/1435982373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f483c0b4040 msgr2=0x7f483c144450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.790+0000 7f483bfff640 1 --2- 172.21.15.16:0/1435982373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f483c0b4040 0x7f483c144450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:09.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.790+0000 7f483bfff640 1 -- 172.21.15.16:0/1435982373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f483c145a60 msgr2=0x7f483c144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.790+0000 7f483bfff640 1 --2- 172.21.15.16:0/1435982373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f483c145a60 0x7f483c144990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.790+0000 7f483bfff640 1 -- 172.21.15.16:0/1435982373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483c1e62b0 con 0x7f483c06bc10 2024-09-19T07:50:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.790+0000 7f4840b9c640 1 --2- 172.21.15.16:0/1435982373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f483c145a60 0x7f483c144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.791+0000 7f483bfff640 1 --2- 172.21.15.16:0/1435982373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f483c06bc10 0x7f483c1e5b70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f482c02a1c0 tx=0x7f482c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.791+0000 7f4838ff9640 1 -- 172.21.15.16:0/1435982373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c041020 con 0x7f483c06bc10 2024-09-19T07:50:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.791+0000 7f4838ff9640 1 -- 172.21.15.16:0/1435982373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f482c03d620 con 0x7f483c06bc10 2024-09-19T07:50:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.791+0000 7f484139d640 1 -- 172.21.15.16:0/1435982373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483c1e6590 con 0x7f483c06bc10 2024-09-19T07:50:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.791+0000 7f4838ff9640 1 -- 172.21.15.16:0/1435982373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c037040 con 0x7f483c06bc10 2024-09-19T07:50:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.791+0000 7f484139d640 1 -- 172.21.15.16:0/1435982373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f483c1e6b50 con 0x7f483c06bc10 2024-09-19T07:50:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.792+0000 7f484139d640 1 -- 172.21.15.16:0/1435982373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f483c0b44a0 con 0x7f483c06bc10 2024-09-19T07:50:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.792+0000 7f4838ff9640 1 -- 172.21.15.16:0/1435982373 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f482c044020 con 0x7f483c06bc10 2024-09-19T07:50:09.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.793+0000 7f4838ff9640 1 --2- 172.21.15.16:0/1435982373 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f482806c2b0 0x7f482806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:09.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.793+0000 7f4838ff9640 1 -- 172.21.15.16:0/1435982373 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5335+0+0 (secure 0 0 0) 0x7f482c0ab910 con 0x7f483c06bc10 2024-09-19T07:50:09.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.793+0000 7f483b7fe640 1 --2- 172.21.15.16:0/1435982373 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f482806c2b0 0x7f482806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:09.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.794+0000 7f483b7fe640 1 --2- 172.21.15.16:0/1435982373 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f482806c2b0 0x7f482806e770 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4830009560 tx=0x7f4830009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:09.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.797+0000 7f4838ff9640 1 -- 172.21.15.16:0/1435982373 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f482c04b030 con 0x7f483c06bc10 2024-09-19T07:50:09.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.972+0000 7f484139d640 1 -- 172.21.15.16:0/1435982373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f483c1e8de0 con 0x7f483c06bc10 2024-09-19T07:50:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.975+0000 7f4838ff9640 1 -- 172.21.15.16:0/1435982373 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v31)=0 v31) ==== 122+0+0 (secure 0 0 0) 0x7f482c07a200 con 0x7f483c06bc10 2024-09-19T07:50:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.977+0000 7f484139d640 1 -- 172.21.15.16:0/1435982373 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f482806c2b0 msgr2=0x7f482806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.977+0000 7f484139d640 1 --2- 172.21.15.16:0/1435982373 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f482806c2b0 0x7f482806e770 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4830009560 tx=0x7f4830009190 comp rx=0 tx=0).stop 2024-09-19T07:50:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.977+0000 7f484139d640 1 -- 172.21.15.16:0/1435982373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f483c06bc10 msgr2=0x7f483c1e5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.977+0000 7f484139d640 1 --2- 172.21.15.16:0/1435982373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f483c06bc10 0x7f483c1e5b70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f482c02a1c0 tx=0x7f482c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:50:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.977+0000 7f484139d640 1 -- 172.21.15.16:0/1435982373 shutdown_connections 2024-09-19T07:50:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.977+0000 7f484139d640 1 --2- 172.21.15.16:0/1435982373 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f482806c2b0 0x7f482806e770 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.977+0000 7f484139d640 1 --2- 172.21.15.16:0/1435982373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f483c145a60 0x7f483c144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.977+0000 7f484139d640 1 --2- 172.21.15.16:0/1435982373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f483c0b4040 0x7f483c144450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.977+0000 7f484139d640 1 --2- 172.21.15.16:0/1435982373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f483c06bc10 0x7f483c1e5b70 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.977+0000 7f484139d640 1 -- 172.21.15.16:0/1435982373 >> 172.21.15.16:0/1435982373 conn(0x7f483c0b2c60 msgr2=0x7f483c0b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.977+0000 7f484139d640 1 -- 172.21.15.16:0/1435982373 shutdown_connections 2024-09-19T07:50:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:09.977+0000 7f484139d640 1 -- 172.21.15.16:0/1435982373 wait complete. 2024-09-19T07:50:09.988 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:50:09.988 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:50:10.050 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:50:10.097 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:50:10.097 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:50:10.097 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:50:10.113 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:50:10.113 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:50:10.113 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:50:10.131 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:50:10.131 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:50:10.131 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:50:10.169 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:50:10.169 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:50:10.169 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:50:10.188 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:50:10.188 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T07:50:10.188 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:50:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.310+0000 7f1c72ad2640 1 Processor -- start 2024-09-19T07:50:10.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.310+0000 7f1c72ad2640 1 -- start start 2024-09-19T07:50:10.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.311+0000 7f1c72ad2640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1c6c161eb0 0x7f1c6c1622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:10.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.311+0000 7f1c72ad2640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c6c1425e0 0x7f1c6c142a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:10.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.311+0000 7f1c72ad2640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c6c162b70 con 0x7f1c6c06cd20 2024-09-19T07:50:10.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.311+0000 7f1c72ad2640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f1c6c162e50 con 0x7f1c6c161eb0 2024-09-19T07:50:10.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.311+0000 7f1c72ad2640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1c6c142150 con 0x7f1c6c1425e0 2024-09-19T07:50:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.311+0000 7f1c7086e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1c6c06cd20 0x7f1c6c06d100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35668/0 (socket says 172.21.15.16:35668) 2024-09-19T07:50:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.311+0000 7f1c7086e640 1 -- 172.21.15.16:0/2143949484 learned_addr learned my addr 172.21.15.16:0/2143949484 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.311+0000 7f1c7106f640 1 --2- 172.21.15.16:0/2143949484 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c6c1425e0 0x7f1c6c142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.311+0000 7f1c6bfff640 1 --2- 172.21.15.16:0/2143949484 >> v2:172.21.15.146:3300/0 conn(0x7f1c6c161eb0 0x7f1c6c1622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.311+0000 7f1c6b7fe640 1 -- 172.21.15.16:0/2143949484 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4217335493 0 0) 0x7f1c6c162b70 con 0x7f1c6c06cd20 2024-09-19T07:50:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.311+0000 7f1c6b7fe640 1 -- 172.21.15.16:0/2143949484 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c54003520 con 0x7f1c6c06cd20 2024-09-19T07:50:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6b7fe640 1 -- 172.21.15.16:0/2143949484 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3434923101 0 0) 0x7f1c54003520 con 0x7f1c6c06cd20 2024-09-19T07:50:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6b7fe640 1 -- 172.21.15.16:0/2143949484 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c6c1425e0 msgr2=0x7f1c6c142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6b7fe640 1 --2- 172.21.15.16:0/2143949484 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c6c1425e0 0x7f1c6c142a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6b7fe640 1 -- 172.21.15.16:0/2143949484 >> v2:172.21.15.146:3300/0 conn(0x7f1c6c161eb0 msgr2=0x7f1c6c1622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6b7fe640 1 --2- 172.21.15.16:0/2143949484 >> v2:172.21.15.146:3300/0 conn(0x7f1c6c161eb0 0x7f1c6c1622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6b7fe640 1 -- 172.21.15.16:0/2143949484 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6c141b00 con 0x7f1c6c06cd20 2024-09-19T07:50:10.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6b7fe640 1 -- 172.21.15.16:0/2143949484 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f1c640019f0 con 0x7f1c6c06cd20 2024-09-19T07:50:10.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6b7fe640 1 -- 172.21.15.16:0/2143949484 >> v1:172.21.15.16:6789/0 conn(0x7f1c6c06cd20 legacy=0x7f1c6c06d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:10.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6b7fe640 1 --2- 172.21.15.16:0/2143949484 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c6c1425e0 0x7f1c540039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:10.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6b7fe640 1 --2- 172.21.15.16:0/2143949484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c54003f30 0x7f1c54004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:10.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6b7fe640 1 --2- 172.21.15.16:0/2143949484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c54004d40 0x7f1c540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:10.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6b7fe640 1 -- 172.21.15.16:0/2143949484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c162e50 con 0x7f1c54003f30 2024-09-19T07:50:10.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6b7fe640 1 -- 172.21.15.16:0/2143949484 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c142150 con 0x7f1c54004d40 2024-09-19T07:50:10.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6b7fe640 1 -- 172.21.15.16:0/2143949484 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1c5400ba10 con 0x7f1c6c1425e0 2024-09-19T07:50:10.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6bfff640 1 --2- 172.21.15.16:0/2143949484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c54003f30 0x7f1c54004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:10.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c7086e640 1 --2- 172.21.15.16:0/2143949484 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c6c1425e0 0x7f1c540039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:10.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c7106f640 1 --2- 172.21.15.16:0/2143949484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c54004d40 0x7f1c540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:10.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6bfff640 1 -- 172.21.15.16:0/2143949484 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c6c1425e0 msgr2=0x7f1c540039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:10.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6bfff640 1 --2- 172.21.15.16:0/2143949484 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c6c1425e0 0x7f1c540039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:10.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6bfff640 1 -- 172.21.15.16:0/2143949484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c54004d40 msgr2=0x7f1c540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:10.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6bfff640 1 --2- 172.21.15.16:0/2143949484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c54004d40 0x7f1c540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:10.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.312+0000 7f1c6bfff640 1 -- 172.21.15.16:0/2143949484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c54003520 con 0x7f1c54003f30 2024-09-19T07:50:10.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.313+0000 7f1c6bfff640 1 --2- 172.21.15.16:0/2143949484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c54003f30 0x7f1c54004800 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009af0 tx=0x7f1c54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4912880a9afa0416 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:10.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.313+0000 7f1c6b7fe640 1 -- 172.21.15.16:0/2143949484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60036070 con 0x7f1c54003f30 2024-09-19T07:50:10.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.313+0000 7f1c6b7fe640 1 -- 172.21.15.16:0/2143949484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c60031400 con 0x7f1c54003f30 2024-09-19T07:50:10.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.313+0000 7f1c6b7fe640 1 -- 172.21.15.16:0/2143949484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60009690 con 0x7f1c54003f30 2024-09-19T07:50:10.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.314+0000 7f1c72ad2640 1 -- 172.21.15.16:0/2143949484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c54003f30 msgr2=0x7f1c54004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:10.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.314+0000 7f1c72ad2640 1 --2- 172.21.15.16:0/2143949484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c54003f30 0x7f1c54004800 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009af0 tx=0x7f1c54005830 comp rx=0 tx=0).stop 2024-09-19T07:50:10.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.314+0000 7f1c7106f640 1 -- 172.21.15.16:0/2143949484 reap_dead start 2024-09-19T07:50:10.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.314+0000 7f1c72ad2640 1 -- 172.21.15.16:0/2143949484 shutdown_connections 2024-09-19T07:50:10.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.314+0000 7f1c72ad2640 1 -- 172.21.15.16:0/2143949484 >> 172.21.15.16:0/2143949484 conn(0x7f1c6c0b3ed0 msgr2=0x7f1c6c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:10.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.314+0000 7f1c72ad2640 1 -- 172.21.15.16:0/2143949484 shutdown_connections 2024-09-19T07:50:10.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.314+0000 7f1c72ad2640 1 -- 172.21.15.16:0/2143949484 wait complete. 2024-09-19T07:50:10.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.315+0000 7f1c72ad2640 1 Processor -- start 2024-09-19T07:50:10.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.315+0000 7f1c72ad2640 1 -- start start 2024-09-19T07:50:10.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.316+0000 7f1c72ad2640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c54003f30 0x7f1c6c140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:10.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.316+0000 7f1c72ad2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0b7bb0 0x7f1c6c0c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:10.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.316+0000 7f1c72ad2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c6c0c8800 0x7f1c6c0c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:10.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.316+0000 7f1c72ad2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c141600 con 0x7f1c6c0b7bb0 2024-09-19T07:50:10.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.316+0000 7f1c72ad2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c162830 con 0x7f1c6c0c8800 2024-09-19T07:50:10.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.316+0000 7f1c72ad2640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1c6c142a70 con 0x7f1c54003f30 2024-09-19T07:50:10.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.316+0000 7f1c6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0b7bb0 0x7f1c6c0c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:10.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.316+0000 7f1c6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0b7bb0 0x7f1c6c0c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39012/0 (socket says 172.21.15.16:39012) 2024-09-19T07:50:10.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.316+0000 7f1c6bfff640 1 -- 172.21.15.16:0/1892976965 learned_addr learned my addr 172.21.15.16:0/1892976965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:10.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.316+0000 7f1c7086e640 1 --2- 172.21.15.16:0/1892976965 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c54003f30 0x7f1c6c140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:10.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.316+0000 7f1c7106f640 1 --2- 172.21.15.16:0/1892976965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c6c0c8800 0x7f1c6c0c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:10.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.316+0000 7f1c6bfff640 1 -- 172.21.15.16:0/1892976965 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c54003f30 msgr2=0x7f1c6c140be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:10.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.316+0000 7f1c6bfff640 1 --2- 172.21.15.16:0/1892976965 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c54003f30 0x7f1c6c140be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:10.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.316+0000 7f1c6bfff640 1 -- 172.21.15.16:0/1892976965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c6c0c8800 msgr2=0x7f1c6c0c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:10.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.316+0000 7f1c6bfff640 1 --2- 172.21.15.16:0/1892976965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c6c0c8800 0x7f1c6c0c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:10.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.316+0000 7f1c6bfff640 1 -- 172.21.15.16:0/1892976965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6c146c00 con 0x7f1c6c0b7bb0 2024-09-19T07:50:10.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.317+0000 7f1c6bfff640 1 --2- 172.21.15.16:0/1892976965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0b7bb0 0x7f1c6c0c82c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009c10 tx=0x7f1c60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:10.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.317+0000 7f1c697fa640 1 -- 172.21.15.16:0/1892976965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60040020 con 0x7f1c6c0b7bb0 2024-09-19T07:50:10.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.317+0000 7f1c697fa640 1 -- 172.21.15.16:0/1892976965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c60003610 con 0x7f1c6c0b7bb0 2024-09-19T07:50:10.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.317+0000 7f1c72ad2640 1 -- 172.21.15.16:0/1892976965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6c0b6220 con 0x7f1c6c0b7bb0 2024-09-19T07:50:10.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.317+0000 7f1c697fa640 1 -- 172.21.15.16:0/1892976965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60036040 con 0x7f1c6c0b7bb0 2024-09-19T07:50:10.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.317+0000 7f1c72ad2640 1 -- 172.21.15.16:0/1892976965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c6c0b67e0 con 0x7f1c6c0b7bb0 2024-09-19T07:50:10.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.318+0000 7f1c72ad2640 1 -- 172.21.15.16:0/1892976965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c6c162650 con 0x7f1c6c0b7bb0 2024-09-19T07:50:10.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.318+0000 7f1c697fa640 1 -- 172.21.15.16:0/1892976965 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1c60043020 con 0x7f1c6c0b7bb0 2024-09-19T07:50:10.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.319+0000 7f1c697fa640 1 --2- 172.21.15.16:0/1892976965 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1c5806c2b0 0x7f1c5806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:10.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.319+0000 7f1c697fa640 1 -- 172.21.15.16:0/1892976965 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5335+0+0 (secure 0 0 0) 0x7f1c600ab340 con 0x7f1c6c0b7bb0 2024-09-19T07:50:10.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.319+0000 7f1c7086e640 1 --2- 172.21.15.16:0/1892976965 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1c5806c2b0 0x7f1c5806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:10.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.320+0000 7f1c7086e640 1 --2- 172.21.15.16:0/1892976965 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1c5806c2b0 0x7f1c5806e770 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1c64004d00 tx=0x7f1c64006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:10.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.324+0000 7f1c697fa640 1 -- 172.21.15.16:0/1892976965 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1c6c162650 con 0x7f1c6c0b7bb0 2024-09-19T07:50:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.538+0000 7f1c72ad2640 1 -- 172.21.15.16:0/1892976965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c6c0c8be0 con 0x7f1c6c0b7bb0 2024-09-19T07:50:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.538+0000 7f1c697fa640 1 -- 172.21.15.16:0/1892976965 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+3908 (secure 0 0 0) 0x7f1c6004a030 con 0x7f1c6c0b7bb0 2024-09-19T07:50:10.539 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:10.539 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":43,"btime":"2024-09-19T07:50:03:646715+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/945862061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":945862061},{"type":"v1","addr":"172.21.15.146:6836","nonce":945862061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6022,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1656482127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1656482127},{"type":"v1","addr":"172.21.15.146:6837","nonce":1656482127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/42753795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":42753795},{"type":"v1","addr":"172.21.15.16:6834","nonce":42753795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6081,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1389787480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1389787480},{"type":"v1","addr":"172.21.15.16:6835","nonce":1389787480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[]} 2024-09-19T07:50:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 43 2024-09-19T07:50:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.541+0000 7f1c72ad2640 1 -- 172.21.15.16:0/1892976965 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1c5806c2b0 msgr2=0x7f1c5806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.541+0000 7f1c72ad2640 1 --2- 172.21.15.16:0/1892976965 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1c5806c2b0 0x7f1c5806e770 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1c64004d00 tx=0x7f1c64006040 comp rx=0 tx=0).stop 2024-09-19T07:50:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.541+0000 7f1c72ad2640 1 -- 172.21.15.16:0/1892976965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0b7bb0 msgr2=0x7f1c6c0c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.541+0000 7f1c72ad2640 1 --2- 172.21.15.16:0/1892976965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0b7bb0 0x7f1c6c0c82c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009c10 tx=0x7f1c60004730 comp rx=0 tx=0).stop 2024-09-19T07:50:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.541+0000 7f1c72ad2640 1 -- 172.21.15.16:0/1892976965 shutdown_connections 2024-09-19T07:50:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.541+0000 7f1c72ad2640 1 --2- 172.21.15.16:0/1892976965 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1c5806c2b0 0x7f1c5806e770 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.541+0000 7f1c72ad2640 1 --2- 172.21.15.16:0/1892976965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c6c0c8800 0x7f1c6c0c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.541+0000 7f1c72ad2640 1 --2- 172.21.15.16:0/1892976965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c0b7bb0 0x7f1c6c0c82c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.541+0000 7f1c72ad2640 1 --2- 172.21.15.16:0/1892976965 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c54003f30 0x7f1c6c140be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.541+0000 7f1c72ad2640 1 -- 172.21.15.16:0/1892976965 >> 172.21.15.16:0/1892976965 conn(0x7f1c6c0b3ed0 msgr2=0x7f1c6c0b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.541+0000 7f1c72ad2640 1 -- 172.21.15.16:0/1892976965 shutdown_connections 2024-09-19T07:50:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.541+0000 7f1c72ad2640 1 -- 172.21.15.16:0/1892976965 wait complete. 2024-09-19T07:50:10.552 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T07:50:10.552 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T07:50:10.553 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T07:50:10.553 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:50:10.553 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T07:50:10.553 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T07:50:10.553 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T07:50:10.553 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:50:10.553 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T07:50:10.554 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T07:50:10.554 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T07:50:10.554 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:50:10.554 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T07:50:10.554 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T07:50:10.554 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T07:50:10.554 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:50:10.562 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T07:50:10.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T07:50:10.563 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T07:50:10.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T07:50:10.567 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T07:50:10.567 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T07:50:10.568 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T07:50:10.568 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T07:50:10.606 INFO:tasks.ceph.mds.c:Started 2024-09-19T07:50:10.606 DEBUG:teuthology.parallel:result is None 2024-09-19T07:50:10.606 INFO:tasks.ceph.mds.a:Started 2024-09-19T07:50:10.607 DEBUG:teuthology.parallel:result is None 2024-09-19T07:50:10.609 INFO:tasks.ceph.mds.d:Started 2024-09-19T07:50:10.609 DEBUG:teuthology.parallel:result is None 2024-09-19T07:50:10.610 INFO:tasks.ceph.mds.b:Started 2024-09-19T07:50:10.610 DEBUG:teuthology.parallel:result is None 2024-09-19T07:50:10.610 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T07:50:10.683 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-19T07:50:10.684 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-19T07:50:10.693 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2024-09-19T07:50:10.694 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2024-09-19T07:50:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.718+0000 7f73bfffd640 1 Processor -- start 2024-09-19T07:50:10.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.719+0000 7f73bfffd640 1 -- start start 2024-09-19T07:50:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.719+0000 7f73bfffd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73b80d4840 0x7f73b815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.720+0000 7f73bfffd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b80d1b10 con 0x7f73b815f0d0 2024-09-19T07:50:10.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.720+0000 7f73bfffd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b80cf2b0 con 0x7f73b80d0070 2024-09-19T07:50:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.720+0000 7f73bfffd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f73b80cf590 con 0x7f73b80d4840 2024-09-19T07:50:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.720+0000 7f73bdd99640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f73b815f0d0 0x7f73b80d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35682/0 (socket says 172.21.15.16:35682) 2024-09-19T07:50:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.720+0000 7f73bdd99640 1 -- 172.21.15.16:0/2177498661 learned_addr learned my addr 172.21.15.16:0/2177498661 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.720+0000 7f73be59a640 1 --2- 172.21.15.16:0/2177498661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73b80d4840 0x7f73b815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.720+0000 7f73bcd97640 1 -- 172.21.15.16:0/2177498661 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3938982864 0 0) 0x7f73b80d1b10 con 0x7f73b815f0d0 2024-09-19T07:50:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.720+0000 7f73bcd97640 1 -- 172.21.15.16:0/2177498661 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73a4003520 con 0x7f73b815f0d0 2024-09-19T07:50:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.720+0000 7f73bcd97640 1 -- 172.21.15.16:0/2177498661 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2608169462 0 0) 0x7f73b80cf2b0 con 0x7f73b80d0070 2024-09-19T07:50:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.720+0000 7f73be59a640 1 -- 172.21.15.16:0/2177498661 >> v1:172.21.15.146:6789/0 conn(0x7f73b80d0070 legacy=0x7f73b80d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.720+0000 7f73be59a640 1 -- 172.21.15.16:0/2177498661 >> v1:172.21.15.16:6789/0 conn(0x7f73b815f0d0 legacy=0x7f73b80d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.720+0000 7f73be59a640 1 -- 172.21.15.16:0/2177498661 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b80cf870 con 0x7f73b80d4840 2024-09-19T07:50:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.721+0000 7f73be59a640 1 --2- 172.21.15.16:0/2177498661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73b80d4840 0x7f73b815f4b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009df0 tx=0x7f73b002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e4d7342845f0b84 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.721+0000 7f73bcd97640 1 -- 172.21.15.16:0/2177498661 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b002ae60 con 0x7f73b80d4840 2024-09-19T07:50:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.721+0000 7f73bcd97640 1 -- 172.21.15.16:0/2177498661 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73b0007740 con 0x7f73b80d4840 2024-09-19T07:50:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.721+0000 7f73bcd97640 1 -- 172.21.15.16:0/2177498661 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b0007b10 con 0x7f73b80d4840 2024-09-19T07:50:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.721+0000 7f73bfffd640 1 -- 172.21.15.16:0/2177498661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73b80d4840 msgr2=0x7f73b815f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.721+0000 7f73bfffd640 1 --2- 172.21.15.16:0/2177498661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73b80d4840 0x7f73b815f4b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009df0 tx=0x7f73b002a680 comp rx=0 tx=0).stop 2024-09-19T07:50:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.722+0000 7f73bfffd640 1 -- 172.21.15.16:0/2177498661 shutdown_connections 2024-09-19T07:50:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.722+0000 7f73bfffd640 1 --2- 172.21.15.16:0/2177498661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73b80d4840 0x7f73b815f4b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.722+0000 7f73bfffd640 1 -- 172.21.15.16:0/2177498661 >> 172.21.15.16:0/2177498661 conn(0x7f73b809d760 msgr2=0x7f73b808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.722+0000 7f73bfffd640 1 -- 172.21.15.16:0/2177498661 shutdown_connections 2024-09-19T07:50:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.722+0000 7f73bfffd640 1 -- 172.21.15.16:0/2177498661 wait complete. 2024-09-19T07:50:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.723+0000 7f73bfffd640 1 Processor -- start 2024-09-19T07:50:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.723+0000 7f73bfffd640 1 -- start start 2024-09-19T07:50:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73bfffd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73b80d0070 0x7f73b8159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73bfffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73b80d4840 0x7f73b80b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73bfffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73b815f0d0 0x7f73b80b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73bfffd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73b80b2f70 con 0x7f73b815f0d0 2024-09-19T07:50:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73bfffd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f73b80cef60 con 0x7f73b80d4840 2024-09-19T07:50:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73bfffd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f73b80b30f0 con 0x7f73b80d0070 2024-09-19T07:50:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73be59a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73b815f0d0 0x7f73b80b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73be59a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73b815f0d0 0x7f73b80b44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39064/0 (socket says 172.21.15.16:39064) 2024-09-19T07:50:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73be59a640 1 -- 172.21.15.16:0/2126177609 learned_addr learned my addr 172.21.15.16:0/2126177609 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73bdd99640 1 --2- 172.21.15.16:0/2126177609 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73b80d0070 0x7f73b8159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73bd598640 1 --2- 172.21.15.16:0/2126177609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73b80d4840 0x7f73b80b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73be59a640 1 -- 172.21.15.16:0/2126177609 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73b80d0070 msgr2=0x7f73b8159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73be59a640 1 --2- 172.21.15.16:0/2126177609 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73b80d0070 0x7f73b8159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73be59a640 1 -- 172.21.15.16:0/2126177609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73b80d4840 msgr2=0x7f73b80b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73be59a640 1 --2- 172.21.15.16:0/2126177609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73b80d4840 0x7f73b80b7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73be59a640 1 -- 172.21.15.16:0/2126177609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b80caf80 con 0x7f73b815f0d0 2024-09-19T07:50:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73be59a640 1 --2- 172.21.15.16:0/2126177609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73b815f0d0 0x7f73b80b44c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009f10 tx=0x7f73b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73ae7fc640 1 -- 172.21.15.16:0/2126177609 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b0040070 con 0x7f73b815f0d0 2024-09-19T07:50:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73ae7fc640 1 -- 172.21.15.16:0/2126177609 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73b002ae60 con 0x7f73b815f0d0 2024-09-19T07:50:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73bfffd640 1 -- 172.21.15.16:0/2126177609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b80b8530 con 0x7f73b815f0d0 2024-09-19T07:50:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.724+0000 7f73ae7fc640 1 -- 172.21.15.16:0/2126177609 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b0009450 con 0x7f73b815f0d0 2024-09-19T07:50:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.725+0000 7f73bfffd640 1 -- 172.21.15.16:0/2126177609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b80b8af0 con 0x7f73b815f0d0 2024-09-19T07:50:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.725+0000 7f73bfffd640 1 -- 172.21.15.16:0/2126177609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b80b8cf0 con 0x7f73b815f0d0 2024-09-19T07:50:10.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.726+0000 7f73ae7fc640 1 -- 172.21.15.16:0/2126177609 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f73b0039040 con 0x7f73b815f0d0 2024-09-19T07:50:10.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.727+0000 7f73ae7fc640 1 --2- 172.21.15.16:0/2126177609 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f739c06c2b0 0x7f739c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:10.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.727+0000 7f73ae7fc640 1 -- 172.21.15.16:0/2126177609 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5335+0+0 (secure 0 0 0) 0x7f73b00ac830 con 0x7f73b815f0d0 2024-09-19T07:50:10.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.727+0000 7f73bdd99640 1 --2- 172.21.15.16:0/2126177609 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f739c06c2b0 0x7f739c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:10.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.727+0000 7f73bdd99640 1 --2- 172.21.15.16:0/2126177609 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f739c06c2b0 0x7f739c06e770 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f73a80011e0 tx=0x7f73a8007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:10.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.731+0000 7f73ae7fc640 1 -- 172.21.15.16:0/2126177609 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f73b80b8cf0 con 0x7f73b815f0d0 2024-09-19T07:50:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:10.919+0000 7f73bfffd640 1 -- 172.21.15.16:0/2126177609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f73b80c7cb0 con 0x7f73b815f0d0 2024-09-19T07:50:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:11.988+0000 7f73ae7fc640 1 -- 172.21.15.16:0/2126177609 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v89) ==== 97+0+0 (secure 0 0 0) 0x7f73b007aad0 con 0x7f73b815f0d0 2024-09-19T07:50:11.989 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-19T07:50:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:11.991+0000 7f73bfffd640 1 -- 172.21.15.16:0/2126177609 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f739c06c2b0 msgr2=0x7f739c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:11.991+0000 7f73bfffd640 1 --2- 172.21.15.16:0/2126177609 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f739c06c2b0 0x7f739c06e770 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f73a80011e0 tx=0x7f73a8007d30 comp rx=0 tx=0).stop 2024-09-19T07:50:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:11.991+0000 7f73bfffd640 1 -- 172.21.15.16:0/2126177609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73b815f0d0 msgr2=0x7f73b80b44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:11.991+0000 7f73bfffd640 1 --2- 172.21.15.16:0/2126177609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73b815f0d0 0x7f73b80b44c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009f10 tx=0x7f73b0004a40 comp rx=0 tx=0).stop 2024-09-19T07:50:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:11.992+0000 7f73bfffd640 1 -- 172.21.15.16:0/2126177609 shutdown_connections 2024-09-19T07:50:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:11.992+0000 7f73bfffd640 1 --2- 172.21.15.16:0/2126177609 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f739c06c2b0 0x7f739c06e770 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:11.992+0000 7f73bfffd640 1 --2- 172.21.15.16:0/2126177609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73b815f0d0 0x7f73b80b44c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:11.992+0000 7f73bfffd640 1 --2- 172.21.15.16:0/2126177609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73b80d4840 0x7f73b80b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:11.992+0000 7f73bfffd640 1 --2- 172.21.15.16:0/2126177609 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73b80d0070 0x7f73b8159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:11.992+0000 7f73bfffd640 1 -- 172.21.15.16:0/2126177609 >> 172.21.15.16:0/2126177609 conn(0x7f73b809d760 msgr2=0x7f73b80d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:11.992+0000 7f73bfffd640 1 -- 172.21.15.16:0/2126177609 shutdown_connections 2024-09-19T07:50:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:11.992+0000 7f73bfffd640 1 -- 172.21.15.16:0/2126177609 wait complete. 2024-09-19T07:50:12.006 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T07:50:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.114+0000 7f17d79a6640 1 Processor -- start 2024-09-19T07:50:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.114+0000 7f17d79a6640 1 -- start start 2024-09-19T07:50:12.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.115+0000 7f17d79a6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f17d015b120 0x7f17d006f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:12.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.115+0000 7f17d79a6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f17d006dec0 0x7f17d006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:12.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.115+0000 7f17d79a6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17d015b500 con 0x7f17d00d04b0 2024-09-19T07:50:12.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.115+0000 7f17d79a6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f17d006e940 con 0x7f17d015b120 2024-09-19T07:50:12.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.115+0000 7f17d79a6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f17d006ec20 con 0x7f17d006dec0 2024-09-19T07:50:12.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.115+0000 7f17d5742640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f17d015b120 0x7f17d006f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:12.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.115+0000 7f17d5742640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f17d015b120 0x7f17d006f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39078/0 (socket says 172.21.15.16:39078) 2024-09-19T07:50:12.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.115+0000 7f17d5742640 1 -- 172.21.15.16:0/1164265592 learned_addr learned my addr 172.21.15.16:0/1164265592 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:12.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.115+0000 7f17d4f41640 1 --2- 172.21.15.16:0/1164265592 >> v2:172.21.15.146:3300/0 conn(0x7f17d006dec0 0x7f17d006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:12.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.115+0000 7f17d5742640 1 -- 172.21.15.16:0/1164265592 >> v2:172.21.15.146:3300/0 conn(0x7f17d006dec0 msgr2=0x7f17d006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:12.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.116+0000 7f17d5742640 1 --2- 172.21.15.16:0/1164265592 >> v2:172.21.15.146:3300/0 conn(0x7f17d006dec0 0x7f17d006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.116+0000 7f17c7fff640 1 -- 172.21.15.16:0/1164265592 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3431210386 0 0) 0x7f17d015b500 con 0x7f17d00d04b0 2024-09-19T07:50:12.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.116+0000 7f17d5742640 1 -- 172.21.15.16:0/1164265592 >> v1:172.21.15.16:6789/0 conn(0x7f17d00d04b0 legacy=0x7f17d00c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:12.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.116+0000 7f17d5742640 1 -- 172.21.15.16:0/1164265592 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d006ef00 con 0x7f17d015b120 2024-09-19T07:50:12.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.116+0000 7f17d4f41640 1 --2- 172.21.15.16:0/1164265592 >> v2:172.21.15.146:3300/0 conn(0x7f17d006dec0 0x7f17d006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:12.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.116+0000 7f17d5742640 1 --2- 172.21.15.16:0/1164265592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d015b120 0x7f17d006f9e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f17d006dcd0 tx=0x7f17cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605c6bad25d25218 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:12.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.116+0000 7f17c7fff640 1 -- 172.21.15.16:0/1164265592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc037070 con 0x7f17d015b120 2024-09-19T07:50:12.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.116+0000 7f17c7fff640 1 -- 172.21.15.16:0/1164265592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17cc032440 con 0x7f17d015b120 2024-09-19T07:50:12.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.116+0000 7f17c7fff640 1 -- 172.21.15.16:0/1164265592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc032810 con 0x7f17d015b120 2024-09-19T07:50:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.117+0000 7f17d79a6640 1 -- 172.21.15.16:0/1164265592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d015b120 msgr2=0x7f17d006f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.117+0000 7f17d79a6640 1 --2- 172.21.15.16:0/1164265592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d015b120 0x7f17d006f9e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f17d006dcd0 tx=0x7f17cc02a270 comp rx=0 tx=0).stop 2024-09-19T07:50:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.117+0000 7f17d79a6640 1 -- 172.21.15.16:0/1164265592 shutdown_connections 2024-09-19T07:50:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.117+0000 7f17d79a6640 1 --2- 172.21.15.16:0/1164265592 >> v2:172.21.15.146:3300/0 conn(0x7f17d006dec0 0x7f17d006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.117+0000 7f17d79a6640 1 --2- 172.21.15.16:0/1164265592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d015b120 0x7f17d006f9e0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.117+0000 7f17d79a6640 1 -- 172.21.15.16:0/1164265592 >> 172.21.15.16:0/1164265592 conn(0x7f17d006bdf0 msgr2=0x7f17d006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.117+0000 7f17d79a6640 1 -- 172.21.15.16:0/1164265592 shutdown_connections 2024-09-19T07:50:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.117+0000 7f17d79a6640 1 -- 172.21.15.16:0/1164265592 wait complete. 2024-09-19T07:50:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.118+0000 7f17d79a6640 1 Processor -- start 2024-09-19T07:50:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.118+0000 7f17d79a6640 1 -- start start 2024-09-19T07:50:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.119+0000 7f17d79a6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17d006dec0 0x7f17d015e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.119+0000 7f17d79a6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17d00d04b0 0x7f17d0151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.119+0000 7f17d79a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d015b120 0x7f17d015b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.119+0000 7f17d79a6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17d00ce060 con 0x7f17d015b120 2024-09-19T07:50:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.119+0000 7f17d79a6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f17d006f4c0 con 0x7f17d006dec0 2024-09-19T07:50:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.119+0000 7f17d79a6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f17d00d0900 con 0x7f17d00d04b0 2024-09-19T07:50:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.119+0000 7f17d5f43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d015b120 0x7f17d015b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.119+0000 7f17d5f43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d015b120 0x7f17d015b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39090/0 (socket says 172.21.15.16:39090) 2024-09-19T07:50:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.119+0000 7f17d5f43640 1 -- 172.21.15.16:0/2068677880 learned_addr learned my addr 172.21.15.16:0/2068677880 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.119+0000 7f17d5742640 1 --2- 172.21.15.16:0/2068677880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17d006dec0 0x7f17d015e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.119+0000 7f17d5f43640 1 -- 172.21.15.16:0/2068677880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17d00d04b0 msgr2=0x7f17d0151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.119+0000 7f17d4f41640 1 --2- 172.21.15.16:0/2068677880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17d00d04b0 0x7f17d0151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.119+0000 7f17d5f43640 1 --2- 172.21.15.16:0/2068677880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17d00d04b0 0x7f17d0151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.119+0000 7f17d5f43640 1 -- 172.21.15.16:0/2068677880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17d006dec0 msgr2=0x7f17d015e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.120+0000 7f17d5f43640 1 --2- 172.21.15.16:0/2068677880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17d006dec0 0x7f17d015e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.120+0000 7f17d5f43640 1 -- 172.21.15.16:0/2068677880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d00b0e10 con 0x7f17d015b120 2024-09-19T07:50:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.120+0000 7f17d4f41640 1 --2- 172.21.15.16:0/2068677880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17d00d04b0 0x7f17d0151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:12.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.120+0000 7f17d5742640 1 --2- 172.21.15.16:0/2068677880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17d006dec0 0x7f17d015e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:12.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.120+0000 7f17d5f43640 1 --2- 172.21.15.16:0/2068677880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d015b120 0x7f17d015b9f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f17c800c760 tx=0x7f17c800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:12.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.120+0000 7f17c5ffb640 1 -- 172.21.15.16:0/2068677880 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8003da0 con 0x7f17d015b120 2024-09-19T07:50:12.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.120+0000 7f17c5ffb640 1 -- 172.21.15.16:0/2068677880 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c8005070 con 0x7f17d015b120 2024-09-19T07:50:12.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.120+0000 7f17c5ffb640 1 -- 172.21.15.16:0/2068677880 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8005460 con 0x7f17d015b120 2024-09-19T07:50:12.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.120+0000 7f17d79a6640 1 -- 172.21.15.16:0/2068677880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d00b10f0 con 0x7f17d015b120 2024-09-19T07:50:12.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.120+0000 7f17d79a6640 1 -- 172.21.15.16:0/2068677880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d00b16b0 con 0x7f17d015b120 2024-09-19T07:50:12.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.121+0000 7f17d79a6640 1 -- 172.21.15.16:0/2068677880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d00b1b90 con 0x7f17d015b120 2024-09-19T07:50:12.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.121+0000 7f17c5ffb640 1 -- 172.21.15.16:0/2068677880 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f17c8005600 con 0x7f17d015b120 2024-09-19T07:50:12.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.122+0000 7f17c5ffb640 1 --2- 172.21.15.16:0/2068677880 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f17b406c2b0 0x7f17b406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:12.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.122+0000 7f17c5ffb640 1 -- 172.21.15.16:0/2068677880 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5249+0+0 (secure 0 0 0) 0x7f17c808b1e0 con 0x7f17d015b120 2024-09-19T07:50:12.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.122+0000 7f17d5742640 1 --2- 172.21.15.16:0/2068677880 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f17b406c2b0 0x7f17b406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:12.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.123+0000 7f17d5742640 1 --2- 172.21.15.16:0/2068677880 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f17b406c2b0 0x7f17b406e770 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f17cc034000 tx=0x7f17cc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:12.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.126+0000 7f17c5ffb640 1 -- 172.21.15.16:0/2068677880 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f17d00b1b90 con 0x7f17d015b120 2024-09-19T07:50:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.365+0000 7f17d79a6640 1 -- 172.21.15.16:0/2068677880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f17d00d0cd0 con 0x7f17d015b120 2024-09-19T07:50:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.365+0000 7f17c5ffb640 1 -- 172.21.15.16:0/2068677880 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f17c8059b10 con 0x7f17d015b120 2024-09-19T07:50:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:12.366 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:50:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-19T07:50:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-19T07:50:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etms80KDxAAqTrG88LTEJIj9TWJu0witQ==", 2024-09-19T07:50:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:50:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:50:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:50:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:50:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:50:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-19T07:50:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etm4T0KGxAAcfDOKrSdKzhJ4xvTFIyxcQ==", 2024-09-19T07:50:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:50:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:50:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:50:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:50:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:50:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-19T07:50:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNDBGFhAAtBioJJ1+NtIDLhd/Hg7aZA==", 2024-09-19T07:50:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:50:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:50:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:50:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:50:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:50:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-19T07:50:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmaHEiIhAACqqjCRMV+xlGh9WqYsj+MA==", 2024-09-19T07:50:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:50:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:50:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:50:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:50:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:50:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-19T07:50:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDT1etm09plCxAAGvJYF6JrKw49FtXb71ox3A==", 2024-09-19T07:50:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:50:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:50:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:50:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:50:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-19T07:50:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDW1etmGy8yHRAAlcDsHLnkjtS6sCCR8Olv2A==", 2024-09-19T07:50:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:50:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:50:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:50:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:50:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-19T07:50:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDZ1etmMhGTHxAATQaW2kgJhHKftutQQtUa6A==", 2024-09-19T07:50:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:50:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:50:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:50:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:50:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-19T07:50:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDc1etmD3sLIhAAIdY00buvb3bIFA5ZpgxxjA==", 2024-09-19T07:50:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:50:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:50:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:50:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:50:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-19T07:50:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDi1etmgutcLRAAHJEl5SfQ7N0T4YoYeBfrQg==", 2024-09-19T07:50:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:50:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:50:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:50:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:50:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-19T07:50:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDm1etmCV3xARAA+5yhQEmIpyNId3RL1Xsxhw==", 2024-09-19T07:50:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:50:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:50:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:50:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:50:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-19T07:50:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDp1etm9XC+AxAA+Lrs0ExfSKaixgOxfX8Nug==", 2024-09-19T07:50:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:50:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:50:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:50:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:50:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-19T07:50:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDs1etmhmvMBhAAhvW/udIE2gCd8IrJCSFuvw==", 2024-09-19T07:50:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:50:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:50:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:50:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:50:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-19T07:50:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA==", 2024-09-19T07:50:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:50:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:50:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-19T07:50:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T07:50:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-19T07:50:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmc05KLxAAPQl6wOHqFIf8LVxskiuLVg==", 2024-09-19T07:50:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:50:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:50:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:50:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:50:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:50:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-19T07:50:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmjFsCLBAANa7gCMvDw84Ovo7u6RRJ3Q==", 2024-09-19T07:50:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:50:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:50:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:50:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:50:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:50:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-19T07:50:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmdUbgNBAAfwstyOeeoZfetgVZu4z5xg==", 2024-09-19T07:50:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:50:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:50:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:50:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:50:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:50:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-19T07:50:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDQ1etm9/vTLhAA9PRfAOA8R4oXp3x0dU8l3g==", 2024-09-19T07:50:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:50:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:50:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:50:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:50:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:50:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-19T07:50:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNQ68CxAANkK+D3u/oTEz8Su4iuZrCg==", 2024-09-19T07:50:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:50:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:50:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:50:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:50:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-19T07:50:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmp+gSCBAAGXzmKfFNd6K2fWeqYhlHAg==", 2024-09-19T07:50:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:50:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:50:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:50:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:50:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T07:50:12.380 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:50:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.367+0000 7f17d79a6640 1 -- 172.21.15.16:0/2068677880 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f17b406c2b0 msgr2=0x7f17b406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.367+0000 7f17d79a6640 1 --2- 172.21.15.16:0/2068677880 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f17b406c2b0 0x7f17b406e770 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f17cc034000 tx=0x7f17cc02e000 comp rx=0 tx=0).stop 2024-09-19T07:50:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.367+0000 7f17d79a6640 1 -- 172.21.15.16:0/2068677880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d015b120 msgr2=0x7f17d015b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.367+0000 7f17d79a6640 1 --2- 172.21.15.16:0/2068677880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d015b120 0x7f17d015b9f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f17c800c760 tx=0x7f17c800cc20 comp rx=0 tx=0).stop 2024-09-19T07:50:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.367+0000 7f17d79a6640 1 -- 172.21.15.16:0/2068677880 shutdown_connections 2024-09-19T07:50:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.367+0000 7f17d79a6640 1 --2- 172.21.15.16:0/2068677880 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f17b406c2b0 0x7f17b406e770 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.367+0000 7f17d79a6640 1 --2- 172.21.15.16:0/2068677880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d015b120 0x7f17d015b9f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.367+0000 7f17d79a6640 1 --2- 172.21.15.16:0/2068677880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17d00d04b0 0x7f17d0151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.367+0000 7f17d79a6640 1 --2- 172.21.15.16:0/2068677880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17d006dec0 0x7f17d015e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.367+0000 7f17d79a6640 1 -- 172.21.15.16:0/2068677880 >> 172.21.15.16:0/2068677880 conn(0x7f17d006bdf0 msgr2=0x7f17d00c5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.367+0000 7f17d79a6640 1 -- 172.21.15.16:0/2068677880 shutdown_connections 2024-09-19T07:50:12.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.367+0000 7f17d79a6640 1 -- 172.21.15.16:0/2068677880 wait complete. 2024-09-19T07:50:12.383 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T07:50:12.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.485+0000 7f9ba23bd640 1 Processor -- start 2024-09-19T07:50:12.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.485+0000 7f9ba23bd640 1 -- start start 2024-09-19T07:50:12.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.485+0000 7f9ba23bd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b9c1621a0 0x7f9b9c141a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:12.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.486+0000 7f9ba23bd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9b9c0c7770 0x7f9b9c162580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:12.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.486+0000 7f9ba23bd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b9c143b40 con 0x7f9b9c15ea80 2024-09-19T07:50:12.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.486+0000 7f9ba23bd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9b9c15efb0 con 0x7f9b9c1621a0 2024-09-19T07:50:12.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.486+0000 7f9ba23bd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f9b9c15f290 con 0x7f9b9c0c7770 2024-09-19T07:50:12.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.486+0000 7f9b9bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b9c1621a0 0x7f9b9c141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:12.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.486+0000 7f9b93fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9b9c15ea80 0x7f9b9c0c7060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50588/0 (socket says 172.21.15.16:50588) 2024-09-19T07:50:12.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.486+0000 7f9b9bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b9c1621a0 0x7f9b9c141a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39898/0 (socket says 172.21.15.16:39898) 2024-09-19T07:50:12.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.486+0000 7f9b93fff640 1 -- 172.21.15.16:0/2663598061 learned_addr learned my addr 172.21.15.16:0/2663598061 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:12.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.486+0000 7f9ba095a640 1 --2- 172.21.15.16:0/2663598061 >> v2:172.21.15.146:3300/0 conn(0x7f9b9c0c7770 0x7f9b9c162580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:12.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.486+0000 7f9b9bfff640 1 -- 172.21.15.16:0/2663598061 >> v2:172.21.15.146:3300/0 conn(0x7f9b9c0c7770 msgr2=0x7f9b9c162580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:12.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.486+0000 7f9b9bfff640 1 --2- 172.21.15.16:0/2663598061 >> v2:172.21.15.146:3300/0 conn(0x7f9b9c0c7770 0x7f9b9c162580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.486+0000 7f9b9bfff640 1 -- 172.21.15.16:0/2663598061 >> v1:172.21.15.16:6789/0 conn(0x7f9b9c15ea80 legacy=0x7f9b9c0c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:12.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.486+0000 7f9b9bfff640 1 -- 172.21.15.16:0/2663598061 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9c15f570 con 0x7f9b9c1621a0 2024-09-19T07:50:12.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.486+0000 7f9b9b7fe640 1 -- 172.21.15.16:0/2663598061 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3005213045 0 0) 0x7f9b9c143b40 con 0x7f9b9c15ea80 2024-09-19T07:50:12.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.486+0000 7f9ba095a640 1 --2- 172.21.15.16:0/2663598061 >> v2:172.21.15.146:3300/0 conn(0x7f9b9c0c7770 0x7f9b9c162580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:12.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.487+0000 7f9b9bfff640 1 --2- 172.21.15.16:0/2663598061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9c1621a0 0x7f9b9c141a90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0c7d60 tx=0x7f9b8c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ee834ce424c098 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:12.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.487+0000 7f9b9b7fe640 1 -- 172.21.15.16:0/2663598061 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c037070 con 0x7f9b9c1621a0 2024-09-19T07:50:12.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.487+0000 7f9b9b7fe640 1 -- 172.21.15.16:0/2663598061 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b8c032440 con 0x7f9b9c1621a0 2024-09-19T07:50:12.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.487+0000 7f9b9b7fe640 1 -- 172.21.15.16:0/2663598061 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c032810 con 0x7f9b9c1621a0 2024-09-19T07:50:12.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.488+0000 7f9ba23bd640 1 -- 172.21.15.16:0/2663598061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9c1621a0 msgr2=0x7f9b9c141a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:12.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.488+0000 7f9ba23bd640 1 --2- 172.21.15.16:0/2663598061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9c1621a0 0x7f9b9c141a90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0c7d60 tx=0x7f9b8c02a1a0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.488+0000 7f9ba23bd640 1 -- 172.21.15.16:0/2663598061 shutdown_connections 2024-09-19T07:50:12.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.488+0000 7f9ba23bd640 1 --2- 172.21.15.16:0/2663598061 >> v2:172.21.15.146:3300/0 conn(0x7f9b9c0c7770 0x7f9b9c162580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.488+0000 7f9ba23bd640 1 --2- 172.21.15.16:0/2663598061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9c1621a0 0x7f9b9c141a90 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.488+0000 7f9ba23bd640 1 -- 172.21.15.16:0/2663598061 >> 172.21.15.16:0/2663598061 conn(0x7f9b9c067fe0 msgr2=0x7f9b9c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:12.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.488+0000 7f9ba23bd640 1 -- 172.21.15.16:0/2663598061 shutdown_connections 2024-09-19T07:50:12.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.488+0000 7f9ba23bd640 1 -- 172.21.15.16:0/2663598061 wait complete. 2024-09-19T07:50:12.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.489+0000 7f9ba23bd640 1 Processor -- start 2024-09-19T07:50:12.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.489+0000 7f9ba23bd640 1 -- start start 2024-09-19T07:50:12.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9ba23bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9c0c7770 0x7f9b9c158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:12.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9ba23bd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b9c15ea80 0x7f9b9c0c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:12.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9ba23bd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b9c1621a0 0x7f9b9c0b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:12.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9ba23bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c0cfa50 con 0x7f9b9c0c7770 2024-09-19T07:50:12.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9ba23bd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c141fd0 con 0x7f9b9c15ea80 2024-09-19T07:50:12.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9ba23bd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9b9c13e960 con 0x7f9b9c1621a0 2024-09-19T07:50:12.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9b9bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9c0c7770 0x7f9b9c158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:12.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9b9bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9c0c7770 0x7f9b9c158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39910/0 (socket says 172.21.15.16:39910) 2024-09-19T07:50:12.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9b9bfff640 1 -- 172.21.15.16:0/1173253723 learned_addr learned my addr 172.21.15.16:0/1173253723 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9ba095a640 1 --2- 172.21.15.16:0/1173253723 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b9c1621a0 0x7f9b9c0b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9b93fff640 1 --2- 172.21.15.16:0/1173253723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b9c15ea80 0x7f9b9c0c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9b9bfff640 1 -- 172.21.15.16:0/1173253723 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b9c1621a0 msgr2=0x7f9b9c0b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9b9bfff640 1 --2- 172.21.15.16:0/1173253723 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b9c1621a0 0x7f9b9c0b50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9b9bfff640 1 -- 172.21.15.16:0/1173253723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b9c15ea80 msgr2=0x7f9b9c0c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9b9bfff640 1 --2- 172.21.15.16:0/1173253723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b9c15ea80 0x7f9b9c0c9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9b9bfff640 1 -- 172.21.15.16:0/1173253723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9c13ec40 con 0x7f9b9c0c7770 2024-09-19T07:50:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9b93fff640 1 --2- 172.21.15.16:0/1173253723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b9c15ea80 0x7f9b9c0c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9b9bfff640 1 --2- 172.21.15.16:0/1173253723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9c0c7770 0x7f9b9c158240 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c000c00 tx=0x7f9b8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9b997fa640 1 -- 172.21.15.16:0/1173253723 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c037050 con 0x7f9b9c0c7770 2024-09-19T07:50:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9b997fa640 1 -- 172.21.15.16:0/1173253723 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b8c038040 con 0x7f9b9c0c7770 2024-09-19T07:50:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9ba23bd640 1 -- 172.21.15.16:0/1173253723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b9c13ef20 con 0x7f9b9c0c7770 2024-09-19T07:50:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.490+0000 7f9b997fa640 1 -- 172.21.15.16:0/1173253723 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c004470 con 0x7f9b9c0c7770 2024-09-19T07:50:12.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.491+0000 7f9ba23bd640 1 -- 172.21.15.16:0/1173253723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b9c13f4e0 con 0x7f9b9c0c7770 2024-09-19T07:50:12.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.491+0000 7f9ba23bd640 1 -- 172.21.15.16:0/1173253723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b9c1588a0 con 0x7f9b9c0c7770 2024-09-19T07:50:12.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.492+0000 7f9b997fa640 1 -- 172.21.15.16:0/1173253723 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9b8c030980 con 0x7f9b9c0c7770 2024-09-19T07:50:12.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.493+0000 7f9b997fa640 1 --2- 172.21.15.16:0/1173253723 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b7c06c2b0 0x7f9b7c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:12.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.493+0000 7f9b997fa640 1 -- 172.21.15.16:0/1173253723 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5249+0+0 (secure 0 0 0) 0x7f9b8c0abfb0 con 0x7f9b9c0c7770 2024-09-19T07:50:12.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.493+0000 7f9b93fff640 1 --2- 172.21.15.16:0/1173253723 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b7c06c2b0 0x7f9b7c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:12.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.493+0000 7f9b93fff640 1 --2- 172.21.15.16:0/1173253723 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b7c06c2b0 0x7f9b7c06e770 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c140c50 tx=0x7f9b84009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:12.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.497+0000 7f9b997fa640 1 -- 172.21.15.16:0/1173253723 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9b8c07a9f0 con 0x7f9b9c0c7770 2024-09-19T07:50:12.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.675+0000 7f9ba23bd640 1 -- 172.21.15.16:0/1173253723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9b9c13f6e0 con 0x7f9b9c0c7770 2024-09-19T07:50:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.675+0000 7f9b997fa640 1 -- 172.21.15.16:0/1173253723 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v89) ==== 84+0+65 (secure 0 0 0) 0x7f9b8c07a2f0 con 0x7f9b9c0c7770 2024-09-19T07:50:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:12.676 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-19T07:50:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-19T07:50:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-19T07:50:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:12.677 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-19T07:50:12.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.678+0000 7f9ba23bd640 1 -- 172.21.15.16:0/1173253723 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b7c06c2b0 msgr2=0x7f9b7c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:12.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.678+0000 7f9ba23bd640 1 --2- 172.21.15.16:0/1173253723 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b7c06c2b0 0x7f9b7c06e770 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c140c50 tx=0x7f9b84009040 comp rx=0 tx=0).stop 2024-09-19T07:50:12.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.678+0000 7f9ba23bd640 1 -- 172.21.15.16:0/1173253723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9c0c7770 msgr2=0x7f9b9c158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:12.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.678+0000 7f9ba23bd640 1 --2- 172.21.15.16:0/1173253723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9c0c7770 0x7f9b9c158240 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c000c00 tx=0x7f9b8c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.678+0000 7f9ba23bd640 1 -- 172.21.15.16:0/1173253723 shutdown_connections 2024-09-19T07:50:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.678+0000 7f9ba23bd640 1 --2- 172.21.15.16:0/1173253723 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b7c06c2b0 0x7f9b7c06e770 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.678+0000 7f9ba23bd640 1 --2- 172.21.15.16:0/1173253723 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b9c1621a0 0x7f9b9c0b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.678+0000 7f9ba23bd640 1 --2- 172.21.15.16:0/1173253723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b9c15ea80 0x7f9b9c0c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.678+0000 7f9ba23bd640 1 --2- 172.21.15.16:0/1173253723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9c0c7770 0x7f9b9c158240 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.678+0000 7f9ba23bd640 1 -- 172.21.15.16:0/1173253723 >> 172.21.15.16:0/1173253723 conn(0x7f9b9c067fe0 msgr2=0x7f9b9c0d2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.678+0000 7f9ba23bd640 1 -- 172.21.15.16:0/1173253723 shutdown_connections 2024-09-19T07:50:12.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.678+0000 7f9ba23bd640 1 -- 172.21.15.16:0/1173253723 wait complete. 2024-09-19T07:50:12.689 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T07:50:12.689 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T07:50:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.796+0000 7faf76331640 1 Processor -- start 2024-09-19T07:50:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.796+0000 7faf76331640 1 -- start start 2024-09-19T07:50:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.797+0000 7faf76331640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf7015e2a0 0x7faf7006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.797+0000 7faf76331640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faf7015e680 0x7faf700cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.797+0000 7faf76331640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf700d1880 con 0x7faf7006d960 2024-09-19T07:50:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.797+0000 7faf76331640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7faf700d1b60 con 0x7faf7015e680 2024-09-19T07:50:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.797+0000 7faf76331640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faf700cf2d0 con 0x7faf7015e2a0 2024-09-19T07:50:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.797+0000 7faf6ffff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf7015e2a0 0x7faf7006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.797+0000 7faf6ffff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf7015e2a0 0x7faf7006d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:53864/0 (socket says 172.21.15.16:53864) 2024-09-19T07:50:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.797+0000 7faf6f7fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faf7015e680 0x7faf700cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.797+0000 7faf6ffff640 1 -- 172.21.15.16:0/1057594747 learned_addr learned my addr 172.21.15.16:0/1057594747 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.797+0000 7faf6ffff640 1 -- 172.21.15.16:0/1057594747 >> v2:172.21.15.146:3300/0 conn(0x7faf7015e680 msgr2=0x7faf700cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.797+0000 7faf6ffff640 1 --2- 172.21.15.16:0/1057594747 >> v2:172.21.15.146:3300/0 conn(0x7faf7015e680 0x7faf700cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.797+0000 7faf6ffff640 1 -- 172.21.15.16:0/1057594747 >> v1:172.21.15.146:6789/0 conn(0x7faf7006d960 legacy=0x7faf700d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:12.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.797+0000 7faf6ffff640 1 -- 172.21.15.16:0/1057594747 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf700cf5b0 con 0x7faf7015e2a0 2024-09-19T07:50:12.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.797+0000 7faf6f7fe640 1 --2- 172.21.15.16:0/1057594747 >> v2:172.21.15.146:3300/0 conn(0x7faf7015e680 0x7faf700cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.798+0000 7faf6ffff640 1 --2- 172.21.15.16:0/1057594747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf7015e2a0 0x7faf7006d420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faf60009870 tx=0x7faf6002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ce8cac83b0ff93c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.798+0000 7faf6effd640 1 -- 172.21.15.16:0/1057594747 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60037070 con 0x7faf7015e2a0 2024-09-19T07:50:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.798+0000 7faf6effd640 1 -- 172.21.15.16:0/1057594747 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf60032440 con 0x7faf7015e2a0 2024-09-19T07:50:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.799+0000 7faf6effd640 1 -- 172.21.15.16:0/1057594747 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60032810 con 0x7faf7015e2a0 2024-09-19T07:50:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.799+0000 7faf76331640 1 -- 172.21.15.16:0/1057594747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf7015e2a0 msgr2=0x7faf7006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.799+0000 7faf76331640 1 --2- 172.21.15.16:0/1057594747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf7015e2a0 0x7faf7006d420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faf60009870 tx=0x7faf6002a270 comp rx=0 tx=0).stop 2024-09-19T07:50:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.799+0000 7faf76331640 1 -- 172.21.15.16:0/1057594747 shutdown_connections 2024-09-19T07:50:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.799+0000 7faf76331640 1 --2- 172.21.15.16:0/1057594747 >> v2:172.21.15.146:3300/0 conn(0x7faf7015e680 0x7faf700cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.799+0000 7faf76331640 1 --2- 172.21.15.16:0/1057594747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf7015e2a0 0x7faf7006d420 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.799+0000 7faf76331640 1 -- 172.21.15.16:0/1057594747 >> 172.21.15.16:0/1057594747 conn(0x7faf7009d760 msgr2=0x7faf7008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:12.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.799+0000 7faf76331640 1 -- 172.21.15.16:0/1057594747 shutdown_connections 2024-09-19T07:50:12.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.799+0000 7faf76331640 1 -- 172.21.15.16:0/1057594747 wait complete. 2024-09-19T07:50:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.800+0000 7faf76331640 1 Processor -- start 2024-09-19T07:50:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.800+0000 7faf76331640 1 -- start start 2024-09-19T07:50:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.801+0000 7faf76331640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf7006d960 0x7faf700cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.801+0000 7faf76331640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf7015e2a0 0x7faf70158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.801+0000 7faf76331640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7015e680 0x7faf70159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.801+0000 7faf76331640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf700d4c00 con 0x7faf7015e680 2024-09-19T07:50:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.801+0000 7faf76331640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faf700d4a80 con 0x7faf7006d960 2024-09-19T07:50:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.801+0000 7faf76331640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faf700d0010 con 0x7faf7015e2a0 2024-09-19T07:50:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.801+0000 7faf748ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7015e680 0x7faf70159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.801+0000 7faf748ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7015e680 0x7faf70159490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39916/0 (socket says 172.21.15.16:39916) 2024-09-19T07:50:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.801+0000 7faf748ce640 1 -- 172.21.15.16:0/4252522587 learned_addr learned my addr 172.21.15.16:0/4252522587 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.801+0000 7faf6ffff640 1 --2- 172.21.15.16:0/4252522587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf7006d960 0x7faf700cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.801+0000 7faf6f7fe640 1 --2- 172.21.15.16:0/4252522587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf7015e2a0 0x7faf70158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.802+0000 7faf748ce640 1 -- 172.21.15.16:0/4252522587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf7015e2a0 msgr2=0x7faf70158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.802+0000 7faf748ce640 1 --2- 172.21.15.16:0/4252522587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf7015e2a0 0x7faf70158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.802+0000 7faf748ce640 1 -- 172.21.15.16:0/4252522587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf7006d960 msgr2=0x7faf700cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.802+0000 7faf748ce640 1 --2- 172.21.15.16:0/4252522587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf7006d960 0x7faf700cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.802+0000 7faf748ce640 1 -- 172.21.15.16:0/4252522587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf700b9df0 con 0x7faf7015e680 2024-09-19T07:50:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.802+0000 7faf748ce640 1 --2- 172.21.15.16:0/4252522587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7015e680 0x7faf70159490 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7faf6800c7a0 tx=0x7faf6800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.802+0000 7faf6cff9640 1 -- 172.21.15.16:0/4252522587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf68003b70 con 0x7faf7015e680 2024-09-19T07:50:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.802+0000 7faf6cff9640 1 -- 172.21.15.16:0/4252522587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf68003d30 con 0x7faf7015e680 2024-09-19T07:50:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.802+0000 7faf76331640 1 -- 172.21.15.16:0/4252522587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf700ba0d0 con 0x7faf7015e680 2024-09-19T07:50:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.802+0000 7faf6cff9640 1 -- 172.21.15.16:0/4252522587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6800aa30 con 0x7faf7015e680 2024-09-19T07:50:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.802+0000 7faf76331640 1 -- 172.21.15.16:0/4252522587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf700c88f0 con 0x7faf7015e680 2024-09-19T07:50:12.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.803+0000 7faf76331640 1 -- 172.21.15.16:0/4252522587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf7006d420 con 0x7faf7015e680 2024-09-19T07:50:12.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.804+0000 7faf6cff9640 1 -- 172.21.15.16:0/4252522587 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7faf6800abd0 con 0x7faf7015e680 2024-09-19T07:50:12.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.804+0000 7faf6cff9640 1 --2- 172.21.15.16:0/4252522587 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faf5c06c2b0 0x7faf5c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:12.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.804+0000 7faf6cff9640 1 -- 172.21.15.16:0/4252522587 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5249+0+0 (secure 0 0 0) 0x7faf6808b090 con 0x7faf7015e680 2024-09-19T07:50:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.804+0000 7faf6ffff640 1 --2- 172.21.15.16:0/4252522587 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faf5c06c2b0 0x7faf5c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.805+0000 7faf6ffff640 1 --2- 172.21.15.16:0/4252522587 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faf5c06c2b0 0x7faf5c06e770 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7faf60034000 tx=0x7faf6002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.808+0000 7faf6cff9640 1 -- 172.21.15.16:0/4252522587 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7faf680592e0 con 0x7faf7015e680 2024-09-19T07:50:12.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:12.984+0000 7faf76331640 1 -- 172.21.15.16:0/4252522587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7faf7006cf90 con 0x7faf7015e680 2024-09-19T07:50:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.678+0000 7faf6cff9640 1 -- 172.21.15.16:0/4252522587 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v90) ==== 138+0+0 (secure 0 0 0) 0x7faf6805d5e0 con 0x7faf7015e680 2024-09-19T07:50:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-19T07:50:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.681+0000 7faf76331640 1 -- 172.21.15.16:0/4252522587 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faf5c06c2b0 msgr2=0x7faf5c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.681+0000 7faf76331640 1 --2- 172.21.15.16:0/4252522587 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faf5c06c2b0 0x7faf5c06e770 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7faf60034000 tx=0x7faf6002e000 comp rx=0 tx=0).stop 2024-09-19T07:50:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.681+0000 7faf76331640 1 -- 172.21.15.16:0/4252522587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7015e680 msgr2=0x7faf70159490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.681+0000 7faf76331640 1 --2- 172.21.15.16:0/4252522587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7015e680 0x7faf70159490 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7faf6800c7a0 tx=0x7faf6800cc60 comp rx=0 tx=0).stop 2024-09-19T07:50:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.681+0000 7faf76331640 1 -- 172.21.15.16:0/4252522587 shutdown_connections 2024-09-19T07:50:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.681+0000 7faf76331640 1 --2- 172.21.15.16:0/4252522587 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faf5c06c2b0 0x7faf5c06e770 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.681+0000 7faf76331640 1 --2- 172.21.15.16:0/4252522587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7015e680 0x7faf70159490 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:13.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.681+0000 7faf76331640 1 --2- 172.21.15.16:0/4252522587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf7015e2a0 0x7faf70158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:13.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.681+0000 7faf76331640 1 --2- 172.21.15.16:0/4252522587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf7006d960 0x7faf700cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:13.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.681+0000 7faf76331640 1 -- 172.21.15.16:0/4252522587 >> 172.21.15.16:0/4252522587 conn(0x7faf7009d760 msgr2=0x7faf700d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:13.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.682+0000 7faf76331640 1 -- 172.21.15.16:0/4252522587 shutdown_connections 2024-09-19T07:50:13.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.682+0000 7faf76331640 1 -- 172.21.15.16:0/4252522587 wait complete. 2024-09-19T07:50:13.698 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T07:50:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.812+0000 7f44ec5fd640 1 Processor -- start 2024-09-19T07:50:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.812+0000 7f44ec5fd640 1 -- start start 2024-09-19T07:50:13.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.813+0000 7f44ec5fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44e415b110 0x7f44e409a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:13.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.813+0000 7f44ec5fd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f44e40b4080 0x7f44e40b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:13.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.813+0000 7f44ec5fd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44e40b4eb0 con 0x7f44e40b4a00 2024-09-19T07:50:13.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.813+0000 7f44ec5fd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f44e406eec0 con 0x7f44e40b4080 2024-09-19T07:50:13.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.813+0000 7f44ec5fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f44e40af9e0 con 0x7f44e415b110 2024-09-19T07:50:13.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.814+0000 7f44e9b98640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f44e40b4080 0x7f44e40b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:13.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.814+0000 7f44ea399640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44e415b110 0x7f44e409a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:13.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.814+0000 7f44e9b98640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f44e40b4080 0x7f44e40b44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:39938/0 (socket says 172.21.15.16:39938) 2024-09-19T07:50:13.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.814+0000 7f44e9b98640 1 -- 172.21.15.16:0/525151455 learned_addr learned my addr 172.21.15.16:0/525151455 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.814+0000 7f44e9b98640 1 -- 172.21.15.16:0/525151455 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44e415b110 msgr2=0x7f44e409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.814+0000 7f44e9b98640 1 --2- 172.21.15.16:0/525151455 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44e415b110 0x7f44e409a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.814+0000 7f44e9b98640 1 -- 172.21.15.16:0/525151455 >> v1:172.21.15.146:6789/0 conn(0x7f44e40b4a00 legacy=0x7f44e40ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.814+0000 7f44e9b98640 1 -- 172.21.15.16:0/525151455 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e40afcc0 con 0x7f44e40b4080 2024-09-19T07:50:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.814+0000 7f44ea399640 1 --2- 172.21.15.16:0/525151455 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44e415b110 0x7f44e409a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.815+0000 7f44e9b98640 1 --2- 172.21.15.16:0/525151455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44e40b4080 0x7f44e40b44c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009870 tx=0x7f44d402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97e603049c0a9427 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.815+0000 7f44e9397640 1 -- 172.21.15.16:0/525151455 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d4037070 con 0x7f44e40b4080 2024-09-19T07:50:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.815+0000 7f44e9397640 1 -- 172.21.15.16:0/525151455 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d4032440 con 0x7f44e40b4080 2024-09-19T07:50:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.815+0000 7f44e9397640 1 -- 172.21.15.16:0/525151455 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d4032810 con 0x7f44e40b4080 2024-09-19T07:50:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.816+0000 7f44ec5fd640 1 -- 172.21.15.16:0/525151455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44e40b4080 msgr2=0x7f44e40b44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.816+0000 7f44ec5fd640 1 --2- 172.21.15.16:0/525151455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44e40b4080 0x7f44e40b44c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009870 tx=0x7f44d402a250 comp rx=0 tx=0).stop 2024-09-19T07:50:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.816+0000 7f44ec5fd640 1 -- 172.21.15.16:0/525151455 shutdown_connections 2024-09-19T07:50:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.816+0000 7f44ec5fd640 1 --2- 172.21.15.16:0/525151455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44e40b4080 0x7f44e40b44c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.816+0000 7f44ec5fd640 1 --2- 172.21.15.16:0/525151455 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44e415b110 0x7f44e409a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.816+0000 7f44ec5fd640 1 -- 172.21.15.16:0/525151455 >> 172.21.15.16:0/525151455 conn(0x7f44e40b67c0 msgr2=0x7f44e40b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.816+0000 7f44ec5fd640 1 -- 172.21.15.16:0/525151455 shutdown_connections 2024-09-19T07:50:13.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.816+0000 7f44ec5fd640 1 -- 172.21.15.16:0/525151455 wait complete. 2024-09-19T07:50:13.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.817+0000 7f44ec5fd640 1 Processor -- start 2024-09-19T07:50:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.817+0000 7f44ec5fd640 1 -- start start 2024-09-19T07:50:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.818+0000 7f44ec5fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e40b4080 0x7f44e40ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.818+0000 7f44ec5fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44e40b4a00 0x7f44e40c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.818+0000 7f44ec5fd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44e415b110 0x7f44e414fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.818+0000 7f44ec5fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f44e415b4f0 con 0x7f44e40b4080 2024-09-19T07:50:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.818+0000 7f44ec5fd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f44e40b0280 con 0x7f44e415b110 2024-09-19T07:50:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.818+0000 7f44ec5fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f44e4070360 con 0x7f44e40b4a00 2024-09-19T07:50:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.818+0000 7f44ea399640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e40b4080 0x7f44e40ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.818+0000 7f44ea399640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e40b4080 0x7f44e40ca900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39932/0 (socket says 172.21.15.16:39932) 2024-09-19T07:50:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.818+0000 7f44ea399640 1 -- 172.21.15.16:0/3866355924 learned_addr learned my addr 172.21.15.16:0/3866355924 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.818+0000 7f44e9b98640 1 --2- 172.21.15.16:0/3866355924 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44e40b4a00 0x7f44e40c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.818+0000 7f44eab9a640 1 --2- 172.21.15.16:0/3866355924 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44e415b110 0x7f44e414fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.818+0000 7f44ea399640 1 -- 172.21.15.16:0/3866355924 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44e40b4a00 msgr2=0x7f44e40c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.818+0000 7f44ea399640 1 --2- 172.21.15.16:0/3866355924 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44e40b4a00 0x7f44e40c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.818+0000 7f44ea399640 1 -- 172.21.15.16:0/3866355924 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44e415b110 msgr2=0x7f44e414fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.818+0000 7f44ea399640 1 --2- 172.21.15.16:0/3866355924 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44e415b110 0x7f44e414fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.818+0000 7f44ea399640 1 -- 172.21.15.16:0/3866355924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e40c8740 con 0x7f44e40b4080 2024-09-19T07:50:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.818+0000 7f44eab9a640 1 --2- 172.21.15.16:0/3866355924 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44e415b110 0x7f44e414fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.818+0000 7f44ea399640 1 --2- 172.21.15.16:0/3866355924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e40b4080 0x7f44e40ca900 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f44e000efc0 tx=0x7f44e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.819+0000 7f44daffd640 1 -- 172.21.15.16:0/3866355924 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0018020 con 0x7f44e40b4080 2024-09-19T07:50:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.819+0000 7f44daffd640 1 -- 172.21.15.16:0/3866355924 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e0014680 con 0x7f44e40b4080 2024-09-19T07:50:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.819+0000 7f44ec5fd640 1 -- 172.21.15.16:0/3866355924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e40c89c0 con 0x7f44e40b4080 2024-09-19T07:50:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.819+0000 7f44daffd640 1 -- 172.21.15.16:0/3866355924 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0014a50 con 0x7f44e40b4080 2024-09-19T07:50:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.819+0000 7f44ec5fd640 1 -- 172.21.15.16:0/3866355924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e40c8f80 con 0x7f44e40b4080 2024-09-19T07:50:13.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.820+0000 7f44ec5fd640 1 -- 172.21.15.16:0/3866355924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e40b44c0 con 0x7f44e40b4080 2024-09-19T07:50:13.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.820+0000 7f44daffd640 1 -- 172.21.15.16:0/3866355924 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f44e0005d10 con 0x7f44e40b4080 2024-09-19T07:50:13.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.821+0000 7f44daffd640 1 --2- 172.21.15.16:0/3866355924 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f44c806c2b0 0x7f44c806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:13.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.821+0000 7f44daffd640 1 -- 172.21.15.16:0/3866355924 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5619+0+0 (secure 0 0 0) 0x7f44e008c060 con 0x7f44e40b4080 2024-09-19T07:50:13.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.821+0000 7f44e9b98640 1 --2- 172.21.15.16:0/3866355924 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f44c806c2b0 0x7f44c806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:13.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.822+0000 7f44e9b98640 1 --2- 172.21.15.16:0/3866355924 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f44c806c2b0 0x7f44c806e770 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f44d4034000 tx=0x7f44d402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:13.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.825+0000 7f44daffd640 1 -- 172.21.15.16:0/3866355924 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f44e0022030 con 0x7f44e40b4080 2024-09-19T07:50:13.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:13.999+0000 7f44ec5fd640 1 -- 172.21.15.16:0/3866355924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f44e415bd60 con 0x7f44e40b4080 2024-09-19T07:50:14.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.684+0000 7f44daffd640 1 -- 172.21.15.16:0/3866355924 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v91) ==== 170+0+0 (secure 0 0 0) 0x7f44e40c8740 con 0x7f44e40b4080 2024-09-19T07:50:14.685 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-19T07:50:14.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.687+0000 7f44ec5fd640 1 -- 172.21.15.16:0/3866355924 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f44c806c2b0 msgr2=0x7f44c806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:14.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.687+0000 7f44ec5fd640 1 --2- 172.21.15.16:0/3866355924 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f44c806c2b0 0x7f44c806e770 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f44d4034000 tx=0x7f44d402e000 comp rx=0 tx=0).stop 2024-09-19T07:50:14.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.687+0000 7f44ec5fd640 1 -- 172.21.15.16:0/3866355924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e40b4080 msgr2=0x7f44e40ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:14.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.687+0000 7f44ec5fd640 1 --2- 172.21.15.16:0/3866355924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e40b4080 0x7f44e40ca900 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f44e000efc0 tx=0x7f44e000c490 comp rx=0 tx=0).stop 2024-09-19T07:50:14.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.687+0000 7f44ec5fd640 1 -- 172.21.15.16:0/3866355924 shutdown_connections 2024-09-19T07:50:14.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.687+0000 7f44ec5fd640 1 --2- 172.21.15.16:0/3866355924 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f44c806c2b0 0x7f44c806e770 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.687+0000 7f44ec5fd640 1 --2- 172.21.15.16:0/3866355924 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44e415b110 0x7f44e414fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.687+0000 7f44ec5fd640 1 --2- 172.21.15.16:0/3866355924 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44e40b4a00 0x7f44e40c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.687+0000 7f44ec5fd640 1 --2- 172.21.15.16:0/3866355924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e40b4080 0x7f44e40ca900 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.687+0000 7f44ec5fd640 1 -- 172.21.15.16:0/3866355924 >> 172.21.15.16:0/3866355924 conn(0x7f44e40b67c0 msgr2=0x7f44e40aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.688+0000 7f44ec5fd640 1 -- 172.21.15.16:0/3866355924 shutdown_connections 2024-09-19T07:50:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.688+0000 7f44ec5fd640 1 -- 172.21.15.16:0/3866355924 wait complete. 2024-09-19T07:50:14.717 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T07:50:14.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.826+0000 7f7d27d8d640 1 Processor -- start 2024-09-19T07:50:14.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.826+0000 7f7d27d8d640 1 -- start start 2024-09-19T07:50:14.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.827+0000 7f7d27d8d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d200d21f0 0x7f7d200d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:14.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.827+0000 7f7d27d8d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d20072590 0x7f7d2006d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:14.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.827+0000 7f7d27d8d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d200d2b10 con 0x7f7d20143a60 2024-09-19T07:50:14.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.827+0000 7f7d27d8d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f7d20141750 con 0x7f7d20072590 2024-09-19T07:50:14.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.827+0000 7f7d27d8d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7d201620b0 con 0x7f7d200d21f0 2024-09-19T07:50:14.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.827+0000 7f7d25328640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d20072590 0x7f7d2006d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:14.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.827+0000 7f7d25328640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d20072590 0x7f7d2006d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39936/0 (socket says 172.21.15.16:39936) 2024-09-19T07:50:14.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.827+0000 7f7d25328640 1 -- 172.21.15.16:0/2519047934 learned_addr learned my addr 172.21.15.16:0/2519047934 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:14.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.827+0000 7f7d25b29640 1 --2- 172.21.15.16:0/2519047934 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d200d21f0 0x7f7d200d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:14.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.827+0000 7f7d25328640 1 -- 172.21.15.16:0/2519047934 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d200d21f0 msgr2=0x7f7d200d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:14.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.827+0000 7f7d25328640 1 --2- 172.21.15.16:0/2519047934 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d200d21f0 0x7f7d200d25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:14.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.827+0000 7f7d25328640 1 -- 172.21.15.16:0/2519047934 >> v1:172.21.15.146:6789/0 conn(0x7f7d20143a60 legacy=0x7f7d20143f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:14.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.827+0000 7f7d25328640 1 -- 172.21.15.16:0/2519047934 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d20162370 con 0x7f7d20072590 2024-09-19T07:50:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.828+0000 7f7d25328640 1 --2- 172.21.15.16:0/2519047934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d20072590 0x7f7d2006d0e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009870 tx=0x7f7d1002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9dab41b238644a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.828+0000 7f7d24b27640 1 -- 172.21.15.16:0/2519047934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10037070 con 0x7f7d20072590 2024-09-19T07:50:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.828+0000 7f7d24b27640 1 -- 172.21.15.16:0/2519047934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d10032440 con 0x7f7d20072590 2024-09-19T07:50:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.828+0000 7f7d24b27640 1 -- 172.21.15.16:0/2519047934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10032810 con 0x7f7d20072590 2024-09-19T07:50:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.829+0000 7f7d27d8d640 1 -- 172.21.15.16:0/2519047934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d20072590 msgr2=0x7f7d2006d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.829+0000 7f7d27d8d640 1 --2- 172.21.15.16:0/2519047934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d20072590 0x7f7d2006d0e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009870 tx=0x7f7d1002a180 comp rx=0 tx=0).stop 2024-09-19T07:50:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.829+0000 7f7d27d8d640 1 -- 172.21.15.16:0/2519047934 shutdown_connections 2024-09-19T07:50:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.829+0000 7f7d27d8d640 1 --2- 172.21.15.16:0/2519047934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d20072590 0x7f7d2006d0e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.829+0000 7f7d27d8d640 1 --2- 172.21.15.16:0/2519047934 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d200d21f0 0x7f7d200d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.829+0000 7f7d27d8d640 1 -- 172.21.15.16:0/2519047934 >> 172.21.15.16:0/2519047934 conn(0x7f7d200b3ed0 msgr2=0x7f7d200b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:14.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.829+0000 7f7d27d8d640 1 -- 172.21.15.16:0/2519047934 shutdown_connections 2024-09-19T07:50:14.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.829+0000 7f7d27d8d640 1 -- 172.21.15.16:0/2519047934 wait complete. 2024-09-19T07:50:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.830+0000 7f7d27d8d640 1 Processor -- start 2024-09-19T07:50:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.830+0000 7f7d27d8d640 1 -- start start 2024-09-19T07:50:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.831+0000 7f7d27d8d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d20072590 0x7f7d200b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.831+0000 7f7d27d8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d200d21f0 0x7f7d200c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.831+0000 7f7d27d8d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d20143a60 0x7f7d200c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.831+0000 7f7d27d8d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d20162c10 con 0x7f7d200d21f0 2024-09-19T07:50:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.831+0000 7f7d27d8d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7d20162930 con 0x7f7d20143a60 2024-09-19T07:50:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.831+0000 7f7d27d8d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7d2006d620 con 0x7f7d20072590 2024-09-19T07:50:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.831+0000 7f7d25328640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d200d21f0 0x7f7d200c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.831+0000 7f7d25328640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d200d21f0 0x7f7d200c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39942/0 (socket says 172.21.15.16:39942) 2024-09-19T07:50:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.831+0000 7f7d25328640 1 -- 172.21.15.16:0/989395044 learned_addr learned my addr 172.21.15.16:0/989395044 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.831+0000 7f7d25b29640 1 --2- 172.21.15.16:0/989395044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d20072590 0x7f7d200b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.831+0000 7f7d2632a640 1 --2- 172.21.15.16:0/989395044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d20143a60 0x7f7d200c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.831+0000 7f7d25328640 1 -- 172.21.15.16:0/989395044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d20072590 msgr2=0x7f7d200b6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.831+0000 7f7d25328640 1 --2- 172.21.15.16:0/989395044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d20072590 0x7f7d200b6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.831+0000 7f7d25328640 1 -- 172.21.15.16:0/989395044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d20143a60 msgr2=0x7f7d200c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.831+0000 7f7d25328640 1 --2- 172.21.15.16:0/989395044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d20143a60 0x7f7d200c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.831+0000 7f7d25328640 1 -- 172.21.15.16:0/989395044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d200b51d0 con 0x7f7d200d21f0 2024-09-19T07:50:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.831+0000 7f7d25b29640 1 --2- 172.21.15.16:0/989395044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d20072590 0x7f7d200b6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.831+0000 7f7d2632a640 1 --2- 172.21.15.16:0/989395044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d20143a60 0x7f7d200c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.831+0000 7f7d25328640 1 --2- 172.21.15.16:0/989395044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d200d21f0 0x7f7d200c8190 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7d10000c00 tx=0x7f7d100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.832+0000 7f7d167fc640 1 -- 172.21.15.16:0/989395044 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10041070 con 0x7f7d200d21f0 2024-09-19T07:50:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.832+0000 7f7d167fc640 1 -- 172.21.15.16:0/989395044 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d1003c890 con 0x7f7d200d21f0 2024-09-19T07:50:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.832+0000 7f7d27d8d640 1 -- 172.21.15.16:0/989395044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2015e000 con 0x7f7d200d21f0 2024-09-19T07:50:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.832+0000 7f7d167fc640 1 -- 172.21.15.16:0/989395044 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10037040 con 0x7f7d200d21f0 2024-09-19T07:50:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.832+0000 7f7d27d8d640 1 -- 172.21.15.16:0/989395044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d200b5770 con 0x7f7d200d21f0 2024-09-19T07:50:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.832+0000 7f7d27d8d640 1 -- 172.21.15.16:0/989395044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d20158c30 con 0x7f7d200d21f0 2024-09-19T07:50:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.833+0000 7f7d167fc640 1 -- 172.21.15.16:0/989395044 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7d10044070 con 0x7f7d200d21f0 2024-09-19T07:50:14.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.834+0000 7f7d167fc640 1 --2- 172.21.15.16:0/989395044 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7d0406c2b0 0x7f7d0406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:14.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.834+0000 7f7d167fc640 1 -- 172.21.15.16:0/989395044 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6001+0+0 (secure 0 0 0) 0x7f7d100ac180 con 0x7f7d200d21f0 2024-09-19T07:50:14.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.834+0000 7f7d25b29640 1 --2- 172.21.15.16:0/989395044 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7d0406c2b0 0x7f7d0406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:14.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.834+0000 7f7d25b29640 1 --2- 172.21.15.16:0/989395044 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7d0406c2b0 0x7f7d0406e770 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c006fd0 tx=0x7f7d1c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:14.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:14.838+0000 7f7d167fc640 1 -- 172.21.15.16:0/989395044 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7d1007a850 con 0x7f7d200d21f0 2024-09-19T07:50:15.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.049+0000 7f7d27d8d640 1 -- 172.21.15.16:0/989395044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7d20072290 con 0x7f7d200d21f0 2024-09-19T07:50:15.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.686+0000 7f7d167fc640 1 -- 172.21.15.16:0/989395044 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v45) ==== 174+0+0 (secure 0 0 0) 0x7f7d1007a150 con 0x7f7d200d21f0 2024-09-19T07:50:15.686 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 13 and data pool 14 2024-09-19T07:50:15.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.690+0000 7f7d27d8d640 1 -- 172.21.15.16:0/989395044 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7d0406c2b0 msgr2=0x7f7d0406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:15.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.690+0000 7f7d27d8d640 1 --2- 172.21.15.16:0/989395044 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7d0406c2b0 0x7f7d0406e770 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c006fd0 tx=0x7f7d1c008040 comp rx=0 tx=0).stop 2024-09-19T07:50:15.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.690+0000 7f7d27d8d640 1 -- 172.21.15.16:0/989395044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d200d21f0 msgr2=0x7f7d200c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:15.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.690+0000 7f7d27d8d640 1 --2- 172.21.15.16:0/989395044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d200d21f0 0x7f7d200c8190 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7d10000c00 tx=0x7f7d100046c0 comp rx=0 tx=0).stop 2024-09-19T07:50:15.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.691+0000 7f7d27d8d640 1 -- 172.21.15.16:0/989395044 shutdown_connections 2024-09-19T07:50:15.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.691+0000 7f7d27d8d640 1 --2- 172.21.15.16:0/989395044 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7d0406c2b0 0x7f7d0406e770 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:15.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.691+0000 7f7d27d8d640 1 --2- 172.21.15.16:0/989395044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d20143a60 0x7f7d200c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:15.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.691+0000 7f7d27d8d640 1 --2- 172.21.15.16:0/989395044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d200d21f0 0x7f7d200c8190 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:15.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.691+0000 7f7d27d8d640 1 --2- 172.21.15.16:0/989395044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d20072590 0x7f7d200b6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:15.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.691+0000 7f7d27d8d640 1 -- 172.21.15.16:0/989395044 >> 172.21.15.16:0/989395044 conn(0x7f7d200b3ed0 msgr2=0x7f7d200b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:15.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.691+0000 7f7d27d8d640 1 -- 172.21.15.16:0/989395044 shutdown_connections 2024-09-19T07:50:15.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.692+0000 7f7d27d8d640 1 -- 172.21.15.16:0/989395044 wait complete. 2024-09-19T07:50:15.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:50:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.829+0000 7ff26e58d640 1 Processor -- start 2024-09-19T07:50:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.829+0000 7ff26e58d640 1 -- start start 2024-09-19T07:50:15.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.830+0000 7ff26e58d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff26815a7b0 0x7ff2680c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:15.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.830+0000 7ff26e58d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2680cd040 0x7ff2680b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:15.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.830+0000 7ff26e58d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2680b4540 con 0x7ff2680cde40 2024-09-19T07:50:15.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.830+0000 7ff26e58d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff2680b4870 con 0x7ff26815a7b0 2024-09-19T07:50:15.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.830+0000 7ff26e58d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff2680cf9b0 con 0x7ff2680cd040 2024-09-19T07:50:15.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.830+0000 7ff267fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff26815a7b0 0x7ff2680c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:15.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.830+0000 7ff267fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff26815a7b0 0x7ff2680c5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39952/0 (socket says 172.21.15.16:39952) 2024-09-19T07:50:15.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.830+0000 7ff267fff640 1 -- 172.21.15.16:0/3587524785 learned_addr learned my addr 172.21.15.16:0/3587524785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:15.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.831+0000 7ff26cb2a640 1 --2- 172.21.15.16:0/3587524785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2680cd040 0x7ff2680b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:15.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.831+0000 7ff267fff640 1 -- 172.21.15.16:0/3587524785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2680cd040 msgr2=0x7ff2680b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:15.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.831+0000 7ff267fff640 1 --2- 172.21.15.16:0/3587524785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2680cd040 0x7ff2680b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:15.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.831+0000 7ff267fff640 1 -- 172.21.15.16:0/3587524785 >> v1:172.21.15.146:6789/0 conn(0x7ff2680cde40 legacy=0x7ff2680cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:15.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.831+0000 7ff267fff640 1 -- 172.21.15.16:0/3587524785 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2680cfc90 con 0x7ff26815a7b0 2024-09-19T07:50:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.831+0000 7ff267fff640 1 --2- 172.21.15.16:0/3587524785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26815a7b0 0x7ff2680c5b00 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff25800aa10 tx=0x7ff25802b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d7f5ebb147428b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.832+0000 7ff266ffd640 1 -- 172.21.15.16:0/3587524785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25800b040 con 0x7ff26815a7b0 2024-09-19T07:50:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.832+0000 7ff266ffd640 1 -- 172.21.15.16:0/3587524785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff258030710 con 0x7ff26815a7b0 2024-09-19T07:50:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.832+0000 7ff266ffd640 1 -- 172.21.15.16:0/3587524785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258030ae0 con 0x7ff26815a7b0 2024-09-19T07:50:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.832+0000 7ff26e58d640 1 -- 172.21.15.16:0/3587524785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26815a7b0 msgr2=0x7ff2680c5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.832+0000 7ff26e58d640 1 --2- 172.21.15.16:0/3587524785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26815a7b0 0x7ff2680c5b00 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff25800aa10 tx=0x7ff25802b610 comp rx=0 tx=0).stop 2024-09-19T07:50:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.832+0000 7ff26e58d640 1 -- 172.21.15.16:0/3587524785 shutdown_connections 2024-09-19T07:50:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.832+0000 7ff26e58d640 1 --2- 172.21.15.16:0/3587524785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2680cd040 0x7ff2680b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.832+0000 7ff26e58d640 1 --2- 172.21.15.16:0/3587524785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26815a7b0 0x7ff2680c5b00 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.832+0000 7ff26e58d640 1 -- 172.21.15.16:0/3587524785 >> 172.21.15.16:0/3587524785 conn(0x7ff26806bdf0 msgr2=0x7ff26806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.832+0000 7ff26e58d640 1 -- 172.21.15.16:0/3587524785 shutdown_connections 2024-09-19T07:50:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.832+0000 7ff26e58d640 1 -- 172.21.15.16:0/3587524785 wait complete. 2024-09-19T07:50:15.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.833+0000 7ff26e58d640 1 Processor -- start 2024-09-19T07:50:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.834+0000 7ff26e58d640 1 -- start start 2024-09-19T07:50:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.834+0000 7ff26e58d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2680cd040 0x7ff2680c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.834+0000 7ff26e58d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2680cde40 0x7ff2680c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.834+0000 7ff26e58d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff26815a7b0 0x7ff2681f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.834+0000 7ff26e58d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2680ca8c0 con 0x7ff2680cd040 2024-09-19T07:50:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.834+0000 7ff26e58d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff26815ab90 con 0x7ff26815a7b0 2024-09-19T07:50:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.834+0000 7ff26e58d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff2680ce280 con 0x7ff2680cde40 2024-09-19T07:50:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.834+0000 7ff267fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2680cd040 0x7ff2680c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.834+0000 7ff267fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2680cd040 0x7ff2680c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39960/0 (socket says 172.21.15.16:39960) 2024-09-19T07:50:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.834+0000 7ff267fff640 1 -- 172.21.15.16:0/1162608269 learned_addr learned my addr 172.21.15.16:0/1162608269 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.834+0000 7ff2677fe640 1 --2- 172.21.15.16:0/1162608269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2680cde40 0x7ff2680c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.835+0000 7ff26cb2a640 1 --2- 172.21.15.16:0/1162608269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff26815a7b0 0x7ff2681f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.835+0000 7ff267fff640 1 -- 172.21.15.16:0/1162608269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2680cde40 msgr2=0x7ff2680c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.835+0000 7ff267fff640 1 --2- 172.21.15.16:0/1162608269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2680cde40 0x7ff2680c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.835+0000 7ff267fff640 1 -- 172.21.15.16:0/1162608269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff26815a7b0 msgr2=0x7ff2681f1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.835+0000 7ff267fff640 1 --2- 172.21.15.16:0/1162608269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff26815a7b0 0x7ff2681f1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.835+0000 7ff267fff640 1 -- 172.21.15.16:0/1162608269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2680c9bb0 con 0x7ff2680cd040 2024-09-19T07:50:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.835+0000 7ff2677fe640 1 --2- 172.21.15.16:0/1162608269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2680cde40 0x7ff2680c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.835+0000 7ff267fff640 1 --2- 172.21.15.16:0/1162608269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2680cd040 0x7ff2680c95d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff25800a1c0 tx=0x7ff2580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.835+0000 7ff264ff9640 1 -- 172.21.15.16:0/1162608269 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258030e30 con 0x7ff2680cd040 2024-09-19T07:50:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.835+0000 7ff264ff9640 1 -- 172.21.15.16:0/1162608269 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff258040070 con 0x7ff2680cd040 2024-09-19T07:50:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.835+0000 7ff26e58d640 1 -- 172.21.15.16:0/1162608269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2680c9e90 con 0x7ff2680cd040 2024-09-19T07:50:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.835+0000 7ff264ff9640 1 -- 172.21.15.16:0/1162608269 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25803b430 con 0x7ff2680cd040 2024-09-19T07:50:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.835+0000 7ff26e58d640 1 -- 172.21.15.16:0/1162608269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2680ca450 con 0x7ff2680cd040 2024-09-19T07:50:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.836+0000 7ff26e58d640 1 -- 172.21.15.16:0/1162608269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2680c5b00 con 0x7ff2680cd040 2024-09-19T07:50:15.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.836+0000 7ff264ff9640 1 -- 172.21.15.16:0/1162608269 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff25803b5d0 con 0x7ff2680cd040 2024-09-19T07:50:15.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.837+0000 7ff264ff9640 1 --2- 172.21.15.16:0/1162608269 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff25406c2b0 0x7ff25406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:15.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.837+0000 7ff264ff9640 1 -- 172.21.15.16:0/1162608269 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7ff2580abfc0 con 0x7ff2680cd040 2024-09-19T07:50:15.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.837+0000 7ff2677fe640 1 --2- 172.21.15.16:0/1162608269 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff25406c2b0 0x7ff25406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:15.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.838+0000 7ff2677fe640 1 --2- 172.21.15.16:0/1162608269 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff25406c2b0 0x7ff25406e770 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff25c007fd0 tx=0x7ff25c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:15.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:15.841+0000 7ff264ff9640 1 -- 172.21.15.16:0/1162608269 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff25807a5a0 con 0x7ff2680cd040 2024-09-19T07:50:16.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.021+0000 7ff26e58d640 1 -- 172.21.15.16:0/1162608269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff26806fcc0 con 0x7ff2680cd040 2024-09-19T07:50:16.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.022+0000 7ff264ff9640 1 -- 172.21.15.16:0/1162608269 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16982 (secure 0 0 0) 0x7ff258079ea0 con 0x7ff2680cd040 2024-09-19T07:50:16.023 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:16.023 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":92,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:50:15.050380+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:50:13.015554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-19T07:50:14.031234+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:50:16.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.025+0000 7ff26e58d640 1 -- 172.21.15.16:0/1162608269 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff25406c2b0 msgr2=0x7ff25406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:16.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.025+0000 7ff26e58d640 1 --2- 172.21.15.16:0/1162608269 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff25406c2b0 0x7ff25406e770 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff25c007fd0 tx=0x7ff25c009040 comp rx=0 tx=0).stop 2024-09-19T07:50:16.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.025+0000 7ff26e58d640 1 -- 172.21.15.16:0/1162608269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2680cd040 msgr2=0x7ff2680c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:16.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.025+0000 7ff26e58d640 1 --2- 172.21.15.16:0/1162608269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2680cd040 0x7ff2680c95d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff25800a1c0 tx=0x7ff2580046c0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.025+0000 7ff26e58d640 1 -- 172.21.15.16:0/1162608269 shutdown_connections 2024-09-19T07:50:16.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.025+0000 7ff26e58d640 1 --2- 172.21.15.16:0/1162608269 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff25406c2b0 0x7ff25406e770 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.025+0000 7ff26e58d640 1 --2- 172.21.15.16:0/1162608269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff26815a7b0 0x7ff2681f1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.025+0000 7ff26e58d640 1 --2- 172.21.15.16:0/1162608269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2680cde40 0x7ff2680c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.025+0000 7ff26e58d640 1 --2- 172.21.15.16:0/1162608269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2680cd040 0x7ff2680c95d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.025+0000 7ff26e58d640 1 -- 172.21.15.16:0/1162608269 >> 172.21.15.16:0/1162608269 conn(0x7ff26806bdf0 msgr2=0x7ff2680d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:16.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.025+0000 7ff26e58d640 1 -- 172.21.15.16:0/1162608269 shutdown_connections 2024-09-19T07:50:16.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.025+0000 7ff26e58d640 1 -- 172.21.15.16:0/1162608269 wait complete. 2024-09-19T07:50:16.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:50:16.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.138+0000 7f161bf31640 1 Processor -- start 2024-09-19T07:50:16.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.138+0000 7f161bf31640 1 -- start start 2024-09-19T07:50:16.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.139+0000 7f161bf31640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1614142170 0x7f1614166d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.139+0000 7f161bf31640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f161415d710 con 0x7f1614141680 2024-09-19T07:50:16.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.139+0000 7f161bf31640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f161415c950 con 0x7f16140d0540 2024-09-19T07:50:16.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.139+0000 7f161bf31640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f161415c6f0 con 0x7f1614142170 2024-09-19T07:50:16.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.139+0000 7f1619ccd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1614141680 0x7f16140cfe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50596/0 (socket says 172.21.15.16:50596) 2024-09-19T07:50:16.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.139+0000 7f1619ccd640 1 -- 172.21.15.16:0/1060696611 learned_addr learned my addr 172.21.15.16:0/1060696611 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:16.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f161a4ce640 1 --2- 172.21.15.16:0/1060696611 >> v2:172.21.15.146:3300/0 conn(0x7f1614142170 0x7f1614166d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1618ccb640 1 -- 172.21.15.16:0/1060696611 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3636893449 0 0) 0x7f161415d710 con 0x7f1614141680 2024-09-19T07:50:16.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1618ccb640 1 -- 172.21.15.16:0/1060696611 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1600003520 con 0x7f1614141680 2024-09-19T07:50:16.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1618ccb640 1 -- 172.21.15.16:0/1060696611 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 817537117 0 0) 0x7f1600003520 con 0x7f1614141680 2024-09-19T07:50:16.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1618ccb640 1 -- 172.21.15.16:0/1060696611 >> v2:172.21.15.146:3300/0 conn(0x7f1614142170 msgr2=0x7f1614166d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:16.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1618ccb640 1 --2- 172.21.15.16:0/1060696611 >> v2:172.21.15.146:3300/0 conn(0x7f1614142170 0x7f1614166d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1618ccb640 1 -- 172.21.15.16:0/1060696611 >> v1:172.21.15.146:6789/0 conn(0x7f16140d0540 legacy=0x7f1614141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:16.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1618ccb640 1 -- 172.21.15.16:0/1060696611 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161415cc30 con 0x7f1614141680 2024-09-19T07:50:16.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1618ccb640 1 -- 172.21.15.16:0/1060696611 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f1610001670 con 0x7f1614141680 2024-09-19T07:50:16.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1618ccb640 1 -- 172.21.15.16:0/1060696611 >> v1:172.21.15.16:6789/0 conn(0x7f1614141680 legacy=0x7f16140cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:16.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1618ccb640 1 --2- 172.21.15.16:0/1060696611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000039b0 0x7f1600003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1618ccb640 1 --2- 172.21.15.16:0/1060696611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1600004800 0x7f1600004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1618ccb640 1 --2- 172.21.15.16:0/1060696611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16000051c0 0x7f160000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1618ccb640 1 -- 172.21.15.16:0/1060696611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f161415c6f0 con 0x7f16000039b0 2024-09-19T07:50:16.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1618ccb640 1 -- 172.21.15.16:0/1060696611 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f160000c060 con 0x7f1600004800 2024-09-19T07:50:16.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1618ccb640 1 -- 172.21.15.16:0/1060696611 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f160000c260 con 0x7f16000051c0 2024-09-19T07:50:16.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1619ccd640 1 --2- 172.21.15.16:0/1060696611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000039b0 0x7f1600003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f16194cc640 1 --2- 172.21.15.16:0/1060696611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1600004800 0x7f1600004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f161a4ce640 1 --2- 172.21.15.16:0/1060696611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16000051c0 0x7f160000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1619ccd640 1 -- 172.21.15.16:0/1060696611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16000051c0 msgr2=0x7f160000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:16.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1619ccd640 1 --2- 172.21.15.16:0/1060696611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16000051c0 0x7f160000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1619ccd640 1 -- 172.21.15.16:0/1060696611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1600004800 msgr2=0x7f1600004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:16.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1619ccd640 1 --2- 172.21.15.16:0/1060696611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1600004800 0x7f1600004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f1619ccd640 1 -- 172.21.15.16:0/1060696611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1600003520 con 0x7f16000039b0 2024-09-19T07:50:16.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.140+0000 7f161a4ce640 1 -- 172.21.15.16:0/1060696611 reap_dead start 2024-09-19T07:50:16.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.141+0000 7f1619ccd640 1 --2- 172.21.15.16:0/1060696611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000039b0 0x7f1600003dd0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f161000c240 tx=0x7f1600005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f77f003d8800d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:16.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.141+0000 7f1618ccb640 1 -- 172.21.15.16:0/1060696611 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610007d10 con 0x7f16000039b0 2024-09-19T07:50:16.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.141+0000 7f1618ccb640 1 -- 172.21.15.16:0/1060696611 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1610005ad0 con 0x7f16000039b0 2024-09-19T07:50:16.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.141+0000 7f1618ccb640 1 -- 172.21.15.16:0/1060696611 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610007070 con 0x7f16000039b0 2024-09-19T07:50:16.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.142+0000 7f161bf31640 1 -- 172.21.15.16:0/1060696611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000039b0 msgr2=0x7f1600003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:16.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.142+0000 7f161bf31640 1 --2- 172.21.15.16:0/1060696611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000039b0 0x7f1600003dd0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f161000c240 tx=0x7f1600005830 comp rx=0 tx=0).stop 2024-09-19T07:50:16.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.142+0000 7f161bf31640 1 -- 172.21.15.16:0/1060696611 shutdown_connections 2024-09-19T07:50:16.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.142+0000 7f161bf31640 1 --2- 172.21.15.16:0/1060696611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000039b0 0x7f1600003dd0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.142+0000 7f161bf31640 1 -- 172.21.15.16:0/1060696611 >> 172.21.15.16:0/1060696611 conn(0x7f16140b3ed0 msgr2=0x7f16140b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:16.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.142+0000 7f161bf31640 1 -- 172.21.15.16:0/1060696611 shutdown_connections 2024-09-19T07:50:16.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.142+0000 7f161bf31640 1 -- 172.21.15.16:0/1060696611 wait complete. 2024-09-19T07:50:16.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.143+0000 7f161bf31640 1 Processor -- start 2024-09-19T07:50:16.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.143+0000 7f161bf31640 1 -- start start 2024-09-19T07:50:16.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.144+0000 7f161bf31640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16000039b0 0x7f1614158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.144+0000 7f161bf31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16141596d0 0x7f16140b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.144+0000 7f161bf31640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1614159b50 0x7f16140b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.144+0000 7f161bf31640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16141425c0 con 0x7f16141596d0 2024-09-19T07:50:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.144+0000 7f161bf31640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f161415d1f0 con 0x7f1614159b50 2024-09-19T07:50:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.144+0000 7f161bf31640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1614141a60 con 0x7f16000039b0 2024-09-19T07:50:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.144+0000 7f16194cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16141596d0 0x7f16140b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.144+0000 7f16194cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16141596d0 0x7f16140b59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39982/0 (socket says 172.21.15.16:39982) 2024-09-19T07:50:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.144+0000 7f16194cc640 1 -- 172.21.15.16:0/4138974773 learned_addr learned my addr 172.21.15.16:0/4138974773 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.144+0000 7f1619ccd640 1 --2- 172.21.15.16:0/4138974773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16000039b0 0x7f1614158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.144+0000 7f161a4ce640 1 --2- 172.21.15.16:0/4138974773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1614159b50 0x7f16140b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.144+0000 7f16194cc640 1 -- 172.21.15.16:0/4138974773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16000039b0 msgr2=0x7f1614158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.144+0000 7f16194cc640 1 --2- 172.21.15.16:0/4138974773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16000039b0 0x7f1614158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.144+0000 7f16194cc640 1 -- 172.21.15.16:0/4138974773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1614159b50 msgr2=0x7f16140b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.144+0000 7f16194cc640 1 --2- 172.21.15.16:0/4138974773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1614159b50 0x7f16140b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.144+0000 7f16194cc640 1 -- 172.21.15.16:0/4138974773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1614155260 con 0x7f16141596d0 2024-09-19T07:50:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.144+0000 7f161a4ce640 1 --2- 172.21.15.16:0/4138974773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1614159b50 0x7f16140b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.144+0000 7f16194cc640 1 --2- 172.21.15.16:0/4138974773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16141596d0 0x7f16140b59c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f160400f5f0 tx=0x7f160400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.145+0000 7f160a7fc640 1 -- 172.21.15.16:0/4138974773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16040052e0 con 0x7f16141596d0 2024-09-19T07:50:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.145+0000 7f160a7fc640 1 -- 172.21.15.16:0/4138974773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1604018070 con 0x7f16141596d0 2024-09-19T07:50:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.145+0000 7f160a7fc640 1 -- 172.21.15.16:0/4138974773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1604003810 con 0x7f16141596d0 2024-09-19T07:50:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.145+0000 7f161bf31640 1 -- 172.21.15.16:0/4138974773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1614155540 con 0x7f16141596d0 2024-09-19T07:50:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.145+0000 7f161bf31640 1 -- 172.21.15.16:0/4138974773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1614155b00 con 0x7f16141596d0 2024-09-19T07:50:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.146+0000 7f161bf31640 1 -- 172.21.15.16:0/4138974773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1614141df0 con 0x7f16141596d0 2024-09-19T07:50:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.146+0000 7f160a7fc640 1 -- 172.21.15.16:0/4138974773 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f16040039b0 con 0x7f16141596d0 2024-09-19T07:50:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.147+0000 7f160a7fc640 1 --2- 172.21.15.16:0/4138974773 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f15f806c2b0 0x7f15f806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.147+0000 7f160a7fc640 1 -- 172.21.15.16:0/4138974773 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f160408b3d0 con 0x7f16141596d0 2024-09-19T07:50:16.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.147+0000 7f1619ccd640 1 --2- 172.21.15.16:0/4138974773 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f15f806c2b0 0x7f15f806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.148+0000 7f1619ccd640 1 --2- 172.21.15.16:0/4138974773 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f15f806c2b0 0x7f15f806e770 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1610032000 tx=0x7f161002d000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:16.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.151+0000 7f160a7fc640 1 -- 172.21.15.16:0/4138974773 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1604059910 con 0x7f16141596d0 2024-09-19T07:50:16.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.327+0000 7f161bf31640 1 -- 172.21.15.16:0/4138974773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16140690c0 con 0x7f16141596d0 2024-09-19T07:50:16.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.328+0000 7f160a7fc640 1 -- 172.21.15.16:0/4138974773 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16982 (secure 0 0 0) 0x7f1604059210 con 0x7f16141596d0 2024-09-19T07:50:16.329 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:16.329 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":92,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:50:15.050380+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:50:13.015554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-19T07:50:14.031234+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:50:16.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.330+0000 7f161bf31640 1 -- 172.21.15.16:0/4138974773 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f15f806c2b0 msgr2=0x7f15f806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.330+0000 7f161bf31640 1 --2- 172.21.15.16:0/4138974773 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f15f806c2b0 0x7f15f806e770 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1610032000 tx=0x7f161002d000 comp rx=0 tx=0).stop 2024-09-19T07:50:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.330+0000 7f161bf31640 1 -- 172.21.15.16:0/4138974773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16141596d0 msgr2=0x7f16140b59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.330+0000 7f161bf31640 1 --2- 172.21.15.16:0/4138974773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16141596d0 0x7f16140b59c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f160400f5f0 tx=0x7f160400fab0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.330+0000 7f161bf31640 1 -- 172.21.15.16:0/4138974773 shutdown_connections 2024-09-19T07:50:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.330+0000 7f161bf31640 1 --2- 172.21.15.16:0/4138974773 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f15f806c2b0 0x7f15f806e770 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.330+0000 7f161bf31640 1 --2- 172.21.15.16:0/4138974773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1614159b50 0x7f16140b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.330+0000 7f161bf31640 1 --2- 172.21.15.16:0/4138974773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16141596d0 0x7f16140b59c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.330+0000 7f161bf31640 1 --2- 172.21.15.16:0/4138974773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16000039b0 0x7f1614158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.330+0000 7f161bf31640 1 -- 172.21.15.16:0/4138974773 >> 172.21.15.16:0/4138974773 conn(0x7f16140b3ed0 msgr2=0x7f16141560b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:16.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.330+0000 7f161bf31640 1 -- 172.21.15.16:0/4138974773 shutdown_connections 2024-09-19T07:50:16.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.330+0000 7f161bf31640 1 -- 172.21.15.16:0/4138974773 wait complete. 2024-09-19T07:50:16.343 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T07:50:16.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.448+0000 7f0d8dea6640 1 Processor -- start 2024-09-19T07:50:16.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.448+0000 7f0d8dea6640 1 -- start start 2024-09-19T07:50:16.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.448+0000 7f0d8dea6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0d8815e2a0 0x7f0d8806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.448+0000 7f0d8dea6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d880d1880 con 0x7f0d8815e680 2024-09-19T07:50:16.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.448+0000 7f0d8dea6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d880d1b60 con 0x7f0d8806d960 2024-09-19T07:50:16.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.448+0000 7f0d8dea6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f0d880cf2d0 con 0x7f0d8815e2a0 2024-09-19T07:50:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.449+0000 7f0d877fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0d8815e2a0 0x7f0d8806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.449+0000 7f0d86ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0d8815e680 0x7f0d880cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50598/0 (socket says 172.21.15.16:50598) 2024-09-19T07:50:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.449+0000 7f0d877fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0d8815e2a0 0x7f0d8806d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39992/0 (socket says 172.21.15.16:39992) 2024-09-19T07:50:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.449+0000 7f0d86ffd640 1 -- 172.21.15.16:0/514352178 learned_addr learned my addr 172.21.15.16:0/514352178 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.449+0000 7f0d877fe640 1 -- 172.21.15.16:0/514352178 >> v1:172.21.15.146:6789/0 conn(0x7f0d8806d960 legacy=0x7f0d880d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.449+0000 7f0d877fe640 1 -- 172.21.15.16:0/514352178 >> v1:172.21.15.16:6789/0 conn(0x7f0d8815e680 legacy=0x7f0d880cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.449+0000 7f0d867fc640 1 -- 172.21.15.16:0/514352178 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2444176028 0 0) 0x7f0d880d1880 con 0x7f0d8815e680 2024-09-19T07:50:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.449+0000 7f0d877fe640 1 -- 172.21.15.16:0/514352178 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d880cf5b0 con 0x7f0d8815e2a0 2024-09-19T07:50:16.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.450+0000 7f0d877fe640 1 --2- 172.21.15.16:0/514352178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d8815e2a0 0x7f0d8806d420 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0d780097a0 tx=0x7f0d7802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386c802ad4f3ca9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:16.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.450+0000 7f0d867fc640 1 -- 172.21.15.16:0/514352178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78037070 con 0x7f0d8815e2a0 2024-09-19T07:50:16.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.450+0000 7f0d867fc640 1 -- 172.21.15.16:0/514352178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d78032440 con 0x7f0d8815e2a0 2024-09-19T07:50:16.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.450+0000 7f0d867fc640 1 -- 172.21.15.16:0/514352178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78032810 con 0x7f0d8815e2a0 2024-09-19T07:50:16.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.450+0000 7f0d8dea6640 1 -- 172.21.15.16:0/514352178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d8815e2a0 msgr2=0x7f0d8806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:16.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.450+0000 7f0d8dea6640 1 --2- 172.21.15.16:0/514352178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d8815e2a0 0x7f0d8806d420 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0d780097a0 tx=0x7f0d7802a1e0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.451+0000 7f0d8dea6640 1 -- 172.21.15.16:0/514352178 shutdown_connections 2024-09-19T07:50:16.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.451+0000 7f0d8dea6640 1 --2- 172.21.15.16:0/514352178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d8815e2a0 0x7f0d8806d420 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.451+0000 7f0d8dea6640 1 -- 172.21.15.16:0/514352178 >> 172.21.15.16:0/514352178 conn(0x7f0d8809d760 msgr2=0x7f0d8808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:16.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.451+0000 7f0d8dea6640 1 -- 172.21.15.16:0/514352178 shutdown_connections 2024-09-19T07:50:16.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.451+0000 7f0d8dea6640 1 -- 172.21.15.16:0/514352178 wait complete. 2024-09-19T07:50:16.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.452+0000 7f0d8dea6640 1 Processor -- start 2024-09-19T07:50:16.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.452+0000 7f0d8dea6640 1 -- start start 2024-09-19T07:50:16.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d8dea6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d8806d960 0x7f0d880b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d8dea6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d8815e2a0 0x7f0d880c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d8dea6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d8815e680 0x7f0d8815d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d8dea6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d880d0360 con 0x7f0d8815e680 2024-09-19T07:50:16.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d8dea6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0d880d01e0 con 0x7f0d8806d960 2024-09-19T07:50:16.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d8dea6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0d880d4a80 con 0x7f0d8815e2a0 2024-09-19T07:50:16.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d8815e680 0x7f0d8815d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d8815e680 0x7f0d8815d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39996/0 (socket says 172.21.15.16:39996) 2024-09-19T07:50:16.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d87fff640 1 -- 172.21.15.16:0/2876519362 learned_addr learned my addr 172.21.15.16:0/2876519362 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:16.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d86ffd640 1 --2- 172.21.15.16:0/2876519362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d8815e2a0 0x7f0d880c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d877fe640 1 --2- 172.21.15.16:0/2876519362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d8806d960 0x7f0d880b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d87fff640 1 -- 172.21.15.16:0/2876519362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d8815e2a0 msgr2=0x7f0d880c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:16.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d87fff640 1 --2- 172.21.15.16:0/2876519362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d8815e2a0 0x7f0d880c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d87fff640 1 -- 172.21.15.16:0/2876519362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d8806d960 msgr2=0x7f0d880b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:16.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d87fff640 1 --2- 172.21.15.16:0/2876519362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d8806d960 0x7f0d880b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d87fff640 1 -- 172.21.15.16:0/2876519362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d880c8550 con 0x7f0d8815e680 2024-09-19T07:50:16.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d877fe640 1 --2- 172.21.15.16:0/2876519362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d8806d960 0x7f0d880b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:16.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d87fff640 1 --2- 172.21.15.16:0/2876519362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d8815e680 0x7f0d8815d3e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000c780 tx=0x7f0d8000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:16.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d6ffff640 1 -- 172.21.15.16:0/2876519362 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80004b60 con 0x7f0d8815e680 2024-09-19T07:50:16.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d6ffff640 1 -- 172.21.15.16:0/2876519362 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d80004d00 con 0x7f0d8815e680 2024-09-19T07:50:16.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.453+0000 7f0d8dea6640 1 -- 172.21.15.16:0/2876519362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d880c8830 con 0x7f0d8815e680 2024-09-19T07:50:16.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.454+0000 7f0d6ffff640 1 -- 172.21.15.16:0/2876519362 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8000aa30 con 0x7f0d8815e680 2024-09-19T07:50:16.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.454+0000 7f0d8dea6640 1 -- 172.21.15.16:0/2876519362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d880b59e0 con 0x7f0d8815e680 2024-09-19T07:50:16.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.454+0000 7f0d8dea6640 1 -- 172.21.15.16:0/2876519362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d8806d420 con 0x7f0d8815e680 2024-09-19T07:50:16.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.455+0000 7f0d6ffff640 1 -- 172.21.15.16:0/2876519362 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0d8000abd0 con 0x7f0d8815e680 2024-09-19T07:50:16.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.455+0000 7f0d6ffff640 1 --2- 172.21.15.16:0/2876519362 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0d7406c2b0 0x7f0d7406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.456+0000 7f0d6ffff640 1 -- 172.21.15.16:0/2876519362 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f0d8008bac0 con 0x7f0d8815e680 2024-09-19T07:50:16.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.456+0000 7f0d877fe640 1 --2- 172.21.15.16:0/2876519362 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0d7406c2b0 0x7f0d7406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.456+0000 7f0d877fe640 1 --2- 172.21.15.16:0/2876519362 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0d7406c2b0 0x7f0d7406e770 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0d78034000 tx=0x7f0d7802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:16.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.460+0000 7f0d6ffff640 1 -- 172.21.15.16:0/2876519362 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f0d8005a090 con 0x7f0d8815e680 2024-09-19T07:50:16.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.671+0000 7f0d8dea6640 1 -- 172.21.15.16:0/2876519362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0d880c7c90 con 0x7f0d8815e680 2024-09-19T07:50:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.699+0000 7f0d6ffff640 1 -- 172.21.15.16:0/2876519362 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v46) ==== 118+0+0 (secure 0 0 0) 0x7f0d80059990 con 0x7f0d8815e680 2024-09-19T07:50:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.702+0000 7f0d8dea6640 1 -- 172.21.15.16:0/2876519362 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0d7406c2b0 msgr2=0x7f0d7406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.702+0000 7f0d8dea6640 1 --2- 172.21.15.16:0/2876519362 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0d7406c2b0 0x7f0d7406e770 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0d78034000 tx=0x7f0d7802e000 comp rx=0 tx=0).stop 2024-09-19T07:50:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.702+0000 7f0d8dea6640 1 -- 172.21.15.16:0/2876519362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d8815e680 msgr2=0x7f0d8815d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.702+0000 7f0d8dea6640 1 --2- 172.21.15.16:0/2876519362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d8815e680 0x7f0d8815d3e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000c780 tx=0x7f0d8000cc40 comp rx=0 tx=0).stop 2024-09-19T07:50:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.702+0000 7f0d8dea6640 1 -- 172.21.15.16:0/2876519362 shutdown_connections 2024-09-19T07:50:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.702+0000 7f0d8dea6640 1 --2- 172.21.15.16:0/2876519362 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0d7406c2b0 0x7f0d7406e770 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.702+0000 7f0d8dea6640 1 --2- 172.21.15.16:0/2876519362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d8815e680 0x7f0d8815d3e0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.702+0000 7f0d8dea6640 1 --2- 172.21.15.16:0/2876519362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d8815e2a0 0x7f0d880c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.702+0000 7f0d8dea6640 1 --2- 172.21.15.16:0/2876519362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d8806d960 0x7f0d880b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.702+0000 7f0d8dea6640 1 -- 172.21.15.16:0/2876519362 >> 172.21.15.16:0/2876519362 conn(0x7f0d8809d760 msgr2=0x7f0d880d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.702+0000 7f0d8dea6640 1 -- 172.21.15.16:0/2876519362 shutdown_connections 2024-09-19T07:50:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.702+0000 7f0d8dea6640 1 -- 172.21.15.16:0/2876519362 wait complete. 2024-09-19T07:50:16.715 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T07:50:16.715 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T07:50:16.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.820+0000 7f9733118640 1 Processor -- start 2024-09-19T07:50:16.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.820+0000 7f9733118640 1 -- start start 2024-09-19T07:50:16.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f9733118640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f972c15e680 0x7f972c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f9733118640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f972c0d0010 0x7f972c0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f9733118640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f972c0d1880 con 0x7f972c15e2a0 2024-09-19T07:50:16.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f9733118640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f972c0d1b60 con 0x7f972c0d0010 2024-09-19T07:50:16.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f9733118640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f972c0cf2d0 con 0x7f972c15e680 2024-09-19T07:50:16.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f9730eb4640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f972c15e2a0 0x7f972c06d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50614/0 (socket says 172.21.15.16:50614) 2024-09-19T07:50:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f9730eb4640 1 -- 172.21.15.16:0/2209825031 learned_addr learned my addr 172.21.15.16:0/2209825031 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f972bfff640 1 --2- 172.21.15.16:0/2209825031 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f972c15e680 0x7f972c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f97316b5640 1 --2- 172.21.15.16:0/2209825031 >> v2:172.21.15.146:3300/0 conn(0x7f972c0d0010 0x7f972c0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f972b7fe640 1 -- 172.21.15.16:0/2209825031 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3920258494 0 0) 0x7f972c0d1880 con 0x7f972c15e2a0 2024-09-19T07:50:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f972b7fe640 1 -- 172.21.15.16:0/2209825031 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f971c003520 con 0x7f972c15e2a0 2024-09-19T07:50:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f972b7fe640 1 -- 172.21.15.16:0/2209825031 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 686341181 0 0) 0x7f971c003520 con 0x7f972c15e2a0 2024-09-19T07:50:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f972b7fe640 1 -- 172.21.15.16:0/2209825031 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f972c15e680 msgr2=0x7f972c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f972b7fe640 1 --2- 172.21.15.16:0/2209825031 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f972c15e680 0x7f972c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f972b7fe640 1 -- 172.21.15.16:0/2209825031 >> v2:172.21.15.146:3300/0 conn(0x7f972c0d0010 msgr2=0x7f972c0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f972b7fe640 1 --2- 172.21.15.16:0/2209825031 >> v2:172.21.15.146:3300/0 conn(0x7f972c0d0010 0x7f972c0d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f972b7fe640 1 -- 172.21.15.16:0/2209825031 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972c0cf5b0 con 0x7f972c15e2a0 2024-09-19T07:50:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f972b7fe640 1 -- 172.21.15.16:0/2209825031 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f9720001670 con 0x7f972c15e2a0 2024-09-19T07:50:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f97316b5640 1 --2- 172.21.15.16:0/2209825031 >> v2:172.21.15.146:3300/0 conn(0x7f972c0d0010 0x7f972c0d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f972b7fe640 1 -- 172.21.15.16:0/2209825031 >> v1:172.21.15.16:6789/0 conn(0x7f972c15e2a0 legacy=0x7f972c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f972b7fe640 1 --2- 172.21.15.16:0/2209825031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f971c0039b0 0x7f971c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f972b7fe640 1 --2- 172.21.15.16:0/2209825031 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f971c004800 0x7f971c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.821+0000 7f972b7fe640 1 --2- 172.21.15.16:0/2209825031 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f971c0051c0 0x7f971c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.822+0000 7f972b7fe640 1 -- 172.21.15.16:0/2209825031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f972c0d1b60 con 0x7f971c0039b0 2024-09-19T07:50:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.822+0000 7f972b7fe640 1 -- 172.21.15.16:0/2209825031 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f972c0cf2d0 con 0x7f971c004800 2024-09-19T07:50:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.822+0000 7f972b7fe640 1 -- 172.21.15.16:0/2209825031 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f971c00bf30 con 0x7f971c0051c0 2024-09-19T07:50:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.822+0000 7f9730eb4640 1 --2- 172.21.15.16:0/2209825031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f971c0039b0 0x7f971c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.822+0000 7f972bfff640 1 --2- 172.21.15.16:0/2209825031 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f972c15e680 0x7f972c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.822+0000 7f97316b5640 1 --2- 172.21.15.16:0/2209825031 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f971c0051c0 0x7f971c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.822+0000 7f972bfff640 1 --2- 172.21.15.16:0/2209825031 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f971c004800 0x7f971c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.822+0000 7f9730eb4640 1 -- 172.21.15.16:0/2209825031 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f971c0051c0 msgr2=0x7f971c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.822+0000 7f9730eb4640 1 --2- 172.21.15.16:0/2209825031 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f971c0051c0 0x7f971c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.822+0000 7f9730eb4640 1 -- 172.21.15.16:0/2209825031 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f971c004800 msgr2=0x7f971c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.822+0000 7f9730eb4640 1 --2- 172.21.15.16:0/2209825031 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f971c004800 0x7f971c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.822+0000 7f9730eb4640 1 -- 172.21.15.16:0/2209825031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971c003520 con 0x7f971c0039b0 2024-09-19T07:50:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.822+0000 7f972bfff640 1 --2- 172.21.15.16:0/2209825031 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f971c004800 0x7f971c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.823+0000 7f9730eb4640 1 --2- 172.21.15.16:0/2209825031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f971c0039b0 0x7f971c003dd0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f972000c210 tx=0x7f971c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6292913bfeb15cb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.823+0000 7f972b7fe640 1 -- 172.21.15.16:0/2209825031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720007d50 con 0x7f971c0039b0 2024-09-19T07:50:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.823+0000 7f972b7fe640 1 -- 172.21.15.16:0/2209825031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f972002e070 con 0x7f971c0039b0 2024-09-19T07:50:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.823+0000 7f972b7fe640 1 -- 172.21.15.16:0/2209825031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720005d30 con 0x7f971c0039b0 2024-09-19T07:50:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.823+0000 7f9733118640 1 -- 172.21.15.16:0/2209825031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f971c0039b0 msgr2=0x7f971c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:16.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.823+0000 7f9733118640 1 --2- 172.21.15.16:0/2209825031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f971c0039b0 0x7f971c003dd0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f972000c210 tx=0x7f971c005830 comp rx=0 tx=0).stop 2024-09-19T07:50:16.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.823+0000 7f97316b5640 1 -- 172.21.15.16:0/2209825031 reap_dead start 2024-09-19T07:50:16.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.823+0000 7f9733118640 1 -- 172.21.15.16:0/2209825031 shutdown_connections 2024-09-19T07:50:16.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.823+0000 7f9733118640 1 -- 172.21.15.16:0/2209825031 >> 172.21.15.16:0/2209825031 conn(0x7f972c09d760 msgr2=0x7f972c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:16.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.823+0000 7f9733118640 1 -- 172.21.15.16:0/2209825031 shutdown_connections 2024-09-19T07:50:16.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.823+0000 7f9733118640 1 -- 172.21.15.16:0/2209825031 wait complete. 2024-09-19T07:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.824+0000 7f9733118640 1 Processor -- start 2024-09-19T07:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.824+0000 7f9733118640 1 -- start start 2024-09-19T07:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.825+0000 7f9733118640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f971c0039b0 0x7f972c0b9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.825+0000 7f9733118640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f972c15cc70 0x7f972c15d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.825+0000 7f9733118640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f972c0c8f60 0x7f972c0c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.825+0000 7f9733118640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f972c06dcb0 con 0x7f971c0039b0 2024-09-19T07:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.825+0000 7f9733118640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f972c06db30 con 0x7f972c15cc70 2024-09-19T07:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.825+0000 7f9733118640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f972c0cfad0 con 0x7f972c0c8f60 2024-09-19T07:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.825+0000 7f9730eb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f971c0039b0 0x7f972c0b9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.826+0000 7f9730eb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f971c0039b0 0x7f972c0b9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40016/0 (socket says 172.21.15.16:40016) 2024-09-19T07:50:16.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.826+0000 7f9730eb4640 1 -- 172.21.15.16:0/4219654582 learned_addr learned my addr 172.21.15.16:0/4219654582 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:16.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.826+0000 7f97316b5640 1 --2- 172.21.15.16:0/4219654582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f972c0c8f60 0x7f972c0c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.826+0000 7f972bfff640 1 --2- 172.21.15.16:0/4219654582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f972c15cc70 0x7f972c15d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.826+0000 7f9730eb4640 1 -- 172.21.15.16:0/4219654582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f972c0c8f60 msgr2=0x7f972c0c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:16.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.826+0000 7f9730eb4640 1 --2- 172.21.15.16:0/4219654582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f972c0c8f60 0x7f972c0c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.826+0000 7f9730eb4640 1 -- 172.21.15.16:0/4219654582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f972c15cc70 msgr2=0x7f972c15d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:16.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.826+0000 7f9730eb4640 1 --2- 172.21.15.16:0/4219654582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f972c15cc70 0x7f972c15d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:16.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.826+0000 7f9730eb4640 1 -- 172.21.15.16:0/4219654582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972c0b8dc0 con 0x7f971c0039b0 2024-09-19T07:50:16.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.826+0000 7f9730eb4640 1 --2- 172.21.15.16:0/4219654582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f971c0039b0 0x7f972c0b9e60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9720034040 tx=0x7f9720001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:16.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.826+0000 7f97297fa640 1 -- 172.21.15.16:0/4219654582 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720004ab0 con 0x7f971c0039b0 2024-09-19T07:50:16.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.826+0000 7f97297fa640 1 -- 172.21.15.16:0/4219654582 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9720004c50 con 0x7f971c0039b0 2024-09-19T07:50:16.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.826+0000 7f9733118640 1 -- 172.21.15.16:0/4219654582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f972c0b90a0 con 0x7f971c0039b0 2024-09-19T07:50:16.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.826+0000 7f97297fa640 1 -- 172.21.15.16:0/4219654582 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720007070 con 0x7f971c0039b0 2024-09-19T07:50:16.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.826+0000 7f9733118640 1 -- 172.21.15.16:0/4219654582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f972c0b9660 con 0x7f971c0039b0 2024-09-19T07:50:16.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.827+0000 7f9733118640 1 -- 172.21.15.16:0/4219654582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f972c06d420 con 0x7f971c0039b0 2024-09-19T07:50:16.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.828+0000 7f97297fa640 1 -- 172.21.15.16:0/4219654582 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9720045020 con 0x7f971c0039b0 2024-09-19T07:50:16.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.828+0000 7f97297fa640 1 --2- 172.21.15.16:0/4219654582 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f971006c2e0 0x7f971006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:16.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.828+0000 7f97297fa640 1 -- 172.21.15.16:0/4219654582 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f97200ab7d0 con 0x7f971c0039b0 2024-09-19T07:50:16.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.828+0000 7f972bfff640 1 --2- 172.21.15.16:0/4219654582 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f971006c2e0 0x7f971006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:16.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.829+0000 7f972bfff640 1 --2- 172.21.15.16:0/4219654582 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f971006c2e0 0x7f971006e7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f972c15d880 tx=0x7f9718008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:16.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:16.833+0000 7f97297fa640 1 -- 172.21.15.16:0/4219654582 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9720079db0 con 0x7f971c0039b0 2024-09-19T07:50:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.045+0000 7f9733118640 1 -- 172.21.15.16:0/4219654582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f972c06cf90 con 0x7f971c0039b0 2024-09-19T07:50:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.704+0000 7f97297fa640 1 -- 172.21.15.16:0/4219654582 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v47) ==== 122+0+0 (secure 0 0 0) 0x7f97200796b0 con 0x7f971c0039b0 2024-09-19T07:50:17.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.707+0000 7f9733118640 1 -- 172.21.15.16:0/4219654582 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f971006c2e0 msgr2=0x7f971006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:17.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.707+0000 7f9733118640 1 --2- 172.21.15.16:0/4219654582 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f971006c2e0 0x7f971006e7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f972c15d880 tx=0x7f9718008040 comp rx=0 tx=0).stop 2024-09-19T07:50:17.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.707+0000 7f9733118640 1 -- 172.21.15.16:0/4219654582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f971c0039b0 msgr2=0x7f972c0b9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:17.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.707+0000 7f9733118640 1 --2- 172.21.15.16:0/4219654582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f971c0039b0 0x7f972c0b9e60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9720034040 tx=0x7f9720001a40 comp rx=0 tx=0).stop 2024-09-19T07:50:17.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.707+0000 7f9733118640 1 -- 172.21.15.16:0/4219654582 shutdown_connections 2024-09-19T07:50:17.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.707+0000 7f9733118640 1 --2- 172.21.15.16:0/4219654582 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f971006c2e0 0x7f971006e7a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:17.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.707+0000 7f9733118640 1 --2- 172.21.15.16:0/4219654582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f972c0c8f60 0x7f972c0c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:17.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.707+0000 7f9733118640 1 --2- 172.21.15.16:0/4219654582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f972c15cc70 0x7f972c15d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:17.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.707+0000 7f9733118640 1 --2- 172.21.15.16:0/4219654582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f971c0039b0 0x7f972c0b9e60 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:17.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.707+0000 7f9733118640 1 -- 172.21.15.16:0/4219654582 >> 172.21.15.16:0/4219654582 conn(0x7f972c09d760 msgr2=0x7f972c15f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:17.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.707+0000 7f9733118640 1 -- 172.21.15.16:0/4219654582 shutdown_connections 2024-09-19T07:50:17.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.708+0000 7f9733118640 1 -- 172.21.15.16:0/4219654582 wait complete. 2024-09-19T07:50:17.721 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T07:50:17.721 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:50:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.830+0000 7fe2d26d9640 1 Processor -- start 2024-09-19T07:50:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.830+0000 7fe2d26d9640 1 -- start start 2024-09-19T07:50:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.831+0000 7fe2d26d9640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe2cc0d0070 0x7fe2cc15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.831+0000 7fe2d26d9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2cc0d1b10 con 0x7fe2cc0d4130 2024-09-19T07:50:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.831+0000 7fe2d26d9640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2cc0cf2b0 con 0x7fe2cc15f0d0 2024-09-19T07:50:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.831+0000 7fe2d26d9640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe2cc0cf590 con 0x7fe2cc0d0070 2024-09-19T07:50:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.831+0000 7fe2d0c76640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe2cc0d4130 0x7fe2cc0d4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50620/0 (socket says 172.21.15.16:50620) 2024-09-19T07:50:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.831+0000 7fe2d0c76640 1 -- 172.21.15.16:0/43959505 learned_addr learned my addr 172.21.15.16:0/43959505 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.831+0000 7fe2c3fff640 1 --2- 172.21.15.16:0/43959505 >> v2:172.21.15.146:3300/0 conn(0x7fe2cc0d0070 0x7fe2cc15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.831+0000 7fe2cb7fe640 1 -- 172.21.15.16:0/43959505 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1792812728 0 0) 0x7fe2cc0d1b10 con 0x7fe2cc0d4130 2024-09-19T07:50:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.831+0000 7fe2cb7fe640 1 -- 172.21.15.16:0/43959505 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2b8003520 con 0x7fe2cc0d4130 2024-09-19T07:50:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.831+0000 7fe2cb7fe640 1 -- 172.21.15.16:0/43959505 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1633630707 0 0) 0x7fe2b8003520 con 0x7fe2cc0d4130 2024-09-19T07:50:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.831+0000 7fe2cb7fe640 1 -- 172.21.15.16:0/43959505 >> v2:172.21.15.146:3300/0 conn(0x7fe2cc0d0070 msgr2=0x7fe2cc15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.831+0000 7fe2cb7fe640 1 --2- 172.21.15.16:0/43959505 >> v2:172.21.15.146:3300/0 conn(0x7fe2cc0d0070 0x7fe2cc15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.831+0000 7fe2cb7fe640 1 -- 172.21.15.16:0/43959505 >> v1:172.21.15.146:6789/0 conn(0x7fe2cc15f0d0 legacy=0x7fe2cc0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.831+0000 7fe2cb7fe640 1 -- 172.21.15.16:0/43959505 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2cc0cf870 con 0x7fe2cc0d4130 2024-09-19T07:50:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.831+0000 7fe2cb7fe640 1 -- 172.21.15.16:0/43959505 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fe2c4001a10 con 0x7fe2cc0d4130 2024-09-19T07:50:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.831+0000 7fe2cb7fe640 1 -- 172.21.15.16:0/43959505 >> v1:172.21.15.16:6789/0 conn(0x7fe2cc0d4130 legacy=0x7fe2cc0d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.831+0000 7fe2cb7fe640 1 --2- 172.21.15.16:0/43959505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2b80039b0 0x7fe2b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.831+0000 7fe2cb7fe640 1 --2- 172.21.15.16:0/43959505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b8004800 0x7fe2b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.831+0000 7fe2cb7fe640 1 --2- 172.21.15.16:0/43959505 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b80052e0 0x7fe2b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.832+0000 7fe2cb7fe640 1 -- 172.21.15.16:0/43959505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc0cf590 con 0x7fe2b8004800 2024-09-19T07:50:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.832+0000 7fe2cb7fe640 1 -- 172.21.15.16:0/43959505 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b800c1b0 con 0x7fe2b80039b0 2024-09-19T07:50:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.832+0000 7fe2cb7fe640 1 -- 172.21.15.16:0/43959505 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b800c3b0 con 0x7fe2b80052e0 2024-09-19T07:50:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.832+0000 7fe2c3fff640 1 --2- 172.21.15.16:0/43959505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b8004800 0x7fe2b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.832+0000 7fe2cbfff640 1 --2- 172.21.15.16:0/43959505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2b80039b0 0x7fe2b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.832+0000 7fe2d0c76640 1 --2- 172.21.15.16:0/43959505 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b80052e0 0x7fe2b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.832+0000 7fe2c3fff640 1 -- 172.21.15.16:0/43959505 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b80052e0 msgr2=0x7fe2b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.832+0000 7fe2c3fff640 1 --2- 172.21.15.16:0/43959505 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b80052e0 0x7fe2b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.832+0000 7fe2c3fff640 1 -- 172.21.15.16:0/43959505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2b80039b0 msgr2=0x7fe2b8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.832+0000 7fe2c3fff640 1 --2- 172.21.15.16:0/43959505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2b80039b0 0x7fe2b8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.832+0000 7fe2c3fff640 1 -- 172.21.15.16:0/43959505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b8003520 con 0x7fe2b8004800 2024-09-19T07:50:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.832+0000 7fe2d0c76640 1 -- 172.21.15.16:0/43959505 reap_dead start 2024-09-19T07:50:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.833+0000 7fe2c3fff640 1 --2- 172.21.15.16:0/43959505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b8004800 0x7fe2b8004c80 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009a90 tx=0x7fe2b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4918a558bde7336 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.833+0000 7fe2cb7fe640 1 -- 172.21.15.16:0/43959505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4036070 con 0x7fe2b8004800 2024-09-19T07:50:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.833+0000 7fe2cb7fe640 1 -- 172.21.15.16:0/43959505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2b4031400 con 0x7fe2b8004800 2024-09-19T07:50:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.833+0000 7fe2cb7fe640 1 -- 172.21.15.16:0/43959505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4009690 con 0x7fe2b8004800 2024-09-19T07:50:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.833+0000 7fe2d26d9640 1 -- 172.21.15.16:0/43959505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b8004800 msgr2=0x7fe2b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.833+0000 7fe2d26d9640 1 --2- 172.21.15.16:0/43959505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b8004800 0x7fe2b8004c80 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009a90 tx=0x7fe2b8005830 comp rx=0 tx=0).stop 2024-09-19T07:50:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.834+0000 7fe2d26d9640 1 -- 172.21.15.16:0/43959505 shutdown_connections 2024-09-19T07:50:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.834+0000 7fe2d26d9640 1 --2- 172.21.15.16:0/43959505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b8004800 0x7fe2b8004c80 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.834+0000 7fe2d26d9640 1 -- 172.21.15.16:0/43959505 >> 172.21.15.16:0/43959505 conn(0x7fe2cc09d760 msgr2=0x7fe2cc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.834+0000 7fe2d26d9640 1 -- 172.21.15.16:0/43959505 shutdown_connections 2024-09-19T07:50:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.834+0000 7fe2d26d9640 1 -- 172.21.15.16:0/43959505 wait complete. 2024-09-19T07:50:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.835+0000 7fe2d26d9640 1 Processor -- start 2024-09-19T07:50:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.835+0000 7fe2d26d9640 1 -- start start 2024-09-19T07:50:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.836+0000 7fe2d26d9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b8004800 0x7fe2cc154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.836+0000 7fe2d26d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc0cc830 0x7fe2cc0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.836+0000 7fe2d26d9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2cc0b99e0 0x7fe2cc0b9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.836+0000 7fe2d26d9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc15f4b0 con 0x7fe2cc0cc830 2024-09-19T07:50:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.836+0000 7fe2d26d9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc0cef60 con 0x7fe2cc0b99e0 2024-09-19T07:50:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.836+0000 7fe2d26d9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe2cc15f630 con 0x7fe2b8004800 2024-09-19T07:50:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.836+0000 7fe2c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc0cc830 0x7fe2cc0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.836+0000 7fe2c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc0cc830 0x7fe2cc0ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40044/0 (socket says 172.21.15.16:40044) 2024-09-19T07:50:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.836+0000 7fe2c3fff640 1 -- 172.21.15.16:0/393964160 learned_addr learned my addr 172.21.15.16:0/393964160 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.836+0000 7fe2cbfff640 1 --2- 172.21.15.16:0/393964160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b8004800 0x7fe2cc154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.836+0000 7fe2d0c76640 1 --2- 172.21.15.16:0/393964160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2cc0b99e0 0x7fe2cc0b9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.836+0000 7fe2c3fff640 1 -- 172.21.15.16:0/393964160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b8004800 msgr2=0x7fe2cc154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.836+0000 7fe2c3fff640 1 --2- 172.21.15.16:0/393964160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b8004800 0x7fe2cc154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.836+0000 7fe2c3fff640 1 -- 172.21.15.16:0/393964160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2cc0b99e0 msgr2=0x7fe2cc0b9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.836+0000 7fe2c3fff640 1 --2- 172.21.15.16:0/393964160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2cc0b99e0 0x7fe2cc0b9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.836+0000 7fe2c3fff640 1 -- 172.21.15.16:0/393964160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2cc0cb4b0 con 0x7fe2cc0cc830 2024-09-19T07:50:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.836+0000 7fe2c3fff640 1 --2- 172.21.15.16:0/393964160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc0cc830 0x7fe2cc0ccc90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe2b402ab50 tx=0x7fe2b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.836+0000 7fe2c97fa640 1 -- 172.21.15.16:0/393964160 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b403f020 con 0x7fe2cc0cc830 2024-09-19T07:50:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.836+0000 7fe2c97fa640 1 -- 172.21.15.16:0/393964160 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2b4037040 con 0x7fe2cc0cc830 2024-09-19T07:50:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.837+0000 7fe2d26d9640 1 -- 172.21.15.16:0/393964160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2cc0cb790 con 0x7fe2cc0cc830 2024-09-19T07:50:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.837+0000 7fe2c97fa640 1 -- 172.21.15.16:0/393964160 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4036070 con 0x7fe2cc0cc830 2024-09-19T07:50:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.837+0000 7fe2d26d9640 1 -- 172.21.15.16:0/393964160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2cc0cbd50 con 0x7fe2cc0cc830 2024-09-19T07:50:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.837+0000 7fe2d26d9640 1 -- 172.21.15.16:0/393964160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2cc0d4560 con 0x7fe2cc0cc830 2024-09-19T07:50:17.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.838+0000 7fe2c97fa640 1 -- 172.21.15.16:0/393964160 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe2b4042020 con 0x7fe2cc0cc830 2024-09-19T07:50:17.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.838+0000 7fe2c97fa640 1 --2- 172.21.15.16:0/393964160 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe2ac06c2b0 0x7fe2ac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:17.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.839+0000 7fe2c97fa640 1 -- 172.21.15.16:0/393964160 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fe2b40ab9a0 con 0x7fe2cc0cc830 2024-09-19T07:50:17.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.839+0000 7fe2cbfff640 1 --2- 172.21.15.16:0/393964160 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe2ac06c2b0 0x7fe2ac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:17.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.839+0000 7fe2cbfff640 1 --2- 172.21.15.16:0/393964160 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe2ac06c2b0 0x7fe2ac06e770 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc001760 tx=0x7fe2bc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:17.843+0000 7fe2c97fa640 1 -- 172.21.15.16:0/393964160 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe2b4049020 con 0x7fe2cc0cc830 2024-09-19T07:50:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.059+0000 7fe2d26d9640 1 -- 172.21.15.16:0/393964160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2cc06cf30 con 0x7fe2cc0cc830 2024-09-19T07:50:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.059+0000 7fe2c97fa640 1 -- 172.21.15.16:0/393964160 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5231 (secure 0 0 0) 0x7fe2b4079950 con 0x7fe2cc0cc830 2024-09-19T07:50:18.060 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:18.060 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":47,"btime":"2024-09-19T07:50:17:700363+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5684,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1865971154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1865971154},{"type":"v1","addr":"172.21.15.146:6837","nonce":1865971154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6094,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/2735152611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2735152611},{"type":"v1","addr":"172.21.15.146:6836","nonce":2735152611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6399,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4204683210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4204683210},{"type":"v1","addr":"172.21.15.16:6834","nonce":4204683210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:50:15.050678+0000","modified":"2024-09-19T07:50:17.046800+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6402":{"gid":6402,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/1305244492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1305244492},{"type":"v1","addr":"172.21.15.16:6835","nonce":1305244492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6402,"qdb_cluster":[6402]},"id":6}]} 2024-09-19T07:50:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 47 2024-09-19T07:50:18.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.061+0000 7fe2d26d9640 1 -- 172.21.15.16:0/393964160 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe2ac06c2b0 msgr2=0x7fe2ac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:18.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.061+0000 7fe2d26d9640 1 --2- 172.21.15.16:0/393964160 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe2ac06c2b0 0x7fe2ac06e770 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc001760 tx=0x7fe2bc009040 comp rx=0 tx=0).stop 2024-09-19T07:50:18.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.061+0000 7fe2d26d9640 1 -- 172.21.15.16:0/393964160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc0cc830 msgr2=0x7fe2cc0ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:18.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.061+0000 7fe2d26d9640 1 --2- 172.21.15.16:0/393964160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc0cc830 0x7fe2cc0ccc90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe2b402ab50 tx=0x7fe2b4004730 comp rx=0 tx=0).stop 2024-09-19T07:50:18.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.061+0000 7fe2d26d9640 1 -- 172.21.15.16:0/393964160 shutdown_connections 2024-09-19T07:50:18.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.061+0000 7fe2d26d9640 1 --2- 172.21.15.16:0/393964160 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe2ac06c2b0 0x7fe2ac06e770 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.061+0000 7fe2d26d9640 1 --2- 172.21.15.16:0/393964160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2cc0b99e0 0x7fe2cc0b9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.061+0000 7fe2d26d9640 1 --2- 172.21.15.16:0/393964160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc0cc830 0x7fe2cc0ccc90 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.061+0000 7fe2d26d9640 1 --2- 172.21.15.16:0/393964160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b8004800 0x7fe2cc154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.061+0000 7fe2d26d9640 1 -- 172.21.15.16:0/393964160 >> 172.21.15.16:0/393964160 conn(0x7fe2cc09d760 msgr2=0x7fe2cc06e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:18.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.061+0000 7fe2d26d9640 1 -- 172.21.15.16:0/393964160 shutdown_connections 2024-09-19T07:50:18.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.061+0000 7fe2d26d9640 1 -- 172.21.15.16:0/393964160 wait complete. 2024-09-19T07:50:18.073 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:50:18.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.175+0000 7f880e865640 1 Processor -- start 2024-09-19T07:50:18.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.175+0000 7f880e865640 1 -- start start 2024-09-19T07:50:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.175+0000 7f880e865640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88080d4370 0x7f88080d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.175+0000 7f880e865640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88080d1880 con 0x7f880815e680 2024-09-19T07:50:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.175+0000 7f880e865640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88080d1b60 con 0x7f880815e2a0 2024-09-19T07:50:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.175+0000 7f880e865640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f88080cf2d0 con 0x7f88080d4370 2024-09-19T07:50:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.176+0000 7f88077fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f880815e680 0x7f88080cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50636/0 (socket says 172.21.15.16:50636) 2024-09-19T07:50:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.176+0000 7f88077fe640 1 -- 172.21.15.16:0/4274758127 learned_addr learned my addr 172.21.15.16:0/4274758127 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.176+0000 7f880ce02640 1 --2- 172.21.15.16:0/4274758127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88080d4370 0x7f88080d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.176+0000 7f8806ffd640 1 -- 172.21.15.16:0/4274758127 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1943694291 0 0) 0x7f88080d1880 con 0x7f880815e680 2024-09-19T07:50:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.176+0000 7f8806ffd640 1 -- 172.21.15.16:0/4274758127 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87f0003520 con 0x7f880815e680 2024-09-19T07:50:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.176+0000 7f8806ffd640 1 -- 172.21.15.16:0/4274758127 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1478598923 0 0) 0x7f88080d1b60 con 0x7f880815e2a0 2024-09-19T07:50:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.176+0000 7f8806ffd640 1 -- 172.21.15.16:0/4274758127 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88080d1880 con 0x7f880815e2a0 2024-09-19T07:50:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.176+0000 7f880ce02640 1 -- 172.21.15.16:0/4274758127 >> v1:172.21.15.146:6789/0 conn(0x7f880815e2a0 legacy=0x7f880806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.176+0000 7f880ce02640 1 -- 172.21.15.16:0/4274758127 >> v1:172.21.15.16:6789/0 conn(0x7f880815e680 legacy=0x7f88080cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.176+0000 7f8806ffd640 1 -- 172.21.15.16:0/4274758127 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 229778499 0 0) 0x7f87f0003520 con 0x7f880815e680 2024-09-19T07:50:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.176+0000 7f880ce02640 1 -- 172.21.15.16:0/4274758127 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88080cf5b0 con 0x7f88080d4370 2024-09-19T07:50:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.177+0000 7f880ce02640 1 --2- 172.21.15.16:0/4274758127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88080d4370 0x7f88080d4820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8800009dc0 tx=0x7f880002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0b50b4cf92111d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.177+0000 7f8806ffd640 1 -- 172.21.15.16:0/4274758127 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc001670 con 0x7f88080d4370 2024-09-19T07:50:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.177+0000 7f8806ffd640 1 -- 172.21.15.16:0/4274758127 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8800007540 con 0x7f88080d4370 2024-09-19T07:50:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.177+0000 7f8806ffd640 1 -- 172.21.15.16:0/4274758127 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88000078f0 con 0x7f88080d4370 2024-09-19T07:50:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.178+0000 7f880e865640 1 -- 172.21.15.16:0/4274758127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88080d4370 msgr2=0x7f88080d4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.178+0000 7f880e865640 1 --2- 172.21.15.16:0/4274758127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88080d4370 0x7f88080d4820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8800009dc0 tx=0x7f880002a660 comp rx=0 tx=0).stop 2024-09-19T07:50:18.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.178+0000 7f880e865640 1 -- 172.21.15.16:0/4274758127 shutdown_connections 2024-09-19T07:50:18.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.178+0000 7f880e865640 1 --2- 172.21.15.16:0/4274758127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88080d4370 0x7f88080d4820 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.178+0000 7f880e865640 1 -- 172.21.15.16:0/4274758127 >> 172.21.15.16:0/4274758127 conn(0x7f880809d760 msgr2=0x7f880808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:18.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.178+0000 7f880e865640 1 -- 172.21.15.16:0/4274758127 shutdown_connections 2024-09-19T07:50:18.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.178+0000 7f880e865640 1 -- 172.21.15.16:0/4274758127 wait complete. 2024-09-19T07:50:18.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.179+0000 7f880e865640 1 Processor -- start 2024-09-19T07:50:18.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.179+0000 7f880e865640 1 -- start start 2024-09-19T07:50:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.180+0000 7f880e865640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88080d4370 0x7f880814b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.180+0000 7f880e865640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f880815e2a0 0x7f88081577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.180+0000 7f880e865640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f880815e680 0x7f8808160660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.180+0000 7f880e865640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f880806dcb0 con 0x7f880815e680 2024-09-19T07:50:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.180+0000 7f880e865640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f880806db30 con 0x7f880815e2a0 2024-09-19T07:50:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.180+0000 7f880e865640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f88080d01e0 con 0x7f88080d4370 2024-09-19T07:50:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.180+0000 7f880ce02640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f880815e680 0x7f8808160660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.180+0000 7f880ce02640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f880815e680 0x7f8808160660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40048/0 (socket says 172.21.15.16:40048) 2024-09-19T07:50:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.180+0000 7f880ce02640 1 -- 172.21.15.16:0/1793686750 learned_addr learned my addr 172.21.15.16:0/1793686750 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.180+0000 7f8807fff640 1 --2- 172.21.15.16:0/1793686750 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88080d4370 0x7f880814b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.180+0000 7f88077fe640 1 --2- 172.21.15.16:0/1793686750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f880815e2a0 0x7f88081577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.181+0000 7f880ce02640 1 -- 172.21.15.16:0/1793686750 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88080d4370 msgr2=0x7f880814b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.181+0000 7f880ce02640 1 --2- 172.21.15.16:0/1793686750 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88080d4370 0x7f880814b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.181+0000 7f880ce02640 1 -- 172.21.15.16:0/1793686750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f880815e2a0 msgr2=0x7f88081577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.181+0000 7f880ce02640 1 --2- 172.21.15.16:0/1793686750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f880815e2a0 0x7f88081577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.181+0000 7f880ce02640 1 -- 172.21.15.16:0/1793686750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88081582d0 con 0x7f880815e680 2024-09-19T07:50:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.181+0000 7f880ce02640 1 --2- 172.21.15.16:0/1793686750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f880815e680 0x7f8808160660 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8800009ee0 tx=0x7f8800004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.181+0000 7f8804ff9640 1 -- 172.21.15.16:0/1793686750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800009450 con 0x7f880815e680 2024-09-19T07:50:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.181+0000 7f8804ff9640 1 -- 172.21.15.16:0/1793686750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8800004240 con 0x7f880815e680 2024-09-19T07:50:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.181+0000 7f8804ff9640 1 -- 172.21.15.16:0/1793686750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88000045f0 con 0x7f880815e680 2024-09-19T07:50:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.181+0000 7f880e865640 1 -- 172.21.15.16:0/1793686750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88081585b0 con 0x7f880815e680 2024-09-19T07:50:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.181+0000 7f880e865640 1 -- 172.21.15.16:0/1793686750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8808158b70 con 0x7f880815e680 2024-09-19T07:50:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.182+0000 7f880e865640 1 -- 172.21.15.16:0/1793686750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88080d4820 con 0x7f880815e680 2024-09-19T07:50:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.182+0000 7f8804ff9640 1 -- 172.21.15.16:0/1793686750 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8800039040 con 0x7f880815e680 2024-09-19T07:50:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.183+0000 7f8804ff9640 1 --2- 172.21.15.16:0/1793686750 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f87f406c2b0 0x7f87f406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.183+0000 7f8804ff9640 1 -- 172.21.15.16:0/1793686750 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f88000acb70 con 0x7f880815e680 2024-09-19T07:50:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.183+0000 7f8807fff640 1 --2- 172.21.15.16:0/1793686750 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f87f406c2b0 0x7f87f406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.184+0000 7f8807fff640 1 --2- 172.21.15.16:0/1793686750 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f87f406c2b0 0x7f87f406e770 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f87f80011e0 tx=0x7f87f8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.187+0000 7f8804ff9640 1 -- 172.21.15.16:0/1793686750 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8800040030 con 0x7f880815e680 2024-09-19T07:50:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.363+0000 7f880e865640 1 -- 172.21.15.16:0/1793686750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f880806cf90 con 0x7f880815e680 2024-09-19T07:50:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.364+0000 7f8804ff9640 1 -- 172.21.15.16:0/1793686750 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16969 (secure 0 0 0) 0x7f880007ac30 con 0x7f880815e680 2024-09-19T07:50:18.364 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:18.365 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:50:16.695783+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:50:13.015554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-19T07:50:14.031234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:50:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.366+0000 7f880e865640 1 -- 172.21.15.16:0/1793686750 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f87f406c2b0 msgr2=0x7f87f406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.366+0000 7f880e865640 1 --2- 172.21.15.16:0/1793686750 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f87f406c2b0 0x7f87f406e770 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f87f80011e0 tx=0x7f87f8009040 comp rx=0 tx=0).stop 2024-09-19T07:50:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.366+0000 7f880e865640 1 -- 172.21.15.16:0/1793686750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f880815e680 msgr2=0x7f8808160660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.366+0000 7f880e865640 1 --2- 172.21.15.16:0/1793686750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f880815e680 0x7f8808160660 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8800009ee0 tx=0x7f8800004a40 comp rx=0 tx=0).stop 2024-09-19T07:50:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.366+0000 7f880e865640 1 -- 172.21.15.16:0/1793686750 shutdown_connections 2024-09-19T07:50:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.366+0000 7f880e865640 1 --2- 172.21.15.16:0/1793686750 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f87f406c2b0 0x7f87f406e770 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.366+0000 7f880e865640 1 --2- 172.21.15.16:0/1793686750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f880815e680 0x7f8808160660 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.366+0000 7f880e865640 1 --2- 172.21.15.16:0/1793686750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f880815e2a0 0x7f88081577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.366+0000 7f880e865640 1 --2- 172.21.15.16:0/1793686750 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88080d4370 0x7f880814b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.366+0000 7f880e865640 1 -- 172.21.15.16:0/1793686750 >> 172.21.15.16:0/1793686750 conn(0x7f880809d760 msgr2=0x7f880806d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.366+0000 7f880e865640 1 -- 172.21.15.16:0/1793686750 shutdown_connections 2024-09-19T07:50:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.366+0000 7f880e865640 1 -- 172.21.15.16:0/1793686750 wait complete. 2024-09-19T07:50:18.378 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T07:50:18.378 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:50:18.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.480+0000 7f2facbb7640 1 Processor -- start 2024-09-19T07:50:18.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.480+0000 7f2facbb7640 1 -- start start 2024-09-19T07:50:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.481+0000 7f2facbb7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fa815e2a0 0x7f2fa806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.481+0000 7f2facbb7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2fa806d960 0x7f2fa80d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.481+0000 7f2facbb7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fa80d1880 con 0x7f2fa815e680 2024-09-19T07:50:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.481+0000 7f2facbb7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2fa80d1b60 con 0x7f2fa806d960 2024-09-19T07:50:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.481+0000 7f2facbb7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa80cf2d0 con 0x7f2fa815e2a0 2024-09-19T07:50:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.481+0000 7f2f9dd9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2fa815e680 0x7f2fa80cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50638/0 (socket says 172.21.15.16:50638) 2024-09-19T07:50:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.481+0000 7f2f9dd9b640 1 -- 172.21.15.16:0/2837683075 learned_addr learned my addr 172.21.15.16:0/2837683075 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.481+0000 7f2fa659c640 1 --2- 172.21.15.16:0/2837683075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fa815e2a0 0x7f2fa806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.481+0000 7f2fa6d9d640 1 --2- 172.21.15.16:0/2837683075 >> v2:172.21.15.146:3300/0 conn(0x7f2fa806d960 0x7f2fa80d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.481+0000 7f2fa5d9b640 1 -- 172.21.15.16:0/2837683075 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3607913723 0 0) 0x7f2fa80d1880 con 0x7f2fa815e680 2024-09-19T07:50:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.481+0000 7f2fa5d9b640 1 -- 172.21.15.16:0/2837683075 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f94003520 con 0x7f2fa815e680 2024-09-19T07:50:18.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.481+0000 7f2fa659c640 1 -- 172.21.15.16:0/2837683075 >> v2:172.21.15.146:3300/0 conn(0x7f2fa806d960 msgr2=0x7f2fa80d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:18.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.481+0000 7f2fa659c640 1 --2- 172.21.15.16:0/2837683075 >> v2:172.21.15.146:3300/0 conn(0x7f2fa806d960 0x7f2fa80d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.481+0000 7f2fa659c640 1 -- 172.21.15.16:0/2837683075 >> v1:172.21.15.16:6789/0 conn(0x7f2fa815e680 legacy=0x7f2fa80cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:18.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.481+0000 7f2fa659c640 1 -- 172.21.15.16:0/2837683075 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa80cf5b0 con 0x7f2fa815e2a0 2024-09-19T07:50:18.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.481+0000 7f2fa6d9d640 1 --2- 172.21.15.16:0/2837683075 >> v2:172.21.15.146:3300/0 conn(0x7f2fa806d960 0x7f2fa80d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:18.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.482+0000 7f2fa659c640 1 --2- 172.21.15.16:0/2837683075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fa815e2a0 0x7f2fa806d420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2f980097a0 tx=0x7f2f9802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1edd01b203fc319 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:18.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.482+0000 7f2fa5d9b640 1 -- 172.21.15.16:0/2837683075 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f98037070 con 0x7f2fa815e2a0 2024-09-19T07:50:18.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.482+0000 7f2fa5d9b640 1 -- 172.21.15.16:0/2837683075 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f98032440 con 0x7f2fa815e2a0 2024-09-19T07:50:18.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.482+0000 7f2fa5d9b640 1 -- 172.21.15.16:0/2837683075 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f980327f0 con 0x7f2fa815e2a0 2024-09-19T07:50:18.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.483+0000 7f2facbb7640 1 -- 172.21.15.16:0/2837683075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fa815e2a0 msgr2=0x7f2fa806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:18.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.483+0000 7f2facbb7640 1 --2- 172.21.15.16:0/2837683075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fa815e2a0 0x7f2fa806d420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2f980097a0 tx=0x7f2f9802a180 comp rx=0 tx=0).stop 2024-09-19T07:50:18.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.483+0000 7f2facbb7640 1 -- 172.21.15.16:0/2837683075 shutdown_connections 2024-09-19T07:50:18.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.483+0000 7f2facbb7640 1 --2- 172.21.15.16:0/2837683075 >> v2:172.21.15.146:3300/0 conn(0x7f2fa806d960 0x7f2fa80d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.483+0000 7f2facbb7640 1 --2- 172.21.15.16:0/2837683075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fa815e2a0 0x7f2fa806d420 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.483+0000 7f2facbb7640 1 -- 172.21.15.16:0/2837683075 >> 172.21.15.16:0/2837683075 conn(0x7f2fa809d760 msgr2=0x7f2fa808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:18.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.483+0000 7f2facbb7640 1 -- 172.21.15.16:0/2837683075 shutdown_connections 2024-09-19T07:50:18.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.483+0000 7f2facbb7640 1 -- 172.21.15.16:0/2837683075 wait complete. 2024-09-19T07:50:18.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.484+0000 7f2facbb7640 1 Processor -- start 2024-09-19T07:50:18.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.484+0000 7f2facbb7640 1 -- start start 2024-09-19T07:50:18.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.485+0000 7f2facbb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa806d960 0x7f2fa80b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:18.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.485+0000 7f2facbb7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fa815e2a0 0x7f2fa8159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:18.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.485+0000 7f2facbb7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fa815e680 0x7f2fa80cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:18.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.485+0000 7f2facbb7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa80d0360 con 0x7f2fa806d960 2024-09-19T07:50:18.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.485+0000 7f2facbb7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa80d01e0 con 0x7f2fa815e680 2024-09-19T07:50:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.485+0000 7f2facbb7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa80d48b0 con 0x7f2fa815e2a0 2024-09-19T07:50:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.485+0000 7f2fa659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa806d960 0x7f2fa80b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.485+0000 7f2fa659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa806d960 0x7f2fa80b7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40052/0 (socket says 172.21.15.16:40052) 2024-09-19T07:50:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.485+0000 7f2fa659c640 1 -- 172.21.15.16:0/833117254 learned_addr learned my addr 172.21.15.16:0/833117254 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.485+0000 7f2f9dd9b640 1 --2- 172.21.15.16:0/833117254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fa815e2a0 0x7f2fa8159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.485+0000 7f2fa6d9d640 1 --2- 172.21.15.16:0/833117254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fa815e680 0x7f2fa80cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.485+0000 7f2fa659c640 1 -- 172.21.15.16:0/833117254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fa815e2a0 msgr2=0x7f2fa8159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.485+0000 7f2fa659c640 1 --2- 172.21.15.16:0/833117254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fa815e2a0 0x7f2fa8159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.485+0000 7f2fa659c640 1 -- 172.21.15.16:0/833117254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fa815e680 msgr2=0x7f2fa80cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.485+0000 7f2fa659c640 1 --2- 172.21.15.16:0/833117254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fa815e680 0x7f2fa80cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.485+0000 7f2fa659c640 1 -- 172.21.15.16:0/833117254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa80cc060 con 0x7f2fa806d960 2024-09-19T07:50:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.486+0000 7f2fa659c640 1 --2- 172.21.15.16:0/833117254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa806d960 0x7f2fa80b7210 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2f980097a0 tx=0x7f2f980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:18.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.486+0000 7f2f9f7fe640 1 -- 172.21.15.16:0/833117254 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f98041070 con 0x7f2fa806d960 2024-09-19T07:50:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.486+0000 7f2f9f7fe640 1 -- 172.21.15.16:0/833117254 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f9803c870 con 0x7f2fa806d960 2024-09-19T07:50:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.486+0000 7f2facbb7640 1 -- 172.21.15.16:0/833117254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa80cc300 con 0x7f2fa806d960 2024-09-19T07:50:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.486+0000 7f2f9f7fe640 1 -- 172.21.15.16:0/833117254 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f98037040 con 0x7f2fa806d960 2024-09-19T07:50:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.486+0000 7f2facbb7640 1 -- 172.21.15.16:0/833117254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa80cc8c0 con 0x7f2fa806d960 2024-09-19T07:50:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.487+0000 7f2facbb7640 1 -- 172.21.15.16:0/833117254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fa80d4c70 con 0x7f2fa806d960 2024-09-19T07:50:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.487+0000 7f2f9f7fe640 1 -- 172.21.15.16:0/833117254 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2f98044070 con 0x7f2fa806d960 2024-09-19T07:50:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.488+0000 7f2f9f7fe640 1 --2- 172.21.15.16:0/833117254 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2f8806c2b0 0x7f2f8806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.488+0000 7f2f9f7fe640 1 -- 172.21.15.16:0/833117254 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f2f980abed0 con 0x7f2fa806d960 2024-09-19T07:50:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.488+0000 7f2f9dd9b640 1 --2- 172.21.15.16:0/833117254 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2f8806c2b0 0x7f2f8806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:18.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.489+0000 7f2f9dd9b640 1 --2- 172.21.15.16:0/833117254 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2f8806c2b0 0x7f2f8806e770 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2fa806d930 tx=0x7f2f90009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:18.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.493+0000 7f2f9f7fe640 1 -- 172.21.15.16:0/833117254 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2f9807a540 con 0x7f2fa806d960 2024-09-19T07:50:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.667+0000 7f2facbb7640 1 -- 172.21.15.16:0/833117254 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2fa80c7c90 con 0x7f2f8806c2b0 2024-09-19T07:50:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.673+0000 7f2f9f7fe640 1 -- 172.21.15.16:0/833117254 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+244023 (secure 0 0 0) 0x7f2fa80c7c90 con 0x7f2f8806c2b0 2024-09-19T07:50:18.676 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:18.678 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:50:18.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.675+0000 7f2facbb7640 1 -- 172.21.15.16:0/833117254 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2f8806c2b0 msgr2=0x7f2f8806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:18.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.676+0000 7f2facbb7640 1 --2- 172.21.15.16:0/833117254 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2f8806c2b0 0x7f2f8806e770 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2fa806d930 tx=0x7f2f90009040 comp rx=0 tx=0).stop 2024-09-19T07:50:18.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.676+0000 7f2facbb7640 1 -- 172.21.15.16:0/833117254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa806d960 msgr2=0x7f2fa80b7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:18.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.676+0000 7f2facbb7640 1 --2- 172.21.15.16:0/833117254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa806d960 0x7f2fa80b7210 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2f980097a0 tx=0x7f2f980046c0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.676+0000 7f2facbb7640 1 -- 172.21.15.16:0/833117254 shutdown_connections 2024-09-19T07:50:18.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.676+0000 7f2facbb7640 1 --2- 172.21.15.16:0/833117254 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2f8806c2b0 0x7f2f8806e770 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.676+0000 7f2facbb7640 1 --2- 172.21.15.16:0/833117254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fa815e680 0x7f2fa80cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.676+0000 7f2facbb7640 1 --2- 172.21.15.16:0/833117254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fa815e2a0 0x7f2fa8159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.676+0000 7f2facbb7640 1 --2- 172.21.15.16:0/833117254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa806d960 0x7f2fa80b7210 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.676+0000 7f2facbb7640 1 -- 172.21.15.16:0/833117254 >> 172.21.15.16:0/833117254 conn(0x7f2fa809d760 msgr2=0x7f2fa815fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:18.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.676+0000 7f2facbb7640 1 -- 172.21.15.16:0/833117254 shutdown_connections 2024-09-19T07:50:18.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.676+0000 7f2facbb7640 1 -- 172.21.15.16:0/833117254 wait complete. 2024-09-19T07:50:18.687 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-19T07:50:18.556331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":110,"acting":110,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":771408,"kb_used_data":9868,"kb_used_omap":12,"kb_used_meta":761331,"kb_avail":754203312,"statfs":{"total":773094113280,"available":772304191488,"internally_reserved":0,"allocated":10104832,"data_stored":5263420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":779603504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.875215"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143560+0000","last_change":"2024-09-19T07:50:15.716195+0000","last_active":"2024-09-19T07:50:17.143560+0000","last_peered":"2024-09-19T07:50:17.143560+0000","last_clean":"2024-09-19T07:50:17.143560+0000","last_became_active":"2024-09-19T07:50:15.715936+0000","last_became_peered":"2024-09-19T07:50:15.715936+0000","last_unstale":"2024-09-19T07:50:17.143560+0000","last_undegraded":"2024-09-19T07:50:17.143560+0000","last_fullsized":"2024-09-19T07:50:17.143560+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:50.198202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143176+0000","last_change":"2024-09-19T07:50:15.706380+0000","last_active":"2024-09-19T07:50:17.143176+0000","last_peered":"2024-09-19T07:50:17.143176+0000","last_clean":"2024-09-19T07:50:17.143176+0000","last_became_active":"2024-09-19T07:50:15.706055+0000","last_became_peered":"2024-09-19T07:50:15.706055+0000","last_unstale":"2024-09-19T07:50:17.143176+0000","last_undegraded":"2024-09-19T07:50:17.143176+0000","last_fullsized":"2024-09-19T07:50:17.143176+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:02.607702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702151+0000","last_change":"2024-09-19T07:50:15.714404+0000","last_active":"2024-09-19T07:50:16.702151+0000","last_peered":"2024-09-19T07:50:16.702151+0000","last_clean":"2024-09-19T07:50:16.702151+0000","last_became_active":"2024-09-19T07:50:15.714029+0000","last_became_peered":"2024-09-19T07:50:15.714029+0000","last_unstale":"2024-09-19T07:50:16.702151+0000","last_undegraded":"2024-09-19T07:50:16.702151+0000","last_fullsized":"2024-09-19T07:50:16.702151+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:52.950164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702212+0000","last_change":"2024-09-19T07:50:15.717130+0000","last_active":"2024-09-19T07:50:16.702212+0000","last_peered":"2024-09-19T07:50:16.702212+0000","last_clean":"2024-09-19T07:50:16.702212+0000","last_became_active":"2024-09-19T07:50:15.716820+0000","last_became_peered":"2024-09-19T07:50:15.716820+0000","last_unstale":"2024-09-19T07:50:16.702212+0000","last_undegraded":"2024-09-19T07:50:16.702212+0000","last_fullsized":"2024-09-19T07:50:16.702212+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:13.211000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.721697+0000","last_change":"2024-09-19T07:50:15.717933+0000","last_active":"2024-09-19T07:50:15.721697+0000","last_peered":"2024-09-19T07:50:15.721697+0000","last_clean":"2024-09-19T07:50:15.721697+0000","last_became_active":"2024-09-19T07:50:15.717636+0000","last_became_peered":"2024-09-19T07:50:15.717636+0000","last_unstale":"2024-09-19T07:50:15.721697+0000","last_undegraded":"2024-09-19T07:50:15.721697+0000","last_fullsized":"2024-09-19T07:50:15.721697+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:11.754897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.703751+0000","last_change":"2024-09-19T07:50:15.702296+0000","last_active":"2024-09-19T07:50:15.703751+0000","last_peered":"2024-09-19T07:50:15.703751+0000","last_clean":"2024-09-19T07:50:15.703751+0000","last_became_active":"2024-09-19T07:50:15.701479+0000","last_became_peered":"2024-09-19T07:50:15.701479+0000","last_unstale":"2024-09-19T07:50:15.703751+0000","last_undegraded":"2024-09-19T07:50:15.703751+0000","last_fullsized":"2024-09-19T07:50:15.703751+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:34.439949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702086+0000","last_change":"2024-09-19T07:50:15.700711+0000","last_active":"2024-09-19T07:50:16.702086+0000","last_peered":"2024-09-19T07:50:16.702086+0000","last_clean":"2024-09-19T07:50:16.702086+0000","last_became_active":"2024-09-19T07:50:15.700431+0000","last_became_peered":"2024-09-19T07:50:15.700431+0000","last_unstale":"2024-09-19T07:50:16.702086+0000","last_undegraded":"2024-09-19T07:50:16.702086+0000","last_fullsized":"2024-09-19T07:50:16.702086+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:55.660212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702008+0000","last_change":"2024-09-19T07:50:15.714991+0000","last_active":"2024-09-19T07:50:16.702008+0000","last_peered":"2024-09-19T07:50:16.702008+0000","last_clean":"2024-09-19T07:50:16.702008+0000","last_became_active":"2024-09-19T07:50:15.714620+0000","last_became_peered":"2024-09-19T07:50:15.714620+0000","last_unstale":"2024-09-19T07:50:16.702008+0000","last_undegraded":"2024-09-19T07:50:16.702008+0000","last_fullsized":"2024-09-19T07:50:16.702008+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:59.294901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143509+0000","last_change":"2024-09-19T07:50:15.697733+0000","last_active":"2024-09-19T07:50:17.143509+0000","last_peered":"2024-09-19T07:50:17.143509+0000","last_clean":"2024-09-19T07:50:17.143509+0000","last_became_active":"2024-09-19T07:50:15.697453+0000","last_became_peered":"2024-09-19T07:50:15.697453+0000","last_unstale":"2024-09-19T07:50:17.143509+0000","last_undegraded":"2024-09-19T07:50:17.143509+0000","last_fullsized":"2024-09-19T07:50:17.143509+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:50.763269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702280+0000","last_change":"2024-09-19T07:50:15.706893+0000","last_active":"2024-09-19T07:50:16.702280+0000","last_peered":"2024-09-19T07:50:16.702280+0000","last_clean":"2024-09-19T07:50:16.702280+0000","last_became_active":"2024-09-19T07:50:15.704710+0000","last_became_peered":"2024-09-19T07:50:15.704710+0000","last_unstale":"2024-09-19T07:50:16.702280+0000","last_undegraded":"2024-09-19T07:50:16.702280+0000","last_fullsized":"2024-09-19T07:50:16.702280+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:33:41.955036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143012+0000","last_change":"2024-09-19T07:50:15.707589+0000","last_active":"2024-09-19T07:50:17.143012+0000","last_peered":"2024-09-19T07:50:17.143012+0000","last_clean":"2024-09-19T07:50:17.143012+0000","last_became_active":"2024-09-19T07:50:15.707303+0000","last_became_peered":"2024-09-19T07:50:15.707303+0000","last_unstale":"2024-09-19T07:50:17.143012+0000","last_undegraded":"2024-09-19T07:50:17.143012+0000","last_fullsized":"2024-09-19T07:50:17.143012+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:02.234544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.701158+0000","last_change":"2024-09-19T07:50:15.700322+0000","last_active":"2024-09-19T07:50:15.701158+0000","last_peered":"2024-09-19T07:50:15.701158+0000","last_clean":"2024-09-19T07:50:15.701158+0000","last_became_active":"2024-09-19T07:50:15.700039+0000","last_became_peered":"2024-09-19T07:50:15.700039+0000","last_unstale":"2024-09-19T07:50:15.701158+0000","last_undegraded":"2024-09-19T07:50:15.701158+0000","last_fullsized":"2024-09-19T07:50:15.701158+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:53.587874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702348+0000","last_change":"2024-09-19T07:50:15.707450+0000","last_active":"2024-09-19T07:50:16.702348+0000","last_peered":"2024-09-19T07:50:16.702348+0000","last_clean":"2024-09-19T07:50:16.702348+0000","last_became_active":"2024-09-19T07:50:15.707187+0000","last_became_peered":"2024-09-19T07:50:15.707187+0000","last_unstale":"2024-09-19T07:50:16.702348+0000","last_undegraded":"2024-09-19T07:50:16.702348+0000","last_fullsized":"2024-09-19T07:50:16.702348+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:48.678322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.688360+0000","last_change":"2024-09-19T07:50:13.688360+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.688360+0000","last_undegraded":"2024-09-19T07:50:13.688360+0000","last_fullsized":"2024-09-19T07:50:13.688360+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.684837+0000","last_change":"2024-09-19T07:50:13.684837+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.684837+0000","last_undegraded":"2024-09-19T07:50:13.684837+0000","last_fullsized":"2024-09-19T07:50:13.684837+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685590+0000","last_change":"2024-09-19T07:50:13.685590+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685590+0000","last_undegraded":"2024-09-19T07:50:13.685590+0000","last_fullsized":"2024-09-19T07:50:13.685590+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.688011+0000","last_change":"2024-09-19T07:50:13.688011+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.688011+0000","last_undegraded":"2024-09-19T07:50:13.688011+0000","last_fullsized":"2024-09-19T07:50:13.688011+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702304+0000","last_change":"2024-09-19T07:50:15.704536+0000","last_active":"2024-09-19T07:50:16.702304+0000","last_peered":"2024-09-19T07:50:16.702304+0000","last_clean":"2024-09-19T07:50:16.702304+0000","last_became_active":"2024-09-19T07:50:15.704223+0000","last_became_peered":"2024-09-19T07:50:15.704223+0000","last_unstale":"2024-09-19T07:50:16.702304+0000","last_undegraded":"2024-09-19T07:50:16.702304+0000","last_fullsized":"2024-09-19T07:50:16.702304+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:41.144795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683040+0000","last_change":"2024-09-19T07:50:13.683040+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683040+0000","last_undegraded":"2024-09-19T07:50:13.683040+0000","last_fullsized":"2024-09-19T07:50:13.683040+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702372+0000","last_change":"2024-09-19T07:50:14.701473+0000","last_active":"2024-09-19T07:50:16.702372+0000","last_peered":"2024-09-19T07:50:16.702372+0000","last_clean":"2024-09-19T07:50:16.702372+0000","last_became_active":"2024-09-19T07:50:14.701196+0000","last_became_peered":"2024-09-19T07:50:14.701196+0000","last_unstale":"2024-09-19T07:50:16.702372+0000","last_undegraded":"2024-09-19T07:50:16.702372+0000","last_fullsized":"2024-09-19T07:50:16.702372+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:45.158919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143734+0000","last_change":"2024-09-19T07:50:14.700842+0000","last_active":"2024-09-19T07:50:17.143734+0000","last_peered":"2024-09-19T07:50:17.143734+0000","last_clean":"2024-09-19T07:50:17.143734+0000","last_became_active":"2024-09-19T07:50:14.700547+0000","last_became_peered":"2024-09-19T07:50:14.700547+0000","last_unstale":"2024-09-19T07:50:17.143734+0000","last_undegraded":"2024-09-19T07:50:17.143734+0000","last_fullsized":"2024-09-19T07:50:17.143734+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:24.868698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702186+0000","last_change":"2024-09-19T07:50:15.703140+0000","last_active":"2024-09-19T07:50:16.702186+0000","last_peered":"2024-09-19T07:50:16.702186+0000","last_clean":"2024-09-19T07:50:16.702186+0000","last_became_active":"2024-09-19T07:50:15.702858+0000","last_became_peered":"2024-09-19T07:50:15.702858+0000","last_unstale":"2024-09-19T07:50:16.702186+0000","last_undegraded":"2024-09-19T07:50:16.702186+0000","last_fullsized":"2024-09-19T07:50:16.702186+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:58.431685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685235+0000","last_change":"2024-09-19T07:50:13.685235+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685235+0000","last_undegraded":"2024-09-19T07:50:13.685235+0000","last_fullsized":"2024-09-19T07:50:13.685235+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683088+0000","last_change":"2024-09-19T07:50:13.683088+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683088+0000","last_undegraded":"2024-09-19T07:50:13.683088+0000","last_fullsized":"2024-09-19T07:50:13.683088+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143660+0000","last_change":"2024-09-19T07:50:14.699921+0000","last_active":"2024-09-19T07:50:17.143660+0000","last_peered":"2024-09-19T07:50:17.143660+0000","last_clean":"2024-09-19T07:50:17.143660+0000","last_became_active":"2024-09-19T07:50:14.699623+0000","last_became_peered":"2024-09-19T07:50:14.699623+0000","last_unstale":"2024-09-19T07:50:17.143660+0000","last_undegraded":"2024-09-19T07:50:17.143660+0000","last_fullsized":"2024-09-19T07:50:17.143660+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:30.030459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143607+0000","last_change":"2024-09-19T07:50:15.724736+0000","last_active":"2024-09-19T07:50:17.143607+0000","last_peered":"2024-09-19T07:50:17.143607+0000","last_clean":"2024-09-19T07:50:17.143607+0000","last_became_active":"2024-09-19T07:50:15.724415+0000","last_became_peered":"2024-09-19T07:50:15.724415+0000","last_unstale":"2024-09-19T07:50:17.143607+0000","last_undegraded":"2024-09-19T07:50:17.143607+0000","last_fullsized":"2024-09-19T07:50:17.143607+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:42.319170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685114+0000","last_change":"2024-09-19T07:50:13.685114+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685114+0000","last_undegraded":"2024-09-19T07:50:13.685114+0000","last_fullsized":"2024-09-19T07:50:13.685114+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702571+0000","last_change":"2024-09-19T07:50:15.701962+0000","last_active":"2024-09-19T07:50:16.702571+0000","last_peered":"2024-09-19T07:50:16.702571+0000","last_clean":"2024-09-19T07:50:16.702571+0000","last_became_active":"2024-09-19T07:50:15.701397+0000","last_became_peered":"2024-09-19T07:50:15.701397+0000","last_unstale":"2024-09-19T07:50:16.702571+0000","last_undegraded":"2024-09-19T07:50:16.702571+0000","last_fullsized":"2024-09-19T07:50:16.702571+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:09.672535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.686167+0000","last_change":"2024-09-19T07:50:13.686167+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.686167+0000","last_undegraded":"2024-09-19T07:50:13.686167+0000","last_fullsized":"2024-09-19T07:50:13.686167+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702370+0000","last_change":"2024-09-19T07:50:15.700668+0000","last_active":"2024-09-19T07:50:16.702370+0000","last_peered":"2024-09-19T07:50:16.702370+0000","last_clean":"2024-09-19T07:50:16.702370+0000","last_became_active":"2024-09-19T07:50:15.700376+0000","last_became_peered":"2024-09-19T07:50:15.700376+0000","last_unstale":"2024-09-19T07:50:16.702370+0000","last_undegraded":"2024-09-19T07:50:16.702370+0000","last_fullsized":"2024-09-19T07:50:16.702370+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:40.316646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685384+0000","last_change":"2024-09-19T07:50:13.685384+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685384+0000","last_undegraded":"2024-09-19T07:50:13.685384+0000","last_fullsized":"2024-09-19T07:50:13.685384+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.686136+0000","last_change":"2024-09-19T07:50:13.686136+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.686136+0000","last_undegraded":"2024-09-19T07:50:13.686136+0000","last_fullsized":"2024-09-19T07:50:13.686136+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.721727+0000","last_change":"2024-09-19T07:50:15.719730+0000","last_active":"2024-09-19T07:50:15.721727+0000","last_peered":"2024-09-19T07:50:15.721727+0000","last_clean":"2024-09-19T07:50:15.721727+0000","last_became_active":"2024-09-19T07:50:15.719453+0000","last_became_peered":"2024-09-19T07:50:15.719453+0000","last_unstale":"2024-09-19T07:50:15.721727+0000","last_undegraded":"2024-09-19T07:50:15.721727+0000","last_fullsized":"2024-09-19T07:50:15.721727+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:43.586941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702027+0000","last_change":"2024-09-19T07:50:14.700632+0000","last_active":"2024-09-19T07:50:16.702027+0000","last_peered":"2024-09-19T07:50:16.702027+0000","last_clean":"2024-09-19T07:50:16.702027+0000","last_became_active":"2024-09-19T07:50:14.700366+0000","last_became_peered":"2024-09-19T07:50:14.700366+0000","last_unstale":"2024-09-19T07:50:16.702027+0000","last_undegraded":"2024-09-19T07:50:16.702027+0000","last_fullsized":"2024-09-19T07:50:16.702027+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:40:38.108015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.709182+0000","last_change":"2024-09-19T07:50:15.707603+0000","last_active":"2024-09-19T07:50:15.709182+0000","last_peered":"2024-09-19T07:50:15.709182+0000","last_clean":"2024-09-19T07:50:15.709182+0000","last_became_active":"2024-09-19T07:50:15.707301+0000","last_became_peered":"2024-09-19T07:50:15.707301+0000","last_unstale":"2024-09-19T07:50:15.709182+0000","last_undegraded":"2024-09-19T07:50:15.709182+0000","last_fullsized":"2024-09-19T07:50:15.709182+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:35.633861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"92'8","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702406+0000","last_change":"2024-09-19T07:50:14.698635+0000","last_active":"2024-09-19T07:50:16.702406+0000","last_peered":"2024-09-19T07:50:16.702406+0000","last_clean":"2024-09-19T07:50:16.702406+0000","last_became_active":"2024-09-19T07:50:14.698359+0000","last_became_peered":"2024-09-19T07:50:14.698359+0000","last_unstale":"2024-09-19T07:50:16.702406+0000","last_undegraded":"2024-09-19T07:50:16.702406+0000","last_fullsized":"2024-09-19T07:50:16.702406+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:57.251575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.687900+0000","last_change":"2024-09-19T07:50:13.687900+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.687900+0000","last_undegraded":"2024-09-19T07:50:13.687900+0000","last_fullsized":"2024-09-19T07:50:13.687900+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685677+0000","last_change":"2024-09-19T07:50:13.685677+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685677+0000","last_undegraded":"2024-09-19T07:50:13.685677+0000","last_fullsized":"2024-09-19T07:50:13.685677+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.684538+0000","last_change":"2024-09-19T07:50:13.684538+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.684538+0000","last_undegraded":"2024-09-19T07:50:13.684538+0000","last_fullsized":"2024-09-19T07:50:13.684538+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683215+0000","last_change":"2024-09-19T07:50:13.683215+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683215+0000","last_undegraded":"2024-09-19T07:50:13.683215+0000","last_fullsized":"2024-09-19T07:50:13.683215+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.693828+0000","last_change":"2024-09-19T07:50:14.695669+0000","last_active":"2024-09-19T07:50:15.693828+0000","last_peered":"2024-09-19T07:50:15.693828+0000","last_clean":"2024-09-19T07:50:15.693828+0000","last_became_active":"2024-09-19T07:50:14.695399+0000","last_became_peered":"2024-09-19T07:50:14.695399+0000","last_unstale":"2024-09-19T07:50:15.693828+0000","last_undegraded":"2024-09-19T07:50:15.693828+0000","last_fullsized":"2024-09-19T07:50:15.693828+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:53.625672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683418+0000","last_change":"2024-09-19T07:50:13.683418+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683418+0000","last_undegraded":"2024-09-19T07:50:13.683418+0000","last_fullsized":"2024-09-19T07:50:13.683418+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.693599+0000","last_change":"2024-09-19T07:50:14.705188+0000","last_active":"2024-09-19T07:50:15.693599+0000","last_peered":"2024-09-19T07:50:15.693599+0000","last_clean":"2024-09-19T07:50:15.693599+0000","last_became_active":"2024-09-19T07:50:14.704804+0000","last_became_peered":"2024-09-19T07:50:14.704804+0000","last_unstale":"2024-09-19T07:50:15.693599+0000","last_undegraded":"2024-09-19T07:50:15.693599+0000","last_fullsized":"2024-09-19T07:50:15.693599+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:49.332075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685337+0000","last_change":"2024-09-19T07:50:13.685337+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685337+0000","last_undegraded":"2024-09-19T07:50:13.685337+0000","last_fullsized":"2024-09-19T07:50:13.685337+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683158+0000","last_change":"2024-09-19T07:50:13.683158+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683158+0000","last_undegraded":"2024-09-19T07:50:13.683158+0000","last_fullsized":"2024-09-19T07:50:13.683158+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683021+0000","last_change":"2024-09-19T07:50:13.683021+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683021+0000","last_undegraded":"2024-09-19T07:50:13.683021+0000","last_fullsized":"2024-09-19T07:50:13.683021+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.687478+0000","last_change":"2024-09-19T07:50:13.687478+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.687478+0000","last_undegraded":"2024-09-19T07:50:13.687478+0000","last_fullsized":"2024-09-19T07:50:13.687478+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.721498+0000","last_change":"2024-09-19T07:50:15.713492+0000","last_active":"2024-09-19T07:50:15.721498+0000","last_peered":"2024-09-19T07:50:15.721498+0000","last_clean":"2024-09-19T07:50:15.721498+0000","last_became_active":"2024-09-19T07:50:15.713228+0000","last_became_peered":"2024-09-19T07:50:15.713228+0000","last_unstale":"2024-09-19T07:50:15.721498+0000","last_undegraded":"2024-09-19T07:50:15.721498+0000","last_fullsized":"2024-09-19T07:50:15.721498+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:21.796433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.686181+0000","last_change":"2024-09-19T07:50:13.686181+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.686181+0000","last_undegraded":"2024-09-19T07:50:13.686181+0000","last_fullsized":"2024-09-19T07:50:13.686181+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143615+0000","last_change":"2024-09-19T07:50:15.708869+0000","last_active":"2024-09-19T07:50:17.143615+0000","last_peered":"2024-09-19T07:50:17.143615+0000","last_clean":"2024-09-19T07:50:17.143615+0000","last_became_active":"2024-09-19T07:50:15.708595+0000","last_became_peered":"2024-09-19T07:50:15.708595+0000","last_unstale":"2024-09-19T07:50:17.143615+0000","last_undegraded":"2024-09-19T07:50:17.143615+0000","last_fullsized":"2024-09-19T07:50:17.143615+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:55.546102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683021+0000","last_change":"2024-09-19T07:50:13.683021+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683021+0000","last_undegraded":"2024-09-19T07:50:13.683021+0000","last_fullsized":"2024-09-19T07:50:13.683021+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.687559+0000","last_change":"2024-09-19T07:50:13.687559+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.687559+0000","last_undegraded":"2024-09-19T07:50:13.687559+0000","last_fullsized":"2024-09-19T07:50:13.687559+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.686119+0000","last_change":"2024-09-19T07:50:13.686119+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.686119+0000","last_undegraded":"2024-09-19T07:50:13.686119+0000","last_fullsized":"2024-09-19T07:50:13.686119+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702502+0000","last_change":"2024-09-19T07:50:14.705646+0000","last_active":"2024-09-19T07:50:16.702502+0000","last_peered":"2024-09-19T07:50:16.702502+0000","last_clean":"2024-09-19T07:50:16.702502+0000","last_became_active":"2024-09-19T07:50:14.705368+0000","last_became_peered":"2024-09-19T07:50:14.705368+0000","last_unstale":"2024-09-19T07:50:16.702502+0000","last_undegraded":"2024-09-19T07:50:16.702502+0000","last_fullsized":"2024-09-19T07:50:16.702502+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:29.229270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":294,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-19T07:50:13.686575+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:50:13.686575+0000","last_peered":"2024-09-19T07:50:13.686575+0000","last_clean":"2024-09-19T07:50:13.686575+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:50:13.686575+0000","last_undegraded":"2024-09-19T07:50:13.686575+0000","last_fullsized":"2024-09-19T07:50:13.686575+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":44,"acting":44,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509531,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":103632,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":101950,"kb_avail":94268208,"statfs":{"total":96636764160,"available":96530644992,"internally_reserved":0,"allocated":1695744,"data_stored":1093834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":104397258},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509530,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100308,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":94271532,"statfs":{"total":96636764160,"available":96534048768,"internally_reserved":0,"allocated":1101824,"data_stored":503466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":101579206},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50312,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321528,"statfs":{"total":96636764160,"available":96585244672,"internally_reserved":0,"allocated":1101824,"data_stored":503466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86348,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285492,"statfs":{"total":96636764160,"available":96548343808,"internally_reserved":0,"allocated":1101824,"data_stored":503466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509530,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100812,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271028,"statfs":{"total":96636764160,"available":96533532672,"internally_reserved":0,"allocated":1695744,"data_stored":1093834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542235,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130800,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94241040,"statfs":{"total":96636764160,"available":96502824960,"internally_reserved":0,"allocated":1138688,"data_stored":523992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542235,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94776,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94277064,"statfs":{"total":96636764160,"available":96539713536,"internally_reserved":0,"allocated":1142784,"data_stored":524062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104420,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267420,"statfs":{"total":96636764160,"available":96529838080,"internally_reserved":0,"allocated":1126400,"data_stored":517300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:50:18.691 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:50:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.828+0000 7f922d0ed640 1 Processor -- start 2024-09-19T07:50:18.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.829+0000 7f922d0ed640 1 -- start start 2024-09-19T07:50:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.829+0000 7f922d0ed640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f922815f0d0 0x7f922815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.829+0000 7f922d0ed640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92280d0070 0x7f92280d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:18.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.829+0000 7f922d0ed640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92280cf2b0 con 0x7f92280d1940 2024-09-19T07:50:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.829+0000 7f922d0ed640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f92280cf590 con 0x7f922815f0d0 2024-09-19T07:50:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.829+0000 7f922d0ed640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f92280cf870 con 0x7f92280d0070 2024-09-19T07:50:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.830+0000 7f922659c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92280d0070 0x7f92280d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.830+0000 7f9226d9d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f922815f0d0 0x7f922815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.830+0000 7f922659c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92280d0070 0x7f92280d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:54054/0 (socket says 172.21.15.16:54054) 2024-09-19T07:50:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.830+0000 7f922659c640 1 -- 172.21.15.16:0/1289019533 learned_addr learned my addr 172.21.15.16:0/1289019533 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.830+0000 7f922659c640 1 -- 172.21.15.16:0/1289019533 >> v2:172.21.15.146:3300/0 conn(0x7f922815f0d0 msgr2=0x7f922815f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.830+0000 7f922659c640 1 --2- 172.21.15.16:0/1289019533 >> v2:172.21.15.146:3300/0 conn(0x7f922815f0d0 0x7f922815f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.830+0000 7f922659c640 1 -- 172.21.15.16:0/1289019533 >> v1:172.21.15.146:6789/0 conn(0x7f92280d1940 legacy=0x7f92280d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.830+0000 7f922659c640 1 -- 172.21.15.16:0/1289019533 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92280cfb50 con 0x7f92280d0070 2024-09-19T07:50:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.830+0000 7f9226d9d640 1 --2- 172.21.15.16:0/1289019533 >> v2:172.21.15.146:3300/0 conn(0x7f922815f0d0 0x7f922815f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:18.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.831+0000 7f922659c640 1 --2- 172.21.15.16:0/1289019533 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92280d0070 0x7f92280d1400 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f921c009870 tx=0x7f921c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8212564f7b79f272 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:18.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.831+0000 7f9225d9b640 1 -- 172.21.15.16:0/1289019533 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c037070 con 0x7f92280d0070 2024-09-19T07:50:18.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.831+0000 7f9225d9b640 1 -- 172.21.15.16:0/1289019533 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f921c032440 con 0x7f92280d0070 2024-09-19T07:50:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.831+0000 7f9225d9b640 1 -- 172.21.15.16:0/1289019533 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c032810 con 0x7f92280d0070 2024-09-19T07:50:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.832+0000 7f922d0ed640 1 -- 172.21.15.16:0/1289019533 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92280d0070 msgr2=0x7f92280d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.832+0000 7f922d0ed640 1 --2- 172.21.15.16:0/1289019533 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92280d0070 0x7f92280d1400 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f921c009870 tx=0x7f921c02a1c0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.832+0000 7f922d0ed640 1 -- 172.21.15.16:0/1289019533 shutdown_connections 2024-09-19T07:50:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.832+0000 7f922d0ed640 1 --2- 172.21.15.16:0/1289019533 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92280d0070 0x7f92280d1400 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.832+0000 7f922d0ed640 1 --2- 172.21.15.16:0/1289019533 >> v2:172.21.15.146:3300/0 conn(0x7f922815f0d0 0x7f922815f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.832+0000 7f922d0ed640 1 -- 172.21.15.16:0/1289019533 >> 172.21.15.16:0/1289019533 conn(0x7f922809d760 msgr2=0x7f922808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.832+0000 7f922d0ed640 1 -- 172.21.15.16:0/1289019533 shutdown_connections 2024-09-19T07:50:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.832+0000 7f922d0ed640 1 -- 172.21.15.16:0/1289019533 wait complete. 2024-09-19T07:50:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.833+0000 7f922d0ed640 1 Processor -- start 2024-09-19T07:50:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.833+0000 7f922d0ed640 1 -- start start 2024-09-19T07:50:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.834+0000 7f922d0ed640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92280d0070 0x7f922815b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.834+0000 7f922d0ed640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92280d1940 0x7f92280c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.834+0000 7f922d0ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f922815f0d0 0x7f922815a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.834+0000 7f922d0ed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f92280d49c0 con 0x7f922815f0d0 2024-09-19T07:50:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.834+0000 7f922d0ed640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f92280d4840 con 0x7f92280d1940 2024-09-19T07:50:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.834+0000 7f922d0ed640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f92280d4b40 con 0x7f92280d0070 2024-09-19T07:50:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.834+0000 7f922759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f922815f0d0 0x7f922815a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.834+0000 7f922759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f922815f0d0 0x7f922815a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40064/0 (socket says 172.21.15.16:40064) 2024-09-19T07:50:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.834+0000 7f922759e640 1 -- 172.21.15.16:0/31190768 learned_addr learned my addr 172.21.15.16:0/31190768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.834+0000 7f9226d9d640 1 --2- 172.21.15.16:0/31190768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92280d0070 0x7f922815b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.834+0000 7f922659c640 1 --2- 172.21.15.16:0/31190768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92280d1940 0x7f92280c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.835+0000 7f922759e640 1 -- 172.21.15.16:0/31190768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92280d0070 msgr2=0x7f922815b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.835+0000 7f922759e640 1 --2- 172.21.15.16:0/31190768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92280d0070 0x7f922815b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.835+0000 7f922759e640 1 -- 172.21.15.16:0/31190768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92280d1940 msgr2=0x7f92280c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.835+0000 7f922759e640 1 --2- 172.21.15.16:0/31190768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92280d1940 0x7f92280c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.835+0000 7f922759e640 1 -- 172.21.15.16:0/31190768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92280b7210 con 0x7f922815f0d0 2024-09-19T07:50:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.835+0000 7f922759e640 1 --2- 172.21.15.16:0/31190768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f922815f0d0 0x7f922815a2b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f922000c3c0 tx=0x7f922000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.835+0000 7f920f7fe640 1 -- 172.21.15.16:0/31190768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9220017070 con 0x7f922815f0d0 2024-09-19T07:50:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.835+0000 7f920f7fe640 1 -- 172.21.15.16:0/31190768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92200052d0 con 0x7f922815f0d0 2024-09-19T07:50:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.835+0000 7f922d0ed640 1 -- 172.21.15.16:0/31190768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92280b74f0 con 0x7f922815f0d0 2024-09-19T07:50:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.835+0000 7f920f7fe640 1 -- 172.21.15.16:0/31190768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9220003850 con 0x7f922815f0d0 2024-09-19T07:50:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.835+0000 7f922d0ed640 1 -- 172.21.15.16:0/31190768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92280b7d90 con 0x7f922815f0d0 2024-09-19T07:50:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.836+0000 7f922d0ed640 1 -- 172.21.15.16:0/31190768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f922806cf30 con 0x7f922815f0d0 2024-09-19T07:50:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.837+0000 7f920f7fe640 1 -- 172.21.15.16:0/31190768 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f922001a070 con 0x7f922815f0d0 2024-09-19T07:50:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.837+0000 7f920f7fe640 1 --2- 172.21.15.16:0/31190768 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f921406c2b0 0x7f921406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.837+0000 7f920f7fe640 1 -- 172.21.15.16:0/31190768 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f922008a820 con 0x7f922815f0d0 2024-09-19T07:50:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.837+0000 7f9226d9d640 1 --2- 172.21.15.16:0/31190768 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f921406c2b0 0x7f921406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.838+0000 7f9226d9d640 1 --2- 172.21.15.16:0/31190768 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f921406c2b0 0x7f921406e770 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9218009bd0 tx=0x7f921800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:18.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:18.841+0000 7f920f7fe640 1 -- 172.21.15.16:0/31190768 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f92200587b0 con 0x7f922815f0d0 2024-09-19T07:50:19.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.015+0000 7f922d0ed640 1 -- 172.21.15.16:0/31190768 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f922806ebc0 con 0x7f921406c2b0 2024-09-19T07:50:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.021+0000 7f920f7fe640 1 -- 172.21.15.16:0/31190768 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+244023 (secure 0 0 0) 0x7f922806ebc0 con 0x7f921406c2b0 2024-09-19T07:50:19.024 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:50:19.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.025+0000 7f922d0ed640 1 -- 172.21.15.16:0/31190768 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f921406c2b0 msgr2=0x7f921406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:19.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.025+0000 7f922d0ed640 1 --2- 172.21.15.16:0/31190768 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f921406c2b0 0x7f921406e770 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9218009bd0 tx=0x7f921800b040 comp rx=0 tx=0).stop 2024-09-19T07:50:19.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.025+0000 7f922d0ed640 1 -- 172.21.15.16:0/31190768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f922815f0d0 msgr2=0x7f922815a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:19.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.025+0000 7f922d0ed640 1 --2- 172.21.15.16:0/31190768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f922815f0d0 0x7f922815a2b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f922000c3c0 tx=0x7f922000c880 comp rx=0 tx=0).stop 2024-09-19T07:50:19.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.025+0000 7f922d0ed640 1 -- 172.21.15.16:0/31190768 shutdown_connections 2024-09-19T07:50:19.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.025+0000 7f922d0ed640 1 --2- 172.21.15.16:0/31190768 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f921406c2b0 0x7f921406e770 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.025+0000 7f922d0ed640 1 --2- 172.21.15.16:0/31190768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f922815f0d0 0x7f922815a2b0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.025+0000 7f922d0ed640 1 --2- 172.21.15.16:0/31190768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92280d1940 0x7f92280c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.025+0000 7f922d0ed640 1 --2- 172.21.15.16:0/31190768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92280d0070 0x7f922815b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.025+0000 7f922d0ed640 1 -- 172.21.15.16:0/31190768 >> 172.21.15.16:0/31190768 conn(0x7f922809d760 msgr2=0x7f922806e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:19.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.026+0000 7f922d0ed640 1 -- 172.21.15.16:0/31190768 shutdown_connections 2024-09-19T07:50:19.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.026+0000 7f922d0ed640 1 -- 172.21.15.16:0/31190768 wait complete. 2024-09-19T07:50:19.041 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-19T07:50:18.556331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":110,"acting":110,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":771408,"kb_used_data":9868,"kb_used_omap":12,"kb_used_meta":761331,"kb_avail":754203312,"statfs":{"total":773094113280,"available":772304191488,"internally_reserved":0,"allocated":10104832,"data_stored":5263420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":779603504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.875215"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143560+0000","last_change":"2024-09-19T07:50:15.716195+0000","last_active":"2024-09-19T07:50:17.143560+0000","last_peered":"2024-09-19T07:50:17.143560+0000","last_clean":"2024-09-19T07:50:17.143560+0000","last_became_active":"2024-09-19T07:50:15.715936+0000","last_became_peered":"2024-09-19T07:50:15.715936+0000","last_unstale":"2024-09-19T07:50:17.143560+0000","last_undegraded":"2024-09-19T07:50:17.143560+0000","last_fullsized":"2024-09-19T07:50:17.143560+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:50.198202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143176+0000","last_change":"2024-09-19T07:50:15.706380+0000","last_active":"2024-09-19T07:50:17.143176+0000","last_peered":"2024-09-19T07:50:17.143176+0000","last_clean":"2024-09-19T07:50:17.143176+0000","last_became_active":"2024-09-19T07:50:15.706055+0000","last_became_peered":"2024-09-19T07:50:15.706055+0000","last_unstale":"2024-09-19T07:50:17.143176+0000","last_undegraded":"2024-09-19T07:50:17.143176+0000","last_fullsized":"2024-09-19T07:50:17.143176+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:02.607702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702151+0000","last_change":"2024-09-19T07:50:15.714404+0000","last_active":"2024-09-19T07:50:16.702151+0000","last_peered":"2024-09-19T07:50:16.702151+0000","last_clean":"2024-09-19T07:50:16.702151+0000","last_became_active":"2024-09-19T07:50:15.714029+0000","last_became_peered":"2024-09-19T07:50:15.714029+0000","last_unstale":"2024-09-19T07:50:16.702151+0000","last_undegraded":"2024-09-19T07:50:16.702151+0000","last_fullsized":"2024-09-19T07:50:16.702151+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:52.950164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702212+0000","last_change":"2024-09-19T07:50:15.717130+0000","last_active":"2024-09-19T07:50:16.702212+0000","last_peered":"2024-09-19T07:50:16.702212+0000","last_clean":"2024-09-19T07:50:16.702212+0000","last_became_active":"2024-09-19T07:50:15.716820+0000","last_became_peered":"2024-09-19T07:50:15.716820+0000","last_unstale":"2024-09-19T07:50:16.702212+0000","last_undegraded":"2024-09-19T07:50:16.702212+0000","last_fullsized":"2024-09-19T07:50:16.702212+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:13.211000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.721697+0000","last_change":"2024-09-19T07:50:15.717933+0000","last_active":"2024-09-19T07:50:15.721697+0000","last_peered":"2024-09-19T07:50:15.721697+0000","last_clean":"2024-09-19T07:50:15.721697+0000","last_became_active":"2024-09-19T07:50:15.717636+0000","last_became_peered":"2024-09-19T07:50:15.717636+0000","last_unstale":"2024-09-19T07:50:15.721697+0000","last_undegraded":"2024-09-19T07:50:15.721697+0000","last_fullsized":"2024-09-19T07:50:15.721697+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:11.754897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.703751+0000","last_change":"2024-09-19T07:50:15.702296+0000","last_active":"2024-09-19T07:50:15.703751+0000","last_peered":"2024-09-19T07:50:15.703751+0000","last_clean":"2024-09-19T07:50:15.703751+0000","last_became_active":"2024-09-19T07:50:15.701479+0000","last_became_peered":"2024-09-19T07:50:15.701479+0000","last_unstale":"2024-09-19T07:50:15.703751+0000","last_undegraded":"2024-09-19T07:50:15.703751+0000","last_fullsized":"2024-09-19T07:50:15.703751+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:34.439949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702086+0000","last_change":"2024-09-19T07:50:15.700711+0000","last_active":"2024-09-19T07:50:16.702086+0000","last_peered":"2024-09-19T07:50:16.702086+0000","last_clean":"2024-09-19T07:50:16.702086+0000","last_became_active":"2024-09-19T07:50:15.700431+0000","last_became_peered":"2024-09-19T07:50:15.700431+0000","last_unstale":"2024-09-19T07:50:16.702086+0000","last_undegraded":"2024-09-19T07:50:16.702086+0000","last_fullsized":"2024-09-19T07:50:16.702086+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:55.660212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702008+0000","last_change":"2024-09-19T07:50:15.714991+0000","last_active":"2024-09-19T07:50:16.702008+0000","last_peered":"2024-09-19T07:50:16.702008+0000","last_clean":"2024-09-19T07:50:16.702008+0000","last_became_active":"2024-09-19T07:50:15.714620+0000","last_became_peered":"2024-09-19T07:50:15.714620+0000","last_unstale":"2024-09-19T07:50:16.702008+0000","last_undegraded":"2024-09-19T07:50:16.702008+0000","last_fullsized":"2024-09-19T07:50:16.702008+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:59.294901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143509+0000","last_change":"2024-09-19T07:50:15.697733+0000","last_active":"2024-09-19T07:50:17.143509+0000","last_peered":"2024-09-19T07:50:17.143509+0000","last_clean":"2024-09-19T07:50:17.143509+0000","last_became_active":"2024-09-19T07:50:15.697453+0000","last_became_peered":"2024-09-19T07:50:15.697453+0000","last_unstale":"2024-09-19T07:50:17.143509+0000","last_undegraded":"2024-09-19T07:50:17.143509+0000","last_fullsized":"2024-09-19T07:50:17.143509+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:50.763269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702280+0000","last_change":"2024-09-19T07:50:15.706893+0000","last_active":"2024-09-19T07:50:16.702280+0000","last_peered":"2024-09-19T07:50:16.702280+0000","last_clean":"2024-09-19T07:50:16.702280+0000","last_became_active":"2024-09-19T07:50:15.704710+0000","last_became_peered":"2024-09-19T07:50:15.704710+0000","last_unstale":"2024-09-19T07:50:16.702280+0000","last_undegraded":"2024-09-19T07:50:16.702280+0000","last_fullsized":"2024-09-19T07:50:16.702280+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:33:41.955036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143012+0000","last_change":"2024-09-19T07:50:15.707589+0000","last_active":"2024-09-19T07:50:17.143012+0000","last_peered":"2024-09-19T07:50:17.143012+0000","last_clean":"2024-09-19T07:50:17.143012+0000","last_became_active":"2024-09-19T07:50:15.707303+0000","last_became_peered":"2024-09-19T07:50:15.707303+0000","last_unstale":"2024-09-19T07:50:17.143012+0000","last_undegraded":"2024-09-19T07:50:17.143012+0000","last_fullsized":"2024-09-19T07:50:17.143012+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:02.234544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.701158+0000","last_change":"2024-09-19T07:50:15.700322+0000","last_active":"2024-09-19T07:50:15.701158+0000","last_peered":"2024-09-19T07:50:15.701158+0000","last_clean":"2024-09-19T07:50:15.701158+0000","last_became_active":"2024-09-19T07:50:15.700039+0000","last_became_peered":"2024-09-19T07:50:15.700039+0000","last_unstale":"2024-09-19T07:50:15.701158+0000","last_undegraded":"2024-09-19T07:50:15.701158+0000","last_fullsized":"2024-09-19T07:50:15.701158+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:53.587874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702348+0000","last_change":"2024-09-19T07:50:15.707450+0000","last_active":"2024-09-19T07:50:16.702348+0000","last_peered":"2024-09-19T07:50:16.702348+0000","last_clean":"2024-09-19T07:50:16.702348+0000","last_became_active":"2024-09-19T07:50:15.707187+0000","last_became_peered":"2024-09-19T07:50:15.707187+0000","last_unstale":"2024-09-19T07:50:16.702348+0000","last_undegraded":"2024-09-19T07:50:16.702348+0000","last_fullsized":"2024-09-19T07:50:16.702348+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:48.678322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.688360+0000","last_change":"2024-09-19T07:50:13.688360+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.688360+0000","last_undegraded":"2024-09-19T07:50:13.688360+0000","last_fullsized":"2024-09-19T07:50:13.688360+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.684837+0000","last_change":"2024-09-19T07:50:13.684837+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.684837+0000","last_undegraded":"2024-09-19T07:50:13.684837+0000","last_fullsized":"2024-09-19T07:50:13.684837+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685590+0000","last_change":"2024-09-19T07:50:13.685590+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685590+0000","last_undegraded":"2024-09-19T07:50:13.685590+0000","last_fullsized":"2024-09-19T07:50:13.685590+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.688011+0000","last_change":"2024-09-19T07:50:13.688011+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.688011+0000","last_undegraded":"2024-09-19T07:50:13.688011+0000","last_fullsized":"2024-09-19T07:50:13.688011+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702304+0000","last_change":"2024-09-19T07:50:15.704536+0000","last_active":"2024-09-19T07:50:16.702304+0000","last_peered":"2024-09-19T07:50:16.702304+0000","last_clean":"2024-09-19T07:50:16.702304+0000","last_became_active":"2024-09-19T07:50:15.704223+0000","last_became_peered":"2024-09-19T07:50:15.704223+0000","last_unstale":"2024-09-19T07:50:16.702304+0000","last_undegraded":"2024-09-19T07:50:16.702304+0000","last_fullsized":"2024-09-19T07:50:16.702304+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:41.144795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683040+0000","last_change":"2024-09-19T07:50:13.683040+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683040+0000","last_undegraded":"2024-09-19T07:50:13.683040+0000","last_fullsized":"2024-09-19T07:50:13.683040+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702372+0000","last_change":"2024-09-19T07:50:14.701473+0000","last_active":"2024-09-19T07:50:16.702372+0000","last_peered":"2024-09-19T07:50:16.702372+0000","last_clean":"2024-09-19T07:50:16.702372+0000","last_became_active":"2024-09-19T07:50:14.701196+0000","last_became_peered":"2024-09-19T07:50:14.701196+0000","last_unstale":"2024-09-19T07:50:16.702372+0000","last_undegraded":"2024-09-19T07:50:16.702372+0000","last_fullsized":"2024-09-19T07:50:16.702372+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:45.158919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143734+0000","last_change":"2024-09-19T07:50:14.700842+0000","last_active":"2024-09-19T07:50:17.143734+0000","last_peered":"2024-09-19T07:50:17.143734+0000","last_clean":"2024-09-19T07:50:17.143734+0000","last_became_active":"2024-09-19T07:50:14.700547+0000","last_became_peered":"2024-09-19T07:50:14.700547+0000","last_unstale":"2024-09-19T07:50:17.143734+0000","last_undegraded":"2024-09-19T07:50:17.143734+0000","last_fullsized":"2024-09-19T07:50:17.143734+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:24.868698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702186+0000","last_change":"2024-09-19T07:50:15.703140+0000","last_active":"2024-09-19T07:50:16.702186+0000","last_peered":"2024-09-19T07:50:16.702186+0000","last_clean":"2024-09-19T07:50:16.702186+0000","last_became_active":"2024-09-19T07:50:15.702858+0000","last_became_peered":"2024-09-19T07:50:15.702858+0000","last_unstale":"2024-09-19T07:50:16.702186+0000","last_undegraded":"2024-09-19T07:50:16.702186+0000","last_fullsized":"2024-09-19T07:50:16.702186+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:58.431685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685235+0000","last_change":"2024-09-19T07:50:13.685235+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685235+0000","last_undegraded":"2024-09-19T07:50:13.685235+0000","last_fullsized":"2024-09-19T07:50:13.685235+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683088+0000","last_change":"2024-09-19T07:50:13.683088+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683088+0000","last_undegraded":"2024-09-19T07:50:13.683088+0000","last_fullsized":"2024-09-19T07:50:13.683088+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143660+0000","last_change":"2024-09-19T07:50:14.699921+0000","last_active":"2024-09-19T07:50:17.143660+0000","last_peered":"2024-09-19T07:50:17.143660+0000","last_clean":"2024-09-19T07:50:17.143660+0000","last_became_active":"2024-09-19T07:50:14.699623+0000","last_became_peered":"2024-09-19T07:50:14.699623+0000","last_unstale":"2024-09-19T07:50:17.143660+0000","last_undegraded":"2024-09-19T07:50:17.143660+0000","last_fullsized":"2024-09-19T07:50:17.143660+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:30.030459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143607+0000","last_change":"2024-09-19T07:50:15.724736+0000","last_active":"2024-09-19T07:50:17.143607+0000","last_peered":"2024-09-19T07:50:17.143607+0000","last_clean":"2024-09-19T07:50:17.143607+0000","last_became_active":"2024-09-19T07:50:15.724415+0000","last_became_peered":"2024-09-19T07:50:15.724415+0000","last_unstale":"2024-09-19T07:50:17.143607+0000","last_undegraded":"2024-09-19T07:50:17.143607+0000","last_fullsized":"2024-09-19T07:50:17.143607+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:42.319170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685114+0000","last_change":"2024-09-19T07:50:13.685114+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685114+0000","last_undegraded":"2024-09-19T07:50:13.685114+0000","last_fullsized":"2024-09-19T07:50:13.685114+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702571+0000","last_change":"2024-09-19T07:50:15.701962+0000","last_active":"2024-09-19T07:50:16.702571+0000","last_peered":"2024-09-19T07:50:16.702571+0000","last_clean":"2024-09-19T07:50:16.702571+0000","last_became_active":"2024-09-19T07:50:15.701397+0000","last_became_peered":"2024-09-19T07:50:15.701397+0000","last_unstale":"2024-09-19T07:50:16.702571+0000","last_undegraded":"2024-09-19T07:50:16.702571+0000","last_fullsized":"2024-09-19T07:50:16.702571+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:09.672535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.686167+0000","last_change":"2024-09-19T07:50:13.686167+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.686167+0000","last_undegraded":"2024-09-19T07:50:13.686167+0000","last_fullsized":"2024-09-19T07:50:13.686167+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702370+0000","last_change":"2024-09-19T07:50:15.700668+0000","last_active":"2024-09-19T07:50:16.702370+0000","last_peered":"2024-09-19T07:50:16.702370+0000","last_clean":"2024-09-19T07:50:16.702370+0000","last_became_active":"2024-09-19T07:50:15.700376+0000","last_became_peered":"2024-09-19T07:50:15.700376+0000","last_unstale":"2024-09-19T07:50:16.702370+0000","last_undegraded":"2024-09-19T07:50:16.702370+0000","last_fullsized":"2024-09-19T07:50:16.702370+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:40.316646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685384+0000","last_change":"2024-09-19T07:50:13.685384+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685384+0000","last_undegraded":"2024-09-19T07:50:13.685384+0000","last_fullsized":"2024-09-19T07:50:13.685384+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.686136+0000","last_change":"2024-09-19T07:50:13.686136+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.686136+0000","last_undegraded":"2024-09-19T07:50:13.686136+0000","last_fullsized":"2024-09-19T07:50:13.686136+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.721727+0000","last_change":"2024-09-19T07:50:15.719730+0000","last_active":"2024-09-19T07:50:15.721727+0000","last_peered":"2024-09-19T07:50:15.721727+0000","last_clean":"2024-09-19T07:50:15.721727+0000","last_became_active":"2024-09-19T07:50:15.719453+0000","last_became_peered":"2024-09-19T07:50:15.719453+0000","last_unstale":"2024-09-19T07:50:15.721727+0000","last_undegraded":"2024-09-19T07:50:15.721727+0000","last_fullsized":"2024-09-19T07:50:15.721727+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:43.586941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702027+0000","last_change":"2024-09-19T07:50:14.700632+0000","last_active":"2024-09-19T07:50:16.702027+0000","last_peered":"2024-09-19T07:50:16.702027+0000","last_clean":"2024-09-19T07:50:16.702027+0000","last_became_active":"2024-09-19T07:50:14.700366+0000","last_became_peered":"2024-09-19T07:50:14.700366+0000","last_unstale":"2024-09-19T07:50:16.702027+0000","last_undegraded":"2024-09-19T07:50:16.702027+0000","last_fullsized":"2024-09-19T07:50:16.702027+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:40:38.108015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.709182+0000","last_change":"2024-09-19T07:50:15.707603+0000","last_active":"2024-09-19T07:50:15.709182+0000","last_peered":"2024-09-19T07:50:15.709182+0000","last_clean":"2024-09-19T07:50:15.709182+0000","last_became_active":"2024-09-19T07:50:15.707301+0000","last_became_peered":"2024-09-19T07:50:15.707301+0000","last_unstale":"2024-09-19T07:50:15.709182+0000","last_undegraded":"2024-09-19T07:50:15.709182+0000","last_fullsized":"2024-09-19T07:50:15.709182+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:35.633861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"92'8","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702406+0000","last_change":"2024-09-19T07:50:14.698635+0000","last_active":"2024-09-19T07:50:16.702406+0000","last_peered":"2024-09-19T07:50:16.702406+0000","last_clean":"2024-09-19T07:50:16.702406+0000","last_became_active":"2024-09-19T07:50:14.698359+0000","last_became_peered":"2024-09-19T07:50:14.698359+0000","last_unstale":"2024-09-19T07:50:16.702406+0000","last_undegraded":"2024-09-19T07:50:16.702406+0000","last_fullsized":"2024-09-19T07:50:16.702406+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:57.251575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.687900+0000","last_change":"2024-09-19T07:50:13.687900+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.687900+0000","last_undegraded":"2024-09-19T07:50:13.687900+0000","last_fullsized":"2024-09-19T07:50:13.687900+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685677+0000","last_change":"2024-09-19T07:50:13.685677+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685677+0000","last_undegraded":"2024-09-19T07:50:13.685677+0000","last_fullsized":"2024-09-19T07:50:13.685677+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.684538+0000","last_change":"2024-09-19T07:50:13.684538+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.684538+0000","last_undegraded":"2024-09-19T07:50:13.684538+0000","last_fullsized":"2024-09-19T07:50:13.684538+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683215+0000","last_change":"2024-09-19T07:50:13.683215+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683215+0000","last_undegraded":"2024-09-19T07:50:13.683215+0000","last_fullsized":"2024-09-19T07:50:13.683215+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.693828+0000","last_change":"2024-09-19T07:50:14.695669+0000","last_active":"2024-09-19T07:50:15.693828+0000","last_peered":"2024-09-19T07:50:15.693828+0000","last_clean":"2024-09-19T07:50:15.693828+0000","last_became_active":"2024-09-19T07:50:14.695399+0000","last_became_peered":"2024-09-19T07:50:14.695399+0000","last_unstale":"2024-09-19T07:50:15.693828+0000","last_undegraded":"2024-09-19T07:50:15.693828+0000","last_fullsized":"2024-09-19T07:50:15.693828+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:53.625672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683418+0000","last_change":"2024-09-19T07:50:13.683418+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683418+0000","last_undegraded":"2024-09-19T07:50:13.683418+0000","last_fullsized":"2024-09-19T07:50:13.683418+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.693599+0000","last_change":"2024-09-19T07:50:14.705188+0000","last_active":"2024-09-19T07:50:15.693599+0000","last_peered":"2024-09-19T07:50:15.693599+0000","last_clean":"2024-09-19T07:50:15.693599+0000","last_became_active":"2024-09-19T07:50:14.704804+0000","last_became_peered":"2024-09-19T07:50:14.704804+0000","last_unstale":"2024-09-19T07:50:15.693599+0000","last_undegraded":"2024-09-19T07:50:15.693599+0000","last_fullsized":"2024-09-19T07:50:15.693599+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:49.332075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685337+0000","last_change":"2024-09-19T07:50:13.685337+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685337+0000","last_undegraded":"2024-09-19T07:50:13.685337+0000","last_fullsized":"2024-09-19T07:50:13.685337+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683158+0000","last_change":"2024-09-19T07:50:13.683158+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683158+0000","last_undegraded":"2024-09-19T07:50:13.683158+0000","last_fullsized":"2024-09-19T07:50:13.683158+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683021+0000","last_change":"2024-09-19T07:50:13.683021+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683021+0000","last_undegraded":"2024-09-19T07:50:13.683021+0000","last_fullsized":"2024-09-19T07:50:13.683021+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.687478+0000","last_change":"2024-09-19T07:50:13.687478+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.687478+0000","last_undegraded":"2024-09-19T07:50:13.687478+0000","last_fullsized":"2024-09-19T07:50:13.687478+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.721498+0000","last_change":"2024-09-19T07:50:15.713492+0000","last_active":"2024-09-19T07:50:15.721498+0000","last_peered":"2024-09-19T07:50:15.721498+0000","last_clean":"2024-09-19T07:50:15.721498+0000","last_became_active":"2024-09-19T07:50:15.713228+0000","last_became_peered":"2024-09-19T07:50:15.713228+0000","last_unstale":"2024-09-19T07:50:15.721498+0000","last_undegraded":"2024-09-19T07:50:15.721498+0000","last_fullsized":"2024-09-19T07:50:15.721498+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:21.796433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.686181+0000","last_change":"2024-09-19T07:50:13.686181+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.686181+0000","last_undegraded":"2024-09-19T07:50:13.686181+0000","last_fullsized":"2024-09-19T07:50:13.686181+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143615+0000","last_change":"2024-09-19T07:50:15.708869+0000","last_active":"2024-09-19T07:50:17.143615+0000","last_peered":"2024-09-19T07:50:17.143615+0000","last_clean":"2024-09-19T07:50:17.143615+0000","last_became_active":"2024-09-19T07:50:15.708595+0000","last_became_peered":"2024-09-19T07:50:15.708595+0000","last_unstale":"2024-09-19T07:50:17.143615+0000","last_undegraded":"2024-09-19T07:50:17.143615+0000","last_fullsized":"2024-09-19T07:50:17.143615+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:55.546102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683021+0000","last_change":"2024-09-19T07:50:13.683021+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683021+0000","last_undegraded":"2024-09-19T07:50:13.683021+0000","last_fullsized":"2024-09-19T07:50:13.683021+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.687559+0000","last_change":"2024-09-19T07:50:13.687559+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.687559+0000","last_undegraded":"2024-09-19T07:50:13.687559+0000","last_fullsized":"2024-09-19T07:50:13.687559+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.686119+0000","last_change":"2024-09-19T07:50:13.686119+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.686119+0000","last_undegraded":"2024-09-19T07:50:13.686119+0000","last_fullsized":"2024-09-19T07:50:13.686119+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702502+0000","last_change":"2024-09-19T07:50:14.705646+0000","last_active":"2024-09-19T07:50:16.702502+0000","last_peered":"2024-09-19T07:50:16.702502+0000","last_clean":"2024-09-19T07:50:16.702502+0000","last_became_active":"2024-09-19T07:50:14.705368+0000","last_became_peered":"2024-09-19T07:50:14.705368+0000","last_unstale":"2024-09-19T07:50:16.702502+0000","last_undegraded":"2024-09-19T07:50:16.702502+0000","last_fullsized":"2024-09-19T07:50:16.702502+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:29.229270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":294,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-19T07:50:13.686575+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:50:13.686575+0000","last_peered":"2024-09-19T07:50:13.686575+0000","last_clean":"2024-09-19T07:50:13.686575+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:50:13.686575+0000","last_undegraded":"2024-09-19T07:50:13.686575+0000","last_fullsized":"2024-09-19T07:50:13.686575+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":44,"acting":44,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509531,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":103632,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":101950,"kb_avail":94268208,"statfs":{"total":96636764160,"available":96530644992,"internally_reserved":0,"allocated":1695744,"data_stored":1093834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":104397258},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509530,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100308,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":94271532,"statfs":{"total":96636764160,"available":96534048768,"internally_reserved":0,"allocated":1101824,"data_stored":503466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":101579206},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50312,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321528,"statfs":{"total":96636764160,"available":96585244672,"internally_reserved":0,"allocated":1101824,"data_stored":503466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86348,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285492,"statfs":{"total":96636764160,"available":96548343808,"internally_reserved":0,"allocated":1101824,"data_stored":503466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509530,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100812,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271028,"statfs":{"total":96636764160,"available":96533532672,"internally_reserved":0,"allocated":1695744,"data_stored":1093834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542235,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130800,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94241040,"statfs":{"total":96636764160,"available":96502824960,"internally_reserved":0,"allocated":1138688,"data_stored":523992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542235,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94776,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94277064,"statfs":{"total":96636764160,"available":96539713536,"internally_reserved":0,"allocated":1142784,"data_stored":524062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104420,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267420,"statfs":{"total":96636764160,"available":96529838080,"internally_reserved":0,"allocated":1126400,"data_stored":517300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:50:19.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:50:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.182+0000 7f5a3390a640 1 Processor -- start 2024-09-19T07:50:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.183+0000 7f5a3390a640 1 -- start start 2024-09-19T07:50:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.183+0000 7f5a3390a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a2c15e2a0 0x7f5a2c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.183+0000 7f5a3390a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5a2c06d960 0x7f5a2c0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.183+0000 7f5a3390a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a2c0d1880 con 0x7f5a2c15e680 2024-09-19T07:50:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.183+0000 7f5a3390a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5a2c0d1b60 con 0x7f5a2c06d960 2024-09-19T07:50:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.183+0000 7f5a3390a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5a2c0cf2d0 con 0x7f5a2c15e2a0 2024-09-19T07:50:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.184+0000 7f5a30ea5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5a2c15e680 0x7f5a2c0cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50648/0 (socket says 172.21.15.16:50648) 2024-09-19T07:50:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.184+0000 7f5a30ea5640 1 -- 172.21.15.16:0/281920968 learned_addr learned my addr 172.21.15.16:0/281920968 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.184+0000 7f5a31ea7640 1 --2- 172.21.15.16:0/281920968 >> v2:172.21.15.146:3300/0 conn(0x7f5a2c06d960 0x7f5a2c0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.184+0000 7f5a316a6640 1 --2- 172.21.15.16:0/281920968 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a2c15e2a0 0x7f5a2c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.184+0000 7f5a23fff640 1 -- 172.21.15.16:0/281920968 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 772514658 0 0) 0x7f5a2c0d1880 con 0x7f5a2c15e680 2024-09-19T07:50:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.184+0000 7f5a23fff640 1 -- 172.21.15.16:0/281920968 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a18003520 con 0x7f5a2c15e680 2024-09-19T07:50:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.184+0000 7f5a31ea7640 1 -- 172.21.15.16:0/281920968 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a2c15e2a0 msgr2=0x7f5a2c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.184+0000 7f5a31ea7640 1 --2- 172.21.15.16:0/281920968 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a2c15e2a0 0x7f5a2c06d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.184+0000 7f5a31ea7640 1 -- 172.21.15.16:0/281920968 >> v1:172.21.15.16:6789/0 conn(0x7f5a2c15e680 legacy=0x7f5a2c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.184+0000 7f5a23fff640 1 -- 172.21.15.16:0/281920968 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1188530499 0 0) 0x7f5a18003520 con 0x7f5a2c15e680 2024-09-19T07:50:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.184+0000 7f5a31ea7640 1 -- 172.21.15.16:0/281920968 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2c0cf5b0 con 0x7f5a2c06d960 2024-09-19T07:50:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.184+0000 7f5a316a6640 1 --2- 172.21.15.16:0/281920968 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a2c15e2a0 0x7f5a2c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.185+0000 7f5a31ea7640 1 --2- 172.21.15.16:0/281920968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a2c06d960 0x7f5a2c0d4370 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009d90 tx=0x7f5a2402a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dd2051fb756527f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.185+0000 7f5a23fff640 1 -- 172.21.15.16:0/281920968 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c001670 con 0x7f5a2c06d960 2024-09-19T07:50:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.185+0000 7f5a23fff640 1 -- 172.21.15.16:0/281920968 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a24007540 con 0x7f5a2c06d960 2024-09-19T07:50:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.185+0000 7f5a23fff640 1 -- 172.21.15.16:0/281920968 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a240078f0 con 0x7f5a2c06d960 2024-09-19T07:50:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.186+0000 7f5a3390a640 1 -- 172.21.15.16:0/281920968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a2c06d960 msgr2=0x7f5a2c0d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.186+0000 7f5a3390a640 1 --2- 172.21.15.16:0/281920968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a2c06d960 0x7f5a2c0d4370 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009d90 tx=0x7f5a2402a5f0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.186+0000 7f5a3390a640 1 -- 172.21.15.16:0/281920968 shutdown_connections 2024-09-19T07:50:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.186+0000 7f5a3390a640 1 --2- 172.21.15.16:0/281920968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a2c06d960 0x7f5a2c0d4370 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.186+0000 7f5a3390a640 1 --2- 172.21.15.16:0/281920968 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a2c15e2a0 0x7f5a2c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.186+0000 7f5a3390a640 1 -- 172.21.15.16:0/281920968 >> 172.21.15.16:0/281920968 conn(0x7f5a2c09d760 msgr2=0x7f5a2c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.186+0000 7f5a3390a640 1 -- 172.21.15.16:0/281920968 shutdown_connections 2024-09-19T07:50:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.186+0000 7f5a3390a640 1 -- 172.21.15.16:0/281920968 wait complete. 2024-09-19T07:50:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.187+0000 7f5a3390a640 1 Processor -- start 2024-09-19T07:50:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.187+0000 7f5a3390a640 1 -- start start 2024-09-19T07:50:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.188+0000 7f5a3390a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a2c06d960 0x7f5a2c1566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.188+0000 7f5a3390a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2c15e2a0 0x7f5a2c0b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.188+0000 7f5a3390a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a2c15e680 0x7f5a2c160e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.188+0000 7f5a3390a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c0d0360 con 0x7f5a2c15e2a0 2024-09-19T07:50:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.188+0000 7f5a3390a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c0d01e0 con 0x7f5a2c06d960 2024-09-19T07:50:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.188+0000 7f5a3390a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5a2c0d48b0 con 0x7f5a2c15e680 2024-09-19T07:50:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.188+0000 7f5a30ea5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2c15e2a0 0x7f5a2c0b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.188+0000 7f5a30ea5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2c15e2a0 0x7f5a2c0b89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40070/0 (socket says 172.21.15.16:40070) 2024-09-19T07:50:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.188+0000 7f5a30ea5640 1 -- 172.21.15.16:0/747249173 learned_addr learned my addr 172.21.15.16:0/747249173 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.188+0000 7f5a31ea7640 1 --2- 172.21.15.16:0/747249173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a2c15e680 0x7f5a2c160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.188+0000 7f5a316a6640 1 --2- 172.21.15.16:0/747249173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a2c06d960 0x7f5a2c1566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.188+0000 7f5a30ea5640 1 -- 172.21.15.16:0/747249173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a2c15e680 msgr2=0x7f5a2c160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.188+0000 7f5a30ea5640 1 --2- 172.21.15.16:0/747249173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a2c15e680 0x7f5a2c160e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.188+0000 7f5a30ea5640 1 -- 172.21.15.16:0/747249173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a2c06d960 msgr2=0x7f5a2c1566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.188+0000 7f5a30ea5640 1 --2- 172.21.15.16:0/747249173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a2c06d960 0x7f5a2c1566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.188+0000 7f5a30ea5640 1 -- 172.21.15.16:0/747249173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2c0c9100 con 0x7f5a2c15e2a0 2024-09-19T07:50:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.188+0000 7f5a31ea7640 1 --2- 172.21.15.16:0/747249173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a2c15e680 0x7f5a2c160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.188+0000 7f5a316a6640 1 --2- 172.21.15.16:0/747249173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a2c06d960 0x7f5a2c1566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.188+0000 7f5a30ea5640 1 --2- 172.21.15.16:0/747249173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2c15e2a0 0x7f5a2c0b89d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00c700 tx=0x7f5a1c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.189+0000 7f5a21ffb640 1 -- 172.21.15.16:0/747249173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c017070 con 0x7f5a2c15e2a0 2024-09-19T07:50:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.189+0000 7f5a21ffb640 1 -- 172.21.15.16:0/747249173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a1c005ad0 con 0x7f5a2c15e2a0 2024-09-19T07:50:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.189+0000 7f5a3390a640 1 -- 172.21.15.16:0/747249173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a2c0b9470 con 0x7f5a2c15e2a0 2024-09-19T07:50:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.189+0000 7f5a21ffb640 1 -- 172.21.15.16:0/747249173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c007070 con 0x7f5a2c15e2a0 2024-09-19T07:50:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.189+0000 7f5a3390a640 1 -- 172.21.15.16:0/747249173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a2c0b9d10 con 0x7f5a2c15e2a0 2024-09-19T07:50:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.190+0000 7f5a3390a640 1 -- 172.21.15.16:0/747249173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a2c0d4370 con 0x7f5a2c15e2a0 2024-09-19T07:50:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.190+0000 7f5a21ffb640 1 -- 172.21.15.16:0/747249173 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5a1c01a070 con 0x7f5a2c15e2a0 2024-09-19T07:50:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.191+0000 7f5a21ffb640 1 --2- 172.21.15.16:0/747249173 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a1006c2b0 0x7f5a1006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.191+0000 7f5a21ffb640 1 -- 172.21.15.16:0/747249173 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f5a1c08a940 con 0x7f5a2c15e2a0 2024-09-19T07:50:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.191+0000 7f5a316a6640 1 --2- 172.21.15.16:0/747249173 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a1006c2b0 0x7f5a1006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.192+0000 7f5a316a6640 1 --2- 172.21.15.16:0/747249173 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a1006c2b0 0x7f5a1006e770 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c160ba0 tx=0x7f5a28008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.195+0000 7f5a21ffb640 1 -- 172.21.15.16:0/747249173 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5a1c0588d0 con 0x7f5a2c15e2a0 2024-09-19T07:50:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.372+0000 7f5a3390a640 1 -- 172.21.15.16:0/747249173 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a2c06cf90 con 0x7f5a1006c2b0 2024-09-19T07:50:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.378+0000 7f5a21ffb640 1 -- 172.21.15.16:0/747249173 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+244023 (secure 0 0 0) 0x7f5a2c06cf90 con 0x7f5a1006c2b0 2024-09-19T07:50:19.380 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:50:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.381+0000 7f5a3390a640 1 -- 172.21.15.16:0/747249173 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a1006c2b0 msgr2=0x7f5a1006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.381+0000 7f5a3390a640 1 --2- 172.21.15.16:0/747249173 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a1006c2b0 0x7f5a1006e770 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c160ba0 tx=0x7f5a28008040 comp rx=0 tx=0).stop 2024-09-19T07:50:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.381+0000 7f5a3390a640 1 -- 172.21.15.16:0/747249173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2c15e2a0 msgr2=0x7f5a2c0b89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.381+0000 7f5a3390a640 1 --2- 172.21.15.16:0/747249173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2c15e2a0 0x7f5a2c0b89d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00c700 tx=0x7f5a1c00cbc0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.381+0000 7f5a3390a640 1 -- 172.21.15.16:0/747249173 shutdown_connections 2024-09-19T07:50:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.381+0000 7f5a3390a640 1 --2- 172.21.15.16:0/747249173 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5a1006c2b0 0x7f5a1006e770 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.381+0000 7f5a3390a640 1 --2- 172.21.15.16:0/747249173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a2c15e680 0x7f5a2c160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.381+0000 7f5a3390a640 1 --2- 172.21.15.16:0/747249173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2c15e2a0 0x7f5a2c0b89d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.381+0000 7f5a3390a640 1 --2- 172.21.15.16:0/747249173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a2c06d960 0x7f5a2c1566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.381+0000 7f5a3390a640 1 -- 172.21.15.16:0/747249173 >> 172.21.15.16:0/747249173 conn(0x7f5a2c09d760 msgr2=0x7f5a2c15f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.381+0000 7f5a3390a640 1 -- 172.21.15.16:0/747249173 shutdown_connections 2024-09-19T07:50:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.381+0000 7f5a3390a640 1 -- 172.21.15.16:0/747249173 wait complete. 2024-09-19T07:50:19.392 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-19T07:50:18.556331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":110,"acting":110,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":771408,"kb_used_data":9868,"kb_used_omap":12,"kb_used_meta":761331,"kb_avail":754203312,"statfs":{"total":773094113280,"available":772304191488,"internally_reserved":0,"allocated":10104832,"data_stored":5263420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":779603504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.875215"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143560+0000","last_change":"2024-09-19T07:50:15.716195+0000","last_active":"2024-09-19T07:50:17.143560+0000","last_peered":"2024-09-19T07:50:17.143560+0000","last_clean":"2024-09-19T07:50:17.143560+0000","last_became_active":"2024-09-19T07:50:15.715936+0000","last_became_peered":"2024-09-19T07:50:15.715936+0000","last_unstale":"2024-09-19T07:50:17.143560+0000","last_undegraded":"2024-09-19T07:50:17.143560+0000","last_fullsized":"2024-09-19T07:50:17.143560+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:50.198202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143176+0000","last_change":"2024-09-19T07:50:15.706380+0000","last_active":"2024-09-19T07:50:17.143176+0000","last_peered":"2024-09-19T07:50:17.143176+0000","last_clean":"2024-09-19T07:50:17.143176+0000","last_became_active":"2024-09-19T07:50:15.706055+0000","last_became_peered":"2024-09-19T07:50:15.706055+0000","last_unstale":"2024-09-19T07:50:17.143176+0000","last_undegraded":"2024-09-19T07:50:17.143176+0000","last_fullsized":"2024-09-19T07:50:17.143176+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:02.607702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702151+0000","last_change":"2024-09-19T07:50:15.714404+0000","last_active":"2024-09-19T07:50:16.702151+0000","last_peered":"2024-09-19T07:50:16.702151+0000","last_clean":"2024-09-19T07:50:16.702151+0000","last_became_active":"2024-09-19T07:50:15.714029+0000","last_became_peered":"2024-09-19T07:50:15.714029+0000","last_unstale":"2024-09-19T07:50:16.702151+0000","last_undegraded":"2024-09-19T07:50:16.702151+0000","last_fullsized":"2024-09-19T07:50:16.702151+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:52.950164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702212+0000","last_change":"2024-09-19T07:50:15.717130+0000","last_active":"2024-09-19T07:50:16.702212+0000","last_peered":"2024-09-19T07:50:16.702212+0000","last_clean":"2024-09-19T07:50:16.702212+0000","last_became_active":"2024-09-19T07:50:15.716820+0000","last_became_peered":"2024-09-19T07:50:15.716820+0000","last_unstale":"2024-09-19T07:50:16.702212+0000","last_undegraded":"2024-09-19T07:50:16.702212+0000","last_fullsized":"2024-09-19T07:50:16.702212+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:13.211000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.721697+0000","last_change":"2024-09-19T07:50:15.717933+0000","last_active":"2024-09-19T07:50:15.721697+0000","last_peered":"2024-09-19T07:50:15.721697+0000","last_clean":"2024-09-19T07:50:15.721697+0000","last_became_active":"2024-09-19T07:50:15.717636+0000","last_became_peered":"2024-09-19T07:50:15.717636+0000","last_unstale":"2024-09-19T07:50:15.721697+0000","last_undegraded":"2024-09-19T07:50:15.721697+0000","last_fullsized":"2024-09-19T07:50:15.721697+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:11.754897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.703751+0000","last_change":"2024-09-19T07:50:15.702296+0000","last_active":"2024-09-19T07:50:15.703751+0000","last_peered":"2024-09-19T07:50:15.703751+0000","last_clean":"2024-09-19T07:50:15.703751+0000","last_became_active":"2024-09-19T07:50:15.701479+0000","last_became_peered":"2024-09-19T07:50:15.701479+0000","last_unstale":"2024-09-19T07:50:15.703751+0000","last_undegraded":"2024-09-19T07:50:15.703751+0000","last_fullsized":"2024-09-19T07:50:15.703751+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:34.439949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702086+0000","last_change":"2024-09-19T07:50:15.700711+0000","last_active":"2024-09-19T07:50:16.702086+0000","last_peered":"2024-09-19T07:50:16.702086+0000","last_clean":"2024-09-19T07:50:16.702086+0000","last_became_active":"2024-09-19T07:50:15.700431+0000","last_became_peered":"2024-09-19T07:50:15.700431+0000","last_unstale":"2024-09-19T07:50:16.702086+0000","last_undegraded":"2024-09-19T07:50:16.702086+0000","last_fullsized":"2024-09-19T07:50:16.702086+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:55.660212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702008+0000","last_change":"2024-09-19T07:50:15.714991+0000","last_active":"2024-09-19T07:50:16.702008+0000","last_peered":"2024-09-19T07:50:16.702008+0000","last_clean":"2024-09-19T07:50:16.702008+0000","last_became_active":"2024-09-19T07:50:15.714620+0000","last_became_peered":"2024-09-19T07:50:15.714620+0000","last_unstale":"2024-09-19T07:50:16.702008+0000","last_undegraded":"2024-09-19T07:50:16.702008+0000","last_fullsized":"2024-09-19T07:50:16.702008+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:59.294901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143509+0000","last_change":"2024-09-19T07:50:15.697733+0000","last_active":"2024-09-19T07:50:17.143509+0000","last_peered":"2024-09-19T07:50:17.143509+0000","last_clean":"2024-09-19T07:50:17.143509+0000","last_became_active":"2024-09-19T07:50:15.697453+0000","last_became_peered":"2024-09-19T07:50:15.697453+0000","last_unstale":"2024-09-19T07:50:17.143509+0000","last_undegraded":"2024-09-19T07:50:17.143509+0000","last_fullsized":"2024-09-19T07:50:17.143509+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:50.763269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702280+0000","last_change":"2024-09-19T07:50:15.706893+0000","last_active":"2024-09-19T07:50:16.702280+0000","last_peered":"2024-09-19T07:50:16.702280+0000","last_clean":"2024-09-19T07:50:16.702280+0000","last_became_active":"2024-09-19T07:50:15.704710+0000","last_became_peered":"2024-09-19T07:50:15.704710+0000","last_unstale":"2024-09-19T07:50:16.702280+0000","last_undegraded":"2024-09-19T07:50:16.702280+0000","last_fullsized":"2024-09-19T07:50:16.702280+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:33:41.955036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143012+0000","last_change":"2024-09-19T07:50:15.707589+0000","last_active":"2024-09-19T07:50:17.143012+0000","last_peered":"2024-09-19T07:50:17.143012+0000","last_clean":"2024-09-19T07:50:17.143012+0000","last_became_active":"2024-09-19T07:50:15.707303+0000","last_became_peered":"2024-09-19T07:50:15.707303+0000","last_unstale":"2024-09-19T07:50:17.143012+0000","last_undegraded":"2024-09-19T07:50:17.143012+0000","last_fullsized":"2024-09-19T07:50:17.143012+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:02.234544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.701158+0000","last_change":"2024-09-19T07:50:15.700322+0000","last_active":"2024-09-19T07:50:15.701158+0000","last_peered":"2024-09-19T07:50:15.701158+0000","last_clean":"2024-09-19T07:50:15.701158+0000","last_became_active":"2024-09-19T07:50:15.700039+0000","last_became_peered":"2024-09-19T07:50:15.700039+0000","last_unstale":"2024-09-19T07:50:15.701158+0000","last_undegraded":"2024-09-19T07:50:15.701158+0000","last_fullsized":"2024-09-19T07:50:15.701158+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:53.587874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702348+0000","last_change":"2024-09-19T07:50:15.707450+0000","last_active":"2024-09-19T07:50:16.702348+0000","last_peered":"2024-09-19T07:50:16.702348+0000","last_clean":"2024-09-19T07:50:16.702348+0000","last_became_active":"2024-09-19T07:50:15.707187+0000","last_became_peered":"2024-09-19T07:50:15.707187+0000","last_unstale":"2024-09-19T07:50:16.702348+0000","last_undegraded":"2024-09-19T07:50:16.702348+0000","last_fullsized":"2024-09-19T07:50:16.702348+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:48.678322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.688360+0000","last_change":"2024-09-19T07:50:13.688360+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.688360+0000","last_undegraded":"2024-09-19T07:50:13.688360+0000","last_fullsized":"2024-09-19T07:50:13.688360+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.684837+0000","last_change":"2024-09-19T07:50:13.684837+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.684837+0000","last_undegraded":"2024-09-19T07:50:13.684837+0000","last_fullsized":"2024-09-19T07:50:13.684837+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685590+0000","last_change":"2024-09-19T07:50:13.685590+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685590+0000","last_undegraded":"2024-09-19T07:50:13.685590+0000","last_fullsized":"2024-09-19T07:50:13.685590+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.688011+0000","last_change":"2024-09-19T07:50:13.688011+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.688011+0000","last_undegraded":"2024-09-19T07:50:13.688011+0000","last_fullsized":"2024-09-19T07:50:13.688011+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702304+0000","last_change":"2024-09-19T07:50:15.704536+0000","last_active":"2024-09-19T07:50:16.702304+0000","last_peered":"2024-09-19T07:50:16.702304+0000","last_clean":"2024-09-19T07:50:16.702304+0000","last_became_active":"2024-09-19T07:50:15.704223+0000","last_became_peered":"2024-09-19T07:50:15.704223+0000","last_unstale":"2024-09-19T07:50:16.702304+0000","last_undegraded":"2024-09-19T07:50:16.702304+0000","last_fullsized":"2024-09-19T07:50:16.702304+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:41.144795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683040+0000","last_change":"2024-09-19T07:50:13.683040+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683040+0000","last_undegraded":"2024-09-19T07:50:13.683040+0000","last_fullsized":"2024-09-19T07:50:13.683040+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702372+0000","last_change":"2024-09-19T07:50:14.701473+0000","last_active":"2024-09-19T07:50:16.702372+0000","last_peered":"2024-09-19T07:50:16.702372+0000","last_clean":"2024-09-19T07:50:16.702372+0000","last_became_active":"2024-09-19T07:50:14.701196+0000","last_became_peered":"2024-09-19T07:50:14.701196+0000","last_unstale":"2024-09-19T07:50:16.702372+0000","last_undegraded":"2024-09-19T07:50:16.702372+0000","last_fullsized":"2024-09-19T07:50:16.702372+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:45.158919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143734+0000","last_change":"2024-09-19T07:50:14.700842+0000","last_active":"2024-09-19T07:50:17.143734+0000","last_peered":"2024-09-19T07:50:17.143734+0000","last_clean":"2024-09-19T07:50:17.143734+0000","last_became_active":"2024-09-19T07:50:14.700547+0000","last_became_peered":"2024-09-19T07:50:14.700547+0000","last_unstale":"2024-09-19T07:50:17.143734+0000","last_undegraded":"2024-09-19T07:50:17.143734+0000","last_fullsized":"2024-09-19T07:50:17.143734+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:24.868698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702186+0000","last_change":"2024-09-19T07:50:15.703140+0000","last_active":"2024-09-19T07:50:16.702186+0000","last_peered":"2024-09-19T07:50:16.702186+0000","last_clean":"2024-09-19T07:50:16.702186+0000","last_became_active":"2024-09-19T07:50:15.702858+0000","last_became_peered":"2024-09-19T07:50:15.702858+0000","last_unstale":"2024-09-19T07:50:16.702186+0000","last_undegraded":"2024-09-19T07:50:16.702186+0000","last_fullsized":"2024-09-19T07:50:16.702186+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:58.431685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685235+0000","last_change":"2024-09-19T07:50:13.685235+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685235+0000","last_undegraded":"2024-09-19T07:50:13.685235+0000","last_fullsized":"2024-09-19T07:50:13.685235+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683088+0000","last_change":"2024-09-19T07:50:13.683088+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683088+0000","last_undegraded":"2024-09-19T07:50:13.683088+0000","last_fullsized":"2024-09-19T07:50:13.683088+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143660+0000","last_change":"2024-09-19T07:50:14.699921+0000","last_active":"2024-09-19T07:50:17.143660+0000","last_peered":"2024-09-19T07:50:17.143660+0000","last_clean":"2024-09-19T07:50:17.143660+0000","last_became_active":"2024-09-19T07:50:14.699623+0000","last_became_peered":"2024-09-19T07:50:14.699623+0000","last_unstale":"2024-09-19T07:50:17.143660+0000","last_undegraded":"2024-09-19T07:50:17.143660+0000","last_fullsized":"2024-09-19T07:50:17.143660+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:30.030459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143607+0000","last_change":"2024-09-19T07:50:15.724736+0000","last_active":"2024-09-19T07:50:17.143607+0000","last_peered":"2024-09-19T07:50:17.143607+0000","last_clean":"2024-09-19T07:50:17.143607+0000","last_became_active":"2024-09-19T07:50:15.724415+0000","last_became_peered":"2024-09-19T07:50:15.724415+0000","last_unstale":"2024-09-19T07:50:17.143607+0000","last_undegraded":"2024-09-19T07:50:17.143607+0000","last_fullsized":"2024-09-19T07:50:17.143607+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:42.319170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685114+0000","last_change":"2024-09-19T07:50:13.685114+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685114+0000","last_undegraded":"2024-09-19T07:50:13.685114+0000","last_fullsized":"2024-09-19T07:50:13.685114+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702571+0000","last_change":"2024-09-19T07:50:15.701962+0000","last_active":"2024-09-19T07:50:16.702571+0000","last_peered":"2024-09-19T07:50:16.702571+0000","last_clean":"2024-09-19T07:50:16.702571+0000","last_became_active":"2024-09-19T07:50:15.701397+0000","last_became_peered":"2024-09-19T07:50:15.701397+0000","last_unstale":"2024-09-19T07:50:16.702571+0000","last_undegraded":"2024-09-19T07:50:16.702571+0000","last_fullsized":"2024-09-19T07:50:16.702571+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:09.672535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.686167+0000","last_change":"2024-09-19T07:50:13.686167+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.686167+0000","last_undegraded":"2024-09-19T07:50:13.686167+0000","last_fullsized":"2024-09-19T07:50:13.686167+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702370+0000","last_change":"2024-09-19T07:50:15.700668+0000","last_active":"2024-09-19T07:50:16.702370+0000","last_peered":"2024-09-19T07:50:16.702370+0000","last_clean":"2024-09-19T07:50:16.702370+0000","last_became_active":"2024-09-19T07:50:15.700376+0000","last_became_peered":"2024-09-19T07:50:15.700376+0000","last_unstale":"2024-09-19T07:50:16.702370+0000","last_undegraded":"2024-09-19T07:50:16.702370+0000","last_fullsized":"2024-09-19T07:50:16.702370+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:40.316646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685384+0000","last_change":"2024-09-19T07:50:13.685384+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685384+0000","last_undegraded":"2024-09-19T07:50:13.685384+0000","last_fullsized":"2024-09-19T07:50:13.685384+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.686136+0000","last_change":"2024-09-19T07:50:13.686136+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.686136+0000","last_undegraded":"2024-09-19T07:50:13.686136+0000","last_fullsized":"2024-09-19T07:50:13.686136+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.721727+0000","last_change":"2024-09-19T07:50:15.719730+0000","last_active":"2024-09-19T07:50:15.721727+0000","last_peered":"2024-09-19T07:50:15.721727+0000","last_clean":"2024-09-19T07:50:15.721727+0000","last_became_active":"2024-09-19T07:50:15.719453+0000","last_became_peered":"2024-09-19T07:50:15.719453+0000","last_unstale":"2024-09-19T07:50:15.721727+0000","last_undegraded":"2024-09-19T07:50:15.721727+0000","last_fullsized":"2024-09-19T07:50:15.721727+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:43.586941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702027+0000","last_change":"2024-09-19T07:50:14.700632+0000","last_active":"2024-09-19T07:50:16.702027+0000","last_peered":"2024-09-19T07:50:16.702027+0000","last_clean":"2024-09-19T07:50:16.702027+0000","last_became_active":"2024-09-19T07:50:14.700366+0000","last_became_peered":"2024-09-19T07:50:14.700366+0000","last_unstale":"2024-09-19T07:50:16.702027+0000","last_undegraded":"2024-09-19T07:50:16.702027+0000","last_fullsized":"2024-09-19T07:50:16.702027+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:40:38.108015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.709182+0000","last_change":"2024-09-19T07:50:15.707603+0000","last_active":"2024-09-19T07:50:15.709182+0000","last_peered":"2024-09-19T07:50:15.709182+0000","last_clean":"2024-09-19T07:50:15.709182+0000","last_became_active":"2024-09-19T07:50:15.707301+0000","last_became_peered":"2024-09-19T07:50:15.707301+0000","last_unstale":"2024-09-19T07:50:15.709182+0000","last_undegraded":"2024-09-19T07:50:15.709182+0000","last_fullsized":"2024-09-19T07:50:15.709182+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:35.633861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"92'8","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702406+0000","last_change":"2024-09-19T07:50:14.698635+0000","last_active":"2024-09-19T07:50:16.702406+0000","last_peered":"2024-09-19T07:50:16.702406+0000","last_clean":"2024-09-19T07:50:16.702406+0000","last_became_active":"2024-09-19T07:50:14.698359+0000","last_became_peered":"2024-09-19T07:50:14.698359+0000","last_unstale":"2024-09-19T07:50:16.702406+0000","last_undegraded":"2024-09-19T07:50:16.702406+0000","last_fullsized":"2024-09-19T07:50:16.702406+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:57.251575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.687900+0000","last_change":"2024-09-19T07:50:13.687900+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.687900+0000","last_undegraded":"2024-09-19T07:50:13.687900+0000","last_fullsized":"2024-09-19T07:50:13.687900+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685677+0000","last_change":"2024-09-19T07:50:13.685677+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685677+0000","last_undegraded":"2024-09-19T07:50:13.685677+0000","last_fullsized":"2024-09-19T07:50:13.685677+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.684538+0000","last_change":"2024-09-19T07:50:13.684538+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.684538+0000","last_undegraded":"2024-09-19T07:50:13.684538+0000","last_fullsized":"2024-09-19T07:50:13.684538+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683215+0000","last_change":"2024-09-19T07:50:13.683215+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683215+0000","last_undegraded":"2024-09-19T07:50:13.683215+0000","last_fullsized":"2024-09-19T07:50:13.683215+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.693828+0000","last_change":"2024-09-19T07:50:14.695669+0000","last_active":"2024-09-19T07:50:15.693828+0000","last_peered":"2024-09-19T07:50:15.693828+0000","last_clean":"2024-09-19T07:50:15.693828+0000","last_became_active":"2024-09-19T07:50:14.695399+0000","last_became_peered":"2024-09-19T07:50:14.695399+0000","last_unstale":"2024-09-19T07:50:15.693828+0000","last_undegraded":"2024-09-19T07:50:15.693828+0000","last_fullsized":"2024-09-19T07:50:15.693828+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:53.625672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683418+0000","last_change":"2024-09-19T07:50:13.683418+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683418+0000","last_undegraded":"2024-09-19T07:50:13.683418+0000","last_fullsized":"2024-09-19T07:50:13.683418+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.693599+0000","last_change":"2024-09-19T07:50:14.705188+0000","last_active":"2024-09-19T07:50:15.693599+0000","last_peered":"2024-09-19T07:50:15.693599+0000","last_clean":"2024-09-19T07:50:15.693599+0000","last_became_active":"2024-09-19T07:50:14.704804+0000","last_became_peered":"2024-09-19T07:50:14.704804+0000","last_unstale":"2024-09-19T07:50:15.693599+0000","last_undegraded":"2024-09-19T07:50:15.693599+0000","last_fullsized":"2024-09-19T07:50:15.693599+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:49.332075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685337+0000","last_change":"2024-09-19T07:50:13.685337+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685337+0000","last_undegraded":"2024-09-19T07:50:13.685337+0000","last_fullsized":"2024-09-19T07:50:13.685337+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683158+0000","last_change":"2024-09-19T07:50:13.683158+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683158+0000","last_undegraded":"2024-09-19T07:50:13.683158+0000","last_fullsized":"2024-09-19T07:50:13.683158+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683021+0000","last_change":"2024-09-19T07:50:13.683021+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683021+0000","last_undegraded":"2024-09-19T07:50:13.683021+0000","last_fullsized":"2024-09-19T07:50:13.683021+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.687478+0000","last_change":"2024-09-19T07:50:13.687478+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.687478+0000","last_undegraded":"2024-09-19T07:50:13.687478+0000","last_fullsized":"2024-09-19T07:50:13.687478+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.721498+0000","last_change":"2024-09-19T07:50:15.713492+0000","last_active":"2024-09-19T07:50:15.721498+0000","last_peered":"2024-09-19T07:50:15.721498+0000","last_clean":"2024-09-19T07:50:15.721498+0000","last_became_active":"2024-09-19T07:50:15.713228+0000","last_became_peered":"2024-09-19T07:50:15.713228+0000","last_unstale":"2024-09-19T07:50:15.721498+0000","last_undegraded":"2024-09-19T07:50:15.721498+0000","last_fullsized":"2024-09-19T07:50:15.721498+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:21.796433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.686181+0000","last_change":"2024-09-19T07:50:13.686181+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.686181+0000","last_undegraded":"2024-09-19T07:50:13.686181+0000","last_fullsized":"2024-09-19T07:50:13.686181+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143615+0000","last_change":"2024-09-19T07:50:15.708869+0000","last_active":"2024-09-19T07:50:17.143615+0000","last_peered":"2024-09-19T07:50:17.143615+0000","last_clean":"2024-09-19T07:50:17.143615+0000","last_became_active":"2024-09-19T07:50:15.708595+0000","last_became_peered":"2024-09-19T07:50:15.708595+0000","last_unstale":"2024-09-19T07:50:17.143615+0000","last_undegraded":"2024-09-19T07:50:17.143615+0000","last_fullsized":"2024-09-19T07:50:17.143615+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:55.546102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683021+0000","last_change":"2024-09-19T07:50:13.683021+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683021+0000","last_undegraded":"2024-09-19T07:50:13.683021+0000","last_fullsized":"2024-09-19T07:50:13.683021+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.687559+0000","last_change":"2024-09-19T07:50:13.687559+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.687559+0000","last_undegraded":"2024-09-19T07:50:13.687559+0000","last_fullsized":"2024-09-19T07:50:13.687559+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.686119+0000","last_change":"2024-09-19T07:50:13.686119+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.686119+0000","last_undegraded":"2024-09-19T07:50:13.686119+0000","last_fullsized":"2024-09-19T07:50:13.686119+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702502+0000","last_change":"2024-09-19T07:50:14.705646+0000","last_active":"2024-09-19T07:50:16.702502+0000","last_peered":"2024-09-19T07:50:16.702502+0000","last_clean":"2024-09-19T07:50:16.702502+0000","last_became_active":"2024-09-19T07:50:14.705368+0000","last_became_peered":"2024-09-19T07:50:14.705368+0000","last_unstale":"2024-09-19T07:50:16.702502+0000","last_undegraded":"2024-09-19T07:50:16.702502+0000","last_fullsized":"2024-09-19T07:50:16.702502+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:29.229270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":294,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-19T07:50:13.686575+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:50:13.686575+0000","last_peered":"2024-09-19T07:50:13.686575+0000","last_clean":"2024-09-19T07:50:13.686575+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:50:13.686575+0000","last_undegraded":"2024-09-19T07:50:13.686575+0000","last_fullsized":"2024-09-19T07:50:13.686575+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":44,"acting":44,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509531,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":103632,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":101950,"kb_avail":94268208,"statfs":{"total":96636764160,"available":96530644992,"internally_reserved":0,"allocated":1695744,"data_stored":1093834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":104397258},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509530,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100308,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":94271532,"statfs":{"total":96636764160,"available":96534048768,"internally_reserved":0,"allocated":1101824,"data_stored":503466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":101579206},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50312,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321528,"statfs":{"total":96636764160,"available":96585244672,"internally_reserved":0,"allocated":1101824,"data_stored":503466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86348,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285492,"statfs":{"total":96636764160,"available":96548343808,"internally_reserved":0,"allocated":1101824,"data_stored":503466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509530,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100812,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271028,"statfs":{"total":96636764160,"available":96533532672,"internally_reserved":0,"allocated":1695744,"data_stored":1093834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542235,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130800,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94241040,"statfs":{"total":96636764160,"available":96502824960,"internally_reserved":0,"allocated":1138688,"data_stored":523992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542235,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94776,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94277064,"statfs":{"total":96636764160,"available":96539713536,"internally_reserved":0,"allocated":1142784,"data_stored":524062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104420,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267420,"statfs":{"total":96636764160,"available":96529838080,"internally_reserved":0,"allocated":1126400,"data_stored":517300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:50:19.397 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3e is not active+clean 2024-09-19T07:50:19.397 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.398 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3b is not active+clean 2024-09-19T07:50:19.398 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.398 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3a is not active+clean 2024-09-19T07:50:19.398 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.398 INFO:tasks.ceph.ceph_manager.ceph:PG 14.39 is not active+clean 2024-09-19T07:50:19.398 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.398 INFO:tasks.ceph.ceph_manager.ceph:PG 14.37 is not active+clean 2024-09-19T07:50:19.398 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.398 INFO:tasks.ceph.ceph_manager.ceph:PG 14.35 is not active+clean 2024-09-19T07:50:19.398 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.398 INFO:tasks.ceph.ceph_manager.ceph:PG 14.32 is not active+clean 2024-09-19T07:50:19.399 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.399 INFO:tasks.ceph.ceph_manager.ceph:PG 14.30 is not active+clean 2024-09-19T07:50:19.399 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.399 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2f is not active+clean 2024-09-19T07:50:19.399 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.399 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2d is not active+clean 2024-09-19T07:50:19.399 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.400 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2b is not active+clean 2024-09-19T07:50:19.400 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.400 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2a is not active+clean 2024-09-19T07:50:19.400 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.400 INFO:tasks.ceph.ceph_manager.ceph:PG 14.29 is not active+clean 2024-09-19T07:50:19.400 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.401 INFO:tasks.ceph.ceph_manager.ceph:PG 14.27 is not active+clean 2024-09-19T07:50:19.401 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.401 INFO:tasks.ceph.ceph_manager.ceph:PG 14.26 is not active+clean 2024-09-19T07:50:19.401 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.401 INFO:tasks.ceph.ceph_manager.ceph:PG 14.23 is not active+clean 2024-09-19T07:50:19.401 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.402 INFO:tasks.ceph.ceph_manager.ceph:PG 14.22 is not active+clean 2024-09-19T07:50:19.402 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.402 INFO:tasks.ceph.ceph_manager.ceph:PG 14.21 is not active+clean 2024-09-19T07:50:19.402 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.402 INFO:tasks.ceph.ceph_manager.ceph:PG 14.20 is not active+clean 2024-09-19T07:50:19.402 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.403 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1c is not active+clean 2024-09-19T07:50:19.403 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.403 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1f is not active+clean 2024-09-19T07:50:19.403 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.688360+0000', 'last_change': '2024-09-19T07:50:13.688360+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.688360+0000', 'last_undegraded': '2024-09-19T07:50:13.688360+0000', 'last_fullsized': '2024-09-19T07:50:13.688360+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T07:50:19.403 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1d is not active+clean 2024-09-19T07:50:19.403 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.403 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1e is not active+clean 2024-09-19T07:50:19.403 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.684837+0000', 'last_change': '2024-09-19T07:50:13.684837+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.684837+0000', 'last_undegraded': '2024-09-19T07:50:13.684837+0000', 'last_fullsized': '2024-09-19T07:50:13.684837+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T07:50:19.404 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1e is not active+clean 2024-09-19T07:50:19.404 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.404 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1d is not active+clean 2024-09-19T07:50:19.404 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.685590+0000', 'last_change': '2024-09-19T07:50:13.685590+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.685590+0000', 'last_undegraded': '2024-09-19T07:50:13.685590+0000', 'last_fullsized': '2024-09-19T07:50:13.685590+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T07:50:19.404 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1f is not active+clean 2024-09-19T07:50:19.404 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.404 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1c is not active+clean 2024-09-19T07:50:19.404 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.688011+0000', 'last_change': '2024-09-19T07:50:13.688011+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.688011+0000', 'last_undegraded': '2024-09-19T07:50:13.688011+0000', 'last_fullsized': '2024-09-19T07:50:13.688011+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T07:50:19.405 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1b is not active+clean 2024-09-19T07:50:19.405 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.683040+0000', 'last_change': '2024-09-19T07:50:13.683040+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.683040+0000', 'last_undegraded': '2024-09-19T07:50:13.683040+0000', 'last_fullsized': '2024-09-19T07:50:13.683040+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T07:50:19.405 INFO:tasks.ceph.ceph_manager.ceph:PG 14.19 is not active+clean 2024-09-19T07:50:19.405 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.405 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1a is not active+clean 2024-09-19T07:50:19.405 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.405 INFO:tasks.ceph.ceph_manager.ceph:PG 13.18 is not active+clean 2024-09-19T07:50:19.405 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.685235+0000', 'last_change': '2024-09-19T07:50:13.685235+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.685235+0000', 'last_undegraded': '2024-09-19T07:50:13.685235+0000', 'last_fullsized': '2024-09-19T07:50:13.685235+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T07:50:19.405 INFO:tasks.ceph.ceph_manager.ceph:PG 14.14 is not active+clean 2024-09-19T07:50:19.406 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.406 INFO:tasks.ceph.ceph_manager.ceph:PG 13.17 is not active+clean 2024-09-19T07:50:19.406 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.683088+0000', 'last_change': '2024-09-19T07:50:13.683088+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.683088+0000', 'last_undegraded': '2024-09-19T07:50:13.683088+0000', 'last_fullsized': '2024-09-19T07:50:13.683088+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T07:50:19.406 INFO:tasks.ceph.ceph_manager.ceph:PG 14.15 is not active+clean 2024-09-19T07:50:19.406 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.406 INFO:tasks.ceph.ceph_manager.ceph:PG 13.15 is not active+clean 2024-09-19T07:50:19.406 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.685114+0000', 'last_change': '2024-09-19T07:50:13.685114+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.685114+0000', 'last_undegraded': '2024-09-19T07:50:13.685114+0000', 'last_fullsized': '2024-09-19T07:50:13.685114+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T07:50:19.406 INFO:tasks.ceph.ceph_manager.ceph:PG 13.14 is not active+clean 2024-09-19T07:50:19.406 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.686167+0000', 'last_change': '2024-09-19T07:50:13.686167+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.686167+0000', 'last_undegraded': '2024-09-19T07:50:13.686167+0000', 'last_fullsized': '2024-09-19T07:50:13.686167+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T07:50:19.406 INFO:tasks.ceph.ceph_manager.ceph:PG 13.13 is not active+clean 2024-09-19T07:50:19.406 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.685384+0000', 'last_change': '2024-09-19T07:50:13.685384+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.685384+0000', 'last_undegraded': '2024-09-19T07:50:13.685384+0000', 'last_fullsized': '2024-09-19T07:50:13.685384+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T07:50:19.407 INFO:tasks.ceph.ceph_manager.ceph:PG 14.11 is not active+clean 2024-09-19T07:50:19.407 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.407 INFO:tasks.ceph.ceph_manager.ceph:PG 13.12 is not active+clean 2024-09-19T07:50:19.407 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.686136+0000', 'last_change': '2024-09-19T07:50:13.686136+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.686136+0000', 'last_undegraded': '2024-09-19T07:50:13.686136+0000', 'last_fullsized': '2024-09-19T07:50:13.686136+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T07:50:19.407 INFO:tasks.ceph.ceph_manager.ceph:PG 14.c is not active+clean 2024-09-19T07:50:19.407 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.407 INFO:tasks.ceph.ceph_manager.ceph:PG 13.f is not active+clean 2024-09-19T07:50:19.407 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.687900+0000', 'last_change': '2024-09-19T07:50:13.687900+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.687900+0000', 'last_undegraded': '2024-09-19T07:50:13.687900+0000', 'last_fullsized': '2024-09-19T07:50:13.687900+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T07:50:19.407 INFO:tasks.ceph.ceph_manager.ceph:PG 14.d is not active+clean 2024-09-19T07:50:19.407 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.407 INFO:tasks.ceph.ceph_manager.ceph:PG 13.e is not active+clean 2024-09-19T07:50:19.408 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.685677+0000', 'last_change': '2024-09-19T07:50:13.685677+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.685677+0000', 'last_undegraded': '2024-09-19T07:50:13.685677+0000', 'last_fullsized': '2024-09-19T07:50:13.685677+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T07:50:19.408 INFO:tasks.ceph.ceph_manager.ceph:PG 14.e is not active+clean 2024-09-19T07:50:19.408 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.408 INFO:tasks.ceph.ceph_manager.ceph:PG 13.d is not active+clean 2024-09-19T07:50:19.408 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.684538+0000', 'last_change': '2024-09-19T07:50:13.684538+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.684538+0000', 'last_undegraded': '2024-09-19T07:50:13.684538+0000', 'last_fullsized': '2024-09-19T07:50:13.684538+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T07:50:19.408 INFO:tasks.ceph.ceph_manager.ceph:PG 14.8 is not active+clean 2024-09-19T07:50:19.408 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.408 INFO:tasks.ceph.ceph_manager.ceph:PG 13.b is not active+clean 2024-09-19T07:50:19.408 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.683215+0000', 'last_change': '2024-09-19T07:50:13.683215+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.683215+0000', 'last_undegraded': '2024-09-19T07:50:13.683215+0000', 'last_fullsized': '2024-09-19T07:50:13.683215+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T07:50:19.408 INFO:tasks.ceph.ceph_manager.ceph:PG 14.9 is not active+clean 2024-09-19T07:50:19.408 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.409 INFO:tasks.ceph.ceph_manager.ceph:PG 14.a is not active+clean 2024-09-19T07:50:19.409 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.409 INFO:tasks.ceph.ceph_manager.ceph:PG 13.9 is not active+clean 2024-09-19T07:50:19.409 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.683418+0000', 'last_change': '2024-09-19T07:50:13.683418+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.683418+0000', 'last_undegraded': '2024-09-19T07:50:13.683418+0000', 'last_fullsized': '2024-09-19T07:50:13.683418+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T07:50:19.409 INFO:tasks.ceph.ceph_manager.ceph:PG 14.b is not active+clean 2024-09-19T07:50:19.409 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.409 INFO:tasks.ceph.ceph_manager.ceph:PG 14.4 is not active+clean 2024-09-19T07:50:19.409 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.409 INFO:tasks.ceph.ceph_manager.ceph:PG 13.7 is not active+clean 2024-09-19T07:50:19.409 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.685337+0000', 'last_change': '2024-09-19T07:50:13.685337+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.685337+0000', 'last_undegraded': '2024-09-19T07:50:13.685337+0000', 'last_fullsized': '2024-09-19T07:50:13.685337+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T07:50:19.409 INFO:tasks.ceph.ceph_manager.ceph:PG 14.5 is not active+clean 2024-09-19T07:50:19.410 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.410 INFO:tasks.ceph.ceph_manager.ceph:PG 13.6 is not active+clean 2024-09-19T07:50:19.410 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.683158+0000', 'last_change': '2024-09-19T07:50:13.683158+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.683158+0000', 'last_undegraded': '2024-09-19T07:50:13.683158+0000', 'last_fullsized': '2024-09-19T07:50:13.683158+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T07:50:19.410 INFO:tasks.ceph.ceph_manager.ceph:PG 14.6 is not active+clean 2024-09-19T07:50:19.410 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.410 INFO:tasks.ceph.ceph_manager.ceph:PG 13.5 is not active+clean 2024-09-19T07:50:19.410 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.683021+0000', 'last_change': '2024-09-19T07:50:13.683021+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.683021+0000', 'last_undegraded': '2024-09-19T07:50:13.683021+0000', 'last_fullsized': '2024-09-19T07:50:13.683021+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T07:50:19.410 INFO:tasks.ceph.ceph_manager.ceph:PG 14.7 is not active+clean 2024-09-19T07:50:19.410 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.410 INFO:tasks.ceph.ceph_manager.ceph:PG 13.4 is not active+clean 2024-09-19T07:50:19.410 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.687478+0000', 'last_change': '2024-09-19T07:50:13.687478+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.687478+0000', 'last_undegraded': '2024-09-19T07:50:13.687478+0000', 'last_fullsized': '2024-09-19T07:50:13.687478+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T07:50:19.411 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3 is not active+clean 2024-09-19T07:50:19.411 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.686181+0000', 'last_change': '2024-09-19T07:50:13.686181+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.686181+0000', 'last_undegraded': '2024-09-19T07:50:13.686181+0000', 'last_fullsized': '2024-09-19T07:50:13.686181+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T07:50:19.411 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2 is not active+clean 2024-09-19T07:50:19.411 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.683021+0000', 'last_change': '2024-09-19T07:50:13.683021+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.683021+0000', 'last_undegraded': '2024-09-19T07:50:13.683021+0000', 'last_fullsized': '2024-09-19T07:50:13.683021+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T07:50:19.411 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2 is not active+clean 2024-09-19T07:50:19.411 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.411 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1 is not active+clean 2024-09-19T07:50:19.411 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.687559+0000', 'last_change': '2024-09-19T07:50:13.687559+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.687559+0000', 'last_undegraded': '2024-09-19T07:50:13.687559+0000', 'last_fullsized': '2024-09-19T07:50:13.687559+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T07:50:19.411 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3 is not active+clean 2024-09-19T07:50:19.411 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.411 INFO:tasks.ceph.ceph_manager.ceph:PG 13.0 is not active+clean 2024-09-19T07:50:19.411 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:50:13.686119+0000', 'last_change': '2024-09-19T07:50:13.686119+0000', 'last_active': '2024-09-19T07:50:13.674730+0000', 'last_peered': '2024-09-19T07:50:13.674730+0000', 'last_clean': '2024-09-19T07:50:13.674730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:13.686119+0000', 'last_undegraded': '2024-09-19T07:50:13.686119+0000', 'last_fullsized': '2024-09-19T07:50:13.686119+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:13.674730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T07:50:19.412 INFO:tasks.ceph.ceph_manager.ceph:PG 14.f is not active+clean 2024-09-19T07:50:19.412 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T07:50:14.680382+0000', 'last_change': '2024-09-19T07:50:14.680382+0000', 'last_active': '2024-09-19T07:50:14.680382+0000', 'last_peered': '2024-09-19T07:50:14.680382+0000', 'last_clean': '2024-09-19T07:50:14.680382+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:50:14.680382+0000', 'last_undegraded': '2024-09-19T07:50:14.680382+0000', 'last_fullsized': '2024-09-19T07:50:14.680382+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'last_clean_scrub_stamp': '2024-09-19T07:50:14.680382+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T07:50:19.412 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T07:50:19.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.535+0000 7f894a331640 1 Processor -- start 2024-09-19T07:50:19.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.535+0000 7f894a331640 1 -- start start 2024-09-19T07:50:19.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.536+0000 7f894a331640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89440d21f0 0x7f89440d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:19.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.536+0000 7f894a331640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89440d2b10 con 0x7f8944141ad0 2024-09-19T07:50:19.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.536+0000 7f894a331640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89441620b0 con 0x7f8944072590 2024-09-19T07:50:19.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.536+0000 7f894a331640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8944141750 con 0x7f89440d21f0 2024-09-19T07:50:19.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.536+0000 7f89488ce640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8944141ad0 0x7f8944141f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50660/0 (socket says 172.21.15.16:50660) 2024-09-19T07:50:19.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.536+0000 7f89488ce640 1 -- 172.21.15.16:0/3723562282 learned_addr learned my addr 172.21.15.16:0/3723562282 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:19.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.536+0000 7f8943fff640 1 --2- 172.21.15.16:0/3723562282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89440d21f0 0x7f89440d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:19.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.536+0000 7f8942ffd640 1 -- 172.21.15.16:0/3723562282 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2545273850 0 0) 0x7f89440d2b10 con 0x7f8944141ad0 2024-09-19T07:50:19.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.536+0000 7f8942ffd640 1 -- 172.21.15.16:0/3723562282 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f892c003520 con 0x7f8944141ad0 2024-09-19T07:50:19.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.536+0000 7f8942ffd640 1 -- 172.21.15.16:0/3723562282 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 141872607 0 0) 0x7f892c003520 con 0x7f8944141ad0 2024-09-19T07:50:19.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.536+0000 7f8942ffd640 1 -- 172.21.15.16:0/3723562282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89440d21f0 msgr2=0x7f89440d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:19.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.536+0000 7f8942ffd640 1 --2- 172.21.15.16:0/3723562282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89440d21f0 0x7f89440d25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.536+0000 7f8942ffd640 1 -- 172.21.15.16:0/3723562282 >> v1:172.21.15.146:6789/0 conn(0x7f8944072590 legacy=0x7f894406d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:19.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.536+0000 7f8942ffd640 1 -- 172.21.15.16:0/3723562282 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8944162370 con 0x7f8944141ad0 2024-09-19T07:50:19.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.536+0000 7f8942ffd640 1 -- 172.21.15.16:0/3723562282 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f893c0019f0 con 0x7f8944141ad0 2024-09-19T07:50:19.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.537+0000 7f8942ffd640 1 -- 172.21.15.16:0/3723562282 >> v1:172.21.15.16:6789/0 conn(0x7f8944141ad0 legacy=0x7f8944141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:19.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.537+0000 7f8942ffd640 1 --2- 172.21.15.16:0/3723562282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89440d21f0 0x7f892c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:19.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.537+0000 7f8942ffd640 1 --2- 172.21.15.16:0/3723562282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892c003f20 0x7f892c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:19.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.537+0000 7f8942ffd640 1 --2- 172.21.15.16:0/3723562282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f892c004d40 0x7f892c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:19.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.537+0000 7f8942ffd640 1 -- 172.21.15.16:0/3723562282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8944141750 con 0x7f892c003f20 2024-09-19T07:50:19.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.537+0000 7f8942ffd640 1 -- 172.21.15.16:0/3723562282 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f892c00bad0 con 0x7f892c004d40 2024-09-19T07:50:19.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.537+0000 7f8942ffd640 1 -- 172.21.15.16:0/3723562282 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f892c00bcd0 con 0x7f89440d21f0 2024-09-19T07:50:19.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.537+0000 7f89437fe640 1 --2- 172.21.15.16:0/3723562282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892c003f20 0x7f892c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:19.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.537+0000 7f8943fff640 1 --2- 172.21.15.16:0/3723562282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89440d21f0 0x7f892c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:19.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.537+0000 7f89488ce640 1 --2- 172.21.15.16:0/3723562282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f892c004d40 0x7f892c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:19.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.537+0000 7f89437fe640 1 -- 172.21.15.16:0/3723562282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89440d21f0 msgr2=0x7f892c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:19.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.537+0000 7f89437fe640 1 --2- 172.21.15.16:0/3723562282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89440d21f0 0x7f892c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.537+0000 7f89437fe640 1 -- 172.21.15.16:0/3723562282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f892c004d40 msgr2=0x7f892c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:19.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.537+0000 7f89437fe640 1 --2- 172.21.15.16:0/3723562282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f892c004d40 0x7f892c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.537+0000 7f89437fe640 1 -- 172.21.15.16:0/3723562282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c003520 con 0x7f892c003f20 2024-09-19T07:50:19.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.537+0000 7f89488ce640 1 --2- 172.21.15.16:0/3723562282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f892c004d40 0x7f892c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:19.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.538+0000 7f89437fe640 1 --2- 172.21.15.16:0/3723562282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892c003f20 0x7f892c004800 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f894406db00 tx=0x7f892c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7325106542f6c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:19.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.538+0000 7f8942ffd640 1 -- 172.21.15.16:0/3723562282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938001460 con 0x7f892c003f20 2024-09-19T07:50:19.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.538+0000 7f8942ffd640 1 -- 172.21.15.16:0/3723562282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f893802ac00 con 0x7f892c003f20 2024-09-19T07:50:19.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.538+0000 7f8942ffd640 1 -- 172.21.15.16:0/3723562282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938007260 con 0x7f892c003f20 2024-09-19T07:50:19.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.538+0000 7f894a331640 1 -- 172.21.15.16:0/3723562282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892c003f20 msgr2=0x7f892c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:19.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.538+0000 7f894a331640 1 --2- 172.21.15.16:0/3723562282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892c003f20 0x7f892c004800 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f894406db00 tx=0x7f892c005830 comp rx=0 tx=0).stop 2024-09-19T07:50:19.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.538+0000 7f89488ce640 1 -- 172.21.15.16:0/3723562282 reap_dead start 2024-09-19T07:50:19.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.538+0000 7f894a331640 1 -- 172.21.15.16:0/3723562282 shutdown_connections 2024-09-19T07:50:19.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.538+0000 7f894a331640 1 -- 172.21.15.16:0/3723562282 >> 172.21.15.16:0/3723562282 conn(0x7f89440b3ed0 msgr2=0x7f89440b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:19.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.538+0000 7f894a331640 1 -- 172.21.15.16:0/3723562282 shutdown_connections 2024-09-19T07:50:19.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.538+0000 7f894a331640 1 -- 172.21.15.16:0/3723562282 wait complete. 2024-09-19T07:50:19.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.540+0000 7f894a331640 1 Processor -- start 2024-09-19T07:50:19.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.540+0000 7f894a331640 1 -- start start 2024-09-19T07:50:19.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.540+0000 7f894a331640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f892c003f20 0x7f8944152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:19.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.540+0000 7f894a331640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8944153de0 0x7f89441408d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:19.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.540+0000 7f894a331640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8944154240 0x7f8944140e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:19.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.540+0000 7f894a331640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8944162930 con 0x7f8944153de0 2024-09-19T07:50:19.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.540+0000 7f894a331640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f894406d7f0 con 0x7f8944154240 2024-09-19T07:50:19.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.540+0000 7f894a331640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8944162c10 con 0x7f892c003f20 2024-09-19T07:50:19.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.541+0000 7f89437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8944153de0 0x7f89441408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:19.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.541+0000 7f89437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8944153de0 0x7f89441408d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40084/0 (socket says 172.21.15.16:40084) 2024-09-19T07:50:19.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.541+0000 7f89437fe640 1 -- 172.21.15.16:0/1581877928 learned_addr learned my addr 172.21.15.16:0/1581877928 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:19.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.541+0000 7f8943fff640 1 --2- 172.21.15.16:0/1581877928 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f892c003f20 0x7f8944152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:19.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.541+0000 7f89488ce640 1 --2- 172.21.15.16:0/1581877928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8944154240 0x7f8944140e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:19.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.541+0000 7f89437fe640 1 -- 172.21.15.16:0/1581877928 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f892c003f20 msgr2=0x7f8944152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:19.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.541+0000 7f89437fe640 1 --2- 172.21.15.16:0/1581877928 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f892c003f20 0x7f8944152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.541+0000 7f89437fe640 1 -- 172.21.15.16:0/1581877928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8944154240 msgr2=0x7f8944140e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:19.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.541+0000 7f89437fe640 1 --2- 172.21.15.16:0/1581877928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8944154240 0x7f8944140e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.541+0000 7f89437fe640 1 -- 172.21.15.16:0/1581877928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8944151480 con 0x7f8944153de0 2024-09-19T07:50:19.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.541+0000 7f89437fe640 1 --2- 172.21.15.16:0/1581877928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8944153de0 0x7f89441408d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f893802a5f0 tx=0x7f8938001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:19.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.541+0000 7f8940ff9640 1 -- 172.21.15.16:0/1581877928 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893803f020 con 0x7f8944153de0 2024-09-19T07:50:19.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.541+0000 7f8940ff9640 1 -- 172.21.15.16:0/1581877928 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89380052e0 con 0x7f8944153de0 2024-09-19T07:50:19.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.541+0000 7f8940ff9640 1 -- 172.21.15.16:0/1581877928 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938001460 con 0x7f8944153de0 2024-09-19T07:50:19.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.541+0000 7f894a331640 1 -- 172.21.15.16:0/1581877928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8944151760 con 0x7f8944153de0 2024-09-19T07:50:19.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.541+0000 7f894a331640 1 -- 172.21.15.16:0/1581877928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8944151d20 con 0x7f8944153de0 2024-09-19T07:50:19.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.542+0000 7f894a331640 1 -- 172.21.15.16:0/1581877928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8944072080 con 0x7f8944153de0 2024-09-19T07:50:19.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.543+0000 7f8940ff9640 1 -- 172.21.15.16:0/1581877928 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8938042020 con 0x7f8944153de0 2024-09-19T07:50:19.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.543+0000 7f8940ff9640 1 --2- 172.21.15.16:0/1581877928 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f893006c2b0 0x7f893006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:19.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.543+0000 7f8940ff9640 1 -- 172.21.15.16:0/1581877928 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f89380abb70 con 0x7f8944153de0 2024-09-19T07:50:19.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.544+0000 7f8943fff640 1 --2- 172.21.15.16:0/1581877928 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f893006c2b0 0x7f893006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:19.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.544+0000 7f8943fff640 1 --2- 172.21.15.16:0/1581877928 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f893006c2b0 0x7f893006e770 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8934009b00 tx=0x7f8934008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:19.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.548+0000 7f8940ff9640 1 -- 172.21.15.16:0/1581877928 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8944072080 con 0x7f8944153de0 2024-09-19T07:50:19.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.796+0000 7f894a331640 1 -- 172.21.15.16:0/1581877928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f8944158c30 con 0x7f8944153de0 2024-09-19T07:50:19.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.796+0000 7f8940ff9640 1 -- 172.21.15.16:0/1581877928 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1213 (secure 0 0 0) 0x7f8938049020 con 0x7f8944153de0 2024-09-19T07:50:19.796 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:19.797 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":457,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":93,"num_osds":8,"num_up_osds":8,"osd_up_since":1726731773,"num_in_osds":8,"osd_in_since":1726731764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":42},{"state_name":"active+clean","count":31},{"state_name":"creating+peering","count":24}],"num_pgs":97,"num_pools":3,"num_objects":6,"data_bytes":590368,"bytes_used":789921792,"bytes_avail":772304191488,"bytes_total":773094113280,"unknown_pgs_ratio":0.4329896867275238,"inactive_pgs_ratio":0.24742268025875092,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":47,"btime":"2024-09-19T07:50:17:700363+0000","id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":6402}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-19T07:46:18.501829+0000","services":{}},"progress_events":{}} 2024-09-19T07:50:19.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.798+0000 7f894a331640 1 -- 172.21.15.16:0/1581877928 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f893006c2b0 msgr2=0x7f893006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:19.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.798+0000 7f894a331640 1 --2- 172.21.15.16:0/1581877928 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f893006c2b0 0x7f893006e770 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8934009b00 tx=0x7f8934008040 comp rx=0 tx=0).stop 2024-09-19T07:50:19.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.798+0000 7f894a331640 1 -- 172.21.15.16:0/1581877928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8944153de0 msgr2=0x7f89441408d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:19.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.798+0000 7f894a331640 1 --2- 172.21.15.16:0/1581877928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8944153de0 0x7f89441408d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f893802a5f0 tx=0x7f8938001230 comp rx=0 tx=0).stop 2024-09-19T07:50:19.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.798+0000 7f894a331640 1 -- 172.21.15.16:0/1581877928 shutdown_connections 2024-09-19T07:50:19.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.798+0000 7f894a331640 1 --2- 172.21.15.16:0/1581877928 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f893006c2b0 0x7f893006e770 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.798+0000 7f894a331640 1 --2- 172.21.15.16:0/1581877928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8944154240 0x7f8944140e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.798+0000 7f894a331640 1 --2- 172.21.15.16:0/1581877928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8944153de0 0x7f89441408d0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.798+0000 7f894a331640 1 --2- 172.21.15.16:0/1581877928 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f892c003f20 0x7f8944152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.798+0000 7f894a331640 1 -- 172.21.15.16:0/1581877928 >> 172.21.15.16:0/1581877928 conn(0x7f89440b3ed0 msgr2=0x7f89440b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:19.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.798+0000 7f894a331640 1 -- 172.21.15.16:0/1581877928 shutdown_connections 2024-09-19T07:50:19.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.798+0000 7f894a331640 1 -- 172.21.15.16:0/1581877928 wait complete. 2024-09-19T07:50:19.810 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-19T07:50:19.810 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:50:19.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.911+0000 7f8c61542640 1 Processor -- start 2024-09-19T07:50:19.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.912+0000 7f8c61542640 1 -- start start 2024-09-19T07:50:19.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.912+0000 7f8c61542640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c5c06f9e0 0x7f8c5c0d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:19.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.912+0000 7f8c61542640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c5c0d0e00 con 0x7f8c5c0d09f0 2024-09-19T07:50:19.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.912+0000 7f8c61542640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c5c15b500 con 0x7f8c5c15b120 2024-09-19T07:50:19.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.912+0000 7f8c61542640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f8c5c06e940 con 0x7f8c5c06f9e0 2024-09-19T07:50:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.913+0000 7f8c5a7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c5c06f9e0 0x7f8c5c0d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.913+0000 7f8c5a7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c5c06f9e0 0x7f8c5c0d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40096/0 (socket says 172.21.15.16:40096) 2024-09-19T07:50:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.913+0000 7f8c5a7fc640 1 -- 172.21.15.16:0/2365485558 learned_addr learned my addr 172.21.15.16:0/2365485558 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.913+0000 7f8c5b7fe640 1 --1- 172.21.15.16:0/2365485558 >> v1:172.21.15.16:6789/0 conn(0x7f8c5c0d09f0 0x7f8c5c0c5160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50666/0 (socket says 172.21.15.16:50666) 2024-09-19T07:50:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.913+0000 7f8c5a7fc640 1 -- 172.21.15.16:0/2365485558 >> v1:172.21.15.146:6789/0 conn(0x7f8c5c15b120 legacy=0x7f8c5c06dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.913+0000 7f8c5a7fc640 1 -- 172.21.15.16:0/2365485558 >> v1:172.21.15.16:6789/0 conn(0x7f8c5c0d09f0 legacy=0x7f8c5c0c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.913+0000 7f8c5a7fc640 1 -- 172.21.15.16:0/2365485558 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5c06ec20 con 0x7f8c5c06f9e0 2024-09-19T07:50:19.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.914+0000 7f8c5a7fc640 1 --2- 172.21.15.16:0/2365485558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c06f9e0 0x7f8c5c0d04b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009870 tx=0x7f8c5002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d101d65ddb3d780d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:19.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.914+0000 7f8c59ffb640 1 -- 172.21.15.16:0/2365485558 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50037070 con 0x7f8c5c06f9e0 2024-09-19T07:50:19.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.914+0000 7f8c59ffb640 1 -- 172.21.15.16:0/2365485558 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c50032440 con 0x7f8c5c06f9e0 2024-09-19T07:50:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.914+0000 7f8c59ffb640 1 -- 172.21.15.16:0/2365485558 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50032810 con 0x7f8c5c06f9e0 2024-09-19T07:50:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.914+0000 7f8c61542640 1 -- 172.21.15.16:0/2365485558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c06f9e0 msgr2=0x7f8c5c0d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.914+0000 7f8c61542640 1 --2- 172.21.15.16:0/2365485558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c06f9e0 0x7f8c5c0d04b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009870 tx=0x7f8c5002a2a0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.914+0000 7f8c61542640 1 -- 172.21.15.16:0/2365485558 shutdown_connections 2024-09-19T07:50:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.914+0000 7f8c61542640 1 --2- 172.21.15.16:0/2365485558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c06f9e0 0x7f8c5c0d04b0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.914+0000 7f8c61542640 1 -- 172.21.15.16:0/2365485558 >> 172.21.15.16:0/2365485558 conn(0x7f8c5c06bdf0 msgr2=0x7f8c5c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.914+0000 7f8c61542640 1 -- 172.21.15.16:0/2365485558 shutdown_connections 2024-09-19T07:50:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.914+0000 7f8c61542640 1 -- 172.21.15.16:0/2365485558 wait complete. 2024-09-19T07:50:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.916+0000 7f8c61542640 1 Processor -- start 2024-09-19T07:50:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.916+0000 7f8c61542640 1 -- start start 2024-09-19T07:50:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c61542640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c5c06f9e0 0x7f8c5c072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c61542640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c5c0d09f0 0x7f8c5c14d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c61542640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c15b120 0x7f8c5c0736d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c61542640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c06f4c0 con 0x7f8c5c15b120 2024-09-19T07:50:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c61542640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c06f1e0 con 0x7f8c5c0d09f0 2024-09-19T07:50:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c61542640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8c5c06e5d0 con 0x7f8c5c06f9e0 2024-09-19T07:50:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c15b120 0x7f8c5c0736d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c15b120 0x7f8c5c0736d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40108/0 (socket says 172.21.15.16:40108) 2024-09-19T07:50:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c5b7fe640 1 -- 172.21.15.16:0/4103950148 learned_addr learned my addr 172.21.15.16:0/4103950148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c5affd640 1 --2- 172.21.15.16:0/4103950148 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c5c06f9e0 0x7f8c5c072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c5a7fc640 1 --2- 172.21.15.16:0/4103950148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c5c0d09f0 0x7f8c5c14d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c5b7fe640 1 -- 172.21.15.16:0/4103950148 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c5c06f9e0 msgr2=0x7f8c5c072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c5b7fe640 1 --2- 172.21.15.16:0/4103950148 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c5c06f9e0 0x7f8c5c072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c5b7fe640 1 -- 172.21.15.16:0/4103950148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c5c0d09f0 msgr2=0x7f8c5c14d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c5b7fe640 1 --2- 172.21.15.16:0/4103950148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c5c0d09f0 0x7f8c5c14d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c5b7fe640 1 -- 172.21.15.16:0/4103950148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5c072020 con 0x7f8c5c15b120 2024-09-19T07:50:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c5b7fe640 1 --2- 172.21.15.16:0/4103950148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c15b120 0x7f8c5c0736d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400c7b0 tx=0x7f8c5400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c437fe640 1 -- 172.21.15.16:0/4103950148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54017070 con 0x7f8c5c15b120 2024-09-19T07:50:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c437fe640 1 -- 172.21.15.16:0/4103950148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c5400a640 con 0x7f8c5c15b120 2024-09-19T07:50:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c61542640 1 -- 172.21.15.16:0/4103950148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5c072300 con 0x7f8c5c15b120 2024-09-19T07:50:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c437fe640 1 -- 172.21.15.16:0/4103950148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5400a9f0 con 0x7f8c5c15b120 2024-09-19T07:50:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.917+0000 7f8c61542640 1 -- 172.21.15.16:0/4103950148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c5c0728c0 con 0x7f8c5c15b120 2024-09-19T07:50:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.918+0000 7f8c61542640 1 -- 172.21.15.16:0/4103950148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c5c0ce060 con 0x7f8c5c15b120 2024-09-19T07:50:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.919+0000 7f8c437fe640 1 -- 172.21.15.16:0/4103950148 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8c5401a070 con 0x7f8c5c15b120 2024-09-19T07:50:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.919+0000 7f8c437fe640 1 --2- 172.21.15.16:0/4103950148 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8c4806c2b0 0x7f8c4806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.919+0000 7f8c437fe640 1 -- 172.21.15.16:0/4103950148 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f8c5408ab10 con 0x7f8c5c15b120 2024-09-19T07:50:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.920+0000 7f8c5affd640 1 --2- 172.21.15.16:0/4103950148 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8c4806c2b0 0x7f8c4806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.920+0000 7f8c5affd640 1 --2- 172.21.15.16:0/4103950148 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8c4806c2b0 0x7f8c4806e770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c006fd0 tx=0x7f8c4c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:19.924+0000 7f8c437fe640 1 -- 172.21.15.16:0/4103950148 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8c5c0d0e00 con 0x7f8c5c15b120 2024-09-19T07:50:20.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:20.095+0000 7f8c61542640 1 -- 172.21.15.16:0/4103950148 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c5c06c300 con 0x7f8c4806c2b0 2024-09-19T07:50:20.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:20.101+0000 7f8c437fe640 1 -- 172.21.15.16:0/4103950148 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+244023 (secure 0 0 0) 0x7f8c5c06c300 con 0x7f8c4806c2b0 2024-09-19T07:50:20.104 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:20.106 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:50:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:20.104+0000 7f8c61542640 1 -- 172.21.15.16:0/4103950148 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8c4806c2b0 msgr2=0x7f8c4806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:20.104+0000 7f8c61542640 1 --2- 172.21.15.16:0/4103950148 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8c4806c2b0 0x7f8c4806e770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c006fd0 tx=0x7f8c4c008040 comp rx=0 tx=0).stop 2024-09-19T07:50:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:20.104+0000 7f8c61542640 1 -- 172.21.15.16:0/4103950148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c15b120 msgr2=0x7f8c5c0736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:20.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:20.104+0000 7f8c61542640 1 --2- 172.21.15.16:0/4103950148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c15b120 0x7f8c5c0736d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400c7b0 tx=0x7f8c5400cc70 comp rx=0 tx=0).stop 2024-09-19T07:50:20.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:20.104+0000 7f8c61542640 1 -- 172.21.15.16:0/4103950148 shutdown_connections 2024-09-19T07:50:20.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:20.104+0000 7f8c61542640 1 --2- 172.21.15.16:0/4103950148 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8c4806c2b0 0x7f8c4806e770 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:20.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:20.104+0000 7f8c61542640 1 --2- 172.21.15.16:0/4103950148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c5c15b120 0x7f8c5c0736d0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:20.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:20.104+0000 7f8c61542640 1 --2- 172.21.15.16:0/4103950148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c5c0d09f0 0x7f8c5c14d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:20.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:20.104+0000 7f8c61542640 1 --2- 172.21.15.16:0/4103950148 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c5c06f9e0 0x7f8c5c072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:20.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:20.104+0000 7f8c61542640 1 -- 172.21.15.16:0/4103950148 >> 172.21.15.16:0/4103950148 conn(0x7f8c5c06bdf0 msgr2=0x7f8c5c06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:20.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:20.104+0000 7f8c61542640 1 -- 172.21.15.16:0/4103950148 shutdown_connections 2024-09-19T07:50:20.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:20.104+0000 7f8c61542640 1 -- 172.21.15.16:0/4103950148 wait complete. 2024-09-19T07:50:20.116 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-19T07:50:18.556331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":110,"acting":110,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":771408,"kb_used_data":9868,"kb_used_omap":12,"kb_used_meta":761331,"kb_avail":754203312,"statfs":{"total":773094113280,"available":772304191488,"internally_reserved":0,"allocated":10104832,"data_stored":5263420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":779603504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.875215"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143560+0000","last_change":"2024-09-19T07:50:15.716195+0000","last_active":"2024-09-19T07:50:17.143560+0000","last_peered":"2024-09-19T07:50:17.143560+0000","last_clean":"2024-09-19T07:50:17.143560+0000","last_became_active":"2024-09-19T07:50:15.715936+0000","last_became_peered":"2024-09-19T07:50:15.715936+0000","last_unstale":"2024-09-19T07:50:17.143560+0000","last_undegraded":"2024-09-19T07:50:17.143560+0000","last_fullsized":"2024-09-19T07:50:17.143560+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:50.198202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143176+0000","last_change":"2024-09-19T07:50:15.706380+0000","last_active":"2024-09-19T07:50:17.143176+0000","last_peered":"2024-09-19T07:50:17.143176+0000","last_clean":"2024-09-19T07:50:17.143176+0000","last_became_active":"2024-09-19T07:50:15.706055+0000","last_became_peered":"2024-09-19T07:50:15.706055+0000","last_unstale":"2024-09-19T07:50:17.143176+0000","last_undegraded":"2024-09-19T07:50:17.143176+0000","last_fullsized":"2024-09-19T07:50:17.143176+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:02.607702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702151+0000","last_change":"2024-09-19T07:50:15.714404+0000","last_active":"2024-09-19T07:50:16.702151+0000","last_peered":"2024-09-19T07:50:16.702151+0000","last_clean":"2024-09-19T07:50:16.702151+0000","last_became_active":"2024-09-19T07:50:15.714029+0000","last_became_peered":"2024-09-19T07:50:15.714029+0000","last_unstale":"2024-09-19T07:50:16.702151+0000","last_undegraded":"2024-09-19T07:50:16.702151+0000","last_fullsized":"2024-09-19T07:50:16.702151+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:52.950164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702212+0000","last_change":"2024-09-19T07:50:15.717130+0000","last_active":"2024-09-19T07:50:16.702212+0000","last_peered":"2024-09-19T07:50:16.702212+0000","last_clean":"2024-09-19T07:50:16.702212+0000","last_became_active":"2024-09-19T07:50:15.716820+0000","last_became_peered":"2024-09-19T07:50:15.716820+0000","last_unstale":"2024-09-19T07:50:16.702212+0000","last_undegraded":"2024-09-19T07:50:16.702212+0000","last_fullsized":"2024-09-19T07:50:16.702212+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:13.211000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.721697+0000","last_change":"2024-09-19T07:50:15.717933+0000","last_active":"2024-09-19T07:50:15.721697+0000","last_peered":"2024-09-19T07:50:15.721697+0000","last_clean":"2024-09-19T07:50:15.721697+0000","last_became_active":"2024-09-19T07:50:15.717636+0000","last_became_peered":"2024-09-19T07:50:15.717636+0000","last_unstale":"2024-09-19T07:50:15.721697+0000","last_undegraded":"2024-09-19T07:50:15.721697+0000","last_fullsized":"2024-09-19T07:50:15.721697+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:11.754897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.703751+0000","last_change":"2024-09-19T07:50:15.702296+0000","last_active":"2024-09-19T07:50:15.703751+0000","last_peered":"2024-09-19T07:50:15.703751+0000","last_clean":"2024-09-19T07:50:15.703751+0000","last_became_active":"2024-09-19T07:50:15.701479+0000","last_became_peered":"2024-09-19T07:50:15.701479+0000","last_unstale":"2024-09-19T07:50:15.703751+0000","last_undegraded":"2024-09-19T07:50:15.703751+0000","last_fullsized":"2024-09-19T07:50:15.703751+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:34.439949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702086+0000","last_change":"2024-09-19T07:50:15.700711+0000","last_active":"2024-09-19T07:50:16.702086+0000","last_peered":"2024-09-19T07:50:16.702086+0000","last_clean":"2024-09-19T07:50:16.702086+0000","last_became_active":"2024-09-19T07:50:15.700431+0000","last_became_peered":"2024-09-19T07:50:15.700431+0000","last_unstale":"2024-09-19T07:50:16.702086+0000","last_undegraded":"2024-09-19T07:50:16.702086+0000","last_fullsized":"2024-09-19T07:50:16.702086+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:55.660212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702008+0000","last_change":"2024-09-19T07:50:15.714991+0000","last_active":"2024-09-19T07:50:16.702008+0000","last_peered":"2024-09-19T07:50:16.702008+0000","last_clean":"2024-09-19T07:50:16.702008+0000","last_became_active":"2024-09-19T07:50:15.714620+0000","last_became_peered":"2024-09-19T07:50:15.714620+0000","last_unstale":"2024-09-19T07:50:16.702008+0000","last_undegraded":"2024-09-19T07:50:16.702008+0000","last_fullsized":"2024-09-19T07:50:16.702008+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:59.294901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143509+0000","last_change":"2024-09-19T07:50:15.697733+0000","last_active":"2024-09-19T07:50:17.143509+0000","last_peered":"2024-09-19T07:50:17.143509+0000","last_clean":"2024-09-19T07:50:17.143509+0000","last_became_active":"2024-09-19T07:50:15.697453+0000","last_became_peered":"2024-09-19T07:50:15.697453+0000","last_unstale":"2024-09-19T07:50:17.143509+0000","last_undegraded":"2024-09-19T07:50:17.143509+0000","last_fullsized":"2024-09-19T07:50:17.143509+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:50.763269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702280+0000","last_change":"2024-09-19T07:50:15.706893+0000","last_active":"2024-09-19T07:50:16.702280+0000","last_peered":"2024-09-19T07:50:16.702280+0000","last_clean":"2024-09-19T07:50:16.702280+0000","last_became_active":"2024-09-19T07:50:15.704710+0000","last_became_peered":"2024-09-19T07:50:15.704710+0000","last_unstale":"2024-09-19T07:50:16.702280+0000","last_undegraded":"2024-09-19T07:50:16.702280+0000","last_fullsized":"2024-09-19T07:50:16.702280+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:33:41.955036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143012+0000","last_change":"2024-09-19T07:50:15.707589+0000","last_active":"2024-09-19T07:50:17.143012+0000","last_peered":"2024-09-19T07:50:17.143012+0000","last_clean":"2024-09-19T07:50:17.143012+0000","last_became_active":"2024-09-19T07:50:15.707303+0000","last_became_peered":"2024-09-19T07:50:15.707303+0000","last_unstale":"2024-09-19T07:50:17.143012+0000","last_undegraded":"2024-09-19T07:50:17.143012+0000","last_fullsized":"2024-09-19T07:50:17.143012+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:02.234544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.701158+0000","last_change":"2024-09-19T07:50:15.700322+0000","last_active":"2024-09-19T07:50:15.701158+0000","last_peered":"2024-09-19T07:50:15.701158+0000","last_clean":"2024-09-19T07:50:15.701158+0000","last_became_active":"2024-09-19T07:50:15.700039+0000","last_became_peered":"2024-09-19T07:50:15.700039+0000","last_unstale":"2024-09-19T07:50:15.701158+0000","last_undegraded":"2024-09-19T07:50:15.701158+0000","last_fullsized":"2024-09-19T07:50:15.701158+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:53.587874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702348+0000","last_change":"2024-09-19T07:50:15.707450+0000","last_active":"2024-09-19T07:50:16.702348+0000","last_peered":"2024-09-19T07:50:16.702348+0000","last_clean":"2024-09-19T07:50:16.702348+0000","last_became_active":"2024-09-19T07:50:15.707187+0000","last_became_peered":"2024-09-19T07:50:15.707187+0000","last_unstale":"2024-09-19T07:50:16.702348+0000","last_undegraded":"2024-09-19T07:50:16.702348+0000","last_fullsized":"2024-09-19T07:50:16.702348+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:48.678322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.688360+0000","last_change":"2024-09-19T07:50:13.688360+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.688360+0000","last_undegraded":"2024-09-19T07:50:13.688360+0000","last_fullsized":"2024-09-19T07:50:13.688360+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.684837+0000","last_change":"2024-09-19T07:50:13.684837+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.684837+0000","last_undegraded":"2024-09-19T07:50:13.684837+0000","last_fullsized":"2024-09-19T07:50:13.684837+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685590+0000","last_change":"2024-09-19T07:50:13.685590+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685590+0000","last_undegraded":"2024-09-19T07:50:13.685590+0000","last_fullsized":"2024-09-19T07:50:13.685590+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.688011+0000","last_change":"2024-09-19T07:50:13.688011+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.688011+0000","last_undegraded":"2024-09-19T07:50:13.688011+0000","last_fullsized":"2024-09-19T07:50:13.688011+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702304+0000","last_change":"2024-09-19T07:50:15.704536+0000","last_active":"2024-09-19T07:50:16.702304+0000","last_peered":"2024-09-19T07:50:16.702304+0000","last_clean":"2024-09-19T07:50:16.702304+0000","last_became_active":"2024-09-19T07:50:15.704223+0000","last_became_peered":"2024-09-19T07:50:15.704223+0000","last_unstale":"2024-09-19T07:50:16.702304+0000","last_undegraded":"2024-09-19T07:50:16.702304+0000","last_fullsized":"2024-09-19T07:50:16.702304+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:41.144795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683040+0000","last_change":"2024-09-19T07:50:13.683040+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683040+0000","last_undegraded":"2024-09-19T07:50:13.683040+0000","last_fullsized":"2024-09-19T07:50:13.683040+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702372+0000","last_change":"2024-09-19T07:50:14.701473+0000","last_active":"2024-09-19T07:50:16.702372+0000","last_peered":"2024-09-19T07:50:16.702372+0000","last_clean":"2024-09-19T07:50:16.702372+0000","last_became_active":"2024-09-19T07:50:14.701196+0000","last_became_peered":"2024-09-19T07:50:14.701196+0000","last_unstale":"2024-09-19T07:50:16.702372+0000","last_undegraded":"2024-09-19T07:50:16.702372+0000","last_fullsized":"2024-09-19T07:50:16.702372+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:45.158919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143734+0000","last_change":"2024-09-19T07:50:14.700842+0000","last_active":"2024-09-19T07:50:17.143734+0000","last_peered":"2024-09-19T07:50:17.143734+0000","last_clean":"2024-09-19T07:50:17.143734+0000","last_became_active":"2024-09-19T07:50:14.700547+0000","last_became_peered":"2024-09-19T07:50:14.700547+0000","last_unstale":"2024-09-19T07:50:17.143734+0000","last_undegraded":"2024-09-19T07:50:17.143734+0000","last_fullsized":"2024-09-19T07:50:17.143734+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:24.868698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702186+0000","last_change":"2024-09-19T07:50:15.703140+0000","last_active":"2024-09-19T07:50:16.702186+0000","last_peered":"2024-09-19T07:50:16.702186+0000","last_clean":"2024-09-19T07:50:16.702186+0000","last_became_active":"2024-09-19T07:50:15.702858+0000","last_became_peered":"2024-09-19T07:50:15.702858+0000","last_unstale":"2024-09-19T07:50:16.702186+0000","last_undegraded":"2024-09-19T07:50:16.702186+0000","last_fullsized":"2024-09-19T07:50:16.702186+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:58.431685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685235+0000","last_change":"2024-09-19T07:50:13.685235+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685235+0000","last_undegraded":"2024-09-19T07:50:13.685235+0000","last_fullsized":"2024-09-19T07:50:13.685235+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683088+0000","last_change":"2024-09-19T07:50:13.683088+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683088+0000","last_undegraded":"2024-09-19T07:50:13.683088+0000","last_fullsized":"2024-09-19T07:50:13.683088+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143660+0000","last_change":"2024-09-19T07:50:14.699921+0000","last_active":"2024-09-19T07:50:17.143660+0000","last_peered":"2024-09-19T07:50:17.143660+0000","last_clean":"2024-09-19T07:50:17.143660+0000","last_became_active":"2024-09-19T07:50:14.699623+0000","last_became_peered":"2024-09-19T07:50:14.699623+0000","last_unstale":"2024-09-19T07:50:17.143660+0000","last_undegraded":"2024-09-19T07:50:17.143660+0000","last_fullsized":"2024-09-19T07:50:17.143660+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:30.030459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143607+0000","last_change":"2024-09-19T07:50:15.724736+0000","last_active":"2024-09-19T07:50:17.143607+0000","last_peered":"2024-09-19T07:50:17.143607+0000","last_clean":"2024-09-19T07:50:17.143607+0000","last_became_active":"2024-09-19T07:50:15.724415+0000","last_became_peered":"2024-09-19T07:50:15.724415+0000","last_unstale":"2024-09-19T07:50:17.143607+0000","last_undegraded":"2024-09-19T07:50:17.143607+0000","last_fullsized":"2024-09-19T07:50:17.143607+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:42.319170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685114+0000","last_change":"2024-09-19T07:50:13.685114+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685114+0000","last_undegraded":"2024-09-19T07:50:13.685114+0000","last_fullsized":"2024-09-19T07:50:13.685114+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702571+0000","last_change":"2024-09-19T07:50:15.701962+0000","last_active":"2024-09-19T07:50:16.702571+0000","last_peered":"2024-09-19T07:50:16.702571+0000","last_clean":"2024-09-19T07:50:16.702571+0000","last_became_active":"2024-09-19T07:50:15.701397+0000","last_became_peered":"2024-09-19T07:50:15.701397+0000","last_unstale":"2024-09-19T07:50:16.702571+0000","last_undegraded":"2024-09-19T07:50:16.702571+0000","last_fullsized":"2024-09-19T07:50:16.702571+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:09.672535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.686167+0000","last_change":"2024-09-19T07:50:13.686167+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.686167+0000","last_undegraded":"2024-09-19T07:50:13.686167+0000","last_fullsized":"2024-09-19T07:50:13.686167+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702370+0000","last_change":"2024-09-19T07:50:15.700668+0000","last_active":"2024-09-19T07:50:16.702370+0000","last_peered":"2024-09-19T07:50:16.702370+0000","last_clean":"2024-09-19T07:50:16.702370+0000","last_became_active":"2024-09-19T07:50:15.700376+0000","last_became_peered":"2024-09-19T07:50:15.700376+0000","last_unstale":"2024-09-19T07:50:16.702370+0000","last_undegraded":"2024-09-19T07:50:16.702370+0000","last_fullsized":"2024-09-19T07:50:16.702370+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:40.316646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685384+0000","last_change":"2024-09-19T07:50:13.685384+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685384+0000","last_undegraded":"2024-09-19T07:50:13.685384+0000","last_fullsized":"2024-09-19T07:50:13.685384+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.686136+0000","last_change":"2024-09-19T07:50:13.686136+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.686136+0000","last_undegraded":"2024-09-19T07:50:13.686136+0000","last_fullsized":"2024-09-19T07:50:13.686136+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.721727+0000","last_change":"2024-09-19T07:50:15.719730+0000","last_active":"2024-09-19T07:50:15.721727+0000","last_peered":"2024-09-19T07:50:15.721727+0000","last_clean":"2024-09-19T07:50:15.721727+0000","last_became_active":"2024-09-19T07:50:15.719453+0000","last_became_peered":"2024-09-19T07:50:15.719453+0000","last_unstale":"2024-09-19T07:50:15.721727+0000","last_undegraded":"2024-09-19T07:50:15.721727+0000","last_fullsized":"2024-09-19T07:50:15.721727+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:43.586941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702027+0000","last_change":"2024-09-19T07:50:14.700632+0000","last_active":"2024-09-19T07:50:16.702027+0000","last_peered":"2024-09-19T07:50:16.702027+0000","last_clean":"2024-09-19T07:50:16.702027+0000","last_became_active":"2024-09-19T07:50:14.700366+0000","last_became_peered":"2024-09-19T07:50:14.700366+0000","last_unstale":"2024-09-19T07:50:16.702027+0000","last_undegraded":"2024-09-19T07:50:16.702027+0000","last_fullsized":"2024-09-19T07:50:16.702027+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:40:38.108015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.709182+0000","last_change":"2024-09-19T07:50:15.707603+0000","last_active":"2024-09-19T07:50:15.709182+0000","last_peered":"2024-09-19T07:50:15.709182+0000","last_clean":"2024-09-19T07:50:15.709182+0000","last_became_active":"2024-09-19T07:50:15.707301+0000","last_became_peered":"2024-09-19T07:50:15.707301+0000","last_unstale":"2024-09-19T07:50:15.709182+0000","last_undegraded":"2024-09-19T07:50:15.709182+0000","last_fullsized":"2024-09-19T07:50:15.709182+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:35.633861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"92'8","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702406+0000","last_change":"2024-09-19T07:50:14.698635+0000","last_active":"2024-09-19T07:50:16.702406+0000","last_peered":"2024-09-19T07:50:16.702406+0000","last_clean":"2024-09-19T07:50:16.702406+0000","last_became_active":"2024-09-19T07:50:14.698359+0000","last_became_peered":"2024-09-19T07:50:14.698359+0000","last_unstale":"2024-09-19T07:50:16.702406+0000","last_undegraded":"2024-09-19T07:50:16.702406+0000","last_fullsized":"2024-09-19T07:50:16.702406+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:57.251575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.687900+0000","last_change":"2024-09-19T07:50:13.687900+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.687900+0000","last_undegraded":"2024-09-19T07:50:13.687900+0000","last_fullsized":"2024-09-19T07:50:13.687900+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685677+0000","last_change":"2024-09-19T07:50:13.685677+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685677+0000","last_undegraded":"2024-09-19T07:50:13.685677+0000","last_fullsized":"2024-09-19T07:50:13.685677+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.684538+0000","last_change":"2024-09-19T07:50:13.684538+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.684538+0000","last_undegraded":"2024-09-19T07:50:13.684538+0000","last_fullsized":"2024-09-19T07:50:13.684538+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683215+0000","last_change":"2024-09-19T07:50:13.683215+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683215+0000","last_undegraded":"2024-09-19T07:50:13.683215+0000","last_fullsized":"2024-09-19T07:50:13.683215+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.693828+0000","last_change":"2024-09-19T07:50:14.695669+0000","last_active":"2024-09-19T07:50:15.693828+0000","last_peered":"2024-09-19T07:50:15.693828+0000","last_clean":"2024-09-19T07:50:15.693828+0000","last_became_active":"2024-09-19T07:50:14.695399+0000","last_became_peered":"2024-09-19T07:50:14.695399+0000","last_unstale":"2024-09-19T07:50:15.693828+0000","last_undegraded":"2024-09-19T07:50:15.693828+0000","last_fullsized":"2024-09-19T07:50:15.693828+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:53.625672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683418+0000","last_change":"2024-09-19T07:50:13.683418+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683418+0000","last_undegraded":"2024-09-19T07:50:13.683418+0000","last_fullsized":"2024-09-19T07:50:13.683418+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.693599+0000","last_change":"2024-09-19T07:50:14.705188+0000","last_active":"2024-09-19T07:50:15.693599+0000","last_peered":"2024-09-19T07:50:15.693599+0000","last_clean":"2024-09-19T07:50:15.693599+0000","last_became_active":"2024-09-19T07:50:14.704804+0000","last_became_peered":"2024-09-19T07:50:14.704804+0000","last_unstale":"2024-09-19T07:50:15.693599+0000","last_undegraded":"2024-09-19T07:50:15.693599+0000","last_fullsized":"2024-09-19T07:50:15.693599+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:49.332075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.685337+0000","last_change":"2024-09-19T07:50:13.685337+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.685337+0000","last_undegraded":"2024-09-19T07:50:13.685337+0000","last_fullsized":"2024-09-19T07:50:13.685337+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683158+0000","last_change":"2024-09-19T07:50:13.683158+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683158+0000","last_undegraded":"2024-09-19T07:50:13.683158+0000","last_fullsized":"2024-09-19T07:50:13.683158+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683021+0000","last_change":"2024-09-19T07:50:13.683021+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683021+0000","last_undegraded":"2024-09-19T07:50:13.683021+0000","last_fullsized":"2024-09-19T07:50:13.683021+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.687478+0000","last_change":"2024-09-19T07:50:13.687478+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.687478+0000","last_undegraded":"2024-09-19T07:50:13.687478+0000","last_fullsized":"2024-09-19T07:50:13.687478+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-19T07:50:15.721498+0000","last_change":"2024-09-19T07:50:15.713492+0000","last_active":"2024-09-19T07:50:15.721498+0000","last_peered":"2024-09-19T07:50:15.721498+0000","last_clean":"2024-09-19T07:50:15.721498+0000","last_became_active":"2024-09-19T07:50:15.713228+0000","last_became_peered":"2024-09-19T07:50:15.713228+0000","last_unstale":"2024-09-19T07:50:15.721498+0000","last_undegraded":"2024-09-19T07:50:15.721498+0000","last_fullsized":"2024-09-19T07:50:15.721498+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:21.796433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.686181+0000","last_change":"2024-09-19T07:50:13.686181+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.686181+0000","last_undegraded":"2024-09-19T07:50:13.686181+0000","last_fullsized":"2024-09-19T07:50:13.686181+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143615+0000","last_change":"2024-09-19T07:50:15.708869+0000","last_active":"2024-09-19T07:50:17.143615+0000","last_peered":"2024-09-19T07:50:17.143615+0000","last_clean":"2024-09-19T07:50:17.143615+0000","last_became_active":"2024-09-19T07:50:15.708595+0000","last_became_peered":"2024-09-19T07:50:15.708595+0000","last_unstale":"2024-09-19T07:50:17.143615+0000","last_undegraded":"2024-09-19T07:50:17.143615+0000","last_fullsized":"2024-09-19T07:50:17.143615+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:55.546102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.683021+0000","last_change":"2024-09-19T07:50:13.683021+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.683021+0000","last_undegraded":"2024-09-19T07:50:13.683021+0000","last_fullsized":"2024-09-19T07:50:13.683021+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.687559+0000","last_change":"2024-09-19T07:50:13.687559+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.687559+0000","last_undegraded":"2024-09-19T07:50:13.687559+0000","last_fullsized":"2024-09-19T07:50:13.687559+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-19T07:50:13.686119+0000","last_change":"2024-09-19T07:50:13.686119+0000","last_active":"2024-09-19T07:50:13.674730+0000","last_peered":"2024-09-19T07:50:13.674730+0000","last_clean":"2024-09-19T07:50:13.674730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:13.686119+0000","last_undegraded":"2024-09-19T07:50:13.686119+0000","last_fullsized":"2024-09-19T07:50:13.686119+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:50:14.680382+0000","last_change":"2024-09-19T07:50:14.680382+0000","last_active":"2024-09-19T07:50:14.680382+0000","last_peered":"2024-09-19T07:50:14.680382+0000","last_clean":"2024-09-19T07:50:14.680382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:50:14.680382+0000","last_undegraded":"2024-09-19T07:50:14.680382+0000","last_fullsized":"2024-09-19T07:50:14.680382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702502+0000","last_change":"2024-09-19T07:50:14.705646+0000","last_active":"2024-09-19T07:50:16.702502+0000","last_peered":"2024-09-19T07:50:16.702502+0000","last_clean":"2024-09-19T07:50:16.702502+0000","last_became_active":"2024-09-19T07:50:14.705368+0000","last_became_peered":"2024-09-19T07:50:14.705368+0000","last_unstale":"2024-09-19T07:50:16.702502+0000","last_undegraded":"2024-09-19T07:50:16.702502+0000","last_fullsized":"2024-09-19T07:50:16.702502+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:29.229270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":294,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-19T07:50:13.686575+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:50:13.686575+0000","last_peered":"2024-09-19T07:50:13.686575+0000","last_clean":"2024-09-19T07:50:13.686575+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:50:13.686575+0000","last_undegraded":"2024-09-19T07:50:13.686575+0000","last_fullsized":"2024-09-19T07:50:13.686575+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":44,"acting":44,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509531,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":103632,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":101950,"kb_avail":94268208,"statfs":{"total":96636764160,"available":96530644992,"internally_reserved":0,"allocated":1695744,"data_stored":1093834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":104397258},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509530,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100308,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":94271532,"statfs":{"total":96636764160,"available":96534048768,"internally_reserved":0,"allocated":1101824,"data_stored":503466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":101579206},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50312,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321528,"statfs":{"total":96636764160,"available":96585244672,"internally_reserved":0,"allocated":1101824,"data_stored":503466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86348,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285492,"statfs":{"total":96636764160,"available":96548343808,"internally_reserved":0,"allocated":1101824,"data_stored":503466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509530,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100812,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271028,"statfs":{"total":96636764160,"available":96533532672,"internally_reserved":0,"allocated":1695744,"data_stored":1093834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542235,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130800,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94241040,"statfs":{"total":96636764160,"available":96502824960,"internally_reserved":0,"allocated":1138688,"data_stored":523992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542235,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94776,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94277064,"statfs":{"total":96636764160,"available":96539713536,"internally_reserved":0,"allocated":1142784,"data_stored":524062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104420,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267420,"statfs":{"total":96636764160,"available":96529838080,"internally_reserved":0,"allocated":1126400,"data_stored":517300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:50:21.678 INFO:tasks.ceph.mds.a.smithi016.stderr:2024-09-19T07:50:21.678+0000 7f36b8b58640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T07:50:23.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:50:23.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.300+0000 7f2a7de1e640 1 Processor -- start 2024-09-19T07:50:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.300+0000 7f2a7de1e640 1 -- start start 2024-09-19T07:50:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.301+0000 7f2a7de1e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2a7815e2a0 0x7f2a7806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.301+0000 7f2a7de1e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a780d1880 con 0x7f2a7815e680 2024-09-19T07:50:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.301+0000 7f2a7de1e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a780d1b60 con 0x7f2a7806d960 2024-09-19T07:50:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.301+0000 7f2a7de1e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2a780cf2d0 con 0x7f2a7815e2a0 2024-09-19T07:50:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.301+0000 7f2a6effd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2a7815e680 0x7f2a780cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41218/0 (socket says 172.21.15.16:41218) 2024-09-19T07:50:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.301+0000 7f2a6effd640 1 -- 172.21.15.16:0/1551701135 learned_addr learned my addr 172.21.15.16:0/1551701135 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.301+0000 7f2a777fe640 1 --2- 172.21.15.16:0/1551701135 >> v2:172.21.15.146:3300/0 conn(0x7f2a7815e2a0 0x7f2a7806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.301+0000 7f2a76ffd640 1 -- 172.21.15.16:0/1551701135 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2806787994 0 0) 0x7f2a780d1880 con 0x7f2a7815e680 2024-09-19T07:50:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.301+0000 7f2a76ffd640 1 -- 172.21.15.16:0/1551701135 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a64003520 con 0x7f2a7815e680 2024-09-19T07:50:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a76ffd640 1 -- 172.21.15.16:0/1551701135 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 141364699 0 0) 0x7f2a64003520 con 0x7f2a7815e680 2024-09-19T07:50:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a76ffd640 1 -- 172.21.15.16:0/1551701135 >> v2:172.21.15.146:3300/0 conn(0x7f2a7815e2a0 msgr2=0x7f2a7806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a76ffd640 1 --2- 172.21.15.16:0/1551701135 >> v2:172.21.15.146:3300/0 conn(0x7f2a7815e2a0 0x7f2a7806d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a76ffd640 1 -- 172.21.15.16:0/1551701135 >> v1:172.21.15.146:6789/0 conn(0x7f2a7806d960 legacy=0x7f2a780d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a76ffd640 1 -- 172.21.15.16:0/1551701135 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a780cf5b0 con 0x7f2a7815e680 2024-09-19T07:50:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a76ffd640 1 -- 172.21.15.16:0/1551701135 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f2a60001670 con 0x7f2a7815e680 2024-09-19T07:50:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a76ffd640 1 -- 172.21.15.16:0/1551701135 >> v1:172.21.15.16:6789/0 conn(0x7f2a7815e680 legacy=0x7f2a780cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a76ffd640 1 --2- 172.21.15.16:0/1551701135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a640039b0 0x7f2a64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a76ffd640 1 --2- 172.21.15.16:0/1551701135 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a64004800 0x7f2a64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a76ffd640 1 --2- 172.21.15.16:0/1551701135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a640051c0 0x7f2a6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a76ffd640 1 -- 172.21.15.16:0/1551701135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2a780cf2d0 con 0x7f2a640039b0 2024-09-19T07:50:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a76ffd640 1 -- 172.21.15.16:0/1551701135 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6400c060 con 0x7f2a640051c0 2024-09-19T07:50:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a76ffd640 1 -- 172.21.15.16:0/1551701135 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2a6400c260 con 0x7f2a64004800 2024-09-19T07:50:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a777fe640 1 --2- 172.21.15.16:0/1551701135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a640039b0 0x7f2a64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a77fff640 1 --2- 172.21.15.16:0/1551701135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a640051c0 0x7f2a6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a6effd640 1 --2- 172.21.15.16:0/1551701135 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a64004800 0x7f2a64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a777fe640 1 -- 172.21.15.16:0/1551701135 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a64004800 msgr2=0x7f2a64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a777fe640 1 --2- 172.21.15.16:0/1551701135 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a64004800 0x7f2a64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a777fe640 1 -- 172.21.15.16:0/1551701135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a640051c0 msgr2=0x7f2a6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a777fe640 1 --2- 172.21.15.16:0/1551701135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a640051c0 0x7f2a6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a777fe640 1 -- 172.21.15.16:0/1551701135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a64003520 con 0x7f2a640039b0 2024-09-19T07:50:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a77fff640 1 -- 172.21.15.16:0/1551701135 reap_dead start 2024-09-19T07:50:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.302+0000 7f2a6effd640 1 --2- 172.21.15.16:0/1551701135 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a64004800 0x7f2a64004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.303+0000 7f2a777fe640 1 --2- 172.21.15.16:0/1551701135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a640039b0 0x7f2a64003dd0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009ac0 tx=0x7f2a64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14e5532564141c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.303+0000 7f2a76ffd640 1 -- 172.21.15.16:0/1551701135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68036070 con 0x7f2a640039b0 2024-09-19T07:50:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.303+0000 7f2a76ffd640 1 -- 172.21.15.16:0/1551701135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a68031400 con 0x7f2a640039b0 2024-09-19T07:50:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.303+0000 7f2a76ffd640 1 -- 172.21.15.16:0/1551701135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68009690 con 0x7f2a640039b0 2024-09-19T07:50:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.304+0000 7f2a7de1e640 1 -- 172.21.15.16:0/1551701135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a640039b0 msgr2=0x7f2a64003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.304+0000 7f2a7de1e640 1 --2- 172.21.15.16:0/1551701135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a640039b0 0x7f2a64003dd0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009ac0 tx=0x7f2a64005830 comp rx=0 tx=0).stop 2024-09-19T07:50:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.304+0000 7f2a7de1e640 1 -- 172.21.15.16:0/1551701135 shutdown_connections 2024-09-19T07:50:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.304+0000 7f2a7de1e640 1 --2- 172.21.15.16:0/1551701135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a640039b0 0x7f2a64003dd0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.304+0000 7f2a7de1e640 1 -- 172.21.15.16:0/1551701135 >> 172.21.15.16:0/1551701135 conn(0x7f2a7809d760 msgr2=0x7f2a7808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.304+0000 7f2a7de1e640 1 -- 172.21.15.16:0/1551701135 shutdown_connections 2024-09-19T07:50:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.304+0000 7f2a7de1e640 1 -- 172.21.15.16:0/1551701135 wait complete. 2024-09-19T07:50:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.305+0000 7f2a7de1e640 1 Processor -- start 2024-09-19T07:50:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.305+0000 7f2a7de1e640 1 -- start start 2024-09-19T07:50:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.306+0000 7f2a7de1e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a640039b0 0x7f2a781567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.306+0000 7f2a7de1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a780c7f10 0x7f2a78156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.306+0000 7f2a7de1e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a780cfce0 0x7f2a780d0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.306+0000 7f2a7de1e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7806d5a0 con 0x7f2a780c7f10 2024-09-19T07:50:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.306+0000 7f2a7de1e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7806d420 con 0x7f2a780cfce0 2024-09-19T07:50:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.306+0000 7f2a7de1e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2a7806d720 con 0x7f2a640039b0 2024-09-19T07:50:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.306+0000 7f2a6effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a780c7f10 0x7f2a78156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.306+0000 7f2a6effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a780c7f10 0x7f2a78156000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50528/0 (socket says 172.21.15.16:50528) 2024-09-19T07:50:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.306+0000 7f2a6effd640 1 -- 172.21.15.16:0/1707803096 learned_addr learned my addr 172.21.15.16:0/1707803096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.306+0000 7f2a77fff640 1 --2- 172.21.15.16:0/1707803096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a780cfce0 0x7f2a780d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.306+0000 7f2a777fe640 1 --2- 172.21.15.16:0/1707803096 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a640039b0 0x7f2a781567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.306+0000 7f2a6effd640 1 -- 172.21.15.16:0/1707803096 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a640039b0 msgr2=0x7f2a781567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.306+0000 7f2a6effd640 1 --2- 172.21.15.16:0/1707803096 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a640039b0 0x7f2a781567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.306+0000 7f2a6effd640 1 -- 172.21.15.16:0/1707803096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a780cfce0 msgr2=0x7f2a780d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.306+0000 7f2a6effd640 1 --2- 172.21.15.16:0/1707803096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a780cfce0 0x7f2a780d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.306+0000 7f2a6effd640 1 -- 172.21.15.16:0/1707803096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a780cd950 con 0x7f2a780c7f10 2024-09-19T07:50:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.306+0000 7f2a777fe640 1 --2- 172.21.15.16:0/1707803096 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a640039b0 0x7f2a781567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.306+0000 7f2a77fff640 1 --2- 172.21.15.16:0/1707803096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a780cfce0 0x7f2a780d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.307+0000 7f2a6effd640 1 --2- 172.21.15.16:0/1707803096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a780c7f10 0x7f2a78156000 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000e850 tx=0x7f2a6000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.307+0000 7f2a74ff9640 1 -- 172.21.15.16:0/1707803096 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6000c870 con 0x7f2a780c7f10 2024-09-19T07:50:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.307+0000 7f2a74ff9640 1 -- 172.21.15.16:0/1707803096 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a60018070 con 0x7f2a780c7f10 2024-09-19T07:50:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.307+0000 7f2a74ff9640 1 -- 172.21.15.16:0/1707803096 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6000cc10 con 0x7f2a780c7f10 2024-09-19T07:50:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.307+0000 7f2a7de1e640 1 -- 172.21.15.16:0/1707803096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a780b9f00 con 0x7f2a780c7f10 2024-09-19T07:50:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.307+0000 7f2a7de1e640 1 -- 172.21.15.16:0/1707803096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a78156540 con 0x7f2a780c7f10 2024-09-19T07:50:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.308+0000 7f2a7de1e640 1 -- 172.21.15.16:0/1707803096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a780cfad0 con 0x7f2a780c7f10 2024-09-19T07:50:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.308+0000 7f2a74ff9640 1 -- 172.21.15.16:0/1707803096 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2a60013420 con 0x7f2a780c7f10 2024-09-19T07:50:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.309+0000 7f2a74ff9640 1 --2- 172.21.15.16:0/1707803096 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2a5806c2e0 0x7f2a5806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.309+0000 7f2a74ff9640 1 -- 172.21.15.16:0/1707803096 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f2a6008b120 con 0x7f2a780c7f10 2024-09-19T07:50:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.309+0000 7f2a777fe640 1 --2- 172.21.15.16:0/1707803096 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2a5806c2e0 0x7f2a5806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.310+0000 7f2a777fe640 1 --2- 172.21.15.16:0/1707803096 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2a5806c2e0 0x7f2a5806e7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2a68033000 tx=0x7f2a68008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.313+0000 7f2a74ff9640 1 -- 172.21.15.16:0/1707803096 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2a60059790 con 0x7f2a780c7f10 2024-09-19T07:50:23.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.485+0000 7f2a7de1e640 1 -- 172.21.15.16:0/1707803096 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2a7806cf90 con 0x7f2a5806c2e0 2024-09-19T07:50:23.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.492+0000 7f2a74ff9640 1 -- 172.21.15.16:0/1707803096 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252073 (secure 0 0 0) 0x7f2a7806cf90 con 0x7f2a5806c2e0 2024-09-19T07:50:23.495 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:23.496 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:50:23.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.495+0000 7f2a7de1e640 1 -- 172.21.15.16:0/1707803096 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2a5806c2e0 msgr2=0x7f2a5806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:23.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.495+0000 7f2a7de1e640 1 --2- 172.21.15.16:0/1707803096 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2a5806c2e0 0x7f2a5806e7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2a68033000 tx=0x7f2a68008030 comp rx=0 tx=0).stop 2024-09-19T07:50:23.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.495+0000 7f2a7de1e640 1 -- 172.21.15.16:0/1707803096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a780c7f10 msgr2=0x7f2a78156000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:23.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.495+0000 7f2a7de1e640 1 --2- 172.21.15.16:0/1707803096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a780c7f10 0x7f2a78156000 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000e850 tx=0x7f2a6000ed10 comp rx=0 tx=0).stop 2024-09-19T07:50:23.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.495+0000 7f2a7de1e640 1 -- 172.21.15.16:0/1707803096 shutdown_connections 2024-09-19T07:50:23.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.495+0000 7f2a7de1e640 1 --2- 172.21.15.16:0/1707803096 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2a5806c2e0 0x7f2a5806e7a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.495+0000 7f2a7de1e640 1 --2- 172.21.15.16:0/1707803096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a780cfce0 0x7f2a780d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.495+0000 7f2a7de1e640 1 --2- 172.21.15.16:0/1707803096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a780c7f10 0x7f2a78156000 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.495+0000 7f2a7de1e640 1 --2- 172.21.15.16:0/1707803096 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a640039b0 0x7f2a781567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.495+0000 7f2a7de1e640 1 -- 172.21.15.16:0/1707803096 >> 172.21.15.16:0/1707803096 conn(0x7f2a7809d760 msgr2=0x7f2a78161500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:23.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.495+0000 7f2a7de1e640 1 -- 172.21.15.16:0/1707803096 shutdown_connections 2024-09-19T07:50:23.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.495+0000 7f2a7de1e640 1 -- 172.21.15.16:0/1707803096 wait complete. 2024-09-19T07:50:23.506 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-09-19T07:50:22.557463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":750796,"kb_used_data":10120,"kb_used_omap":12,"kb_used_meta":740467,"kb_avail":754223924,"statfs":{"total":773094113280,"available":772325298176,"internally_reserved":0,"allocated":10362880,"data_stored":5377492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":758238768},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.866646"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143560+0000","last_change":"2024-09-19T07:50:15.716195+0000","last_active":"2024-09-19T07:50:17.143560+0000","last_peered":"2024-09-19T07:50:17.143560+0000","last_clean":"2024-09-19T07:50:17.143560+0000","last_became_active":"2024-09-19T07:50:15.715936+0000","last_became_peered":"2024-09-19T07:50:15.715936+0000","last_unstale":"2024-09-19T07:50:17.143560+0000","last_undegraded":"2024-09-19T07:50:17.143560+0000","last_fullsized":"2024-09-19T07:50:17.143560+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:50.198202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.144519+0000","last_change":"2024-09-19T07:50:15.701939+0000","last_active":"2024-09-19T07:50:17.144519+0000","last_peered":"2024-09-19T07:50:17.144519+0000","last_clean":"2024-09-19T07:50:17.144519+0000","last_became_active":"2024-09-19T07:50:15.701615+0000","last_became_peered":"2024-09-19T07:50:15.701615+0000","last_unstale":"2024-09-19T07:50:17.144519+0000","last_undegraded":"2024-09-19T07:50:17.144519+0000","last_fullsized":"2024-09-19T07:50:17.144519+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:12.161468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143176+0000","last_change":"2024-09-19T07:50:15.706380+0000","last_active":"2024-09-19T07:50:17.143176+0000","last_peered":"2024-09-19T07:50:17.143176+0000","last_clean":"2024-09-19T07:50:17.143176+0000","last_became_active":"2024-09-19T07:50:15.706055+0000","last_became_peered":"2024-09-19T07:50:15.706055+0000","last_unstale":"2024-09-19T07:50:17.143176+0000","last_undegraded":"2024-09-19T07:50:17.143176+0000","last_fullsized":"2024-09-19T07:50:17.143176+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:02.607702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702151+0000","last_change":"2024-09-19T07:50:15.714404+0000","last_active":"2024-09-19T07:50:16.702151+0000","last_peered":"2024-09-19T07:50:16.702151+0000","last_clean":"2024-09-19T07:50:16.702151+0000","last_became_active":"2024-09-19T07:50:15.714029+0000","last_became_peered":"2024-09-19T07:50:15.714029+0000","last_unstale":"2024-09-19T07:50:16.702151+0000","last_undegraded":"2024-09-19T07:50:16.702151+0000","last_fullsized":"2024-09-19T07:50:16.702151+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:52.950164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.705043+0000","last_change":"2024-09-19T07:50:15.708712+0000","last_active":"2024-09-19T07:50:16.705043+0000","last_peered":"2024-09-19T07:50:16.705043+0000","last_clean":"2024-09-19T07:50:16.705043+0000","last_became_active":"2024-09-19T07:50:15.707724+0000","last_became_peered":"2024-09-19T07:50:15.707724+0000","last_unstale":"2024-09-19T07:50:16.705043+0000","last_undegraded":"2024-09-19T07:50:16.705043+0000","last_fullsized":"2024-09-19T07:50:16.705043+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:41.144795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704422+0000","last_change":"2024-09-19T07:50:15.718733+0000","last_active":"2024-09-19T07:50:16.704422+0000","last_peered":"2024-09-19T07:50:16.704422+0000","last_clean":"2024-09-19T07:50:16.704422+0000","last_became_active":"2024-09-19T07:50:15.718480+0000","last_became_peered":"2024-09-19T07:50:15.718480+0000","last_unstale":"2024-09-19T07:50:16.704422+0000","last_undegraded":"2024-09-19T07:50:16.704422+0000","last_fullsized":"2024-09-19T07:50:16.704422+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:55.546102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143344+0000","last_change":"2024-09-19T07:50:15.702282+0000","last_active":"2024-09-19T07:50:17.143344+0000","last_peered":"2024-09-19T07:50:17.143344+0000","last_clean":"2024-09-19T07:50:17.143344+0000","last_became_active":"2024-09-19T07:50:15.701960+0000","last_became_peered":"2024-09-19T07:50:15.701960+0000","last_unstale":"2024-09-19T07:50:17.143344+0000","last_undegraded":"2024-09-19T07:50:17.143344+0000","last_fullsized":"2024-09-19T07:50:17.143344+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:36.594773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702212+0000","last_change":"2024-09-19T07:50:15.717130+0000","last_active":"2024-09-19T07:50:16.702212+0000","last_peered":"2024-09-19T07:50:16.702212+0000","last_clean":"2024-09-19T07:50:16.702212+0000","last_became_active":"2024-09-19T07:50:15.716820+0000","last_became_peered":"2024-09-19T07:50:15.716820+0000","last_unstale":"2024-09-19T07:50:16.702212+0000","last_undegraded":"2024-09-19T07:50:16.702212+0000","last_fullsized":"2024-09-19T07:50:16.702212+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:13.211000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.705023+0000","last_change":"2024-09-19T07:50:15.710035+0000","last_active":"2024-09-19T07:50:16.705023+0000","last_peered":"2024-09-19T07:50:16.705023+0000","last_clean":"2024-09-19T07:50:16.705023+0000","last_became_active":"2024-09-19T07:50:15.708603+0000","last_became_peered":"2024-09-19T07:50:15.708603+0000","last_unstale":"2024-09-19T07:50:16.705023+0000","last_undegraded":"2024-09-19T07:50:16.705023+0000","last_fullsized":"2024-09-19T07:50:16.705023+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:02.644041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143373+0000","last_change":"2024-09-19T07:50:15.717933+0000","last_active":"2024-09-19T07:50:17.143373+0000","last_peered":"2024-09-19T07:50:17.143373+0000","last_clean":"2024-09-19T07:50:17.143373+0000","last_became_active":"2024-09-19T07:50:15.717636+0000","last_became_peered":"2024-09-19T07:50:15.717636+0000","last_unstale":"2024-09-19T07:50:17.143373+0000","last_undegraded":"2024-09-19T07:50:17.143373+0000","last_fullsized":"2024-09-19T07:50:17.143373+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:30.588427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.705304+0000","last_change":"2024-09-19T07:50:15.715747+0000","last_active":"2024-09-19T07:50:16.705304+0000","last_peered":"2024-09-19T07:50:16.705304+0000","last_clean":"2024-09-19T07:50:16.705304+0000","last_became_active":"2024-09-19T07:50:15.715370+0000","last_became_peered":"2024-09-19T07:50:15.715370+0000","last_unstale":"2024-09-19T07:50:16.705304+0000","last_undegraded":"2024-09-19T07:50:16.705304+0000","last_fullsized":"2024-09-19T07:50:16.705304+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:54.391850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143086+0000","last_change":"2024-09-19T07:50:15.702296+0000","last_active":"2024-09-19T07:50:17.143086+0000","last_peered":"2024-09-19T07:50:17.143086+0000","last_clean":"2024-09-19T07:50:17.143086+0000","last_became_active":"2024-09-19T07:50:15.701479+0000","last_became_peered":"2024-09-19T07:50:15.701479+0000","last_unstale":"2024-09-19T07:50:17.143086+0000","last_undegraded":"2024-09-19T07:50:17.143086+0000","last_fullsized":"2024-09-19T07:50:17.143086+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:42:04.203050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702086+0000","last_change":"2024-09-19T07:50:15.700711+0000","last_active":"2024-09-19T07:50:16.702086+0000","last_peered":"2024-09-19T07:50:16.702086+0000","last_clean":"2024-09-19T07:50:16.702086+0000","last_became_active":"2024-09-19T07:50:15.700431+0000","last_became_peered":"2024-09-19T07:50:15.700431+0000","last_unstale":"2024-09-19T07:50:16.702086+0000","last_undegraded":"2024-09-19T07:50:16.702086+0000","last_fullsized":"2024-09-19T07:50:16.702086+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:55.660212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704622+0000","last_change":"2024-09-19T07:50:15.717199+0000","last_active":"2024-09-19T07:50:16.704622+0000","last_peered":"2024-09-19T07:50:16.704622+0000","last_clean":"2024-09-19T07:50:16.704622+0000","last_became_active":"2024-09-19T07:50:15.716146+0000","last_became_peered":"2024-09-19T07:50:15.716146+0000","last_unstale":"2024-09-19T07:50:16.704622+0000","last_undegraded":"2024-09-19T07:50:16.704622+0000","last_fullsized":"2024-09-19T07:50:16.704622+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:18.255612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702008+0000","last_change":"2024-09-19T07:50:15.714991+0000","last_active":"2024-09-19T07:50:16.702008+0000","last_peered":"2024-09-19T07:50:16.702008+0000","last_clean":"2024-09-19T07:50:16.702008+0000","last_became_active":"2024-09-19T07:50:15.714620+0000","last_became_peered":"2024-09-19T07:50:15.714620+0000","last_unstale":"2024-09-19T07:50:16.702008+0000","last_undegraded":"2024-09-19T07:50:16.702008+0000","last_fullsized":"2024-09-19T07:50:16.702008+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:59.294901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704112+0000","last_change":"2024-09-19T07:50:15.715764+0000","last_active":"2024-09-19T07:50:16.704112+0000","last_peered":"2024-09-19T07:50:16.704112+0000","last_clean":"2024-09-19T07:50:16.704112+0000","last_became_active":"2024-09-19T07:50:15.715283+0000","last_became_peered":"2024-09-19T07:50:15.715283+0000","last_unstale":"2024-09-19T07:50:16.704112+0000","last_undegraded":"2024-09-19T07:50:16.704112+0000","last_fullsized":"2024-09-19T07:50:16.704112+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:02.234544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704539+0000","last_change":"2024-09-19T07:50:15.724199+0000","last_active":"2024-09-19T07:50:16.704539+0000","last_peered":"2024-09-19T07:50:16.704539+0000","last_clean":"2024-09-19T07:50:16.704539+0000","last_became_active":"2024-09-19T07:50:15.723918+0000","last_became_peered":"2024-09-19T07:50:15.723918+0000","last_unstale":"2024-09-19T07:50:16.704539+0000","last_undegraded":"2024-09-19T07:50:16.704539+0000","last_fullsized":"2024-09-19T07:50:16.704539+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:25.464287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143509+0000","last_change":"2024-09-19T07:50:15.697733+0000","last_active":"2024-09-19T07:50:17.143509+0000","last_peered":"2024-09-19T07:50:17.143509+0000","last_clean":"2024-09-19T07:50:17.143509+0000","last_became_active":"2024-09-19T07:50:15.697453+0000","last_became_peered":"2024-09-19T07:50:15.697453+0000","last_unstale":"2024-09-19T07:50:17.143509+0000","last_undegraded":"2024-09-19T07:50:17.143509+0000","last_fullsized":"2024-09-19T07:50:17.143509+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:50.763269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.705196+0000","last_change":"2024-09-19T07:50:15.716339+0000","last_active":"2024-09-19T07:50:16.705196+0000","last_peered":"2024-09-19T07:50:16.705196+0000","last_clean":"2024-09-19T07:50:16.705196+0000","last_became_active":"2024-09-19T07:50:15.716082+0000","last_became_peered":"2024-09-19T07:50:15.716082+0000","last_unstale":"2024-09-19T07:50:16.705196+0000","last_undegraded":"2024-09-19T07:50:16.705196+0000","last_fullsized":"2024-09-19T07:50:16.705196+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:52.950164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702280+0000","last_change":"2024-09-19T07:50:15.706893+0000","last_active":"2024-09-19T07:50:16.702280+0000","last_peered":"2024-09-19T07:50:16.702280+0000","last_clean":"2024-09-19T07:50:16.702280+0000","last_became_active":"2024-09-19T07:50:15.704710+0000","last_became_peered":"2024-09-19T07:50:15.704710+0000","last_unstale":"2024-09-19T07:50:16.702280+0000","last_undegraded":"2024-09-19T07:50:16.702280+0000","last_fullsized":"2024-09-19T07:50:16.702280+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:33:41.955036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.705072+0000","last_change":"2024-09-19T07:50:15.709507+0000","last_active":"2024-09-19T07:50:16.705072+0000","last_peered":"2024-09-19T07:50:16.705072+0000","last_clean":"2024-09-19T07:50:16.705072+0000","last_became_active":"2024-09-19T07:50:15.708113+0000","last_became_peered":"2024-09-19T07:50:15.708113+0000","last_unstale":"2024-09-19T07:50:16.705072+0000","last_undegraded":"2024-09-19T07:50:16.705072+0000","last_fullsized":"2024-09-19T07:50:16.705072+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:19.463235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704599+0000","last_change":"2024-09-19T07:50:15.716928+0000","last_active":"2024-09-19T07:50:16.704599+0000","last_peered":"2024-09-19T07:50:16.704599+0000","last_clean":"2024-09-19T07:50:16.704599+0000","last_became_active":"2024-09-19T07:50:15.715787+0000","last_became_peered":"2024-09-19T07:50:15.715787+0000","last_unstale":"2024-09-19T07:50:16.704599+0000","last_undegraded":"2024-09-19T07:50:16.704599+0000","last_fullsized":"2024-09-19T07:50:16.704599+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:55.103360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143656+0000","last_change":"2024-09-19T07:50:15.716783+0000","last_active":"2024-09-19T07:50:17.143656+0000","last_peered":"2024-09-19T07:50:17.143656+0000","last_clean":"2024-09-19T07:50:17.143656+0000","last_became_active":"2024-09-19T07:50:15.716522+0000","last_became_peered":"2024-09-19T07:50:15.716522+0000","last_unstale":"2024-09-19T07:50:17.143656+0000","last_undegraded":"2024-09-19T07:50:17.143656+0000","last_fullsized":"2024-09-19T07:50:17.143656+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:19.463235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143012+0000","last_change":"2024-09-19T07:50:15.707589+0000","last_active":"2024-09-19T07:50:17.143012+0000","last_peered":"2024-09-19T07:50:17.143012+0000","last_clean":"2024-09-19T07:50:17.143012+0000","last_became_active":"2024-09-19T07:50:15.707303+0000","last_became_peered":"2024-09-19T07:50:15.707303+0000","last_unstale":"2024-09-19T07:50:17.143012+0000","last_undegraded":"2024-09-19T07:50:17.143012+0000","last_fullsized":"2024-09-19T07:50:17.143012+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:02.234544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.144290+0000","last_change":"2024-09-19T07:50:15.703780+0000","last_active":"2024-09-19T07:50:17.144290+0000","last_peered":"2024-09-19T07:50:17.144290+0000","last_clean":"2024-09-19T07:50:17.144290+0000","last_became_active":"2024-09-19T07:50:15.703126+0000","last_became_peered":"2024-09-19T07:50:15.703126+0000","last_unstale":"2024-09-19T07:50:17.144290+0000","last_undegraded":"2024-09-19T07:50:17.144290+0000","last_fullsized":"2024-09-19T07:50:17.144290+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:28.584622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.705321+0000","last_change":"2024-09-19T07:50:15.717810+0000","last_active":"2024-09-19T07:50:16.705321+0000","last_peered":"2024-09-19T07:50:16.705321+0000","last_clean":"2024-09-19T07:50:16.705321+0000","last_became_active":"2024-09-19T07:50:15.717539+0000","last_became_peered":"2024-09-19T07:50:15.717539+0000","last_unstale":"2024-09-19T07:50:16.705321+0000","last_undegraded":"2024-09-19T07:50:16.705321+0000","last_fullsized":"2024-09-19T07:50:16.705321+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:13.211000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143257+0000","last_change":"2024-09-19T07:50:15.700322+0000","last_active":"2024-09-19T07:50:17.143257+0000","last_peered":"2024-09-19T07:50:17.143257+0000","last_clean":"2024-09-19T07:50:17.143257+0000","last_became_active":"2024-09-19T07:50:15.700039+0000","last_became_peered":"2024-09-19T07:50:15.700039+0000","last_unstale":"2024-09-19T07:50:17.143257+0000","last_undegraded":"2024-09-19T07:50:17.143257+0000","last_fullsized":"2024-09-19T07:50:17.143257+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:43.608917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702348+0000","last_change":"2024-09-19T07:50:15.707450+0000","last_active":"2024-09-19T07:50:16.702348+0000","last_peered":"2024-09-19T07:50:16.702348+0000","last_clean":"2024-09-19T07:50:16.702348+0000","last_became_active":"2024-09-19T07:50:15.707187+0000","last_became_peered":"2024-09-19T07:50:15.707187+0000","last_unstale":"2024-09-19T07:50:16.702348+0000","last_undegraded":"2024-09-19T07:50:16.702348+0000","last_fullsized":"2024-09-19T07:50:16.702348+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:48.678322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143740+0000","last_change":"2024-09-19T07:50:15.699206+0000","last_active":"2024-09-19T07:50:17.143740+0000","last_peered":"2024-09-19T07:50:17.143740+0000","last_clean":"2024-09-19T07:50:17.143740+0000","last_became_active":"2024-09-19T07:50:15.698904+0000","last_became_peered":"2024-09-19T07:50:15.698904+0000","last_unstale":"2024-09-19T07:50:17.143740+0000","last_undegraded":"2024-09-19T07:50:17.143740+0000","last_fullsized":"2024-09-19T07:50:17.143740+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:03.771544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704875+0000","last_change":"2024-09-19T07:50:15.709018+0000","last_active":"2024-09-19T07:50:16.704875+0000","last_peered":"2024-09-19T07:50:16.704875+0000","last_clean":"2024-09-19T07:50:16.704875+0000","last_became_active":"2024-09-19T07:50:15.708648+0000","last_became_peered":"2024-09-19T07:50:15.708648+0000","last_unstale":"2024-09-19T07:50:16.704875+0000","last_undegraded":"2024-09-19T07:50:16.704875+0000","last_fullsized":"2024-09-19T07:50:16.704875+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:48.796535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143936+0000","last_change":"2024-09-19T07:50:15.718553+0000","last_active":"2024-09-19T07:50:17.143936+0000","last_peered":"2024-09-19T07:50:17.143936+0000","last_clean":"2024-09-19T07:50:17.143936+0000","last_became_active":"2024-09-19T07:50:15.718269+0000","last_became_peered":"2024-09-19T07:50:15.718269+0000","last_unstale":"2024-09-19T07:50:17.143936+0000","last_undegraded":"2024-09-19T07:50:17.143936+0000","last_fullsized":"2024-09-19T07:50:17.143936+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:12.594351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704128+0000","last_change":"2024-09-19T07:50:15.715541+0000","last_active":"2024-09-19T07:50:16.704128+0000","last_peered":"2024-09-19T07:50:16.704128+0000","last_clean":"2024-09-19T07:50:16.704128+0000","last_became_active":"2024-09-19T07:50:15.714926+0000","last_became_peered":"2024-09-19T07:50:15.714926+0000","last_unstale":"2024-09-19T07:50:16.704128+0000","last_undegraded":"2024-09-19T07:50:16.704128+0000","last_fullsized":"2024-09-19T07:50:16.704128+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:02.607702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704438+0000","last_change":"2024-09-19T07:50:15.714120+0000","last_active":"2024-09-19T07:50:16.704438+0000","last_peered":"2024-09-19T07:50:16.704438+0000","last_clean":"2024-09-19T07:50:16.704438+0000","last_became_active":"2024-09-19T07:50:15.711960+0000","last_became_peered":"2024-09-19T07:50:15.711960+0000","last_unstale":"2024-09-19T07:50:16.704438+0000","last_undegraded":"2024-09-19T07:50:16.704438+0000","last_fullsized":"2024-09-19T07:50:16.704438+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:14:16.662615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"92'4","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143727+0000","last_change":"2024-09-19T07:50:14.702723+0000","last_active":"2024-09-19T07:50:17.143727+0000","last_peered":"2024-09-19T07:50:17.143727+0000","last_clean":"2024-09-19T07:50:17.143727+0000","last_became_active":"2024-09-19T07:50:14.702383+0000","last_became_peered":"2024-09-19T07:50:14.702383+0000","last_unstale":"2024-09-19T07:50:17.143727+0000","last_undegraded":"2024-09-19T07:50:17.143727+0000","last_fullsized":"2024-09-19T07:50:17.143727+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:40:38.108015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704400+0000","last_change":"2024-09-19T07:50:15.713152+0000","last_active":"2024-09-19T07:50:16.704400+0000","last_peered":"2024-09-19T07:50:16.704400+0000","last_clean":"2024-09-19T07:50:16.704400+0000","last_became_active":"2024-09-19T07:50:15.712577+0000","last_became_peered":"2024-09-19T07:50:15.712577+0000","last_unstale":"2024-09-19T07:50:16.704400+0000","last_undegraded":"2024-09-19T07:50:16.704400+0000","last_fullsized":"2024-09-19T07:50:16.704400+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:42.319170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.705186+0000","last_change":"2024-09-19T07:50:14.709537+0000","last_active":"2024-09-19T07:50:16.705186+0000","last_peered":"2024-09-19T07:50:16.705186+0000","last_clean":"2024-09-19T07:50:16.705186+0000","last_became_active":"2024-09-19T07:50:14.709265+0000","last_became_peered":"2024-09-19T07:50:14.709265+0000","last_unstale":"2024-09-19T07:50:16.705186+0000","last_undegraded":"2024-09-19T07:50:16.705186+0000","last_fullsized":"2024-09-19T07:50:16.705186+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:02.679623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143517+0000","last_change":"2024-09-19T07:50:15.701667+0000","last_active":"2024-09-19T07:50:17.143517+0000","last_peered":"2024-09-19T07:50:17.143517+0000","last_clean":"2024-09-19T07:50:17.143517+0000","last_became_active":"2024-09-19T07:50:15.701386+0000","last_became_peered":"2024-09-19T07:50:15.701386+0000","last_unstale":"2024-09-19T07:50:17.143517+0000","last_undegraded":"2024-09-19T07:50:17.143517+0000","last_fullsized":"2024-09-19T07:50:17.143517+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:17:41.185861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143745+0000","last_change":"2024-09-19T07:50:14.701173+0000","last_active":"2024-09-19T07:50:17.143745+0000","last_peered":"2024-09-19T07:50:17.143745+0000","last_clean":"2024-09-19T07:50:17.143745+0000","last_became_active":"2024-09-19T07:50:14.700907+0000","last_became_peered":"2024-09-19T07:50:14.700907+0000","last_unstale":"2024-09-19T07:50:17.143745+0000","last_undegraded":"2024-09-19T07:50:17.143745+0000","last_fullsized":"2024-09-19T07:50:17.143745+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:27.578970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704627+0000","last_change":"2024-09-19T07:50:15.711926+0000","last_active":"2024-09-19T07:50:16.704627+0000","last_peered":"2024-09-19T07:50:16.704627+0000","last_clean":"2024-09-19T07:50:16.704627+0000","last_became_active":"2024-09-19T07:50:15.711539+0000","last_became_peered":"2024-09-19T07:50:15.711539+0000","last_unstale":"2024-09-19T07:50:16.704627+0000","last_undegraded":"2024-09-19T07:50:16.704627+0000","last_fullsized":"2024-09-19T07:50:16.704627+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:22.207031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.144032+0000","last_change":"2024-09-19T07:50:14.707445+0000","last_active":"2024-09-19T07:50:17.144032+0000","last_peered":"2024-09-19T07:50:17.144032+0000","last_clean":"2024-09-19T07:50:17.144032+0000","last_became_active":"2024-09-19T07:50:14.707096+0000","last_became_peered":"2024-09-19T07:50:14.707096+0000","last_unstale":"2024-09-19T07:50:17.144032+0000","last_undegraded":"2024-09-19T07:50:17.144032+0000","last_fullsized":"2024-09-19T07:50:17.144032+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:29.229270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702304+0000","last_change":"2024-09-19T07:50:15.704536+0000","last_active":"2024-09-19T07:50:16.702304+0000","last_peered":"2024-09-19T07:50:16.702304+0000","last_clean":"2024-09-19T07:50:16.702304+0000","last_became_active":"2024-09-19T07:50:15.704223+0000","last_became_peered":"2024-09-19T07:50:15.704223+0000","last_unstale":"2024-09-19T07:50:16.702304+0000","last_undegraded":"2024-09-19T07:50:16.702304+0000","last_fullsized":"2024-09-19T07:50:16.702304+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:41.144795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.705133+0000","last_change":"2024-09-19T07:50:14.706656+0000","last_active":"2024-09-19T07:50:16.705133+0000","last_peered":"2024-09-19T07:50:16.705133+0000","last_clean":"2024-09-19T07:50:16.705133+0000","last_became_active":"2024-09-19T07:50:14.706240+0000","last_became_peered":"2024-09-19T07:50:14.706240+0000","last_unstale":"2024-09-19T07:50:16.705133+0000","last_undegraded":"2024-09-19T07:50:16.705133+0000","last_fullsized":"2024-09-19T07:50:16.705133+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:45.952455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143594+0000","last_change":"2024-09-19T07:50:15.715392+0000","last_active":"2024-09-19T07:50:17.143594+0000","last_peered":"2024-09-19T07:50:17.143594+0000","last_clean":"2024-09-19T07:50:17.143594+0000","last_became_active":"2024-09-19T07:50:15.715109+0000","last_became_peered":"2024-09-19T07:50:15.715109+0000","last_unstale":"2024-09-19T07:50:17.143594+0000","last_undegraded":"2024-09-19T07:50:17.143594+0000","last_fullsized":"2024-09-19T07:50:17.143594+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:54.391850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702372+0000","last_change":"2024-09-19T07:50:14.701473+0000","last_active":"2024-09-19T07:50:16.702372+0000","last_peered":"2024-09-19T07:50:16.702372+0000","last_clean":"2024-09-19T07:50:16.702372+0000","last_became_active":"2024-09-19T07:50:14.701196+0000","last_became_peered":"2024-09-19T07:50:14.701196+0000","last_unstale":"2024-09-19T07:50:16.702372+0000","last_undegraded":"2024-09-19T07:50:16.702372+0000","last_fullsized":"2024-09-19T07:50:16.702372+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:45.158919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704600+0000","last_change":"2024-09-19T07:50:15.718482+0000","last_active":"2024-09-19T07:50:16.704600+0000","last_peered":"2024-09-19T07:50:16.704600+0000","last_clean":"2024-09-19T07:50:16.704600+0000","last_became_active":"2024-09-19T07:50:15.717822+0000","last_became_peered":"2024-09-19T07:50:15.717822+0000","last_unstale":"2024-09-19T07:50:16.704600+0000","last_undegraded":"2024-09-19T07:50:16.704600+0000","last_fullsized":"2024-09-19T07:50:16.704600+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:20.277235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143734+0000","last_change":"2024-09-19T07:50:14.700842+0000","last_active":"2024-09-19T07:50:17.143734+0000","last_peered":"2024-09-19T07:50:17.143734+0000","last_clean":"2024-09-19T07:50:17.143734+0000","last_became_active":"2024-09-19T07:50:14.700547+0000","last_became_peered":"2024-09-19T07:50:14.700547+0000","last_unstale":"2024-09-19T07:50:17.143734+0000","last_undegraded":"2024-09-19T07:50:17.143734+0000","last_fullsized":"2024-09-19T07:50:17.143734+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:24.868698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702186+0000","last_change":"2024-09-19T07:50:15.703140+0000","last_active":"2024-09-19T07:50:16.702186+0000","last_peered":"2024-09-19T07:50:16.702186+0000","last_clean":"2024-09-19T07:50:16.702186+0000","last_became_active":"2024-09-19T07:50:15.702858+0000","last_became_peered":"2024-09-19T07:50:15.702858+0000","last_unstale":"2024-09-19T07:50:16.702186+0000","last_undegraded":"2024-09-19T07:50:16.702186+0000","last_fullsized":"2024-09-19T07:50:16.702186+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:58.431685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143893+0000","last_change":"2024-09-19T07:50:14.702120+0000","last_active":"2024-09-19T07:50:17.143893+0000","last_peered":"2024-09-19T07:50:17.143893+0000","last_clean":"2024-09-19T07:50:17.143893+0000","last_became_active":"2024-09-19T07:50:14.701841+0000","last_became_peered":"2024-09-19T07:50:14.701841+0000","last_unstale":"2024-09-19T07:50:17.143893+0000","last_undegraded":"2024-09-19T07:50:17.143893+0000","last_fullsized":"2024-09-19T07:50:17.143893+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:11.588699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.705121+0000","last_change":"2024-09-19T07:50:15.710780+0000","last_active":"2024-09-19T07:50:16.705121+0000","last_peered":"2024-09-19T07:50:16.705121+0000","last_clean":"2024-09-19T07:50:16.705121+0000","last_became_active":"2024-09-19T07:50:15.710469+0000","last_became_peered":"2024-09-19T07:50:15.710469+0000","last_unstale":"2024-09-19T07:50:16.705121+0000","last_undegraded":"2024-09-19T07:50:16.705121+0000","last_fullsized":"2024-09-19T07:50:16.705121+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:55.660212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.144410+0000","last_change":"2024-09-19T07:50:14.703596+0000","last_active":"2024-09-19T07:50:17.144410+0000","last_peered":"2024-09-19T07:50:17.144410+0000","last_clean":"2024-09-19T07:50:17.144410+0000","last_became_active":"2024-09-19T07:50:14.703311+0000","last_became_peered":"2024-09-19T07:50:14.703311+0000","last_unstale":"2024-09-19T07:50:17.144410+0000","last_undegraded":"2024-09-19T07:50:17.144410+0000","last_fullsized":"2024-09-19T07:50:17.144410+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:51.944511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143923+0000","last_change":"2024-09-19T07:50:15.706466+0000","last_active":"2024-09-19T07:50:17.143923+0000","last_peered":"2024-09-19T07:50:17.143923+0000","last_clean":"2024-09-19T07:50:17.143923+0000","last_became_active":"2024-09-19T07:50:15.706200+0000","last_became_peered":"2024-09-19T07:50:15.706200+0000","last_unstale":"2024-09-19T07:50:17.143923+0000","last_undegraded":"2024-09-19T07:50:17.143923+0000","last_fullsized":"2024-09-19T07:50:17.143923+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:53:59.032473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143660+0000","last_change":"2024-09-19T07:50:14.699921+0000","last_active":"2024-09-19T07:50:17.143660+0000","last_peered":"2024-09-19T07:50:17.143660+0000","last_clean":"2024-09-19T07:50:17.143660+0000","last_became_active":"2024-09-19T07:50:14.699623+0000","last_became_peered":"2024-09-19T07:50:14.699623+0000","last_unstale":"2024-09-19T07:50:17.143660+0000","last_undegraded":"2024-09-19T07:50:17.143660+0000","last_fullsized":"2024-09-19T07:50:17.143660+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:30.030459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143607+0000","last_change":"2024-09-19T07:50:15.724736+0000","last_active":"2024-09-19T07:50:17.143607+0000","last_peered":"2024-09-19T07:50:17.143607+0000","last_clean":"2024-09-19T07:50:17.143607+0000","last_became_active":"2024-09-19T07:50:15.724415+0000","last_became_peered":"2024-09-19T07:50:15.724415+0000","last_unstale":"2024-09-19T07:50:17.143607+0000","last_undegraded":"2024-09-19T07:50:17.143607+0000","last_fullsized":"2024-09-19T07:50:17.143607+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:42.319170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"92'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.705354+0000","last_change":"2024-09-19T07:50:14.711463+0000","last_active":"2024-09-19T07:50:16.705354+0000","last_peered":"2024-09-19T07:50:16.705354+0000","last_clean":"2024-09-19T07:50:16.705354+0000","last_became_active":"2024-09-19T07:50:14.711236+0000","last_became_peered":"2024-09-19T07:50:14.711236+0000","last_unstale":"2024-09-19T07:50:16.705354+0000","last_undegraded":"2024-09-19T07:50:16.705354+0000","last_fullsized":"2024-09-19T07:50:16.705354+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:29:55.559711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702571+0000","last_change":"2024-09-19T07:50:15.701962+0000","last_active":"2024-09-19T07:50:16.702571+0000","last_peered":"2024-09-19T07:50:16.702571+0000","last_clean":"2024-09-19T07:50:16.702571+0000","last_became_active":"2024-09-19T07:50:15.701397+0000","last_became_peered":"2024-09-19T07:50:15.701397+0000","last_unstale":"2024-09-19T07:50:16.702571+0000","last_undegraded":"2024-09-19T07:50:16.702571+0000","last_fullsized":"2024-09-19T07:50:16.702571+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:09.672535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"92'6","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704573+0000","last_change":"2024-09-19T07:50:14.704847+0000","last_active":"2024-09-19T07:50:16.704573+0000","last_peered":"2024-09-19T07:50:16.704573+0000","last_clean":"2024-09-19T07:50:16.704573+0000","last_became_active":"2024-09-19T07:50:14.704561+0000","last_became_peered":"2024-09-19T07:50:14.704561+0000","last_unstale":"2024-09-19T07:50:16.704573+0000","last_undegraded":"2024-09-19T07:50:16.704573+0000","last_fullsized":"2024-09-19T07:50:16.704573+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:07.082219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702370+0000","last_change":"2024-09-19T07:50:15.700668+0000","last_active":"2024-09-19T07:50:16.702370+0000","last_peered":"2024-09-19T07:50:16.702370+0000","last_clean":"2024-09-19T07:50:16.702370+0000","last_became_active":"2024-09-19T07:50:15.700376+0000","last_became_peered":"2024-09-19T07:50:15.700376+0000","last_unstale":"2024-09-19T07:50:16.702370+0000","last_undegraded":"2024-09-19T07:50:16.702370+0000","last_fullsized":"2024-09-19T07:50:16.702370+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:40.316646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.144288+0000","last_change":"2024-09-19T07:50:14.701768+0000","last_active":"2024-09-19T07:50:17.144288+0000","last_peered":"2024-09-19T07:50:17.144288+0000","last_clean":"2024-09-19T07:50:17.144288+0000","last_became_active":"2024-09-19T07:50:14.701507+0000","last_became_peered":"2024-09-19T07:50:14.701507+0000","last_unstale":"2024-09-19T07:50:17.144288+0000","last_undegraded":"2024-09-19T07:50:17.144288+0000","last_fullsized":"2024-09-19T07:50:17.144288+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:08.666882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704605+0000","last_change":"2024-09-19T07:50:15.706185+0000","last_active":"2024-09-19T07:50:16.704605+0000","last_peered":"2024-09-19T07:50:16.704605+0000","last_clean":"2024-09-19T07:50:16.704605+0000","last_became_active":"2024-09-19T07:50:15.705912+0000","last_became_peered":"2024-09-19T07:50:15.705912+0000","last_unstale":"2024-09-19T07:50:16.704605+0000","last_undegraded":"2024-09-19T07:50:16.704605+0000","last_fullsized":"2024-09-19T07:50:16.704605+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:13.143382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704547+0000","last_change":"2024-09-19T07:50:14.710132+0000","last_active":"2024-09-19T07:50:16.704547+0000","last_peered":"2024-09-19T07:50:16.704547+0000","last_clean":"2024-09-19T07:50:16.704547+0000","last_became_active":"2024-09-19T07:50:14.709860+0000","last_became_peered":"2024-09-19T07:50:14.709860+0000","last_unstale":"2024-09-19T07:50:16.704547+0000","last_undegraded":"2024-09-19T07:50:16.704547+0000","last_fullsized":"2024-09-19T07:50:16.704547+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:39.310994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143010+0000","last_change":"2024-09-19T07:50:15.719730+0000","last_active":"2024-09-19T07:50:17.143010+0000","last_peered":"2024-09-19T07:50:17.143010+0000","last_clean":"2024-09-19T07:50:17.143010+0000","last_became_active":"2024-09-19T07:50:15.719453+0000","last_became_peered":"2024-09-19T07:50:15.719453+0000","last_unstale":"2024-09-19T07:50:17.143010+0000","last_undegraded":"2024-09-19T07:50:17.143010+0000","last_fullsized":"2024-09-19T07:50:17.143010+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:20.025331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702027+0000","last_change":"2024-09-19T07:50:14.700632+0000","last_active":"2024-09-19T07:50:16.702027+0000","last_peered":"2024-09-19T07:50:16.702027+0000","last_clean":"2024-09-19T07:50:16.702027+0000","last_became_active":"2024-09-19T07:50:14.700366+0000","last_became_peered":"2024-09-19T07:50:14.700366+0000","last_unstale":"2024-09-19T07:50:16.702027+0000","last_undegraded":"2024-09-19T07:50:16.702027+0000","last_fullsized":"2024-09-19T07:50:16.702027+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:40:38.108015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143354+0000","last_change":"2024-09-19T07:50:15.707603+0000","last_active":"2024-09-19T07:50:17.143354+0000","last_peered":"2024-09-19T07:50:17.143354+0000","last_clean":"2024-09-19T07:50:17.143354+0000","last_became_active":"2024-09-19T07:50:15.707301+0000","last_became_peered":"2024-09-19T07:50:15.707301+0000","last_unstale":"2024-09-19T07:50:17.143354+0000","last_undegraded":"2024-09-19T07:50:17.143354+0000","last_fullsized":"2024-09-19T07:50:17.143354+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:34.324274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"92'8","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702406+0000","last_change":"2024-09-19T07:50:14.698635+0000","last_active":"2024-09-19T07:50:16.702406+0000","last_peered":"2024-09-19T07:50:16.702406+0000","last_clean":"2024-09-19T07:50:16.702406+0000","last_became_active":"2024-09-19T07:50:14.698359+0000","last_became_peered":"2024-09-19T07:50:14.698359+0000","last_unstale":"2024-09-19T07:50:16.702406+0000","last_undegraded":"2024-09-19T07:50:16.702406+0000","last_fullsized":"2024-09-19T07:50:16.702406+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:57.251575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704625+0000","last_change":"2024-09-19T07:50:15.712438+0000","last_active":"2024-09-19T07:50:16.704625+0000","last_peered":"2024-09-19T07:50:16.704625+0000","last_clean":"2024-09-19T07:50:16.704625+0000","last_became_active":"2024-09-19T07:50:15.711642+0000","last_became_peered":"2024-09-19T07:50:15.711642+0000","last_unstale":"2024-09-19T07:50:16.704625+0000","last_undegraded":"2024-09-19T07:50:16.704625+0000","last_fullsized":"2024-09-19T07:50:16.704625+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:59.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143620+0000","last_change":"2024-09-19T07:50:14.702005+0000","last_active":"2024-09-19T07:50:17.143620+0000","last_peered":"2024-09-19T07:50:17.143620+0000","last_clean":"2024-09-19T07:50:17.143620+0000","last_became_active":"2024-09-19T07:50:14.700196+0000","last_became_peered":"2024-09-19T07:50:14.700196+0000","last_unstale":"2024-09-19T07:50:17.143620+0000","last_undegraded":"2024-09-19T07:50:17.143620+0000","last_fullsized":"2024-09-19T07:50:17.143620+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:18.862777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143445+0000","last_change":"2024-09-19T07:50:15.703644+0000","last_active":"2024-09-19T07:50:17.143445+0000","last_peered":"2024-09-19T07:50:17.143445+0000","last_clean":"2024-09-19T07:50:17.143445+0000","last_became_active":"2024-09-19T07:50:15.703310+0000","last_became_peered":"2024-09-19T07:50:15.703310+0000","last_unstale":"2024-09-19T07:50:17.143445+0000","last_undegraded":"2024-09-19T07:50:17.143445+0000","last_fullsized":"2024-09-19T07:50:17.143445+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:02.644041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143423+0000","last_change":"2024-09-19T07:50:14.707449+0000","last_active":"2024-09-19T07:50:17.143423+0000","last_peered":"2024-09-19T07:50:17.143423+0000","last_clean":"2024-09-19T07:50:17.143423+0000","last_became_active":"2024-09-19T07:50:14.707175+0000","last_became_peered":"2024-09-19T07:50:14.707175+0000","last_unstale":"2024-09-19T07:50:17.143423+0000","last_undegraded":"2024-09-19T07:50:17.143423+0000","last_fullsized":"2024-09-19T07:50:17.143423+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:39.310994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.705130+0000","last_change":"2024-09-19T07:50:15.706417+0000","last_active":"2024-09-19T07:50:16.705130+0000","last_peered":"2024-09-19T07:50:16.705130+0000","last_clean":"2024-09-19T07:50:16.705130+0000","last_became_active":"2024-09-19T07:50:15.706136+0000","last_became_peered":"2024-09-19T07:50:15.706136+0000","last_unstale":"2024-09-19T07:50:16.705130+0000","last_undegraded":"2024-09-19T07:50:16.705130+0000","last_fullsized":"2024-09-19T07:50:16.705130+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:59.294901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.705360+0000","last_change":"2024-09-19T07:50:14.705867+0000","last_active":"2024-09-19T07:50:16.705360+0000","last_peered":"2024-09-19T07:50:16.705360+0000","last_clean":"2024-09-19T07:50:16.705360+0000","last_became_active":"2024-09-19T07:50:14.704680+0000","last_became_peered":"2024-09-19T07:50:14.704680+0000","last_unstale":"2024-09-19T07:50:16.705360+0000","last_undegraded":"2024-09-19T07:50:16.705360+0000","last_fullsized":"2024-09-19T07:50:16.705360+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:33.084280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704269+0000","last_change":"2024-09-19T07:50:15.721029+0000","last_active":"2024-09-19T07:50:16.704269+0000","last_peered":"2024-09-19T07:50:16.704269+0000","last_clean":"2024-09-19T07:50:16.704269+0000","last_became_active":"2024-09-19T07:50:15.719136+0000","last_became_peered":"2024-09-19T07:50:15.719136+0000","last_unstale":"2024-09-19T07:50:16.704269+0000","last_undegraded":"2024-09-19T07:50:16.704269+0000","last_fullsized":"2024-09-19T07:50:16.704269+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:50.763269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.144246+0000","last_change":"2024-09-19T07:50:14.706068+0000","last_active":"2024-09-19T07:50:17.144246+0000","last_peered":"2024-09-19T07:50:17.144246+0000","last_clean":"2024-09-19T07:50:17.144246+0000","last_became_active":"2024-09-19T07:50:14.705706+0000","last_became_peered":"2024-09-19T07:50:14.705706+0000","last_unstale":"2024-09-19T07:50:17.144246+0000","last_undegraded":"2024-09-19T07:50:17.144246+0000","last_fullsized":"2024-09-19T07:50:17.144246+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:58.289248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.144116+0000","last_change":"2024-09-19T07:50:15.702502+0000","last_active":"2024-09-19T07:50:17.144116+0000","last_peered":"2024-09-19T07:50:17.144116+0000","last_clean":"2024-09-19T07:50:17.144116+0000","last_became_active":"2024-09-19T07:50:15.702169+0000","last_became_peered":"2024-09-19T07:50:15.702169+0000","last_unstale":"2024-09-19T07:50:17.144116+0000","last_undegraded":"2024-09-19T07:50:17.144116+0000","last_fullsized":"2024-09-19T07:50:17.144116+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:58.187743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143109+0000","last_change":"2024-09-19T07:50:14.695669+0000","last_active":"2024-09-19T07:50:17.143109+0000","last_peered":"2024-09-19T07:50:17.143109+0000","last_clean":"2024-09-19T07:50:17.143109+0000","last_became_active":"2024-09-19T07:50:14.695399+0000","last_became_peered":"2024-09-19T07:50:14.695399+0000","last_unstale":"2024-09-19T07:50:17.143109+0000","last_undegraded":"2024-09-19T07:50:17.143109+0000","last_fullsized":"2024-09-19T07:50:17.143109+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:53.625672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.144424+0000","last_change":"2024-09-19T07:50:15.718195+0000","last_active":"2024-09-19T07:50:17.144424+0000","last_peered":"2024-09-19T07:50:17.144424+0000","last_clean":"2024-09-19T07:50:17.144424+0000","last_became_active":"2024-09-19T07:50:15.717907+0000","last_became_peered":"2024-09-19T07:50:15.717907+0000","last_unstale":"2024-09-19T07:50:17.144424+0000","last_undegraded":"2024-09-19T07:50:17.144424+0000","last_fullsized":"2024-09-19T07:50:17.144424+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:10.319810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.144153+0000","last_change":"2024-09-19T07:50:14.706725+0000","last_active":"2024-09-19T07:50:17.144153+0000","last_peered":"2024-09-19T07:50:17.144153+0000","last_clean":"2024-09-19T07:50:17.144153+0000","last_became_active":"2024-09-19T07:50:14.706331+0000","last_became_peered":"2024-09-19T07:50:14.706331+0000","last_unstale":"2024-09-19T07:50:17.144153+0000","last_undegraded":"2024-09-19T07:50:17.144153+0000","last_fullsized":"2024-09-19T07:50:17.144153+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:12.205347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704499+0000","last_change":"2024-09-19T07:50:15.712032+0000","last_active":"2024-09-19T07:50:16.704499+0000","last_peered":"2024-09-19T07:50:16.704499+0000","last_clean":"2024-09-19T07:50:16.704499+0000","last_became_active":"2024-09-19T07:50:15.711715+0000","last_became_peered":"2024-09-19T07:50:15.711715+0000","last_unstale":"2024-09-19T07:50:16.704499+0000","last_undegraded":"2024-09-19T07:50:16.704499+0000","last_fullsized":"2024-09-19T07:50:16.704499+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:26.933669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.142891+0000","last_change":"2024-09-19T07:50:14.705188+0000","last_active":"2024-09-19T07:50:17.142891+0000","last_peered":"2024-09-19T07:50:17.142891+0000","last_clean":"2024-09-19T07:50:17.142891+0000","last_became_active":"2024-09-19T07:50:14.704804+0000","last_became_peered":"2024-09-19T07:50:14.704804+0000","last_unstale":"2024-09-19T07:50:17.142891+0000","last_undegraded":"2024-09-19T07:50:17.142891+0000","last_fullsized":"2024-09-19T07:50:17.142891+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:49.332075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.144102+0000","last_change":"2024-09-19T07:50:15.701701+0000","last_active":"2024-09-19T07:50:17.144102+0000","last_peered":"2024-09-19T07:50:17.144102+0000","last_clean":"2024-09-19T07:50:17.144102+0000","last_became_active":"2024-09-19T07:50:15.701384+0000","last_became_peered":"2024-09-19T07:50:15.701384+0000","last_unstale":"2024-09-19T07:50:17.144102+0000","last_undegraded":"2024-09-19T07:50:17.144102+0000","last_fullsized":"2024-09-19T07:50:17.144102+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:41.144795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143793+0000","last_change":"2024-09-19T07:50:14.703782+0000","last_active":"2024-09-19T07:50:17.143793+0000","last_peered":"2024-09-19T07:50:17.143793+0000","last_clean":"2024-09-19T07:50:17.143793+0000","last_became_active":"2024-09-19T07:50:14.703494+0000","last_became_peered":"2024-09-19T07:50:14.703494+0000","last_unstale":"2024-09-19T07:50:17.143793+0000","last_undegraded":"2024-09-19T07:50:17.143793+0000","last_fullsized":"2024-09-19T07:50:17.143793+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:57.182091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704511+0000","last_change":"2024-09-19T07:50:15.712913+0000","last_active":"2024-09-19T07:50:16.704511+0000","last_peered":"2024-09-19T07:50:16.704511+0000","last_clean":"2024-09-19T07:50:16.704511+0000","last_became_active":"2024-09-19T07:50:15.712257+0000","last_became_peered":"2024-09-19T07:50:15.712257+0000","last_unstale":"2024-09-19T07:50:16.704511+0000","last_undegraded":"2024-09-19T07:50:16.704511+0000","last_fullsized":"2024-09-19T07:50:16.704511+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:24:36.525952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"93'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.705368+0000","last_change":"2024-09-19T07:50:14.711093+0000","last_active":"2024-09-19T07:50:16.705368+0000","last_peered":"2024-09-19T07:50:16.705368+0000","last_clean":"2024-09-19T07:50:16.705368+0000","last_became_active":"2024-09-19T07:50:14.710723+0000","last_became_peered":"2024-09-19T07:50:14.710723+0000","last_unstale":"2024-09-19T07:50:16.705368+0000","last_undegraded":"2024-09-19T07:50:16.705368+0000","last_fullsized":"2024-09-19T07:50:16.705368+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:49.142051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704518+0000","last_change":"2024-09-19T07:50:15.716329+0000","last_active":"2024-09-19T07:50:16.704518+0000","last_peered":"2024-09-19T07:50:16.704518+0000","last_clean":"2024-09-19T07:50:16.704518+0000","last_became_active":"2024-09-19T07:50:15.716008+0000","last_became_peered":"2024-09-19T07:50:15.716008+0000","last_unstale":"2024-09-19T07:50:16.704518+0000","last_undegraded":"2024-09-19T07:50:16.704518+0000","last_fullsized":"2024-09-19T07:50:16.704518+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:54.932377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.705427+0000","last_change":"2024-09-19T07:50:14.711089+0000","last_active":"2024-09-19T07:50:16.705427+0000","last_peered":"2024-09-19T07:50:16.705427+0000","last_clean":"2024-09-19T07:50:16.705427+0000","last_became_active":"2024-09-19T07:50:14.710627+0000","last_became_peered":"2024-09-19T07:50:14.710627+0000","last_unstale":"2024-09-19T07:50:16.705427+0000","last_undegraded":"2024-09-19T07:50:16.705427+0000","last_fullsized":"2024-09-19T07:50:16.705427+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:41.392228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.144485+0000","last_change":"2024-09-19T07:50:15.707326+0000","last_active":"2024-09-19T07:50:17.144485+0000","last_peered":"2024-09-19T07:50:17.144485+0000","last_clean":"2024-09-19T07:50:17.144485+0000","last_became_active":"2024-09-19T07:50:15.707059+0000","last_became_peered":"2024-09-19T07:50:15.707059+0000","last_unstale":"2024-09-19T07:50:17.144485+0000","last_undegraded":"2024-09-19T07:50:17.144485+0000","last_fullsized":"2024-09-19T07:50:17.144485+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:39:59.551922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.144158+0000","last_change":"2024-09-19T07:50:14.704170+0000","last_active":"2024-09-19T07:50:17.144158+0000","last_peered":"2024-09-19T07:50:17.144158+0000","last_clean":"2024-09-19T07:50:17.144158+0000","last_became_active":"2024-09-19T07:50:14.703560+0000","last_became_peered":"2024-09-19T07:50:14.703560+0000","last_unstale":"2024-09-19T07:50:17.144158+0000","last_undegraded":"2024-09-19T07:50:17.144158+0000","last_fullsized":"2024-09-19T07:50:17.144158+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:45.158919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.142896+0000","last_change":"2024-09-19T07:50:15.713492+0000","last_active":"2024-09-19T07:50:17.142896+0000","last_peered":"2024-09-19T07:50:17.142896+0000","last_clean":"2024-09-19T07:50:17.142896+0000","last_became_active":"2024-09-19T07:50:15.713228+0000","last_became_peered":"2024-09-19T07:50:15.713228+0000","last_unstale":"2024-09-19T07:50:17.142896+0000","last_undegraded":"2024-09-19T07:50:17.142896+0000","last_fullsized":"2024-09-19T07:50:17.142896+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:57:27.199585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704420+0000","last_change":"2024-09-19T07:50:14.711087+0000","last_active":"2024-09-19T07:50:16.704420+0000","last_peered":"2024-09-19T07:50:16.704420+0000","last_clean":"2024-09-19T07:50:16.704420+0000","last_became_active":"2024-09-19T07:50:14.710757+0000","last_became_peered":"2024-09-19T07:50:14.710757+0000","last_unstale":"2024-09-19T07:50:16.704420+0000","last_undegraded":"2024-09-19T07:50:16.704420+0000","last_fullsized":"2024-09-19T07:50:16.704420+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:08.666882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143615+0000","last_change":"2024-09-19T07:50:15.708869+0000","last_active":"2024-09-19T07:50:17.143615+0000","last_peered":"2024-09-19T07:50:17.143615+0000","last_clean":"2024-09-19T07:50:17.143615+0000","last_became_active":"2024-09-19T07:50:15.708595+0000","last_became_peered":"2024-09-19T07:50:15.708595+0000","last_unstale":"2024-09-19T07:50:17.143615+0000","last_undegraded":"2024-09-19T07:50:17.143615+0000","last_fullsized":"2024-09-19T07:50:17.143615+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:55.546102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.144282+0000","last_change":"2024-09-19T07:50:14.705313+0000","last_active":"2024-09-19T07:50:17.144282+0000","last_peered":"2024-09-19T07:50:17.144282+0000","last_clean":"2024-09-19T07:50:17.144282+0000","last_became_active":"2024-09-19T07:50:14.705027+0000","last_became_peered":"2024-09-19T07:50:14.705027+0000","last_unstale":"2024-09-19T07:50:17.144282+0000","last_undegraded":"2024-09-19T07:50:17.144282+0000","last_fullsized":"2024-09-19T07:50:17.144282+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:54.654559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143315+0000","last_change":"2024-09-19T07:50:15.716659+0000","last_active":"2024-09-19T07:50:17.143315+0000","last_peered":"2024-09-19T07:50:17.143315+0000","last_clean":"2024-09-19T07:50:17.143315+0000","last_became_active":"2024-09-19T07:50:15.716396+0000","last_became_peered":"2024-09-19T07:50:15.716396+0000","last_unstale":"2024-09-19T07:50:17.143315+0000","last_undegraded":"2024-09-19T07:50:17.143315+0000","last_fullsized":"2024-09-19T07:50:17.143315+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:48.796535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143726+0000","last_change":"2024-09-19T07:50:14.701715+0000","last_active":"2024-09-19T07:50:17.143726+0000","last_peered":"2024-09-19T07:50:17.143726+0000","last_clean":"2024-09-19T07:50:17.143726+0000","last_became_active":"2024-09-19T07:50:14.701132+0000","last_became_peered":"2024-09-19T07:50:14.701132+0000","last_unstale":"2024-09-19T07:50:17.143726+0000","last_undegraded":"2024-09-19T07:50:17.143726+0000","last_fullsized":"2024-09-19T07:50:17.143726+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:28.986240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.704365+0000","last_change":"2024-09-19T07:50:15.716397+0000","last_active":"2024-09-19T07:50:16.704365+0000","last_peered":"2024-09-19T07:50:16.704365+0000","last_clean":"2024-09-19T07:50:16.704365+0000","last_became_active":"2024-09-19T07:50:15.716131+0000","last_became_peered":"2024-09-19T07:50:15.716131+0000","last_unstale":"2024-09-19T07:50:16.704365+0000","last_undegraded":"2024-09-19T07:50:16.704365+0000","last_fullsized":"2024-09-19T07:50:16.704365+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:50.198202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143219+0000","last_change":"2024-09-19T07:50:14.708289+0000","last_active":"2024-09-19T07:50:17.143219+0000","last_peered":"2024-09-19T07:50:17.143219+0000","last_clean":"2024-09-19T07:50:17.143219+0000","last_became_active":"2024-09-19T07:50:14.707988+0000","last_became_peered":"2024-09-19T07:50:14.707988+0000","last_unstale":"2024-09-19T07:50:17.143219+0000","last_undegraded":"2024-09-19T07:50:17.143219+0000","last_fullsized":"2024-09-19T07:50:17.143219+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:07.082219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143726+0000","last_change":"2024-09-19T07:50:15.702261+0000","last_active":"2024-09-19T07:50:17.143726+0000","last_peered":"2024-09-19T07:50:17.143726+0000","last_clean":"2024-09-19T07:50:17.143726+0000","last_became_active":"2024-09-19T07:50:15.702001+0000","last_became_peered":"2024-09-19T07:50:15.702001+0000","last_unstale":"2024-09-19T07:50:17.143726+0000","last_undegraded":"2024-09-19T07:50:17.143726+0000","last_fullsized":"2024-09-19T07:50:17.143726+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:14.680382+0000","last_clean_scrub_stamp":"2024-09-19T07:50:14.680382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:36.224139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:16.702502+0000","last_change":"2024-09-19T07:50:14.705646+0000","last_active":"2024-09-19T07:50:16.702502+0000","last_peered":"2024-09-19T07:50:16.702502+0000","last_clean":"2024-09-19T07:50:16.702502+0000","last_became_active":"2024-09-19T07:50:14.705368+0000","last_became_peered":"2024-09-19T07:50:14.705368+0000","last_unstale":"2024-09-19T07:50:16.702502+0000","last_undegraded":"2024-09-19T07:50:16.702502+0000","last_fullsized":"2024-09-19T07:50:16.702502+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:50:13.674730+0000","last_clean_scrub_stamp":"2024-09-19T07:50:13.674730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:29.229270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":300,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-19T07:50:17.143996+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:50:17.143996+0000","last_peered":"2024-09-19T07:50:17.143996+0000","last_clean":"2024-09-19T07:50:17.143996+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:50:17.143996+0000","last_undegraded":"2024-09-19T07:50:17.143996+0000","last_fullsized":"2024-09-19T07:50:17.143996+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509532,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82816,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289024,"statfs":{"total":96636764160,"available":96551960576,"internally_reserved":0,"allocated":1740800,"data_stored":1114416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509531,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100352,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":94271488,"statfs":{"total":96636764160,"available":96534003712,"internally_reserved":0,"allocated":1159168,"data_stored":526208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":101579206},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509532,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50352,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321488,"statfs":{"total":96636764160,"available":96585203712,"internally_reserved":0,"allocated":1138688,"data_stored":523992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509532,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86392,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285448,"statfs":{"total":96636764160,"available":96548298752,"internally_reserved":0,"allocated":1146880,"data_stored":524152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509531,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100872,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94270968,"statfs":{"total":96636764160,"available":96533471232,"internally_reserved":0,"allocated":1757184,"data_stored":1116678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542235,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130800,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94241040,"statfs":{"total":96636764160,"available":96502824960,"internally_reserved":0,"allocated":1138688,"data_stored":523992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542235,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94776,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94277064,"statfs":{"total":96636764160,"available":96539713536,"internally_reserved":0,"allocated":1142784,"data_stored":524062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104436,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267404,"statfs":{"total":96636764160,"available":96529821696,"internally_reserved":0,"allocated":1138688,"data_stored":523992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:50:23.510 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T07:50:23.511 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T07:50:23.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.655+0000 7f27feec3640 1 Processor -- start 2024-09-19T07:50:23.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.655+0000 7f27feec3640 1 -- start start 2024-09-19T07:50:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.656+0000 7f27feec3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f27f815e0a0 0x7f27f8162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.656+0000 7f27feec3640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f27f80b5280 0x7f27f80d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.656+0000 7f27feec3640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27f815f250 con 0x7f27f80d43e0 2024-09-19T07:50:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.656+0000 7f27feec3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f27f80d4d60 con 0x7f27f815e0a0 2024-09-19T07:50:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.656+0000 7f27feec3640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f27f815f530 con 0x7f27f80b5280 2024-09-19T07:50:23.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.656+0000 7f27fcc5f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f27f815e0a0 0x7f27f8162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:23.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.656+0000 7f27fcc5f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f27f815e0a0 0x7f27f8162a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50534/0 (socket says 172.21.15.16:50534) 2024-09-19T07:50:23.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.656+0000 7f27fcc5f640 1 -- 172.21.15.16:0/1541983001 learned_addr learned my addr 172.21.15.16:0/1541983001 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:23.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.656+0000 7f27fcc5f640 1 -- 172.21.15.16:0/1541983001 >> v2:172.21.15.146:3300/0 conn(0x7f27f80b5280 msgr2=0x7f27f80d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:23.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.656+0000 7f27fcc5f640 1 --2- 172.21.15.16:0/1541983001 >> v2:172.21.15.146:3300/0 conn(0x7f27f80b5280 0x7f27f80d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.656+0000 7f27fcc5f640 1 -- 172.21.15.16:0/1541983001 >> v1:172.21.15.146:6789/0 conn(0x7f27f80d43e0 legacy=0x7f27f80b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:23.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.656+0000 7f27fcc5f640 1 -- 172.21.15.16:0/1541983001 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f815f810 con 0x7f27f815e0a0 2024-09-19T07:50:23.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.657+0000 7f27fcc5f640 1 --2- 172.21.15.16:0/1541983001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27f815e0a0 0x7f27f8162a70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f27f809ddf0 tx=0x7f27ec02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496f28a59befc243 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:23.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.657+0000 7f27f77fe640 1 -- 172.21.15.16:0/1541983001 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec037070 con 0x7f27f815e0a0 2024-09-19T07:50:23.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.657+0000 7f27f77fe640 1 -- 172.21.15.16:0/1541983001 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27ec032440 con 0x7f27f815e0a0 2024-09-19T07:50:23.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.657+0000 7f27f77fe640 1 -- 172.21.15.16:0/1541983001 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec032810 con 0x7f27f815e0a0 2024-09-19T07:50:23.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.658+0000 7f27feec3640 1 -- 172.21.15.16:0/1541983001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27f815e0a0 msgr2=0x7f27f8162a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:23.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.658+0000 7f27feec3640 1 --2- 172.21.15.16:0/1541983001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27f815e0a0 0x7f27f8162a70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f27f809ddf0 tx=0x7f27ec02a230 comp rx=0 tx=0).stop 2024-09-19T07:50:23.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.658+0000 7f27feec3640 1 -- 172.21.15.16:0/1541983001 shutdown_connections 2024-09-19T07:50:23.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.658+0000 7f27feec3640 1 --2- 172.21.15.16:0/1541983001 >> v2:172.21.15.146:3300/0 conn(0x7f27f80b5280 0x7f27f80d4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.658+0000 7f27feec3640 1 --2- 172.21.15.16:0/1541983001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27f815e0a0 0x7f27f8162a70 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.658+0000 7f27feec3640 1 -- 172.21.15.16:0/1541983001 >> 172.21.15.16:0/1541983001 conn(0x7f27f809d770 msgr2=0x7f27f808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:23.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.658+0000 7f27feec3640 1 -- 172.21.15.16:0/1541983001 shutdown_connections 2024-09-19T07:50:23.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.658+0000 7f27feec3640 1 -- 172.21.15.16:0/1541983001 wait complete. 2024-09-19T07:50:23.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.659+0000 7f27feec3640 1 Processor -- start 2024-09-19T07:50:23.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.659+0000 7f27feec3640 1 -- start start 2024-09-19T07:50:23.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.660+0000 7f27feec3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27f80b5280 0x7f27f80b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:23.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.660+0000 7f27feec3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27f80d43e0 0x7f27f815cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:23.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.660+0000 7f27feec3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27f815e0a0 0x7f27f80d0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:23.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.660+0000 7f27feec3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27f815e600 con 0x7f27f80d43e0 2024-09-19T07:50:23.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.660+0000 7f27feec3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f27f815e480 con 0x7f27f80b5280 2024-09-19T07:50:23.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.660+0000 7f27feec3640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f27f815e780 con 0x7f27f815e0a0 2024-09-19T07:50:23.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.660+0000 7f27f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27f80d43e0 0x7f27f815cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:23.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.660+0000 7f27f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27f80d43e0 0x7f27f815cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50546/0 (socket says 172.21.15.16:50546) 2024-09-19T07:50:23.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.660+0000 7f27f7fff640 1 -- 172.21.15.16:0/3675192212 learned_addr learned my addr 172.21.15.16:0/3675192212 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:23.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.660+0000 7f27fd460640 1 --2- 172.21.15.16:0/3675192212 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27f815e0a0 0x7f27f80d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:23.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.660+0000 7f27fcc5f640 1 --2- 172.21.15.16:0/3675192212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27f80b5280 0x7f27f80b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:23.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.660+0000 7f27f7fff640 1 -- 172.21.15.16:0/3675192212 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27f815e0a0 msgr2=0x7f27f80d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:23.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.660+0000 7f27f7fff640 1 --2- 172.21.15.16:0/3675192212 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27f815e0a0 0x7f27f80d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.660+0000 7f27f7fff640 1 -- 172.21.15.16:0/3675192212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27f80b5280 msgr2=0x7f27f80b7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:23.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.660+0000 7f27f7fff640 1 --2- 172.21.15.16:0/3675192212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27f80b5280 0x7f27f80b7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.660+0000 7f27f7fff640 1 -- 172.21.15.16:0/3675192212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f80b9b90 con 0x7f27f80d43e0 2024-09-19T07:50:23.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.660+0000 7f27fd460640 1 --2- 172.21.15.16:0/3675192212 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27f815e0a0 0x7f27f80d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:23.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.660+0000 7f27f7fff640 1 --2- 172.21.15.16:0/3675192212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27f80d43e0 0x7f27f815cfc0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f27e400c400 tx=0x7f27e400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:23.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.661+0000 7f27f57fa640 1 -- 172.21.15.16:0/3675192212 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e4017070 con 0x7f27f80d43e0 2024-09-19T07:50:23.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.661+0000 7f27f57fa640 1 -- 172.21.15.16:0/3675192212 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27e4003640 con 0x7f27f80d43e0 2024-09-19T07:50:23.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.661+0000 7f27feec3640 1 -- 172.21.15.16:0/3675192212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f80b74d0 con 0x7f27f80d43e0 2024-09-19T07:50:23.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.661+0000 7f27f57fa640 1 -- 172.21.15.16:0/3675192212 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e4003a30 con 0x7f27f80d43e0 2024-09-19T07:50:23.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.661+0000 7f27feec3640 1 -- 172.21.15.16:0/3675192212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f80b7a90 con 0x7f27f80d43e0 2024-09-19T07:50:23.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.662+0000 7f27feec3640 1 -- 172.21.15.16:0/3675192212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27f80d4820 con 0x7f27f80d43e0 2024-09-19T07:50:23.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.662+0000 7f27f57fa640 1 -- 172.21.15.16:0/3675192212 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f27e401a070 con 0x7f27f80d43e0 2024-09-19T07:50:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.663+0000 7f27f57fa640 1 --2- 172.21.15.16:0/3675192212 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f27dc06c2b0 0x7f27dc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.663+0000 7f27f57fa640 1 -- 172.21.15.16:0/3675192212 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f27e408a960 con 0x7f27f80d43e0 2024-09-19T07:50:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.663+0000 7f27fcc5f640 1 --2- 172.21.15.16:0/3675192212 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f27dc06c2b0 0x7f27dc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.663+0000 7f27fcc5f640 1 --2- 172.21.15.16:0/3675192212 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f27dc06c2b0 0x7f27dc06e770 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f27ec034000 tx=0x7f27ec02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.667+0000 7f27f57fa640 1 -- 172.21.15.16:0/3675192212 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f27e4058fd0 con 0x7f27f80d43e0 2024-09-19T07:50:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.913+0000 7f27feec3640 1 -- 172.21.15.16:0/3675192212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f27f80ba3d0 con 0x7f27f80d43e0 2024-09-19T07:50:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.915+0000 7f27f57fa640 1 -- 172.21.15.16:0/3675192212 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f27e40588d0 con 0x7f27f80d43e0 2024-09-19T07:50:23.915 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-19T07:50:23.915 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA== 2024-09-19T07:50:23.915 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-19T07:50:23.915 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-19T07:50:23.916 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T07:50:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-19T07:50:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.917+0000 7f27feec3640 1 -- 172.21.15.16:0/3675192212 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f27dc06c2b0 msgr2=0x7f27dc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.917+0000 7f27feec3640 1 --2- 172.21.15.16:0/3675192212 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f27dc06c2b0 0x7f27dc06e770 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f27ec034000 tx=0x7f27ec02e000 comp rx=0 tx=0).stop 2024-09-19T07:50:23.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.917+0000 7f27feec3640 1 -- 172.21.15.16:0/3675192212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27f80d43e0 msgr2=0x7f27f815cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:23.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.917+0000 7f27feec3640 1 --2- 172.21.15.16:0/3675192212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27f80d43e0 0x7f27f815cfc0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f27e400c400 tx=0x7f27e400c8c0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.917+0000 7f27feec3640 1 -- 172.21.15.16:0/3675192212 shutdown_connections 2024-09-19T07:50:23.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.917+0000 7f27feec3640 1 --2- 172.21.15.16:0/3675192212 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f27dc06c2b0 0x7f27dc06e770 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.917+0000 7f27feec3640 1 --2- 172.21.15.16:0/3675192212 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27f815e0a0 0x7f27f80d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.917+0000 7f27feec3640 1 --2- 172.21.15.16:0/3675192212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27f80d43e0 0x7f27f815cfc0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.917+0000 7f27feec3640 1 --2- 172.21.15.16:0/3675192212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27f80b5280 0x7f27f80b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.917+0000 7f27feec3640 1 -- 172.21.15.16:0/3675192212 >> 172.21.15.16:0/3675192212 conn(0x7f27f809d770 msgr2=0x7f27f80b4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:23.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.917+0000 7f27feec3640 1 -- 172.21.15.16:0/3675192212 shutdown_connections 2024-09-19T07:50:23.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:23.917+0000 7f27feec3640 1 -- 172.21.15.16:0/3675192212 wait complete. 2024-09-19T07:50:23.929 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:50:24.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.030+0000 7fe61c79a640 1 Processor -- start 2024-09-19T07:50:24.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.030+0000 7fe61c79a640 1 -- start start 2024-09-19T07:50:24.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.031+0000 7fe61c79a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61415f0d0 0x7fe61415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.031+0000 7fe61c79a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe6140d0070 0x7fe6140d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.031+0000 7fe61c79a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe6140d1940 0x7fe6140d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.031+0000 7fe61c79a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fe6140cf2b0 con 0x7fe6140d1940 2024-09-19T07:50:24.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.031+0000 7fe61c79a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe6140cf590 con 0x7fe6140d0070 2024-09-19T07:50:24.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.031+0000 7fe61c79a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe6140cf870 con 0x7fe61415f0d0 2024-09-19T07:50:24.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.031+0000 7fe61ad37640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe6140d1940 0x7fe6140d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.031+0000 7fe61ad37640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe6140d1940 0x7fe6140d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50560/0 (socket says 172.21.15.16:50560) 2024-09-19T07:50:24.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.032+0000 7fe61ad37640 1 -- 172.21.15.16:0/1428281786 learned_addr learned my addr 172.21.15.16:0/1428281786 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:24.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.032+0000 7fe619d35640 1 --2- 172.21.15.16:0/1428281786 >> v2:172.21.15.146:3300/0 conn(0x7fe6140d0070 0x7fe6140d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.032+0000 7fe61a536640 1 --2- 172.21.15.16:0/1428281786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61415f0d0 0x7fe61415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.032+0000 7fe61ad37640 1 -- 172.21.15.16:0/1428281786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61415f0d0 msgr2=0x7fe61415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:24.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.032+0000 7fe61ad37640 1 --2- 172.21.15.16:0/1428281786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61415f0d0 0x7fe61415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.032+0000 7fe61ad37640 1 -- 172.21.15.16:0/1428281786 >> v2:172.21.15.146:3300/0 conn(0x7fe6140d0070 msgr2=0x7fe6140d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:24.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.032+0000 7fe61ad37640 1 --2- 172.21.15.16:0/1428281786 >> v2:172.21.15.146:3300/0 conn(0x7fe6140d0070 0x7fe6140d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.032+0000 7fe61ad37640 1 -- 172.21.15.16:0/1428281786 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6140cfb50 con 0x7fe6140d1940 2024-09-19T07:50:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.033+0000 7fe61ad37640 1 --2- 172.21.15.16:0/1428281786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6140d1940 0x7fe6140d4130 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe6140d1f60 tx=0x7fe60c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270c73442895c902 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.033+0000 7fe619534640 1 -- 172.21.15.16:0/1428281786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c037070 con 0x7fe6140d1940 2024-09-19T07:50:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.033+0000 7fe619534640 1 -- 172.21.15.16:0/1428281786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe60c032440 con 0x7fe6140d1940 2024-09-19T07:50:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.033+0000 7fe619534640 1 -- 172.21.15.16:0/1428281786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c032810 con 0x7fe6140d1940 2024-09-19T07:50:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.033+0000 7fe61c79a640 1 -- 172.21.15.16:0/1428281786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6140d1940 msgr2=0x7fe6140d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.033+0000 7fe61c79a640 1 --2- 172.21.15.16:0/1428281786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6140d1940 0x7fe6140d4130 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe6140d1f60 tx=0x7fe60c02a140 comp rx=0 tx=0).stop 2024-09-19T07:50:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.033+0000 7fe61c79a640 1 -- 172.21.15.16:0/1428281786 shutdown_connections 2024-09-19T07:50:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.033+0000 7fe61c79a640 1 --2- 172.21.15.16:0/1428281786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6140d1940 0x7fe6140d4130 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.033+0000 7fe61c79a640 1 --2- 172.21.15.16:0/1428281786 >> v2:172.21.15.146:3300/0 conn(0x7fe6140d0070 0x7fe6140d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.033+0000 7fe61c79a640 1 --2- 172.21.15.16:0/1428281786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61415f0d0 0x7fe61415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.033+0000 7fe61c79a640 1 -- 172.21.15.16:0/1428281786 >> 172.21.15.16:0/1428281786 conn(0x7fe61409d760 msgr2=0x7fe61408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.033+0000 7fe61c79a640 1 -- 172.21.15.16:0/1428281786 shutdown_connections 2024-09-19T07:50:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.033+0000 7fe61c79a640 1 -- 172.21.15.16:0/1428281786 wait complete. 2024-09-19T07:50:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.035+0000 7fe61c79a640 1 Processor -- start 2024-09-19T07:50:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.035+0000 7fe61c79a640 1 -- start start 2024-09-19T07:50:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.035+0000 7fe61c79a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6140d0070 0x7fe6140ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.035+0000 7fe61c79a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6140d1940 0x7fe614154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.035+0000 7fe61c79a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61415f0d0 0x7fe614155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.035+0000 7fe61c79a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6140d47f0 con 0x7fe6140d1940 2024-09-19T07:50:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.035+0000 7fe61c79a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe6140d4670 con 0x7fe6140d0070 2024-09-19T07:50:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.035+0000 7fe61c79a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe6140d4970 con 0x7fe61415f0d0 2024-09-19T07:50:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.035+0000 7fe619d35640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6140d1940 0x7fe614154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.035+0000 7fe619d35640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6140d1940 0x7fe614154c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50568/0 (socket says 172.21.15.16:50568) 2024-09-19T07:50:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.035+0000 7fe619d35640 1 -- 172.21.15.16:0/3306640090 learned_addr learned my addr 172.21.15.16:0/3306640090 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.035+0000 7fe61ad37640 1 --2- 172.21.15.16:0/3306640090 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61415f0d0 0x7fe614155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.036+0000 7fe61a536640 1 --2- 172.21.15.16:0/3306640090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6140d0070 0x7fe6140ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.036+0000 7fe619d35640 1 -- 172.21.15.16:0/3306640090 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61415f0d0 msgr2=0x7fe614155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.036+0000 7fe619d35640 1 --2- 172.21.15.16:0/3306640090 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61415f0d0 0x7fe614155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.036+0000 7fe619d35640 1 -- 172.21.15.16:0/3306640090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6140d0070 msgr2=0x7fe6140ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.036+0000 7fe619d35640 1 --2- 172.21.15.16:0/3306640090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6140d0070 0x7fe6140ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.036+0000 7fe619d35640 1 -- 172.21.15.16:0/3306640090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6141562d0 con 0x7fe6140d1940 2024-09-19T07:50:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.036+0000 7fe619d35640 1 --2- 172.21.15.16:0/3306640090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6140d1940 0x7fe614154c10 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe60800d5f0 tx=0x7fe60800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.036+0000 7fe612ffd640 1 -- 172.21.15.16:0/3306640090 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608004240 con 0x7fe6140d1940 2024-09-19T07:50:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.036+0000 7fe612ffd640 1 -- 172.21.15.16:0/3306640090 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6080043e0 con 0x7fe6140d1940 2024-09-19T07:50:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.036+0000 7fe61c79a640 1 -- 172.21.15.16:0/3306640090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe614156570 con 0x7fe6140d1940 2024-09-19T07:50:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.036+0000 7fe612ffd640 1 -- 172.21.15.16:0/3306640090 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608003610 con 0x7fe6140d1940 2024-09-19T07:50:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.036+0000 7fe61c79a640 1 -- 172.21.15.16:0/3306640090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6140b86c0 con 0x7fe6140d1940 2024-09-19T07:50:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.037+0000 7fe61c79a640 1 -- 172.21.15.16:0/3306640090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6140d1400 con 0x7fe6140d1940 2024-09-19T07:50:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.038+0000 7fe612ffd640 1 -- 172.21.15.16:0/3306640090 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe60801b020 con 0x7fe6140d1940 2024-09-19T07:50:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.038+0000 7fe612ffd640 1 --2- 172.21.15.16:0/3306640090 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe60006c2b0 0x7fe60006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.038+0000 7fe612ffd640 1 -- 172.21.15.16:0/3306640090 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fe60808b160 con 0x7fe6140d1940 2024-09-19T07:50:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.038+0000 7fe61a536640 1 --2- 172.21.15.16:0/3306640090 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe60006c2b0 0x7fe60006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.039+0000 7fe61a536640 1 --2- 172.21.15.16:0/3306640090 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe60006c2b0 0x7fe60006e770 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe6140cdf30 tx=0x7fe604009830 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.042+0000 7fe612ffd640 1 -- 172.21.15.16:0/3306640090 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe6080590f0 con 0x7fe6140d1940 2024-09-19T07:50:24.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.255+0000 7fe61c79a640 1 -- 172.21.15.16:0/3306640090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe61415be80 con 0x7fe6140d1940 2024-09-19T07:50:24.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.256+0000 7fe612ffd640 1 -- 172.21.15.16:0/3306640090 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5231 (secure 0 0 0) 0x7fe60805d3f0 con 0x7fe6140d1940 2024-09-19T07:50:24.257 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:24.257 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":47,"btime":"2024-09-19T07:50:17:700363+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5684,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1865971154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1865971154},{"type":"v1","addr":"172.21.15.146:6837","nonce":1865971154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6094,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/2735152611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2735152611},{"type":"v1","addr":"172.21.15.146:6836","nonce":2735152611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6399,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4204683210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4204683210},{"type":"v1","addr":"172.21.15.16:6834","nonce":4204683210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:50:15.050678+0000","modified":"2024-09-19T07:50:17.046800+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6402":{"gid":6402,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/1305244492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1305244492},{"type":"v1","addr":"172.21.15.16:6835","nonce":1305244492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6402,"qdb_cluster":[6402]},"id":6}]} 2024-09-19T07:50:24.257 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 47 2024-09-19T07:50:24.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.258+0000 7fe61c79a640 1 -- 172.21.15.16:0/3306640090 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe60006c2b0 msgr2=0x7fe60006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:24.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.258+0000 7fe61c79a640 1 --2- 172.21.15.16:0/3306640090 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe60006c2b0 0x7fe60006e770 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe6140cdf30 tx=0x7fe604009830 comp rx=0 tx=0).stop 2024-09-19T07:50:24.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.258+0000 7fe61c79a640 1 -- 172.21.15.16:0/3306640090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6140d1940 msgr2=0x7fe614154c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:24.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.258+0000 7fe61c79a640 1 --2- 172.21.15.16:0/3306640090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6140d1940 0x7fe614154c10 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe60800d5f0 tx=0x7fe60800dab0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.258+0000 7fe61c79a640 1 -- 172.21.15.16:0/3306640090 shutdown_connections 2024-09-19T07:50:24.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.258+0000 7fe61c79a640 1 --2- 172.21.15.16:0/3306640090 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe60006c2b0 0x7fe60006e770 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.258+0000 7fe61c79a640 1 --2- 172.21.15.16:0/3306640090 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61415f0d0 0x7fe614155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.258+0000 7fe61c79a640 1 --2- 172.21.15.16:0/3306640090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6140d1940 0x7fe614154c10 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.258+0000 7fe61c79a640 1 --2- 172.21.15.16:0/3306640090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6140d0070 0x7fe6140ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.258+0000 7fe61c79a640 1 -- 172.21.15.16:0/3306640090 >> 172.21.15.16:0/3306640090 conn(0x7fe61409d760 msgr2=0x7fe61406e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.258+0000 7fe61c79a640 1 -- 172.21.15.16:0/3306640090 shutdown_connections 2024-09-19T07:50:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.258+0000 7fe61c79a640 1 -- 172.21.15.16:0/3306640090 wait complete. 2024-09-19T07:50:24.672 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T07:50:15.050678+0000', 'modified': '2024-09-19T07:50:17.046800+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6402}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6402': {'gid': 6402, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6835/1305244492', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6833', 'nonce': 1305244492}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 1305244492}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6402, 'qdb_cluster': [6402]} 2024-09-19T07:50:24.672 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T07:50:24.673 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T07:50:24.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.781+0000 7f83670b7640 1 Processor -- start 2024-09-19T07:50:24.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.781+0000 7f83670b7640 1 -- start start 2024-09-19T07:50:24.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.782+0000 7f83670b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f836015e2a0 0x7f836006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.782+0000 7f83670b7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f836015e680 0x7f83600cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.782+0000 7f83670b7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f836006d960 0x7f83600d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.782+0000 7f83670b7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f83600d1880 con 0x7f836015e680 2024-09-19T07:50:24.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.782+0000 7f83670b7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f83600d1b60 con 0x7f836006d960 2024-09-19T07:50:24.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.782+0000 7f83670b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f83600cf2d0 con 0x7f836015e2a0 2024-09-19T07:50:24.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.782+0000 7f835ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f836015e680 0x7f83600cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.782+0000 7f835ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f836015e680 0x7f83600cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50582/0 (socket says 172.21.15.16:50582) 2024-09-19T07:50:24.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.782+0000 7f835ffff640 1 -- 172.21.15.16:0/2682027159 learned_addr learned my addr 172.21.15.16:0/2682027159 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:24.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.782+0000 7f8364e53640 1 --2- 172.21.15.16:0/2682027159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f836015e2a0 0x7f836006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.782+0000 7f8365654640 1 --2- 172.21.15.16:0/2682027159 >> v2:172.21.15.146:3300/0 conn(0x7f836006d960 0x7f83600d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.782+0000 7f835ffff640 1 -- 172.21.15.16:0/2682027159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f836015e2a0 msgr2=0x7f836006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:24.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.782+0000 7f835ffff640 1 --2- 172.21.15.16:0/2682027159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f836015e2a0 0x7f836006d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.782+0000 7f835ffff640 1 -- 172.21.15.16:0/2682027159 >> v2:172.21.15.146:3300/0 conn(0x7f836006d960 msgr2=0x7f83600d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:24.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.783+0000 7f835ffff640 1 --2- 172.21.15.16:0/2682027159 >> v2:172.21.15.146:3300/0 conn(0x7f836006d960 0x7f83600d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.783+0000 7f835ffff640 1 -- 172.21.15.16:0/2682027159 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83600cf5b0 con 0x7f836015e680 2024-09-19T07:50:24.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.783+0000 7f8365654640 1 --2- 172.21.15.16:0/2682027159 >> v2:172.21.15.146:3300/0 conn(0x7f836006d960 0x7f83600d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:24.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.783+0000 7f835ffff640 1 --2- 172.21.15.16:0/2682027159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f836015e680 0x7f83600cfad0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f836006cd80 tx=0x7f835402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63bd32782585ad3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:24.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.783+0000 7f835f7fe640 1 -- 172.21.15.16:0/2682027159 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354037070 con 0x7f836015e680 2024-09-19T07:50:24.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.783+0000 7f835f7fe640 1 -- 172.21.15.16:0/2682027159 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8354032440 con 0x7f836015e680 2024-09-19T07:50:24.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.783+0000 7f835f7fe640 1 -- 172.21.15.16:0/2682027159 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354032810 con 0x7f836015e680 2024-09-19T07:50:24.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.784+0000 7f83670b7640 1 -- 172.21.15.16:0/2682027159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f836015e680 msgr2=0x7f83600cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:24.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.784+0000 7f83670b7640 1 --2- 172.21.15.16:0/2682027159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f836015e680 0x7f83600cfad0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f836006cd80 tx=0x7f835402a1e0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.784+0000 7f83670b7640 1 -- 172.21.15.16:0/2682027159 shutdown_connections 2024-09-19T07:50:24.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.784+0000 7f83670b7640 1 --2- 172.21.15.16:0/2682027159 >> v2:172.21.15.146:3300/0 conn(0x7f836006d960 0x7f83600d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.784+0000 7f83670b7640 1 --2- 172.21.15.16:0/2682027159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f836015e680 0x7f83600cfad0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.784+0000 7f83670b7640 1 --2- 172.21.15.16:0/2682027159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f836015e2a0 0x7f836006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.784+0000 7f83670b7640 1 -- 172.21.15.16:0/2682027159 >> 172.21.15.16:0/2682027159 conn(0x7f836009d760 msgr2=0x7f836008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:24.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.784+0000 7f83670b7640 1 -- 172.21.15.16:0/2682027159 shutdown_connections 2024-09-19T07:50:24.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.784+0000 7f83670b7640 1 -- 172.21.15.16:0/2682027159 wait complete. 2024-09-19T07:50:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.785+0000 7f83670b7640 1 Processor -- start 2024-09-19T07:50:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.785+0000 7f83670b7640 1 -- start start 2024-09-19T07:50:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.786+0000 7f83670b7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f836006d960 0x7f83600b9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.786+0000 7f83670b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f836015e2a0 0x7f83600ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.786+0000 7f83670b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f836015e680 0x7f836014d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.786+0000 7f83670b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83600d44f0 con 0x7f836015e680 2024-09-19T07:50:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.786+0000 7f83670b7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f83600d4370 con 0x7f836006d960 2024-09-19T07:50:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.786+0000 7f83670b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f83600d4670 con 0x7f836015e2a0 2024-09-19T07:50:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.786+0000 7f8365654640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f836015e680 0x7f836014d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.786+0000 7f8365654640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f836015e680 0x7f836014d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50594/0 (socket says 172.21.15.16:50594) 2024-09-19T07:50:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.786+0000 7f8365654640 1 -- 172.21.15.16:0/72048304 learned_addr learned my addr 172.21.15.16:0/72048304 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.787+0000 7f8365654640 1 -- 172.21.15.16:0/72048304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f836015e2a0 msgr2=0x7f83600ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.787+0000 7f835ffff640 1 --2- 172.21.15.16:0/72048304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f836015e2a0 0x7f83600ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.787+0000 7f8364e53640 1 --2- 172.21.15.16:0/72048304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f836006d960 0x7f83600b9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.787+0000 7f8365654640 1 --2- 172.21.15.16:0/72048304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f836015e2a0 0x7f83600ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.787+0000 7f8365654640 1 -- 172.21.15.16:0/72048304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f836006d960 msgr2=0x7f83600b9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:24.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.787+0000 7f8365654640 1 --2- 172.21.15.16:0/72048304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f836006d960 0x7f83600b9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.787+0000 7f8365654640 1 -- 172.21.15.16:0/72048304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83600ba140 con 0x7f836015e680 2024-09-19T07:50:24.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.787+0000 7f8364e53640 1 --2- 172.21.15.16:0/72048304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f836006d960 0x7f83600b9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:24.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.787+0000 7f835ffff640 1 --2- 172.21.15.16:0/72048304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f836015e2a0 0x7f83600ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:24.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.787+0000 7f8365654640 1 --2- 172.21.15.16:0/72048304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f836015e680 0x7f836014d2e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8358009fa0 tx=0x7f835800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:24.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.787+0000 7f835d7fa640 1 -- 172.21.15.16:0/72048304 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835801a070 con 0x7f836015e680 2024-09-19T07:50:24.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.787+0000 7f835d7fa640 1 -- 172.21.15.16:0/72048304 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8358003bb0 con 0x7f836015e680 2024-09-19T07:50:24.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.787+0000 7f83670b7640 1 -- 172.21.15.16:0/72048304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83600c94e0 con 0x7f836015e680 2024-09-19T07:50:24.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.787+0000 7f835d7fa640 1 -- 172.21.15.16:0/72048304 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358004030 con 0x7f836015e680 2024-09-19T07:50:24.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.787+0000 7f83670b7640 1 -- 172.21.15.16:0/72048304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83600c9aa0 con 0x7f836015e680 2024-09-19T07:50:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.789+0000 7f835d7fa640 1 -- 172.21.15.16:0/72048304 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f835801d070 con 0x7f836015e680 2024-09-19T07:50:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.789+0000 7f835d7fa640 1 --2- 172.21.15.16:0/72048304 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f834c06c2b0 0x7f834c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.789+0000 7f835d7fa640 1 -- 172.21.15.16:0/72048304 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f835808b900 con 0x7f836015e680 2024-09-19T07:50:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.789+0000 7f8364e53640 1 --2- 172.21.15.16:0/72048304 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f834c06c2b0 0x7f834c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.790+0000 7f8364e53640 1 --2- 172.21.15.16:0/72048304 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f834c06c2b0 0x7f834c06e770 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8350006fd0 tx=0x7f8350008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.790+0000 7f8369e50740 1 Processor -- start 2024-09-19T07:50:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.790+0000 7f8369e50740 1 -- start start 2024-09-19T07:50:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.790+0000 7f8369e50740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e468632c0 0x563e468a8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.791+0000 7f8369e50740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e46860a80 0x563e468636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.791+0000 7f8369e50740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563e46864a50 0x563e4685d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.791+0000 7f8369e50740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563e467c5520 con 0x563e468632c0 2024-09-19T07:50:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.791+0000 7f8369e50740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563e4676d3e0 con 0x563e46864a50 2024-09-19T07:50:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.791+0000 7f8369e50740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x563e467c14e0 con 0x563e46860a80 2024-09-19T07:50:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.791+0000 7f8364e53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e468632c0 0x563e468a8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.791+0000 7f8364e53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e468632c0 0x563e468a8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50600/0 (socket says 172.21.15.16:50600) 2024-09-19T07:50:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.791+0000 7f8364e53640 1 -- 172.21.15.16:0/3778108681 learned_addr learned my addr 172.21.15.16:0/3778108681 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.791+0000 7f8365654640 1 --2- 172.21.15.16:0/3778108681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563e46864a50 0x563e4685d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.791+0000 7f835ffff640 1 --2- 172.21.15.16:0/3778108681 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e46860a80 0x563e468636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.791+0000 7f8364e53640 1 -- 172.21.15.16:0/3778108681 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e46860a80 msgr2=0x563e468636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.791+0000 7f8364e53640 1 --2- 172.21.15.16:0/3778108681 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e46860a80 0x563e468636a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.791+0000 7f8364e53640 1 -- 172.21.15.16:0/3778108681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563e46864a50 msgr2=0x563e4685d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.791+0000 7f8364e53640 1 --2- 172.21.15.16:0/3778108681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563e46864a50 0x563e4685d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.791+0000 7f8364e53640 1 -- 172.21.15.16:0/3778108681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e467882f0 con 0x563e468632c0 2024-09-19T07:50:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.791+0000 7f8364e53640 1 --2- 172.21.15.16:0/3778108681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e468632c0 0x563e468a8a60 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f83500043e0 tx=0x7f8350011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb4f7de5b29e656 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.791+0000 7f83467fc640 1 -- 172.21.15.16:0/3778108681 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835000f040 con 0x563e468632c0 2024-09-19T07:50:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.791+0000 7f83467fc640 1 -- 172.21.15.16:0/3778108681 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8350004410 con 0x563e468632c0 2024-09-19T07:50:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.791+0000 7f83467fc640 1 -- 172.21.15.16:0/3778108681 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835000ee80 con 0x563e468632c0 2024-09-19T07:50:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.792+0000 7f8369e50740 1 -- 172.21.15.16:0/3778108681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e468632c0 msgr2=0x563e468a8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.792+0000 7f8369e50740 1 --2- 172.21.15.16:0/3778108681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e468632c0 0x563e468a8a60 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f83500043e0 tx=0x7f8350011000 comp rx=0 tx=0).stop 2024-09-19T07:50:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.792+0000 7f8369e50740 1 -- 172.21.15.16:0/3778108681 shutdown_connections 2024-09-19T07:50:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.792+0000 7f8369e50740 1 --2- 172.21.15.16:0/3778108681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563e46864a50 0x563e4685d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.792+0000 7f8369e50740 1 --2- 172.21.15.16:0/3778108681 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e46860a80 0x563e468636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.792+0000 7f8369e50740 1 --2- 172.21.15.16:0/3778108681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e468632c0 0x563e468a8a60 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.792+0000 7f8369e50740 1 -- 172.21.15.16:0/3778108681 >> 172.21.15.16:0/3778108681 conn(0x563e4685fa80 msgr2=0x563e468a7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.792+0000 7f8369e50740 1 -- 172.21.15.16:0/3778108681 shutdown_connections 2024-09-19T07:50:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.792+0000 7f8369e50740 1 -- 172.21.15.16:0/3778108681 wait complete. 2024-09-19T07:50:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.792+0000 7f8369e50740 1 -- start start 2024-09-19T07:50:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.793+0000 7f8369e50740 1 Processor -- start 2024-09-19T07:50:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.793+0000 7f8369e50740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563e468632c0 0x563e468b9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.793+0000 7f8369e50740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e46864a50 0x563e4685ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.793+0000 7f8369e50740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e4687ea10 0x563e46883cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f8369e50740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563e4685ef20 con 0x563e46864a50 2024-09-19T07:50:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f8369e50740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563e4674e450 con 0x563e468632c0 2024-09-19T07:50:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f8369e50740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x563e468a9290 con 0x563e4687ea10 2024-09-19T07:50:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f8369e50740 10 client.0 _ll_register_callbacks cb 0x563e464d7ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f835ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e46864a50 0x563e4685ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f835ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e46864a50 0x563e4685ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50602/0 (socket says 172.21.15.16:50602) 2024-09-19T07:50:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f835ffff640 1 -- 172.21.15.16:0/1371820701 learned_addr learned my addr 172.21.15.16:0/1371820701 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f8365654640 1 --2- 172.21.15.16:0/1371820701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e4687ea10 0x563e46883cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f8364e53640 1 --2- 172.21.15.16:0/1371820701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563e468632c0 0x563e468b9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f835ffff640 1 -- 172.21.15.16:0/1371820701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e4687ea10 msgr2=0x563e46883cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f835ffff640 1 --2- 172.21.15.16:0/1371820701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e4687ea10 0x563e46883cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f835ffff640 1 -- 172.21.15.16:0/1371820701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563e468632c0 msgr2=0x563e468b9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f835ffff640 1 --2- 172.21.15.16:0/1371820701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563e468632c0 0x563e468b9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f835ffff640 1 -- 172.21.15.16:0/1371820701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e464d82d0 con 0x563e46864a50 2024-09-19T07:50:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f835ffff640 1 --2- 172.21.15.16:0/1371820701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e46864a50 0x563e4685ca00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8354000c00 tx=0x7f8354004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd0cf7484885933 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f833f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:50:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f833f7fe640 1 -- 172.21.15.16:0/1371820701 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354042070 con 0x563e46864a50 2024-09-19T07:50:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f833f7fe640 1 -- 172.21.15.16:0/1371820701 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f8354047c40 con 0x563e46864a50 2024-09-19T07:50:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f8369e50740 1 -- 172.21.15.16:0/1371820701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8350004410 con 0x563e46864a50 2024-09-19T07:50:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f833f7fe640 1 -- 172.21.15.16:0/1371820701 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8354037040 con 0x563e46864a50 2024-09-19T07:50:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f8369e50740 10 client.6555 fetch_fsmap learned FSMap version 47 2024-09-19T07:50:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.794+0000 7f8369e50740 1 -- 172.21.15.16:0/1371820701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) -- 0x563e465a26b0 con 0x563e46864a50 2024-09-19T07:50:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.795+0000 7f833f7fe640 1 -- 172.21.15.16:0/1371820701 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8350004410 con 0x563e46864a50 2024-09-19T07:50:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.795+0000 7f833f7fe640 1 -- 172.21.15.16:0/1371820701 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 47) ==== 3502+0+0 (secure 0 0 0) 0x7f835403c410 con 0x563e46864a50 2024-09-19T07:50:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.795+0000 7f8369e50740 10 client.6555 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T07:50:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.795+0000 7f8369e50740 10 client.6555 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T07:50:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.795+0000 7f8369e50740 20 client.6555 populate_metadata read hostname 'smithi016' 2024-09-19T07:50:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.795+0000 7f8369e50740 1 --2- 172.21.15.16:0/1371820701 >> [v2:172.21.15.16:6833/1305244492,v1:172.21.15.16:6835/1305244492] conn(0x563e468bc5c0 0x563e468bc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.795+0000 7f8369e50740 4 client.6555 mds_command: new command op to 6402 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:50:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.795+0000 7f8369e50740 1 -- 172.21.15.16:0/1371820701 --> [v2:172.21.15.16:6833/1305244492,v1:172.21.15.16:6835/1305244492] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e467d2c50 con 0x563e468bc5c0 2024-09-19T07:50:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.795+0000 7f8364e53640 1 --2- 172.21.15.16:0/1371820701 >> [v2:172.21.15.16:6833/1305244492,v1:172.21.15.16:6835/1305244492] conn(0x563e468bc5c0 0x563e468bc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.795+0000 7f833f7fe640 10 client.6555 ms_handle_connect on v2:172.21.15.16:6833/1305244492 2024-09-19T07:50:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.795+0000 7f8364e53640 1 --2- 172.21.15.16:0/1371820701 >> [v2:172.21.15.16:6833/1305244492,v1:172.21.15.16:6835/1305244492] conn(0x563e468bc5c0 0x563e468bc9a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7110cbb247469ac server_cookie=a456c5f9e8669c61 in_seq=0 out_seq=0 2024-09-19T07:50:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.796+0000 7f833f7fe640 1 -- 172.21.15.16:0/1371820701 <== mds.0 v2:172.21.15.16:6833/1305244492 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e467d2c50 con 0x563e468bc5c0 2024-09-19T07:50:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.796+0000 7f833f7fe640 10 client.6555 handle_command_reply: tid=0 2024-09-19T07:50:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.796+0000 7f8369e50740 1 client.6555 shutdown 2024-09-19T07:50:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.796+0000 7f8369e50740 1 -- 172.21.15.16:0/1371820701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e46864a50 msgr2=0x563e4685ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.796+0000 7f8369e50740 1 --2- 172.21.15.16:0/1371820701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e46864a50 0x563e4685ca00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8354000c00 tx=0x7f8354004730 comp rx=0 tx=0).stop 2024-09-19T07:50:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.796+0000 7f8369e50740 1 -- 172.21.15.16:0/1371820701 shutdown_connections 2024-09-19T07:50:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.796+0000 7f8369e50740 1 --2- 172.21.15.16:0/1371820701 >> [v2:172.21.15.16:6833/1305244492,v1:172.21.15.16:6835/1305244492] conn(0x563e468bc5c0 0x563e468bc9a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.796+0000 7f8369e50740 1 --2- 172.21.15.16:0/1371820701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e4687ea10 0x563e46883cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.796+0000 7f833f7fe640 0 client.6555 ms_handle_reset on v2:172.21.15.16:6833/1305244492 2024-09-19T07:50:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.796+0000 7f8369e50740 1 --2- 172.21.15.16:0/1371820701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e46864a50 0x563e4685ca00 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.796+0000 7f8369e50740 1 --2- 172.21.15.16:0/1371820701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563e468632c0 0x563e468b9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.796+0000 7f8369e50740 1 -- 172.21.15.16:0/1371820701 >> 172.21.15.16:0/1371820701 conn(0x563e46860a80 msgr2=0x563e468a5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.796+0000 7f8369e50740 1 -- 172.21.15.16:0/1371820701 shutdown_connections 2024-09-19T07:50:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.796+0000 7f8369e50740 1 -- 172.21.15.16:0/1371820701 wait complete. 2024-09-19T07:50:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.796+0000 7f8369e50740 20 client.6555 trim_cache size 0 max 16384 2024-09-19T07:50:24.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.814+0000 7f8369e50740 1 Processor -- start 2024-09-19T07:50:24.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.814+0000 7f8369e50740 1 -- start start 2024-09-19T07:50:24.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.815+0000 7f8369e50740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e468632c0 0x563e468bb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.815+0000 7f8369e50740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563e46864a50 0x563e468bbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.815+0000 7f8369e50740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4687ea10 0x563e468cc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.815+0000 7f8369e50740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83601ffef0 con 0x563e4687ea10 2024-09-19T07:50:24.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.815+0000 7f8369e50740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f83601ff250 con 0x563e46864a50 2024-09-19T07:50:24.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.815+0000 7f8369e50740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f83601ff8a0 con 0x563e468632c0 2024-09-19T07:50:24.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.815+0000 7f8365654640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4687ea10 0x563e468cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.815+0000 7f8365654640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4687ea10 0x563e468cc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50618/0 (socket says 172.21.15.16:50618) 2024-09-19T07:50:24.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.815+0000 7f8365654640 1 -- 172.21.15.16:0/990554319 learned_addr learned my addr 172.21.15.16:0/990554319 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:24.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.815+0000 7f8364e53640 1 --2- 172.21.15.16:0/990554319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e468632c0 0x563e468bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.815+0000 7f8365654640 1 -- 172.21.15.16:0/990554319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e468632c0 msgr2=0x563e468bb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:24.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.815+0000 7f8365654640 1 --2- 172.21.15.16:0/990554319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e468632c0 0x563e468bb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.815+0000 7f8365654640 1 -- 172.21.15.16:0/990554319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563e46864a50 msgr2=0x563e468bbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:24.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.815+0000 7f8365654640 1 --2- 172.21.15.16:0/990554319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563e46864a50 0x563e468bbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.815+0000 7f8365654640 1 -- 172.21.15.16:0/990554319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e468598b0 con 0x563e4687ea10 2024-09-19T07:50:24.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.815+0000 7f8365654640 1 --2- 172.21.15.16:0/990554319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4687ea10 0x563e468cc970 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8358055c40 tx=0x7f8358059320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c403a3108594a63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:24.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.816+0000 7f8345ffb640 1 -- 172.21.15.16:0/990554319 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83580600c0 con 0x563e4687ea10 2024-09-19T07:50:24.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.816+0000 7f8345ffb640 1 -- 172.21.15.16:0/990554319 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8358071070 con 0x563e4687ea10 2024-09-19T07:50:24.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.816+0000 7f8345ffb640 1 -- 172.21.15.16:0/990554319 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358059a40 con 0x563e4687ea10 2024-09-19T07:50:24.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.816+0000 7f8369e50740 1 -- 172.21.15.16:0/990554319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4687ea10 msgr2=0x563e468cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:24.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.816+0000 7f8369e50740 1 --2- 172.21.15.16:0/990554319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4687ea10 0x563e468cc970 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8358055c40 tx=0x7f8358059320 comp rx=0 tx=0).stop 2024-09-19T07:50:24.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.816+0000 7f8369e50740 1 -- 172.21.15.16:0/990554319 shutdown_connections 2024-09-19T07:50:24.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.816+0000 7f8369e50740 1 --2- 172.21.15.16:0/990554319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4687ea10 0x563e468cc970 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.816+0000 7f8369e50740 1 --2- 172.21.15.16:0/990554319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563e46864a50 0x563e468bbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.816+0000 7f8369e50740 1 --2- 172.21.15.16:0/990554319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e468632c0 0x563e468bb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.816+0000 7f8369e50740 1 -- 172.21.15.16:0/990554319 >> 172.21.15.16:0/990554319 conn(0x563e46860a80 msgr2=0x563e4688aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:24.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.816+0000 7f8369e50740 1 -- 172.21.15.16:0/990554319 shutdown_connections 2024-09-19T07:50:24.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.816+0000 7f8369e50740 1 -- 172.21.15.16:0/990554319 wait complete. 2024-09-19T07:50:24.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.817+0000 7f8369e50740 1 -- start start 2024-09-19T07:50:24.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.817+0000 7f8369e50740 1 Processor -- start 2024-09-19T07:50:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f8369e50740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e468632c0 0x563e468b5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f8369e50740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563e46864a50 0x563e4685d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f8369e50740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4687ea10 0x563e4685db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f8369e50740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563e4688d730 con 0x563e4687ea10 2024-09-19T07:50:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f8369e50740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563e4685cf40 con 0x563e46864a50 2024-09-19T07:50:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f8369e50740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x563e468843c0 con 0x563e468632c0 2024-09-19T07:50:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f8369e50740 10 client.0 _ll_register_callbacks cb 0x7f83540096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f8365654640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4687ea10 0x563e4685db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f8365654640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4687ea10 0x563e4685db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50626/0 (socket says 172.21.15.16:50626) 2024-09-19T07:50:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f8365654640 1 -- 172.21.15.16:0/667764218 learned_addr learned my addr 172.21.15.16:0/667764218 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f835ffff640 1 --2- 172.21.15.16:0/667764218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563e46864a50 0x563e4685d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f8364e53640 1 --2- 172.21.15.16:0/667764218 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e468632c0 0x563e468b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f8365654640 1 -- 172.21.15.16:0/667764218 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e468632c0 msgr2=0x563e468b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f8365654640 1 --2- 172.21.15.16:0/667764218 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e468632c0 0x563e468b5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f8365654640 1 -- 172.21.15.16:0/667764218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563e46864a50 msgr2=0x563e4685d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f8365654640 1 --2- 172.21.15.16:0/667764218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563e46864a50 0x563e4685d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f8365654640 1 -- 172.21.15.16:0/667764218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e468a9200 con 0x563e4687ea10 2024-09-19T07:50:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f8365654640 1 --2- 172.21.15.16:0/667764218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4687ea10 0x563e4685db30 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8358062580 tx=0x7f835806f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8921e33efb17456b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f833f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:50:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.818+0000 7f833f7fe640 1 -- 172.21.15.16:0/667764218 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358059ee0 con 0x563e4687ea10 2024-09-19T07:50:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.819+0000 7f833f7fe640 1 -- 172.21.15.16:0/667764218 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f8358061f30 con 0x563e4687ea10 2024-09-19T07:50:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.819+0000 7f8369e50740 1 -- 172.21.15.16:0/667764218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8358071070 con 0x563e4687ea10 2024-09-19T07:50:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.819+0000 7f833f7fe640 1 -- 172.21.15.16:0/667764218 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8358062130 con 0x563e4687ea10 2024-09-19T07:50:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.819+0000 7f8369e50740 10 client.6561 fetch_fsmap learned FSMap version 47 2024-09-19T07:50:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.819+0000 7f8369e50740 1 -- 172.21.15.16:0/667764218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) -- 0x563e468b5b40 con 0x563e4687ea10 2024-09-19T07:50:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.819+0000 7f833f7fe640 1 -- 172.21.15.16:0/667764218 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8358071070 con 0x563e4687ea10 2024-09-19T07:50:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.819+0000 7f833f7fe640 1 -- 172.21.15.16:0/667764218 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 47) ==== 3502+0+0 (secure 0 0 0) 0x7f8358078b80 con 0x563e4687ea10 2024-09-19T07:50:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.819+0000 7f8369e50740 10 client.6561 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T07:50:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.819+0000 7f8369e50740 10 client.6561 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T07:50:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.819+0000 7f8369e50740 20 client.6561 populate_metadata read hostname 'smithi016' 2024-09-19T07:50:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.819+0000 7f8369e50740 1 --2- 172.21.15.16:0/667764218 >> [v2:172.21.15.16:6833/1305244492,v1:172.21.15.16:6835/1305244492] conn(0x563e468849b0 0x563e468619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.819+0000 7f8369e50740 4 client.6561 mds_command: new command op to 6402 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:50:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.819+0000 7f8369e50740 1 -- 172.21.15.16:0/667764218 --> [v2:172.21.15.16:6833/1305244492,v1:172.21.15.16:6835/1305244492] -- command(tid 0: {"prefix": "status"}) -- 0x563e4688d0e0 con 0x563e468849b0 2024-09-19T07:50:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.819+0000 7f8364e53640 1 --2- 172.21.15.16:0/667764218 >> [v2:172.21.15.16:6833/1305244492,v1:172.21.15.16:6835/1305244492] conn(0x563e468849b0 0x563e468619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.819+0000 7f833f7fe640 10 client.6561 ms_handle_connect on v2:172.21.15.16:6833/1305244492 2024-09-19T07:50:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.819+0000 7f8364e53640 1 --2- 172.21.15.16:0/667764218 >> [v2:172.21.15.16:6833/1305244492,v1:172.21.15.16:6835/1305244492] conn(0x563e468849b0 0x563e468619f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70bf91169ea02fe6 server_cookie=6ef78c57c06111dc in_seq=0 out_seq=0 2024-09-19T07:50:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.820+0000 7f833f7fe640 1 -- 172.21.15.16:0/667764218 <== mds.0 v2:172.21.15.16:6833/1305244492 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x563e4688d0e0 con 0x563e468849b0 2024-09-19T07:50:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.820+0000 7f833f7fe640 10 client.6561 handle_command_reply: tid=0 2024-09-19T07:50:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.820+0000 7f8369e50740 1 client.6561 shutdown 2024-09-19T07:50:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.820+0000 7f8369e50740 1 -- 172.21.15.16:0/667764218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4687ea10 msgr2=0x563e4685db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.820+0000 7f8369e50740 1 --2- 172.21.15.16:0/667764218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4687ea10 0x563e4685db30 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8358062580 tx=0x7f835806f070 comp rx=0 tx=0).stop 2024-09-19T07:50:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.820+0000 7f8369e50740 1 -- 172.21.15.16:0/667764218 shutdown_connections 2024-09-19T07:50:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.820+0000 7f8369e50740 1 --2- 172.21.15.16:0/667764218 >> [v2:172.21.15.16:6833/1305244492,v1:172.21.15.16:6835/1305244492] conn(0x563e468849b0 0x563e468619f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.820+0000 7f8369e50740 1 --2- 172.21.15.16:0/667764218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e4687ea10 0x563e4685db30 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.820+0000 7f8369e50740 1 --2- 172.21.15.16:0/667764218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563e46864a50 0x563e4685d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.820+0000 7f833f7fe640 0 client.6561 ms_handle_reset on v2:172.21.15.16:6833/1305244492 2024-09-19T07:50:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.820+0000 7f8369e50740 1 --2- 172.21.15.16:0/667764218 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563e468632c0 0x563e468b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.820+0000 7f8369e50740 1 -- 172.21.15.16:0/667764218 >> 172.21.15.16:0/667764218 conn(0x563e46860a80 msgr2=0x563e468bb760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.820+0000 7f8369e50740 1 -- 172.21.15.16:0/667764218 shutdown_connections 2024-09-19T07:50:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.820+0000 7f8369e50740 1 -- 172.21.15.16:0/667764218 wait complete. 2024-09-19T07:50:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.820+0000 7f8369e50740 20 client.6561 trim_cache size 0 max 16384 2024-09-19T07:50:24.827 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:50:24.827 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb", 2024-09-19T07:50:24.827 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-19T07:50:24.827 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 6402, 2024-09-19T07:50:24.827 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T07:50:24.828 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T07:50:24.828 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T07:50:24.828 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 9.1332128010000009, 2024-09-19T07:50:24.828 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 47, 2024-09-19T07:50:24.828 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 93, 2024-09-19T07:50:24.828 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 93, 2024-09-19T07:50:24.828 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 14.134801728999999 2024-09-19T07:50:24.828 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:50:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.821+0000 7f83670b7640 1 -- 172.21.15.16:0/72048304 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f834c06c2b0 msgr2=0x7f834c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.821+0000 7f83670b7640 1 --2- 172.21.15.16:0/72048304 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f834c06c2b0 0x7f834c06e770 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8350006fd0 tx=0x7f8350008040 comp rx=0 tx=0).stop 2024-09-19T07:50:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.821+0000 7f83670b7640 1 -- 172.21.15.16:0/72048304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f836015e680 msgr2=0x7f836014d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.821+0000 7f83670b7640 1 --2- 172.21.15.16:0/72048304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f836015e680 0x7f836014d2e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8358009fa0 tx=0x7f835800d560 comp rx=0 tx=0).stop 2024-09-19T07:50:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.821+0000 7f83670b7640 1 -- 172.21.15.16:0/72048304 shutdown_connections 2024-09-19T07:50:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.821+0000 7f83670b7640 1 --2- 172.21.15.16:0/72048304 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f834c06c2b0 0x7f834c06e770 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.821+0000 7f83670b7640 1 --2- 172.21.15.16:0/72048304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f836015e680 0x7f836014d2e0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.821+0000 7f83670b7640 1 --2- 172.21.15.16:0/72048304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f836015e2a0 0x7f83600ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.821+0000 7f83670b7640 1 --2- 172.21.15.16:0/72048304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f836006d960 0x7f83600b9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.821+0000 7f83670b7640 1 -- 172.21.15.16:0/72048304 >> 172.21.15.16:0/72048304 conn(0x7f836009d760 msgr2=0x7f8360161f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:24.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.821+0000 7f83670b7640 1 -- 172.21.15.16:0/72048304 shutdown_connections 2024-09-19T07:50:24.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.821+0000 7f83670b7640 1 -- 172.21.15.16:0/72048304 wait complete. 2024-09-19T07:50:24.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:50:24.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.934+0000 7f95aab4b640 1 Processor -- start 2024-09-19T07:50:24.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.934+0000 7f95aab4b640 1 -- start start 2024-09-19T07:50:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.935+0000 7f95aab4b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f95a415e2a0 0x7f95a406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.935+0000 7f95aab4b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95a415e680 0x7f95a40cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.935+0000 7f95aab4b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a40d1880 con 0x7f95a406d960 2024-09-19T07:50:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.935+0000 7f95aab4b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f95a40d1b60 con 0x7f95a415e2a0 2024-09-19T07:50:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.935+0000 7f95aab4b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f95a40cf2d0 con 0x7f95a415e680 2024-09-19T07:50:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.935+0000 7f95a88e7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f95a415e2a0 0x7f95a406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.935+0000 7f95a88e7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f95a415e2a0 0x7f95a406d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50640/0 (socket says 172.21.15.16:50640) 2024-09-19T07:50:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.935+0000 7f95a88e7640 1 -- 172.21.15.16:0/3661683594 learned_addr learned my addr 172.21.15.16:0/3661683594 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.935+0000 7f95a3fff640 1 --2- 172.21.15.16:0/3661683594 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95a415e680 0x7f95a40cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.936+0000 7f95a88e7640 1 -- 172.21.15.16:0/3661683594 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95a415e680 msgr2=0x7f95a40cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.936+0000 7f95a88e7640 1 --2- 172.21.15.16:0/3661683594 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95a415e680 0x7f95a40cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.936+0000 7f95a88e7640 1 -- 172.21.15.16:0/3661683594 >> v1:172.21.15.146:6789/0 conn(0x7f95a406d960 legacy=0x7f95a40d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.936+0000 7f95a88e7640 1 -- 172.21.15.16:0/3661683594 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a40cf5b0 con 0x7f95a415e2a0 2024-09-19T07:50:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.936+0000 7f95a88e7640 1 --2- 172.21.15.16:0/3661683594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a415e2a0 0x7f95a406d420 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9594009870 tx=0x7f959402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5260ef4f6a7f8c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.936+0000 7f95a37fe640 1 -- 172.21.15.16:0/3661683594 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594037070 con 0x7f95a415e2a0 2024-09-19T07:50:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.937+0000 7f95a37fe640 1 -- 172.21.15.16:0/3661683594 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9594032440 con 0x7f95a415e2a0 2024-09-19T07:50:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.937+0000 7f95a37fe640 1 -- 172.21.15.16:0/3661683594 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594032810 con 0x7f95a415e2a0 2024-09-19T07:50:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.937+0000 7f95aab4b640 1 -- 172.21.15.16:0/3661683594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a415e2a0 msgr2=0x7f95a406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.937+0000 7f95aab4b640 1 --2- 172.21.15.16:0/3661683594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a415e2a0 0x7f95a406d420 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9594009870 tx=0x7f959402a210 comp rx=0 tx=0).stop 2024-09-19T07:50:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.937+0000 7f95aab4b640 1 -- 172.21.15.16:0/3661683594 shutdown_connections 2024-09-19T07:50:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.937+0000 7f95aab4b640 1 --2- 172.21.15.16:0/3661683594 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95a415e680 0x7f95a40cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.937+0000 7f95aab4b640 1 --2- 172.21.15.16:0/3661683594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a415e2a0 0x7f95a406d420 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.937+0000 7f95aab4b640 1 -- 172.21.15.16:0/3661683594 >> 172.21.15.16:0/3661683594 conn(0x7f95a409d760 msgr2=0x7f95a408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.937+0000 7f95aab4b640 1 -- 172.21.15.16:0/3661683594 shutdown_connections 2024-09-19T07:50:24.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.937+0000 7f95aab4b640 1 -- 172.21.15.16:0/3661683594 wait complete. 2024-09-19T07:50:24.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.938+0000 7f95aab4b640 1 Processor -- start 2024-09-19T07:50:24.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.938+0000 7f95aab4b640 1 -- start start 2024-09-19T07:50:24.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.939+0000 7f95aab4b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95a406d960 0x7f95a40c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.939+0000 7f95aab4b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95a415e2a0 0x7f95a40cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.939+0000 7f95aab4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a415e680 0x7f95a4151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.939+0000 7f95aab4b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f95a40d4c00 con 0x7f95a415e680 2024-09-19T07:50:24.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.939+0000 7f95aab4b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f95a40d4a80 con 0x7f95a415e2a0 2024-09-19T07:50:24.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.939+0000 7f95aab4b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f95a40d0010 con 0x7f95a406d960 2024-09-19T07:50:24.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.939+0000 7f95a90e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a415e680 0x7f95a4151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.939+0000 7f95a90e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a415e680 0x7f95a4151850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50656/0 (socket says 172.21.15.16:50656) 2024-09-19T07:50:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.939+0000 7f95a90e8640 1 -- 172.21.15.16:0/1666006349 learned_addr learned my addr 172.21.15.16:0/1666006349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.940+0000 7f95a88e7640 1 --2- 172.21.15.16:0/1666006349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95a406d960 0x7f95a40c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.940+0000 7f95a3fff640 1 --2- 172.21.15.16:0/1666006349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95a415e2a0 0x7f95a40cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.940+0000 7f95a90e8640 1 -- 172.21.15.16:0/1666006349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95a406d960 msgr2=0x7f95a40c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.940+0000 7f95a90e8640 1 --2- 172.21.15.16:0/1666006349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95a406d960 0x7f95a40c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.940+0000 7f95a90e8640 1 -- 172.21.15.16:0/1666006349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95a415e2a0 msgr2=0x7f95a40cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.940+0000 7f95a90e8640 1 --2- 172.21.15.16:0/1666006349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95a415e2a0 0x7f95a40cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.940+0000 7f95a90e8640 1 -- 172.21.15.16:0/1666006349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a4151d90 con 0x7f95a415e680 2024-09-19T07:50:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.940+0000 7f95a90e8640 1 --2- 172.21.15.16:0/1666006349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a415e680 0x7f95a4151850 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f959c00c780 tx=0x7f959c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.940+0000 7f95a17fa640 1 -- 172.21.15.16:0/1666006349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c017070 con 0x7f95a415e680 2024-09-19T07:50:24.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.940+0000 7f95a17fa640 1 -- 172.21.15.16:0/1666006349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f959c003da0 con 0x7f95a415e680 2024-09-19T07:50:24.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.940+0000 7f95a17fa640 1 -- 172.21.15.16:0/1666006349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c0052b0 con 0x7f95a415e680 2024-09-19T07:50:24.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.940+0000 7f95aab4b640 1 -- 172.21.15.16:0/1666006349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a40c9cc0 con 0x7f95a415e680 2024-09-19T07:50:24.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.940+0000 7f95aab4b640 1 -- 172.21.15.16:0/1666006349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a40ca280 con 0x7f95a415e680 2024-09-19T07:50:24.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.941+0000 7f95aab4b640 1 -- 172.21.15.16:0/1666006349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a406d420 con 0x7f95a415e680 2024-09-19T07:50:24.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.942+0000 7f95a17fa640 1 -- 172.21.15.16:0/1666006349 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f959c005450 con 0x7f95a415e680 2024-09-19T07:50:24.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.942+0000 7f95a17fa640 1 --2- 172.21.15.16:0/1666006349 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f959006c2e0 0x7f959006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:24.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.942+0000 7f95a17fa640 1 -- 172.21.15.16:0/1666006349 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f959c08abd0 con 0x7f95a415e680 2024-09-19T07:50:24.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.942+0000 7f95a88e7640 1 --2- 172.21.15.16:0/1666006349 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f959006c2e0 0x7f959006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:24.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.943+0000 7f95a88e7640 1 --2- 172.21.15.16:0/1666006349 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f959006c2e0 0x7f959006e7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9594034000 tx=0x7f959402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:24.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:24.947+0000 7f95a17fa640 1 -- 172.21.15.16:0/1666006349 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f95a4151d90 con 0x7f95a415e680 2024-09-19T07:50:25.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:25.158+0000 7f95aab4b640 1 -- 172.21.15.16:0/1666006349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f95a40756d0 con 0x7f95a415e680 2024-09-19T07:50:25.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:25.158+0000 7f95a17fa640 1 -- 172.21.15.16:0/1666006349 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) ==== 53+0+73 (secure 0 0 0) 0x7f959c059090 con 0x7f95a415e680 2024-09-19T07:50:25.159 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:50:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:25.160+0000 7f95aab4b640 1 -- 172.21.15.16:0/1666006349 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f959006c2e0 msgr2=0x7f959006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:25.160+0000 7f95aab4b640 1 --2- 172.21.15.16:0/1666006349 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f959006c2e0 0x7f959006e7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9594034000 tx=0x7f959402e000 comp rx=0 tx=0).stop 2024-09-19T07:50:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:25.160+0000 7f95aab4b640 1 -- 172.21.15.16:0/1666006349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a415e680 msgr2=0x7f95a4151850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:25.160+0000 7f95aab4b640 1 --2- 172.21.15.16:0/1666006349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a415e680 0x7f95a4151850 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f959c00c780 tx=0x7f959c00cc40 comp rx=0 tx=0).stop 2024-09-19T07:50:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:25.160+0000 7f95aab4b640 1 -- 172.21.15.16:0/1666006349 shutdown_connections 2024-09-19T07:50:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:25.160+0000 7f95aab4b640 1 --2- 172.21.15.16:0/1666006349 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f959006c2e0 0x7f959006e7a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:25.160+0000 7f95aab4b640 1 --2- 172.21.15.16:0/1666006349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95a415e680 0x7f95a4151850 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:25.160+0000 7f95aab4b640 1 --2- 172.21.15.16:0/1666006349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95a415e2a0 0x7f95a40cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:25.160+0000 7f95aab4b640 1 --2- 172.21.15.16:0/1666006349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95a406d960 0x7f95a40c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:25.160+0000 7f95aab4b640 1 -- 172.21.15.16:0/1666006349 >> 172.21.15.16:0/1666006349 conn(0x7f95a409d760 msgr2=0x7f95a415fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:25.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:25.160+0000 7f95aab4b640 1 -- 172.21.15.16:0/1666006349 shutdown_connections 2024-09-19T07:50:25.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:25.160+0000 7f95aab4b640 1 -- 172.21.15.16:0/1666006349 wait complete. 2024-09-19T07:50:25.172 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:50:25.172 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:50:25.172 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T07:50:25.172 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T07:50:25.172 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:50:25.173 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:50:25.173 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:50:25.173 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:50:25.173 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:50:25.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:25.173 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T07:50:25.190 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:50:25.190 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:50:25.190 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:50:25.190 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:50:25.190 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:50:25.191 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:50:25.191 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:50:25.191 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:50:25.191 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T07:50:25.191 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:50:25.191 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:50:25.191 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T07:50:25.192 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:50:25.192 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:50:25.192 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T07:50:25.192 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T07:50:25.192 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42364sec preferred_lft 42364sec 2024-09-19T07:50:25.192 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T07:50:25.192 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:50:25.192 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:50:25.192 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:50:25.192 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T07:50:25.192 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:50:25.193 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:50:25.193 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:50:25.193 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:50:25.193 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::32:7bff:fe92:8895/64 scope link 2024-09-19T07:50:25.193 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:50:25.193 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:50:25.193 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether e6:fd:21:6a:43:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:50:25.193 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e4fd:21ff:fe6a:4369/64 scope link 2024-09-19T07:50:25.193 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:50:25.193 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:50:25.194 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:50:25.194 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::80f4:81ff:fe7d:b7f0/64 scope link 2024-09-19T07:50:25.194 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:50:25.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:25.194 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:50:25.255 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:50:25.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:25.256 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T07:50:25.313 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T07:50:25.314 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:50:25.314 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T07:50:25.314 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T07:50:25.314 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:50:25.314 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:50:25.314 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:50:25.314 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T07:50:25.314 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T07:50:25.315 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T07:50:25.315 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:50:25.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:50:25.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:50:25.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:25.444 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:50:25.461 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T07:50:25.461 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:50:25.461 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:50:25.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:25.461 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-19T07:50:25.527 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T07:50:25.528 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:50:25.528 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T07:50:25.584 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:50:25.585 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:50:25.585 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:50:25.641 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:50:25.708 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:50:25.766 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:50:25.766 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:50:25.766 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:50:25.766 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:50:25.766 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:50:25.766 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:50:25.766 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:50:25.766 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:50:25.766 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:50:25.767 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:50:25.767 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:50:25.767 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:50:25.767 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:50:25.767 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:50:25.767 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:50:25.767 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:50:25.767 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:50:25.767 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:50:25.767 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:50:25.767 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:50:25.767 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:50:25.767 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:50:25.768 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:50:25.768 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:50:25.768 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:50:25.768 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:50:25.768 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:50:25.768 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:50:25.768 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:50:25.768 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:50:25.768 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:50:25.768 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:50:25.768 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:50:25.768 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:50:25.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:25.769 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:50:25.822 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:50:25.822 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T07:50:25.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:50:25.891 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:50:25.948 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:50:25.948 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:50:25.948 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:50:25.948 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:50:25.949 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:50:25.949 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:50:25.949 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:50:25.949 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:50:25.949 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:50:25.949 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:50:25.949 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:50:25.949 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:50:25.949 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:50:25.950 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:50:25.950 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:50:25.950 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:50:25.950 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:50:25.950 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:50:25.950 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:50:25.950 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:50:25.950 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:50:25.950 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:50:25.950 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:50:25.950 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:50:25.950 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:50:25.950 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:50:25.950 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:50:25.951 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:50:25.951 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:50:25.951 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:50:25.951 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:50:25.951 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:50:25.951 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:50:25.951 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:50:25.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:25.951 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:50:25.952 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-19T07:50:25.949+0000 7f7ccc84b480 -1 init, newargv = 0x55a279e79840 newargc=15 2024-09-19T07:50:25.952 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[61430]: starting ceph client 2024-09-19T07:50:25.954 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[61430]: starting fuse 2024-09-19T07:50:25.968 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:50:25.969 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-19T07:50:25.969 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:50:25.969 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:50:25.969 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:50:25.969 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:50:25.969 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:50:25.969 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:50:25.969 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:50:25.970 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:50:25.970 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:50:25.970 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:50:25.970 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:50:25.970 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:50:25.970 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:50:25.970 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:50:25.970 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:50:25.970 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:50:25.970 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:50:25.970 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:50:25.970 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:50:25.970 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:50:25.970 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:50:25.970 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:50:25.971 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:50:25.971 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:50:25.971 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:50:25.971 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:50:25.971 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:50:26.089 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.61430.asok 2024-09-19T07:50:26.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:50:26.100 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.61430.asok 2024-09-19T07:50:26.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:26.100 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.61430.asok status 2024-09-19T07:50:26.236 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:50:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:50:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:50:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:50:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:50:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:50:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:50:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "61430", 2024-09-19T07:50:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:50:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-19T07:50:26.237 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-19T07:50:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 6576, 2024-09-19T07:50:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:50:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:50:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:50:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 6576 2024-09-19T07:50:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:50:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:50:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:50:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1620886677 2024-09-19T07:50:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:26.238 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:50:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:50:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:50:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1620886677 2024-09-19T07:50:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.6576 192.168.0.1:0/1620886677", 2024-09-19T07:50:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/1620886677", 2024-09-19T07:50:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-19T07:50:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 47, 2024-09-19T07:50:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 93, 2024-09-19T07:50:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:50:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:50:26.239 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:50:26.239 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:50:26.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:26.245 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-19T07:50:26.245 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T07:50:26.262 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-19T07:50:26.262 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T07:50:26.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:26.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:50:26.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:50:26.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.471+0000 7f147f0af640 1 Processor -- start 2024-09-19T07:50:26.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.471+0000 7f147f0af640 1 -- start start 2024-09-19T07:50:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.472+0000 7f147f0af640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f147815e4f0 0x7f14780b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.472+0000 7f147f0af640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1478070930 0x7f14780b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.472+0000 7f147f0af640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14780b5590 con 0x7f14780add30 2024-09-19T07:50:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.472+0000 7f147f0af640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f1478070ea0 con 0x7f1478070930 2024-09-19T07:50:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.472+0000 7f147f0af640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f147806e8c0 con 0x7f147815e4f0 2024-09-19T07:50:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.472+0000 7f147d64c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1478070930 0x7f14780b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.472+0000 7f147ce4b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f14780add30 0x7f14780ae110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41224/0 (socket says 172.21.15.16:41224) 2024-09-19T07:50:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.472+0000 7f147ce4b640 1 -- 172.21.15.16:0/3096490494 learned_addr learned my addr 172.21.15.16:0/3096490494 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.472+0000 7f1477fff640 1 --2- 172.21.15.16:0/3096490494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f147815e4f0 0x7f14780b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:26.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.472+0000 7f147d64c640 1 -- 172.21.15.16:0/3096490494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f147815e4f0 msgr2=0x7f14780b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:26.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.472+0000 7f147d64c640 1 --2- 172.21.15.16:0/3096490494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f147815e4f0 0x7f14780b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:26.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.472+0000 7f147d64c640 1 -- 172.21.15.16:0/3096490494 >> v1:172.21.15.16:6789/0 conn(0x7f14780add30 legacy=0x7f14780ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:26.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.472+0000 7f147d64c640 1 -- 172.21.15.16:0/3096490494 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147806eba0 con 0x7f1478070930 2024-09-19T07:50:26.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.472+0000 7f14777fe640 1 -- 172.21.15.16:0/3096490494 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 280220797 0 0) 0x7f14780b5590 con 0x7f14780add30 2024-09-19T07:50:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.473+0000 7f147d64c640 1 --2- 172.21.15.16:0/3096490494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1478070930 0x7f14780b00f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f14700097a0 tx=0x7f147002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6646085f66c3732 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.473+0000 7f14777fe640 1 -- 172.21.15.16:0/3096490494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470037070 con 0x7f1478070930 2024-09-19T07:50:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.473+0000 7f14777fe640 1 -- 172.21.15.16:0/3096490494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1470032440 con 0x7f1478070930 2024-09-19T07:50:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.473+0000 7f14777fe640 1 -- 172.21.15.16:0/3096490494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470032810 con 0x7f1478070930 2024-09-19T07:50:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.474+0000 7f147f0af640 1 -- 172.21.15.16:0/3096490494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1478070930 msgr2=0x7f14780b00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.474+0000 7f147f0af640 1 --2- 172.21.15.16:0/3096490494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1478070930 0x7f14780b00f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f14700097a0 tx=0x7f147002a180 comp rx=0 tx=0).stop 2024-09-19T07:50:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.474+0000 7f147f0af640 1 -- 172.21.15.16:0/3096490494 shutdown_connections 2024-09-19T07:50:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.474+0000 7f147f0af640 1 --2- 172.21.15.16:0/3096490494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1478070930 0x7f14780b00f0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.474+0000 7f147f0af640 1 --2- 172.21.15.16:0/3096490494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f147815e4f0 0x7f14780b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.474+0000 7f147f0af640 1 -- 172.21.15.16:0/3096490494 >> 172.21.15.16:0/3096490494 conn(0x7f147806bc10 msgr2=0x7f147806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.474+0000 7f147f0af640 1 -- 172.21.15.16:0/3096490494 shutdown_connections 2024-09-19T07:50:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.474+0000 7f147f0af640 1 -- 172.21.15.16:0/3096490494 wait complete. 2024-09-19T07:50:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.475+0000 7f147f0af640 1 Processor -- start 2024-09-19T07:50:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.475+0000 7f147f0af640 1 -- start start 2024-09-19T07:50:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.476+0000 7f147f0af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1478070930 0x7f1478157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.476+0000 7f147f0af640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14780add30 0x7f1478157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.476+0000 7f147f0af640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f147815e4f0 0x7f147814aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.476+0000 7f147f0af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f147806f3a0 con 0x7f1478070930 2024-09-19T07:50:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.476+0000 7f147f0af640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f14780b0630 con 0x7f14780add30 2024-09-19T07:50:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.476+0000 7f147f0af640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f147806f520 con 0x7f147815e4f0 2024-09-19T07:50:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.476+0000 7f147ce4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1478070930 0x7f1478157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.476+0000 7f147ce4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1478070930 0x7f1478157820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50666/0 (socket says 172.21.15.16:50666) 2024-09-19T07:50:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.476+0000 7f147ce4b640 1 -- 172.21.15.16:0/926829025 learned_addr learned my addr 172.21.15.16:0/926829025 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.476+0000 7f147d64c640 1 --2- 172.21.15.16:0/926829025 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f147815e4f0 0x7f147814aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.476+0000 7f1477fff640 1 --2- 172.21.15.16:0/926829025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14780add30 0x7f1478157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.476+0000 7f147ce4b640 1 -- 172.21.15.16:0/926829025 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f147815e4f0 msgr2=0x7f147814aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.476+0000 7f147ce4b640 1 --2- 172.21.15.16:0/926829025 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f147815e4f0 0x7f147814aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.476+0000 7f147ce4b640 1 -- 172.21.15.16:0/926829025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14780add30 msgr2=0x7f1478157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.476+0000 7f147ce4b640 1 --2- 172.21.15.16:0/926829025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14780add30 0x7f1478157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.476+0000 7f147ce4b640 1 -- 172.21.15.16:0/926829025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14780a9ca0 con 0x7f1478070930 2024-09-19T07:50:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.476+0000 7f1477fff640 1 --2- 172.21.15.16:0/926829025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14780add30 0x7f1478157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.477+0000 7f147ce4b640 1 --2- 172.21.15.16:0/926829025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1478070930 0x7f1478157820 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f146800c4d0 tx=0x7f146800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.477+0000 7f14757fa640 1 -- 172.21.15.16:0/926829025 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468018070 con 0x7f1478070930 2024-09-19T07:50:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.477+0000 7f14757fa640 1 -- 172.21.15.16:0/926829025 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1468007500 con 0x7f1478070930 2024-09-19T07:50:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.477+0000 7f147f0af640 1 -- 172.21.15.16:0/926829025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14780a99c0 con 0x7f1478070930 2024-09-19T07:50:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.477+0000 7f14757fa640 1 -- 172.21.15.16:0/926829025 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468004a20 con 0x7f1478070930 2024-09-19T07:50:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.477+0000 7f147f0af640 1 -- 172.21.15.16:0/926829025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1478148bc0 con 0x7f1478070930 2024-09-19T07:50:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.478+0000 7f147f0af640 1 -- 172.21.15.16:0/926829025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14780b00f0 con 0x7f1478070930 2024-09-19T07:50:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.478+0000 7f14757fa640 1 -- 172.21.15.16:0/926829025 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f146801b070 con 0x7f1478070930 2024-09-19T07:50:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.479+0000 7f14757fa640 1 --2- 172.21.15.16:0/926829025 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f145806c2b0 0x7f145806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.479+0000 7f14757fa640 1 -- 172.21.15.16:0/926829025 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f146808a840 con 0x7f1478070930 2024-09-19T07:50:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.479+0000 7f1477fff640 1 --2- 172.21.15.16:0/926829025 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f145806c2b0 0x7f145806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.480+0000 7f1477fff640 1 --2- 172.21.15.16:0/926829025 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f145806c2b0 0x7f145806e770 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f147814ee50 tx=0x7f14600039d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:26.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.483+0000 7f14757fa640 1 -- 172.21.15.16:0/926829025 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f14680587d0 con 0x7f1478070930 2024-09-19T07:50:26.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.694+0000 7f147f0af640 1 -- 172.21.15.16:0/926829025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f14780b1890 con 0x7f1478070930 2024-09-19T07:50:26.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.695+0000 7f14757fa640 1 -- 172.21.15.16:0/926829025 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) ==== 53+0+73 (secure 0 0 0) 0x7f146805cad0 con 0x7f1478070930 2024-09-19T07:50:26.695 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:50:26.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.697+0000 7f147f0af640 1 -- 172.21.15.16:0/926829025 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f145806c2b0 msgr2=0x7f145806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:26.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.697+0000 7f147f0af640 1 --2- 172.21.15.16:0/926829025 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f145806c2b0 0x7f145806e770 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f147814ee50 tx=0x7f14600039d0 comp rx=0 tx=0).stop 2024-09-19T07:50:26.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.697+0000 7f147f0af640 1 -- 172.21.15.16:0/926829025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1478070930 msgr2=0x7f1478157820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:26.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.697+0000 7f147f0af640 1 --2- 172.21.15.16:0/926829025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1478070930 0x7f1478157820 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f146800c4d0 tx=0x7f146800c990 comp rx=0 tx=0).stop 2024-09-19T07:50:26.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.697+0000 7f147f0af640 1 -- 172.21.15.16:0/926829025 shutdown_connections 2024-09-19T07:50:26.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.697+0000 7f147f0af640 1 --2- 172.21.15.16:0/926829025 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f145806c2b0 0x7f145806e770 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:26.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.697+0000 7f147f0af640 1 --2- 172.21.15.16:0/926829025 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f147815e4f0 0x7f147814aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:26.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.697+0000 7f147f0af640 1 --2- 172.21.15.16:0/926829025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f14780add30 0x7f1478157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:26.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.697+0000 7f147f0af640 1 --2- 172.21.15.16:0/926829025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1478070930 0x7f1478157820 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:26.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.697+0000 7f147f0af640 1 -- 172.21.15.16:0/926829025 >> 172.21.15.16:0/926829025 conn(0x7f147806bc10 msgr2=0x7f1478074ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:26.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.697+0000 7f147f0af640 1 -- 172.21.15.16:0/926829025 shutdown_connections 2024-09-19T07:50:26.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:26.697+0000 7f147f0af640 1 -- 172.21.15.16:0/926829025 wait complete. 2024-09-19T07:50:26.709 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:50:26.709 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:50:26.709 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-09-19T07:50:26.709 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T07:50:26.709 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T07:50:26.709 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:50:26.710 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:50:26.710 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:50:26.710 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:50:26.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:26.710 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-09-19T07:50:26.730 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:50:26.730 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:50:26.730 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:50:26.730 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:50:26.730 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-09-19T07:50:26.730 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:50:26.730 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:50:26.731 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-09-19T07:50:26.731 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2024-09-19T07:50:26.731 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:50:26.731 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-09-19T07:50:26.731 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2024-09-19T07:50:26.731 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:50:26.731 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-09-19T07:50:26.731 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:50:26.731 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-09-19T07:50:26.731 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T07:50:26.731 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42330sec preferred_lft 42330sec 2024-09-19T07:50:26.731 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link noprefixroute 2024-09-19T07:50:26.731 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:50:26.731 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:50:26.732 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:50:26.732 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:50:26.732 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:50:26.732 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::e47f:32ff:fe68:31ca/64 scope link 2024-09-19T07:50:26.732 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:50:26.732 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:50:26.732 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether f6:1f:22:44:9d:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:50:26.732 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::f41f:22ff:fe44:9dcd/64 scope link 2024-09-19T07:50:26.732 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:50:26.732 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:50:26.732 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:50:26.732 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::8459:eeff:fe21:a113/64 scope link 2024-09-19T07:50:26.732 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:50:26.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:26.733 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:50:26.799 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-09-19T07:50:26.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:26.802 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-09-19T07:50:26.861 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-09-19T07:50:26.861 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:50:26.862 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T07:50:26.862 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T07:50:26.862 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:50:26.862 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:50:26.862 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-19T07:50:26.862 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T07:50:26.862 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T07:50:26.862 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T07:50:26.862 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-19T07:50:26.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:50:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:50:26.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:50:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:50:26.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:26.999 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-09-19T07:50:27.015 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T07:50:27.015 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T07:50:27.015 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:50:27.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:27.015 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2024-09-19T07:50:27.081 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T07:50:27.081 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:50:27.081 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T07:50:27.136 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:50:27.137 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:50:27.137 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:50:27.192 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:50:27.257 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:50:27.313 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:50:27.313 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:50:27.313 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:50:27.313 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:50:27.314 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:50:27.314 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:50:27.314 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:50:27.314 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:50:27.314 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:50:27.314 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:50:27.314 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:50:27.314 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:50:27.314 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:50:27.314 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:50:27.314 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:50:27.314 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:50:27.314 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:50:27.315 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:50:27.315 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:50:27.315 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:50:27.315 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:50:27.315 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:50:27.315 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:50:27.315 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:50:27.315 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:50:27.315 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:50:27.315 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:50:27.315 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:50:27.315 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:50:27.315 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:50:27.316 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:50:27.316 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:50:27.316 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:50:27.316 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:50:27.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:27.316 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:50:27.371 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:50:27.372 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T07:50:27.414 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:50:27.439 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:50:27.496 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:50:27.496 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:50:27.496 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:50:27.496 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:50:27.496 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:50:27.496 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:50:27.496 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:50:27.496 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:50:27.496 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:50:27.497 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:50:27.497 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:50:27.497 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:50:27.497 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:50:27.497 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:50:27.497 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:50:27.497 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:50:27.497 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:50:27.497 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:50:27.497 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:50:27.497 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:50:27.498 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:50:27.498 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:50:27.498 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:50:27.498 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:50:27.498 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:50:27.498 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:50:27.498 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:50:27.498 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:50:27.498 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:50:27.498 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:50:27.498 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:50:27.498 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:50:27.498 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:50:27.499 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:50:27.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:27.499 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:50:27.500 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2024-09-19T07:50:27.496+0000 7f3407f1d480 -1 init, newargv = 0x55e5a1521a30 newargc=15 2024-09-19T07:50:27.500 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[46954]: starting ceph client 2024-09-19T07:50:27.501 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[46954]: starting fuse 2024-09-19T07:50:27.515 INFO:teuthology.orchestra.run.smithi146.stdout:42 2024-09-19T07:50:27.516 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-19T07:50:27.516 DEBUG:teuthology.orchestra.run.smithi146:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:50:27.516 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2024-09-19T07:50:27.516 DEBUG:teuthology.orchestra.run.smithi146:> import re 2024-09-19T07:50:27.516 DEBUG:teuthology.orchestra.run.smithi146:> import os 2024-09-19T07:50:27.516 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2024-09-19T07:50:27.516 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:50:27.516 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2024-09-19T07:50:27.516 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T07:50:27.516 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2024-09-19T07:50:27.517 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T07:50:27.517 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:50:27.517 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2024-09-19T07:50:27.517 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2024-09-19T07:50:27.517 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2024-09-19T07:50:27.517 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2024-09-19T07:50:27.517 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:50:27.517 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2024-09-19T07:50:27.517 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:50:27.517 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:50:27.517 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:50:27.517 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2024-09-19T07:50:27.517 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2024-09-19T07:50:27.517 DEBUG:teuthology.orchestra.run.smithi146:> return f 2024-09-19T07:50:27.518 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:50:27.518 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:50:27.518 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2024-09-19T07:50:27.518 DEBUG:teuthology.orchestra.run.smithi146:> ' 2024-09-19T07:50:27.634 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.46954.asok 2024-09-19T07:50:27.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:50:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:50:27.645 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.46954.asok 2024-09-19T07:50:27.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:27.645 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.46954.asok status 2024-09-19T07:50:27.779 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-19T07:50:27.779 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2024-09-19T07:50:27.779 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:50:27.779 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:50:27.779 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2024-09-19T07:50:27.779 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2024-09-19T07:50:27.779 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T07:50:27.779 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "46954", 2024-09-19T07:50:27.779 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2024-09-19T07:50:27.779 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:50:27.780 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2024-09-19T07:50:27.780 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2024-09-19T07:50:27.780 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 6226, 2024-09-19T07:50:27.780 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2024-09-19T07:50:27.780 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2024-09-19T07:50:27.780 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2024-09-19T07:50:27.780 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 6226 2024-09-19T07:50:27.780 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:50:27.780 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:50:27.780 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:50:27.780 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:50:27.780 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 4223664118 2024-09-19T07:50:27.780 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-19T07:50:27.781 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:50:27.781 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:50:27.781 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:50:27.781 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:50:27.781 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 4223664118 2024-09-19T07:50:27.781 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:50:27.781 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.6226 192.168.0.1:0/4223664118", 2024-09-19T07:50:27.781 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/4223664118", 2024-09-19T07:50:27.781 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2024-09-19T07:50:27.781 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 47, 2024-09-19T07:50:27.781 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 93, 2024-09-19T07:50:27.781 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:50:27.781 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2024-09-19T07:50:27.782 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2024-09-19T07:50:27.782 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-19T07:50:27.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:27.788 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2024-09-19T07:50:27.788 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T07:50:27.804 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2024-09-19T07:50:27.804 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T07:50:27.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:27.804 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:50:27.869 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-09-19T07:50:27.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.988+0000 7f375d896640 1 Processor -- start 2024-09-19T07:50:27.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.988+0000 7f375d896640 1 -- start start 2024-09-19T07:50:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.989+0000 7f375d896640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f375815c270 0x7f375806c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.989+0000 7f375d896640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37580cfda0 con 0x7f37580ae270 2024-09-19T07:50:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.989+0000 7f375d896640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37580ccf80 con 0x7f37580cf680 2024-09-19T07:50:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.989+0000 7f375d896640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f37580cd260 con 0x7f375815c270 2024-09-19T07:50:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.989+0000 7f3756ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f375815c270 0x7f375806c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.989+0000 7f37567fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f37580ae270 0x7f37580cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41226/0 (socket says 172.21.15.16:41226) 2024-09-19T07:50:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.989+0000 7f37567fc640 1 -- 172.21.15.16:0/1498320712 learned_addr learned my addr 172.21.15.16:0/1498320712 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.989+0000 7f3756ffd640 1 -- 172.21.15.16:0/1498320712 >> v1:172.21.15.146:6789/0 conn(0x7f37580cf680 legacy=0x7f37580cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:27.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.989+0000 7f3756ffd640 1 -- 172.21.15.16:0/1498320712 >> v1:172.21.15.16:6789/0 conn(0x7f37580ae270 legacy=0x7f37580cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.989+0000 7f3755ffb640 1 -- 172.21.15.16:0/1498320712 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 497019294 0 0) 0x7f3740001110 con 0x7f37580ae270 2024-09-19T07:50:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.990+0000 7f3756ffd640 1 -- 172.21.15.16:0/1498320712 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37580cd540 con 0x7f375815c270 2024-09-19T07:50:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.990+0000 7f3756ffd640 1 --2- 172.21.15.16:0/1498320712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375815c270 0x7f375806c620 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f37480097a0 tx=0x7f374802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96735d9b86420751 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.990+0000 7f3755ffb640 1 -- 172.21.15.16:0/1498320712 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3748037070 con 0x7f375815c270 2024-09-19T07:50:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.990+0000 7f3755ffb640 1 -- 172.21.15.16:0/1498320712 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3748032440 con 0x7f375815c270 2024-09-19T07:50:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.990+0000 7f3755ffb640 1 -- 172.21.15.16:0/1498320712 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3748032810 con 0x7f375815c270 2024-09-19T07:50:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.991+0000 7f375d896640 1 -- 172.21.15.16:0/1498320712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375815c270 msgr2=0x7f375806c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.991+0000 7f375d896640 1 --2- 172.21.15.16:0/1498320712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375815c270 0x7f375806c620 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f37480097a0 tx=0x7f374802a1a0 comp rx=0 tx=0).stop 2024-09-19T07:50:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.991+0000 7f375d896640 1 -- 172.21.15.16:0/1498320712 shutdown_connections 2024-09-19T07:50:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.991+0000 7f375d896640 1 --2- 172.21.15.16:0/1498320712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375815c270 0x7f375806c620 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.991+0000 7f375d896640 1 -- 172.21.15.16:0/1498320712 >> 172.21.15.16:0/1498320712 conn(0x7f375809a9f0 msgr2=0x7f37580b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.991+0000 7f375d896640 1 -- 172.21.15.16:0/1498320712 shutdown_connections 2024-09-19T07:50:27.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.991+0000 7f375d896640 1 -- 172.21.15.16:0/1498320712 wait complete. 2024-09-19T07:50:27.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.992+0000 7f375d896640 1 Processor -- start 2024-09-19T07:50:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.992+0000 7f375d896640 1 -- start start 2024-09-19T07:50:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.993+0000 7f375d896640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37580ae270 0x7f37581f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.993+0000 7f375d896640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37580cf680 0x7f37580b5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.993+0000 7f375d896640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375815c270 0x7f37580b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.993+0000 7f375d896640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f375815c7d0 con 0x7f375815c270 2024-09-19T07:50:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.993+0000 7f375d896640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f375815c650 con 0x7f37580ae270 2024-09-19T07:50:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.993+0000 7f375d896640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f37580ccba0 con 0x7f37580cf680 2024-09-19T07:50:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.993+0000 7f37577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375815c270 0x7f37580b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.993+0000 7f3756ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37580ae270 0x7f37581f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.993+0000 7f37577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375815c270 0x7f37580b5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50682/0 (socket says 172.21.15.16:50682) 2024-09-19T07:50:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.993+0000 7f37577fe640 1 -- 172.21.15.16:0/17548720 learned_addr learned my addr 172.21.15.16:0/17548720 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:27.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.993+0000 7f37567fc640 1 --2- 172.21.15.16:0/17548720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37580cf680 0x7f37580b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.994+0000 7f37577fe640 1 -- 172.21.15.16:0/17548720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37580cf680 msgr2=0x7f37580b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.994+0000 7f37577fe640 1 --2- 172.21.15.16:0/17548720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37580cf680 0x7f37580b5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.994+0000 7f37577fe640 1 -- 172.21.15.16:0/17548720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37580ae270 msgr2=0x7f37581f32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.994+0000 7f37577fe640 1 --2- 172.21.15.16:0/17548720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37580ae270 0x7f37581f32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.994+0000 7f37577fe640 1 -- 172.21.15.16:0/17548720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37580b6200 con 0x7f375815c270 2024-09-19T07:50:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.994+0000 7f3756ffd640 1 --2- 172.21.15.16:0/17548720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37580ae270 0x7f37581f32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.994+0000 7f37577fe640 1 --2- 172.21.15.16:0/17548720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375815c270 0x7f37580b5cc0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f375000c7a0 tx=0x7f375000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.994+0000 7f374ffff640 1 -- 172.21.15.16:0/17548720 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750003b70 con 0x7f375815c270 2024-09-19T07:50:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.994+0000 7f374ffff640 1 -- 172.21.15.16:0/17548720 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3750003d30 con 0x7f375815c270 2024-09-19T07:50:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.994+0000 7f375d896640 1 -- 172.21.15.16:0/17548720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37580b49c0 con 0x7f375815c270 2024-09-19T07:50:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.994+0000 7f374ffff640 1 -- 172.21.15.16:0/17548720 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375000aa90 con 0x7f375815c270 2024-09-19T07:50:27.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.994+0000 7f375d896640 1 -- 172.21.15.16:0/17548720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37580b4f80 con 0x7f375815c270 2024-09-19T07:50:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.995+0000 7f375d896640 1 -- 172.21.15.16:0/17548720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37580cdf20 con 0x7f375815c270 2024-09-19T07:50:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.996+0000 7f374ffff640 1 -- 172.21.15.16:0/17548720 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f375000ac30 con 0x7f375815c270 2024-09-19T07:50:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.996+0000 7f374ffff640 1 --2- 172.21.15.16:0/17548720 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f373806c2b0 0x7f373806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:27.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.996+0000 7f374ffff640 1 -- 172.21.15.16:0/17548720 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f375008b3d0 con 0x7f375815c270 2024-09-19T07:50:27.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.997+0000 7f3756ffd640 1 --2- 172.21.15.16:0/17548720 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f373806c2b0 0x7f373806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:27.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:27.997+0000 7f3756ffd640 1 --2- 172.21.15.16:0/17548720 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f373806c2b0 0x7f373806e770 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3748034000 tx=0x7f374802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:28.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.001+0000 7f374ffff640 1 -- 172.21.15.16:0/17548720 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f37580b6200 con 0x7f375815c270 2024-09-19T07:50:28.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.177+0000 7f375d896640 1 -- 172.21.15.16:0/17548720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7f37580aeed0 con 0x7f375815c270 2024-09-19T07:50:28.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.180+0000 7f374ffff640 1 -- 172.21.15.16:0/17548720 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v32)=0 v32) ==== 121+0+0 (secure 0 0 0) 0x7f3750059360 con 0x7f375815c270 2024-09-19T07:50:28.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.182+0000 7f375d896640 1 -- 172.21.15.16:0/17548720 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f373806c2b0 msgr2=0x7f373806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.182+0000 7f375d896640 1 --2- 172.21.15.16:0/17548720 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f373806c2b0 0x7f373806e770 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3748034000 tx=0x7f374802e000 comp rx=0 tx=0).stop 2024-09-19T07:50:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.182+0000 7f375d896640 1 -- 172.21.15.16:0/17548720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375815c270 msgr2=0x7f37580b5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.182+0000 7f375d896640 1 --2- 172.21.15.16:0/17548720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375815c270 0x7f37580b5cc0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f375000c7a0 tx=0x7f375000cc60 comp rx=0 tx=0).stop 2024-09-19T07:50:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.182+0000 7f375d896640 1 -- 172.21.15.16:0/17548720 shutdown_connections 2024-09-19T07:50:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.182+0000 7f375d896640 1 --2- 172.21.15.16:0/17548720 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f373806c2b0 0x7f373806e770 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.182+0000 7f375d896640 1 --2- 172.21.15.16:0/17548720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375815c270 0x7f37580b5cc0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.182+0000 7f375d896640 1 --2- 172.21.15.16:0/17548720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37580cf680 0x7f37580b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.182+0000 7f375d896640 1 --2- 172.21.15.16:0/17548720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37580ae270 0x7f37581f32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.182+0000 7f375d896640 1 -- 172.21.15.16:0/17548720 >> 172.21.15.16:0/17548720 conn(0x7f375809a9f0 msgr2=0x7f37580af550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.182+0000 7f375d896640 1 -- 172.21.15.16:0/17548720 shutdown_connections 2024-09-19T07:50:28.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.183+0000 7f375d896640 1 -- 172.21.15.16:0/17548720 wait complete. 2024-09-19T07:50:28.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 200 2024-09-19T07:50:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.299+0000 7f0bae8d9640 1 Processor -- start 2024-09-19T07:50:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.299+0000 7f0bae8d9640 1 -- start start 2024-09-19T07:50:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.299+0000 7f0bae8d9640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0ba815f0d0 0x7f0ba815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.299+0000 7f0bae8d9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ba80d4130 0x7f0ba80d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.299+0000 7f0bae8d9640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ba80d4ae0 con 0x7f0ba80d0070 2024-09-19T07:50:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.299+0000 7f0bae8d9640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f0ba80d1b10 con 0x7f0ba815f0d0 2024-09-19T07:50:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.299+0000 7f0bae8d9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0ba80cf2b0 con 0x7f0ba80d4130 2024-09-19T07:50:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.300+0000 7f0bace76640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ba80d4130 0x7f0ba80d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.300+0000 7f0bace76640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ba80d4130 0x7f0ba80d45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:44712/0 (socket says 172.21.15.16:44712) 2024-09-19T07:50:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.300+0000 7f0bace76640 1 -- 172.21.15.16:0/3707677041 learned_addr learned my addr 172.21.15.16:0/3707677041 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.300+0000 7f0ba77fe640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f0ba80d0070 0x7f0ba80d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:35766/0 (socket says 172.21.15.16:35766) 2024-09-19T07:50:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.300+0000 7f0ba7fff640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0ba815f0d0 0x7f0ba815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.300+0000 7f0bace76640 1 -- 172.21.15.16:0/3707677041 >> v2:172.21.15.146:3300/0 conn(0x7f0ba815f0d0 msgr2=0x7f0ba815f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.300+0000 7f0bace76640 1 --2- 172.21.15.16:0/3707677041 >> v2:172.21.15.146:3300/0 conn(0x7f0ba815f0d0 0x7f0ba815f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.300+0000 7f0bace76640 1 -- 172.21.15.16:0/3707677041 >> v1:172.21.15.146:6789/0 conn(0x7f0ba80d0070 legacy=0x7f0ba80d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.300+0000 7f0bace76640 1 -- 172.21.15.16:0/3707677041 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba80cf590 con 0x7f0ba80d4130 2024-09-19T07:50:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.301+0000 7f0bace76640 1 --2- 172.21.15.16:0/3707677041 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ba80d4130 0x7f0ba80d45a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009e50 tx=0x7f0ba002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3d8094529345652 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.301+0000 7f0ba6ffd640 1 -- 172.21.15.16:0/3707677041 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0036070 con 0x7f0ba80d4130 2024-09-19T07:50:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.301+0000 7f0ba6ffd640 1 -- 172.21.15.16:0/3707677041 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba0031600 con 0x7f0ba80d4130 2024-09-19T07:50:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.301+0000 7f0ba6ffd640 1 -- 172.21.15.16:0/3707677041 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba00319d0 con 0x7f0ba80d4130 2024-09-19T07:50:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.302+0000 7f0bae8d9640 1 -- 172.21.15.16:0/3707677041 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ba80d4130 msgr2=0x7f0ba80d45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.302+0000 7f0bae8d9640 1 --2- 172.21.15.16:0/3707677041 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ba80d4130 0x7f0ba80d45a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009e50 tx=0x7f0ba002a680 comp rx=0 tx=0).stop 2024-09-19T07:50:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.302+0000 7f0bae8d9640 1 -- 172.21.15.16:0/3707677041 shutdown_connections 2024-09-19T07:50:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.302+0000 7f0bae8d9640 1 --2- 172.21.15.16:0/3707677041 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ba80d4130 0x7f0ba80d45a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.302+0000 7f0bae8d9640 1 --2- 172.21.15.16:0/3707677041 >> v2:172.21.15.146:3300/0 conn(0x7f0ba815f0d0 0x7f0ba815f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.302+0000 7f0bae8d9640 1 -- 172.21.15.16:0/3707677041 >> 172.21.15.16:0/3707677041 conn(0x7f0ba809d760 msgr2=0x7f0ba808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.302+0000 7f0bae8d9640 1 -- 172.21.15.16:0/3707677041 shutdown_connections 2024-09-19T07:50:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.302+0000 7f0bae8d9640 1 -- 172.21.15.16:0/3707677041 wait complete. 2024-09-19T07:50:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.303+0000 7f0bae8d9640 1 Processor -- start 2024-09-19T07:50:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.303+0000 7f0bae8d9640 1 -- start start 2024-09-19T07:50:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.304+0000 7f0bae8d9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ba80d0070 0x7f0ba80ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.304+0000 7f0bae8d9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ba80d4130 0x7f0ba814e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.304+0000 7f0bae8d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ba815f0d0 0x7f0ba80cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.304+0000 7f0bae8d9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba80cef60 con 0x7f0ba815f0d0 2024-09-19T07:50:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.304+0000 7f0bae8d9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba80cfb50 con 0x7f0ba80d4130 2024-09-19T07:50:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.304+0000 7f0bae8d9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0ba80b2f70 con 0x7f0ba80d0070 2024-09-19T07:50:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.304+0000 7f0bace76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ba815f0d0 0x7f0ba80cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.304+0000 7f0ba7fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ba80d0070 0x7f0ba80ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.304+0000 7f0bace76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ba815f0d0 0x7f0ba80cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50696/0 (socket says 172.21.15.16:50696) 2024-09-19T07:50:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.304+0000 7f0ba7fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ba80d0070 0x7f0ba80ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:44728/0 (socket says 172.21.15.16:44728) 2024-09-19T07:50:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.304+0000 7f0ba77fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ba80d4130 0x7f0ba814e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.304+0000 7f0bace76640 1 -- 172.21.15.16:0/3540603065 learned_addr learned my addr 172.21.15.16:0/3540603065 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.304+0000 7f0bace76640 1 -- 172.21.15.16:0/3540603065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ba80d0070 msgr2=0x7f0ba80ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.304+0000 7f0bace76640 1 --2- 172.21.15.16:0/3540603065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ba80d0070 0x7f0ba80ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.304+0000 7f0bace76640 1 -- 172.21.15.16:0/3540603065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ba80d4130 msgr2=0x7f0ba814e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.304+0000 7f0bace76640 1 --2- 172.21.15.16:0/3540603065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ba80d4130 0x7f0ba814e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.305+0000 7f0bace76640 1 -- 172.21.15.16:0/3540603065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba80cbeb0 con 0x7f0ba815f0d0 2024-09-19T07:50:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.305+0000 7f0ba7fff640 1 --2- 172.21.15.16:0/3540603065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ba80d0070 0x7f0ba80ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.305+0000 7f0bace76640 1 --2- 172.21.15.16:0/3540603065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ba815f0d0 0x7f0ba80cb4b0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0034040 tx=0x7f0ba0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.305+0000 7f0ba4ff9640 1 -- 172.21.15.16:0/3540603065 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0036050 con 0x7f0ba815f0d0 2024-09-19T07:50:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.305+0000 7f0ba4ff9640 1 -- 172.21.15.16:0/3540603065 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba0042070 con 0x7f0ba815f0d0 2024-09-19T07:50:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.305+0000 7f0bae8d9640 1 -- 172.21.15.16:0/3540603065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba80cc170 con 0x7f0ba815f0d0 2024-09-19T07:50:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.305+0000 7f0ba4ff9640 1 -- 172.21.15.16:0/3540603065 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0009510 con 0x7f0ba815f0d0 2024-09-19T07:50:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.305+0000 7f0bae8d9640 1 -- 172.21.15.16:0/3540603065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba80cc730 con 0x7f0ba815f0d0 2024-09-19T07:50:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.306+0000 7f0bae8d9640 1 -- 172.21.15.16:0/3540603065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba80d45a0 con 0x7f0ba815f0d0 2024-09-19T07:50:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.306+0000 7f0ba4ff9640 1 -- 172.21.15.16:0/3540603065 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0ba0007c70 con 0x7f0ba815f0d0 2024-09-19T07:50:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.307+0000 7f0ba4ff9640 1 --2- 172.21.15.16:0/3540603065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0b9406c2b0 0x7f0b9406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.307+0000 7f0ba4ff9640 1 -- 172.21.15.16:0/3540603065 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f0ba00ac220 con 0x7f0ba815f0d0 2024-09-19T07:50:28.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.307+0000 7f0ba7fff640 1 --2- 172.21.15.16:0/3540603065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0b9406c2b0 0x7f0b9406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.307+0000 7f0ba7fff640 1 --2- 172.21.15.16:0/3540603065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0b9406c2b0 0x7f0b9406e770 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009e90 tx=0x7f0b98009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:28.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.311+0000 7f0ba4ff9640 1 -- 172.21.15.16:0/3540603065 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f0ba007a240 con 0x7f0ba815f0d0 2024-09-19T07:50:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.488+0000 7f0bae8d9640 1 -- 172.21.15.16:0/3540603065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7f0ba815d880 con 0x7f0ba815f0d0 2024-09-19T07:50:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.490+0000 7f0ba4ff9640 1 -- 172.21.15.16:0/3540603065 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v33)=0 v33) ==== 119+0+0 (secure 0 0 0) 0x7f0ba007e540 con 0x7f0ba815f0d0 2024-09-19T07:50:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.493+0000 7f0bae8d9640 1 -- 172.21.15.16:0/3540603065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0b9406c2b0 msgr2=0x7f0b9406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.493+0000 7f0bae8d9640 1 --2- 172.21.15.16:0/3540603065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0b9406c2b0 0x7f0b9406e770 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009e90 tx=0x7f0b98009190 comp rx=0 tx=0).stop 2024-09-19T07:50:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.493+0000 7f0bae8d9640 1 -- 172.21.15.16:0/3540603065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ba815f0d0 msgr2=0x7f0ba80cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.493+0000 7f0bae8d9640 1 --2- 172.21.15.16:0/3540603065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ba815f0d0 0x7f0ba80cb4b0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0034040 tx=0x7f0ba0004a40 comp rx=0 tx=0).stop 2024-09-19T07:50:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.493+0000 7f0bae8d9640 1 -- 172.21.15.16:0/3540603065 shutdown_connections 2024-09-19T07:50:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.493+0000 7f0bae8d9640 1 --2- 172.21.15.16:0/3540603065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0b9406c2b0 0x7f0b9406e770 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.493+0000 7f0bae8d9640 1 --2- 172.21.15.16:0/3540603065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ba815f0d0 0x7f0ba80cb4b0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.493+0000 7f0bae8d9640 1 --2- 172.21.15.16:0/3540603065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ba80d4130 0x7f0ba814e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.493+0000 7f0bae8d9640 1 --2- 172.21.15.16:0/3540603065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ba80d0070 0x7f0ba80ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.493+0000 7f0bae8d9640 1 -- 172.21.15.16:0/3540603065 >> 172.21.15.16:0/3540603065 conn(0x7f0ba809d760 msgr2=0x7f0ba815dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.493+0000 7f0bae8d9640 1 -- 172.21.15.16:0/3540603065 shutdown_connections 2024-09-19T07:50:28.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.493+0000 7f0bae8d9640 1 -- 172.21.15.16:0/3540603065 wait complete. 2024-09-19T07:50:28.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 400 2024-09-19T07:50:28.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.609+0000 7fd990cd0640 1 Processor -- start 2024-09-19T07:50:28.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.609+0000 7fd990cd0640 1 -- start start 2024-09-19T07:50:28.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.610+0000 7fd990cd0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd98c142ae0 0x7fd98c1692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.610+0000 7fd990cd0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd98c1620b0 con 0x7fd98c0d21f0 2024-09-19T07:50:28.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.610+0000 7fd990cd0640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd98c162370 con 0x7fd98c072590 2024-09-19T07:50:28.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.610+0000 7fd990cd0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd98c141750 con 0x7fd98c142ae0 2024-09-19T07:50:28.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.610+0000 7fd98a59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd98c0d21f0 0x7fd98c0d25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41240/0 (socket says 172.21.15.16:41240) 2024-09-19T07:50:28.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.610+0000 7fd98a59c640 1 -- 172.21.15.16:0/2407990179 learned_addr learned my addr 172.21.15.16:0/2407990179 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:28.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.610+0000 7fd98ad9d640 1 --2- 172.21.15.16:0/2407990179 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd98c142ae0 0x7fd98c1692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.611+0000 7fd98959a640 1 -- 172.21.15.16:0/2407990179 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3947400227 0 0) 0x7fd98c1620b0 con 0x7fd98c0d21f0 2024-09-19T07:50:28.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.611+0000 7fd98959a640 1 -- 172.21.15.16:0/2407990179 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd978003520 con 0x7fd98c0d21f0 2024-09-19T07:50:28.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.611+0000 7fd98959a640 1 -- 172.21.15.16:0/2407990179 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3544724550 0 0) 0x7fd98c162370 con 0x7fd98c072590 2024-09-19T07:50:28.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.611+0000 7fd98ad9d640 1 -- 172.21.15.16:0/2407990179 >> v1:172.21.15.146:6789/0 conn(0x7fd98c072590 legacy=0x7fd98c06d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.611+0000 7fd98ad9d640 1 -- 172.21.15.16:0/2407990179 >> v1:172.21.15.16:6789/0 conn(0x7fd98c0d21f0 legacy=0x7fd98c0d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.611+0000 7fd98ad9d640 1 -- 172.21.15.16:0/2407990179 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98c162650 con 0x7fd98c142ae0 2024-09-19T07:50:28.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.612+0000 7fd98ad9d640 1 --2- 172.21.15.16:0/2407990179 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd98c142ae0 0x7fd98c1692c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009dc0 tx=0x7fd97c02a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee1f88e6f83159c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:28.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.612+0000 7fd98959a640 1 -- 172.21.15.16:0/2407990179 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c02ae60 con 0x7fd98c142ae0 2024-09-19T07:50:28.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.612+0000 7fd98959a640 1 -- 172.21.15.16:0/2407990179 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd97c007740 con 0x7fd98c142ae0 2024-09-19T07:50:28.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.612+0000 7fd98959a640 1 -- 172.21.15.16:0/2407990179 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c007b10 con 0x7fd98c142ae0 2024-09-19T07:50:28.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.612+0000 7fd990cd0640 1 -- 172.21.15.16:0/2407990179 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd98c142ae0 msgr2=0x7fd98c1692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.612+0000 7fd990cd0640 1 --2- 172.21.15.16:0/2407990179 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd98c142ae0 0x7fd98c1692c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009dc0 tx=0x7fd97c02a640 comp rx=0 tx=0).stop 2024-09-19T07:50:28.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.612+0000 7fd990cd0640 1 -- 172.21.15.16:0/2407990179 shutdown_connections 2024-09-19T07:50:28.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.612+0000 7fd990cd0640 1 --2- 172.21.15.16:0/2407990179 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd98c142ae0 0x7fd98c1692c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.612+0000 7fd990cd0640 1 -- 172.21.15.16:0/2407990179 >> 172.21.15.16:0/2407990179 conn(0x7fd98c0b3ed0 msgr2=0x7fd98c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:28.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.612+0000 7fd990cd0640 1 -- 172.21.15.16:0/2407990179 shutdown_connections 2024-09-19T07:50:28.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.612+0000 7fd990cd0640 1 -- 172.21.15.16:0/2407990179 wait complete. 2024-09-19T07:50:28.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.614+0000 7fd990cd0640 1 Processor -- start 2024-09-19T07:50:28.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.614+0000 7fd990cd0640 1 -- start start 2024-09-19T07:50:28.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.614+0000 7fd990cd0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd98c072590 0x7fd98c0b6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.614+0000 7fd990cd0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd98c0d21f0 0x7fd98c15baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.614+0000 7fd990cd0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd98c142ae0 0x7fd98c158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.614+0000 7fd990cd0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd98c162c10 con 0x7fd98c072590 2024-09-19T07:50:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.614+0000 7fd990cd0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd98c06d7f0 con 0x7fd98c142ae0 2024-09-19T07:50:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.614+0000 7fd990cd0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd98c158c30 con 0x7fd98c0d21f0 2024-09-19T07:50:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.615+0000 7fd98a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd98c072590 0x7fd98c0b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.615+0000 7fd98a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd98c072590 0x7fd98c0b6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50706/0 (socket says 172.21.15.16:50706) 2024-09-19T07:50:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.615+0000 7fd98a59c640 1 -- 172.21.15.16:0/3150994651 learned_addr learned my addr 172.21.15.16:0/3150994651 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.615+0000 7fd989d9b640 1 --2- 172.21.15.16:0/3150994651 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd98c0d21f0 0x7fd98c15baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.615+0000 7fd98ad9d640 1 --2- 172.21.15.16:0/3150994651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd98c142ae0 0x7fd98c158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.615+0000 7fd98a59c640 1 -- 172.21.15.16:0/3150994651 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd98c0d21f0 msgr2=0x7fd98c15baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.615+0000 7fd98a59c640 1 --2- 172.21.15.16:0/3150994651 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd98c0d21f0 0x7fd98c15baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.615+0000 7fd98a59c640 1 -- 172.21.15.16:0/3150994651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd98c142ae0 msgr2=0x7fd98c158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.615+0000 7fd98a59c640 1 --2- 172.21.15.16:0/3150994651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd98c142ae0 0x7fd98c158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.615+0000 7fd98a59c640 1 -- 172.21.15.16:0/3150994651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98c0b5d50 con 0x7fd98c072590 2024-09-19T07:50:28.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.615+0000 7fd98a59c640 1 --2- 172.21.15.16:0/3150994651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd98c072590 0x7fd98c0b6210 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd98400c430 tx=0x7fd98400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:28.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.615+0000 7fd982ffd640 1 -- 172.21.15.16:0/3150994651 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984017070 con 0x7fd98c072590 2024-09-19T07:50:28.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.615+0000 7fd982ffd640 1 -- 172.21.15.16:0/3150994651 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd984005ce0 con 0x7fd98c072590 2024-09-19T07:50:28.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.615+0000 7fd982ffd640 1 -- 172.21.15.16:0/3150994651 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9840072f0 con 0x7fd98c072590 2024-09-19T07:50:28.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.615+0000 7fd990cd0640 1 -- 172.21.15.16:0/3150994651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd98c13e870 con 0x7fd98c072590 2024-09-19T07:50:28.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.615+0000 7fd990cd0640 1 -- 172.21.15.16:0/3150994651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd98c13ee30 con 0x7fd98c072590 2024-09-19T07:50:28.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.616+0000 7fd990cd0640 1 -- 172.21.15.16:0/3150994651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd98c06d0e0 con 0x7fd98c072590 2024-09-19T07:50:28.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.617+0000 7fd982ffd640 1 -- 172.21.15.16:0/3150994651 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd98401a070 con 0x7fd98c072590 2024-09-19T07:50:28.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.617+0000 7fd982ffd640 1 --2- 172.21.15.16:0/3150994651 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd96c06c2e0 0x7fd96c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.617+0000 7fd982ffd640 1 -- 172.21.15.16:0/3150994651 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fd98408a910 con 0x7fd98c072590 2024-09-19T07:50:28.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.617+0000 7fd989d9b640 1 --2- 172.21.15.16:0/3150994651 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd96c06c2e0 0x7fd96c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.618+0000 7fd989d9b640 1 --2- 172.21.15.16:0/3150994651 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd96c06c2e0 0x7fd96c06e7a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd98c1f6ec0 tx=0x7fd974009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:28.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.621+0000 7fd982ffd640 1 -- 172.21.15.16:0/3150994651 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd984058930 con 0x7fd98c072590 2024-09-19T07:50:28.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.797+0000 7fd990cd0640 1 -- 172.21.15.16:0/3150994651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7fd98c13f000 con 0x7fd98c072590 2024-09-19T07:50:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.800+0000 7fd982ffd640 1 -- 172.21.15.16:0/3150994651 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v34)=0 v34) ==== 128+0+0 (secure 0 0 0) 0x7fd98405cc30 con 0x7fd98c072590 2024-09-19T07:50:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.802+0000 7fd990cd0640 1 -- 172.21.15.16:0/3150994651 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd96c06c2e0 msgr2=0x7fd96c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.802+0000 7fd990cd0640 1 --2- 172.21.15.16:0/3150994651 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd96c06c2e0 0x7fd96c06e7a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd98c1f6ec0 tx=0x7fd974009040 comp rx=0 tx=0).stop 2024-09-19T07:50:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.802+0000 7fd990cd0640 1 -- 172.21.15.16:0/3150994651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd98c072590 msgr2=0x7fd98c0b6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.802+0000 7fd990cd0640 1 --2- 172.21.15.16:0/3150994651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd98c072590 0x7fd98c0b6210 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd98400c430 tx=0x7fd98400c8f0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.802+0000 7fd990cd0640 1 -- 172.21.15.16:0/3150994651 shutdown_connections 2024-09-19T07:50:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.802+0000 7fd990cd0640 1 --2- 172.21.15.16:0/3150994651 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd96c06c2e0 0x7fd96c06e7a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.802+0000 7fd990cd0640 1 --2- 172.21.15.16:0/3150994651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd98c142ae0 0x7fd98c158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.802+0000 7fd990cd0640 1 --2- 172.21.15.16:0/3150994651 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd98c0d21f0 0x7fd98c15baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.802+0000 7fd990cd0640 1 --2- 172.21.15.16:0/3150994651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd98c072590 0x7fd98c0b6210 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.802+0000 7fd990cd0640 1 -- 172.21.15.16:0/3150994651 >> 172.21.15.16:0/3150994651 conn(0x7fd98c0b3ed0 msgr2=0x7fd98c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.802+0000 7fd990cd0640 1 -- 172.21.15.16:0/3150994651 shutdown_connections 2024-09-19T07:50:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.802+0000 7fd990cd0640 1 -- 172.21.15.16:0/3150994651 wait complete. 2024-09-19T07:50:28.813 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 800 2024-09-19T07:50:28.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.918+0000 7f6e28fe4640 1 Processor -- start 2024-09-19T07:50:28.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.918+0000 7f6e28fe4640 1 -- start start 2024-09-19T07:50:28.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.919+0000 7f6e28fe4640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6e241619b0 0x7f6e240d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.919+0000 7f6e28fe4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e240ce6a0 con 0x7f6e240b0a80 2024-09-19T07:50:28.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.919+0000 7f6e28fe4640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e2406c990 con 0x7f6e24161d90 2024-09-19T07:50:28.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.919+0000 7f6e28fe4640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f6e240cec10 con 0x7f6e241619b0 2024-09-19T07:50:28.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.919+0000 7f6e21d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6e240b0a80 0x7f6e240cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41256/0 (socket says 172.21.15.16:41256) 2024-09-19T07:50:28.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.919+0000 7f6e21d9b640 1 -- 172.21.15.16:0/580789011 learned_addr learned my addr 172.21.15.16:0/580789011 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:28.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.919+0000 7f6e2259c640 1 --2- 172.21.15.16:0/580789011 >> v2:172.21.15.146:3300/0 conn(0x7f6e241619b0 0x7f6e240d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.919+0000 7f6e2159a640 1 -- 172.21.15.16:0/580789011 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3090566456 0 0) 0x7f6e240ce6a0 con 0x7f6e240b0a80 2024-09-19T07:50:28.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.919+0000 7f6e2159a640 1 -- 172.21.15.16:0/580789011 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e10003520 con 0x7f6e240b0a80 2024-09-19T07:50:28.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.920+0000 7f6e2259c640 1 -- 172.21.15.16:0/580789011 >> v1:172.21.15.146:6789/0 conn(0x7f6e24161d90 legacy=0x7f6e240cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.920+0000 7f6e2159a640 1 -- 172.21.15.16:0/580789011 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4101069968 0 0) 0x7f6e10003520 con 0x7f6e240b0a80 2024-09-19T07:50:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.920+0000 7f6e2259c640 1 -- 172.21.15.16:0/580789011 >> v1:172.21.15.16:6789/0 conn(0x7f6e240b0a80 legacy=0x7f6e240cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.920+0000 7f6e2259c640 1 -- 172.21.15.16:0/580789011 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e240ce930 con 0x7f6e241619b0 2024-09-19T07:50:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.920+0000 7f6e2259c640 1 --2- 172.21.15.16:0/580789011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e241619b0 0x7f6e240d0440 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009d90 tx=0x7f6e1402a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b82cdb86316454c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.921+0000 7f6e2159a640 1 -- 172.21.15.16:0/580789011 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e14035020 con 0x7f6e241619b0 2024-09-19T07:50:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.921+0000 7f6e2159a640 1 -- 172.21.15.16:0/580789011 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e14007540 con 0x7f6e241619b0 2024-09-19T07:50:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.921+0000 7f6e2159a640 1 -- 172.21.15.16:0/580789011 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c001670 con 0x7f6e241619b0 2024-09-19T07:50:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.921+0000 7f6e28fe4640 1 -- 172.21.15.16:0/580789011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e241619b0 msgr2=0x7f6e240d0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.921+0000 7f6e28fe4640 1 --2- 172.21.15.16:0/580789011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e241619b0 0x7f6e240d0440 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009d90 tx=0x7f6e1402a5b0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.921+0000 7f6e28fe4640 1 -- 172.21.15.16:0/580789011 shutdown_connections 2024-09-19T07:50:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.921+0000 7f6e28fe4640 1 --2- 172.21.15.16:0/580789011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e241619b0 0x7f6e240d0440 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.921+0000 7f6e28fe4640 1 -- 172.21.15.16:0/580789011 >> 172.21.15.16:0/580789011 conn(0x7f6e2409a9f0 msgr2=0x7f6e240b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.921+0000 7f6e28fe4640 1 -- 172.21.15.16:0/580789011 shutdown_connections 2024-09-19T07:50:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.921+0000 7f6e28fe4640 1 -- 172.21.15.16:0/580789011 wait complete. 2024-09-19T07:50:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.923+0000 7f6e28fe4640 1 Processor -- start 2024-09-19T07:50:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.923+0000 7f6e28fe4640 1 -- start start 2024-09-19T07:50:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.923+0000 7f6e28fe4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e240b0a80 0x7f6e241569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.923+0000 7f6e28fe4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6e241619b0 0x7f6e240c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.923+0000 7f6e28fe4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e24161d90 0x7f6e240b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.923+0000 7f6e28fe4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2406c620 con 0x7f6e240b0a80 2024-09-19T07:50:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.923+0000 7f6e28fe4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6e240b0820 con 0x7f6e24161d90 2024-09-19T07:50:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.923+0000 7f6e28fe4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6e240ccce0 con 0x7f6e241619b0 2024-09-19T07:50:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.923+0000 7f6e2259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e240b0a80 0x7f6e241569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.924+0000 7f6e2259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e240b0a80 0x7f6e241569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50710/0 (socket says 172.21.15.16:50710) 2024-09-19T07:50:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.924+0000 7f6e2259c640 1 -- 172.21.15.16:0/4141930217 learned_addr learned my addr 172.21.15.16:0/4141930217 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.924+0000 7f6e22d9d640 1 --2- 172.21.15.16:0/4141930217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e24161d90 0x7f6e240b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.924+0000 7f6e21d9b640 1 --2- 172.21.15.16:0/4141930217 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6e241619b0 0x7f6e240c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.924+0000 7f6e2259c640 1 -- 172.21.15.16:0/4141930217 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6e241619b0 msgr2=0x7f6e240c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.924+0000 7f6e2259c640 1 --2- 172.21.15.16:0/4141930217 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6e241619b0 0x7f6e240c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.924+0000 7f6e2259c640 1 -- 172.21.15.16:0/4141930217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e24161d90 msgr2=0x7f6e240b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.924+0000 7f6e2259c640 1 --2- 172.21.15.16:0/4141930217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e24161d90 0x7f6e240b5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.924+0000 7f6e2259c640 1 -- 172.21.15.16:0/4141930217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e240c65e0 con 0x7f6e240b0a80 2024-09-19T07:50:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.924+0000 7f6e22d9d640 1 --2- 172.21.15.16:0/4141930217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e24161d90 0x7f6e240b5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.924+0000 7f6e21d9b640 1 --2- 172.21.15.16:0/4141930217 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6e241619b0 0x7f6e240c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.924+0000 7f6e2259c640 1 --2- 172.21.15.16:0/4141930217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e240b0a80 0x7f6e241569b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009eb0 tx=0x7f6e14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.924+0000 7f6e1affd640 1 -- 172.21.15.16:0/4141930217 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1403f070 con 0x7f6e240b0a80 2024-09-19T07:50:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.924+0000 7f6e1affd640 1 -- 172.21.15.16:0/4141930217 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e14031c40 con 0x7f6e240b0a80 2024-09-19T07:50:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.924+0000 7f6e28fe4640 1 -- 172.21.15.16:0/4141930217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e240c68c0 con 0x7f6e240b0a80 2024-09-19T07:50:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.924+0000 7f6e1affd640 1 -- 172.21.15.16:0/4141930217 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e14035070 con 0x7f6e240b0a80 2024-09-19T07:50:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.924+0000 7f6e28fe4640 1 -- 172.21.15.16:0/4141930217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e240c6e80 con 0x7f6e240b0a80 2024-09-19T07:50:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.925+0000 7f6e28fe4640 1 -- 172.21.15.16:0/4141930217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e240cd330 con 0x7f6e240b0a80 2024-09-19T07:50:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.926+0000 7f6e1affd640 1 -- 172.21.15.16:0/4141930217 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6e14042070 con 0x7f6e240b0a80 2024-09-19T07:50:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.926+0000 7f6e1affd640 1 --2- 172.21.15.16:0/4141930217 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6e0406c2b0 0x7f6e0406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.926+0000 7f6e1affd640 1 -- 172.21.15.16:0/4141930217 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f6e140ac670 con 0x7f6e240b0a80 2024-09-19T07:50:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.927+0000 7f6e21d9b640 1 --2- 172.21.15.16:0/4141930217 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6e0406c2b0 0x7f6e0406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.927+0000 7f6e21d9b640 1 --2- 172.21.15.16:0/4141930217 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6e0406c2b0 0x7f6e0406e770 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6e240d19a0 tx=0x7f6e0c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:28.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:28.931+0000 7f6e1affd640 1 -- 172.21.15.16:0/4141930217 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6e1407ac50 con 0x7f6e240b0a80 2024-09-19T07:50:29.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.108+0000 7f6e28fe4640 1 -- 172.21.15.16:0/4141930217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7f6e2406df30 con 0x7f6e240b0a80 2024-09-19T07:50:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.111+0000 7f6e1affd640 1 -- 172.21.15.16:0/4141930217 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v35)=0 v35) ==== 124+0+0 (secure 0 0 0) 0x7f6e1407a550 con 0x7f6e240b0a80 2024-09-19T07:50:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.113+0000 7f6e28fe4640 1 -- 172.21.15.16:0/4141930217 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6e0406c2b0 msgr2=0x7f6e0406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.113+0000 7f6e28fe4640 1 --2- 172.21.15.16:0/4141930217 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6e0406c2b0 0x7f6e0406e770 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6e240d19a0 tx=0x7f6e0c009040 comp rx=0 tx=0).stop 2024-09-19T07:50:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.113+0000 7f6e28fe4640 1 -- 172.21.15.16:0/4141930217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e240b0a80 msgr2=0x7f6e241569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.113+0000 7f6e28fe4640 1 --2- 172.21.15.16:0/4141930217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e240b0a80 0x7f6e241569b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009eb0 tx=0x7f6e14004a40 comp rx=0 tx=0).stop 2024-09-19T07:50:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.113+0000 7f6e28fe4640 1 -- 172.21.15.16:0/4141930217 shutdown_connections 2024-09-19T07:50:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.113+0000 7f6e28fe4640 1 --2- 172.21.15.16:0/4141930217 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6e0406c2b0 0x7f6e0406e770 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.113+0000 7f6e28fe4640 1 --2- 172.21.15.16:0/4141930217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e24161d90 0x7f6e240b5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.113+0000 7f6e28fe4640 1 --2- 172.21.15.16:0/4141930217 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6e241619b0 0x7f6e240c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.113+0000 7f6e28fe4640 1 --2- 172.21.15.16:0/4141930217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e240b0a80 0x7f6e241569b0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.113+0000 7f6e28fe4640 1 -- 172.21.15.16:0/4141930217 >> 172.21.15.16:0/4141930217 conn(0x7f6e2409a9f0 msgr2=0x7f6e240706f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.113+0000 7f6e28fe4640 1 -- 172.21.15.16:0/4141930217 shutdown_connections 2024-09-19T07:50:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.113+0000 7f6e28fe4640 1 -- 172.21.15.16:0/4141930217 wait complete. 2024-09-19T07:50:29.125 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-09-19T07:50:29.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.228+0000 7f3a8e931640 1 Processor -- start 2024-09-19T07:50:29.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.228+0000 7f3a8e931640 1 -- start start 2024-09-19T07:50:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.229+0000 7f3a8e931640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3a880d21f0 0x7f3a880d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.229+0000 7f3a8e931640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a88072590 0x7f3a8806d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.229+0000 7f3a8e931640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a880d2b10 con 0x7f3a88143a60 2024-09-19T07:50:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.229+0000 7f3a8e931640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f3a88141750 con 0x7f3a88072590 2024-09-19T07:50:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.229+0000 7f3a8e931640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f3a881620b0 con 0x7f3a880d21f0 2024-09-19T07:50:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.229+0000 7f3a877fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a88072590 0x7f3a8806d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.230+0000 7f3a877fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a88072590 0x7f3a8806d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50716/0 (socket says 172.21.15.16:50716) 2024-09-19T07:50:29.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.230+0000 7f3a877fe640 1 -- 172.21.15.16:0/397843714 learned_addr learned my addr 172.21.15.16:0/397843714 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:29.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.230+0000 7f3a877fe640 1 -- 172.21.15.16:0/397843714 >> v2:172.21.15.146:3300/0 conn(0x7f3a880d21f0 msgr2=0x7f3a880d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:29.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.230+0000 7f3a87fff640 1 --2- 172.21.15.16:0/397843714 >> v2:172.21.15.146:3300/0 conn(0x7f3a880d21f0 0x7f3a880d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:29.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.230+0000 7f3a877fe640 1 --2- 172.21.15.16:0/397843714 >> v2:172.21.15.146:3300/0 conn(0x7f3a880d21f0 0x7f3a880d25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.230+0000 7f3a877fe640 1 -- 172.21.15.16:0/397843714 >> v1:172.21.15.146:6789/0 conn(0x7f3a88143a60 legacy=0x7f3a88143f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:29.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.230+0000 7f3a877fe640 1 -- 172.21.15.16:0/397843714 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a88162370 con 0x7f3a88072590 2024-09-19T07:50:29.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.231+0000 7f3a877fe640 1 --2- 172.21.15.16:0/397843714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a88072590 0x7f3a8806d0e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f3a880cfd80 tx=0x7f3a7c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d966c45a6fa1fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:29.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.231+0000 7f3a86ffd640 1 -- 172.21.15.16:0/397843714 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c037070 con 0x7f3a88072590 2024-09-19T07:50:29.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.231+0000 7f3a86ffd640 1 -- 172.21.15.16:0/397843714 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a7c032440 con 0x7f3a88072590 2024-09-19T07:50:29.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.231+0000 7f3a86ffd640 1 -- 172.21.15.16:0/397843714 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c032810 con 0x7f3a88072590 2024-09-19T07:50:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.231+0000 7f3a8e931640 1 -- 172.21.15.16:0/397843714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a88072590 msgr2=0x7f3a8806d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.231+0000 7f3a8e931640 1 --2- 172.21.15.16:0/397843714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a88072590 0x7f3a8806d0e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f3a880cfd80 tx=0x7f3a7c02a1a0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.231+0000 7f3a8e931640 1 -- 172.21.15.16:0/397843714 shutdown_connections 2024-09-19T07:50:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.231+0000 7f3a8e931640 1 --2- 172.21.15.16:0/397843714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a88072590 0x7f3a8806d0e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.231+0000 7f3a8e931640 1 --2- 172.21.15.16:0/397843714 >> v2:172.21.15.146:3300/0 conn(0x7f3a880d21f0 0x7f3a880d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.231+0000 7f3a8e931640 1 -- 172.21.15.16:0/397843714 >> 172.21.15.16:0/397843714 conn(0x7f3a880b3ed0 msgr2=0x7f3a880b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.231+0000 7f3a8e931640 1 -- 172.21.15.16:0/397843714 shutdown_connections 2024-09-19T07:50:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.231+0000 7f3a8e931640 1 -- 172.21.15.16:0/397843714 wait complete. 2024-09-19T07:50:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.232+0000 7f3a8e931640 1 Processor -- start 2024-09-19T07:50:29.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.233+0000 7f3a8e931640 1 -- start start 2024-09-19T07:50:29.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.233+0000 7f3a8e931640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a88072590 0x7f3a88140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:29.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.233+0000 7f3a8e931640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a880d21f0 0x7f3a880b7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:29.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.233+0000 7f3a8e931640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a88143a60 0x7f3a880c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:29.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.233+0000 7f3a8e931640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3a88162c10 con 0x7f3a88143a60 2024-09-19T07:50:29.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.233+0000 7f3a8e931640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3a88162930 con 0x7f3a880d21f0 2024-09-19T07:50:29.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.233+0000 7f3a8e931640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3a8806d620 con 0x7f3a88072590 2024-09-19T07:50:29.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.234+0000 7f3a8cece640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a88143a60 0x7f3a880c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:29.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.234+0000 7f3a8cece640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a88143a60 0x7f3a880c86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50720/0 (socket says 172.21.15.16:50720) 2024-09-19T07:50:29.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.234+0000 7f3a8cece640 1 -- 172.21.15.16:0/261486510 learned_addr learned my addr 172.21.15.16:0/261486510 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:29.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.234+0000 7f3a87fff640 1 --2- 172.21.15.16:0/261486510 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a88072590 0x7f3a88140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:29.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.234+0000 7f3a877fe640 1 --2- 172.21.15.16:0/261486510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a880d21f0 0x7f3a880b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:29.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.234+0000 7f3a8cece640 1 -- 172.21.15.16:0/261486510 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a88072590 msgr2=0x7f3a88140be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:29.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.234+0000 7f3a8cece640 1 --2- 172.21.15.16:0/261486510 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a88072590 0x7f3a88140be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.234+0000 7f3a8cece640 1 -- 172.21.15.16:0/261486510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a880d21f0 msgr2=0x7f3a880b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:29.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.234+0000 7f3a8cece640 1 --2- 172.21.15.16:0/261486510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a880d21f0 0x7f3a880b7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.234+0000 7f3a8cece640 1 -- 172.21.15.16:0/261486510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a880b5fb0 con 0x7f3a88143a60 2024-09-19T07:50:29.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.234+0000 7f3a8cece640 1 --2- 172.21.15.16:0/261486510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a88143a60 0x7f3a880c86a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000b970 tx=0x7f3a8000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:29.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.234+0000 7f3a84ff9640 1 -- 172.21.15.16:0/261486510 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80019070 con 0x7f3a88143a60 2024-09-19T07:50:29.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.234+0000 7f3a84ff9640 1 -- 172.21.15.16:0/261486510 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a8000cb40 con 0x7f3a88143a60 2024-09-19T07:50:29.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.234+0000 7f3a84ff9640 1 -- 172.21.15.16:0/261486510 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80014440 con 0x7f3a88143a60 2024-09-19T07:50:29.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.234+0000 7f3a8e931640 1 -- 172.21.15.16:0/261486510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a880b6290 con 0x7f3a88143a60 2024-09-19T07:50:29.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.234+0000 7f3a8e931640 1 -- 172.21.15.16:0/261486510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a880b6850 con 0x7f3a88143a60 2024-09-19T07:50:29.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.235+0000 7f3a8e931640 1 -- 172.21.15.16:0/261486510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a880690c0 con 0x7f3a88143a60 2024-09-19T07:50:29.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.236+0000 7f3a84ff9640 1 -- 172.21.15.16:0/261486510 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3a8001c070 con 0x7f3a88143a60 2024-09-19T07:50:29.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.236+0000 7f3a84ff9640 1 --2- 172.21.15.16:0/261486510 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3a7406c2e0 0x7f3a7406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:29.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.237+0000 7f3a84ff9640 1 -- 172.21.15.16:0/261486510 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f3a8008b1d0 con 0x7f3a88143a60 2024-09-19T07:50:29.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.237+0000 7f3a87fff640 1 --2- 172.21.15.16:0/261486510 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3a7406c2e0 0x7f3a7406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:29.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.237+0000 7f3a87fff640 1 --2- 172.21.15.16:0/261486510 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3a7406c2e0 0x7f3a7406e7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3a8815a000 tx=0x7f3a78008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:29.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.241+0000 7f3a84ff9640 1 -- 172.21.15.16:0/261486510 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3a800590c0 con 0x7f3a88143a60 2024-09-19T07:50:29.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.418+0000 7f3a8e931640 1 -- 172.21.15.16:0/261486510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7f3a88072290 con 0x7f3a88143a60 2024-09-19T07:50:29.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.418+0000 7f3a84ff9640 1 -- 172.21.15.16:0/261486510 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v35) ==== 106+0+4 (secure 0 0 0) 0x7f3a8005d3c0 con 0x7f3a88143a60 2024-09-19T07:50:29.418 INFO:teuthology.orchestra.run.smithi016.stdout:100 2024-09-19T07:50:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.420+0000 7f3a8e931640 1 -- 172.21.15.16:0/261486510 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3a7406c2e0 msgr2=0x7f3a7406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.420+0000 7f3a8e931640 1 --2- 172.21.15.16:0/261486510 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3a7406c2e0 0x7f3a7406e7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3a8815a000 tx=0x7f3a78008040 comp rx=0 tx=0).stop 2024-09-19T07:50:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.420+0000 7f3a8e931640 1 -- 172.21.15.16:0/261486510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a88143a60 msgr2=0x7f3a880c86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.420+0000 7f3a8e931640 1 --2- 172.21.15.16:0/261486510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a88143a60 0x7f3a880c86a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000b970 tx=0x7f3a8000be30 comp rx=0 tx=0).stop 2024-09-19T07:50:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.420+0000 7f3a8e931640 1 -- 172.21.15.16:0/261486510 shutdown_connections 2024-09-19T07:50:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.420+0000 7f3a8e931640 1 --2- 172.21.15.16:0/261486510 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3a7406c2e0 0x7f3a7406e7a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.420+0000 7f3a8e931640 1 --2- 172.21.15.16:0/261486510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a88143a60 0x7f3a880c86a0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.420+0000 7f3a8e931640 1 --2- 172.21.15.16:0/261486510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a880d21f0 0x7f3a880b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.420+0000 7f3a8e931640 1 --2- 172.21.15.16:0/261486510 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a88072590 0x7f3a88140be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.420+0000 7f3a8e931640 1 -- 172.21.15.16:0/261486510 >> 172.21.15.16:0/261486510 conn(0x7f3a880b3ed0 msgr2=0x7f3a881e9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:29.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.420+0000 7f3a8e931640 1 -- 172.21.15.16:0/261486510 shutdown_connections 2024-09-19T07:50:29.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.420+0000 7f3a8e931640 1 -- 172.21.15.16:0/261486510 wait complete. 2024-09-19T07:50:29.432 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-09-19T07:50:29.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.536+0000 7f1350e6e640 1 Processor -- start 2024-09-19T07:50:29.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.536+0000 7f1350e6e640 1 -- start start 2024-09-19T07:50:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.537+0000 7f1350e6e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f134c15f0d0 0x7f134c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.537+0000 7f1350e6e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f134c0d1b10 con 0x7f134c0d4130 2024-09-19T07:50:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.537+0000 7f1350e6e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f134c0cf2b0 con 0x7f134c0d0070 2024-09-19T07:50:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.537+0000 7f1350e6e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f134c0cf590 con 0x7f134c15f0d0 2024-09-19T07:50:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.537+0000 7f134ad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f134c0d4130 0x7f134c0d4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41268/0 (socket says 172.21.15.16:41268) 2024-09-19T07:50:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.537+0000 7f134ad9d640 1 -- 172.21.15.16:0/517251241 learned_addr learned my addr 172.21.15.16:0/517251241 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f134a59c640 1 --2- 172.21.15.16:0/517251241 >> v2:172.21.15.146:3300/0 conn(0x7f134c15f0d0 0x7f134c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f134959a640 1 -- 172.21.15.16:0/517251241 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 615072104 0 0) 0x7f134c0d1b10 con 0x7f134c0d4130 2024-09-19T07:50:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f134959a640 1 -- 172.21.15.16:0/517251241 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1334003520 con 0x7f134c0d4130 2024-09-19T07:50:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f134959a640 1 -- 172.21.15.16:0/517251241 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4212541815 0 0) 0x7f1334003520 con 0x7f134c0d4130 2024-09-19T07:50:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f134959a640 1 -- 172.21.15.16:0/517251241 >> v2:172.21.15.146:3300/0 conn(0x7f134c15f0d0 msgr2=0x7f134c15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f134959a640 1 --2- 172.21.15.16:0/517251241 >> v2:172.21.15.146:3300/0 conn(0x7f134c15f0d0 0x7f134c15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f134959a640 1 -- 172.21.15.16:0/517251241 >> v1:172.21.15.146:6789/0 conn(0x7f134c0d0070 legacy=0x7f134c0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f134959a640 1 -- 172.21.15.16:0/517251241 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134c0cf870 con 0x7f134c0d4130 2024-09-19T07:50:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f134959a640 1 -- 172.21.15.16:0/517251241 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f1344001690 con 0x7f134c0d4130 2024-09-19T07:50:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f134959a640 1 -- 172.21.15.16:0/517251241 >> v1:172.21.15.16:6789/0 conn(0x7f134c0d4130 legacy=0x7f134c0d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f134959a640 1 --2- 172.21.15.16:0/517251241 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13340039b0 0x7f1334003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f134959a640 1 --2- 172.21.15.16:0/517251241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334004800 0x7f1334004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f134959a640 1 --2- 172.21.15.16:0/517251241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13340052e0 0x7f133400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f134959a640 1 -- 172.21.15.16:0/517251241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f134c0cf590 con 0x7f1334004800 2024-09-19T07:50:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f134959a640 1 -- 172.21.15.16:0/517251241 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f133400c1b0 con 0x7f13340052e0 2024-09-19T07:50:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f134959a640 1 -- 172.21.15.16:0/517251241 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f133400c3b0 con 0x7f13340039b0 2024-09-19T07:50:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f1349d9b640 1 --2- 172.21.15.16:0/517251241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334004800 0x7f1334004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f134a59c640 1 --2- 172.21.15.16:0/517251241 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13340039b0 0x7f1334003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f134ad9d640 1 --2- 172.21.15.16:0/517251241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13340052e0 0x7f133400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f1349d9b640 1 -- 172.21.15.16:0/517251241 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13340039b0 msgr2=0x7f1334003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f1349d9b640 1 --2- 172.21.15.16:0/517251241 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13340039b0 0x7f1334003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f1349d9b640 1 -- 172.21.15.16:0/517251241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13340052e0 msgr2=0x7f133400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.538+0000 7f1349d9b640 1 --2- 172.21.15.16:0/517251241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13340052e0 0x7f133400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.539+0000 7f134ad9d640 1 -- 172.21.15.16:0/517251241 reap_dead start 2024-09-19T07:50:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.539+0000 7f1349d9b640 1 -- 172.21.15.16:0/517251241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1334003520 con 0x7f1334004800 2024-09-19T07:50:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.539+0000 7f1349d9b640 1 --2- 172.21.15.16:0/517251241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334004800 0x7f1334004c80 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1340001720 tx=0x7f1334005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ca5347e91a44de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.539+0000 7f134959a640 1 -- 172.21.15.16:0/517251241 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340001370 con 0x7f1334004800 2024-09-19T07:50:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.539+0000 7f134959a640 1 -- 172.21.15.16:0/517251241 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13400052d0 con 0x7f1334004800 2024-09-19T07:50:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.539+0000 7f134959a640 1 -- 172.21.15.16:0/517251241 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134002ab90 con 0x7f1334004800 2024-09-19T07:50:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.540+0000 7f1350e6e640 1 -- 172.21.15.16:0/517251241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334004800 msgr2=0x7f1334004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.540+0000 7f1350e6e640 1 --2- 172.21.15.16:0/517251241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334004800 0x7f1334004c80 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1340001720 tx=0x7f1334005830 comp rx=0 tx=0).stop 2024-09-19T07:50:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.540+0000 7f1350e6e640 1 -- 172.21.15.16:0/517251241 shutdown_connections 2024-09-19T07:50:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.540+0000 7f1350e6e640 1 --2- 172.21.15.16:0/517251241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334004800 0x7f1334004c80 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.540+0000 7f1350e6e640 1 -- 172.21.15.16:0/517251241 >> 172.21.15.16:0/517251241 conn(0x7f134c09d760 msgr2=0x7f134c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.540+0000 7f1350e6e640 1 -- 172.21.15.16:0/517251241 shutdown_connections 2024-09-19T07:50:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.540+0000 7f1350e6e640 1 -- 172.21.15.16:0/517251241 wait complete. 2024-09-19T07:50:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.541+0000 7f1350e6e640 1 Processor -- start 2024-09-19T07:50:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.542+0000 7f1350e6e640 1 -- start start 2024-09-19T07:50:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.542+0000 7f1350e6e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334004800 0x7f134c152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.542+0000 7f1350e6e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f134c1537f0 0x7f134c0b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.542+0000 7f1350e6e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f134c0b96a0 0x7f134c0b9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.542+0000 7f1350e6e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f134c15f4b0 con 0x7f1334004800 2024-09-19T07:50:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.542+0000 7f1350e6e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f134c0cef60 con 0x7f134c1537f0 2024-09-19T07:50:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.542+0000 7f1350e6e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f134c15f630 con 0x7f134c0b96a0 2024-09-19T07:50:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.542+0000 7f134a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334004800 0x7f134c152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.542+0000 7f134a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334004800 0x7f134c152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50744/0 (socket says 172.21.15.16:50744) 2024-09-19T07:50:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.542+0000 7f134a59c640 1 -- 172.21.15.16:0/2218589568 learned_addr learned my addr 172.21.15.16:0/2218589568 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.543+0000 7f1349d9b640 1 --2- 172.21.15.16:0/2218589568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f134c1537f0 0x7f134c0b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.543+0000 7f134ad9d640 1 --2- 172.21.15.16:0/2218589568 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f134c0b96a0 0x7f134c0b9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.543+0000 7f134a59c640 1 -- 172.21.15.16:0/2218589568 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f134c0b96a0 msgr2=0x7f134c0b9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.543+0000 7f134a59c640 1 --2- 172.21.15.16:0/2218589568 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f134c0b96a0 0x7f134c0b9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.543+0000 7f134a59c640 1 -- 172.21.15.16:0/2218589568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f134c1537f0 msgr2=0x7f134c0b7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.543+0000 7f134a59c640 1 --2- 172.21.15.16:0/2218589568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f134c1537f0 0x7f134c0b7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.543+0000 7f134a59c640 1 -- 172.21.15.16:0/2218589568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134c0cc090 con 0x7f1334004800 2024-09-19T07:50:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.543+0000 7f1349d9b640 1 --2- 172.21.15.16:0/2218589568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f134c1537f0 0x7f134c0b7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.543+0000 7f134ad9d640 1 --2- 172.21.15.16:0/2218589568 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f134c0b96a0 0x7f134c0b9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.543+0000 7f134a59c640 1 --2- 172.21.15.16:0/2218589568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334004800 0x7f134c152f80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f133c009fc0 tx=0x7f133c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.543+0000 7f133affd640 1 -- 172.21.15.16:0/2218589568 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c00fea0 con 0x7f1334004800 2024-09-19T07:50:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.543+0000 7f133affd640 1 -- 172.21.15.16:0/2218589568 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133c015630 con 0x7f1334004800 2024-09-19T07:50:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.543+0000 7f133affd640 1 -- 172.21.15.16:0/2218589568 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c015a20 con 0x7f1334004800 2024-09-19T07:50:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.543+0000 7f1350e6e640 1 -- 172.21.15.16:0/2218589568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f134c158ee0 con 0x7f1334004800 2024-09-19T07:50:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.543+0000 7f1350e6e640 1 -- 172.21.15.16:0/2218589568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f134c1594a0 con 0x7f1334004800 2024-09-19T07:50:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.544+0000 7f1350e6e640 1 -- 172.21.15.16:0/2218589568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134c0d4560 con 0x7f1334004800 2024-09-19T07:50:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.545+0000 7f133affd640 1 -- 172.21.15.16:0/2218589568 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f133c015bc0 con 0x7f1334004800 2024-09-19T07:50:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.545+0000 7f133affd640 1 --2- 172.21.15.16:0/2218589568 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f132c06c2e0 0x7f132c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.545+0000 7f133affd640 1 -- 172.21.15.16:0/2218589568 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f133c08c720 con 0x7f1334004800 2024-09-19T07:50:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.545+0000 7f1349d9b640 1 --2- 172.21.15.16:0/2218589568 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f132c06c2e0 0x7f132c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.546+0000 7f1349d9b640 1 --2- 172.21.15.16:0/2218589568 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f132c06c2e0 0x7f132c06e7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1340032000 tx=0x7f134002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:29.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.549+0000 7f133affd640 1 -- 172.21.15.16:0/2218589568 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f133c019030 con 0x7f1334004800 2024-09-19T07:50:29.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.727+0000 7f1350e6e640 1 -- 172.21.15.16:0/2218589568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7f134c06e0b0 con 0x7f1334004800 2024-09-19T07:50:29.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.727+0000 7f133affd640 1 -- 172.21.15.16:0/2218589568 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v35) ==== 112+0+10 (secure 0 0 0) 0x7f133c010040 con 0x7f1334004800 2024-09-19T07:50:29.728 INFO:teuthology.orchestra.run.smithi016.stdout:60.000000 2024-09-19T07:50:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.729+0000 7f1350e6e640 1 -- 172.21.15.16:0/2218589568 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f132c06c2e0 msgr2=0x7f132c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.729+0000 7f1350e6e640 1 --2- 172.21.15.16:0/2218589568 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f132c06c2e0 0x7f132c06e7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1340032000 tx=0x7f134002e000 comp rx=0 tx=0).stop 2024-09-19T07:50:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.729+0000 7f1350e6e640 1 -- 172.21.15.16:0/2218589568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334004800 msgr2=0x7f134c152f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.729+0000 7f1350e6e640 1 --2- 172.21.15.16:0/2218589568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334004800 0x7f134c152f80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f133c009fc0 tx=0x7f133c00f580 comp rx=0 tx=0).stop 2024-09-19T07:50:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.729+0000 7f1350e6e640 1 -- 172.21.15.16:0/2218589568 shutdown_connections 2024-09-19T07:50:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.729+0000 7f1350e6e640 1 --2- 172.21.15.16:0/2218589568 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f132c06c2e0 0x7f132c06e7a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.729+0000 7f1350e6e640 1 --2- 172.21.15.16:0/2218589568 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f134c0b96a0 0x7f134c0b9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.729+0000 7f1350e6e640 1 --2- 172.21.15.16:0/2218589568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f134c1537f0 0x7f134c0b7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.729+0000 7f1350e6e640 1 --2- 172.21.15.16:0/2218589568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1334004800 0x7f134c152f80 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:29.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.729+0000 7f1350e6e640 1 -- 172.21.15.16:0/2218589568 >> 172.21.15.16:0/2218589568 conn(0x7f134c09d760 msgr2=0x7f134c15f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:29.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.730+0000 7f1350e6e640 1 -- 172.21.15.16:0/2218589568 shutdown_connections 2024-09-19T07:50:29.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29.730+0000 7f1350e6e640 1 -- 172.21.15.16:0/2218589568 wait complete. 2024-09-19T07:50:29.741 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:50:29.741 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:50:29.741 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:50:29.741 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:50:29.741 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:50:29.741 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:50:29.741 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:50:29.741 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:50:29.741 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:50:29.741 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:50:29.742 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:50:29.742 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:50:29.742 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:50:29.742 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:50:29.742 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:50:29.742 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:50:29.742 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:50:29.742 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:50:29.742 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:50:29.742 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:50:29.742 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:50:29.742 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:50:29.742 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:50:29.742 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:50:29.742 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:50:29.742 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:50:29.743 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:50:29.822 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.61430.asok 2024-09-19T07:50:29.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:50:29.832 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.61430.asok 2024-09-19T07:50:29.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:50:29.833 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.61430.asok mds_sessions 2024-09-19T07:50:29.968 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:50:29.969 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 6576, 2024-09-19T07:50:29.969 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:50:29.969 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:50:29.969 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:50:29.969 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 6576 2024-09-19T07:50:29.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:29.969 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:50:29.969 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:50:29.969 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:50:29.969 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1620886677 2024-09-19T07:50:29.970 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:29.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:29.970 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.6576 v1:192.168.0.1:0/1620886677", 2024-09-19T07:50:29.970 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "v1:192.168.0.1:0/1620886677", 2024-09-19T07:50:29.970 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions": [ 2024-09-19T07:50:29.970 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:29.970 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": 0, 2024-09-19T07:50:29.970 INFO:teuthology.orchestra.run.smithi016.stdout: "addrs": { 2024-09-19T07:50:29.970 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-19T07:50:29.970 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:29.970 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-19T07:50:29.970 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6833", 2024-09-19T07:50:29.971 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1305244492 2024-09-19T07:50:29.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:29.971 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:50:29.971 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:50:29.971 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6835", 2024-09-19T07:50:29.971 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1305244492 2024-09-19T07:50:29.971 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:29.971 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T07:50:29.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:50:29.971 INFO:teuthology.orchestra.run.smithi016.stdout: "seq": 0, 2024-09-19T07:50:29.971 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_gen": 0, 2024-09-19T07:50:29.971 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_ttl": "2024-09-19T07:51:26.952015+0000", 2024-09-19T07:50:29.971 INFO:teuthology.orchestra.run.smithi016.stdout: "last_cap_renew_request": "2024-09-19T07:50:26.952015+0000", 2024-09-19T07:50:29.971 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_renew_seq": 2, 2024-09-19T07:50:29.972 INFO:teuthology.orchestra.run.smithi016.stdout: "num_caps": 1, 2024-09-19T07:50:29.972 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "open" 2024-09-19T07:50:29.972 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:50:29.972 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:50:29.972 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 47 2024-09-19T07:50:29.972 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:50:29.978 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:50:29.978 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:50:29.995 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-19T07:50:29.995 DEBUG:teuthology.orchestra.run.smithi016:> import fcntl 2024-09-19T07:50:29.995 DEBUG:teuthology.orchestra.run.smithi016:> import sys 2024-09-19T07:50:29.995 DEBUG:teuthology.orchestra.run.smithi016:> import time 2024-09-19T07:50:29.995 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:50:29.995 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:50:29.995 DEBUG:teuthology.orchestra.run.smithi016:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-19T07:50:29.995 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:50:29.995 DEBUG:teuthology.orchestra.run.smithi016:> n = 400 2024-09-19T07:50:29.995 DEBUG:teuthology.orchestra.run.smithi016:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2024-09-19T07:50:29.995 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:50:29.995 DEBUG:teuthology.orchestra.run.smithi016:> if not os.path.exists(abs_path): 2024-09-19T07:50:29.995 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(abs_path) 2024-09-19T07:50:29.996 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:50:29.996 DEBUG:teuthology.orchestra.run.smithi016:> handles = [] 2024-09-19T07:50:29.996 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, n): 2024-09-19T07:50:29.996 DEBUG:teuthology.orchestra.run.smithi016:> fname = "file_"+str(i) 2024-09-19T07:50:29.996 DEBUG:teuthology.orchestra.run.smithi016:> path = os.path.join(abs_path, fname) 2024-09-19T07:50:29.996 DEBUG:teuthology.orchestra.run.smithi016:> handles.append(open(path, '"'"'w'"'"')) 2024-09-19T07:50:29.996 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:50:29.996 DEBUG:teuthology.orchestra.run.smithi016:> print("waiting with handles open", file=sys.stderr) 2024-09-19T07:50:29.997 DEBUG:teuthology.orchestra.run.smithi016:> while True: 2024-09-19T07:50:29.997 DEBUG:teuthology.orchestra.run.smithi016:> print("open_n_background: keeping files open", file=sys.stderr) 2024-09-19T07:50:29.997 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-19T07:50:29.997 DEBUG:teuthology.orchestra.run.smithi016:> if os.read(0, 4096) == b"": 2024-09-19T07:50:29.997 DEBUG:teuthology.orchestra.run.smithi016:> break 2024-09-19T07:50:29.997 DEBUG:teuthology.orchestra.run.smithi016:> except BlockingIOError: 2024-09-19T07:50:29.997 DEBUG:teuthology.orchestra.run.smithi016:> pass 2024-09-19T07:50:29.997 DEBUG:teuthology.orchestra.run.smithi016:> time.sleep(2) 2024-09-19T07:50:29.997 DEBUG:teuthology.orchestra.run.smithi016:> print("stdin closed, goodbye!", file=sys.stderr) 2024-09-19T07:50:29.997 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:50:30.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:50:30.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.144+0000 7f84e9477640 1 Processor -- start 2024-09-19T07:50:30.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.144+0000 7f84e9477640 1 -- start start 2024-09-19T07:50:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.145+0000 7f84e9477640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84e4072250 0x7f84e40cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.145+0000 7f84e9477640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f84e40726c0 0x7f84e40d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.145+0000 7f84e9477640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84e40d2b30 con 0x7f84e40d2750 2024-09-19T07:50:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.145+0000 7f84e9477640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f84e4145bd0 con 0x7f84e4072250 2024-09-19T07:50:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.145+0000 7f84e9477640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f84e41427e0 con 0x7f84e40726c0 2024-09-19T07:50:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.145+0000 7f84e27fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84e4072250 0x7f84e40cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.145+0000 7f84e2ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f84e40d2750 0x7f84e413f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41282/0 (socket says 172.21.15.16:41282) 2024-09-19T07:50:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.145+0000 7f84e27fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84e4072250 0x7f84e40cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50750/0 (socket says 172.21.15.16:50750) 2024-09-19T07:50:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.145+0000 7f84e2ffd640 1 -- 172.21.15.16:0/3998198313 learned_addr learned my addr 172.21.15.16:0/3998198313 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.146+0000 7f84e37fe640 1 --2- 172.21.15.16:0/3998198313 >> v2:172.21.15.146:3300/0 conn(0x7f84e40726c0 0x7f84e40d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.146+0000 7f84e27fc640 1 -- 172.21.15.16:0/3998198313 >> v2:172.21.15.146:3300/0 conn(0x7f84e40726c0 msgr2=0x7f84e40d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.146+0000 7f84e27fc640 1 --2- 172.21.15.16:0/3998198313 >> v2:172.21.15.146:3300/0 conn(0x7f84e40726c0 0x7f84e40d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.146+0000 7f84e1ffb640 1 -- 172.21.15.16:0/3998198313 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1969279250 0 0) 0x7f84e40d2b30 con 0x7f84e40d2750 2024-09-19T07:50:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.146+0000 7f84e27fc640 1 -- 172.21.15.16:0/3998198313 >> v1:172.21.15.16:6789/0 conn(0x7f84e40d2750 legacy=0x7f84e413f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.146+0000 7f84e27fc640 1 -- 172.21.15.16:0/3998198313 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e4141c60 con 0x7f84e4072250 2024-09-19T07:50:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.146+0000 7f84e37fe640 1 --2- 172.21.15.16:0/3998198313 >> v2:172.21.15.146:3300/0 conn(0x7f84e40726c0 0x7f84e40d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.147+0000 7f84e27fc640 1 --2- 172.21.15.16:0/3998198313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e4072250 0x7f84e40cfdc0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f84e40b1d60 tx=0x7f84d802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f565623fd27944 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.147+0000 7f84e1ffb640 1 -- 172.21.15.16:0/3998198313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8037070 con 0x7f84e4072250 2024-09-19T07:50:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.147+0000 7f84e1ffb640 1 -- 172.21.15.16:0/3998198313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d8032440 con 0x7f84e4072250 2024-09-19T07:50:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.147+0000 7f84e1ffb640 1 -- 172.21.15.16:0/3998198313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d802ae60 con 0x7f84e4072250 2024-09-19T07:50:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.147+0000 7f84e9477640 1 -- 172.21.15.16:0/3998198313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e4072250 msgr2=0x7f84e40cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.147+0000 7f84e9477640 1 --2- 172.21.15.16:0/3998198313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e4072250 0x7f84e40cfdc0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f84e40b1d60 tx=0x7f84d802a110 comp rx=0 tx=0).stop 2024-09-19T07:50:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.147+0000 7f84e9477640 1 -- 172.21.15.16:0/3998198313 shutdown_connections 2024-09-19T07:50:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.147+0000 7f84e9477640 1 --2- 172.21.15.16:0/3998198313 >> v2:172.21.15.146:3300/0 conn(0x7f84e40726c0 0x7f84e40d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.147+0000 7f84e9477640 1 --2- 172.21.15.16:0/3998198313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e4072250 0x7f84e40cfdc0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.148+0000 7f84e9477640 1 -- 172.21.15.16:0/3998198313 >> 172.21.15.16:0/3998198313 conn(0x7f84e40b4040 msgr2=0x7f84e40b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.148+0000 7f84e9477640 1 -- 172.21.15.16:0/3998198313 shutdown_connections 2024-09-19T07:50:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.148+0000 7f84e9477640 1 -- 172.21.15.16:0/3998198313 wait complete. 2024-09-19T07:50:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.149+0000 7f84e9477640 1 Processor -- start 2024-09-19T07:50:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.149+0000 7f84e9477640 1 -- start start 2024-09-19T07:50:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.150+0000 7f84e9477640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84e4072250 0x7f84e415b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.150+0000 7f84e9477640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40726c0 0x7f84e41f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.150+0000 7f84e9477640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84e40d2750 0x7f84e41f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.150+0000 7f84e9477640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84e4142500 con 0x7f84e40726c0 2024-09-19T07:50:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.150+0000 7f84e9477640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f84e4142220 con 0x7f84e4072250 2024-09-19T07:50:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.150+0000 7f84e9477640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f84e41418b0 con 0x7f84e40d2750 2024-09-19T07:50:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.150+0000 7f84e27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40726c0 0x7f84e41f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.150+0000 7f84e27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40726c0 0x7f84e41f78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50752/0 (socket says 172.21.15.16:50752) 2024-09-19T07:50:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.150+0000 7f84e27fc640 1 -- 172.21.15.16:0/2539066044 learned_addr learned my addr 172.21.15.16:0/2539066044 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.150+0000 7f84e2ffd640 1 --2- 172.21.15.16:0/2539066044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84e4072250 0x7f84e415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.150+0000 7f84e37fe640 1 --2- 172.21.15.16:0/2539066044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84e40d2750 0x7f84e41f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.151+0000 7f84e27fc640 1 -- 172.21.15.16:0/2539066044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84e40d2750 msgr2=0x7f84e41f7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.151+0000 7f84e27fc640 1 --2- 172.21.15.16:0/2539066044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84e40d2750 0x7f84e41f7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.151+0000 7f84e27fc640 1 -- 172.21.15.16:0/2539066044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84e4072250 msgr2=0x7f84e415b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.151+0000 7f84e27fc640 1 --2- 172.21.15.16:0/2539066044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84e4072250 0x7f84e415b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.151+0000 7f84e27fc640 1 -- 172.21.15.16:0/2539066044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e40c9510 con 0x7f84e40726c0 2024-09-19T07:50:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.151+0000 7f84e37fe640 1 --2- 172.21.15.16:0/2539066044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84e40d2750 0x7f84e41f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.151+0000 7f84e2ffd640 1 --2- 172.21.15.16:0/2539066044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84e4072250 0x7f84e415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.151+0000 7f84e27fc640 1 --2- 172.21.15.16:0/2539066044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40726c0 0x7f84e41f78a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f84d8000c00 tx=0x7f84d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.151+0000 7f84cb7fe640 1 -- 172.21.15.16:0/2539066044 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8037040 con 0x7f84e40726c0 2024-09-19T07:50:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.151+0000 7f84cb7fe640 1 -- 172.21.15.16:0/2539066044 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d8042070 con 0x7f84e40726c0 2024-09-19T07:50:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.151+0000 7f84cb7fe640 1 -- 172.21.15.16:0/2539066044 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8030a50 con 0x7f84e40726c0 2024-09-19T07:50:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.151+0000 7f84e9477640 1 -- 172.21.15.16:0/2539066044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e40c97f0 con 0x7f84e40726c0 2024-09-19T07:50:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.151+0000 7f84e9477640 1 -- 172.21.15.16:0/2539066044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e415bc70 con 0x7f84e40726c0 2024-09-19T07:50:30.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.152+0000 7f84e9477640 1 -- 172.21.15.16:0/2539066044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e40c92a0 con 0x7f84e40726c0 2024-09-19T07:50:30.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.153+0000 7f84cb7fe640 1 -- 172.21.15.16:0/2539066044 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f84d8030bf0 con 0x7f84e40726c0 2024-09-19T07:50:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.153+0000 7f84cb7fe640 1 --2- 172.21.15.16:0/2539066044 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f84d006c2b0 0x7f84d006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.154+0000 7f84cb7fe640 1 -- 172.21.15.16:0/2539066044 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f84d80ac030 con 0x7f84e40726c0 2024-09-19T07:50:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.154+0000 7f84e2ffd640 1 --2- 172.21.15.16:0/2539066044 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f84d006c2b0 0x7f84d006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.154+0000 7f84e2ffd640 1 --2- 172.21.15.16:0/2539066044 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f84d006c2b0 0x7f84d006e770 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f84d4000f60 tx=0x7f84d4007c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:30.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.158+0000 7f84cb7fe640 1 -- 172.21.15.16:0/2539066044 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f84d8045360 con 0x7f84e40726c0 2024-09-19T07:50:30.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.383+0000 7f84e9477640 1 -- 172.21.15.16:0/2539066044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84e40b7960 con 0x7f84e40726c0 2024-09-19T07:50:30.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.383+0000 7f84cb7fe640 1 -- 172.21.15.16:0/2539066044 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5231 (secure 0 0 0) 0x7f84d807a6a0 con 0x7f84e40726c0 2024-09-19T07:50:30.384 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:30.384 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":47,"btime":"2024-09-19T07:50:17:700363+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5684,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1865971154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1865971154},{"type":"v1","addr":"172.21.15.146:6837","nonce":1865971154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6094,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/2735152611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2735152611},{"type":"v1","addr":"172.21.15.146:6836","nonce":2735152611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6399,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4204683210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4204683210},{"type":"v1","addr":"172.21.15.16:6834","nonce":4204683210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:50:15.050678+0000","modified":"2024-09-19T07:50:17.046800+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6402":{"gid":6402,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/1305244492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1305244492},{"type":"v1","addr":"172.21.15.16:6835","nonce":1305244492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6402,"qdb_cluster":[6402]},"id":6}]} 2024-09-19T07:50:30.384 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 47 2024-09-19T07:50:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.385+0000 7f84e9477640 1 -- 172.21.15.16:0/2539066044 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f84d006c2b0 msgr2=0x7f84d006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.385+0000 7f84e9477640 1 --2- 172.21.15.16:0/2539066044 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f84d006c2b0 0x7f84d006e770 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f84d4000f60 tx=0x7f84d4007c10 comp rx=0 tx=0).stop 2024-09-19T07:50:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.385+0000 7f84e9477640 1 -- 172.21.15.16:0/2539066044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40726c0 msgr2=0x7f84e41f78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.385+0000 7f84e9477640 1 --2- 172.21.15.16:0/2539066044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40726c0 0x7f84e41f78a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f84d8000c00 tx=0x7f84d80046c0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.385+0000 7f84e9477640 1 -- 172.21.15.16:0/2539066044 shutdown_connections 2024-09-19T07:50:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.385+0000 7f84e9477640 1 --2- 172.21.15.16:0/2539066044 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f84d006c2b0 0x7f84d006e770 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.385+0000 7f84e9477640 1 --2- 172.21.15.16:0/2539066044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84e40d2750 0x7f84e41f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.385+0000 7f84e9477640 1 --2- 172.21.15.16:0/2539066044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40726c0 0x7f84e41f78a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.385+0000 7f84e9477640 1 --2- 172.21.15.16:0/2539066044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84e4072250 0x7f84e415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.385+0000 7f84e9477640 1 -- 172.21.15.16:0/2539066044 >> 172.21.15.16:0/2539066044 conn(0x7f84e40b4040 msgr2=0x7f84e413fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.385+0000 7f84e9477640 1 -- 172.21.15.16:0/2539066044 shutdown_connections 2024-09-19T07:50:30.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:30.385+0000 7f84e9477640 1 -- 172.21.15.16:0/2539066044 wait complete. 2024-09-19T07:50:30.398 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-19T07:50:30.545 INFO:teuthology.orchestra.run.smithi016.stdout:[{"id":6226,"entity":{"name":{"type":"client","num":6226},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":4223664118}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":3.0359578049999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9910308817710756,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6226 192.168.0.1:0/4223664118","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"46954","root":"/"}},{"id":6576,"entity":{"name":{"type":"client","num":6576},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1620886677}},"state":"open","num_leases":1,"num_caps":276,"request_load_avg":277,"uptime":4.5829363049999996,"requests_in_flight":274,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":553.703617283423,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":5,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6576 192.168.0.1:0/1620886677","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"0","hostname":"smithi016","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"61430","root":"/"}}] 2024-09-19T07:50:30.545 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "46954", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4223664118, "type": "any" }, "name": { "num": 6226, "type": "client" } }, "id": 6226, "inst": "client.6226 192.168.0.1:0/4223664118", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9910308817710756 }, "state": "open", "uptime": 3.035957805 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "0", "hostname": "smithi016", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "61430", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1620886677, "type": "any" }, "name": { "num": 6576, "type": "client" } }, "id": 6576, "inst": "client.6576 192.168.0.1:0/1620886677", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 5, "num_caps": 276, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 277, "requests_in_flight": 274, "session_cache_liveness": { "halflife": 300, "value": 553.703617283423 }, "state": "open", "uptime": 4.582936305 } ] 2024-09-19T07:50:30.545 DEBUG:tasks.ceph_test_case:wait_until_equal: 276 != 402, waiting (timeout=600)... 2024-09-19T07:50:30.715 INFO:teuthology.orchestra.run.smithi016.stderr:waiting with handles open 2024-09-19T07:50:30.715 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:50:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:50:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:50:35.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:50:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.662+0000 7f3c18ddf640 1 Processor -- start 2024-09-19T07:50:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.662+0000 7f3c18ddf640 1 -- start start 2024-09-19T07:50:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.663+0000 7f3c18ddf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c14072590 0x7f3c1406d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.663+0000 7f3c18ddf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c141620b0 con 0x7f3c140d21f0 2024-09-19T07:50:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.663+0000 7f3c18ddf640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c14162370 con 0x7f3c14141ad0 2024-09-19T07:50:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.663+0000 7f3c18ddf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3c14141750 con 0x7f3c14072590 2024-09-19T07:50:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.663+0000 7f3c1259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3c140d21f0 0x7f3c140d25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52408/0 (socket says 172.21.15.16:52408) 2024-09-19T07:50:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.663+0000 7f3c1259c640 1 -- 172.21.15.16:0/1837904778 learned_addr learned my addr 172.21.15.16:0/1837904778 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.663+0000 7f3c0bfff640 1 --2- 172.21.15.16:0/1837904778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c14072590 0x7f3c1406d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.663+0000 7f3c11d9b640 1 -- 172.21.15.16:0/1837904778 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2517197459 0 0) 0x7f3c141620b0 con 0x7f3c140d21f0 2024-09-19T07:50:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.663+0000 7f3c11d9b640 1 -- 172.21.15.16:0/1837904778 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c00003520 con 0x7f3c140d21f0 2024-09-19T07:50:35.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.663+0000 7f3c11d9b640 1 -- 172.21.15.16:0/1837904778 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1912650159 0 0) 0x7f3c14162370 con 0x7f3c14141ad0 2024-09-19T07:50:35.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.664+0000 7f3c11d9b640 1 -- 172.21.15.16:0/1837904778 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c141620b0 con 0x7f3c14141ad0 2024-09-19T07:50:35.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.664+0000 7f3c11d9b640 1 -- 172.21.15.16:0/1837904778 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3219350126 0 0) 0x7f3c00003520 con 0x7f3c140d21f0 2024-09-19T07:50:35.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.664+0000 7f3c0bfff640 1 -- 172.21.15.16:0/1837904778 >> v1:172.21.15.146:6789/0 conn(0x7f3c14141ad0 legacy=0x7f3c14141f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:35.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.664+0000 7f3c0bfff640 1 -- 172.21.15.16:0/1837904778 >> v1:172.21.15.16:6789/0 conn(0x7f3c140d21f0 legacy=0x7f3c140d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:35.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.664+0000 7f3c0bfff640 1 -- 172.21.15.16:0/1837904778 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c14162650 con 0x7f3c14072590 2024-09-19T07:50:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.664+0000 7f3c0bfff640 1 --2- 172.21.15.16:0/1837904778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c14072590 0x7f3c1406d0e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0097d0 tx=0x7f3bfc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a50bc180b9755a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.665+0000 7f3c11d9b640 1 -- 172.21.15.16:0/1837904778 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc037070 con 0x7f3c14072590 2024-09-19T07:50:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.665+0000 7f3c11d9b640 1 -- 172.21.15.16:0/1837904778 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bfc02ab90 con 0x7f3c14072590 2024-09-19T07:50:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.665+0000 7f3c11d9b640 1 -- 172.21.15.16:0/1837904778 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04001670 con 0x7f3c14072590 2024-09-19T07:50:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.665+0000 7f3c18ddf640 1 -- 172.21.15.16:0/1837904778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c14072590 msgr2=0x7f3c1406d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.665+0000 7f3c18ddf640 1 --2- 172.21.15.16:0/1837904778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c14072590 0x7f3c1406d0e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0097d0 tx=0x7f3bfc02a100 comp rx=0 tx=0).stop 2024-09-19T07:50:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.665+0000 7f3c18ddf640 1 -- 172.21.15.16:0/1837904778 shutdown_connections 2024-09-19T07:50:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.665+0000 7f3c18ddf640 1 --2- 172.21.15.16:0/1837904778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c14072590 0x7f3c1406d0e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.665+0000 7f3c18ddf640 1 -- 172.21.15.16:0/1837904778 >> 172.21.15.16:0/1837904778 conn(0x7f3c140b3ed0 msgr2=0x7f3c140b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.665+0000 7f3c18ddf640 1 -- 172.21.15.16:0/1837904778 shutdown_connections 2024-09-19T07:50:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.665+0000 7f3c18ddf640 1 -- 172.21.15.16:0/1837904778 wait complete. 2024-09-19T07:50:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.667+0000 7f3c18ddf640 1 Processor -- start 2024-09-19T07:50:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.667+0000 7f3c18ddf640 1 -- start start 2024-09-19T07:50:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.667+0000 7f3c18ddf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c14072590 0x7f3c14140ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.667+0000 7f3c18ddf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c140d21f0 0x7f3c140c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c18ddf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c14141ad0 0x7f3c140c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c18ddf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1406d620 con 0x7f3c14072590 2024-09-19T07:50:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c18ddf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3c14162c10 con 0x7f3c140d21f0 2024-09-19T07:50:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c18ddf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1406d7a0 con 0x7f3c14141ad0 2024-09-19T07:50:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c1259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c14072590 0x7f3c14140ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c1259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c14072590 0x7f3c14140ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49584/0 (socket says 172.21.15.16:49584) 2024-09-19T07:50:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c1259c640 1 -- 172.21.15.16:0/3029775338 learned_addr learned my addr 172.21.15.16:0/3029775338 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c12d9d640 1 --2- 172.21.15.16:0/3029775338 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c14141ad0 0x7f3c140c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c0bfff640 1 --2- 172.21.15.16:0/3029775338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c140d21f0 0x7f3c140c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c1259c640 1 -- 172.21.15.16:0/3029775338 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c14141ad0 msgr2=0x7f3c140c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c1259c640 1 --2- 172.21.15.16:0/3029775338 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c14141ad0 0x7f3c140c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c1259c640 1 -- 172.21.15.16:0/3029775338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c140d21f0 msgr2=0x7f3c140c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c1259c640 1 --2- 172.21.15.16:0/3029775338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c140d21f0 0x7f3c140c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c1259c640 1 -- 172.21.15.16:0/3029775338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c14159ab0 con 0x7f3c14072590 2024-09-19T07:50:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c1259c640 1 --2- 172.21.15.16:0/3029775338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c14072590 0x7f3c14140ce0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3c04001eb0 tx=0x7f3c0400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c0affd640 1 -- 172.21.15.16:0/3029775338 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04017070 con 0x7f3c14072590 2024-09-19T07:50:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c0affd640 1 -- 172.21.15.16:0/3029775338 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c04005ad0 con 0x7f3c14072590 2024-09-19T07:50:35.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c0affd640 1 -- 172.21.15.16:0/3029775338 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04007070 con 0x7f3c14072590 2024-09-19T07:50:35.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c18ddf640 1 -- 172.21.15.16:0/3029775338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c14159d90 con 0x7f3c14072590 2024-09-19T07:50:35.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.668+0000 7f3c18ddf640 1 -- 172.21.15.16:0/3029775338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c1415a350 con 0x7f3c14072590 2024-09-19T07:50:35.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.670+0000 7f3c18ddf640 1 -- 172.21.15.16:0/3029775338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c14158c30 con 0x7f3c14072590 2024-09-19T07:50:35.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.670+0000 7f3c0affd640 1 -- 172.21.15.16:0/3029775338 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3c0401a070 con 0x7f3c14072590 2024-09-19T07:50:35.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.670+0000 7f3c0affd640 1 --2- 172.21.15.16:0/3029775338 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3bf406c2b0 0x7f3bf406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.670+0000 7f3c0affd640 1 -- 172.21.15.16:0/3029775338 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f3c0408ab50 con 0x7f3c14072590 2024-09-19T07:50:35.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.671+0000 7f3c0bfff640 1 --2- 172.21.15.16:0/3029775338 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3bf406c2b0 0x7f3bf406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.671+0000 7f3c0bfff640 1 --2- 172.21.15.16:0/3029775338 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3bf406c2b0 0x7f3bf406e770 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc034000 tx=0x7f3bfc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.675+0000 7f3c0affd640 1 -- 172.21.15.16:0/3029775338 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3c04059130 con 0x7f3c14072590 2024-09-19T07:50:35.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.889+0000 7f3c18ddf640 1 -- 172.21.15.16:0/3029775338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c140690c0 con 0x7f3c14072590 2024-09-19T07:50:35.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.890+0000 7f3c0affd640 1 -- 172.21.15.16:0/3029775338 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5231 (secure 0 0 0) 0x7f3c04058a30 con 0x7f3c14072590 2024-09-19T07:50:35.890 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:35.890 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":48,"btime":"2024-09-19T07:50:32:187543+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5684,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1865971154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1865971154},{"type":"v1","addr":"172.21.15.146:6837","nonce":1865971154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6094,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/2735152611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2735152611},{"type":"v1","addr":"172.21.15.146:6836","nonce":2735152611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6399,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4204683210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4204683210},{"type":"v1","addr":"172.21.15.16:6834","nonce":4204683210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:50:15.050678+0000","modified":"2024-09-19T07:50:31.729510+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6402":{"gid":6402,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":7,"addr":"172.21.15.16:6835/1305244492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1305244492},{"type":"v1","addr":"172.21.15.16:6835","nonce":1305244492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6402,"qdb_cluster":[6402]},"id":6}]} 2024-09-19T07:50:35.891 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 48 2024-09-19T07:50:35.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.892+0000 7f3c18ddf640 1 -- 172.21.15.16:0/3029775338 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3bf406c2b0 msgr2=0x7f3bf406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.892+0000 7f3c18ddf640 1 --2- 172.21.15.16:0/3029775338 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3bf406c2b0 0x7f3bf406e770 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc034000 tx=0x7f3bfc02e000 comp rx=0 tx=0).stop 2024-09-19T07:50:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.892+0000 7f3c18ddf640 1 -- 172.21.15.16:0/3029775338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c14072590 msgr2=0x7f3c14140ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.892+0000 7f3c18ddf640 1 --2- 172.21.15.16:0/3029775338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c14072590 0x7f3c14140ce0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3c04001eb0 tx=0x7f3c0400cbe0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.892+0000 7f3c18ddf640 1 -- 172.21.15.16:0/3029775338 shutdown_connections 2024-09-19T07:50:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.892+0000 7f3c18ddf640 1 --2- 172.21.15.16:0/3029775338 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3bf406c2b0 0x7f3bf406e770 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.892+0000 7f3c18ddf640 1 --2- 172.21.15.16:0/3029775338 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c14141ad0 0x7f3c140c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.892+0000 7f3c18ddf640 1 --2- 172.21.15.16:0/3029775338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c140d21f0 0x7f3c140c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.892+0000 7f3c18ddf640 1 --2- 172.21.15.16:0/3029775338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c14072590 0x7f3c14140ce0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.892+0000 7f3c18ddf640 1 -- 172.21.15.16:0/3029775338 >> 172.21.15.16:0/3029775338 conn(0x7f3c140b3ed0 msgr2=0x7f3c140b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.892+0000 7f3c18ddf640 1 -- 172.21.15.16:0/3029775338 shutdown_connections 2024-09-19T07:50:35.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:35.892+0000 7f3c18ddf640 1 -- 172.21.15.16:0/3029775338 wait complete. 2024-09-19T07:50:35.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-19T07:50:36.026 INFO:teuthology.orchestra.run.smithi016.stdout:[{"id":6226,"entity":{"name":{"type":"client","num":6226},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":4223664118}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":8.5168816290000002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9533924170089287,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6226 192.168.0.1:0/4223664118","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"46954","root":"/"}},{"id":6576,"entity":{"name":{"type":"client","num":6576},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1620886677}},"state":"open","num_leases":1,"num_caps":402,"request_load_avg":379,"uptime":10.063860129,"requests_in_flight":0,"num_completed_requests":369,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":1161.8326481062811,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":438.25760867024445,"halflife":1.5},"recall_caps_throttle2o":{"value":172.6765465365375,"halflife":0.5},"session_cache_liveness":{"value":795.61328356339118,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":36,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6576 192.168.0.1:0/1620886677","completed_requests":[{"tid":36,"created_ino":"0x10000000214"},{"tid":37,"created_ino":"0x10000000215"},{"tid":38,"created_ino":"0x10000000216"},{"tid":39,"created_ino":"0x10000000217"},{"tid":40,"created_ino":"0x10000000218"},{"tid":41,"created_ino":"0x10000000219"},{"tid":42,"created_ino":"0x1000000021a"},{"tid":43,"created_ino":"0x1000000021b"},{"tid":44,"created_ino":"0x1000000021c"},{"tid":45,"created_ino":"0x1000000021d"},{"tid":46,"created_ino":"0x1000000021e"},{"tid":47,"created_ino":"0x1000000021f"},{"tid":48,"created_ino":"0x10000000220"},{"tid":49,"created_ino":"0x10000000221"},{"tid":50,"created_ino":"0x10000000222"},{"tid":51,"created_ino":"0x10000000223"},{"tid":52,"created_ino":"0x10000000224"},{"tid":53,"created_ino":"0x10000000225"},{"tid":54,"created_ino":"0x10000000226"},{"tid":55,"created_ino":"0x10000000227"},{"tid":56,"created_ino":"0x10000000228"},{"tid":57,"created_ino":"0x10000000229"},{"tid":58,"created_ino":"0x1000000022a"},{"tid":59,"created_ino":"0x1000000022b"},{"tid":60,"created_ino":"0x1000000022c"},{"tid":61,"created_ino":"0x1000000022d"},{"tid":62,"created_ino":"0x1000000022e"},{"tid":63,"created_ino":"0x1000000022f"},{"tid":64,"created_ino":"0x10000000230"},{"tid":65,"created_ino":"0x10000000231"},{"tid":66,"created_ino":"0x10000000232"},{"tid":67,"created_ino":"0x10000000233"},{"tid":68,"created_ino":"0x10000000234"},{"tid":69,"created_ino":"0x10000000235"},{"tid":70,"created_ino":"0x10000000236"},{"tid":71,"created_ino":"0x10000000237"},{"tid":72,"created_ino":"0x10000000238"},{"tid":73,"created_ino":"0x10000000239"},{"tid":74,"created_ino":"0x1000000023a"},{"tid":75,"created_ino":"0x1000000023b"},{"tid":76,"created_ino":"0x1000000023c"},{"tid":77,"created_ino":"0x1000000023d"},{"tid":78,"created_ino":"0x1000000023e"},{"tid":79,"created_ino":"0x1000000023f"},{"tid":80,"created_ino":"0x10000000240"},{"tid":81,"created_ino":"0x10000000241"},{"tid":82,"created_ino":"0x10000000242"},{"tid":83,"created_ino":"0x10000000243"},{"tid":84,"created_ino":"0x10000000244"},{"tid":85,"created_ino":"0x10000000245"},{"tid":86,"created_ino":"0x10000000246"},{"tid":87,"created_ino":"0x10000000247"},{"tid":88,"created_ino":"0x10000000248"},{"tid":89,"created_ino":"0x10000000249"},{"tid":90,"created_ino":"0x1000000024a"},{"tid":91,"created_ino":"0x1000000024b"},{"tid":92,"created_ino":"0x1000000024c"},{"tid":93,"created_ino":"0x1000000024d"},{"tid":94,"created_ino":"0x1000000024e"},{"tid":95,"created_ino":"0x1000000024f"},{"tid":96,"created_ino":"0x10000000250"},{"tid":97,"created_ino":"0x10000000251"},{"tid":98,"created_ino":"0x10000000252"},{"tid":99,"created_ino":"0x10000000253"},{"tid":100,"created_ino":"0x10000000254"},{"tid":101,"created_ino":"0x10000000255"},{"tid":102,"created_ino":"0x10000000256"},{"tid":103,"created_ino":"0x10000000257"},{"tid":104,"created_ino":"0x10000000258"},{"tid":105,"created_ino":"0x10000000259"},{"tid":106,"created_ino":"0x1000000025a"},{"tid":107,"created_ino":"0x1000000025b"},{"tid":108,"created_ino":"0x1000000025c"},{"tid":109,"created_ino":"0x1000000025d"},{"tid":110,"created_ino":"0x1000000025e"},{"tid":111,"created_ino":"0x1000000025f"},{"tid":112,"created_ino":"0x10000000260"},{"tid":113,"created_ino":"0x10000000261"},{"tid":114,"created_ino":"0x10000000262"},{"tid":115,"created_ino":"0x10000000263"},{"tid":116,"created_ino":"0x10000000264"},{"tid":117,"created_ino":"0x10000000265"},{"tid":118,"created_ino":"0x10000000266"},{"tid":119,"created_ino":"0x10000000267"},{"tid":120,"created_ino":"0x10000000268"},{"tid":121,"created_ino":"0x10000000269"},{"tid":122,"created_ino":"0x1000000026a"},{"tid":123,"created_ino":"0x1000000026b"},{"tid":124,"created_ino":"0x1000000026c"},{"tid":125,"created_ino":"0x1000000026d"},{"tid":126,"created_ino":"0x1000000026e"},{"tid":127,"created_ino":"0x1000000026f"},{"tid":128,"created_ino":"0x10000000270"},{"tid":129,"created_ino":"0x10000000271"},{"tid":130,"created_ino":"0x10000000272"},{"tid":131,"created_ino":"0x10000000273"},{"tid":132,"created_ino":"0x10000000274"},{"tid":133,"created_ino":"0x10000000275"},{"tid":134,"created_ino":"0x10000000276"},{"tid":135,"created_ino":"0x10000000277"},{"tid":136,"created_ino":"0x10000000278"},{"tid":137,"created_ino":"0x10000000279"},{"tid":138,"created_ino":"0x1000000027a"},{"tid":139,"created_ino":"0x1000000027b"},{"tid":140,"created_ino":"0x1000000027c"},{"tid":141,"created_ino":"0x1000000027d"},{"tid":142,"created_ino":"0x1000000027e"},{"tid":143,"created_ino":"0x1000000027f"},{"tid":144,"created_ino":"0x10000000280"},{"tid":145,"created_ino":"0x10000000281"},{"tid":146,"created_ino":"0x10000000282"},{"tid":147,"created_ino":"0x10000000283"},{"tid":148,"created_ino":"0x10000000284"},{"tid":149,"created_ino":"0x10000000285"},{"tid":150,"created_ino":"0x10000000286"},{"tid":151,"created_ino":"0x10000000287"},{"tid":152,"created_ino":"0x10000000288"},{"tid":153,"created_ino":"0x10000000289"},{"tid":154,"created_ino":"0x1000000028a"},{"tid":155,"created_ino":"0x1000000028b"},{"tid":156,"created_ino":"0x1000000028c"},{"tid":157,"created_ino":"0x1000000028d"},{"tid":158,"created_ino":"0x1000000028e"},{"tid":159,"created_ino":"0x1000000028f"},{"tid":160,"created_ino":"0x10000000290"},{"tid":161,"created_ino":"0x10000000291"},{"tid":162,"created_ino":"0x10000000292"},{"tid":163,"created_ino":"0x10000000293"},{"tid":164,"created_ino":"0x10000000294"},{"tid":165,"created_ino":"0x10000000295"},{"tid":166,"created_ino":"0x10000000296"},{"tid":167,"created_ino":"0x10000000297"},{"tid":168,"created_ino":"0x10000000298"},{"tid":169,"created_ino":"0x10000000299"},{"tid":170,"created_ino":"0x1000000029a"},{"tid":171,"created_ino":"0x1000000029b"},{"tid":172,"created_ino":"0x1000000029c"},{"tid":173,"created_ino":"0x1000000029d"},{"tid":174,"created_ino":"0x1000000029e"},{"tid":175,"created_ino":"0x1000000029f"},{"tid":176,"created_ino":"0x100000002a0"},{"tid":177,"created_ino":"0x100000002a1"},{"tid":178,"created_ino":"0x100000002a2"},{"tid":179,"created_ino":"0x100000002a3"},{"tid":180,"created_ino":"0x100000002a4"},{"tid":181,"created_ino":"0x100000002a5"},{"tid":182,"created_ino":"0x100000002a6"},{"tid":183,"created_ino":"0x100000002a7"},{"tid":184,"created_ino":"0x100000002a8"},{"tid":185,"created_ino":"0x100000002a9"},{"tid":186,"created_ino":"0x100000002aa"},{"tid":187,"created_ino":"0x100000002ab"},{"tid":188,"created_ino":"0x100000002ac"},{"tid":189,"created_ino":"0x100000002ad"},{"tid":190,"created_ino":"0x100000002ae"},{"tid":191,"created_ino":"0x100000002af"},{"tid":192,"created_ino":"0x100000002b0"},{"tid":193,"created_ino":"0x100000002b1"},{"tid":194,"created_ino":"0x100000002b2"},{"tid":195,"created_ino":"0x100000002b3"},{"tid":196,"created_ino":"0x100000002b4"},{"tid":197,"created_ino":"0x100000002b5"},{"tid":198,"created_ino":"0x100000002b6"},{"tid":199,"created_ino":"0x100000002b7"},{"tid":200,"created_ino":"0x100000002b8"},{"tid":201,"created_ino":"0x100000002b9"},{"tid":202,"created_ino":"0x100000002ba"},{"tid":203,"created_ino":"0x100000002bb"},{"tid":204,"created_ino":"0x100000002bc"},{"tid":205,"created_ino":"0x100000002bd"},{"tid":206,"created_ino":"0x100000002be"},{"tid":207,"created_ino":"0x100000002bf"},{"tid":208,"created_ino":"0x100000002c0"},{"tid":209,"created_ino":"0x100000002c1"},{"tid":210,"created_ino":"0x100000002c2"},{"tid":211,"created_ino":"0x100000002c3"},{"tid":212,"created_ino":"0x100000002c4"},{"tid":213,"created_ino":"0x100000002c5"},{"tid":214,"created_ino":"0x100000002c6"},{"tid":215,"created_ino":"0x100000002c7"},{"tid":216,"created_ino":"0x100000002c8"},{"tid":217,"created_ino":"0x100000002c9"},{"tid":218,"created_ino":"0x100000002ca"},{"tid":219,"created_ino":"0x100000002cb"},{"tid":220,"created_ino":"0x100000002cc"},{"tid":221,"created_ino":"0x100000002cd"},{"tid":222,"created_ino":"0x100000002ce"},{"tid":223,"created_ino":"0x100000002cf"},{"tid":224,"created_ino":"0x100000002d0"},{"tid":225,"created_ino":"0x100000002d1"},{"tid":226,"created_ino":"0x100000002d2"},{"tid":227,"created_ino":"0x100000002d3"},{"tid":228,"created_ino":"0x100000002d4"},{"tid":229,"created_ino":"0x100000002d5"},{"tid":230,"created_ino":"0x100000002d6"},{"tid":231,"created_ino":"0x100000002d7"},{"tid":232,"created_ino":"0x100000002d8"},{"tid":233,"created_ino":"0x100000002d9"},{"tid":234,"created_ino":"0x100000002da"},{"tid":235,"created_ino":"0x100000002db"},{"tid":236,"created_ino":"0x100000002dc"},{"tid":237,"created_ino":"0x100000002dd"},{"tid":238,"created_ino":"0x100000002de"},{"tid":239,"created_ino":"0x100000002df"},{"tid":240,"created_ino":"0x100000002e0"},{"tid":241,"created_ino":"0x100000002e1"},{"tid":242,"created_ino":"0x100000002e2"},{"tid":243,"created_ino":"0x100000002e3"},{"tid":244,"created_ino":"0x100000002e4"},{"tid":245,"created_ino":"0x100000002e5"},{"tid":246,"created_ino":"0x100000002e6"},{"tid":247,"created_ino":"0x100000002e7"},{"tid":248,"created_ino":"0x100000002e8"},{"tid":249,"created_ino":"0x100000002e9"},{"tid":250,"created_ino":"0x100000002ea"},{"tid":251,"created_ino":"0x100000002eb"},{"tid":252,"created_ino":"0x100000002ec"},{"tid":253,"created_ino":"0x100000002ed"},{"tid":254,"created_ino":"0x100000002ee"},{"tid":255,"created_ino":"0x100000002ef"},{"tid":256,"created_ino":"0x100000002f0"},{"tid":257,"created_ino":"0x100000002f1"},{"tid":258,"created_ino":"0x100000002f2"},{"tid":259,"created_ino":"0x100000002f3"},{"tid":260,"created_ino":"0x100000002f4"},{"tid":261,"created_ino":"0x100000002f5"},{"tid":262,"created_ino":"0x100000002f6"},{"tid":263,"created_ino":"0x100000002f7"},{"tid":264,"created_ino":"0x100000002f8"},{"tid":265,"created_ino":"0x100000002f9"},{"tid":266,"created_ino":"0x100000002fa"},{"tid":267,"created_ino":"0x100000002fb"},{"tid":268,"created_ino":"0x100000002fc"},{"tid":269,"created_ino":"0x100000002fd"},{"tid":270,"created_ino":"0x100000002fe"},{"tid":271,"created_ino":"0x100000002ff"},{"tid":272,"created_ino":"0x10000000300"},{"tid":273,"created_ino":"0x10000000301"},{"tid":274,"created_ino":"0x10000000302"},{"tid":275,"created_ino":"0x10000000303"},{"tid":276,"created_ino":"0x10000000304"},{"tid":277,"created_ino":"0x10000000305"},{"tid":278,"created_ino":"0x10000000306"},{"tid":279,"created_ino":"0x10000000307"},{"tid":280,"created_ino":"0x10000000308"},{"tid":281,"created_ino":"0x10000000309"},{"tid":282,"created_ino":"0x1000000030a"},{"tid":283,"created_ino":"0x1000000030b"},{"tid":284,"created_ino":"0x1000000030c"},{"tid":285,"created_ino":"0x1000000030d"},{"tid":286,"created_ino":"0x1000000030e"},{"tid":287,"created_ino":"0x1000000030f"},{"tid":288,"created_ino":"0x10000000310"},{"tid":289,"created_ino":"0x10000000311"},{"tid":290,"created_ino":"0x10000000312"},{"tid":291,"created_ino":"0x10000000313"},{"tid":292,"created_ino":"0x10000000314"},{"tid":293,"created_ino":"0x10000000315"},{"tid":294,"created_ino":"0x10000000316"},{"tid":295,"created_ino":"0x10000000317"},{"tid":296,"created_ino":"0x10000000318"},{"tid":297,"created_ino":"0x10000000319"},{"tid":298,"created_ino":"0x1000000031a"},{"tid":299,"created_ino":"0x1000000031b"},{"tid":300,"created_ino":"0x1000000031c"},{"tid":301,"created_ino":"0x1000000031d"},{"tid":302,"created_ino":"0x1000000031e"},{"tid":303,"created_ino":"0x1000000031f"},{"tid":304,"created_ino":"0x10000000320"},{"tid":305,"created_ino":"0x10000000321"},{"tid":306,"created_ino":"0x10000000322"},{"tid":307,"created_ino":"0x10000000323"},{"tid":308,"created_ino":"0x10000000324"},{"tid":309,"created_ino":"0x10000000325"},{"tid":310,"created_ino":"0x10000000326"},{"tid":311,"created_ino":"0x10000000327"},{"tid":312,"created_ino":"0x10000000328"},{"tid":313,"created_ino":"0x10000000329"},{"tid":314,"created_ino":"0x1000000032a"},{"tid":315,"created_ino":"0x1000000032b"},{"tid":316,"created_ino":"0x1000000032c"},{"tid":317,"created_ino":"0x1000000032d"},{"tid":318,"created_ino":"0x1000000032e"},{"tid":319,"created_ino":"0x1000000032f"},{"tid":320,"created_ino":"0x10000000330"},{"tid":321,"created_ino":"0x10000000331"},{"tid":322,"created_ino":"0x10000000332"},{"tid":323,"created_ino":"0x10000000333"},{"tid":324,"created_ino":"0x10000000334"},{"tid":325,"created_ino":"0x10000000335"},{"tid":326,"created_ino":"0x10000000336"},{"tid":327,"created_ino":"0x10000000337"},{"tid":328,"created_ino":"0x10000000338"},{"tid":329,"created_ino":"0x10000000339"},{"tid":330,"created_ino":"0x1000000033a"},{"tid":331,"created_ino":"0x1000000033b"},{"tid":332,"created_ino":"0x1000000033c"},{"tid":333,"created_ino":"0x1000000033d"},{"tid":334,"created_ino":"0x1000000033e"},{"tid":335,"created_ino":"0x1000000033f"},{"tid":336,"created_ino":"0x10000000340"},{"tid":337,"created_ino":"0x10000000341"},{"tid":338,"created_ino":"0x10000000342"},{"tid":339,"created_ino":"0x10000000343"},{"tid":340,"created_ino":"0x10000000344"},{"tid":341,"created_ino":"0x10000000345"},{"tid":342,"created_ino":"0x10000000346"},{"tid":343,"created_ino":"0x10000000347"},{"tid":344,"created_ino":"0x10000000348"},{"tid":345,"created_ino":"0x10000000349"},{"tid":346,"created_ino":"0x1000000034a"},{"tid":347,"created_ino":"0x1000000034b"},{"tid":348,"created_ino":"0x1000000034c"},{"tid":349,"created_ino":"0x1000000034d"},{"tid":350,"created_ino":"0x1000000034e"},{"tid":351,"created_ino":"0x1000000034f"},{"tid":352,"created_ino":"0x10000000350"},{"tid":353,"created_ino":"0x10000000351"},{"tid":354,"created_ino":"0x10000000352"},{"tid":355,"created_ino":"0x10000000353"},{"tid":356,"created_ino":"0x10000000354"},{"tid":357,"created_ino":"0x10000000355"},{"tid":358,"created_ino":"0x10000000356"},{"tid":359,"created_ino":"0x10000000357"},{"tid":360,"created_ino":"0x10000000358"},{"tid":361,"created_ino":"0x10000000359"},{"tid":362,"created_ino":"0x1000000035a"},{"tid":363,"created_ino":"0x1000000035b"},{"tid":364,"created_ino":"0x1000000035c"},{"tid":365,"created_ino":"0x1000000035d"},{"tid":366,"created_ino":"0x1000000035e"},{"tid":367,"created_ino":"0x1000000035f"},{"tid":368,"created_ino":"0x10000000360"},{"tid":369,"created_ino":"0x10000000361"},{"tid":370,"created_ino":"0x10000000362"},{"tid":371,"created_ino":"0x10000000363"},{"tid":372,"created_ino":"0x10000000364"},{"tid":373,"created_ino":"0x10000000365"},{"tid":374,"created_ino":"0x10000000366"},{"tid":375,"created_ino":"0x10000000367"},{"tid":376,"created_ino":"0x10000000368"},{"tid":377,"created_ino":"0x10000000369"},{"tid":378,"created_ino":"0x1000000036a"},{"tid":379,"created_ino":"0x1000000036b"},{"tid":380,"created_ino":"0x1000000036c"},{"tid":381,"created_ino":"0x1000000036d"},{"tid":382,"created_ino":"0x1000000036e"},{"tid":383,"created_ino":"0x1000000036f"},{"tid":384,"created_ino":"0x10000000370"},{"tid":385,"created_ino":"0x10000000371"},{"tid":386,"created_ino":"0x10000000372"},{"tid":387,"created_ino":"0x10000000373"},{"tid":388,"created_ino":"0x10000000374"},{"tid":389,"created_ino":"0x10000000375"},{"tid":390,"created_ino":"0x10000000376"},{"tid":391,"created_ino":"0x10000000377"},{"tid":392,"created_ino":"0x10000000378"},{"tid":393,"created_ino":"0x10000000379"},{"tid":394,"created_ino":"0x1000000037a"},{"tid":395,"created_ino":"0x1000000037b"},{"tid":396,"created_ino":"0x1000000037c"},{"tid":397,"created_ino":"0x1000000037d"},{"tid":398,"created_ino":"0x1000000037e"},{"tid":399,"created_ino":"0x1000000037f"},{"tid":400,"created_ino":"0x10000000380"},{"tid":402,"created_ino":"0x10000000381"},{"tid":403,"created_ino":"0x10000000382"},{"tid":404,"created_ino":"0x10000000383"},{"tid":405,"created_ino":"0x10000000384"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000385","length":602}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"0","hostname":"smithi016","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"61430","root":"/"}}] 2024-09-19T07:50:36.029 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "46954", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4223664118, "type": "any" }, "name": { "num": 6226, "type": "client" } }, "id": 6226, "inst": "client.6226 192.168.0.1:0/4223664118", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9533924170089287 }, "state": "open", "uptime": 8.516881629 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "0", "hostname": "smithi016", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "61430", "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000214", "tid": 36 }, { "created_ino": "0x10000000215", "tid": 37 }, { "created_ino": "0x10000000216", "tid": 38 }, { "created_ino": "0x10000000217", "tid": 39 }, { "created_ino": "0x10000000218", "tid": 40 }, { "created_ino": "0x10000000219", "tid": 41 }, { "created_ino": "0x1000000021a", "tid": 42 }, { "created_ino": "0x1000000021b", "tid": 43 }, { "created_ino": "0x1000000021c", "tid": 44 }, { "created_ino": "0x1000000021d", "tid": 45 }, { "created_ino": "0x1000000021e", "tid": 46 }, { "created_ino": "0x1000000021f", "tid": 47 }, { "created_ino": "0x10000000220", "tid": 48 }, { "created_ino": "0x10000000221", "tid": 49 }, { "created_ino": "0x10000000222", "tid": 50 }, { "created_ino": "0x10000000223", "tid": 51 }, { "created_ino": "0x10000000224", "tid": 52 }, { "created_ino": "0x10000000225", "tid": 53 }, { "created_ino": "0x10000000226", "tid": 54 }, { "created_ino": "0x10000000227", "tid": 55 }, { "created_ino": "0x10000000228", "tid": 56 }, { "created_ino": "0x10000000229", "tid": 57 }, { "created_ino": "0x1000000022a", "tid": 58 }, { "created_ino": "0x1000000022b", "tid": 59 }, { "created_ino": "0x1000000022c", "tid": 60 }, { "created_ino": "0x1000000022d", "tid": 61 }, { "created_ino": "0x1000000022e", "tid": 62 }, { "created_ino": "0x1000000022f", "tid": 63 }, { "created_ino": "0x10000000230", "tid": 64 }, { "created_ino": "0x10000000231", "tid": 65 }, { "created_ino": "0x10000000232", "tid": 66 }, { "created_ino": "0x10000000233", "tid": 67 }, { "created_ino": "0x10000000234", "tid": 68 }, { "created_ino": "0x10000000235", "tid": 69 }, { "created_ino": "0x10000000236", "tid": 70 }, { "created_ino": "0x10000000237", "tid": 71 }, { "created_ino": "0x10000000238", "tid": 72 }, { "created_ino": "0x10000000239", "tid": 73 }, { "created_ino": "0x1000000023a", "tid": 74 }, { "created_ino": "0x1000000023b", "tid": 75 }, { "created_ino": "0x1000000023c", "tid": 76 }, { "created_ino": "0x1000000023d", "tid": 77 }, { "created_ino": "0x1000000023e", "tid": 78 }, { "created_ino": "0x1000000023f", "tid": 79 }, { "created_ino": "0x10000000240", "tid": 80 }, { "created_ino": "0x10000000241", "tid": 81 }, { "created_ino": "0x10000000242", "tid": 82 }, { "created_ino": "0x10000000243", "tid": 83 }, { "created_ino": "0x10000000244", "tid": 84 }, { "created_ino": "0x10000000245", "tid": 85 }, { "created_ino": "0x10000000246", "tid": 86 }, { "created_ino": "0x10000000247", "tid": 87 }, { "created_ino": "0x10000000248", "tid": 88 }, { "created_ino": "0x10000000249", "tid": 89 }, { "created_ino": "0x1000000024a", "tid": 90 }, { "created_ino": "0x1000000024b", "tid": 91 }, { "created_ino": "0x1000000024c", "tid": 92 }, { "created_ino": "0x1000000024d", "tid": 93 }, { "created_ino": "0x1000000024e", "tid": 94 }, { "created_ino": "0x1000000024f", "tid": 95 }, { "created_ino": "0x10000000250", "tid": 96 }, { "created_ino": "0x10000000251", "tid": 97 }, { "created_ino": "0x10000000252", "tid": 98 }, { "created_ino": "0x10000000253", "tid": 99 }, { "created_ino": "0x10000000254", "tid": 100 }, { "created_ino": "0x10000000255", "tid": 101 }, { "created_ino": "0x10000000256", "tid": 102 }, { "created_ino": "0x10000000257", "tid": 103 }, { "created_ino": "0x10000000258", "tid": 104 }, { "created_ino": "0x10000000259", "tid": 105 }, { "created_ino": "0x1000000025a", "tid": 106 }, { "created_ino": "0x1000000025b", "tid": 107 }, { "created_ino": "0x1000000025c", "tid": 108 }, { "created_ino": "0x1000000025d", "tid": 109 }, { "created_ino": "0x1000000025e", "tid": 110 }, { "created_ino": "0x1000000025f", "tid": 111 }, { "created_ino": "0x10000000260", "tid": 112 }, { "created_ino": "0x10000000261", "tid": 113 }, { "created_ino": "0x10000000262", "tid": 114 }, { "created_ino": "0x10000000263", "tid": 115 }, { "created_ino": "0x10000000264", "tid": 116 }, { "created_ino": "0x10000000265", "tid": 117 }, { "created_ino": "0x10000000266", "tid": 118 }, { "created_ino": "0x10000000267", "tid": 119 }, { "created_ino": "0x10000000268", "tid": 120 }, { "created_ino": "0x10000000269", "tid": 121 }, { "created_ino": "0x1000000026a", "tid": 122 }, { "created_ino": "0x1000000026b", "tid": 123 }, { "created_ino": "0x1000000026c", "tid": 124 }, { "created_ino": "0x1000000026d", "tid": 125 }, { "created_ino": "0x1000000026e", "tid": 126 }, { "created_ino": "0x1000000026f", "tid": 127 }, { "created_ino": "0x10000000270", "tid": 128 }, { "created_ino": "0x10000000271", "tid": 129 }, { "created_ino": "0x10000000272", "tid": 130 }, { "created_ino": "0x10000000273", "tid": 131 }, { "created_ino": "0x10000000274", "tid": 132 }, { "created_ino": "0x10000000275", "tid": 133 }, { "created_ino": "0x10000000276", "tid": 134 }, { "created_ino": "0x10000000277", "tid": 135 }, { "created_ino": "0x10000000278", "tid": 136 }, { "created_ino": "0x10000000279", "tid": 137 }, { "created_ino": "0x1000000027a", "tid": 138 }, { "created_ino": "0x1000000027b", "tid": 139 }, { "created_ino": "0x1000000027c", "tid": 140 }, { "created_ino": "0x1000000027d", "tid": 141 }, { "created_ino": "0x1000000027e", "tid": 142 }, { "created_ino": "0x1000000027f", "tid": 143 }, { "created_ino": "0x10000000280", "tid": 144 }, { "created_ino": "0x10000000281", "tid": 145 }, { "created_ino": "0x10000000282", "tid": 146 }, { "created_ino": "0x10000000283", "tid": 147 }, { "created_ino": "0x10000000284", "tid": 148 }, { "created_ino": "0x10000000285", "tid": 149 }, { "created_ino": "0x10000000286", "tid": 150 }, { "created_ino": "0x10000000287", "tid": 151 }, { "created_ino": "0x10000000288", "tid": 152 }, { "created_ino": "0x10000000289", "tid": 153 }, { "created_ino": "0x1000000028a", "tid": 154 }, { "created_ino": "0x1000000028b", "tid": 155 }, { "created_ino": "0x1000000028c", "tid": 156 }, { "created_ino": "0x1000000028d", "tid": 157 }, { "created_ino": "0x1000000028e", "tid": 158 }, { "created_ino": "0x1000000028f", "tid": 159 }, { "created_ino": "0x10000000290", "tid": 160 }, { "created_ino": "0x10000000291", "tid": 161 }, { "created_ino": "0x10000000292", "tid": 162 }, { "created_ino": "0x10000000293", "tid": 163 }, { "created_ino": "0x10000000294", "tid": 164 }, { "created_ino": "0x10000000295", "tid": 165 }, { "created_ino": "0x10000000296", "tid": 166 }, { "created_ino": "0x10000000297", "tid": 167 }, { "created_ino": "0x10000000298", "tid": 168 }, { "created_ino": "0x10000000299", "tid": 169 }, { "created_ino": "0x1000000029a", "tid": 170 }, { "created_ino": "0x1000000029b", "tid": 171 }, { "created_ino": "0x1000000029c", "tid": 172 }, { "created_ino": "0x1000000029d", "tid": 173 }, { "created_ino": "0x1000000029e", "tid": 174 }, { "created_ino": "0x1000000029f", "tid": 175 }, { "created_ino": "0x100000002a0", "tid": 176 }, { "created_ino": "0x100000002a1", "tid": 177 }, { "created_ino": "0x100000002a2", "tid": 178 }, { "created_ino": "0x100000002a3", "tid": 179 }, { "created_ino": "0x100000002a4", "tid": 180 }, { "created_ino": "0x100000002a5", "tid": 181 }, { "created_ino": "0x100000002a6", "tid": 182 }, { "created_ino": "0x100000002a7", "tid": 183 }, { "created_ino": "0x100000002a8", "tid": 184 }, { "created_ino": "0x100000002a9", "tid": 185 }, { "created_ino": "0x100000002aa", "tid": 186 }, { "created_ino": "0x100000002ab", "tid": 187 }, { "created_ino": "0x100000002ac", "tid": 188 }, { "created_ino": "0x100000002ad", "tid": 189 }, { "created_ino": "0x100000002ae", "tid": 190 }, { "created_ino": "0x100000002af", "tid": 191 }, { "created_ino": "0x100000002b0", "tid": 192 }, { "created_ino": "0x100000002b1", "tid": 193 }, { "created_ino": "0x100000002b2", "tid": 194 }, { "created_ino": "0x100000002b3", "tid": 195 }, { "created_ino": "0x100000002b4", "tid": 196 }, { "created_ino": "0x100000002b5", "tid": 197 }, { "created_ino": "0x100000002b6", "tid": 198 }, { "created_ino": "0x100000002b7", "tid": 199 }, { "created_ino": "0x100000002b8", "tid": 200 }, { "created_ino": "0x100000002b9", "tid": 201 }, { "created_ino": "0x100000002ba", "tid": 202 }, { "created_ino": "0x100000002bb", "tid": 203 }, { "created_ino": "0x100000002bc", "tid": 204 }, { "created_ino": "0x100000002bd", "tid": 205 }, { "created_ino": "0x100000002be", "tid": 206 }, { "created_ino": "0x100000002bf", "tid": 207 }, { "created_ino": "0x100000002c0", "tid": 208 }, { "created_ino": "0x100000002c1", "tid": 209 }, { "created_ino": "0x100000002c2", "tid": 210 }, { "created_ino": "0x100000002c3", "tid": 211 }, { "created_ino": "0x100000002c4", "tid": 212 }, { "created_ino": "0x100000002c5", "tid": 213 }, { "created_ino": "0x100000002c6", "tid": 214 }, { "created_ino": "0x100000002c7", "tid": 215 }, { "created_ino": "0x100000002c8", "tid": 216 }, { "created_ino": "0x100000002c9", "tid": 217 }, { "created_ino": "0x100000002ca", "tid": 218 }, { "created_ino": "0x100000002cb", "tid": 219 }, { "created_ino": "0x100000002cc", "tid": 220 }, { "created_ino": "0x100000002cd", "tid": 221 }, { "created_ino": "0x100000002ce", "tid": 222 }, { "created_ino": "0x100000002cf", "tid": 223 }, { "created_ino": "0x100000002d0", "tid": 224 }, { "created_ino": "0x100000002d1", "tid": 225 }, { "created_ino": "0x100000002d2", "tid": 226 }, { "created_ino": "0x100000002d3", "tid": 227 }, { "created_ino": "0x100000002d4", "tid": 228 }, { "created_ino": "0x100000002d5", "tid": 229 }, { "created_ino": "0x100000002d6", "tid": 230 }, { "created_ino": "0x100000002d7", "tid": 231 }, { "created_ino": "0x100000002d8", "tid": 232 }, { "created_ino": "0x100000002d9", "tid": 233 }, { "created_ino": "0x100000002da", "tid": 234 }, { "created_ino": "0x100000002db", "tid": 235 }, { "created_ino": "0x100000002dc", "tid": 236 }, { "created_ino": "0x100000002dd", "tid": 237 }, { "created_ino": "0x100000002de", "tid": 238 }, { "created_ino": "0x100000002df", "tid": 239 }, { "created_ino": "0x100000002e0", "tid": 240 }, { "created_ino": "0x100000002e1", "tid": 241 }, { "created_ino": "0x100000002e2", "tid": 242 }, { "created_ino": "0x100000002e3", "tid": 243 }, { "created_ino": "0x100000002e4", "tid": 244 }, { "created_ino": "0x100000002e5", "tid": 245 }, { "created_ino": "0x100000002e6", "tid": 246 }, { "created_ino": "0x100000002e7", "tid": 247 }, { "created_ino": "0x100000002e8", "tid": 248 }, { "created_ino": "0x100000002e9", "tid": 249 }, { "created_ino": "0x100000002ea", "tid": 250 }, { "created_ino": "0x100000002eb", "tid": 251 }, { "created_ino": "0x100000002ec", "tid": 252 }, { "created_ino": "0x100000002ed", "tid": 253 }, { "created_ino": "0x100000002ee", "tid": 254 }, { "created_ino": "0x100000002ef", "tid": 255 }, { "created_ino": "0x100000002f0", "tid": 256 }, { "created_ino": "0x100000002f1", "tid": 257 }, { "created_ino": "0x100000002f2", "tid": 258 }, { "created_ino": "0x100000002f3", "tid": 259 }, { "created_ino": "0x100000002f4", "tid": 260 }, { "created_ino": "0x100000002f5", "tid": 261 }, { "created_ino": "0x100000002f6", "tid": 262 }, { "created_ino": "0x100000002f7", "tid": 263 }, { "created_ino": "0x100000002f8", "tid": 264 }, { "created_ino": "0x100000002f9", "tid": 265 }, { "created_ino": "0x100000002fa", "tid": 266 }, { "created_ino": "0x100000002fb", "tid": 267 }, { "created_ino": "0x100000002fc", "tid": 268 }, { "created_ino": "0x100000002fd", "tid": 269 }, { "created_ino": "0x100000002fe", "tid": 270 }, { "created_ino": "0x100000002ff", "tid": 271 }, { "created_ino": "0x10000000300", "tid": 272 }, { "created_ino": "0x10000000301", "tid": 273 }, { "created_ino": "0x10000000302", "tid": 274 }, { "created_ino": "0x10000000303", "tid": 275 }, { "created_ino": "0x10000000304", "tid": 276 }, { "created_ino": "0x10000000305", "tid": 277 }, { "created_ino": "0x10000000306", "tid": 278 }, { "created_ino": "0x10000000307", "tid": 279 }, { "created_ino": "0x10000000308", "tid": 280 }, { "created_ino": "0x10000000309", "tid": 281 }, { "created_ino": "0x1000000030a", "tid": 282 }, { "created_ino": "0x1000000030b", "tid": 283 }, { "created_ino": "0x1000000030c", "tid": 284 }, { "created_ino": "0x1000000030d", "tid": 285 }, { "created_ino": "0x1000000030e", "tid": 286 }, { "created_ino": "0x1000000030f", "tid": 287 }, { "created_ino": "0x10000000310", "tid": 288 }, { "created_ino": "0x10000000311", "tid": 289 }, { "created_ino": "0x10000000312", "tid": 290 }, { "created_ino": "0x10000000313", "tid": 291 }, { "created_ino": "0x10000000314", "tid": 292 }, { "created_ino": "0x10000000315", "tid": 293 }, { "created_ino": "0x10000000316", "tid": 294 }, { "created_ino": "0x10000000317", "tid": 295 }, { "created_ino": "0x10000000318", "tid": 296 }, { "created_ino": "0x10000000319", "tid": 297 }, { "created_ino": "0x1000000031a", "tid": 298 }, { "created_ino": "0x1000000031b", "tid": 299 }, { "created_ino": "0x1000000031c", "tid": 300 }, { "created_ino": "0x1000000031d", "tid": 301 }, { "created_ino": "0x1000000031e", "tid": 302 }, { "created_ino": "0x1000000031f", "tid": 303 }, { "created_ino": "0x10000000320", "tid": 304 }, { "created_ino": "0x10000000321", "tid": 305 }, { "created_ino": "0x10000000322", "tid": 306 }, { "created_ino": "0x10000000323", "tid": 307 }, { "created_ino": "0x10000000324", "tid": 308 }, { "created_ino": "0x10000000325", "tid": 309 }, { "created_ino": "0x10000000326", "tid": 310 }, { "created_ino": "0x10000000327", "tid": 311 }, { "created_ino": "0x10000000328", "tid": 312 }, { "created_ino": "0x10000000329", "tid": 313 }, { "created_ino": "0x1000000032a", "tid": 314 }, { "created_ino": "0x1000000032b", "tid": 315 }, { "created_ino": "0x1000000032c", "tid": 316 }, { "created_ino": "0x1000000032d", "tid": 317 }, { "created_ino": "0x1000000032e", "tid": 318 }, { "created_ino": "0x1000000032f", "tid": 319 }, { "created_ino": "0x10000000330", "tid": 320 }, { "created_ino": "0x10000000331", "tid": 321 }, { "created_ino": "0x10000000332", "tid": 322 }, { "created_ino": "0x10000000333", "tid": 323 }, { "created_ino": "0x10000000334", "tid": 324 }, { "created_ino": "0x10000000335", "tid": 325 }, { "created_ino": "0x10000000336", "tid": 326 }, { "created_ino": "0x10000000337", "tid": 327 }, { "created_ino": "0x10000000338", "tid": 328 }, { "created_ino": "0x10000000339", "tid": 329 }, { "created_ino": "0x1000000033a", "tid": 330 }, { "created_ino": "0x1000000033b", "tid": 331 }, { "created_ino": "0x1000000033c", "tid": 332 }, { "created_ino": "0x1000000033d", "tid": 333 }, { "created_ino": "0x1000000033e", "tid": 334 }, { "created_ino": "0x1000000033f", "tid": 335 }, { "created_ino": "0x10000000340", "tid": 336 }, { "created_ino": "0x10000000341", "tid": 337 }, { "created_ino": "0x10000000342", "tid": 338 }, { "created_ino": "0x10000000343", "tid": 339 }, { "created_ino": "0x10000000344", "tid": 340 }, { "created_ino": "0x10000000345", "tid": 341 }, { "created_ino": "0x10000000346", "tid": 342 }, { "created_ino": "0x10000000347", "tid": 343 }, { "created_ino": "0x10000000348", "tid": 344 }, { "created_ino": "0x10000000349", "tid": 345 }, { "created_ino": "0x1000000034a", "tid": 346 }, { "created_ino": "0x1000000034b", "tid": 347 }, { "created_ino": "0x1000000034c", "tid": 348 }, { "created_ino": "0x1000000034d", "tid": 349 }, { "created_ino": "0x1000000034e", "tid": 350 }, { "created_ino": "0x1000000034f", "tid": 351 }, { "created_ino": "0x10000000350", "tid": 352 }, { "created_ino": "0x10000000351", "tid": 353 }, { "created_ino": "0x10000000352", "tid": 354 }, { "created_ino": "0x10000000353", "tid": 355 }, { "created_ino": "0x10000000354", "tid": 356 }, { "created_ino": "0x10000000355", "tid": 357 }, { "created_ino": "0x10000000356", "tid": 358 }, { "created_ino": "0x10000000357", "tid": 359 }, { "created_ino": "0x10000000358", "tid": 360 }, { "created_ino": "0x10000000359", "tid": 361 }, { "created_ino": "0x1000000035a", "tid": 362 }, { "created_ino": "0x1000000035b", "tid": 363 }, { "created_ino": "0x1000000035c", "tid": 364 }, { "created_ino": "0x1000000035d", "tid": 365 }, { "created_ino": "0x1000000035e", "tid": 366 }, { "created_ino": "0x1000000035f", "tid": 367 }, { "created_ino": "0x10000000360", "tid": 368 }, { "created_ino": "0x10000000361", "tid": 369 }, { "created_ino": "0x10000000362", "tid": 370 }, { "created_ino": "0x10000000363", "tid": 371 }, { "created_ino": "0x10000000364", "tid": 372 }, { "created_ino": "0x10000000365", "tid": 373 }, { "created_ino": "0x10000000366", "tid": 374 }, { "created_ino": "0x10000000367", "tid": 375 }, { "created_ino": "0x10000000368", "tid": 376 }, { "created_ino": "0x10000000369", "tid": 377 }, { "created_ino": "0x1000000036a", "tid": 378 }, { "created_ino": "0x1000000036b", "tid": 379 }, { "created_ino": "0x1000000036c", "tid": 380 }, { "created_ino": "0x1000000036d", "tid": 381 }, { "created_ino": "0x1000000036e", "tid": 382 }, { "created_ino": "0x1000000036f", "tid": 383 }, { "created_ino": "0x10000000370", "tid": 384 }, { "created_ino": "0x10000000371", "tid": 385 }, { "created_ino": "0x10000000372", "tid": 386 }, { "created_ino": "0x10000000373", "tid": 387 }, { "created_ino": "0x10000000374", "tid": 388 }, { "created_ino": "0x10000000375", "tid": 389 }, { "created_ino": "0x10000000376", "tid": 390 }, { "created_ino": "0x10000000377", "tid": 391 }, { "created_ino": "0x10000000378", "tid": 392 }, { "created_ino": "0x10000000379", "tid": 393 }, { "created_ino": "0x1000000037a", "tid": 394 }, { "created_ino": "0x1000000037b", "tid": 395 }, { "created_ino": "0x1000000037c", "tid": 396 }, { "created_ino": "0x1000000037d", "tid": 397 }, { "created_ino": "0x1000000037e", "tid": 398 }, { "created_ino": "0x1000000037f", "tid": 399 }, { "created_ino": "0x10000000380", "tid": 400 }, { "created_ino": "0x10000000381", "tid": 402 }, { "created_ino": "0x10000000382", "tid": 403 }, { "created_ino": "0x10000000383", "tid": 404 }, { "created_ino": "0x10000000384", "tid": 405 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1620886677, "type": "any" }, "name": { "num": 6576, "type": "client" } }, "id": 6576, "inst": "client.6576 192.168.0.1:0/1620886677", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 36, "num_caps": 402, "num_completed_flushes": 0, "num_completed_requests": 369, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 602, "start": "0x10000000385" } ], "recall_caps": { "halflife": 60, "value": 1161.832648106281 }, "recall_caps_throttle": { "halflife": 1.5, "value": 438.25760867024445 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 172.6765465365375 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 379, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 795.6132835633912 }, "state": "open", "uptime": 10.063860129 } ] 2024-09-19T07:50:36.030 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-09-19T07:50:36.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:50:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.131+0000 7f03f64b7640 1 Processor -- start 2024-09-19T07:50:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.131+0000 7f03f64b7640 1 -- start start 2024-09-19T07:50:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.131+0000 7f03f64b7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f03f015f0d0 0x7f03f015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.131+0000 7f03f64b7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f03f00d0070 0x7f03f00d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.131+0000 7f03f64b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03f00d1940 0x7f03f00d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.131+0000 7f03f64b7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f03f00cf2b0 con 0x7f03f015f0d0 2024-09-19T07:50:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.131+0000 7f03f64b7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f03f00cf590 con 0x7f03f00d0070 2024-09-19T07:50:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.131+0000 7f03f64b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f03f00cf870 con 0x7f03f00d1940 2024-09-19T07:50:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.132+0000 7f03effff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f03f015f0d0 0x7f03f015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.132+0000 7f03effff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f03f015f0d0 0x7f03f015f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49600/0 (socket says 172.21.15.16:49600) 2024-09-19T07:50:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.132+0000 7f03effff640 1 -- 172.21.15.16:0/3364330013 learned_addr learned my addr 172.21.15.16:0/3364330013 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:36.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.132+0000 7f03f4a54640 1 --2- 172.21.15.16:0/3364330013 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03f00d1940 0x7f03f00d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.132+0000 7f03ef7fe640 1 --2- 172.21.15.16:0/3364330013 >> v2:172.21.15.146:3300/0 conn(0x7f03f00d0070 0x7f03f00d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.132+0000 7f03effff640 1 -- 172.21.15.16:0/3364330013 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03f00d1940 msgr2=0x7f03f00d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.132+0000 7f03effff640 1 --2- 172.21.15.16:0/3364330013 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03f00d1940 0x7f03f00d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.132+0000 7f03effff640 1 -- 172.21.15.16:0/3364330013 >> v2:172.21.15.146:3300/0 conn(0x7f03f00d0070 msgr2=0x7f03f00d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.132+0000 7f03effff640 1 --2- 172.21.15.16:0/3364330013 >> v2:172.21.15.146:3300/0 conn(0x7f03f00d0070 0x7f03f00d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.132+0000 7f03effff640 1 -- 172.21.15.16:0/3364330013 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f00cfb50 con 0x7f03f015f0d0 2024-09-19T07:50:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.132+0000 7f03f4a54640 1 --2- 172.21.15.16:0/3364330013 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03f00d1940 0x7f03f00d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.132+0000 7f03ef7fe640 1 --2- 172.21.15.16:0/3364330013 >> v2:172.21.15.146:3300/0 conn(0x7f03f00d0070 0x7f03f00d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.133+0000 7f03effff640 1 --2- 172.21.15.16:0/3364330013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f015f0d0 0x7f03f015f4b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f03f0162fc0 tx=0x7f03e002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386ef730989202b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.133+0000 7f03eeffd640 1 -- 172.21.15.16:0/3364330013 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0037070 con 0x7f03f015f0d0 2024-09-19T07:50:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.133+0000 7f03eeffd640 1 -- 172.21.15.16:0/3364330013 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03e0032440 con 0x7f03f015f0d0 2024-09-19T07:50:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.133+0000 7f03eeffd640 1 -- 172.21.15.16:0/3364330013 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0032810 con 0x7f03f015f0d0 2024-09-19T07:50:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.133+0000 7f03f64b7640 1 -- 172.21.15.16:0/3364330013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f015f0d0 msgr2=0x7f03f015f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:36.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.133+0000 7f03f64b7640 1 --2- 172.21.15.16:0/3364330013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f015f0d0 0x7f03f015f4b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f03f0162fc0 tx=0x7f03e002a140 comp rx=0 tx=0).stop 2024-09-19T07:50:36.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.133+0000 7f03f64b7640 1 -- 172.21.15.16:0/3364330013 shutdown_connections 2024-09-19T07:50:36.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.133+0000 7f03f64b7640 1 --2- 172.21.15.16:0/3364330013 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03f00d1940 0x7f03f00d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.133+0000 7f03f64b7640 1 --2- 172.21.15.16:0/3364330013 >> v2:172.21.15.146:3300/0 conn(0x7f03f00d0070 0x7f03f00d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.133+0000 7f03f64b7640 1 --2- 172.21.15.16:0/3364330013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f015f0d0 0x7f03f015f4b0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.133+0000 7f03f64b7640 1 -- 172.21.15.16:0/3364330013 >> 172.21.15.16:0/3364330013 conn(0x7f03f009d760 msgr2=0x7f03f008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:36.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.133+0000 7f03f64b7640 1 -- 172.21.15.16:0/3364330013 shutdown_connections 2024-09-19T07:50:36.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.133+0000 7f03f64b7640 1 -- 172.21.15.16:0/3364330013 wait complete. 2024-09-19T07:50:36.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.134+0000 7f03f64b7640 1 Processor -- start 2024-09-19T07:50:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.135+0000 7f03f64b7640 1 -- start start 2024-09-19T07:50:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.135+0000 7f03f64b7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03f00d0070 0x7f03f00b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.135+0000 7f03f64b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03f00d1940 0x7f03f00d3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.135+0000 7f03f64b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f015f0d0 0x7f03f00d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.135+0000 7f03f64b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03f00d47f0 con 0x7f03f015f0d0 2024-09-19T07:50:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.135+0000 7f03f64b7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f03f00d4670 con 0x7f03f00d0070 2024-09-19T07:50:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.135+0000 7f03f64b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f03f00d4970 con 0x7f03f00d1940 2024-09-19T07:50:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.136+0000 7f03f4a54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f015f0d0 0x7f03f00d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.136+0000 7f03f4a54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f015f0d0 0x7f03f00d3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49606/0 (socket says 172.21.15.16:49606) 2024-09-19T07:50:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.136+0000 7f03f4a54640 1 -- 172.21.15.16:0/855649291 learned_addr learned my addr 172.21.15.16:0/855649291 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.136+0000 7f03effff640 1 --2- 172.21.15.16:0/855649291 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03f00d0070 0x7f03f00b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:36.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.136+0000 7f03ef7fe640 1 --2- 172.21.15.16:0/855649291 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03f00d1940 0x7f03f00d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.136+0000 7f03f4a54640 1 -- 172.21.15.16:0/855649291 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03f00d1940 msgr2=0x7f03f00d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.136+0000 7f03f4a54640 1 --2- 172.21.15.16:0/855649291 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03f00d1940 0x7f03f00d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.136+0000 7f03f4a54640 1 -- 172.21.15.16:0/855649291 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03f00d0070 msgr2=0x7f03f00b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.136+0000 7f03f4a54640 1 --2- 172.21.15.16:0/855649291 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03f00d0070 0x7f03f00b7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.136+0000 7f03f4a54640 1 -- 172.21.15.16:0/855649291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f0154ec0 con 0x7f03f015f0d0 2024-09-19T07:50:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.136+0000 7f03f4a54640 1 --2- 172.21.15.16:0/855649291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f015f0d0 0x7f03f00d3b70 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f03e800cea0 tx=0x7f03e800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.136+0000 7f03ecff9640 1 -- 172.21.15.16:0/855649291 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8003b90 con 0x7f03f015f0d0 2024-09-19T07:50:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.136+0000 7f03ecff9640 1 -- 172.21.15.16:0/855649291 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03e8003d50 con 0x7f03f015f0d0 2024-09-19T07:50:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.136+0000 7f03f64b7640 1 -- 172.21.15.16:0/855649291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f01467a0 con 0x7f03f015f0d0 2024-09-19T07:50:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.136+0000 7f03ecff9640 1 -- 172.21.15.16:0/855649291 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8004260 con 0x7f03f015f0d0 2024-09-19T07:50:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.136+0000 7f03f64b7640 1 -- 172.21.15.16:0/855649291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f0146a40 con 0x7f03f015f0d0 2024-09-19T07:50:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.137+0000 7f03f64b7640 1 -- 172.21.15.16:0/855649291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f00d1400 con 0x7f03f015f0d0 2024-09-19T07:50:36.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.138+0000 7f03ecff9640 1 -- 172.21.15.16:0/855649291 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f03e8004400 con 0x7f03f015f0d0 2024-09-19T07:50:36.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.138+0000 7f03ecff9640 1 --2- 172.21.15.16:0/855649291 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f03dc06c2e0 0x7f03dc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:36.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.138+0000 7f03ecff9640 1 -- 172.21.15.16:0/855649291 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f03e808bf70 con 0x7f03f015f0d0 2024-09-19T07:50:36.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.139+0000 7f03effff640 1 --2- 172.21.15.16:0/855649291 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f03dc06c2e0 0x7f03dc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:36.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.139+0000 7f03effff640 1 --2- 172.21.15.16:0/855649291 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f03dc06c2e0 0x7f03dc06e7a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f03e0034000 tx=0x7f03e002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:36.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.143+0000 7f03ecff9640 1 -- 172.21.15.16:0/855649291 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f03f0154ec0 con 0x7f03f015f0d0 2024-09-19T07:50:36.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.356+0000 7f03f64b7640 1 -- 172.21.15.16:0/855649291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f03f006e0b0 con 0x7f03f015f0d0 2024-09-19T07:50:36.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.357+0000 7f03ecff9640 1 -- 172.21.15.16:0/855649291 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f03e8059e70 con 0x7f03f015f0d0 2024-09-19T07:50:36.358 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:36.358 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:50:36.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.359+0000 7f03f64b7640 1 -- 172.21.15.16:0/855649291 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f03dc06c2e0 msgr2=0x7f03dc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.359+0000 7f03f64b7640 1 --2- 172.21.15.16:0/855649291 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f03dc06c2e0 0x7f03dc06e7a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f03e0034000 tx=0x7f03e002e000 comp rx=0 tx=0).stop 2024-09-19T07:50:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.359+0000 7f03f64b7640 1 -- 172.21.15.16:0/855649291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f015f0d0 msgr2=0x7f03f00d3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.359+0000 7f03f64b7640 1 --2- 172.21.15.16:0/855649291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f015f0d0 0x7f03f00d3b70 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f03e800cea0 tx=0x7f03e800cf90 comp rx=0 tx=0).stop 2024-09-19T07:50:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.359+0000 7f03f64b7640 1 -- 172.21.15.16:0/855649291 shutdown_connections 2024-09-19T07:50:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.359+0000 7f03f64b7640 1 --2- 172.21.15.16:0/855649291 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f03dc06c2e0 0x7f03dc06e7a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.359+0000 7f03f64b7640 1 --2- 172.21.15.16:0/855649291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f015f0d0 0x7f03f00d3b70 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.359+0000 7f03f64b7640 1 --2- 172.21.15.16:0/855649291 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03f00d1940 0x7f03f00d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.359+0000 7f03f64b7640 1 --2- 172.21.15.16:0/855649291 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03f00d0070 0x7f03f00b7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.359+0000 7f03f64b7640 1 -- 172.21.15.16:0/855649291 >> 172.21.15.16:0/855649291 conn(0x7f03f009d760 msgr2=0x7f03f015c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.359+0000 7f03f64b7640 1 -- 172.21.15.16:0/855649291 shutdown_connections 2024-09-19T07:50:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.359+0000 7f03f64b7640 1 -- 172.21.15.16:0/855649291 wait complete. 2024-09-19T07:50:36.370 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-19T07:50:36.370 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-09-19T07:50:36.370 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:50:36.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.471+0000 7fc3eecba640 1 Processor -- start 2024-09-19T07:50:36.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.471+0000 7fc3eecba640 1 -- start start 2024-09-19T07:50:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.472+0000 7fc3eecba640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc3e8161eb0 0x7fc3e81622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.472+0000 7fc3eecba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3e81425e0 0x7fc3e8142a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.472+0000 7fc3eecba640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3e8162b70 con 0x7fc3e806cd20 2024-09-19T07:50:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.472+0000 7fc3eecba640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc3e8162e50 con 0x7fc3e81425e0 2024-09-19T07:50:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.472+0000 7fc3eecba640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc3e8142150 con 0x7fc3e8161eb0 2024-09-19T07:50:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.472+0000 7fc3eca56640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc3e806cd20 0x7fc3e806d100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52424/0 (socket says 172.21.15.16:52424) 2024-09-19T07:50:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.472+0000 7fc3eca56640 1 -- 172.21.15.16:0/148154081 learned_addr learned my addr 172.21.15.16:0/148154081 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.472+0000 7fc3ed257640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3e81425e0 0x7fc3e8142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.473+0000 7fc3e77fe640 1 -- 172.21.15.16:0/148154081 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1663504382 0 0) 0x7fc3e8162b70 con 0x7fc3e806cd20 2024-09-19T07:50:36.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.473+0000 7fc3e77fe640 1 -- 172.21.15.16:0/148154081 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3d0003520 con 0x7fc3e806cd20 2024-09-19T07:50:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.473+0000 7fc3e7fff640 1 --2- 172.21.15.16:0/148154081 >> v2:172.21.15.146:3300/0 conn(0x7fc3e8161eb0 0x7fc3e81622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.473+0000 7fc3ed257640 1 -- 172.21.15.16:0/148154081 >> v2:172.21.15.146:3300/0 conn(0x7fc3e8161eb0 msgr2=0x7fc3e81622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.473+0000 7fc3ed257640 1 --2- 172.21.15.16:0/148154081 >> v2:172.21.15.146:3300/0 conn(0x7fc3e8161eb0 0x7fc3e81622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.473+0000 7fc3ed257640 1 -- 172.21.15.16:0/148154081 >> v1:172.21.15.16:6789/0 conn(0x7fc3e806cd20 legacy=0x7fc3e806d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.473+0000 7fc3ed257640 1 -- 172.21.15.16:0/148154081 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e8141b00 con 0x7fc3e81425e0 2024-09-19T07:50:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.473+0000 7fc3e7fff640 1 --2- 172.21.15.16:0/148154081 >> v2:172.21.15.146:3300/0 conn(0x7fc3e8161eb0 0x7fc3e81622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.473+0000 7fc3ed257640 1 --2- 172.21.15.16:0/148154081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81425e0 0x7fc3e8142a70 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc3e806ef00 tx=0x7fc3e002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f794b48ca97d8b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.474+0000 7fc3e77fe640 1 -- 172.21.15.16:0/148154081 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0007740 con 0x7fc3e81425e0 2024-09-19T07:50:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.474+0000 7fc3e77fe640 1 -- 172.21.15.16:0/148154081 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3e00078e0 con 0x7fc3e81425e0 2024-09-19T07:50:36.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.474+0000 7fc3e77fe640 1 -- 172.21.15.16:0/148154081 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0007c90 con 0x7fc3e81425e0 2024-09-19T07:50:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.474+0000 7fc3eecba640 1 -- 172.21.15.16:0/148154081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81425e0 msgr2=0x7fc3e8142a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.474+0000 7fc3eecba640 1 --2- 172.21.15.16:0/148154081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81425e0 0x7fc3e8142a70 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc3e806ef00 tx=0x7fc3e002a5f0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.474+0000 7fc3eecba640 1 -- 172.21.15.16:0/148154081 shutdown_connections 2024-09-19T07:50:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.474+0000 7fc3eecba640 1 --2- 172.21.15.16:0/148154081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81425e0 0x7fc3e8142a70 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.474+0000 7fc3eecba640 1 --2- 172.21.15.16:0/148154081 >> v2:172.21.15.146:3300/0 conn(0x7fc3e8161eb0 0x7fc3e81622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.474+0000 7fc3eecba640 1 -- 172.21.15.16:0/148154081 >> 172.21.15.16:0/148154081 conn(0x7fc3e80b3ed0 msgr2=0x7fc3e80b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.474+0000 7fc3eecba640 1 -- 172.21.15.16:0/148154081 shutdown_connections 2024-09-19T07:50:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.474+0000 7fc3eecba640 1 -- 172.21.15.16:0/148154081 wait complete. 2024-09-19T07:50:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.475+0000 7fc3eecba640 1 Processor -- start 2024-09-19T07:50:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.475+0000 7fc3eecba640 1 -- start start 2024-09-19T07:50:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.476+0000 7fc3eecba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3e806cd20 0x7fc3e81f3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.476+0000 7fc3eecba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81425e0 0x7fc3e81559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.476+0000 7fc3eecba640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3e8161eb0 0x7fc3e8155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.476+0000 7fc3eecba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8141600 con 0x7fc3e81425e0 2024-09-19T07:50:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.476+0000 7fc3eecba640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8162830 con 0x7fc3e8161eb0 2024-09-19T07:50:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.476+0000 7fc3eecba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e8142fb0 con 0x7fc3e806cd20 2024-09-19T07:50:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.476+0000 7fc3e7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81425e0 0x7fc3e81559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.477+0000 7fc3e7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81425e0 0x7fc3e81559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49610/0 (socket says 172.21.15.16:49610) 2024-09-19T07:50:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.477+0000 7fc3e7fff640 1 -- 172.21.15.16:0/757865729 learned_addr learned my addr 172.21.15.16:0/757865729 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.477+0000 7fc3eca56640 1 --2- 172.21.15.16:0/757865729 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3e806cd20 0x7fc3e81f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.477+0000 7fc3ed257640 1 --2- 172.21.15.16:0/757865729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3e8161eb0 0x7fc3e8155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.477+0000 7fc3e7fff640 1 -- 172.21.15.16:0/757865729 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3e806cd20 msgr2=0x7fc3e81f3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.477+0000 7fc3e7fff640 1 --2- 172.21.15.16:0/757865729 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3e806cd20 0x7fc3e81f3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.477+0000 7fc3e7fff640 1 -- 172.21.15.16:0/757865729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3e8161eb0 msgr2=0x7fc3e8155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.477+0000 7fc3e7fff640 1 --2- 172.21.15.16:0/757865729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3e8161eb0 0x7fc3e8155f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.477+0000 7fc3e7fff640 1 -- 172.21.15.16:0/757865729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e80b54d0 con 0x7fc3e81425e0 2024-09-19T07:50:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.477+0000 7fc3e7fff640 1 --2- 172.21.15.16:0/757865729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81425e0 0x7fc3e81559e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00c820 tx=0x7fc3dc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.477+0000 7fc3e57fa640 1 -- 172.21.15.16:0/757865729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc015070 con 0x7fc3e81425e0 2024-09-19T07:50:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.477+0000 7fc3e57fa640 1 -- 172.21.15.16:0/757865729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3dc007a30 con 0x7fc3e81425e0 2024-09-19T07:50:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.477+0000 7fc3eecba640 1 -- 172.21.15.16:0/757865729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e8156550 con 0x7fc3e81425e0 2024-09-19T07:50:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.477+0000 7fc3e57fa640 1 -- 172.21.15.16:0/757865729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc007de0 con 0x7fc3e81425e0 2024-09-19T07:50:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.477+0000 7fc3eecba640 1 -- 172.21.15.16:0/757865729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e8156b10 con 0x7fc3e81425e0 2024-09-19T07:50:36.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.478+0000 7fc3eecba640 1 -- 172.21.15.16:0/757865729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e8162b70 con 0x7fc3e81425e0 2024-09-19T07:50:36.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.479+0000 7fc3e57fa640 1 -- 172.21.15.16:0/757865729 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc3dc018070 con 0x7fc3e81425e0 2024-09-19T07:50:36.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.479+0000 7fc3e57fa640 1 --2- 172.21.15.16:0/757865729 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc3d406c2b0 0x7fc3d406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:36.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.479+0000 7fc3e57fa640 1 -- 172.21.15.16:0/757865729 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fc3dc08b490 con 0x7fc3e81425e0 2024-09-19T07:50:36.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.479+0000 7fc3eca56640 1 --2- 172.21.15.16:0/757865729 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc3d406c2b0 0x7fc3d406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:36.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.480+0000 7fc3eca56640 1 --2- 172.21.15.16:0/757865729 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc3d406c2b0 0x7fc3d406e770 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8000f60 tx=0x7fc3d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:36.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.484+0000 7fc3e57fa640 1 -- 172.21.15.16:0/757865729 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc3dc059a70 con 0x7fc3e81425e0 2024-09-19T07:50:36.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.702+0000 7fc3eecba640 1 -- 172.21.15.16:0/757865729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc3e8072220 con 0x7fc3e81425e0 2024-09-19T07:50:36.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.702+0000 7fc3e57fa640 1 -- 172.21.15.16:0/757865729 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fc3dc059370 con 0x7fc3e81425e0 2024-09-19T07:50:36.703 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:36.703 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:50:36.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.705+0000 7fc3eecba640 1 -- 172.21.15.16:0/757865729 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc3d406c2b0 msgr2=0x7fc3d406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:36.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.705+0000 7fc3eecba640 1 --2- 172.21.15.16:0/757865729 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc3d406c2b0 0x7fc3d406e770 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8000f60 tx=0x7fc3d8009040 comp rx=0 tx=0).stop 2024-09-19T07:50:36.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.705+0000 7fc3eecba640 1 -- 172.21.15.16:0/757865729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81425e0 msgr2=0x7fc3e81559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:36.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.705+0000 7fc3eecba640 1 --2- 172.21.15.16:0/757865729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81425e0 0x7fc3e81559e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00c820 tx=0x7fc3dc00cce0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.705+0000 7fc3eecba640 1 -- 172.21.15.16:0/757865729 shutdown_connections 2024-09-19T07:50:36.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.705+0000 7fc3eecba640 1 --2- 172.21.15.16:0/757865729 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc3d406c2b0 0x7fc3d406e770 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.705+0000 7fc3eecba640 1 --2- 172.21.15.16:0/757865729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3e8161eb0 0x7fc3e8155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.705+0000 7fc3eecba640 1 --2- 172.21.15.16:0/757865729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81425e0 0x7fc3e81559e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.705+0000 7fc3eecba640 1 --2- 172.21.15.16:0/757865729 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3e806cd20 0x7fc3e81f3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:36.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.705+0000 7fc3eecba640 1 -- 172.21.15.16:0/757865729 >> 172.21.15.16:0/757865729 conn(0x7fc3e80b3ed0 msgr2=0x7fc3e80c7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:36.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.705+0000 7fc3eecba640 1 -- 172.21.15.16:0/757865729 shutdown_connections 2024-09-19T07:50:36.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:36.705+0000 7fc3eecba640 1 -- 172.21.15.16:0/757865729 wait complete. 2024-09-19T07:50:36.716 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:50:36.716 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:50:36.721 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:50:38.396 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:50:38.400 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:50:38.723 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:50:40.725 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:50:41.717 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:50:41.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.830+0000 7fa33085c640 1 Processor -- start 2024-09-19T07:50:41.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.830+0000 7fa33085c640 1 -- start start 2024-09-19T07:50:41.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.831+0000 7fa33085c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa32c15f0d0 0x7fa32c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:41.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.831+0000 7fa33085c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa32c0d0070 0x7fa32c0d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:41.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.831+0000 7fa33085c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa32c0d1940 0x7fa32c0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:41.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.831+0000 7fa33085c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa32c0cf2b0 con 0x7fa32c0d1940 2024-09-19T07:50:41.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.831+0000 7fa33085c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa32c0cf590 con 0x7fa32c15f0d0 2024-09-19T07:50:41.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.831+0000 7fa33085c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa32c0cf870 con 0x7fa32c0d0070 2024-09-19T07:50:41.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.832+0000 7fa32ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa32c0d1940 0x7fa32c0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:41.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.832+0000 7fa32ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa32c0d1940 0x7fa32c0d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49612/0 (socket says 172.21.15.16:49612) 2024-09-19T07:50:41.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.832+0000 7fa32ad9d640 1 -- 172.21.15.16:0/1680261447 learned_addr learned my addr 172.21.15.16:0/1680261447 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:41.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.832+0000 7fa329d9b640 1 --2- 172.21.15.16:0/1680261447 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa32c0d0070 0x7fa32c0d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:41.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.832+0000 7fa32a59c640 1 --2- 172.21.15.16:0/1680261447 >> v2:172.21.15.146:3300/0 conn(0x7fa32c15f0d0 0x7fa32c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:41.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.832+0000 7fa32ad9d640 1 -- 172.21.15.16:0/1680261447 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa32c0d0070 msgr2=0x7fa32c0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:41.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.832+0000 7fa32ad9d640 1 --2- 172.21.15.16:0/1680261447 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa32c0d0070 0x7fa32c0d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:41.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.832+0000 7fa32ad9d640 1 -- 172.21.15.16:0/1680261447 >> v2:172.21.15.146:3300/0 conn(0x7fa32c15f0d0 msgr2=0x7fa32c15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:41.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.832+0000 7fa32ad9d640 1 --2- 172.21.15.16:0/1680261447 >> v2:172.21.15.146:3300/0 conn(0x7fa32c15f0d0 0x7fa32c15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:41.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.832+0000 7fa32ad9d640 1 -- 172.21.15.16:0/1680261447 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32c0cfb50 con 0x7fa32c0d1940 2024-09-19T07:50:41.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.832+0000 7fa32a59c640 1 --2- 172.21.15.16:0/1680261447 >> v2:172.21.15.146:3300/0 conn(0x7fa32c15f0d0 0x7fa32c15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.833+0000 7fa32ad9d640 1 --2- 172.21.15.16:0/1680261447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa32c0d1940 0x7fa32c0d4130 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0b6f40 tx=0x7fa32402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbf90c42c77e89c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.833+0000 7fa32959a640 1 -- 172.21.15.16:0/1680261447 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324036070 con 0x7fa32c0d1940 2024-09-19T07:50:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.833+0000 7fa32959a640 1 -- 172.21.15.16:0/1680261447 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa324031600 con 0x7fa32c0d1940 2024-09-19T07:50:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.833+0000 7fa32959a640 1 -- 172.21.15.16:0/1680261447 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3240319d0 con 0x7fa32c0d1940 2024-09-19T07:50:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.833+0000 7fa33085c640 1 -- 172.21.15.16:0/1680261447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa32c0d1940 msgr2=0x7fa32c0d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.833+0000 7fa33085c640 1 --2- 172.21.15.16:0/1680261447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa32c0d1940 0x7fa32c0d4130 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0b6f40 tx=0x7fa32402a610 comp rx=0 tx=0).stop 2024-09-19T07:50:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.834+0000 7fa33085c640 1 -- 172.21.15.16:0/1680261447 shutdown_connections 2024-09-19T07:50:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.834+0000 7fa33085c640 1 --2- 172.21.15.16:0/1680261447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa32c0d1940 0x7fa32c0d4130 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.834+0000 7fa33085c640 1 --2- 172.21.15.16:0/1680261447 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa32c0d0070 0x7fa32c0d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.834+0000 7fa33085c640 1 --2- 172.21.15.16:0/1680261447 >> v2:172.21.15.146:3300/0 conn(0x7fa32c15f0d0 0x7fa32c15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.834+0000 7fa33085c640 1 -- 172.21.15.16:0/1680261447 >> 172.21.15.16:0/1680261447 conn(0x7fa32c09d760 msgr2=0x7fa32c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.834+0000 7fa33085c640 1 -- 172.21.15.16:0/1680261447 shutdown_connections 2024-09-19T07:50:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.834+0000 7fa33085c640 1 -- 172.21.15.16:0/1680261447 wait complete. 2024-09-19T07:50:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.835+0000 7fa33085c640 1 Processor -- start 2024-09-19T07:50:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.835+0000 7fa33085c640 1 -- start start 2024-09-19T07:50:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.836+0000 7fa33085c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa32c0d0070 0x7fa32c15b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.836+0000 7fa33085c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa32c0d1940 0x7fa32c0a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.836+0000 7fa33085c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa32c15f0d0 0x7fa32c0cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.836+0000 7fa33085c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa32c0d47f0 con 0x7fa32c15f0d0 2024-09-19T07:50:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.836+0000 7fa33085c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa32c0d4670 con 0x7fa32c0d0070 2024-09-19T07:50:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.836+0000 7fa33085c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa32c0d4970 con 0x7fa32c0d1940 2024-09-19T07:50:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.836+0000 7fa32ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa32c15f0d0 0x7fa32c0cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.836+0000 7fa32ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa32c15f0d0 0x7fa32c0cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49624/0 (socket says 172.21.15.16:49624) 2024-09-19T07:50:41.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.836+0000 7fa32ad9d640 1 -- 172.21.15.16:0/548655220 learned_addr learned my addr 172.21.15.16:0/548655220 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.836+0000 7fa32a59c640 1 --2- 172.21.15.16:0/548655220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa32c0d0070 0x7fa32c15b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.836+0000 7fa329d9b640 1 --2- 172.21.15.16:0/548655220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa32c0d1940 0x7fa32c0a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.836+0000 7fa32ad9d640 1 -- 172.21.15.16:0/548655220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa32c0d1940 msgr2=0x7fa32c0a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.836+0000 7fa32ad9d640 1 --2- 172.21.15.16:0/548655220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa32c0d1940 0x7fa32c0a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.836+0000 7fa32ad9d640 1 -- 172.21.15.16:0/548655220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa32c0d0070 msgr2=0x7fa32c15b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.836+0000 7fa32ad9d640 1 --2- 172.21.15.16:0/548655220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa32c0d0070 0x7fa32c15b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.836+0000 7fa32ad9d640 1 -- 172.21.15.16:0/548655220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32c09e270 con 0x7fa32c15f0d0 2024-09-19T07:50:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.837+0000 7fa32ad9d640 1 --2- 172.21.15.16:0/548655220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa32c15f0d0 0x7fa32c0cd930 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa324009f40 tx=0x7fa324004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.837+0000 7fa322ffd640 1 -- 172.21.15.16:0/548655220 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324036050 con 0x7fa32c15f0d0 2024-09-19T07:50:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.837+0000 7fa322ffd640 1 -- 172.21.15.16:0/548655220 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa324042070 con 0x7fa32c15f0d0 2024-09-19T07:50:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.837+0000 7fa33085c640 1 -- 172.21.15.16:0/548655220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa32c09e550 con 0x7fa32c15f0d0 2024-09-19T07:50:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.837+0000 7fa322ffd640 1 -- 172.21.15.16:0/548655220 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324009510 con 0x7fa32c15f0d0 2024-09-19T07:50:41.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.837+0000 7fa33085c640 1 -- 172.21.15.16:0/548655220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa32c09eb10 con 0x7fa32c15f0d0 2024-09-19T07:50:41.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.838+0000 7fa33085c640 1 -- 172.21.15.16:0/548655220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa32c0d1400 con 0x7fa32c15f0d0 2024-09-19T07:50:41.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.838+0000 7fa322ffd640 1 -- 172.21.15.16:0/548655220 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa324007bb0 con 0x7fa32c15f0d0 2024-09-19T07:50:41.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.839+0000 7fa322ffd640 1 --2- 172.21.15.16:0/548655220 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa30c06c2e0 0x7fa30c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.839+0000 7fa322ffd640 1 -- 172.21.15.16:0/548655220 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fa3240ac2b0 con 0x7fa32c15f0d0 2024-09-19T07:50:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.839+0000 7fa32a59c640 1 --2- 172.21.15.16:0/548655220 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa30c06c2e0 0x7fa30c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.840+0000 7fa32a59c640 1 --2- 172.21.15.16:0/548655220 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa30c06c2e0 0x7fa30c06e7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa32c15c050 tx=0x7fa31c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:41.843+0000 7fa322ffd640 1 -- 172.21.15.16:0/548655220 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa32407a240 con 0x7fa32c15f0d0 2024-09-19T07:50:42.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:42.057+0000 7fa33085c640 1 -- 172.21.15.16:0/548655220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa32c1624e0 con 0x7fa32c15f0d0 2024-09-19T07:50:42.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:42.057+0000 7fa322ffd640 1 -- 172.21.15.16:0/548655220 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fa32407e540 con 0x7fa32c15f0d0 2024-09-19T07:50:42.057 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:42.057 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:50:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:42.059+0000 7fa33085c640 1 -- 172.21.15.16:0/548655220 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa30c06c2e0 msgr2=0x7fa30c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:42.059+0000 7fa33085c640 1 --2- 172.21.15.16:0/548655220 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa30c06c2e0 0x7fa30c06e7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa32c15c050 tx=0x7fa31c008040 comp rx=0 tx=0).stop 2024-09-19T07:50:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:42.059+0000 7fa33085c640 1 -- 172.21.15.16:0/548655220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa32c15f0d0 msgr2=0x7fa32c0cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:42.059+0000 7fa33085c640 1 --2- 172.21.15.16:0/548655220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa32c15f0d0 0x7fa32c0cd930 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa324009f40 tx=0x7fa324004a40 comp rx=0 tx=0).stop 2024-09-19T07:50:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:42.059+0000 7fa33085c640 1 -- 172.21.15.16:0/548655220 shutdown_connections 2024-09-19T07:50:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:42.059+0000 7fa33085c640 1 --2- 172.21.15.16:0/548655220 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa30c06c2e0 0x7fa30c06e7a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:42.059+0000 7fa33085c640 1 --2- 172.21.15.16:0/548655220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa32c15f0d0 0x7fa32c0cd930 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:42.059+0000 7fa33085c640 1 --2- 172.21.15.16:0/548655220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa32c0d1940 0x7fa32c0a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:42.059+0000 7fa33085c640 1 --2- 172.21.15.16:0/548655220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa32c0d0070 0x7fa32c15b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:42.059+0000 7fa33085c640 1 -- 172.21.15.16:0/548655220 >> 172.21.15.16:0/548655220 conn(0x7fa32c09d760 msgr2=0x7fa32c15c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:42.059+0000 7fa33085c640 1 -- 172.21.15.16:0/548655220 shutdown_connections 2024-09-19T07:50:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:42.059+0000 7fa33085c640 1 -- 172.21.15.16:0/548655220 wait complete. 2024-09-19T07:50:42.071 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:50:42.071 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:50:42.727 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:50:44.729 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:50:46.731 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:50:47.071 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:50:47.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.187+0000 7f4d2f674640 1 Processor -- start 2024-09-19T07:50:47.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.187+0000 7f4d2f674640 1 -- start start 2024-09-19T07:50:47.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.188+0000 7f4d2f674640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d2806f9d0 0x7f4d280b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:47.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.188+0000 7f4d2f674640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d2815c520 con 0x7f4d280b6480 2024-09-19T07:50:47.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.188+0000 7f4d2f674640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d2815c7f0 con 0x7f4d2815c140 2024-09-19T07:50:47.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.188+0000 7f4d2f674640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f4d280b68b0 con 0x7f4d2806f9d0 2024-09-19T07:50:47.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.188+0000 7f4d2cc0f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d2806f9d0 0x7f4d280b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:47.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.188+0000 7f4d2dc11640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4d280b6480 0x7f4d2815d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35494/0 (socket says 172.21.15.16:35494) 2024-09-19T07:50:47.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.188+0000 7f4d2dc11640 1 -- 172.21.15.16:0/2712348256 learned_addr learned my addr 172.21.15.16:0/2712348256 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:47.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.189+0000 7f4d27fff640 1 -- 172.21.15.16:0/2712348256 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 751411033 0 0) 0x7f4d2815c520 con 0x7f4d280b6480 2024-09-19T07:50:47.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.189+0000 7f4d27fff640 1 -- 172.21.15.16:0/2712348256 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d10003520 con 0x7f4d280b6480 2024-09-19T07:50:47.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.189+0000 7f4d2cc0f640 1 -- 172.21.15.16:0/2712348256 >> v1:172.21.15.146:6789/0 conn(0x7f4d2815c140 legacy=0x7f4d28160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:47.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.189+0000 7f4d2cc0f640 1 -- 172.21.15.16:0/2712348256 >> v1:172.21.15.16:6789/0 conn(0x7f4d280b6480 legacy=0x7f4d2815d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:47.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.189+0000 7f4d2cc0f640 1 -- 172.21.15.16:0/2712348256 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2815cad0 con 0x7f4d2806f9d0 2024-09-19T07:50:47.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.189+0000 7f4d2cc0f640 1 --2- 172.21.15.16:0/2712348256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d2806f9d0 0x7f4d280b5f40 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0097a0 tx=0x7f4d1c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bcbf0aec51f38e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:47.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.190+0000 7f4d27fff640 1 -- 172.21.15.16:0/2712348256 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c037070 con 0x7f4d2806f9d0 2024-09-19T07:50:47.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.190+0000 7f4d27fff640 1 -- 172.21.15.16:0/2712348256 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d1c032440 con 0x7f4d2806f9d0 2024-09-19T07:50:47.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.190+0000 7f4d27fff640 1 -- 172.21.15.16:0/2712348256 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c0327f0 con 0x7f4d2806f9d0 2024-09-19T07:50:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.190+0000 7f4d2f674640 1 -- 172.21.15.16:0/2712348256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d2806f9d0 msgr2=0x7f4d280b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.190+0000 7f4d2f674640 1 --2- 172.21.15.16:0/2712348256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d2806f9d0 0x7f4d280b5f40 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0097a0 tx=0x7f4d1c02a230 comp rx=0 tx=0).stop 2024-09-19T07:50:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.190+0000 7f4d2f674640 1 -- 172.21.15.16:0/2712348256 shutdown_connections 2024-09-19T07:50:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.190+0000 7f4d2f674640 1 --2- 172.21.15.16:0/2712348256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d2806f9d0 0x7f4d280b5f40 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.190+0000 7f4d2f674640 1 -- 172.21.15.16:0/2712348256 >> 172.21.15.16:0/2712348256 conn(0x7f4d2806bc10 msgr2=0x7f4d2806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.190+0000 7f4d2f674640 1 -- 172.21.15.16:0/2712348256 shutdown_connections 2024-09-19T07:50:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.190+0000 7f4d2f674640 1 -- 172.21.15.16:0/2712348256 wait complete. 2024-09-19T07:50:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.191+0000 7f4d2f674640 1 Processor -- start 2024-09-19T07:50:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.191+0000 7f4d2f674640 1 -- start start 2024-09-19T07:50:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.192+0000 7f4d2f674640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d2806f9d0 0x7f4d281541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.192+0000 7f4d2f674640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d280b6480 0x7f4d2815f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.192+0000 7f4d2f674640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d2815c140 0x7f4d280b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.192+0000 7f4d2f674640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2815d370 con 0x7f4d280b6480 2024-09-19T07:50:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.192+0000 7f4d2f674640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2815d090 con 0x7f4d2806f9d0 2024-09-19T07:50:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.192+0000 7f4d2f674640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2815dfa0 con 0x7f4d2815c140 2024-09-19T07:50:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.192+0000 7f4d2cc0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d280b6480 0x7f4d2815f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.192+0000 7f4d2cc0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d280b6480 0x7f4d2815f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41402/0 (socket says 172.21.15.16:41402) 2024-09-19T07:50:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.192+0000 7f4d2cc0f640 1 -- 172.21.15.16:0/3319873641 learned_addr learned my addr 172.21.15.16:0/3319873641 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.192+0000 7f4d2d410640 1 --2- 172.21.15.16:0/3319873641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d2806f9d0 0x7f4d281541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.192+0000 7f4d2dc11640 1 --2- 172.21.15.16:0/3319873641 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d2815c140 0x7f4d280b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.193+0000 7f4d2cc0f640 1 -- 172.21.15.16:0/3319873641 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d2815c140 msgr2=0x7f4d280b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.193+0000 7f4d2cc0f640 1 --2- 172.21.15.16:0/3319873641 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d2815c140 0x7f4d280b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.193+0000 7f4d2cc0f640 1 -- 172.21.15.16:0/3319873641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d2806f9d0 msgr2=0x7f4d281541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.193+0000 7f4d2cc0f640 1 --2- 172.21.15.16:0/3319873641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d2806f9d0 0x7f4d281541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.193+0000 7f4d2cc0f640 1 -- 172.21.15.16:0/3319873641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d280ae9e0 con 0x7f4d280b6480 2024-09-19T07:50:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.193+0000 7f4d2dc11640 1 --2- 172.21.15.16:0/3319873641 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d2815c140 0x7f4d280b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.193+0000 7f4d2cc0f640 1 --2- 172.21.15.16:0/3319873641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d280b6480 0x7f4d2815f570 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0097a0 tx=0x7f4d1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.193+0000 7f4d25ffb640 1 -- 172.21.15.16:0/3319873641 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c037050 con 0x7f4d280b6480 2024-09-19T07:50:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.193+0000 7f4d25ffb640 1 -- 172.21.15.16:0/3319873641 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d1c043070 con 0x7f4d280b6480 2024-09-19T07:50:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.193+0000 7f4d2f674640 1 -- 172.21.15.16:0/3319873641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d280aecc0 con 0x7f4d280b6480 2024-09-19T07:50:47.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.193+0000 7f4d25ffb640 1 -- 172.21.15.16:0/3319873641 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c032810 con 0x7f4d280b6480 2024-09-19T07:50:47.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.193+0000 7f4d2f674640 1 -- 172.21.15.16:0/3319873641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d280af280 con 0x7f4d280b6480 2024-09-19T07:50:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.194+0000 7f4d2f674640 1 -- 172.21.15.16:0/3319873641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d280af660 con 0x7f4d280b6480 2024-09-19T07:50:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.195+0000 7f4d25ffb640 1 -- 172.21.15.16:0/3319873641 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4d1c0329b0 con 0x7f4d280b6480 2024-09-19T07:50:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.195+0000 7f4d25ffb640 1 --2- 172.21.15.16:0/3319873641 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4d1406c2b0 0x7f4d1406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.196+0000 7f4d25ffb640 1 -- 172.21.15.16:0/3319873641 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f4d1c0abf60 con 0x7f4d280b6480 2024-09-19T07:50:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.196+0000 7f4d2d410640 1 --2- 172.21.15.16:0/3319873641 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4d1406c2b0 0x7f4d1406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.196+0000 7f4d2d410640 1 --2- 172.21.15.16:0/3319873641 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4d1406c2b0 0x7f4d1406e770 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4d18001e70 tx=0x7f4d18009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.200+0000 7f4d25ffb640 1 -- 172.21.15.16:0/3319873641 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4d280af660 con 0x7f4d280b6480 2024-09-19T07:50:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.417+0000 7f4d2f674640 1 -- 172.21.15.16:0/3319873641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4d280b01c0 con 0x7f4d280b6480 2024-09-19T07:50:47.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.417+0000 7f4d25ffb640 1 -- 172.21.15.16:0/3319873641 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f4d1c07a5d0 con 0x7f4d280b6480 2024-09-19T07:50:47.418 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:47.418 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:50:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.419+0000 7f4d2f674640 1 -- 172.21.15.16:0/3319873641 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4d1406c2b0 msgr2=0x7f4d1406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.419+0000 7f4d2f674640 1 --2- 172.21.15.16:0/3319873641 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4d1406c2b0 0x7f4d1406e770 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4d18001e70 tx=0x7f4d18009040 comp rx=0 tx=0).stop 2024-09-19T07:50:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.419+0000 7f4d2f674640 1 -- 172.21.15.16:0/3319873641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d280b6480 msgr2=0x7f4d2815f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.419+0000 7f4d2f674640 1 --2- 172.21.15.16:0/3319873641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d280b6480 0x7f4d2815f570 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0097a0 tx=0x7f4d1c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:50:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.419+0000 7f4d2f674640 1 -- 172.21.15.16:0/3319873641 shutdown_connections 2024-09-19T07:50:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.419+0000 7f4d2f674640 1 --2- 172.21.15.16:0/3319873641 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4d1406c2b0 0x7f4d1406e770 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.419+0000 7f4d2f674640 1 --2- 172.21.15.16:0/3319873641 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d2815c140 0x7f4d280b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.419+0000 7f4d2f674640 1 --2- 172.21.15.16:0/3319873641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d280b6480 0x7f4d2815f570 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.419+0000 7f4d2f674640 1 --2- 172.21.15.16:0/3319873641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d2806f9d0 0x7f4d281541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.419+0000 7f4d2f674640 1 -- 172.21.15.16:0/3319873641 >> 172.21.15.16:0/3319873641 conn(0x7f4d2806bc10 msgr2=0x7f4d28074ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.419+0000 7f4d2f674640 1 -- 172.21.15.16:0/3319873641 shutdown_connections 2024-09-19T07:50:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:47.419+0000 7f4d2f674640 1 -- 172.21.15.16:0/3319873641 wait complete. 2024-09-19T07:50:47.430 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:50:47.431 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:50:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:50:50.735 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:50:52.431 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:50:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.545+0000 7f1476531640 1 Processor -- start 2024-09-19T07:50:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.545+0000 7f1476531640 1 -- start start 2024-09-19T07:50:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.546+0000 7f1476531640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f147015e680 0x7f14700cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.546+0000 7f1476531640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14700d1880 con 0x7f14700d0010 2024-09-19T07:50:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.546+0000 7f1476531640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14700d1b60 con 0x7f147015e2a0 2024-09-19T07:50:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.546+0000 7f1476531640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f14700cf2d0 con 0x7f147015e680 2024-09-19T07:50:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.546+0000 7f146f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f147015e680 0x7f14700cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.546+0000 7f1474ace640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f14700d0010 0x7f14700d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47600/0 (socket says 172.21.15.16:47600) 2024-09-19T07:50:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.546+0000 7f1474ace640 1 -- 172.21.15.16:0/528774896 learned_addr learned my addr 172.21.15.16:0/528774896 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.546+0000 7f146f7fe640 1 -- 172.21.15.16:0/528774896 >> v1:172.21.15.146:6789/0 conn(0x7f147015e2a0 legacy=0x7f147006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.546+0000 7f146f7fe640 1 -- 172.21.15.16:0/528774896 >> v1:172.21.15.16:6789/0 conn(0x7f14700d0010 legacy=0x7f14700d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.546+0000 7f146f7fe640 1 -- 172.21.15.16:0/528774896 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14700cf5b0 con 0x7f147015e680 2024-09-19T07:50:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.547+0000 7f146f7fe640 1 --2- 172.21.15.16:0/528774896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147015e680 0x7f14700cfad0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f14640097a0 tx=0x7f146402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bae8accb1b37cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.547+0000 7f146effd640 1 -- 172.21.15.16:0/528774896 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464037070 con 0x7f147015e680 2024-09-19T07:50:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.547+0000 7f146effd640 1 -- 172.21.15.16:0/528774896 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1464032440 con 0x7f147015e680 2024-09-19T07:50:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.547+0000 7f146effd640 1 -- 172.21.15.16:0/528774896 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464032810 con 0x7f147015e680 2024-09-19T07:50:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.548+0000 7f1476531640 1 -- 172.21.15.16:0/528774896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147015e680 msgr2=0x7f14700cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.548+0000 7f1476531640 1 --2- 172.21.15.16:0/528774896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147015e680 0x7f14700cfad0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f14640097a0 tx=0x7f146402a1a0 comp rx=0 tx=0).stop 2024-09-19T07:50:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.548+0000 7f1476531640 1 -- 172.21.15.16:0/528774896 shutdown_connections 2024-09-19T07:50:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.548+0000 7f1476531640 1 --2- 172.21.15.16:0/528774896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147015e680 0x7f14700cfad0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.548+0000 7f1476531640 1 -- 172.21.15.16:0/528774896 >> 172.21.15.16:0/528774896 conn(0x7f147009d760 msgr2=0x7f147008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.548+0000 7f1476531640 1 -- 172.21.15.16:0/528774896 shutdown_connections 2024-09-19T07:50:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.548+0000 7f1476531640 1 -- 172.21.15.16:0/528774896 wait complete. 2024-09-19T07:50:52.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.549+0000 7f1476531640 1 Processor -- start 2024-09-19T07:50:52.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.549+0000 7f1476531640 1 -- start start 2024-09-19T07:50:52.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.550+0000 7f1476531640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14700d0010 0x7f14700ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:52.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.550+0000 7f1476531640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f147015e2a0 0x7f147015cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:52.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.550+0000 7f1476531640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147015e680 0x7f147015d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.550+0000 7f1476531640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f147006dcb0 con 0x7f147015e680 2024-09-19T07:50:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.550+0000 7f1476531640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f147006db30 con 0x7f147015e2a0 2024-09-19T07:50:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.550+0000 7f1476531640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f14700d4a80 con 0x7f14700d0010 2024-09-19T07:50:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.550+0000 7f1474ace640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147015e680 0x7f147015d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.550+0000 7f1474ace640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147015e680 0x7f147015d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40218/0 (socket says 172.21.15.16:40218) 2024-09-19T07:50:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.550+0000 7f1474ace640 1 -- 172.21.15.16:0/1847431411 learned_addr learned my addr 172.21.15.16:0/1847431411 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.550+0000 7f146ffff640 1 --2- 172.21.15.16:0/1847431411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14700d0010 0x7f14700ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.550+0000 7f146f7fe640 1 --2- 172.21.15.16:0/1847431411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f147015e2a0 0x7f147015cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.551+0000 7f1474ace640 1 -- 172.21.15.16:0/1847431411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14700d0010 msgr2=0x7f14700ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.551+0000 7f1474ace640 1 --2- 172.21.15.16:0/1847431411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14700d0010 0x7f14700ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.551+0000 7f1474ace640 1 -- 172.21.15.16:0/1847431411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f147015e2a0 msgr2=0x7f147015cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.551+0000 7f1474ace640 1 --2- 172.21.15.16:0/1847431411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f147015e2a0 0x7f147015cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.551+0000 7f1474ace640 1 -- 172.21.15.16:0/1847431411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14700cd530 con 0x7f147015e680 2024-09-19T07:50:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.551+0000 7f1474ace640 1 --2- 172.21.15.16:0/1847431411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147015e680 0x7f147015d420 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f146800cbc0 tx=0x7f146800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.551+0000 7f146cff9640 1 -- 172.21.15.16:0/1847431411 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468017070 con 0x7f147015e680 2024-09-19T07:50:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.551+0000 7f146cff9640 1 -- 172.21.15.16:0/1847431411 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f146800ad40 con 0x7f147015e680 2024-09-19T07:50:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.551+0000 7f1476531640 1 -- 172.21.15.16:0/1847431411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14700cd810 con 0x7f147015e680 2024-09-19T07:50:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.551+0000 7f146cff9640 1 -- 172.21.15.16:0/1847431411 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468012600 con 0x7f147015e680 2024-09-19T07:50:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.551+0000 7f1476531640 1 -- 172.21.15.16:0/1847431411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14700cddd0 con 0x7f147015e680 2024-09-19T07:50:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.552+0000 7f1476531640 1 -- 172.21.15.16:0/1847431411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14700cfad0 con 0x7f147015e680 2024-09-19T07:50:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.553+0000 7f146cff9640 1 -- 172.21.15.16:0/1847431411 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f146801a070 con 0x7f147015e680 2024-09-19T07:50:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.553+0000 7f146cff9640 1 --2- 172.21.15.16:0/1847431411 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f145c06c2b0 0x7f145c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.553+0000 7f146cff9640 1 -- 172.21.15.16:0/1847431411 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f146808b320 con 0x7f147015e680 2024-09-19T07:50:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.554+0000 7f146ffff640 1 --2- 172.21.15.16:0/1847431411 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f145c06c2b0 0x7f145c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.554+0000 7f146ffff640 1 --2- 172.21.15.16:0/1847431411 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f145c06c2b0 0x7f145c06e770 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1460000f60 tx=0x7f1460009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.558+0000 7f146cff9640 1 -- 172.21.15.16:0/1847431411 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1468059240 con 0x7f147015e680 2024-09-19T07:50:52.736 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:50:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.771+0000 7f1476531640 1 -- 172.21.15.16:0/1847431411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f147006cf90 con 0x7f147015e680 2024-09-19T07:50:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.772+0000 7f146cff9640 1 -- 172.21.15.16:0/1847431411 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f146805d540 con 0x7f147015e680 2024-09-19T07:50:52.773 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:52.773 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:50:52.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.773+0000 7f1476531640 1 -- 172.21.15.16:0/1847431411 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f145c06c2b0 msgr2=0x7f145c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:52.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.773+0000 7f1476531640 1 --2- 172.21.15.16:0/1847431411 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f145c06c2b0 0x7f145c06e770 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1460000f60 tx=0x7f1460009040 comp rx=0 tx=0).stop 2024-09-19T07:50:52.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.773+0000 7f1476531640 1 -- 172.21.15.16:0/1847431411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147015e680 msgr2=0x7f147015d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:52.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.773+0000 7f1476531640 1 --2- 172.21.15.16:0/1847431411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147015e680 0x7f147015d420 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f146800cbc0 tx=0x7f146800a430 comp rx=0 tx=0).stop 2024-09-19T07:50:52.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.774+0000 7f1476531640 1 -- 172.21.15.16:0/1847431411 shutdown_connections 2024-09-19T07:50:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.774+0000 7f1476531640 1 --2- 172.21.15.16:0/1847431411 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f145c06c2b0 0x7f145c06e770 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.774+0000 7f1476531640 1 --2- 172.21.15.16:0/1847431411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f147015e680 0x7f147015d420 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.774+0000 7f1476531640 1 --2- 172.21.15.16:0/1847431411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f147015e2a0 0x7f147015cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.774+0000 7f1476531640 1 --2- 172.21.15.16:0/1847431411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f14700d0010 0x7f14700ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.774+0000 7f1476531640 1 -- 172.21.15.16:0/1847431411 >> 172.21.15.16:0/1847431411 conn(0x7f147009d760 msgr2=0x7f14700d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.774+0000 7f1476531640 1 -- 172.21.15.16:0/1847431411 shutdown_connections 2024-09-19T07:50:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:52.774+0000 7f1476531640 1 -- 172.21.15.16:0/1847431411 wait complete. 2024-09-19T07:50:52.786 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:50:52.786 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:50:54.738 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:50:56.739 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:50:57.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:50:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.902+0000 7f0b6a45e640 1 Processor -- start 2024-09-19T07:50:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.902+0000 7f0b6a45e640 1 -- start start 2024-09-19T07:50:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.903+0000 7f0b6a45e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0b6415f0d0 0x7f0b6415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:57.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.903+0000 7f0b6a45e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b640d0070 0x7f0b640d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.903+0000 7f0b6a45e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b640cf2b0 con 0x7f0b640d1940 2024-09-19T07:50:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.903+0000 7f0b6a45e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f0b640cf590 con 0x7f0b6415f0d0 2024-09-19T07:50:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.903+0000 7f0b6a45e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0b640cf870 con 0x7f0b640d0070 2024-09-19T07:50:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.903+0000 7f0b689fb640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0b640d1940 0x7f0b640d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47602/0 (socket says 172.21.15.16:47602) 2024-09-19T07:50:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.903+0000 7f0b689fb640 1 -- 172.21.15.16:0/3846484720 learned_addr learned my addr 172.21.15.16:0/3846484720 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:50:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.903+0000 7f0b637fe640 1 --2- 172.21.15.16:0/3846484720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b640d0070 0x7f0b640d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.903+0000 7f0b63fff640 1 --2- 172.21.15.16:0/3846484720 >> v2:172.21.15.146:3300/0 conn(0x7f0b6415f0d0 0x7f0b6415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.903+0000 7f0b62ffd640 1 -- 172.21.15.16:0/3846484720 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1896353443 0 0) 0x7f0b640cf2b0 con 0x7f0b640d1940 2024-09-19T07:50:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.903+0000 7f0b62ffd640 1 -- 172.21.15.16:0/3846484720 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b4c003520 con 0x7f0b640d1940 2024-09-19T07:50:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.903+0000 7f0b62ffd640 1 -- 172.21.15.16:0/3846484720 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1229042156 0 0) 0x7f0b4c003520 con 0x7f0b640d1940 2024-09-19T07:50:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.903+0000 7f0b62ffd640 1 -- 172.21.15.16:0/3846484720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b640d0070 msgr2=0x7f0b640d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.903+0000 7f0b62ffd640 1 --2- 172.21.15.16:0/3846484720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b640d0070 0x7f0b640d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.903+0000 7f0b62ffd640 1 -- 172.21.15.16:0/3846484720 >> v2:172.21.15.146:3300/0 conn(0x7f0b6415f0d0 msgr2=0x7f0b6415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.903+0000 7f0b62ffd640 1 --2- 172.21.15.16:0/3846484720 >> v2:172.21.15.146:3300/0 conn(0x7f0b6415f0d0 0x7f0b6415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.903+0000 7f0b62ffd640 1 -- 172.21.15.16:0/3846484720 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b640cfb50 con 0x7f0b640d1940 2024-09-19T07:50:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.903+0000 7f0b62ffd640 1 -- 172.21.15.16:0/3846484720 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f0b5c001a10 con 0x7f0b640d1940 2024-09-19T07:50:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.903+0000 7f0b62ffd640 1 -- 172.21.15.16:0/3846484720 >> v1:172.21.15.16:6789/0 conn(0x7f0b640d1940 legacy=0x7f0b640d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.904+0000 7f0b62ffd640 1 --2- 172.21.15.16:0/3846484720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b4c0039b0 0x7f0b4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.904+0000 7f0b62ffd640 1 --2- 172.21.15.16:0/3846484720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b4c004800 0x7f0b4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.904+0000 7f0b62ffd640 1 --2- 172.21.15.16:0/3846484720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b640d0070 0x7f0b4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.904+0000 7f0b62ffd640 1 -- 172.21.15.16:0/3846484720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b640cf590 con 0x7f0b4c004800 2024-09-19T07:50:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.904+0000 7f0b62ffd640 1 -- 172.21.15.16:0/3846484720 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0b640cf870 con 0x7f0b4c0039b0 2024-09-19T07:50:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.904+0000 7f0b62ffd640 1 -- 172.21.15.16:0/3846484720 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0b4c00b9f0 con 0x7f0b640d0070 2024-09-19T07:50:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.904+0000 7f0b637fe640 1 --2- 172.21.15.16:0/3846484720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b4c004800 0x7f0b4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.904+0000 7f0b63fff640 1 --2- 172.21.15.16:0/3846484720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b4c0039b0 0x7f0b4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.904+0000 7f0b689fb640 1 --2- 172.21.15.16:0/3846484720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b640d0070 0x7f0b4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.904+0000 7f0b637fe640 1 -- 172.21.15.16:0/3846484720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b640d0070 msgr2=0x7f0b4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.904+0000 7f0b637fe640 1 --2- 172.21.15.16:0/3846484720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b640d0070 0x7f0b4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.904+0000 7f0b637fe640 1 -- 172.21.15.16:0/3846484720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b4c0039b0 msgr2=0x7f0b4c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.904+0000 7f0b637fe640 1 --2- 172.21.15.16:0/3846484720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b4c0039b0 0x7f0b4c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.904+0000 7f0b637fe640 1 -- 172.21.15.16:0/3846484720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4c003520 con 0x7f0b4c004800 2024-09-19T07:50:57.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.905+0000 7f0b637fe640 1 --2- 172.21.15.16:0/3846484720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b4c004800 0x7f0b4c004c80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009ac0 tx=0x7f0b4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e3f20cbac417fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.905+0000 7f0b62ffd640 1 -- 172.21.15.16:0/3846484720 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58036070 con 0x7f0b4c004800 2024-09-19T07:50:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.905+0000 7f0b62ffd640 1 -- 172.21.15.16:0/3846484720 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b58031400 con 0x7f0b4c004800 2024-09-19T07:50:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.905+0000 7f0b62ffd640 1 -- 172.21.15.16:0/3846484720 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58009690 con 0x7f0b4c004800 2024-09-19T07:50:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.906+0000 7f0b6a45e640 1 -- 172.21.15.16:0/3846484720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b4c004800 msgr2=0x7f0b4c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.906+0000 7f0b6a45e640 1 --2- 172.21.15.16:0/3846484720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b4c004800 0x7f0b4c004c80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009ac0 tx=0x7f0b4c005830 comp rx=0 tx=0).stop 2024-09-19T07:50:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.906+0000 7f0b689fb640 1 -- 172.21.15.16:0/3846484720 reap_dead start 2024-09-19T07:50:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.906+0000 7f0b6a45e640 1 -- 172.21.15.16:0/3846484720 shutdown_connections 2024-09-19T07:50:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.906+0000 7f0b6a45e640 1 -- 172.21.15.16:0/3846484720 >> 172.21.15.16:0/3846484720 conn(0x7f0b6409d760 msgr2=0x7f0b6408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.906+0000 7f0b6a45e640 1 -- 172.21.15.16:0/3846484720 shutdown_connections 2024-09-19T07:50:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.906+0000 7f0b6a45e640 1 -- 172.21.15.16:0/3846484720 wait complete. 2024-09-19T07:50:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.907+0000 7f0b6a45e640 1 Processor -- start 2024-09-19T07:50:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.907+0000 7f0b6a45e640 1 -- start start 2024-09-19T07:50:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.908+0000 7f0b6a45e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b4c004800 0x7f0b6415b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.908+0000 7f0b6a45e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b641f8230 0x7f0b64157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.908+0000 7f0b6a45e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b64158390 0x7f0b64161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.908+0000 7f0b6a45e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b640d49c0 con 0x7f0b64158390 2024-09-19T07:50:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.908+0000 7f0b6a45e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0b640d4840 con 0x7f0b641f8230 2024-09-19T07:50:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.908+0000 7f0b6a45e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0b640d4b40 con 0x7f0b4c004800 2024-09-19T07:50:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.908+0000 7f0b689fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b64158390 0x7f0b64161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.908+0000 7f0b689fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b64158390 0x7f0b64161eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40236/0 (socket says 172.21.15.16:40236) 2024-09-19T07:50:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.908+0000 7f0b689fb640 1 -- 172.21.15.16:0/1278606748 learned_addr learned my addr 172.21.15.16:0/1278606748 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:50:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.908+0000 7f0b63fff640 1 --2- 172.21.15.16:0/1278606748 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b4c004800 0x7f0b6415b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.908+0000 7f0b637fe640 1 --2- 172.21.15.16:0/1278606748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b641f8230 0x7f0b64157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.908+0000 7f0b689fb640 1 -- 172.21.15.16:0/1278606748 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b4c004800 msgr2=0x7f0b6415b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.908+0000 7f0b689fb640 1 --2- 172.21.15.16:0/1278606748 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b4c004800 0x7f0b6415b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.908+0000 7f0b689fb640 1 -- 172.21.15.16:0/1278606748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b641f8230 msgr2=0x7f0b64157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.908+0000 7f0b689fb640 1 --2- 172.21.15.16:0/1278606748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b641f8230 0x7f0b64157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.908+0000 7f0b689fb640 1 -- 172.21.15.16:0/1278606748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b64158900 con 0x7f0b64158390 2024-09-19T07:50:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.908+0000 7f0b689fb640 1 --2- 172.21.15.16:0/1278606748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b64158390 0x7f0b64161eb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0b640ce9e0 tx=0x7f0b5c007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.909+0000 7f0b60ff9640 1 -- 172.21.15.16:0/1278606748 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c017020 con 0x7f0b64158390 2024-09-19T07:50:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.909+0000 7f0b60ff9640 1 -- 172.21.15.16:0/1278606748 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b5c00ab80 con 0x7f0b64158390 2024-09-19T07:50:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.909+0000 7f0b6a45e640 1 -- 172.21.15.16:0/1278606748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b641f7750 con 0x7f0b64158390 2024-09-19T07:50:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.909+0000 7f0b60ff9640 1 -- 172.21.15.16:0/1278606748 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0133f0 con 0x7f0b64158390 2024-09-19T07:50:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.909+0000 7f0b6a45e640 1 -- 172.21.15.16:0/1278606748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b641f7d10 con 0x7f0b64158390 2024-09-19T07:50:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.910+0000 7f0b6a45e640 1 -- 172.21.15.16:0/1278606748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b641f7570 con 0x7f0b64158390 2024-09-19T07:50:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.910+0000 7f0b60ff9640 1 -- 172.21.15.16:0/1278606748 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0b5c01a020 con 0x7f0b64158390 2024-09-19T07:50:57.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.911+0000 7f0b60ff9640 1 --2- 172.21.15.16:0/1278606748 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0b5006c2b0 0x7f0b5006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:57.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.911+0000 7f0b60ff9640 1 -- 172.21.15.16:0/1278606748 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f0b5c08bd30 con 0x7f0b64158390 2024-09-19T07:50:57.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.911+0000 7f0b63fff640 1 --2- 172.21.15.16:0/1278606748 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0b5006c2b0 0x7f0b5006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:57.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.912+0000 7f0b63fff640 1 --2- 172.21.15.16:0/1278606748 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0b5006c2b0 0x7f0b5006e770 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009a00 tx=0x7f0b54008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:57.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:57.915+0000 7f0b60ff9640 1 -- 172.21.15.16:0/1278606748 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f0b641f7570 con 0x7f0b64158390 2024-09-19T07:50:58.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:58.127+0000 7f0b6a45e640 1 -- 172.21.15.16:0/1278606748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0b6415f4b0 con 0x7f0b64158390 2024-09-19T07:50:58.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:58.128+0000 7f0b60ff9640 1 -- 172.21.15.16:0/1278606748 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f0b5c021020 con 0x7f0b64158390 2024-09-19T07:50:58.128 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:50:58.128 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:50:58.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:58.130+0000 7f0b6a45e640 1 -- 172.21.15.16:0/1278606748 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0b5006c2b0 msgr2=0x7f0b5006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:58.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:58.130+0000 7f0b6a45e640 1 --2- 172.21.15.16:0/1278606748 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0b5006c2b0 0x7f0b5006e770 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009a00 tx=0x7f0b54008040 comp rx=0 tx=0).stop 2024-09-19T07:50:58.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:58.130+0000 7f0b6a45e640 1 -- 172.21.15.16:0/1278606748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b64158390 msgr2=0x7f0b64161eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:58.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:58.130+0000 7f0b6a45e640 1 --2- 172.21.15.16:0/1278606748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b64158390 0x7f0b64161eb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0b640ce9e0 tx=0x7f0b5c007a90 comp rx=0 tx=0).stop 2024-09-19T07:50:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:58.130+0000 7f0b6a45e640 1 -- 172.21.15.16:0/1278606748 shutdown_connections 2024-09-19T07:50:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:58.130+0000 7f0b6a45e640 1 --2- 172.21.15.16:0/1278606748 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0b5006c2b0 0x7f0b5006e770 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:58.130+0000 7f0b6a45e640 1 --2- 172.21.15.16:0/1278606748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b64158390 0x7f0b64161eb0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:58.130+0000 7f0b6a45e640 1 --2- 172.21.15.16:0/1278606748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b641f8230 0x7f0b64157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:58.130+0000 7f0b6a45e640 1 --2- 172.21.15.16:0/1278606748 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b4c004800 0x7f0b6415b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:58.130+0000 7f0b6a45e640 1 -- 172.21.15.16:0/1278606748 >> 172.21.15.16:0/1278606748 conn(0x7f0b6409d760 msgr2=0x7f0b6415c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:58.130+0000 7f0b6a45e640 1 -- 172.21.15.16:0/1278606748 shutdown_connections 2024-09-19T07:50:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:50:58.130+0000 7f0b6a45e640 1 -- 172.21.15.16:0/1278606748 wait complete. 2024-09-19T07:50:58.141 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:50:58.141 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:50:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:02.745 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:03.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:51:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.257+0000 7ff09c027640 1 Processor -- start 2024-09-19T07:51:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.257+0000 7ff09c027640 1 -- start start 2024-09-19T07:51:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.258+0000 7ff09c027640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff09413f0c0 0x7ff09415d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.258+0000 7ff09c027640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff0940d01c0 0x7ff094141e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.258+0000 7ff09c027640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff09415d6b0 con 0x7ff0941419d0 2024-09-19T07:51:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.258+0000 7ff09c027640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff094142380 con 0x7ff0940d01c0 2024-09-19T07:51:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.258+0000 7ff09c027640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff094072450 con 0x7ff09413f0c0 2024-09-19T07:51:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.258+0000 7ff09a5c4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff0940d01c0 0x7ff094141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.258+0000 7ff09a5c4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff0940d01c0 0x7ff094141e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37978/0 (socket says 172.21.15.16:37978) 2024-09-19T07:51:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.258+0000 7ff09a5c4640 1 -- 172.21.15.16:0/3072839892 learned_addr learned my addr 172.21.15.16:0/3072839892 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:51:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.258+0000 7ff099dc3640 1 --2- 172.21.15.16:0/3072839892 >> v2:172.21.15.146:3300/0 conn(0x7ff09413f0c0 0x7ff09415d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:03.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.258+0000 7ff09a5c4640 1 -- 172.21.15.16:0/3072839892 >> v2:172.21.15.146:3300/0 conn(0x7ff09413f0c0 msgr2=0x7ff09415d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:03.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.258+0000 7ff09a5c4640 1 --2- 172.21.15.16:0/3072839892 >> v2:172.21.15.146:3300/0 conn(0x7ff09413f0c0 0x7ff09415d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:03.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.258+0000 7ff09a5c4640 1 -- 172.21.15.16:0/3072839892 >> v1:172.21.15.146:6789/0 conn(0x7ff0941419d0 legacy=0x7ff09415fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:03.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.258+0000 7ff09a5c4640 1 -- 172.21.15.16:0/3072839892 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff094072730 con 0x7ff0940d01c0 2024-09-19T07:51:03.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.259+0000 7ff09a5c4640 1 --2- 172.21.15.16:0/3072839892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0940d01c0 0x7ff094141e40 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff0940683f0 tx=0x7ff08c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecffa04bd0eeff35 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:03.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.259+0000 7ff098dc1640 1 -- 172.21.15.16:0/3072839892 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c036070 con 0x7ff0940d01c0 2024-09-19T07:51:03.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.259+0000 7ff098dc1640 1 -- 172.21.15.16:0/3072839892 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff08c031600 con 0x7ff0940d01c0 2024-09-19T07:51:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.259+0000 7ff098dc1640 1 -- 172.21.15.16:0/3072839892 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0319d0 con 0x7ff0940d01c0 2024-09-19T07:51:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.260+0000 7ff09c027640 1 -- 172.21.15.16:0/3072839892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0940d01c0 msgr2=0x7ff094141e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.260+0000 7ff09c027640 1 --2- 172.21.15.16:0/3072839892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0940d01c0 0x7ff094141e40 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff0940683f0 tx=0x7ff08c02a6f0 comp rx=0 tx=0).stop 2024-09-19T07:51:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.260+0000 7ff09c027640 1 -- 172.21.15.16:0/3072839892 shutdown_connections 2024-09-19T07:51:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.260+0000 7ff09c027640 1 --2- 172.21.15.16:0/3072839892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0940d01c0 0x7ff094141e40 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.260+0000 7ff09c027640 1 --2- 172.21.15.16:0/3072839892 >> v2:172.21.15.146:3300/0 conn(0x7ff09413f0c0 0x7ff09415d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.260+0000 7ff09c027640 1 -- 172.21.15.16:0/3072839892 >> 172.21.15.16:0/3072839892 conn(0x7ff094067d70 msgr2=0x7ff09406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.260+0000 7ff09c027640 1 -- 172.21.15.16:0/3072839892 shutdown_connections 2024-09-19T07:51:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.260+0000 7ff09c027640 1 -- 172.21.15.16:0/3072839892 wait complete. 2024-09-19T07:51:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.261+0000 7ff09c027640 1 Processor -- start 2024-09-19T07:51:03.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.261+0000 7ff09c027640 1 -- start start 2024-09-19T07:51:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.262+0000 7ff09c027640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0940d01c0 0x7ff0940b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.262+0000 7ff09c027640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09413f0c0 0x7ff0941591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.262+0000 7ff09c027640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0941419d0 0x7ff0940b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.262+0000 7ff09c027640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff094072fd0 con 0x7ff09413f0c0 2024-09-19T07:51:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.262+0000 7ff09c027640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff094072cf0 con 0x7ff0941419d0 2024-09-19T07:51:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.262+0000 7ff09c027640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff0941427a0 con 0x7ff0940d01c0 2024-09-19T07:51:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.262+0000 7ff0995c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09413f0c0 0x7ff0941591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.262+0000 7ff0995c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09413f0c0 0x7ff0941591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37980/0 (socket says 172.21.15.16:37980) 2024-09-19T07:51:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.262+0000 7ff0995c2640 1 -- 172.21.15.16:0/2918803626 learned_addr learned my addr 172.21.15.16:0/2918803626 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:51:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.262+0000 7ff099dc3640 1 --2- 172.21.15.16:0/2918803626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0940d01c0 0x7ff0940b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:03.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.262+0000 7ff09a5c4640 1 --2- 172.21.15.16:0/2918803626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0941419d0 0x7ff0940b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.262+0000 7ff0995c2640 1 -- 172.21.15.16:0/2918803626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0940d01c0 msgr2=0x7ff0940b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.262+0000 7ff0995c2640 1 --2- 172.21.15.16:0/2918803626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0940d01c0 0x7ff0940b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.262+0000 7ff0995c2640 1 -- 172.21.15.16:0/2918803626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0941419d0 msgr2=0x7ff0940b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.262+0000 7ff0995c2640 1 --2- 172.21.15.16:0/2918803626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0941419d0 0x7ff0940b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.262+0000 7ff0995c2640 1 -- 172.21.15.16:0/2918803626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff094159710 con 0x7ff09413f0c0 2024-09-19T07:51:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.262+0000 7ff0995c2640 1 --2- 172.21.15.16:0/2918803626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09413f0c0 0x7ff0941591d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff08800b3f0 tx=0x7ff08800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.263+0000 7ff0867fc640 1 -- 172.21.15.16:0/2918803626 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088015020 con 0x7ff09413f0c0 2024-09-19T07:51:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.263+0000 7ff0867fc640 1 -- 172.21.15.16:0/2918803626 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff088011830 con 0x7ff09413f0c0 2024-09-19T07:51:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.263+0000 7ff09c027640 1 -- 172.21.15.16:0/2918803626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0940b1e40 con 0x7ff09413f0c0 2024-09-19T07:51:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.263+0000 7ff0867fc640 1 -- 172.21.15.16:0/2918803626 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088011c20 con 0x7ff09413f0c0 2024-09-19T07:51:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.263+0000 7ff09c027640 1 -- 172.21.15.16:0/2918803626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0940b2400 con 0x7ff09413f0c0 2024-09-19T07:51:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.264+0000 7ff09c027640 1 -- 172.21.15.16:0/2918803626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0940720b0 con 0x7ff09413f0c0 2024-09-19T07:51:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.264+0000 7ff0867fc640 1 -- 172.21.15.16:0/2918803626 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff088018020 con 0x7ff09413f0c0 2024-09-19T07:51:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.265+0000 7ff0867fc640 1 --2- 172.21.15.16:0/2918803626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff07806c2b0 0x7ff07806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.265+0000 7ff0867fc640 1 -- 172.21.15.16:0/2918803626 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7ff08808b140 con 0x7ff09413f0c0 2024-09-19T07:51:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.265+0000 7ff099dc3640 1 --2- 172.21.15.16:0/2918803626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff07806c2b0 0x7ff07806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.266+0000 7ff099dc3640 1 --2- 172.21.15.16:0/2918803626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff07806c2b0 0x7ff07806e770 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff090006fd0 tx=0x7ff090008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:03.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.269+0000 7ff0867fc640 1 -- 172.21.15.16:0/2918803626 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff08801f030 con 0x7ff09413f0c0 2024-09-19T07:51:03.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.482+0000 7ff09c027640 1 -- 172.21.15.16:0/2918803626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff0940d05a0 con 0x7ff09413f0c0 2024-09-19T07:51:03.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.483+0000 7ff0867fc640 1 -- 172.21.15.16:0/2918803626 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7ff088059050 con 0x7ff09413f0c0 2024-09-19T07:51:03.483 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:51:03.483 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:51:03.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.485+0000 7ff09c027640 1 -- 172.21.15.16:0/2918803626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff07806c2b0 msgr2=0x7ff07806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:03.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.485+0000 7ff09c027640 1 --2- 172.21.15.16:0/2918803626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff07806c2b0 0x7ff07806e770 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff090006fd0 tx=0x7ff090008040 comp rx=0 tx=0).stop 2024-09-19T07:51:03.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.485+0000 7ff09c027640 1 -- 172.21.15.16:0/2918803626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09413f0c0 msgr2=0x7ff0941591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:03.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.485+0000 7ff09c027640 1 --2- 172.21.15.16:0/2918803626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09413f0c0 0x7ff0941591d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff08800b3f0 tx=0x7ff08800b8b0 comp rx=0 tx=0).stop 2024-09-19T07:51:03.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.485+0000 7ff09c027640 1 -- 172.21.15.16:0/2918803626 shutdown_connections 2024-09-19T07:51:03.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.485+0000 7ff09c027640 1 --2- 172.21.15.16:0/2918803626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff07806c2b0 0x7ff07806e770 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:03.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.485+0000 7ff09c027640 1 --2- 172.21.15.16:0/2918803626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0941419d0 0x7ff0940b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:03.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.485+0000 7ff09c027640 1 --2- 172.21.15.16:0/2918803626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff09413f0c0 0x7ff0941591d0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:03.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.485+0000 7ff09c027640 1 --2- 172.21.15.16:0/2918803626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0940d01c0 0x7ff0940b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:03.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.485+0000 7ff09c027640 1 -- 172.21.15.16:0/2918803626 >> 172.21.15.16:0/2918803626 conn(0x7ff094067d70 msgr2=0x7ff0940cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:03.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.485+0000 7ff09c027640 1 -- 172.21.15.16:0/2918803626 shutdown_connections 2024-09-19T07:51:03.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:03.485+0000 7ff09c027640 1 -- 172.21.15.16:0/2918803626 wait complete. 2024-09-19T07:51:03.496 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:51:03.496 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:51:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:06.749 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:08.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:51:08.482 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:51:08.497 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:51:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.633+0000 7f404427b640 1 Processor -- start 2024-09-19T07:51:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.633+0000 7f404427b640 1 -- start start 2024-09-19T07:51:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.634+0000 7f404427b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c0b0a80 0x7f403c0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.634+0000 7f404427b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f403c161d90 0x7f403c0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.634+0000 7f404427b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f403c06c990 con 0x7f403c1619b0 2024-09-19T07:51:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.634+0000 7f404427b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f403c0cec10 con 0x7f403c161d90 2024-09-19T07:51:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.634+0000 7f404427b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f403c0ce930 con 0x7f403c0b0a80 2024-09-19T07:51:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.634+0000 7f4042818640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f403c161d90 0x7f403c0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.634+0000 7f4042818640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f403c161d90 0x7f403c0cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37982/0 (socket says 172.21.15.16:37982) 2024-09-19T07:51:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.634+0000 7f4042818640 1 -- 172.21.15.16:0/2172374006 learned_addr learned my addr 172.21.15.16:0/2172374006 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:51:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.634+0000 7f4041816640 1 --2- 172.21.15.16:0/2172374006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c0b0a80 0x7f403c0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.634+0000 7f4042818640 1 -- 172.21.15.16:0/2172374006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c0b0a80 msgr2=0x7f403c0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.634+0000 7f4042818640 1 --2- 172.21.15.16:0/2172374006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c0b0a80 0x7f403c0cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.634+0000 7f4042818640 1 -- 172.21.15.16:0/2172374006 >> v1:172.21.15.146:6789/0 conn(0x7f403c1619b0 legacy=0x7f403c0d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.634+0000 7f4042818640 1 -- 172.21.15.16:0/2172374006 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c0b0540 con 0x7f403c161d90 2024-09-19T07:51:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.635+0000 7f4042818640 1 --2- 172.21.15.16:0/2172374006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f403c161d90 0x7f403c0cc210 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f4034009e50 tx=0x7f403402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d0aa812d940fdf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.635+0000 7f4041015640 1 -- 172.21.15.16:0/2172374006 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034036070 con 0x7f403c161d90 2024-09-19T07:51:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.635+0000 7f4041015640 1 -- 172.21.15.16:0/2172374006 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4034031600 con 0x7f403c161d90 2024-09-19T07:51:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.635+0000 7f4041015640 1 -- 172.21.15.16:0/2172374006 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40340319d0 con 0x7f403c161d90 2024-09-19T07:51:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.636+0000 7f404427b640 1 -- 172.21.15.16:0/2172374006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f403c161d90 msgr2=0x7f403c0cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.636+0000 7f404427b640 1 --2- 172.21.15.16:0/2172374006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f403c161d90 0x7f403c0cc210 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f4034009e50 tx=0x7f403402a710 comp rx=0 tx=0).stop 2024-09-19T07:51:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.636+0000 7f404427b640 1 -- 172.21.15.16:0/2172374006 shutdown_connections 2024-09-19T07:51:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.636+0000 7f404427b640 1 --2- 172.21.15.16:0/2172374006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f403c161d90 0x7f403c0cc210 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.636+0000 7f404427b640 1 --2- 172.21.15.16:0/2172374006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c0b0a80 0x7f403c0cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.636+0000 7f404427b640 1 -- 172.21.15.16:0/2172374006 >> 172.21.15.16:0/2172374006 conn(0x7f403c09a9f0 msgr2=0x7f403c0b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.636+0000 7f404427b640 1 -- 172.21.15.16:0/2172374006 shutdown_connections 2024-09-19T07:51:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.636+0000 7f404427b640 1 -- 172.21.15.16:0/2172374006 wait complete. 2024-09-19T07:51:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.637+0000 7f404427b640 1 Processor -- start 2024-09-19T07:51:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.637+0000 7f404427b640 1 -- start start 2024-09-19T07:51:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.638+0000 7f404427b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f403c0b0a80 0x7f403c158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.638+0000 7f404427b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f403c1619b0 0x7f403c15da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.638+0000 7f404427b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c161d90 0x7f403c15dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.638+0000 7f404427b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f403c0cd330 con 0x7f403c0b0a80 2024-09-19T07:51:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.638+0000 7f404427b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f403c06c620 con 0x7f403c1619b0 2024-09-19T07:51:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.638+0000 7f404427b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f403c0cd4b0 con 0x7f403c161d90 2024-09-19T07:51:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.638+0000 7f4042017640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f403c0b0a80 0x7f403c158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.638+0000 7f4042017640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f403c0b0a80 0x7f403c158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37994/0 (socket says 172.21.15.16:37994) 2024-09-19T07:51:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.638+0000 7f4042017640 1 -- 172.21.15.16:0/473594033 learned_addr learned my addr 172.21.15.16:0/473594033 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:51:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.638+0000 7f4042017640 1 -- 172.21.15.16:0/473594033 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c161d90 msgr2=0x7f403c15dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.638+0000 7f4042818640 1 --2- 172.21.15.16:0/473594033 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c161d90 0x7f403c15dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.638+0000 7f4041816640 1 --2- 172.21.15.16:0/473594033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f403c1619b0 0x7f403c15da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.638+0000 7f4042017640 1 --2- 172.21.15.16:0/473594033 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c161d90 0x7f403c15dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.638+0000 7f4042017640 1 -- 172.21.15.16:0/473594033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f403c1619b0 msgr2=0x7f403c15da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.638+0000 7f4042017640 1 --2- 172.21.15.16:0/473594033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f403c1619b0 0x7f403c15da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.638+0000 7f4042017640 1 -- 172.21.15.16:0/473594033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c0c68c0 con 0x7f403c0b0a80 2024-09-19T07:51:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.638+0000 7f4042017640 1 --2- 172.21.15.16:0/473594033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f403c0b0a80 0x7f403c158240 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f403800c430 tx=0x7f403800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.638+0000 7f4032ffd640 1 -- 172.21.15.16:0/473594033 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40380052d0 con 0x7f403c0b0a80 2024-09-19T07:51:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.639+0000 7f4032ffd640 1 -- 172.21.15.16:0/473594033 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4038005490 con 0x7f403c0b0a80 2024-09-19T07:51:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.639+0000 7f404427b640 1 -- 172.21.15.16:0/473594033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403c156680 con 0x7f403c0b0a80 2024-09-19T07:51:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.639+0000 7f4032ffd640 1 -- 172.21.15.16:0/473594033 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038007260 con 0x7f403c0b0a80 2024-09-19T07:51:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.639+0000 7f404427b640 1 -- 172.21.15.16:0/473594033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f403c156c40 con 0x7f403c0b0a80 2024-09-19T07:51:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.640+0000 7f404427b640 1 -- 172.21.15.16:0/473594033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f403c0d0440 con 0x7f403c0b0a80 2024-09-19T07:51:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.640+0000 7f4032ffd640 1 -- 172.21.15.16:0/473594033 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4038007400 con 0x7f403c0b0a80 2024-09-19T07:51:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.641+0000 7f4032ffd640 1 --2- 172.21.15.16:0/473594033 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f402006c2b0 0x7f402006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.641+0000 7f4032ffd640 1 -- 172.21.15.16:0/473594033 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f403808ab70 con 0x7f403c0b0a80 2024-09-19T07:51:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.641+0000 7f4041816640 1 --2- 172.21.15.16:0/473594033 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f402006c2b0 0x7f402006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.641+0000 7f4041816640 1 --2- 172.21.15.16:0/473594033 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f402006c2b0 0x7f402006e770 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f403c0d1520 tx=0x7f402c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.645+0000 7f4032ffd640 1 -- 172.21.15.16:0/473594033 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f40380590e0 con 0x7f403c0b0a80 2024-09-19T07:51:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:08.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.858+0000 7f404427b640 1 -- 172.21.15.16:0/473594033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f403c15f800 con 0x7f403c0b0a80 2024-09-19T07:51:08.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.859+0000 7f4032ffd640 1 -- 172.21.15.16:0/473594033 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f40380589e0 con 0x7f403c0b0a80 2024-09-19T07:51:08.859 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:51:08.860 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:51:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.861+0000 7f404427b640 1 -- 172.21.15.16:0/473594033 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f402006c2b0 msgr2=0x7f402006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.861+0000 7f404427b640 1 --2- 172.21.15.16:0/473594033 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f402006c2b0 0x7f402006e770 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f403c0d1520 tx=0x7f402c008040 comp rx=0 tx=0).stop 2024-09-19T07:51:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.861+0000 7f404427b640 1 -- 172.21.15.16:0/473594033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f403c0b0a80 msgr2=0x7f403c158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.861+0000 7f404427b640 1 --2- 172.21.15.16:0/473594033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f403c0b0a80 0x7f403c158240 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f403800c430 tx=0x7f403800c8f0 comp rx=0 tx=0).stop 2024-09-19T07:51:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.861+0000 7f404427b640 1 -- 172.21.15.16:0/473594033 shutdown_connections 2024-09-19T07:51:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.861+0000 7f404427b640 1 --2- 172.21.15.16:0/473594033 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f402006c2b0 0x7f402006e770 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.861+0000 7f404427b640 1 --2- 172.21.15.16:0/473594033 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f403c161d90 0x7f403c15dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.861+0000 7f404427b640 1 --2- 172.21.15.16:0/473594033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f403c1619b0 0x7f403c15da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.861+0000 7f404427b640 1 --2- 172.21.15.16:0/473594033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f403c0b0a80 0x7f403c158240 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.861+0000 7f404427b640 1 -- 172.21.15.16:0/473594033 >> 172.21.15.16:0/473594033 conn(0x7f403c09a9f0 msgr2=0x7f403c0ccb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.861+0000 7f404427b640 1 -- 172.21.15.16:0/473594033 shutdown_connections 2024-09-19T07:51:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:08.861+0000 7f404427b640 1 -- 172.21.15.16:0/473594033 wait complete. 2024-09-19T07:51:08.872 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:51:08.872 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:51:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:12.754 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:13.873 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:51:13.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.989+0000 7f04a2531640 1 Processor -- start 2024-09-19T07:51:13.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.989+0000 7f04a2531640 1 -- start start 2024-09-19T07:51:13.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.989+0000 7f04a2531640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f049c0d0bc0 0x7f049c0cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.989+0000 7f04a2531640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f049c0cdee0 con 0x7f049c15c2c0 2024-09-19T07:51:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.989+0000 7f04a2531640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f049c0ce1c0 con 0x7f049c15bee0 2024-09-19T07:51:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.989+0000 7f04a2531640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f049c0ceaf0 con 0x7f049c0d0bc0 2024-09-19T07:51:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.990+0000 7f04a0ace640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f049c15c2c0 0x7f049c0cf740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32830/0 (socket says 172.21.15.16:32830) 2024-09-19T07:51:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.990+0000 7f04a0ace640 1 -- 172.21.15.16:0/2906988129 learned_addr learned my addr 172.21.15.16:0/2906988129 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:51:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.990+0000 7f049b7fe640 1 --2- 172.21.15.16:0/2906988129 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f049c0d0bc0 0x7f049c0cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.990+0000 7f049affd640 1 -- 172.21.15.16:0/2906988129 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3441351530 0 0) 0x7f049c0cdee0 con 0x7f049c15c2c0 2024-09-19T07:51:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.990+0000 7f049affd640 1 -- 172.21.15.16:0/2906988129 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0484003520 con 0x7f049c15c2c0 2024-09-19T07:51:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.990+0000 7f049affd640 1 -- 172.21.15.16:0/2906988129 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 539256796 0 0) 0x7f049c0ce1c0 con 0x7f049c15bee0 2024-09-19T07:51:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.990+0000 7f049affd640 1 -- 172.21.15.16:0/2906988129 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f049c0cdee0 con 0x7f049c15bee0 2024-09-19T07:51:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.990+0000 7f049b7fe640 1 -- 172.21.15.16:0/2906988129 >> v1:172.21.15.146:6789/0 conn(0x7f049c15bee0 legacy=0x7f049c0d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.990+0000 7f049b7fe640 1 -- 172.21.15.16:0/2906988129 >> v1:172.21.15.16:6789/0 conn(0x7f049c15c2c0 legacy=0x7f049c0cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:13.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.990+0000 7f049affd640 1 -- 172.21.15.16:0/2906988129 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1070640406 0 0) 0x7f0484003520 con 0x7f049c15c2c0 2024-09-19T07:51:13.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.990+0000 7f049b7fe640 1 -- 172.21.15.16:0/2906988129 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049c0ce4a0 con 0x7f049c0d0bc0 2024-09-19T07:51:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.991+0000 7f049b7fe640 1 --2- 172.21.15.16:0/2906988129 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f049c0d0bc0 0x7f049c0cf200 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0490009800 tx=0x7f049002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7097ef3e4b59df4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.991+0000 7f049affd640 1 -- 172.21.15.16:0/2906988129 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0490037070 con 0x7f049c0d0bc0 2024-09-19T07:51:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.991+0000 7f049affd640 1 -- 172.21.15.16:0/2906988129 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f049002ac50 con 0x7f049c0d0bc0 2024-09-19T07:51:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.991+0000 7f049affd640 1 -- 172.21.15.16:0/2906988129 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04940019f0 con 0x7f049c0d0bc0 2024-09-19T07:51:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.992+0000 7f04a2531640 1 -- 172.21.15.16:0/2906988129 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f049c0d0bc0 msgr2=0x7f049c0cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.992+0000 7f04a2531640 1 --2- 172.21.15.16:0/2906988129 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f049c0d0bc0 0x7f049c0cf200 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0490009800 tx=0x7f049002a1c0 comp rx=0 tx=0).stop 2024-09-19T07:51:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.992+0000 7f04a2531640 1 -- 172.21.15.16:0/2906988129 shutdown_connections 2024-09-19T07:51:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.992+0000 7f04a2531640 1 --2- 172.21.15.16:0/2906988129 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f049c0d0bc0 0x7f049c0cf200 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.992+0000 7f04a2531640 1 -- 172.21.15.16:0/2906988129 >> 172.21.15.16:0/2906988129 conn(0x7f049c06bdf0 msgr2=0x7f049c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.992+0000 7f04a2531640 1 -- 172.21.15.16:0/2906988129 shutdown_connections 2024-09-19T07:51:13.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.992+0000 7f04a2531640 1 -- 172.21.15.16:0/2906988129 wait complete. 2024-09-19T07:51:13.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.993+0000 7f04a2531640 1 Processor -- start 2024-09-19T07:51:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.993+0000 7f04a2531640 1 -- start start 2024-09-19T07:51:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.994+0000 7f04a2531640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049c0d0bc0 0x7f049c151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.994+0000 7f04a2531640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f049c15bee0 0x7f049c0ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.994+0000 7f04a2531640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c15c2c0 0x7f049c0ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.994+0000 7f04a2531640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f049c06f050 con 0x7f049c15c2c0 2024-09-19T07:51:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.994+0000 7f04a2531640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f049c06eed0 con 0x7f049c0d0bc0 2024-09-19T07:51:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.994+0000 7f04a2531640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f049c166ea0 con 0x7f049c15bee0 2024-09-19T07:51:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.994+0000 7f04a0ace640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c15c2c0 0x7f049c0ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.994+0000 7f04a0ace640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c15c2c0 0x7f049c0ae800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34540/0 (socket says 172.21.15.16:34540) 2024-09-19T07:51:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.994+0000 7f04a0ace640 1 -- 172.21.15.16:0/3833986822 learned_addr learned my addr 172.21.15.16:0/3833986822 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:51:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.994+0000 7f049bfff640 1 --2- 172.21.15.16:0/3833986822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049c0d0bc0 0x7f049c151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.994+0000 7f049b7fe640 1 --2- 172.21.15.16:0/3833986822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f049c15bee0 0x7f049c0ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:13.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.995+0000 7f04a0ace640 1 -- 172.21.15.16:0/3833986822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f049c15bee0 msgr2=0x7f049c0ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:13.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.995+0000 7f04a0ace640 1 --2- 172.21.15.16:0/3833986822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f049c15bee0 0x7f049c0ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:13.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.995+0000 7f04a0ace640 1 -- 172.21.15.16:0/3833986822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049c0d0bc0 msgr2=0x7f049c151440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:13.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.995+0000 7f04a0ace640 1 --2- 172.21.15.16:0/3833986822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049c0d0bc0 0x7f049c151440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:13.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.995+0000 7f04a0ace640 1 -- 172.21.15.16:0/3833986822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049c156dc0 con 0x7f049c15c2c0 2024-09-19T07:51:13.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.995+0000 7f04a0ace640 1 --2- 172.21.15.16:0/3833986822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c15c2c0 0x7f049c0ae800 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f0494001350 tx=0x7f049400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:13.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.995+0000 7f0498ff9640 1 -- 172.21.15.16:0/3833986822 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0494017070 con 0x7f049c15c2c0 2024-09-19T07:51:13.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.995+0000 7f0498ff9640 1 -- 172.21.15.16:0/3833986822 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f049400ad40 con 0x7f049c15c2c0 2024-09-19T07:51:13.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.995+0000 7f0498ff9640 1 -- 172.21.15.16:0/3833986822 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0494012640 con 0x7f049c15c2c0 2024-09-19T07:51:13.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.995+0000 7f04a2531640 1 -- 172.21.15.16:0/3833986822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f049c1570a0 con 0x7f049c15c2c0 2024-09-19T07:51:13.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.995+0000 7f04a2531640 1 -- 172.21.15.16:0/3833986822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f049c157660 con 0x7f049c15c2c0 2024-09-19T07:51:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.996+0000 7f04a2531640 1 -- 172.21.15.16:0/3833986822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f049c0d04b0 con 0x7f049c15c2c0 2024-09-19T07:51:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.997+0000 7f0498ff9640 1 -- 172.21.15.16:0/3833986822 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f049401a070 con 0x7f049c15c2c0 2024-09-19T07:51:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.997+0000 7f0498ff9640 1 --2- 172.21.15.16:0/3833986822 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f048806c2b0 0x7f048806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.998+0000 7f0498ff9640 1 -- 172.21.15.16:0/3833986822 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f049408b630 con 0x7f049c15c2c0 2024-09-19T07:51:13.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.998+0000 7f049bfff640 1 --2- 172.21.15.16:0/3833986822 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f048806c2b0 0x7f048806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:13.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:13.998+0000 7f049bfff640 1 --2- 172.21.15.16:0/3833986822 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f048806c2b0 0x7f048806e770 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f048c0011e0 tx=0x7f048c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:14.002+0000 7f0498ff9640 1 -- 172.21.15.16:0/3833986822 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f04940595e0 con 0x7f049c15c2c0 2024-09-19T07:51:14.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:14.216+0000 7f04a2531640 1 -- 172.21.15.16:0/3833986822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f049c154d60 con 0x7f049c15c2c0 2024-09-19T07:51:14.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:14.216+0000 7f0498ff9640 1 -- 172.21.15.16:0/3833986822 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f049405d8e0 con 0x7f049c15c2c0 2024-09-19T07:51:14.217 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:51:14.217 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:51:14.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:14.218+0000 7f04a2531640 1 -- 172.21.15.16:0/3833986822 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f048806c2b0 msgr2=0x7f048806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:14.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:14.218+0000 7f04a2531640 1 --2- 172.21.15.16:0/3833986822 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f048806c2b0 0x7f048806e770 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f048c0011e0 tx=0x7f048c009040 comp rx=0 tx=0).stop 2024-09-19T07:51:14.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:14.218+0000 7f04a2531640 1 -- 172.21.15.16:0/3833986822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c15c2c0 msgr2=0x7f049c0ae800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:14.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:14.218+0000 7f04a2531640 1 --2- 172.21.15.16:0/3833986822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c15c2c0 0x7f049c0ae800 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f0494001350 tx=0x7f049400a430 comp rx=0 tx=0).stop 2024-09-19T07:51:14.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:14.218+0000 7f04a2531640 1 -- 172.21.15.16:0/3833986822 shutdown_connections 2024-09-19T07:51:14.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:14.218+0000 7f04a2531640 1 --2- 172.21.15.16:0/3833986822 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f048806c2b0 0x7f048806e770 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:14.218+0000 7f04a2531640 1 --2- 172.21.15.16:0/3833986822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c15c2c0 0x7f049c0ae800 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:14.218+0000 7f04a2531640 1 --2- 172.21.15.16:0/3833986822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f049c15bee0 0x7f049c0ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:14.218+0000 7f04a2531640 1 --2- 172.21.15.16:0/3833986822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f049c0d0bc0 0x7f049c151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:14.218+0000 7f04a2531640 1 -- 172.21.15.16:0/3833986822 >> 172.21.15.16:0/3833986822 conn(0x7f049c06bdf0 msgr2=0x7f049c0cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:14.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:14.218+0000 7f04a2531640 1 -- 172.21.15.16:0/3833986822 shutdown_connections 2024-09-19T07:51:14.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:14.218+0000 7f04a2531640 1 -- 172.21.15.16:0/3833986822 wait complete. 2024-09-19T07:51:14.229 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:51:14.230 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:51:14.756 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:16.758 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:19.230 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:51:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.345+0000 7fcc80c65640 1 Processor -- start 2024-09-19T07:51:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.345+0000 7fcc80c65640 1 -- start start 2024-09-19T07:51:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.346+0000 7fcc80c65640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcc7c06cd20 0x7fcc7c06d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.346+0000 7fcc80c65640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc7c161eb0 0x7fcc7c1622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.346+0000 7fcc80c65640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc7c06d640 con 0x7fcc7c143a20 2024-09-19T07:51:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.346+0000 7fcc80c65640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fcc7c162e50 con 0x7fcc7c161eb0 2024-09-19T07:51:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.346+0000 7fcc80c65640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fcc7c142150 con 0x7fcc7c06cd20 2024-09-19T07:51:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.346+0000 7fcc73fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc7c161eb0 0x7fcc7c1622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.346+0000 7fcc7ad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcc7c143a20 0x7fcc7c143eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32846/0 (socket says 172.21.15.16:32846) 2024-09-19T07:51:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.346+0000 7fcc7ad9d640 1 -- 172.21.15.16:0/3238263948 learned_addr learned my addr 172.21.15.16:0/3238263948 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:51:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.346+0000 7fcc7a59c640 1 --2- 172.21.15.16:0/3238263948 >> v2:172.21.15.146:3300/0 conn(0x7fcc7c06cd20 0x7fcc7c06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.346+0000 7fcc73fff640 1 -- 172.21.15.16:0/3238263948 >> v2:172.21.15.146:3300/0 conn(0x7fcc7c06cd20 msgr2=0x7fcc7c06d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.346+0000 7fcc73fff640 1 --2- 172.21.15.16:0/3238263948 >> v2:172.21.15.146:3300/0 conn(0x7fcc7c06cd20 0x7fcc7c06d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.346+0000 7fcc73fff640 1 -- 172.21.15.16:0/3238263948 >> v1:172.21.15.16:6789/0 conn(0x7fcc7c143a20 legacy=0x7fcc7c143eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.346+0000 7fcc73fff640 1 -- 172.21.15.16:0/3238263948 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7c162b70 con 0x7fcc7c161eb0 2024-09-19T07:51:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.346+0000 7fcc79d9b640 1 -- 172.21.15.16:0/3238263948 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1044031 0 0) 0x7fcc7c06d640 con 0x7fcc7c143a20 2024-09-19T07:51:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.346+0000 7fcc7a59c640 1 --2- 172.21.15.16:0/3238263948 >> v2:172.21.15.146:3300/0 conn(0x7fcc7c06cd20 0x7fcc7c06d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:51:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.347+0000 7fcc73fff640 1 --2- 172.21.15.16:0/3238263948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c161eb0 0x7fcc7c1622f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c070ff0 tx=0x7fcc6402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ef5a8fca884216 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.347+0000 7fcc79d9b640 1 -- 172.21.15.16:0/3238263948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64037070 con 0x7fcc7c161eb0 2024-09-19T07:51:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.347+0000 7fcc79d9b640 1 -- 172.21.15.16:0/3238263948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc64032440 con 0x7fcc7c161eb0 2024-09-19T07:51:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.347+0000 7fcc79d9b640 1 -- 172.21.15.16:0/3238263948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6402ae60 con 0x7fcc7c161eb0 2024-09-19T07:51:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.348+0000 7fcc80c65640 1 -- 172.21.15.16:0/3238263948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c161eb0 msgr2=0x7fcc7c1622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.348+0000 7fcc80c65640 1 --2- 172.21.15.16:0/3238263948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c161eb0 0x7fcc7c1622f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c070ff0 tx=0x7fcc6402a110 comp rx=0 tx=0).stop 2024-09-19T07:51:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.348+0000 7fcc80c65640 1 -- 172.21.15.16:0/3238263948 shutdown_connections 2024-09-19T07:51:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.348+0000 7fcc80c65640 1 --2- 172.21.15.16:0/3238263948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c161eb0 0x7fcc7c1622f0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.348+0000 7fcc80c65640 1 --2- 172.21.15.16:0/3238263948 >> v2:172.21.15.146:3300/0 conn(0x7fcc7c06cd20 0x7fcc7c06d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.348+0000 7fcc80c65640 1 -- 172.21.15.16:0/3238263948 >> 172.21.15.16:0/3238263948 conn(0x7fcc7c0b3ed0 msgr2=0x7fcc7c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.348+0000 7fcc80c65640 1 -- 172.21.15.16:0/3238263948 shutdown_connections 2024-09-19T07:51:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.348+0000 7fcc80c65640 1 -- 172.21.15.16:0/3238263948 wait complete. 2024-09-19T07:51:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.349+0000 7fcc80c65640 1 Processor -- start 2024-09-19T07:51:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.349+0000 7fcc80c65640 1 -- start start 2024-09-19T07:51:19.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.350+0000 7fcc80c65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c06cd20 0x7fcc7c15bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.350+0000 7fcc80c65640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc7c143a20 0x7fcc7c15b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.350+0000 7fcc80c65640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc7c161eb0 0x7fcc7c12caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.350+0000 7fcc80c65640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c162830 con 0x7fcc7c06cd20 2024-09-19T07:51:19.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.350+0000 7fcc80c65640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c141de0 con 0x7fcc7c143a20 2024-09-19T07:51:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.350+0000 7fcc80c65640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcc7c141600 con 0x7fcc7c161eb0 2024-09-19T07:51:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.350+0000 7fcc7a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c06cd20 0x7fcc7c15bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.350+0000 7fcc7a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c06cd20 0x7fcc7c15bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34554/0 (socket says 172.21.15.16:34554) 2024-09-19T07:51:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.350+0000 7fcc7a59c640 1 -- 172.21.15.16:0/3155602197 learned_addr learned my addr 172.21.15.16:0/3155602197 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:51:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.351+0000 7fcc73fff640 1 --2- 172.21.15.16:0/3155602197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc7c143a20 0x7fcc7c15b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.351+0000 7fcc7ad9d640 1 --2- 172.21.15.16:0/3155602197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc7c161eb0 0x7fcc7c12caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.351+0000 7fcc7a59c640 1 -- 172.21.15.16:0/3155602197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc7c161eb0 msgr2=0x7fcc7c12caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.351+0000 7fcc7a59c640 1 --2- 172.21.15.16:0/3155602197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc7c161eb0 0x7fcc7c12caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.351+0000 7fcc7a59c640 1 -- 172.21.15.16:0/3155602197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc7c143a20 msgr2=0x7fcc7c15b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.351+0000 7fcc7a59c640 1 --2- 172.21.15.16:0/3155602197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc7c143a20 0x7fcc7c15b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.351+0000 7fcc7a59c640 1 -- 172.21.15.16:0/3155602197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7c12bd70 con 0x7fcc7c06cd20 2024-09-19T07:51:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.351+0000 7fcc73fff640 1 --2- 172.21.15.16:0/3155602197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc7c143a20 0x7fcc7c15b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:51:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.351+0000 7fcc7a59c640 1 --2- 172.21.15.16:0/3155602197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c06cd20 0x7fcc7c15bac0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00c820 tx=0x7fcc6c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:19.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.351+0000 7fcc72ffd640 1 -- 172.21.15.16:0/3155602197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c015070 con 0x7fcc7c06cd20 2024-09-19T07:51:19.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.351+0000 7fcc72ffd640 1 -- 172.21.15.16:0/3155602197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc6c007a30 con 0x7fcc7c06cd20 2024-09-19T07:51:19.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.351+0000 7fcc72ffd640 1 -- 172.21.15.16:0/3155602197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c007de0 con 0x7fcc7c06cd20 2024-09-19T07:51:19.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.351+0000 7fcc80c65640 1 -- 172.21.15.16:0/3155602197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc7c12c010 con 0x7fcc7c06cd20 2024-09-19T07:51:19.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.351+0000 7fcc80c65640 1 -- 172.21.15.16:0/3155602197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc7c1e8280 con 0x7fcc7c06cd20 2024-09-19T07:51:19.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.352+0000 7fcc80c65640 1 -- 172.21.15.16:0/3155602197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc7c1622f0 con 0x7fcc7c06cd20 2024-09-19T07:51:19.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.353+0000 7fcc72ffd640 1 -- 172.21.15.16:0/3155602197 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcc6c018070 con 0x7fcc7c06cd20 2024-09-19T07:51:19.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.353+0000 7fcc72ffd640 1 --2- 172.21.15.16:0/3155602197 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcc5c06c2b0 0x7fcc5c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.353+0000 7fcc72ffd640 1 -- 172.21.15.16:0/3155602197 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7fcc6c08b550 con 0x7fcc7c06cd20 2024-09-19T07:51:19.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.353+0000 7fcc73fff640 1 --2- 172.21.15.16:0/3155602197 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcc5c06c2b0 0x7fcc5c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.354+0000 7fcc73fff640 1 --2- 172.21.15.16:0/3155602197 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcc5c06c2b0 0x7fcc5c06e770 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcc64034000 tx=0x7fcc6402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:19.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.358+0000 7fcc72ffd640 1 -- 172.21.15.16:0/3155602197 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fcc6c059ac0 con 0x7fcc7c06cd20 2024-09-19T07:51:19.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.572+0000 7fcc80c65640 1 -- 172.21.15.16:0/3155602197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fcc7c0690c0 con 0x7fcc7c06cd20 2024-09-19T07:51:19.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.572+0000 7fcc72ffd640 1 -- 172.21.15.16:0/3155602197 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fcc6c0593c0 con 0x7fcc7c06cd20 2024-09-19T07:51:19.573 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:51:19.573 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:51:19.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.574+0000 7fcc80c65640 1 -- 172.21.15.16:0/3155602197 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcc5c06c2b0 msgr2=0x7fcc5c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:19.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.574+0000 7fcc80c65640 1 --2- 172.21.15.16:0/3155602197 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcc5c06c2b0 0x7fcc5c06e770 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcc64034000 tx=0x7fcc6402e000 comp rx=0 tx=0).stop 2024-09-19T07:51:19.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.574+0000 7fcc80c65640 1 -- 172.21.15.16:0/3155602197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c06cd20 msgr2=0x7fcc7c15bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:19.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.574+0000 7fcc80c65640 1 --2- 172.21.15.16:0/3155602197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c06cd20 0x7fcc7c15bac0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00c820 tx=0x7fcc6c00cce0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.574+0000 7fcc80c65640 1 -- 172.21.15.16:0/3155602197 shutdown_connections 2024-09-19T07:51:19.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.574+0000 7fcc80c65640 1 --2- 172.21.15.16:0/3155602197 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcc5c06c2b0 0x7fcc5c06e770 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.574+0000 7fcc80c65640 1 --2- 172.21.15.16:0/3155602197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc7c161eb0 0x7fcc7c12caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.574+0000 7fcc80c65640 1 --2- 172.21.15.16:0/3155602197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc7c143a20 0x7fcc7c15b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.574+0000 7fcc80c65640 1 --2- 172.21.15.16:0/3155602197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc7c06cd20 0x7fcc7c15bac0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.574+0000 7fcc80c65640 1 -- 172.21.15.16:0/3155602197 >> 172.21.15.16:0/3155602197 conn(0x7fcc7c0b3ed0 msgr2=0x7fcc7c15fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.574+0000 7fcc80c65640 1 -- 172.21.15.16:0/3155602197 shutdown_connections 2024-09-19T07:51:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:19.575+0000 7fcc80c65640 1 -- 172.21.15.16:0/3155602197 wait complete. 2024-09-19T07:51:19.586 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:51:19.586 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:51:20.762 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:24.587 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:51:24.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.701+0000 7f641100a640 1 Processor -- start 2024-09-19T07:51:24.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.701+0000 7f641100a640 1 -- start start 2024-09-19T07:51:24.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.702+0000 7f641100a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f640c161eb0 0x7f640c1622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:24.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.702+0000 7f641100a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f640c162b70 con 0x7f640c06cd20 2024-09-19T07:51:24.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.702+0000 7f641100a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f640c141b00 con 0x7f640c1425e0 2024-09-19T07:51:24.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.702+0000 7f641100a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f640c162e50 con 0x7f640c161eb0 2024-09-19T07:51:24.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.702+0000 7f640a59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f640c161eb0 0x7f640c1622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:24.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.702+0000 7f640ad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f640c06cd20 0x7f640c06d100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34604/0 (socket says 172.21.15.16:34604) 2024-09-19T07:51:24.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.702+0000 7f640ad9d640 1 -- 172.21.15.16:0/3331986273 learned_addr learned my addr 172.21.15.16:0/3331986273 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:51:24.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.702+0000 7f640a59c640 1 -- 172.21.15.16:0/3331986273 >> v1:172.21.15.146:6789/0 conn(0x7f640c1425e0 legacy=0x7f640c142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:24.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.702+0000 7f640a59c640 1 -- 172.21.15.16:0/3331986273 >> v1:172.21.15.16:6789/0 conn(0x7f640c06cd20 legacy=0x7f640c06d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:24.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.702+0000 7f640a59c640 1 -- 172.21.15.16:0/3331986273 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640c141de0 con 0x7f640c161eb0 2024-09-19T07:51:24.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.703+0000 7f640a59c640 1 --2- 172.21.15.16:0/3331986273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c161eb0 0x7f640c1622f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f64000097a0 tx=0x7f640002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c44ecc03114b8dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:24.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.703+0000 7f6409d9b640 1 -- 172.21.15.16:0/3331986273 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400037070 con 0x7f640c161eb0 2024-09-19T07:51:24.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.703+0000 7f6409d9b640 1 -- 172.21.15.16:0/3331986273 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6400032440 con 0x7f640c161eb0 2024-09-19T07:51:24.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.703+0000 7f6409d9b640 1 -- 172.21.15.16:0/3331986273 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400032810 con 0x7f640c161eb0 2024-09-19T07:51:24.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.704+0000 7f641100a640 1 -- 172.21.15.16:0/3331986273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c161eb0 msgr2=0x7f640c1622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:24.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.704+0000 7f641100a640 1 --2- 172.21.15.16:0/3331986273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c161eb0 0x7f640c1622f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f64000097a0 tx=0x7f640002a1a0 comp rx=0 tx=0).stop 2024-09-19T07:51:24.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.704+0000 7f641100a640 1 -- 172.21.15.16:0/3331986273 shutdown_connections 2024-09-19T07:51:24.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.704+0000 7f641100a640 1 --2- 172.21.15.16:0/3331986273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c161eb0 0x7f640c1622f0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.704+0000 7f641100a640 1 -- 172.21.15.16:0/3331986273 >> 172.21.15.16:0/3331986273 conn(0x7f640c0b3ed0 msgr2=0x7f640c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.704+0000 7f641100a640 1 -- 172.21.15.16:0/3331986273 shutdown_connections 2024-09-19T07:51:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.704+0000 7f641100a640 1 -- 172.21.15.16:0/3331986273 wait complete. 2024-09-19T07:51:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.705+0000 7f641100a640 1 Processor -- start 2024-09-19T07:51:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.705+0000 7f641100a640 1 -- start start 2024-09-19T07:51:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.706+0000 7f641100a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f640c06cd20 0x7f640c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.706+0000 7f641100a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c1425e0 0x7f640c0b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.706+0000 7f641100a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f640c161eb0 0x7f640c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.706+0000 7f641100a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f640c141600 con 0x7f640c1425e0 2024-09-19T07:51:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.706+0000 7f641100a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f640c142150 con 0x7f640c161eb0 2024-09-19T07:51:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.706+0000 7f641100a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f640c143180 con 0x7f640c06cd20 2024-09-19T07:51:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.706+0000 7f640a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c1425e0 0x7f640c0b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.706+0000 7f640a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c1425e0 0x7f640c0b7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38100/0 (socket says 172.21.15.16:38100) 2024-09-19T07:51:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.706+0000 7f640a59c640 1 -- 172.21.15.16:0/2250421992 learned_addr learned my addr 172.21.15.16:0/2250421992 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:51:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.706+0000 7f640b59e640 1 --2- 172.21.15.16:0/2250421992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f640c161eb0 0x7f640c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.706+0000 7f640ad9d640 1 --2- 172.21.15.16:0/2250421992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f640c06cd20 0x7f640c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.706+0000 7f640a59c640 1 -- 172.21.15.16:0/2250421992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f640c06cd20 msgr2=0x7f640c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.706+0000 7f640a59c640 1 --2- 172.21.15.16:0/2250421992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f640c06cd20 0x7f640c0b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.706+0000 7f640a59c640 1 -- 172.21.15.16:0/2250421992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f640c161eb0 msgr2=0x7f640c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.706+0000 7f640a59c640 1 --2- 172.21.15.16:0/2250421992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f640c161eb0 0x7f640c0cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.707+0000 7f640a59c640 1 -- 172.21.15.16:0/2250421992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640c0ce320 con 0x7f640c1425e0 2024-09-19T07:51:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.707+0000 7f640b59e640 1 --2- 172.21.15.16:0/2250421992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f640c161eb0 0x7f640c0cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:51:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.707+0000 7f640ad9d640 1 --2- 172.21.15.16:0/2250421992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f640c06cd20 0x7f640c0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:51:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.707+0000 7f640a59c640 1 --2- 172.21.15.16:0/2250421992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c1425e0 0x7f640c0b7a10 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6400035040 tx=0x7f64000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.707+0000 7f63f37fe640 1 -- 172.21.15.16:0/2250421992 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400037040 con 0x7f640c1425e0 2024-09-19T07:51:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.707+0000 7f63f37fe640 1 -- 172.21.15.16:0/2250421992 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6400030770 con 0x7f640c1425e0 2024-09-19T07:51:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.707+0000 7f641100a640 1 -- 172.21.15.16:0/2250421992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640c0cf5b0 con 0x7f640c1425e0 2024-09-19T07:51:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.707+0000 7f63f37fe640 1 -- 172.21.15.16:0/2250421992 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400030b80 con 0x7f640c1425e0 2024-09-19T07:51:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.707+0000 7f641100a640 1 -- 172.21.15.16:0/2250421992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640c0ce870 con 0x7f640c1425e0 2024-09-19T07:51:24.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.708+0000 7f641100a640 1 -- 172.21.15.16:0/2250421992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f640c143510 con 0x7f640c1425e0 2024-09-19T07:51:24.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.709+0000 7f63f37fe640 1 -- 172.21.15.16:0/2250421992 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6400044020 con 0x7f640c1425e0 2024-09-19T07:51:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.709+0000 7f63f37fe640 1 --2- 172.21.15.16:0/2250421992 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f63f806c2b0 0x7f63f806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.709+0000 7f63f37fe640 1 -- 172.21.15.16:0/2250421992 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f64000abbd0 con 0x7f640c1425e0 2024-09-19T07:51:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.710+0000 7f640ad9d640 1 --2- 172.21.15.16:0/2250421992 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f63f806c2b0 0x7f63f806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.710+0000 7f640ad9d640 1 --2- 172.21.15.16:0/2250421992 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f63f806c2b0 0x7f63f806e770 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f63fc007fd0 tx=0x7f63fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.713+0000 7f63f37fe640 1 -- 172.21.15.16:0/2250421992 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6400032e20 con 0x7f640c1425e0 2024-09-19T07:51:24.766 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:24.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.932+0000 7f641100a640 1 -- 172.21.15.16:0/2250421992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f640c0690c0 con 0x7f640c1425e0 2024-09-19T07:51:24.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.932+0000 7f63f37fe640 1 -- 172.21.15.16:0/2250421992 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f640007a1d0 con 0x7f640c1425e0 2024-09-19T07:51:24.933 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:51:24.933 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:51:24.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.934+0000 7f641100a640 1 -- 172.21.15.16:0/2250421992 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f63f806c2b0 msgr2=0x7f63f806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:24.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.934+0000 7f641100a640 1 --2- 172.21.15.16:0/2250421992 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f63f806c2b0 0x7f63f806e770 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f63fc007fd0 tx=0x7f63fc009040 comp rx=0 tx=0).stop 2024-09-19T07:51:24.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.934+0000 7f641100a640 1 -- 172.21.15.16:0/2250421992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c1425e0 msgr2=0x7f640c0b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:24.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.934+0000 7f641100a640 1 --2- 172.21.15.16:0/2250421992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c1425e0 0x7f640c0b7a10 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6400035040 tx=0x7f64000046c0 comp rx=0 tx=0).stop 2024-09-19T07:51:24.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.934+0000 7f641100a640 1 -- 172.21.15.16:0/2250421992 shutdown_connections 2024-09-19T07:51:24.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.934+0000 7f641100a640 1 --2- 172.21.15.16:0/2250421992 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f63f806c2b0 0x7f63f806e770 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:24.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.934+0000 7f641100a640 1 --2- 172.21.15.16:0/2250421992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f640c161eb0 0x7f640c0cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:24.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.934+0000 7f641100a640 1 --2- 172.21.15.16:0/2250421992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c1425e0 0x7f640c0b7a10 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:24.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.934+0000 7f641100a640 1 --2- 172.21.15.16:0/2250421992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f640c06cd20 0x7f640c0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:24.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.934+0000 7f641100a640 1 -- 172.21.15.16:0/2250421992 >> 172.21.15.16:0/2250421992 conn(0x7f640c0b3ed0 msgr2=0x7f640c06dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:24.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.935+0000 7f641100a640 1 -- 172.21.15.16:0/2250421992 shutdown_connections 2024-09-19T07:51:24.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:24.935+0000 7f641100a640 1 -- 172.21.15.16:0/2250421992 wait complete. 2024-09-19T07:51:24.946 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:51:24.946 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:51:26.768 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:29.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:51:30.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.061+0000 7f2ced6f1640 1 Processor -- start 2024-09-19T07:51:30.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.061+0000 7f2ced6f1640 1 -- start start 2024-09-19T07:51:30.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.062+0000 7f2ced6f1640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2ce80b0a80 0x7f2ce80cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:30.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.062+0000 7f2ced6f1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ce806c990 con 0x7f2ce81619b0 2024-09-19T07:51:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.062+0000 7f2ced6f1640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ce80cec10 con 0x7f2ce8161d90 2024-09-19T07:51:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.062+0000 7f2ced6f1640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2ce80ce930 con 0x7f2ce80b0a80 2024-09-19T07:51:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.062+0000 7f2ce6ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2ce81619b0 0x7f2ce80d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34610/0 (socket says 172.21.15.16:34610) 2024-09-19T07:51:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.062+0000 7f2ce6ffd640 1 -- 172.21.15.16:0/3058990537 learned_addr learned my addr 172.21.15.16:0/3058990537 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:51:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.063+0000 7f2ce67fc640 1 --2- 172.21.15.16:0/3058990537 >> v2:172.21.15.146:3300/0 conn(0x7f2ce80b0a80 0x7f2ce80cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.063+0000 7f2ce5ffb640 1 -- 172.21.15.16:0/3058990537 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3717716335 0 0) 0x7f2ce806c990 con 0x7f2ce81619b0 2024-09-19T07:51:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.063+0000 7f2ce5ffb640 1 -- 172.21.15.16:0/3058990537 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2cd0003520 con 0x7f2ce81619b0 2024-09-19T07:51:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.063+0000 7f2ce5ffb640 1 -- 172.21.15.16:0/3058990537 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1269060348 0 0) 0x7f2ce80cec10 con 0x7f2ce8161d90 2024-09-19T07:51:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.063+0000 7f2ce5ffb640 1 -- 172.21.15.16:0/3058990537 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ce806c990 con 0x7f2ce8161d90 2024-09-19T07:51:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.063+0000 7f2ce67fc640 1 -- 172.21.15.16:0/3058990537 >> v1:172.21.15.146:6789/0 conn(0x7f2ce8161d90 legacy=0x7f2ce80cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.063+0000 7f2ce5ffb640 1 -- 172.21.15.16:0/3058990537 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4213599499 0 0) 0x7f2cd0003520 con 0x7f2ce81619b0 2024-09-19T07:51:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.063+0000 7f2ce67fc640 1 -- 172.21.15.16:0/3058990537 >> v1:172.21.15.16:6789/0 conn(0x7f2ce81619b0 legacy=0x7f2ce80d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.063+0000 7f2ce67fc640 1 -- 172.21.15.16:0/3058990537 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce80b0540 con 0x7f2ce80b0a80 2024-09-19T07:51:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.064+0000 7f2ce67fc640 1 --2- 172.21.15.16:0/3058990537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ce80b0a80 0x7f2ce80cbcd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0097a0 tx=0x7f2cdc02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13c474dbbe96b186 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.064+0000 7f2ce5ffb640 1 -- 172.21.15.16:0/3058990537 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc037070 con 0x7f2ce80b0a80 2024-09-19T07:51:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.064+0000 7f2ce5ffb640 1 -- 172.21.15.16:0/3058990537 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cdc02ac50 con 0x7f2ce80b0a80 2024-09-19T07:51:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.064+0000 7f2ce5ffb640 1 -- 172.21.15.16:0/3058990537 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8001670 con 0x7f2ce80b0a80 2024-09-19T07:51:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.065+0000 7f2ced6f1640 1 -- 172.21.15.16:0/3058990537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ce80b0a80 msgr2=0x7f2ce80cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.065+0000 7f2ced6f1640 1 --2- 172.21.15.16:0/3058990537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ce80b0a80 0x7f2ce80cbcd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0097a0 tx=0x7f2cdc02a1c0 comp rx=0 tx=0).stop 2024-09-19T07:51:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.065+0000 7f2ced6f1640 1 -- 172.21.15.16:0/3058990537 shutdown_connections 2024-09-19T07:51:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.065+0000 7f2ced6f1640 1 --2- 172.21.15.16:0/3058990537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ce80b0a80 0x7f2ce80cbcd0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.065+0000 7f2ced6f1640 1 -- 172.21.15.16:0/3058990537 >> 172.21.15.16:0/3058990537 conn(0x7f2ce809a9f0 msgr2=0x7f2ce80b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.065+0000 7f2ced6f1640 1 -- 172.21.15.16:0/3058990537 shutdown_connections 2024-09-19T07:51:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.065+0000 7f2ced6f1640 1 -- 172.21.15.16:0/3058990537 wait complete. 2024-09-19T07:51:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.066+0000 7f2ced6f1640 1 Processor -- start 2024-09-19T07:51:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.066+0000 7f2ced6f1640 1 -- start start 2024-09-19T07:51:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.067+0000 7f2ced6f1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ce80b0a80 0x7f2ce815f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.067+0000 7f2ced6f1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ce81619b0 0x7f2ce8148260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.067+0000 7f2ced6f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ce8161d90 0x7f2ce81487a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.067+0000 7f2ced6f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce80cd330 con 0x7f2ce8161d90 2024-09-19T07:51:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.067+0000 7f2ced6f1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce806c620 con 0x7f2ce80b0a80 2024-09-19T07:51:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.067+0000 7f2ced6f1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce80cd4b0 con 0x7f2ce81619b0 2024-09-19T07:51:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.067+0000 7f2ce77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ce8161d90 0x7f2ce81487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.067+0000 7f2ce77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ce8161d90 0x7f2ce81487a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38104/0 (socket says 172.21.15.16:38104) 2024-09-19T07:51:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.067+0000 7f2ce77fe640 1 -- 172.21.15.16:0/1451590412 learned_addr learned my addr 172.21.15.16:0/1451590412 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:51:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.067+0000 7f2ce67fc640 1 --2- 172.21.15.16:0/1451590412 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ce81619b0 0x7f2ce8148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.067+0000 7f2ce6ffd640 1 --2- 172.21.15.16:0/1451590412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ce80b0a80 0x7f2ce815f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.067+0000 7f2ce77fe640 1 -- 172.21.15.16:0/1451590412 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ce81619b0 msgr2=0x7f2ce8148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.067+0000 7f2ce77fe640 1 --2- 172.21.15.16:0/1451590412 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ce81619b0 0x7f2ce8148260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.067+0000 7f2ce77fe640 1 -- 172.21.15.16:0/1451590412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ce80b0a80 msgr2=0x7f2ce815f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.067+0000 7f2ce77fe640 1 --2- 172.21.15.16:0/1451590412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ce80b0a80 0x7f2ce815f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.067+0000 7f2ce77fe640 1 -- 172.21.15.16:0/1451590412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce815f800 con 0x7f2ce8161d90 2024-09-19T07:51:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.068+0000 7f2ce77fe640 1 --2- 172.21.15.16:0/1451590412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ce8161d90 0x7f2ce81487a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000c6e0 tx=0x7f2ce000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.068+0000 7f2ccf7fe640 1 -- 172.21.15.16:0/1451590412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0003b70 con 0x7f2ce8161d90 2024-09-19T07:51:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.068+0000 7f2ccf7fe640 1 -- 172.21.15.16:0/1451590412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ce0003d10 con 0x7f2ce8161d90 2024-09-19T07:51:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.068+0000 7f2ccf7fe640 1 -- 172.21.15.16:0/1451590412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce00052d0 con 0x7f2ce8161d90 2024-09-19T07:51:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.068+0000 7f2ced6f1640 1 -- 172.21.15.16:0/1451590412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce80c69e0 con 0x7f2ce8161d90 2024-09-19T07:51:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.068+0000 7f2ced6f1640 1 -- 172.21.15.16:0/1451590412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce80c6fa0 con 0x7f2ce8161d90 2024-09-19T07:51:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.069+0000 7f2ced6f1640 1 -- 172.21.15.16:0/1451590412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce81592b0 con 0x7f2ce8161d90 2024-09-19T07:51:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.069+0000 7f2ccf7fe640 1 -- 172.21.15.16:0/1451590412 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2ce0005470 con 0x7f2ce8161d90 2024-09-19T07:51:30.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.070+0000 7f2ccf7fe640 1 --2- 172.21.15.16:0/1451590412 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2cd406c2b0 0x7f2cd406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.070+0000 7f2ccf7fe640 1 -- 172.21.15.16:0/1451590412 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f2ce008b470 con 0x7f2ce8161d90 2024-09-19T07:51:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.070+0000 7f2ce6ffd640 1 --2- 172.21.15.16:0/1451590412 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2cd406c2b0 0x7f2cd406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.071+0000 7f2ce6ffd640 1 --2- 172.21.15.16:0/1451590412 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2cd406c2b0 0x7f2cd406e770 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8001340 tx=0x7f2cd8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.074+0000 7f2ccf7fe640 1 -- 172.21.15.16:0/1451590412 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2ce81592b0 con 0x7f2ce8161d90 2024-09-19T07:51:30.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.287+0000 7f2ced6f1640 1 -- 172.21.15.16:0/1451590412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2ce80d0440 con 0x7f2ce8161d90 2024-09-19T07:51:30.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.287+0000 7f2ccf7fe640 1 -- 172.21.15.16:0/1451590412 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f2ce0059a70 con 0x7f2ce8161d90 2024-09-19T07:51:30.288 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:51:30.288 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:51:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.289+0000 7f2ced6f1640 1 -- 172.21.15.16:0/1451590412 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2cd406c2b0 msgr2=0x7f2cd406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.289+0000 7f2ced6f1640 1 --2- 172.21.15.16:0/1451590412 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2cd406c2b0 0x7f2cd406e770 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8001340 tx=0x7f2cd8009040 comp rx=0 tx=0).stop 2024-09-19T07:51:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.289+0000 7f2ced6f1640 1 -- 172.21.15.16:0/1451590412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ce8161d90 msgr2=0x7f2ce81487a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.289+0000 7f2ced6f1640 1 --2- 172.21.15.16:0/1451590412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ce8161d90 0x7f2ce81487a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000c6e0 tx=0x7f2ce000cba0 comp rx=0 tx=0).stop 2024-09-19T07:51:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.289+0000 7f2ced6f1640 1 -- 172.21.15.16:0/1451590412 shutdown_connections 2024-09-19T07:51:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.289+0000 7f2ced6f1640 1 --2- 172.21.15.16:0/1451590412 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2cd406c2b0 0x7f2cd406e770 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.289+0000 7f2ced6f1640 1 --2- 172.21.15.16:0/1451590412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ce8161d90 0x7f2ce81487a0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.289+0000 7f2ced6f1640 1 --2- 172.21.15.16:0/1451590412 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ce81619b0 0x7f2ce8148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.289+0000 7f2ced6f1640 1 --2- 172.21.15.16:0/1451590412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ce80b0a80 0x7f2ce815f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.289+0000 7f2ced6f1640 1 -- 172.21.15.16:0/1451590412 >> 172.21.15.16:0/1451590412 conn(0x7f2ce809a9f0 msgr2=0x7f2ce80cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.289+0000 7f2ced6f1640 1 -- 172.21.15.16:0/1451590412 shutdown_connections 2024-09-19T07:51:30.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:30.289+0000 7f2ced6f1640 1 -- 172.21.15.16:0/1451590412 wait complete. 2024-09-19T07:51:30.300 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:51:30.300 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:51:30.772 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:32.774 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:35.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:51:35.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.416+0000 7f5479b78640 1 Processor -- start 2024-09-19T07:51:35.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.416+0000 7f5479b78640 1 -- start start 2024-09-19T07:51:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.417+0000 7f5479b78640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f547415bee0 0x7f54740d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.417+0000 7f5479b78640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f547415c2c0 0x7f54740d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.417+0000 7f5479b78640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f54740cf200 0x7f54740cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.417+0000 7f5479b78640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f54740ceaf0 con 0x7f547415c2c0 2024-09-19T07:51:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.417+0000 7f5479b78640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f54740cdee0 con 0x7f54740cf200 2024-09-19T07:51:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.417+0000 7f5479b78640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f54740ce1c0 con 0x7f547415bee0 2024-09-19T07:51:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.417+0000 7f5472ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f547415c2c0 0x7f54740d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.417+0000 7f5472ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f547415c2c0 0x7f54740d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54246/0 (socket says 172.21.15.16:54246) 2024-09-19T07:51:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.417+0000 7f5472ffd640 1 -- 172.21.15.16:0/2811634730 learned_addr learned my addr 172.21.15.16:0/2811634730 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:51:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.417+0000 7f54737fe640 1 --2- 172.21.15.16:0/2811634730 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f547415bee0 0x7f54740d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.417+0000 7f5473fff640 1 --2- 172.21.15.16:0/2811634730 >> v2:172.21.15.146:3300/0 conn(0x7f54740cf200 0x7f54740cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.417+0000 7f5472ffd640 1 -- 172.21.15.16:0/2811634730 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f547415bee0 msgr2=0x7f54740d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.417+0000 7f5472ffd640 1 --2- 172.21.15.16:0/2811634730 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f547415bee0 0x7f54740d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.417+0000 7f5472ffd640 1 -- 172.21.15.16:0/2811634730 >> v2:172.21.15.146:3300/0 conn(0x7f54740cf200 msgr2=0x7f54740cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.417+0000 7f5472ffd640 1 --2- 172.21.15.16:0/2811634730 >> v2:172.21.15.146:3300/0 conn(0x7f54740cf200 0x7f54740cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.417+0000 7f5472ffd640 1 -- 172.21.15.16:0/2811634730 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54740ce4a0 con 0x7f547415c2c0 2024-09-19T07:51:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.418+0000 7f5472ffd640 1 --2- 172.21.15.16:0/2811634730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f547415c2c0 0x7f54740d09f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f54740af910 tx=0x7f546802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1476de70be30764f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:35.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.418+0000 7f54727fc640 1 -- 172.21.15.16:0/2811634730 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468037070 con 0x7f547415c2c0 2024-09-19T07:51:35.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.418+0000 7f54727fc640 1 -- 172.21.15.16:0/2811634730 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5468032440 con 0x7f547415c2c0 2024-09-19T07:51:35.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.418+0000 7f54727fc640 1 -- 172.21.15.16:0/2811634730 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468032810 con 0x7f547415c2c0 2024-09-19T07:51:35.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.419+0000 7f5479b78640 1 -- 172.21.15.16:0/2811634730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f547415c2c0 msgr2=0x7f54740d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:35.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.419+0000 7f5479b78640 1 --2- 172.21.15.16:0/2811634730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f547415c2c0 0x7f54740d09f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f54740af910 tx=0x7f546802a1e0 comp rx=0 tx=0).stop 2024-09-19T07:51:35.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.419+0000 7f5479b78640 1 -- 172.21.15.16:0/2811634730 shutdown_connections 2024-09-19T07:51:35.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.419+0000 7f5479b78640 1 --2- 172.21.15.16:0/2811634730 >> v2:172.21.15.146:3300/0 conn(0x7f54740cf200 0x7f54740cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:35.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.419+0000 7f5479b78640 1 --2- 172.21.15.16:0/2811634730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f547415c2c0 0x7f54740d09f0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:35.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.419+0000 7f5479b78640 1 --2- 172.21.15.16:0/2811634730 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f547415bee0 0x7f54740d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:35.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.419+0000 7f5479b78640 1 -- 172.21.15.16:0/2811634730 >> 172.21.15.16:0/2811634730 conn(0x7f547406bdf0 msgr2=0x7f547406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:35.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.419+0000 7f5479b78640 1 -- 172.21.15.16:0/2811634730 shutdown_connections 2024-09-19T07:51:35.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.419+0000 7f5479b78640 1 -- 172.21.15.16:0/2811634730 wait complete. 2024-09-19T07:51:35.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.420+0000 7f5479b78640 1 Processor -- start 2024-09-19T07:51:35.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.420+0000 7f5479b78640 1 -- start start 2024-09-19T07:51:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.421+0000 7f5479b78640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54740cf200 0x7f54741f4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.421+0000 7f5479b78640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f547415bee0 0x7f54740c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.421+0000 7f5479b78640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f547415c2c0 0x7f54740c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.421+0000 7f5479b78640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f547415fd40 con 0x7f54740cf200 2024-09-19T07:51:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.421+0000 7f5479b78640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f54740cfbf0 con 0x7f547415c2c0 2024-09-19T07:51:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.421+0000 7f5479b78640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f547415fec0 con 0x7f547415bee0 2024-09-19T07:51:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.421+0000 7f54737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54740cf200 0x7f54741f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.421+0000 7f54737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54740cf200 0x7f54741f4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54252/0 (socket says 172.21.15.16:54252) 2024-09-19T07:51:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.421+0000 7f54737fe640 1 -- 172.21.15.16:0/1170775043 learned_addr learned my addr 172.21.15.16:0/1170775043 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:51:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.422+0000 7f54737fe640 1 -- 172.21.15.16:0/1170775043 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f547415bee0 msgr2=0x7f54740c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.422+0000 7f54737fe640 1 --2- 172.21.15.16:0/1170775043 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f547415bee0 0x7f54740c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.422+0000 7f54737fe640 1 -- 172.21.15.16:0/1170775043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f547415c2c0 msgr2=0x7f54740c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.422+0000 7f54737fe640 1 --2- 172.21.15.16:0/1170775043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f547415c2c0 0x7f54740c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.422+0000 7f54737fe640 1 -- 172.21.15.16:0/1170775043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54740c5630 con 0x7f54740cf200 2024-09-19T07:51:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.422+0000 7f54737fe640 1 --2- 172.21.15.16:0/1170775043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54740cf200 0x7f54741f4860 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f546400e9e0 tx=0x7f546400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.422+0000 7f545bfff640 1 -- 172.21.15.16:0/1170775043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464017070 con 0x7f54740cf200 2024-09-19T07:51:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.422+0000 7f545bfff640 1 -- 172.21.15.16:0/1170775043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f546400c950 con 0x7f54740cf200 2024-09-19T07:51:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.422+0000 7f5479b78640 1 -- 172.21.15.16:0/1170775043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54740c58d0 con 0x7f54740cf200 2024-09-19T07:51:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.422+0000 7f545bfff640 1 -- 172.21.15.16:0/1170775043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546400cd40 con 0x7f54740cf200 2024-09-19T07:51:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.422+0000 7f5479b78640 1 -- 172.21.15.16:0/1170775043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54740c5e90 con 0x7f54740cf200 2024-09-19T07:51:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.423+0000 7f5479b78640 1 -- 172.21.15.16:0/1170775043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54740cf6b0 con 0x7f54740cf200 2024-09-19T07:51:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.423+0000 7f545bfff640 1 -- 172.21.15.16:0/1170775043 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f546401a070 con 0x7f54740cf200 2024-09-19T07:51:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.424+0000 7f545bfff640 1 --2- 172.21.15.16:0/1170775043 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f546006c2b0 0x7f546006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.424+0000 7f545bfff640 1 -- 172.21.15.16:0/1170775043 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f546408b5a0 con 0x7f54740cf200 2024-09-19T07:51:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.424+0000 7f5472ffd640 1 --2- 172.21.15.16:0/1170775043 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f546006c2b0 0x7f546006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.425+0000 7f5472ffd640 1 --2- 172.21.15.16:0/1170775043 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f546006c2b0 0x7f546006e770 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5468034000 tx=0x7f546802c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:35.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.428+0000 7f545bfff640 1 -- 172.21.15.16:0/1170775043 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5464059b10 con 0x7f54740cf200 2024-09-19T07:51:35.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.644+0000 7f5479b78640 1 -- 172.21.15.16:0/1170775043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f54740b4d80 con 0x7f54740cf200 2024-09-19T07:51:35.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.644+0000 7f545bfff640 1 -- 172.21.15.16:0/1170775043 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f5464059410 con 0x7f54740cf200 2024-09-19T07:51:35.645 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:51:35.645 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:51:35.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.647+0000 7f5479b78640 1 -- 172.21.15.16:0/1170775043 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f546006c2b0 msgr2=0x7f546006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:35.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.647+0000 7f5479b78640 1 --2- 172.21.15.16:0/1170775043 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f546006c2b0 0x7f546006e770 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5468034000 tx=0x7f546802c000 comp rx=0 tx=0).stop 2024-09-19T07:51:35.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.647+0000 7f5479b78640 1 -- 172.21.15.16:0/1170775043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54740cf200 msgr2=0x7f54741f4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:35.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.647+0000 7f5479b78640 1 --2- 172.21.15.16:0/1170775043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54740cf200 0x7f54741f4860 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f546400e9e0 tx=0x7f546400eea0 comp rx=0 tx=0).stop 2024-09-19T07:51:35.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.647+0000 7f5479b78640 1 -- 172.21.15.16:0/1170775043 shutdown_connections 2024-09-19T07:51:35.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.647+0000 7f5479b78640 1 --2- 172.21.15.16:0/1170775043 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f546006c2b0 0x7f546006e770 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.647+0000 7f5479b78640 1 --2- 172.21.15.16:0/1170775043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f547415c2c0 0x7f54740c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.647+0000 7f5479b78640 1 --2- 172.21.15.16:0/1170775043 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f547415bee0 0x7f54740c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.647+0000 7f5479b78640 1 --2- 172.21.15.16:0/1170775043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54740cf200 0x7f54741f4860 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.647+0000 7f5479b78640 1 -- 172.21.15.16:0/1170775043 >> 172.21.15.16:0/1170775043 conn(0x7f547406bdf0 msgr2=0x7f547406c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.647+0000 7f5479b78640 1 -- 172.21.15.16:0/1170775043 shutdown_connections 2024-09-19T07:51:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:35.647+0000 7f5479b78640 1 -- 172.21.15.16:0/1170775043 wait complete. 2024-09-19T07:51:35.658 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:51:35.658 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:51:36.778 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:38.617 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:51:38.705 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:51:38.781 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:40.659 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:51:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.774+0000 7fae664d7640 1 Processor -- start 2024-09-19T07:51:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.774+0000 7fae664d7640 1 -- start start 2024-09-19T07:51:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae664d7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fae6015b120 0x7fae6006f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae664d7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae6015b500 con 0x7fae6006dec0 2024-09-19T07:51:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae664d7640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae6006e940 con 0x7fae600d0bc0 2024-09-19T07:51:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae664d7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fae6006ec20 con 0x7fae6015b120 2024-09-19T07:51:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae5f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fae6006dec0 0x7fae600d04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50424/0 (socket says 172.21.15.16:50424) 2024-09-19T07:51:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae5f7fe640 1 -- 172.21.15.16:0/3084138275 learned_addr learned my addr 172.21.15.16:0/3084138275 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:51:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae5ffff640 1 --2- 172.21.15.16:0/3084138275 >> v2:172.21.15.146:3300/0 conn(0x7fae6015b120 0x7fae6006f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae5effd640 1 -- 172.21.15.16:0/3084138275 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1811038084 0 0) 0x7fae6015b500 con 0x7fae6006dec0 2024-09-19T07:51:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae5effd640 1 -- 172.21.15.16:0/3084138275 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae48003520 con 0x7fae6006dec0 2024-09-19T07:51:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae5effd640 1 -- 172.21.15.16:0/3084138275 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2185382373 0 0) 0x7fae48003520 con 0x7fae6006dec0 2024-09-19T07:51:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae5effd640 1 -- 172.21.15.16:0/3084138275 >> v2:172.21.15.146:3300/0 conn(0x7fae6015b120 msgr2=0x7fae6006f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae5effd640 1 --2- 172.21.15.16:0/3084138275 >> v2:172.21.15.146:3300/0 conn(0x7fae6015b120 0x7fae6006f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae5effd640 1 -- 172.21.15.16:0/3084138275 >> v1:172.21.15.146:6789/0 conn(0x7fae600d0bc0 legacy=0x7fae600c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae5effd640 1 -- 172.21.15.16:0/3084138275 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6006ef00 con 0x7fae6006dec0 2024-09-19T07:51:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae5effd640 1 -- 172.21.15.16:0/3084138275 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fae54001670 con 0x7fae6006dec0 2024-09-19T07:51:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae5effd640 1 -- 172.21.15.16:0/3084138275 >> v1:172.21.15.16:6789/0 conn(0x7fae6006dec0 legacy=0x7fae600d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae5effd640 1 --2- 172.21.15.16:0/3084138275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae480039b0 0x7fae48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae5effd640 1 --2- 172.21.15.16:0/3084138275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae48004800 0x7fae48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae5effd640 1 --2- 172.21.15.16:0/3084138275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae480051c0 0x7fae4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae5effd640 1 -- 172.21.15.16:0/3084138275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fae6006ec20 con 0x7fae480039b0 2024-09-19T07:51:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae5effd640 1 -- 172.21.15.16:0/3084138275 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fae4800c060 con 0x7fae480051c0 2024-09-19T07:51:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.775+0000 7fae5effd640 1 -- 172.21.15.16:0/3084138275 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fae4800c260 con 0x7fae48004800 2024-09-19T07:51:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.776+0000 7fae5ffff640 1 --2- 172.21.15.16:0/3084138275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae480039b0 0x7fae48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.776+0000 7fae5f7fe640 1 --2- 172.21.15.16:0/3084138275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae48004800 0x7fae48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.776+0000 7fae64a74640 1 --2- 172.21.15.16:0/3084138275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae480051c0 0x7fae4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.776+0000 7fae5ffff640 1 -- 172.21.15.16:0/3084138275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae48004800 msgr2=0x7fae48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.776+0000 7fae5ffff640 1 --2- 172.21.15.16:0/3084138275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae48004800 0x7fae48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.776+0000 7fae5ffff640 1 -- 172.21.15.16:0/3084138275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae480051c0 msgr2=0x7fae4800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.776+0000 7fae5ffff640 1 --2- 172.21.15.16:0/3084138275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae480051c0 0x7fae4800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.776+0000 7fae64a74640 1 -- 172.21.15.16:0/3084138275 reap_dead start 2024-09-19T07:51:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.776+0000 7fae5ffff640 1 -- 172.21.15.16:0/3084138275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae48003520 con 0x7fae480039b0 2024-09-19T07:51:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.776+0000 7fae5f7fe640 1 --2- 172.21.15.16:0/3084138275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae48004800 0x7fae48004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:51:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.777+0000 7fae5ffff640 1 --2- 172.21.15.16:0/3084138275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae480039b0 0x7fae48003dd0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fae50009ac0 tx=0x7fae48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cafbcc8b869d95e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.777+0000 7fae5effd640 1 -- 172.21.15.16:0/3084138275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50036070 con 0x7fae480039b0 2024-09-19T07:51:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.777+0000 7fae5effd640 1 -- 172.21.15.16:0/3084138275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae50031400 con 0x7fae480039b0 2024-09-19T07:51:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.777+0000 7fae5effd640 1 -- 172.21.15.16:0/3084138275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50009690 con 0x7fae480039b0 2024-09-19T07:51:40.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.777+0000 7fae664d7640 1 -- 172.21.15.16:0/3084138275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae480039b0 msgr2=0x7fae48003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:40.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.777+0000 7fae664d7640 1 --2- 172.21.15.16:0/3084138275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae480039b0 0x7fae48003dd0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fae50009ac0 tx=0x7fae48005830 comp rx=0 tx=0).stop 2024-09-19T07:51:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.777+0000 7fae664d7640 1 -- 172.21.15.16:0/3084138275 shutdown_connections 2024-09-19T07:51:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.777+0000 7fae664d7640 1 --2- 172.21.15.16:0/3084138275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae480039b0 0x7fae48003dd0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.777+0000 7fae664d7640 1 -- 172.21.15.16:0/3084138275 >> 172.21.15.16:0/3084138275 conn(0x7fae6006bdf0 msgr2=0x7fae6006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.777+0000 7fae664d7640 1 -- 172.21.15.16:0/3084138275 shutdown_connections 2024-09-19T07:51:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.777+0000 7fae664d7640 1 -- 172.21.15.16:0/3084138275 wait complete. 2024-09-19T07:51:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.779+0000 7fae664d7640 1 Processor -- start 2024-09-19T07:51:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.779+0000 7fae664d7640 1 -- start start 2024-09-19T07:51:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae664d7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae480039b0 0x7fae600c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae664d7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae60145160 0x7fae601e4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae664d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae601f35f0 0x7fae601f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae664d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fae600ce060 con 0x7fae601f35f0 2024-09-19T07:51:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae664d7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fae6006f4c0 con 0x7fae60145160 2024-09-19T07:51:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae664d7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fae6006f9e0 con 0x7fae480039b0 2024-09-19T07:51:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae64a74640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae601f35f0 0x7fae601f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:40.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae64a74640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae601f35f0 0x7fae601f3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54260/0 (socket says 172.21.15.16:54260) 2024-09-19T07:51:40.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae64a74640 1 -- 172.21.15.16:0/3096192664 learned_addr learned my addr 172.21.15.16:0/3096192664 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:51:40.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae5ffff640 1 --2- 172.21.15.16:0/3096192664 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae480039b0 0x7fae600c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:40.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae5f7fe640 1 --2- 172.21.15.16:0/3096192664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae60145160 0x7fae601e4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:40.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae64a74640 1 -- 172.21.15.16:0/3096192664 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae480039b0 msgr2=0x7fae600c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:40.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae64a74640 1 --2- 172.21.15.16:0/3096192664 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae480039b0 0x7fae600c7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:40.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae64a74640 1 -- 172.21.15.16:0/3096192664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae60145160 msgr2=0x7fae601e4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:40.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae64a74640 1 --2- 172.21.15.16:0/3096192664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae60145160 0x7fae601e4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:40.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae64a74640 1 -- 172.21.15.16:0/3096192664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae60145710 con 0x7fae601f35f0 2024-09-19T07:51:40.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae64a74640 1 --2- 172.21.15.16:0/3096192664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae601f35f0 0x7fae601f3a50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fae58007fd0 tx=0x7fae5800e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:40.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae5cff9640 1 -- 172.21.15.16:0/3096192664 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58019070 con 0x7fae601f35f0 2024-09-19T07:51:40.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae5cff9640 1 -- 172.21.15.16:0/3096192664 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae58006d60 con 0x7fae601f35f0 2024-09-19T07:51:40.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae664d7640 1 -- 172.21.15.16:0/3096192664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae601459f0 con 0x7fae601f35f0 2024-09-19T07:51:40.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae5cff9640 1 -- 172.21.15.16:0/3096192664 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58003b90 con 0x7fae601f35f0 2024-09-19T07:51:40.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.780+0000 7fae664d7640 1 -- 172.21.15.16:0/3096192664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae60145fb0 con 0x7fae601f35f0 2024-09-19T07:51:40.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.781+0000 7fae664d7640 1 -- 172.21.15.16:0/3096192664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae6006e330 con 0x7fae601f35f0 2024-09-19T07:51:40.785 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:40.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.782+0000 7fae5cff9640 1 -- 172.21.15.16:0/3096192664 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fae5801c070 con 0x7fae601f35f0 2024-09-19T07:51:40.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.782+0000 7fae5cff9640 1 --2- 172.21.15.16:0/3096192664 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fae4c06c2e0 0x7fae4c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:40.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.783+0000 7fae5cff9640 1 -- 172.21.15.16:0/3096192664 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7fae5808ab90 con 0x7fae601f35f0 2024-09-19T07:51:40.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.783+0000 7fae5ffff640 1 --2- 172.21.15.16:0/3096192664 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fae4c06c2e0 0x7fae4c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:40.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.783+0000 7fae5ffff640 1 --2- 172.21.15.16:0/3096192664 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fae4c06c2e0 0x7fae4c06e7a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fae50033000 tx=0x7fae50008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:40.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:40.787+0000 7fae5cff9640 1 -- 172.21.15.16:0/3096192664 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fae58058a20 con 0x7fae601f35f0 2024-09-19T07:51:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:41.000+0000 7fae664d7640 1 -- 172.21.15.16:0/3096192664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fae600cac20 con 0x7fae601f35f0 2024-09-19T07:51:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:41.000+0000 7fae5cff9640 1 -- 172.21.15.16:0/3096192664 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fae5805cd20 con 0x7fae601f35f0 2024-09-19T07:51:41.001 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:51:41.001 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:51:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:41.002+0000 7fae664d7640 1 -- 172.21.15.16:0/3096192664 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fae4c06c2e0 msgr2=0x7fae4c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:41.002+0000 7fae664d7640 1 --2- 172.21.15.16:0/3096192664 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fae4c06c2e0 0x7fae4c06e7a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fae50033000 tx=0x7fae50008030 comp rx=0 tx=0).stop 2024-09-19T07:51:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:41.002+0000 7fae664d7640 1 -- 172.21.15.16:0/3096192664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae601f35f0 msgr2=0x7fae601f3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:41.002+0000 7fae664d7640 1 --2- 172.21.15.16:0/3096192664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae601f35f0 0x7fae601f3a50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fae58007fd0 tx=0x7fae5800e9e0 comp rx=0 tx=0).stop 2024-09-19T07:51:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:41.002+0000 7fae664d7640 1 -- 172.21.15.16:0/3096192664 shutdown_connections 2024-09-19T07:51:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:41.002+0000 7fae664d7640 1 --2- 172.21.15.16:0/3096192664 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fae4c06c2e0 0x7fae4c06e7a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:41.002+0000 7fae664d7640 1 --2- 172.21.15.16:0/3096192664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae601f35f0 0x7fae601f3a50 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:41.002+0000 7fae664d7640 1 --2- 172.21.15.16:0/3096192664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae60145160 0x7fae601e4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:41.002+0000 7fae664d7640 1 --2- 172.21.15.16:0/3096192664 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae480039b0 0x7fae600c7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:41.002+0000 7fae664d7640 1 -- 172.21.15.16:0/3096192664 >> 172.21.15.16:0/3096192664 conn(0x7fae6006bdf0 msgr2=0x7fae60075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:41.002+0000 7fae664d7640 1 -- 172.21.15.16:0/3096192664 shutdown_connections 2024-09-19T07:51:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:41.003+0000 7fae664d7640 1 -- 172.21.15.16:0/3096192664 wait complete. 2024-09-19T07:51:41.013 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:51:41.013 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:51:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:44.786 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:46.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:51:46.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.129+0000 7f07a7c77640 1 Processor -- start 2024-09-19T07:51:46.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.129+0000 7f07a7c77640 1 -- start start 2024-09-19T07:51:46.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.130+0000 7f07a7c77640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07a015e2a0 0x7f07a006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:46.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.130+0000 7f07a7c77640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07a006d960 0x7f07a00d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:46.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.130+0000 7f07a7c77640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07a00d1880 con 0x7f07a015e680 2024-09-19T07:51:46.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.130+0000 7f07a7c77640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f07a00d1b60 con 0x7f07a015e2a0 2024-09-19T07:51:46.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.130+0000 7f07a7c77640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f07a00cf2d0 con 0x7f07a006d960 2024-09-19T07:51:46.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.130+0000 7f07a5a13640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07a015e2a0 0x7f07a006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:46.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.130+0000 7f07a5a13640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07a015e2a0 0x7f07a006d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35380/0 (socket says 172.21.15.16:35380) 2024-09-19T07:51:46.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.130+0000 7f07a5a13640 1 -- 172.21.15.16:0/1259545047 learned_addr learned my addr 172.21.15.16:0/1259545047 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:51:46.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.130+0000 7f07a6214640 1 --2- 172.21.15.16:0/1259545047 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07a006d960 0x7f07a00d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:46.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.131+0000 7f07a5a13640 1 -- 172.21.15.16:0/1259545047 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07a006d960 msgr2=0x7f07a00d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:46.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.131+0000 7f07a5a13640 1 --2- 172.21.15.16:0/1259545047 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07a006d960 0x7f07a00d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:46.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.131+0000 7f07a5a13640 1 -- 172.21.15.16:0/1259545047 >> v1:172.21.15.146:6789/0 conn(0x7f07a015e680 legacy=0x7f07a00cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:46.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.131+0000 7f07a5a13640 1 -- 172.21.15.16:0/1259545047 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a00cf5b0 con 0x7f07a015e2a0 2024-09-19T07:51:46.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.132+0000 7f07a5a13640 1 --2- 172.21.15.16:0/1259545047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a015e2a0 0x7f07a006d420 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f079c009870 tx=0x7f079c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4669271c2969f0ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:46.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.132+0000 7f07a4a11640 1 -- 172.21.15.16:0/1259545047 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c037070 con 0x7f07a015e2a0 2024-09-19T07:51:46.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.132+0000 7f07a4a11640 1 -- 172.21.15.16:0/1259545047 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f079c032440 con 0x7f07a015e2a0 2024-09-19T07:51:46.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.132+0000 7f07a4a11640 1 -- 172.21.15.16:0/1259545047 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c032810 con 0x7f07a015e2a0 2024-09-19T07:51:46.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.132+0000 7f07a7c77640 1 -- 172.21.15.16:0/1259545047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a015e2a0 msgr2=0x7f07a006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:46.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.132+0000 7f07a7c77640 1 --2- 172.21.15.16:0/1259545047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a015e2a0 0x7f07a006d420 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f079c009870 tx=0x7f079c02a270 comp rx=0 tx=0).stop 2024-09-19T07:51:46.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.132+0000 7f07a7c77640 1 -- 172.21.15.16:0/1259545047 shutdown_connections 2024-09-19T07:51:46.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.132+0000 7f07a7c77640 1 --2- 172.21.15.16:0/1259545047 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07a006d960 0x7f07a00d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:46.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.132+0000 7f07a7c77640 1 --2- 172.21.15.16:0/1259545047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a015e2a0 0x7f07a006d420 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:46.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.132+0000 7f07a7c77640 1 -- 172.21.15.16:0/1259545047 >> 172.21.15.16:0/1259545047 conn(0x7f07a009d760 msgr2=0x7f07a008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:46.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.132+0000 7f07a7c77640 1 -- 172.21.15.16:0/1259545047 shutdown_connections 2024-09-19T07:51:46.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.132+0000 7f07a7c77640 1 -- 172.21.15.16:0/1259545047 wait complete. 2024-09-19T07:51:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.134+0000 7f07a7c77640 1 Processor -- start 2024-09-19T07:51:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.134+0000 7f07a7c77640 1 -- start start 2024-09-19T07:51:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.134+0000 7f07a7c77640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07a006d960 0x7f07a014a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.134+0000 7f07a7c77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a015e2a0 0x7f07a00cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.134+0000 7f07a7c77640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07a015e680 0x7f07a01e7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.134+0000 7f07a7c77640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07a00d0360 con 0x7f07a015e2a0 2024-09-19T07:51:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.134+0000 7f07a7c77640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f07a00d01e0 con 0x7f07a006d960 2024-09-19T07:51:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.134+0000 7f07a7c77640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f07a00d48b0 con 0x7f07a015e680 2024-09-19T07:51:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.135+0000 7f07a5212640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a015e2a0 0x7f07a00cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.135+0000 7f07a5212640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a015e2a0 0x7f07a00cab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35384/0 (socket says 172.21.15.16:35384) 2024-09-19T07:51:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.135+0000 7f07a5212640 1 -- 172.21.15.16:0/2699699138 learned_addr learned my addr 172.21.15.16:0/2699699138 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:51:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.135+0000 7f07a5a13640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07a006d960 0x7f07a014a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:46.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.135+0000 7f07a6214640 1 --2- 172.21.15.16:0/2699699138 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07a015e680 0x7f07a01e7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:46.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.135+0000 7f07a5212640 1 -- 172.21.15.16:0/2699699138 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07a015e680 msgr2=0x7f07a01e7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:46.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.135+0000 7f07a5212640 1 --2- 172.21.15.16:0/2699699138 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07a015e680 0x7f07a01e7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:46.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.135+0000 7f07a5212640 1 -- 172.21.15.16:0/2699699138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07a006d960 msgr2=0x7f07a014a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:46.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.135+0000 7f07a5212640 1 --2- 172.21.15.16:0/2699699138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07a006d960 0x7f07a014a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:46.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.135+0000 7f07a5212640 1 -- 172.21.15.16:0/2699699138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a014b3f0 con 0x7f07a015e2a0 2024-09-19T07:51:46.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.135+0000 7f07a5212640 1 --2- 172.21.15.16:0/2699699138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a015e2a0 0x7f07a00cab90 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f079000c440 tx=0x7f079000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:46.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.135+0000 7f07967fc640 1 -- 172.21.15.16:0/2699699138 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07900052d0 con 0x7f07a015e2a0 2024-09-19T07:51:46.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.135+0000 7f07967fc640 1 -- 172.21.15.16:0/2699699138 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0790005470 con 0x7f07a015e2a0 2024-09-19T07:51:46.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.135+0000 7f07967fc640 1 -- 172.21.15.16:0/2699699138 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790007260 con 0x7f07a015e2a0 2024-09-19T07:51:46.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.135+0000 7f07a7c77640 1 -- 172.21.15.16:0/2699699138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a014b6d0 con 0x7f07a015e2a0 2024-09-19T07:51:46.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.135+0000 7f07a7c77640 1 -- 172.21.15.16:0/2699699138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a014bc90 con 0x7f07a015e2a0 2024-09-19T07:51:46.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.136+0000 7f07a7c77640 1 -- 172.21.15.16:0/2699699138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07a00d4c70 con 0x7f07a015e2a0 2024-09-19T07:51:46.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.137+0000 7f07967fc640 1 -- 172.21.15.16:0/2699699138 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f07900074c0 con 0x7f07a015e2a0 2024-09-19T07:51:46.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.137+0000 7f07967fc640 1 --2- 172.21.15.16:0/2699699138 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f078406c2b0 0x7f078406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:46.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.137+0000 7f07967fc640 1 -- 172.21.15.16:0/2699699138 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f079008aa10 con 0x7f07a015e2a0 2024-09-19T07:51:46.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.138+0000 7f07a5a13640 1 --2- 172.21.15.16:0/2699699138 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f078406c2b0 0x7f078406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:46.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.138+0000 7f07a5a13640 1 --2- 172.21.15.16:0/2699699138 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f078406c2b0 0x7f078406e770 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f079c034000 tx=0x7f079c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:46.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.142+0000 7f07967fc640 1 -- 172.21.15.16:0/2699699138 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f0790058930 con 0x7f07a015e2a0 2024-09-19T07:51:46.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.354+0000 7f07a7c77640 1 -- 172.21.15.16:0/2699699138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f07a006cf90 con 0x7f07a015e2a0 2024-09-19T07:51:46.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.355+0000 7f07967fc640 1 -- 172.21.15.16:0/2699699138 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f079005cc30 con 0x7f07a015e2a0 2024-09-19T07:51:46.355 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:51:46.355 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:51:46.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.356+0000 7f07a7c77640 1 -- 172.21.15.16:0/2699699138 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f078406c2b0 msgr2=0x7f078406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:46.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.357+0000 7f07a7c77640 1 --2- 172.21.15.16:0/2699699138 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f078406c2b0 0x7f078406e770 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f079c034000 tx=0x7f079c02e000 comp rx=0 tx=0).stop 2024-09-19T07:51:46.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.357+0000 7f07a7c77640 1 -- 172.21.15.16:0/2699699138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a015e2a0 msgr2=0x7f07a00cab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:46.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.357+0000 7f07a7c77640 1 --2- 172.21.15.16:0/2699699138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a015e2a0 0x7f07a00cab90 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f079000c440 tx=0x7f079000c900 comp rx=0 tx=0).stop 2024-09-19T07:51:46.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.357+0000 7f07a7c77640 1 -- 172.21.15.16:0/2699699138 shutdown_connections 2024-09-19T07:51:46.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.357+0000 7f07a7c77640 1 --2- 172.21.15.16:0/2699699138 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f078406c2b0 0x7f078406e770 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:46.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.357+0000 7f07a7c77640 1 --2- 172.21.15.16:0/2699699138 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07a015e680 0x7f07a01e7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:46.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.357+0000 7f07a7c77640 1 --2- 172.21.15.16:0/2699699138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a015e2a0 0x7f07a00cab90 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:46.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.357+0000 7f07a7c77640 1 --2- 172.21.15.16:0/2699699138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07a006d960 0x7f07a014a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:46.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.357+0000 7f07a7c77640 1 -- 172.21.15.16:0/2699699138 >> 172.21.15.16:0/2699699138 conn(0x7f07a009d760 msgr2=0x7f07a00b9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:46.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.357+0000 7f07a7c77640 1 -- 172.21.15.16:0/2699699138 shutdown_connections 2024-09-19T07:51:46.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:46.357+0000 7f07a7c77640 1 -- 172.21.15.16:0/2699699138 wait complete. 2024-09-19T07:51:46.368 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:51:46.368 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:51:46.788 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:48.790 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:50.792 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:51.369 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:51:51.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.483+0000 7fa3ccde8640 1 Processor -- start 2024-09-19T07:51:51.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.483+0000 7fa3ccde8640 1 -- start start 2024-09-19T07:51:51.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.484+0000 7fa3ccde8640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa3c815e2a0 0x7fa3c806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:51.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.484+0000 7fa3ccde8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3c80d1880 con 0x7fa3c806d960 2024-09-19T07:51:51.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.484+0000 7fa3ccde8640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3c80d1b60 con 0x7fa3c815e680 2024-09-19T07:51:51.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.484+0000 7fa3ccde8640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa3c80cf2d0 con 0x7fa3c815e2a0 2024-09-19T07:51:51.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c6d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa3c806d960 0x7fa3c80d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53668/0 (socket says 172.21.15.16:53668) 2024-09-19T07:51:51.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c6d9d640 1 -- 172.21.15.16:0/3336572852 learned_addr learned my addr 172.21.15.16:0/3336572852 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:51:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c659c640 1 --2- 172.21.15.16:0/3336572852 >> v2:172.21.15.146:3300/0 conn(0x7fa3c815e2a0 0x7fa3c806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c559a640 1 -- 172.21.15.16:0/3336572852 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 856197562 0 0) 0x7fa3c80d1880 con 0x7fa3c806d960 2024-09-19T07:51:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c559a640 1 -- 172.21.15.16:0/3336572852 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3b0003520 con 0x7fa3c806d960 2024-09-19T07:51:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c559a640 1 -- 172.21.15.16:0/3336572852 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 980238348 0 0) 0x7fa3b0003520 con 0x7fa3c806d960 2024-09-19T07:51:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c559a640 1 -- 172.21.15.16:0/3336572852 >> v2:172.21.15.146:3300/0 conn(0x7fa3c815e2a0 msgr2=0x7fa3c806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c559a640 1 --2- 172.21.15.16:0/3336572852 >> v2:172.21.15.146:3300/0 conn(0x7fa3c815e2a0 0x7fa3c806d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c559a640 1 -- 172.21.15.16:0/3336572852 >> v1:172.21.15.146:6789/0 conn(0x7fa3c815e680 legacy=0x7fa3c80cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c559a640 1 -- 172.21.15.16:0/3336572852 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c80cf5b0 con 0x7fa3c806d960 2024-09-19T07:51:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c559a640 1 -- 172.21.15.16:0/3336572852 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fa3c0001670 con 0x7fa3c806d960 2024-09-19T07:51:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c559a640 1 -- 172.21.15.16:0/3336572852 >> v1:172.21.15.16:6789/0 conn(0x7fa3c806d960 legacy=0x7fa3c80d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c559a640 1 --2- 172.21.15.16:0/3336572852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3b00039b0 0x7fa3b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c559a640 1 --2- 172.21.15.16:0/3336572852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa3b0004800 0x7fa3b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c559a640 1 --2- 172.21.15.16:0/3336572852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3b00051c0 0x7fa3b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c559a640 1 -- 172.21.15.16:0/3336572852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c80cf2d0 con 0x7fa3b00051c0 2024-09-19T07:51:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c559a640 1 -- 172.21.15.16:0/3336572852 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b000c060 con 0x7fa3b00039b0 2024-09-19T07:51:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c559a640 1 -- 172.21.15.16:0/3336572852 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa3b000c260 con 0x7fa3b0004800 2024-09-19T07:51:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c6d9d640 1 --2- 172.21.15.16:0/3336572852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3b00051c0 0x7fa3b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.485+0000 7fa3c659c640 1 --2- 172.21.15.16:0/3336572852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3b00039b0 0x7fa3b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.486+0000 7fa3c5d9b640 1 --2- 172.21.15.16:0/3336572852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa3b0004800 0x7fa3b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.486+0000 7fa3c6d9d640 1 -- 172.21.15.16:0/3336572852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa3b0004800 msgr2=0x7fa3b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.486+0000 7fa3c6d9d640 1 --2- 172.21.15.16:0/3336572852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa3b0004800 0x7fa3b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.486+0000 7fa3c6d9d640 1 -- 172.21.15.16:0/3336572852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3b00039b0 msgr2=0x7fa3b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.486+0000 7fa3c6d9d640 1 --2- 172.21.15.16:0/3336572852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3b00039b0 0x7fa3b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.486+0000 7fa3c6d9d640 1 -- 172.21.15.16:0/3336572852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b0003520 con 0x7fa3b00051c0 2024-09-19T07:51:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.486+0000 7fa3c6d9d640 1 -- 172.21.15.16:0/3336572852 reap_dead start 2024-09-19T07:51:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.486+0000 7fa3c6d9d640 1 --2- 172.21.15.16:0/3336572852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3b00051c0 0x7fa3b000b9c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000c210 tx=0x7fa3b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fdadc869422055 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.487+0000 7fa3c559a640 1 -- 172.21.15.16:0/3336572852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0007d30 con 0x7fa3b00051c0 2024-09-19T07:51:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.487+0000 7fa3c559a640 1 -- 172.21.15.16:0/3336572852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c0005ad0 con 0x7fa3b00051c0 2024-09-19T07:51:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.487+0000 7fa3c559a640 1 -- 172.21.15.16:0/3336572852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0007070 con 0x7fa3b00051c0 2024-09-19T07:51:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.487+0000 7fa3ccde8640 1 -- 172.21.15.16:0/3336572852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3b00051c0 msgr2=0x7fa3b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.487+0000 7fa3ccde8640 1 --2- 172.21.15.16:0/3336572852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3b00051c0 0x7fa3b000b9c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000c210 tx=0x7fa3b0005830 comp rx=0 tx=0).stop 2024-09-19T07:51:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.487+0000 7fa3ccde8640 1 -- 172.21.15.16:0/3336572852 shutdown_connections 2024-09-19T07:51:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.487+0000 7fa3ccde8640 1 --2- 172.21.15.16:0/3336572852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3b00051c0 0x7fa3b000b9c0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.487+0000 7fa3ccde8640 1 -- 172.21.15.16:0/3336572852 >> 172.21.15.16:0/3336572852 conn(0x7fa3c809d760 msgr2=0x7fa3c808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.487+0000 7fa3ccde8640 1 -- 172.21.15.16:0/3336572852 shutdown_connections 2024-09-19T07:51:51.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.487+0000 7fa3ccde8640 1 -- 172.21.15.16:0/3336572852 wait complete. 2024-09-19T07:51:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.489+0000 7fa3ccde8640 1 Processor -- start 2024-09-19T07:51:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.489+0000 7fa3ccde8640 1 -- start start 2024-09-19T07:51:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3ccde8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3b00051c0 0x7fa3c80d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3ccde8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c80cca90 0x7fa3c80d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3ccde8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa3c80b97f0 0x7fa3c80b9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3ccde8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c806d5a0 con 0x7fa3c80cca90 2024-09-19T07:51:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3ccde8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c806d420 con 0x7fa3b00051c0 2024-09-19T07:51:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3ccde8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c806d720 con 0x7fa3c80b97f0 2024-09-19T07:51:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3c5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c80cca90 0x7fa3c80d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3c5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c80cca90 0x7fa3c80d0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35410/0 (socket says 172.21.15.16:35410) 2024-09-19T07:51:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3c5d9b640 1 -- 172.21.15.16:0/2854068017 learned_addr learned my addr 172.21.15.16:0/2854068017 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:51:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3c659c640 1 --2- 172.21.15.16:0/2854068017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3b00051c0 0x7fa3c80d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3c6d9d640 1 --2- 172.21.15.16:0/2854068017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa3c80b97f0 0x7fa3c80b9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3c5d9b640 1 -- 172.21.15.16:0/2854068017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa3c80b97f0 msgr2=0x7fa3c80b9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3c5d9b640 1 --2- 172.21.15.16:0/2854068017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa3c80b97f0 0x7fa3c80b9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3c5d9b640 1 -- 172.21.15.16:0/2854068017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3b00051c0 msgr2=0x7fa3c80d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3c5d9b640 1 --2- 172.21.15.16:0/2854068017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3b00051c0 0x7fa3c80d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3c5d9b640 1 -- 172.21.15.16:0/2854068017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c80ce010 con 0x7fa3c80cca90 2024-09-19T07:51:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3c5d9b640 1 --2- 172.21.15.16:0/2854068017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c80cca90 0x7fa3c80d0de0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00f960 tx=0x7fa3bc00fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3b6ffd640 1 -- 172.21.15.16:0/2854068017 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc003b80 con 0x7fa3c80cca90 2024-09-19T07:51:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3b6ffd640 1 -- 172.21.15.16:0/2854068017 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3bc018070 con 0x7fa3c80cca90 2024-09-19T07:51:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3b6ffd640 1 -- 172.21.15.16:0/2854068017 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc00d790 con 0x7fa3c80cca90 2024-09-19T07:51:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.490+0000 7fa3ccde8640 1 -- 172.21.15.16:0/2854068017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c80ccf70 con 0x7fa3c80cca90 2024-09-19T07:51:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.491+0000 7fa3ccde8640 1 -- 172.21.15.16:0/2854068017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c80cd530 con 0x7fa3c80cca90 2024-09-19T07:51:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.491+0000 7fa3ccde8640 1 -- 172.21.15.16:0/2854068017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c80cfad0 con 0x7fa3c80cca90 2024-09-19T07:51:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.492+0000 7fa3b6ffd640 1 -- 172.21.15.16:0/2854068017 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa3bc00d930 con 0x7fa3c80cca90 2024-09-19T07:51:51.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.492+0000 7fa3b6ffd640 1 --2- 172.21.15.16:0/2854068017 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa3a806c2b0 0x7fa3a806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:51.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.493+0000 7fa3b6ffd640 1 -- 172.21.15.16:0/2854068017 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7fa3bc08b6f0 con 0x7fa3c80cca90 2024-09-19T07:51:51.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.493+0000 7fa3c659c640 1 --2- 172.21.15.16:0/2854068017 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa3a806c2b0 0x7fa3a806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:51.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.493+0000 7fa3c659c640 1 --2- 172.21.15.16:0/2854068017 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa3a806c2b0 0x7fa3a806e770 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009a80 tx=0x7fa3b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:51.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.497+0000 7fa3b6ffd640 1 -- 172.21.15.16:0/2854068017 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa3bc01a310 con 0x7fa3c80cca90 2024-09-19T07:51:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.710+0000 7fa3ccde8640 1 -- 172.21.15.16:0/2854068017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa3c815a060 con 0x7fa3c80cca90 2024-09-19T07:51:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.710+0000 7fa3b6ffd640 1 -- 172.21.15.16:0/2854068017 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fa3bc059c60 con 0x7fa3c80cca90 2024-09-19T07:51:51.710 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:51:51.711 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:51:51.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.712+0000 7fa3ccde8640 1 -- 172.21.15.16:0/2854068017 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa3a806c2b0 msgr2=0x7fa3a806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.712+0000 7fa3ccde8640 1 --2- 172.21.15.16:0/2854068017 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa3a806c2b0 0x7fa3a806e770 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009a80 tx=0x7fa3b8008040 comp rx=0 tx=0).stop 2024-09-19T07:51:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.712+0000 7fa3ccde8640 1 -- 172.21.15.16:0/2854068017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c80cca90 msgr2=0x7fa3c80d0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.712+0000 7fa3ccde8640 1 --2- 172.21.15.16:0/2854068017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c80cca90 0x7fa3c80d0de0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00f960 tx=0x7fa3bc00fe20 comp rx=0 tx=0).stop 2024-09-19T07:51:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.712+0000 7fa3ccde8640 1 -- 172.21.15.16:0/2854068017 shutdown_connections 2024-09-19T07:51:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.712+0000 7fa3ccde8640 1 --2- 172.21.15.16:0/2854068017 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa3a806c2b0 0x7fa3a806e770 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.712+0000 7fa3ccde8640 1 --2- 172.21.15.16:0/2854068017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa3c80b97f0 0x7fa3c80b9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.712+0000 7fa3ccde8640 1 --2- 172.21.15.16:0/2854068017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c80cca90 0x7fa3c80d0de0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.712+0000 7fa3ccde8640 1 --2- 172.21.15.16:0/2854068017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3b00051c0 0x7fa3c80d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.712+0000 7fa3ccde8640 1 -- 172.21.15.16:0/2854068017 >> 172.21.15.16:0/2854068017 conn(0x7fa3c809d760 msgr2=0x7fa3c80d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.712+0000 7fa3ccde8640 1 -- 172.21.15.16:0/2854068017 shutdown_connections 2024-09-19T07:51:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:51.712+0000 7fa3ccde8640 1 -- 172.21.15.16:0/2854068017 wait complete. 2024-09-19T07:51:51.724 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:51:51.724 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:51:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:56.725 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:51:56.799 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:51:56.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.839+0000 7f57dbf5f640 1 Processor -- start 2024-09-19T07:51:56.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.839+0000 7f57dbf5f640 1 -- start start 2024-09-19T07:51:56.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.840+0000 7f57dbf5f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57d4141680 0x7f57d40cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:56.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.840+0000 7f57dbf5f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57d415d710 con 0x7f57d4142170 2024-09-19T07:51:56.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.840+0000 7f57dbf5f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57d4142600 con 0x7f57d40d0370 2024-09-19T07:51:56.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.840+0000 7f57dbf5f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f57d40d07b0 con 0x7f57d4141680 2024-09-19T07:51:56.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.840+0000 7f57da4fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f57d4142170 0x7f57d413f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37518/0 (socket says 172.21.15.16:37518) 2024-09-19T07:51:56.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.840+0000 7f57da4fc640 1 -- 172.21.15.16:0/679501399 learned_addr learned my addr 172.21.15.16:0/679501399 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:51:56.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d9cfb640 1 --2- 172.21.15.16:0/679501399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57d4141680 0x7f57d40cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:56.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d8cf9640 1 -- 172.21.15.16:0/679501399 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 821857181 0 0) 0x7f57d415d710 con 0x7f57d4142170 2024-09-19T07:51:56.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d8cf9640 1 -- 172.21.15.16:0/679501399 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57c0003520 con 0x7f57d4142170 2024-09-19T07:51:56.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d8cf9640 1 -- 172.21.15.16:0/679501399 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2998153316 0 0) 0x7f57c0003520 con 0x7f57d4142170 2024-09-19T07:51:56.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d8cf9640 1 -- 172.21.15.16:0/679501399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57d4141680 msgr2=0x7f57d40cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:56.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d8cf9640 1 --2- 172.21.15.16:0/679501399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57d4141680 0x7f57d40cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:56.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d8cf9640 1 -- 172.21.15.16:0/679501399 >> v1:172.21.15.146:6789/0 conn(0x7f57d40d0370 legacy=0x7f57d4141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:56.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d8cf9640 1 -- 172.21.15.16:0/679501399 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d415c950 con 0x7f57d4142170 2024-09-19T07:51:56.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d8cf9640 1 -- 172.21.15.16:0/679501399 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f57cc0019f0 con 0x7f57d4142170 2024-09-19T07:51:56.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d8cf9640 1 -- 172.21.15.16:0/679501399 >> v1:172.21.15.16:6789/0 conn(0x7f57d4142170 legacy=0x7f57d413f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:56.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d8cf9640 1 --2- 172.21.15.16:0/679501399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c00039e0 0x7f57c0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:56.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d8cf9640 1 --2- 172.21.15.16:0/679501399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c0004800 0x7f57c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:56.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d8cf9640 1 --2- 172.21.15.16:0/679501399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57d4141680 0x7f57c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:56.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d8cf9640 1 -- 172.21.15.16:0/679501399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57d40d07b0 con 0x7f57c00039e0 2024-09-19T07:51:56.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d8cf9640 1 -- 172.21.15.16:0/679501399 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f57c000ba70 con 0x7f57c0004800 2024-09-19T07:51:56.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d8cf9640 1 -- 172.21.15.16:0/679501399 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f57c000bc70 con 0x7f57d4141680 2024-09-19T07:51:56.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d9cfb640 1 --2- 172.21.15.16:0/679501399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c00039e0 0x7f57c0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:56.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d94fa640 1 --2- 172.21.15.16:0/679501399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c0004800 0x7f57c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:56.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57da4fc640 1 --2- 172.21.15.16:0/679501399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57d4141680 0x7f57c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:56.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d9cfb640 1 -- 172.21.15.16:0/679501399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57d4141680 msgr2=0x7f57c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:56.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d9cfb640 1 --2- 172.21.15.16:0/679501399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57d4141680 0x7f57c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:56.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d9cfb640 1 -- 172.21.15.16:0/679501399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c0004800 msgr2=0x7f57c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:56.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d9cfb640 1 --2- 172.21.15.16:0/679501399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c0004800 0x7f57c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:56.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57d9cfb640 1 -- 172.21.15.16:0/679501399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c0003520 con 0x7f57c00039e0 2024-09-19T07:51:56.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.841+0000 7f57da4fc640 1 --2- 172.21.15.16:0/679501399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57d4141680 0x7f57c00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:51:56.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.842+0000 7f57d9cfb640 1 --2- 172.21.15.16:0/679501399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c00039e0 0x7f57c0003e20 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009a90 tx=0x7f57c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68bb0252f840d8aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:56.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.842+0000 7f57d8cf9640 1 -- 172.21.15.16:0/679501399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0036070 con 0x7f57c00039e0 2024-09-19T07:51:56.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.842+0000 7f57d8cf9640 1 -- 172.21.15.16:0/679501399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d0031400 con 0x7f57c00039e0 2024-09-19T07:51:56.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.842+0000 7f57d8cf9640 1 -- 172.21.15.16:0/679501399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0009690 con 0x7f57c00039e0 2024-09-19T07:51:56.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.843+0000 7f57dbf5f640 1 -- 172.21.15.16:0/679501399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c00039e0 msgr2=0x7f57c0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:56.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.843+0000 7f57dbf5f640 1 --2- 172.21.15.16:0/679501399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c00039e0 0x7f57c0003e20 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009a90 tx=0x7f57c0005830 comp rx=0 tx=0).stop 2024-09-19T07:51:56.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.843+0000 7f57da4fc640 1 -- 172.21.15.16:0/679501399 reap_dead start 2024-09-19T07:51:56.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.843+0000 7f57dbf5f640 1 -- 172.21.15.16:0/679501399 shutdown_connections 2024-09-19T07:51:56.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.843+0000 7f57dbf5f640 1 -- 172.21.15.16:0/679501399 >> 172.21.15.16:0/679501399 conn(0x7f57d40b3ed0 msgr2=0x7f57d40b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:56.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.843+0000 7f57dbf5f640 1 -- 172.21.15.16:0/679501399 shutdown_connections 2024-09-19T07:51:56.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.843+0000 7f57dbf5f640 1 -- 172.21.15.16:0/679501399 wait complete. 2024-09-19T07:51:56.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.844+0000 7f57dbf5f640 1 Processor -- start 2024-09-19T07:51:56.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.844+0000 7f57dbf5f640 1 -- start start 2024-09-19T07:51:56.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.845+0000 7f57dbf5f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c00039e0 0x7f57d41564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:56.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.845+0000 7f57dbf5f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57d40b51d0 0x7f57d40d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:56.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.845+0000 7f57dbf5f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57d40b5650 0x7f57d40d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:56.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.845+0000 7f57dbf5f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57d415cf10 con 0x7f57d40b5650 2024-09-19T07:51:56.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.845+0000 7f57dbf5f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f57d415c6f0 con 0x7f57c00039e0 2024-09-19T07:51:56.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.845+0000 7f57dbf5f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f57d415d1f0 con 0x7f57d40b51d0 2024-09-19T07:51:56.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.845+0000 7f57da4fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57d40b5650 0x7f57d40d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:56.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.845+0000 7f57da4fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57d40b5650 0x7f57d40d22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52622/0 (socket says 172.21.15.16:52622) 2024-09-19T07:51:56.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.845+0000 7f57da4fc640 1 -- 172.21.15.16:0/3748098173 learned_addr learned my addr 172.21.15.16:0/3748098173 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:51:56.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.845+0000 7f57d9cfb640 1 --2- 172.21.15.16:0/3748098173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c00039e0 0x7f57d41564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:56.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.845+0000 7f57d94fa640 1 --2- 172.21.15.16:0/3748098173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57d40b51d0 0x7f57d40d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:56.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.846+0000 7f57da4fc640 1 -- 172.21.15.16:0/3748098173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57d40b51d0 msgr2=0x7f57d40d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:56.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.846+0000 7f57da4fc640 1 --2- 172.21.15.16:0/3748098173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57d40b51d0 0x7f57d40d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:56.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.846+0000 7f57da4fc640 1 -- 172.21.15.16:0/3748098173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c00039e0 msgr2=0x7f57d41564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:56.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.846+0000 7f57da4fc640 1 --2- 172.21.15.16:0/3748098173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c00039e0 0x7f57d41564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:56.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.846+0000 7f57da4fc640 1 -- 172.21.15.16:0/3748098173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d40b5bb0 con 0x7f57d40b5650 2024-09-19T07:51:56.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.846+0000 7f57d9cfb640 1 --2- 172.21.15.16:0/3748098173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c00039e0 0x7f57d41564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:51:56.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.846+0000 7f57da4fc640 1 --2- 172.21.15.16:0/3748098173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57d40b5650 0x7f57d40d22f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f57d40cc840 tx=0x7f57cc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:56.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.846+0000 7f57ca7fc640 1 -- 172.21.15.16:0/3748098173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc00eb90 con 0x7f57d40b5650 2024-09-19T07:51:56.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.846+0000 7f57ca7fc640 1 -- 172.21.15.16:0/3748098173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57cc00ed30 con 0x7f57d40b5650 2024-09-19T07:51:56.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.846+0000 7f57dbf5f640 1 -- 172.21.15.16:0/3748098173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d40b5e90 con 0x7f57d40b5650 2024-09-19T07:51:56.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.846+0000 7f57ca7fc640 1 -- 172.21.15.16:0/3748098173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc014650 con 0x7f57d40b5650 2024-09-19T07:51:56.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.846+0000 7f57dbf5f640 1 -- 172.21.15.16:0/3748098173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d40b6d90 con 0x7f57d40b5650 2024-09-19T07:51:56.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.847+0000 7f57dbf5f640 1 -- 172.21.15.16:0/3748098173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d40b63b0 con 0x7f57d40b5650 2024-09-19T07:51:56.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.848+0000 7f57ca7fc640 1 -- 172.21.15.16:0/3748098173 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f57cc0147f0 con 0x7f57d40b5650 2024-09-19T07:51:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.848+0000 7f57ca7fc640 1 --2- 172.21.15.16:0/3748098173 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f57b806c2e0 0x7f57b806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.848+0000 7f57ca7fc640 1 -- 172.21.15.16:0/3748098173 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f57cc08b3d0 con 0x7f57d40b5650 2024-09-19T07:51:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.849+0000 7f57d9cfb640 1 --2- 172.21.15.16:0/3748098173 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f57b806c2e0 0x7f57b806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.849+0000 7f57d9cfb640 1 --2- 172.21.15.16:0/3748098173 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f57b806c2e0 0x7f57b806e7a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f57d0033000 tx=0x7f57d0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:56.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:56.853+0000 7f57ca7fc640 1 -- 172.21.15.16:0/3748098173 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f57cc059940 con 0x7f57d40b5650 2024-09-19T07:51:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:57.065+0000 7f57dbf5f640 1 -- 172.21.15.16:0/3748098173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f57d40cfe30 con 0x7f57d40b5650 2024-09-19T07:51:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:57.066+0000 7f57ca7fc640 1 -- 172.21.15.16:0/3748098173 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f57cc059240 con 0x7f57d40b5650 2024-09-19T07:51:57.066 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:51:57.066 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:51:57.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:57.068+0000 7f57dbf5f640 1 -- 172.21.15.16:0/3748098173 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f57b806c2e0 msgr2=0x7f57b806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:57.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:57.068+0000 7f57dbf5f640 1 --2- 172.21.15.16:0/3748098173 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f57b806c2e0 0x7f57b806e7a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f57d0033000 tx=0x7f57d0008030 comp rx=0 tx=0).stop 2024-09-19T07:51:57.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:57.068+0000 7f57dbf5f640 1 -- 172.21.15.16:0/3748098173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57d40b5650 msgr2=0x7f57d40d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:57.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:57.068+0000 7f57dbf5f640 1 --2- 172.21.15.16:0/3748098173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57d40b5650 0x7f57d40d22f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f57d40cc840 tx=0x7f57cc00e530 comp rx=0 tx=0).stop 2024-09-19T07:51:57.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:57.068+0000 7f57dbf5f640 1 -- 172.21.15.16:0/3748098173 shutdown_connections 2024-09-19T07:51:57.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:57.068+0000 7f57dbf5f640 1 --2- 172.21.15.16:0/3748098173 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f57b806c2e0 0x7f57b806e7a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:57.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:57.068+0000 7f57dbf5f640 1 --2- 172.21.15.16:0/3748098173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57d40b5650 0x7f57d40d22f0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:57.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:57.068+0000 7f57dbf5f640 1 --2- 172.21.15.16:0/3748098173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57d40b51d0 0x7f57d40d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:57.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:57.068+0000 7f57dbf5f640 1 --2- 172.21.15.16:0/3748098173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c00039e0 0x7f57d41564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:57.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:57.068+0000 7f57dbf5f640 1 -- 172.21.15.16:0/3748098173 >> 172.21.15.16:0/3748098173 conn(0x7f57d40b3ed0 msgr2=0x7f57d41401c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:57.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:57.068+0000 7f57dbf5f640 1 -- 172.21.15.16:0/3748098173 shutdown_connections 2024-09-19T07:51:57.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:51:57.068+0000 7f57dbf5f640 1 -- 172.21.15.16:0/3748098173 wait complete. 2024-09-19T07:51:57.079 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:51:57.079 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:51:58.801 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:00.803 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:02.080 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:52:02.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.193+0000 7fcf4d4a4640 1 Processor -- start 2024-09-19T07:52:02.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.194+0000 7fcf4d4a4640 1 -- start start 2024-09-19T07:52:02.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.194+0000 7fcf4d4a4640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcf48072250 0x7fcf480cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:02.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.194+0000 7fcf4d4a4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf480726c0 0x7fcf480d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:02.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.194+0000 7fcf4d4a4640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf480d2b30 con 0x7fcf480d2750 2024-09-19T07:52:02.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.194+0000 7fcf4d4a4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fcf48145bd0 con 0x7fcf480726c0 2024-09-19T07:52:02.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.194+0000 7fcf4d4a4640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fcf481427e0 con 0x7fcf48072250 2024-09-19T07:52:02.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.195+0000 7fcf477fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf480726c0 0x7fcf480d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:02.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.195+0000 7fcf477fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf480726c0 0x7fcf480d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52626/0 (socket says 172.21.15.16:52626) 2024-09-19T07:52:02.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.195+0000 7fcf477fe640 1 -- 172.21.15.16:0/2838221342 learned_addr learned my addr 172.21.15.16:0/2838221342 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:02.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.195+0000 7fcf467fc640 1 --2- 172.21.15.16:0/2838221342 >> v2:172.21.15.146:3300/0 conn(0x7fcf48072250 0x7fcf480cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:02.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.195+0000 7fcf477fe640 1 -- 172.21.15.16:0/2838221342 >> v2:172.21.15.146:3300/0 conn(0x7fcf48072250 msgr2=0x7fcf480cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:02.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.195+0000 7fcf477fe640 1 --2- 172.21.15.16:0/2838221342 >> v2:172.21.15.146:3300/0 conn(0x7fcf48072250 0x7fcf480cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:02.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.195+0000 7fcf477fe640 1 -- 172.21.15.16:0/2838221342 >> v1:172.21.15.146:6789/0 conn(0x7fcf480d2750 legacy=0x7fcf4813f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:02.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.195+0000 7fcf477fe640 1 -- 172.21.15.16:0/2838221342 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf48141c60 con 0x7fcf480726c0 2024-09-19T07:52:02.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.196+0000 7fcf477fe640 1 --2- 172.21.15.16:0/2838221342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf480726c0 0x7fcf480d0300 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fcf480d3260 tx=0x7fcf3802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b79b849da7ed9e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:02.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.196+0000 7fcf45ffb640 1 -- 172.21.15.16:0/2838221342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38037070 con 0x7fcf480726c0 2024-09-19T07:52:02.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.196+0000 7fcf45ffb640 1 -- 172.21.15.16:0/2838221342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf38032440 con 0x7fcf480726c0 2024-09-19T07:52:02.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.196+0000 7fcf45ffb640 1 -- 172.21.15.16:0/2838221342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38032810 con 0x7fcf480726c0 2024-09-19T07:52:02.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.196+0000 7fcf4d4a4640 1 -- 172.21.15.16:0/2838221342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf480726c0 msgr2=0x7fcf480d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:02.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.196+0000 7fcf4d4a4640 1 --2- 172.21.15.16:0/2838221342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf480726c0 0x7fcf480d0300 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fcf480d3260 tx=0x7fcf3802a1e0 comp rx=0 tx=0).stop 2024-09-19T07:52:02.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.196+0000 7fcf4d4a4640 1 -- 172.21.15.16:0/2838221342 shutdown_connections 2024-09-19T07:52:02.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.196+0000 7fcf4d4a4640 1 --2- 172.21.15.16:0/2838221342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf480726c0 0x7fcf480d0300 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:02.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.196+0000 7fcf4d4a4640 1 --2- 172.21.15.16:0/2838221342 >> v2:172.21.15.146:3300/0 conn(0x7fcf48072250 0x7fcf480cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:02.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.197+0000 7fcf4d4a4640 1 -- 172.21.15.16:0/2838221342 >> 172.21.15.16:0/2838221342 conn(0x7fcf480b4040 msgr2=0x7fcf480b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:02.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.197+0000 7fcf4d4a4640 1 -- 172.21.15.16:0/2838221342 shutdown_connections 2024-09-19T07:52:02.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.197+0000 7fcf4d4a4640 1 -- 172.21.15.16:0/2838221342 wait complete. 2024-09-19T07:52:02.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.198+0000 7fcf4d4a4640 1 Processor -- start 2024-09-19T07:52:02.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.198+0000 7fcf4d4a4640 1 -- start start 2024-09-19T07:52:02.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.199+0000 7fcf4d4a4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf48072250 0x7fcf48151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:02.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.199+0000 7fcf4d4a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf480726c0 0x7fcf48144200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:02.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.199+0000 7fcf4d4a4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf480d2750 0x7fcf480cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:02.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.199+0000 7fcf4d4a4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf48142500 con 0x7fcf480726c0 2024-09-19T07:52:02.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.199+0000 7fcf4d4a4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcf48142220 con 0x7fcf480d2750 2024-09-19T07:52:02.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.199+0000 7fcf4d4a4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcf481418b0 con 0x7fcf48072250 2024-09-19T07:52:02.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.199+0000 7fcf467fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf480726c0 0x7fcf48144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:02.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.199+0000 7fcf467fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf480726c0 0x7fcf48144200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52630/0 (socket says 172.21.15.16:52630) 2024-09-19T07:52:02.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.199+0000 7fcf467fc640 1 -- 172.21.15.16:0/3610263450 learned_addr learned my addr 172.21.15.16:0/3610263450 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:02.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.199+0000 7fcf46ffd640 1 --2- 172.21.15.16:0/3610263450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf48072250 0x7fcf48151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:02.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.199+0000 7fcf477fe640 1 --2- 172.21.15.16:0/3610263450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf480d2750 0x7fcf480cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:02.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.199+0000 7fcf467fc640 1 -- 172.21.15.16:0/3610263450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf48072250 msgr2=0x7fcf48151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:02.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.199+0000 7fcf467fc640 1 --2- 172.21.15.16:0/3610263450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf48072250 0x7fcf48151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:02.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.199+0000 7fcf467fc640 1 -- 172.21.15.16:0/3610263450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf480d2750 msgr2=0x7fcf480cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:02.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.199+0000 7fcf467fc640 1 --2- 172.21.15.16:0/3610263450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf480d2750 0x7fcf480cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:02.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.199+0000 7fcf467fc640 1 -- 172.21.15.16:0/3610263450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf480b5dc0 con 0x7fcf480726c0 2024-09-19T07:52:02.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.199+0000 7fcf467fc640 1 --2- 172.21.15.16:0/3610263450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf480726c0 0x7fcf48144200 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000d620 tx=0x7fcf3000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:02.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.200+0000 7fcf3f7fe640 1 -- 172.21.15.16:0/3610263450 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000b620 con 0x7fcf480726c0 2024-09-19T07:52:02.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.200+0000 7fcf3f7fe640 1 -- 172.21.15.16:0/3610263450 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf30019070 con 0x7fcf480726c0 2024-09-19T07:52:02.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.200+0000 7fcf3f7fe640 1 -- 172.21.15.16:0/3610263450 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000ba00 con 0x7fcf480726c0 2024-09-19T07:52:02.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.200+0000 7fcf4d4a4640 1 -- 172.21.15.16:0/3610263450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf480b60a0 con 0x7fcf480726c0 2024-09-19T07:52:02.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.200+0000 7fcf4d4a4640 1 -- 172.21.15.16:0/3610263450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf480b6660 con 0x7fcf480726c0 2024-09-19T07:52:02.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.201+0000 7fcf4d4a4640 1 -- 172.21.15.16:0/3610263450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf48141440 con 0x7fcf480726c0 2024-09-19T07:52:02.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.201+0000 7fcf3f7fe640 1 -- 172.21.15.16:0/3610263450 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcf3000bba0 con 0x7fcf480726c0 2024-09-19T07:52:02.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.202+0000 7fcf3f7fe640 1 --2- 172.21.15.16:0/3610263450 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcf2806c2b0 0x7fcf2806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:02.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.202+0000 7fcf3f7fe640 1 -- 172.21.15.16:0/3610263450 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7fcf3008b8e0 con 0x7fcf480726c0 2024-09-19T07:52:02.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.202+0000 7fcf46ffd640 1 --2- 172.21.15.16:0/3610263450 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcf2806c2b0 0x7fcf2806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:02.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.202+0000 7fcf46ffd640 1 --2- 172.21.15.16:0/3610263450 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcf2806c2b0 0x7fcf2806e770 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcf40001f10 tx=0x7fcf40009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:02.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.206+0000 7fcf3f7fe640 1 -- 172.21.15.16:0/3610263450 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fcf48141440 con 0x7fcf480726c0 2024-09-19T07:52:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.421+0000 7fcf4d4a4640 1 -- 172.21.15.16:0/3610263450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fcf48140020 con 0x7fcf480726c0 2024-09-19T07:52:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.421+0000 7fcf3f7fe640 1 -- 172.21.15.16:0/3610263450 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fcf30059800 con 0x7fcf480726c0 2024-09-19T07:52:02.422 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:52:02.422 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:52:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.423+0000 7fcf4d4a4640 1 -- 172.21.15.16:0/3610263450 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcf2806c2b0 msgr2=0x7fcf2806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.423+0000 7fcf4d4a4640 1 --2- 172.21.15.16:0/3610263450 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcf2806c2b0 0x7fcf2806e770 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcf40001f10 tx=0x7fcf40009040 comp rx=0 tx=0).stop 2024-09-19T07:52:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.423+0000 7fcf4d4a4640 1 -- 172.21.15.16:0/3610263450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf480726c0 msgr2=0x7fcf48144200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.423+0000 7fcf4d4a4640 1 --2- 172.21.15.16:0/3610263450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf480726c0 0x7fcf48144200 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000d620 tx=0x7fcf3000dae0 comp rx=0 tx=0).stop 2024-09-19T07:52:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.423+0000 7fcf4d4a4640 1 -- 172.21.15.16:0/3610263450 shutdown_connections 2024-09-19T07:52:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.423+0000 7fcf4d4a4640 1 --2- 172.21.15.16:0/3610263450 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcf2806c2b0 0x7fcf2806e770 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.423+0000 7fcf4d4a4640 1 --2- 172.21.15.16:0/3610263450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf480d2750 0x7fcf480cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.423+0000 7fcf4d4a4640 1 --2- 172.21.15.16:0/3610263450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf480726c0 0x7fcf48144200 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.423+0000 7fcf4d4a4640 1 --2- 172.21.15.16:0/3610263450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf48072250 0x7fcf48151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.423+0000 7fcf4d4a4640 1 -- 172.21.15.16:0/3610263450 >> 172.21.15.16:0/3610263450 conn(0x7fcf480b4040 msgr2=0x7fcf480cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.423+0000 7fcf4d4a4640 1 -- 172.21.15.16:0/3610263450 shutdown_connections 2024-09-19T07:52:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:02.423+0000 7fcf4d4a4640 1 -- 172.21.15.16:0/3610263450 wait complete. 2024-09-19T07:52:02.435 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:52:02.435 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:52:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:07.436 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:52:07.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.549+0000 7fab94f03640 1 Processor -- start 2024-09-19T07:52:07.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.549+0000 7fab94f03640 1 -- start start 2024-09-19T07:52:07.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.550+0000 7fab94f03640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab900d21f0 0x7fab900d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.550+0000 7fab94f03640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fab90072590 0x7fab9006d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.550+0000 7fab94f03640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab900d2b10 con 0x7fab90143a60 2024-09-19T07:52:07.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.550+0000 7fab94f03640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fab90141750 con 0x7fab90072590 2024-09-19T07:52:07.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.550+0000 7fab94f03640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fab901620b0 con 0x7fab900d21f0 2024-09-19T07:52:07.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.550+0000 7fab8dd9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fab90072590 0x7fab9006d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.550+0000 7fab8dd9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fab90072590 0x7fab9006d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50908/0 (socket says 172.21.15.16:50908) 2024-09-19T07:52:07.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.550+0000 7fab8ed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fab90143a60 0x7fab90143f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48726/0 (socket says 172.21.15.16:48726) 2024-09-19T07:52:07.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.550+0000 7fab8dd9b640 1 -- 172.21.15.16:0/1970573592 learned_addr learned my addr 172.21.15.16:0/1970573592 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:07.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.550+0000 7fab8e59c640 1 --2- 172.21.15.16:0/1970573592 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab900d21f0 0x7fab900d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.551+0000 7fab8dd9b640 1 -- 172.21.15.16:0/1970573592 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab900d21f0 msgr2=0x7fab900d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:07.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.551+0000 7fab8dd9b640 1 --2- 172.21.15.16:0/1970573592 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab900d21f0 0x7fab900d25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.551+0000 7fab8dd9b640 1 -- 172.21.15.16:0/1970573592 >> v1:172.21.15.16:6789/0 conn(0x7fab90143a60 legacy=0x7fab90143f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:07.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.551+0000 7fab8dd9b640 1 -- 172.21.15.16:0/1970573592 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab90162370 con 0x7fab90072590 2024-09-19T07:52:07.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.551+0000 7fab8d59a640 1 -- 172.21.15.16:0/1970573592 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3391377130 0 0) 0x7fab900d2b10 con 0x7fab90143a60 2024-09-19T07:52:07.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.551+0000 7fab8dd9b640 1 --2- 172.21.15.16:0/1970573592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90072590 0x7fab9006d0e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fab78009870 tx=0x7fab7802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8806dcd0d75d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:07.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.551+0000 7fab8d59a640 1 -- 172.21.15.16:0/1970573592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78037070 con 0x7fab90072590 2024-09-19T07:52:07.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.552+0000 7fab8d59a640 1 -- 172.21.15.16:0/1970573592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab78032440 con 0x7fab90072590 2024-09-19T07:52:07.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.552+0000 7fab8d59a640 1 -- 172.21.15.16:0/1970573592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78032810 con 0x7fab90072590 2024-09-19T07:52:07.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.552+0000 7fab94f03640 1 -- 172.21.15.16:0/1970573592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90072590 msgr2=0x7fab9006d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:07.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.552+0000 7fab94f03640 1 --2- 172.21.15.16:0/1970573592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90072590 0x7fab9006d0e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fab78009870 tx=0x7fab7802a1c0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.552+0000 7fab94f03640 1 -- 172.21.15.16:0/1970573592 shutdown_connections 2024-09-19T07:52:07.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.552+0000 7fab94f03640 1 --2- 172.21.15.16:0/1970573592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90072590 0x7fab9006d0e0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.552+0000 7fab94f03640 1 --2- 172.21.15.16:0/1970573592 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab900d21f0 0x7fab900d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.552+0000 7fab94f03640 1 -- 172.21.15.16:0/1970573592 >> 172.21.15.16:0/1970573592 conn(0x7fab900b3ed0 msgr2=0x7fab900b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:07.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.552+0000 7fab94f03640 1 -- 172.21.15.16:0/1970573592 shutdown_connections 2024-09-19T07:52:07.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.552+0000 7fab94f03640 1 -- 172.21.15.16:0/1970573592 wait complete. 2024-09-19T07:52:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.554+0000 7fab94f03640 1 Processor -- start 2024-09-19T07:52:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.554+0000 7fab94f03640 1 -- start start 2024-09-19T07:52:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.554+0000 7fab94f03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90072590 0x7fab90158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.554+0000 7fab94f03640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab900d21f0 0x7fab900c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.554+0000 7fab94f03640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab90143a60 0x7fab900c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.554+0000 7fab94f03640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab90162c10 con 0x7fab90072590 2024-09-19T07:52:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.554+0000 7fab94f03640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fab90162930 con 0x7fab900d21f0 2024-09-19T07:52:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.554+0000 7fab94f03640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fab9006d620 con 0x7fab90143a60 2024-09-19T07:52:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.554+0000 7fab8e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90072590 0x7fab90158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.554+0000 7fab8e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90072590 0x7fab90158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50910/0 (socket says 172.21.15.16:50910) 2024-09-19T07:52:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.554+0000 7fab8e59c640 1 -- 172.21.15.16:0/673685899 learned_addr learned my addr 172.21.15.16:0/673685899 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.555+0000 7fab8ed9d640 1 --2- 172.21.15.16:0/673685899 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab90143a60 0x7fab900c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.555+0000 7fab8dd9b640 1 --2- 172.21.15.16:0/673685899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab900d21f0 0x7fab900c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.555+0000 7fab8e59c640 1 -- 172.21.15.16:0/673685899 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab90143a60 msgr2=0x7fab900c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.555+0000 7fab8e59c640 1 --2- 172.21.15.16:0/673685899 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab90143a60 0x7fab900c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.555+0000 7fab8e59c640 1 -- 172.21.15.16:0/673685899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab900d21f0 msgr2=0x7fab900c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.555+0000 7fab8e59c640 1 --2- 172.21.15.16:0/673685899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab900d21f0 0x7fab900c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.555+0000 7fab8e59c640 1 -- 172.21.15.16:0/673685899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab90159540 con 0x7fab90072590 2024-09-19T07:52:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.555+0000 7fab8ed9d640 1 --2- 172.21.15.16:0/673685899 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab90143a60 0x7fab900c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:52:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.555+0000 7fab8dd9b640 1 --2- 172.21.15.16:0/673685899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab900d21f0 0x7fab900c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:52:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.555+0000 7fab8e59c640 1 --2- 172.21.15.16:0/673685899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90072590 0x7fab90158e40 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fab8800e9b0 tx=0x7fab8800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.555+0000 7fab86ffd640 1 -- 172.21.15.16:0/673685899 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88017070 con 0x7fab90072590 2024-09-19T07:52:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.555+0000 7fab86ffd640 1 -- 172.21.15.16:0/673685899 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8800c9a0 con 0x7fab90072590 2024-09-19T07:52:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.555+0000 7fab94f03640 1 -- 172.21.15.16:0/673685899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab900b5a90 con 0x7fab90072590 2024-09-19T07:52:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.555+0000 7fab86ffd640 1 -- 172.21.15.16:0/673685899 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8800cd90 con 0x7fab90072590 2024-09-19T07:52:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.555+0000 7fab94f03640 1 -- 172.21.15.16:0/673685899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab9015b480 con 0x7fab90072590 2024-09-19T07:52:07.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.556+0000 7fab94f03640 1 -- 172.21.15.16:0/673685899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab90158c30 con 0x7fab90072590 2024-09-19T07:52:07.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.557+0000 7fab86ffd640 1 -- 172.21.15.16:0/673685899 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fab8801a070 con 0x7fab90072590 2024-09-19T07:52:07.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.557+0000 7fab86ffd640 1 --2- 172.21.15.16:0/673685899 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fab7006c2b0 0x7fab7006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.558+0000 7fab86ffd640 1 -- 172.21.15.16:0/673685899 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7fab8808ba40 con 0x7fab90072590 2024-09-19T07:52:07.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.558+0000 7fab8dd9b640 1 --2- 172.21.15.16:0/673685899 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fab7006c2b0 0x7fab7006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.558+0000 7fab8dd9b640 1 --2- 172.21.15.16:0/673685899 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fab7006c2b0 0x7fab7006e770 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fab78034000 tx=0x7fab7802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:07.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.562+0000 7fab86ffd640 1 -- 172.21.15.16:0/673685899 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fab88059fa0 con 0x7fab90072590 2024-09-19T07:52:07.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.776+0000 7fab94f03640 1 -- 172.21.15.16:0/673685899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fab900690c0 con 0x7fab90072590 2024-09-19T07:52:07.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.776+0000 7fab86ffd640 1 -- 172.21.15.16:0/673685899 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fab880598a0 con 0x7fab90072590 2024-09-19T07:52:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:52:07.778 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:52:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.779+0000 7fab94f03640 1 -- 172.21.15.16:0/673685899 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fab7006c2b0 msgr2=0x7fab7006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.779+0000 7fab94f03640 1 --2- 172.21.15.16:0/673685899 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fab7006c2b0 0x7fab7006e770 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fab78034000 tx=0x7fab7802e000 comp rx=0 tx=0).stop 2024-09-19T07:52:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.779+0000 7fab94f03640 1 -- 172.21.15.16:0/673685899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90072590 msgr2=0x7fab90158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.779+0000 7fab94f03640 1 --2- 172.21.15.16:0/673685899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90072590 0x7fab90158e40 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fab8800e9b0 tx=0x7fab8800ee70 comp rx=0 tx=0).stop 2024-09-19T07:52:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.779+0000 7fab94f03640 1 -- 172.21.15.16:0/673685899 shutdown_connections 2024-09-19T07:52:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.779+0000 7fab94f03640 1 --2- 172.21.15.16:0/673685899 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fab7006c2b0 0x7fab7006e770 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.779+0000 7fab94f03640 1 --2- 172.21.15.16:0/673685899 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab90143a60 0x7fab900c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.779+0000 7fab94f03640 1 --2- 172.21.15.16:0/673685899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab900d21f0 0x7fab900c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.779+0000 7fab94f03640 1 --2- 172.21.15.16:0/673685899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab90072590 0x7fab90158e40 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.779+0000 7fab94f03640 1 -- 172.21.15.16:0/673685899 >> 172.21.15.16:0/673685899 conn(0x7fab900b3ed0 msgr2=0x7fab9015cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.779+0000 7fab94f03640 1 -- 172.21.15.16:0/673685899 shutdown_connections 2024-09-19T07:52:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:07.779+0000 7fab94f03640 1 -- 172.21.15.16:0/673685899 wait complete. 2024-09-19T07:52:07.790 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:52:07.790 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:52:08.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:52:08.765 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:52:08.812 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:10.814 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:12.790 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:52:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.903+0000 7f40e9c8b640 1 Processor -- start 2024-09-19T07:52:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.904+0000 7f40e9c8b640 1 -- start start 2024-09-19T07:52:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.904+0000 7f40e9c8b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f40e415e2a0 0x7f40e406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.904+0000 7f40e9c8b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f40e406d960 0x7f40e40d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.904+0000 7f40e9c8b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40e40d1880 con 0x7f40e415e680 2024-09-19T07:52:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.904+0000 7f40e9c8b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f40e40d1b60 con 0x7f40e415e2a0 2024-09-19T07:52:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.904+0000 7f40e9c8b640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f40e40cf2d0 con 0x7f40e406d960 2024-09-19T07:52:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.904+0000 7f40e37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f40e415e2a0 0x7f40e406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.905+0000 7f40e37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f40e415e2a0 0x7f40e406d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42902/0 (socket says 172.21.15.16:42902) 2024-09-19T07:52:12.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.905+0000 7f40e37fe640 1 -- 172.21.15.16:0/3720250849 learned_addr learned my addr 172.21.15.16:0/3720250849 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:12.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.905+0000 7f40e3fff640 1 --2- 172.21.15.16:0/3720250849 >> v2:172.21.15.146:3300/0 conn(0x7f40e406d960 0x7f40e40d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.905+0000 7f40e37fe640 1 -- 172.21.15.16:0/3720250849 >> v2:172.21.15.146:3300/0 conn(0x7f40e406d960 msgr2=0x7f40e40d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:12.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.905+0000 7f40e37fe640 1 --2- 172.21.15.16:0/3720250849 >> v2:172.21.15.146:3300/0 conn(0x7f40e406d960 0x7f40e40d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.905+0000 7f40e37fe640 1 -- 172.21.15.16:0/3720250849 >> v1:172.21.15.146:6789/0 conn(0x7f40e415e680 legacy=0x7f40e40cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:12.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.905+0000 7f40e37fe640 1 -- 172.21.15.16:0/3720250849 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e40cf5b0 con 0x7f40e415e2a0 2024-09-19T07:52:12.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.906+0000 7f40e37fe640 1 --2- 172.21.15.16:0/3720250849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e415e2a0 0x7f40e406d420 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f40e4163050 tx=0x7f40d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60dce1ac0755eb52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:12.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.906+0000 7f40e27fc640 1 -- 172.21.15.16:0/3720250849 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4037070 con 0x7f40e415e2a0 2024-09-19T07:52:12.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.906+0000 7f40e27fc640 1 -- 172.21.15.16:0/3720250849 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40d4032440 con 0x7f40e415e2a0 2024-09-19T07:52:12.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.906+0000 7f40e27fc640 1 -- 172.21.15.16:0/3720250849 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4032810 con 0x7f40e415e2a0 2024-09-19T07:52:12.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.906+0000 7f40e9c8b640 1 -- 172.21.15.16:0/3720250849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e415e2a0 msgr2=0x7f40e406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.906+0000 7f40e9c8b640 1 --2- 172.21.15.16:0/3720250849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e415e2a0 0x7f40e406d420 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f40e4163050 tx=0x7f40d402a270 comp rx=0 tx=0).stop 2024-09-19T07:52:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.906+0000 7f40e9c8b640 1 -- 172.21.15.16:0/3720250849 shutdown_connections 2024-09-19T07:52:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.906+0000 7f40e9c8b640 1 --2- 172.21.15.16:0/3720250849 >> v2:172.21.15.146:3300/0 conn(0x7f40e406d960 0x7f40e40d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.906+0000 7f40e9c8b640 1 --2- 172.21.15.16:0/3720250849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e415e2a0 0x7f40e406d420 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.906+0000 7f40e9c8b640 1 -- 172.21.15.16:0/3720250849 >> 172.21.15.16:0/3720250849 conn(0x7f40e409d760 msgr2=0x7f40e408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.906+0000 7f40e9c8b640 1 -- 172.21.15.16:0/3720250849 shutdown_connections 2024-09-19T07:52:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.907+0000 7f40e9c8b640 1 -- 172.21.15.16:0/3720250849 wait complete. 2024-09-19T07:52:12.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.908+0000 7f40e9c8b640 1 Processor -- start 2024-09-19T07:52:12.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.908+0000 7f40e9c8b640 1 -- start start 2024-09-19T07:52:12.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.909+0000 7f40e9c8b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40e406d960 0x7f40e415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.909+0000 7f40e9c8b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40e415e2a0 0x7f40e409e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.909+0000 7f40e9c8b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e415e680 0x7f40e409ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.909+0000 7f40e9c8b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40e40d0360 con 0x7f40e415e680 2024-09-19T07:52:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.909+0000 7f40e9c8b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f40e40d01e0 con 0x7f40e415e2a0 2024-09-19T07:52:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.909+0000 7f40e9c8b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f40e40d48b0 con 0x7f40e406d960 2024-09-19T07:52:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.909+0000 7f40e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e415e680 0x7f40e409ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.909+0000 7f40e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e415e680 0x7f40e409ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42908/0 (socket says 172.21.15.16:42908) 2024-09-19T07:52:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.909+0000 7f40e3fff640 1 -- 172.21.15.16:0/2444212075 learned_addr learned my addr 172.21.15.16:0/2444212075 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.909+0000 7f40e2ffd640 1 --2- 172.21.15.16:0/2444212075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40e415e2a0 0x7f40e409e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.909+0000 7f40e37fe640 1 --2- 172.21.15.16:0/2444212075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40e406d960 0x7f40e415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.909+0000 7f40e3fff640 1 -- 172.21.15.16:0/2444212075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40e406d960 msgr2=0x7f40e415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.909+0000 7f40e3fff640 1 --2- 172.21.15.16:0/2444212075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40e406d960 0x7f40e415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.909+0000 7f40e3fff640 1 -- 172.21.15.16:0/2444212075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40e415e2a0 msgr2=0x7f40e409e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.909+0000 7f40e3fff640 1 --2- 172.21.15.16:0/2444212075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40e415e2a0 0x7f40e409e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.909+0000 7f40e3fff640 1 -- 172.21.15.16:0/2444212075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e40ce8f0 con 0x7f40e415e680 2024-09-19T07:52:12.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.909+0000 7f40e3fff640 1 --2- 172.21.15.16:0/2444212075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e415e680 0x7f40e409ed10 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f40dc007e70 tx=0x7f40dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:12.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.910+0000 7f40cbfff640 1 -- 172.21.15.16:0/2444212075 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc014420 con 0x7f40e415e680 2024-09-19T07:52:12.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.910+0000 7f40cbfff640 1 -- 172.21.15.16:0/2444212075 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40dc019070 con 0x7f40e415e680 2024-09-19T07:52:12.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.910+0000 7f40e9c8b640 1 -- 172.21.15.16:0/2444212075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e409f250 con 0x7f40e415e680 2024-09-19T07:52:12.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.910+0000 7f40cbfff640 1 -- 172.21.15.16:0/2444212075 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc014820 con 0x7f40e415e680 2024-09-19T07:52:12.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.910+0000 7f40e9c8b640 1 -- 172.21.15.16:0/2444212075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e409f7d0 con 0x7f40e415e680 2024-09-19T07:52:12.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.911+0000 7f40e9c8b640 1 -- 172.21.15.16:0/2444212075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40e40d4c70 con 0x7f40e415e680 2024-09-19T07:52:12.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.911+0000 7f40cbfff640 1 -- 172.21.15.16:0/2444212075 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f40dc0149c0 con 0x7f40e415e680 2024-09-19T07:52:12.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.912+0000 7f40cbfff640 1 --2- 172.21.15.16:0/2444212075 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f40d006c2b0 0x7f40d006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.912+0000 7f40cbfff640 1 -- 172.21.15.16:0/2444212075 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f40dc08bd70 con 0x7f40e415e680 2024-09-19T07:52:12.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.912+0000 7f40e37fe640 1 --2- 172.21.15.16:0/2444212075 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f40d006c2b0 0x7f40d006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.913+0000 7f40e37fe640 1 --2- 172.21.15.16:0/2444212075 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f40d006c2b0 0x7f40d006e770 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f40d4034000 tx=0x7f40d402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:12.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:12.917+0000 7f40cbfff640 1 -- 172.21.15.16:0/2444212075 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f40dc059c80 con 0x7f40e415e680 2024-09-19T07:52:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:13.133+0000 7f40e9c8b640 1 -- 172.21.15.16:0/2444212075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f40e406cf90 con 0x7f40e415e680 2024-09-19T07:52:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:13.134+0000 7f40cbfff640 1 -- 172.21.15.16:0/2444212075 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f40dc05df80 con 0x7f40e415e680 2024-09-19T07:52:13.134 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:52:13.134 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:52:13.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:13.136+0000 7f40e9c8b640 1 -- 172.21.15.16:0/2444212075 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f40d006c2b0 msgr2=0x7f40d006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:13.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:13.136+0000 7f40e9c8b640 1 --2- 172.21.15.16:0/2444212075 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f40d006c2b0 0x7f40d006e770 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f40d4034000 tx=0x7f40d402e000 comp rx=0 tx=0).stop 2024-09-19T07:52:13.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:13.136+0000 7f40e9c8b640 1 -- 172.21.15.16:0/2444212075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e415e680 msgr2=0x7f40e409ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:13.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:13.136+0000 7f40e9c8b640 1 --2- 172.21.15.16:0/2444212075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e415e680 0x7f40e409ed10 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f40dc007e70 tx=0x7f40dc00e4b0 comp rx=0 tx=0).stop 2024-09-19T07:52:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:13.136+0000 7f40e9c8b640 1 -- 172.21.15.16:0/2444212075 shutdown_connections 2024-09-19T07:52:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:13.136+0000 7f40e9c8b640 1 --2- 172.21.15.16:0/2444212075 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f40d006c2b0 0x7f40d006e770 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:13.136+0000 7f40e9c8b640 1 --2- 172.21.15.16:0/2444212075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e415e680 0x7f40e409ed10 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:13.136+0000 7f40e9c8b640 1 --2- 172.21.15.16:0/2444212075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40e415e2a0 0x7f40e409e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:13.136+0000 7f40e9c8b640 1 --2- 172.21.15.16:0/2444212075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40e406d960 0x7f40e415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:13.136+0000 7f40e9c8b640 1 -- 172.21.15.16:0/2444212075 >> 172.21.15.16:0/2444212075 conn(0x7f40e409d760 msgr2=0x7f40e4160ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:13.136+0000 7f40e9c8b640 1 -- 172.21.15.16:0/2444212075 shutdown_connections 2024-09-19T07:52:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:13.136+0000 7f40e9c8b640 1 -- 172.21.15.16:0/2444212075 wait complete. 2024-09-19T07:52:13.147 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:52:13.147 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:52:14.818 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:16.819 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:18.148 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:52:18.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.262+0000 7f97166f1640 1 Processor -- start 2024-09-19T07:52:18.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.262+0000 7f97166f1640 1 -- start start 2024-09-19T07:52:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f97166f1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97101619b0 0x7f97100d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f97166f1640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f97100b0a80 0x7f97100d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f97166f1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f971006c990 con 0x7f9710161d90 2024-09-19T07:52:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f97166f1640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f97100d0ec0 con 0x7f97100b0a80 2024-09-19T07:52:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f97166f1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f97100cec10 con 0x7f97101619b0 2024-09-19T07:52:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f9714c8e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9710161d90 0x7f97100cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42620/0 (socket says 172.21.15.16:42620) 2024-09-19T07:52:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f9714c8e640 1 -- 172.21.15.16:0/2476396575 learned_addr learned my addr 172.21.15.16:0/2476396575 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:52:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f970ffff640 1 --2- 172.21.15.16:0/2476396575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97101619b0 0x7f97100d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f970effd640 1 -- 172.21.15.16:0/2476396575 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1041065278 0 0) 0x7f971006c990 con 0x7f9710161d90 2024-09-19T07:52:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f970effd640 1 -- 172.21.15.16:0/2476396575 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96f8003520 con 0x7f9710161d90 2024-09-19T07:52:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f970f7fe640 1 --2- 172.21.15.16:0/2476396575 >> v2:172.21.15.146:3300/0 conn(0x7f97100b0a80 0x7f97100d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f970effd640 1 -- 172.21.15.16:0/2476396575 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1672718789 0 0) 0x7f96f8003520 con 0x7f9710161d90 2024-09-19T07:52:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f970effd640 1 -- 172.21.15.16:0/2476396575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97101619b0 msgr2=0x7f97100d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f970effd640 1 --2- 172.21.15.16:0/2476396575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97101619b0 0x7f97100d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f970effd640 1 -- 172.21.15.16:0/2476396575 >> v2:172.21.15.146:3300/0 conn(0x7f97100b0a80 msgr2=0x7f97100d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f970effd640 1 --2- 172.21.15.16:0/2476396575 >> v2:172.21.15.146:3300/0 conn(0x7f97100b0a80 0x7f97100d0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f970effd640 1 -- 172.21.15.16:0/2476396575 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97100ce930 con 0x7f9710161d90 2024-09-19T07:52:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f970effd640 1 -- 172.21.15.16:0/2476396575 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f97080019f0 con 0x7f9710161d90 2024-09-19T07:52:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f970effd640 1 -- 172.21.15.16:0/2476396575 >> v1:172.21.15.16:6789/0 conn(0x7f9710161d90 legacy=0x7f97100cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.263+0000 7f970effd640 1 --2- 172.21.15.16:0/2476396575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96f80039b0 0x7f96f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.264+0000 7f970effd640 1 --2- 172.21.15.16:0/2476396575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97101619b0 0x7f96f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.264+0000 7f970effd640 1 --2- 172.21.15.16:0/2476396575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96f8004d40 0x7f96f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.264+0000 7f970effd640 1 -- 172.21.15.16:0/2476396575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97100d0ec0 con 0x7f96f8004d40 2024-09-19T07:52:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.264+0000 7f970effd640 1 -- 172.21.15.16:0/2476396575 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f97100cec10 con 0x7f96f80039b0 2024-09-19T07:52:18.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.264+0000 7f970effd640 1 -- 172.21.15.16:0/2476396575 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f96f800b9c0 con 0x7f97101619b0 2024-09-19T07:52:18.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.264+0000 7f9714c8e640 1 --2- 172.21.15.16:0/2476396575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96f8004d40 0x7f96f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:18.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.264+0000 7f970ffff640 1 --2- 172.21.15.16:0/2476396575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96f80039b0 0x7f96f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:18.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.264+0000 7f970f7fe640 1 --2- 172.21.15.16:0/2476396575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97101619b0 0x7f96f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:18.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.264+0000 7f9714c8e640 1 -- 172.21.15.16:0/2476396575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97101619b0 msgr2=0x7f96f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:18.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.264+0000 7f9714c8e640 1 --2- 172.21.15.16:0/2476396575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97101619b0 0x7f96f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:18.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.264+0000 7f9714c8e640 1 -- 172.21.15.16:0/2476396575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96f80039b0 msgr2=0x7f96f8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:18.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.264+0000 7f9714c8e640 1 --2- 172.21.15.16:0/2476396575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96f80039b0 0x7f96f8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:18.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.264+0000 7f9714c8e640 1 -- 172.21.15.16:0/2476396575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f8003520 con 0x7f96f8004d40 2024-09-19T07:52:18.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.265+0000 7f9714c8e640 1 --2- 172.21.15.16:0/2476396575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96f8004d40 0x7f96f80051c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f970800a880 tx=0x7f96f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76156aff17f32c14 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:18.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.265+0000 7f970effd640 1 -- 172.21.15.16:0/2476396575 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970800a4a0 con 0x7f96f8004d40 2024-09-19T07:52:18.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.265+0000 7f970effd640 1 -- 172.21.15.16:0/2476396575 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f970802f070 con 0x7f96f8004d40 2024-09-19T07:52:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.265+0000 7f970effd640 1 -- 172.21.15.16:0/2476396575 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970802bd20 con 0x7f96f8004d40 2024-09-19T07:52:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.265+0000 7f97166f1640 1 -- 172.21.15.16:0/2476396575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96f8004d40 msgr2=0x7f96f80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.265+0000 7f97166f1640 1 --2- 172.21.15.16:0/2476396575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96f8004d40 0x7f96f80051c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f970800a880 tx=0x7f96f8005830 comp rx=0 tx=0).stop 2024-09-19T07:52:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.265+0000 7f9714c8e640 1 -- 172.21.15.16:0/2476396575 reap_dead start 2024-09-19T07:52:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.265+0000 7f97166f1640 1 -- 172.21.15.16:0/2476396575 shutdown_connections 2024-09-19T07:52:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.265+0000 7f97166f1640 1 -- 172.21.15.16:0/2476396575 >> 172.21.15.16:0/2476396575 conn(0x7f971009a9f0 msgr2=0x7f97100b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.266+0000 7f97166f1640 1 -- 172.21.15.16:0/2476396575 shutdown_connections 2024-09-19T07:52:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.266+0000 7f97166f1640 1 -- 172.21.15.16:0/2476396575 wait complete. 2024-09-19T07:52:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.267+0000 7f97166f1640 1 Processor -- start 2024-09-19T07:52:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.267+0000 7f97166f1640 1 -- start start 2024-09-19T07:52:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.268+0000 7f97166f1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96f8004d40 0x7f97100c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.268+0000 7f97166f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97100b66d0 0x7f971015b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.268+0000 7f97166f1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97100cbbd0 0x7f97100cc050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.268+0000 7f97166f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f971006c620 con 0x7f97100b66d0 2024-09-19T07:52:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.268+0000 7f97166f1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f97100b0820 con 0x7f97100cbbd0 2024-09-19T07:52:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.268+0000 7f97166f1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f97100ccce0 con 0x7f96f8004d40 2024-09-19T07:52:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.268+0000 7f970f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97100b66d0 0x7f971015b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.268+0000 7f970f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97100b66d0 0x7f971015b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42940/0 (socket says 172.21.15.16:42940) 2024-09-19T07:52:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.268+0000 7f970f7fe640 1 -- 172.21.15.16:0/1436099196 learned_addr learned my addr 172.21.15.16:0/1436099196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.268+0000 7f970f7fe640 1 -- 172.21.15.16:0/1436099196 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96f8004d40 msgr2=0x7f97100c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.268+0000 7f970ffff640 1 --2- 172.21.15.16:0/1436099196 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96f8004d40 0x7f97100c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.268+0000 7f9714c8e640 1 --2- 172.21.15.16:0/1436099196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97100cbbd0 0x7f97100cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.268+0000 7f970f7fe640 1 --2- 172.21.15.16:0/1436099196 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96f8004d40 0x7f97100c78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.268+0000 7f970f7fe640 1 -- 172.21.15.16:0/1436099196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97100cbbd0 msgr2=0x7f97100cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.268+0000 7f970f7fe640 1 --2- 172.21.15.16:0/1436099196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97100cbbd0 0x7f97100cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.268+0000 7f970f7fe640 1 -- 172.21.15.16:0/1436099196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97100b6e70 con 0x7f97100b66d0 2024-09-19T07:52:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.268+0000 7f970ffff640 1 --2- 172.21.15.16:0/1436099196 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96f8004d40 0x7f97100c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:52:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.268+0000 7f9714c8e640 1 --2- 172.21.15.16:0/1436099196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97100cbbd0 0x7f97100cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:52:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.268+0000 7f970f7fe640 1 --2- 172.21.15.16:0/1436099196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97100b66d0 0x7f971015b850 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9704011960 tx=0x7f9704011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.269+0000 7f970cff9640 1 -- 172.21.15.16:0/1436099196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704019070 con 0x7f97100b66d0 2024-09-19T07:52:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.269+0000 7f970cff9640 1 -- 172.21.15.16:0/1436099196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9704004240 con 0x7f97100b66d0 2024-09-19T07:52:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.269+0000 7f970cff9640 1 -- 172.21.15.16:0/1436099196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970400f8e0 con 0x7f97100b66d0 2024-09-19T07:52:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.269+0000 7f97166f1640 1 -- 172.21.15.16:0/1436099196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97100b7150 con 0x7f97100b66d0 2024-09-19T07:52:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.269+0000 7f97166f1640 1 -- 172.21.15.16:0/1436099196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97100c51d0 con 0x7f97100b66d0 2024-09-19T07:52:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.270+0000 7f97166f1640 1 -- 172.21.15.16:0/1436099196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97100cd330 con 0x7f97100b66d0 2024-09-19T07:52:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.270+0000 7f970cff9640 1 -- 172.21.15.16:0/1436099196 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f970401c070 con 0x7f97100b66d0 2024-09-19T07:52:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.271+0000 7f970cff9640 1 --2- 172.21.15.16:0/1436099196 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f96fc06c2b0 0x7f96fc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.271+0000 7f970cff9640 1 -- 172.21.15.16:0/1436099196 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f970408bbe0 con 0x7f97100b66d0 2024-09-19T07:52:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.271+0000 7f970ffff640 1 --2- 172.21.15.16:0/1436099196 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f96fc06c2b0 0x7f96fc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.272+0000 7f970ffff640 1 --2- 172.21.15.16:0/1436099196 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f96fc06c2b0 0x7f96fc06e770 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9700009a40 tx=0x7f9700008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:18.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.275+0000 7f970cff9640 1 -- 172.21.15.16:0/1436099196 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f970405a0b0 con 0x7f97100b66d0 2024-09-19T07:52:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.488+0000 7f97166f1640 1 -- 172.21.15.16:0/1436099196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f97100cd5b0 con 0x7f97100b66d0 2024-09-19T07:52:18.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.489+0000 7f970cff9640 1 -- 172.21.15.16:0/1436099196 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f97040599b0 con 0x7f97100b66d0 2024-09-19T07:52:18.489 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:52:18.489 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:52:18.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.491+0000 7f97166f1640 1 -- 172.21.15.16:0/1436099196 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f96fc06c2b0 msgr2=0x7f96fc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:18.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.491+0000 7f97166f1640 1 --2- 172.21.15.16:0/1436099196 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f96fc06c2b0 0x7f96fc06e770 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9700009a40 tx=0x7f9700008040 comp rx=0 tx=0).stop 2024-09-19T07:52:18.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.491+0000 7f97166f1640 1 -- 172.21.15.16:0/1436099196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97100b66d0 msgr2=0x7f971015b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:18.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.491+0000 7f97166f1640 1 --2- 172.21.15.16:0/1436099196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97100b66d0 0x7f971015b850 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9704011960 tx=0x7f9704011e20 comp rx=0 tx=0).stop 2024-09-19T07:52:18.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.491+0000 7f97166f1640 1 -- 172.21.15.16:0/1436099196 shutdown_connections 2024-09-19T07:52:18.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.491+0000 7f97166f1640 1 --2- 172.21.15.16:0/1436099196 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f96fc06c2b0 0x7f96fc06e770 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:18.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.491+0000 7f97166f1640 1 --2- 172.21.15.16:0/1436099196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97100cbbd0 0x7f97100cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:18.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.491+0000 7f97166f1640 1 --2- 172.21.15.16:0/1436099196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97100b66d0 0x7f971015b850 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:18.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.491+0000 7f97166f1640 1 --2- 172.21.15.16:0/1436099196 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96f8004d40 0x7f97100c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:18.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.491+0000 7f97166f1640 1 -- 172.21.15.16:0/1436099196 >> 172.21.15.16:0/1436099196 conn(0x7f971009a9f0 msgr2=0x7f97100af0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:18.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.491+0000 7f97166f1640 1 -- 172.21.15.16:0/1436099196 shutdown_connections 2024-09-19T07:52:18.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:18.491+0000 7f97166f1640 1 -- 172.21.15.16:0/1436099196 wait complete. 2024-09-19T07:52:18.502 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:52:18.502 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:52:18.821 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:20.823 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:23.503 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:52:23.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.617+0000 7fb4108f8640 1 Processor -- start 2024-09-19T07:52:23.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.617+0000 7fb4108f8640 1 -- start start 2024-09-19T07:52:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.618+0000 7fb4108f8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb40c15e2a0 0x7fb40c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.618+0000 7fb4108f8640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb40c06d960 0x7fb40c0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.618+0000 7fb4108f8640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb40c0d1880 con 0x7fb40c15e680 2024-09-19T07:52:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.618+0000 7fb4108f8640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb40c0d1b60 con 0x7fb40c06d960 2024-09-19T07:52:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.618+0000 7fb4108f8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb40c0cf2d0 con 0x7fb40c15e2a0 2024-09-19T07:52:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.618+0000 7fb40ad9d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb40c06d960 0x7fb40c0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.618+0000 7fb40a59c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb40c15e2a0 0x7fb40c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.618+0000 7fb40a59c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb40c15e2a0 0x7fb40c06d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:60656/0 (socket says 172.21.15.16:60656) 2024-09-19T07:52:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.618+0000 7fb409d9b640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fb40c15e680 0x7fb40c0cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:48064/0 (socket says 172.21.15.16:48064) 2024-09-19T07:52:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.618+0000 7fb40ad9d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb40c06d960 0x7fb40c0d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:48186/0 (socket says 172.21.15.16:48186) 2024-09-19T07:52:23.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.618+0000 7fb40a59c640 1 -- 172.21.15.16:0/1698254233 learned_addr learned my addr 172.21.15.16:0/1698254233 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:23.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.619+0000 7fb40a59c640 1 -- 172.21.15.16:0/1698254233 >> v2:172.21.15.146:3300/0 conn(0x7fb40c06d960 msgr2=0x7fb40c0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:23.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.619+0000 7fb40a59c640 1 --2- 172.21.15.16:0/1698254233 >> v2:172.21.15.146:3300/0 conn(0x7fb40c06d960 0x7fb40c0d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:23.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.619+0000 7fb40a59c640 1 -- 172.21.15.16:0/1698254233 >> v1:172.21.15.146:6789/0 conn(0x7fb40c15e680 legacy=0x7fb40c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:23.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.619+0000 7fb40a59c640 1 -- 172.21.15.16:0/1698254233 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c0cf5b0 con 0x7fb40c15e2a0 2024-09-19T07:52:23.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.619+0000 7fb40a59c640 1 --2- 172.21.15.16:0/1698254233 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb40c15e2a0 0x7fb40c06d420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc009870 tx=0x7fb3fc02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfc89016d1b8e1af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:23.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.620+0000 7fb40959a640 1 -- 172.21.15.16:0/1698254233 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc037070 con 0x7fb40c15e2a0 2024-09-19T07:52:23.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.620+0000 7fb40959a640 1 -- 172.21.15.16:0/1698254233 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3fc032440 con 0x7fb40c15e2a0 2024-09-19T07:52:23.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.620+0000 7fb40959a640 1 -- 172.21.15.16:0/1698254233 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc032810 con 0x7fb40c15e2a0 2024-09-19T07:52:23.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.620+0000 7fb4108f8640 1 -- 172.21.15.16:0/1698254233 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb40c15e2a0 msgr2=0x7fb40c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:23.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.620+0000 7fb4108f8640 1 --2- 172.21.15.16:0/1698254233 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb40c15e2a0 0x7fb40c06d420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc009870 tx=0x7fb3fc02a270 comp rx=0 tx=0).stop 2024-09-19T07:52:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.620+0000 7fb4108f8640 1 -- 172.21.15.16:0/1698254233 shutdown_connections 2024-09-19T07:52:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.620+0000 7fb4108f8640 1 --2- 172.21.15.16:0/1698254233 >> v2:172.21.15.146:3300/0 conn(0x7fb40c06d960 0x7fb40c0d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.620+0000 7fb4108f8640 1 --2- 172.21.15.16:0/1698254233 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb40c15e2a0 0x7fb40c06d420 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.620+0000 7fb4108f8640 1 -- 172.21.15.16:0/1698254233 >> 172.21.15.16:0/1698254233 conn(0x7fb40c09d760 msgr2=0x7fb40c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.620+0000 7fb4108f8640 1 -- 172.21.15.16:0/1698254233 shutdown_connections 2024-09-19T07:52:23.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.620+0000 7fb4108f8640 1 -- 172.21.15.16:0/1698254233 wait complete. 2024-09-19T07:52:23.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.622+0000 7fb4108f8640 1 Processor -- start 2024-09-19T07:52:23.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.622+0000 7fb4108f8640 1 -- start start 2024-09-19T07:52:23.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb4108f8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb40c06d960 0x7fb40c158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:23.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb4108f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40c15e2a0 0x7fb40c0cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:23.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb4108f8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb40c15e680 0x7fb40c156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:23.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb4108f8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c0d0360 con 0x7fb40c15e2a0 2024-09-19T07:52:23.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb4108f8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c0d01e0 con 0x7fb40c15e680 2024-09-19T07:52:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb4108f8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb40c0d48b0 con 0x7fb40c06d960 2024-09-19T07:52:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb409d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40c15e2a0 0x7fb40c0cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb409d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40c15e2a0 0x7fb40c0cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33596/0 (socket says 172.21.15.16:33596) 2024-09-19T07:52:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb409d9b640 1 -- 172.21.15.16:0/3448443792 learned_addr learned my addr 172.21.15.16:0/3448443792 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb40a59c640 1 --2- 172.21.15.16:0/3448443792 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb40c06d960 0x7fb40c158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb40ad9d640 1 --2- 172.21.15.16:0/3448443792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb40c15e680 0x7fb40c156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb409d9b640 1 -- 172.21.15.16:0/3448443792 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb40c06d960 msgr2=0x7fb40c158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb409d9b640 1 --2- 172.21.15.16:0/3448443792 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb40c06d960 0x7fb40c158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb409d9b640 1 -- 172.21.15.16:0/3448443792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb40c15e680 msgr2=0x7fb40c156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb409d9b640 1 --2- 172.21.15.16:0/3448443792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb40c15e680 0x7fb40c156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb409d9b640 1 -- 172.21.15.16:0/3448443792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c156e50 con 0x7fb40c15e2a0 2024-09-19T07:52:23.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb409d9b640 1 --2- 172.21.15.16:0/3448443792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40c15e2a0 0x7fb40c0cc390 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb40000c390 tx=0x7fb40000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb3faffd640 1 -- 172.21.15.16:0/3448443792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4000074e0 con 0x7fb40c15e2a0 2024-09-19T07:52:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb3faffd640 1 -- 172.21.15.16:0/3448443792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4000047e0 con 0x7fb40c15e2a0 2024-09-19T07:52:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb3faffd640 1 -- 172.21.15.16:0/3448443792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400004bd0 con 0x7fb40c15e2a0 2024-09-19T07:52:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.623+0000 7fb4108f8640 1 -- 172.21.15.16:0/3448443792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40c158770 con 0x7fb40c15e2a0 2024-09-19T07:52:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.624+0000 7fb4108f8640 1 -- 172.21.15.16:0/3448443792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40c158d30 con 0x7fb40c15e2a0 2024-09-19T07:52:23.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.625+0000 7fb4108f8640 1 -- 172.21.15.16:0/3448443792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb40c0d4c70 con 0x7fb40c15e2a0 2024-09-19T07:52:23.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.625+0000 7fb3faffd640 1 -- 172.21.15.16:0/3448443792 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb40001a020 con 0x7fb40c15e2a0 2024-09-19T07:52:23.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.626+0000 7fb3faffd640 1 --2- 172.21.15.16:0/3448443792 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb3ec06c2b0 0x7fb3ec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:23.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.626+0000 7fb3faffd640 1 -- 172.21.15.16:0/3448443792 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7fb40008ad50 con 0x7fb40c15e2a0 2024-09-19T07:52:23.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.626+0000 7fb40a59c640 1 --2- 172.21.15.16:0/3448443792 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb3ec06c2b0 0x7fb3ec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:23.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.626+0000 7fb40a59c640 1 --2- 172.21.15.16:0/3448443792 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb3ec06c2b0 0x7fb3ec06e770 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc034000 tx=0x7fb3fc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.630+0000 7fb3faffd640 1 -- 172.21.15.16:0/3448443792 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb40c156e50 con 0x7fb40c15e2a0 2024-09-19T07:52:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.848+0000 7fb4108f8640 1 -- 172.21.15.16:0/3448443792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb40c06cf90 con 0x7fb40c15e2a0 2024-09-19T07:52:23.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.849+0000 7fb3faffd640 1 -- 172.21.15.16:0/3448443792 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fb400059220 con 0x7fb40c15e2a0 2024-09-19T07:52:23.849 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:52:23.849 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:52:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.851+0000 7fb4108f8640 1 -- 172.21.15.16:0/3448443792 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb3ec06c2b0 msgr2=0x7fb3ec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.851+0000 7fb4108f8640 1 --2- 172.21.15.16:0/3448443792 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb3ec06c2b0 0x7fb3ec06e770 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc034000 tx=0x7fb3fc02e000 comp rx=0 tx=0).stop 2024-09-19T07:52:23.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.851+0000 7fb4108f8640 1 -- 172.21.15.16:0/3448443792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40c15e2a0 msgr2=0x7fb40c0cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:23.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.851+0000 7fb4108f8640 1 --2- 172.21.15.16:0/3448443792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40c15e2a0 0x7fb40c0cc390 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb40000c390 tx=0x7fb40000c790 comp rx=0 tx=0).stop 2024-09-19T07:52:23.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.851+0000 7fb4108f8640 1 -- 172.21.15.16:0/3448443792 shutdown_connections 2024-09-19T07:52:23.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.851+0000 7fb4108f8640 1 --2- 172.21.15.16:0/3448443792 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb3ec06c2b0 0x7fb3ec06e770 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:23.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.851+0000 7fb4108f8640 1 --2- 172.21.15.16:0/3448443792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb40c15e680 0x7fb40c156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:23.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.851+0000 7fb4108f8640 1 --2- 172.21.15.16:0/3448443792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40c15e2a0 0x7fb40c0cc390 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:23.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.851+0000 7fb4108f8640 1 --2- 172.21.15.16:0/3448443792 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb40c06d960 0x7fb40c158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:23.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.851+0000 7fb4108f8640 1 -- 172.21.15.16:0/3448443792 >> 172.21.15.16:0/3448443792 conn(0x7fb40c09d760 msgr2=0x7fb40c0d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:23.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.851+0000 7fb4108f8640 1 -- 172.21.15.16:0/3448443792 shutdown_connections 2024-09-19T07:52:23.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:23.851+0000 7fb4108f8640 1 -- 172.21.15.16:0/3448443792 wait complete. 2024-09-19T07:52:23.862 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:52:23.862 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:52:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:26.829 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:28.832 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:28.862 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:52:28.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.976+0000 7f0279247640 1 Processor -- start 2024-09-19T07:52:28.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.976+0000 7f0279247640 1 -- start start 2024-09-19T07:52:28.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.976+0000 7f0279247640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02740d21f0 0x7f02740d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:28.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.976+0000 7f0279247640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02740d2b10 con 0x7f0274141ad0 2024-09-19T07:52:28.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.976+0000 7f0279247640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02741620b0 con 0x7f0274072590 2024-09-19T07:52:28.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.976+0000 7f0279247640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0274141750 con 0x7f02740d21f0 2024-09-19T07:52:28.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.977+0000 7f027359e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0274141ad0 0x7f0274141f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44780/0 (socket says 172.21.15.16:44780) 2024-09-19T07:52:28.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.977+0000 7f027359e640 1 -- 172.21.15.16:0/268479204 learned_addr learned my addr 172.21.15.16:0/268479204 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:52:28.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.977+0000 7f0272d9d640 1 --2- 172.21.15.16:0/268479204 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02740d21f0 0x7f02740d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:28.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.977+0000 7f0271d9b640 1 -- 172.21.15.16:0/268479204 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 158024073 0 0) 0x7f02740d2b10 con 0x7f0274141ad0 2024-09-19T07:52:28.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.977+0000 7f0271d9b640 1 -- 172.21.15.16:0/268479204 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0260003520 con 0x7f0274141ad0 2024-09-19T07:52:28.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.977+0000 7f0271d9b640 1 -- 172.21.15.16:0/268479204 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 851555900 0 0) 0x7f02741620b0 con 0x7f0274072590 2024-09-19T07:52:28.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.977+0000 7f0271d9b640 1 -- 172.21.15.16:0/268479204 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02740d2b10 con 0x7f0274072590 2024-09-19T07:52:28.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.977+0000 7f0271d9b640 1 -- 172.21.15.16:0/268479204 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2672484258 0 0) 0x7f0260003520 con 0x7f0274141ad0 2024-09-19T07:52:28.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.977+0000 7f0272d9d640 1 -- 172.21.15.16:0/268479204 >> v1:172.21.15.146:6789/0 conn(0x7f0274072590 legacy=0x7f027406d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:28.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.977+0000 7f0272d9d640 1 -- 172.21.15.16:0/268479204 >> v1:172.21.15.16:6789/0 conn(0x7f0274141ad0 legacy=0x7f0274141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:28.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.977+0000 7f0272d9d640 1 -- 172.21.15.16:0/268479204 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0274162370 con 0x7f02740d21f0 2024-09-19T07:52:28.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.978+0000 7f0272d9d640 1 --2- 172.21.15.16:0/268479204 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02740d21f0 0x7f02740d25d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f026c0097d0 tx=0x7f026c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f996395204cabc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:28.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.978+0000 7f0271d9b640 1 -- 172.21.15.16:0/268479204 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c037070 con 0x7f02740d21f0 2024-09-19T07:52:28.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.978+0000 7f0271d9b640 1 -- 172.21.15.16:0/268479204 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f026c02ab90 con 0x7f02740d21f0 2024-09-19T07:52:28.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.978+0000 7f0271d9b640 1 -- 172.21.15.16:0/268479204 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02640019f0 con 0x7f02740d21f0 2024-09-19T07:52:28.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.979+0000 7f0279247640 1 -- 172.21.15.16:0/268479204 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02740d21f0 msgr2=0x7f02740d25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:28.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.979+0000 7f0279247640 1 --2- 172.21.15.16:0/268479204 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02740d21f0 0x7f02740d25d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f026c0097d0 tx=0x7f026c02a100 comp rx=0 tx=0).stop 2024-09-19T07:52:28.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.979+0000 7f0279247640 1 -- 172.21.15.16:0/268479204 shutdown_connections 2024-09-19T07:52:28.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.979+0000 7f0279247640 1 --2- 172.21.15.16:0/268479204 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02740d21f0 0x7f02740d25d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:28.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.979+0000 7f0279247640 1 -- 172.21.15.16:0/268479204 >> 172.21.15.16:0/268479204 conn(0x7f02740b3ed0 msgr2=0x7f02740b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:28.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.979+0000 7f0279247640 1 -- 172.21.15.16:0/268479204 shutdown_connections 2024-09-19T07:52:28.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.979+0000 7f0279247640 1 -- 172.21.15.16:0/268479204 wait complete. 2024-09-19T07:52:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.981+0000 7f0279247640 1 Processor -- start 2024-09-19T07:52:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.981+0000 7f0279247640 1 -- start start 2024-09-19T07:52:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.981+0000 7f0279247640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0274072590 0x7f0274159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.981+0000 7f0279247640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02740d21f0 0x7f0274154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.981+0000 7f0279247640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0274141ad0 0x7f0274140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.981+0000 7f0279247640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0274162930 con 0x7f0274072590 2024-09-19T07:52:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.981+0000 7f0279247640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f027406d7f0 con 0x7f0274141ad0 2024-09-19T07:52:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.981+0000 7f0279247640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0274162c10 con 0x7f02740d21f0 2024-09-19T07:52:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.982+0000 7f0272d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0274072590 0x7f0274159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.982+0000 7f0272d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0274072590 0x7f0274159850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33604/0 (socket says 172.21.15.16:33604) 2024-09-19T07:52:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.982+0000 7f0272d9d640 1 -- 172.21.15.16:0/3619542324 learned_addr learned my addr 172.21.15.16:0/3619542324 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.982+0000 7f027359e640 1 --2- 172.21.15.16:0/3619542324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0274141ad0 0x7f0274140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.982+0000 7f027259c640 1 --2- 172.21.15.16:0/3619542324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02740d21f0 0x7f0274154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.982+0000 7f0272d9d640 1 -- 172.21.15.16:0/3619542324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02740d21f0 msgr2=0x7f0274154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.982+0000 7f0272d9d640 1 --2- 172.21.15.16:0/3619542324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02740d21f0 0x7f0274154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.982+0000 7f0272d9d640 1 -- 172.21.15.16:0/3619542324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0274141ad0 msgr2=0x7f0274140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:28.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.982+0000 7f0272d9d640 1 --2- 172.21.15.16:0/3619542324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0274141ad0 0x7f0274140c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:28.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.982+0000 7f0272d9d640 1 -- 172.21.15.16:0/3619542324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02741548a0 con 0x7f0274072590 2024-09-19T07:52:28.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.982+0000 7f027359e640 1 --2- 172.21.15.16:0/3619542324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0274141ad0 0x7f0274140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:52:28.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.982+0000 7f0272d9d640 1 --2- 172.21.15.16:0/3619542324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0274072590 0x7f0274159850 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f026c0097d0 tx=0x7f026c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:28.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.982+0000 7f026b7fe640 1 -- 172.21.15.16:0/3619542324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c042020 con 0x7f0274072590 2024-09-19T07:52:28.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.982+0000 7f026b7fe640 1 -- 172.21.15.16:0/3619542324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f026c03d670 con 0x7f0274072590 2024-09-19T07:52:28.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.982+0000 7f026b7fe640 1 -- 172.21.15.16:0/3619542324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c037040 con 0x7f0274072590 2024-09-19T07:52:28.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.982+0000 7f0279247640 1 -- 172.21.15.16:0/3619542324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0274151620 con 0x7f0274072590 2024-09-19T07:52:28.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.983+0000 7f0279247640 1 -- 172.21.15.16:0/3619542324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0274151ec0 con 0x7f0274072590 2024-09-19T07:52:28.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.983+0000 7f0279247640 1 -- 172.21.15.16:0/3619542324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0274158c30 con 0x7f0274072590 2024-09-19T07:52:28.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.984+0000 7f026b7fe640 1 -- 172.21.15.16:0/3619542324 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f026c045020 con 0x7f0274072590 2024-09-19T07:52:28.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.984+0000 7f026b7fe640 1 --2- 172.21.15.16:0/3619542324 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f025406c2b0 0x7f025406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:28.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.985+0000 7f026b7fe640 1 -- 172.21.15.16:0/3619542324 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f026c0abfa0 con 0x7f0274072590 2024-09-19T07:52:28.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.985+0000 7f027259c640 1 --2- 172.21.15.16:0/3619542324 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f025406c2b0 0x7f025406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:28.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.985+0000 7f027259c640 1 --2- 172.21.15.16:0/3619542324 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f025406c2b0 0x7f025406e770 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f02740b7740 tx=0x7f025c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:28.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:28.989+0000 7f026b7fe640 1 -- 172.21.15.16:0/3619542324 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f026c04c030 con 0x7f0274072590 2024-09-19T07:52:29.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:29.203+0000 7f0279247640 1 -- 172.21.15.16:0/3619542324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0274072290 con 0x7f0274072590 2024-09-19T07:52:29.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:29.204+0000 7f026b7fe640 1 -- 172.21.15.16:0/3619542324 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f026c07a470 con 0x7f0274072590 2024-09-19T07:52:29.204 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:52:29.204 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:52:29.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:29.205+0000 7f0279247640 1 -- 172.21.15.16:0/3619542324 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f025406c2b0 msgr2=0x7f025406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:29.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:29.205+0000 7f0279247640 1 --2- 172.21.15.16:0/3619542324 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f025406c2b0 0x7f025406e770 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f02740b7740 tx=0x7f025c009040 comp rx=0 tx=0).stop 2024-09-19T07:52:29.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:29.205+0000 7f0279247640 1 -- 172.21.15.16:0/3619542324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0274072590 msgr2=0x7f0274159850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:29.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:29.205+0000 7f0279247640 1 --2- 172.21.15.16:0/3619542324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0274072590 0x7f0274159850 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f026c0097d0 tx=0x7f026c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:52:29.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:29.206+0000 7f0279247640 1 -- 172.21.15.16:0/3619542324 shutdown_connections 2024-09-19T07:52:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:29.206+0000 7f0279247640 1 --2- 172.21.15.16:0/3619542324 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f025406c2b0 0x7f025406e770 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:29.206+0000 7f0279247640 1 --2- 172.21.15.16:0/3619542324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0274141ad0 0x7f0274140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:29.206+0000 7f0279247640 1 --2- 172.21.15.16:0/3619542324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02740d21f0 0x7f0274154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:29.206+0000 7f0279247640 1 --2- 172.21.15.16:0/3619542324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0274072590 0x7f0274159850 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:29.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:29.206+0000 7f0279247640 1 -- 172.21.15.16:0/3619542324 >> 172.21.15.16:0/3619542324 conn(0x7f02740b3ed0 msgr2=0x7f02740cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:29.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:29.206+0000 7f0279247640 1 -- 172.21.15.16:0/3619542324 shutdown_connections 2024-09-19T07:52:29.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:29.206+0000 7f0279247640 1 -- 172.21.15.16:0/3619542324 wait complete. 2024-09-19T07:52:29.217 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:52:29.217 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:52:30.834 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:32.836 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:34.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:52:34.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.332+0000 7f9aaf05e640 1 Processor -- start 2024-09-19T07:52:34.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.332+0000 7f9aaf05e640 1 -- start start 2024-09-19T07:52:34.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.333+0000 7f9aaf05e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9aa80d0540 0x7f9aa8141a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:34.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.333+0000 7f9aaf05e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9aa8141fa0 0x7f9aa813f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:34.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.333+0000 7f9aaf05e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aa815d710 con 0x7f9aa8141680 2024-09-19T07:52:34.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.333+0000 7f9aaf05e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9aa815c6f0 con 0x7f9aa8141fa0 2024-09-19T07:52:34.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.333+0000 7f9aaf05e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa815c950 con 0x7f9aa80d0540 2024-09-19T07:52:34.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.333+0000 7f9aad5fb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9aa8141fa0 0x7f9aa813f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:34.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.333+0000 7f9aacdfa640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9aa8141680 0x7f9aa80cfe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42810/0 (socket says 172.21.15.16:42810) 2024-09-19T07:52:34.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.333+0000 7f9aacdfa640 1 -- 172.21.15.16:0/4288115902 learned_addr learned my addr 172.21.15.16:0/4288115902 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:52:34.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.333+0000 7f9aa7fff640 1 --2- 172.21.15.16:0/4288115902 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9aa80d0540 0x7f9aa8141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:34.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.333+0000 7f9aad5fb640 1 -- 172.21.15.16:0/4288115902 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9aa80d0540 msgr2=0x7f9aa8141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:34.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.333+0000 7f9aad5fb640 1 --2- 172.21.15.16:0/4288115902 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9aa80d0540 0x7f9aa8141a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:34.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.333+0000 7f9aad5fb640 1 -- 172.21.15.16:0/4288115902 >> v1:172.21.15.16:6789/0 conn(0x7f9aa8141680 legacy=0x7f9aa80cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:34.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.333+0000 7f9aad5fb640 1 -- 172.21.15.16:0/4288115902 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa815cc30 con 0x7f9aa8141fa0 2024-09-19T07:52:34.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.333+0000 7f9aa77fe640 1 -- 172.21.15.16:0/4288115902 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2406157162 0 0) 0x7f9aa815d710 con 0x7f9aa8141680 2024-09-19T07:52:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.334+0000 7f9aad5fb640 1 --2- 172.21.15.16:0/4288115902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9aa8141fa0 0x7f9aa813f8b0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009d90 tx=0x7f9aa002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400c56bad3f3c9b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.334+0000 7f9aa77fe640 1 -- 172.21.15.16:0/4288115902 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0007740 con 0x7f9aa8141fa0 2024-09-19T07:52:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.334+0000 7f9aa77fe640 1 -- 172.21.15.16:0/4288115902 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa00078e0 con 0x7f9aa8141fa0 2024-09-19T07:52:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.334+0000 7f9aa77fe640 1 -- 172.21.15.16:0/4288115902 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0007cb0 con 0x7f9aa8141fa0 2024-09-19T07:52:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.335+0000 7f9aaf05e640 1 -- 172.21.15.16:0/4288115902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9aa8141fa0 msgr2=0x7f9aa813f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.335+0000 7f9aaf05e640 1 --2- 172.21.15.16:0/4288115902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9aa8141fa0 0x7f9aa813f8b0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009d90 tx=0x7f9aa002a660 comp rx=0 tx=0).stop 2024-09-19T07:52:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.335+0000 7f9aaf05e640 1 -- 172.21.15.16:0/4288115902 shutdown_connections 2024-09-19T07:52:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.335+0000 7f9aaf05e640 1 --2- 172.21.15.16:0/4288115902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9aa8141fa0 0x7f9aa813f8b0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:34.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.335+0000 7f9aaf05e640 1 --2- 172.21.15.16:0/4288115902 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9aa80d0540 0x7f9aa8141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:34.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.335+0000 7f9aaf05e640 1 -- 172.21.15.16:0/4288115902 >> 172.21.15.16:0/4288115902 conn(0x7f9aa80b3ed0 msgr2=0x7f9aa80b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:34.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.335+0000 7f9aaf05e640 1 -- 172.21.15.16:0/4288115902 shutdown_connections 2024-09-19T07:52:34.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.335+0000 7f9aaf05e640 1 -- 172.21.15.16:0/4288115902 wait complete. 2024-09-19T07:52:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.336+0000 7f9aaf05e640 1 Processor -- start 2024-09-19T07:52:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.336+0000 7f9aaf05e640 1 -- start start 2024-09-19T07:52:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.337+0000 7f9aaf05e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9aa80d0540 0x7f9aa813ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.337+0000 7f9aaf05e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9aa8141680 0x7f9aa80b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.337+0000 7f9aaf05e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9aa8141fa0 0x7f9aa81e3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.337+0000 7f9aaf05e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa8142430 con 0x7f9aa8141fa0 2024-09-19T07:52:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.337+0000 7f9aaf05e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa815d1f0 con 0x7f9aa80d0540 2024-09-19T07:52:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.337+0000 7f9aaf05e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa81425b0 con 0x7f9aa8141680 2024-09-19T07:52:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.337+0000 7f9aad5fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9aa8141fa0 0x7f9aa81e3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.337+0000 7f9aad5fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9aa8141fa0 0x7f9aa81e3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41274/0 (socket says 172.21.15.16:41274) 2024-09-19T07:52:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.337+0000 7f9aad5fb640 1 -- 172.21.15.16:0/1397303994 learned_addr learned my addr 172.21.15.16:0/1397303994 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.337+0000 7f9aacdfa640 1 --2- 172.21.15.16:0/1397303994 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9aa80d0540 0x7f9aa813ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.337+0000 7f9aa7fff640 1 --2- 172.21.15.16:0/1397303994 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9aa8141680 0x7f9aa80b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.337+0000 7f9aad5fb640 1 -- 172.21.15.16:0/1397303994 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9aa8141680 msgr2=0x7f9aa80b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.337+0000 7f9aad5fb640 1 --2- 172.21.15.16:0/1397303994 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9aa8141680 0x7f9aa80b6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.337+0000 7f9aad5fb640 1 -- 172.21.15.16:0/1397303994 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9aa80d0540 msgr2=0x7f9aa813ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.337+0000 7f9aad5fb640 1 --2- 172.21.15.16:0/1397303994 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9aa80d0540 0x7f9aa813ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.337+0000 7f9aad5fb640 1 -- 172.21.15.16:0/1397303994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa80b7fa0 con 0x7f9aa8141fa0 2024-09-19T07:52:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.338+0000 7f9aad5fb640 1 --2- 172.21.15.16:0/1397303994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9aa8141fa0 0x7f9aa81e3620 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009d90 tx=0x7f9aa0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.338+0000 7f9aa57fa640 1 -- 172.21.15.16:0/1397303994 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0040070 con 0x7f9aa8141fa0 2024-09-19T07:52:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.338+0000 7f9aa57fa640 1 -- 172.21.15.16:0/1397303994 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa0031c40 con 0x7f9aa8141fa0 2024-09-19T07:52:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.338+0000 7f9aaf05e640 1 -- 172.21.15.16:0/1397303994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa80b7cc0 con 0x7f9aa8141fa0 2024-09-19T07:52:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.338+0000 7f9aa57fa640 1 -- 172.21.15.16:0/1397303994 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0009450 con 0x7f9aa8141fa0 2024-09-19T07:52:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.338+0000 7f9aaf05e640 1 -- 172.21.15.16:0/1397303994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa80b6240 con 0x7f9aa8141fa0 2024-09-19T07:52:34.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.339+0000 7f9aaf05e640 1 -- 172.21.15.16:0/1397303994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aa8141a60 con 0x7f9aa8141fa0 2024-09-19T07:52:34.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.340+0000 7f9aa57fa640 1 -- 172.21.15.16:0/1397303994 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9aa0039040 con 0x7f9aa8141fa0 2024-09-19T07:52:34.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.340+0000 7f9aa57fa640 1 --2- 172.21.15.16:0/1397303994 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9a8806c2b0 0x7f9a8806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:34.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.340+0000 7f9aa57fa640 1 -- 172.21.15.16:0/1397303994 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f9aa00ac4d0 con 0x7f9aa8141fa0 2024-09-19T07:52:34.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.341+0000 7f9aacdfa640 1 --2- 172.21.15.16:0/1397303994 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9a8806c2b0 0x7f9a8806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:34.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.341+0000 7f9aacdfa640 1 --2- 172.21.15.16:0/1397303994 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9a8806c2b0 0x7f9a8806e770 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9a90007fd0 tx=0x7f9a90007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.345+0000 7f9aa57fa640 1 -- 172.21.15.16:0/1397303994 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9aa007a400 con 0x7f9aa8141fa0 2024-09-19T07:52:34.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.557+0000 7f9aaf05e640 1 -- 172.21.15.16:0/1397303994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9aa80690c0 con 0x7f9aa8141fa0 2024-09-19T07:52:34.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.557+0000 7f9aa57fa640 1 -- 172.21.15.16:0/1397303994 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f9aa007e700 con 0x7f9aa8141fa0 2024-09-19T07:52:34.558 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:52:34.558 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:52:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.559+0000 7f9aaf05e640 1 -- 172.21.15.16:0/1397303994 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9a8806c2b0 msgr2=0x7f9a8806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.559+0000 7f9aaf05e640 1 --2- 172.21.15.16:0/1397303994 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9a8806c2b0 0x7f9a8806e770 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9a90007fd0 tx=0x7f9a90007de0 comp rx=0 tx=0).stop 2024-09-19T07:52:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.559+0000 7f9aaf05e640 1 -- 172.21.15.16:0/1397303994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9aa8141fa0 msgr2=0x7f9aa81e3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.559+0000 7f9aaf05e640 1 --2- 172.21.15.16:0/1397303994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9aa8141fa0 0x7f9aa81e3620 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009d90 tx=0x7f9aa0004a40 comp rx=0 tx=0).stop 2024-09-19T07:52:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.559+0000 7f9aaf05e640 1 -- 172.21.15.16:0/1397303994 shutdown_connections 2024-09-19T07:52:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.559+0000 7f9aaf05e640 1 --2- 172.21.15.16:0/1397303994 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9a8806c2b0 0x7f9a8806e770 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.559+0000 7f9aaf05e640 1 --2- 172.21.15.16:0/1397303994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9aa8141fa0 0x7f9aa81e3620 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.559+0000 7f9aaf05e640 1 --2- 172.21.15.16:0/1397303994 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9aa8141680 0x7f9aa80b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.559+0000 7f9aaf05e640 1 --2- 172.21.15.16:0/1397303994 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9aa80d0540 0x7f9aa813ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.559+0000 7f9aaf05e640 1 -- 172.21.15.16:0/1397303994 >> 172.21.15.16:0/1397303994 conn(0x7f9aa80b3ed0 msgr2=0x7f9aa80cec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.560+0000 7f9aaf05e640 1 -- 172.21.15.16:0/1397303994 shutdown_connections 2024-09-19T07:52:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:34.560+0000 7f9aaf05e640 1 -- 172.21.15.16:0/1397303994 wait complete. 2024-09-19T07:52:34.570 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:52:34.571 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:52:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:38.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:52:38.897 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:52:39.572 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:52:39.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.687+0000 7fb317cc3640 1 Processor -- start 2024-09-19T07:52:39.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.687+0000 7fb317cc3640 1 -- start start 2024-09-19T07:52:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.688+0000 7fb317cc3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb31015e2a0 0x7fb31006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.688+0000 7fb317cc3640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb31015e680 0x7fb3100cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.688+0000 7fb317cc3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3100d1880 con 0x7fb31006d960 2024-09-19T07:52:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.688+0000 7fb317cc3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb3100d1b60 con 0x7fb31015e2a0 2024-09-19T07:52:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.688+0000 7fb317cc3640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb3100cf2d0 con 0x7fb31015e680 2024-09-19T07:52:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.688+0000 7fb315a5f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb31015e2a0 0x7fb31006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.688+0000 7fb316260640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb31006d960 0x7fb3100d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42824/0 (socket says 172.21.15.16:42824) 2024-09-19T07:52:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.688+0000 7fb316260640 1 -- 172.21.15.16:0/4150254351 learned_addr learned my addr 172.21.15.16:0/4150254351 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:52:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.688+0000 7fb31525e640 1 --2- 172.21.15.16:0/4150254351 >> v2:172.21.15.146:3300/0 conn(0x7fb31015e680 0x7fb3100cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:39.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.688+0000 7fb315a5f640 1 -- 172.21.15.16:0/4150254351 >> v2:172.21.15.146:3300/0 conn(0x7fb31015e680 msgr2=0x7fb3100cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.688+0000 7fb315a5f640 1 --2- 172.21.15.16:0/4150254351 >> v2:172.21.15.146:3300/0 conn(0x7fb31015e680 0x7fb3100cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.688+0000 7fb315a5f640 1 -- 172.21.15.16:0/4150254351 >> v1:172.21.15.16:6789/0 conn(0x7fb31006d960 legacy=0x7fb3100d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.688+0000 7fb315a5f640 1 -- 172.21.15.16:0/4150254351 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3100cf5b0 con 0x7fb31015e2a0 2024-09-19T07:52:39.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.688+0000 7fb314a5d640 1 -- 172.21.15.16:0/4150254351 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3211309590 0 0) 0x7fb3100d1880 con 0x7fb31006d960 2024-09-19T07:52:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.689+0000 7fb315a5f640 1 --2- 172.21.15.16:0/4150254351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb31015e2a0 0x7fb31006d420 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb3100a22a0 tx=0x7fb30c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1721260e260ec512 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.689+0000 7fb314a5d640 1 -- 172.21.15.16:0/4150254351 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c037070 con 0x7fb31015e2a0 2024-09-19T07:52:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.689+0000 7fb314a5d640 1 -- 172.21.15.16:0/4150254351 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb30c032440 con 0x7fb31015e2a0 2024-09-19T07:52:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.689+0000 7fb314a5d640 1 -- 172.21.15.16:0/4150254351 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c032810 con 0x7fb31015e2a0 2024-09-19T07:52:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.690+0000 7fb317cc3640 1 -- 172.21.15.16:0/4150254351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb31015e2a0 msgr2=0x7fb31006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.690+0000 7fb317cc3640 1 --2- 172.21.15.16:0/4150254351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb31015e2a0 0x7fb31006d420 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb3100a22a0 tx=0x7fb30c02a1a0 comp rx=0 tx=0).stop 2024-09-19T07:52:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.690+0000 7fb317cc3640 1 -- 172.21.15.16:0/4150254351 shutdown_connections 2024-09-19T07:52:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.690+0000 7fb317cc3640 1 --2- 172.21.15.16:0/4150254351 >> v2:172.21.15.146:3300/0 conn(0x7fb31015e680 0x7fb3100cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.690+0000 7fb317cc3640 1 --2- 172.21.15.16:0/4150254351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb31015e2a0 0x7fb31006d420 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:39.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.690+0000 7fb317cc3640 1 -- 172.21.15.16:0/4150254351 >> 172.21.15.16:0/4150254351 conn(0x7fb31009d760 msgr2=0x7fb31008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:39.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.690+0000 7fb317cc3640 1 -- 172.21.15.16:0/4150254351 shutdown_connections 2024-09-19T07:52:39.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.690+0000 7fb317cc3640 1 -- 172.21.15.16:0/4150254351 wait complete. 2024-09-19T07:52:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.691+0000 7fb317cc3640 1 Processor -- start 2024-09-19T07:52:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.691+0000 7fb317cc3640 1 -- start start 2024-09-19T07:52:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.692+0000 7fb317cc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb31006d960 0x7fb31015be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.692+0000 7fb317cc3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb31015e2a0 0x7fb3100c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.692+0000 7fb317cc3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb31015e680 0x7fb3100c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.692+0000 7fb317cc3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3100d4c00 con 0x7fb31006d960 2024-09-19T07:52:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.692+0000 7fb317cc3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb3100d4a80 con 0x7fb31015e680 2024-09-19T07:52:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.692+0000 7fb317cc3640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb3100d0010 con 0x7fb31015e2a0 2024-09-19T07:52:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.692+0000 7fb315a5f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb31006d960 0x7fb31015be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:39.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.692+0000 7fb315a5f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb31006d960 0x7fb31015be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41292/0 (socket says 172.21.15.16:41292) 2024-09-19T07:52:39.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.692+0000 7fb315a5f640 1 -- 172.21.15.16:0/1163514328 learned_addr learned my addr 172.21.15.16:0/1163514328 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:39.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.692+0000 7fb316260640 1 --2- 172.21.15.16:0/1163514328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb31015e680 0x7fb3100c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:39.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.692+0000 7fb31525e640 1 --2- 172.21.15.16:0/1163514328 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb31015e2a0 0x7fb3100c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:39.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.693+0000 7fb315a5f640 1 -- 172.21.15.16:0/1163514328 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb31015e2a0 msgr2=0x7fb3100c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:39.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.693+0000 7fb315a5f640 1 --2- 172.21.15.16:0/1163514328 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb31015e2a0 0x7fb3100c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:39.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.693+0000 7fb315a5f640 1 -- 172.21.15.16:0/1163514328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb31015e680 msgr2=0x7fb3100c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:39.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.693+0000 7fb315a5f640 1 --2- 172.21.15.16:0/1163514328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb31015e680 0x7fb3100c9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:39.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.693+0000 7fb315a5f640 1 -- 172.21.15.16:0/1163514328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3100c8880 con 0x7fb31006d960 2024-09-19T07:52:39.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.693+0000 7fb315a5f640 1 --2- 172.21.15.16:0/1163514328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb31006d960 0x7fb31015be40 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fb30c000c00 tx=0x7fb30c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.693+0000 7fb3067fc640 1 -- 172.21.15.16:0/1163514328 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c037040 con 0x7fb31006d960 2024-09-19T07:52:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.693+0000 7fb3067fc640 1 -- 172.21.15.16:0/1163514328 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb30c042070 con 0x7fb31006d960 2024-09-19T07:52:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.693+0000 7fb317cc3640 1 -- 172.21.15.16:0/1163514328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3100c8b60 con 0x7fb31006d960 2024-09-19T07:52:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.693+0000 7fb3067fc640 1 -- 172.21.15.16:0/1163514328 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c030bd0 con 0x7fb31006d960 2024-09-19T07:52:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.693+0000 7fb317cc3640 1 -- 172.21.15.16:0/1163514328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3100c9120 con 0x7fb31006d960 2024-09-19T07:52:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.694+0000 7fb317cc3640 1 -- 172.21.15.16:0/1163514328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb31006d420 con 0x7fb31006d960 2024-09-19T07:52:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.695+0000 7fb3067fc640 1 -- 172.21.15.16:0/1163514328 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb30c047460 con 0x7fb31006d960 2024-09-19T07:52:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.695+0000 7fb3067fc640 1 --2- 172.21.15.16:0/1163514328 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb2f406c2b0 0x7fb2f406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.695+0000 7fb3067fc640 1 -- 172.21.15.16:0/1163514328 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7fb30c0ac090 con 0x7fb31006d960 2024-09-19T07:52:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.696+0000 7fb31525e640 1 --2- 172.21.15.16:0/1163514328 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb2f406c2b0 0x7fb2f406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.696+0000 7fb31525e640 1 --2- 172.21.15.16:0/1163514328 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb2f406c2b0 0x7fb2f406e770 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb310160c10 tx=0x7fb300008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:39.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.700+0000 7fb3067fc640 1 -- 172.21.15.16:0/1163514328 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb30c032e20 con 0x7fb31006d960 2024-09-19T07:52:39.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.921+0000 7fb317cc3640 1 -- 172.21.15.16:0/1163514328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb3100d0850 con 0x7fb31006d960 2024-09-19T07:52:39.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.921+0000 7fb3067fc640 1 -- 172.21.15.16:0/1163514328 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fb30c079fb0 con 0x7fb31006d960 2024-09-19T07:52:39.921 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:52:39.922 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:52:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.923+0000 7fb317cc3640 1 -- 172.21.15.16:0/1163514328 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb2f406c2b0 msgr2=0x7fb2f406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.923+0000 7fb317cc3640 1 --2- 172.21.15.16:0/1163514328 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb2f406c2b0 0x7fb2f406e770 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb310160c10 tx=0x7fb300008040 comp rx=0 tx=0).stop 2024-09-19T07:52:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.923+0000 7fb317cc3640 1 -- 172.21.15.16:0/1163514328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb31006d960 msgr2=0x7fb31015be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.923+0000 7fb317cc3640 1 --2- 172.21.15.16:0/1163514328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb31006d960 0x7fb31015be40 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fb30c000c00 tx=0x7fb30c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:52:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.923+0000 7fb317cc3640 1 -- 172.21.15.16:0/1163514328 shutdown_connections 2024-09-19T07:52:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.923+0000 7fb317cc3640 1 --2- 172.21.15.16:0/1163514328 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb2f406c2b0 0x7fb2f406e770 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.923+0000 7fb317cc3640 1 --2- 172.21.15.16:0/1163514328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb31015e680 0x7fb3100c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.923+0000 7fb317cc3640 1 --2- 172.21.15.16:0/1163514328 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb31015e2a0 0x7fb3100c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.923+0000 7fb317cc3640 1 --2- 172.21.15.16:0/1163514328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb31006d960 0x7fb31015be40 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:39.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.923+0000 7fb317cc3640 1 -- 172.21.15.16:0/1163514328 >> 172.21.15.16:0/1163514328 conn(0x7fb31009d760 msgr2=0x7fb31015f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.923+0000 7fb317cc3640 1 -- 172.21.15.16:0/1163514328 shutdown_connections 2024-09-19T07:52:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:39.923+0000 7fb317cc3640 1 -- 172.21.15.16:0/1163514328 wait complete. 2024-09-19T07:52:39.934 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:52:39.934 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:52:40.844 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:44.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:52:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.049+0000 7fab0ca77640 1 Processor -- start 2024-09-19T07:52:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.049+0000 7fab0ca77640 1 -- start start 2024-09-19T07:52:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.050+0000 7fab0ca77640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fab0815e2a0 0x7fab0806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.050+0000 7fab0ca77640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab080d1880 con 0x7fab0806d960 2024-09-19T07:52:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.050+0000 7fab0ca77640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab080d1b60 con 0x7fab0815e680 2024-09-19T07:52:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.050+0000 7fab0ca77640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fab080cf2d0 con 0x7fab0815e2a0 2024-09-19T07:52:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.050+0000 7fab06d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fab0806d960 0x7fab080d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54402/0 (socket says 172.21.15.16:54402) 2024-09-19T07:52:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.050+0000 7fab06d9d640 1 -- 172.21.15.16:0/4228610960 learned_addr learned my addr 172.21.15.16:0/4228610960 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:52:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.050+0000 7fab0659c640 1 --2- 172.21.15.16:0/4228610960 >> v2:172.21.15.146:3300/0 conn(0x7fab0815e2a0 0x7fab0806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.050+0000 7fab05d9b640 1 -- 172.21.15.16:0/4228610960 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1612380943 0 0) 0x7fab080d1880 con 0x7fab0806d960 2024-09-19T07:52:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.050+0000 7fab05d9b640 1 -- 172.21.15.16:0/4228610960 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faaf4003520 con 0x7fab0806d960 2024-09-19T07:52:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab05d9b640 1 -- 172.21.15.16:0/4228610960 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 293269777 0 0) 0x7faaf4003520 con 0x7fab0806d960 2024-09-19T07:52:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab05d9b640 1 -- 172.21.15.16:0/4228610960 >> v2:172.21.15.146:3300/0 conn(0x7fab0815e2a0 msgr2=0x7fab0806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab05d9b640 1 --2- 172.21.15.16:0/4228610960 >> v2:172.21.15.146:3300/0 conn(0x7fab0815e2a0 0x7fab0806d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab05d9b640 1 -- 172.21.15.16:0/4228610960 >> v1:172.21.15.146:6789/0 conn(0x7fab0815e680 legacy=0x7fab080cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab05d9b640 1 -- 172.21.15.16:0/4228610960 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab080cf5b0 con 0x7fab0806d960 2024-09-19T07:52:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab05d9b640 1 -- 172.21.15.16:0/4228610960 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fab000019f0 con 0x7fab0806d960 2024-09-19T07:52:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab05d9b640 1 -- 172.21.15.16:0/4228610960 >> v1:172.21.15.16:6789/0 conn(0x7fab0806d960 legacy=0x7fab080d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab05d9b640 1 --2- 172.21.15.16:0/4228610960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faaf40039b0 0x7faaf4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab05d9b640 1 --2- 172.21.15.16:0/4228610960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaf40043b0 0x7faaf4004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab05d9b640 1 --2- 172.21.15.16:0/4228610960 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faaf4005080 0x7faaf400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab05d9b640 1 -- 172.21.15.16:0/4228610960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab080cf2d0 con 0x7faaf40039b0 2024-09-19T07:52:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab05d9b640 1 -- 172.21.15.16:0/4228610960 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faaf400bfe0 con 0x7faaf40043b0 2024-09-19T07:52:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab05d9b640 1 -- 172.21.15.16:0/4228610960 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faaf400c1e0 con 0x7faaf4005080 2024-09-19T07:52:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab0659c640 1 --2- 172.21.15.16:0/4228610960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faaf40039b0 0x7faaf4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7faafffff640 1 --2- 172.21.15.16:0/4228610960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaf40043b0 0x7faaf4004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab06d9d640 1 --2- 172.21.15.16:0/4228610960 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faaf4005080 0x7faaf400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab0659c640 1 -- 172.21.15.16:0/4228610960 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faaf4005080 msgr2=0x7faaf400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab0659c640 1 --2- 172.21.15.16:0/4228610960 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faaf4005080 0x7faaf400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab0659c640 1 -- 172.21.15.16:0/4228610960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaf40043b0 msgr2=0x7faaf4004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab0659c640 1 --2- 172.21.15.16:0/4228610960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaf40043b0 0x7faaf4004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab06d9d640 1 -- 172.21.15.16:0/4228610960 reap_dead start 2024-09-19T07:52:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.051+0000 7fab0659c640 1 -- 172.21.15.16:0/4228610960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf4003520 con 0x7faaf40039b0 2024-09-19T07:52:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.052+0000 7fab0659c640 1 --2- 172.21.15.16:0/4228610960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faaf40039b0 0x7faaf4003d90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009ac0 tx=0x7faaf4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2a27f8a0a67625 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.052+0000 7fab05d9b640 1 -- 172.21.15.16:0/4228610960 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8036070 con 0x7faaf40039b0 2024-09-19T07:52:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.052+0000 7fab05d9b640 1 -- 172.21.15.16:0/4228610960 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf8031400 con 0x7faaf40039b0 2024-09-19T07:52:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.052+0000 7fab05d9b640 1 -- 172.21.15.16:0/4228610960 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8009690 con 0x7faaf40039b0 2024-09-19T07:52:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.053+0000 7fab0ca77640 1 -- 172.21.15.16:0/4228610960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faaf40039b0 msgr2=0x7faaf4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.053+0000 7fab0ca77640 1 --2- 172.21.15.16:0/4228610960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faaf40039b0 0x7faaf4003d90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009ac0 tx=0x7faaf4005830 comp rx=0 tx=0).stop 2024-09-19T07:52:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.053+0000 7fab0ca77640 1 -- 172.21.15.16:0/4228610960 shutdown_connections 2024-09-19T07:52:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.053+0000 7fab0ca77640 1 --2- 172.21.15.16:0/4228610960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faaf40039b0 0x7faaf4003d90 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.053+0000 7fab0ca77640 1 -- 172.21.15.16:0/4228610960 >> 172.21.15.16:0/4228610960 conn(0x7fab0809d760 msgr2=0x7fab0808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.053+0000 7fab0ca77640 1 -- 172.21.15.16:0/4228610960 shutdown_connections 2024-09-19T07:52:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.053+0000 7fab0ca77640 1 -- 172.21.15.16:0/4228610960 wait complete. 2024-09-19T07:52:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.054+0000 7fab0ca77640 1 Processor -- start 2024-09-19T07:52:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.054+0000 7fab0ca77640 1 -- start start 2024-09-19T07:52:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.055+0000 7fab0ca77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faaf40039b0 0x7fab08156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.055+0000 7fab0ca77640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab080b9b60 0x7fab080ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.055+0000 7fab0ca77640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab080b8f50 0x7fab080b93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.055+0000 7fab0ca77640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab0806d5a0 con 0x7faaf40039b0 2024-09-19T07:52:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.055+0000 7fab0ca77640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fab0806d420 con 0x7fab080b9b60 2024-09-19T07:52:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.055+0000 7fab0ca77640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fab0806d720 con 0x7fab080b8f50 2024-09-19T07:52:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.055+0000 7fab0659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faaf40039b0 0x7fab08156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.055+0000 7fab0659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faaf40039b0 0x7fab08156ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41426/0 (socket says 172.21.15.16:41426) 2024-09-19T07:52:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.055+0000 7fab0659c640 1 -- 172.21.15.16:0/1195268393 learned_addr learned my addr 172.21.15.16:0/1195268393 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.055+0000 7fab06d9d640 1 --2- 172.21.15.16:0/1195268393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab080b8f50 0x7fab080b93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.055+0000 7faafffff640 1 --2- 172.21.15.16:0/1195268393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab080b9b60 0x7fab080ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.055+0000 7fab0659c640 1 -- 172.21.15.16:0/1195268393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab080b8f50 msgr2=0x7fab080b93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.055+0000 7fab0659c640 1 --2- 172.21.15.16:0/1195268393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab080b8f50 0x7fab080b93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.055+0000 7fab0659c640 1 -- 172.21.15.16:0/1195268393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab080b9b60 msgr2=0x7fab080ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.055+0000 7fab0659c640 1 --2- 172.21.15.16:0/1195268393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab080b9b60 0x7fab080ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:45.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.055+0000 7fab0659c640 1 -- 172.21.15.16:0/1195268393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab080cd130 con 0x7faaf40039b0 2024-09-19T07:52:45.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.056+0000 7fab0659c640 1 --2- 172.21.15.16:0/1195268393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faaf40039b0 0x7fab08156ac0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009be0 tx=0x7faaf8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:45.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.056+0000 7faafeffd640 1 -- 172.21.15.16:0/1195268393 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf803f020 con 0x7faaf40039b0 2024-09-19T07:52:45.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.056+0000 7faafeffd640 1 -- 172.21.15.16:0/1195268393 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf8037040 con 0x7faaf40039b0 2024-09-19T07:52:45.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.056+0000 7fab0ca77640 1 -- 172.21.15.16:0/1195268393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab080ca440 con 0x7faaf40039b0 2024-09-19T07:52:45.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.056+0000 7faafeffd640 1 -- 172.21.15.16:0/1195268393 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8036070 con 0x7faaf40039b0 2024-09-19T07:52:45.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.056+0000 7fab0ca77640 1 -- 172.21.15.16:0/1195268393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab080caa00 con 0x7faaf40039b0 2024-09-19T07:52:45.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.057+0000 7fab0ca77640 1 -- 172.21.15.16:0/1195268393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab080cfad0 con 0x7faaf40039b0 2024-09-19T07:52:45.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.057+0000 7faafeffd640 1 -- 172.21.15.16:0/1195268393 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7faaf8042020 con 0x7faaf40039b0 2024-09-19T07:52:45.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.058+0000 7faafeffd640 1 --2- 172.21.15.16:0/1195268393 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faae806c2b0 0x7faae806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:45.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.058+0000 7faafeffd640 1 -- 172.21.15.16:0/1195268393 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7faaf80ab300 con 0x7faaf40039b0 2024-09-19T07:52:45.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.058+0000 7faafffff640 1 --2- 172.21.15.16:0/1195268393 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faae806c2b0 0x7faae806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:45.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.059+0000 7faafffff640 1 --2- 172.21.15.16:0/1195268393 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faae806c2b0 0x7faae806e770 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fab081f7c10 tx=0x7faaf0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:45.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.062+0000 7faafeffd640 1 -- 172.21.15.16:0/1195268393 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7faaf8049070 con 0x7faaf40039b0 2024-09-19T07:52:45.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.279+0000 7fab0ca77640 1 -- 172.21.15.16:0/1195268393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fab0806cf90 con 0x7faaf40039b0 2024-09-19T07:52:45.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.280+0000 7faafeffd640 1 -- 172.21.15.16:0/1195268393 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7faaf8079900 con 0x7faaf40039b0 2024-09-19T07:52:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:52:45.281 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:52:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.282+0000 7fab0ca77640 1 -- 172.21.15.16:0/1195268393 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faae806c2b0 msgr2=0x7faae806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.282+0000 7fab0ca77640 1 --2- 172.21.15.16:0/1195268393 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faae806c2b0 0x7faae806e770 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fab081f7c10 tx=0x7faaf0009040 comp rx=0 tx=0).stop 2024-09-19T07:52:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.282+0000 7fab0ca77640 1 -- 172.21.15.16:0/1195268393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faaf40039b0 msgr2=0x7fab08156ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.282+0000 7fab0ca77640 1 --2- 172.21.15.16:0/1195268393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faaf40039b0 0x7fab08156ac0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009be0 tx=0x7faaf8004730 comp rx=0 tx=0).stop 2024-09-19T07:52:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.282+0000 7fab0ca77640 1 -- 172.21.15.16:0/1195268393 shutdown_connections 2024-09-19T07:52:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.282+0000 7fab0ca77640 1 --2- 172.21.15.16:0/1195268393 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faae806c2b0 0x7faae806e770 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.282+0000 7fab0ca77640 1 --2- 172.21.15.16:0/1195268393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab080b8f50 0x7fab080b93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.282+0000 7fab0ca77640 1 --2- 172.21.15.16:0/1195268393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab080b9b60 0x7fab080ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.282+0000 7fab0ca77640 1 --2- 172.21.15.16:0/1195268393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faaf40039b0 0x7fab08156ac0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.282+0000 7fab0ca77640 1 -- 172.21.15.16:0/1195268393 >> 172.21.15.16:0/1195268393 conn(0x7fab0809d760 msgr2=0x7fab08161f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.282+0000 7fab0ca77640 1 -- 172.21.15.16:0/1195268393 shutdown_connections 2024-09-19T07:52:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45.282+0000 7fab0ca77640 1 -- 172.21.15.16:0/1195268393 wait complete. 2024-09-19T07:52:45.293 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-19T07:52:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:45 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-19T07:52:46.851 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:52:46.851 INFO:teuthology.orchestra.run.smithi016.stderr:stdin closed, goodbye! 2024-09-19T07:52:46.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:52:46.898 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:52:46.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:52:46.981 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:52:47.025 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:52:47.025 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:52:47.025 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 400\nabs_path = "/home/ubuntu/cephtest/mnt.0/subdir"\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi016') 2024-09-19T07:52:47.025 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:52:47.025 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:52:47.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:52:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.195+0000 7f58e1f9f640 1 Processor -- start 2024-09-19T07:52:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.195+0000 7f58e1f9f640 1 -- start start 2024-09-19T07:52:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.195+0000 7f58e1f9f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f58dc0adf70 0x7f58dc0b3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.195+0000 7f58e1f9f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58dc0700d0 con 0x7f58dc0b2bf0 2024-09-19T07:52:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58e1f9f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58dc070340 con 0x7f58dc146e50 2024-09-19T07:52:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58e1f9f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f58dc0b35a0 con 0x7f58dc0adf70 2024-09-19T07:52:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58dbfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f58dc0b2bf0 0x7f58dc0ad860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54412/0 (socket says 172.21.15.16:54412) 2024-09-19T07:52:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58dbfff640 1 -- 172.21.15.16:0/1802267586 learned_addr learned my addr 172.21.15.16:0/1802267586 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:52:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58e179e640 1 --2- 172.21.15.16:0/1802267586 >> v2:172.21.15.146:3300/0 conn(0x7f58dc0adf70 0x7f58dc0b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58db7fe640 1 -- 172.21.15.16:0/1802267586 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2270298202 0 0) 0x7f58dc0700d0 con 0x7f58dc0b2bf0 2024-09-19T07:52:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58db7fe640 1 -- 172.21.15.16:0/1802267586 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58c8003520 con 0x7f58dc0b2bf0 2024-09-19T07:52:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58db7fe640 1 -- 172.21.15.16:0/1802267586 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2677608525 0 0) 0x7f58c8003520 con 0x7f58dc0b2bf0 2024-09-19T07:52:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58db7fe640 1 -- 172.21.15.16:0/1802267586 >> v2:172.21.15.146:3300/0 conn(0x7f58dc0adf70 msgr2=0x7f58dc0b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58db7fe640 1 --2- 172.21.15.16:0/1802267586 >> v2:172.21.15.146:3300/0 conn(0x7f58dc0adf70 0x7f58dc0b3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58db7fe640 1 -- 172.21.15.16:0/1802267586 >> v1:172.21.15.146:6789/0 conn(0x7f58dc146e50 legacy=0x7f58dc070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58db7fe640 1 -- 172.21.15.16:0/1802267586 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc070620 con 0x7f58dc0b2bf0 2024-09-19T07:52:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58db7fe640 1 -- 172.21.15.16:0/1802267586 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f58c40019f0 con 0x7f58dc0b2bf0 2024-09-19T07:52:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58db7fe640 1 -- 172.21.15.16:0/1802267586 >> v1:172.21.15.16:6789/0 conn(0x7f58dc0b2bf0 legacy=0x7f58dc0ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58db7fe640 1 --2- 172.21.15.16:0/1802267586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58c80039b0 0x7f58c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58db7fe640 1 --2- 172.21.15.16:0/1802267586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58c8004800 0x7f58c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58db7fe640 1 --2- 172.21.15.16:0/1802267586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c80051c0 0x7f58c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58db7fe640 1 -- 172.21.15.16:0/1802267586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc0b35a0 con 0x7f58c80051c0 2024-09-19T07:52:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58db7fe640 1 -- 172.21.15.16:0/1802267586 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f58c800c160 con 0x7f58c8004800 2024-09-19T07:52:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58db7fe640 1 -- 172.21.15.16:0/1802267586 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f58c800c360 con 0x7f58c80039b0 2024-09-19T07:52:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58e179e640 1 --2- 172.21.15.16:0/1802267586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c80051c0 0x7f58c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58dbfff640 1 --2- 172.21.15.16:0/1802267586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58c8004800 0x7f58c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.196+0000 7f58e0f9d640 1 --2- 172.21.15.16:0/1802267586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58c80039b0 0x7f58c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.197+0000 7f58e179e640 1 -- 172.21.15.16:0/1802267586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58c80039b0 msgr2=0x7f58c8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.197+0000 7f58e179e640 1 --2- 172.21.15.16:0/1802267586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58c80039b0 0x7f58c8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.197+0000 7f58e179e640 1 -- 172.21.15.16:0/1802267586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58c8004800 msgr2=0x7f58c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.197+0000 7f58e179e640 1 --2- 172.21.15.16:0/1802267586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58c8004800 0x7f58c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.197+0000 7f58e179e640 1 -- 172.21.15.16:0/1802267586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c8003520 con 0x7f58c80051c0 2024-09-19T07:52:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.197+0000 7f58e179e640 1 -- 172.21.15.16:0/1802267586 reap_dead start 2024-09-19T07:52:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.197+0000 7f58e179e640 1 --2- 172.21.15.16:0/1802267586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c80051c0 0x7f58c800b9c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009ac0 tx=0x7f58c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee407a83dd7ef25 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:47.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.198+0000 7f58db7fe640 1 -- 172.21.15.16:0/1802267586 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4009690 con 0x7f58c80051c0 2024-09-19T07:52:47.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.198+0000 7f58db7fe640 1 -- 172.21.15.16:0/1802267586 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d4004020 con 0x7f58c80051c0 2024-09-19T07:52:47.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.198+0000 7f58db7fe640 1 -- 172.21.15.16:0/1802267586 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4004410 con 0x7f58c80051c0 2024-09-19T07:52:47.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.198+0000 7f58e1f9f640 1 -- 172.21.15.16:0/1802267586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c80051c0 msgr2=0x7f58c800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:47.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.198+0000 7f58e1f9f640 1 --2- 172.21.15.16:0/1802267586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c80051c0 0x7f58c800b9c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009ac0 tx=0x7f58c8005830 comp rx=0 tx=0).stop 2024-09-19T07:52:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.198+0000 7f58e1f9f640 1 -- 172.21.15.16:0/1802267586 shutdown_connections 2024-09-19T07:52:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.198+0000 7f58e1f9f640 1 --2- 172.21.15.16:0/1802267586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c80051c0 0x7f58c800b9c0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.198+0000 7f58e1f9f640 1 -- 172.21.15.16:0/1802267586 >> 172.21.15.16:0/1802267586 conn(0x7f58dc06ed90 msgr2=0x7f58dc06f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.198+0000 7f58e1f9f640 1 -- 172.21.15.16:0/1802267586 shutdown_connections 2024-09-19T07:52:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.198+0000 7f58e1f9f640 1 -- 172.21.15.16:0/1802267586 wait complete. 2024-09-19T07:52:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.199+0000 7f58e1f9f640 1 Processor -- start 2024-09-19T07:52:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.200+0000 7f58e1f9f640 1 -- start start 2024-09-19T07:52:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.200+0000 7f58e1f9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c80051c0 0x7f58dc13ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.200+0000 7f58e1f9f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58dc15e890 0x7f58dc15ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.200+0000 7f58e1f9f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58dc160750 0x7f58dc160c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.200+0000 7f58e1f9f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc147230 con 0x7f58c80051c0 2024-09-19T07:52:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.200+0000 7f58e1f9f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc073750 con 0x7f58dc160750 2024-09-19T07:52:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.200+0000 7f58e1f9f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f58dc15f400 con 0x7f58dc15e890 2024-09-19T07:52:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.201+0000 7f58e0f9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c80051c0 0x7f58dc13ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.201+0000 7f58e0f9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c80051c0 0x7f58dc13ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41446/0 (socket says 172.21.15.16:41446) 2024-09-19T07:52:47.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.201+0000 7f58e0f9d640 1 -- 172.21.15.16:0/955934169 learned_addr learned my addr 172.21.15.16:0/955934169 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:47.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.201+0000 7f58dbfff640 1 --2- 172.21.15.16:0/955934169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58dc15e890 0x7f58dc15ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:47.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.201+0000 7f58e179e640 1 --2- 172.21.15.16:0/955934169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58dc160750 0x7f58dc160c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:47.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.201+0000 7f58e0f9d640 1 -- 172.21.15.16:0/955934169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58dc15e890 msgr2=0x7f58dc15ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:47.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.201+0000 7f58e0f9d640 1 --2- 172.21.15.16:0/955934169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58dc15e890 0x7f58dc15ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:47.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.201+0000 7f58e0f9d640 1 -- 172.21.15.16:0/955934169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58dc160750 msgr2=0x7f58dc160c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:47.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.201+0000 7f58e0f9d640 1 --2- 172.21.15.16:0/955934169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58dc160750 0x7f58dc160c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:47.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.201+0000 7f58e0f9d640 1 -- 172.21.15.16:0/955934169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc15f6e0 con 0x7f58c80051c0 2024-09-19T07:52:47.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.201+0000 7f58e0f9d640 1 --2- 172.21.15.16:0/955934169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c80051c0 0x7f58dc13ea10 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00f620 tx=0x7f58cc00fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:47.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.201+0000 7f58d97fa640 1 -- 172.21.15.16:0/955934169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc0052e0 con 0x7f58c80051c0 2024-09-19T07:52:47.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.201+0000 7f58d97fa640 1 -- 172.21.15.16:0/955934169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58cc018070 con 0x7f58c80051c0 2024-09-19T07:52:47.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.201+0000 7f58e1f9f640 1 -- 172.21.15.16:0/955934169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58dc15f9c0 con 0x7f58c80051c0 2024-09-19T07:52:47.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.201+0000 7f58d97fa640 1 -- 172.21.15.16:0/955934169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc003810 con 0x7f58c80051c0 2024-09-19T07:52:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.201+0000 7f58e1f9f640 1 -- 172.21.15.16:0/955934169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58dc13ef50 con 0x7f58c80051c0 2024-09-19T07:52:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.202+0000 7f58e1f9f640 1 -- 172.21.15.16:0/955934169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58dc160080 con 0x7f58c80051c0 2024-09-19T07:52:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.203+0000 7f58d97fa640 1 -- 172.21.15.16:0/955934169 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f58cc0039b0 con 0x7f58c80051c0 2024-09-19T07:52:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.203+0000 7f58d97fa640 1 --2- 172.21.15.16:0/955934169 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58bc06c2e0 0x7f58bc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:47.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.203+0000 7f58d97fa640 1 -- 172.21.15.16:0/955934169 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f58cc08b270 con 0x7f58c80051c0 2024-09-19T07:52:47.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.203+0000 7f58dbfff640 1 --2- 172.21.15.16:0/955934169 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58bc06c2e0 0x7f58bc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:47.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.204+0000 7f58dbfff640 1 --2- 172.21.15.16:0/955934169 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58bc06c2e0 0x7f58bc06e7a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f58dc1554d0 tx=0x7f58c4006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:47.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.208+0000 7f58d97fa640 1 -- 172.21.15.16:0/955934169 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f58dc160080 con 0x7f58c80051c0 2024-09-19T07:52:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.396+0000 7f58e1f9f640 1 -- 172.21.15.16:0/955934169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f58dc0b3060 con 0x7f58c80051c0 2024-09-19T07:52:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.396+0000 7f58d97fa640 1 -- 172.21.15.16:0/955934169 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v95) ==== 80+0+0 (secure 0 0 0) 0x7f58cc059190 con 0x7f58c80051c0 2024-09-19T07:52:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:52:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.398+0000 7f58e1f9f640 1 -- 172.21.15.16:0/955934169 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58bc06c2e0 msgr2=0x7f58bc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.398+0000 7f58e1f9f640 1 --2- 172.21.15.16:0/955934169 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58bc06c2e0 0x7f58bc06e7a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f58dc1554d0 tx=0x7f58c4006040 comp rx=0 tx=0).stop 2024-09-19T07:52:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.398+0000 7f58e1f9f640 1 -- 172.21.15.16:0/955934169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c80051c0 msgr2=0x7f58dc13ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.398+0000 7f58e1f9f640 1 --2- 172.21.15.16:0/955934169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c80051c0 0x7f58dc13ea10 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00f620 tx=0x7f58cc00fae0 comp rx=0 tx=0).stop 2024-09-19T07:52:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.398+0000 7f58e1f9f640 1 -- 172.21.15.16:0/955934169 shutdown_connections 2024-09-19T07:52:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.398+0000 7f58e1f9f640 1 --2- 172.21.15.16:0/955934169 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58bc06c2e0 0x7f58bc06e7a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.398+0000 7f58e1f9f640 1 --2- 172.21.15.16:0/955934169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58dc160750 0x7f58dc160c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.398+0000 7f58e1f9f640 1 --2- 172.21.15.16:0/955934169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58dc15e890 0x7f58dc15ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.398+0000 7f58e1f9f640 1 --2- 172.21.15.16:0/955934169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c80051c0 0x7f58dc13ea10 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.398+0000 7f58e1f9f640 1 -- 172.21.15.16:0/955934169 >> 172.21.15.16:0/955934169 conn(0x7f58dc06ed90 msgr2=0x7f58dc144490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.398+0000 7f58e1f9f640 1 -- 172.21.15.16:0/955934169 shutdown_connections 2024-09-19T07:52:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.398+0000 7f58e1f9f640 1 -- 172.21.15.16:0/955934169 wait complete. 2024-09-19T07:52:47.410 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-19T07:52:47.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:52:47.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T07:52:47.434 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[61430]: fuse finished with error 0 and tester_r 0 2024-09-19T07:52:47.436 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:52:47.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:52:47.437 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:52:47.626 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:52:47.626 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:52:47.626 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:52:47.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:52:47.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.772+0000 7f2519778640 1 Processor -- start 2024-09-19T07:52:47.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.772+0000 7f2519778640 1 -- start start 2024-09-19T07:52:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.773+0000 7f2519778640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f25140726c0 0x7f2514160010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.773+0000 7f2519778640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25140d2b30 con 0x7f25140d2750 2024-09-19T07:52:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.773+0000 7f2519778640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2514141c60 con 0x7f2514072250 2024-09-19T07:52:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.773+0000 7f2519778640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2514145bd0 con 0x7f25140726c0 2024-09-19T07:52:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.773+0000 7f25137fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f25140726c0 0x7f2514160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.773+0000 7f2512ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f25140d2750 0x7f251413f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54414/0 (socket says 172.21.15.16:54414) 2024-09-19T07:52:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.773+0000 7f2512ffd640 1 -- 172.21.15.16:0/882231839 learned_addr learned my addr 172.21.15.16:0/882231839 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:52:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.774+0000 7f25137fe640 1 -- 172.21.15.16:0/882231839 >> v1:172.21.15.146:6789/0 conn(0x7f2514072250 legacy=0x7f25140cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.774+0000 7f25137fe640 1 -- 172.21.15.16:0/882231839 >> v1:172.21.15.16:6789/0 conn(0x7f25140d2750 legacy=0x7f251413f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.774+0000 7f25137fe640 1 -- 172.21.15.16:0/882231839 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2514141f40 con 0x7f25140726c0 2024-09-19T07:52:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.774+0000 7f2511ffb640 1 -- 172.21.15.16:0/882231839 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3000043420 0 0) 0x7f25140d2b30 con 0x7f25140d2750 2024-09-19T07:52:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.774+0000 7f25137fe640 1 --2- 172.21.15.16:0/882231839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25140726c0 0x7f2514160010 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f250c009d90 tx=0x7f250c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a8b3fa34def9ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.775+0000 7f2511ffb640 1 -- 172.21.15.16:0/882231839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c031600 con 0x7f25140726c0 2024-09-19T07:52:47.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.775+0000 7f2511ffb640 1 -- 172.21.15.16:0/882231839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f250c0317a0 con 0x7f25140726c0 2024-09-19T07:52:47.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.775+0000 7f2511ffb640 1 -- 172.21.15.16:0/882231839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c031b70 con 0x7f25140726c0 2024-09-19T07:52:47.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.775+0000 7f2519778640 1 -- 172.21.15.16:0/882231839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25140726c0 msgr2=0x7f2514160010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:47.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.775+0000 7f2519778640 1 --2- 172.21.15.16:0/882231839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25140726c0 0x7f2514160010 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f250c009d90 tx=0x7f250c02a610 comp rx=0 tx=0).stop 2024-09-19T07:52:47.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.775+0000 7f2519778640 1 -- 172.21.15.16:0/882231839 shutdown_connections 2024-09-19T07:52:47.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.775+0000 7f2519778640 1 --2- 172.21.15.16:0/882231839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25140726c0 0x7f2514160010 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:47.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.775+0000 7f2519778640 1 -- 172.21.15.16:0/882231839 >> 172.21.15.16:0/882231839 conn(0x7f25140b4040 msgr2=0x7f25140b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:47.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.775+0000 7f2519778640 1 -- 172.21.15.16:0/882231839 shutdown_connections 2024-09-19T07:52:47.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.775+0000 7f2519778640 1 -- 172.21.15.16:0/882231839 wait complete. 2024-09-19T07:52:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.776+0000 7f2519778640 1 Processor -- start 2024-09-19T07:52:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.777+0000 7f2519778640 1 -- start start 2024-09-19T07:52:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.777+0000 7f2519778640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2514072250 0x7f251415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.777+0000 7f2519778640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25140726c0 0x7f25141f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.777+0000 7f2519778640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25140d2750 0x7f25141f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.777+0000 7f2519778640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2514142500 con 0x7f25140726c0 2024-09-19T07:52:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.777+0000 7f2519778640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f25141427e0 con 0x7f25140d2750 2024-09-19T07:52:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.777+0000 7f2519778640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f25141418b0 con 0x7f2514072250 2024-09-19T07:52:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.777+0000 7f25127fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25140726c0 0x7f25141f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.777+0000 7f25127fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25140726c0 0x7f25141f4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41456/0 (socket says 172.21.15.16:41456) 2024-09-19T07:52:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.777+0000 7f25127fc640 1 -- 172.21.15.16:0/1352010805 learned_addr learned my addr 172.21.15.16:0/1352010805 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:47.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.778+0000 7f25137fe640 1 --2- 172.21.15.16:0/1352010805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25140d2750 0x7f25141f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.778+0000 7f2512ffd640 1 --2- 172.21.15.16:0/1352010805 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2514072250 0x7f251415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.778+0000 7f25127fc640 1 -- 172.21.15.16:0/1352010805 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2514072250 msgr2=0x7f251415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.778+0000 7f25127fc640 1 --2- 172.21.15.16:0/1352010805 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2514072250 0x7f251415c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.778+0000 7f25127fc640 1 -- 172.21.15.16:0/1352010805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25140d2750 msgr2=0x7f25141f36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.778+0000 7f25127fc640 1 --2- 172.21.15.16:0/1352010805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25140d2750 0x7f25141f36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.778+0000 7f25127fc640 1 -- 172.21.15.16:0/1352010805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25141f3c30 con 0x7f25140726c0 2024-09-19T07:52:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.778+0000 7f2512ffd640 1 --2- 172.21.15.16:0/1352010805 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2514072250 0x7f251415c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:52:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.778+0000 7f25127fc640 1 --2- 172.21.15.16:0/1352010805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25140726c0 0x7f25141f4830 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00b3f0 tx=0x7f24fc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.778+0000 7f250b7fe640 1 -- 172.21.15.16:0/1352010805 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc015020 con 0x7f25140726c0 2024-09-19T07:52:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.778+0000 7f250b7fe640 1 -- 172.21.15.16:0/1352010805 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24fc011830 con 0x7f25140726c0 2024-09-19T07:52:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.778+0000 7f2519778640 1 -- 172.21.15.16:0/1352010805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251415ca70 con 0x7f25140726c0 2024-09-19T07:52:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.778+0000 7f250b7fe640 1 -- 172.21.15.16:0/1352010805 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc011c20 con 0x7f25140726c0 2024-09-19T07:52:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.778+0000 7f2519778640 1 -- 172.21.15.16:0/1352010805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f251415d030 con 0x7f25140726c0 2024-09-19T07:52:47.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.779+0000 7f2519778640 1 -- 172.21.15.16:0/1352010805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25141409e0 con 0x7f25140726c0 2024-09-19T07:52:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.780+0000 7f250b7fe640 1 -- 172.21.15.16:0/1352010805 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f24fc018020 con 0x7f25140726c0 2024-09-19T07:52:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.780+0000 7f250b7fe640 1 --2- 172.21.15.16:0/1352010805 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f24f406c2b0 0x7f24f406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.780+0000 7f250b7fe640 1 -- 172.21.15.16:0/1352010805 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f24fc08b1d0 con 0x7f25140726c0 2024-09-19T07:52:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.780+0000 7f2512ffd640 1 --2- 172.21.15.16:0/1352010805 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f24f406c2b0 0x7f24f406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.781+0000 7f2512ffd640 1 --2- 172.21.15.16:0/1352010805 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f24f406c2b0 0x7f24f406e770 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2504000f60 tx=0x7f2504007c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.785+0000 7f250b7fe640 1 -- 172.21.15.16:0/1352010805 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f24fc01f030 con 0x7f25140726c0 2024-09-19T07:52:47.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.974+0000 7f2519778640 1 -- 172.21.15.16:0/1352010805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f25140b3490 con 0x7f25140726c0 2024-09-19T07:52:47.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.974+0000 7f250b7fe640 1 -- 172.21.15.16:0/1352010805 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v95) ==== 80+0+0 (secure 0 0 0) 0x7f24fc059740 con 0x7f25140726c0 2024-09-19T07:52:47.975 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:52:47.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.976+0000 7f2519778640 1 -- 172.21.15.16:0/1352010805 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f24f406c2b0 msgr2=0x7f24f406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:47.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.976+0000 7f2519778640 1 --2- 172.21.15.16:0/1352010805 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f24f406c2b0 0x7f24f406e770 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2504000f60 tx=0x7f2504007c10 comp rx=0 tx=0).stop 2024-09-19T07:52:47.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.976+0000 7f2519778640 1 -- 172.21.15.16:0/1352010805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25140726c0 msgr2=0x7f25141f4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:47.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.976+0000 7f2519778640 1 --2- 172.21.15.16:0/1352010805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25140726c0 0x7f25141f4830 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00b3f0 tx=0x7f24fc00b8b0 comp rx=0 tx=0).stop 2024-09-19T07:52:47.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.976+0000 7f2519778640 1 -- 172.21.15.16:0/1352010805 shutdown_connections 2024-09-19T07:52:47.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.976+0000 7f2519778640 1 --2- 172.21.15.16:0/1352010805 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f24f406c2b0 0x7f24f406e770 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:47.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.976+0000 7f2519778640 1 --2- 172.21.15.16:0/1352010805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25140d2750 0x7f25141f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:47.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.976+0000 7f2519778640 1 --2- 172.21.15.16:0/1352010805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25140726c0 0x7f25141f4830 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:47.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.976+0000 7f2519778640 1 --2- 172.21.15.16:0/1352010805 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2514072250 0x7f251415c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:47.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.976+0000 7f2519778640 1 -- 172.21.15.16:0/1352010805 >> 172.21.15.16:0/1352010805 conn(0x7f25140b4040 msgr2=0x7f25140b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:47.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.977+0000 7f2519778640 1 -- 172.21.15.16:0/1352010805 shutdown_connections 2024-09-19T07:52:47.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:47.977+0000 7f2519778640 1 -- 172.21.15.16:0/1352010805 wait complete. 2024-09-19T07:52:47.988 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2024-09-19T07:52:47.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:52:47.988 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T07:52:48.017 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[46954]: fuse finished with error 0 and tester_r 0 2024-09-19T07:52:48.019 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:52:48.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:52:48.019 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T07:52:48.081 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:52:48.082 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:52:48.082 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:52:48.099 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:52:48.099 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:52:48.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:52:48.099 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T07:52:48.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:52:48.156 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T07:52:48.156 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:52:48.156 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:52:48.156 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:52:48.175 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:52:48.175 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:52:48.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:52:48.175 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T07:52:48.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:52:48.233 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T07:52:48.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:52:48.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.335+0000 7faabf02e640 1 Processor -- start 2024-09-19T07:52:48.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.335+0000 7faabf02e640 1 -- start start 2024-09-19T07:52:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.336+0000 7faabf02e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faab80d21f0 0x7faab80d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.336+0000 7faabf02e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faab8141ad0 0x7faab8141f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.336+0000 7faabf02e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faab80d2b10 con 0x7faab8072590 2024-09-19T07:52:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.336+0000 7faabf02e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7faab8141750 con 0x7faab8141ad0 2024-09-19T07:52:48.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.336+0000 7faabf02e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faab806d7f0 con 0x7faab80d21f0 2024-09-19T07:52:48.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.336+0000 7faabd5cb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faab8141ad0 0x7faab8141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:48.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.336+0000 7faab7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7faab8072590 0x7faab806d0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54416/0 (socket says 172.21.15.16:54416) 2024-09-19T07:52:48.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.336+0000 7faab7fff640 1 -- 172.21.15.16:0/921153979 learned_addr learned my addr 172.21.15.16:0/921153979 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:52:48.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.336+0000 7faabcdca640 1 --2- 172.21.15.16:0/921153979 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faab80d21f0 0x7faab80d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:48.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.337+0000 7faabd5cb640 1 -- 172.21.15.16:0/921153979 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faab80d21f0 msgr2=0x7faab80d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:48.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.337+0000 7faab77fe640 1 -- 172.21.15.16:0/921153979 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3700706715 0 0) 0x7faab80d2b10 con 0x7faab8072590 2024-09-19T07:52:48.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.337+0000 7faabd5cb640 1 --2- 172.21.15.16:0/921153979 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faab80d21f0 0x7faab80d25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:48.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.337+0000 7faabd5cb640 1 -- 172.21.15.16:0/921153979 >> v1:172.21.15.16:6789/0 conn(0x7faab8072590 legacy=0x7faab806d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:48.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.337+0000 7faabd5cb640 1 -- 172.21.15.16:0/921153979 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab81620b0 con 0x7faab8141ad0 2024-09-19T07:52:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.337+0000 7faabd5cb640 1 --2- 172.21.15.16:0/921153979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8141ad0 0x7faab8141f80 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009dc0 tx=0x7faaa802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed02ce89f2841724 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.338+0000 7faab77fe640 1 -- 172.21.15.16:0/921153979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8031600 con 0x7faab8141ad0 2024-09-19T07:52:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.338+0000 7faab77fe640 1 -- 172.21.15.16:0/921153979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa80317a0 con 0x7faab8141ad0 2024-09-19T07:52:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.338+0000 7faab77fe640 1 -- 172.21.15.16:0/921153979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8031b70 con 0x7faab8141ad0 2024-09-19T07:52:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.338+0000 7faabf02e640 1 -- 172.21.15.16:0/921153979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8141ad0 msgr2=0x7faab8141f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.338+0000 7faabf02e640 1 --2- 172.21.15.16:0/921153979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8141ad0 0x7faab8141f80 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009dc0 tx=0x7faaa802a610 comp rx=0 tx=0).stop 2024-09-19T07:52:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.338+0000 7faabf02e640 1 -- 172.21.15.16:0/921153979 shutdown_connections 2024-09-19T07:52:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.338+0000 7faabf02e640 1 --2- 172.21.15.16:0/921153979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8141ad0 0x7faab8141f80 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.338+0000 7faabf02e640 1 --2- 172.21.15.16:0/921153979 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faab80d21f0 0x7faab80d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.338+0000 7faabf02e640 1 -- 172.21.15.16:0/921153979 >> 172.21.15.16:0/921153979 conn(0x7faab80b3ed0 msgr2=0x7faab80b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.338+0000 7faabf02e640 1 -- 172.21.15.16:0/921153979 shutdown_connections 2024-09-19T07:52:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.338+0000 7faabf02e640 1 -- 172.21.15.16:0/921153979 wait complete. 2024-09-19T07:52:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.339+0000 7faabf02e640 1 Processor -- start 2024-09-19T07:52:48.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.339+0000 7faabf02e640 1 -- start start 2024-09-19T07:52:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.340+0000 7faabf02e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faab8072590 0x7faab8151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.340+0000 7faabf02e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faab80d21f0 0x7faab8153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.340+0000 7faabf02e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8141ad0 0x7faab81537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.340+0000 7faabf02e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faab8162930 con 0x7faab8141ad0 2024-09-19T07:52:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.340+0000 7faabf02e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faab8162650 con 0x7faab80d21f0 2024-09-19T07:52:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.340+0000 7faabf02e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faab8162c10 con 0x7faab8072590 2024-09-19T07:52:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.340+0000 7faabd5cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8141ad0 0x7faab81537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.340+0000 7faabd5cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8141ad0 0x7faab81537d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41468/0 (socket says 172.21.15.16:41468) 2024-09-19T07:52:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.340+0000 7faabd5cb640 1 -- 172.21.15.16:0/3922842547 learned_addr learned my addr 172.21.15.16:0/3922842547 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.341+0000 7faabcdca640 1 --2- 172.21.15.16:0/3922842547 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faab8072590 0x7faab8151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.341+0000 7faab7fff640 1 --2- 172.21.15.16:0/3922842547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faab80d21f0 0x7faab8153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.341+0000 7faabd5cb640 1 -- 172.21.15.16:0/3922842547 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faab8072590 msgr2=0x7faab8151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.341+0000 7faabd5cb640 1 --2- 172.21.15.16:0/3922842547 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faab8072590 0x7faab8151280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.341+0000 7faabd5cb640 1 -- 172.21.15.16:0/3922842547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faab80d21f0 msgr2=0x7faab8153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.341+0000 7faabd5cb640 1 --2- 172.21.15.16:0/3922842547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faab80d21f0 0x7faab8153290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.341+0000 7faabd5cb640 1 -- 172.21.15.16:0/3922842547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab8153ec0 con 0x7faab8141ad0 2024-09-19T07:52:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.341+0000 7faabd5cb640 1 --2- 172.21.15.16:0/3922842547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8141ad0 0x7faab81537d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009ee0 tx=0x7faaa8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.341+0000 7faab57fa640 1 -- 172.21.15.16:0/3922842547 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8040070 con 0x7faab8141ad0 2024-09-19T07:52:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.341+0000 7faab57fa640 1 -- 172.21.15.16:0/3922842547 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa8030b60 con 0x7faab8141ad0 2024-09-19T07:52:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.341+0000 7faab57fa640 1 -- 172.21.15.16:0/3922842547 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8009410 con 0x7faab8141ad0 2024-09-19T07:52:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.341+0000 7faabf02e640 1 -- 172.21.15.16:0/3922842547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab81541a0 con 0x7faab8141ad0 2024-09-19T07:52:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.341+0000 7faabf02e640 1 -- 172.21.15.16:0/3922842547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab81596f0 con 0x7faab8141ad0 2024-09-19T07:52:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.342+0000 7faabf02e640 1 -- 172.21.15.16:0/3922842547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faab8158c30 con 0x7faab8141ad0 2024-09-19T07:52:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.343+0000 7faab57fa640 1 -- 172.21.15.16:0/3922842547 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7faaa8039040 con 0x7faab8141ad0 2024-09-19T07:52:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.343+0000 7faab57fa640 1 --2- 172.21.15.16:0/3922842547 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faa9806c2b0 0x7faa9806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.343+0000 7faab57fa640 1 -- 172.21.15.16:0/3922842547 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7faaa80acbe0 con 0x7faab8141ad0 2024-09-19T07:52:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.343+0000 7faabcdca640 1 --2- 172.21.15.16:0/3922842547 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faa9806c2b0 0x7faa9806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.344+0000 7faabcdca640 1 --2- 172.21.15.16:0/3922842547 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faa9806c2b0 0x7faa9806e770 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7faab0006fd0 tx=0x7faab0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:48.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.348+0000 7faab57fa640 1 -- 172.21.15.16:0/3922842547 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7faaa807b150 con 0x7faab8141ad0 2024-09-19T07:52:48.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.560+0000 7faabf02e640 1 -- 172.21.15.16:0/3922842547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faab80690c0 con 0x7faab8141ad0 2024-09-19T07:52:48.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.560+0000 7faab57fa640 1 -- 172.21.15.16:0/3922842547 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5231 (secure 0 0 0) 0x7faaa807aa50 con 0x7faab8141ad0 2024-09-19T07:52:48.561 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:52:48.561 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":49,"btime":"2024-09-19T07:50:36:200205+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5684,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1865971154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1865971154},{"type":"v1","addr":"172.21.15.146:6837","nonce":1865971154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6094,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/2735152611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2735152611},{"type":"v1","addr":"172.21.15.146:6836","nonce":2735152611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6399,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4204683210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4204683210},{"type":"v1","addr":"172.21.15.16:6834","nonce":4204683210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:50:15.050678+0000","modified":"2024-09-19T07:50:35.729727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6402":{"gid":6402,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/1305244492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1305244492},{"type":"v1","addr":"172.21.15.16:6835","nonce":1305244492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6402,"qdb_cluster":[6402]},"id":6}]} 2024-09-19T07:52:48.561 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 49 2024-09-19T07:52:48.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.562+0000 7faabf02e640 1 -- 172.21.15.16:0/3922842547 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faa9806c2b0 msgr2=0x7faa9806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:48.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.562+0000 7faabf02e640 1 --2- 172.21.15.16:0/3922842547 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faa9806c2b0 0x7faa9806e770 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7faab0006fd0 tx=0x7faab0008040 comp rx=0 tx=0).stop 2024-09-19T07:52:48.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.562+0000 7faabf02e640 1 -- 172.21.15.16:0/3922842547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8141ad0 msgr2=0x7faab81537d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:48.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.562+0000 7faabf02e640 1 --2- 172.21.15.16:0/3922842547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8141ad0 0x7faab81537d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009ee0 tx=0x7faaa8004a40 comp rx=0 tx=0).stop 2024-09-19T07:52:48.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.562+0000 7faabf02e640 1 -- 172.21.15.16:0/3922842547 shutdown_connections 2024-09-19T07:52:48.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.562+0000 7faabf02e640 1 --2- 172.21.15.16:0/3922842547 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faa9806c2b0 0x7faa9806e770 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:48.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.562+0000 7faabf02e640 1 --2- 172.21.15.16:0/3922842547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8141ad0 0x7faab81537d0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:48.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.562+0000 7faabf02e640 1 --2- 172.21.15.16:0/3922842547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faab80d21f0 0x7faab8153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:48.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.562+0000 7faabf02e640 1 --2- 172.21.15.16:0/3922842547 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faab8072590 0x7faab8151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:48.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.563+0000 7faabf02e640 1 -- 172.21.15.16:0/3922842547 >> 172.21.15.16:0/3922842547 conn(0x7faab80b3ed0 msgr2=0x7faab80d0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:48.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.563+0000 7faabf02e640 1 -- 172.21.15.16:0/3922842547 shutdown_connections 2024-09-19T07:52:48.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.563+0000 7faabf02e640 1 -- 172.21.15.16:0/3922842547 wait complete. 2024-09-19T07:52:48.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:52:48.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.675+0000 7f574b7cf640 1 Processor -- start 2024-09-19T07:52:48.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.676+0000 7f574b7cf640 1 -- start start 2024-09-19T07:52:48.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.676+0000 7f574b7cf640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f574415f0d0 0x7f574415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:48.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.676+0000 7f574b7cf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57440d1b10 con 0x7f57440d4130 2024-09-19T07:52:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.676+0000 7f574b7cf640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57440cf2b0 con 0x7f57440d0070 2024-09-19T07:52:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.676+0000 7f574b7cf640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f57440cf590 con 0x7f574415f0d0 2024-09-19T07:52:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.677+0000 7f5749d6c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f57440d4130 0x7f57440d4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54418/0 (socket says 172.21.15.16:54418) 2024-09-19T07:52:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.677+0000 7f5749d6c640 1 -- 172.21.15.16:0/1846294646 learned_addr learned my addr 172.21.15.16:0/1846294646 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:52:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.677+0000 7f574956b640 1 --2- 172.21.15.16:0/1846294646 >> v2:172.21.15.146:3300/0 conn(0x7f574415f0d0 0x7f574415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.677+0000 7f5743fff640 1 -- 172.21.15.16:0/1846294646 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1707275821 0 0) 0x7f57440d1b10 con 0x7f57440d4130 2024-09-19T07:52:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.677+0000 7f5743fff640 1 -- 172.21.15.16:0/1846294646 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f572c003520 con 0x7f57440d4130 2024-09-19T07:52:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.677+0000 7f574956b640 1 -- 172.21.15.16:0/1846294646 >> v1:172.21.15.146:6789/0 conn(0x7f57440d0070 legacy=0x7f57440d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.677+0000 7f574956b640 1 -- 172.21.15.16:0/1846294646 >> v1:172.21.15.16:6789/0 conn(0x7f57440d4130 legacy=0x7f57440d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.677+0000 7f5743fff640 1 -- 172.21.15.16:0/1846294646 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1254789675 0 0) 0x7f57440cf2b0 con 0x7f57440d0070 2024-09-19T07:52:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.677+0000 7f574956b640 1 -- 172.21.15.16:0/1846294646 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57440cf870 con 0x7f574415f0d0 2024-09-19T07:52:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.678+0000 7f574956b640 1 --2- 172.21.15.16:0/1846294646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f574415f0d0 0x7f574415f4b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f573c0097a0 tx=0x7f573c02a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b00689f0ba7cb2c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.678+0000 7f5743fff640 1 -- 172.21.15.16:0/1846294646 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c037070 con 0x7f574415f0d0 2024-09-19T07:52:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.678+0000 7f5743fff640 1 -- 172.21.15.16:0/1846294646 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f573c02ab10 con 0x7f574415f0d0 2024-09-19T07:52:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.678+0000 7f5743fff640 1 -- 172.21.15.16:0/1846294646 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5734001990 con 0x7f574415f0d0 2024-09-19T07:52:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.679+0000 7f574b7cf640 1 -- 172.21.15.16:0/1846294646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f574415f0d0 msgr2=0x7f574415f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.679+0000 7f574b7cf640 1 --2- 172.21.15.16:0/1846294646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f574415f0d0 0x7f574415f4b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f573c0097a0 tx=0x7f573c02a080 comp rx=0 tx=0).stop 2024-09-19T07:52:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.679+0000 7f574b7cf640 1 -- 172.21.15.16:0/1846294646 shutdown_connections 2024-09-19T07:52:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.679+0000 7f574b7cf640 1 --2- 172.21.15.16:0/1846294646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f574415f0d0 0x7f574415f4b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.679+0000 7f574b7cf640 1 -- 172.21.15.16:0/1846294646 >> 172.21.15.16:0/1846294646 conn(0x7f574409d760 msgr2=0x7f574408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.679+0000 7f574b7cf640 1 -- 172.21.15.16:0/1846294646 shutdown_connections 2024-09-19T07:52:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.679+0000 7f574b7cf640 1 -- 172.21.15.16:0/1846294646 wait complete. 2024-09-19T07:52:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.680+0000 7f574b7cf640 1 Processor -- start 2024-09-19T07:52:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.680+0000 7f574b7cf640 1 -- start start 2024-09-19T07:52:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.681+0000 7f574b7cf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57440d0070 0x7f5744157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.681+0000 7f574b7cf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57440d4130 0x7f5744160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.681+0000 7f574b7cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f574415f0d0 0x7f5744158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.681+0000 7f574b7cf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57440b2f70 con 0x7f574415f0d0 2024-09-19T07:52:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.681+0000 7f574b7cf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f57440cef60 con 0x7f57440d4130 2024-09-19T07:52:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.681+0000 7f574b7cf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f57440b30f0 con 0x7f57440d0070 2024-09-19T07:52:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.681+0000 7f5749d6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f574415f0d0 0x7f5744158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.681+0000 7f5749d6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f574415f0d0 0x7f5744158c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41474/0 (socket says 172.21.15.16:41474) 2024-09-19T07:52:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.681+0000 7f5749d6c640 1 -- 172.21.15.16:0/3416601064 learned_addr learned my addr 172.21.15.16:0/3416601064 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.681+0000 7f574956b640 1 --2- 172.21.15.16:0/3416601064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57440d0070 0x7f5744157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.681+0000 7f5749d6c640 1 -- 172.21.15.16:0/3416601064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57440d0070 msgr2=0x7f5744157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.681+0000 7f5749d6c640 1 --2- 172.21.15.16:0/3416601064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57440d0070 0x7f5744157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.681+0000 7f5748d6a640 1 --2- 172.21.15.16:0/3416601064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57440d4130 0x7f5744160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.681+0000 7f5749d6c640 1 -- 172.21.15.16:0/3416601064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57440d4130 msgr2=0x7f5744160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.681+0000 7f5749d6c640 1 --2- 172.21.15.16:0/3416601064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57440d4130 0x7f5744160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.681+0000 7f5749d6c640 1 -- 172.21.15.16:0/3416601064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57441614a0 con 0x7f574415f0d0 2024-09-19T07:52:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.681+0000 7f5748d6a640 1 --2- 172.21.15.16:0/3416601064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57440d4130 0x7f5744160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:52:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.681+0000 7f5749d6c640 1 --2- 172.21.15.16:0/3416601064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f574415f0d0 0x7f5744158c80 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f573400cab0 tx=0x7f573400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.682+0000 7f5741ffb640 1 -- 172.21.15.16:0/3416601064 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5734018070 con 0x7f574415f0d0 2024-09-19T07:52:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.682+0000 7f5741ffb640 1 -- 172.21.15.16:0/3416601064 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f573400a730 con 0x7f574415f0d0 2024-09-19T07:52:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.682+0000 7f5741ffb640 1 -- 172.21.15.16:0/3416601064 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573400ab00 con 0x7f574415f0d0 2024-09-19T07:52:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.682+0000 7f574b7cf640 1 -- 172.21.15.16:0/3416601064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57441f7830 con 0x7f574415f0d0 2024-09-19T07:52:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.682+0000 7f574b7cf640 1 -- 172.21.15.16:0/3416601064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57441f7df0 con 0x7f574415f0d0 2024-09-19T07:52:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.683+0000 7f574b7cf640 1 -- 172.21.15.16:0/3416601064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57441f7ff0 con 0x7f574415f0d0 2024-09-19T07:52:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.683+0000 7f5741ffb640 1 -- 172.21.15.16:0/3416601064 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f573401b070 con 0x7f574415f0d0 2024-09-19T07:52:48.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.684+0000 7f5741ffb640 1 --2- 172.21.15.16:0/3416601064 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f573006c2b0 0x7f573006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:48.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.684+0000 7f5741ffb640 1 -- 172.21.15.16:0/3416601064 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f573408aee0 con 0x7f574415f0d0 2024-09-19T07:52:48.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.684+0000 7f574956b640 1 --2- 172.21.15.16:0/3416601064 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f573006c2b0 0x7f573006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:48.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.685+0000 7f574956b640 1 --2- 172.21.15.16:0/3416601064 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f573006c2b0 0x7f573006e770 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f573c034000 tx=0x7f573c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:48.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.688+0000 7f5741ffb640 1 -- 172.21.15.16:0/3416601064 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f57441f7ff0 con 0x7f574415f0d0 2024-09-19T07:52:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.903+0000 7f574b7cf640 1 -- 172.21.15.16:0/3416601064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f574406cf30 con 0x7f574415f0d0 2024-09-19T07:52:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.903+0000 7f5741ffb640 1 -- 172.21.15.16:0/3416601064 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5231 (secure 0 0 0) 0x7f5734059450 con 0x7f574415f0d0 2024-09-19T07:52:48.904 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:52:48.904 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":49,"btime":"2024-09-19T07:50:36:200205+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5684,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1865971154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1865971154},{"type":"v1","addr":"172.21.15.146:6837","nonce":1865971154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6094,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/2735152611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2735152611},{"type":"v1","addr":"172.21.15.146:6836","nonce":2735152611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6399,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4204683210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4204683210},{"type":"v1","addr":"172.21.15.16:6834","nonce":4204683210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:50:15.050678+0000","modified":"2024-09-19T07:50:35.729727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6402":{"gid":6402,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/1305244492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1305244492},{"type":"v1","addr":"172.21.15.16:6835","nonce":1305244492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6402,"qdb_cluster":[6402]},"id":6}]} 2024-09-19T07:52:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 49 2024-09-19T07:52:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.905+0000 7f574b7cf640 1 -- 172.21.15.16:0/3416601064 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f573006c2b0 msgr2=0x7f573006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.905+0000 7f574b7cf640 1 --2- 172.21.15.16:0/3416601064 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f573006c2b0 0x7f573006e770 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f573c034000 tx=0x7f573c02e000 comp rx=0 tx=0).stop 2024-09-19T07:52:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.905+0000 7f574b7cf640 1 -- 172.21.15.16:0/3416601064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f574415f0d0 msgr2=0x7f5744158c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.905+0000 7f574b7cf640 1 --2- 172.21.15.16:0/3416601064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f574415f0d0 0x7f5744158c80 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f573400cab0 tx=0x7f573400cf70 comp rx=0 tx=0).stop 2024-09-19T07:52:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.905+0000 7f574b7cf640 1 -- 172.21.15.16:0/3416601064 shutdown_connections 2024-09-19T07:52:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.905+0000 7f574b7cf640 1 --2- 172.21.15.16:0/3416601064 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f573006c2b0 0x7f573006e770 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.905+0000 7f574b7cf640 1 --2- 172.21.15.16:0/3416601064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f574415f0d0 0x7f5744158c80 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.905+0000 7f574b7cf640 1 --2- 172.21.15.16:0/3416601064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57440d4130 0x7f5744160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.905+0000 7f574b7cf640 1 --2- 172.21.15.16:0/3416601064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57440d0070 0x7f5744157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.905+0000 7f574b7cf640 1 -- 172.21.15.16:0/3416601064 >> 172.21.15.16:0/3416601064 conn(0x7f574409d760 msgr2=0x7f574406dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.905+0000 7f574b7cf640 1 -- 172.21.15.16:0/3416601064 shutdown_connections 2024-09-19T07:52:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:48.905+0000 7f574b7cf640 1 -- 172.21.15.16:0/3416601064 wait complete. 2024-09-19T07:52:48.917 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:52:49.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.018+0000 7f01c62ae640 1 Processor -- start 2024-09-19T07:52:49.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.018+0000 7f01c62ae640 1 -- start start 2024-09-19T07:52:49.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.019+0000 7f01c62ae640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f01c0072590 0x7f01c006d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:49.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.019+0000 7f01c62ae640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01c01620b0 con 0x7f01c00d21f0 2024-09-19T07:52:49.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.019+0000 7f01c62ae640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01c0162370 con 0x7f01c0141ad0 2024-09-19T07:52:49.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.019+0000 7f01c62ae640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f01c0141750 con 0x7f01c0072590 2024-09-19T07:52:49.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.019+0000 7f01bffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f01c00d21f0 0x7f01c00d25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54434/0 (socket says 172.21.15.16:54434) 2024-09-19T07:52:49.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.019+0000 7f01bffff640 1 -- 172.21.15.16:0/4088599133 learned_addr learned my addr 172.21.15.16:0/4088599133 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:52:49.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.019+0000 7f01bf7fe640 1 --2- 172.21.15.16:0/4088599133 >> v2:172.21.15.146:3300/0 conn(0x7f01c0072590 0x7f01c006d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:49.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.019+0000 7f01beffd640 1 -- 172.21.15.16:0/4088599133 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2402187729 0 0) 0x7f01c01620b0 con 0x7f01c00d21f0 2024-09-19T07:52:49.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.019+0000 7f01beffd640 1 -- 172.21.15.16:0/4088599133 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01ac003520 con 0x7f01c00d21f0 2024-09-19T07:52:49.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.019+0000 7f01beffd640 1 -- 172.21.15.16:0/4088599133 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2524117180 0 0) 0x7f01c0162370 con 0x7f01c0141ad0 2024-09-19T07:52:49.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.019+0000 7f01beffd640 1 -- 172.21.15.16:0/4088599133 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01c01620b0 con 0x7f01c0141ad0 2024-09-19T07:52:49.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.020+0000 7f01bf7fe640 1 -- 172.21.15.16:0/4088599133 >> v1:172.21.15.146:6789/0 conn(0x7f01c0141ad0 legacy=0x7f01c0141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:49.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.020+0000 7f01bf7fe640 1 -- 172.21.15.16:0/4088599133 >> v1:172.21.15.16:6789/0 conn(0x7f01c00d21f0 legacy=0x7f01c00d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:49.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.020+0000 7f01bf7fe640 1 -- 172.21.15.16:0/4088599133 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c0162650 con 0x7f01c0072590 2024-09-19T07:52:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.020+0000 7f01bf7fe640 1 --2- 172.21.15.16:0/4088599133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01c0072590 0x7f01c006d0e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f01a80097a0 tx=0x7f01a802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59af41e79b6427a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.021+0000 7f01beffd640 1 -- 172.21.15.16:0/4088599133 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8037070 con 0x7f01c0072590 2024-09-19T07:52:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.021+0000 7f01beffd640 1 -- 172.21.15.16:0/4088599133 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a8032440 con 0x7f01c0072590 2024-09-19T07:52:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.021+0000 7f01beffd640 1 -- 172.21.15.16:0/4088599133 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8032810 con 0x7f01c0072590 2024-09-19T07:52:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.021+0000 7f01c62ae640 1 -- 172.21.15.16:0/4088599133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01c0072590 msgr2=0x7f01c006d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.021+0000 7f01c62ae640 1 --2- 172.21.15.16:0/4088599133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01c0072590 0x7f01c006d0e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f01a80097a0 tx=0x7f01a802a1a0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.021+0000 7f01c62ae640 1 -- 172.21.15.16:0/4088599133 shutdown_connections 2024-09-19T07:52:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.021+0000 7f01c62ae640 1 --2- 172.21.15.16:0/4088599133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01c0072590 0x7f01c006d0e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.021+0000 7f01c62ae640 1 -- 172.21.15.16:0/4088599133 >> 172.21.15.16:0/4088599133 conn(0x7f01c00b3ed0 msgr2=0x7f01c00b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.021+0000 7f01c62ae640 1 -- 172.21.15.16:0/4088599133 shutdown_connections 2024-09-19T07:52:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.021+0000 7f01c62ae640 1 -- 172.21.15.16:0/4088599133 wait complete. 2024-09-19T07:52:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.022+0000 7f01c62ae640 1 Processor -- start 2024-09-19T07:52:49.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.022+0000 7f01c62ae640 1 -- start start 2024-09-19T07:52:49.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.023+0000 7f01c62ae640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01c0072590 0x7f01c0159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:49.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.023+0000 7f01c62ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c00d21f0 0x7f01c0140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:49.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.023+0000 7f01c62ae640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01c0141ad0 0x7f01c015e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:49.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.023+0000 7f01c62ae640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01c006d620 con 0x7f01c00d21f0 2024-09-19T07:52:49.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.023+0000 7f01c62ae640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0162c10 con 0x7f01c0072590 2024-09-19T07:52:49.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.023+0000 7f01c62ae640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f01c006d7a0 con 0x7f01c0141ad0 2024-09-19T07:52:49.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.023+0000 7f01bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c00d21f0 0x7f01c0140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:49.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.023+0000 7f01bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c00d21f0 0x7f01c0140c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41478/0 (socket says 172.21.15.16:41478) 2024-09-19T07:52:49.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.023+0000 7f01bf7fe640 1 -- 172.21.15.16:0/2516756210 learned_addr learned my addr 172.21.15.16:0/2516756210 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:49.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.023+0000 7f01bffff640 1 --2- 172.21.15.16:0/2516756210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01c0072590 0x7f01c0159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:49.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.023+0000 7f01c484b640 1 --2- 172.21.15.16:0/2516756210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01c0141ad0 0x7f01c015e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:49.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.024+0000 7f01bf7fe640 1 -- 172.21.15.16:0/2516756210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01c0141ad0 msgr2=0x7f01c015e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:49.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.024+0000 7f01bf7fe640 1 --2- 172.21.15.16:0/2516756210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01c0141ad0 0x7f01c015e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.024+0000 7f01bf7fe640 1 -- 172.21.15.16:0/2516756210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01c0072590 msgr2=0x7f01c0159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:49.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.024+0000 7f01bf7fe640 1 --2- 172.21.15.16:0/2516756210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01c0072590 0x7f01c0159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.024+0000 7f01bf7fe640 1 -- 172.21.15.16:0/2516756210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c00b5fb0 con 0x7f01c00d21f0 2024-09-19T07:52:49.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.024+0000 7f01bf7fe640 1 --2- 172.21.15.16:0/2516756210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c00d21f0 0x7f01c0140c70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f01a8035040 tx=0x7f01a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:49.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.024+0000 7f01bcff9640 1 -- 172.21.15.16:0/2516756210 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8041030 con 0x7f01c00d21f0 2024-09-19T07:52:49.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.024+0000 7f01bcff9640 1 -- 172.21.15.16:0/2516756210 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a803d4e0 con 0x7f01c00d21f0 2024-09-19T07:52:49.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.024+0000 7f01c62ae640 1 -- 172.21.15.16:0/2516756210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c015d750 con 0x7f01c00d21f0 2024-09-19T07:52:49.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.024+0000 7f01bcff9640 1 -- 172.21.15.16:0/2516756210 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8037040 con 0x7f01c00d21f0 2024-09-19T07:52:49.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.024+0000 7f01c62ae640 1 -- 172.21.15.16:0/2516756210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c015dd10 con 0x7f01c00d21f0 2024-09-19T07:52:49.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.025+0000 7f01c62ae640 1 -- 172.21.15.16:0/2516756210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c0158c30 con 0x7f01c00d21f0 2024-09-19T07:52:49.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.025+0000 7f01bcff9640 1 -- 172.21.15.16:0/2516756210 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f01a8044070 con 0x7f01c00d21f0 2024-09-19T07:52:49.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.026+0000 7f01bcff9640 1 --2- 172.21.15.16:0/2516756210 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01a006c2b0 0x7f01a006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:49.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.026+0000 7f01bcff9640 1 -- 172.21.15.16:0/2516756210 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f01a80abca0 con 0x7f01c00d21f0 2024-09-19T07:52:49.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.026+0000 7f01bffff640 1 --2- 172.21.15.16:0/2516756210 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01a006c2b0 0x7f01a006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:49.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.027+0000 7f01bffff640 1 --2- 172.21.15.16:0/2516756210 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01a006c2b0 0x7f01a006e770 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f01b0000f60 tx=0x7f01b0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:49.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.030+0000 7f01bcff9640 1 -- 172.21.15.16:0/2516756210 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f01a807a220 con 0x7f01c00d21f0 2024-09-19T07:52:49.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.206+0000 7f01c62ae640 1 -- 172.21.15.16:0/2516756210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01c00722c0 con 0x7f01c00d21f0 2024-09-19T07:52:49.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.208+0000 7f01bcff9640 1 -- 172.21.15.16:0/2516756210 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+17163 (secure 0 0 0) 0x7f01a8079b20 con 0x7f01c00d21f0 2024-09-19T07:52:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:52:49.208 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":95,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:50:49.229309+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:50:13.015554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-19T07:50:14.031234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.21","mappings":[{"from":1,"to":2}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]},{"pgid":"14.2c","mappings":[{"from":1,"to":5}]},{"pgid":"14.3c","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:52:49.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.209+0000 7f01c62ae640 1 -- 172.21.15.16:0/2516756210 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01a006c2b0 msgr2=0x7f01a006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:49.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.209+0000 7f01c62ae640 1 --2- 172.21.15.16:0/2516756210 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01a006c2b0 0x7f01a006e770 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f01b0000f60 tx=0x7f01b0009040 comp rx=0 tx=0).stop 2024-09-19T07:52:49.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.209+0000 7f01c62ae640 1 -- 172.21.15.16:0/2516756210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c00d21f0 msgr2=0x7f01c0140c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:49.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.209+0000 7f01c62ae640 1 --2- 172.21.15.16:0/2516756210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c00d21f0 0x7f01c0140c70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f01a8035040 tx=0x7f01a80046c0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.210+0000 7f01c62ae640 1 -- 172.21.15.16:0/2516756210 shutdown_connections 2024-09-19T07:52:49.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.210+0000 7f01c62ae640 1 --2- 172.21.15.16:0/2516756210 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01a006c2b0 0x7f01a006e770 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.210+0000 7f01c62ae640 1 --2- 172.21.15.16:0/2516756210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01c0141ad0 0x7f01c015e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.210+0000 7f01c62ae640 1 --2- 172.21.15.16:0/2516756210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c00d21f0 0x7f01c0140c70 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.210+0000 7f01c62ae640 1 --2- 172.21.15.16:0/2516756210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01c0072590 0x7f01c0159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.210+0000 7f01c62ae640 1 -- 172.21.15.16:0/2516756210 >> 172.21.15.16:0/2516756210 conn(0x7f01c00b3ed0 msgr2=0x7f01c00b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:49.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.210+0000 7f01c62ae640 1 -- 172.21.15.16:0/2516756210 shutdown_connections 2024-09-19T07:52:49.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.210+0000 7f01c62ae640 1 -- 172.21.15.16:0/2516756210 wait complete. 2024-09-19T07:52:49.221 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T07:52:49.221 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:52:49.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.324+0000 7fbf1b5a6640 1 Processor -- start 2024-09-19T07:52:49.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.324+0000 7fbf1b5a6640 1 -- start start 2024-09-19T07:52:49.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.325+0000 7fbf1b5a6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbf1415e2a0 0x7fbf1406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:49.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.325+0000 7fbf1b5a6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbf1406d960 0x7fbf140d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:49.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.325+0000 7fbf1b5a6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf140d1880 con 0x7fbf1415e680 2024-09-19T07:52:49.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.325+0000 7fbf1b5a6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fbf140d1b60 con 0x7fbf1415e2a0 2024-09-19T07:52:49.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.325+0000 7fbf1b5a6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fbf140cf2d0 con 0x7fbf1406d960 2024-09-19T07:52:49.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.325+0000 7fbf19342640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbf1415e2a0 0x7fbf1406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:49.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.325+0000 7fbf19342640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbf1415e2a0 0x7fbf1406d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41480/0 (socket says 172.21.15.16:41480) 2024-09-19T07:52:49.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.325+0000 7fbf19342640 1 -- 172.21.15.16:0/1946733071 learned_addr learned my addr 172.21.15.16:0/1946733071 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:49.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.325+0000 7fbf19b43640 1 --2- 172.21.15.16:0/1946733071 >> v2:172.21.15.146:3300/0 conn(0x7fbf1406d960 0x7fbf140d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:49.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.325+0000 7fbf19342640 1 -- 172.21.15.16:0/1946733071 >> v2:172.21.15.146:3300/0 conn(0x7fbf1406d960 msgr2=0x7fbf140d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:49.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.325+0000 7fbf19342640 1 --2- 172.21.15.16:0/1946733071 >> v2:172.21.15.146:3300/0 conn(0x7fbf1406d960 0x7fbf140d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.325+0000 7fbf19342640 1 -- 172.21.15.16:0/1946733071 >> v1:172.21.15.146:6789/0 conn(0x7fbf1415e680 legacy=0x7fbf140cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:49.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.325+0000 7fbf19342640 1 -- 172.21.15.16:0/1946733071 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf140cf5b0 con 0x7fbf1415e2a0 2024-09-19T07:52:49.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.326+0000 7fbf19342640 1 --2- 172.21.15.16:0/1946733071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1415e2a0 0x7fbf1406d420 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fbf14163050 tx=0x7fbf1002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c578fdcdffacda server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:49.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.326+0000 7fbf07fff640 1 -- 172.21.15.16:0/1946733071 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10037070 con 0x7fbf1415e2a0 2024-09-19T07:52:49.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.326+0000 7fbf07fff640 1 -- 172.21.15.16:0/1946733071 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf10032440 con 0x7fbf1415e2a0 2024-09-19T07:52:49.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.326+0000 7fbf07fff640 1 -- 172.21.15.16:0/1946733071 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10032810 con 0x7fbf1415e2a0 2024-09-19T07:52:49.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.327+0000 7fbf1b5a6640 1 -- 172.21.15.16:0/1946733071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1415e2a0 msgr2=0x7fbf1406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:49.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.327+0000 7fbf1b5a6640 1 --2- 172.21.15.16:0/1946733071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1415e2a0 0x7fbf1406d420 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fbf14163050 tx=0x7fbf1002a230 comp rx=0 tx=0).stop 2024-09-19T07:52:49.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.327+0000 7fbf1b5a6640 1 -- 172.21.15.16:0/1946733071 shutdown_connections 2024-09-19T07:52:49.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.327+0000 7fbf1b5a6640 1 --2- 172.21.15.16:0/1946733071 >> v2:172.21.15.146:3300/0 conn(0x7fbf1406d960 0x7fbf140d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.327+0000 7fbf1b5a6640 1 --2- 172.21.15.16:0/1946733071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1415e2a0 0x7fbf1406d420 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.327+0000 7fbf1b5a6640 1 -- 172.21.15.16:0/1946733071 >> 172.21.15.16:0/1946733071 conn(0x7fbf1409d760 msgr2=0x7fbf1408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:49.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.327+0000 7fbf1b5a6640 1 -- 172.21.15.16:0/1946733071 shutdown_connections 2024-09-19T07:52:49.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.327+0000 7fbf1b5a6640 1 -- 172.21.15.16:0/1946733071 wait complete. 2024-09-19T07:52:49.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.328+0000 7fbf1b5a6640 1 Processor -- start 2024-09-19T07:52:49.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.328+0000 7fbf1b5a6640 1 -- start start 2024-09-19T07:52:49.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.329+0000 7fbf1b5a6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf1406d960 0x7fbf140ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:49.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.329+0000 7fbf1b5a6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf1415e2a0 0x7fbf140ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:49.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.329+0000 7fbf1b5a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1415e680 0x7fbf140ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:49.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.329+0000 7fbf1b5a6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbf140d0360 con 0x7fbf1415e680 2024-09-19T07:52:49.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.329+0000 7fbf1b5a6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbf140d01e0 con 0x7fbf1406d960 2024-09-19T07:52:49.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.329+0000 7fbf1b5a6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbf140d48b0 con 0x7fbf1415e2a0 2024-09-19T07:52:49.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.329+0000 7fbf19b43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1415e680 0x7fbf140ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:49.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.329+0000 7fbf19b43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1415e680 0x7fbf140ccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41488/0 (socket says 172.21.15.16:41488) 2024-09-19T07:52:49.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.329+0000 7fbf19b43640 1 -- 172.21.15.16:0/3832460048 learned_addr learned my addr 172.21.15.16:0/3832460048 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:49.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.329+0000 7fbf19342640 1 --2- 172.21.15.16:0/3832460048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf1406d960 0x7fbf140ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:49.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.329+0000 7fbf18b41640 1 --2- 172.21.15.16:0/3832460048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf1415e2a0 0x7fbf140ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:49.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.329+0000 7fbf19b43640 1 -- 172.21.15.16:0/3832460048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf1415e2a0 msgr2=0x7fbf140ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:49.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.329+0000 7fbf19b43640 1 --2- 172.21.15.16:0/3832460048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf1415e2a0 0x7fbf140ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.329+0000 7fbf19b43640 1 -- 172.21.15.16:0/3832460048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf1406d960 msgr2=0x7fbf140ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:49.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.329+0000 7fbf19b43640 1 --2- 172.21.15.16:0/3832460048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf1406d960 0x7fbf140ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.329+0000 7fbf19b43640 1 -- 172.21.15.16:0/3832460048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf140d0e10 con 0x7fbf1415e680 2024-09-19T07:52:49.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.329+0000 7fbf18b41640 1 --2- 172.21.15.16:0/3832460048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf1415e2a0 0x7fbf140ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:52:49.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.329+0000 7fbf19b43640 1 --2- 172.21.15.16:0/3832460048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1415e680 0x7fbf140ccb40 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c007e40 tx=0x7fbf0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:49.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.330+0000 7fbf05ffb640 1 -- 172.21.15.16:0/3832460048 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c014620 con 0x7fbf1415e680 2024-09-19T07:52:49.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.330+0000 7fbf05ffb640 1 -- 172.21.15.16:0/3832460048 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf0c019070 con 0x7fbf1415e680 2024-09-19T07:52:49.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.330+0000 7fbf1b5a6640 1 -- 172.21.15.16:0/3832460048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf140cc060 con 0x7fbf1415e680 2024-09-19T07:52:49.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.330+0000 7fbf05ffb640 1 -- 172.21.15.16:0/3832460048 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c014a20 con 0x7fbf1415e680 2024-09-19T07:52:49.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.330+0000 7fbf1b5a6640 1 -- 172.21.15.16:0/3832460048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf140cc620 con 0x7fbf1415e680 2024-09-19T07:52:49.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.331+0000 7fbf1b5a6640 1 -- 172.21.15.16:0/3832460048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf140cbe90 con 0x7fbf1415e680 2024-09-19T07:52:49.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.331+0000 7fbf05ffb640 1 -- 172.21.15.16:0/3832460048 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbf0c014bc0 con 0x7fbf1415e680 2024-09-19T07:52:49.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.331+0000 7fbf05ffb640 1 --2- 172.21.15.16:0/3832460048 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbef806c2b0 0x7fbef806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:49.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.332+0000 7fbf05ffb640 1 -- 172.21.15.16:0/3832460048 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7fbf0c08beb0 con 0x7fbf1415e680 2024-09-19T07:52:49.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.332+0000 7fbf19342640 1 --2- 172.21.15.16:0/3832460048 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbef806c2b0 0x7fbef806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:49.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.332+0000 7fbf19342640 1 --2- 172.21.15.16:0/3832460048 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbef806c2b0 0x7fbef806e770 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbf10034000 tx=0x7fbf1002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:49.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.336+0000 7fbf05ffb640 1 -- 172.21.15.16:0/3832460048 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fbf0c05a410 con 0x7fbf1415e680 2024-09-19T07:52:49.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.549+0000 7fbf1b5a6640 1 -- 172.21.15.16:0/3832460048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf140d4c70 con 0x7fbf1415e680 2024-09-19T07:52:49.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.549+0000 7fbf05ffb640 1 -- 172.21.15.16:0/3832460048 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5231 (secure 0 0 0) 0x7fbf0c059d10 con 0x7fbf1415e680 2024-09-19T07:52:49.550 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:52:49.550 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":49,"btime":"2024-09-19T07:50:36:200205+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5684,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1865971154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1865971154},{"type":"v1","addr":"172.21.15.146:6837","nonce":1865971154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6094,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/2735152611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2735152611},{"type":"v1","addr":"172.21.15.146:6836","nonce":2735152611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6399,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4204683210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4204683210},{"type":"v1","addr":"172.21.15.16:6834","nonce":4204683210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:50:15.050678+0000","modified":"2024-09-19T07:50:35.729727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6402":{"gid":6402,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/1305244492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1305244492},{"type":"v1","addr":"172.21.15.16:6835","nonce":1305244492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6402,"qdb_cluster":[6402]},"id":6}]} 2024-09-19T07:52:49.550 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 49 2024-09-19T07:52:49.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.552+0000 7fbf1b5a6640 1 -- 172.21.15.16:0/3832460048 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbef806c2b0 msgr2=0x7fbef806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:49.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.552+0000 7fbf1b5a6640 1 --2- 172.21.15.16:0/3832460048 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbef806c2b0 0x7fbef806e770 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbf10034000 tx=0x7fbf1002e000 comp rx=0 tx=0).stop 2024-09-19T07:52:49.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.552+0000 7fbf1b5a6640 1 -- 172.21.15.16:0/3832460048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1415e680 msgr2=0x7fbf140ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:49.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.552+0000 7fbf1b5a6640 1 --2- 172.21.15.16:0/3832460048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1415e680 0x7fbf140ccb40 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c007e40 tx=0x7fbf0c00e4b0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.552+0000 7fbf1b5a6640 1 -- 172.21.15.16:0/3832460048 shutdown_connections 2024-09-19T07:52:49.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.552+0000 7fbf1b5a6640 1 --2- 172.21.15.16:0/3832460048 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbef806c2b0 0x7fbef806e770 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.552+0000 7fbf1b5a6640 1 --2- 172.21.15.16:0/3832460048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf1415e680 0x7fbf140ccb40 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.552+0000 7fbf1b5a6640 1 --2- 172.21.15.16:0/3832460048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf1415e2a0 0x7fbf140ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.552+0000 7fbf1b5a6640 1 --2- 172.21.15.16:0/3832460048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf1406d960 0x7fbf140ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.552+0000 7fbf1b5a6640 1 -- 172.21.15.16:0/3832460048 >> 172.21.15.16:0/3832460048 conn(0x7fbf1409d760 msgr2=0x7fbf140d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:49.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.552+0000 7fbf1b5a6640 1 -- 172.21.15.16:0/3832460048 shutdown_connections 2024-09-19T07:52:49.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.552+0000 7fbf1b5a6640 1 -- 172.21.15.16:0/3832460048 wait complete. 2024-09-19T07:52:49.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:52:49.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.664+0000 7f0b25ad7640 1 Processor -- start 2024-09-19T07:52:49.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.664+0000 7f0b25ad7640 1 -- start start 2024-09-19T07:52:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.664+0000 7f0b25ad7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0b201619b0 0x7f0b200d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.664+0000 7f0b25ad7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b200b0a80 0x7f0b200d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.664+0000 7f0b25ad7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0b20161d90 0x7f0b200cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.664+0000 7f0b25ad7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f0b200d0ec0 con 0x7f0b201619b0 2024-09-19T07:52:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.665+0000 7f0b25ad7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f0b2006c990 con 0x7f0b20161d90 2024-09-19T07:52:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.665+0000 7f0b25ad7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0b200cec10 con 0x7f0b200b0a80 2024-09-19T07:52:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.665+0000 7f0b1f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0b201619b0 0x7f0b200d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.665+0000 7f0b1f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0b201619b0 0x7f0b200d0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41494/0 (socket says 172.21.15.16:41494) 2024-09-19T07:52:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.665+0000 7f0b1f7fe640 1 -- 172.21.15.16:0/349160411 learned_addr learned my addr 172.21.15.16:0/349160411 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.665+0000 7f0b1f7fe640 1 -- 172.21.15.16:0/349160411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b200b0a80 msgr2=0x7f0b200d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.665+0000 7f0b1effd640 1 --2- 172.21.15.16:0/349160411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b200b0a80 0x7f0b200d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.665+0000 7f0b1ffff640 1 --2- 172.21.15.16:0/349160411 >> v2:172.21.15.146:3300/0 conn(0x7f0b20161d90 0x7f0b200cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.665+0000 7f0b1f7fe640 1 --2- 172.21.15.16:0/349160411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b200b0a80 0x7f0b200d0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.665+0000 7f0b1f7fe640 1 -- 172.21.15.16:0/349160411 >> v2:172.21.15.146:3300/0 conn(0x7f0b20161d90 msgr2=0x7f0b200cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.665+0000 7f0b1f7fe640 1 --2- 172.21.15.16:0/349160411 >> v2:172.21.15.146:3300/0 conn(0x7f0b20161d90 0x7f0b200cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.665+0000 7f0b1f7fe640 1 -- 172.21.15.16:0/349160411 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b200ce930 con 0x7f0b201619b0 2024-09-19T07:52:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.666+0000 7f0b1f7fe640 1 --2- 172.21.15.16:0/349160411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b201619b0 0x7f0b200d0440 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f0b200cd2c0 tx=0x7f0b1002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbb6e1ff788f6d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.666+0000 7f0b1e7fc640 1 -- 172.21.15.16:0/349160411 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10037070 con 0x7f0b201619b0 2024-09-19T07:52:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.666+0000 7f0b1e7fc640 1 -- 172.21.15.16:0/349160411 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b10032440 con 0x7f0b201619b0 2024-09-19T07:52:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.666+0000 7f0b1e7fc640 1 -- 172.21.15.16:0/349160411 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10032810 con 0x7f0b201619b0 2024-09-19T07:52:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.666+0000 7f0b25ad7640 1 -- 172.21.15.16:0/349160411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b201619b0 msgr2=0x7f0b200d0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.666+0000 7f0b25ad7640 1 --2- 172.21.15.16:0/349160411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b201619b0 0x7f0b200d0440 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f0b200cd2c0 tx=0x7f0b1002a1e0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.666+0000 7f0b25ad7640 1 -- 172.21.15.16:0/349160411 shutdown_connections 2024-09-19T07:52:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.666+0000 7f0b25ad7640 1 --2- 172.21.15.16:0/349160411 >> v2:172.21.15.146:3300/0 conn(0x7f0b20161d90 0x7f0b200cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.666+0000 7f0b25ad7640 1 --2- 172.21.15.16:0/349160411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b200b0a80 0x7f0b200d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.666+0000 7f0b25ad7640 1 --2- 172.21.15.16:0/349160411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b201619b0 0x7f0b200d0440 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.666+0000 7f0b25ad7640 1 -- 172.21.15.16:0/349160411 >> 172.21.15.16:0/349160411 conn(0x7f0b2009a9f0 msgr2=0x7f0b200b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.667+0000 7f0b25ad7640 1 -- 172.21.15.16:0/349160411 shutdown_connections 2024-09-19T07:52:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.667+0000 7f0b25ad7640 1 -- 172.21.15.16:0/349160411 wait complete. 2024-09-19T07:52:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.668+0000 7f0b25ad7640 1 Processor -- start 2024-09-19T07:52:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.668+0000 7f0b25ad7640 1 -- start start 2024-09-19T07:52:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b25ad7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b200b0a80 0x7f0b20158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b25ad7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b201619b0 0x7f0b200d1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b25ad7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b20161d90 0x7f0b200d1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b25ad7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2006c620 con 0x7f0b201619b0 2024-09-19T07:52:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b25ad7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0b200b0820 con 0x7f0b200b0a80 2024-09-19T07:52:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b25ad7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0b200cd330 con 0x7f0b20161d90 2024-09-19T07:52:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b1effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b201619b0 0x7f0b200d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b1effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b201619b0 0x7f0b200d1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41498/0 (socket says 172.21.15.16:41498) 2024-09-19T07:52:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b1effd640 1 -- 172.21.15.16:0/61280674 learned_addr learned my addr 172.21.15.16:0/61280674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b1ffff640 1 --2- 172.21.15.16:0/61280674 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b20161d90 0x7f0b200d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b1f7fe640 1 --2- 172.21.15.16:0/61280674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b200b0a80 0x7f0b20158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b1effd640 1 -- 172.21.15.16:0/61280674 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b20161d90 msgr2=0x7f0b200d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b1effd640 1 --2- 172.21.15.16:0/61280674 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b20161d90 0x7f0b200d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b1effd640 1 -- 172.21.15.16:0/61280674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b200b0a80 msgr2=0x7f0b20158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b1effd640 1 --2- 172.21.15.16:0/61280674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b200b0a80 0x7f0b20158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b1effd640 1 -- 172.21.15.16:0/61280674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b201f4ed0 con 0x7f0b201619b0 2024-09-19T07:52:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b1effd640 1 --2- 172.21.15.16:0/61280674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b201619b0 0x7f0b200d1040 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400c5a0 tx=0x7f0b1400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b07fff640 1 -- 172.21.15.16:0/61280674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14004230 con 0x7f0b201619b0 2024-09-19T07:52:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b07fff640 1 -- 172.21.15.16:0/61280674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b140043d0 con 0x7f0b201619b0 2024-09-19T07:52:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b25ad7640 1 -- 172.21.15.16:0/61280674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b200c85a0 con 0x7f0b201619b0 2024-09-19T07:52:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.669+0000 7f0b07fff640 1 -- 172.21.15.16:0/61280674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14003830 con 0x7f0b201619b0 2024-09-19T07:52:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.670+0000 7f0b25ad7640 1 -- 172.21.15.16:0/61280674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b200c8b60 con 0x7f0b201619b0 2024-09-19T07:52:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.670+0000 7f0b25ad7640 1 -- 172.21.15.16:0/61280674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b200d0440 con 0x7f0b201619b0 2024-09-19T07:52:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.671+0000 7f0b07fff640 1 -- 172.21.15.16:0/61280674 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0b140039d0 con 0x7f0b201619b0 2024-09-19T07:52:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.671+0000 7f0b07fff640 1 --2- 172.21.15.16:0/61280674 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0b0c06c2b0 0x7f0b0c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.672+0000 7f0b07fff640 1 -- 172.21.15.16:0/61280674 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f0b1408b4b0 con 0x7f0b201619b0 2024-09-19T07:52:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.672+0000 7f0b1f7fe640 1 --2- 172.21.15.16:0/61280674 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0b0c06c2b0 0x7f0b0c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.672+0000 7f0b1f7fe640 1 --2- 172.21.15.16:0/61280674 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0b0c06c2b0 0x7f0b0c06e770 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0b10034000 tx=0x7f0b1002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.676+0000 7f0b07fff640 1 -- 172.21.15.16:0/61280674 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f0b14015020 con 0x7f0b201619b0 2024-09-19T07:52:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.886+0000 7f0b25ad7640 1 -- 172.21.15.16:0/61280674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b200cc1f0 con 0x7f0b201619b0 2024-09-19T07:52:49.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.887+0000 7f0b07fff640 1 -- 172.21.15.16:0/61280674 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5231 (secure 0 0 0) 0x7f0b140593c0 con 0x7f0b201619b0 2024-09-19T07:52:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:52:49.887 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":49,"btime":"2024-09-19T07:50:36:200205+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5684,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1865971154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1865971154},{"type":"v1","addr":"172.21.15.146:6837","nonce":1865971154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6094,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/2735152611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2735152611},{"type":"v1","addr":"172.21.15.146:6836","nonce":2735152611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6399,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4204683210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4204683210},{"type":"v1","addr":"172.21.15.16:6834","nonce":4204683210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:50:15.050678+0000","modified":"2024-09-19T07:50:35.729727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6402":{"gid":6402,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/1305244492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1305244492},{"type":"v1","addr":"172.21.15.16:6835","nonce":1305244492}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6402,"qdb_cluster":[6402]},"id":6}]} 2024-09-19T07:52:49.887 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 49 2024-09-19T07:52:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.888+0000 7f0b25ad7640 1 -- 172.21.15.16:0/61280674 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0b0c06c2b0 msgr2=0x7f0b0c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.888+0000 7f0b25ad7640 1 --2- 172.21.15.16:0/61280674 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0b0c06c2b0 0x7f0b0c06e770 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0b10034000 tx=0x7f0b1002e000 comp rx=0 tx=0).stop 2024-09-19T07:52:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.888+0000 7f0b25ad7640 1 -- 172.21.15.16:0/61280674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b201619b0 msgr2=0x7f0b200d1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.888+0000 7f0b25ad7640 1 --2- 172.21.15.16:0/61280674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b201619b0 0x7f0b200d1040 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400c5a0 tx=0x7f0b1400ca60 comp rx=0 tx=0).stop 2024-09-19T07:52:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.889+0000 7f0b25ad7640 1 -- 172.21.15.16:0/61280674 shutdown_connections 2024-09-19T07:52:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.889+0000 7f0b25ad7640 1 --2- 172.21.15.16:0/61280674 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0b0c06c2b0 0x7f0b0c06e770 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.889+0000 7f0b25ad7640 1 --2- 172.21.15.16:0/61280674 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0b20161d90 0x7f0b200d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.889+0000 7f0b25ad7640 1 --2- 172.21.15.16:0/61280674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b201619b0 0x7f0b200d1040 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.889+0000 7f0b25ad7640 1 --2- 172.21.15.16:0/61280674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0b200b0a80 0x7f0b20158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:49.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.889+0000 7f0b25ad7640 1 -- 172.21.15.16:0/61280674 >> 172.21.15.16:0/61280674 conn(0x7f0b2009a9f0 msgr2=0x7f0b2006be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:49.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.889+0000 7f0b25ad7640 1 -- 172.21.15.16:0/61280674 shutdown_connections 2024-09-19T07:52:49.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:49.889+0000 7f0b25ad7640 1 -- 172.21.15.16:0/61280674 wait complete. 2024-09-19T07:52:49.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:52:50.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.001+0000 7f62303cd640 1 Processor -- start 2024-09-19T07:52:50.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.001+0000 7f62303cd640 1 -- start start 2024-09-19T07:52:50.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.002+0000 7f62303cd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f622815b120 0x7f622806f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.002+0000 7f62303cd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f622806dec0 0x7f622806e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.002+0000 7f62303cd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f622815b500 con 0x7f62280d04b0 2024-09-19T07:52:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.002+0000 7f62303cd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f622806e940 con 0x7f622815b120 2024-09-19T07:52:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.002+0000 7f62303cd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f622806ec20 con 0x7f622806dec0 2024-09-19T07:52:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.002+0000 7f622e169640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f622815b120 0x7f622806f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.002+0000 7f622e169640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f622815b120 0x7f622806f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41510/0 (socket says 172.21.15.16:41510) 2024-09-19T07:52:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.002+0000 7f622e169640 1 -- 172.21.15.16:0/1941745338 learned_addr learned my addr 172.21.15.16:0/1941745338 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.002+0000 7f622d968640 1 --2- 172.21.15.16:0/1941745338 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f622806dec0 0x7f622806e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.002+0000 7f622e169640 1 -- 172.21.15.16:0/1941745338 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f622806dec0 msgr2=0x7f622806e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.002+0000 7f622e169640 1 --2- 172.21.15.16:0/1941745338 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f622806dec0 0x7f622806e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.002+0000 7f622e169640 1 -- 172.21.15.16:0/1941745338 >> v1:172.21.15.146:6789/0 conn(0x7f62280d04b0 legacy=0x7f62280c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.002+0000 7f622e169640 1 -- 172.21.15.16:0/1941745338 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622806ef00 con 0x7f622815b120 2024-09-19T07:52:50.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.003+0000 7f622e169640 1 --2- 172.21.15.16:0/1941745338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f622815b120 0x7f622806f9e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6224009870 tx=0x7f622402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168876d25840691d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.003+0000 7f622d167640 1 -- 172.21.15.16:0/1941745338 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224037070 con 0x7f622815b120 2024-09-19T07:52:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.003+0000 7f622d167640 1 -- 172.21.15.16:0/1941745338 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6224032440 con 0x7f622815b120 2024-09-19T07:52:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.003+0000 7f622d167640 1 -- 172.21.15.16:0/1941745338 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224032810 con 0x7f622815b120 2024-09-19T07:52:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.003+0000 7f62303cd640 1 -- 172.21.15.16:0/1941745338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f622815b120 msgr2=0x7f622806f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.003+0000 7f62303cd640 1 --2- 172.21.15.16:0/1941745338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f622815b120 0x7f622806f9e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6224009870 tx=0x7f622402a250 comp rx=0 tx=0).stop 2024-09-19T07:52:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.004+0000 7f62303cd640 1 -- 172.21.15.16:0/1941745338 shutdown_connections 2024-09-19T07:52:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.004+0000 7f62303cd640 1 --2- 172.21.15.16:0/1941745338 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f622806dec0 0x7f622806e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.004+0000 7f62303cd640 1 --2- 172.21.15.16:0/1941745338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f622815b120 0x7f622806f9e0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.004+0000 7f62303cd640 1 -- 172.21.15.16:0/1941745338 >> 172.21.15.16:0/1941745338 conn(0x7f622806bdf0 msgr2=0x7f622806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.004+0000 7f62303cd640 1 -- 172.21.15.16:0/1941745338 shutdown_connections 2024-09-19T07:52:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.004+0000 7f62303cd640 1 -- 172.21.15.16:0/1941745338 wait complete. 2024-09-19T07:52:50.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.005+0000 7f62303cd640 1 Processor -- start 2024-09-19T07:52:50.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.005+0000 7f62303cd640 1 -- start start 2024-09-19T07:52:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f62303cd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f622806dec0 0x7f6228153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f62303cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62280d04b0 0x7f622814ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f62303cd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f622815b120 0x7f62281543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f62303cd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62280ce060 con 0x7f62280d04b0 2024-09-19T07:52:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f62303cd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f622806f4c0 con 0x7f622806dec0 2024-09-19T07:52:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f62303cd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f62280d0900 con 0x7f622815b120 2024-09-19T07:52:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f622d968640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62280d04b0 0x7f622814ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f622d968640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62280d04b0 0x7f622814ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41518/0 (socket says 172.21.15.16:41518) 2024-09-19T07:52:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f622d968640 1 -- 172.21.15.16:0/1016570945 learned_addr learned my addr 172.21.15.16:0/1016570945 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f622e96a640 1 --2- 172.21.15.16:0/1016570945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f622815b120 0x7f62281543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f622e169640 1 --2- 172.21.15.16:0/1016570945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f622806dec0 0x7f6228153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f622d968640 1 -- 172.21.15.16:0/1016570945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f622815b120 msgr2=0x7f62281543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f622d968640 1 --2- 172.21.15.16:0/1016570945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f622815b120 0x7f62281543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f622d968640 1 -- 172.21.15.16:0/1016570945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f622806dec0 msgr2=0x7f6228153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f622d968640 1 --2- 172.21.15.16:0/1016570945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f622806dec0 0x7f6228153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f622d968640 1 -- 172.21.15.16:0/1016570945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6228155230 con 0x7f62280d04b0 2024-09-19T07:52:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f622e169640 1 --2- 172.21.15.16:0/1016570945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f622806dec0 0x7f6228153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:52:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f622e96a640 1 --2- 172.21.15.16:0/1016570945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f622815b120 0x7f62281543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:52:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f622d968640 1 --2- 172.21.15.16:0/1016570945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62280d04b0 0x7f622814ec20 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f621800d800 tx=0x7f621800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f621effd640 1 -- 172.21.15.16:0/1016570945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621800b6c0 con 0x7f62280d04b0 2024-09-19T07:52:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f621effd640 1 -- 172.21.15.16:0/1016570945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6218019070 con 0x7f62280d04b0 2024-09-19T07:52:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f621effd640 1 -- 172.21.15.16:0/1016570945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621800ba80 con 0x7f62280d04b0 2024-09-19T07:52:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f62303cd640 1 -- 172.21.15.16:0/1016570945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6228154c70 con 0x7f62280d04b0 2024-09-19T07:52:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.006+0000 7f62303cd640 1 -- 172.21.15.16:0/1016570945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62280c5a80 con 0x7f62280d04b0 2024-09-19T07:52:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.007+0000 7f62303cd640 1 -- 172.21.15.16:0/1016570945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62280d0d60 con 0x7f62280d04b0 2024-09-19T07:52:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.008+0000 7f621effd640 1 -- 172.21.15.16:0/1016570945 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f621800bc20 con 0x7f62280d04b0 2024-09-19T07:52:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.008+0000 7f621effd640 1 --2- 172.21.15.16:0/1016570945 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f620c06c2b0 0x7f620c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.008+0000 7f621effd640 1 -- 172.21.15.16:0/1016570945 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f621808bb00 con 0x7f62280d04b0 2024-09-19T07:52:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.009+0000 7f622e169640 1 --2- 172.21.15.16:0/1016570945 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f620c06c2b0 0x7f620c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.009+0000 7f622e169640 1 --2- 172.21.15.16:0/1016570945 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f620c06c2b0 0x7f620c06e770 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6224034000 tx=0x7f622402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:50.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.013+0000 7f621effd640 1 -- 172.21.15.16:0/1016570945 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f621805a070 con 0x7f62280d04b0 2024-09-19T07:52:50.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.189+0000 7f62303cd640 1 -- 172.21.15.16:0/1016570945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6228157da0 con 0x7f62280d04b0 2024-09-19T07:52:50.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.190+0000 7f621effd640 1 -- 172.21.15.16:0/1016570945 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+17163 (secure 0 0 0) 0x7f6218059970 con 0x7f62280d04b0 2024-09-19T07:52:50.190 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:52:50.191 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":95,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:50:49.229309+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:50:13.015554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-19T07:50:14.031234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.21","mappings":[{"from":1,"to":2}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]},{"pgid":"14.2c","mappings":[{"from":1,"to":5}]},{"pgid":"14.3c","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:52:50.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.192+0000 7f62303cd640 1 -- 172.21.15.16:0/1016570945 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f620c06c2b0 msgr2=0x7f620c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:50.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.192+0000 7f62303cd640 1 --2- 172.21.15.16:0/1016570945 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f620c06c2b0 0x7f620c06e770 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6224034000 tx=0x7f622402e000 comp rx=0 tx=0).stop 2024-09-19T07:52:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.192+0000 7f62303cd640 1 -- 172.21.15.16:0/1016570945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62280d04b0 msgr2=0x7f622814ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.192+0000 7f62303cd640 1 --2- 172.21.15.16:0/1016570945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62280d04b0 0x7f622814ec20 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f621800d800 tx=0x7f621800dcc0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.192+0000 7f62303cd640 1 -- 172.21.15.16:0/1016570945 shutdown_connections 2024-09-19T07:52:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.192+0000 7f62303cd640 1 --2- 172.21.15.16:0/1016570945 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f620c06c2b0 0x7f620c06e770 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.192+0000 7f62303cd640 1 --2- 172.21.15.16:0/1016570945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f622815b120 0x7f62281543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.192+0000 7f62303cd640 1 --2- 172.21.15.16:0/1016570945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62280d04b0 0x7f622814ec20 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.192+0000 7f62303cd640 1 --2- 172.21.15.16:0/1016570945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f622806dec0 0x7f6228153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.192+0000 7f62303cd640 1 -- 172.21.15.16:0/1016570945 >> 172.21.15.16:0/1016570945 conn(0x7f622806bdf0 msgr2=0x7f6228075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.192+0000 7f62303cd640 1 -- 172.21.15.16:0/1016570945 shutdown_connections 2024-09-19T07:52:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.192+0000 7f62303cd640 1 -- 172.21.15.16:0/1016570945 wait complete. 2024-09-19T07:52:50.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T07:52:50.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.306+0000 7f58e2ad0640 1 Processor -- start 2024-09-19T07:52:50.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.306+0000 7f58e2ad0640 1 -- start start 2024-09-19T07:52:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.306+0000 7f58e2ad0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58dc15f0d0 0x7f58dc15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.306+0000 7f58e2ad0640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f58dc0d4130 0x7f58dc0d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.306+0000 7f58e2ad0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58dc0d1b10 con 0x7f58dc0d0070 2024-09-19T07:52:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.306+0000 7f58e2ad0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f58dc0cf2b0 con 0x7f58dc15f0d0 2024-09-19T07:52:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.306+0000 7f58e2ad0640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f58dc0cf590 con 0x7f58dc0d4130 2024-09-19T07:52:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.307+0000 7f58e086c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58dc15f0d0 0x7f58dc15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.307+0000 7f58dbfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f58dc0d0070 0x7f58dc0d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54440/0 (socket says 172.21.15.16:54440) 2024-09-19T07:52:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.307+0000 7f58dbfff640 1 -- 172.21.15.16:0/3419695314 learned_addr learned my addr 172.21.15.16:0/3419695314 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:52:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.307+0000 7f58e106d640 1 --2- 172.21.15.16:0/3419695314 >> v2:172.21.15.146:3300/0 conn(0x7f58dc0d4130 0x7f58dc0d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.307+0000 7f58e086c640 1 -- 172.21.15.16:0/3419695314 >> v2:172.21.15.146:3300/0 conn(0x7f58dc0d4130 msgr2=0x7f58dc0d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.307+0000 7f58e086c640 1 --2- 172.21.15.16:0/3419695314 >> v2:172.21.15.146:3300/0 conn(0x7f58dc0d4130 0x7f58dc0d4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.307+0000 7f58e086c640 1 -- 172.21.15.16:0/3419695314 >> v1:172.21.15.16:6789/0 conn(0x7f58dc0d0070 legacy=0x7f58dc0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.307+0000 7f58e086c640 1 -- 172.21.15.16:0/3419695314 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc0cf870 con 0x7f58dc15f0d0 2024-09-19T07:52:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.307+0000 7f58db7fe640 1 -- 172.21.15.16:0/3419695314 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2134829896 0 0) 0x7f58dc0d1b10 con 0x7f58dc0d0070 2024-09-19T07:52:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.308+0000 7f58e086c640 1 --2- 172.21.15.16:0/3419695314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58dc15f0d0 0x7f58dc15f4b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0d04e0 tx=0x7f58d002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cbbb26a5ab9031 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.308+0000 7f58db7fe640 1 -- 172.21.15.16:0/3419695314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0037070 con 0x7f58dc15f0d0 2024-09-19T07:52:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.308+0000 7f58db7fe640 1 -- 172.21.15.16:0/3419695314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d0032440 con 0x7f58dc15f0d0 2024-09-19T07:52:50.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.308+0000 7f58db7fe640 1 -- 172.21.15.16:0/3419695314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d002ae60 con 0x7f58dc15f0d0 2024-09-19T07:52:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.308+0000 7f58e2ad0640 1 -- 172.21.15.16:0/3419695314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58dc15f0d0 msgr2=0x7f58dc15f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.308+0000 7f58e2ad0640 1 --2- 172.21.15.16:0/3419695314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58dc15f0d0 0x7f58dc15f4b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0d04e0 tx=0x7f58d002a110 comp rx=0 tx=0).stop 2024-09-19T07:52:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.308+0000 7f58e2ad0640 1 -- 172.21.15.16:0/3419695314 shutdown_connections 2024-09-19T07:52:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.308+0000 7f58e2ad0640 1 --2- 172.21.15.16:0/3419695314 >> v2:172.21.15.146:3300/0 conn(0x7f58dc0d4130 0x7f58dc0d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.308+0000 7f58e2ad0640 1 --2- 172.21.15.16:0/3419695314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58dc15f0d0 0x7f58dc15f4b0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.308+0000 7f58e2ad0640 1 -- 172.21.15.16:0/3419695314 >> 172.21.15.16:0/3419695314 conn(0x7f58dc09d760 msgr2=0x7f58dc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.308+0000 7f58e2ad0640 1 -- 172.21.15.16:0/3419695314 shutdown_connections 2024-09-19T07:52:50.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.308+0000 7f58e2ad0640 1 -- 172.21.15.16:0/3419695314 wait complete. 2024-09-19T07:52:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.310+0000 7f58e2ad0640 1 Processor -- start 2024-09-19T07:52:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.310+0000 7f58e2ad0640 1 -- start start 2024-09-19T07:52:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.310+0000 7f58e2ad0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58dc0d0070 0x7f58dc0b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.310+0000 7f58e2ad0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58dc0d4130 0x7f58dc159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.310+0000 7f58e2ad0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58dc15f0d0 0x7f58dc0b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.310+0000 7f58e2ad0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc0cef60 con 0x7f58dc0d0070 2024-09-19T07:52:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.310+0000 7f58e2ad0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc0d4aa0 con 0x7f58dc15f0d0 2024-09-19T07:52:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.311+0000 7f58e2ad0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f58dc0b2f70 con 0x7f58dc0d4130 2024-09-19T07:52:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.311+0000 7f58e086c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58dc0d0070 0x7f58dc0b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.311+0000 7f58e086c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58dc0d0070 0x7f58dc0b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41538/0 (socket says 172.21.15.16:41538) 2024-09-19T07:52:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.311+0000 7f58e086c640 1 -- 172.21.15.16:0/2597887189 learned_addr learned my addr 172.21.15.16:0/2597887189 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.311+0000 7f58dbfff640 1 --2- 172.21.15.16:0/2597887189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58dc0d4130 0x7f58dc159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.311+0000 7f58e106d640 1 --2- 172.21.15.16:0/2597887189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58dc15f0d0 0x7f58dc0b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.311+0000 7f58e086c640 1 -- 172.21.15.16:0/2597887189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58dc0d4130 msgr2=0x7f58dc159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.311+0000 7f58e086c640 1 --2- 172.21.15.16:0/2597887189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58dc0d4130 0x7f58dc159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.311+0000 7f58e086c640 1 -- 172.21.15.16:0/2597887189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58dc15f0d0 msgr2=0x7f58dc0b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.311+0000 7f58e086c640 1 --2- 172.21.15.16:0/2597887189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58dc15f0d0 0x7f58dc0b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.311+0000 7f58e086c640 1 -- 172.21.15.16:0/2597887189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc0b4300 con 0x7f58dc0d0070 2024-09-19T07:52:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.311+0000 7f58e106d640 1 --2- 172.21.15.16:0/2597887189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58dc15f0d0 0x7f58dc0b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:52:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.311+0000 7f58e086c640 1 --2- 172.21.15.16:0/2597887189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58dc0d0070 0x7f58dc0b3d10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f58d0000c00 tx=0x7f58d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.311+0000 7f58d97fa640 1 -- 172.21.15.16:0/2597887189 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0037040 con 0x7f58dc0d0070 2024-09-19T07:52:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.311+0000 7f58d97fa640 1 -- 172.21.15.16:0/2597887189 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d0042070 con 0x7f58dc0d0070 2024-09-19T07:52:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.311+0000 7f58e2ad0640 1 -- 172.21.15.16:0/2597887189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58dc15fa70 con 0x7f58dc0d0070 2024-09-19T07:52:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.311+0000 7f58d97fa640 1 -- 172.21.15.16:0/2597887189 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0030a50 con 0x7f58dc0d0070 2024-09-19T07:52:50.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.311+0000 7f58e2ad0640 1 -- 172.21.15.16:0/2597887189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58dc159c90 con 0x7f58dc0d0070 2024-09-19T07:52:50.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.312+0000 7f58e2ad0640 1 -- 172.21.15.16:0/2597887189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58dc15fe60 con 0x7f58dc0d0070 2024-09-19T07:52:50.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.313+0000 7f58d97fa640 1 -- 172.21.15.16:0/2597887189 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f58d0030bf0 con 0x7f58dc0d0070 2024-09-19T07:52:50.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.313+0000 7f58d97fa640 1 --2- 172.21.15.16:0/2597887189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58c006c2b0 0x7f58c006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.313+0000 7f58d97fa640 1 -- 172.21.15.16:0/2597887189 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7f58d00ac1d0 con 0x7f58dc0d0070 2024-09-19T07:52:50.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.314+0000 7f58dbfff640 1 --2- 172.21.15.16:0/2597887189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58c006c2b0 0x7f58c006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.314+0000 7f58dbfff640 1 --2- 172.21.15.16:0/2597887189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f58c006c2b0 0x7f58c006e770 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f58dc158540 tx=0x7f58c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:50.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.317+0000 7f58d97fa640 1 -- 172.21.15.16:0/2597887189 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f58d0045360 con 0x7f58dc0d0070 2024-09-19T07:52:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.530+0000 7f58e2ad0640 1 -- 172.21.15.16:0/2597887189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f58dc0d4560 con 0x7f58dc0d0070 2024-09-19T07:52:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.530+0000 7f58d97fa640 1 -- 172.21.15.16:0/2597887189 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=-1 MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it v49) ==== 316+0+0 (secure 0 0 0) 0x7f58d007a6a0 con 0x7f58dc0d0070 2024-09-19T07:52:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:Error EPERM: MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it 2024-09-19T07:52:50.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:52:50.537 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs --yes-i-really-mean-it 2024-09-19T07:52:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.641+0000 7eff0a113640 1 Processor -- start 2024-09-19T07:52:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.641+0000 7eff0a113640 1 -- start start 2024-09-19T07:52:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.642+0000 7eff0a113640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7eff04141680 0x7eff040cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.642+0000 7eff0a113640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff040d0370 0x7eff04141a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.642+0000 7eff0a113640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff0415d710 con 0x7eff04141fa0 2024-09-19T07:52:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.642+0000 7eff0a113640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7eff040d07b0 con 0x7eff04141680 2024-09-19T07:52:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.642+0000 7eff0a113640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7eff0415c6f0 con 0x7eff040d0370 2024-09-19T07:52:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.642+0000 7eff037fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7eff04141680 0x7eff040cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.642+0000 7eff037fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7eff04141680 0x7eff040cfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:50144/0 (socket says 172.21.15.16:50144) 2024-09-19T07:52:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.642+0000 7eff037fe640 1 -- 172.21.15.16:0/1971481896 learned_addr learned my addr 172.21.15.16:0/1971481896 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.642+0000 7eff02ffd640 1 --2- 172.21.15.16:0/1971481896 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff040d0370 0x7eff04141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.643+0000 7eff037fe640 1 -- 172.21.15.16:0/1971481896 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff040d0370 msgr2=0x7eff04141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.643+0000 7eff037fe640 1 --2- 172.21.15.16:0/1971481896 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff040d0370 0x7eff04141a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.643+0000 7eff037fe640 1 -- 172.21.15.16:0/1971481896 >> v1:172.21.15.146:6789/0 conn(0x7eff04141fa0 legacy=0x7eff040cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.643+0000 7eff037fe640 1 -- 172.21.15.16:0/1971481896 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0415c950 con 0x7eff04141680 2024-09-19T07:52:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.643+0000 7eff02ffd640 1 --2- 172.21.15.16:0/1971481896 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff040d0370 0x7eff04141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:52:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.643+0000 7eff037fe640 1 --2- 172.21.15.16:0/1971481896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff04141680 0x7eff040cfe30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efef4009870 tx=0x7efef402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c100c229ad96b2f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.644+0000 7eff027fc640 1 -- 172.21.15.16:0/1971481896 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4037070 con 0x7eff04141680 2024-09-19T07:52:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.644+0000 7eff027fc640 1 -- 172.21.15.16:0/1971481896 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef4032440 con 0x7eff04141680 2024-09-19T07:52:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.644+0000 7eff027fc640 1 -- 172.21.15.16:0/1971481896 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4032810 con 0x7eff04141680 2024-09-19T07:52:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.644+0000 7eff0a113640 1 -- 172.21.15.16:0/1971481896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff04141680 msgr2=0x7eff040cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.644+0000 7eff0a113640 1 --2- 172.21.15.16:0/1971481896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff04141680 0x7eff040cfe30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efef4009870 tx=0x7efef402a270 comp rx=0 tx=0).stop 2024-09-19T07:52:50.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.644+0000 7eff0a113640 1 -- 172.21.15.16:0/1971481896 shutdown_connections 2024-09-19T07:52:50.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.644+0000 7eff0a113640 1 --2- 172.21.15.16:0/1971481896 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff040d0370 0x7eff04141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.644+0000 7eff0a113640 1 --2- 172.21.15.16:0/1971481896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff04141680 0x7eff040cfe30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.644+0000 7eff0a113640 1 -- 172.21.15.16:0/1971481896 >> 172.21.15.16:0/1971481896 conn(0x7eff040b3ed0 msgr2=0x7eff040b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:50.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.644+0000 7eff0a113640 1 -- 172.21.15.16:0/1971481896 shutdown_connections 2024-09-19T07:52:50.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.644+0000 7eff0a113640 1 -- 172.21.15.16:0/1971481896 wait complete. 2024-09-19T07:52:50.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.646+0000 7eff0a113640 1 Processor -- start 2024-09-19T07:52:50.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.646+0000 7eff0a113640 1 -- start start 2024-09-19T07:52:50.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.646+0000 7eff0a113640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff040d0370 0x7eff0406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.646+0000 7eff0a113640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff04141680 0x7eff04155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.646+0000 7eff0a113640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff04141fa0 0x7eff0413df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.647+0000 7eff0a113640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff0415d1f0 con 0x7eff04141680 2024-09-19T07:52:50.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.647+0000 7eff0a113640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7eff0415cf10 con 0x7eff040d0370 2024-09-19T07:52:50.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.647+0000 7eff0a113640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7eff041408c0 con 0x7eff04141fa0 2024-09-19T07:52:50.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.647+0000 7eff02ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff04141680 0x7eff04155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.647+0000 7eff02ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff04141680 0x7eff04155180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41540/0 (socket says 172.21.15.16:41540) 2024-09-19T07:52:50.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.647+0000 7eff02ffd640 1 -- 172.21.15.16:0/1921300276 learned_addr learned my addr 172.21.15.16:0/1921300276 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:50.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.647+0000 7eff037fe640 1 --2- 172.21.15.16:0/1921300276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff040d0370 0x7eff0406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.647+0000 7eff03fff640 1 --2- 172.21.15.16:0/1921300276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff04141fa0 0x7eff0413df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.647+0000 7eff02ffd640 1 -- 172.21.15.16:0/1921300276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff04141fa0 msgr2=0x7eff0413df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:50.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.647+0000 7eff02ffd640 1 --2- 172.21.15.16:0/1921300276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff04141fa0 0x7eff0413df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.647+0000 7eff02ffd640 1 -- 172.21.15.16:0/1921300276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff040d0370 msgr2=0x7eff0406da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:50.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.647+0000 7eff02ffd640 1 --2- 172.21.15.16:0/1921300276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff040d0370 0x7eff0406da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.647+0000 7eff02ffd640 1 -- 172.21.15.16:0/1921300276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0413eac0 con 0x7eff04141680 2024-09-19T07:52:50.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.647+0000 7eff02ffd640 1 --2- 172.21.15.16:0/1921300276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff04141680 0x7eff04155180 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7efef800ebe0 tx=0x7efef800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:50.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.647+0000 7efeebfff640 1 -- 172.21.15.16:0/1921300276 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef8017070 con 0x7eff04141680 2024-09-19T07:52:50.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.647+0000 7efeebfff640 1 -- 172.21.15.16:0/1921300276 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef800cc30 con 0x7eff04141680 2024-09-19T07:52:50.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.647+0000 7eff0a113640 1 -- 172.21.15.16:0/1921300276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff0406cf30 con 0x7eff04141680 2024-09-19T07:52:50.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.647+0000 7efeebfff640 1 -- 172.21.15.16:0/1921300276 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef8013470 con 0x7eff04141680 2024-09-19T07:52:50.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.647+0000 7eff0a113640 1 -- 172.21.15.16:0/1921300276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff0406d4f0 con 0x7eff04141680 2024-09-19T07:52:50.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.648+0000 7eff0a113640 1 -- 172.21.15.16:0/1921300276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff04142410 con 0x7eff04141680 2024-09-19T07:52:50.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.649+0000 7efeebfff640 1 -- 172.21.15.16:0/1921300276 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7efef801a070 con 0x7eff04141680 2024-09-19T07:52:50.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.649+0000 7efeebfff640 1 --2- 172.21.15.16:0/1921300276 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efef006c2e0 0x7efef006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.649+0000 7efeebfff640 1 -- 172.21.15.16:0/1921300276 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6217+0+0 (secure 0 0 0) 0x7efef808b7e0 con 0x7eff04141680 2024-09-19T07:52:50.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.650+0000 7eff037fe640 1 --2- 172.21.15.16:0/1921300276 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efef006c2e0 0x7efef006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.650+0000 7eff037fe640 1 --2- 172.21.15.16:0/1921300276 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efef006c2e0 0x7efef006e7a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efef4034000 tx=0x7efef402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.654+0000 7efeebfff640 1 -- 172.21.15.16:0/1921300276 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7efef8059790 con 0x7eff04141680 2024-09-19T07:52:50.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.871+0000 7eff0a113640 1 -- 172.21.15.16:0/1921300276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7eff040690c0 con 0x7eff04141680 2024-09-19T07:52:50.872 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-19T07:52:50.872+0000 7f6950fea640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:52:50.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.873+0000 7efeebfff640 1 -- 172.21.15.16:0/1921300276 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) ==== 191+0+0 (secure 0 0 0) 0x7efef805da90 con 0x7eff04141680 2024-09-19T07:52:50.874 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T07:52:50.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.876+0000 7eff0a113640 1 -- 172.21.15.16:0/1921300276 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efef006c2e0 msgr2=0x7efef006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.876+0000 7eff0a113640 1 --2- 172.21.15.16:0/1921300276 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efef006c2e0 0x7efef006e7a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efef4034000 tx=0x7efef402e000 comp rx=0 tx=0).stop 2024-09-19T07:52:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.876+0000 7eff0a113640 1 -- 172.21.15.16:0/1921300276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff04141680 msgr2=0x7eff04155180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.876+0000 7eff0a113640 1 --2- 172.21.15.16:0/1921300276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff04141680 0x7eff04155180 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7efef800ebe0 tx=0x7efef800c520 comp rx=0 tx=0).stop 2024-09-19T07:52:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.876+0000 7eff0a113640 1 -- 172.21.15.16:0/1921300276 shutdown_connections 2024-09-19T07:52:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.876+0000 7eff0a113640 1 --2- 172.21.15.16:0/1921300276 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efef006c2e0 0x7efef006e7a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.876+0000 7eff0a113640 1 --2- 172.21.15.16:0/1921300276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff04141fa0 0x7eff0413df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.876+0000 7eff0a113640 1 --2- 172.21.15.16:0/1921300276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff04141680 0x7eff04155180 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.876+0000 7eff0a113640 1 --2- 172.21.15.16:0/1921300276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff040d0370 0x7eff0406da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.876+0000 7eff0a113640 1 -- 172.21.15.16:0/1921300276 >> 172.21.15.16:0/1921300276 conn(0x7eff040b3ed0 msgr2=0x7eff040b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.876+0000 7eff0a113640 1 -- 172.21.15.16:0/1921300276 shutdown_connections 2024-09-19T07:52:50.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.876+0000 7eff0a113640 1 -- 172.21.15.16:0/1921300276 wait complete. 2024-09-19T07:52:50.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T07:52:50.915 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-19T07:52:50.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.993+0000 7fc9ec4b7640 1 Processor -- start 2024-09-19T07:52:50.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.993+0000 7fc9ec4b7640 1 -- start start 2024-09-19T07:52:50.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.994+0000 7fc9ec4b7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc9e415e2a0 0x7fc9e406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.994+0000 7fc9ec4b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e415e680 0x7fc9e40cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:50.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.994+0000 7fc9ec4b7640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e40d1880 con 0x7fc9e406d960 2024-09-19T07:52:50.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.994+0000 7fc9ec4b7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc9e40d1b60 con 0x7fc9e415e2a0 2024-09-19T07:52:50.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.994+0000 7fc9ec4b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e40cf2d0 con 0x7fc9e415e680 2024-09-19T07:52:50.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.994+0000 7fc9ea253640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc9e415e2a0 0x7fc9e406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.994+0000 7fc9e9a52640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e415e680 0x7fc9e40cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:50.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.994+0000 7fc9e9a52640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e415e680 0x7fc9e40cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:57414/0 (socket says 172.21.15.16:57414) 2024-09-19T07:52:50.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.994+0000 7fc9ea253640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc9e415e2a0 0x7fc9e406d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:50168/0 (socket says 172.21.15.16:50168) 2024-09-19T07:52:50.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.994+0000 7fc9e9a52640 1 -- 172.21.15.16:0/3169836987 learned_addr learned my addr 172.21.15.16:0/3169836987 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:50.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.994+0000 7fc9eaa54640 1 --1- 172.21.15.16:0/3169836987 >> v1:172.21.15.146:6789/0 conn(0x7fc9e406d960 0x7fc9e40d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:59578/0 (socket says 172.21.15.16:59578) 2024-09-19T07:52:50.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.995+0000 7fc9e9a52640 1 -- 172.21.15.16:0/3169836987 >> v2:172.21.15.146:3300/0 conn(0x7fc9e415e2a0 msgr2=0x7fc9e406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:50.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.995+0000 7fc9e9a52640 1 --2- 172.21.15.16:0/3169836987 >> v2:172.21.15.146:3300/0 conn(0x7fc9e415e2a0 0x7fc9e406d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:50.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.995+0000 7fc9e9a52640 1 -- 172.21.15.16:0/3169836987 >> v1:172.21.15.146:6789/0 conn(0x7fc9e406d960 legacy=0x7fc9e40d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:50.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.995+0000 7fc9e9a52640 1 -- 172.21.15.16:0/3169836987 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e40cf5b0 con 0x7fc9e415e680 2024-09-19T07:52:50.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.996+0000 7fc9e9a52640 1 --2- 172.21.15.16:0/3169836987 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e415e680 0x7fc9e40cfad0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009870 tx=0x7fc9d402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66b31cc55780d188 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.996+0000 7fc9e9251640 1 -- 172.21.15.16:0/3169836987 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4037070 con 0x7fc9e415e680 2024-09-19T07:52:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.996+0000 7fc9e9251640 1 -- 172.21.15.16:0/3169836987 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d4032440 con 0x7fc9e415e680 2024-09-19T07:52:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.996+0000 7fc9e9251640 1 -- 172.21.15.16:0/3169836987 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4032810 con 0x7fc9e415e680 2024-09-19T07:52:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.996+0000 7fc9ec4b7640 1 -- 172.21.15.16:0/3169836987 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e415e680 msgr2=0x7fc9e40cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.996+0000 7fc9ec4b7640 1 --2- 172.21.15.16:0/3169836987 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e415e680 0x7fc9e40cfad0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009870 tx=0x7fc9d402a270 comp rx=0 tx=0).stop 2024-09-19T07:52:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.996+0000 7fc9ec4b7640 1 -- 172.21.15.16:0/3169836987 shutdown_connections 2024-09-19T07:52:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.996+0000 7fc9ec4b7640 1 --2- 172.21.15.16:0/3169836987 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e415e680 0x7fc9e40cfad0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.996+0000 7fc9ec4b7640 1 --2- 172.21.15.16:0/3169836987 >> v2:172.21.15.146:3300/0 conn(0x7fc9e415e2a0 0x7fc9e406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.996+0000 7fc9ec4b7640 1 -- 172.21.15.16:0/3169836987 >> 172.21.15.16:0/3169836987 conn(0x7fc9e409d760 msgr2=0x7fc9e408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.996+0000 7fc9ec4b7640 1 -- 172.21.15.16:0/3169836987 shutdown_connections 2024-09-19T07:52:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.996+0000 7fc9ec4b7640 1 -- 172.21.15.16:0/3169836987 wait complete. 2024-09-19T07:52:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.998+0000 7fc9ec4b7640 1 Processor -- start 2024-09-19T07:52:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.998+0000 7fc9ec4b7640 1 -- start start 2024-09-19T07:52:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.998+0000 7fc9ec4b7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9e406d960 0x7fc9e40cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.998+0000 7fc9ec4b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e415e2a0 0x7fc9e40ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.998+0000 7fc9ec4b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e415e680 0x7fc9e4156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.998+0000 7fc9ec4b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e40d4c00 con 0x7fc9e415e2a0 2024-09-19T07:52:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.998+0000 7fc9ec4b7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e40d4a80 con 0x7fc9e406d960 2024-09-19T07:52:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.998+0000 7fc9ec4b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e40d0010 con 0x7fc9e415e680 2024-09-19T07:52:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.999+0000 7fc9e9a52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e415e2a0 0x7fc9e40ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.999+0000 7fc9e9a52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e415e2a0 0x7fc9e40ccd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41560/0 (socket says 172.21.15.16:41560) 2024-09-19T07:52:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.999+0000 7fc9e9a52640 1 -- 172.21.15.16:0/3887436762 learned_addr learned my addr 172.21.15.16:0/3887436762 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.999+0000 7fc9ea253640 1 --2- 172.21.15.16:0/3887436762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9e406d960 0x7fc9e40cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.999+0000 7fc9eaa54640 1 --2- 172.21.15.16:0/3887436762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e415e680 0x7fc9e4156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.999+0000 7fc9e9a52640 1 -- 172.21.15.16:0/3887436762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e415e680 msgr2=0x7fc9e4156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.999+0000 7fc9e9a52640 1 --2- 172.21.15.16:0/3887436762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e415e680 0x7fc9e4156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.999+0000 7fc9e9a52640 1 -- 172.21.15.16:0/3887436762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9e406d960 msgr2=0x7fc9e40cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.999+0000 7fc9e9a52640 1 --2- 172.21.15.16:0/3887436762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9e406d960 0x7fc9e40cad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.999+0000 7fc9e9a52640 1 -- 172.21.15.16:0/3887436762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e40ca3e0 con 0x7fc9e415e2a0 2024-09-19T07:52:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.999+0000 7fc9e9a52640 1 --2- 172.21.15.16:0/3887436762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e415e2a0 0x7fc9e40ccd10 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40090b0 tx=0x7fc9d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.999+0000 7fc9daffd640 1 -- 172.21.15.16:0/3887436762 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4041070 con 0x7fc9e415e2a0 2024-09-19T07:52:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.999+0000 7fc9daffd640 1 -- 172.21.15.16:0/3887436762 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d403c8b0 con 0x7fc9e415e2a0 2024-09-19T07:52:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.999+0000 7fc9ec4b7640 1 -- 172.21.15.16:0/3887436762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e40ca650 con 0x7fc9e415e2a0 2024-09-19T07:52:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.999+0000 7fc9daffd640 1 -- 172.21.15.16:0/3887436762 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4037040 con 0x7fc9e415e2a0 2024-09-19T07:52:51.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:50.999+0000 7fc9ec4b7640 1 -- 172.21.15.16:0/3887436762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e40b7e70 con 0x7fc9e415e2a0 2024-09-19T07:52:51.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.000+0000 7fc9ec4b7640 1 -- 172.21.15.16:0/3887436762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e406d420 con 0x7fc9e415e2a0 2024-09-19T07:52:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.001+0000 7fc9daffd640 1 -- 172.21.15.16:0/3887436762 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc9d4044070 con 0x7fc9e415e2a0 2024-09-19T07:52:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.001+0000 7fc9daffd640 1 --2- 172.21.15.16:0/3887436762 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc9c806c2b0 0x7fc9c806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.001+0000 7fc9daffd640 1 -- 172.21.15.16:0/3887436762 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6303+0+0 (secure 0 0 0) 0x7fc9d40ac370 con 0x7fc9e415e2a0 2024-09-19T07:52:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.001+0000 7fc9ea253640 1 --2- 172.21.15.16:0/3887436762 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc9c806c2b0 0x7fc9c806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.002+0000 7fc9ea253640 1 --2- 172.21.15.16:0/3887436762 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc9c806c2b0 0x7fc9c806e770 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009b40 tx=0x7fc9e0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:51.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.006+0000 7fc9daffd640 1 -- 172.21.15.16:0/3887436762 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc9d407a360 con 0x7fc9e415e2a0 2024-09-19T07:52:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.219+0000 7fc9ec4b7640 1 -- 172.21.15.16:0/3887436762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc9e406cf90 con 0x7fc9e415e2a0 2024-09-19T07:52:51.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.875+0000 7fc9daffd640 1 -- 172.21.15.16:0/3887436762 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) ==== 104+0+0 (secure 0 0 0) 0x7fc9d407e660 con 0x7fc9e415e2a0 2024-09-19T07:52:51.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.878+0000 7fc9ec4b7640 1 -- 172.21.15.16:0/3887436762 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc9c806c2b0 msgr2=0x7fc9c806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:51.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.878+0000 7fc9ec4b7640 1 --2- 172.21.15.16:0/3887436762 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc9c806c2b0 0x7fc9c806e770 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009b40 tx=0x7fc9e0009190 comp rx=0 tx=0).stop 2024-09-19T07:52:51.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.878+0000 7fc9ec4b7640 1 -- 172.21.15.16:0/3887436762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e415e2a0 msgr2=0x7fc9e40ccd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:51.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.878+0000 7fc9ec4b7640 1 --2- 172.21.15.16:0/3887436762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e415e2a0 0x7fc9e40ccd10 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40090b0 tx=0x7fc9d40046c0 comp rx=0 tx=0).stop 2024-09-19T07:52:51.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.878+0000 7fc9ec4b7640 1 -- 172.21.15.16:0/3887436762 shutdown_connections 2024-09-19T07:52:51.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.878+0000 7fc9ec4b7640 1 --2- 172.21.15.16:0/3887436762 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc9c806c2b0 0x7fc9c806e770 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:51.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.878+0000 7fc9ec4b7640 1 --2- 172.21.15.16:0/3887436762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e415e680 0x7fc9e4156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:51.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.878+0000 7fc9ec4b7640 1 --2- 172.21.15.16:0/3887436762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e415e2a0 0x7fc9e40ccd10 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:51.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.878+0000 7fc9ec4b7640 1 --2- 172.21.15.16:0/3887436762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9e406d960 0x7fc9e40cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:51.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.878+0000 7fc9ec4b7640 1 -- 172.21.15.16:0/3887436762 >> 172.21.15.16:0/3887436762 conn(0x7fc9e409d760 msgr2=0x7fc9e4161f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:51.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.878+0000 7fc9ec4b7640 1 -- 172.21.15.16:0/3887436762 shutdown_connections 2024-09-19T07:52:51.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:51.878+0000 7fc9ec4b7640 1 -- 172.21.15.16:0/3887436762 wait complete. 2024-09-19T07:52:51.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T07:52:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.047+0000 7f92a73d8640 1 Processor -- start 2024-09-19T07:52:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.047+0000 7f92a73d8640 1 -- start start 2024-09-19T07:52:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.048+0000 7f92a73d8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f92a015b120 0x7f92a006f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.048+0000 7f92a73d8640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f92a00d0bc0 0x7f92a006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.048+0000 7f92a73d8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92a015b500 con 0x7f92a006dec0 2024-09-19T07:52:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.048+0000 7f92a73d8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f92a006e940 con 0x7f92a015b120 2024-09-19T07:52:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.048+0000 7f92a73d8640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f92a006ec20 con 0x7f92a00d0bc0 2024-09-19T07:52:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.048+0000 7f92a5174640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f92a015b120 0x7f92a006f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.048+0000 7f92a5174640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f92a015b120 0x7f92a006f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41562/0 (socket says 172.21.15.16:41562) 2024-09-19T07:52:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.048+0000 7f92a4973640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f92a006dec0 0x7f92a00d04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54460/0 (socket says 172.21.15.16:54460) 2024-09-19T07:52:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.048+0000 7f92a5174640 1 -- 172.21.15.16:0/2538145365 learned_addr learned my addr 172.21.15.16:0/2538145365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.049+0000 7f92a5975640 1 --2- 172.21.15.16:0/2538145365 >> v2:172.21.15.146:3300/0 conn(0x7f92a00d0bc0 0x7f92a006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.049+0000 7f92a5174640 1 -- 172.21.15.16:0/2538145365 >> v2:172.21.15.146:3300/0 conn(0x7f92a00d0bc0 msgr2=0x7f92a006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.049+0000 7f92a5174640 1 --2- 172.21.15.16:0/2538145365 >> v2:172.21.15.146:3300/0 conn(0x7f92a00d0bc0 0x7f92a006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.049+0000 7f92a5174640 1 -- 172.21.15.16:0/2538145365 >> v1:172.21.15.16:6789/0 conn(0x7f92a006dec0 legacy=0x7f92a00d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.049+0000 7f92a5174640 1 -- 172.21.15.16:0/2538145365 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a006ef00 con 0x7f92a015b120 2024-09-19T07:52:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.049+0000 7f92a5174640 1 --2- 172.21.15.16:0/2538145365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92a015b120 0x7f92a006f9e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f92a015eb60 tx=0x7f929c02c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f858d5da3f20718a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.050+0000 7f9297fff640 1 -- 172.21.15.16:0/2538145365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c036020 con 0x7f92a015b120 2024-09-19T07:52:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.050+0000 7f9297fff640 1 -- 172.21.15.16:0/2538145365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f929c032640 con 0x7f92a015b120 2024-09-19T07:52:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.050+0000 7f9297fff640 1 -- 172.21.15.16:0/2538145365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c032a10 con 0x7f92a015b120 2024-09-19T07:52:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.050+0000 7f92a73d8640 1 -- 172.21.15.16:0/2538145365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92a015b120 msgr2=0x7f92a006f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.050+0000 7f92a73d8640 1 --2- 172.21.15.16:0/2538145365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92a015b120 0x7f92a006f9e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f92a015eb60 tx=0x7f929c02c940 comp rx=0 tx=0).stop 2024-09-19T07:52:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.050+0000 7f92a73d8640 1 -- 172.21.15.16:0/2538145365 shutdown_connections 2024-09-19T07:52:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.050+0000 7f92a73d8640 1 --2- 172.21.15.16:0/2538145365 >> v2:172.21.15.146:3300/0 conn(0x7f92a00d0bc0 0x7f92a006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.050+0000 7f92a73d8640 1 --2- 172.21.15.16:0/2538145365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92a015b120 0x7f92a006f9e0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.050+0000 7f92a73d8640 1 -- 172.21.15.16:0/2538145365 >> 172.21.15.16:0/2538145365 conn(0x7f92a006bdf0 msgr2=0x7f92a006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.050+0000 7f92a73d8640 1 -- 172.21.15.16:0/2538145365 shutdown_connections 2024-09-19T07:52:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.050+0000 7f92a73d8640 1 -- 172.21.15.16:0/2538145365 wait complete. 2024-09-19T07:52:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.052+0000 7f92a73d8640 1 Processor -- start 2024-09-19T07:52:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.052+0000 7f92a73d8640 1 -- start start 2024-09-19T07:52:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.052+0000 7f92a73d8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92a006dec0 0x7f92a01557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.052+0000 7f92a73d8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92a00d0bc0 0x7f92a00c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.052+0000 7f92a73d8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92a015b120 0x7f92a0157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.052+0000 7f92a73d8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f92a00ce060 con 0x7f92a015b120 2024-09-19T07:52:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.052+0000 7f92a73d8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f92a006f4c0 con 0x7f92a00d0bc0 2024-09-19T07:52:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.052+0000 7f92a73d8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f92a00c5160 con 0x7f92a006dec0 2024-09-19T07:52:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.052+0000 7f92a5975640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92a015b120 0x7f92a0157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.053+0000 7f92a5975640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92a015b120 0x7f92a0157570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41576/0 (socket says 172.21.15.16:41576) 2024-09-19T07:52:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.053+0000 7f92a5975640 1 -- 172.21.15.16:0/20224909 learned_addr learned my addr 172.21.15.16:0/20224909 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.053+0000 7f92a5174640 1 --2- 172.21.15.16:0/20224909 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92a006dec0 0x7f92a01557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.053+0000 7f92a4973640 1 --2- 172.21.15.16:0/20224909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92a00d0bc0 0x7f92a00c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.053+0000 7f92a5975640 1 -- 172.21.15.16:0/20224909 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92a006dec0 msgr2=0x7f92a01557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.053+0000 7f92a5975640 1 --2- 172.21.15.16:0/20224909 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92a006dec0 0x7f92a01557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.053+0000 7f92a5975640 1 -- 172.21.15.16:0/20224909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92a00d0bc0 msgr2=0x7f92a00c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.053+0000 7f92a5975640 1 --2- 172.21.15.16:0/20224909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92a00d0bc0 0x7f92a00c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.053+0000 7f92a5975640 1 -- 172.21.15.16:0/20224909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a0155cf0 con 0x7f92a015b120 2024-09-19T07:52:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.053+0000 7f92a4973640 1 --2- 172.21.15.16:0/20224909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92a00d0bc0 0x7f92a00c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:52:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.053+0000 7f92a5975640 1 --2- 172.21.15.16:0/20224909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92a015b120 0x7f92a0157570 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f9298007e30 tx=0x7f929800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.053+0000 7f9295ffb640 1 -- 172.21.15.16:0/20224909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298014620 con 0x7f92a015b120 2024-09-19T07:52:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.053+0000 7f9295ffb640 1 -- 172.21.15.16:0/20224909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9298019070 con 0x7f92a015b120 2024-09-19T07:52:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.053+0000 7f92a73d8640 1 -- 172.21.15.16:0/20224909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a00c73e0 con 0x7f92a015b120 2024-09-19T07:52:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.053+0000 7f9295ffb640 1 -- 172.21.15.16:0/20224909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298014a00 con 0x7f92a015b120 2024-09-19T07:52:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.053+0000 7f92a73d8640 1 -- 172.21.15.16:0/20224909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a00c7c80 con 0x7f92a015b120 2024-09-19T07:52:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.055+0000 7f92a73d8640 1 -- 172.21.15.16:0/20224909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a00c77d0 con 0x7f92a015b120 2024-09-19T07:52:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.055+0000 7f9295ffb640 1 -- 172.21.15.16:0/20224909 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9298014ba0 con 0x7f92a015b120 2024-09-19T07:52:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.056+0000 7f9295ffb640 1 --2- 172.21.15.16:0/20224909 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f928406c2b0 0x7f928406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:52.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.056+0000 7f9295ffb640 1 -- 172.21.15.16:0/20224909 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6303+0+0 (secure 0 0 0) 0x7f929808bed0 con 0x7f92a015b120 2024-09-19T07:52:52.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.056+0000 7f92a5174640 1 --2- 172.21.15.16:0/20224909 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f928406c2b0 0x7f928406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:52.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.056+0000 7f92a5174640 1 --2- 172.21.15.16:0/20224909 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f928406c2b0 0x7f928406e770 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f929c037000 tx=0x7f929c032000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.060+0000 7f9295ffb640 1 -- 172.21.15.16:0/20224909 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f929805a3f0 con 0x7f92a015b120 2024-09-19T07:52:52.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.237+0000 7f92a73d8640 1 -- 172.21.15.16:0/20224909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f92a006e330 con 0x7f92a015b120 2024-09-19T07:52:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.880+0000 7f9295ffb640 1 -- 172.21.15.16:0/20224909 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) ==== 181+0+0 (secure 0 0 0) 0x7f9298059cf0 con 0x7f92a015b120 2024-09-19T07:52:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-19T07:52:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.885+0000 7f92a73d8640 1 -- 172.21.15.16:0/20224909 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f928406c2b0 msgr2=0x7f928406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.885+0000 7f92a73d8640 1 --2- 172.21.15.16:0/20224909 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f928406c2b0 0x7f928406e770 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f929c037000 tx=0x7f929c032000 comp rx=0 tx=0).stop 2024-09-19T07:52:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.885+0000 7f92a73d8640 1 -- 172.21.15.16:0/20224909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92a015b120 msgr2=0x7f92a0157570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.885+0000 7f92a73d8640 1 --2- 172.21.15.16:0/20224909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92a015b120 0x7f92a0157570 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f9298007e30 tx=0x7f929800e4b0 comp rx=0 tx=0).stop 2024-09-19T07:52:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.886+0000 7f92a73d8640 1 -- 172.21.15.16:0/20224909 shutdown_connections 2024-09-19T07:52:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.886+0000 7f92a73d8640 1 --2- 172.21.15.16:0/20224909 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f928406c2b0 0x7f928406e770 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.886+0000 7f92a73d8640 1 --2- 172.21.15.16:0/20224909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92a015b120 0x7f92a0157570 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.886+0000 7f92a73d8640 1 --2- 172.21.15.16:0/20224909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92a00d0bc0 0x7f92a00c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.886+0000 7f92a73d8640 1 --2- 172.21.15.16:0/20224909 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92a006dec0 0x7f92a01557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.886+0000 7f92a73d8640 1 -- 172.21.15.16:0/20224909 >> 172.21.15.16:0/20224909 conn(0x7f92a006bdf0 msgr2=0x7f92a00caba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.887+0000 7f92a73d8640 1 -- 172.21.15.16:0/20224909 shutdown_connections 2024-09-19T07:52:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:52.887+0000 7f92a73d8640 1 -- 172.21.15.16:0/20224909 wait complete. 2024-09-19T07:52:52.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T07:52:53.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.016+0000 7fa73116d640 1 Processor -- start 2024-09-19T07:52:53.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.016+0000 7fa73116d640 1 -- start start 2024-09-19T07:52:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.017+0000 7fa73116d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa72c06d430 0x7fa72c1439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.017+0000 7fa73116d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa72c143f30 0x7fa72c0d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.017+0000 7fa73116d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa72c0c7140 con 0x7fa72c0c8d20 2024-09-19T07:52:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.017+0000 7fa73116d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa72c141960 con 0x7fa72c06d430 2024-09-19T07:52:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.017+0000 7fa73116d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa72c142200 con 0x7fa72c143f30 2024-09-19T07:52:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.017+0000 7fa72a59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa72c06d430 0x7fa72c1439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.017+0000 7fa72a59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa72c06d430 0x7fa72c1439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36718/0 (socket says 172.21.15.16:36718) 2024-09-19T07:52:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.017+0000 7fa72a59c640 1 -- 172.21.15.16:0/3533785269 learned_addr learned my addr 172.21.15.16:0/3533785269 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:53.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.017+0000 7fa72a59c640 1 -- 172.21.15.16:0/3533785269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa72c143f30 msgr2=0x7fa72c0d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:53.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.017+0000 7fa72b59e640 1 --2- 172.21.15.16:0/3533785269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa72c143f30 0x7fa72c0d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:53.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.017+0000 7fa72a59c640 1 --2- 172.21.15.16:0/3533785269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa72c143f30 0x7fa72c0d1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:53.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.017+0000 7fa72a59c640 1 -- 172.21.15.16:0/3533785269 >> v1:172.21.15.146:6789/0 conn(0x7fa72c0c8d20 legacy=0x7fa72c06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:53.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.017+0000 7fa72a59c640 1 -- 172.21.15.16:0/3533785269 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72c0c9100 con 0x7fa72c06d430 2024-09-19T07:52:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.018+0000 7fa72a59c640 1 --2- 172.21.15.16:0/3533785269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c06d430 0x7fa72c1439f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa714009870 tx=0x7fa71402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e6d8da5fb2e837 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.018+0000 7fa729d9b640 1 -- 172.21.15.16:0/3533785269 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714037070 con 0x7fa72c06d430 2024-09-19T07:52:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.018+0000 7fa729d9b640 1 -- 172.21.15.16:0/3533785269 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa71402ae10 con 0x7fa72c06d430 2024-09-19T07:52:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.018+0000 7fa729d9b640 1 -- 172.21.15.16:0/3533785269 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714032660 con 0x7fa72c06d430 2024-09-19T07:52:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.019+0000 7fa73116d640 1 -- 172.21.15.16:0/3533785269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c06d430 msgr2=0x7fa72c1439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.019+0000 7fa73116d640 1 --2- 172.21.15.16:0/3533785269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c06d430 0x7fa72c1439f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa714009870 tx=0x7fa71402a100 comp rx=0 tx=0).stop 2024-09-19T07:52:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.019+0000 7fa73116d640 1 -- 172.21.15.16:0/3533785269 shutdown_connections 2024-09-19T07:52:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.019+0000 7fa73116d640 1 --2- 172.21.15.16:0/3533785269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa72c143f30 0x7fa72c0d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.019+0000 7fa73116d640 1 --2- 172.21.15.16:0/3533785269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c06d430 0x7fa72c1439f0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.019+0000 7fa73116d640 1 -- 172.21.15.16:0/3533785269 >> 172.21.15.16:0/3533785269 conn(0x7fa72c0b3ed0 msgr2=0x7fa72c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.019+0000 7fa73116d640 1 -- 172.21.15.16:0/3533785269 shutdown_connections 2024-09-19T07:52:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.019+0000 7fa73116d640 1 -- 172.21.15.16:0/3533785269 wait complete. 2024-09-19T07:52:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.020+0000 7fa73116d640 1 Processor -- start 2024-09-19T07:52:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.020+0000 7fa73116d640 1 -- start start 2024-09-19T07:52:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.021+0000 7fa73116d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa72c06d430 0x7fa72c1405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.021+0000 7fa73116d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa72c0c8d20 0x7fa72c0b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.021+0000 7fa73116d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c143f30 0x7fa72c0b7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.021+0000 7fa73116d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa72c141e90 con 0x7fa72c143f30 2024-09-19T07:52:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.021+0000 7fa73116d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa72c141bb0 con 0x7fa72c06d430 2024-09-19T07:52:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.021+0000 7fa73116d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa72c144350 con 0x7fa72c0c8d20 2024-09-19T07:52:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.021+0000 7fa72b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c143f30 0x7fa72c0b7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.021+0000 7fa72b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c143f30 0x7fa72c0b7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36728/0 (socket says 172.21.15.16:36728) 2024-09-19T07:52:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.021+0000 7fa72b59e640 1 -- 172.21.15.16:0/3663531065 learned_addr learned my addr 172.21.15.16:0/3663531065 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.021+0000 7fa72a59c640 1 --2- 172.21.15.16:0/3663531065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa72c0c8d20 0x7fa72c0b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.021+0000 7fa72ad9d640 1 --2- 172.21.15.16:0/3663531065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa72c06d430 0x7fa72c1405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.022+0000 7fa72b59e640 1 -- 172.21.15.16:0/3663531065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa72c0c8d20 msgr2=0x7fa72c0b71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.022+0000 7fa72b59e640 1 --2- 172.21.15.16:0/3663531065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa72c0c8d20 0x7fa72c0b71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.022+0000 7fa72b59e640 1 -- 172.21.15.16:0/3663531065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa72c06d430 msgr2=0x7fa72c1405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.022+0000 7fa72b59e640 1 --2- 172.21.15.16:0/3663531065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa72c06d430 0x7fa72c1405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.022+0000 7fa72b59e640 1 -- 172.21.15.16:0/3663531065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72c0cf5e0 con 0x7fa72c143f30 2024-09-19T07:52:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.022+0000 7fa72ad9d640 1 --2- 172.21.15.16:0/3663531065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa72c06d430 0x7fa72c1405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:52:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.022+0000 7fa72b59e640 1 --2- 172.21.15.16:0/3663531065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c143f30 0x7fa72c0b7720 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa72400c580 tx=0x7fa72400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.022+0000 7fa7237fe640 1 -- 172.21.15.16:0/3663531065 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa724015070 con 0x7fa72c143f30 2024-09-19T07:52:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.022+0000 7fa7237fe640 1 -- 172.21.15.16:0/3663531065 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa724004220 con 0x7fa72c143f30 2024-09-19T07:52:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.022+0000 7fa7237fe640 1 -- 172.21.15.16:0/3663531065 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa724003610 con 0x7fa72c143f30 2024-09-19T07:52:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.022+0000 7fa73116d640 1 -- 172.21.15.16:0/3663531065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72c0d05c0 con 0x7fa72c143f30 2024-09-19T07:52:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.022+0000 7fa73116d640 1 -- 172.21.15.16:0/3663531065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa72c140b30 con 0x7fa72c143f30 2024-09-19T07:52:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.023+0000 7fa73116d640 1 -- 172.21.15.16:0/3663531065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa72c0690c0 con 0x7fa72c143f30 2024-09-19T07:52:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.024+0000 7fa7237fe640 1 -- 172.21.15.16:0/3663531065 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa7240037b0 con 0x7fa72c143f30 2024-09-19T07:52:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.024+0000 7fa7237fe640 1 --2- 172.21.15.16:0/3663531065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa70c06c2b0 0x7fa70c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.024+0000 7fa7237fe640 1 -- 172.21.15.16:0/3663531065 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5865+0+0 (secure 0 0 0) 0x7fa72408b1d0 con 0x7fa72c143f30 2024-09-19T07:52:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.024+0000 7fa72ad9d640 1 --2- 172.21.15.16:0/3663531065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa70c06c2b0 0x7fa70c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.025+0000 7fa72ad9d640 1 --2- 172.21.15.16:0/3663531065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa70c06c2b0 0x7fa70c06e770 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0b7ec0 tx=0x7fa71c009590 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.029+0000 7fa7237fe640 1 -- 172.21.15.16:0/3663531065 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa724059260 con 0x7fa72c143f30 2024-09-19T07:52:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.206+0000 7fa73116d640 1 -- 172.21.15.16:0/3663531065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa72c06ec80 con 0x7fa72c143f30 2024-09-19T07:52:53.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.886+0000 7fa7237fe640 1 -- 172.21.15.16:0/3663531065 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) ==== 169+0+0 (secure 0 0 0) 0x7fa72405d560 con 0x7fa72c143f30 2024-09-19T07:52:53.886 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-19T07:52:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.889+0000 7fa73116d640 1 -- 172.21.15.16:0/3663531065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa70c06c2b0 msgr2=0x7fa70c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.889+0000 7fa73116d640 1 --2- 172.21.15.16:0/3663531065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa70c06c2b0 0x7fa70c06e770 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0b7ec0 tx=0x7fa71c009590 comp rx=0 tx=0).stop 2024-09-19T07:52:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.889+0000 7fa73116d640 1 -- 172.21.15.16:0/3663531065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c143f30 msgr2=0x7fa72c0b7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.889+0000 7fa73116d640 1 --2- 172.21.15.16:0/3663531065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c143f30 0x7fa72c0b7720 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa72400c580 tx=0x7fa72400ca40 comp rx=0 tx=0).stop 2024-09-19T07:52:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.890+0000 7fa73116d640 1 -- 172.21.15.16:0/3663531065 shutdown_connections 2024-09-19T07:52:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.890+0000 7fa73116d640 1 --2- 172.21.15.16:0/3663531065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa70c06c2b0 0x7fa70c06e770 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.890+0000 7fa73116d640 1 --2- 172.21.15.16:0/3663531065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c143f30 0x7fa72c0b7720 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.890+0000 7fa73116d640 1 --2- 172.21.15.16:0/3663531065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa72c0c8d20 0x7fa72c0b71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.890+0000 7fa73116d640 1 --2- 172.21.15.16:0/3663531065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa72c06d430 0x7fa72c1405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.890+0000 7fa73116d640 1 -- 172.21.15.16:0/3663531065 >> 172.21.15.16:0/3663531065 conn(0x7fa72c0b3ed0 msgr2=0x7fa72c15fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.891+0000 7fa73116d640 1 -- 172.21.15.16:0/3663531065 shutdown_connections 2024-09-19T07:52:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:53.891+0000 7fa73116d640 1 -- 172.21.15.16:0/3663531065 wait complete. 2024-09-19T07:52:53.907 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2024-09-19T07:52:54.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.017+0000 7ff62c949640 1 Processor -- start 2024-09-19T07:52:54.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.017+0000 7ff62c949640 1 -- start start 2024-09-19T07:52:54.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.018+0000 7ff62c949640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff6280d4370 0x7ff6280d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.018+0000 7ff62c949640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6280d1880 con 0x7ff62815e680 2024-09-19T07:52:54.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.018+0000 7ff62c949640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6280d1b60 con 0x7ff62815e2a0 2024-09-19T07:52:54.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.018+0000 7ff62c949640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff6280cf2d0 con 0x7ff6280d4370 2024-09-19T07:52:54.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.018+0000 7ff61dd9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff62815e680 0x7ff6280cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51438/0 (socket says 172.21.15.16:51438) 2024-09-19T07:52:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.018+0000 7ff61dd9b640 1 -- 172.21.15.16:0/205385217 learned_addr learned my addr 172.21.15.16:0/205385217 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:52:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.018+0000 7ff626d9d640 1 --2- 172.21.15.16:0/205385217 >> v2:172.21.15.146:3300/0 conn(0x7ff6280d4370 0x7ff6280d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.018+0000 7ff625d9b640 1 -- 172.21.15.16:0/205385217 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1190106011 0 0) 0x7ff6280d1880 con 0x7ff62815e680 2024-09-19T07:52:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.018+0000 7ff625d9b640 1 -- 172.21.15.16:0/205385217 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff614003520 con 0x7ff62815e680 2024-09-19T07:52:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.018+0000 7ff625d9b640 1 -- 172.21.15.16:0/205385217 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2049837104 0 0) 0x7ff614003520 con 0x7ff62815e680 2024-09-19T07:52:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.018+0000 7ff625d9b640 1 -- 172.21.15.16:0/205385217 >> v2:172.21.15.146:3300/0 conn(0x7ff6280d4370 msgr2=0x7ff6280d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.018+0000 7ff625d9b640 1 --2- 172.21.15.16:0/205385217 >> v2:172.21.15.146:3300/0 conn(0x7ff6280d4370 0x7ff6280d4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff625d9b640 1 -- 172.21.15.16:0/205385217 >> v1:172.21.15.146:6789/0 conn(0x7ff62815e2a0 legacy=0x7ff62806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff625d9b640 1 -- 172.21.15.16:0/205385217 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6280cf5b0 con 0x7ff62815e680 2024-09-19T07:52:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff625d9b640 1 -- 172.21.15.16:0/205385217 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7ff610001670 con 0x7ff62815e680 2024-09-19T07:52:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff625d9b640 1 -- 172.21.15.16:0/205385217 >> v1:172.21.15.16:6789/0 conn(0x7ff62815e680 legacy=0x7ff6280cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff625d9b640 1 --2- 172.21.15.16:0/205385217 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6140039b0 0x7ff614003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff625d9b640 1 --2- 172.21.15.16:0/205385217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff614004800 0x7ff614004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff625d9b640 1 --2- 172.21.15.16:0/205385217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6140051c0 0x7ff61400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff625d9b640 1 -- 172.21.15.16:0/205385217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6280cf2d0 con 0x7ff614004800 2024-09-19T07:52:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff625d9b640 1 -- 172.21.15.16:0/205385217 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff61400c060 con 0x7ff6140051c0 2024-09-19T07:52:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff625d9b640 1 -- 172.21.15.16:0/205385217 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff61400c260 con 0x7ff6140039b0 2024-09-19T07:52:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff61dd9b640 1 --2- 172.21.15.16:0/205385217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff614004800 0x7ff614004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff62659c640 1 --2- 172.21.15.16:0/205385217 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6140039b0 0x7ff614003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff626d9d640 1 --2- 172.21.15.16:0/205385217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6140051c0 0x7ff61400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff61dd9b640 1 -- 172.21.15.16:0/205385217 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6140039b0 msgr2=0x7ff614003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff61dd9b640 1 --2- 172.21.15.16:0/205385217 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6140039b0 0x7ff614003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff61dd9b640 1 -- 172.21.15.16:0/205385217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6140051c0 msgr2=0x7ff61400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff61dd9b640 1 --2- 172.21.15.16:0/205385217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6140051c0 0x7ff61400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff61dd9b640 1 -- 172.21.15.16:0/205385217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff614003520 con 0x7ff614004800 2024-09-19T07:52:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff626d9d640 1 --2- 172.21.15.16:0/205385217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6140051c0 0x7ff61400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:52:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.019+0000 7ff626d9d640 1 -- 172.21.15.16:0/205385217 reap_dead start 2024-09-19T07:52:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.020+0000 7ff61dd9b640 1 --2- 172.21.15.16:0/205385217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff614004800 0x7ff614004c80 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7ff62815ea80 tx=0x7ff614005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8b002b954608b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.020+0000 7ff625d9b640 1 -- 172.21.15.16:0/205385217 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610007d30 con 0x7ff614004800 2024-09-19T07:52:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.020+0000 7ff625d9b640 1 -- 172.21.15.16:0/205385217 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff610005ad0 con 0x7ff614004800 2024-09-19T07:52:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.020+0000 7ff625d9b640 1 -- 172.21.15.16:0/205385217 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610007070 con 0x7ff614004800 2024-09-19T07:52:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.020+0000 7ff62c949640 1 -- 172.21.15.16:0/205385217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff614004800 msgr2=0x7ff614004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.020+0000 7ff62c949640 1 --2- 172.21.15.16:0/205385217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff614004800 0x7ff614004c80 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7ff62815ea80 tx=0x7ff614005830 comp rx=0 tx=0).stop 2024-09-19T07:52:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.021+0000 7ff62c949640 1 -- 172.21.15.16:0/205385217 shutdown_connections 2024-09-19T07:52:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.021+0000 7ff62c949640 1 --2- 172.21.15.16:0/205385217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff614004800 0x7ff614004c80 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.021+0000 7ff62c949640 1 -- 172.21.15.16:0/205385217 >> 172.21.15.16:0/205385217 conn(0x7ff62809d760 msgr2=0x7ff62808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:54.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.021+0000 7ff62c949640 1 -- 172.21.15.16:0/205385217 shutdown_connections 2024-09-19T07:52:54.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.021+0000 7ff62c949640 1 -- 172.21.15.16:0/205385217 wait complete. 2024-09-19T07:52:54.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.022+0000 7ff62c949640 1 Processor -- start 2024-09-19T07:52:54.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.022+0000 7ff62c949640 1 -- start start 2024-09-19T07:52:54.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.023+0000 7ff62c949640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff614004800 0x7ff6281ea840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.023+0000 7ff62c949640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6281e9a30 0x7ff6281568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.023+0000 7ff62c949640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6281596e0 0x7ff628159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.023+0000 7ff62c949640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6280d49a0 con 0x7ff6281596e0 2024-09-19T07:52:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.023+0000 7ff62c949640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff6280d4820 con 0x7ff614004800 2024-09-19T07:52:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.023+0000 7ff62c949640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff6280d4b20 con 0x7ff6281e9a30 2024-09-19T07:52:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.023+0000 7ff626d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6281596e0 0x7ff628159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.023+0000 7ff626d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6281596e0 0x7ff628159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36740/0 (socket says 172.21.15.16:36740) 2024-09-19T07:52:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.023+0000 7ff626d9d640 1 -- 172.21.15.16:0/2358611230 learned_addr learned my addr 172.21.15.16:0/2358611230 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.023+0000 7ff61dd9b640 1 --2- 172.21.15.16:0/2358611230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6281e9a30 0x7ff6281568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.023+0000 7ff62659c640 1 --2- 172.21.15.16:0/2358611230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff614004800 0x7ff6281ea840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.023+0000 7ff626d9d640 1 -- 172.21.15.16:0/2358611230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6281e9a30 msgr2=0x7ff6281568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.023+0000 7ff626d9d640 1 --2- 172.21.15.16:0/2358611230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6281e9a30 0x7ff6281568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.023+0000 7ff626d9d640 1 -- 172.21.15.16:0/2358611230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff614004800 msgr2=0x7ff6281ea840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.023+0000 7ff626d9d640 1 --2- 172.21.15.16:0/2358611230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff614004800 0x7ff6281ea840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.023+0000 7ff626d9d640 1 -- 172.21.15.16:0/2358611230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6281e8150 con 0x7ff6281596e0 2024-09-19T07:52:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.023+0000 7ff61dd9b640 1 --2- 172.21.15.16:0/2358611230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6281e9a30 0x7ff6281568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:52:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.023+0000 7ff62659c640 1 --2- 172.21.15.16:0/2358611230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff614004800 0x7ff6281ea840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:52:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.024+0000 7ff626d9d640 1 --2- 172.21.15.16:0/2358611230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6281596e0 0x7ff628159b70 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff62806eda0 tx=0x7ff620010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.024+0000 7ff61f7fe640 1 -- 172.21.15.16:0/2358611230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62001a030 con 0x7ff6281596e0 2024-09-19T07:52:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.024+0000 7ff61f7fe640 1 -- 172.21.15.16:0/2358611230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff620010da0 con 0x7ff6281596e0 2024-09-19T07:52:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.024+0000 7ff62c949640 1 -- 172.21.15.16:0/2358611230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6281e8430 con 0x7ff6281596e0 2024-09-19T07:52:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.024+0000 7ff61f7fe640 1 -- 172.21.15.16:0/2358611230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620003bd0 con 0x7ff6281596e0 2024-09-19T07:52:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.024+0000 7ff62c949640 1 -- 172.21.15.16:0/2358611230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6281e89f0 con 0x7ff6281596e0 2024-09-19T07:52:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.025+0000 7ff62c949640 1 -- 172.21.15.16:0/2358611230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff62806d420 con 0x7ff6281596e0 2024-09-19T07:52:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.025+0000 7ff61f7fe640 1 -- 172.21.15.16:0/2358611230 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff62001d070 con 0x7ff6281596e0 2024-09-19T07:52:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.026+0000 7ff61f7fe640 1 --2- 172.21.15.16:0/2358611230 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff60806c2e0 0x7ff60806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.026+0000 7ff61f7fe640 1 -- 172.21.15.16:0/2358611230 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5335+0+0 (secure 0 0 0) 0x7ff62008b8a0 con 0x7ff6281596e0 2024-09-19T07:52:54.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.026+0000 7ff62659c640 1 --2- 172.21.15.16:0/2358611230 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff60806c2e0 0x7ff60806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.027+0000 7ff62659c640 1 --2- 172.21.15.16:0/2358611230 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff60806c2e0 0x7ff60806e7a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff6180036b0 tx=0x7ff618009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:54.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.030+0000 7ff61f7fe640 1 -- 172.21.15.16:0/2358611230 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff62005a210 con 0x7ff6281596e0 2024-09-19T07:52:54.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.205+0000 7ff62c949640 1 -- 172.21.15.16:0/2358611230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) -- 0x7ff6280ce8a0 con 0x7ff6281596e0 2024-09-19T07:52:54.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.208+0000 7ff61f7fe640 1 -- 172.21.15.16:0/2358611230 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v36) ==== 111+0+0 (secure 0 0 0) 0x7ff620059b10 con 0x7ff6281596e0 2024-09-19T07:52:54.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.210+0000 7ff62c949640 1 -- 172.21.15.16:0/2358611230 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff60806c2e0 msgr2=0x7ff60806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.210+0000 7ff62c949640 1 --2- 172.21.15.16:0/2358611230 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff60806c2e0 0x7ff60806e7a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff6180036b0 tx=0x7ff618009040 comp rx=0 tx=0).stop 2024-09-19T07:52:54.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.210+0000 7ff62c949640 1 -- 172.21.15.16:0/2358611230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6281596e0 msgr2=0x7ff628159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.210+0000 7ff62c949640 1 --2- 172.21.15.16:0/2358611230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6281596e0 0x7ff628159b70 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff62806eda0 tx=0x7ff620010590 comp rx=0 tx=0).stop 2024-09-19T07:52:54.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.210+0000 7ff62c949640 1 -- 172.21.15.16:0/2358611230 shutdown_connections 2024-09-19T07:52:54.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.210+0000 7ff62c949640 1 --2- 172.21.15.16:0/2358611230 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff60806c2e0 0x7ff60806e7a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.210+0000 7ff62c949640 1 --2- 172.21.15.16:0/2358611230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6281596e0 0x7ff628159b70 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.210+0000 7ff62c949640 1 --2- 172.21.15.16:0/2358611230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6281e9a30 0x7ff6281568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.210+0000 7ff62c949640 1 --2- 172.21.15.16:0/2358611230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff614004800 0x7ff6281ea840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.210+0000 7ff62c949640 1 -- 172.21.15.16:0/2358611230 >> 172.21.15.16:0/2358611230 conn(0x7ff62809d760 msgr2=0x7ff62815cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:54.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.210+0000 7ff62c949640 1 -- 172.21.15.16:0/2358611230 shutdown_connections 2024-09-19T07:52:54.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.210+0000 7ff62c949640 1 -- 172.21.15.16:0/2358611230 wait complete. 2024-09-19T07:52:54.221 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2024-09-19T07:52:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.332+0000 7ff228b8d640 1 Processor -- start 2024-09-19T07:52:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.332+0000 7ff228b8d640 1 -- start start 2024-09-19T07:52:54.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.333+0000 7ff228b8d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff2240d4370 0x7ff2240d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.333+0000 7ff228b8d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2240d1880 con 0x7ff22415e680 2024-09-19T07:52:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.333+0000 7ff228b8d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2240d1b60 con 0x7ff22415e2a0 2024-09-19T07:52:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.333+0000 7ff228b8d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff2240cf2d0 con 0x7ff2240d4370 2024-09-19T07:52:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.333+0000 7ff221d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff22415e680 0x7ff2240cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51444/0 (socket says 172.21.15.16:51444) 2024-09-19T07:52:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.333+0000 7ff221d9b640 1 -- 172.21.15.16:0/3305801028 learned_addr learned my addr 172.21.15.16:0/3305801028 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:52:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.333+0000 7ff22159a640 1 -- 172.21.15.16:0/3305801028 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 739008327 0 0) 0x7ff2240d1880 con 0x7ff22415e680 2024-09-19T07:52:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.333+0000 7ff222d9d640 1 --2- 172.21.15.16:0/3305801028 >> v2:172.21.15.146:3300/0 conn(0x7ff2240d4370 0x7ff2240d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.333+0000 7ff22159a640 1 -- 172.21.15.16:0/3305801028 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff20c003520 con 0x7ff22415e680 2024-09-19T07:52:54.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff22159a640 1 -- 172.21.15.16:0/3305801028 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4127738203 0 0) 0x7ff20c003520 con 0x7ff22415e680 2024-09-19T07:52:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff22159a640 1 -- 172.21.15.16:0/3305801028 >> v2:172.21.15.146:3300/0 conn(0x7ff2240d4370 msgr2=0x7ff2240d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff22159a640 1 --2- 172.21.15.16:0/3305801028 >> v2:172.21.15.146:3300/0 conn(0x7ff2240d4370 0x7ff2240d4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff22159a640 1 -- 172.21.15.16:0/3305801028 >> v1:172.21.15.146:6789/0 conn(0x7ff22415e2a0 legacy=0x7ff22406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff22159a640 1 -- 172.21.15.16:0/3305801028 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2240cf5b0 con 0x7ff22415e680 2024-09-19T07:52:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff22159a640 1 -- 172.21.15.16:0/3305801028 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7ff218001670 con 0x7ff22415e680 2024-09-19T07:52:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff22159a640 1 -- 172.21.15.16:0/3305801028 >> v1:172.21.15.16:6789/0 conn(0x7ff22415e680 legacy=0x7ff2240cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff22159a640 1 --2- 172.21.15.16:0/3305801028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20c0039b0 0x7ff20c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff22159a640 1 --2- 172.21.15.16:0/3305801028 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff20c0043b0 0x7ff20c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff22159a640 1 --2- 172.21.15.16:0/3305801028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff20c005080 0x7ff20c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff22159a640 1 -- 172.21.15.16:0/3305801028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2240cf2d0 con 0x7ff20c0039b0 2024-09-19T07:52:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff22159a640 1 -- 172.21.15.16:0/3305801028 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c00bfe0 con 0x7ff20c005080 2024-09-19T07:52:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff22159a640 1 -- 172.21.15.16:0/3305801028 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff20c00c1e0 con 0x7ff20c0043b0 2024-09-19T07:52:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff22259c640 1 --2- 172.21.15.16:0/3305801028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20c0039b0 0x7ff20c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff221d9b640 1 --2- 172.21.15.16:0/3305801028 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff20c0043b0 0x7ff20c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff222d9d640 1 --2- 172.21.15.16:0/3305801028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff20c005080 0x7ff20c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff22259c640 1 -- 172.21.15.16:0/3305801028 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff20c0043b0 msgr2=0x7ff20c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff22259c640 1 --2- 172.21.15.16:0/3305801028 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff20c0043b0 0x7ff20c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff22259c640 1 -- 172.21.15.16:0/3305801028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff20c005080 msgr2=0x7ff20c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff22259c640 1 --2- 172.21.15.16:0/3305801028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff20c005080 0x7ff20c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff222d9d640 1 -- 172.21.15.16:0/3305801028 reap_dead start 2024-09-19T07:52:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.334+0000 7ff22259c640 1 -- 172.21.15.16:0/3305801028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20c003520 con 0x7ff20c0039b0 2024-09-19T07:52:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.335+0000 7ff22259c640 1 --2- 172.21.15.16:0/3305801028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20c0039b0 0x7ff20c003d90 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff214001860 tx=0x7ff20c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b3d14aa213f2d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.335+0000 7ff22159a640 1 -- 172.21.15.16:0/3305801028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214037070 con 0x7ff20c0039b0 2024-09-19T07:52:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.335+0000 7ff22159a640 1 -- 172.21.15.16:0/3305801028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff21402abf0 con 0x7ff20c0039b0 2024-09-19T07:52:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.335+0000 7ff22159a640 1 -- 172.21.15.16:0/3305801028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214001430 con 0x7ff20c0039b0 2024-09-19T07:52:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.336+0000 7ff228b8d640 1 -- 172.21.15.16:0/3305801028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20c0039b0 msgr2=0x7ff20c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.336+0000 7ff228b8d640 1 --2- 172.21.15.16:0/3305801028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20c0039b0 0x7ff20c003d90 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff214001860 tx=0x7ff20c005830 comp rx=0 tx=0).stop 2024-09-19T07:52:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.336+0000 7ff228b8d640 1 -- 172.21.15.16:0/3305801028 shutdown_connections 2024-09-19T07:52:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.336+0000 7ff228b8d640 1 --2- 172.21.15.16:0/3305801028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20c0039b0 0x7ff20c003d90 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.336+0000 7ff228b8d640 1 -- 172.21.15.16:0/3305801028 >> 172.21.15.16:0/3305801028 conn(0x7ff22409d760 msgr2=0x7ff22408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.336+0000 7ff228b8d640 1 -- 172.21.15.16:0/3305801028 shutdown_connections 2024-09-19T07:52:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.336+0000 7ff228b8d640 1 -- 172.21.15.16:0/3305801028 wait complete. 2024-09-19T07:52:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.337+0000 7ff228b8d640 1 Processor -- start 2024-09-19T07:52:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.337+0000 7ff228b8d640 1 -- start start 2024-09-19T07:52:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.338+0000 7ff228b8d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff20c0039b0 0x7ff2241f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.338+0000 7ff228b8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff224151720 0x7ff2240c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.338+0000 7ff228b8d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2240c8e40 0x7ff2240cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.338+0000 7ff228b8d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2240d49a0 con 0x7ff224151720 2024-09-19T07:52:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.338+0000 7ff228b8d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff2240d4820 con 0x7ff2240c8e40 2024-09-19T07:52:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.338+0000 7ff228b8d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff2240d4b20 con 0x7ff20c0039b0 2024-09-19T07:52:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.338+0000 7ff221d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff224151720 0x7ff2240c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.338+0000 7ff221d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff224151720 0x7ff2240c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36760/0 (socket says 172.21.15.16:36760) 2024-09-19T07:52:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.338+0000 7ff221d9b640 1 -- 172.21.15.16:0/3308267774 learned_addr learned my addr 172.21.15.16:0/3308267774 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.338+0000 7ff22259c640 1 --2- 172.21.15.16:0/3308267774 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff20c0039b0 0x7ff2241f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.338+0000 7ff222d9d640 1 --2- 172.21.15.16:0/3308267774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2240c8e40 0x7ff2240cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.338+0000 7ff221d9b640 1 -- 172.21.15.16:0/3308267774 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff20c0039b0 msgr2=0x7ff2241f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.338+0000 7ff221d9b640 1 --2- 172.21.15.16:0/3308267774 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff20c0039b0 0x7ff2241f4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.338+0000 7ff221d9b640 1 -- 172.21.15.16:0/3308267774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2240c8e40 msgr2=0x7ff2240cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.338+0000 7ff221d9b640 1 --2- 172.21.15.16:0/3308267774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2240c8e40 0x7ff2240cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.338+0000 7ff221d9b640 1 -- 172.21.15.16:0/3308267774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22415bf00 con 0x7ff224151720 2024-09-19T07:52:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.339+0000 7ff221d9b640 1 --2- 172.21.15.16:0/3308267774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff224151720 0x7ff2240c8900 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff218004fa0 tx=0x7ff2180074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.339+0000 7ff212ffd640 1 -- 172.21.15.16:0/3308267774 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21801a070 con 0x7ff224151720 2024-09-19T07:52:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.339+0000 7ff212ffd640 1 -- 172.21.15.16:0/3308267774 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff218007740 con 0x7ff224151720 2024-09-19T07:52:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.339+0000 7ff228b8d640 1 -- 172.21.15.16:0/3308267774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2240b83c0 con 0x7ff224151720 2024-09-19T07:52:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.339+0000 7ff212ffd640 1 -- 172.21.15.16:0/3308267774 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21800fbd0 con 0x7ff224151720 2024-09-19T07:52:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.339+0000 7ff228b8d640 1 -- 172.21.15.16:0/3308267774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2240b8980 con 0x7ff224151720 2024-09-19T07:52:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.340+0000 7ff228b8d640 1 -- 172.21.15.16:0/3308267774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff22406dbe0 con 0x7ff224151720 2024-09-19T07:52:54.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.340+0000 7ff212ffd640 1 -- 172.21.15.16:0/3308267774 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff21801d070 con 0x7ff224151720 2024-09-19T07:52:54.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.341+0000 7ff212ffd640 1 --2- 172.21.15.16:0/3308267774 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff20406c2e0 0x7ff20406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.341+0000 7ff212ffd640 1 -- 172.21.15.16:0/3308267774 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5335+0+0 (secure 0 0 0) 0x7ff21808aab0 con 0x7ff224151720 2024-09-19T07:52:54.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.341+0000 7ff22259c640 1 --2- 172.21.15.16:0/3308267774 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff20406c2e0 0x7ff20406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.342+0000 7ff22259c640 1 --2- 172.21.15.16:0/3308267774 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff20406c2e0 0x7ff20406e7a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff214034000 tx=0x7ff214009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.345+0000 7ff212ffd640 1 -- 172.21.15.16:0/3308267774 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff218058d40 con 0x7ff224151720 2024-09-19T07:52:54.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.521+0000 7ff228b8d640 1 -- 172.21.15.16:0/3308267774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) -- 0x7ff22406cf90 con 0x7ff224151720 2024-09-19T07:52:54.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.525+0000 7ff212ffd640 1 -- 172.21.15.16:0/3308267774 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v37) ==== 107+0+0 (secure 0 0 0) 0x7ff21805d040 con 0x7ff224151720 2024-09-19T07:52:54.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.527+0000 7ff228b8d640 1 -- 172.21.15.16:0/3308267774 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff20406c2e0 msgr2=0x7ff20406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.527+0000 7ff228b8d640 1 --2- 172.21.15.16:0/3308267774 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff20406c2e0 0x7ff20406e7a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff214034000 tx=0x7ff214009030 comp rx=0 tx=0).stop 2024-09-19T07:52:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.527+0000 7ff228b8d640 1 -- 172.21.15.16:0/3308267774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff224151720 msgr2=0x7ff2240c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.527+0000 7ff228b8d640 1 --2- 172.21.15.16:0/3308267774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff224151720 0x7ff2240c8900 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff218004fa0 tx=0x7ff2180074f0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.527+0000 7ff228b8d640 1 -- 172.21.15.16:0/3308267774 shutdown_connections 2024-09-19T07:52:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.527+0000 7ff228b8d640 1 --2- 172.21.15.16:0/3308267774 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff20406c2e0 0x7ff20406e7a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.527+0000 7ff228b8d640 1 --2- 172.21.15.16:0/3308267774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2240c8e40 0x7ff2240cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.527+0000 7ff228b8d640 1 --2- 172.21.15.16:0/3308267774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff224151720 0x7ff2240c8900 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.527+0000 7ff228b8d640 1 --2- 172.21.15.16:0/3308267774 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff20c0039b0 0x7ff2241f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.527+0000 7ff228b8d640 1 -- 172.21.15.16:0/3308267774 >> 172.21.15.16:0/3308267774 conn(0x7ff22409d760 msgr2=0x7ff22415c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.527+0000 7ff228b8d640 1 -- 172.21.15.16:0/3308267774 shutdown_connections 2024-09-19T07:52:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.527+0000 7ff228b8d640 1 -- 172.21.15.16:0/3308267774 wait complete. 2024-09-19T07:52:54.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2024-09-19T07:52:54.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.643+0000 7fb0b279f640 1 Processor -- start 2024-09-19T07:52:54.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.644+0000 7fb0b279f640 1 -- start start 2024-09-19T07:52:54.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.644+0000 7fb0b279f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0ac15c0c0 0x7fb0ac0b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.644+0000 7fb0b279f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb0ac0ccd10 0x7fb0ac09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.644+0000 7fb0b279f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0ac0cd1d0 con 0x7fb0ac09af30 2024-09-19T07:52:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.644+0000 7fb0b279f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb0ac09b380 con 0x7fb0ac0ccd10 2024-09-19T07:52:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.644+0000 7fb0b279f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb0ac0cd4b0 con 0x7fb0ac15c0c0 2024-09-19T07:52:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.645+0000 7fb0b179d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0ac15c0c0 0x7fb0ac0b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.645+0000 7fb0b0f9c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb0ac0ccd10 0x7fb0ac09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.645+0000 7fb0b179d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0ac15c0c0 0x7fb0ac0b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:51660/0 (socket says 172.21.15.16:51660) 2024-09-19T07:52:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.645+0000 7fb0b179d640 1 -- 172.21.15.16:0/3374909188 learned_addr learned my addr 172.21.15.16:0/3374909188 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.645+0000 7fb0b0f9c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb0ac0ccd10 0x7fb0ac09a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:52092/0 (socket says 172.21.15.16:52092) 2024-09-19T07:52:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.645+0000 7fb0b0f9c640 1 -- 172.21.15.16:0/3374909188 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0ac15c0c0 msgr2=0x7fb0ac0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.645+0000 7fb0b0f9c640 1 --2- 172.21.15.16:0/3374909188 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0ac15c0c0 0x7fb0ac0b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.645+0000 7fb0b0f9c640 1 -- 172.21.15.16:0/3374909188 >> v1:172.21.15.146:6789/0 conn(0x7fb0ac09af30 legacy=0x7fb0ac06ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.645+0000 7fb0abfff640 1 -- 172.21.15.16:0/3374909188 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3891213599 0 0) 0x7fb0ac0cd1d0 con 0x7fb0ac09af30 2024-09-19T07:52:54.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.645+0000 7fb0b0f9c640 1 -- 172.21.15.16:0/3374909188 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ac0cd790 con 0x7fb0ac0ccd10 2024-09-19T07:52:54.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.645+0000 7fb0b179d640 1 --2- 172.21.15.16:0/3374909188 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0ac15c0c0 0x7fb0ac0b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:52:54.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.646+0000 7fb0b0f9c640 1 --2- 172.21.15.16:0/3374909188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0ac0ccd10 0x7fb0ac09a9f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00097a0 tx=0x7fb0a002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf59a74f508b70a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.646+0000 7fb0abfff640 1 -- 172.21.15.16:0/3374909188 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0037070 con 0x7fb0ac0ccd10 2024-09-19T07:52:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.646+0000 7fb0abfff640 1 -- 172.21.15.16:0/3374909188 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0a002ae50 con 0x7fb0ac0ccd10 2024-09-19T07:52:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.646+0000 7fb0abfff640 1 -- 172.21.15.16:0/3374909188 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0032660 con 0x7fb0ac0ccd10 2024-09-19T07:52:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.647+0000 7fb0b279f640 1 -- 172.21.15.16:0/3374909188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0ac0ccd10 msgr2=0x7fb0ac09a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.647+0000 7fb0b279f640 1 --2- 172.21.15.16:0/3374909188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0ac0ccd10 0x7fb0ac09a9f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00097a0 tx=0x7fb0a002a100 comp rx=0 tx=0).stop 2024-09-19T07:52:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.647+0000 7fb0b279f640 1 -- 172.21.15.16:0/3374909188 shutdown_connections 2024-09-19T07:52:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.647+0000 7fb0b279f640 1 --2- 172.21.15.16:0/3374909188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0ac0ccd10 0x7fb0ac09a9f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.647+0000 7fb0b279f640 1 --2- 172.21.15.16:0/3374909188 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0ac15c0c0 0x7fb0ac0b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.647+0000 7fb0b279f640 1 -- 172.21.15.16:0/3374909188 >> 172.21.15.16:0/3374909188 conn(0x7fb0ac0b3ac0 msgr2=0x7fb0ac0b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.647+0000 7fb0b279f640 1 -- 172.21.15.16:0/3374909188 shutdown_connections 2024-09-19T07:52:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.647+0000 7fb0b279f640 1 -- 172.21.15.16:0/3374909188 wait complete. 2024-09-19T07:52:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.648+0000 7fb0b279f640 1 Processor -- start 2024-09-19T07:52:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.648+0000 7fb0b279f640 1 -- start start 2024-09-19T07:52:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.649+0000 7fb0b279f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0ac09af30 0x7fb0ac0aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.649+0000 7fb0b279f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0ac0ccd10 0x7fb0ac0b7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.649+0000 7fb0b279f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0ac15c0c0 0x7fb0ac150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.649+0000 7fb0b279f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac0b1570 con 0x7fb0ac15c0c0 2024-09-19T07:52:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.649+0000 7fb0b279f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac0b13f0 con 0x7fb0ac09af30 2024-09-19T07:52:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.649+0000 7fb0b279f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb0ac15c4a0 con 0x7fb0ac0ccd10 2024-09-19T07:52:54.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.649+0000 7fb0b1f9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0ac15c0c0 0x7fb0ac150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.649+0000 7fb0b1f9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0ac15c0c0 0x7fb0ac150850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36774/0 (socket says 172.21.15.16:36774) 2024-09-19T07:52:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.649+0000 7fb0b1f9e640 1 -- 172.21.15.16:0/618545762 learned_addr learned my addr 172.21.15.16:0/618545762 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.649+0000 7fb0b179d640 1 --2- 172.21.15.16:0/618545762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0ac09af30 0x7fb0ac0aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.649+0000 7fb0b0f9c640 1 --2- 172.21.15.16:0/618545762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0ac0ccd10 0x7fb0ac0b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.649+0000 7fb0b1f9e640 1 -- 172.21.15.16:0/618545762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0ac0ccd10 msgr2=0x7fb0ac0b7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.649+0000 7fb0b1f9e640 1 --2- 172.21.15.16:0/618545762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0ac0ccd10 0x7fb0ac0b7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.649+0000 7fb0b1f9e640 1 -- 172.21.15.16:0/618545762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0ac09af30 msgr2=0x7fb0ac0aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.649+0000 7fb0b1f9e640 1 --2- 172.21.15.16:0/618545762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0ac09af30 0x7fb0ac0aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.649+0000 7fb0b1f9e640 1 -- 172.21.15.16:0/618545762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ac14f770 con 0x7fb0ac15c0c0 2024-09-19T07:52:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.650+0000 7fb0b1f9e640 1 --2- 172.21.15.16:0/618545762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0ac15c0c0 0x7fb0ac150850 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400c3e0 tx=0x7fb0a400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.650+0000 7fb0a9ffb640 1 -- 172.21.15.16:0/618545762 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4017070 con 0x7fb0ac15c0c0 2024-09-19T07:52:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.650+0000 7fb0a9ffb640 1 -- 172.21.15.16:0/618545762 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0a40035f0 con 0x7fb0ac15c0c0 2024-09-19T07:52:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.650+0000 7fb0b279f640 1 -- 172.21.15.16:0/618545762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ac14fa10 con 0x7fb0ac15c0c0 2024-09-19T07:52:54.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.650+0000 7fb0a9ffb640 1 -- 172.21.15.16:0/618545762 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a40039e0 con 0x7fb0ac15c0c0 2024-09-19T07:52:54.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.650+0000 7fb0b279f640 1 -- 172.21.15.16:0/618545762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ac150330 con 0x7fb0ac15c0c0 2024-09-19T07:52:54.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.651+0000 7fb0b279f640 1 -- 172.21.15.16:0/618545762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0ac09a9f0 con 0x7fb0ac15c0c0 2024-09-19T07:52:54.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.652+0000 7fb0a9ffb640 1 -- 172.21.15.16:0/618545762 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb0a401a070 con 0x7fb0ac15c0c0 2024-09-19T07:52:54.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.652+0000 7fb0a9ffb640 1 --2- 172.21.15.16:0/618545762 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb09806c2e0 0x7fb09806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.652+0000 7fb0a9ffb640 1 -- 172.21.15.16:0/618545762 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5335+0+0 (secure 0 0 0) 0x7fb0a408a3c0 con 0x7fb0ac15c0c0 2024-09-19T07:52:54.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.652+0000 7fb0b179d640 1 --2- 172.21.15.16:0/618545762 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb09806c2e0 0x7fb09806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.653+0000 7fb0b179d640 1 --2- 172.21.15.16:0/618545762 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb09806c2e0 0x7fb09806e7a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009c90 tx=0x7fb09c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:54.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.657+0000 7fb0a9ffb640 1 -- 172.21.15.16:0/618545762 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb0a40586e0 con 0x7fb0ac15c0c0 2024-09-19T07:52:54.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.834+0000 7fb0b279f640 1 -- 172.21.15.16:0/618545762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) -- 0x7fb0ac09b960 con 0x7fb0ac15c0c0 2024-09-19T07:52:54.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.837+0000 7fb0a9ffb640 1 -- 172.21.15.16:0/618545762 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v38) ==== 102+0+0 (secure 0 0 0) 0x7fb0a405c9e0 con 0x7fb0ac15c0c0 2024-09-19T07:52:54.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.840+0000 7fb0b279f640 1 -- 172.21.15.16:0/618545762 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb09806c2e0 msgr2=0x7fb09806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.840+0000 7fb0b279f640 1 --2- 172.21.15.16:0/618545762 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb09806c2e0 0x7fb09806e7a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009c90 tx=0x7fb09c00b040 comp rx=0 tx=0).stop 2024-09-19T07:52:54.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.840+0000 7fb0b279f640 1 -- 172.21.15.16:0/618545762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0ac15c0c0 msgr2=0x7fb0ac150850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.840+0000 7fb0b279f640 1 --2- 172.21.15.16:0/618545762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0ac15c0c0 0x7fb0ac150850 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400c3e0 tx=0x7fb0a400c8a0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.840+0000 7fb0b279f640 1 -- 172.21.15.16:0/618545762 shutdown_connections 2024-09-19T07:52:54.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.840+0000 7fb0b279f640 1 --2- 172.21.15.16:0/618545762 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb09806c2e0 0x7fb09806e7a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.840+0000 7fb0b279f640 1 --2- 172.21.15.16:0/618545762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0ac15c0c0 0x7fb0ac150850 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.840+0000 7fb0b279f640 1 --2- 172.21.15.16:0/618545762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0ac0ccd10 0x7fb0ac0b7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.840+0000 7fb0b279f640 1 --2- 172.21.15.16:0/618545762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0ac09af30 0x7fb0ac0aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.840+0000 7fb0b279f640 1 -- 172.21.15.16:0/618545762 >> 172.21.15.16:0/618545762 conn(0x7fb0ac0b3ac0 msgr2=0x7fb0ac0cc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:54.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.840+0000 7fb0b279f640 1 -- 172.21.15.16:0/618545762 shutdown_connections 2024-09-19T07:52:54.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.840+0000 7fb0b279f640 1 -- 172.21.15.16:0/618545762 wait complete. 2024-09-19T07:52:54.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T07:52:54.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.956+0000 7ffaf6f5b640 1 Processor -- start 2024-09-19T07:52:54.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.957+0000 7ffaf6f5b640 1 -- start start 2024-09-19T07:52:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.957+0000 7ffaf6f5b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffaf0142170 0x7ffaf0166d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.957+0000 7ffaf6f5b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaf015d710 con 0x7ffaf0141680 2024-09-19T07:52:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.957+0000 7ffaf6f5b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaf015c950 con 0x7ffaf00d0540 2024-09-19T07:52:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.957+0000 7ffaf6f5b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ffaf015c6f0 con 0x7ffaf0142170 2024-09-19T07:52:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.958+0000 7ffaf4cf7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffaf0141680 0x7ffaf00cfe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51452/0 (socket says 172.21.15.16:51452) 2024-09-19T07:52:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.958+0000 7ffaf4cf7640 1 -- 172.21.15.16:0/2485259568 learned_addr learned my addr 172.21.15.16:0/2485259568 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:52:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.958+0000 7ffaf54f8640 1 --2- 172.21.15.16:0/2485259568 >> v2:172.21.15.16:3300/0 conn(0x7ffaf0142170 0x7ffaf0166d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.958+0000 7ffaf54f8640 1 -- 172.21.15.16:0/2485259568 >> v1:172.21.15.146:6789/0 conn(0x7ffaf00d0540 legacy=0x7ffaf0141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.958+0000 7ffaf54f8640 1 -- 172.21.15.16:0/2485259568 >> v1:172.21.15.16:6789/0 conn(0x7ffaf0141680 legacy=0x7ffaf00cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.958+0000 7ffaf54f8640 1 -- 172.21.15.16:0/2485259568 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf015cc30 con 0x7ffaf0142170 2024-09-19T07:52:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.959+0000 7ffaf54f8640 1 --2- 172.21.15.16:0/2485259568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf0142170 0x7ffaf0166d70 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009d90 tx=0x7ffae802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f6609c76aacb9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.959+0000 7ffaef7fe640 1 -- 172.21.15.16:0/2485259568 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae8007740 con 0x7ffaf0142170 2024-09-19T07:52:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.959+0000 7ffaef7fe640 1 -- 172.21.15.16:0/2485259568 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffae80078e0 con 0x7ffaf0142170 2024-09-19T07:52:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.959+0000 7ffaef7fe640 1 -- 172.21.15.16:0/2485259568 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae8007cb0 con 0x7ffaf0142170 2024-09-19T07:52:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.959+0000 7ffaf6f5b640 1 -- 172.21.15.16:0/2485259568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf0142170 msgr2=0x7ffaf0166d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.959+0000 7ffaf6f5b640 1 --2- 172.21.15.16:0/2485259568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf0142170 0x7ffaf0166d70 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009d90 tx=0x7ffae802a6a0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.959+0000 7ffaf6f5b640 1 -- 172.21.15.16:0/2485259568 shutdown_connections 2024-09-19T07:52:54.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.959+0000 7ffaf6f5b640 1 --2- 172.21.15.16:0/2485259568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf0142170 0x7ffaf0166d70 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.959+0000 7ffaf6f5b640 1 -- 172.21.15.16:0/2485259568 >> 172.21.15.16:0/2485259568 conn(0x7ffaf00b3ed0 msgr2=0x7ffaf00b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:54.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.959+0000 7ffaf6f5b640 1 -- 172.21.15.16:0/2485259568 shutdown_connections 2024-09-19T07:52:54.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.959+0000 7ffaf6f5b640 1 -- 172.21.15.16:0/2485259568 wait complete. 2024-09-19T07:52:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.960+0000 7ffaf6f5b640 1 Processor -- start 2024-09-19T07:52:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.961+0000 7ffaf6f5b640 1 -- start start 2024-09-19T07:52:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.961+0000 7ffaf6f5b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf00d0540 0x7ffaf0155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.961+0000 7ffaf6f5b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaf0141680 0x7ffaf00b6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.961+0000 7ffaf6f5b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffaf0142170 0x7ffaf00c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.961+0000 7ffaf6f5b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf01425c0 con 0x7ffaf00d0540 2024-09-19T07:52:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.961+0000 7ffaf6f5b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf015d1f0 con 0x7ffaf0141680 2024-09-19T07:52:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.961+0000 7ffaf6f5b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ffaf01408c0 con 0x7ffaf0142170 2024-09-19T07:52:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.961+0000 7ffaf4cf7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf00d0540 0x7ffaf0155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.961+0000 7ffaf4cf7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf00d0540 0x7ffaf0155a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36788/0 (socket says 172.21.15.16:36788) 2024-09-19T07:52:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.962+0000 7ffaf4cf7640 1 -- 172.21.15.16:0/2374512295 learned_addr learned my addr 172.21.15.16:0/2374512295 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.962+0000 7ffaf54f8640 1 --2- 172.21.15.16:0/2374512295 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffaf0142170 0x7ffaf00c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.962+0000 7ffaf4cf7640 1 -- 172.21.15.16:0/2374512295 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffaf0142170 msgr2=0x7ffaf00c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.962+0000 7ffaeffff640 1 --2- 172.21.15.16:0/2374512295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaf0141680 0x7ffaf00b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.962+0000 7ffaf4cf7640 1 --2- 172.21.15.16:0/2374512295 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffaf0142170 0x7ffaf00c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.962+0000 7ffaf4cf7640 1 -- 172.21.15.16:0/2374512295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaf0141680 msgr2=0x7ffaf00b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:54.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.962+0000 7ffaf4cf7640 1 --2- 172.21.15.16:0/2374512295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaf0141680 0x7ffaf00b6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:54.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.962+0000 7ffaf4cf7640 1 -- 172.21.15.16:0/2374512295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf0155fd0 con 0x7ffaf00d0540 2024-09-19T07:52:54.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.962+0000 7ffaf4cf7640 1 --2- 172.21.15.16:0/2374512295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf00d0540 0x7ffaf0155a90 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ffae000c430 tx=0x7ffae000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:54.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.962+0000 7ffaed7fa640 1 -- 172.21.15.16:0/2374512295 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae0017070 con 0x7ffaf00d0540 2024-09-19T07:52:54.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.962+0000 7ffaed7fa640 1 -- 172.21.15.16:0/2374512295 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffae0005ce0 con 0x7ffaf00d0540 2024-09-19T07:52:54.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.962+0000 7ffaf6f5b640 1 -- 172.21.15.16:0/2374512295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf00b70e0 con 0x7ffaf00d0540 2024-09-19T07:52:54.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.962+0000 7ffaed7fa640 1 -- 172.21.15.16:0/2374512295 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae00072f0 con 0x7ffaf00d0540 2024-09-19T07:52:54.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.962+0000 7ffaf6f5b640 1 -- 172.21.15.16:0/2374512295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf00b76a0 con 0x7ffaf00d0540 2024-09-19T07:52:54.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.963+0000 7ffaf6f5b640 1 -- 172.21.15.16:0/2374512295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaf0141a60 con 0x7ffaf00d0540 2024-09-19T07:52:54.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.964+0000 7ffaed7fa640 1 -- 172.21.15.16:0/2374512295 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ffae001a070 con 0x7ffaf00d0540 2024-09-19T07:52:54.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.964+0000 7ffaed7fa640 1 --2- 172.21.15.16:0/2374512295 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffadc06c2b0 0x7ffadc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:54.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.964+0000 7ffaed7fa640 1 -- 172.21.15.16:0/2374512295 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5335+0+0 (secure 0 0 0) 0x7ffae008a630 con 0x7ffaf00d0540 2024-09-19T07:52:54.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.964+0000 7ffaeffff640 1 --2- 172.21.15.16:0/2374512295 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffadc06c2b0 0x7ffadc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:54.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.965+0000 7ffaeffff640 1 --2- 172.21.15.16:0/2374512295 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffadc06c2b0 0x7ffadc06e770 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00c92b0 tx=0x7ffae4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:54.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:54.969+0000 7ffaed7fa640 1 -- 172.21.15.16:0/2374512295 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ffaf0155fd0 con 0x7ffaf00d0540 2024-09-19T07:52:55.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.146+0000 7ffaf6f5b640 1 -- 172.21.15.16:0/2374512295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7ffaf00d2ab0 con 0x7ffaf00d0540 2024-09-19T07:52:55.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.150+0000 7ffaed7fa640 1 -- 172.21.15.16:0/2374512295 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7ffae0058fa0 con 0x7ffaf00d0540 2024-09-19T07:52:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.152+0000 7ffaf6f5b640 1 -- 172.21.15.16:0/2374512295 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffadc06c2b0 msgr2=0x7ffadc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.152+0000 7ffaf6f5b640 1 --2- 172.21.15.16:0/2374512295 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffadc06c2b0 0x7ffadc06e770 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00c92b0 tx=0x7ffae4008040 comp rx=0 tx=0).stop 2024-09-19T07:52:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.152+0000 7ffaf6f5b640 1 -- 172.21.15.16:0/2374512295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf00d0540 msgr2=0x7ffaf0155a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.152+0000 7ffaf6f5b640 1 --2- 172.21.15.16:0/2374512295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf00d0540 0x7ffaf0155a90 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ffae000c430 tx=0x7ffae000c8f0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.152+0000 7ffaf6f5b640 1 -- 172.21.15.16:0/2374512295 shutdown_connections 2024-09-19T07:52:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.152+0000 7ffaf6f5b640 1 --2- 172.21.15.16:0/2374512295 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffadc06c2b0 0x7ffadc06e770 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.152+0000 7ffaf6f5b640 1 --2- 172.21.15.16:0/2374512295 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffaf0142170 0x7ffaf00c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.152+0000 7ffaf6f5b640 1 --2- 172.21.15.16:0/2374512295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaf0141680 0x7ffaf00b6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.152+0000 7ffaf6f5b640 1 --2- 172.21.15.16:0/2374512295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf00d0540 0x7ffaf0155a90 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.152+0000 7ffaf6f5b640 1 -- 172.21.15.16:0/2374512295 >> 172.21.15.16:0/2374512295 conn(0x7ffaf00b3ed0 msgr2=0x7ffaf015b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:55.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.152+0000 7ffaf6f5b640 1 -- 172.21.15.16:0/2374512295 shutdown_connections 2024-09-19T07:52:55.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.153+0000 7ffaf6f5b640 1 -- 172.21.15.16:0/2374512295 wait complete. 2024-09-19T07:52:55.164 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2024-09-19T07:52:55.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.268+0000 7f00e4ba6640 1 Processor -- start 2024-09-19T07:52:55.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.268+0000 7f00e4ba6640 1 -- start start 2024-09-19T07:52:55.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.269+0000 7f00e4ba6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f00e015bee0 0x7f00e00d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:55.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.269+0000 7f00e4ba6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f00e015c2c0 0x7f00e00d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:55.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.269+0000 7f00e4ba6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e00cdee0 con 0x7f00e00cf200 2024-09-19T07:52:55.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.269+0000 7f00e4ba6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f00e00ceaf0 con 0x7f00e015c2c0 2024-09-19T07:52:55.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.269+0000 7f00e4ba6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f00e00ce1c0 con 0x7f00e015bee0 2024-09-19T07:52:55.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.269+0000 7f00d7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f00e015c2c0 0x7f00e00d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:55.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.269+0000 7f00d7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f00e015c2c0 0x7f00e00d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36792/0 (socket says 172.21.15.16:36792) 2024-09-19T07:52:55.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.269+0000 7f00d7fff640 1 -- 172.21.15.16:0/2395777310 learned_addr learned my addr 172.21.15.16:0/2395777310 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:55.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.269+0000 7f00de59c640 1 --2- 172.21.15.16:0/2395777310 >> v2:172.21.15.146:3300/0 conn(0x7f00e015bee0 0x7f00e00d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:55.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.269+0000 7f00d7fff640 1 -- 172.21.15.16:0/2395777310 >> v2:172.21.15.146:3300/0 conn(0x7f00e015bee0 msgr2=0x7f00e00d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:55.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.269+0000 7f00d7fff640 1 --2- 172.21.15.16:0/2395777310 >> v2:172.21.15.146:3300/0 conn(0x7f00e015bee0 0x7f00e00d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.269+0000 7f00d7fff640 1 -- 172.21.15.16:0/2395777310 >> v1:172.21.15.146:6789/0 conn(0x7f00e00cf200 legacy=0x7f00e00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:55.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.269+0000 7f00d7fff640 1 -- 172.21.15.16:0/2395777310 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e00ce4a0 con 0x7f00e015c2c0 2024-09-19T07:52:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.270+0000 7f00d7fff640 1 --2- 172.21.15.16:0/2395777310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e015c2c0 0x7f00e00d09f0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f00e00b2f40 tx=0x7f00c802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57a1d3eb2471cad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.270+0000 7f00ddd9b640 1 -- 172.21.15.16:0/2395777310 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8037070 con 0x7f00e015c2c0 2024-09-19T07:52:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.270+0000 7f00ddd9b640 1 -- 172.21.15.16:0/2395777310 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00c8032440 con 0x7f00e015c2c0 2024-09-19T07:52:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.270+0000 7f00ddd9b640 1 -- 172.21.15.16:0/2395777310 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8032810 con 0x7f00e015c2c0 2024-09-19T07:52:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.271+0000 7f00e4ba6640 1 -- 172.21.15.16:0/2395777310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e015c2c0 msgr2=0x7f00e00d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.271+0000 7f00e4ba6640 1 --2- 172.21.15.16:0/2395777310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e015c2c0 0x7f00e00d09f0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f00e00b2f40 tx=0x7f00c802a230 comp rx=0 tx=0).stop 2024-09-19T07:52:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.271+0000 7f00e4ba6640 1 -- 172.21.15.16:0/2395777310 shutdown_connections 2024-09-19T07:52:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.271+0000 7f00e4ba6640 1 --2- 172.21.15.16:0/2395777310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e015c2c0 0x7f00e00d09f0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.271+0000 7f00e4ba6640 1 --2- 172.21.15.16:0/2395777310 >> v2:172.21.15.146:3300/0 conn(0x7f00e015bee0 0x7f00e00d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.271+0000 7f00e4ba6640 1 -- 172.21.15.16:0/2395777310 >> 172.21.15.16:0/2395777310 conn(0x7f00e006bdf0 msgr2=0x7f00e006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.271+0000 7f00e4ba6640 1 -- 172.21.15.16:0/2395777310 shutdown_connections 2024-09-19T07:52:55.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.271+0000 7f00e4ba6640 1 -- 172.21.15.16:0/2395777310 wait complete. 2024-09-19T07:52:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.272+0000 7f00e4ba6640 1 Processor -- start 2024-09-19T07:52:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.272+0000 7f00e4ba6640 1 -- start start 2024-09-19T07:52:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.273+0000 7f00e4ba6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00e00cf200 0x7f00e015f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:55.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.273+0000 7f00e4ba6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e015bee0 0x7f00e0153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.273+0000 7f00e4ba6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00e015c2c0 0x7f00e00ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.273+0000 7f00e4ba6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00e006e040 con 0x7f00e015bee0 2024-09-19T07:52:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.273+0000 7f00e4ba6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f00e006dec0 con 0x7f00e00cf200 2024-09-19T07:52:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.273+0000 7f00e4ba6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f00e006e1c0 con 0x7f00e015c2c0 2024-09-19T07:52:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.273+0000 7f00d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e015bee0 0x7f00e0153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.273+0000 7f00d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e015bee0 0x7f00e0153b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36802/0 (socket says 172.21.15.16:36802) 2024-09-19T07:52:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.273+0000 7f00d7fff640 1 -- 172.21.15.16:0/1972338011 learned_addr learned my addr 172.21.15.16:0/1972338011 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.273+0000 7f00de59c640 1 --2- 172.21.15.16:0/1972338011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00e00cf200 0x7f00e015f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.273+0000 7f00ded9d640 1 --2- 172.21.15.16:0/1972338011 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00e015c2c0 0x7f00e00ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.273+0000 7f00d7fff640 1 -- 172.21.15.16:0/1972338011 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00e015c2c0 msgr2=0x7f00e00ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.273+0000 7f00d7fff640 1 --2- 172.21.15.16:0/1972338011 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00e015c2c0 0x7f00e00ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.273+0000 7f00d7fff640 1 -- 172.21.15.16:0/1972338011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00e00cf200 msgr2=0x7f00e015f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:55.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.273+0000 7f00d7fff640 1 --2- 172.21.15.16:0/1972338011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00e00cf200 0x7f00e015f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.273+0000 7f00d7fff640 1 -- 172.21.15.16:0/1972338011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e01f5c80 con 0x7f00e015bee0 2024-09-19T07:52:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.274+0000 7f00d7fff640 1 --2- 172.21.15.16:0/1972338011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e015bee0 0x7f00e0153b60 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f00c802a270 tx=0x7f00c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.274+0000 7f00d6ffd640 1 -- 172.21.15.16:0/1972338011 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8041020 con 0x7f00e015bee0 2024-09-19T07:52:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.274+0000 7f00d6ffd640 1 -- 172.21.15.16:0/1972338011 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00c80306c0 con 0x7f00e015bee0 2024-09-19T07:52:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.274+0000 7f00e4ba6640 1 -- 172.21.15.16:0/1972338011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e01f5f60 con 0x7f00e015bee0 2024-09-19T07:52:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.274+0000 7f00d6ffd640 1 -- 172.21.15.16:0/1972338011 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8037040 con 0x7f00e015bee0 2024-09-19T07:52:55.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.274+0000 7f00e4ba6640 1 -- 172.21.15.16:0/1972338011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e01f6520 con 0x7f00e015bee0 2024-09-19T07:52:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.275+0000 7f00e4ba6640 1 -- 172.21.15.16:0/1972338011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e00cf690 con 0x7f00e015bee0 2024-09-19T07:52:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.275+0000 7f00d6ffd640 1 -- 172.21.15.16:0/1972338011 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f00c8044020 con 0x7f00e015bee0 2024-09-19T07:52:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.276+0000 7f00d6ffd640 1 --2- 172.21.15.16:0/1972338011 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f00c006c2b0 0x7f00c006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.276+0000 7f00d6ffd640 1 -- 172.21.15.16:0/1972338011 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5335+0+0 (secure 0 0 0) 0x7f00c80ab920 con 0x7f00e015bee0 2024-09-19T07:52:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.276+0000 7f00de59c640 1 --2- 172.21.15.16:0/1972338011 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f00c006c2b0 0x7f00c006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:55.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.277+0000 7f00de59c640 1 --2- 172.21.15.16:0/1972338011 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f00c006c2b0 0x7f00c006e770 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f00d0006fd0 tx=0x7f00d0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:55.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.280+0000 7f00d6ffd640 1 -- 172.21.15.16:0/1972338011 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f00c804b030 con 0x7f00e015bee0 2024-09-19T07:52:55.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.457+0000 7f00e4ba6640 1 -- 172.21.15.16:0/1972338011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) -- 0x7f00e0160050 con 0x7f00e015bee0 2024-09-19T07:52:55.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.460+0000 7f00d6ffd640 1 -- 172.21.15.16:0/1972338011 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v40) ==== 105+0+0 (secure 0 0 0) 0x7f00c8079c40 con 0x7f00e015bee0 2024-09-19T07:52:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.462+0000 7f00e4ba6640 1 -- 172.21.15.16:0/1972338011 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f00c006c2b0 msgr2=0x7f00c006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.462+0000 7f00e4ba6640 1 --2- 172.21.15.16:0/1972338011 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f00c006c2b0 0x7f00c006e770 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f00d0006fd0 tx=0x7f00d0008040 comp rx=0 tx=0).stop 2024-09-19T07:52:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.462+0000 7f00e4ba6640 1 -- 172.21.15.16:0/1972338011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e015bee0 msgr2=0x7f00e0153b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.462+0000 7f00e4ba6640 1 --2- 172.21.15.16:0/1972338011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e015bee0 0x7f00e0153b60 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f00c802a270 tx=0x7f00c80046c0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.462+0000 7f00e4ba6640 1 -- 172.21.15.16:0/1972338011 shutdown_connections 2024-09-19T07:52:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.462+0000 7f00e4ba6640 1 --2- 172.21.15.16:0/1972338011 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f00c006c2b0 0x7f00c006e770 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.462+0000 7f00e4ba6640 1 --2- 172.21.15.16:0/1972338011 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00e015c2c0 0x7f00e00ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.462+0000 7f00e4ba6640 1 --2- 172.21.15.16:0/1972338011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e015bee0 0x7f00e0153b60 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.462+0000 7f00e4ba6640 1 --2- 172.21.15.16:0/1972338011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00e00cf200 0x7f00e015f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.462+0000 7f00e4ba6640 1 -- 172.21.15.16:0/1972338011 >> 172.21.15.16:0/1972338011 conn(0x7f00e006bdf0 msgr2=0x7f00e0160570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.462+0000 7f00e4ba6640 1 -- 172.21.15.16:0/1972338011 shutdown_connections 2024-09-19T07:52:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.462+0000 7f00e4ba6640 1 -- 172.21.15.16:0/1972338011 wait complete. 2024-09-19T07:52:55.474 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set' 2024-09-19T07:52:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.576+0000 7f5ffee9b640 1 Processor -- start 2024-09-19T07:52:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.576+0000 7f5ffee9b640 1 -- start start 2024-09-19T07:52:55.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.576+0000 7f5ffee9b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5ff806f9e0 0x7f5ff80d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:55.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ffee9b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ff80d0e00 con 0x7f5ff80d09f0 2024-09-19T07:52:55.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ffee9b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ff815b500 con 0x7f5ff815b120 2024-09-19T07:52:55.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ffee9b640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5ff806e940 con 0x7f5ff806f9e0 2024-09-19T07:52:55.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ffd438640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5ff80d09f0 0x7f5ff80c5160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51456/0 (socket says 172.21.15.16:51456) 2024-09-19T07:52:55.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ffd438640 1 -- 172.21.15.16:0/4108780400 learned_addr learned my addr 172.21.15.16:0/4108780400 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:52:55.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ff77fe640 1 -- 172.21.15.16:0/4108780400 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2585499281 0 0) 0x7f5ff0001490 con 0x7f5ff80d09f0 2024-09-19T07:52:55.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ff77fe640 1 -- 172.21.15.16:0/4108780400 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fe8003520 con 0x7f5ff80d09f0 2024-09-19T07:52:55.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ff7fff640 1 --2- 172.21.15.16:0/4108780400 >> v2:172.21.15.146:3300/0 conn(0x7f5ff806f9e0 0x7f5ff80d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:55.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ff77fe640 1 -- 172.21.15.16:0/4108780400 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3012666441 0 0) 0x7f5fe8003520 con 0x7f5ff80d09f0 2024-09-19T07:52:55.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ff77fe640 1 -- 172.21.15.16:0/4108780400 >> v2:172.21.15.146:3300/0 conn(0x7f5ff806f9e0 msgr2=0x7f5ff80d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:55.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ff77fe640 1 --2- 172.21.15.16:0/4108780400 >> v2:172.21.15.146:3300/0 conn(0x7f5ff806f9e0 0x7f5ff80d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ff77fe640 1 -- 172.21.15.16:0/4108780400 >> v1:172.21.15.146:6789/0 conn(0x7f5ff815b120 legacy=0x7f5ff806dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ff77fe640 1 -- 172.21.15.16:0/4108780400 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff806ec20 con 0x7f5ff80d09f0 2024-09-19T07:52:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ff77fe640 1 -- 172.21.15.16:0/4108780400 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f5ff0001bc0 con 0x7f5ff80d09f0 2024-09-19T07:52:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ff77fe640 1 -- 172.21.15.16:0/4108780400 >> v1:172.21.15.16:6789/0 conn(0x7f5ff80d09f0 legacy=0x7f5ff80c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ff77fe640 1 --2- 172.21.15.16:0/4108780400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5fe80039b0 0x7f5fe8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ff77fe640 1 --2- 172.21.15.16:0/4108780400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fe8004800 0x7f5fe8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ff77fe640 1 --2- 172.21.15.16:0/4108780400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe80051c0 0x7f5fe800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ff77fe640 1 -- 172.21.15.16:0/4108780400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff806e940 con 0x7f5fe80051c0 2024-09-19T07:52:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ff77fe640 1 -- 172.21.15.16:0/4108780400 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe800c060 con 0x7f5fe8004800 2024-09-19T07:52:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ff77fe640 1 -- 172.21.15.16:0/4108780400 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe800c260 con 0x7f5fe80039b0 2024-09-19T07:52:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.577+0000 7f5ffd438640 1 --2- 172.21.15.16:0/4108780400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe80051c0 0x7f5fe800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.578+0000 7f5ff7fff640 1 --2- 172.21.15.16:0/4108780400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fe8004800 0x7f5fe8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.578+0000 7f5ffcc37640 1 --2- 172.21.15.16:0/4108780400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5fe80039b0 0x7f5fe8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.578+0000 7f5ffd438640 1 -- 172.21.15.16:0/4108780400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5fe80039b0 msgr2=0x7f5fe8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.578+0000 7f5ffd438640 1 --2- 172.21.15.16:0/4108780400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5fe80039b0 0x7f5fe8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.578+0000 7f5ffd438640 1 -- 172.21.15.16:0/4108780400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fe8004800 msgr2=0x7f5fe8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:55.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.578+0000 7f5ffd438640 1 --2- 172.21.15.16:0/4108780400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fe8004800 0x7f5fe8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.578+0000 7f5ffd438640 1 -- 172.21.15.16:0/4108780400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe8003520 con 0x7f5fe80051c0 2024-09-19T07:52:55.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.578+0000 7f5ffd438640 1 -- 172.21.15.16:0/4108780400 reap_dead start 2024-09-19T07:52:55.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.579+0000 7f5ffd438640 1 --2- 172.21.15.16:0/4108780400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe80051c0 0x7f5fe800b9c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000aa30 tx=0x7f5fe8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1344af7a37a6be54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:55.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.579+0000 7f5ff77fe640 1 -- 172.21.15.16:0/4108780400 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff000a650 con 0x7f5fe80051c0 2024-09-19T07:52:55.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.579+0000 7f5ff77fe640 1 -- 172.21.15.16:0/4108780400 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff002f070 con 0x7f5fe80051c0 2024-09-19T07:52:55.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.579+0000 7f5ff77fe640 1 -- 172.21.15.16:0/4108780400 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0034470 con 0x7f5fe80051c0 2024-09-19T07:52:55.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.579+0000 7f5ffee9b640 1 -- 172.21.15.16:0/4108780400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe80051c0 msgr2=0x7f5fe800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:55.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.579+0000 7f5ffee9b640 1 --2- 172.21.15.16:0/4108780400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe80051c0 0x7f5fe800b9c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000aa30 tx=0x7f5fe8005830 comp rx=0 tx=0).stop 2024-09-19T07:52:55.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.579+0000 7f5ffee9b640 1 -- 172.21.15.16:0/4108780400 shutdown_connections 2024-09-19T07:52:55.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.579+0000 7f5ffee9b640 1 --2- 172.21.15.16:0/4108780400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe80051c0 0x7f5fe800b9c0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.579+0000 7f5ffee9b640 1 -- 172.21.15.16:0/4108780400 >> 172.21.15.16:0/4108780400 conn(0x7f5ff806bdf0 msgr2=0x7f5ff806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:55.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.579+0000 7f5ffee9b640 1 -- 172.21.15.16:0/4108780400 shutdown_connections 2024-09-19T07:52:55.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.579+0000 7f5ffee9b640 1 -- 172.21.15.16:0/4108780400 wait complete. 2024-09-19T07:52:55.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.580+0000 7f5ffee9b640 1 Processor -- start 2024-09-19T07:52:55.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.581+0000 7f5ffee9b640 1 -- start start 2024-09-19T07:52:55.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.581+0000 7f5ffee9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe80051c0 0x7f5ff814e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:55.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.581+0000 7f5ffee9b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ff815a2c0 0x7f5ff8159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:55.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.581+0000 7f5ffee9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ff80b0d90 0x7f5ff80ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:55.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.581+0000 7f5ffee9b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff806f4c0 con 0x7f5fe80051c0 2024-09-19T07:52:55.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.581+0000 7f5ffee9b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff806f1e0 con 0x7f5ff80b0d90 2024-09-19T07:52:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.581+0000 7f5ffee9b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff80ce060 con 0x7f5ff815a2c0 2024-09-19T07:52:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.581+0000 7f5ffcc37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe80051c0 0x7f5ff814e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.582+0000 7f5ffcc37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe80051c0 0x7f5ff814e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36820/0 (socket says 172.21.15.16:36820) 2024-09-19T07:52:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.582+0000 7f5ffcc37640 1 -- 172.21.15.16:0/4135613113 learned_addr learned my addr 172.21.15.16:0/4135613113 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.582+0000 7f5ffd438640 1 --2- 172.21.15.16:0/4135613113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ff80b0d90 0x7f5ff80ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.582+0000 7f5ff7fff640 1 --2- 172.21.15.16:0/4135613113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ff815a2c0 0x7f5ff8159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.582+0000 7f5ffcc37640 1 -- 172.21.15.16:0/4135613113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ff815a2c0 msgr2=0x7f5ff8159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.582+0000 7f5ffcc37640 1 --2- 172.21.15.16:0/4135613113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ff815a2c0 0x7f5ff8159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.582+0000 7f5ffcc37640 1 -- 172.21.15.16:0/4135613113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ff80b0d90 msgr2=0x7f5ff80ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.582+0000 7f5ffcc37640 1 --2- 172.21.15.16:0/4135613113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ff80b0d90 0x7f5ff80ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.582+0000 7f5ffcc37640 1 -- 172.21.15.16:0/4135613113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff814ee50 con 0x7f5fe80051c0 2024-09-19T07:52:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.582+0000 7f5ffcc37640 1 --2- 172.21.15.16:0/4135613113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe80051c0 0x7f5ff814e910 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00f5f0 tx=0x7f5fec00f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.582+0000 7f5ff57fa640 1 -- 172.21.15.16:0/4135613113 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec0052e0 con 0x7f5fe80051c0 2024-09-19T07:52:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.582+0000 7f5ff57fa640 1 -- 172.21.15.16:0/4135613113 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fec018070 con 0x7f5fe80051c0 2024-09-19T07:52:55.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.582+0000 7f5ffee9b640 1 -- 172.21.15.16:0/4135613113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff80affd0 con 0x7f5fe80051c0 2024-09-19T07:52:55.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.582+0000 7f5ff57fa640 1 -- 172.21.15.16:0/4135613113 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec003810 con 0x7f5fe80051c0 2024-09-19T07:52:55.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.582+0000 7f5ffee9b640 1 -- 172.21.15.16:0/4135613113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff80b0590 con 0x7f5fe80051c0 2024-09-19T07:52:55.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.583+0000 7f5ffee9b640 1 -- 172.21.15.16:0/4135613113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff80d04b0 con 0x7f5fe80051c0 2024-09-19T07:52:55.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.584+0000 7f5ff57fa640 1 -- 172.21.15.16:0/4135613113 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5fec0039b0 con 0x7f5fe80051c0 2024-09-19T07:52:55.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.584+0000 7f5ff57fa640 1 --2- 172.21.15.16:0/4135613113 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5fdc06c2e0 0x7f5fdc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:55.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.584+0000 7f5ff57fa640 1 -- 172.21.15.16:0/4135613113 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5335+0+0 (secure 0 0 0) 0x7f5fec08b000 con 0x7f5fe80051c0 2024-09-19T07:52:55.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.584+0000 7f5ff7fff640 1 --2- 172.21.15.16:0/4135613113 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5fdc06c2e0 0x7f5fdc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:55.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.585+0000 7f5ff7fff640 1 --2- 172.21.15.16:0/4135613113 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5fdc06c2e0 0x7f5fdc06e7a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80c9db0 tx=0x7f5fe4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:55.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.589+0000 7f5ff57fa640 1 -- 172.21.15.16:0/4135613113 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5fec059840 con 0x7f5fe80051c0 2024-09-19T07:52:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.832+0000 7f5ffee9b640 1 -- 172.21.15.16:0/4135613113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]} v 0) -- 0x7f5ff806c510 con 0x7f5fe80051c0 2024-09-19T07:52:55.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.888+0000 7f5ff57fa640 1 -- 172.21.15.16:0/4135613113 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]}]=0 v427) ==== 160+0+0 (secure 0 0 0) 0x7f5fec059140 con 0x7f5fe80051c0 2024-09-19T07:52:55.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.890+0000 7f5ffee9b640 1 -- 172.21.15.16:0/4135613113 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5fdc06c2e0 msgr2=0x7f5fdc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:55.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.890+0000 7f5ffee9b640 1 --2- 172.21.15.16:0/4135613113 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5fdc06c2e0 0x7f5fdc06e7a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80c9db0 tx=0x7f5fe4009190 comp rx=0 tx=0).stop 2024-09-19T07:52:55.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.890+0000 7f5ffee9b640 1 -- 172.21.15.16:0/4135613113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe80051c0 msgr2=0x7f5ff814e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:55.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.890+0000 7f5ffee9b640 1 --2- 172.21.15.16:0/4135613113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe80051c0 0x7f5ff814e910 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00f5f0 tx=0x7f5fec00f9f0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.891+0000 7f5ffee9b640 1 -- 172.21.15.16:0/4135613113 shutdown_connections 2024-09-19T07:52:55.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.891+0000 7f5ffee9b640 1 --2- 172.21.15.16:0/4135613113 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5fdc06c2e0 0x7f5fdc06e7a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.891+0000 7f5ffee9b640 1 --2- 172.21.15.16:0/4135613113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ff80b0d90 0x7f5ff80ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.891+0000 7f5ffee9b640 1 --2- 172.21.15.16:0/4135613113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ff815a2c0 0x7f5ff8159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.891+0000 7f5ffee9b640 1 --2- 172.21.15.16:0/4135613113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe80051c0 0x7f5ff814e910 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:55.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.891+0000 7f5ffee9b640 1 -- 172.21.15.16:0/4135613113 >> 172.21.15.16:0/4135613113 conn(0x7f5ff806bdf0 msgr2=0x7f5ff8071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:55.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.891+0000 7f5ffee9b640 1 -- 172.21.15.16:0/4135613113 shutdown_connections 2024-09-19T07:52:55.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:55.891+0000 7f5ffee9b640 1 -- 172.21.15.16:0/4135613113 wait complete. 2024-09-19T07:52:55.902 INFO:tasks.cephfs_test_runner:When a client has inodes pinned in its cache (open files), that the MDS ... ok 2024-09-19T07:52:55.902 INFO:tasks.cephfs_test_runner:Starting test: test_client_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) When a client does not advance its oldest tid, the MDS should notice that 2024-09-19T07:52:55.903 INFO:tasks.cephfs_test_runner:test_client_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-19T07:52:55.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:52:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.008+0000 7fd8215f6640 1 Processor -- start 2024-09-19T07:52:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.008+0000 7fd8215f6640 1 -- start start 2024-09-19T07:52:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.009+0000 7fd8215f6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd81c15e2a0 0x7fd81c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.009+0000 7fd8215f6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd81c15e680 0x7fd81c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.009+0000 7fd8215f6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd81c0d1880 con 0x7fd81c06d960 2024-09-19T07:52:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.009+0000 7fd8215f6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd81c0d1b60 con 0x7fd81c15e2a0 2024-09-19T07:52:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.009+0000 7fd8215f6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd81c0cf2d0 con 0x7fd81c15e680 2024-09-19T07:52:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.009+0000 7fd81affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd81c15e2a0 0x7fd81c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.009+0000 7fd81affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd81c15e2a0 0x7fd81c06d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36830/0 (socket says 172.21.15.16:36830) 2024-09-19T07:52:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.009+0000 7fd81affd640 1 -- 172.21.15.16:0/3363275863 learned_addr learned my addr 172.21.15.16:0/3363275863 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.009+0000 7fd81a7fc640 1 --2- 172.21.15.16:0/3363275863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd81c15e680 0x7fd81c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.009+0000 7fd81affd640 1 -- 172.21.15.16:0/3363275863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd81c15e680 msgr2=0x7fd81c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.009+0000 7fd81affd640 1 --2- 172.21.15.16:0/3363275863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd81c15e680 0x7fd81c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.009+0000 7fd81affd640 1 -- 172.21.15.16:0/3363275863 >> v1:172.21.15.146:6789/0 conn(0x7fd81c06d960 legacy=0x7fd81c0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.009+0000 7fd81affd640 1 -- 172.21.15.16:0/3363275863 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81c0cf5b0 con 0x7fd81c15e2a0 2024-09-19T07:52:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.009+0000 7fd81a7fc640 1 --2- 172.21.15.16:0/3363275863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd81c15e680 0x7fd81c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:52:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.010+0000 7fd81affd640 1 --2- 172.21.15.16:0/3363275863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd81c15e2a0 0x7fd81c06d420 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009870 tx=0x7fd80c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6527c4e1173fbc0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.010+0000 7fd819ffb640 1 -- 172.21.15.16:0/3363275863 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c037070 con 0x7fd81c15e2a0 2024-09-19T07:52:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.010+0000 7fd819ffb640 1 -- 172.21.15.16:0/3363275863 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd80c032440 con 0x7fd81c15e2a0 2024-09-19T07:52:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.010+0000 7fd819ffb640 1 -- 172.21.15.16:0/3363275863 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c032810 con 0x7fd81c15e2a0 2024-09-19T07:52:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.011+0000 7fd8215f6640 1 -- 172.21.15.16:0/3363275863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd81c15e2a0 msgr2=0x7fd81c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.011+0000 7fd8215f6640 1 --2- 172.21.15.16:0/3363275863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd81c15e2a0 0x7fd81c06d420 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009870 tx=0x7fd80c02a200 comp rx=0 tx=0).stop 2024-09-19T07:52:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.011+0000 7fd8215f6640 1 -- 172.21.15.16:0/3363275863 shutdown_connections 2024-09-19T07:52:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.011+0000 7fd8215f6640 1 --2- 172.21.15.16:0/3363275863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd81c15e680 0x7fd81c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.011+0000 7fd8215f6640 1 --2- 172.21.15.16:0/3363275863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd81c15e2a0 0x7fd81c06d420 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.011+0000 7fd8215f6640 1 -- 172.21.15.16:0/3363275863 >> 172.21.15.16:0/3363275863 conn(0x7fd81c09d760 msgr2=0x7fd81c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.011+0000 7fd8215f6640 1 -- 172.21.15.16:0/3363275863 shutdown_connections 2024-09-19T07:52:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.011+0000 7fd8215f6640 1 -- 172.21.15.16:0/3363275863 wait complete. 2024-09-19T07:52:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.012+0000 7fd8215f6640 1 Processor -- start 2024-09-19T07:52:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.012+0000 7fd8215f6640 1 -- start start 2024-09-19T07:52:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.013+0000 7fd8215f6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd81c06d960 0x7fd81c0cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.013+0000 7fd8215f6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd81c15e2a0 0x7fd81c0cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.013+0000 7fd8215f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd81c15e680 0x7fd81c1f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.013+0000 7fd8215f6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c0d4c00 con 0x7fd81c15e680 2024-09-19T07:52:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.013+0000 7fd8215f6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c0d4a80 con 0x7fd81c06d960 2024-09-19T07:52:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.013+0000 7fd8215f6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd81c0d0010 con 0x7fd81c15e2a0 2024-09-19T07:52:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.013+0000 7fd81b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd81c15e680 0x7fd81c1f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.013+0000 7fd81b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd81c15e680 0x7fd81c1f48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36832/0 (socket says 172.21.15.16:36832) 2024-09-19T07:52:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.013+0000 7fd81b7fe640 1 -- 172.21.15.16:0/1773987857 learned_addr learned my addr 172.21.15.16:0/1773987857 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.013+0000 7fd81a7fc640 1 --2- 172.21.15.16:0/1773987857 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd81c15e2a0 0x7fd81c0cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.013+0000 7fd81affd640 1 --2- 172.21.15.16:0/1773987857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd81c06d960 0x7fd81c0cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.013+0000 7fd81b7fe640 1 -- 172.21.15.16:0/1773987857 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd81c15e2a0 msgr2=0x7fd81c0cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.013+0000 7fd81b7fe640 1 --2- 172.21.15.16:0/1773987857 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd81c15e2a0 0x7fd81c0cb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.013+0000 7fd81b7fe640 1 -- 172.21.15.16:0/1773987857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd81c06d960 msgr2=0x7fd81c0cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.013+0000 7fd81b7fe640 1 --2- 172.21.15.16:0/1773987857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd81c06d960 0x7fd81c0cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.013+0000 7fd81b7fe640 1 -- 172.21.15.16:0/1773987857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81c0ce210 con 0x7fd81c15e680 2024-09-19T07:52:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.014+0000 7fd81b7fe640 1 --2- 172.21.15.16:0/1773987857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd81c15e680 0x7fd81c1f48b0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fd81400c780 tx=0x7fd81400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.014+0000 7fd8037fe640 1 -- 172.21.15.16:0/1773987857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd814004b60 con 0x7fd81c15e680 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.014+0000 7fd8037fe640 1 -- 172.21.15.16:0/1773987857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd814004d00 con 0x7fd81c15e680 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.014+0000 7fd8215f6640 1 -- 172.21.15.16:0/1773987857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81c0ce4f0 con 0x7fd81c15e680 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.014+0000 7fd8037fe640 1 -- 172.21.15.16:0/1773987857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81400aa30 con 0x7fd81c15e680 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.014+0000 7fd8215f6640 1 -- 172.21.15.16:0/1773987857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd81c1e8e00 con 0x7fd81c15e680 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.015+0000 7fd8215f6640 1 -- 172.21.15.16:0/1773987857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd81c06d420 con 0x7fd81c15e680 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.015+0000 7fd8037fe640 1 -- 172.21.15.16:0/1773987857 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd81400abd0 con 0x7fd81c15e680 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.016+0000 7fd8037fe640 1 --2- 172.21.15.16:0/1773987857 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd80806c2e0 0x7fd80806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.016+0000 7fd8037fe640 1 -- 172.21.15.16:0/1773987857 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5335+0+0 (secure 0 0 0) 0x7fd81408b2e0 con 0x7fd81c15e680 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.016+0000 7fd81affd640 1 --2- 172.21.15.16:0/1773987857 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd80806c2e0 0x7fd80806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.017+0000 7fd81affd640 1 --2- 172.21.15.16:0/1773987857 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd80806c2e0 0x7fd80806e7a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd80c034000 tx=0x7fd80c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:56.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.020+0000 7fd8037fe640 1 -- 172.21.15.16:0/1773987857 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd814020070 con 0x7fd81c15e680 2024-09-19T07:52:56.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.196+0000 7fd8215f6640 1 -- 172.21.15.16:0/1773987857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd81c06cf90 con 0x7fd81c15e680 2024-09-19T07:52:56.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.197+0000 7fd8037fe640 1 -- 172.21.15.16:0/1773987857 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+13330 (secure 0 0 0) 0x7fd814059600 con 0x7fd81c15e680 2024-09-19T07:52:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:52:56.198 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":98,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:52:53.882081+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/1305244492":"2024-09-20T07:52:50.873199+0000","172.21.15.16:6833/1305244492":"2024-09-20T07:52:50.873199+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:52:56.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.199+0000 7fd8215f6640 1 -- 172.21.15.16:0/1773987857 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd80806c2e0 msgr2=0x7fd80806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.199+0000 7fd8215f6640 1 --2- 172.21.15.16:0/1773987857 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd80806c2e0 0x7fd80806e7a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd80c034000 tx=0x7fd80c02e000 comp rx=0 tx=0).stop 2024-09-19T07:52:56.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.199+0000 7fd8215f6640 1 -- 172.21.15.16:0/1773987857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd81c15e680 msgr2=0x7fd81c1f48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.199+0000 7fd8215f6640 1 --2- 172.21.15.16:0/1773987857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd81c15e680 0x7fd81c1f48b0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fd81400c780 tx=0x7fd81400cc40 comp rx=0 tx=0).stop 2024-09-19T07:52:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.199+0000 7fd8215f6640 1 -- 172.21.15.16:0/1773987857 shutdown_connections 2024-09-19T07:52:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.199+0000 7fd8215f6640 1 --2- 172.21.15.16:0/1773987857 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd80806c2e0 0x7fd80806e7a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.199+0000 7fd8215f6640 1 --2- 172.21.15.16:0/1773987857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd81c15e680 0x7fd81c1f48b0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.199+0000 7fd8215f6640 1 --2- 172.21.15.16:0/1773987857 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd81c15e2a0 0x7fd81c0cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.199+0000 7fd8215f6640 1 --2- 172.21.15.16:0/1773987857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd81c06d960 0x7fd81c0cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.199+0000 7fd8215f6640 1 -- 172.21.15.16:0/1773987857 >> 172.21.15.16:0/1773987857 conn(0x7fd81c09d760 msgr2=0x7fd81c15f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.200+0000 7fd8215f6640 1 -- 172.21.15.16:0/1773987857 shutdown_connections 2024-09-19T07:52:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.200+0000 7fd8215f6640 1 -- 172.21.15.16:0/1773987857 wait complete. 2024-09-19T07:52:56.211 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T07:42:54.500338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T07:52:56.211 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T07:52:56.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.322+0000 7f41d5827640 1 Processor -- start 2024-09-19T07:52:56.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.322+0000 7f41d5827640 1 -- start start 2024-09-19T07:52:56.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.323+0000 7f41d5827640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f41d015b120 0x7f41d006f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.323+0000 7f41d5827640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41d00d0bc0 0x7f41d006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.323+0000 7f41d5827640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d015b500 con 0x7f41d006dec0 2024-09-19T07:52:56.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.323+0000 7f41d5827640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f41d006e940 con 0x7f41d015b120 2024-09-19T07:52:56.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.323+0000 7f41d5827640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f41d006ec20 con 0x7f41d00d0bc0 2024-09-19T07:52:56.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.323+0000 7f41ceffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f41d015b120 0x7f41d006f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.323+0000 7f41ceffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f41d015b120 0x7f41d006f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36842/0 (socket says 172.21.15.16:36842) 2024-09-19T07:52:56.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.323+0000 7f41ceffd640 1 -- 172.21.15.16:0/782347123 learned_addr learned my addr 172.21.15.16:0/782347123 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:56.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.323+0000 7f41cf7fe640 1 --2- 172.21.15.16:0/782347123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41d00d0bc0 0x7f41d006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.323+0000 7f41ceffd640 1 -- 172.21.15.16:0/782347123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41d00d0bc0 msgr2=0x7f41d006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:56.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.323+0000 7f41ceffd640 1 --2- 172.21.15.16:0/782347123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41d00d0bc0 0x7f41d006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.323+0000 7f41ceffd640 1 -- 172.21.15.16:0/782347123 >> v1:172.21.15.146:6789/0 conn(0x7f41d006dec0 legacy=0x7f41d00d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.323+0000 7f41ceffd640 1 -- 172.21.15.16:0/782347123 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d006ef00 con 0x7f41d015b120 2024-09-19T07:52:56.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.323+0000 7f41cf7fe640 1 --2- 172.21.15.16:0/782347123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41d00d0bc0 0x7f41d006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:52:56.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.324+0000 7f41ceffd640 1 --2- 172.21.15.16:0/782347123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41d015b120 0x7f41d006f9e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009870 tx=0x7f41c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f51cd6849d6af0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:56.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.324+0000 7f41cdffb640 1 -- 172.21.15.16:0/782347123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0037070 con 0x7f41d015b120 2024-09-19T07:52:56.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.324+0000 7f41cdffb640 1 -- 172.21.15.16:0/782347123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c0032440 con 0x7f41d015b120 2024-09-19T07:52:56.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.324+0000 7f41cdffb640 1 -- 172.21.15.16:0/782347123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0032810 con 0x7f41d015b120 2024-09-19T07:52:56.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.325+0000 7f41d5827640 1 -- 172.21.15.16:0/782347123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41d015b120 msgr2=0x7f41d006f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.325+0000 7f41d5827640 1 --2- 172.21.15.16:0/782347123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41d015b120 0x7f41d006f9e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009870 tx=0x7f41c002a270 comp rx=0 tx=0).stop 2024-09-19T07:52:56.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.325+0000 7f41d5827640 1 -- 172.21.15.16:0/782347123 shutdown_connections 2024-09-19T07:52:56.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.325+0000 7f41d5827640 1 --2- 172.21.15.16:0/782347123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41d00d0bc0 0x7f41d006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.325+0000 7f41d5827640 1 --2- 172.21.15.16:0/782347123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41d015b120 0x7f41d006f9e0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.325+0000 7f41d5827640 1 -- 172.21.15.16:0/782347123 >> 172.21.15.16:0/782347123 conn(0x7f41d006bdf0 msgr2=0x7f41d006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:56.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.325+0000 7f41d5827640 1 -- 172.21.15.16:0/782347123 shutdown_connections 2024-09-19T07:52:56.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.325+0000 7f41d5827640 1 -- 172.21.15.16:0/782347123 wait complete. 2024-09-19T07:52:56.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.326+0000 7f41d5827640 1 Processor -- start 2024-09-19T07:52:56.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.326+0000 7f41d5827640 1 -- start start 2024-09-19T07:52:56.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.327+0000 7f41d5827640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41d006dec0 0x7f41d014e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.327+0000 7f41d5827640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41d00d0bc0 0x7f41d0071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.327+0000 7f41d5827640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41d015b120 0x7f41d0072380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.327+0000 7f41d5827640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41d00ce060 con 0x7f41d015b120 2024-09-19T07:52:56.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.327+0000 7f41d5827640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f41d006f4c0 con 0x7f41d006dec0 2024-09-19T07:52:56.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.327+0000 7f41d5827640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f41d00c5160 con 0x7f41d00d0bc0 2024-09-19T07:52:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.327+0000 7f41cf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41d015b120 0x7f41d0072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.327+0000 7f41cf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41d015b120 0x7f41d0072380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36844/0 (socket says 172.21.15.16:36844) 2024-09-19T07:52:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.327+0000 7f41cf7fe640 1 -- 172.21.15.16:0/782323534 learned_addr learned my addr 172.21.15.16:0/782323534 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.327+0000 7f41ceffd640 1 --2- 172.21.15.16:0/782323534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41d006dec0 0x7f41d014e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.327+0000 7f41ce7fc640 1 --2- 172.21.15.16:0/782323534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41d00d0bc0 0x7f41d0071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.328+0000 7f41cf7fe640 1 -- 172.21.15.16:0/782323534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41d00d0bc0 msgr2=0x7f41d0071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.328+0000 7f41cf7fe640 1 --2- 172.21.15.16:0/782323534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41d00d0bc0 0x7f41d0071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.328+0000 7f41cf7fe640 1 -- 172.21.15.16:0/782323534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41d006dec0 msgr2=0x7f41d014e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.328+0000 7f41cf7fe640 1 --2- 172.21.15.16:0/782323534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41d006dec0 0x7f41d014e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.328+0000 7f41cf7fe640 1 -- 172.21.15.16:0/782323534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d015e040 con 0x7f41d015b120 2024-09-19T07:52:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.328+0000 7f41ceffd640 1 --2- 172.21.15.16:0/782323534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41d006dec0 0x7f41d014e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:52:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.328+0000 7f41cf7fe640 1 --2- 172.21.15.16:0/782323534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41d015b120 0x7f41d0072380 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f41c8007ec0 tx=0x7f41c800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:56.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.328+0000 7f41b7fff640 1 -- 172.21.15.16:0/782323534 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c801a070 con 0x7f41d015b120 2024-09-19T07:52:56.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.328+0000 7f41b7fff640 1 -- 172.21.15.16:0/782323534 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c8015600 con 0x7f41d015b120 2024-09-19T07:52:56.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.328+0000 7f41d5827640 1 -- 172.21.15.16:0/782323534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d015e320 con 0x7f41d015b120 2024-09-19T07:52:56.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.328+0000 7f41b7fff640 1 -- 172.21.15.16:0/782323534 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c80159f0 con 0x7f41d015b120 2024-09-19T07:52:56.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.328+0000 7f41d5827640 1 -- 172.21.15.16:0/782323534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d0073260 con 0x7f41d015b120 2024-09-19T07:52:56.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.329+0000 7f41d5827640 1 -- 172.21.15.16:0/782323534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d006e330 con 0x7f41d015b120 2024-09-19T07:52:56.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.329+0000 7f41b7fff640 1 -- 172.21.15.16:0/782323534 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f41c801d070 con 0x7f41d015b120 2024-09-19T07:52:56.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.330+0000 7f41b7fff640 1 --2- 172.21.15.16:0/782323534 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f41bc06c2b0 0x7f41bc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.330+0000 7f41b7fff640 1 -- 172.21.15.16:0/782323534 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5335+0+0 (secure 0 0 0) 0x7f41c808b5c0 con 0x7f41d015b120 2024-09-19T07:52:56.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.330+0000 7f41ceffd640 1 --2- 172.21.15.16:0/782323534 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f41bc06c2b0 0x7f41bc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.331+0000 7f41ceffd640 1 --2- 172.21.15.16:0/782323534 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f41bc06c2b0 0x7f41bc06e770 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f41c0034000 tx=0x7f41c002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:56.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.334+0000 7f41b7fff640 1 -- 172.21.15.16:0/782323534 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f41c8010070 con 0x7f41d015b120 2024-09-19T07:52:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.513+0000 7f41d5827640 1 -- 172.21.15.16:0/782323534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f41d00cc750 con 0x7f41d015b120 2024-09-19T07:52:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.513+0000 7f41b7fff640 1 -- 172.21.15.16:0/782323534 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7f41c8059ea0 con 0x7f41d015b120 2024-09-19T07:52:56.513 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-19T07:52:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.515+0000 7f41d5827640 1 -- 172.21.15.16:0/782323534 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f41bc06c2b0 msgr2=0x7f41bc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.515+0000 7f41d5827640 1 --2- 172.21.15.16:0/782323534 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f41bc06c2b0 0x7f41bc06e770 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f41c0034000 tx=0x7f41c002e000 comp rx=0 tx=0).stop 2024-09-19T07:52:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.515+0000 7f41d5827640 1 -- 172.21.15.16:0/782323534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41d015b120 msgr2=0x7f41d0072380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.515+0000 7f41d5827640 1 --2- 172.21.15.16:0/782323534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41d015b120 0x7f41d0072380 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f41c8007ec0 tx=0x7f41c800d4a0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.515+0000 7f41d5827640 1 -- 172.21.15.16:0/782323534 shutdown_connections 2024-09-19T07:52:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.515+0000 7f41d5827640 1 --2- 172.21.15.16:0/782323534 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f41bc06c2b0 0x7f41bc06e770 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.515+0000 7f41d5827640 1 --2- 172.21.15.16:0/782323534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41d015b120 0x7f41d0072380 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.515+0000 7f41d5827640 1 --2- 172.21.15.16:0/782323534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41d00d0bc0 0x7f41d0071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.515+0000 7f41d5827640 1 --2- 172.21.15.16:0/782323534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41d006dec0 0x7f41d014e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.515+0000 7f41d5827640 1 -- 172.21.15.16:0/782323534 >> 172.21.15.16:0/782323534 conn(0x7f41d006bdf0 msgr2=0x7f41d00cbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.515+0000 7f41d5827640 1 -- 172.21.15.16:0/782323534 shutdown_connections 2024-09-19T07:52:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.515+0000 7f41d5827640 1 -- 172.21.15.16:0/782323534 wait complete. 2024-09-19T07:52:56.527 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid' 2024-09-19T07:52:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.629+0000 7f0aa395f640 1 Processor -- start 2024-09-19T07:52:56.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.629+0000 7f0aa395f640 1 -- start start 2024-09-19T07:52:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.630+0000 7f0aa395f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0a9c15f0d0 0x7f0a9c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.630+0000 7f0aa395f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0a9c0d0070 0x7f0a9c0d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.630+0000 7f0aa395f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0a9c0d1940 0x7f0a9c0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.630+0000 7f0aa395f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f0a9c0cf2b0 con 0x7f0a9c0d0070 2024-09-19T07:52:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.630+0000 7f0aa395f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f0a9c0cf590 con 0x7f0a9c15f0d0 2024-09-19T07:52:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.630+0000 7f0aa395f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0a9c0cf870 con 0x7f0a9c0d1940 2024-09-19T07:52:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.630+0000 7f0aa0efa640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0a9c0d0070 0x7f0a9c0d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.630+0000 7f0aa0efa640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0a9c0d0070 0x7f0a9c0d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36854/0 (socket says 172.21.15.16:36854) 2024-09-19T07:52:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.630+0000 7f0aa0efa640 1 -- 172.21.15.16:0/4065939804 learned_addr learned my addr 172.21.15.16:0/4065939804 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.630+0000 7f0aa1efc640 1 --2- 172.21.15.16:0/4065939804 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0a9c0d1940 0x7f0a9c0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.630+0000 7f0aa16fb640 1 --2- 172.21.15.16:0/4065939804 >> v2:172.21.15.146:3300/0 conn(0x7f0a9c15f0d0 0x7f0a9c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.631+0000 7f0aa0efa640 1 -- 172.21.15.16:0/4065939804 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0a9c0d1940 msgr2=0x7f0a9c0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.631+0000 7f0aa0efa640 1 --2- 172.21.15.16:0/4065939804 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0a9c0d1940 0x7f0a9c0d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.631+0000 7f0aa0efa640 1 -- 172.21.15.16:0/4065939804 >> v2:172.21.15.146:3300/0 conn(0x7f0a9c15f0d0 msgr2=0x7f0a9c15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.631+0000 7f0aa0efa640 1 --2- 172.21.15.16:0/4065939804 >> v2:172.21.15.146:3300/0 conn(0x7f0a9c15f0d0 0x7f0a9c15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.631+0000 7f0aa0efa640 1 -- 172.21.15.16:0/4065939804 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9c0cfb50 con 0x7f0a9c0d0070 2024-09-19T07:52:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.631+0000 7f0aa16fb640 1 --2- 172.21.15.16:0/4065939804 >> v2:172.21.15.146:3300/0 conn(0x7f0a9c15f0d0 0x7f0a9c15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:52:56.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.631+0000 7f0aa0efa640 1 --2- 172.21.15.16:0/4065939804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0d0070 0x7f0a9c0d1400 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0b6f40 tx=0x7f0a8c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb101f82b2cca679 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:56.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.632+0000 7f0a93fff640 1 -- 172.21.15.16:0/4065939804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c037070 con 0x7f0a9c0d0070 2024-09-19T07:52:56.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.632+0000 7f0a93fff640 1 -- 172.21.15.16:0/4065939804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a8c032440 con 0x7f0a9c0d0070 2024-09-19T07:52:56.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.632+0000 7f0a93fff640 1 -- 172.21.15.16:0/4065939804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c032810 con 0x7f0a9c0d0070 2024-09-19T07:52:56.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.632+0000 7f0aa395f640 1 -- 172.21.15.16:0/4065939804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0d0070 msgr2=0x7f0a9c0d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.632+0000 7f0aa395f640 1 --2- 172.21.15.16:0/4065939804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0d0070 0x7f0a9c0d1400 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0b6f40 tx=0x7f0a8c02a140 comp rx=0 tx=0).stop 2024-09-19T07:52:56.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.632+0000 7f0aa395f640 1 -- 172.21.15.16:0/4065939804 shutdown_connections 2024-09-19T07:52:56.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.632+0000 7f0aa395f640 1 --2- 172.21.15.16:0/4065939804 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0a9c0d1940 0x7f0a9c0d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.632+0000 7f0aa395f640 1 --2- 172.21.15.16:0/4065939804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0d0070 0x7f0a9c0d1400 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.632+0000 7f0aa395f640 1 --2- 172.21.15.16:0/4065939804 >> v2:172.21.15.146:3300/0 conn(0x7f0a9c15f0d0 0x7f0a9c15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.632+0000 7f0aa395f640 1 -- 172.21.15.16:0/4065939804 >> 172.21.15.16:0/4065939804 conn(0x7f0a9c09d760 msgr2=0x7f0a9c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:56.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.632+0000 7f0aa395f640 1 -- 172.21.15.16:0/4065939804 shutdown_connections 2024-09-19T07:52:56.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.632+0000 7f0aa395f640 1 -- 172.21.15.16:0/4065939804 wait complete. 2024-09-19T07:52:56.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.633+0000 7f0aa395f640 1 Processor -- start 2024-09-19T07:52:56.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.633+0000 7f0aa395f640 1 -- start start 2024-09-19T07:52:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.634+0000 7f0aa395f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0a9c0d0070 0x7f0a9c15b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.634+0000 7f0aa395f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0d1940 0x7f0a9c0c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.634+0000 7f0aa395f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a9c15f0d0 0x7f0a9c0ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.634+0000 7f0aa395f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c0d47f0 con 0x7f0a9c0d1940 2024-09-19T07:52:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.634+0000 7f0aa395f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c0d4670 con 0x7f0a9c15f0d0 2024-09-19T07:52:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.634+0000 7f0aa395f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0a9c0d4970 con 0x7f0a9c0d0070 2024-09-19T07:52:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.634+0000 7f0aa0efa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0d1940 0x7f0a9c0c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.634+0000 7f0aa0efa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0d1940 0x7f0a9c0c9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36866/0 (socket says 172.21.15.16:36866) 2024-09-19T07:52:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.634+0000 7f0aa0efa640 1 -- 172.21.15.16:0/1286598019 learned_addr learned my addr 172.21.15.16:0/1286598019 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.634+0000 7f0aa16fb640 1 --2- 172.21.15.16:0/1286598019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0a9c0d0070 0x7f0a9c15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.634+0000 7f0aa1efc640 1 --2- 172.21.15.16:0/1286598019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a9c15f0d0 0x7f0a9c0ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.635+0000 7f0aa0efa640 1 -- 172.21.15.16:0/1286598019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0a9c0d0070 msgr2=0x7f0a9c15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.635+0000 7f0aa0efa640 1 --2- 172.21.15.16:0/1286598019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0a9c0d0070 0x7f0a9c15b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.635+0000 7f0aa0efa640 1 -- 172.21.15.16:0/1286598019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a9c15f0d0 msgr2=0x7f0a9c0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.635+0000 7f0aa0efa640 1 --2- 172.21.15.16:0/1286598019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a9c15f0d0 0x7f0a9c0ca440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.635+0000 7f0aa0efa640 1 -- 172.21.15.16:0/1286598019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9c0cb4b0 con 0x7f0a9c0d1940 2024-09-19T07:52:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.635+0000 7f0aa0efa640 1 --2- 172.21.15.16:0/1286598019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0d1940 0x7f0a9c0c9f00 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c02a180 tx=0x7f0a8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.635+0000 7f0a91ffb640 1 -- 172.21.15.16:0/1286598019 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c037040 con 0x7f0a9c0d1940 2024-09-19T07:52:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.635+0000 7f0a91ffb640 1 -- 172.21.15.16:0/1286598019 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a8c042070 con 0x7f0a9c0d1940 2024-09-19T07:52:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.635+0000 7f0a91ffb640 1 -- 172.21.15.16:0/1286598019 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c030880 con 0x7f0a9c0d1940 2024-09-19T07:52:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.635+0000 7f0aa395f640 1 -- 172.21.15.16:0/1286598019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9c06dab0 con 0x7f0a9c0d1940 2024-09-19T07:52:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.635+0000 7f0aa395f640 1 -- 172.21.15.16:0/1286598019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a9c0b7280 con 0x7f0a9c0d1940 2024-09-19T07:52:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.636+0000 7f0aa395f640 1 -- 172.21.15.16:0/1286598019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a9c0d1400 con 0x7f0a9c0d1940 2024-09-19T07:52:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.637+0000 7f0a91ffb640 1 -- 172.21.15.16:0/1286598019 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0a8c030a20 con 0x7f0a9c0d1940 2024-09-19T07:52:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.637+0000 7f0a91ffb640 1 --2- 172.21.15.16:0/1286598019 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0a8006c2e0 0x7f0a8006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.637+0000 7f0a91ffb640 1 -- 172.21.15.16:0/1286598019 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5335+0+0 (secure 0 0 0) 0x7f0a8c0abb10 con 0x7f0a9c0d1940 2024-09-19T07:52:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.637+0000 7f0aa16fb640 1 --2- 172.21.15.16:0/1286598019 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0a8006c2e0 0x7f0a8006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.638+0000 7f0aa16fb640 1 --2- 172.21.15.16:0/1286598019 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0a8006c2e0 0x7f0a8006e7a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0c7d10 tx=0x7f0a98008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:56.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.641+0000 7f0a91ffb640 1 -- 172.21.15.16:0/1286598019 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f0a8c07a3f0 con 0x7f0a9c0d1940 2024-09-19T07:52:56.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.885+0000 7f0aa395f640 1 -- 172.21.15.16:0/1286598019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]} v 0) -- 0x7f0a9c0d3600 con 0x7f0a9c0d1940 2024-09-19T07:52:56.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.894+0000 7f0a91ffb640 1 -- 172.21.15.16:0/1286598019 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]}]=0 v428) ==== 153+0+0 (secure 0 0 0) 0x7f0a8c079cf0 con 0x7f0a9c0d1940 2024-09-19T07:52:56.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.896+0000 7f0aa395f640 1 -- 172.21.15.16:0/1286598019 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0a8006c2e0 msgr2=0x7f0a8006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.896+0000 7f0aa395f640 1 --2- 172.21.15.16:0/1286598019 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0a8006c2e0 0x7f0a8006e7a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0c7d10 tx=0x7f0a98008040 comp rx=0 tx=0).stop 2024-09-19T07:52:56.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.896+0000 7f0aa395f640 1 -- 172.21.15.16:0/1286598019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0d1940 msgr2=0x7f0a9c0c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.896+0000 7f0aa395f640 1 --2- 172.21.15.16:0/1286598019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0d1940 0x7f0a9c0c9f00 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c02a180 tx=0x7f0a8c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.896+0000 7f0aa395f640 1 -- 172.21.15.16:0/1286598019 shutdown_connections 2024-09-19T07:52:56.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.896+0000 7f0aa395f640 1 --2- 172.21.15.16:0/1286598019 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0a8006c2e0 0x7f0a8006e7a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.896+0000 7f0aa395f640 1 --2- 172.21.15.16:0/1286598019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a9c15f0d0 0x7f0a9c0ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.896+0000 7f0aa395f640 1 --2- 172.21.15.16:0/1286598019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a9c0d1940 0x7f0a9c0c9f00 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.896+0000 7f0aa395f640 1 --2- 172.21.15.16:0/1286598019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0a9c0d0070 0x7f0a9c15b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.896+0000 7f0aa395f640 1 -- 172.21.15.16:0/1286598019 >> 172.21.15.16:0/1286598019 conn(0x7f0a9c09d760 msgr2=0x7f0a9c15d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:56.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.896+0000 7f0aa395f640 1 -- 172.21.15.16:0/1286598019 shutdown_connections 2024-09-19T07:52:56.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:56.896+0000 7f0aa395f640 1 -- 172.21.15.16:0/1286598019 wait complete. 2024-09-19T07:52:56.907 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T07:52:57.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.012+0000 7f2acd70a640 1 Processor -- start 2024-09-19T07:52:57.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.012+0000 7f2acd70a640 1 -- start start 2024-09-19T07:52:57.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.012+0000 7f2acd70a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ac815bee0 0x7f2ac80d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.012+0000 7f2acd70a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2ac80cf910 0x7f2ac80d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.012+0000 7f2acd70a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ac80cdee0 con 0x7f2ac815c2c0 2024-09-19T07:52:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.012+0000 7f2acd70a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2ac80ceaf0 con 0x7f2ac815bee0 2024-09-19T07:52:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.012+0000 7f2acd70a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2ac80ce1c0 con 0x7f2ac80cf910 2024-09-19T07:52:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.013+0000 7f2ac6ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ac815bee0 0x7f2ac80d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.013+0000 7f2ac6ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ac815bee0 0x7f2ac80d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36878/0 (socket says 172.21.15.16:36878) 2024-09-19T07:52:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.013+0000 7f2ac6ffd640 1 -- 172.21.15.16:0/1297431460 learned_addr learned my addr 172.21.15.16:0/1297431460 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.013+0000 7f2ac77fe640 1 --2- 172.21.15.16:0/1297431460 >> v2:172.21.15.146:3300/0 conn(0x7f2ac80cf910 0x7f2ac80d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.013+0000 7f2ac6ffd640 1 -- 172.21.15.16:0/1297431460 >> v2:172.21.15.146:3300/0 conn(0x7f2ac80cf910 msgr2=0x7f2ac80d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.013+0000 7f2ac6ffd640 1 --2- 172.21.15.16:0/1297431460 >> v2:172.21.15.146:3300/0 conn(0x7f2ac80cf910 0x7f2ac80d09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.013+0000 7f2ac6ffd640 1 -- 172.21.15.16:0/1297431460 >> v1:172.21.15.146:6789/0 conn(0x7f2ac815c2c0 legacy=0x7f2ac80cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.013+0000 7f2ac6ffd640 1 -- 172.21.15.16:0/1297431460 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac80ce4a0 con 0x7f2ac815bee0 2024-09-19T07:52:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.013+0000 7f2ac77fe640 1 --2- 172.21.15.16:0/1297431460 >> v2:172.21.15.146:3300/0 conn(0x7f2ac80cf910 0x7f2ac80d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:52:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.014+0000 7f2ac6ffd640 1 --2- 172.21.15.16:0/1297431460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac815bee0 0x7f2ac80d04b0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f2ac80743d0 tx=0x7f2ab802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b500a714a7b29047 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.014+0000 7f2ac5ffb640 1 -- 172.21.15.16:0/1297431460 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8037070 con 0x7f2ac815bee0 2024-09-19T07:52:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.014+0000 7f2ac5ffb640 1 -- 172.21.15.16:0/1297431460 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab8032440 con 0x7f2ac815bee0 2024-09-19T07:52:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.014+0000 7f2ac5ffb640 1 -- 172.21.15.16:0/1297431460 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8032810 con 0x7f2ac815bee0 2024-09-19T07:52:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.014+0000 7f2acd70a640 1 -- 172.21.15.16:0/1297431460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac815bee0 msgr2=0x7f2ac80d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.014+0000 7f2acd70a640 1 --2- 172.21.15.16:0/1297431460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac815bee0 0x7f2ac80d04b0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f2ac80743d0 tx=0x7f2ab802a270 comp rx=0 tx=0).stop 2024-09-19T07:52:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.015+0000 7f2acd70a640 1 -- 172.21.15.16:0/1297431460 shutdown_connections 2024-09-19T07:52:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.015+0000 7f2acd70a640 1 --2- 172.21.15.16:0/1297431460 >> v2:172.21.15.146:3300/0 conn(0x7f2ac80cf910 0x7f2ac80d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.015+0000 7f2acd70a640 1 --2- 172.21.15.16:0/1297431460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac815bee0 0x7f2ac80d04b0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.015+0000 7f2acd70a640 1 -- 172.21.15.16:0/1297431460 >> 172.21.15.16:0/1297431460 conn(0x7f2ac806bdf0 msgr2=0x7f2ac806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.015+0000 7f2acd70a640 1 -- 172.21.15.16:0/1297431460 shutdown_connections 2024-09-19T07:52:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.015+0000 7f2acd70a640 1 -- 172.21.15.16:0/1297431460 wait complete. 2024-09-19T07:52:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.016+0000 7f2acd70a640 1 Processor -- start 2024-09-19T07:52:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.016+0000 7f2acd70a640 1 -- start start 2024-09-19T07:52:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.017+0000 7f2acd70a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ac80cf910 0x7f2ac80732f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.017+0000 7f2acd70a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ac815bee0 0x7f2ac80cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.017+0000 7f2acd70a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac815c2c0 0x7f2ac80ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.017+0000 7f2acd70a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac806e040 con 0x7f2ac815c2c0 2024-09-19T07:52:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.017+0000 7f2acd70a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac806dec0 con 0x7f2ac815bee0 2024-09-19T07:52:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.017+0000 7f2acd70a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac806e1c0 con 0x7f2ac80cf910 2024-09-19T07:52:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.017+0000 7f2ac77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac815c2c0 0x7f2ac80ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.017+0000 7f2ac77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac815c2c0 0x7f2ac80ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36886/0 (socket says 172.21.15.16:36886) 2024-09-19T07:52:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.017+0000 7f2ac77fe640 1 -- 172.21.15.16:0/1248215752 learned_addr learned my addr 172.21.15.16:0/1248215752 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.017+0000 7f2ac6ffd640 1 --2- 172.21.15.16:0/1248215752 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ac80cf910 0x7f2ac80732f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.017+0000 7f2ac67fc640 1 --2- 172.21.15.16:0/1248215752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ac815bee0 0x7f2ac80cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.017+0000 7f2ac77fe640 1 -- 172.21.15.16:0/1248215752 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ac80cf910 msgr2=0x7f2ac80732f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.017+0000 7f2ac77fe640 1 --2- 172.21.15.16:0/1248215752 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ac80cf910 0x7f2ac80732f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.017+0000 7f2ac77fe640 1 -- 172.21.15.16:0/1248215752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ac815bee0 msgr2=0x7f2ac80cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.017+0000 7f2ac77fe640 1 --2- 172.21.15.16:0/1248215752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ac815bee0 0x7f2ac80cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.017+0000 7f2ac77fe640 1 -- 172.21.15.16:0/1248215752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac80c5ff0 con 0x7f2ac815c2c0 2024-09-19T07:52:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.017+0000 7f2ac77fe640 1 --2- 172.21.15.16:0/1248215752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac815c2c0 0x7f2ac80ccd30 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0007ea0 tx=0x7f2ac0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.018+0000 7f2aaf7fe640 1 -- 172.21.15.16:0/1248215752 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac001a070 con 0x7f2ac815c2c0 2024-09-19T07:52:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.018+0000 7f2aaf7fe640 1 -- 172.21.15.16:0/1248215752 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac0003bb0 con 0x7f2ac815c2c0 2024-09-19T07:52:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.018+0000 7f2acd70a640 1 -- 172.21.15.16:0/1248215752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac80c62d0 con 0x7f2ac815c2c0 2024-09-19T07:52:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.018+0000 7f2aaf7fe640 1 -- 172.21.15.16:0/1248215752 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac000de70 con 0x7f2ac815c2c0 2024-09-19T07:52:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.018+0000 7f2acd70a640 1 -- 172.21.15.16:0/1248215752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac80c6890 con 0x7f2ac815c2c0 2024-09-19T07:52:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.019+0000 7f2acd70a640 1 -- 172.21.15.16:0/1248215752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac80b2d90 con 0x7f2ac815c2c0 2024-09-19T07:52:57.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.019+0000 7f2aaf7fe640 1 -- 172.21.15.16:0/1248215752 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2ac001d070 con 0x7f2ac815c2c0 2024-09-19T07:52:57.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.020+0000 7f2aaf7fe640 1 --2- 172.21.15.16:0/1248215752 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2ab406c2b0 0x7f2ab406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.020+0000 7f2aaf7fe640 1 -- 172.21.15.16:0/1248215752 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5335+0+0 (secure 0 0 0) 0x7f2ac008b4a0 con 0x7f2ac815c2c0 2024-09-19T07:52:57.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.020+0000 7f2ac6ffd640 1 --2- 172.21.15.16:0/1248215752 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2ab406c2b0 0x7f2ab406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.021+0000 7f2ac6ffd640 1 --2- 172.21.15.16:0/1248215752 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2ab406c2b0 0x7f2ab406e770 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8034000 tx=0x7f2ab802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:57.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.024+0000 7f2aaf7fe640 1 -- 172.21.15.16:0/1248215752 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2ac80b2d90 con 0x7f2ac815c2c0 2024-09-19T07:52:57.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.199+0000 7f2acd70a640 1 -- 172.21.15.16:0/1248215752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2ac80ccb10 con 0x7f2ac815c2c0 2024-09-19T07:52:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.203+0000 7f2aaf7fe640 1 -- 172.21.15.16:0/1248215752 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v41)=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7f2ac0010070 con 0x7f2ac815c2c0 2024-09-19T07:52:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.205+0000 7f2acd70a640 1 -- 172.21.15.16:0/1248215752 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2ab406c2b0 msgr2=0x7f2ab406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.205+0000 7f2acd70a640 1 --2- 172.21.15.16:0/1248215752 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2ab406c2b0 0x7f2ab406e770 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8034000 tx=0x7f2ab802e000 comp rx=0 tx=0).stop 2024-09-19T07:52:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.205+0000 7f2acd70a640 1 -- 172.21.15.16:0/1248215752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac815c2c0 msgr2=0x7f2ac80ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.205+0000 7f2acd70a640 1 --2- 172.21.15.16:0/1248215752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac815c2c0 0x7f2ac80ccd30 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0007ea0 tx=0x7f2ac0007f90 comp rx=0 tx=0).stop 2024-09-19T07:52:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.205+0000 7f2acd70a640 1 -- 172.21.15.16:0/1248215752 shutdown_connections 2024-09-19T07:52:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.205+0000 7f2acd70a640 1 --2- 172.21.15.16:0/1248215752 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2ab406c2b0 0x7f2ab406e770 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.205+0000 7f2acd70a640 1 --2- 172.21.15.16:0/1248215752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac815c2c0 0x7f2ac80ccd30 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.205+0000 7f2acd70a640 1 --2- 172.21.15.16:0/1248215752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ac815bee0 0x7f2ac80cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.205+0000 7f2acd70a640 1 --2- 172.21.15.16:0/1248215752 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ac80cf910 0x7f2ac80732f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.205+0000 7f2acd70a640 1 -- 172.21.15.16:0/1248215752 >> 172.21.15.16:0/1248215752 conn(0x7f2ac806bdf0 msgr2=0x7f2ac80b2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.205+0000 7f2acd70a640 1 -- 172.21.15.16:0/1248215752 shutdown_connections 2024-09-19T07:52:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.205+0000 7f2acd70a640 1 -- 172.21.15.16:0/1248215752 wait complete. 2024-09-19T07:52:57.216 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:52:57.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:52:57.257 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:52:57.308 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:52:57.308 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:52:57.308 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:52:57.324 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:52:57.324 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:52:57.324 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:52:57.342 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:52:57.343 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:52:57.343 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:52:57.380 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:52:57.380 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:52:57.380 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:52:57.400 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:52:57.400 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T07:52:57.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:52:57.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.522+0000 7f63e9798640 1 Processor -- start 2024-09-19T07:52:57.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.522+0000 7f63e9798640 1 -- start start 2024-09-19T07:52:57.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.523+0000 7f63e9798640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f63e415e2a0 0x7f63e406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.523+0000 7f63e9798640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63e415e680 0x7f63e40cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.523+0000 7f63e9798640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63e40d1880 con 0x7f63e406d960 2024-09-19T07:52:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.523+0000 7f63e9798640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f63e40d1b60 con 0x7f63e415e2a0 2024-09-19T07:52:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.523+0000 7f63e9798640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f63e40cf2d0 con 0x7f63e415e680 2024-09-19T07:52:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.523+0000 7f63e37fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f63e406d960 0x7f63e40d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51458/0 (socket says 172.21.15.16:51458) 2024-09-19T07:52:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.523+0000 7f63e37fe640 1 -- 172.21.15.16:0/1575253113 learned_addr learned my addr 172.21.15.16:0/1575253113 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:52:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.523+0000 7f63e27fc640 1 --2- 172.21.15.16:0/1575253113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63e415e680 0x7f63e40cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.523+0000 7f63e2ffd640 1 --2- 172.21.15.16:0/1575253113 >> v2:172.21.15.146:3300/0 conn(0x7f63e415e2a0 0x7f63e406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.523+0000 7f63e1ffb640 1 -- 172.21.15.16:0/1575253113 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1745752693 0 0) 0x7f63e40d1880 con 0x7f63e406d960 2024-09-19T07:52:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e1ffb640 1 -- 172.21.15.16:0/1575253113 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63d0003520 con 0x7f63e406d960 2024-09-19T07:52:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e1ffb640 1 -- 172.21.15.16:0/1575253113 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2147117203 0 0) 0x7f63d0003520 con 0x7f63e406d960 2024-09-19T07:52:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e1ffb640 1 -- 172.21.15.16:0/1575253113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63e415e680 msgr2=0x7f63e40cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e1ffb640 1 --2- 172.21.15.16:0/1575253113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63e415e680 0x7f63e40cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e1ffb640 1 -- 172.21.15.16:0/1575253113 >> v2:172.21.15.146:3300/0 conn(0x7f63e415e2a0 msgr2=0x7f63e406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e1ffb640 1 --2- 172.21.15.16:0/1575253113 >> v2:172.21.15.146:3300/0 conn(0x7f63e415e2a0 0x7f63e406d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e1ffb640 1 -- 172.21.15.16:0/1575253113 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e40cf5b0 con 0x7f63e406d960 2024-09-19T07:52:57.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e1ffb640 1 -- 172.21.15.16:0/1575253113 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f63dc001670 con 0x7f63e406d960 2024-09-19T07:52:57.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e1ffb640 1 -- 172.21.15.16:0/1575253113 >> v1:172.21.15.16:6789/0 conn(0x7f63e406d960 legacy=0x7f63e40d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:57.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e1ffb640 1 --2- 172.21.15.16:0/1575253113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d00039b0 0x7f63d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e1ffb640 1 --2- 172.21.15.16:0/1575253113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63d0004800 0x7f63d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e1ffb640 1 --2- 172.21.15.16:0/1575253113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63e415e680 0x7f63d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e1ffb640 1 -- 172.21.15.16:0/1575253113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f63e40d1b60 con 0x7f63d00039b0 2024-09-19T07:52:57.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e1ffb640 1 -- 172.21.15.16:0/1575253113 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f63e40cf2d0 con 0x7f63d0004800 2024-09-19T07:52:57.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e1ffb640 1 -- 172.21.15.16:0/1575253113 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f63d000b9c0 con 0x7f63e415e680 2024-09-19T07:52:57.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e2ffd640 1 --2- 172.21.15.16:0/1575253113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d00039b0 0x7f63d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e27fc640 1 --2- 172.21.15.16:0/1575253113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63d0004800 0x7f63d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e37fe640 1 --2- 172.21.15.16:0/1575253113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63e415e680 0x7f63d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e2ffd640 1 -- 172.21.15.16:0/1575253113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63e415e680 msgr2=0x7f63d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:57.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e2ffd640 1 --2- 172.21.15.16:0/1575253113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63e415e680 0x7f63d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e2ffd640 1 -- 172.21.15.16:0/1575253113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63d0004800 msgr2=0x7f63d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:57.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e2ffd640 1 --2- 172.21.15.16:0/1575253113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63d0004800 0x7f63d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.524+0000 7f63e2ffd640 1 -- 172.21.15.16:0/1575253113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d0003520 con 0x7f63d00039b0 2024-09-19T07:52:57.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.525+0000 7f63e2ffd640 1 --2- 172.21.15.16:0/1575253113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d00039b0 0x7f63d0003dd0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009fc0 tx=0x7f63d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576a88c9c8cc48a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:57.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.525+0000 7f63e1ffb640 1 -- 172.21.15.16:0/1575253113 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d4009c10 con 0x7f63d00039b0 2024-09-19T07:52:57.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.525+0000 7f63e1ffb640 1 -- 172.21.15.16:0/1575253113 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63d4036070 con 0x7f63d00039b0 2024-09-19T07:52:57.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.525+0000 7f63e1ffb640 1 -- 172.21.15.16:0/1575253113 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d402d070 con 0x7f63d00039b0 2024-09-19T07:52:57.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.526+0000 7f63e9798640 1 -- 172.21.15.16:0/1575253113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d00039b0 msgr2=0x7f63d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:57.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.526+0000 7f63e9798640 1 --2- 172.21.15.16:0/1575253113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d00039b0 0x7f63d0003dd0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009fc0 tx=0x7f63d0005830 comp rx=0 tx=0).stop 2024-09-19T07:52:57.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.526+0000 7f63e37fe640 1 -- 172.21.15.16:0/1575253113 reap_dead start 2024-09-19T07:52:57.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.526+0000 7f63e9798640 1 -- 172.21.15.16:0/1575253113 shutdown_connections 2024-09-19T07:52:57.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.526+0000 7f63e9798640 1 -- 172.21.15.16:0/1575253113 >> 172.21.15.16:0/1575253113 conn(0x7f63e409d760 msgr2=0x7f63e408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:57.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.526+0000 7f63e9798640 1 -- 172.21.15.16:0/1575253113 shutdown_connections 2024-09-19T07:52:57.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.526+0000 7f63e9798640 1 -- 172.21.15.16:0/1575253113 wait complete. 2024-09-19T07:52:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.527+0000 7f63e9798640 1 Processor -- start 2024-09-19T07:52:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.527+0000 7f63e9798640 1 -- start start 2024-09-19T07:52:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.528+0000 7f63e9798640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63d00039b0 0x7f63e415c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.528+0000 7f63e9798640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63e415d4e0 0x7f63e41e7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.528+0000 7f63e9798640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63e415cf00 0x7f63e41e82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.528+0000 7f63e9798640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f63e40d4c00 con 0x7f63e415cf00 2024-09-19T07:52:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.528+0000 7f63e9798640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f63e40d4a80 con 0x7f63e415d4e0 2024-09-19T07:52:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.528+0000 7f63e9798640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f63e40cfad0 con 0x7f63d00039b0 2024-09-19T07:52:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.528+0000 7f63e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63e415cf00 0x7f63e41e82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.528+0000 7f63e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63e415cf00 0x7f63e41e82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36904/0 (socket says 172.21.15.16:36904) 2024-09-19T07:52:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.528+0000 7f63e37fe640 1 -- 172.21.15.16:0/1021951906 learned_addr learned my addr 172.21.15.16:0/1021951906 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.528+0000 7f63e2ffd640 1 --2- 172.21.15.16:0/1021951906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63d00039b0 0x7f63e415c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.528+0000 7f63e27fc640 1 --2- 172.21.15.16:0/1021951906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63e415d4e0 0x7f63e41e7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.528+0000 7f63e37fe640 1 -- 172.21.15.16:0/1021951906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63d00039b0 msgr2=0x7f63e415c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.528+0000 7f63e37fe640 1 --2- 172.21.15.16:0/1021951906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63d00039b0 0x7f63e415c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.528+0000 7f63e37fe640 1 -- 172.21.15.16:0/1021951906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63e415d4e0 msgr2=0x7f63e41e7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.528+0000 7f63e37fe640 1 --2- 172.21.15.16:0/1021951906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63e415d4e0 0x7f63e41e7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.529+0000 7f63e37fe640 1 -- 172.21.15.16:0/1021951906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e41e8cc0 con 0x7f63e415cf00 2024-09-19T07:52:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.529+0000 7f63e37fe640 1 --2- 172.21.15.16:0/1021951906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63e415cf00 0x7f63e41e82f0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f63e406eda0 tx=0x7f63dc007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.529+0000 7f63db7fe640 1 -- 172.21.15.16:0/1021951906 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc001640 con 0x7f63e415cf00 2024-09-19T07:52:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.529+0000 7f63db7fe640 1 -- 172.21.15.16:0/1021951906 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63dc018070 con 0x7f63e415cf00 2024-09-19T07:52:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.529+0000 7f63e9798640 1 -- 172.21.15.16:0/1021951906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e41e8fa0 con 0x7f63e415cf00 2024-09-19T07:52:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.529+0000 7f63db7fe640 1 -- 172.21.15.16:0/1021951906 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc00faf0 con 0x7f63e415cf00 2024-09-19T07:52:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.529+0000 7f63e9798640 1 -- 172.21.15.16:0/1021951906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e40ccb20 con 0x7f63e415cf00 2024-09-19T07:52:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.530+0000 7f63e9798640 1 -- 172.21.15.16:0/1021951906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e4074b80 con 0x7f63e415cf00 2024-09-19T07:52:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.530+0000 7f63db7fe640 1 -- 172.21.15.16:0/1021951906 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f63dc00fc90 con 0x7f63e415cf00 2024-09-19T07:52:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.531+0000 7f63db7fe640 1 --2- 172.21.15.16:0/1021951906 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f63c406c2e0 0x7f63c406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.531+0000 7f63db7fe640 1 -- 172.21.15.16:0/1021951906 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5335+0+0 (secure 0 0 0) 0x7f63dc08b0e0 con 0x7f63e415cf00 2024-09-19T07:52:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.531+0000 7f63e2ffd640 1 --2- 172.21.15.16:0/1021951906 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f63c406c2e0 0x7f63c406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.532+0000 7f63e2ffd640 1 --2- 172.21.15.16:0/1021951906 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f63c406c2e0 0x7f63c406e7a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f63d4035000 tx=0x7f63d402f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:57.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.535+0000 7f63db7fe640 1 -- 172.21.15.16:0/1021951906 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f63dc059370 con 0x7f63e415cf00 2024-09-19T07:52:57.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.746+0000 7f63e9798640 1 -- 172.21.15.16:0/1021951906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63e406d420 con 0x7f63e415cf00 2024-09-19T07:52:57.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.747+0000 7f63db7fe640 1 -- 172.21.15.16:0/1021951906 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+3917 (secure 0 0 0) 0x7f63dc05d670 con 0x7f63e415cf00 2024-09-19T07:52:57.747 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:52:57.747 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":51,"btime":"2024-09-19T07:52:51:874738+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5684,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1865971154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1865971154},{"type":"v1","addr":"172.21.15.146:6837","nonce":1865971154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6094,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/2735152611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2735152611},{"type":"v1","addr":"172.21.15.146:6836","nonce":2735152611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6399,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4204683210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4204683210},{"type":"v1","addr":"172.21.15.16:6834","nonce":4204683210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6894,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2942722084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2942722084},{"type":"v1","addr":"172.21.15.16:6835","nonce":2942722084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[]} 2024-09-19T07:52:57.747 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 51 2024-09-19T07:52:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.748+0000 7f63e9798640 1 -- 172.21.15.16:0/1021951906 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f63c406c2e0 msgr2=0x7f63c406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.748+0000 7f63e9798640 1 --2- 172.21.15.16:0/1021951906 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f63c406c2e0 0x7f63c406e7a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f63d4035000 tx=0x7f63d402f000 comp rx=0 tx=0).stop 2024-09-19T07:52:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.748+0000 7f63e9798640 1 -- 172.21.15.16:0/1021951906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63e415cf00 msgr2=0x7f63e41e82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.748+0000 7f63e9798640 1 --2- 172.21.15.16:0/1021951906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63e415cf00 0x7f63e41e82f0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f63e406eda0 tx=0x7f63dc007720 comp rx=0 tx=0).stop 2024-09-19T07:52:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.749+0000 7f63e9798640 1 -- 172.21.15.16:0/1021951906 shutdown_connections 2024-09-19T07:52:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.749+0000 7f63e9798640 1 --2- 172.21.15.16:0/1021951906 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f63c406c2e0 0x7f63c406e7a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.749+0000 7f63e9798640 1 --2- 172.21.15.16:0/1021951906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63e415cf00 0x7f63e41e82f0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.749+0000 7f63e9798640 1 --2- 172.21.15.16:0/1021951906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63e415d4e0 0x7f63e41e7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.749+0000 7f63e9798640 1 --2- 172.21.15.16:0/1021951906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63d00039b0 0x7f63e415c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.749+0000 7f63e9798640 1 -- 172.21.15.16:0/1021951906 >> 172.21.15.16:0/1021951906 conn(0x7f63e409d760 msgr2=0x7f63e40d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.749+0000 7f63e9798640 1 -- 172.21.15.16:0/1021951906 shutdown_connections 2024-09-19T07:52:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.749+0000 7f63e9798640 1 -- 172.21.15.16:0/1021951906 wait complete. 2024-09-19T07:52:57.760 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T07:52:57.760 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T07:52:57.761 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T07:52:57.761 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:52:57.761 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T07:52:57.761 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T07:52:57.761 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T07:52:57.761 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:52:57.761 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T07:52:57.761 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T07:52:57.762 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T07:52:57.762 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:52:57.762 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T07:52:57.762 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T07:52:57.762 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T07:52:57.762 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:52:57.770 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T07:52:57.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T07:52:57.770 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T07:52:57.771 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T07:52:57.774 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T07:52:57.774 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T07:52:57.775 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T07:52:57.775 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T07:52:57.812 INFO:tasks.ceph.mds.a:Started 2024-09-19T07:52:57.812 DEBUG:teuthology.parallel:result is None 2024-09-19T07:52:57.813 INFO:tasks.ceph.mds.c:Started 2024-09-19T07:52:57.813 DEBUG:teuthology.parallel:result is None 2024-09-19T07:52:57.816 INFO:tasks.ceph.mds.d:Started 2024-09-19T07:52:57.816 DEBUG:teuthology.parallel:result is None 2024-09-19T07:52:57.817 INFO:tasks.ceph.mds.b:Started 2024-09-19T07:52:57.817 DEBUG:teuthology.parallel:result is None 2024-09-19T07:52:57.817 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T07:52:57.889 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-19T07:52:57.889 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-19T07:52:57.902 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2024-09-19T07:52:57.903 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2024-09-19T07:52:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.927+0000 7fddbb20e640 1 Processor -- start 2024-09-19T07:52:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.927+0000 7fddbb20e640 1 -- start start 2024-09-19T07:52:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.927+0000 7fddbb20e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fddb40d0070 0x7fddb415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.927+0000 7fddbb20e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddb40d4130 0x7fddb40d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.928+0000 7fddbb20e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddb40d4ae0 con 0x7fddb415f0d0 2024-09-19T07:52:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.928+0000 7fddbb20e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fddb40d1b10 con 0x7fddb40d0070 2024-09-19T07:52:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.928+0000 7fddbb20e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fddb40cf2b0 con 0x7fddb40d4130 2024-09-19T07:52:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.928+0000 7fddb3fff640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fddb40d0070 0x7fddb415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.928+0000 7fddb97ab640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddb40d4130 0x7fddb40d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.928+0000 7fddb97ab640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddb40d4130 0x7fddb40d45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:51810/0 (socket says 172.21.15.16:51810) 2024-09-19T07:52:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.928+0000 7fddb97ab640 1 -- 172.21.15.16:0/3949224475 learned_addr learned my addr 172.21.15.16:0/3949224475 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.928+0000 7fddb8faa640 1 --1- 172.21.15.16:0/3949224475 >> v1:172.21.15.146:6789/0 conn(0x7fddb415f0d0 0x7fddb40d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:57714/0 (socket says 172.21.15.16:57714) 2024-09-19T07:52:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.928+0000 7fddb97ab640 1 -- 172.21.15.16:0/3949224475 >> v2:172.21.15.146:3300/0 conn(0x7fddb40d0070 msgr2=0x7fddb415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.928+0000 7fddb97ab640 1 --2- 172.21.15.16:0/3949224475 >> v2:172.21.15.146:3300/0 conn(0x7fddb40d0070 0x7fddb415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.928+0000 7fddb97ab640 1 -- 172.21.15.16:0/3949224475 >> v1:172.21.15.146:6789/0 conn(0x7fddb415f0d0 legacy=0x7fddb40d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.928+0000 7fddb97ab640 1 -- 172.21.15.16:0/3949224475 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb40cf590 con 0x7fddb40d4130 2024-09-19T07:52:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.929+0000 7fddb97ab640 1 --2- 172.21.15.16:0/3949224475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddb40d4130 0x7fddb40d45a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fddac009e50 tx=0x7fddac02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=960204a5e4f8af66 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.929+0000 7fddb37fe640 1 -- 172.21.15.16:0/3949224475 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac036070 con 0x7fddb40d4130 2024-09-19T07:52:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.929+0000 7fddb37fe640 1 -- 172.21.15.16:0/3949224475 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddac031600 con 0x7fddb40d4130 2024-09-19T07:52:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.929+0000 7fddb37fe640 1 -- 172.21.15.16:0/3949224475 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac0319d0 con 0x7fddb40d4130 2024-09-19T07:52:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.929+0000 7fddbb20e640 1 -- 172.21.15.16:0/3949224475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddb40d4130 msgr2=0x7fddb40d45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.929+0000 7fddbb20e640 1 --2- 172.21.15.16:0/3949224475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddb40d4130 0x7fddb40d45a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fddac009e50 tx=0x7fddac02a680 comp rx=0 tx=0).stop 2024-09-19T07:52:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.930+0000 7fddbb20e640 1 -- 172.21.15.16:0/3949224475 shutdown_connections 2024-09-19T07:52:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.930+0000 7fddbb20e640 1 --2- 172.21.15.16:0/3949224475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddb40d4130 0x7fddb40d45a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.930+0000 7fddbb20e640 1 --2- 172.21.15.16:0/3949224475 >> v2:172.21.15.146:3300/0 conn(0x7fddb40d0070 0x7fddb415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.930+0000 7fddbb20e640 1 -- 172.21.15.16:0/3949224475 >> 172.21.15.16:0/3949224475 conn(0x7fddb409d760 msgr2=0x7fddb408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.930+0000 7fddbb20e640 1 -- 172.21.15.16:0/3949224475 shutdown_connections 2024-09-19T07:52:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.930+0000 7fddbb20e640 1 -- 172.21.15.16:0/3949224475 wait complete. 2024-09-19T07:52:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.931+0000 7fddbb20e640 1 Processor -- start 2024-09-19T07:52:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.931+0000 7fddbb20e640 1 -- start start 2024-09-19T07:52:57.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.932+0000 7fddbb20e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddb40d0070 0x7fddb415b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.932+0000 7fddbb20e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddb40d4130 0x7fddb40b9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.932+0000 7fddbb20e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb415f0d0 0x7fddb41e81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.932+0000 7fddbb20e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fddb40cef60 con 0x7fddb415f0d0 2024-09-19T07:52:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.932+0000 7fddbb20e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fddb40cfb50 con 0x7fddb40d4130 2024-09-19T07:52:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.932+0000 7fddbb20e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fddb40b2f70 con 0x7fddb40d0070 2024-09-19T07:52:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.932+0000 7fddb97ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb415f0d0 0x7fddb41e81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.932+0000 7fddb97ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb415f0d0 0x7fddb41e81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36966/0 (socket says 172.21.15.16:36966) 2024-09-19T07:52:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.932+0000 7fddb97ab640 1 -- 172.21.15.16:0/1019551778 learned_addr learned my addr 172.21.15.16:0/1019551778 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.932+0000 7fddb8faa640 1 --2- 172.21.15.16:0/1019551778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddb40d0070 0x7fddb415b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.932+0000 7fddb3fff640 1 --2- 172.21.15.16:0/1019551778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddb40d4130 0x7fddb40b9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.932+0000 7fddb97ab640 1 -- 172.21.15.16:0/1019551778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddb40d0070 msgr2=0x7fddb415b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.932+0000 7fddb97ab640 1 --2- 172.21.15.16:0/1019551778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddb40d0070 0x7fddb415b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.932+0000 7fddb97ab640 1 -- 172.21.15.16:0/1019551778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddb40d4130 msgr2=0x7fddb40b9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.932+0000 7fddb97ab640 1 --2- 172.21.15.16:0/1019551778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddb40d4130 0x7fddb40b9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.932+0000 7fddb97ab640 1 -- 172.21.15.16:0/1019551778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb40b8070 con 0x7fddb415f0d0 2024-09-19T07:52:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.932+0000 7fddb97ab640 1 --2- 172.21.15.16:0/1019551778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb415f0d0 0x7fddb41e81f0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fddac009e50 tx=0x7fddac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.932+0000 7fddb17fa640 1 -- 172.21.15.16:0/1019551778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac036050 con 0x7fddb415f0d0 2024-09-19T07:52:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.932+0000 7fddb17fa640 1 -- 172.21.15.16:0/1019551778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddac042070 con 0x7fddb415f0d0 2024-09-19T07:52:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.933+0000 7fddb17fa640 1 -- 172.21.15.16:0/1019551778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac009510 con 0x7fddb415f0d0 2024-09-19T07:52:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.933+0000 7fddbb20e640 1 -- 172.21.15.16:0/1019551778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb40b8350 con 0x7fddb415f0d0 2024-09-19T07:52:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.933+0000 7fddbb20e640 1 -- 172.21.15.16:0/1019551778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb40b8bf0 con 0x7fddb415f0d0 2024-09-19T07:52:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.934+0000 7fddbb20e640 1 -- 172.21.15.16:0/1019551778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb40d45a0 con 0x7fddb415f0d0 2024-09-19T07:52:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.934+0000 7fddb17fa640 1 -- 172.21.15.16:0/1019551778 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fddac007bf0 con 0x7fddb415f0d0 2024-09-19T07:52:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.934+0000 7fddb17fa640 1 --2- 172.21.15.16:0/1019551778 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdda006c2b0 0x7fdda006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.935+0000 7fddb17fa640 1 -- 172.21.15.16:0/1019551778 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5335+0+0 (secure 0 0 0) 0x7fddac0ac070 con 0x7fddb415f0d0 2024-09-19T07:52:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.935+0000 7fddb8faa640 1 --2- 172.21.15.16:0/1019551778 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdda006c2b0 0x7fdda006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.935+0000 7fddb8faa640 1 --2- 172.21.15.16:0/1019551778 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdda006c2b0 0x7fdda006e770 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdda4007fd0 tx=0x7fdda4007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:57.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:57.939+0000 7fddb17fa640 1 -- 172.21.15.16:0/1019551778 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fddac07a1f0 con 0x7fddb415f0d0 2024-09-19T07:52:58.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:58.129+0000 7fddbb20e640 1 -- 172.21.15.16:0/1019551778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fddb406cf30 con 0x7fddb415f0d0 2024-09-19T07:52:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:58.900+0000 7fddb17fa640 1 -- 172.21.15.16:0/1019551778 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v102) ==== 97+0+0 (secure 0 0 0) 0x7fddac07e4f0 con 0x7fddb415f0d0 2024-09-19T07:52:58.901 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-19T07:52:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:58.903+0000 7fddbb20e640 1 -- 172.21.15.16:0/1019551778 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdda006c2b0 msgr2=0x7fdda006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:58.903+0000 7fddbb20e640 1 --2- 172.21.15.16:0/1019551778 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdda006c2b0 0x7fdda006e770 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdda4007fd0 tx=0x7fdda4007e80 comp rx=0 tx=0).stop 2024-09-19T07:52:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:58.903+0000 7fddbb20e640 1 -- 172.21.15.16:0/1019551778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb415f0d0 msgr2=0x7fddb41e81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:58.903+0000 7fddbb20e640 1 --2- 172.21.15.16:0/1019551778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb415f0d0 0x7fddb41e81f0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fddac009e50 tx=0x7fddac004a40 comp rx=0 tx=0).stop 2024-09-19T07:52:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:58.903+0000 7fddbb20e640 1 -- 172.21.15.16:0/1019551778 shutdown_connections 2024-09-19T07:52:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:58.903+0000 7fddbb20e640 1 --2- 172.21.15.16:0/1019551778 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdda006c2b0 0x7fdda006e770 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:58.903+0000 7fddbb20e640 1 --2- 172.21.15.16:0/1019551778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb415f0d0 0x7fddb41e81f0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:58.903+0000 7fddbb20e640 1 --2- 172.21.15.16:0/1019551778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddb40d4130 0x7fddb40b9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:58.903+0000 7fddbb20e640 1 --2- 172.21.15.16:0/1019551778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddb40d0070 0x7fddb415b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:58.903+0000 7fddbb20e640 1 -- 172.21.15.16:0/1019551778 >> 172.21.15.16:0/1019551778 conn(0x7fddb409d760 msgr2=0x7fddb415d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:58.903+0000 7fddbb20e640 1 -- 172.21.15.16:0/1019551778 shutdown_connections 2024-09-19T07:52:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:58.904+0000 7fddbb20e640 1 -- 172.21.15.16:0/1019551778 wait complete. 2024-09-19T07:52:58.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T07:52:59.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.027+0000 7fe4629cf640 1 Processor -- start 2024-09-19T07:52:59.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.027+0000 7fe4629cf640 1 -- start start 2024-09-19T07:52:59.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.028+0000 7fe4629cf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe45c0d0bc0 0x7fe45c0cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.028+0000 7fe4629cf640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe45c15c2c0 0x7fe45c0cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.028+0000 7fe4629cf640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe45c0cfc80 con 0x7fe45c15bee0 2024-09-19T07:52:59.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.028+0000 7fe4629cf640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe45c0ceaf0 con 0x7fe45c15c2c0 2024-09-19T07:52:59.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.028+0000 7fe4629cf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe45c0cdee0 con 0x7fe45c0d0bc0 2024-09-19T07:52:59.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.028+0000 7fe45b7fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe45c0d0bc0 0x7fe45c0cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.028+0000 7fe460f6c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe45c15c2c0 0x7fe45c0cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.028+0000 7fe45b7fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe45c0d0bc0 0x7fe45c0cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:51820/0 (socket says 172.21.15.16:51820) 2024-09-19T07:52:59.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.028+0000 7fe45b7fe640 1 -- 172.21.15.16:0/2365574964 learned_addr learned my addr 172.21.15.16:0/2365574964 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:59.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.028+0000 7fe45bfff640 1 --1- 172.21.15.16:0/2365574964 >> v1:172.21.15.146:6789/0 conn(0x7fe45c15bee0 0x7fe45c0d04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:57722/0 (socket says 172.21.15.16:57722) 2024-09-19T07:52:59.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.029+0000 7fe460f6c640 1 -- 172.21.15.16:0/2365574964 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe45c0d0bc0 msgr2=0x7fe45c0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:59.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.029+0000 7fe460f6c640 1 --2- 172.21.15.16:0/2365574964 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe45c0d0bc0 0x7fe45c0cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.029+0000 7fe460f6c640 1 -- 172.21.15.16:0/2365574964 >> v1:172.21.15.146:6789/0 conn(0x7fe45c15bee0 legacy=0x7fe45c0d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:59.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.029+0000 7fe460f6c640 1 -- 172.21.15.16:0/2365574964 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45c0ce1c0 con 0x7fe45c15c2c0 2024-09-19T07:52:59.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.029+0000 7fe45b7fe640 1 --2- 172.21.15.16:0/2365574964 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe45c0d0bc0 0x7fe45c0cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:52:59.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.029+0000 7fe460f6c640 1 --2- 172.21.15.16:0/2365574964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe45c15c2c0 0x7fe45c0cf740 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe454009d90 tx=0x7fe45402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bc625904c81e68a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:59.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.030+0000 7fe45affd640 1 -- 172.21.15.16:0/2365574964 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454007740 con 0x7fe45c15c2c0 2024-09-19T07:52:59.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.030+0000 7fe45affd640 1 -- 172.21.15.16:0/2365574964 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4540078e0 con 0x7fe45c15c2c0 2024-09-19T07:52:59.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.030+0000 7fe45affd640 1 -- 172.21.15.16:0/2365574964 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454007cb0 con 0x7fe45c15c2c0 2024-09-19T07:52:59.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.030+0000 7fe4629cf640 1 -- 172.21.15.16:0/2365574964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe45c15c2c0 msgr2=0x7fe45c0cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:59.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.030+0000 7fe4629cf640 1 --2- 172.21.15.16:0/2365574964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe45c15c2c0 0x7fe45c0cf740 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe454009d90 tx=0x7fe45402a660 comp rx=0 tx=0).stop 2024-09-19T07:52:59.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.030+0000 7fe4629cf640 1 -- 172.21.15.16:0/2365574964 shutdown_connections 2024-09-19T07:52:59.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.030+0000 7fe4629cf640 1 --2- 172.21.15.16:0/2365574964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe45c15c2c0 0x7fe45c0cf740 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.030+0000 7fe4629cf640 1 --2- 172.21.15.16:0/2365574964 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe45c0d0bc0 0x7fe45c0cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.030+0000 7fe4629cf640 1 -- 172.21.15.16:0/2365574964 >> 172.21.15.16:0/2365574964 conn(0x7fe45c06bdf0 msgr2=0x7fe45c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:59.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.030+0000 7fe4629cf640 1 -- 172.21.15.16:0/2365574964 shutdown_connections 2024-09-19T07:52:59.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.030+0000 7fe4629cf640 1 -- 172.21.15.16:0/2365574964 wait complete. 2024-09-19T07:52:59.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.031+0000 7fe4629cf640 1 Processor -- start 2024-09-19T07:52:59.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.032+0000 7fe4629cf640 1 -- start start 2024-09-19T07:52:59.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.032+0000 7fe4629cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe45c0d0bc0 0x7fe45c0c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.032+0000 7fe4629cf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe45c15bee0 0x7fe45c1f1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.032+0000 7fe4629cf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe45c15c2c0 0x7fe45c1f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.032+0000 7fe4629cf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c06dec0 con 0x7fe45c0d0bc0 2024-09-19T07:52:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.032+0000 7fe4629cf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c0ce780 con 0x7fe45c15c2c0 2024-09-19T07:52:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.032+0000 7fe4629cf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe45c06e040 con 0x7fe45c15bee0 2024-09-19T07:52:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.032+0000 7fe45bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe45c0d0bc0 0x7fe45c0c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.032+0000 7fe45bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe45c0d0bc0 0x7fe45c0c55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36976/0 (socket says 172.21.15.16:36976) 2024-09-19T07:52:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.033+0000 7fe45bfff640 1 -- 172.21.15.16:0/3399269400 learned_addr learned my addr 172.21.15.16:0/3399269400 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.033+0000 7fe460f6c640 1 --2- 172.21.15.16:0/3399269400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe45c15c2c0 0x7fe45c1f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.033+0000 7fe45b7fe640 1 --2- 172.21.15.16:0/3399269400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe45c15bee0 0x7fe45c1f1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.033+0000 7fe45bfff640 1 -- 172.21.15.16:0/3399269400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe45c15bee0 msgr2=0x7fe45c1f1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.033+0000 7fe45bfff640 1 --2- 172.21.15.16:0/3399269400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe45c15bee0 0x7fe45c1f1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.033+0000 7fe45bfff640 1 -- 172.21.15.16:0/3399269400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe45c15c2c0 msgr2=0x7fe45c1f3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.033+0000 7fe45bfff640 1 --2- 172.21.15.16:0/3399269400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe45c15c2c0 0x7fe45c1f3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.033+0000 7fe45bfff640 1 -- 172.21.15.16:0/3399269400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45c1f49d0 con 0x7fe45c0d0bc0 2024-09-19T07:52:59.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.033+0000 7fe45bfff640 1 --2- 172.21.15.16:0/3399269400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe45c0d0bc0 0x7fe45c0c55c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00c430 tx=0x7fe44c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:59.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.033+0000 7fe458ff9640 1 -- 172.21.15.16:0/3399269400 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c0074e0 con 0x7fe45c0d0bc0 2024-09-19T07:52:59.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.033+0000 7fe458ff9640 1 -- 172.21.15.16:0/3399269400 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe44c007680 con 0x7fe45c0d0bc0 2024-09-19T07:52:59.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.033+0000 7fe4629cf640 1 -- 172.21.15.16:0/3399269400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe45c1f4cb0 con 0x7fe45c0d0bc0 2024-09-19T07:52:59.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.033+0000 7fe458ff9640 1 -- 172.21.15.16:0/3399269400 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c004a00 con 0x7fe45c0d0bc0 2024-09-19T07:52:59.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.033+0000 7fe4629cf640 1 -- 172.21.15.16:0/3399269400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe45c0cbdf0 con 0x7fe45c0d0bc0 2024-09-19T07:52:59.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.034+0000 7fe4629cf640 1 -- 172.21.15.16:0/3399269400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe45c0d04b0 con 0x7fe45c0d0bc0 2024-09-19T07:52:59.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.035+0000 7fe458ff9640 1 -- 172.21.15.16:0/3399269400 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe44c01a020 con 0x7fe45c0d0bc0 2024-09-19T07:52:59.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.035+0000 7fe458ff9640 1 --2- 172.21.15.16:0/3399269400 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe44806c2b0 0x7fe44806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.035+0000 7fe458ff9640 1 -- 172.21.15.16:0/3399269400 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5249+0+0 (secure 0 0 0) 0x7fe44c08a960 con 0x7fe45c0d0bc0 2024-09-19T07:52:59.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.036+0000 7fe45b7fe640 1 --2- 172.21.15.16:0/3399269400 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe44806c2b0 0x7fe44806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.036+0000 7fe45b7fe640 1 --2- 172.21.15.16:0/3399269400 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe44806c2b0 0x7fe44806e770 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe45c15b730 tx=0x7fe450009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.040+0000 7fe458ff9640 1 -- 172.21.15.16:0/3399269400 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe44c058c50 con 0x7fe45c0d0bc0 2024-09-19T07:52:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.281+0000 7fe4629cf640 1 -- 172.21.15.16:0/3399269400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe45c15b480 con 0x7fe45c0d0bc0 2024-09-19T07:52:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.282+0000 7fe458ff9640 1 -- 172.21.15.16:0/3399269400 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fe44c05cf50 con 0x7fe45c0d0bc0 2024-09-19T07:52:59.282 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:52:59.282 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:52:59.282 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-19T07:52:59.283 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.283 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-19T07:52:59.283 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etms80KDxAAqTrG88LTEJIj9TWJu0witQ==", 2024-09-19T07:52:59.283 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:52:59.283 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:52:59.283 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:52:59.283 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:52:59.283 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:52:59.283 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:52:59.283 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.283 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-19T07:52:59.284 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etm4T0KGxAAcfDOKrSdKzhJ4xvTFIyxcQ==", 2024-09-19T07:52:59.284 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:52:59.284 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:52:59.284 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:52:59.284 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:52:59.284 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:52:59.284 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.284 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:52:59.284 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.284 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-19T07:52:59.284 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNDBGFhAAtBioJJ1+NtIDLhd/Hg7aZA==", 2024-09-19T07:52:59.284 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:52:59.284 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:52:59.285 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:52:59.285 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:52:59.285 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:52:59.285 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:52:59.285 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.285 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-19T07:52:59.285 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmaHEiIhAACqqjCRMV+xlGh9WqYsj+MA==", 2024-09-19T07:52:59.285 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:52:59.285 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:52:59.285 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:52:59.285 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:52:59.285 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:52:59.286 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:52:59.286 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.286 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-19T07:52:59.286 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDT1etm09plCxAAGvJYF6JrKw49FtXb71ox3A==", 2024-09-19T07:52:59.286 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:52:59.286 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:52:59.286 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:52:59.286 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:52:59.286 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:52:59.286 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.286 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-19T07:52:59.287 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDW1etmGy8yHRAAlcDsHLnkjtS6sCCR8Olv2A==", 2024-09-19T07:52:59.287 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:52:59.287 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:52:59.287 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:52:59.287 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:52:59.287 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:52:59.287 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.287 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-19T07:52:59.287 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDZ1etmMhGTHxAATQaW2kgJhHKftutQQtUa6A==", 2024-09-19T07:52:59.287 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:52:59.287 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:52:59.288 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:52:59.288 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:52:59.288 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:52:59.288 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.288 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-19T07:52:59.288 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDc1etmD3sLIhAAIdY00buvb3bIFA5ZpgxxjA==", 2024-09-19T07:52:59.288 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:52:59.288 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:52:59.288 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:52:59.288 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:52:59.288 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:52:59.288 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.289 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-19T07:52:59.289 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDi1etmgutcLRAAHJEl5SfQ7N0T4YoYeBfrQg==", 2024-09-19T07:52:59.289 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:52:59.289 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:52:59.289 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:52:59.289 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:52:59.289 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:52:59.289 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.289 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-19T07:52:59.289 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDm1etmCV3xARAA+5yhQEmIpyNId3RL1Xsxhw==", 2024-09-19T07:52:59.289 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:52:59.289 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:52:59.290 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:52:59.290 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:52:59.290 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:52:59.290 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.290 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-19T07:52:59.290 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDp1etm9XC+AxAA+Lrs0ExfSKaixgOxfX8Nug==", 2024-09-19T07:52:59.290 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:52:59.290 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:52:59.290 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:52:59.290 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:52:59.290 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:52:59.290 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.290 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-19T07:52:59.291 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDs1etmhmvMBhAAhvW/udIE2gCd8IrJCSFuvw==", 2024-09-19T07:52:59.291 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:52:59.291 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:52:59.291 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:52:59.291 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:52:59.291 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:52:59.291 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.291 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-19T07:52:59.291 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA==", 2024-09-19T07:52:59.291 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:52:59.291 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:52:59.292 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-19T07:52:59.292 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T07:52:59.292 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:52:59.292 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.292 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-19T07:52:59.292 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmc05KLxAAPQl6wOHqFIf8LVxskiuLVg==", 2024-09-19T07:52:59.292 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:52:59.292 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:52:59.292 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:52:59.292 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:52:59.292 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:52:59.292 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:52:59.293 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.293 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-19T07:52:59.293 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmjFsCLBAANa7gCMvDw84Ovo7u6RRJ3Q==", 2024-09-19T07:52:59.293 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:52:59.293 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:52:59.293 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:52:59.293 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:52:59.293 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:52:59.293 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:52:59.293 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.293 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-19T07:52:59.293 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmdUbgNBAAfwstyOeeoZfetgVZu4z5xg==", 2024-09-19T07:52:59.294 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:52:59.294 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:52:59.294 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:52:59.294 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:52:59.294 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:52:59.294 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:52:59.294 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.294 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-19T07:52:59.294 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDQ1etm9/vTLhAA9PRfAOA8R4oXp3x0dU8l3g==", 2024-09-19T07:52:59.294 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:52:59.294 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:52:59.295 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:52:59.295 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:52:59.295 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:52:59.295 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.295 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:52:59.295 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.295 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-19T07:52:59.295 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNQ68CxAANkK+D3u/oTEz8Su4iuZrCg==", 2024-09-19T07:52:59.295 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:52:59.295 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:52:59.295 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:52:59.295 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:52:59.295 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.295 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:52:59.296 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.296 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-19T07:52:59.296 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmp+gSCBAAGXzmKfFNd6K2fWeqYhlHAg==", 2024-09-19T07:52:59.296 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:52:59.296 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:52:59.296 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:52:59.296 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:52:59.296 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.296 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.296 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T07:52:59.296 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:52:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.284+0000 7fe4629cf640 1 -- 172.21.15.16:0/3399269400 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe44806c2b0 msgr2=0x7fe44806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.284+0000 7fe4629cf640 1 --2- 172.21.15.16:0/3399269400 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe44806c2b0 0x7fe44806e770 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe45c15b730 tx=0x7fe450009240 comp rx=0 tx=0).stop 2024-09-19T07:52:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.284+0000 7fe4629cf640 1 -- 172.21.15.16:0/3399269400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe45c0d0bc0 msgr2=0x7fe45c0c55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.284+0000 7fe4629cf640 1 --2- 172.21.15.16:0/3399269400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe45c0d0bc0 0x7fe45c0c55c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00c430 tx=0x7fe44c00c8f0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.284+0000 7fe4629cf640 1 -- 172.21.15.16:0/3399269400 shutdown_connections 2024-09-19T07:52:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.284+0000 7fe4629cf640 1 --2- 172.21.15.16:0/3399269400 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe44806c2b0 0x7fe44806e770 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.284+0000 7fe4629cf640 1 --2- 172.21.15.16:0/3399269400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe45c15c2c0 0x7fe45c1f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.284+0000 7fe4629cf640 1 --2- 172.21.15.16:0/3399269400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe45c15bee0 0x7fe45c1f1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.284+0000 7fe4629cf640 1 --2- 172.21.15.16:0/3399269400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe45c0d0bc0 0x7fe45c0c55c0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.284+0000 7fe4629cf640 1 -- 172.21.15.16:0/3399269400 >> 172.21.15.16:0/3399269400 conn(0x7fe45c06bdf0 msgr2=0x7fe45c0afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.284+0000 7fe4629cf640 1 -- 172.21.15.16:0/3399269400 shutdown_connections 2024-09-19T07:52:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.284+0000 7fe4629cf640 1 -- 172.21.15.16:0/3399269400 wait complete. 2024-09-19T07:52:59.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T07:52:59.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.401+0000 7f95fa047640 1 Processor -- start 2024-09-19T07:52:59.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.401+0000 7f95fa047640 1 -- start start 2024-09-19T07:52:59.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.402+0000 7f95fa047640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95f415e2a0 0x7f95f406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.402+0000 7f95fa047640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f95f406d960 0x7f95f40d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.402+0000 7f95fa047640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95f40d1880 con 0x7f95f415e680 2024-09-19T07:52:59.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.402+0000 7f95fa047640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f95f40d1b60 con 0x7f95f406d960 2024-09-19T07:52:59.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.402+0000 7f95fa047640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f95f40cf2d0 con 0x7f95f415e2a0 2024-09-19T07:52:59.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.402+0000 7f95f3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f95f406d960 0x7f95f40d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.402+0000 7f95f3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f95f406d960 0x7f95f40d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36978/0 (socket says 172.21.15.16:36978) 2024-09-19T07:52:59.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.402+0000 7f95f3fff640 1 -- 172.21.15.16:0/3144677398 learned_addr learned my addr 172.21.15.16:0/3144677398 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:59.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.402+0000 7f95f3fff640 1 -- 172.21.15.16:0/3144677398 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95f415e2a0 msgr2=0x7f95f406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:59.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.402+0000 7f95f3fff640 1 --2- 172.21.15.16:0/3144677398 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95f415e2a0 0x7f95f406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.402+0000 7f95f3fff640 1 -- 172.21.15.16:0/3144677398 >> v1:172.21.15.146:6789/0 conn(0x7f95f415e680 legacy=0x7f95f40cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:59.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.402+0000 7f95f3fff640 1 -- 172.21.15.16:0/3144677398 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f40cf5b0 con 0x7f95f406d960 2024-09-19T07:52:59.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.403+0000 7f95f3fff640 1 --2- 172.21.15.16:0/3144677398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95f406d960 0x7f95f40d4370 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009e50 tx=0x7f95ec02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76558239f691231 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:59.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.403+0000 7f95f27fc640 1 -- 172.21.15.16:0/3144677398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec036070 con 0x7f95f406d960 2024-09-19T07:52:59.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.403+0000 7f95f27fc640 1 -- 172.21.15.16:0/3144677398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95ec031600 con 0x7f95f406d960 2024-09-19T07:52:59.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.403+0000 7f95f27fc640 1 -- 172.21.15.16:0/3144677398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec0319d0 con 0x7f95f406d960 2024-09-19T07:52:59.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.404+0000 7f95fa047640 1 -- 172.21.15.16:0/3144677398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95f406d960 msgr2=0x7f95f40d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:59.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.404+0000 7f95fa047640 1 --2- 172.21.15.16:0/3144677398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95f406d960 0x7f95f40d4370 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009e50 tx=0x7f95ec02a6c0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.404+0000 7f95fa047640 1 -- 172.21.15.16:0/3144677398 shutdown_connections 2024-09-19T07:52:59.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.404+0000 7f95fa047640 1 --2- 172.21.15.16:0/3144677398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95f406d960 0x7f95f40d4370 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.404+0000 7f95fa047640 1 --2- 172.21.15.16:0/3144677398 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95f415e2a0 0x7f95f406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.404+0000 7f95fa047640 1 -- 172.21.15.16:0/3144677398 >> 172.21.15.16:0/3144677398 conn(0x7f95f409d760 msgr2=0x7f95f408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:59.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.404+0000 7f95fa047640 1 -- 172.21.15.16:0/3144677398 shutdown_connections 2024-09-19T07:52:59.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.404+0000 7f95fa047640 1 -- 172.21.15.16:0/3144677398 wait complete. 2024-09-19T07:52:59.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.405+0000 7f95fa047640 1 Processor -- start 2024-09-19T07:52:59.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.405+0000 7f95fa047640 1 -- start start 2024-09-19T07:52:59.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.406+0000 7f95fa047640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95f406d960 0x7f95f41f5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.406+0000 7f95fa047640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95f415e2a0 0x7f95f40c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.406+0000 7f95fa047640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95f415e680 0x7f95f41f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.406+0000 7f95fa047640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f95f40d0360 con 0x7f95f406d960 2024-09-19T07:52:59.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.406+0000 7f95fa047640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f95f40d01e0 con 0x7f95f415e680 2024-09-19T07:52:59.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.406+0000 7f95fa047640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f95f40d48b0 con 0x7f95f415e2a0 2024-09-19T07:52:59.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.406+0000 7f95f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95f406d960 0x7f95f41f5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.406+0000 7f95f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95f406d960 0x7f95f41f5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36988/0 (socket says 172.21.15.16:36988) 2024-09-19T07:52:59.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.406+0000 7f95f37fe640 1 -- 172.21.15.16:0/2536497965 learned_addr learned my addr 172.21.15.16:0/2536497965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:59.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.406+0000 7f95f2ffd640 1 --2- 172.21.15.16:0/2536497965 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95f415e2a0 0x7f95f40c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.406+0000 7f95f37fe640 1 -- 172.21.15.16:0/2536497965 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95f415e2a0 msgr2=0x7f95f40c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:59.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.406+0000 7f95f3fff640 1 --2- 172.21.15.16:0/2536497965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95f415e680 0x7f95f41f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.406+0000 7f95f37fe640 1 --2- 172.21.15.16:0/2536497965 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95f415e2a0 0x7f95f40c9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.406+0000 7f95f37fe640 1 -- 172.21.15.16:0/2536497965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95f415e680 msgr2=0x7f95f41f7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:59.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.407+0000 7f95f37fe640 1 --2- 172.21.15.16:0/2536497965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95f415e680 0x7f95f41f7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.407+0000 7f95f37fe640 1 -- 172.21.15.16:0/2536497965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f41f8850 con 0x7f95f406d960 2024-09-19T07:52:59.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.407+0000 7f95f37fe640 1 --2- 172.21.15.16:0/2536497965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95f406d960 0x7f95f41f5290 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f95e400b4a0 tx=0x7f95e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:59.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.407+0000 7f95dbfff640 1 -- 172.21.15.16:0/2536497965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4014070 con 0x7f95f406d960 2024-09-19T07:52:59.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.407+0000 7f95dbfff640 1 -- 172.21.15.16:0/2536497965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e4010610 con 0x7f95f406d960 2024-09-19T07:52:59.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.407+0000 7f95dbfff640 1 -- 172.21.15.16:0/2536497965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4010a00 con 0x7f95f406d960 2024-09-19T07:52:59.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.407+0000 7f95fa047640 1 -- 172.21.15.16:0/2536497965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f41f8b30 con 0x7f95f406d960 2024-09-19T07:52:59.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.407+0000 7f95fa047640 1 -- 172.21.15.16:0/2536497965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f41f57d0 con 0x7f95f406d960 2024-09-19T07:52:59.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.408+0000 7f95fa047640 1 -- 172.21.15.16:0/2536497965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f41f4880 con 0x7f95f406d960 2024-09-19T07:52:59.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.408+0000 7f95dbfff640 1 -- 172.21.15.16:0/2536497965 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f95e4017070 con 0x7f95f406d960 2024-09-19T07:52:59.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.409+0000 7f95dbfff640 1 --2- 172.21.15.16:0/2536497965 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f95e006c2b0 0x7f95e006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.409+0000 7f95dbfff640 1 -- 172.21.15.16:0/2536497965 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5249+0+0 (secure 0 0 0) 0x7f95e408ada0 con 0x7f95f406d960 2024-09-19T07:52:59.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.409+0000 7f95f2ffd640 1 --2- 172.21.15.16:0/2536497965 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f95e006c2b0 0x7f95e006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.410+0000 7f95f2ffd640 1 --2- 172.21.15.16:0/2536497965 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f95e006c2b0 0x7f95e006e770 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f95e8006fd0 tx=0x7f95e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:59.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.413+0000 7f95dbfff640 1 -- 172.21.15.16:0/2536497965 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f95e4059630 con 0x7f95f406d960 2024-09-19T07:52:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.589+0000 7f95fa047640 1 -- 172.21.15.16:0/2536497965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f95f40d4370 con 0x7f95f406d960 2024-09-19T07:52:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.590+0000 7f95dbfff640 1 -- 172.21.15.16:0/2536497965 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v102) ==== 84+0+65 (secure 0 0 0) 0x7f95e4058f30 con 0x7f95f406d960 2024-09-19T07:52:59.590 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:52:59.590 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-19T07:52:59.590 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:52:59.591 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-19T07:52:59.591 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-19T07:52:59.591 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:52:59.591 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-19T07:52:59.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.592+0000 7f95fa047640 1 -- 172.21.15.16:0/2536497965 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f95e006c2b0 msgr2=0x7f95e006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:59.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.592+0000 7f95fa047640 1 --2- 172.21.15.16:0/2536497965 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f95e006c2b0 0x7f95e006e770 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f95e8006fd0 tx=0x7f95e8008040 comp rx=0 tx=0).stop 2024-09-19T07:52:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.592+0000 7f95fa047640 1 -- 172.21.15.16:0/2536497965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95f406d960 msgr2=0x7f95f41f5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.592+0000 7f95fa047640 1 --2- 172.21.15.16:0/2536497965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95f406d960 0x7f95f41f5290 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f95e400b4a0 tx=0x7f95e400b960 comp rx=0 tx=0).stop 2024-09-19T07:52:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.592+0000 7f95fa047640 1 -- 172.21.15.16:0/2536497965 shutdown_connections 2024-09-19T07:52:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.592+0000 7f95fa047640 1 --2- 172.21.15.16:0/2536497965 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f95e006c2b0 0x7f95e006e770 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.592+0000 7f95fa047640 1 --2- 172.21.15.16:0/2536497965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95f415e680 0x7f95f41f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.592+0000 7f95fa047640 1 --2- 172.21.15.16:0/2536497965 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95f415e2a0 0x7f95f40c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.592+0000 7f95fa047640 1 --2- 172.21.15.16:0/2536497965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95f406d960 0x7f95f41f5290 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.592+0000 7f95fa047640 1 -- 172.21.15.16:0/2536497965 >> 172.21.15.16:0/2536497965 conn(0x7f95f409d760 msgr2=0x7f95f415fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.592+0000 7f95fa047640 1 -- 172.21.15.16:0/2536497965 shutdown_connections 2024-09-19T07:52:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.592+0000 7f95fa047640 1 -- 172.21.15.16:0/2536497965 wait complete. 2024-09-19T07:52:59.603 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T07:52:59.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T07:52:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.710+0000 7f98fe7bd640 1 Processor -- start 2024-09-19T07:52:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.710+0000 7f98fe7bd640 1 -- start start 2024-09-19T07:52:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98fe7bd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f98f80d0200 0x7f98f80d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98fe7bd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98f80adeb0 0x7f98f80ae320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98fe7bd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98f806eb70 con 0x7f98f815e2b0 2024-09-19T07:52:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98fe7bd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f98f80d0bb0 con 0x7f98f80d0200 2024-09-19T07:52:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98fe7bd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f98f806ee50 con 0x7f98f80adeb0 2024-09-19T07:52:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98f7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f98f815e2b0 0x7f98f806e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51482/0 (socket says 172.21.15.16:51482) 2024-09-19T07:52:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98f7fff640 1 -- 172.21.15.16:0/3750683349 learned_addr learned my addr 172.21.15.16:0/3750683349 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:52:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98f6ffd640 1 -- 172.21.15.16:0/3750683349 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3245026439 0 0) 0x7f98f806eb70 con 0x7f98f815e2b0 2024-09-19T07:52:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98fcd5a640 1 --2- 172.21.15.16:0/3750683349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98f80adeb0 0x7f98f80ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98f6ffd640 1 -- 172.21.15.16:0/3750683349 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98e0003520 con 0x7f98f815e2b0 2024-09-19T07:52:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98f77fe640 1 --2- 172.21.15.16:0/3750683349 >> v2:172.21.15.146:3300/0 conn(0x7f98f80d0200 0x7f98f80d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98f6ffd640 1 -- 172.21.15.16:0/3750683349 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2680534984 0 0) 0x7f98e0003520 con 0x7f98f815e2b0 2024-09-19T07:52:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98f6ffd640 1 -- 172.21.15.16:0/3750683349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98f80adeb0 msgr2=0x7f98f80ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98f6ffd640 1 --2- 172.21.15.16:0/3750683349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98f80adeb0 0x7f98f80ae320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98f6ffd640 1 -- 172.21.15.16:0/3750683349 >> v2:172.21.15.146:3300/0 conn(0x7f98f80d0200 msgr2=0x7f98f80d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98f6ffd640 1 --2- 172.21.15.16:0/3750683349 >> v2:172.21.15.146:3300/0 conn(0x7f98f80d0200 0x7f98f80d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98f6ffd640 1 -- 172.21.15.16:0/3750683349 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f806f130 con 0x7f98f815e2b0 2024-09-19T07:52:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98f6ffd640 1 -- 172.21.15.16:0/3750683349 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f98e8001670 con 0x7f98f815e2b0 2024-09-19T07:52:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98f6ffd640 1 -- 172.21.15.16:0/3750683349 >> v1:172.21.15.16:6789/0 conn(0x7f98f815e2b0 legacy=0x7f98f806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98f6ffd640 1 --2- 172.21.15.16:0/3750683349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98e00039b0 0x7f98e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98f6ffd640 1 --2- 172.21.15.16:0/3750683349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0004800 0x7f98e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98f6ffd640 1 --2- 172.21.15.16:0/3750683349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98f80adeb0 0x7f98e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98f6ffd640 1 -- 172.21.15.16:0/3750683349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98f80d0bb0 con 0x7f98e0004800 2024-09-19T07:52:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98f6ffd640 1 -- 172.21.15.16:0/3750683349 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f98f806ee50 con 0x7f98e00039b0 2024-09-19T07:52:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.711+0000 7f98f6ffd640 1 -- 172.21.15.16:0/3750683349 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f98e000b9c0 con 0x7f98f80adeb0 2024-09-19T07:52:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.712+0000 7f98f77fe640 1 --2- 172.21.15.16:0/3750683349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0004800 0x7f98e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.712+0000 7f98f7fff640 1 --2- 172.21.15.16:0/3750683349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98e00039b0 0x7f98e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.712+0000 7f98fcd5a640 1 --2- 172.21.15.16:0/3750683349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98f80adeb0 0x7f98e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.712+0000 7f98f77fe640 1 -- 172.21.15.16:0/3750683349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98f80adeb0 msgr2=0x7f98e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.712+0000 7f98f77fe640 1 --2- 172.21.15.16:0/3750683349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98f80adeb0 0x7f98e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.712+0000 7f98f77fe640 1 -- 172.21.15.16:0/3750683349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98e00039b0 msgr2=0x7f98e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.712+0000 7f98f77fe640 1 --2- 172.21.15.16:0/3750683349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98e00039b0 0x7f98e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.712+0000 7f98f77fe640 1 -- 172.21.15.16:0/3750683349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e0003520 con 0x7f98e0004800 2024-09-19T07:52:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.712+0000 7f98f7fff640 1 --2- 172.21.15.16:0/3750683349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98e00039b0 0x7f98e0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:52:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.713+0000 7f98f77fe640 1 --2- 172.21.15.16:0/3750683349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0004800 0x7f98e0004c80 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009ad0 tx=0x7f98e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe7942039bfe276 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.713+0000 7f98f6ffd640 1 -- 172.21.15.16:0/3750683349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec036070 con 0x7f98e0004800 2024-09-19T07:52:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.713+0000 7f98f6ffd640 1 -- 172.21.15.16:0/3750683349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ec031400 con 0x7f98e0004800 2024-09-19T07:52:59.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.713+0000 7f98f6ffd640 1 -- 172.21.15.16:0/3750683349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec009670 con 0x7f98e0004800 2024-09-19T07:52:59.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.713+0000 7f98fe7bd640 1 -- 172.21.15.16:0/3750683349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0004800 msgr2=0x7f98e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:59.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.713+0000 7f98fe7bd640 1 --2- 172.21.15.16:0/3750683349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0004800 0x7f98e0004c80 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009ad0 tx=0x7f98e0005830 comp rx=0 tx=0).stop 2024-09-19T07:52:59.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.713+0000 7f98fcd5a640 1 -- 172.21.15.16:0/3750683349 reap_dead start 2024-09-19T07:52:59.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.713+0000 7f98fe7bd640 1 -- 172.21.15.16:0/3750683349 shutdown_connections 2024-09-19T07:52:59.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.713+0000 7f98fe7bd640 1 -- 172.21.15.16:0/3750683349 >> 172.21.15.16:0/3750683349 conn(0x7f98f8067d70 msgr2=0x7f98f806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:59.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.713+0000 7f98fe7bd640 1 -- 172.21.15.16:0/3750683349 shutdown_connections 2024-09-19T07:52:59.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.713+0000 7f98fe7bd640 1 -- 172.21.15.16:0/3750683349 wait complete. 2024-09-19T07:52:59.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.714+0000 7f98fe7bd640 1 Processor -- start 2024-09-19T07:52:59.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.715+0000 7f98fe7bd640 1 -- start start 2024-09-19T07:52:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.715+0000 7f98fe7bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0004800 0x7f98f8146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.715+0000 7f98fe7bd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98f8159c90 0x7f98f815a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.715+0000 7f98fe7bd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98f8152110 0x7f98f81525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.715+0000 7f98fe7bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98f806e7c0 con 0x7f98e0004800 2024-09-19T07:52:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.715+0000 7f98fe7bd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f98f806c9a0 con 0x7f98f8159c90 2024-09-19T07:52:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.715+0000 7f98fe7bd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f98f8152b00 con 0x7f98f8152110 2024-09-19T07:52:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.716+0000 7f98f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0004800 0x7f98f8146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.716+0000 7f98f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0004800 0x7f98f8146f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37000/0 (socket says 172.21.15.16:37000) 2024-09-19T07:52:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.716+0000 7f98f7fff640 1 -- 172.21.15.16:0/846351950 learned_addr learned my addr 172.21.15.16:0/846351950 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:52:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.716+0000 7f98f77fe640 1 --2- 172.21.15.16:0/846351950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98f8159c90 0x7f98f815a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.716+0000 7f98fcd5a640 1 --2- 172.21.15.16:0/846351950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98f8152110 0x7f98f81525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.716+0000 7f98f7fff640 1 -- 172.21.15.16:0/846351950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98f8152110 msgr2=0x7f98f81525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.716+0000 7f98f7fff640 1 --2- 172.21.15.16:0/846351950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98f8152110 0x7f98f81525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.716+0000 7f98f7fff640 1 -- 172.21.15.16:0/846351950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98f8159c90 msgr2=0x7f98f815a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.716+0000 7f98f7fff640 1 --2- 172.21.15.16:0/846351950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98f8159c90 0x7f98f815a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.716+0000 7f98f7fff640 1 -- 172.21.15.16:0/846351950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f8152d90 con 0x7f98e0004800 2024-09-19T07:52:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.716+0000 7f98f7fff640 1 --2- 172.21.15.16:0/846351950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0004800 0x7f98f8146f20 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f98e8001d30 tx=0x7f98e8004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.716+0000 7f98f4ff9640 1 -- 172.21.15.16:0/846351950 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e800fa80 con 0x7f98e0004800 2024-09-19T07:52:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.716+0000 7f98f4ff9640 1 -- 172.21.15.16:0/846351950 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e8018070 con 0x7f98e0004800 2024-09-19T07:52:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.716+0000 7f98fe7bd640 1 -- 172.21.15.16:0/846351950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f8153070 con 0x7f98e0004800 2024-09-19T07:52:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.716+0000 7f98f4ff9640 1 -- 172.21.15.16:0/846351950 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e800fe40 con 0x7f98e0004800 2024-09-19T07:52:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.716+0000 7f98fe7bd640 1 -- 172.21.15.16:0/846351950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f8153630 con 0x7f98e0004800 2024-09-19T07:52:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.717+0000 7f98fe7bd640 1 -- 172.21.15.16:0/846351950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98f80ae610 con 0x7f98e0004800 2024-09-19T07:52:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.718+0000 7f98f4ff9640 1 -- 172.21.15.16:0/846351950 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f98e801d460 con 0x7f98e0004800 2024-09-19T07:52:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.718+0000 7f98f4ff9640 1 --2- 172.21.15.16:0/846351950 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f98e406c2b0 0x7f98e406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.718+0000 7f98f4ff9640 1 -- 172.21.15.16:0/846351950 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5249+0+0 (secure 0 0 0) 0x7f98e808b550 con 0x7f98e0004800 2024-09-19T07:52:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.718+0000 7f98f77fe640 1 --2- 172.21.15.16:0/846351950 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f98e406c2b0 0x7f98e406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.719+0000 7f98f77fe640 1 --2- 172.21.15.16:0/846351950 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f98e406c2b0 0x7f98e406e770 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f98ec033000 tx=0x7f98ec008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.722+0000 7f98f4ff9640 1 -- 172.21.15.16:0/846351950 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f98e8059f10 con 0x7f98e0004800 2024-09-19T07:52:59.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:52:59.898+0000 7f98fe7bd640 1 -- 172.21.15.16:0/846351950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f98f80ae330 con 0x7f98e0004800 2024-09-19T07:53:00.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:00.900+0000 7f98f4ff9640 1 -- 172.21.15.16:0/846351950 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v103) ==== 138+0+0 (secure 0 0 0) 0x7f98e8059810 con 0x7f98e0004800 2024-09-19T07:53:00.900 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-19T07:53:00.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:00.903+0000 7f98fe7bd640 1 -- 172.21.15.16:0/846351950 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f98e406c2b0 msgr2=0x7f98e406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:00.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:00.903+0000 7f98fe7bd640 1 --2- 172.21.15.16:0/846351950 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f98e406c2b0 0x7f98e406e770 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f98ec033000 tx=0x7f98ec008030 comp rx=0 tx=0).stop 2024-09-19T07:53:00.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:00.903+0000 7f98fe7bd640 1 -- 172.21.15.16:0/846351950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0004800 msgr2=0x7f98f8146f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:00.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:00.903+0000 7f98fe7bd640 1 --2- 172.21.15.16:0/846351950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0004800 0x7f98f8146f20 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f98e8001d30 tx=0x7f98e8004f90 comp rx=0 tx=0).stop 2024-09-19T07:53:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:00.904+0000 7f98fe7bd640 1 -- 172.21.15.16:0/846351950 shutdown_connections 2024-09-19T07:53:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:00.904+0000 7f98fe7bd640 1 --2- 172.21.15.16:0/846351950 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f98e406c2b0 0x7f98e406e770 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:00.904+0000 7f98fe7bd640 1 --2- 172.21.15.16:0/846351950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98f8152110 0x7f98f81525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:00.904+0000 7f98fe7bd640 1 --2- 172.21.15.16:0/846351950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98f8159c90 0x7f98f815a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:00.904+0000 7f98fe7bd640 1 --2- 172.21.15.16:0/846351950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0004800 0x7f98f8146f20 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:00.904+0000 7f98fe7bd640 1 -- 172.21.15.16:0/846351950 >> 172.21.15.16:0/846351950 conn(0x7f98f8067d70 msgr2=0x7f98f80b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:00.904+0000 7f98fe7bd640 1 -- 172.21.15.16:0/846351950 shutdown_connections 2024-09-19T07:53:00.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:00.904+0000 7f98fe7bd640 1 -- 172.21.15.16:0/846351950 wait complete. 2024-09-19T07:53:00.922 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T07:53:01.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.037+0000 7f2cf32d0640 1 Processor -- start 2024-09-19T07:53:01.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.037+0000 7f2cf32d0640 1 -- start start 2024-09-19T07:53:01.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.038+0000 7f2cf32d0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2cec161da0 0x7f2cec0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.038+0000 7f2cf32d0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2cec0b3b10 0x7f2cec0d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.038+0000 7f2cf32d0640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2cec0d2740 0x7f2cec0d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.038+0000 7f2cf32d0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2cec0cf0d0 con 0x7f2cec161da0 2024-09-19T07:53:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.038+0000 7f2cf32d0640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2cec0d2c40 con 0x7f2cec0d2740 2024-09-19T07:53:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.038+0000 7f2cf32d0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2cec0d2f20 con 0x7f2cec0b3b10 2024-09-19T07:53:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.038+0000 7f2cf106c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2cec161da0 0x7f2cec0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.038+0000 7f2cf106c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2cec161da0 0x7f2cec0b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37010/0 (socket says 172.21.15.16:37010) 2024-09-19T07:53:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.038+0000 7f2cf106c640 1 -- 172.21.15.16:0/3796152620 learned_addr learned my addr 172.21.15.16:0/3796152620 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.038+0000 7f2cf186d640 1 --2- 172.21.15.16:0/3796152620 >> v2:172.21.15.146:3300/0 conn(0x7f2cec0d2740 0x7f2cec0d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.038+0000 7f2cf086b640 1 --2- 172.21.15.16:0/3796152620 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2cec0b3b10 0x7f2cec0d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:01.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.038+0000 7f2cf106c640 1 -- 172.21.15.16:0/3796152620 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2cec0b3b10 msgr2=0x7f2cec0d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:01.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.038+0000 7f2cf106c640 1 --2- 172.21.15.16:0/3796152620 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2cec0b3b10 0x7f2cec0d0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.038+0000 7f2cf106c640 1 -- 172.21.15.16:0/3796152620 >> v2:172.21.15.146:3300/0 conn(0x7f2cec0d2740 msgr2=0x7f2cec0d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:01.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.038+0000 7f2cf106c640 1 --2- 172.21.15.16:0/3796152620 >> v2:172.21.15.146:3300/0 conn(0x7f2cec0d2740 0x7f2cec0d1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.038+0000 7f2cf106c640 1 -- 172.21.15.16:0/3796152620 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cec0d3200 con 0x7f2cec161da0 2024-09-19T07:53:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.039+0000 7f2cf106c640 1 --2- 172.21.15.16:0/3796152620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec161da0 0x7f2cec0b30e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2cec08d660 tx=0x7f2ce802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a970d44ab7423807 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.039+0000 7f2ce3fff640 1 -- 172.21.15.16:0/3796152620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8037070 con 0x7f2cec161da0 2024-09-19T07:53:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.039+0000 7f2ce3fff640 1 -- 172.21.15.16:0/3796152620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ce802ae00 con 0x7f2cec161da0 2024-09-19T07:53:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.039+0000 7f2ce3fff640 1 -- 172.21.15.16:0/3796152620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8032660 con 0x7f2cec161da0 2024-09-19T07:53:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.040+0000 7f2cf32d0640 1 -- 172.21.15.16:0/3796152620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec161da0 msgr2=0x7f2cec0b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.040+0000 7f2cf32d0640 1 --2- 172.21.15.16:0/3796152620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec161da0 0x7f2cec0b30e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2cec08d660 tx=0x7f2ce802a0b0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.040+0000 7f2cf32d0640 1 -- 172.21.15.16:0/3796152620 shutdown_connections 2024-09-19T07:53:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.040+0000 7f2cf32d0640 1 --2- 172.21.15.16:0/3796152620 >> v2:172.21.15.146:3300/0 conn(0x7f2cec0d2740 0x7f2cec0d1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.040+0000 7f2cf32d0640 1 --2- 172.21.15.16:0/3796152620 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2cec0b3b10 0x7f2cec0d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.040+0000 7f2cf32d0640 1 --2- 172.21.15.16:0/3796152620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec161da0 0x7f2cec0b30e0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.040+0000 7f2cf32d0640 1 -- 172.21.15.16:0/3796152620 >> 172.21.15.16:0/3796152620 conn(0x7f2cec08bf80 msgr2=0x7f2cec0d4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.040+0000 7f2cf32d0640 1 -- 172.21.15.16:0/3796152620 shutdown_connections 2024-09-19T07:53:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.040+0000 7f2cf32d0640 1 -- 172.21.15.16:0/3796152620 wait complete. 2024-09-19T07:53:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.041+0000 7f2cf32d0640 1 Processor -- start 2024-09-19T07:53:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.041+0000 7f2cf32d0640 1 -- start start 2024-09-19T07:53:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.042+0000 7f2cf32d0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2cec0b3b10 0x7f2cec156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.042+0000 7f2cf32d0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cec0d2740 0x7f2cec15b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.042+0000 7f2cf32d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec161da0 0x7f2cec15b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.042+0000 7f2cf32d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec0b3620 con 0x7f2cec161da0 2024-09-19T07:53:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.042+0000 7f2cf32d0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec0d34e0 con 0x7f2cec0d2740 2024-09-19T07:53:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.042+0000 7f2cf32d0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2cec0b37a0 con 0x7f2cec0b3b10 2024-09-19T07:53:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.042+0000 7f2cf186d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec161da0 0x7f2cec15b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.042+0000 7f2cf186d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec161da0 0x7f2cec15b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37012/0 (socket says 172.21.15.16:37012) 2024-09-19T07:53:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.042+0000 7f2cf186d640 1 -- 172.21.15.16:0/434950707 learned_addr learned my addr 172.21.15.16:0/434950707 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:01.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.042+0000 7f2cf086b640 1 --2- 172.21.15.16:0/434950707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cec0d2740 0x7f2cec15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.042+0000 7f2cf106c640 1 --2- 172.21.15.16:0/434950707 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2cec0b3b10 0x7f2cec156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.042+0000 7f2cf186d640 1 -- 172.21.15.16:0/434950707 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2cec0b3b10 msgr2=0x7f2cec156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.042+0000 7f2cf186d640 1 --2- 172.21.15.16:0/434950707 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2cec0b3b10 0x7f2cec156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.042+0000 7f2cf186d640 1 -- 172.21.15.16:0/434950707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cec0d2740 msgr2=0x7f2cec15b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.042+0000 7f2cf186d640 1 --2- 172.21.15.16:0/434950707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cec0d2740 0x7f2cec15b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.042+0000 7f2cf186d640 1 -- 172.21.15.16:0/434950707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cec1569c0 con 0x7f2cec161da0 2024-09-19T07:53:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.042+0000 7f2cf186d640 1 --2- 172.21.15.16:0/434950707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec161da0 0x7f2cec15b990 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4007e10 tx=0x7f2ce400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.043+0000 7f2ce1ffb640 1 -- 172.21.15.16:0/434950707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4014620 con 0x7f2cec161da0 2024-09-19T07:53:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.043+0000 7f2ce1ffb640 1 -- 172.21.15.16:0/434950707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ce4019070 con 0x7f2cec161da0 2024-09-19T07:53:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.043+0000 7f2cf32d0640 1 -- 172.21.15.16:0/434950707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cec0b1d40 con 0x7f2cec161da0 2024-09-19T07:53:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.043+0000 7f2ce1ffb640 1 -- 172.21.15.16:0/434950707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4014a20 con 0x7f2cec161da0 2024-09-19T07:53:01.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.043+0000 7f2cf32d0640 1 -- 172.21.15.16:0/434950707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cec0b2300 con 0x7f2cec161da0 2024-09-19T07:53:01.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.044+0000 7f2cf32d0640 1 -- 172.21.15.16:0/434950707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cec0b30e0 con 0x7f2cec161da0 2024-09-19T07:53:01.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.044+0000 7f2ce1ffb640 1 -- 172.21.15.16:0/434950707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2ce4014bc0 con 0x7f2cec161da0 2024-09-19T07:53:01.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.045+0000 7f2ce1ffb640 1 --2- 172.21.15.16:0/434950707 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2cd006c2b0 0x7f2cd006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:01.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.045+0000 7f2ce1ffb640 1 -- 172.21.15.16:0/434950707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5619+0+0 (secure 0 0 0) 0x7f2ce408bb60 con 0x7f2cec161da0 2024-09-19T07:53:01.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.045+0000 7f2cf106c640 1 --2- 172.21.15.16:0/434950707 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2cd006c2b0 0x7f2cd006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:01.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.046+0000 7f2cf106c640 1 --2- 172.21.15.16:0/434950707 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2cd006c2b0 0x7f2cd006e770 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8034000 tx=0x7f2ce802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:01.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.050+0000 7f2ce1ffb640 1 -- 172.21.15.16:0/434950707 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2cec1569c0 con 0x7f2cec161da0 2024-09-19T07:53:01.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.226+0000 7f2cf32d0640 1 -- 172.21.15.16:0/434950707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2cec1615c0 con 0x7f2cec161da0 2024-09-19T07:53:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.902+0000 7f2ce1ffb640 1 -- 172.21.15.16:0/434950707 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v104) ==== 170+0+0 (secure 0 0 0) 0x7f2ce4059cd0 con 0x7f2cec161da0 2024-09-19T07:53:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-19T07:53:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.906+0000 7f2cf32d0640 1 -- 172.21.15.16:0/434950707 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2cd006c2b0 msgr2=0x7f2cd006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.906+0000 7f2cf32d0640 1 --2- 172.21.15.16:0/434950707 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2cd006c2b0 0x7f2cd006e770 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8034000 tx=0x7f2ce802e000 comp rx=0 tx=0).stop 2024-09-19T07:53:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.906+0000 7f2cf32d0640 1 -- 172.21.15.16:0/434950707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec161da0 msgr2=0x7f2cec15b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.906+0000 7f2cf32d0640 1 --2- 172.21.15.16:0/434950707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec161da0 0x7f2cec15b990 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4007e10 tx=0x7f2ce400e4b0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.906+0000 7f2cf32d0640 1 -- 172.21.15.16:0/434950707 shutdown_connections 2024-09-19T07:53:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.906+0000 7f2cf32d0640 1 --2- 172.21.15.16:0/434950707 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2cd006c2b0 0x7f2cd006e770 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.906+0000 7f2cf32d0640 1 --2- 172.21.15.16:0/434950707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cec161da0 0x7f2cec15b990 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.906+0000 7f2cf32d0640 1 --2- 172.21.15.16:0/434950707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cec0d2740 0x7f2cec15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.906+0000 7f2cf32d0640 1 --2- 172.21.15.16:0/434950707 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2cec0b3b10 0x7f2cec156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.906+0000 7f2cf32d0640 1 -- 172.21.15.16:0/434950707 >> 172.21.15.16:0/434950707 conn(0x7f2cec08bf80 msgr2=0x7f2cec06cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.906+0000 7f2cf32d0640 1 -- 172.21.15.16:0/434950707 shutdown_connections 2024-09-19T07:53:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:01.906+0000 7f2cf32d0640 1 -- 172.21.15.16:0/434950707 wait complete. 2024-09-19T07:53:01.930 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T07:53:02.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.040+0000 7f19f861e640 1 Processor -- start 2024-09-19T07:53:02.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.040+0000 7f19f861e640 1 -- start start 2024-09-19T07:53:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.041+0000 7f19f861e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19f015f0d0 0x7f19f015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.041+0000 7f19f861e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19f00d0070 0x7f19f00d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.041+0000 7f19f861e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19f00cf2b0 con 0x7f19f00d1940 2024-09-19T07:53:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.041+0000 7f19f861e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f19f00cf590 con 0x7f19f00d0070 2024-09-19T07:53:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.041+0000 7f19f861e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f19f00cf870 con 0x7f19f015f0d0 2024-09-19T07:53:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.041+0000 7f19f5bb9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19f00d0070 0x7f19f00d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.041+0000 7f19f5bb9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19f00d0070 0x7f19f00d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37020/0 (socket says 172.21.15.16:37020) 2024-09-19T07:53:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.041+0000 7f19f5bb9640 1 -- 172.21.15.16:0/2500506476 learned_addr learned my addr 172.21.15.16:0/2500506476 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.041+0000 7f19f6bbb640 1 --1- 172.21.15.16:0/2500506476 >> v1:172.21.15.16:6789/0 conn(0x7f19f00d1940 0x7f19f00d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51496/0 (socket says 172.21.15.16:51496) 2024-09-19T07:53:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.041+0000 7f19f63ba640 1 --2- 172.21.15.16:0/2500506476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19f015f0d0 0x7f19f015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.041+0000 7f19f5bb9640 1 -- 172.21.15.16:0/2500506476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19f015f0d0 msgr2=0x7f19f015f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.041+0000 7f19f5bb9640 1 --2- 172.21.15.16:0/2500506476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19f015f0d0 0x7f19f015f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.041+0000 7f19f5bb9640 1 -- 172.21.15.16:0/2500506476 >> v1:172.21.15.16:6789/0 conn(0x7f19f00d1940 legacy=0x7f19f00d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.041+0000 7f19f5bb9640 1 -- 172.21.15.16:0/2500506476 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f00cfb50 con 0x7f19f00d0070 2024-09-19T07:53:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.041+0000 7f19f53b8640 1 -- 172.21.15.16:0/2500506476 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2263258640 0 0) 0x7f19f00cf2b0 con 0x7f19f00d1940 2024-09-19T07:53:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.042+0000 7f19f5bb9640 1 --2- 172.21.15.16:0/2500506476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f00d0070 0x7f19f00d1400 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009870 tx=0x7f19e002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2215ca6f011b5b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.042+0000 7f19f53b8640 1 -- 172.21.15.16:0/2500506476 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0037070 con 0x7f19f00d0070 2024-09-19T07:53:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.042+0000 7f19f53b8640 1 -- 172.21.15.16:0/2500506476 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e0032440 con 0x7f19f00d0070 2024-09-19T07:53:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.042+0000 7f19f53b8640 1 -- 172.21.15.16:0/2500506476 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0032810 con 0x7f19f00d0070 2024-09-19T07:53:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.043+0000 7f19f861e640 1 -- 172.21.15.16:0/2500506476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f00d0070 msgr2=0x7f19f00d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.043+0000 7f19f861e640 1 --2- 172.21.15.16:0/2500506476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f00d0070 0x7f19f00d1400 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009870 tx=0x7f19e002a1c0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.043+0000 7f19f861e640 1 -- 172.21.15.16:0/2500506476 shutdown_connections 2024-09-19T07:53:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.043+0000 7f19f861e640 1 --2- 172.21.15.16:0/2500506476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f00d0070 0x7f19f00d1400 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.043+0000 7f19f861e640 1 --2- 172.21.15.16:0/2500506476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19f015f0d0 0x7f19f015f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.043+0000 7f19f861e640 1 -- 172.21.15.16:0/2500506476 >> 172.21.15.16:0/2500506476 conn(0x7f19f009d760 msgr2=0x7f19f008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.043+0000 7f19f861e640 1 -- 172.21.15.16:0/2500506476 shutdown_connections 2024-09-19T07:53:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.043+0000 7f19f861e640 1 -- 172.21.15.16:0/2500506476 wait complete. 2024-09-19T07:53:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.044+0000 7f19f861e640 1 Processor -- start 2024-09-19T07:53:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.044+0000 7f19f861e640 1 -- start start 2024-09-19T07:53:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.045+0000 7f19f861e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19f00d0070 0x7f19f0155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.045+0000 7f19f861e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f00d1940 0x7f19f015a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.045+0000 7f19f861e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19f015f0d0 0x7f19f015af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.045+0000 7f19f861e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19f00d49c0 con 0x7f19f00d1940 2024-09-19T07:53:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.045+0000 7f19f861e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f19f00d4840 con 0x7f19f015f0d0 2024-09-19T07:53:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.045+0000 7f19f861e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f19f00d4b40 con 0x7f19f00d0070 2024-09-19T07:53:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.045+0000 7f19f5bb9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f00d1940 0x7f19f015a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.045+0000 7f19f5bb9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f00d1940 0x7f19f015a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37028/0 (socket says 172.21.15.16:37028) 2024-09-19T07:53:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.045+0000 7f19f5bb9640 1 -- 172.21.15.16:0/1917045550 learned_addr learned my addr 172.21.15.16:0/1917045550 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.045+0000 7f19f63ba640 1 --2- 172.21.15.16:0/1917045550 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19f00d0070 0x7f19f0155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.045+0000 7f19f6bbb640 1 --2- 172.21.15.16:0/1917045550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19f015f0d0 0x7f19f015af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.045+0000 7f19f5bb9640 1 -- 172.21.15.16:0/1917045550 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19f00d0070 msgr2=0x7f19f0155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.045+0000 7f19f5bb9640 1 --2- 172.21.15.16:0/1917045550 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19f00d0070 0x7f19f0155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.045+0000 7f19f5bb9640 1 -- 172.21.15.16:0/1917045550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19f015f0d0 msgr2=0x7f19f015af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.045+0000 7f19f5bb9640 1 --2- 172.21.15.16:0/1917045550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19f015f0d0 0x7f19f015af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.045+0000 7f19f5bb9640 1 -- 172.21.15.16:0/1917045550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f00cb700 con 0x7f19f00d1940 2024-09-19T07:53:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.046+0000 7f19f5bb9640 1 --2- 172.21.15.16:0/1917045550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f00d1940 0x7f19f015a9e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009870 tx=0x7f19e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.046+0000 7f19e6ffd640 1 -- 172.21.15.16:0/1917045550 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0037040 con 0x7f19f00d1940 2024-09-19T07:53:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.046+0000 7f19e6ffd640 1 -- 172.21.15.16:0/1917045550 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e0004250 con 0x7f19f00d1940 2024-09-19T07:53:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.046+0000 7f19f861e640 1 -- 172.21.15.16:0/1917045550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f00cb9e0 con 0x7f19f00d1940 2024-09-19T07:53:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.046+0000 7f19e6ffd640 1 -- 172.21.15.16:0/1917045550 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0003610 con 0x7f19f00d1940 2024-09-19T07:53:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.046+0000 7f19f861e640 1 -- 172.21.15.16:0/1917045550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f00cbfa0 con 0x7f19f00d1940 2024-09-19T07:53:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.047+0000 7f19f861e640 1 -- 172.21.15.16:0/1917045550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f00d1400 con 0x7f19f00d1940 2024-09-19T07:53:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.047+0000 7f19e6ffd640 1 -- 172.21.15.16:0/1917045550 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f19e002ad10 con 0x7f19f00d1940 2024-09-19T07:53:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.048+0000 7f19e6ffd640 1 --2- 172.21.15.16:0/1917045550 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f19d406c2b0 0x7f19d406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.048+0000 7f19e6ffd640 1 -- 172.21.15.16:0/1917045550 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6001+0+0 (secure 0 0 0) 0x7f19e00abe90 con 0x7f19f00d1940 2024-09-19T07:53:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.048+0000 7f19f63ba640 1 --2- 172.21.15.16:0/1917045550 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f19d406c2b0 0x7f19d406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.049+0000 7f19f63ba640 1 --2- 172.21.15.16:0/1917045550 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f19d406c2b0 0x7f19d406e770 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009aa0 tx=0x7f19ec009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.053+0000 7f19e6ffd640 1 -- 172.21.15.16:0/1917045550 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f19e0079fa0 con 0x7f19f00d1940 2024-09-19T07:53:02.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.263+0000 7f19f861e640 1 -- 172.21.15.16:0/1917045550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f19f00d0e00 con 0x7f19f00d1940 2024-09-19T07:53:02.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.949+0000 7f19e6ffd640 1 -- 172.21.15.16:0/1917045550 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v54) ==== 174+0+0 (secure 0 0 0) 0x7f19e007e2a0 con 0x7f19f00d1940 2024-09-19T07:53:02.955 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 15 and data pool 16 2024-09-19T07:53:02.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.957+0000 7f19f861e640 1 -- 172.21.15.16:0/1917045550 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f19d406c2b0 msgr2=0x7f19d406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:02.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.957+0000 7f19f861e640 1 --2- 172.21.15.16:0/1917045550 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f19d406c2b0 0x7f19d406e770 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009aa0 tx=0x7f19ec009190 comp rx=0 tx=0).stop 2024-09-19T07:53:02.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.957+0000 7f19f861e640 1 -- 172.21.15.16:0/1917045550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f00d1940 msgr2=0x7f19f015a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:02.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.957+0000 7f19f861e640 1 --2- 172.21.15.16:0/1917045550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f00d1940 0x7f19f015a9e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009870 tx=0x7f19e00046c0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.961+0000 7f19f861e640 1 -- 172.21.15.16:0/1917045550 shutdown_connections 2024-09-19T07:53:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.961+0000 7f19f861e640 1 --2- 172.21.15.16:0/1917045550 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f19d406c2b0 0x7f19d406e770 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.961+0000 7f19f861e640 1 --2- 172.21.15.16:0/1917045550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19f015f0d0 0x7f19f015af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.961+0000 7f19f861e640 1 --2- 172.21.15.16:0/1917045550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f00d1940 0x7f19f015a9e0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.961+0000 7f19f861e640 1 --2- 172.21.15.16:0/1917045550 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19f00d0070 0x7f19f0155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.961+0000 7f19f861e640 1 -- 172.21.15.16:0/1917045550 >> 172.21.15.16:0/1917045550 conn(0x7f19f009d760 msgr2=0x7f19f00d25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:02.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.965+0000 7f19f861e640 1 -- 172.21.15.16:0/1917045550 shutdown_connections 2024-09-19T07:53:02.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:02.966+0000 7f19f861e640 1 -- 172.21.15.16:0/1917045550 wait complete. 2024-09-19T07:53:02.984 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:53:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.090+0000 7f80425bd640 1 Processor -- start 2024-09-19T07:53:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.090+0000 7f80425bd640 1 -- start start 2024-09-19T07:53:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.090+0000 7f80425bd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f803c15f0d0 0x7f803c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.090+0000 7f80425bd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f803c0d4130 0x7f803c0d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.090+0000 7f80425bd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f803c0d4b20 con 0x7f803c0d0070 2024-09-19T07:53:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.090+0000 7f80425bd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f803c0d1b10 con 0x7f803c0d4130 2024-09-19T07:53:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.090+0000 7f80425bd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f803c0cf2b0 con 0x7f803c15f0d0 2024-09-19T07:53:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.091+0000 7f8040b5a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f803c0d4130 0x7f803c0d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.091+0000 7f803bfff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f803c15f0d0 0x7f803c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.091+0000 7f803bfff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f803c15f0d0 0x7f803c15f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:48618/0 (socket says 172.21.15.16:48618) 2024-09-19T07:53:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.091+0000 7f803bfff640 1 -- 172.21.15.16:0/371101077 learned_addr learned my addr 172.21.15.16:0/371101077 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.091+0000 7f803b7fe640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f803c0d0070 0x7f803c0d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:34396/0 (socket says 172.21.15.16:34396) 2024-09-19T07:53:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.091+0000 7f803bfff640 1 -- 172.21.15.16:0/371101077 >> v2:172.21.15.146:3300/0 conn(0x7f803c0d4130 msgr2=0x7f803c0d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.091+0000 7f803bfff640 1 --2- 172.21.15.16:0/371101077 >> v2:172.21.15.146:3300/0 conn(0x7f803c0d4130 0x7f803c0d45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.091+0000 7f803bfff640 1 -- 172.21.15.16:0/371101077 >> v1:172.21.15.146:6789/0 conn(0x7f803c0d0070 legacy=0x7f803c0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.091+0000 7f803bfff640 1 -- 172.21.15.16:0/371101077 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c0cf590 con 0x7f803c15f0d0 2024-09-19T07:53:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.091+0000 7f8040b5a640 1 --2- 172.21.15.16:0/371101077 >> v2:172.21.15.146:3300/0 conn(0x7f803c0d4130 0x7f803c0d45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:53:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.092+0000 7f803bfff640 1 --2- 172.21.15.16:0/371101077 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f803c15f0d0 0x7f803c15f4b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f803c0d2fc0 tx=0x7f802c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75fb817bcdf7a31c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.092+0000 7f803affd640 1 -- 172.21.15.16:0/371101077 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c037070 con 0x7f803c15f0d0 2024-09-19T07:53:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.092+0000 7f803affd640 1 -- 172.21.15.16:0/371101077 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f802c032440 con 0x7f803c15f0d0 2024-09-19T07:53:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.092+0000 7f803affd640 1 -- 172.21.15.16:0/371101077 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c032810 con 0x7f803c15f0d0 2024-09-19T07:53:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.093+0000 7f80425bd640 1 -- 172.21.15.16:0/371101077 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f803c15f0d0 msgr2=0x7f803c15f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.093+0000 7f80425bd640 1 --2- 172.21.15.16:0/371101077 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f803c15f0d0 0x7f803c15f4b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f803c0d2fc0 tx=0x7f802c02a1c0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.093+0000 7f80425bd640 1 -- 172.21.15.16:0/371101077 shutdown_connections 2024-09-19T07:53:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.093+0000 7f80425bd640 1 --2- 172.21.15.16:0/371101077 >> v2:172.21.15.146:3300/0 conn(0x7f803c0d4130 0x7f803c0d45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.093+0000 7f80425bd640 1 --2- 172.21.15.16:0/371101077 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f803c15f0d0 0x7f803c15f4b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.093+0000 7f80425bd640 1 -- 172.21.15.16:0/371101077 >> 172.21.15.16:0/371101077 conn(0x7f803c09d760 msgr2=0x7f803c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.093+0000 7f80425bd640 1 -- 172.21.15.16:0/371101077 shutdown_connections 2024-09-19T07:53:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.093+0000 7f80425bd640 1 -- 172.21.15.16:0/371101077 wait complete. 2024-09-19T07:53:03.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.094+0000 7f80425bd640 1 Processor -- start 2024-09-19T07:53:03.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.094+0000 7f80425bd640 1 -- start start 2024-09-19T07:53:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.095+0000 7f80425bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c0d0070 0x7f803c0c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.095+0000 7f80425bd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f803c0d4130 0x7f803c1e8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.095+0000 7f80425bd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f803c15f0d0 0x7f803c0b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.095+0000 7f80425bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f803c0cef60 con 0x7f803c0d0070 2024-09-19T07:53:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.095+0000 7f80425bd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f803c0cfb50 con 0x7f803c15f0d0 2024-09-19T07:53:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.095+0000 7f80425bd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f803c0b2f70 con 0x7f803c0d4130 2024-09-19T07:53:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.095+0000 7f803bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c0d0070 0x7f803c0c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.095+0000 7f803bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c0d0070 0x7f803c0c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60622/0 (socket says 172.21.15.16:60622) 2024-09-19T07:53:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.095+0000 7f803bfff640 1 -- 172.21.15.16:0/1715980216 learned_addr learned my addr 172.21.15.16:0/1715980216 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.095+0000 7f803b7fe640 1 --2- 172.21.15.16:0/1715980216 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f803c0d4130 0x7f803c1e8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.095+0000 7f8040b5a640 1 --2- 172.21.15.16:0/1715980216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f803c15f0d0 0x7f803c0b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.095+0000 7f803bfff640 1 -- 172.21.15.16:0/1715980216 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f803c0d4130 msgr2=0x7f803c1e8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.095+0000 7f803bfff640 1 --2- 172.21.15.16:0/1715980216 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f803c0d4130 0x7f803c1e8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.095+0000 7f803bfff640 1 -- 172.21.15.16:0/1715980216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f803c15f0d0 msgr2=0x7f803c0b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.095+0000 7f803bfff640 1 --2- 172.21.15.16:0/1715980216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f803c15f0d0 0x7f803c0b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.095+0000 7f803bfff640 1 -- 172.21.15.16:0/1715980216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c156160 con 0x7f803c0d0070 2024-09-19T07:53:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.095+0000 7f803bfff640 1 --2- 172.21.15.16:0/1715980216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c0d0070 0x7f803c0c7580 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f802c02a200 tx=0x7f802c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.096+0000 7f8038ff9640 1 -- 172.21.15.16:0/1715980216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c037040 con 0x7f803c0d0070 2024-09-19T07:53:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.096+0000 7f8038ff9640 1 -- 172.21.15.16:0/1715980216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f802c042070 con 0x7f803c0d0070 2024-09-19T07:53:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.096+0000 7f80425bd640 1 -- 172.21.15.16:0/1715980216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f803c159e60 con 0x7f803c0d0070 2024-09-19T07:53:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.096+0000 7f8038ff9640 1 -- 172.21.15.16:0/1715980216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c030880 con 0x7f803c0d0070 2024-09-19T07:53:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.096+0000 7f80425bd640 1 -- 172.21.15.16:0/1715980216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f803c15a420 con 0x7f803c0d0070 2024-09-19T07:53:03.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.097+0000 7f80425bd640 1 -- 172.21.15.16:0/1715980216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f803c0d45e0 con 0x7f803c0d0070 2024-09-19T07:53:03.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.097+0000 7f8038ff9640 1 -- 172.21.15.16:0/1715980216 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f802c02ad10 con 0x7f803c0d0070 2024-09-19T07:53:03.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.098+0000 7f8038ff9640 1 --2- 172.21.15.16:0/1715980216 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f802806c2b0 0x7f802806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.098+0000 7f8038ff9640 1 -- 172.21.15.16:0/1715980216 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f802c0aba30 con 0x7f803c0d0070 2024-09-19T07:53:03.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.098+0000 7f803b7fe640 1 --2- 172.21.15.16:0/1715980216 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f802806c2b0 0x7f802806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.099+0000 7f803b7fe640 1 --2- 172.21.15.16:0/1715980216 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f802806c2b0 0x7f802806e770 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f803c160600 tx=0x7f8030007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:03.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.102+0000 7f8038ff9640 1 -- 172.21.15.16:0/1715980216 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f802c07a010 con 0x7f803c0d0070 2024-09-19T07:53:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.282+0000 7f80425bd640 1 -- 172.21.15.16:0/1715980216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f803c15e980 con 0x7f803c0d0070 2024-09-19T07:53:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.283+0000 7f8038ff9640 1 -- 172.21.15.16:0/1715980216 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16981 (secure 0 0 0) 0x7f802c079910 con 0x7f803c0d0070 2024-09-19T07:53:03.283 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:03.283 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":105,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:53:02.264492+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:52:59.931281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-19T07:53:01.258168+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:53:03.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.285+0000 7f80425bd640 1 -- 172.21.15.16:0/1715980216 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f802806c2b0 msgr2=0x7f802806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.285+0000 7f80425bd640 1 --2- 172.21.15.16:0/1715980216 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f802806c2b0 0x7f802806e770 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f803c160600 tx=0x7f8030007e80 comp rx=0 tx=0).stop 2024-09-19T07:53:03.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.285+0000 7f80425bd640 1 -- 172.21.15.16:0/1715980216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c0d0070 msgr2=0x7f803c0c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.285+0000 7f80425bd640 1 --2- 172.21.15.16:0/1715980216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c0d0070 0x7f803c0c7580 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f802c02a200 tx=0x7f802c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.285+0000 7f80425bd640 1 -- 172.21.15.16:0/1715980216 shutdown_connections 2024-09-19T07:53:03.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.285+0000 7f80425bd640 1 --2- 172.21.15.16:0/1715980216 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f802806c2b0 0x7f802806e770 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.285+0000 7f80425bd640 1 --2- 172.21.15.16:0/1715980216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f803c15f0d0 0x7f803c0b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.285+0000 7f80425bd640 1 --2- 172.21.15.16:0/1715980216 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f803c0d4130 0x7f803c1e8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.285+0000 7f80425bd640 1 --2- 172.21.15.16:0/1715980216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f803c0d0070 0x7f803c0c7580 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.285+0000 7f80425bd640 1 -- 172.21.15.16:0/1715980216 >> 172.21.15.16:0/1715980216 conn(0x7f803c09d760 msgr2=0x7f803c0b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.285+0000 7f80425bd640 1 -- 172.21.15.16:0/1715980216 shutdown_connections 2024-09-19T07:53:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.285+0000 7f80425bd640 1 -- 172.21.15.16:0/1715980216 wait complete. 2024-09-19T07:53:03.297 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:53:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.398+0000 7f4020cac640 1 Processor -- start 2024-09-19T07:53:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.398+0000 7f4020cac640 1 -- start start 2024-09-19T07:53:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.399+0000 7f4020cac640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f401c141680 0x7f401c0cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.399+0000 7f4020cac640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f401c0d0370 0x7f401c141a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.399+0000 7f4020cac640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f401c15d710 con 0x7f401c141fa0 2024-09-19T07:53:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.399+0000 7f4020cac640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f401c0d07b0 con 0x7f401c0d0370 2024-09-19T07:53:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.399+0000 7f4020cac640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f401c15c6f0 con 0x7f401c141680 2024-09-19T07:53:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.400+0000 7f401a59c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f401c141680 0x7f401c0cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.400+0000 7f4019d9b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f401c0d0370 0x7f401c141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.400+0000 7f401a59c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f401c141680 0x7f401c0cfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:48634/0 (socket says 172.21.15.16:48634) 2024-09-19T07:53:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.400+0000 7f401a59c640 1 -- 172.21.15.16:0/2614242110 learned_addr learned my addr 172.21.15.16:0/2614242110 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.400+0000 7f401a59c640 1 -- 172.21.15.16:0/2614242110 >> v2:172.21.15.146:3300/0 conn(0x7f401c0d0370 msgr2=0x7f401c141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.400+0000 7f401a59c640 1 --2- 172.21.15.16:0/2614242110 >> v2:172.21.15.146:3300/0 conn(0x7f401c0d0370 0x7f401c141a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.400+0000 7f401a59c640 1 -- 172.21.15.16:0/2614242110 >> v1:172.21.15.146:6789/0 conn(0x7f401c141fa0 legacy=0x7f401c0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.400+0000 7f401a59c640 1 -- 172.21.15.16:0/2614242110 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401c15c950 con 0x7f401c141680 2024-09-19T07:53:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.401+0000 7f401a59c640 1 --2- 172.21.15.16:0/2614242110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f401c141680 0x7f401c0cfe30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4014009870 tx=0x7f401402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e266c13e06be78c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.401+0000 7f401959a640 1 -- 172.21.15.16:0/2614242110 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014037070 con 0x7f401c141680 2024-09-19T07:53:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.401+0000 7f401959a640 1 -- 172.21.15.16:0/2614242110 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4014032440 con 0x7f401c141680 2024-09-19T07:53:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.401+0000 7f401959a640 1 -- 172.21.15.16:0/2614242110 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014032810 con 0x7f401c141680 2024-09-19T07:53:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.401+0000 7f4020cac640 1 -- 172.21.15.16:0/2614242110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f401c141680 msgr2=0x7f401c0cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.401+0000 7f4020cac640 1 --2- 172.21.15.16:0/2614242110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f401c141680 0x7f401c0cfe30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4014009870 tx=0x7f401402a270 comp rx=0 tx=0).stop 2024-09-19T07:53:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.402+0000 7f4020cac640 1 -- 172.21.15.16:0/2614242110 shutdown_connections 2024-09-19T07:53:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.402+0000 7f4020cac640 1 --2- 172.21.15.16:0/2614242110 >> v2:172.21.15.146:3300/0 conn(0x7f401c0d0370 0x7f401c141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.402+0000 7f4020cac640 1 --2- 172.21.15.16:0/2614242110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f401c141680 0x7f401c0cfe30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.402+0000 7f4020cac640 1 -- 172.21.15.16:0/2614242110 >> 172.21.15.16:0/2614242110 conn(0x7f401c0b3ed0 msgr2=0x7f401c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.402+0000 7f4020cac640 1 -- 172.21.15.16:0/2614242110 shutdown_connections 2024-09-19T07:53:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.402+0000 7f4020cac640 1 -- 172.21.15.16:0/2614242110 wait complete. 2024-09-19T07:53:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.403+0000 7f4020cac640 1 Processor -- start 2024-09-19T07:53:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.403+0000 7f4020cac640 1 -- start start 2024-09-19T07:53:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.404+0000 7f4020cac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0d0370 0x7f401c13e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.404+0000 7f4020cac640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f401c141680 0x7f401c0b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.404+0000 7f4020cac640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f401c141fa0 0x7f401c06dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.404+0000 7f4020cac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f401c15d1f0 con 0x7f401c0d0370 2024-09-19T07:53:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.404+0000 7f4020cac640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f401c15cf10 con 0x7f401c141fa0 2024-09-19T07:53:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.404+0000 7f4020cac640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f401c1408c0 con 0x7f401c141680 2024-09-19T07:53:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.404+0000 7f401a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0d0370 0x7f401c13e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.404+0000 7f401a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0d0370 0x7f401c13e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60634/0 (socket says 172.21.15.16:60634) 2024-09-19T07:53:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.404+0000 7f401a59c640 1 -- 172.21.15.16:0/1048602661 learned_addr learned my addr 172.21.15.16:0/1048602661 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.404+0000 7f4019d9b640 1 --2- 172.21.15.16:0/1048602661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f401c141680 0x7f401c0b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.404+0000 7f401ad9d640 1 --2- 172.21.15.16:0/1048602661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f401c141fa0 0x7f401c06dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.404+0000 7f401a59c640 1 -- 172.21.15.16:0/1048602661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f401c141680 msgr2=0x7f401c0b7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.404+0000 7f401a59c640 1 --2- 172.21.15.16:0/1048602661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f401c141680 0x7f401c0b7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.404+0000 7f401a59c640 1 -- 172.21.15.16:0/1048602661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f401c141fa0 msgr2=0x7f401c06dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.404+0000 7f401a59c640 1 --2- 172.21.15.16:0/1048602661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f401c141fa0 0x7f401c06dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.404+0000 7f401a59c640 1 -- 172.21.15.16:0/1048602661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401c0b5270 con 0x7f401c0d0370 2024-09-19T07:53:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.404+0000 7f401a59c640 1 --2- 172.21.15.16:0/1048602661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0d0370 0x7f401c13e970 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4014000c00 tx=0x7f40140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.405+0000 7f4012ffd640 1 -- 172.21.15.16:0/1048602661 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014037040 con 0x7f401c0d0370 2024-09-19T07:53:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.405+0000 7f4012ffd640 1 -- 172.21.15.16:0/1048602661 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4014042070 con 0x7f401c0d0370 2024-09-19T07:53:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.405+0000 7f4012ffd640 1 -- 172.21.15.16:0/1048602661 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014030880 con 0x7f401c0d0370 2024-09-19T07:53:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.405+0000 7f4020cac640 1 -- 172.21.15.16:0/1048602661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401c0b5550 con 0x7f401c0d0370 2024-09-19T07:53:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.405+0000 7f4020cac640 1 -- 172.21.15.16:0/1048602661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f401c0b5b10 con 0x7f401c0d0370 2024-09-19T07:53:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.406+0000 7f4020cac640 1 -- 172.21.15.16:0/1048602661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f401c142430 con 0x7f401c0d0370 2024-09-19T07:53:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.406+0000 7f4012ffd640 1 -- 172.21.15.16:0/1048602661 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4014030a20 con 0x7f401c0d0370 2024-09-19T07:53:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.407+0000 7f4012ffd640 1 --2- 172.21.15.16:0/1048602661 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3ffc06c2b0 0x7f3ffc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.407+0000 7f4012ffd640 1 -- 172.21.15.16:0/1048602661 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f40140abd00 con 0x7f401c0d0370 2024-09-19T07:53:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.407+0000 7f4019d9b640 1 --2- 172.21.15.16:0/1048602661 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3ffc06c2b0 0x7f3ffc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.408+0000 7f4019d9b640 1 --2- 172.21.15.16:0/1048602661 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3ffc06c2b0 0x7f3ffc06e770 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f401c0ce420 tx=0x7f4004009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:03.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.411+0000 7f4012ffd640 1 -- 172.21.15.16:0/1048602661 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4014079c90 con 0x7f401c0d0370 2024-09-19T07:53:03.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.591+0000 7f4020cac640 1 -- 172.21.15.16:0/1048602661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f401c0690c0 con 0x7f401c0d0370 2024-09-19T07:53:03.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.593+0000 7f4012ffd640 1 -- 172.21.15.16:0/1048602661 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16981 (secure 0 0 0) 0x7f401407df90 con 0x7f401c0d0370 2024-09-19T07:53:03.593 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:03.593 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":105,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:53:02.264492+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:52:59.931281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-19T07:53:01.258168+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:53:03.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.595+0000 7f4020cac640 1 -- 172.21.15.16:0/1048602661 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3ffc06c2b0 msgr2=0x7f3ffc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.595+0000 7f4020cac640 1 --2- 172.21.15.16:0/1048602661 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3ffc06c2b0 0x7f3ffc06e770 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f401c0ce420 tx=0x7f4004009190 comp rx=0 tx=0).stop 2024-09-19T07:53:03.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.595+0000 7f4020cac640 1 -- 172.21.15.16:0/1048602661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0d0370 msgr2=0x7f401c13e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.595+0000 7f4020cac640 1 --2- 172.21.15.16:0/1048602661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0d0370 0x7f401c13e970 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4014000c00 tx=0x7f40140046c0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.595+0000 7f4020cac640 1 -- 172.21.15.16:0/1048602661 shutdown_connections 2024-09-19T07:53:03.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.595+0000 7f4020cac640 1 --2- 172.21.15.16:0/1048602661 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3ffc06c2b0 0x7f3ffc06e770 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.595+0000 7f4020cac640 1 --2- 172.21.15.16:0/1048602661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f401c141fa0 0x7f401c06dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.595+0000 7f4020cac640 1 --2- 172.21.15.16:0/1048602661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f401c141680 0x7f401c0b7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.595+0000 7f4020cac640 1 --2- 172.21.15.16:0/1048602661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0d0370 0x7f401c13e970 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.595+0000 7f4020cac640 1 -- 172.21.15.16:0/1048602661 >> 172.21.15.16:0/1048602661 conn(0x7f401c0b3ed0 msgr2=0x7f401c070670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:03.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.595+0000 7f4020cac640 1 -- 172.21.15.16:0/1048602661 shutdown_connections 2024-09-19T07:53:03.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.595+0000 7f4020cac640 1 -- 172.21.15.16:0/1048602661 wait complete. 2024-09-19T07:53:03.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T07:53:03.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.711+0000 7fbb09477640 1 Processor -- start 2024-09-19T07:53:03.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.712+0000 7fbb09477640 1 -- start start 2024-09-19T07:53:03.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.712+0000 7fbb09477640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb0415b300 0x7fbb040cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.712+0000 7fbb09477640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbb040ccbd0 0x7fbb040cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.712+0000 7fbb09477640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb040cd910 con 0x7fbb0415b6e0 2024-09-19T07:53:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.712+0000 7fbb09477640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fbb040cd070 con 0x7fbb040ccbd0 2024-09-19T07:53:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.712+0000 7fbb09477640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbb040cd350 con 0x7fbb0415b300 2024-09-19T07:53:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.713+0000 7fbb02ffd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb0415b300 0x7fbb040cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.713+0000 7fbb02ffd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb0415b300 0x7fbb040cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:48656/0 (socket says 172.21.15.16:48656) 2024-09-19T07:53:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.713+0000 7fbb027fc640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbb040ccbd0 0x7fbb040cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.713+0000 7fbb02ffd640 1 -- 172.21.15.16:0/3643892470 learned_addr learned my addr 172.21.15.16:0/3643892470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.713+0000 7fbb02ffd640 1 -- 172.21.15.16:0/3643892470 >> v2:172.21.15.146:3300/0 conn(0x7fbb040ccbd0 msgr2=0x7fbb040cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.713+0000 7fbb02ffd640 1 --2- 172.21.15.16:0/3643892470 >> v2:172.21.15.146:3300/0 conn(0x7fbb040ccbd0 0x7fbb040cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.713+0000 7fbb02ffd640 1 -- 172.21.15.16:0/3643892470 >> v1:172.21.15.146:6789/0 conn(0x7fbb0415b6e0 legacy=0x7fbb040cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.713+0000 7fbb02ffd640 1 -- 172.21.15.16:0/3643892470 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb040cd630 con 0x7fbb0415b300 2024-09-19T07:53:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.714+0000 7fbb02ffd640 1 --2- 172.21.15.16:0/3643892470 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb0415b300 0x7fbb040cc690 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009e50 tx=0x7fbafc02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43cef4fc013110ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.714+0000 7fbb01ffb640 1 -- 172.21.15.16:0/3643892470 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc036070 con 0x7fbb0415b300 2024-09-19T07:53:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.714+0000 7fbb01ffb640 1 -- 172.21.15.16:0/3643892470 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc031600 con 0x7fbb0415b300 2024-09-19T07:53:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.714+0000 7fbb01ffb640 1 -- 172.21.15.16:0/3643892470 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0319d0 con 0x7fbb0415b300 2024-09-19T07:53:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.715+0000 7fbb09477640 1 -- 172.21.15.16:0/3643892470 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb0415b300 msgr2=0x7fbb040cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.715+0000 7fbb09477640 1 --2- 172.21.15.16:0/3643892470 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb0415b300 0x7fbb040cc690 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009e50 tx=0x7fbafc02a710 comp rx=0 tx=0).stop 2024-09-19T07:53:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.715+0000 7fbb09477640 1 -- 172.21.15.16:0/3643892470 shutdown_connections 2024-09-19T07:53:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.715+0000 7fbb09477640 1 --2- 172.21.15.16:0/3643892470 >> v2:172.21.15.146:3300/0 conn(0x7fbb040ccbd0 0x7fbb040cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.715+0000 7fbb09477640 1 --2- 172.21.15.16:0/3643892470 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb0415b300 0x7fbb040cc690 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.715+0000 7fbb09477640 1 -- 172.21.15.16:0/3643892470 >> 172.21.15.16:0/3643892470 conn(0x7fbb040b2d10 msgr2=0x7fbb040b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.715+0000 7fbb09477640 1 -- 172.21.15.16:0/3643892470 shutdown_connections 2024-09-19T07:53:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.715+0000 7fbb09477640 1 -- 172.21.15.16:0/3643892470 wait complete. 2024-09-19T07:53:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.716+0000 7fbb09477640 1 Processor -- start 2024-09-19T07:53:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.716+0000 7fbb09477640 1 -- start start 2024-09-19T07:53:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.717+0000 7fbb09477640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb040ccbd0 0x7fbb0414e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.717+0000 7fbb09477640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb0415b300 0x7fbb040b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.717+0000 7fbb09477640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0415b6e0 0x7fbb04151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.717+0000 7fbb09477640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb040b3b70 con 0x7fbb0415b6e0 2024-09-19T07:53:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.717+0000 7fbb09477640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbb040b39f0 con 0x7fbb0415b300 2024-09-19T07:53:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.717+0000 7fbb09477640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbb040b2060 con 0x7fbb040ccbd0 2024-09-19T07:53:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.717+0000 7fbb037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0415b6e0 0x7fbb04151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.717+0000 7fbb037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0415b6e0 0x7fbb04151890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60650/0 (socket says 172.21.15.16:60650) 2024-09-19T07:53:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.717+0000 7fbb037fe640 1 -- 172.21.15.16:0/3111045799 learned_addr learned my addr 172.21.15.16:0/3111045799 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.717+0000 7fbb02ffd640 1 --2- 172.21.15.16:0/3111045799 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb040ccbd0 0x7fbb0414e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.717+0000 7fbb027fc640 1 --2- 172.21.15.16:0/3111045799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb0415b300 0x7fbb040b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.717+0000 7fbb037fe640 1 -- 172.21.15.16:0/3111045799 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb040ccbd0 msgr2=0x7fbb0414e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.717+0000 7fbb037fe640 1 --2- 172.21.15.16:0/3111045799 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb040ccbd0 0x7fbb0414e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.717+0000 7fbb037fe640 1 -- 172.21.15.16:0/3111045799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb0415b300 msgr2=0x7fbb040b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.717+0000 7fbb037fe640 1 --2- 172.21.15.16:0/3111045799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb0415b300 0x7fbb040b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.717+0000 7fbb037fe640 1 -- 172.21.15.16:0/3111045799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0414e6e0 con 0x7fbb0415b6e0 2024-09-19T07:53:03.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.717+0000 7fbb037fe640 1 --2- 172.21.15.16:0/3111045799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0415b6e0 0x7fbb04151890 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400c390 tx=0x7fbaf400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:03.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.718+0000 7fbafb7fe640 1 -- 172.21.15.16:0/3111045799 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4017070 con 0x7fbb0415b6e0 2024-09-19T07:53:03.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.718+0000 7fbafb7fe640 1 -- 172.21.15.16:0/3111045799 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf40052d0 con 0x7fbb0415b6e0 2024-09-19T07:53:03.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.718+0000 7fbb09477640 1 -- 172.21.15.16:0/3111045799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb040b5290 con 0x7fbb0415b6e0 2024-09-19T07:53:03.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.718+0000 7fbafb7fe640 1 -- 172.21.15.16:0/3111045799 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4003810 con 0x7fbb0415b6e0 2024-09-19T07:53:03.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.718+0000 7fbb09477640 1 -- 172.21.15.16:0/3111045799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb040b4cd0 con 0x7fbb0415b6e0 2024-09-19T07:53:03.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.719+0000 7fbb09477640 1 -- 172.21.15.16:0/3111045799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb040cc690 con 0x7fbb0415b6e0 2024-09-19T07:53:03.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.719+0000 7fbafb7fe640 1 -- 172.21.15.16:0/3111045799 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbaf401a070 con 0x7fbb0415b6e0 2024-09-19T07:53:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.720+0000 7fbafb7fe640 1 --2- 172.21.15.16:0/3111045799 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbae406c2b0 0x7fbae406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.720+0000 7fbafb7fe640 1 -- 172.21.15.16:0/3111045799 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7fbaf408a810 con 0x7fbb0415b6e0 2024-09-19T07:53:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.720+0000 7fbb02ffd640 1 --2- 172.21.15.16:0/3111045799 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbae406c2b0 0x7fbae406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.720+0000 7fbb02ffd640 1 --2- 172.21.15.16:0/3111045799 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbae406c2b0 0x7fbae406e770 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbafc033000 tx=0x7fbafc009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:03.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.724+0000 7fbafb7fe640 1 -- 172.21.15.16:0/3111045799 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fbb0414e6e0 con 0x7fbb0415b6e0 2024-09-19T07:53:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.934+0000 7fbb09477640 1 -- 172.21.15.16:0/3111045799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbb040b1080 con 0x7fbb0415b6e0 2024-09-19T07:53:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.952+0000 7fbafb7fe640 1 -- 172.21.15.16:0/3111045799 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) ==== 118+0+0 (secure 0 0 0) 0x7fbaf40587a0 con 0x7fbb0415b6e0 2024-09-19T07:53:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.954+0000 7fbb09477640 1 -- 172.21.15.16:0/3111045799 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbae406c2b0 msgr2=0x7fbae406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.954+0000 7fbb09477640 1 --2- 172.21.15.16:0/3111045799 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbae406c2b0 0x7fbae406e770 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbafc033000 tx=0x7fbafc009000 comp rx=0 tx=0).stop 2024-09-19T07:53:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.954+0000 7fbb09477640 1 -- 172.21.15.16:0/3111045799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0415b6e0 msgr2=0x7fbb04151890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.954+0000 7fbb09477640 1 --2- 172.21.15.16:0/3111045799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0415b6e0 0x7fbb04151890 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400c390 tx=0x7fbaf400c850 comp rx=0 tx=0).stop 2024-09-19T07:53:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.954+0000 7fbb09477640 1 -- 172.21.15.16:0/3111045799 shutdown_connections 2024-09-19T07:53:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.954+0000 7fbb09477640 1 --2- 172.21.15.16:0/3111045799 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbae406c2b0 0x7fbae406e770 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.954+0000 7fbb09477640 1 --2- 172.21.15.16:0/3111045799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0415b6e0 0x7fbb04151890 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.954+0000 7fbb09477640 1 --2- 172.21.15.16:0/3111045799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb0415b300 0x7fbb040b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.954+0000 7fbb09477640 1 --2- 172.21.15.16:0/3111045799 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb040ccbd0 0x7fbb0414e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.954+0000 7fbb09477640 1 -- 172.21.15.16:0/3111045799 >> 172.21.15.16:0/3111045799 conn(0x7fbb040b2d10 msgr2=0x7fbb0406f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.954+0000 7fbb09477640 1 -- 172.21.15.16:0/3111045799 shutdown_connections 2024-09-19T07:53:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:03.954+0000 7fbb09477640 1 -- 172.21.15.16:0/3111045799 wait complete. 2024-09-19T07:53:03.967 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T07:53:03.967 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T07:53:04.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.072+0000 7f5a21386640 1 Processor -- start 2024-09-19T07:53:04.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.072+0000 7f5a21386640 1 -- start start 2024-09-19T07:53:04.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.073+0000 7f5a21386640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a1c072590 0x7f5a1c06d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:04.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.073+0000 7f5a21386640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a1c1620b0 con 0x7f5a1c141ad0 2024-09-19T07:53:04.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.073+0000 7f5a21386640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a1c162370 con 0x7f5a1c0d21f0 2024-09-19T07:53:04.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.073+0000 7f5a21386640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f5a1c141750 con 0x7f5a1c072590 2024-09-19T07:53:04.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.073+0000 7f5a1a7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a1c072590 0x7f5a1c06d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.073+0000 7f5a1b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5a1c141ad0 0x7f5a1c141f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60134/0 (socket says 172.21.15.16:60134) 2024-09-19T07:53:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.073+0000 7f5a1a7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a1c072590 0x7f5a1c06d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60662/0 (socket says 172.21.15.16:60662) 2024-09-19T07:53:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.073+0000 7f5a1a7fc640 1 -- 172.21.15.16:0/3678974048 learned_addr learned my addr 172.21.15.16:0/3678974048 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.073+0000 7f5a1a7fc640 1 -- 172.21.15.16:0/3678974048 >> v1:172.21.15.146:6789/0 conn(0x7f5a1c0d21f0 legacy=0x7f5a1c0d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.073+0000 7f5a1a7fc640 1 -- 172.21.15.16:0/3678974048 >> v1:172.21.15.16:6789/0 conn(0x7f5a1c141ad0 legacy=0x7f5a1c141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.073+0000 7f5a19ffb640 1 -- 172.21.15.16:0/3678974048 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 594964879 0 0) 0x7f5a1c1620b0 con 0x7f5a1c141ad0 2024-09-19T07:53:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.074+0000 7f5a1a7fc640 1 -- 172.21.15.16:0/3678974048 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1c162650 con 0x7f5a1c072590 2024-09-19T07:53:04.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.074+0000 7f5a1a7fc640 1 --2- 172.21.15.16:0/3678974048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c072590 0x7f5a1c06d0e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f5a04009870 tx=0x7f5a0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81243972dde0829b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:04.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.074+0000 7f5a19ffb640 1 -- 172.21.15.16:0/3678974048 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04037070 con 0x7f5a1c072590 2024-09-19T07:53:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.074+0000 7f5a19ffb640 1 -- 172.21.15.16:0/3678974048 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a04032440 con 0x7f5a1c072590 2024-09-19T07:53:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.074+0000 7f5a19ffb640 1 -- 172.21.15.16:0/3678974048 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04032810 con 0x7f5a1c072590 2024-09-19T07:53:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.075+0000 7f5a21386640 1 -- 172.21.15.16:0/3678974048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c072590 msgr2=0x7f5a1c06d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.075+0000 7f5a21386640 1 --2- 172.21.15.16:0/3678974048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c072590 0x7f5a1c06d0e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f5a04009870 tx=0x7f5a0402a270 comp rx=0 tx=0).stop 2024-09-19T07:53:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.075+0000 7f5a21386640 1 -- 172.21.15.16:0/3678974048 shutdown_connections 2024-09-19T07:53:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.075+0000 7f5a21386640 1 --2- 172.21.15.16:0/3678974048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c072590 0x7f5a1c06d0e0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.075+0000 7f5a21386640 1 -- 172.21.15.16:0/3678974048 >> 172.21.15.16:0/3678974048 conn(0x7f5a1c0b3ed0 msgr2=0x7f5a1c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.075+0000 7f5a21386640 1 -- 172.21.15.16:0/3678974048 shutdown_connections 2024-09-19T07:53:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.075+0000 7f5a21386640 1 -- 172.21.15.16:0/3678974048 wait complete. 2024-09-19T07:53:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.076+0000 7f5a21386640 1 Processor -- start 2024-09-19T07:53:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.076+0000 7f5a21386640 1 -- start start 2024-09-19T07:53:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.077+0000 7f5a21386640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a1c072590 0x7f5a1c140120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.077+0000 7f5a21386640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a1c0d21f0 0x7f5a1c0b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.077+0000 7f5a21386640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c141ad0 0x7f5a1c0b6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.077+0000 7f5a21386640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c06d620 con 0x7f5a1c141ad0 2024-09-19T07:53:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.077+0000 7f5a21386640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c162c10 con 0x7f5a1c0d21f0 2024-09-19T07:53:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.077+0000 7f5a21386640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5a1c06d7a0 con 0x7f5a1c072590 2024-09-19T07:53:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.077+0000 7f5a1b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c141ad0 0x7f5a1c0b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.077+0000 7f5a1b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c141ad0 0x7f5a1c0b6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60674/0 (socket says 172.21.15.16:60674) 2024-09-19T07:53:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.077+0000 7f5a1b7fe640 1 -- 172.21.15.16:0/1758664071 learned_addr learned my addr 172.21.15.16:0/1758664071 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.077+0000 7f5a1affd640 1 --2- 172.21.15.16:0/1758664071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a1c072590 0x7f5a1c140120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.077+0000 7f5a1a7fc640 1 --2- 172.21.15.16:0/1758664071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a1c0d21f0 0x7f5a1c0b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.078+0000 7f5a1b7fe640 1 -- 172.21.15.16:0/1758664071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a1c072590 msgr2=0x7f5a1c140120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.078+0000 7f5a1b7fe640 1 --2- 172.21.15.16:0/1758664071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a1c072590 0x7f5a1c140120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.078+0000 7f5a1b7fe640 1 -- 172.21.15.16:0/1758664071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a1c0d21f0 msgr2=0x7f5a1c0b6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.078+0000 7f5a1b7fe640 1 --2- 172.21.15.16:0/1758664071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a1c0d21f0 0x7f5a1c0b6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.078+0000 7f5a1b7fe640 1 -- 172.21.15.16:0/1758664071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1c15d350 con 0x7f5a1c141ad0 2024-09-19T07:53:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.078+0000 7f5a1a7fc640 1 --2- 172.21.15.16:0/1758664071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a1c0d21f0 0x7f5a1c0b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.078+0000 7f5a1b7fe640 1 --2- 172.21.15.16:0/1758664071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c141ad0 0x7f5a1c0b6760 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00c700 tx=0x7f5a0c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.078+0000 7f5a137fe640 1 -- 172.21.15.16:0/1758664071 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c018070 con 0x7f5a1c141ad0 2024-09-19T07:53:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.078+0000 7f5a137fe640 1 -- 172.21.15.16:0/1758664071 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a0c00ce10 con 0x7f5a1c141ad0 2024-09-19T07:53:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.078+0000 7f5a137fe640 1 -- 172.21.15.16:0/1758664071 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c00a860 con 0x7f5a1c141ad0 2024-09-19T07:53:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.078+0000 7f5a21386640 1 -- 172.21.15.16:0/1758664071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1c1f4270 con 0x7f5a1c141ad0 2024-09-19T07:53:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.078+0000 7f5a21386640 1 -- 172.21.15.16:0/1758664071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a1c1f4830 con 0x7f5a1c141ad0 2024-09-19T07:53:04.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.079+0000 7f5a21386640 1 -- 172.21.15.16:0/1758664071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a1c158c30 con 0x7f5a1c141ad0 2024-09-19T07:53:04.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.079+0000 7f5a137fe640 1 -- 172.21.15.16:0/1758664071 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5a0c01b070 con 0x7f5a1c141ad0 2024-09-19T07:53:04.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.080+0000 7f5a137fe640 1 --2- 172.21.15.16:0/1758664071 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f59fc06c2b0 0x7f59fc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:04.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.080+0000 7f5a137fe640 1 -- 172.21.15.16:0/1758664071 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f5a0c08ab50 con 0x7f5a1c141ad0 2024-09-19T07:53:04.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.080+0000 7f5a1affd640 1 --2- 172.21.15.16:0/1758664071 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f59fc06c2b0 0x7f59fc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:04.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.081+0000 7f5a1affd640 1 --2- 172.21.15.16:0/1758664071 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f59fc06c2b0 0x7f59fc06e770 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c161a60 tx=0x7f5a14009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:04.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.084+0000 7f5a137fe640 1 -- 172.21.15.16:0/1758664071 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5a0c0591c0 con 0x7f5a1c141ad0 2024-09-19T07:53:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.297+0000 7f5a21386640 1 -- 172.21.15.16:0/1758664071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f5a1c072290 con 0x7f5a1c141ad0 2024-09-19T07:53:04.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.958+0000 7f5a137fe640 1 -- 172.21.15.16:0/1758664071 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) ==== 122+0+0 (secure 0 0 0) 0x7f5a0c058ac0 con 0x7f5a1c141ad0 2024-09-19T07:53:04.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.961+0000 7f5a21386640 1 -- 172.21.15.16:0/1758664071 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f59fc06c2b0 msgr2=0x7f59fc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:04.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.961+0000 7f5a21386640 1 --2- 172.21.15.16:0/1758664071 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f59fc06c2b0 0x7f59fc06e770 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c161a60 tx=0x7f5a14009040 comp rx=0 tx=0).stop 2024-09-19T07:53:04.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.961+0000 7f5a21386640 1 -- 172.21.15.16:0/1758664071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c141ad0 msgr2=0x7f5a1c0b6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:04.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.961+0000 7f5a21386640 1 --2- 172.21.15.16:0/1758664071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c141ad0 0x7f5a1c0b6760 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00c700 tx=0x7f5a0c00cbc0 comp rx=0 tx=0).stop 2024-09-19T07:53:04.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.961+0000 7f5a21386640 1 -- 172.21.15.16:0/1758664071 shutdown_connections 2024-09-19T07:53:04.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.961+0000 7f5a21386640 1 --2- 172.21.15.16:0/1758664071 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f59fc06c2b0 0x7f59fc06e770 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:04.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.961+0000 7f5a21386640 1 --2- 172.21.15.16:0/1758664071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c141ad0 0x7f5a1c0b6760 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:04.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.961+0000 7f5a21386640 1 --2- 172.21.15.16:0/1758664071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a1c0d21f0 0x7f5a1c0b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:04.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.961+0000 7f5a21386640 1 --2- 172.21.15.16:0/1758664071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a1c072590 0x7f5a1c140120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:04.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.961+0000 7f5a21386640 1 -- 172.21.15.16:0/1758664071 >> 172.21.15.16:0/1758664071 conn(0x7f5a1c0b3ed0 msgr2=0x7f5a1c160380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:04.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.961+0000 7f5a21386640 1 -- 172.21.15.16:0/1758664071 shutdown_connections 2024-09-19T07:53:04.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:04.961+0000 7f5a21386640 1 -- 172.21.15.16:0/1758664071 wait complete. 2024-09-19T07:53:04.974 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T07:53:04.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:53:05.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.083+0000 7fe7a86d9640 1 Processor -- start 2024-09-19T07:53:05.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.083+0000 7fe7a86d9640 1 -- start start 2024-09-19T07:53:05.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.084+0000 7fe7a86d9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe7a015e2a0 0x7fe7a006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:05.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.084+0000 7fe7a86d9640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe7a015e680 0x7fe7a00cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:05.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.084+0000 7fe7a86d9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7a006d960 0x7fe7a00d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:05.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.084+0000 7fe7a86d9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fe7a00d1880 con 0x7fe7a015e2a0 2024-09-19T07:53:05.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.084+0000 7fe7a86d9640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe7a00d1b60 con 0x7fe7a015e680 2024-09-19T07:53:05.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.084+0000 7fe7a86d9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe7a00cf2d0 con 0x7fe7a006d960 2024-09-19T07:53:05.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.084+0000 7fe7a6475640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe7a015e2a0 0x7fe7a006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:05.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.084+0000 7fe7a6475640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe7a015e2a0 0x7fe7a006d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60676/0 (socket says 172.21.15.16:60676) 2024-09-19T07:53:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.084+0000 7fe7a6475640 1 -- 172.21.15.16:0/716046788 learned_addr learned my addr 172.21.15.16:0/716046788 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.084+0000 7fe7a6475640 1 -- 172.21.15.16:0/716046788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7a006d960 msgr2=0x7fe7a00d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.084+0000 7fe7a5c74640 1 --2- 172.21.15.16:0/716046788 >> v2:172.21.15.146:3300/0 conn(0x7fe7a015e680 0x7fe7a00cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.084+0000 7fe7a6c76640 1 --2- 172.21.15.16:0/716046788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7a006d960 0x7fe7a00d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.084+0000 7fe7a6475640 1 --2- 172.21.15.16:0/716046788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7a006d960 0x7fe7a00d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.084+0000 7fe7a6475640 1 -- 172.21.15.16:0/716046788 >> v2:172.21.15.146:3300/0 conn(0x7fe7a015e680 msgr2=0x7fe7a00cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.084+0000 7fe7a6475640 1 --2- 172.21.15.16:0/716046788 >> v2:172.21.15.146:3300/0 conn(0x7fe7a015e680 0x7fe7a00cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.084+0000 7fe7a6475640 1 -- 172.21.15.16:0/716046788 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a00cf5b0 con 0x7fe7a015e2a0 2024-09-19T07:53:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.084+0000 7fe7a5c74640 1 --2- 172.21.15.16:0/716046788 >> v2:172.21.15.146:3300/0 conn(0x7fe7a015e680 0x7fe7a00cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:05.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.084+0000 7fe7a6c76640 1 --2- 172.21.15.16:0/716046788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7a006d960 0x7fe7a00d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.085+0000 7fe7a6475640 1 --2- 172.21.15.16:0/716046788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7a015e2a0 0x7fe7a006d420 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00a22a0 tx=0x7fe79c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbf716e8663170d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.085+0000 7fe7a5473640 1 -- 172.21.15.16:0/716046788 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c037070 con 0x7fe7a015e2a0 2024-09-19T07:53:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.085+0000 7fe7a5473640 1 -- 172.21.15.16:0/716046788 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe79c032440 con 0x7fe7a015e2a0 2024-09-19T07:53:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.085+0000 7fe7a5473640 1 -- 172.21.15.16:0/716046788 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c032810 con 0x7fe7a015e2a0 2024-09-19T07:53:05.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.086+0000 7fe7a86d9640 1 -- 172.21.15.16:0/716046788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7a015e2a0 msgr2=0x7fe7a006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.086+0000 7fe7a86d9640 1 --2- 172.21.15.16:0/716046788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7a015e2a0 0x7fe7a006d420 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00a22a0 tx=0x7fe79c02a1e0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.086+0000 7fe7a86d9640 1 -- 172.21.15.16:0/716046788 shutdown_connections 2024-09-19T07:53:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.086+0000 7fe7a86d9640 1 --2- 172.21.15.16:0/716046788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7a006d960 0x7fe7a00d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.086+0000 7fe7a86d9640 1 --2- 172.21.15.16:0/716046788 >> v2:172.21.15.146:3300/0 conn(0x7fe7a015e680 0x7fe7a00cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.086+0000 7fe7a86d9640 1 --2- 172.21.15.16:0/716046788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7a015e2a0 0x7fe7a006d420 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.086+0000 7fe7a86d9640 1 -- 172.21.15.16:0/716046788 >> 172.21.15.16:0/716046788 conn(0x7fe7a009d760 msgr2=0x7fe7a008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.086+0000 7fe7a86d9640 1 -- 172.21.15.16:0/716046788 shutdown_connections 2024-09-19T07:53:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.086+0000 7fe7a86d9640 1 -- 172.21.15.16:0/716046788 wait complete. 2024-09-19T07:53:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.087+0000 7fe7a86d9640 1 Processor -- start 2024-09-19T07:53:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.087+0000 7fe7a86d9640 1 -- start start 2024-09-19T07:53:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.088+0000 7fe7a86d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7a006d960 0x7fe7a00caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.088+0000 7fe7a86d9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7a015e2a0 0x7fe7a00ce450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.088+0000 7fe7a86d9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7a015e680 0x7fe7a01e9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:05.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.088+0000 7fe7a86d9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a00d44f0 con 0x7fe7a006d960 2024-09-19T07:53:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.088+0000 7fe7a86d9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a00d4370 con 0x7fe7a015e2a0 2024-09-19T07:53:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.088+0000 7fe7a86d9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe7a00d4670 con 0x7fe7a015e680 2024-09-19T07:53:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.088+0000 7fe7a6475640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7a006d960 0x7fe7a00caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.088+0000 7fe7a6475640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7a006d960 0x7fe7a00caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60684/0 (socket says 172.21.15.16:60684) 2024-09-19T07:53:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.088+0000 7fe7a6475640 1 -- 172.21.15.16:0/3622037588 learned_addr learned my addr 172.21.15.16:0/3622037588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.088+0000 7fe7a6475640 1 -- 172.21.15.16:0/3622037588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7a015e680 msgr2=0x7fe7a01e9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.088+0000 7fe7a6475640 1 --2- 172.21.15.16:0/3622037588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7a015e680 0x7fe7a01e9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.088+0000 7fe7a6475640 1 -- 172.21.15.16:0/3622037588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7a015e2a0 msgr2=0x7fe7a00ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.088+0000 7fe7a6475640 1 --2- 172.21.15.16:0/3622037588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7a015e2a0 0x7fe7a00ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.088+0000 7fe7a6475640 1 -- 172.21.15.16:0/3622037588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a00cb100 con 0x7fe7a006d960 2024-09-19T07:53:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.088+0000 7fe7a6475640 1 --2- 172.21.15.16:0/3622037588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7a006d960 0x7fe7a00caae0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe79c02a220 tx=0x7fe79c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.088+0000 7fe792ffd640 1 -- 172.21.15.16:0/3622037588 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c037040 con 0x7fe7a006d960 2024-09-19T07:53:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.089+0000 7fe792ffd640 1 -- 172.21.15.16:0/3622037588 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe79c042070 con 0x7fe7a006d960 2024-09-19T07:53:05.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.089+0000 7fe7a86d9640 1 -- 172.21.15.16:0/3622037588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a01e8f30 con 0x7fe7a006d960 2024-09-19T07:53:05.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.089+0000 7fe792ffd640 1 -- 172.21.15.16:0/3622037588 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c030bb0 con 0x7fe7a006d960 2024-09-19T07:53:05.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.089+0000 7fe7a86d9640 1 -- 172.21.15.16:0/3622037588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a01e94f0 con 0x7fe7a006d960 2024-09-19T07:53:05.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.089+0000 7fe7a86d9640 1 -- 172.21.15.16:0/3622037588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7a006d420 con 0x7fe7a006d960 2024-09-19T07:53:05.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.090+0000 7fe792ffd640 1 -- 172.21.15.16:0/3622037588 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe79c047460 con 0x7fe7a006d960 2024-09-19T07:53:05.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.091+0000 7fe792ffd640 1 --2- 172.21.15.16:0/3622037588 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe78406c2b0 0x7fe78406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:05.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.091+0000 7fe792ffd640 1 -- 172.21.15.16:0/3622037588 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fe79c0ac290 con 0x7fe7a006d960 2024-09-19T07:53:05.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.091+0000 7fe7a5c74640 1 --2- 172.21.15.16:0/3622037588 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe78406c2b0 0x7fe78406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:05.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.091+0000 7fe7a5c74640 1 --2- 172.21.15.16:0/3622037588 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe78406c2b0 0x7fe78406e770 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe794006b10 tx=0x7fe794008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:05.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.095+0000 7fe792ffd640 1 -- 172.21.15.16:0/3622037588 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe79c045360 con 0x7fe7a006d960 2024-09-19T07:53:05.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.307+0000 7fe7a86d9640 1 -- 172.21.15.16:0/3622037588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7a006cf90 con 0x7fe7a006d960 2024-09-19T07:53:05.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.307+0000 7fe792ffd640 1 -- 172.21.15.16:0/3622037588 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5228 (secure 0 0 0) 0x7fe79c07a870 con 0x7fe7a006d960 2024-09-19T07:53:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:05.308 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":56,"btime":"2024-09-19T07:53:04:953484+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6017,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2955839490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":2955839490},{"type":"v1","addr":"172.21.15.146:6837","nonce":2955839490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6550,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/395357736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":395357736},{"type":"v1","addr":"172.21.15.146:6836","nonce":395357736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3539638739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3539638739},{"type":"v1","addr":"172.21.15.16:6834","nonce":3539638739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:53:02.264799+0000","modified":"2024-09-19T07:53:04.298337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/4185904511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4185904511},{"type":"v1","addr":"172.21.15.16:6835","nonce":4185904511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":7}]} 2024-09-19T07:53:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 56 2024-09-19T07:53:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.309+0000 7fe7a86d9640 1 -- 172.21.15.16:0/3622037588 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe78406c2b0 msgr2=0x7fe78406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.309+0000 7fe7a86d9640 1 --2- 172.21.15.16:0/3622037588 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe78406c2b0 0x7fe78406e770 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe794006b10 tx=0x7fe794008040 comp rx=0 tx=0).stop 2024-09-19T07:53:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.309+0000 7fe7a86d9640 1 -- 172.21.15.16:0/3622037588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7a006d960 msgr2=0x7fe7a00caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.310+0000 7fe7a86d9640 1 --2- 172.21.15.16:0/3622037588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7a006d960 0x7fe7a00caae0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe79c02a220 tx=0x7fe79c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.310+0000 7fe7a86d9640 1 -- 172.21.15.16:0/3622037588 shutdown_connections 2024-09-19T07:53:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.310+0000 7fe7a86d9640 1 --2- 172.21.15.16:0/3622037588 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe78406c2b0 0x7fe78406e770 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.310+0000 7fe7a86d9640 1 --2- 172.21.15.16:0/3622037588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7a015e680 0x7fe7a01e9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.310+0000 7fe7a86d9640 1 --2- 172.21.15.16:0/3622037588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7a015e2a0 0x7fe7a00ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.310+0000 7fe7a86d9640 1 --2- 172.21.15.16:0/3622037588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7a006d960 0x7fe7a00caae0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.310+0000 7fe7a86d9640 1 -- 172.21.15.16:0/3622037588 >> 172.21.15.16:0/3622037588 conn(0x7fe7a009d760 msgr2=0x7fe7a00d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:05.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.310+0000 7fe7a86d9640 1 -- 172.21.15.16:0/3622037588 shutdown_connections 2024-09-19T07:53:05.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.310+0000 7fe7a86d9640 1 -- 172.21.15.16:0/3622037588 wait complete. 2024-09-19T07:53:05.322 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:53:05.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.424+0000 7fdb919b7640 1 Processor -- start 2024-09-19T07:53:05.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.424+0000 7fdb919b7640 1 -- start start 2024-09-19T07:53:05.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.425+0000 7fdb919b7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdb8c15e680 0x7fdb8c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:05.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.425+0000 7fdb919b7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdb8c0d0010 0x7fdb8c0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:05.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.425+0000 7fdb919b7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb8c0d1880 con 0x7fdb8c15e2a0 2024-09-19T07:53:05.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.425+0000 7fdb919b7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fdb8c0d1b60 con 0x7fdb8c0d0010 2024-09-19T07:53:05.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.425+0000 7fdb919b7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fdb8c0cf2d0 con 0x7fdb8c15e680 2024-09-19T07:53:05.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.425+0000 7fdb8b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdb8c0d0010 0x7fdb8c0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:05.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.425+0000 7fdb8affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdb8c15e2a0 0x7fdb8c06d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60150/0 (socket says 172.21.15.16:60150) 2024-09-19T07:53:05.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.425+0000 7fdb8affd640 1 -- 172.21.15.16:0/2059313622 learned_addr learned my addr 172.21.15.16:0/2059313622 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:05.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.426+0000 7fdb8b7fe640 1 -- 172.21.15.16:0/2059313622 >> v2:172.21.15.146:3300/0 conn(0x7fdb8c15e680 msgr2=0x7fdb8c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:05.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.426+0000 7fdb8a7fc640 1 --2- 172.21.15.16:0/2059313622 >> v2:172.21.15.146:3300/0 conn(0x7fdb8c15e680 0x7fdb8c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:05.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.426+0000 7fdb89ffb640 1 -- 172.21.15.16:0/2059313622 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2576372709 0 0) 0x7fdb8c0d1880 con 0x7fdb8c15e2a0 2024-09-19T07:53:05.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.426+0000 7fdb8b7fe640 1 --2- 172.21.15.16:0/2059313622 >> v2:172.21.15.146:3300/0 conn(0x7fdb8c15e680 0x7fdb8c0cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.426+0000 7fdb8b7fe640 1 -- 172.21.15.16:0/2059313622 >> v1:172.21.15.16:6789/0 conn(0x7fdb8c15e2a0 legacy=0x7fdb8c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:05.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.426+0000 7fdb8b7fe640 1 -- 172.21.15.16:0/2059313622 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8c0cf5b0 con 0x7fdb8c0d0010 2024-09-19T07:53:05.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.426+0000 7fdb8b7fe640 1 --2- 172.21.15.16:0/2059313622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c0d0010 0x7fdb8c0d4370 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0a22a0 tx=0x7fdb8402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96ddd3011f4ef64 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:05.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.427+0000 7fdb89ffb640 1 -- 172.21.15.16:0/2059313622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84007740 con 0x7fdb8c0d0010 2024-09-19T07:53:05.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.427+0000 7fdb89ffb640 1 -- 172.21.15.16:0/2059313622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb840078e0 con 0x7fdb8c0d0010 2024-09-19T07:53:05.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.427+0000 7fdb89ffb640 1 -- 172.21.15.16:0/2059313622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84007cb0 con 0x7fdb8c0d0010 2024-09-19T07:53:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.427+0000 7fdb919b7640 1 -- 172.21.15.16:0/2059313622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c0d0010 msgr2=0x7fdb8c0d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.427+0000 7fdb919b7640 1 --2- 172.21.15.16:0/2059313622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c0d0010 0x7fdb8c0d4370 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0a22a0 tx=0x7fdb8402a5f0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.427+0000 7fdb919b7640 1 -- 172.21.15.16:0/2059313622 shutdown_connections 2024-09-19T07:53:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.427+0000 7fdb919b7640 1 --2- 172.21.15.16:0/2059313622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c0d0010 0x7fdb8c0d4370 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.427+0000 7fdb919b7640 1 --2- 172.21.15.16:0/2059313622 >> v2:172.21.15.146:3300/0 conn(0x7fdb8c15e680 0x7fdb8c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.427+0000 7fdb919b7640 1 -- 172.21.15.16:0/2059313622 >> 172.21.15.16:0/2059313622 conn(0x7fdb8c09d760 msgr2=0x7fdb8c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.427+0000 7fdb919b7640 1 -- 172.21.15.16:0/2059313622 shutdown_connections 2024-09-19T07:53:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.427+0000 7fdb919b7640 1 -- 172.21.15.16:0/2059313622 wait complete. 2024-09-19T07:53:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.428+0000 7fdb919b7640 1 Processor -- start 2024-09-19T07:53:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.429+0000 7fdb919b7640 1 -- start start 2024-09-19T07:53:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.429+0000 7fdb919b7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb8c0d0010 0x7fdb8c154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.429+0000 7fdb919b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb8c15e2a0 0x7fdb8c0a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.429+0000 7fdb919b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c15e680 0x7fdb8c0a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.429+0000 7fdb919b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8c06dcb0 con 0x7fdb8c15e680 2024-09-19T07:53:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.429+0000 7fdb919b7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8c06db30 con 0x7fdb8c0d0010 2024-09-19T07:53:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.429+0000 7fdb919b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdb8c0d48b0 con 0x7fdb8c15e2a0 2024-09-19T07:53:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.429+0000 7fdb8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c15e680 0x7fdb8c0a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.429+0000 7fdb8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c15e680 0x7fdb8c0a1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60706/0 (socket says 172.21.15.16:60706) 2024-09-19T07:53:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.429+0000 7fdb8b7fe640 1 -- 172.21.15.16:0/1112076627 learned_addr learned my addr 172.21.15.16:0/1112076627 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.429+0000 7fdb8affd640 1 --2- 172.21.15.16:0/1112076627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb8c0d0010 0x7fdb8c154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.429+0000 7fdb8a7fc640 1 --2- 172.21.15.16:0/1112076627 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb8c15e2a0 0x7fdb8c0a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.430+0000 7fdb8b7fe640 1 -- 172.21.15.16:0/1112076627 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb8c15e2a0 msgr2=0x7fdb8c0a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.430+0000 7fdb8b7fe640 1 --2- 172.21.15.16:0/1112076627 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb8c15e2a0 0x7fdb8c0a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.430+0000 7fdb8b7fe640 1 -- 172.21.15.16:0/1112076627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb8c0d0010 msgr2=0x7fdb8c154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.430+0000 7fdb8b7fe640 1 --2- 172.21.15.16:0/1112076627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb8c0d0010 0x7fdb8c154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.430+0000 7fdb8b7fe640 1 -- 172.21.15.16:0/1112076627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8c0ce190 con 0x7fdb8c15e680 2024-09-19T07:53:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.430+0000 7fdb8b7fe640 1 --2- 172.21.15.16:0/1112076627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c15e680 0x7fdb8c0a1240 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009e80 tx=0x7fdb84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.430+0000 7fdb73fff640 1 -- 172.21.15.16:0/1112076627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84040070 con 0x7fdb8c15e680 2024-09-19T07:53:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.430+0000 7fdb73fff640 1 -- 172.21.15.16:0/1112076627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb84031c40 con 0x7fdb8c15e680 2024-09-19T07:53:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.430+0000 7fdb919b7640 1 -- 172.21.15.16:0/1112076627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8c0ce470 con 0x7fdb8c15e680 2024-09-19T07:53:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.430+0000 7fdb73fff640 1 -- 172.21.15.16:0/1112076627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84003990 con 0x7fdb8c15e680 2024-09-19T07:53:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.430+0000 7fdb919b7640 1 -- 172.21.15.16:0/1112076627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb8c0cb9e0 con 0x7fdb8c15e680 2024-09-19T07:53:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.431+0000 7fdb919b7640 1 -- 172.21.15.16:0/1112076627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb8c0cbbe0 con 0x7fdb8c15e680 2024-09-19T07:53:05.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.432+0000 7fdb73fff640 1 -- 172.21.15.16:0/1112076627 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdb84039040 con 0x7fdb8c15e680 2024-09-19T07:53:05.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.432+0000 7fdb73fff640 1 --2- 172.21.15.16:0/1112076627 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdb7806c2b0 0x7fdb7806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:05.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.432+0000 7fdb73fff640 1 -- 172.21.15.16:0/1112076627 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fdb840ac890 con 0x7fdb8c15e680 2024-09-19T07:53:05.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.432+0000 7fdb8affd640 1 --2- 172.21.15.16:0/1112076627 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdb7806c2b0 0x7fdb7806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:05.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.433+0000 7fdb8affd640 1 --2- 172.21.15.16:0/1112076627 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdb7806c2b0 0x7fdb7806e770 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c000f60 tx=0x7fdb7c007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.437+0000 7fdb73fff640 1 -- 172.21.15.16:0/1112076627 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fdb8c0cbbe0 con 0x7fdb8c15e680 2024-09-19T07:53:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.612+0000 7fdb919b7640 1 -- 172.21.15.16:0/1112076627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb8c06cf90 con 0x7fdb8c15e680 2024-09-19T07:53:05.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.613+0000 7fdb73fff640 1 -- 172.21.15.16:0/1112076627 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16968 (secure 0 0 0) 0x7fdb8407a820 con 0x7fdb8c15e680 2024-09-19T07:53:05.614 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:05.614 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":106,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:53:03.939259+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:52:59.931281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-19T07:53:01.258168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:53:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.616+0000 7fdb919b7640 1 -- 172.21.15.16:0/1112076627 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdb7806c2b0 msgr2=0x7fdb7806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.616+0000 7fdb919b7640 1 --2- 172.21.15.16:0/1112076627 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdb7806c2b0 0x7fdb7806e770 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c000f60 tx=0x7fdb7c007d50 comp rx=0 tx=0).stop 2024-09-19T07:53:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.616+0000 7fdb919b7640 1 -- 172.21.15.16:0/1112076627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c15e680 msgr2=0x7fdb8c0a1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.616+0000 7fdb919b7640 1 --2- 172.21.15.16:0/1112076627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c15e680 0x7fdb8c0a1240 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009e80 tx=0x7fdb84004a40 comp rx=0 tx=0).stop 2024-09-19T07:53:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.616+0000 7fdb919b7640 1 -- 172.21.15.16:0/1112076627 shutdown_connections 2024-09-19T07:53:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.616+0000 7fdb919b7640 1 --2- 172.21.15.16:0/1112076627 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdb7806c2b0 0x7fdb7806e770 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.616+0000 7fdb919b7640 1 --2- 172.21.15.16:0/1112076627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c15e680 0x7fdb8c0a1240 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.616+0000 7fdb919b7640 1 --2- 172.21.15.16:0/1112076627 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb8c15e2a0 0x7fdb8c0a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.616+0000 7fdb919b7640 1 --2- 172.21.15.16:0/1112076627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb8c0d0010 0x7fdb8c154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.616+0000 7fdb919b7640 1 -- 172.21.15.16:0/1112076627 >> 172.21.15.16:0/1112076627 conn(0x7fdb8c09d760 msgr2=0x7fdb8c0b9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.616+0000 7fdb919b7640 1 -- 172.21.15.16:0/1112076627 shutdown_connections 2024-09-19T07:53:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.616+0000 7fdb919b7640 1 -- 172.21.15.16:0/1112076627 wait complete. 2024-09-19T07:53:05.627 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T07:53:05.628 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:53:05.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.729+0000 7f13bef86640 1 Processor -- start 2024-09-19T07:53:05.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.729+0000 7f13bef86640 1 -- start start 2024-09-19T07:53:05.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.730+0000 7f13bef86640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b80b03c0 0x7f13b80cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:05.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.730+0000 7f13bef86640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f13b80cc680 0x7f13b80ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:05.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.730+0000 7f13bef86640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13b80b3c10 con 0x7f13b815e7f0 2024-09-19T07:53:05.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.730+0000 7f13bef86640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f13b80cc1f0 con 0x7f13b80cc680 2024-09-19T07:53:05.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.730+0000 7f13bef86640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f13b80cb900 con 0x7f13b80b03c0 2024-09-19T07:53:05.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.730+0000 7f13bd523640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f13b80cc680 0x7f13b80ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:05.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.730+0000 7f13b7fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b80b03c0 0x7f13b80cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:05.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.730+0000 7f13bd523640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f13b80cc680 0x7f13b80ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:48058/0 (socket says 172.21.15.16:48058) 2024-09-19T07:53:05.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.730+0000 7f13bd523640 1 -- 172.21.15.16:0/3195510531 learned_addr learned my addr 172.21.15.16:0/3195510531 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:05.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.731+0000 7f13b77fe640 1 -- 172.21.15.16:0/3195510531 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 505946724 0 0) 0x7f13b80b3c10 con 0x7f13b815e7f0 2024-09-19T07:53:05.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.731+0000 7f13b77fe640 1 -- 172.21.15.16:0/3195510531 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13a0003520 con 0x7f13b815e7f0 2024-09-19T07:53:05.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.731+0000 7f13bd523640 1 -- 172.21.15.16:0/3195510531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b80b03c0 msgr2=0x7f13b80cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:05.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.731+0000 7f13bd523640 1 --2- 172.21.15.16:0/3195510531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b80b03c0 0x7f13b80cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.731+0000 7f13bd523640 1 -- 172.21.15.16:0/3195510531 >> v1:172.21.15.146:6789/0 conn(0x7f13b815e7f0 legacy=0x7f13b80adff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:05.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.731+0000 7f13bd523640 1 -- 172.21.15.16:0/3195510531 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b815ebd0 con 0x7f13b80cc680 2024-09-19T07:53:05.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.731+0000 7f13b7fff640 1 --2- 172.21.15.16:0/3195510531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b80b03c0 0x7f13b80cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:53:05.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.731+0000 7f13bd523640 1 --2- 172.21.15.16:0/3195510531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13b80cc680 0x7f13b80ccb30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009e80 tx=0x7f13b002a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=331af49410f32680 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:05.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.732+0000 7f13b77fe640 1 -- 172.21.15.16:0/3195510531 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0036070 con 0x7f13b80cc680 2024-09-19T07:53:05.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.732+0000 7f13b77fe640 1 -- 172.21.15.16:0/3195510531 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b0031600 con 0x7f13b80cc680 2024-09-19T07:53:05.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.732+0000 7f13b77fe640 1 -- 172.21.15.16:0/3195510531 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b00319d0 con 0x7f13b80cc680 2024-09-19T07:53:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.732+0000 7f13bef86640 1 -- 172.21.15.16:0/3195510531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13b80cc680 msgr2=0x7f13b80ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.732+0000 7f13bef86640 1 --2- 172.21.15.16:0/3195510531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13b80cc680 0x7f13b80ccb30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009e80 tx=0x7f13b002a6c0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.732+0000 7f13bef86640 1 -- 172.21.15.16:0/3195510531 shutdown_connections 2024-09-19T07:53:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.732+0000 7f13bef86640 1 --2- 172.21.15.16:0/3195510531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13b80cc680 0x7f13b80ccb30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.732+0000 7f13bef86640 1 --2- 172.21.15.16:0/3195510531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b80b03c0 0x7f13b80cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.732+0000 7f13bef86640 1 -- 172.21.15.16:0/3195510531 >> 172.21.15.16:0/3195510531 conn(0x7f13b80b2d10 msgr2=0x7f13b80b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.732+0000 7f13bef86640 1 -- 172.21.15.16:0/3195510531 shutdown_connections 2024-09-19T07:53:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.732+0000 7f13bef86640 1 -- 172.21.15.16:0/3195510531 wait complete. 2024-09-19T07:53:05.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.733+0000 7f13bef86640 1 Processor -- start 2024-09-19T07:53:05.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.733+0000 7f13bef86640 1 -- start start 2024-09-19T07:53:05.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.734+0000 7f13bef86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b80b03c0 0x7f13b8149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:05.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.734+0000 7f13bef86640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13b80cc680 0x7f13b80c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:05.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.734+0000 7f13bef86640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b815e7f0 0x7f13b80c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:05.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.734+0000 7f13bef86640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13b80ae700 con 0x7f13b80b03c0 2024-09-19T07:53:05.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.734+0000 7f13bef86640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f13b80cbe80 con 0x7f13b80cc680 2024-09-19T07:53:05.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.734+0000 7f13bef86640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f13b80b3830 con 0x7f13b815e7f0 2024-09-19T07:53:05.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.734+0000 7f13bcd22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b80b03c0 0x7f13b8149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:05.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.734+0000 7f13bcd22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b80b03c0 0x7f13b8149470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60720/0 (socket says 172.21.15.16:60720) 2024-09-19T07:53:05.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.734+0000 7f13bcd22640 1 -- 172.21.15.16:0/2189997130 learned_addr learned my addr 172.21.15.16:0/2189997130 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.735+0000 7f13bd523640 1 --2- 172.21.15.16:0/2189997130 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b815e7f0 0x7f13b80c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.735+0000 7f13b7fff640 1 --2- 172.21.15.16:0/2189997130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13b80cc680 0x7f13b80c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.735+0000 7f13bcd22640 1 -- 172.21.15.16:0/2189997130 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b815e7f0 msgr2=0x7f13b80c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.735+0000 7f13bcd22640 1 --2- 172.21.15.16:0/2189997130 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b815e7f0 0x7f13b80c6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.735+0000 7f13bcd22640 1 -- 172.21.15.16:0/2189997130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13b80cc680 msgr2=0x7f13b80c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.735+0000 7f13bcd22640 1 --2- 172.21.15.16:0/2189997130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13b80cc680 0x7f13b80c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.735+0000 7f13bcd22640 1 -- 172.21.15.16:0/2189997130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b81499b0 con 0x7f13b80b03c0 2024-09-19T07:53:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.735+0000 7f13bcd22640 1 --2- 172.21.15.16:0/2189997130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b80b03c0 0x7f13b8149470 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f13a800c530 tx=0x7f13a800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.735+0000 7f13b57fa640 1 -- 172.21.15.16:0/2189997130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a80052d0 con 0x7f13b80b03c0 2024-09-19T07:53:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.735+0000 7f13b57fa640 1 -- 172.21.15.16:0/2189997130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13a8005490 con 0x7f13b80b03c0 2024-09-19T07:53:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.735+0000 7f13bef86640 1 -- 172.21.15.16:0/2189997130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b80c9030 con 0x7f13b80b03c0 2024-09-19T07:53:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.735+0000 7f13b57fa640 1 -- 172.21.15.16:0/2189997130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8012870 con 0x7f13b80b03c0 2024-09-19T07:53:05.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.735+0000 7f13bef86640 1 -- 172.21.15.16:0/2189997130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b80c95f0 con 0x7f13b80b03c0 2024-09-19T07:53:05.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.736+0000 7f13bef86640 1 -- 172.21.15.16:0/2189997130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13b80cd360 con 0x7f13b80b03c0 2024-09-19T07:53:05.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.737+0000 7f13b57fa640 1 -- 172.21.15.16:0/2189997130 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f13a8012a10 con 0x7f13b80b03c0 2024-09-19T07:53:05.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.737+0000 7f13b57fa640 1 --2- 172.21.15.16:0/2189997130 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f13a406c2b0 0x7f13a406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:05.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.737+0000 7f13b57fa640 1 -- 172.21.15.16:0/2189997130 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f13a808b170 con 0x7f13b80b03c0 2024-09-19T07:53:05.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.737+0000 7f13b7fff640 1 --2- 172.21.15.16:0/2189997130 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f13a406c2b0 0x7f13a406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:05.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.738+0000 7f13b7fff640 1 --2- 172.21.15.16:0/2189997130 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f13a406c2b0 0x7f13a406e770 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009560 tx=0x7f13ac009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:05.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.742+0000 7f13b57fa640 1 -- 172.21.15.16:0/2189997130 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f13a8059070 con 0x7f13b80b03c0 2024-09-19T07:53:05.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.914+0000 7f13bef86640 1 -- 172.21.15.16:0/2189997130 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f13b80b2ac0 con 0x7f13a406c2b0 2024-09-19T07:53:05.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.920+0000 7f13b57fa640 1 -- 172.21.15.16:0/2189997130 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250259 (secure 0 0 0) 0x7f13b80b2ac0 con 0x7f13a406c2b0 2024-09-19T07:53:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:53:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.923+0000 7f13bef86640 1 -- 172.21.15.16:0/2189997130 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f13a406c2b0 msgr2=0x7f13a406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.923+0000 7f13bef86640 1 --2- 172.21.15.16:0/2189997130 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f13a406c2b0 0x7f13a406e770 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009560 tx=0x7f13ac009190 comp rx=0 tx=0).stop 2024-09-19T07:53:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.923+0000 7f13bef86640 1 -- 172.21.15.16:0/2189997130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b80b03c0 msgr2=0x7f13b8149470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.923+0000 7f13bef86640 1 --2- 172.21.15.16:0/2189997130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b80b03c0 0x7f13b8149470 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f13a800c530 tx=0x7f13a800c9f0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.923+0000 7f13bef86640 1 -- 172.21.15.16:0/2189997130 shutdown_connections 2024-09-19T07:53:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.923+0000 7f13bef86640 1 --2- 172.21.15.16:0/2189997130 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f13a406c2b0 0x7f13a406e770 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.923+0000 7f13bef86640 1 --2- 172.21.15.16:0/2189997130 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13b815e7f0 0x7f13b80c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.923+0000 7f13bef86640 1 --2- 172.21.15.16:0/2189997130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13b80cc680 0x7f13b80c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.923+0000 7f13bef86640 1 --2- 172.21.15.16:0/2189997130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b80b03c0 0x7f13b8149470 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.923+0000 7f13bef86640 1 -- 172.21.15.16:0/2189997130 >> 172.21.15.16:0/2189997130 conn(0x7f13b80b2d10 msgr2=0x7f13b80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.923+0000 7f13bef86640 1 -- 172.21.15.16:0/2189997130 shutdown_connections 2024-09-19T07:53:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:05.923+0000 7f13bef86640 1 -- 172.21.15.16:0/2189997130 wait complete. 2024-09-19T07:53:05.934 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-09-19T07:53:04.600078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":200,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":756168,"kb_used_data":11344,"kb_used_omap":12,"kb_used_meta":744627,"kb_avail":754218552,"statfs":{"total":773094113280,"available":772319797248,"internally_reserved":0,"allocated":11616256,"data_stored":5977828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":762498608},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000976"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959791+0000","last_change":"2024-09-19T07:53:02.969667+0000","last_active":"2024-09-19T07:53:03.959791+0000","last_peered":"2024-09-19T07:53:03.959791+0000","last_clean":"2024-09-19T07:53:03.959791+0000","last_became_active":"2024-09-19T07:53:02.969269+0000","last_became_peered":"2024-09-19T07:53:02.969269+0000","last_unstale":"2024-09-19T07:53:03.959791+0000","last_undegraded":"2024-09-19T07:53:03.959791+0000","last_fullsized":"2024-09-19T07:53:03.959791+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:21.528651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089709+0000","last_change":"2024-09-19T07:53:02.960077+0000","last_active":"2024-09-19T07:53:04.089709+0000","last_peered":"2024-09-19T07:53:04.089709+0000","last_clean":"2024-09-19T07:53:04.089709+0000","last_became_active":"2024-09-19T07:53:02.959395+0000","last_became_peered":"2024-09-19T07:53:02.959395+0000","last_unstale":"2024-09-19T07:53:04.089709+0000","last_undegraded":"2024-09-19T07:53:04.089709+0000","last_fullsized":"2024-09-19T07:53:04.089709+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959657+0000","last_change":"2024-09-19T07:53:02.961895+0000","last_active":"2024-09-19T07:53:03.959657+0000","last_peered":"2024-09-19T07:53:03.959657+0000","last_clean":"2024-09-19T07:53:03.959657+0000","last_became_active":"2024-09-19T07:53:02.961616+0000","last_became_peered":"2024-09-19T07:53:02.961616+0000","last_unstale":"2024-09-19T07:53:03.959657+0000","last_undegraded":"2024-09-19T07:53:03.959657+0000","last_fullsized":"2024-09-19T07:53:03.959657+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:01.957812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.962372+0000","last_change":"2024-09-19T07:53:02.961725+0000","last_active":"2024-09-19T07:53:02.962372+0000","last_peered":"2024-09-19T07:53:02.962372+0000","last_clean":"2024-09-19T07:53:02.962372+0000","last_became_active":"2024-09-19T07:53:02.961334+0000","last_became_peered":"2024-09-19T07:53:02.961334+0000","last_unstale":"2024-09-19T07:53:02.962372+0000","last_undegraded":"2024-09-19T07:53:02.962372+0000","last_fullsized":"2024-09-19T07:53:02.962372+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:04.795110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.946689+0000","last_change":"2024-09-19T07:53:02.945749+0000","last_active":"2024-09-19T07:53:02.946689+0000","last_peered":"2024-09-19T07:53:02.946689+0000","last_clean":"2024-09-19T07:53:02.946689+0000","last_became_active":"2024-09-19T07:53:02.945466+0000","last_became_peered":"2024-09-19T07:53:02.945466+0000","last_unstale":"2024-09-19T07:53:02.946689+0000","last_undegraded":"2024-09-19T07:53:02.946689+0000","last_fullsized":"2024-09-19T07:53:02.946689+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:28.288866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088937+0000","last_change":"2024-09-19T07:53:02.960167+0000","last_active":"2024-09-19T07:53:04.088937+0000","last_peered":"2024-09-19T07:53:04.088937+0000","last_clean":"2024-09-19T07:53:04.088937+0000","last_became_active":"2024-09-19T07:53:02.959874+0000","last_became_peered":"2024-09-19T07:53:02.959874+0000","last_unstale":"2024-09-19T07:53:04.088937+0000","last_undegraded":"2024-09-19T07:53:04.088937+0000","last_fullsized":"2024-09-19T07:53:04.088937+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:17.832687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088822+0000","last_change":"2024-09-19T07:53:02.973657+0000","last_active":"2024-09-19T07:53:04.088822+0000","last_peered":"2024-09-19T07:53:04.088822+0000","last_clean":"2024-09-19T07:53:04.088822+0000","last_became_active":"2024-09-19T07:53:02.973421+0000","last_became_peered":"2024-09-19T07:53:02.973421+0000","last_unstale":"2024-09-19T07:53:04.088822+0000","last_undegraded":"2024-09-19T07:53:04.088822+0000","last_fullsized":"2024-09-19T07:53:04.088822+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:28.288866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089588+0000","last_change":"2024-09-19T07:53:02.966890+0000","last_active":"2024-09-19T07:53:04.089588+0000","last_peered":"2024-09-19T07:53:04.089588+0000","last_clean":"2024-09-19T07:53:04.089588+0000","last_became_active":"2024-09-19T07:53:02.966573+0000","last_became_peered":"2024-09-19T07:53:02.966573+0000","last_unstale":"2024-09-19T07:53:04.089588+0000","last_undegraded":"2024-09-19T07:53:04.089588+0000","last_fullsized":"2024-09-19T07:53:04.089588+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:22.222446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.972902+0000","last_change":"2024-09-19T07:53:02.969544+0000","last_active":"2024-09-19T07:53:02.972902+0000","last_peered":"2024-09-19T07:53:02.972902+0000","last_clean":"2024-09-19T07:53:02.972902+0000","last_became_active":"2024-09-19T07:53:02.969117+0000","last_became_peered":"2024-09-19T07:53:02.969117+0000","last_unstale":"2024-09-19T07:53:02.972902+0000","last_undegraded":"2024-09-19T07:53:02.972902+0000","last_fullsized":"2024-09-19T07:53:02.972902+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:22.222446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952211+0000","last_change":"2024-09-19T07:53:02.960894+0000","last_active":"2024-09-19T07:53:03.952211+0000","last_peered":"2024-09-19T07:53:03.952211+0000","last_clean":"2024-09-19T07:53:03.952211+0000","last_became_active":"2024-09-19T07:53:02.960589+0000","last_became_peered":"2024-09-19T07:53:02.960589+0000","last_unstale":"2024-09-19T07:53:03.952211+0000","last_undegraded":"2024-09-19T07:53:03.952211+0000","last_fullsized":"2024-09-19T07:53:03.952211+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:28.417746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951606+0000","last_change":"2024-09-19T07:53:02.964794+0000","last_active":"2024-09-19T07:53:03.951606+0000","last_peered":"2024-09-19T07:53:03.951606+0000","last_clean":"2024-09-19T07:53:03.951606+0000","last_became_active":"2024-09-19T07:53:02.964326+0000","last_became_peered":"2024-09-19T07:53:02.964326+0000","last_unstale":"2024-09-19T07:53:03.951606+0000","last_undegraded":"2024-09-19T07:53:03.951606+0000","last_fullsized":"2024-09-19T07:53:03.951606+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:22.222446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.970212+0000","last_change":"2024-09-19T07:53:02.966603+0000","last_active":"2024-09-19T07:53:02.970212+0000","last_peered":"2024-09-19T07:53:02.970212+0000","last_clean":"2024-09-19T07:53:02.970212+0000","last_became_active":"2024-09-19T07:53:02.966330+0000","last_became_peered":"2024-09-19T07:53:02.966330+0000","last_unstale":"2024-09-19T07:53:02.970212+0000","last_undegraded":"2024-09-19T07:53:02.970212+0000","last_fullsized":"2024-09-19T07:53:02.970212+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:15.018246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089112+0000","last_change":"2024-09-19T07:53:02.962007+0000","last_active":"2024-09-19T07:53:04.089112+0000","last_peered":"2024-09-19T07:53:04.089112+0000","last_clean":"2024-09-19T07:53:04.089112+0000","last_became_active":"2024-09-19T07:53:02.961724+0000","last_became_peered":"2024-09-19T07:53:02.961724+0000","last_unstale":"2024-09-19T07:53:04.089112+0000","last_undegraded":"2024-09-19T07:53:04.089112+0000","last_fullsized":"2024-09-19T07:53:04.089112+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:39.353799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088923+0000","last_change":"2024-09-19T07:53:02.957971+0000","last_active":"2024-09-19T07:53:04.088923+0000","last_peered":"2024-09-19T07:53:04.088923+0000","last_clean":"2024-09-19T07:53:04.088923+0000","last_became_active":"2024-09-19T07:53:02.957708+0000","last_became_peered":"2024-09-19T07:53:02.957708+0000","last_unstale":"2024-09-19T07:53:04.088923+0000","last_undegraded":"2024-09-19T07:53:04.088923+0000","last_fullsized":"2024-09-19T07:53:04.088923+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:56.130236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088952+0000","last_change":"2024-09-19T07:53:02.954436+0000","last_active":"2024-09-19T07:53:04.088952+0000","last_peered":"2024-09-19T07:53:04.088952+0000","last_clean":"2024-09-19T07:53:04.088952+0000","last_became_active":"2024-09-19T07:53:02.954126+0000","last_became_peered":"2024-09-19T07:53:02.954126+0000","last_unstale":"2024-09-19T07:53:04.088952+0000","last_undegraded":"2024-09-19T07:53:04.088952+0000","last_fullsized":"2024-09-19T07:53:04.088952+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:21.471735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959826+0000","last_change":"2024-09-19T07:53:02.957284+0000","last_active":"2024-09-19T07:53:03.959826+0000","last_peered":"2024-09-19T07:53:03.959826+0000","last_clean":"2024-09-19T07:53:03.959826+0000","last_became_active":"2024-09-19T07:53:02.957008+0000","last_became_peered":"2024-09-19T07:53:02.957008+0000","last_unstale":"2024-09-19T07:53:03.959826+0000","last_undegraded":"2024-09-19T07:53:03.959826+0000","last_fullsized":"2024-09-19T07:53:03.959826+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:24.443678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089749+0000","last_change":"2024-09-19T07:53:02.959833+0000","last_active":"2024-09-19T07:53:04.089749+0000","last_peered":"2024-09-19T07:53:04.089749+0000","last_clean":"2024-09-19T07:53:04.089749+0000","last_became_active":"2024-09-19T07:53:02.959076+0000","last_became_peered":"2024-09-19T07:53:02.959076+0000","last_unstale":"2024-09-19T07:53:04.089749+0000","last_undegraded":"2024-09-19T07:53:04.089749+0000","last_fullsized":"2024-09-19T07:53:04.089749+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:31.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959718+0000","last_change":"2024-09-19T07:53:02.970410+0000","last_active":"2024-09-19T07:53:03.959718+0000","last_peered":"2024-09-19T07:53:03.959718+0000","last_clean":"2024-09-19T07:53:03.959718+0000","last_became_active":"2024-09-19T07:53:02.967129+0000","last_became_peered":"2024-09-19T07:53:02.967129+0000","last_unstale":"2024-09-19T07:53:03.959718+0000","last_undegraded":"2024-09-19T07:53:03.959718+0000","last_fullsized":"2024-09-19T07:53:03.959718+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:46.990754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951748+0000","last_change":"2024-09-19T07:53:02.953069+0000","last_active":"2024-09-19T07:53:03.951748+0000","last_peered":"2024-09-19T07:53:03.951748+0000","last_clean":"2024-09-19T07:53:03.951748+0000","last_became_active":"2024-09-19T07:53:02.952498+0000","last_became_peered":"2024-09-19T07:53:02.952498+0000","last_unstale":"2024-09-19T07:53:03.951748+0000","last_undegraded":"2024-09-19T07:53:03.951748+0000","last_fullsized":"2024-09-19T07:53:03.951748+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:31.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.969961+0000","last_change":"2024-09-19T07:53:02.960655+0000","last_active":"2024-09-19T07:53:02.969961+0000","last_peered":"2024-09-19T07:53:02.969961+0000","last_clean":"2024-09-19T07:53:02.969961+0000","last_became_active":"2024-09-19T07:53:02.960373+0000","last_became_peered":"2024-09-19T07:53:02.960373+0000","last_unstale":"2024-09-19T07:53:02.969961+0000","last_undegraded":"2024-09-19T07:53:02.969961+0000","last_fullsized":"2024-09-19T07:53:02.969961+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:42.406919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959448+0000","last_change":"2024-09-19T07:53:02.958018+0000","last_active":"2024-09-19T07:53:03.959448+0000","last_peered":"2024-09-19T07:53:03.959448+0000","last_clean":"2024-09-19T07:53:03.959448+0000","last_became_active":"2024-09-19T07:53:02.957706+0000","last_became_peered":"2024-09-19T07:53:02.957706+0000","last_unstale":"2024-09-19T07:53:03.959448+0000","last_undegraded":"2024-09-19T07:53:03.959448+0000","last_fullsized":"2024-09-19T07:53:03.959448+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:08.021359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.954880+0000","last_change":"2024-09-19T07:53:02.951003+0000","last_active":"2024-09-19T07:53:02.954880+0000","last_peered":"2024-09-19T07:53:02.954880+0000","last_clean":"2024-09-19T07:53:02.954880+0000","last_became_active":"2024-09-19T07:53:02.950262+0000","last_became_peered":"2024-09-19T07:53:02.950262+0000","last_unstale":"2024-09-19T07:53:02.954880+0000","last_undegraded":"2024-09-19T07:53:02.954880+0000","last_fullsized":"2024-09-19T07:53:02.954880+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:17.832687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.953137+0000","last_change":"2024-09-19T07:53:02.955789+0000","last_active":"2024-09-19T07:53:03.953137+0000","last_peered":"2024-09-19T07:53:03.953137+0000","last_clean":"2024-09-19T07:53:03.953137+0000","last_became_active":"2024-09-19T07:53:02.955480+0000","last_became_peered":"2024-09-19T07:53:02.955480+0000","last_unstale":"2024-09-19T07:53:03.953137+0000","last_undegraded":"2024-09-19T07:53:03.953137+0000","last_fullsized":"2024-09-19T07:53:03.953137+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:00.688869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952726+0000","last_change":"2024-09-19T07:53:02.962786+0000","last_active":"2024-09-19T07:53:03.952726+0000","last_peered":"2024-09-19T07:53:03.952726+0000","last_clean":"2024-09-19T07:53:03.952726+0000","last_became_active":"2024-09-19T07:53:02.961758+0000","last_became_peered":"2024-09-19T07:53:02.961758+0000","last_unstale":"2024-09-19T07:53:03.952726+0000","last_undegraded":"2024-09-19T07:53:03.952726+0000","last_fullsized":"2024-09-19T07:53:03.952726+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:43.470589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952391+0000","last_change":"2024-09-19T07:53:02.955834+0000","last_active":"2024-09-19T07:53:03.952391+0000","last_peered":"2024-09-19T07:53:03.952391+0000","last_clean":"2024-09-19T07:53:03.952391+0000","last_became_active":"2024-09-19T07:53:02.955070+0000","last_became_peered":"2024-09-19T07:53:02.955070+0000","last_unstale":"2024-09-19T07:53:03.952391+0000","last_undegraded":"2024-09-19T07:53:03.952391+0000","last_fullsized":"2024-09-19T07:53:03.952391+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:24.696392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089356+0000","last_change":"2024-09-19T07:53:02.963635+0000","last_active":"2024-09-19T07:53:04.089356+0000","last_peered":"2024-09-19T07:53:04.089356+0000","last_clean":"2024-09-19T07:53:04.089356+0000","last_became_active":"2024-09-19T07:53:02.963299+0000","last_became_peered":"2024-09-19T07:53:02.963299+0000","last_unstale":"2024-09-19T07:53:04.089356+0000","last_undegraded":"2024-09-19T07:53:04.089356+0000","last_fullsized":"2024-09-19T07:53:04.089356+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:03.858829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952126+0000","last_change":"2024-09-19T07:53:02.959004+0000","last_active":"2024-09-19T07:53:03.952126+0000","last_peered":"2024-09-19T07:53:03.952126+0000","last_clean":"2024-09-19T07:53:03.952126+0000","last_became_active":"2024-09-19T07:53:02.958656+0000","last_became_peered":"2024-09-19T07:53:02.958656+0000","last_unstale":"2024-09-19T07:53:03.952126+0000","last_undegraded":"2024-09-19T07:53:03.952126+0000","last_fullsized":"2024-09-19T07:53:03.952126+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:58.202929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089316+0000","last_change":"2024-09-19T07:53:02.968735+0000","last_active":"2024-09-19T07:53:04.089316+0000","last_peered":"2024-09-19T07:53:04.089316+0000","last_clean":"2024-09-19T07:53:04.089316+0000","last_became_active":"2024-09-19T07:53:02.968434+0000","last_became_peered":"2024-09-19T07:53:02.968434+0000","last_unstale":"2024-09-19T07:53:04.089316+0000","last_undegraded":"2024-09-19T07:53:04.089316+0000","last_fullsized":"2024-09-19T07:53:04.089316+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:57.435952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959514+0000","last_change":"2024-09-19T07:53:02.964781+0000","last_active":"2024-09-19T07:53:03.959514+0000","last_peered":"2024-09-19T07:53:03.959514+0000","last_clean":"2024-09-19T07:53:03.959514+0000","last_became_active":"2024-09-19T07:53:02.964513+0000","last_became_peered":"2024-09-19T07:53:02.964513+0000","last_unstale":"2024-09-19T07:53:03.959514+0000","last_undegraded":"2024-09-19T07:53:03.959514+0000","last_fullsized":"2024-09-19T07:53:03.959514+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:26.718784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"105'4","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.971186+0000","last_change":"2024-09-19T07:53:01.917100+0000","last_active":"2024-09-19T07:53:02.971186+0000","last_peered":"2024-09-19T07:53:02.971186+0000","last_clean":"2024-09-19T07:53:02.971186+0000","last_became_active":"2024-09-19T07:53:01.916801+0000","last_became_peered":"2024-09-19T07:53:01.916801+0000","last_unstale":"2024-09-19T07:53:02.971186+0000","last_undegraded":"2024-09-19T07:53:02.971186+0000","last_fullsized":"2024-09-19T07:53:02.971186+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:25:44.686436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-19T07:53:00.908452+0000","last_change":"2024-09-19T07:53:00.908452+0000","last_active":"2024-09-19T07:53:00.899035+0000","last_peered":"2024-09-19T07:53:00.899035+0000","last_clean":"2024-09-19T07:53:00.899035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:00.908452+0000","last_undegraded":"2024-09-19T07:53:00.908452+0000","last_fullsized":"2024-09-19T07:53:00.908452+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952282+0000","last_change":"2024-09-19T07:53:02.950143+0000","last_active":"2024-09-19T07:53:03.952282+0000","last_peered":"2024-09-19T07:53:03.952282+0000","last_clean":"2024-09-19T07:53:03.952282+0000","last_became_active":"2024-09-19T07:53:02.949855+0000","last_became_peered":"2024-09-19T07:53:02.949855+0000","last_unstale":"2024-09-19T07:53:03.952282+0000","last_undegraded":"2024-09-19T07:53:03.952282+0000","last_fullsized":"2024-09-19T07:53:03.952282+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:22.009944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951518+0000","last_change":"2024-09-19T07:53:01.926136+0000","last_active":"2024-09-19T07:53:03.951518+0000","last_peered":"2024-09-19T07:53:03.951518+0000","last_clean":"2024-09-19T07:53:03.951518+0000","last_became_active":"2024-09-19T07:53:01.924857+0000","last_became_peered":"2024-09-19T07:53:01.924857+0000","last_unstale":"2024-09-19T07:53:03.951518+0000","last_undegraded":"2024-09-19T07:53:03.951518+0000","last_fullsized":"2024-09-19T07:53:03.951518+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:26.340346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952916+0000","last_change":"2024-09-19T07:53:02.956369+0000","last_active":"2024-09-19T07:53:03.952916+0000","last_peered":"2024-09-19T07:53:03.952916+0000","last_clean":"2024-09-19T07:53:03.952916+0000","last_became_active":"2024-09-19T07:53:02.956011+0000","last_became_peered":"2024-09-19T07:53:02.956011+0000","last_unstale":"2024-09-19T07:53:03.952916+0000","last_undegraded":"2024-09-19T07:53:03.952916+0000","last_fullsized":"2024-09-19T07:53:03.952916+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:21.471735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.969829+0000","last_change":"2024-09-19T07:53:01.919358+0000","last_active":"2024-09-19T07:53:02.969829+0000","last_peered":"2024-09-19T07:53:02.969829+0000","last_clean":"2024-09-19T07:53:02.969829+0000","last_became_active":"2024-09-19T07:53:01.914884+0000","last_became_peered":"2024-09-19T07:53:01.914884+0000","last_unstale":"2024-09-19T07:53:02.969829+0000","last_undegraded":"2024-09-19T07:53:02.969829+0000","last_fullsized":"2024-09-19T07:53:02.969829+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:26.340346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.971960+0000","last_change":"2024-09-19T07:53:02.971101+0000","last_active":"2024-09-19T07:53:02.971960+0000","last_peered":"2024-09-19T07:53:02.971960+0000","last_clean":"2024-09-19T07:53:02.971960+0000","last_became_active":"2024-09-19T07:53:02.970767+0000","last_became_peered":"2024-09-19T07:53:02.970767+0000","last_unstale":"2024-09-19T07:53:02.971960+0000","last_undegraded":"2024-09-19T07:53:02.971960+0000","last_fullsized":"2024-09-19T07:53:02.971960+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:07.231140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089007+0000","last_change":"2024-09-19T07:53:01.923137+0000","last_active":"2024-09-19T07:53:04.089007+0000","last_peered":"2024-09-19T07:53:04.089007+0000","last_clean":"2024-09-19T07:53:04.089007+0000","last_became_active":"2024-09-19T07:53:01.922852+0000","last_became_peered":"2024-09-19T07:53:01.922852+0000","last_unstale":"2024-09-19T07:53:04.089007+0000","last_undegraded":"2024-09-19T07:53:04.089007+0000","last_fullsized":"2024-09-19T07:53:04.089007+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:31.183313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959859+0000","last_change":"2024-09-19T07:53:02.967452+0000","last_active":"2024-09-19T07:53:03.959859+0000","last_peered":"2024-09-19T07:53:03.959859+0000","last_clean":"2024-09-19T07:53:03.959859+0000","last_became_active":"2024-09-19T07:53:02.966768+0000","last_became_peered":"2024-09-19T07:53:02.966768+0000","last_unstale":"2024-09-19T07:53:03.959859+0000","last_undegraded":"2024-09-19T07:53:03.959859+0000","last_fullsized":"2024-09-19T07:53:03.959859+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:04.735526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952599+0000","last_change":"2024-09-19T07:53:01.919808+0000","last_active":"2024-09-19T07:53:03.952599+0000","last_peered":"2024-09-19T07:53:03.952599+0000","last_clean":"2024-09-19T07:53:03.952599+0000","last_became_active":"2024-09-19T07:53:01.919477+0000","last_became_peered":"2024-09-19T07:53:01.919477+0000","last_unstale":"2024-09-19T07:53:03.952599+0000","last_undegraded":"2024-09-19T07:53:03.952599+0000","last_fullsized":"2024-09-19T07:53:03.952599+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:21.008103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089819+0000","last_change":"2024-09-19T07:53:02.960753+0000","last_active":"2024-09-19T07:53:04.089819+0000","last_peered":"2024-09-19T07:53:04.089819+0000","last_clean":"2024-09-19T07:53:04.089819+0000","last_became_active":"2024-09-19T07:53:02.960465+0000","last_became_peered":"2024-09-19T07:53:02.960465+0000","last_unstale":"2024-09-19T07:53:04.089819+0000","last_undegraded":"2024-09-19T07:53:04.089819+0000","last_fullsized":"2024-09-19T07:53:04.089819+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:58.202929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951787+0000","last_change":"2024-09-19T07:53:01.916796+0000","last_active":"2024-09-19T07:53:03.951787+0000","last_peered":"2024-09-19T07:53:03.951787+0000","last_clean":"2024-09-19T07:53:03.951787+0000","last_became_active":"2024-09-19T07:53:01.916506+0000","last_became_peered":"2024-09-19T07:53:01.916506+0000","last_unstale":"2024-09-19T07:53:03.951787+0000","last_undegraded":"2024-09-19T07:53:03.951787+0000","last_fullsized":"2024-09-19T07:53:03.951787+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:50.457075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.965750+0000","last_change":"2024-09-19T07:53:02.963768+0000","last_active":"2024-09-19T07:53:02.965750+0000","last_peered":"2024-09-19T07:53:02.965750+0000","last_clean":"2024-09-19T07:53:02.965750+0000","last_became_active":"2024-09-19T07:53:02.963316+0000","last_became_peered":"2024-09-19T07:53:02.963316+0000","last_unstale":"2024-09-19T07:53:02.965750+0000","last_undegraded":"2024-09-19T07:53:02.965750+0000","last_fullsized":"2024-09-19T07:53:02.965750+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:40.843150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.947396+0000","last_change":"2024-09-19T07:53:01.922316+0000","last_active":"2024-09-19T07:53:02.947396+0000","last_peered":"2024-09-19T07:53:02.947396+0000","last_clean":"2024-09-19T07:53:02.947396+0000","last_became_active":"2024-09-19T07:53:01.922051+0000","last_became_peered":"2024-09-19T07:53:01.922051+0000","last_unstale":"2024-09-19T07:53:02.947396+0000","last_undegraded":"2024-09-19T07:53:02.947396+0000","last_fullsized":"2024-09-19T07:53:02.947396+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:20.469894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959407+0000","last_change":"2024-09-19T07:53:02.967631+0000","last_active":"2024-09-19T07:53:03.959407+0000","last_peered":"2024-09-19T07:53:03.959407+0000","last_clean":"2024-09-19T07:53:03.959407+0000","last_became_active":"2024-09-19T07:53:02.966419+0000","last_became_peered":"2024-09-19T07:53:02.966419+0000","last_unstale":"2024-09-19T07:53:03.959407+0000","last_undegraded":"2024-09-19T07:53:03.959407+0000","last_fullsized":"2024-09-19T07:53:03.959407+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:00.201893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.953111+0000","last_change":"2024-09-19T07:53:01.917284+0000","last_active":"2024-09-19T07:53:03.953111+0000","last_peered":"2024-09-19T07:53:03.953111+0000","last_clean":"2024-09-19T07:53:03.953111+0000","last_became_active":"2024-09-19T07:53:01.917007+0000","last_became_peered":"2024-09-19T07:53:01.917007+0000","last_unstale":"2024-09-19T07:53:03.953111+0000","last_undegraded":"2024-09-19T07:53:03.953111+0000","last_fullsized":"2024-09-19T07:53:03.953111+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:56.053928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.957662+0000","last_change":"2024-09-19T07:53:02.951942+0000","last_active":"2024-09-19T07:53:02.957662+0000","last_peered":"2024-09-19T07:53:02.957662+0000","last_clean":"2024-09-19T07:53:02.957662+0000","last_became_active":"2024-09-19T07:53:02.950538+0000","last_became_peered":"2024-09-19T07:53:02.950538+0000","last_unstale":"2024-09-19T07:53:02.957662+0000","last_undegraded":"2024-09-19T07:53:02.957662+0000","last_fullsized":"2024-09-19T07:53:02.957662+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:50.313345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959653+0000","last_change":"2024-09-19T07:53:01.923243+0000","last_active":"2024-09-19T07:53:03.959653+0000","last_peered":"2024-09-19T07:53:03.959653+0000","last_clean":"2024-09-19T07:53:03.959653+0000","last_became_active":"2024-09-19T07:53:01.922939+0000","last_became_peered":"2024-09-19T07:53:01.922939+0000","last_unstale":"2024-09-19T07:53:03.959653+0000","last_undegraded":"2024-09-19T07:53:03.959653+0000","last_fullsized":"2024-09-19T07:53:03.959653+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:55.701057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"105'3","reported_seq":17,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089046+0000","last_change":"2024-09-19T07:53:01.923641+0000","last_active":"2024-09-19T07:53:04.089046+0000","last_peered":"2024-09-19T07:53:04.089046+0000","last_clean":"2024-09-19T07:53:04.089046+0000","last_became_active":"2024-09-19T07:53:01.923379+0000","last_became_peered":"2024-09-19T07:53:01.923379+0000","last_unstale":"2024-09-19T07:53:04.089046+0000","last_undegraded":"2024-09-19T07:53:04.089046+0000","last_fullsized":"2024-09-19T07:53:04.089046+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:51:46.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.955166+0000","last_change":"2024-09-19T07:53:02.953561+0000","last_active":"2024-09-19T07:53:02.955166+0000","last_peered":"2024-09-19T07:53:02.955166+0000","last_clean":"2024-09-19T07:53:02.955166+0000","last_became_active":"2024-09-19T07:53:02.953304+0000","last_became_peered":"2024-09-19T07:53:02.953304+0000","last_unstale":"2024-09-19T07:53:02.955166+0000","last_undegraded":"2024-09-19T07:53:02.955166+0000","last_fullsized":"2024-09-19T07:53:02.955166+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:03.858829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"105'6","reported_seq":20,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089486+0000","last_change":"2024-09-19T07:53:01.924833+0000","last_active":"2024-09-19T07:53:04.089486+0000","last_peered":"2024-09-19T07:53:04.089486+0000","last_clean":"2024-09-19T07:53:04.089486+0000","last_became_active":"2024-09-19T07:53:01.924568+0000","last_became_peered":"2024-09-19T07:53:01.924568+0000","last_unstale":"2024-09-19T07:53:04.089486+0000","last_undegraded":"2024-09-19T07:53:04.089486+0000","last_fullsized":"2024-09-19T07:53:04.089486+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:49.033408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952811+0000","last_change":"2024-09-19T07:53:02.955505+0000","last_active":"2024-09-19T07:53:03.952811+0000","last_peered":"2024-09-19T07:53:03.952811+0000","last_clean":"2024-09-19T07:53:03.952811+0000","last_became_active":"2024-09-19T07:53:02.955208+0000","last_became_peered":"2024-09-19T07:53:02.955208+0000","last_unstale":"2024-09-19T07:53:03.952811+0000","last_undegraded":"2024-09-19T07:53:03.952811+0000","last_fullsized":"2024-09-19T07:53:03.952811+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:56.130236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.969986+0000","last_change":"2024-09-19T07:53:01.920568+0000","last_active":"2024-09-19T07:53:02.969986+0000","last_peered":"2024-09-19T07:53:02.969986+0000","last_clean":"2024-09-19T07:53:02.969986+0000","last_became_active":"2024-09-19T07:53:01.920282+0000","last_became_peered":"2024-09-19T07:53:01.920282+0000","last_unstale":"2024-09-19T07:53:02.969986+0000","last_undegraded":"2024-09-19T07:53:02.969986+0000","last_fullsized":"2024-09-19T07:53:02.969986+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:11.122190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089492+0000","last_change":"2024-09-19T07:53:02.963599+0000","last_active":"2024-09-19T07:53:04.089492+0000","last_peered":"2024-09-19T07:53:04.089492+0000","last_clean":"2024-09-19T07:53:04.089492+0000","last_became_active":"2024-09-19T07:53:02.963303+0000","last_became_peered":"2024-09-19T07:53:02.963303+0000","last_unstale":"2024-09-19T07:53:04.089492+0000","last_undegraded":"2024-09-19T07:53:04.089492+0000","last_fullsized":"2024-09-19T07:53:04.089492+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:42.406919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.950643+0000","last_change":"2024-09-19T07:53:01.919711+0000","last_active":"2024-09-19T07:53:02.950643+0000","last_peered":"2024-09-19T07:53:02.950643+0000","last_clean":"2024-09-19T07:53:02.950643+0000","last_became_active":"2024-09-19T07:53:01.919443+0000","last_became_peered":"2024-09-19T07:53:01.919443+0000","last_unstale":"2024-09-19T07:53:02.950643+0000","last_undegraded":"2024-09-19T07:53:02.950643+0000","last_fullsized":"2024-09-19T07:53:02.950643+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:38.351958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-19T07:53:00.907480+0000","last_change":"2024-09-19T07:53:00.907480+0000","last_active":"2024-09-19T07:53:00.899035+0000","last_peered":"2024-09-19T07:53:00.899035+0000","last_clean":"2024-09-19T07:53:00.899035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:00.907480+0000","last_undegraded":"2024-09-19T07:53:00.907480+0000","last_fullsized":"2024-09-19T07:53:00.907480+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959932+0000","last_change":"2024-09-19T07:53:02.962652+0000","last_active":"2024-09-19T07:53:03.959932+0000","last_peered":"2024-09-19T07:53:03.959932+0000","last_clean":"2024-09-19T07:53:03.959932+0000","last_became_active":"2024-09-19T07:53:02.962394+0000","last_became_peered":"2024-09-19T07:53:02.962394+0000","last_unstale":"2024-09-19T07:53:03.959932+0000","last_undegraded":"2024-09-19T07:53:03.959932+0000","last_fullsized":"2024-09-19T07:53:03.959932+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:46.985872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"105'8","reported_seq":22,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951439+0000","last_change":"2024-09-19T07:53:01.918580+0000","last_active":"2024-09-19T07:53:03.951439+0000","last_peered":"2024-09-19T07:53:03.951439+0000","last_clean":"2024-09-19T07:53:03.951439+0000","last_became_active":"2024-09-19T07:53:01.918222+0000","last_became_peered":"2024-09-19T07:53:01.918222+0000","last_unstale":"2024-09-19T07:53:03.951439+0000","last_undegraded":"2024-09-19T07:53:03.951439+0000","last_fullsized":"2024-09-19T07:53:03.951439+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:06.785416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089618+0000","last_change":"2024-09-19T07:53:01.922470+0000","last_active":"2024-09-19T07:53:04.089618+0000","last_peered":"2024-09-19T07:53:04.089618+0000","last_clean":"2024-09-19T07:53:04.089618+0000","last_became_active":"2024-09-19T07:53:01.922181+0000","last_became_peered":"2024-09-19T07:53:01.922181+0000","last_unstale":"2024-09-19T07:53:04.089618+0000","last_undegraded":"2024-09-19T07:53:04.089618+0000","last_fullsized":"2024-09-19T07:53:04.089618+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:47.737086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089090+0000","last_change":"2024-09-19T07:53:02.965849+0000","last_active":"2024-09-19T07:53:04.089090+0000","last_peered":"2024-09-19T07:53:04.089090+0000","last_clean":"2024-09-19T07:53:04.089090+0000","last_became_active":"2024-09-19T07:53:02.965562+0000","last_became_peered":"2024-09-19T07:53:02.965562+0000","last_unstale":"2024-09-19T07:53:04.089090+0000","last_undegraded":"2024-09-19T07:53:04.089090+0000","last_fullsized":"2024-09-19T07:53:04.089090+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:50.313345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089146+0000","last_change":"2024-09-19T07:53:01.922247+0000","last_active":"2024-09-19T07:53:04.089146+0000","last_peered":"2024-09-19T07:53:04.089146+0000","last_clean":"2024-09-19T07:53:04.089146+0000","last_became_active":"2024-09-19T07:53:01.921944+0000","last_became_peered":"2024-09-19T07:53:01.921944+0000","last_unstale":"2024-09-19T07:53:04.089146+0000","last_undegraded":"2024-09-19T07:53:04.089146+0000","last_fullsized":"2024-09-19T07:53:04.089146+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:40.569155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.955145+0000","last_change":"2024-09-19T07:53:02.952785+0000","last_active":"2024-09-19T07:53:02.955145+0000","last_peered":"2024-09-19T07:53:02.955145+0000","last_clean":"2024-09-19T07:53:02.955145+0000","last_became_active":"2024-09-19T07:53:02.952535+0000","last_became_peered":"2024-09-19T07:53:02.952535+0000","last_unstale":"2024-09-19T07:53:02.955145+0000","last_undegraded":"2024-09-19T07:53:02.955145+0000","last_fullsized":"2024-09-19T07:53:02.955145+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:57.435952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952771+0000","last_change":"2024-09-19T07:53:01.925865+0000","last_active":"2024-09-19T07:53:03.952771+0000","last_peered":"2024-09-19T07:53:03.952771+0000","last_clean":"2024-09-19T07:53:03.952771+0000","last_became_active":"2024-09-19T07:53:01.925544+0000","last_became_peered":"2024-09-19T07:53:01.925544+0000","last_unstale":"2024-09-19T07:53:03.952771+0000","last_undegraded":"2024-09-19T07:53:03.952771+0000","last_fullsized":"2024-09-19T07:53:03.952771+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:57.201088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959514+0000","last_change":"2024-09-19T07:53:02.970190+0000","last_active":"2024-09-19T07:53:03.959514+0000","last_peered":"2024-09-19T07:53:03.959514+0000","last_clean":"2024-09-19T07:53:03.959514+0000","last_became_active":"2024-09-19T07:53:02.969897+0000","last_became_peered":"2024-09-19T07:53:02.969897+0000","last_unstale":"2024-09-19T07:53:03.959514+0000","last_undegraded":"2024-09-19T07:53:03.959514+0000","last_fullsized":"2024-09-19T07:53:03.959514+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:24.195258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-19T07:53:00.907636+0000","last_change":"2024-09-19T07:53:00.907636+0000","last_active":"2024-09-19T07:53:00.899035+0000","last_peered":"2024-09-19T07:53:00.899035+0000","last_clean":"2024-09-19T07:53:00.899035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:00.907636+0000","last_undegraded":"2024-09-19T07:53:00.907636+0000","last_fullsized":"2024-09-19T07:53:00.907636+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959770+0000","last_change":"2024-09-19T07:53:02.946678+0000","last_active":"2024-09-19T07:53:03.959770+0000","last_peered":"2024-09-19T07:53:03.959770+0000","last_clean":"2024-09-19T07:53:03.959770+0000","last_became_active":"2024-09-19T07:53:02.946357+0000","last_became_peered":"2024-09-19T07:53:02.946357+0000","last_unstale":"2024-09-19T07:53:03.959770+0000","last_undegraded":"2024-09-19T07:53:03.959770+0000","last_fullsized":"2024-09-19T07:53:03.959770+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:20.948125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-19T07:53:00.908527+0000","last_change":"2024-09-19T07:53:00.908527+0000","last_active":"2024-09-19T07:53:00.899035+0000","last_peered":"2024-09-19T07:53:00.899035+0000","last_clean":"2024-09-19T07:53:00.899035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:00.908527+0000","last_undegraded":"2024-09-19T07:53:00.908527+0000","last_fullsized":"2024-09-19T07:53:00.908527+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.968168+0000","last_change":"2024-09-19T07:53:02.965053+0000","last_active":"2024-09-19T07:53:02.968168+0000","last_peered":"2024-09-19T07:53:02.968168+0000","last_clean":"2024-09-19T07:53:02.968168+0000","last_became_active":"2024-09-19T07:53:02.964769+0000","last_became_peered":"2024-09-19T07:53:02.964769+0000","last_unstale":"2024-09-19T07:53:02.968168+0000","last_undegraded":"2024-09-19T07:53:02.968168+0000","last_fullsized":"2024-09-19T07:53:02.968168+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:40.771106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089316+0000","last_change":"2024-09-19T07:53:01.923054+0000","last_active":"2024-09-19T07:53:04.089316+0000","last_peered":"2024-09-19T07:53:04.089316+0000","last_clean":"2024-09-19T07:53:04.089316+0000","last_became_active":"2024-09-19T07:53:01.922799+0000","last_became_peered":"2024-09-19T07:53:01.922799+0000","last_unstale":"2024-09-19T07:53:04.089316+0000","last_undegraded":"2024-09-19T07:53:04.089316+0000","last_fullsized":"2024-09-19T07:53:04.089316+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:47.745785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-19T07:53:00.907876+0000","last_change":"2024-09-19T07:53:00.907876+0000","last_active":"2024-09-19T07:53:00.899035+0000","last_peered":"2024-09-19T07:53:00.899035+0000","last_clean":"2024-09-19T07:53:00.899035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:00.907876+0000","last_undegraded":"2024-09-19T07:53:00.907876+0000","last_fullsized":"2024-09-19T07:53:00.907876+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959382+0000","last_change":"2024-09-19T07:53:02.968250+0000","last_active":"2024-09-19T07:53:03.959382+0000","last_peered":"2024-09-19T07:53:03.959382+0000","last_clean":"2024-09-19T07:53:03.959382+0000","last_became_active":"2024-09-19T07:53:02.967961+0000","last_became_peered":"2024-09-19T07:53:02.967961+0000","last_unstale":"2024-09-19T07:53:03.959382+0000","last_undegraded":"2024-09-19T07:53:03.959382+0000","last_fullsized":"2024-09-19T07:53:03.959382+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:03.803861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.969747+0000","last_change":"2024-09-19T07:53:01.922110+0000","last_active":"2024-09-19T07:53:02.969747+0000","last_peered":"2024-09-19T07:53:02.969747+0000","last_clean":"2024-09-19T07:53:02.969747+0000","last_became_active":"2024-09-19T07:53:01.921769+0000","last_became_peered":"2024-09-19T07:53:01.921769+0000","last_unstale":"2024-09-19T07:53:02.969747+0000","last_undegraded":"2024-09-19T07:53:02.969747+0000","last_fullsized":"2024-09-19T07:53:02.969747+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:56.053928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951681+0000","last_change":"2024-09-19T07:53:02.963616+0000","last_active":"2024-09-19T07:53:03.951681+0000","last_peered":"2024-09-19T07:53:03.951681+0000","last_clean":"2024-09-19T07:53:03.951681+0000","last_became_active":"2024-09-19T07:53:02.963333+0000","last_became_peered":"2024-09-19T07:53:02.963333+0000","last_unstale":"2024-09-19T07:53:03.951681+0000","last_undegraded":"2024-09-19T07:53:03.951681+0000","last_fullsized":"2024-09-19T07:53:03.951681+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"106'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.961104+0000","last_change":"2024-09-19T07:53:01.922601+0000","last_active":"2024-09-19T07:53:03.961104+0000","last_peered":"2024-09-19T07:53:03.961104+0000","last_clean":"2024-09-19T07:53:03.961104+0000","last_became_active":"2024-09-19T07:53:01.922324+0000","last_became_peered":"2024-09-19T07:53:01.922324+0000","last_unstale":"2024-09-19T07:53:03.961104+0000","last_undegraded":"2024-09-19T07:53:03.961104+0000","last_fullsized":"2024-09-19T07:53:03.961104+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:02.428894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.962345+0000","last_change":"2024-09-19T07:53:02.961218+0000","last_active":"2024-09-19T07:53:02.962345+0000","last_peered":"2024-09-19T07:53:02.962345+0000","last_clean":"2024-09-19T07:53:02.962345+0000","last_became_active":"2024-09-19T07:53:02.960873+0000","last_became_peered":"2024-09-19T07:53:02.960873+0000","last_unstale":"2024-09-19T07:53:02.962345+0000","last_undegraded":"2024-09-19T07:53:02.962345+0000","last_fullsized":"2024-09-19T07:53:02.962345+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:27.358096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089482+0000","last_change":"2024-09-19T07:53:01.920744+0000","last_active":"2024-09-19T07:53:04.089482+0000","last_peered":"2024-09-19T07:53:04.089482+0000","last_clean":"2024-09-19T07:53:04.089482+0000","last_became_active":"2024-09-19T07:53:01.920476+0000","last_became_peered":"2024-09-19T07:53:01.920476+0000","last_unstale":"2024-09-19T07:53:04.089482+0000","last_undegraded":"2024-09-19T07:53:04.089482+0000","last_fullsized":"2024-09-19T07:53:04.089482+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:11.662918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952379+0000","last_change":"2024-09-19T07:53:02.961727+0000","last_active":"2024-09-19T07:53:03.952379+0000","last_peered":"2024-09-19T07:53:03.952379+0000","last_clean":"2024-09-19T07:53:03.952379+0000","last_became_active":"2024-09-19T07:53:02.961447+0000","last_became_peered":"2024-09-19T07:53:02.961447+0000","last_unstale":"2024-09-19T07:53:03.952379+0000","last_undegraded":"2024-09-19T07:53:03.952379+0000","last_fullsized":"2024-09-19T07:53:03.952379+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:26:55.837328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.940797+0000","last_change":"2024-09-19T07:53:01.922583+0000","last_active":"2024-09-19T07:53:02.940797+0000","last_peered":"2024-09-19T07:53:02.940797+0000","last_clean":"2024-09-19T07:53:02.940797+0000","last_became_active":"2024-09-19T07:53:01.922318+0000","last_became_peered":"2024-09-19T07:53:01.922318+0000","last_unstale":"2024-09-19T07:53:02.940797+0000","last_undegraded":"2024-09-19T07:53:02.940797+0000","last_fullsized":"2024-09-19T07:53:02.940797+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:14.312065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952032+0000","last_change":"2024-09-19T07:53:01.918087+0000","last_active":"2024-09-19T07:53:03.952032+0000","last_peered":"2024-09-19T07:53:03.952032+0000","last_clean":"2024-09-19T07:53:03.952032+0000","last_became_active":"2024-09-19T07:53:01.917738+0000","last_became_peered":"2024-09-19T07:53:01.917738+0000","last_unstale":"2024-09-19T07:53:03.952032+0000","last_undegraded":"2024-09-19T07:53:03.952032+0000","last_fullsized":"2024-09-19T07:53:03.952032+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:14.312065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-19T07:53:00.908081+0000","last_change":"2024-09-19T07:53:00.908081+0000","last_active":"2024-09-19T07:53:00.899035+0000","last_peered":"2024-09-19T07:53:00.899035+0000","last_clean":"2024-09-19T07:53:00.899035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:00.908081+0000","last_undegraded":"2024-09-19T07:53:00.908081+0000","last_fullsized":"2024-09-19T07:53:00.908081+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089415+0000","last_change":"2024-09-19T07:53:02.957282+0000","last_active":"2024-09-19T07:53:04.089415+0000","last_peered":"2024-09-19T07:53:04.089415+0000","last_clean":"2024-09-19T07:53:04.089415+0000","last_became_active":"2024-09-19T07:53:02.957004+0000","last_became_peered":"2024-09-19T07:53:02.957004+0000","last_unstale":"2024-09-19T07:53:04.089415+0000","last_undegraded":"2024-09-19T07:53:04.089415+0000","last_fullsized":"2024-09-19T07:53:04.089415+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:25:45.688277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089434+0000","last_change":"2024-09-19T07:53:01.921607+0000","last_active":"2024-09-19T07:53:04.089434+0000","last_peered":"2024-09-19T07:53:04.089434+0000","last_clean":"2024-09-19T07:53:04.089434+0000","last_became_active":"2024-09-19T07:53:01.920937+0000","last_became_peered":"2024-09-19T07:53:01.920937+0000","last_unstale":"2024-09-19T07:53:04.089434+0000","last_undegraded":"2024-09-19T07:53:04.089434+0000","last_fullsized":"2024-09-19T07:53:04.089434+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:35.664386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.953006+0000","last_change":"2024-09-19T07:53:02.956981+0000","last_active":"2024-09-19T07:53:03.953006+0000","last_peered":"2024-09-19T07:53:03.953006+0000","last_clean":"2024-09-19T07:53:03.953006+0000","last_became_active":"2024-09-19T07:53:02.955937+0000","last_became_peered":"2024-09-19T07:53:02.955937+0000","last_unstale":"2024-09-19T07:53:03.953006+0000","last_undegraded":"2024-09-19T07:53:03.953006+0000","last_fullsized":"2024-09-19T07:53:03.953006+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:39.353799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-19T07:53:00.907369+0000","last_change":"2024-09-19T07:53:00.907369+0000","last_active":"2024-09-19T07:53:00.899035+0000","last_peered":"2024-09-19T07:53:00.899035+0000","last_clean":"2024-09-19T07:53:00.899035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:00.907369+0000","last_undegraded":"2024-09-19T07:53:00.907369+0000","last_fullsized":"2024-09-19T07:53:00.907369+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951881+0000","last_change":"2024-09-19T07:53:02.955750+0000","last_active":"2024-09-19T07:53:03.951881+0000","last_peered":"2024-09-19T07:53:03.951881+0000","last_clean":"2024-09-19T07:53:03.951881+0000","last_became_active":"2024-09-19T07:53:02.955250+0000","last_became_peered":"2024-09-19T07:53:02.955250+0000","last_unstale":"2024-09-19T07:53:03.951881+0000","last_undegraded":"2024-09-19T07:53:03.951881+0000","last_fullsized":"2024-09-19T07:53:03.951881+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:22.292252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-19T07:53:00.907937+0000","last_change":"2024-09-19T07:53:00.907937+0000","last_active":"2024-09-19T07:53:00.899035+0000","last_peered":"2024-09-19T07:53:00.899035+0000","last_clean":"2024-09-19T07:53:00.899035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:00.907937+0000","last_undegraded":"2024-09-19T07:53:00.907937+0000","last_fullsized":"2024-09-19T07:53:00.907937+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":325,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088582+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:53:04.088582+0000","last_peered":"2024-09-19T07:53:04.088582+0000","last_clean":"2024-09-19T07:53:04.088582+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:53:04.088582+0000","last_undegraded":"2024-09-19T07:53:04.088582+0000","last_fullsized":"2024-09-19T07:53:04.088582+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":106,"acting":106,"num_store_stats":7},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509565,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82960,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288880,"statfs":{"total":96636764160,"available":96551813120,"internally_reserved":0,"allocated":1896448,"data_stored":1188786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509564,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100520,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":94271320,"statfs":{"total":96636764160,"available":96533831680,"internally_reserved":0,"allocated":1327104,"data_stored":607166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":101579206},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509565,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50512,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321328,"statfs":{"total":96636764160,"available":96585039872,"internally_reserved":0,"allocated":1306624,"data_stored":604950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509565,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86552,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285288,"statfs":{"total":96636764160,"available":96548134912,"internally_reserved":0,"allocated":1314816,"data_stored":605042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509564,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":101020,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94270820,"statfs":{"total":96636764160,"available":96533319680,"internally_reserved":0,"allocated":1904640,"data_stored":1190328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542268,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135120,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236720,"statfs":{"total":96636764160,"available":96498401280,"internally_reserved":0,"allocated":1298432,"data_stored":598840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":137034183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542268,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94928,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276912,"statfs":{"total":96636764160,"available":96539557888,"internally_reserved":0,"allocated":1298432,"data_stored":598292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104556,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267284,"statfs":{"total":96636764160,"available":96529698816,"internally_reserved":0,"allocated":1269760,"data_stored":584424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:53:05.939 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:53:06.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.076+0000 7f46cc8e6640 1 Processor -- start 2024-09-19T07:53:06.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.076+0000 7f46cc8e6640 1 -- start start 2024-09-19T07:53:06.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.077+0000 7f46cc8e6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f46c815e680 0x7f46c80cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.077+0000 7f46cc8e6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f46c80d0010 0x7f46c80d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.077+0000 7f46cc8e6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46c80d1880 con 0x7f46c815e2a0 2024-09-19T07:53:06.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.077+0000 7f46cc8e6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f46c80d1b60 con 0x7f46c80d0010 2024-09-19T07:53:06.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.077+0000 7f46cc8e6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f46c80cf2d0 con 0x7f46c815e680 2024-09-19T07:53:06.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.077+0000 7f46c6d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f46c80d0010 0x7f46c80d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.077+0000 7f46c6d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f46c80d0010 0x7f46c80d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60728/0 (socket says 172.21.15.16:60728) 2024-09-19T07:53:06.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.077+0000 7f46c6d9d640 1 -- 172.21.15.16:0/1173543702 learned_addr learned my addr 172.21.15.16:0/1173543702 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:06.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.077+0000 7f46c5d9b640 1 --2- 172.21.15.16:0/1173543702 >> v2:172.21.15.146:3300/0 conn(0x7f46c815e680 0x7f46c80cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.078+0000 7f46c6d9d640 1 -- 172.21.15.16:0/1173543702 >> v2:172.21.15.146:3300/0 conn(0x7f46c815e680 msgr2=0x7f46c80cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:06.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.078+0000 7f46c6d9d640 1 --2- 172.21.15.16:0/1173543702 >> v2:172.21.15.146:3300/0 conn(0x7f46c815e680 0x7f46c80cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.078+0000 7f46c6d9d640 1 -- 172.21.15.16:0/1173543702 >> v1:172.21.15.146:6789/0 conn(0x7f46c815e2a0 legacy=0x7f46c806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.078+0000 7f46c6d9d640 1 -- 172.21.15.16:0/1173543702 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c80cf5b0 con 0x7f46c80d0010 2024-09-19T07:53:06.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.078+0000 7f46c6d9d640 1 --2- 172.21.15.16:0/1173543702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c80d0010 0x7f46c80d4370 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f46c80a22a0 tx=0x7f46c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948a4336b05ef6cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:06.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.079+0000 7f46c559a640 1 -- 172.21.15.16:0/1173543702 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0037070 con 0x7f46c80d0010 2024-09-19T07:53:06.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.079+0000 7f46c559a640 1 -- 172.21.15.16:0/1173543702 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c0032440 con 0x7f46c80d0010 2024-09-19T07:53:06.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.079+0000 7f46c559a640 1 -- 172.21.15.16:0/1173543702 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0032810 con 0x7f46c80d0010 2024-09-19T07:53:06.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.079+0000 7f46cc8e6640 1 -- 172.21.15.16:0/1173543702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c80d0010 msgr2=0x7f46c80d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.079+0000 7f46cc8e6640 1 --2- 172.21.15.16:0/1173543702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c80d0010 0x7f46c80d4370 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f46c80a22a0 tx=0x7f46c002a1e0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.079+0000 7f46cc8e6640 1 -- 172.21.15.16:0/1173543702 shutdown_connections 2024-09-19T07:53:06.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.079+0000 7f46cc8e6640 1 --2- 172.21.15.16:0/1173543702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c80d0010 0x7f46c80d4370 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.079+0000 7f46cc8e6640 1 --2- 172.21.15.16:0/1173543702 >> v2:172.21.15.146:3300/0 conn(0x7f46c815e680 0x7f46c80cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.079+0000 7f46cc8e6640 1 -- 172.21.15.16:0/1173543702 >> 172.21.15.16:0/1173543702 conn(0x7f46c809d760 msgr2=0x7f46c808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:06.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.079+0000 7f46cc8e6640 1 -- 172.21.15.16:0/1173543702 shutdown_connections 2024-09-19T07:53:06.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.079+0000 7f46cc8e6640 1 -- 172.21.15.16:0/1173543702 wait complete. 2024-09-19T07:53:06.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.081+0000 7f46cc8e6640 1 Processor -- start 2024-09-19T07:53:06.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.081+0000 7f46cc8e6640 1 -- start start 2024-09-19T07:53:06.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.081+0000 7f46cc8e6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46c80d0010 0x7f46c81528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.081+0000 7f46cc8e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c815e2a0 0x7f46c80c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.081+0000 7f46cc8e6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f46c815e680 0x7f46c815fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.081+0000 7f46cc8e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f46c806dcb0 con 0x7f46c815e2a0 2024-09-19T07:53:06.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.081+0000 7f46cc8e6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f46c806db30 con 0x7f46c80d0010 2024-09-19T07:53:06.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.081+0000 7f46cc8e6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f46c80d48b0 con 0x7f46c815e680 2024-09-19T07:53:06.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.082+0000 7f46c5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c815e2a0 0x7f46c80c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.082+0000 7f46c5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c815e2a0 0x7f46c80c9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60738/0 (socket says 172.21.15.16:60738) 2024-09-19T07:53:06.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.082+0000 7f46c5d9b640 1 -- 172.21.15.16:0/3228975406 learned_addr learned my addr 172.21.15.16:0/3228975406 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:06.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.082+0000 7f46c659c640 1 --2- 172.21.15.16:0/3228975406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46c80d0010 0x7f46c81528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.082+0000 7f46c6d9d640 1 --2- 172.21.15.16:0/3228975406 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f46c815e680 0x7f46c815fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.082+0000 7f46c5d9b640 1 -- 172.21.15.16:0/3228975406 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f46c815e680 msgr2=0x7f46c815fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.082+0000 7f46c5d9b640 1 --2- 172.21.15.16:0/3228975406 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f46c815e680 0x7f46c815fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.082+0000 7f46c5d9b640 1 -- 172.21.15.16:0/3228975406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46c80d0010 msgr2=0x7f46c81528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.082+0000 7f46c5d9b640 1 --2- 172.21.15.16:0/3228975406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46c80d0010 0x7f46c81528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.082+0000 7f46c5d9b640 1 -- 172.21.15.16:0/3228975406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c80ca510 con 0x7f46c815e2a0 2024-09-19T07:53:06.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.082+0000 7f46c5d9b640 1 --2- 172.21.15.16:0/3228975406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c815e2a0 0x7f46c80c9bf0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f46b000e840 tx=0x7f46b000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:06.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.082+0000 7f46beffd640 1 -- 172.21.15.16:0/3228975406 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0017070 con 0x7f46c815e2a0 2024-09-19T07:53:06.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.082+0000 7f46beffd640 1 -- 172.21.15.16:0/3228975406 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b000c940 con 0x7f46c815e2a0 2024-09-19T07:53:06.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.082+0000 7f46cc8e6640 1 -- 172.21.15.16:0/3228975406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c80ca7f0 con 0x7f46c815e2a0 2024-09-19T07:53:06.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.082+0000 7f46beffd640 1 -- 172.21.15.16:0/3228975406 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b000cd10 con 0x7f46c815e2a0 2024-09-19T07:53:06.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.082+0000 7f46cc8e6640 1 -- 172.21.15.16:0/3228975406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c80cadb0 con 0x7f46c815e2a0 2024-09-19T07:53:06.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.083+0000 7f46cc8e6640 1 -- 172.21.15.16:0/3228975406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c80d4c70 con 0x7f46c815e2a0 2024-09-19T07:53:06.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.084+0000 7f46beffd640 1 -- 172.21.15.16:0/3228975406 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f46b001a070 con 0x7f46c815e2a0 2024-09-19T07:53:06.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.084+0000 7f46beffd640 1 --2- 172.21.15.16:0/3228975406 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f46a806c2b0 0x7f46a806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.084+0000 7f46beffd640 1 -- 172.21.15.16:0/3228975406 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f46b008b760 con 0x7f46c815e2a0 2024-09-19T07:53:06.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.084+0000 7f46c659c640 1 --2- 172.21.15.16:0/3228975406 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f46a806c2b0 0x7f46a806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.085+0000 7f46c659c640 1 --2- 172.21.15.16:0/3228975406 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f46a806c2b0 0x7f46a806e770 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f46c8160ac0 tx=0x7f46b8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:06.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.089+0000 7f46beffd640 1 -- 172.21.15.16:0/3228975406 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f46b00596e0 con 0x7f46c815e2a0 2024-09-19T07:53:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.263+0000 7f46cc8e6640 1 -- 172.21.15.16:0/3228975406 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f46c80c7c90 con 0x7f46a806c2b0 2024-09-19T07:53:06.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.269+0000 7f46beffd640 1 -- 172.21.15.16:0/3228975406 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250259 (secure 0 0 0) 0x7f46c80c7c90 con 0x7f46a806c2b0 2024-09-19T07:53:06.272 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:06.273 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:53:06.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.272+0000 7f46cc8e6640 1 -- 172.21.15.16:0/3228975406 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f46a806c2b0 msgr2=0x7f46a806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.272+0000 7f46cc8e6640 1 --2- 172.21.15.16:0/3228975406 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f46a806c2b0 0x7f46a806e770 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f46c8160ac0 tx=0x7f46b8009040 comp rx=0 tx=0).stop 2024-09-19T07:53:06.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.272+0000 7f46cc8e6640 1 -- 172.21.15.16:0/3228975406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c815e2a0 msgr2=0x7f46c80c9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.272+0000 7f46cc8e6640 1 --2- 172.21.15.16:0/3228975406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c815e2a0 0x7f46c80c9bf0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f46b000e840 tx=0x7f46b000ed00 comp rx=0 tx=0).stop 2024-09-19T07:53:06.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.272+0000 7f46cc8e6640 1 -- 172.21.15.16:0/3228975406 shutdown_connections 2024-09-19T07:53:06.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.272+0000 7f46cc8e6640 1 --2- 172.21.15.16:0/3228975406 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f46a806c2b0 0x7f46a806e770 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.272+0000 7f46cc8e6640 1 --2- 172.21.15.16:0/3228975406 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f46c815e680 0x7f46c815fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.272+0000 7f46cc8e6640 1 --2- 172.21.15.16:0/3228975406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c815e2a0 0x7f46c80c9bf0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.272+0000 7f46cc8e6640 1 --2- 172.21.15.16:0/3228975406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46c80d0010 0x7f46c81528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.272+0000 7f46cc8e6640 1 -- 172.21.15.16:0/3228975406 >> 172.21.15.16:0/3228975406 conn(0x7f46c809d760 msgr2=0x7f46c8161020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:06.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.272+0000 7f46cc8e6640 1 -- 172.21.15.16:0/3228975406 shutdown_connections 2024-09-19T07:53:06.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.272+0000 7f46cc8e6640 1 -- 172.21.15.16:0/3228975406 wait complete. 2024-09-19T07:53:06.283 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-09-19T07:53:04.600078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":200,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":756168,"kb_used_data":11344,"kb_used_omap":12,"kb_used_meta":744627,"kb_avail":754218552,"statfs":{"total":773094113280,"available":772319797248,"internally_reserved":0,"allocated":11616256,"data_stored":5977828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":762498608},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000976"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959791+0000","last_change":"2024-09-19T07:53:02.969667+0000","last_active":"2024-09-19T07:53:03.959791+0000","last_peered":"2024-09-19T07:53:03.959791+0000","last_clean":"2024-09-19T07:53:03.959791+0000","last_became_active":"2024-09-19T07:53:02.969269+0000","last_became_peered":"2024-09-19T07:53:02.969269+0000","last_unstale":"2024-09-19T07:53:03.959791+0000","last_undegraded":"2024-09-19T07:53:03.959791+0000","last_fullsized":"2024-09-19T07:53:03.959791+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:21.528651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089709+0000","last_change":"2024-09-19T07:53:02.960077+0000","last_active":"2024-09-19T07:53:04.089709+0000","last_peered":"2024-09-19T07:53:04.089709+0000","last_clean":"2024-09-19T07:53:04.089709+0000","last_became_active":"2024-09-19T07:53:02.959395+0000","last_became_peered":"2024-09-19T07:53:02.959395+0000","last_unstale":"2024-09-19T07:53:04.089709+0000","last_undegraded":"2024-09-19T07:53:04.089709+0000","last_fullsized":"2024-09-19T07:53:04.089709+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959657+0000","last_change":"2024-09-19T07:53:02.961895+0000","last_active":"2024-09-19T07:53:03.959657+0000","last_peered":"2024-09-19T07:53:03.959657+0000","last_clean":"2024-09-19T07:53:03.959657+0000","last_became_active":"2024-09-19T07:53:02.961616+0000","last_became_peered":"2024-09-19T07:53:02.961616+0000","last_unstale":"2024-09-19T07:53:03.959657+0000","last_undegraded":"2024-09-19T07:53:03.959657+0000","last_fullsized":"2024-09-19T07:53:03.959657+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:01.957812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.962372+0000","last_change":"2024-09-19T07:53:02.961725+0000","last_active":"2024-09-19T07:53:02.962372+0000","last_peered":"2024-09-19T07:53:02.962372+0000","last_clean":"2024-09-19T07:53:02.962372+0000","last_became_active":"2024-09-19T07:53:02.961334+0000","last_became_peered":"2024-09-19T07:53:02.961334+0000","last_unstale":"2024-09-19T07:53:02.962372+0000","last_undegraded":"2024-09-19T07:53:02.962372+0000","last_fullsized":"2024-09-19T07:53:02.962372+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:04.795110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.946689+0000","last_change":"2024-09-19T07:53:02.945749+0000","last_active":"2024-09-19T07:53:02.946689+0000","last_peered":"2024-09-19T07:53:02.946689+0000","last_clean":"2024-09-19T07:53:02.946689+0000","last_became_active":"2024-09-19T07:53:02.945466+0000","last_became_peered":"2024-09-19T07:53:02.945466+0000","last_unstale":"2024-09-19T07:53:02.946689+0000","last_undegraded":"2024-09-19T07:53:02.946689+0000","last_fullsized":"2024-09-19T07:53:02.946689+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:28.288866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088937+0000","last_change":"2024-09-19T07:53:02.960167+0000","last_active":"2024-09-19T07:53:04.088937+0000","last_peered":"2024-09-19T07:53:04.088937+0000","last_clean":"2024-09-19T07:53:04.088937+0000","last_became_active":"2024-09-19T07:53:02.959874+0000","last_became_peered":"2024-09-19T07:53:02.959874+0000","last_unstale":"2024-09-19T07:53:04.088937+0000","last_undegraded":"2024-09-19T07:53:04.088937+0000","last_fullsized":"2024-09-19T07:53:04.088937+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:17.832687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088822+0000","last_change":"2024-09-19T07:53:02.973657+0000","last_active":"2024-09-19T07:53:04.088822+0000","last_peered":"2024-09-19T07:53:04.088822+0000","last_clean":"2024-09-19T07:53:04.088822+0000","last_became_active":"2024-09-19T07:53:02.973421+0000","last_became_peered":"2024-09-19T07:53:02.973421+0000","last_unstale":"2024-09-19T07:53:04.088822+0000","last_undegraded":"2024-09-19T07:53:04.088822+0000","last_fullsized":"2024-09-19T07:53:04.088822+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:28.288866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089588+0000","last_change":"2024-09-19T07:53:02.966890+0000","last_active":"2024-09-19T07:53:04.089588+0000","last_peered":"2024-09-19T07:53:04.089588+0000","last_clean":"2024-09-19T07:53:04.089588+0000","last_became_active":"2024-09-19T07:53:02.966573+0000","last_became_peered":"2024-09-19T07:53:02.966573+0000","last_unstale":"2024-09-19T07:53:04.089588+0000","last_undegraded":"2024-09-19T07:53:04.089588+0000","last_fullsized":"2024-09-19T07:53:04.089588+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:22.222446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.972902+0000","last_change":"2024-09-19T07:53:02.969544+0000","last_active":"2024-09-19T07:53:02.972902+0000","last_peered":"2024-09-19T07:53:02.972902+0000","last_clean":"2024-09-19T07:53:02.972902+0000","last_became_active":"2024-09-19T07:53:02.969117+0000","last_became_peered":"2024-09-19T07:53:02.969117+0000","last_unstale":"2024-09-19T07:53:02.972902+0000","last_undegraded":"2024-09-19T07:53:02.972902+0000","last_fullsized":"2024-09-19T07:53:02.972902+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:22.222446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952211+0000","last_change":"2024-09-19T07:53:02.960894+0000","last_active":"2024-09-19T07:53:03.952211+0000","last_peered":"2024-09-19T07:53:03.952211+0000","last_clean":"2024-09-19T07:53:03.952211+0000","last_became_active":"2024-09-19T07:53:02.960589+0000","last_became_peered":"2024-09-19T07:53:02.960589+0000","last_unstale":"2024-09-19T07:53:03.952211+0000","last_undegraded":"2024-09-19T07:53:03.952211+0000","last_fullsized":"2024-09-19T07:53:03.952211+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:28.417746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951606+0000","last_change":"2024-09-19T07:53:02.964794+0000","last_active":"2024-09-19T07:53:03.951606+0000","last_peered":"2024-09-19T07:53:03.951606+0000","last_clean":"2024-09-19T07:53:03.951606+0000","last_became_active":"2024-09-19T07:53:02.964326+0000","last_became_peered":"2024-09-19T07:53:02.964326+0000","last_unstale":"2024-09-19T07:53:03.951606+0000","last_undegraded":"2024-09-19T07:53:03.951606+0000","last_fullsized":"2024-09-19T07:53:03.951606+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:22.222446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.970212+0000","last_change":"2024-09-19T07:53:02.966603+0000","last_active":"2024-09-19T07:53:02.970212+0000","last_peered":"2024-09-19T07:53:02.970212+0000","last_clean":"2024-09-19T07:53:02.970212+0000","last_became_active":"2024-09-19T07:53:02.966330+0000","last_became_peered":"2024-09-19T07:53:02.966330+0000","last_unstale":"2024-09-19T07:53:02.970212+0000","last_undegraded":"2024-09-19T07:53:02.970212+0000","last_fullsized":"2024-09-19T07:53:02.970212+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:15.018246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089112+0000","last_change":"2024-09-19T07:53:02.962007+0000","last_active":"2024-09-19T07:53:04.089112+0000","last_peered":"2024-09-19T07:53:04.089112+0000","last_clean":"2024-09-19T07:53:04.089112+0000","last_became_active":"2024-09-19T07:53:02.961724+0000","last_became_peered":"2024-09-19T07:53:02.961724+0000","last_unstale":"2024-09-19T07:53:04.089112+0000","last_undegraded":"2024-09-19T07:53:04.089112+0000","last_fullsized":"2024-09-19T07:53:04.089112+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:39.353799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088923+0000","last_change":"2024-09-19T07:53:02.957971+0000","last_active":"2024-09-19T07:53:04.088923+0000","last_peered":"2024-09-19T07:53:04.088923+0000","last_clean":"2024-09-19T07:53:04.088923+0000","last_became_active":"2024-09-19T07:53:02.957708+0000","last_became_peered":"2024-09-19T07:53:02.957708+0000","last_unstale":"2024-09-19T07:53:04.088923+0000","last_undegraded":"2024-09-19T07:53:04.088923+0000","last_fullsized":"2024-09-19T07:53:04.088923+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:56.130236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088952+0000","last_change":"2024-09-19T07:53:02.954436+0000","last_active":"2024-09-19T07:53:04.088952+0000","last_peered":"2024-09-19T07:53:04.088952+0000","last_clean":"2024-09-19T07:53:04.088952+0000","last_became_active":"2024-09-19T07:53:02.954126+0000","last_became_peered":"2024-09-19T07:53:02.954126+0000","last_unstale":"2024-09-19T07:53:04.088952+0000","last_undegraded":"2024-09-19T07:53:04.088952+0000","last_fullsized":"2024-09-19T07:53:04.088952+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:21.471735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959826+0000","last_change":"2024-09-19T07:53:02.957284+0000","last_active":"2024-09-19T07:53:03.959826+0000","last_peered":"2024-09-19T07:53:03.959826+0000","last_clean":"2024-09-19T07:53:03.959826+0000","last_became_active":"2024-09-19T07:53:02.957008+0000","last_became_peered":"2024-09-19T07:53:02.957008+0000","last_unstale":"2024-09-19T07:53:03.959826+0000","last_undegraded":"2024-09-19T07:53:03.959826+0000","last_fullsized":"2024-09-19T07:53:03.959826+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:24.443678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089749+0000","last_change":"2024-09-19T07:53:02.959833+0000","last_active":"2024-09-19T07:53:04.089749+0000","last_peered":"2024-09-19T07:53:04.089749+0000","last_clean":"2024-09-19T07:53:04.089749+0000","last_became_active":"2024-09-19T07:53:02.959076+0000","last_became_peered":"2024-09-19T07:53:02.959076+0000","last_unstale":"2024-09-19T07:53:04.089749+0000","last_undegraded":"2024-09-19T07:53:04.089749+0000","last_fullsized":"2024-09-19T07:53:04.089749+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:31.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959718+0000","last_change":"2024-09-19T07:53:02.970410+0000","last_active":"2024-09-19T07:53:03.959718+0000","last_peered":"2024-09-19T07:53:03.959718+0000","last_clean":"2024-09-19T07:53:03.959718+0000","last_became_active":"2024-09-19T07:53:02.967129+0000","last_became_peered":"2024-09-19T07:53:02.967129+0000","last_unstale":"2024-09-19T07:53:03.959718+0000","last_undegraded":"2024-09-19T07:53:03.959718+0000","last_fullsized":"2024-09-19T07:53:03.959718+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:46.990754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951748+0000","last_change":"2024-09-19T07:53:02.953069+0000","last_active":"2024-09-19T07:53:03.951748+0000","last_peered":"2024-09-19T07:53:03.951748+0000","last_clean":"2024-09-19T07:53:03.951748+0000","last_became_active":"2024-09-19T07:53:02.952498+0000","last_became_peered":"2024-09-19T07:53:02.952498+0000","last_unstale":"2024-09-19T07:53:03.951748+0000","last_undegraded":"2024-09-19T07:53:03.951748+0000","last_fullsized":"2024-09-19T07:53:03.951748+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:31.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.969961+0000","last_change":"2024-09-19T07:53:02.960655+0000","last_active":"2024-09-19T07:53:02.969961+0000","last_peered":"2024-09-19T07:53:02.969961+0000","last_clean":"2024-09-19T07:53:02.969961+0000","last_became_active":"2024-09-19T07:53:02.960373+0000","last_became_peered":"2024-09-19T07:53:02.960373+0000","last_unstale":"2024-09-19T07:53:02.969961+0000","last_undegraded":"2024-09-19T07:53:02.969961+0000","last_fullsized":"2024-09-19T07:53:02.969961+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:42.406919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959448+0000","last_change":"2024-09-19T07:53:02.958018+0000","last_active":"2024-09-19T07:53:03.959448+0000","last_peered":"2024-09-19T07:53:03.959448+0000","last_clean":"2024-09-19T07:53:03.959448+0000","last_became_active":"2024-09-19T07:53:02.957706+0000","last_became_peered":"2024-09-19T07:53:02.957706+0000","last_unstale":"2024-09-19T07:53:03.959448+0000","last_undegraded":"2024-09-19T07:53:03.959448+0000","last_fullsized":"2024-09-19T07:53:03.959448+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:08.021359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.954880+0000","last_change":"2024-09-19T07:53:02.951003+0000","last_active":"2024-09-19T07:53:02.954880+0000","last_peered":"2024-09-19T07:53:02.954880+0000","last_clean":"2024-09-19T07:53:02.954880+0000","last_became_active":"2024-09-19T07:53:02.950262+0000","last_became_peered":"2024-09-19T07:53:02.950262+0000","last_unstale":"2024-09-19T07:53:02.954880+0000","last_undegraded":"2024-09-19T07:53:02.954880+0000","last_fullsized":"2024-09-19T07:53:02.954880+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:17.832687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.953137+0000","last_change":"2024-09-19T07:53:02.955789+0000","last_active":"2024-09-19T07:53:03.953137+0000","last_peered":"2024-09-19T07:53:03.953137+0000","last_clean":"2024-09-19T07:53:03.953137+0000","last_became_active":"2024-09-19T07:53:02.955480+0000","last_became_peered":"2024-09-19T07:53:02.955480+0000","last_unstale":"2024-09-19T07:53:03.953137+0000","last_undegraded":"2024-09-19T07:53:03.953137+0000","last_fullsized":"2024-09-19T07:53:03.953137+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:00.688869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952726+0000","last_change":"2024-09-19T07:53:02.962786+0000","last_active":"2024-09-19T07:53:03.952726+0000","last_peered":"2024-09-19T07:53:03.952726+0000","last_clean":"2024-09-19T07:53:03.952726+0000","last_became_active":"2024-09-19T07:53:02.961758+0000","last_became_peered":"2024-09-19T07:53:02.961758+0000","last_unstale":"2024-09-19T07:53:03.952726+0000","last_undegraded":"2024-09-19T07:53:03.952726+0000","last_fullsized":"2024-09-19T07:53:03.952726+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:43.470589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952391+0000","last_change":"2024-09-19T07:53:02.955834+0000","last_active":"2024-09-19T07:53:03.952391+0000","last_peered":"2024-09-19T07:53:03.952391+0000","last_clean":"2024-09-19T07:53:03.952391+0000","last_became_active":"2024-09-19T07:53:02.955070+0000","last_became_peered":"2024-09-19T07:53:02.955070+0000","last_unstale":"2024-09-19T07:53:03.952391+0000","last_undegraded":"2024-09-19T07:53:03.952391+0000","last_fullsized":"2024-09-19T07:53:03.952391+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:24.696392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089356+0000","last_change":"2024-09-19T07:53:02.963635+0000","last_active":"2024-09-19T07:53:04.089356+0000","last_peered":"2024-09-19T07:53:04.089356+0000","last_clean":"2024-09-19T07:53:04.089356+0000","last_became_active":"2024-09-19T07:53:02.963299+0000","last_became_peered":"2024-09-19T07:53:02.963299+0000","last_unstale":"2024-09-19T07:53:04.089356+0000","last_undegraded":"2024-09-19T07:53:04.089356+0000","last_fullsized":"2024-09-19T07:53:04.089356+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:03.858829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952126+0000","last_change":"2024-09-19T07:53:02.959004+0000","last_active":"2024-09-19T07:53:03.952126+0000","last_peered":"2024-09-19T07:53:03.952126+0000","last_clean":"2024-09-19T07:53:03.952126+0000","last_became_active":"2024-09-19T07:53:02.958656+0000","last_became_peered":"2024-09-19T07:53:02.958656+0000","last_unstale":"2024-09-19T07:53:03.952126+0000","last_undegraded":"2024-09-19T07:53:03.952126+0000","last_fullsized":"2024-09-19T07:53:03.952126+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:58.202929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089316+0000","last_change":"2024-09-19T07:53:02.968735+0000","last_active":"2024-09-19T07:53:04.089316+0000","last_peered":"2024-09-19T07:53:04.089316+0000","last_clean":"2024-09-19T07:53:04.089316+0000","last_became_active":"2024-09-19T07:53:02.968434+0000","last_became_peered":"2024-09-19T07:53:02.968434+0000","last_unstale":"2024-09-19T07:53:04.089316+0000","last_undegraded":"2024-09-19T07:53:04.089316+0000","last_fullsized":"2024-09-19T07:53:04.089316+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:57.435952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959514+0000","last_change":"2024-09-19T07:53:02.964781+0000","last_active":"2024-09-19T07:53:03.959514+0000","last_peered":"2024-09-19T07:53:03.959514+0000","last_clean":"2024-09-19T07:53:03.959514+0000","last_became_active":"2024-09-19T07:53:02.964513+0000","last_became_peered":"2024-09-19T07:53:02.964513+0000","last_unstale":"2024-09-19T07:53:03.959514+0000","last_undegraded":"2024-09-19T07:53:03.959514+0000","last_fullsized":"2024-09-19T07:53:03.959514+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:26.718784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"105'4","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.971186+0000","last_change":"2024-09-19T07:53:01.917100+0000","last_active":"2024-09-19T07:53:02.971186+0000","last_peered":"2024-09-19T07:53:02.971186+0000","last_clean":"2024-09-19T07:53:02.971186+0000","last_became_active":"2024-09-19T07:53:01.916801+0000","last_became_peered":"2024-09-19T07:53:01.916801+0000","last_unstale":"2024-09-19T07:53:02.971186+0000","last_undegraded":"2024-09-19T07:53:02.971186+0000","last_fullsized":"2024-09-19T07:53:02.971186+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:25:44.686436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-19T07:53:00.908452+0000","last_change":"2024-09-19T07:53:00.908452+0000","last_active":"2024-09-19T07:53:00.899035+0000","last_peered":"2024-09-19T07:53:00.899035+0000","last_clean":"2024-09-19T07:53:00.899035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:00.908452+0000","last_undegraded":"2024-09-19T07:53:00.908452+0000","last_fullsized":"2024-09-19T07:53:00.908452+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952282+0000","last_change":"2024-09-19T07:53:02.950143+0000","last_active":"2024-09-19T07:53:03.952282+0000","last_peered":"2024-09-19T07:53:03.952282+0000","last_clean":"2024-09-19T07:53:03.952282+0000","last_became_active":"2024-09-19T07:53:02.949855+0000","last_became_peered":"2024-09-19T07:53:02.949855+0000","last_unstale":"2024-09-19T07:53:03.952282+0000","last_undegraded":"2024-09-19T07:53:03.952282+0000","last_fullsized":"2024-09-19T07:53:03.952282+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:22.009944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951518+0000","last_change":"2024-09-19T07:53:01.926136+0000","last_active":"2024-09-19T07:53:03.951518+0000","last_peered":"2024-09-19T07:53:03.951518+0000","last_clean":"2024-09-19T07:53:03.951518+0000","last_became_active":"2024-09-19T07:53:01.924857+0000","last_became_peered":"2024-09-19T07:53:01.924857+0000","last_unstale":"2024-09-19T07:53:03.951518+0000","last_undegraded":"2024-09-19T07:53:03.951518+0000","last_fullsized":"2024-09-19T07:53:03.951518+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:26.340346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952916+0000","last_change":"2024-09-19T07:53:02.956369+0000","last_active":"2024-09-19T07:53:03.952916+0000","last_peered":"2024-09-19T07:53:03.952916+0000","last_clean":"2024-09-19T07:53:03.952916+0000","last_became_active":"2024-09-19T07:53:02.956011+0000","last_became_peered":"2024-09-19T07:53:02.956011+0000","last_unstale":"2024-09-19T07:53:03.952916+0000","last_undegraded":"2024-09-19T07:53:03.952916+0000","last_fullsized":"2024-09-19T07:53:03.952916+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:21.471735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.969829+0000","last_change":"2024-09-19T07:53:01.919358+0000","last_active":"2024-09-19T07:53:02.969829+0000","last_peered":"2024-09-19T07:53:02.969829+0000","last_clean":"2024-09-19T07:53:02.969829+0000","last_became_active":"2024-09-19T07:53:01.914884+0000","last_became_peered":"2024-09-19T07:53:01.914884+0000","last_unstale":"2024-09-19T07:53:02.969829+0000","last_undegraded":"2024-09-19T07:53:02.969829+0000","last_fullsized":"2024-09-19T07:53:02.969829+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:26.340346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.971960+0000","last_change":"2024-09-19T07:53:02.971101+0000","last_active":"2024-09-19T07:53:02.971960+0000","last_peered":"2024-09-19T07:53:02.971960+0000","last_clean":"2024-09-19T07:53:02.971960+0000","last_became_active":"2024-09-19T07:53:02.970767+0000","last_became_peered":"2024-09-19T07:53:02.970767+0000","last_unstale":"2024-09-19T07:53:02.971960+0000","last_undegraded":"2024-09-19T07:53:02.971960+0000","last_fullsized":"2024-09-19T07:53:02.971960+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:07.231140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089007+0000","last_change":"2024-09-19T07:53:01.923137+0000","last_active":"2024-09-19T07:53:04.089007+0000","last_peered":"2024-09-19T07:53:04.089007+0000","last_clean":"2024-09-19T07:53:04.089007+0000","last_became_active":"2024-09-19T07:53:01.922852+0000","last_became_peered":"2024-09-19T07:53:01.922852+0000","last_unstale":"2024-09-19T07:53:04.089007+0000","last_undegraded":"2024-09-19T07:53:04.089007+0000","last_fullsized":"2024-09-19T07:53:04.089007+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:31.183313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959859+0000","last_change":"2024-09-19T07:53:02.967452+0000","last_active":"2024-09-19T07:53:03.959859+0000","last_peered":"2024-09-19T07:53:03.959859+0000","last_clean":"2024-09-19T07:53:03.959859+0000","last_became_active":"2024-09-19T07:53:02.966768+0000","last_became_peered":"2024-09-19T07:53:02.966768+0000","last_unstale":"2024-09-19T07:53:03.959859+0000","last_undegraded":"2024-09-19T07:53:03.959859+0000","last_fullsized":"2024-09-19T07:53:03.959859+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:04.735526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952599+0000","last_change":"2024-09-19T07:53:01.919808+0000","last_active":"2024-09-19T07:53:03.952599+0000","last_peered":"2024-09-19T07:53:03.952599+0000","last_clean":"2024-09-19T07:53:03.952599+0000","last_became_active":"2024-09-19T07:53:01.919477+0000","last_became_peered":"2024-09-19T07:53:01.919477+0000","last_unstale":"2024-09-19T07:53:03.952599+0000","last_undegraded":"2024-09-19T07:53:03.952599+0000","last_fullsized":"2024-09-19T07:53:03.952599+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:21.008103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089819+0000","last_change":"2024-09-19T07:53:02.960753+0000","last_active":"2024-09-19T07:53:04.089819+0000","last_peered":"2024-09-19T07:53:04.089819+0000","last_clean":"2024-09-19T07:53:04.089819+0000","last_became_active":"2024-09-19T07:53:02.960465+0000","last_became_peered":"2024-09-19T07:53:02.960465+0000","last_unstale":"2024-09-19T07:53:04.089819+0000","last_undegraded":"2024-09-19T07:53:04.089819+0000","last_fullsized":"2024-09-19T07:53:04.089819+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:58.202929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951787+0000","last_change":"2024-09-19T07:53:01.916796+0000","last_active":"2024-09-19T07:53:03.951787+0000","last_peered":"2024-09-19T07:53:03.951787+0000","last_clean":"2024-09-19T07:53:03.951787+0000","last_became_active":"2024-09-19T07:53:01.916506+0000","last_became_peered":"2024-09-19T07:53:01.916506+0000","last_unstale":"2024-09-19T07:53:03.951787+0000","last_undegraded":"2024-09-19T07:53:03.951787+0000","last_fullsized":"2024-09-19T07:53:03.951787+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:50.457075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.965750+0000","last_change":"2024-09-19T07:53:02.963768+0000","last_active":"2024-09-19T07:53:02.965750+0000","last_peered":"2024-09-19T07:53:02.965750+0000","last_clean":"2024-09-19T07:53:02.965750+0000","last_became_active":"2024-09-19T07:53:02.963316+0000","last_became_peered":"2024-09-19T07:53:02.963316+0000","last_unstale":"2024-09-19T07:53:02.965750+0000","last_undegraded":"2024-09-19T07:53:02.965750+0000","last_fullsized":"2024-09-19T07:53:02.965750+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:40.843150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.947396+0000","last_change":"2024-09-19T07:53:01.922316+0000","last_active":"2024-09-19T07:53:02.947396+0000","last_peered":"2024-09-19T07:53:02.947396+0000","last_clean":"2024-09-19T07:53:02.947396+0000","last_became_active":"2024-09-19T07:53:01.922051+0000","last_became_peered":"2024-09-19T07:53:01.922051+0000","last_unstale":"2024-09-19T07:53:02.947396+0000","last_undegraded":"2024-09-19T07:53:02.947396+0000","last_fullsized":"2024-09-19T07:53:02.947396+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:20.469894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959407+0000","last_change":"2024-09-19T07:53:02.967631+0000","last_active":"2024-09-19T07:53:03.959407+0000","last_peered":"2024-09-19T07:53:03.959407+0000","last_clean":"2024-09-19T07:53:03.959407+0000","last_became_active":"2024-09-19T07:53:02.966419+0000","last_became_peered":"2024-09-19T07:53:02.966419+0000","last_unstale":"2024-09-19T07:53:03.959407+0000","last_undegraded":"2024-09-19T07:53:03.959407+0000","last_fullsized":"2024-09-19T07:53:03.959407+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:00.201893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.953111+0000","last_change":"2024-09-19T07:53:01.917284+0000","last_active":"2024-09-19T07:53:03.953111+0000","last_peered":"2024-09-19T07:53:03.953111+0000","last_clean":"2024-09-19T07:53:03.953111+0000","last_became_active":"2024-09-19T07:53:01.917007+0000","last_became_peered":"2024-09-19T07:53:01.917007+0000","last_unstale":"2024-09-19T07:53:03.953111+0000","last_undegraded":"2024-09-19T07:53:03.953111+0000","last_fullsized":"2024-09-19T07:53:03.953111+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:56.053928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.957662+0000","last_change":"2024-09-19T07:53:02.951942+0000","last_active":"2024-09-19T07:53:02.957662+0000","last_peered":"2024-09-19T07:53:02.957662+0000","last_clean":"2024-09-19T07:53:02.957662+0000","last_became_active":"2024-09-19T07:53:02.950538+0000","last_became_peered":"2024-09-19T07:53:02.950538+0000","last_unstale":"2024-09-19T07:53:02.957662+0000","last_undegraded":"2024-09-19T07:53:02.957662+0000","last_fullsized":"2024-09-19T07:53:02.957662+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:50.313345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959653+0000","last_change":"2024-09-19T07:53:01.923243+0000","last_active":"2024-09-19T07:53:03.959653+0000","last_peered":"2024-09-19T07:53:03.959653+0000","last_clean":"2024-09-19T07:53:03.959653+0000","last_became_active":"2024-09-19T07:53:01.922939+0000","last_became_peered":"2024-09-19T07:53:01.922939+0000","last_unstale":"2024-09-19T07:53:03.959653+0000","last_undegraded":"2024-09-19T07:53:03.959653+0000","last_fullsized":"2024-09-19T07:53:03.959653+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:55.701057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"105'3","reported_seq":17,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089046+0000","last_change":"2024-09-19T07:53:01.923641+0000","last_active":"2024-09-19T07:53:04.089046+0000","last_peered":"2024-09-19T07:53:04.089046+0000","last_clean":"2024-09-19T07:53:04.089046+0000","last_became_active":"2024-09-19T07:53:01.923379+0000","last_became_peered":"2024-09-19T07:53:01.923379+0000","last_unstale":"2024-09-19T07:53:04.089046+0000","last_undegraded":"2024-09-19T07:53:04.089046+0000","last_fullsized":"2024-09-19T07:53:04.089046+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:51:46.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.955166+0000","last_change":"2024-09-19T07:53:02.953561+0000","last_active":"2024-09-19T07:53:02.955166+0000","last_peered":"2024-09-19T07:53:02.955166+0000","last_clean":"2024-09-19T07:53:02.955166+0000","last_became_active":"2024-09-19T07:53:02.953304+0000","last_became_peered":"2024-09-19T07:53:02.953304+0000","last_unstale":"2024-09-19T07:53:02.955166+0000","last_undegraded":"2024-09-19T07:53:02.955166+0000","last_fullsized":"2024-09-19T07:53:02.955166+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:03.858829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"105'6","reported_seq":20,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089486+0000","last_change":"2024-09-19T07:53:01.924833+0000","last_active":"2024-09-19T07:53:04.089486+0000","last_peered":"2024-09-19T07:53:04.089486+0000","last_clean":"2024-09-19T07:53:04.089486+0000","last_became_active":"2024-09-19T07:53:01.924568+0000","last_became_peered":"2024-09-19T07:53:01.924568+0000","last_unstale":"2024-09-19T07:53:04.089486+0000","last_undegraded":"2024-09-19T07:53:04.089486+0000","last_fullsized":"2024-09-19T07:53:04.089486+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:49.033408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952811+0000","last_change":"2024-09-19T07:53:02.955505+0000","last_active":"2024-09-19T07:53:03.952811+0000","last_peered":"2024-09-19T07:53:03.952811+0000","last_clean":"2024-09-19T07:53:03.952811+0000","last_became_active":"2024-09-19T07:53:02.955208+0000","last_became_peered":"2024-09-19T07:53:02.955208+0000","last_unstale":"2024-09-19T07:53:03.952811+0000","last_undegraded":"2024-09-19T07:53:03.952811+0000","last_fullsized":"2024-09-19T07:53:03.952811+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:56.130236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.969986+0000","last_change":"2024-09-19T07:53:01.920568+0000","last_active":"2024-09-19T07:53:02.969986+0000","last_peered":"2024-09-19T07:53:02.969986+0000","last_clean":"2024-09-19T07:53:02.969986+0000","last_became_active":"2024-09-19T07:53:01.920282+0000","last_became_peered":"2024-09-19T07:53:01.920282+0000","last_unstale":"2024-09-19T07:53:02.969986+0000","last_undegraded":"2024-09-19T07:53:02.969986+0000","last_fullsized":"2024-09-19T07:53:02.969986+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:11.122190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089492+0000","last_change":"2024-09-19T07:53:02.963599+0000","last_active":"2024-09-19T07:53:04.089492+0000","last_peered":"2024-09-19T07:53:04.089492+0000","last_clean":"2024-09-19T07:53:04.089492+0000","last_became_active":"2024-09-19T07:53:02.963303+0000","last_became_peered":"2024-09-19T07:53:02.963303+0000","last_unstale":"2024-09-19T07:53:04.089492+0000","last_undegraded":"2024-09-19T07:53:04.089492+0000","last_fullsized":"2024-09-19T07:53:04.089492+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:42.406919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.950643+0000","last_change":"2024-09-19T07:53:01.919711+0000","last_active":"2024-09-19T07:53:02.950643+0000","last_peered":"2024-09-19T07:53:02.950643+0000","last_clean":"2024-09-19T07:53:02.950643+0000","last_became_active":"2024-09-19T07:53:01.919443+0000","last_became_peered":"2024-09-19T07:53:01.919443+0000","last_unstale":"2024-09-19T07:53:02.950643+0000","last_undegraded":"2024-09-19T07:53:02.950643+0000","last_fullsized":"2024-09-19T07:53:02.950643+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:38.351958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-19T07:53:00.907480+0000","last_change":"2024-09-19T07:53:00.907480+0000","last_active":"2024-09-19T07:53:00.899035+0000","last_peered":"2024-09-19T07:53:00.899035+0000","last_clean":"2024-09-19T07:53:00.899035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:00.907480+0000","last_undegraded":"2024-09-19T07:53:00.907480+0000","last_fullsized":"2024-09-19T07:53:00.907480+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959932+0000","last_change":"2024-09-19T07:53:02.962652+0000","last_active":"2024-09-19T07:53:03.959932+0000","last_peered":"2024-09-19T07:53:03.959932+0000","last_clean":"2024-09-19T07:53:03.959932+0000","last_became_active":"2024-09-19T07:53:02.962394+0000","last_became_peered":"2024-09-19T07:53:02.962394+0000","last_unstale":"2024-09-19T07:53:03.959932+0000","last_undegraded":"2024-09-19T07:53:03.959932+0000","last_fullsized":"2024-09-19T07:53:03.959932+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:46.985872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"105'8","reported_seq":22,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951439+0000","last_change":"2024-09-19T07:53:01.918580+0000","last_active":"2024-09-19T07:53:03.951439+0000","last_peered":"2024-09-19T07:53:03.951439+0000","last_clean":"2024-09-19T07:53:03.951439+0000","last_became_active":"2024-09-19T07:53:01.918222+0000","last_became_peered":"2024-09-19T07:53:01.918222+0000","last_unstale":"2024-09-19T07:53:03.951439+0000","last_undegraded":"2024-09-19T07:53:03.951439+0000","last_fullsized":"2024-09-19T07:53:03.951439+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:06.785416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089618+0000","last_change":"2024-09-19T07:53:01.922470+0000","last_active":"2024-09-19T07:53:04.089618+0000","last_peered":"2024-09-19T07:53:04.089618+0000","last_clean":"2024-09-19T07:53:04.089618+0000","last_became_active":"2024-09-19T07:53:01.922181+0000","last_became_peered":"2024-09-19T07:53:01.922181+0000","last_unstale":"2024-09-19T07:53:04.089618+0000","last_undegraded":"2024-09-19T07:53:04.089618+0000","last_fullsized":"2024-09-19T07:53:04.089618+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:47.737086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089090+0000","last_change":"2024-09-19T07:53:02.965849+0000","last_active":"2024-09-19T07:53:04.089090+0000","last_peered":"2024-09-19T07:53:04.089090+0000","last_clean":"2024-09-19T07:53:04.089090+0000","last_became_active":"2024-09-19T07:53:02.965562+0000","last_became_peered":"2024-09-19T07:53:02.965562+0000","last_unstale":"2024-09-19T07:53:04.089090+0000","last_undegraded":"2024-09-19T07:53:04.089090+0000","last_fullsized":"2024-09-19T07:53:04.089090+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:50.313345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089146+0000","last_change":"2024-09-19T07:53:01.922247+0000","last_active":"2024-09-19T07:53:04.089146+0000","last_peered":"2024-09-19T07:53:04.089146+0000","last_clean":"2024-09-19T07:53:04.089146+0000","last_became_active":"2024-09-19T07:53:01.921944+0000","last_became_peered":"2024-09-19T07:53:01.921944+0000","last_unstale":"2024-09-19T07:53:04.089146+0000","last_undegraded":"2024-09-19T07:53:04.089146+0000","last_fullsized":"2024-09-19T07:53:04.089146+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:40.569155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.955145+0000","last_change":"2024-09-19T07:53:02.952785+0000","last_active":"2024-09-19T07:53:02.955145+0000","last_peered":"2024-09-19T07:53:02.955145+0000","last_clean":"2024-09-19T07:53:02.955145+0000","last_became_active":"2024-09-19T07:53:02.952535+0000","last_became_peered":"2024-09-19T07:53:02.952535+0000","last_unstale":"2024-09-19T07:53:02.955145+0000","last_undegraded":"2024-09-19T07:53:02.955145+0000","last_fullsized":"2024-09-19T07:53:02.955145+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:57.435952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952771+0000","last_change":"2024-09-19T07:53:01.925865+0000","last_active":"2024-09-19T07:53:03.952771+0000","last_peered":"2024-09-19T07:53:03.952771+0000","last_clean":"2024-09-19T07:53:03.952771+0000","last_became_active":"2024-09-19T07:53:01.925544+0000","last_became_peered":"2024-09-19T07:53:01.925544+0000","last_unstale":"2024-09-19T07:53:03.952771+0000","last_undegraded":"2024-09-19T07:53:03.952771+0000","last_fullsized":"2024-09-19T07:53:03.952771+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:57.201088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959514+0000","last_change":"2024-09-19T07:53:02.970190+0000","last_active":"2024-09-19T07:53:03.959514+0000","last_peered":"2024-09-19T07:53:03.959514+0000","last_clean":"2024-09-19T07:53:03.959514+0000","last_became_active":"2024-09-19T07:53:02.969897+0000","last_became_peered":"2024-09-19T07:53:02.969897+0000","last_unstale":"2024-09-19T07:53:03.959514+0000","last_undegraded":"2024-09-19T07:53:03.959514+0000","last_fullsized":"2024-09-19T07:53:03.959514+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:24.195258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-19T07:53:00.907636+0000","last_change":"2024-09-19T07:53:00.907636+0000","last_active":"2024-09-19T07:53:00.899035+0000","last_peered":"2024-09-19T07:53:00.899035+0000","last_clean":"2024-09-19T07:53:00.899035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:00.907636+0000","last_undegraded":"2024-09-19T07:53:00.907636+0000","last_fullsized":"2024-09-19T07:53:00.907636+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959770+0000","last_change":"2024-09-19T07:53:02.946678+0000","last_active":"2024-09-19T07:53:03.959770+0000","last_peered":"2024-09-19T07:53:03.959770+0000","last_clean":"2024-09-19T07:53:03.959770+0000","last_became_active":"2024-09-19T07:53:02.946357+0000","last_became_peered":"2024-09-19T07:53:02.946357+0000","last_unstale":"2024-09-19T07:53:03.959770+0000","last_undegraded":"2024-09-19T07:53:03.959770+0000","last_fullsized":"2024-09-19T07:53:03.959770+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:20.948125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-19T07:53:00.908527+0000","last_change":"2024-09-19T07:53:00.908527+0000","last_active":"2024-09-19T07:53:00.899035+0000","last_peered":"2024-09-19T07:53:00.899035+0000","last_clean":"2024-09-19T07:53:00.899035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:00.908527+0000","last_undegraded":"2024-09-19T07:53:00.908527+0000","last_fullsized":"2024-09-19T07:53:00.908527+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.968168+0000","last_change":"2024-09-19T07:53:02.965053+0000","last_active":"2024-09-19T07:53:02.968168+0000","last_peered":"2024-09-19T07:53:02.968168+0000","last_clean":"2024-09-19T07:53:02.968168+0000","last_became_active":"2024-09-19T07:53:02.964769+0000","last_became_peered":"2024-09-19T07:53:02.964769+0000","last_unstale":"2024-09-19T07:53:02.968168+0000","last_undegraded":"2024-09-19T07:53:02.968168+0000","last_fullsized":"2024-09-19T07:53:02.968168+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:40.771106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089316+0000","last_change":"2024-09-19T07:53:01.923054+0000","last_active":"2024-09-19T07:53:04.089316+0000","last_peered":"2024-09-19T07:53:04.089316+0000","last_clean":"2024-09-19T07:53:04.089316+0000","last_became_active":"2024-09-19T07:53:01.922799+0000","last_became_peered":"2024-09-19T07:53:01.922799+0000","last_unstale":"2024-09-19T07:53:04.089316+0000","last_undegraded":"2024-09-19T07:53:04.089316+0000","last_fullsized":"2024-09-19T07:53:04.089316+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:47.745785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-19T07:53:00.907876+0000","last_change":"2024-09-19T07:53:00.907876+0000","last_active":"2024-09-19T07:53:00.899035+0000","last_peered":"2024-09-19T07:53:00.899035+0000","last_clean":"2024-09-19T07:53:00.899035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:00.907876+0000","last_undegraded":"2024-09-19T07:53:00.907876+0000","last_fullsized":"2024-09-19T07:53:00.907876+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959382+0000","last_change":"2024-09-19T07:53:02.968250+0000","last_active":"2024-09-19T07:53:03.959382+0000","last_peered":"2024-09-19T07:53:03.959382+0000","last_clean":"2024-09-19T07:53:03.959382+0000","last_became_active":"2024-09-19T07:53:02.967961+0000","last_became_peered":"2024-09-19T07:53:02.967961+0000","last_unstale":"2024-09-19T07:53:03.959382+0000","last_undegraded":"2024-09-19T07:53:03.959382+0000","last_fullsized":"2024-09-19T07:53:03.959382+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:03.803861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.969747+0000","last_change":"2024-09-19T07:53:01.922110+0000","last_active":"2024-09-19T07:53:02.969747+0000","last_peered":"2024-09-19T07:53:02.969747+0000","last_clean":"2024-09-19T07:53:02.969747+0000","last_became_active":"2024-09-19T07:53:01.921769+0000","last_became_peered":"2024-09-19T07:53:01.921769+0000","last_unstale":"2024-09-19T07:53:02.969747+0000","last_undegraded":"2024-09-19T07:53:02.969747+0000","last_fullsized":"2024-09-19T07:53:02.969747+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:56.053928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951681+0000","last_change":"2024-09-19T07:53:02.963616+0000","last_active":"2024-09-19T07:53:03.951681+0000","last_peered":"2024-09-19T07:53:03.951681+0000","last_clean":"2024-09-19T07:53:03.951681+0000","last_became_active":"2024-09-19T07:53:02.963333+0000","last_became_peered":"2024-09-19T07:53:02.963333+0000","last_unstale":"2024-09-19T07:53:03.951681+0000","last_undegraded":"2024-09-19T07:53:03.951681+0000","last_fullsized":"2024-09-19T07:53:03.951681+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"106'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.961104+0000","last_change":"2024-09-19T07:53:01.922601+0000","last_active":"2024-09-19T07:53:03.961104+0000","last_peered":"2024-09-19T07:53:03.961104+0000","last_clean":"2024-09-19T07:53:03.961104+0000","last_became_active":"2024-09-19T07:53:01.922324+0000","last_became_peered":"2024-09-19T07:53:01.922324+0000","last_unstale":"2024-09-19T07:53:03.961104+0000","last_undegraded":"2024-09-19T07:53:03.961104+0000","last_fullsized":"2024-09-19T07:53:03.961104+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:02.428894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.962345+0000","last_change":"2024-09-19T07:53:02.961218+0000","last_active":"2024-09-19T07:53:02.962345+0000","last_peered":"2024-09-19T07:53:02.962345+0000","last_clean":"2024-09-19T07:53:02.962345+0000","last_became_active":"2024-09-19T07:53:02.960873+0000","last_became_peered":"2024-09-19T07:53:02.960873+0000","last_unstale":"2024-09-19T07:53:02.962345+0000","last_undegraded":"2024-09-19T07:53:02.962345+0000","last_fullsized":"2024-09-19T07:53:02.962345+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:27.358096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089482+0000","last_change":"2024-09-19T07:53:01.920744+0000","last_active":"2024-09-19T07:53:04.089482+0000","last_peered":"2024-09-19T07:53:04.089482+0000","last_clean":"2024-09-19T07:53:04.089482+0000","last_became_active":"2024-09-19T07:53:01.920476+0000","last_became_peered":"2024-09-19T07:53:01.920476+0000","last_unstale":"2024-09-19T07:53:04.089482+0000","last_undegraded":"2024-09-19T07:53:04.089482+0000","last_fullsized":"2024-09-19T07:53:04.089482+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:11.662918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952379+0000","last_change":"2024-09-19T07:53:02.961727+0000","last_active":"2024-09-19T07:53:03.952379+0000","last_peered":"2024-09-19T07:53:03.952379+0000","last_clean":"2024-09-19T07:53:03.952379+0000","last_became_active":"2024-09-19T07:53:02.961447+0000","last_became_peered":"2024-09-19T07:53:02.961447+0000","last_unstale":"2024-09-19T07:53:03.952379+0000","last_undegraded":"2024-09-19T07:53:03.952379+0000","last_fullsized":"2024-09-19T07:53:03.952379+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:26:55.837328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.940797+0000","last_change":"2024-09-19T07:53:01.922583+0000","last_active":"2024-09-19T07:53:02.940797+0000","last_peered":"2024-09-19T07:53:02.940797+0000","last_clean":"2024-09-19T07:53:02.940797+0000","last_became_active":"2024-09-19T07:53:01.922318+0000","last_became_peered":"2024-09-19T07:53:01.922318+0000","last_unstale":"2024-09-19T07:53:02.940797+0000","last_undegraded":"2024-09-19T07:53:02.940797+0000","last_fullsized":"2024-09-19T07:53:02.940797+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:14.312065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952032+0000","last_change":"2024-09-19T07:53:01.918087+0000","last_active":"2024-09-19T07:53:03.952032+0000","last_peered":"2024-09-19T07:53:03.952032+0000","last_clean":"2024-09-19T07:53:03.952032+0000","last_became_active":"2024-09-19T07:53:01.917738+0000","last_became_peered":"2024-09-19T07:53:01.917738+0000","last_unstale":"2024-09-19T07:53:03.952032+0000","last_undegraded":"2024-09-19T07:53:03.952032+0000","last_fullsized":"2024-09-19T07:53:03.952032+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:14.312065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:53:01.900875+0000","last_change":"2024-09-19T07:53:01.900875+0000","last_active":"2024-09-19T07:53:01.900875+0000","last_peered":"2024-09-19T07:53:01.900875+0000","last_clean":"2024-09-19T07:53:01.900875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:01.900875+0000","last_undegraded":"2024-09-19T07:53:01.900875+0000","last_fullsized":"2024-09-19T07:53:01.900875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-19T07:53:00.908081+0000","last_change":"2024-09-19T07:53:00.908081+0000","last_active":"2024-09-19T07:53:00.899035+0000","last_peered":"2024-09-19T07:53:00.899035+0000","last_clean":"2024-09-19T07:53:00.899035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:00.908081+0000","last_undegraded":"2024-09-19T07:53:00.908081+0000","last_fullsized":"2024-09-19T07:53:00.908081+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089415+0000","last_change":"2024-09-19T07:53:02.957282+0000","last_active":"2024-09-19T07:53:04.089415+0000","last_peered":"2024-09-19T07:53:04.089415+0000","last_clean":"2024-09-19T07:53:04.089415+0000","last_became_active":"2024-09-19T07:53:02.957004+0000","last_became_peered":"2024-09-19T07:53:02.957004+0000","last_unstale":"2024-09-19T07:53:04.089415+0000","last_undegraded":"2024-09-19T07:53:04.089415+0000","last_fullsized":"2024-09-19T07:53:04.089415+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:25:45.688277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089434+0000","last_change":"2024-09-19T07:53:01.921607+0000","last_active":"2024-09-19T07:53:04.089434+0000","last_peered":"2024-09-19T07:53:04.089434+0000","last_clean":"2024-09-19T07:53:04.089434+0000","last_became_active":"2024-09-19T07:53:01.920937+0000","last_became_peered":"2024-09-19T07:53:01.920937+0000","last_unstale":"2024-09-19T07:53:04.089434+0000","last_undegraded":"2024-09-19T07:53:04.089434+0000","last_fullsized":"2024-09-19T07:53:04.089434+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:35.664386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.953006+0000","last_change":"2024-09-19T07:53:02.956981+0000","last_active":"2024-09-19T07:53:03.953006+0000","last_peered":"2024-09-19T07:53:03.953006+0000","last_clean":"2024-09-19T07:53:03.953006+0000","last_became_active":"2024-09-19T07:53:02.955937+0000","last_became_peered":"2024-09-19T07:53:02.955937+0000","last_unstale":"2024-09-19T07:53:03.953006+0000","last_undegraded":"2024-09-19T07:53:03.953006+0000","last_fullsized":"2024-09-19T07:53:03.953006+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:39.353799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-19T07:53:00.907369+0000","last_change":"2024-09-19T07:53:00.907369+0000","last_active":"2024-09-19T07:53:00.899035+0000","last_peered":"2024-09-19T07:53:00.899035+0000","last_clean":"2024-09-19T07:53:00.899035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:00.907369+0000","last_undegraded":"2024-09-19T07:53:00.907369+0000","last_fullsized":"2024-09-19T07:53:00.907369+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951881+0000","last_change":"2024-09-19T07:53:02.955750+0000","last_active":"2024-09-19T07:53:03.951881+0000","last_peered":"2024-09-19T07:53:03.951881+0000","last_clean":"2024-09-19T07:53:03.951881+0000","last_became_active":"2024-09-19T07:53:02.955250+0000","last_became_peered":"2024-09-19T07:53:02.955250+0000","last_unstale":"2024-09-19T07:53:03.951881+0000","last_undegraded":"2024-09-19T07:53:03.951881+0000","last_fullsized":"2024-09-19T07:53:03.951881+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:22.292252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-19T07:53:00.907937+0000","last_change":"2024-09-19T07:53:00.907937+0000","last_active":"2024-09-19T07:53:00.899035+0000","last_peered":"2024-09-19T07:53:00.899035+0000","last_clean":"2024-09-19T07:53:00.899035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:00.907937+0000","last_undegraded":"2024-09-19T07:53:00.907937+0000","last_fullsized":"2024-09-19T07:53:00.907937+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":325,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088582+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:53:04.088582+0000","last_peered":"2024-09-19T07:53:04.088582+0000","last_clean":"2024-09-19T07:53:04.088582+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:53:04.088582+0000","last_undegraded":"2024-09-19T07:53:04.088582+0000","last_fullsized":"2024-09-19T07:53:04.088582+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":106,"acting":106,"num_store_stats":7},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509565,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82960,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288880,"statfs":{"total":96636764160,"available":96551813120,"internally_reserved":0,"allocated":1896448,"data_stored":1188786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509564,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100520,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":94271320,"statfs":{"total":96636764160,"available":96533831680,"internally_reserved":0,"allocated":1327104,"data_stored":607166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":101579206},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509565,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50512,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321328,"statfs":{"total":96636764160,"available":96585039872,"internally_reserved":0,"allocated":1306624,"data_stored":604950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509565,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86552,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285288,"statfs":{"total":96636764160,"available":96548134912,"internally_reserved":0,"allocated":1314816,"data_stored":605042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509564,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":101020,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94270820,"statfs":{"total":96636764160,"available":96533319680,"internally_reserved":0,"allocated":1904640,"data_stored":1190328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542268,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135120,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236720,"statfs":{"total":96636764160,"available":96498401280,"internally_reserved":0,"allocated":1298432,"data_stored":598840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":137034183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542268,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94928,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276912,"statfs":{"total":96636764160,"available":96539557888,"internally_reserved":0,"allocated":1298432,"data_stored":598292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104556,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267284,"statfs":{"total":96636764160,"available":96529698816,"internally_reserved":0,"allocated":1269760,"data_stored":584424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:53:06.287 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:53:06.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.426+0000 7fd5d172a640 1 Processor -- start 2024-09-19T07:53:06.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.426+0000 7fd5d172a640 1 -- start start 2024-09-19T07:53:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.427+0000 7fd5d172a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5cc141470 0x7fd5cc141b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.427+0000 7fd5d172a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd5cc0d2130 0x7fd5cc0d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.427+0000 7fd5d172a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5cc0d2ae0 con 0x7fd5cc06ce70 2024-09-19T07:53:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.427+0000 7fd5d172a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fd5cc161f10 con 0x7fd5cc0d2130 2024-09-19T07:53:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.427+0000 7fd5d172a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd5cc141850 con 0x7fd5cc141470 2024-09-19T07:53:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.427+0000 7fd5ca7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd5cc06ce70 0x7fd5cc06d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60158/0 (socket says 172.21.15.16:60158) 2024-09-19T07:53:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.427+0000 7fd5ca7fc640 1 -- 172.21.15.16:0/2117439459 learned_addr learned my addr 172.21.15.16:0/2117439459 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5caffd640 1 --2- 172.21.15.16:0/2117439459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5cc141470 0x7fd5cc141b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5c9ffb640 1 -- 172.21.15.16:0/2117439459 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2860592443 0 0) 0x7fd5cc0d2ae0 con 0x7fd5cc06ce70 2024-09-19T07:53:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5cb7fe640 1 --2- 172.21.15.16:0/2117439459 >> v2:172.21.15.146:3300/0 conn(0x7fd5cc0d2130 0x7fd5cc0d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5c9ffb640 1 -- 172.21.15.16:0/2117439459 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5b4003520 con 0x7fd5cc06ce70 2024-09-19T07:53:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5c9ffb640 1 -- 172.21.15.16:0/2117439459 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1583351638 0 0) 0x7fd5b4003520 con 0x7fd5cc06ce70 2024-09-19T07:53:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5c9ffb640 1 -- 172.21.15.16:0/2117439459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5cc141470 msgr2=0x7fd5cc141b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5c9ffb640 1 --2- 172.21.15.16:0/2117439459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5cc141470 0x7fd5cc141b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5c9ffb640 1 -- 172.21.15.16:0/2117439459 >> v2:172.21.15.146:3300/0 conn(0x7fd5cc0d2130 msgr2=0x7fd5cc0d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5c9ffb640 1 --2- 172.21.15.16:0/2117439459 >> v2:172.21.15.146:3300/0 conn(0x7fd5cc0d2130 0x7fd5cc0d25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5c9ffb640 1 -- 172.21.15.16:0/2117439459 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5cc1621b0 con 0x7fd5cc06ce70 2024-09-19T07:53:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5c9ffb640 1 -- 172.21.15.16:0/2117439459 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fd5c0001670 con 0x7fd5cc06ce70 2024-09-19T07:53:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5c9ffb640 1 -- 172.21.15.16:0/2117439459 >> v1:172.21.15.16:6789/0 conn(0x7fd5cc06ce70 legacy=0x7fd5cc06d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5c9ffb640 1 --2- 172.21.15.16:0/2117439459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b40039b0 0x7fd5b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5c9ffb640 1 --2- 172.21.15.16:0/2117439459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5b4004800 0x7fd5b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5c9ffb640 1 --2- 172.21.15.16:0/2117439459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5cc141470 0x7fd5b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5c9ffb640 1 -- 172.21.15.16:0/2117439459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc161f10 con 0x7fd5b4004800 2024-09-19T07:53:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5c9ffb640 1 -- 172.21.15.16:0/2117439459 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc141850 con 0x7fd5b40039b0 2024-09-19T07:53:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5c9ffb640 1 -- 172.21.15.16:0/2117439459 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b400b9c0 con 0x7fd5cc141470 2024-09-19T07:53:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5ca7fc640 1 --2- 172.21.15.16:0/2117439459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5b4004800 0x7fd5b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5caffd640 1 --2- 172.21.15.16:0/2117439459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b40039b0 0x7fd5b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5cb7fe640 1 --2- 172.21.15.16:0/2117439459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5cc141470 0x7fd5b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5ca7fc640 1 -- 172.21.15.16:0/2117439459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5cc141470 msgr2=0x7fd5b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5ca7fc640 1 --2- 172.21.15.16:0/2117439459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5cc141470 0x7fd5b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5ca7fc640 1 -- 172.21.15.16:0/2117439459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b40039b0 msgr2=0x7fd5b4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5ca7fc640 1 --2- 172.21.15.16:0/2117439459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b40039b0 0x7fd5b4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5ca7fc640 1 -- 172.21.15.16:0/2117439459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b4003520 con 0x7fd5b4004800 2024-09-19T07:53:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.428+0000 7fd5cb7fe640 1 --2- 172.21.15.16:0/2117439459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5cc141470 0x7fd5b40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.429+0000 7fd5ca7fc640 1 --2- 172.21.15.16:0/2117439459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5b4004800 0x7fd5b4004c80 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0001050 tx=0x7fd5b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef4cd6ed7e3e28d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.429+0000 7fd5c9ffb640 1 -- 172.21.15.16:0/2117439459 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0007d70 con 0x7fd5b4004800 2024-09-19T07:53:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.429+0000 7fd5c9ffb640 1 -- 172.21.15.16:0/2117439459 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c0005af0 con 0x7fd5b4004800 2024-09-19T07:53:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.429+0000 7fd5c9ffb640 1 -- 172.21.15.16:0/2117439459 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0007070 con 0x7fd5b4004800 2024-09-19T07:53:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.430+0000 7fd5d172a640 1 -- 172.21.15.16:0/2117439459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5b4004800 msgr2=0x7fd5b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.430+0000 7fd5d172a640 1 --2- 172.21.15.16:0/2117439459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5b4004800 0x7fd5b4004c80 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0001050 tx=0x7fd5b4005830 comp rx=0 tx=0).stop 2024-09-19T07:53:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.430+0000 7fd5cb7fe640 1 -- 172.21.15.16:0/2117439459 reap_dead start 2024-09-19T07:53:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.430+0000 7fd5d172a640 1 -- 172.21.15.16:0/2117439459 shutdown_connections 2024-09-19T07:53:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.430+0000 7fd5d172a640 1 -- 172.21.15.16:0/2117439459 >> 172.21.15.16:0/2117439459 conn(0x7fd5cc0b4040 msgr2=0x7fd5cc0b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.430+0000 7fd5d172a640 1 -- 172.21.15.16:0/2117439459 shutdown_connections 2024-09-19T07:53:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.430+0000 7fd5d172a640 1 -- 172.21.15.16:0/2117439459 wait complete. 2024-09-19T07:53:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.431+0000 7fd5d172a640 1 Processor -- start 2024-09-19T07:53:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.431+0000 7fd5d172a640 1 -- start start 2024-09-19T07:53:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.432+0000 7fd5d172a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5b4004800 0x7fd5cc1476f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.432+0000 7fd5d172a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5cc1f4500 0x7fd5cc160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.432+0000 7fd5d172a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5cc161010 0x7fd5cc15d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.432+0000 7fd5d172a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc162a50 con 0x7fd5cc161010 2024-09-19T07:53:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.432+0000 7fd5d172a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc162770 con 0x7fd5cc1f4500 2024-09-19T07:53:06.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.432+0000 7fd5d172a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd5cc162d30 con 0x7fd5b4004800 2024-09-19T07:53:06.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.432+0000 7fd5cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5cc161010 0x7fd5cc15d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.432+0000 7fd5cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5cc161010 0x7fd5cc15d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60756/0 (socket says 172.21.15.16:60756) 2024-09-19T07:53:06.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.432+0000 7fd5cb7fe640 1 -- 172.21.15.16:0/718361865 learned_addr learned my addr 172.21.15.16:0/718361865 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:06.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.432+0000 7fd5ca7fc640 1 --2- 172.21.15.16:0/718361865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5cc1f4500 0x7fd5cc160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.432+0000 7fd5caffd640 1 --2- 172.21.15.16:0/718361865 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5b4004800 0x7fd5cc1476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.432+0000 7fd5cb7fe640 1 -- 172.21.15.16:0/718361865 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5b4004800 msgr2=0x7fd5cc1476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.432+0000 7fd5cb7fe640 1 --2- 172.21.15.16:0/718361865 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5b4004800 0x7fd5cc1476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.432+0000 7fd5cb7fe640 1 -- 172.21.15.16:0/718361865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5cc1f4500 msgr2=0x7fd5cc160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.432+0000 7fd5cb7fe640 1 --2- 172.21.15.16:0/718361865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5cc1f4500 0x7fd5cc160ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.432+0000 7fd5cb7fe640 1 -- 172.21.15.16:0/718361865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5cc161490 con 0x7fd5cc161010 2024-09-19T07:53:06.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.433+0000 7fd5cb7fe640 1 --2- 172.21.15.16:0/718361865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5cc161010 0x7fd5cc15d340 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc071930 tx=0x7fd5c4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:06.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.433+0000 7fd5b37fe640 1 -- 172.21.15.16:0/718361865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4010e50 con 0x7fd5cc161010 2024-09-19T07:53:06.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.433+0000 7fd5b37fe640 1 -- 172.21.15.16:0/718361865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c4003990 con 0x7fd5cc161010 2024-09-19T07:53:06.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.433+0000 7fd5b37fe640 1 -- 172.21.15.16:0/718361865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4003d80 con 0x7fd5cc161010 2024-09-19T07:53:06.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.433+0000 7fd5d172a640 1 -- 172.21.15.16:0/718361865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5cc1f4980 con 0x7fd5cc161010 2024-09-19T07:53:06.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.433+0000 7fd5d172a640 1 -- 172.21.15.16:0/718361865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5cc146930 con 0x7fd5cc161010 2024-09-19T07:53:06.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.434+0000 7fd5d172a640 1 -- 172.21.15.16:0/718361865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5cc142cf0 con 0x7fd5cc161010 2024-09-19T07:53:06.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.434+0000 7fd5b37fe640 1 -- 172.21.15.16:0/718361865 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd5c4005d10 con 0x7fd5cc161010 2024-09-19T07:53:06.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.435+0000 7fd5b37fe640 1 --2- 172.21.15.16:0/718361865 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd5b806c2e0 0x7fd5b806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.435+0000 7fd5b37fe640 1 -- 172.21.15.16:0/718361865 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fd5c408bdb0 con 0x7fd5cc161010 2024-09-19T07:53:06.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.435+0000 7fd5caffd640 1 --2- 172.21.15.16:0/718361865 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd5b806c2e0 0x7fd5b806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.436+0000 7fd5caffd640 1 --2- 172.21.15.16:0/718361865 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd5b806c2e0 0x7fd5b806e7a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0099c0 tx=0x7fd5bc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:06.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.439+0000 7fd5b37fe640 1 -- 172.21.15.16:0/718361865 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd5c4025020 con 0x7fd5cc161010 2024-09-19T07:53:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.613+0000 7fd5d172a640 1 -- 172.21.15.16:0/718361865 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd5cc0b3ac0 con 0x7fd5b806c2e0 2024-09-19T07:53:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.620+0000 7fd5b37fe640 1 -- 172.21.15.16:0/718361865 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252479 (secure 0 0 0) 0x7fd5cc0b3ac0 con 0x7fd5b806c2e0 2024-09-19T07:53:06.623 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:53:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.623+0000 7fd5d172a640 1 -- 172.21.15.16:0/718361865 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd5b806c2e0 msgr2=0x7fd5b806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.623+0000 7fd5d172a640 1 --2- 172.21.15.16:0/718361865 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd5b806c2e0 0x7fd5b806e7a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0099c0 tx=0x7fd5bc008040 comp rx=0 tx=0).stop 2024-09-19T07:53:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.623+0000 7fd5d172a640 1 -- 172.21.15.16:0/718361865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5cc161010 msgr2=0x7fd5cc15d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.623+0000 7fd5d172a640 1 --2- 172.21.15.16:0/718361865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5cc161010 0x7fd5cc15d340 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc071930 tx=0x7fd5c4009f90 comp rx=0 tx=0).stop 2024-09-19T07:53:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.623+0000 7fd5d172a640 1 -- 172.21.15.16:0/718361865 shutdown_connections 2024-09-19T07:53:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.623+0000 7fd5d172a640 1 --2- 172.21.15.16:0/718361865 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd5b806c2e0 0x7fd5b806e7a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.623+0000 7fd5d172a640 1 --2- 172.21.15.16:0/718361865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5cc161010 0x7fd5cc15d340 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.623+0000 7fd5d172a640 1 --2- 172.21.15.16:0/718361865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5cc1f4500 0x7fd5cc160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.623+0000 7fd5d172a640 1 --2- 172.21.15.16:0/718361865 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5b4004800 0x7fd5cc1476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.623+0000 7fd5d172a640 1 -- 172.21.15.16:0/718361865 >> 172.21.15.16:0/718361865 conn(0x7fd5cc0b4040 msgr2=0x7fd5cc0b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.623+0000 7fd5d172a640 1 -- 172.21.15.16:0/718361865 shutdown_connections 2024-09-19T07:53:06.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.623+0000 7fd5d172a640 1 -- 172.21.15.16:0/718361865 wait complete. 2024-09-19T07:53:06.634 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-09-19T07:53:06.600556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":756212,"kb_used_data":11384,"kb_used_omap":12,"kb_used_meta":744627,"kb_avail":754218508,"statfs":{"total":773094113280,"available":772319752192,"internally_reserved":0,"allocated":11657216,"data_stored":5998388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":762498608},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.698634"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959791+0000","last_change":"2024-09-19T07:53:02.969667+0000","last_active":"2024-09-19T07:53:03.959791+0000","last_peered":"2024-09-19T07:53:03.959791+0000","last_clean":"2024-09-19T07:53:03.959791+0000","last_became_active":"2024-09-19T07:53:02.969269+0000","last_became_peered":"2024-09-19T07:53:02.969269+0000","last_unstale":"2024-09-19T07:53:03.959791+0000","last_undegraded":"2024-09-19T07:53:03.959791+0000","last_fullsized":"2024-09-19T07:53:03.959791+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:21.528651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089709+0000","last_change":"2024-09-19T07:53:02.960077+0000","last_active":"2024-09-19T07:53:04.089709+0000","last_peered":"2024-09-19T07:53:04.089709+0000","last_clean":"2024-09-19T07:53:04.089709+0000","last_became_active":"2024-09-19T07:53:02.959395+0000","last_became_peered":"2024-09-19T07:53:02.959395+0000","last_unstale":"2024-09-19T07:53:04.089709+0000","last_undegraded":"2024-09-19T07:53:04.089709+0000","last_fullsized":"2024-09-19T07:53:04.089709+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959657+0000","last_change":"2024-09-19T07:53:02.961895+0000","last_active":"2024-09-19T07:53:03.959657+0000","last_peered":"2024-09-19T07:53:03.959657+0000","last_clean":"2024-09-19T07:53:03.959657+0000","last_became_active":"2024-09-19T07:53:02.961616+0000","last_became_peered":"2024-09-19T07:53:02.961616+0000","last_unstale":"2024-09-19T07:53:03.959657+0000","last_undegraded":"2024-09-19T07:53:03.959657+0000","last_fullsized":"2024-09-19T07:53:03.959657+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:01.957812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.962372+0000","last_change":"2024-09-19T07:53:02.961725+0000","last_active":"2024-09-19T07:53:02.962372+0000","last_peered":"2024-09-19T07:53:02.962372+0000","last_clean":"2024-09-19T07:53:02.962372+0000","last_became_active":"2024-09-19T07:53:02.961334+0000","last_became_peered":"2024-09-19T07:53:02.961334+0000","last_unstale":"2024-09-19T07:53:02.962372+0000","last_undegraded":"2024-09-19T07:53:02.962372+0000","last_fullsized":"2024-09-19T07:53:02.962372+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:04.795110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.946689+0000","last_change":"2024-09-19T07:53:02.945749+0000","last_active":"2024-09-19T07:53:02.946689+0000","last_peered":"2024-09-19T07:53:02.946689+0000","last_clean":"2024-09-19T07:53:02.946689+0000","last_became_active":"2024-09-19T07:53:02.945466+0000","last_became_peered":"2024-09-19T07:53:02.945466+0000","last_unstale":"2024-09-19T07:53:02.946689+0000","last_undegraded":"2024-09-19T07:53:02.946689+0000","last_fullsized":"2024-09-19T07:53:02.946689+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:28.288866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088937+0000","last_change":"2024-09-19T07:53:02.960167+0000","last_active":"2024-09-19T07:53:04.088937+0000","last_peered":"2024-09-19T07:53:04.088937+0000","last_clean":"2024-09-19T07:53:04.088937+0000","last_became_active":"2024-09-19T07:53:02.959874+0000","last_became_peered":"2024-09-19T07:53:02.959874+0000","last_unstale":"2024-09-19T07:53:04.088937+0000","last_undegraded":"2024-09-19T07:53:04.088937+0000","last_fullsized":"2024-09-19T07:53:04.088937+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:17.832687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088822+0000","last_change":"2024-09-19T07:53:02.973657+0000","last_active":"2024-09-19T07:53:04.088822+0000","last_peered":"2024-09-19T07:53:04.088822+0000","last_clean":"2024-09-19T07:53:04.088822+0000","last_became_active":"2024-09-19T07:53:02.973421+0000","last_became_peered":"2024-09-19T07:53:02.973421+0000","last_unstale":"2024-09-19T07:53:04.088822+0000","last_undegraded":"2024-09-19T07:53:04.088822+0000","last_fullsized":"2024-09-19T07:53:04.088822+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:28.288866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088738+0000","last_change":"2024-09-19T07:53:02.959944+0000","last_active":"2024-09-19T07:53:04.088738+0000","last_peered":"2024-09-19T07:53:04.088738+0000","last_clean":"2024-09-19T07:53:04.088738+0000","last_became_active":"2024-09-19T07:53:02.959654+0000","last_became_peered":"2024-09-19T07:53:02.959654+0000","last_unstale":"2024-09-19T07:53:04.088738+0000","last_undegraded":"2024-09-19T07:53:04.088738+0000","last_fullsized":"2024-09-19T07:53:04.088738+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:48.854939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089588+0000","last_change":"2024-09-19T07:53:02.966890+0000","last_active":"2024-09-19T07:53:04.089588+0000","last_peered":"2024-09-19T07:53:04.089588+0000","last_clean":"2024-09-19T07:53:04.089588+0000","last_became_active":"2024-09-19T07:53:02.966573+0000","last_became_peered":"2024-09-19T07:53:02.966573+0000","last_unstale":"2024-09-19T07:53:04.089588+0000","last_undegraded":"2024-09-19T07:53:04.089588+0000","last_fullsized":"2024-09-19T07:53:04.089588+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:22.222446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.972902+0000","last_change":"2024-09-19T07:53:02.969544+0000","last_active":"2024-09-19T07:53:02.972902+0000","last_peered":"2024-09-19T07:53:02.972902+0000","last_clean":"2024-09-19T07:53:02.972902+0000","last_became_active":"2024-09-19T07:53:02.969117+0000","last_became_peered":"2024-09-19T07:53:02.969117+0000","last_unstale":"2024-09-19T07:53:02.972902+0000","last_undegraded":"2024-09-19T07:53:02.972902+0000","last_fullsized":"2024-09-19T07:53:02.972902+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:22.222446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952211+0000","last_change":"2024-09-19T07:53:02.960894+0000","last_active":"2024-09-19T07:53:03.952211+0000","last_peered":"2024-09-19T07:53:03.952211+0000","last_clean":"2024-09-19T07:53:03.952211+0000","last_became_active":"2024-09-19T07:53:02.960589+0000","last_became_peered":"2024-09-19T07:53:02.960589+0000","last_unstale":"2024-09-19T07:53:03.952211+0000","last_undegraded":"2024-09-19T07:53:03.952211+0000","last_fullsized":"2024-09-19T07:53:03.952211+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:28.417746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951606+0000","last_change":"2024-09-19T07:53:02.964794+0000","last_active":"2024-09-19T07:53:03.951606+0000","last_peered":"2024-09-19T07:53:03.951606+0000","last_clean":"2024-09-19T07:53:03.951606+0000","last_became_active":"2024-09-19T07:53:02.964326+0000","last_became_peered":"2024-09-19T07:53:02.964326+0000","last_unstale":"2024-09-19T07:53:03.951606+0000","last_undegraded":"2024-09-19T07:53:03.951606+0000","last_fullsized":"2024-09-19T07:53:03.951606+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:22.222446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.970212+0000","last_change":"2024-09-19T07:53:02.966603+0000","last_active":"2024-09-19T07:53:02.970212+0000","last_peered":"2024-09-19T07:53:02.970212+0000","last_clean":"2024-09-19T07:53:02.970212+0000","last_became_active":"2024-09-19T07:53:02.966330+0000","last_became_peered":"2024-09-19T07:53:02.966330+0000","last_unstale":"2024-09-19T07:53:02.970212+0000","last_undegraded":"2024-09-19T07:53:02.970212+0000","last_fullsized":"2024-09-19T07:53:02.970212+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:15.018246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089112+0000","last_change":"2024-09-19T07:53:02.962007+0000","last_active":"2024-09-19T07:53:04.089112+0000","last_peered":"2024-09-19T07:53:04.089112+0000","last_clean":"2024-09-19T07:53:04.089112+0000","last_became_active":"2024-09-19T07:53:02.961724+0000","last_became_peered":"2024-09-19T07:53:02.961724+0000","last_unstale":"2024-09-19T07:53:04.089112+0000","last_undegraded":"2024-09-19T07:53:04.089112+0000","last_fullsized":"2024-09-19T07:53:04.089112+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:39.353799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088923+0000","last_change":"2024-09-19T07:53:02.957971+0000","last_active":"2024-09-19T07:53:04.088923+0000","last_peered":"2024-09-19T07:53:04.088923+0000","last_clean":"2024-09-19T07:53:04.088923+0000","last_became_active":"2024-09-19T07:53:02.957708+0000","last_became_peered":"2024-09-19T07:53:02.957708+0000","last_unstale":"2024-09-19T07:53:04.088923+0000","last_undegraded":"2024-09-19T07:53:04.088923+0000","last_fullsized":"2024-09-19T07:53:04.088923+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:56.130236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088952+0000","last_change":"2024-09-19T07:53:02.954436+0000","last_active":"2024-09-19T07:53:04.088952+0000","last_peered":"2024-09-19T07:53:04.088952+0000","last_clean":"2024-09-19T07:53:04.088952+0000","last_became_active":"2024-09-19T07:53:02.954126+0000","last_became_peered":"2024-09-19T07:53:02.954126+0000","last_unstale":"2024-09-19T07:53:04.088952+0000","last_undegraded":"2024-09-19T07:53:04.088952+0000","last_fullsized":"2024-09-19T07:53:04.088952+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:21.471735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959826+0000","last_change":"2024-09-19T07:53:02.957284+0000","last_active":"2024-09-19T07:53:03.959826+0000","last_peered":"2024-09-19T07:53:03.959826+0000","last_clean":"2024-09-19T07:53:03.959826+0000","last_became_active":"2024-09-19T07:53:02.957008+0000","last_became_peered":"2024-09-19T07:53:02.957008+0000","last_unstale":"2024-09-19T07:53:03.959826+0000","last_undegraded":"2024-09-19T07:53:03.959826+0000","last_fullsized":"2024-09-19T07:53:03.959826+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:24.443678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089749+0000","last_change":"2024-09-19T07:53:02.959833+0000","last_active":"2024-09-19T07:53:04.089749+0000","last_peered":"2024-09-19T07:53:04.089749+0000","last_clean":"2024-09-19T07:53:04.089749+0000","last_became_active":"2024-09-19T07:53:02.959076+0000","last_became_peered":"2024-09-19T07:53:02.959076+0000","last_unstale":"2024-09-19T07:53:04.089749+0000","last_undegraded":"2024-09-19T07:53:04.089749+0000","last_fullsized":"2024-09-19T07:53:04.089749+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:31.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959718+0000","last_change":"2024-09-19T07:53:02.970410+0000","last_active":"2024-09-19T07:53:03.959718+0000","last_peered":"2024-09-19T07:53:03.959718+0000","last_clean":"2024-09-19T07:53:03.959718+0000","last_became_active":"2024-09-19T07:53:02.967129+0000","last_became_peered":"2024-09-19T07:53:02.967129+0000","last_unstale":"2024-09-19T07:53:03.959718+0000","last_undegraded":"2024-09-19T07:53:03.959718+0000","last_fullsized":"2024-09-19T07:53:03.959718+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:46.990754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951748+0000","last_change":"2024-09-19T07:53:02.953069+0000","last_active":"2024-09-19T07:53:03.951748+0000","last_peered":"2024-09-19T07:53:03.951748+0000","last_clean":"2024-09-19T07:53:03.951748+0000","last_became_active":"2024-09-19T07:53:02.952498+0000","last_became_peered":"2024-09-19T07:53:02.952498+0000","last_unstale":"2024-09-19T07:53:03.951748+0000","last_undegraded":"2024-09-19T07:53:03.951748+0000","last_fullsized":"2024-09-19T07:53:03.951748+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:31.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089071+0000","last_change":"2024-09-19T07:53:02.965042+0000","last_active":"2024-09-19T07:53:04.089071+0000","last_peered":"2024-09-19T07:53:04.089071+0000","last_clean":"2024-09-19T07:53:04.089071+0000","last_became_active":"2024-09-19T07:53:02.964720+0000","last_became_peered":"2024-09-19T07:53:02.964720+0000","last_unstale":"2024-09-19T07:53:04.089071+0000","last_undegraded":"2024-09-19T07:53:04.089071+0000","last_fullsized":"2024-09-19T07:53:04.089071+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:49.707375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.969961+0000","last_change":"2024-09-19T07:53:02.960655+0000","last_active":"2024-09-19T07:53:02.969961+0000","last_peered":"2024-09-19T07:53:02.969961+0000","last_clean":"2024-09-19T07:53:02.969961+0000","last_became_active":"2024-09-19T07:53:02.960373+0000","last_became_peered":"2024-09-19T07:53:02.960373+0000","last_unstale":"2024-09-19T07:53:02.969961+0000","last_undegraded":"2024-09-19T07:53:02.969961+0000","last_fullsized":"2024-09-19T07:53:02.969961+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:42.406919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959448+0000","last_change":"2024-09-19T07:53:02.958018+0000","last_active":"2024-09-19T07:53:03.959448+0000","last_peered":"2024-09-19T07:53:03.959448+0000","last_clean":"2024-09-19T07:53:03.959448+0000","last_became_active":"2024-09-19T07:53:02.957706+0000","last_became_peered":"2024-09-19T07:53:02.957706+0000","last_unstale":"2024-09-19T07:53:03.959448+0000","last_undegraded":"2024-09-19T07:53:03.959448+0000","last_fullsized":"2024-09-19T07:53:03.959448+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:08.021359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.954880+0000","last_change":"2024-09-19T07:53:02.951003+0000","last_active":"2024-09-19T07:53:02.954880+0000","last_peered":"2024-09-19T07:53:02.954880+0000","last_clean":"2024-09-19T07:53:02.954880+0000","last_became_active":"2024-09-19T07:53:02.950262+0000","last_became_peered":"2024-09-19T07:53:02.950262+0000","last_unstale":"2024-09-19T07:53:02.954880+0000","last_undegraded":"2024-09-19T07:53:02.954880+0000","last_fullsized":"2024-09-19T07:53:02.954880+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:17.832687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.953137+0000","last_change":"2024-09-19T07:53:02.955789+0000","last_active":"2024-09-19T07:53:03.953137+0000","last_peered":"2024-09-19T07:53:03.953137+0000","last_clean":"2024-09-19T07:53:03.953137+0000","last_became_active":"2024-09-19T07:53:02.955480+0000","last_became_peered":"2024-09-19T07:53:02.955480+0000","last_unstale":"2024-09-19T07:53:03.953137+0000","last_undegraded":"2024-09-19T07:53:03.953137+0000","last_fullsized":"2024-09-19T07:53:03.953137+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:00.688869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952726+0000","last_change":"2024-09-19T07:53:02.962786+0000","last_active":"2024-09-19T07:53:03.952726+0000","last_peered":"2024-09-19T07:53:03.952726+0000","last_clean":"2024-09-19T07:53:03.952726+0000","last_became_active":"2024-09-19T07:53:02.961758+0000","last_became_peered":"2024-09-19T07:53:02.961758+0000","last_unstale":"2024-09-19T07:53:03.952726+0000","last_undegraded":"2024-09-19T07:53:03.952726+0000","last_fullsized":"2024-09-19T07:53:03.952726+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:43.470589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952391+0000","last_change":"2024-09-19T07:53:02.955834+0000","last_active":"2024-09-19T07:53:03.952391+0000","last_peered":"2024-09-19T07:53:03.952391+0000","last_clean":"2024-09-19T07:53:03.952391+0000","last_became_active":"2024-09-19T07:53:02.955070+0000","last_became_peered":"2024-09-19T07:53:02.955070+0000","last_unstale":"2024-09-19T07:53:03.952391+0000","last_undegraded":"2024-09-19T07:53:03.952391+0000","last_fullsized":"2024-09-19T07:53:03.952391+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:24.696392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089356+0000","last_change":"2024-09-19T07:53:02.963635+0000","last_active":"2024-09-19T07:53:04.089356+0000","last_peered":"2024-09-19T07:53:04.089356+0000","last_clean":"2024-09-19T07:53:04.089356+0000","last_became_active":"2024-09-19T07:53:02.963299+0000","last_became_peered":"2024-09-19T07:53:02.963299+0000","last_unstale":"2024-09-19T07:53:04.089356+0000","last_undegraded":"2024-09-19T07:53:04.089356+0000","last_fullsized":"2024-09-19T07:53:04.089356+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:03.858829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089467+0000","last_change":"2024-09-19T07:53:02.953500+0000","last_active":"2024-09-19T07:53:04.089467+0000","last_peered":"2024-09-19T07:53:04.089467+0000","last_clean":"2024-09-19T07:53:04.089467+0000","last_became_active":"2024-09-19T07:53:02.953221+0000","last_became_peered":"2024-09-19T07:53:02.953221+0000","last_unstale":"2024-09-19T07:53:04.089467+0000","last_undegraded":"2024-09-19T07:53:04.089467+0000","last_fullsized":"2024-09-19T07:53:04.089467+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:33.997609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952126+0000","last_change":"2024-09-19T07:53:02.959004+0000","last_active":"2024-09-19T07:53:03.952126+0000","last_peered":"2024-09-19T07:53:03.952126+0000","last_clean":"2024-09-19T07:53:03.952126+0000","last_became_active":"2024-09-19T07:53:02.958656+0000","last_became_peered":"2024-09-19T07:53:02.958656+0000","last_unstale":"2024-09-19T07:53:03.952126+0000","last_undegraded":"2024-09-19T07:53:03.952126+0000","last_fullsized":"2024-09-19T07:53:03.952126+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:58.202929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089316+0000","last_change":"2024-09-19T07:53:02.968735+0000","last_active":"2024-09-19T07:53:04.089316+0000","last_peered":"2024-09-19T07:53:04.089316+0000","last_clean":"2024-09-19T07:53:04.089316+0000","last_became_active":"2024-09-19T07:53:02.968434+0000","last_became_peered":"2024-09-19T07:53:02.968434+0000","last_unstale":"2024-09-19T07:53:04.089316+0000","last_undegraded":"2024-09-19T07:53:04.089316+0000","last_fullsized":"2024-09-19T07:53:04.089316+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:57.435952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088787+0000","last_change":"2024-09-19T07:53:02.954559+0000","last_active":"2024-09-19T07:53:04.088787+0000","last_peered":"2024-09-19T07:53:04.088787+0000","last_clean":"2024-09-19T07:53:04.088787+0000","last_became_active":"2024-09-19T07:53:02.954302+0000","last_became_peered":"2024-09-19T07:53:02.954302+0000","last_unstale":"2024-09-19T07:53:04.088787+0000","last_undegraded":"2024-09-19T07:53:04.088787+0000","last_fullsized":"2024-09-19T07:53:04.088787+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:39.717541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959514+0000","last_change":"2024-09-19T07:53:02.964781+0000","last_active":"2024-09-19T07:53:03.959514+0000","last_peered":"2024-09-19T07:53:03.959514+0000","last_clean":"2024-09-19T07:53:03.959514+0000","last_became_active":"2024-09-19T07:53:02.964513+0000","last_became_peered":"2024-09-19T07:53:02.964513+0000","last_unstale":"2024-09-19T07:53:03.959514+0000","last_undegraded":"2024-09-19T07:53:03.959514+0000","last_fullsized":"2024-09-19T07:53:03.959514+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:26.718784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"105'4","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.971186+0000","last_change":"2024-09-19T07:53:01.917100+0000","last_active":"2024-09-19T07:53:02.971186+0000","last_peered":"2024-09-19T07:53:02.971186+0000","last_clean":"2024-09-19T07:53:02.971186+0000","last_became_active":"2024-09-19T07:53:01.916801+0000","last_became_peered":"2024-09-19T07:53:01.916801+0000","last_unstale":"2024-09-19T07:53:02.971186+0000","last_undegraded":"2024-09-19T07:53:02.971186+0000","last_fullsized":"2024-09-19T07:53:02.971186+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:25:44.686436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088766+0000","last_change":"2024-09-19T07:53:02.959134+0000","last_active":"2024-09-19T07:53:04.088766+0000","last_peered":"2024-09-19T07:53:04.088766+0000","last_clean":"2024-09-19T07:53:04.088766+0000","last_became_active":"2024-09-19T07:53:02.958328+0000","last_became_peered":"2024-09-19T07:53:02.958328+0000","last_unstale":"2024-09-19T07:53:04.088766+0000","last_undegraded":"2024-09-19T07:53:04.088766+0000","last_fullsized":"2024-09-19T07:53:04.088766+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:42.128496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089533+0000","last_change":"2024-09-19T07:53:01.927253+0000","last_active":"2024-09-19T07:53:04.089533+0000","last_peered":"2024-09-19T07:53:04.089533+0000","last_clean":"2024-09-19T07:53:04.089533+0000","last_became_active":"2024-09-19T07:53:01.926918+0000","last_became_peered":"2024-09-19T07:53:01.926918+0000","last_unstale":"2024-09-19T07:53:04.089533+0000","last_undegraded":"2024-09-19T07:53:04.089533+0000","last_fullsized":"2024-09-19T07:53:04.089533+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:55:57.845393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952282+0000","last_change":"2024-09-19T07:53:02.950143+0000","last_active":"2024-09-19T07:53:03.952282+0000","last_peered":"2024-09-19T07:53:03.952282+0000","last_clean":"2024-09-19T07:53:03.952282+0000","last_became_active":"2024-09-19T07:53:02.949855+0000","last_became_peered":"2024-09-19T07:53:02.949855+0000","last_unstale":"2024-09-19T07:53:03.952282+0000","last_undegraded":"2024-09-19T07:53:03.952282+0000","last_fullsized":"2024-09-19T07:53:03.952282+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:22.009944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951518+0000","last_change":"2024-09-19T07:53:01.926136+0000","last_active":"2024-09-19T07:53:03.951518+0000","last_peered":"2024-09-19T07:53:03.951518+0000","last_clean":"2024-09-19T07:53:03.951518+0000","last_became_active":"2024-09-19T07:53:01.924857+0000","last_became_peered":"2024-09-19T07:53:01.924857+0000","last_unstale":"2024-09-19T07:53:03.951518+0000","last_undegraded":"2024-09-19T07:53:03.951518+0000","last_fullsized":"2024-09-19T07:53:03.951518+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:26.340346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952916+0000","last_change":"2024-09-19T07:53:02.956369+0000","last_active":"2024-09-19T07:53:03.952916+0000","last_peered":"2024-09-19T07:53:03.952916+0000","last_clean":"2024-09-19T07:53:03.952916+0000","last_became_active":"2024-09-19T07:53:02.956011+0000","last_became_peered":"2024-09-19T07:53:02.956011+0000","last_unstale":"2024-09-19T07:53:03.952916+0000","last_undegraded":"2024-09-19T07:53:03.952916+0000","last_fullsized":"2024-09-19T07:53:03.952916+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:21.471735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.969829+0000","last_change":"2024-09-19T07:53:01.919358+0000","last_active":"2024-09-19T07:53:02.969829+0000","last_peered":"2024-09-19T07:53:02.969829+0000","last_clean":"2024-09-19T07:53:02.969829+0000","last_became_active":"2024-09-19T07:53:01.914884+0000","last_became_peered":"2024-09-19T07:53:01.914884+0000","last_unstale":"2024-09-19T07:53:02.969829+0000","last_undegraded":"2024-09-19T07:53:02.969829+0000","last_fullsized":"2024-09-19T07:53:02.969829+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:26.340346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.971960+0000","last_change":"2024-09-19T07:53:02.971101+0000","last_active":"2024-09-19T07:53:02.971960+0000","last_peered":"2024-09-19T07:53:02.971960+0000","last_clean":"2024-09-19T07:53:02.971960+0000","last_became_active":"2024-09-19T07:53:02.970767+0000","last_became_peered":"2024-09-19T07:53:02.970767+0000","last_unstale":"2024-09-19T07:53:02.971960+0000","last_undegraded":"2024-09-19T07:53:02.971960+0000","last_fullsized":"2024-09-19T07:53:02.971960+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:07.231140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089007+0000","last_change":"2024-09-19T07:53:01.923137+0000","last_active":"2024-09-19T07:53:04.089007+0000","last_peered":"2024-09-19T07:53:04.089007+0000","last_clean":"2024-09-19T07:53:04.089007+0000","last_became_active":"2024-09-19T07:53:01.922852+0000","last_became_peered":"2024-09-19T07:53:01.922852+0000","last_unstale":"2024-09-19T07:53:04.089007+0000","last_undegraded":"2024-09-19T07:53:04.089007+0000","last_fullsized":"2024-09-19T07:53:04.089007+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:31.183313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959859+0000","last_change":"2024-09-19T07:53:02.967452+0000","last_active":"2024-09-19T07:53:03.959859+0000","last_peered":"2024-09-19T07:53:03.959859+0000","last_clean":"2024-09-19T07:53:03.959859+0000","last_became_active":"2024-09-19T07:53:02.966768+0000","last_became_peered":"2024-09-19T07:53:02.966768+0000","last_unstale":"2024-09-19T07:53:03.959859+0000","last_undegraded":"2024-09-19T07:53:03.959859+0000","last_fullsized":"2024-09-19T07:53:03.959859+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:04.735526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952599+0000","last_change":"2024-09-19T07:53:01.919808+0000","last_active":"2024-09-19T07:53:03.952599+0000","last_peered":"2024-09-19T07:53:03.952599+0000","last_clean":"2024-09-19T07:53:03.952599+0000","last_became_active":"2024-09-19T07:53:01.919477+0000","last_became_peered":"2024-09-19T07:53:01.919477+0000","last_unstale":"2024-09-19T07:53:03.952599+0000","last_undegraded":"2024-09-19T07:53:03.952599+0000","last_fullsized":"2024-09-19T07:53:03.952599+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:21.008103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089819+0000","last_change":"2024-09-19T07:53:02.960753+0000","last_active":"2024-09-19T07:53:04.089819+0000","last_peered":"2024-09-19T07:53:04.089819+0000","last_clean":"2024-09-19T07:53:04.089819+0000","last_became_active":"2024-09-19T07:53:02.960465+0000","last_became_peered":"2024-09-19T07:53:02.960465+0000","last_unstale":"2024-09-19T07:53:04.089819+0000","last_undegraded":"2024-09-19T07:53:04.089819+0000","last_fullsized":"2024-09-19T07:53:04.089819+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:58.202929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951787+0000","last_change":"2024-09-19T07:53:01.916796+0000","last_active":"2024-09-19T07:53:03.951787+0000","last_peered":"2024-09-19T07:53:03.951787+0000","last_clean":"2024-09-19T07:53:03.951787+0000","last_became_active":"2024-09-19T07:53:01.916506+0000","last_became_peered":"2024-09-19T07:53:01.916506+0000","last_unstale":"2024-09-19T07:53:03.951787+0000","last_undegraded":"2024-09-19T07:53:03.951787+0000","last_fullsized":"2024-09-19T07:53:03.951787+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:50.457075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.965750+0000","last_change":"2024-09-19T07:53:02.963768+0000","last_active":"2024-09-19T07:53:02.965750+0000","last_peered":"2024-09-19T07:53:02.965750+0000","last_clean":"2024-09-19T07:53:02.965750+0000","last_became_active":"2024-09-19T07:53:02.963316+0000","last_became_peered":"2024-09-19T07:53:02.963316+0000","last_unstale":"2024-09-19T07:53:02.965750+0000","last_undegraded":"2024-09-19T07:53:02.965750+0000","last_fullsized":"2024-09-19T07:53:02.965750+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:40.843150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.947396+0000","last_change":"2024-09-19T07:53:01.922316+0000","last_active":"2024-09-19T07:53:02.947396+0000","last_peered":"2024-09-19T07:53:02.947396+0000","last_clean":"2024-09-19T07:53:02.947396+0000","last_became_active":"2024-09-19T07:53:01.922051+0000","last_became_peered":"2024-09-19T07:53:01.922051+0000","last_unstale":"2024-09-19T07:53:02.947396+0000","last_undegraded":"2024-09-19T07:53:02.947396+0000","last_fullsized":"2024-09-19T07:53:02.947396+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:20.469894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959407+0000","last_change":"2024-09-19T07:53:02.967631+0000","last_active":"2024-09-19T07:53:03.959407+0000","last_peered":"2024-09-19T07:53:03.959407+0000","last_clean":"2024-09-19T07:53:03.959407+0000","last_became_active":"2024-09-19T07:53:02.966419+0000","last_became_peered":"2024-09-19T07:53:02.966419+0000","last_unstale":"2024-09-19T07:53:03.959407+0000","last_undegraded":"2024-09-19T07:53:03.959407+0000","last_fullsized":"2024-09-19T07:53:03.959407+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:00.201893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.953111+0000","last_change":"2024-09-19T07:53:01.917284+0000","last_active":"2024-09-19T07:53:03.953111+0000","last_peered":"2024-09-19T07:53:03.953111+0000","last_clean":"2024-09-19T07:53:03.953111+0000","last_became_active":"2024-09-19T07:53:01.917007+0000","last_became_peered":"2024-09-19T07:53:01.917007+0000","last_unstale":"2024-09-19T07:53:03.953111+0000","last_undegraded":"2024-09-19T07:53:03.953111+0000","last_fullsized":"2024-09-19T07:53:03.953111+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:56.053928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.957662+0000","last_change":"2024-09-19T07:53:02.951942+0000","last_active":"2024-09-19T07:53:02.957662+0000","last_peered":"2024-09-19T07:53:02.957662+0000","last_clean":"2024-09-19T07:53:02.957662+0000","last_became_active":"2024-09-19T07:53:02.950538+0000","last_became_peered":"2024-09-19T07:53:02.950538+0000","last_unstale":"2024-09-19T07:53:02.957662+0000","last_undegraded":"2024-09-19T07:53:02.957662+0000","last_fullsized":"2024-09-19T07:53:02.957662+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:50.313345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959653+0000","last_change":"2024-09-19T07:53:01.923243+0000","last_active":"2024-09-19T07:53:03.959653+0000","last_peered":"2024-09-19T07:53:03.959653+0000","last_clean":"2024-09-19T07:53:03.959653+0000","last_became_active":"2024-09-19T07:53:01.922939+0000","last_became_peered":"2024-09-19T07:53:01.922939+0000","last_unstale":"2024-09-19T07:53:03.959653+0000","last_undegraded":"2024-09-19T07:53:03.959653+0000","last_fullsized":"2024-09-19T07:53:03.959653+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:55.701057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088818+0000","last_change":"2024-09-19T07:53:02.960750+0000","last_active":"2024-09-19T07:53:04.088818+0000","last_peered":"2024-09-19T07:53:04.088818+0000","last_clean":"2024-09-19T07:53:04.088818+0000","last_became_active":"2024-09-19T07:53:02.960480+0000","last_became_peered":"2024-09-19T07:53:02.960480+0000","last_unstale":"2024-09-19T07:53:04.088818+0000","last_undegraded":"2024-09-19T07:53:04.088818+0000","last_fullsized":"2024-09-19T07:53:04.088818+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:39.185837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"105'3","reported_seq":17,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089046+0000","last_change":"2024-09-19T07:53:01.923641+0000","last_active":"2024-09-19T07:53:04.089046+0000","last_peered":"2024-09-19T07:53:04.089046+0000","last_clean":"2024-09-19T07:53:04.089046+0000","last_became_active":"2024-09-19T07:53:01.923379+0000","last_became_peered":"2024-09-19T07:53:01.923379+0000","last_unstale":"2024-09-19T07:53:04.089046+0000","last_undegraded":"2024-09-19T07:53:04.089046+0000","last_fullsized":"2024-09-19T07:53:04.089046+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:51:46.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.955166+0000","last_change":"2024-09-19T07:53:02.953561+0000","last_active":"2024-09-19T07:53:02.955166+0000","last_peered":"2024-09-19T07:53:02.955166+0000","last_clean":"2024-09-19T07:53:02.955166+0000","last_became_active":"2024-09-19T07:53:02.953304+0000","last_became_peered":"2024-09-19T07:53:02.953304+0000","last_unstale":"2024-09-19T07:53:02.955166+0000","last_undegraded":"2024-09-19T07:53:02.955166+0000","last_fullsized":"2024-09-19T07:53:02.955166+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:03.858829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"105'6","reported_seq":20,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089486+0000","last_change":"2024-09-19T07:53:01.924833+0000","last_active":"2024-09-19T07:53:04.089486+0000","last_peered":"2024-09-19T07:53:04.089486+0000","last_clean":"2024-09-19T07:53:04.089486+0000","last_became_active":"2024-09-19T07:53:01.924568+0000","last_became_peered":"2024-09-19T07:53:01.924568+0000","last_unstale":"2024-09-19T07:53:04.089486+0000","last_undegraded":"2024-09-19T07:53:04.089486+0000","last_fullsized":"2024-09-19T07:53:04.089486+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:49.033408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952811+0000","last_change":"2024-09-19T07:53:02.955505+0000","last_active":"2024-09-19T07:53:03.952811+0000","last_peered":"2024-09-19T07:53:03.952811+0000","last_clean":"2024-09-19T07:53:03.952811+0000","last_became_active":"2024-09-19T07:53:02.955208+0000","last_became_peered":"2024-09-19T07:53:02.955208+0000","last_unstale":"2024-09-19T07:53:03.952811+0000","last_undegraded":"2024-09-19T07:53:03.952811+0000","last_fullsized":"2024-09-19T07:53:03.952811+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:56.130236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.969986+0000","last_change":"2024-09-19T07:53:01.920568+0000","last_active":"2024-09-19T07:53:02.969986+0000","last_peered":"2024-09-19T07:53:02.969986+0000","last_clean":"2024-09-19T07:53:02.969986+0000","last_became_active":"2024-09-19T07:53:01.920282+0000","last_became_peered":"2024-09-19T07:53:01.920282+0000","last_unstale":"2024-09-19T07:53:02.969986+0000","last_undegraded":"2024-09-19T07:53:02.969986+0000","last_fullsized":"2024-09-19T07:53:02.969986+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:11.122190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089492+0000","last_change":"2024-09-19T07:53:02.963599+0000","last_active":"2024-09-19T07:53:04.089492+0000","last_peered":"2024-09-19T07:53:04.089492+0000","last_clean":"2024-09-19T07:53:04.089492+0000","last_became_active":"2024-09-19T07:53:02.963303+0000","last_became_peered":"2024-09-19T07:53:02.963303+0000","last_unstale":"2024-09-19T07:53:04.089492+0000","last_undegraded":"2024-09-19T07:53:04.089492+0000","last_fullsized":"2024-09-19T07:53:04.089492+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:42.406919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.950643+0000","last_change":"2024-09-19T07:53:01.919711+0000","last_active":"2024-09-19T07:53:02.950643+0000","last_peered":"2024-09-19T07:53:02.950643+0000","last_clean":"2024-09-19T07:53:02.950643+0000","last_became_active":"2024-09-19T07:53:01.919443+0000","last_became_peered":"2024-09-19T07:53:01.919443+0000","last_unstale":"2024-09-19T07:53:02.950643+0000","last_undegraded":"2024-09-19T07:53:02.950643+0000","last_fullsized":"2024-09-19T07:53:02.950643+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:38.351958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089167+0000","last_change":"2024-09-19T07:53:02.955371+0000","last_active":"2024-09-19T07:53:04.089167+0000","last_peered":"2024-09-19T07:53:04.089167+0000","last_clean":"2024-09-19T07:53:04.089167+0000","last_became_active":"2024-09-19T07:53:02.955075+0000","last_became_peered":"2024-09-19T07:53:02.955075+0000","last_unstale":"2024-09-19T07:53:04.089167+0000","last_undegraded":"2024-09-19T07:53:04.089167+0000","last_fullsized":"2024-09-19T07:53:04.089167+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:37:25.460499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088708+0000","last_change":"2024-09-19T07:53:01.918535+0000","last_active":"2024-09-19T07:53:04.088708+0000","last_peered":"2024-09-19T07:53:04.088708+0000","last_clean":"2024-09-19T07:53:04.088708+0000","last_became_active":"2024-09-19T07:53:01.918258+0000","last_became_peered":"2024-09-19T07:53:01.918258+0000","last_unstale":"2024-09-19T07:53:04.088708+0000","last_undegraded":"2024-09-19T07:53:04.088708+0000","last_fullsized":"2024-09-19T07:53:04.088708+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:33.989659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959932+0000","last_change":"2024-09-19T07:53:02.962652+0000","last_active":"2024-09-19T07:53:03.959932+0000","last_peered":"2024-09-19T07:53:03.959932+0000","last_clean":"2024-09-19T07:53:03.959932+0000","last_became_active":"2024-09-19T07:53:02.962394+0000","last_became_peered":"2024-09-19T07:53:02.962394+0000","last_unstale":"2024-09-19T07:53:03.959932+0000","last_undegraded":"2024-09-19T07:53:03.959932+0000","last_fullsized":"2024-09-19T07:53:03.959932+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:46.985872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"105'8","reported_seq":22,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951439+0000","last_change":"2024-09-19T07:53:01.918580+0000","last_active":"2024-09-19T07:53:03.951439+0000","last_peered":"2024-09-19T07:53:03.951439+0000","last_clean":"2024-09-19T07:53:03.951439+0000","last_became_active":"2024-09-19T07:53:01.918222+0000","last_became_peered":"2024-09-19T07:53:01.918222+0000","last_unstale":"2024-09-19T07:53:03.951439+0000","last_undegraded":"2024-09-19T07:53:03.951439+0000","last_fullsized":"2024-09-19T07:53:03.951439+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:06.785416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088884+0000","last_change":"2024-09-19T07:53:02.952071+0000","last_active":"2024-09-19T07:53:04.088884+0000","last_peered":"2024-09-19T07:53:04.088884+0000","last_clean":"2024-09-19T07:53:04.088884+0000","last_became_active":"2024-09-19T07:53:02.951783+0000","last_became_peered":"2024-09-19T07:53:02.951783+0000","last_unstale":"2024-09-19T07:53:04.088884+0000","last_undegraded":"2024-09-19T07:53:04.088884+0000","last_fullsized":"2024-09-19T07:53:04.088884+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:02.793228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089618+0000","last_change":"2024-09-19T07:53:01.922470+0000","last_active":"2024-09-19T07:53:04.089618+0000","last_peered":"2024-09-19T07:53:04.089618+0000","last_clean":"2024-09-19T07:53:04.089618+0000","last_became_active":"2024-09-19T07:53:01.922181+0000","last_became_peered":"2024-09-19T07:53:01.922181+0000","last_unstale":"2024-09-19T07:53:04.089618+0000","last_undegraded":"2024-09-19T07:53:04.089618+0000","last_fullsized":"2024-09-19T07:53:04.089618+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:47.737086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089090+0000","last_change":"2024-09-19T07:53:02.965849+0000","last_active":"2024-09-19T07:53:04.089090+0000","last_peered":"2024-09-19T07:53:04.089090+0000","last_clean":"2024-09-19T07:53:04.089090+0000","last_became_active":"2024-09-19T07:53:02.965562+0000","last_became_peered":"2024-09-19T07:53:02.965562+0000","last_unstale":"2024-09-19T07:53:04.089090+0000","last_undegraded":"2024-09-19T07:53:04.089090+0000","last_fullsized":"2024-09-19T07:53:04.089090+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:50.313345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089146+0000","last_change":"2024-09-19T07:53:01.922247+0000","last_active":"2024-09-19T07:53:04.089146+0000","last_peered":"2024-09-19T07:53:04.089146+0000","last_clean":"2024-09-19T07:53:04.089146+0000","last_became_active":"2024-09-19T07:53:01.921944+0000","last_became_peered":"2024-09-19T07:53:01.921944+0000","last_unstale":"2024-09-19T07:53:04.089146+0000","last_undegraded":"2024-09-19T07:53:04.089146+0000","last_fullsized":"2024-09-19T07:53:04.089146+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:40.569155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.955145+0000","last_change":"2024-09-19T07:53:02.952785+0000","last_active":"2024-09-19T07:53:02.955145+0000","last_peered":"2024-09-19T07:53:02.955145+0000","last_clean":"2024-09-19T07:53:02.955145+0000","last_became_active":"2024-09-19T07:53:02.952535+0000","last_became_peered":"2024-09-19T07:53:02.952535+0000","last_unstale":"2024-09-19T07:53:02.955145+0000","last_undegraded":"2024-09-19T07:53:02.955145+0000","last_fullsized":"2024-09-19T07:53:02.955145+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:57.435952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952771+0000","last_change":"2024-09-19T07:53:01.925865+0000","last_active":"2024-09-19T07:53:03.952771+0000","last_peered":"2024-09-19T07:53:03.952771+0000","last_clean":"2024-09-19T07:53:03.952771+0000","last_became_active":"2024-09-19T07:53:01.925544+0000","last_became_peered":"2024-09-19T07:53:01.925544+0000","last_unstale":"2024-09-19T07:53:03.952771+0000","last_undegraded":"2024-09-19T07:53:03.952771+0000","last_fullsized":"2024-09-19T07:53:03.952771+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:57.201088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959514+0000","last_change":"2024-09-19T07:53:02.970190+0000","last_active":"2024-09-19T07:53:03.959514+0000","last_peered":"2024-09-19T07:53:03.959514+0000","last_clean":"2024-09-19T07:53:03.959514+0000","last_became_active":"2024-09-19T07:53:02.969897+0000","last_became_peered":"2024-09-19T07:53:02.969897+0000","last_unstale":"2024-09-19T07:53:03.959514+0000","last_undegraded":"2024-09-19T07:53:03.959514+0000","last_fullsized":"2024-09-19T07:53:03.959514+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:24.195258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089522+0000","last_change":"2024-09-19T07:53:01.926600+0000","last_active":"2024-09-19T07:53:04.089522+0000","last_peered":"2024-09-19T07:53:04.089522+0000","last_clean":"2024-09-19T07:53:04.089522+0000","last_became_active":"2024-09-19T07:53:01.925587+0000","last_became_peered":"2024-09-19T07:53:01.925587+0000","last_unstale":"2024-09-19T07:53:04.089522+0000","last_undegraded":"2024-09-19T07:53:04.089522+0000","last_fullsized":"2024-09-19T07:53:04.089522+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:13:51.412984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959770+0000","last_change":"2024-09-19T07:53:02.946678+0000","last_active":"2024-09-19T07:53:03.959770+0000","last_peered":"2024-09-19T07:53:03.959770+0000","last_clean":"2024-09-19T07:53:03.959770+0000","last_became_active":"2024-09-19T07:53:02.946357+0000","last_became_peered":"2024-09-19T07:53:02.946357+0000","last_unstale":"2024-09-19T07:53:03.959770+0000","last_undegraded":"2024-09-19T07:53:03.959770+0000","last_fullsized":"2024-09-19T07:53:03.959770+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:20.948125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088503+0000","last_change":"2024-09-19T07:53:01.926198+0000","last_active":"2024-09-19T07:53:04.088503+0000","last_peered":"2024-09-19T07:53:04.088503+0000","last_clean":"2024-09-19T07:53:04.088503+0000","last_became_active":"2024-09-19T07:53:01.925873+0000","last_became_peered":"2024-09-19T07:53:01.925873+0000","last_unstale":"2024-09-19T07:53:04.088503+0000","last_undegraded":"2024-09-19T07:53:04.088503+0000","last_fullsized":"2024-09-19T07:53:04.088503+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:23:00.870329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.968168+0000","last_change":"2024-09-19T07:53:02.965053+0000","last_active":"2024-09-19T07:53:02.968168+0000","last_peered":"2024-09-19T07:53:02.968168+0000","last_clean":"2024-09-19T07:53:02.968168+0000","last_became_active":"2024-09-19T07:53:02.964769+0000","last_became_peered":"2024-09-19T07:53:02.964769+0000","last_unstale":"2024-09-19T07:53:02.968168+0000","last_undegraded":"2024-09-19T07:53:02.968168+0000","last_fullsized":"2024-09-19T07:53:02.968168+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:40.771106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089316+0000","last_change":"2024-09-19T07:53:01.923054+0000","last_active":"2024-09-19T07:53:04.089316+0000","last_peered":"2024-09-19T07:53:04.089316+0000","last_clean":"2024-09-19T07:53:04.089316+0000","last_became_active":"2024-09-19T07:53:01.922799+0000","last_became_peered":"2024-09-19T07:53:01.922799+0000","last_unstale":"2024-09-19T07:53:04.089316+0000","last_undegraded":"2024-09-19T07:53:04.089316+0000","last_fullsized":"2024-09-19T07:53:04.089316+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:47.745785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089660+0000","last_change":"2024-09-19T07:53:02.957041+0000","last_active":"2024-09-19T07:53:04.089660+0000","last_peered":"2024-09-19T07:53:04.089660+0000","last_clean":"2024-09-19T07:53:04.089660+0000","last_became_active":"2024-09-19T07:53:02.956729+0000","last_became_peered":"2024-09-19T07:53:02.956729+0000","last_unstale":"2024-09-19T07:53:04.089660+0000","last_undegraded":"2024-09-19T07:53:04.089660+0000","last_fullsized":"2024-09-19T07:53:04.089660+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:24:06.262792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088870+0000","last_change":"2024-09-19T07:53:01.923097+0000","last_active":"2024-09-19T07:53:04.088870+0000","last_peered":"2024-09-19T07:53:04.088870+0000","last_clean":"2024-09-19T07:53:04.088870+0000","last_became_active":"2024-09-19T07:53:01.922794+0000","last_became_peered":"2024-09-19T07:53:01.922794+0000","last_unstale":"2024-09-19T07:53:04.088870+0000","last_undegraded":"2024-09-19T07:53:04.088870+0000","last_fullsized":"2024-09-19T07:53:04.088870+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:29.994651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959382+0000","last_change":"2024-09-19T07:53:02.968250+0000","last_active":"2024-09-19T07:53:03.959382+0000","last_peered":"2024-09-19T07:53:03.959382+0000","last_clean":"2024-09-19T07:53:03.959382+0000","last_became_active":"2024-09-19T07:53:02.967961+0000","last_became_peered":"2024-09-19T07:53:02.967961+0000","last_unstale":"2024-09-19T07:53:03.959382+0000","last_undegraded":"2024-09-19T07:53:03.959382+0000","last_fullsized":"2024-09-19T07:53:03.959382+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:03.803861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.969747+0000","last_change":"2024-09-19T07:53:01.922110+0000","last_active":"2024-09-19T07:53:02.969747+0000","last_peered":"2024-09-19T07:53:02.969747+0000","last_clean":"2024-09-19T07:53:02.969747+0000","last_became_active":"2024-09-19T07:53:01.921769+0000","last_became_peered":"2024-09-19T07:53:01.921769+0000","last_unstale":"2024-09-19T07:53:02.969747+0000","last_undegraded":"2024-09-19T07:53:02.969747+0000","last_fullsized":"2024-09-19T07:53:02.969747+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:56.053928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951681+0000","last_change":"2024-09-19T07:53:02.963616+0000","last_active":"2024-09-19T07:53:03.951681+0000","last_peered":"2024-09-19T07:53:03.951681+0000","last_clean":"2024-09-19T07:53:03.951681+0000","last_became_active":"2024-09-19T07:53:02.963333+0000","last_became_peered":"2024-09-19T07:53:02.963333+0000","last_unstale":"2024-09-19T07:53:03.951681+0000","last_undegraded":"2024-09-19T07:53:03.951681+0000","last_fullsized":"2024-09-19T07:53:03.951681+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"106'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.961104+0000","last_change":"2024-09-19T07:53:01.922601+0000","last_active":"2024-09-19T07:53:03.961104+0000","last_peered":"2024-09-19T07:53:03.961104+0000","last_clean":"2024-09-19T07:53:03.961104+0000","last_became_active":"2024-09-19T07:53:01.922324+0000","last_became_peered":"2024-09-19T07:53:01.922324+0000","last_unstale":"2024-09-19T07:53:03.961104+0000","last_undegraded":"2024-09-19T07:53:03.961104+0000","last_fullsized":"2024-09-19T07:53:03.961104+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:02.428894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.962345+0000","last_change":"2024-09-19T07:53:02.961218+0000","last_active":"2024-09-19T07:53:02.962345+0000","last_peered":"2024-09-19T07:53:02.962345+0000","last_clean":"2024-09-19T07:53:02.962345+0000","last_became_active":"2024-09-19T07:53:02.960873+0000","last_became_peered":"2024-09-19T07:53:02.960873+0000","last_unstale":"2024-09-19T07:53:02.962345+0000","last_undegraded":"2024-09-19T07:53:02.962345+0000","last_fullsized":"2024-09-19T07:53:02.962345+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:27.358096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089482+0000","last_change":"2024-09-19T07:53:01.920744+0000","last_active":"2024-09-19T07:53:04.089482+0000","last_peered":"2024-09-19T07:53:04.089482+0000","last_clean":"2024-09-19T07:53:04.089482+0000","last_became_active":"2024-09-19T07:53:01.920476+0000","last_became_peered":"2024-09-19T07:53:01.920476+0000","last_unstale":"2024-09-19T07:53:04.089482+0000","last_undegraded":"2024-09-19T07:53:04.089482+0000","last_fullsized":"2024-09-19T07:53:04.089482+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:11.662918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952379+0000","last_change":"2024-09-19T07:53:02.961727+0000","last_active":"2024-09-19T07:53:03.952379+0000","last_peered":"2024-09-19T07:53:03.952379+0000","last_clean":"2024-09-19T07:53:03.952379+0000","last_became_active":"2024-09-19T07:53:02.961447+0000","last_became_peered":"2024-09-19T07:53:02.961447+0000","last_unstale":"2024-09-19T07:53:03.952379+0000","last_undegraded":"2024-09-19T07:53:03.952379+0000","last_fullsized":"2024-09-19T07:53:03.952379+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:26:55.837328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.940797+0000","last_change":"2024-09-19T07:53:01.922583+0000","last_active":"2024-09-19T07:53:02.940797+0000","last_peered":"2024-09-19T07:53:02.940797+0000","last_clean":"2024-09-19T07:53:02.940797+0000","last_became_active":"2024-09-19T07:53:01.922318+0000","last_became_peered":"2024-09-19T07:53:01.922318+0000","last_unstale":"2024-09-19T07:53:02.940797+0000","last_undegraded":"2024-09-19T07:53:02.940797+0000","last_fullsized":"2024-09-19T07:53:02.940797+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:14.312065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089440+0000","last_change":"2024-09-19T07:53:02.957752+0000","last_active":"2024-09-19T07:53:04.089440+0000","last_peered":"2024-09-19T07:53:04.089440+0000","last_clean":"2024-09-19T07:53:04.089440+0000","last_became_active":"2024-09-19T07:53:02.957469+0000","last_became_peered":"2024-09-19T07:53:02.957469+0000","last_unstale":"2024-09-19T07:53:04.089440+0000","last_undegraded":"2024-09-19T07:53:04.089440+0000","last_fullsized":"2024-09-19T07:53:04.089440+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:18.785138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952032+0000","last_change":"2024-09-19T07:53:01.918087+0000","last_active":"2024-09-19T07:53:03.952032+0000","last_peered":"2024-09-19T07:53:03.952032+0000","last_clean":"2024-09-19T07:53:03.952032+0000","last_became_active":"2024-09-19T07:53:01.917738+0000","last_became_peered":"2024-09-19T07:53:01.917738+0000","last_unstale":"2024-09-19T07:53:03.952032+0000","last_undegraded":"2024-09-19T07:53:03.952032+0000","last_fullsized":"2024-09-19T07:53:03.952032+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:14.312065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089339+0000","last_change":"2024-09-19T07:53:02.961489+0000","last_active":"2024-09-19T07:53:04.089339+0000","last_peered":"2024-09-19T07:53:04.089339+0000","last_clean":"2024-09-19T07:53:04.089339+0000","last_became_active":"2024-09-19T07:53:02.961212+0000","last_became_peered":"2024-09-19T07:53:02.961212+0000","last_unstale":"2024-09-19T07:53:04.089339+0000","last_undegraded":"2024-09-19T07:53:04.089339+0000","last_fullsized":"2024-09-19T07:53:04.089339+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:15.778175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088861+0000","last_change":"2024-09-19T07:53:01.920116+0000","last_active":"2024-09-19T07:53:04.088861+0000","last_peered":"2024-09-19T07:53:04.088861+0000","last_clean":"2024-09-19T07:53:04.088861+0000","last_became_active":"2024-09-19T07:53:01.919834+0000","last_became_peered":"2024-09-19T07:53:01.919834+0000","last_unstale":"2024-09-19T07:53:04.088861+0000","last_undegraded":"2024-09-19T07:53:04.088861+0000","last_fullsized":"2024-09-19T07:53:04.088861+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:18.763151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089415+0000","last_change":"2024-09-19T07:53:02.957282+0000","last_active":"2024-09-19T07:53:04.089415+0000","last_peered":"2024-09-19T07:53:04.089415+0000","last_clean":"2024-09-19T07:53:04.089415+0000","last_became_active":"2024-09-19T07:53:02.957004+0000","last_became_peered":"2024-09-19T07:53:02.957004+0000","last_unstale":"2024-09-19T07:53:04.089415+0000","last_undegraded":"2024-09-19T07:53:04.089415+0000","last_fullsized":"2024-09-19T07:53:04.089415+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:25:45.688277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089434+0000","last_change":"2024-09-19T07:53:01.921607+0000","last_active":"2024-09-19T07:53:04.089434+0000","last_peered":"2024-09-19T07:53:04.089434+0000","last_clean":"2024-09-19T07:53:04.089434+0000","last_became_active":"2024-09-19T07:53:01.920937+0000","last_became_peered":"2024-09-19T07:53:01.920937+0000","last_unstale":"2024-09-19T07:53:04.089434+0000","last_undegraded":"2024-09-19T07:53:04.089434+0000","last_fullsized":"2024-09-19T07:53:04.089434+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:35.664386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.953006+0000","last_change":"2024-09-19T07:53:02.956981+0000","last_active":"2024-09-19T07:53:03.953006+0000","last_peered":"2024-09-19T07:53:03.953006+0000","last_clean":"2024-09-19T07:53:03.953006+0000","last_became_active":"2024-09-19T07:53:02.955937+0000","last_became_peered":"2024-09-19T07:53:02.955937+0000","last_unstale":"2024-09-19T07:53:03.953006+0000","last_undegraded":"2024-09-19T07:53:03.953006+0000","last_fullsized":"2024-09-19T07:53:03.953006+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:39.353799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088926+0000","last_change":"2024-09-19T07:53:01.924267+0000","last_active":"2024-09-19T07:53:04.088926+0000","last_peered":"2024-09-19T07:53:04.088926+0000","last_clean":"2024-09-19T07:53:04.088926+0000","last_became_active":"2024-09-19T07:53:01.924001+0000","last_became_peered":"2024-09-19T07:53:01.924001+0000","last_unstale":"2024-09-19T07:53:04.088926+0000","last_undegraded":"2024-09-19T07:53:04.088926+0000","last_fullsized":"2024-09-19T07:53:04.088926+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:15.069406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951881+0000","last_change":"2024-09-19T07:53:02.955750+0000","last_active":"2024-09-19T07:53:03.951881+0000","last_peered":"2024-09-19T07:53:03.951881+0000","last_clean":"2024-09-19T07:53:03.951881+0000","last_became_active":"2024-09-19T07:53:02.955250+0000","last_became_peered":"2024-09-19T07:53:02.955250+0000","last_unstale":"2024-09-19T07:53:03.951881+0000","last_undegraded":"2024-09-19T07:53:03.951881+0000","last_fullsized":"2024-09-19T07:53:03.951881+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:22.292252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089547+0000","last_change":"2024-09-19T07:53:01.921714+0000","last_active":"2024-09-19T07:53:04.089547+0000","last_peered":"2024-09-19T07:53:04.089547+0000","last_clean":"2024-09-19T07:53:04.089547+0000","last_became_active":"2024-09-19T07:53:01.921436+0000","last_became_peered":"2024-09-19T07:53:01.921436+0000","last_unstale":"2024-09-19T07:53:04.089547+0000","last_undegraded":"2024-09-19T07:53:04.089547+0000","last_fullsized":"2024-09-19T07:53:04.089547+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:04.601086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":325,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088582+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:53:04.088582+0000","last_peered":"2024-09-19T07:53:04.088582+0000","last_clean":"2024-09-19T07:53:04.088582+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:53:04.088582+0000","last_undegraded":"2024-09-19T07:53:04.088582+0000","last_fullsized":"2024-09-19T07:53:04.088582+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509565,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82960,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288880,"statfs":{"total":96636764160,"available":96551813120,"internally_reserved":0,"allocated":1896448,"data_stored":1188786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509564,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100520,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":94271320,"statfs":{"total":96636764160,"available":96533831680,"internally_reserved":0,"allocated":1327104,"data_stored":607166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":101579206},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509565,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50512,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321328,"statfs":{"total":96636764160,"available":96585039872,"internally_reserved":0,"allocated":1306624,"data_stored":604950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509565,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86552,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285288,"statfs":{"total":96636764160,"available":96548134912,"internally_reserved":0,"allocated":1314816,"data_stored":605042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509564,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":101020,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94270820,"statfs":{"total":96636764160,"available":96533319680,"internally_reserved":0,"allocated":1904640,"data_stored":1190328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542268,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135120,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236720,"statfs":{"total":96636764160,"available":96498401280,"internally_reserved":0,"allocated":1298432,"data_stored":598840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":137034183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542268,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94928,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276912,"statfs":{"total":96636764160,"available":96539557888,"internally_reserved":0,"allocated":1298432,"data_stored":598292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574974,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104600,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267240,"statfs":{"total":96636764160,"available":96529653760,"internally_reserved":0,"allocated":1310720,"data_stored":604984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:53:06.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T07:53:06.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.775+0000 7f7f2ef11640 1 Processor -- start 2024-09-19T07:53:06.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.775+0000 7f7f2ef11640 1 -- start start 2024-09-19T07:53:06.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.776+0000 7f7f2ef11640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f2815bee0 0x7f7f280d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.776+0000 7f7f2ef11640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7f280cf910 0x7f7f280d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.776+0000 7f7f2ef11640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f280cdee0 con 0x7f7f2815c2c0 2024-09-19T07:53:06.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.776+0000 7f7f2ef11640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f7f280ceaf0 con 0x7f7f280cf910 2024-09-19T07:53:06.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.776+0000 7f7f2ef11640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7f280ce1c0 con 0x7f7f2815bee0 2024-09-19T07:53:06.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.776+0000 7f7f2d4ae640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7f280cf910 0x7f7f280d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.776+0000 7f7f2d4ae640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7f280cf910 0x7f7f280d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60762/0 (socket says 172.21.15.16:60762) 2024-09-19T07:53:06.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.776+0000 7f7f2d4ae640 1 -- 172.21.15.16:0/1172723240 learned_addr learned my addr 172.21.15.16:0/1172723240 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:06.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.776+0000 7f7f2d4ae640 1 -- 172.21.15.16:0/1172723240 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f2815bee0 msgr2=0x7f7f280d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:06.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.776+0000 7f7f2ccad640 1 --2- 172.21.15.16:0/1172723240 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f2815bee0 0x7f7f280d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.776+0000 7f7f2d4ae640 1 --2- 172.21.15.16:0/1172723240 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f2815bee0 0x7f7f280d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.776+0000 7f7f2d4ae640 1 -- 172.21.15.16:0/1172723240 >> v1:172.21.15.146:6789/0 conn(0x7f7f2815c2c0 legacy=0x7f7f280cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:06.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.776+0000 7f7f2d4ae640 1 -- 172.21.15.16:0/1172723240 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f280ce4a0 con 0x7f7f280cf910 2024-09-19T07:53:06.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.776+0000 7f7f2ccad640 1 --2- 172.21.15.16:0/1172723240 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f2815bee0 0x7f7f280d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:06.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.777+0000 7f7f2d4ae640 1 --2- 172.21.15.16:0/1172723240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f280cf910 0x7f7f280d09f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009870 tx=0x7f7f2002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8149abbe1264c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:06.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.777+0000 7f7f277fe640 1 -- 172.21.15.16:0/1172723240 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20037070 con 0x7f7f280cf910 2024-09-19T07:53:06.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.777+0000 7f7f277fe640 1 -- 172.21.15.16:0/1172723240 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f20032440 con 0x7f7f280cf910 2024-09-19T07:53:06.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.777+0000 7f7f277fe640 1 -- 172.21.15.16:0/1172723240 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20032810 con 0x7f7f280cf910 2024-09-19T07:53:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.778+0000 7f7f2ef11640 1 -- 172.21.15.16:0/1172723240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f280cf910 msgr2=0x7f7f280d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.778+0000 7f7f2ef11640 1 --2- 172.21.15.16:0/1172723240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f280cf910 0x7f7f280d09f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009870 tx=0x7f7f2002a200 comp rx=0 tx=0).stop 2024-09-19T07:53:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.778+0000 7f7f2ef11640 1 -- 172.21.15.16:0/1172723240 shutdown_connections 2024-09-19T07:53:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.778+0000 7f7f2ef11640 1 --2- 172.21.15.16:0/1172723240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f280cf910 0x7f7f280d09f0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.778+0000 7f7f2ef11640 1 --2- 172.21.15.16:0/1172723240 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f2815bee0 0x7f7f280d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.778+0000 7f7f2ef11640 1 -- 172.21.15.16:0/1172723240 >> 172.21.15.16:0/1172723240 conn(0x7f7f2806bdf0 msgr2=0x7f7f2806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.778+0000 7f7f2ef11640 1 -- 172.21.15.16:0/1172723240 shutdown_connections 2024-09-19T07:53:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.778+0000 7f7f2ef11640 1 -- 172.21.15.16:0/1172723240 wait complete. 2024-09-19T07:53:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.779+0000 7f7f2ef11640 1 Processor -- start 2024-09-19T07:53:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.779+0000 7f7f2ef11640 1 -- start start 2024-09-19T07:53:06.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.780+0000 7f7f2ef11640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f280cf910 0x7f7f2815c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.780+0000 7f7f2ef11640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f2815bee0 0x7f7f28154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.780+0000 7f7f2ef11640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f2815c2c0 0x7f7f280b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.780+0000 7f7f2ef11640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2806e040 con 0x7f7f280cf910 2024-09-19T07:53:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.780+0000 7f7f2ef11640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2806dec0 con 0x7f7f2815c2c0 2024-09-19T07:53:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.780+0000 7f7f2ef11640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7f2806e1c0 con 0x7f7f2815bee0 2024-09-19T07:53:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.780+0000 7f7f2ccad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f280cf910 0x7f7f2815c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.780+0000 7f7f2ccad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f280cf910 0x7f7f2815c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60776/0 (socket says 172.21.15.16:60776) 2024-09-19T07:53:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.780+0000 7f7f2ccad640 1 -- 172.21.15.16:0/106837445 learned_addr learned my addr 172.21.15.16:0/106837445 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.780+0000 7f7f2ccad640 1 -- 172.21.15.16:0/106837445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f2815bee0 msgr2=0x7f7f28154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.780+0000 7f7f2d4ae640 1 --2- 172.21.15.16:0/106837445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f2815c2c0 0x7f7f280b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.780+0000 7f7f27fff640 1 --2- 172.21.15.16:0/106837445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f2815bee0 0x7f7f28154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.780+0000 7f7f2ccad640 1 --2- 172.21.15.16:0/106837445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f2815bee0 0x7f7f28154470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.780+0000 7f7f2ccad640 1 -- 172.21.15.16:0/106837445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f2815c2c0 msgr2=0x7f7f280b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.780+0000 7f7f2ccad640 1 --2- 172.21.15.16:0/106837445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f2815c2c0 0x7f7f280b0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.781+0000 7f7f2ccad640 1 -- 172.21.15.16:0/106837445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f280c85c0 con 0x7f7f280cf910 2024-09-19T07:53:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.781+0000 7f7f2ccad640 1 --2- 172.21.15.16:0/106837445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f280cf910 0x7f7f2815c7e0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800c7f0 tx=0x7f7f1800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.781+0000 7f7f257fa640 1 -- 172.21.15.16:0/106837445 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18015070 con 0x7f7f280cf910 2024-09-19T07:53:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.781+0000 7f7f257fa640 1 -- 172.21.15.16:0/106837445 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f18007a30 con 0x7f7f280cf910 2024-09-19T07:53:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.781+0000 7f7f2ef11640 1 -- 172.21.15.16:0/106837445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2815cee0 con 0x7f7f280cf910 2024-09-19T07:53:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.781+0000 7f7f257fa640 1 -- 172.21.15.16:0/106837445 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18007e00 con 0x7f7f280cf910 2024-09-19T07:53:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.781+0000 7f7f2ef11640 1 -- 172.21.15.16:0/106837445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f2815d4a0 con 0x7f7f280cf910 2024-09-19T07:53:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.782+0000 7f7f2ef11640 1 -- 172.21.15.16:0/106837445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f280cf200 con 0x7f7f280cf910 2024-09-19T07:53:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.782+0000 7f7f257fa640 1 -- 172.21.15.16:0/106837445 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7f18018070 con 0x7f7f280cf910 2024-09-19T07:53:06.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.783+0000 7f7f257fa640 1 --2- 172.21.15.16:0/106837445 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7f1406c2b0 0x7f7f1406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.783+0000 7f7f257fa640 1 -- 172.21.15.16:0/106837445 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f7f1808b650 con 0x7f7f280cf910 2024-09-19T07:53:06.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.783+0000 7f7f27fff640 1 --2- 172.21.15.16:0/106837445 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7f1406c2b0 0x7f7f1406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.784+0000 7f7f27fff640 1 --2- 172.21.15.16:0/106837445 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7f1406c2b0 0x7f7f1406e770 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c007fd0 tx=0x7f7f1c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:06.787+0000 7f7f257fa640 1 -- 172.21.15.16:0/106837445 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7f18059b90 con 0x7f7f280cf910 2024-09-19T07:53:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.033+0000 7f7f2ef11640 1 -- 172.21.15.16:0/106837445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f7f280c8790 con 0x7f7f280cf910 2024-09-19T07:53:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.033+0000 7f7f257fa640 1 -- 172.21.15.16:0/106837445 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1057 (secure 0 0 0) 0x7f7f18059490 con 0x7f7f280cf910 2024-09-19T07:53:07.034 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:07.034 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":624,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1726731773,"num_in_osds":8,"osd_in_since":1726731764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":774361088,"bytes_avail":772319752192,"bytes_total":773094113280,"write_bytes_sec":2515,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":56,"btime":"2024-09-19T07:53:04:953484+0000","id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":6993}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-19T07:51:22.573775+0000","services":{}},"progress_events":{}} 2024-09-19T07:53:07.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.035+0000 7f7f2ef11640 1 -- 172.21.15.16:0/106837445 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7f1406c2b0 msgr2=0x7f7f1406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:07.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.035+0000 7f7f2ef11640 1 --2- 172.21.15.16:0/106837445 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7f1406c2b0 0x7f7f1406e770 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c007fd0 tx=0x7f7f1c009040 comp rx=0 tx=0).stop 2024-09-19T07:53:07.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.035+0000 7f7f2ef11640 1 -- 172.21.15.16:0/106837445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f280cf910 msgr2=0x7f7f2815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:07.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.035+0000 7f7f2ef11640 1 --2- 172.21.15.16:0/106837445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f280cf910 0x7f7f2815c7e0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800c7f0 tx=0x7f7f1800cbf0 comp rx=0 tx=0).stop 2024-09-19T07:53:07.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.035+0000 7f7f2ef11640 1 -- 172.21.15.16:0/106837445 shutdown_connections 2024-09-19T07:53:07.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.035+0000 7f7f2ef11640 1 --2- 172.21.15.16:0/106837445 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7f1406c2b0 0x7f7f1406e770 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:07.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.035+0000 7f7f2ef11640 1 --2- 172.21.15.16:0/106837445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f2815c2c0 0x7f7f280b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:07.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.035+0000 7f7f2ef11640 1 --2- 172.21.15.16:0/106837445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f2815bee0 0x7f7f28154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:07.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.035+0000 7f7f2ef11640 1 --2- 172.21.15.16:0/106837445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f280cf910 0x7f7f2815c7e0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:07.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.035+0000 7f7f2ef11640 1 -- 172.21.15.16:0/106837445 >> 172.21.15.16:0/106837445 conn(0x7f7f2806bdf0 msgr2=0x7f7f280b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:07.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.035+0000 7f7f2ef11640 1 -- 172.21.15.16:0/106837445 shutdown_connections 2024-09-19T07:53:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.035+0000 7f7f2ef11640 1 -- 172.21.15.16:0/106837445 wait complete. 2024-09-19T07:53:07.046 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-19T07:53:07.046 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:53:07.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.148+0000 7f861b591640 1 Processor -- start 2024-09-19T07:53:07.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.148+0000 7f861b591640 1 -- start start 2024-09-19T07:53:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.149+0000 7f861b591640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f861415e2a0 0x7f861406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.149+0000 7f861b591640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f861406d960 0x7f86140d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.149+0000 7f861b591640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86140d1880 con 0x7f861415e680 2024-09-19T07:53:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.149+0000 7f861b591640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f86140d1b60 con 0x7f861415e2a0 2024-09-19T07:53:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.149+0000 7f861b591640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f86140cf2d0 con 0x7f861406d960 2024-09-19T07:53:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.149+0000 7f861932d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f861415e2a0 0x7f861406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.149+0000 7f861932d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f861415e2a0 0x7f861406d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60790/0 (socket says 172.21.15.16:60790) 2024-09-19T07:53:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.149+0000 7f861932d640 1 -- 172.21.15.16:0/1043673065 learned_addr learned my addr 172.21.15.16:0/1043673065 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.149+0000 7f8618b2c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f861415e680 0x7f86140cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60166/0 (socket says 172.21.15.16:60166) 2024-09-19T07:53:07.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.149+0000 7f8619b2e640 1 --2- 172.21.15.16:0/1043673065 >> v2:172.21.15.146:3300/0 conn(0x7f861406d960 0x7f86140d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.150+0000 7f861932d640 1 -- 172.21.15.16:0/1043673065 >> v2:172.21.15.146:3300/0 conn(0x7f861406d960 msgr2=0x7f86140d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.150+0000 7f861932d640 1 --2- 172.21.15.16:0/1043673065 >> v2:172.21.15.146:3300/0 conn(0x7f861406d960 0x7f86140d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.150+0000 7f861932d640 1 -- 172.21.15.16:0/1043673065 >> v1:172.21.15.16:6789/0 conn(0x7f861415e680 legacy=0x7f86140cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.150+0000 7f860bfff640 1 -- 172.21.15.16:0/1043673065 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2398723747 0 0) 0x7f86140d1880 con 0x7f861415e680 2024-09-19T07:53:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.150+0000 7f861932d640 1 -- 172.21.15.16:0/1043673065 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86140cf5b0 con 0x7f861415e2a0 2024-09-19T07:53:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.150+0000 7f8619b2e640 1 --2- 172.21.15.16:0/1043673065 >> v2:172.21.15.146:3300/0 conn(0x7f861406d960 0x7f86140d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:07.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.150+0000 7f861932d640 1 --2- 172.21.15.16:0/1043673065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861415e2a0 0x7f861406d420 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f8614163050 tx=0x7f861002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6de8bef5df31c90 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:07.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.150+0000 7f860bfff640 1 -- 172.21.15.16:0/1043673065 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610037070 con 0x7f861415e2a0 2024-09-19T07:53:07.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.150+0000 7f860bfff640 1 -- 172.21.15.16:0/1043673065 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8610032440 con 0x7f861415e2a0 2024-09-19T07:53:07.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.150+0000 7f860bfff640 1 -- 172.21.15.16:0/1043673065 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610032810 con 0x7f861415e2a0 2024-09-19T07:53:07.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.151+0000 7f861b591640 1 -- 172.21.15.16:0/1043673065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861415e2a0 msgr2=0x7f861406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:07.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.151+0000 7f861b591640 1 --2- 172.21.15.16:0/1043673065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861415e2a0 0x7f861406d420 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f8614163050 tx=0x7f861002a270 comp rx=0 tx=0).stop 2024-09-19T07:53:07.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.151+0000 7f861b591640 1 -- 172.21.15.16:0/1043673065 shutdown_connections 2024-09-19T07:53:07.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.151+0000 7f861b591640 1 --2- 172.21.15.16:0/1043673065 >> v2:172.21.15.146:3300/0 conn(0x7f861406d960 0x7f86140d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:07.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.151+0000 7f861b591640 1 --2- 172.21.15.16:0/1043673065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861415e2a0 0x7f861406d420 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:07.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.151+0000 7f861b591640 1 -- 172.21.15.16:0/1043673065 >> 172.21.15.16:0/1043673065 conn(0x7f861409d760 msgr2=0x7f861408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:07.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.151+0000 7f861b591640 1 -- 172.21.15.16:0/1043673065 shutdown_connections 2024-09-19T07:53:07.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.151+0000 7f861b591640 1 -- 172.21.15.16:0/1043673065 wait complete. 2024-09-19T07:53:07.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.152+0000 7f861b591640 1 Processor -- start 2024-09-19T07:53:07.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.152+0000 7f861b591640 1 -- start start 2024-09-19T07:53:07.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.153+0000 7f861b591640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861406d960 0x7f861415d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:07.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.153+0000 7f861b591640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f861415e2a0 0x7f86140a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:07.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.153+0000 7f861b591640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f861415e680 0x7f861409e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:07.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.153+0000 7f861b591640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86140d0360 con 0x7f861406d960 2024-09-19T07:53:07.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.153+0000 7f861b591640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f86140d01e0 con 0x7f861415e680 2024-09-19T07:53:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.153+0000 7f861b591640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f86140d48b0 con 0x7f861415e2a0 2024-09-19T07:53:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.153+0000 7f861932d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861406d960 0x7f861415d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.153+0000 7f861932d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861406d960 0x7f861415d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60806/0 (socket says 172.21.15.16:60806) 2024-09-19T07:53:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.153+0000 7f861932d640 1 -- 172.21.15.16:0/502391595 learned_addr learned my addr 172.21.15.16:0/502391595 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.153+0000 7f861932d640 1 -- 172.21.15.16:0/502391595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f861415e2a0 msgr2=0x7f86140a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.153+0000 7f8618b2c640 1 --2- 172.21.15.16:0/502391595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f861415e2a0 0x7f86140a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.153+0000 7f861932d640 1 --2- 172.21.15.16:0/502391595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f861415e2a0 0x7f86140a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.153+0000 7f8619b2e640 1 --2- 172.21.15.16:0/502391595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f861415e680 0x7f861409e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.153+0000 7f861932d640 1 -- 172.21.15.16:0/502391595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f861415e680 msgr2=0x7f861409e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.154+0000 7f861932d640 1 --2- 172.21.15.16:0/502391595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f861415e680 0x7f861409e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.154+0000 7f861932d640 1 -- 172.21.15.16:0/502391595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86140a0e90 con 0x7f861406d960 2024-09-19T07:53:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.154+0000 7f8618b2c640 1 --2- 172.21.15.16:0/502391595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f861415e2a0 0x7f86140a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.154+0000 7f861932d640 1 --2- 172.21.15.16:0/502391595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861406d960 0x7f861415d770 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f861002a2b0 tx=0x7f86100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.154+0000 7f8609ffb640 1 -- 172.21.15.16:0/502391595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610041020 con 0x7f861406d960 2024-09-19T07:53:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.154+0000 7f8609ffb640 1 -- 172.21.15.16:0/502391595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86100306c0 con 0x7f861406d960 2024-09-19T07:53:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.154+0000 7f861b591640 1 -- 172.21.15.16:0/502391595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86140a1170 con 0x7f861406d960 2024-09-19T07:53:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.154+0000 7f8609ffb640 1 -- 172.21.15.16:0/502391595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610037040 con 0x7f861406d960 2024-09-19T07:53:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.154+0000 7f861b591640 1 -- 172.21.15.16:0/502391595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86140a1730 con 0x7f861406d960 2024-09-19T07:53:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.155+0000 7f861b591640 1 -- 172.21.15.16:0/502391595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86140d4c70 con 0x7f861406d960 2024-09-19T07:53:07.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.156+0000 7f8609ffb640 1 -- 172.21.15.16:0/502391595 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8610044020 con 0x7f861406d960 2024-09-19T07:53:07.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.156+0000 7f8609ffb640 1 --2- 172.21.15.16:0/502391595 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f85f806c2b0 0x7f85f806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:07.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.156+0000 7f8609ffb640 1 -- 172.21.15.16:0/502391595 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f86100abd70 con 0x7f861406d960 2024-09-19T07:53:07.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.157+0000 7f8618b2c640 1 --2- 172.21.15.16:0/502391595 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f85f806c2b0 0x7f85f806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:07.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.157+0000 7f8618b2c640 1 --2- 172.21.15.16:0/502391595 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f85f806c2b0 0x7f85f806e770 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f86140ba2d0 tx=0x7f8604009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.161+0000 7f8609ffb640 1 -- 172.21.15.16:0/502391595 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f861004b030 con 0x7f861406d960 2024-09-19T07:53:07.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.331+0000 7f861b591640 1 -- 172.21.15.16:0/502391595 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8614160000 con 0x7f85f806c2b0 2024-09-19T07:53:07.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.338+0000 7f8609ffb640 1 -- 172.21.15.16:0/502391595 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252479 (secure 0 0 0) 0x7f8614160000 con 0x7f85f806c2b0 2024-09-19T07:53:07.341 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:07.342 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:53:07.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.340+0000 7f861b591640 1 -- 172.21.15.16:0/502391595 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f85f806c2b0 msgr2=0x7f85f806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.340+0000 7f861b591640 1 --2- 172.21.15.16:0/502391595 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f85f806c2b0 0x7f85f806e770 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f86140ba2d0 tx=0x7f8604009040 comp rx=0 tx=0).stop 2024-09-19T07:53:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.340+0000 7f861b591640 1 -- 172.21.15.16:0/502391595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861406d960 msgr2=0x7f861415d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.340+0000 7f861b591640 1 --2- 172.21.15.16:0/502391595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861406d960 0x7f861415d770 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f861002a2b0 tx=0x7f86100046c0 comp rx=0 tx=0).stop 2024-09-19T07:53:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.341+0000 7f861b591640 1 -- 172.21.15.16:0/502391595 shutdown_connections 2024-09-19T07:53:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.341+0000 7f861b591640 1 --2- 172.21.15.16:0/502391595 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f85f806c2b0 0x7f85f806e770 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.341+0000 7f861b591640 1 --2- 172.21.15.16:0/502391595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f861415e680 0x7f861409e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.341+0000 7f861b591640 1 --2- 172.21.15.16:0/502391595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f861415e2a0 0x7f86140a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.341+0000 7f861b591640 1 --2- 172.21.15.16:0/502391595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861406d960 0x7f861415d770 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.341+0000 7f861b591640 1 -- 172.21.15.16:0/502391595 >> 172.21.15.16:0/502391595 conn(0x7f861409d760 msgr2=0x7f86140d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.341+0000 7f861b591640 1 -- 172.21.15.16:0/502391595 shutdown_connections 2024-09-19T07:53:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:07.341+0000 7f861b591640 1 -- 172.21.15.16:0/502391595 wait complete. 2024-09-19T07:53:07.352 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-09-19T07:53:06.600556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":756212,"kb_used_data":11384,"kb_used_omap":12,"kb_used_meta":744627,"kb_avail":754218508,"statfs":{"total":773094113280,"available":772319752192,"internally_reserved":0,"allocated":11657216,"data_stored":5998388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":762498608},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.698634"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959791+0000","last_change":"2024-09-19T07:53:02.969667+0000","last_active":"2024-09-19T07:53:03.959791+0000","last_peered":"2024-09-19T07:53:03.959791+0000","last_clean":"2024-09-19T07:53:03.959791+0000","last_became_active":"2024-09-19T07:53:02.969269+0000","last_became_peered":"2024-09-19T07:53:02.969269+0000","last_unstale":"2024-09-19T07:53:03.959791+0000","last_undegraded":"2024-09-19T07:53:03.959791+0000","last_fullsized":"2024-09-19T07:53:03.959791+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:21.528651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089709+0000","last_change":"2024-09-19T07:53:02.960077+0000","last_active":"2024-09-19T07:53:04.089709+0000","last_peered":"2024-09-19T07:53:04.089709+0000","last_clean":"2024-09-19T07:53:04.089709+0000","last_became_active":"2024-09-19T07:53:02.959395+0000","last_became_peered":"2024-09-19T07:53:02.959395+0000","last_unstale":"2024-09-19T07:53:04.089709+0000","last_undegraded":"2024-09-19T07:53:04.089709+0000","last_fullsized":"2024-09-19T07:53:04.089709+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959657+0000","last_change":"2024-09-19T07:53:02.961895+0000","last_active":"2024-09-19T07:53:03.959657+0000","last_peered":"2024-09-19T07:53:03.959657+0000","last_clean":"2024-09-19T07:53:03.959657+0000","last_became_active":"2024-09-19T07:53:02.961616+0000","last_became_peered":"2024-09-19T07:53:02.961616+0000","last_unstale":"2024-09-19T07:53:03.959657+0000","last_undegraded":"2024-09-19T07:53:03.959657+0000","last_fullsized":"2024-09-19T07:53:03.959657+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:01.957812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.962372+0000","last_change":"2024-09-19T07:53:02.961725+0000","last_active":"2024-09-19T07:53:02.962372+0000","last_peered":"2024-09-19T07:53:02.962372+0000","last_clean":"2024-09-19T07:53:02.962372+0000","last_became_active":"2024-09-19T07:53:02.961334+0000","last_became_peered":"2024-09-19T07:53:02.961334+0000","last_unstale":"2024-09-19T07:53:02.962372+0000","last_undegraded":"2024-09-19T07:53:02.962372+0000","last_fullsized":"2024-09-19T07:53:02.962372+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:04.795110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.946689+0000","last_change":"2024-09-19T07:53:02.945749+0000","last_active":"2024-09-19T07:53:02.946689+0000","last_peered":"2024-09-19T07:53:02.946689+0000","last_clean":"2024-09-19T07:53:02.946689+0000","last_became_active":"2024-09-19T07:53:02.945466+0000","last_became_peered":"2024-09-19T07:53:02.945466+0000","last_unstale":"2024-09-19T07:53:02.946689+0000","last_undegraded":"2024-09-19T07:53:02.946689+0000","last_fullsized":"2024-09-19T07:53:02.946689+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:28.288866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088937+0000","last_change":"2024-09-19T07:53:02.960167+0000","last_active":"2024-09-19T07:53:04.088937+0000","last_peered":"2024-09-19T07:53:04.088937+0000","last_clean":"2024-09-19T07:53:04.088937+0000","last_became_active":"2024-09-19T07:53:02.959874+0000","last_became_peered":"2024-09-19T07:53:02.959874+0000","last_unstale":"2024-09-19T07:53:04.088937+0000","last_undegraded":"2024-09-19T07:53:04.088937+0000","last_fullsized":"2024-09-19T07:53:04.088937+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:17.832687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088822+0000","last_change":"2024-09-19T07:53:02.973657+0000","last_active":"2024-09-19T07:53:04.088822+0000","last_peered":"2024-09-19T07:53:04.088822+0000","last_clean":"2024-09-19T07:53:04.088822+0000","last_became_active":"2024-09-19T07:53:02.973421+0000","last_became_peered":"2024-09-19T07:53:02.973421+0000","last_unstale":"2024-09-19T07:53:04.088822+0000","last_undegraded":"2024-09-19T07:53:04.088822+0000","last_fullsized":"2024-09-19T07:53:04.088822+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:28.288866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088738+0000","last_change":"2024-09-19T07:53:02.959944+0000","last_active":"2024-09-19T07:53:04.088738+0000","last_peered":"2024-09-19T07:53:04.088738+0000","last_clean":"2024-09-19T07:53:04.088738+0000","last_became_active":"2024-09-19T07:53:02.959654+0000","last_became_peered":"2024-09-19T07:53:02.959654+0000","last_unstale":"2024-09-19T07:53:04.088738+0000","last_undegraded":"2024-09-19T07:53:04.088738+0000","last_fullsized":"2024-09-19T07:53:04.088738+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:48.854939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089588+0000","last_change":"2024-09-19T07:53:02.966890+0000","last_active":"2024-09-19T07:53:04.089588+0000","last_peered":"2024-09-19T07:53:04.089588+0000","last_clean":"2024-09-19T07:53:04.089588+0000","last_became_active":"2024-09-19T07:53:02.966573+0000","last_became_peered":"2024-09-19T07:53:02.966573+0000","last_unstale":"2024-09-19T07:53:04.089588+0000","last_undegraded":"2024-09-19T07:53:04.089588+0000","last_fullsized":"2024-09-19T07:53:04.089588+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:22.222446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.972902+0000","last_change":"2024-09-19T07:53:02.969544+0000","last_active":"2024-09-19T07:53:02.972902+0000","last_peered":"2024-09-19T07:53:02.972902+0000","last_clean":"2024-09-19T07:53:02.972902+0000","last_became_active":"2024-09-19T07:53:02.969117+0000","last_became_peered":"2024-09-19T07:53:02.969117+0000","last_unstale":"2024-09-19T07:53:02.972902+0000","last_undegraded":"2024-09-19T07:53:02.972902+0000","last_fullsized":"2024-09-19T07:53:02.972902+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:22.222446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952211+0000","last_change":"2024-09-19T07:53:02.960894+0000","last_active":"2024-09-19T07:53:03.952211+0000","last_peered":"2024-09-19T07:53:03.952211+0000","last_clean":"2024-09-19T07:53:03.952211+0000","last_became_active":"2024-09-19T07:53:02.960589+0000","last_became_peered":"2024-09-19T07:53:02.960589+0000","last_unstale":"2024-09-19T07:53:03.952211+0000","last_undegraded":"2024-09-19T07:53:03.952211+0000","last_fullsized":"2024-09-19T07:53:03.952211+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:28.417746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951606+0000","last_change":"2024-09-19T07:53:02.964794+0000","last_active":"2024-09-19T07:53:03.951606+0000","last_peered":"2024-09-19T07:53:03.951606+0000","last_clean":"2024-09-19T07:53:03.951606+0000","last_became_active":"2024-09-19T07:53:02.964326+0000","last_became_peered":"2024-09-19T07:53:02.964326+0000","last_unstale":"2024-09-19T07:53:03.951606+0000","last_undegraded":"2024-09-19T07:53:03.951606+0000","last_fullsized":"2024-09-19T07:53:03.951606+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:22.222446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.970212+0000","last_change":"2024-09-19T07:53:02.966603+0000","last_active":"2024-09-19T07:53:02.970212+0000","last_peered":"2024-09-19T07:53:02.970212+0000","last_clean":"2024-09-19T07:53:02.970212+0000","last_became_active":"2024-09-19T07:53:02.966330+0000","last_became_peered":"2024-09-19T07:53:02.966330+0000","last_unstale":"2024-09-19T07:53:02.970212+0000","last_undegraded":"2024-09-19T07:53:02.970212+0000","last_fullsized":"2024-09-19T07:53:02.970212+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:15.018246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089112+0000","last_change":"2024-09-19T07:53:02.962007+0000","last_active":"2024-09-19T07:53:04.089112+0000","last_peered":"2024-09-19T07:53:04.089112+0000","last_clean":"2024-09-19T07:53:04.089112+0000","last_became_active":"2024-09-19T07:53:02.961724+0000","last_became_peered":"2024-09-19T07:53:02.961724+0000","last_unstale":"2024-09-19T07:53:04.089112+0000","last_undegraded":"2024-09-19T07:53:04.089112+0000","last_fullsized":"2024-09-19T07:53:04.089112+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:39.353799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088923+0000","last_change":"2024-09-19T07:53:02.957971+0000","last_active":"2024-09-19T07:53:04.088923+0000","last_peered":"2024-09-19T07:53:04.088923+0000","last_clean":"2024-09-19T07:53:04.088923+0000","last_became_active":"2024-09-19T07:53:02.957708+0000","last_became_peered":"2024-09-19T07:53:02.957708+0000","last_unstale":"2024-09-19T07:53:04.088923+0000","last_undegraded":"2024-09-19T07:53:04.088923+0000","last_fullsized":"2024-09-19T07:53:04.088923+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:56.130236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088952+0000","last_change":"2024-09-19T07:53:02.954436+0000","last_active":"2024-09-19T07:53:04.088952+0000","last_peered":"2024-09-19T07:53:04.088952+0000","last_clean":"2024-09-19T07:53:04.088952+0000","last_became_active":"2024-09-19T07:53:02.954126+0000","last_became_peered":"2024-09-19T07:53:02.954126+0000","last_unstale":"2024-09-19T07:53:04.088952+0000","last_undegraded":"2024-09-19T07:53:04.088952+0000","last_fullsized":"2024-09-19T07:53:04.088952+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:21.471735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959826+0000","last_change":"2024-09-19T07:53:02.957284+0000","last_active":"2024-09-19T07:53:03.959826+0000","last_peered":"2024-09-19T07:53:03.959826+0000","last_clean":"2024-09-19T07:53:03.959826+0000","last_became_active":"2024-09-19T07:53:02.957008+0000","last_became_peered":"2024-09-19T07:53:02.957008+0000","last_unstale":"2024-09-19T07:53:03.959826+0000","last_undegraded":"2024-09-19T07:53:03.959826+0000","last_fullsized":"2024-09-19T07:53:03.959826+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:24.443678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089749+0000","last_change":"2024-09-19T07:53:02.959833+0000","last_active":"2024-09-19T07:53:04.089749+0000","last_peered":"2024-09-19T07:53:04.089749+0000","last_clean":"2024-09-19T07:53:04.089749+0000","last_became_active":"2024-09-19T07:53:02.959076+0000","last_became_peered":"2024-09-19T07:53:02.959076+0000","last_unstale":"2024-09-19T07:53:04.089749+0000","last_undegraded":"2024-09-19T07:53:04.089749+0000","last_fullsized":"2024-09-19T07:53:04.089749+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:31.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959718+0000","last_change":"2024-09-19T07:53:02.970410+0000","last_active":"2024-09-19T07:53:03.959718+0000","last_peered":"2024-09-19T07:53:03.959718+0000","last_clean":"2024-09-19T07:53:03.959718+0000","last_became_active":"2024-09-19T07:53:02.967129+0000","last_became_peered":"2024-09-19T07:53:02.967129+0000","last_unstale":"2024-09-19T07:53:03.959718+0000","last_undegraded":"2024-09-19T07:53:03.959718+0000","last_fullsized":"2024-09-19T07:53:03.959718+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:46.990754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951748+0000","last_change":"2024-09-19T07:53:02.953069+0000","last_active":"2024-09-19T07:53:03.951748+0000","last_peered":"2024-09-19T07:53:03.951748+0000","last_clean":"2024-09-19T07:53:03.951748+0000","last_became_active":"2024-09-19T07:53:02.952498+0000","last_became_peered":"2024-09-19T07:53:02.952498+0000","last_unstale":"2024-09-19T07:53:03.951748+0000","last_undegraded":"2024-09-19T07:53:03.951748+0000","last_fullsized":"2024-09-19T07:53:03.951748+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:31.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089071+0000","last_change":"2024-09-19T07:53:02.965042+0000","last_active":"2024-09-19T07:53:04.089071+0000","last_peered":"2024-09-19T07:53:04.089071+0000","last_clean":"2024-09-19T07:53:04.089071+0000","last_became_active":"2024-09-19T07:53:02.964720+0000","last_became_peered":"2024-09-19T07:53:02.964720+0000","last_unstale":"2024-09-19T07:53:04.089071+0000","last_undegraded":"2024-09-19T07:53:04.089071+0000","last_fullsized":"2024-09-19T07:53:04.089071+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:49.707375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.969961+0000","last_change":"2024-09-19T07:53:02.960655+0000","last_active":"2024-09-19T07:53:02.969961+0000","last_peered":"2024-09-19T07:53:02.969961+0000","last_clean":"2024-09-19T07:53:02.969961+0000","last_became_active":"2024-09-19T07:53:02.960373+0000","last_became_peered":"2024-09-19T07:53:02.960373+0000","last_unstale":"2024-09-19T07:53:02.969961+0000","last_undegraded":"2024-09-19T07:53:02.969961+0000","last_fullsized":"2024-09-19T07:53:02.969961+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:42.406919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959448+0000","last_change":"2024-09-19T07:53:02.958018+0000","last_active":"2024-09-19T07:53:03.959448+0000","last_peered":"2024-09-19T07:53:03.959448+0000","last_clean":"2024-09-19T07:53:03.959448+0000","last_became_active":"2024-09-19T07:53:02.957706+0000","last_became_peered":"2024-09-19T07:53:02.957706+0000","last_unstale":"2024-09-19T07:53:03.959448+0000","last_undegraded":"2024-09-19T07:53:03.959448+0000","last_fullsized":"2024-09-19T07:53:03.959448+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:08.021359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.954880+0000","last_change":"2024-09-19T07:53:02.951003+0000","last_active":"2024-09-19T07:53:02.954880+0000","last_peered":"2024-09-19T07:53:02.954880+0000","last_clean":"2024-09-19T07:53:02.954880+0000","last_became_active":"2024-09-19T07:53:02.950262+0000","last_became_peered":"2024-09-19T07:53:02.950262+0000","last_unstale":"2024-09-19T07:53:02.954880+0000","last_undegraded":"2024-09-19T07:53:02.954880+0000","last_fullsized":"2024-09-19T07:53:02.954880+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:17.832687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.953137+0000","last_change":"2024-09-19T07:53:02.955789+0000","last_active":"2024-09-19T07:53:03.953137+0000","last_peered":"2024-09-19T07:53:03.953137+0000","last_clean":"2024-09-19T07:53:03.953137+0000","last_became_active":"2024-09-19T07:53:02.955480+0000","last_became_peered":"2024-09-19T07:53:02.955480+0000","last_unstale":"2024-09-19T07:53:03.953137+0000","last_undegraded":"2024-09-19T07:53:03.953137+0000","last_fullsized":"2024-09-19T07:53:03.953137+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:00.688869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952726+0000","last_change":"2024-09-19T07:53:02.962786+0000","last_active":"2024-09-19T07:53:03.952726+0000","last_peered":"2024-09-19T07:53:03.952726+0000","last_clean":"2024-09-19T07:53:03.952726+0000","last_became_active":"2024-09-19T07:53:02.961758+0000","last_became_peered":"2024-09-19T07:53:02.961758+0000","last_unstale":"2024-09-19T07:53:03.952726+0000","last_undegraded":"2024-09-19T07:53:03.952726+0000","last_fullsized":"2024-09-19T07:53:03.952726+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:43.470589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952391+0000","last_change":"2024-09-19T07:53:02.955834+0000","last_active":"2024-09-19T07:53:03.952391+0000","last_peered":"2024-09-19T07:53:03.952391+0000","last_clean":"2024-09-19T07:53:03.952391+0000","last_became_active":"2024-09-19T07:53:02.955070+0000","last_became_peered":"2024-09-19T07:53:02.955070+0000","last_unstale":"2024-09-19T07:53:03.952391+0000","last_undegraded":"2024-09-19T07:53:03.952391+0000","last_fullsized":"2024-09-19T07:53:03.952391+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:24.696392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089356+0000","last_change":"2024-09-19T07:53:02.963635+0000","last_active":"2024-09-19T07:53:04.089356+0000","last_peered":"2024-09-19T07:53:04.089356+0000","last_clean":"2024-09-19T07:53:04.089356+0000","last_became_active":"2024-09-19T07:53:02.963299+0000","last_became_peered":"2024-09-19T07:53:02.963299+0000","last_unstale":"2024-09-19T07:53:04.089356+0000","last_undegraded":"2024-09-19T07:53:04.089356+0000","last_fullsized":"2024-09-19T07:53:04.089356+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:03.858829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089467+0000","last_change":"2024-09-19T07:53:02.953500+0000","last_active":"2024-09-19T07:53:04.089467+0000","last_peered":"2024-09-19T07:53:04.089467+0000","last_clean":"2024-09-19T07:53:04.089467+0000","last_became_active":"2024-09-19T07:53:02.953221+0000","last_became_peered":"2024-09-19T07:53:02.953221+0000","last_unstale":"2024-09-19T07:53:04.089467+0000","last_undegraded":"2024-09-19T07:53:04.089467+0000","last_fullsized":"2024-09-19T07:53:04.089467+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:33.997609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952126+0000","last_change":"2024-09-19T07:53:02.959004+0000","last_active":"2024-09-19T07:53:03.952126+0000","last_peered":"2024-09-19T07:53:03.952126+0000","last_clean":"2024-09-19T07:53:03.952126+0000","last_became_active":"2024-09-19T07:53:02.958656+0000","last_became_peered":"2024-09-19T07:53:02.958656+0000","last_unstale":"2024-09-19T07:53:03.952126+0000","last_undegraded":"2024-09-19T07:53:03.952126+0000","last_fullsized":"2024-09-19T07:53:03.952126+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:58.202929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089316+0000","last_change":"2024-09-19T07:53:02.968735+0000","last_active":"2024-09-19T07:53:04.089316+0000","last_peered":"2024-09-19T07:53:04.089316+0000","last_clean":"2024-09-19T07:53:04.089316+0000","last_became_active":"2024-09-19T07:53:02.968434+0000","last_became_peered":"2024-09-19T07:53:02.968434+0000","last_unstale":"2024-09-19T07:53:04.089316+0000","last_undegraded":"2024-09-19T07:53:04.089316+0000","last_fullsized":"2024-09-19T07:53:04.089316+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:57.435952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088787+0000","last_change":"2024-09-19T07:53:02.954559+0000","last_active":"2024-09-19T07:53:04.088787+0000","last_peered":"2024-09-19T07:53:04.088787+0000","last_clean":"2024-09-19T07:53:04.088787+0000","last_became_active":"2024-09-19T07:53:02.954302+0000","last_became_peered":"2024-09-19T07:53:02.954302+0000","last_unstale":"2024-09-19T07:53:04.088787+0000","last_undegraded":"2024-09-19T07:53:04.088787+0000","last_fullsized":"2024-09-19T07:53:04.088787+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:39.717541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959514+0000","last_change":"2024-09-19T07:53:02.964781+0000","last_active":"2024-09-19T07:53:03.959514+0000","last_peered":"2024-09-19T07:53:03.959514+0000","last_clean":"2024-09-19T07:53:03.959514+0000","last_became_active":"2024-09-19T07:53:02.964513+0000","last_became_peered":"2024-09-19T07:53:02.964513+0000","last_unstale":"2024-09-19T07:53:03.959514+0000","last_undegraded":"2024-09-19T07:53:03.959514+0000","last_fullsized":"2024-09-19T07:53:03.959514+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:26.718784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"105'4","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.971186+0000","last_change":"2024-09-19T07:53:01.917100+0000","last_active":"2024-09-19T07:53:02.971186+0000","last_peered":"2024-09-19T07:53:02.971186+0000","last_clean":"2024-09-19T07:53:02.971186+0000","last_became_active":"2024-09-19T07:53:01.916801+0000","last_became_peered":"2024-09-19T07:53:01.916801+0000","last_unstale":"2024-09-19T07:53:02.971186+0000","last_undegraded":"2024-09-19T07:53:02.971186+0000","last_fullsized":"2024-09-19T07:53:02.971186+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:25:44.686436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088766+0000","last_change":"2024-09-19T07:53:02.959134+0000","last_active":"2024-09-19T07:53:04.088766+0000","last_peered":"2024-09-19T07:53:04.088766+0000","last_clean":"2024-09-19T07:53:04.088766+0000","last_became_active":"2024-09-19T07:53:02.958328+0000","last_became_peered":"2024-09-19T07:53:02.958328+0000","last_unstale":"2024-09-19T07:53:04.088766+0000","last_undegraded":"2024-09-19T07:53:04.088766+0000","last_fullsized":"2024-09-19T07:53:04.088766+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:42.128496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089533+0000","last_change":"2024-09-19T07:53:01.927253+0000","last_active":"2024-09-19T07:53:04.089533+0000","last_peered":"2024-09-19T07:53:04.089533+0000","last_clean":"2024-09-19T07:53:04.089533+0000","last_became_active":"2024-09-19T07:53:01.926918+0000","last_became_peered":"2024-09-19T07:53:01.926918+0000","last_unstale":"2024-09-19T07:53:04.089533+0000","last_undegraded":"2024-09-19T07:53:04.089533+0000","last_fullsized":"2024-09-19T07:53:04.089533+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:55:57.845393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952282+0000","last_change":"2024-09-19T07:53:02.950143+0000","last_active":"2024-09-19T07:53:03.952282+0000","last_peered":"2024-09-19T07:53:03.952282+0000","last_clean":"2024-09-19T07:53:03.952282+0000","last_became_active":"2024-09-19T07:53:02.949855+0000","last_became_peered":"2024-09-19T07:53:02.949855+0000","last_unstale":"2024-09-19T07:53:03.952282+0000","last_undegraded":"2024-09-19T07:53:03.952282+0000","last_fullsized":"2024-09-19T07:53:03.952282+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:22.009944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951518+0000","last_change":"2024-09-19T07:53:01.926136+0000","last_active":"2024-09-19T07:53:03.951518+0000","last_peered":"2024-09-19T07:53:03.951518+0000","last_clean":"2024-09-19T07:53:03.951518+0000","last_became_active":"2024-09-19T07:53:01.924857+0000","last_became_peered":"2024-09-19T07:53:01.924857+0000","last_unstale":"2024-09-19T07:53:03.951518+0000","last_undegraded":"2024-09-19T07:53:03.951518+0000","last_fullsized":"2024-09-19T07:53:03.951518+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:26.340346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952916+0000","last_change":"2024-09-19T07:53:02.956369+0000","last_active":"2024-09-19T07:53:03.952916+0000","last_peered":"2024-09-19T07:53:03.952916+0000","last_clean":"2024-09-19T07:53:03.952916+0000","last_became_active":"2024-09-19T07:53:02.956011+0000","last_became_peered":"2024-09-19T07:53:02.956011+0000","last_unstale":"2024-09-19T07:53:03.952916+0000","last_undegraded":"2024-09-19T07:53:03.952916+0000","last_fullsized":"2024-09-19T07:53:03.952916+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:21.471735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.969829+0000","last_change":"2024-09-19T07:53:01.919358+0000","last_active":"2024-09-19T07:53:02.969829+0000","last_peered":"2024-09-19T07:53:02.969829+0000","last_clean":"2024-09-19T07:53:02.969829+0000","last_became_active":"2024-09-19T07:53:01.914884+0000","last_became_peered":"2024-09-19T07:53:01.914884+0000","last_unstale":"2024-09-19T07:53:02.969829+0000","last_undegraded":"2024-09-19T07:53:02.969829+0000","last_fullsized":"2024-09-19T07:53:02.969829+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:26.340346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.971960+0000","last_change":"2024-09-19T07:53:02.971101+0000","last_active":"2024-09-19T07:53:02.971960+0000","last_peered":"2024-09-19T07:53:02.971960+0000","last_clean":"2024-09-19T07:53:02.971960+0000","last_became_active":"2024-09-19T07:53:02.970767+0000","last_became_peered":"2024-09-19T07:53:02.970767+0000","last_unstale":"2024-09-19T07:53:02.971960+0000","last_undegraded":"2024-09-19T07:53:02.971960+0000","last_fullsized":"2024-09-19T07:53:02.971960+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:07.231140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089007+0000","last_change":"2024-09-19T07:53:01.923137+0000","last_active":"2024-09-19T07:53:04.089007+0000","last_peered":"2024-09-19T07:53:04.089007+0000","last_clean":"2024-09-19T07:53:04.089007+0000","last_became_active":"2024-09-19T07:53:01.922852+0000","last_became_peered":"2024-09-19T07:53:01.922852+0000","last_unstale":"2024-09-19T07:53:04.089007+0000","last_undegraded":"2024-09-19T07:53:04.089007+0000","last_fullsized":"2024-09-19T07:53:04.089007+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:31.183313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959859+0000","last_change":"2024-09-19T07:53:02.967452+0000","last_active":"2024-09-19T07:53:03.959859+0000","last_peered":"2024-09-19T07:53:03.959859+0000","last_clean":"2024-09-19T07:53:03.959859+0000","last_became_active":"2024-09-19T07:53:02.966768+0000","last_became_peered":"2024-09-19T07:53:02.966768+0000","last_unstale":"2024-09-19T07:53:03.959859+0000","last_undegraded":"2024-09-19T07:53:03.959859+0000","last_fullsized":"2024-09-19T07:53:03.959859+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:04.735526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952599+0000","last_change":"2024-09-19T07:53:01.919808+0000","last_active":"2024-09-19T07:53:03.952599+0000","last_peered":"2024-09-19T07:53:03.952599+0000","last_clean":"2024-09-19T07:53:03.952599+0000","last_became_active":"2024-09-19T07:53:01.919477+0000","last_became_peered":"2024-09-19T07:53:01.919477+0000","last_unstale":"2024-09-19T07:53:03.952599+0000","last_undegraded":"2024-09-19T07:53:03.952599+0000","last_fullsized":"2024-09-19T07:53:03.952599+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:21.008103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089819+0000","last_change":"2024-09-19T07:53:02.960753+0000","last_active":"2024-09-19T07:53:04.089819+0000","last_peered":"2024-09-19T07:53:04.089819+0000","last_clean":"2024-09-19T07:53:04.089819+0000","last_became_active":"2024-09-19T07:53:02.960465+0000","last_became_peered":"2024-09-19T07:53:02.960465+0000","last_unstale":"2024-09-19T07:53:04.089819+0000","last_undegraded":"2024-09-19T07:53:04.089819+0000","last_fullsized":"2024-09-19T07:53:04.089819+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:58.202929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951787+0000","last_change":"2024-09-19T07:53:01.916796+0000","last_active":"2024-09-19T07:53:03.951787+0000","last_peered":"2024-09-19T07:53:03.951787+0000","last_clean":"2024-09-19T07:53:03.951787+0000","last_became_active":"2024-09-19T07:53:01.916506+0000","last_became_peered":"2024-09-19T07:53:01.916506+0000","last_unstale":"2024-09-19T07:53:03.951787+0000","last_undegraded":"2024-09-19T07:53:03.951787+0000","last_fullsized":"2024-09-19T07:53:03.951787+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:50.457075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.965750+0000","last_change":"2024-09-19T07:53:02.963768+0000","last_active":"2024-09-19T07:53:02.965750+0000","last_peered":"2024-09-19T07:53:02.965750+0000","last_clean":"2024-09-19T07:53:02.965750+0000","last_became_active":"2024-09-19T07:53:02.963316+0000","last_became_peered":"2024-09-19T07:53:02.963316+0000","last_unstale":"2024-09-19T07:53:02.965750+0000","last_undegraded":"2024-09-19T07:53:02.965750+0000","last_fullsized":"2024-09-19T07:53:02.965750+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:40.843150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.947396+0000","last_change":"2024-09-19T07:53:01.922316+0000","last_active":"2024-09-19T07:53:02.947396+0000","last_peered":"2024-09-19T07:53:02.947396+0000","last_clean":"2024-09-19T07:53:02.947396+0000","last_became_active":"2024-09-19T07:53:01.922051+0000","last_became_peered":"2024-09-19T07:53:01.922051+0000","last_unstale":"2024-09-19T07:53:02.947396+0000","last_undegraded":"2024-09-19T07:53:02.947396+0000","last_fullsized":"2024-09-19T07:53:02.947396+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:20.469894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959407+0000","last_change":"2024-09-19T07:53:02.967631+0000","last_active":"2024-09-19T07:53:03.959407+0000","last_peered":"2024-09-19T07:53:03.959407+0000","last_clean":"2024-09-19T07:53:03.959407+0000","last_became_active":"2024-09-19T07:53:02.966419+0000","last_became_peered":"2024-09-19T07:53:02.966419+0000","last_unstale":"2024-09-19T07:53:03.959407+0000","last_undegraded":"2024-09-19T07:53:03.959407+0000","last_fullsized":"2024-09-19T07:53:03.959407+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:00.201893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.953111+0000","last_change":"2024-09-19T07:53:01.917284+0000","last_active":"2024-09-19T07:53:03.953111+0000","last_peered":"2024-09-19T07:53:03.953111+0000","last_clean":"2024-09-19T07:53:03.953111+0000","last_became_active":"2024-09-19T07:53:01.917007+0000","last_became_peered":"2024-09-19T07:53:01.917007+0000","last_unstale":"2024-09-19T07:53:03.953111+0000","last_undegraded":"2024-09-19T07:53:03.953111+0000","last_fullsized":"2024-09-19T07:53:03.953111+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:56.053928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.957662+0000","last_change":"2024-09-19T07:53:02.951942+0000","last_active":"2024-09-19T07:53:02.957662+0000","last_peered":"2024-09-19T07:53:02.957662+0000","last_clean":"2024-09-19T07:53:02.957662+0000","last_became_active":"2024-09-19T07:53:02.950538+0000","last_became_peered":"2024-09-19T07:53:02.950538+0000","last_unstale":"2024-09-19T07:53:02.957662+0000","last_undegraded":"2024-09-19T07:53:02.957662+0000","last_fullsized":"2024-09-19T07:53:02.957662+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:50.313345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959653+0000","last_change":"2024-09-19T07:53:01.923243+0000","last_active":"2024-09-19T07:53:03.959653+0000","last_peered":"2024-09-19T07:53:03.959653+0000","last_clean":"2024-09-19T07:53:03.959653+0000","last_became_active":"2024-09-19T07:53:01.922939+0000","last_became_peered":"2024-09-19T07:53:01.922939+0000","last_unstale":"2024-09-19T07:53:03.959653+0000","last_undegraded":"2024-09-19T07:53:03.959653+0000","last_fullsized":"2024-09-19T07:53:03.959653+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:55.701057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088818+0000","last_change":"2024-09-19T07:53:02.960750+0000","last_active":"2024-09-19T07:53:04.088818+0000","last_peered":"2024-09-19T07:53:04.088818+0000","last_clean":"2024-09-19T07:53:04.088818+0000","last_became_active":"2024-09-19T07:53:02.960480+0000","last_became_peered":"2024-09-19T07:53:02.960480+0000","last_unstale":"2024-09-19T07:53:04.088818+0000","last_undegraded":"2024-09-19T07:53:04.088818+0000","last_fullsized":"2024-09-19T07:53:04.088818+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:39.185837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"105'3","reported_seq":17,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089046+0000","last_change":"2024-09-19T07:53:01.923641+0000","last_active":"2024-09-19T07:53:04.089046+0000","last_peered":"2024-09-19T07:53:04.089046+0000","last_clean":"2024-09-19T07:53:04.089046+0000","last_became_active":"2024-09-19T07:53:01.923379+0000","last_became_peered":"2024-09-19T07:53:01.923379+0000","last_unstale":"2024-09-19T07:53:04.089046+0000","last_undegraded":"2024-09-19T07:53:04.089046+0000","last_fullsized":"2024-09-19T07:53:04.089046+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:51:46.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.955166+0000","last_change":"2024-09-19T07:53:02.953561+0000","last_active":"2024-09-19T07:53:02.955166+0000","last_peered":"2024-09-19T07:53:02.955166+0000","last_clean":"2024-09-19T07:53:02.955166+0000","last_became_active":"2024-09-19T07:53:02.953304+0000","last_became_peered":"2024-09-19T07:53:02.953304+0000","last_unstale":"2024-09-19T07:53:02.955166+0000","last_undegraded":"2024-09-19T07:53:02.955166+0000","last_fullsized":"2024-09-19T07:53:02.955166+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:03.858829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"105'6","reported_seq":20,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089486+0000","last_change":"2024-09-19T07:53:01.924833+0000","last_active":"2024-09-19T07:53:04.089486+0000","last_peered":"2024-09-19T07:53:04.089486+0000","last_clean":"2024-09-19T07:53:04.089486+0000","last_became_active":"2024-09-19T07:53:01.924568+0000","last_became_peered":"2024-09-19T07:53:01.924568+0000","last_unstale":"2024-09-19T07:53:04.089486+0000","last_undegraded":"2024-09-19T07:53:04.089486+0000","last_fullsized":"2024-09-19T07:53:04.089486+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:49.033408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952811+0000","last_change":"2024-09-19T07:53:02.955505+0000","last_active":"2024-09-19T07:53:03.952811+0000","last_peered":"2024-09-19T07:53:03.952811+0000","last_clean":"2024-09-19T07:53:03.952811+0000","last_became_active":"2024-09-19T07:53:02.955208+0000","last_became_peered":"2024-09-19T07:53:02.955208+0000","last_unstale":"2024-09-19T07:53:03.952811+0000","last_undegraded":"2024-09-19T07:53:03.952811+0000","last_fullsized":"2024-09-19T07:53:03.952811+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:56.130236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.969986+0000","last_change":"2024-09-19T07:53:01.920568+0000","last_active":"2024-09-19T07:53:02.969986+0000","last_peered":"2024-09-19T07:53:02.969986+0000","last_clean":"2024-09-19T07:53:02.969986+0000","last_became_active":"2024-09-19T07:53:01.920282+0000","last_became_peered":"2024-09-19T07:53:01.920282+0000","last_unstale":"2024-09-19T07:53:02.969986+0000","last_undegraded":"2024-09-19T07:53:02.969986+0000","last_fullsized":"2024-09-19T07:53:02.969986+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:11.122190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089492+0000","last_change":"2024-09-19T07:53:02.963599+0000","last_active":"2024-09-19T07:53:04.089492+0000","last_peered":"2024-09-19T07:53:04.089492+0000","last_clean":"2024-09-19T07:53:04.089492+0000","last_became_active":"2024-09-19T07:53:02.963303+0000","last_became_peered":"2024-09-19T07:53:02.963303+0000","last_unstale":"2024-09-19T07:53:04.089492+0000","last_undegraded":"2024-09-19T07:53:04.089492+0000","last_fullsized":"2024-09-19T07:53:04.089492+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:42.406919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.950643+0000","last_change":"2024-09-19T07:53:01.919711+0000","last_active":"2024-09-19T07:53:02.950643+0000","last_peered":"2024-09-19T07:53:02.950643+0000","last_clean":"2024-09-19T07:53:02.950643+0000","last_became_active":"2024-09-19T07:53:01.919443+0000","last_became_peered":"2024-09-19T07:53:01.919443+0000","last_unstale":"2024-09-19T07:53:02.950643+0000","last_undegraded":"2024-09-19T07:53:02.950643+0000","last_fullsized":"2024-09-19T07:53:02.950643+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:38.351958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089167+0000","last_change":"2024-09-19T07:53:02.955371+0000","last_active":"2024-09-19T07:53:04.089167+0000","last_peered":"2024-09-19T07:53:04.089167+0000","last_clean":"2024-09-19T07:53:04.089167+0000","last_became_active":"2024-09-19T07:53:02.955075+0000","last_became_peered":"2024-09-19T07:53:02.955075+0000","last_unstale":"2024-09-19T07:53:04.089167+0000","last_undegraded":"2024-09-19T07:53:04.089167+0000","last_fullsized":"2024-09-19T07:53:04.089167+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:37:25.460499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088708+0000","last_change":"2024-09-19T07:53:01.918535+0000","last_active":"2024-09-19T07:53:04.088708+0000","last_peered":"2024-09-19T07:53:04.088708+0000","last_clean":"2024-09-19T07:53:04.088708+0000","last_became_active":"2024-09-19T07:53:01.918258+0000","last_became_peered":"2024-09-19T07:53:01.918258+0000","last_unstale":"2024-09-19T07:53:04.088708+0000","last_undegraded":"2024-09-19T07:53:04.088708+0000","last_fullsized":"2024-09-19T07:53:04.088708+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:33.989659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959932+0000","last_change":"2024-09-19T07:53:02.962652+0000","last_active":"2024-09-19T07:53:03.959932+0000","last_peered":"2024-09-19T07:53:03.959932+0000","last_clean":"2024-09-19T07:53:03.959932+0000","last_became_active":"2024-09-19T07:53:02.962394+0000","last_became_peered":"2024-09-19T07:53:02.962394+0000","last_unstale":"2024-09-19T07:53:03.959932+0000","last_undegraded":"2024-09-19T07:53:03.959932+0000","last_fullsized":"2024-09-19T07:53:03.959932+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:46.985872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"105'8","reported_seq":22,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951439+0000","last_change":"2024-09-19T07:53:01.918580+0000","last_active":"2024-09-19T07:53:03.951439+0000","last_peered":"2024-09-19T07:53:03.951439+0000","last_clean":"2024-09-19T07:53:03.951439+0000","last_became_active":"2024-09-19T07:53:01.918222+0000","last_became_peered":"2024-09-19T07:53:01.918222+0000","last_unstale":"2024-09-19T07:53:03.951439+0000","last_undegraded":"2024-09-19T07:53:03.951439+0000","last_fullsized":"2024-09-19T07:53:03.951439+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:06.785416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088884+0000","last_change":"2024-09-19T07:53:02.952071+0000","last_active":"2024-09-19T07:53:04.088884+0000","last_peered":"2024-09-19T07:53:04.088884+0000","last_clean":"2024-09-19T07:53:04.088884+0000","last_became_active":"2024-09-19T07:53:02.951783+0000","last_became_peered":"2024-09-19T07:53:02.951783+0000","last_unstale":"2024-09-19T07:53:04.088884+0000","last_undegraded":"2024-09-19T07:53:04.088884+0000","last_fullsized":"2024-09-19T07:53:04.088884+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:02.793228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089618+0000","last_change":"2024-09-19T07:53:01.922470+0000","last_active":"2024-09-19T07:53:04.089618+0000","last_peered":"2024-09-19T07:53:04.089618+0000","last_clean":"2024-09-19T07:53:04.089618+0000","last_became_active":"2024-09-19T07:53:01.922181+0000","last_became_peered":"2024-09-19T07:53:01.922181+0000","last_unstale":"2024-09-19T07:53:04.089618+0000","last_undegraded":"2024-09-19T07:53:04.089618+0000","last_fullsized":"2024-09-19T07:53:04.089618+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:47.737086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089090+0000","last_change":"2024-09-19T07:53:02.965849+0000","last_active":"2024-09-19T07:53:04.089090+0000","last_peered":"2024-09-19T07:53:04.089090+0000","last_clean":"2024-09-19T07:53:04.089090+0000","last_became_active":"2024-09-19T07:53:02.965562+0000","last_became_peered":"2024-09-19T07:53:02.965562+0000","last_unstale":"2024-09-19T07:53:04.089090+0000","last_undegraded":"2024-09-19T07:53:04.089090+0000","last_fullsized":"2024-09-19T07:53:04.089090+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:50.313345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089146+0000","last_change":"2024-09-19T07:53:01.922247+0000","last_active":"2024-09-19T07:53:04.089146+0000","last_peered":"2024-09-19T07:53:04.089146+0000","last_clean":"2024-09-19T07:53:04.089146+0000","last_became_active":"2024-09-19T07:53:01.921944+0000","last_became_peered":"2024-09-19T07:53:01.921944+0000","last_unstale":"2024-09-19T07:53:04.089146+0000","last_undegraded":"2024-09-19T07:53:04.089146+0000","last_fullsized":"2024-09-19T07:53:04.089146+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:40.569155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.955145+0000","last_change":"2024-09-19T07:53:02.952785+0000","last_active":"2024-09-19T07:53:02.955145+0000","last_peered":"2024-09-19T07:53:02.955145+0000","last_clean":"2024-09-19T07:53:02.955145+0000","last_became_active":"2024-09-19T07:53:02.952535+0000","last_became_peered":"2024-09-19T07:53:02.952535+0000","last_unstale":"2024-09-19T07:53:02.955145+0000","last_undegraded":"2024-09-19T07:53:02.955145+0000","last_fullsized":"2024-09-19T07:53:02.955145+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:57.435952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952771+0000","last_change":"2024-09-19T07:53:01.925865+0000","last_active":"2024-09-19T07:53:03.952771+0000","last_peered":"2024-09-19T07:53:03.952771+0000","last_clean":"2024-09-19T07:53:03.952771+0000","last_became_active":"2024-09-19T07:53:01.925544+0000","last_became_peered":"2024-09-19T07:53:01.925544+0000","last_unstale":"2024-09-19T07:53:03.952771+0000","last_undegraded":"2024-09-19T07:53:03.952771+0000","last_fullsized":"2024-09-19T07:53:03.952771+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:57.201088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959514+0000","last_change":"2024-09-19T07:53:02.970190+0000","last_active":"2024-09-19T07:53:03.959514+0000","last_peered":"2024-09-19T07:53:03.959514+0000","last_clean":"2024-09-19T07:53:03.959514+0000","last_became_active":"2024-09-19T07:53:02.969897+0000","last_became_peered":"2024-09-19T07:53:02.969897+0000","last_unstale":"2024-09-19T07:53:03.959514+0000","last_undegraded":"2024-09-19T07:53:03.959514+0000","last_fullsized":"2024-09-19T07:53:03.959514+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:24.195258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089522+0000","last_change":"2024-09-19T07:53:01.926600+0000","last_active":"2024-09-19T07:53:04.089522+0000","last_peered":"2024-09-19T07:53:04.089522+0000","last_clean":"2024-09-19T07:53:04.089522+0000","last_became_active":"2024-09-19T07:53:01.925587+0000","last_became_peered":"2024-09-19T07:53:01.925587+0000","last_unstale":"2024-09-19T07:53:04.089522+0000","last_undegraded":"2024-09-19T07:53:04.089522+0000","last_fullsized":"2024-09-19T07:53:04.089522+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:13:51.412984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959770+0000","last_change":"2024-09-19T07:53:02.946678+0000","last_active":"2024-09-19T07:53:03.959770+0000","last_peered":"2024-09-19T07:53:03.959770+0000","last_clean":"2024-09-19T07:53:03.959770+0000","last_became_active":"2024-09-19T07:53:02.946357+0000","last_became_peered":"2024-09-19T07:53:02.946357+0000","last_unstale":"2024-09-19T07:53:03.959770+0000","last_undegraded":"2024-09-19T07:53:03.959770+0000","last_fullsized":"2024-09-19T07:53:03.959770+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:20.948125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088503+0000","last_change":"2024-09-19T07:53:01.926198+0000","last_active":"2024-09-19T07:53:04.088503+0000","last_peered":"2024-09-19T07:53:04.088503+0000","last_clean":"2024-09-19T07:53:04.088503+0000","last_became_active":"2024-09-19T07:53:01.925873+0000","last_became_peered":"2024-09-19T07:53:01.925873+0000","last_unstale":"2024-09-19T07:53:04.088503+0000","last_undegraded":"2024-09-19T07:53:04.088503+0000","last_fullsized":"2024-09-19T07:53:04.088503+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:23:00.870329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.968168+0000","last_change":"2024-09-19T07:53:02.965053+0000","last_active":"2024-09-19T07:53:02.968168+0000","last_peered":"2024-09-19T07:53:02.968168+0000","last_clean":"2024-09-19T07:53:02.968168+0000","last_became_active":"2024-09-19T07:53:02.964769+0000","last_became_peered":"2024-09-19T07:53:02.964769+0000","last_unstale":"2024-09-19T07:53:02.968168+0000","last_undegraded":"2024-09-19T07:53:02.968168+0000","last_fullsized":"2024-09-19T07:53:02.968168+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:40.771106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089316+0000","last_change":"2024-09-19T07:53:01.923054+0000","last_active":"2024-09-19T07:53:04.089316+0000","last_peered":"2024-09-19T07:53:04.089316+0000","last_clean":"2024-09-19T07:53:04.089316+0000","last_became_active":"2024-09-19T07:53:01.922799+0000","last_became_peered":"2024-09-19T07:53:01.922799+0000","last_unstale":"2024-09-19T07:53:04.089316+0000","last_undegraded":"2024-09-19T07:53:04.089316+0000","last_fullsized":"2024-09-19T07:53:04.089316+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:47.745785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089660+0000","last_change":"2024-09-19T07:53:02.957041+0000","last_active":"2024-09-19T07:53:04.089660+0000","last_peered":"2024-09-19T07:53:04.089660+0000","last_clean":"2024-09-19T07:53:04.089660+0000","last_became_active":"2024-09-19T07:53:02.956729+0000","last_became_peered":"2024-09-19T07:53:02.956729+0000","last_unstale":"2024-09-19T07:53:04.089660+0000","last_undegraded":"2024-09-19T07:53:04.089660+0000","last_fullsized":"2024-09-19T07:53:04.089660+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:24:06.262792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088870+0000","last_change":"2024-09-19T07:53:01.923097+0000","last_active":"2024-09-19T07:53:04.088870+0000","last_peered":"2024-09-19T07:53:04.088870+0000","last_clean":"2024-09-19T07:53:04.088870+0000","last_became_active":"2024-09-19T07:53:01.922794+0000","last_became_peered":"2024-09-19T07:53:01.922794+0000","last_unstale":"2024-09-19T07:53:04.088870+0000","last_undegraded":"2024-09-19T07:53:04.088870+0000","last_fullsized":"2024-09-19T07:53:04.088870+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:29.994651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959382+0000","last_change":"2024-09-19T07:53:02.968250+0000","last_active":"2024-09-19T07:53:03.959382+0000","last_peered":"2024-09-19T07:53:03.959382+0000","last_clean":"2024-09-19T07:53:03.959382+0000","last_became_active":"2024-09-19T07:53:02.967961+0000","last_became_peered":"2024-09-19T07:53:02.967961+0000","last_unstale":"2024-09-19T07:53:03.959382+0000","last_undegraded":"2024-09-19T07:53:03.959382+0000","last_fullsized":"2024-09-19T07:53:03.959382+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:03.803861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.969747+0000","last_change":"2024-09-19T07:53:01.922110+0000","last_active":"2024-09-19T07:53:02.969747+0000","last_peered":"2024-09-19T07:53:02.969747+0000","last_clean":"2024-09-19T07:53:02.969747+0000","last_became_active":"2024-09-19T07:53:01.921769+0000","last_became_peered":"2024-09-19T07:53:01.921769+0000","last_unstale":"2024-09-19T07:53:02.969747+0000","last_undegraded":"2024-09-19T07:53:02.969747+0000","last_fullsized":"2024-09-19T07:53:02.969747+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:56.053928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951681+0000","last_change":"2024-09-19T07:53:02.963616+0000","last_active":"2024-09-19T07:53:03.951681+0000","last_peered":"2024-09-19T07:53:03.951681+0000","last_clean":"2024-09-19T07:53:03.951681+0000","last_became_active":"2024-09-19T07:53:02.963333+0000","last_became_peered":"2024-09-19T07:53:02.963333+0000","last_unstale":"2024-09-19T07:53:03.951681+0000","last_undegraded":"2024-09-19T07:53:03.951681+0000","last_fullsized":"2024-09-19T07:53:03.951681+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"106'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.961104+0000","last_change":"2024-09-19T07:53:01.922601+0000","last_active":"2024-09-19T07:53:03.961104+0000","last_peered":"2024-09-19T07:53:03.961104+0000","last_clean":"2024-09-19T07:53:03.961104+0000","last_became_active":"2024-09-19T07:53:01.922324+0000","last_became_peered":"2024-09-19T07:53:01.922324+0000","last_unstale":"2024-09-19T07:53:03.961104+0000","last_undegraded":"2024-09-19T07:53:03.961104+0000","last_fullsized":"2024-09-19T07:53:03.961104+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:02.428894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.962345+0000","last_change":"2024-09-19T07:53:02.961218+0000","last_active":"2024-09-19T07:53:02.962345+0000","last_peered":"2024-09-19T07:53:02.962345+0000","last_clean":"2024-09-19T07:53:02.962345+0000","last_became_active":"2024-09-19T07:53:02.960873+0000","last_became_peered":"2024-09-19T07:53:02.960873+0000","last_unstale":"2024-09-19T07:53:02.962345+0000","last_undegraded":"2024-09-19T07:53:02.962345+0000","last_fullsized":"2024-09-19T07:53:02.962345+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:27.358096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089482+0000","last_change":"2024-09-19T07:53:01.920744+0000","last_active":"2024-09-19T07:53:04.089482+0000","last_peered":"2024-09-19T07:53:04.089482+0000","last_clean":"2024-09-19T07:53:04.089482+0000","last_became_active":"2024-09-19T07:53:01.920476+0000","last_became_peered":"2024-09-19T07:53:01.920476+0000","last_unstale":"2024-09-19T07:53:04.089482+0000","last_undegraded":"2024-09-19T07:53:04.089482+0000","last_fullsized":"2024-09-19T07:53:04.089482+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:11.662918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952379+0000","last_change":"2024-09-19T07:53:02.961727+0000","last_active":"2024-09-19T07:53:03.952379+0000","last_peered":"2024-09-19T07:53:03.952379+0000","last_clean":"2024-09-19T07:53:03.952379+0000","last_became_active":"2024-09-19T07:53:02.961447+0000","last_became_peered":"2024-09-19T07:53:02.961447+0000","last_unstale":"2024-09-19T07:53:03.952379+0000","last_undegraded":"2024-09-19T07:53:03.952379+0000","last_fullsized":"2024-09-19T07:53:03.952379+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:26:55.837328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-19T07:53:02.940797+0000","last_change":"2024-09-19T07:53:01.922583+0000","last_active":"2024-09-19T07:53:02.940797+0000","last_peered":"2024-09-19T07:53:02.940797+0000","last_clean":"2024-09-19T07:53:02.940797+0000","last_became_active":"2024-09-19T07:53:01.922318+0000","last_became_peered":"2024-09-19T07:53:01.922318+0000","last_unstale":"2024-09-19T07:53:02.940797+0000","last_undegraded":"2024-09-19T07:53:02.940797+0000","last_fullsized":"2024-09-19T07:53:02.940797+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:14.312065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089440+0000","last_change":"2024-09-19T07:53:02.957752+0000","last_active":"2024-09-19T07:53:04.089440+0000","last_peered":"2024-09-19T07:53:04.089440+0000","last_clean":"2024-09-19T07:53:04.089440+0000","last_became_active":"2024-09-19T07:53:02.957469+0000","last_became_peered":"2024-09-19T07:53:02.957469+0000","last_unstale":"2024-09-19T07:53:04.089440+0000","last_undegraded":"2024-09-19T07:53:04.089440+0000","last_fullsized":"2024-09-19T07:53:04.089440+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:18.785138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952032+0000","last_change":"2024-09-19T07:53:01.918087+0000","last_active":"2024-09-19T07:53:03.952032+0000","last_peered":"2024-09-19T07:53:03.952032+0000","last_clean":"2024-09-19T07:53:03.952032+0000","last_became_active":"2024-09-19T07:53:01.917738+0000","last_became_peered":"2024-09-19T07:53:01.917738+0000","last_unstale":"2024-09-19T07:53:03.952032+0000","last_undegraded":"2024-09-19T07:53:03.952032+0000","last_fullsized":"2024-09-19T07:53:03.952032+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:14.312065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089339+0000","last_change":"2024-09-19T07:53:02.961489+0000","last_active":"2024-09-19T07:53:04.089339+0000","last_peered":"2024-09-19T07:53:04.089339+0000","last_clean":"2024-09-19T07:53:04.089339+0000","last_became_active":"2024-09-19T07:53:02.961212+0000","last_became_peered":"2024-09-19T07:53:02.961212+0000","last_unstale":"2024-09-19T07:53:04.089339+0000","last_undegraded":"2024-09-19T07:53:04.089339+0000","last_fullsized":"2024-09-19T07:53:04.089339+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:15.778175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088861+0000","last_change":"2024-09-19T07:53:01.920116+0000","last_active":"2024-09-19T07:53:04.088861+0000","last_peered":"2024-09-19T07:53:04.088861+0000","last_clean":"2024-09-19T07:53:04.088861+0000","last_became_active":"2024-09-19T07:53:01.919834+0000","last_became_peered":"2024-09-19T07:53:01.919834+0000","last_unstale":"2024-09-19T07:53:04.088861+0000","last_undegraded":"2024-09-19T07:53:04.088861+0000","last_fullsized":"2024-09-19T07:53:04.088861+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:18.763151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089415+0000","last_change":"2024-09-19T07:53:02.957282+0000","last_active":"2024-09-19T07:53:04.089415+0000","last_peered":"2024-09-19T07:53:04.089415+0000","last_clean":"2024-09-19T07:53:04.089415+0000","last_became_active":"2024-09-19T07:53:02.957004+0000","last_became_peered":"2024-09-19T07:53:02.957004+0000","last_unstale":"2024-09-19T07:53:04.089415+0000","last_undegraded":"2024-09-19T07:53:04.089415+0000","last_fullsized":"2024-09-19T07:53:04.089415+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:25:45.688277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089434+0000","last_change":"2024-09-19T07:53:01.921607+0000","last_active":"2024-09-19T07:53:04.089434+0000","last_peered":"2024-09-19T07:53:04.089434+0000","last_clean":"2024-09-19T07:53:04.089434+0000","last_became_active":"2024-09-19T07:53:01.920937+0000","last_became_peered":"2024-09-19T07:53:01.920937+0000","last_unstale":"2024-09-19T07:53:04.089434+0000","last_undegraded":"2024-09-19T07:53:04.089434+0000","last_fullsized":"2024-09-19T07:53:04.089434+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:35.664386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.953006+0000","last_change":"2024-09-19T07:53:02.956981+0000","last_active":"2024-09-19T07:53:03.953006+0000","last_peered":"2024-09-19T07:53:03.953006+0000","last_clean":"2024-09-19T07:53:03.953006+0000","last_became_active":"2024-09-19T07:53:02.955937+0000","last_became_peered":"2024-09-19T07:53:02.955937+0000","last_unstale":"2024-09-19T07:53:03.953006+0000","last_undegraded":"2024-09-19T07:53:03.953006+0000","last_fullsized":"2024-09-19T07:53:03.953006+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:39.353799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088926+0000","last_change":"2024-09-19T07:53:01.924267+0000","last_active":"2024-09-19T07:53:04.088926+0000","last_peered":"2024-09-19T07:53:04.088926+0000","last_clean":"2024-09-19T07:53:04.088926+0000","last_became_active":"2024-09-19T07:53:01.924001+0000","last_became_peered":"2024-09-19T07:53:01.924001+0000","last_unstale":"2024-09-19T07:53:04.088926+0000","last_undegraded":"2024-09-19T07:53:04.088926+0000","last_fullsized":"2024-09-19T07:53:04.088926+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:15.069406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951881+0000","last_change":"2024-09-19T07:53:02.955750+0000","last_active":"2024-09-19T07:53:03.951881+0000","last_peered":"2024-09-19T07:53:03.951881+0000","last_clean":"2024-09-19T07:53:03.951881+0000","last_became_active":"2024-09-19T07:53:02.955250+0000","last_became_peered":"2024-09-19T07:53:02.955250+0000","last_unstale":"2024-09-19T07:53:03.951881+0000","last_undegraded":"2024-09-19T07:53:03.951881+0000","last_fullsized":"2024-09-19T07:53:03.951881+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:22.292252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089547+0000","last_change":"2024-09-19T07:53:01.921714+0000","last_active":"2024-09-19T07:53:04.089547+0000","last_peered":"2024-09-19T07:53:04.089547+0000","last_clean":"2024-09-19T07:53:04.089547+0000","last_became_active":"2024-09-19T07:53:01.921436+0000","last_became_peered":"2024-09-19T07:53:01.921436+0000","last_unstale":"2024-09-19T07:53:04.089547+0000","last_undegraded":"2024-09-19T07:53:04.089547+0000","last_fullsized":"2024-09-19T07:53:04.089547+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:04.601086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":325,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088582+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:53:04.088582+0000","last_peered":"2024-09-19T07:53:04.088582+0000","last_clean":"2024-09-19T07:53:04.088582+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:53:04.088582+0000","last_undegraded":"2024-09-19T07:53:04.088582+0000","last_fullsized":"2024-09-19T07:53:04.088582+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509565,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82960,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288880,"statfs":{"total":96636764160,"available":96551813120,"internally_reserved":0,"allocated":1896448,"data_stored":1188786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509564,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100520,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":94271320,"statfs":{"total":96636764160,"available":96533831680,"internally_reserved":0,"allocated":1327104,"data_stored":607166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":101579206},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509565,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50512,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321328,"statfs":{"total":96636764160,"available":96585039872,"internally_reserved":0,"allocated":1306624,"data_stored":604950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509565,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86552,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285288,"statfs":{"total":96636764160,"available":96548134912,"internally_reserved":0,"allocated":1314816,"data_stored":605042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509564,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":101020,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94270820,"statfs":{"total":96636764160,"available":96533319680,"internally_reserved":0,"allocated":1904640,"data_stored":1190328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542268,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135120,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236720,"statfs":{"total":96636764160,"available":96498401280,"internally_reserved":0,"allocated":1298432,"data_stored":598840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":137034183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542268,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94928,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276912,"statfs":{"total":96636764160,"available":96539557888,"internally_reserved":0,"allocated":1298432,"data_stored":598292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574974,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104600,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267240,"statfs":{"total":96636764160,"available":96529653760,"internally_reserved":0,"allocated":1310720,"data_stored":604984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:53:08.209 INFO:tasks.ceph.mds.c.smithi016.stderr:2024-09-19T07:53:08.209+0000 7fe2e869e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T07:53:09.058 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:53:09.061 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:53:10.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:53:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.472+0000 7f1e800b7640 1 Processor -- start 2024-09-19T07:53:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.472+0000 7f1e800b7640 1 -- start start 2024-09-19T07:53:10.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.473+0000 7f1e800b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e78141680 0x7f1e780cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:10.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.473+0000 7f1e800b7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1e78142170 0x7f1e7813f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:10.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.473+0000 7f1e800b7640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e7815d710 con 0x7f1e780d0370 2024-09-19T07:53:10.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.473+0000 7f1e800b7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f1e780d07b0 con 0x7f1e78142170 2024-09-19T07:53:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.473+0000 7f1e800b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1e7815c6f0 con 0x7f1e78141680 2024-09-19T07:53:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.474+0000 7f1e7e654640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1e78142170 0x7f1e7813f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.474+0000 7f1e7e654640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1e78142170 0x7f1e7813f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60820/0 (socket says 172.21.15.16:60820) 2024-09-19T07:53:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.474+0000 7f1e7e654640 1 -- 172.21.15.16:0/2678797878 learned_addr learned my addr 172.21.15.16:0/2678797878 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.474+0000 7f1e7de53640 1 --2- 172.21.15.16:0/2678797878 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e78141680 0x7f1e780cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.474+0000 7f1e7e654640 1 -- 172.21.15.16:0/2678797878 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e78141680 msgr2=0x7f1e780cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.474+0000 7f1e7e654640 1 --2- 172.21.15.16:0/2678797878 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e78141680 0x7f1e780cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.474+0000 7f1e7e654640 1 -- 172.21.15.16:0/2678797878 >> v1:172.21.15.146:6789/0 conn(0x7f1e780d0370 legacy=0x7f1e78141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.474+0000 7f1e7e654640 1 -- 172.21.15.16:0/2678797878 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e78142600 con 0x7f1e78142170 2024-09-19T07:53:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.475+0000 7f1e7e654640 1 --2- 172.21.15.16:0/2678797878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e78142170 0x7f1e7813f8b0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009870 tx=0x7f1e7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618f52c28ac9200a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.475+0000 7f1e7ce51640 1 -- 172.21.15.16:0/2678797878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70037070 con 0x7f1e78142170 2024-09-19T07:53:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.475+0000 7f1e7ce51640 1 -- 172.21.15.16:0/2678797878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e70032440 con 0x7f1e78142170 2024-09-19T07:53:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.475+0000 7f1e7ce51640 1 -- 172.21.15.16:0/2678797878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70032810 con 0x7f1e78142170 2024-09-19T07:53:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.475+0000 7f1e800b7640 1 -- 172.21.15.16:0/2678797878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e78142170 msgr2=0x7f1e7813f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.475+0000 7f1e800b7640 1 --2- 172.21.15.16:0/2678797878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e78142170 0x7f1e7813f8b0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009870 tx=0x7f1e7002a270 comp rx=0 tx=0).stop 2024-09-19T07:53:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.475+0000 7f1e800b7640 1 -- 172.21.15.16:0/2678797878 shutdown_connections 2024-09-19T07:53:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.475+0000 7f1e800b7640 1 --2- 172.21.15.16:0/2678797878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e78142170 0x7f1e7813f8b0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.475+0000 7f1e800b7640 1 --2- 172.21.15.16:0/2678797878 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e78141680 0x7f1e780cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.475+0000 7f1e800b7640 1 -- 172.21.15.16:0/2678797878 >> 172.21.15.16:0/2678797878 conn(0x7f1e780b3ed0 msgr2=0x7f1e780b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.476+0000 7f1e800b7640 1 -- 172.21.15.16:0/2678797878 shutdown_connections 2024-09-19T07:53:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.476+0000 7f1e800b7640 1 -- 172.21.15.16:0/2678797878 wait complete. 2024-09-19T07:53:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.477+0000 7f1e800b7640 1 Processor -- start 2024-09-19T07:53:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.477+0000 7f1e800b7640 1 -- start start 2024-09-19T07:53:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e800b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e780d0370 0x7f1e7813ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e800b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e78141680 0x7f1e781f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e800b7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e78142170 0x7f1e780cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e800b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7815cf10 con 0x7f1e78141680 2024-09-19T07:53:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e800b7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7815cc30 con 0x7f1e78142170 2024-09-19T07:53:10.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e800b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1e7815d1f0 con 0x7f1e780d0370 2024-09-19T07:53:10.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e7d652640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e78141680 0x7f1e781f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:10.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e7d652640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e78141680 0x7f1e781f1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60830/0 (socket says 172.21.15.16:60830) 2024-09-19T07:53:10.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e7d652640 1 -- 172.21.15.16:0/1362143399 learned_addr learned my addr 172.21.15.16:0/1362143399 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:10.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e7de53640 1 --2- 172.21.15.16:0/1362143399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e780d0370 0x7f1e7813ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:10.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e7d652640 1 -- 172.21.15.16:0/1362143399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e780d0370 msgr2=0x7f1e7813ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:10.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e7d652640 1 --2- 172.21.15.16:0/1362143399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e780d0370 0x7f1e7813ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e7e654640 1 --2- 172.21.15.16:0/1362143399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e78142170 0x7f1e780cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:10.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e7d652640 1 -- 172.21.15.16:0/1362143399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e78142170 msgr2=0x7f1e780cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:10.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e7d652640 1 --2- 172.21.15.16:0/1362143399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e78142170 0x7f1e780cacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e7d652640 1 -- 172.21.15.16:0/1362143399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7813f120 con 0x7f1e78141680 2024-09-19T07:53:10.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e7d652640 1 --2- 172.21.15.16:0/1362143399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e78141680 0x7f1e781f1d10 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c440 tx=0x7f1e6c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:10.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e767fc640 1 -- 172.21.15.16:0/1362143399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c0052d0 con 0x7f1e78141680 2024-09-19T07:53:10.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e767fc640 1 -- 172.21.15.16:0/1362143399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e6c005470 con 0x7f1e78141680 2024-09-19T07:53:10.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e800b7640 1 -- 172.21.15.16:0/1362143399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e780c9c50 con 0x7f1e78141680 2024-09-19T07:53:10.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e767fc640 1 -- 172.21.15.16:0/1362143399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c007260 con 0x7f1e78141680 2024-09-19T07:53:10.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.478+0000 7f1e800b7640 1 -- 172.21.15.16:0/1362143399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e780ca210 con 0x7f1e78141680 2024-09-19T07:53:10.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.479+0000 7f1e800b7640 1 -- 172.21.15.16:0/1362143399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e780cfe30 con 0x7f1e78141680 2024-09-19T07:53:10.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.480+0000 7f1e767fc640 1 -- 172.21.15.16:0/1362143399 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1e6c007400 con 0x7f1e78141680 2024-09-19T07:53:10.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.480+0000 7f1e767fc640 1 --2- 172.21.15.16:0/1362143399 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1e6406c2b0 0x7f1e6406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:10.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.480+0000 7f1e767fc640 1 -- 172.21.15.16:0/1362143399 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f1e6c08aa80 con 0x7f1e78141680 2024-09-19T07:53:10.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.481+0000 7f1e7de53640 1 --2- 172.21.15.16:0/1362143399 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1e6406c2b0 0x7f1e6406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:10.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.481+0000 7f1e7de53640 1 --2- 172.21.15.16:0/1362143399 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1e6406c2b0 0x7f1e6406e770 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1e68007330 tx=0x7f1e6800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:10.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.485+0000 7f1e767fc640 1 -- 172.21.15.16:0/1362143399 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1e6c0590f0 con 0x7f1e78141680 2024-09-19T07:53:10.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.658+0000 7f1e800b7640 1 -- 172.21.15.16:0/1362143399 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1e780b2780 con 0x7f1e6406c2b0 2024-09-19T07:53:10.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.664+0000 7f1e767fc640 1 -- 172.21.15.16:0/1362143399 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252408 (secure 0 0 0) 0x7f1e780b2780 con 0x7f1e6406c2b0 2024-09-19T07:53:10.667 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:10.669 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:53:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.667+0000 7f1e800b7640 1 -- 172.21.15.16:0/1362143399 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1e6406c2b0 msgr2=0x7f1e6406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.667+0000 7f1e800b7640 1 --2- 172.21.15.16:0/1362143399 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1e6406c2b0 0x7f1e6406e770 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1e68007330 tx=0x7f1e6800b040 comp rx=0 tx=0).stop 2024-09-19T07:53:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.667+0000 7f1e800b7640 1 -- 172.21.15.16:0/1362143399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e78141680 msgr2=0x7f1e781f1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.667+0000 7f1e800b7640 1 --2- 172.21.15.16:0/1362143399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e78141680 0x7f1e781f1d10 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c440 tx=0x7f1e6c001e70 comp rx=0 tx=0).stop 2024-09-19T07:53:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.667+0000 7f1e800b7640 1 -- 172.21.15.16:0/1362143399 shutdown_connections 2024-09-19T07:53:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.667+0000 7f1e800b7640 1 --2- 172.21.15.16:0/1362143399 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1e6406c2b0 0x7f1e6406e770 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.667+0000 7f1e800b7640 1 --2- 172.21.15.16:0/1362143399 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e78142170 0x7f1e780cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.667+0000 7f1e800b7640 1 --2- 172.21.15.16:0/1362143399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e78141680 0x7f1e781f1d10 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.667+0000 7f1e800b7640 1 --2- 172.21.15.16:0/1362143399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e780d0370 0x7f1e7813ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.667+0000 7f1e800b7640 1 -- 172.21.15.16:0/1362143399 >> 172.21.15.16:0/1362143399 conn(0x7f1e780b3ed0 msgr2=0x7f1e780b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.668+0000 7f1e800b7640 1 -- 172.21.15.16:0/1362143399 shutdown_connections 2024-09-19T07:53:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.668+0000 7f1e800b7640 1 -- 172.21.15.16:0/1362143399 wait complete. 2024-09-19T07:53:10.679 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-09-19T07:53:10.601578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":756252,"kb_used_data":11432,"kb_used_omap":12,"kb_used_meta":744627,"kb_avail":754218468,"statfs":{"total":773094113280,"available":772319711232,"internally_reserved":0,"allocated":11706368,"data_stored":6025156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":762498608},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001993"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959791+0000","last_change":"2024-09-19T07:53:02.969667+0000","last_active":"2024-09-19T07:53:03.959791+0000","last_peered":"2024-09-19T07:53:03.959791+0000","last_clean":"2024-09-19T07:53:03.959791+0000","last_became_active":"2024-09-19T07:53:02.969269+0000","last_became_peered":"2024-09-19T07:53:02.969269+0000","last_unstale":"2024-09-19T07:53:03.959791+0000","last_undegraded":"2024-09-19T07:53:03.959791+0000","last_fullsized":"2024-09-19T07:53:03.959791+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:21.528651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089709+0000","last_change":"2024-09-19T07:53:02.960077+0000","last_active":"2024-09-19T07:53:04.089709+0000","last_peered":"2024-09-19T07:53:04.089709+0000","last_clean":"2024-09-19T07:53:04.089709+0000","last_became_active":"2024-09-19T07:53:02.959395+0000","last_became_peered":"2024-09-19T07:53:02.959395+0000","last_unstale":"2024-09-19T07:53:04.089709+0000","last_undegraded":"2024-09-19T07:53:04.089709+0000","last_fullsized":"2024-09-19T07:53:04.089709+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959657+0000","last_change":"2024-09-19T07:53:02.961895+0000","last_active":"2024-09-19T07:53:03.959657+0000","last_peered":"2024-09-19T07:53:03.959657+0000","last_clean":"2024-09-19T07:53:03.959657+0000","last_became_active":"2024-09-19T07:53:02.961616+0000","last_became_peered":"2024-09-19T07:53:02.961616+0000","last_unstale":"2024-09-19T07:53:03.959657+0000","last_undegraded":"2024-09-19T07:53:03.959657+0000","last_fullsized":"2024-09-19T07:53:03.959657+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:01.957812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.948131+0000","last_change":"2024-09-19T07:53:02.961725+0000","last_active":"2024-09-19T07:53:03.948131+0000","last_peered":"2024-09-19T07:53:03.948131+0000","last_clean":"2024-09-19T07:53:03.948131+0000","last_became_active":"2024-09-19T07:53:02.961334+0000","last_became_peered":"2024-09-19T07:53:02.961334+0000","last_unstale":"2024-09-19T07:53:03.948131+0000","last_undegraded":"2024-09-19T07:53:03.948131+0000","last_fullsized":"2024-09-19T07:53:03.948131+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:02.766217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089015+0000","last_change":"2024-09-19T07:53:02.945749+0000","last_active":"2024-09-19T07:53:04.089015+0000","last_peered":"2024-09-19T07:53:04.089015+0000","last_clean":"2024-09-19T07:53:04.089015+0000","last_became_active":"2024-09-19T07:53:02.945466+0000","last_became_peered":"2024-09-19T07:53:02.945466+0000","last_unstale":"2024-09-19T07:53:04.089015+0000","last_undegraded":"2024-09-19T07:53:04.089015+0000","last_fullsized":"2024-09-19T07:53:04.089015+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:58.202929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088937+0000","last_change":"2024-09-19T07:53:02.960167+0000","last_active":"2024-09-19T07:53:04.088937+0000","last_peered":"2024-09-19T07:53:04.088937+0000","last_clean":"2024-09-19T07:53:04.088937+0000","last_became_active":"2024-09-19T07:53:02.959874+0000","last_became_peered":"2024-09-19T07:53:02.959874+0000","last_unstale":"2024-09-19T07:53:04.088937+0000","last_undegraded":"2024-09-19T07:53:04.088937+0000","last_fullsized":"2024-09-19T07:53:04.088937+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:17.832687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088822+0000","last_change":"2024-09-19T07:53:02.973657+0000","last_active":"2024-09-19T07:53:04.088822+0000","last_peered":"2024-09-19T07:53:04.088822+0000","last_clean":"2024-09-19T07:53:04.088822+0000","last_became_active":"2024-09-19T07:53:02.973421+0000","last_became_peered":"2024-09-19T07:53:02.973421+0000","last_unstale":"2024-09-19T07:53:04.088822+0000","last_undegraded":"2024-09-19T07:53:04.088822+0000","last_fullsized":"2024-09-19T07:53:04.088822+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:28.288866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088738+0000","last_change":"2024-09-19T07:53:02.959944+0000","last_active":"2024-09-19T07:53:04.088738+0000","last_peered":"2024-09-19T07:53:04.088738+0000","last_clean":"2024-09-19T07:53:04.088738+0000","last_became_active":"2024-09-19T07:53:02.959654+0000","last_became_peered":"2024-09-19T07:53:02.959654+0000","last_unstale":"2024-09-19T07:53:04.088738+0000","last_undegraded":"2024-09-19T07:53:04.088738+0000","last_fullsized":"2024-09-19T07:53:04.088738+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:48.854939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089588+0000","last_change":"2024-09-19T07:53:02.966890+0000","last_active":"2024-09-19T07:53:04.089588+0000","last_peered":"2024-09-19T07:53:04.089588+0000","last_clean":"2024-09-19T07:53:04.089588+0000","last_became_active":"2024-09-19T07:53:02.966573+0000","last_became_peered":"2024-09-19T07:53:02.966573+0000","last_unstale":"2024-09-19T07:53:04.089588+0000","last_undegraded":"2024-09-19T07:53:04.089588+0000","last_fullsized":"2024-09-19T07:53:04.089588+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:22.222446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089386+0000","last_change":"2024-09-19T07:53:02.969544+0000","last_active":"2024-09-19T07:53:04.089386+0000","last_peered":"2024-09-19T07:53:04.089386+0000","last_clean":"2024-09-19T07:53:04.089386+0000","last_became_active":"2024-09-19T07:53:02.969117+0000","last_became_peered":"2024-09-19T07:53:02.969117+0000","last_unstale":"2024-09-19T07:53:04.089386+0000","last_undegraded":"2024-09-19T07:53:04.089386+0000","last_fullsized":"2024-09-19T07:53:04.089386+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:30.807092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952211+0000","last_change":"2024-09-19T07:53:02.960894+0000","last_active":"2024-09-19T07:53:03.952211+0000","last_peered":"2024-09-19T07:53:03.952211+0000","last_clean":"2024-09-19T07:53:03.952211+0000","last_became_active":"2024-09-19T07:53:02.960589+0000","last_became_peered":"2024-09-19T07:53:02.960589+0000","last_unstale":"2024-09-19T07:53:03.952211+0000","last_undegraded":"2024-09-19T07:53:03.952211+0000","last_fullsized":"2024-09-19T07:53:03.952211+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:24:28.417746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951606+0000","last_change":"2024-09-19T07:53:02.964794+0000","last_active":"2024-09-19T07:53:03.951606+0000","last_peered":"2024-09-19T07:53:03.951606+0000","last_clean":"2024-09-19T07:53:03.951606+0000","last_became_active":"2024-09-19T07:53:02.964326+0000","last_became_peered":"2024-09-19T07:53:02.964326+0000","last_unstale":"2024-09-19T07:53:03.951606+0000","last_undegraded":"2024-09-19T07:53:03.951606+0000","last_fullsized":"2024-09-19T07:53:03.951606+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:22.222446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.947972+0000","last_change":"2024-09-19T07:53:02.966603+0000","last_active":"2024-09-19T07:53:03.947972+0000","last_peered":"2024-09-19T07:53:03.947972+0000","last_clean":"2024-09-19T07:53:03.947972+0000","last_became_active":"2024-09-19T07:53:02.966330+0000","last_became_peered":"2024-09-19T07:53:02.966330+0000","last_unstale":"2024-09-19T07:53:03.947972+0000","last_undegraded":"2024-09-19T07:53:03.947972+0000","last_fullsized":"2024-09-19T07:53:03.947972+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:56.702898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089112+0000","last_change":"2024-09-19T07:53:02.962007+0000","last_active":"2024-09-19T07:53:04.089112+0000","last_peered":"2024-09-19T07:53:04.089112+0000","last_clean":"2024-09-19T07:53:04.089112+0000","last_became_active":"2024-09-19T07:53:02.961724+0000","last_became_peered":"2024-09-19T07:53:02.961724+0000","last_unstale":"2024-09-19T07:53:04.089112+0000","last_undegraded":"2024-09-19T07:53:04.089112+0000","last_fullsized":"2024-09-19T07:53:04.089112+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:39.353799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088923+0000","last_change":"2024-09-19T07:53:02.957971+0000","last_active":"2024-09-19T07:53:04.088923+0000","last_peered":"2024-09-19T07:53:04.088923+0000","last_clean":"2024-09-19T07:53:04.088923+0000","last_became_active":"2024-09-19T07:53:02.957708+0000","last_became_peered":"2024-09-19T07:53:02.957708+0000","last_unstale":"2024-09-19T07:53:04.088923+0000","last_undegraded":"2024-09-19T07:53:04.088923+0000","last_fullsized":"2024-09-19T07:53:04.088923+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:56.130236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088952+0000","last_change":"2024-09-19T07:53:02.954436+0000","last_active":"2024-09-19T07:53:04.088952+0000","last_peered":"2024-09-19T07:53:04.088952+0000","last_clean":"2024-09-19T07:53:04.088952+0000","last_became_active":"2024-09-19T07:53:02.954126+0000","last_became_peered":"2024-09-19T07:53:02.954126+0000","last_unstale":"2024-09-19T07:53:04.088952+0000","last_undegraded":"2024-09-19T07:53:04.088952+0000","last_fullsized":"2024-09-19T07:53:04.088952+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:21.471735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959826+0000","last_change":"2024-09-19T07:53:02.957284+0000","last_active":"2024-09-19T07:53:03.959826+0000","last_peered":"2024-09-19T07:53:03.959826+0000","last_clean":"2024-09-19T07:53:03.959826+0000","last_became_active":"2024-09-19T07:53:02.957008+0000","last_became_peered":"2024-09-19T07:53:02.957008+0000","last_unstale":"2024-09-19T07:53:03.959826+0000","last_undegraded":"2024-09-19T07:53:03.959826+0000","last_fullsized":"2024-09-19T07:53:03.959826+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:24.443678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089749+0000","last_change":"2024-09-19T07:53:02.959833+0000","last_active":"2024-09-19T07:53:04.089749+0000","last_peered":"2024-09-19T07:53:04.089749+0000","last_clean":"2024-09-19T07:53:04.089749+0000","last_became_active":"2024-09-19T07:53:02.959076+0000","last_became_peered":"2024-09-19T07:53:02.959076+0000","last_unstale":"2024-09-19T07:53:04.089749+0000","last_undegraded":"2024-09-19T07:53:04.089749+0000","last_fullsized":"2024-09-19T07:53:04.089749+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:31.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959718+0000","last_change":"2024-09-19T07:53:02.970410+0000","last_active":"2024-09-19T07:53:03.959718+0000","last_peered":"2024-09-19T07:53:03.959718+0000","last_clean":"2024-09-19T07:53:03.959718+0000","last_became_active":"2024-09-19T07:53:02.967129+0000","last_became_peered":"2024-09-19T07:53:02.967129+0000","last_unstale":"2024-09-19T07:53:03.959718+0000","last_undegraded":"2024-09-19T07:53:03.959718+0000","last_fullsized":"2024-09-19T07:53:03.959718+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:46.990754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951748+0000","last_change":"2024-09-19T07:53:02.953069+0000","last_active":"2024-09-19T07:53:03.951748+0000","last_peered":"2024-09-19T07:53:03.951748+0000","last_clean":"2024-09-19T07:53:03.951748+0000","last_became_active":"2024-09-19T07:53:02.952498+0000","last_became_peered":"2024-09-19T07:53:02.952498+0000","last_unstale":"2024-09-19T07:53:03.951748+0000","last_undegraded":"2024-09-19T07:53:03.951748+0000","last_fullsized":"2024-09-19T07:53:03.951748+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:31.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089071+0000","last_change":"2024-09-19T07:53:02.965042+0000","last_active":"2024-09-19T07:53:04.089071+0000","last_peered":"2024-09-19T07:53:04.089071+0000","last_clean":"2024-09-19T07:53:04.089071+0000","last_became_active":"2024-09-19T07:53:02.964720+0000","last_became_peered":"2024-09-19T07:53:02.964720+0000","last_unstale":"2024-09-19T07:53:04.089071+0000","last_undegraded":"2024-09-19T07:53:04.089071+0000","last_fullsized":"2024-09-19T07:53:04.089071+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:49.707375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089076+0000","last_change":"2024-09-19T07:53:02.960655+0000","last_active":"2024-09-19T07:53:04.089076+0000","last_peered":"2024-09-19T07:53:04.089076+0000","last_clean":"2024-09-19T07:53:04.089076+0000","last_became_active":"2024-09-19T07:53:02.960373+0000","last_became_peered":"2024-09-19T07:53:02.960373+0000","last_unstale":"2024-09-19T07:53:04.089076+0000","last_undegraded":"2024-09-19T07:53:04.089076+0000","last_fullsized":"2024-09-19T07:53:04.089076+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:22.009944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959448+0000","last_change":"2024-09-19T07:53:02.958018+0000","last_active":"2024-09-19T07:53:03.959448+0000","last_peered":"2024-09-19T07:53:03.959448+0000","last_clean":"2024-09-19T07:53:03.959448+0000","last_became_active":"2024-09-19T07:53:02.957706+0000","last_became_peered":"2024-09-19T07:53:02.957706+0000","last_unstale":"2024-09-19T07:53:03.959448+0000","last_undegraded":"2024-09-19T07:53:03.959448+0000","last_fullsized":"2024-09-19T07:53:03.959448+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:08.021359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088577+0000","last_change":"2024-09-19T07:53:02.951003+0000","last_active":"2024-09-19T07:53:04.088577+0000","last_peered":"2024-09-19T07:53:04.088577+0000","last_clean":"2024-09-19T07:53:04.088577+0000","last_became_active":"2024-09-19T07:53:02.950262+0000","last_became_peered":"2024-09-19T07:53:02.950262+0000","last_unstale":"2024-09-19T07:53:04.088577+0000","last_undegraded":"2024-09-19T07:53:04.088577+0000","last_fullsized":"2024-09-19T07:53:04.088577+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.953137+0000","last_change":"2024-09-19T07:53:02.955789+0000","last_active":"2024-09-19T07:53:03.953137+0000","last_peered":"2024-09-19T07:53:03.953137+0000","last_clean":"2024-09-19T07:53:03.953137+0000","last_became_active":"2024-09-19T07:53:02.955480+0000","last_became_peered":"2024-09-19T07:53:02.955480+0000","last_unstale":"2024-09-19T07:53:03.953137+0000","last_undegraded":"2024-09-19T07:53:03.953137+0000","last_fullsized":"2024-09-19T07:53:03.953137+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:00.688869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952726+0000","last_change":"2024-09-19T07:53:02.962786+0000","last_active":"2024-09-19T07:53:03.952726+0000","last_peered":"2024-09-19T07:53:03.952726+0000","last_clean":"2024-09-19T07:53:03.952726+0000","last_became_active":"2024-09-19T07:53:02.961758+0000","last_became_peered":"2024-09-19T07:53:02.961758+0000","last_unstale":"2024-09-19T07:53:03.952726+0000","last_undegraded":"2024-09-19T07:53:03.952726+0000","last_fullsized":"2024-09-19T07:53:03.952726+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:43.470589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952391+0000","last_change":"2024-09-19T07:53:02.955834+0000","last_active":"2024-09-19T07:53:03.952391+0000","last_peered":"2024-09-19T07:53:03.952391+0000","last_clean":"2024-09-19T07:53:03.952391+0000","last_became_active":"2024-09-19T07:53:02.955070+0000","last_became_peered":"2024-09-19T07:53:02.955070+0000","last_unstale":"2024-09-19T07:53:03.952391+0000","last_undegraded":"2024-09-19T07:53:03.952391+0000","last_fullsized":"2024-09-19T07:53:03.952391+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:24.696392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089356+0000","last_change":"2024-09-19T07:53:02.963635+0000","last_active":"2024-09-19T07:53:04.089356+0000","last_peered":"2024-09-19T07:53:04.089356+0000","last_clean":"2024-09-19T07:53:04.089356+0000","last_became_active":"2024-09-19T07:53:02.963299+0000","last_became_peered":"2024-09-19T07:53:02.963299+0000","last_unstale":"2024-09-19T07:53:04.089356+0000","last_undegraded":"2024-09-19T07:53:04.089356+0000","last_fullsized":"2024-09-19T07:53:04.089356+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:03.858829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089467+0000","last_change":"2024-09-19T07:53:02.953500+0000","last_active":"2024-09-19T07:53:04.089467+0000","last_peered":"2024-09-19T07:53:04.089467+0000","last_clean":"2024-09-19T07:53:04.089467+0000","last_became_active":"2024-09-19T07:53:02.953221+0000","last_became_peered":"2024-09-19T07:53:02.953221+0000","last_unstale":"2024-09-19T07:53:04.089467+0000","last_undegraded":"2024-09-19T07:53:04.089467+0000","last_fullsized":"2024-09-19T07:53:04.089467+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:33.997609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952126+0000","last_change":"2024-09-19T07:53:02.959004+0000","last_active":"2024-09-19T07:53:03.952126+0000","last_peered":"2024-09-19T07:53:03.952126+0000","last_clean":"2024-09-19T07:53:03.952126+0000","last_became_active":"2024-09-19T07:53:02.958656+0000","last_became_peered":"2024-09-19T07:53:02.958656+0000","last_unstale":"2024-09-19T07:53:03.952126+0000","last_undegraded":"2024-09-19T07:53:03.952126+0000","last_fullsized":"2024-09-19T07:53:03.952126+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:58.202929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089316+0000","last_change":"2024-09-19T07:53:02.968735+0000","last_active":"2024-09-19T07:53:04.089316+0000","last_peered":"2024-09-19T07:53:04.089316+0000","last_clean":"2024-09-19T07:53:04.089316+0000","last_became_active":"2024-09-19T07:53:02.968434+0000","last_became_peered":"2024-09-19T07:53:02.968434+0000","last_unstale":"2024-09-19T07:53:04.089316+0000","last_undegraded":"2024-09-19T07:53:04.089316+0000","last_fullsized":"2024-09-19T07:53:04.089316+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:57.435952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088787+0000","last_change":"2024-09-19T07:53:02.954559+0000","last_active":"2024-09-19T07:53:04.088787+0000","last_peered":"2024-09-19T07:53:04.088787+0000","last_clean":"2024-09-19T07:53:04.088787+0000","last_became_active":"2024-09-19T07:53:02.954302+0000","last_became_peered":"2024-09-19T07:53:02.954302+0000","last_unstale":"2024-09-19T07:53:04.088787+0000","last_undegraded":"2024-09-19T07:53:04.088787+0000","last_fullsized":"2024-09-19T07:53:04.088787+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:39.717541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959514+0000","last_change":"2024-09-19T07:53:02.964781+0000","last_active":"2024-09-19T07:53:03.959514+0000","last_peered":"2024-09-19T07:53:03.959514+0000","last_clean":"2024-09-19T07:53:03.959514+0000","last_became_active":"2024-09-19T07:53:02.964513+0000","last_became_peered":"2024-09-19T07:53:02.964513+0000","last_unstale":"2024-09-19T07:53:03.959514+0000","last_undegraded":"2024-09-19T07:53:03.959514+0000","last_fullsized":"2024-09-19T07:53:03.959514+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:26.718784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"105'4","reported_seq":18,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.948194+0000","last_change":"2024-09-19T07:53:01.917100+0000","last_active":"2024-09-19T07:53:03.948194+0000","last_peered":"2024-09-19T07:53:03.948194+0000","last_clean":"2024-09-19T07:53:03.948194+0000","last_became_active":"2024-09-19T07:53:01.916801+0000","last_became_peered":"2024-09-19T07:53:01.916801+0000","last_unstale":"2024-09-19T07:53:03.948194+0000","last_undegraded":"2024-09-19T07:53:03.948194+0000","last_fullsized":"2024-09-19T07:53:03.948194+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:25:44.686436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088766+0000","last_change":"2024-09-19T07:53:02.959134+0000","last_active":"2024-09-19T07:53:04.088766+0000","last_peered":"2024-09-19T07:53:04.088766+0000","last_clean":"2024-09-19T07:53:04.088766+0000","last_became_active":"2024-09-19T07:53:02.958328+0000","last_became_peered":"2024-09-19T07:53:02.958328+0000","last_unstale":"2024-09-19T07:53:04.088766+0000","last_undegraded":"2024-09-19T07:53:04.088766+0000","last_fullsized":"2024-09-19T07:53:04.088766+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:42.128496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089533+0000","last_change":"2024-09-19T07:53:01.927253+0000","last_active":"2024-09-19T07:53:04.089533+0000","last_peered":"2024-09-19T07:53:04.089533+0000","last_clean":"2024-09-19T07:53:04.089533+0000","last_became_active":"2024-09-19T07:53:01.926918+0000","last_became_peered":"2024-09-19T07:53:01.926918+0000","last_unstale":"2024-09-19T07:53:04.089533+0000","last_undegraded":"2024-09-19T07:53:04.089533+0000","last_fullsized":"2024-09-19T07:53:04.089533+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:55:57.845393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952282+0000","last_change":"2024-09-19T07:53:02.950143+0000","last_active":"2024-09-19T07:53:03.952282+0000","last_peered":"2024-09-19T07:53:03.952282+0000","last_clean":"2024-09-19T07:53:03.952282+0000","last_became_active":"2024-09-19T07:53:02.949855+0000","last_became_peered":"2024-09-19T07:53:02.949855+0000","last_unstale":"2024-09-19T07:53:03.952282+0000","last_undegraded":"2024-09-19T07:53:03.952282+0000","last_fullsized":"2024-09-19T07:53:03.952282+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:22.009944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951518+0000","last_change":"2024-09-19T07:53:01.926136+0000","last_active":"2024-09-19T07:53:03.951518+0000","last_peered":"2024-09-19T07:53:03.951518+0000","last_clean":"2024-09-19T07:53:03.951518+0000","last_became_active":"2024-09-19T07:53:01.924857+0000","last_became_peered":"2024-09-19T07:53:01.924857+0000","last_unstale":"2024-09-19T07:53:03.951518+0000","last_undegraded":"2024-09-19T07:53:03.951518+0000","last_fullsized":"2024-09-19T07:53:03.951518+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:26.340346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952916+0000","last_change":"2024-09-19T07:53:02.956369+0000","last_active":"2024-09-19T07:53:03.952916+0000","last_peered":"2024-09-19T07:53:03.952916+0000","last_clean":"2024-09-19T07:53:03.952916+0000","last_became_active":"2024-09-19T07:53:02.956011+0000","last_became_peered":"2024-09-19T07:53:02.956011+0000","last_unstale":"2024-09-19T07:53:03.952916+0000","last_undegraded":"2024-09-19T07:53:03.952916+0000","last_fullsized":"2024-09-19T07:53:03.952916+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:21.471735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089161+0000","last_change":"2024-09-19T07:53:01.919358+0000","last_active":"2024-09-19T07:53:04.089161+0000","last_peered":"2024-09-19T07:53:04.089161+0000","last_clean":"2024-09-19T07:53:04.089161+0000","last_became_active":"2024-09-19T07:53:01.914884+0000","last_became_peered":"2024-09-19T07:53:01.914884+0000","last_unstale":"2024-09-19T07:53:04.089161+0000","last_undegraded":"2024-09-19T07:53:04.089161+0000","last_fullsized":"2024-09-19T07:53:04.089161+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:26.340346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.948238+0000","last_change":"2024-09-19T07:53:02.971101+0000","last_active":"2024-09-19T07:53:03.948238+0000","last_peered":"2024-09-19T07:53:03.948238+0000","last_clean":"2024-09-19T07:53:03.948238+0000","last_became_active":"2024-09-19T07:53:02.970767+0000","last_became_peered":"2024-09-19T07:53:02.970767+0000","last_unstale":"2024-09-19T07:53:03.948238+0000","last_undegraded":"2024-09-19T07:53:03.948238+0000","last_fullsized":"2024-09-19T07:53:03.948238+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:35:43.936759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089007+0000","last_change":"2024-09-19T07:53:01.923137+0000","last_active":"2024-09-19T07:53:04.089007+0000","last_peered":"2024-09-19T07:53:04.089007+0000","last_clean":"2024-09-19T07:53:04.089007+0000","last_became_active":"2024-09-19T07:53:01.922852+0000","last_became_peered":"2024-09-19T07:53:01.922852+0000","last_unstale":"2024-09-19T07:53:04.089007+0000","last_undegraded":"2024-09-19T07:53:04.089007+0000","last_fullsized":"2024-09-19T07:53:04.089007+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:31.183313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959859+0000","last_change":"2024-09-19T07:53:02.967452+0000","last_active":"2024-09-19T07:53:03.959859+0000","last_peered":"2024-09-19T07:53:03.959859+0000","last_clean":"2024-09-19T07:53:03.959859+0000","last_became_active":"2024-09-19T07:53:02.966768+0000","last_became_peered":"2024-09-19T07:53:02.966768+0000","last_unstale":"2024-09-19T07:53:03.959859+0000","last_undegraded":"2024-09-19T07:53:03.959859+0000","last_fullsized":"2024-09-19T07:53:03.959859+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:04.735526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952599+0000","last_change":"2024-09-19T07:53:01.919808+0000","last_active":"2024-09-19T07:53:03.952599+0000","last_peered":"2024-09-19T07:53:03.952599+0000","last_clean":"2024-09-19T07:53:03.952599+0000","last_became_active":"2024-09-19T07:53:01.919477+0000","last_became_peered":"2024-09-19T07:53:01.919477+0000","last_unstale":"2024-09-19T07:53:03.952599+0000","last_undegraded":"2024-09-19T07:53:03.952599+0000","last_fullsized":"2024-09-19T07:53:03.952599+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:21.008103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089819+0000","last_change":"2024-09-19T07:53:02.960753+0000","last_active":"2024-09-19T07:53:04.089819+0000","last_peered":"2024-09-19T07:53:04.089819+0000","last_clean":"2024-09-19T07:53:04.089819+0000","last_became_active":"2024-09-19T07:53:02.960465+0000","last_became_peered":"2024-09-19T07:53:02.960465+0000","last_unstale":"2024-09-19T07:53:04.089819+0000","last_undegraded":"2024-09-19T07:53:04.089819+0000","last_fullsized":"2024-09-19T07:53:04.089819+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:58.202929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951787+0000","last_change":"2024-09-19T07:53:01.916796+0000","last_active":"2024-09-19T07:53:03.951787+0000","last_peered":"2024-09-19T07:53:03.951787+0000","last_clean":"2024-09-19T07:53:03.951787+0000","last_became_active":"2024-09-19T07:53:01.916506+0000","last_became_peered":"2024-09-19T07:53:01.916506+0000","last_unstale":"2024-09-19T07:53:03.951787+0000","last_undegraded":"2024-09-19T07:53:03.951787+0000","last_fullsized":"2024-09-19T07:53:03.951787+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:50.457075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.948283+0000","last_change":"2024-09-19T07:53:02.963768+0000","last_active":"2024-09-19T07:53:03.948283+0000","last_peered":"2024-09-19T07:53:03.948283+0000","last_clean":"2024-09-19T07:53:03.948283+0000","last_became_active":"2024-09-19T07:53:02.963316+0000","last_became_peered":"2024-09-19T07:53:02.963316+0000","last_unstale":"2024-09-19T07:53:03.948283+0000","last_undegraded":"2024-09-19T07:53:03.948283+0000","last_fullsized":"2024-09-19T07:53:03.948283+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:23.087788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.947815+0000","last_change":"2024-09-19T07:53:01.922316+0000","last_active":"2024-09-19T07:53:03.947815+0000","last_peered":"2024-09-19T07:53:03.947815+0000","last_clean":"2024-09-19T07:53:03.947815+0000","last_became_active":"2024-09-19T07:53:01.922051+0000","last_became_peered":"2024-09-19T07:53:01.922051+0000","last_unstale":"2024-09-19T07:53:03.947815+0000","last_undegraded":"2024-09-19T07:53:03.947815+0000","last_fullsized":"2024-09-19T07:53:03.947815+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:20.469894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959407+0000","last_change":"2024-09-19T07:53:02.967631+0000","last_active":"2024-09-19T07:53:03.959407+0000","last_peered":"2024-09-19T07:53:03.959407+0000","last_clean":"2024-09-19T07:53:03.959407+0000","last_became_active":"2024-09-19T07:53:02.966419+0000","last_became_peered":"2024-09-19T07:53:02.966419+0000","last_unstale":"2024-09-19T07:53:03.959407+0000","last_undegraded":"2024-09-19T07:53:03.959407+0000","last_fullsized":"2024-09-19T07:53:03.959407+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:00.201893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.953111+0000","last_change":"2024-09-19T07:53:01.917284+0000","last_active":"2024-09-19T07:53:03.953111+0000","last_peered":"2024-09-19T07:53:03.953111+0000","last_clean":"2024-09-19T07:53:03.953111+0000","last_became_active":"2024-09-19T07:53:01.917007+0000","last_became_peered":"2024-09-19T07:53:01.917007+0000","last_unstale":"2024-09-19T07:53:03.953111+0000","last_undegraded":"2024-09-19T07:53:03.953111+0000","last_fullsized":"2024-09-19T07:53:03.953111+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:56.053928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088729+0000","last_change":"2024-09-19T07:53:02.951942+0000","last_active":"2024-09-19T07:53:04.088729+0000","last_peered":"2024-09-19T07:53:04.088729+0000","last_clean":"2024-09-19T07:53:04.088729+0000","last_became_active":"2024-09-19T07:53:02.950538+0000","last_became_peered":"2024-09-19T07:53:02.950538+0000","last_unstale":"2024-09-19T07:53:04.088729+0000","last_undegraded":"2024-09-19T07:53:04.088729+0000","last_fullsized":"2024-09-19T07:53:04.088729+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:31.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959653+0000","last_change":"2024-09-19T07:53:01.923243+0000","last_active":"2024-09-19T07:53:03.959653+0000","last_peered":"2024-09-19T07:53:03.959653+0000","last_clean":"2024-09-19T07:53:03.959653+0000","last_became_active":"2024-09-19T07:53:01.922939+0000","last_became_peered":"2024-09-19T07:53:01.922939+0000","last_unstale":"2024-09-19T07:53:03.959653+0000","last_undegraded":"2024-09-19T07:53:03.959653+0000","last_fullsized":"2024-09-19T07:53:03.959653+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:55.701057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088818+0000","last_change":"2024-09-19T07:53:02.960750+0000","last_active":"2024-09-19T07:53:04.088818+0000","last_peered":"2024-09-19T07:53:04.088818+0000","last_clean":"2024-09-19T07:53:04.088818+0000","last_became_active":"2024-09-19T07:53:02.960480+0000","last_became_peered":"2024-09-19T07:53:02.960480+0000","last_unstale":"2024-09-19T07:53:04.088818+0000","last_undegraded":"2024-09-19T07:53:04.088818+0000","last_fullsized":"2024-09-19T07:53:04.088818+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:39.185837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"105'3","reported_seq":17,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089046+0000","last_change":"2024-09-19T07:53:01.923641+0000","last_active":"2024-09-19T07:53:04.089046+0000","last_peered":"2024-09-19T07:53:04.089046+0000","last_clean":"2024-09-19T07:53:04.089046+0000","last_became_active":"2024-09-19T07:53:01.923379+0000","last_became_peered":"2024-09-19T07:53:01.923379+0000","last_unstale":"2024-09-19T07:53:04.089046+0000","last_undegraded":"2024-09-19T07:53:04.089046+0000","last_fullsized":"2024-09-19T07:53:04.089046+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:51:46.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089260+0000","last_change":"2024-09-19T07:53:02.953561+0000","last_active":"2024-09-19T07:53:04.089260+0000","last_peered":"2024-09-19T07:53:04.089260+0000","last_clean":"2024-09-19T07:53:04.089260+0000","last_became_active":"2024-09-19T07:53:02.953304+0000","last_became_peered":"2024-09-19T07:53:02.953304+0000","last_unstale":"2024-09-19T07:53:04.089260+0000","last_undegraded":"2024-09-19T07:53:04.089260+0000","last_fullsized":"2024-09-19T07:53:04.089260+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:43.470589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"105'6","reported_seq":20,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089486+0000","last_change":"2024-09-19T07:53:01.924833+0000","last_active":"2024-09-19T07:53:04.089486+0000","last_peered":"2024-09-19T07:53:04.089486+0000","last_clean":"2024-09-19T07:53:04.089486+0000","last_became_active":"2024-09-19T07:53:01.924568+0000","last_became_peered":"2024-09-19T07:53:01.924568+0000","last_unstale":"2024-09-19T07:53:04.089486+0000","last_undegraded":"2024-09-19T07:53:04.089486+0000","last_fullsized":"2024-09-19T07:53:04.089486+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:49.033408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952811+0000","last_change":"2024-09-19T07:53:02.955505+0000","last_active":"2024-09-19T07:53:03.952811+0000","last_peered":"2024-09-19T07:53:03.952811+0000","last_clean":"2024-09-19T07:53:03.952811+0000","last_became_active":"2024-09-19T07:53:02.955208+0000","last_became_peered":"2024-09-19T07:53:02.955208+0000","last_unstale":"2024-09-19T07:53:03.952811+0000","last_undegraded":"2024-09-19T07:53:03.952811+0000","last_fullsized":"2024-09-19T07:53:03.952811+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:56.130236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089356+0000","last_change":"2024-09-19T07:53:01.920568+0000","last_active":"2024-09-19T07:53:04.089356+0000","last_peered":"2024-09-19T07:53:04.089356+0000","last_clean":"2024-09-19T07:53:04.089356+0000","last_became_active":"2024-09-19T07:53:01.920282+0000","last_became_peered":"2024-09-19T07:53:01.920282+0000","last_unstale":"2024-09-19T07:53:04.089356+0000","last_undegraded":"2024-09-19T07:53:04.089356+0000","last_fullsized":"2024-09-19T07:53:04.089356+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:11.122190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089492+0000","last_change":"2024-09-19T07:53:02.963599+0000","last_active":"2024-09-19T07:53:04.089492+0000","last_peered":"2024-09-19T07:53:04.089492+0000","last_clean":"2024-09-19T07:53:04.089492+0000","last_became_active":"2024-09-19T07:53:02.963303+0000","last_became_peered":"2024-09-19T07:53:02.963303+0000","last_unstale":"2024-09-19T07:53:04.089492+0000","last_undegraded":"2024-09-19T07:53:04.089492+0000","last_fullsized":"2024-09-19T07:53:04.089492+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:42.406919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.948091+0000","last_change":"2024-09-19T07:53:01.919711+0000","last_active":"2024-09-19T07:53:03.948091+0000","last_peered":"2024-09-19T07:53:03.948091+0000","last_clean":"2024-09-19T07:53:03.948091+0000","last_became_active":"2024-09-19T07:53:01.919443+0000","last_became_peered":"2024-09-19T07:53:01.919443+0000","last_unstale":"2024-09-19T07:53:03.948091+0000","last_undegraded":"2024-09-19T07:53:03.948091+0000","last_fullsized":"2024-09-19T07:53:03.948091+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:38.351958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089167+0000","last_change":"2024-09-19T07:53:02.955371+0000","last_active":"2024-09-19T07:53:04.089167+0000","last_peered":"2024-09-19T07:53:04.089167+0000","last_clean":"2024-09-19T07:53:04.089167+0000","last_became_active":"2024-09-19T07:53:02.955075+0000","last_became_peered":"2024-09-19T07:53:02.955075+0000","last_unstale":"2024-09-19T07:53:04.089167+0000","last_undegraded":"2024-09-19T07:53:04.089167+0000","last_fullsized":"2024-09-19T07:53:04.089167+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:37:25.460499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088708+0000","last_change":"2024-09-19T07:53:01.918535+0000","last_active":"2024-09-19T07:53:04.088708+0000","last_peered":"2024-09-19T07:53:04.088708+0000","last_clean":"2024-09-19T07:53:04.088708+0000","last_became_active":"2024-09-19T07:53:01.918258+0000","last_became_peered":"2024-09-19T07:53:01.918258+0000","last_unstale":"2024-09-19T07:53:04.088708+0000","last_undegraded":"2024-09-19T07:53:04.088708+0000","last_fullsized":"2024-09-19T07:53:04.088708+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:33.989659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959932+0000","last_change":"2024-09-19T07:53:02.962652+0000","last_active":"2024-09-19T07:53:03.959932+0000","last_peered":"2024-09-19T07:53:03.959932+0000","last_clean":"2024-09-19T07:53:03.959932+0000","last_became_active":"2024-09-19T07:53:02.962394+0000","last_became_peered":"2024-09-19T07:53:02.962394+0000","last_unstale":"2024-09-19T07:53:03.959932+0000","last_undegraded":"2024-09-19T07:53:03.959932+0000","last_fullsized":"2024-09-19T07:53:03.959932+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:46.985872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"105'8","reported_seq":22,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951439+0000","last_change":"2024-09-19T07:53:01.918580+0000","last_active":"2024-09-19T07:53:03.951439+0000","last_peered":"2024-09-19T07:53:03.951439+0000","last_clean":"2024-09-19T07:53:03.951439+0000","last_became_active":"2024-09-19T07:53:01.918222+0000","last_became_peered":"2024-09-19T07:53:01.918222+0000","last_unstale":"2024-09-19T07:53:03.951439+0000","last_undegraded":"2024-09-19T07:53:03.951439+0000","last_fullsized":"2024-09-19T07:53:03.951439+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:06.785416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088884+0000","last_change":"2024-09-19T07:53:02.952071+0000","last_active":"2024-09-19T07:53:04.088884+0000","last_peered":"2024-09-19T07:53:04.088884+0000","last_clean":"2024-09-19T07:53:04.088884+0000","last_became_active":"2024-09-19T07:53:02.951783+0000","last_became_peered":"2024-09-19T07:53:02.951783+0000","last_unstale":"2024-09-19T07:53:04.088884+0000","last_undegraded":"2024-09-19T07:53:04.088884+0000","last_fullsized":"2024-09-19T07:53:04.088884+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:02.793228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089618+0000","last_change":"2024-09-19T07:53:01.922470+0000","last_active":"2024-09-19T07:53:04.089618+0000","last_peered":"2024-09-19T07:53:04.089618+0000","last_clean":"2024-09-19T07:53:04.089618+0000","last_became_active":"2024-09-19T07:53:01.922181+0000","last_became_peered":"2024-09-19T07:53:01.922181+0000","last_unstale":"2024-09-19T07:53:04.089618+0000","last_undegraded":"2024-09-19T07:53:04.089618+0000","last_fullsized":"2024-09-19T07:53:04.089618+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:47.737086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089090+0000","last_change":"2024-09-19T07:53:02.965849+0000","last_active":"2024-09-19T07:53:04.089090+0000","last_peered":"2024-09-19T07:53:04.089090+0000","last_clean":"2024-09-19T07:53:04.089090+0000","last_became_active":"2024-09-19T07:53:02.965562+0000","last_became_peered":"2024-09-19T07:53:02.965562+0000","last_unstale":"2024-09-19T07:53:04.089090+0000","last_undegraded":"2024-09-19T07:53:04.089090+0000","last_fullsized":"2024-09-19T07:53:04.089090+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:50.313345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089146+0000","last_change":"2024-09-19T07:53:01.922247+0000","last_active":"2024-09-19T07:53:04.089146+0000","last_peered":"2024-09-19T07:53:04.089146+0000","last_clean":"2024-09-19T07:53:04.089146+0000","last_became_active":"2024-09-19T07:53:01.921944+0000","last_became_peered":"2024-09-19T07:53:01.921944+0000","last_unstale":"2024-09-19T07:53:04.089146+0000","last_undegraded":"2024-09-19T07:53:04.089146+0000","last_fullsized":"2024-09-19T07:53:04.089146+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:40.569155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089176+0000","last_change":"2024-09-19T07:53:02.952785+0000","last_active":"2024-09-19T07:53:04.089176+0000","last_peered":"2024-09-19T07:53:04.089176+0000","last_clean":"2024-09-19T07:53:04.089176+0000","last_became_active":"2024-09-19T07:53:02.952535+0000","last_became_peered":"2024-09-19T07:53:02.952535+0000","last_unstale":"2024-09-19T07:53:04.089176+0000","last_undegraded":"2024-09-19T07:53:04.089176+0000","last_fullsized":"2024-09-19T07:53:04.089176+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:00.688869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952771+0000","last_change":"2024-09-19T07:53:01.925865+0000","last_active":"2024-09-19T07:53:03.952771+0000","last_peered":"2024-09-19T07:53:03.952771+0000","last_clean":"2024-09-19T07:53:03.952771+0000","last_became_active":"2024-09-19T07:53:01.925544+0000","last_became_peered":"2024-09-19T07:53:01.925544+0000","last_unstale":"2024-09-19T07:53:03.952771+0000","last_undegraded":"2024-09-19T07:53:03.952771+0000","last_fullsized":"2024-09-19T07:53:03.952771+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:57.201088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959514+0000","last_change":"2024-09-19T07:53:02.970190+0000","last_active":"2024-09-19T07:53:03.959514+0000","last_peered":"2024-09-19T07:53:03.959514+0000","last_clean":"2024-09-19T07:53:03.959514+0000","last_became_active":"2024-09-19T07:53:02.969897+0000","last_became_peered":"2024-09-19T07:53:02.969897+0000","last_unstale":"2024-09-19T07:53:03.959514+0000","last_undegraded":"2024-09-19T07:53:03.959514+0000","last_fullsized":"2024-09-19T07:53:03.959514+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:24.195258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089522+0000","last_change":"2024-09-19T07:53:01.926600+0000","last_active":"2024-09-19T07:53:04.089522+0000","last_peered":"2024-09-19T07:53:04.089522+0000","last_clean":"2024-09-19T07:53:04.089522+0000","last_became_active":"2024-09-19T07:53:01.925587+0000","last_became_peered":"2024-09-19T07:53:01.925587+0000","last_unstale":"2024-09-19T07:53:04.089522+0000","last_undegraded":"2024-09-19T07:53:04.089522+0000","last_fullsized":"2024-09-19T07:53:04.089522+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:13:51.412984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959770+0000","last_change":"2024-09-19T07:53:02.946678+0000","last_active":"2024-09-19T07:53:03.959770+0000","last_peered":"2024-09-19T07:53:03.959770+0000","last_clean":"2024-09-19T07:53:03.959770+0000","last_became_active":"2024-09-19T07:53:02.946357+0000","last_became_peered":"2024-09-19T07:53:02.946357+0000","last_unstale":"2024-09-19T07:53:03.959770+0000","last_undegraded":"2024-09-19T07:53:03.959770+0000","last_fullsized":"2024-09-19T07:53:03.959770+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:20.948125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088503+0000","last_change":"2024-09-19T07:53:01.926198+0000","last_active":"2024-09-19T07:53:04.088503+0000","last_peered":"2024-09-19T07:53:04.088503+0000","last_clean":"2024-09-19T07:53:04.088503+0000","last_became_active":"2024-09-19T07:53:01.925873+0000","last_became_peered":"2024-09-19T07:53:01.925873+0000","last_unstale":"2024-09-19T07:53:04.088503+0000","last_undegraded":"2024-09-19T07:53:04.088503+0000","last_fullsized":"2024-09-19T07:53:04.088503+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:23:00.870329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.948189+0000","last_change":"2024-09-19T07:53:02.965053+0000","last_active":"2024-09-19T07:53:03.948189+0000","last_peered":"2024-09-19T07:53:03.948189+0000","last_clean":"2024-09-19T07:53:03.948189+0000","last_became_active":"2024-09-19T07:53:02.964769+0000","last_became_peered":"2024-09-19T07:53:02.964769+0000","last_unstale":"2024-09-19T07:53:03.948189+0000","last_undegraded":"2024-09-19T07:53:03.948189+0000","last_fullsized":"2024-09-19T07:53:03.948189+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:58.660852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089316+0000","last_change":"2024-09-19T07:53:01.923054+0000","last_active":"2024-09-19T07:53:04.089316+0000","last_peered":"2024-09-19T07:53:04.089316+0000","last_clean":"2024-09-19T07:53:04.089316+0000","last_became_active":"2024-09-19T07:53:01.922799+0000","last_became_peered":"2024-09-19T07:53:01.922799+0000","last_unstale":"2024-09-19T07:53:04.089316+0000","last_undegraded":"2024-09-19T07:53:04.089316+0000","last_fullsized":"2024-09-19T07:53:04.089316+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:47.745785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089660+0000","last_change":"2024-09-19T07:53:02.957041+0000","last_active":"2024-09-19T07:53:04.089660+0000","last_peered":"2024-09-19T07:53:04.089660+0000","last_clean":"2024-09-19T07:53:04.089660+0000","last_became_active":"2024-09-19T07:53:02.956729+0000","last_became_peered":"2024-09-19T07:53:02.956729+0000","last_unstale":"2024-09-19T07:53:04.089660+0000","last_undegraded":"2024-09-19T07:53:04.089660+0000","last_fullsized":"2024-09-19T07:53:04.089660+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:24:06.262792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088870+0000","last_change":"2024-09-19T07:53:01.923097+0000","last_active":"2024-09-19T07:53:04.088870+0000","last_peered":"2024-09-19T07:53:04.088870+0000","last_clean":"2024-09-19T07:53:04.088870+0000","last_became_active":"2024-09-19T07:53:01.922794+0000","last_became_peered":"2024-09-19T07:53:01.922794+0000","last_unstale":"2024-09-19T07:53:04.088870+0000","last_undegraded":"2024-09-19T07:53:04.088870+0000","last_fullsized":"2024-09-19T07:53:04.088870+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:29.994651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.959382+0000","last_change":"2024-09-19T07:53:02.968250+0000","last_active":"2024-09-19T07:53:03.959382+0000","last_peered":"2024-09-19T07:53:03.959382+0000","last_clean":"2024-09-19T07:53:03.959382+0000","last_became_active":"2024-09-19T07:53:02.967961+0000","last_became_peered":"2024-09-19T07:53:02.967961+0000","last_unstale":"2024-09-19T07:53:03.959382+0000","last_undegraded":"2024-09-19T07:53:03.959382+0000","last_fullsized":"2024-09-19T07:53:03.959382+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:03.803861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089294+0000","last_change":"2024-09-19T07:53:01.922110+0000","last_active":"2024-09-19T07:53:04.089294+0000","last_peered":"2024-09-19T07:53:04.089294+0000","last_clean":"2024-09-19T07:53:04.089294+0000","last_became_active":"2024-09-19T07:53:01.921769+0000","last_became_peered":"2024-09-19T07:53:01.921769+0000","last_unstale":"2024-09-19T07:53:04.089294+0000","last_undegraded":"2024-09-19T07:53:04.089294+0000","last_fullsized":"2024-09-19T07:53:04.089294+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:56.053928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951681+0000","last_change":"2024-09-19T07:53:02.963616+0000","last_active":"2024-09-19T07:53:03.951681+0000","last_peered":"2024-09-19T07:53:03.951681+0000","last_clean":"2024-09-19T07:53:03.951681+0000","last_became_active":"2024-09-19T07:53:02.963333+0000","last_became_peered":"2024-09-19T07:53:02.963333+0000","last_unstale":"2024-09-19T07:53:03.951681+0000","last_undegraded":"2024-09-19T07:53:03.951681+0000","last_fullsized":"2024-09-19T07:53:03.951681+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"106'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.961104+0000","last_change":"2024-09-19T07:53:01.922601+0000","last_active":"2024-09-19T07:53:03.961104+0000","last_peered":"2024-09-19T07:53:03.961104+0000","last_clean":"2024-09-19T07:53:03.961104+0000","last_became_active":"2024-09-19T07:53:01.922324+0000","last_became_peered":"2024-09-19T07:53:01.922324+0000","last_unstale":"2024-09-19T07:53:03.961104+0000","last_undegraded":"2024-09-19T07:53:03.961104+0000","last_fullsized":"2024-09-19T07:53:03.961104+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:02.428894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.947991+0000","last_change":"2024-09-19T07:53:02.961218+0000","last_active":"2024-09-19T07:53:03.947991+0000","last_peered":"2024-09-19T07:53:03.947991+0000","last_clean":"2024-09-19T07:53:03.947991+0000","last_became_active":"2024-09-19T07:53:02.960873+0000","last_became_peered":"2024-09-19T07:53:02.960873+0000","last_unstale":"2024-09-19T07:53:03.947991+0000","last_undegraded":"2024-09-19T07:53:03.947991+0000","last_fullsized":"2024-09-19T07:53:03.947991+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:03.430735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089482+0000","last_change":"2024-09-19T07:53:01.920744+0000","last_active":"2024-09-19T07:53:04.089482+0000","last_peered":"2024-09-19T07:53:04.089482+0000","last_clean":"2024-09-19T07:53:04.089482+0000","last_became_active":"2024-09-19T07:53:01.920476+0000","last_became_peered":"2024-09-19T07:53:01.920476+0000","last_unstale":"2024-09-19T07:53:04.089482+0000","last_undegraded":"2024-09-19T07:53:04.089482+0000","last_fullsized":"2024-09-19T07:53:04.089482+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:11.662918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952379+0000","last_change":"2024-09-19T07:53:02.961727+0000","last_active":"2024-09-19T07:53:03.952379+0000","last_peered":"2024-09-19T07:53:03.952379+0000","last_clean":"2024-09-19T07:53:03.952379+0000","last_became_active":"2024-09-19T07:53:02.961447+0000","last_became_peered":"2024-09-19T07:53:02.961447+0000","last_unstale":"2024-09-19T07:53:03.952379+0000","last_undegraded":"2024-09-19T07:53:03.952379+0000","last_fullsized":"2024-09-19T07:53:03.952379+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:26:55.837328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089205+0000","last_change":"2024-09-19T07:53:01.922583+0000","last_active":"2024-09-19T07:53:04.089205+0000","last_peered":"2024-09-19T07:53:04.089205+0000","last_clean":"2024-09-19T07:53:04.089205+0000","last_became_active":"2024-09-19T07:53:01.922318+0000","last_became_peered":"2024-09-19T07:53:01.922318+0000","last_unstale":"2024-09-19T07:53:04.089205+0000","last_undegraded":"2024-09-19T07:53:04.089205+0000","last_fullsized":"2024-09-19T07:53:04.089205+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:14.312065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089440+0000","last_change":"2024-09-19T07:53:02.957752+0000","last_active":"2024-09-19T07:53:04.089440+0000","last_peered":"2024-09-19T07:53:04.089440+0000","last_clean":"2024-09-19T07:53:04.089440+0000","last_became_active":"2024-09-19T07:53:02.957469+0000","last_became_peered":"2024-09-19T07:53:02.957469+0000","last_unstale":"2024-09-19T07:53:04.089440+0000","last_undegraded":"2024-09-19T07:53:04.089440+0000","last_fullsized":"2024-09-19T07:53:04.089440+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:18.785138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.952032+0000","last_change":"2024-09-19T07:53:01.918087+0000","last_active":"2024-09-19T07:53:03.952032+0000","last_peered":"2024-09-19T07:53:03.952032+0000","last_clean":"2024-09-19T07:53:03.952032+0000","last_became_active":"2024-09-19T07:53:01.917738+0000","last_became_peered":"2024-09-19T07:53:01.917738+0000","last_unstale":"2024-09-19T07:53:03.952032+0000","last_undegraded":"2024-09-19T07:53:03.952032+0000","last_fullsized":"2024-09-19T07:53:03.952032+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:14.312065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089339+0000","last_change":"2024-09-19T07:53:02.961489+0000","last_active":"2024-09-19T07:53:04.089339+0000","last_peered":"2024-09-19T07:53:04.089339+0000","last_clean":"2024-09-19T07:53:04.089339+0000","last_became_active":"2024-09-19T07:53:02.961212+0000","last_became_peered":"2024-09-19T07:53:02.961212+0000","last_unstale":"2024-09-19T07:53:04.089339+0000","last_undegraded":"2024-09-19T07:53:04.089339+0000","last_fullsized":"2024-09-19T07:53:04.089339+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:15.778175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088861+0000","last_change":"2024-09-19T07:53:01.920116+0000","last_active":"2024-09-19T07:53:04.088861+0000","last_peered":"2024-09-19T07:53:04.088861+0000","last_clean":"2024-09-19T07:53:04.088861+0000","last_became_active":"2024-09-19T07:53:01.919834+0000","last_became_peered":"2024-09-19T07:53:01.919834+0000","last_unstale":"2024-09-19T07:53:04.088861+0000","last_undegraded":"2024-09-19T07:53:04.088861+0000","last_fullsized":"2024-09-19T07:53:04.088861+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:18.763151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089415+0000","last_change":"2024-09-19T07:53:02.957282+0000","last_active":"2024-09-19T07:53:04.089415+0000","last_peered":"2024-09-19T07:53:04.089415+0000","last_clean":"2024-09-19T07:53:04.089415+0000","last_became_active":"2024-09-19T07:53:02.957004+0000","last_became_peered":"2024-09-19T07:53:02.957004+0000","last_unstale":"2024-09-19T07:53:04.089415+0000","last_undegraded":"2024-09-19T07:53:04.089415+0000","last_fullsized":"2024-09-19T07:53:04.089415+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:25:45.688277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089434+0000","last_change":"2024-09-19T07:53:01.921607+0000","last_active":"2024-09-19T07:53:04.089434+0000","last_peered":"2024-09-19T07:53:04.089434+0000","last_clean":"2024-09-19T07:53:04.089434+0000","last_became_active":"2024-09-19T07:53:01.920937+0000","last_became_peered":"2024-09-19T07:53:01.920937+0000","last_unstale":"2024-09-19T07:53:04.089434+0000","last_undegraded":"2024-09-19T07:53:04.089434+0000","last_fullsized":"2024-09-19T07:53:04.089434+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:35.664386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.953006+0000","last_change":"2024-09-19T07:53:02.956981+0000","last_active":"2024-09-19T07:53:03.953006+0000","last_peered":"2024-09-19T07:53:03.953006+0000","last_clean":"2024-09-19T07:53:03.953006+0000","last_became_active":"2024-09-19T07:53:02.955937+0000","last_became_peered":"2024-09-19T07:53:02.955937+0000","last_unstale":"2024-09-19T07:53:03.953006+0000","last_undegraded":"2024-09-19T07:53:03.953006+0000","last_fullsized":"2024-09-19T07:53:03.953006+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:39.353799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088926+0000","last_change":"2024-09-19T07:53:01.924267+0000","last_active":"2024-09-19T07:53:04.088926+0000","last_peered":"2024-09-19T07:53:04.088926+0000","last_clean":"2024-09-19T07:53:04.088926+0000","last_became_active":"2024-09-19T07:53:01.924001+0000","last_became_peered":"2024-09-19T07:53:01.924001+0000","last_unstale":"2024-09-19T07:53:04.088926+0000","last_undegraded":"2024-09-19T07:53:04.088926+0000","last_fullsized":"2024-09-19T07:53:04.088926+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:15.069406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:03.951881+0000","last_change":"2024-09-19T07:53:02.955750+0000","last_active":"2024-09-19T07:53:03.951881+0000","last_peered":"2024-09-19T07:53:03.951881+0000","last_clean":"2024-09-19T07:53:03.951881+0000","last_became_active":"2024-09-19T07:53:02.955250+0000","last_became_peered":"2024-09-19T07:53:02.955250+0000","last_unstale":"2024-09-19T07:53:03.951881+0000","last_undegraded":"2024-09-19T07:53:03.951881+0000","last_fullsized":"2024-09-19T07:53:03.951881+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:01.900875+0000","last_clean_scrub_stamp":"2024-09-19T07:53:01.900875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:22.292252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.089547+0000","last_change":"2024-09-19T07:53:01.921714+0000","last_active":"2024-09-19T07:53:04.089547+0000","last_peered":"2024-09-19T07:53:04.089547+0000","last_clean":"2024-09-19T07:53:04.089547+0000","last_became_active":"2024-09-19T07:53:01.921436+0000","last_became_peered":"2024-09-19T07:53:01.921436+0000","last_unstale":"2024-09-19T07:53:04.089547+0000","last_undegraded":"2024-09-19T07:53:04.089547+0000","last_fullsized":"2024-09-19T07:53:04.089547+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:00.899035+0000","last_clean_scrub_stamp":"2024-09-19T07:53:00.899035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:04.601086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":325,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-19T07:53:04.088582+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:53:04.088582+0000","last_peered":"2024-09-19T07:53:04.088582+0000","last_clean":"2024-09-19T07:53:04.088582+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:53:04.088582+0000","last_undegraded":"2024-09-19T07:53:04.088582+0000","last_fullsized":"2024-09-19T07:53:04.088582+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509566,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82976,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288864,"statfs":{"total":96636764160,"available":96551796736,"internally_reserved":0,"allocated":1908736,"data_stored":1195478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509565,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100520,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":94271320,"statfs":{"total":96636764160,"available":96533831680,"internally_reserved":0,"allocated":1327104,"data_stored":607166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":101579206},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509566,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50512,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321328,"statfs":{"total":96636764160,"available":96585039872,"internally_reserved":0,"allocated":1306624,"data_stored":604950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509566,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86552,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285288,"statfs":{"total":96636764160,"available":96548134912,"internally_reserved":0,"allocated":1314816,"data_stored":605042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509565,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":101036,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94270804,"statfs":{"total":96636764160,"available":96533303296,"internally_reserved":0,"allocated":1916928,"data_stored":1197020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542269,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135124,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236716,"statfs":{"total":96636764160,"available":96498397184,"internally_reserved":0,"allocated":1310720,"data_stored":605532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":137034183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542269,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94932,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94276908,"statfs":{"total":96636764160,"available":96539553792,"internally_reserved":0,"allocated":1310720,"data_stored":604984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574974,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104600,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267240,"statfs":{"total":96636764160,"available":96529653760,"internally_reserved":0,"allocated":1310720,"data_stored":604984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:53:10.683 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T07:53:10.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T07:53:10.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.828+0000 7f1c03b03640 1 Processor -- start 2024-09-19T07:53:10.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.828+0000 7f1c03b03640 1 -- start start 2024-09-19T07:53:10.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.829+0000 7f1c03b03640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1bfc15e390 0x7f1bfc071d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:10.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.829+0000 7f1c03b03640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bfc06e730 0x7f1bfc06eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:10.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.829+0000 7f1c03b03640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bfc0ce1d0 con 0x7f1bfc0b4260 2024-09-19T07:53:10.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.829+0000 7f1c03b03640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f1bfc0cdf80 con 0x7f1bfc15e390 2024-09-19T07:53:10.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.829+0000 7f1c03b03640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1bfc0ce4b0 con 0x7f1bfc06e730 2024-09-19T07:53:10.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.829+0000 7f1c020a0640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1bfc0b4260 0x7f1bfc0b46d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60182/0 (socket says 172.21.15.16:60182) 2024-09-19T07:53:10.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.829+0000 7f1c020a0640 1 -- 172.21.15.16:0/1238144846 learned_addr learned my addr 172.21.15.16:0/1238144846 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:10.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.829+0000 7f1c0109e640 1 --2- 172.21.15.16:0/1238144846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bfc06e730 0x7f1bfc06eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:10.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c0189f640 1 --2- 172.21.15.16:0/1238144846 >> v2:172.21.15.146:3300/0 conn(0x7f1bfc15e390 0x7f1bfc071d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:10.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c0089d640 1 -- 172.21.15.16:0/1238144846 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 117544250 0 0) 0x7f1bfc0ce1d0 con 0x7f1bfc0b4260 2024-09-19T07:53:10.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c0089d640 1 -- 172.21.15.16:0/1238144846 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1be8003520 con 0x7f1bfc0b4260 2024-09-19T07:53:10.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c0089d640 1 -- 172.21.15.16:0/1238144846 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 844301751 0 0) 0x7f1be8003520 con 0x7f1bfc0b4260 2024-09-19T07:53:10.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c0089d640 1 -- 172.21.15.16:0/1238144846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bfc06e730 msgr2=0x7f1bfc06eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:10.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c0089d640 1 --2- 172.21.15.16:0/1238144846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bfc06e730 0x7f1bfc06eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c0089d640 1 -- 172.21.15.16:0/1238144846 >> v2:172.21.15.146:3300/0 conn(0x7f1bfc15e390 msgr2=0x7f1bfc071d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:10.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c0089d640 1 --2- 172.21.15.16:0/1238144846 >> v2:172.21.15.146:3300/0 conn(0x7f1bfc15e390 0x7f1bfc071d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c0089d640 1 -- 172.21.15.16:0/1238144846 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bfc0ce790 con 0x7f1bfc0b4260 2024-09-19T07:53:10.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c0089d640 1 -- 172.21.15.16:0/1238144846 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f1bf40019f0 con 0x7f1bfc0b4260 2024-09-19T07:53:10.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c0089d640 1 -- 172.21.15.16:0/1238144846 >> v1:172.21.15.16:6789/0 conn(0x7f1bfc0b4260 legacy=0x7f1bfc0b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:10.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c0089d640 1 --2- 172.21.15.16:0/1238144846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1be80039b0 0x7f1be8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:10.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c0089d640 1 --2- 172.21.15.16:0/1238144846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bfc06e730 0x7f1be8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:10.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c0089d640 1 --2- 172.21.15.16:0/1238144846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be8004d40 0x7f1be80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:10.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c0089d640 1 -- 172.21.15.16:0/1238144846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc0cdf80 con 0x7f1be8004d40 2024-09-19T07:53:10.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c0089d640 1 -- 172.21.15.16:0/1238144846 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc0ce4b0 con 0x7f1be80039b0 2024-09-19T07:53:10.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c0089d640 1 -- 172.21.15.16:0/1238144846 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1be800b9c0 con 0x7f1bfc06e730 2024-09-19T07:53:10.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c020a0640 1 --2- 172.21.15.16:0/1238144846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be8004d40 0x7f1be80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:10.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c0189f640 1 --2- 172.21.15.16:0/1238144846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1be80039b0 0x7f1be8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:10.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c0109e640 1 --2- 172.21.15.16:0/1238144846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bfc06e730 0x7f1be8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:10.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c020a0640 1 -- 172.21.15.16:0/1238144846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bfc06e730 msgr2=0x7f1be8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:10.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c020a0640 1 --2- 172.21.15.16:0/1238144846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bfc06e730 0x7f1be8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c020a0640 1 -- 172.21.15.16:0/1238144846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1be80039b0 msgr2=0x7f1be8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:10.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c020a0640 1 --2- 172.21.15.16:0/1238144846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1be80039b0 0x7f1be8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.830+0000 7f1c020a0640 1 -- 172.21.15.16:0/1238144846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be8003520 con 0x7f1be8004d40 2024-09-19T07:53:10.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.831+0000 7f1c020a0640 1 --2- 172.21.15.16:0/1238144846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be8004d40 0x7f1be80051c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400a920 tx=0x7f1be8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4255cae793a2680f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:10.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.831+0000 7f1c0089d640 1 -- 172.21.15.16:0/1238144846 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf400a4c0 con 0x7f1be8004d40 2024-09-19T07:53:10.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.831+0000 7f1c0089d640 1 -- 172.21.15.16:0/1238144846 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bf402f070 con 0x7f1be8004d40 2024-09-19T07:53:10.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.831+0000 7f1c0089d640 1 -- 172.21.15.16:0/1238144846 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf402bd00 con 0x7f1be8004d40 2024-09-19T07:53:10.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.832+0000 7f1c03b03640 1 -- 172.21.15.16:0/1238144846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be8004d40 msgr2=0x7f1be80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:10.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.832+0000 7f1c03b03640 1 --2- 172.21.15.16:0/1238144846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be8004d40 0x7f1be80051c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400a920 tx=0x7f1be8005830 comp rx=0 tx=0).stop 2024-09-19T07:53:10.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.832+0000 7f1c020a0640 1 -- 172.21.15.16:0/1238144846 reap_dead start 2024-09-19T07:53:10.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.832+0000 7f1c03b03640 1 -- 172.21.15.16:0/1238144846 shutdown_connections 2024-09-19T07:53:10.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.832+0000 7f1c03b03640 1 -- 172.21.15.16:0/1238144846 >> 172.21.15.16:0/1238144846 conn(0x7f1bfc0b1c70 msgr2=0x7f1bfc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:10.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.832+0000 7f1c03b03640 1 -- 172.21.15.16:0/1238144846 shutdown_connections 2024-09-19T07:53:10.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.832+0000 7f1c03b03640 1 -- 172.21.15.16:0/1238144846 wait complete. 2024-09-19T07:53:10.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.833+0000 7f1c03b03640 1 Processor -- start 2024-09-19T07:53:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.833+0000 7f1c03b03640 1 -- start start 2024-09-19T07:53:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.834+0000 7f1c03b03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be8004d40 0x7f1bfc159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.834+0000 7f1c03b03640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bfc15b630 0x7f1bfc15bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.834+0000 7f1c03b03640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bfc161900 0x7f1bfc161d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.834+0000 7f1c03b03640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc072250 con 0x7f1be8004d40 2024-09-19T07:53:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.834+0000 7f1c03b03640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc0ced50 con 0x7f1bfc161900 2024-09-19T07:53:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.834+0000 7f1c03b03640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1bfc0723d0 con 0x7f1bfc15b630 2024-09-19T07:53:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.834+0000 7f1c0189f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be8004d40 0x7f1bfc159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.834+0000 7f1c0189f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be8004d40 0x7f1bfc159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60838/0 (socket says 172.21.15.16:60838) 2024-09-19T07:53:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.834+0000 7f1c0189f640 1 -- 172.21.15.16:0/1090756750 learned_addr learned my addr 172.21.15.16:0/1090756750 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.834+0000 7f1c020a0640 1 --2- 172.21.15.16:0/1090756750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bfc161900 0x7f1bfc161d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.834+0000 7f1c0109e640 1 --2- 172.21.15.16:0/1090756750 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bfc15b630 0x7f1bfc15bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.834+0000 7f1c0189f640 1 -- 172.21.15.16:0/1090756750 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bfc15b630 msgr2=0x7f1bfc15bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:10.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.834+0000 7f1c0189f640 1 --2- 172.21.15.16:0/1090756750 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bfc15b630 0x7f1bfc15bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.834+0000 7f1c0189f640 1 -- 172.21.15.16:0/1090756750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bfc161900 msgr2=0x7f1bfc161d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:10.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.834+0000 7f1c0189f640 1 --2- 172.21.15.16:0/1090756750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bfc161900 0x7f1bfc161d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.834+0000 7f1c0189f640 1 -- 172.21.15.16:0/1090756750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bfc15f6a0 con 0x7f1be8004d40 2024-09-19T07:53:10.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.835+0000 7f1c0189f640 1 --2- 172.21.15.16:0/1090756750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be8004d40 0x7f1bfc159c30 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8011960 tx=0x7f1bf8011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:10.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.835+0000 7f1bf27fc640 1 -- 172.21.15.16:0/1090756750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf800faf0 con 0x7f1be8004d40 2024-09-19T07:53:10.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.835+0000 7f1bf27fc640 1 -- 172.21.15.16:0/1090756750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bf800fc90 con 0x7f1be8004d40 2024-09-19T07:53:10.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.835+0000 7f1c03b03640 1 -- 172.21.15.16:0/1090756750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bfc1622a0 con 0x7f1be8004d40 2024-09-19T07:53:10.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.835+0000 7f1bf27fc640 1 -- 172.21.15.16:0/1090756750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8014440 con 0x7f1be8004d40 2024-09-19T07:53:10.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.835+0000 7f1c03b03640 1 -- 172.21.15.16:0/1090756750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bfc1f4250 con 0x7f1be8004d40 2024-09-19T07:53:10.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.836+0000 7f1c03b03640 1 -- 172.21.15.16:0/1090756750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bfc15e770 con 0x7f1be8004d40 2024-09-19T07:53:10.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.836+0000 7f1bf27fc640 1 -- 172.21.15.16:0/1090756750 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1bf8012040 con 0x7f1be8004d40 2024-09-19T07:53:10.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.837+0000 7f1bf27fc640 1 --2- 172.21.15.16:0/1090756750 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1be006c2b0 0x7f1be006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:10.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.837+0000 7f1bf27fc640 1 -- 172.21.15.16:0/1090756750 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f1bf808bc60 con 0x7f1be8004d40 2024-09-19T07:53:10.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.837+0000 7f1c0109e640 1 --2- 172.21.15.16:0/1090756750 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1be006c2b0 0x7f1be006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:10.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.837+0000 7f1c0109e640 1 --2- 172.21.15.16:0/1090756750 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1be006c2b0 0x7f1be006e770 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc09c160 tx=0x7f1bec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:10.841+0000 7f1bf27fc640 1 -- 172.21.15.16:0/1090756750 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1bf8019030 con 0x7f1be8004d40 2024-09-19T07:53:11.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.089+0000 7f1c03b03640 1 -- 172.21.15.16:0/1090756750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f1bfc15d4e0 con 0x7f1be8004d40 2024-09-19T07:53:11.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.091+0000 7f1bf27fc640 1 -- 172.21.15.16:0/1090756750 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f1bf8059bf0 con 0x7f1be8004d40 2024-09-19T07:53:11.092 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-19T07:53:11.092 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA== 2024-09-19T07:53:11.092 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-19T07:53:11.092 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-19T07:53:11.092 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T07:53:11.092 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-19T07:53:11.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.094+0000 7f1c03b03640 1 -- 172.21.15.16:0/1090756750 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1be006c2b0 msgr2=0x7f1be006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:11.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.094+0000 7f1c03b03640 1 --2- 172.21.15.16:0/1090756750 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1be006c2b0 0x7f1be006e770 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc09c160 tx=0x7f1bec008040 comp rx=0 tx=0).stop 2024-09-19T07:53:11.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.094+0000 7f1c03b03640 1 -- 172.21.15.16:0/1090756750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be8004d40 msgr2=0x7f1bfc159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:11.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.094+0000 7f1c03b03640 1 --2- 172.21.15.16:0/1090756750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be8004d40 0x7f1bfc159c30 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8011960 tx=0x7f1bf8011e20 comp rx=0 tx=0).stop 2024-09-19T07:53:11.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.094+0000 7f1c03b03640 1 -- 172.21.15.16:0/1090756750 shutdown_connections 2024-09-19T07:53:11.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.094+0000 7f1c03b03640 1 --2- 172.21.15.16:0/1090756750 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1be006c2b0 0x7f1be006e770 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.094+0000 7f1c03b03640 1 --2- 172.21.15.16:0/1090756750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bfc161900 0x7f1bfc161d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.094+0000 7f1c03b03640 1 --2- 172.21.15.16:0/1090756750 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bfc15b630 0x7f1bfc15bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.094+0000 7f1c03b03640 1 --2- 172.21.15.16:0/1090756750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be8004d40 0x7f1bfc159c30 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.094+0000 7f1c03b03640 1 -- 172.21.15.16:0/1090756750 >> 172.21.15.16:0/1090756750 conn(0x7f1bfc0b1c70 msgr2=0x7f1bfc162680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:11.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.094+0000 7f1c03b03640 1 -- 172.21.15.16:0/1090756750 shutdown_connections 2024-09-19T07:53:11.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.094+0000 7f1c03b03640 1 -- 172.21.15.16:0/1090756750 wait complete. 2024-09-19T07:53:11.105 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:53:11.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.207+0000 7fe8b7f6d640 1 Processor -- start 2024-09-19T07:53:11.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.207+0000 7fe8b7f6d640 1 -- start start 2024-09-19T07:53:11.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.208+0000 7fe8b7f6d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe8b00d0400 0x7fe8b006fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.208+0000 7fe8b7f6d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8b015f150 con 0x7fe8b015ea20 2024-09-19T07:53:11.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.208+0000 7fe8b7f6d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8b006ebb0 con 0x7fe8b006e720 2024-09-19T07:53:11.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.208+0000 7fe8b7f6d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe8b006ee90 con 0x7fe8b00d0400 2024-09-19T07:53:11.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.208+0000 7fe8b650a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe8b015ea20 0x7fe8b00ceed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60184/0 (socket says 172.21.15.16:60184) 2024-09-19T07:53:11.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.208+0000 7fe8b650a640 1 -- 172.21.15.16:0/1141883064 learned_addr learned my addr 172.21.15.16:0/1141883064 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:11.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.208+0000 7fe8b5d09640 1 --2- 172.21.15.16:0/1141883064 >> v2:172.21.15.146:3300/0 conn(0x7fe8b00d0400 0x7fe8b006fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.208+0000 7fe8b4d07640 1 -- 172.21.15.16:0/1141883064 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3588560727 0 0) 0x7fe8b015f150 con 0x7fe8b015ea20 2024-09-19T07:53:11.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.208+0000 7fe8b4d07640 1 -- 172.21.15.16:0/1141883064 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe89c003520 con 0x7fe8b015ea20 2024-09-19T07:53:11.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b4d07640 1 -- 172.21.15.16:0/1141883064 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1566477300 0 0) 0x7fe89c003520 con 0x7fe8b015ea20 2024-09-19T07:53:11.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b4d07640 1 -- 172.21.15.16:0/1141883064 >> v2:172.21.15.146:3300/0 conn(0x7fe8b00d0400 msgr2=0x7fe8b006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:11.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b4d07640 1 --2- 172.21.15.16:0/1141883064 >> v2:172.21.15.146:3300/0 conn(0x7fe8b00d0400 0x7fe8b006fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b4d07640 1 -- 172.21.15.16:0/1141883064 >> v1:172.21.15.146:6789/0 conn(0x7fe8b006e720 legacy=0x7fe8b015e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:11.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b4d07640 1 -- 172.21.15.16:0/1141883064 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b006f170 con 0x7fe8b015ea20 2024-09-19T07:53:11.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b4d07640 1 -- 172.21.15.16:0/1141883064 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fe8a8001a10 con 0x7fe8b015ea20 2024-09-19T07:53:11.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b4d07640 1 -- 172.21.15.16:0/1141883064 >> v1:172.21.15.16:6789/0 conn(0x7fe8b015ea20 legacy=0x7fe8b00ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:11.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b4d07640 1 --2- 172.21.15.16:0/1141883064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe89c0039b0 0x7fe89c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b4d07640 1 --2- 172.21.15.16:0/1141883064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe89c004800 0x7fe89c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b4d07640 1 --2- 172.21.15.16:0/1141883064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe89c0051c0 0x7fe89c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b4d07640 1 -- 172.21.15.16:0/1141883064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b006ee90 con 0x7fe89c0039b0 2024-09-19T07:53:11.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b4d07640 1 -- 172.21.15.16:0/1141883064 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c00c160 con 0x7fe89c004800 2024-09-19T07:53:11.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b4d07640 1 -- 172.21.15.16:0/1141883064 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe89c00c360 con 0x7fe89c0051c0 2024-09-19T07:53:11.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b5d09640 1 --2- 172.21.15.16:0/1141883064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe89c0039b0 0x7fe89c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b5508640 1 --2- 172.21.15.16:0/1141883064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe89c004800 0x7fe89c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b650a640 1 --2- 172.21.15.16:0/1141883064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe89c0051c0 0x7fe89c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b5d09640 1 -- 172.21.15.16:0/1141883064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe89c0051c0 msgr2=0x7fe89c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:11.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b5d09640 1 --2- 172.21.15.16:0/1141883064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe89c0051c0 0x7fe89c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b5d09640 1 -- 172.21.15.16:0/1141883064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe89c004800 msgr2=0x7fe89c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:11.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b5d09640 1 --2- 172.21.15.16:0/1141883064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe89c004800 0x7fe89c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b5d09640 1 -- 172.21.15.16:0/1141883064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89c003520 con 0x7fe89c0039b0 2024-09-19T07:53:11.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b5508640 1 --2- 172.21.15.16:0/1141883064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe89c004800 0x7fe89c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:11.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.209+0000 7fe8b650a640 1 -- 172.21.15.16:0/1141883064 reap_dead start 2024-09-19T07:53:11.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.210+0000 7fe8b5d09640 1 --2- 172.21.15.16:0/1141883064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe89c0039b0 0x7fe89c003dd0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00bd60 tx=0x7fe89c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2aad9c412d5031d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:11.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.210+0000 7fe8b4d07640 1 -- 172.21.15.16:0/1141883064 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac03a070 con 0x7fe89c0039b0 2024-09-19T07:53:11.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.210+0000 7fe8b4d07640 1 -- 172.21.15.16:0/1141883064 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8ac035440 con 0x7fe89c0039b0 2024-09-19T07:53:11.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.210+0000 7fe8b4d07640 1 -- 172.21.15.16:0/1141883064 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac00b930 con 0x7fe89c0039b0 2024-09-19T07:53:11.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.210+0000 7fe8b7f6d640 1 -- 172.21.15.16:0/1141883064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe89c0039b0 msgr2=0x7fe89c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:11.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.210+0000 7fe8b7f6d640 1 --2- 172.21.15.16:0/1141883064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe89c0039b0 0x7fe89c003dd0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00bd60 tx=0x7fe89c005830 comp rx=0 tx=0).stop 2024-09-19T07:53:11.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.211+0000 7fe8b7f6d640 1 -- 172.21.15.16:0/1141883064 shutdown_connections 2024-09-19T07:53:11.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.211+0000 7fe8b7f6d640 1 --2- 172.21.15.16:0/1141883064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe89c0039b0 0x7fe89c003dd0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.211+0000 7fe8b7f6d640 1 -- 172.21.15.16:0/1141883064 >> 172.21.15.16:0/1141883064 conn(0x7fe8b006bc10 msgr2=0x7fe8b006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:11.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.211+0000 7fe8b7f6d640 1 -- 172.21.15.16:0/1141883064 shutdown_connections 2024-09-19T07:53:11.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.211+0000 7fe8b7f6d640 1 -- 172.21.15.16:0/1141883064 wait complete. 2024-09-19T07:53:11.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.212+0000 7fe8b7f6d640 1 Processor -- start 2024-09-19T07:53:11.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.212+0000 7fe8b7f6d640 1 -- start start 2024-09-19T07:53:11.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.213+0000 7fe8b7f6d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe89c0039b0 0x7fe8b0156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.213+0000 7fe8b7f6d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8b0156b40 0x7fe8b01f3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.213+0000 7fe8b7f6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8b01f3860 0x7fe8b013cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.213+0000 7fe8b7f6d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b00d07e0 con 0x7fe8b01f3860 2024-09-19T07:53:11.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.213+0000 7fe8b7f6d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b006f730 con 0x7fe89c0039b0 2024-09-19T07:53:11.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.213+0000 7fe8b7f6d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe8b00d0960 con 0x7fe8b0156b40 2024-09-19T07:53:11.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.213+0000 7fe8b650a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8b01f3860 0x7fe8b013cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.213+0000 7fe8b650a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8b01f3860 0x7fe8b013cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60854/0 (socket says 172.21.15.16:60854) 2024-09-19T07:53:11.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.213+0000 7fe8b650a640 1 -- 172.21.15.16:0/3709061669 learned_addr learned my addr 172.21.15.16:0/3709061669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:11.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.213+0000 7fe8b5508640 1 --2- 172.21.15.16:0/3709061669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8b0156b40 0x7fe8b01f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.213+0000 7fe8b5d09640 1 --2- 172.21.15.16:0/3709061669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe89c0039b0 0x7fe8b0156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.213+0000 7fe8b650a640 1 -- 172.21.15.16:0/3709061669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8b0156b40 msgr2=0x7fe8b01f3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.213+0000 7fe8b650a640 1 --2- 172.21.15.16:0/3709061669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8b0156b40 0x7fe8b01f3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.213+0000 7fe8b650a640 1 -- 172.21.15.16:0/3709061669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe89c0039b0 msgr2=0x7fe8b0156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.213+0000 7fe8b650a640 1 --2- 172.21.15.16:0/3709061669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe89c0039b0 0x7fe8b0156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.213+0000 7fe8b650a640 1 -- 172.21.15.16:0/3709061669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b01f3f50 con 0x7fe8b01f3860 2024-09-19T07:53:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.213+0000 7fe8b650a640 1 --2- 172.21.15.16:0/3709061669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8b01f3860 0x7fe8b013cd50 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00ce1e0 tx=0x7fe8a800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.213+0000 7fe8a67fc640 1 -- 172.21.15.16:0/3709061669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a800eb90 con 0x7fe8b01f3860 2024-09-19T07:53:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.214+0000 7fe8a67fc640 1 -- 172.21.15.16:0/3709061669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a8018070 con 0x7fe8b01f3860 2024-09-19T07:53:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.214+0000 7fe8b7f6d640 1 -- 172.21.15.16:0/3709061669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b00aa2e0 con 0x7fe8b01f3860 2024-09-19T07:53:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.214+0000 7fe8a67fc640 1 -- 172.21.15.16:0/3709061669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8013420 con 0x7fe8b01f3860 2024-09-19T07:53:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.214+0000 7fe8b7f6d640 1 -- 172.21.15.16:0/3709061669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b00aa000 con 0x7fe8b01f3860 2024-09-19T07:53:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.215+0000 7fe8b7f6d640 1 -- 172.21.15.16:0/3709061669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b006fd70 con 0x7fe8b01f3860 2024-09-19T07:53:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.215+0000 7fe8a67fc640 1 -- 172.21.15.16:0/3709061669 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe8a80135c0 con 0x7fe8b01f3860 2024-09-19T07:53:11.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.216+0000 7fe8a67fc640 1 --2- 172.21.15.16:0/3709061669 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe89406c2b0 0x7fe89406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.216+0000 7fe8a67fc640 1 -- 172.21.15.16:0/3709061669 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fe8a808b880 con 0x7fe8b01f3860 2024-09-19T07:53:11.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.216+0000 7fe8b5d09640 1 --2- 172.21.15.16:0/3709061669 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe89406c2b0 0x7fe89406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.217+0000 7fe8b5d09640 1 --2- 172.21.15.16:0/3709061669 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe89406c2b0 0x7fe89406e770 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac037000 tx=0x7fe8ac031000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:11.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.220+0000 7fe8a67fc640 1 -- 172.21.15.16:0/3709061669 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe8a8059810 con 0x7fe8b01f3860 2024-09-19T07:53:11.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.434+0000 7fe8b7f6d640 1 -- 172.21.15.16:0/3709061669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8b00b6c20 con 0x7fe8b01f3860 2024-09-19T07:53:11.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.435+0000 7fe8a67fc640 1 -- 172.21.15.16:0/3709061669 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5228 (secure 0 0 0) 0x7fe8a805db10 con 0x7fe8b01f3860 2024-09-19T07:53:11.435 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:11.435 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":56,"btime":"2024-09-19T07:53:04:953484+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6017,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2955839490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":2955839490},{"type":"v1","addr":"172.21.15.146:6837","nonce":2955839490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6550,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/395357736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":395357736},{"type":"v1","addr":"172.21.15.146:6836","nonce":395357736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3539638739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3539638739},{"type":"v1","addr":"172.21.15.16:6834","nonce":3539638739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:53:02.264799+0000","modified":"2024-09-19T07:53:04.298337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/4185904511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4185904511},{"type":"v1","addr":"172.21.15.16:6835","nonce":4185904511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":7}]} 2024-09-19T07:53:11.436 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 56 2024-09-19T07:53:11.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.437+0000 7fe8b7f6d640 1 -- 172.21.15.16:0/3709061669 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe89406c2b0 msgr2=0x7fe89406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:11.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.437+0000 7fe8b7f6d640 1 --2- 172.21.15.16:0/3709061669 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe89406c2b0 0x7fe89406e770 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac037000 tx=0x7fe8ac031000 comp rx=0 tx=0).stop 2024-09-19T07:53:11.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.437+0000 7fe8b7f6d640 1 -- 172.21.15.16:0/3709061669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8b01f3860 msgr2=0x7fe8b013cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.437+0000 7fe8b7f6d640 1 --2- 172.21.15.16:0/3709061669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8b01f3860 0x7fe8b013cd50 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00ce1e0 tx=0x7fe8a800e530 comp rx=0 tx=0).stop 2024-09-19T07:53:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.437+0000 7fe8b7f6d640 1 -- 172.21.15.16:0/3709061669 shutdown_connections 2024-09-19T07:53:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.437+0000 7fe8b7f6d640 1 --2- 172.21.15.16:0/3709061669 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe89406c2b0 0x7fe89406e770 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.437+0000 7fe8b7f6d640 1 --2- 172.21.15.16:0/3709061669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8b01f3860 0x7fe8b013cd50 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.437+0000 7fe8b7f6d640 1 --2- 172.21.15.16:0/3709061669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8b0156b40 0x7fe8b01f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.437+0000 7fe8b7f6d640 1 --2- 172.21.15.16:0/3709061669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe89c0039b0 0x7fe8b0156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.437+0000 7fe8b7f6d640 1 -- 172.21.15.16:0/3709061669 >> 172.21.15.16:0/3709061669 conn(0x7fe8b006bc10 msgr2=0x7fe8b00b62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.437+0000 7fe8b7f6d640 1 -- 172.21.15.16:0/3709061669 shutdown_connections 2024-09-19T07:53:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.437+0000 7fe8b7f6d640 1 -- 172.21.15.16:0/3709061669 wait complete. 2024-09-19T07:53:11.852 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T07:53:02.264799+0000', 'modified': '2024-09-19T07:53:04.298337+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6993}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6993': {'gid': 6993, 'name': 'c', 'rank': 0, 'incarnation': 54, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6835/4185904511', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6833', 'nonce': 4185904511}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 4185904511}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6993, 'qdb_cluster': [6993]} 2024-09-19T07:53:11.852 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T07:53:11.853 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-19T07:53:11.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.961+0000 7fd1b5227640 1 Processor -- start 2024-09-19T07:53:11.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.961+0000 7fd1b5227640 1 -- start start 2024-09-19T07:53:11.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.962+0000 7fd1b5227640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1b00d21f0 0x7fd1b00d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.962+0000 7fd1b5227640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd1b0141ad0 0x7fd1b0141f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.962+0000 7fd1b5227640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1b00d2b10 con 0x7fd1b0072590 2024-09-19T07:53:11.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.962+0000 7fd1b5227640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd1b0141750 con 0x7fd1b0141ad0 2024-09-19T07:53:11.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.962+0000 7fd1b5227640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b006d7f0 con 0x7fd1b00d21f0 2024-09-19T07:53:11.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.962+0000 7fd1af59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd1b0141ad0 0x7fd1b0141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.962+0000 7fd1ae59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd1b0072590 0x7fd1b006d0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60186/0 (socket says 172.21.15.16:60186) 2024-09-19T07:53:11.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.962+0000 7fd1ae59c640 1 -- 172.21.15.16:0/886708197 learned_addr learned my addr 172.21.15.16:0/886708197 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:11.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.962+0000 7fd1aed9d640 1 --2- 172.21.15.16:0/886708197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1b00d21f0 0x7fd1b00d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.963+0000 7fd1af59e640 1 -- 172.21.15.16:0/886708197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1b00d21f0 msgr2=0x7fd1b00d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:11.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.963+0000 7fd1af59e640 1 --2- 172.21.15.16:0/886708197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1b00d21f0 0x7fd1b00d25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.963+0000 7fd1af59e640 1 -- 172.21.15.16:0/886708197 >> v1:172.21.15.16:6789/0 conn(0x7fd1b0072590 legacy=0x7fd1b006d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:11.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.963+0000 7fd1af59e640 1 -- 172.21.15.16:0/886708197 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b01620b0 con 0x7fd1b0141ad0 2024-09-19T07:53:11.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.963+0000 7fd1af59e640 1 --2- 172.21.15.16:0/886708197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b0141ad0 0x7fd1b0141f80 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009dc0 tx=0x7fd1a802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d0452aa1997848 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:11.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.964+0000 7fd1add9b640 1 -- 172.21.15.16:0/886708197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8007740 con 0x7fd1b0141ad0 2024-09-19T07:53:11.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.964+0000 7fd1add9b640 1 -- 172.21.15.16:0/886708197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1a80078e0 con 0x7fd1b0141ad0 2024-09-19T07:53:11.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.964+0000 7fd1add9b640 1 -- 172.21.15.16:0/886708197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8007cb0 con 0x7fd1b0141ad0 2024-09-19T07:53:11.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.964+0000 7fd1b5227640 1 -- 172.21.15.16:0/886708197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b0141ad0 msgr2=0x7fd1b0141f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:11.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.964+0000 7fd1b5227640 1 --2- 172.21.15.16:0/886708197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b0141ad0 0x7fd1b0141f80 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009dc0 tx=0x7fd1a802a5f0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.964+0000 7fd1b5227640 1 -- 172.21.15.16:0/886708197 shutdown_connections 2024-09-19T07:53:11.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.964+0000 7fd1b5227640 1 --2- 172.21.15.16:0/886708197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b0141ad0 0x7fd1b0141f80 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.964+0000 7fd1b5227640 1 --2- 172.21.15.16:0/886708197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1b00d21f0 0x7fd1b00d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.964+0000 7fd1b5227640 1 -- 172.21.15.16:0/886708197 >> 172.21.15.16:0/886708197 conn(0x7fd1b00b3ed0 msgr2=0x7fd1b00b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:11.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.964+0000 7fd1b5227640 1 -- 172.21.15.16:0/886708197 shutdown_connections 2024-09-19T07:53:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.964+0000 7fd1b5227640 1 -- 172.21.15.16:0/886708197 wait complete. 2024-09-19T07:53:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.966+0000 7fd1b5227640 1 Processor -- start 2024-09-19T07:53:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.966+0000 7fd1b5227640 1 -- start start 2024-09-19T07:53:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.967+0000 7fd1b5227640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1b0072590 0x7fd1b0158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.967+0000 7fd1b5227640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b00d21f0 0x7fd1b01f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.967+0000 7fd1b5227640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1b0141ad0 0x7fd1b01f7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.967+0000 7fd1b5227640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b0162930 con 0x7fd1b00d21f0 2024-09-19T07:53:11.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.967+0000 7fd1b5227640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b0162650 con 0x7fd1b0141ad0 2024-09-19T07:53:11.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.967+0000 7fd1b5227640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b0162c10 con 0x7fd1b0072590 2024-09-19T07:53:11.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.967+0000 7fd1ae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b00d21f0 0x7fd1b01f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.967+0000 7fd1ae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b00d21f0 0x7fd1b01f4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60874/0 (socket says 172.21.15.16:60874) 2024-09-19T07:53:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.967+0000 7fd1ae59c640 1 -- 172.21.15.16:0/2531331631 learned_addr learned my addr 172.21.15.16:0/2531331631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.967+0000 7fd1aed9d640 1 --2- 172.21.15.16:0/2531331631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1b0072590 0x7fd1b0158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.967+0000 7fd1af59e640 1 --2- 172.21.15.16:0/2531331631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1b0141ad0 0x7fd1b01f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.967+0000 7fd1ae59c640 1 -- 172.21.15.16:0/2531331631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1b0072590 msgr2=0x7fd1b0158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.967+0000 7fd1ae59c640 1 --2- 172.21.15.16:0/2531331631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1b0072590 0x7fd1b0158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.967+0000 7fd1ae59c640 1 -- 172.21.15.16:0/2531331631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1b0141ad0 msgr2=0x7fd1b01f7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.967+0000 7fd1ae59c640 1 --2- 172.21.15.16:0/2531331631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1b0141ad0 0x7fd1b01f7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.967+0000 7fd1ae59c640 1 -- 172.21.15.16:0/2531331631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b015bab0 con 0x7fd1b00d21f0 2024-09-19T07:53:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.967+0000 7fd1ae59c640 1 --2- 172.21.15.16:0/2531331631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b00d21f0 0x7fd1b01f4360 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00d3330 tx=0x7fd19800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.968+0000 7fd1a77fe640 1 -- 172.21.15.16:0/2531331631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd198017070 con 0x7fd1b00d21f0 2024-09-19T07:53:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.968+0000 7fd1a77fe640 1 -- 172.21.15.16:0/2531331631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd198005ce0 con 0x7fd1b00d21f0 2024-09-19T07:53:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.968+0000 7fd1b5227640 1 -- 172.21.15.16:0/2531331631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b015bd90 con 0x7fd1b00d21f0 2024-09-19T07:53:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.968+0000 7fd1a77fe640 1 -- 172.21.15.16:0/2531331631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd198007310 con 0x7fd1b00d21f0 2024-09-19T07:53:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.968+0000 7fd1b5227640 1 -- 172.21.15.16:0/2531331631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b015c070 con 0x7fd1b00d21f0 2024-09-19T07:53:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.969+0000 7fd1a77fe640 1 -- 172.21.15.16:0/2531331631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd19801a070 con 0x7fd1b00d21f0 2024-09-19T07:53:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.970+0000 7fd1a77fe640 1 --2- 172.21.15.16:0/2531331631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd19006c2b0 0x7fd19006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.970+0000 7fd1a77fe640 1 -- 172.21.15.16:0/2531331631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fd19808a7f0 con 0x7fd1b00d21f0 2024-09-19T07:53:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.970+0000 7fd1aed9d640 1 --2- 172.21.15.16:0/2531331631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd19006c2b0 0x7fd19006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.970+0000 7fd1b7fc7740 1 Processor -- start 2024-09-19T07:53:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.970+0000 7fd1aed9d640 1 --2- 172.21.15.16:0/2531331631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd19006c2b0 0x7fd19006e770 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0006fd0 tx=0x7fd1a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.971+0000 7fd1b7fc7740 1 -- start start 2024-09-19T07:53:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.971+0000 7fd1b7fc7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7ab942c0 0x563d7abd9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.971+0000 7fd1b7fc7740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab91a80 0x563d7ab946a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.971+0000 7fd1b7fc7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7ab95a50 0x563d7ab8e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.971+0000 7fd1b7fc7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563d7aaf6520 con 0x563d7ab95a50 2024-09-19T07:53:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.971+0000 7fd1b7fc7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563d7aa9e3e0 con 0x563d7ab942c0 2024-09-19T07:53:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.971+0000 7fd1b7fc7740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x563d7aaf24e0 con 0x563d7ab91a80 2024-09-19T07:53:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.972+0000 7fd1af59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7ab95a50 0x563d7ab8e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.972+0000 7fd1af59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7ab95a50 0x563d7ab8e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60890/0 (socket says 172.21.15.16:60890) 2024-09-19T07:53:11.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.972+0000 7fd1af59e640 1 -- 172.21.15.16:0/2663288298 learned_addr learned my addr 172.21.15.16:0/2663288298 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:11.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.972+0000 7fd1aed9d640 1 --2- 172.21.15.16:0/2663288298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7ab942c0 0x563d7abd9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.972+0000 7fd1ae59c640 1 --2- 172.21.15.16:0/2663288298 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab91a80 0x563d7ab946a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.972+0000 7fd1af59e640 1 -- 172.21.15.16:0/2663288298 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab91a80 msgr2=0x563d7ab946a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:11.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.972+0000 7fd1af59e640 1 --2- 172.21.15.16:0/2663288298 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab91a80 0x563d7ab946a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.972+0000 7fd1af59e640 1 -- 172.21.15.16:0/2663288298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7ab942c0 msgr2=0x563d7abd9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:11.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.972+0000 7fd1af59e640 1 --2- 172.21.15.16:0/2663288298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7ab942c0 0x563d7abd9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.972+0000 7fd1af59e640 1 -- 172.21.15.16:0/2663288298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d7aab92f0 con 0x563d7ab95a50 2024-09-19T07:53:11.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.972+0000 7fd1af59e640 1 --2- 172.21.15.16:0/2663288298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7ab95a50 0x563d7ab8e8b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009eb0 tx=0x7fd1a802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0dc29a669117a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:11.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.972+0000 7fd1a4ff9640 1 -- 172.21.15.16:0/2663288298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8007a60 con 0x563d7ab95a50 2024-09-19T07:53:11.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.972+0000 7fd1a4ff9640 1 -- 172.21.15.16:0/2663288298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1a8007c00 con 0x563d7ab95a50 2024-09-19T07:53:11.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.972+0000 7fd1a4ff9640 1 -- 172.21.15.16:0/2663288298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8045430 con 0x563d7ab95a50 2024-09-19T07:53:11.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.973+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2663288298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7ab95a50 msgr2=0x563d7ab8e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:11.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.973+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2663288298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7ab95a50 0x563d7ab8e8b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009eb0 tx=0x7fd1a802e000 comp rx=0 tx=0).stop 2024-09-19T07:53:11.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.973+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2663288298 shutdown_connections 2024-09-19T07:53:11.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.973+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2663288298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7ab95a50 0x563d7ab8e8b0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.973+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2663288298 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab91a80 0x563d7ab946a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.973+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2663288298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7ab942c0 0x563d7abd9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.973+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2663288298 >> 172.21.15.16:0/2663288298 conn(0x563d7ab90a80 msgr2=0x563d7abd8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.973+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2663288298 shutdown_connections 2024-09-19T07:53:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.973+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2663288298 wait complete. 2024-09-19T07:53:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.974+0000 7fd1b7fc7740 1 -- start start 2024-09-19T07:53:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.974+0000 7fd1b7fc7740 1 Processor -- start 2024-09-19T07:53:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.974+0000 7fd1b7fc7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7ab942c0 0x563d7abebb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd1b7fc7740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab95a50 0x563d7abbbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd1b7fc7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7abbc370 0x563d7abbe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd1b7fc7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563d7ab8ff20 con 0x563d7ab942c0 2024-09-19T07:53:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd1b7fc7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563d7aa7f450 con 0x563d7abbc370 2024-09-19T07:53:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd1b7fc7740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x563d7abda290 con 0x563d7ab95a50 2024-09-19T07:53:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd1b7fc7740 10 client.0 _ll_register_callbacks cb 0x563d7a808ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd1aed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7ab942c0 0x563d7abebb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd1aed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7ab942c0 0x563d7abebb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60902/0 (socket says 172.21.15.16:60902) 2024-09-19T07:53:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd1aed9d640 1 -- 172.21.15.16:0/2093624198 learned_addr learned my addr 172.21.15.16:0/2093624198 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd1ae59c640 1 --2- 172.21.15.16:0/2093624198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab95a50 0x563d7abbbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd1af59e640 1 --2- 172.21.15.16:0/2093624198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7abbc370 0x563d7abbe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd1aed9d640 1 -- 172.21.15.16:0/2093624198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab95a50 msgr2=0x563d7abbbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd1aed9d640 1 --2- 172.21.15.16:0/2093624198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab95a50 0x563d7abbbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd1aed9d640 1 -- 172.21.15.16:0/2093624198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7abbc370 msgr2=0x563d7abbe360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd1aed9d640 1 --2- 172.21.15.16:0/2093624198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7abbc370 0x563d7abbe360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd1aed9d640 1 -- 172.21.15.16:0/2093624198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d7a8092d0 con 0x563d7ab942c0 2024-09-19T07:53:11.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd1af59e640 1 --2- 172.21.15.16:0/2093624198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7abbc370 0x563d7abbe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:11.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd1aed9d640 1 --2- 172.21.15.16:0/2093624198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7ab942c0 0x563d7abebb40 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0016820 tx=0x7fd1a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2726b581201225bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:11.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd18dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:53:11.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd18dffb640 1 -- 172.21.15.16:0/2093624198 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a001c040 con 0x563d7ab942c0 2024-09-19T07:53:11.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd18dffb640 1 -- 172.21.15.16:0/2093624198 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fd1a0004030 con 0x563d7ab942c0 2024-09-19T07:53:11.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.975+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2093624198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1a8007c00 con 0x563d7ab942c0 2024-09-19T07:53:11.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.976+0000 7fd18dffb640 1 -- 172.21.15.16:0/2093624198 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fd1a0030050 con 0x563d7ab942c0 2024-09-19T07:53:11.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.976+0000 7fd1b7fc7740 10 client.7134 fetch_fsmap learned FSMap version 56 2024-09-19T07:53:11.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.976+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2093624198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x563d7a8d36b0 con 0x563d7ab942c0 2024-09-19T07:53:11.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.976+0000 7fd18dffb640 1 -- 172.21.15.16:0/2093624198 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1a8007c00 con 0x563d7ab942c0 2024-09-19T07:53:11.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.976+0000 7fd18dffb640 1 -- 172.21.15.16:0/2093624198 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 56) ==== 3502+0+0 (secure 0 0 0) 0x7fd1a0033020 con 0x563d7ab942c0 2024-09-19T07:53:11.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.976+0000 7fd1b7fc7740 10 client.7134 fetch_fsmap finished waiting for FSMap version 56 2024-09-19T07:53:11.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.976+0000 7fd1b7fc7740 10 client.7134 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T07:53:11.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.976+0000 7fd1b7fc7740 20 client.7134 populate_metadata read hostname 'smithi016' 2024-09-19T07:53:11.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.976+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2093624198 >> [v2:172.21.15.16:6833/4185904511,v1:172.21.15.16:6835/4185904511] conn(0x563d7abb02e0 0x563d7abb7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:11.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.976+0000 7fd1b7fc7740 4 client.7134 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:53:11.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.976+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2093624198 --> [v2:172.21.15.16:6833/4185904511,v1:172.21.15.16:6835/4185904511] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d7ab03c50 con 0x563d7abb02e0 2024-09-19T07:53:11.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.976+0000 7fd1ae59c640 1 --2- 172.21.15.16:0/2093624198 >> [v2:172.21.15.16:6833/4185904511,v1:172.21.15.16:6835/4185904511] conn(0x563d7abb02e0 0x563d7abb7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:11.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.976+0000 7fd18dffb640 10 client.7134 ms_handle_connect on v2:172.21.15.16:6833/4185904511 2024-09-19T07:53:11.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.976+0000 7fd1ae59c640 1 --2- 172.21.15.16:0/2093624198 >> [v2:172.21.15.16:6833/4185904511,v1:172.21.15.16:6835/4185904511] conn(0x563d7abb02e0 0x563d7abb7a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef956a1ab680030c server_cookie=ee4ad3712bd7b83a in_seq=0 out_seq=0 2024-09-19T07:53:11.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.977+0000 7fd18dffb640 1 -- 172.21.15.16:0/2093624198 <== mds.0 v2:172.21.15.16:6833/4185904511 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d7ab03c50 con 0x563d7abb02e0 2024-09-19T07:53:11.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.977+0000 7fd18dffb640 10 client.7134 handle_command_reply: tid=0 2024-09-19T07:53:11.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.977+0000 7fd1b7fc7740 1 client.7134 shutdown 2024-09-19T07:53:11.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.977+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2093624198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7ab942c0 msgr2=0x563d7abebb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:11.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.977+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2093624198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7ab942c0 0x563d7abebb40 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0016820 tx=0x7fd1a00046c0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.977+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2093624198 shutdown_connections 2024-09-19T07:53:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.977+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2093624198 >> [v2:172.21.15.16:6833/4185904511,v1:172.21.15.16:6835/4185904511] conn(0x563d7abb02e0 0x563d7abb7a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.977+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2093624198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7abbc370 0x563d7abbe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.977+0000 7fd18dffb640 0 client.7134 ms_handle_reset on v2:172.21.15.16:6833/4185904511 2024-09-19T07:53:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.977+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2093624198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab95a50 0x563d7abbbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.977+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2093624198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7ab942c0 0x563d7abebb40 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.977+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2093624198 >> 172.21.15.16:0/2093624198 conn(0x563d7ab91a80 msgr2=0x563d7abd6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.977+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2093624198 shutdown_connections 2024-09-19T07:53:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.977+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2093624198 wait complete. 2024-09-19T07:53:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.977+0000 7fd1b7fc7740 20 client.7134 trim_cache size 0 max 16384 2024-09-19T07:53:12.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.995+0000 7fd1b7fc7740 1 Processor -- start 2024-09-19T07:53:12.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.995+0000 7fd1b7fc7740 1 -- start start 2024-09-19T07:53:12.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.996+0000 7fd1b7fc7740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab942c0 0x563d7abd9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:12.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.996+0000 7fd1b7fc7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7ab95a50 0x563d7abeba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:12.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.996+0000 7fd1b7fc7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7abbc370 0x563d7abb7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.996+0000 7fd1b7fc7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b0200000 con 0x563d7abbc370 2024-09-19T07:53:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.996+0000 7fd1b7fc7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b00b5ae0 con 0x563d7ab95a50 2024-09-19T07:53:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.996+0000 7fd1b7fc7740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b01ffb50 con 0x563d7ab942c0 2024-09-19T07:53:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.996+0000 7fd1af59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7abbc370 0x563d7abb7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.996+0000 7fd1af59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7abbc370 0x563d7abb7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60908/0 (socket says 172.21.15.16:60908) 2024-09-19T07:53:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.996+0000 7fd1af59e640 1 -- 172.21.15.16:0/2949460338 learned_addr learned my addr 172.21.15.16:0/2949460338 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.996+0000 7fd1aed9d640 1 --2- 172.21.15.16:0/2949460338 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab942c0 0x563d7abd9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.996+0000 7fd1af59e640 1 -- 172.21.15.16:0/2949460338 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab942c0 msgr2=0x563d7abd9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.996+0000 7fd1af59e640 1 --2- 172.21.15.16:0/2949460338 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab942c0 0x563d7abd9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.996+0000 7fd1af59e640 1 -- 172.21.15.16:0/2949460338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7ab95a50 msgr2=0x563d7abeba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.996+0000 7fd1af59e640 1 --2- 172.21.15.16:0/2949460338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7ab95a50 0x563d7abeba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.996+0000 7fd1af59e640 1 -- 172.21.15.16:0/2949460338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d7abb66e0 con 0x563d7abbc370 2024-09-19T07:53:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.997+0000 7fd1af59e640 1 --2- 172.21.15.16:0/2949460338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7abbc370 0x563d7abb7a50 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd1a803e000 tx=0x7fd1a8030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a26d5d1aea60a3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.997+0000 7fd18ffff640 1 -- 172.21.15.16:0/2949460338 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8003ba0 con 0x563d7abbc370 2024-09-19T07:53:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.997+0000 7fd18ffff640 1 -- 172.21.15.16:0/2949460338 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1a8003d40 con 0x563d7abbc370 2024-09-19T07:53:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.997+0000 7fd18ffff640 1 -- 172.21.15.16:0/2949460338 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8007a60 con 0x563d7abbc370 2024-09-19T07:53:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.997+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2949460338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7abbc370 msgr2=0x563d7abb7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.997+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2949460338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7abbc370 0x563d7abb7a50 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd1a803e000 tx=0x7fd1a8030000 comp rx=0 tx=0).stop 2024-09-19T07:53:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.997+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2949460338 shutdown_connections 2024-09-19T07:53:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.997+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2949460338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7abbc370 0x563d7abb7a50 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.997+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2949460338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7ab95a50 0x563d7abeba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.997+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2949460338 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab942c0 0x563d7abd9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.997+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2949460338 >> 172.21.15.16:0/2949460338 conn(0x563d7ab91a80 msgr2=0x563d7abb51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.997+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2949460338 shutdown_connections 2024-09-19T07:53:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.997+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2949460338 wait complete. 2024-09-19T07:53:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.998+0000 7fd1b7fc7740 1 -- start start 2024-09-19T07:53:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.998+0000 7fd1b7fc7740 1 Processor -- start 2024-09-19T07:53:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1b7fc7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7ab942c0 0x563d7abea3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1b7fc7740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab95a50 0x563d7abb6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1b7fc7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7abbc370 0x563d7abdaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1b7fc7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563d7abec260 con 0x563d7abbc370 2024-09-19T07:53:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1b7fc7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563d7abea250 con 0x563d7ab942c0 2024-09-19T07:53:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1b7fc7740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x563d7abb07b0 con 0x563d7ab95a50 2024-09-19T07:53:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1b7fc7740 10 client.0 _ll_register_callbacks cb 0x7fd1a0007790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1af59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7abbc370 0x563d7abdaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1af59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7abbc370 0x563d7abdaa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60914/0 (socket says 172.21.15.16:60914) 2024-09-19T07:53:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1af59e640 1 -- 172.21.15.16:0/2010945458 learned_addr learned my addr 172.21.15.16:0/2010945458 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1aed9d640 1 --2- 172.21.15.16:0/2010945458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7ab942c0 0x563d7abea3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1ae59c640 1 --2- 172.21.15.16:0/2010945458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab95a50 0x563d7abb6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1af59e640 1 -- 172.21.15.16:0/2010945458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab95a50 msgr2=0x563d7abb6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1af59e640 1 --2- 172.21.15.16:0/2010945458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab95a50 0x563d7abb6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1af59e640 1 -- 172.21.15.16:0/2010945458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7ab942c0 msgr2=0x563d7abea3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1af59e640 1 --2- 172.21.15.16:0/2010945458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7ab942c0 0x563d7abea3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1af59e640 1 -- 172.21.15.16:0/2010945458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d7abb5aa0 con 0x563d7abbc370 2024-09-19T07:53:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1ae59c640 1 --2- 172.21.15.16:0/2010945458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab95a50 0x563d7abb6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1aed9d640 1 --2- 172.21.15.16:0/2010945458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7ab942c0 0x563d7abea3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd1af59e640 1 --2- 172.21.15.16:0/2010945458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7abbc370 0x563d7abdaa90 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8000f80 tx=0x7fd1a8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc280c5ea1eba2c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:11.999+0000 7fd18dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:53:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.000+0000 7fd18dffb640 1 -- 172.21.15.16:0/2010945458 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8054070 con 0x563d7abbc370 2024-09-19T07:53:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.000+0000 7fd18dffb640 1 -- 172.21.15.16:0/2010945458 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fd1a8004100 con 0x563d7abbc370 2024-09-19T07:53:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.000+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2010945458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1a8003d40 con 0x563d7abbc370 2024-09-19T07:53:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.000+0000 7fd18dffb640 1 -- 172.21.15.16:0/2010945458 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fd1a805f050 con 0x563d7abbc370 2024-09-19T07:53:12.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.000+0000 7fd1b7fc7740 10 client.7140 fetch_fsmap learned FSMap version 56 2024-09-19T07:53:12.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.000+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2010945458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x563d7abb71c0 con 0x563d7abbc370 2024-09-19T07:53:12.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.000+0000 7fd18dffb640 1 -- 172.21.15.16:0/2010945458 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1a8003d40 con 0x563d7abbc370 2024-09-19T07:53:12.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.000+0000 7fd18dffb640 1 -- 172.21.15.16:0/2010945458 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 56) ==== 3502+0+0 (secure 0 0 0) 0x7fd1a8062020 con 0x563d7abbc370 2024-09-19T07:53:12.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.000+0000 7fd1b7fc7740 10 client.7140 fetch_fsmap finished waiting for FSMap version 56 2024-09-19T07:53:12.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.000+0000 7fd1b7fc7740 10 client.7140 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T07:53:12.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.000+0000 7fd1b7fc7740 20 client.7140 populate_metadata read hostname 'smithi016' 2024-09-19T07:53:12.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.000+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2010945458 >> [v2:172.21.15.16:6833/4185904511,v1:172.21.15.16:6835/4185904511] conn(0x563d7ac5b1c0 0x563d7ac5b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:12.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.000+0000 7fd1b7fc7740 4 client.7140 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:53:12.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.000+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2010945458 --> [v2:172.21.15.16:6833/4185904511,v1:172.21.15.16:6835/4185904511] -- command(tid 0: {"prefix": "status"}) -- 0x563d7aafdfa0 con 0x563d7ac5b1c0 2024-09-19T07:53:12.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.000+0000 7fd1aed9d640 1 --2- 172.21.15.16:0/2010945458 >> [v2:172.21.15.16:6833/4185904511,v1:172.21.15.16:6835/4185904511] conn(0x563d7ac5b1c0 0x563d7ac5b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:12.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.000+0000 7fd18dffb640 10 client.7140 ms_handle_connect on v2:172.21.15.16:6833/4185904511 2024-09-19T07:53:12.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.000+0000 7fd1aed9d640 1 --2- 172.21.15.16:0/2010945458 >> [v2:172.21.15.16:6833/4185904511,v1:172.21.15.16:6835/4185904511] conn(0x563d7ac5b1c0 0x563d7ac5b5a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f466988f25bedec server_cookie=51446ccd0175b437 in_seq=0 out_seq=0 2024-09-19T07:53:12.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.001+0000 7fd18dffb640 1 -- 172.21.15.16:0/2010945458 <== mds.0 v2:172.21.15.16:6833/4185904511 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563d7aafdfa0 con 0x563d7ac5b1c0 2024-09-19T07:53:12.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.001+0000 7fd18dffb640 10 client.7140 handle_command_reply: tid=0 2024-09-19T07:53:12.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.001+0000 7fd1b7fc7740 1 client.7140 shutdown 2024-09-19T07:53:12.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.001+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2010945458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7abbc370 msgr2=0x563d7abdaa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:12.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.001+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2010945458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7abbc370 0x563d7abdaa90 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8000f80 tx=0x7fd1a8004ab0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.001+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2010945458 shutdown_connections 2024-09-19T07:53:12.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.001+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2010945458 >> [v2:172.21.15.16:6833/4185904511,v1:172.21.15.16:6835/4185904511] conn(0x563d7ac5b1c0 0x563d7ac5b5a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.001+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2010945458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d7abbc370 0x563d7abdaa90 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.001+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2010945458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x563d7ab95a50 0x563d7abb6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.001+0000 7fd1b7fc7740 1 --2- 172.21.15.16:0/2010945458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563d7ab942c0 0x563d7abea3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.001+0000 7fd18dffb640 0 client.7140 ms_handle_reset on v2:172.21.15.16:6833/4185904511 2024-09-19T07:53:12.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.001+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2010945458 >> 172.21.15.16:0/2010945458 conn(0x563d7ab91a80 msgr2=0x563d7abbd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:12.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.001+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2010945458 shutdown_connections 2024-09-19T07:53:12.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.001+0000 7fd1b7fc7740 1 -- 172.21.15.16:0/2010945458 wait complete. 2024-09-19T07:53:12.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.001+0000 7fd1b7fc7740 20 client.7140 trim_cache size 0 max 16384 2024-09-19T07:53:12.020 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:53:12.020 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb", 2024-09-19T07:53:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-19T07:53:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 6993, 2024-09-19T07:53:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T07:53:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T07:53:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T07:53:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 9.0508390280000004, 2024-09-19T07:53:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 56, 2024-09-19T07:53:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 106, 2024-09-19T07:53:12.022 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 106, 2024-09-19T07:53:12.022 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 14.109853847 2024-09-19T07:53:12.022 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:53:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.001+0000 7fd1b5227640 1 -- 172.21.15.16:0/2531331631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd19006c2b0 msgr2=0x7fd19006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.001+0000 7fd1b5227640 1 --2- 172.21.15.16:0/2531331631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd19006c2b0 0x7fd19006e770 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0006fd0 tx=0x7fd1a0008040 comp rx=0 tx=0).stop 2024-09-19T07:53:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.001+0000 7fd1b5227640 1 -- 172.21.15.16:0/2531331631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b00d21f0 msgr2=0x7fd1b01f4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.001+0000 7fd1b5227640 1 --2- 172.21.15.16:0/2531331631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b00d21f0 0x7fd1b01f4360 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00d3330 tx=0x7fd19800c800 comp rx=0 tx=0).stop 2024-09-19T07:53:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.002+0000 7fd1b5227640 1 -- 172.21.15.16:0/2531331631 shutdown_connections 2024-09-19T07:53:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.002+0000 7fd1b5227640 1 --2- 172.21.15.16:0/2531331631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd19006c2b0 0x7fd19006e770 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.002+0000 7fd1b5227640 1 --2- 172.21.15.16:0/2531331631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1b0141ad0 0x7fd1b01f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.002+0000 7fd1b5227640 1 --2- 172.21.15.16:0/2531331631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1b00d21f0 0x7fd1b01f4360 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.002+0000 7fd1b5227640 1 --2- 172.21.15.16:0/2531331631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1b0072590 0x7fd1b0158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.002+0000 7fd1b5227640 1 -- 172.21.15.16:0/2531331631 >> 172.21.15.16:0/2531331631 conn(0x7fd1b00b3ed0 msgr2=0x7fd1b015d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.002+0000 7fd1b5227640 1 -- 172.21.15.16:0/2531331631 shutdown_connections 2024-09-19T07:53:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.002+0000 7fd1b5227640 1 -- 172.21.15.16:0/2531331631 wait complete. 2024-09-19T07:53:12.026 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:53:12.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.128+0000 7fe6207df640 1 Processor -- start 2024-09-19T07:53:12.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.128+0000 7fe6207df640 1 -- start start 2024-09-19T07:53:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.129+0000 7fe6207df640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe61815e2a0 0x7fe61806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.129+0000 7fe6207df640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61815e680 0x7fe6180cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.129+0000 7fe6207df640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6180d1880 con 0x7fe61806d960 2024-09-19T07:53:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.129+0000 7fe6207df640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe6180d1b60 con 0x7fe61815e2a0 2024-09-19T07:53:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.129+0000 7fe6207df640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe6180cf2d0 con 0x7fe61815e680 2024-09-19T07:53:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.129+0000 7fe61dd7a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61815e680 0x7fe6180cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.129+0000 7fe61e57b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe61815e2a0 0x7fe61806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.129+0000 7fe61dd7a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61815e680 0x7fe6180cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:48920/0 (socket says 172.21.15.16:48920) 2024-09-19T07:53:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.129+0000 7fe61dd7a640 1 -- 172.21.15.16:0/609465836 learned_addr learned my addr 172.21.15.16:0/609465836 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.130+0000 7fe61dd7a640 1 -- 172.21.15.16:0/609465836 >> v2:172.21.15.146:3300/0 conn(0x7fe61815e2a0 msgr2=0x7fe61806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.130+0000 7fe61dd7a640 1 --2- 172.21.15.16:0/609465836 >> v2:172.21.15.146:3300/0 conn(0x7fe61815e2a0 0x7fe61806d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.130+0000 7fe61dd7a640 1 -- 172.21.15.16:0/609465836 >> v1:172.21.15.146:6789/0 conn(0x7fe61806d960 legacy=0x7fe6180d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.130+0000 7fe61dd7a640 1 -- 172.21.15.16:0/609465836 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6180cf5b0 con 0x7fe61815e680 2024-09-19T07:53:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.130+0000 7fe61dd7a640 1 --2- 172.21.15.16:0/609465836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61815e680 0x7fe6180cfad0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe608009870 tx=0x7fe60802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff24c0460f077fbb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.131+0000 7fe61d579640 1 -- 172.21.15.16:0/609465836 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608037070 con 0x7fe61815e680 2024-09-19T07:53:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.131+0000 7fe61d579640 1 -- 172.21.15.16:0/609465836 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe608032440 con 0x7fe61815e680 2024-09-19T07:53:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.131+0000 7fe61d579640 1 -- 172.21.15.16:0/609465836 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608032810 con 0x7fe61815e680 2024-09-19T07:53:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.131+0000 7fe6207df640 1 -- 172.21.15.16:0/609465836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61815e680 msgr2=0x7fe6180cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.131+0000 7fe6207df640 1 --2- 172.21.15.16:0/609465836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61815e680 0x7fe6180cfad0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe608009870 tx=0x7fe60802a270 comp rx=0 tx=0).stop 2024-09-19T07:53:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.131+0000 7fe6207df640 1 -- 172.21.15.16:0/609465836 shutdown_connections 2024-09-19T07:53:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.131+0000 7fe6207df640 1 --2- 172.21.15.16:0/609465836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61815e680 0x7fe6180cfad0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.131+0000 7fe6207df640 1 --2- 172.21.15.16:0/609465836 >> v2:172.21.15.146:3300/0 conn(0x7fe61815e2a0 0x7fe61806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.131+0000 7fe6207df640 1 -- 172.21.15.16:0/609465836 >> 172.21.15.16:0/609465836 conn(0x7fe61809d760 msgr2=0x7fe61808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.131+0000 7fe6207df640 1 -- 172.21.15.16:0/609465836 shutdown_connections 2024-09-19T07:53:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.131+0000 7fe6207df640 1 -- 172.21.15.16:0/609465836 wait complete. 2024-09-19T07:53:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.132+0000 7fe6207df640 1 Processor -- start 2024-09-19T07:53:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.132+0000 7fe6207df640 1 -- start start 2024-09-19T07:53:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.133+0000 7fe6207df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61806d960 0x7fe6181ea7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.133+0000 7fe6207df640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe61815e2a0 0x7fe61815cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.133+0000 7fe6207df640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61815e680 0x7fe61815d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.133+0000 7fe6207df640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6180d4c00 con 0x7fe61806d960 2024-09-19T07:53:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.133+0000 7fe6207df640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe6180d4a80 con 0x7fe61815e2a0 2024-09-19T07:53:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.133+0000 7fe6207df640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe6180d0010 con 0x7fe61815e680 2024-09-19T07:53:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.133+0000 7fe61e57b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61806d960 0x7fe6181ea7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.133+0000 7fe61e57b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61806d960 0x7fe6181ea7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60928/0 (socket says 172.21.15.16:60928) 2024-09-19T07:53:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.133+0000 7fe61e57b640 1 -- 172.21.15.16:0/1191372593 learned_addr learned my addr 172.21.15.16:0/1191372593 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.133+0000 7fe61dd7a640 1 --2- 172.21.15.16:0/1191372593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe61815e2a0 0x7fe61815cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.133+0000 7fe61ed7c640 1 --2- 172.21.15.16:0/1191372593 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61815e680 0x7fe61815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.134+0000 7fe61e57b640 1 -- 172.21.15.16:0/1191372593 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61815e680 msgr2=0x7fe61815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.134+0000 7fe61e57b640 1 --2- 172.21.15.16:0/1191372593 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61815e680 0x7fe61815d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.134+0000 7fe61e57b640 1 -- 172.21.15.16:0/1191372593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe61815e2a0 msgr2=0x7fe61815cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.134+0000 7fe61e57b640 1 --2- 172.21.15.16:0/1191372593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe61815e2a0 0x7fe61815cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.134+0000 7fe61e57b640 1 -- 172.21.15.16:0/1191372593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6180b9dd0 con 0x7fe61806d960 2024-09-19T07:53:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.134+0000 7fe61e57b640 1 --2- 172.21.15.16:0/1191372593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61806d960 0x7fe6181ea7e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe61400e9b0 tx=0x7fe61400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.134+0000 7fe60effd640 1 -- 172.21.15.16:0/1191372593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614017070 con 0x7fe61806d960 2024-09-19T07:53:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.134+0000 7fe60effd640 1 -- 172.21.15.16:0/1191372593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe61400c9a0 con 0x7fe61806d960 2024-09-19T07:53:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.134+0000 7fe6207df640 1 -- 172.21.15.16:0/1191372593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6180cb5e0 con 0x7fe61806d960 2024-09-19T07:53:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.134+0000 7fe60effd640 1 -- 172.21.15.16:0/1191372593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61400cd90 con 0x7fe61806d960 2024-09-19T07:53:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.134+0000 7fe6207df640 1 -- 172.21.15.16:0/1191372593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6180cbba0 con 0x7fe61806d960 2024-09-19T07:53:12.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.135+0000 7fe6207df640 1 -- 172.21.15.16:0/1191372593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe61806d420 con 0x7fe61806d960 2024-09-19T07:53:12.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.136+0000 7fe60effd640 1 -- 172.21.15.16:0/1191372593 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe61401a070 con 0x7fe61806d960 2024-09-19T07:53:12.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.136+0000 7fe60effd640 1 --2- 172.21.15.16:0/1191372593 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe5fc06c2b0 0x7fe5fc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:12.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.136+0000 7fe60effd640 1 -- 172.21.15.16:0/1191372593 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fe61408b800 con 0x7fe61806d960 2024-09-19T07:53:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.136+0000 7fe61dd7a640 1 --2- 172.21.15.16:0/1191372593 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe5fc06c2b0 0x7fe5fc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.137+0000 7fe61dd7a640 1 --2- 172.21.15.16:0/1191372593 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe5fc06c2b0 0x7fe5fc06e770 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe608034000 tx=0x7fe60802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.140+0000 7fe60effd640 1 -- 172.21.15.16:0/1191372593 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe614059820 con 0x7fe61806d960 2024-09-19T07:53:12.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.352+0000 7fe6207df640 1 -- 172.21.15.16:0/1191372593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe6180d45e0 con 0x7fe61806d960 2024-09-19T07:53:12.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.353+0000 7fe60effd640 1 -- 172.21.15.16:0/1191372593 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) ==== 53+0+73 (secure 0 0 0) 0x7fe61405db20 con 0x7fe61806d960 2024-09-19T07:53:12.353 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:53:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.355+0000 7fe6207df640 1 -- 172.21.15.16:0/1191372593 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe5fc06c2b0 msgr2=0x7fe5fc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.355+0000 7fe6207df640 1 --2- 172.21.15.16:0/1191372593 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe5fc06c2b0 0x7fe5fc06e770 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe608034000 tx=0x7fe60802e000 comp rx=0 tx=0).stop 2024-09-19T07:53:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.355+0000 7fe6207df640 1 -- 172.21.15.16:0/1191372593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61806d960 msgr2=0x7fe6181ea7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.355+0000 7fe6207df640 1 --2- 172.21.15.16:0/1191372593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61806d960 0x7fe6181ea7e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe61400e9b0 tx=0x7fe61400ee70 comp rx=0 tx=0).stop 2024-09-19T07:53:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.355+0000 7fe6207df640 1 -- 172.21.15.16:0/1191372593 shutdown_connections 2024-09-19T07:53:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.355+0000 7fe6207df640 1 --2- 172.21.15.16:0/1191372593 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe5fc06c2b0 0x7fe5fc06e770 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.355+0000 7fe6207df640 1 --2- 172.21.15.16:0/1191372593 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe61815e680 0x7fe61815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.355+0000 7fe6207df640 1 --2- 172.21.15.16:0/1191372593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe61815e2a0 0x7fe61815cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.355+0000 7fe6207df640 1 --2- 172.21.15.16:0/1191372593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61806d960 0x7fe6181ea7e0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.355+0000 7fe6207df640 1 -- 172.21.15.16:0/1191372593 >> 172.21.15.16:0/1191372593 conn(0x7fe61809d760 msgr2=0x7fe61815bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.355+0000 7fe6207df640 1 -- 172.21.15.16:0/1191372593 shutdown_connections 2024-09-19T07:53:12.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12.355+0000 7fe6207df640 1 -- 172.21.15.16:0/1191372593 wait complete. 2024-09-19T07:53:12.366 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:53:12.367 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:53:12.367 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T07:53:12.367 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T07:53:12.367 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:12.367 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:53:12.367 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:53:12.367 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:53:12.367 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:53:12.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:12.368 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T07:53:12.385 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:53:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:53:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:53:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:53:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:12.385 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:53:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T07:53:12.385 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:53:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T07:53:12.386 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:53:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T07:53:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T07:53:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42197sec preferred_lft 42197sec 2024-09-19T07:53:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T07:53:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:12.386 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:53:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T07:53:12.386 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:53:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:53:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::32:7bff:fe92:8895/64 scope link 2024-09-19T07:53:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:12.387 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:53:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether e6:fd:21:6a:43:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e4fd:21ff:fe6a:4369/64 scope link 2024-09-19T07:53:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:12.387 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:53:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::80f4:81ff:fe7d:b7f0/64 scope link 2024-09-19T07:53:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:12.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:12.388 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:53:12.449 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:53:12.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:12.451 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T07:53:12.508 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T07:53:12.508 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:53:12.508 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T07:53:12.508 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T07:53:12.509 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:53:12.509 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:53:12.509 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:53:12.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T07:53:12.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T07:53:12.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T07:53:12.509 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:53:12.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:53:12.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:53:12.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:12.638 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:53:12.654 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T07:53:12.654 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:53:12.654 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:53:12.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:12.654 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-19T07:53:12.720 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T07:53:12.721 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:53:12.721 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:12.777 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:53:12.777 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:53:12.778 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:12.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:53:12.899 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:53:12.955 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:53:12.955 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:53:12.955 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:53:12.955 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:53:12.955 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:53:12.955 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:53:12.955 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:53:12.955 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:53:12.956 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:53:12.956 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:53:12.956 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:53:12.956 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:53:12.956 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:53:12.956 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:53:12.956 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:53:12.956 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:53:12.956 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:53:12.956 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:53:12.956 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:53:12.956 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:53:12.956 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:53:12.957 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:53:12.957 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:53:12.957 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:53:12.957 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:53:12.957 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:53:12.957 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:53:12.957 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:53:12.957 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:53:12.957 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:53:12.957 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:12.957 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:12.957 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:12.957 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:12.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:12.958 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:53:13.015 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:53:13.015 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T07:53:13.059 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:53:13.085 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:53:13.142 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:53:13.142 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:53:13.142 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:53:13.142 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:53:13.142 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:53:13.142 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:53:13.142 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:53:13.142 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:53:13.142 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:53:13.142 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:53:13.142 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:53:13.143 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:53:13.143 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:53:13.143 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:53:13.143 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:53:13.143 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:53:13.143 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:53:13.143 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:53:13.143 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:53:13.143 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:53:13.143 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:53:13.143 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:53:13.143 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:53:13.143 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:53:13.144 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:53:13.144 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:53:13.144 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:53:13.144 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:53:13.144 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:53:13.144 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:53:13.144 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:13.144 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:13.144 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:13.144 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:13.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:13.145 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:53:13.145 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-19T07:53:13.143+0000 7f7d06f88480 -1 init, newargv = 0x5627679a0160 newargc=15 2024-09-19T07:53:13.145 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[65864]: starting ceph client 2024-09-19T07:53:13.148 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[65864]: starting fuse 2024-09-19T07:53:13.161 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:53:13.161 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-19T07:53:13.161 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:53:13.161 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:53:13.162 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:53:13.162 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:53:13.162 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:53:13.162 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:13.162 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:53:13.162 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:53:13.162 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:53:13.162 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:53:13.162 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:13.162 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:53:13.162 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:53:13.162 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:53:13.162 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:53:13.162 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:13.162 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:53:13.163 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:53:13.163 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:53:13.163 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:53:13.163 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:53:13.163 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:53:13.163 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:53:13.163 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:53:13.163 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:13.163 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:53:13.163 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:53:13.281 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.65864.asok 2024-09-19T07:53:13.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:53:13.291 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.65864.asok 2024-09-19T07:53:13.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:13.292 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.65864.asok status 2024-09-19T07:53:13.429 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:53:13.429 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:53:13.429 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:53:13.429 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:53:13.429 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:53:13.429 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:53:13.429 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:53:13.429 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "65864", 2024-09-19T07:53:13.429 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:53:13.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:13.430 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-19T07:53:13.430 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-19T07:53:13.430 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7149, 2024-09-19T07:53:13.430 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:53:13.430 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:53:13.430 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:53:13.430 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 7149 2024-09-19T07:53:13.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:13.430 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:53:13.430 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:53:13.430 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:53:13.430 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 212674707 2024-09-19T07:53:13.431 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:13.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:13.431 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:53:13.431 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:53:13.431 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:53:13.431 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 212674707 2024-09-19T07:53:13.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:13.431 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.7149 192.168.0.1:0/212674707", 2024-09-19T07:53:13.431 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/212674707", 2024-09-19T07:53:13.431 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-19T07:53:13.431 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 56, 2024-09-19T07:53:13.431 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 106, 2024-09-19T07:53:13.431 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:53:13.432 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:53:13.432 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:53:13.432 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:53:13.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:13.437 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-19T07:53:13.438 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:13.453 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-19T07:53:13.454 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:13.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:13.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:13.520 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:53:13.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.662+0000 7f03a34d7640 1 Processor -- start 2024-09-19T07:53:13.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.662+0000 7f03a34d7640 1 -- start start 2024-09-19T07:53:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.663+0000 7f03a34d7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f039c0c8d20 0x7f039c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.663+0000 7f03a34d7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f039c144100 0x7f039c0d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.663+0000 7f03a34d7640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f039c0c7140 con 0x7f039c06d260 2024-09-19T07:53:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.663+0000 7f03a34d7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f039c144530 con 0x7f039c144100 2024-09-19T07:53:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.663+0000 7f03a34d7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f039c141960 con 0x7f039c0c8d20 2024-09-19T07:53:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.663+0000 7f03a1a74640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f039c144100 0x7f039c0d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.663+0000 7f03a1a74640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f039c144100 0x7f039c0d1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38776/0 (socket says 172.21.15.16:38776) 2024-09-19T07:53:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.663+0000 7f03a1a74640 1 -- 172.21.15.16:0/3213709310 learned_addr learned my addr 172.21.15.16:0/3213709310 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.663+0000 7f03a1273640 1 --2- 172.21.15.16:0/3213709310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f039c0c8d20 0x7f039c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.663+0000 7f03a1a74640 1 -- 172.21.15.16:0/3213709310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f039c0c8d20 msgr2=0x7f039c06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.663+0000 7f03a1a74640 1 --2- 172.21.15.16:0/3213709310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f039c0c8d20 0x7f039c06cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.663+0000 7f03a1a74640 1 -- 172.21.15.16:0/3213709310 >> v1:172.21.15.146:6789/0 conn(0x7f039c06d260 legacy=0x7f039c1439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.663+0000 7f03a1a74640 1 -- 172.21.15.16:0/3213709310 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c0c9100 con 0x7f039c144100 2024-09-19T07:53:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.664+0000 7f03a1a74640 1 --2- 172.21.15.16:0/3213709310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f039c144100 0x7f039c0d1e50 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f0394009e50 tx=0x7f039402a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb2566a6bc7e9a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.664+0000 7f0393fff640 1 -- 172.21.15.16:0/3213709310 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0394036070 con 0x7f039c144100 2024-09-19T07:53:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.664+0000 7f0393fff640 1 -- 172.21.15.16:0/3213709310 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0394031600 con 0x7f039c144100 2024-09-19T07:53:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.664+0000 7f0393fff640 1 -- 172.21.15.16:0/3213709310 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03940319d0 con 0x7f039c144100 2024-09-19T07:53:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.665+0000 7f03a34d7640 1 -- 172.21.15.16:0/3213709310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f039c144100 msgr2=0x7f039c0d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.665+0000 7f03a34d7640 1 --2- 172.21.15.16:0/3213709310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f039c144100 0x7f039c0d1e50 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f0394009e50 tx=0x7f039402a5b0 comp rx=0 tx=0).stop 2024-09-19T07:53:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.665+0000 7f03a34d7640 1 -- 172.21.15.16:0/3213709310 shutdown_connections 2024-09-19T07:53:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.665+0000 7f03a34d7640 1 --2- 172.21.15.16:0/3213709310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f039c144100 0x7f039c0d1e50 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.665+0000 7f03a34d7640 1 --2- 172.21.15.16:0/3213709310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f039c0c8d20 0x7f039c06cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.665+0000 7f03a34d7640 1 -- 172.21.15.16:0/3213709310 >> 172.21.15.16:0/3213709310 conn(0x7f039c0b3ed0 msgr2=0x7f039c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.665+0000 7f03a34d7640 1 -- 172.21.15.16:0/3213709310 shutdown_connections 2024-09-19T07:53:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.665+0000 7f03a34d7640 1 -- 172.21.15.16:0/3213709310 wait complete. 2024-09-19T07:53:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.666+0000 7f03a34d7640 1 Processor -- start 2024-09-19T07:53:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.666+0000 7f03a34d7640 1 -- start start 2024-09-19T07:53:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.667+0000 7f03a34d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f039c06d260 0x7f039c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.667+0000 7f03a34d7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f039c0c8d20 0x7f039c0b6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.667+0000 7f03a34d7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f039c144100 0x7f039c0b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.667+0000 7f03a34d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f039c06d6a0 con 0x7f039c06d260 2024-09-19T07:53:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.667+0000 7f03a34d7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f039c142200 con 0x7f039c144100 2024-09-19T07:53:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.667+0000 7f03a34d7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f039c141bb0 con 0x7f039c0c8d20 2024-09-19T07:53:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.667+0000 7f03a1273640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f039c06d260 0x7f039c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.667+0000 7f03a1273640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f039c06d260 0x7f039c13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38788/0 (socket says 172.21.15.16:38788) 2024-09-19T07:53:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.667+0000 7f03a1273640 1 -- 172.21.15.16:0/718313230 learned_addr learned my addr 172.21.15.16:0/718313230 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.667+0000 7f03a0a72640 1 --2- 172.21.15.16:0/718313230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f039c0c8d20 0x7f039c0b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.667+0000 7f03a1a74640 1 --2- 172.21.15.16:0/718313230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f039c144100 0x7f039c0b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.667+0000 7f03a1273640 1 -- 172.21.15.16:0/718313230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f039c0c8d20 msgr2=0x7f039c0b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.667+0000 7f03a1273640 1 --2- 172.21.15.16:0/718313230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f039c0c8d20 0x7f039c0b6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.667+0000 7f03a1273640 1 -- 172.21.15.16:0/718313230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f039c144100 msgr2=0x7f039c0b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.667+0000 7f03a1273640 1 --2- 172.21.15.16:0/718313230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f039c144100 0x7f039c0b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.667+0000 7f03a1273640 1 -- 172.21.15.16:0/718313230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c0b71d0 con 0x7f039c06d260 2024-09-19T07:53:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.667+0000 7f03a1273640 1 --2- 172.21.15.16:0/718313230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f039c06d260 0x7f039c13b160 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f039800d5f0 tx=0x7f039800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.668+0000 7f0391ffb640 1 -- 172.21.15.16:0/718313230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03980125f0 con 0x7f039c06d260 2024-09-19T07:53:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.668+0000 7f0391ffb640 1 -- 172.21.15.16:0/718313230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0398019070 con 0x7f039c06d260 2024-09-19T07:53:13.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.668+0000 7f0391ffb640 1 -- 172.21.15.16:0/718313230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03980129d0 con 0x7f039c06d260 2024-09-19T07:53:13.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.668+0000 7f03a34d7640 1 -- 172.21.15.16:0/718313230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039c0b74b0 con 0x7f039c06d260 2024-09-19T07:53:13.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.668+0000 7f03a34d7640 1 -- 172.21.15.16:0/718313230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f039c0b7a70 con 0x7f039c06d260 2024-09-19T07:53:13.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.669+0000 7f03a34d7640 1 -- 172.21.15.16:0/718313230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f039c06cd20 con 0x7f039c06d260 2024-09-19T07:53:13.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.669+0000 7f0391ffb640 1 -- 172.21.15.16:0/718313230 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0398012b70 con 0x7f039c06d260 2024-09-19T07:53:13.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.670+0000 7f0391ffb640 1 --2- 172.21.15.16:0/718313230 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f038006c2e0 0x7f038006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:13.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.670+0000 7f0391ffb640 1 -- 172.21.15.16:0/718313230 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f039808b8c0 con 0x7f039c06d260 2024-09-19T07:53:13.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.670+0000 7f03a0a72640 1 --2- 172.21.15.16:0/718313230 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f038006c2e0 0x7f038006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:13.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.670+0000 7f03a0a72640 1 --2- 172.21.15.16:0/718313230 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f038006c2e0 0x7f038006e7a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f038c006fd0 tx=0x7f038c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.674+0000 7f0391ffb640 1 -- 172.21.15.16:0/718313230 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f0398059f30 con 0x7f039c06d260 2024-09-19T07:53:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.886+0000 7f03a34d7640 1 -- 172.21.15.16:0/718313230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f039c0690c0 con 0x7f039c06d260 2024-09-19T07:53:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.886+0000 7f0391ffb640 1 -- 172.21.15.16:0/718313230 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) ==== 53+0+73 (secure 0 0 0) 0x7f0398059830 con 0x7f039c06d260 2024-09-19T07:53:13.888 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:53:13.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.888+0000 7f03a34d7640 1 -- 172.21.15.16:0/718313230 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f038006c2e0 msgr2=0x7f038006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:13.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.888+0000 7f03a34d7640 1 --2- 172.21.15.16:0/718313230 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f038006c2e0 0x7f038006e7a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f038c006fd0 tx=0x7f038c008040 comp rx=0 tx=0).stop 2024-09-19T07:53:13.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.888+0000 7f03a34d7640 1 -- 172.21.15.16:0/718313230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f039c06d260 msgr2=0x7f039c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:13.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.888+0000 7f03a34d7640 1 --2- 172.21.15.16:0/718313230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f039c06d260 0x7f039c13b160 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f039800d5f0 tx=0x7f039800dab0 comp rx=0 tx=0).stop 2024-09-19T07:53:13.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.888+0000 7f03a34d7640 1 -- 172.21.15.16:0/718313230 shutdown_connections 2024-09-19T07:53:13.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.888+0000 7f03a34d7640 1 --2- 172.21.15.16:0/718313230 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f038006c2e0 0x7f038006e7a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:13.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.888+0000 7f03a34d7640 1 --2- 172.21.15.16:0/718313230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f039c144100 0x7f039c0b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:13.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.888+0000 7f03a34d7640 1 --2- 172.21.15.16:0/718313230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f039c0c8d20 0x7f039c0b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:13.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.888+0000 7f03a34d7640 1 --2- 172.21.15.16:0/718313230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f039c06d260 0x7f039c13b160 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:13.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.888+0000 7f03a34d7640 1 -- 172.21.15.16:0/718313230 >> 172.21.15.16:0/718313230 conn(0x7f039c0b3ed0 msgr2=0x7f039c15ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:13.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.888+0000 7f03a34d7640 1 -- 172.21.15.16:0/718313230 shutdown_connections 2024-09-19T07:53:13.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:13.888+0000 7f03a34d7640 1 -- 172.21.15.16:0/718313230 wait complete. 2024-09-19T07:53:13.901 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:53:13.901 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:53:13.901 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-09-19T07:53:13.901 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T07:53:13.901 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T07:53:13.901 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:53:13.901 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:53:13.901 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:53:13.901 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:53:13.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:13.901 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-09-19T07:53:13.921 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:53:13.921 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:53:13.921 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:53:13.921 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:13.921 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-09-19T07:53:13.921 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:13.921 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:53:13.921 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:13.921 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2024-09-19T07:53:13.922 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:53:13.922 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:13.922 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2024-09-19T07:53:13.922 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:53:13.922 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:13.922 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:53:13.922 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:13.922 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T07:53:13.922 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42162sec preferred_lft 42162sec 2024-09-19T07:53:13.922 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link noprefixroute 2024-09-19T07:53:13.922 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:13.922 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:53:13.922 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:13.923 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:53:13.923 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:13.923 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::e47f:32ff:fe68:31ca/64 scope link 2024-09-19T07:53:13.923 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:13.923 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:53:13.923 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether f6:1f:22:44:9d:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:53:13.923 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::f41f:22ff:fe44:9dcd/64 scope link 2024-09-19T07:53:13.923 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:13.923 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:53:13.923 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:53:13.923 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::8459:eeff:fe21:a113/64 scope link 2024-09-19T07:53:13.923 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:13.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:13.924 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:53:13.989 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-09-19T07:53:13.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:13.991 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-09-19T07:53:14.049 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-09-19T07:53:14.050 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:53:14.050 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T07:53:14.050 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T07:53:14.050 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:53:14.050 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:53:14.050 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-19T07:53:14.050 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T07:53:14.050 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T07:53:14.051 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T07:53:14.051 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-19T07:53:14.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:53:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:53:14.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:53:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:53:14.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:14.182 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-09-19T07:53:14.198 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T07:53:14.198 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T07:53:14.198 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:53:14.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:14.198 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2024-09-19T07:53:14.265 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T07:53:14.265 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:53:14.265 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T07:53:14.320 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:53:14.320 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:53:14.320 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:53:14.375 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:53:14.440 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:53:14.496 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:53:14.496 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:53:14.496 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:53:14.497 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:53:14.497 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:53:14.497 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:53:14.497 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:53:14.497 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:53:14.497 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:53:14.497 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:53:14.497 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:53:14.497 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:53:14.497 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:53:14.497 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:53:14.497 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:53:14.497 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:53:14.498 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:53:14.498 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:53:14.498 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:53:14.498 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:53:14.498 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:53:14.498 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:53:14.498 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:53:14.498 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:53:14.498 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:53:14.498 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:53:14.498 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:53:14.498 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:53:14.498 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:53:14.498 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:53:14.499 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:53:14.499 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:53:14.499 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:53:14.499 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:53:14.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:14.499 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:53:14.552 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:53:14.552 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T07:53:14.594 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:53:14.618 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:53:14.677 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:53:14.677 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:53:14.677 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:53:14.677 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:53:14.677 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:53:14.677 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:53:14.678 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:53:14.678 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:53:14.678 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:53:14.678 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:53:14.678 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:53:14.678 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:53:14.678 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:53:14.678 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:53:14.678 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:53:14.678 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:53:14.679 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:53:14.679 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:53:14.679 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:53:14.679 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:53:14.679 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:53:14.679 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:53:14.679 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:53:14.679 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:53:14.679 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:53:14.679 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:53:14.679 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:53:14.680 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:53:14.680 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:53:14.680 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:53:14.680 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:53:14.680 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:53:14.680 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:53:14.680 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:53:14.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:14.681 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:53:14.682 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2024-09-19T07:53:14.675+0000 7fdc5dc47480 -1 init, newargv = 0x55c49a077670 newargc=15 2024-09-19T07:53:14.682 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[47679]: starting ceph client 2024-09-19T07:53:14.682 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[47679]: starting fuse 2024-09-19T07:53:14.696 INFO:teuthology.orchestra.run.smithi146.stdout:42 2024-09-19T07:53:14.696 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-19T07:53:14.696 DEBUG:teuthology.orchestra.run.smithi146:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:53:14.697 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2024-09-19T07:53:14.697 DEBUG:teuthology.orchestra.run.smithi146:> import re 2024-09-19T07:53:14.697 DEBUG:teuthology.orchestra.run.smithi146:> import os 2024-09-19T07:53:14.697 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2024-09-19T07:53:14.697 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:53:14.697 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2024-09-19T07:53:14.697 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T07:53:14.697 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2024-09-19T07:53:14.697 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T07:53:14.697 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:53:14.697 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2024-09-19T07:53:14.698 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2024-09-19T07:53:14.698 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2024-09-19T07:53:14.698 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2024-09-19T07:53:14.698 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:53:14.698 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2024-09-19T07:53:14.698 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:53:14.698 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:53:14.698 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:53:14.698 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2024-09-19T07:53:14.698 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2024-09-19T07:53:14.698 DEBUG:teuthology.orchestra.run.smithi146:> return f 2024-09-19T07:53:14.698 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:53:14.698 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:53:14.699 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2024-09-19T07:53:14.699 DEBUG:teuthology.orchestra.run.smithi146:> ' 2024-09-19T07:53:14.813 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.47679.asok 2024-09-19T07:53:14.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:53:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:53:14.823 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.47679.asok 2024-09-19T07:53:14.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:14.823 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.47679.asok status 2024-09-19T07:53:14.955 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-19T07:53:14.955 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2024-09-19T07:53:14.956 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:53:14.956 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:53:14.956 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2024-09-19T07:53:14.956 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2024-09-19T07:53:14.956 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T07:53:14.956 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "47679", 2024-09-19T07:53:14.956 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2024-09-19T07:53:14.956 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:53:14.956 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2024-09-19T07:53:14.956 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2024-09-19T07:53:14.956 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 6700, 2024-09-19T07:53:14.956 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2024-09-19T07:53:14.956 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2024-09-19T07:53:14.957 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2024-09-19T07:53:14.957 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 6700 2024-09-19T07:53:14.957 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:53:14.957 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:53:14.957 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:53:14.957 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:53:14.957 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 1089738884 2024-09-19T07:53:14.957 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-19T07:53:14.957 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:53:14.957 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:53:14.957 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:53:14.957 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:53:14.957 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 1089738884 2024-09-19T07:53:14.958 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:53:14.958 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.6700 192.168.0.1:0/1089738884", 2024-09-19T07:53:14.958 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/1089738884", 2024-09-19T07:53:14.958 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2024-09-19T07:53:14.958 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 56, 2024-09-19T07:53:14.958 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 106, 2024-09-19T07:53:14.958 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:53:14.958 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2024-09-19T07:53:14.958 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2024-09-19T07:53:14.958 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-19T07:53:14.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:14.964 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2024-09-19T07:53:14.964 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T07:53:14.979 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2024-09-19T07:53:14.980 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T07:53:14.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:14.980 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:53:15.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T07:53:15.048 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T07:53:15.100 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-09-19T07:53:15.100 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2024-09-19T07:53:15.100 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T07:53:15.100 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2024-09-19T07:53:15.100 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.100 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2024-09-19T07:53:15.100 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.101 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2024-09-19T07:53:15.101 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.101 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2024-09-19T07:53:15.101 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2024-09-19T07:53:15.101 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.101 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2024-09-19T07:53:15.101 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2024-09-19T07:53:15.101 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.101 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T07:53:15.101 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2024-09-19T07:53:15.101 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.101 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T07:53:15.101 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T07:53:15.102 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.102 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2024-09-19T07:53:15.102 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.102 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2024-09-19T07:53:15.102 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2024-09-19T07:53:15.102 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2024-09-19T07:53:15.102 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2024-09-19T07:53:15.102 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2024-09-19T07:53:15.102 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2024-09-19T07:53:15.102 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2024-09-19T07:53:15.102 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T07:53:15.102 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T07:53:15.102 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2024-09-19T07:53:15.102 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.103 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T07:53:15.103 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.103 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2024-09-19T07:53:15.103 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.103 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2024-09-19T07:53:15.103 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.103 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2024-09-19T07:53:15.103 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2024-09-19T07:53:15.103 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2024-09-19T07:53:15.103 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-09-19T07:53:15.103 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb 2024-09-19T07:53:15.103 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.103 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2024-09-19T07:53:15.103 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2024-09-19T07:53:15.103 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.104 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2024-09-19T07:53:15.104 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2024-09-19T07:53:15.104 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2024-09-19T07:53:15.104 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.104 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2024-09-19T07:53:15.104 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2024-09-19T07:53:15.104 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.104 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2024-09-19T07:53:15.104 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2024-09-19T07:53:15.104 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2024-09-19T07:53:15.104 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2024-09-19T07:53:15.104 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.104 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2024-09-19T07:53:15.105 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2024-09-19T07:53:15.105 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.105 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2024-09-19T07:53:15.105 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.105 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2024-09-19T07:53:15.105 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.105 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2024-09-19T07:53:15.105 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.105 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2024-09-19T07:53:15.105 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2024-09-19T07:53:15.105 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.105 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2024-09-19T07:53:15.105 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2024-09-19T07:53:15.105 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2024-09-19T07:53:15.106 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2024-09-19T07:53:15.106 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2024-09-19T07:53:15.106 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2024-09-19T07:53:15.106 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:53:15.106 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2024-09-19T07:53:15.106 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2024-09-19T07:53:15.106 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T07:53:15.106 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2024-09-19T07:53:15.106 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2024-09-19T07:53:15.106 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2024-09-19T07:53:15.106 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2024-09-19T07:53:15.106 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-09-19T07:53:15.106 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.106 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2024-09-19T07:53:15.107 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:53:15.107 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2024-09-19T07:53:15.107 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-19T07:53:15.107 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-19T07:53:15.107 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:53:15.107 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:53:15.107 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2024-09-19T07:53:15.107 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-09-19T07:53:15.107 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.107 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2024-09-19T07:53:15.107 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:53:15.107 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-19T07:53:15.107 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2024-09-19T07:53:15.108 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-19T07:53:15.108 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2024-09-19T07:53:15.108 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2024-09-19T07:53:15.108 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:53:15.108 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2024-09-19T07:53:15.108 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2024-09-19T07:53:15.108 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:53:15.108 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.108 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T07:53:15.108 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T07:53:15.108 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T07:53:15.108 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.108 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T07:53:15.108 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T07:53:15.108 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T07:53:15.109 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.109 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2024-09-19T07:53:15.109 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2024-09-19T07:53:15.109 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.109 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2024-09-19T07:53:15.109 INFO:teuthology.orchestra.run.smithi016.stdout: mon down mkfs grace = 300 2024-09-19T07:53:15.109 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2024-09-19T07:53:15.109 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on pool no app = False 2024-09-19T07:53:15.109 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:15.109 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2024-09-19T07:53:15.109 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2024-09-19T07:53:15.109 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2024-09-19T07:53:15.109 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2024-09-19T07:53:15.109 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T07:53:15.110 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T07:53:15.110 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2024-09-19T07:53:15.110 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-09-19T07:53:15.110 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:53:15.110 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2024-09-19T07:53:15.110 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-09-19T07:53:15.110 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-09-19T07:53:15.110 INFO:teuthology.orchestra.run.smithi016.stdout: client inject fixed oldest tid = true 2024-09-19T07:53:15.110 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2024-09-19T07:53:15.110 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2024-09-19T07:53:15.110 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2024-09-19T07:53:15.110 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-09-19T07:53:15.110 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-09-19T07:53:15.110 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-09-19T07:53:15.110 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-09-19T07:53:15.111 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2024-09-19T07:53:15.111 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2024-09-19T07:53:15.111 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2024-09-19T07:53:15.111 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:53:15.111 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2024-09-19T07:53:15.111 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2024-09-19T07:53:15.111 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2024-09-19T07:53:15.111 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = True 2024-09-19T07:53:15.111 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-09-19T07:53:15.111 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-09-19T07:53:15.111 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-09-19T07:53:15.128 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2024-09-19T07:53:15.128 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2024-09-19T07:53:15.128 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T07:53:15.128 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2024-09-19T07:53:15.128 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.128 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2024-09-19T07:53:15.128 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.128 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2024-09-19T07:53:15.128 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.128 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2024-09-19T07:53:15.128 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2024-09-19T07:53:15.129 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.129 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2024-09-19T07:53:15.129 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2024-09-19T07:53:15.129 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.129 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T07:53:15.129 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2024-09-19T07:53:15.129 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.129 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T07:53:15.129 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T07:53:15.129 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.129 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2024-09-19T07:53:15.129 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.129 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2024-09-19T07:53:15.129 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2024-09-19T07:53:15.130 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2024-09-19T07:53:15.130 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2024-09-19T07:53:15.130 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2024-09-19T07:53:15.130 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2024-09-19T07:53:15.130 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2024-09-19T07:53:15.130 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T07:53:15.130 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T07:53:15.130 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2024-09-19T07:53:15.130 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.130 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T07:53:15.130 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.130 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2024-09-19T07:53:15.130 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.130 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2024-09-19T07:53:15.131 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.131 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2024-09-19T07:53:15.131 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2024-09-19T07:53:15.131 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2024-09-19T07:53:15.131 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.16,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-09-19T07:53:15.131 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = 6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb 2024-09-19T07:53:15.131 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.131 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2024-09-19T07:53:15.131 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2024-09-19T07:53:15.131 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.131 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2024-09-19T07:53:15.131 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2024-09-19T07:53:15.131 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2024-09-19T07:53:15.131 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.132 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2024-09-19T07:53:15.132 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2024-09-19T07:53:15.132 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.132 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2024-09-19T07:53:15.132 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2024-09-19T07:53:15.132 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2024-09-19T07:53:15.132 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug trim objects = true 2024-09-19T07:53:15.132 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.132 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2024-09-19T07:53:15.132 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2024-09-19T07:53:15.132 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.132 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2024-09-19T07:53:15.132 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.132 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2024-09-19T07:53:15.133 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.133 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2024-09-19T07:53:15.133 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.133 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2024-09-19T07:53:15.133 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2024-09-19T07:53:15.133 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.133 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2024-09-19T07:53:15.133 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2024-09-19T07:53:15.133 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2024-09-19T07:53:15.133 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2024-09-19T07:53:15.133 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 20 2024-09-19T07:53:15.133 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 20 2024-09-19T07:53:15.133 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:53:15.133 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2024-09-19T07:53:15.133 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 10 2024-09-19T07:53:15.134 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T07:53:15.134 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2024-09-19T07:53:15.134 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2024-09-19T07:53:15.134 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2024-09-19T07:53:15.134 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2024-09-19T07:53:15.134 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-09-19T07:53:15.134 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.134 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2024-09-19T07:53:15.134 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:53:15.134 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2024-09-19T07:53:15.134 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-19T07:53:15.134 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-19T07:53:15.134 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:53:15.134 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:53:15.135 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2024-09-19T07:53:15.135 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-09-19T07:53:15.135 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.135 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2024-09-19T07:53:15.135 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:53:15.135 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-19T07:53:15.135 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2024-09-19T07:53:15.135 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-19T07:53:15.135 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2024-09-19T07:53:15.135 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2024-09-19T07:53:15.135 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:53:15.135 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2024-09-19T07:53:15.135 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2024-09-19T07:53:15.135 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:53:15.136 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.136 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T07:53:15.136 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T07:53:15.136 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T07:53:15.136 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.136 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T07:53:15.136 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T07:53:15.136 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T07:53:15.136 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.136 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2024-09-19T07:53:15.136 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2024-09-19T07:53:15.136 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.136 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2024-09-19T07:53:15.136 INFO:teuthology.orchestra.run.smithi146.stdout: mon down mkfs grace = 300 2024-09-19T07:53:15.137 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2024-09-19T07:53:15.137 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on pool no app = False 2024-09-19T07:53:15.137 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:15.137 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2024-09-19T07:53:15.137 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2024-09-19T07:53:15.137 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2024-09-19T07:53:15.137 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2024-09-19T07:53:15.137 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T07:53:15.137 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T07:53:15.137 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2024-09-19T07:53:15.137 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-09-19T07:53:15.137 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:53:15.137 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = "" 2024-09-19T07:53:15.137 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-09-19T07:53:15.137 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-09-19T07:53:15.138 INFO:teuthology.orchestra.run.smithi146.stdout: client inject fixed oldest tid = true 2024-09-19T07:53:15.138 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2024-09-19T07:53:15.138 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2024-09-19T07:53:15.138 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2024-09-19T07:53:15.138 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2024-09-19T07:53:15.138 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2024-09-19T07:53:15.138 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2024-09-19T07:53:15.138 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2024-09-19T07:53:15.138 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2024-09-19T07:53:15.138 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2024-09-19T07:53:15.138 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2024-09-19T07:53:15.138 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:53:15.138 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2024-09-19T07:53:15.138 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2024-09-19T07:53:15.139 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2024-09-19T07:53:15.139 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2024-09-19T07:53:15.139 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-09-19T07:53:15.139 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-09-19T07:53:15.139 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-09-19T07:53:15.140 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:53:15.140 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:53:15.140 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:53:15.173 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:53:15.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.322+0000 7f90dafb7640 1 Processor -- start 2024-09-19T07:53:15.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.322+0000 7f90dafb7640 1 -- start start 2024-09-19T07:53:15.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.323+0000 7f90dafb7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90d4161e00 0x7f90d40cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:15.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.323+0000 7f90dafb7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90d40ce670 0x7f90d40cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:15.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.323+0000 7f90dafb7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90d40d0230 con 0x7f90d40cf000 2024-09-19T07:53:15.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.323+0000 7f90dafb7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f90d40d32d0 con 0x7f90d4161e00 2024-09-19T07:53:15.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.323+0000 7f90dafb7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f90d40b5e20 con 0x7f90d40ce670 2024-09-19T07:53:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.323+0000 7f90d8d53640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90d4161e00 0x7f90d40cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.323+0000 7f90d9554640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f90d40cf000 0x7f90d40d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45676/0 (socket says 172.21.15.16:45676) 2024-09-19T07:53:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.323+0000 7f90d9554640 1 -- 172.21.15.16:0/3013196293 learned_addr learned my addr 172.21.15.16:0/3013196293 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.323+0000 7f90d3fff640 1 --2- 172.21.15.16:0/3013196293 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90d40ce670 0x7f90d40cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.323+0000 7f90d37fe640 1 -- 172.21.15.16:0/3013196293 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 660481989 0 0) 0x7f90d40d0230 con 0x7f90d40cf000 2024-09-19T07:53:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.323+0000 7f90d37fe640 1 -- 172.21.15.16:0/3013196293 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90c4003520 con 0x7f90d40cf000 2024-09-19T07:53:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.323+0000 7f90d8d53640 1 -- 172.21.15.16:0/3013196293 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90d40ce670 msgr2=0x7f90d40cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.323+0000 7f90d8d53640 1 --2- 172.21.15.16:0/3013196293 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90d40ce670 0x7f90d40cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.323+0000 7f90d8d53640 1 -- 172.21.15.16:0/3013196293 >> v1:172.21.15.16:6789/0 conn(0x7f90d40cf000 legacy=0x7f90d40d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.323+0000 7f90d8d53640 1 -- 172.21.15.16:0/3013196293 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d40b60e0 con 0x7f90d4161e00 2024-09-19T07:53:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.324+0000 7f90d8d53640 1 --2- 172.21.15.16:0/3013196293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d4161e00 0x7f90d40cb5b0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f90c80097a0 tx=0x7f90c802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3b3a6978065163 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.324+0000 7f90d37fe640 1 -- 172.21.15.16:0/3013196293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8037070 con 0x7f90d4161e00 2024-09-19T07:53:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.324+0000 7f90d37fe640 1 -- 172.21.15.16:0/3013196293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c802ae50 con 0x7f90d4161e00 2024-09-19T07:53:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.324+0000 7f90d37fe640 1 -- 172.21.15.16:0/3013196293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8032660 con 0x7f90d4161e00 2024-09-19T07:53:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.325+0000 7f90dafb7640 1 -- 172.21.15.16:0/3013196293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d4161e00 msgr2=0x7f90d40cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.325+0000 7f90dafb7640 1 --2- 172.21.15.16:0/3013196293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d4161e00 0x7f90d40cb5b0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f90c80097a0 tx=0x7f90c802a100 comp rx=0 tx=0).stop 2024-09-19T07:53:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.325+0000 7f90dafb7640 1 -- 172.21.15.16:0/3013196293 shutdown_connections 2024-09-19T07:53:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.325+0000 7f90dafb7640 1 --2- 172.21.15.16:0/3013196293 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90d40ce670 0x7f90d40cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.325+0000 7f90dafb7640 1 --2- 172.21.15.16:0/3013196293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d4161e00 0x7f90d40cb5b0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.325+0000 7f90dafb7640 1 -- 172.21.15.16:0/3013196293 >> 172.21.15.16:0/3013196293 conn(0x7f90d408bfe0 msgr2=0x7f90d40d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.325+0000 7f90dafb7640 1 -- 172.21.15.16:0/3013196293 shutdown_connections 2024-09-19T07:53:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.325+0000 7f90dafb7640 1 -- 172.21.15.16:0/3013196293 wait complete. 2024-09-19T07:53:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.326+0000 7f90dafb7640 1 Processor -- start 2024-09-19T07:53:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.326+0000 7f90dafb7640 1 -- start start 2024-09-19T07:53:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.327+0000 7f90dafb7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90d40ce670 0x7f90d4156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.327+0000 7f90dafb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d40cf000 0x7f90d40cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.327+0000 7f90dafb7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d4161e00 0x7f90d41e95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.327+0000 7f90dafb7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f90d40b6980 con 0x7f90d40cf000 2024-09-19T07:53:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.327+0000 7f90dafb7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f90d40b66a0 con 0x7f90d4161e00 2024-09-19T07:53:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.327+0000 7f90dafb7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f90d40d3e70 con 0x7f90d40ce670 2024-09-19T07:53:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.327+0000 7f90d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d40cf000 0x7f90d40cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.327+0000 7f90d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d40cf000 0x7f90d40cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38806/0 (socket says 172.21.15.16:38806) 2024-09-19T07:53:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.327+0000 7f90d3fff640 1 -- 172.21.15.16:0/543509917 learned_addr learned my addr 172.21.15.16:0/543509917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.327+0000 7f90d8d53640 1 --2- 172.21.15.16:0/543509917 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90d40ce670 0x7f90d4156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.327+0000 7f90d9554640 1 --2- 172.21.15.16:0/543509917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d4161e00 0x7f90d41e95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.327+0000 7f90d3fff640 1 -- 172.21.15.16:0/543509917 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90d40ce670 msgr2=0x7f90d4156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.327+0000 7f90d3fff640 1 --2- 172.21.15.16:0/543509917 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90d40ce670 0x7f90d4156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.327+0000 7f90d3fff640 1 -- 172.21.15.16:0/543509917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d4161e00 msgr2=0x7f90d41e95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.327+0000 7f90d3fff640 1 --2- 172.21.15.16:0/543509917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d4161e00 0x7f90d41e95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.327+0000 7f90d3fff640 1 -- 172.21.15.16:0/543509917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d4156a90 con 0x7f90d40cf000 2024-09-19T07:53:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.328+0000 7f90d3fff640 1 --2- 172.21.15.16:0/543509917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d40cf000 0x7f90d40cdc10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f90c000e870 tx=0x7f90c000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.328+0000 7f90d17fa640 1 -- 172.21.15.16:0/543509917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0017070 con 0x7f90d40cf000 2024-09-19T07:53:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.328+0000 7f90d17fa640 1 -- 172.21.15.16:0/543509917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c000c940 con 0x7f90d40cf000 2024-09-19T07:53:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.328+0000 7f90dafb7640 1 -- 172.21.15.16:0/543509917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d41e8540 con 0x7f90d40cf000 2024-09-19T07:53:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.328+0000 7f90d17fa640 1 -- 172.21.15.16:0/543509917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c000cd10 con 0x7f90d40cf000 2024-09-19T07:53:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.328+0000 7f90dafb7640 1 -- 172.21.15.16:0/543509917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d41e8b00 con 0x7f90d40cf000 2024-09-19T07:53:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.329+0000 7f90dafb7640 1 -- 172.21.15.16:0/543509917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90d41613f0 con 0x7f90d40cf000 2024-09-19T07:53:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.329+0000 7f90d17fa640 1 -- 172.21.15.16:0/543509917 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f90c001a070 con 0x7f90d40cf000 2024-09-19T07:53:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.330+0000 7f90d17fa640 1 --2- 172.21.15.16:0/543509917 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f90b806c2b0 0x7f90b806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.330+0000 7f90d17fa640 1 -- 172.21.15.16:0/543509917 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f90c008b320 con 0x7f90d40cf000 2024-09-19T07:53:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.330+0000 7f90d8d53640 1 --2- 172.21.15.16:0/543509917 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f90b806c2b0 0x7f90b806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.330+0000 7f90d8d53640 1 --2- 172.21.15.16:0/543509917 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f90b806c2b0 0x7f90b806e770 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f90c8034000 tx=0x7f90c802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:15.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.334+0000 7f90d17fa640 1 -- 172.21.15.16:0/543509917 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f90c00592b0 con 0x7f90d40cf000 2024-09-19T07:53:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.521+0000 7f90dafb7640 1 -- 172.21.15.16:0/543509917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f90d40d4f20 con 0x7f90d40cf000 2024-09-19T07:53:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.521+0000 7f90d17fa640 1 -- 172.21.15.16:0/543509917 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) ==== 80+0+0 (secure 0 0 0) 0x7f90c005d5b0 con 0x7f90d40cf000 2024-09-19T07:53:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:53:15.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.523+0000 7f90dafb7640 1 -- 172.21.15.16:0/543509917 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f90b806c2b0 msgr2=0x7f90b806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:15.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.523+0000 7f90dafb7640 1 --2- 172.21.15.16:0/543509917 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f90b806c2b0 0x7f90b806e770 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f90c8034000 tx=0x7f90c802e000 comp rx=0 tx=0).stop 2024-09-19T07:53:15.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.523+0000 7f90dafb7640 1 -- 172.21.15.16:0/543509917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d40cf000 msgr2=0x7f90d40cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:15.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.523+0000 7f90dafb7640 1 --2- 172.21.15.16:0/543509917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d40cf000 0x7f90d40cdc10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f90c000e870 tx=0x7f90c000ed30 comp rx=0 tx=0).stop 2024-09-19T07:53:15.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.524+0000 7f90dafb7640 1 -- 172.21.15.16:0/543509917 shutdown_connections 2024-09-19T07:53:15.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.524+0000 7f90dafb7640 1 --2- 172.21.15.16:0/543509917 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f90b806c2b0 0x7f90b806e770 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:15.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.524+0000 7f90dafb7640 1 --2- 172.21.15.16:0/543509917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d4161e00 0x7f90d41e95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:15.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.524+0000 7f90dafb7640 1 --2- 172.21.15.16:0/543509917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d40cf000 0x7f90d40cdc10 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:15.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.524+0000 7f90dafb7640 1 --2- 172.21.15.16:0/543509917 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90d40ce670 0x7f90d4156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:15.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.524+0000 7f90dafb7640 1 -- 172.21.15.16:0/543509917 >> 172.21.15.16:0/543509917 conn(0x7f90d408bfe0 msgr2=0x7f90d40c8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:15.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.524+0000 7f90dafb7640 1 -- 172.21.15.16:0/543509917 shutdown_connections 2024-09-19T07:53:15.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.524+0000 7f90dafb7640 1 -- 172.21.15.16:0/543509917 wait complete. 2024-09-19T07:53:15.536 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-19T07:53:15.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:15.536 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:15.560 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[65864]: fuse finished with error 0 and tester_r 0 2024-09-19T07:53:15.562 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:53:15.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:15.562 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:53:15.729 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:53:15.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.870+0000 7f529c318640 1 Processor -- start 2024-09-19T07:53:15.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.870+0000 7f529c318640 1 -- start start 2024-09-19T07:53:15.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.871+0000 7f529c318640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52940704c0 0x7f52940d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:15.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.871+0000 7f529c318640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f529409ab50 0x7f529409b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:15.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.871+0000 7f529c318640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52940713a0 con 0x7f529415f6b0 2024-09-19T07:53:15.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.871+0000 7f529c318640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f529415eb00 con 0x7f529409ab50 2024-09-19T07:53:15.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.871+0000 7f529c318640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f52940d16e0 con 0x7f52940704c0 2024-09-19T07:53:15.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.871+0000 7f529a8b5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f529409ab50 0x7f529409b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:15.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.871+0000 7f529a8b5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f529409ab50 0x7f529409b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38822/0 (socket says 172.21.15.16:38822) 2024-09-19T07:53:15.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.871+0000 7f52998b3640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f529415f6b0 0x7f52940708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45678/0 (socket says 172.21.15.16:45678) 2024-09-19T07:53:15.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.871+0000 7f529a8b5640 1 -- 172.21.15.16:0/1493276006 learned_addr learned my addr 172.21.15.16:0/1493276006 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:15.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.872+0000 7f529a0b4640 1 --2- 172.21.15.16:0/1493276006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52940704c0 0x7f52940d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:15.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.872+0000 7f529a8b5640 1 -- 172.21.15.16:0/1493276006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52940704c0 msgr2=0x7f52940d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:15.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.872+0000 7f529a8b5640 1 --2- 172.21.15.16:0/1493276006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52940704c0 0x7f52940d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:15.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.872+0000 7f529a8b5640 1 -- 172.21.15.16:0/1493276006 >> v1:172.21.15.16:6789/0 conn(0x7f529415f6b0 legacy=0x7f52940708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:15.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.872+0000 7f529a8b5640 1 -- 172.21.15.16:0/1493276006 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52940d0ad0 con 0x7f529409ab50 2024-09-19T07:53:15.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.872+0000 7f52990b2640 1 -- 172.21.15.16:0/1493276006 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3025035208 0 0) 0x7f52940713a0 con 0x7f529415f6b0 2024-09-19T07:53:15.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.872+0000 7f529a0b4640 1 --2- 172.21.15.16:0/1493276006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52940704c0 0x7f52940d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:15.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.872+0000 7f529a8b5640 1 --2- 172.21.15.16:0/1493276006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529409ab50 0x7f529409b000 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f528c009e80 tx=0x7f528c02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8fe5d518372ed1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.873+0000 7f52990b2640 1 -- 172.21.15.16:0/1493276006 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c036070 con 0x7f529409ab50 2024-09-19T07:53:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.873+0000 7f52990b2640 1 -- 172.21.15.16:0/1493276006 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f528c031600 con 0x7f529409ab50 2024-09-19T07:53:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.873+0000 7f52990b2640 1 -- 172.21.15.16:0/1493276006 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c0319d0 con 0x7f529409ab50 2024-09-19T07:53:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.873+0000 7f529c318640 1 -- 172.21.15.16:0/1493276006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529409ab50 msgr2=0x7f529409b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.873+0000 7f529c318640 1 --2- 172.21.15.16:0/1493276006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529409ab50 0x7f529409b000 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f528c009e80 tx=0x7f528c02a750 comp rx=0 tx=0).stop 2024-09-19T07:53:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.873+0000 7f529c318640 1 -- 172.21.15.16:0/1493276006 shutdown_connections 2024-09-19T07:53:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.873+0000 7f529c318640 1 --2- 172.21.15.16:0/1493276006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529409ab50 0x7f529409b000 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.873+0000 7f529c318640 1 --2- 172.21.15.16:0/1493276006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52940704c0 0x7f52940d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.873+0000 7f529c318640 1 -- 172.21.15.16:0/1493276006 >> 172.21.15.16:0/1493276006 conn(0x7f52940b3e00 msgr2=0x7f52940b4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.873+0000 7f529c318640 1 -- 172.21.15.16:0/1493276006 shutdown_connections 2024-09-19T07:53:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.873+0000 7f529c318640 1 -- 172.21.15.16:0/1493276006 wait complete. 2024-09-19T07:53:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.874+0000 7f529c318640 1 Processor -- start 2024-09-19T07:53:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.874+0000 7f529c318640 1 -- start start 2024-09-19T07:53:15.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.875+0000 7f529c318640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52940704c0 0x7f52941f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:15.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.875+0000 7f529c318640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529409ab50 0x7f5294161990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:15.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.875+0000 7f529c318640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f529415f6b0 0x7f52940cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:15.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.875+0000 7f529c318640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f52940d1370 con 0x7f529409ab50 2024-09-19T07:53:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.875+0000 7f529c318640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f52940d1090 con 0x7f52940704c0 2024-09-19T07:53:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.875+0000 7f529c318640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f529415faf0 con 0x7f529415f6b0 2024-09-19T07:53:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.875+0000 7f52998b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529409ab50 0x7f5294161990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.875+0000 7f52998b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529409ab50 0x7f5294161990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38832/0 (socket says 172.21.15.16:38832) 2024-09-19T07:53:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.875+0000 7f52998b3640 1 -- 172.21.15.16:0/227870438 learned_addr learned my addr 172.21.15.16:0/227870438 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.875+0000 7f529a8b5640 1 --2- 172.21.15.16:0/227870438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f529415f6b0 0x7f52940cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.875+0000 7f529a0b4640 1 --2- 172.21.15.16:0/227870438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52940704c0 0x7f52941f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.876+0000 7f52998b3640 1 -- 172.21.15.16:0/227870438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f529415f6b0 msgr2=0x7f52940cabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.876+0000 7f52998b3640 1 --2- 172.21.15.16:0/227870438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f529415f6b0 0x7f52940cabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.876+0000 7f52998b3640 1 -- 172.21.15.16:0/227870438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52940704c0 msgr2=0x7f52941f8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.876+0000 7f52998b3640 1 --2- 172.21.15.16:0/227870438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52940704c0 0x7f52941f8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.876+0000 7f52998b3640 1 -- 172.21.15.16:0/227870438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529415ff00 con 0x7f529409ab50 2024-09-19T07:53:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.876+0000 7f52998b3640 1 --2- 172.21.15.16:0/227870438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529409ab50 0x7f5294161990 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f528800c4b0 tx=0x7f528800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.876+0000 7f5292ffd640 1 -- 172.21.15.16:0/227870438 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288018070 con 0x7f529409ab50 2024-09-19T07:53:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.876+0000 7f5292ffd640 1 -- 172.21.15.16:0/227870438 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5288007500 con 0x7f529409ab50 2024-09-19T07:53:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.876+0000 7f529c318640 1 -- 172.21.15.16:0/227870438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52941601a0 con 0x7f529409ab50 2024-09-19T07:53:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.876+0000 7f5292ffd640 1 -- 172.21.15.16:0/227870438 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288004a20 con 0x7f529409ab50 2024-09-19T07:53:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.876+0000 7f529c318640 1 -- 172.21.15.16:0/227870438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5294161ed0 con 0x7f529409ab50 2024-09-19T07:53:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.877+0000 7f529c318640 1 -- 172.21.15.16:0/227870438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52940d04a0 con 0x7f529409ab50 2024-09-19T07:53:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.877+0000 7f5292ffd640 1 -- 172.21.15.16:0/227870438 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f528801b070 con 0x7f529409ab50 2024-09-19T07:53:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.878+0000 7f5292ffd640 1 --2- 172.21.15.16:0/227870438 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f528006c2b0 0x7f528006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.878+0000 7f5292ffd640 1 -- 172.21.15.16:0/227870438 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f528808a7f0 con 0x7f529409ab50 2024-09-19T07:53:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.878+0000 7f529a0b4640 1 --2- 172.21.15.16:0/227870438 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f528006c2b0 0x7f528006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.878+0000 7f529a0b4640 1 --2- 172.21.15.16:0/227870438 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f528006c2b0 0x7f528006e770 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f528400a950 tx=0x7f5284008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:15.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:15.882+0000 7f5292ffd640 1 -- 172.21.15.16:0/227870438 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5288058f20 con 0x7f529409ab50 2024-09-19T07:53:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:16.096+0000 7f529c318640 1 -- 172.21.15.16:0/227870438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f529415e290 con 0x7f529409ab50 2024-09-19T07:53:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:16.096+0000 7f5292ffd640 1 -- 172.21.15.16:0/227870438 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) ==== 53+0+73 (secure 0 0 0) 0x7f5288058820 con 0x7f529409ab50 2024-09-19T07:53:16.097 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:53:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:16.098+0000 7f529c318640 1 -- 172.21.15.16:0/227870438 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f528006c2b0 msgr2=0x7f528006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:16.098+0000 7f529c318640 1 --2- 172.21.15.16:0/227870438 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f528006c2b0 0x7f528006e770 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f528400a950 tx=0x7f5284008040 comp rx=0 tx=0).stop 2024-09-19T07:53:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:16.098+0000 7f529c318640 1 -- 172.21.15.16:0/227870438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529409ab50 msgr2=0x7f5294161990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:16.098+0000 7f529c318640 1 --2- 172.21.15.16:0/227870438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529409ab50 0x7f5294161990 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f528800c4b0 tx=0x7f528800c970 comp rx=0 tx=0).stop 2024-09-19T07:53:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:16.098+0000 7f529c318640 1 -- 172.21.15.16:0/227870438 shutdown_connections 2024-09-19T07:53:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:16.098+0000 7f529c318640 1 --2- 172.21.15.16:0/227870438 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f528006c2b0 0x7f528006e770 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:16.098+0000 7f529c318640 1 --2- 172.21.15.16:0/227870438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f529415f6b0 0x7f52940cabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:16.098+0000 7f529c318640 1 --2- 172.21.15.16:0/227870438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529409ab50 0x7f5294161990 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:16.098+0000 7f529c318640 1 --2- 172.21.15.16:0/227870438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52940704c0 0x7f52941f8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:16.098+0000 7f529c318640 1 -- 172.21.15.16:0/227870438 >> 172.21.15.16:0/227870438 conn(0x7f52940b3e00 msgr2=0x7f529406e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:16.098+0000 7f529c318640 1 -- 172.21.15.16:0/227870438 shutdown_connections 2024-09-19T07:53:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:16.098+0000 7f529c318640 1 -- 172.21.15.16:0/227870438 wait complete. 2024-09-19T07:53:16.110 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:53:16.110 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:53:16.110 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T07:53:16.110 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T07:53:16.110 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:16.110 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:53:16.110 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:53:16.110 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:53:16.110 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:53:16.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:16.111 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T07:53:16.127 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:53:16.127 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:53:16.127 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:53:16.127 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:16.127 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:53:16.127 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:16.127 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:53:16.127 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:16.127 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T07:53:16.127 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:53:16.128 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:16.128 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T07:53:16.128 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:53:16.128 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:16.128 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T07:53:16.128 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T07:53:16.128 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42193sec preferred_lft 42193sec 2024-09-19T07:53:16.128 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T07:53:16.128 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:16.128 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:53:16.128 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:16.128 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T07:53:16.128 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:53:16.129 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:16.129 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:53:16.129 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:16.129 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::32:7bff:fe92:8895/64 scope link 2024-09-19T07:53:16.129 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:16.129 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:53:16.129 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether e6:fd:21:6a:43:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:16.129 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e4fd:21ff:fe6a:4369/64 scope link 2024-09-19T07:53:16.129 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:16.129 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:53:16.129 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:16.129 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::80f4:81ff:fe7d:b7f0/64 scope link 2024-09-19T07:53:16.129 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:16.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:16.130 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:53:16.191 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:53:16.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:16.193 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T07:53:16.250 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T07:53:16.250 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:53:16.250 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T07:53:16.250 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T07:53:16.250 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:53:16.251 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:53:16.251 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:53:16.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T07:53:16.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T07:53:16.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T07:53:16.251 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:53:16.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:53:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:53:16.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:16.379 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:53:16.395 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T07:53:16.395 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:53:16.395 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:53:16.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:16.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-19T07:53:16.461 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T07:53:16.461 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:53:16.461 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:16.517 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:53:16.517 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:53:16.518 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:16.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:53:16.640 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:53:16.697 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:53:16.697 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:53:16.697 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:53:16.698 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:53:16.698 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:53:16.698 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:53:16.698 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:53:16.698 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:53:16.698 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:53:16.698 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:53:16.698 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:53:16.698 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:53:16.698 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:53:16.698 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:53:16.699 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:53:16.699 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:53:16.699 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:53:16.699 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:53:16.699 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:53:16.699 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:53:16.699 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:53:16.699 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:53:16.699 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:53:16.699 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:53:16.699 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:53:16.699 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:53:16.700 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:53:16.700 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:53:16.700 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:53:16.700 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:53:16.700 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:16.700 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:16.700 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:16.700 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:16.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:16.701 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:53:16.754 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:53:16.754 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T07:53:16.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:53:16.822 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:53:16.880 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:53:16.952 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:53:16.952 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:53:16.952 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:53:16.952 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:53:16.952 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:53:16.952 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:53:16.952 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:53:16.952 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:53:16.952 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:53:16.952 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:53:16.953 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:53:16.953 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:53:16.953 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:53:16.953 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:53:16.953 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:53:16.953 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:53:16.953 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:53:16.953 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:53:16.953 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:53:16.953 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:53:16.953 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:53:16.953 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:53:16.953 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:53:16.954 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:53:16.954 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:53:16.954 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:53:16.954 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:53:16.954 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:53:16.954 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:53:16.954 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:16.954 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:16.954 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:16.954 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:16.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:16.955 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:53:16.956 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-19T07:53:16.880+0000 7f911af86480 -1 init, newargv = 0x5558ecc27890 newargc=15 2024-09-19T07:53:16.956 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[66444]: starting ceph client 2024-09-19T07:53:16.956 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[66444]: starting fuse 2024-09-19T07:53:16.972 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:53:16.972 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-19T07:53:16.972 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:53:16.972 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:53:16.972 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:53:16.972 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:53:16.972 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:53:16.972 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:16.973 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:53:16.973 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:53:16.973 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:53:16.973 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:53:16.973 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:16.973 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:53:16.973 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:53:16.973 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:53:16.973 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:53:16.973 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:16.973 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:53:16.973 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:53:16.973 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:53:16.973 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:53:16.973 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:53:16.974 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:53:16.974 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:53:16.974 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:53:16.974 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:16.974 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:53:16.974 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:53:17.091 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.66444.asok 2024-09-19T07:53:17.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:53:17.102 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.66444.asok 2024-09-19T07:53:17.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:17.103 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.66444.asok status 2024-09-19T07:53:17.240 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:53:17.240 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:53:17.240 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:53:17.240 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:53:17.240 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:53:17.240 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:53:17.240 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:53:17.240 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "66444", 2024-09-19T07:53:17.241 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:53:17.241 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:17.241 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-19T07:53:17.241 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-19T07:53:17.241 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7185, 2024-09-19T07:53:17.241 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:53:17.241 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:53:17.241 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:53:17.241 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 7185 2024-09-19T07:53:17.241 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:17.241 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:53:17.241 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:53:17.242 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:53:17.242 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3748638091 2024-09-19T07:53:17.242 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:17.242 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:17.242 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:53:17.242 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:53:17.242 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:53:17.242 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3748638091 2024-09-19T07:53:17.242 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:17.242 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.7185 192.168.0.1:0/3748638091", 2024-09-19T07:53:17.242 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/3748638091", 2024-09-19T07:53:17.242 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-19T07:53:17.242 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 56, 2024-09-19T07:53:17.242 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 106, 2024-09-19T07:53:17.243 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:53:17.243 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:53:17.243 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:53:17.243 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:53:17.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:17.248 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-19T07:53:17.249 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:17.265 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-19T07:53:17.265 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:17.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:17.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:17.331 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:53:17.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.472+0000 7fec0467b640 1 Processor -- start 2024-09-19T07:53:17.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.472+0000 7fec0467b640 1 -- start start 2024-09-19T07:53:17.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.473+0000 7fec0467b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7febfc070910 0x7febfc070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:17.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.473+0000 7fec0467b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7febfc09b460 0x7febfc0d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:17.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.473+0000 7fec0467b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febfc071230 con 0x7febfc09bb20 2024-09-19T07:53:17.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.473+0000 7fec0467b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7febfc070030 con 0x7febfc070910 2024-09-19T07:53:17.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.473+0000 7fec0467b640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7febfc15ee40 con 0x7febfc09b460 2024-09-19T07:53:17.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.473+0000 7fec02417640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7febfc070910 0x7febfc070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:17.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.473+0000 7fec02417640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7febfc070910 0x7febfc070cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38840/0 (socket says 172.21.15.16:38840) 2024-09-19T07:53:17.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.473+0000 7fec02417640 1 -- 172.21.15.16:0/2877116359 learned_addr learned my addr 172.21.15.16:0/2877116359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:17.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.473+0000 7fec02417640 1 -- 172.21.15.16:0/2877116359 >> v2:172.21.15.146:3300/0 conn(0x7febfc09b460 msgr2=0x7febfc0d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:17.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.473+0000 7fec02417640 1 --2- 172.21.15.16:0/2877116359 >> v2:172.21.15.146:3300/0 conn(0x7febfc09b460 0x7febfc0d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:17.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.473+0000 7fec02417640 1 -- 172.21.15.16:0/2877116359 >> v1:172.21.15.146:6789/0 conn(0x7febfc09bb20 legacy=0x7febfc15f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:17.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.473+0000 7fec02417640 1 -- 172.21.15.16:0/2877116359 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febfc15f9e0 con 0x7febfc070910 2024-09-19T07:53:17.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.474+0000 7fec02417640 1 --2- 172.21.15.16:0/2877116359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febfc070910 0x7febfc070cf0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7febfc072f10 tx=0x7febf802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40cee4d19ae6a222 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:17.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.474+0000 7fec01415640 1 -- 172.21.15.16:0/2877116359 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8037070 con 0x7febfc070910 2024-09-19T07:53:17.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.474+0000 7fec01415640 1 -- 172.21.15.16:0/2877116359 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf8032440 con 0x7febfc070910 2024-09-19T07:53:17.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.474+0000 7fec01415640 1 -- 172.21.15.16:0/2877116359 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8032810 con 0x7febfc070910 2024-09-19T07:53:17.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.474+0000 7fec0467b640 1 -- 172.21.15.16:0/2877116359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febfc070910 msgr2=0x7febfc070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:17.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.475+0000 7fec0467b640 1 --2- 172.21.15.16:0/2877116359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febfc070910 0x7febfc070cf0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7febfc072f10 tx=0x7febf802a230 comp rx=0 tx=0).stop 2024-09-19T07:53:17.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.475+0000 7fec0467b640 1 -- 172.21.15.16:0/2877116359 shutdown_connections 2024-09-19T07:53:17.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.475+0000 7fec0467b640 1 --2- 172.21.15.16:0/2877116359 >> v2:172.21.15.146:3300/0 conn(0x7febfc09b460 0x7febfc0d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:17.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.475+0000 7fec0467b640 1 --2- 172.21.15.16:0/2877116359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febfc070910 0x7febfc070cf0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:17.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.475+0000 7fec0467b640 1 -- 172.21.15.16:0/2877116359 >> 172.21.15.16:0/2877116359 conn(0x7febfc0b3e00 msgr2=0x7febfc0b4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:17.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.475+0000 7fec0467b640 1 -- 172.21.15.16:0/2877116359 shutdown_connections 2024-09-19T07:53:17.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.475+0000 7fec0467b640 1 -- 172.21.15.16:0/2877116359 wait complete. 2024-09-19T07:53:17.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.476+0000 7fec0467b640 1 Processor -- start 2024-09-19T07:53:17.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.476+0000 7fec0467b640 1 -- start start 2024-09-19T07:53:17.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7fec0467b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febfc070910 0x7febfc1f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:17.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7fec0467b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febfc09b460 0x7febfc0b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:17.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7fec0467b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7febfc09bb20 0x7febfc0b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:17.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7fec0467b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7febfc15ea90 con 0x7febfc09b460 2024-09-19T07:53:17.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7fec0467b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7febfc070560 con 0x7febfc070910 2024-09-19T07:53:17.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7fec0467b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7febfc0b1530 con 0x7febfc09bb20 2024-09-19T07:53:17.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7fec01c16640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febfc09b460 0x7febfc0b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:17.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7fec01c16640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febfc09b460 0x7febfc0b2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38856/0 (socket says 172.21.15.16:38856) 2024-09-19T07:53:17.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7fec01c16640 1 -- 172.21.15.16:0/216286087 learned_addr learned my addr 172.21.15.16:0/216286087 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:17.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7fec02c18640 1 --2- 172.21.15.16:0/216286087 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7febfc09bb20 0x7febfc0b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:17.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7fec02417640 1 --2- 172.21.15.16:0/216286087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febfc070910 0x7febfc1f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:17.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7fec01c16640 1 -- 172.21.15.16:0/216286087 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7febfc09bb20 msgr2=0x7febfc0b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:17.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7fec01c16640 1 --2- 172.21.15.16:0/216286087 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7febfc09bb20 0x7febfc0b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:17.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7fec01c16640 1 -- 172.21.15.16:0/216286087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febfc070910 msgr2=0x7febfc1f5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:17.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7fec01c16640 1 --2- 172.21.15.16:0/216286087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febfc070910 0x7febfc1f5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:17.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7fec01c16640 1 -- 172.21.15.16:0/216286087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febfc1f5d50 con 0x7febfc09b460 2024-09-19T07:53:17.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7fec01c16640 1 --2- 172.21.15.16:0/216286087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febfc09b460 0x7febfc0b2dc0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7febec00c400 tx=0x7febec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:17.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7febf2ffd640 1 -- 172.21.15.16:0/216286087 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec017070 con 0x7febfc09b460 2024-09-19T07:53:17.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7febf2ffd640 1 -- 172.21.15.16:0/216286087 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febec0035f0 con 0x7febfc09b460 2024-09-19T07:53:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7fec0467b640 1 -- 172.21.15.16:0/216286087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febfc159830 con 0x7febfc09b460 2024-09-19T07:53:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7febf2ffd640 1 -- 172.21.15.16:0/216286087 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec0039e0 con 0x7febfc09b460 2024-09-19T07:53:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.477+0000 7fec0467b640 1 -- 172.21.15.16:0/216286087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febfc15d980 con 0x7febfc09b460 2024-09-19T07:53:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.478+0000 7fec0467b640 1 -- 172.21.15.16:0/216286087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febfc070cf0 con 0x7febfc09b460 2024-09-19T07:53:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.479+0000 7febf2ffd640 1 -- 172.21.15.16:0/216286087 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7febec01a070 con 0x7febfc09b460 2024-09-19T07:53:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.479+0000 7febf2ffd640 1 --2- 172.21.15.16:0/216286087 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7febe006c2b0 0x7febe006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.479+0000 7febf2ffd640 1 -- 172.21.15.16:0/216286087 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7febec08a980 con 0x7febfc09b460 2024-09-19T07:53:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.480+0000 7fec02417640 1 --2- 172.21.15.16:0/216286087 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7febe006c2b0 0x7febe006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.480+0000 7fec02417640 1 --2- 172.21.15.16:0/216286087 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7febe006c2b0 0x7febe006e770 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7febf8034000 tx=0x7febf802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:17.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.484+0000 7febf2ffd640 1 -- 172.21.15.16:0/216286087 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7febec058870 con 0x7febfc09b460 2024-09-19T07:53:17.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.696+0000 7fec0467b640 1 -- 172.21.15.16:0/216286087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febfc0b0fc0 con 0x7febfc09b460 2024-09-19T07:53:17.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.697+0000 7febf2ffd640 1 -- 172.21.15.16:0/216286087 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5228 (secure 0 0 0) 0x7febec05cb70 con 0x7febfc09b460 2024-09-19T07:53:17.698 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:17.698 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":56,"btime":"2024-09-19T07:53:04:953484+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6017,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2955839490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":2955839490},{"type":"v1","addr":"172.21.15.146:6837","nonce":2955839490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6550,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/395357736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":395357736},{"type":"v1","addr":"172.21.15.146:6836","nonce":395357736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3539638739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3539638739},{"type":"v1","addr":"172.21.15.16:6834","nonce":3539638739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:53:02.264799+0000","modified":"2024-09-19T07:53:04.298337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/4185904511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4185904511},{"type":"v1","addr":"172.21.15.16:6835","nonce":4185904511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":7}]} 2024-09-19T07:53:17.699 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 56 2024-09-19T07:53:17.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.699+0000 7fec0467b640 1 -- 172.21.15.16:0/216286087 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7febe006c2b0 msgr2=0x7febe006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:17.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.699+0000 7fec0467b640 1 --2- 172.21.15.16:0/216286087 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7febe006c2b0 0x7febe006e770 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7febf8034000 tx=0x7febf802e000 comp rx=0 tx=0).stop 2024-09-19T07:53:17.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.699+0000 7fec0467b640 1 -- 172.21.15.16:0/216286087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febfc09b460 msgr2=0x7febfc0b2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:17.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.699+0000 7fec0467b640 1 --2- 172.21.15.16:0/216286087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febfc09b460 0x7febfc0b2dc0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7febec00c400 tx=0x7febec00c8c0 comp rx=0 tx=0).stop 2024-09-19T07:53:17.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.699+0000 7fec0467b640 1 -- 172.21.15.16:0/216286087 shutdown_connections 2024-09-19T07:53:17.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.699+0000 7fec0467b640 1 --2- 172.21.15.16:0/216286087 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7febe006c2b0 0x7febe006e770 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:17.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.699+0000 7fec0467b640 1 --2- 172.21.15.16:0/216286087 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7febfc09bb20 0x7febfc0b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:17.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.699+0000 7fec0467b640 1 --2- 172.21.15.16:0/216286087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febfc09b460 0x7febfc0b2dc0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:17.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.699+0000 7fec0467b640 1 --2- 172.21.15.16:0/216286087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febfc070910 0x7febfc1f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:17.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.699+0000 7fec0467b640 1 -- 172.21.15.16:0/216286087 >> 172.21.15.16:0/216286087 conn(0x7febfc0b3e00 msgr2=0x7febfc15f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:17.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.699+0000 7fec0467b640 1 -- 172.21.15.16:0/216286087 shutdown_connections 2024-09-19T07:53:17.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:17.699+0000 7fec0467b640 1 -- 172.21.15.16:0/216286087 wait complete. 2024-09-19T07:53:17.712 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_max_completed_requests 1000 2024-09-19T07:53:17.829 INFO:teuthology.orchestra.run.smithi016.stdout:{"success":"mds_max_completed_requests = '' (not observed, change may require restart) "} 2024-09-19T07:53:17.829 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_max_completed_requests = '' (not observed, change may require restart) " } 2024-09-19T07:53:17.829 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:53:17.830 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:53:17.846 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-19T07:53:17.846 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:53:17.846 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-19T07:53:17.846 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:17.846 DEBUG:teuthology.orchestra.run.smithi016:> n = 1100 2024-09-19T07:53:17.846 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-19T07:53:17.846 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-19T07:53:17.846 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-19T07:53:17.847 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file1" 2024-09-19T07:53:17.847 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:17.847 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-19T07:53:17.847 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-19T07:53:17.847 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-19T07:53:17.847 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:17.847 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-19T07:53:17.847 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-19T07:53:17.847 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:17.847 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-19T07:53:17.847 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-19T07:53:17.847 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-19T07:53:17.847 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-19T07:53:17.847 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:53:17.848 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-19T07:53:17.848 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-19T07:53:17.848 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:53:17.848 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-19T07:53:17.848 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:53:17.848 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:53:17.848 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-19T07:53:17.848 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-19T07:53:17.848 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-19T07:53:17.848 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:53:17.848 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:53:17.848 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-19T07:53:17.848 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-19T07:53:17.848 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:53:20.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:53:20.176 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:53:20.176 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:53:20.233 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-19T07:53:20.233 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:53:20.233 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-19T07:53:20.233 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:20.233 DEBUG:teuthology.orchestra.run.smithi016:> n = 5 2024-09-19T07:53:20.233 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-19T07:53:20.233 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-19T07:53:20.233 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-19T07:53:20.233 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file2" 2024-09-19T07:53:20.234 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:20.234 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-19T07:53:20.234 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-19T07:53:20.234 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-19T07:53:20.234 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:20.234 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-19T07:53:20.234 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-19T07:53:20.234 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:20.234 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-19T07:53:20.234 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-19T07:53:20.234 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-19T07:53:20.234 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-19T07:53:20.234 DEBUG:teuthology.orchestra.run.smithi016:> if True: 2024-09-19T07:53:20.234 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-19T07:53:20.234 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-19T07:53:20.235 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:53:20.235 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-19T07:53:20.235 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:53:20.235 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:53:20.235 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-19T07:53:20.235 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-19T07:53:20.235 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-19T07:53:20.235 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-19T07:53:20.235 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-19T07:53:20.235 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-19T07:53:20.235 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-19T07:53:20.235 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:53:20.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:53:20.401 INFO:tasks.ceph_test_case:waiting 100s for health warning matching MDS_CLIENT_OLDEST_TID 2024-09-19T07:53:20.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health detail --format=json 2024-09-19T07:53:20.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.543+0000 7fc9d9265640 1 Processor -- start 2024-09-19T07:53:20.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.543+0000 7fc9d9265640 1 -- start start 2024-09-19T07:53:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.544+0000 7fc9d9265640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9d40b1420 0x7fc9d40cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.544+0000 7fc9d9265640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc9d40cdbd0 0x7fc9d4160350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.544+0000 7fc9d9265640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9d40ceaa0 con 0x7fc9d415ff70 2024-09-19T07:53:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.544+0000 7fc9d9265640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc9d40d0a90 con 0x7fc9d40cdbd0 2024-09-19T07:53:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.544+0000 7fc9d9265640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc9d40d0d70 con 0x7fc9d40b1420 2024-09-19T07:53:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.544+0000 7fc9d37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc9d40cdbd0 0x7fc9d4160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.544+0000 7fc9d37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc9d40cdbd0 0x7fc9d4160350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38870/0 (socket says 172.21.15.16:38870) 2024-09-19T07:53:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.544+0000 7fc9d37fe640 1 -- 172.21.15.16:0/2196948656 learned_addr learned my addr 172.21.15.16:0/2196948656 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.544+0000 7fc9d27fc640 1 --2- 172.21.15.16:0/2196948656 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9d40b1420 0x7fc9d40cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.544+0000 7fc9d37fe640 1 -- 172.21.15.16:0/2196948656 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9d40b1420 msgr2=0x7fc9d40cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.544+0000 7fc9d37fe640 1 --2- 172.21.15.16:0/2196948656 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9d40b1420 0x7fc9d40cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.544+0000 7fc9d37fe640 1 -- 172.21.15.16:0/2196948656 >> v1:172.21.15.146:6789/0 conn(0x7fc9d415ff70 legacy=0x7fc9d409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.544+0000 7fc9d37fe640 1 -- 172.21.15.16:0/2196948656 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d40d1050 con 0x7fc9d40cdbd0 2024-09-19T07:53:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.544+0000 7fc9d27fc640 1 --2- 172.21.15.16:0/2196948656 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9d40b1420 0x7fc9d40cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.545+0000 7fc9d37fe640 1 --2- 172.21.15.16:0/2196948656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9d40cdbd0 0x7fc9d4160350 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009e50 tx=0x7fc9cc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881fd571a36f1e7b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.545+0000 7fc9d1ffb640 1 -- 172.21.15.16:0/2196948656 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc036070 con 0x7fc9d40cdbd0 2024-09-19T07:53:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.545+0000 7fc9d1ffb640 1 -- 172.21.15.16:0/2196948656 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9cc031600 con 0x7fc9d40cdbd0 2024-09-19T07:53:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.545+0000 7fc9d1ffb640 1 -- 172.21.15.16:0/2196948656 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc0319d0 con 0x7fc9d40cdbd0 2024-09-19T07:53:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.546+0000 7fc9d9265640 1 -- 172.21.15.16:0/2196948656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9d40cdbd0 msgr2=0x7fc9d4160350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.546+0000 7fc9d9265640 1 --2- 172.21.15.16:0/2196948656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9d40cdbd0 0x7fc9d4160350 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009e50 tx=0x7fc9cc02a710 comp rx=0 tx=0).stop 2024-09-19T07:53:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.546+0000 7fc9d9265640 1 -- 172.21.15.16:0/2196948656 shutdown_connections 2024-09-19T07:53:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.546+0000 7fc9d9265640 1 --2- 172.21.15.16:0/2196948656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9d40cdbd0 0x7fc9d4160350 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.546+0000 7fc9d9265640 1 --2- 172.21.15.16:0/2196948656 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9d40b1420 0x7fc9d40cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.546+0000 7fc9d9265640 1 -- 172.21.15.16:0/2196948656 >> 172.21.15.16:0/2196948656 conn(0x7fc9d40b3e00 msgr2=0x7fc9d40b4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.546+0000 7fc9d9265640 1 -- 172.21.15.16:0/2196948656 shutdown_connections 2024-09-19T07:53:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.546+0000 7fc9d9265640 1 -- 172.21.15.16:0/2196948656 wait complete. 2024-09-19T07:53:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.547+0000 7fc9d9265640 1 Processor -- start 2024-09-19T07:53:20.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.547+0000 7fc9d9265640 1 -- start start 2024-09-19T07:53:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.548+0000 7fc9d9265640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9d40b1420 0x7fc9d40cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.548+0000 7fc9d9265640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9d40cdbd0 0x7fc9d4151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.548+0000 7fc9d9265640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9d415ff70 0x7fc9d40b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.548+0000 7fc9d9265640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d40ce050 con 0x7fc9d40b1420 2024-09-19T07:53:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.548+0000 7fc9d9265640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d40cce20 con 0x7fc9d415ff70 2024-09-19T07:53:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.548+0000 7fc9d9265640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc9d40ce1d0 con 0x7fc9d40cdbd0 2024-09-19T07:53:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.548+0000 7fc9d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9d40b1420 0x7fc9d40cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.548+0000 7fc9d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9d40b1420 0x7fc9d40cba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38872/0 (socket says 172.21.15.16:38872) 2024-09-19T07:53:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.548+0000 7fc9d2ffd640 1 -- 172.21.15.16:0/1184583292 learned_addr learned my addr 172.21.15.16:0/1184583292 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.548+0000 7fc9d27fc640 1 --2- 172.21.15.16:0/1184583292 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9d40cdbd0 0x7fc9d4151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.548+0000 7fc9d37fe640 1 --2- 172.21.15.16:0/1184583292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9d415ff70 0x7fc9d40b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.548+0000 7fc9d2ffd640 1 -- 172.21.15.16:0/1184583292 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9d40cdbd0 msgr2=0x7fc9d4151650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.548+0000 7fc9d2ffd640 1 --2- 172.21.15.16:0/1184583292 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9d40cdbd0 0x7fc9d4151650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.548+0000 7fc9d2ffd640 1 -- 172.21.15.16:0/1184583292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9d415ff70 msgr2=0x7fc9d40b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.548+0000 7fc9d2ffd640 1 --2- 172.21.15.16:0/1184583292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9d415ff70 0x7fc9d40b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.548+0000 7fc9d2ffd640 1 -- 172.21.15.16:0/1184583292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d40b6810 con 0x7fc9d40b1420 2024-09-19T07:53:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.549+0000 7fc9d2ffd640 1 --2- 172.21.15.16:0/1184583292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9d40b1420 0x7fc9d40cba70 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400c3c0 tx=0x7fc9c400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.549+0000 7fc9bb7fe640 1 -- 172.21.15.16:0/1184583292 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c40052d0 con 0x7fc9d40b1420 2024-09-19T07:53:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.549+0000 7fc9bb7fe640 1 -- 172.21.15.16:0/1184583292 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c4005490 con 0x7fc9d40b1420 2024-09-19T07:53:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.549+0000 7fc9d9265640 1 -- 172.21.15.16:0/1184583292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d40b6af0 con 0x7fc9d40b1420 2024-09-19T07:53:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.549+0000 7fc9bb7fe640 1 -- 172.21.15.16:0/1184583292 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4007260 con 0x7fc9d40b1420 2024-09-19T07:53:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.549+0000 7fc9d9265640 1 -- 172.21.15.16:0/1184583292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d40b70b0 con 0x7fc9d40b1420 2024-09-19T07:53:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.550+0000 7fc9d9265640 1 -- 172.21.15.16:0/1184583292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d40cd690 con 0x7fc9d40b1420 2024-09-19T07:53:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.550+0000 7fc9bb7fe640 1 -- 172.21.15.16:0/1184583292 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc9c4007400 con 0x7fc9d40b1420 2024-09-19T07:53:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.551+0000 7fc9bb7fe640 1 --2- 172.21.15.16:0/1184583292 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc9c006c2b0 0x7fc9c006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.551+0000 7fc9bb7fe640 1 -- 172.21.15.16:0/1184583292 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fc9c408a9a0 con 0x7fc9d40b1420 2024-09-19T07:53:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.551+0000 7fc9d27fc640 1 --2- 172.21.15.16:0/1184583292 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc9c006c2b0 0x7fc9c006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.551+0000 7fc9d27fc640 1 --2- 172.21.15.16:0/1184583292 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc9c006c2b0 0x7fc9c006e770 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40c5990 tx=0x7fc9c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:20.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.555+0000 7fc9bb7fe640 1 -- 172.21.15.16:0/1184583292 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc9c40589c0 con 0x7fc9d40b1420 2024-09-19T07:53:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.798+0000 7fc9d9265640 1 -- 172.21.15.16:0/1184583292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail", "format": "json"} v 0) -- 0x7fc9d40b8290 con 0x7fc9d40b1420 2024-09-19T07:53:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.798+0000 7fc9bb7fe640 1 -- 172.21.15.16:0/1184583292 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail", "format": "json"}]=0 v0) ==== 92+0+46 (secure 0 0 0) 0x7fc9c405ccc0 con 0x7fc9d40b1420 2024-09-19T07:53:20.799 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:20.799 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T07:53:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.800+0000 7fc9d9265640 1 -- 172.21.15.16:0/1184583292 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc9c006c2b0 msgr2=0x7fc9c006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.800+0000 7fc9d9265640 1 --2- 172.21.15.16:0/1184583292 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc9c006c2b0 0x7fc9c006e770 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40c5990 tx=0x7fc9c8008040 comp rx=0 tx=0).stop 2024-09-19T07:53:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.800+0000 7fc9d9265640 1 -- 172.21.15.16:0/1184583292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9d40b1420 msgr2=0x7fc9d40cba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.800+0000 7fc9d9265640 1 --2- 172.21.15.16:0/1184583292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9d40b1420 0x7fc9d40cba70 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400c3c0 tx=0x7fc9c400c880 comp rx=0 tx=0).stop 2024-09-19T07:53:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.800+0000 7fc9d9265640 1 -- 172.21.15.16:0/1184583292 shutdown_connections 2024-09-19T07:53:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.800+0000 7fc9d9265640 1 --2- 172.21.15.16:0/1184583292 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc9c006c2b0 0x7fc9c006e770 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.800+0000 7fc9d9265640 1 --2- 172.21.15.16:0/1184583292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9d415ff70 0x7fc9d40b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.800+0000 7fc9d9265640 1 --2- 172.21.15.16:0/1184583292 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9d40cdbd0 0x7fc9d4151650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.800+0000 7fc9d9265640 1 --2- 172.21.15.16:0/1184583292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9d40b1420 0x7fc9d40cba70 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.800+0000 7fc9d9265640 1 -- 172.21.15.16:0/1184583292 >> 172.21.15.16:0/1184583292 conn(0x7fc9d40b3e00 msgr2=0x7fc9d415b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.800+0000 7fc9d9265640 1 -- 172.21.15.16:0/1184583292 shutdown_connections 2024-09-19T07:53:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:20.800+0000 7fc9d9265640 1 -- 172.21.15.16:0/1184583292 wait complete. 2024-09-19T07:53:20.810 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-19T07:53:20.811 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2024-09-19T07:53:25.812 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health detail --format=json 2024-09-19T07:53:25.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.927+0000 7f9e72aa5640 1 Processor -- start 2024-09-19T07:53:25.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.927+0000 7f9e72aa5640 1 -- start start 2024-09-19T07:53:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.928+0000 7f9e72aa5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e6c15e640 0x7f9e6c162250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.928+0000 7f9e72aa5640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9e6c162ea0 0x7f9e6c0b1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.928+0000 7f9e72aa5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e6c0cf8b0 con 0x7f9e6c0d5040 2024-09-19T07:53:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.928+0000 7f9e72aa5640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f9e6c0b12e0 con 0x7f9e6c162ea0 2024-09-19T07:53:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.928+0000 7f9e72aa5640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9e6c0cfb90 con 0x7f9e6c15e640 2024-09-19T07:53:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.928+0000 7f9e6bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9e6c0d5040 0x7f9e6c162790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49508/0 (socket says 172.21.15.16:49508) 2024-09-19T07:53:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e6bfff640 1 -- 172.21.15.16:0/3075186599 learned_addr learned my addr 172.21.15.16:0/3075186599 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e70841640 1 --2- 172.21.15.16:0/3075186599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e6c15e640 0x7f9e6c162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:25.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e71042640 1 --2- 172.21.15.16:0/3075186599 >> v2:172.21.15.146:3300/0 conn(0x7f9e6c162ea0 0x7f9e6c0b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:25.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e6b7fe640 1 -- 172.21.15.16:0/3075186599 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2232260889 0 0) 0x7f9e6c0cf8b0 con 0x7f9e6c0d5040 2024-09-19T07:53:25.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e6b7fe640 1 -- 172.21.15.16:0/3075186599 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e58003520 con 0x7f9e6c0d5040 2024-09-19T07:53:25.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e6b7fe640 1 -- 172.21.15.16:0/3075186599 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 284916166 0 0) 0x7f9e58003520 con 0x7f9e6c0d5040 2024-09-19T07:53:25.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e6b7fe640 1 -- 172.21.15.16:0/3075186599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e6c15e640 msgr2=0x7f9e6c162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:25.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e6b7fe640 1 --2- 172.21.15.16:0/3075186599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e6c15e640 0x7f9e6c162250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:25.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e6b7fe640 1 -- 172.21.15.16:0/3075186599 >> v2:172.21.15.146:3300/0 conn(0x7f9e6c162ea0 msgr2=0x7f9e6c0b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:25.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e6b7fe640 1 --2- 172.21.15.16:0/3075186599 >> v2:172.21.15.146:3300/0 conn(0x7f9e6c162ea0 0x7f9e6c0b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:25.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e6b7fe640 1 -- 172.21.15.16:0/3075186599 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6c0cfe70 con 0x7f9e6c0d5040 2024-09-19T07:53:25.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e6b7fe640 1 -- 172.21.15.16:0/3075186599 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f9e54001b70 con 0x7f9e6c0d5040 2024-09-19T07:53:25.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e6b7fe640 1 -- 172.21.15.16:0/3075186599 >> v1:172.21.15.16:6789/0 conn(0x7f9e6c0d5040 legacy=0x7f9e6c162790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:25.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e6b7fe640 1 --2- 172.21.15.16:0/3075186599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e580039b0 0x7f9e58003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:25.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e6b7fe640 1 --2- 172.21.15.16:0/3075186599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e58004800 0x7f9e58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e6b7fe640 1 --2- 172.21.15.16:0/3075186599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e6c15e640 0x7f9e580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e6b7fe640 1 -- 172.21.15.16:0/3075186599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c0b12e0 con 0x7f9e580039b0 2024-09-19T07:53:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e6b7fe640 1 -- 172.21.15.16:0/3075186599 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c0cfb90 con 0x7f9e58004800 2024-09-19T07:53:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e6b7fe640 1 -- 172.21.15.16:0/3075186599 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9e5800b9c0 con 0x7f9e6c15e640 2024-09-19T07:53:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e70841640 1 --2- 172.21.15.16:0/3075186599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e580039b0 0x7f9e58003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e71042640 1 --2- 172.21.15.16:0/3075186599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e6c15e640 0x7f9e580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.929+0000 7f9e6bfff640 1 --2- 172.21.15.16:0/3075186599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e58004800 0x7f9e58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.930+0000 7f9e70841640 1 -- 172.21.15.16:0/3075186599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e6c15e640 msgr2=0x7f9e580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.930+0000 7f9e70841640 1 --2- 172.21.15.16:0/3075186599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e6c15e640 0x7f9e580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.930+0000 7f9e70841640 1 -- 172.21.15.16:0/3075186599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e58004800 msgr2=0x7f9e58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.930+0000 7f9e70841640 1 --2- 172.21.15.16:0/3075186599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e58004800 0x7f9e58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.930+0000 7f9e70841640 1 -- 172.21.15.16:0/3075186599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e58003520 con 0x7f9e580039b0 2024-09-19T07:53:25.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.930+0000 7f9e70841640 1 --2- 172.21.15.16:0/3075186599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e580039b0 0x7f9e58003dd0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c009ac0 tx=0x7f9e58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c46d607ac3cb93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:25.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.931+0000 7f9e6b7fe640 1 -- 172.21.15.16:0/3075186599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c036070 con 0x7f9e580039b0 2024-09-19T07:53:25.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.931+0000 7f9e6b7fe640 1 -- 172.21.15.16:0/3075186599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e5c031400 con 0x7f9e580039b0 2024-09-19T07:53:25.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.931+0000 7f9e6b7fe640 1 -- 172.21.15.16:0/3075186599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c009690 con 0x7f9e580039b0 2024-09-19T07:53:25.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.931+0000 7f9e72aa5640 1 -- 172.21.15.16:0/3075186599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e580039b0 msgr2=0x7f9e58003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:25.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.931+0000 7f9e72aa5640 1 --2- 172.21.15.16:0/3075186599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e580039b0 0x7f9e58003dd0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c009ac0 tx=0x7f9e58005830 comp rx=0 tx=0).stop 2024-09-19T07:53:25.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.931+0000 7f9e71042640 1 -- 172.21.15.16:0/3075186599 reap_dead start 2024-09-19T07:53:25.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.931+0000 7f9e72aa5640 1 -- 172.21.15.16:0/3075186599 shutdown_connections 2024-09-19T07:53:25.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.931+0000 7f9e72aa5640 1 -- 172.21.15.16:0/3075186599 >> 172.21.15.16:0/3075186599 conn(0x7f9e6c08bfe0 msgr2=0x7f9e6c0d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:25.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.931+0000 7f9e72aa5640 1 -- 172.21.15.16:0/3075186599 shutdown_connections 2024-09-19T07:53:25.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.931+0000 7f9e72aa5640 1 -- 172.21.15.16:0/3075186599 wait complete. 2024-09-19T07:53:25.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.932+0000 7f9e72aa5640 1 Processor -- start 2024-09-19T07:53:25.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.933+0000 7f9e72aa5640 1 -- start start 2024-09-19T07:53:25.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.933+0000 7f9e72aa5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e580039b0 0x7f9e6c0cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:25.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.933+0000 7f9e72aa5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6c14f040 0x7f9e6c0cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:25.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.933+0000 7f9e72aa5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e6c14f4a0 0x7f9e6c0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:25.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.933+0000 7f9e72aa5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c15eba0 con 0x7f9e6c14f040 2024-09-19T07:53:25.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.933+0000 7f9e72aa5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c15ea20 con 0x7f9e580039b0 2024-09-19T07:53:25.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.933+0000 7f9e72aa5640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9e6c162250 con 0x7f9e6c14f4a0 2024-09-19T07:53:25.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.934+0000 7f9e6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6c14f040 0x7f9e6c0cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:25.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.934+0000 7f9e6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6c14f040 0x7f9e6c0cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40508/0 (socket says 172.21.15.16:40508) 2024-09-19T07:53:25.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.934+0000 7f9e6bfff640 1 -- 172.21.15.16:0/1889741755 learned_addr learned my addr 172.21.15.16:0/1889741755 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:25.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.934+0000 7f9e71042640 1 --2- 172.21.15.16:0/1889741755 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e6c14f4a0 0x7f9e6c0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:25.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.934+0000 7f9e70841640 1 --2- 172.21.15.16:0/1889741755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e580039b0 0x7f9e6c0cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:25.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.934+0000 7f9e6bfff640 1 -- 172.21.15.16:0/1889741755 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e6c14f4a0 msgr2=0x7f9e6c0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:25.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.934+0000 7f9e6bfff640 1 --2- 172.21.15.16:0/1889741755 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e6c14f4a0 0x7f9e6c0cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:25.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.934+0000 7f9e6bfff640 1 -- 172.21.15.16:0/1889741755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e580039b0 msgr2=0x7f9e6c0cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:25.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.934+0000 7f9e6bfff640 1 --2- 172.21.15.16:0/1889741755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e580039b0 0x7f9e6c0cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:25.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.934+0000 7f9e6bfff640 1 -- 172.21.15.16:0/1889741755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6c0cbd00 con 0x7f9e6c14f040 2024-09-19T07:53:25.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.934+0000 7f9e6bfff640 1 --2- 172.21.15.16:0/1889741755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6c14f040 0x7f9e6c0cd270 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9e5400fb90 tx=0x7f9e5400ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:25.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.934+0000 7f9e697fa640 1 -- 172.21.15.16:0/1889741755 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e54005df0 con 0x7f9e6c14f040 2024-09-19T07:53:25.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.934+0000 7f9e697fa640 1 -- 172.21.15.16:0/1889741755 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e54018070 con 0x7f9e6c14f040 2024-09-19T07:53:25.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.934+0000 7f9e72aa5640 1 -- 172.21.15.16:0/1889741755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6c0cbfe0 con 0x7f9e6c14f040 2024-09-19T07:53:25.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.934+0000 7f9e697fa640 1 -- 172.21.15.16:0/1889741755 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5400db30 con 0x7f9e6c14f040 2024-09-19T07:53:25.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.934+0000 7f9e72aa5640 1 -- 172.21.15.16:0/1889741755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e6c0cc5a0 con 0x7f9e6c14f040 2024-09-19T07:53:25.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.935+0000 7f9e72aa5640 1 -- 172.21.15.16:0/1889741755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e6c162610 con 0x7f9e6c14f040 2024-09-19T07:53:25.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.936+0000 7f9e697fa640 1 -- 172.21.15.16:0/1889741755 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9e5400dcd0 con 0x7f9e6c14f040 2024-09-19T07:53:25.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.936+0000 7f9e697fa640 1 --2- 172.21.15.16:0/1889741755 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9e4c06c2b0 0x7f9e4c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:25.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.936+0000 7f9e697fa640 1 -- 172.21.15.16:0/1889741755 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f9e5408b8b0 con 0x7f9e6c14f040 2024-09-19T07:53:25.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.937+0000 7f9e70841640 1 --2- 172.21.15.16:0/1889741755 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9e4c06c2b0 0x7f9e4c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:25.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.937+0000 7f9e70841640 1 --2- 172.21.15.16:0/1889741755 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9e4c06c2b0 0x7f9e4c06e770 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c033000 tx=0x7f9e5c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:25.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:25.941+0000 7f9e697fa640 1 -- 172.21.15.16:0/1889741755 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9e54059f20 con 0x7f9e6c14f040 2024-09-19T07:53:26.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.223+0000 7f9e72aa5640 1 -- 172.21.15.16:0/1889741755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail", "format": "json"} v 0) -- 0x7f9e6c0b6ce0 con 0x7f9e6c14f040 2024-09-19T07:53:26.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.223+0000 7f9e697fa640 1 -- 172.21.15.16:0/1889741755 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail", "format": "json"}]=0 v0) ==== 92+0+322 (secure 0 0 0) 0x7f9e54059820 con 0x7f9e6c14f040 2024-09-19T07:53:26.224 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.224 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CLIENT_OLDEST_TID":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to advance oldest client/flush tid","count":1},"detail":[{"message":"mds.c(mds.0): Client smithi016:0 failing to advance its oldest client/flush tid. client_id: 7185"}],"muted":false}},"mutes":[]} 2024-09-19T07:53:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.225+0000 7f9e72aa5640 1 -- 172.21.15.16:0/1889741755 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9e4c06c2b0 msgr2=0x7f9e4c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.225+0000 7f9e72aa5640 1 --2- 172.21.15.16:0/1889741755 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9e4c06c2b0 0x7f9e4c06e770 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c033000 tx=0x7f9e5c008030 comp rx=0 tx=0).stop 2024-09-19T07:53:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.226+0000 7f9e72aa5640 1 -- 172.21.15.16:0/1889741755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6c14f040 msgr2=0x7f9e6c0cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.226+0000 7f9e72aa5640 1 --2- 172.21.15.16:0/1889741755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6c14f040 0x7f9e6c0cd270 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9e5400fb90 tx=0x7f9e5400ff90 comp rx=0 tx=0).stop 2024-09-19T07:53:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.226+0000 7f9e72aa5640 1 -- 172.21.15.16:0/1889741755 shutdown_connections 2024-09-19T07:53:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.226+0000 7f9e72aa5640 1 --2- 172.21.15.16:0/1889741755 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9e4c06c2b0 0x7f9e4c06e770 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.226+0000 7f9e72aa5640 1 --2- 172.21.15.16:0/1889741755 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e6c14f4a0 0x7f9e6c0cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.226+0000 7f9e72aa5640 1 --2- 172.21.15.16:0/1889741755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6c14f040 0x7f9e6c0cd270 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.226+0000 7f9e72aa5640 1 --2- 172.21.15.16:0/1889741755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e580039b0 0x7f9e6c0cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.226+0000 7f9e72aa5640 1 -- 172.21.15.16:0/1889741755 >> 172.21.15.16:0/1889741755 conn(0x7f9e6c08bfe0 msgr2=0x7f9e6c0d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.226+0000 7f9e72aa5640 1 -- 172.21.15.16:0/1889741755 shutdown_connections 2024-09-19T07:53:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.226+0000 7f9e72aa5640 1 -- 172.21.15.16:0/1889741755 wait complete. 2024-09-19T07:53:26.237 DEBUG:tasks.ceph_test_case:detail_strings: ['mds.c(mds.0): Client smithi016:0 failing to advance its oldest client/flush tid. client_id: 7185'] 2024-09-19T07:53:26.237 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-19T07:53:26.238 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T07:53:26.240 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T07:53:26.282 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-09-19T07:53:26.282 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2024-09-19T07:53:26.282 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T07:53:26.282 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2024-09-19T07:53:26.282 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2024-09-19T07:53:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2024-09-19T07:53:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2024-09-19T07:53:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2024-09-19T07:53:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2024-09-19T07:53:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2024-09-19T07:53:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.283 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T07:53:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2024-09-19T07:53:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T07:53:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T07:53:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2024-09-19T07:53:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2024-09-19T07:53:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2024-09-19T07:53:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2024-09-19T07:53:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2024-09-19T07:53:26.284 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2024-09-19T07:53:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2024-09-19T07:53:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2024-09-19T07:53:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T07:53:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T07:53:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2024-09-19T07:53:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T07:53:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2024-09-19T07:53:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2024-09-19T07:53:26.285 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2024-09-19T07:53:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2024-09-19T07:53:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2024-09-19T07:53:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-09-19T07:53:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb 2024-09-19T07:53:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.286 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2024-09-19T07:53:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2024-09-19T07:53:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2024-09-19T07:53:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2024-09-19T07:53:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2024-09-19T07:53:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2024-09-19T07:53:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2024-09-19T07:53:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2024-09-19T07:53:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2024-09-19T07:53:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2024-09-19T07:53:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2024-09-19T07:53:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2024-09-19T07:53:26.287 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2024-09-19T07:53:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2024-09-19T07:53:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2024-09-19T07:53:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2024-09-19T07:53:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2024-09-19T07:53:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2024-09-19T07:53:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2024-09-19T07:53:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2024-09-19T07:53:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2024-09-19T07:53:26.288 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2024-09-19T07:53:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2024-09-19T07:53:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2024-09-19T07:53:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:53:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2024-09-19T07:53:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2024-09-19T07:53:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T07:53:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2024-09-19T07:53:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2024-09-19T07:53:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2024-09-19T07:53:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2024-09-19T07:53:26.289 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-09-19T07:53:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.290 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2024-09-19T07:53:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:53:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2024-09-19T07:53:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-19T07:53:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-19T07:53:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:53:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:53:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2024-09-19T07:53:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-09-19T07:53:26.290 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.290 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2024-09-19T07:53:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:53:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-19T07:53:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2024-09-19T07:53:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-19T07:53:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2024-09-19T07:53:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2024-09-19T07:53:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:53:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2024-09-19T07:53:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2024-09-19T07:53:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:53:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.291 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T07:53:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T07:53:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T07:53:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T07:53:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T07:53:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T07:53:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2024-09-19T07:53:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2024-09-19T07:53:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2024-09-19T07:53:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: mon down mkfs grace = 300 2024-09-19T07:53:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2024-09-19T07:53:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on pool no app = False 2024-09-19T07:53:26.292 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:26.293 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2024-09-19T07:53:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2024-09-19T07:53:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2024-09-19T07:53:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2024-09-19T07:53:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T07:53:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T07:53:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2024-09-19T07:53:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-09-19T07:53:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:53:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2024-09-19T07:53:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-09-19T07:53:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-09-19T07:53:26.293 INFO:teuthology.orchestra.run.smithi016.stdout: client inject fixed oldest tid = false 2024-09-19T07:53:26.294 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2024-09-19T07:53:26.294 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2024-09-19T07:53:26.294 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2024-09-19T07:53:26.294 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-09-19T07:53:26.294 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-09-19T07:53:26.294 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-09-19T07:53:26.294 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-09-19T07:53:26.294 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2024-09-19T07:53:26.294 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2024-09-19T07:53:26.294 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2024-09-19T07:53:26.294 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:53:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2024-09-19T07:53:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2024-09-19T07:53:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2024-09-19T07:53:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = True 2024-09-19T07:53:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-09-19T07:53:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-09-19T07:53:26.295 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-09-19T07:53:26.296 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2024-09-19T07:53:26.296 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2024-09-19T07:53:26.296 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T07:53:26.296 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2024-09-19T07:53:26.296 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.296 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2024-09-19T07:53:26.296 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.296 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2024-09-19T07:53:26.296 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.296 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2024-09-19T07:53:26.296 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2024-09-19T07:53:26.297 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.297 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2024-09-19T07:53:26.297 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2024-09-19T07:53:26.297 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.297 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T07:53:26.297 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2024-09-19T07:53:26.297 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.297 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T07:53:26.297 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T07:53:26.297 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.297 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2024-09-19T07:53:26.297 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.297 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2024-09-19T07:53:26.297 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2024-09-19T07:53:26.298 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2024-09-19T07:53:26.298 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2024-09-19T07:53:26.298 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2024-09-19T07:53:26.298 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2024-09-19T07:53:26.298 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2024-09-19T07:53:26.298 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T07:53:26.298 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T07:53:26.298 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2024-09-19T07:53:26.298 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.298 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T07:53:26.298 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.298 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2024-09-19T07:53:26.298 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.298 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2024-09-19T07:53:26.299 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.299 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2024-09-19T07:53:26.299 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2024-09-19T07:53:26.299 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2024-09-19T07:53:26.299 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.16,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-09-19T07:53:26.299 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = 6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb 2024-09-19T07:53:26.299 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.299 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2024-09-19T07:53:26.299 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2024-09-19T07:53:26.299 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.299 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2024-09-19T07:53:26.299 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2024-09-19T07:53:26.299 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2024-09-19T07:53:26.299 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.300 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2024-09-19T07:53:26.300 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2024-09-19T07:53:26.300 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.300 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2024-09-19T07:53:26.300 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2024-09-19T07:53:26.300 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2024-09-19T07:53:26.300 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug trim objects = true 2024-09-19T07:53:26.300 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.300 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2024-09-19T07:53:26.300 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2024-09-19T07:53:26.300 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.300 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2024-09-19T07:53:26.300 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.300 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2024-09-19T07:53:26.301 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.301 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2024-09-19T07:53:26.301 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.301 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2024-09-19T07:53:26.301 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2024-09-19T07:53:26.301 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.301 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2024-09-19T07:53:26.301 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2024-09-19T07:53:26.301 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2024-09-19T07:53:26.301 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2024-09-19T07:53:26.301 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 20 2024-09-19T07:53:26.301 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 20 2024-09-19T07:53:26.301 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:53:26.301 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2024-09-19T07:53:26.302 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 10 2024-09-19T07:53:26.302 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T07:53:26.302 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2024-09-19T07:53:26.302 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2024-09-19T07:53:26.302 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2024-09-19T07:53:26.302 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2024-09-19T07:53:26.302 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-09-19T07:53:26.302 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.302 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2024-09-19T07:53:26.302 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:53:26.302 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2024-09-19T07:53:26.302 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-19T07:53:26.302 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-19T07:53:26.302 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:53:26.302 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:53:26.303 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2024-09-19T07:53:26.303 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-09-19T07:53:26.303 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.303 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2024-09-19T07:53:26.303 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:53:26.303 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-19T07:53:26.303 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2024-09-19T07:53:26.303 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-19T07:53:26.303 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2024-09-19T07:53:26.303 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2024-09-19T07:53:26.303 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:53:26.303 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2024-09-19T07:53:26.303 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2024-09-19T07:53:26.304 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:53:26.304 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.304 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T07:53:26.304 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T07:53:26.304 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T07:53:26.304 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.304 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T07:53:26.304 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T07:53:26.304 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T07:53:26.304 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.304 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2024-09-19T07:53:26.304 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2024-09-19T07:53:26.304 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.304 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2024-09-19T07:53:26.304 INFO:teuthology.orchestra.run.smithi146.stdout: mon down mkfs grace = 300 2024-09-19T07:53:26.305 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2024-09-19T07:53:26.305 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on pool no app = False 2024-09-19T07:53:26.305 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:26.305 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2024-09-19T07:53:26.305 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2024-09-19T07:53:26.305 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2024-09-19T07:53:26.305 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2024-09-19T07:53:26.305 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T07:53:26.305 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T07:53:26.305 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2024-09-19T07:53:26.305 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-09-19T07:53:26.305 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:53:26.306 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = "" 2024-09-19T07:53:26.306 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-09-19T07:53:26.306 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-09-19T07:53:26.306 INFO:teuthology.orchestra.run.smithi146.stdout: client inject fixed oldest tid = false 2024-09-19T07:53:26.306 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2024-09-19T07:53:26.306 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2024-09-19T07:53:26.306 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2024-09-19T07:53:26.306 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2024-09-19T07:53:26.306 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2024-09-19T07:53:26.306 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2024-09-19T07:53:26.306 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2024-09-19T07:53:26.306 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2024-09-19T07:53:26.306 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2024-09-19T07:53:26.306 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2024-09-19T07:53:26.306 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:53:26.307 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2024-09-19T07:53:26.307 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2024-09-19T07:53:26.307 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2024-09-19T07:53:26.307 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2024-09-19T07:53:26.307 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-09-19T07:53:26.307 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-09-19T07:53:26.307 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-09-19T07:53:26.308 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:53:26.309 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:53:26.309 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:53:26.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:53:26.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.509+0000 7fd34bb0a640 1 Processor -- start 2024-09-19T07:53:26.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.509+0000 7fd34bb0a640 1 -- start start 2024-09-19T07:53:26.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.510+0000 7fd34bb0a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd34415f9d0 0x7fd344160200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:26.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.510+0000 7fd34bb0a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd34415fdb0 con 0x7fd344162aa0 2024-09-19T07:53:26.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.510+0000 7fd34bb0a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3440b3600 con 0x7fd34409bb60 2024-09-19T07:53:26.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.510+0000 7fd34bb0a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd3440d0c40 con 0x7fd34415f9d0 2024-09-19T07:53:26.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.510+0000 7fd3498a6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd34415f9d0 0x7fd344160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:26.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.510+0000 7fd3498a6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd34415f9d0 0x7fd344160200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40512/0 (socket says 172.21.15.16:40512) 2024-09-19T07:53:26.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.510+0000 7fd34a0a7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd344162aa0 0x7fd3440d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49520/0 (socket says 172.21.15.16:49520) 2024-09-19T07:53:26.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.510+0000 7fd3498a6640 1 -- 172.21.15.16:0/157664738 learned_addr learned my addr 172.21.15.16:0/157664738 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:26.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.510+0000 7fd3498a6640 1 -- 172.21.15.16:0/157664738 >> v1:172.21.15.146:6789/0 conn(0x7fd34409bb60 legacy=0x7fd344162390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:26.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.510+0000 7fd3498a6640 1 -- 172.21.15.16:0/157664738 >> v1:172.21.15.16:6789/0 conn(0x7fd344162aa0 legacy=0x7fd3440d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:26.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.510+0000 7fd3488a4640 1 -- 172.21.15.16:0/157664738 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2487421006 0 0) 0x7fd34415fdb0 con 0x7fd344162aa0 2024-09-19T07:53:26.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.510+0000 7fd3498a6640 1 -- 172.21.15.16:0/157664738 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3440b38e0 con 0x7fd34415f9d0 2024-09-19T07:53:26.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.511+0000 7fd3498a6640 1 --2- 172.21.15.16:0/157664738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34415f9d0 0x7fd344160200 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fd340009870 tx=0x7fd34002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15174b6804525b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:26.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.511+0000 7fd3488a4640 1 -- 172.21.15.16:0/157664738 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340037070 con 0x7fd34415f9d0 2024-09-19T07:53:26.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.511+0000 7fd3488a4640 1 -- 172.21.15.16:0/157664738 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd340032440 con 0x7fd34415f9d0 2024-09-19T07:53:26.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.511+0000 7fd3488a4640 1 -- 172.21.15.16:0/157664738 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340032810 con 0x7fd34415f9d0 2024-09-19T07:53:26.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.512+0000 7fd34bb0a640 1 -- 172.21.15.16:0/157664738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34415f9d0 msgr2=0x7fd344160200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:26.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.512+0000 7fd34bb0a640 1 --2- 172.21.15.16:0/157664738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34415f9d0 0x7fd344160200 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fd340009870 tx=0x7fd34002a2a0 comp rx=0 tx=0).stop 2024-09-19T07:53:26.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.512+0000 7fd34bb0a640 1 -- 172.21.15.16:0/157664738 shutdown_connections 2024-09-19T07:53:26.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.512+0000 7fd34bb0a640 1 --2- 172.21.15.16:0/157664738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34415f9d0 0x7fd344160200 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:26.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.512+0000 7fd34bb0a640 1 -- 172.21.15.16:0/157664738 >> 172.21.15.16:0/157664738 conn(0x7fd3440d2360 msgr2=0x7fd3440d2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:26.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.512+0000 7fd34bb0a640 1 -- 172.21.15.16:0/157664738 shutdown_connections 2024-09-19T07:53:26.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.512+0000 7fd34bb0a640 1 -- 172.21.15.16:0/157664738 wait complete. 2024-09-19T07:53:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.513+0000 7fd34bb0a640 1 Processor -- start 2024-09-19T07:53:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.513+0000 7fd34bb0a640 1 -- start start 2024-09-19T07:53:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd34bb0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34409bb60 0x7fd3441550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd34bb0a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd34415f9d0 0x7fd344152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd34bb0a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd344162aa0 0x7fd3441f7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd34bb0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3440d04f0 con 0x7fd34409bb60 2024-09-19T07:53:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd34bb0a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd3440b32b0 con 0x7fd34415f9d0 2024-09-19T07:53:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd34bb0a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd3440d0670 con 0x7fd344162aa0 2024-09-19T07:53:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd3498a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34409bb60 0x7fd3441550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd3498a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34409bb60 0x7fd3441550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40516/0 (socket says 172.21.15.16:40516) 2024-09-19T07:53:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd3498a6640 1 -- 172.21.15.16:0/858306653 learned_addr learned my addr 172.21.15.16:0/858306653 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd34a0a7640 1 --2- 172.21.15.16:0/858306653 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd344162aa0 0x7fd3441f7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd3490a5640 1 --2- 172.21.15.16:0/858306653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd34415f9d0 0x7fd344152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd3498a6640 1 -- 172.21.15.16:0/858306653 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd344162aa0 msgr2=0x7fd3441f7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd3498a6640 1 --2- 172.21.15.16:0/858306653 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd344162aa0 0x7fd3441f7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd3498a6640 1 -- 172.21.15.16:0/858306653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd34415f9d0 msgr2=0x7fd344152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd3498a6640 1 --2- 172.21.15.16:0/858306653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd34415f9d0 0x7fd344152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd3498a6640 1 -- 172.21.15.16:0/858306653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3440c8720 con 0x7fd34409bb60 2024-09-19T07:53:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd34a0a7640 1 --2- 172.21.15.16:0/858306653 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd344162aa0 0x7fd3441f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd3490a5640 1 --2- 172.21.15.16:0/858306653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd34415f9d0 0x7fd344152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd3498a6640 1 --2- 172.21.15.16:0/858306653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34409bb60 0x7fd3441550d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd340009870 tx=0x7fd3400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd33a7fc640 1 -- 172.21.15.16:0/858306653 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340041070 con 0x7fd34409bb60 2024-09-19T07:53:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd33a7fc640 1 -- 172.21.15.16:0/858306653 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd34003c870 con 0x7fd34409bb60 2024-09-19T07:53:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.514+0000 7fd33a7fc640 1 -- 172.21.15.16:0/858306653 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340037040 con 0x7fd34409bb60 2024-09-19T07:53:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.515+0000 7fd34bb0a640 1 -- 172.21.15.16:0/858306653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3440c8a00 con 0x7fd34409bb60 2024-09-19T07:53:26.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.515+0000 7fd34bb0a640 1 -- 172.21.15.16:0/858306653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd344156d60 con 0x7fd34409bb60 2024-09-19T07:53:26.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.516+0000 7fd34bb0a640 1 -- 172.21.15.16:0/858306653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3440d2f10 con 0x7fd34409bb60 2024-09-19T07:53:26.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.516+0000 7fd33a7fc640 1 -- 172.21.15.16:0/858306653 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd340044070 con 0x7fd34409bb60 2024-09-19T07:53:26.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.517+0000 7fd33a7fc640 1 --2- 172.21.15.16:0/858306653 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd32806c2e0 0x7fd32806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:26.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.517+0000 7fd33a7fc640 1 -- 172.21.15.16:0/858306653 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fd3400abfd0 con 0x7fd34409bb60 2024-09-19T07:53:26.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.517+0000 7fd3490a5640 1 --2- 172.21.15.16:0/858306653 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd32806c2e0 0x7fd32806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:26.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.518+0000 7fd3490a5640 1 --2- 172.21.15.16:0/858306653 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd32806c2e0 0x7fd32806e7a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd334006fd0 tx=0x7fd334008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:26.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.521+0000 7fd33a7fc640 1 -- 172.21.15.16:0/858306653 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd34007a640 con 0x7fd34409bb60 2024-09-19T07:53:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.708+0000 7fd34bb0a640 1 -- 172.21.15.16:0/858306653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd34406d050 con 0x7fd34409bb60 2024-09-19T07:53:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.708+0000 7fd33a7fc640 1 -- 172.21.15.16:0/858306653 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) ==== 80+0+0 (secure 0 0 0) 0x7fd340079f40 con 0x7fd34409bb60 2024-09-19T07:53:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:53:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.710+0000 7fd34bb0a640 1 -- 172.21.15.16:0/858306653 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd32806c2e0 msgr2=0x7fd32806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.710+0000 7fd34bb0a640 1 --2- 172.21.15.16:0/858306653 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd32806c2e0 0x7fd32806e7a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd334006fd0 tx=0x7fd334008040 comp rx=0 tx=0).stop 2024-09-19T07:53:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.710+0000 7fd34bb0a640 1 -- 172.21.15.16:0/858306653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34409bb60 msgr2=0x7fd3441550d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.710+0000 7fd34bb0a640 1 --2- 172.21.15.16:0/858306653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34409bb60 0x7fd3441550d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd340009870 tx=0x7fd3400046c0 comp rx=0 tx=0).stop 2024-09-19T07:53:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.710+0000 7fd34bb0a640 1 -- 172.21.15.16:0/858306653 shutdown_connections 2024-09-19T07:53:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.710+0000 7fd34bb0a640 1 --2- 172.21.15.16:0/858306653 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd32806c2e0 0x7fd32806e7a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.710+0000 7fd34bb0a640 1 --2- 172.21.15.16:0/858306653 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd344162aa0 0x7fd3441f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.710+0000 7fd34bb0a640 1 --2- 172.21.15.16:0/858306653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd34415f9d0 0x7fd344152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.710+0000 7fd34bb0a640 1 --2- 172.21.15.16:0/858306653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34409bb60 0x7fd3441550d0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:26.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.710+0000 7fd34bb0a640 1 -- 172.21.15.16:0/858306653 >> 172.21.15.16:0/858306653 conn(0x7fd3440d2360 msgr2=0x7fd34406d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:26.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.710+0000 7fd34bb0a640 1 -- 172.21.15.16:0/858306653 shutdown_connections 2024-09-19T07:53:26.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:26.710+0000 7fd34bb0a640 1 -- 172.21.15.16:0/858306653 wait complete. 2024-09-19T07:53:26.721 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-19T07:53:26.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:26.721 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:26.745 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[66444]: fuse finished with error 0 and tester_r 0 2024-09-19T07:53:26.748 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:53:26.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:26.749 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:53:26.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:53:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.061+0000 7f16d7311640 1 Processor -- start 2024-09-19T07:53:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.062+0000 7f16d7311640 1 -- start start 2024-09-19T07:53:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.062+0000 7f16d7311640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16d015e640 0x7f16d0162250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.062+0000 7f16d7311640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16d0162ea0 0x7f16d00b1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.062+0000 7f16d7311640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16d00cf8b0 con 0x7f16d00d5040 2024-09-19T07:53:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.062+0000 7f16d7311640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f16d00b12e0 con 0x7f16d015e640 2024-09-19T07:53:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.062+0000 7f16d7311640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f16d00cfb90 con 0x7f16d0162ea0 2024-09-19T07:53:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.063+0000 7f16d50ad640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16d015e640 0x7f16d0162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.063+0000 7f16d50ad640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16d015e640 0x7f16d0162250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40526/0 (socket says 172.21.15.16:40526) 2024-09-19T07:53:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.063+0000 7f16d50ad640 1 -- 172.21.15.16:0/2984368311 learned_addr learned my addr 172.21.15.16:0/2984368311 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.063+0000 7f16d58ae640 1 --2- 172.21.15.16:0/2984368311 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16d0162ea0 0x7f16d00b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.063+0000 7f16d50ad640 1 -- 172.21.15.16:0/2984368311 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16d0162ea0 msgr2=0x7f16d00b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.063+0000 7f16d50ad640 1 --2- 172.21.15.16:0/2984368311 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16d0162ea0 0x7f16d00b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.063+0000 7f16d50ad640 1 -- 172.21.15.16:0/2984368311 >> v1:172.21.15.146:6789/0 conn(0x7f16d00d5040 legacy=0x7f16d0162790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.063+0000 7f16d50ad640 1 -- 172.21.15.16:0/2984368311 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d00cfe70 con 0x7f16d015e640 2024-09-19T07:53:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.064+0000 7f16d50ad640 1 --2- 172.21.15.16:0/2984368311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d015e640 0x7f16d0162250 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009870 tx=0x7f16c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b57789aea3a5f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.064+0000 7f16cffff640 1 -- 172.21.15.16:0/2984368311 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8037070 con 0x7f16d015e640 2024-09-19T07:53:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.064+0000 7f16cffff640 1 -- 172.21.15.16:0/2984368311 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16c8032440 con 0x7f16d015e640 2024-09-19T07:53:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.064+0000 7f16cffff640 1 -- 172.21.15.16:0/2984368311 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8032810 con 0x7f16d015e640 2024-09-19T07:53:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.064+0000 7f16d7311640 1 -- 172.21.15.16:0/2984368311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d015e640 msgr2=0x7f16d0162250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.064+0000 7f16d7311640 1 --2- 172.21.15.16:0/2984368311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d015e640 0x7f16d0162250 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009870 tx=0x7f16c802a270 comp rx=0 tx=0).stop 2024-09-19T07:53:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.064+0000 7f16d7311640 1 -- 172.21.15.16:0/2984368311 shutdown_connections 2024-09-19T07:53:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.064+0000 7f16d7311640 1 --2- 172.21.15.16:0/2984368311 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16d0162ea0 0x7f16d00b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.064+0000 7f16d7311640 1 --2- 172.21.15.16:0/2984368311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d015e640 0x7f16d0162250 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.064+0000 7f16d7311640 1 -- 172.21.15.16:0/2984368311 >> 172.21.15.16:0/2984368311 conn(0x7f16d008bfe0 msgr2=0x7f16d00d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.064+0000 7f16d7311640 1 -- 172.21.15.16:0/2984368311 shutdown_connections 2024-09-19T07:53:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.064+0000 7f16d7311640 1 -- 172.21.15.16:0/2984368311 wait complete. 2024-09-19T07:53:27.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.065+0000 7f16d7311640 1 Processor -- start 2024-09-19T07:53:27.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.066+0000 7f16d7311640 1 -- start start 2024-09-19T07:53:27.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.066+0000 7f16d7311640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16d00d5040 0x7f16d00c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:27.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.066+0000 7f16d7311640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d015e640 0x7f16d014eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:27.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.066+0000 7f16d7311640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16d0162ea0 0x7f16d0161150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:27.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.066+0000 7f16d7311640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16d015eba0 con 0x7f16d015e640 2024-09-19T07:53:27.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.066+0000 7f16d7311640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f16d015ea20 con 0x7f16d0162ea0 2024-09-19T07:53:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.066+0000 7f16d7311640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f16d00b1cb0 con 0x7f16d00d5040 2024-09-19T07:53:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.066+0000 7f16d48ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d015e640 0x7f16d014eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.066+0000 7f16d48ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d015e640 0x7f16d014eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40530/0 (socket says 172.21.15.16:40530) 2024-09-19T07:53:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.066+0000 7f16d48ac640 1 -- 172.21.15.16:0/245969043 learned_addr learned my addr 172.21.15.16:0/245969043 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.067+0000 7f16d50ad640 1 --2- 172.21.15.16:0/245969043 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16d00d5040 0x7f16d00c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.067+0000 7f16d58ae640 1 --2- 172.21.15.16:0/245969043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16d0162ea0 0x7f16d0161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.067+0000 7f16d48ac640 1 -- 172.21.15.16:0/245969043 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16d00d5040 msgr2=0x7f16d00c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.067+0000 7f16d48ac640 1 --2- 172.21.15.16:0/245969043 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16d00d5040 0x7f16d00c8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.067+0000 7f16d48ac640 1 -- 172.21.15.16:0/245969043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16d0162ea0 msgr2=0x7f16d0161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.067+0000 7f16d48ac640 1 --2- 172.21.15.16:0/245969043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16d0162ea0 0x7f16d0161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.067+0000 7f16d48ac640 1 -- 172.21.15.16:0/245969043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d00ba1c0 con 0x7f16d015e640 2024-09-19T07:53:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.067+0000 7f16d48ac640 1 --2- 172.21.15.16:0/245969043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d015e640 0x7f16d014eab0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f16c400c3f0 tx=0x7f16c400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.067+0000 7f16cdffb640 1 -- 172.21.15.16:0/245969043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c40052d0 con 0x7f16d015e640 2024-09-19T07:53:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.067+0000 7f16cdffb640 1 -- 172.21.15.16:0/245969043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16c4005490 con 0x7f16d015e640 2024-09-19T07:53:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.067+0000 7f16d7311640 1 -- 172.21.15.16:0/245969043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d00c8100 con 0x7f16d015e640 2024-09-19T07:53:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.067+0000 7f16cdffb640 1 -- 172.21.15.16:0/245969043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4006a60 con 0x7f16d015e640 2024-09-19T07:53:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.067+0000 7f16d7311640 1 -- 172.21.15.16:0/245969043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d00c86c0 con 0x7f16d015e640 2024-09-19T07:53:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.068+0000 7f16d7311640 1 -- 172.21.15.16:0/245969043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d00b6ce0 con 0x7f16d015e640 2024-09-19T07:53:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.069+0000 7f16cdffb640 1 -- 172.21.15.16:0/245969043 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f16c4003640 con 0x7f16d015e640 2024-09-19T07:53:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.069+0000 7f16cdffb640 1 --2- 172.21.15.16:0/245969043 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f16bc06c2b0 0x7f16bc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.070+0000 7f16cdffb640 1 -- 172.21.15.16:0/245969043 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f16c408aa20 con 0x7f16d015e640 2024-09-19T07:53:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.070+0000 7f16d50ad640 1 --2- 172.21.15.16:0/245969043 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f16bc06c2b0 0x7f16bc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.070+0000 7f16d50ad640 1 --2- 172.21.15.16:0/245969043 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f16bc06c2b0 0x7f16bc06e770 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f16c8034000 tx=0x7f16c802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:27.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.074+0000 7f16cdffb640 1 -- 172.21.15.16:0/245969043 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f16c4059090 con 0x7f16d015e640 2024-09-19T07:53:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.283+0000 7f16d7311640 1 -- 172.21.15.16:0/245969043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f16d015a150 con 0x7f16d015e640 2024-09-19T07:53:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.283+0000 7f16cdffb640 1 -- 172.21.15.16:0/245969043 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) ==== 53+0+73 (secure 0 0 0) 0x7f16c4058990 con 0x7f16d015e640 2024-09-19T07:53:27.283 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:53:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.285+0000 7f16d7311640 1 -- 172.21.15.16:0/245969043 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f16bc06c2b0 msgr2=0x7f16bc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.285+0000 7f16d7311640 1 --2- 172.21.15.16:0/245969043 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f16bc06c2b0 0x7f16bc06e770 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f16c8034000 tx=0x7f16c802e000 comp rx=0 tx=0).stop 2024-09-19T07:53:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.285+0000 7f16d7311640 1 -- 172.21.15.16:0/245969043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d015e640 msgr2=0x7f16d014eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.285+0000 7f16d7311640 1 --2- 172.21.15.16:0/245969043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d015e640 0x7f16d014eab0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f16c400c3f0 tx=0x7f16c400c8b0 comp rx=0 tx=0).stop 2024-09-19T07:53:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.285+0000 7f16d7311640 1 -- 172.21.15.16:0/245969043 shutdown_connections 2024-09-19T07:53:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.285+0000 7f16d7311640 1 --2- 172.21.15.16:0/245969043 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f16bc06c2b0 0x7f16bc06e770 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.285+0000 7f16d7311640 1 --2- 172.21.15.16:0/245969043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16d0162ea0 0x7f16d0161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.285+0000 7f16d7311640 1 --2- 172.21.15.16:0/245969043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16d015e640 0x7f16d014eab0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.285+0000 7f16d7311640 1 --2- 172.21.15.16:0/245969043 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16d00d5040 0x7f16d00c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.285+0000 7f16d7311640 1 -- 172.21.15.16:0/245969043 >> 172.21.15.16:0/245969043 conn(0x7f16d008bfe0 msgr2=0x7f16d015f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.285+0000 7f16d7311640 1 -- 172.21.15.16:0/245969043 shutdown_connections 2024-09-19T07:53:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27.285+0000 7f16d7311640 1 -- 172.21.15.16:0/245969043 wait complete. 2024-09-19T07:53:27.296 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:53:27.296 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:53:27.296 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T07:53:27.297 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T07:53:27.297 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:27.297 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:53:27.297 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:53:27.297 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:53:27.297 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:53:27.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:27.297 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T07:53:27.313 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:53:27.314 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:53:27.314 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:53:27.314 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:27.314 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:53:27.314 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:27.314 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:53:27.314 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:27.314 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T07:53:27.314 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:53:27.314 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:27.314 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T07:53:27.315 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:53:27.315 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:27.315 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T07:53:27.315 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T07:53:27.315 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42182sec preferred_lft 42182sec 2024-09-19T07:53:27.315 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T07:53:27.315 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:27.315 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:53:27.315 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:27.315 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T07:53:27.315 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:53:27.315 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:27.315 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:53:27.316 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:27.316 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::32:7bff:fe92:8895/64 scope link 2024-09-19T07:53:27.316 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:27.316 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:53:27.316 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether e6:fd:21:6a:43:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:27.316 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e4fd:21ff:fe6a:4369/64 scope link 2024-09-19T07:53:27.316 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:27.316 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:53:27.316 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:27.316 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::80f4:81ff:fe7d:b7f0/64 scope link 2024-09-19T07:53:27.316 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:27.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:27.317 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:53:27.380 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:53:27.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:27.382 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T07:53:27.438 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T07:53:27.438 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:53:27.439 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T07:53:27.439 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T07:53:27.439 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:53:27.439 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:53:27.439 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:53:27.439 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T07:53:27.439 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T07:53:27.439 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T07:53:27.440 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:53:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:53:27.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:53:27.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:27.568 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:53:27.584 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T07:53:27.584 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:53:27.584 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:53:27.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:27.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-19T07:53:27.649 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T07:53:27.650 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:53:27.650 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:27.706 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:53:27.706 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:53:27.706 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:27.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:53:27.827 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:53:27.884 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:53:27.884 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:53:27.884 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:53:27.884 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:53:27.884 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:53:27.884 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:53:27.884 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:53:27.884 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:53:27.884 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:53:27.885 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:53:27.885 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:53:27.885 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:53:27.885 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:53:27.885 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:53:27.885 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:53:27.885 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:53:27.885 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:53:27.885 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:53:27.885 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:53:27.885 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:53:27.885 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:53:27.885 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:53:27.885 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:53:27.886 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:53:27.886 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:53:27.886 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:53:27.886 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:53:27.886 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:53:27.886 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:53:27.886 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:53:27.886 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:27.886 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:27.886 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:27.886 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:27.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:27.887 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:53:27.941 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:53:27.941 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T07:53:27.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:53:28.009 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:53:28.066 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:53:28.066 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:53:28.066 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:53:28.066 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:53:28.066 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:53:28.066 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:53:28.066 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:53:28.066 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:53:28.066 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:53:28.067 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:53:28.067 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:53:28.067 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:53:28.067 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:53:28.067 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:53:28.067 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:53:28.067 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:53:28.067 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:53:28.067 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:53:28.067 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:53:28.067 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:53:28.067 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:53:28.067 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:53:28.068 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:53:28.068 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:53:28.068 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:53:28.068 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:53:28.068 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:53:28.068 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:53:28.068 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:53:28.068 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:53:28.068 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:28.068 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:28.068 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:28.068 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:28.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:28.069 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:53:28.069 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-19T07:53:28.068+0000 7fabe1280480 -1 init, newargv = 0x5566be621930 newargc=15 2024-09-19T07:53:28.070 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[67195]: starting ceph client 2024-09-19T07:53:28.072 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[67195]: starting fuse 2024-09-19T07:53:28.084 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:53:28.085 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-19T07:53:28.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:53:28.085 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:53:28.085 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:53:28.085 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:53:28.085 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:53:28.085 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:28.085 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:53:28.085 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:53:28.086 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:53:28.086 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:53:28.086 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:28.086 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:53:28.086 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:53:28.086 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:53:28.086 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:53:28.086 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:28.086 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:53:28.086 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:53:28.086 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:53:28.086 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:53:28.086 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:53:28.086 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:53:28.087 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:53:28.087 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:53:28.087 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:28.087 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:53:28.087 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:53:28.204 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.67195.asok 2024-09-19T07:53:28.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:53:28.215 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.67195.asok 2024-09-19T07:53:28.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:28.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.67195.asok status 2024-09-19T07:53:28.352 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:53:28.352 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:53:28.352 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:53:28.352 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:53:28.353 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:53:28.353 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:53:28.353 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:53:28.353 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "67195", 2024-09-19T07:53:28.353 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:53:28.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:28.353 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-19T07:53:28.353 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-19T07:53:28.353 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7224, 2024-09-19T07:53:28.353 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:53:28.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:53:28.353 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:53:28.353 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 7224 2024-09-19T07:53:28.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:28.354 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:53:28.354 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:53:28.354 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:53:28.354 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 219321101 2024-09-19T07:53:28.354 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:28.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:28.354 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:53:28.354 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:53:28.354 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:53:28.354 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 219321101 2024-09-19T07:53:28.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:28.354 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.7224 192.168.0.1:0/219321101", 2024-09-19T07:53:28.355 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/219321101", 2024-09-19T07:53:28.355 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-19T07:53:28.355 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 57, 2024-09-19T07:53:28.355 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 106, 2024-09-19T07:53:28.355 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:53:28.355 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:53:28.355 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:53:28.355 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:53:28.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:28.360 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-19T07:53:28.361 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:28.377 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-19T07:53:28.380 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:28.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:28.381 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:28.443 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:53:28.444 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:53:28.531 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:53:28.573 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:53:28.573 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:53:28.574 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:53:28.574 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:53:28.590 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:53:28.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.732+0000 7f2dc262c640 1 Processor -- start 2024-09-19T07:53:28.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.732+0000 7f2dc262c640 1 -- start start 2024-09-19T07:53:28.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.733+0000 7f2dc262c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2dbc15f9d0 0x7f2dbc160200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:28.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.733+0000 7f2dc262c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2dbc162aa0 0x7f2dbc0d2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:28.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.733+0000 7f2dc262c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dbc15fdb0 con 0x7f2dbc09bb60 2024-09-19T07:53:28.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.733+0000 7f2dc262c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2dbc0d0c40 con 0x7f2dbc162aa0 2024-09-19T07:53:28.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.733+0000 7f2dc262c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2dbc0b3600 con 0x7f2dbc15f9d0 2024-09-19T07:53:28.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.733+0000 7f2dbbfff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2dbc15f9d0 0x7f2dbc160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:28.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.733+0000 7f2dc0bc9640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2dbc162aa0 0x7f2dbc0d2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:28.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.733+0000 7f2dc0bc9640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2dbc162aa0 0x7f2dbc0d2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:38806/0 (socket says 172.21.15.16:38806) 2024-09-19T07:53:28.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.733+0000 7f2dbb7fe640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f2dbc09bb60 0x7f2dbc162390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:59844/0 (socket says 172.21.15.16:59844) 2024-09-19T07:53:28.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.733+0000 7f2dc0bc9640 1 -- 172.21.15.16:0/615485234 learned_addr learned my addr 172.21.15.16:0/615485234 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:28.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.733+0000 7f2dbbfff640 1 --2- 172.21.15.16:0/615485234 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2dbc15f9d0 0x7f2dbc160200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:42084/0 (socket says 172.21.15.16:42084) 2024-09-19T07:53:28.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.734+0000 7f2dc0bc9640 1 -- 172.21.15.16:0/615485234 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2dbc15f9d0 msgr2=0x7f2dbc160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:28.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.734+0000 7f2dc0bc9640 1 --2- 172.21.15.16:0/615485234 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2dbc15f9d0 0x7f2dbc160200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:28.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.734+0000 7f2dc0bc9640 1 -- 172.21.15.16:0/615485234 >> v1:172.21.15.146:6789/0 conn(0x7f2dbc09bb60 legacy=0x7f2dbc162390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:28.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.734+0000 7f2dbaffd640 1 -- 172.21.15.16:0/615485234 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3817596206 0 0) 0x7f2dbc15fdb0 con 0x7f2dbc09bb60 2024-09-19T07:53:28.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.734+0000 7f2dc0bc9640 1 -- 172.21.15.16:0/615485234 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dbc0b38e0 con 0x7f2dbc162aa0 2024-09-19T07:53:28.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.735+0000 7f2dc0bc9640 1 --2- 172.21.15.16:0/615485234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dbc162aa0 0x7f2dbc0d2e80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009e50 tx=0x7f2db402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7367bd9df206684b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:28.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.735+0000 7f2dbaffd640 1 -- 172.21.15.16:0/615485234 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db4036070 con 0x7f2dbc162aa0 2024-09-19T07:53:28.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.735+0000 7f2dbaffd640 1 -- 172.21.15.16:0/615485234 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db4031600 con 0x7f2dbc162aa0 2024-09-19T07:53:28.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.735+0000 7f2dbaffd640 1 -- 172.21.15.16:0/615485234 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db40319d0 con 0x7f2dbc162aa0 2024-09-19T07:53:28.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.735+0000 7f2dc262c640 1 -- 172.21.15.16:0/615485234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dbc162aa0 msgr2=0x7f2dbc0d2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:28.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.735+0000 7f2dc262c640 1 --2- 172.21.15.16:0/615485234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dbc162aa0 0x7f2dbc0d2e80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009e50 tx=0x7f2db402a710 comp rx=0 tx=0).stop 2024-09-19T07:53:28.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.735+0000 7f2dc262c640 1 -- 172.21.15.16:0/615485234 shutdown_connections 2024-09-19T07:53:28.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.735+0000 7f2dc262c640 1 --2- 172.21.15.16:0/615485234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dbc162aa0 0x7f2dbc0d2e80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:28.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.735+0000 7f2dc262c640 1 --2- 172.21.15.16:0/615485234 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2dbc15f9d0 0x7f2dbc160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:28.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.735+0000 7f2dc262c640 1 -- 172.21.15.16:0/615485234 >> 172.21.15.16:0/615485234 conn(0x7f2dbc0d2360 msgr2=0x7f2dbc0d2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:28.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.735+0000 7f2dc262c640 1 -- 172.21.15.16:0/615485234 shutdown_connections 2024-09-19T07:53:28.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.735+0000 7f2dc262c640 1 -- 172.21.15.16:0/615485234 wait complete. 2024-09-19T07:53:28.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.736+0000 7f2dc262c640 1 Processor -- start 2024-09-19T07:53:28.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.737+0000 7f2dc262c640 1 -- start start 2024-09-19T07:53:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.737+0000 7f2dc262c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dbc09bb60 0x7f2dbc14dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.737+0000 7f2dc262c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dbc15f9d0 0x7f2dbc15c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.737+0000 7f2dc262c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2dbc162aa0 0x7f2dbc15ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.737+0000 7f2dc262c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc0d04f0 con 0x7f2dbc15f9d0 2024-09-19T07:53:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.737+0000 7f2dc262c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc0b32b0 con 0x7f2dbc09bb60 2024-09-19T07:53:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.737+0000 7f2dc262c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2dbc0d0670 con 0x7f2dbc162aa0 2024-09-19T07:53:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.738+0000 7f2dbb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dbc15f9d0 0x7f2dbc15c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.738+0000 7f2dbb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dbc15f9d0 0x7f2dbc15c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40540/0 (socket says 172.21.15.16:40540) 2024-09-19T07:53:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.738+0000 7f2dbb7fe640 1 -- 172.21.15.16:0/2643975158 learned_addr learned my addr 172.21.15.16:0/2643975158 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.738+0000 7f2dbbfff640 1 --2- 172.21.15.16:0/2643975158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dbc09bb60 0x7f2dbc14dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.738+0000 7f2dc0bc9640 1 --2- 172.21.15.16:0/2643975158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2dbc162aa0 0x7f2dbc15ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.738+0000 7f2dbb7fe640 1 -- 172.21.15.16:0/2643975158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2dbc162aa0 msgr2=0x7f2dbc15ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.738+0000 7f2dbb7fe640 1 --2- 172.21.15.16:0/2643975158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2dbc162aa0 0x7f2dbc15ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.738+0000 7f2dbb7fe640 1 -- 172.21.15.16:0/2643975158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dbc09bb60 msgr2=0x7f2dbc14dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.738+0000 7f2dbb7fe640 1 --2- 172.21.15.16:0/2643975158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dbc09bb60 0x7f2dbc14dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.738+0000 7f2dbb7fe640 1 -- 172.21.15.16:0/2643975158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dbc0b5230 con 0x7f2dbc15f9d0 2024-09-19T07:53:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.738+0000 7f2dbb7fe640 1 --2- 172.21.15.16:0/2643975158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dbc15f9d0 0x7f2dbc15c900 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2db000c4b0 tx=0x7f2db000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.738+0000 7f2db8ff9640 1 -- 172.21.15.16:0/2643975158 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0018070 con 0x7f2dbc15f9d0 2024-09-19T07:53:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.738+0000 7f2db8ff9640 1 -- 172.21.15.16:0/2643975158 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db0007500 con 0x7f2dbc15f9d0 2024-09-19T07:53:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.738+0000 7f2dc262c640 1 -- 172.21.15.16:0/2643975158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dbc0b5510 con 0x7f2dbc15f9d0 2024-09-19T07:53:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.738+0000 7f2db8ff9640 1 -- 172.21.15.16:0/2643975158 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0004a20 con 0x7f2dbc15f9d0 2024-09-19T07:53:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.738+0000 7f2dc262c640 1 -- 172.21.15.16:0/2643975158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dbc0b5ad0 con 0x7f2dbc15f9d0 2024-09-19T07:53:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.739+0000 7f2dc262c640 1 -- 172.21.15.16:0/2643975158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dbc160200 con 0x7f2dbc15f9d0 2024-09-19T07:53:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.740+0000 7f2db8ff9640 1 -- 172.21.15.16:0/2643975158 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2db001b070 con 0x7f2dbc15f9d0 2024-09-19T07:53:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.740+0000 7f2db8ff9640 1 --2- 172.21.15.16:0/2643975158 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2da806c2b0 0x7f2da806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.740+0000 7f2db8ff9640 1 -- 172.21.15.16:0/2643975158 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f2db008a8b0 con 0x7f2dbc15f9d0 2024-09-19T07:53:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.740+0000 7f2dbbfff640 1 --2- 172.21.15.16:0/2643975158 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2da806c2b0 0x7f2da806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.741+0000 7f2dbbfff640 1 --2- 172.21.15.16:0/2643975158 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2da806c2b0 0x7f2da806e770 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009b40 tx=0x7f2dac009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.745+0000 7f2db8ff9640 1 -- 172.21.15.16:0/2643975158 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2db0058900 con 0x7f2dbc15f9d0 2024-09-19T07:53:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.933+0000 7f2dc262c640 1 -- 172.21.15.16:0/2643975158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2dbc06d050 con 0x7f2dbc15f9d0 2024-09-19T07:53:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.933+0000 7f2db8ff9640 1 -- 172.21.15.16:0/2643975158 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) ==== 80+0+0 (secure 0 0 0) 0x7f2db005cc00 con 0x7f2dbc15f9d0 2024-09-19T07:53:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:53:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.935+0000 7f2dc262c640 1 -- 172.21.15.16:0/2643975158 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2da806c2b0 msgr2=0x7f2da806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.935+0000 7f2dc262c640 1 --2- 172.21.15.16:0/2643975158 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2da806c2b0 0x7f2da806e770 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009b40 tx=0x7f2dac009190 comp rx=0 tx=0).stop 2024-09-19T07:53:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.935+0000 7f2dc262c640 1 -- 172.21.15.16:0/2643975158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dbc15f9d0 msgr2=0x7f2dbc15c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.935+0000 7f2dc262c640 1 --2- 172.21.15.16:0/2643975158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dbc15f9d0 0x7f2dbc15c900 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2db000c4b0 tx=0x7f2db000c970 comp rx=0 tx=0).stop 2024-09-19T07:53:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.935+0000 7f2dc262c640 1 -- 172.21.15.16:0/2643975158 shutdown_connections 2024-09-19T07:53:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.935+0000 7f2dc262c640 1 --2- 172.21.15.16:0/2643975158 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2da806c2b0 0x7f2da806e770 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.935+0000 7f2dc262c640 1 --2- 172.21.15.16:0/2643975158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2dbc162aa0 0x7f2dbc15ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.935+0000 7f2dc262c640 1 --2- 172.21.15.16:0/2643975158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dbc15f9d0 0x7f2dbc15c900 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:28.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.935+0000 7f2dc262c640 1 --2- 172.21.15.16:0/2643975158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2dbc09bb60 0x7f2dbc14dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:28.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.935+0000 7f2dc262c640 1 -- 172.21.15.16:0/2643975158 >> 172.21.15.16:0/2643975158 conn(0x7f2dbc0d2360 msgr2=0x7f2dbc15dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:28.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.935+0000 7f2dc262c640 1 -- 172.21.15.16:0/2643975158 shutdown_connections 2024-09-19T07:53:28.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:28.935+0000 7f2dc262c640 1 -- 172.21.15.16:0/2643975158 wait complete. 2024-09-19T07:53:28.946 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-19T07:53:28.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:28.946 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:28.970 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[67195]: fuse finished with error 0 and tester_r 0 2024-09-19T07:53:28.972 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:53:28.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:28.972 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:53:29.057 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:53:29.057 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:53:29.057 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:53:29.076 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:53:29.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.199+0000 7f8c14f98640 1 Processor -- start 2024-09-19T07:53:29.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.199+0000 7f8c14f98640 1 -- start start 2024-09-19T07:53:29.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.200+0000 7f8c14f98640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c1015e7b0 0x7f8c100d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:29.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.200+0000 7f8c14f98640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c1009bad0 0x7f8c10162210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:29.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.200+0000 7f8c14f98640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8c100d16a0 0x7f8c10162750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:29.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.200+0000 7f8c14f98640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f8c1009aa50 con 0x7f8c1009bad0 2024-09-19T07:53:29.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.200+0000 7f8c14f98640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f8c1009ad10 con 0x7f8c100d16a0 2024-09-19T07:53:29.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.200+0000 7f8c14f98640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8c1009aff0 con 0x7f8c1015e7b0 2024-09-19T07:53:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.200+0000 7f8c0dd9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c1009bad0 0x7f8c10162210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.200+0000 7f8c0dd9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c1009bad0 0x7f8c10162210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40556/0 (socket says 172.21.15.16:40556) 2024-09-19T07:53:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.200+0000 7f8c0dd9b640 1 -- 172.21.15.16:0/1706907801 learned_addr learned my addr 172.21.15.16:0/1706907801 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.201+0000 7f8c0e59c640 1 --2- 172.21.15.16:0/1706907801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c1015e7b0 0x7f8c100d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.201+0000 7f8c0ed9d640 1 --2- 172.21.15.16:0/1706907801 >> v2:172.21.15.146:3300/0 conn(0x7f8c100d16a0 0x7f8c10162750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.201+0000 7f8c0dd9b640 1 -- 172.21.15.16:0/1706907801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c1015e7b0 msgr2=0x7f8c100d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.201+0000 7f8c0dd9b640 1 --2- 172.21.15.16:0/1706907801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c1015e7b0 0x7f8c100d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.201+0000 7f8c0dd9b640 1 -- 172.21.15.16:0/1706907801 >> v2:172.21.15.146:3300/0 conn(0x7f8c100d16a0 msgr2=0x7f8c10162750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.201+0000 7f8c0dd9b640 1 --2- 172.21.15.16:0/1706907801 >> v2:172.21.15.146:3300/0 conn(0x7f8c100d16a0 0x7f8c10162750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.201+0000 7f8c0dd9b640 1 -- 172.21.15.16:0/1706907801 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1009b2d0 con 0x7f8c1009bad0 2024-09-19T07:53:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.201+0000 7f8c0e59c640 1 --2- 172.21.15.16:0/1706907801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c1015e7b0 0x7f8c100d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.201+0000 7f8c0ed9d640 1 --2- 172.21.15.16:0/1706907801 >> v2:172.21.15.146:3300/0 conn(0x7f8c100d16a0 0x7f8c10162750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.202+0000 7f8c0dd9b640 1 --2- 172.21.15.16:0/1706907801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c1009bad0 0x7f8c10162210 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f8c1006f6d0 tx=0x7f8bf802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ef7cc672160517 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.202+0000 7f8c0d59a640 1 -- 172.21.15.16:0/1706907801 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8037070 con 0x7f8c1009bad0 2024-09-19T07:53:29.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.202+0000 7f8c0d59a640 1 -- 172.21.15.16:0/1706907801 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf8032440 con 0x7f8c1009bad0 2024-09-19T07:53:29.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.202+0000 7f8c0d59a640 1 -- 172.21.15.16:0/1706907801 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8032810 con 0x7f8c1009bad0 2024-09-19T07:53:29.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.202+0000 7f8c14f98640 1 -- 172.21.15.16:0/1706907801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c1009bad0 msgr2=0x7f8c10162210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:29.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.202+0000 7f8c14f98640 1 --2- 172.21.15.16:0/1706907801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c1009bad0 0x7f8c10162210 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f8c1006f6d0 tx=0x7f8bf802a1e0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.202+0000 7f8c14f98640 1 -- 172.21.15.16:0/1706907801 shutdown_connections 2024-09-19T07:53:29.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.202+0000 7f8c14f98640 1 --2- 172.21.15.16:0/1706907801 >> v2:172.21.15.146:3300/0 conn(0x7f8c100d16a0 0x7f8c10162750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.202+0000 7f8c14f98640 1 --2- 172.21.15.16:0/1706907801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c1009bad0 0x7f8c10162210 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.202+0000 7f8c14f98640 1 --2- 172.21.15.16:0/1706907801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c1015e7b0 0x7f8c100d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.202+0000 7f8c14f98640 1 -- 172.21.15.16:0/1706907801 >> 172.21.15.16:0/1706907801 conn(0x7f8c100b3e70 msgr2=0x7f8c100b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.202+0000 7f8c14f98640 1 -- 172.21.15.16:0/1706907801 shutdown_connections 2024-09-19T07:53:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.202+0000 7f8c14f98640 1 -- 172.21.15.16:0/1706907801 wait complete. 2024-09-19T07:53:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.204+0000 7f8c14f98640 1 Processor -- start 2024-09-19T07:53:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.204+0000 7f8c14f98640 1 -- start start 2024-09-19T07:53:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.204+0000 7f8c14f98640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c1009bad0 0x7f8c10156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.204+0000 7f8c14f98640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c100d16a0 0x7f8c100b0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.204+0000 7f8c14f98640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c1015e7b0 0x7f8c100afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.204+0000 7f8c14f98640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c10070010 con 0x7f8c1015e7b0 2024-09-19T07:53:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.204+0000 7f8c14f98640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8c10162fb0 con 0x7f8c100d16a0 2024-09-19T07:53:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.204+0000 7f8c14f98640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8c10070190 con 0x7f8c1009bad0 2024-09-19T07:53:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.205+0000 7f8c0ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c1015e7b0 0x7f8c100afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.205+0000 7f8c0ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c1015e7b0 0x7f8c100afb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40562/0 (socket says 172.21.15.16:40562) 2024-09-19T07:53:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.205+0000 7f8c0ed9d640 1 -- 172.21.15.16:0/903253138 learned_addr learned my addr 172.21.15.16:0/903253138 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.205+0000 7f8c0e59c640 1 --2- 172.21.15.16:0/903253138 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c1009bad0 0x7f8c10156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.205+0000 7f8c0dd9b640 1 --2- 172.21.15.16:0/903253138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c100d16a0 0x7f8c100b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.205+0000 7f8c0ed9d640 1 -- 172.21.15.16:0/903253138 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c1009bad0 msgr2=0x7f8c10156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.205+0000 7f8c0ed9d640 1 --2- 172.21.15.16:0/903253138 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c1009bad0 0x7f8c10156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.205+0000 7f8c0ed9d640 1 -- 172.21.15.16:0/903253138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c100d16a0 msgr2=0x7f8c100b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.205+0000 7f8c0ed9d640 1 --2- 172.21.15.16:0/903253138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c100d16a0 0x7f8c100b0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.205+0000 7f8c0ed9d640 1 -- 172.21.15.16:0/903253138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c101f47d0 con 0x7f8c1015e7b0 2024-09-19T07:53:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.205+0000 7f8c0e59c640 1 --2- 172.21.15.16:0/903253138 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c1009bad0 0x7f8c10156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.205+0000 7f8c0dd9b640 1 --2- 172.21.15.16:0/903253138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c100d16a0 0x7f8c100b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:29.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.205+0000 7f8c0ed9d640 1 --2- 172.21.15.16:0/903253138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c1015e7b0 0x7f8c100afb00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000c820 tx=0x7f8c0000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:29.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.205+0000 7f8c06ffd640 1 -- 172.21.15.16:0/903253138 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00015070 con 0x7f8c1015e7b0 2024-09-19T07:53:29.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.205+0000 7f8c06ffd640 1 -- 172.21.15.16:0/903253138 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c00004240 con 0x7f8c1015e7b0 2024-09-19T07:53:29.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.205+0000 7f8c14f98640 1 -- 172.21.15.16:0/903253138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c101f4ab0 con 0x7f8c1015e7b0 2024-09-19T07:53:29.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.205+0000 7f8c06ffd640 1 -- 172.21.15.16:0/903253138 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0000ce10 con 0x7f8c1015e7b0 2024-09-19T07:53:29.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.205+0000 7f8c14f98640 1 -- 172.21.15.16:0/903253138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c101f5070 con 0x7f8c1015e7b0 2024-09-19T07:53:29.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.206+0000 7f8c14f98640 1 -- 172.21.15.16:0/903253138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c100d1160 con 0x7f8c1015e7b0 2024-09-19T07:53:29.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.207+0000 7f8c06ffd640 1 -- 172.21.15.16:0/903253138 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8c00007830 con 0x7f8c1015e7b0 2024-09-19T07:53:29.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.207+0000 7f8c06ffd640 1 --2- 172.21.15.16:0/903253138 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8bf006c2b0 0x7f8bf006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:29.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.207+0000 7f8c06ffd640 1 -- 172.21.15.16:0/903253138 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f8c0008b570 con 0x7f8c1015e7b0 2024-09-19T07:53:29.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.207+0000 7f8c0e59c640 1 --2- 172.21.15.16:0/903253138 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8bf006c2b0 0x7f8bf006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:29.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.208+0000 7f8c0e59c640 1 --2- 172.21.15.16:0/903253138 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8bf006c2b0 0x7f8bf006e770 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8c08007130 tx=0x7f8c08007080 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.212+0000 7f8c06ffd640 1 -- 172.21.15.16:0/903253138 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8c00059ab0 con 0x7f8c1015e7b0 2024-09-19T07:53:29.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.397+0000 7f8c14f98640 1 -- 172.21.15.16:0/903253138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8c100b4d80 con 0x7f8c1015e7b0 2024-09-19T07:53:29.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.398+0000 7f8c06ffd640 1 -- 172.21.15.16:0/903253138 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) ==== 80+0+0 (secure 0 0 0) 0x7f8c000593b0 con 0x7f8c1015e7b0 2024-09-19T07:53:29.398 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:53:29.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.400+0000 7f8c14f98640 1 -- 172.21.15.16:0/903253138 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8bf006c2b0 msgr2=0x7f8bf006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:29.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.400+0000 7f8c14f98640 1 --2- 172.21.15.16:0/903253138 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8bf006c2b0 0x7f8bf006e770 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8c08007130 tx=0x7f8c08007080 comp rx=0 tx=0).stop 2024-09-19T07:53:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.400+0000 7f8c14f98640 1 -- 172.21.15.16:0/903253138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c1015e7b0 msgr2=0x7f8c100afb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.400+0000 7f8c14f98640 1 --2- 172.21.15.16:0/903253138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c1015e7b0 0x7f8c100afb00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000c820 tx=0x7f8c0000cce0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.400+0000 7f8c14f98640 1 -- 172.21.15.16:0/903253138 shutdown_connections 2024-09-19T07:53:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.400+0000 7f8c14f98640 1 --2- 172.21.15.16:0/903253138 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8bf006c2b0 0x7f8bf006e770 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.400+0000 7f8c14f98640 1 --2- 172.21.15.16:0/903253138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c1015e7b0 0x7f8c100afb00 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.400+0000 7f8c14f98640 1 --2- 172.21.15.16:0/903253138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c100d16a0 0x7f8c100b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.400+0000 7f8c14f98640 1 --2- 172.21.15.16:0/903253138 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c1009bad0 0x7f8c10156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.400+0000 7f8c14f98640 1 -- 172.21.15.16:0/903253138 >> 172.21.15.16:0/903253138 conn(0x7f8c100b3e70 msgr2=0x7f8c1015eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.400+0000 7f8c14f98640 1 -- 172.21.15.16:0/903253138 shutdown_connections 2024-09-19T07:53:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.400+0000 7f8c14f98640 1 -- 172.21.15.16:0/903253138 wait complete. 2024-09-19T07:53:29.411 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2024-09-19T07:53:29.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:29.412 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T07:53:29.440 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[47679]: fuse finished with error 0 and tester_r 0 2024-09-19T07:53:29.442 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:53:29.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:29.442 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T07:53:29.501 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:53:29.501 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:53:29.502 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:53:29.518 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:53:29.519 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:53:29.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:29.519 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T07:53:29.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:53:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T07:53:29.576 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:53:29.576 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:53:29.576 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:53:29.594 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:53:29.594 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:53:29.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:29.594 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T07:53:29.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:53:29.653 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T07:53:29.653 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:53:29.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.755+0000 7fa86f82e640 1 Processor -- start 2024-09-19T07:53:29.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.756+0000 7fa86f82e640 1 -- start start 2024-09-19T07:53:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.756+0000 7fa86f82e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa86815e640 0x7fa868162250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.756+0000 7fa86f82e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa868162ea0 0x7fa8680b1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.756+0000 7fa86f82e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8680cf8b0 con 0x7fa8680d5040 2024-09-19T07:53:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.756+0000 7fa86f82e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa8680b12e0 con 0x7fa868162ea0 2024-09-19T07:53:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.756+0000 7fa86f82e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa8680cfb90 con 0x7fa86815e640 2024-09-19T07:53:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa86cdc9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa8680d5040 0x7fa868162790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49524/0 (socket says 172.21.15.16:49524) 2024-09-19T07:53:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa86cdc9640 1 -- 172.21.15.16:0/782798185 learned_addr learned my addr 172.21.15.16:0/782798185 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa85ffff640 1 -- 172.21.15.16:0/782798185 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1831510223 0 0) 0x7fa8680cf8b0 con 0x7fa8680d5040 2024-09-19T07:53:29.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa85ffff640 1 -- 172.21.15.16:0/782798185 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa854003520 con 0x7fa8680d5040 2024-09-19T07:53:29.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa86d5ca640 1 --2- 172.21.15.16:0/782798185 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa86815e640 0x7fa868162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:29.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa86ddcb640 1 --2- 172.21.15.16:0/782798185 >> v2:172.21.15.146:3300/0 conn(0x7fa868162ea0 0x7fa8680b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:29.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa85ffff640 1 -- 172.21.15.16:0/782798185 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1396681782 0 0) 0x7fa854003520 con 0x7fa8680d5040 2024-09-19T07:53:29.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa85ffff640 1 -- 172.21.15.16:0/782798185 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa86815e640 msgr2=0x7fa868162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:29.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa85ffff640 1 --2- 172.21.15.16:0/782798185 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa86815e640 0x7fa868162250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa85ffff640 1 -- 172.21.15.16:0/782798185 >> v2:172.21.15.146:3300/0 conn(0x7fa868162ea0 msgr2=0x7fa8680b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:29.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa85ffff640 1 --2- 172.21.15.16:0/782798185 >> v2:172.21.15.146:3300/0 conn(0x7fa868162ea0 0x7fa8680b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa85ffff640 1 -- 172.21.15.16:0/782798185 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8680cfe70 con 0x7fa8680d5040 2024-09-19T07:53:29.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa85ffff640 1 -- 172.21.15.16:0/782798185 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fa8580017f0 con 0x7fa8680d5040 2024-09-19T07:53:29.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa85ffff640 1 -- 172.21.15.16:0/782798185 >> v1:172.21.15.16:6789/0 conn(0x7fa8680d5040 legacy=0x7fa868162790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:29.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa85ffff640 1 --2- 172.21.15.16:0/782798185 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8540039b0 0x7fa854003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:29.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa85ffff640 1 --2- 172.21.15.16:0/782798185 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa86815e640 0x7fa854004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:29.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa85ffff640 1 --2- 172.21.15.16:0/782798185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa854004d40 0x7fa8540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:29.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa85ffff640 1 -- 172.21.15.16:0/782798185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8680b12e0 con 0x7fa854004d40 2024-09-19T07:53:29.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa85ffff640 1 -- 172.21.15.16:0/782798185 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa8680cfb90 con 0x7fa8540039b0 2024-09-19T07:53:29.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa85ffff640 1 -- 172.21.15.16:0/782798185 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa85400b9c0 con 0x7fa86815e640 2024-09-19T07:53:29.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa86ddcb640 1 --2- 172.21.15.16:0/782798185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa854004d40 0x7fa8540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:29.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa86d5ca640 1 --2- 172.21.15.16:0/782798185 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8540039b0 0x7fa854003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:29.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.757+0000 7fa86cdc9640 1 --2- 172.21.15.16:0/782798185 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa86815e640 0x7fa854004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:29.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.758+0000 7fa86ddcb640 1 -- 172.21.15.16:0/782798185 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa86815e640 msgr2=0x7fa854004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:29.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.758+0000 7fa86ddcb640 1 --2- 172.21.15.16:0/782798185 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa86815e640 0x7fa854004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.758+0000 7fa86ddcb640 1 -- 172.21.15.16:0/782798185 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8540039b0 msgr2=0x7fa854003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:29.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.758+0000 7fa86ddcb640 1 --2- 172.21.15.16:0/782798185 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8540039b0 0x7fa854003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.758+0000 7fa86ddcb640 1 -- 172.21.15.16:0/782798185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa854003520 con 0x7fa854004d40 2024-09-19T07:53:29.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.758+0000 7fa86ddcb640 1 --2- 172.21.15.16:0/782798185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa854004d40 0x7fa8540051c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa860009fc0 tx=0x7fa854005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c983d88abba3ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:29.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.758+0000 7fa85ffff640 1 -- 172.21.15.16:0/782798185 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa860009c10 con 0x7fa854004d40 2024-09-19T07:53:29.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.759+0000 7fa85ffff640 1 -- 172.21.15.16:0/782798185 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa860036070 con 0x7fa854004d40 2024-09-19T07:53:29.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.759+0000 7fa85ffff640 1 -- 172.21.15.16:0/782798185 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86002d070 con 0x7fa854004d40 2024-09-19T07:53:29.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.759+0000 7fa86f82e640 1 -- 172.21.15.16:0/782798185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa854004d40 msgr2=0x7fa8540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:29.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.759+0000 7fa86f82e640 1 --2- 172.21.15.16:0/782798185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa854004d40 0x7fa8540051c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa860009fc0 tx=0x7fa854005830 comp rx=0 tx=0).stop 2024-09-19T07:53:29.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.759+0000 7fa86ddcb640 1 -- 172.21.15.16:0/782798185 reap_dead start 2024-09-19T07:53:29.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.759+0000 7fa86f82e640 1 -- 172.21.15.16:0/782798185 shutdown_connections 2024-09-19T07:53:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.759+0000 7fa86f82e640 1 -- 172.21.15.16:0/782798185 >> 172.21.15.16:0/782798185 conn(0x7fa86808bfe0 msgr2=0x7fa8680d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.759+0000 7fa86f82e640 1 -- 172.21.15.16:0/782798185 shutdown_connections 2024-09-19T07:53:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.759+0000 7fa86f82e640 1 -- 172.21.15.16:0/782798185 wait complete. 2024-09-19T07:53:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.760+0000 7fa86f82e640 1 Processor -- start 2024-09-19T07:53:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.760+0000 7fa86f82e640 1 -- start start 2024-09-19T07:53:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.761+0000 7fa86f82e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa854004d40 0x7fa868158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.761+0000 7fa86f82e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8680a0c50 0x7fa8680a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.761+0000 7fa86f82e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8680d34a0 0x7fa8680d3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.761+0000 7fa86f82e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa86815eba0 con 0x7fa8680a0c50 2024-09-19T07:53:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.761+0000 7fa86f82e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa86815ea20 con 0x7fa8680d34a0 2024-09-19T07:53:29.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.761+0000 7fa86f82e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa868162250 con 0x7fa854004d40 2024-09-19T07:53:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.761+0000 7fa86cdc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8680a0c50 0x7fa8680a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.761+0000 7fa86cdc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8680a0c50 0x7fa8680a10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40586/0 (socket says 172.21.15.16:40586) 2024-09-19T07:53:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.761+0000 7fa86cdc9640 1 -- 172.21.15.16:0/503711633 learned_addr learned my addr 172.21.15.16:0/503711633 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.762+0000 7fa86cdc9640 1 -- 172.21.15.16:0/503711633 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa854004d40 msgr2=0x7fa868158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.762+0000 7fa86ddcb640 1 --2- 172.21.15.16:0/503711633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8680d34a0 0x7fa8680d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.762+0000 7fa86d5ca640 1 --2- 172.21.15.16:0/503711633 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa854004d40 0x7fa868158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.762+0000 7fa86cdc9640 1 --2- 172.21.15.16:0/503711633 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa854004d40 0x7fa868158570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.762+0000 7fa86cdc9640 1 -- 172.21.15.16:0/503711633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8680d34a0 msgr2=0x7fa8680d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.762+0000 7fa86cdc9640 1 --2- 172.21.15.16:0/503711633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8680d34a0 0x7fa8680d3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.762+0000 7fa86cdc9640 1 -- 172.21.15.16:0/503711633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86809e530 con 0x7fa8680a0c50 2024-09-19T07:53:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.762+0000 7fa86cdc9640 1 --2- 172.21.15.16:0/503711633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8680a0c50 0x7fa8680a10d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fa85800f930 tx=0x7fa85800fdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.762+0000 7fa85dffb640 1 -- 172.21.15.16:0/503711633 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8580072e0 con 0x7fa8680a0c50 2024-09-19T07:53:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.762+0000 7fa85dffb640 1 -- 172.21.15.16:0/503711633 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa858018070 con 0x7fa8680a0c50 2024-09-19T07:53:29.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.762+0000 7fa86f82e640 1 -- 172.21.15.16:0/503711633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa86809e810 con 0x7fa8680a0c50 2024-09-19T07:53:29.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.762+0000 7fa85dffb640 1 -- 172.21.15.16:0/503711633 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8580057a0 con 0x7fa8680a0c50 2024-09-19T07:53:29.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.762+0000 7fa86f82e640 1 -- 172.21.15.16:0/503711633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8680cc000 con 0x7fa8680a0c50 2024-09-19T07:53:29.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.763+0000 7fa86f82e640 1 -- 172.21.15.16:0/503711633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa868162610 con 0x7fa8680a0c50 2024-09-19T07:53:29.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.764+0000 7fa85dffb640 1 -- 172.21.15.16:0/503711633 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa858005940 con 0x7fa8680a0c50 2024-09-19T07:53:29.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.764+0000 7fa85dffb640 1 --2- 172.21.15.16:0/503711633 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa84c06c2e0 0x7fa84c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:29.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.764+0000 7fa85dffb640 1 -- 172.21.15.16:0/503711633 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fa85808b600 con 0x7fa8680a0c50 2024-09-19T07:53:29.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.764+0000 7fa86d5ca640 1 --2- 172.21.15.16:0/503711633 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa84c06c2e0 0x7fa84c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:29.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.765+0000 7fa86d5ca640 1 --2- 172.21.15.16:0/503711633 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa84c06c2e0 0x7fa84c06e7a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa864009980 tx=0x7fa864008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:29.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.768+0000 7fa85dffb640 1 -- 172.21.15.16:0/503711633 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa858059c70 con 0x7fa8680a0c50 2024-09-19T07:53:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.982+0000 7fa86f82e640 1 -- 172.21.15.16:0/503711633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8680b6ce0 con 0x7fa8680a0c50 2024-09-19T07:53:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.982+0000 7fa85dffb640 1 -- 172.21.15.16:0/503711633 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5228 (secure 0 0 0) 0x7fa858059570 con 0x7fa8680a0c50 2024-09-19T07:53:29.983 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:29.983 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":57,"btime":"2024-09-19T07:53:22:988089+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6017,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2955839490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":2955839490},{"type":"v1","addr":"172.21.15.146:6837","nonce":2955839490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6550,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/395357736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":395357736},{"type":"v1","addr":"172.21.15.146:6836","nonce":395357736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3539638739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3539638739},{"type":"v1","addr":"172.21.15.16:6834","nonce":3539638739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:53:02.264799+0000","modified":"2024-09-19T07:53:22.976173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/4185904511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4185904511},{"type":"v1","addr":"172.21.15.16:6835","nonce":4185904511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":7}]} 2024-09-19T07:53:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 57 2024-09-19T07:53:29.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.984+0000 7fa86f82e640 1 -- 172.21.15.16:0/503711633 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa84c06c2e0 msgr2=0x7fa84c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:29.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.984+0000 7fa86f82e640 1 --2- 172.21.15.16:0/503711633 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa84c06c2e0 0x7fa84c06e7a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa864009980 tx=0x7fa864008040 comp rx=0 tx=0).stop 2024-09-19T07:53:29.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.984+0000 7fa86f82e640 1 -- 172.21.15.16:0/503711633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8680a0c50 msgr2=0x7fa8680a10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:29.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.984+0000 7fa86f82e640 1 --2- 172.21.15.16:0/503711633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8680a0c50 0x7fa8680a10d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fa85800f930 tx=0x7fa85800fdf0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.984+0000 7fa86f82e640 1 -- 172.21.15.16:0/503711633 shutdown_connections 2024-09-19T07:53:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.984+0000 7fa86f82e640 1 --2- 172.21.15.16:0/503711633 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa84c06c2e0 0x7fa84c06e7a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.984+0000 7fa86f82e640 1 --2- 172.21.15.16:0/503711633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8680d34a0 0x7fa8680d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.984+0000 7fa86f82e640 1 --2- 172.21.15.16:0/503711633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8680a0c50 0x7fa8680a10d0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.984+0000 7fa86f82e640 1 --2- 172.21.15.16:0/503711633 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa854004d40 0x7fa868158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.984+0000 7fa86f82e640 1 -- 172.21.15.16:0/503711633 >> 172.21.15.16:0/503711633 conn(0x7fa86808bfe0 msgr2=0x7fa86815f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.984+0000 7fa86f82e640 1 -- 172.21.15.16:0/503711633 shutdown_connections 2024-09-19T07:53:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:29.984+0000 7fa86f82e640 1 -- 172.21.15.16:0/503711633 wait complete. 2024-09-19T07:53:29.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:53:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.097+0000 7f6dff742640 1 Processor -- start 2024-09-19T07:53:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.098+0000 7f6dff742640 1 -- start start 2024-09-19T07:53:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.098+0000 7f6dff742640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6df80b2780 0x7f6df8169580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.098+0000 7f6dff742640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6df80cf8b0 con 0x7f6df815e640 2024-09-19T07:53:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.098+0000 7f6dff742640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6df80cfb90 con 0x7f6df80d5040 2024-09-19T07:53:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.098+0000 7f6dff742640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6df80b12e0 con 0x7f6df80b2780 2024-09-19T07:53:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.098+0000 7f6dfd4de640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6df815e640 0x7f6df8162250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49528/0 (socket says 172.21.15.16:49528) 2024-09-19T07:53:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.098+0000 7f6dfd4de640 1 -- 172.21.15.16:0/1618844317 learned_addr learned my addr 172.21.15.16:0/1618844317 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.099+0000 7f6dfdcdf640 1 --2- 172.21.15.16:0/1618844317 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6df80b2780 0x7f6df8169580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.099+0000 7f6df7fff640 1 -- 172.21.15.16:0/1618844317 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3529720963 0 0) 0x7f6df80cf8b0 con 0x7f6df815e640 2024-09-19T07:53:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.099+0000 7f6df7fff640 1 -- 172.21.15.16:0/1618844317 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6de0003520 con 0x7f6df815e640 2024-09-19T07:53:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.099+0000 7f6df7fff640 1 -- 172.21.15.16:0/1618844317 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 343808039 0 0) 0x7f6de0003520 con 0x7f6df815e640 2024-09-19T07:53:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.099+0000 7f6df7fff640 1 -- 172.21.15.16:0/1618844317 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6df80b2780 msgr2=0x7f6df8169580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.099+0000 7f6df7fff640 1 --2- 172.21.15.16:0/1618844317 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6df80b2780 0x7f6df8169580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.099+0000 7f6df7fff640 1 -- 172.21.15.16:0/1618844317 >> v1:172.21.15.146:6789/0 conn(0x7f6df80d5040 legacy=0x7f6df8162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.099+0000 7f6df7fff640 1 -- 172.21.15.16:0/1618844317 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df80cfe70 con 0x7f6df815e640 2024-09-19T07:53:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.099+0000 7f6df7fff640 1 -- 172.21.15.16:0/1618844317 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f6de80019f0 con 0x7f6df815e640 2024-09-19T07:53:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.099+0000 7f6df7fff640 1 -- 172.21.15.16:0/1618844317 >> v1:172.21.15.16:6789/0 conn(0x7f6df815e640 legacy=0x7f6df8162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.099+0000 7f6df7fff640 1 --2- 172.21.15.16:0/1618844317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de00039e0 0x7f6de0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.099+0000 7f6df7fff640 1 --2- 172.21.15.16:0/1618844317 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6df80b2780 0x7f6de0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.099+0000 7f6df7fff640 1 --2- 172.21.15.16:0/1618844317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de0004d40 0x7f6de00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.099+0000 7f6df7fff640 1 -- 172.21.15.16:0/1618844317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6df80b12e0 con 0x7f6de00039e0 2024-09-19T07:53:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.099+0000 7f6df7fff640 1 -- 172.21.15.16:0/1618844317 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6de000baa0 con 0x7f6de0004d40 2024-09-19T07:53:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.099+0000 7f6df7fff640 1 -- 172.21.15.16:0/1618844317 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6de000bca0 con 0x7f6df80b2780 2024-09-19T07:53:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.099+0000 7f6dfd4de640 1 --2- 172.21.15.16:0/1618844317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de00039e0 0x7f6de0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.099+0000 7f6dfdcdf640 1 --2- 172.21.15.16:0/1618844317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de0004d40 0x7f6de00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.099+0000 7f6dfccdd640 1 --2- 172.21.15.16:0/1618844317 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6df80b2780 0x7f6de0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.100+0000 7f6dfd4de640 1 -- 172.21.15.16:0/1618844317 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6df80b2780 msgr2=0x7f6de0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.100+0000 7f6dfd4de640 1 --2- 172.21.15.16:0/1618844317 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6df80b2780 0x7f6de0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.100+0000 7f6dfd4de640 1 -- 172.21.15.16:0/1618844317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de0004d40 msgr2=0x7f6de00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.100+0000 7f6dfd4de640 1 --2- 172.21.15.16:0/1618844317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de0004d40 0x7f6de00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.100+0000 7f6dfd4de640 1 -- 172.21.15.16:0/1618844317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de0003520 con 0x7f6de00039e0 2024-09-19T07:53:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.100+0000 7f6dfdcdf640 1 --2- 172.21.15.16:0/1618844317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de0004d40 0x7f6de00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.100+0000 7f6dfccdd640 1 --2- 172.21.15.16:0/1618844317 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6df80b2780 0x7f6de0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.100+0000 7f6dfd4de640 1 --2- 172.21.15.16:0/1618844317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de00039e0 0x7f6de0003e00 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6de80013d0 tx=0x7f6de0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c65e384d8e1323a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.100+0000 7f6df7fff640 1 -- 172.21.15.16:0/1618844317 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de800a4a0 con 0x7f6de00039e0 2024-09-19T07:53:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.100+0000 7f6df7fff640 1 -- 172.21.15.16:0/1618844317 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6de802f070 con 0x7f6de00039e0 2024-09-19T07:53:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.100+0000 7f6df7fff640 1 -- 172.21.15.16:0/1618844317 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de802bc90 con 0x7f6de00039e0 2024-09-19T07:53:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.101+0000 7f6dff742640 1 -- 172.21.15.16:0/1618844317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de00039e0 msgr2=0x7f6de0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.101+0000 7f6dff742640 1 --2- 172.21.15.16:0/1618844317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de00039e0 0x7f6de0003e00 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6de80013d0 tx=0x7f6de0005830 comp rx=0 tx=0).stop 2024-09-19T07:53:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.101+0000 7f6dfdcdf640 1 -- 172.21.15.16:0/1618844317 reap_dead start 2024-09-19T07:53:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.101+0000 7f6dff742640 1 -- 172.21.15.16:0/1618844317 shutdown_connections 2024-09-19T07:53:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.101+0000 7f6dff742640 1 -- 172.21.15.16:0/1618844317 >> 172.21.15.16:0/1618844317 conn(0x7f6df808bfe0 msgr2=0x7f6df80d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.101+0000 7f6dff742640 1 -- 172.21.15.16:0/1618844317 shutdown_connections 2024-09-19T07:53:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.101+0000 7f6dff742640 1 -- 172.21.15.16:0/1618844317 wait complete. 2024-09-19T07:53:30.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.102+0000 7f6dff742640 1 Processor -- start 2024-09-19T07:53:30.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.102+0000 7f6dff742640 1 -- start start 2024-09-19T07:53:30.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.103+0000 7f6dff742640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de00039e0 0x7f6df8150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.103+0000 7f6dff742640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df814f6b0 0x7f6df815a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.103+0000 7f6dff742640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6df814eb90 0x7f6df80ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.103+0000 7f6dff742640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6df815ea20 con 0x7f6df814f6b0 2024-09-19T07:53:30.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.103+0000 7f6dff742640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6df80cf540 con 0x7f6de00039e0 2024-09-19T07:53:30.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.103+0000 7f6dff742640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6df815eba0 con 0x7f6df814eb90 2024-09-19T07:53:30.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.103+0000 7f6dfccdd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df814f6b0 0x7f6df815a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.103+0000 7f6dfccdd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df814f6b0 0x7f6df815a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40598/0 (socket says 172.21.15.16:40598) 2024-09-19T07:53:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.103+0000 7f6dfccdd640 1 -- 172.21.15.16:0/422520798 learned_addr learned my addr 172.21.15.16:0/422520798 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.103+0000 7f6dfd4de640 1 --2- 172.21.15.16:0/422520798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de00039e0 0x7f6df8150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.103+0000 7f6dfdcdf640 1 --2- 172.21.15.16:0/422520798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6df814eb90 0x7f6df80ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.103+0000 7f6dfccdd640 1 -- 172.21.15.16:0/422520798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6df814eb90 msgr2=0x7f6df80ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.103+0000 7f6dfccdd640 1 --2- 172.21.15.16:0/422520798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6df814eb90 0x7f6df80ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.103+0000 7f6dfccdd640 1 -- 172.21.15.16:0/422520798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de00039e0 msgr2=0x7f6df8150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.103+0000 7f6dfccdd640 1 --2- 172.21.15.16:0/422520798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de00039e0 0x7f6df8150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.103+0000 7f6dfccdd640 1 -- 172.21.15.16:0/422520798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df815aad0 con 0x7f6df814f6b0 2024-09-19T07:53:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.103+0000 7f6dfd4de640 1 --2- 172.21.15.16:0/422520798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de00039e0 0x7f6df8150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.104+0000 7f6dfccdd640 1 --2- 172.21.15.16:0/422520798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df814f6b0 0x7f6df815a590 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00d800 tx=0x7f6dec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.104+0000 7f6df5ffb640 1 -- 172.21.15.16:0/422520798 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec005070 con 0x7f6df814f6b0 2024-09-19T07:53:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.104+0000 7f6df5ffb640 1 -- 172.21.15.16:0/422520798 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dec005210 con 0x7f6df814f6b0 2024-09-19T07:53:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.104+0000 7f6dff742640 1 -- 172.21.15.16:0/422520798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df8151520 con 0x7f6df814f6b0 2024-09-19T07:53:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.104+0000 7f6df5ffb640 1 -- 172.21.15.16:0/422520798 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec006820 con 0x7f6df814f6b0 2024-09-19T07:53:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.104+0000 7f6dff742640 1 -- 172.21.15.16:0/422520798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df8151ae0 con 0x7f6df814f6b0 2024-09-19T07:53:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.105+0000 7f6dff742640 1 -- 172.21.15.16:0/422520798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df8162250 con 0x7f6df814f6b0 2024-09-19T07:53:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.106+0000 7f6df5ffb640 1 -- 172.21.15.16:0/422520798 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6dec01a020 con 0x7f6df814f6b0 2024-09-19T07:53:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.106+0000 7f6df5ffb640 1 --2- 172.21.15.16:0/422520798 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6de406c2b0 0x7f6de406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.106+0000 7f6df5ffb640 1 -- 172.21.15.16:0/422520798 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f6dec08acf0 con 0x7f6df814f6b0 2024-09-19T07:53:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.106+0000 7f6dfd4de640 1 --2- 172.21.15.16:0/422520798 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6de406c2b0 0x7f6de406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.107+0000 7f6dfd4de640 1 --2- 172.21.15.16:0/422520798 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6de406c2b0 0x7f6de406e770 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6de8033000 tx=0x7f6de8007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.110+0000 7f6df5ffb640 1 -- 172.21.15.16:0/422520798 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6dec08f070 con 0x7f6df814f6b0 2024-09-19T07:53:30.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.322+0000 7f6dff742640 1 -- 172.21.15.16:0/422520798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6df806dba0 con 0x7f6df814f6b0 2024-09-19T07:53:30.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.322+0000 7f6df5ffb640 1 -- 172.21.15.16:0/422520798 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5228 (secure 0 0 0) 0x7f6dec059360 con 0x7f6df814f6b0 2024-09-19T07:53:30.323 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:30.323 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":57,"btime":"2024-09-19T07:53:22:988089+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6017,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2955839490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":2955839490},{"type":"v1","addr":"172.21.15.146:6837","nonce":2955839490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6550,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/395357736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":395357736},{"type":"v1","addr":"172.21.15.146:6836","nonce":395357736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3539638739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3539638739},{"type":"v1","addr":"172.21.15.16:6834","nonce":3539638739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:53:02.264799+0000","modified":"2024-09-19T07:53:22.976173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/4185904511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4185904511},{"type":"v1","addr":"172.21.15.16:6835","nonce":4185904511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":7}]} 2024-09-19T07:53:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 57 2024-09-19T07:53:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.324+0000 7f6dff742640 1 -- 172.21.15.16:0/422520798 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6de406c2b0 msgr2=0x7f6de406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.324+0000 7f6dff742640 1 --2- 172.21.15.16:0/422520798 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6de406c2b0 0x7f6de406e770 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6de8033000 tx=0x7f6de8007000 comp rx=0 tx=0).stop 2024-09-19T07:53:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.324+0000 7f6dff742640 1 -- 172.21.15.16:0/422520798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df814f6b0 msgr2=0x7f6df815a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.324+0000 7f6dff742640 1 --2- 172.21.15.16:0/422520798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df814f6b0 0x7f6df815a590 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00d800 tx=0x7f6dec00dcc0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.325+0000 7f6dff742640 1 -- 172.21.15.16:0/422520798 shutdown_connections 2024-09-19T07:53:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.325+0000 7f6dff742640 1 --2- 172.21.15.16:0/422520798 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6de406c2b0 0x7f6de406e770 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.325+0000 7f6dff742640 1 --2- 172.21.15.16:0/422520798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6df814eb90 0x7f6df80ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.325+0000 7f6dff742640 1 --2- 172.21.15.16:0/422520798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df814f6b0 0x7f6df815a590 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.325+0000 7f6dff742640 1 --2- 172.21.15.16:0/422520798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de00039e0 0x7f6df8150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.325+0000 7f6dff742640 1 -- 172.21.15.16:0/422520798 >> 172.21.15.16:0/422520798 conn(0x7f6df808bfe0 msgr2=0x7f6df806e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.325+0000 7f6dff742640 1 -- 172.21.15.16:0/422520798 shutdown_connections 2024-09-19T07:53:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.325+0000 7f6dff742640 1 -- 172.21.15.16:0/422520798 wait complete. 2024-09-19T07:53:30.336 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:53:30.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.437+0000 7f4942318640 1 Processor -- start 2024-09-19T07:53:30.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.437+0000 7f4942318640 1 -- start start 2024-09-19T07:53:30.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.438+0000 7f4942318640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f493c161e00 0x7f493c0cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.438+0000 7f4942318640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f493c0ce670 0x7f493c0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.438+0000 7f4942318640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f493c0d0230 con 0x7f493c0cf000 2024-09-19T07:53:30.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.438+0000 7f4942318640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f493c0d32d0 con 0x7f493c161e00 2024-09-19T07:53:30.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.438+0000 7f4942318640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f493c0b5e20 con 0x7f493c0ce670 2024-09-19T07:53:30.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.438+0000 7f49408b5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f493c0cf000 0x7f493c0d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49534/0 (socket says 172.21.15.16:49534) 2024-09-19T07:53:30.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.438+0000 7f49408b5640 1 -- 172.21.15.16:0/453472783 learned_addr learned my addr 172.21.15.16:0/453472783 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:30.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.438+0000 7f493b7fe640 1 --2- 172.21.15.16:0/453472783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f493c0ce670 0x7f493c0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.438+0000 7f493affd640 1 -- 172.21.15.16:0/453472783 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3555865633 0 0) 0x7f493c0d0230 con 0x7f493c0cf000 2024-09-19T07:53:30.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493affd640 1 -- 172.21.15.16:0/453472783 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4924003520 con 0x7f493c0cf000 2024-09-19T07:53:30.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493bfff640 1 --2- 172.21.15.16:0/453472783 >> v2:172.21.15.146:3300/0 conn(0x7f493c161e00 0x7f493c0cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493affd640 1 -- 172.21.15.16:0/453472783 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3845005414 0 0) 0x7f4924003520 con 0x7f493c0cf000 2024-09-19T07:53:30.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493affd640 1 -- 172.21.15.16:0/453472783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f493c0ce670 msgr2=0x7f493c0cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:30.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493affd640 1 --2- 172.21.15.16:0/453472783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f493c0ce670 0x7f493c0cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493affd640 1 -- 172.21.15.16:0/453472783 >> v2:172.21.15.146:3300/0 conn(0x7f493c161e00 msgr2=0x7f493c0cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:30.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493affd640 1 --2- 172.21.15.16:0/453472783 >> v2:172.21.15.146:3300/0 conn(0x7f493c161e00 0x7f493c0cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493affd640 1 -- 172.21.15.16:0/453472783 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493c0b60e0 con 0x7f493c0cf000 2024-09-19T07:53:30.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493affd640 1 -- 172.21.15.16:0/453472783 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f49340019f0 con 0x7f493c0cf000 2024-09-19T07:53:30.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493affd640 1 -- 172.21.15.16:0/453472783 >> v1:172.21.15.16:6789/0 conn(0x7f493c0cf000 legacy=0x7f493c0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:30.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493affd640 1 --2- 172.21.15.16:0/453472783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49240039b0 0x7f4924003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493affd640 1 --2- 172.21.15.16:0/453472783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4924004800 0x7f4924004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493affd640 1 --2- 172.21.15.16:0/453472783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f493c0ce670 0x7f49240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493affd640 1 -- 172.21.15.16:0/453472783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f493c0b5e20 con 0x7f4924004800 2024-09-19T07:53:30.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493affd640 1 -- 172.21.15.16:0/453472783 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f492400b9c0 con 0x7f49240039b0 2024-09-19T07:53:30.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493affd640 1 -- 172.21.15.16:0/453472783 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f492400bb40 con 0x7f493c0ce670 2024-09-19T07:53:30.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493b7fe640 1 --2- 172.21.15.16:0/453472783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4924004800 0x7f4924004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493bfff640 1 --2- 172.21.15.16:0/453472783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49240039b0 0x7f4924003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f49408b5640 1 --2- 172.21.15.16:0/453472783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f493c0ce670 0x7f49240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493b7fe640 1 -- 172.21.15.16:0/453472783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f493c0ce670 msgr2=0x7f49240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:30.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493b7fe640 1 --2- 172.21.15.16:0/453472783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f493c0ce670 0x7f49240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493b7fe640 1 -- 172.21.15.16:0/453472783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49240039b0 msgr2=0x7f4924003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:30.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493b7fe640 1 --2- 172.21.15.16:0/453472783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49240039b0 0x7f4924003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.439+0000 7f493b7fe640 1 -- 172.21.15.16:0/453472783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4924003520 con 0x7f4924004800 2024-09-19T07:53:30.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.440+0000 7f493b7fe640 1 --2- 172.21.15.16:0/453472783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4924004800 0x7f4924004c80 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f4930009ac0 tx=0x7f4924005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3667c1fed275886 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:30.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.440+0000 7f493affd640 1 -- 172.21.15.16:0/453472783 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930036070 con 0x7f4924004800 2024-09-19T07:53:30.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.440+0000 7f493affd640 1 -- 172.21.15.16:0/453472783 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4930031400 con 0x7f4924004800 2024-09-19T07:53:30.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.440+0000 7f493affd640 1 -- 172.21.15.16:0/453472783 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930009690 con 0x7f4924004800 2024-09-19T07:53:30.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.441+0000 7f4942318640 1 -- 172.21.15.16:0/453472783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4924004800 msgr2=0x7f4924004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:30.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.441+0000 7f4942318640 1 --2- 172.21.15.16:0/453472783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4924004800 0x7f4924004c80 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f4930009ac0 tx=0x7f4924005830 comp rx=0 tx=0).stop 2024-09-19T07:53:30.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.441+0000 7f49408b5640 1 -- 172.21.15.16:0/453472783 reap_dead start 2024-09-19T07:53:30.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.441+0000 7f4942318640 1 -- 172.21.15.16:0/453472783 shutdown_connections 2024-09-19T07:53:30.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.441+0000 7f4942318640 1 -- 172.21.15.16:0/453472783 >> 172.21.15.16:0/453472783 conn(0x7f493c08bfe0 msgr2=0x7f493c0d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:30.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.441+0000 7f4942318640 1 -- 172.21.15.16:0/453472783 shutdown_connections 2024-09-19T07:53:30.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.441+0000 7f4942318640 1 -- 172.21.15.16:0/453472783 wait complete. 2024-09-19T07:53:30.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.442+0000 7f4942318640 1 Processor -- start 2024-09-19T07:53:30.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.442+0000 7f4942318640 1 -- start start 2024-09-19T07:53:30.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.443+0000 7f4942318640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4924004800 0x7f493c1ea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.443+0000 7f4942318640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f493c0cdf60 0x7f493c1eac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.443+0000 7f4942318640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c0cb5b0 0x7f493c0cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.443+0000 7f4942318640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f493c0b6980 con 0x7f4924004800 2024-09-19T07:53:30.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.443+0000 7f4942318640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f493c0b66a0 con 0x7f493c0cb5b0 2024-09-19T07:53:30.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.443+0000 7f4942318640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f493c0d3e70 con 0x7f493c0cdf60 2024-09-19T07:53:30.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.443+0000 7f493bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4924004800 0x7f493c1ea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.443+0000 7f493bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4924004800 0x7f493c1ea6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40610/0 (socket says 172.21.15.16:40610) 2024-09-19T07:53:30.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.443+0000 7f493bfff640 1 -- 172.21.15.16:0/3328144528 learned_addr learned my addr 172.21.15.16:0/3328144528 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:30.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.443+0000 7f49408b5640 1 --2- 172.21.15.16:0/3328144528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c0cb5b0 0x7f493c0cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.443+0000 7f493b7fe640 1 --2- 172.21.15.16:0/3328144528 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f493c0cdf60 0x7f493c1eac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.443+0000 7f493bfff640 1 -- 172.21.15.16:0/3328144528 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f493c0cdf60 msgr2=0x7f493c1eac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:30.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.443+0000 7f493bfff640 1 --2- 172.21.15.16:0/3328144528 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f493c0cdf60 0x7f493c1eac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.443+0000 7f493bfff640 1 -- 172.21.15.16:0/3328144528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c0cb5b0 msgr2=0x7f493c0cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:30.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.443+0000 7f493bfff640 1 --2- 172.21.15.16:0/3328144528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c0cb5b0 0x7f493c0cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.443+0000 7f493bfff640 1 -- 172.21.15.16:0/3328144528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493c1575b0 con 0x7f4924004800 2024-09-19T07:53:30.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.443+0000 7f49408b5640 1 --2- 172.21.15.16:0/3328144528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c0cb5b0 0x7f493c0cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:30.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.443+0000 7f493bfff640 1 --2- 172.21.15.16:0/3328144528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4924004800 0x7f493c1ea6d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f492c011990 tx=0x7f492c011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:30.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.444+0000 7f4938ff9640 1 -- 172.21.15.16:0/3328144528 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c00fad0 con 0x7f4924004800 2024-09-19T07:53:30.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.444+0000 7f4938ff9640 1 -- 172.21.15.16:0/3328144528 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f492c00fc70 con 0x7f4924004800 2024-09-19T07:53:30.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.444+0000 7f4942318640 1 -- 172.21.15.16:0/3328144528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f493c157890 con 0x7f4924004800 2024-09-19T07:53:30.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.444+0000 7f4938ff9640 1 -- 172.21.15.16:0/3328144528 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c014440 con 0x7f4924004800 2024-09-19T07:53:30.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.444+0000 7f4942318640 1 -- 172.21.15.16:0/3328144528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f493c157e50 con 0x7f4924004800 2024-09-19T07:53:30.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.445+0000 7f4942318640 1 -- 172.21.15.16:0/3328144528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f493c0d3760 con 0x7f4924004800 2024-09-19T07:53:30.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.445+0000 7f4938ff9640 1 -- 172.21.15.16:0/3328144528 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f492c012040 con 0x7f4924004800 2024-09-19T07:53:30.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.446+0000 7f4938ff9640 1 --2- 172.21.15.16:0/3328144528 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f492806c2b0 0x7f492806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.446+0000 7f4938ff9640 1 -- 172.21.15.16:0/3328144528 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f492c08bc40 con 0x7f4924004800 2024-09-19T07:53:30.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.446+0000 7f493b7fe640 1 --2- 172.21.15.16:0/3328144528 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f492806c2b0 0x7f492806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.447+0000 7f493b7fe640 1 --2- 172.21.15.16:0/3328144528 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f492806c2b0 0x7f492806e770 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4930033000 tx=0x7f4930008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:30.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.450+0000 7f4938ff9640 1 -- 172.21.15.16:0/3328144528 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f492c019030 con 0x7f4924004800 2024-09-19T07:53:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.625+0000 7f4942318640 1 -- 172.21.15.16:0/3328144528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f493c162520 con 0x7f4924004800 2024-09-19T07:53:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.626+0000 7f4938ff9640 1 -- 172.21.15.16:0/3328144528 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16968 (secure 0 0 0) 0x7f492c05a220 con 0x7f4924004800 2024-09-19T07:53:30.627 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:30.627 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":106,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:53:03.939259+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:52:59.931281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-19T07:53:01.258168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:53:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.628+0000 7f4942318640 1 -- 172.21.15.16:0/3328144528 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f492806c2b0 msgr2=0x7f492806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.628+0000 7f4942318640 1 --2- 172.21.15.16:0/3328144528 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f492806c2b0 0x7f492806e770 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4930033000 tx=0x7f4930008030 comp rx=0 tx=0).stop 2024-09-19T07:53:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.628+0000 7f4942318640 1 -- 172.21.15.16:0/3328144528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4924004800 msgr2=0x7f493c1ea6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.628+0000 7f4942318640 1 --2- 172.21.15.16:0/3328144528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4924004800 0x7f493c1ea6d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f492c011990 tx=0x7f492c011e50 comp rx=0 tx=0).stop 2024-09-19T07:53:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.628+0000 7f4942318640 1 -- 172.21.15.16:0/3328144528 shutdown_connections 2024-09-19T07:53:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.628+0000 7f4942318640 1 --2- 172.21.15.16:0/3328144528 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f492806c2b0 0x7f492806e770 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.628+0000 7f4942318640 1 --2- 172.21.15.16:0/3328144528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c0cb5b0 0x7f493c0cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.628+0000 7f4942318640 1 --2- 172.21.15.16:0/3328144528 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f493c0cdf60 0x7f493c1eac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.628+0000 7f4942318640 1 --2- 172.21.15.16:0/3328144528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4924004800 0x7f493c1ea6d0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.628+0000 7f4942318640 1 -- 172.21.15.16:0/3328144528 >> 172.21.15.16:0/3328144528 conn(0x7f493c08bfe0 msgr2=0x7f493c06cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.628+0000 7f4942318640 1 -- 172.21.15.16:0/3328144528 shutdown_connections 2024-09-19T07:53:30.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.628+0000 7f4942318640 1 -- 172.21.15.16:0/3328144528 wait complete. 2024-09-19T07:53:30.642 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T07:53:30.642 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:53:30.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.744+0000 7f76fe531640 1 Processor -- start 2024-09-19T07:53:30.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.744+0000 7f76fe531640 1 -- start start 2024-09-19T07:53:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.745+0000 7f76fe531640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76f815c7f0 0x7f76f80af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.745+0000 7f76fe531640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f76f80ad0b0 0x7f76f80ad500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.745+0000 7f76fe531640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76f815ce70 con 0x7f76f8156890 2024-09-19T07:53:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.745+0000 7f76fe531640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f76f80b37e0 con 0x7f76f815c7f0 2024-09-19T07:53:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.745+0000 7f76fe531640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f76f80afa30 con 0x7f76f80ad0b0 2024-09-19T07:53:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.745+0000 7f76f77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76f815c7f0 0x7f76f80af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.745+0000 7f76f77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76f815c7f0 0x7f76f80af4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40616/0 (socket says 172.21.15.16:40616) 2024-09-19T07:53:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.745+0000 7f76f77fe640 1 -- 172.21.15.16:0/526895136 learned_addr learned my addr 172.21.15.16:0/526895136 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.745+0000 7f76fcace640 1 --2- 172.21.15.16:0/526895136 >> v2:172.21.15.146:3300/0 conn(0x7f76f80ad0b0 0x7f76f80ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.745+0000 7f76f77fe640 1 -- 172.21.15.16:0/526895136 >> v2:172.21.15.146:3300/0 conn(0x7f76f80ad0b0 msgr2=0x7f76f80ad500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.745+0000 7f76f77fe640 1 --2- 172.21.15.16:0/526895136 >> v2:172.21.15.146:3300/0 conn(0x7f76f80ad0b0 0x7f76f80ad500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.745+0000 7f76f77fe640 1 -- 172.21.15.16:0/526895136 >> v1:172.21.15.146:6789/0 conn(0x7f76f8156890 legacy=0x7f76f80b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.745+0000 7f76f77fe640 1 -- 172.21.15.16:0/526895136 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f815d150 con 0x7f76f815c7f0 2024-09-19T07:53:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.745+0000 7f76fcace640 1 --2- 172.21.15.16:0/526895136 >> v2:172.21.15.146:3300/0 conn(0x7f76f80ad0b0 0x7f76f80ad500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.746+0000 7f76f77fe640 1 --2- 172.21.15.16:0/526895136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f815c7f0 0x7f76f80af4f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f76f80d0cf0 tx=0x7f76ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9460df1c77c29e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:30.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.746+0000 7f76f6ffd640 1 -- 172.21.15.16:0/526895136 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec037070 con 0x7f76f815c7f0 2024-09-19T07:53:30.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.746+0000 7f76f6ffd640 1 -- 172.21.15.16:0/526895136 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76ec032440 con 0x7f76f815c7f0 2024-09-19T07:53:30.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.746+0000 7f76f6ffd640 1 -- 172.21.15.16:0/526895136 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec032810 con 0x7f76f815c7f0 2024-09-19T07:53:30.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.747+0000 7f76fe531640 1 -- 172.21.15.16:0/526895136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f815c7f0 msgr2=0x7f76f80af4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:30.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.747+0000 7f76fe531640 1 --2- 172.21.15.16:0/526895136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f815c7f0 0x7f76f80af4f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f76f80d0cf0 tx=0x7f76ec02a270 comp rx=0 tx=0).stop 2024-09-19T07:53:30.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.747+0000 7f76fe531640 1 -- 172.21.15.16:0/526895136 shutdown_connections 2024-09-19T07:53:30.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.747+0000 7f76fe531640 1 --2- 172.21.15.16:0/526895136 >> v2:172.21.15.146:3300/0 conn(0x7f76f80ad0b0 0x7f76f80ad500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.747+0000 7f76fe531640 1 --2- 172.21.15.16:0/526895136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f815c7f0 0x7f76f80af4f0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.747+0000 7f76fe531640 1 -- 172.21.15.16:0/526895136 >> 172.21.15.16:0/526895136 conn(0x7f76f806bdf0 msgr2=0x7f76f806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:30.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.747+0000 7f76fe531640 1 -- 172.21.15.16:0/526895136 shutdown_connections 2024-09-19T07:53:30.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.747+0000 7f76fe531640 1 -- 172.21.15.16:0/526895136 wait complete. 2024-09-19T07:53:30.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.748+0000 7f76fe531640 1 Processor -- start 2024-09-19T07:53:30.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.749+0000 7f76fe531640 1 -- start start 2024-09-19T07:53:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.749+0000 7f76fe531640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76f80ad0b0 0x7f76f809ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.749+0000 7f76fe531640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f8156890 0x7f76f809b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.749+0000 7f76fe531640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76f815c7f0 0x7f76f809bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.749+0000 7f76fe531640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76f80ada40 con 0x7f76f8156890 2024-09-19T07:53:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.749+0000 7f76fe531640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f76f815cc50 con 0x7f76f815c7f0 2024-09-19T07:53:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.749+0000 7f76fe531640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f76f806ded0 con 0x7f76f80ad0b0 2024-09-19T07:53:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.749+0000 7f76f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f8156890 0x7f76f809b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.749+0000 7f76f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f8156890 0x7f76f809b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40624/0 (socket says 172.21.15.16:40624) 2024-09-19T07:53:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.749+0000 7f76f77fe640 1 -- 172.21.15.16:0/905811491 learned_addr learned my addr 172.21.15.16:0/905811491 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.750+0000 7f76f7fff640 1 --2- 172.21.15.16:0/905811491 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76f80ad0b0 0x7f76f809ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.750+0000 7f76fcace640 1 --2- 172.21.15.16:0/905811491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76f815c7f0 0x7f76f809bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.750+0000 7f76f77fe640 1 -- 172.21.15.16:0/905811491 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76f80ad0b0 msgr2=0x7f76f809ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.750+0000 7f76f77fe640 1 --2- 172.21.15.16:0/905811491 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76f80ad0b0 0x7f76f809ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.750+0000 7f76f77fe640 1 -- 172.21.15.16:0/905811491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76f815c7f0 msgr2=0x7f76f809bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.750+0000 7f76f77fe640 1 --2- 172.21.15.16:0/905811491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76f815c7f0 0x7f76f809bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.750+0000 7f76f77fe640 1 -- 172.21.15.16:0/905811491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f80cf6e0 con 0x7f76f8156890 2024-09-19T07:53:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.750+0000 7f76fcace640 1 --2- 172.21.15.16:0/905811491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76f815c7f0 0x7f76f809bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.750+0000 7f76f77fe640 1 --2- 172.21.15.16:0/905811491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f8156890 0x7f76f809b5c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f76ec02a2b0 tx=0x7f76ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.750+0000 7f76f4ff9640 1 -- 172.21.15.16:0/905811491 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec041020 con 0x7f76f8156890 2024-09-19T07:53:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.750+0000 7f76f4ff9640 1 -- 172.21.15.16:0/905811491 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76ec0306c0 con 0x7f76f8156890 2024-09-19T07:53:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.750+0000 7f76fe531640 1 -- 172.21.15.16:0/905811491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f809a260 con 0x7f76f8156890 2024-09-19T07:53:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.750+0000 7f76f4ff9640 1 -- 172.21.15.16:0/905811491 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec037040 con 0x7f76f8156890 2024-09-19T07:53:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.750+0000 7f76fe531640 1 -- 172.21.15.16:0/905811491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f809a820 con 0x7f76f8156890 2024-09-19T07:53:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.751+0000 7f76fe531640 1 -- 172.21.15.16:0/905811491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f8156c70 con 0x7f76f8156890 2024-09-19T07:53:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.752+0000 7f76f4ff9640 1 -- 172.21.15.16:0/905811491 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f76ec044020 con 0x7f76f8156890 2024-09-19T07:53:30.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.752+0000 7f76f4ff9640 1 --2- 172.21.15.16:0/905811491 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f76e406c2b0 0x7f76e406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:30.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.752+0000 7f76f4ff9640 1 -- 172.21.15.16:0/905811491 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f76ec0abc80 con 0x7f76f8156890 2024-09-19T07:53:30.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.752+0000 7f76f7fff640 1 --2- 172.21.15.16:0/905811491 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f76e406c2b0 0x7f76e406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:30.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.753+0000 7f76f7fff640 1 --2- 172.21.15.16:0/905811491 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f76e406c2b0 0x7f76e406e770 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f76e8001780 tx=0x7f76e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:30.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.756+0000 7f76f4ff9640 1 -- 172.21.15.16:0/905811491 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f76ec04b030 con 0x7f76f8156890 2024-09-19T07:53:30.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.967+0000 7f76fe531640 1 -- 172.21.15.16:0/905811491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76f806fbe0 con 0x7f76f8156890 2024-09-19T07:53:30.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.968+0000 7f76f4ff9640 1 -- 172.21.15.16:0/905811491 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5228 (secure 0 0 0) 0x7f76ec07a2f0 con 0x7f76f8156890 2024-09-19T07:53:30.968 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:30.969 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":57,"btime":"2024-09-19T07:53:22:988089+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6017,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2955839490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":2955839490},{"type":"v1","addr":"172.21.15.146:6837","nonce":2955839490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6550,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/395357736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":395357736},{"type":"v1","addr":"172.21.15.146:6836","nonce":395357736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3539638739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3539638739},{"type":"v1","addr":"172.21.15.16:6834","nonce":3539638739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:53:02.264799+0000","modified":"2024-09-19T07:53:22.976173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/4185904511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4185904511},{"type":"v1","addr":"172.21.15.16:6835","nonce":4185904511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":7}]} 2024-09-19T07:53:30.969 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 57 2024-09-19T07:53:30.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.970+0000 7f76fe531640 1 -- 172.21.15.16:0/905811491 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f76e406c2b0 msgr2=0x7f76e406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:30.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.970+0000 7f76fe531640 1 --2- 172.21.15.16:0/905811491 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f76e406c2b0 0x7f76e406e770 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f76e8001780 tx=0x7f76e8009040 comp rx=0 tx=0).stop 2024-09-19T07:53:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.970+0000 7f76fe531640 1 -- 172.21.15.16:0/905811491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f8156890 msgr2=0x7f76f809b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.970+0000 7f76fe531640 1 --2- 172.21.15.16:0/905811491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f8156890 0x7f76f809b5c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f76ec02a2b0 tx=0x7f76ec0046c0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.970+0000 7f76fe531640 1 -- 172.21.15.16:0/905811491 shutdown_connections 2024-09-19T07:53:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.970+0000 7f76fe531640 1 --2- 172.21.15.16:0/905811491 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f76e406c2b0 0x7f76e406e770 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.970+0000 7f76fe531640 1 --2- 172.21.15.16:0/905811491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76f815c7f0 0x7f76f809bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.970+0000 7f76fe531640 1 --2- 172.21.15.16:0/905811491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f8156890 0x7f76f809b5c0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.970+0000 7f76fe531640 1 --2- 172.21.15.16:0/905811491 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76f80ad0b0 0x7f76f809ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.970+0000 7f76fe531640 1 -- 172.21.15.16:0/905811491 >> 172.21.15.16:0/905811491 conn(0x7f76f806bdf0 msgr2=0x7f76f80afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.970+0000 7f76fe531640 1 -- 172.21.15.16:0/905811491 shutdown_connections 2024-09-19T07:53:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:30.970+0000 7f76fe531640 1 -- 172.21.15.16:0/905811491 wait complete. 2024-09-19T07:53:30.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:53:31.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.084+0000 7f6ce431f640 1 Processor -- start 2024-09-19T07:53:31.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.084+0000 7f6ce431f640 1 -- start start 2024-09-19T07:53:31.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.085+0000 7f6ce431f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6cdc1589c0 0x7f6cdc0733b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:31.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.085+0000 7f6ce431f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6cdc15f610 0x7f6cdc06bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:31.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.085+0000 7f6ce431f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cdc0b44d0 con 0x7f6cdc0b3b30 2024-09-19T07:53:31.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.085+0000 7f6ce431f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f6cdc15eb30 con 0x7f6cdc1589c0 2024-09-19T07:53:31.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.085+0000 7f6ce431f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f6cdc15ee10 con 0x7f6cdc15f610 2024-09-19T07:53:31.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.085+0000 7f6ce20bb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6cdc1589c0 0x7f6cdc0733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:31.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.085+0000 7f6ce20bb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6cdc1589c0 0x7f6cdc0733b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40630/0 (socket says 172.21.15.16:40630) 2024-09-19T07:53:31.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.085+0000 7f6ce20bb640 1 -- 172.21.15.16:0/3408285712 learned_addr learned my addr 172.21.15.16:0/3408285712 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:31.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.085+0000 7f6ce18ba640 1 --2- 172.21.15.16:0/3408285712 >> v2:172.21.15.146:3300/0 conn(0x7f6cdc15f610 0x7f6cdc06bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:31.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.085+0000 7f6ce20bb640 1 -- 172.21.15.16:0/3408285712 >> v2:172.21.15.146:3300/0 conn(0x7f6cdc15f610 msgr2=0x7f6cdc06bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:31.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.085+0000 7f6ce20bb640 1 --2- 172.21.15.16:0/3408285712 >> v2:172.21.15.146:3300/0 conn(0x7f6cdc15f610 0x7f6cdc06bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.085+0000 7f6ce20bb640 1 -- 172.21.15.16:0/3408285712 >> v1:172.21.15.146:6789/0 conn(0x7f6cdc0b3b30 legacy=0x7f6cdc0b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.085+0000 7f6ce20bb640 1 -- 172.21.15.16:0/3408285712 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cdc15f0f0 con 0x7f6cdc1589c0 2024-09-19T07:53:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.086+0000 7f6ce20bb640 1 --2- 172.21.15.16:0/3408285712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cdc1589c0 0x7f6cdc0733b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009870 tx=0x7f6cd802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750a0e0efa1b7da2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.086+0000 7f6ce10b9640 1 -- 172.21.15.16:0/3408285712 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8037070 con 0x7f6cdc1589c0 2024-09-19T07:53:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.086+0000 7f6ce10b9640 1 -- 172.21.15.16:0/3408285712 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cd8032440 con 0x7f6cdc1589c0 2024-09-19T07:53:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.086+0000 7f6ce10b9640 1 -- 172.21.15.16:0/3408285712 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8032810 con 0x7f6cdc1589c0 2024-09-19T07:53:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.087+0000 7f6ce431f640 1 -- 172.21.15.16:0/3408285712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cdc1589c0 msgr2=0x7f6cdc0733b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.087+0000 7f6ce431f640 1 --2- 172.21.15.16:0/3408285712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cdc1589c0 0x7f6cdc0733b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009870 tx=0x7f6cd802a270 comp rx=0 tx=0).stop 2024-09-19T07:53:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.087+0000 7f6ce431f640 1 -- 172.21.15.16:0/3408285712 shutdown_connections 2024-09-19T07:53:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.087+0000 7f6ce431f640 1 --2- 172.21.15.16:0/3408285712 >> v2:172.21.15.146:3300/0 conn(0x7f6cdc15f610 0x7f6cdc06bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.087+0000 7f6ce431f640 1 --2- 172.21.15.16:0/3408285712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cdc1589c0 0x7f6cdc0733b0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.087+0000 7f6ce431f640 1 -- 172.21.15.16:0/3408285712 >> 172.21.15.16:0/3408285712 conn(0x7f6cdc0b6a30 msgr2=0x7f6cdc0b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.087+0000 7f6ce431f640 1 -- 172.21.15.16:0/3408285712 shutdown_connections 2024-09-19T07:53:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.087+0000 7f6ce431f640 1 -- 172.21.15.16:0/3408285712 wait complete. 2024-09-19T07:53:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.088+0000 7f6ce431f640 1 Processor -- start 2024-09-19T07:53:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.088+0000 7f6ce431f640 1 -- start start 2024-09-19T07:53:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.089+0000 7f6ce431f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cdc0b3b30 0x7f6cdc161e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.089+0000 7f6ce431f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cdc1589c0 0x7f6cdc13fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.089+0000 7f6ce431f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6cdc15f610 0x7f6cdc140130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.089+0000 7f6ce431f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc0b4930 con 0x7f6cdc1589c0 2024-09-19T07:53:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.089+0000 7f6ce431f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc0b47b0 con 0x7f6cdc0b3b30 2024-09-19T07:53:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.089+0000 7f6ce431f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6cdc06c150 con 0x7f6cdc15f610 2024-09-19T07:53:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.089+0000 7f6ce18ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cdc1589c0 0x7f6cdc13fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.089+0000 7f6ce18ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cdc1589c0 0x7f6cdc13fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40646/0 (socket says 172.21.15.16:40646) 2024-09-19T07:53:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.089+0000 7f6ce18ba640 1 -- 172.21.15.16:0/2607922171 learned_addr learned my addr 172.21.15.16:0/2607922171 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.089+0000 7f6ce18ba640 1 -- 172.21.15.16:0/2607922171 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6cdc15f610 msgr2=0x7f6cdc140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.089+0000 7f6ce18ba640 1 --2- 172.21.15.16:0/2607922171 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6cdc15f610 0x7f6cdc140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.089+0000 7f6ce18ba640 1 -- 172.21.15.16:0/2607922171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cdc0b3b30 msgr2=0x7f6cdc161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.089+0000 7f6ce18ba640 1 --2- 172.21.15.16:0/2607922171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cdc0b3b30 0x7f6cdc161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.089+0000 7f6ce18ba640 1 -- 172.21.15.16:0/2607922171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cdc154df0 con 0x7f6cdc1589c0 2024-09-19T07:53:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.089+0000 7f6ce18ba640 1 --2- 172.21.15.16:0/2607922171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cdc1589c0 0x7f6cdc13fbf0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00d620 tx=0x7f6ccc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.089+0000 7f6cd2ffd640 1 -- 172.21.15.16:0/2607922171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc00b620 con 0x7f6cdc1589c0 2024-09-19T07:53:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.089+0000 7f6cd2ffd640 1 -- 172.21.15.16:0/2607922171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ccc019070 con 0x7f6cdc1589c0 2024-09-19T07:53:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.089+0000 7f6ce431f640 1 -- 172.21.15.16:0/2607922171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cdc1550d0 con 0x7f6cdc1589c0 2024-09-19T07:53:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.089+0000 7f6cd2ffd640 1 -- 172.21.15.16:0/2607922171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc00ba00 con 0x7f6cdc1589c0 2024-09-19T07:53:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.090+0000 7f6ce431f640 1 -- 172.21.15.16:0/2607922171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cdc161930 con 0x7f6cdc1589c0 2024-09-19T07:53:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.090+0000 7f6ce431f640 1 -- 172.21.15.16:0/2607922171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cdc06c510 con 0x7f6cdc1589c0 2024-09-19T07:53:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.091+0000 7f6cd2ffd640 1 -- 172.21.15.16:0/2607922171 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6ccc00bba0 con 0x7f6cdc1589c0 2024-09-19T07:53:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.091+0000 7f6cd2ffd640 1 --2- 172.21.15.16:0/2607922171 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6cc006c2b0 0x7f6cc006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.091+0000 7f6cd2ffd640 1 -- 172.21.15.16:0/2607922171 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f6ccc08b9d0 con 0x7f6cdc1589c0 2024-09-19T07:53:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.092+0000 7f6ce20bb640 1 --2- 172.21.15.16:0/2607922171 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6cc006c2b0 0x7f6cc006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.092+0000 7f6ce20bb640 1 --2- 172.21.15.16:0/2607922171 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6cc006c2b0 0x7f6cc006e770 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8034000 tx=0x7f6cd802c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:31.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.096+0000 7f6cd2ffd640 1 -- 172.21.15.16:0/2607922171 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6ccc059fb0 con 0x7f6cdc1589c0 2024-09-19T07:53:31.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.310+0000 7f6ce431f640 1 -- 172.21.15.16:0/2607922171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cdc1549d0 con 0x7f6cdc1589c0 2024-09-19T07:53:31.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.310+0000 7f6cd2ffd640 1 -- 172.21.15.16:0/2607922171 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5228 (secure 0 0 0) 0x7f6ccc0598b0 con 0x7f6cdc1589c0 2024-09-19T07:53:31.311 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:31.311 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":57,"btime":"2024-09-19T07:53:22:988089+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6017,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2955839490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":2955839490},{"type":"v1","addr":"172.21.15.146:6837","nonce":2955839490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6550,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/395357736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":395357736},{"type":"v1","addr":"172.21.15.146:6836","nonce":395357736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3539638739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3539638739},{"type":"v1","addr":"172.21.15.16:6834","nonce":3539638739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:53:02.264799+0000","modified":"2024-09-19T07:53:22.976173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/4185904511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4185904511},{"type":"v1","addr":"172.21.15.16:6835","nonce":4185904511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":7}]} 2024-09-19T07:53:31.311 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 57 2024-09-19T07:53:31.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.313+0000 7f6ce431f640 1 -- 172.21.15.16:0/2607922171 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6cc006c2b0 msgr2=0x7f6cc006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:31.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.313+0000 7f6ce431f640 1 --2- 172.21.15.16:0/2607922171 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6cc006c2b0 0x7f6cc006e770 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8034000 tx=0x7f6cd802c000 comp rx=0 tx=0).stop 2024-09-19T07:53:31.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.313+0000 7f6ce431f640 1 -- 172.21.15.16:0/2607922171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cdc1589c0 msgr2=0x7f6cdc13fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:31.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.313+0000 7f6ce431f640 1 --2- 172.21.15.16:0/2607922171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cdc1589c0 0x7f6cdc13fbf0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00d620 tx=0x7f6ccc00dae0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.313+0000 7f6ce431f640 1 -- 172.21.15.16:0/2607922171 shutdown_connections 2024-09-19T07:53:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.313+0000 7f6ce431f640 1 --2- 172.21.15.16:0/2607922171 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6cc006c2b0 0x7f6cc006e770 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.313+0000 7f6ce431f640 1 --2- 172.21.15.16:0/2607922171 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6cdc15f610 0x7f6cdc140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.313+0000 7f6ce431f640 1 --2- 172.21.15.16:0/2607922171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cdc1589c0 0x7f6cdc13fbf0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.313+0000 7f6ce431f640 1 --2- 172.21.15.16:0/2607922171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cdc0b3b30 0x7f6cdc161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.313+0000 7f6ce431f640 1 -- 172.21.15.16:0/2607922171 >> 172.21.15.16:0/2607922171 conn(0x7f6cdc0b6a30 msgr2=0x7f6cdc15e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.313+0000 7f6ce431f640 1 -- 172.21.15.16:0/2607922171 shutdown_connections 2024-09-19T07:53:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.313+0000 7f6ce431f640 1 -- 172.21.15.16:0/2607922171 wait complete. 2024-09-19T07:53:31.324 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:53:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.425+0000 7f54aba01640 1 Processor -- start 2024-09-19T07:53:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.425+0000 7f54aba01640 1 -- start start 2024-09-19T07:53:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.426+0000 7f54aba01640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54a40d5040 0x7f54a4162960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.426+0000 7f54aba01640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54a40cf8b0 con 0x7f54a4162ea0 2024-09-19T07:53:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.426+0000 7f54aba01640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54a40cfb90 con 0x7f54a415e640 2024-09-19T07:53:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.426+0000 7f54aba01640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f54a40b12e0 con 0x7f54a40d5040 2024-09-19T07:53:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.426+0000 7f54a8f9c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54a40d5040 0x7f54a4162960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.426+0000 7f54a9f9e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f54a4162ea0 0x7f54a40b1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49540/0 (socket says 172.21.15.16:49540) 2024-09-19T07:53:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.426+0000 7f54a8f9c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54a40d5040 0x7f54a4162960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40658/0 (socket says 172.21.15.16:40658) 2024-09-19T07:53:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.426+0000 7f54a9f9e640 1 -- 172.21.15.16:0/3179362282 learned_addr learned my addr 172.21.15.16:0/3179362282 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.427+0000 7f54a8f9c640 1 -- 172.21.15.16:0/3179362282 >> v1:172.21.15.146:6789/0 conn(0x7f54a415e640 legacy=0x7f54a4162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.427+0000 7f54a8f9c640 1 -- 172.21.15.16:0/3179362282 >> v1:172.21.15.16:6789/0 conn(0x7f54a4162ea0 legacy=0x7f54a40b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.427+0000 7f54a8f9c640 1 -- 172.21.15.16:0/3179362282 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a40cfe70 con 0x7f54a40d5040 2024-09-19T07:53:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.427+0000 7f54a3fff640 1 -- 172.21.15.16:0/3179362282 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2087964949 0 0) 0x7f54a40cf8b0 con 0x7f54a4162ea0 2024-09-19T07:53:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.427+0000 7f54a8f9c640 1 --2- 172.21.15.16:0/3179362282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a40d5040 0x7f54a4162960 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f5498009790 tx=0x7f549802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b2d98e4d07d80f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.428+0000 7f54a3fff640 1 -- 172.21.15.16:0/3179362282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498037070 con 0x7f54a40d5040 2024-09-19T07:53:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.428+0000 7f54a3fff640 1 -- 172.21.15.16:0/3179362282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5498032440 con 0x7f54a40d5040 2024-09-19T07:53:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.428+0000 7f54a3fff640 1 -- 172.21.15.16:0/3179362282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498032810 con 0x7f54a40d5040 2024-09-19T07:53:31.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.428+0000 7f54aba01640 1 -- 172.21.15.16:0/3179362282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a40d5040 msgr2=0x7f54a4162960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.428+0000 7f54aba01640 1 --2- 172.21.15.16:0/3179362282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a40d5040 0x7f54a4162960 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f5498009790 tx=0x7f549802a1c0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.428+0000 7f54aba01640 1 -- 172.21.15.16:0/3179362282 shutdown_connections 2024-09-19T07:53:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.428+0000 7f54aba01640 1 --2- 172.21.15.16:0/3179362282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a40d5040 0x7f54a4162960 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.428+0000 7f54aba01640 1 -- 172.21.15.16:0/3179362282 >> 172.21.15.16:0/3179362282 conn(0x7f54a408bfe0 msgr2=0x7f54a40d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.428+0000 7f54aba01640 1 -- 172.21.15.16:0/3179362282 shutdown_connections 2024-09-19T07:53:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.428+0000 7f54aba01640 1 -- 172.21.15.16:0/3179362282 wait complete. 2024-09-19T07:53:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.429+0000 7f54aba01640 1 Processor -- start 2024-09-19T07:53:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.430+0000 7f54aba01640 1 -- start start 2024-09-19T07:53:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.430+0000 7f54aba01640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a40d5040 0x7f54a4160c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.430+0000 7f54aba01640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54a415e640 0x7f54a41494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.430+0000 7f54aba01640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54a4162ea0 0x7f54a41499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.430+0000 7f54aba01640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54a415eba0 con 0x7f54a40d5040 2024-09-19T07:53:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.430+0000 7f54aba01640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f54a415ea20 con 0x7f54a415e640 2024-09-19T07:53:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.430+0000 7f54aba01640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f54a40b1e80 con 0x7f54a4162ea0 2024-09-19T07:53:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.430+0000 7f54a979d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a40d5040 0x7f54a4160c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.430+0000 7f54a979d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a40d5040 0x7f54a4160c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40672/0 (socket says 172.21.15.16:40672) 2024-09-19T07:53:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.430+0000 7f54a979d640 1 -- 172.21.15.16:0/3257704101 learned_addr learned my addr 172.21.15.16:0/3257704101 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.431+0000 7f54a979d640 1 -- 172.21.15.16:0/3257704101 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54a4162ea0 msgr2=0x7f54a41499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.431+0000 7f54a979d640 1 --2- 172.21.15.16:0/3257704101 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54a4162ea0 0x7f54a41499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.431+0000 7f54a979d640 1 -- 172.21.15.16:0/3257704101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54a415e640 msgr2=0x7f54a41494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.431+0000 7f54a979d640 1 --2- 172.21.15.16:0/3257704101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54a415e640 0x7f54a41494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.431+0000 7f54a979d640 1 -- 172.21.15.16:0/3257704101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a41e8f80 con 0x7f54a40d5040 2024-09-19T07:53:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.431+0000 7f54a979d640 1 --2- 172.21.15.16:0/3257704101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a40d5040 0x7f54a4160c00 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f549400c7f0 tx=0x7f549400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.431+0000 7f54a1ffb640 1 -- 172.21.15.16:0/3257704101 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494016020 con 0x7f54a40d5040 2024-09-19T07:53:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.431+0000 7f54a1ffb640 1 -- 172.21.15.16:0/3257704101 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f549400aa50 con 0x7f54a40d5040 2024-09-19T07:53:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.431+0000 7f54aba01640 1 -- 172.21.15.16:0/3257704101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a40c7f50 con 0x7f54a40d5040 2024-09-19T07:53:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.431+0000 7f54a1ffb640 1 -- 172.21.15.16:0/3257704101 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549400ae40 con 0x7f54a40d5040 2024-09-19T07:53:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.431+0000 7f54aba01640 1 -- 172.21.15.16:0/3257704101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a40c8510 con 0x7f54a40d5040 2024-09-19T07:53:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.432+0000 7f54aba01640 1 -- 172.21.15.16:0/3257704101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a40b1750 con 0x7f54a40d5040 2024-09-19T07:53:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.433+0000 7f54a1ffb640 1 -- 172.21.15.16:0/3257704101 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5494019020 con 0x7f54a40d5040 2024-09-19T07:53:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.433+0000 7f54a1ffb640 1 --2- 172.21.15.16:0/3257704101 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f549006c2b0 0x7f549006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.433+0000 7f54a1ffb640 1 -- 172.21.15.16:0/3257704101 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f549408b270 con 0x7f54a40d5040 2024-09-19T07:53:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.433+0000 7f54a8f9c640 1 --2- 172.21.15.16:0/3257704101 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f549006c2b0 0x7f549006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.434+0000 7f54a8f9c640 1 --2- 172.21.15.16:0/3257704101 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f549006c2b0 0x7f549006e770 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5498034000 tx=0x7f549802c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:31.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.437+0000 7f54a1ffb640 1 -- 172.21.15.16:0/3257704101 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5494020070 con 0x7f54a40d5040 2024-09-19T07:53:31.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.614+0000 7f54aba01640 1 -- 172.21.15.16:0/3257704101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54a40d3670 con 0x7f54a40d5040 2024-09-19T07:53:31.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.615+0000 7f54a1ffb640 1 -- 172.21.15.16:0/3257704101 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16968 (secure 0 0 0) 0x7f54940598e0 con 0x7f54a40d5040 2024-09-19T07:53:31.615 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:31.615 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":106,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:53:03.939259+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:52:59.931281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-19T07:53:01.258168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:53:31.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.617+0000 7f54aba01640 1 -- 172.21.15.16:0/3257704101 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f549006c2b0 msgr2=0x7f549006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:31.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.617+0000 7f54aba01640 1 --2- 172.21.15.16:0/3257704101 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f549006c2b0 0x7f549006e770 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5498034000 tx=0x7f549802c000 comp rx=0 tx=0).stop 2024-09-19T07:53:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.617+0000 7f54aba01640 1 -- 172.21.15.16:0/3257704101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a40d5040 msgr2=0x7f54a4160c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.617+0000 7f54aba01640 1 --2- 172.21.15.16:0/3257704101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a40d5040 0x7f54a4160c00 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f549400c7f0 tx=0x7f549400ccb0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.617+0000 7f54aba01640 1 -- 172.21.15.16:0/3257704101 shutdown_connections 2024-09-19T07:53:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.617+0000 7f54aba01640 1 --2- 172.21.15.16:0/3257704101 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f549006c2b0 0x7f549006e770 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.617+0000 7f54aba01640 1 --2- 172.21.15.16:0/3257704101 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54a4162ea0 0x7f54a41499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.617+0000 7f54aba01640 1 --2- 172.21.15.16:0/3257704101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54a415e640 0x7f54a41494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.617+0000 7f54aba01640 1 --2- 172.21.15.16:0/3257704101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54a40d5040 0x7f54a4160c00 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.617+0000 7f54aba01640 1 -- 172.21.15.16:0/3257704101 >> 172.21.15.16:0/3257704101 conn(0x7f54a408bfe0 msgr2=0x7f54a40d2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.617+0000 7f54aba01640 1 -- 172.21.15.16:0/3257704101 shutdown_connections 2024-09-19T07:53:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.617+0000 7f54aba01640 1 -- 172.21.15.16:0/3257704101 wait complete. 2024-09-19T07:53:31.628 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T07:53:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.730+0000 7f855485e640 1 Processor -- start 2024-09-19T07:53:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.730+0000 7f855485e640 1 -- start start 2024-09-19T07:53:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.731+0000 7f855485e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f85500d1630 0x7f85500d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.731+0000 7f855485e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85500b4480 0x7f85500b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.731+0000 7f855485e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85500b4b80 con 0x7f855015e6d0 2024-09-19T07:53:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.731+0000 7f855485e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f85500b4e60 con 0x7f85500b4480 2024-09-19T07:53:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.731+0000 7f855485e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f85500b5140 con 0x7f85500d1630 2024-09-19T07:53:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.731+0000 7f854ed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85500b4480 0x7f85500b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.732+0000 7f854ed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85500b4480 0x7f85500b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40678/0 (socket says 172.21.15.16:40678) 2024-09-19T07:53:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.732+0000 7f854ed9d640 1 -- 172.21.15.16:0/3564916544 learned_addr learned my addr 172.21.15.16:0/3564916544 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:31.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.732+0000 7f854dd9b640 1 --2- 172.21.15.16:0/3564916544 >> v2:172.21.15.146:3300/0 conn(0x7f85500d1630 0x7f85500d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:31.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.732+0000 7f854ed9d640 1 -- 172.21.15.16:0/3564916544 >> v2:172.21.15.146:3300/0 conn(0x7f85500d1630 msgr2=0x7f85500d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:31.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.732+0000 7f854ed9d640 1 --2- 172.21.15.16:0/3564916544 >> v2:172.21.15.146:3300/0 conn(0x7f85500d1630 0x7f85500d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.732+0000 7f854ed9d640 1 -- 172.21.15.16:0/3564916544 >> v1:172.21.15.146:6789/0 conn(0x7f855015e6d0 legacy=0x7f85500d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:31.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.732+0000 7f854ed9d640 1 -- 172.21.15.16:0/3564916544 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85500b5420 con 0x7f85500b4480 2024-09-19T07:53:31.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.732+0000 7f854ed9d640 1 --2- 172.21.15.16:0/3564916544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85500b4480 0x7f85500b5c20 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f85500c7c40 tx=0x7f854002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef115021f7065730 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:31.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.733+0000 7f854d59a640 1 -- 172.21.15.16:0/3564916544 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540036070 con 0x7f85500b4480 2024-09-19T07:53:31.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.733+0000 7f854d59a640 1 -- 172.21.15.16:0/3564916544 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8540031600 con 0x7f85500b4480 2024-09-19T07:53:31.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.733+0000 7f854d59a640 1 -- 172.21.15.16:0/3564916544 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85400319d0 con 0x7f85500b4480 2024-09-19T07:53:31.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.733+0000 7f855485e640 1 -- 172.21.15.16:0/3564916544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85500b4480 msgr2=0x7f85500b5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:31.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.733+0000 7f855485e640 1 --2- 172.21.15.16:0/3564916544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85500b4480 0x7f85500b5c20 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f85500c7c40 tx=0x7f854002a6f0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.733+0000 7f855485e640 1 -- 172.21.15.16:0/3564916544 shutdown_connections 2024-09-19T07:53:31.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.733+0000 7f855485e640 1 --2- 172.21.15.16:0/3564916544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85500b4480 0x7f85500b5c20 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.733+0000 7f855485e640 1 --2- 172.21.15.16:0/3564916544 >> v2:172.21.15.146:3300/0 conn(0x7f85500d1630 0x7f85500d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.733+0000 7f855485e640 1 -- 172.21.15.16:0/3564916544 >> 172.21.15.16:0/3564916544 conn(0x7f855009d860 msgr2=0x7f855008bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:31.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.733+0000 7f855485e640 1 -- 172.21.15.16:0/3564916544 shutdown_connections 2024-09-19T07:53:31.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.733+0000 7f855485e640 1 -- 172.21.15.16:0/3564916544 wait complete. 2024-09-19T07:53:31.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.735+0000 7f855485e640 1 Processor -- start 2024-09-19T07:53:31.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.735+0000 7f855485e640 1 -- start start 2024-09-19T07:53:31.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.735+0000 7f855485e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85500b4480 0x7f8550159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:31.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.735+0000 7f855485e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85500d1630 0x7f85500cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:31.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.735+0000 7f855485e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855015e6d0 0x7f855015ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:31.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.735+0000 7f855485e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85500b6160 con 0x7f855015e6d0 2024-09-19T07:53:31.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.735+0000 7f855485e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f855015eab0 con 0x7f85500d1630 2024-09-19T07:53:31.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.735+0000 7f855485e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f85500b62e0 con 0x7f85500b4480 2024-09-19T07:53:31.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.736+0000 7f854ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855015e6d0 0x7f855015ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:31.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.736+0000 7f854ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855015e6d0 0x7f855015ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40692/0 (socket says 172.21.15.16:40692) 2024-09-19T07:53:31.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.736+0000 7f854ed9d640 1 -- 172.21.15.16:0/534830450 learned_addr learned my addr 172.21.15.16:0/534830450 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:31.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.736+0000 7f854e59c640 1 --2- 172.21.15.16:0/534830450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85500b4480 0x7f8550159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:31.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.736+0000 7f854dd9b640 1 --2- 172.21.15.16:0/534830450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85500d1630 0x7f85500cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:31.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.736+0000 7f854ed9d640 1 -- 172.21.15.16:0/534830450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85500b4480 msgr2=0x7f8550159080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:31.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.736+0000 7f854ed9d640 1 --2- 172.21.15.16:0/534830450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85500b4480 0x7f8550159080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.736+0000 7f854ed9d640 1 -- 172.21.15.16:0/534830450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85500d1630 msgr2=0x7f85500cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:31.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.736+0000 7f854ed9d640 1 --2- 172.21.15.16:0/534830450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85500d1630 0x7f85500cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.736+0000 7f854ed9d640 1 -- 172.21.15.16:0/534830450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855015f3a0 con 0x7f855015e6d0 2024-09-19T07:53:31.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.736+0000 7f854dd9b640 1 --2- 172.21.15.16:0/534830450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85500d1630 0x7f85500cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:31.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.736+0000 7f854ed9d640 1 --2- 172.21.15.16:0/534830450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855015e6d0 0x7f855015ec30 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f8540009f40 tx=0x7f8540004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:31.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.736+0000 7f8546ffd640 1 -- 172.21.15.16:0/534830450 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540036050 con 0x7f855015e6d0 2024-09-19T07:53:31.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.736+0000 7f8546ffd640 1 -- 172.21.15.16:0/534830450 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8540042070 con 0x7f855015e6d0 2024-09-19T07:53:31.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.736+0000 7f855485e640 1 -- 172.21.15.16:0/534830450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f855015f680 con 0x7f855015e6d0 2024-09-19T07:53:31.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.736+0000 7f8546ffd640 1 -- 172.21.15.16:0/534830450 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540009510 con 0x7f855015e6d0 2024-09-19T07:53:31.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.736+0000 7f855485e640 1 -- 172.21.15.16:0/534830450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f855015fc40 con 0x7f855015e6d0 2024-09-19T07:53:31.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.737+0000 7f855485e640 1 -- 172.21.15.16:0/534830450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85500b5c20 con 0x7f855015e6d0 2024-09-19T07:53:31.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.738+0000 7f8546ffd640 1 -- 172.21.15.16:0/534830450 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8540007bd0 con 0x7f855015e6d0 2024-09-19T07:53:31.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.738+0000 7f8546ffd640 1 --2- 172.21.15.16:0/534830450 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f853006c2e0 0x7f853006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:31.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.738+0000 7f8546ffd640 1 -- 172.21.15.16:0/534830450 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f85400ac580 con 0x7f855015e6d0 2024-09-19T07:53:31.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.739+0000 7f854e59c640 1 --2- 172.21.15.16:0/534830450 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f853006c2e0 0x7f853006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:31.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.739+0000 7f854e59c640 1 --2- 172.21.15.16:0/534830450 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f853006c2e0 0x7f853006e7a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f85500d5500 tx=0x7f8548008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:31.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.743+0000 7f8546ffd640 1 -- 172.21.15.16:0/534830450 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8540045360 con 0x7f855015e6d0 2024-09-19T07:53:31.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.952+0000 7f855485e640 1 -- 172.21.15.16:0/534830450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f85500b30f0 con 0x7f855015e6d0 2024-09-19T07:53:31.952 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-19T07:53:31.952+0000 7f6950fea640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:53:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.953+0000 7f8546ffd640 1 -- 172.21.15.16:0/534830450 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) ==== 161+0+0 (secure 0 0 0) 0x7f854007ab50 con 0x7f855015e6d0 2024-09-19T07:53:31.954 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T07:53:31.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.956+0000 7f855485e640 1 -- 172.21.15.16:0/534830450 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f853006c2e0 msgr2=0x7f853006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:31.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.956+0000 7f855485e640 1 --2- 172.21.15.16:0/534830450 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f853006c2e0 0x7f853006e7a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f85500d5500 tx=0x7f8548008040 comp rx=0 tx=0).stop 2024-09-19T07:53:31.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.956+0000 7f855485e640 1 -- 172.21.15.16:0/534830450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855015e6d0 msgr2=0x7f855015ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:31.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.956+0000 7f855485e640 1 --2- 172.21.15.16:0/534830450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855015e6d0 0x7f855015ec30 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f8540009f40 tx=0x7f8540004a40 comp rx=0 tx=0).stop 2024-09-19T07:53:31.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.956+0000 7f855485e640 1 -- 172.21.15.16:0/534830450 shutdown_connections 2024-09-19T07:53:31.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.956+0000 7f855485e640 1 --2- 172.21.15.16:0/534830450 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f853006c2e0 0x7f853006e7a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.957+0000 7f855485e640 1 --2- 172.21.15.16:0/534830450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f855015e6d0 0x7f855015ec30 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.957+0000 7f855485e640 1 --2- 172.21.15.16:0/534830450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85500d1630 0x7f85500cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.957+0000 7f855485e640 1 --2- 172.21.15.16:0/534830450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85500b4480 0x7f8550159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:31.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.957+0000 7f855485e640 1 -- 172.21.15.16:0/534830450 >> 172.21.15.16:0/534830450 conn(0x7f855009d860 msgr2=0x7f855008bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:31.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.957+0000 7f855485e640 1 -- 172.21.15.16:0/534830450 shutdown_connections 2024-09-19T07:53:31.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:31.957+0000 7f855485e640 1 -- 172.21.15.16:0/534830450 wait complete. 2024-09-19T07:53:31.970 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T07:53:31.998 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-19T07:53:32.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.076+0000 7f293dc23640 1 Processor -- start 2024-09-19T07:53:32.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.076+0000 7f293dc23640 1 -- start start 2024-09-19T07:53:32.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.077+0000 7f293dc23640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2938161e00 0x7f29380cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:32.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.077+0000 7f293dc23640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f29380ce670 0x7f29380cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:32.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.077+0000 7f293dc23640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29380d0230 con 0x7f29380cf000 2024-09-19T07:53:32.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.077+0000 7f293dc23640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f29380d32d0 con 0x7f29380ce670 2024-09-19T07:53:32.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.077+0000 7f293dc23640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f29380b5e20 con 0x7f2938161e00 2024-09-19T07:53:32.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.077+0000 7f29377fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2938161e00 0x7f29380cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:32.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.077+0000 7f2936ffd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f29380ce670 0x7f29380cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:32.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.077+0000 7f29377fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2938161e00 0x7f29380cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:42260/0 (socket says 172.21.15.16:42260) 2024-09-19T07:53:32.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.077+0000 7f29377fe640 1 -- 172.21.15.16:0/397447869 learned_addr learned my addr 172.21.15.16:0/397447869 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:32.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.077+0000 7f29367fc640 1 -- 172.21.15.16:0/397447869 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3937837926 0 0) 0x7f29380d0230 con 0x7f29380cf000 2024-09-19T07:53:32.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.077+0000 7f29367fc640 1 -- 172.21.15.16:0/397447869 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2920003520 con 0x7f29380cf000 2024-09-19T07:53:32.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.078+0000 7f29377fe640 1 -- 172.21.15.16:0/397447869 >> v2:172.21.15.146:3300/0 conn(0x7f29380ce670 msgr2=0x7f29380cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:32.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.078+0000 7f29377fe640 1 --2- 172.21.15.16:0/397447869 >> v2:172.21.15.146:3300/0 conn(0x7f29380ce670 0x7f29380cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:32.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.078+0000 7f29377fe640 1 -- 172.21.15.16:0/397447869 >> v1:172.21.15.146:6789/0 conn(0x7f29380cf000 legacy=0x7f29380d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:32.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.078+0000 7f29377fe640 1 -- 172.21.15.16:0/397447869 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29380b60e0 con 0x7f2938161e00 2024-09-19T07:53:32.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.078+0000 7f2936ffd640 1 --2- 172.21.15.16:0/397447869 >> v2:172.21.15.146:3300/0 conn(0x7f29380ce670 0x7f29380cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:53:32.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.078+0000 7f29377fe640 1 --2- 172.21.15.16:0/397447869 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2938161e00 0x7f29380cb5b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2928009870 tx=0x7f292802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e2cae9f5647e246 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:32.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.078+0000 7f29367fc640 1 -- 172.21.15.16:0/397447869 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928037070 con 0x7f2938161e00 2024-09-19T07:53:32.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.079+0000 7f29367fc640 1 -- 172.21.15.16:0/397447869 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2928032440 con 0x7f2938161e00 2024-09-19T07:53:32.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.079+0000 7f29367fc640 1 -- 172.21.15.16:0/397447869 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928032810 con 0x7f2938161e00 2024-09-19T07:53:32.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.079+0000 7f293dc23640 1 -- 172.21.15.16:0/397447869 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2938161e00 msgr2=0x7f29380cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:32.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.079+0000 7f293dc23640 1 --2- 172.21.15.16:0/397447869 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2938161e00 0x7f29380cb5b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2928009870 tx=0x7f292802a1c0 comp rx=0 tx=0).stop 2024-09-19T07:53:32.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.079+0000 7f293dc23640 1 -- 172.21.15.16:0/397447869 shutdown_connections 2024-09-19T07:53:32.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.079+0000 7f293dc23640 1 --2- 172.21.15.16:0/397447869 >> v2:172.21.15.146:3300/0 conn(0x7f29380ce670 0x7f29380cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:32.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.079+0000 7f293dc23640 1 --2- 172.21.15.16:0/397447869 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2938161e00 0x7f29380cb5b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:32.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.079+0000 7f293dc23640 1 -- 172.21.15.16:0/397447869 >> 172.21.15.16:0/397447869 conn(0x7f293808bfe0 msgr2=0x7f29380d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:32.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.079+0000 7f293dc23640 1 -- 172.21.15.16:0/397447869 shutdown_connections 2024-09-19T07:53:32.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.079+0000 7f293dc23640 1 -- 172.21.15.16:0/397447869 wait complete. 2024-09-19T07:53:32.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.080+0000 7f293dc23640 1 Processor -- start 2024-09-19T07:53:32.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.081+0000 7f293dc23640 1 -- start start 2024-09-19T07:53:32.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.081+0000 7f293dc23640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29380ce670 0x7f2938156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:32.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.081+0000 7f293dc23640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29380cf000 0x7f29380ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:32.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.081+0000 7f293dc23640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2938161e00 0x7f29380cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:32.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.081+0000 7f293dc23640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29380b6980 con 0x7f29380ce670 2024-09-19T07:53:32.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.081+0000 7f293dc23640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f29380b66a0 con 0x7f2938161e00 2024-09-19T07:53:32.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.081+0000 7f293dc23640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f29380d3e70 con 0x7f29380cf000 2024-09-19T07:53:32.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.082+0000 7f29377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29380ce670 0x7f2938156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:32.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.082+0000 7f29377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29380ce670 0x7f2938156550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40724/0 (socket says 172.21.15.16:40724) 2024-09-19T07:53:32.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.082+0000 7f29377fe640 1 -- 172.21.15.16:0/1289056512 learned_addr learned my addr 172.21.15.16:0/1289056512 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:32.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.082+0000 7f2936ffd640 1 --2- 172.21.15.16:0/1289056512 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29380cf000 0x7f29380ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:32.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.082+0000 7f2937fff640 1 --2- 172.21.15.16:0/1289056512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2938161e00 0x7f29380cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:32.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.082+0000 7f29377fe640 1 -- 172.21.15.16:0/1289056512 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29380cf000 msgr2=0x7f29380ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:32.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.082+0000 7f29377fe640 1 --2- 172.21.15.16:0/1289056512 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29380cf000 0x7f29380ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:32.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.082+0000 7f29377fe640 1 -- 172.21.15.16:0/1289056512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2938161e00 msgr2=0x7f29380cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:32.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.082+0000 7f29377fe640 1 --2- 172.21.15.16:0/1289056512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2938161e00 0x7f29380cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:32.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.082+0000 7f29377fe640 1 -- 172.21.15.16:0/1289056512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29380cbae0 con 0x7f29380ce670 2024-09-19T07:53:32.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.082+0000 7f29377fe640 1 --2- 172.21.15.16:0/1289056512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29380ce670 0x7f2938156550 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f292802a200 tx=0x7f29280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:32.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.082+0000 7f291ffff640 1 -- 172.21.15.16:0/1289056512 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928041020 con 0x7f29380ce670 2024-09-19T07:53:32.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.082+0000 7f291ffff640 1 -- 172.21.15.16:0/1289056512 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f292803d640 con 0x7f29380ce670 2024-09-19T07:53:32.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.082+0000 7f293dc23640 1 -- 172.21.15.16:0/1289056512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29380cbdc0 con 0x7f29380ce670 2024-09-19T07:53:32.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.082+0000 7f291ffff640 1 -- 172.21.15.16:0/1289056512 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928037040 con 0x7f29380ce670 2024-09-19T07:53:32.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.082+0000 7f293dc23640 1 -- 172.21.15.16:0/1289056512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29380cc380 con 0x7f29380ce670 2024-09-19T07:53:32.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.083+0000 7f293dc23640 1 -- 172.21.15.16:0/1289056512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29380d3760 con 0x7f29380ce670 2024-09-19T07:53:32.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.084+0000 7f291ffff640 1 -- 172.21.15.16:0/1289056512 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2928044020 con 0x7f29380ce670 2024-09-19T07:53:32.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.084+0000 7f291ffff640 1 --2- 172.21.15.16:0/1289056512 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f292406c2b0 0x7f292406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:32.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.084+0000 7f291ffff640 1 -- 172.21.15.16:0/1289056512 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6187+0+0 (secure 0 0 0) 0x7f29280abb70 con 0x7f29380ce670 2024-09-19T07:53:32.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.084+0000 7f2936ffd640 1 --2- 172.21.15.16:0/1289056512 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f292406c2b0 0x7f292406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:32.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.085+0000 7f2936ffd640 1 --2- 172.21.15.16:0/1289056512 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f292406c2b0 0x7f292406e770 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f292c009560 tx=0x7f292c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:32.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.088+0000 7f291ffff640 1 -- 172.21.15.16:0/1289056512 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f292804b030 con 0x7f29380ce670 2024-09-19T07:53:32.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.301+0000 7f293dc23640 1 -- 172.21.15.16:0/1289056512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f29380d4f20 con 0x7f29380ce670 2024-09-19T07:53:32.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.955+0000 7f291ffff640 1 -- 172.21.15.16:0/1289056512 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) ==== 104+0+0 (secure 0 0 0) 0x7f2928079ab0 con 0x7f29380ce670 2024-09-19T07:53:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.958+0000 7f293dc23640 1 -- 172.21.15.16:0/1289056512 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f292406c2b0 msgr2=0x7f292406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.958+0000 7f293dc23640 1 --2- 172.21.15.16:0/1289056512 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f292406c2b0 0x7f292406e770 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f292c009560 tx=0x7f292c009190 comp rx=0 tx=0).stop 2024-09-19T07:53:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.958+0000 7f293dc23640 1 -- 172.21.15.16:0/1289056512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29380ce670 msgr2=0x7f2938156550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.958+0000 7f293dc23640 1 --2- 172.21.15.16:0/1289056512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29380ce670 0x7f2938156550 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f292802a200 tx=0x7f29280046c0 comp rx=0 tx=0).stop 2024-09-19T07:53:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.959+0000 7f293dc23640 1 -- 172.21.15.16:0/1289056512 shutdown_connections 2024-09-19T07:53:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.959+0000 7f293dc23640 1 --2- 172.21.15.16:0/1289056512 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f292406c2b0 0x7f292406e770 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.959+0000 7f293dc23640 1 --2- 172.21.15.16:0/1289056512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2938161e00 0x7f29380cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.959+0000 7f293dc23640 1 --2- 172.21.15.16:0/1289056512 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29380cf000 0x7f29380ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.959+0000 7f293dc23640 1 --2- 172.21.15.16:0/1289056512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29380ce670 0x7f2938156550 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.959+0000 7f293dc23640 1 -- 172.21.15.16:0/1289056512 >> 172.21.15.16:0/1289056512 conn(0x7f293808bfe0 msgr2=0x7f29380c8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.959+0000 7f293dc23640 1 -- 172.21.15.16:0/1289056512 shutdown_connections 2024-09-19T07:53:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:32.959+0000 7f293dc23640 1 -- 172.21.15.16:0/1289056512 wait complete. 2024-09-19T07:53:32.974 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T07:53:33.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.131+0000 7f4b51931640 1 Processor -- start 2024-09-19T07:53:33.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.131+0000 7f4b51931640 1 -- start start 2024-09-19T07:53:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.132+0000 7f4b51931640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4b4c161e90 0x7f4b4c0b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.132+0000 7f4b51931640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4b4c0d3db0 0x7f4b4c0b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.132+0000 7f4b51931640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b4c06cba0 0x7f4b4c06cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.132+0000 7f4b51931640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f4b4c162270 con 0x7f4b4c0d3db0 2024-09-19T07:53:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.132+0000 7f4b51931640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4b4c0d3a00 con 0x7f4b4c161e90 2024-09-19T07:53:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.132+0000 7f4b51931640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4b4c0b6d00 con 0x7f4b4c06cba0 2024-09-19T07:53:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.132+0000 7f4b4a7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4b4c0d3db0 0x7f4b4c0b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.132+0000 7f4b4a7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4b4c0d3db0 0x7f4b4c0b64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38874/0 (socket says 172.21.15.16:38874) 2024-09-19T07:53:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.132+0000 7f4b4a7fc640 1 -- 172.21.15.16:0/2407982709 learned_addr learned my addr 172.21.15.16:0/2407982709 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.132+0000 7f4b4b7fe640 1 --2- 172.21.15.16:0/2407982709 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b4c06cba0 0x7f4b4c06cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.132+0000 7f4b4affd640 1 --2- 172.21.15.16:0/2407982709 >> v2:172.21.15.146:3300/0 conn(0x7f4b4c161e90 0x7f4b4c0b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.132+0000 7f4b4a7fc640 1 -- 172.21.15.16:0/2407982709 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b4c06cba0 msgr2=0x7f4b4c06cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.132+0000 7f4b4a7fc640 1 --2- 172.21.15.16:0/2407982709 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b4c06cba0 0x7f4b4c06cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.132+0000 7f4b4a7fc640 1 -- 172.21.15.16:0/2407982709 >> v2:172.21.15.146:3300/0 conn(0x7f4b4c161e90 msgr2=0x7f4b4c0b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.132+0000 7f4b4a7fc640 1 --2- 172.21.15.16:0/2407982709 >> v2:172.21.15.146:3300/0 conn(0x7f4b4c161e90 0x7f4b4c0b5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.132+0000 7f4b4a7fc640 1 -- 172.21.15.16:0/2407982709 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c0cfd80 con 0x7f4b4c0d3db0 2024-09-19T07:53:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.133+0000 7f4b4a7fc640 1 --2- 172.21.15.16:0/2407982709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c0d3db0 0x7f4b4c0b64c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c08d410 tx=0x7f4b4002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeaeb05f62c3a112 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.133+0000 7f4b49ffb640 1 -- 172.21.15.16:0/2407982709 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40037070 con 0x7f4b4c0d3db0 2024-09-19T07:53:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.133+0000 7f4b49ffb640 1 -- 172.21.15.16:0/2407982709 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b40032440 con 0x7f4b4c0d3db0 2024-09-19T07:53:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.133+0000 7f4b49ffb640 1 -- 172.21.15.16:0/2407982709 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40032810 con 0x7f4b4c0d3db0 2024-09-19T07:53:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.134+0000 7f4b51931640 1 -- 172.21.15.16:0/2407982709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c0d3db0 msgr2=0x7f4b4c0b64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.134+0000 7f4b51931640 1 --2- 172.21.15.16:0/2407982709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c0d3db0 0x7f4b4c0b64c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c08d410 tx=0x7f4b4002a140 comp rx=0 tx=0).stop 2024-09-19T07:53:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.134+0000 7f4b51931640 1 -- 172.21.15.16:0/2407982709 shutdown_connections 2024-09-19T07:53:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.134+0000 7f4b51931640 1 --2- 172.21.15.16:0/2407982709 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b4c06cba0 0x7f4b4c06cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.134+0000 7f4b51931640 1 --2- 172.21.15.16:0/2407982709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c0d3db0 0x7f4b4c0b64c0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.134+0000 7f4b51931640 1 --2- 172.21.15.16:0/2407982709 >> v2:172.21.15.146:3300/0 conn(0x7f4b4c161e90 0x7f4b4c0b5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.134+0000 7f4b51931640 1 -- 172.21.15.16:0/2407982709 >> 172.21.15.16:0/2407982709 conn(0x7f4b4c0d4500 msgr2=0x7f4b4c0d4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.134+0000 7f4b51931640 1 -- 172.21.15.16:0/2407982709 shutdown_connections 2024-09-19T07:53:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.134+0000 7f4b51931640 1 -- 172.21.15.16:0/2407982709 wait complete. 2024-09-19T07:53:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.135+0000 7f4b51931640 1 Processor -- start 2024-09-19T07:53:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.135+0000 7f4b51931640 1 -- start start 2024-09-19T07:53:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.136+0000 7f4b51931640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b4c06cba0 0x7f4b4c15a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.136+0000 7f4b51931640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c0d3db0 0x7f4b4c1f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.136+0000 7f4b51931640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b4c161e90 0x7f4b4c0c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.136+0000 7f4b51931640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c0d53b0 con 0x7f4b4c0d3db0 2024-09-19T07:53:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.136+0000 7f4b51931640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c0d5230 con 0x7f4b4c06cba0 2024-09-19T07:53:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.136+0000 7f4b51931640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4b4c0d02a0 con 0x7f4b4c161e90 2024-09-19T07:53:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.136+0000 7f4b4a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c0d3db0 0x7f4b4c1f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.136+0000 7f4b4a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c0d3db0 0x7f4b4c1f76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38876/0 (socket says 172.21.15.16:38876) 2024-09-19T07:53:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.136+0000 7f4b4a7fc640 1 -- 172.21.15.16:0/301468067 learned_addr learned my addr 172.21.15.16:0/301468067 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.136+0000 7f4b4affd640 1 --2- 172.21.15.16:0/301468067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b4c06cba0 0x7f4b4c15a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.136+0000 7f4b4b7fe640 1 --2- 172.21.15.16:0/301468067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b4c161e90 0x7f4b4c0c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.137+0000 7f4b4a7fc640 1 -- 172.21.15.16:0/301468067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b4c161e90 msgr2=0x7f4b4c0c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.137+0000 7f4b4a7fc640 1 --2- 172.21.15.16:0/301468067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b4c161e90 0x7f4b4c0c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.137+0000 7f4b4a7fc640 1 -- 172.21.15.16:0/301468067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b4c06cba0 msgr2=0x7f4b4c15a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.137+0000 7f4b4a7fc640 1 --2- 172.21.15.16:0/301468067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b4c06cba0 0x7f4b4c15a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.137+0000 7f4b4a7fc640 1 -- 172.21.15.16:0/301468067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c15d960 con 0x7f4b4c0d3db0 2024-09-19T07:53:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.137+0000 7f4b4b7fe640 1 --2- 172.21.15.16:0/301468067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b4c161e90 0x7f4b4c0c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.137+0000 7f4b4a7fc640 1 --2- 172.21.15.16:0/301468067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c0d3db0 0x7f4b4c1f76a0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f4b4002a180 tx=0x7f4b400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.137+0000 7f4b33fff640 1 -- 172.21.15.16:0/301468067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40037040 con 0x7f4b4c0d3db0 2024-09-19T07:53:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.137+0000 7f4b33fff640 1 -- 172.21.15.16:0/301468067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b40042070 con 0x7f4b4c0d3db0 2024-09-19T07:53:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.137+0000 7f4b33fff640 1 -- 172.21.15.16:0/301468067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40030bb0 con 0x7f4b4c0d3db0 2024-09-19T07:53:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.137+0000 7f4b51931640 1 -- 172.21.15.16:0/301468067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4c15d3a0 con 0x7f4b4c0d3db0 2024-09-19T07:53:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.137+0000 7f4b51931640 1 -- 172.21.15.16:0/301468067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4c14b3f0 con 0x7f4b4c0d3db0 2024-09-19T07:53:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.138+0000 7f4b51931640 1 -- 172.21.15.16:0/301468067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b4c0b5f80 con 0x7f4b4c0d3db0 2024-09-19T07:53:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.139+0000 7f4b33fff640 1 -- 172.21.15.16:0/301468067 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4b40047460 con 0x7f4b4c0d3db0 2024-09-19T07:53:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.139+0000 7f4b33fff640 1 --2- 172.21.15.16:0/301468067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4b3806c2b0 0x7f4b3806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.139+0000 7f4b33fff640 1 -- 172.21.15.16:0/301468067 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6187+0+0 (secure 0 0 0) 0x7f4b400ac110 con 0x7f4b4c0d3db0 2024-09-19T07:53:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.139+0000 7f4b4affd640 1 --2- 172.21.15.16:0/301468067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4b3806c2b0 0x7f4b3806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.140+0000 7f4b4affd640 1 --2- 172.21.15.16:0/301468067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4b3806c2b0 0x7f4b3806e770 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0ba4e0 tx=0x7f4b3c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:33.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.144+0000 7f4b33fff640 1 -- 172.21.15.16:0/301468067 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4b4007a7c0 con 0x7f4b4c0d3db0 2024-09-19T07:53:33.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:33.320+0000 7f4b51931640 1 -- 172.21.15.16:0/301468067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4b4c0756d0 con 0x7f4b4c0d3db0 2024-09-19T07:53:34.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.075+0000 7f4b33fff640 1 -- 172.21.15.16:0/301468067 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) ==== 181+0+0 (secure 0 0 0) 0x7f4b4007a0c0 con 0x7f4b4c0d3db0 2024-09-19T07:53:34.077 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-19T07:53:34.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.079+0000 7f4b51931640 1 -- 172.21.15.16:0/301468067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4b3806c2b0 msgr2=0x7f4b3806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.079+0000 7f4b51931640 1 --2- 172.21.15.16:0/301468067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4b3806c2b0 0x7f4b3806e770 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0ba4e0 tx=0x7f4b3c008040 comp rx=0 tx=0).stop 2024-09-19T07:53:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.079+0000 7f4b51931640 1 -- 172.21.15.16:0/301468067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c0d3db0 msgr2=0x7f4b4c1f76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.079+0000 7f4b51931640 1 --2- 172.21.15.16:0/301468067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c0d3db0 0x7f4b4c1f76a0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f4b4002a180 tx=0x7f4b400046c0 comp rx=0 tx=0).stop 2024-09-19T07:53:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.079+0000 7f4b51931640 1 -- 172.21.15.16:0/301468067 shutdown_connections 2024-09-19T07:53:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.079+0000 7f4b51931640 1 --2- 172.21.15.16:0/301468067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4b3806c2b0 0x7f4b3806e770 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.079+0000 7f4b51931640 1 --2- 172.21.15.16:0/301468067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b4c161e90 0x7f4b4c0c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.079+0000 7f4b51931640 1 --2- 172.21.15.16:0/301468067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b4c0d3db0 0x7f4b4c1f76a0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.079+0000 7f4b51931640 1 --2- 172.21.15.16:0/301468067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b4c06cba0 0x7f4b4c15a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.079+0000 7f4b51931640 1 -- 172.21.15.16:0/301468067 >> 172.21.15.16:0/301468067 conn(0x7f4b4c0d4500 msgr2=0x7f4b4c15e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.080+0000 7f4b51931640 1 -- 172.21.15.16:0/301468067 shutdown_connections 2024-09-19T07:53:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.080+0000 7f4b51931640 1 -- 172.21.15.16:0/301468067 wait complete. 2024-09-19T07:53:34.095 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T07:53:34.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.208+0000 7f7faaa80640 1 Processor -- start 2024-09-19T07:53:34.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.208+0000 7f7faaa80640 1 -- start start 2024-09-19T07:53:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.209+0000 7f7faaa80640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7fa40b4cf0 0x7f7fa406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.209+0000 7f7faaa80640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fa40ad6e0 0x7f7fa40b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.209+0000 7f7faaa80640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fa406f5e0 con 0x7f7fa4145520 2024-09-19T07:53:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.209+0000 7f7faaa80640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f7fa40b50d0 con 0x7f7fa40b4cf0 2024-09-19T07:53:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.209+0000 7f7faaa80640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa406e540 con 0x7f7fa40ad6e0 2024-09-19T07:53:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.209+0000 7f7fa881c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7fa40b4cf0 0x7f7fa406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.209+0000 7f7fa881c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7fa40b4cf0 0x7f7fa406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:44174/0 (socket says 172.21.15.16:44174) 2024-09-19T07:53:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.209+0000 7f7fa881c640 1 -- 172.21.15.16:0/2950769416 learned_addr learned my addr 172.21.15.16:0/2950769416 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.209+0000 7f7f9bfff640 1 --1- 172.21.15.16:0/2950769416 >> v1:172.21.15.146:6789/0 conn(0x7f7fa4145520 0x7f7fa40acfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:36956/0 (socket says 172.21.15.16:36956) 2024-09-19T07:53:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.209+0000 7f7fa901d640 1 --2- 172.21.15.16:0/2950769416 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fa40ad6e0 0x7f7fa40b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.210+0000 7f7fa881c640 1 -- 172.21.15.16:0/2950769416 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fa40ad6e0 msgr2=0x7f7fa40b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.210+0000 7f7fa881c640 1 --2- 172.21.15.16:0/2950769416 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fa40ad6e0 0x7f7fa40b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.210+0000 7f7fa881c640 1 -- 172.21.15.16:0/2950769416 >> v1:172.21.15.146:6789/0 conn(0x7f7fa4145520 legacy=0x7f7fa40acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.210+0000 7f7fa881c640 1 -- 172.21.15.16:0/2950769416 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa406e820 con 0x7f7fa40b4cf0 2024-09-19T07:53:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.210+0000 7f7f9b7fe640 1 -- 172.21.15.16:0/2950769416 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2758616644 0 0) 0x7f7fa406f5e0 con 0x7f7fa4145520 2024-09-19T07:53:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.210+0000 7f7fa881c640 1 --2- 172.21.15.16:0/2950769416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fa40b4cf0 0x7f7fa406dcf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009e50 tx=0x7f7fa002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bdc424fc39b7ff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.211+0000 7f7f9b7fe640 1 -- 172.21.15.16:0/2950769416 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0036070 con 0x7f7fa40b4cf0 2024-09-19T07:53:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.211+0000 7f7f9b7fe640 1 -- 172.21.15.16:0/2950769416 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fa0031600 con 0x7f7fa40b4cf0 2024-09-19T07:53:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.211+0000 7f7f9b7fe640 1 -- 172.21.15.16:0/2950769416 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa00319d0 con 0x7f7fa40b4cf0 2024-09-19T07:53:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.211+0000 7f7faaa80640 1 -- 172.21.15.16:0/2950769416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fa40b4cf0 msgr2=0x7f7fa406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.211+0000 7f7faaa80640 1 --2- 172.21.15.16:0/2950769416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fa40b4cf0 0x7f7fa406dcf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009e50 tx=0x7f7fa002a710 comp rx=0 tx=0).stop 2024-09-19T07:53:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.211+0000 7f7faaa80640 1 -- 172.21.15.16:0/2950769416 shutdown_connections 2024-09-19T07:53:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.211+0000 7f7faaa80640 1 --2- 172.21.15.16:0/2950769416 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fa40ad6e0 0x7f7fa40b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.211+0000 7f7faaa80640 1 --2- 172.21.15.16:0/2950769416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fa40b4cf0 0x7f7fa406dcf0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.211+0000 7f7faaa80640 1 -- 172.21.15.16:0/2950769416 >> 172.21.15.16:0/2950769416 conn(0x7f7fa40b1cf0 msgr2=0x7f7fa40b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.211+0000 7f7faaa80640 1 -- 172.21.15.16:0/2950769416 shutdown_connections 2024-09-19T07:53:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.211+0000 7f7faaa80640 1 -- 172.21.15.16:0/2950769416 wait complete. 2024-09-19T07:53:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.212+0000 7f7faaa80640 1 Processor -- start 2024-09-19T07:53:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.213+0000 7f7faaa80640 1 -- start start 2024-09-19T07:53:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.213+0000 7f7faaa80640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fa40ad6e0 0x7f7fa4099a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.213+0000 7f7faaa80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa40b4cf0 0x7f7fa4157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.213+0000 7f7faaa80640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fa4145520 0x7f7fa41e7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.213+0000 7f7faaa80640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa406f0c0 con 0x7f7fa40b4cf0 2024-09-19T07:53:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.213+0000 7f7faaa80640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa406ede0 con 0x7f7fa4145520 2024-09-19T07:53:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.213+0000 7f7faaa80640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa406e230 con 0x7f7fa40ad6e0 2024-09-19T07:53:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.214+0000 7f7f9bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa40b4cf0 0x7f7fa4157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.214+0000 7f7f9bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa40b4cf0 0x7f7fa4157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38880/0 (socket says 172.21.15.16:38880) 2024-09-19T07:53:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.214+0000 7f7f9bfff640 1 -- 172.21.15.16:0/2218595576 learned_addr learned my addr 172.21.15.16:0/2218595576 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.214+0000 7f7fa881c640 1 --2- 172.21.15.16:0/2218595576 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fa40ad6e0 0x7f7fa4099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.214+0000 7f7fa901d640 1 --2- 172.21.15.16:0/2218595576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fa4145520 0x7f7fa41e7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.214+0000 7f7f9bfff640 1 -- 172.21.15.16:0/2218595576 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fa40ad6e0 msgr2=0x7f7fa4099a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.214+0000 7f7f9bfff640 1 --2- 172.21.15.16:0/2218595576 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fa40ad6e0 0x7f7fa4099a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.214+0000 7f7f9bfff640 1 -- 172.21.15.16:0/2218595576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fa4145520 msgr2=0x7f7fa41e7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.214+0000 7f7f9bfff640 1 --2- 172.21.15.16:0/2218595576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fa4145520 0x7f7fa41e7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.214+0000 7f7f9bfff640 1 -- 172.21.15.16:0/2218595576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa4156960 con 0x7f7fa40b4cf0 2024-09-19T07:53:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.214+0000 7f7f9bfff640 1 --2- 172.21.15.16:0/2218595576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa40b4cf0 0x7f7fa4157a00 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400c430 tx=0x7f7f9400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.214+0000 7f7f997fa640 1 -- 172.21.15.16:0/2218595576 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f940074e0 con 0x7f7fa40b4cf0 2024-09-19T07:53:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.214+0000 7f7f997fa640 1 -- 172.21.15.16:0/2218595576 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f940047e0 con 0x7f7fa40b4cf0 2024-09-19T07:53:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.214+0000 7f7f997fa640 1 -- 172.21.15.16:0/2218595576 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f94004bd0 con 0x7f7fa40b4cf0 2024-09-19T07:53:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.214+0000 7f7faaa80640 1 -- 172.21.15.16:0/2218595576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa4156c40 con 0x7f7fa40b4cf0 2024-09-19T07:53:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.214+0000 7f7faaa80640 1 -- 172.21.15.16:0/2218595576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa4157200 con 0x7f7fa40b4cf0 2024-09-19T07:53:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.215+0000 7f7faaa80640 1 -- 172.21.15.16:0/2218595576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa406dcf0 con 0x7f7fa40b4cf0 2024-09-19T07:53:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.216+0000 7f7f997fa640 1 -- 172.21.15.16:0/2218595576 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7f94011070 con 0x7f7fa40b4cf0 2024-09-19T07:53:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.216+0000 7f7f997fa640 1 --2- 172.21.15.16:0/2218595576 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7f9006c2e0 0x7f7f9006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.216+0000 7f7f997fa640 1 -- 172.21.15.16:0/2218595576 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5749+0+0 (secure 0 0 0) 0x7f7f9408b020 con 0x7f7fa40b4cf0 2024-09-19T07:53:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.217+0000 7f7fa881c640 1 --2- 172.21.15.16:0/2218595576 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7f9006c2e0 0x7f7f9006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.217+0000 7f7fa881c640 1 --2- 172.21.15.16:0/2218595576 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7f9006c2e0 0x7f7f9006e7a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0033000 tx=0x7f7fa0009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.221+0000 7f7f997fa640 1 -- 172.21.15.16:0/2218595576 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7f94059070 con 0x7f7fa40b4cf0 2024-09-19T07:53:34.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:34.400+0000 7f7faaa80640 1 -- 172.21.15.16:0/2218595576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7fa40b2b60 con 0x7f7fa40b4cf0 2024-09-19T07:53:35.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.083+0000 7f7f997fa640 1 -- 172.21.15.16:0/2218595576 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) ==== 169+0+0 (secure 0 0 0) 0x7f7f9405d370 con 0x7f7fa40b4cf0 2024-09-19T07:53:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-19T07:53:35.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.087+0000 7f7faaa80640 1 -- 172.21.15.16:0/2218595576 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7f9006c2e0 msgr2=0x7f7f9006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:35.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.087+0000 7f7faaa80640 1 --2- 172.21.15.16:0/2218595576 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7f9006c2e0 0x7f7f9006e7a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0033000 tx=0x7f7fa0009000 comp rx=0 tx=0).stop 2024-09-19T07:53:35.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.087+0000 7f7faaa80640 1 -- 172.21.15.16:0/2218595576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa40b4cf0 msgr2=0x7f7fa4157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:35.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.087+0000 7f7faaa80640 1 --2- 172.21.15.16:0/2218595576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa40b4cf0 0x7f7fa4157a00 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400c430 tx=0x7f7f9400c8f0 comp rx=0 tx=0).stop 2024-09-19T07:53:35.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.087+0000 7f7faaa80640 1 -- 172.21.15.16:0/2218595576 shutdown_connections 2024-09-19T07:53:35.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.087+0000 7f7faaa80640 1 --2- 172.21.15.16:0/2218595576 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7f9006c2e0 0x7f7f9006e7a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:35.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.087+0000 7f7faaa80640 1 --2- 172.21.15.16:0/2218595576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fa4145520 0x7f7fa41e7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:35.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.087+0000 7f7faaa80640 1 --2- 172.21.15.16:0/2218595576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa40b4cf0 0x7f7fa4157a00 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:35.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.087+0000 7f7faaa80640 1 --2- 172.21.15.16:0/2218595576 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fa40ad6e0 0x7f7fa4099a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:35.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.087+0000 7f7faaa80640 1 -- 172.21.15.16:0/2218595576 >> 172.21.15.16:0/2218595576 conn(0x7f7fa40b1cf0 msgr2=0x7f7fa40b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:35.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.087+0000 7f7faaa80640 1 -- 172.21.15.16:0/2218595576 shutdown_connections 2024-09-19T07:53:35.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.087+0000 7f7faaa80640 1 -- 172.21.15.16:0/2218595576 wait complete. 2024-09-19T07:53:35.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T07:53:35.111 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T07:53:35.157 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-09-19T07:53:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2024-09-19T07:53:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T07:53:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2024-09-19T07:53:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2024-09-19T07:53:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2024-09-19T07:53:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2024-09-19T07:53:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2024-09-19T07:53:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2024-09-19T07:53:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2024-09-19T07:53:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T07:53:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2024-09-19T07:53:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T07:53:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T07:53:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2024-09-19T07:53:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2024-09-19T07:53:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2024-09-19T07:53:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2024-09-19T07:53:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2024-09-19T07:53:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2024-09-19T07:53:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2024-09-19T07:53:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2024-09-19T07:53:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T07:53:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T07:53:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2024-09-19T07:53:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T07:53:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2024-09-19T07:53:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2024-09-19T07:53:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2024-09-19T07:53:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2024-09-19T07:53:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2024-09-19T07:53:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-09-19T07:53:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb 2024-09-19T07:53:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.161 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2024-09-19T07:53:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2024-09-19T07:53:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2024-09-19T07:53:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2024-09-19T07:53:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2024-09-19T07:53:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2024-09-19T07:53:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2024-09-19T07:53:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2024-09-19T07:53:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2024-09-19T07:53:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2024-09-19T07:53:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2024-09-19T07:53:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2024-09-19T07:53:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2024-09-19T07:53:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2024-09-19T07:53:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2024-09-19T07:53:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2024-09-19T07:53:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2024-09-19T07:53:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2024-09-19T07:53:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2024-09-19T07:53:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2024-09-19T07:53:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2024-09-19T07:53:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2024-09-19T07:53:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2024-09-19T07:53:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2024-09-19T07:53:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:53:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2024-09-19T07:53:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2024-09-19T07:53:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T07:53:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2024-09-19T07:53:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2024-09-19T07:53:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2024-09-19T07:53:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2024-09-19T07:53:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-09-19T07:53:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.164 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2024-09-19T07:53:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:53:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2024-09-19T07:53:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-19T07:53:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-19T07:53:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:53:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:53:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2024-09-19T07:53:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-09-19T07:53:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.165 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2024-09-19T07:53:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:53:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-19T07:53:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2024-09-19T07:53:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-19T07:53:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2024-09-19T07:53:35.166 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2024-09-19T07:53:35.166 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:53:35.166 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2024-09-19T07:53:35.166 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2024-09-19T07:53:35.166 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:53:35.166 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.166 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T07:53:35.166 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T07:53:35.166 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T07:53:35.166 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.166 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T07:53:35.166 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T07:53:35.167 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T07:53:35.167 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.167 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2024-09-19T07:53:35.167 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2024-09-19T07:53:35.167 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.167 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2024-09-19T07:53:35.167 INFO:teuthology.orchestra.run.smithi016.stdout: mon down mkfs grace = 300 2024-09-19T07:53:35.167 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2024-09-19T07:53:35.167 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on pool no app = False 2024-09-19T07:53:35.167 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:35.167 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2024-09-19T07:53:35.167 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2024-09-19T07:53:35.167 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2024-09-19T07:53:35.167 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2024-09-19T07:53:35.167 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T07:53:35.168 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T07:53:35.168 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2024-09-19T07:53:35.168 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-09-19T07:53:35.168 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:53:35.168 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2024-09-19T07:53:35.168 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-09-19T07:53:35.168 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-09-19T07:53:35.168 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2024-09-19T07:53:35.168 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2024-09-19T07:53:35.168 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2024-09-19T07:53:35.168 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-09-19T07:53:35.168 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-09-19T07:53:35.169 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-09-19T07:53:35.169 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-09-19T07:53:35.169 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2024-09-19T07:53:35.169 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2024-09-19T07:53:35.169 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2024-09-19T07:53:35.169 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:53:35.169 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2024-09-19T07:53:35.169 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2024-09-19T07:53:35.169 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2024-09-19T07:53:35.169 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = True 2024-09-19T07:53:35.169 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-09-19T07:53:35.169 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-09-19T07:53:35.169 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-09-19T07:53:35.170 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2024-09-19T07:53:35.170 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2024-09-19T07:53:35.170 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T07:53:35.170 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2024-09-19T07:53:35.170 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.171 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2024-09-19T07:53:35.171 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.171 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2024-09-19T07:53:35.171 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.171 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2024-09-19T07:53:35.171 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2024-09-19T07:53:35.171 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.171 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2024-09-19T07:53:35.171 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2024-09-19T07:53:35.171 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.171 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T07:53:35.171 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2024-09-19T07:53:35.171 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.171 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T07:53:35.172 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T07:53:35.172 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.172 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2024-09-19T07:53:35.172 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.172 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2024-09-19T07:53:35.172 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2024-09-19T07:53:35.172 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2024-09-19T07:53:35.172 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2024-09-19T07:53:35.172 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2024-09-19T07:53:35.172 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2024-09-19T07:53:35.172 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2024-09-19T07:53:35.172 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T07:53:35.172 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T07:53:35.172 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2024-09-19T07:53:35.173 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.173 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T07:53:35.173 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.173 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2024-09-19T07:53:35.173 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.173 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2024-09-19T07:53:35.173 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.173 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2024-09-19T07:53:35.173 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2024-09-19T07:53:35.173 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2024-09-19T07:53:35.173 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.16,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-09-19T07:53:35.173 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = 6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb 2024-09-19T07:53:35.174 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.174 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2024-09-19T07:53:35.174 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2024-09-19T07:53:35.174 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.174 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2024-09-19T07:53:35.174 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2024-09-19T07:53:35.174 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2024-09-19T07:53:35.174 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.174 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2024-09-19T07:53:35.174 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2024-09-19T07:53:35.174 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.175 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2024-09-19T07:53:35.175 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2024-09-19T07:53:35.175 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2024-09-19T07:53:35.175 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug trim objects = true 2024-09-19T07:53:35.175 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.175 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2024-09-19T07:53:35.175 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2024-09-19T07:53:35.175 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.175 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2024-09-19T07:53:35.175 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.175 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2024-09-19T07:53:35.175 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.176 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2024-09-19T07:53:35.176 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.176 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2024-09-19T07:53:35.176 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2024-09-19T07:53:35.176 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.176 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2024-09-19T07:53:35.176 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2024-09-19T07:53:35.176 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2024-09-19T07:53:35.176 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2024-09-19T07:53:35.176 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 20 2024-09-19T07:53:35.176 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 20 2024-09-19T07:53:35.176 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:53:35.177 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2024-09-19T07:53:35.177 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 10 2024-09-19T07:53:35.177 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T07:53:35.177 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2024-09-19T07:53:35.177 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2024-09-19T07:53:35.177 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2024-09-19T07:53:35.177 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2024-09-19T07:53:35.177 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-09-19T07:53:35.177 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.177 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2024-09-19T07:53:35.177 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:53:35.177 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2024-09-19T07:53:35.177 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-19T07:53:35.178 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-19T07:53:35.178 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:53:35.178 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:53:35.178 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2024-09-19T07:53:35.178 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-09-19T07:53:35.178 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.178 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2024-09-19T07:53:35.178 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:53:35.178 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-19T07:53:35.178 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2024-09-19T07:53:35.179 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-19T07:53:35.179 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2024-09-19T07:53:35.179 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2024-09-19T07:53:35.179 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:53:35.179 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2024-09-19T07:53:35.179 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2024-09-19T07:53:35.179 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:53:35.179 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.179 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T07:53:35.180 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T07:53:35.180 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T07:53:35.180 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.180 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T07:53:35.180 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T07:53:35.180 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T07:53:35.180 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.180 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2024-09-19T07:53:35.180 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2024-09-19T07:53:35.180 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.181 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2024-09-19T07:53:35.181 INFO:teuthology.orchestra.run.smithi146.stdout: mon down mkfs grace = 300 2024-09-19T07:53:35.181 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2024-09-19T07:53:35.181 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on pool no app = False 2024-09-19T07:53:35.181 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:53:35.181 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2024-09-19T07:53:35.181 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2024-09-19T07:53:35.181 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2024-09-19T07:53:35.181 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2024-09-19T07:53:35.181 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T07:53:35.181 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T07:53:35.181 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2024-09-19T07:53:35.182 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-09-19T07:53:35.182 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:53:35.182 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = "" 2024-09-19T07:53:35.182 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-09-19T07:53:35.182 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-09-19T07:53:35.182 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2024-09-19T07:53:35.182 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2024-09-19T07:53:35.182 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2024-09-19T07:53:35.182 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2024-09-19T07:53:35.182 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2024-09-19T07:53:35.182 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2024-09-19T07:53:35.182 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2024-09-19T07:53:35.182 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2024-09-19T07:53:35.182 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2024-09-19T07:53:35.183 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2024-09-19T07:53:35.183 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:53:35.183 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2024-09-19T07:53:35.183 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2024-09-19T07:53:35.183 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2024-09-19T07:53:35.183 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2024-09-19T07:53:35.183 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-09-19T07:53:35.183 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-09-19T07:53:35.183 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-09-19T07:53:35.185 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T07:53:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.290+0000 7f5d2c21e640 1 Processor -- start 2024-09-19T07:53:35.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.290+0000 7f5d2c21e640 1 -- start start 2024-09-19T07:53:35.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.291+0000 7f5d2c21e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d2415e2a0 0x7f5d2406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.291+0000 7f5d2c21e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d240d1880 con 0x7f5d2415e680 2024-09-19T07:53:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.291+0000 7f5d2c21e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d240d1b60 con 0x7f5d2406d960 2024-09-19T07:53:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.291+0000 7f5d2c21e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f5d240cf2d0 con 0x7f5d2415e2a0 2024-09-19T07:53:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.291+0000 7f5d29fba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d2415e2a0 0x7f5d2406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.291+0000 7f5d297b9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5d2415e680 0x7f5d240cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40066/0 (socket says 172.21.15.16:40066) 2024-09-19T07:53:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.291+0000 7f5d29fba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d2415e2a0 0x7f5d2406d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38890/0 (socket says 172.21.15.16:38890) 2024-09-19T07:53:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.291+0000 7f5d297b9640 1 -- 172.21.15.16:0/2030542124 learned_addr learned my addr 172.21.15.16:0/2030542124 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.292+0000 7f5d29fba640 1 -- 172.21.15.16:0/2030542124 >> v1:172.21.15.146:6789/0 conn(0x7f5d2406d960 legacy=0x7f5d240d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.292+0000 7f5d29fba640 1 -- 172.21.15.16:0/2030542124 >> v1:172.21.15.16:6789/0 conn(0x7f5d2415e680 legacy=0x7f5d240cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.292+0000 7f5d29fba640 1 -- 172.21.15.16:0/2030542124 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d240cf5b0 con 0x7f5d2415e2a0 2024-09-19T07:53:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.292+0000 7f5d29fba640 1 --2- 172.21.15.16:0/2030542124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2415e2a0 0x7f5d2406d420 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0097a0 tx=0x7f5d1c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768da01d3de122e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.292+0000 7f5d28fb8640 1 -- 172.21.15.16:0/2030542124 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c037070 con 0x7f5d2415e2a0 2024-09-19T07:53:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.292+0000 7f5d28fb8640 1 -- 172.21.15.16:0/2030542124 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d1c032440 con 0x7f5d2415e2a0 2024-09-19T07:53:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.292+0000 7f5d28fb8640 1 -- 172.21.15.16:0/2030542124 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c032810 con 0x7f5d2415e2a0 2024-09-19T07:53:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.293+0000 7f5d2c21e640 1 -- 172.21.15.16:0/2030542124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2415e2a0 msgr2=0x7f5d2406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.293+0000 7f5d2c21e640 1 --2- 172.21.15.16:0/2030542124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2415e2a0 0x7f5d2406d420 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0097a0 tx=0x7f5d1c02a1e0 comp rx=0 tx=0).stop 2024-09-19T07:53:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.293+0000 7f5d2c21e640 1 -- 172.21.15.16:0/2030542124 shutdown_connections 2024-09-19T07:53:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.293+0000 7f5d2c21e640 1 --2- 172.21.15.16:0/2030542124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2415e2a0 0x7f5d2406d420 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.293+0000 7f5d2c21e640 1 -- 172.21.15.16:0/2030542124 >> 172.21.15.16:0/2030542124 conn(0x7f5d2409d760 msgr2=0x7f5d2408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.293+0000 7f5d2c21e640 1 -- 172.21.15.16:0/2030542124 shutdown_connections 2024-09-19T07:53:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.293+0000 7f5d2c21e640 1 -- 172.21.15.16:0/2030542124 wait complete. 2024-09-19T07:53:35.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.294+0000 7f5d2c21e640 1 Processor -- start 2024-09-19T07:53:35.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.294+0000 7f5d2c21e640 1 -- start start 2024-09-19T07:53:35.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.295+0000 7f5d2c21e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d2406d960 0x7f5d2415c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:35.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.295+0000 7f5d2c21e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d2415e2a0 0x7f5d240cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:35.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.295+0000 7f5d2c21e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2415e680 0x7f5d240c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:35.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.295+0000 7f5d2c21e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d240d0360 con 0x7f5d2415e680 2024-09-19T07:53:35.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.295+0000 7f5d2c21e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5d240d01e0 con 0x7f5d2406d960 2024-09-19T07:53:35.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.295+0000 7f5d2c21e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5d240d4a80 con 0x7f5d2415e2a0 2024-09-19T07:53:35.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.295+0000 7f5d2a7bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2415e680 0x7f5d240c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:35.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.295+0000 7f5d2a7bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2415e680 0x7f5d240c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38902/0 (socket says 172.21.15.16:38902) 2024-09-19T07:53:35.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.295+0000 7f5d2a7bb640 1 -- 172.21.15.16:0/2844119771 learned_addr learned my addr 172.21.15.16:0/2844119771 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:35.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.295+0000 7f5d297b9640 1 --2- 172.21.15.16:0/2844119771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d2415e2a0 0x7f5d240cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:35.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.295+0000 7f5d29fba640 1 --2- 172.21.15.16:0/2844119771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d2406d960 0x7f5d2415c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:35.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.295+0000 7f5d2a7bb640 1 -- 172.21.15.16:0/2844119771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d2415e2a0 msgr2=0x7f5d240cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:35.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.296+0000 7f5d2a7bb640 1 --2- 172.21.15.16:0/2844119771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d2415e2a0 0x7f5d240cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:35.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.296+0000 7f5d2a7bb640 1 -- 172.21.15.16:0/2844119771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d2406d960 msgr2=0x7f5d2415c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:35.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.296+0000 7f5d2a7bb640 1 --2- 172.21.15.16:0/2844119771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d2406d960 0x7f5d2415c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:35.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.296+0000 7f5d2a7bb640 1 -- 172.21.15.16:0/2844119771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2415c9d0 con 0x7f5d2415e680 2024-09-19T07:53:35.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.296+0000 7f5d2a7bb640 1 --2- 172.21.15.16:0/2844119771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2415e680 0x7f5d240c9ed0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400c7b0 tx=0x7f5d1400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:35.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.296+0000 7f5d227fc640 1 -- 172.21.15.16:0/2844119771 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d14004b60 con 0x7f5d2415e680 2024-09-19T07:53:35.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.296+0000 7f5d227fc640 1 -- 172.21.15.16:0/2844119771 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d14004d00 con 0x7f5d2415e680 2024-09-19T07:53:35.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.296+0000 7f5d2c21e640 1 -- 172.21.15.16:0/2844119771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d240c8db0 con 0x7f5d2415e680 2024-09-19T07:53:35.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.296+0000 7f5d227fc640 1 -- 172.21.15.16:0/2844119771 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1400aa90 con 0x7f5d2415e680 2024-09-19T07:53:35.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.296+0000 7f5d2c21e640 1 -- 172.21.15.16:0/2844119771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d240c9370 con 0x7f5d2415e680 2024-09-19T07:53:35.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.297+0000 7f5d2c21e640 1 -- 172.21.15.16:0/2844119771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d2406d420 con 0x7f5d2415e680 2024-09-19T07:53:35.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.297+0000 7f5d227fc640 1 -- 172.21.15.16:0/2844119771 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5d1400ac30 con 0x7f5d2415e680 2024-09-19T07:53:35.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.298+0000 7f5d227fc640 1 --2- 172.21.15.16:0/2844119771 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5d1006c2b0 0x7f5d1006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:35.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.298+0000 7f5d227fc640 1 -- 172.21.15.16:0/2844119771 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5335+0+0 (secure 0 0 0) 0x7f5d1408b860 con 0x7f5d2415e680 2024-09-19T07:53:35.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.298+0000 7f5d29fba640 1 --2- 172.21.15.16:0/2844119771 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5d1006c2b0 0x7f5d1006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:35.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.299+0000 7f5d29fba640 1 --2- 172.21.15.16:0/2844119771 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5d1006c2b0 0x7f5d1006e770 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c034000 tx=0x7f5d1c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.302+0000 7f5d227fc640 1 -- 172.21.15.16:0/2844119771 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5d2415c9d0 con 0x7f5d2415e680 2024-09-19T07:53:35.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.478+0000 7f5d2c21e640 1 -- 172.21.15.16:0/2844119771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f5d240c7c90 con 0x7f5d2415e680 2024-09-19T07:53:35.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.481+0000 7f5d227fc640 1 -- 172.21.15.16:0/2844119771 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v42) ==== 104+0+0 (secure 0 0 0) 0x7f5d1405a1d0 con 0x7f5d2415e680 2024-09-19T07:53:35.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.483+0000 7f5d2c21e640 1 -- 172.21.15.16:0/2844119771 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5d1006c2b0 msgr2=0x7f5d1006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:35.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.483+0000 7f5d2c21e640 1 --2- 172.21.15.16:0/2844119771 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5d1006c2b0 0x7f5d1006e770 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c034000 tx=0x7f5d1c02e000 comp rx=0 tx=0).stop 2024-09-19T07:53:35.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.483+0000 7f5d2c21e640 1 -- 172.21.15.16:0/2844119771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2415e680 msgr2=0x7f5d240c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:35.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.483+0000 7f5d2c21e640 1 --2- 172.21.15.16:0/2844119771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2415e680 0x7f5d240c9ed0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400c7b0 tx=0x7f5d1400cc70 comp rx=0 tx=0).stop 2024-09-19T07:53:35.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.483+0000 7f5d2c21e640 1 -- 172.21.15.16:0/2844119771 shutdown_connections 2024-09-19T07:53:35.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.483+0000 7f5d2c21e640 1 --2- 172.21.15.16:0/2844119771 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5d1006c2b0 0x7f5d1006e770 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:35.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.483+0000 7f5d2c21e640 1 --2- 172.21.15.16:0/2844119771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2415e680 0x7f5d240c9ed0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:35.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.483+0000 7f5d2c21e640 1 --2- 172.21.15.16:0/2844119771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d2415e2a0 0x7f5d240cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:35.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.483+0000 7f5d2c21e640 1 --2- 172.21.15.16:0/2844119771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d2406d960 0x7f5d2415c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:35.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.483+0000 7f5d2c21e640 1 -- 172.21.15.16:0/2844119771 >> 172.21.15.16:0/2844119771 conn(0x7f5d2409d760 msgr2=0x7f5d240d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:35.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.483+0000 7f5d2c21e640 1 -- 172.21.15.16:0/2844119771 shutdown_connections 2024-09-19T07:53:35.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.483+0000 7f5d2c21e640 1 -- 172.21.15.16:0/2844119771 wait complete. 2024-09-19T07:53:35.494 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid' 2024-09-19T07:53:35.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.597+0000 7f020e3bd640 1 Processor -- start 2024-09-19T07:53:35.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.597+0000 7f020e3bd640 1 -- start start 2024-09-19T07:53:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.597+0000 7f020e3bd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f020815f0d0 0x7f020815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:35.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.597+0000 7f020e3bd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02080d1b10 con 0x7f02080d4130 2024-09-19T07:53:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.597+0000 7f020e3bd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02080cf2b0 con 0x7f02080d0070 2024-09-19T07:53:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.597+0000 7f020e3bd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f02080cf590 con 0x7f020815f0d0 2024-09-19T07:53:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.598+0000 7f020c95a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f02080d4130 0x7f02080d45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40074/0 (socket says 172.21.15.16:40074) 2024-09-19T07:53:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.598+0000 7f020c95a640 1 -- 172.21.15.16:0/3569192599 learned_addr learned my addr 172.21.15.16:0/3569192599 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.598+0000 7f0207fff640 1 --2- 172.21.15.16:0/3569192599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f020815f0d0 0x7f020815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.598+0000 7f0206ffd640 1 -- 172.21.15.16:0/3569192599 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3786083612 0 0) 0x7f02080d1b10 con 0x7f02080d4130 2024-09-19T07:53:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.598+0000 7f0206ffd640 1 -- 172.21.15.16:0/3569192599 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01f0003520 con 0x7f02080d4130 2024-09-19T07:53:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.598+0000 7f0206ffd640 1 -- 172.21.15.16:0/3569192599 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1369227396 0 0) 0x7f02080cf2b0 con 0x7f02080d0070 2024-09-19T07:53:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.598+0000 7f0206ffd640 1 -- 172.21.15.16:0/3569192599 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02080d1b10 con 0x7f02080d0070 2024-09-19T07:53:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.598+0000 7f0207fff640 1 -- 172.21.15.16:0/3569192599 >> v1:172.21.15.146:6789/0 conn(0x7f02080d0070 legacy=0x7f02080d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.598+0000 7f0206ffd640 1 -- 172.21.15.16:0/3569192599 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1475802416 0 0) 0x7f01f0003520 con 0x7f02080d4130 2024-09-19T07:53:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.598+0000 7f0207fff640 1 -- 172.21.15.16:0/3569192599 >> v1:172.21.15.16:6789/0 conn(0x7f02080d4130 legacy=0x7f02080d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.598+0000 7f0207fff640 1 -- 172.21.15.16:0/3569192599 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02080cf870 con 0x7f020815f0d0 2024-09-19T07:53:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.599+0000 7f0207fff640 1 --2- 172.21.15.16:0/3569192599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f020815f0d0 0x7f020815f4b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f01f80097d0 tx=0x7f01f802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=166cc1d0229dcec7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.599+0000 7f0206ffd640 1 -- 172.21.15.16:0/3569192599 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8037070 con 0x7f020815f0d0 2024-09-19T07:53:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.599+0000 7f0206ffd640 1 -- 172.21.15.16:0/3569192599 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f802ab90 con 0x7f020815f0d0 2024-09-19T07:53:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.599+0000 7f0206ffd640 1 -- 172.21.15.16:0/3569192599 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02000019f0 con 0x7f020815f0d0 2024-09-19T07:53:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.600+0000 7f020e3bd640 1 -- 172.21.15.16:0/3569192599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f020815f0d0 msgr2=0x7f020815f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.600+0000 7f020e3bd640 1 --2- 172.21.15.16:0/3569192599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f020815f0d0 0x7f020815f4b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f01f80097d0 tx=0x7f01f802a100 comp rx=0 tx=0).stop 2024-09-19T07:53:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.600+0000 7f020e3bd640 1 -- 172.21.15.16:0/3569192599 shutdown_connections 2024-09-19T07:53:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.600+0000 7f020e3bd640 1 --2- 172.21.15.16:0/3569192599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f020815f0d0 0x7f020815f4b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.600+0000 7f020e3bd640 1 -- 172.21.15.16:0/3569192599 >> 172.21.15.16:0/3569192599 conn(0x7f020809d760 msgr2=0x7f020808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.600+0000 7f020e3bd640 1 -- 172.21.15.16:0/3569192599 shutdown_connections 2024-09-19T07:53:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.600+0000 7f020e3bd640 1 -- 172.21.15.16:0/3569192599 wait complete. 2024-09-19T07:53:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.601+0000 7f020e3bd640 1 Processor -- start 2024-09-19T07:53:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.601+0000 7f020e3bd640 1 -- start start 2024-09-19T07:53:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.602+0000 7f020e3bd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02080d0070 0x7f02080cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.602+0000 7f020e3bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02080d4130 0x7f0208159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.602+0000 7f020e3bd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f020815f0d0 0x7f02080b8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.602+0000 7f020e3bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f02080b2f70 con 0x7f02080d4130 2024-09-19T07:53:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.602+0000 7f020e3bd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f02080cef60 con 0x7f02080d0070 2024-09-19T07:53:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.602+0000 7f020e3bd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f02080b30f0 con 0x7f020815f0d0 2024-09-19T07:53:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.602+0000 7f02077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02080d4130 0x7f0208159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.602+0000 7f02077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02080d4130 0x7f0208159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38916/0 (socket says 172.21.15.16:38916) 2024-09-19T07:53:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.602+0000 7f02077fe640 1 -- 172.21.15.16:0/432604928 learned_addr learned my addr 172.21.15.16:0/432604928 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.602+0000 7f020c95a640 1 --2- 172.21.15.16:0/432604928 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f020815f0d0 0x7f02080b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.602+0000 7f0207fff640 1 --2- 172.21.15.16:0/432604928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02080d0070 0x7f02080cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.602+0000 7f02077fe640 1 -- 172.21.15.16:0/432604928 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f020815f0d0 msgr2=0x7f02080b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.602+0000 7f02077fe640 1 --2- 172.21.15.16:0/432604928 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f020815f0d0 0x7f02080b8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.602+0000 7f02077fe640 1 -- 172.21.15.16:0/432604928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02080d0070 msgr2=0x7f02080cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.602+0000 7f02077fe640 1 --2- 172.21.15.16:0/432604928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02080d0070 0x7f02080cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.602+0000 7f02077fe640 1 -- 172.21.15.16:0/432604928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02080cc380 con 0x7f02080d4130 2024-09-19T07:53:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.602+0000 7f02077fe640 1 --2- 172.21.15.16:0/432604928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02080d4130 0x7f0208159610 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00c480 tx=0x7f01fc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.603+0000 7f0204ff9640 1 -- 172.21.15.16:0/432604928 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc0052d0 con 0x7f02080d4130 2024-09-19T07:53:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.603+0000 7f0204ff9640 1 -- 172.21.15.16:0/432604928 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01fc005470 con 0x7f02080d4130 2024-09-19T07:53:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.603+0000 7f020e3bd640 1 -- 172.21.15.16:0/432604928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02080cc660 con 0x7f02080d4130 2024-09-19T07:53:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.603+0000 7f0204ff9640 1 -- 172.21.15.16:0/432604928 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc012870 con 0x7f02080d4130 2024-09-19T07:53:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.603+0000 7f020e3bd640 1 -- 172.21.15.16:0/432604928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02080ccc20 con 0x7f02080d4130 2024-09-19T07:53:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.604+0000 7f020e3bd640 1 -- 172.21.15.16:0/432604928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f020815f4b0 con 0x7f02080d4130 2024-09-19T07:53:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.604+0000 7f0204ff9640 1 -- 172.21.15.16:0/432604928 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f01fc012a10 con 0x7f02080d4130 2024-09-19T07:53:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.605+0000 7f0204ff9640 1 --2- 172.21.15.16:0/432604928 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01f406c2b0 0x7f01f406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.605+0000 7f0204ff9640 1 -- 172.21.15.16:0/432604928 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5335+0+0 (secure 0 0 0) 0x7f01fc08af20 con 0x7f02080d4130 2024-09-19T07:53:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.605+0000 7f0207fff640 1 --2- 172.21.15.16:0/432604928 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01f406c2b0 0x7f01f406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.606+0000 7f0207fff640 1 --2- 172.21.15.16:0/432604928 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01f406c2b0 0x7f01f406e770 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f01f8034000 tx=0x7f01f802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.609+0000 7f0204ff9640 1 -- 172.21.15.16:0/432604928 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f01fc059950 con 0x7f02080d4130 2024-09-19T07:53:35.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:35.849+0000 7f020e3bd640 1 -- 172.21.15.16:0/432604928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]} v 0) -- 0x7f020806cf30 con 0x7f02080d4130 2024-09-19T07:53:36.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.085+0000 7f0204ff9640 1 -- 172.21.15.16:0/432604928 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]}]=0 v462) ==== 150+0+0 (secure 0 0 0) 0x7f01fc059250 con 0x7f02080d4130 2024-09-19T07:53:36.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.087+0000 7f020e3bd640 1 -- 172.21.15.16:0/432604928 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01f406c2b0 msgr2=0x7f01f406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:36.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.087+0000 7f020e3bd640 1 --2- 172.21.15.16:0/432604928 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01f406c2b0 0x7f01f406e770 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f01f8034000 tx=0x7f01f802e000 comp rx=0 tx=0).stop 2024-09-19T07:53:36.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.087+0000 7f020e3bd640 1 -- 172.21.15.16:0/432604928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02080d4130 msgr2=0x7f0208159610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:36.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.087+0000 7f020e3bd640 1 --2- 172.21.15.16:0/432604928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02080d4130 0x7f0208159610 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00c480 tx=0x7f01fc00c940 comp rx=0 tx=0).stop 2024-09-19T07:53:36.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.087+0000 7f020e3bd640 1 -- 172.21.15.16:0/432604928 shutdown_connections 2024-09-19T07:53:36.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.087+0000 7f020e3bd640 1 --2- 172.21.15.16:0/432604928 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01f406c2b0 0x7f01f406e770 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.087+0000 7f020e3bd640 1 --2- 172.21.15.16:0/432604928 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f020815f0d0 0x7f02080b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.087+0000 7f020e3bd640 1 --2- 172.21.15.16:0/432604928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02080d4130 0x7f0208159610 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.087+0000 7f020e3bd640 1 --2- 172.21.15.16:0/432604928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02080d0070 0x7f02080cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.087+0000 7f020e3bd640 1 -- 172.21.15.16:0/432604928 >> 172.21.15.16:0/432604928 conn(0x7f020809d760 msgr2=0x7f02080d1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:36.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.087+0000 7f020e3bd640 1 -- 172.21.15.16:0/432604928 shutdown_connections 2024-09-19T07:53:36.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.087+0000 7f020e3bd640 1 -- 172.21.15.16:0/432604928 wait complete. 2024-09-19T07:53:36.099 INFO:tasks.cephfs_test_runner:When a client does not advance its oldest tid, the MDS should notice that ... ok 2024-09-19T07:53:36.099 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-19T07:53:36.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:53:36.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.203+0000 7f40760dd640 1 Processor -- start 2024-09-19T07:53:36.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.203+0000 7f40760dd640 1 -- start start 2024-09-19T07:53:36.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.203+0000 7f40760dd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f40700d0070 0x7f407015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:36.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.203+0000 7f40760dd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40700d4130 0x7f40700d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:36.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.203+0000 7f40760dd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40700d4ae0 con 0x7f407015f0d0 2024-09-19T07:53:36.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.203+0000 7f40760dd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f40700d1b10 con 0x7f40700d0070 2024-09-19T07:53:36.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.203+0000 7f40760dd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f40700cf2b0 con 0x7f40700d4130 2024-09-19T07:53:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.204+0000 7f406f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f407015f0d0 0x7f40700d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40082/0 (socket says 172.21.15.16:40082) 2024-09-19T07:53:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.204+0000 7f406f7fe640 1 -- 172.21.15.16:0/393881102 learned_addr learned my addr 172.21.15.16:0/393881102 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.204+0000 7f406effd640 1 --2- 172.21.15.16:0/393881102 >> v2:172.21.15.146:3300/0 conn(0x7f40700d0070 0x7f407015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.204+0000 7f406ffff640 1 --2- 172.21.15.16:0/393881102 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40700d4130 0x7f40700d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.204+0000 7f406e7fc640 1 -- 172.21.15.16:0/393881102 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2195358460 0 0) 0x7f40700d4ae0 con 0x7f407015f0d0 2024-09-19T07:53:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.204+0000 7f406e7fc640 1 -- 172.21.15.16:0/393881102 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4058003520 con 0x7f407015f0d0 2024-09-19T07:53:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.204+0000 7f406effd640 1 -- 172.21.15.16:0/393881102 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40700d4130 msgr2=0x7f40700d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.204+0000 7f406effd640 1 --2- 172.21.15.16:0/393881102 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40700d4130 0x7f40700d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.204+0000 7f406effd640 1 -- 172.21.15.16:0/393881102 >> v1:172.21.15.16:6789/0 conn(0x7f407015f0d0 legacy=0x7f40700d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.204+0000 7f406e7fc640 1 -- 172.21.15.16:0/393881102 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2257160174 0 0) 0x7f4058003520 con 0x7f407015f0d0 2024-09-19T07:53:36.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.204+0000 7f406effd640 1 -- 172.21.15.16:0/393881102 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40700cf590 con 0x7f40700d0070 2024-09-19T07:53:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.205+0000 7f406effd640 1 --2- 172.21.15.16:0/393881102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40700d0070 0x7f407015f4b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f40640097a0 tx=0x7f406402a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4c5cd15f92250fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.205+0000 7f406e7fc640 1 -- 172.21.15.16:0/393881102 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064037070 con 0x7f40700d0070 2024-09-19T07:53:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.205+0000 7f406e7fc640 1 -- 172.21.15.16:0/393881102 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f406402ab10 con 0x7f40700d0070 2024-09-19T07:53:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.205+0000 7f406e7fc640 1 -- 172.21.15.16:0/393881102 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060001670 con 0x7f40700d0070 2024-09-19T07:53:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.206+0000 7f40760dd640 1 -- 172.21.15.16:0/393881102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40700d0070 msgr2=0x7f407015f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.206+0000 7f40760dd640 1 --2- 172.21.15.16:0/393881102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40700d0070 0x7f407015f4b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f40640097a0 tx=0x7f406402a080 comp rx=0 tx=0).stop 2024-09-19T07:53:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.206+0000 7f40760dd640 1 -- 172.21.15.16:0/393881102 shutdown_connections 2024-09-19T07:53:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.206+0000 7f40760dd640 1 --2- 172.21.15.16:0/393881102 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40700d4130 0x7f40700d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.206+0000 7f40760dd640 1 --2- 172.21.15.16:0/393881102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40700d0070 0x7f407015f4b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.206+0000 7f40760dd640 1 -- 172.21.15.16:0/393881102 >> 172.21.15.16:0/393881102 conn(0x7f407009d760 msgr2=0x7f407008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.206+0000 7f40760dd640 1 -- 172.21.15.16:0/393881102 shutdown_connections 2024-09-19T07:53:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.206+0000 7f40760dd640 1 -- 172.21.15.16:0/393881102 wait complete. 2024-09-19T07:53:36.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.207+0000 7f40760dd640 1 Processor -- start 2024-09-19T07:53:36.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.207+0000 7f40760dd640 1 -- start start 2024-09-19T07:53:36.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.208+0000 7f40760dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40700d0070 0x7f407015b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:36.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.208+0000 7f40760dd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40700d4130 0x7f40700cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.208+0000 7f40760dd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f407015f0d0 0x7f40700cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.208+0000 7f40760dd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40700cef60 con 0x7f40700d0070 2024-09-19T07:53:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.208+0000 7f40760dd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f40700cfb50 con 0x7f407015f0d0 2024-09-19T07:53:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.208+0000 7f40760dd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f40700b2f70 con 0x7f40700d4130 2024-09-19T07:53:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.208+0000 7f406f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40700d0070 0x7f407015b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.208+0000 7f406f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40700d0070 0x7f407015b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38920/0 (socket says 172.21.15.16:38920) 2024-09-19T07:53:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.208+0000 7f406f7fe640 1 -- 172.21.15.16:0/917386826 learned_addr learned my addr 172.21.15.16:0/917386826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.208+0000 7f406f7fe640 1 -- 172.21.15.16:0/917386826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40700d4130 msgr2=0x7f40700cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.208+0000 7f406f7fe640 1 --2- 172.21.15.16:0/917386826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40700d4130 0x7f40700cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.208+0000 7f406f7fe640 1 -- 172.21.15.16:0/917386826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f407015f0d0 msgr2=0x7f40700cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.208+0000 7f406f7fe640 1 --2- 172.21.15.16:0/917386826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f407015f0d0 0x7f40700cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.208+0000 7f406f7fe640 1 -- 172.21.15.16:0/917386826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407015b950 con 0x7f40700d0070 2024-09-19T07:53:36.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.208+0000 7f406f7fe640 1 --2- 172.21.15.16:0/917386826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40700d0070 0x7f407015b410 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f406000c730 tx=0x7f406000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:36.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.209+0000 7f4057fff640 1 -- 172.21.15.16:0/917386826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060017070 con 0x7f40700d0070 2024-09-19T07:53:36.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.209+0000 7f4057fff640 1 -- 172.21.15.16:0/917386826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4060005ad0 con 0x7f40700d0070 2024-09-19T07:53:36.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.209+0000 7f40760dd640 1 -- 172.21.15.16:0/917386826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40700cb5e0 con 0x7f40700d0070 2024-09-19T07:53:36.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.209+0000 7f4057fff640 1 -- 172.21.15.16:0/917386826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060007070 con 0x7f40700d0070 2024-09-19T07:53:36.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.209+0000 7f40760dd640 1 -- 172.21.15.16:0/917386826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40700cbba0 con 0x7f40700d0070 2024-09-19T07:53:36.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.210+0000 7f40760dd640 1 -- 172.21.15.16:0/917386826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f407015f4b0 con 0x7f40700d0070 2024-09-19T07:53:36.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.210+0000 7f4057fff640 1 -- 172.21.15.16:0/917386826 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f406001a070 con 0x7f40700d0070 2024-09-19T07:53:36.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.211+0000 7f4057fff640 1 --2- 172.21.15.16:0/917386826 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f405c06c2b0 0x7f405c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:36.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.211+0000 7f4057fff640 1 -- 172.21.15.16:0/917386826 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5335+0+0 (secure 0 0 0) 0x7f406008a5f0 con 0x7f40700d0070 2024-09-19T07:53:36.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.211+0000 7f406effd640 1 --2- 172.21.15.16:0/917386826 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f405c06c2b0 0x7f405c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:36.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.211+0000 7f406effd640 1 --2- 172.21.15.16:0/917386826 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f405c06c2b0 0x7f405c06e770 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4064034000 tx=0x7f406402c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:36.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.215+0000 7f4057fff640 1 -- 172.21.15.16:0/917386826 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f407015b950 con 0x7f40700d0070 2024-09-19T07:53:36.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.395+0000 7f40760dd640 1 -- 172.21.15.16:0/917386826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f407006cf30 con 0x7f40700d0070 2024-09-19T07:53:36.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.396+0000 7f4057fff640 1 -- 172.21.15.16:0/917386826 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+13339 (secure 0 0 0) 0x7f4060058870 con 0x7f40700d0070 2024-09-19T07:53:36.396 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:36.396 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":109,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:53:35.077170+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/4185904511":"2024-09-20T07:53:31.953250+0000","172.21.15.16:6833/4185904511":"2024-09-20T07:53:31.953250+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:53:36.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.398+0000 7f40760dd640 1 -- 172.21.15.16:0/917386826 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f405c06c2b0 msgr2=0x7f405c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:36.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.398+0000 7f40760dd640 1 --2- 172.21.15.16:0/917386826 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f405c06c2b0 0x7f405c06e770 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4064034000 tx=0x7f406402c000 comp rx=0 tx=0).stop 2024-09-19T07:53:36.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.398+0000 7f40760dd640 1 -- 172.21.15.16:0/917386826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40700d0070 msgr2=0x7f407015b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:36.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.398+0000 7f40760dd640 1 --2- 172.21.15.16:0/917386826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40700d0070 0x7f407015b410 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f406000c730 tx=0x7f406000cbf0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.398+0000 7f40760dd640 1 -- 172.21.15.16:0/917386826 shutdown_connections 2024-09-19T07:53:36.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.398+0000 7f40760dd640 1 --2- 172.21.15.16:0/917386826 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f405c06c2b0 0x7f405c06e770 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.398+0000 7f40760dd640 1 --2- 172.21.15.16:0/917386826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f407015f0d0 0x7f40700cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.398+0000 7f40760dd640 1 --2- 172.21.15.16:0/917386826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40700d4130 0x7f40700cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.398+0000 7f40760dd640 1 --2- 172.21.15.16:0/917386826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40700d0070 0x7f407015b410 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.398+0000 7f40760dd640 1 -- 172.21.15.16:0/917386826 >> 172.21.15.16:0/917386826 conn(0x7f407009d760 msgr2=0x7f407006d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:36.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.398+0000 7f40760dd640 1 -- 172.21.15.16:0/917386826 shutdown_connections 2024-09-19T07:53:36.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.398+0000 7f40760dd640 1 -- 172.21.15.16:0/917386826 wait complete. 2024-09-19T07:53:36.409 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T07:42:54.500338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T07:53:36.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T07:53:36.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.515+0000 7fb2083c4640 1 Processor -- start 2024-09-19T07:53:36.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.515+0000 7fb2083c4640 1 -- start start 2024-09-19T07:53:36.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.516+0000 7fb2083c4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb2000d21f0 0x7fb2000d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:36.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.516+0000 7fb2083c4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2000d2b10 con 0x7fb200141ad0 2024-09-19T07:53:36.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.516+0000 7fb2083c4640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2001620b0 con 0x7fb200072590 2024-09-19T07:53:36.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.516+0000 7fb2083c4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb200141750 con 0x7fb2000d21f0 2024-09-19T07:53:36.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.516+0000 7fb206160640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb2000d21f0 0x7fb2000d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:36.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.516+0000 7fb206961640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb200141ad0 0x7fb200141f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40096/0 (socket says 172.21.15.16:40096) 2024-09-19T07:53:36.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.516+0000 7fb206961640 1 -- 172.21.15.16:0/4271895598 learned_addr learned my addr 172.21.15.16:0/4271895598 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:36.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.516+0000 7fb206160640 1 -- 172.21.15.16:0/4271895598 >> v1:172.21.15.146:6789/0 conn(0x7fb200072590 legacy=0x7fb20006d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:36.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.516+0000 7fb206160640 1 -- 172.21.15.16:0/4271895598 >> v1:172.21.15.16:6789/0 conn(0x7fb200141ad0 legacy=0x7fb200141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:36.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.516+0000 7fb206160640 1 -- 172.21.15.16:0/4271895598 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb200162370 con 0x7fb2000d21f0 2024-09-19T07:53:36.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.516+0000 7fb20515e640 1 -- 172.21.15.16:0/4271895598 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 888162229 0 0) 0x7fb2000d2b10 con 0x7fb200141ad0 2024-09-19T07:53:36.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.517+0000 7fb206160640 1 --2- 172.21.15.16:0/4271895598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2000d21f0 0x7fb2000d25d0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0097a0 tx=0x7fb1fc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75e1a1b4892b9cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:36.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.517+0000 7fb20515e640 1 -- 172.21.15.16:0/4271895598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc037070 con 0x7fb2000d21f0 2024-09-19T07:53:36.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.517+0000 7fb20515e640 1 -- 172.21.15.16:0/4271895598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1fc02ae50 con 0x7fb2000d21f0 2024-09-19T07:53:36.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.517+0000 7fb20515e640 1 -- 172.21.15.16:0/4271895598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc032660 con 0x7fb2000d21f0 2024-09-19T07:53:36.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.518+0000 7fb2083c4640 1 -- 172.21.15.16:0/4271895598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2000d21f0 msgr2=0x7fb2000d25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:36.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.518+0000 7fb2083c4640 1 --2- 172.21.15.16:0/4271895598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2000d21f0 0x7fb2000d25d0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0097a0 tx=0x7fb1fc02a140 comp rx=0 tx=0).stop 2024-09-19T07:53:36.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.518+0000 7fb2083c4640 1 -- 172.21.15.16:0/4271895598 shutdown_connections 2024-09-19T07:53:36.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.518+0000 7fb2083c4640 1 --2- 172.21.15.16:0/4271895598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2000d21f0 0x7fb2000d25d0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.518+0000 7fb2083c4640 1 -- 172.21.15.16:0/4271895598 >> 172.21.15.16:0/4271895598 conn(0x7fb2000b3ed0 msgr2=0x7fb2000b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:36.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.518+0000 7fb2083c4640 1 -- 172.21.15.16:0/4271895598 shutdown_connections 2024-09-19T07:53:36.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.518+0000 7fb2083c4640 1 -- 172.21.15.16:0/4271895598 wait complete. 2024-09-19T07:53:36.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.519+0000 7fb2083c4640 1 Processor -- start 2024-09-19T07:53:36.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.519+0000 7fb2083c4640 1 -- start start 2024-09-19T07:53:36.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb2083c4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb200072590 0x7fb2001f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:36.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb2083c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2000d21f0 0x7fb200158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:36.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb2083c4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb200141ad0 0x7fb200157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:36.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb2083c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb200162930 con 0x7fb2000d21f0 2024-09-19T07:53:36.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb2083c4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb20006d7f0 con 0x7fb200141ad0 2024-09-19T07:53:36.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb2083c4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb200162c10 con 0x7fb200072590 2024-09-19T07:53:36.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb20595f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2000d21f0 0x7fb200158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:36.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb20595f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2000d21f0 0x7fb200158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38942/0 (socket says 172.21.15.16:38942) 2024-09-19T07:53:36.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb20595f640 1 -- 172.21.15.16:0/3576615899 learned_addr learned my addr 172.21.15.16:0/3576615899 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:36.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb206961640 1 --2- 172.21.15.16:0/3576615899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb200141ad0 0x7fb200157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:36.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb206160640 1 --2- 172.21.15.16:0/3576615899 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb200072590 0x7fb2001f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:36.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb20595f640 1 -- 172.21.15.16:0/3576615899 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb200072590 msgr2=0x7fb2001f4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:36.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb20595f640 1 --2- 172.21.15.16:0/3576615899 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb200072590 0x7fb2001f4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb20595f640 1 -- 172.21.15.16:0/3576615899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb200141ad0 msgr2=0x7fb200157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:36.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb20595f640 1 --2- 172.21.15.16:0/3576615899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb200141ad0 0x7fb200157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb20595f640 1 -- 172.21.15.16:0/3576615899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb200157d70 con 0x7fb2000d21f0 2024-09-19T07:53:36.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb206961640 1 --2- 172.21.15.16:0/3576615899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb200141ad0 0x7fb200157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:36.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb20595f640 1 --2- 172.21.15.16:0/3576615899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2000d21f0 0x7fb200158e40 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000c430 tx=0x7fb1f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:36.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb1f6ffd640 1 -- 172.21.15.16:0/3576615899 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0011790 con 0x7fb2000d21f0 2024-09-19T07:53:36.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb1f6ffd640 1 -- 172.21.15.16:0/3576615899 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f0017070 con 0x7fb2000d21f0 2024-09-19T07:53:36.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb2083c4640 1 -- 172.21.15.16:0/3576615899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb200158180 con 0x7fb2000d21f0 2024-09-19T07:53:36.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb1f6ffd640 1 -- 172.21.15.16:0/3576615899 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0011bb0 con 0x7fb2000d21f0 2024-09-19T07:53:36.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.520+0000 7fb2083c4640 1 -- 172.21.15.16:0/3576615899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb200159820 con 0x7fb2000d21f0 2024-09-19T07:53:36.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.521+0000 7fb2083c4640 1 -- 172.21.15.16:0/3576615899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb200158c30 con 0x7fb2000d21f0 2024-09-19T07:53:36.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.522+0000 7fb1f6ffd640 1 -- 172.21.15.16:0/3576615899 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb1f001c460 con 0x7fb2000d21f0 2024-09-19T07:53:36.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.522+0000 7fb1f6ffd640 1 --2- 172.21.15.16:0/3576615899 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb1e406c2b0 0x7fb1e406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:36.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.522+0000 7fb1f6ffd640 1 -- 172.21.15.16:0/3576615899 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5335+0+0 (secure 0 0 0) 0x7fb1f008b1a0 con 0x7fb2000d21f0 2024-09-19T07:53:36.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.523+0000 7fb206160640 1 --2- 172.21.15.16:0/3576615899 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb1e406c2b0 0x7fb1e406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:36.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.523+0000 7fb206160640 1 --2- 172.21.15.16:0/3576615899 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb1e406c2b0 0x7fb1e406e770 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc034000 tx=0x7fb1fc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:36.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.527+0000 7fb1f6ffd640 1 -- 172.21.15.16:0/3576615899 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb200157d70 con 0x7fb2000d21f0 2024-09-19T07:53:36.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.714+0000 7fb2083c4640 1 -- 172.21.15.16:0/3576615899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb2000690c0 con 0x7fb2000d21f0 2024-09-19T07:53:36.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.715+0000 7fb1f6ffd640 1 -- 172.21.15.16:0/3576615899 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7fb1f0059a80 con 0x7fb2000d21f0 2024-09-19T07:53:36.715 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-19T07:53:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.717+0000 7fb2083c4640 1 -- 172.21.15.16:0/3576615899 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb1e406c2b0 msgr2=0x7fb1e406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:36.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.717+0000 7fb2083c4640 1 --2- 172.21.15.16:0/3576615899 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb1e406c2b0 0x7fb1e406e770 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc034000 tx=0x7fb1fc02e000 comp rx=0 tx=0).stop 2024-09-19T07:53:36.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.717+0000 7fb2083c4640 1 -- 172.21.15.16:0/3576615899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2000d21f0 msgr2=0x7fb200158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:36.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.717+0000 7fb2083c4640 1 --2- 172.21.15.16:0/3576615899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2000d21f0 0x7fb200158e40 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000c430 tx=0x7fb1f000c8f0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.717+0000 7fb2083c4640 1 -- 172.21.15.16:0/3576615899 shutdown_connections 2024-09-19T07:53:36.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.717+0000 7fb2083c4640 1 --2- 172.21.15.16:0/3576615899 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb1e406c2b0 0x7fb1e406e770 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.717+0000 7fb2083c4640 1 --2- 172.21.15.16:0/3576615899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb200141ad0 0x7fb200157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.717+0000 7fb2083c4640 1 --2- 172.21.15.16:0/3576615899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2000d21f0 0x7fb200158e40 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.717+0000 7fb2083c4640 1 --2- 172.21.15.16:0/3576615899 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb200072590 0x7fb2001f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.717+0000 7fb2083c4640 1 -- 172.21.15.16:0/3576615899 >> 172.21.15.16:0/3576615899 conn(0x7fb2000b3ed0 msgr2=0x7fb2000b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:36.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.717+0000 7fb2083c4640 1 -- 172.21.15.16:0/3576615899 shutdown_connections 2024-09-19T07:53:36.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.717+0000 7fb2083c4640 1 -- 172.21.15.16:0/3576615899 wait complete. 2024-09-19T07:53:36.728 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin' 2024-09-19T07:53:36.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.831+0000 7f99c542e640 1 Processor -- start 2024-09-19T07:53:36.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.831+0000 7f99c542e640 1 -- start start 2024-09-19T07:53:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.831+0000 7f99c542e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99c015e2a0 0x7f99c006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.831+0000 7f99c542e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99c00d1880 con 0x7f99c015e680 2024-09-19T07:53:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.831+0000 7f99c542e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99c00d1b60 con 0x7f99c006d960 2024-09-19T07:53:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.831+0000 7f99c542e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f99c00cf2d0 con 0x7f99c015e2a0 2024-09-19T07:53:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.832+0000 7f99beffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99c015e2a0 0x7f99c006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.832+0000 7f99be7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f99c015e680 0x7f99c00cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40108/0 (socket says 172.21.15.16:40108) 2024-09-19T07:53:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.832+0000 7f99beffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99c015e2a0 0x7f99c006d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38954/0 (socket says 172.21.15.16:38954) 2024-09-19T07:53:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.832+0000 7f99be7fc640 1 -- 172.21.15.16:0/2981001996 learned_addr learned my addr 172.21.15.16:0/2981001996 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.832+0000 7f99beffd640 1 -- 172.21.15.16:0/2981001996 >> v1:172.21.15.146:6789/0 conn(0x7f99c006d960 legacy=0x7f99c00d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.832+0000 7f99beffd640 1 -- 172.21.15.16:0/2981001996 >> v1:172.21.15.16:6789/0 conn(0x7f99c015e680 legacy=0x7f99c00cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.832+0000 7f99bdffb640 1 -- 172.21.15.16:0/2981001996 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1973094520 0 0) 0x7f99c00d1880 con 0x7f99c015e680 2024-09-19T07:53:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.832+0000 7f99beffd640 1 -- 172.21.15.16:0/2981001996 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c00cf5b0 con 0x7f99c015e2a0 2024-09-19T07:53:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.833+0000 7f99beffd640 1 --2- 172.21.15.16:0/2981001996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c015e2a0 0x7f99c006d420 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f99b00097a0 tx=0x7f99b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb885070f6cccf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.833+0000 7f99bdffb640 1 -- 172.21.15.16:0/2981001996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0037070 con 0x7f99c015e2a0 2024-09-19T07:53:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.833+0000 7f99bdffb640 1 -- 172.21.15.16:0/2981001996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b0032440 con 0x7f99c015e2a0 2024-09-19T07:53:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.833+0000 7f99bdffb640 1 -- 172.21.15.16:0/2981001996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0032810 con 0x7f99c015e2a0 2024-09-19T07:53:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.833+0000 7f99c542e640 1 -- 172.21.15.16:0/2981001996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c015e2a0 msgr2=0x7f99c006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.833+0000 7f99c542e640 1 --2- 172.21.15.16:0/2981001996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c015e2a0 0x7f99c006d420 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f99b00097a0 tx=0x7f99b002a1e0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.833+0000 7f99c542e640 1 -- 172.21.15.16:0/2981001996 shutdown_connections 2024-09-19T07:53:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.833+0000 7f99c542e640 1 --2- 172.21.15.16:0/2981001996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c015e2a0 0x7f99c006d420 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.833+0000 7f99c542e640 1 -- 172.21.15.16:0/2981001996 >> 172.21.15.16:0/2981001996 conn(0x7f99c009d760 msgr2=0x7f99c008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:36.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.833+0000 7f99c542e640 1 -- 172.21.15.16:0/2981001996 shutdown_connections 2024-09-19T07:53:36.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.833+0000 7f99c542e640 1 -- 172.21.15.16:0/2981001996 wait complete. 2024-09-19T07:53:36.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.835+0000 7f99c542e640 1 Processor -- start 2024-09-19T07:53:36.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.835+0000 7f99c542e640 1 -- start start 2024-09-19T07:53:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.835+0000 7f99c542e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c006d960 0x7f99c01f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.835+0000 7f99c542e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99c015e2a0 0x7f99c01f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.835+0000 7f99c542e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99c015e680 0x7f99c01f8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.835+0000 7f99c542e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f99c00d0360 con 0x7f99c006d960 2024-09-19T07:53:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.835+0000 7f99c542e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f99c00d01e0 con 0x7f99c015e2a0 2024-09-19T07:53:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.835+0000 7f99c542e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f99c00d4a80 con 0x7f99c015e680 2024-09-19T07:53:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.836+0000 7f99beffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c006d960 0x7f99c01f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.836+0000 7f99beffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c006d960 0x7f99c01f7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38960/0 (socket says 172.21.15.16:38960) 2024-09-19T07:53:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.836+0000 7f99beffd640 1 -- 172.21.15.16:0/2304556279 learned_addr learned my addr 172.21.15.16:0/2304556279 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.836+0000 7f99be7fc640 1 --2- 172.21.15.16:0/2304556279 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99c015e2a0 0x7f99c01f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.836+0000 7f99bf7fe640 1 --2- 172.21.15.16:0/2304556279 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99c015e680 0x7f99c01f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.836+0000 7f99beffd640 1 -- 172.21.15.16:0/2304556279 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99c015e680 msgr2=0x7f99c01f8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.836+0000 7f99beffd640 1 --2- 172.21.15.16:0/2304556279 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99c015e680 0x7f99c01f8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.836+0000 7f99beffd640 1 -- 172.21.15.16:0/2304556279 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99c015e2a0 msgr2=0x7f99c01f5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.836+0000 7f99beffd640 1 --2- 172.21.15.16:0/2304556279 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99c015e2a0 0x7f99c01f5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.836+0000 7f99beffd640 1 -- 172.21.15.16:0/2304556279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c00cb200 con 0x7f99c006d960 2024-09-19T07:53:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.836+0000 7f99beffd640 1 --2- 172.21.15.16:0/2304556279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c006d960 0x7f99c01f7a80 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f99b00098c0 tx=0x7f99b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.836+0000 7f99a77fe640 1 -- 172.21.15.16:0/2304556279 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0041070 con 0x7f99c006d960 2024-09-19T07:53:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.836+0000 7f99a77fe640 1 -- 172.21.15.16:0/2304556279 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b003c870 con 0x7f99c006d960 2024-09-19T07:53:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.836+0000 7f99c542e640 1 -- 172.21.15.16:0/2304556279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c00cc5a0 con 0x7f99c006d960 2024-09-19T07:53:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.836+0000 7f99a77fe640 1 -- 172.21.15.16:0/2304556279 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0037040 con 0x7f99c006d960 2024-09-19T07:53:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.836+0000 7f99c542e640 1 -- 172.21.15.16:0/2304556279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c00ccb60 con 0x7f99c006d960 2024-09-19T07:53:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.837+0000 7f99c542e640 1 -- 172.21.15.16:0/2304556279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c006de70 con 0x7f99c006d960 2024-09-19T07:53:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.838+0000 7f99a77fe640 1 -- 172.21.15.16:0/2304556279 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f99b0044070 con 0x7f99c006d960 2024-09-19T07:53:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.838+0000 7f99a77fe640 1 --2- 172.21.15.16:0/2304556279 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f99ac06c2b0 0x7f99ac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.838+0000 7f99a77fe640 1 -- 172.21.15.16:0/2304556279 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5335+0+0 (secure 0 0 0) 0x7f99b00abe80 con 0x7f99c006d960 2024-09-19T07:53:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.839+0000 7f99be7fc640 1 --2- 172.21.15.16:0/2304556279 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f99ac06c2b0 0x7f99ac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.839+0000 7f99be7fc640 1 --2- 172.21.15.16:0/2304556279 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f99ac06c2b0 0x7f99ac06e770 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f99c00ba490 tx=0x7f99b4007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:36.843+0000 7f99a77fe640 1 -- 172.21.15.16:0/2304556279 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f99c006de70 con 0x7f99c006d960 2024-09-19T07:53:37.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:37.087+0000 7f99c542e640 1 -- 172.21.15.16:0/2304556279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]} v 0) -- 0x7f99c006d420 con 0x7f99c006d960 2024-09-19T07:53:38.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.090+0000 7f99a77fe640 1 -- 172.21.15.16:0/2304556279 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]}]=0 v464) ==== 146+0+0 (secure 0 0 0) 0x7f99b007a760 con 0x7f99c006d960 2024-09-19T07:53:38.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.094+0000 7f99c542e640 1 -- 172.21.15.16:0/2304556279 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f99ac06c2b0 msgr2=0x7f99ac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.094+0000 7f99c542e640 1 --2- 172.21.15.16:0/2304556279 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f99ac06c2b0 0x7f99ac06e770 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f99c00ba490 tx=0x7f99b4007c00 comp rx=0 tx=0).stop 2024-09-19T07:53:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.094+0000 7f99c542e640 1 -- 172.21.15.16:0/2304556279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c006d960 msgr2=0x7f99c01f7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.094+0000 7f99c542e640 1 --2- 172.21.15.16:0/2304556279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c006d960 0x7f99c01f7a80 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f99b00098c0 tx=0x7f99b00046c0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.094+0000 7f99c542e640 1 -- 172.21.15.16:0/2304556279 shutdown_connections 2024-09-19T07:53:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.094+0000 7f99c542e640 1 --2- 172.21.15.16:0/2304556279 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f99ac06c2b0 0x7f99ac06e770 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.094+0000 7f99c542e640 1 --2- 172.21.15.16:0/2304556279 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99c015e680 0x7f99c01f8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.094+0000 7f99c542e640 1 --2- 172.21.15.16:0/2304556279 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99c015e2a0 0x7f99c01f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.094+0000 7f99c542e640 1 --2- 172.21.15.16:0/2304556279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c006d960 0x7f99c01f7a80 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.094+0000 7f99c542e640 1 -- 172.21.15.16:0/2304556279 >> 172.21.15.16:0/2304556279 conn(0x7f99c009d760 msgr2=0x7f99c0160880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.094+0000 7f99c542e640 1 -- 172.21.15.16:0/2304556279 shutdown_connections 2024-09-19T07:53:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.094+0000 7f99c542e640 1 -- 172.21.15.16:0/2304556279 wait complete. 2024-09-19T07:53:38.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T07:53:38.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.219+0000 7f3243bf8640 1 Processor -- start 2024-09-19T07:53:38.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.220+0000 7f3243bf8640 1 -- start start 2024-09-19T07:53:38.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.220+0000 7f3243bf8640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f323c15e4d0 0x7f323c0d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:38.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.220+0000 7f3243bf8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f323c0d29c0 0x7f323c0d2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:38.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.220+0000 7f3243bf8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f323c0d2250 con 0x7f323c15e8b0 2024-09-19T07:53:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.220+0000 7f3243bf8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f323c0d3370 con 0x7f323c0d29c0 2024-09-19T07:53:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.220+0000 7f3243bf8640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f323c0d1170 con 0x7f323c15e4d0 2024-09-19T07:53:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.221+0000 7f3241193640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f323c0d29c0 0x7f323c0d2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.221+0000 7f3241193640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f323c0d29c0 0x7f323c0d2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38974/0 (socket says 172.21.15.16:38974) 2024-09-19T07:53:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.221+0000 7f3241193640 1 -- 172.21.15.16:0/4207338603 learned_addr learned my addr 172.21.15.16:0/4207338603 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.221+0000 7f3241994640 1 --2- 172.21.15.16:0/4207338603 >> v2:172.21.15.146:3300/0 conn(0x7f323c15e4d0 0x7f323c0d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.221+0000 7f3241193640 1 -- 172.21.15.16:0/4207338603 >> v2:172.21.15.146:3300/0 conn(0x7f323c15e4d0 msgr2=0x7f323c0d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.221+0000 7f3241193640 1 --2- 172.21.15.16:0/4207338603 >> v2:172.21.15.146:3300/0 conn(0x7f323c15e4d0 0x7f323c0d1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.221+0000 7f3241193640 1 -- 172.21.15.16:0/4207338603 >> v1:172.21.15.16:6789/0 conn(0x7f323c15e8b0 legacy=0x7f323c15ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.221+0000 7f3241193640 1 -- 172.21.15.16:0/4207338603 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323c15f520 con 0x7f323c0d29c0 2024-09-19T07:53:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.221+0000 7f3240992640 1 -- 172.21.15.16:0/4207338603 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2157435240 0 0) 0x7f323c0d2250 con 0x7f323c15e8b0 2024-09-19T07:53:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.221+0000 7f3241994640 1 --2- 172.21.15.16:0/4207338603 >> v2:172.21.15.146:3300/0 conn(0x7f323c15e4d0 0x7f323c0d1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.222+0000 7f3241193640 1 --2- 172.21.15.16:0/4207338603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323c0d29c0 0x7f323c0d2e30 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f323c0d4340 tx=0x7f323002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fb58afcd0d8361 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.222+0000 7f3240992640 1 -- 172.21.15.16:0/4207338603 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230004030 con 0x7f323c0d29c0 2024-09-19T07:53:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.222+0000 7f3240992640 1 -- 172.21.15.16:0/4207338603 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32300041d0 con 0x7f323c0d29c0 2024-09-19T07:53:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.222+0000 7f3240992640 1 -- 172.21.15.16:0/4207338603 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230003610 con 0x7f323c0d29c0 2024-09-19T07:53:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.222+0000 7f3243bf8640 1 -- 172.21.15.16:0/4207338603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323c0d29c0 msgr2=0x7f323c0d2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.222+0000 7f3243bf8640 1 --2- 172.21.15.16:0/4207338603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323c0d29c0 0x7f323c0d2e30 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f323c0d4340 tx=0x7f323002a250 comp rx=0 tx=0).stop 2024-09-19T07:53:38.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.222+0000 7f3243bf8640 1 -- 172.21.15.16:0/4207338603 shutdown_connections 2024-09-19T07:53:38.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.222+0000 7f3243bf8640 1 --2- 172.21.15.16:0/4207338603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323c0d29c0 0x7f323c0d2e30 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.222+0000 7f3243bf8640 1 --2- 172.21.15.16:0/4207338603 >> v2:172.21.15.146:3300/0 conn(0x7f323c15e4d0 0x7f323c0d1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.222+0000 7f3243bf8640 1 -- 172.21.15.16:0/4207338603 >> 172.21.15.16:0/4207338603 conn(0x7f323c09d770 msgr2=0x7f323c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:38.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.222+0000 7f3243bf8640 1 -- 172.21.15.16:0/4207338603 shutdown_connections 2024-09-19T07:53:38.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.222+0000 7f3243bf8640 1 -- 172.21.15.16:0/4207338603 wait complete. 2024-09-19T07:53:38.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.224+0000 7f3243bf8640 1 Processor -- start 2024-09-19T07:53:38.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.224+0000 7f3243bf8640 1 -- start start 2024-09-19T07:53:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.224+0000 7f3243bf8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f323c0d29c0 0x7f323c0cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.224+0000 7f3243bf8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f323c15e4d0 0x7f323c160660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.224+0000 7f3243bf8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323c15e8b0 0x7f323c1f5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.224+0000 7f3243bf8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f323c0d0e00 con 0x7f323c15e8b0 2024-09-19T07:53:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.224+0000 7f3243bf8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f323c0d0b20 con 0x7f323c0d29c0 2024-09-19T07:53:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.224+0000 7f3243bf8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f323c0d1b40 con 0x7f323c15e4d0 2024-09-19T07:53:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.225+0000 7f3242195640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323c15e8b0 0x7f323c1f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.225+0000 7f3242195640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323c15e8b0 0x7f323c1f5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38990/0 (socket says 172.21.15.16:38990) 2024-09-19T07:53:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.225+0000 7f3242195640 1 -- 172.21.15.16:0/3085075077 learned_addr learned my addr 172.21.15.16:0/3085075077 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.225+0000 7f3241994640 1 --2- 172.21.15.16:0/3085075077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f323c0d29c0 0x7f323c0cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.225+0000 7f3241193640 1 --2- 172.21.15.16:0/3085075077 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f323c15e4d0 0x7f323c160660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.225+0000 7f3242195640 1 -- 172.21.15.16:0/3085075077 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f323c15e4d0 msgr2=0x7f323c160660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.225+0000 7f3242195640 1 --2- 172.21.15.16:0/3085075077 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f323c15e4d0 0x7f323c160660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.225+0000 7f3242195640 1 -- 172.21.15.16:0/3085075077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f323c0d29c0 msgr2=0x7f323c0cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.225+0000 7f3242195640 1 --2- 172.21.15.16:0/3085075077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f323c0d29c0 0x7f323c0cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.225+0000 7f3242195640 1 -- 172.21.15.16:0/3085075077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323c160ba0 con 0x7f323c15e8b0 2024-09-19T07:53:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.225+0000 7f3242195640 1 --2- 172.21.15.16:0/3085075077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323c15e8b0 0x7f323c1f5110 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f322c00c750 tx=0x7f322c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.225+0000 7f323a7fc640 1 -- 172.21.15.16:0/3085075077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c003970 con 0x7f323c15e8b0 2024-09-19T07:53:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.225+0000 7f323a7fc640 1 -- 172.21.15.16:0/3085075077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f322c003b30 con 0x7f323c15e8b0 2024-09-19T07:53:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.225+0000 7f3243bf8640 1 -- 172.21.15.16:0/3085075077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f323c1f5650 con 0x7f323c15e8b0 2024-09-19T07:53:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.225+0000 7f323a7fc640 1 -- 172.21.15.16:0/3085075077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c00a850 con 0x7f323c15e8b0 2024-09-19T07:53:38.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.225+0000 7f3243bf8640 1 -- 172.21.15.16:0/3085075077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f323c0cb830 con 0x7f323c15e8b0 2024-09-19T07:53:38.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.226+0000 7f3243bf8640 1 -- 172.21.15.16:0/3085075077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f323c0d2e30 con 0x7f323c15e8b0 2024-09-19T07:53:38.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.227+0000 7f323a7fc640 1 -- 172.21.15.16:0/3085075077 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f322c00a9f0 con 0x7f323c15e8b0 2024-09-19T07:53:38.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.228+0000 7f323a7fc640 1 --2- 172.21.15.16:0/3085075077 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f322806c2b0 0x7f322806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:38.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.228+0000 7f323a7fc640 1 -- 172.21.15.16:0/3085075077 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5335+0+0 (secure 0 0 0) 0x7f322c08afc0 con 0x7f323c15e8b0 2024-09-19T07:53:38.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.228+0000 7f3241994640 1 --2- 172.21.15.16:0/3085075077 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f322806c2b0 0x7f322806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:38.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.228+0000 7f3241994640 1 --2- 172.21.15.16:0/3085075077 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f322806c2b0 0x7f322806e770 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f323c0b28c0 tx=0x7f3234008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:38.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.232+0000 7f323a7fc640 1 -- 172.21.15.16:0/3085075077 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f322c059240 con 0x7f323c15e8b0 2024-09-19T07:53:38.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.407+0000 7f3243bf8640 1 -- 172.21.15.16:0/3085075077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f323c06cf90 con 0x7f323c15e8b0 2024-09-19T07:53:38.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.410+0000 7f323a7fc640 1 -- 172.21.15.16:0/3085075077 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v43)=0 v43) ==== 122+0+0 (secure 0 0 0) 0x7f322c05d540 con 0x7f323c15e8b0 2024-09-19T07:53:38.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.413+0000 7f3243bf8640 1 -- 172.21.15.16:0/3085075077 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f322806c2b0 msgr2=0x7f322806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:38.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.413+0000 7f3243bf8640 1 --2- 172.21.15.16:0/3085075077 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f322806c2b0 0x7f322806e770 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f323c0b28c0 tx=0x7f3234008040 comp rx=0 tx=0).stop 2024-09-19T07:53:38.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.413+0000 7f3243bf8640 1 -- 172.21.15.16:0/3085075077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323c15e8b0 msgr2=0x7f323c1f5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:38.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.413+0000 7f3243bf8640 1 --2- 172.21.15.16:0/3085075077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323c15e8b0 0x7f323c1f5110 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f322c00c750 tx=0x7f322c00cc10 comp rx=0 tx=0).stop 2024-09-19T07:53:38.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.413+0000 7f3243bf8640 1 -- 172.21.15.16:0/3085075077 shutdown_connections 2024-09-19T07:53:38.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.413+0000 7f3243bf8640 1 --2- 172.21.15.16:0/3085075077 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f322806c2b0 0x7f322806e770 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.413+0000 7f3243bf8640 1 --2- 172.21.15.16:0/3085075077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323c15e8b0 0x7f323c1f5110 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.413+0000 7f3243bf8640 1 --2- 172.21.15.16:0/3085075077 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f323c15e4d0 0x7f323c160660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.413+0000 7f3243bf8640 1 --2- 172.21.15.16:0/3085075077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f323c0d29c0 0x7f323c0cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.413+0000 7f3243bf8640 1 -- 172.21.15.16:0/3085075077 >> 172.21.15.16:0/3085075077 conn(0x7f323c09d770 msgr2=0x7f323c06da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:38.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.413+0000 7f3243bf8640 1 -- 172.21.15.16:0/3085075077 shutdown_connections 2024-09-19T07:53:38.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.413+0000 7f3243bf8640 1 -- 172.21.15.16:0/3085075077 wait complete. 2024-09-19T07:53:38.424 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:53:38.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:53:38.471 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:53:38.520 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:53:38.520 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:53:38.520 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:53:38.536 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:53:38.536 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:53:38.537 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:53:38.554 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:53:38.555 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:53:38.555 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:53:38.593 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:53:38.593 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:53:38.593 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:53:38.613 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:53:38.614 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T07:53:38.614 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:53:38.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.735+0000 7fe88cb5f640 1 Processor -- start 2024-09-19T07:53:38.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.735+0000 7fe88cb5f640 1 -- start start 2024-09-19T07:53:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.736+0000 7fe88cb5f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe88815b120 0x7fe88806f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.736+0000 7fe88cb5f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe8880d0bc0 0x7fe88806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.736+0000 7fe88cb5f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe88815b500 con 0x7fe88806dec0 2024-09-19T07:53:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.736+0000 7fe88cb5f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fe88806e940 con 0x7fe88815b120 2024-09-19T07:53:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.736+0000 7fe88cb5f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe88806ec20 con 0x7fe8880d0bc0 2024-09-19T07:53:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.736+0000 7fe88659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe88815b120 0x7fe88806f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.736+0000 7fe885d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe88806dec0 0x7fe8880d04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40126/0 (socket says 172.21.15.16:40126) 2024-09-19T07:53:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.736+0000 7fe88659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe88815b120 0x7fe88806f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38996/0 (socket says 172.21.15.16:38996) 2024-09-19T07:53:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.736+0000 7fe885d9b640 1 -- 172.21.15.16:0/3868993865 learned_addr learned my addr 172.21.15.16:0/3868993865 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.736+0000 7fe886d9d640 1 --2- 172.21.15.16:0/3868993865 >> v2:172.21.15.146:3300/0 conn(0x7fe8880d0bc0 0x7fe88806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:38.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.736+0000 7fe88659c640 1 -- 172.21.15.16:0/3868993865 >> v2:172.21.15.146:3300/0 conn(0x7fe8880d0bc0 msgr2=0x7fe88806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:38.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.736+0000 7fe88659c640 1 --2- 172.21.15.16:0/3868993865 >> v2:172.21.15.146:3300/0 conn(0x7fe8880d0bc0 0x7fe88806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.736+0000 7fe88659c640 1 -- 172.21.15.16:0/3868993865 >> v1:172.21.15.16:6789/0 conn(0x7fe88806dec0 legacy=0x7fe8880d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:38.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.736+0000 7fe88659c640 1 -- 172.21.15.16:0/3868993865 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88806ef00 con 0x7fe88815b120 2024-09-19T07:53:38.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.736+0000 7fe88559a640 1 -- 172.21.15.16:0/3868993865 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1126420065 0 0) 0x7fe88815b500 con 0x7fe88806dec0 2024-09-19T07:53:38.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.737+0000 7fe88659c640 1 --2- 172.21.15.16:0/3868993865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe88815b120 0x7fe88806f9e0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fe88815eb60 tx=0x7fe87802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f6296b9e5d41a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:38.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.737+0000 7fe88559a640 1 -- 172.21.15.16:0/3868993865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878037070 con 0x7fe88815b120 2024-09-19T07:53:38.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.737+0000 7fe88559a640 1 -- 172.21.15.16:0/3868993865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe878032440 con 0x7fe88815b120 2024-09-19T07:53:38.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.737+0000 7fe88559a640 1 -- 172.21.15.16:0/3868993865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878032810 con 0x7fe88815b120 2024-09-19T07:53:38.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.738+0000 7fe88cb5f640 1 -- 172.21.15.16:0/3868993865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe88815b120 msgr2=0x7fe88806f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:38.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.738+0000 7fe88cb5f640 1 --2- 172.21.15.16:0/3868993865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe88815b120 0x7fe88806f9e0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fe88815eb60 tx=0x7fe87802a1a0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.738+0000 7fe88cb5f640 1 -- 172.21.15.16:0/3868993865 shutdown_connections 2024-09-19T07:53:38.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.738+0000 7fe88cb5f640 1 --2- 172.21.15.16:0/3868993865 >> v2:172.21.15.146:3300/0 conn(0x7fe8880d0bc0 0x7fe88806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.738+0000 7fe88cb5f640 1 --2- 172.21.15.16:0/3868993865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe88815b120 0x7fe88806f9e0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.738+0000 7fe88cb5f640 1 -- 172.21.15.16:0/3868993865 >> 172.21.15.16:0/3868993865 conn(0x7fe88806bdf0 msgr2=0x7fe88806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:38.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.738+0000 7fe88cb5f640 1 -- 172.21.15.16:0/3868993865 shutdown_connections 2024-09-19T07:53:38.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.738+0000 7fe88cb5f640 1 -- 172.21.15.16:0/3868993865 wait complete. 2024-09-19T07:53:38.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.739+0000 7fe88cb5f640 1 Processor -- start 2024-09-19T07:53:38.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.739+0000 7fe88cb5f640 1 -- start start 2024-09-19T07:53:38.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.740+0000 7fe88cb5f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe88806dec0 0x7fe888158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:38.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.740+0000 7fe88cb5f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8880d0bc0 0x7fe8881573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:38.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.740+0000 7fe88cb5f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe88815b120 0x7fe88814f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:38.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.740+0000 7fe88cb5f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe8880ce060 con 0x7fe8880d0bc0 2024-09-19T07:53:38.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.740+0000 7fe88cb5f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe88806f4c0 con 0x7fe88806dec0 2024-09-19T07:53:38.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.740+0000 7fe88cb5f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe8880c5160 con 0x7fe88815b120 2024-09-19T07:53:38.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.740+0000 7fe885d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8880d0bc0 0x7fe8881573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:38.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.740+0000 7fe885d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8880d0bc0 0x7fe8881573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39000/0 (socket says 172.21.15.16:39000) 2024-09-19T07:53:38.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.740+0000 7fe885d9b640 1 -- 172.21.15.16:0/1813006065 learned_addr learned my addr 172.21.15.16:0/1813006065 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:38.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.740+0000 7fe886d9d640 1 --2- 172.21.15.16:0/1813006065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe88815b120 0x7fe88814f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:38.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.740+0000 7fe88659c640 1 --2- 172.21.15.16:0/1813006065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe88806dec0 0x7fe888158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:38.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.740+0000 7fe885d9b640 1 -- 172.21.15.16:0/1813006065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe88815b120 msgr2=0x7fe88814f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:38.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.740+0000 7fe885d9b640 1 --2- 172.21.15.16:0/1813006065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe88815b120 0x7fe88814f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.740+0000 7fe885d9b640 1 -- 172.21.15.16:0/1813006065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe88806dec0 msgr2=0x7fe888158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:38.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.740+0000 7fe885d9b640 1 --2- 172.21.15.16:0/1813006065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe88806dec0 0x7fe888158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.740+0000 7fe885d9b640 1 -- 172.21.15.16:0/1813006065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8881543f0 con 0x7fe8880d0bc0 2024-09-19T07:53:38.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.740+0000 7fe886d9d640 1 --2- 172.21.15.16:0/1813006065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe88815b120 0x7fe88814f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:38.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.740+0000 7fe885d9b640 1 --2- 172.21.15.16:0/1813006065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8880d0bc0 0x7fe8881573e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00c500 tx=0x7fe87c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:38.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.740+0000 7fe876ffd640 1 -- 172.21.15.16:0/1813006065 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c018070 con 0x7fe8880d0bc0 2024-09-19T07:53:38.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.741+0000 7fe876ffd640 1 -- 172.21.15.16:0/1813006065 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe87c007580 con 0x7fe8880d0bc0 2024-09-19T07:53:38.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.741+0000 7fe88cb5f640 1 -- 172.21.15.16:0/1813006065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8881546d0 con 0x7fe8880d0bc0 2024-09-19T07:53:38.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.741+0000 7fe876ffd640 1 -- 172.21.15.16:0/1813006065 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c004a20 con 0x7fe8880d0bc0 2024-09-19T07:53:38.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.741+0000 7fe88cb5f640 1 -- 172.21.15.16:0/1813006065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe888154f70 con 0x7fe8880d0bc0 2024-09-19T07:53:38.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.742+0000 7fe88cb5f640 1 -- 172.21.15.16:0/1813006065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8881548a0 con 0x7fe8880d0bc0 2024-09-19T07:53:38.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.742+0000 7fe876ffd640 1 -- 172.21.15.16:0/1813006065 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe87c01b070 con 0x7fe8880d0bc0 2024-09-19T07:53:38.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.742+0000 7fe876ffd640 1 --2- 172.21.15.16:0/1813006065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe86806c2b0 0x7fe86806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:38.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.742+0000 7fe876ffd640 1 -- 172.21.15.16:0/1813006065 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5335+0+0 (secure 0 0 0) 0x7fe87c08a5a0 con 0x7fe8880d0bc0 2024-09-19T07:53:38.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.743+0000 7fe88659c640 1 --2- 172.21.15.16:0/1813006065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe86806c2b0 0x7fe86806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:38.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.743+0000 7fe88659c640 1 --2- 172.21.15.16:0/1813006065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe86806c2b0 0x7fe86806e770 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe878034000 tx=0x7fe87802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.747+0000 7fe876ffd640 1 -- 172.21.15.16:0/1813006065 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe87c058f10 con 0x7fe8880d0bc0 2024-09-19T07:53:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.957+0000 7fe88cb5f640 1 -- 172.21.15.16:0/1813006065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe888155140 con 0x7fe8880d0bc0 2024-09-19T07:53:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.958+0000 7fe876ffd640 1 -- 172.21.15.16:0/1813006065 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+3914 (secure 0 0 0) 0x7fe87c058810 con 0x7fe8880d0bc0 2024-09-19T07:53:38.958 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:38.959 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":59,"btime":"2024-09-19T07:53:32:954789+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6017,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2955839490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":2955839490},{"type":"v1","addr":"172.21.15.146:6837","nonce":2955839490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6550,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/395357736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":395357736},{"type":"v1","addr":"172.21.15.146:6836","nonce":395357736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3539638739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3539638739},{"type":"v1","addr":"172.21.15.16:6834","nonce":3539638739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":7284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4078410600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4078410600},{"type":"v1","addr":"172.21.15.16:6835","nonce":4078410600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[]} 2024-09-19T07:53:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 59 2024-09-19T07:53:38.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.960+0000 7fe88cb5f640 1 -- 172.21.15.16:0/1813006065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe86806c2b0 msgr2=0x7fe86806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:38.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.960+0000 7fe88cb5f640 1 --2- 172.21.15.16:0/1813006065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe86806c2b0 0x7fe86806e770 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe878034000 tx=0x7fe87802e000 comp rx=0 tx=0).stop 2024-09-19T07:53:38.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.960+0000 7fe88cb5f640 1 -- 172.21.15.16:0/1813006065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8880d0bc0 msgr2=0x7fe8881573e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:38.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.960+0000 7fe88cb5f640 1 --2- 172.21.15.16:0/1813006065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8880d0bc0 0x7fe8881573e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00c500 tx=0x7fe87c00c9c0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.960+0000 7fe88cb5f640 1 -- 172.21.15.16:0/1813006065 shutdown_connections 2024-09-19T07:53:38.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.960+0000 7fe88cb5f640 1 --2- 172.21.15.16:0/1813006065 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe86806c2b0 0x7fe86806e770 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.960+0000 7fe88cb5f640 1 --2- 172.21.15.16:0/1813006065 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe88815b120 0x7fe88814f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.960+0000 7fe88cb5f640 1 --2- 172.21.15.16:0/1813006065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8880d0bc0 0x7fe8881573e0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.960+0000 7fe88cb5f640 1 --2- 172.21.15.16:0/1813006065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe88806dec0 0x7fe888158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:38.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.960+0000 7fe88cb5f640 1 -- 172.21.15.16:0/1813006065 >> 172.21.15.16:0/1813006065 conn(0x7fe88806bdf0 msgr2=0x7fe8880cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:38.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.960+0000 7fe88cb5f640 1 -- 172.21.15.16:0/1813006065 shutdown_connections 2024-09-19T07:53:38.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:38.960+0000 7fe88cb5f640 1 -- 172.21.15.16:0/1813006065 wait complete. 2024-09-19T07:53:38.972 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T07:53:38.972 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T07:53:38.972 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T07:53:38.972 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:53:38.972 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T07:53:38.972 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T07:53:38.973 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T07:53:38.973 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:53:38.973 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T07:53:38.973 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T07:53:38.973 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T07:53:38.973 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:53:38.973 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T07:53:38.973 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T07:53:38.974 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T07:53:38.974 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:53:38.981 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T07:53:38.981 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T07:53:38.982 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T07:53:38.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T07:53:38.985 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T07:53:38.985 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T07:53:38.986 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T07:53:38.986 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T07:53:39.023 INFO:tasks.ceph.mds.a:Started 2024-09-19T07:53:39.024 DEBUG:teuthology.parallel:result is None 2024-09-19T07:53:39.024 INFO:tasks.ceph.mds.c:Started 2024-09-19T07:53:39.025 DEBUG:teuthology.parallel:result is None 2024-09-19T07:53:39.027 INFO:tasks.ceph.mds.b:Started 2024-09-19T07:53:39.027 DEBUG:teuthology.parallel:result is None 2024-09-19T07:53:39.028 INFO:tasks.ceph.mds.d:Started 2024-09-19T07:53:39.028 DEBUG:teuthology.parallel:result is None 2024-09-19T07:53:39.028 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T07:53:39.101 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-19T07:53:39.102 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-19T07:53:39.108 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2024-09-19T07:53:39.109 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2024-09-19T07:53:39.122 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:53:39.125 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:53:39.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.139+0000 7fcfe0bd8640 1 Processor -- start 2024-09-19T07:53:39.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.139+0000 7fcfe0bd8640 1 -- start start 2024-09-19T07:53:39.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.140+0000 7fcfe0bd8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcfdc15cb00 0x7fcfdc0d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:39.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.140+0000 7fcfe0bd8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcfdc15cee0 0x7fcfdc15d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:39.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.140+0000 7fcfe0bd8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfdc0d22d0 con 0x7fcfdc0cfdb0 2024-09-19T07:53:39.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.140+0000 7fcfe0bd8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fcfdc0b40b0 con 0x7fcfdc15cee0 2024-09-19T07:53:39.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.140+0000 7fcfe0bd8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcfdc0d25b0 con 0x7fcfdc15cb00 2024-09-19T07:53:39.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.141+0000 7fcfd9d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcfdc15cee0 0x7fcfdc15d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:39.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.141+0000 7fcfdad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcfdc0cfdb0 0x7fcfdc1562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40134/0 (socket says 172.21.15.16:40134) 2024-09-19T07:53:39.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.141+0000 7fcfdad9d640 1 -- 172.21.15.16:0/2301349842 learned_addr learned my addr 172.21.15.16:0/2301349842 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:39.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.141+0000 7fcfda59c640 1 --2- 172.21.15.16:0/2301349842 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcfdc15cb00 0x7fcfdc0d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:39.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.141+0000 7fcfd9d9b640 1 -- 172.21.15.16:0/2301349842 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcfdc15cb00 msgr2=0x7fcfdc0d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:39.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.141+0000 7fcfd9d9b640 1 --2- 172.21.15.16:0/2301349842 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcfdc15cb00 0x7fcfdc0d1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:39.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.141+0000 7fcfd9d9b640 1 -- 172.21.15.16:0/2301349842 >> v1:172.21.15.16:6789/0 conn(0x7fcfdc0cfdb0 legacy=0x7fcfdc1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:39.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.141+0000 7fcfd959a640 1 -- 172.21.15.16:0/2301349842 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 937436556 0 0) 0x7fcfdc0d22d0 con 0x7fcfdc0cfdb0 2024-09-19T07:53:39.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.141+0000 7fcfd9d9b640 1 -- 172.21.15.16:0/2301349842 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc0d2890 con 0x7fcfdc15cee0 2024-09-19T07:53:39.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.142+0000 7fcfd9d9b640 1 --2- 172.21.15.16:0/2301349842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfdc15cee0 0x7fcfdc15d340 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40097a0 tx=0x7fcfc402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4973588141a2ea3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:39.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.142+0000 7fcfd959a640 1 -- 172.21.15.16:0/2301349842 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4037070 con 0x7fcfdc15cee0 2024-09-19T07:53:39.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.142+0000 7fcfd959a640 1 -- 172.21.15.16:0/2301349842 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfc402ae50 con 0x7fcfdc15cee0 2024-09-19T07:53:39.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.142+0000 7fcfd959a640 1 -- 172.21.15.16:0/2301349842 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4032660 con 0x7fcfdc15cee0 2024-09-19T07:53:39.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.142+0000 7fcfe0bd8640 1 -- 172.21.15.16:0/2301349842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfdc15cee0 msgr2=0x7fcfdc15d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:39.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.142+0000 7fcfe0bd8640 1 --2- 172.21.15.16:0/2301349842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfdc15cee0 0x7fcfdc15d340 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40097a0 tx=0x7fcfc402a100 comp rx=0 tx=0).stop 2024-09-19T07:53:39.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.143+0000 7fcfe0bd8640 1 -- 172.21.15.16:0/2301349842 shutdown_connections 2024-09-19T07:53:39.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.143+0000 7fcfe0bd8640 1 --2- 172.21.15.16:0/2301349842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfdc15cee0 0x7fcfdc15d340 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:39.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.143+0000 7fcfe0bd8640 1 --2- 172.21.15.16:0/2301349842 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcfdc15cb00 0x7fcfdc0d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:39.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.143+0000 7fcfe0bd8640 1 -- 172.21.15.16:0/2301349842 >> 172.21.15.16:0/2301349842 conn(0x7fcfdc067fe0 msgr2=0x7fcfdc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:39.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.143+0000 7fcfe0bd8640 1 -- 172.21.15.16:0/2301349842 shutdown_connections 2024-09-19T07:53:39.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.143+0000 7fcfe0bd8640 1 -- 172.21.15.16:0/2301349842 wait complete. 2024-09-19T07:53:39.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.144+0000 7fcfe0bd8640 1 Processor -- start 2024-09-19T07:53:39.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.144+0000 7fcfe0bd8640 1 -- start start 2024-09-19T07:53:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.145+0000 7fcfe0bd8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfdc0cfdb0 0x7fcfdc13ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.145+0000 7fcfe0bd8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfdc15cb00 0x7fcfdc15e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.145+0000 7fcfe0bd8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcfdc15cee0 0x7fcfdc15eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.145+0000 7fcfe0bd8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc156b50 con 0x7fcfdc15cb00 2024-09-19T07:53:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.145+0000 7fcfe0bd8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1569d0 con 0x7fcfdc0cfdb0 2024-09-19T07:53:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.145+0000 7fcfe0bd8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcfdc156cd0 con 0x7fcfdc15cee0 2024-09-19T07:53:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.145+0000 7fcfd9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfdc15cb00 0x7fcfdc15e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.145+0000 7fcfd9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfdc15cb00 0x7fcfdc15e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39056/0 (socket says 172.21.15.16:39056) 2024-09-19T07:53:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.145+0000 7fcfd9d9b640 1 -- 172.21.15.16:0/3542840289 learned_addr learned my addr 172.21.15.16:0/3542840289 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.145+0000 7fcfda59c640 1 --2- 172.21.15.16:0/3542840289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfdc0cfdb0 0x7fcfdc13ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.145+0000 7fcfdad9d640 1 --2- 172.21.15.16:0/3542840289 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcfdc15cee0 0x7fcfdc15eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.146+0000 7fcfda59c640 1 -- 172.21.15.16:0/3542840289 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcfdc15cee0 msgr2=0x7fcfdc15eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.146+0000 7fcfda59c640 1 --2- 172.21.15.16:0/3542840289 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcfdc15cee0 0x7fcfdc15eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.146+0000 7fcfda59c640 1 -- 172.21.15.16:0/3542840289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfdc15cb00 msgr2=0x7fcfdc15e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.146+0000 7fcfda59c640 1 --2- 172.21.15.16:0/3542840289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfdc15cb00 0x7fcfdc15e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.146+0000 7fcfda59c640 1 -- 172.21.15.16:0/3542840289 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc13f110 con 0x7fcfdc0cfdb0 2024-09-19T07:53:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.146+0000 7fcfda59c640 1 --2- 172.21.15.16:0/3542840289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfdc0cfdb0 0x7fcfdc13ebd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00e8a0 tx=0x7fcfcc00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.146+0000 7fcfd2ffd640 1 -- 172.21.15.16:0/3542840289 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc017070 con 0x7fcfdc0cfdb0 2024-09-19T07:53:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.146+0000 7fcfd2ffd640 1 -- 172.21.15.16:0/3542840289 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfcc00c940 con 0x7fcfdc0cfdb0 2024-09-19T07:53:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.146+0000 7fcfd2ffd640 1 -- 172.21.15.16:0/3542840289 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc009e60 con 0x7fcfdc0cfdb0 2024-09-19T07:53:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.146+0000 7fcfe0bd8640 1 -- 172.21.15.16:0/3542840289 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfdc0c9350 con 0x7fcfdc0cfdb0 2024-09-19T07:53:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.146+0000 7fcfe0bd8640 1 -- 172.21.15.16:0/3542840289 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfdc0c98f0 con 0x7fcfdc0cfdb0 2024-09-19T07:53:39.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.147+0000 7fcfe0bd8640 1 -- 172.21.15.16:0/3542840289 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfdc0d1ca0 con 0x7fcfdc0cfdb0 2024-09-19T07:53:39.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.148+0000 7fcfd2ffd640 1 -- 172.21.15.16:0/3542840289 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcfcc01a070 con 0x7fcfdc0cfdb0 2024-09-19T07:53:39.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.148+0000 7fcfd2ffd640 1 --2- 172.21.15.16:0/3542840289 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcfbc06c2b0 0x7fcfbc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:39.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.149+0000 7fcfd2ffd640 1 -- 172.21.15.16:0/3542840289 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5335+0+0 (secure 0 0 0) 0x7fcfcc08b420 con 0x7fcfdc0cfdb0 2024-09-19T07:53:39.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.149+0000 7fcfd9d9b640 1 --2- 172.21.15.16:0/3542840289 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcfbc06c2b0 0x7fcfbc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:39.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.149+0000 7fcfd9d9b640 1 --2- 172.21.15.16:0/3542840289 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcfbc06c2b0 0x7fcfbc06e770 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcfc402a140 tx=0x7fcfc4040040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:39.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.153+0000 7fcfd2ffd640 1 -- 172.21.15.16:0/3542840289 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fcfdc13f110 con 0x7fcfdc0cfdb0 2024-09-19T07:53:39.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:39.340+0000 7fcfe0bd8640 1 -- 172.21.15.16:0/3542840289 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fcfdc0d1520 con 0x7fcfdc0cfdb0 2024-09-19T07:53:40.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.440+0000 7fcfd2ffd640 1 -- 172.21.15.16:0/3542840289 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v114) ==== 97+0+0 (secure 0 0 0) 0x7fcfcc059610 con 0x7fcfdc0cfdb0 2024-09-19T07:53:40.440 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-19T07:53:40.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.442+0000 7fcfe0bd8640 1 -- 172.21.15.16:0/3542840289 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcfbc06c2b0 msgr2=0x7fcfbc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:40.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.442+0000 7fcfe0bd8640 1 --2- 172.21.15.16:0/3542840289 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcfbc06c2b0 0x7fcfbc06e770 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcfc402a140 tx=0x7fcfc4040040 comp rx=0 tx=0).stop 2024-09-19T07:53:40.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.442+0000 7fcfe0bd8640 1 -- 172.21.15.16:0/3542840289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfdc0cfdb0 msgr2=0x7fcfdc13ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:40.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.442+0000 7fcfe0bd8640 1 --2- 172.21.15.16:0/3542840289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfdc0cfdb0 0x7fcfdc13ebd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00e8a0 tx=0x7fcfcc00ed60 comp rx=0 tx=0).stop 2024-09-19T07:53:40.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.442+0000 7fcfe0bd8640 1 -- 172.21.15.16:0/3542840289 shutdown_connections 2024-09-19T07:53:40.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.442+0000 7fcfe0bd8640 1 --2- 172.21.15.16:0/3542840289 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcfbc06c2b0 0x7fcfbc06e770 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.442+0000 7fcfe0bd8640 1 --2- 172.21.15.16:0/3542840289 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcfdc15cee0 0x7fcfdc15eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.442+0000 7fcfe0bd8640 1 --2- 172.21.15.16:0/3542840289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfdc15cb00 0x7fcfdc15e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.442+0000 7fcfe0bd8640 1 --2- 172.21.15.16:0/3542840289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfdc0cfdb0 0x7fcfdc13ebd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.442+0000 7fcfe0bd8640 1 -- 172.21.15.16:0/3542840289 >> 172.21.15.16:0/3542840289 conn(0x7fcfdc067fe0 msgr2=0x7fcfdc15b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:40.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.443+0000 7fcfe0bd8640 1 -- 172.21.15.16:0/3542840289 shutdown_connections 2024-09-19T07:53:40.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.443+0000 7fcfe0bd8640 1 -- 172.21.15.16:0/3542840289 wait complete. 2024-09-19T07:53:40.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T07:53:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.565+0000 7fd1cf442640 1 Processor -- start 2024-09-19T07:53:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.566+0000 7fd1cf442640 1 -- start start 2024-09-19T07:53:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.566+0000 7fd1cf442640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd1c815f0d0 0x7fd1c815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.566+0000 7fd1cf442640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd1c80d0070 0x7fd1c80d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.566+0000 7fd1cf442640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1c80cf2b0 con 0x7fd1c80d1940 2024-09-19T07:53:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.566+0000 7fd1cf442640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd1c80cf590 con 0x7fd1c80d0070 2024-09-19T07:53:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.566+0000 7fd1cf442640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fd1c80cf870 con 0x7fd1c815f0d0 2024-09-19T07:53:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.567+0000 7fd1cc9dd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd1c80d0070 0x7fd1c80d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.567+0000 7fd1cd9df640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd1c80d1940 0x7fd1c80d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40146/0 (socket says 172.21.15.16:40146) 2024-09-19T07:53:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.567+0000 7fd1cc9dd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd1c80d0070 0x7fd1c80d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39062/0 (socket says 172.21.15.16:39062) 2024-09-19T07:53:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.567+0000 7fd1cd9df640 1 -- 172.21.15.16:0/1996507716 learned_addr learned my addr 172.21.15.16:0/1996507716 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.567+0000 7fd1cd1de640 1 --2- 172.21.15.16:0/1996507716 >> v2:172.21.15.146:3300/0 conn(0x7fd1c815f0d0 0x7fd1c815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.567+0000 7fd1cc9dd640 1 -- 172.21.15.16:0/1996507716 >> v2:172.21.15.146:3300/0 conn(0x7fd1c815f0d0 msgr2=0x7fd1c815f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.567+0000 7fd1cc9dd640 1 --2- 172.21.15.16:0/1996507716 >> v2:172.21.15.146:3300/0 conn(0x7fd1c815f0d0 0x7fd1c815f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.567+0000 7fd1cc9dd640 1 -- 172.21.15.16:0/1996507716 >> v1:172.21.15.16:6789/0 conn(0x7fd1c80d1940 legacy=0x7fd1c80d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.567+0000 7fd1cc9dd640 1 -- 172.21.15.16:0/1996507716 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c80cfb50 con 0x7fd1c80d0070 2024-09-19T07:53:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.568+0000 7fd1cc9dd640 1 --2- 172.21.15.16:0/1996507716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c80d0070 0x7fd1c80d1400 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80b6f40 tx=0x7fd1bc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43a7f1e67f7ce51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.568+0000 7fd1c7fff640 1 -- 172.21.15.16:0/1996507716 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc037070 con 0x7fd1c80d0070 2024-09-19T07:53:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.568+0000 7fd1c7fff640 1 -- 172.21.15.16:0/1996507716 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1bc032440 con 0x7fd1c80d0070 2024-09-19T07:53:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.568+0000 7fd1c7fff640 1 -- 172.21.15.16:0/1996507716 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc02ae60 con 0x7fd1c80d0070 2024-09-19T07:53:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.568+0000 7fd1cf442640 1 -- 172.21.15.16:0/1996507716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c80d0070 msgr2=0x7fd1c80d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:40.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.568+0000 7fd1cf442640 1 --2- 172.21.15.16:0/1996507716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c80d0070 0x7fd1c80d1400 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80b6f40 tx=0x7fd1bc02a110 comp rx=0 tx=0).stop 2024-09-19T07:53:40.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.568+0000 7fd1cf442640 1 -- 172.21.15.16:0/1996507716 shutdown_connections 2024-09-19T07:53:40.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.568+0000 7fd1cf442640 1 --2- 172.21.15.16:0/1996507716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c80d0070 0x7fd1c80d1400 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.568+0000 7fd1cf442640 1 --2- 172.21.15.16:0/1996507716 >> v2:172.21.15.146:3300/0 conn(0x7fd1c815f0d0 0x7fd1c815f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.568+0000 7fd1cf442640 1 -- 172.21.15.16:0/1996507716 >> 172.21.15.16:0/1996507716 conn(0x7fd1c809d760 msgr2=0x7fd1c808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:40.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.569+0000 7fd1cf442640 1 -- 172.21.15.16:0/1996507716 shutdown_connections 2024-09-19T07:53:40.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.569+0000 7fd1cf442640 1 -- 172.21.15.16:0/1996507716 wait complete. 2024-09-19T07:53:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.570+0000 7fd1cf442640 1 Processor -- start 2024-09-19T07:53:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.570+0000 7fd1cf442640 1 -- start start 2024-09-19T07:53:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.570+0000 7fd1cf442640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1c80d0070 0x7fd1c80b9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1cf442640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1c80d1940 0x7fd1c81e9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1cf442640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c815f0d0 0x7fd1c81ea470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1cf442640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c80d49c0 con 0x7fd1c815f0d0 2024-09-19T07:53:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1cf442640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c80d4840 con 0x7fd1c80d1940 2024-09-19T07:53:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1cf442640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd1c80d4b40 con 0x7fd1c80d0070 2024-09-19T07:53:40.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1cd9df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c815f0d0 0x7fd1c81ea470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:40.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1cd9df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c815f0d0 0x7fd1c81ea470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39070/0 (socket says 172.21.15.16:39070) 2024-09-19T07:53:40.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1cd9df640 1 -- 172.21.15.16:0/3605560767 learned_addr learned my addr 172.21.15.16:0/3605560767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:40.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1cd1de640 1 --2- 172.21.15.16:0/3605560767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1c80d0070 0x7fd1c80b9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:40.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1cc9dd640 1 --2- 172.21.15.16:0/3605560767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1c80d1940 0x7fd1c81e9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:40.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1cd9df640 1 -- 172.21.15.16:0/3605560767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1c80d0070 msgr2=0x7fd1c80b9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:40.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1cd9df640 1 --2- 172.21.15.16:0/3605560767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1c80d0070 0x7fd1c80b9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1cd9df640 1 -- 172.21.15.16:0/3605560767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1c80d1940 msgr2=0x7fd1c81e9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:40.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1cd9df640 1 --2- 172.21.15.16:0/3605560767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1c80d1940 0x7fd1c81e9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1cd9df640 1 -- 172.21.15.16:0/3605560767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c80cb4b0 con 0x7fd1c815f0d0 2024-09-19T07:53:40.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1cd9df640 1 --2- 172.21.15.16:0/3605560767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c815f0d0 0x7fd1c81ea470 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8002230 tx=0x7fd1b800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:40.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1c5ffb640 1 -- 172.21.15.16:0/3605560767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8017070 con 0x7fd1c815f0d0 2024-09-19T07:53:40.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1c5ffb640 1 -- 172.21.15.16:0/3605560767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b800ac30 con 0x7fd1c815f0d0 2024-09-19T07:53:40.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1cf442640 1 -- 172.21.15.16:0/3605560767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c80cb790 con 0x7fd1c815f0d0 2024-09-19T07:53:40.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.571+0000 7fd1c5ffb640 1 -- 172.21.15.16:0/3605560767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8012400 con 0x7fd1c815f0d0 2024-09-19T07:53:40.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.572+0000 7fd1cf442640 1 -- 172.21.15.16:0/3605560767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c80cbd50 con 0x7fd1c815f0d0 2024-09-19T07:53:40.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.573+0000 7fd1cf442640 1 -- 172.21.15.16:0/3605560767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c80d1400 con 0x7fd1c815f0d0 2024-09-19T07:53:40.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.573+0000 7fd1c5ffb640 1 -- 172.21.15.16:0/3605560767 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd1b801a070 con 0x7fd1c815f0d0 2024-09-19T07:53:40.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.573+0000 7fd1c5ffb640 1 --2- 172.21.15.16:0/3605560767 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd1b406c2b0 0x7fd1b406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:40.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.574+0000 7fd1c5ffb640 1 -- 172.21.15.16:0/3605560767 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5249+0+0 (secure 0 0 0) 0x7fd1b808adf0 con 0x7fd1c815f0d0 2024-09-19T07:53:40.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.574+0000 7fd1cd1de640 1 --2- 172.21.15.16:0/3605560767 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd1b406c2b0 0x7fd1b406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:40.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.574+0000 7fd1cd1de640 1 --2- 172.21.15.16:0/3605560767 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd1b406c2b0 0x7fd1b406e770 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd1c81eab50 tx=0x7fd1c0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:40.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.578+0000 7fd1c5ffb640 1 -- 172.21.15.16:0/3605560767 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd1b8059160 con 0x7fd1c815f0d0 2024-09-19T07:53:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.822+0000 7fd1cf442640 1 -- 172.21.15.16:0/3605560767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd1c815e2b0 con 0x7fd1c815f0d0 2024-09-19T07:53:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.822+0000 7fd1c5ffb640 1 -- 172.21.15.16:0/3605560767 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7fd1b805d460 con 0x7fd1c815f0d0 2024-09-19T07:53:40.822 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:40.823 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:53:40.823 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-19T07:53:40.823 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:40.823 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-19T07:53:40.823 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etms80KDxAAqTrG88LTEJIj9TWJu0witQ==", 2024-09-19T07:53:40.823 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:53:40.823 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:53:40.823 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:53:40.823 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:53:40.823 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:53:40.823 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-19T07:53:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etm4T0KGxAAcfDOKrSdKzhJ4xvTFIyxcQ==", 2024-09-19T07:53:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:53:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:53:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:53:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:53:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:53:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-19T07:53:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNDBGFhAAtBioJJ1+NtIDLhd/Hg7aZA==", 2024-09-19T07:53:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:53:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:53:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:53:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:53:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:53:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-19T07:53:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmaHEiIhAACqqjCRMV+xlGh9WqYsj+MA==", 2024-09-19T07:53:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:53:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:53:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:53:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:53:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:53:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-19T07:53:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDT1etm09plCxAAGvJYF6JrKw49FtXb71ox3A==", 2024-09-19T07:53:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:53:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:53:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:53:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:53:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-19T07:53:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDW1etmGy8yHRAAlcDsHLnkjtS6sCCR8Olv2A==", 2024-09-19T07:53:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:53:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:53:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:53:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:53:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-19T07:53:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDZ1etmMhGTHxAATQaW2kgJhHKftutQQtUa6A==", 2024-09-19T07:53:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:53:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:53:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:53:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:53:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-19T07:53:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDc1etmD3sLIhAAIdY00buvb3bIFA5ZpgxxjA==", 2024-09-19T07:53:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:53:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:53:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:53:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:53:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-19T07:53:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDi1etmgutcLRAAHJEl5SfQ7N0T4YoYeBfrQg==", 2024-09-19T07:53:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:53:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:53:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:53:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:53:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-19T07:53:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDm1etmCV3xARAA+5yhQEmIpyNId3RL1Xsxhw==", 2024-09-19T07:53:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:53:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:53:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:53:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:53:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-19T07:53:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDp1etm9XC+AxAA+Lrs0ExfSKaixgOxfX8Nug==", 2024-09-19T07:53:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:53:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:53:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:53:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:53:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-19T07:53:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDs1etmhmvMBhAAhvW/udIE2gCd8IrJCSFuvw==", 2024-09-19T07:53:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:53:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:53:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:53:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:53:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-19T07:53:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA==", 2024-09-19T07:53:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:53:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:53:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-19T07:53:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T07:53:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-19T07:53:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmc05KLxAAPQl6wOHqFIf8LVxskiuLVg==", 2024-09-19T07:53:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:53:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:53:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:53:40.833 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:53:40.833 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:53:40.833 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.833 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:40.833 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:40.833 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-19T07:53:40.833 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmjFsCLBAANa7gCMvDw84Ovo7u6RRJ3Q==", 2024-09-19T07:53:40.833 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:53:40.833 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:53:40.833 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:53:40.833 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:53:40.833 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:53:40.833 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:40.834 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:40.834 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-19T07:53:40.834 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmdUbgNBAAfwstyOeeoZfetgVZu4z5xg==", 2024-09-19T07:53:40.834 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:53:40.834 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:53:40.834 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:53:40.834 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:53:40.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:53:40.834 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:40.834 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:40.834 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-19T07:53:40.835 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDQ1etm9/vTLhAA9PRfAOA8R4oXp3x0dU8l3g==", 2024-09-19T07:53:40.835 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:53:40.835 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:53:40.835 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:53:40.835 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:53:40.835 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:53:40.835 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.835 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:40.835 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:40.835 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-19T07:53:40.835 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNQ68CxAANkK+D3u/oTEz8Su4iuZrCg==", 2024-09-19T07:53:40.835 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:53:40.835 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:53:40.835 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:53:40.836 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:53:40.836 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:40.836 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:40.836 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-19T07:53:40.836 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmp+gSCBAAGXzmKfFNd6K2fWeqYhlHAg==", 2024-09-19T07:53:40.836 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:53:40.836 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:53:40.836 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:53:40.836 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:53:40.836 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.836 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:40.836 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T07:53:40.837 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:53:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.824+0000 7fd1cf442640 1 -- 172.21.15.16:0/3605560767 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd1b406c2b0 msgr2=0x7fd1b406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.824+0000 7fd1cf442640 1 --2- 172.21.15.16:0/3605560767 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd1b406c2b0 0x7fd1b406e770 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd1c81eab50 tx=0x7fd1c0008040 comp rx=0 tx=0).stop 2024-09-19T07:53:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.824+0000 7fd1cf442640 1 -- 172.21.15.16:0/3605560767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c815f0d0 msgr2=0x7fd1c81ea470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.824+0000 7fd1cf442640 1 --2- 172.21.15.16:0/3605560767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c815f0d0 0x7fd1c81ea470 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8002230 tx=0x7fd1b800cf70 comp rx=0 tx=0).stop 2024-09-19T07:53:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.824+0000 7fd1cf442640 1 -- 172.21.15.16:0/3605560767 shutdown_connections 2024-09-19T07:53:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.824+0000 7fd1cf442640 1 --2- 172.21.15.16:0/3605560767 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd1b406c2b0 0x7fd1b406e770 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.824+0000 7fd1cf442640 1 --2- 172.21.15.16:0/3605560767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1c815f0d0 0x7fd1c81ea470 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.824+0000 7fd1cf442640 1 --2- 172.21.15.16:0/3605560767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1c80d1940 0x7fd1c81e9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.824+0000 7fd1cf442640 1 --2- 172.21.15.16:0/3605560767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1c80d0070 0x7fd1c80b9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.824+0000 7fd1cf442640 1 -- 172.21.15.16:0/3605560767 >> 172.21.15.16:0/3605560767 conn(0x7fd1c809d760 msgr2=0x7fd1c808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:40.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.824+0000 7fd1cf442640 1 -- 172.21.15.16:0/3605560767 shutdown_connections 2024-09-19T07:53:40.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.824+0000 7fd1cf442640 1 -- 172.21.15.16:0/3605560767 wait complete. 2024-09-19T07:53:40.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T07:53:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.941+0000 7f99d9637640 1 Processor -- start 2024-09-19T07:53:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.941+0000 7f99d9637640 1 -- start start 2024-09-19T07:53:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.942+0000 7f99d9637640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99d415bee0 0x7f99d40d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.942+0000 7f99d9637640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99d415c2c0 0x7f99d40d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.942+0000 7f99d9637640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f99d40cf200 0x7f99d40cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.942+0000 7f99d9637640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f99d40ceaf0 con 0x7f99d415c2c0 2024-09-19T07:53:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.942+0000 7f99d9637640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f99d40cdee0 con 0x7f99d40cf200 2024-09-19T07:53:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.942+0000 7f99d9637640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f99d40ce1c0 con 0x7f99d415bee0 2024-09-19T07:53:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.942+0000 7f99d27fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99d415c2c0 0x7f99d40d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.942+0000 7f99d27fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99d415c2c0 0x7f99d40d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39078/0 (socket says 172.21.15.16:39078) 2024-09-19T07:53:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.942+0000 7f99d27fc640 1 -- 172.21.15.16:0/106059777 learned_addr learned my addr 172.21.15.16:0/106059777 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.942+0000 7f99d2ffd640 1 --2- 172.21.15.16:0/106059777 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99d415bee0 0x7f99d40d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.942+0000 7f99d37fe640 1 --2- 172.21.15.16:0/106059777 >> v2:172.21.15.146:3300/0 conn(0x7f99d40cf200 0x7f99d40cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.942+0000 7f99d27fc640 1 -- 172.21.15.16:0/106059777 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99d415bee0 msgr2=0x7f99d40d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.942+0000 7f99d27fc640 1 --2- 172.21.15.16:0/106059777 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99d415bee0 0x7f99d40d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.942+0000 7f99d27fc640 1 -- 172.21.15.16:0/106059777 >> v2:172.21.15.146:3300/0 conn(0x7f99d40cf200 msgr2=0x7f99d40cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.942+0000 7f99d27fc640 1 --2- 172.21.15.16:0/106059777 >> v2:172.21.15.146:3300/0 conn(0x7f99d40cf200 0x7f99d40cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.942+0000 7f99d27fc640 1 -- 172.21.15.16:0/106059777 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d40ce4a0 con 0x7f99d415c2c0 2024-09-19T07:53:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.943+0000 7f99d27fc640 1 --2- 172.21.15.16:0/106059777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99d415c2c0 0x7f99d40d09f0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f99d40af910 tx=0x7f99c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21dc1f556ddc76c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.943+0000 7f99d1ffb640 1 -- 172.21.15.16:0/106059777 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8037070 con 0x7f99d415c2c0 2024-09-19T07:53:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.943+0000 7f99d1ffb640 1 -- 172.21.15.16:0/106059777 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99c8032440 con 0x7f99d415c2c0 2024-09-19T07:53:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.943+0000 7f99d1ffb640 1 -- 172.21.15.16:0/106059777 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8032810 con 0x7f99d415c2c0 2024-09-19T07:53:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.944+0000 7f99d9637640 1 -- 172.21.15.16:0/106059777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99d415c2c0 msgr2=0x7f99d40d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.944+0000 7f99d9637640 1 --2- 172.21.15.16:0/106059777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99d415c2c0 0x7f99d40d09f0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f99d40af910 tx=0x7f99c802a1e0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.944+0000 7f99d9637640 1 -- 172.21.15.16:0/106059777 shutdown_connections 2024-09-19T07:53:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.944+0000 7f99d9637640 1 --2- 172.21.15.16:0/106059777 >> v2:172.21.15.146:3300/0 conn(0x7f99d40cf200 0x7f99d40cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.944+0000 7f99d9637640 1 --2- 172.21.15.16:0/106059777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99d415c2c0 0x7f99d40d09f0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.944+0000 7f99d9637640 1 --2- 172.21.15.16:0/106059777 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99d415bee0 0x7f99d40d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.944+0000 7f99d9637640 1 -- 172.21.15.16:0/106059777 >> 172.21.15.16:0/106059777 conn(0x7f99d406bdf0 msgr2=0x7f99d406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.944+0000 7f99d9637640 1 -- 172.21.15.16:0/106059777 shutdown_connections 2024-09-19T07:53:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.944+0000 7f99d9637640 1 -- 172.21.15.16:0/106059777 wait complete. 2024-09-19T07:53:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.945+0000 7f99d9637640 1 Processor -- start 2024-09-19T07:53:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.945+0000 7f99d9637640 1 -- start start 2024-09-19T07:53:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.946+0000 7f99d9637640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99d40cf200 0x7f99d4157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.946+0000 7f99d9637640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99d415bee0 0x7f99d41f2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.946+0000 7f99d9637640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99d415c2c0 0x7f99d40c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.946+0000 7f99d9637640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f99d415fd40 con 0x7f99d40cf200 2024-09-19T07:53:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.946+0000 7f99d9637640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f99d40cfbf0 con 0x7f99d415c2c0 2024-09-19T07:53:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.946+0000 7f99d9637640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f99d415fec0 con 0x7f99d415bee0 2024-09-19T07:53:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.946+0000 7f99d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99d40cf200 0x7f99d4157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.946+0000 7f99d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99d40cf200 0x7f99d4157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39094/0 (socket says 172.21.15.16:39094) 2024-09-19T07:53:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.946+0000 7f99d2ffd640 1 -- 172.21.15.16:0/2467779096 learned_addr learned my addr 172.21.15.16:0/2467779096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.946+0000 7f99d2ffd640 1 -- 172.21.15.16:0/2467779096 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99d415bee0 msgr2=0x7f99d41f2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.946+0000 7f99d27fc640 1 --2- 172.21.15.16:0/2467779096 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99d415bee0 0x7f99d41f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.946+0000 7f99d37fe640 1 --2- 172.21.15.16:0/2467779096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99d415c2c0 0x7f99d40c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.946+0000 7f99d2ffd640 1 --2- 172.21.15.16:0/2467779096 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99d415bee0 0x7f99d41f2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.946+0000 7f99d2ffd640 1 -- 172.21.15.16:0/2467779096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99d415c2c0 msgr2=0x7f99d40c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.946+0000 7f99d2ffd640 1 --2- 172.21.15.16:0/2467779096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99d415c2c0 0x7f99d40c9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.946+0000 7f99d2ffd640 1 -- 172.21.15.16:0/2467779096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d41f2a90 con 0x7f99d40cf200 2024-09-19T07:53:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.946+0000 7f99d27fc640 1 --2- 172.21.15.16:0/2467779096 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99d415bee0 0x7f99d41f2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.946+0000 7f99d37fe640 1 --2- 172.21.15.16:0/2467779096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99d415c2c0 0x7f99d40c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.946+0000 7f99d2ffd640 1 --2- 172.21.15.16:0/2467779096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99d40cf200 0x7f99d4157910 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f99c400e9e0 tx=0x7f99c400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.947+0000 7f99bb7fe640 1 -- 172.21.15.16:0/2467779096 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4017070 con 0x7f99d40cf200 2024-09-19T07:53:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.947+0000 7f99bb7fe640 1 -- 172.21.15.16:0/2467779096 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99c400c9d0 con 0x7f99d40cf200 2024-09-19T07:53:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.947+0000 7f99d9637640 1 -- 172.21.15.16:0/2467779096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d40c7520 con 0x7f99d40cf200 2024-09-19T07:53:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.947+0000 7f99bb7fe640 1 -- 172.21.15.16:0/2467779096 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c400cdc0 con 0x7f99d40cf200 2024-09-19T07:53:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.947+0000 7f99d9637640 1 -- 172.21.15.16:0/2467779096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d4071f40 con 0x7f99d40cf200 2024-09-19T07:53:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.948+0000 7f99d9637640 1 -- 172.21.15.16:0/2467779096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99d4071d60 con 0x7f99d40cf200 2024-09-19T07:53:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.948+0000 7f99bb7fe640 1 -- 172.21.15.16:0/2467779096 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f99c401a070 con 0x7f99d40cf200 2024-09-19T07:53:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.949+0000 7f99bb7fe640 1 --2- 172.21.15.16:0/2467779096 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f99c006c2b0 0x7f99c006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.949+0000 7f99bb7fe640 1 -- 172.21.15.16:0/2467779096 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5249+0+0 (secure 0 0 0) 0x7f99c408b1a0 con 0x7f99d40cf200 2024-09-19T07:53:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.949+0000 7f99d27fc640 1 --2- 172.21.15.16:0/2467779096 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f99c006c2b0 0x7f99c006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.949+0000 7f99d27fc640 1 --2- 172.21.15.16:0/2467779096 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f99c006c2b0 0x7f99c006e770 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f99c8034000 tx=0x7f99c802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:40.953+0000 7f99bb7fe640 1 -- 172.21.15.16:0/2467779096 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f99d4071d60 con 0x7f99d40cf200 2024-09-19T07:53:41.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.131+0000 7f99d9637640 1 -- 172.21.15.16:0/2467779096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f99d40cf6b0 con 0x7f99d40cf200 2024-09-19T07:53:41.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.132+0000 7f99bb7fe640 1 -- 172.21.15.16:0/2467779096 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v114) ==== 84+0+65 (secure 0 0 0) 0x7f99d41f2a90 con 0x7f99d40cf200 2024-09-19T07:53:41.132 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:41.132 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-19T07:53:41.133 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:41.133 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-19T07:53:41.133 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-19T07:53:41.133 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:41.133 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-19T07:53:41.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.134+0000 7f99d9637640 1 -- 172.21.15.16:0/2467779096 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f99c006c2b0 msgr2=0x7f99c006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:41.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.134+0000 7f99d9637640 1 --2- 172.21.15.16:0/2467779096 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f99c006c2b0 0x7f99c006e770 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f99c8034000 tx=0x7f99c802e000 comp rx=0 tx=0).stop 2024-09-19T07:53:41.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.134+0000 7f99d9637640 1 -- 172.21.15.16:0/2467779096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99d40cf200 msgr2=0x7f99d4157910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.134+0000 7f99d9637640 1 --2- 172.21.15.16:0/2467779096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99d40cf200 0x7f99d4157910 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f99c400e9e0 tx=0x7f99c400eea0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.134+0000 7f99d9637640 1 -- 172.21.15.16:0/2467779096 shutdown_connections 2024-09-19T07:53:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.134+0000 7f99d9637640 1 --2- 172.21.15.16:0/2467779096 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f99c006c2b0 0x7f99c006e770 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.134+0000 7f99d9637640 1 --2- 172.21.15.16:0/2467779096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99d415c2c0 0x7f99d40c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.134+0000 7f99d9637640 1 --2- 172.21.15.16:0/2467779096 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99d415bee0 0x7f99d41f2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.134+0000 7f99d9637640 1 --2- 172.21.15.16:0/2467779096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99d40cf200 0x7f99d4157910 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.134+0000 7f99d9637640 1 -- 172.21.15.16:0/2467779096 >> 172.21.15.16:0/2467779096 conn(0x7f99d406bdf0 msgr2=0x7f99d406c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.134+0000 7f99d9637640 1 -- 172.21.15.16:0/2467779096 shutdown_connections 2024-09-19T07:53:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.134+0000 7f99d9637640 1 -- 172.21.15.16:0/2467779096 wait complete. 2024-09-19T07:53:41.145 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T07:53:41.145 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T07:53:41.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.252+0000 7f6bef806640 1 Processor -- start 2024-09-19T07:53:41.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.252+0000 7f6bef806640 1 -- start start 2024-09-19T07:53:41.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.253+0000 7f6bef806640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6be815f0d0 0x7f6be815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.253+0000 7f6bef806640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6be80d4130 0x7f6be80d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.253+0000 7f6bef806640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6be80d4ae0 con 0x7f6be80d0070 2024-09-19T07:53:41.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.253+0000 7f6bef806640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f6be80d1b10 con 0x7f6be815f0d0 2024-09-19T07:53:41.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.253+0000 7f6bef806640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6be80cf2b0 con 0x7f6be80d4130 2024-09-19T07:53:41.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.253+0000 7f6bedda3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6be80d4130 0x7f6be80d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.253+0000 7f6bed5a2640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6be815f0d0 0x7f6be815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.253+0000 7f6becda1640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f6be80d0070 0x7f6be80d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:37032/0 (socket says 172.21.15.16:37032) 2024-09-19T07:53:41.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.254+0000 7f6becda1640 1 -- 172.21.15.16:0/2859439705 learned_addr learned my addr 172.21.15.16:0/2859439705 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:41.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.254+0000 7f6be7fff640 1 -- 172.21.15.16:0/2859439705 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 30206290 0 0) 0x7f6be80d4ae0 con 0x7f6be80d0070 2024-09-19T07:53:41.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.254+0000 7f6be7fff640 1 -- 172.21.15.16:0/2859439705 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6bd0003520 con 0x7f6be80d0070 2024-09-19T07:53:41.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.254+0000 7f6bedda3640 1 -- 172.21.15.16:0/2859439705 >> v2:172.21.15.146:3300/0 conn(0x7f6be815f0d0 msgr2=0x7f6be815f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:41.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.254+0000 7f6bedda3640 1 --2- 172.21.15.16:0/2859439705 >> v2:172.21.15.146:3300/0 conn(0x7f6be815f0d0 0x7f6be815f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.254+0000 7f6bedda3640 1 -- 172.21.15.16:0/2859439705 >> v1:172.21.15.146:6789/0 conn(0x7f6be80d0070 legacy=0x7f6be80d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:41.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.254+0000 7f6bedda3640 1 -- 172.21.15.16:0/2859439705 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be80cf590 con 0x7f6be80d4130 2024-09-19T07:53:41.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.254+0000 7f6bed5a2640 1 --2- 172.21.15.16:0/2859439705 >> v2:172.21.15.146:3300/0 conn(0x7f6be815f0d0 0x7f6be815f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:53:41.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.255+0000 7f6bedda3640 1 --2- 172.21.15.16:0/2859439705 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6be80d4130 0x7f6be80d45a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6be00097a0 tx=0x7f6be002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b899a2f3faef942 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:41.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.255+0000 7f6be7fff640 1 -- 172.21.15.16:0/2859439705 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0037070 con 0x7f6be80d4130 2024-09-19T07:53:41.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.255+0000 7f6be7fff640 1 -- 172.21.15.16:0/2859439705 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6be002ae50 con 0x7f6be80d4130 2024-09-19T07:53:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.255+0000 7f6be7fff640 1 -- 172.21.15.16:0/2859439705 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0032640 con 0x7f6be80d4130 2024-09-19T07:53:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.255+0000 7f6bef806640 1 -- 172.21.15.16:0/2859439705 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6be80d4130 msgr2=0x7f6be80d45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.255+0000 7f6bef806640 1 --2- 172.21.15.16:0/2859439705 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6be80d4130 0x7f6be80d45a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6be00097a0 tx=0x7f6be002a100 comp rx=0 tx=0).stop 2024-09-19T07:53:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.255+0000 7f6bef806640 1 -- 172.21.15.16:0/2859439705 shutdown_connections 2024-09-19T07:53:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.255+0000 7f6bef806640 1 --2- 172.21.15.16:0/2859439705 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6be80d4130 0x7f6be80d45a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.255+0000 7f6bef806640 1 --2- 172.21.15.16:0/2859439705 >> v2:172.21.15.146:3300/0 conn(0x7f6be815f0d0 0x7f6be815f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.255+0000 7f6bef806640 1 -- 172.21.15.16:0/2859439705 >> 172.21.15.16:0/2859439705 conn(0x7f6be809d760 msgr2=0x7f6be808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.256+0000 7f6bef806640 1 -- 172.21.15.16:0/2859439705 shutdown_connections 2024-09-19T07:53:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.256+0000 7f6bef806640 1 -- 172.21.15.16:0/2859439705 wait complete. 2024-09-19T07:53:41.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.257+0000 7f6bef806640 1 Processor -- start 2024-09-19T07:53:41.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.257+0000 7f6bef806640 1 -- start start 2024-09-19T07:53:41.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.257+0000 7f6bef806640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6be80d0070 0x7f6be815f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.257+0000 7f6bef806640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6be80d4130 0x7f6be8151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.257+0000 7f6bef806640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be815f0d0 0x7f6be80b98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.257+0000 7f6bef806640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6be80cef60 con 0x7f6be815f0d0 2024-09-19T07:53:41.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.257+0000 7f6bef806640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6be80cfb50 con 0x7f6be80d0070 2024-09-19T07:53:41.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.257+0000 7f6bef806640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6be80b2f70 con 0x7f6be80d4130 2024-09-19T07:53:41.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.258+0000 7f6bedda3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be815f0d0 0x7f6be80b98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.258+0000 7f6bedda3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be815f0d0 0x7f6be80b98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39100/0 (socket says 172.21.15.16:39100) 2024-09-19T07:53:41.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.258+0000 7f6bedda3640 1 -- 172.21.15.16:0/177350198 learned_addr learned my addr 172.21.15.16:0/177350198 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:41.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.258+0000 7f6becda1640 1 --2- 172.21.15.16:0/177350198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6be80d4130 0x7f6be8151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.258+0000 7f6bed5a2640 1 --2- 172.21.15.16:0/177350198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6be80d0070 0x7f6be815f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.258+0000 7f6bedda3640 1 -- 172.21.15.16:0/177350198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6be80d4130 msgr2=0x7f6be8151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:41.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.258+0000 7f6bedda3640 1 --2- 172.21.15.16:0/177350198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6be80d4130 0x7f6be8151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.258+0000 7f6bedda3640 1 -- 172.21.15.16:0/177350198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6be80d0070 msgr2=0x7f6be815f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:41.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.258+0000 7f6bedda3640 1 --2- 172.21.15.16:0/177350198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6be80d0070 0x7f6be815f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.258+0000 7f6bedda3640 1 -- 172.21.15.16:0/177350198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be80ba2b0 con 0x7f6be815f0d0 2024-09-19T07:53:41.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.258+0000 7f6bedda3640 1 --2- 172.21.15.16:0/177350198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be815f0d0 0x7f6be80b98a0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6be0035040 tx=0x7f6be00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:41.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.258+0000 7f6be5ffb640 1 -- 172.21.15.16:0/177350198 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0037050 con 0x7f6be815f0d0 2024-09-19T07:53:41.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.258+0000 7f6be5ffb640 1 -- 172.21.15.16:0/177350198 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6be0043070 con 0x7f6be815f0d0 2024-09-19T07:53:41.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.258+0000 7f6bef806640 1 -- 172.21.15.16:0/177350198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be80cb5e0 con 0x7f6be815f0d0 2024-09-19T07:53:41.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.258+0000 7f6be5ffb640 1 -- 172.21.15.16:0/177350198 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be00044e0 con 0x7f6be815f0d0 2024-09-19T07:53:41.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.259+0000 7f6bef806640 1 -- 172.21.15.16:0/177350198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be80cbba0 con 0x7f6be815f0d0 2024-09-19T07:53:41.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.260+0000 7f6bef806640 1 -- 172.21.15.16:0/177350198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be815f4b0 con 0x7f6be815f0d0 2024-09-19T07:53:41.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.260+0000 7f6be5ffb640 1 -- 172.21.15.16:0/177350198 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6be003e670 con 0x7f6be815f0d0 2024-09-19T07:53:41.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.261+0000 7f6be5ffb640 1 --2- 172.21.15.16:0/177350198 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6bd406c2b0 0x7f6bd406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.261+0000 7f6be5ffb640 1 -- 172.21.15.16:0/177350198 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5249+0+0 (secure 0 0 0) 0x7f6be00ab9b0 con 0x7f6be815f0d0 2024-09-19T07:53:41.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.261+0000 7f6bed5a2640 1 --2- 172.21.15.16:0/177350198 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6bd406c2b0 0x7f6bd406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.261+0000 7f6bed5a2640 1 --2- 172.21.15.16:0/177350198 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6bd406c2b0 0x7f6bd406e770 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009560 tx=0x7f6bd8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:41.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.265+0000 7f6be5ffb640 1 -- 172.21.15.16:0/177350198 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6be004d020 con 0x7f6be815f0d0 2024-09-19T07:53:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:41.443+0000 7f6bef806640 1 -- 172.21.15.16:0/177350198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6be806cf30 con 0x7f6be815f0d0 2024-09-19T07:53:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.439+0000 7f6be5ffb640 1 -- 172.21.15.16:0/177350198 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v115) ==== 138+0+0 (secure 0 0 0) 0x7f6be0079c80 con 0x7f6be815f0d0 2024-09-19T07:53:42.440 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-19T07:53:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.442+0000 7f6bef806640 1 -- 172.21.15.16:0/177350198 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6bd406c2b0 msgr2=0x7f6bd406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.442+0000 7f6bef806640 1 --2- 172.21.15.16:0/177350198 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6bd406c2b0 0x7f6bd406e770 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009560 tx=0x7f6bd8009190 comp rx=0 tx=0).stop 2024-09-19T07:53:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.442+0000 7f6bef806640 1 -- 172.21.15.16:0/177350198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be815f0d0 msgr2=0x7f6be80b98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.442+0000 7f6bef806640 1 --2- 172.21.15.16:0/177350198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be815f0d0 0x7f6be80b98a0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6be0035040 tx=0x7f6be00046c0 comp rx=0 tx=0).stop 2024-09-19T07:53:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.442+0000 7f6bef806640 1 -- 172.21.15.16:0/177350198 shutdown_connections 2024-09-19T07:53:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.442+0000 7f6bef806640 1 --2- 172.21.15.16:0/177350198 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6bd406c2b0 0x7f6bd406e770 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.442+0000 7f6bef806640 1 --2- 172.21.15.16:0/177350198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be815f0d0 0x7f6be80b98a0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.442+0000 7f6bef806640 1 --2- 172.21.15.16:0/177350198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6be80d4130 0x7f6be8151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.442+0000 7f6bef806640 1 --2- 172.21.15.16:0/177350198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6be80d0070 0x7f6be815f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.442+0000 7f6bef806640 1 -- 172.21.15.16:0/177350198 >> 172.21.15.16:0/177350198 conn(0x7f6be809d760 msgr2=0x7f6be80b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.443+0000 7f6bef806640 1 -- 172.21.15.16:0/177350198 shutdown_connections 2024-09-19T07:53:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.443+0000 7f6bef806640 1 -- 172.21.15.16:0/177350198 wait complete. 2024-09-19T07:53:42.460 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T07:53:42.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.574+0000 7f054deaf640 1 Processor -- start 2024-09-19T07:53:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.574+0000 7f054deaf640 1 -- start start 2024-09-19T07:53:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.575+0000 7f054deaf640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f054815e0a0 0x7f0548162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.575+0000 7f054deaf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05480b5280 0x7f05480d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.575+0000 7f054deaf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f054815f250 con 0x7f05480d43e0 2024-09-19T07:53:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.575+0000 7f054deaf640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f05480d4d60 con 0x7f05480b5280 2024-09-19T07:53:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.575+0000 7f054deaf640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f054815f530 con 0x7f054815e0a0 2024-09-19T07:53:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.575+0000 7f0547fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05480b5280 0x7f05480d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.575+0000 7f0546ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f05480d43e0 0x7f05480b4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35592/0 (socket says 172.21.15.16:35592) 2024-09-19T07:53:42.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.575+0000 7f0546ffd640 1 -- 172.21.15.16:0/3935562670 learned_addr learned my addr 172.21.15.16:0/3935562670 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:42.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.576+0000 7f05477fe640 1 --2- 172.21.15.16:0/3935562670 >> v2:172.21.15.146:3300/0 conn(0x7f054815e0a0 0x7f0548162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:42.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.576+0000 7f0547fff640 1 -- 172.21.15.16:0/3935562670 >> v2:172.21.15.146:3300/0 conn(0x7f054815e0a0 msgr2=0x7f0548162a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:42.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.576+0000 7f0547fff640 1 --2- 172.21.15.16:0/3935562670 >> v2:172.21.15.146:3300/0 conn(0x7f054815e0a0 0x7f0548162a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:42.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.576+0000 7f0547fff640 1 -- 172.21.15.16:0/3935562670 >> v1:172.21.15.16:6789/0 conn(0x7f05480d43e0 legacy=0x7f05480b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:42.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.576+0000 7f0547fff640 1 -- 172.21.15.16:0/3935562670 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054815f810 con 0x7f05480b5280 2024-09-19T07:53:42.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.576+0000 7f0547fff640 1 --2- 172.21.15.16:0/3935562670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05480b5280 0x7f05480d4820 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f05480d43b0 tx=0x7f054002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24526ee02e91554e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:42.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.577+0000 7f05467fc640 1 -- 172.21.15.16:0/3935562670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0540007740 con 0x7f05480b5280 2024-09-19T07:53:42.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.577+0000 7f05467fc640 1 -- 172.21.15.16:0/3935562670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05400078e0 con 0x7f05480b5280 2024-09-19T07:53:42.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.577+0000 7f05467fc640 1 -- 172.21.15.16:0/3935562670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0540007cb0 con 0x7f05480b5280 2024-09-19T07:53:42.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.577+0000 7f054deaf640 1 -- 172.21.15.16:0/3935562670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05480b5280 msgr2=0x7f05480d4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:42.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.577+0000 7f054deaf640 1 --2- 172.21.15.16:0/3935562670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05480b5280 0x7f05480d4820 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f05480d43b0 tx=0x7f054002a680 comp rx=0 tx=0).stop 2024-09-19T07:53:42.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.577+0000 7f054deaf640 1 -- 172.21.15.16:0/3935562670 shutdown_connections 2024-09-19T07:53:42.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.577+0000 7f054deaf640 1 --2- 172.21.15.16:0/3935562670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05480b5280 0x7f05480d4820 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:42.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.577+0000 7f054deaf640 1 --2- 172.21.15.16:0/3935562670 >> v2:172.21.15.146:3300/0 conn(0x7f054815e0a0 0x7f0548162a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:42.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.577+0000 7f054deaf640 1 -- 172.21.15.16:0/3935562670 >> 172.21.15.16:0/3935562670 conn(0x7f054809d770 msgr2=0x7f054808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:42.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.577+0000 7f054deaf640 1 -- 172.21.15.16:0/3935562670 shutdown_connections 2024-09-19T07:53:42.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.577+0000 7f054deaf640 1 -- 172.21.15.16:0/3935562670 wait complete. 2024-09-19T07:53:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.578+0000 7f054deaf640 1 Processor -- start 2024-09-19T07:53:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.579+0000 7f054deaf640 1 -- start start 2024-09-19T07:53:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.579+0000 7f054deaf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05480b5280 0x7f05480cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.579+0000 7f054deaf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05480d43e0 0x7f05480b9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.579+0000 7f054deaf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054815e0a0 0x7f05481f7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.579+0000 7f054deaf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f054815e600 con 0x7f054815e0a0 2024-09-19T07:53:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.579+0000 7f054deaf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f054815e480 con 0x7f05480d43e0 2024-09-19T07:53:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.579+0000 7f054deaf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f054815e780 con 0x7f05480b5280 2024-09-19T07:53:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.579+0000 7f0547fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054815e0a0 0x7f05481f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.579+0000 7f0547fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054815e0a0 0x7f05481f7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52616/0 (socket says 172.21.15.16:52616) 2024-09-19T07:53:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.580+0000 7f0547fff640 1 -- 172.21.15.16:0/552063210 learned_addr learned my addr 172.21.15.16:0/552063210 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.580+0000 7f0546ffd640 1 --2- 172.21.15.16:0/552063210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05480d43e0 0x7f05480b9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.580+0000 7f05477fe640 1 --2- 172.21.15.16:0/552063210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05480b5280 0x7f05480cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.580+0000 7f0547fff640 1 -- 172.21.15.16:0/552063210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05480b5280 msgr2=0x7f05480cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.580+0000 7f0547fff640 1 --2- 172.21.15.16:0/552063210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05480b5280 0x7f05480cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.580+0000 7f0547fff640 1 -- 172.21.15.16:0/552063210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05480d43e0 msgr2=0x7f05480b9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.580+0000 7f0547fff640 1 --2- 172.21.15.16:0/552063210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05480d43e0 0x7f05480b9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.580+0000 7f0547fff640 1 -- 172.21.15.16:0/552063210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05480cbb90 con 0x7f054815e0a0 2024-09-19T07:53:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.580+0000 7f0546ffd640 1 --2- 172.21.15.16:0/552063210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05480d43e0 0x7f05480b9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.580+0000 7f05477fe640 1 --2- 172.21.15.16:0/552063210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05480b5280 0x7f05480cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.580+0000 7f0547fff640 1 --2- 172.21.15.16:0/552063210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054815e0a0 0x7f05481f7490 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f054002ac70 tx=0x7f0540004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.580+0000 7f052ffff640 1 -- 172.21.15.16:0/552063210 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0540040070 con 0x7f054815e0a0 2024-09-19T07:53:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.580+0000 7f052ffff640 1 -- 172.21.15.16:0/552063210 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0540031c60 con 0x7f054815e0a0 2024-09-19T07:53:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.580+0000 7f054deaf640 1 -- 172.21.15.16:0/552063210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05480cbe70 con 0x7f054815e0a0 2024-09-19T07:53:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.580+0000 7f052ffff640 1 -- 172.21.15.16:0/552063210 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054003c660 con 0x7f054815e0a0 2024-09-19T07:53:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.580+0000 7f054deaf640 1 -- 172.21.15.16:0/552063210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05480cc430 con 0x7f054815e0a0 2024-09-19T07:53:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.581+0000 7f054deaf640 1 -- 172.21.15.16:0/552063210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05480d4820 con 0x7f054815e0a0 2024-09-19T07:53:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.582+0000 7f052ffff640 1 -- 172.21.15.16:0/552063210 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0540039040 con 0x7f054815e0a0 2024-09-19T07:53:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.582+0000 7f052ffff640 1 --2- 172.21.15.16:0/552063210 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f053406c2e0 0x7f053406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.582+0000 7f052ffff640 1 -- 172.21.15.16:0/552063210 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5619+0+0 (secure 0 0 0) 0x7f05400ac910 con 0x7f054815e0a0 2024-09-19T07:53:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.583+0000 7f05477fe640 1 --2- 172.21.15.16:0/552063210 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f053406c2e0 0x7f053406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.583+0000 7f05477fe640 1 --2- 172.21.15.16:0/552063210 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f053406c2e0 0x7f053406e7a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0538006fd0 tx=0x7f0538008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.587+0000 7f052ffff640 1 -- 172.21.15.16:0/552063210 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f054007b160 con 0x7f054815e0a0 2024-09-19T07:53:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:42.767+0000 7f054deaf640 1 -- 172.21.15.16:0/552063210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f054815ef90 con 0x7f054815e0a0 2024-09-19T07:53:43.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.444+0000 7f052ffff640 1 -- 172.21.15.16:0/552063210 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v116) ==== 170+0+0 (secure 0 0 0) 0x7f054007aa60 con 0x7f054815e0a0 2024-09-19T07:53:43.444 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-19T07:53:43.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.447+0000 7f054deaf640 1 -- 172.21.15.16:0/552063210 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f053406c2e0 msgr2=0x7f053406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:43.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.447+0000 7f054deaf640 1 --2- 172.21.15.16:0/552063210 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f053406c2e0 0x7f053406e7a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0538006fd0 tx=0x7f0538008040 comp rx=0 tx=0).stop 2024-09-19T07:53:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.447+0000 7f054deaf640 1 -- 172.21.15.16:0/552063210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054815e0a0 msgr2=0x7f05481f7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.447+0000 7f054deaf640 1 --2- 172.21.15.16:0/552063210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054815e0a0 0x7f05481f7490 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f054002ac70 tx=0x7f0540004a40 comp rx=0 tx=0).stop 2024-09-19T07:53:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.447+0000 7f054deaf640 1 -- 172.21.15.16:0/552063210 shutdown_connections 2024-09-19T07:53:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.447+0000 7f054deaf640 1 --2- 172.21.15.16:0/552063210 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f053406c2e0 0x7f053406e7a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.447+0000 7f054deaf640 1 --2- 172.21.15.16:0/552063210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054815e0a0 0x7f05481f7490 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.447+0000 7f054deaf640 1 --2- 172.21.15.16:0/552063210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05480d43e0 0x7f05480b9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.447+0000 7f054deaf640 1 --2- 172.21.15.16:0/552063210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05480b5280 0x7f05480cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.447+0000 7f054deaf640 1 -- 172.21.15.16:0/552063210 >> 172.21.15.16:0/552063210 conn(0x7f054809d770 msgr2=0x7f054808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.448+0000 7f054deaf640 1 -- 172.21.15.16:0/552063210 shutdown_connections 2024-09-19T07:53:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.448+0000 7f054deaf640 1 -- 172.21.15.16:0/552063210 wait complete. 2024-09-19T07:53:43.476 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T07:53:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.583+0000 7f11c4ec9640 1 Processor -- start 2024-09-19T07:53:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.583+0000 7f11c4ec9640 1 -- start start 2024-09-19T07:53:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.584+0000 7f11c4ec9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f11c015e2a0 0x7f11c006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.584+0000 7f11c4ec9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11c00d1880 con 0x7f11c006d960 2024-09-19T07:53:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.584+0000 7f11c4ec9640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11c00d1b60 con 0x7f11c015e680 2024-09-19T07:53:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.584+0000 7f11c4ec9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f11c00cf2d0 con 0x7f11c015e2a0 2024-09-19T07:53:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.585+0000 7f11be59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f11c015e2a0 0x7f11c006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.585+0000 7f11bed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f11c006d960 0x7f11c00d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35598/0 (socket says 172.21.15.16:35598) 2024-09-19T07:53:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.585+0000 7f11bed9d640 1 -- 172.21.15.16:0/3752796372 learned_addr learned my addr 172.21.15.16:0/3752796372 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.585+0000 7f11be59c640 1 -- 172.21.15.16:0/3752796372 >> v1:172.21.15.146:6789/0 conn(0x7f11c015e680 legacy=0x7f11c00cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.585+0000 7f11be59c640 1 -- 172.21.15.16:0/3752796372 >> v1:172.21.15.16:6789/0 conn(0x7f11c006d960 legacy=0x7f11c00d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.585+0000 7f11be59c640 1 -- 172.21.15.16:0/3752796372 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c00cf5b0 con 0x7f11c015e2a0 2024-09-19T07:53:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.585+0000 7f11bd59a640 1 -- 172.21.15.16:0/3752796372 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2666108664 0 0) 0x7f11c00d1880 con 0x7f11c006d960 2024-09-19T07:53:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.586+0000 7f11be59c640 1 --2- 172.21.15.16:0/3752796372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c015e2a0 0x7f11c006d420 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f11b80097a0 tx=0x7f11b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5ddabaa6125b89 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.586+0000 7f11bd59a640 1 -- 172.21.15.16:0/3752796372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8037070 con 0x7f11c015e2a0 2024-09-19T07:53:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.586+0000 7f11bd59a640 1 -- 172.21.15.16:0/3752796372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b8032440 con 0x7f11c015e2a0 2024-09-19T07:53:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.586+0000 7f11bd59a640 1 -- 172.21.15.16:0/3752796372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8032810 con 0x7f11c015e2a0 2024-09-19T07:53:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.586+0000 7f11c4ec9640 1 -- 172.21.15.16:0/3752796372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c015e2a0 msgr2=0x7f11c006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.586+0000 7f11c4ec9640 1 --2- 172.21.15.16:0/3752796372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c015e2a0 0x7f11c006d420 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f11b80097a0 tx=0x7f11b802a1e0 comp rx=0 tx=0).stop 2024-09-19T07:53:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.586+0000 7f11c4ec9640 1 -- 172.21.15.16:0/3752796372 shutdown_connections 2024-09-19T07:53:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.586+0000 7f11c4ec9640 1 --2- 172.21.15.16:0/3752796372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c015e2a0 0x7f11c006d420 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.586+0000 7f11c4ec9640 1 -- 172.21.15.16:0/3752796372 >> 172.21.15.16:0/3752796372 conn(0x7f11c009d760 msgr2=0x7f11c008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.586+0000 7f11c4ec9640 1 -- 172.21.15.16:0/3752796372 shutdown_connections 2024-09-19T07:53:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.586+0000 7f11c4ec9640 1 -- 172.21.15.16:0/3752796372 wait complete. 2024-09-19T07:53:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.588+0000 7f11c4ec9640 1 Processor -- start 2024-09-19T07:53:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.588+0000 7f11c4ec9640 1 -- start start 2024-09-19T07:53:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.588+0000 7f11c4ec9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c006d960 0x7f11c00b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11c4ec9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11c015e2a0 0x7f11c00cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11c4ec9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11c015e680 0x7f11c01f7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11c4ec9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11c00d0360 con 0x7f11c006d960 2024-09-19T07:53:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11c4ec9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f11c00d01e0 con 0x7f11c015e680 2024-09-19T07:53:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11c4ec9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f11c00d4a80 con 0x7f11c015e2a0 2024-09-19T07:53:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11be59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c006d960 0x7f11c00b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11be59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c006d960 0x7f11c00b8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52632/0 (socket says 172.21.15.16:52632) 2024-09-19T07:53:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11be59c640 1 -- 172.21.15.16:0/1761820729 learned_addr learned my addr 172.21.15.16:0/1761820729 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11bdd9b640 1 --2- 172.21.15.16:0/1761820729 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11c015e2a0 0x7f11c00cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11bed9d640 1 --2- 172.21.15.16:0/1761820729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11c015e680 0x7f11c01f7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11be59c640 1 -- 172.21.15.16:0/1761820729 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11c015e2a0 msgr2=0x7f11c00cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11be59c640 1 --2- 172.21.15.16:0/1761820729 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11c015e2a0 0x7f11c00cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11be59c640 1 -- 172.21.15.16:0/1761820729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11c015e680 msgr2=0x7f11c01f7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11be59c640 1 --2- 172.21.15.16:0/1761820729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11c015e680 0x7f11c01f7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11be59c640 1 -- 172.21.15.16:0/1761820729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c00b9350 con 0x7f11c006d960 2024-09-19T07:53:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11be59c640 1 --2- 172.21.15.16:0/1761820729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c006d960 0x7f11c00b8c70 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f11b80097a0 tx=0x7f11b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11b6ffd640 1 -- 172.21.15.16:0/1761820729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8037040 con 0x7f11c006d960 2024-09-19T07:53:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11b6ffd640 1 -- 172.21.15.16:0/1761820729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b8004250 con 0x7f11c006d960 2024-09-19T07:53:43.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11c4ec9640 1 -- 172.21.15.16:0/1761820729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c00b9630 con 0x7f11c006d960 2024-09-19T07:53:43.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11b6ffd640 1 -- 172.21.15.16:0/1761820729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8032e20 con 0x7f11c006d960 2024-09-19T07:53:43.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.589+0000 7f11c4ec9640 1 -- 172.21.15.16:0/1761820729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c01ea900 con 0x7f11c006d960 2024-09-19T07:53:43.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.590+0000 7f11c4ec9640 1 -- 172.21.15.16:0/1761820729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11c006d420 con 0x7f11c006d960 2024-09-19T07:53:43.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.591+0000 7f11b6ffd640 1 -- 172.21.15.16:0/1761820729 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f11b8030770 con 0x7f11c006d960 2024-09-19T07:53:43.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.592+0000 7f11b6ffd640 1 --2- 172.21.15.16:0/1761820729 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f11a006c2b0 0x7f11a006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:43.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.592+0000 7f11b6ffd640 1 -- 172.21.15.16:0/1761820729 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6001+0+0 (secure 0 0 0) 0x7f11b80ac740 con 0x7f11c006d960 2024-09-19T07:53:43.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.592+0000 7f11bdd9b640 1 --2- 172.21.15.16:0/1761820729 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f11a006c2b0 0x7f11a006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:43.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.593+0000 7f11bdd9b640 1 --2- 172.21.15.16:0/1761820729 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f11a006c2b0 0x7f11a006e770 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f11c00ce7e0 tx=0x7f11a8009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.597+0000 7f11b6ffd640 1 -- 172.21.15.16:0/1761820729 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f11b807ae10 con 0x7f11c006d960 2024-09-19T07:53:43.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:43.810+0000 7f11c4ec9640 1 -- 172.21.15.16:0/1761820729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f11c006cf90 con 0x7f11c006d960 2024-09-19T07:53:44.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.446+0000 7f11b6ffd640 1 -- 172.21.15.16:0/1761820729 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v61) ==== 174+0+0 (secure 0 0 0) 0x7f11b807a710 con 0x7f11c006d960 2024-09-19T07:53:44.446 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 17 and data pool 18 2024-09-19T07:53:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.450+0000 7f11c4ec9640 1 -- 172.21.15.16:0/1761820729 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f11a006c2b0 msgr2=0x7f11a006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.450+0000 7f11c4ec9640 1 --2- 172.21.15.16:0/1761820729 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f11a006c2b0 0x7f11a006e770 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f11c00ce7e0 tx=0x7f11a8009060 comp rx=0 tx=0).stop 2024-09-19T07:53:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.450+0000 7f11c4ec9640 1 -- 172.21.15.16:0/1761820729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c006d960 msgr2=0x7f11c00b8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.450+0000 7f11c4ec9640 1 --2- 172.21.15.16:0/1761820729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c006d960 0x7f11c00b8c70 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f11b80097a0 tx=0x7f11b80046c0 comp rx=0 tx=0).stop 2024-09-19T07:53:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.450+0000 7f11c4ec9640 1 -- 172.21.15.16:0/1761820729 shutdown_connections 2024-09-19T07:53:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.450+0000 7f11c4ec9640 1 --2- 172.21.15.16:0/1761820729 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f11a006c2b0 0x7f11a006e770 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.450+0000 7f11c4ec9640 1 --2- 172.21.15.16:0/1761820729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11c015e680 0x7f11c01f7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.450+0000 7f11c4ec9640 1 --2- 172.21.15.16:0/1761820729 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11c015e2a0 0x7f11c00cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.450+0000 7f11c4ec9640 1 --2- 172.21.15.16:0/1761820729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c006d960 0x7f11c00b8c70 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.450+0000 7f11c4ec9640 1 -- 172.21.15.16:0/1761820729 >> 172.21.15.16:0/1761820729 conn(0x7f11c009d760 msgr2=0x7f11c00d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.451+0000 7f11c4ec9640 1 -- 172.21.15.16:0/1761820729 shutdown_connections 2024-09-19T07:53:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.452+0000 7f11c4ec9640 1 -- 172.21.15.16:0/1761820729 wait complete. 2024-09-19T07:53:44.483 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:53:44.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.590+0000 7f75536e4640 1 Processor -- start 2024-09-19T07:53:44.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.590+0000 7f75536e4640 1 -- start start 2024-09-19T07:53:44.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.591+0000 7f75536e4640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f754c15b120 0x7f754c06f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:44.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.591+0000 7f75536e4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f754c06dec0 0x7f754c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:44.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.591+0000 7f75536e4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f754c15b500 con 0x7f754c0d04b0 2024-09-19T07:53:44.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.591+0000 7f75536e4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f754c06e940 con 0x7f754c06dec0 2024-09-19T07:53:44.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.591+0000 7f75536e4640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f754c06ec20 con 0x7f754c15b120 2024-09-19T07:53:44.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.591+0000 7f7550c7f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f754c06dec0 0x7f754c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:44.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.591+0000 7f7551c81640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f754c0d04b0 0x7f754c0c5160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35600/0 (socket says 172.21.15.16:35600) 2024-09-19T07:53:44.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.591+0000 7f7551c81640 1 -- 172.21.15.16:0/1969752028 learned_addr learned my addr 172.21.15.16:0/1969752028 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:44.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.591+0000 7f7551480640 1 --2- 172.21.15.16:0/1969752028 >> v2:172.21.15.146:3300/0 conn(0x7f754c15b120 0x7f754c06f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:44.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.591+0000 7f7550c7f640 1 -- 172.21.15.16:0/1969752028 >> v2:172.21.15.146:3300/0 conn(0x7f754c15b120 msgr2=0x7f754c06f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.591+0000 7f7550c7f640 1 --2- 172.21.15.16:0/1969752028 >> v2:172.21.15.146:3300/0 conn(0x7f754c15b120 0x7f754c06f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.591+0000 7f7550c7f640 1 -- 172.21.15.16:0/1969752028 >> v1:172.21.15.16:6789/0 conn(0x7f754c0d04b0 legacy=0x7f754c0c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.591+0000 7f7550c7f640 1 -- 172.21.15.16:0/1969752028 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c06ef00 con 0x7f754c06dec0 2024-09-19T07:53:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.591+0000 7f7543fff640 1 -- 172.21.15.16:0/1969752028 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3162902035 0 0) 0x7f754c15b500 con 0x7f754c0d04b0 2024-09-19T07:53:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.591+0000 7f7551480640 1 --2- 172.21.15.16:0/1969752028 >> v2:172.21.15.146:3300/0 conn(0x7f754c15b120 0x7f754c06f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.592+0000 7f7550c7f640 1 --2- 172.21.15.16:0/1969752028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c06dec0 0x7f754c06e330 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f754c06cb50 tx=0x7f753c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e416fb9fb173cec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.592+0000 7f7543fff640 1 -- 172.21.15.16:0/1969752028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c037070 con 0x7f754c06dec0 2024-09-19T07:53:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.592+0000 7f7543fff640 1 -- 172.21.15.16:0/1969752028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f753c032440 con 0x7f754c06dec0 2024-09-19T07:53:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.592+0000 7f7543fff640 1 -- 172.21.15.16:0/1969752028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c032810 con 0x7f754c06dec0 2024-09-19T07:53:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.593+0000 7f75536e4640 1 -- 172.21.15.16:0/1969752028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c06dec0 msgr2=0x7f754c06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.593+0000 7f75536e4640 1 --2- 172.21.15.16:0/1969752028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c06dec0 0x7f754c06e330 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f754c06cb50 tx=0x7f753c02a1a0 comp rx=0 tx=0).stop 2024-09-19T07:53:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.593+0000 7f75536e4640 1 -- 172.21.15.16:0/1969752028 shutdown_connections 2024-09-19T07:53:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.593+0000 7f75536e4640 1 --2- 172.21.15.16:0/1969752028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c06dec0 0x7f754c06e330 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.593+0000 7f75536e4640 1 --2- 172.21.15.16:0/1969752028 >> v2:172.21.15.146:3300/0 conn(0x7f754c15b120 0x7f754c06f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.593+0000 7f75536e4640 1 -- 172.21.15.16:0/1969752028 >> 172.21.15.16:0/1969752028 conn(0x7f754c06bdf0 msgr2=0x7f754c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.593+0000 7f75536e4640 1 -- 172.21.15.16:0/1969752028 shutdown_connections 2024-09-19T07:53:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.593+0000 7f75536e4640 1 -- 172.21.15.16:0/1969752028 wait complete. 2024-09-19T07:53:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.594+0000 7f75536e4640 1 Processor -- start 2024-09-19T07:53:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.594+0000 7f75536e4640 1 -- start start 2024-09-19T07:53:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.595+0000 7f75536e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c06dec0 0x7f754c1f1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.595+0000 7f75536e4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f754c0d04b0 0x7f754c0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.595+0000 7f75536e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f754c15b120 0x7f754c0cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.595+0000 7f75536e4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f754c0ce060 con 0x7f754c06dec0 2024-09-19T07:53:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.595+0000 7f75536e4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f754c06f4c0 con 0x7f754c15b120 2024-09-19T07:53:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.595+0000 7f75536e4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f754c0d0900 con 0x7f754c0d04b0 2024-09-19T07:53:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.595+0000 7f7551480640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c06dec0 0x7f754c1f1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.595+0000 7f7551480640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c06dec0 0x7f754c1f1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52654/0 (socket says 172.21.15.16:52654) 2024-09-19T07:53:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.595+0000 7f7551480640 1 -- 172.21.15.16:0/917362667 learned_addr learned my addr 172.21.15.16:0/917362667 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.595+0000 7f7551480640 1 -- 172.21.15.16:0/917362667 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f754c0d04b0 msgr2=0x7f754c0b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.595+0000 7f7551480640 1 --2- 172.21.15.16:0/917362667 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f754c0d04b0 0x7f754c0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.595+0000 7f7551c81640 1 --2- 172.21.15.16:0/917362667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f754c15b120 0x7f754c0cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.595+0000 7f7551480640 1 -- 172.21.15.16:0/917362667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f754c15b120 msgr2=0x7f754c0cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.595+0000 7f7551480640 1 --2- 172.21.15.16:0/917362667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f754c15b120 0x7f754c0cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.595+0000 7f7551480640 1 -- 172.21.15.16:0/917362667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c0b1cf0 con 0x7f754c06dec0 2024-09-19T07:53:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.596+0000 7f7551480640 1 --2- 172.21.15.16:0/917362667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c06dec0 0x7f754c1f1250 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f754800d7d0 tx=0x7f754800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.596+0000 7f7541ffb640 1 -- 172.21.15.16:0/917362667 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548004030 con 0x7f754c06dec0 2024-09-19T07:53:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.596+0000 7f7541ffb640 1 -- 172.21.15.16:0/917362667 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75480041d0 con 0x7f754c06dec0 2024-09-19T07:53:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.596+0000 7f75536e4640 1 -- 172.21.15.16:0/917362667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754c0b1fd0 con 0x7f754c06dec0 2024-09-19T07:53:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.596+0000 7f7541ffb640 1 -- 172.21.15.16:0/917362667 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754800b6c0 con 0x7f754c06dec0 2024-09-19T07:53:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.596+0000 7f75536e4640 1 -- 172.21.15.16:0/917362667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754c0b2590 con 0x7f754c06dec0 2024-09-19T07:53:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.597+0000 7f75536e4640 1 -- 172.21.15.16:0/917362667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f754c0b29d0 con 0x7f754c06dec0 2024-09-19T07:53:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.597+0000 7f7541ffb640 1 -- 172.21.15.16:0/917362667 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f754800b860 con 0x7f754c06dec0 2024-09-19T07:53:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.598+0000 7f7541ffb640 1 --2- 172.21.15.16:0/917362667 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f753006c2b0 0x7f753006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.598+0000 7f7541ffb640 1 -- 172.21.15.16:0/917362667 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6101+0+0 (secure 0 0 0) 0x7f754808b8e0 con 0x7f754c06dec0 2024-09-19T07:53:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.598+0000 7f7550c7f640 1 --2- 172.21.15.16:0/917362667 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f753006c2b0 0x7f753006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.599+0000 7f7550c7f640 1 --2- 172.21.15.16:0/917362667 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f753006c2b0 0x7f753006e770 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f753c034000 tx=0x7f753c02c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.602+0000 7f7541ffb640 1 -- 172.21.15.16:0/917362667 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7548059eb0 con 0x7f754c06dec0 2024-09-19T07:53:44.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.781+0000 7f75536e4640 1 -- 172.21.15.16:0/917362667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f754c0d0cd0 con 0x7f754c06dec0 2024-09-19T07:53:44.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.782+0000 7f7541ffb640 1 -- 172.21.15.16:0/917362667 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+17049 (secure 0 0 0) 0x7f75480597b0 con 0x7f754c06dec0 2024-09-19T07:53:44.783 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:44.783 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":117,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:53:43.811746+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:53:41.474707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-19T07:53:42.798313+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:53:44.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.784+0000 7f75536e4640 1 -- 172.21.15.16:0/917362667 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f753006c2b0 msgr2=0x7f753006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:44.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.784+0000 7f75536e4640 1 --2- 172.21.15.16:0/917362667 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f753006c2b0 0x7f753006e770 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f753c034000 tx=0x7f753c02c000 comp rx=0 tx=0).stop 2024-09-19T07:53:44.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.784+0000 7f75536e4640 1 -- 172.21.15.16:0/917362667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c06dec0 msgr2=0x7f754c1f1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:44.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.784+0000 7f75536e4640 1 --2- 172.21.15.16:0/917362667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c06dec0 0x7f754c1f1250 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f754800d7d0 tx=0x7f754800dc90 comp rx=0 tx=0).stop 2024-09-19T07:53:44.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.784+0000 7f75536e4640 1 -- 172.21.15.16:0/917362667 shutdown_connections 2024-09-19T07:53:44.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.784+0000 7f75536e4640 1 --2- 172.21.15.16:0/917362667 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f753006c2b0 0x7f753006e770 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:44.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.784+0000 7f75536e4640 1 --2- 172.21.15.16:0/917362667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f754c15b120 0x7f754c0cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:44.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.784+0000 7f75536e4640 1 --2- 172.21.15.16:0/917362667 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f754c0d04b0 0x7f754c0b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:44.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.784+0000 7f75536e4640 1 --2- 172.21.15.16:0/917362667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f754c06dec0 0x7f754c1f1250 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:44.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.785+0000 7f75536e4640 1 -- 172.21.15.16:0/917362667 >> 172.21.15.16:0/917362667 conn(0x7f754c06bdf0 msgr2=0x7f754c0b3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:44.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.785+0000 7f75536e4640 1 -- 172.21.15.16:0/917362667 shutdown_connections 2024-09-19T07:53:44.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.785+0000 7f75536e4640 1 -- 172.21.15.16:0/917362667 wait complete. 2024-09-19T07:53:44.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:53:44.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.897+0000 7fe18bf3b640 1 Processor -- start 2024-09-19T07:53:44.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.898+0000 7fe18bf3b640 1 -- start start 2024-09-19T07:53:44.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.899+0000 7fe18bf3b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe1840d0bc0 0x7fe1840c5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:44.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.899+0000 7fe18bf3b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe18415b500 con 0x7fe18415b120 2024-09-19T07:53:44.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.899+0000 7fe18bf3b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe18406e940 con 0x7fe18406f9e0 2024-09-19T07:53:44.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.899+0000 7fe18bf3b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fe18406ec20 con 0x7fe1840d0bc0 2024-09-19T07:53:44.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.899+0000 7fe18a4d8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe1840d0bc0 0x7fe1840c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:44.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.899+0000 7fe189cd7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe18415b120 0x7fe18406dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35616/0 (socket says 172.21.15.16:35616) 2024-09-19T07:53:44.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.899+0000 7fe189cd7640 1 -- 172.21.15.16:0/937100401 learned_addr learned my addr 172.21.15.16:0/937100401 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:44.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.899+0000 7fe18a4d8640 1 -- 172.21.15.16:0/937100401 >> v1:172.21.15.146:6789/0 conn(0x7fe18406f9e0 legacy=0x7fe1840d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:44.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.899+0000 7fe18a4d8640 1 -- 172.21.15.16:0/937100401 >> v1:172.21.15.16:6789/0 conn(0x7fe18415b120 legacy=0x7fe18406dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:44.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.899+0000 7fe18a4d8640 1 -- 172.21.15.16:0/937100401 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18406ef00 con 0x7fe1840d0bc0 2024-09-19T07:53:44.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.900+0000 7fe18a4d8640 1 --2- 172.21.15.16:0/937100401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1840d0bc0 0x7fe1840c5160 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009d90 tx=0x7fe17c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbddfcb5868220c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:44.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.900+0000 7fe188cd5640 1 -- 172.21.15.16:0/937100401 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c007740 con 0x7fe1840d0bc0 2024-09-19T07:53:44.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.900+0000 7fe188cd5640 1 -- 172.21.15.16:0/937100401 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe17c0078e0 con 0x7fe1840d0bc0 2024-09-19T07:53:44.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.900+0000 7fe188cd5640 1 -- 172.21.15.16:0/937100401 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c007cb0 con 0x7fe1840d0bc0 2024-09-19T07:53:44.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.900+0000 7fe18bf3b640 1 -- 172.21.15.16:0/937100401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1840d0bc0 msgr2=0x7fe1840c5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:44.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.900+0000 7fe18bf3b640 1 --2- 172.21.15.16:0/937100401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1840d0bc0 0x7fe1840c5160 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009d90 tx=0x7fe17c02a6f0 comp rx=0 tx=0).stop 2024-09-19T07:53:44.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.901+0000 7fe18bf3b640 1 -- 172.21.15.16:0/937100401 shutdown_connections 2024-09-19T07:53:44.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.901+0000 7fe18bf3b640 1 --2- 172.21.15.16:0/937100401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1840d0bc0 0x7fe1840c5160 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:44.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.901+0000 7fe18bf3b640 1 -- 172.21.15.16:0/937100401 >> 172.21.15.16:0/937100401 conn(0x7fe18406bdf0 msgr2=0x7fe18406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:44.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.901+0000 7fe18bf3b640 1 -- 172.21.15.16:0/937100401 shutdown_connections 2024-09-19T07:53:44.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.901+0000 7fe18bf3b640 1 -- 172.21.15.16:0/937100401 wait complete. 2024-09-19T07:53:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.902+0000 7fe18bf3b640 1 Processor -- start 2024-09-19T07:53:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.902+0000 7fe18bf3b640 1 -- start start 2024-09-19T07:53:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe18bf3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe18406f9e0 0x7fe184145760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe18bf3b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1840d0bc0 0x7fe184148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe18bf3b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe18415b120 0x7fe1840b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe18bf3b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe18406e5d0 con 0x7fe18406f9e0 2024-09-19T07:53:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe18bf3b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe18406f4c0 con 0x7fe18415b120 2024-09-19T07:53:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe18bf3b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe1840ce060 con 0x7fe1840d0bc0 2024-09-19T07:53:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe189cd7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe18406f9e0 0x7fe184145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe189cd7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe18406f9e0 0x7fe184145760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52664/0 (socket says 172.21.15.16:52664) 2024-09-19T07:53:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe189cd7640 1 -- 172.21.15.16:0/1344865809 learned_addr learned my addr 172.21.15.16:0/1344865809 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe1894d6640 1 --2- 172.21.15.16:0/1344865809 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1840d0bc0 0x7fe184148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe18a4d8640 1 --2- 172.21.15.16:0/1344865809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe18415b120 0x7fe1840b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe189cd7640 1 -- 172.21.15.16:0/1344865809 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1840d0bc0 msgr2=0x7fe184148950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe189cd7640 1 --2- 172.21.15.16:0/1344865809 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1840d0bc0 0x7fe184148950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe189cd7640 1 -- 172.21.15.16:0/1344865809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe18415b120 msgr2=0x7fe1840b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe189cd7640 1 --2- 172.21.15.16:0/1344865809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe18415b120 0x7fe1840b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe189cd7640 1 -- 172.21.15.16:0/1344865809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1840b0330 con 0x7fe18406f9e0 2024-09-19T07:53:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe189cd7640 1 --2- 172.21.15.16:0/1344865809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe18406f9e0 0x7fe184145760 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fe18000c4b0 tx=0x7fe18000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe17a7fc640 1 -- 172.21.15.16:0/1344865809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180017070 con 0x7fe18406f9e0 2024-09-19T07:53:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe17a7fc640 1 -- 172.21.15.16:0/1344865809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe180005ce0 con 0x7fe18406f9e0 2024-09-19T07:53:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe18bf3b640 1 -- 172.21.15.16:0/1344865809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1840b0610 con 0x7fe18406f9e0 2024-09-19T07:53:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.903+0000 7fe17a7fc640 1 -- 172.21.15.16:0/1344865809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180007270 con 0x7fe18406f9e0 2024-09-19T07:53:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.904+0000 7fe18bf3b640 1 -- 172.21.15.16:0/1344865809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1840b0bd0 con 0x7fe18406f9e0 2024-09-19T07:53:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.904+0000 7fe18bf3b640 1 -- 172.21.15.16:0/1344865809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1840d04b0 con 0x7fe18406f9e0 2024-09-19T07:53:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.905+0000 7fe17a7fc640 1 -- 172.21.15.16:0/1344865809 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe18001a070 con 0x7fe18406f9e0 2024-09-19T07:53:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.905+0000 7fe17a7fc640 1 --2- 172.21.15.16:0/1344865809 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe16806c2b0 0x7fe16806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.905+0000 7fe17a7fc640 1 -- 172.21.15.16:0/1344865809 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6101+0+0 (secure 0 0 0) 0x7fe18008a7c0 con 0x7fe18406f9e0 2024-09-19T07:53:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.906+0000 7fe1894d6640 1 --2- 172.21.15.16:0/1344865809 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe16806c2b0 0x7fe16806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.906+0000 7fe1894d6640 1 --2- 172.21.15.16:0/1344865809 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe16806c2b0 0x7fe16806e770 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe1840b4440 tx=0x7fe174009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:44.910+0000 7fe17a7fc640 1 -- 172.21.15.16:0/1344865809 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe1800587e0 con 0x7fe18406f9e0 2024-09-19T07:53:45.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.089+0000 7fe18bf3b640 1 -- 172.21.15.16:0/1344865809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1840c6930 con 0x7fe18406f9e0 2024-09-19T07:53:45.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.090+0000 7fe17a7fc640 1 -- 172.21.15.16:0/1344865809 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+17049 (secure 0 0 0) 0x7fe18005cae0 con 0x7fe18406f9e0 2024-09-19T07:53:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:45.092 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":117,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:53:43.811746+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:53:41.474707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-19T07:53:42.798313+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:53:45.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.093+0000 7fe18bf3b640 1 -- 172.21.15.16:0/1344865809 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe16806c2b0 msgr2=0x7fe16806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:45.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.093+0000 7fe18bf3b640 1 --2- 172.21.15.16:0/1344865809 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe16806c2b0 0x7fe16806e770 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe1840b4440 tx=0x7fe174009040 comp rx=0 tx=0).stop 2024-09-19T07:53:45.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.093+0000 7fe18bf3b640 1 -- 172.21.15.16:0/1344865809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe18406f9e0 msgr2=0x7fe184145760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:45.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.093+0000 7fe18bf3b640 1 --2- 172.21.15.16:0/1344865809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe18406f9e0 0x7fe184145760 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fe18000c4b0 tx=0x7fe18000c970 comp rx=0 tx=0).stop 2024-09-19T07:53:45.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.093+0000 7fe18bf3b640 1 -- 172.21.15.16:0/1344865809 shutdown_connections 2024-09-19T07:53:45.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.093+0000 7fe18bf3b640 1 --2- 172.21.15.16:0/1344865809 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe16806c2b0 0x7fe16806e770 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:45.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.093+0000 7fe18bf3b640 1 --2- 172.21.15.16:0/1344865809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe18415b120 0x7fe1840b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:45.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.093+0000 7fe18bf3b640 1 --2- 172.21.15.16:0/1344865809 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1840d0bc0 0x7fe184148950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:45.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.093+0000 7fe18bf3b640 1 --2- 172.21.15.16:0/1344865809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe18406f9e0 0x7fe184145760 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:45.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.093+0000 7fe18bf3b640 1 -- 172.21.15.16:0/1344865809 >> 172.21.15.16:0/1344865809 conn(0x7fe18406bdf0 msgr2=0x7fe1840cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:45.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.093+0000 7fe18bf3b640 1 -- 172.21.15.16:0/1344865809 shutdown_connections 2024-09-19T07:53:45.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.093+0000 7fe18bf3b640 1 -- 172.21.15.16:0/1344865809 wait complete. 2024-09-19T07:53:45.104 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T07:53:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.209+0000 7fe8dcc17640 1 Processor -- start 2024-09-19T07:53:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.209+0000 7fe8dcc17640 1 -- start start 2024-09-19T07:53:45.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.210+0000 7fe8dcc17640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe8d8141680 0x7fe8d80cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:45.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.210+0000 7fe8dcc17640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8d815d710 con 0x7fe8d80d0370 2024-09-19T07:53:45.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.210+0000 7fe8dcc17640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8d815c950 con 0x7fe8d8142170 2024-09-19T07:53:45.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.210+0000 7fe8dcc17640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe8d80d07e0 con 0x7fe8d8141680 2024-09-19T07:53:45.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.210+0000 7fe8d5d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe8d80d0370 0x7fe8d8141a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35620/0 (socket says 172.21.15.16:35620) 2024-09-19T07:53:45.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.210+0000 7fe8d5d9b640 1 -- 172.21.15.16:0/3929164962 learned_addr learned my addr 172.21.15.16:0/3929164962 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:45.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.210+0000 7fe8d659c640 1 --2- 172.21.15.16:0/3929164962 >> v2:172.21.15.146:3300/0 conn(0x7fe8d8141680 0x7fe8d80cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:45.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.210+0000 7fe8d559a640 1 -- 172.21.15.16:0/3929164962 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2219065154 0 0) 0x7fe8d815d710 con 0x7fe8d80d0370 2024-09-19T07:53:45.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.210+0000 7fe8d559a640 1 -- 172.21.15.16:0/3929164962 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8c0003520 con 0x7fe8d80d0370 2024-09-19T07:53:45.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.211+0000 7fe8d659c640 1 -- 172.21.15.16:0/3929164962 >> v1:172.21.15.146:6789/0 conn(0x7fe8d8142170 legacy=0x7fe8d813f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:45.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.211+0000 7fe8d559a640 1 -- 172.21.15.16:0/3929164962 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1570423185 0 0) 0x7fe8d815c950 con 0x7fe8d8142170 2024-09-19T07:53:45.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.211+0000 7fe8d659c640 1 -- 172.21.15.16:0/3929164962 >> v1:172.21.15.16:6789/0 conn(0x7fe8d80d0370 legacy=0x7fe8d8141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:45.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.211+0000 7fe8d659c640 1 -- 172.21.15.16:0/3929164962 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d815cc30 con 0x7fe8d8141680 2024-09-19T07:53:45.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.211+0000 7fe8d659c640 1 --2- 172.21.15.16:0/3929164962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8d8141680 0x7fe8d80cfe30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80097a0 tx=0x7fe8c802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2480ca4e74c14eef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:45.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.212+0000 7fe8d559a640 1 -- 172.21.15.16:0/3929164962 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8037070 con 0x7fe8d8141680 2024-09-19T07:53:45.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.212+0000 7fe8d559a640 1 -- 172.21.15.16:0/3929164962 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c8032440 con 0x7fe8d8141680 2024-09-19T07:53:45.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.212+0000 7fe8d559a640 1 -- 172.21.15.16:0/3929164962 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8032810 con 0x7fe8d8141680 2024-09-19T07:53:45.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.212+0000 7fe8dcc17640 1 -- 172.21.15.16:0/3929164962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8d8141680 msgr2=0x7fe8d80cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:45.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.212+0000 7fe8dcc17640 1 --2- 172.21.15.16:0/3929164962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8d8141680 0x7fe8d80cfe30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80097a0 tx=0x7fe8c802a230 comp rx=0 tx=0).stop 2024-09-19T07:53:45.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.212+0000 7fe8dcc17640 1 -- 172.21.15.16:0/3929164962 shutdown_connections 2024-09-19T07:53:45.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.212+0000 7fe8dcc17640 1 --2- 172.21.15.16:0/3929164962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8d8141680 0x7fe8d80cfe30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:45.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.212+0000 7fe8dcc17640 1 -- 172.21.15.16:0/3929164962 >> 172.21.15.16:0/3929164962 conn(0x7fe8d80b3ed0 msgr2=0x7fe8d80b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:45.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.212+0000 7fe8dcc17640 1 -- 172.21.15.16:0/3929164962 shutdown_connections 2024-09-19T07:53:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.212+0000 7fe8dcc17640 1 -- 172.21.15.16:0/3929164962 wait complete. 2024-09-19T07:53:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.214+0000 7fe8dcc17640 1 Processor -- start 2024-09-19T07:53:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.214+0000 7fe8dcc17640 1 -- start start 2024-09-19T07:53:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.214+0000 7fe8dcc17640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8d80d0370 0x7fe8d81f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.214+0000 7fe8dcc17640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d8141680 0x7fe8d81f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.214+0000 7fe8dcc17640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8d8142170 0x7fe8d80b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.214+0000 7fe8dcc17640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d815d1f0 con 0x7fe8d8141680 2024-09-19T07:53:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.214+0000 7fe8dcc17640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d815c6f0 con 0x7fe8d8142170 2024-09-19T07:53:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.214+0000 7fe8dcc17640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe8d8142570 con 0x7fe8d80d0370 2024-09-19T07:53:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.214+0000 7fe8d5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d8141680 0x7fe8d81f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.215+0000 7fe8d5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d8141680 0x7fe8d81f4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52674/0 (socket says 172.21.15.16:52674) 2024-09-19T07:53:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.215+0000 7fe8d5d9b640 1 -- 172.21.15.16:0/3871117423 learned_addr learned my addr 172.21.15.16:0/3871117423 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.215+0000 7fe8d6d9d640 1 --2- 172.21.15.16:0/3871117423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8d8142170 0x7fe8d80b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.215+0000 7fe8d659c640 1 --2- 172.21.15.16:0/3871117423 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8d80d0370 0x7fe8d81f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.215+0000 7fe8d5d9b640 1 -- 172.21.15.16:0/3871117423 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8d80d0370 msgr2=0x7fe8d81f2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.215+0000 7fe8d5d9b640 1 --2- 172.21.15.16:0/3871117423 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8d80d0370 0x7fe8d81f2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.215+0000 7fe8d5d9b640 1 -- 172.21.15.16:0/3871117423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8d8142170 msgr2=0x7fe8d80b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.215+0000 7fe8d5d9b640 1 --2- 172.21.15.16:0/3871117423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8d8142170 0x7fe8d80b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.215+0000 7fe8d5d9b640 1 -- 172.21.15.16:0/3871117423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d80c9580 con 0x7fe8d8141680 2024-09-19T07:53:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.215+0000 7fe8d6d9d640 1 --2- 172.21.15.16:0/3871117423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8d8142170 0x7fe8d80b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.215+0000 7fe8d5d9b640 1 --2- 172.21.15.16:0/3871117423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d8141680 0x7fe8d81f4a30 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00c390 tx=0x7fe8cc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.215+0000 7fe8c6ffd640 1 -- 172.21.15.16:0/3871117423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc017070 con 0x7fe8d8141680 2024-09-19T07:53:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.215+0000 7fe8c6ffd640 1 -- 172.21.15.16:0/3871117423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8cc005ce0 con 0x7fe8d8141680 2024-09-19T07:53:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.215+0000 7fe8dcc17640 1 -- 172.21.15.16:0/3871117423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d80c9860 con 0x7fe8d8141680 2024-09-19T07:53:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.215+0000 7fe8c6ffd640 1 -- 172.21.15.16:0/3871117423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0072f0 con 0x7fe8d8141680 2024-09-19T07:53:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.215+0000 7fe8dcc17640 1 -- 172.21.15.16:0/3871117423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d80c9e20 con 0x7fe8d8141680 2024-09-19T07:53:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.216+0000 7fe8dcc17640 1 -- 172.21.15.16:0/3871117423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d80c9310 con 0x7fe8d8141680 2024-09-19T07:53:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.217+0000 7fe8c6ffd640 1 -- 172.21.15.16:0/3871117423 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe8cc01a070 con 0x7fe8d8141680 2024-09-19T07:53:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.217+0000 7fe8c6ffd640 1 --2- 172.21.15.16:0/3871117423 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe8b806c2b0 0x7fe8b806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.217+0000 7fe8c6ffd640 1 -- 172.21.15.16:0/3871117423 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6101+0+0 (secure 0 0 0) 0x7fe8cc08aa70 con 0x7fe8d8141680 2024-09-19T07:53:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.217+0000 7fe8d659c640 1 --2- 172.21.15.16:0/3871117423 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe8b806c2b0 0x7fe8b806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.218+0000 7fe8d659c640 1 --2- 172.21.15.16:0/3871117423 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe8b806c2b0 0x7fe8b806e770 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8034000 tx=0x7fe8c802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.222+0000 7fe8c6ffd640 1 -- 172.21.15.16:0/3871117423 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe8cc0588e0 con 0x7fe8d8141680 2024-09-19T07:53:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.437+0000 7fe8dcc17640 1 -- 172.21.15.16:0/3871117423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe8d80690c0 con 0x7fe8d8141680 2024-09-19T07:53:45.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.465+0000 7fe8c6ffd640 1 -- 172.21.15.16:0/3871117423 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v62) ==== 118+0+0 (secure 0 0 0) 0x7fe8cc05cbe0 con 0x7fe8d8141680 2024-09-19T07:53:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.467+0000 7fe8dcc17640 1 -- 172.21.15.16:0/3871117423 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe8b806c2b0 msgr2=0x7fe8b806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.467+0000 7fe8dcc17640 1 --2- 172.21.15.16:0/3871117423 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe8b806c2b0 0x7fe8b806e770 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8034000 tx=0x7fe8c802e000 comp rx=0 tx=0).stop 2024-09-19T07:53:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.467+0000 7fe8dcc17640 1 -- 172.21.15.16:0/3871117423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d8141680 msgr2=0x7fe8d81f4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.467+0000 7fe8dcc17640 1 --2- 172.21.15.16:0/3871117423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d8141680 0x7fe8d81f4a30 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00c390 tx=0x7fe8cc00c850 comp rx=0 tx=0).stop 2024-09-19T07:53:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.467+0000 7fe8dcc17640 1 -- 172.21.15.16:0/3871117423 shutdown_connections 2024-09-19T07:53:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.467+0000 7fe8dcc17640 1 --2- 172.21.15.16:0/3871117423 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe8b806c2b0 0x7fe8b806e770 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.467+0000 7fe8dcc17640 1 --2- 172.21.15.16:0/3871117423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8d8142170 0x7fe8d80b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.467+0000 7fe8dcc17640 1 --2- 172.21.15.16:0/3871117423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8d8141680 0x7fe8d81f4a30 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:45.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.467+0000 7fe8dcc17640 1 --2- 172.21.15.16:0/3871117423 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8d80d0370 0x7fe8d81f2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:45.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.467+0000 7fe8dcc17640 1 -- 172.21.15.16:0/3871117423 >> 172.21.15.16:0/3871117423 conn(0x7fe8d80b3ed0 msgr2=0x7fe8d80b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:45.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.467+0000 7fe8dcc17640 1 -- 172.21.15.16:0/3871117423 shutdown_connections 2024-09-19T07:53:45.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.467+0000 7fe8dcc17640 1 -- 172.21.15.16:0/3871117423 wait complete. 2024-09-19T07:53:45.479 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T07:53:45.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T07:53:45.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.584+0000 7fbbe0b11640 1 Processor -- start 2024-09-19T07:53:45.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.584+0000 7fbbe0b11640 1 -- start start 2024-09-19T07:53:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.585+0000 7fbbe0b11640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbdc15f0d0 0x7fbbdc15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.585+0000 7fbbe0b11640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbbdc0d0070 0x7fbbdc0d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.585+0000 7fbbe0b11640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbbdc0d1940 0x7fbbdc0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.585+0000 7fbbe0b11640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fbbdc0cf2b0 con 0x7fbbdc0d0070 2024-09-19T07:53:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.585+0000 7fbbe0b11640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fbbdc0cf590 con 0x7fbbdc0d1940 2024-09-19T07:53:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.585+0000 7fbbe0b11640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbbdc0cf870 con 0x7fbbdc15f0d0 2024-09-19T07:53:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.585+0000 7fbbd1d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbbdc0d0070 0x7fbbdc0d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.585+0000 7fbbd1d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbbdc0d0070 0x7fbbdc0d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52678/0 (socket says 172.21.15.16:52678) 2024-09-19T07:53:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.585+0000 7fbbd1d9b640 1 -- 172.21.15.16:0/2642602636 learned_addr learned my addr 172.21.15.16:0/2642602636 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.585+0000 7fbbd1d9b640 1 -- 172.21.15.16:0/2642602636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbdc15f0d0 msgr2=0x7fbbdc15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.585+0000 7fbbda59c640 1 --2- 172.21.15.16:0/2642602636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbdc15f0d0 0x7fbbdc15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.585+0000 7fbbdad9d640 1 --2- 172.21.15.16:0/2642602636 >> v2:172.21.15.146:3300/0 conn(0x7fbbdc0d1940 0x7fbbdc0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.585+0000 7fbbd1d9b640 1 --2- 172.21.15.16:0/2642602636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbdc15f0d0 0x7fbbdc15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.585+0000 7fbbd1d9b640 1 -- 172.21.15.16:0/2642602636 >> v2:172.21.15.146:3300/0 conn(0x7fbbdc0d1940 msgr2=0x7fbbdc0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.585+0000 7fbbd1d9b640 1 --2- 172.21.15.16:0/2642602636 >> v2:172.21.15.146:3300/0 conn(0x7fbbdc0d1940 0x7fbbdc0d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.585+0000 7fbbd1d9b640 1 -- 172.21.15.16:0/2642602636 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbdc0cfb50 con 0x7fbbdc0d0070 2024-09-19T07:53:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.585+0000 7fbbdad9d640 1 --2- 172.21.15.16:0/2642602636 >> v2:172.21.15.146:3300/0 conn(0x7fbbdc0d1940 0x7fbbdc0d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.585+0000 7fbbda59c640 1 --2- 172.21.15.16:0/2642602636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbdc15f0d0 0x7fbbdc15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.586+0000 7fbbd1d9b640 1 --2- 172.21.15.16:0/2642602636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbdc0d0070 0x7fbbdc0d1400 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc161e80 tx=0x7fbbc402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286a11777599dfcc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.586+0000 7fbbd9d9b640 1 -- 172.21.15.16:0/2642602636 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4037070 con 0x7fbbdc0d0070 2024-09-19T07:53:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.586+0000 7fbbd9d9b640 1 -- 172.21.15.16:0/2642602636 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbc4032440 con 0x7fbbdc0d0070 2024-09-19T07:53:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.586+0000 7fbbd9d9b640 1 -- 172.21.15.16:0/2642602636 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4032810 con 0x7fbbdc0d0070 2024-09-19T07:53:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.587+0000 7fbbe0b11640 1 -- 172.21.15.16:0/2642602636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbdc0d0070 msgr2=0x7fbbdc0d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.587+0000 7fbbe0b11640 1 --2- 172.21.15.16:0/2642602636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbdc0d0070 0x7fbbdc0d1400 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc161e80 tx=0x7fbbc402a140 comp rx=0 tx=0).stop 2024-09-19T07:53:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.587+0000 7fbbe0b11640 1 -- 172.21.15.16:0/2642602636 shutdown_connections 2024-09-19T07:53:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.587+0000 7fbbe0b11640 1 --2- 172.21.15.16:0/2642602636 >> v2:172.21.15.146:3300/0 conn(0x7fbbdc0d1940 0x7fbbdc0d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.587+0000 7fbbe0b11640 1 --2- 172.21.15.16:0/2642602636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbdc0d0070 0x7fbbdc0d1400 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.587+0000 7fbbe0b11640 1 --2- 172.21.15.16:0/2642602636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbdc15f0d0 0x7fbbdc15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.587+0000 7fbbe0b11640 1 -- 172.21.15.16:0/2642602636 >> 172.21.15.16:0/2642602636 conn(0x7fbbdc09d760 msgr2=0x7fbbdc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.587+0000 7fbbe0b11640 1 -- 172.21.15.16:0/2642602636 shutdown_connections 2024-09-19T07:53:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.587+0000 7fbbe0b11640 1 -- 172.21.15.16:0/2642602636 wait complete. 2024-09-19T07:53:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.588+0000 7fbbe0b11640 1 Processor -- start 2024-09-19T07:53:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.588+0000 7fbbe0b11640 1 -- start start 2024-09-19T07:53:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.589+0000 7fbbe0b11640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbdc0d0070 0x7fbbdc0cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.589+0000 7fbbe0b11640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbdc0d1940 0x7fbbdc0b87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.589+0000 7fbbe0b11640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbdc15f0d0 0x7fbbdc0b5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.589+0000 7fbbe0b11640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc0d47f0 con 0x7fbbdc0d0070 2024-09-19T07:53:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.589+0000 7fbbe0b11640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc0d4670 con 0x7fbbdc15f0d0 2024-09-19T07:53:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.589+0000 7fbbe0b11640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbbdc0d4970 con 0x7fbbdc0d1940 2024-09-19T07:53:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.589+0000 7fbbda59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbdc0d0070 0x7fbbdc0cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.589+0000 7fbbda59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbdc0d0070 0x7fbbdc0cd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52684/0 (socket says 172.21.15.16:52684) 2024-09-19T07:53:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.589+0000 7fbbda59c640 1 -- 172.21.15.16:0/1877210560 learned_addr learned my addr 172.21.15.16:0/1877210560 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.589+0000 7fbbdad9d640 1 --2- 172.21.15.16:0/1877210560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbdc15f0d0 0x7fbbdc0b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.589+0000 7fbbd1d9b640 1 --2- 172.21.15.16:0/1877210560 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbdc0d1940 0x7fbbdc0b87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.589+0000 7fbbda59c640 1 -- 172.21.15.16:0/1877210560 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbdc0d1940 msgr2=0x7fbbdc0b87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.589+0000 7fbbda59c640 1 --2- 172.21.15.16:0/1877210560 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbdc0d1940 0x7fbbdc0b87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.589+0000 7fbbda59c640 1 -- 172.21.15.16:0/1877210560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbdc15f0d0 msgr2=0x7fbbdc0b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.590+0000 7fbbda59c640 1 --2- 172.21.15.16:0/1877210560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbdc15f0d0 0x7fbbdc0b5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.590+0000 7fbbda59c640 1 -- 172.21.15.16:0/1877210560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbdc15a280 con 0x7fbbdc0d0070 2024-09-19T07:53:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.590+0000 7fbbdad9d640 1 --2- 172.21.15.16:0/1877210560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbdc15f0d0 0x7fbbdc0b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.590+0000 7fbbd1d9b640 1 --2- 172.21.15.16:0/1877210560 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbdc0d1940 0x7fbbdc0b87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.590+0000 7fbbda59c640 1 --2- 172.21.15.16:0/1877210560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbdc0d0070 0x7fbbdc0cd790 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00c820 tx=0x7fbbcc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.590+0000 7fbbd37fe640 1 -- 172.21.15.16:0/1877210560 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc015070 con 0x7fbbdc0d0070 2024-09-19T07:53:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.590+0000 7fbbd37fe640 1 -- 172.21.15.16:0/1877210560 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbcc007a30 con 0x7fbbdc0d0070 2024-09-19T07:53:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.590+0000 7fbbe0b11640 1 -- 172.21.15.16:0/1877210560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbdc15a560 con 0x7fbbdc0d0070 2024-09-19T07:53:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.590+0000 7fbbd37fe640 1 -- 172.21.15.16:0/1877210560 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc007de0 con 0x7fbbdc0d0070 2024-09-19T07:53:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.590+0000 7fbbe0b11640 1 -- 172.21.15.16:0/1877210560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbdc15ab20 con 0x7fbbdc0d0070 2024-09-19T07:53:45.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.591+0000 7fbbe0b11640 1 -- 172.21.15.16:0/1877210560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbdc0d1400 con 0x7fbbdc0d0070 2024-09-19T07:53:45.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.591+0000 7fbbd37fe640 1 -- 172.21.15.16:0/1877210560 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbbcc018070 con 0x7fbbdc0d0070 2024-09-19T07:53:45.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.592+0000 7fbbd37fe640 1 --2- 172.21.15.16:0/1877210560 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbbbc06c2b0 0x7fbbbc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:45.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.592+0000 7fbbd37fe640 1 -- 172.21.15.16:0/1877210560 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fbbcc08b580 con 0x7fbbdc0d0070 2024-09-19T07:53:45.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.592+0000 7fbbd1d9b640 1 --2- 172.21.15.16:0/1877210560 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbbbc06c2b0 0x7fbbbc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.593+0000 7fbbd1d9b640 1 --2- 172.21.15.16:0/1877210560 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbbbc06c2b0 0x7fbbbc06e770 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4034000 tx=0x7fbbc402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:45.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.596+0000 7fbbd37fe640 1 -- 172.21.15.16:0/1877210560 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fbbcc059ac0 con 0x7fbbdc0d0070 2024-09-19T07:53:45.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:45.808+0000 7fbbe0b11640 1 -- 172.21.15.16:0/1877210560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbbdc0d2ed0 con 0x7fbbdc0d0070 2024-09-19T07:53:46.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.470+0000 7fbbd37fe640 1 -- 172.21.15.16:0/1877210560 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v63) ==== 122+0+0 (secure 0 0 0) 0x7fbbcc0593c0 con 0x7fbbdc0d0070 2024-09-19T07:53:46.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.473+0000 7fbbe0b11640 1 -- 172.21.15.16:0/1877210560 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbbbc06c2b0 msgr2=0x7fbbbc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:46.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.473+0000 7fbbe0b11640 1 --2- 172.21.15.16:0/1877210560 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbbbc06c2b0 0x7fbbbc06e770 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4034000 tx=0x7fbbc402e000 comp rx=0 tx=0).stop 2024-09-19T07:53:46.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.473+0000 7fbbe0b11640 1 -- 172.21.15.16:0/1877210560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbdc0d0070 msgr2=0x7fbbdc0cd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:46.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.473+0000 7fbbe0b11640 1 --2- 172.21.15.16:0/1877210560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbdc0d0070 0x7fbbdc0cd790 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00c820 tx=0x7fbbcc00cce0 comp rx=0 tx=0).stop 2024-09-19T07:53:46.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.473+0000 7fbbe0b11640 1 -- 172.21.15.16:0/1877210560 shutdown_connections 2024-09-19T07:53:46.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.473+0000 7fbbe0b11640 1 --2- 172.21.15.16:0/1877210560 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbbbc06c2b0 0x7fbbbc06e770 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:46.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.473+0000 7fbbe0b11640 1 --2- 172.21.15.16:0/1877210560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbdc15f0d0 0x7fbbdc0b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:46.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.473+0000 7fbbe0b11640 1 --2- 172.21.15.16:0/1877210560 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbdc0d1940 0x7fbbdc0b87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:46.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.473+0000 7fbbe0b11640 1 --2- 172.21.15.16:0/1877210560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbdc0d0070 0x7fbbdc0cd790 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:46.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.473+0000 7fbbe0b11640 1 -- 172.21.15.16:0/1877210560 >> 172.21.15.16:0/1877210560 conn(0x7fbbdc09d760 msgr2=0x7fbbdc1626a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:46.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.473+0000 7fbbe0b11640 1 -- 172.21.15.16:0/1877210560 shutdown_connections 2024-09-19T07:53:46.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.473+0000 7fbbe0b11640 1 -- 172.21.15.16:0/1877210560 wait complete. 2024-09-19T07:53:46.487 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T07:53:46.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:53:46.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.596+0000 7fa02aab7640 1 Processor -- start 2024-09-19T07:53:46.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.596+0000 7fa02aab7640 1 -- start start 2024-09-19T07:53:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.597+0000 7fa02aab7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa024147e70 0x7fa0240cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.597+0000 7fa02aab7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa02415ef80 0x7fa02406d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.597+0000 7fa02aab7640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0240cfb00 con 0x7fa02406cd20 2024-09-19T07:53:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.597+0000 7fa02aab7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa02406d6d0 con 0x7fa024147e70 2024-09-19T07:53:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.597+0000 7fa02aab7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa024148250 con 0x7fa02415ef80 2024-09-19T07:53:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.597+0000 7fa028853640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa024147e70 0x7fa0240cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.597+0000 7fa028853640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa024147e70 0x7fa0240cf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52694/0 (socket says 172.21.15.16:52694) 2024-09-19T07:53:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.597+0000 7fa028853640 1 -- 172.21.15.16:0/1371332829 learned_addr learned my addr 172.21.15.16:0/1371332829 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.597+0000 7fa028853640 1 -- 172.21.15.16:0/1371332829 >> v2:172.21.15.146:3300/0 conn(0x7fa02415ef80 msgr2=0x7fa02406d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.597+0000 7fa029054640 1 --2- 172.21.15.16:0/1371332829 >> v2:172.21.15.146:3300/0 conn(0x7fa02415ef80 0x7fa02406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.597+0000 7fa028853640 1 --2- 172.21.15.16:0/1371332829 >> v2:172.21.15.146:3300/0 conn(0x7fa02415ef80 0x7fa02406d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.597+0000 7fa028853640 1 -- 172.21.15.16:0/1371332829 >> v1:172.21.15.146:6789/0 conn(0x7fa02406cd20 legacy=0x7fa02415e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.597+0000 7fa028853640 1 -- 172.21.15.16:0/1371332829 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa02415f450 con 0x7fa024147e70 2024-09-19T07:53:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.597+0000 7fa029054640 1 --2- 172.21.15.16:0/1371332829 >> v2:172.21.15.146:3300/0 conn(0x7fa02415ef80 0x7fa02406d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.598+0000 7fa028853640 1 --2- 172.21.15.16:0/1371332829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa024147e70 0x7fa0240cf5c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa0240b1df0 tx=0x7fa02002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ebb62f8c5daebc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.598+0000 7fa01b7fe640 1 -- 172.21.15.16:0/1371332829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020037070 con 0x7fa024147e70 2024-09-19T07:53:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.598+0000 7fa01b7fe640 1 -- 172.21.15.16:0/1371332829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa020032440 con 0x7fa024147e70 2024-09-19T07:53:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.598+0000 7fa01b7fe640 1 -- 172.21.15.16:0/1371332829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020032810 con 0x7fa024147e70 2024-09-19T07:53:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.599+0000 7fa02aab7640 1 -- 172.21.15.16:0/1371332829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa024147e70 msgr2=0x7fa0240cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.599+0000 7fa02aab7640 1 --2- 172.21.15.16:0/1371332829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa024147e70 0x7fa0240cf5c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa0240b1df0 tx=0x7fa02002a1a0 comp rx=0 tx=0).stop 2024-09-19T07:53:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.599+0000 7fa02aab7640 1 -- 172.21.15.16:0/1371332829 shutdown_connections 2024-09-19T07:53:46.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.599+0000 7fa02aab7640 1 --2- 172.21.15.16:0/1371332829 >> v2:172.21.15.146:3300/0 conn(0x7fa02415ef80 0x7fa02406d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:46.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.599+0000 7fa02aab7640 1 --2- 172.21.15.16:0/1371332829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa024147e70 0x7fa0240cf5c0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:46.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.599+0000 7fa02aab7640 1 -- 172.21.15.16:0/1371332829 >> 172.21.15.16:0/1371332829 conn(0x7fa0240b3ed0 msgr2=0x7fa0240b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:46.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.599+0000 7fa02aab7640 1 -- 172.21.15.16:0/1371332829 shutdown_connections 2024-09-19T07:53:46.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.599+0000 7fa02aab7640 1 -- 172.21.15.16:0/1371332829 wait complete. 2024-09-19T07:53:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.600+0000 7fa02aab7640 1 Processor -- start 2024-09-19T07:53:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.600+0000 7fa02aab7640 1 -- start start 2024-09-19T07:53:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.601+0000 7fa02aab7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa02406cd20 0x7fa0241463e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.601+0000 7fa02aab7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa024147e70 0x7fa0240c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.601+0000 7fa02aab7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa02415ef80 0x7fa0241ea1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.601+0000 7fa02aab7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0240ceae0 con 0x7fa024147e70 2024-09-19T07:53:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.601+0000 7fa02aab7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa0240cf130 con 0x7fa02415ef80 2024-09-19T07:53:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.601+0000 7fa02aab7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa0240cedc0 con 0x7fa02406cd20 2024-09-19T07:53:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.601+0000 7fa01bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa024147e70 0x7fa0240c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.601+0000 7fa01bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa024147e70 0x7fa0240c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52708/0 (socket says 172.21.15.16:52708) 2024-09-19T07:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.601+0000 7fa01bfff640 1 -- 172.21.15.16:0/1076955928 learned_addr learned my addr 172.21.15.16:0/1076955928 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.601+0000 7fa029054640 1 --2- 172.21.15.16:0/1076955928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa02415ef80 0x7fa0241ea1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.601+0000 7fa028853640 1 --2- 172.21.15.16:0/1076955928 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa02406cd20 0x7fa0241463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.602+0000 7fa01bfff640 1 -- 172.21.15.16:0/1076955928 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa02406cd20 msgr2=0x7fa0241463e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.602+0000 7fa01bfff640 1 --2- 172.21.15.16:0/1076955928 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa02406cd20 0x7fa0241463e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.602+0000 7fa01bfff640 1 -- 172.21.15.16:0/1076955928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa02415ef80 msgr2=0x7fa0241ea1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.602+0000 7fa01bfff640 1 --2- 172.21.15.16:0/1076955928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa02415ef80 0x7fa0241ea1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.602+0000 7fa01bfff640 1 -- 172.21.15.16:0/1076955928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0241e9170 con 0x7fa024147e70 2024-09-19T07:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.602+0000 7fa028853640 1 --2- 172.21.15.16:0/1076955928 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa02406cd20 0x7fa0241463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.602+0000 7fa029054640 1 --2- 172.21.15.16:0/1076955928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa02415ef80 0x7fa0241ea1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.602+0000 7fa01bfff640 1 --2- 172.21.15.16:0/1076955928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa024147e70 0x7fa0240c9d40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa01400c430 tx=0x7fa01400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.602+0000 7fa0197fa640 1 -- 172.21.15.16:0/1076955928 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014011790 con 0x7fa024147e70 2024-09-19T07:53:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.602+0000 7fa0197fa640 1 -- 172.21.15.16:0/1076955928 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa014017070 con 0x7fa024147e70 2024-09-19T07:53:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.602+0000 7fa02aab7640 1 -- 172.21.15.16:0/1076955928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0241e93f0 con 0x7fa024147e70 2024-09-19T07:53:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.602+0000 7fa0197fa640 1 -- 172.21.15.16:0/1076955928 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014011bb0 con 0x7fa024147e70 2024-09-19T07:53:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.602+0000 7fa02aab7640 1 -- 172.21.15.16:0/1076955928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0241e99b0 con 0x7fa024147e70 2024-09-19T07:53:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.603+0000 7fa02aab7640 1 -- 172.21.15.16:0/1076955928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa024161090 con 0x7fa024147e70 2024-09-19T07:53:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.603+0000 7fa0197fa640 1 -- 172.21.15.16:0/1076955928 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa01401c460 con 0x7fa024147e70 2024-09-19T07:53:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.604+0000 7fa0197fa640 1 --2- 172.21.15.16:0/1076955928 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa01006c2e0 0x7fa01006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.604+0000 7fa0197fa640 1 -- 172.21.15.16:0/1076955928 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fa01408b400 con 0x7fa024147e70 2024-09-19T07:53:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.605+0000 7fa028853640 1 --2- 172.21.15.16:0/1076955928 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa01006c2e0 0x7fa01006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.605+0000 7fa028853640 1 --2- 172.21.15.16:0/1076955928 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa01006c2e0 0x7fa01006e7a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa020034000 tx=0x7fa02002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.609+0000 7fa0197fa640 1 -- 172.21.15.16:0/1076955928 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa024161090 con 0x7fa024147e70 2024-09-19T07:53:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.820+0000 7fa02aab7640 1 -- 172.21.15.16:0/1076955928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0240722f0 con 0x7fa024147e70 2024-09-19T07:53:46.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.821+0000 7fa0197fa640 1 -- 172.21.15.16:0/1076955928 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+5231 (secure 0 0 0) 0x7fa0241e9170 con 0x7fa024147e70 2024-09-19T07:53:46.821 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:46.821 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":63,"btime":"2024-09-19T07:53:46:466514+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2471408352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2471408352},{"type":"v1","addr":"172.21.15.16:6833","nonce":2471408352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/1281369866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1281369866},{"type":"v1","addr":"172.21.15.146:6836","nonce":1281369866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1377303883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1377303883},{"type":"v1","addr":"172.21.15.146:6837","nonce":1377303883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:53:43.812066+0000","modified":"2024-09-19T07:53:45.809688+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7368":{"gid":7368,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2050126415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2050126415},{"type":"v1","addr":"172.21.15.16:6835","nonce":2050126415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7368,"qdb_cluster":[7368]},"id":8}]} 2024-09-19T07:53:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 63 2024-09-19T07:53:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.823+0000 7fa02aab7640 1 -- 172.21.15.16:0/1076955928 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa01006c2e0 msgr2=0x7fa01006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.823+0000 7fa02aab7640 1 --2- 172.21.15.16:0/1076955928 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa01006c2e0 0x7fa01006e7a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa020034000 tx=0x7fa02002e000 comp rx=0 tx=0).stop 2024-09-19T07:53:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.823+0000 7fa02aab7640 1 -- 172.21.15.16:0/1076955928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa024147e70 msgr2=0x7fa0240c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.823+0000 7fa02aab7640 1 --2- 172.21.15.16:0/1076955928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa024147e70 0x7fa0240c9d40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa01400c430 tx=0x7fa01400c8f0 comp rx=0 tx=0).stop 2024-09-19T07:53:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.823+0000 7fa02aab7640 1 -- 172.21.15.16:0/1076955928 shutdown_connections 2024-09-19T07:53:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.823+0000 7fa02aab7640 1 --2- 172.21.15.16:0/1076955928 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa01006c2e0 0x7fa01006e7a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.823+0000 7fa02aab7640 1 --2- 172.21.15.16:0/1076955928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa02415ef80 0x7fa0241ea1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.823+0000 7fa02aab7640 1 --2- 172.21.15.16:0/1076955928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa024147e70 0x7fa0240c9d40 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.823+0000 7fa02aab7640 1 --2- 172.21.15.16:0/1076955928 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa02406cd20 0x7fa0241463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.823+0000 7fa02aab7640 1 -- 172.21.15.16:0/1076955928 >> 172.21.15.16:0/1076955928 conn(0x7fa0240b3ed0 msgr2=0x7fa0240b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.823+0000 7fa02aab7640 1 -- 172.21.15.16:0/1076955928 shutdown_connections 2024-09-19T07:53:46.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.823+0000 7fa02aab7640 1 -- 172.21.15.16:0/1076955928 wait complete. 2024-09-19T07:53:46.835 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:53:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.936+0000 7fc6016c0640 1 Processor -- start 2024-09-19T07:53:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.936+0000 7fc6016c0640 1 -- start start 2024-09-19T07:53:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.937+0000 7fc6016c0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc5fc072590 0x7fc5fc06d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.937+0000 7fc6016c0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5fc1620b0 con 0x7fc5fc141ad0 2024-09-19T07:53:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.937+0000 7fc6016c0640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5fc162370 con 0x7fc5fc0d21f0 2024-09-19T07:53:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.937+0000 7fc6016c0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc5fc141750 con 0x7fc5fc072590 2024-09-19T07:53:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.937+0000 7fc5fa7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc5fc072590 0x7fc5fc06d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.937+0000 7fc5fb7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc5fc141ad0 0x7fc5fc141f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35630/0 (socket says 172.21.15.16:35630) 2024-09-19T07:53:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.937+0000 7fc5fb7fe640 1 -- 172.21.15.16:0/417630654 learned_addr learned my addr 172.21.15.16:0/417630654 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.937+0000 7fc5fa7fc640 1 -- 172.21.15.16:0/417630654 >> v1:172.21.15.146:6789/0 conn(0x7fc5fc0d21f0 legacy=0x7fc5fc0d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.937+0000 7fc5fa7fc640 1 -- 172.21.15.16:0/417630654 >> v1:172.21.15.16:6789/0 conn(0x7fc5fc141ad0 legacy=0x7fc5fc141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.937+0000 7fc5fa7fc640 1 -- 172.21.15.16:0/417630654 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5fc162650 con 0x7fc5fc072590 2024-09-19T07:53:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.937+0000 7fc5f9ffb640 1 -- 172.21.15.16:0/417630654 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 690283254 0 0) 0x7fc5fc1620b0 con 0x7fc5fc141ad0 2024-09-19T07:53:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.938+0000 7fc5fa7fc640 1 --2- 172.21.15.16:0/417630654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5fc072590 0x7fc5fc06d0e0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fc5f00097a0 tx=0x7fc5f002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9dbd6ce7883a99 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:46.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.938+0000 7fc5f9ffb640 1 -- 172.21.15.16:0/417630654 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0037070 con 0x7fc5fc072590 2024-09-19T07:53:46.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.938+0000 7fc5f9ffb640 1 -- 172.21.15.16:0/417630654 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f002ae50 con 0x7fc5fc072590 2024-09-19T07:53:46.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.938+0000 7fc5f9ffb640 1 -- 172.21.15.16:0/417630654 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0032660 con 0x7fc5fc072590 2024-09-19T07:53:46.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.938+0000 7fc6016c0640 1 -- 172.21.15.16:0/417630654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5fc072590 msgr2=0x7fc5fc06d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:46.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.938+0000 7fc6016c0640 1 --2- 172.21.15.16:0/417630654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5fc072590 0x7fc5fc06d0e0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fc5f00097a0 tx=0x7fc5f002a140 comp rx=0 tx=0).stop 2024-09-19T07:53:46.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.938+0000 7fc6016c0640 1 -- 172.21.15.16:0/417630654 shutdown_connections 2024-09-19T07:53:46.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.938+0000 7fc6016c0640 1 --2- 172.21.15.16:0/417630654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5fc072590 0x7fc5fc06d0e0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.938+0000 7fc6016c0640 1 -- 172.21.15.16:0/417630654 >> 172.21.15.16:0/417630654 conn(0x7fc5fc0b3ed0 msgr2=0x7fc5fc0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.939+0000 7fc6016c0640 1 -- 172.21.15.16:0/417630654 shutdown_connections 2024-09-19T07:53:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.939+0000 7fc6016c0640 1 -- 172.21.15.16:0/417630654 wait complete. 2024-09-19T07:53:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.940+0000 7fc6016c0640 1 Processor -- start 2024-09-19T07:53:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.940+0000 7fc6016c0640 1 -- start start 2024-09-19T07:53:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc6016c0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5fc072590 0x7fc5fc1405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:46.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc6016c0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5fc0d21f0 0x7fc5fc15b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:46.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc6016c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5fc141ad0 0x7fc5fc15beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:46.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc6016c0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc06d620 con 0x7fc5fc141ad0 2024-09-19T07:53:46.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc6016c0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc162c10 con 0x7fc5fc0d21f0 2024-09-19T07:53:46.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc6016c0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc5fc06d7a0 con 0x7fc5fc072590 2024-09-19T07:53:46.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc5fb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5fc141ad0 0x7fc5fc15beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:46.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc5fb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5fc141ad0 0x7fc5fc15beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52728/0 (socket says 172.21.15.16:52728) 2024-09-19T07:53:46.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc5fb7fe640 1 -- 172.21.15.16:0/2402147558 learned_addr learned my addr 172.21.15.16:0/2402147558 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:46.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc5fa7fc640 1 --2- 172.21.15.16:0/2402147558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5fc0d21f0 0x7fc5fc15b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:46.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc5faffd640 1 --2- 172.21.15.16:0/2402147558 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5fc072590 0x7fc5fc1405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:46.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc5fb7fe640 1 -- 172.21.15.16:0/2402147558 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5fc072590 msgr2=0x7fc5fc1405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:46.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc5fb7fe640 1 --2- 172.21.15.16:0/2402147558 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5fc072590 0x7fc5fc1405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:46.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc5fb7fe640 1 -- 172.21.15.16:0/2402147558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5fc0d21f0 msgr2=0x7fc5fc15b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:46.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc5fb7fe640 1 --2- 172.21.15.16:0/2402147558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5fc0d21f0 0x7fc5fc15b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:46.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc5fb7fe640 1 -- 172.21.15.16:0/2402147558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5fc1f4220 con 0x7fc5fc141ad0 2024-09-19T07:53:46.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc5fb7fe640 1 --2- 172.21.15.16:0/2402147558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5fc141ad0 0x7fc5fc15beb0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400c700 tx=0x7fc5f400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:46.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc5e37fe640 1 -- 172.21.15.16:0/2402147558 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4018070 con 0x7fc5fc141ad0 2024-09-19T07:53:46.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc5e37fe640 1 -- 172.21.15.16:0/2402147558 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f400ce10 con 0x7fc5fc141ad0 2024-09-19T07:53:46.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc6016c0640 1 -- 172.21.15.16:0/2402147558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5fc1f4500 con 0x7fc5fc141ad0 2024-09-19T07:53:46.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.941+0000 7fc5e37fe640 1 -- 172.21.15.16:0/2402147558 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f400a860 con 0x7fc5fc141ad0 2024-09-19T07:53:46.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.942+0000 7fc6016c0640 1 -- 172.21.15.16:0/2402147558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5fc1f4ac0 con 0x7fc5fc141ad0 2024-09-19T07:53:46.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.943+0000 7fc6016c0640 1 -- 172.21.15.16:0/2402147558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5fc158c30 con 0x7fc5fc141ad0 2024-09-19T07:53:46.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.943+0000 7fc5e37fe640 1 -- 172.21.15.16:0/2402147558 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc5f401b070 con 0x7fc5fc141ad0 2024-09-19T07:53:46.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.943+0000 7fc5e37fe640 1 --2- 172.21.15.16:0/2402147558 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc5e806c2b0 0x7fc5e806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:46.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.943+0000 7fc5e37fe640 1 -- 172.21.15.16:0/2402147558 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fc5f408aaa0 con 0x7fc5fc141ad0 2024-09-19T07:53:46.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.944+0000 7fc5faffd640 1 --2- 172.21.15.16:0/2402147558 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc5e806c2b0 0x7fc5e806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:46.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.944+0000 7fc5faffd640 1 --2- 172.21.15.16:0/2402147558 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc5e806c2b0 0x7fc5e806e770 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec004f60 tx=0x7fc5ec007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:46.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:46.948+0000 7fc5e37fe640 1 -- 172.21.15.16:0/2402147558 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc5f4058ac0 con 0x7fc5fc141ad0 2024-09-19T07:53:47.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.127+0000 7fc6016c0640 1 -- 172.21.15.16:0/2402147558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5fc0690c0 con 0x7fc5fc141ad0 2024-09-19T07:53:47.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.128+0000 7fc5e37fe640 1 -- 172.21.15.16:0/2402147558 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+17036 (secure 0 0 0) 0x7fc5f405cdc0 con 0x7fc5fc141ad0 2024-09-19T07:53:47.128 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:47.129 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":118,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:53:45.447012+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:53:41.474707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-19T07:53:42.798313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:53:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.130+0000 7fc6016c0640 1 -- 172.21.15.16:0/2402147558 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc5e806c2b0 msgr2=0x7fc5e806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.130+0000 7fc6016c0640 1 --2- 172.21.15.16:0/2402147558 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc5e806c2b0 0x7fc5e806e770 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec004f60 tx=0x7fc5ec007e30 comp rx=0 tx=0).stop 2024-09-19T07:53:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.130+0000 7fc6016c0640 1 -- 172.21.15.16:0/2402147558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5fc141ad0 msgr2=0x7fc5fc15beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.130+0000 7fc6016c0640 1 --2- 172.21.15.16:0/2402147558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5fc141ad0 0x7fc5fc15beb0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400c700 tx=0x7fc5f400cbc0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.130+0000 7fc6016c0640 1 -- 172.21.15.16:0/2402147558 shutdown_connections 2024-09-19T07:53:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.130+0000 7fc6016c0640 1 --2- 172.21.15.16:0/2402147558 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc5e806c2b0 0x7fc5e806e770 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.130+0000 7fc6016c0640 1 --2- 172.21.15.16:0/2402147558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5fc141ad0 0x7fc5fc15beb0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.130+0000 7fc6016c0640 1 --2- 172.21.15.16:0/2402147558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5fc0d21f0 0x7fc5fc15b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.130+0000 7fc6016c0640 1 --2- 172.21.15.16:0/2402147558 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5fc072590 0x7fc5fc1405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.130+0000 7fc6016c0640 1 -- 172.21.15.16:0/2402147558 >> 172.21.15.16:0/2402147558 conn(0x7fc5fc0b3ed0 msgr2=0x7fc5fc0b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:47.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.130+0000 7fc6016c0640 1 -- 172.21.15.16:0/2402147558 shutdown_connections 2024-09-19T07:53:47.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.130+0000 7fc6016c0640 1 -- 172.21.15.16:0/2402147558 wait complete. 2024-09-19T07:53:47.143 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T07:53:47.143 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:53:47.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.245+0000 7f75be00a640 1 Processor -- start 2024-09-19T07:53:47.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.245+0000 7f75be00a640 1 -- start start 2024-09-19T07:53:47.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.246+0000 7f75be00a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f75b806cd20 0x7f75b806d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.246+0000 7f75be00a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f75b81425e0 0x7f75b8142a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.246+0000 7f75be00a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75b806d640 con 0x7f75b8161eb0 2024-09-19T07:53:47.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.246+0000 7f75be00a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f75b8162e50 con 0x7f75b806cd20 2024-09-19T07:53:47.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.246+0000 7f75be00a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f75b8142150 con 0x7f75b81425e0 2024-09-19T07:53:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.246+0000 7f75b77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f75b806cd20 0x7f75b806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.246+0000 7f75b6ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f75b8161eb0 0x7f75b81622f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35634/0 (socket says 172.21.15.16:35634) 2024-09-19T07:53:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.246+0000 7f75b6ffd640 1 -- 172.21.15.16:0/2052585629 learned_addr learned my addr 172.21.15.16:0/2052585629 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.247+0000 7f75b7fff640 1 --2- 172.21.15.16:0/2052585629 >> v2:172.21.15.146:3300/0 conn(0x7f75b81425e0 0x7f75b8142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.247+0000 7f75b77fe640 1 -- 172.21.15.16:0/2052585629 >> v2:172.21.15.146:3300/0 conn(0x7f75b81425e0 msgr2=0x7f75b8142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.247+0000 7f75b77fe640 1 --2- 172.21.15.16:0/2052585629 >> v2:172.21.15.146:3300/0 conn(0x7f75b81425e0 0x7f75b8142a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.247+0000 7f75b77fe640 1 -- 172.21.15.16:0/2052585629 >> v1:172.21.15.16:6789/0 conn(0x7f75b8161eb0 legacy=0x7f75b81622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.247+0000 7f75b77fe640 1 -- 172.21.15.16:0/2052585629 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b8162b70 con 0x7f75b806cd20 2024-09-19T07:53:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.247+0000 7f75b67fc640 1 -- 172.21.15.16:0/2052585629 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1575948688 0 0) 0x7f75b806d640 con 0x7f75b8161eb0 2024-09-19T07:53:47.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.248+0000 7f75b77fe640 1 --2- 172.21.15.16:0/2052585629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b806cd20 0x7f75b806d100 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f75b815c490 tx=0x7f75a802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7127f6d7a8df5d1b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:47.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.248+0000 7f75b67fc640 1 -- 172.21.15.16:0/2052585629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8037070 con 0x7f75b806cd20 2024-09-19T07:53:47.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.248+0000 7f75b67fc640 1 -- 172.21.15.16:0/2052585629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a8032440 con 0x7f75b806cd20 2024-09-19T07:53:47.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.248+0000 7f75b67fc640 1 -- 172.21.15.16:0/2052585629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a802ae60 con 0x7f75b806cd20 2024-09-19T07:53:47.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.248+0000 7f75be00a640 1 -- 172.21.15.16:0/2052585629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b806cd20 msgr2=0x7f75b806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.248+0000 7f75be00a640 1 --2- 172.21.15.16:0/2052585629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b806cd20 0x7f75b806d100 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f75b815c490 tx=0x7f75a802a110 comp rx=0 tx=0).stop 2024-09-19T07:53:47.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.248+0000 7f75be00a640 1 -- 172.21.15.16:0/2052585629 shutdown_connections 2024-09-19T07:53:47.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.248+0000 7f75be00a640 1 --2- 172.21.15.16:0/2052585629 >> v2:172.21.15.146:3300/0 conn(0x7f75b81425e0 0x7f75b8142a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.248+0000 7f75be00a640 1 --2- 172.21.15.16:0/2052585629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b806cd20 0x7f75b806d100 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.248+0000 7f75be00a640 1 -- 172.21.15.16:0/2052585629 >> 172.21.15.16:0/2052585629 conn(0x7f75b80b3ed0 msgr2=0x7f75b80b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:47.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.248+0000 7f75be00a640 1 -- 172.21.15.16:0/2052585629 shutdown_connections 2024-09-19T07:53:47.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.248+0000 7f75be00a640 1 -- 172.21.15.16:0/2052585629 wait complete. 2024-09-19T07:53:47.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.249+0000 7f75be00a640 1 Processor -- start 2024-09-19T07:53:47.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.250+0000 7f75be00a640 1 -- start start 2024-09-19T07:53:47.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.250+0000 7f75be00a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75b806cd20 0x7f75b81586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.250+0000 7f75be00a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75b81425e0 0x7f75b815bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.250+0000 7f75be00a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b8161eb0 0x7f75b8158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.250+0000 7f75be00a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8141600 con 0x7f75b8161eb0 2024-09-19T07:53:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.250+0000 7f75be00a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8141de0 con 0x7f75b806cd20 2024-09-19T07:53:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.250+0000 7f75be00a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f75b8142fb0 con 0x7f75b81425e0 2024-09-19T07:53:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.251+0000 7f75b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b8161eb0 0x7f75b8158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.251+0000 7f75b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b8161eb0 0x7f75b8158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52734/0 (socket says 172.21.15.16:52734) 2024-09-19T07:53:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.251+0000 7f75b7fff640 1 -- 172.21.15.16:0/4198256287 learned_addr learned my addr 172.21.15.16:0/4198256287 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.251+0000 7f75b6ffd640 1 --2- 172.21.15.16:0/4198256287 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75b81425e0 0x7f75b815bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.251+0000 7f75b77fe640 1 --2- 172.21.15.16:0/4198256287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75b806cd20 0x7f75b81586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.251+0000 7f75b7fff640 1 -- 172.21.15.16:0/4198256287 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75b81425e0 msgr2=0x7f75b815bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.251+0000 7f75b7fff640 1 --2- 172.21.15.16:0/4198256287 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75b81425e0 0x7f75b815bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.251+0000 7f75b7fff640 1 -- 172.21.15.16:0/4198256287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75b806cd20 msgr2=0x7f75b81586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.251+0000 7f75b7fff640 1 --2- 172.21.15.16:0/4198256287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75b806cd20 0x7f75b81586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.251+0000 7f75b7fff640 1 -- 172.21.15.16:0/4198256287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b8159b50 con 0x7f75b8161eb0 2024-09-19T07:53:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.251+0000 7f75b6ffd640 1 --2- 172.21.15.16:0/4198256287 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75b81425e0 0x7f75b815bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.251+0000 7f75b77fe640 1 --2- 172.21.15.16:0/4198256287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75b806cd20 0x7f75b81586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.251+0000 7f75b7fff640 1 --2- 172.21.15.16:0/4198256287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b8161eb0 0x7f75b8158f80 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f75b000efc0 tx=0x7f75b000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.251+0000 7f759ffff640 1 -- 172.21.15.16:0/4198256287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b0018020 con 0x7f75b8161eb0 2024-09-19T07:53:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.251+0000 7f759ffff640 1 -- 172.21.15.16:0/4198256287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75b0014480 con 0x7f75b8161eb0 2024-09-19T07:53:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.251+0000 7f75be00a640 1 -- 172.21.15.16:0/4198256287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b8159e30 con 0x7f75b8161eb0 2024-09-19T07:53:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.251+0000 7f759ffff640 1 -- 172.21.15.16:0/4198256287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b0014870 con 0x7f75b8161eb0 2024-09-19T07:53:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.251+0000 7f75be00a640 1 -- 172.21.15.16:0/4198256287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b80b73d0 con 0x7f75b8161eb0 2024-09-19T07:53:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.252+0000 7f75be00a640 1 -- 172.21.15.16:0/4198256287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b8142a70 con 0x7f75b8161eb0 2024-09-19T07:53:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.253+0000 7f759ffff640 1 -- 172.21.15.16:0/4198256287 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f75b0005d10 con 0x7f75b8161eb0 2024-09-19T07:53:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.253+0000 7f759ffff640 1 --2- 172.21.15.16:0/4198256287 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f75a406c2b0 0x7f75a406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.253+0000 7f759ffff640 1 -- 172.21.15.16:0/4198256287 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f75b008c4a0 con 0x7f75b8161eb0 2024-09-19T07:53:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.254+0000 7f75b77fe640 1 --2- 172.21.15.16:0/4198256287 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f75a406c2b0 0x7f75a406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.254+0000 7f75b77fe640 1 --2- 172.21.15.16:0/4198256287 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f75a406c2b0 0x7f75a406e770 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f75a8034000 tx=0x7f75a802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:47.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.258+0000 7f759ffff640 1 -- 172.21.15.16:0/4198256287 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f75b0022030 con 0x7f75b8161eb0 2024-09-19T07:53:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.429+0000 7f75be00a640 1 -- 172.21.15.16:0/4198256287 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f75b80690c0 con 0x7f75a406c2b0 2024-09-19T07:53:47.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.435+0000 7f759ffff640 1 -- 172.21.15.16:0/4198256287 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250763 (secure 0 0 0) 0x7f75b80690c0 con 0x7f75a406c2b0 2024-09-19T07:53:47.439 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:53:47.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.438+0000 7f75be00a640 1 -- 172.21.15.16:0/4198256287 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f75a406c2b0 msgr2=0x7f75a406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.438+0000 7f75be00a640 1 --2- 172.21.15.16:0/4198256287 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f75a406c2b0 0x7f75a406e770 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f75a8034000 tx=0x7f75a802e000 comp rx=0 tx=0).stop 2024-09-19T07:53:47.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.438+0000 7f75be00a640 1 -- 172.21.15.16:0/4198256287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b8161eb0 msgr2=0x7f75b8158f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.438+0000 7f75be00a640 1 --2- 172.21.15.16:0/4198256287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b8161eb0 0x7f75b8158f80 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f75b000efc0 tx=0x7f75b000c490 comp rx=0 tx=0).stop 2024-09-19T07:53:47.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.438+0000 7f75be00a640 1 -- 172.21.15.16:0/4198256287 shutdown_connections 2024-09-19T07:53:47.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.438+0000 7f75be00a640 1 --2- 172.21.15.16:0/4198256287 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f75a406c2b0 0x7f75a406e770 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.438+0000 7f75be00a640 1 --2- 172.21.15.16:0/4198256287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b8161eb0 0x7f75b8158f80 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.438+0000 7f75be00a640 1 --2- 172.21.15.16:0/4198256287 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f75b81425e0 0x7f75b815bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.438+0000 7f75be00a640 1 --2- 172.21.15.16:0/4198256287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75b806cd20 0x7f75b81586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.438+0000 7f75be00a640 1 -- 172.21.15.16:0/4198256287 >> 172.21.15.16:0/4198256287 conn(0x7f75b80b3ed0 msgr2=0x7f75b80b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:47.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.439+0000 7f75be00a640 1 -- 172.21.15.16:0/4198256287 shutdown_connections 2024-09-19T07:53:47.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.439+0000 7f75be00a640 1 -- 172.21.15.16:0/4198256287 wait complete. 2024-09-19T07:53:47.450 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-09-19T07:53:46.610133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591110,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":178,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":762724,"kb_used_data":12392,"kb_used_omap":12,"kb_used_meta":750131,"kb_avail":754211996,"statfs":{"total":773094113280,"available":772313083904,"internally_reserved":0,"allocated":12689408,"data_stored":6495741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":768134704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":742,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.167829"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455920+0000","last_change":"2024-09-19T07:53:44.454037+0000","last_active":"2024-09-19T07:53:44.455920+0000","last_peered":"2024-09-19T07:53:44.455920+0000","last_clean":"2024-09-19T07:53:44.455920+0000","last_became_active":"2024-09-19T07:53:44.453756+0000","last_became_peered":"2024-09-19T07:53:44.453756+0000","last_unstale":"2024-09-19T07:53:44.455920+0000","last_undegraded":"2024-09-19T07:53:44.455920+0000","last_fullsized":"2024-09-19T07:53:44.455920+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:44.305938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.455258+0000","last_change":"2024-09-19T07:53:43.455258+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.455258+0000","last_undegraded":"2024-09-19T07:53:43.455258+0000","last_fullsized":"2024-09-19T07:53:43.455258+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.454692+0000","last_change":"2024-09-19T07:53:43.454692+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.454692+0000","last_undegraded":"2024-09-19T07:53:43.454692+0000","last_fullsized":"2024-09-19T07:53:43.454692+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.471544+0000","last_change":"2024-09-19T07:53:44.463853+0000","last_active":"2024-09-19T07:53:44.471544+0000","last_peered":"2024-09-19T07:53:44.471544+0000","last_clean":"2024-09-19T07:53:44.471544+0000","last_became_active":"2024-09-19T07:53:44.463569+0000","last_became_peered":"2024-09-19T07:53:44.463569+0000","last_unstale":"2024-09-19T07:53:44.471544+0000","last_undegraded":"2024-09-19T07:53:44.471544+0000","last_fullsized":"2024-09-19T07:53:44.471544+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:43.497533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.462115+0000","last_change":"2024-09-19T07:53:44.460542+0000","last_active":"2024-09-19T07:53:44.462115+0000","last_peered":"2024-09-19T07:53:44.462115+0000","last_clean":"2024-09-19T07:53:44.462115+0000","last_became_active":"2024-09-19T07:53:44.460245+0000","last_became_peered":"2024-09-19T07:53:44.460245+0000","last_unstale":"2024-09-19T07:53:44.462115+0000","last_undegraded":"2024-09-19T07:53:44.462115+0000","last_fullsized":"2024-09-19T07:53:44.462115+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:39:04.627509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.482181+0000","last_change":"2024-09-19T07:53:44.480188+0000","last_active":"2024-09-19T07:53:44.482181+0000","last_peered":"2024-09-19T07:53:44.482181+0000","last_clean":"2024-09-19T07:53:44.482181+0000","last_became_active":"2024-09-19T07:53:44.479867+0000","last_became_peered":"2024-09-19T07:53:44.479867+0000","last_unstale":"2024-09-19T07:53:44.482181+0000","last_undegraded":"2024-09-19T07:53:44.482181+0000","last_fullsized":"2024-09-19T07:53:44.482181+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:21.929840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.477127+0000","last_change":"2024-09-19T07:53:44.475378+0000","last_active":"2024-09-19T07:53:44.477127+0000","last_peered":"2024-09-19T07:53:44.477127+0000","last_clean":"2024-09-19T07:53:44.477127+0000","last_became_active":"2024-09-19T07:53:44.475024+0000","last_became_peered":"2024-09-19T07:53:44.475024+0000","last_unstale":"2024-09-19T07:53:44.477127+0000","last_undegraded":"2024-09-19T07:53:44.477127+0000","last_fullsized":"2024-09-19T07:53:44.477127+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:13.004954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.454146+0000","last_change":"2024-09-19T07:53:43.454146+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.454146+0000","last_undegraded":"2024-09-19T07:53:43.454146+0000","last_fullsized":"2024-09-19T07:53:43.454146+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.474046+0000","last_change":"2024-09-19T07:53:44.472235+0000","last_active":"2024-09-19T07:53:44.474046+0000","last_peered":"2024-09-19T07:53:44.474046+0000","last_clean":"2024-09-19T07:53:44.474046+0000","last_became_active":"2024-09-19T07:53:44.471961+0000","last_became_peered":"2024-09-19T07:53:44.471961+0000","last_unstale":"2024-09-19T07:53:44.474046+0000","last_undegraded":"2024-09-19T07:53:44.474046+0000","last_fullsized":"2024-09-19T07:53:44.474046+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:03.068372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.450946+0000","last_change":"2024-09-19T07:53:43.450946+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.450946+0000","last_undegraded":"2024-09-19T07:53:43.450946+0000","last_fullsized":"2024-09-19T07:53:43.450946+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.462065+0000","last_change":"2024-09-19T07:53:44.460409+0000","last_active":"2024-09-19T07:53:44.462065+0000","last_peered":"2024-09-19T07:53:44.462065+0000","last_clean":"2024-09-19T07:53:44.462065+0000","last_became_active":"2024-09-19T07:53:44.460137+0000","last_became_peered":"2024-09-19T07:53:44.460137+0000","last_unstale":"2024-09-19T07:53:44.462065+0000","last_undegraded":"2024-09-19T07:53:44.462065+0000","last_fullsized":"2024-09-19T07:53:44.462065+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:25.476480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.472912+0000","last_change":"2024-09-19T07:53:44.467120+0000","last_active":"2024-09-19T07:53:44.472912+0000","last_peered":"2024-09-19T07:53:44.472912+0000","last_clean":"2024-09-19T07:53:44.472912+0000","last_became_active":"2024-09-19T07:53:44.466828+0000","last_became_peered":"2024-09-19T07:53:44.466828+0000","last_unstale":"2024-09-19T07:53:44.472912+0000","last_undegraded":"2024-09-19T07:53:44.472912+0000","last_fullsized":"2024-09-19T07:53:44.472912+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:00.047940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.454391+0000","last_change":"2024-09-19T07:53:43.454391+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.454391+0000","last_undegraded":"2024-09-19T07:53:43.454391+0000","last_fullsized":"2024-09-19T07:53:43.454391+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453644+0000","last_change":"2024-09-19T07:53:43.453644+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453644+0000","last_undegraded":"2024-09-19T07:53:43.453644+0000","last_fullsized":"2024-09-19T07:53:43.453644+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.456674+0000","last_change":"2024-09-19T07:53:44.455460+0000","last_active":"2024-09-19T07:53:44.456674+0000","last_peered":"2024-09-19T07:53:44.456674+0000","last_clean":"2024-09-19T07:53:44.456674+0000","last_became_active":"2024-09-19T07:53:44.455164+0000","last_became_peered":"2024-09-19T07:53:44.455164+0000","last_unstale":"2024-09-19T07:53:44.456674+0000","last_undegraded":"2024-09-19T07:53:44.456674+0000","last_fullsized":"2024-09-19T07:53:44.456674+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:40.200573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.476417+0000","last_change":"2024-09-19T07:53:44.473808+0000","last_active":"2024-09-19T07:53:44.476417+0000","last_peered":"2024-09-19T07:53:44.476417+0000","last_clean":"2024-09-19T07:53:44.476417+0000","last_became_active":"2024-09-19T07:53:44.473528+0000","last_became_peered":"2024-09-19T07:53:44.473528+0000","last_unstale":"2024-09-19T07:53:44.476417+0000","last_undegraded":"2024-09-19T07:53:44.476417+0000","last_fullsized":"2024-09-19T07:53:44.476417+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:19.095190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453835+0000","last_change":"2024-09-19T07:53:43.453835+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453835+0000","last_undegraded":"2024-09-19T07:53:43.453835+0000","last_fullsized":"2024-09-19T07:53:43.453835+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453214+0000","last_change":"2024-09-19T07:53:43.453214+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453214+0000","last_undegraded":"2024-09-19T07:53:43.453214+0000","last_fullsized":"2024-09-19T07:53:43.453214+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.471284+0000","last_change":"2024-09-19T07:53:44.465304+0000","last_active":"2024-09-19T07:53:44.471284+0000","last_peered":"2024-09-19T07:53:44.471284+0000","last_clean":"2024-09-19T07:53:44.471284+0000","last_became_active":"2024-09-19T07:53:44.465044+0000","last_became_peered":"2024-09-19T07:53:44.465044+0000","last_unstale":"2024-09-19T07:53:44.471284+0000","last_undegraded":"2024-09-19T07:53:44.471284+0000","last_fullsized":"2024-09-19T07:53:44.471284+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:40.200573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955372+0000","last_change":"2024-09-19T07:53:44.466484+0000","last_active":"2024-09-19T07:53:45.955372+0000","last_peered":"2024-09-19T07:53:45.955372+0000","last_clean":"2024-09-19T07:53:45.955372+0000","last_became_active":"2024-09-19T07:53:44.466195+0000","last_became_peered":"2024-09-19T07:53:44.466195+0000","last_unstale":"2024-09-19T07:53:45.955372+0000","last_undegraded":"2024-09-19T07:53:45.955372+0000","last_fullsized":"2024-09-19T07:53:45.955372+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:20.436455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.473757+0000","last_change":"2024-09-19T07:53:44.472944+0000","last_active":"2024-09-19T07:53:44.473757+0000","last_peered":"2024-09-19T07:53:44.473757+0000","last_clean":"2024-09-19T07:53:44.473757+0000","last_became_active":"2024-09-19T07:53:44.472669+0000","last_became_peered":"2024-09-19T07:53:44.472669+0000","last_unstale":"2024-09-19T07:53:44.473757+0000","last_undegraded":"2024-09-19T07:53:44.473757+0000","last_fullsized":"2024-09-19T07:53:44.473757+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:05.983399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470250+0000","last_change":"2024-09-19T07:53:44.466834+0000","last_active":"2024-09-19T07:53:44.470250+0000","last_peered":"2024-09-19T07:53:44.470250+0000","last_clean":"2024-09-19T07:53:44.470250+0000","last_became_active":"2024-09-19T07:53:44.466204+0000","last_became_peered":"2024-09-19T07:53:44.466204+0000","last_unstale":"2024-09-19T07:53:44.470250+0000","last_undegraded":"2024-09-19T07:53:44.470250+0000","last_fullsized":"2024-09-19T07:53:44.470250+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:00.929562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.481470+0000","last_change":"2024-09-19T07:53:44.480566+0000","last_active":"2024-09-19T07:53:44.481470+0000","last_peered":"2024-09-19T07:53:44.481470+0000","last_clean":"2024-09-19T07:53:44.481470+0000","last_became_active":"2024-09-19T07:53:44.480280+0000","last_became_peered":"2024-09-19T07:53:44.480280+0000","last_unstale":"2024-09-19T07:53:44.481470+0000","last_undegraded":"2024-09-19T07:53:44.481470+0000","last_fullsized":"2024-09-19T07:53:44.481470+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:46.275247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.464980+0000","last_change":"2024-09-19T07:53:44.462832+0000","last_active":"2024-09-19T07:53:44.464980+0000","last_peered":"2024-09-19T07:53:44.464980+0000","last_clean":"2024-09-19T07:53:44.464980+0000","last_became_active":"2024-09-19T07:53:44.462538+0000","last_became_peered":"2024-09-19T07:53:44.462538+0000","last_unstale":"2024-09-19T07:53:44.464980+0000","last_undegraded":"2024-09-19T07:53:44.464980+0000","last_fullsized":"2024-09-19T07:53:44.464980+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:45.343582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.465738+0000","last_change":"2024-09-19T07:53:44.460249+0000","last_active":"2024-09-19T07:53:44.465738+0000","last_peered":"2024-09-19T07:53:44.465738+0000","last_clean":"2024-09-19T07:53:44.465738+0000","last_became_active":"2024-09-19T07:53:44.458793+0000","last_became_peered":"2024-09-19T07:53:44.458793+0000","last_unstale":"2024-09-19T07:53:44.465738+0000","last_undegraded":"2024-09-19T07:53:44.465738+0000","last_fullsized":"2024-09-19T07:53:44.465738+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:37.126896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955551+0000","last_change":"2024-09-19T07:53:44.467665+0000","last_active":"2024-09-19T07:53:45.955551+0000","last_peered":"2024-09-19T07:53:45.955551+0000","last_clean":"2024-09-19T07:53:45.955551+0000","last_became_active":"2024-09-19T07:53:44.467388+0000","last_became_peered":"2024-09-19T07:53:44.467388+0000","last_unstale":"2024-09-19T07:53:45.955551+0000","last_undegraded":"2024-09-19T07:53:45.955551+0000","last_fullsized":"2024-09-19T07:53:45.955551+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:09.122218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.464494+0000","last_change":"2024-09-19T07:53:44.463945+0000","last_active":"2024-09-19T07:53:44.464494+0000","last_peered":"2024-09-19T07:53:44.464494+0000","last_clean":"2024-09-19T07:53:44.464494+0000","last_became_active":"2024-09-19T07:53:44.463622+0000","last_became_peered":"2024-09-19T07:53:44.463622+0000","last_unstale":"2024-09-19T07:53:44.464494+0000","last_undegraded":"2024-09-19T07:53:44.464494+0000","last_fullsized":"2024-09-19T07:53:44.464494+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:44.970456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.480727+0000","last_change":"2024-09-19T07:53:44.479121+0000","last_active":"2024-09-19T07:53:44.480727+0000","last_peered":"2024-09-19T07:53:44.480727+0000","last_clean":"2024-09-19T07:53:44.480727+0000","last_became_active":"2024-09-19T07:53:44.478632+0000","last_became_peered":"2024-09-19T07:53:44.478632+0000","last_unstale":"2024-09-19T07:53:44.480727+0000","last_undegraded":"2024-09-19T07:53:44.480727+0000","last_fullsized":"2024-09-19T07:53:44.480727+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:58.195995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955651+0000","last_change":"2024-09-19T07:53:44.471531+0000","last_active":"2024-09-19T07:53:45.955651+0000","last_peered":"2024-09-19T07:53:45.955651+0000","last_clean":"2024-09-19T07:53:45.955651+0000","last_became_active":"2024-09-19T07:53:44.471247+0000","last_became_peered":"2024-09-19T07:53:44.471247+0000","last_unstale":"2024-09-19T07:53:45.955651+0000","last_undegraded":"2024-09-19T07:53:45.955651+0000","last_fullsized":"2024-09-19T07:53:45.955651+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:27:11.047914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.452495+0000","last_change":"2024-09-19T07:53:43.452495+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.452495+0000","last_undegraded":"2024-09-19T07:53:43.452495+0000","last_fullsized":"2024-09-19T07:53:43.452495+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.464284+0000","last_change":"2024-09-19T07:53:44.461893+0000","last_active":"2024-09-19T07:53:44.464284+0000","last_peered":"2024-09-19T07:53:44.464284+0000","last_clean":"2024-09-19T07:53:44.464284+0000","last_became_active":"2024-09-19T07:53:44.461543+0000","last_became_peered":"2024-09-19T07:53:44.461543+0000","last_unstale":"2024-09-19T07:53:44.464284+0000","last_undegraded":"2024-09-19T07:53:44.464284+0000","last_fullsized":"2024-09-19T07:53:44.464284+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:02.071494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.464955+0000","last_change":"2024-09-19T07:53:44.462124+0000","last_active":"2024-09-19T07:53:44.464955+0000","last_peered":"2024-09-19T07:53:44.464955+0000","last_clean":"2024-09-19T07:53:44.464955+0000","last_became_active":"2024-09-19T07:53:44.461823+0000","last_became_peered":"2024-09-19T07:53:44.461823+0000","last_unstale":"2024-09-19T07:53:44.464955+0000","last_undegraded":"2024-09-19T07:53:44.464955+0000","last_fullsized":"2024-09-19T07:53:44.464955+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:19.276802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453302+0000","last_change":"2024-09-19T07:53:43.453302+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453302+0000","last_undegraded":"2024-09-19T07:53:43.453302+0000","last_fullsized":"2024-09-19T07:53:43.453302+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"117'4","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475117+0000","last_change":"2024-09-19T07:53:43.462786+0000","last_active":"2024-09-19T07:53:44.475117+0000","last_peered":"2024-09-19T07:53:44.475117+0000","last_clean":"2024-09-19T07:53:44.475117+0000","last_became_active":"2024-09-19T07:53:43.462517+0000","last_became_peered":"2024-09-19T07:53:43.462517+0000","last_unstale":"2024-09-19T07:53:44.475117+0000","last_undegraded":"2024-09-19T07:53:44.475117+0000","last_fullsized":"2024-09-19T07:53:44.475117+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:55.555956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.483675+0000","last_change":"2024-09-19T07:53:44.482782+0000","last_active":"2024-09-19T07:53:44.483675+0000","last_peered":"2024-09-19T07:53:44.483675+0000","last_clean":"2024-09-19T07:53:44.483675+0000","last_became_active":"2024-09-19T07:53:44.482497+0000","last_became_peered":"2024-09-19T07:53:44.482497+0000","last_unstale":"2024-09-19T07:53:44.483675+0000","last_undegraded":"2024-09-19T07:53:44.483675+0000","last_fullsized":"2024-09-19T07:53:44.483675+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:03.706766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.460833+0000","last_change":"2024-09-19T07:53:43.457859+0000","last_active":"2024-09-19T07:53:43.460833+0000","last_peered":"2024-09-19T07:53:43.460833+0000","last_clean":"2024-09-19T07:53:43.460833+0000","last_became_active":"2024-09-19T07:53:43.457574+0000","last_became_peered":"2024-09-19T07:53:43.457574+0000","last_unstale":"2024-09-19T07:53:43.460833+0000","last_undegraded":"2024-09-19T07:53:43.460833+0000","last_fullsized":"2024-09-19T07:53:43.460833+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:21.380860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.469707+0000","last_change":"2024-09-19T07:53:44.468508+0000","last_active":"2024-09-19T07:53:44.469707+0000","last_peered":"2024-09-19T07:53:44.469707+0000","last_clean":"2024-09-19T07:53:44.469707+0000","last_became_active":"2024-09-19T07:53:44.468228+0000","last_became_peered":"2024-09-19T07:53:44.468228+0000","last_unstale":"2024-09-19T07:53:44.469707+0000","last_undegraded":"2024-09-19T07:53:44.469707+0000","last_fullsized":"2024-09-19T07:53:44.469707+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.258505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.480926+0000","last_change":"2024-09-19T07:53:43.464281+0000","last_active":"2024-09-19T07:53:44.480926+0000","last_peered":"2024-09-19T07:53:44.480926+0000","last_clean":"2024-09-19T07:53:44.480926+0000","last_became_active":"2024-09-19T07:53:43.464005+0000","last_became_peered":"2024-09-19T07:53:43.464005+0000","last_unstale":"2024-09-19T07:53:44.480926+0000","last_undegraded":"2024-09-19T07:53:44.480926+0000","last_fullsized":"2024-09-19T07:53:44.480926+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:40.739604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470859+0000","last_change":"2024-09-19T07:53:44.463658+0000","last_active":"2024-09-19T07:53:44.470859+0000","last_peered":"2024-09-19T07:53:44.470859+0000","last_clean":"2024-09-19T07:53:44.470859+0000","last_became_active":"2024-09-19T07:53:44.463373+0000","last_became_peered":"2024-09-19T07:53:44.463373+0000","last_unstale":"2024-09-19T07:53:44.470859+0000","last_undegraded":"2024-09-19T07:53:44.470859+0000","last_fullsized":"2024-09-19T07:53:44.470859+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:05.734979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.461452+0000","last_change":"2024-09-19T07:53:43.459863+0000","last_active":"2024-09-19T07:53:43.461452+0000","last_peered":"2024-09-19T07:53:43.461452+0000","last_clean":"2024-09-19T07:53:43.461452+0000","last_became_active":"2024-09-19T07:53:43.459538+0000","last_became_peered":"2024-09-19T07:53:43.459538+0000","last_unstale":"2024-09-19T07:53:43.461452+0000","last_undegraded":"2024-09-19T07:53:43.461452+0000","last_fullsized":"2024-09-19T07:53:43.461452+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:21.308817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.450580+0000","last_change":"2024-09-19T07:53:43.450580+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.450580+0000","last_undegraded":"2024-09-19T07:53:43.450580+0000","last_fullsized":"2024-09-19T07:53:43.450580+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"117'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.467472+0000","last_change":"2024-09-19T07:53:43.459625+0000","last_active":"2024-09-19T07:53:44.467472+0000","last_peered":"2024-09-19T07:53:44.467472+0000","last_clean":"2024-09-19T07:53:44.467472+0000","last_became_active":"2024-09-19T07:53:43.459283+0000","last_became_peered":"2024-09-19T07:53:43.459283+0000","last_unstale":"2024-09-19T07:53:44.467472+0000","last_undegraded":"2024-09-19T07:53:44.467472+0000","last_fullsized":"2024-09-19T07:53:44.467472+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:37.240608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.463008+0000","last_change":"2024-09-19T07:53:44.461446+0000","last_active":"2024-09-19T07:53:44.463008+0000","last_peered":"2024-09-19T07:53:44.463008+0000","last_clean":"2024-09-19T07:53:44.463008+0000","last_became_active":"2024-09-19T07:53:44.461158+0000","last_became_peered":"2024-09-19T07:53:44.461158+0000","last_unstale":"2024-09-19T07:53:44.463008+0000","last_undegraded":"2024-09-19T07:53:44.463008+0000","last_fullsized":"2024-09-19T07:53:44.463008+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:13.056229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"117'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.485413+0000","last_change":"2024-09-19T07:53:43.467953+0000","last_active":"2024-09-19T07:53:44.485413+0000","last_peered":"2024-09-19T07:53:44.485413+0000","last_clean":"2024-09-19T07:53:44.485413+0000","last_became_active":"2024-09-19T07:53:43.467684+0000","last_became_peered":"2024-09-19T07:53:43.467684+0000","last_unstale":"2024-09-19T07:53:44.485413+0000","last_undegraded":"2024-09-19T07:53:44.485413+0000","last_fullsized":"2024-09-19T07:53:44.485413+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:46.197768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.451717+0000","last_change":"2024-09-19T07:53:43.451717+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.451717+0000","last_undegraded":"2024-09-19T07:53:43.451717+0000","last_fullsized":"2024-09-19T07:53:43.451717+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.453345+0000","last_change":"2024-09-19T07:53:43.458791+0000","last_active":"2024-09-19T07:53:44.453345+0000","last_peered":"2024-09-19T07:53:44.453345+0000","last_clean":"2024-09-19T07:53:44.453345+0000","last_became_active":"2024-09-19T07:53:43.458509+0000","last_became_peered":"2024-09-19T07:53:43.458509+0000","last_unstale":"2024-09-19T07:53:44.453345+0000","last_undegraded":"2024-09-19T07:53:44.453345+0000","last_fullsized":"2024-09-19T07:53:44.453345+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:54.555152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453976+0000","last_change":"2024-09-19T07:53:43.453976+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453976+0000","last_undegraded":"2024-09-19T07:53:43.453976+0000","last_fullsized":"2024-09-19T07:53:43.453976+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.460241+0000","last_change":"2024-09-19T07:53:43.466824+0000","last_active":"2024-09-19T07:53:44.460241+0000","last_peered":"2024-09-19T07:53:44.460241+0000","last_clean":"2024-09-19T07:53:44.460241+0000","last_became_active":"2024-09-19T07:53:43.466546+0000","last_became_peered":"2024-09-19T07:53:43.466546+0000","last_unstale":"2024-09-19T07:53:44.460241+0000","last_undegraded":"2024-09-19T07:53:44.460241+0000","last_fullsized":"2024-09-19T07:53:44.460241+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:52.318252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470048+0000","last_change":"2024-09-19T07:53:44.464599+0000","last_active":"2024-09-19T07:53:44.470048+0000","last_peered":"2024-09-19T07:53:44.470048+0000","last_clean":"2024-09-19T07:53:44.470048+0000","last_became_active":"2024-09-19T07:53:44.464335+0000","last_became_peered":"2024-09-19T07:53:44.464335+0000","last_unstale":"2024-09-19T07:53:44.470048+0000","last_undegraded":"2024-09-19T07:53:44.470048+0000","last_fullsized":"2024-09-19T07:53:44.470048+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:44.305938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.456285+0000","last_change":"2024-09-19T07:53:43.468918+0000","last_active":"2024-09-19T07:53:44.456285+0000","last_peered":"2024-09-19T07:53:44.456285+0000","last_clean":"2024-09-19T07:53:44.456285+0000","last_became_active":"2024-09-19T07:53:43.468594+0000","last_became_peered":"2024-09-19T07:53:43.468594+0000","last_unstale":"2024-09-19T07:53:44.456285+0000","last_undegraded":"2024-09-19T07:53:44.456285+0000","last_fullsized":"2024-09-19T07:53:44.456285+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:47.768851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.469144+0000","last_change":"2024-09-19T07:53:44.466217+0000","last_active":"2024-09-19T07:53:44.469144+0000","last_peered":"2024-09-19T07:53:44.469144+0000","last_clean":"2024-09-19T07:53:44.469144+0000","last_became_active":"2024-09-19T07:53:44.465405+0000","last_became_peered":"2024-09-19T07:53:44.465405+0000","last_unstale":"2024-09-19T07:53:44.469144+0000","last_undegraded":"2024-09-19T07:53:44.469144+0000","last_fullsized":"2024-09-19T07:53:44.469144+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:41.741615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.454379+0000","last_change":"2024-09-19T07:53:43.463233+0000","last_active":"2024-09-19T07:53:44.454379+0000","last_peered":"2024-09-19T07:53:44.454379+0000","last_clean":"2024-09-19T07:53:44.454379+0000","last_became_active":"2024-09-19T07:53:43.462620+0000","last_became_peered":"2024-09-19T07:53:43.462620+0000","last_unstale":"2024-09-19T07:53:44.454379+0000","last_undegraded":"2024-09-19T07:53:44.454379+0000","last_fullsized":"2024-09-19T07:53:44.454379+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:57:59.927552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.450354+0000","last_change":"2024-09-19T07:53:43.450354+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.450354+0000","last_undegraded":"2024-09-19T07:53:43.450354+0000","last_fullsized":"2024-09-19T07:53:43.450354+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"117'3","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475321+0000","last_change":"2024-09-19T07:53:43.463469+0000","last_active":"2024-09-19T07:53:44.475321+0000","last_peered":"2024-09-19T07:53:44.475321+0000","last_clean":"2024-09-19T07:53:44.475321+0000","last_became_active":"2024-09-19T07:53:43.463042+0000","last_became_peered":"2024-09-19T07:53:43.463042+0000","last_unstale":"2024-09-19T07:53:44.475321+0000","last_undegraded":"2024-09-19T07:53:44.475321+0000","last_fullsized":"2024-09-19T07:53:44.475321+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:32.163267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.469249+0000","last_change":"2024-09-19T07:53:44.466514+0000","last_active":"2024-09-19T07:53:44.469249+0000","last_peered":"2024-09-19T07:53:44.469249+0000","last_clean":"2024-09-19T07:53:44.469249+0000","last_became_active":"2024-09-19T07:53:44.464961+0000","last_became_peered":"2024-09-19T07:53:44.464961+0000","last_unstale":"2024-09-19T07:53:44.469249+0000","last_undegraded":"2024-09-19T07:53:44.469249+0000","last_fullsized":"2024-09-19T07:53:44.469249+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:49.561080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.465379+0000","last_change":"2024-09-19T07:53:43.460277+0000","last_active":"2024-09-19T07:53:43.465379+0000","last_peered":"2024-09-19T07:53:43.465379+0000","last_clean":"2024-09-19T07:53:43.465379+0000","last_became_active":"2024-09-19T07:53:43.460028+0000","last_became_peered":"2024-09-19T07:53:43.460028+0000","last_unstale":"2024-09-19T07:53:43.465379+0000","last_undegraded":"2024-09-19T07:53:43.465379+0000","last_fullsized":"2024-09-19T07:53:43.465379+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:27.528465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475155+0000","last_change":"2024-09-19T07:53:44.474267+0000","last_active":"2024-09-19T07:53:44.475155+0000","last_peered":"2024-09-19T07:53:44.475155+0000","last_clean":"2024-09-19T07:53:44.475155+0000","last_became_active":"2024-09-19T07:53:44.473919+0000","last_became_peered":"2024-09-19T07:53:44.473919+0000","last_unstale":"2024-09-19T07:53:44.475155+0000","last_undegraded":"2024-09-19T07:53:44.475155+0000","last_fullsized":"2024-09-19T07:53:44.475155+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:25.476480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.465360+0000","last_change":"2024-09-19T07:53:43.463274+0000","last_active":"2024-09-19T07:53:43.465360+0000","last_peered":"2024-09-19T07:53:43.465360+0000","last_clean":"2024-09-19T07:53:43.465360+0000","last_became_active":"2024-09-19T07:53:43.463007+0000","last_became_peered":"2024-09-19T07:53:43.463007+0000","last_unstale":"2024-09-19T07:53:43.465360+0000","last_undegraded":"2024-09-19T07:53:43.465360+0000","last_fullsized":"2024-09-19T07:53:43.465360+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:01.485835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453196+0000","last_change":"2024-09-19T07:53:43.453196+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453196+0000","last_undegraded":"2024-09-19T07:53:43.453196+0000","last_fullsized":"2024-09-19T07:53:43.453196+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.457723+0000","last_change":"2024-09-19T07:53:43.455019+0000","last_active":"2024-09-19T07:53:43.457723+0000","last_peered":"2024-09-19T07:53:43.457723+0000","last_clean":"2024-09-19T07:53:43.457723+0000","last_became_active":"2024-09-19T07:53:43.454686+0000","last_became_peered":"2024-09-19T07:53:43.454686+0000","last_unstale":"2024-09-19T07:53:43.457723+0000","last_undegraded":"2024-09-19T07:53:43.457723+0000","last_fullsized":"2024-09-19T07:53:43.457723+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:07.256495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956646+0000","last_change":"2024-09-19T07:53:44.457728+0000","last_active":"2024-09-19T07:53:45.956646+0000","last_peered":"2024-09-19T07:53:45.956646+0000","last_clean":"2024-09-19T07:53:45.956646+0000","last_became_active":"2024-09-19T07:53:44.457456+0000","last_became_peered":"2024-09-19T07:53:44.457456+0000","last_unstale":"2024-09-19T07:53:45.956646+0000","last_undegraded":"2024-09-19T07:53:45.956646+0000","last_fullsized":"2024-09-19T07:53:45.956646+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:37:56.076282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.468120+0000","last_change":"2024-09-19T07:53:43.466167+0000","last_active":"2024-09-19T07:53:43.468120+0000","last_peered":"2024-09-19T07:53:43.468120+0000","last_clean":"2024-09-19T07:53:43.468120+0000","last_became_active":"2024-09-19T07:53:43.465882+0000","last_became_peered":"2024-09-19T07:53:43.465882+0000","last_unstale":"2024-09-19T07:53:43.468120+0000","last_undegraded":"2024-09-19T07:53:43.468120+0000","last_fullsized":"2024-09-19T07:53:43.468120+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:04.981388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470860+0000","last_change":"2024-09-19T07:53:44.469161+0000","last_active":"2024-09-19T07:53:44.470860+0000","last_peered":"2024-09-19T07:53:44.470860+0000","last_clean":"2024-09-19T07:53:44.470860+0000","last_became_active":"2024-09-19T07:53:44.468896+0000","last_became_peered":"2024-09-19T07:53:44.468896+0000","last_unstale":"2024-09-19T07:53:44.470860+0000","last_undegraded":"2024-09-19T07:53:44.470860+0000","last_fullsized":"2024-09-19T07:53:44.470860+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:43.497533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.469677+0000","last_change":"2024-09-19T07:53:43.468990+0000","last_active":"2024-09-19T07:53:43.469677+0000","last_peered":"2024-09-19T07:53:43.469677+0000","last_clean":"2024-09-19T07:53:43.469677+0000","last_became_active":"2024-09-19T07:53:43.468625+0000","last_became_peered":"2024-09-19T07:53:43.468625+0000","last_unstale":"2024-09-19T07:53:43.469677+0000","last_undegraded":"2024-09-19T07:53:43.469677+0000","last_fullsized":"2024-09-19T07:53:43.469677+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:55.555956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470204+0000","last_change":"2024-09-19T07:53:44.466474+0000","last_active":"2024-09-19T07:53:44.470204+0000","last_peered":"2024-09-19T07:53:44.470204+0000","last_clean":"2024-09-19T07:53:44.470204+0000","last_became_active":"2024-09-19T07:53:44.466230+0000","last_became_peered":"2024-09-19T07:53:44.466230+0000","last_unstale":"2024-09-19T07:53:44.470204+0000","last_undegraded":"2024-09-19T07:53:44.470204+0000","last_fullsized":"2024-09-19T07:53:44.470204+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:55.557163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955645+0000","last_change":"2024-09-19T07:53:43.463386+0000","last_active":"2024-09-19T07:53:45.955645+0000","last_peered":"2024-09-19T07:53:45.955645+0000","last_clean":"2024-09-19T07:53:45.955645+0000","last_became_active":"2024-09-19T07:53:43.463020+0000","last_became_peered":"2024-09-19T07:53:43.463020+0000","last_unstale":"2024-09-19T07:53:45.955645+0000","last_undegraded":"2024-09-19T07:53:45.955645+0000","last_fullsized":"2024-09-19T07:53:45.955645+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:01.309102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.482421+0000","last_change":"2024-09-19T07:53:44.480939+0000","last_active":"2024-09-19T07:53:44.482421+0000","last_peered":"2024-09-19T07:53:44.482421+0000","last_clean":"2024-09-19T07:53:44.482421+0000","last_became_active":"2024-09-19T07:53:44.480634+0000","last_became_peered":"2024-09-19T07:53:44.480634+0000","last_unstale":"2024-09-19T07:53:44.482421+0000","last_undegraded":"2024-09-19T07:53:44.482421+0000","last_fullsized":"2024-09-19T07:53:44.482421+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:58.089951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"117'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475895+0000","last_change":"2024-09-19T07:53:43.464937+0000","last_active":"2024-09-19T07:53:44.475895+0000","last_peered":"2024-09-19T07:53:44.475895+0000","last_clean":"2024-09-19T07:53:44.475895+0000","last_became_active":"2024-09-19T07:53:43.464614+0000","last_became_peered":"2024-09-19T07:53:43.464614+0000","last_unstale":"2024-09-19T07:53:44.475895+0000","last_undegraded":"2024-09-19T07:53:44.475895+0000","last_fullsized":"2024-09-19T07:53:44.475895+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:48.559070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.469270+0000","last_change":"2024-09-19T07:53:44.466748+0000","last_active":"2024-09-19T07:53:44.469270+0000","last_peered":"2024-09-19T07:53:44.469270+0000","last_clean":"2024-09-19T07:53:44.469270+0000","last_became_active":"2024-09-19T07:53:44.465355+0000","last_became_peered":"2024-09-19T07:53:44.465355+0000","last_unstale":"2024-09-19T07:53:44.469270+0000","last_undegraded":"2024-09-19T07:53:44.469270+0000","last_fullsized":"2024-09-19T07:53:44.469270+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:05.734979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.456785+0000","last_change":"2024-09-19T07:53:43.466472+0000","last_active":"2024-09-19T07:53:44.456785+0000","last_peered":"2024-09-19T07:53:44.456785+0000","last_clean":"2024-09-19T07:53:44.456785+0000","last_became_active":"2024-09-19T07:53:43.466216+0000","last_became_peered":"2024-09-19T07:53:43.466216+0000","last_unstale":"2024-09-19T07:53:44.456785+0000","last_undegraded":"2024-09-19T07:53:44.456785+0000","last_fullsized":"2024-09-19T07:53:44.456785+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:44.341571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.482841+0000","last_change":"2024-09-19T07:53:44.480405+0000","last_active":"2024-09-19T07:53:44.482841+0000","last_peered":"2024-09-19T07:53:44.482841+0000","last_clean":"2024-09-19T07:53:44.482841+0000","last_became_active":"2024-09-19T07:53:44.479696+0000","last_became_peered":"2024-09-19T07:53:44.479696+0000","last_unstale":"2024-09-19T07:53:44.482841+0000","last_undegraded":"2024-09-19T07:53:44.482841+0000","last_fullsized":"2024-09-19T07:53:44.482841+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:23.633013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.476888+0000","last_change":"2024-09-19T07:53:43.463463+0000","last_active":"2024-09-19T07:53:44.476888+0000","last_peered":"2024-09-19T07:53:44.476888+0000","last_clean":"2024-09-19T07:53:44.476888+0000","last_became_active":"2024-09-19T07:53:43.463197+0000","last_became_peered":"2024-09-19T07:53:43.463197+0000","last_unstale":"2024-09-19T07:53:44.476888+0000","last_undegraded":"2024-09-19T07:53:44.476888+0000","last_fullsized":"2024-09-19T07:53:44.476888+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:27.266942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.451887+0000","last_change":"2024-09-19T07:53:43.451887+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.451887+0000","last_undegraded":"2024-09-19T07:53:43.451887+0000","last_fullsized":"2024-09-19T07:53:43.451887+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.456607+0000","last_change":"2024-09-19T07:53:43.465167+0000","last_active":"2024-09-19T07:53:44.456607+0000","last_peered":"2024-09-19T07:53:44.456607+0000","last_clean":"2024-09-19T07:53:44.456607+0000","last_became_active":"2024-09-19T07:53:43.464875+0000","last_became_peered":"2024-09-19T07:53:43.464875+0000","last_unstale":"2024-09-19T07:53:44.456607+0000","last_undegraded":"2024-09-19T07:53:44.456607+0000","last_fullsized":"2024-09-19T07:53:44.456607+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:18.274791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.450312+0000","last_change":"2024-09-19T07:53:43.450312+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.450312+0000","last_undegraded":"2024-09-19T07:53:43.450312+0000","last_fullsized":"2024-09-19T07:53:43.450312+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.468095+0000","last_change":"2024-09-19T07:53:43.465392+0000","last_active":"2024-09-19T07:53:43.468095+0000","last_peered":"2024-09-19T07:53:43.468095+0000","last_clean":"2024-09-19T07:53:43.468095+0000","last_became_active":"2024-09-19T07:53:43.465112+0000","last_became_peered":"2024-09-19T07:53:43.465112+0000","last_unstale":"2024-09-19T07:53:43.468095+0000","last_undegraded":"2024-09-19T07:53:43.468095+0000","last_fullsized":"2024-09-19T07:53:43.468095+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:02.066362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.462713+0000","last_change":"2024-09-19T07:53:44.461101+0000","last_active":"2024-09-19T07:53:44.462713+0000","last_peered":"2024-09-19T07:53:44.462713+0000","last_clean":"2024-09-19T07:53:44.462713+0000","last_became_active":"2024-09-19T07:53:44.460795+0000","last_became_peered":"2024-09-19T07:53:44.460795+0000","last_unstale":"2024-09-19T07:53:44.462713+0000","last_undegraded":"2024-09-19T07:53:44.462713+0000","last_fullsized":"2024-09-19T07:53:44.462713+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:59.717556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455598+0000","last_change":"2024-09-19T07:53:43.467250+0000","last_active":"2024-09-19T07:53:44.455598+0000","last_peered":"2024-09-19T07:53:44.455598+0000","last_clean":"2024-09-19T07:53:44.455598+0000","last_became_active":"2024-09-19T07:53:43.467030+0000","last_became_peered":"2024-09-19T07:53:43.467030+0000","last_unstale":"2024-09-19T07:53:44.455598+0000","last_undegraded":"2024-09-19T07:53:44.455598+0000","last_fullsized":"2024-09-19T07:53:44.455598+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:58.715546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.472936+0000","last_change":"2024-09-19T07:53:44.469053+0000","last_active":"2024-09-19T07:53:44.472936+0000","last_peered":"2024-09-19T07:53:44.472936+0000","last_clean":"2024-09-19T07:53:44.472936+0000","last_became_active":"2024-09-19T07:53:44.468779+0000","last_became_peered":"2024-09-19T07:53:44.468779+0000","last_unstale":"2024-09-19T07:53:44.472936+0000","last_undegraded":"2024-09-19T07:53:44.472936+0000","last_fullsized":"2024-09-19T07:53:44.472936+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:35.653193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475147+0000","last_change":"2024-09-19T07:53:43.464536+0000","last_active":"2024-09-19T07:53:44.475147+0000","last_peered":"2024-09-19T07:53:44.475147+0000","last_clean":"2024-09-19T07:53:44.475147+0000","last_became_active":"2024-09-19T07:53:43.462203+0000","last_became_peered":"2024-09-19T07:53:43.462203+0000","last_unstale":"2024-09-19T07:53:44.475147+0000","last_undegraded":"2024-09-19T07:53:44.475147+0000","last_fullsized":"2024-09-19T07:53:44.475147+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:35.927023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453890+0000","last_change":"2024-09-19T07:53:43.453890+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453890+0000","last_undegraded":"2024-09-19T07:53:43.453890+0000","last_fullsized":"2024-09-19T07:53:43.453890+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455906+0000","last_change":"2024-09-19T07:53:43.468893+0000","last_active":"2024-09-19T07:53:44.455906+0000","last_peered":"2024-09-19T07:53:44.455906+0000","last_clean":"2024-09-19T07:53:44.455906+0000","last_became_active":"2024-09-19T07:53:43.468581+0000","last_became_peered":"2024-09-19T07:53:43.468581+0000","last_unstale":"2024-09-19T07:53:44.455906+0000","last_undegraded":"2024-09-19T07:53:44.455906+0000","last_fullsized":"2024-09-19T07:53:44.455906+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:21.308817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.471585+0000","last_change":"2024-09-19T07:53:44.463751+0000","last_active":"2024-09-19T07:53:44.471585+0000","last_peered":"2024-09-19T07:53:44.471585+0000","last_clean":"2024-09-19T07:53:44.471585+0000","last_became_active":"2024-09-19T07:53:44.463490+0000","last_became_peered":"2024-09-19T07:53:44.463490+0000","last_unstale":"2024-09-19T07:53:44.471585+0000","last_undegraded":"2024-09-19T07:53:44.471585+0000","last_fullsized":"2024-09-19T07:53:44.471585+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.258505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.458476+0000","last_change":"2024-09-19T07:53:43.468830+0000","last_active":"2024-09-19T07:53:44.458476+0000","last_peered":"2024-09-19T07:53:44.458476+0000","last_clean":"2024-09-19T07:53:44.458476+0000","last_became_active":"2024-09-19T07:53:43.468521+0000","last_became_peered":"2024-09-19T07:53:43.468521+0000","last_unstale":"2024-09-19T07:53:44.458476+0000","last_undegraded":"2024-09-19T07:53:44.458476+0000","last_fullsized":"2024-09-19T07:53:44.458476+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:37.240608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956008+0000","last_change":"2024-09-19T07:53:44.472102+0000","last_active":"2024-09-19T07:53:45.956008+0000","last_peered":"2024-09-19T07:53:45.956008+0000","last_clean":"2024-09-19T07:53:45.956008+0000","last_became_active":"2024-09-19T07:53:44.471800+0000","last_became_peered":"2024-09-19T07:53:44.471800+0000","last_unstale":"2024-09-19T07:53:45.956008+0000","last_undegraded":"2024-09-19T07:53:45.956008+0000","last_fullsized":"2024-09-19T07:53:45.956008+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:06.662848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.459993+0000","last_change":"2024-09-19T07:53:43.464402+0000","last_active":"2024-09-19T07:53:44.459993+0000","last_peered":"2024-09-19T07:53:44.459993+0000","last_clean":"2024-09-19T07:53:44.459993+0000","last_became_active":"2024-09-19T07:53:43.464122+0000","last_became_peered":"2024-09-19T07:53:43.464122+0000","last_unstale":"2024-09-19T07:53:44.459993+0000","last_undegraded":"2024-09-19T07:53:44.459993+0000","last_fullsized":"2024-09-19T07:53:44.459993+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:34.228041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.472855+0000","last_change":"2024-09-19T07:53:44.467967+0000","last_active":"2024-09-19T07:53:44.472855+0000","last_peered":"2024-09-19T07:53:44.472855+0000","last_clean":"2024-09-19T07:53:44.472855+0000","last_became_active":"2024-09-19T07:53:44.467699+0000","last_became_peered":"2024-09-19T07:53:44.467699+0000","last_unstale":"2024-09-19T07:53:44.472855+0000","last_undegraded":"2024-09-19T07:53:44.472855+0000","last_fullsized":"2024-09-19T07:53:44.472855+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:53.377199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.451335+0000","last_change":"2024-09-19T07:53:43.454771+0000","last_active":"2024-09-19T07:53:44.451335+0000","last_peered":"2024-09-19T07:53:44.451335+0000","last_clean":"2024-09-19T07:53:44.451335+0000","last_became_active":"2024-09-19T07:53:43.454309+0000","last_became_peered":"2024-09-19T07:53:43.454309+0000","last_unstale":"2024-09-19T07:53:44.451335+0000","last_undegraded":"2024-09-19T07:53:44.451335+0000","last_fullsized":"2024-09-19T07:53:44.451335+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:43.968446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.451146+0000","last_change":"2024-09-19T07:53:43.451146+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.451146+0000","last_undegraded":"2024-09-19T07:53:43.451146+0000","last_fullsized":"2024-09-19T07:53:43.451146+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.451114+0000","last_change":"2024-09-19T07:53:43.459958+0000","last_active":"2024-09-19T07:53:44.451114+0000","last_peered":"2024-09-19T07:53:44.451114+0000","last_clean":"2024-09-19T07:53:44.451114+0000","last_became_active":"2024-09-19T07:53:43.459470+0000","last_became_peered":"2024-09-19T07:53:43.459470+0000","last_unstale":"2024-09-19T07:53:44.451114+0000","last_undegraded":"2024-09-19T07:53:44.451114+0000","last_fullsized":"2024-09-19T07:53:44.451114+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:47.768851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.473383+0000","last_change":"2024-09-19T07:53:44.470487+0000","last_active":"2024-09-19T07:53:44.473383+0000","last_peered":"2024-09-19T07:53:44.473383+0000","last_clean":"2024-09-19T07:53:44.473383+0000","last_became_active":"2024-09-19T07:53:44.470216+0000","last_became_peered":"2024-09-19T07:53:44.470216+0000","last_unstale":"2024-09-19T07:53:44.473383+0000","last_undegraded":"2024-09-19T07:53:44.473383+0000","last_fullsized":"2024-09-19T07:53:44.473383+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:02.487846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.460679+0000","last_change":"2024-09-19T07:53:43.467051+0000","last_active":"2024-09-19T07:53:44.460679+0000","last_peered":"2024-09-19T07:53:44.460679+0000","last_clean":"2024-09-19T07:53:44.460679+0000","last_became_active":"2024-09-19T07:53:43.466667+0000","last_became_peered":"2024-09-19T07:53:43.466667+0000","last_unstale":"2024-09-19T07:53:44.460679+0000","last_undegraded":"2024-09-19T07:53:44.460679+0000","last_fullsized":"2024-09-19T07:53:44.460679+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:49.561304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.450576+0000","last_change":"2024-09-19T07:53:43.450576+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.450576+0000","last_undegraded":"2024-09-19T07:53:43.450576+0000","last_fullsized":"2024-09-19T07:53:43.450576+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455682+0000","last_change":"2024-09-19T07:53:43.465164+0000","last_active":"2024-09-19T07:53:44.455682+0000","last_peered":"2024-09-19T07:53:44.455682+0000","last_clean":"2024-09-19T07:53:44.455682+0000","last_became_active":"2024-09-19T07:53:43.464888+0000","last_became_peered":"2024-09-19T07:53:43.464888+0000","last_unstale":"2024-09-19T07:53:44.455682+0000","last_undegraded":"2024-09-19T07:53:44.455682+0000","last_fullsized":"2024-09-19T07:53:44.455682+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:01.069484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.471656+0000","last_change":"2024-09-19T07:53:44.464883+0000","last_active":"2024-09-19T07:53:44.471656+0000","last_peered":"2024-09-19T07:53:44.471656+0000","last_clean":"2024-09-19T07:53:44.471656+0000","last_became_active":"2024-09-19T07:53:44.464617+0000","last_became_peered":"2024-09-19T07:53:44.464617+0000","last_unstale":"2024-09-19T07:53:44.471656+0000","last_undegraded":"2024-09-19T07:53:44.471656+0000","last_fullsized":"2024-09-19T07:53:44.471656+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:28.530475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.465335+0000","last_change":"2024-09-19T07:53:43.459564+0000","last_active":"2024-09-19T07:53:43.465335+0000","last_peered":"2024-09-19T07:53:43.465335+0000","last_clean":"2024-09-19T07:53:43.465335+0000","last_became_active":"2024-09-19T07:53:43.459126+0000","last_became_peered":"2024-09-19T07:53:43.459126+0000","last_unstale":"2024-09-19T07:53:43.465335+0000","last_undegraded":"2024-09-19T07:53:43.465335+0000","last_fullsized":"2024-09-19T07:53:43.465335+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:42.495522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":344,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455739+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:53:44.455739+0000","last_peered":"2024-09-19T07:53:44.455739+0000","last_clean":"2024-09-19T07:53:44.455739+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:53:44.455739+0000","last_undegraded":"2024-09-19T07:53:44.455739+0000","last_fullsized":"2024-09-19T07:53:44.455739+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":742,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509573,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83096,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288744,"statfs":{"total":96636764160,"available":96551673856,"internally_reserved":0,"allocated":2031616,"data_stored":1254678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509572,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104788,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94267052,"statfs":{"total":96636764160,"available":96529461248,"internally_reserved":0,"allocated":1437696,"data_stored":664310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509573,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72784,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299056,"statfs":{"total":96636764160,"available":96562233344,"internally_reserved":0,"allocated":1437696,"data_stored":664310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509573,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86676,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285164,"statfs":{"total":96636764160,"available":96548007936,"internally_reserved":0,"allocated":1437696,"data_stored":664310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509572,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105308,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266532,"statfs":{"total":96636764160,"available":96528928768,"internally_reserved":0,"allocated":2031616,"data_stored":1254678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542276,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118604,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253236,"statfs":{"total":96636764160,"available":96515313664,"internally_reserved":0,"allocated":1425408,"data_stored":657643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542276,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82564,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94289276,"statfs":{"total":96636764160,"available":96552218624,"internally_reserved":0,"allocated":1425408,"data_stored":657643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574982,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":108904,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262936,"statfs":{"total":96636764160,"available":96525246464,"internally_reserved":0,"allocated":1462272,"data_stored":678169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:53:47.456 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:53:47.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.593+0000 7fa0be533640 1 Processor -- start 2024-09-19T07:53:47.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.593+0000 7fa0be533640 1 -- start start 2024-09-19T07:53:47.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.593+0000 7fa0be533640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0b80d0070 0x7fa0b815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.593+0000 7fa0be533640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b80d1b10 con 0x7fa0b815f0d0 2024-09-19T07:53:47.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.594+0000 7fa0be533640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b80cf2b0 con 0x7fa0b80d4130 2024-09-19T07:53:47.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.594+0000 7fa0be533640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b80cf590 con 0x7fa0b80d0070 2024-09-19T07:53:47.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.594+0000 7fa0b7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa0b815f0d0 0x7fa0b80d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35640/0 (socket says 172.21.15.16:35640) 2024-09-19T07:53:47.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.594+0000 7fa0b7fff640 1 -- 172.21.15.16:0/1813543782 learned_addr learned my addr 172.21.15.16:0/1813543782 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:47.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.594+0000 7fa0b77fe640 1 --2- 172.21.15.16:0/1813543782 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0b80d0070 0x7fa0b815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.594+0000 7fa0b6ffd640 1 -- 172.21.15.16:0/1813543782 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1517275908 0 0) 0x7fa0b80d1b10 con 0x7fa0b815f0d0 2024-09-19T07:53:47.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.594+0000 7fa0b6ffd640 1 -- 172.21.15.16:0/1813543782 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0a0003520 con 0x7fa0b815f0d0 2024-09-19T07:53:47.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.594+0000 7fa0b6ffd640 1 -- 172.21.15.16:0/1813543782 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2598240966 0 0) 0x7fa0b80cf2b0 con 0x7fa0b80d4130 2024-09-19T07:53:47.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.594+0000 7fa0b6ffd640 1 -- 172.21.15.16:0/1813543782 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0b80d1b10 con 0x7fa0b80d4130 2024-09-19T07:53:47.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.594+0000 7fa0b6ffd640 1 -- 172.21.15.16:0/1813543782 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2579972129 0 0) 0x7fa0a0003520 con 0x7fa0b815f0d0 2024-09-19T07:53:47.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.594+0000 7fa0b77fe640 1 -- 172.21.15.16:0/1813543782 >> v1:172.21.15.146:6789/0 conn(0x7fa0b80d4130 legacy=0x7fa0b80d45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.594+0000 7fa0b77fe640 1 -- 172.21.15.16:0/1813543782 >> v1:172.21.15.16:6789/0 conn(0x7fa0b815f0d0 legacy=0x7fa0b80d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.594+0000 7fa0b77fe640 1 -- 172.21.15.16:0/1813543782 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b80cf870 con 0x7fa0b80d0070 2024-09-19T07:53:47.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.595+0000 7fa0b77fe640 1 --2- 172.21.15.16:0/1813543782 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0b80d0070 0x7fa0b815f4b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac0097d0 tx=0x7fa0ac02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f51b99f279c56527 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:47.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.595+0000 7fa0b6ffd640 1 -- 172.21.15.16:0/1813543782 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac037070 con 0x7fa0b80d0070 2024-09-19T07:53:47.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.595+0000 7fa0b6ffd640 1 -- 172.21.15.16:0/1813543782 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0ac02ab90 con 0x7fa0b80d0070 2024-09-19T07:53:47.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.595+0000 7fa0b6ffd640 1 -- 172.21.15.16:0/1813543782 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8001670 con 0x7fa0b80d0070 2024-09-19T07:53:47.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.596+0000 7fa0be533640 1 -- 172.21.15.16:0/1813543782 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0b80d0070 msgr2=0x7fa0b815f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.596+0000 7fa0be533640 1 --2- 172.21.15.16:0/1813543782 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0b80d0070 0x7fa0b815f4b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac0097d0 tx=0x7fa0ac02a100 comp rx=0 tx=0).stop 2024-09-19T07:53:47.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.596+0000 7fa0be533640 1 -- 172.21.15.16:0/1813543782 shutdown_connections 2024-09-19T07:53:47.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.596+0000 7fa0be533640 1 --2- 172.21.15.16:0/1813543782 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0b80d0070 0x7fa0b815f4b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.596+0000 7fa0be533640 1 -- 172.21.15.16:0/1813543782 >> 172.21.15.16:0/1813543782 conn(0x7fa0b809d760 msgr2=0x7fa0b808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:47.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.596+0000 7fa0be533640 1 -- 172.21.15.16:0/1813543782 shutdown_connections 2024-09-19T07:53:47.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.596+0000 7fa0be533640 1 -- 172.21.15.16:0/1813543782 wait complete. 2024-09-19T07:53:47.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.597+0000 7fa0be533640 1 Processor -- start 2024-09-19T07:53:47.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.597+0000 7fa0be533640 1 -- start start 2024-09-19T07:53:47.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.598+0000 7fa0be533640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0b80d0070 0x7fa0b80b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.598+0000 7fa0be533640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0b80d4130 0x7fa0b80b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.598+0000 7fa0be533640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b815f0d0 0x7fa0b8150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.598+0000 7fa0be533640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b80b2f70 con 0x7fa0b815f0d0 2024-09-19T07:53:47.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.598+0000 7fa0be533640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b80cef60 con 0x7fa0b80d0070 2024-09-19T07:53:47.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.598+0000 7fa0be533640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b80b30f0 con 0x7fa0b80d4130 2024-09-19T07:53:47.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.598+0000 7fa0bcad0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b815f0d0 0x7fa0b8150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.598+0000 7fa0bcad0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b815f0d0 0x7fa0b8150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52744/0 (socket says 172.21.15.16:52744) 2024-09-19T07:53:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.598+0000 7fa0bcad0640 1 -- 172.21.15.16:0/3009162661 learned_addr learned my addr 172.21.15.16:0/3009162661 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.598+0000 7fa0b7fff640 1 --2- 172.21.15.16:0/3009162661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0b80d0070 0x7fa0b80b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.598+0000 7fa0b77fe640 1 --2- 172.21.15.16:0/3009162661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0b80d4130 0x7fa0b80b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.599+0000 7fa0bcad0640 1 -- 172.21.15.16:0/3009162661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0b80d4130 msgr2=0x7fa0b80b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.599+0000 7fa0bcad0640 1 --2- 172.21.15.16:0/3009162661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0b80d4130 0x7fa0b80b5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.599+0000 7fa0bcad0640 1 -- 172.21.15.16:0/3009162661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0b80d0070 msgr2=0x7fa0b80b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.599+0000 7fa0bcad0640 1 --2- 172.21.15.16:0/3009162661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0b80d0070 0x7fa0b80b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.599+0000 7fa0bcad0640 1 -- 172.21.15.16:0/3009162661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b8150ea0 con 0x7fa0b815f0d0 2024-09-19T07:53:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.599+0000 7fa0bcad0640 1 --2- 172.21.15.16:0/3009162661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b815f0d0 0x7fa0b8150960 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000c7a0 tx=0x7fa0b000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.599+0000 7fa0b4ff9640 1 -- 172.21.15.16:0/3009162661 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0003b70 con 0x7fa0b815f0d0 2024-09-19T07:53:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.599+0000 7fa0b4ff9640 1 -- 172.21.15.16:0/3009162661 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0b0003d10 con 0x7fa0b815f0d0 2024-09-19T07:53:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.599+0000 7fa0b4ff9640 1 -- 172.21.15.16:0/3009162661 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b000aa90 con 0x7fa0b815f0d0 2024-09-19T07:53:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.599+0000 7fa0be533640 1 -- 172.21.15.16:0/3009162661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b80b78a0 con 0x7fa0b815f0d0 2024-09-19T07:53:47.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.599+0000 7fa0be533640 1 -- 172.21.15.16:0/3009162661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b80b7e60 con 0x7fa0b815f0d0 2024-09-19T07:53:47.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.600+0000 7fa0be533640 1 -- 172.21.15.16:0/3009162661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b80b5cb0 con 0x7fa0b815f0d0 2024-09-19T07:53:47.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.601+0000 7fa0b4ff9640 1 -- 172.21.15.16:0/3009162661 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa0b000ac30 con 0x7fa0b815f0d0 2024-09-19T07:53:47.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.601+0000 7fa0b4ff9640 1 --2- 172.21.15.16:0/3009162661 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa0a406c2b0 0x7fa0a406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.601+0000 7fa0b4ff9640 1 -- 172.21.15.16:0/3009162661 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fa0b008b400 con 0x7fa0b815f0d0 2024-09-19T07:53:47.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.602+0000 7fa0b7fff640 1 --2- 172.21.15.16:0/3009162661 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa0a406c2b0 0x7fa0a406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.602+0000 7fa0b7fff640 1 --2- 172.21.15.16:0/3009162661 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa0a406c2b0 0x7fa0a406e770 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80017e0 tx=0x7fa0a8001610 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:47.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.606+0000 7fa0b4ff9640 1 -- 172.21.15.16:0/3009162661 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa0b8150ea0 con 0x7fa0b815f0d0 2024-09-19T07:53:47.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.778+0000 7fa0be533640 1 -- 172.21.15.16:0/3009162661 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa0b806cf30 con 0x7fa0a406c2b0 2024-09-19T07:53:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.784+0000 7fa0b4ff9640 1 -- 172.21.15.16:0/3009162661 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250763 (secure 0 0 0) 0x7fa0b806cf30 con 0x7fa0a406c2b0 2024-09-19T07:53:47.787 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:47.789 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:53:47.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.787+0000 7fa0be533640 1 -- 172.21.15.16:0/3009162661 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa0a406c2b0 msgr2=0x7fa0a406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.787+0000 7fa0be533640 1 --2- 172.21.15.16:0/3009162661 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa0a406c2b0 0x7fa0a406e770 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80017e0 tx=0x7fa0a8001610 comp rx=0 tx=0).stop 2024-09-19T07:53:47.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.787+0000 7fa0be533640 1 -- 172.21.15.16:0/3009162661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b815f0d0 msgr2=0x7fa0b8150960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.787+0000 7fa0be533640 1 --2- 172.21.15.16:0/3009162661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b815f0d0 0x7fa0b8150960 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000c7a0 tx=0x7fa0b000cc60 comp rx=0 tx=0).stop 2024-09-19T07:53:47.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.788+0000 7fa0be533640 1 -- 172.21.15.16:0/3009162661 shutdown_connections 2024-09-19T07:53:47.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.788+0000 7fa0be533640 1 --2- 172.21.15.16:0/3009162661 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa0a406c2b0 0x7fa0a406e770 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.788+0000 7fa0be533640 1 --2- 172.21.15.16:0/3009162661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b815f0d0 0x7fa0b8150960 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.788+0000 7fa0be533640 1 --2- 172.21.15.16:0/3009162661 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa0b80d4130 0x7fa0b80b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.788+0000 7fa0be533640 1 --2- 172.21.15.16:0/3009162661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0b80d0070 0x7fa0b80b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.788+0000 7fa0be533640 1 -- 172.21.15.16:0/3009162661 >> 172.21.15.16:0/3009162661 conn(0x7fa0b809d760 msgr2=0x7fa0b808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.788+0000 7fa0be533640 1 -- 172.21.15.16:0/3009162661 shutdown_connections 2024-09-19T07:53:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.788+0000 7fa0be533640 1 -- 172.21.15.16:0/3009162661 wait complete. 2024-09-19T07:53:47.799 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-09-19T07:53:46.610133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591110,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":178,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":762724,"kb_used_data":12392,"kb_used_omap":12,"kb_used_meta":750131,"kb_avail":754211996,"statfs":{"total":773094113280,"available":772313083904,"internally_reserved":0,"allocated":12689408,"data_stored":6495741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":768134704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":742,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.167829"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455920+0000","last_change":"2024-09-19T07:53:44.454037+0000","last_active":"2024-09-19T07:53:44.455920+0000","last_peered":"2024-09-19T07:53:44.455920+0000","last_clean":"2024-09-19T07:53:44.455920+0000","last_became_active":"2024-09-19T07:53:44.453756+0000","last_became_peered":"2024-09-19T07:53:44.453756+0000","last_unstale":"2024-09-19T07:53:44.455920+0000","last_undegraded":"2024-09-19T07:53:44.455920+0000","last_fullsized":"2024-09-19T07:53:44.455920+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:44.305938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.455258+0000","last_change":"2024-09-19T07:53:43.455258+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.455258+0000","last_undegraded":"2024-09-19T07:53:43.455258+0000","last_fullsized":"2024-09-19T07:53:43.455258+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.454692+0000","last_change":"2024-09-19T07:53:43.454692+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.454692+0000","last_undegraded":"2024-09-19T07:53:43.454692+0000","last_fullsized":"2024-09-19T07:53:43.454692+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.471544+0000","last_change":"2024-09-19T07:53:44.463853+0000","last_active":"2024-09-19T07:53:44.471544+0000","last_peered":"2024-09-19T07:53:44.471544+0000","last_clean":"2024-09-19T07:53:44.471544+0000","last_became_active":"2024-09-19T07:53:44.463569+0000","last_became_peered":"2024-09-19T07:53:44.463569+0000","last_unstale":"2024-09-19T07:53:44.471544+0000","last_undegraded":"2024-09-19T07:53:44.471544+0000","last_fullsized":"2024-09-19T07:53:44.471544+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:43.497533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.462115+0000","last_change":"2024-09-19T07:53:44.460542+0000","last_active":"2024-09-19T07:53:44.462115+0000","last_peered":"2024-09-19T07:53:44.462115+0000","last_clean":"2024-09-19T07:53:44.462115+0000","last_became_active":"2024-09-19T07:53:44.460245+0000","last_became_peered":"2024-09-19T07:53:44.460245+0000","last_unstale":"2024-09-19T07:53:44.462115+0000","last_undegraded":"2024-09-19T07:53:44.462115+0000","last_fullsized":"2024-09-19T07:53:44.462115+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:39:04.627509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.482181+0000","last_change":"2024-09-19T07:53:44.480188+0000","last_active":"2024-09-19T07:53:44.482181+0000","last_peered":"2024-09-19T07:53:44.482181+0000","last_clean":"2024-09-19T07:53:44.482181+0000","last_became_active":"2024-09-19T07:53:44.479867+0000","last_became_peered":"2024-09-19T07:53:44.479867+0000","last_unstale":"2024-09-19T07:53:44.482181+0000","last_undegraded":"2024-09-19T07:53:44.482181+0000","last_fullsized":"2024-09-19T07:53:44.482181+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:21.929840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.477127+0000","last_change":"2024-09-19T07:53:44.475378+0000","last_active":"2024-09-19T07:53:44.477127+0000","last_peered":"2024-09-19T07:53:44.477127+0000","last_clean":"2024-09-19T07:53:44.477127+0000","last_became_active":"2024-09-19T07:53:44.475024+0000","last_became_peered":"2024-09-19T07:53:44.475024+0000","last_unstale":"2024-09-19T07:53:44.477127+0000","last_undegraded":"2024-09-19T07:53:44.477127+0000","last_fullsized":"2024-09-19T07:53:44.477127+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:13.004954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.454146+0000","last_change":"2024-09-19T07:53:43.454146+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.454146+0000","last_undegraded":"2024-09-19T07:53:43.454146+0000","last_fullsized":"2024-09-19T07:53:43.454146+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.474046+0000","last_change":"2024-09-19T07:53:44.472235+0000","last_active":"2024-09-19T07:53:44.474046+0000","last_peered":"2024-09-19T07:53:44.474046+0000","last_clean":"2024-09-19T07:53:44.474046+0000","last_became_active":"2024-09-19T07:53:44.471961+0000","last_became_peered":"2024-09-19T07:53:44.471961+0000","last_unstale":"2024-09-19T07:53:44.474046+0000","last_undegraded":"2024-09-19T07:53:44.474046+0000","last_fullsized":"2024-09-19T07:53:44.474046+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:03.068372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.450946+0000","last_change":"2024-09-19T07:53:43.450946+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.450946+0000","last_undegraded":"2024-09-19T07:53:43.450946+0000","last_fullsized":"2024-09-19T07:53:43.450946+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.462065+0000","last_change":"2024-09-19T07:53:44.460409+0000","last_active":"2024-09-19T07:53:44.462065+0000","last_peered":"2024-09-19T07:53:44.462065+0000","last_clean":"2024-09-19T07:53:44.462065+0000","last_became_active":"2024-09-19T07:53:44.460137+0000","last_became_peered":"2024-09-19T07:53:44.460137+0000","last_unstale":"2024-09-19T07:53:44.462065+0000","last_undegraded":"2024-09-19T07:53:44.462065+0000","last_fullsized":"2024-09-19T07:53:44.462065+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:25.476480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.472912+0000","last_change":"2024-09-19T07:53:44.467120+0000","last_active":"2024-09-19T07:53:44.472912+0000","last_peered":"2024-09-19T07:53:44.472912+0000","last_clean":"2024-09-19T07:53:44.472912+0000","last_became_active":"2024-09-19T07:53:44.466828+0000","last_became_peered":"2024-09-19T07:53:44.466828+0000","last_unstale":"2024-09-19T07:53:44.472912+0000","last_undegraded":"2024-09-19T07:53:44.472912+0000","last_fullsized":"2024-09-19T07:53:44.472912+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:00.047940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.454391+0000","last_change":"2024-09-19T07:53:43.454391+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.454391+0000","last_undegraded":"2024-09-19T07:53:43.454391+0000","last_fullsized":"2024-09-19T07:53:43.454391+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453644+0000","last_change":"2024-09-19T07:53:43.453644+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453644+0000","last_undegraded":"2024-09-19T07:53:43.453644+0000","last_fullsized":"2024-09-19T07:53:43.453644+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.456674+0000","last_change":"2024-09-19T07:53:44.455460+0000","last_active":"2024-09-19T07:53:44.456674+0000","last_peered":"2024-09-19T07:53:44.456674+0000","last_clean":"2024-09-19T07:53:44.456674+0000","last_became_active":"2024-09-19T07:53:44.455164+0000","last_became_peered":"2024-09-19T07:53:44.455164+0000","last_unstale":"2024-09-19T07:53:44.456674+0000","last_undegraded":"2024-09-19T07:53:44.456674+0000","last_fullsized":"2024-09-19T07:53:44.456674+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:40.200573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.476417+0000","last_change":"2024-09-19T07:53:44.473808+0000","last_active":"2024-09-19T07:53:44.476417+0000","last_peered":"2024-09-19T07:53:44.476417+0000","last_clean":"2024-09-19T07:53:44.476417+0000","last_became_active":"2024-09-19T07:53:44.473528+0000","last_became_peered":"2024-09-19T07:53:44.473528+0000","last_unstale":"2024-09-19T07:53:44.476417+0000","last_undegraded":"2024-09-19T07:53:44.476417+0000","last_fullsized":"2024-09-19T07:53:44.476417+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:19.095190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453835+0000","last_change":"2024-09-19T07:53:43.453835+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453835+0000","last_undegraded":"2024-09-19T07:53:43.453835+0000","last_fullsized":"2024-09-19T07:53:43.453835+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453214+0000","last_change":"2024-09-19T07:53:43.453214+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453214+0000","last_undegraded":"2024-09-19T07:53:43.453214+0000","last_fullsized":"2024-09-19T07:53:43.453214+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.471284+0000","last_change":"2024-09-19T07:53:44.465304+0000","last_active":"2024-09-19T07:53:44.471284+0000","last_peered":"2024-09-19T07:53:44.471284+0000","last_clean":"2024-09-19T07:53:44.471284+0000","last_became_active":"2024-09-19T07:53:44.465044+0000","last_became_peered":"2024-09-19T07:53:44.465044+0000","last_unstale":"2024-09-19T07:53:44.471284+0000","last_undegraded":"2024-09-19T07:53:44.471284+0000","last_fullsized":"2024-09-19T07:53:44.471284+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:40.200573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955372+0000","last_change":"2024-09-19T07:53:44.466484+0000","last_active":"2024-09-19T07:53:45.955372+0000","last_peered":"2024-09-19T07:53:45.955372+0000","last_clean":"2024-09-19T07:53:45.955372+0000","last_became_active":"2024-09-19T07:53:44.466195+0000","last_became_peered":"2024-09-19T07:53:44.466195+0000","last_unstale":"2024-09-19T07:53:45.955372+0000","last_undegraded":"2024-09-19T07:53:45.955372+0000","last_fullsized":"2024-09-19T07:53:45.955372+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:20.436455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.473757+0000","last_change":"2024-09-19T07:53:44.472944+0000","last_active":"2024-09-19T07:53:44.473757+0000","last_peered":"2024-09-19T07:53:44.473757+0000","last_clean":"2024-09-19T07:53:44.473757+0000","last_became_active":"2024-09-19T07:53:44.472669+0000","last_became_peered":"2024-09-19T07:53:44.472669+0000","last_unstale":"2024-09-19T07:53:44.473757+0000","last_undegraded":"2024-09-19T07:53:44.473757+0000","last_fullsized":"2024-09-19T07:53:44.473757+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:05.983399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470250+0000","last_change":"2024-09-19T07:53:44.466834+0000","last_active":"2024-09-19T07:53:44.470250+0000","last_peered":"2024-09-19T07:53:44.470250+0000","last_clean":"2024-09-19T07:53:44.470250+0000","last_became_active":"2024-09-19T07:53:44.466204+0000","last_became_peered":"2024-09-19T07:53:44.466204+0000","last_unstale":"2024-09-19T07:53:44.470250+0000","last_undegraded":"2024-09-19T07:53:44.470250+0000","last_fullsized":"2024-09-19T07:53:44.470250+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:00.929562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.481470+0000","last_change":"2024-09-19T07:53:44.480566+0000","last_active":"2024-09-19T07:53:44.481470+0000","last_peered":"2024-09-19T07:53:44.481470+0000","last_clean":"2024-09-19T07:53:44.481470+0000","last_became_active":"2024-09-19T07:53:44.480280+0000","last_became_peered":"2024-09-19T07:53:44.480280+0000","last_unstale":"2024-09-19T07:53:44.481470+0000","last_undegraded":"2024-09-19T07:53:44.481470+0000","last_fullsized":"2024-09-19T07:53:44.481470+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:46.275247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.464980+0000","last_change":"2024-09-19T07:53:44.462832+0000","last_active":"2024-09-19T07:53:44.464980+0000","last_peered":"2024-09-19T07:53:44.464980+0000","last_clean":"2024-09-19T07:53:44.464980+0000","last_became_active":"2024-09-19T07:53:44.462538+0000","last_became_peered":"2024-09-19T07:53:44.462538+0000","last_unstale":"2024-09-19T07:53:44.464980+0000","last_undegraded":"2024-09-19T07:53:44.464980+0000","last_fullsized":"2024-09-19T07:53:44.464980+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:45.343582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.465738+0000","last_change":"2024-09-19T07:53:44.460249+0000","last_active":"2024-09-19T07:53:44.465738+0000","last_peered":"2024-09-19T07:53:44.465738+0000","last_clean":"2024-09-19T07:53:44.465738+0000","last_became_active":"2024-09-19T07:53:44.458793+0000","last_became_peered":"2024-09-19T07:53:44.458793+0000","last_unstale":"2024-09-19T07:53:44.465738+0000","last_undegraded":"2024-09-19T07:53:44.465738+0000","last_fullsized":"2024-09-19T07:53:44.465738+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:37.126896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955551+0000","last_change":"2024-09-19T07:53:44.467665+0000","last_active":"2024-09-19T07:53:45.955551+0000","last_peered":"2024-09-19T07:53:45.955551+0000","last_clean":"2024-09-19T07:53:45.955551+0000","last_became_active":"2024-09-19T07:53:44.467388+0000","last_became_peered":"2024-09-19T07:53:44.467388+0000","last_unstale":"2024-09-19T07:53:45.955551+0000","last_undegraded":"2024-09-19T07:53:45.955551+0000","last_fullsized":"2024-09-19T07:53:45.955551+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:09.122218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.464494+0000","last_change":"2024-09-19T07:53:44.463945+0000","last_active":"2024-09-19T07:53:44.464494+0000","last_peered":"2024-09-19T07:53:44.464494+0000","last_clean":"2024-09-19T07:53:44.464494+0000","last_became_active":"2024-09-19T07:53:44.463622+0000","last_became_peered":"2024-09-19T07:53:44.463622+0000","last_unstale":"2024-09-19T07:53:44.464494+0000","last_undegraded":"2024-09-19T07:53:44.464494+0000","last_fullsized":"2024-09-19T07:53:44.464494+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:44.970456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.480727+0000","last_change":"2024-09-19T07:53:44.479121+0000","last_active":"2024-09-19T07:53:44.480727+0000","last_peered":"2024-09-19T07:53:44.480727+0000","last_clean":"2024-09-19T07:53:44.480727+0000","last_became_active":"2024-09-19T07:53:44.478632+0000","last_became_peered":"2024-09-19T07:53:44.478632+0000","last_unstale":"2024-09-19T07:53:44.480727+0000","last_undegraded":"2024-09-19T07:53:44.480727+0000","last_fullsized":"2024-09-19T07:53:44.480727+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:58.195995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955651+0000","last_change":"2024-09-19T07:53:44.471531+0000","last_active":"2024-09-19T07:53:45.955651+0000","last_peered":"2024-09-19T07:53:45.955651+0000","last_clean":"2024-09-19T07:53:45.955651+0000","last_became_active":"2024-09-19T07:53:44.471247+0000","last_became_peered":"2024-09-19T07:53:44.471247+0000","last_unstale":"2024-09-19T07:53:45.955651+0000","last_undegraded":"2024-09-19T07:53:45.955651+0000","last_fullsized":"2024-09-19T07:53:45.955651+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:27:11.047914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.452495+0000","last_change":"2024-09-19T07:53:43.452495+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.452495+0000","last_undegraded":"2024-09-19T07:53:43.452495+0000","last_fullsized":"2024-09-19T07:53:43.452495+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.464284+0000","last_change":"2024-09-19T07:53:44.461893+0000","last_active":"2024-09-19T07:53:44.464284+0000","last_peered":"2024-09-19T07:53:44.464284+0000","last_clean":"2024-09-19T07:53:44.464284+0000","last_became_active":"2024-09-19T07:53:44.461543+0000","last_became_peered":"2024-09-19T07:53:44.461543+0000","last_unstale":"2024-09-19T07:53:44.464284+0000","last_undegraded":"2024-09-19T07:53:44.464284+0000","last_fullsized":"2024-09-19T07:53:44.464284+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:02.071494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.464955+0000","last_change":"2024-09-19T07:53:44.462124+0000","last_active":"2024-09-19T07:53:44.464955+0000","last_peered":"2024-09-19T07:53:44.464955+0000","last_clean":"2024-09-19T07:53:44.464955+0000","last_became_active":"2024-09-19T07:53:44.461823+0000","last_became_peered":"2024-09-19T07:53:44.461823+0000","last_unstale":"2024-09-19T07:53:44.464955+0000","last_undegraded":"2024-09-19T07:53:44.464955+0000","last_fullsized":"2024-09-19T07:53:44.464955+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:19.276802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453302+0000","last_change":"2024-09-19T07:53:43.453302+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453302+0000","last_undegraded":"2024-09-19T07:53:43.453302+0000","last_fullsized":"2024-09-19T07:53:43.453302+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"117'4","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475117+0000","last_change":"2024-09-19T07:53:43.462786+0000","last_active":"2024-09-19T07:53:44.475117+0000","last_peered":"2024-09-19T07:53:44.475117+0000","last_clean":"2024-09-19T07:53:44.475117+0000","last_became_active":"2024-09-19T07:53:43.462517+0000","last_became_peered":"2024-09-19T07:53:43.462517+0000","last_unstale":"2024-09-19T07:53:44.475117+0000","last_undegraded":"2024-09-19T07:53:44.475117+0000","last_fullsized":"2024-09-19T07:53:44.475117+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:55.555956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.483675+0000","last_change":"2024-09-19T07:53:44.482782+0000","last_active":"2024-09-19T07:53:44.483675+0000","last_peered":"2024-09-19T07:53:44.483675+0000","last_clean":"2024-09-19T07:53:44.483675+0000","last_became_active":"2024-09-19T07:53:44.482497+0000","last_became_peered":"2024-09-19T07:53:44.482497+0000","last_unstale":"2024-09-19T07:53:44.483675+0000","last_undegraded":"2024-09-19T07:53:44.483675+0000","last_fullsized":"2024-09-19T07:53:44.483675+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:03.706766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.460833+0000","last_change":"2024-09-19T07:53:43.457859+0000","last_active":"2024-09-19T07:53:43.460833+0000","last_peered":"2024-09-19T07:53:43.460833+0000","last_clean":"2024-09-19T07:53:43.460833+0000","last_became_active":"2024-09-19T07:53:43.457574+0000","last_became_peered":"2024-09-19T07:53:43.457574+0000","last_unstale":"2024-09-19T07:53:43.460833+0000","last_undegraded":"2024-09-19T07:53:43.460833+0000","last_fullsized":"2024-09-19T07:53:43.460833+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:21.380860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.469707+0000","last_change":"2024-09-19T07:53:44.468508+0000","last_active":"2024-09-19T07:53:44.469707+0000","last_peered":"2024-09-19T07:53:44.469707+0000","last_clean":"2024-09-19T07:53:44.469707+0000","last_became_active":"2024-09-19T07:53:44.468228+0000","last_became_peered":"2024-09-19T07:53:44.468228+0000","last_unstale":"2024-09-19T07:53:44.469707+0000","last_undegraded":"2024-09-19T07:53:44.469707+0000","last_fullsized":"2024-09-19T07:53:44.469707+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.258505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.480926+0000","last_change":"2024-09-19T07:53:43.464281+0000","last_active":"2024-09-19T07:53:44.480926+0000","last_peered":"2024-09-19T07:53:44.480926+0000","last_clean":"2024-09-19T07:53:44.480926+0000","last_became_active":"2024-09-19T07:53:43.464005+0000","last_became_peered":"2024-09-19T07:53:43.464005+0000","last_unstale":"2024-09-19T07:53:44.480926+0000","last_undegraded":"2024-09-19T07:53:44.480926+0000","last_fullsized":"2024-09-19T07:53:44.480926+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:40.739604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470859+0000","last_change":"2024-09-19T07:53:44.463658+0000","last_active":"2024-09-19T07:53:44.470859+0000","last_peered":"2024-09-19T07:53:44.470859+0000","last_clean":"2024-09-19T07:53:44.470859+0000","last_became_active":"2024-09-19T07:53:44.463373+0000","last_became_peered":"2024-09-19T07:53:44.463373+0000","last_unstale":"2024-09-19T07:53:44.470859+0000","last_undegraded":"2024-09-19T07:53:44.470859+0000","last_fullsized":"2024-09-19T07:53:44.470859+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:05.734979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.461452+0000","last_change":"2024-09-19T07:53:43.459863+0000","last_active":"2024-09-19T07:53:43.461452+0000","last_peered":"2024-09-19T07:53:43.461452+0000","last_clean":"2024-09-19T07:53:43.461452+0000","last_became_active":"2024-09-19T07:53:43.459538+0000","last_became_peered":"2024-09-19T07:53:43.459538+0000","last_unstale":"2024-09-19T07:53:43.461452+0000","last_undegraded":"2024-09-19T07:53:43.461452+0000","last_fullsized":"2024-09-19T07:53:43.461452+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:21.308817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.450580+0000","last_change":"2024-09-19T07:53:43.450580+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.450580+0000","last_undegraded":"2024-09-19T07:53:43.450580+0000","last_fullsized":"2024-09-19T07:53:43.450580+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"117'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.467472+0000","last_change":"2024-09-19T07:53:43.459625+0000","last_active":"2024-09-19T07:53:44.467472+0000","last_peered":"2024-09-19T07:53:44.467472+0000","last_clean":"2024-09-19T07:53:44.467472+0000","last_became_active":"2024-09-19T07:53:43.459283+0000","last_became_peered":"2024-09-19T07:53:43.459283+0000","last_unstale":"2024-09-19T07:53:44.467472+0000","last_undegraded":"2024-09-19T07:53:44.467472+0000","last_fullsized":"2024-09-19T07:53:44.467472+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:37.240608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.463008+0000","last_change":"2024-09-19T07:53:44.461446+0000","last_active":"2024-09-19T07:53:44.463008+0000","last_peered":"2024-09-19T07:53:44.463008+0000","last_clean":"2024-09-19T07:53:44.463008+0000","last_became_active":"2024-09-19T07:53:44.461158+0000","last_became_peered":"2024-09-19T07:53:44.461158+0000","last_unstale":"2024-09-19T07:53:44.463008+0000","last_undegraded":"2024-09-19T07:53:44.463008+0000","last_fullsized":"2024-09-19T07:53:44.463008+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:13.056229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"117'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.485413+0000","last_change":"2024-09-19T07:53:43.467953+0000","last_active":"2024-09-19T07:53:44.485413+0000","last_peered":"2024-09-19T07:53:44.485413+0000","last_clean":"2024-09-19T07:53:44.485413+0000","last_became_active":"2024-09-19T07:53:43.467684+0000","last_became_peered":"2024-09-19T07:53:43.467684+0000","last_unstale":"2024-09-19T07:53:44.485413+0000","last_undegraded":"2024-09-19T07:53:44.485413+0000","last_fullsized":"2024-09-19T07:53:44.485413+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:46.197768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.451717+0000","last_change":"2024-09-19T07:53:43.451717+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.451717+0000","last_undegraded":"2024-09-19T07:53:43.451717+0000","last_fullsized":"2024-09-19T07:53:43.451717+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.453345+0000","last_change":"2024-09-19T07:53:43.458791+0000","last_active":"2024-09-19T07:53:44.453345+0000","last_peered":"2024-09-19T07:53:44.453345+0000","last_clean":"2024-09-19T07:53:44.453345+0000","last_became_active":"2024-09-19T07:53:43.458509+0000","last_became_peered":"2024-09-19T07:53:43.458509+0000","last_unstale":"2024-09-19T07:53:44.453345+0000","last_undegraded":"2024-09-19T07:53:44.453345+0000","last_fullsized":"2024-09-19T07:53:44.453345+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:54.555152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453976+0000","last_change":"2024-09-19T07:53:43.453976+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453976+0000","last_undegraded":"2024-09-19T07:53:43.453976+0000","last_fullsized":"2024-09-19T07:53:43.453976+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.460241+0000","last_change":"2024-09-19T07:53:43.466824+0000","last_active":"2024-09-19T07:53:44.460241+0000","last_peered":"2024-09-19T07:53:44.460241+0000","last_clean":"2024-09-19T07:53:44.460241+0000","last_became_active":"2024-09-19T07:53:43.466546+0000","last_became_peered":"2024-09-19T07:53:43.466546+0000","last_unstale":"2024-09-19T07:53:44.460241+0000","last_undegraded":"2024-09-19T07:53:44.460241+0000","last_fullsized":"2024-09-19T07:53:44.460241+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:52.318252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470048+0000","last_change":"2024-09-19T07:53:44.464599+0000","last_active":"2024-09-19T07:53:44.470048+0000","last_peered":"2024-09-19T07:53:44.470048+0000","last_clean":"2024-09-19T07:53:44.470048+0000","last_became_active":"2024-09-19T07:53:44.464335+0000","last_became_peered":"2024-09-19T07:53:44.464335+0000","last_unstale":"2024-09-19T07:53:44.470048+0000","last_undegraded":"2024-09-19T07:53:44.470048+0000","last_fullsized":"2024-09-19T07:53:44.470048+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:44.305938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.456285+0000","last_change":"2024-09-19T07:53:43.468918+0000","last_active":"2024-09-19T07:53:44.456285+0000","last_peered":"2024-09-19T07:53:44.456285+0000","last_clean":"2024-09-19T07:53:44.456285+0000","last_became_active":"2024-09-19T07:53:43.468594+0000","last_became_peered":"2024-09-19T07:53:43.468594+0000","last_unstale":"2024-09-19T07:53:44.456285+0000","last_undegraded":"2024-09-19T07:53:44.456285+0000","last_fullsized":"2024-09-19T07:53:44.456285+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:47.768851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.469144+0000","last_change":"2024-09-19T07:53:44.466217+0000","last_active":"2024-09-19T07:53:44.469144+0000","last_peered":"2024-09-19T07:53:44.469144+0000","last_clean":"2024-09-19T07:53:44.469144+0000","last_became_active":"2024-09-19T07:53:44.465405+0000","last_became_peered":"2024-09-19T07:53:44.465405+0000","last_unstale":"2024-09-19T07:53:44.469144+0000","last_undegraded":"2024-09-19T07:53:44.469144+0000","last_fullsized":"2024-09-19T07:53:44.469144+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:41.741615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.454379+0000","last_change":"2024-09-19T07:53:43.463233+0000","last_active":"2024-09-19T07:53:44.454379+0000","last_peered":"2024-09-19T07:53:44.454379+0000","last_clean":"2024-09-19T07:53:44.454379+0000","last_became_active":"2024-09-19T07:53:43.462620+0000","last_became_peered":"2024-09-19T07:53:43.462620+0000","last_unstale":"2024-09-19T07:53:44.454379+0000","last_undegraded":"2024-09-19T07:53:44.454379+0000","last_fullsized":"2024-09-19T07:53:44.454379+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:57:59.927552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.450354+0000","last_change":"2024-09-19T07:53:43.450354+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.450354+0000","last_undegraded":"2024-09-19T07:53:43.450354+0000","last_fullsized":"2024-09-19T07:53:43.450354+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"117'3","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475321+0000","last_change":"2024-09-19T07:53:43.463469+0000","last_active":"2024-09-19T07:53:44.475321+0000","last_peered":"2024-09-19T07:53:44.475321+0000","last_clean":"2024-09-19T07:53:44.475321+0000","last_became_active":"2024-09-19T07:53:43.463042+0000","last_became_peered":"2024-09-19T07:53:43.463042+0000","last_unstale":"2024-09-19T07:53:44.475321+0000","last_undegraded":"2024-09-19T07:53:44.475321+0000","last_fullsized":"2024-09-19T07:53:44.475321+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:32.163267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.469249+0000","last_change":"2024-09-19T07:53:44.466514+0000","last_active":"2024-09-19T07:53:44.469249+0000","last_peered":"2024-09-19T07:53:44.469249+0000","last_clean":"2024-09-19T07:53:44.469249+0000","last_became_active":"2024-09-19T07:53:44.464961+0000","last_became_peered":"2024-09-19T07:53:44.464961+0000","last_unstale":"2024-09-19T07:53:44.469249+0000","last_undegraded":"2024-09-19T07:53:44.469249+0000","last_fullsized":"2024-09-19T07:53:44.469249+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:49.561080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.465379+0000","last_change":"2024-09-19T07:53:43.460277+0000","last_active":"2024-09-19T07:53:43.465379+0000","last_peered":"2024-09-19T07:53:43.465379+0000","last_clean":"2024-09-19T07:53:43.465379+0000","last_became_active":"2024-09-19T07:53:43.460028+0000","last_became_peered":"2024-09-19T07:53:43.460028+0000","last_unstale":"2024-09-19T07:53:43.465379+0000","last_undegraded":"2024-09-19T07:53:43.465379+0000","last_fullsized":"2024-09-19T07:53:43.465379+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:27.528465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475155+0000","last_change":"2024-09-19T07:53:44.474267+0000","last_active":"2024-09-19T07:53:44.475155+0000","last_peered":"2024-09-19T07:53:44.475155+0000","last_clean":"2024-09-19T07:53:44.475155+0000","last_became_active":"2024-09-19T07:53:44.473919+0000","last_became_peered":"2024-09-19T07:53:44.473919+0000","last_unstale":"2024-09-19T07:53:44.475155+0000","last_undegraded":"2024-09-19T07:53:44.475155+0000","last_fullsized":"2024-09-19T07:53:44.475155+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:25.476480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.465360+0000","last_change":"2024-09-19T07:53:43.463274+0000","last_active":"2024-09-19T07:53:43.465360+0000","last_peered":"2024-09-19T07:53:43.465360+0000","last_clean":"2024-09-19T07:53:43.465360+0000","last_became_active":"2024-09-19T07:53:43.463007+0000","last_became_peered":"2024-09-19T07:53:43.463007+0000","last_unstale":"2024-09-19T07:53:43.465360+0000","last_undegraded":"2024-09-19T07:53:43.465360+0000","last_fullsized":"2024-09-19T07:53:43.465360+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:01.485835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453196+0000","last_change":"2024-09-19T07:53:43.453196+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453196+0000","last_undegraded":"2024-09-19T07:53:43.453196+0000","last_fullsized":"2024-09-19T07:53:43.453196+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.457723+0000","last_change":"2024-09-19T07:53:43.455019+0000","last_active":"2024-09-19T07:53:43.457723+0000","last_peered":"2024-09-19T07:53:43.457723+0000","last_clean":"2024-09-19T07:53:43.457723+0000","last_became_active":"2024-09-19T07:53:43.454686+0000","last_became_peered":"2024-09-19T07:53:43.454686+0000","last_unstale":"2024-09-19T07:53:43.457723+0000","last_undegraded":"2024-09-19T07:53:43.457723+0000","last_fullsized":"2024-09-19T07:53:43.457723+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:07.256495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956646+0000","last_change":"2024-09-19T07:53:44.457728+0000","last_active":"2024-09-19T07:53:45.956646+0000","last_peered":"2024-09-19T07:53:45.956646+0000","last_clean":"2024-09-19T07:53:45.956646+0000","last_became_active":"2024-09-19T07:53:44.457456+0000","last_became_peered":"2024-09-19T07:53:44.457456+0000","last_unstale":"2024-09-19T07:53:45.956646+0000","last_undegraded":"2024-09-19T07:53:45.956646+0000","last_fullsized":"2024-09-19T07:53:45.956646+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:37:56.076282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.468120+0000","last_change":"2024-09-19T07:53:43.466167+0000","last_active":"2024-09-19T07:53:43.468120+0000","last_peered":"2024-09-19T07:53:43.468120+0000","last_clean":"2024-09-19T07:53:43.468120+0000","last_became_active":"2024-09-19T07:53:43.465882+0000","last_became_peered":"2024-09-19T07:53:43.465882+0000","last_unstale":"2024-09-19T07:53:43.468120+0000","last_undegraded":"2024-09-19T07:53:43.468120+0000","last_fullsized":"2024-09-19T07:53:43.468120+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:04.981388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470860+0000","last_change":"2024-09-19T07:53:44.469161+0000","last_active":"2024-09-19T07:53:44.470860+0000","last_peered":"2024-09-19T07:53:44.470860+0000","last_clean":"2024-09-19T07:53:44.470860+0000","last_became_active":"2024-09-19T07:53:44.468896+0000","last_became_peered":"2024-09-19T07:53:44.468896+0000","last_unstale":"2024-09-19T07:53:44.470860+0000","last_undegraded":"2024-09-19T07:53:44.470860+0000","last_fullsized":"2024-09-19T07:53:44.470860+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:43.497533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.469677+0000","last_change":"2024-09-19T07:53:43.468990+0000","last_active":"2024-09-19T07:53:43.469677+0000","last_peered":"2024-09-19T07:53:43.469677+0000","last_clean":"2024-09-19T07:53:43.469677+0000","last_became_active":"2024-09-19T07:53:43.468625+0000","last_became_peered":"2024-09-19T07:53:43.468625+0000","last_unstale":"2024-09-19T07:53:43.469677+0000","last_undegraded":"2024-09-19T07:53:43.469677+0000","last_fullsized":"2024-09-19T07:53:43.469677+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:55.555956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470204+0000","last_change":"2024-09-19T07:53:44.466474+0000","last_active":"2024-09-19T07:53:44.470204+0000","last_peered":"2024-09-19T07:53:44.470204+0000","last_clean":"2024-09-19T07:53:44.470204+0000","last_became_active":"2024-09-19T07:53:44.466230+0000","last_became_peered":"2024-09-19T07:53:44.466230+0000","last_unstale":"2024-09-19T07:53:44.470204+0000","last_undegraded":"2024-09-19T07:53:44.470204+0000","last_fullsized":"2024-09-19T07:53:44.470204+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:55.557163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955645+0000","last_change":"2024-09-19T07:53:43.463386+0000","last_active":"2024-09-19T07:53:45.955645+0000","last_peered":"2024-09-19T07:53:45.955645+0000","last_clean":"2024-09-19T07:53:45.955645+0000","last_became_active":"2024-09-19T07:53:43.463020+0000","last_became_peered":"2024-09-19T07:53:43.463020+0000","last_unstale":"2024-09-19T07:53:45.955645+0000","last_undegraded":"2024-09-19T07:53:45.955645+0000","last_fullsized":"2024-09-19T07:53:45.955645+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:01.309102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.482421+0000","last_change":"2024-09-19T07:53:44.480939+0000","last_active":"2024-09-19T07:53:44.482421+0000","last_peered":"2024-09-19T07:53:44.482421+0000","last_clean":"2024-09-19T07:53:44.482421+0000","last_became_active":"2024-09-19T07:53:44.480634+0000","last_became_peered":"2024-09-19T07:53:44.480634+0000","last_unstale":"2024-09-19T07:53:44.482421+0000","last_undegraded":"2024-09-19T07:53:44.482421+0000","last_fullsized":"2024-09-19T07:53:44.482421+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:58.089951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"117'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475895+0000","last_change":"2024-09-19T07:53:43.464937+0000","last_active":"2024-09-19T07:53:44.475895+0000","last_peered":"2024-09-19T07:53:44.475895+0000","last_clean":"2024-09-19T07:53:44.475895+0000","last_became_active":"2024-09-19T07:53:43.464614+0000","last_became_peered":"2024-09-19T07:53:43.464614+0000","last_unstale":"2024-09-19T07:53:44.475895+0000","last_undegraded":"2024-09-19T07:53:44.475895+0000","last_fullsized":"2024-09-19T07:53:44.475895+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:48.559070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.469270+0000","last_change":"2024-09-19T07:53:44.466748+0000","last_active":"2024-09-19T07:53:44.469270+0000","last_peered":"2024-09-19T07:53:44.469270+0000","last_clean":"2024-09-19T07:53:44.469270+0000","last_became_active":"2024-09-19T07:53:44.465355+0000","last_became_peered":"2024-09-19T07:53:44.465355+0000","last_unstale":"2024-09-19T07:53:44.469270+0000","last_undegraded":"2024-09-19T07:53:44.469270+0000","last_fullsized":"2024-09-19T07:53:44.469270+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:05.734979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.456785+0000","last_change":"2024-09-19T07:53:43.466472+0000","last_active":"2024-09-19T07:53:44.456785+0000","last_peered":"2024-09-19T07:53:44.456785+0000","last_clean":"2024-09-19T07:53:44.456785+0000","last_became_active":"2024-09-19T07:53:43.466216+0000","last_became_peered":"2024-09-19T07:53:43.466216+0000","last_unstale":"2024-09-19T07:53:44.456785+0000","last_undegraded":"2024-09-19T07:53:44.456785+0000","last_fullsized":"2024-09-19T07:53:44.456785+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:44.341571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.482841+0000","last_change":"2024-09-19T07:53:44.480405+0000","last_active":"2024-09-19T07:53:44.482841+0000","last_peered":"2024-09-19T07:53:44.482841+0000","last_clean":"2024-09-19T07:53:44.482841+0000","last_became_active":"2024-09-19T07:53:44.479696+0000","last_became_peered":"2024-09-19T07:53:44.479696+0000","last_unstale":"2024-09-19T07:53:44.482841+0000","last_undegraded":"2024-09-19T07:53:44.482841+0000","last_fullsized":"2024-09-19T07:53:44.482841+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:23.633013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.476888+0000","last_change":"2024-09-19T07:53:43.463463+0000","last_active":"2024-09-19T07:53:44.476888+0000","last_peered":"2024-09-19T07:53:44.476888+0000","last_clean":"2024-09-19T07:53:44.476888+0000","last_became_active":"2024-09-19T07:53:43.463197+0000","last_became_peered":"2024-09-19T07:53:43.463197+0000","last_unstale":"2024-09-19T07:53:44.476888+0000","last_undegraded":"2024-09-19T07:53:44.476888+0000","last_fullsized":"2024-09-19T07:53:44.476888+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:27.266942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.451887+0000","last_change":"2024-09-19T07:53:43.451887+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.451887+0000","last_undegraded":"2024-09-19T07:53:43.451887+0000","last_fullsized":"2024-09-19T07:53:43.451887+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.456607+0000","last_change":"2024-09-19T07:53:43.465167+0000","last_active":"2024-09-19T07:53:44.456607+0000","last_peered":"2024-09-19T07:53:44.456607+0000","last_clean":"2024-09-19T07:53:44.456607+0000","last_became_active":"2024-09-19T07:53:43.464875+0000","last_became_peered":"2024-09-19T07:53:43.464875+0000","last_unstale":"2024-09-19T07:53:44.456607+0000","last_undegraded":"2024-09-19T07:53:44.456607+0000","last_fullsized":"2024-09-19T07:53:44.456607+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:18.274791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.450312+0000","last_change":"2024-09-19T07:53:43.450312+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.450312+0000","last_undegraded":"2024-09-19T07:53:43.450312+0000","last_fullsized":"2024-09-19T07:53:43.450312+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.468095+0000","last_change":"2024-09-19T07:53:43.465392+0000","last_active":"2024-09-19T07:53:43.468095+0000","last_peered":"2024-09-19T07:53:43.468095+0000","last_clean":"2024-09-19T07:53:43.468095+0000","last_became_active":"2024-09-19T07:53:43.465112+0000","last_became_peered":"2024-09-19T07:53:43.465112+0000","last_unstale":"2024-09-19T07:53:43.468095+0000","last_undegraded":"2024-09-19T07:53:43.468095+0000","last_fullsized":"2024-09-19T07:53:43.468095+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:02.066362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.462713+0000","last_change":"2024-09-19T07:53:44.461101+0000","last_active":"2024-09-19T07:53:44.462713+0000","last_peered":"2024-09-19T07:53:44.462713+0000","last_clean":"2024-09-19T07:53:44.462713+0000","last_became_active":"2024-09-19T07:53:44.460795+0000","last_became_peered":"2024-09-19T07:53:44.460795+0000","last_unstale":"2024-09-19T07:53:44.462713+0000","last_undegraded":"2024-09-19T07:53:44.462713+0000","last_fullsized":"2024-09-19T07:53:44.462713+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:59.717556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455598+0000","last_change":"2024-09-19T07:53:43.467250+0000","last_active":"2024-09-19T07:53:44.455598+0000","last_peered":"2024-09-19T07:53:44.455598+0000","last_clean":"2024-09-19T07:53:44.455598+0000","last_became_active":"2024-09-19T07:53:43.467030+0000","last_became_peered":"2024-09-19T07:53:43.467030+0000","last_unstale":"2024-09-19T07:53:44.455598+0000","last_undegraded":"2024-09-19T07:53:44.455598+0000","last_fullsized":"2024-09-19T07:53:44.455598+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:58.715546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.472936+0000","last_change":"2024-09-19T07:53:44.469053+0000","last_active":"2024-09-19T07:53:44.472936+0000","last_peered":"2024-09-19T07:53:44.472936+0000","last_clean":"2024-09-19T07:53:44.472936+0000","last_became_active":"2024-09-19T07:53:44.468779+0000","last_became_peered":"2024-09-19T07:53:44.468779+0000","last_unstale":"2024-09-19T07:53:44.472936+0000","last_undegraded":"2024-09-19T07:53:44.472936+0000","last_fullsized":"2024-09-19T07:53:44.472936+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:35.653193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475147+0000","last_change":"2024-09-19T07:53:43.464536+0000","last_active":"2024-09-19T07:53:44.475147+0000","last_peered":"2024-09-19T07:53:44.475147+0000","last_clean":"2024-09-19T07:53:44.475147+0000","last_became_active":"2024-09-19T07:53:43.462203+0000","last_became_peered":"2024-09-19T07:53:43.462203+0000","last_unstale":"2024-09-19T07:53:44.475147+0000","last_undegraded":"2024-09-19T07:53:44.475147+0000","last_fullsized":"2024-09-19T07:53:44.475147+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:35.927023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453890+0000","last_change":"2024-09-19T07:53:43.453890+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453890+0000","last_undegraded":"2024-09-19T07:53:43.453890+0000","last_fullsized":"2024-09-19T07:53:43.453890+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455906+0000","last_change":"2024-09-19T07:53:43.468893+0000","last_active":"2024-09-19T07:53:44.455906+0000","last_peered":"2024-09-19T07:53:44.455906+0000","last_clean":"2024-09-19T07:53:44.455906+0000","last_became_active":"2024-09-19T07:53:43.468581+0000","last_became_peered":"2024-09-19T07:53:43.468581+0000","last_unstale":"2024-09-19T07:53:44.455906+0000","last_undegraded":"2024-09-19T07:53:44.455906+0000","last_fullsized":"2024-09-19T07:53:44.455906+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:21.308817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.471585+0000","last_change":"2024-09-19T07:53:44.463751+0000","last_active":"2024-09-19T07:53:44.471585+0000","last_peered":"2024-09-19T07:53:44.471585+0000","last_clean":"2024-09-19T07:53:44.471585+0000","last_became_active":"2024-09-19T07:53:44.463490+0000","last_became_peered":"2024-09-19T07:53:44.463490+0000","last_unstale":"2024-09-19T07:53:44.471585+0000","last_undegraded":"2024-09-19T07:53:44.471585+0000","last_fullsized":"2024-09-19T07:53:44.471585+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.258505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.458476+0000","last_change":"2024-09-19T07:53:43.468830+0000","last_active":"2024-09-19T07:53:44.458476+0000","last_peered":"2024-09-19T07:53:44.458476+0000","last_clean":"2024-09-19T07:53:44.458476+0000","last_became_active":"2024-09-19T07:53:43.468521+0000","last_became_peered":"2024-09-19T07:53:43.468521+0000","last_unstale":"2024-09-19T07:53:44.458476+0000","last_undegraded":"2024-09-19T07:53:44.458476+0000","last_fullsized":"2024-09-19T07:53:44.458476+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:37.240608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956008+0000","last_change":"2024-09-19T07:53:44.472102+0000","last_active":"2024-09-19T07:53:45.956008+0000","last_peered":"2024-09-19T07:53:45.956008+0000","last_clean":"2024-09-19T07:53:45.956008+0000","last_became_active":"2024-09-19T07:53:44.471800+0000","last_became_peered":"2024-09-19T07:53:44.471800+0000","last_unstale":"2024-09-19T07:53:45.956008+0000","last_undegraded":"2024-09-19T07:53:45.956008+0000","last_fullsized":"2024-09-19T07:53:45.956008+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:06.662848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.459993+0000","last_change":"2024-09-19T07:53:43.464402+0000","last_active":"2024-09-19T07:53:44.459993+0000","last_peered":"2024-09-19T07:53:44.459993+0000","last_clean":"2024-09-19T07:53:44.459993+0000","last_became_active":"2024-09-19T07:53:43.464122+0000","last_became_peered":"2024-09-19T07:53:43.464122+0000","last_unstale":"2024-09-19T07:53:44.459993+0000","last_undegraded":"2024-09-19T07:53:44.459993+0000","last_fullsized":"2024-09-19T07:53:44.459993+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:34.228041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.472855+0000","last_change":"2024-09-19T07:53:44.467967+0000","last_active":"2024-09-19T07:53:44.472855+0000","last_peered":"2024-09-19T07:53:44.472855+0000","last_clean":"2024-09-19T07:53:44.472855+0000","last_became_active":"2024-09-19T07:53:44.467699+0000","last_became_peered":"2024-09-19T07:53:44.467699+0000","last_unstale":"2024-09-19T07:53:44.472855+0000","last_undegraded":"2024-09-19T07:53:44.472855+0000","last_fullsized":"2024-09-19T07:53:44.472855+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:53.377199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.451335+0000","last_change":"2024-09-19T07:53:43.454771+0000","last_active":"2024-09-19T07:53:44.451335+0000","last_peered":"2024-09-19T07:53:44.451335+0000","last_clean":"2024-09-19T07:53:44.451335+0000","last_became_active":"2024-09-19T07:53:43.454309+0000","last_became_peered":"2024-09-19T07:53:43.454309+0000","last_unstale":"2024-09-19T07:53:44.451335+0000","last_undegraded":"2024-09-19T07:53:44.451335+0000","last_fullsized":"2024-09-19T07:53:44.451335+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:43.968446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.451146+0000","last_change":"2024-09-19T07:53:43.451146+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.451146+0000","last_undegraded":"2024-09-19T07:53:43.451146+0000","last_fullsized":"2024-09-19T07:53:43.451146+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.451114+0000","last_change":"2024-09-19T07:53:43.459958+0000","last_active":"2024-09-19T07:53:44.451114+0000","last_peered":"2024-09-19T07:53:44.451114+0000","last_clean":"2024-09-19T07:53:44.451114+0000","last_became_active":"2024-09-19T07:53:43.459470+0000","last_became_peered":"2024-09-19T07:53:43.459470+0000","last_unstale":"2024-09-19T07:53:44.451114+0000","last_undegraded":"2024-09-19T07:53:44.451114+0000","last_fullsized":"2024-09-19T07:53:44.451114+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:47.768851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.473383+0000","last_change":"2024-09-19T07:53:44.470487+0000","last_active":"2024-09-19T07:53:44.473383+0000","last_peered":"2024-09-19T07:53:44.473383+0000","last_clean":"2024-09-19T07:53:44.473383+0000","last_became_active":"2024-09-19T07:53:44.470216+0000","last_became_peered":"2024-09-19T07:53:44.470216+0000","last_unstale":"2024-09-19T07:53:44.473383+0000","last_undegraded":"2024-09-19T07:53:44.473383+0000","last_fullsized":"2024-09-19T07:53:44.473383+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:02.487846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.460679+0000","last_change":"2024-09-19T07:53:43.467051+0000","last_active":"2024-09-19T07:53:44.460679+0000","last_peered":"2024-09-19T07:53:44.460679+0000","last_clean":"2024-09-19T07:53:44.460679+0000","last_became_active":"2024-09-19T07:53:43.466667+0000","last_became_peered":"2024-09-19T07:53:43.466667+0000","last_unstale":"2024-09-19T07:53:44.460679+0000","last_undegraded":"2024-09-19T07:53:44.460679+0000","last_fullsized":"2024-09-19T07:53:44.460679+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:49.561304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.450576+0000","last_change":"2024-09-19T07:53:43.450576+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.450576+0000","last_undegraded":"2024-09-19T07:53:43.450576+0000","last_fullsized":"2024-09-19T07:53:43.450576+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455682+0000","last_change":"2024-09-19T07:53:43.465164+0000","last_active":"2024-09-19T07:53:44.455682+0000","last_peered":"2024-09-19T07:53:44.455682+0000","last_clean":"2024-09-19T07:53:44.455682+0000","last_became_active":"2024-09-19T07:53:43.464888+0000","last_became_peered":"2024-09-19T07:53:43.464888+0000","last_unstale":"2024-09-19T07:53:44.455682+0000","last_undegraded":"2024-09-19T07:53:44.455682+0000","last_fullsized":"2024-09-19T07:53:44.455682+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:01.069484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.471656+0000","last_change":"2024-09-19T07:53:44.464883+0000","last_active":"2024-09-19T07:53:44.471656+0000","last_peered":"2024-09-19T07:53:44.471656+0000","last_clean":"2024-09-19T07:53:44.471656+0000","last_became_active":"2024-09-19T07:53:44.464617+0000","last_became_peered":"2024-09-19T07:53:44.464617+0000","last_unstale":"2024-09-19T07:53:44.471656+0000","last_undegraded":"2024-09-19T07:53:44.471656+0000","last_fullsized":"2024-09-19T07:53:44.471656+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:28.530475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.465335+0000","last_change":"2024-09-19T07:53:43.459564+0000","last_active":"2024-09-19T07:53:43.465335+0000","last_peered":"2024-09-19T07:53:43.465335+0000","last_clean":"2024-09-19T07:53:43.465335+0000","last_became_active":"2024-09-19T07:53:43.459126+0000","last_became_peered":"2024-09-19T07:53:43.459126+0000","last_unstale":"2024-09-19T07:53:43.465335+0000","last_undegraded":"2024-09-19T07:53:43.465335+0000","last_fullsized":"2024-09-19T07:53:43.465335+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:42.495522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":344,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455739+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:53:44.455739+0000","last_peered":"2024-09-19T07:53:44.455739+0000","last_clean":"2024-09-19T07:53:44.455739+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:53:44.455739+0000","last_undegraded":"2024-09-19T07:53:44.455739+0000","last_fullsized":"2024-09-19T07:53:44.455739+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":742,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509573,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83096,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288744,"statfs":{"total":96636764160,"available":96551673856,"internally_reserved":0,"allocated":2031616,"data_stored":1254678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509572,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104788,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94267052,"statfs":{"total":96636764160,"available":96529461248,"internally_reserved":0,"allocated":1437696,"data_stored":664310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509573,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72784,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299056,"statfs":{"total":96636764160,"available":96562233344,"internally_reserved":0,"allocated":1437696,"data_stored":664310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509573,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86676,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285164,"statfs":{"total":96636764160,"available":96548007936,"internally_reserved":0,"allocated":1437696,"data_stored":664310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509572,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105308,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266532,"statfs":{"total":96636764160,"available":96528928768,"internally_reserved":0,"allocated":2031616,"data_stored":1254678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542276,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118604,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253236,"statfs":{"total":96636764160,"available":96515313664,"internally_reserved":0,"allocated":1425408,"data_stored":657643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542276,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82564,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94289276,"statfs":{"total":96636764160,"available":96552218624,"internally_reserved":0,"allocated":1425408,"data_stored":657643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574982,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":108904,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262936,"statfs":{"total":96636764160,"available":96525246464,"internally_reserved":0,"allocated":1462272,"data_stored":678169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:53:47.807 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:53:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.941+0000 7f52801fd640 1 Processor -- start 2024-09-19T07:53:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.941+0000 7f52801fd640 1 -- start start 2024-09-19T07:53:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.942+0000 7f52801fd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f527815e680 0x7f52780cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.942+0000 7f52801fd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f52780d0010 0x7f52780d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.942+0000 7f52801fd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52780d1880 con 0x7f527815e2a0 2024-09-19T07:53:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.942+0000 7f52801fd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f52780d1b60 con 0x7f52780d0010 2024-09-19T07:53:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.942+0000 7f52801fd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f52780cf2d0 con 0x7f527815e680 2024-09-19T07:53:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.942+0000 7f527e79a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f52780d0010 0x7f52780d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.942+0000 7f527df99640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f527815e2a0 0x7f527806d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35650/0 (socket says 172.21.15.16:35650) 2024-09-19T07:53:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.942+0000 7f527df99640 1 -- 172.21.15.16:0/1314934804 learned_addr learned my addr 172.21.15.16:0/1314934804 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.942+0000 7f527d798640 1 --2- 172.21.15.16:0/1314934804 >> v2:172.21.15.146:3300/0 conn(0x7f527815e680 0x7f52780cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.943+0000 7f527e79a640 1 -- 172.21.15.16:0/1314934804 >> v2:172.21.15.146:3300/0 conn(0x7f527815e680 msgr2=0x7f52780cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.943+0000 7f527e79a640 1 --2- 172.21.15.16:0/1314934804 >> v2:172.21.15.146:3300/0 conn(0x7f527815e680 0x7f52780cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.943+0000 7f527e79a640 1 -- 172.21.15.16:0/1314934804 >> v1:172.21.15.16:6789/0 conn(0x7f527815e2a0 legacy=0x7f527806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.943+0000 7f527cf97640 1 -- 172.21.15.16:0/1314934804 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2497300982 0 0) 0x7f52780d1880 con 0x7f527815e2a0 2024-09-19T07:53:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.943+0000 7f527e79a640 1 -- 172.21.15.16:0/1314934804 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52780cf5b0 con 0x7f52780d0010 2024-09-19T07:53:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.943+0000 7f527e79a640 1 --2- 172.21.15.16:0/1314934804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52780d0010 0x7f52780d4370 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f52780a22a0 tx=0x7f527002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb60fc3f6b3405a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.943+0000 7f527cf97640 1 -- 172.21.15.16:0/1314934804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270037070 con 0x7f52780d0010 2024-09-19T07:53:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.943+0000 7f527cf97640 1 -- 172.21.15.16:0/1314934804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5270032440 con 0x7f52780d0010 2024-09-19T07:53:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.943+0000 7f527cf97640 1 -- 172.21.15.16:0/1314934804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527002ae60 con 0x7f52780d0010 2024-09-19T07:53:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.944+0000 7f52801fd640 1 -- 172.21.15.16:0/1314934804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52780d0010 msgr2=0x7f52780d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.944+0000 7f52801fd640 1 --2- 172.21.15.16:0/1314934804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52780d0010 0x7f52780d4370 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f52780a22a0 tx=0x7f527002a110 comp rx=0 tx=0).stop 2024-09-19T07:53:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.944+0000 7f52801fd640 1 -- 172.21.15.16:0/1314934804 shutdown_connections 2024-09-19T07:53:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.944+0000 7f52801fd640 1 --2- 172.21.15.16:0/1314934804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52780d0010 0x7f52780d4370 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.944+0000 7f52801fd640 1 --2- 172.21.15.16:0/1314934804 >> v2:172.21.15.146:3300/0 conn(0x7f527815e680 0x7f52780cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.944+0000 7f52801fd640 1 -- 172.21.15.16:0/1314934804 >> 172.21.15.16:0/1314934804 conn(0x7f527809d760 msgr2=0x7f527808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.944+0000 7f52801fd640 1 -- 172.21.15.16:0/1314934804 shutdown_connections 2024-09-19T07:53:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.944+0000 7f52801fd640 1 -- 172.21.15.16:0/1314934804 wait complete. 2024-09-19T07:53:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.945+0000 7f52801fd640 1 Processor -- start 2024-09-19T07:53:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.945+0000 7f52801fd640 1 -- start start 2024-09-19T07:53:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.946+0000 7f52801fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52780d0010 0x7f52780b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.946+0000 7f52801fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f527815e2a0 0x7f52780cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.946+0000 7f52801fd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f527815e680 0x7f527815b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.946+0000 7f52801fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f527806dcb0 con 0x7f52780d0010 2024-09-19T07:53:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.946+0000 7f52801fd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f527806db30 con 0x7f527815e680 2024-09-19T07:53:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.946+0000 7f52801fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f52780d48b0 con 0x7f527815e2a0 2024-09-19T07:53:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.946+0000 7f527df99640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52780d0010 0x7f52780b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.946+0000 7f527df99640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52780d0010 0x7f52780b0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52762/0 (socket says 172.21.15.16:52762) 2024-09-19T07:53:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.946+0000 7f527df99640 1 -- 172.21.15.16:0/3143607382 learned_addr learned my addr 172.21.15.16:0/3143607382 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.946+0000 7f527d798640 1 --2- 172.21.15.16:0/3143607382 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f527815e2a0 0x7f52780cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.946+0000 7f527e79a640 1 --2- 172.21.15.16:0/3143607382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f527815e680 0x7f527815b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.946+0000 7f527df99640 1 -- 172.21.15.16:0/3143607382 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f527815e2a0 msgr2=0x7f52780cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.946+0000 7f527df99640 1 --2- 172.21.15.16:0/3143607382 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f527815e2a0 0x7f52780cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.946+0000 7f527df99640 1 -- 172.21.15.16:0/3143607382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f527815e680 msgr2=0x7f527815b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.946+0000 7f527df99640 1 --2- 172.21.15.16:0/3143607382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f527815e680 0x7f527815b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.946+0000 7f527df99640 1 -- 172.21.15.16:0/3143607382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52780cc510 con 0x7f52780d0010 2024-09-19T07:53:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.947+0000 7f527df99640 1 --2- 172.21.15.16:0/3143607382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52780d0010 0x7f52780b0f50 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f526800c890 tx=0x7f526800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.947+0000 7f52767fc640 1 -- 172.21.15.16:0/3143607382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268018070 con 0x7f52780d0010 2024-09-19T07:53:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.947+0000 7f52767fc640 1 -- 172.21.15.16:0/3143607382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f526800a640 con 0x7f52780d0010 2024-09-19T07:53:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.947+0000 7f52801fd640 1 -- 172.21.15.16:0/3143607382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52780cc7f0 con 0x7f52780d0010 2024-09-19T07:53:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.947+0000 7f52767fc640 1 -- 172.21.15.16:0/3143607382 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526800aa30 con 0x7f52780d0010 2024-09-19T07:53:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.947+0000 7f52801fd640 1 -- 172.21.15.16:0/3143607382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52780c9680 con 0x7f52780d0010 2024-09-19T07:53:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.948+0000 7f52801fd640 1 -- 172.21.15.16:0/3143607382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52780d4c70 con 0x7f52780d0010 2024-09-19T07:53:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.948+0000 7f52767fc640 1 -- 172.21.15.16:0/3143607382 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f526801b070 con 0x7f52780d0010 2024-09-19T07:53:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.949+0000 7f52767fc640 1 --2- 172.21.15.16:0/3143607382 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f526406c2b0 0x7f526406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.949+0000 7f52767fc640 1 -- 172.21.15.16:0/3143607382 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f526808ac70 con 0x7f52780d0010 2024-09-19T07:53:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.949+0000 7f527d798640 1 --2- 172.21.15.16:0/3143607382 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f526406c2b0 0x7f526406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.950+0000 7f527d798640 1 --2- 172.21.15.16:0/3143607382 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f526406c2b0 0x7f526406e770 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f526c006fd0 tx=0x7f526c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:47.954+0000 7f52767fc640 1 -- 172.21.15.16:0/3143607382 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f52680592e0 con 0x7f52780d0010 2024-09-19T07:53:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.126+0000 7f52801fd640 1 -- 172.21.15.16:0/3143607382 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f52780cb180 con 0x7f526406c2b0 2024-09-19T07:53:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.132+0000 7f52767fc640 1 -- 172.21.15.16:0/3143607382 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250763 (secure 0 0 0) 0x7f52780cb180 con 0x7f526406c2b0 2024-09-19T07:53:48.472 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:53:48.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.135+0000 7f52801fd640 1 -- 172.21.15.16:0/3143607382 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f526406c2b0 msgr2=0x7f526406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:48.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.135+0000 7f52801fd640 1 --2- 172.21.15.16:0/3143607382 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f526406c2b0 0x7f526406e770 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f526c006fd0 tx=0x7f526c008040 comp rx=0 tx=0).stop 2024-09-19T07:53:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.135+0000 7f52801fd640 1 -- 172.21.15.16:0/3143607382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52780d0010 msgr2=0x7f52780b0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.135+0000 7f52801fd640 1 --2- 172.21.15.16:0/3143607382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52780d0010 0x7f52780b0f50 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f526800c890 tx=0x7f526800cd50 comp rx=0 tx=0).stop 2024-09-19T07:53:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.135+0000 7f52801fd640 1 -- 172.21.15.16:0/3143607382 shutdown_connections 2024-09-19T07:53:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.135+0000 7f52801fd640 1 --2- 172.21.15.16:0/3143607382 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f526406c2b0 0x7f526406e770 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.135+0000 7f52801fd640 1 --2- 172.21.15.16:0/3143607382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f527815e680 0x7f527815b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.135+0000 7f52801fd640 1 --2- 172.21.15.16:0/3143607382 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f527815e2a0 0x7f52780cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.135+0000 7f52801fd640 1 --2- 172.21.15.16:0/3143607382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52780d0010 0x7f52780b0f50 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.135+0000 7f52801fd640 1 -- 172.21.15.16:0/3143607382 >> 172.21.15.16:0/3143607382 conn(0x7f527809d760 msgr2=0x7f52780d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.135+0000 7f52801fd640 1 -- 172.21.15.16:0/3143607382 shutdown_connections 2024-09-19T07:53:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.135+0000 7f52801fd640 1 -- 172.21.15.16:0/3143607382 wait complete. 2024-09-19T07:53:48.476 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-09-19T07:53:46.610133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591110,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":178,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":762724,"kb_used_data":12392,"kb_used_omap":12,"kb_used_meta":750131,"kb_avail":754211996,"statfs":{"total":773094113280,"available":772313083904,"internally_reserved":0,"allocated":12689408,"data_stored":6495741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":768134704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":742,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.167829"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455920+0000","last_change":"2024-09-19T07:53:44.454037+0000","last_active":"2024-09-19T07:53:44.455920+0000","last_peered":"2024-09-19T07:53:44.455920+0000","last_clean":"2024-09-19T07:53:44.455920+0000","last_became_active":"2024-09-19T07:53:44.453756+0000","last_became_peered":"2024-09-19T07:53:44.453756+0000","last_unstale":"2024-09-19T07:53:44.455920+0000","last_undegraded":"2024-09-19T07:53:44.455920+0000","last_fullsized":"2024-09-19T07:53:44.455920+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:44.305938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.455258+0000","last_change":"2024-09-19T07:53:43.455258+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.455258+0000","last_undegraded":"2024-09-19T07:53:43.455258+0000","last_fullsized":"2024-09-19T07:53:43.455258+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.454692+0000","last_change":"2024-09-19T07:53:43.454692+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.454692+0000","last_undegraded":"2024-09-19T07:53:43.454692+0000","last_fullsized":"2024-09-19T07:53:43.454692+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.471544+0000","last_change":"2024-09-19T07:53:44.463853+0000","last_active":"2024-09-19T07:53:44.471544+0000","last_peered":"2024-09-19T07:53:44.471544+0000","last_clean":"2024-09-19T07:53:44.471544+0000","last_became_active":"2024-09-19T07:53:44.463569+0000","last_became_peered":"2024-09-19T07:53:44.463569+0000","last_unstale":"2024-09-19T07:53:44.471544+0000","last_undegraded":"2024-09-19T07:53:44.471544+0000","last_fullsized":"2024-09-19T07:53:44.471544+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:43.497533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.462115+0000","last_change":"2024-09-19T07:53:44.460542+0000","last_active":"2024-09-19T07:53:44.462115+0000","last_peered":"2024-09-19T07:53:44.462115+0000","last_clean":"2024-09-19T07:53:44.462115+0000","last_became_active":"2024-09-19T07:53:44.460245+0000","last_became_peered":"2024-09-19T07:53:44.460245+0000","last_unstale":"2024-09-19T07:53:44.462115+0000","last_undegraded":"2024-09-19T07:53:44.462115+0000","last_fullsized":"2024-09-19T07:53:44.462115+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:39:04.627509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.482181+0000","last_change":"2024-09-19T07:53:44.480188+0000","last_active":"2024-09-19T07:53:44.482181+0000","last_peered":"2024-09-19T07:53:44.482181+0000","last_clean":"2024-09-19T07:53:44.482181+0000","last_became_active":"2024-09-19T07:53:44.479867+0000","last_became_peered":"2024-09-19T07:53:44.479867+0000","last_unstale":"2024-09-19T07:53:44.482181+0000","last_undegraded":"2024-09-19T07:53:44.482181+0000","last_fullsized":"2024-09-19T07:53:44.482181+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:21.929840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.477127+0000","last_change":"2024-09-19T07:53:44.475378+0000","last_active":"2024-09-19T07:53:44.477127+0000","last_peered":"2024-09-19T07:53:44.477127+0000","last_clean":"2024-09-19T07:53:44.477127+0000","last_became_active":"2024-09-19T07:53:44.475024+0000","last_became_peered":"2024-09-19T07:53:44.475024+0000","last_unstale":"2024-09-19T07:53:44.477127+0000","last_undegraded":"2024-09-19T07:53:44.477127+0000","last_fullsized":"2024-09-19T07:53:44.477127+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:13.004954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.454146+0000","last_change":"2024-09-19T07:53:43.454146+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.454146+0000","last_undegraded":"2024-09-19T07:53:43.454146+0000","last_fullsized":"2024-09-19T07:53:43.454146+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.474046+0000","last_change":"2024-09-19T07:53:44.472235+0000","last_active":"2024-09-19T07:53:44.474046+0000","last_peered":"2024-09-19T07:53:44.474046+0000","last_clean":"2024-09-19T07:53:44.474046+0000","last_became_active":"2024-09-19T07:53:44.471961+0000","last_became_peered":"2024-09-19T07:53:44.471961+0000","last_unstale":"2024-09-19T07:53:44.474046+0000","last_undegraded":"2024-09-19T07:53:44.474046+0000","last_fullsized":"2024-09-19T07:53:44.474046+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:03.068372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.450946+0000","last_change":"2024-09-19T07:53:43.450946+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.450946+0000","last_undegraded":"2024-09-19T07:53:43.450946+0000","last_fullsized":"2024-09-19T07:53:43.450946+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.462065+0000","last_change":"2024-09-19T07:53:44.460409+0000","last_active":"2024-09-19T07:53:44.462065+0000","last_peered":"2024-09-19T07:53:44.462065+0000","last_clean":"2024-09-19T07:53:44.462065+0000","last_became_active":"2024-09-19T07:53:44.460137+0000","last_became_peered":"2024-09-19T07:53:44.460137+0000","last_unstale":"2024-09-19T07:53:44.462065+0000","last_undegraded":"2024-09-19T07:53:44.462065+0000","last_fullsized":"2024-09-19T07:53:44.462065+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:25.476480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.472912+0000","last_change":"2024-09-19T07:53:44.467120+0000","last_active":"2024-09-19T07:53:44.472912+0000","last_peered":"2024-09-19T07:53:44.472912+0000","last_clean":"2024-09-19T07:53:44.472912+0000","last_became_active":"2024-09-19T07:53:44.466828+0000","last_became_peered":"2024-09-19T07:53:44.466828+0000","last_unstale":"2024-09-19T07:53:44.472912+0000","last_undegraded":"2024-09-19T07:53:44.472912+0000","last_fullsized":"2024-09-19T07:53:44.472912+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:00.047940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.454391+0000","last_change":"2024-09-19T07:53:43.454391+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.454391+0000","last_undegraded":"2024-09-19T07:53:43.454391+0000","last_fullsized":"2024-09-19T07:53:43.454391+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453644+0000","last_change":"2024-09-19T07:53:43.453644+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453644+0000","last_undegraded":"2024-09-19T07:53:43.453644+0000","last_fullsized":"2024-09-19T07:53:43.453644+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.456674+0000","last_change":"2024-09-19T07:53:44.455460+0000","last_active":"2024-09-19T07:53:44.456674+0000","last_peered":"2024-09-19T07:53:44.456674+0000","last_clean":"2024-09-19T07:53:44.456674+0000","last_became_active":"2024-09-19T07:53:44.455164+0000","last_became_peered":"2024-09-19T07:53:44.455164+0000","last_unstale":"2024-09-19T07:53:44.456674+0000","last_undegraded":"2024-09-19T07:53:44.456674+0000","last_fullsized":"2024-09-19T07:53:44.456674+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:40.200573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.476417+0000","last_change":"2024-09-19T07:53:44.473808+0000","last_active":"2024-09-19T07:53:44.476417+0000","last_peered":"2024-09-19T07:53:44.476417+0000","last_clean":"2024-09-19T07:53:44.476417+0000","last_became_active":"2024-09-19T07:53:44.473528+0000","last_became_peered":"2024-09-19T07:53:44.473528+0000","last_unstale":"2024-09-19T07:53:44.476417+0000","last_undegraded":"2024-09-19T07:53:44.476417+0000","last_fullsized":"2024-09-19T07:53:44.476417+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:19.095190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453835+0000","last_change":"2024-09-19T07:53:43.453835+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453835+0000","last_undegraded":"2024-09-19T07:53:43.453835+0000","last_fullsized":"2024-09-19T07:53:43.453835+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453214+0000","last_change":"2024-09-19T07:53:43.453214+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453214+0000","last_undegraded":"2024-09-19T07:53:43.453214+0000","last_fullsized":"2024-09-19T07:53:43.453214+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.471284+0000","last_change":"2024-09-19T07:53:44.465304+0000","last_active":"2024-09-19T07:53:44.471284+0000","last_peered":"2024-09-19T07:53:44.471284+0000","last_clean":"2024-09-19T07:53:44.471284+0000","last_became_active":"2024-09-19T07:53:44.465044+0000","last_became_peered":"2024-09-19T07:53:44.465044+0000","last_unstale":"2024-09-19T07:53:44.471284+0000","last_undegraded":"2024-09-19T07:53:44.471284+0000","last_fullsized":"2024-09-19T07:53:44.471284+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:40.200573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955372+0000","last_change":"2024-09-19T07:53:44.466484+0000","last_active":"2024-09-19T07:53:45.955372+0000","last_peered":"2024-09-19T07:53:45.955372+0000","last_clean":"2024-09-19T07:53:45.955372+0000","last_became_active":"2024-09-19T07:53:44.466195+0000","last_became_peered":"2024-09-19T07:53:44.466195+0000","last_unstale":"2024-09-19T07:53:45.955372+0000","last_undegraded":"2024-09-19T07:53:45.955372+0000","last_fullsized":"2024-09-19T07:53:45.955372+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:20.436455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.473757+0000","last_change":"2024-09-19T07:53:44.472944+0000","last_active":"2024-09-19T07:53:44.473757+0000","last_peered":"2024-09-19T07:53:44.473757+0000","last_clean":"2024-09-19T07:53:44.473757+0000","last_became_active":"2024-09-19T07:53:44.472669+0000","last_became_peered":"2024-09-19T07:53:44.472669+0000","last_unstale":"2024-09-19T07:53:44.473757+0000","last_undegraded":"2024-09-19T07:53:44.473757+0000","last_fullsized":"2024-09-19T07:53:44.473757+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:05.983399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470250+0000","last_change":"2024-09-19T07:53:44.466834+0000","last_active":"2024-09-19T07:53:44.470250+0000","last_peered":"2024-09-19T07:53:44.470250+0000","last_clean":"2024-09-19T07:53:44.470250+0000","last_became_active":"2024-09-19T07:53:44.466204+0000","last_became_peered":"2024-09-19T07:53:44.466204+0000","last_unstale":"2024-09-19T07:53:44.470250+0000","last_undegraded":"2024-09-19T07:53:44.470250+0000","last_fullsized":"2024-09-19T07:53:44.470250+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:00.929562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.481470+0000","last_change":"2024-09-19T07:53:44.480566+0000","last_active":"2024-09-19T07:53:44.481470+0000","last_peered":"2024-09-19T07:53:44.481470+0000","last_clean":"2024-09-19T07:53:44.481470+0000","last_became_active":"2024-09-19T07:53:44.480280+0000","last_became_peered":"2024-09-19T07:53:44.480280+0000","last_unstale":"2024-09-19T07:53:44.481470+0000","last_undegraded":"2024-09-19T07:53:44.481470+0000","last_fullsized":"2024-09-19T07:53:44.481470+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:46.275247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.464980+0000","last_change":"2024-09-19T07:53:44.462832+0000","last_active":"2024-09-19T07:53:44.464980+0000","last_peered":"2024-09-19T07:53:44.464980+0000","last_clean":"2024-09-19T07:53:44.464980+0000","last_became_active":"2024-09-19T07:53:44.462538+0000","last_became_peered":"2024-09-19T07:53:44.462538+0000","last_unstale":"2024-09-19T07:53:44.464980+0000","last_undegraded":"2024-09-19T07:53:44.464980+0000","last_fullsized":"2024-09-19T07:53:44.464980+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:45.343582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.465738+0000","last_change":"2024-09-19T07:53:44.460249+0000","last_active":"2024-09-19T07:53:44.465738+0000","last_peered":"2024-09-19T07:53:44.465738+0000","last_clean":"2024-09-19T07:53:44.465738+0000","last_became_active":"2024-09-19T07:53:44.458793+0000","last_became_peered":"2024-09-19T07:53:44.458793+0000","last_unstale":"2024-09-19T07:53:44.465738+0000","last_undegraded":"2024-09-19T07:53:44.465738+0000","last_fullsized":"2024-09-19T07:53:44.465738+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:37.126896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955551+0000","last_change":"2024-09-19T07:53:44.467665+0000","last_active":"2024-09-19T07:53:45.955551+0000","last_peered":"2024-09-19T07:53:45.955551+0000","last_clean":"2024-09-19T07:53:45.955551+0000","last_became_active":"2024-09-19T07:53:44.467388+0000","last_became_peered":"2024-09-19T07:53:44.467388+0000","last_unstale":"2024-09-19T07:53:45.955551+0000","last_undegraded":"2024-09-19T07:53:45.955551+0000","last_fullsized":"2024-09-19T07:53:45.955551+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:09.122218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.464494+0000","last_change":"2024-09-19T07:53:44.463945+0000","last_active":"2024-09-19T07:53:44.464494+0000","last_peered":"2024-09-19T07:53:44.464494+0000","last_clean":"2024-09-19T07:53:44.464494+0000","last_became_active":"2024-09-19T07:53:44.463622+0000","last_became_peered":"2024-09-19T07:53:44.463622+0000","last_unstale":"2024-09-19T07:53:44.464494+0000","last_undegraded":"2024-09-19T07:53:44.464494+0000","last_fullsized":"2024-09-19T07:53:44.464494+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:44.970456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.480727+0000","last_change":"2024-09-19T07:53:44.479121+0000","last_active":"2024-09-19T07:53:44.480727+0000","last_peered":"2024-09-19T07:53:44.480727+0000","last_clean":"2024-09-19T07:53:44.480727+0000","last_became_active":"2024-09-19T07:53:44.478632+0000","last_became_peered":"2024-09-19T07:53:44.478632+0000","last_unstale":"2024-09-19T07:53:44.480727+0000","last_undegraded":"2024-09-19T07:53:44.480727+0000","last_fullsized":"2024-09-19T07:53:44.480727+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:58.195995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955651+0000","last_change":"2024-09-19T07:53:44.471531+0000","last_active":"2024-09-19T07:53:45.955651+0000","last_peered":"2024-09-19T07:53:45.955651+0000","last_clean":"2024-09-19T07:53:45.955651+0000","last_became_active":"2024-09-19T07:53:44.471247+0000","last_became_peered":"2024-09-19T07:53:44.471247+0000","last_unstale":"2024-09-19T07:53:45.955651+0000","last_undegraded":"2024-09-19T07:53:45.955651+0000","last_fullsized":"2024-09-19T07:53:45.955651+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:27:11.047914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.452495+0000","last_change":"2024-09-19T07:53:43.452495+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.452495+0000","last_undegraded":"2024-09-19T07:53:43.452495+0000","last_fullsized":"2024-09-19T07:53:43.452495+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.464284+0000","last_change":"2024-09-19T07:53:44.461893+0000","last_active":"2024-09-19T07:53:44.464284+0000","last_peered":"2024-09-19T07:53:44.464284+0000","last_clean":"2024-09-19T07:53:44.464284+0000","last_became_active":"2024-09-19T07:53:44.461543+0000","last_became_peered":"2024-09-19T07:53:44.461543+0000","last_unstale":"2024-09-19T07:53:44.464284+0000","last_undegraded":"2024-09-19T07:53:44.464284+0000","last_fullsized":"2024-09-19T07:53:44.464284+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:02.071494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.464955+0000","last_change":"2024-09-19T07:53:44.462124+0000","last_active":"2024-09-19T07:53:44.464955+0000","last_peered":"2024-09-19T07:53:44.464955+0000","last_clean":"2024-09-19T07:53:44.464955+0000","last_became_active":"2024-09-19T07:53:44.461823+0000","last_became_peered":"2024-09-19T07:53:44.461823+0000","last_unstale":"2024-09-19T07:53:44.464955+0000","last_undegraded":"2024-09-19T07:53:44.464955+0000","last_fullsized":"2024-09-19T07:53:44.464955+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:19.276802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453302+0000","last_change":"2024-09-19T07:53:43.453302+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453302+0000","last_undegraded":"2024-09-19T07:53:43.453302+0000","last_fullsized":"2024-09-19T07:53:43.453302+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"117'4","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475117+0000","last_change":"2024-09-19T07:53:43.462786+0000","last_active":"2024-09-19T07:53:44.475117+0000","last_peered":"2024-09-19T07:53:44.475117+0000","last_clean":"2024-09-19T07:53:44.475117+0000","last_became_active":"2024-09-19T07:53:43.462517+0000","last_became_peered":"2024-09-19T07:53:43.462517+0000","last_unstale":"2024-09-19T07:53:44.475117+0000","last_undegraded":"2024-09-19T07:53:44.475117+0000","last_fullsized":"2024-09-19T07:53:44.475117+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:55.555956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.483675+0000","last_change":"2024-09-19T07:53:44.482782+0000","last_active":"2024-09-19T07:53:44.483675+0000","last_peered":"2024-09-19T07:53:44.483675+0000","last_clean":"2024-09-19T07:53:44.483675+0000","last_became_active":"2024-09-19T07:53:44.482497+0000","last_became_peered":"2024-09-19T07:53:44.482497+0000","last_unstale":"2024-09-19T07:53:44.483675+0000","last_undegraded":"2024-09-19T07:53:44.483675+0000","last_fullsized":"2024-09-19T07:53:44.483675+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:03.706766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.460833+0000","last_change":"2024-09-19T07:53:43.457859+0000","last_active":"2024-09-19T07:53:43.460833+0000","last_peered":"2024-09-19T07:53:43.460833+0000","last_clean":"2024-09-19T07:53:43.460833+0000","last_became_active":"2024-09-19T07:53:43.457574+0000","last_became_peered":"2024-09-19T07:53:43.457574+0000","last_unstale":"2024-09-19T07:53:43.460833+0000","last_undegraded":"2024-09-19T07:53:43.460833+0000","last_fullsized":"2024-09-19T07:53:43.460833+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:21.380860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.469707+0000","last_change":"2024-09-19T07:53:44.468508+0000","last_active":"2024-09-19T07:53:44.469707+0000","last_peered":"2024-09-19T07:53:44.469707+0000","last_clean":"2024-09-19T07:53:44.469707+0000","last_became_active":"2024-09-19T07:53:44.468228+0000","last_became_peered":"2024-09-19T07:53:44.468228+0000","last_unstale":"2024-09-19T07:53:44.469707+0000","last_undegraded":"2024-09-19T07:53:44.469707+0000","last_fullsized":"2024-09-19T07:53:44.469707+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.258505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.480926+0000","last_change":"2024-09-19T07:53:43.464281+0000","last_active":"2024-09-19T07:53:44.480926+0000","last_peered":"2024-09-19T07:53:44.480926+0000","last_clean":"2024-09-19T07:53:44.480926+0000","last_became_active":"2024-09-19T07:53:43.464005+0000","last_became_peered":"2024-09-19T07:53:43.464005+0000","last_unstale":"2024-09-19T07:53:44.480926+0000","last_undegraded":"2024-09-19T07:53:44.480926+0000","last_fullsized":"2024-09-19T07:53:44.480926+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:40.739604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470859+0000","last_change":"2024-09-19T07:53:44.463658+0000","last_active":"2024-09-19T07:53:44.470859+0000","last_peered":"2024-09-19T07:53:44.470859+0000","last_clean":"2024-09-19T07:53:44.470859+0000","last_became_active":"2024-09-19T07:53:44.463373+0000","last_became_peered":"2024-09-19T07:53:44.463373+0000","last_unstale":"2024-09-19T07:53:44.470859+0000","last_undegraded":"2024-09-19T07:53:44.470859+0000","last_fullsized":"2024-09-19T07:53:44.470859+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:05.734979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.461452+0000","last_change":"2024-09-19T07:53:43.459863+0000","last_active":"2024-09-19T07:53:43.461452+0000","last_peered":"2024-09-19T07:53:43.461452+0000","last_clean":"2024-09-19T07:53:43.461452+0000","last_became_active":"2024-09-19T07:53:43.459538+0000","last_became_peered":"2024-09-19T07:53:43.459538+0000","last_unstale":"2024-09-19T07:53:43.461452+0000","last_undegraded":"2024-09-19T07:53:43.461452+0000","last_fullsized":"2024-09-19T07:53:43.461452+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:21.308817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.450580+0000","last_change":"2024-09-19T07:53:43.450580+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.450580+0000","last_undegraded":"2024-09-19T07:53:43.450580+0000","last_fullsized":"2024-09-19T07:53:43.450580+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"117'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.467472+0000","last_change":"2024-09-19T07:53:43.459625+0000","last_active":"2024-09-19T07:53:44.467472+0000","last_peered":"2024-09-19T07:53:44.467472+0000","last_clean":"2024-09-19T07:53:44.467472+0000","last_became_active":"2024-09-19T07:53:43.459283+0000","last_became_peered":"2024-09-19T07:53:43.459283+0000","last_unstale":"2024-09-19T07:53:44.467472+0000","last_undegraded":"2024-09-19T07:53:44.467472+0000","last_fullsized":"2024-09-19T07:53:44.467472+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:37.240608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.463008+0000","last_change":"2024-09-19T07:53:44.461446+0000","last_active":"2024-09-19T07:53:44.463008+0000","last_peered":"2024-09-19T07:53:44.463008+0000","last_clean":"2024-09-19T07:53:44.463008+0000","last_became_active":"2024-09-19T07:53:44.461158+0000","last_became_peered":"2024-09-19T07:53:44.461158+0000","last_unstale":"2024-09-19T07:53:44.463008+0000","last_undegraded":"2024-09-19T07:53:44.463008+0000","last_fullsized":"2024-09-19T07:53:44.463008+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:13.056229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"117'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.485413+0000","last_change":"2024-09-19T07:53:43.467953+0000","last_active":"2024-09-19T07:53:44.485413+0000","last_peered":"2024-09-19T07:53:44.485413+0000","last_clean":"2024-09-19T07:53:44.485413+0000","last_became_active":"2024-09-19T07:53:43.467684+0000","last_became_peered":"2024-09-19T07:53:43.467684+0000","last_unstale":"2024-09-19T07:53:44.485413+0000","last_undegraded":"2024-09-19T07:53:44.485413+0000","last_fullsized":"2024-09-19T07:53:44.485413+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:46.197768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.451717+0000","last_change":"2024-09-19T07:53:43.451717+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.451717+0000","last_undegraded":"2024-09-19T07:53:43.451717+0000","last_fullsized":"2024-09-19T07:53:43.451717+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.453345+0000","last_change":"2024-09-19T07:53:43.458791+0000","last_active":"2024-09-19T07:53:44.453345+0000","last_peered":"2024-09-19T07:53:44.453345+0000","last_clean":"2024-09-19T07:53:44.453345+0000","last_became_active":"2024-09-19T07:53:43.458509+0000","last_became_peered":"2024-09-19T07:53:43.458509+0000","last_unstale":"2024-09-19T07:53:44.453345+0000","last_undegraded":"2024-09-19T07:53:44.453345+0000","last_fullsized":"2024-09-19T07:53:44.453345+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:54.555152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453976+0000","last_change":"2024-09-19T07:53:43.453976+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453976+0000","last_undegraded":"2024-09-19T07:53:43.453976+0000","last_fullsized":"2024-09-19T07:53:43.453976+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.460241+0000","last_change":"2024-09-19T07:53:43.466824+0000","last_active":"2024-09-19T07:53:44.460241+0000","last_peered":"2024-09-19T07:53:44.460241+0000","last_clean":"2024-09-19T07:53:44.460241+0000","last_became_active":"2024-09-19T07:53:43.466546+0000","last_became_peered":"2024-09-19T07:53:43.466546+0000","last_unstale":"2024-09-19T07:53:44.460241+0000","last_undegraded":"2024-09-19T07:53:44.460241+0000","last_fullsized":"2024-09-19T07:53:44.460241+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:52.318252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470048+0000","last_change":"2024-09-19T07:53:44.464599+0000","last_active":"2024-09-19T07:53:44.470048+0000","last_peered":"2024-09-19T07:53:44.470048+0000","last_clean":"2024-09-19T07:53:44.470048+0000","last_became_active":"2024-09-19T07:53:44.464335+0000","last_became_peered":"2024-09-19T07:53:44.464335+0000","last_unstale":"2024-09-19T07:53:44.470048+0000","last_undegraded":"2024-09-19T07:53:44.470048+0000","last_fullsized":"2024-09-19T07:53:44.470048+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:44.305938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.456285+0000","last_change":"2024-09-19T07:53:43.468918+0000","last_active":"2024-09-19T07:53:44.456285+0000","last_peered":"2024-09-19T07:53:44.456285+0000","last_clean":"2024-09-19T07:53:44.456285+0000","last_became_active":"2024-09-19T07:53:43.468594+0000","last_became_peered":"2024-09-19T07:53:43.468594+0000","last_unstale":"2024-09-19T07:53:44.456285+0000","last_undegraded":"2024-09-19T07:53:44.456285+0000","last_fullsized":"2024-09-19T07:53:44.456285+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:47.768851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.469144+0000","last_change":"2024-09-19T07:53:44.466217+0000","last_active":"2024-09-19T07:53:44.469144+0000","last_peered":"2024-09-19T07:53:44.469144+0000","last_clean":"2024-09-19T07:53:44.469144+0000","last_became_active":"2024-09-19T07:53:44.465405+0000","last_became_peered":"2024-09-19T07:53:44.465405+0000","last_unstale":"2024-09-19T07:53:44.469144+0000","last_undegraded":"2024-09-19T07:53:44.469144+0000","last_fullsized":"2024-09-19T07:53:44.469144+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:41.741615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.454379+0000","last_change":"2024-09-19T07:53:43.463233+0000","last_active":"2024-09-19T07:53:44.454379+0000","last_peered":"2024-09-19T07:53:44.454379+0000","last_clean":"2024-09-19T07:53:44.454379+0000","last_became_active":"2024-09-19T07:53:43.462620+0000","last_became_peered":"2024-09-19T07:53:43.462620+0000","last_unstale":"2024-09-19T07:53:44.454379+0000","last_undegraded":"2024-09-19T07:53:44.454379+0000","last_fullsized":"2024-09-19T07:53:44.454379+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:57:59.927552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.450354+0000","last_change":"2024-09-19T07:53:43.450354+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.450354+0000","last_undegraded":"2024-09-19T07:53:43.450354+0000","last_fullsized":"2024-09-19T07:53:43.450354+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"117'3","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475321+0000","last_change":"2024-09-19T07:53:43.463469+0000","last_active":"2024-09-19T07:53:44.475321+0000","last_peered":"2024-09-19T07:53:44.475321+0000","last_clean":"2024-09-19T07:53:44.475321+0000","last_became_active":"2024-09-19T07:53:43.463042+0000","last_became_peered":"2024-09-19T07:53:43.463042+0000","last_unstale":"2024-09-19T07:53:44.475321+0000","last_undegraded":"2024-09-19T07:53:44.475321+0000","last_fullsized":"2024-09-19T07:53:44.475321+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:32.163267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.469249+0000","last_change":"2024-09-19T07:53:44.466514+0000","last_active":"2024-09-19T07:53:44.469249+0000","last_peered":"2024-09-19T07:53:44.469249+0000","last_clean":"2024-09-19T07:53:44.469249+0000","last_became_active":"2024-09-19T07:53:44.464961+0000","last_became_peered":"2024-09-19T07:53:44.464961+0000","last_unstale":"2024-09-19T07:53:44.469249+0000","last_undegraded":"2024-09-19T07:53:44.469249+0000","last_fullsized":"2024-09-19T07:53:44.469249+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:49.561080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.465379+0000","last_change":"2024-09-19T07:53:43.460277+0000","last_active":"2024-09-19T07:53:43.465379+0000","last_peered":"2024-09-19T07:53:43.465379+0000","last_clean":"2024-09-19T07:53:43.465379+0000","last_became_active":"2024-09-19T07:53:43.460028+0000","last_became_peered":"2024-09-19T07:53:43.460028+0000","last_unstale":"2024-09-19T07:53:43.465379+0000","last_undegraded":"2024-09-19T07:53:43.465379+0000","last_fullsized":"2024-09-19T07:53:43.465379+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:27.528465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475155+0000","last_change":"2024-09-19T07:53:44.474267+0000","last_active":"2024-09-19T07:53:44.475155+0000","last_peered":"2024-09-19T07:53:44.475155+0000","last_clean":"2024-09-19T07:53:44.475155+0000","last_became_active":"2024-09-19T07:53:44.473919+0000","last_became_peered":"2024-09-19T07:53:44.473919+0000","last_unstale":"2024-09-19T07:53:44.475155+0000","last_undegraded":"2024-09-19T07:53:44.475155+0000","last_fullsized":"2024-09-19T07:53:44.475155+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:25.476480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.465360+0000","last_change":"2024-09-19T07:53:43.463274+0000","last_active":"2024-09-19T07:53:43.465360+0000","last_peered":"2024-09-19T07:53:43.465360+0000","last_clean":"2024-09-19T07:53:43.465360+0000","last_became_active":"2024-09-19T07:53:43.463007+0000","last_became_peered":"2024-09-19T07:53:43.463007+0000","last_unstale":"2024-09-19T07:53:43.465360+0000","last_undegraded":"2024-09-19T07:53:43.465360+0000","last_fullsized":"2024-09-19T07:53:43.465360+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:01.485835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453196+0000","last_change":"2024-09-19T07:53:43.453196+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453196+0000","last_undegraded":"2024-09-19T07:53:43.453196+0000","last_fullsized":"2024-09-19T07:53:43.453196+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.457723+0000","last_change":"2024-09-19T07:53:43.455019+0000","last_active":"2024-09-19T07:53:43.457723+0000","last_peered":"2024-09-19T07:53:43.457723+0000","last_clean":"2024-09-19T07:53:43.457723+0000","last_became_active":"2024-09-19T07:53:43.454686+0000","last_became_peered":"2024-09-19T07:53:43.454686+0000","last_unstale":"2024-09-19T07:53:43.457723+0000","last_undegraded":"2024-09-19T07:53:43.457723+0000","last_fullsized":"2024-09-19T07:53:43.457723+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:07.256495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956646+0000","last_change":"2024-09-19T07:53:44.457728+0000","last_active":"2024-09-19T07:53:45.956646+0000","last_peered":"2024-09-19T07:53:45.956646+0000","last_clean":"2024-09-19T07:53:45.956646+0000","last_became_active":"2024-09-19T07:53:44.457456+0000","last_became_peered":"2024-09-19T07:53:44.457456+0000","last_unstale":"2024-09-19T07:53:45.956646+0000","last_undegraded":"2024-09-19T07:53:45.956646+0000","last_fullsized":"2024-09-19T07:53:45.956646+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:37:56.076282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.468120+0000","last_change":"2024-09-19T07:53:43.466167+0000","last_active":"2024-09-19T07:53:43.468120+0000","last_peered":"2024-09-19T07:53:43.468120+0000","last_clean":"2024-09-19T07:53:43.468120+0000","last_became_active":"2024-09-19T07:53:43.465882+0000","last_became_peered":"2024-09-19T07:53:43.465882+0000","last_unstale":"2024-09-19T07:53:43.468120+0000","last_undegraded":"2024-09-19T07:53:43.468120+0000","last_fullsized":"2024-09-19T07:53:43.468120+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:04.981388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470860+0000","last_change":"2024-09-19T07:53:44.469161+0000","last_active":"2024-09-19T07:53:44.470860+0000","last_peered":"2024-09-19T07:53:44.470860+0000","last_clean":"2024-09-19T07:53:44.470860+0000","last_became_active":"2024-09-19T07:53:44.468896+0000","last_became_peered":"2024-09-19T07:53:44.468896+0000","last_unstale":"2024-09-19T07:53:44.470860+0000","last_undegraded":"2024-09-19T07:53:44.470860+0000","last_fullsized":"2024-09-19T07:53:44.470860+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:43.497533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.469677+0000","last_change":"2024-09-19T07:53:43.468990+0000","last_active":"2024-09-19T07:53:43.469677+0000","last_peered":"2024-09-19T07:53:43.469677+0000","last_clean":"2024-09-19T07:53:43.469677+0000","last_became_active":"2024-09-19T07:53:43.468625+0000","last_became_peered":"2024-09-19T07:53:43.468625+0000","last_unstale":"2024-09-19T07:53:43.469677+0000","last_undegraded":"2024-09-19T07:53:43.469677+0000","last_fullsized":"2024-09-19T07:53:43.469677+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:55.555956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470204+0000","last_change":"2024-09-19T07:53:44.466474+0000","last_active":"2024-09-19T07:53:44.470204+0000","last_peered":"2024-09-19T07:53:44.470204+0000","last_clean":"2024-09-19T07:53:44.470204+0000","last_became_active":"2024-09-19T07:53:44.466230+0000","last_became_peered":"2024-09-19T07:53:44.466230+0000","last_unstale":"2024-09-19T07:53:44.470204+0000","last_undegraded":"2024-09-19T07:53:44.470204+0000","last_fullsized":"2024-09-19T07:53:44.470204+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:55.557163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955645+0000","last_change":"2024-09-19T07:53:43.463386+0000","last_active":"2024-09-19T07:53:45.955645+0000","last_peered":"2024-09-19T07:53:45.955645+0000","last_clean":"2024-09-19T07:53:45.955645+0000","last_became_active":"2024-09-19T07:53:43.463020+0000","last_became_peered":"2024-09-19T07:53:43.463020+0000","last_unstale":"2024-09-19T07:53:45.955645+0000","last_undegraded":"2024-09-19T07:53:45.955645+0000","last_fullsized":"2024-09-19T07:53:45.955645+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:01.309102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.482421+0000","last_change":"2024-09-19T07:53:44.480939+0000","last_active":"2024-09-19T07:53:44.482421+0000","last_peered":"2024-09-19T07:53:44.482421+0000","last_clean":"2024-09-19T07:53:44.482421+0000","last_became_active":"2024-09-19T07:53:44.480634+0000","last_became_peered":"2024-09-19T07:53:44.480634+0000","last_unstale":"2024-09-19T07:53:44.482421+0000","last_undegraded":"2024-09-19T07:53:44.482421+0000","last_fullsized":"2024-09-19T07:53:44.482421+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:58.089951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"117'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475895+0000","last_change":"2024-09-19T07:53:43.464937+0000","last_active":"2024-09-19T07:53:44.475895+0000","last_peered":"2024-09-19T07:53:44.475895+0000","last_clean":"2024-09-19T07:53:44.475895+0000","last_became_active":"2024-09-19T07:53:43.464614+0000","last_became_peered":"2024-09-19T07:53:43.464614+0000","last_unstale":"2024-09-19T07:53:44.475895+0000","last_undegraded":"2024-09-19T07:53:44.475895+0000","last_fullsized":"2024-09-19T07:53:44.475895+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:48.559070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.469270+0000","last_change":"2024-09-19T07:53:44.466748+0000","last_active":"2024-09-19T07:53:44.469270+0000","last_peered":"2024-09-19T07:53:44.469270+0000","last_clean":"2024-09-19T07:53:44.469270+0000","last_became_active":"2024-09-19T07:53:44.465355+0000","last_became_peered":"2024-09-19T07:53:44.465355+0000","last_unstale":"2024-09-19T07:53:44.469270+0000","last_undegraded":"2024-09-19T07:53:44.469270+0000","last_fullsized":"2024-09-19T07:53:44.469270+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:05.734979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.456785+0000","last_change":"2024-09-19T07:53:43.466472+0000","last_active":"2024-09-19T07:53:44.456785+0000","last_peered":"2024-09-19T07:53:44.456785+0000","last_clean":"2024-09-19T07:53:44.456785+0000","last_became_active":"2024-09-19T07:53:43.466216+0000","last_became_peered":"2024-09-19T07:53:43.466216+0000","last_unstale":"2024-09-19T07:53:44.456785+0000","last_undegraded":"2024-09-19T07:53:44.456785+0000","last_fullsized":"2024-09-19T07:53:44.456785+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:44.341571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.482841+0000","last_change":"2024-09-19T07:53:44.480405+0000","last_active":"2024-09-19T07:53:44.482841+0000","last_peered":"2024-09-19T07:53:44.482841+0000","last_clean":"2024-09-19T07:53:44.482841+0000","last_became_active":"2024-09-19T07:53:44.479696+0000","last_became_peered":"2024-09-19T07:53:44.479696+0000","last_unstale":"2024-09-19T07:53:44.482841+0000","last_undegraded":"2024-09-19T07:53:44.482841+0000","last_fullsized":"2024-09-19T07:53:44.482841+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:23.633013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.476888+0000","last_change":"2024-09-19T07:53:43.463463+0000","last_active":"2024-09-19T07:53:44.476888+0000","last_peered":"2024-09-19T07:53:44.476888+0000","last_clean":"2024-09-19T07:53:44.476888+0000","last_became_active":"2024-09-19T07:53:43.463197+0000","last_became_peered":"2024-09-19T07:53:43.463197+0000","last_unstale":"2024-09-19T07:53:44.476888+0000","last_undegraded":"2024-09-19T07:53:44.476888+0000","last_fullsized":"2024-09-19T07:53:44.476888+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:27.266942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.451887+0000","last_change":"2024-09-19T07:53:43.451887+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.451887+0000","last_undegraded":"2024-09-19T07:53:43.451887+0000","last_fullsized":"2024-09-19T07:53:43.451887+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.456607+0000","last_change":"2024-09-19T07:53:43.465167+0000","last_active":"2024-09-19T07:53:44.456607+0000","last_peered":"2024-09-19T07:53:44.456607+0000","last_clean":"2024-09-19T07:53:44.456607+0000","last_became_active":"2024-09-19T07:53:43.464875+0000","last_became_peered":"2024-09-19T07:53:43.464875+0000","last_unstale":"2024-09-19T07:53:44.456607+0000","last_undegraded":"2024-09-19T07:53:44.456607+0000","last_fullsized":"2024-09-19T07:53:44.456607+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:18.274791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.450312+0000","last_change":"2024-09-19T07:53:43.450312+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.450312+0000","last_undegraded":"2024-09-19T07:53:43.450312+0000","last_fullsized":"2024-09-19T07:53:43.450312+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.468095+0000","last_change":"2024-09-19T07:53:43.465392+0000","last_active":"2024-09-19T07:53:43.468095+0000","last_peered":"2024-09-19T07:53:43.468095+0000","last_clean":"2024-09-19T07:53:43.468095+0000","last_became_active":"2024-09-19T07:53:43.465112+0000","last_became_peered":"2024-09-19T07:53:43.465112+0000","last_unstale":"2024-09-19T07:53:43.468095+0000","last_undegraded":"2024-09-19T07:53:43.468095+0000","last_fullsized":"2024-09-19T07:53:43.468095+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:02.066362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.462713+0000","last_change":"2024-09-19T07:53:44.461101+0000","last_active":"2024-09-19T07:53:44.462713+0000","last_peered":"2024-09-19T07:53:44.462713+0000","last_clean":"2024-09-19T07:53:44.462713+0000","last_became_active":"2024-09-19T07:53:44.460795+0000","last_became_peered":"2024-09-19T07:53:44.460795+0000","last_unstale":"2024-09-19T07:53:44.462713+0000","last_undegraded":"2024-09-19T07:53:44.462713+0000","last_fullsized":"2024-09-19T07:53:44.462713+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:59.717556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455598+0000","last_change":"2024-09-19T07:53:43.467250+0000","last_active":"2024-09-19T07:53:44.455598+0000","last_peered":"2024-09-19T07:53:44.455598+0000","last_clean":"2024-09-19T07:53:44.455598+0000","last_became_active":"2024-09-19T07:53:43.467030+0000","last_became_peered":"2024-09-19T07:53:43.467030+0000","last_unstale":"2024-09-19T07:53:44.455598+0000","last_undegraded":"2024-09-19T07:53:44.455598+0000","last_fullsized":"2024-09-19T07:53:44.455598+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:58.715546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.472936+0000","last_change":"2024-09-19T07:53:44.469053+0000","last_active":"2024-09-19T07:53:44.472936+0000","last_peered":"2024-09-19T07:53:44.472936+0000","last_clean":"2024-09-19T07:53:44.472936+0000","last_became_active":"2024-09-19T07:53:44.468779+0000","last_became_peered":"2024-09-19T07:53:44.468779+0000","last_unstale":"2024-09-19T07:53:44.472936+0000","last_undegraded":"2024-09-19T07:53:44.472936+0000","last_fullsized":"2024-09-19T07:53:44.472936+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:35.653193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475147+0000","last_change":"2024-09-19T07:53:43.464536+0000","last_active":"2024-09-19T07:53:44.475147+0000","last_peered":"2024-09-19T07:53:44.475147+0000","last_clean":"2024-09-19T07:53:44.475147+0000","last_became_active":"2024-09-19T07:53:43.462203+0000","last_became_peered":"2024-09-19T07:53:43.462203+0000","last_unstale":"2024-09-19T07:53:44.475147+0000","last_undegraded":"2024-09-19T07:53:44.475147+0000","last_fullsized":"2024-09-19T07:53:44.475147+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:35.927023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.453890+0000","last_change":"2024-09-19T07:53:43.453890+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.453890+0000","last_undegraded":"2024-09-19T07:53:43.453890+0000","last_fullsized":"2024-09-19T07:53:43.453890+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455906+0000","last_change":"2024-09-19T07:53:43.468893+0000","last_active":"2024-09-19T07:53:44.455906+0000","last_peered":"2024-09-19T07:53:44.455906+0000","last_clean":"2024-09-19T07:53:44.455906+0000","last_became_active":"2024-09-19T07:53:43.468581+0000","last_became_peered":"2024-09-19T07:53:43.468581+0000","last_unstale":"2024-09-19T07:53:44.455906+0000","last_undegraded":"2024-09-19T07:53:44.455906+0000","last_fullsized":"2024-09-19T07:53:44.455906+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:21.308817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.471585+0000","last_change":"2024-09-19T07:53:44.463751+0000","last_active":"2024-09-19T07:53:44.471585+0000","last_peered":"2024-09-19T07:53:44.471585+0000","last_clean":"2024-09-19T07:53:44.471585+0000","last_became_active":"2024-09-19T07:53:44.463490+0000","last_became_peered":"2024-09-19T07:53:44.463490+0000","last_unstale":"2024-09-19T07:53:44.471585+0000","last_undegraded":"2024-09-19T07:53:44.471585+0000","last_fullsized":"2024-09-19T07:53:44.471585+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.258505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.458476+0000","last_change":"2024-09-19T07:53:43.468830+0000","last_active":"2024-09-19T07:53:44.458476+0000","last_peered":"2024-09-19T07:53:44.458476+0000","last_clean":"2024-09-19T07:53:44.458476+0000","last_became_active":"2024-09-19T07:53:43.468521+0000","last_became_peered":"2024-09-19T07:53:43.468521+0000","last_unstale":"2024-09-19T07:53:44.458476+0000","last_undegraded":"2024-09-19T07:53:44.458476+0000","last_fullsized":"2024-09-19T07:53:44.458476+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:37.240608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956008+0000","last_change":"2024-09-19T07:53:44.472102+0000","last_active":"2024-09-19T07:53:45.956008+0000","last_peered":"2024-09-19T07:53:45.956008+0000","last_clean":"2024-09-19T07:53:45.956008+0000","last_became_active":"2024-09-19T07:53:44.471800+0000","last_became_peered":"2024-09-19T07:53:44.471800+0000","last_unstale":"2024-09-19T07:53:45.956008+0000","last_undegraded":"2024-09-19T07:53:45.956008+0000","last_fullsized":"2024-09-19T07:53:45.956008+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:06.662848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.459993+0000","last_change":"2024-09-19T07:53:43.464402+0000","last_active":"2024-09-19T07:53:44.459993+0000","last_peered":"2024-09-19T07:53:44.459993+0000","last_clean":"2024-09-19T07:53:44.459993+0000","last_became_active":"2024-09-19T07:53:43.464122+0000","last_became_peered":"2024-09-19T07:53:43.464122+0000","last_unstale":"2024-09-19T07:53:44.459993+0000","last_undegraded":"2024-09-19T07:53:44.459993+0000","last_fullsized":"2024-09-19T07:53:44.459993+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:34.228041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.472855+0000","last_change":"2024-09-19T07:53:44.467967+0000","last_active":"2024-09-19T07:53:44.472855+0000","last_peered":"2024-09-19T07:53:44.472855+0000","last_clean":"2024-09-19T07:53:44.472855+0000","last_became_active":"2024-09-19T07:53:44.467699+0000","last_became_peered":"2024-09-19T07:53:44.467699+0000","last_unstale":"2024-09-19T07:53:44.472855+0000","last_undegraded":"2024-09-19T07:53:44.472855+0000","last_fullsized":"2024-09-19T07:53:44.472855+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:53.377199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.451335+0000","last_change":"2024-09-19T07:53:43.454771+0000","last_active":"2024-09-19T07:53:44.451335+0000","last_peered":"2024-09-19T07:53:44.451335+0000","last_clean":"2024-09-19T07:53:44.451335+0000","last_became_active":"2024-09-19T07:53:43.454309+0000","last_became_peered":"2024-09-19T07:53:43.454309+0000","last_unstale":"2024-09-19T07:53:44.451335+0000","last_undegraded":"2024-09-19T07:53:44.451335+0000","last_fullsized":"2024-09-19T07:53:44.451335+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:43.968446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.451146+0000","last_change":"2024-09-19T07:53:43.451146+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.451146+0000","last_undegraded":"2024-09-19T07:53:43.451146+0000","last_fullsized":"2024-09-19T07:53:43.451146+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.451114+0000","last_change":"2024-09-19T07:53:43.459958+0000","last_active":"2024-09-19T07:53:44.451114+0000","last_peered":"2024-09-19T07:53:44.451114+0000","last_clean":"2024-09-19T07:53:44.451114+0000","last_became_active":"2024-09-19T07:53:43.459470+0000","last_became_peered":"2024-09-19T07:53:43.459470+0000","last_unstale":"2024-09-19T07:53:44.451114+0000","last_undegraded":"2024-09-19T07:53:44.451114+0000","last_fullsized":"2024-09-19T07:53:44.451114+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:47.768851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.473383+0000","last_change":"2024-09-19T07:53:44.470487+0000","last_active":"2024-09-19T07:53:44.473383+0000","last_peered":"2024-09-19T07:53:44.473383+0000","last_clean":"2024-09-19T07:53:44.473383+0000","last_became_active":"2024-09-19T07:53:44.470216+0000","last_became_peered":"2024-09-19T07:53:44.470216+0000","last_unstale":"2024-09-19T07:53:44.473383+0000","last_undegraded":"2024-09-19T07:53:44.473383+0000","last_fullsized":"2024-09-19T07:53:44.473383+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:02.487846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.460679+0000","last_change":"2024-09-19T07:53:43.467051+0000","last_active":"2024-09-19T07:53:44.460679+0000","last_peered":"2024-09-19T07:53:44.460679+0000","last_clean":"2024-09-19T07:53:44.460679+0000","last_became_active":"2024-09-19T07:53:43.466667+0000","last_became_peered":"2024-09-19T07:53:43.466667+0000","last_unstale":"2024-09-19T07:53:44.460679+0000","last_undegraded":"2024-09-19T07:53:44.460679+0000","last_fullsized":"2024-09-19T07:53:44.460679+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:49.561304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-19T07:53:43.450576+0000","last_change":"2024-09-19T07:53:43.450576+0000","last_active":"2024-09-19T07:53:43.440597+0000","last_peered":"2024-09-19T07:53:43.440597+0000","last_clean":"2024-09-19T07:53:43.440597+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:53:43.450576+0000","last_undegraded":"2024-09-19T07:53:43.450576+0000","last_fullsized":"2024-09-19T07:53:43.450576+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455682+0000","last_change":"2024-09-19T07:53:43.465164+0000","last_active":"2024-09-19T07:53:44.455682+0000","last_peered":"2024-09-19T07:53:44.455682+0000","last_clean":"2024-09-19T07:53:44.455682+0000","last_became_active":"2024-09-19T07:53:43.464888+0000","last_became_peered":"2024-09-19T07:53:43.464888+0000","last_unstale":"2024-09-19T07:53:44.455682+0000","last_undegraded":"2024-09-19T07:53:44.455682+0000","last_fullsized":"2024-09-19T07:53:44.455682+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:01.069484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.471656+0000","last_change":"2024-09-19T07:53:44.464883+0000","last_active":"2024-09-19T07:53:44.471656+0000","last_peered":"2024-09-19T07:53:44.471656+0000","last_clean":"2024-09-19T07:53:44.471656+0000","last_became_active":"2024-09-19T07:53:44.464617+0000","last_became_peered":"2024-09-19T07:53:44.464617+0000","last_unstale":"2024-09-19T07:53:44.471656+0000","last_undegraded":"2024-09-19T07:53:44.471656+0000","last_fullsized":"2024-09-19T07:53:44.471656+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:28.530475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-19T07:53:43.465335+0000","last_change":"2024-09-19T07:53:43.459564+0000","last_active":"2024-09-19T07:53:43.465335+0000","last_peered":"2024-09-19T07:53:43.465335+0000","last_clean":"2024-09-19T07:53:43.465335+0000","last_became_active":"2024-09-19T07:53:43.459126+0000","last_became_peered":"2024-09-19T07:53:43.459126+0000","last_unstale":"2024-09-19T07:53:43.465335+0000","last_undegraded":"2024-09-19T07:53:43.465335+0000","last_fullsized":"2024-09-19T07:53:43.465335+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:42.495522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":344,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455739+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:53:44.455739+0000","last_peered":"2024-09-19T07:53:44.455739+0000","last_clean":"2024-09-19T07:53:44.455739+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:53:44.455739+0000","last_undegraded":"2024-09-19T07:53:44.455739+0000","last_fullsized":"2024-09-19T07:53:44.455739+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":742,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509573,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83096,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288744,"statfs":{"total":96636764160,"available":96551673856,"internally_reserved":0,"allocated":2031616,"data_stored":1254678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509572,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104788,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94267052,"statfs":{"total":96636764160,"available":96529461248,"internally_reserved":0,"allocated":1437696,"data_stored":664310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509573,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72784,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299056,"statfs":{"total":96636764160,"available":96562233344,"internally_reserved":0,"allocated":1437696,"data_stored":664310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509573,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86676,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285164,"statfs":{"total":96636764160,"available":96548007936,"internally_reserved":0,"allocated":1437696,"data_stored":664310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509572,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105308,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266532,"statfs":{"total":96636764160,"available":96528928768,"internally_reserved":0,"allocated":2031616,"data_stored":1254678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542276,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118604,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253236,"statfs":{"total":96636764160,"available":96515313664,"internally_reserved":0,"allocated":1425408,"data_stored":657643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542276,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82564,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94289276,"statfs":{"total":96636764160,"available":96552218624,"internally_reserved":0,"allocated":1425408,"data_stored":657643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574982,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":108904,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262936,"statfs":{"total":96636764160,"available":96525246464,"internally_reserved":0,"allocated":1462272,"data_stored":678169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:53:48.480 INFO:tasks.ceph.ceph_manager.ceph:PG 18.3e is not active+clean 2024-09-19T07:53:48.480 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.455258+0000', 'last_change': '2024-09-19T07:53:43.455258+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.455258+0000', 'last_undegraded': '2024-09-19T07:53:43.455258+0000', 'last_fullsized': '2024-09-19T07:53:43.455258+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T07:53:48.480 INFO:tasks.ceph.ceph_manager.ceph:PG 18.3d is not active+clean 2024-09-19T07:53:48.480 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.454692+0000', 'last_change': '2024-09-19T07:53:43.454692+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.454692+0000', 'last_undegraded': '2024-09-19T07:53:43.454692+0000', 'last_fullsized': '2024-09-19T07:53:43.454692+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T07:53:48.481 INFO:tasks.ceph.ceph_manager.ceph:PG 18.38 is not active+clean 2024-09-19T07:53:48.481 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.454146+0000', 'last_change': '2024-09-19T07:53:43.454146+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.454146+0000', 'last_undegraded': '2024-09-19T07:53:43.454146+0000', 'last_fullsized': '2024-09-19T07:53:43.454146+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T07:53:48.481 INFO:tasks.ceph.ceph_manager.ceph:PG 18.36 is not active+clean 2024-09-19T07:53:48.481 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.450946+0000', 'last_change': '2024-09-19T07:53:43.450946+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.450946+0000', 'last_undegraded': '2024-09-19T07:53:43.450946+0000', 'last_fullsized': '2024-09-19T07:53:43.450946+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T07:53:48.481 INFO:tasks.ceph.ceph_manager.ceph:PG 18.33 is not active+clean 2024-09-19T07:53:48.481 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.454391+0000', 'last_change': '2024-09-19T07:53:43.454391+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.454391+0000', 'last_undegraded': '2024-09-19T07:53:43.454391+0000', 'last_fullsized': '2024-09-19T07:53:43.454391+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T07:53:48.481 INFO:tasks.ceph.ceph_manager.ceph:PG 18.32 is not active+clean 2024-09-19T07:53:48.481 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.453644+0000', 'last_change': '2024-09-19T07:53:43.453644+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.453644+0000', 'last_undegraded': '2024-09-19T07:53:43.453644+0000', 'last_fullsized': '2024-09-19T07:53:43.453644+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T07:53:48.481 INFO:tasks.ceph.ceph_manager.ceph:PG 18.2f is not active+clean 2024-09-19T07:53:48.481 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.453835+0000', 'last_change': '2024-09-19T07:53:43.453835+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.453835+0000', 'last_undegraded': '2024-09-19T07:53:43.453835+0000', 'last_fullsized': '2024-09-19T07:53:43.453835+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T07:53:48.481 INFO:tasks.ceph.ceph_manager.ceph:PG 18.2e is not active+clean 2024-09-19T07:53:48.482 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.453214+0000', 'last_change': '2024-09-19T07:53:43.453214+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.453214+0000', 'last_undegraded': '2024-09-19T07:53:43.453214+0000', 'last_fullsized': '2024-09-19T07:53:43.453214+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T07:53:48.482 INFO:tasks.ceph.ceph_manager.ceph:PG 18.22 is not active+clean 2024-09-19T07:53:48.482 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.452495+0000', 'last_change': '2024-09-19T07:53:43.452495+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.452495+0000', 'last_undegraded': '2024-09-19T07:53:43.452495+0000', 'last_fullsized': '2024-09-19T07:53:43.452495+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T07:53:48.482 INFO:tasks.ceph.ceph_manager.ceph:PG 18.1c is not active+clean 2024-09-19T07:53:48.482 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.453302+0000', 'last_change': '2024-09-19T07:53:43.453302+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.453302+0000', 'last_undegraded': '2024-09-19T07:53:43.453302+0000', 'last_fullsized': '2024-09-19T07:53:43.453302+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T07:53:48.482 INFO:tasks.ceph.ceph_manager.ceph:PG 18.18 is not active+clean 2024-09-19T07:53:48.482 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.450580+0000', 'last_change': '2024-09-19T07:53:43.450580+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.450580+0000', 'last_undegraded': '2024-09-19T07:53:43.450580+0000', 'last_fullsized': '2024-09-19T07:53:43.450580+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T07:53:48.482 INFO:tasks.ceph.ceph_manager.ceph:PG 18.1a is not active+clean 2024-09-19T07:53:48.482 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.451717+0000', 'last_change': '2024-09-19T07:53:43.451717+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.451717+0000', 'last_undegraded': '2024-09-19T07:53:43.451717+0000', 'last_fullsized': '2024-09-19T07:53:43.451717+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T07:53:48.482 INFO:tasks.ceph.ceph_manager.ceph:PG 18.1b is not active+clean 2024-09-19T07:53:48.482 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.453976+0000', 'last_change': '2024-09-19T07:53:43.453976+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.453976+0000', 'last_undegraded': '2024-09-19T07:53:43.453976+0000', 'last_fullsized': '2024-09-19T07:53:43.453976+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T07:53:48.483 INFO:tasks.ceph.ceph_manager.ceph:PG 18.16 is not active+clean 2024-09-19T07:53:48.483 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.450354+0000', 'last_change': '2024-09-19T07:53:43.450354+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.450354+0000', 'last_undegraded': '2024-09-19T07:53:43.450354+0000', 'last_fullsized': '2024-09-19T07:53:43.450354+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T07:53:48.483 INFO:tasks.ceph.ceph_manager.ceph:PG 18.11 is not active+clean 2024-09-19T07:53:48.483 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.453196+0000', 'last_change': '2024-09-19T07:53:43.453196+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.453196+0000', 'last_undegraded': '2024-09-19T07:53:43.453196+0000', 'last_fullsized': '2024-09-19T07:53:43.453196+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T07:53:48.483 INFO:tasks.ceph.ceph_manager.ceph:PG 18.9 is not active+clean 2024-09-19T07:53:48.483 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.451887+0000', 'last_change': '2024-09-19T07:53:43.451887+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.451887+0000', 'last_undegraded': '2024-09-19T07:53:43.451887+0000', 'last_fullsized': '2024-09-19T07:53:43.451887+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T07:53:48.483 INFO:tasks.ceph.ceph_manager.ceph:PG 18.a is not active+clean 2024-09-19T07:53:48.483 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.450312+0000', 'last_change': '2024-09-19T07:53:43.450312+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.450312+0000', 'last_undegraded': '2024-09-19T07:53:43.450312+0000', 'last_fullsized': '2024-09-19T07:53:43.450312+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T07:53:48.483 INFO:tasks.ceph.ceph_manager.ceph:PG 18.5 is not active+clean 2024-09-19T07:53:48.483 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.453890+0000', 'last_change': '2024-09-19T07:53:43.453890+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.453890+0000', 'last_undegraded': '2024-09-19T07:53:43.453890+0000', 'last_fullsized': '2024-09-19T07:53:43.453890+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T07:53:48.484 INFO:tasks.ceph.ceph_manager.ceph:PG 18.1 is not active+clean 2024-09-19T07:53:48.484 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.451146+0000', 'last_change': '2024-09-19T07:53:43.451146+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.451146+0000', 'last_undegraded': '2024-09-19T07:53:43.451146+0000', 'last_fullsized': '2024-09-19T07:53:43.451146+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T07:53:48.484 INFO:tasks.ceph.ceph_manager.ceph:PG 18.3 is not active+clean 2024-09-19T07:53:48.484 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:53:43.450576+0000', 'last_change': '2024-09-19T07:53:43.450576+0000', 'last_active': '2024-09-19T07:53:43.440597+0000', 'last_peered': '2024-09-19T07:53:43.440597+0000', 'last_clean': '2024-09-19T07:53:43.440597+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:53:43.450576+0000', 'last_undegraded': '2024-09-19T07:53:43.450576+0000', 'last_fullsized': '2024-09-19T07:53:43.450576+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'last_clean_scrub_stamp': '2024-09-19T07:53:43.440597+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T07:53:48.484 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T07:53:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.590+0000 7f01f6f9f640 1 Processor -- start 2024-09-19T07:53:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.590+0000 7f01f6f9f640 1 -- start start 2024-09-19T07:53:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.591+0000 7f01f6f9f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f01f0162a00 0x7f01f00d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.591+0000 7f01f6f9f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01f00d2110 con 0x7f01f00cfe10 2024-09-19T07:53:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.591+0000 7f01f6f9f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01f00d23f0 con 0x7f01f00b4f80 2024-09-19T07:53:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.591+0000 7f01f6f9f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f01f00d26d0 con 0x7f01f0162a00 2024-09-19T07:53:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.591+0000 7f01f679e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f01f00cfe10 0x7f01f00b7730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35662/0 (socket says 172.21.15.16:35662) 2024-09-19T07:53:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.591+0000 7f01f679e640 1 -- 172.21.15.16:0/4074573106 learned_addr learned my addr 172.21.15.16:0/4074573106 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.591+0000 7f01f5f9d640 1 --2- 172.21.15.16:0/4074573106 >> v2:172.21.15.146:3300/0 conn(0x7f01f0162a00 0x7f01f00d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.591+0000 7f01f4f9b640 1 -- 172.21.15.16:0/4074573106 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3440217014 0 0) 0x7f01f00d2110 con 0x7f01f00cfe10 2024-09-19T07:53:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.591+0000 7f01f4f9b640 1 -- 172.21.15.16:0/4074573106 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01dc003520 con 0x7f01f00cfe10 2024-09-19T07:53:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f4f9b640 1 -- 172.21.15.16:0/4074573106 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3575720804 0 0) 0x7f01dc003520 con 0x7f01f00cfe10 2024-09-19T07:53:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f4f9b640 1 -- 172.21.15.16:0/4074573106 >> v2:172.21.15.146:3300/0 conn(0x7f01f0162a00 msgr2=0x7f01f00d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f4f9b640 1 --2- 172.21.15.16:0/4074573106 >> v2:172.21.15.146:3300/0 conn(0x7f01f0162a00 0x7f01f00d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f4f9b640 1 -- 172.21.15.16:0/4074573106 >> v1:172.21.15.146:6789/0 conn(0x7f01f00b4f80 legacy=0x7f01f00b7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f4f9b640 1 -- 172.21.15.16:0/4074573106 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f00d29b0 con 0x7f01f00cfe10 2024-09-19T07:53:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f4f9b640 1 -- 172.21.15.16:0/4074573106 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f01e80019f0 con 0x7f01f00cfe10 2024-09-19T07:53:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f4f9b640 1 -- 172.21.15.16:0/4074573106 >> v1:172.21.15.16:6789/0 conn(0x7f01f00cfe10 legacy=0x7f01f00b7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f4f9b640 1 --2- 172.21.15.16:0/4074573106 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01dc0039b0 0x7f01dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f4f9b640 1 --2- 172.21.15.16:0/4074573106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01dc004800 0x7f01dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f4f9b640 1 --2- 172.21.15.16:0/4074573106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01dc0051c0 0x7f01dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f4f9b640 1 -- 172.21.15.16:0/4074573106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01f00d26d0 con 0x7f01dc0051c0 2024-09-19T07:53:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f4f9b640 1 -- 172.21.15.16:0/4074573106 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc00c160 con 0x7f01dc004800 2024-09-19T07:53:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f4f9b640 1 -- 172.21.15.16:0/4074573106 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f01dc00c360 con 0x7f01dc0039b0 2024-09-19T07:53:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f679e640 1 --2- 172.21.15.16:0/4074573106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01dc0051c0 0x7f01dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f579c640 1 --2- 172.21.15.16:0/4074573106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01dc004800 0x7f01dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f5f9d640 1 --2- 172.21.15.16:0/4074573106 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01dc0039b0 0x7f01dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f679e640 1 -- 172.21.15.16:0/4074573106 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01dc0039b0 msgr2=0x7f01dc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f679e640 1 --2- 172.21.15.16:0/4074573106 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01dc0039b0 0x7f01dc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f679e640 1 -- 172.21.15.16:0/4074573106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01dc004800 msgr2=0x7f01dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f679e640 1 --2- 172.21.15.16:0/4074573106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01dc004800 0x7f01dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.592+0000 7f01f679e640 1 -- 172.21.15.16:0/4074573106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01dc003520 con 0x7f01dc0051c0 2024-09-19T07:53:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.593+0000 7f01f679e640 1 -- 172.21.15.16:0/4074573106 reap_dead start 2024-09-19T07:53:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.593+0000 7f01f679e640 1 --2- 172.21.15.16:0/4074573106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01dc0051c0 0x7f01dc00b9c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f01e800a860 tx=0x7f01dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac1c547a3f8a4fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.593+0000 7f01f4f9b640 1 -- 172.21.15.16:0/4074573106 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e800a480 con 0x7f01dc0051c0 2024-09-19T07:53:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.593+0000 7f01f4f9b640 1 -- 172.21.15.16:0/4074573106 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e802f070 con 0x7f01dc0051c0 2024-09-19T07:53:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.593+0000 7f01f4f9b640 1 -- 172.21.15.16:0/4074573106 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e802bca0 con 0x7f01dc0051c0 2024-09-19T07:53:48.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.593+0000 7f01f6f9f640 1 -- 172.21.15.16:0/4074573106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01dc0051c0 msgr2=0x7f01dc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:48.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.593+0000 7f01f6f9f640 1 --2- 172.21.15.16:0/4074573106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01dc0051c0 0x7f01dc00b9c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f01e800a860 tx=0x7f01dc005830 comp rx=0 tx=0).stop 2024-09-19T07:53:48.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.594+0000 7f01f6f9f640 1 -- 172.21.15.16:0/4074573106 shutdown_connections 2024-09-19T07:53:48.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.594+0000 7f01f6f9f640 1 --2- 172.21.15.16:0/4074573106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01dc0051c0 0x7f01dc00b9c0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.594+0000 7f01f6f9f640 1 -- 172.21.15.16:0/4074573106 >> 172.21.15.16:0/4074573106 conn(0x7f01f009d520 msgr2=0x7f01f00d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:48.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.594+0000 7f01f6f9f640 1 -- 172.21.15.16:0/4074573106 shutdown_connections 2024-09-19T07:53:48.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.594+0000 7f01f6f9f640 1 -- 172.21.15.16:0/4074573106 wait complete. 2024-09-19T07:53:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.595+0000 7f01f6f9f640 1 Processor -- start 2024-09-19T07:53:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.595+0000 7f01f6f9f640 1 -- start start 2024-09-19T07:53:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01f6f9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01dc0051c0 0x7f01f00c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01f6f9f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01f01607c0 0x7f01f01ea600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01f6f9f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01f01eab40 0x7f01f01eb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01f6f9f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01f00d3330 con 0x7f01dc0051c0 2024-09-19T07:53:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01f6f9f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f01f00d31b0 con 0x7f01f01eab40 2024-09-19T07:53:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01f6f9f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f01f0162de0 con 0x7f01f01607c0 2024-09-19T07:53:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01f5f9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01dc0051c0 0x7f01f00c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01f5f9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01dc0051c0 0x7f01f00c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52782/0 (socket says 172.21.15.16:52782) 2024-09-19T07:53:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01f5f9d640 1 -- 172.21.15.16:0/643238587 learned_addr learned my addr 172.21.15.16:0/643238587 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01f579c640 1 --2- 172.21.15.16:0/643238587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01f01607c0 0x7f01f01ea600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01f679e640 1 --2- 172.21.15.16:0/643238587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01f01eab40 0x7f01f01eb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01f5f9d640 1 -- 172.21.15.16:0/643238587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01f01607c0 msgr2=0x7f01f01ea600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01f5f9d640 1 --2- 172.21.15.16:0/643238587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01f01607c0 0x7f01f01ea600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01f5f9d640 1 -- 172.21.15.16:0/643238587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01f01eab40 msgr2=0x7f01f01eb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01f5f9d640 1 --2- 172.21.15.16:0/643238587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01f01eab40 0x7f01f01eb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01f5f9d640 1 -- 172.21.15.16:0/643238587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f015b330 con 0x7f01dc0051c0 2024-09-19T07:53:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01f5f9d640 1 --2- 172.21.15.16:0/643238587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01dc0051c0 0x7f01f00c7150 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f01ec011910 tx=0x7f01ec011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01e67fc640 1 -- 172.21.15.16:0/643238587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00fa90 con 0x7f01dc0051c0 2024-09-19T07:53:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01e67fc640 1 -- 172.21.15.16:0/643238587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ec00fc30 con 0x7f01dc0051c0 2024-09-19T07:53:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01f6f9f640 1 -- 172.21.15.16:0/643238587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f01e88a0 con 0x7f01dc0051c0 2024-09-19T07:53:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01e67fc640 1 -- 172.21.15.16:0/643238587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec014440 con 0x7f01dc0051c0 2024-09-19T07:53:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.596+0000 7f01f6f9f640 1 -- 172.21.15.16:0/643238587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f01e8e60 con 0x7f01dc0051c0 2024-09-19T07:53:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.597+0000 7f01f6f9f640 1 -- 172.21.15.16:0/643238587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f00d0900 con 0x7f01dc0051c0 2024-09-19T07:53:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.598+0000 7f01e67fc640 1 -- 172.21.15.16:0/643238587 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f01ec012040 con 0x7f01dc0051c0 2024-09-19T07:53:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.599+0000 7f01e67fc640 1 --2- 172.21.15.16:0/643238587 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01d406c2b0 0x7f01d406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.599+0000 7f01e67fc640 1 -- 172.21.15.16:0/643238587 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f01ec08bb10 con 0x7f01dc0051c0 2024-09-19T07:53:48.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.599+0000 7f01f579c640 1 --2- 172.21.15.16:0/643238587 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01d406c2b0 0x7f01d406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.599+0000 7f01f579c640 1 --2- 172.21.15.16:0/643238587 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01d406c2b0 0x7f01d406e770 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f01e0000c00 tx=0x7f01e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:48.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.603+0000 7f01e67fc640 1 -- 172.21.15.16:0/643238587 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f01f015b330 con 0x7f01dc0051c0 2024-09-19T07:53:48.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.847+0000 7f01f6f9f640 1 -- 172.21.15.16:0/643238587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f01f00b5a20 con 0x7f01dc0051c0 2024-09-19T07:53:48.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.848+0000 7f01e67fc640 1 -- 172.21.15.16:0/643238587 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1561 (secure 0 0 0) 0x7f01ec019030 con 0x7f01dc0051c0 2024-09-19T07:53:48.848 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:48.848 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":666,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":118,"num_osds":8,"num_up_osds":8,"osd_up_since":1726731773,"num_in_osds":8,"osd_in_since":1726731764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":77},{"state_name":"creating+peering","count":20}],"num_pgs":97,"num_pools":3,"num_objects":11,"data_bytes":591110,"bytes_used":781029376,"bytes_avail":772313083904,"bytes_total":773094113280,"inactive_pgs_ratio":0.20618556439876556,"write_bytes_sec":737,"read_op_per_sec":0,"write_op_per_sec":4},"fsmap":{"epoch":63,"btime":"2024-09-19T07:53:46:466514+0000","id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":7368}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-19T07:53:32.607149+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T07:53:48.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.849+0000 7f01f6f9f640 1 -- 172.21.15.16:0/643238587 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01d406c2b0 msgr2=0x7f01d406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:48.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.849+0000 7f01f6f9f640 1 --2- 172.21.15.16:0/643238587 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01d406c2b0 0x7f01d406e770 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f01e0000c00 tx=0x7f01e0009040 comp rx=0 tx=0).stop 2024-09-19T07:53:48.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.849+0000 7f01f6f9f640 1 -- 172.21.15.16:0/643238587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01dc0051c0 msgr2=0x7f01f00c7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:48.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.849+0000 7f01f6f9f640 1 --2- 172.21.15.16:0/643238587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01dc0051c0 0x7f01f00c7150 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f01ec011910 tx=0x7f01ec011dd0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.849+0000 7f01f6f9f640 1 -- 172.21.15.16:0/643238587 shutdown_connections 2024-09-19T07:53:48.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.849+0000 7f01f6f9f640 1 --2- 172.21.15.16:0/643238587 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01d406c2b0 0x7f01d406e770 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.849+0000 7f01f6f9f640 1 --2- 172.21.15.16:0/643238587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01f01eab40 0x7f01f01eb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.849+0000 7f01f6f9f640 1 --2- 172.21.15.16:0/643238587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01f01607c0 0x7f01f01ea600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.849+0000 7f01f6f9f640 1 --2- 172.21.15.16:0/643238587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01dc0051c0 0x7f01f00c7150 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.849+0000 7f01f6f9f640 1 -- 172.21.15.16:0/643238587 >> 172.21.15.16:0/643238587 conn(0x7f01f009d520 msgr2=0x7f01f00b8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:48.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.850+0000 7f01f6f9f640 1 -- 172.21.15.16:0/643238587 shutdown_connections 2024-09-19T07:53:48.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.850+0000 7f01f6f9f640 1 -- 172.21.15.16:0/643238587 wait complete. 2024-09-19T07:53:48.861 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-19T07:53:48.861 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:53:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.963+0000 7fa348c0a640 1 Processor -- start 2024-09-19T07:53:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.963+0000 7fa348c0a640 1 -- start start 2024-09-19T07:53:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.964+0000 7fa348c0a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa34415e2a0 0x7fa34406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.964+0000 7fa348c0a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa34415e680 0x7fa3440cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.964+0000 7fa348c0a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa34406d960 0x7fa3440d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.964+0000 7fa348c0a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa3440d1880 con 0x7fa34415e2a0 2024-09-19T07:53:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.964+0000 7fa348c0a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa3440d1b60 con 0x7fa34415e680 2024-09-19T07:53:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.964+0000 7fa348c0a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa3440cf2d0 con 0x7fa34406d960 2024-09-19T07:53:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.964+0000 7fa34259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa34415e2a0 0x7fa34406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.964+0000 7fa34259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa34415e2a0 0x7fa34406d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52792/0 (socket says 172.21.15.16:52792) 2024-09-19T07:53:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.964+0000 7fa34259c640 1 -- 172.21.15.16:0/1737932623 learned_addr learned my addr 172.21.15.16:0/1737932623 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.964+0000 7fa34259c640 1 -- 172.21.15.16:0/1737932623 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa34406d960 msgr2=0x7fa3440d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.964+0000 7fa34259c640 1 --2- 172.21.15.16:0/1737932623 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa34406d960 0x7fa3440d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.964+0000 7fa34259c640 1 -- 172.21.15.16:0/1737932623 >> v2:172.21.15.146:3300/0 conn(0x7fa34415e680 msgr2=0x7fa3440cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.964+0000 7fa341d9b640 1 --2- 172.21.15.16:0/1737932623 >> v2:172.21.15.146:3300/0 conn(0x7fa34415e680 0x7fa3440cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.964+0000 7fa34259c640 1 --2- 172.21.15.16:0/1737932623 >> v2:172.21.15.146:3300/0 conn(0x7fa34415e680 0x7fa3440cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.964+0000 7fa34259c640 1 -- 172.21.15.16:0/1737932623 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3440cf5b0 con 0x7fa34415e2a0 2024-09-19T07:53:48.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.965+0000 7fa34259c640 1 --2- 172.21.15.16:0/1737932623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa34415e2a0 0x7fa34406d420 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa3440a22a0 tx=0x7fa33402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93cabb3976a4e2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:48.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.965+0000 7fa34159a640 1 -- 172.21.15.16:0/1737932623 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334037070 con 0x7fa34415e2a0 2024-09-19T07:53:48.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.965+0000 7fa34159a640 1 -- 172.21.15.16:0/1737932623 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa334032440 con 0x7fa34415e2a0 2024-09-19T07:53:48.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.965+0000 7fa34159a640 1 -- 172.21.15.16:0/1737932623 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334032810 con 0x7fa34415e2a0 2024-09-19T07:53:48.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.966+0000 7fa348c0a640 1 -- 172.21.15.16:0/1737932623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa34415e2a0 msgr2=0x7fa34406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.966+0000 7fa348c0a640 1 --2- 172.21.15.16:0/1737932623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa34415e2a0 0x7fa34406d420 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa3440a22a0 tx=0x7fa33402a1e0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.966+0000 7fa348c0a640 1 -- 172.21.15.16:0/1737932623 shutdown_connections 2024-09-19T07:53:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.966+0000 7fa348c0a640 1 --2- 172.21.15.16:0/1737932623 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa34406d960 0x7fa3440d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.966+0000 7fa348c0a640 1 --2- 172.21.15.16:0/1737932623 >> v2:172.21.15.146:3300/0 conn(0x7fa34415e680 0x7fa3440cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.966+0000 7fa348c0a640 1 --2- 172.21.15.16:0/1737932623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa34415e2a0 0x7fa34406d420 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.966+0000 7fa348c0a640 1 -- 172.21.15.16:0/1737932623 >> 172.21.15.16:0/1737932623 conn(0x7fa34409d760 msgr2=0x7fa34408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.966+0000 7fa348c0a640 1 -- 172.21.15.16:0/1737932623 shutdown_connections 2024-09-19T07:53:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.966+0000 7fa348c0a640 1 -- 172.21.15.16:0/1737932623 wait complete. 2024-09-19T07:53:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.967+0000 7fa348c0a640 1 Processor -- start 2024-09-19T07:53:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.967+0000 7fa348c0a640 1 -- start start 2024-09-19T07:53:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.968+0000 7fa348c0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa34406d960 0x7fa344157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.968+0000 7fa348c0a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa34415e2a0 0x7fa34414c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.968+0000 7fa348c0a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa34415e680 0x7fa3440c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.968+0000 7fa348c0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3440d44f0 con 0x7fa34406d960 2024-09-19T07:53:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.968+0000 7fa348c0a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa3440d4370 con 0x7fa34415e680 2024-09-19T07:53:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.968+0000 7fa348c0a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa3440d4670 con 0x7fa34415e2a0 2024-09-19T07:53:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.968+0000 7fa34259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa34406d960 0x7fa344157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.968+0000 7fa34259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa34406d960 0x7fa344157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52804/0 (socket says 172.21.15.16:52804) 2024-09-19T07:53:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.968+0000 7fa34259c640 1 -- 172.21.15.16:0/3816395427 learned_addr learned my addr 172.21.15.16:0/3816395427 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.968+0000 7fa341d9b640 1 --2- 172.21.15.16:0/3816395427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa34415e2a0 0x7fa34414c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.968+0000 7fa342d9d640 1 --2- 172.21.15.16:0/3816395427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa34415e680 0x7fa3440c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.968+0000 7fa34259c640 1 -- 172.21.15.16:0/3816395427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa34415e2a0 msgr2=0x7fa34414c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.968+0000 7fa34259c640 1 --2- 172.21.15.16:0/3816395427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa34415e2a0 0x7fa34414c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.968+0000 7fa34259c640 1 -- 172.21.15.16:0/3816395427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa34415e680 msgr2=0x7fa3440c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.968+0000 7fa34259c640 1 --2- 172.21.15.16:0/3816395427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa34415e680 0x7fa3440c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.968+0000 7fa34259c640 1 -- 172.21.15.16:0/3816395427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3440b9b60 con 0x7fa34406d960 2024-09-19T07:53:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.969+0000 7fa34259c640 1 --2- 172.21.15.16:0/3816395427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa34406d960 0x7fa344157250 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fa334000c00 tx=0x7fa3340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.969+0000 7fa332ffd640 1 -- 172.21.15.16:0/3816395427 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334041020 con 0x7fa34406d960 2024-09-19T07:53:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.969+0000 7fa332ffd640 1 -- 172.21.15.16:0/3816395427 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa334030970 con 0x7fa34406d960 2024-09-19T07:53:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.969+0000 7fa348c0a640 1 -- 172.21.15.16:0/3816395427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3440b8da0 con 0x7fa34406d960 2024-09-19T07:53:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.969+0000 7fa332ffd640 1 -- 172.21.15.16:0/3816395427 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334037040 con 0x7fa34406d960 2024-09-19T07:53:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.969+0000 7fa348c0a640 1 -- 172.21.15.16:0/3816395427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3440b9360 con 0x7fa34406d960 2024-09-19T07:53:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.970+0000 7fa348c0a640 1 -- 172.21.15.16:0/3816395427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa34406d420 con 0x7fa34406d960 2024-09-19T07:53:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.970+0000 7fa332ffd640 1 -- 172.21.15.16:0/3816395427 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa334044020 con 0x7fa34406d960 2024-09-19T07:53:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.971+0000 7fa332ffd640 1 --2- 172.21.15.16:0/3816395427 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa32406c2b0 0x7fa32406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.971+0000 7fa332ffd640 1 -- 172.21.15.16:0/3816395427 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fa3340abf60 con 0x7fa34406d960 2024-09-19T07:53:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.971+0000 7fa341d9b640 1 --2- 172.21.15.16:0/3816395427 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa32406c2b0 0x7fa32406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.972+0000 7fa341d9b640 1 --2- 172.21.15.16:0/3816395427 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa32406c2b0 0x7fa32406e770 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa338006fd0 tx=0x7fa338008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:48.975+0000 7fa332ffd640 1 -- 172.21.15.16:0/3816395427 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa33404b030 con 0x7fa34406d960 2024-09-19T07:53:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:49.147+0000 7fa348c0a640 1 -- 172.21.15.16:0/3816395427 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa3440cb230 con 0x7fa32406c2b0 2024-09-19T07:53:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:49.153+0000 7fa332ffd640 1 -- 172.21.15.16:0/3816395427 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252454 (secure 0 0 0) 0x7fa3440cb230 con 0x7fa32406c2b0 2024-09-19T07:53:49.156 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:49.157 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:53:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:49.157+0000 7fa348c0a640 1 -- 172.21.15.16:0/3816395427 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa32406c2b0 msgr2=0x7fa32406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:49.157+0000 7fa348c0a640 1 --2- 172.21.15.16:0/3816395427 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa32406c2b0 0x7fa32406e770 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa338006fd0 tx=0x7fa338008040 comp rx=0 tx=0).stop 2024-09-19T07:53:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:49.157+0000 7fa348c0a640 1 -- 172.21.15.16:0/3816395427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa34406d960 msgr2=0x7fa344157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:49.157+0000 7fa348c0a640 1 --2- 172.21.15.16:0/3816395427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa34406d960 0x7fa344157250 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fa334000c00 tx=0x7fa3340046c0 comp rx=0 tx=0).stop 2024-09-19T07:53:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:49.157+0000 7fa348c0a640 1 -- 172.21.15.16:0/3816395427 shutdown_connections 2024-09-19T07:53:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:49.157+0000 7fa348c0a640 1 --2- 172.21.15.16:0/3816395427 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa32406c2b0 0x7fa32406e770 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:49.157+0000 7fa348c0a640 1 --2- 172.21.15.16:0/3816395427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa34415e680 0x7fa3440c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:49.157+0000 7fa348c0a640 1 --2- 172.21.15.16:0/3816395427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa34415e2a0 0x7fa34414c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:49.157+0000 7fa348c0a640 1 --2- 172.21.15.16:0/3816395427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa34406d960 0x7fa344157250 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:49.157+0000 7fa348c0a640 1 -- 172.21.15.16:0/3816395427 >> 172.21.15.16:0/3816395427 conn(0x7fa34409d760 msgr2=0x7fa34415f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:49.157+0000 7fa348c0a640 1 -- 172.21.15.16:0/3816395427 shutdown_connections 2024-09-19T07:53:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:49.157+0000 7fa348c0a640 1 -- 172.21.15.16:0/3816395427 wait complete. 2024-09-19T07:53:49.168 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-09-19T07:53:48.610533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":762808,"kb_used_data":12484,"kb_used_omap":12,"kb_used_meta":750131,"kb_avail":754211912,"statfs":{"total":773094113280,"available":772312997888,"internally_reserved":0,"allocated":12783616,"data_stored":6538461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":768134704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001389"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455920+0000","last_change":"2024-09-19T07:53:44.454037+0000","last_active":"2024-09-19T07:53:44.455920+0000","last_peered":"2024-09-19T07:53:44.455920+0000","last_clean":"2024-09-19T07:53:44.455920+0000","last_became_active":"2024-09-19T07:53:44.453756+0000","last_became_peered":"2024-09-19T07:53:44.453756+0000","last_unstale":"2024-09-19T07:53:44.455920+0000","last_undegraded":"2024-09-19T07:53:44.455920+0000","last_fullsized":"2024-09-19T07:53:44.455920+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:44.305938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955591+0000","last_change":"2024-09-19T07:53:44.474712+0000","last_active":"2024-09-19T07:53:45.955591+0000","last_peered":"2024-09-19T07:53:45.955591+0000","last_clean":"2024-09-19T07:53:45.955591+0000","last_became_active":"2024-09-19T07:53:44.474083+0000","last_became_peered":"2024-09-19T07:53:44.474083+0000","last_unstale":"2024-09-19T07:53:45.955591+0000","last_undegraded":"2024-09-19T07:53:45.955591+0000","last_fullsized":"2024-09-19T07:53:45.955591+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:36.929034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956412+0000","last_change":"2024-09-19T07:53:44.481309+0000","last_active":"2024-09-19T07:53:45.956412+0000","last_peered":"2024-09-19T07:53:45.956412+0000","last_clean":"2024-09-19T07:53:45.956412+0000","last_became_active":"2024-09-19T07:53:44.480479+0000","last_became_peered":"2024-09-19T07:53:44.480479+0000","last_unstale":"2024-09-19T07:53:45.956412+0000","last_undegraded":"2024-09-19T07:53:45.956412+0000","last_fullsized":"2024-09-19T07:53:45.956412+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:53.377199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.471544+0000","last_change":"2024-09-19T07:53:44.463853+0000","last_active":"2024-09-19T07:53:44.471544+0000","last_peered":"2024-09-19T07:53:44.471544+0000","last_clean":"2024-09-19T07:53:44.471544+0000","last_became_active":"2024-09-19T07:53:44.463569+0000","last_became_peered":"2024-09-19T07:53:44.463569+0000","last_unstale":"2024-09-19T07:53:44.471544+0000","last_undegraded":"2024-09-19T07:53:44.471544+0000","last_fullsized":"2024-09-19T07:53:44.471544+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:43.497533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.462115+0000","last_change":"2024-09-19T07:53:44.460542+0000","last_active":"2024-09-19T07:53:44.462115+0000","last_peered":"2024-09-19T07:53:44.462115+0000","last_clean":"2024-09-19T07:53:44.462115+0000","last_became_active":"2024-09-19T07:53:44.460245+0000","last_became_peered":"2024-09-19T07:53:44.460245+0000","last_unstale":"2024-09-19T07:53:44.462115+0000","last_undegraded":"2024-09-19T07:53:44.462115+0000","last_fullsized":"2024-09-19T07:53:44.462115+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:39:04.627509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.482181+0000","last_change":"2024-09-19T07:53:44.480188+0000","last_active":"2024-09-19T07:53:44.482181+0000","last_peered":"2024-09-19T07:53:44.482181+0000","last_clean":"2024-09-19T07:53:44.482181+0000","last_became_active":"2024-09-19T07:53:44.479867+0000","last_became_peered":"2024-09-19T07:53:44.479867+0000","last_unstale":"2024-09-19T07:53:44.482181+0000","last_undegraded":"2024-09-19T07:53:44.482181+0000","last_fullsized":"2024-09-19T07:53:44.482181+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:21.929840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.477127+0000","last_change":"2024-09-19T07:53:44.475378+0000","last_active":"2024-09-19T07:53:44.477127+0000","last_peered":"2024-09-19T07:53:44.477127+0000","last_clean":"2024-09-19T07:53:44.477127+0000","last_became_active":"2024-09-19T07:53:44.475024+0000","last_became_peered":"2024-09-19T07:53:44.475024+0000","last_unstale":"2024-09-19T07:53:44.477127+0000","last_undegraded":"2024-09-19T07:53:44.477127+0000","last_fullsized":"2024-09-19T07:53:44.477127+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:13.004954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955416+0000","last_change":"2024-09-19T07:53:44.483364+0000","last_active":"2024-09-19T07:53:45.955416+0000","last_peered":"2024-09-19T07:53:45.955416+0000","last_clean":"2024-09-19T07:53:45.955416+0000","last_became_active":"2024-09-19T07:53:44.481115+0000","last_became_peered":"2024-09-19T07:53:44.481115+0000","last_unstale":"2024-09-19T07:53:45.955416+0000","last_undegraded":"2024-09-19T07:53:45.955416+0000","last_fullsized":"2024-09-19T07:53:45.955416+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:26.365967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.474046+0000","last_change":"2024-09-19T07:53:44.472235+0000","last_active":"2024-09-19T07:53:44.474046+0000","last_peered":"2024-09-19T07:53:44.474046+0000","last_clean":"2024-09-19T07:53:44.474046+0000","last_became_active":"2024-09-19T07:53:44.471961+0000","last_became_peered":"2024-09-19T07:53:44.471961+0000","last_unstale":"2024-09-19T07:53:44.474046+0000","last_undegraded":"2024-09-19T07:53:44.474046+0000","last_fullsized":"2024-09-19T07:53:44.474046+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:03.068372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.450177+0000","last_change":"2024-09-19T07:53:44.469391+0000","last_active":"2024-09-19T07:53:45.450177+0000","last_peered":"2024-09-19T07:53:45.450177+0000","last_clean":"2024-09-19T07:53:45.450177+0000","last_became_active":"2024-09-19T07:53:44.469111+0000","last_became_peered":"2024-09-19T07:53:44.469111+0000","last_unstale":"2024-09-19T07:53:45.450177+0000","last_undegraded":"2024-09-19T07:53:45.450177+0000","last_fullsized":"2024-09-19T07:53:45.450177+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:28.530475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.462065+0000","last_change":"2024-09-19T07:53:44.460409+0000","last_active":"2024-09-19T07:53:44.462065+0000","last_peered":"2024-09-19T07:53:44.462065+0000","last_clean":"2024-09-19T07:53:44.462065+0000","last_became_active":"2024-09-19T07:53:44.460137+0000","last_became_peered":"2024-09-19T07:53:44.460137+0000","last_unstale":"2024-09-19T07:53:44.462065+0000","last_undegraded":"2024-09-19T07:53:44.462065+0000","last_fullsized":"2024-09-19T07:53:44.462065+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:25.476480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.472912+0000","last_change":"2024-09-19T07:53:44.467120+0000","last_active":"2024-09-19T07:53:44.472912+0000","last_peered":"2024-09-19T07:53:44.472912+0000","last_clean":"2024-09-19T07:53:44.472912+0000","last_became_active":"2024-09-19T07:53:44.466828+0000","last_became_peered":"2024-09-19T07:53:44.466828+0000","last_unstale":"2024-09-19T07:53:44.472912+0000","last_undegraded":"2024-09-19T07:53:44.472912+0000","last_fullsized":"2024-09-19T07:53:44.472912+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:00.047940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.449972+0000","last_change":"2024-09-19T07:53:44.455777+0000","last_active":"2024-09-19T07:53:45.449972+0000","last_peered":"2024-09-19T07:53:45.449972+0000","last_clean":"2024-09-19T07:53:45.449972+0000","last_became_active":"2024-09-19T07:53:44.455467+0000","last_became_peered":"2024-09-19T07:53:44.455467+0000","last_unstale":"2024-09-19T07:53:45.449972+0000","last_undegraded":"2024-09-19T07:53:45.449972+0000","last_fullsized":"2024-09-19T07:53:45.449972+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:41.741615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955692+0000","last_change":"2024-09-19T07:53:44.481813+0000","last_active":"2024-09-19T07:53:45.955692+0000","last_peered":"2024-09-19T07:53:45.955692+0000","last_clean":"2024-09-19T07:53:45.955692+0000","last_became_active":"2024-09-19T07:53:44.480915+0000","last_became_peered":"2024-09-19T07:53:44.480915+0000","last_unstale":"2024-09-19T07:53:45.955692+0000","last_undegraded":"2024-09-19T07:53:45.955692+0000","last_fullsized":"2024-09-19T07:53:45.955692+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:47.199779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.456674+0000","last_change":"2024-09-19T07:53:44.455460+0000","last_active":"2024-09-19T07:53:44.456674+0000","last_peered":"2024-09-19T07:53:44.456674+0000","last_clean":"2024-09-19T07:53:44.456674+0000","last_became_active":"2024-09-19T07:53:44.455164+0000","last_became_peered":"2024-09-19T07:53:44.455164+0000","last_unstale":"2024-09-19T07:53:44.456674+0000","last_undegraded":"2024-09-19T07:53:44.456674+0000","last_fullsized":"2024-09-19T07:53:44.456674+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:40.200573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.476417+0000","last_change":"2024-09-19T07:53:44.473808+0000","last_active":"2024-09-19T07:53:44.476417+0000","last_peered":"2024-09-19T07:53:44.476417+0000","last_clean":"2024-09-19T07:53:44.476417+0000","last_became_active":"2024-09-19T07:53:44.473528+0000","last_became_peered":"2024-09-19T07:53:44.473528+0000","last_unstale":"2024-09-19T07:53:44.476417+0000","last_undegraded":"2024-09-19T07:53:44.476417+0000","last_fullsized":"2024-09-19T07:53:44.476417+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:19.095190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.450130+0000","last_change":"2024-09-19T07:53:44.458967+0000","last_active":"2024-09-19T07:53:45.450130+0000","last_peered":"2024-09-19T07:53:45.450130+0000","last_clean":"2024-09-19T07:53:45.450130+0000","last_became_active":"2024-09-19T07:53:44.458687+0000","last_became_peered":"2024-09-19T07:53:44.458687+0000","last_unstale":"2024-09-19T07:53:45.450130+0000","last_undegraded":"2024-09-19T07:53:45.450130+0000","last_fullsized":"2024-09-19T07:53:45.450130+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:43.497533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955733+0000","last_change":"2024-09-19T07:53:44.474993+0000","last_active":"2024-09-19T07:53:45.955733+0000","last_peered":"2024-09-19T07:53:45.955733+0000","last_clean":"2024-09-19T07:53:45.955733+0000","last_became_active":"2024-09-19T07:53:44.474454+0000","last_became_peered":"2024-09-19T07:53:44.474454+0000","last_unstale":"2024-09-19T07:53:45.955733+0000","last_undegraded":"2024-09-19T07:53:45.955733+0000","last_fullsized":"2024-09-19T07:53:45.955733+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:28.268952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.471284+0000","last_change":"2024-09-19T07:53:44.465304+0000","last_active":"2024-09-19T07:53:44.471284+0000","last_peered":"2024-09-19T07:53:44.471284+0000","last_clean":"2024-09-19T07:53:44.471284+0000","last_became_active":"2024-09-19T07:53:44.465044+0000","last_became_peered":"2024-09-19T07:53:44.465044+0000","last_unstale":"2024-09-19T07:53:44.471284+0000","last_undegraded":"2024-09-19T07:53:44.471284+0000","last_fullsized":"2024-09-19T07:53:44.471284+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:40.200573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955372+0000","last_change":"2024-09-19T07:53:44.466484+0000","last_active":"2024-09-19T07:53:45.955372+0000","last_peered":"2024-09-19T07:53:45.955372+0000","last_clean":"2024-09-19T07:53:45.955372+0000","last_became_active":"2024-09-19T07:53:44.466195+0000","last_became_peered":"2024-09-19T07:53:44.466195+0000","last_unstale":"2024-09-19T07:53:45.955372+0000","last_undegraded":"2024-09-19T07:53:45.955372+0000","last_fullsized":"2024-09-19T07:53:45.955372+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:20.436455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.473757+0000","last_change":"2024-09-19T07:53:44.472944+0000","last_active":"2024-09-19T07:53:44.473757+0000","last_peered":"2024-09-19T07:53:44.473757+0000","last_clean":"2024-09-19T07:53:44.473757+0000","last_became_active":"2024-09-19T07:53:44.472669+0000","last_became_peered":"2024-09-19T07:53:44.472669+0000","last_unstale":"2024-09-19T07:53:44.473757+0000","last_undegraded":"2024-09-19T07:53:44.473757+0000","last_fullsized":"2024-09-19T07:53:44.473757+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:05.983399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470250+0000","last_change":"2024-09-19T07:53:44.466834+0000","last_active":"2024-09-19T07:53:44.470250+0000","last_peered":"2024-09-19T07:53:44.470250+0000","last_clean":"2024-09-19T07:53:44.470250+0000","last_became_active":"2024-09-19T07:53:44.466204+0000","last_became_peered":"2024-09-19T07:53:44.466204+0000","last_unstale":"2024-09-19T07:53:44.470250+0000","last_undegraded":"2024-09-19T07:53:44.470250+0000","last_fullsized":"2024-09-19T07:53:44.470250+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:00.929562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.481470+0000","last_change":"2024-09-19T07:53:44.480566+0000","last_active":"2024-09-19T07:53:44.481470+0000","last_peered":"2024-09-19T07:53:44.481470+0000","last_clean":"2024-09-19T07:53:44.481470+0000","last_became_active":"2024-09-19T07:53:44.480280+0000","last_became_peered":"2024-09-19T07:53:44.480280+0000","last_unstale":"2024-09-19T07:53:44.481470+0000","last_undegraded":"2024-09-19T07:53:44.481470+0000","last_fullsized":"2024-09-19T07:53:44.481470+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:46.275247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.464980+0000","last_change":"2024-09-19T07:53:44.462832+0000","last_active":"2024-09-19T07:53:44.464980+0000","last_peered":"2024-09-19T07:53:44.464980+0000","last_clean":"2024-09-19T07:53:44.464980+0000","last_became_active":"2024-09-19T07:53:44.462538+0000","last_became_peered":"2024-09-19T07:53:44.462538+0000","last_unstale":"2024-09-19T07:53:44.464980+0000","last_undegraded":"2024-09-19T07:53:44.464980+0000","last_fullsized":"2024-09-19T07:53:44.464980+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:45.343582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.465738+0000","last_change":"2024-09-19T07:53:44.460249+0000","last_active":"2024-09-19T07:53:44.465738+0000","last_peered":"2024-09-19T07:53:44.465738+0000","last_clean":"2024-09-19T07:53:44.465738+0000","last_became_active":"2024-09-19T07:53:44.458793+0000","last_became_peered":"2024-09-19T07:53:44.458793+0000","last_unstale":"2024-09-19T07:53:44.465738+0000","last_undegraded":"2024-09-19T07:53:44.465738+0000","last_fullsized":"2024-09-19T07:53:44.465738+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:37.126896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955551+0000","last_change":"2024-09-19T07:53:44.467665+0000","last_active":"2024-09-19T07:53:45.955551+0000","last_peered":"2024-09-19T07:53:45.955551+0000","last_clean":"2024-09-19T07:53:45.955551+0000","last_became_active":"2024-09-19T07:53:44.467388+0000","last_became_peered":"2024-09-19T07:53:44.467388+0000","last_unstale":"2024-09-19T07:53:45.955551+0000","last_undegraded":"2024-09-19T07:53:45.955551+0000","last_fullsized":"2024-09-19T07:53:45.955551+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:09.122218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.464494+0000","last_change":"2024-09-19T07:53:44.463945+0000","last_active":"2024-09-19T07:53:44.464494+0000","last_peered":"2024-09-19T07:53:44.464494+0000","last_clean":"2024-09-19T07:53:44.464494+0000","last_became_active":"2024-09-19T07:53:44.463622+0000","last_became_peered":"2024-09-19T07:53:44.463622+0000","last_unstale":"2024-09-19T07:53:44.464494+0000","last_undegraded":"2024-09-19T07:53:44.464494+0000","last_fullsized":"2024-09-19T07:53:44.464494+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:44.970456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.480727+0000","last_change":"2024-09-19T07:53:44.479121+0000","last_active":"2024-09-19T07:53:44.480727+0000","last_peered":"2024-09-19T07:53:44.480727+0000","last_clean":"2024-09-19T07:53:44.480727+0000","last_became_active":"2024-09-19T07:53:44.478632+0000","last_became_peered":"2024-09-19T07:53:44.478632+0000","last_unstale":"2024-09-19T07:53:44.480727+0000","last_undegraded":"2024-09-19T07:53:44.480727+0000","last_fullsized":"2024-09-19T07:53:44.480727+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:58.195995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955651+0000","last_change":"2024-09-19T07:53:44.471531+0000","last_active":"2024-09-19T07:53:45.955651+0000","last_peered":"2024-09-19T07:53:45.955651+0000","last_clean":"2024-09-19T07:53:45.955651+0000","last_became_active":"2024-09-19T07:53:44.471247+0000","last_became_peered":"2024-09-19T07:53:44.471247+0000","last_unstale":"2024-09-19T07:53:45.955651+0000","last_undegraded":"2024-09-19T07:53:45.955651+0000","last_fullsized":"2024-09-19T07:53:45.955651+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:27:11.047914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955832+0000","last_change":"2024-09-19T07:53:44.472849+0000","last_active":"2024-09-19T07:53:45.955832+0000","last_peered":"2024-09-19T07:53:45.955832+0000","last_clean":"2024-09-19T07:53:45.955832+0000","last_became_active":"2024-09-19T07:53:44.472565+0000","last_became_peered":"2024-09-19T07:53:44.472565+0000","last_unstale":"2024-09-19T07:53:45.955832+0000","last_undegraded":"2024-09-19T07:53:45.955832+0000","last_fullsized":"2024-09-19T07:53:45.955832+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:35.230052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.464284+0000","last_change":"2024-09-19T07:53:44.461893+0000","last_active":"2024-09-19T07:53:44.464284+0000","last_peered":"2024-09-19T07:53:44.464284+0000","last_clean":"2024-09-19T07:53:44.464284+0000","last_became_active":"2024-09-19T07:53:44.461543+0000","last_became_peered":"2024-09-19T07:53:44.461543+0000","last_unstale":"2024-09-19T07:53:44.464284+0000","last_undegraded":"2024-09-19T07:53:44.464284+0000","last_fullsized":"2024-09-19T07:53:44.464284+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:02.071494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.464955+0000","last_change":"2024-09-19T07:53:44.462124+0000","last_active":"2024-09-19T07:53:44.464955+0000","last_peered":"2024-09-19T07:53:44.464955+0000","last_clean":"2024-09-19T07:53:44.464955+0000","last_became_active":"2024-09-19T07:53:44.461823+0000","last_became_peered":"2024-09-19T07:53:44.461823+0000","last_unstale":"2024-09-19T07:53:44.464955+0000","last_undegraded":"2024-09-19T07:53:44.464955+0000","last_fullsized":"2024-09-19T07:53:44.464955+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:19.276802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.450047+0000","last_change":"2024-09-19T07:53:44.470121+0000","last_active":"2024-09-19T07:53:45.450047+0000","last_peered":"2024-09-19T07:53:45.450047+0000","last_clean":"2024-09-19T07:53:45.450047+0000","last_became_active":"2024-09-19T07:53:44.469821+0000","last_became_peered":"2024-09-19T07:53:44.469821+0000","last_unstale":"2024-09-19T07:53:45.450047+0000","last_undegraded":"2024-09-19T07:53:45.450047+0000","last_fullsized":"2024-09-19T07:53:45.450047+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:05.734979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"117'4","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475117+0000","last_change":"2024-09-19T07:53:43.462786+0000","last_active":"2024-09-19T07:53:44.475117+0000","last_peered":"2024-09-19T07:53:44.475117+0000","last_clean":"2024-09-19T07:53:44.475117+0000","last_became_active":"2024-09-19T07:53:43.462517+0000","last_became_peered":"2024-09-19T07:53:43.462517+0000","last_unstale":"2024-09-19T07:53:44.475117+0000","last_undegraded":"2024-09-19T07:53:44.475117+0000","last_fullsized":"2024-09-19T07:53:44.475117+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:55.555956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.483675+0000","last_change":"2024-09-19T07:53:44.482782+0000","last_active":"2024-09-19T07:53:44.483675+0000","last_peered":"2024-09-19T07:53:44.483675+0000","last_clean":"2024-09-19T07:53:44.483675+0000","last_became_active":"2024-09-19T07:53:44.482497+0000","last_became_peered":"2024-09-19T07:53:44.482497+0000","last_unstale":"2024-09-19T07:53:44.483675+0000","last_undegraded":"2024-09-19T07:53:44.483675+0000","last_fullsized":"2024-09-19T07:53:44.483675+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:03.706766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.450173+0000","last_change":"2024-09-19T07:53:43.457859+0000","last_active":"2024-09-19T07:53:45.450173+0000","last_peered":"2024-09-19T07:53:45.450173+0000","last_clean":"2024-09-19T07:53:45.450173+0000","last_became_active":"2024-09-19T07:53:43.457574+0000","last_became_peered":"2024-09-19T07:53:43.457574+0000","last_unstale":"2024-09-19T07:53:45.450173+0000","last_undegraded":"2024-09-19T07:53:45.450173+0000","last_fullsized":"2024-09-19T07:53:45.450173+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:37.240608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.469707+0000","last_change":"2024-09-19T07:53:44.468508+0000","last_active":"2024-09-19T07:53:44.469707+0000","last_peered":"2024-09-19T07:53:44.469707+0000","last_clean":"2024-09-19T07:53:44.469707+0000","last_became_active":"2024-09-19T07:53:44.468228+0000","last_became_peered":"2024-09-19T07:53:44.468228+0000","last_unstale":"2024-09-19T07:53:44.469707+0000","last_undegraded":"2024-09-19T07:53:44.469707+0000","last_fullsized":"2024-09-19T07:53:44.469707+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.258505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.480926+0000","last_change":"2024-09-19T07:53:43.464281+0000","last_active":"2024-09-19T07:53:44.480926+0000","last_peered":"2024-09-19T07:53:44.480926+0000","last_clean":"2024-09-19T07:53:44.480926+0000","last_became_active":"2024-09-19T07:53:43.464005+0000","last_became_peered":"2024-09-19T07:53:43.464005+0000","last_unstale":"2024-09-19T07:53:44.480926+0000","last_undegraded":"2024-09-19T07:53:44.480926+0000","last_fullsized":"2024-09-19T07:53:44.480926+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:40.739604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470859+0000","last_change":"2024-09-19T07:53:44.463658+0000","last_active":"2024-09-19T07:53:44.470859+0000","last_peered":"2024-09-19T07:53:44.470859+0000","last_clean":"2024-09-19T07:53:44.470859+0000","last_became_active":"2024-09-19T07:53:44.463373+0000","last_became_peered":"2024-09-19T07:53:44.463373+0000","last_unstale":"2024-09-19T07:53:44.470859+0000","last_undegraded":"2024-09-19T07:53:44.470859+0000","last_fullsized":"2024-09-19T07:53:44.470859+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:05.734979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.449990+0000","last_change":"2024-09-19T07:53:43.459863+0000","last_active":"2024-09-19T07:53:45.449990+0000","last_peered":"2024-09-19T07:53:45.449990+0000","last_clean":"2024-09-19T07:53:45.449990+0000","last_became_active":"2024-09-19T07:53:43.459538+0000","last_became_peered":"2024-09-19T07:53:43.459538+0000","last_unstale":"2024-09-19T07:53:45.449990+0000","last_undegraded":"2024-09-19T07:53:45.449990+0000","last_fullsized":"2024-09-19T07:53:45.449990+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:47.768851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.449836+0000","last_change":"2024-09-19T07:53:44.460976+0000","last_active":"2024-09-19T07:53:45.449836+0000","last_peered":"2024-09-19T07:53:45.449836+0000","last_clean":"2024-09-19T07:53:45.449836+0000","last_became_active":"2024-09-19T07:53:44.460685+0000","last_became_peered":"2024-09-19T07:53:44.460685+0000","last_unstale":"2024-09-19T07:53:45.449836+0000","last_undegraded":"2024-09-19T07:53:45.449836+0000","last_fullsized":"2024-09-19T07:53:45.449836+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:19.276802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"117'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.467472+0000","last_change":"2024-09-19T07:53:43.459625+0000","last_active":"2024-09-19T07:53:44.467472+0000","last_peered":"2024-09-19T07:53:44.467472+0000","last_clean":"2024-09-19T07:53:44.467472+0000","last_became_active":"2024-09-19T07:53:43.459283+0000","last_became_peered":"2024-09-19T07:53:43.459283+0000","last_unstale":"2024-09-19T07:53:44.467472+0000","last_undegraded":"2024-09-19T07:53:44.467472+0000","last_fullsized":"2024-09-19T07:53:44.467472+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:37.240608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.463008+0000","last_change":"2024-09-19T07:53:44.461446+0000","last_active":"2024-09-19T07:53:44.463008+0000","last_peered":"2024-09-19T07:53:44.463008+0000","last_clean":"2024-09-19T07:53:44.463008+0000","last_became_active":"2024-09-19T07:53:44.461158+0000","last_became_peered":"2024-09-19T07:53:44.461158+0000","last_unstale":"2024-09-19T07:53:44.463008+0000","last_undegraded":"2024-09-19T07:53:44.463008+0000","last_fullsized":"2024-09-19T07:53:44.463008+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:13.056229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"117'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.485413+0000","last_change":"2024-09-19T07:53:43.467953+0000","last_active":"2024-09-19T07:53:44.485413+0000","last_peered":"2024-09-19T07:53:44.485413+0000","last_clean":"2024-09-19T07:53:44.485413+0000","last_became_active":"2024-09-19T07:53:43.467684+0000","last_became_peered":"2024-09-19T07:53:43.467684+0000","last_unstale":"2024-09-19T07:53:44.485413+0000","last_undegraded":"2024-09-19T07:53:44.485413+0000","last_fullsized":"2024-09-19T07:53:44.485413+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:46.197768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955975+0000","last_change":"2024-09-19T07:53:44.471846+0000","last_active":"2024-09-19T07:53:45.955975+0000","last_peered":"2024-09-19T07:53:45.955975+0000","last_clean":"2024-09-19T07:53:45.955975+0000","last_became_active":"2024-09-19T07:53:44.471573+0000","last_became_peered":"2024-09-19T07:53:44.471573+0000","last_unstale":"2024-09-19T07:53:45.955975+0000","last_undegraded":"2024-09-19T07:53:45.955975+0000","last_fullsized":"2024-09-19T07:53:45.955975+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:50.563314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.453345+0000","last_change":"2024-09-19T07:53:43.458791+0000","last_active":"2024-09-19T07:53:44.453345+0000","last_peered":"2024-09-19T07:53:44.453345+0000","last_clean":"2024-09-19T07:53:44.453345+0000","last_became_active":"2024-09-19T07:53:43.458509+0000","last_became_peered":"2024-09-19T07:53:43.458509+0000","last_unstale":"2024-09-19T07:53:44.453345+0000","last_undegraded":"2024-09-19T07:53:44.453345+0000","last_fullsized":"2024-09-19T07:53:44.453345+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:54.555152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.450043+0000","last_change":"2024-09-19T07:53:44.468647+0000","last_active":"2024-09-19T07:53:45.450043+0000","last_peered":"2024-09-19T07:53:45.450043+0000","last_clean":"2024-09-19T07:53:45.450043+0000","last_became_active":"2024-09-19T07:53:44.468375+0000","last_became_peered":"2024-09-19T07:53:44.468375+0000","last_unstale":"2024-09-19T07:53:45.450043+0000","last_undegraded":"2024-09-19T07:53:45.450043+0000","last_fullsized":"2024-09-19T07:53:45.450043+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:49.561080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.460241+0000","last_change":"2024-09-19T07:53:43.466824+0000","last_active":"2024-09-19T07:53:44.460241+0000","last_peered":"2024-09-19T07:53:44.460241+0000","last_clean":"2024-09-19T07:53:44.460241+0000","last_became_active":"2024-09-19T07:53:43.466546+0000","last_became_peered":"2024-09-19T07:53:43.466546+0000","last_unstale":"2024-09-19T07:53:44.460241+0000","last_undegraded":"2024-09-19T07:53:44.460241+0000","last_fullsized":"2024-09-19T07:53:44.460241+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:52.318252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470048+0000","last_change":"2024-09-19T07:53:44.464599+0000","last_active":"2024-09-19T07:53:44.470048+0000","last_peered":"2024-09-19T07:53:44.470048+0000","last_clean":"2024-09-19T07:53:44.470048+0000","last_became_active":"2024-09-19T07:53:44.464335+0000","last_became_peered":"2024-09-19T07:53:44.464335+0000","last_unstale":"2024-09-19T07:53:44.470048+0000","last_undegraded":"2024-09-19T07:53:44.470048+0000","last_fullsized":"2024-09-19T07:53:44.470048+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:44.305938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.456285+0000","last_change":"2024-09-19T07:53:43.468918+0000","last_active":"2024-09-19T07:53:44.456285+0000","last_peered":"2024-09-19T07:53:44.456285+0000","last_clean":"2024-09-19T07:53:44.456285+0000","last_became_active":"2024-09-19T07:53:43.468594+0000","last_became_peered":"2024-09-19T07:53:43.468594+0000","last_unstale":"2024-09-19T07:53:44.456285+0000","last_undegraded":"2024-09-19T07:53:44.456285+0000","last_fullsized":"2024-09-19T07:53:44.456285+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:47.768851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.469144+0000","last_change":"2024-09-19T07:53:44.466217+0000","last_active":"2024-09-19T07:53:44.469144+0000","last_peered":"2024-09-19T07:53:44.469144+0000","last_clean":"2024-09-19T07:53:44.469144+0000","last_became_active":"2024-09-19T07:53:44.465405+0000","last_became_peered":"2024-09-19T07:53:44.465405+0000","last_unstale":"2024-09-19T07:53:44.469144+0000","last_undegraded":"2024-09-19T07:53:44.469144+0000","last_fullsized":"2024-09-19T07:53:44.469144+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:41.741615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.454379+0000","last_change":"2024-09-19T07:53:43.463233+0000","last_active":"2024-09-19T07:53:44.454379+0000","last_peered":"2024-09-19T07:53:44.454379+0000","last_clean":"2024-09-19T07:53:44.454379+0000","last_became_active":"2024-09-19T07:53:43.462620+0000","last_became_peered":"2024-09-19T07:53:43.462620+0000","last_unstale":"2024-09-19T07:53:44.454379+0000","last_undegraded":"2024-09-19T07:53:44.454379+0000","last_fullsized":"2024-09-19T07:53:44.454379+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:57:59.927552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955871+0000","last_change":"2024-09-19T07:53:44.481007+0000","last_active":"2024-09-19T07:53:45.955871+0000","last_peered":"2024-09-19T07:53:45.955871+0000","last_clean":"2024-09-19T07:53:45.955871+0000","last_became_active":"2024-09-19T07:53:44.479548+0000","last_became_peered":"2024-09-19T07:53:44.479548+0000","last_unstale":"2024-09-19T07:53:45.955871+0000","last_undegraded":"2024-09-19T07:53:45.955871+0000","last_fullsized":"2024-09-19T07:53:45.955871+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:53.320262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"117'3","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475321+0000","last_change":"2024-09-19T07:53:43.463469+0000","last_active":"2024-09-19T07:53:44.475321+0000","last_peered":"2024-09-19T07:53:44.475321+0000","last_clean":"2024-09-19T07:53:44.475321+0000","last_became_active":"2024-09-19T07:53:43.463042+0000","last_became_peered":"2024-09-19T07:53:43.463042+0000","last_unstale":"2024-09-19T07:53:44.475321+0000","last_undegraded":"2024-09-19T07:53:44.475321+0000","last_fullsized":"2024-09-19T07:53:44.475321+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:32.163267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.469249+0000","last_change":"2024-09-19T07:53:44.466514+0000","last_active":"2024-09-19T07:53:44.469249+0000","last_peered":"2024-09-19T07:53:44.469249+0000","last_clean":"2024-09-19T07:53:44.469249+0000","last_became_active":"2024-09-19T07:53:44.464961+0000","last_became_peered":"2024-09-19T07:53:44.464961+0000","last_unstale":"2024-09-19T07:53:44.469249+0000","last_undegraded":"2024-09-19T07:53:44.469249+0000","last_fullsized":"2024-09-19T07:53:44.469249+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:49.561080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"117'6","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956390+0000","last_change":"2024-09-19T07:53:43.460277+0000","last_active":"2024-09-19T07:53:45.956390+0000","last_peered":"2024-09-19T07:53:45.956390+0000","last_clean":"2024-09-19T07:53:45.956390+0000","last_became_active":"2024-09-19T07:53:43.460028+0000","last_became_peered":"2024-09-19T07:53:43.460028+0000","last_unstale":"2024-09-19T07:53:45.956390+0000","last_undegraded":"2024-09-19T07:53:45.956390+0000","last_fullsized":"2024-09-19T07:53:45.956390+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:29.545776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475155+0000","last_change":"2024-09-19T07:53:44.474267+0000","last_active":"2024-09-19T07:53:44.475155+0000","last_peered":"2024-09-19T07:53:44.475155+0000","last_clean":"2024-09-19T07:53:44.475155+0000","last_became_active":"2024-09-19T07:53:44.473919+0000","last_became_peered":"2024-09-19T07:53:44.473919+0000","last_unstale":"2024-09-19T07:53:44.475155+0000","last_undegraded":"2024-09-19T07:53:44.475155+0000","last_fullsized":"2024-09-19T07:53:44.475155+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:25.476480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956214+0000","last_change":"2024-09-19T07:53:43.463274+0000","last_active":"2024-09-19T07:53:45.956214+0000","last_peered":"2024-09-19T07:53:45.956214+0000","last_clean":"2024-09-19T07:53:45.956214+0000","last_became_active":"2024-09-19T07:53:43.463007+0000","last_became_peered":"2024-09-19T07:53:43.463007+0000","last_unstale":"2024-09-19T07:53:45.956214+0000","last_undegraded":"2024-09-19T07:53:45.956214+0000","last_fullsized":"2024-09-19T07:53:45.956214+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:06.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.449876+0000","last_change":"2024-09-19T07:53:44.463985+0000","last_active":"2024-09-19T07:53:45.449876+0000","last_peered":"2024-09-19T07:53:45.449876+0000","last_clean":"2024-09-19T07:53:45.449876+0000","last_became_active":"2024-09-19T07:53:44.463678+0000","last_became_peered":"2024-09-19T07:53:44.463678+0000","last_unstale":"2024-09-19T07:53:45.449876+0000","last_undegraded":"2024-09-19T07:53:45.449876+0000","last_fullsized":"2024-09-19T07:53:45.449876+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:45.343582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956260+0000","last_change":"2024-09-19T07:53:43.455019+0000","last_active":"2024-09-19T07:53:45.956260+0000","last_peered":"2024-09-19T07:53:45.956260+0000","last_clean":"2024-09-19T07:53:45.956260+0000","last_became_active":"2024-09-19T07:53:43.454686+0000","last_became_peered":"2024-09-19T07:53:43.454686+0000","last_unstale":"2024-09-19T07:53:45.956260+0000","last_undegraded":"2024-09-19T07:53:45.956260+0000","last_fullsized":"2024-09-19T07:53:45.956260+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:20.992974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956646+0000","last_change":"2024-09-19T07:53:44.457728+0000","last_active":"2024-09-19T07:53:45.956646+0000","last_peered":"2024-09-19T07:53:45.956646+0000","last_clean":"2024-09-19T07:53:45.956646+0000","last_became_active":"2024-09-19T07:53:44.457456+0000","last_became_peered":"2024-09-19T07:53:44.457456+0000","last_unstale":"2024-09-19T07:53:45.956646+0000","last_undegraded":"2024-09-19T07:53:45.956646+0000","last_fullsized":"2024-09-19T07:53:45.956646+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:37:56.076282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956614+0000","last_change":"2024-09-19T07:53:43.466167+0000","last_active":"2024-09-19T07:53:45.956614+0000","last_peered":"2024-09-19T07:53:45.956614+0000","last_clean":"2024-09-19T07:53:45.956614+0000","last_became_active":"2024-09-19T07:53:43.465882+0000","last_became_peered":"2024-09-19T07:53:43.465882+0000","last_unstale":"2024-09-19T07:53:45.956614+0000","last_undegraded":"2024-09-19T07:53:45.956614+0000","last_fullsized":"2024-09-19T07:53:45.956614+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:07.875623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470860+0000","last_change":"2024-09-19T07:53:44.469161+0000","last_active":"2024-09-19T07:53:44.470860+0000","last_peered":"2024-09-19T07:53:44.470860+0000","last_clean":"2024-09-19T07:53:44.470860+0000","last_became_active":"2024-09-19T07:53:44.468896+0000","last_became_peered":"2024-09-19T07:53:44.468896+0000","last_unstale":"2024-09-19T07:53:44.470860+0000","last_undegraded":"2024-09-19T07:53:44.470860+0000","last_fullsized":"2024-09-19T07:53:44.470860+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:43.497533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.450013+0000","last_change":"2024-09-19T07:53:43.468990+0000","last_active":"2024-09-19T07:53:45.450013+0000","last_peered":"2024-09-19T07:53:45.450013+0000","last_clean":"2024-09-19T07:53:45.450013+0000","last_became_active":"2024-09-19T07:53:43.468625+0000","last_became_peered":"2024-09-19T07:53:43.468625+0000","last_unstale":"2024-09-19T07:53:45.450013+0000","last_undegraded":"2024-09-19T07:53:45.450013+0000","last_fullsized":"2024-09-19T07:53:45.450013+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:43.968446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.470204+0000","last_change":"2024-09-19T07:53:44.466474+0000","last_active":"2024-09-19T07:53:44.470204+0000","last_peered":"2024-09-19T07:53:44.470204+0000","last_clean":"2024-09-19T07:53:44.470204+0000","last_became_active":"2024-09-19T07:53:44.466230+0000","last_became_peered":"2024-09-19T07:53:44.466230+0000","last_unstale":"2024-09-19T07:53:44.470204+0000","last_undegraded":"2024-09-19T07:53:44.470204+0000","last_fullsized":"2024-09-19T07:53:44.470204+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:55.557163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955645+0000","last_change":"2024-09-19T07:53:43.463386+0000","last_active":"2024-09-19T07:53:45.955645+0000","last_peered":"2024-09-19T07:53:45.955645+0000","last_clean":"2024-09-19T07:53:45.955645+0000","last_became_active":"2024-09-19T07:53:43.463020+0000","last_became_peered":"2024-09-19T07:53:43.463020+0000","last_unstale":"2024-09-19T07:53:45.955645+0000","last_undegraded":"2024-09-19T07:53:45.955645+0000","last_fullsized":"2024-09-19T07:53:45.955645+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:01.309102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.482421+0000","last_change":"2024-09-19T07:53:44.480939+0000","last_active":"2024-09-19T07:53:44.482421+0000","last_peered":"2024-09-19T07:53:44.482421+0000","last_clean":"2024-09-19T07:53:44.482421+0000","last_became_active":"2024-09-19T07:53:44.480634+0000","last_became_peered":"2024-09-19T07:53:44.480634+0000","last_unstale":"2024-09-19T07:53:44.482421+0000","last_undegraded":"2024-09-19T07:53:44.482421+0000","last_fullsized":"2024-09-19T07:53:44.482421+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:58.089951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"117'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475895+0000","last_change":"2024-09-19T07:53:43.464937+0000","last_active":"2024-09-19T07:53:44.475895+0000","last_peered":"2024-09-19T07:53:44.475895+0000","last_clean":"2024-09-19T07:53:44.475895+0000","last_became_active":"2024-09-19T07:53:43.464614+0000","last_became_peered":"2024-09-19T07:53:43.464614+0000","last_unstale":"2024-09-19T07:53:44.475895+0000","last_undegraded":"2024-09-19T07:53:44.475895+0000","last_fullsized":"2024-09-19T07:53:44.475895+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:48.559070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.469270+0000","last_change":"2024-09-19T07:53:44.466748+0000","last_active":"2024-09-19T07:53:44.469270+0000","last_peered":"2024-09-19T07:53:44.469270+0000","last_clean":"2024-09-19T07:53:44.469270+0000","last_became_active":"2024-09-19T07:53:44.465355+0000","last_became_peered":"2024-09-19T07:53:44.465355+0000","last_unstale":"2024-09-19T07:53:44.469270+0000","last_undegraded":"2024-09-19T07:53:44.469270+0000","last_fullsized":"2024-09-19T07:53:44.469270+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:05.734979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.456785+0000","last_change":"2024-09-19T07:53:43.466472+0000","last_active":"2024-09-19T07:53:44.456785+0000","last_peered":"2024-09-19T07:53:44.456785+0000","last_clean":"2024-09-19T07:53:44.456785+0000","last_became_active":"2024-09-19T07:53:43.466216+0000","last_became_peered":"2024-09-19T07:53:43.466216+0000","last_unstale":"2024-09-19T07:53:44.456785+0000","last_undegraded":"2024-09-19T07:53:44.456785+0000","last_fullsized":"2024-09-19T07:53:44.456785+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:44.341571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.482841+0000","last_change":"2024-09-19T07:53:44.480405+0000","last_active":"2024-09-19T07:53:44.482841+0000","last_peered":"2024-09-19T07:53:44.482841+0000","last_clean":"2024-09-19T07:53:44.482841+0000","last_became_active":"2024-09-19T07:53:44.479696+0000","last_became_peered":"2024-09-19T07:53:44.479696+0000","last_unstale":"2024-09-19T07:53:44.482841+0000","last_undegraded":"2024-09-19T07:53:44.482841+0000","last_fullsized":"2024-09-19T07:53:44.482841+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:23.633013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.476888+0000","last_change":"2024-09-19T07:53:43.463463+0000","last_active":"2024-09-19T07:53:44.476888+0000","last_peered":"2024-09-19T07:53:44.476888+0000","last_clean":"2024-09-19T07:53:44.476888+0000","last_became_active":"2024-09-19T07:53:43.463197+0000","last_became_peered":"2024-09-19T07:53:43.463197+0000","last_unstale":"2024-09-19T07:53:44.476888+0000","last_undegraded":"2024-09-19T07:53:44.476888+0000","last_fullsized":"2024-09-19T07:53:44.476888+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:27.266942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956498+0000","last_change":"2024-09-19T07:53:44.466715+0000","last_active":"2024-09-19T07:53:45.956498+0000","last_peered":"2024-09-19T07:53:45.956498+0000","last_clean":"2024-09-19T07:53:45.956498+0000","last_became_active":"2024-09-19T07:53:44.466187+0000","last_became_peered":"2024-09-19T07:53:44.466187+0000","last_unstale":"2024-09-19T07:53:45.956498+0000","last_undegraded":"2024-09-19T07:53:45.956498+0000","last_fullsized":"2024-09-19T07:53:45.956498+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:35.653193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.456607+0000","last_change":"2024-09-19T07:53:43.465167+0000","last_active":"2024-09-19T07:53:44.456607+0000","last_peered":"2024-09-19T07:53:44.456607+0000","last_clean":"2024-09-19T07:53:44.456607+0000","last_became_active":"2024-09-19T07:53:43.464875+0000","last_became_peered":"2024-09-19T07:53:43.464875+0000","last_unstale":"2024-09-19T07:53:44.456607+0000","last_undegraded":"2024-09-19T07:53:44.456607+0000","last_fullsized":"2024-09-19T07:53:44.456607+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:18.274791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956174+0000","last_change":"2024-09-19T07:53:44.481091+0000","last_active":"2024-09-19T07:53:45.956174+0000","last_peered":"2024-09-19T07:53:45.956174+0000","last_clean":"2024-09-19T07:53:45.956174+0000","last_became_active":"2024-09-19T07:53:44.480524+0000","last_became_peered":"2024-09-19T07:53:44.480524+0000","last_unstale":"2024-09-19T07:53:45.956174+0000","last_undegraded":"2024-09-19T07:53:45.956174+0000","last_fullsized":"2024-09-19T07:53:45.956174+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:00.047940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956719+0000","last_change":"2024-09-19T07:53:43.465392+0000","last_active":"2024-09-19T07:53:45.956719+0000","last_peered":"2024-09-19T07:53:45.956719+0000","last_clean":"2024-09-19T07:53:45.956719+0000","last_became_active":"2024-09-19T07:53:43.465112+0000","last_became_peered":"2024-09-19T07:53:43.465112+0000","last_unstale":"2024-09-19T07:53:45.956719+0000","last_undegraded":"2024-09-19T07:53:45.956719+0000","last_fullsized":"2024-09-19T07:53:45.956719+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:24.215511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.462713+0000","last_change":"2024-09-19T07:53:44.461101+0000","last_active":"2024-09-19T07:53:44.462713+0000","last_peered":"2024-09-19T07:53:44.462713+0000","last_clean":"2024-09-19T07:53:44.462713+0000","last_became_active":"2024-09-19T07:53:44.460795+0000","last_became_peered":"2024-09-19T07:53:44.460795+0000","last_unstale":"2024-09-19T07:53:44.462713+0000","last_undegraded":"2024-09-19T07:53:44.462713+0000","last_fullsized":"2024-09-19T07:53:44.462713+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:59.717556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455598+0000","last_change":"2024-09-19T07:53:43.467250+0000","last_active":"2024-09-19T07:53:44.455598+0000","last_peered":"2024-09-19T07:53:44.455598+0000","last_clean":"2024-09-19T07:53:44.455598+0000","last_became_active":"2024-09-19T07:53:43.467030+0000","last_became_peered":"2024-09-19T07:53:43.467030+0000","last_unstale":"2024-09-19T07:53:44.455598+0000","last_undegraded":"2024-09-19T07:53:44.455598+0000","last_fullsized":"2024-09-19T07:53:44.455598+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:58.715546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.472936+0000","last_change":"2024-09-19T07:53:44.469053+0000","last_active":"2024-09-19T07:53:44.472936+0000","last_peered":"2024-09-19T07:53:44.472936+0000","last_clean":"2024-09-19T07:53:44.472936+0000","last_became_active":"2024-09-19T07:53:44.468779+0000","last_became_peered":"2024-09-19T07:53:44.468779+0000","last_unstale":"2024-09-19T07:53:44.472936+0000","last_undegraded":"2024-09-19T07:53:44.472936+0000","last_fullsized":"2024-09-19T07:53:44.472936+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:35.653193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.475147+0000","last_change":"2024-09-19T07:53:43.464536+0000","last_active":"2024-09-19T07:53:44.475147+0000","last_peered":"2024-09-19T07:53:44.475147+0000","last_clean":"2024-09-19T07:53:44.475147+0000","last_became_active":"2024-09-19T07:53:43.462203+0000","last_became_peered":"2024-09-19T07:53:43.462203+0000","last_unstale":"2024-09-19T07:53:44.475147+0000","last_undegraded":"2024-09-19T07:53:44.475147+0000","last_fullsized":"2024-09-19T07:53:44.475147+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:35.927023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.450207+0000","last_change":"2024-09-19T07:53:44.462128+0000","last_active":"2024-09-19T07:53:45.450207+0000","last_peered":"2024-09-19T07:53:45.450207+0000","last_clean":"2024-09-19T07:53:45.450207+0000","last_became_active":"2024-09-19T07:53:44.461851+0000","last_became_peered":"2024-09-19T07:53:44.461851+0000","last_unstale":"2024-09-19T07:53:45.450207+0000","last_undegraded":"2024-09-19T07:53:45.450207+0000","last_fullsized":"2024-09-19T07:53:45.450207+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:02.487846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455906+0000","last_change":"2024-09-19T07:53:43.468893+0000","last_active":"2024-09-19T07:53:44.455906+0000","last_peered":"2024-09-19T07:53:44.455906+0000","last_clean":"2024-09-19T07:53:44.455906+0000","last_became_active":"2024-09-19T07:53:43.468581+0000","last_became_peered":"2024-09-19T07:53:43.468581+0000","last_unstale":"2024-09-19T07:53:44.455906+0000","last_undegraded":"2024-09-19T07:53:44.455906+0000","last_fullsized":"2024-09-19T07:53:44.455906+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:21.308817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.471585+0000","last_change":"2024-09-19T07:53:44.463751+0000","last_active":"2024-09-19T07:53:44.471585+0000","last_peered":"2024-09-19T07:53:44.471585+0000","last_clean":"2024-09-19T07:53:44.471585+0000","last_became_active":"2024-09-19T07:53:44.463490+0000","last_became_peered":"2024-09-19T07:53:44.463490+0000","last_unstale":"2024-09-19T07:53:44.471585+0000","last_undegraded":"2024-09-19T07:53:44.471585+0000","last_fullsized":"2024-09-19T07:53:44.471585+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.258505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.458476+0000","last_change":"2024-09-19T07:53:43.468830+0000","last_active":"2024-09-19T07:53:44.458476+0000","last_peered":"2024-09-19T07:53:44.458476+0000","last_clean":"2024-09-19T07:53:44.458476+0000","last_became_active":"2024-09-19T07:53:43.468521+0000","last_became_peered":"2024-09-19T07:53:43.468521+0000","last_unstale":"2024-09-19T07:53:44.458476+0000","last_undegraded":"2024-09-19T07:53:44.458476+0000","last_fullsized":"2024-09-19T07:53:44.458476+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:37.240608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956008+0000","last_change":"2024-09-19T07:53:44.472102+0000","last_active":"2024-09-19T07:53:45.956008+0000","last_peered":"2024-09-19T07:53:45.956008+0000","last_clean":"2024-09-19T07:53:45.956008+0000","last_became_active":"2024-09-19T07:53:44.471800+0000","last_became_peered":"2024-09-19T07:53:44.471800+0000","last_unstale":"2024-09-19T07:53:45.956008+0000","last_undegraded":"2024-09-19T07:53:45.956008+0000","last_fullsized":"2024-09-19T07:53:45.956008+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:06.662848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.459993+0000","last_change":"2024-09-19T07:53:43.464402+0000","last_active":"2024-09-19T07:53:44.459993+0000","last_peered":"2024-09-19T07:53:44.459993+0000","last_clean":"2024-09-19T07:53:44.459993+0000","last_became_active":"2024-09-19T07:53:43.464122+0000","last_became_peered":"2024-09-19T07:53:43.464122+0000","last_unstale":"2024-09-19T07:53:44.459993+0000","last_undegraded":"2024-09-19T07:53:44.459993+0000","last_fullsized":"2024-09-19T07:53:44.459993+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:34.228041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.472855+0000","last_change":"2024-09-19T07:53:44.467967+0000","last_active":"2024-09-19T07:53:44.472855+0000","last_peered":"2024-09-19T07:53:44.472855+0000","last_clean":"2024-09-19T07:53:44.472855+0000","last_became_active":"2024-09-19T07:53:44.467699+0000","last_became_peered":"2024-09-19T07:53:44.467699+0000","last_unstale":"2024-09-19T07:53:44.472855+0000","last_undegraded":"2024-09-19T07:53:44.472855+0000","last_fullsized":"2024-09-19T07:53:44.472855+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:53.377199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.451335+0000","last_change":"2024-09-19T07:53:43.454771+0000","last_active":"2024-09-19T07:53:44.451335+0000","last_peered":"2024-09-19T07:53:44.451335+0000","last_clean":"2024-09-19T07:53:44.451335+0000","last_became_active":"2024-09-19T07:53:43.454309+0000","last_became_peered":"2024-09-19T07:53:43.454309+0000","last_unstale":"2024-09-19T07:53:44.451335+0000","last_undegraded":"2024-09-19T07:53:44.451335+0000","last_fullsized":"2024-09-19T07:53:44.451335+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:43.968446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956628+0000","last_change":"2024-09-19T07:53:44.466447+0000","last_active":"2024-09-19T07:53:45.956628+0000","last_peered":"2024-09-19T07:53:45.956628+0000","last_clean":"2024-09-19T07:53:45.956628+0000","last_became_active":"2024-09-19T07:53:44.465888+0000","last_became_peered":"2024-09-19T07:53:44.465888+0000","last_unstale":"2024-09-19T07:53:45.956628+0000","last_undegraded":"2024-09-19T07:53:45.956628+0000","last_fullsized":"2024-09-19T07:53:45.956628+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:19.095190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.451114+0000","last_change":"2024-09-19T07:53:43.459958+0000","last_active":"2024-09-19T07:53:44.451114+0000","last_peered":"2024-09-19T07:53:44.451114+0000","last_clean":"2024-09-19T07:53:44.451114+0000","last_became_active":"2024-09-19T07:53:43.459470+0000","last_became_peered":"2024-09-19T07:53:43.459470+0000","last_unstale":"2024-09-19T07:53:44.451114+0000","last_undegraded":"2024-09-19T07:53:44.451114+0000","last_fullsized":"2024-09-19T07:53:44.451114+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:47.768851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.473383+0000","last_change":"2024-09-19T07:53:44.470487+0000","last_active":"2024-09-19T07:53:44.473383+0000","last_peered":"2024-09-19T07:53:44.473383+0000","last_clean":"2024-09-19T07:53:44.473383+0000","last_became_active":"2024-09-19T07:53:44.470216+0000","last_became_peered":"2024-09-19T07:53:44.470216+0000","last_unstale":"2024-09-19T07:53:44.473383+0000","last_undegraded":"2024-09-19T07:53:44.473383+0000","last_fullsized":"2024-09-19T07:53:44.473383+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:02.487846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.460679+0000","last_change":"2024-09-19T07:53:43.467051+0000","last_active":"2024-09-19T07:53:44.460679+0000","last_peered":"2024-09-19T07:53:44.460679+0000","last_clean":"2024-09-19T07:53:44.460679+0000","last_became_active":"2024-09-19T07:53:43.466667+0000","last_became_peered":"2024-09-19T07:53:43.466667+0000","last_unstale":"2024-09-19T07:53:44.460679+0000","last_undegraded":"2024-09-19T07:53:44.460679+0000","last_fullsized":"2024-09-19T07:53:44.460679+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:49.561304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.450124+0000","last_change":"2024-09-19T07:53:44.464319+0000","last_active":"2024-09-19T07:53:45.450124+0000","last_peered":"2024-09-19T07:53:45.450124+0000","last_clean":"2024-09-19T07:53:45.450124+0000","last_became_active":"2024-09-19T07:53:44.463940+0000","last_became_peered":"2024-09-19T07:53:44.463940+0000","last_unstale":"2024-09-19T07:53:45.450124+0000","last_undegraded":"2024-09-19T07:53:45.450124+0000","last_fullsized":"2024-09-19T07:53:45.450124+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.258505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455682+0000","last_change":"2024-09-19T07:53:43.465164+0000","last_active":"2024-09-19T07:53:44.455682+0000","last_peered":"2024-09-19T07:53:44.455682+0000","last_clean":"2024-09-19T07:53:44.455682+0000","last_became_active":"2024-09-19T07:53:43.464888+0000","last_became_peered":"2024-09-19T07:53:43.464888+0000","last_unstale":"2024-09-19T07:53:44.455682+0000","last_undegraded":"2024-09-19T07:53:44.455682+0000","last_fullsized":"2024-09-19T07:53:44.455682+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:01.069484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.471656+0000","last_change":"2024-09-19T07:53:44.464883+0000","last_active":"2024-09-19T07:53:44.471656+0000","last_peered":"2024-09-19T07:53:44.471656+0000","last_clean":"2024-09-19T07:53:44.471656+0000","last_became_active":"2024-09-19T07:53:44.464617+0000","last_became_peered":"2024-09-19T07:53:44.464617+0000","last_unstale":"2024-09-19T07:53:44.471656+0000","last_undegraded":"2024-09-19T07:53:44.471656+0000","last_fullsized":"2024-09-19T07:53:44.471656+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:28.530475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"117'8","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955380+0000","last_change":"2024-09-19T07:53:43.459564+0000","last_active":"2024-09-19T07:53:45.955380+0000","last_peered":"2024-09-19T07:53:45.955380+0000","last_clean":"2024-09-19T07:53:45.955380+0000","last_became_active":"2024-09-19T07:53:43.459126+0000","last_became_peered":"2024-09-19T07:53:43.459126+0000","last_unstale":"2024-09-19T07:53:45.955380+0000","last_undegraded":"2024-09-19T07:53:45.955380+0000","last_fullsized":"2024-09-19T07:53:45.955380+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:27.523582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":344,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-19T07:53:44.455739+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:53:44.455739+0000","last_peered":"2024-09-19T07:53:44.455739+0000","last_clean":"2024-09-19T07:53:44.455739+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:53:44.455739+0000","last_undegraded":"2024-09-19T07:53:44.455739+0000","last_fullsized":"2024-09-19T07:53:44.455739+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509573,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83096,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288744,"statfs":{"total":96636764160,"available":96551673856,"internally_reserved":0,"allocated":2031616,"data_stored":1254678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509572,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104788,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94267052,"statfs":{"total":96636764160,"available":96529461248,"internally_reserved":0,"allocated":1437696,"data_stored":664310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509573,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72784,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299056,"statfs":{"total":96636764160,"available":96562233344,"internally_reserved":0,"allocated":1437696,"data_stored":664310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509573,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86676,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285164,"statfs":{"total":96636764160,"available":96548007936,"internally_reserved":0,"allocated":1437696,"data_stored":664310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509572,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105308,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266532,"statfs":{"total":96636764160,"available":96528928768,"internally_reserved":0,"allocated":2031616,"data_stored":1254678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542277,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118648,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253192,"statfs":{"total":96636764160,"available":96515268608,"internally_reserved":0,"allocated":1478656,"data_stored":679815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542277,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82604,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94289236,"statfs":{"total":96636764160,"available":96552177664,"internally_reserved":0,"allocated":1466368,"data_stored":678191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574982,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":108904,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262936,"statfs":{"total":96636764160,"available":96525246464,"internally_reserved":0,"allocated":1462272,"data_stored":678169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:53:50.103 INFO:tasks.ceph.mds.a.smithi016.stderr:2024-09-19T07:53:50.103+0000 7f167b99f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T07:53:52.173 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:53:52.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.289+0000 7f22f35b7640 1 Processor -- start 2024-09-19T07:53:52.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.290+0000 7f22f35b7640 1 -- start start 2024-09-19T07:53:52.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.290+0000 7f22f35b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22ec15f0d0 0x7f22ec15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:52.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.290+0000 7f22f35b7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f22ec0d0070 0x7f22ec0d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:52.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.290+0000 7f22f35b7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22ec0cf2b0 con 0x7f22ec0d1940 2024-09-19T07:53:52.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.290+0000 7f22f35b7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f22ec0cf590 con 0x7f22ec0d0070 2024-09-19T07:53:52.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.290+0000 7f22f35b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f22ec0cf870 con 0x7f22ec15f0d0 2024-09-19T07:53:52.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22f1b54640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f22ec0d1940 0x7f22ec0d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35676/0 (socket says 172.21.15.16:35676) 2024-09-19T07:53:52.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22f1b54640 1 -- 172.21.15.16:0/1902836 learned_addr learned my addr 172.21.15.16:0/1902836 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:52.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22f0b52640 1 --2- 172.21.15.16:0/1902836 >> v2:172.21.15.146:3300/0 conn(0x7f22ec0d0070 0x7f22ec0d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:52.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22f1353640 1 --2- 172.21.15.16:0/1902836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22ec15f0d0 0x7f22ec15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:52.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22e3fff640 1 -- 172.21.15.16:0/1902836 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1005893001 0 0) 0x7f22ec0cf2b0 con 0x7f22ec0d1940 2024-09-19T07:53:52.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22e3fff640 1 -- 172.21.15.16:0/1902836 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22d8003520 con 0x7f22ec0d1940 2024-09-19T07:53:52.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22e3fff640 1 -- 172.21.15.16:0/1902836 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2022489609 0 0) 0x7f22d8003520 con 0x7f22ec0d1940 2024-09-19T07:53:52.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22e3fff640 1 -- 172.21.15.16:0/1902836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22ec15f0d0 msgr2=0x7f22ec15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:52.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22e3fff640 1 --2- 172.21.15.16:0/1902836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22ec15f0d0 0x7f22ec15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22e3fff640 1 -- 172.21.15.16:0/1902836 >> v2:172.21.15.146:3300/0 conn(0x7f22ec0d0070 msgr2=0x7f22ec0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22e3fff640 1 --2- 172.21.15.16:0/1902836 >> v2:172.21.15.146:3300/0 conn(0x7f22ec0d0070 0x7f22ec0d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22e3fff640 1 -- 172.21.15.16:0/1902836 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec0cfb50 con 0x7f22ec0d1940 2024-09-19T07:53:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22e3fff640 1 -- 172.21.15.16:0/1902836 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f22e40019f0 con 0x7f22ec0d1940 2024-09-19T07:53:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22f0b52640 1 --2- 172.21.15.16:0/1902836 >> v2:172.21.15.146:3300/0 conn(0x7f22ec0d0070 0x7f22ec0d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:53:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22e3fff640 1 -- 172.21.15.16:0/1902836 >> v1:172.21.15.16:6789/0 conn(0x7f22ec0d1940 legacy=0x7f22ec0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22e3fff640 1 --2- 172.21.15.16:0/1902836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d80039b0 0x7f22d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22e3fff640 1 --2- 172.21.15.16:0/1902836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22d8004800 0x7f22d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22e3fff640 1 --2- 172.21.15.16:0/1902836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22ec15f0d0 0x7f22d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22e3fff640 1 -- 172.21.15.16:0/1902836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec0cf590 con 0x7f22d80039b0 2024-09-19T07:53:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22e3fff640 1 -- 172.21.15.16:0/1902836 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec0cf870 con 0x7f22d8004800 2024-09-19T07:53:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22e3fff640 1 -- 172.21.15.16:0/1902836 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f22d800b9c0 con 0x7f22ec15f0d0 2024-09-19T07:53:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.291+0000 7f22f1353640 1 --2- 172.21.15.16:0/1902836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d80039b0 0x7f22d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.292+0000 7f22f1b54640 1 --2- 172.21.15.16:0/1902836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22ec15f0d0 0x7f22d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.292+0000 7f22f0b52640 1 --2- 172.21.15.16:0/1902836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22d8004800 0x7f22d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.292+0000 7f22f1353640 1 -- 172.21.15.16:0/1902836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22ec15f0d0 msgr2=0x7f22d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.292+0000 7f22f1353640 1 --2- 172.21.15.16:0/1902836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22ec15f0d0 0x7f22d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.292+0000 7f22f1353640 1 -- 172.21.15.16:0/1902836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22d8004800 msgr2=0x7f22d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.292+0000 7f22f1353640 1 --2- 172.21.15.16:0/1902836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22d8004800 0x7f22d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.292+0000 7f22f1353640 1 -- 172.21.15.16:0/1902836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d8003520 con 0x7f22d80039b0 2024-09-19T07:53:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.293+0000 7f22f1353640 1 --2- 172.21.15.16:0/1902836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d80039b0 0x7f22d8003dd0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009af0 tx=0x7f22d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9348d44fd7b4b1bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.293+0000 7f22e3fff640 1 -- 172.21.15.16:0/1902836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e8036070 con 0x7f22d80039b0 2024-09-19T07:53:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.293+0000 7f22e3fff640 1 -- 172.21.15.16:0/1902836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e8031400 con 0x7f22d80039b0 2024-09-19T07:53:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.293+0000 7f22e3fff640 1 -- 172.21.15.16:0/1902836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e8009690 con 0x7f22d80039b0 2024-09-19T07:53:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.293+0000 7f22f35b7640 1 -- 172.21.15.16:0/1902836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d80039b0 msgr2=0x7f22d8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.293+0000 7f22f35b7640 1 --2- 172.21.15.16:0/1902836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d80039b0 0x7f22d8003dd0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009af0 tx=0x7f22d8005830 comp rx=0 tx=0).stop 2024-09-19T07:53:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.293+0000 7f22f1b54640 1 -- 172.21.15.16:0/1902836 reap_dead start 2024-09-19T07:53:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.293+0000 7f22f35b7640 1 -- 172.21.15.16:0/1902836 shutdown_connections 2024-09-19T07:53:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.293+0000 7f22f35b7640 1 -- 172.21.15.16:0/1902836 >> 172.21.15.16:0/1902836 conn(0x7f22ec09d760 msgr2=0x7f22ec08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.293+0000 7f22f35b7640 1 -- 172.21.15.16:0/1902836 shutdown_connections 2024-09-19T07:53:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.293+0000 7f22f35b7640 1 -- 172.21.15.16:0/1902836 wait complete. 2024-09-19T07:53:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.295+0000 7f22f35b7640 1 Processor -- start 2024-09-19T07:53:52.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.295+0000 7f22f35b7640 1 -- start start 2024-09-19T07:53:52.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22f35b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22d80039b0 0x7f22ec0b8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:52.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22f35b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22ec159dc0 0x7f22ec0cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:52.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22f35b7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22ec159870 0x7f22ec1f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:52.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22f35b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec0d49c0 con 0x7f22ec159dc0 2024-09-19T07:53:52.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22f35b7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec0d4840 con 0x7f22ec159870 2024-09-19T07:53:52.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22f35b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f22ec0d4b40 con 0x7f22d80039b0 2024-09-19T07:53:52.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22f0b52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22ec159dc0 0x7f22ec0cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:52.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22f0b52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22ec159dc0 0x7f22ec0cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52832/0 (socket says 172.21.15.16:52832) 2024-09-19T07:53:52.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22f0b52640 1 -- 172.21.15.16:0/2546905140 learned_addr learned my addr 172.21.15.16:0/2546905140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:52.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22f1b54640 1 --2- 172.21.15.16:0/2546905140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22ec159870 0x7f22ec1f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22f1353640 1 --2- 172.21.15.16:0/2546905140 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22d80039b0 0x7f22ec0b8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22f0b52640 1 -- 172.21.15.16:0/2546905140 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22d80039b0 msgr2=0x7f22ec0b8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22f0b52640 1 --2- 172.21.15.16:0/2546905140 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22d80039b0 0x7f22ec0b8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22f0b52640 1 -- 172.21.15.16:0/2546905140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22ec159870 msgr2=0x7f22ec1f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22f0b52640 1 --2- 172.21.15.16:0/2546905140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22ec159870 0x7f22ec1f4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22f0b52640 1 -- 172.21.15.16:0/2546905140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec0cb5e0 con 0x7f22ec159dc0 2024-09-19T07:53:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22f1b54640 1 --2- 172.21.15.16:0/2546905140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22ec159870 0x7f22ec1f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22f0b52640 1 --2- 172.21.15.16:0/2546905140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22ec159dc0 0x7f22ec0cc680 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f22dc011cd0 tx=0x7f22dc00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22e1ffb640 1 -- 172.21.15.16:0/2546905140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00fd20 con 0x7f22ec159dc0 2024-09-19T07:53:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22e1ffb640 1 -- 172.21.15.16:0/2546905140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22dc014440 con 0x7f22ec159dc0 2024-09-19T07:53:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22f35b7640 1 -- 172.21.15.16:0/2546905140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ec0cb8c0 con 0x7f22ec159dc0 2024-09-19T07:53:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.296+0000 7f22e1ffb640 1 -- 172.21.15.16:0/2546905140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0147f0 con 0x7f22ec159dc0 2024-09-19T07:53:52.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.297+0000 7f22f35b7640 1 -- 172.21.15.16:0/2546905140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ec0cbe80 con 0x7f22ec159dc0 2024-09-19T07:53:52.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.298+0000 7f22f35b7640 1 -- 172.21.15.16:0/2546905140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22ec0d1400 con 0x7f22ec159dc0 2024-09-19T07:53:52.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.298+0000 7f22e1ffb640 1 -- 172.21.15.16:0/2546905140 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f22dc012040 con 0x7f22ec159dc0 2024-09-19T07:53:52.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.298+0000 7f22e1ffb640 1 --2- 172.21.15.16:0/2546905140 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22d006c2e0 0x7f22d006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:52.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.299+0000 7f22e1ffb640 1 -- 172.21.15.16:0/2546905140 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f22dc08bc50 con 0x7f22ec159dc0 2024-09-19T07:53:52.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.299+0000 7f22f1353640 1 --2- 172.21.15.16:0/2546905140 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22d006c2e0 0x7f22d006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:52.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.299+0000 7f22f1353640 1 --2- 172.21.15.16:0/2546905140 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22d006c2e0 0x7f22d006e7a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f22e8033000 tx=0x7f22e8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:52.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.303+0000 7f22e1ffb640 1 -- 172.21.15.16:0/2546905140 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f22dc091020 con 0x7f22ec159dc0 2024-09-19T07:53:52.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.477+0000 7f22f35b7640 1 -- 172.21.15.16:0/2546905140 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f22ec15dc40 con 0x7f22d006c2e0 2024-09-19T07:53:52.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.484+0000 7f22e1ffb640 1 -- 172.21.15.16:0/2546905140 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252464 (secure 0 0 0) 0x7f22ec15dc40 con 0x7f22d006c2e0 2024-09-19T07:53:52.489 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:52.491 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:53:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.487+0000 7f22f35b7640 1 -- 172.21.15.16:0/2546905140 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22d006c2e0 msgr2=0x7f22d006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.487+0000 7f22f35b7640 1 --2- 172.21.15.16:0/2546905140 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22d006c2e0 0x7f22d006e7a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f22e8033000 tx=0x7f22e8008030 comp rx=0 tx=0).stop 2024-09-19T07:53:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.487+0000 7f22f35b7640 1 -- 172.21.15.16:0/2546905140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22ec159dc0 msgr2=0x7f22ec0cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.487+0000 7f22f35b7640 1 --2- 172.21.15.16:0/2546905140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22ec159dc0 0x7f22ec0cc680 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f22dc011cd0 tx=0x7f22dc00f610 comp rx=0 tx=0).stop 2024-09-19T07:53:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.487+0000 7f22f35b7640 1 -- 172.21.15.16:0/2546905140 shutdown_connections 2024-09-19T07:53:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.487+0000 7f22f35b7640 1 --2- 172.21.15.16:0/2546905140 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22d006c2e0 0x7f22d006e7a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.487+0000 7f22f35b7640 1 --2- 172.21.15.16:0/2546905140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22ec159870 0x7f22ec1f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.487+0000 7f22f35b7640 1 --2- 172.21.15.16:0/2546905140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22ec159dc0 0x7f22ec0cc680 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.487+0000 7f22f35b7640 1 --2- 172.21.15.16:0/2546905140 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22d80039b0 0x7f22ec0b8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.487+0000 7f22f35b7640 1 -- 172.21.15.16:0/2546905140 >> 172.21.15.16:0/2546905140 conn(0x7f22ec09d760 msgr2=0x7f22ec15c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.487+0000 7f22f35b7640 1 -- 172.21.15.16:0/2546905140 shutdown_connections 2024-09-19T07:53:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.487+0000 7f22f35b7640 1 -- 172.21.15.16:0/2546905140 wait complete. 2024-09-19T07:53:52.499 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-09-19T07:53:50.611061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":762964,"kb_used_data":12648,"kb_used_omap":12,"kb_used_meta":750131,"kb_avail":754211756,"statfs":{"total":773094113280,"available":772312838144,"internally_reserved":0,"allocated":12951552,"data_stored":6610908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":768134704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.164369"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.456511+0000","last_change":"2024-09-19T07:53:44.454037+0000","last_active":"2024-09-19T07:53:45.456511+0000","last_peered":"2024-09-19T07:53:45.456511+0000","last_clean":"2024-09-19T07:53:45.456511+0000","last_became_active":"2024-09-19T07:53:44.453756+0000","last_became_peered":"2024-09-19T07:53:44.453756+0000","last_unstale":"2024-09-19T07:53:45.456511+0000","last_undegraded":"2024-09-19T07:53:45.456511+0000","last_fullsized":"2024-09-19T07:53:45.456511+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:41.741615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955591+0000","last_change":"2024-09-19T07:53:44.474712+0000","last_active":"2024-09-19T07:53:45.955591+0000","last_peered":"2024-09-19T07:53:45.955591+0000","last_clean":"2024-09-19T07:53:45.955591+0000","last_became_active":"2024-09-19T07:53:44.474083+0000","last_became_peered":"2024-09-19T07:53:44.474083+0000","last_unstale":"2024-09-19T07:53:45.955591+0000","last_undegraded":"2024-09-19T07:53:45.955591+0000","last_fullsized":"2024-09-19T07:53:45.955591+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:36.929034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956412+0000","last_change":"2024-09-19T07:53:44.481309+0000","last_active":"2024-09-19T07:53:45.956412+0000","last_peered":"2024-09-19T07:53:45.956412+0000","last_clean":"2024-09-19T07:53:45.956412+0000","last_became_active":"2024-09-19T07:53:44.480479+0000","last_became_peered":"2024-09-19T07:53:44.480479+0000","last_unstale":"2024-09-19T07:53:45.956412+0000","last_undegraded":"2024-09-19T07:53:45.956412+0000","last_fullsized":"2024-09-19T07:53:45.956412+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:53.377199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956367+0000","last_change":"2024-09-19T07:53:44.463853+0000","last_active":"2024-09-19T07:53:45.956367+0000","last_peered":"2024-09-19T07:53:45.956367+0000","last_clean":"2024-09-19T07:53:45.956367+0000","last_became_active":"2024-09-19T07:53:44.463569+0000","last_became_peered":"2024-09-19T07:53:44.463569+0000","last_unstale":"2024-09-19T07:53:45.956367+0000","last_undegraded":"2024-09-19T07:53:45.956367+0000","last_fullsized":"2024-09-19T07:53:45.956367+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:30.547787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.456253+0000","last_change":"2024-09-19T07:53:44.460542+0000","last_active":"2024-09-19T07:53:45.456253+0000","last_peered":"2024-09-19T07:53:45.456253+0000","last_clean":"2024-09-19T07:53:45.456253+0000","last_became_active":"2024-09-19T07:53:44.460245+0000","last_became_peered":"2024-09-19T07:53:44.460245+0000","last_unstale":"2024-09-19T07:53:45.456253+0000","last_undegraded":"2024-09-19T07:53:45.456253+0000","last_fullsized":"2024-09-19T07:53:45.456253+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:59.717556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955328+0000","last_change":"2024-09-19T07:53:44.480188+0000","last_active":"2024-09-19T07:53:45.955328+0000","last_peered":"2024-09-19T07:53:45.955328+0000","last_clean":"2024-09-19T07:53:45.955328+0000","last_became_active":"2024-09-19T07:53:44.479867+0000","last_became_peered":"2024-09-19T07:53:44.479867+0000","last_unstale":"2024-09-19T07:53:45.955328+0000","last_undegraded":"2024-09-19T07:53:45.955328+0000","last_fullsized":"2024-09-19T07:53:45.955328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:22.045158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955793+0000","last_change":"2024-09-19T07:53:44.475378+0000","last_active":"2024-09-19T07:53:45.955793+0000","last_peered":"2024-09-19T07:53:45.955793+0000","last_clean":"2024-09-19T07:53:45.955793+0000","last_became_active":"2024-09-19T07:53:44.475024+0000","last_became_peered":"2024-09-19T07:53:44.475024+0000","last_unstale":"2024-09-19T07:53:45.955793+0000","last_undegraded":"2024-09-19T07:53:45.955793+0000","last_fullsized":"2024-09-19T07:53:45.955793+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:02.187400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955416+0000","last_change":"2024-09-19T07:53:44.483364+0000","last_active":"2024-09-19T07:53:45.955416+0000","last_peered":"2024-09-19T07:53:45.955416+0000","last_clean":"2024-09-19T07:53:45.955416+0000","last_became_active":"2024-09-19T07:53:44.481115+0000","last_became_peered":"2024-09-19T07:53:44.481115+0000","last_unstale":"2024-09-19T07:53:45.955416+0000","last_undegraded":"2024-09-19T07:53:45.955416+0000","last_fullsized":"2024-09-19T07:53:45.955416+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:26.365967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956319+0000","last_change":"2024-09-19T07:53:44.472235+0000","last_active":"2024-09-19T07:53:45.956319+0000","last_peered":"2024-09-19T07:53:45.956319+0000","last_clean":"2024-09-19T07:53:45.956319+0000","last_became_active":"2024-09-19T07:53:44.471961+0000","last_became_peered":"2024-09-19T07:53:44.471961+0000","last_unstale":"2024-09-19T07:53:45.956319+0000","last_undegraded":"2024-09-19T07:53:45.956319+0000","last_fullsized":"2024-09-19T07:53:45.956319+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:21.994984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.450177+0000","last_change":"2024-09-19T07:53:44.469391+0000","last_active":"2024-09-19T07:53:45.450177+0000","last_peered":"2024-09-19T07:53:45.450177+0000","last_clean":"2024-09-19T07:53:45.450177+0000","last_became_active":"2024-09-19T07:53:44.469111+0000","last_became_peered":"2024-09-19T07:53:44.469111+0000","last_unstale":"2024-09-19T07:53:45.450177+0000","last_undegraded":"2024-09-19T07:53:45.450177+0000","last_fullsized":"2024-09-19T07:53:45.450177+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:28.530475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.456361+0000","last_change":"2024-09-19T07:53:44.460409+0000","last_active":"2024-09-19T07:53:45.456361+0000","last_peered":"2024-09-19T07:53:45.456361+0000","last_clean":"2024-09-19T07:53:45.456361+0000","last_became_active":"2024-09-19T07:53:44.460137+0000","last_became_peered":"2024-09-19T07:53:44.460137+0000","last_unstale":"2024-09-19T07:53:45.456361+0000","last_undegraded":"2024-09-19T07:53:45.456361+0000","last_fullsized":"2024-09-19T07:53:45.456361+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:19.276802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955949+0000","last_change":"2024-09-19T07:53:44.467120+0000","last_active":"2024-09-19T07:53:45.955949+0000","last_peered":"2024-09-19T07:53:45.955949+0000","last_clean":"2024-09-19T07:53:45.955949+0000","last_became_active":"2024-09-19T07:53:44.466828+0000","last_became_peered":"2024-09-19T07:53:44.466828+0000","last_unstale":"2024-09-19T07:53:45.955949+0000","last_undegraded":"2024-09-19T07:53:45.955949+0000","last_fullsized":"2024-09-19T07:53:45.955949+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:50.813956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.449972+0000","last_change":"2024-09-19T07:53:44.455777+0000","last_active":"2024-09-19T07:53:45.449972+0000","last_peered":"2024-09-19T07:53:45.449972+0000","last_clean":"2024-09-19T07:53:45.449972+0000","last_became_active":"2024-09-19T07:53:44.455467+0000","last_became_peered":"2024-09-19T07:53:44.455467+0000","last_unstale":"2024-09-19T07:53:45.449972+0000","last_undegraded":"2024-09-19T07:53:45.449972+0000","last_fullsized":"2024-09-19T07:53:45.449972+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:41.741615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955692+0000","last_change":"2024-09-19T07:53:44.481813+0000","last_active":"2024-09-19T07:53:45.955692+0000","last_peered":"2024-09-19T07:53:45.955692+0000","last_clean":"2024-09-19T07:53:45.955692+0000","last_became_active":"2024-09-19T07:53:44.480915+0000","last_became_peered":"2024-09-19T07:53:44.480915+0000","last_unstale":"2024-09-19T07:53:45.955692+0000","last_undegraded":"2024-09-19T07:53:45.955692+0000","last_fullsized":"2024-09-19T07:53:45.955692+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:47.199779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.456105+0000","last_change":"2024-09-19T07:53:44.455460+0000","last_active":"2024-09-19T07:53:45.456105+0000","last_peered":"2024-09-19T07:53:45.456105+0000","last_clean":"2024-09-19T07:53:45.456105+0000","last_became_active":"2024-09-19T07:53:44.455164+0000","last_became_peered":"2024-09-19T07:53:44.455164+0000","last_unstale":"2024-09-19T07:53:45.456105+0000","last_undegraded":"2024-09-19T07:53:45.456105+0000","last_fullsized":"2024-09-19T07:53:45.456105+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:33.165277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956592+0000","last_change":"2024-09-19T07:53:44.473808+0000","last_active":"2024-09-19T07:53:45.956592+0000","last_peered":"2024-09-19T07:53:45.956592+0000","last_clean":"2024-09-19T07:53:45.956592+0000","last_became_active":"2024-09-19T07:53:44.473528+0000","last_became_peered":"2024-09-19T07:53:44.473528+0000","last_unstale":"2024-09-19T07:53:45.956592+0000","last_undegraded":"2024-09-19T07:53:45.956592+0000","last_fullsized":"2024-09-19T07:53:45.956592+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:41.452472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.450130+0000","last_change":"2024-09-19T07:53:44.458967+0000","last_active":"2024-09-19T07:53:45.450130+0000","last_peered":"2024-09-19T07:53:45.450130+0000","last_clean":"2024-09-19T07:53:45.450130+0000","last_became_active":"2024-09-19T07:53:44.458687+0000","last_became_peered":"2024-09-19T07:53:44.458687+0000","last_unstale":"2024-09-19T07:53:45.450130+0000","last_undegraded":"2024-09-19T07:53:45.450130+0000","last_fullsized":"2024-09-19T07:53:45.450130+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:43.497533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955733+0000","last_change":"2024-09-19T07:53:44.474993+0000","last_active":"2024-09-19T07:53:45.955733+0000","last_peered":"2024-09-19T07:53:45.955733+0000","last_clean":"2024-09-19T07:53:45.955733+0000","last_became_active":"2024-09-19T07:53:44.474454+0000","last_became_peered":"2024-09-19T07:53:44.474454+0000","last_unstale":"2024-09-19T07:53:45.955733+0000","last_undegraded":"2024-09-19T07:53:45.955733+0000","last_fullsized":"2024-09-19T07:53:45.955733+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:28.268952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955824+0000","last_change":"2024-09-19T07:53:44.465304+0000","last_active":"2024-09-19T07:53:45.955824+0000","last_peered":"2024-09-19T07:53:45.955824+0000","last_clean":"2024-09-19T07:53:45.955824+0000","last_became_active":"2024-09-19T07:53:44.465044+0000","last_became_peered":"2024-09-19T07:53:44.465044+0000","last_unstale":"2024-09-19T07:53:45.955824+0000","last_undegraded":"2024-09-19T07:53:45.955824+0000","last_fullsized":"2024-09-19T07:53:45.955824+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:55.557163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955372+0000","last_change":"2024-09-19T07:53:44.466484+0000","last_active":"2024-09-19T07:53:45.955372+0000","last_peered":"2024-09-19T07:53:45.955372+0000","last_clean":"2024-09-19T07:53:45.955372+0000","last_became_active":"2024-09-19T07:53:44.466195+0000","last_became_peered":"2024-09-19T07:53:44.466195+0000","last_unstale":"2024-09-19T07:53:45.955372+0000","last_undegraded":"2024-09-19T07:53:45.955372+0000","last_fullsized":"2024-09-19T07:53:45.955372+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:20.436455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956289+0000","last_change":"2024-09-19T07:53:44.472944+0000","last_active":"2024-09-19T07:53:45.956289+0000","last_peered":"2024-09-19T07:53:45.956289+0000","last_clean":"2024-09-19T07:53:45.956289+0000","last_became_active":"2024-09-19T07:53:44.472669+0000","last_became_peered":"2024-09-19T07:53:44.472669+0000","last_unstale":"2024-09-19T07:53:45.956289+0000","last_undegraded":"2024-09-19T07:53:45.956289+0000","last_fullsized":"2024-09-19T07:53:45.956289+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:07.395824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.456420+0000","last_change":"2024-09-19T07:53:44.466834+0000","last_active":"2024-09-19T07:53:45.456420+0000","last_peered":"2024-09-19T07:53:45.456420+0000","last_clean":"2024-09-19T07:53:45.456420+0000","last_became_active":"2024-09-19T07:53:44.466204+0000","last_became_peered":"2024-09-19T07:53:44.466204+0000","last_unstale":"2024-09-19T07:53:45.456420+0000","last_undegraded":"2024-09-19T07:53:45.456420+0000","last_fullsized":"2024-09-19T07:53:45.456420+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:45.343582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955815+0000","last_change":"2024-09-19T07:53:44.480566+0000","last_active":"2024-09-19T07:53:45.955815+0000","last_peered":"2024-09-19T07:53:45.955815+0000","last_clean":"2024-09-19T07:53:45.955815+0000","last_became_active":"2024-09-19T07:53:44.480280+0000","last_became_peered":"2024-09-19T07:53:44.480280+0000","last_unstale":"2024-09-19T07:53:45.955815+0000","last_undegraded":"2024-09-19T07:53:45.955815+0000","last_fullsized":"2024-09-19T07:53:45.955815+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:28.525593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.454872+0000","last_change":"2024-09-19T07:53:44.462832+0000","last_active":"2024-09-19T07:53:45.454872+0000","last_peered":"2024-09-19T07:53:45.454872+0000","last_clean":"2024-09-19T07:53:45.454872+0000","last_became_active":"2024-09-19T07:53:44.462538+0000","last_became_peered":"2024-09-19T07:53:44.462538+0000","last_unstale":"2024-09-19T07:53:45.454872+0000","last_undegraded":"2024-09-19T07:53:45.454872+0000","last_fullsized":"2024-09-19T07:53:45.454872+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:02.487846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.455485+0000","last_change":"2024-09-19T07:53:44.460249+0000","last_active":"2024-09-19T07:53:45.455485+0000","last_peered":"2024-09-19T07:53:45.455485+0000","last_clean":"2024-09-19T07:53:45.455485+0000","last_became_active":"2024-09-19T07:53:44.458793+0000","last_became_peered":"2024-09-19T07:53:44.458793+0000","last_unstale":"2024-09-19T07:53:45.455485+0000","last_undegraded":"2024-09-19T07:53:45.455485+0000","last_fullsized":"2024-09-19T07:53:45.455485+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:25.217522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955551+0000","last_change":"2024-09-19T07:53:44.467665+0000","last_active":"2024-09-19T07:53:45.955551+0000","last_peered":"2024-09-19T07:53:45.955551+0000","last_clean":"2024-09-19T07:53:45.955551+0000","last_became_active":"2024-09-19T07:53:44.467388+0000","last_became_peered":"2024-09-19T07:53:44.467388+0000","last_unstale":"2024-09-19T07:53:45.955551+0000","last_undegraded":"2024-09-19T07:53:45.955551+0000","last_fullsized":"2024-09-19T07:53:45.955551+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:09.122218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956000+0000","last_change":"2024-09-19T07:53:44.463945+0000","last_active":"2024-09-19T07:53:45.956000+0000","last_peered":"2024-09-19T07:53:45.956000+0000","last_clean":"2024-09-19T07:53:45.956000+0000","last_became_active":"2024-09-19T07:53:44.463622+0000","last_became_peered":"2024-09-19T07:53:44.463622+0000","last_unstale":"2024-09-19T07:53:45.956000+0000","last_undegraded":"2024-09-19T07:53:45.956000+0000","last_fullsized":"2024-09-19T07:53:45.956000+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:00.929562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956092+0000","last_change":"2024-09-19T07:53:44.479121+0000","last_active":"2024-09-19T07:53:45.956092+0000","last_peered":"2024-09-19T07:53:45.956092+0000","last_clean":"2024-09-19T07:53:45.956092+0000","last_became_active":"2024-09-19T07:53:44.478632+0000","last_became_peered":"2024-09-19T07:53:44.478632+0000","last_unstale":"2024-09-19T07:53:45.956092+0000","last_undegraded":"2024-09-19T07:53:45.956092+0000","last_fullsized":"2024-09-19T07:53:45.956092+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:40.802408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955651+0000","last_change":"2024-09-19T07:53:44.471531+0000","last_active":"2024-09-19T07:53:45.955651+0000","last_peered":"2024-09-19T07:53:45.955651+0000","last_clean":"2024-09-19T07:53:45.955651+0000","last_became_active":"2024-09-19T07:53:44.471247+0000","last_became_peered":"2024-09-19T07:53:44.471247+0000","last_unstale":"2024-09-19T07:53:45.955651+0000","last_undegraded":"2024-09-19T07:53:45.955651+0000","last_fullsized":"2024-09-19T07:53:45.955651+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:27:11.047914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955832+0000","last_change":"2024-09-19T07:53:44.472849+0000","last_active":"2024-09-19T07:53:45.955832+0000","last_peered":"2024-09-19T07:53:45.955832+0000","last_clean":"2024-09-19T07:53:45.955832+0000","last_became_active":"2024-09-19T07:53:44.472565+0000","last_became_peered":"2024-09-19T07:53:44.472565+0000","last_unstale":"2024-09-19T07:53:45.955832+0000","last_undegraded":"2024-09-19T07:53:45.955832+0000","last_fullsized":"2024-09-19T07:53:45.955832+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:35.230052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.455025+0000","last_change":"2024-09-19T07:53:44.461893+0000","last_active":"2024-09-19T07:53:45.455025+0000","last_peered":"2024-09-19T07:53:45.455025+0000","last_clean":"2024-09-19T07:53:45.455025+0000","last_became_active":"2024-09-19T07:53:44.461543+0000","last_became_peered":"2024-09-19T07:53:44.461543+0000","last_unstale":"2024-09-19T07:53:45.455025+0000","last_undegraded":"2024-09-19T07:53:45.455025+0000","last_fullsized":"2024-09-19T07:53:45.455025+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:03.068372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.454874+0000","last_change":"2024-09-19T07:53:44.462124+0000","last_active":"2024-09-19T07:53:45.454874+0000","last_peered":"2024-09-19T07:53:45.454874+0000","last_clean":"2024-09-19T07:53:45.454874+0000","last_became_active":"2024-09-19T07:53:44.461823+0000","last_became_peered":"2024-09-19T07:53:44.461823+0000","last_unstale":"2024-09-19T07:53:45.454874+0000","last_undegraded":"2024-09-19T07:53:45.454874+0000","last_fullsized":"2024-09-19T07:53:45.454874+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:28.530475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.450047+0000","last_change":"2024-09-19T07:53:44.470121+0000","last_active":"2024-09-19T07:53:45.450047+0000","last_peered":"2024-09-19T07:53:45.450047+0000","last_clean":"2024-09-19T07:53:45.450047+0000","last_became_active":"2024-09-19T07:53:44.469821+0000","last_became_peered":"2024-09-19T07:53:44.469821+0000","last_unstale":"2024-09-19T07:53:45.450047+0000","last_undegraded":"2024-09-19T07:53:45.450047+0000","last_fullsized":"2024-09-19T07:53:45.450047+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:05.734979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"117'4","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955755+0000","last_change":"2024-09-19T07:53:43.462786+0000","last_active":"2024-09-19T07:53:45.955755+0000","last_peered":"2024-09-19T07:53:45.955755+0000","last_clean":"2024-09-19T07:53:45.955755+0000","last_became_active":"2024-09-19T07:53:43.462517+0000","last_became_peered":"2024-09-19T07:53:43.462517+0000","last_unstale":"2024-09-19T07:53:45.955755+0000","last_undegraded":"2024-09-19T07:53:45.955755+0000","last_fullsized":"2024-09-19T07:53:45.955755+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:55.555956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956162+0000","last_change":"2024-09-19T07:53:44.482782+0000","last_active":"2024-09-19T07:53:45.956162+0000","last_peered":"2024-09-19T07:53:45.956162+0000","last_clean":"2024-09-19T07:53:45.956162+0000","last_became_active":"2024-09-19T07:53:44.482497+0000","last_became_peered":"2024-09-19T07:53:44.482497+0000","last_unstale":"2024-09-19T07:53:45.956162+0000","last_undegraded":"2024-09-19T07:53:45.956162+0000","last_fullsized":"2024-09-19T07:53:45.956162+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:42.271648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.450173+0000","last_change":"2024-09-19T07:53:43.457859+0000","last_active":"2024-09-19T07:53:45.450173+0000","last_peered":"2024-09-19T07:53:45.450173+0000","last_clean":"2024-09-19T07:53:45.450173+0000","last_became_active":"2024-09-19T07:53:43.457574+0000","last_became_peered":"2024-09-19T07:53:43.457574+0000","last_unstale":"2024-09-19T07:53:45.450173+0000","last_undegraded":"2024-09-19T07:53:45.450173+0000","last_fullsized":"2024-09-19T07:53:45.450173+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:37.240608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.455301+0000","last_change":"2024-09-19T07:53:44.468508+0000","last_active":"2024-09-19T07:53:45.455301+0000","last_peered":"2024-09-19T07:53:45.455301+0000","last_clean":"2024-09-19T07:53:45.455301+0000","last_became_active":"2024-09-19T07:53:44.468228+0000","last_became_peered":"2024-09-19T07:53:44.468228+0000","last_unstale":"2024-09-19T07:53:45.455301+0000","last_undegraded":"2024-09-19T07:53:45.455301+0000","last_fullsized":"2024-09-19T07:53:45.455301+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:46.275247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956300+0000","last_change":"2024-09-19T07:53:43.464281+0000","last_active":"2024-09-19T07:53:45.956300+0000","last_peered":"2024-09-19T07:53:45.956300+0000","last_clean":"2024-09-19T07:53:45.956300+0000","last_became_active":"2024-09-19T07:53:43.464005+0000","last_became_peered":"2024-09-19T07:53:43.464005+0000","last_unstale":"2024-09-19T07:53:45.956300+0000","last_undegraded":"2024-09-19T07:53:45.956300+0000","last_fullsized":"2024-09-19T07:53:45.956300+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:40.739604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956261+0000","last_change":"2024-09-19T07:53:44.463658+0000","last_active":"2024-09-19T07:53:45.956261+0000","last_peered":"2024-09-19T07:53:45.956261+0000","last_clean":"2024-09-19T07:53:45.956261+0000","last_became_active":"2024-09-19T07:53:44.463373+0000","last_became_peered":"2024-09-19T07:53:44.463373+0000","last_unstale":"2024-09-19T07:53:45.956261+0000","last_undegraded":"2024-09-19T07:53:45.956261+0000","last_fullsized":"2024-09-19T07:53:45.956261+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:25.217522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.449990+0000","last_change":"2024-09-19T07:53:43.459863+0000","last_active":"2024-09-19T07:53:45.449990+0000","last_peered":"2024-09-19T07:53:45.449990+0000","last_clean":"2024-09-19T07:53:45.449990+0000","last_became_active":"2024-09-19T07:53:43.459538+0000","last_became_peered":"2024-09-19T07:53:43.459538+0000","last_unstale":"2024-09-19T07:53:45.449990+0000","last_undegraded":"2024-09-19T07:53:45.449990+0000","last_fullsized":"2024-09-19T07:53:45.449990+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:47.768851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.449836+0000","last_change":"2024-09-19T07:53:44.460976+0000","last_active":"2024-09-19T07:53:45.449836+0000","last_peered":"2024-09-19T07:53:45.449836+0000","last_clean":"2024-09-19T07:53:45.449836+0000","last_became_active":"2024-09-19T07:53:44.460685+0000","last_became_peered":"2024-09-19T07:53:44.460685+0000","last_unstale":"2024-09-19T07:53:45.449836+0000","last_undegraded":"2024-09-19T07:53:45.449836+0000","last_fullsized":"2024-09-19T07:53:45.449836+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:19.276802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.456127+0000","last_change":"2024-09-19T07:53:43.459625+0000","last_active":"2024-09-19T07:53:45.456127+0000","last_peered":"2024-09-19T07:53:45.456127+0000","last_clean":"2024-09-19T07:53:45.456127+0000","last_became_active":"2024-09-19T07:53:43.459283+0000","last_became_peered":"2024-09-19T07:53:43.459283+0000","last_unstale":"2024-09-19T07:53:45.456127+0000","last_undegraded":"2024-09-19T07:53:45.456127+0000","last_fullsized":"2024-09-19T07:53:45.456127+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:37.240608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.456113+0000","last_change":"2024-09-19T07:53:44.461446+0000","last_active":"2024-09-19T07:53:45.456113+0000","last_peered":"2024-09-19T07:53:45.456113+0000","last_clean":"2024-09-19T07:53:45.456113+0000","last_became_active":"2024-09-19T07:53:44.461158+0000","last_became_peered":"2024-09-19T07:53:44.461158+0000","last_unstale":"2024-09-19T07:53:45.456113+0000","last_undegraded":"2024-09-19T07:53:45.456113+0000","last_fullsized":"2024-09-19T07:53:45.456113+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:37.126896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955923+0000","last_change":"2024-09-19T07:53:43.467953+0000","last_active":"2024-09-19T07:53:45.955923+0000","last_peered":"2024-09-19T07:53:45.955923+0000","last_clean":"2024-09-19T07:53:45.955923+0000","last_became_active":"2024-09-19T07:53:43.467684+0000","last_became_peered":"2024-09-19T07:53:43.467684+0000","last_unstale":"2024-09-19T07:53:45.955923+0000","last_undegraded":"2024-09-19T07:53:45.955923+0000","last_fullsized":"2024-09-19T07:53:45.955923+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:46.197768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955975+0000","last_change":"2024-09-19T07:53:44.471846+0000","last_active":"2024-09-19T07:53:45.955975+0000","last_peered":"2024-09-19T07:53:45.955975+0000","last_clean":"2024-09-19T07:53:45.955975+0000","last_became_active":"2024-09-19T07:53:44.471573+0000","last_became_peered":"2024-09-19T07:53:44.471573+0000","last_unstale":"2024-09-19T07:53:45.955975+0000","last_undegraded":"2024-09-19T07:53:45.955975+0000","last_fullsized":"2024-09-19T07:53:45.955975+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:50.563314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.455084+0000","last_change":"2024-09-19T07:53:43.458791+0000","last_active":"2024-09-19T07:53:45.455084+0000","last_peered":"2024-09-19T07:53:45.455084+0000","last_clean":"2024-09-19T07:53:45.455084+0000","last_became_active":"2024-09-19T07:53:43.458509+0000","last_became_peered":"2024-09-19T07:53:43.458509+0000","last_unstale":"2024-09-19T07:53:45.455084+0000","last_undegraded":"2024-09-19T07:53:45.455084+0000","last_fullsized":"2024-09-19T07:53:45.455084+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:54.555152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.450043+0000","last_change":"2024-09-19T07:53:44.468647+0000","last_active":"2024-09-19T07:53:45.450043+0000","last_peered":"2024-09-19T07:53:45.450043+0000","last_clean":"2024-09-19T07:53:45.450043+0000","last_became_active":"2024-09-19T07:53:44.468375+0000","last_became_peered":"2024-09-19T07:53:44.468375+0000","last_unstale":"2024-09-19T07:53:45.450043+0000","last_undegraded":"2024-09-19T07:53:45.450043+0000","last_fullsized":"2024-09-19T07:53:45.450043+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:49.561080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956676+0000","last_change":"2024-09-19T07:53:43.466824+0000","last_active":"2024-09-19T07:53:45.956676+0000","last_peered":"2024-09-19T07:53:45.956676+0000","last_clean":"2024-09-19T07:53:45.956676+0000","last_became_active":"2024-09-19T07:53:43.466546+0000","last_became_peered":"2024-09-19T07:53:43.466546+0000","last_unstale":"2024-09-19T07:53:45.956676+0000","last_undegraded":"2024-09-19T07:53:45.956676+0000","last_fullsized":"2024-09-19T07:53:45.956676+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:52.318252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955632+0000","last_change":"2024-09-19T07:53:44.464599+0000","last_active":"2024-09-19T07:53:45.955632+0000","last_peered":"2024-09-19T07:53:45.955632+0000","last_clean":"2024-09-19T07:53:45.955632+0000","last_became_active":"2024-09-19T07:53:44.464335+0000","last_became_peered":"2024-09-19T07:53:44.464335+0000","last_unstale":"2024-09-19T07:53:45.955632+0000","last_undegraded":"2024-09-19T07:53:45.955632+0000","last_fullsized":"2024-09-19T07:53:45.955632+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:13.056229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955886+0000","last_change":"2024-09-19T07:53:43.468918+0000","last_active":"2024-09-19T07:53:45.955886+0000","last_peered":"2024-09-19T07:53:45.955886+0000","last_clean":"2024-09-19T07:53:45.955886+0000","last_became_active":"2024-09-19T07:53:43.468594+0000","last_became_peered":"2024-09-19T07:53:43.468594+0000","last_unstale":"2024-09-19T07:53:45.955886+0000","last_undegraded":"2024-09-19T07:53:45.955886+0000","last_fullsized":"2024-09-19T07:53:45.955886+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:47.768851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.455307+0000","last_change":"2024-09-19T07:53:44.466217+0000","last_active":"2024-09-19T07:53:45.455307+0000","last_peered":"2024-09-19T07:53:45.455307+0000","last_clean":"2024-09-19T07:53:45.455307+0000","last_became_active":"2024-09-19T07:53:44.465405+0000","last_became_peered":"2024-09-19T07:53:44.465405+0000","last_unstale":"2024-09-19T07:53:45.455307+0000","last_undegraded":"2024-09-19T07:53:45.455307+0000","last_fullsized":"2024-09-19T07:53:45.455307+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:28.525593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.455158+0000","last_change":"2024-09-19T07:53:43.463233+0000","last_active":"2024-09-19T07:53:45.455158+0000","last_peered":"2024-09-19T07:53:45.455158+0000","last_clean":"2024-09-19T07:53:45.455158+0000","last_became_active":"2024-09-19T07:53:43.462620+0000","last_became_peered":"2024-09-19T07:53:43.462620+0000","last_unstale":"2024-09-19T07:53:45.455158+0000","last_undegraded":"2024-09-19T07:53:45.455158+0000","last_fullsized":"2024-09-19T07:53:45.455158+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:57:59.927552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955871+0000","last_change":"2024-09-19T07:53:44.481007+0000","last_active":"2024-09-19T07:53:45.955871+0000","last_peered":"2024-09-19T07:53:45.955871+0000","last_clean":"2024-09-19T07:53:45.955871+0000","last_became_active":"2024-09-19T07:53:44.479548+0000","last_became_peered":"2024-09-19T07:53:44.479548+0000","last_unstale":"2024-09-19T07:53:45.955871+0000","last_undegraded":"2024-09-19T07:53:45.955871+0000","last_fullsized":"2024-09-19T07:53:45.955871+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:53.320262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"117'3","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.455284+0000","last_change":"2024-09-19T07:53:43.463469+0000","last_active":"2024-09-19T07:53:45.455284+0000","last_peered":"2024-09-19T07:53:45.455284+0000","last_clean":"2024-09-19T07:53:45.455284+0000","last_became_active":"2024-09-19T07:53:43.463042+0000","last_became_peered":"2024-09-19T07:53:43.463042+0000","last_unstale":"2024-09-19T07:53:45.455284+0000","last_undegraded":"2024-09-19T07:53:45.455284+0000","last_fullsized":"2024-09-19T07:53:45.455284+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:32.163267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.455518+0000","last_change":"2024-09-19T07:53:44.466514+0000","last_active":"2024-09-19T07:53:45.455518+0000","last_peered":"2024-09-19T07:53:45.455518+0000","last_clean":"2024-09-19T07:53:45.455518+0000","last_became_active":"2024-09-19T07:53:44.464961+0000","last_became_peered":"2024-09-19T07:53:44.464961+0000","last_unstale":"2024-09-19T07:53:45.455518+0000","last_undegraded":"2024-09-19T07:53:45.455518+0000","last_fullsized":"2024-09-19T07:53:45.455518+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:07.395824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"117'6","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956390+0000","last_change":"2024-09-19T07:53:43.460277+0000","last_active":"2024-09-19T07:53:45.956390+0000","last_peered":"2024-09-19T07:53:45.956390+0000","last_clean":"2024-09-19T07:53:45.956390+0000","last_became_active":"2024-09-19T07:53:43.460028+0000","last_became_peered":"2024-09-19T07:53:43.460028+0000","last_unstale":"2024-09-19T07:53:45.956390+0000","last_undegraded":"2024-09-19T07:53:45.956390+0000","last_fullsized":"2024-09-19T07:53:45.956390+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:29.545776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955634+0000","last_change":"2024-09-19T07:53:44.474267+0000","last_active":"2024-09-19T07:53:45.955634+0000","last_peered":"2024-09-19T07:53:45.955634+0000","last_clean":"2024-09-19T07:53:45.955634+0000","last_became_active":"2024-09-19T07:53:44.473919+0000","last_became_peered":"2024-09-19T07:53:44.473919+0000","last_unstale":"2024-09-19T07:53:45.955634+0000","last_undegraded":"2024-09-19T07:53:45.955634+0000","last_fullsized":"2024-09-19T07:53:45.955634+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:39:04.627509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956214+0000","last_change":"2024-09-19T07:53:43.463274+0000","last_active":"2024-09-19T07:53:45.956214+0000","last_peered":"2024-09-19T07:53:45.956214+0000","last_clean":"2024-09-19T07:53:45.956214+0000","last_became_active":"2024-09-19T07:53:43.463007+0000","last_became_peered":"2024-09-19T07:53:43.463007+0000","last_unstale":"2024-09-19T07:53:45.956214+0000","last_undegraded":"2024-09-19T07:53:45.956214+0000","last_fullsized":"2024-09-19T07:53:45.956214+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:06.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.449876+0000","last_change":"2024-09-19T07:53:44.463985+0000","last_active":"2024-09-19T07:53:45.449876+0000","last_peered":"2024-09-19T07:53:45.449876+0000","last_clean":"2024-09-19T07:53:45.449876+0000","last_became_active":"2024-09-19T07:53:44.463678+0000","last_became_peered":"2024-09-19T07:53:44.463678+0000","last_unstale":"2024-09-19T07:53:45.449876+0000","last_undegraded":"2024-09-19T07:53:45.449876+0000","last_fullsized":"2024-09-19T07:53:45.449876+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:45.343582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956260+0000","last_change":"2024-09-19T07:53:43.455019+0000","last_active":"2024-09-19T07:53:45.956260+0000","last_peered":"2024-09-19T07:53:45.956260+0000","last_clean":"2024-09-19T07:53:45.956260+0000","last_became_active":"2024-09-19T07:53:43.454686+0000","last_became_peered":"2024-09-19T07:53:43.454686+0000","last_unstale":"2024-09-19T07:53:45.956260+0000","last_undegraded":"2024-09-19T07:53:45.956260+0000","last_fullsized":"2024-09-19T07:53:45.956260+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:20.992974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956646+0000","last_change":"2024-09-19T07:53:44.457728+0000","last_active":"2024-09-19T07:53:45.956646+0000","last_peered":"2024-09-19T07:53:45.956646+0000","last_clean":"2024-09-19T07:53:45.956646+0000","last_became_active":"2024-09-19T07:53:44.457456+0000","last_became_peered":"2024-09-19T07:53:44.457456+0000","last_unstale":"2024-09-19T07:53:45.956646+0000","last_undegraded":"2024-09-19T07:53:45.956646+0000","last_fullsized":"2024-09-19T07:53:45.956646+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:37:56.076282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956614+0000","last_change":"2024-09-19T07:53:43.466167+0000","last_active":"2024-09-19T07:53:45.956614+0000","last_peered":"2024-09-19T07:53:45.956614+0000","last_clean":"2024-09-19T07:53:45.956614+0000","last_became_active":"2024-09-19T07:53:43.465882+0000","last_became_peered":"2024-09-19T07:53:43.465882+0000","last_unstale":"2024-09-19T07:53:45.956614+0000","last_undegraded":"2024-09-19T07:53:45.956614+0000","last_fullsized":"2024-09-19T07:53:45.956614+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:07.875623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.455208+0000","last_change":"2024-09-19T07:53:44.469161+0000","last_active":"2024-09-19T07:53:45.455208+0000","last_peered":"2024-09-19T07:53:45.455208+0000","last_clean":"2024-09-19T07:53:45.455208+0000","last_became_active":"2024-09-19T07:53:44.468896+0000","last_became_peered":"2024-09-19T07:53:44.468896+0000","last_unstale":"2024-09-19T07:53:45.455208+0000","last_undegraded":"2024-09-19T07:53:45.455208+0000","last_fullsized":"2024-09-19T07:53:45.455208+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:05.983399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.450013+0000","last_change":"2024-09-19T07:53:43.468990+0000","last_active":"2024-09-19T07:53:45.450013+0000","last_peered":"2024-09-19T07:53:45.450013+0000","last_clean":"2024-09-19T07:53:45.450013+0000","last_became_active":"2024-09-19T07:53:43.468625+0000","last_became_peered":"2024-09-19T07:53:43.468625+0000","last_unstale":"2024-09-19T07:53:45.450013+0000","last_undegraded":"2024-09-19T07:53:45.450013+0000","last_fullsized":"2024-09-19T07:53:45.450013+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:43.968446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.456340+0000","last_change":"2024-09-19T07:53:44.466474+0000","last_active":"2024-09-19T07:53:45.456340+0000","last_peered":"2024-09-19T07:53:45.456340+0000","last_clean":"2024-09-19T07:53:45.456340+0000","last_became_active":"2024-09-19T07:53:44.466230+0000","last_became_peered":"2024-09-19T07:53:44.466230+0000","last_unstale":"2024-09-19T07:53:45.456340+0000","last_undegraded":"2024-09-19T07:53:45.456340+0000","last_fullsized":"2024-09-19T07:53:45.456340+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:02.071494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955645+0000","last_change":"2024-09-19T07:53:43.463386+0000","last_active":"2024-09-19T07:53:45.955645+0000","last_peered":"2024-09-19T07:53:45.955645+0000","last_clean":"2024-09-19T07:53:45.955645+0000","last_became_active":"2024-09-19T07:53:43.463020+0000","last_became_peered":"2024-09-19T07:53:43.463020+0000","last_unstale":"2024-09-19T07:53:45.955645+0000","last_undegraded":"2024-09-19T07:53:45.955645+0000","last_fullsized":"2024-09-19T07:53:45.955645+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:01.309102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956576+0000","last_change":"2024-09-19T07:53:44.480939+0000","last_active":"2024-09-19T07:53:45.956576+0000","last_peered":"2024-09-19T07:53:45.956576+0000","last_clean":"2024-09-19T07:53:45.956576+0000","last_became_active":"2024-09-19T07:53:44.480634+0000","last_became_peered":"2024-09-19T07:53:44.480634+0000","last_unstale":"2024-09-19T07:53:45.956576+0000","last_undegraded":"2024-09-19T07:53:45.956576+0000","last_fullsized":"2024-09-19T07:53:45.956576+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:13:47.945892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956394+0000","last_change":"2024-09-19T07:53:43.464937+0000","last_active":"2024-09-19T07:53:45.956394+0000","last_peered":"2024-09-19T07:53:45.956394+0000","last_clean":"2024-09-19T07:53:45.956394+0000","last_became_active":"2024-09-19T07:53:43.464614+0000","last_became_peered":"2024-09-19T07:53:43.464614+0000","last_unstale":"2024-09-19T07:53:45.956394+0000","last_undegraded":"2024-09-19T07:53:45.956394+0000","last_fullsized":"2024-09-19T07:53:45.956394+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:48.559070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.455583+0000","last_change":"2024-09-19T07:53:44.466748+0000","last_active":"2024-09-19T07:53:45.455583+0000","last_peered":"2024-09-19T07:53:45.455583+0000","last_clean":"2024-09-19T07:53:45.455583+0000","last_became_active":"2024-09-19T07:53:44.465355+0000","last_became_peered":"2024-09-19T07:53:44.465355+0000","last_unstale":"2024-09-19T07:53:45.455583+0000","last_undegraded":"2024-09-19T07:53:45.455583+0000","last_fullsized":"2024-09-19T07:53:45.455583+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:21.994984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956268+0000","last_change":"2024-09-19T07:53:43.466472+0000","last_active":"2024-09-19T07:53:45.956268+0000","last_peered":"2024-09-19T07:53:45.956268+0000","last_clean":"2024-09-19T07:53:45.956268+0000","last_became_active":"2024-09-19T07:53:43.466216+0000","last_became_peered":"2024-09-19T07:53:43.466216+0000","last_unstale":"2024-09-19T07:53:45.956268+0000","last_undegraded":"2024-09-19T07:53:45.956268+0000","last_fullsized":"2024-09-19T07:53:45.956268+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:44.341571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956320+0000","last_change":"2024-09-19T07:53:44.480405+0000","last_active":"2024-09-19T07:53:45.956320+0000","last_peered":"2024-09-19T07:53:45.956320+0000","last_clean":"2024-09-19T07:53:45.956320+0000","last_became_active":"2024-09-19T07:53:44.479696+0000","last_became_peered":"2024-09-19T07:53:44.479696+0000","last_unstale":"2024-09-19T07:53:45.956320+0000","last_undegraded":"2024-09-19T07:53:45.956320+0000","last_fullsized":"2024-09-19T07:53:45.956320+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:56.319634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955834+0000","last_change":"2024-09-19T07:53:43.463463+0000","last_active":"2024-09-19T07:53:45.955834+0000","last_peered":"2024-09-19T07:53:45.955834+0000","last_clean":"2024-09-19T07:53:45.955834+0000","last_became_active":"2024-09-19T07:53:43.463197+0000","last_became_peered":"2024-09-19T07:53:43.463197+0000","last_unstale":"2024-09-19T07:53:45.955834+0000","last_undegraded":"2024-09-19T07:53:45.955834+0000","last_fullsized":"2024-09-19T07:53:45.955834+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:27.266942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956498+0000","last_change":"2024-09-19T07:53:44.466715+0000","last_active":"2024-09-19T07:53:45.956498+0000","last_peered":"2024-09-19T07:53:45.956498+0000","last_clean":"2024-09-19T07:53:45.956498+0000","last_became_active":"2024-09-19T07:53:44.466187+0000","last_became_peered":"2024-09-19T07:53:44.466187+0000","last_unstale":"2024-09-19T07:53:45.956498+0000","last_undegraded":"2024-09-19T07:53:45.956498+0000","last_fullsized":"2024-09-19T07:53:45.956498+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:35.653193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955830+0000","last_change":"2024-09-19T07:53:43.465167+0000","last_active":"2024-09-19T07:53:45.955830+0000","last_peered":"2024-09-19T07:53:45.955830+0000","last_clean":"2024-09-19T07:53:45.955830+0000","last_became_active":"2024-09-19T07:53:43.464875+0000","last_became_peered":"2024-09-19T07:53:43.464875+0000","last_unstale":"2024-09-19T07:53:45.955830+0000","last_undegraded":"2024-09-19T07:53:45.955830+0000","last_fullsized":"2024-09-19T07:53:45.955830+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:18.274791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956174+0000","last_change":"2024-09-19T07:53:44.481091+0000","last_active":"2024-09-19T07:53:45.956174+0000","last_peered":"2024-09-19T07:53:45.956174+0000","last_clean":"2024-09-19T07:53:45.956174+0000","last_became_active":"2024-09-19T07:53:44.480524+0000","last_became_peered":"2024-09-19T07:53:44.480524+0000","last_unstale":"2024-09-19T07:53:45.956174+0000","last_undegraded":"2024-09-19T07:53:45.956174+0000","last_fullsized":"2024-09-19T07:53:45.956174+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:00.047940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956719+0000","last_change":"2024-09-19T07:53:43.465392+0000","last_active":"2024-09-19T07:53:45.956719+0000","last_peered":"2024-09-19T07:53:45.956719+0000","last_clean":"2024-09-19T07:53:45.956719+0000","last_became_active":"2024-09-19T07:53:43.465112+0000","last_became_peered":"2024-09-19T07:53:43.465112+0000","last_unstale":"2024-09-19T07:53:45.956719+0000","last_undegraded":"2024-09-19T07:53:45.956719+0000","last_fullsized":"2024-09-19T07:53:45.956719+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:24.215511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.455394+0000","last_change":"2024-09-19T07:53:44.461101+0000","last_active":"2024-09-19T07:53:45.455394+0000","last_peered":"2024-09-19T07:53:45.455394+0000","last_clean":"2024-09-19T07:53:45.455394+0000","last_became_active":"2024-09-19T07:53:44.460795+0000","last_became_peered":"2024-09-19T07:53:44.460795+0000","last_unstale":"2024-09-19T07:53:45.455394+0000","last_undegraded":"2024-09-19T07:53:45.455394+0000","last_fullsized":"2024-09-19T07:53:45.455394+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:08.877634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955294+0000","last_change":"2024-09-19T07:53:43.467250+0000","last_active":"2024-09-19T07:53:45.955294+0000","last_peered":"2024-09-19T07:53:45.955294+0000","last_clean":"2024-09-19T07:53:45.955294+0000","last_became_active":"2024-09-19T07:53:43.467030+0000","last_became_peered":"2024-09-19T07:53:43.467030+0000","last_unstale":"2024-09-19T07:53:45.955294+0000","last_undegraded":"2024-09-19T07:53:45.955294+0000","last_fullsized":"2024-09-19T07:53:45.955294+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:58.715546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956238+0000","last_change":"2024-09-19T07:53:44.469053+0000","last_active":"2024-09-19T07:53:45.956238+0000","last_peered":"2024-09-19T07:53:45.956238+0000","last_clean":"2024-09-19T07:53:45.956238+0000","last_became_active":"2024-09-19T07:53:44.468779+0000","last_became_peered":"2024-09-19T07:53:44.468779+0000","last_unstale":"2024-09-19T07:53:45.956238+0000","last_undegraded":"2024-09-19T07:53:45.956238+0000","last_fullsized":"2024-09-19T07:53:45.956238+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:27:38.786468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955790+0000","last_change":"2024-09-19T07:53:43.464536+0000","last_active":"2024-09-19T07:53:45.955790+0000","last_peered":"2024-09-19T07:53:45.955790+0000","last_clean":"2024-09-19T07:53:45.955790+0000","last_became_active":"2024-09-19T07:53:43.462203+0000","last_became_peered":"2024-09-19T07:53:43.462203+0000","last_unstale":"2024-09-19T07:53:45.955790+0000","last_undegraded":"2024-09-19T07:53:45.955790+0000","last_fullsized":"2024-09-19T07:53:45.955790+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:35.927023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.450207+0000","last_change":"2024-09-19T07:53:44.462128+0000","last_active":"2024-09-19T07:53:45.450207+0000","last_peered":"2024-09-19T07:53:45.450207+0000","last_clean":"2024-09-19T07:53:45.450207+0000","last_became_active":"2024-09-19T07:53:44.461851+0000","last_became_peered":"2024-09-19T07:53:44.461851+0000","last_unstale":"2024-09-19T07:53:45.450207+0000","last_undegraded":"2024-09-19T07:53:45.450207+0000","last_fullsized":"2024-09-19T07:53:45.450207+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:02.487846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"118'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955880+0000","last_change":"2024-09-19T07:53:43.468893+0000","last_active":"2024-09-19T07:53:45.955880+0000","last_peered":"2024-09-19T07:53:45.955880+0000","last_clean":"2024-09-19T07:53:45.955880+0000","last_became_active":"2024-09-19T07:53:43.468581+0000","last_became_peered":"2024-09-19T07:53:43.468581+0000","last_unstale":"2024-09-19T07:53:45.955880+0000","last_undegraded":"2024-09-19T07:53:45.955880+0000","last_fullsized":"2024-09-19T07:53:45.955880+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:21.308817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956279+0000","last_change":"2024-09-19T07:53:44.463751+0000","last_active":"2024-09-19T07:53:45.956279+0000","last_peered":"2024-09-19T07:53:45.956279+0000","last_clean":"2024-09-19T07:53:45.956279+0000","last_became_active":"2024-09-19T07:53:44.463490+0000","last_became_peered":"2024-09-19T07:53:44.463490+0000","last_unstale":"2024-09-19T07:53:45.956279+0000","last_undegraded":"2024-09-19T07:53:45.956279+0000","last_fullsized":"2024-09-19T07:53:45.956279+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:02.197826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955925+0000","last_change":"2024-09-19T07:53:43.468830+0000","last_active":"2024-09-19T07:53:45.955925+0000","last_peered":"2024-09-19T07:53:45.955925+0000","last_clean":"2024-09-19T07:53:45.955925+0000","last_became_active":"2024-09-19T07:53:43.468521+0000","last_became_peered":"2024-09-19T07:53:43.468521+0000","last_unstale":"2024-09-19T07:53:45.955925+0000","last_undegraded":"2024-09-19T07:53:45.955925+0000","last_fullsized":"2024-09-19T07:53:45.955925+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:37.240608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956008+0000","last_change":"2024-09-19T07:53:44.472102+0000","last_active":"2024-09-19T07:53:45.956008+0000","last_peered":"2024-09-19T07:53:45.956008+0000","last_clean":"2024-09-19T07:53:45.956008+0000","last_became_active":"2024-09-19T07:53:44.471800+0000","last_became_peered":"2024-09-19T07:53:44.471800+0000","last_unstale":"2024-09-19T07:53:45.956008+0000","last_undegraded":"2024-09-19T07:53:45.956008+0000","last_fullsized":"2024-09-19T07:53:45.956008+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:06.662848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956330+0000","last_change":"2024-09-19T07:53:43.464402+0000","last_active":"2024-09-19T07:53:45.956330+0000","last_peered":"2024-09-19T07:53:45.956330+0000","last_clean":"2024-09-19T07:53:45.956330+0000","last_became_active":"2024-09-19T07:53:43.464122+0000","last_became_peered":"2024-09-19T07:53:43.464122+0000","last_unstale":"2024-09-19T07:53:45.956330+0000","last_undegraded":"2024-09-19T07:53:45.956330+0000","last_fullsized":"2024-09-19T07:53:45.956330+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:34.228041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956454+0000","last_change":"2024-09-19T07:53:44.467967+0000","last_active":"2024-09-19T07:53:45.956454+0000","last_peered":"2024-09-19T07:53:45.956454+0000","last_clean":"2024-09-19T07:53:45.956454+0000","last_became_active":"2024-09-19T07:53:44.467699+0000","last_became_peered":"2024-09-19T07:53:44.467699+0000","last_unstale":"2024-09-19T07:53:45.956454+0000","last_undegraded":"2024-09-19T07:53:45.956454+0000","last_fullsized":"2024-09-19T07:53:45.956454+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:04.391779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.456398+0000","last_change":"2024-09-19T07:53:43.454771+0000","last_active":"2024-09-19T07:53:45.456398+0000","last_peered":"2024-09-19T07:53:45.456398+0000","last_clean":"2024-09-19T07:53:45.456398+0000","last_became_active":"2024-09-19T07:53:43.454309+0000","last_became_peered":"2024-09-19T07:53:43.454309+0000","last_unstale":"2024-09-19T07:53:45.456398+0000","last_undegraded":"2024-09-19T07:53:45.456398+0000","last_fullsized":"2024-09-19T07:53:45.456398+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:43.968446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956628+0000","last_change":"2024-09-19T07:53:44.466447+0000","last_active":"2024-09-19T07:53:45.956628+0000","last_peered":"2024-09-19T07:53:45.956628+0000","last_clean":"2024-09-19T07:53:45.956628+0000","last_became_active":"2024-09-19T07:53:44.465888+0000","last_became_peered":"2024-09-19T07:53:44.465888+0000","last_unstale":"2024-09-19T07:53:45.956628+0000","last_undegraded":"2024-09-19T07:53:45.956628+0000","last_fullsized":"2024-09-19T07:53:45.956628+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:19.095190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.456533+0000","last_change":"2024-09-19T07:53:43.459958+0000","last_active":"2024-09-19T07:53:45.456533+0000","last_peered":"2024-09-19T07:53:45.456533+0000","last_clean":"2024-09-19T07:53:45.456533+0000","last_became_active":"2024-09-19T07:53:43.459470+0000","last_became_peered":"2024-09-19T07:53:43.459470+0000","last_unstale":"2024-09-19T07:53:45.456533+0000","last_undegraded":"2024-09-19T07:53:45.456533+0000","last_fullsized":"2024-09-19T07:53:45.456533+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:47.768851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955783+0000","last_change":"2024-09-19T07:53:44.470487+0000","last_active":"2024-09-19T07:53:45.955783+0000","last_peered":"2024-09-19T07:53:45.955783+0000","last_clean":"2024-09-19T07:53:45.955783+0000","last_became_active":"2024-09-19T07:53:44.470216+0000","last_became_peered":"2024-09-19T07:53:44.470216+0000","last_unstale":"2024-09-19T07:53:45.955783+0000","last_undegraded":"2024-09-19T07:53:45.955783+0000","last_fullsized":"2024-09-19T07:53:45.955783+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:08.877634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956065+0000","last_change":"2024-09-19T07:53:43.467051+0000","last_active":"2024-09-19T07:53:45.956065+0000","last_peered":"2024-09-19T07:53:45.956065+0000","last_clean":"2024-09-19T07:53:45.956065+0000","last_became_active":"2024-09-19T07:53:43.466667+0000","last_became_peered":"2024-09-19T07:53:43.466667+0000","last_unstale":"2024-09-19T07:53:45.956065+0000","last_undegraded":"2024-09-19T07:53:45.956065+0000","last_fullsized":"2024-09-19T07:53:45.956065+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:49.561304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.450124+0000","last_change":"2024-09-19T07:53:44.464319+0000","last_active":"2024-09-19T07:53:45.450124+0000","last_peered":"2024-09-19T07:53:45.450124+0000","last_clean":"2024-09-19T07:53:45.450124+0000","last_became_active":"2024-09-19T07:53:44.463940+0000","last_became_peered":"2024-09-19T07:53:44.463940+0000","last_unstale":"2024-09-19T07:53:45.450124+0000","last_undegraded":"2024-09-19T07:53:45.450124+0000","last_fullsized":"2024-09-19T07:53:45.450124+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.258505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955933+0000","last_change":"2024-09-19T07:53:43.465164+0000","last_active":"2024-09-19T07:53:45.955933+0000","last_peered":"2024-09-19T07:53:45.955933+0000","last_clean":"2024-09-19T07:53:45.955933+0000","last_became_active":"2024-09-19T07:53:43.464888+0000","last_became_peered":"2024-09-19T07:53:43.464888+0000","last_unstale":"2024-09-19T07:53:45.955933+0000","last_undegraded":"2024-09-19T07:53:45.955933+0000","last_fullsized":"2024-09-19T07:53:45.955933+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:01.069484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.956512+0000","last_change":"2024-09-19T07:53:44.464883+0000","last_active":"2024-09-19T07:53:45.956512+0000","last_peered":"2024-09-19T07:53:45.956512+0000","last_clean":"2024-09-19T07:53:45.956512+0000","last_became_active":"2024-09-19T07:53:44.464617+0000","last_became_peered":"2024-09-19T07:53:44.464617+0000","last_unstale":"2024-09-19T07:53:45.956512+0000","last_undegraded":"2024-09-19T07:53:45.956512+0000","last_fullsized":"2024-09-19T07:53:45.956512+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:43.440597+0000","last_clean_scrub_stamp":"2024-09-19T07:53:43.440597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:23.524844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"117'8","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955380+0000","last_change":"2024-09-19T07:53:43.459564+0000","last_active":"2024-09-19T07:53:45.955380+0000","last_peered":"2024-09-19T07:53:45.955380+0000","last_clean":"2024-09-19T07:53:45.955380+0000","last_became_active":"2024-09-19T07:53:43.459126+0000","last_became_peered":"2024-09-19T07:53:43.459126+0000","last_unstale":"2024-09-19T07:53:45.955380+0000","last_undegraded":"2024-09-19T07:53:45.955380+0000","last_fullsized":"2024-09-19T07:53:45.955380+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:53:42.438586+0000","last_clean_scrub_stamp":"2024-09-19T07:53:42.438586+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:27.523582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":346,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T07:53:45.955606+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:53:45.955606+0000","last_peered":"2024-09-19T07:53:45.955606+0000","last_clean":"2024-09-19T07:53:45.955606+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:53:45.955606+0000","last_undegraded":"2024-09-19T07:53:45.955606+0000","last_fullsized":"2024-09-19T07:53:45.955606+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509574,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83132,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288708,"statfs":{"total":96636764160,"available":96551636992,"internally_reserved":0,"allocated":2072576,"data_stored":1269313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509573,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104812,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94267028,"statfs":{"total":96636764160,"available":96529436672,"internally_reserved":0,"allocated":1466368,"data_stored":678751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509574,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72824,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299016,"statfs":{"total":96636764160,"available":96562192384,"internally_reserved":0,"allocated":1470464,"data_stored":678329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509574,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86708,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285132,"statfs":{"total":96636764160,"available":96547975168,"internally_reserved":0,"allocated":1474560,"data_stored":679781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509573,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105332,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266508,"statfs":{"total":96636764160,"available":96528904192,"internally_reserved":0,"allocated":2060288,"data_stored":1268559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542277,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118648,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253192,"statfs":{"total":96636764160,"available":96515268608,"internally_reserved":0,"allocated":1478656,"data_stored":679815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542277,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82604,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94289236,"statfs":{"total":96636764160,"available":96552177664,"internally_reserved":0,"allocated":1466368,"data_stored":678191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574982,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":108904,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262936,"statfs":{"total":96636764160,"available":96525246464,"internally_reserved":0,"allocated":1462272,"data_stored":678169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:53:52.504 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T07:53:52.504 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T07:53:52.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.647+0000 7f7b0ff42640 1 Processor -- start 2024-09-19T07:53:52.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.648+0000 7f7b0ff42640 1 -- start start 2024-09-19T07:53:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.648+0000 7f7b0ff42640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b080d4af0 0x7f7b08162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.648+0000 7f7b0ff42640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b080b4b70 0x7f7b080b5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.648+0000 7f7b0ff42640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b0815f250 con 0x7f7b0815e0a0 2024-09-19T07:53:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.648+0000 7f7b0ff42640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f7b080b5560 con 0x7f7b080d4af0 2024-09-19T07:53:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.648+0000 7f7b0ff42640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7b0815f530 con 0x7f7b080b4b70 2024-09-19T07:53:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.649+0000 7f7b0d4dd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b080d4af0 0x7f7b08162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.649+0000 7f7b0d4dd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b080d4af0 0x7f7b08162a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33616/0 (socket says 172.21.15.16:33616) 2024-09-19T07:53:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.649+0000 7f7b0d4dd640 1 -- 172.21.15.16:0/2252376346 learned_addr learned my addr 172.21.15.16:0/2252376346 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.649+0000 7f7b0e4df640 1 --2- 172.21.15.16:0/2252376346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b080b4b70 0x7f7b080b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.649+0000 7f7b0d4dd640 1 -- 172.21.15.16:0/2252376346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b080b4b70 msgr2=0x7f7b080b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.649+0000 7f7b0d4dd640 1 --2- 172.21.15.16:0/2252376346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b080b4b70 0x7f7b080b5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.649+0000 7f7b0d4dd640 1 -- 172.21.15.16:0/2252376346 >> v1:172.21.15.146:6789/0 conn(0x7f7b0815e0a0 legacy=0x7f7b080d43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.649+0000 7f7b0d4dd640 1 -- 172.21.15.16:0/2252376346 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0815f810 con 0x7f7b080d4af0 2024-09-19T07:53:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.650+0000 7f7b0d4dd640 1 --2- 172.21.15.16:0/2252376346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b080d4af0 0x7f7b08162a70 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f7af8009850 tx=0x7f7af802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e19514da7f8d89 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.650+0000 7f7b0ccdc640 1 -- 172.21.15.16:0/2252376346 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8004030 con 0x7f7b080d4af0 2024-09-19T07:53:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.650+0000 7f7b0ccdc640 1 -- 172.21.15.16:0/2252376346 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7af80041d0 con 0x7f7b080d4af0 2024-09-19T07:53:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.650+0000 7f7b0ccdc640 1 -- 172.21.15.16:0/2252376346 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8003610 con 0x7f7b080d4af0 2024-09-19T07:53:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.650+0000 7f7b0ff42640 1 -- 172.21.15.16:0/2252376346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b080d4af0 msgr2=0x7f7b08162a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.650+0000 7f7b0ff42640 1 --2- 172.21.15.16:0/2252376346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b080d4af0 0x7f7b08162a70 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f7af8009850 tx=0x7f7af802a230 comp rx=0 tx=0).stop 2024-09-19T07:53:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.650+0000 7f7b0ff42640 1 -- 172.21.15.16:0/2252376346 shutdown_connections 2024-09-19T07:53:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.650+0000 7f7b0ff42640 1 --2- 172.21.15.16:0/2252376346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b080b4b70 0x7f7b080b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.650+0000 7f7b0ff42640 1 --2- 172.21.15.16:0/2252376346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b080d4af0 0x7f7b08162a70 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.650+0000 7f7b0ff42640 1 -- 172.21.15.16:0/2252376346 >> 172.21.15.16:0/2252376346 conn(0x7f7b0809d770 msgr2=0x7f7b0808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.650+0000 7f7b0ff42640 1 -- 172.21.15.16:0/2252376346 shutdown_connections 2024-09-19T07:53:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.650+0000 7f7b0ff42640 1 -- 172.21.15.16:0/2252376346 wait complete. 2024-09-19T07:53:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.652+0000 7f7b0ff42640 1 Processor -- start 2024-09-19T07:53:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.652+0000 7f7b0ff42640 1 -- start start 2024-09-19T07:53:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.652+0000 7f7b0ff42640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b080b4b70 0x7f7b0815c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.652+0000 7f7b0ff42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b080d4af0 0x7f7b080d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:52.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.652+0000 7f7b0ff42640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b0815e0a0 0x7f7b080d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:52.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.652+0000 7f7b0ff42640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0815e600 con 0x7f7b080d4af0 2024-09-19T07:53:52.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.652+0000 7f7b0ff42640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0815e480 con 0x7f7b080b4b70 2024-09-19T07:53:52.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.652+0000 7f7b0ff42640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7b0815e780 con 0x7f7b0815e0a0 2024-09-19T07:53:52.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.652+0000 7f7b0d4dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b080d4af0 0x7f7b080d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:52.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.653+0000 7f7b0d4dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b080d4af0 0x7f7b080d2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33632/0 (socket says 172.21.15.16:33632) 2024-09-19T07:53:52.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.653+0000 7f7b0d4dd640 1 -- 172.21.15.16:0/97339269 learned_addr learned my addr 172.21.15.16:0/97339269 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:52.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.653+0000 7f7b0dcde640 1 --2- 172.21.15.16:0/97339269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b080b4b70 0x7f7b0815c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:52.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.653+0000 7f7b0e4df640 1 --2- 172.21.15.16:0/97339269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b0815e0a0 0x7f7b080d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:52.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.653+0000 7f7b0d4dd640 1 -- 172.21.15.16:0/97339269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b0815e0a0 msgr2=0x7f7b080d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.653+0000 7f7b0d4dd640 1 --2- 172.21.15.16:0/97339269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b0815e0a0 0x7f7b080d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.653+0000 7f7b0d4dd640 1 -- 172.21.15.16:0/97339269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b080b4b70 msgr2=0x7f7b0815c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.653+0000 7f7b0d4dd640 1 --2- 172.21.15.16:0/97339269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b080b4b70 0x7f7b0815c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.653+0000 7f7b0d4dd640 1 -- 172.21.15.16:0/97339269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b081f4720 con 0x7f7b080d4af0 2024-09-19T07:53:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.653+0000 7f7b0d4dd640 1 --2- 172.21.15.16:0/97339269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b080d4af0 0x7f7b080d2e90 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f7af802a270 tx=0x7f7af80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.653+0000 7f7afe7fc640 1 -- 172.21.15.16:0/97339269 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8041020 con 0x7f7b080d4af0 2024-09-19T07:53:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.653+0000 7f7afe7fc640 1 -- 172.21.15.16:0/97339269 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7af8003c50 con 0x7f7b080d4af0 2024-09-19T07:53:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.653+0000 7f7b0ff42640 1 -- 172.21.15.16:0/97339269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b081f49e0 con 0x7f7b080d4af0 2024-09-19T07:53:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.653+0000 7f7afe7fc640 1 -- 172.21.15.16:0/97339269 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8004030 con 0x7f7b080d4af0 2024-09-19T07:53:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.653+0000 7f7b0ff42640 1 -- 172.21.15.16:0/97339269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b081f4fa0 con 0x7f7b080d4af0 2024-09-19T07:53:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.654+0000 7f7b0ff42640 1 -- 172.21.15.16:0/97339269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b080b5020 con 0x7f7b080d4af0 2024-09-19T07:53:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.655+0000 7f7afe7fc640 1 -- 172.21.15.16:0/97339269 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7af8044020 con 0x7f7b080d4af0 2024-09-19T07:53:52.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.655+0000 7f7afe7fc640 1 --2- 172.21.15.16:0/97339269 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7aec06c2b0 0x7f7aec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:52.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.655+0000 7f7afe7fc640 1 -- 172.21.15.16:0/97339269 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f7af80ac060 con 0x7f7b080d4af0 2024-09-19T07:53:52.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.655+0000 7f7b0dcde640 1 --2- 172.21.15.16:0/97339269 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7aec06c2b0 0x7f7aec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:52.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.656+0000 7f7b0dcde640 1 --2- 172.21.15.16:0/97339269 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7aec06c2b0 0x7f7aec06e770 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7b040016e0 tx=0x7f7b04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:52.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.660+0000 7f7afe7fc640 1 -- 172.21.15.16:0/97339269 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7af804b030 con 0x7f7b080d4af0 2024-09-19T07:53:52.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.900+0000 7f7b0ff42640 1 -- 172.21.15.16:0/97339269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7b0806e060 con 0x7f7b080d4af0 2024-09-19T07:53:52.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.902+0000 7f7afe7fc640 1 -- 172.21.15.16:0/97339269 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f7af807a080 con 0x7f7b080d4af0 2024-09-19T07:53:52.902 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-19T07:53:52.902 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA== 2024-09-19T07:53:52.902 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-19T07:53:52.902 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-19T07:53:52.903 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T07:53:52.903 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-19T07:53:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.904+0000 7f7b0ff42640 1 -- 172.21.15.16:0/97339269 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7aec06c2b0 msgr2=0x7f7aec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.904+0000 7f7b0ff42640 1 --2- 172.21.15.16:0/97339269 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7aec06c2b0 0x7f7aec06e770 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7b040016e0 tx=0x7f7b04009040 comp rx=0 tx=0).stop 2024-09-19T07:53:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.904+0000 7f7b0ff42640 1 -- 172.21.15.16:0/97339269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b080d4af0 msgr2=0x7f7b080d2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.904+0000 7f7b0ff42640 1 --2- 172.21.15.16:0/97339269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b080d4af0 0x7f7b080d2e90 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f7af802a270 tx=0x7f7af80046c0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.904+0000 7f7b0ff42640 1 -- 172.21.15.16:0/97339269 shutdown_connections 2024-09-19T07:53:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.904+0000 7f7b0ff42640 1 --2- 172.21.15.16:0/97339269 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7aec06c2b0 0x7f7aec06e770 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.904+0000 7f7b0ff42640 1 --2- 172.21.15.16:0/97339269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b0815e0a0 0x7f7b080d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.904+0000 7f7b0ff42640 1 --2- 172.21.15.16:0/97339269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b080d4af0 0x7f7b080d2e90 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.904+0000 7f7b0ff42640 1 --2- 172.21.15.16:0/97339269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b080b4b70 0x7f7b0815c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.904+0000 7f7b0ff42640 1 -- 172.21.15.16:0/97339269 >> 172.21.15.16:0/97339269 conn(0x7f7b0809d770 msgr2=0x7f7b0806e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.904+0000 7f7b0ff42640 1 -- 172.21.15.16:0/97339269 shutdown_connections 2024-09-19T07:53:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:52.904+0000 7f7b0ff42640 1 -- 172.21.15.16:0/97339269 wait complete. 2024-09-19T07:53:52.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:53:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.018+0000 7ff3830dd640 1 Processor -- start 2024-09-19T07:53:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.018+0000 7ff3830dd640 1 -- start start 2024-09-19T07:53:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.018+0000 7ff3830dd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0d0070 0x7ff37c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff3830dd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff37c0d1b10 con 0x7ff37c15f0d0 2024-09-19T07:53:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff3830dd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff37c0cf2b0 con 0x7ff37c0d4130 2024-09-19T07:53:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff3830dd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff37c0cf590 con 0x7ff37c0d0070 2024-09-19T07:53:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff380e79640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff37c15f0d0 0x7ff37c0d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35180/0 (socket says 172.21.15.16:35180) 2024-09-19T07:53:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff380e79640 1 -- 172.21.15.16:0/2394715481 learned_addr learned my addr 172.21.15.16:0/2394715481 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff37b7fe640 1 -- 172.21.15.16:0/2394715481 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2986039877 0 0) 0x7ff37c0d1b10 con 0x7ff37c15f0d0 2024-09-19T07:53:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff37bfff640 1 --2- 172.21.15.16:0/2394715481 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0d0070 0x7ff37c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff37b7fe640 1 -- 172.21.15.16:0/2394715481 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff368003520 con 0x7ff37c15f0d0 2024-09-19T07:53:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff37b7fe640 1 -- 172.21.15.16:0/2394715481 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2120162402 0 0) 0x7ff37c0cf2b0 con 0x7ff37c0d4130 2024-09-19T07:53:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff37b7fe640 1 -- 172.21.15.16:0/2394715481 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff37c0d1b10 con 0x7ff37c0d4130 2024-09-19T07:53:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff37b7fe640 1 -- 172.21.15.16:0/2394715481 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 125228521 0 0) 0x7ff368003520 con 0x7ff37c15f0d0 2024-09-19T07:53:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff37b7fe640 1 -- 172.21.15.16:0/2394715481 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0d0070 msgr2=0x7ff37c15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff37b7fe640 1 --2- 172.21.15.16:0/2394715481 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0d0070 0x7ff37c15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff37b7fe640 1 -- 172.21.15.16:0/2394715481 >> v1:172.21.15.146:6789/0 conn(0x7ff37c0d4130 legacy=0x7ff37c0d45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff37b7fe640 1 -- 172.21.15.16:0/2394715481 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37c0cf870 con 0x7ff37c15f0d0 2024-09-19T07:53:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff37b7fe640 1 -- 172.21.15.16:0/2394715481 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7ff36c001670 con 0x7ff37c15f0d0 2024-09-19T07:53:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff37b7fe640 1 -- 172.21.15.16:0/2394715481 >> v1:172.21.15.16:6789/0 conn(0x7ff37c15f0d0 legacy=0x7ff37c0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff37b7fe640 1 --2- 172.21.15.16:0/2394715481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff368003b70 0x7ff368003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff37b7fe640 1 --2- 172.21.15.16:0/2394715481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff368004800 0x7ff368004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff37b7fe640 1 --2- 172.21.15.16:0/2394715481 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0d0070 0x7ff3680051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff37b7fe640 1 -- 172.21.15.16:0/2394715481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c0cf590 con 0x7ff368003b70 2024-09-19T07:53:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff37b7fe640 1 -- 172.21.15.16:0/2394715481 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff36800bb00 con 0x7ff368004800 2024-09-19T07:53:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff37b7fe640 1 -- 172.21.15.16:0/2394715481 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff36800bd00 con 0x7ff37c0d0070 2024-09-19T07:53:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.019+0000 7ff380e79640 1 --2- 172.21.15.16:0/2394715481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff368003b70 0x7ff368003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.020+0000 7ff37bfff640 1 --2- 172.21.15.16:0/2394715481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff368004800 0x7ff368004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.020+0000 7ff38167a640 1 --2- 172.21.15.16:0/2394715481 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0d0070 0x7ff3680051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.020+0000 7ff380e79640 1 -- 172.21.15.16:0/2394715481 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0d0070 msgr2=0x7ff3680051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.020+0000 7ff380e79640 1 --2- 172.21.15.16:0/2394715481 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0d0070 0x7ff3680051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.020+0000 7ff380e79640 1 -- 172.21.15.16:0/2394715481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff368004800 msgr2=0x7ff368004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.020+0000 7ff380e79640 1 --2- 172.21.15.16:0/2394715481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff368004800 0x7ff368004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.020+0000 7ff380e79640 1 -- 172.21.15.16:0/2394715481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff368003520 con 0x7ff368003b70 2024-09-19T07:53:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.020+0000 7ff380e79640 1 --2- 172.21.15.16:0/2394715481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff368003b70 0x7ff368003f70 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00c190 tx=0x7ff368005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c206cd8011233e83 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.021+0000 7ff37b7fe640 1 -- 172.21.15.16:0/2394715481 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c007ce0 con 0x7ff368003b70 2024-09-19T07:53:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.021+0000 7ff37b7fe640 1 -- 172.21.15.16:0/2394715481 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff36c005ad0 con 0x7ff368003b70 2024-09-19T07:53:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.021+0000 7ff37b7fe640 1 -- 172.21.15.16:0/2394715481 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c007070 con 0x7ff368003b70 2024-09-19T07:53:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.021+0000 7ff3830dd640 1 -- 172.21.15.16:0/2394715481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff368003b70 msgr2=0x7ff368003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.021+0000 7ff3830dd640 1 --2- 172.21.15.16:0/2394715481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff368003b70 0x7ff368003f70 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00c190 tx=0x7ff368005830 comp rx=0 tx=0).stop 2024-09-19T07:53:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.021+0000 7ff38167a640 1 -- 172.21.15.16:0/2394715481 reap_dead start 2024-09-19T07:53:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.021+0000 7ff3830dd640 1 -- 172.21.15.16:0/2394715481 shutdown_connections 2024-09-19T07:53:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.021+0000 7ff3830dd640 1 -- 172.21.15.16:0/2394715481 >> 172.21.15.16:0/2394715481 conn(0x7ff37c09d760 msgr2=0x7ff37c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.021+0000 7ff3830dd640 1 -- 172.21.15.16:0/2394715481 shutdown_connections 2024-09-19T07:53:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.021+0000 7ff3830dd640 1 -- 172.21.15.16:0/2394715481 wait complete. 2024-09-19T07:53:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.022+0000 7ff3830dd640 1 Processor -- start 2024-09-19T07:53:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.022+0000 7ff3830dd640 1 -- start start 2024-09-19T07:53:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.023+0000 7ff3830dd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff368003b70 0x7ff37c15a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.023+0000 7ff3830dd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0ce370 0x7ff37c0cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.023+0000 7ff3830dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff37c09eac0 0x7ff37c09ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.023+0000 7ff3830dd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c15f4b0 con 0x7ff37c09eac0 2024-09-19T07:53:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.023+0000 7ff3830dd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c0cef60 con 0x7ff368003b70 2024-09-19T07:53:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.023+0000 7ff3830dd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff37c15f630 con 0x7ff37c0ce370 2024-09-19T07:53:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.023+0000 7ff38167a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff37c09eac0 0x7ff37c09ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.023+0000 7ff38167a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff37c09eac0 0x7ff37c09ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33660/0 (socket says 172.21.15.16:33660) 2024-09-19T07:53:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.023+0000 7ff38167a640 1 -- 172.21.15.16:0/541937131 learned_addr learned my addr 172.21.15.16:0/541937131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.024+0000 7ff380e79640 1 --2- 172.21.15.16:0/541937131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff368003b70 0x7ff37c15a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.024+0000 7ff37bfff640 1 --2- 172.21.15.16:0/541937131 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0ce370 0x7ff37c0cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.024+0000 7ff38167a640 1 -- 172.21.15.16:0/541937131 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0ce370 msgr2=0x7ff37c0cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.024+0000 7ff38167a640 1 --2- 172.21.15.16:0/541937131 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0ce370 0x7ff37c0cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.024+0000 7ff38167a640 1 -- 172.21.15.16:0/541937131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff368003b70 msgr2=0x7ff37c15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.024+0000 7ff38167a640 1 --2- 172.21.15.16:0/541937131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff368003b70 0x7ff37c15a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.024+0000 7ff38167a640 1 -- 172.21.15.16:0/541937131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37c0ccb90 con 0x7ff37c09eac0 2024-09-19T07:53:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.024+0000 7ff380e79640 1 --2- 172.21.15.16:0/541937131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff368003b70 0x7ff37c15a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.024+0000 7ff37bfff640 1 --2- 172.21.15.16:0/541937131 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0ce370 0x7ff37c0cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.024+0000 7ff38167a640 1 --2- 172.21.15.16:0/541937131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff37c09eac0 0x7ff37c09ef70 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0ce9c0 tx=0x7ff37400e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.024+0000 7ff3797fa640 1 -- 172.21.15.16:0/541937131 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374005030 con 0x7ff37c09eac0 2024-09-19T07:53:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.024+0000 7ff3797fa640 1 -- 172.21.15.16:0/541937131 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3740051d0 con 0x7ff37c09eac0 2024-09-19T07:53:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.024+0000 7ff3830dd640 1 -- 172.21.15.16:0/541937131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff37c0a1d40 con 0x7ff37c09eac0 2024-09-19T07:53:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.024+0000 7ff3797fa640 1 -- 172.21.15.16:0/541937131 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3740069e0 con 0x7ff37c09eac0 2024-09-19T07:53:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.024+0000 7ff3830dd640 1 -- 172.21.15.16:0/541937131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff37c0cd890 con 0x7ff37c09eac0 2024-09-19T07:53:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.025+0000 7ff3830dd640 1 -- 172.21.15.16:0/541937131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff37c0d45c0 con 0x7ff37c09eac0 2024-09-19T07:53:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.026+0000 7ff3797fa640 1 -- 172.21.15.16:0/541937131 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff374006b80 con 0x7ff37c09eac0 2024-09-19T07:53:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.026+0000 7ff3797fa640 1 --2- 172.21.15.16:0/541937131 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff35c06c2e0 0x7ff35c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.026+0000 7ff3797fa640 1 -- 172.21.15.16:0/541937131 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7ff37408aef0 con 0x7ff37c09eac0 2024-09-19T07:53:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.026+0000 7ff380e79640 1 --2- 172.21.15.16:0/541937131 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff35c06c2e0 0x7ff35c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.027+0000 7ff380e79640 1 --2- 172.21.15.16:0/541937131 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff35c06c2e0 0x7ff35c06e7a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff36c032000 tx=0x7ff36c009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.030+0000 7ff3797fa640 1 -- 172.21.15.16:0/541937131 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff37401b060 con 0x7ff37c09eac0 2024-09-19T07:53:53.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.241+0000 7ff3830dd640 1 -- 172.21.15.16:0/541937131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff37c06cf30 con 0x7ff37c09eac0 2024-09-19T07:53:53.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.241+0000 7ff3797fa640 1 -- 172.21.15.16:0/541937131 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+5231 (secure 0 0 0) 0x7ff374059430 con 0x7ff37c09eac0 2024-09-19T07:53:53.242 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:53.242 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":63,"btime":"2024-09-19T07:53:46:466514+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2471408352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2471408352},{"type":"v1","addr":"172.21.15.16:6833","nonce":2471408352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/1281369866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1281369866},{"type":"v1","addr":"172.21.15.146:6836","nonce":1281369866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1377303883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1377303883},{"type":"v1","addr":"172.21.15.146:6837","nonce":1377303883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:53:43.812066+0000","modified":"2024-09-19T07:53:45.809688+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7368":{"gid":7368,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2050126415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2050126415},{"type":"v1","addr":"172.21.15.16:6835","nonce":2050126415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7368,"qdb_cluster":[7368]},"id":8}]} 2024-09-19T07:53:53.242 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 63 2024-09-19T07:53:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.243+0000 7ff3830dd640 1 -- 172.21.15.16:0/541937131 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff35c06c2e0 msgr2=0x7ff35c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.243+0000 7ff3830dd640 1 --2- 172.21.15.16:0/541937131 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff35c06c2e0 0x7ff35c06e7a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff36c032000 tx=0x7ff36c009030 comp rx=0 tx=0).stop 2024-09-19T07:53:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.243+0000 7ff3830dd640 1 -- 172.21.15.16:0/541937131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff37c09eac0 msgr2=0x7ff37c09ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.243+0000 7ff3830dd640 1 --2- 172.21.15.16:0/541937131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff37c09eac0 0x7ff37c09ef70 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0ce9c0 tx=0x7ff37400e820 comp rx=0 tx=0).stop 2024-09-19T07:53:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.243+0000 7ff3830dd640 1 -- 172.21.15.16:0/541937131 shutdown_connections 2024-09-19T07:53:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.244+0000 7ff3830dd640 1 --2- 172.21.15.16:0/541937131 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff35c06c2e0 0x7ff35c06e7a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.244+0000 7ff3830dd640 1 --2- 172.21.15.16:0/541937131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff37c09eac0 0x7ff37c09ef70 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.244+0000 7ff3830dd640 1 --2- 172.21.15.16:0/541937131 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff37c0ce370 0x7ff37c0cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.244+0000 7ff3830dd640 1 --2- 172.21.15.16:0/541937131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff368003b70 0x7ff37c15a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.244+0000 7ff3830dd640 1 -- 172.21.15.16:0/541937131 >> 172.21.15.16:0/541937131 conn(0x7ff37c09d760 msgr2=0x7ff37c15b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:53.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.244+0000 7ff3830dd640 1 -- 172.21.15.16:0/541937131 shutdown_connections 2024-09-19T07:53:53.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.244+0000 7ff3830dd640 1 -- 172.21.15.16:0/541937131 wait complete. 2024-09-19T07:53:53.657 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T07:53:43.812066+0000', 'modified': '2024-09-19T07:53:45.809688+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7368}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7368': {'gid': 7368, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6835/2050126415', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 2050126415}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 2050126415}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7368, 'qdb_cluster': [7368]} 2024-09-19T07:53:53.658 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T07:53:53.658 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T07:53:53.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.764+0000 7f26c4f86640 1 Processor -- start 2024-09-19T07:53:53.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.764+0000 7f26c4f86640 1 -- start start 2024-09-19T07:53:53.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.765+0000 7f26c4f86640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f26c006f9e0 0x7f26c00d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.765+0000 7f26c4f86640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26c00d09f0 0x7f26c00c5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.765+0000 7f26c4f86640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26c015b500 con 0x7f26c015b120 2024-09-19T07:53:53.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.765+0000 7f26c4f86640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f26c00d0e40 con 0x7f26c006f9e0 2024-09-19T07:53:53.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.765+0000 7f26c4f86640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f26c006e940 con 0x7f26c00d09f0 2024-09-19T07:53:53.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.766+0000 7f26bdd9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f26c006f9e0 0x7f26c00d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.766+0000 7f26be59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f26c015b120 0x7f26c006dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35186/0 (socket says 172.21.15.16:35186) 2024-09-19T07:53:53.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.766+0000 7f26be59c640 1 -- 172.21.15.16:0/2262217983 learned_addr learned my addr 172.21.15.16:0/2262217983 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:53.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.766+0000 7f26bed9d640 1 --2- 172.21.15.16:0/2262217983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26c00d09f0 0x7f26c00c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.766+0000 7f26bdd9b640 1 -- 172.21.15.16:0/2262217983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26c00d09f0 msgr2=0x7f26c00c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:53.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.766+0000 7f26bdd9b640 1 --2- 172.21.15.16:0/2262217983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26c00d09f0 0x7f26c00c5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.766+0000 7f26bdd9b640 1 -- 172.21.15.16:0/2262217983 >> v1:172.21.15.16:6789/0 conn(0x7f26c015b120 legacy=0x7f26c006dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.766+0000 7f26bdd9b640 1 -- 172.21.15.16:0/2262217983 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c006ec20 con 0x7f26c006f9e0 2024-09-19T07:53:53.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.767+0000 7f26bdd9b640 1 --2- 172.21.15.16:0/2262217983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c006f9e0 0x7f26c00d04b0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f26b40097a0 tx=0x7f26b402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32346248d1707ae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:53.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.767+0000 7f26bd59a640 1 -- 172.21.15.16:0/2262217983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b4037070 con 0x7f26c006f9e0 2024-09-19T07:53:53.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.767+0000 7f26bd59a640 1 -- 172.21.15.16:0/2262217983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b4032440 con 0x7f26c006f9e0 2024-09-19T07:53:53.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.767+0000 7f26bd59a640 1 -- 172.21.15.16:0/2262217983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b4032810 con 0x7f26c006f9e0 2024-09-19T07:53:53.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.767+0000 7f26c4f86640 1 -- 172.21.15.16:0/2262217983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c006f9e0 msgr2=0x7f26c00d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.767+0000 7f26c4f86640 1 --2- 172.21.15.16:0/2262217983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c006f9e0 0x7f26c00d04b0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f26b40097a0 tx=0x7f26b402a180 comp rx=0 tx=0).stop 2024-09-19T07:53:53.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.767+0000 7f26c4f86640 1 -- 172.21.15.16:0/2262217983 shutdown_connections 2024-09-19T07:53:53.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.767+0000 7f26c4f86640 1 --2- 172.21.15.16:0/2262217983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26c00d09f0 0x7f26c00c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.767+0000 7f26c4f86640 1 --2- 172.21.15.16:0/2262217983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c006f9e0 0x7f26c00d04b0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.767+0000 7f26c4f86640 1 -- 172.21.15.16:0/2262217983 >> 172.21.15.16:0/2262217983 conn(0x7f26c006bdf0 msgr2=0x7f26c006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:53.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.767+0000 7f26c4f86640 1 -- 172.21.15.16:0/2262217983 shutdown_connections 2024-09-19T07:53:53.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.767+0000 7f26c4f86640 1 -- 172.21.15.16:0/2262217983 wait complete. 2024-09-19T07:53:53.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.769+0000 7f26c4f86640 1 Processor -- start 2024-09-19T07:53:53.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.769+0000 7f26c4f86640 1 -- start start 2024-09-19T07:53:53.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.769+0000 7f26c4f86640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26c006f9e0 0x7f26c0157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26c4f86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c00d09f0 0x7f26c0073450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26c4f86640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26c015b120 0x7f26c00996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26c4f86640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26c006f4c0 con 0x7f26c00d09f0 2024-09-19T07:53:53.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26c4f86640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f26c006f1e0 con 0x7f26c015b120 2024-09-19T07:53:53.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26c4f86640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f26c006e5d0 con 0x7f26c006f9e0 2024-09-19T07:53:53.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26bdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c00d09f0 0x7f26c0073450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26bdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c00d09f0 0x7f26c0073450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33672/0 (socket says 172.21.15.16:33672) 2024-09-19T07:53:53.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26bdd9b640 1 -- 172.21.15.16:0/1258497075 learned_addr learned my addr 172.21.15.16:0/1258497075 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:53.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26be59c640 1 --2- 172.21.15.16:0/1258497075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26c006f9e0 0x7f26c0157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26bed9d640 1 --2- 172.21.15.16:0/1258497075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26c015b120 0x7f26c00996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26bdd9b640 1 -- 172.21.15.16:0/1258497075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26c006f9e0 msgr2=0x7f26c0157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26bdd9b640 1 --2- 172.21.15.16:0/1258497075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26c006f9e0 0x7f26c0157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26bdd9b640 1 -- 172.21.15.16:0/1258497075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26c015b120 msgr2=0x7f26c00996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26bdd9b640 1 --2- 172.21.15.16:0/1258497075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26c015b120 0x7f26c00996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26bdd9b640 1 -- 172.21.15.16:0/1258497075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c00723b0 con 0x7f26c00d09f0 2024-09-19T07:53:53.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26be59c640 1 --2- 172.21.15.16:0/1258497075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26c006f9e0 0x7f26c0157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:53.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26bed9d640 1 --2- 172.21.15.16:0/1258497075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26c015b120 0x7f26c00996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26bdd9b640 1 --2- 172.21.15.16:0/1258497075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c00d09f0 0x7f26c0073450 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f26b4035040 tx=0x7f26b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26aeffd640 1 -- 172.21.15.16:0/1258497075 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b4037040 con 0x7f26c00d09f0 2024-09-19T07:53:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26aeffd640 1 -- 172.21.15.16:0/1258497075 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b4030770 con 0x7f26c00d09f0 2024-09-19T07:53:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26aeffd640 1 -- 172.21.15.16:0/1258497075 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b4030b40 con 0x7f26c00d09f0 2024-09-19T07:53:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26c4f86640 1 -- 172.21.15.16:0/1258497075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c0072690 con 0x7f26c00d09f0 2024-09-19T07:53:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.770+0000 7f26c4f86640 1 -- 172.21.15.16:0/1258497075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c0072c50 con 0x7f26c00d09f0 2024-09-19T07:53:53.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.772+0000 7f26aeffd640 1 -- 172.21.15.16:0/1258497075 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f26b4044020 con 0x7f26c00d09f0 2024-09-19T07:53:53.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.772+0000 7f26aeffd640 1 --2- 172.21.15.16:0/1258497075 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f26a006c2b0 0x7f26a006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.773+0000 7f26aeffd640 1 -- 172.21.15.16:0/1258497075 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f26b40abbe0 con 0x7f26c00d09f0 2024-09-19T07:53:53.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.773+0000 7f26be59c640 1 --2- 172.21.15.16:0/1258497075 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f26a006c2b0 0x7f26a006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.773+0000 7f26c7dfb740 1 Processor -- start 2024-09-19T07:53:53.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.774+0000 7f26be59c640 1 --2- 172.21.15.16:0/1258497075 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f26a006c2b0 0x7f26a006e770 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f26b0000f60 tx=0x7f26b0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:53.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.774+0000 7f26c7dfb740 1 -- start start 2024-09-19T07:53:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.774+0000 7f26c7dfb740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 0x5603cf994a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.774+0000 7f26c7dfb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf94ca80 0x5603cf94f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.774+0000 7f26c7dfb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf950a50 0x5603cf9498b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.774+0000 7f26c7dfb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5603cf8b1520 con 0x5603cf950a50 2024-09-19T07:53:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.774+0000 7f26c7dfb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5603cf8593e0 con 0x5603cf94ca80 2024-09-19T07:53:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.774+0000 7f26c7dfb740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x5603cf8ad4e0 con 0x5603cf94f2c0 2024-09-19T07:53:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.774+0000 7f26bed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf950a50 0x5603cf9498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.775+0000 7f26bed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf950a50 0x5603cf9498b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33676/0 (socket says 172.21.15.16:33676) 2024-09-19T07:53:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.775+0000 7f26bed9d640 1 -- 172.21.15.16:0/1643396374 learned_addr learned my addr 172.21.15.16:0/1643396374 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.775+0000 7f26be59c640 1 --2- 172.21.15.16:0/1643396374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 0x5603cf994a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.775+0000 7f26bdd9b640 1 --2- 172.21.15.16:0/1643396374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf94ca80 0x5603cf94f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.775+0000 7f26bed9d640 1 -- 172.21.15.16:0/1643396374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 msgr2=0x5603cf994a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.775+0000 7f26bed9d640 1 --2- 172.21.15.16:0/1643396374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 0x5603cf994a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.775+0000 7f26bed9d640 1 -- 172.21.15.16:0/1643396374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf94ca80 msgr2=0x5603cf94f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.775+0000 7f26bed9d640 1 --2- 172.21.15.16:0/1643396374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf94ca80 0x5603cf94f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.775+0000 7f26bed9d640 1 -- 172.21.15.16:0/1643396374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603cf8742f0 con 0x5603cf950a50 2024-09-19T07:53:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.775+0000 7f26bed9d640 1 --2- 172.21.15.16:0/1643396374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf950a50 0x5603cf9498b0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f26b800e4f0 tx=0x7f26b800e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999699757611e58d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.775+0000 7f269ffff640 1 -- 172.21.15.16:0/1643396374 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8003da0 con 0x5603cf950a50 2024-09-19T07:53:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.775+0000 7f269ffff640 1 -- 172.21.15.16:0/1643396374 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b8013070 con 0x5603cf950a50 2024-09-19T07:53:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.775+0000 7f269ffff640 1 -- 172.21.15.16:0/1643396374 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8004230 con 0x5603cf950a50 2024-09-19T07:53:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.776+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1643396374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf950a50 msgr2=0x5603cf9498b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.776+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/1643396374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf950a50 0x5603cf9498b0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f26b800e4f0 tx=0x7f26b800e9b0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.776+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1643396374 shutdown_connections 2024-09-19T07:53:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.776+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/1643396374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf950a50 0x5603cf9498b0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.776+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/1643396374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf94ca80 0x5603cf94f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.776+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/1643396374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 0x5603cf994a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.776+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1643396374 >> 172.21.15.16:0/1643396374 conn(0x5603cf94ba80 msgr2=0x5603cf993f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.776+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1643396374 shutdown_connections 2024-09-19T07:53:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.776+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1643396374 wait complete. 2024-09-19T07:53:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.776+0000 7f26c7dfb740 1 -- start start 2024-09-19T07:53:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.777+0000 7f26c7dfb740 1 Processor -- start 2024-09-19T07:53:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.777+0000 7f26c7dfb740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 0x5603cf973f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.777+0000 7f26c7dfb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf950a50 0x5603cf97c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.777+0000 7f26c7dfb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf976ca0 0x5603cf97cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.777+0000 7f26c7dfb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5603cf94af20 con 0x5603cf976ca0 2024-09-19T07:53:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.777+0000 7f26c7dfb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5603cf83a450 con 0x5603cf950a50 2024-09-19T07:53:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.777+0000 7f26c7dfb740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x5603cf995290 con 0x5603cf94f2c0 2024-09-19T07:53:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.777+0000 7f26c7dfb740 10 client.0 _ll_register_callbacks cb 0x5603cf5c3ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.778+0000 7f26bed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf976ca0 0x5603cf97cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.778+0000 7f26bed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf976ca0 0x5603cf97cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33678/0 (socket says 172.21.15.16:33678) 2024-09-19T07:53:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.778+0000 7f26bed9d640 1 -- 172.21.15.16:0/1964548125 learned_addr learned my addr 172.21.15.16:0/1964548125 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.778+0000 7f26bdd9b640 1 --2- 172.21.15.16:0/1964548125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf950a50 0x5603cf97c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.778+0000 7f26be59c640 1 --2- 172.21.15.16:0/1964548125 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 0x5603cf973f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.778+0000 7f26bed9d640 1 -- 172.21.15.16:0/1964548125 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 msgr2=0x5603cf973f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.778+0000 7f26bed9d640 1 --2- 172.21.15.16:0/1964548125 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 0x5603cf973f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.778+0000 7f26bed9d640 1 -- 172.21.15.16:0/1964548125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf950a50 msgr2=0x5603cf97c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.778+0000 7f26bed9d640 1 --2- 172.21.15.16:0/1964548125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf950a50 0x5603cf97c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.778+0000 7f26bed9d640 1 -- 172.21.15.16:0/1964548125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603cf5c42d0 con 0x5603cf976ca0 2024-09-19T07:53:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.778+0000 7f26bed9d640 1 --2- 172.21.15.16:0/1964548125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf976ca0 0x5603cf97cc50 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f26b8007e30 tx=0x7f26b8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc4de2f2e9bfe66 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.778+0000 7f269d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:53:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.778+0000 7f269d7fa640 1 -- 172.21.15.16:0/1964548125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8029060 con 0x5603cf976ca0 2024-09-19T07:53:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.778+0000 7f269d7fa640 1 -- 172.21.15.16:0/1964548125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f26b8003db0 con 0x5603cf976ca0 2024-09-19T07:53:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.778+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1964548125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26b8013070 con 0x5603cf976ca0 2024-09-19T07:53:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.778+0000 7f269d7fa640 1 -- 172.21.15.16:0/1964548125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f26b802f050 con 0x5603cf976ca0 2024-09-19T07:53:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.778+0000 7f26c7dfb740 10 client.7536 fetch_fsmap learned FSMap version 63 2024-09-19T07:53:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.778+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1964548125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) -- 0x5603cf68e6b0 con 0x5603cf976ca0 2024-09-19T07:53:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.779+0000 7f269d7fa640 1 -- 172.21.15.16:0/1964548125 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b8013070 con 0x5603cf976ca0 2024-09-19T07:53:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.779+0000 7f269d7fa640 1 -- 172.21.15.16:0/1964548125 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 63) ==== 3502+0+0 (secure 0 0 0) 0x7f26b8032020 con 0x5603cf976ca0 2024-09-19T07:53:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.779+0000 7f26c7dfb740 10 client.7536 fetch_fsmap finished waiting for FSMap version 63 2024-09-19T07:53:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.779+0000 7f26c7dfb740 10 client.7536 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T07:53:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.779+0000 7f26c7dfb740 20 client.7536 populate_metadata read hostname 'smithi016' 2024-09-19T07:53:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.779+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/1964548125 >> [v2:172.21.15.16:6834/2050126415,v1:172.21.15.16:6835/2050126415] conn(0x5603cf97a2e0 0x5603cf97a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.779+0000 7f26c7dfb740 4 client.7536 mds_command: new command op to 7368 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:53:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.779+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1964548125 --> [v2:172.21.15.16:6834/2050126415,v1:172.21.15.16:6835/2050126415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603cf8bec50 con 0x5603cf97a2e0 2024-09-19T07:53:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.779+0000 7f26be59c640 1 --2- 172.21.15.16:0/1964548125 >> [v2:172.21.15.16:6834/2050126415,v1:172.21.15.16:6835/2050126415] conn(0x5603cf97a2e0 0x5603cf97a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.779+0000 7f269d7fa640 10 client.7536 ms_handle_connect on v2:172.21.15.16:6834/2050126415 2024-09-19T07:53:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.779+0000 7f26be59c640 1 --2- 172.21.15.16:0/1964548125 >> [v2:172.21.15.16:6834/2050126415,v1:172.21.15.16:6835/2050126415] conn(0x5603cf97a2e0 0x5603cf97a6c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4e6bf393e9027d0 server_cookie=9cf6c8b9280f5eac in_seq=0 out_seq=0 2024-09-19T07:53:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.780+0000 7f269d7fa640 1 -- 172.21.15.16:0/1964548125 <== mds.0 v2:172.21.15.16:6834/2050126415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5603cf8bec50 con 0x5603cf97a2e0 2024-09-19T07:53:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.780+0000 7f269d7fa640 10 client.7536 handle_command_reply: tid=0 2024-09-19T07:53:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.780+0000 7f26c7dfb740 1 client.7536 shutdown 2024-09-19T07:53:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.780+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1964548125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf976ca0 msgr2=0x5603cf97cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.780+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/1964548125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf976ca0 0x5603cf97cc50 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f26b8007e30 tx=0x7f26b8004ab0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.780+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1964548125 shutdown_connections 2024-09-19T07:53:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.780+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/1964548125 >> [v2:172.21.15.16:6834/2050126415,v1:172.21.15.16:6835/2050126415] conn(0x5603cf97a2e0 0x5603cf97a6c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.780+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/1964548125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf976ca0 0x5603cf97cc50 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.780+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/1964548125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf950a50 0x5603cf97c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.780+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/1964548125 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 0x5603cf973f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.780+0000 7f269d7fa640 0 client.7536 ms_handle_reset on v2:172.21.15.16:6834/2050126415 2024-09-19T07:53:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.780+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1964548125 >> 172.21.15.16:0/1964548125 conn(0x5603cf94ca80 msgr2=0x5603cf9935a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.780+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1964548125 shutdown_connections 2024-09-19T07:53:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.780+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1964548125 wait complete. 2024-09-19T07:53:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.780+0000 7f26c7dfb740 20 client.7536 trim_cache size 0 max 16384 2024-09-19T07:53:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.798+0000 7f26c7dfb740 1 Processor -- start 2024-09-19T07:53:53.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.798+0000 7f26c7dfb740 1 -- start start 2024-09-19T07:53:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.799+0000 7f26c7dfb740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 0x5603cf9921a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.799+0000 7f26c7dfb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf950a50 0x5603cf993f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.799+0000 7f26c7dfb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf976ca0 0x5603cf9b6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.799+0000 7f26c7dfb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26c02000f0 con 0x5603cf976ca0 2024-09-19T07:53:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.799+0000 7f26c7dfb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f26c00c6bd0 con 0x5603cf950a50 2024-09-19T07:53:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.799+0000 7f26c7dfb740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f26c01f5350 con 0x5603cf94f2c0 2024-09-19T07:53:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.799+0000 7f26bed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf976ca0 0x5603cf9b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.799+0000 7f26bed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf976ca0 0x5603cf9b6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33690/0 (socket says 172.21.15.16:33690) 2024-09-19T07:53:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.799+0000 7f26bed9d640 1 -- 172.21.15.16:0/3536078515 learned_addr learned my addr 172.21.15.16:0/3536078515 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.799+0000 7f26be59c640 1 --2- 172.21.15.16:0/3536078515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 0x5603cf9921a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.799+0000 7f26bdd9b640 1 --2- 172.21.15.16:0/3536078515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf950a50 0x5603cf993f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.799+0000 7f26bed9d640 1 -- 172.21.15.16:0/3536078515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 msgr2=0x5603cf9921a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.799+0000 7f26bed9d640 1 --2- 172.21.15.16:0/3536078515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 0x5603cf9921a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.799+0000 7f26bed9d640 1 -- 172.21.15.16:0/3536078515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf950a50 msgr2=0x5603cf993f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.799+0000 7f26bed9d640 1 --2- 172.21.15.16:0/3536078515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf950a50 0x5603cf993f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.799+0000 7f26bed9d640 1 -- 172.21.15.16:0/3536078515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603cf9742d0 con 0x5603cf976ca0 2024-09-19T07:53:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.799+0000 7f26bed9d640 1 --2- 172.21.15.16:0/3536078515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf976ca0 0x5603cf9b6670 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f26b800f010 tx=0x7f26b801d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95fda16624ea3ee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.800+0000 7f269f7fe640 1 -- 172.21.15.16:0/3536078515 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8004370 con 0x5603cf976ca0 2024-09-19T07:53:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.800+0000 7f269f7fe640 1 -- 172.21.15.16:0/3536078515 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b8013070 con 0x5603cf976ca0 2024-09-19T07:53:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.800+0000 7f269f7fe640 1 -- 172.21.15.16:0/3536078515 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8004550 con 0x5603cf976ca0 2024-09-19T07:53:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.800+0000 7f26c7dfb740 1 -- 172.21.15.16:0/3536078515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf976ca0 msgr2=0x5603cf9b6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.800+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/3536078515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf976ca0 0x5603cf9b6670 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f26b800f010 tx=0x7f26b801d070 comp rx=0 tx=0).stop 2024-09-19T07:53:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.800+0000 7f26c7dfb740 1 -- 172.21.15.16:0/3536078515 shutdown_connections 2024-09-19T07:53:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.800+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/3536078515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf976ca0 0x5603cf9b6670 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.800+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/3536078515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf950a50 0x5603cf993f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.800+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/3536078515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 0x5603cf9921a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.800+0000 7f26c7dfb740 1 -- 172.21.15.16:0/3536078515 >> 172.21.15.16:0/3536078515 conn(0x5603cf94ca80 msgr2=0x5603cf995cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:53.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.800+0000 7f26c7dfb740 1 -- 172.21.15.16:0/3536078515 shutdown_connections 2024-09-19T07:53:53.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.800+0000 7f26c7dfb740 1 -- 172.21.15.16:0/3536078515 wait complete. 2024-09-19T07:53:53.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.801+0000 7f26c7dfb740 1 -- start start 2024-09-19T07:53:53.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.801+0000 7f26c7dfb740 1 Processor -- start 2024-09-19T07:53:53.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26c7dfb740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 0x5603cfa0ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26c7dfb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf950a50 0x5603cfa118a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26c7dfb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf976ca0 0x5603cfa11de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26c7dfb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5603cf9744a0 con 0x5603cf950a50 2024-09-19T07:53:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26c7dfb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5603cf9a4ad0 con 0x5603cf976ca0 2024-09-19T07:53:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26c7dfb740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x5603cf9a3e00 con 0x5603cf94f2c0 2024-09-19T07:53:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26c7dfb740 10 client.0 _ll_register_callbacks cb 0x7f26b4009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26bdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf950a50 0x5603cfa118a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26bdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf950a50 0x5603cfa118a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33702/0 (socket says 172.21.15.16:33702) 2024-09-19T07:53:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26bdd9b640 1 -- 172.21.15.16:0/1167854186 learned_addr learned my addr 172.21.15.16:0/1167854186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26bed9d640 1 --2- 172.21.15.16:0/1167854186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf976ca0 0x5603cfa11de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26be59c640 1 --2- 172.21.15.16:0/1167854186 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 0x5603cfa0ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26bdd9b640 1 -- 172.21.15.16:0/1167854186 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 msgr2=0x5603cfa0ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26bdd9b640 1 --2- 172.21.15.16:0/1167854186 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 0x5603cfa0ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26bdd9b640 1 -- 172.21.15.16:0/1167854186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf976ca0 msgr2=0x5603cfa11de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26bdd9b640 1 --2- 172.21.15.16:0/1167854186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf976ca0 0x5603cfa11de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26bdd9b640 1 -- 172.21.15.16:0/1167854186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603cf971370 con 0x5603cf950a50 2024-09-19T07:53:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26be59c640 1 --2- 172.21.15.16:0/1167854186 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 0x5603cfa0ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26bdd9b640 1 --2- 172.21.15.16:0/1167854186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf950a50 0x5603cfa118a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f26b4075f10 tx=0x7f26b40094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db1dc31b91ac0d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f269d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:53:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f269d7fa640 1 -- 172.21.15.16:0/1167854186 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b40a7020 con 0x5603cf950a50 2024-09-19T07:53:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f269d7fa640 1 -- 172.21.15.16:0/1167854186 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f26b4076120 con 0x5603cf950a50 2024-09-19T07:53:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.802+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1167854186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26b8013070 con 0x5603cf950a50 2024-09-19T07:53:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.803+0000 7f269d7fa640 1 -- 172.21.15.16:0/1167854186 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f26b4082690 con 0x5603cf950a50 2024-09-19T07:53:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.803+0000 7f26c7dfb740 10 client.7542 fetch_fsmap learned FSMap version 63 2024-09-19T07:53:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.803+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1167854186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) -- 0x5603cfa126d0 con 0x5603cf950a50 2024-09-19T07:53:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.803+0000 7f269d7fa640 1 -- 172.21.15.16:0/1167854186 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b8013070 con 0x5603cf950a50 2024-09-19T07:53:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.803+0000 7f269d7fa640 1 -- 172.21.15.16:0/1167854186 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 63) ==== 3502+0+0 (secure 0 0 0) 0x7f26b403a030 con 0x5603cf950a50 2024-09-19T07:53:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.803+0000 7f26c7dfb740 10 client.7542 fetch_fsmap finished waiting for FSMap version 63 2024-09-19T07:53:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.803+0000 7f26c7dfb740 10 client.7542 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T07:53:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.803+0000 7f26c7dfb740 20 client.7542 populate_metadata read hostname 'smithi016' 2024-09-19T07:53:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.803+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/1167854186 >> [v2:172.21.15.16:6834/2050126415,v1:172.21.15.16:6835/2050126415] conn(0x5603cf9a2fa0 0x5603cf997440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.803+0000 7f26c7dfb740 4 client.7542 mds_command: new command op to 7368 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:53:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.803+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1167854186 --> [v2:172.21.15.16:6834/2050126415,v1:172.21.15.16:6835/2050126415] -- command(tid 0: {"prefix": "status"}) -- 0x5603cf8b8fa0 con 0x5603cf9a2fa0 2024-09-19T07:53:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.803+0000 7f26be59c640 1 --2- 172.21.15.16:0/1167854186 >> [v2:172.21.15.16:6834/2050126415,v1:172.21.15.16:6835/2050126415] conn(0x5603cf9a2fa0 0x5603cf997440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.803+0000 7f269d7fa640 10 client.7542 ms_handle_connect on v2:172.21.15.16:6834/2050126415 2024-09-19T07:53:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.803+0000 7f26be59c640 1 --2- 172.21.15.16:0/1167854186 >> [v2:172.21.15.16:6834/2050126415,v1:172.21.15.16:6835/2050126415] conn(0x5603cf9a2fa0 0x5603cf997440 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2149c6b5e002f668 server_cookie=b1c47786ebf7658f in_seq=0 out_seq=0 2024-09-19T07:53:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f269d7fa640 1 -- 172.21.15.16:0/1167854186 <== mds.0 v2:172.21.15.16:6834/2050126415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5603cf8b8fa0 con 0x5603cf9a2fa0 2024-09-19T07:53:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f269d7fa640 10 client.7542 handle_command_reply: tid=0 2024-09-19T07:53:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c7dfb740 1 client.7542 shutdown 2024-09-19T07:53:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1167854186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf950a50 msgr2=0x5603cfa118a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/1167854186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf950a50 0x5603cfa118a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f26b4075f10 tx=0x7f26b40094b0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1167854186 shutdown_connections 2024-09-19T07:53:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/1167854186 >> [v2:172.21.15.16:6834/2050126415,v1:172.21.15.16:6835/2050126415] conn(0x5603cf9a2fa0 0x5603cf997440 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/1167854186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5603cf976ca0 0x5603cfa11de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/1167854186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5603cf950a50 0x5603cfa118a0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c7dfb740 1 --2- 172.21.15.16:0/1167854186 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5603cf94f2c0 0x5603cfa0ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f269d7fa640 0 client.7542 ms_handle_reset on v2:172.21.15.16:6834/2050126415 2024-09-19T07:53:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1167854186 >> 172.21.15.16:0/1167854186 conn(0x5603cf94ca80 msgr2=0x5603cf944ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1167854186 shutdown_connections 2024-09-19T07:53:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c7dfb740 1 -- 172.21.15.16:0/1167854186 wait complete. 2024-09-19T07:53:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c7dfb740 20 client.7542 trim_cache size 0 max 16384 2024-09-19T07:53:53.811 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:53:53.811 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb", 2024-09-19T07:53:53.811 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-19T07:53:53.811 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7368, 2024-09-19T07:53:53.811 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T07:53:53.811 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T07:53:53.811 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T07:53:53.811 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 9.3570201490000002, 2024-09-19T07:53:53.811 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 63, 2024-09-19T07:53:53.811 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 118, 2024-09-19T07:53:53.811 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 118, 2024-09-19T07:53:53.811 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 14.699702875 2024-09-19T07:53:53.811 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:53:53.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c4f86640 1 -- 172.21.15.16:0/1258497075 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f26a006c2b0 msgr2=0x7f26a006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c4f86640 1 --2- 172.21.15.16:0/1258497075 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f26a006c2b0 0x7f26a006e770 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f26b0000f60 tx=0x7f26b0009040 comp rx=0 tx=0).stop 2024-09-19T07:53:53.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c4f86640 1 -- 172.21.15.16:0/1258497075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c00d09f0 msgr2=0x7f26c0073450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c4f86640 1 --2- 172.21.15.16:0/1258497075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c00d09f0 0x7f26c0073450 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f26b4035040 tx=0x7f26b40046c0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c4f86640 1 -- 172.21.15.16:0/1258497075 shutdown_connections 2024-09-19T07:53:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c4f86640 1 --2- 172.21.15.16:0/1258497075 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f26a006c2b0 0x7f26a006e770 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c4f86640 1 --2- 172.21.15.16:0/1258497075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26c015b120 0x7f26c00996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c4f86640 1 --2- 172.21.15.16:0/1258497075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c00d09f0 0x7f26c0073450 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.804+0000 7f26c4f86640 1 --2- 172.21.15.16:0/1258497075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26c006f9e0 0x7f26c0157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.805+0000 7f26c4f86640 1 -- 172.21.15.16:0/1258497075 >> 172.21.15.16:0/1258497075 conn(0x7f26c006bdf0 msgr2=0x7f26c00caa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.805+0000 7f26c4f86640 1 -- 172.21.15.16:0/1258497075 shutdown_connections 2024-09-19T07:53:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.805+0000 7f26c4f86640 1 -- 172.21.15.16:0/1258497075 wait complete. 2024-09-19T07:53:53.817 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:53:53.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.919+0000 7f207e445640 1 Processor -- start 2024-09-19T07:53:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.919+0000 7f207e445640 1 -- start start 2024-09-19T07:53:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.920+0000 7f207e445640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f207815e680 0x7f20780cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.920+0000 7f207e445640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20780d0010 0x7f20780d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.920+0000 7f207e445640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20780d1880 con 0x7f207815e2a0 2024-09-19T07:53:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.920+0000 7f207e445640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f20780d1b60 con 0x7f20780d0010 2024-09-19T07:53:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.920+0000 7f207e445640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f20780cf2d0 con 0x7f207815e680 2024-09-19T07:53:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.920+0000 7f207c9e2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20780d0010 0x7f20780d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.920+0000 7f207c9e2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20780d0010 0x7f20780d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33708/0 (socket says 172.21.15.16:33708) 2024-09-19T07:53:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.920+0000 7f207c9e2640 1 -- 172.21.15.16:0/1133053203 learned_addr learned my addr 172.21.15.16:0/1133053203 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.920+0000 7f20777fe640 1 --2- 172.21.15.16:0/1133053203 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f207815e680 0x7f20780cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.920+0000 7f207c9e2640 1 -- 172.21.15.16:0/1133053203 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f207815e680 msgr2=0x7f20780cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.920+0000 7f207c9e2640 1 --2- 172.21.15.16:0/1133053203 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f207815e680 0x7f20780cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.920+0000 7f207c9e2640 1 -- 172.21.15.16:0/1133053203 >> v1:172.21.15.146:6789/0 conn(0x7f207815e2a0 legacy=0x7f207806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.920+0000 7f207c9e2640 1 -- 172.21.15.16:0/1133053203 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20780cf5b0 con 0x7f20780d0010 2024-09-19T07:53:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.921+0000 7f207c9e2640 1 --2- 172.21.15.16:0/1133053203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20780d0010 0x7f20780d4370 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f2070009e50 tx=0x7f207002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742caff6f2f801cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.921+0000 7f2076ffd640 1 -- 172.21.15.16:0/1133053203 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070036070 con 0x7f20780d0010 2024-09-19T07:53:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.921+0000 7f2076ffd640 1 -- 172.21.15.16:0/1133053203 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2070031600 con 0x7f20780d0010 2024-09-19T07:53:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.921+0000 7f2076ffd640 1 -- 172.21.15.16:0/1133053203 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20700319d0 con 0x7f20780d0010 2024-09-19T07:53:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.922+0000 7f207e445640 1 -- 172.21.15.16:0/1133053203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20780d0010 msgr2=0x7f20780d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.922+0000 7f207e445640 1 --2- 172.21.15.16:0/1133053203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20780d0010 0x7f20780d4370 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f2070009e50 tx=0x7f207002a680 comp rx=0 tx=0).stop 2024-09-19T07:53:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.922+0000 7f207e445640 1 -- 172.21.15.16:0/1133053203 shutdown_connections 2024-09-19T07:53:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.922+0000 7f207e445640 1 --2- 172.21.15.16:0/1133053203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20780d0010 0x7f20780d4370 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.922+0000 7f207e445640 1 --2- 172.21.15.16:0/1133053203 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f207815e680 0x7f20780cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.922+0000 7f207e445640 1 -- 172.21.15.16:0/1133053203 >> 172.21.15.16:0/1133053203 conn(0x7f207809d760 msgr2=0x7f207808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:53.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.922+0000 7f207e445640 1 -- 172.21.15.16:0/1133053203 shutdown_connections 2024-09-19T07:53:53.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.922+0000 7f207e445640 1 -- 172.21.15.16:0/1133053203 wait complete. 2024-09-19T07:53:53.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.923+0000 7f207e445640 1 Processor -- start 2024-09-19T07:53:53.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.923+0000 7f207e445640 1 -- start start 2024-09-19T07:53:53.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.924+0000 7f207e445640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20780d0010 0x7f20780ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.924+0000 7f207e445640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f207815e2a0 0x7f20780c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.924+0000 7f207e445640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f207815e680 0x7f2078160ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.924+0000 7f207e445640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f207806dcb0 con 0x7f207815e2a0 2024-09-19T07:53:53.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.924+0000 7f207e445640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f207806db30 con 0x7f20780d0010 2024-09-19T07:53:53.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.924+0000 7f207e445640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f20780d48b0 con 0x7f207815e680 2024-09-19T07:53:53.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.924+0000 7f20777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f207815e2a0 0x7f20780c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.924+0000 7f20777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f207815e2a0 0x7f20780c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33720/0 (socket says 172.21.15.16:33720) 2024-09-19T07:53:53.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.924+0000 7f20777fe640 1 -- 172.21.15.16:0/537114494 learned_addr learned my addr 172.21.15.16:0/537114494 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:53.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.924+0000 7f207c9e2640 1 --2- 172.21.15.16:0/537114494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f207815e680 0x7f2078160ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.924+0000 7f2077fff640 1 --2- 172.21.15.16:0/537114494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20780d0010 0x7f20780ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.925+0000 7f20777fe640 1 -- 172.21.15.16:0/537114494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f207815e680 msgr2=0x7f2078160ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.925+0000 7f20777fe640 1 --2- 172.21.15.16:0/537114494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f207815e680 0x7f2078160ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.925+0000 7f20777fe640 1 -- 172.21.15.16:0/537114494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20780d0010 msgr2=0x7f20780ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.925+0000 7f20777fe640 1 --2- 172.21.15.16:0/537114494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20780d0010 0x7f20780ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.925+0000 7f20777fe640 1 -- 172.21.15.16:0/537114494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20780cd330 con 0x7f207815e2a0 2024-09-19T07:53:53.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.925+0000 7f20777fe640 1 --2- 172.21.15.16:0/537114494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f207815e2a0 0x7f20780c7f10 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f206c00e840 tx=0x7f206c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:53.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.925+0000 7f2074ff9640 1 -- 172.21.15.16:0/537114494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c017070 con 0x7f207815e2a0 2024-09-19T07:53:53.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.925+0000 7f2074ff9640 1 -- 172.21.15.16:0/537114494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f206c00c730 con 0x7f207815e2a0 2024-09-19T07:53:53.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.925+0000 7f207e445640 1 -- 172.21.15.16:0/537114494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20780cd770 con 0x7f207815e2a0 2024-09-19T07:53:53.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.925+0000 7f2074ff9640 1 -- 172.21.15.16:0/537114494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c00cb40 con 0x7f207815e2a0 2024-09-19T07:53:53.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.925+0000 7f207e445640 1 -- 172.21.15.16:0/537114494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20780cdd30 con 0x7f207815e2a0 2024-09-19T07:53:53.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.926+0000 7f207e445640 1 -- 172.21.15.16:0/537114494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f207806e040 con 0x7f207815e2a0 2024-09-19T07:53:53.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.926+0000 7f2074ff9640 1 -- 172.21.15.16:0/537114494 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f206c01a070 con 0x7f207815e2a0 2024-09-19T07:53:53.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.927+0000 7f2074ff9640 1 --2- 172.21.15.16:0/537114494 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f206406c2b0 0x7f206406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.927+0000 7f2074ff9640 1 -- 172.21.15.16:0/537114494 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f206c08b2c0 con 0x7f207815e2a0 2024-09-19T07:53:53.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.927+0000 7f2077fff640 1 --2- 172.21.15.16:0/537114494 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f206406c2b0 0x7f206406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.928+0000 7f2077fff640 1 --2- 172.21.15.16:0/537114494 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f206406c2b0 0x7f206406e770 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2068001780 tx=0x7f2068009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:53.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:53.931+0000 7f2074ff9640 1 -- 172.21.15.16:0/537114494 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f206c0592e0 con 0x7f207815e2a0 2024-09-19T07:53:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:54.143+0000 7f207e445640 1 -- 172.21.15.16:0/537114494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f20780d4370 con 0x7f207815e2a0 2024-09-19T07:53:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:54.144+0000 7f2074ff9640 1 -- 172.21.15.16:0/537114494 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v63) ==== 53+0+73 (secure 0 0 0) 0x7f206c05d5e0 con 0x7f207815e2a0 2024-09-19T07:53:54.144 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:53:54.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:54.146+0000 7f207e445640 1 -- 172.21.15.16:0/537114494 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f206406c2b0 msgr2=0x7f206406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:54.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:54.146+0000 7f207e445640 1 --2- 172.21.15.16:0/537114494 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f206406c2b0 0x7f206406e770 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2068001780 tx=0x7f2068009040 comp rx=0 tx=0).stop 2024-09-19T07:53:54.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:54.146+0000 7f207e445640 1 -- 172.21.15.16:0/537114494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f207815e2a0 msgr2=0x7f20780c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:54.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:54.146+0000 7f207e445640 1 --2- 172.21.15.16:0/537114494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f207815e2a0 0x7f20780c7f10 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f206c00e840 tx=0x7f206c00ed00 comp rx=0 tx=0).stop 2024-09-19T07:53:54.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:54.146+0000 7f207e445640 1 -- 172.21.15.16:0/537114494 shutdown_connections 2024-09-19T07:53:54.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:54.146+0000 7f207e445640 1 --2- 172.21.15.16:0/537114494 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f206406c2b0 0x7f206406e770 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:54.146+0000 7f207e445640 1 --2- 172.21.15.16:0/537114494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f207815e680 0x7f2078160ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:54.146+0000 7f207e445640 1 --2- 172.21.15.16:0/537114494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f207815e2a0 0x7f20780c7f10 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:54.146+0000 7f207e445640 1 --2- 172.21.15.16:0/537114494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20780d0010 0x7f20780ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:54.146+0000 7f207e445640 1 -- 172.21.15.16:0/537114494 >> 172.21.15.16:0/537114494 conn(0x7f207809d760 msgr2=0x7f20780d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:54.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:54.146+0000 7f207e445640 1 -- 172.21.15.16:0/537114494 shutdown_connections 2024-09-19T07:53:54.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:54.146+0000 7f207e445640 1 -- 172.21.15.16:0/537114494 wait complete. 2024-09-19T07:53:54.157 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:53:54.157 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:53:54.158 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T07:53:54.158 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T07:53:54.158 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:54.158 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:53:54.158 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:53:54.158 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:53:54.158 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:53:54.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:54.158 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T07:53:54.175 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:53:54.175 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:53:54.175 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:53:54.175 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:54.175 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:53:54.175 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:54.175 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:53:54.176 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:54.176 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T07:53:54.176 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:53:54.176 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:54.176 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T07:53:54.176 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:53:54.176 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:54.176 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T07:53:54.176 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T07:53:54.176 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42155sec preferred_lft 42155sec 2024-09-19T07:53:54.176 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T07:53:54.176 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:54.176 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:53:54.177 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:54.177 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T07:53:54.177 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:53:54.177 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:54.177 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:53:54.177 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:54.177 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::32:7bff:fe92:8895/64 scope link 2024-09-19T07:53:54.177 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:54.177 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:53:54.177 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether e6:fd:21:6a:43:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:54.177 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e4fd:21ff:fe6a:4369/64 scope link 2024-09-19T07:53:54.177 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:54.177 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:53:54.178 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:54.178 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::80f4:81ff:fe7d:b7f0/64 scope link 2024-09-19T07:53:54.178 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:54.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:54.178 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:53:54.239 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:53:54.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:54.241 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T07:53:54.298 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T07:53:54.298 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:53:54.298 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T07:53:54.298 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T07:53:54.298 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:53:54.299 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:53:54.299 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:53:54.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T07:53:54.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T07:53:54.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T07:53:54.299 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:53:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:53:54.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:53:54.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:54.426 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:53:54.442 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T07:53:54.442 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:53:54.443 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:53:54.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:54.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-19T07:53:54.509 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T07:53:54.509 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:53:54.509 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:54.565 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:53:54.565 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:53:54.566 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:54.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:53:54.689 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:53:54.746 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:53:54.746 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:53:54.746 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:53:54.746 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:53:54.746 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:53:54.746 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:53:54.747 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:53:54.747 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:53:54.747 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:53:54.747 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:53:54.747 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:53:54.747 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:53:54.747 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:53:54.747 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:53:54.747 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:53:54.747 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:53:54.747 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:53:54.748 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:53:54.748 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:53:54.748 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:53:54.748 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:53:54.748 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:53:54.748 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:53:54.748 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:53:54.748 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:53:54.748 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:53:54.748 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:53:54.748 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:53:54.748 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:53:54.748 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:53:54.749 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:54.749 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:54.749 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:54.749 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:54.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:54.749 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:53:54.802 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:53:54.802 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T07:53:54.845 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:53:54.870 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:53:54.927 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:53:54.927 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:53:54.927 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:53:54.927 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:53:54.927 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:53:54.928 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:53:54.928 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:53:54.928 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:53:54.928 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:53:54.928 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:53:54.928 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:53:54.928 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:53:54.928 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:53:54.928 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:53:54.928 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:53:54.928 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:53:54.928 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:53:54.928 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:53:54.929 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:53:54.929 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:53:54.929 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:53:54.929 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:53:54.929 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:53:54.929 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:53:54.929 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:53:54.929 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:53:54.929 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:53:54.929 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:53:54.929 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:53:54.929 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:53:54.929 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:54.929 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:53:54.930 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:54.930 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:53:54.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:54.930 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:53:54.931 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-19T07:53:54.929+0000 7fd93ab78480 -1 init, newargv = 0x55f39b544a10 newargc=15 2024-09-19T07:53:54.931 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[69634]: starting ceph client 2024-09-19T07:53:54.934 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[69634]: starting fuse 2024-09-19T07:53:54.947 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:53:54.947 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-19T07:53:54.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:53:54.947 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:53:54.948 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:53:54.948 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:53:54.948 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:53:54.948 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:54.948 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:53:54.948 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:53:54.948 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:53:54.948 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:53:54.948 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:54.948 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:53:54.948 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:53:54.948 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:53:54.948 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:53:54.948 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:54.948 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:53:54.949 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:53:54.949 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:53:54.949 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:53:54.949 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:53:54.949 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:53:54.949 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:53:54.949 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:53:54.949 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:54.949 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:53:54.949 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:53:55.067 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.69634.asok 2024-09-19T07:53:55.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:53:55.077 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.69634.asok 2024-09-19T07:53:55.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:55.078 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.69634.asok status 2024-09-19T07:53:55.213 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:53:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:53:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:53:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:53:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:53:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:53:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:53:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "69634", 2024-09-19T07:53:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:53:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-19T07:53:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-19T07:53:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7551, 2024-09-19T07:53:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:53:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:53:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:53:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 7551 2024-09-19T07:53:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:53:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:53:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:53:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1440089416 2024-09-19T07:53:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:53:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:53:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:53:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1440089416 2024-09-19T07:53:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.7551 192.168.0.1:0/1440089416", 2024-09-19T07:53:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/1440089416", 2024-09-19T07:53:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-19T07:53:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 63, 2024-09-19T07:53:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 118, 2024-09-19T07:53:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:53:55.217 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:53:55.217 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:53:55.217 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:53:55.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:55.222 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-19T07:53:55.222 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:55.238 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-19T07:53:55.239 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:55.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:55.239 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:53:55.306 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:53:55.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.448+0000 7f274ebd8640 1 Processor -- start 2024-09-19T07:53:55.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.448+0000 7f274ebd8640 1 -- start start 2024-09-19T07:53:55.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.449+0000 7f274ebd8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f274806f9e0 0x7f27480d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.449+0000 7f274ebd8640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f27480d09f0 0x7f27480c5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.449+0000 7f274ebd8640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f274815b500 con 0x7f274815b120 2024-09-19T07:53:55.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.449+0000 7f274ebd8640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f27480d0e40 con 0x7f27480d09f0 2024-09-19T07:53:55.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.449+0000 7f274ebd8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f274806e940 con 0x7f274806f9e0 2024-09-19T07:53:55.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.449+0000 7f274d175640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f27480d09f0 0x7f27480c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.449+0000 7f273ffff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f274806f9e0 0x7f27480d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.449+0000 7f274c974640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f274815b120 0x7f274806dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:58134/0 (socket says 172.21.15.16:58134) 2024-09-19T07:53:55.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.449+0000 7f274c974640 1 -- 172.21.15.16:0/1576052908 learned_addr learned my addr 172.21.15.16:0/1576052908 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:55.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.449+0000 7f273ffff640 1 -- 172.21.15.16:0/1576052908 >> v2:172.21.15.146:3300/0 conn(0x7f27480d09f0 msgr2=0x7f27480c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:55.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.449+0000 7f273ffff640 1 --2- 172.21.15.16:0/1576052908 >> v2:172.21.15.146:3300/0 conn(0x7f27480d09f0 0x7f27480c5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.450+0000 7f273ffff640 1 -- 172.21.15.16:0/1576052908 >> v1:172.21.15.146:6789/0 conn(0x7f274815b120 legacy=0x7f274806dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:55.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.450+0000 7f273ffff640 1 -- 172.21.15.16:0/1576052908 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274806ec20 con 0x7f274806f9e0 2024-09-19T07:53:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.450+0000 7f274d175640 1 --2- 172.21.15.16:0/1576052908 >> v2:172.21.15.146:3300/0 conn(0x7f27480d09f0 0x7f27480c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:53:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.450+0000 7f273ffff640 1 --2- 172.21.15.16:0/1576052908 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f274806f9e0 0x7f27480d04b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f27380097a0 tx=0x7f273802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5974aa108ec0fca0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.450+0000 7f273f7fe640 1 -- 172.21.15.16:0/1576052908 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2738037070 con 0x7f274806f9e0 2024-09-19T07:53:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.450+0000 7f273f7fe640 1 -- 172.21.15.16:0/1576052908 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2738032440 con 0x7f274806f9e0 2024-09-19T07:53:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.451+0000 7f273f7fe640 1 -- 172.21.15.16:0/1576052908 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2738032810 con 0x7f274806f9e0 2024-09-19T07:53:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.451+0000 7f274ebd8640 1 -- 172.21.15.16:0/1576052908 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f274806f9e0 msgr2=0x7f27480d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.451+0000 7f274ebd8640 1 --2- 172.21.15.16:0/1576052908 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f274806f9e0 0x7f27480d04b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f27380097a0 tx=0x7f273802a180 comp rx=0 tx=0).stop 2024-09-19T07:53:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.451+0000 7f274ebd8640 1 -- 172.21.15.16:0/1576052908 shutdown_connections 2024-09-19T07:53:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.451+0000 7f274ebd8640 1 --2- 172.21.15.16:0/1576052908 >> v2:172.21.15.146:3300/0 conn(0x7f27480d09f0 0x7f27480c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.451+0000 7f274ebd8640 1 --2- 172.21.15.16:0/1576052908 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f274806f9e0 0x7f27480d04b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.451+0000 7f274ebd8640 1 -- 172.21.15.16:0/1576052908 >> 172.21.15.16:0/1576052908 conn(0x7f274806bdf0 msgr2=0x7f274806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.451+0000 7f274ebd8640 1 -- 172.21.15.16:0/1576052908 shutdown_connections 2024-09-19T07:53:55.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.451+0000 7f274ebd8640 1 -- 172.21.15.16:0/1576052908 wait complete. 2024-09-19T07:53:55.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.452+0000 7f274ebd8640 1 Processor -- start 2024-09-19T07:53:55.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.452+0000 7f274ebd8640 1 -- start start 2024-09-19T07:53:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.453+0000 7f274ebd8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274806f9e0 0x7f2748157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.453+0000 7f274ebd8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27480d09f0 0x7f27481561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.453+0000 7f274ebd8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f274815b120 0x7f27481566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.453+0000 7f274ebd8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f274806f4c0 con 0x7f274806f9e0 2024-09-19T07:53:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.453+0000 7f274ebd8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f274806f1e0 con 0x7f274815b120 2024-09-19T07:53:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.453+0000 7f274ebd8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f274806e5d0 con 0x7f27480d09f0 2024-09-19T07:53:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.453+0000 7f274c974640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274806f9e0 0x7f2748157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.453+0000 7f274c974640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274806f9e0 0x7f2748157db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33722/0 (socket says 172.21.15.16:33722) 2024-09-19T07:53:55.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.453+0000 7f274c974640 1 -- 172.21.15.16:0/2603338557 learned_addr learned my addr 172.21.15.16:0/2603338557 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.453+0000 7f273ffff640 1 --2- 172.21.15.16:0/2603338557 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27480d09f0 0x7f27481561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.453+0000 7f274d175640 1 --2- 172.21.15.16:0/2603338557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f274815b120 0x7f27481566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.454+0000 7f274c974640 1 -- 172.21.15.16:0/2603338557 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27480d09f0 msgr2=0x7f27481561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.454+0000 7f274c974640 1 --2- 172.21.15.16:0/2603338557 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27480d09f0 0x7f27481561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.454+0000 7f274c974640 1 -- 172.21.15.16:0/2603338557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f274815b120 msgr2=0x7f27481566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.454+0000 7f274c974640 1 --2- 172.21.15.16:0/2603338557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f274815b120 0x7f27481566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.454+0000 7f274c974640 1 -- 172.21.15.16:0/2603338557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2748154860 con 0x7f274806f9e0 2024-09-19T07:53:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.454+0000 7f273ffff640 1 --2- 172.21.15.16:0/2603338557 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27480d09f0 0x7f27481561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.454+0000 7f274d175640 1 --2- 172.21.15.16:0/2603338557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f274815b120 0x7f27481566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.454+0000 7f274c974640 1 --2- 172.21.15.16:0/2603338557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274806f9e0 0x7f2748157db0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f274400c3c0 tx=0x7f274400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.454+0000 7f273d7fa640 1 -- 172.21.15.16:0/2603338557 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744017070 con 0x7f274806f9e0 2024-09-19T07:53:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.454+0000 7f273d7fa640 1 -- 172.21.15.16:0/2603338557 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2744005ce0 con 0x7f274806f9e0 2024-09-19T07:53:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.454+0000 7f274ebd8640 1 -- 172.21.15.16:0/2603338557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2748156d10 con 0x7f274806f9e0 2024-09-19T07:53:55.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.454+0000 7f273d7fa640 1 -- 172.21.15.16:0/2603338557 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27440072f0 con 0x7f274806f9e0 2024-09-19T07:53:55.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.454+0000 7f274ebd8640 1 -- 172.21.15.16:0/2603338557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27481572d0 con 0x7f274806f9e0 2024-09-19T07:53:55.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.455+0000 7f274ebd8640 1 -- 172.21.15.16:0/2603338557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27480ce060 con 0x7f274806f9e0 2024-09-19T07:53:55.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.455+0000 7f273d7fa640 1 -- 172.21.15.16:0/2603338557 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f274401a070 con 0x7f274806f9e0 2024-09-19T07:53:55.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.456+0000 7f273d7fa640 1 --2- 172.21.15.16:0/2603338557 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f273406c2b0 0x7f273406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.456+0000 7f273d7fa640 1 -- 172.21.15.16:0/2603338557 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f274408a9f0 con 0x7f274806f9e0 2024-09-19T07:53:55.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.456+0000 7f273ffff640 1 --2- 172.21.15.16:0/2603338557 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f273406c2b0 0x7f273406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.457+0000 7f273ffff640 1 --2- 172.21.15.16:0/2603338557 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f273406c2b0 0x7f273406e770 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2738034000 tx=0x7f273802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:55.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.460+0000 7f273d7fa640 1 -- 172.21.15.16:0/2603338557 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2744058f50 con 0x7f274806f9e0 2024-09-19T07:53:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.671+0000 7f274ebd8640 1 -- 172.21.15.16:0/2603338557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f27481e3d00 con 0x7f274806f9e0 2024-09-19T07:53:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.671+0000 7f273d7fa640 1 -- 172.21.15.16:0/2603338557 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v63) ==== 53+0+73 (secure 0 0 0) 0x7f274408a070 con 0x7f274806f9e0 2024-09-19T07:53:55.672 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:53:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.673+0000 7f274ebd8640 1 -- 172.21.15.16:0/2603338557 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f273406c2b0 msgr2=0x7f273406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.673+0000 7f274ebd8640 1 --2- 172.21.15.16:0/2603338557 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f273406c2b0 0x7f273406e770 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2738034000 tx=0x7f273802e000 comp rx=0 tx=0).stop 2024-09-19T07:53:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.673+0000 7f274ebd8640 1 -- 172.21.15.16:0/2603338557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274806f9e0 msgr2=0x7f2748157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.673+0000 7f274ebd8640 1 --2- 172.21.15.16:0/2603338557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274806f9e0 0x7f2748157db0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f274400c3c0 tx=0x7f274400c880 comp rx=0 tx=0).stop 2024-09-19T07:53:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.673+0000 7f274ebd8640 1 -- 172.21.15.16:0/2603338557 shutdown_connections 2024-09-19T07:53:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.673+0000 7f274ebd8640 1 --2- 172.21.15.16:0/2603338557 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f273406c2b0 0x7f273406e770 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.673+0000 7f274ebd8640 1 --2- 172.21.15.16:0/2603338557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f274815b120 0x7f27481566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.673+0000 7f274ebd8640 1 --2- 172.21.15.16:0/2603338557 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27480d09f0 0x7f27481561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.673+0000 7f274ebd8640 1 --2- 172.21.15.16:0/2603338557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274806f9e0 0x7f2748157db0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.673+0000 7f274ebd8640 1 -- 172.21.15.16:0/2603338557 >> 172.21.15.16:0/2603338557 conn(0x7f274806bdf0 msgr2=0x7f274806dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.673+0000 7f274ebd8640 1 -- 172.21.15.16:0/2603338557 shutdown_connections 2024-09-19T07:53:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:55.673+0000 7f274ebd8640 1 -- 172.21.15.16:0/2603338557 wait complete. 2024-09-19T07:53:55.685 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:53:55.685 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:53:55.685 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-09-19T07:53:55.685 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T07:53:55.685 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T07:53:55.685 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:53:55.685 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:53:55.685 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:53:55.685 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:53:55.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:55.686 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-09-19T07:53:55.705 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:53:55.705 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:53:55.705 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:53:55.706 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:55.706 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-09-19T07:53:55.706 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:55.706 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:53:55.706 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:55.706 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2024-09-19T07:53:55.706 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:53:55.706 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:55.706 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2024-09-19T07:53:55.706 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:53:55.706 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:55.706 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:53:55.706 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:55.706 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T07:53:55.707 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42121sec preferred_lft 42121sec 2024-09-19T07:53:55.707 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link noprefixroute 2024-09-19T07:53:55.707 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:55.707 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:53:55.707 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:53:55.707 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:53:55.707 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:55.707 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::e47f:32ff:fe68:31ca/64 scope link 2024-09-19T07:53:55.707 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:55.707 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:53:55.707 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether f6:1f:22:44:9d:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:53:55.707 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::f41f:22ff:fe44:9dcd/64 scope link 2024-09-19T07:53:55.707 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:55.707 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:53:55.708 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:53:55.708 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::8459:eeff:fe21:a113/64 scope link 2024-09-19T07:53:55.708 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:53:55.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:55.708 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:53:55.773 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-09-19T07:53:55.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:55.774 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-09-19T07:53:55.833 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-09-19T07:53:55.833 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:53:55.833 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T07:53:55.834 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T07:53:55.834 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:53:55.834 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:53:55.834 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-19T07:53:55.834 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T07:53:55.834 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T07:53:55.834 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T07:53:55.835 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-19T07:53:55.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:53:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:53:55.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:53:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:53:55.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:55.966 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-09-19T07:53:55.984 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T07:53:55.984 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T07:53:55.984 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:53:55.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:55.984 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2024-09-19T07:53:56.047 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T07:53:56.047 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:53:56.047 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T07:53:56.103 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:53:56.103 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:53:56.103 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:53:56.159 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:53:56.224 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:53:56.280 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:53:56.280 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:53:56.281 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:53:56.281 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:53:56.281 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:53:56.281 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:53:56.281 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:53:56.281 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:53:56.281 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:53:56.281 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:53:56.281 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:53:56.281 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:53:56.281 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:53:56.281 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:53:56.281 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:53:56.282 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:53:56.282 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:53:56.282 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:53:56.282 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:53:56.282 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:53:56.282 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:53:56.282 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:53:56.282 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:53:56.282 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:53:56.282 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:53:56.282 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:53:56.282 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:53:56.282 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:53:56.283 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:53:56.283 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:53:56.283 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:53:56.283 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:53:56.283 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:53:56.283 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:53:56.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:56.283 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:53:56.337 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:53:56.337 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T07:53:56.379 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:53:56.404 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:53:56.460 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:53:56.460 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:53:56.461 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:53:56.461 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:53:56.461 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:53:56.461 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:53:56.461 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:53:56.461 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:53:56.461 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:53:56.461 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:53:56.461 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:53:56.461 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:53:56.462 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:53:56.462 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:53:56.462 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:53:56.462 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:53:56.462 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:53:56.462 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:53:56.462 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:53:56.462 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:53:56.462 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:53:56.462 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:53:56.462 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:53:56.463 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:53:56.463 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:53:56.463 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:53:56.463 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:53:56.463 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:53:56.463 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:53:56.463 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:53:56.463 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:53:56.463 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:53:56.463 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:53:56.463 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:53:56.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:56.464 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:53:56.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2024-09-19T07:53:56.461+0000 7fabb2e21480 -1 init, newargv = 0x55dc99384800 newargc=15 2024-09-19T07:53:56.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[48393]: starting ceph client 2024-09-19T07:53:56.467 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[48393]: starting fuse 2024-09-19T07:53:56.479 INFO:teuthology.orchestra.run.smithi146.stdout:42 2024-09-19T07:53:56.480 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-19T07:53:56.480 DEBUG:teuthology.orchestra.run.smithi146:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:53:56.480 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2024-09-19T07:53:56.480 DEBUG:teuthology.orchestra.run.smithi146:> import re 2024-09-19T07:53:56.480 DEBUG:teuthology.orchestra.run.smithi146:> import os 2024-09-19T07:53:56.480 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2024-09-19T07:53:56.480 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:53:56.480 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2024-09-19T07:53:56.480 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T07:53:56.481 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2024-09-19T07:53:56.481 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T07:53:56.481 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:53:56.481 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2024-09-19T07:53:56.481 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2024-09-19T07:53:56.481 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2024-09-19T07:53:56.481 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2024-09-19T07:53:56.481 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:53:56.481 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2024-09-19T07:53:56.481 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:53:56.481 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:53:56.481 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:53:56.481 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2024-09-19T07:53:56.482 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2024-09-19T07:53:56.482 DEBUG:teuthology.orchestra.run.smithi146:> return f 2024-09-19T07:53:56.482 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:53:56.482 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:53:56.482 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2024-09-19T07:53:56.482 DEBUG:teuthology.orchestra.run.smithi146:> ' 2024-09-19T07:53:56.597 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.48393.asok 2024-09-19T07:53:56.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:53:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:53:56.607 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.48393.asok 2024-09-19T07:53:56.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:56.608 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.48393.asok status 2024-09-19T07:53:56.742 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-19T07:53:56.742 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2024-09-19T07:53:56.743 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:53:56.743 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:53:56.743 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2024-09-19T07:53:56.743 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2024-09-19T07:53:56.743 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T07:53:56.743 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "48393", 2024-09-19T07:53:56.743 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2024-09-19T07:53:56.743 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:53:56.743 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2024-09-19T07:53:56.743 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2024-09-19T07:53:56.743 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 6374, 2024-09-19T07:53:56.743 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2024-09-19T07:53:56.743 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2024-09-19T07:53:56.744 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2024-09-19T07:53:56.744 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 6374 2024-09-19T07:53:56.744 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:53:56.744 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:53:56.744 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:53:56.744 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:53:56.744 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 3044582375 2024-09-19T07:53:56.744 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-19T07:53:56.744 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:53:56.744 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:53:56.744 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:53:56.744 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:53:56.744 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 3044582375 2024-09-19T07:53:56.745 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:53:56.745 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.6374 192.168.0.1:0/3044582375", 2024-09-19T07:53:56.745 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/3044582375", 2024-09-19T07:53:56.745 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2024-09-19T07:53:56.745 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 63, 2024-09-19T07:53:56.745 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 118, 2024-09-19T07:53:56.745 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:53:56.745 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2024-09-19T07:53:56.745 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2024-09-19T07:53:56.745 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-19T07:53:56.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:56.751 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2024-09-19T07:53:56.751 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T07:53:56.766 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2024-09-19T07:53:56.767 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T07:53:56.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:56.767 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:53:56.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-09-19T07:53:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.951+0000 7fed33766640 1 Processor -- start 2024-09-19T07:53:56.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.951+0000 7fed33766640 1 -- start start 2024-09-19T07:53:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.952+0000 7fed33766640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c0d21f0 0x7fed2c0d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.952+0000 7fed33766640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed2c0d2b10 con 0x7fed2c141ad0 2024-09-19T07:53:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.952+0000 7fed33766640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed2c1620b0 con 0x7fed2c072590 2024-09-19T07:53:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.952+0000 7fed33766640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fed2c141750 con 0x7fed2c0d21f0 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.952+0000 7fed31d03640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fed2c141ad0 0x7fed2c141f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35202/0 (socket says 172.21.15.16:35202) 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.952+0000 7fed31d03640 1 -- 172.21.15.16:0/3622643827 learned_addr learned my addr 172.21.15.16:0/3622643827 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.952+0000 7fed31502640 1 --2- 172.21.15.16:0/3622643827 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c0d21f0 0x7fed2c0d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.952+0000 7fed23fff640 1 -- 172.21.15.16:0/3622643827 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1581554215 0 0) 0x7fed2c0d2b10 con 0x7fed2c141ad0 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.953+0000 7fed23fff640 1 -- 172.21.15.16:0/3622643827 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed18003520 con 0x7fed2c141ad0 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.953+0000 7fed23fff640 1 -- 172.21.15.16:0/3622643827 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1228201118 0 0) 0x7fed2c1620b0 con 0x7fed2c072590 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.953+0000 7fed23fff640 1 -- 172.21.15.16:0/3622643827 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed2c0d2b10 con 0x7fed2c072590 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.953+0000 7fed31502640 1 -- 172.21.15.16:0/3622643827 >> v1:172.21.15.146:6789/0 conn(0x7fed2c072590 legacy=0x7fed2c06d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.953+0000 7fed31502640 1 -- 172.21.15.16:0/3622643827 >> v1:172.21.15.16:6789/0 conn(0x7fed2c141ad0 legacy=0x7fed2c141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.953+0000 7fed31502640 1 -- 172.21.15.16:0/3622643827 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c162370 con 0x7fed2c0d21f0 2024-09-19T07:53:56.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.954+0000 7fed31502640 1 --2- 172.21.15.16:0/3622643827 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c0d21f0 0x7fed2c0d25d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fed28009dc0 tx=0x7fed2802a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67348cc5b93b73b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:56.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.954+0000 7fed23fff640 1 -- 172.21.15.16:0/3622643827 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2802ae60 con 0x7fed2c0d21f0 2024-09-19T07:53:56.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.954+0000 7fed23fff640 1 -- 172.21.15.16:0/3622643827 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed28007740 con 0x7fed2c0d21f0 2024-09-19T07:53:56.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.954+0000 7fed23fff640 1 -- 172.21.15.16:0/3622643827 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28007b10 con 0x7fed2c0d21f0 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.954+0000 7fed33766640 1 -- 172.21.15.16:0/3622643827 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c0d21f0 msgr2=0x7fed2c0d25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.954+0000 7fed33766640 1 --2- 172.21.15.16:0/3622643827 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c0d21f0 0x7fed2c0d25d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fed28009dc0 tx=0x7fed2802a640 comp rx=0 tx=0).stop 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.954+0000 7fed33766640 1 -- 172.21.15.16:0/3622643827 shutdown_connections 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.954+0000 7fed33766640 1 --2- 172.21.15.16:0/3622643827 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c0d21f0 0x7fed2c0d25d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.954+0000 7fed33766640 1 -- 172.21.15.16:0/3622643827 >> 172.21.15.16:0/3622643827 conn(0x7fed2c0b3ed0 msgr2=0x7fed2c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.954+0000 7fed33766640 1 -- 172.21.15.16:0/3622643827 shutdown_connections 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.954+0000 7fed33766640 1 -- 172.21.15.16:0/3622643827 wait complete. 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.956+0000 7fed33766640 1 Processor -- start 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.956+0000 7fed33766640 1 -- start start 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed33766640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed2c072590 0x7fed2c140de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed33766640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c0d21f0 0x7fed2c160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed33766640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed2c141ad0 0x7fed2c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed33766640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c162930 con 0x7fed2c072590 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed33766640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c06d7f0 con 0x7fed2c141ad0 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed33766640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fed2c162c10 con 0x7fed2c0d21f0 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed31502640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed2c072590 0x7fed2c140de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed31502640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed2c072590 0x7fed2c140de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33738/0 (socket says 172.21.15.16:33738) 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed31502640 1 -- 172.21.15.16:0/1379618352 learned_addr learned my addr 172.21.15.16:0/1379618352 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed31d03640 1 --2- 172.21.15.16:0/1379618352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed2c141ad0 0x7fed2c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed30d01640 1 --2- 172.21.15.16:0/1379618352 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c0d21f0 0x7fed2c160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed31502640 1 -- 172.21.15.16:0/1379618352 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c0d21f0 msgr2=0x7fed2c160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed31502640 1 --2- 172.21.15.16:0/1379618352 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c0d21f0 0x7fed2c160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed31502640 1 -- 172.21.15.16:0/1379618352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed2c141ad0 msgr2=0x7fed2c0b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed31502640 1 --2- 172.21.15.16:0/1379618352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed2c141ad0 0x7fed2c0b6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed31502640 1 -- 172.21.15.16:0/1379618352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c0b6080 con 0x7fed2c072590 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed31d03640 1 --2- 172.21.15.16:0/1379618352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed2c141ad0 0x7fed2c0b6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed31502640 1 --2- 172.21.15.16:0/1379618352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed2c072590 0x7fed2c140de0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fed28000f80 tx=0x7fed28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed21ffb640 1 -- 172.21.15.16:0/1379618352 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28040070 con 0x7fed2c072590 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed21ffb640 1 -- 172.21.15.16:0/1379618352 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed2802ae60 con 0x7fed2c072590 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed21ffb640 1 -- 172.21.15.16:0/1379618352 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28031be0 con 0x7fed2c072590 2024-09-19T07:53:56.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.957+0000 7fed33766640 1 -- 172.21.15.16:0/1379618352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2c15f680 con 0x7fed2c072590 2024-09-19T07:53:56.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.958+0000 7fed33766640 1 -- 172.21.15.16:0/1379618352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2c15fc40 con 0x7fed2c072590 2024-09-19T07:53:56.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.959+0000 7fed33766640 1 -- 172.21.15.16:0/1379618352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed2c0690c0 con 0x7fed2c072590 2024-09-19T07:53:56.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.959+0000 7fed21ffb640 1 -- 172.21.15.16:0/1379618352 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fed28039040 con 0x7fed2c072590 2024-09-19T07:53:56.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.959+0000 7fed21ffb640 1 --2- 172.21.15.16:0/1379618352 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fed1006c2b0 0x7fed1006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.960+0000 7fed21ffb640 1 -- 172.21.15.16:0/1379618352 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fed280accc0 con 0x7fed2c072590 2024-09-19T07:53:56.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.960+0000 7fed30d01640 1 --2- 172.21.15.16:0/1379618352 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fed1006c2b0 0x7fed1006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.961+0000 7fed30d01640 1 --2- 172.21.15.16:0/1379618352 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fed1006c2b0 0x7fed1006e770 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fed2c158310 tx=0x7fed1c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:56.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:56.964+0000 7fed21ffb640 1 -- 172.21.15.16:0/1379618352 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fed2807b3f0 con 0x7fed2c072590 2024-09-19T07:53:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.142+0000 7fed33766640 1 -- 172.21.15.16:0/1379618352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7fed2c072290 con 0x7fed2c072590 2024-09-19T07:53:57.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.145+0000 7fed21ffb640 1 -- 172.21.15.16:0/1379618352 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v44)=0 v44) ==== 121+0+0 (secure 0 0 0) 0x7fed2804ee30 con 0x7fed2c072590 2024-09-19T07:53:57.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.147+0000 7fed33766640 1 -- 172.21.15.16:0/1379618352 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fed1006c2b0 msgr2=0x7fed1006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:57.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.147+0000 7fed33766640 1 --2- 172.21.15.16:0/1379618352 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fed1006c2b0 0x7fed1006e770 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fed2c158310 tx=0x7fed1c009040 comp rx=0 tx=0).stop 2024-09-19T07:53:57.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.147+0000 7fed33766640 1 -- 172.21.15.16:0/1379618352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed2c072590 msgr2=0x7fed2c140de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:57.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.147+0000 7fed33766640 1 --2- 172.21.15.16:0/1379618352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed2c072590 0x7fed2c140de0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fed28000f80 tx=0x7fed28004a40 comp rx=0 tx=0).stop 2024-09-19T07:53:57.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.147+0000 7fed33766640 1 -- 172.21.15.16:0/1379618352 shutdown_connections 2024-09-19T07:53:57.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.147+0000 7fed33766640 1 --2- 172.21.15.16:0/1379618352 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fed1006c2b0 0x7fed1006e770 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.147+0000 7fed33766640 1 --2- 172.21.15.16:0/1379618352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed2c141ad0 0x7fed2c0b6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.147+0000 7fed33766640 1 --2- 172.21.15.16:0/1379618352 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed2c0d21f0 0x7fed2c160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.147+0000 7fed33766640 1 --2- 172.21.15.16:0/1379618352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed2c072590 0x7fed2c140de0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.147+0000 7fed33766640 1 -- 172.21.15.16:0/1379618352 >> 172.21.15.16:0/1379618352 conn(0x7fed2c0b3ed0 msgr2=0x7fed2c0c8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:57.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.148+0000 7fed33766640 1 -- 172.21.15.16:0/1379618352 shutdown_connections 2024-09-19T07:53:57.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.148+0000 7fed33766640 1 -- 172.21.15.16:0/1379618352 wait complete. 2024-09-19T07:53:57.316 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 400 2024-09-19T07:53:57.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.424+0000 7f80adae6640 1 Processor -- start 2024-09-19T07:53:57.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.424+0000 7f80adae6640 1 -- start start 2024-09-19T07:53:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.424+0000 7f80adae6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f80a81619b0 0x7f80a80d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.424+0000 7f80adae6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80a80b0a80 0x7f80a80d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.424+0000 7f80adae6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80a806c990 con 0x7f80a8161d90 2024-09-19T07:53:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.424+0000 7f80adae6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f80a80d0ec0 con 0x7f80a80b0a80 2024-09-19T07:53:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.424+0000 7f80adae6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f80a80cec10 con 0x7f80a81619b0 2024-09-19T07:53:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.425+0000 7f80a6ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80a80b0a80 0x7f80a80d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.425+0000 7f80a6ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80a80b0a80 0x7f80a80d0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33752/0 (socket says 172.21.15.16:33752) 2024-09-19T07:53:57.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.425+0000 7f80a6ffd640 1 -- 172.21.15.16:0/2773084150 learned_addr learned my addr 172.21.15.16:0/2773084150 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:57.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.425+0000 7f80a7fff640 1 --1- 172.21.15.16:0/2773084150 >> v1:172.21.15.16:6789/0 conn(0x7f80a8161d90 0x7f80a80cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35206/0 (socket says 172.21.15.16:35206) 2024-09-19T07:53:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.425+0000 7f80a77fe640 1 --2- 172.21.15.16:0/2773084150 >> v2:172.21.15.146:3300/0 conn(0x7f80a81619b0 0x7f80a80d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.425+0000 7f80a6ffd640 1 -- 172.21.15.16:0/2773084150 >> v2:172.21.15.146:3300/0 conn(0x7f80a81619b0 msgr2=0x7f80a80d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.425+0000 7f80a6ffd640 1 --2- 172.21.15.16:0/2773084150 >> v2:172.21.15.146:3300/0 conn(0x7f80a81619b0 0x7f80a80d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.425+0000 7f80a6ffd640 1 -- 172.21.15.16:0/2773084150 >> v1:172.21.15.16:6789/0 conn(0x7f80a8161d90 legacy=0x7f80a80cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.425+0000 7f80a67fc640 1 -- 172.21.15.16:0/2773084150 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3262217503 0 0) 0x7f80a806c990 con 0x7f80a8161d90 2024-09-19T07:53:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.425+0000 7f80a6ffd640 1 -- 172.21.15.16:0/2773084150 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a80ce930 con 0x7f80a80b0a80 2024-09-19T07:53:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.426+0000 7f80a6ffd640 1 --2- 172.21.15.16:0/2773084150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a80b0a80 0x7f80a80d0980 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f80a80d1040 tx=0x7f809c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9431769625fd6ce9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.426+0000 7f80a67fc640 1 -- 172.21.15.16:0/2773084150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c037070 con 0x7f80a80b0a80 2024-09-19T07:53:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.426+0000 7f80a67fc640 1 -- 172.21.15.16:0/2773084150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f809c032440 con 0x7f80a80b0a80 2024-09-19T07:53:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.426+0000 7f80a67fc640 1 -- 172.21.15.16:0/2773084150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c032810 con 0x7f80a80b0a80 2024-09-19T07:53:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.426+0000 7f80adae6640 1 -- 172.21.15.16:0/2773084150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a80b0a80 msgr2=0x7f80a80d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.426+0000 7f80adae6640 1 --2- 172.21.15.16:0/2773084150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a80b0a80 0x7f80a80d0980 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f80a80d1040 tx=0x7f809c02a270 comp rx=0 tx=0).stop 2024-09-19T07:53:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.426+0000 7f80adae6640 1 -- 172.21.15.16:0/2773084150 shutdown_connections 2024-09-19T07:53:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.426+0000 7f80adae6640 1 --2- 172.21.15.16:0/2773084150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a80b0a80 0x7f80a80d0980 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.426+0000 7f80adae6640 1 --2- 172.21.15.16:0/2773084150 >> v2:172.21.15.146:3300/0 conn(0x7f80a81619b0 0x7f80a80d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.426+0000 7f80adae6640 1 -- 172.21.15.16:0/2773084150 >> 172.21.15.16:0/2773084150 conn(0x7f80a809a9f0 msgr2=0x7f80a80b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.427+0000 7f80adae6640 1 -- 172.21.15.16:0/2773084150 shutdown_connections 2024-09-19T07:53:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.427+0000 7f80adae6640 1 -- 172.21.15.16:0/2773084150 wait complete. 2024-09-19T07:53:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.428+0000 7f80adae6640 1 Processor -- start 2024-09-19T07:53:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.428+0000 7f80adae6640 1 -- start start 2024-09-19T07:53:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f80adae6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a80b0a80 0x7f80a80b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f80adae6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80a81619b0 0x7f80a81f3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f80adae6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a8161d90 0x7f80a80c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f80adae6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80a806c620 con 0x7f80a8161d90 2024-09-19T07:53:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f80adae6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f80a80b0820 con 0x7f80a80b0a80 2024-09-19T07:53:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f80adae6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f80a80ccce0 con 0x7f80a81619b0 2024-09-19T07:53:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f80a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a8161d90 0x7f80a80c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f80a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a8161d90 0x7f80a80c6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33768/0 (socket says 172.21.15.16:33768) 2024-09-19T07:53:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f80a7fff640 1 -- 172.21.15.16:0/645555103 learned_addr learned my addr 172.21.15.16:0/645555103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f80a77fe640 1 --2- 172.21.15.16:0/645555103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a80b0a80 0x7f80a80b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f80a6ffd640 1 --2- 172.21.15.16:0/645555103 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80a81619b0 0x7f80a81f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f80a7fff640 1 -- 172.21.15.16:0/645555103 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80a81619b0 msgr2=0x7f80a81f3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f80a7fff640 1 --2- 172.21.15.16:0/645555103 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80a81619b0 0x7f80a81f3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f80a7fff640 1 -- 172.21.15.16:0/645555103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a80b0a80 msgr2=0x7f80a80b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f80a7fff640 1 --2- 172.21.15.16:0/645555103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a80b0a80 0x7f80a80b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f80a7fff640 1 -- 172.21.15.16:0/645555103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a80b2490 con 0x7f80a8161d90 2024-09-19T07:53:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f80a7fff640 1 --2- 172.21.15.16:0/645555103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a8161d90 0x7f80a80c6ff0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f80a000c700 tx=0x7f80a000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:57.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f808ffff640 1 -- 172.21.15.16:0/645555103 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0018070 con 0x7f80a8161d90 2024-09-19T07:53:57.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f808ffff640 1 -- 172.21.15.16:0/645555103 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a000ce10 con 0x7f80a8161d90 2024-09-19T07:53:57.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f808ffff640 1 -- 172.21.15.16:0/645555103 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a000a860 con 0x7f80a8161d90 2024-09-19T07:53:57.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.429+0000 7f80adae6640 1 -- 172.21.15.16:0/645555103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a80b2770 con 0x7f80a8161d90 2024-09-19T07:53:57.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.430+0000 7f80adae6640 1 -- 172.21.15.16:0/645555103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a80b2d30 con 0x7f80a8161d90 2024-09-19T07:53:57.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.431+0000 7f80adae6640 1 -- 172.21.15.16:0/645555103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a8160ed0 con 0x7f80a8161d90 2024-09-19T07:53:57.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.431+0000 7f808ffff640 1 -- 172.21.15.16:0/645555103 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f80a001b070 con 0x7f80a8161d90 2024-09-19T07:53:57.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.432+0000 7f808ffff640 1 --2- 172.21.15.16:0/645555103 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f809406c2b0 0x7f809406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:57.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.432+0000 7f808ffff640 1 -- 172.21.15.16:0/645555103 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f80a008ab30 con 0x7f80a8161d90 2024-09-19T07:53:57.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.432+0000 7f80a77fe640 1 --2- 172.21.15.16:0/645555103 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f809406c2b0 0x7f809406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:57.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.432+0000 7f80a77fe640 1 --2- 172.21.15.16:0/645555103 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f809406c2b0 0x7f809406e770 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8098006fd0 tx=0x7f8098008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:57.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.436+0000 7f808ffff640 1 -- 172.21.15.16:0/645555103 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f80a8160ed0 con 0x7f80a8161d90 2024-09-19T07:53:57.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.615+0000 7f80adae6640 1 -- 172.21.15.16:0/645555103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7f80a806c360 con 0x7f80a8161d90 2024-09-19T07:53:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.618+0000 7f808ffff640 1 -- 172.21.15.16:0/645555103 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v45)=0 v45) ==== 119+0+0 (secure 0 0 0) 0x7f80a806c360 con 0x7f80a8161d90 2024-09-19T07:53:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.620+0000 7f80adae6640 1 -- 172.21.15.16:0/645555103 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f809406c2b0 msgr2=0x7f809406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.620+0000 7f80adae6640 1 --2- 172.21.15.16:0/645555103 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f809406c2b0 0x7f809406e770 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8098006fd0 tx=0x7f8098008040 comp rx=0 tx=0).stop 2024-09-19T07:53:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.620+0000 7f80adae6640 1 -- 172.21.15.16:0/645555103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a8161d90 msgr2=0x7f80a80c6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.620+0000 7f80adae6640 1 --2- 172.21.15.16:0/645555103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a8161d90 0x7f80a80c6ff0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f80a000c700 tx=0x7f80a000cbc0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.620+0000 7f80adae6640 1 -- 172.21.15.16:0/645555103 shutdown_connections 2024-09-19T07:53:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.620+0000 7f80adae6640 1 --2- 172.21.15.16:0/645555103 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f809406c2b0 0x7f809406e770 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.620+0000 7f80adae6640 1 --2- 172.21.15.16:0/645555103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a8161d90 0x7f80a80c6ff0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.620+0000 7f80adae6640 1 --2- 172.21.15.16:0/645555103 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80a81619b0 0x7f80a81f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.620+0000 7f80adae6640 1 --2- 172.21.15.16:0/645555103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80a80b0a80 0x7f80a80b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.620+0000 7f80adae6640 1 -- 172.21.15.16:0/645555103 >> 172.21.15.16:0/645555103 conn(0x7f80a809a9f0 msgr2=0x7f80a806f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.620+0000 7f80adae6640 1 -- 172.21.15.16:0/645555103 shutdown_connections 2024-09-19T07:53:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.620+0000 7f80adae6640 1 -- 172.21.15.16:0/645555103 wait complete. 2024-09-19T07:53:57.631 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 800 2024-09-19T07:53:57.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.736+0000 7f9b39e5e640 1 Processor -- start 2024-09-19T07:53:57.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.736+0000 7f9b39e5e640 1 -- start start 2024-09-19T07:53:57.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.737+0000 7f9b39e5e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b3415e2a0 0x7f9b3406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.737+0000 7f9b39e5e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b340d1880 con 0x7f9b3415e680 2024-09-19T07:53:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.737+0000 7f9b39e5e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b340d1b60 con 0x7f9b3406d960 2024-09-19T07:53:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.737+0000 7f9b39e5e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9b340cf2d0 con 0x7f9b3415e2a0 2024-09-19T07:53:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.737+0000 7f9b337fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b3415e2a0 0x7f9b3406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.737+0000 7f9b32ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9b3415e680 0x7f9b340cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35214/0 (socket says 172.21.15.16:35214) 2024-09-19T07:53:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.737+0000 7f9b32ffd640 1 -- 172.21.15.16:0/2569388582 learned_addr learned my addr 172.21.15.16:0/2569388582 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.737+0000 7f9b337fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b3415e2a0 0x7f9b3406d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33776/0 (socket says 172.21.15.16:33776) 2024-09-19T07:53:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.737+0000 7f9b337fe640 1 -- 172.21.15.16:0/2569388582 >> v1:172.21.15.146:6789/0 conn(0x7f9b3406d960 legacy=0x7f9b340d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.737+0000 7f9b337fe640 1 -- 172.21.15.16:0/2569388582 >> v1:172.21.15.16:6789/0 conn(0x7f9b3415e680 legacy=0x7f9b340cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.737+0000 7f9b327fc640 1 -- 172.21.15.16:0/2569388582 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4195972616 0 0) 0x7f9b340d1880 con 0x7f9b3415e680 2024-09-19T07:53:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.737+0000 7f9b337fe640 1 -- 172.21.15.16:0/2569388582 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b340cf5b0 con 0x7f9b3415e2a0 2024-09-19T07:53:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.738+0000 7f9b337fe640 1 --2- 172.21.15.16:0/2569388582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3415e2a0 0x7f9b3406d420 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f9b240097a0 tx=0x7f9b2402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5d50921d365cc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:57.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.738+0000 7f9b327fc640 1 -- 172.21.15.16:0/2569388582 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24037070 con 0x7f9b3415e2a0 2024-09-19T07:53:57.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.738+0000 7f9b327fc640 1 -- 172.21.15.16:0/2569388582 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b24032440 con 0x7f9b3415e2a0 2024-09-19T07:53:57.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.738+0000 7f9b327fc640 1 -- 172.21.15.16:0/2569388582 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24032810 con 0x7f9b3415e2a0 2024-09-19T07:53:57.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.739+0000 7f9b39e5e640 1 -- 172.21.15.16:0/2569388582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3415e2a0 msgr2=0x7f9b3406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:57.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.739+0000 7f9b39e5e640 1 --2- 172.21.15.16:0/2569388582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3415e2a0 0x7f9b3406d420 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f9b240097a0 tx=0x7f9b2402a1e0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.739+0000 7f9b39e5e640 1 -- 172.21.15.16:0/2569388582 shutdown_connections 2024-09-19T07:53:57.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.739+0000 7f9b39e5e640 1 --2- 172.21.15.16:0/2569388582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3415e2a0 0x7f9b3406d420 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.739+0000 7f9b39e5e640 1 -- 172.21.15.16:0/2569388582 >> 172.21.15.16:0/2569388582 conn(0x7f9b3409d760 msgr2=0x7f9b3408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:57.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.739+0000 7f9b39e5e640 1 -- 172.21.15.16:0/2569388582 shutdown_connections 2024-09-19T07:53:57.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.739+0000 7f9b39e5e640 1 -- 172.21.15.16:0/2569388582 wait complete. 2024-09-19T07:53:57.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.740+0000 7f9b39e5e640 1 Processor -- start 2024-09-19T07:53:57.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.740+0000 7f9b39e5e640 1 -- start start 2024-09-19T07:53:57.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.741+0000 7f9b39e5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3406d960 0x7f9b34156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:57.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.741+0000 7f9b39e5e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b3415e2a0 0x7f9b341e7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:57.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.741+0000 7f9b39e5e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b3415e680 0x7f9b340b9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:57.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.741+0000 7f9b39e5e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b340d0360 con 0x7f9b3406d960 2024-09-19T07:53:57.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.741+0000 7f9b39e5e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9b340d01e0 con 0x7f9b3415e680 2024-09-19T07:53:57.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.741+0000 7f9b39e5e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9b340d4a80 con 0x7f9b3415e2a0 2024-09-19T07:53:57.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.741+0000 7f9b337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3406d960 0x7f9b34156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:57.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.741+0000 7f9b337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3406d960 0x7f9b34156ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33792/0 (socket says 172.21.15.16:33792) 2024-09-19T07:53:57.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.741+0000 7f9b337fe640 1 -- 172.21.15.16:0/2270131378 learned_addr learned my addr 172.21.15.16:0/2270131378 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:57.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.741+0000 7f9b32ffd640 1 --2- 172.21.15.16:0/2270131378 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b3415e2a0 0x7f9b341e7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:57.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.741+0000 7f9b33fff640 1 --2- 172.21.15.16:0/2270131378 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b3415e680 0x7f9b340b9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:57.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.742+0000 7f9b337fe640 1 -- 172.21.15.16:0/2270131378 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b3415e2a0 msgr2=0x7f9b341e7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:57.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.742+0000 7f9b337fe640 1 --2- 172.21.15.16:0/2270131378 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b3415e2a0 0x7f9b341e7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.742+0000 7f9b337fe640 1 -- 172.21.15.16:0/2270131378 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b3415e680 msgr2=0x7f9b340b9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.742+0000 7f9b337fe640 1 --2- 172.21.15.16:0/2270131378 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b3415e680 0x7f9b340b9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.742+0000 7f9b337fe640 1 -- 172.21.15.16:0/2270131378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b340b8af0 con 0x7f9b3406d960 2024-09-19T07:53:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.742+0000 7f9b337fe640 1 --2- 172.21.15.16:0/2270131378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3406d960 0x7f9b34156ac0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f9b240098c0 tx=0x7f9b240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.742+0000 7f9b1bfff640 1 -- 172.21.15.16:0/2270131378 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24041070 con 0x7f9b3406d960 2024-09-19T07:53:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.742+0000 7f9b1bfff640 1 -- 172.21.15.16:0/2270131378 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b2403c870 con 0x7f9b3406d960 2024-09-19T07:53:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.742+0000 7f9b39e5e640 1 -- 172.21.15.16:0/2270131378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b340cd0f0 con 0x7f9b3406d960 2024-09-19T07:53:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.742+0000 7f9b1bfff640 1 -- 172.21.15.16:0/2270131378 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24037040 con 0x7f9b3406d960 2024-09-19T07:53:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.742+0000 7f9b39e5e640 1 -- 172.21.15.16:0/2270131378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b340cd6b0 con 0x7f9b3406d960 2024-09-19T07:53:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.743+0000 7f9b39e5e640 1 -- 172.21.15.16:0/2270131378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b340ccf10 con 0x7f9b3406d960 2024-09-19T07:53:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.744+0000 7f9b1bfff640 1 -- 172.21.15.16:0/2270131378 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9b24044070 con 0x7f9b3406d960 2024-09-19T07:53:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.744+0000 7f9b1bfff640 1 --2- 172.21.15.16:0/2270131378 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b2006c2b0 0x7f9b2006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:57.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.744+0000 7f9b1bfff640 1 -- 172.21.15.16:0/2270131378 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f9b240ac180 con 0x7f9b3406d960 2024-09-19T07:53:57.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.744+0000 7f9b32ffd640 1 --2- 172.21.15.16:0/2270131378 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b2006c2b0 0x7f9b2006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:57.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.745+0000 7f9b32ffd640 1 --2- 172.21.15.16:0/2270131378 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b2006c2b0 0x7f9b2006e770 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9b34157000 tx=0x7f9b28007d00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:57.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.749+0000 7f9b1bfff640 1 -- 172.21.15.16:0/2270131378 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9b340ccf10 con 0x7f9b3406d960 2024-09-19T07:53:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.922+0000 7f9b39e5e640 1 -- 172.21.15.16:0/2270131378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7f9b3406d420 con 0x7f9b3406d960 2024-09-19T07:53:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.925+0000 7f9b1bfff640 1 -- 172.21.15.16:0/2270131378 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v46)=0 v46) ==== 128+0+0 (secure 0 0 0) 0x7f9b2407a880 con 0x7f9b3406d960 2024-09-19T07:53:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.927+0000 7f9b39e5e640 1 -- 172.21.15.16:0/2270131378 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b2006c2b0 msgr2=0x7f9b2006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.927+0000 7f9b39e5e640 1 --2- 172.21.15.16:0/2270131378 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b2006c2b0 0x7f9b2006e770 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9b34157000 tx=0x7f9b28007d00 comp rx=0 tx=0).stop 2024-09-19T07:53:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.927+0000 7f9b39e5e640 1 -- 172.21.15.16:0/2270131378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3406d960 msgr2=0x7f9b34156ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.927+0000 7f9b39e5e640 1 --2- 172.21.15.16:0/2270131378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3406d960 0x7f9b34156ac0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f9b240098c0 tx=0x7f9b240046c0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.927+0000 7f9b39e5e640 1 -- 172.21.15.16:0/2270131378 shutdown_connections 2024-09-19T07:53:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.927+0000 7f9b39e5e640 1 --2- 172.21.15.16:0/2270131378 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b2006c2b0 0x7f9b2006e770 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.927+0000 7f9b39e5e640 1 --2- 172.21.15.16:0/2270131378 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b3415e680 0x7f9b340b9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.927+0000 7f9b39e5e640 1 --2- 172.21.15.16:0/2270131378 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b3415e2a0 0x7f9b341e7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.927+0000 7f9b39e5e640 1 --2- 172.21.15.16:0/2270131378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3406d960 0x7f9b34156ac0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.927+0000 7f9b39e5e640 1 -- 172.21.15.16:0/2270131378 >> 172.21.15.16:0/2270131378 conn(0x7f9b3409d760 msgr2=0x7f9b340d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.927+0000 7f9b39e5e640 1 -- 172.21.15.16:0/2270131378 shutdown_connections 2024-09-19T07:53:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:57.927+0000 7f9b39e5e640 1 -- 172.21.15.16:0/2270131378 wait complete. 2024-09-19T07:53:57.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-09-19T07:53:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.043+0000 7fe0e598d640 1 Processor -- start 2024-09-19T07:53:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.043+0000 7fe0e598d640 1 -- start start 2024-09-19T07:53:58.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.044+0000 7fe0e598d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe0e015e2a0 0x7fe0e006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.044+0000 7fe0e598d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0e015e680 0x7fe0e00cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.044+0000 7fe0e598d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0e00d1880 con 0x7fe0e006d960 2024-09-19T07:53:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.044+0000 7fe0e598d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fe0e00d1b60 con 0x7fe0e015e2a0 2024-09-19T07:53:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.044+0000 7fe0e598d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e00cf2d0 con 0x7fe0e015e680 2024-09-19T07:53:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.044+0000 7fe0deffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe0e015e2a0 0x7fe0e006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.044+0000 7fe0df7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe0e006d960 0x7fe0e00d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35224/0 (socket says 172.21.15.16:35224) 2024-09-19T07:53:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.044+0000 7fe0df7fe640 1 -- 172.21.15.16:0/828551812 learned_addr learned my addr 172.21.15.16:0/828551812 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.044+0000 7fe0deffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe0e015e2a0 0x7fe0e006d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33808/0 (socket says 172.21.15.16:33808) 2024-09-19T07:53:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.044+0000 7fe0deffd640 1 -- 172.21.15.16:0/828551812 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0e015e680 msgr2=0x7fe0e00cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.044+0000 7fe0deffd640 1 --2- 172.21.15.16:0/828551812 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0e015e680 0x7fe0e00cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.044+0000 7fe0deffd640 1 -- 172.21.15.16:0/828551812 >> v1:172.21.15.16:6789/0 conn(0x7fe0e006d960 legacy=0x7fe0e00d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.044+0000 7fe0ddffb640 1 -- 172.21.15.16:0/828551812 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 453418205 0 0) 0x7fe0e00d1880 con 0x7fe0e006d960 2024-09-19T07:53:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.044+0000 7fe0deffd640 1 -- 172.21.15.16:0/828551812 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e00cf5b0 con 0x7fe0e015e2a0 2024-09-19T07:53:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.045+0000 7fe0deffd640 1 --2- 172.21.15.16:0/828551812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e015e2a0 0x7fe0e006d420 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe0d00097a0 tx=0x7fe0d002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3739f7c8a3188c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.045+0000 7fe0ddffb640 1 -- 172.21.15.16:0/828551812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0037070 con 0x7fe0e015e2a0 2024-09-19T07:53:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.045+0000 7fe0ddffb640 1 -- 172.21.15.16:0/828551812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d0032440 con 0x7fe0e015e2a0 2024-09-19T07:53:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.045+0000 7fe0ddffb640 1 -- 172.21.15.16:0/828551812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0032810 con 0x7fe0e015e2a0 2024-09-19T07:53:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.046+0000 7fe0e598d640 1 -- 172.21.15.16:0/828551812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e015e2a0 msgr2=0x7fe0e006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.046+0000 7fe0e598d640 1 --2- 172.21.15.16:0/828551812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e015e2a0 0x7fe0e006d420 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe0d00097a0 tx=0x7fe0d002a1a0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.046+0000 7fe0e598d640 1 -- 172.21.15.16:0/828551812 shutdown_connections 2024-09-19T07:53:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.046+0000 7fe0e598d640 1 --2- 172.21.15.16:0/828551812 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0e015e680 0x7fe0e00cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.046+0000 7fe0e598d640 1 --2- 172.21.15.16:0/828551812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e015e2a0 0x7fe0e006d420 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.046+0000 7fe0e598d640 1 -- 172.21.15.16:0/828551812 >> 172.21.15.16:0/828551812 conn(0x7fe0e009d760 msgr2=0x7fe0e008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.046+0000 7fe0e598d640 1 -- 172.21.15.16:0/828551812 shutdown_connections 2024-09-19T07:53:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.046+0000 7fe0e598d640 1 -- 172.21.15.16:0/828551812 wait complete. 2024-09-19T07:53:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.047+0000 7fe0e598d640 1 Processor -- start 2024-09-19T07:53:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.047+0000 7fe0e598d640 1 -- start start 2024-09-19T07:53:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.048+0000 7fe0e598d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0e006d960 0x7fe0e014c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.048+0000 7fe0e598d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e015e2a0 0x7fe0e00b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.048+0000 7fe0e598d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0e015e680 0x7fe0e00cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.048+0000 7fe0e598d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e00d4c00 con 0x7fe0e015e2a0 2024-09-19T07:53:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.048+0000 7fe0e598d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e00d4a80 con 0x7fe0e006d960 2024-09-19T07:53:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.048+0000 7fe0e598d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e00d0010 con 0x7fe0e015e680 2024-09-19T07:53:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.048+0000 7fe0de7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e015e2a0 0x7fe0e00b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.048+0000 7fe0de7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e015e2a0 0x7fe0e00b7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33824/0 (socket says 172.21.15.16:33824) 2024-09-19T07:53:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.048+0000 7fe0de7fc640 1 -- 172.21.15.16:0/704174209 learned_addr learned my addr 172.21.15.16:0/704174209 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.048+0000 7fe0de7fc640 1 -- 172.21.15.16:0/704174209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0e015e680 msgr2=0x7fe0e00cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.048+0000 7fe0de7fc640 1 --2- 172.21.15.16:0/704174209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0e015e680 0x7fe0e00cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.048+0000 7fe0de7fc640 1 -- 172.21.15.16:0/704174209 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0e006d960 msgr2=0x7fe0e014c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.048+0000 7fe0de7fc640 1 --2- 172.21.15.16:0/704174209 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0e006d960 0x7fe0e014c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.048+0000 7fe0de7fc640 1 -- 172.21.15.16:0/704174209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e014d010 con 0x7fe0e015e2a0 2024-09-19T07:53:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.048+0000 7fe0de7fc640 1 --2- 172.21.15.16:0/704174209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e015e2a0 0x7fe0e00b7a30 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800b390 tx=0x7fe0c800b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.049+0000 7fe0d7fff640 1 -- 172.21.15.16:0/704174209 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8015020 con 0x7fe0e015e2a0 2024-09-19T07:53:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.049+0000 7fe0d7fff640 1 -- 172.21.15.16:0/704174209 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c8011830 con 0x7fe0e015e2a0 2024-09-19T07:53:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.049+0000 7fe0e598d640 1 -- 172.21.15.16:0/704174209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e00c8a00 con 0x7fe0e015e2a0 2024-09-19T07:53:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.049+0000 7fe0d7fff640 1 -- 172.21.15.16:0/704174209 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8011c20 con 0x7fe0e015e2a0 2024-09-19T07:53:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.049+0000 7fe0e598d640 1 -- 172.21.15.16:0/704174209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e00c8fc0 con 0x7fe0e015e2a0 2024-09-19T07:53:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.050+0000 7fe0e598d640 1 -- 172.21.15.16:0/704174209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0e006d420 con 0x7fe0e015e2a0 2024-09-19T07:53:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.050+0000 7fe0d7fff640 1 -- 172.21.15.16:0/704174209 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe0c8018020 con 0x7fe0e015e2a0 2024-09-19T07:53:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.051+0000 7fe0d7fff640 1 --2- 172.21.15.16:0/704174209 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe0c006c2b0 0x7fe0c006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.051+0000 7fe0d7fff640 1 -- 172.21.15.16:0/704174209 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fe0c808b3b0 con 0x7fe0e015e2a0 2024-09-19T07:53:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.051+0000 7fe0deffd640 1 --2- 172.21.15.16:0/704174209 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe0c006c2b0 0x7fe0c006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.052+0000 7fe0deffd640 1 --2- 172.21.15.16:0/704174209 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe0c006c2b0 0x7fe0c006e770 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0034000 tx=0x7fe0d002c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.055+0000 7fe0d7fff640 1 -- 172.21.15.16:0/704174209 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe0c801f020 con 0x7fe0e015e2a0 2024-09-19T07:53:58.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.231+0000 7fe0e598d640 1 -- 172.21.15.16:0/704174209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7fe0e006cf90 con 0x7fe0e015e2a0 2024-09-19T07:53:58.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.232+0000 7fe0d7fff640 1 -- 172.21.15.16:0/704174209 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v46) ==== 106+0+4 (secure 0 0 0) 0x7fe0c8059a20 con 0x7fe0e015e2a0 2024-09-19T07:53:58.232 INFO:teuthology.orchestra.run.smithi016.stdout:100 2024-09-19T07:53:58.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.234+0000 7fe0e598d640 1 -- 172.21.15.16:0/704174209 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe0c006c2b0 msgr2=0x7fe0c006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.234+0000 7fe0e598d640 1 --2- 172.21.15.16:0/704174209 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe0c006c2b0 0x7fe0c006e770 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0034000 tx=0x7fe0d002c000 comp rx=0 tx=0).stop 2024-09-19T07:53:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.234+0000 7fe0e598d640 1 -- 172.21.15.16:0/704174209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e015e2a0 msgr2=0x7fe0e00b7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.234+0000 7fe0e598d640 1 --2- 172.21.15.16:0/704174209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e015e2a0 0x7fe0e00b7a30 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800b390 tx=0x7fe0c800b850 comp rx=0 tx=0).stop 2024-09-19T07:53:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.234+0000 7fe0e598d640 1 -- 172.21.15.16:0/704174209 shutdown_connections 2024-09-19T07:53:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.234+0000 7fe0e598d640 1 --2- 172.21.15.16:0/704174209 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe0c006c2b0 0x7fe0c006e770 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.234+0000 7fe0e598d640 1 --2- 172.21.15.16:0/704174209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0e015e680 0x7fe0e00cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.234+0000 7fe0e598d640 1 --2- 172.21.15.16:0/704174209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e015e2a0 0x7fe0e00b7a30 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.234+0000 7fe0e598d640 1 --2- 172.21.15.16:0/704174209 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0e006d960 0x7fe0e014c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.234+0000 7fe0e598d640 1 -- 172.21.15.16:0/704174209 >> 172.21.15.16:0/704174209 conn(0x7fe0e009d760 msgr2=0x7fe0e00d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.234+0000 7fe0e598d640 1 -- 172.21.15.16:0/704174209 shutdown_connections 2024-09-19T07:53:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.234+0000 7fe0e598d640 1 -- 172.21.15.16:0/704174209 wait complete. 2024-09-19T07:53:58.245 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2024-09-19T07:53:58.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.350+0000 7f2142253640 1 Processor -- start 2024-09-19T07:53:58.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.350+0000 7f2142253640 1 -- start start 2024-09-19T07:53:58.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.351+0000 7f2142253640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f213c06f9e0 0x7f213c0d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.351+0000 7f2142253640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f213c0d09f0 0x7f213c0c5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.351+0000 7f2142253640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f213c15b500 con 0x7f213c15b120 2024-09-19T07:53:58.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.351+0000 7f2142253640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f213c0d0e40 con 0x7f213c06f9e0 2024-09-19T07:53:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.351+0000 7f2142253640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f213c06e940 con 0x7f213c0d09f0 2024-09-19T07:53:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.351+0000 7f213bfff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f213c0d09f0 0x7f213c0c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.351+0000 7f213affd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f213c06f9e0 0x7f213c0d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.351+0000 7f213bfff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f213c0d09f0 0x7f213c0c5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:39998/0 (socket says 172.21.15.16:39998) 2024-09-19T07:53:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.351+0000 7f213bfff640 1 -- 172.21.15.16:0/1913629858 learned_addr learned my addr 172.21.15.16:0/1913629858 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.351+0000 7f213bfff640 1 -- 172.21.15.16:0/1913629858 >> v2:172.21.15.146:3300/0 conn(0x7f213c06f9e0 msgr2=0x7f213c0d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.351+0000 7f213bfff640 1 --2- 172.21.15.16:0/1913629858 >> v2:172.21.15.146:3300/0 conn(0x7f213c06f9e0 0x7f213c0d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.351+0000 7f213bfff640 1 -- 172.21.15.16:0/1913629858 >> v1:172.21.15.146:6789/0 conn(0x7f213c15b120 legacy=0x7f213c06dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.351+0000 7f213bfff640 1 -- 172.21.15.16:0/1913629858 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c06ec20 con 0x7f213c0d09f0 2024-09-19T07:53:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.352+0000 7f213affd640 1 --2- 172.21.15.16:0/1913629858 >> v2:172.21.15.146:3300/0 conn(0x7f213c06f9e0 0x7f213c0d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:53:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.352+0000 7f213bfff640 1 --2- 172.21.15.16:0/1913629858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f213c0d09f0 0x7f213c0c5160 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2134009e50 tx=0x7f213402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48cbb274c7348471 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.352+0000 7f213a7fc640 1 -- 172.21.15.16:0/1913629858 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2134036070 con 0x7f213c0d09f0 2024-09-19T07:53:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.352+0000 7f213a7fc640 1 -- 172.21.15.16:0/1913629858 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2134031600 con 0x7f213c0d09f0 2024-09-19T07:53:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.352+0000 7f213a7fc640 1 -- 172.21.15.16:0/1913629858 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21340319d0 con 0x7f213c0d09f0 2024-09-19T07:53:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.353+0000 7f2142253640 1 -- 172.21.15.16:0/1913629858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f213c0d09f0 msgr2=0x7f213c0c5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.353+0000 7f2142253640 1 --2- 172.21.15.16:0/1913629858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f213c0d09f0 0x7f213c0c5160 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2134009e50 tx=0x7f213402a710 comp rx=0 tx=0).stop 2024-09-19T07:53:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.353+0000 7f2142253640 1 -- 172.21.15.16:0/1913629858 shutdown_connections 2024-09-19T07:53:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.353+0000 7f2142253640 1 --2- 172.21.15.16:0/1913629858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f213c0d09f0 0x7f213c0c5160 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.353+0000 7f2142253640 1 --2- 172.21.15.16:0/1913629858 >> v2:172.21.15.146:3300/0 conn(0x7f213c06f9e0 0x7f213c0d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.353+0000 7f2142253640 1 -- 172.21.15.16:0/1913629858 >> 172.21.15.16:0/1913629858 conn(0x7f213c06bdf0 msgr2=0x7f213c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.353+0000 7f2142253640 1 -- 172.21.15.16:0/1913629858 shutdown_connections 2024-09-19T07:53:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.353+0000 7f2142253640 1 -- 172.21.15.16:0/1913629858 wait complete. 2024-09-19T07:53:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.354+0000 7f2142253640 1 Processor -- start 2024-09-19T07:53:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.355+0000 7f2142253640 1 -- start start 2024-09-19T07:53:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.355+0000 7f2142253640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f213c06f9e0 0x7f213c14ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.355+0000 7f2142253640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213c0d09f0 0x7f213c0b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.355+0000 7f2142253640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f213c15b120 0x7f213c1f1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.355+0000 7f2142253640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f213c06f4c0 con 0x7f213c0d09f0 2024-09-19T07:53:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.355+0000 7f2142253640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f213c06f1e0 con 0x7f213c15b120 2024-09-19T07:53:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.355+0000 7f2142253640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f213c06e5d0 con 0x7f213c06f9e0 2024-09-19T07:53:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.355+0000 7f213affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213c0d09f0 0x7f213c0b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.355+0000 7f213affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213c0d09f0 0x7f213c0b0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33840/0 (socket says 172.21.15.16:33840) 2024-09-19T07:53:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.355+0000 7f213affd640 1 -- 172.21.15.16:0/2468123827 learned_addr learned my addr 172.21.15.16:0/2468123827 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.355+0000 7f213b7fe640 1 --2- 172.21.15.16:0/2468123827 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f213c06f9e0 0x7f213c14ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.356+0000 7f213bfff640 1 --2- 172.21.15.16:0/2468123827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f213c15b120 0x7f213c1f1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.356+0000 7f213affd640 1 -- 172.21.15.16:0/2468123827 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f213c06f9e0 msgr2=0x7f213c14ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.356+0000 7f213affd640 1 --2- 172.21.15.16:0/2468123827 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f213c06f9e0 0x7f213c14ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.356+0000 7f213affd640 1 -- 172.21.15.16:0/2468123827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f213c15b120 msgr2=0x7f213c1f1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.356+0000 7f213affd640 1 --2- 172.21.15.16:0/2468123827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f213c15b120 0x7f213c1f1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.356+0000 7f213affd640 1 -- 172.21.15.16:0/2468123827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c14dda0 con 0x7f213c0d09f0 2024-09-19T07:53:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.356+0000 7f213affd640 1 --2- 172.21.15.16:0/2468123827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213c0d09f0 0x7f213c0b0820 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f212400ebe0 tx=0x7f212400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.356+0000 7f2133fff640 1 -- 172.21.15.16:0/2468123827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2124017070 con 0x7f213c0d09f0 2024-09-19T07:53:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.356+0000 7f2133fff640 1 -- 172.21.15.16:0/2468123827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f212400cc30 con 0x7f213c0d09f0 2024-09-19T07:53:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.356+0000 7f2133fff640 1 -- 172.21.15.16:0/2468123827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2124013470 con 0x7f213c0d09f0 2024-09-19T07:53:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.356+0000 7f2142253640 1 -- 172.21.15.16:0/2468123827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f213c14d0e0 con 0x7f213c0d09f0 2024-09-19T07:53:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.356+0000 7f2142253640 1 -- 172.21.15.16:0/2468123827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f213c15cb80 con 0x7f213c0d09f0 2024-09-19T07:53:58.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.357+0000 7f2142253640 1 -- 172.21.15.16:0/2468123827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f213c0ce060 con 0x7f213c0d09f0 2024-09-19T07:53:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.358+0000 7f2133fff640 1 -- 172.21.15.16:0/2468123827 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f212401a070 con 0x7f213c0d09f0 2024-09-19T07:53:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.358+0000 7f2133fff640 1 --2- 172.21.15.16:0/2468123827 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f211c06c2b0 0x7f211c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.358+0000 7f2133fff640 1 -- 172.21.15.16:0/2468123827 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f212408b3c0 con 0x7f213c0d09f0 2024-09-19T07:53:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.358+0000 7f213b7fe640 1 --2- 172.21.15.16:0/2468123827 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f211c06c2b0 0x7f211c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.359+0000 7f213b7fe640 1 --2- 172.21.15.16:0/2468123827 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f211c06c2b0 0x7f211c06e770 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f212c0016e0 tx=0x7f212c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.363+0000 7f2133fff640 1 -- 172.21.15.16:0/2468123827 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f21240593e0 con 0x7f213c0d09f0 2024-09-19T07:53:58.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.538+0000 7f2142253640 1 -- 172.21.15.16:0/2468123827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7f213c0cb000 con 0x7f213c0d09f0 2024-09-19T07:53:58.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.541+0000 7f2133fff640 1 -- 172.21.15.16:0/2468123827 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v47)=0 v47) ==== 124+0+0 (secure 0 0 0) 0x7f212405d6e0 con 0x7f213c0d09f0 2024-09-19T07:53:58.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.543+0000 7f2142253640 1 -- 172.21.15.16:0/2468123827 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f211c06c2b0 msgr2=0x7f211c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.543+0000 7f2142253640 1 --2- 172.21.15.16:0/2468123827 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f211c06c2b0 0x7f211c06e770 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f212c0016e0 tx=0x7f212c009040 comp rx=0 tx=0).stop 2024-09-19T07:53:58.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.543+0000 7f2142253640 1 -- 172.21.15.16:0/2468123827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213c0d09f0 msgr2=0x7f213c0b0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.543+0000 7f2142253640 1 --2- 172.21.15.16:0/2468123827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213c0d09f0 0x7f213c0b0820 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f212400ebe0 tx=0x7f212400c520 comp rx=0 tx=0).stop 2024-09-19T07:53:58.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.543+0000 7f2142253640 1 -- 172.21.15.16:0/2468123827 shutdown_connections 2024-09-19T07:53:58.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.543+0000 7f2142253640 1 --2- 172.21.15.16:0/2468123827 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f211c06c2b0 0x7f211c06e770 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.544+0000 7f2142253640 1 --2- 172.21.15.16:0/2468123827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f213c15b120 0x7f213c1f1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.544+0000 7f2142253640 1 --2- 172.21.15.16:0/2468123827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213c0d09f0 0x7f213c0b0820 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.544+0000 7f2142253640 1 --2- 172.21.15.16:0/2468123827 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f213c06f9e0 0x7f213c14ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.544+0000 7f2142253640 1 -- 172.21.15.16:0/2468123827 >> 172.21.15.16:0/2468123827 conn(0x7f213c06bdf0 msgr2=0x7f213c0720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.544+0000 7f2142253640 1 -- 172.21.15.16:0/2468123827 shutdown_connections 2024-09-19T07:53:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.544+0000 7f2142253640 1 -- 172.21.15.16:0/2468123827 wait complete. 2024-09-19T07:53:58.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2024-09-19T07:53:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.660+0000 7f00b2198640 1 Processor -- start 2024-09-19T07:53:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.660+0000 7f00b2198640 1 -- start start 2024-09-19T07:53:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00b2198640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f00ac15e2a0 0x7f00ac06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00b2198640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00ac06d960 0x7f00ac0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00b2198640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00ac0d1880 con 0x7f00ac15e680 2024-09-19T07:53:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00b2198640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f00ac0d1b60 con 0x7f00ac15e2a0 2024-09-19T07:53:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00b2198640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f00ac0cf2d0 con 0x7f00ac06d960 2024-09-19T07:53:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00aaffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f00ac15e680 0x7f00ac0cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35226/0 (socket says 172.21.15.16:35226) 2024-09-19T07:53:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00aaffd640 1 -- 172.21.15.16:0/4103894508 learned_addr learned my addr 172.21.15.16:0/4103894508 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:53:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00ab7fe640 1 --2- 172.21.15.16:0/4103894508 >> v2:172.21.15.146:3300/0 conn(0x7f00ac15e2a0 0x7f00ac06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00abfff640 1 --2- 172.21.15.16:0/4103894508 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00ac06d960 0x7f00ac0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00aa7fc640 1 -- 172.21.15.16:0/4103894508 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 354448961 0 0) 0x7f00ac0d1880 con 0x7f00ac15e680 2024-09-19T07:53:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00aa7fc640 1 -- 172.21.15.16:0/4103894508 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0094003520 con 0x7f00ac15e680 2024-09-19T07:53:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00aa7fc640 1 -- 172.21.15.16:0/4103894508 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2283070731 0 0) 0x7f0094003520 con 0x7f00ac15e680 2024-09-19T07:53:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00aa7fc640 1 -- 172.21.15.16:0/4103894508 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00ac06d960 msgr2=0x7f00ac0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00aa7fc640 1 --2- 172.21.15.16:0/4103894508 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00ac06d960 0x7f00ac0d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00aa7fc640 1 -- 172.21.15.16:0/4103894508 >> v2:172.21.15.146:3300/0 conn(0x7f00ac15e2a0 msgr2=0x7f00ac06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00aa7fc640 1 --2- 172.21.15.16:0/4103894508 >> v2:172.21.15.146:3300/0 conn(0x7f00ac15e2a0 0x7f00ac06d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00aa7fc640 1 -- 172.21.15.16:0/4103894508 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ac0cf5b0 con 0x7f00ac15e680 2024-09-19T07:53:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00aa7fc640 1 -- 172.21.15.16:0/4103894508 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f00a0001670 con 0x7f00ac15e680 2024-09-19T07:53:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00aa7fc640 1 -- 172.21.15.16:0/4103894508 >> v1:172.21.15.16:6789/0 conn(0x7f00ac15e680 legacy=0x7f00ac0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00aa7fc640 1 --2- 172.21.15.16:0/4103894508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00940039b0 0x7f0094003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00aa7fc640 1 --2- 172.21.15.16:0/4103894508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0094004800 0x7f0094004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00aa7fc640 1 --2- 172.21.15.16:0/4103894508 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00ac06d960 0x7f00940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00aa7fc640 1 -- 172.21.15.16:0/4103894508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac0d1b60 con 0x7f0094004800 2024-09-19T07:53:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00aa7fc640 1 -- 172.21.15.16:0/4103894508 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac0cf2d0 con 0x7f00940039b0 2024-09-19T07:53:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00aa7fc640 1 -- 172.21.15.16:0/4103894508 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f009400b9c0 con 0x7f00ac06d960 2024-09-19T07:53:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.661+0000 7f00aaffd640 1 --2- 172.21.15.16:0/4103894508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0094004800 0x7f0094004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.662+0000 7f00ab7fe640 1 --2- 172.21.15.16:0/4103894508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00940039b0 0x7f0094003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.662+0000 7f00abfff640 1 --2- 172.21.15.16:0/4103894508 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00ac06d960 0x7f00940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.662+0000 7f00aaffd640 1 -- 172.21.15.16:0/4103894508 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00ac06d960 msgr2=0x7f00940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.662+0000 7f00aaffd640 1 --2- 172.21.15.16:0/4103894508 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00ac06d960 0x7f00940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.662+0000 7f00aaffd640 1 -- 172.21.15.16:0/4103894508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00940039b0 msgr2=0x7f0094003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.662+0000 7f00aaffd640 1 --2- 172.21.15.16:0/4103894508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00940039b0 0x7f0094003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.662+0000 7f00aaffd640 1 -- 172.21.15.16:0/4103894508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0094003520 con 0x7f0094004800 2024-09-19T07:53:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.662+0000 7f00ab7fe640 1 --2- 172.21.15.16:0/4103894508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00940039b0 0x7f0094003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.662+0000 7f00aaffd640 1 --2- 172.21.15.16:0/4103894508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0094004800 0x7f0094004c80 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f00a0001050 tx=0x7f0094005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4d21719c6ac7ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.663+0000 7f00aa7fc640 1 -- 172.21.15.16:0/4103894508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a0007d70 con 0x7f0094004800 2024-09-19T07:53:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.663+0000 7f00aa7fc640 1 -- 172.21.15.16:0/4103894508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a0005af0 con 0x7f0094004800 2024-09-19T07:53:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.663+0000 7f00aa7fc640 1 -- 172.21.15.16:0/4103894508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a0007070 con 0x7f0094004800 2024-09-19T07:53:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.663+0000 7f00b2198640 1 -- 172.21.15.16:0/4103894508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0094004800 msgr2=0x7f0094004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.663+0000 7f00b2198640 1 --2- 172.21.15.16:0/4103894508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0094004800 0x7f0094004c80 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f00a0001050 tx=0x7f0094005830 comp rx=0 tx=0).stop 2024-09-19T07:53:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.663+0000 7f00abfff640 1 -- 172.21.15.16:0/4103894508 reap_dead start 2024-09-19T07:53:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.663+0000 7f00b2198640 1 -- 172.21.15.16:0/4103894508 shutdown_connections 2024-09-19T07:53:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.663+0000 7f00b2198640 1 -- 172.21.15.16:0/4103894508 >> 172.21.15.16:0/4103894508 conn(0x7f00ac09d760 msgr2=0x7f00ac08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.663+0000 7f00b2198640 1 -- 172.21.15.16:0/4103894508 shutdown_connections 2024-09-19T07:53:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.663+0000 7f00b2198640 1 -- 172.21.15.16:0/4103894508 wait complete. 2024-09-19T07:53:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.664+0000 7f00b2198640 1 Processor -- start 2024-09-19T07:53:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.665+0000 7f00b2198640 1 -- start start 2024-09-19T07:53:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.665+0000 7f00b2198640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0094004800 0x7f00ac1f8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.665+0000 7f00b2198640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00ac1e6cc0 0x7f00ac0ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.665+0000 7f00b2198640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00ac1f3610 0x7f00ac1f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.665+0000 7f00b2198640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac0d0360 con 0x7f0094004800 2024-09-19T07:53:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.665+0000 7f00b2198640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac0d01e0 con 0x7f00ac1e6cc0 2024-09-19T07:53:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.665+0000 7f00b2198640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f00ac0d4370 con 0x7f00ac1f3610 2024-09-19T07:53:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.665+0000 7f00ab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0094004800 0x7f00ac1f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.665+0000 7f00ab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0094004800 0x7f00ac1f8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33870/0 (socket says 172.21.15.16:33870) 2024-09-19T07:53:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.665+0000 7f00ab7fe640 1 -- 172.21.15.16:0/3516907202 learned_addr learned my addr 172.21.15.16:0/3516907202 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.665+0000 7f00aaffd640 1 --2- 172.21.15.16:0/3516907202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00ac1e6cc0 0x7f00ac0ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.665+0000 7f00abfff640 1 --2- 172.21.15.16:0/3516907202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00ac1f3610 0x7f00ac1f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.666+0000 7f00ab7fe640 1 -- 172.21.15.16:0/3516907202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00ac1f3610 msgr2=0x7f00ac1f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.666+0000 7f00ab7fe640 1 --2- 172.21.15.16:0/3516907202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00ac1f3610 0x7f00ac1f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.666+0000 7f00ab7fe640 1 -- 172.21.15.16:0/3516907202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00ac1e6cc0 msgr2=0x7f00ac0ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.666+0000 7f00ab7fe640 1 --2- 172.21.15.16:0/3516907202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00ac1e6cc0 0x7f00ac0ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.666+0000 7f00ab7fe640 1 -- 172.21.15.16:0/3516907202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ac1f87d0 con 0x7f0094004800 2024-09-19T07:53:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.666+0000 7f00abfff640 1 --2- 172.21.15.16:0/3516907202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00ac1f3610 0x7f00ac1f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.666+0000 7f00ab7fe640 1 --2- 172.21.15.16:0/3516907202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0094004800 0x7f00ac1f8290 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f009c011600 tx=0x7f009c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.666+0000 7f0093fff640 1 -- 172.21.15.16:0/3516907202 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c00f920 con 0x7f0094004800 2024-09-19T07:53:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.666+0000 7f0093fff640 1 -- 172.21.15.16:0/3516907202 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f009c00fac0 con 0x7f0094004800 2024-09-19T07:53:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.666+0000 7f00b2198640 1 -- 172.21.15.16:0/3516907202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ac1e76f0 con 0x7f0094004800 2024-09-19T07:53:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.666+0000 7f0093fff640 1 -- 172.21.15.16:0/3516907202 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c00fe90 con 0x7f0094004800 2024-09-19T07:53:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.666+0000 7f00b2198640 1 -- 172.21.15.16:0/3516907202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ac1e7cb0 con 0x7f0094004800 2024-09-19T07:53:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.667+0000 7f00b2198640 1 -- 172.21.15.16:0/3516907202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00ac06d420 con 0x7f0094004800 2024-09-19T07:53:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.668+0000 7f0093fff640 1 -- 172.21.15.16:0/3516907202 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f009c015460 con 0x7f0094004800 2024-09-19T07:53:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.668+0000 7f0093fff640 1 --2- 172.21.15.16:0/3516907202 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f009806c450 0x7f009806e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.668+0000 7f0093fff640 1 -- 172.21.15.16:0/3516907202 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f009c08be60 con 0x7f0094004800 2024-09-19T07:53:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.668+0000 7f00aaffd640 1 --2- 172.21.15.16:0/3516907202 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f009806c450 0x7f009806e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.669+0000 7f00aaffd640 1 --2- 172.21.15.16:0/3516907202 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f009806c450 0x7f009806e910 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f00a0033000 tx=0x7f00a0009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:58.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.672+0000 7f0093fff640 1 -- 172.21.15.16:0/3516907202 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f00ac1f87d0 con 0x7f0094004800 2024-09-19T07:53:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.848+0000 7f00b2198640 1 -- 172.21.15.16:0/3516907202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) -- 0x7f00ac06cf90 con 0x7f0094004800 2024-09-19T07:53:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.849+0000 7f0093fff640 1 -- 172.21.15.16:0/3516907202 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v47) ==== 106+0+8 (secure 0 0 0) 0x7f009c019030 con 0x7f0094004800 2024-09-19T07:53:58.849 INFO:teuthology.orchestra.run.smithi016.stdout:1048576 2024-09-19T07:53:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.851+0000 7f00b2198640 1 -- 172.21.15.16:0/3516907202 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f009806c450 msgr2=0x7f009806e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.851+0000 7f00b2198640 1 --2- 172.21.15.16:0/3516907202 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f009806c450 0x7f009806e910 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f00a0033000 tx=0x7f00a0009030 comp rx=0 tx=0).stop 2024-09-19T07:53:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.851+0000 7f00b2198640 1 -- 172.21.15.16:0/3516907202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0094004800 msgr2=0x7f00ac1f8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.851+0000 7f00b2198640 1 --2- 172.21.15.16:0/3516907202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0094004800 0x7f00ac1f8290 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f009c011600 tx=0x7f009c011ac0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.851+0000 7f00b2198640 1 -- 172.21.15.16:0/3516907202 shutdown_connections 2024-09-19T07:53:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.851+0000 7f00b2198640 1 --2- 172.21.15.16:0/3516907202 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f009806c450 0x7f009806e910 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.851+0000 7f00b2198640 1 --2- 172.21.15.16:0/3516907202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00ac1f3610 0x7f00ac1f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.851+0000 7f00b2198640 1 --2- 172.21.15.16:0/3516907202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00ac1e6cc0 0x7f00ac0ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.851+0000 7f00b2198640 1 --2- 172.21.15.16:0/3516907202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0094004800 0x7f00ac1f8290 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.851+0000 7f00b2198640 1 -- 172.21.15.16:0/3516907202 >> 172.21.15.16:0/3516907202 conn(0x7f00ac09d760 msgr2=0x7f00ac0b9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.851+0000 7f00b2198640 1 -- 172.21.15.16:0/3516907202 shutdown_connections 2024-09-19T07:53:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.851+0000 7f00b2198640 1 -- 172.21.15.16:0/3516907202 wait complete. 2024-09-19T07:53:58.862 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-09-19T07:53:58.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.966+0000 7f91c24f1640 1 Processor -- start 2024-09-19T07:53:58.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.966+0000 7f91c24f1640 1 -- start start 2024-09-19T07:53:58.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.967+0000 7f91c24f1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91bc15e2a0 0x7f91bc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.967+0000 7f91c24f1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f91bc15e680 0x7f91bc0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.967+0000 7f91c24f1640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91bc0d1880 con 0x7f91bc06d960 2024-09-19T07:53:58.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.967+0000 7f91c24f1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f91bc0d1b60 con 0x7f91bc15e680 2024-09-19T07:53:58.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.967+0000 7f91c24f1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f91bc0cf2d0 con 0x7f91bc15e2a0 2024-09-19T07:53:58.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.967+0000 7f91b3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f91bc15e680 0x7f91bc0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.967+0000 7f91b3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f91bc15e680 0x7f91bc0cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33882/0 (socket says 172.21.15.16:33882) 2024-09-19T07:53:58.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.967+0000 7f91b3fff640 1 -- 172.21.15.16:0/2950354360 learned_addr learned my addr 172.21.15.16:0/2950354360 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:58.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.967+0000 7f91bbfff640 1 --2- 172.21.15.16:0/2950354360 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91bc15e2a0 0x7f91bc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.968+0000 7f91b3fff640 1 -- 172.21.15.16:0/2950354360 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91bc15e2a0 msgr2=0x7f91bc06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:58.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.968+0000 7f91b3fff640 1 --2- 172.21.15.16:0/2950354360 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91bc15e2a0 0x7f91bc06d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.968+0000 7f91b3fff640 1 -- 172.21.15.16:0/2950354360 >> v1:172.21.15.146:6789/0 conn(0x7f91bc06d960 legacy=0x7f91bc0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.968+0000 7f91b3fff640 1 -- 172.21.15.16:0/2950354360 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91bc0cf5b0 con 0x7f91bc15e680 2024-09-19T07:53:58.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.968+0000 7f91b3fff640 1 --2- 172.21.15.16:0/2950354360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc15e680 0x7f91bc0cfad0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009870 tx=0x7f91a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0241ccde55bd60 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:58.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.969+0000 7f91bb7fe640 1 -- 172.21.15.16:0/2950354360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4037070 con 0x7f91bc15e680 2024-09-19T07:53:58.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.969+0000 7f91bb7fe640 1 -- 172.21.15.16:0/2950354360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a4032440 con 0x7f91bc15e680 2024-09-19T07:53:58.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.969+0000 7f91bb7fe640 1 -- 172.21.15.16:0/2950354360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4032810 con 0x7f91bc15e680 2024-09-19T07:53:58.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.969+0000 7f91c24f1640 1 -- 172.21.15.16:0/2950354360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc15e680 msgr2=0x7f91bc0cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.969+0000 7f91c24f1640 1 --2- 172.21.15.16:0/2950354360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc15e680 0x7f91bc0cfad0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009870 tx=0x7f91a402a270 comp rx=0 tx=0).stop 2024-09-19T07:53:58.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.969+0000 7f91c24f1640 1 -- 172.21.15.16:0/2950354360 shutdown_connections 2024-09-19T07:53:58.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.969+0000 7f91c24f1640 1 --2- 172.21.15.16:0/2950354360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc15e680 0x7f91bc0cfad0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.969+0000 7f91c24f1640 1 --2- 172.21.15.16:0/2950354360 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91bc15e2a0 0x7f91bc06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.969+0000 7f91c24f1640 1 -- 172.21.15.16:0/2950354360 >> 172.21.15.16:0/2950354360 conn(0x7f91bc09d760 msgr2=0x7f91bc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:58.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.969+0000 7f91c24f1640 1 -- 172.21.15.16:0/2950354360 shutdown_connections 2024-09-19T07:53:58.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.969+0000 7f91c24f1640 1 -- 172.21.15.16:0/2950354360 wait complete. 2024-09-19T07:53:58.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.970+0000 7f91c24f1640 1 Processor -- start 2024-09-19T07:53:58.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.970+0000 7f91c24f1640 1 -- start start 2024-09-19T07:53:58.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.971+0000 7f91c24f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc06d960 0x7f91bc0c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.971+0000 7f91c24f1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91bc15e2a0 0x7f91bc156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.971+0000 7f91c24f1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91bc15e680 0x7f91bc0cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.971+0000 7f91c24f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc0d4c00 con 0x7f91bc06d960 2024-09-19T07:53:58.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.971+0000 7f91c24f1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc0d4a80 con 0x7f91bc15e680 2024-09-19T07:53:58.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.971+0000 7f91c24f1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f91bc0d0010 con 0x7f91bc15e2a0 2024-09-19T07:53:58.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.971+0000 7f91bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc06d960 0x7f91bc0c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.971+0000 7f91bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc06d960 0x7f91bc0c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33890/0 (socket says 172.21.15.16:33890) 2024-09-19T07:53:58.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.971+0000 7f91bbfff640 1 -- 172.21.15.16:0/3195635626 learned_addr learned my addr 172.21.15.16:0/3195635626 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:58.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.971+0000 7f91b3fff640 1 --2- 172.21.15.16:0/3195635626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91bc15e2a0 0x7f91bc156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.971+0000 7f91c0a8e640 1 --2- 172.21.15.16:0/3195635626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91bc15e680 0x7f91bc0cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.972+0000 7f91bbfff640 1 -- 172.21.15.16:0/3195635626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91bc15e2a0 msgr2=0x7f91bc156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.972+0000 7f91bbfff640 1 --2- 172.21.15.16:0/3195635626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91bc15e2a0 0x7f91bc156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.972+0000 7f91bbfff640 1 -- 172.21.15.16:0/3195635626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91bc15e680 msgr2=0x7f91bc0cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.972+0000 7f91bbfff640 1 --2- 172.21.15.16:0/3195635626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91bc15e680 0x7f91bc0cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.972+0000 7f91bbfff640 1 -- 172.21.15.16:0/3195635626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91bc0caf20 con 0x7f91bc06d960 2024-09-19T07:53:58.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.972+0000 7f91bbfff640 1 --2- 172.21.15.16:0/3195635626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc06d960 0x7f91bc0c7f10 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00e9b0 tx=0x7f91ac00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:58.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.972+0000 7f91b97fa640 1 -- 172.21.15.16:0/3195635626 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac017070 con 0x7f91bc06d960 2024-09-19T07:53:58.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.972+0000 7f91b97fa640 1 -- 172.21.15.16:0/3195635626 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91ac00c9a0 con 0x7f91bc06d960 2024-09-19T07:53:58.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.972+0000 7f91c24f1640 1 -- 172.21.15.16:0/3195635626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91bc0cc4e0 con 0x7f91bc06d960 2024-09-19T07:53:58.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.972+0000 7f91b97fa640 1 -- 172.21.15.16:0/3195635626 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac00cd90 con 0x7f91bc06d960 2024-09-19T07:53:58.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.972+0000 7f91c24f1640 1 -- 172.21.15.16:0/3195635626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91bc0ccaa0 con 0x7f91bc06d960 2024-09-19T07:53:58.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.973+0000 7f91c24f1640 1 -- 172.21.15.16:0/3195635626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91bc06d420 con 0x7f91bc06d960 2024-09-19T07:53:58.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.973+0000 7f91b97fa640 1 -- 172.21.15.16:0/3195635626 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f91ac01a070 con 0x7f91bc06d960 2024-09-19T07:53:58.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.974+0000 7f91b97fa640 1 --2- 172.21.15.16:0/3195635626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f919c06c2b0 0x7f919c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.974+0000 7f91b97fa640 1 -- 172.21.15.16:0/3195635626 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f91ac08b9b0 con 0x7f91bc06d960 2024-09-19T07:53:58.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.974+0000 7f91b3fff640 1 --2- 172.21.15.16:0/3195635626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f919c06c2b0 0x7f919c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.975+0000 7f91b3fff640 1 --2- 172.21.15.16:0/3195635626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f919c06c2b0 0x7f919c06e770 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f91a4034000 tx=0x7f91a402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:58.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:58.979+0000 7f91b97fa640 1 -- 172.21.15.16:0/3195635626 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f91ac0598a0 con 0x7f91bc06d960 2024-09-19T07:53:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.154+0000 7f91c24f1640 1 -- 172.21.15.16:0/3195635626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7f91bc06cf90 con 0x7f91bc06d960 2024-09-19T07:53:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.155+0000 7f91b97fa640 1 -- 172.21.15.16:0/3195635626 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v47) ==== 112+0+10 (secure 0 0 0) 0x7f91ac05dba0 con 0x7f91bc06d960 2024-09-19T07:53:59.155 INFO:teuthology.orchestra.run.smithi016.stdout:60.000000 2024-09-19T07:53:59.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.156+0000 7f91c24f1640 1 -- 172.21.15.16:0/3195635626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f919c06c2b0 msgr2=0x7f919c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.156+0000 7f91c24f1640 1 --2- 172.21.15.16:0/3195635626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f919c06c2b0 0x7f919c06e770 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f91a4034000 tx=0x7f91a402e000 comp rx=0 tx=0).stop 2024-09-19T07:53:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.156+0000 7f91c24f1640 1 -- 172.21.15.16:0/3195635626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc06d960 msgr2=0x7f91bc0c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.156+0000 7f91c24f1640 1 --2- 172.21.15.16:0/3195635626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc06d960 0x7f91bc0c7f10 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00e9b0 tx=0x7f91ac00ee70 comp rx=0 tx=0).stop 2024-09-19T07:53:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.156+0000 7f91c24f1640 1 -- 172.21.15.16:0/3195635626 shutdown_connections 2024-09-19T07:53:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.156+0000 7f91c24f1640 1 --2- 172.21.15.16:0/3195635626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f919c06c2b0 0x7f919c06e770 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.156+0000 7f91c24f1640 1 --2- 172.21.15.16:0/3195635626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91bc15e680 0x7f91bc0cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.156+0000 7f91c24f1640 1 --2- 172.21.15.16:0/3195635626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91bc15e2a0 0x7f91bc156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.156+0000 7f91c24f1640 1 --2- 172.21.15.16:0/3195635626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc06d960 0x7f91bc0c7f10 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:59.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.157+0000 7f91c24f1640 1 -- 172.21.15.16:0/3195635626 >> 172.21.15.16:0/3195635626 conn(0x7f91bc09d760 msgr2=0x7f91bc0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:59.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.157+0000 7f91c24f1640 1 -- 172.21.15.16:0/3195635626 shutdown_connections 2024-09-19T07:53:59.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.157+0000 7f91c24f1640 1 -- 172.21.15.16:0/3195635626 wait complete. 2024-09-19T07:53:59.168 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:53:59.168 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:53:59.169 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:53:59.169 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:53:59.169 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:53:59.169 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:59.169 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:53:59.169 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:53:59.169 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:53:59.169 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:53:59.169 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:59.170 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:53:59.170 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:53:59.170 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:53:59.170 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:53:59.170 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:59.170 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:53:59.170 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:53:59.170 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:53:59.170 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:53:59.170 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:53:59.170 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:53:59.170 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:53:59.170 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:53:59.170 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:59.170 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:53:59.171 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:53:59.250 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.69634.asok 2024-09-19T07:53:59.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:53:59.261 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.69634.asok 2024-09-19T07:53:59.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:53:59.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.69634.asok mds_sessions 2024-09-19T07:53:59.398 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:53:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7551, 2024-09-19T07:53:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:53:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:53:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:53:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 7551 2024-09-19T07:53:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:53:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:53:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:53:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1440089416 2024-09-19T07:53:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.7551 v1:192.168.0.1:0/1440089416", 2024-09-19T07:53:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "v1:192.168.0.1:0/1440089416", 2024-09-19T07:53:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions": [ 2024-09-19T07:53:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": 0, 2024-09-19T07:53:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: "addrs": { 2024-09-19T07:53:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-19T07:53:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-19T07:53:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6834", 2024-09-19T07:53:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2050126415 2024-09-19T07:53:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:53:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:53:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6835", 2024-09-19T07:53:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2050126415 2024-09-19T07:53:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T07:53:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:53:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: "seq": 0, 2024-09-19T07:53:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_gen": 0, 2024-09-19T07:53:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_ttl": "2024-09-19T07:54:55.931993+0000", 2024-09-19T07:53:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: "last_cap_renew_request": "2024-09-19T07:53:55.931993+0000", 2024-09-19T07:53:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_renew_seq": 2, 2024-09-19T07:53:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: "num_caps": 1, 2024-09-19T07:53:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "open" 2024-09-19T07:53:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:53:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:53:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 63 2024-09-19T07:53:59.401 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:53:59.406 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:53:59.406 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:53:59.423 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-19T07:53:59.423 DEBUG:teuthology.orchestra.run.smithi016:> import fcntl 2024-09-19T07:53:59.423 DEBUG:teuthology.orchestra.run.smithi016:> import sys 2024-09-19T07:53:59.423 DEBUG:teuthology.orchestra.run.smithi016:> import time 2024-09-19T07:53:59.423 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:53:59.423 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:59.423 DEBUG:teuthology.orchestra.run.smithi016:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-19T07:53:59.423 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:59.423 DEBUG:teuthology.orchestra.run.smithi016:> n = 800 2024-09-19T07:53:59.424 DEBUG:teuthology.orchestra.run.smithi016:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2024-09-19T07:53:59.424 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:59.424 DEBUG:teuthology.orchestra.run.smithi016:> if not os.path.exists(abs_path): 2024-09-19T07:53:59.424 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(abs_path) 2024-09-19T07:53:59.424 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:59.424 DEBUG:teuthology.orchestra.run.smithi016:> handles = [] 2024-09-19T07:53:59.424 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, n): 2024-09-19T07:53:59.424 DEBUG:teuthology.orchestra.run.smithi016:> fname = "file_"+str(i) 2024-09-19T07:53:59.424 DEBUG:teuthology.orchestra.run.smithi016:> path = os.path.join(abs_path, fname) 2024-09-19T07:53:59.424 DEBUG:teuthology.orchestra.run.smithi016:> handles.append(open(path, '"'"'w'"'"')) 2024-09-19T07:53:59.424 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:53:59.424 DEBUG:teuthology.orchestra.run.smithi016:> print("waiting with handles open", file=sys.stderr) 2024-09-19T07:53:59.424 DEBUG:teuthology.orchestra.run.smithi016:> while True: 2024-09-19T07:53:59.424 DEBUG:teuthology.orchestra.run.smithi016:> print("open_n_background: keeping files open", file=sys.stderr) 2024-09-19T07:53:59.425 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-19T07:53:59.425 DEBUG:teuthology.orchestra.run.smithi016:> if os.read(0, 4096) == b"": 2024-09-19T07:53:59.425 DEBUG:teuthology.orchestra.run.smithi016:> break 2024-09-19T07:53:59.425 DEBUG:teuthology.orchestra.run.smithi016:> except BlockingIOError: 2024-09-19T07:53:59.425 DEBUG:teuthology.orchestra.run.smithi016:> pass 2024-09-19T07:53:59.425 DEBUG:teuthology.orchestra.run.smithi016:> time.sleep(2) 2024-09-19T07:53:59.425 DEBUG:teuthology.orchestra.run.smithi016:> print("stdin closed, goodbye!", file=sys.stderr) 2024-09-19T07:53:59.425 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:53:59.469 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:53:59.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.575+0000 7f323aa47640 1 Processor -- start 2024-09-19T07:53:59.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.575+0000 7f323aa47640 1 -- start start 2024-09-19T07:53:59.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.576+0000 7f323aa47640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32340b03c0 0x7f32340cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:59.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.576+0000 7f323aa47640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32340b3c10 con 0x7f323415e7f0 2024-09-19T07:53:59.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.576+0000 7f323aa47640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32340cb900 con 0x7f32340cc680 2024-09-19T07:53:59.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.576+0000 7f323aa47640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f32340cc1f0 con 0x7f32340b03c0 2024-09-19T07:53:59.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.576+0000 7f32337fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32340b03c0 0x7f32340cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:59.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.576+0000 7f32337fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32340b03c0 0x7f32340cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33894/0 (socket says 172.21.15.16:33894) 2024-09-19T07:53:59.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.576+0000 7f3233fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f323415e7f0 0x7f32340adff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35230/0 (socket says 172.21.15.16:35230) 2024-09-19T07:53:59.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.576+0000 7f32337fe640 1 -- 172.21.15.16:0/4187948186 learned_addr learned my addr 172.21.15.16:0/4187948186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:59.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.576+0000 7f32337fe640 1 -- 172.21.15.16:0/4187948186 >> v1:172.21.15.146:6789/0 conn(0x7f32340cc680 legacy=0x7f323406bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:59.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.576+0000 7f32337fe640 1 -- 172.21.15.16:0/4187948186 >> v1:172.21.15.16:6789/0 conn(0x7f323415e7f0 legacy=0x7f32340adff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:59.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.576+0000 7f32337fe640 1 -- 172.21.15.16:0/4187948186 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323415ebd0 con 0x7f32340b03c0 2024-09-19T07:53:59.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.577+0000 7f32337fe640 1 --2- 172.21.15.16:0/4187948186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32340b03c0 0x7f32340cd360 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f3228009870 tx=0x7f322802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae97bf241f278 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:59.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.577+0000 7f3232ffd640 1 -- 172.21.15.16:0/4187948186 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228037070 con 0x7f32340b03c0 2024-09-19T07:53:59.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.577+0000 7f3232ffd640 1 -- 172.21.15.16:0/4187948186 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3228032440 con 0x7f32340b03c0 2024-09-19T07:53:59.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.577+0000 7f3232ffd640 1 -- 172.21.15.16:0/4187948186 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228032810 con 0x7f32340b03c0 2024-09-19T07:53:59.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.578+0000 7f323aa47640 1 -- 172.21.15.16:0/4187948186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32340b03c0 msgr2=0x7f32340cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:59.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.578+0000 7f323aa47640 1 --2- 172.21.15.16:0/4187948186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32340b03c0 0x7f32340cd360 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f3228009870 tx=0x7f322802a210 comp rx=0 tx=0).stop 2024-09-19T07:53:59.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.578+0000 7f323aa47640 1 -- 172.21.15.16:0/4187948186 shutdown_connections 2024-09-19T07:53:59.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.578+0000 7f323aa47640 1 --2- 172.21.15.16:0/4187948186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32340b03c0 0x7f32340cd360 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:59.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.578+0000 7f323aa47640 1 -- 172.21.15.16:0/4187948186 >> 172.21.15.16:0/4187948186 conn(0x7f32340b2d10 msgr2=0x7f32340b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:59.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.578+0000 7f323aa47640 1 -- 172.21.15.16:0/4187948186 shutdown_connections 2024-09-19T07:53:59.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.578+0000 7f323aa47640 1 -- 172.21.15.16:0/4187948186 wait complete. 2024-09-19T07:53:59.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.579+0000 7f323aa47640 1 Processor -- start 2024-09-19T07:53:59.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.579+0000 7f323aa47640 1 -- start start 2024-09-19T07:53:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.580+0000 7f323aa47640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32340b03c0 0x7f32340b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.580+0000 7f323aa47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32340cc680 0x7f32340c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.580+0000 7f323aa47640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f323415e7f0 0x7f32340b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.580+0000 7f323aa47640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32340ae700 con 0x7f32340cc680 2024-09-19T07:53:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.580+0000 7f323aa47640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f32340cbe80 con 0x7f32340b03c0 2024-09-19T07:53:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.580+0000 7f323aa47640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f32340b3830 con 0x7f323415e7f0 2024-09-19T07:53:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.580+0000 7f32337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32340cc680 0x7f32340c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.580+0000 7f32337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32340cc680 0x7f32340c5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33902/0 (socket says 172.21.15.16:33902) 2024-09-19T07:53:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.580+0000 7f32337fe640 1 -- 172.21.15.16:0/802712996 learned_addr learned my addr 172.21.15.16:0/802712996 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:53:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.581+0000 7f3233fff640 1 --2- 172.21.15.16:0/802712996 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32340b03c0 0x7f32340b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.581+0000 7f3238fe4640 1 --2- 172.21.15.16:0/802712996 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f323415e7f0 0x7f32340b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.581+0000 7f32337fe640 1 -- 172.21.15.16:0/802712996 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f323415e7f0 msgr2=0x7f32340b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.581+0000 7f32337fe640 1 --2- 172.21.15.16:0/802712996 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f323415e7f0 0x7f32340b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.581+0000 7f32337fe640 1 -- 172.21.15.16:0/802712996 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32340b03c0 msgr2=0x7f32340b6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.581+0000 7f32337fe640 1 --2- 172.21.15.16:0/802712996 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32340b03c0 0x7f32340b6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.581+0000 7f32337fe640 1 -- 172.21.15.16:0/802712996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32340b5e90 con 0x7f32340cc680 2024-09-19T07:53:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.581+0000 7f32337fe640 1 --2- 172.21.15.16:0/802712996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32340cc680 0x7f32340c5c40 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f322802a250 tx=0x7f32280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.581+0000 7f3230ff9640 1 -- 172.21.15.16:0/802712996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228037040 con 0x7f32340cc680 2024-09-19T07:53:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.581+0000 7f3230ff9640 1 -- 172.21.15.16:0/802712996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3228042070 con 0x7f32340cc680 2024-09-19T07:53:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.581+0000 7f323aa47640 1 -- 172.21.15.16:0/802712996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32340b6170 con 0x7f32340cc680 2024-09-19T07:53:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.581+0000 7f3230ff9640 1 -- 172.21.15.16:0/802712996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228030bd0 con 0x7f32340cc680 2024-09-19T07:53:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.581+0000 7f323aa47640 1 -- 172.21.15.16:0/802712996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32340c6320 con 0x7f32340cc680 2024-09-19T07:53:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.582+0000 7f323aa47640 1 -- 172.21.15.16:0/802712996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32340cd360 con 0x7f32340cc680 2024-09-19T07:53:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.583+0000 7f3230ff9640 1 -- 172.21.15.16:0/802712996 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3228047460 con 0x7f32340cc680 2024-09-19T07:53:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.583+0000 7f3230ff9640 1 --2- 172.21.15.16:0/802712996 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f322006c2b0 0x7f322006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.583+0000 7f3230ff9640 1 -- 172.21.15.16:0/802712996 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f32280ac1a0 con 0x7f32340cc680 2024-09-19T07:53:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.584+0000 7f3233fff640 1 --2- 172.21.15.16:0/802712996 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f322006c2b0 0x7f322006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.584+0000 7f3233fff640 1 --2- 172.21.15.16:0/802712996 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f322006c2b0 0x7f322006e770 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3224000f60 tx=0x7f3224007cc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.588+0000 7f3230ff9640 1 -- 172.21.15.16:0/802712996 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3228045360 con 0x7f32340cc680 2024-09-19T07:53:59.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.868+0000 7f323aa47640 1 -- 172.21.15.16:0/802712996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32340b2ac0 con 0x7f32340cc680 2024-09-19T07:53:59.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.869+0000 7f3230ff9640 1 -- 172.21.15.16:0/802712996 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+5231 (secure 0 0 0) 0x7f322807a780 con 0x7f32340cc680 2024-09-19T07:53:59.870 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:53:59.870 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":63,"btime":"2024-09-19T07:53:46:466514+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2471408352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2471408352},{"type":"v1","addr":"172.21.15.16:6833","nonce":2471408352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/1281369866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1281369866},{"type":"v1","addr":"172.21.15.146:6836","nonce":1281369866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1377303883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1377303883},{"type":"v1","addr":"172.21.15.146:6837","nonce":1377303883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:53:43.812066+0000","modified":"2024-09-19T07:53:45.809688+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7368":{"gid":7368,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2050126415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2050126415},{"type":"v1","addr":"172.21.15.16:6835","nonce":2050126415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7368,"qdb_cluster":[7368]},"id":8}]} 2024-09-19T07:53:59.870 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 63 2024-09-19T07:53:59.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.871+0000 7f323aa47640 1 -- 172.21.15.16:0/802712996 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f322006c2b0 msgr2=0x7f322006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:59.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.871+0000 7f323aa47640 1 --2- 172.21.15.16:0/802712996 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f322006c2b0 0x7f322006e770 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3224000f60 tx=0x7f3224007cc0 comp rx=0 tx=0).stop 2024-09-19T07:53:59.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.871+0000 7f323aa47640 1 -- 172.21.15.16:0/802712996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32340cc680 msgr2=0x7f32340c5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:59.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.871+0000 7f323aa47640 1 --2- 172.21.15.16:0/802712996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32340cc680 0x7f32340c5c40 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f322802a250 tx=0x7f32280046c0 comp rx=0 tx=0).stop 2024-09-19T07:53:59.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.871+0000 7f323aa47640 1 -- 172.21.15.16:0/802712996 shutdown_connections 2024-09-19T07:53:59.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.871+0000 7f323aa47640 1 --2- 172.21.15.16:0/802712996 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f322006c2b0 0x7f322006e770 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:59.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.871+0000 7f323aa47640 1 --2- 172.21.15.16:0/802712996 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f323415e7f0 0x7f32340b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:59.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.871+0000 7f323aa47640 1 --2- 172.21.15.16:0/802712996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32340cc680 0x7f32340c5c40 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:59.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.871+0000 7f323aa47640 1 --2- 172.21.15.16:0/802712996 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32340b03c0 0x7f32340b6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:59.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.871+0000 7f323aa47640 1 -- 172.21.15.16:0/802712996 >> 172.21.15.16:0/802712996 conn(0x7f32340b2d10 msgr2=0x7f32340afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:59.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.872+0000 7f323aa47640 1 -- 172.21.15.16:0/802712996 shutdown_connections 2024-09-19T07:53:59.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:53:59.872+0000 7f323aa47640 1 -- 172.21.15.16:0/802712996 wait complete. 2024-09-19T07:53:59.885 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-19T07:54:00.017 INFO:teuthology.orchestra.run.smithi016.stdout:[{"id":6374,"entity":{"name":{"type":"client","num":6374},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3044582375}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":3.5439502780000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.989665252386438,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6374 192.168.0.1:0/3044582375","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"48393","root":"/"}},{"id":7551,"entity":{"name":{"type":"client","num":7551},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1440089416}},"state":"open","num_leases":1,"num_caps":379,"request_load_avg":379,"uptime":5.0759287850000003,"requests_in_flight":377,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":759.56348328189665,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":5,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7551 192.168.0.1:0/1440089416","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"0","hostname":"smithi016","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"69634","root":"/"}}] 2024-09-19T07:54:00.018 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "48393", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3044582375, "type": "any" }, "name": { "num": 6374, "type": "client" } }, "id": 6374, "inst": "client.6374 192.168.0.1:0/3044582375", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.989665252386438 }, "state": "open", "uptime": 3.543950278 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "0", "hostname": "smithi016", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "69634", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1440089416, "type": "any" }, "name": { "num": 7551, "type": "client" } }, "id": 7551, "inst": "client.7551 192.168.0.1:0/1440089416", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 5, "num_caps": 379, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 379, "requests_in_flight": 377, "session_cache_liveness": { "halflife": 300, "value": 759.5634832818967 }, "state": "open", "uptime": 5.075928785 } ] 2024-09-19T07:54:00.018 DEBUG:tasks.ceph_test_case:wait_until_equal: 379 != 802, waiting (timeout=600)... 2024-09-19T07:54:00.865 INFO:teuthology.orchestra.run.smithi016.stderr:waiting with handles open 2024-09-19T07:54:00.866 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:54:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:54:04.869 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:54:05.019 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:54:05.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.134+0000 7f13aac7b640 1 Processor -- start 2024-09-19T07:54:05.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.134+0000 7f13aac7b640 1 -- start start 2024-09-19T07:54:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.135+0000 7f13aac7b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13a40d0bc0 0x7f13a40cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.135+0000 7f13aac7b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13a415c2c0 0x7f13a40cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.135+0000 7f13aac7b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13a40cfc80 con 0x7f13a415bee0 2024-09-19T07:54:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.135+0000 7f13aac7b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f13a40ceaf0 con 0x7f13a415c2c0 2024-09-19T07:54:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.135+0000 7f13aac7b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f13a40cdee0 con 0x7f13a40d0bc0 2024-09-19T07:54:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.135+0000 7f13a9218640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13a415c2c0 0x7f13a40cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.135+0000 7f13a9218640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13a415c2c0 0x7f13a40cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38398/0 (socket says 172.21.15.16:38398) 2024-09-19T07:54:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.135+0000 7f13a9218640 1 -- 172.21.15.16:0/265722030 learned_addr learned my addr 172.21.15.16:0/265722030 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.135+0000 7f139bfff640 1 --2- 172.21.15.16:0/265722030 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13a40d0bc0 0x7f13a40cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.135+0000 7f13a9218640 1 -- 172.21.15.16:0/265722030 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13a40d0bc0 msgr2=0x7f13a40cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.135+0000 7f13a9218640 1 --2- 172.21.15.16:0/265722030 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13a40d0bc0 0x7f13a40cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.135+0000 7f13a9218640 1 -- 172.21.15.16:0/265722030 >> v1:172.21.15.146:6789/0 conn(0x7f13a415bee0 legacy=0x7f13a40d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.135+0000 7f13a9218640 1 -- 172.21.15.16:0/265722030 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a40ce1c0 con 0x7f13a415c2c0 2024-09-19T07:54:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.136+0000 7f13a9218640 1 --2- 172.21.15.16:0/265722030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13a415c2c0 0x7f13a40cf740 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f139c009e50 tx=0x7f139c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91ecb20e4dbf954 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.136+0000 7f139b7fe640 1 -- 172.21.15.16:0/265722030 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c036070 con 0x7f13a415c2c0 2024-09-19T07:54:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.136+0000 7f139b7fe640 1 -- 172.21.15.16:0/265722030 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f139c031600 con 0x7f13a415c2c0 2024-09-19T07:54:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.136+0000 7f139b7fe640 1 -- 172.21.15.16:0/265722030 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c0319d0 con 0x7f13a415c2c0 2024-09-19T07:54:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.137+0000 7f13aac7b640 1 -- 172.21.15.16:0/265722030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13a415c2c0 msgr2=0x7f13a40cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:05.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.137+0000 7f13aac7b640 1 --2- 172.21.15.16:0/265722030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13a415c2c0 0x7f13a40cf740 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f139c009e50 tx=0x7f139c02a710 comp rx=0 tx=0).stop 2024-09-19T07:54:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.137+0000 7f13aac7b640 1 -- 172.21.15.16:0/265722030 shutdown_connections 2024-09-19T07:54:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.137+0000 7f13aac7b640 1 --2- 172.21.15.16:0/265722030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13a415c2c0 0x7f13a40cf740 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.137+0000 7f13aac7b640 1 --2- 172.21.15.16:0/265722030 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13a40d0bc0 0x7f13a40cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.137+0000 7f13aac7b640 1 -- 172.21.15.16:0/265722030 >> 172.21.15.16:0/265722030 conn(0x7f13a406bdf0 msgr2=0x7f13a406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.137+0000 7f13aac7b640 1 -- 172.21.15.16:0/265722030 shutdown_connections 2024-09-19T07:54:05.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.137+0000 7f13aac7b640 1 -- 172.21.15.16:0/265722030 wait complete. 2024-09-19T07:54:05.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.138+0000 7f13aac7b640 1 Processor -- start 2024-09-19T07:54:05.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.138+0000 7f13aac7b640 1 -- start start 2024-09-19T07:54:05.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.139+0000 7f13aac7b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13a40d0bc0 0x7f13a4142200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.139+0000 7f13aac7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13a415bee0 0x7f13a4146210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.139+0000 7f13aac7b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13a415c2c0 0x7f13a4140170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.139+0000 7f13aac7b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13a406dec0 con 0x7f13a415bee0 2024-09-19T07:54:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.139+0000 7f13aac7b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f13a40ce780 con 0x7f13a40d0bc0 2024-09-19T07:54:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.139+0000 7f13aac7b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f13a406e040 con 0x7f13a415c2c0 2024-09-19T07:54:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.140+0000 7f139bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13a415bee0 0x7f13a4146210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.140+0000 7f139bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13a415bee0 0x7f13a4146210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38410/0 (socket says 172.21.15.16:38410) 2024-09-19T07:54:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.140+0000 7f139bfff640 1 -- 172.21.15.16:0/1151942587 learned_addr learned my addr 172.21.15.16:0/1151942587 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.140+0000 7f13a8a17640 1 --2- 172.21.15.16:0/1151942587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13a40d0bc0 0x7f13a4142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.140+0000 7f13a9218640 1 --2- 172.21.15.16:0/1151942587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13a415c2c0 0x7f13a4140170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.140+0000 7f139bfff640 1 -- 172.21.15.16:0/1151942587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13a415c2c0 msgr2=0x7f13a4140170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:05.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.140+0000 7f139bfff640 1 --2- 172.21.15.16:0/1151942587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13a415c2c0 0x7f13a4140170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.140+0000 7f139bfff640 1 -- 172.21.15.16:0/1151942587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13a40d0bc0 msgr2=0x7f13a4142200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:05.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.140+0000 7f139bfff640 1 --2- 172.21.15.16:0/1151942587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13a40d0bc0 0x7f13a4142200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.140+0000 7f139bfff640 1 -- 172.21.15.16:0/1151942587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a4142850 con 0x7f13a415bee0 2024-09-19T07:54:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.140+0000 7f139bfff640 1 --2- 172.21.15.16:0/1151942587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13a415bee0 0x7f13a4146210 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f139400d5d0 tx=0x7f139400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.140+0000 7f13997fa640 1 -- 172.21.15.16:0/1151942587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394004240 con 0x7f13a415bee0 2024-09-19T07:54:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.140+0000 7f13997fa640 1 -- 172.21.15.16:0/1151942587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13940043e0 con 0x7f13a415bee0 2024-09-19T07:54:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.140+0000 7f13aac7b640 1 -- 172.21.15.16:0/1151942587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a4142b30 con 0x7f13a415bee0 2024-09-19T07:54:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.140+0000 7f13997fa640 1 -- 172.21.15.16:0/1151942587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394003830 con 0x7f13a415bee0 2024-09-19T07:54:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.140+0000 7f13aac7b640 1 -- 172.21.15.16:0/1151942587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a41430f0 con 0x7f13a415bee0 2024-09-19T07:54:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.141+0000 7f13aac7b640 1 -- 172.21.15.16:0/1151942587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a40d04b0 con 0x7f13a415bee0 2024-09-19T07:54:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.142+0000 7f13997fa640 1 -- 172.21.15.16:0/1151942587 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f13940039d0 con 0x7f13a415bee0 2024-09-19T07:54:05.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.142+0000 7f13997fa640 1 --2- 172.21.15.16:0/1151942587 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f139006c2b0 0x7f139006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.142+0000 7f13997fa640 1 -- 172.21.15.16:0/1151942587 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f139408b700 con 0x7f13a415bee0 2024-09-19T07:54:05.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.143+0000 7f13a8a17640 1 --2- 172.21.15.16:0/1151942587 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f139006c2b0 0x7f139006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.143+0000 7f13a8a17640 1 --2- 172.21.15.16:0/1151942587 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f139006c2b0 0x7f139006e770 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f13a0001780 tx=0x7f13a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:05.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.147+0000 7f13997fa640 1 -- 172.21.15.16:0/1151942587 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f13940596a0 con 0x7f13a415bee0 2024-09-19T07:54:05.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.359+0000 7f13aac7b640 1 -- 172.21.15.16:0/1151942587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13a40af170 con 0x7f13a415bee0 2024-09-19T07:54:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.360+0000 7f13997fa640 1 -- 172.21.15.16:0/1151942587 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5231 (secure 0 0 0) 0x7f139405d9a0 con 0x7f13a415bee0 2024-09-19T07:54:05.360 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:05.360 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":65,"btime":"2024-09-19T07:54:05:204476+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2471408352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2471408352},{"type":"v1","addr":"172.21.15.16:6833","nonce":2471408352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/1281369866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1281369866},{"type":"v1","addr":"172.21.15.146:6836","nonce":1281369866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1377303883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1377303883},{"type":"v1","addr":"172.21.15.146:6837","nonce":1377303883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:53:43.812066+0000","modified":"2024-09-19T07:54:04.488075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7368":{"gid":7368,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/2050126415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2050126415},{"type":"v1","addr":"172.21.15.16:6835","nonce":2050126415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7368,"qdb_cluster":[7368]},"id":8}]} 2024-09-19T07:54:05.361 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 65 2024-09-19T07:54:05.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.362+0000 7f13aac7b640 1 -- 172.21.15.16:0/1151942587 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f139006c2b0 msgr2=0x7f139006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:05.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.362+0000 7f13aac7b640 1 --2- 172.21.15.16:0/1151942587 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f139006c2b0 0x7f139006e770 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f13a0001780 tx=0x7f13a0009040 comp rx=0 tx=0).stop 2024-09-19T07:54:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.362+0000 7f13aac7b640 1 -- 172.21.15.16:0/1151942587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13a415bee0 msgr2=0x7f13a4146210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.362+0000 7f13aac7b640 1 --2- 172.21.15.16:0/1151942587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13a415bee0 0x7f13a4146210 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f139400d5d0 tx=0x7f139400da90 comp rx=0 tx=0).stop 2024-09-19T07:54:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.362+0000 7f13aac7b640 1 -- 172.21.15.16:0/1151942587 shutdown_connections 2024-09-19T07:54:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.362+0000 7f13aac7b640 1 --2- 172.21.15.16:0/1151942587 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f139006c2b0 0x7f139006e770 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.362+0000 7f13aac7b640 1 --2- 172.21.15.16:0/1151942587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13a415c2c0 0x7f13a4140170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.362+0000 7f13aac7b640 1 --2- 172.21.15.16:0/1151942587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13a415bee0 0x7f13a4146210 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.362+0000 7f13aac7b640 1 --2- 172.21.15.16:0/1151942587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13a40d0bc0 0x7f13a4142200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.362+0000 7f13aac7b640 1 -- 172.21.15.16:0/1151942587 >> 172.21.15.16:0/1151942587 conn(0x7f13a406bdf0 msgr2=0x7f13a40b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.362+0000 7f13aac7b640 1 -- 172.21.15.16:0/1151942587 shutdown_connections 2024-09-19T07:54:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.362+0000 7f13aac7b640 1 -- 172.21.15.16:0/1151942587 wait complete. 2024-09-19T07:54:05.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-19T07:54:05.496 INFO:teuthology.orchestra.run.smithi016.stdout:[{"id":6374,"entity":{"name":{"type":"client","num":6374},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3044582375}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":9.0228734070000005,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9520576185111578,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6374 192.168.0.1:0/3044582375","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"48393","root":"/"}},{"id":7551,"entity":{"name":{"type":"client","num":7551},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1440089416}},"state":"open","num_leases":1,"num_caps":802,"request_load_avg":1095,"uptime":10.554851914,"requests_in_flight":0,"num_completed_requests":299,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":2331.4225414343996,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":1001.4289494857182,"halflife":1.5},"recall_caps_throttle2o":{"value":514.86246561826761,"halflife":0.5},"session_cache_liveness":{"value":1586.161946597183,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":506,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7551 192.168.0.1:0/1440089416","completed_requests":[{"tid":506,"created_ino":"0x100000003e9"},{"tid":507,"created_ino":"0x100000003ea"},{"tid":508,"created_ino":"0x100000003eb"},{"tid":509,"created_ino":"0x100000003ec"},{"tid":510,"created_ino":"0x100000003ed"},{"tid":511,"created_ino":"0x100000003ee"},{"tid":512,"created_ino":"0x100000003ef"},{"tid":513,"created_ino":"0x100000003f0"},{"tid":514,"created_ino":"0x100000003f1"},{"tid":515,"created_ino":"0x100000003f2"},{"tid":516,"created_ino":"0x100000003f3"},{"tid":517,"created_ino":"0x100000003f4"},{"tid":518,"created_ino":"0x100000003f5"},{"tid":519,"created_ino":"0x100000003f6"},{"tid":520,"created_ino":"0x100000003f7"},{"tid":521,"created_ino":"0x100000003f8"},{"tid":522,"created_ino":"0x100000003f9"},{"tid":523,"created_ino":"0x100000003fa"},{"tid":524,"created_ino":"0x100000003fb"},{"tid":525,"created_ino":"0x100000003fc"},{"tid":526,"created_ino":"0x100000003fd"},{"tid":527,"created_ino":"0x100000003fe"},{"tid":528,"created_ino":"0x100000003ff"},{"tid":529,"created_ino":"0x10000000400"},{"tid":530,"created_ino":"0x10000000401"},{"tid":531,"created_ino":"0x10000000402"},{"tid":532,"created_ino":"0x10000000403"},{"tid":533,"created_ino":"0x10000000404"},{"tid":534,"created_ino":"0x10000000405"},{"tid":535,"created_ino":"0x10000000406"},{"tid":536,"created_ino":"0x10000000407"},{"tid":537,"created_ino":"0x10000000408"},{"tid":538,"created_ino":"0x10000000409"},{"tid":539,"created_ino":"0x1000000040a"},{"tid":540,"created_ino":"0x1000000040b"},{"tid":541,"created_ino":"0x1000000040c"},{"tid":542,"created_ino":"0x1000000040d"},{"tid":543,"created_ino":"0x1000000040e"},{"tid":544,"created_ino":"0x1000000040f"},{"tid":545,"created_ino":"0x10000000410"},{"tid":546,"created_ino":"0x10000000411"},{"tid":547,"created_ino":"0x10000000412"},{"tid":548,"created_ino":"0x10000000413"},{"tid":549,"created_ino":"0x10000000414"},{"tid":550,"created_ino":"0x10000000415"},{"tid":551,"created_ino":"0x10000000416"},{"tid":552,"created_ino":"0x10000000417"},{"tid":553,"created_ino":"0x10000000418"},{"tid":554,"created_ino":"0x10000000419"},{"tid":555,"created_ino":"0x1000000041a"},{"tid":556,"created_ino":"0x1000000041b"},{"tid":557,"created_ino":"0x1000000041c"},{"tid":558,"created_ino":"0x1000000041d"},{"tid":559,"created_ino":"0x1000000041e"},{"tid":560,"created_ino":"0x1000000041f"},{"tid":561,"created_ino":"0x10000000420"},{"tid":562,"created_ino":"0x10000000421"},{"tid":563,"created_ino":"0x10000000422"},{"tid":564,"created_ino":"0x10000000423"},{"tid":565,"created_ino":"0x10000000424"},{"tid":566,"created_ino":"0x10000000425"},{"tid":567,"created_ino":"0x10000000426"},{"tid":568,"created_ino":"0x10000000427"},{"tid":569,"created_ino":"0x10000000428"},{"tid":570,"created_ino":"0x10000000429"},{"tid":571,"created_ino":"0x1000000042a"},{"tid":572,"created_ino":"0x1000000042b"},{"tid":573,"created_ino":"0x1000000042c"},{"tid":574,"created_ino":"0x1000000042d"},{"tid":575,"created_ino":"0x1000000042e"},{"tid":576,"created_ino":"0x1000000042f"},{"tid":577,"created_ino":"0x10000000430"},{"tid":578,"created_ino":"0x10000000431"},{"tid":579,"created_ino":"0x10000000432"},{"tid":580,"created_ino":"0x10000000433"},{"tid":581,"created_ino":"0x10000000434"},{"tid":582,"created_ino":"0x10000000435"},{"tid":583,"created_ino":"0x10000000436"},{"tid":584,"created_ino":"0x10000000437"},{"tid":585,"created_ino":"0x10000000438"},{"tid":586,"created_ino":"0x10000000439"},{"tid":587,"created_ino":"0x1000000043a"},{"tid":588,"created_ino":"0x1000000043b"},{"tid":589,"created_ino":"0x1000000043c"},{"tid":590,"created_ino":"0x1000000043d"},{"tid":591,"created_ino":"0x1000000043e"},{"tid":592,"created_ino":"0x1000000043f"},{"tid":593,"created_ino":"0x10000000440"},{"tid":594,"created_ino":"0x10000000441"},{"tid":595,"created_ino":"0x10000000442"},{"tid":596,"created_ino":"0x10000000443"},{"tid":597,"created_ino":"0x10000000444"},{"tid":598,"created_ino":"0x10000000445"},{"tid":599,"created_ino":"0x10000000446"},{"tid":600,"created_ino":"0x10000000447"},{"tid":601,"created_ino":"0x10000000448"},{"tid":602,"created_ino":"0x10000000449"},{"tid":603,"created_ino":"0x1000000044a"},{"tid":604,"created_ino":"0x1000000044b"},{"tid":605,"created_ino":"0x1000000044c"},{"tid":606,"created_ino":"0x1000000044d"},{"tid":607,"created_ino":"0x1000000044e"},{"tid":608,"created_ino":"0x1000000044f"},{"tid":609,"created_ino":"0x10000000450"},{"tid":610,"created_ino":"0x10000000451"},{"tid":611,"created_ino":"0x10000000452"},{"tid":612,"created_ino":"0x10000000453"},{"tid":613,"created_ino":"0x10000000454"},{"tid":614,"created_ino":"0x10000000455"},{"tid":615,"created_ino":"0x10000000456"},{"tid":616,"created_ino":"0x10000000457"},{"tid":617,"created_ino":"0x10000000458"},{"tid":618,"created_ino":"0x10000000459"},{"tid":619,"created_ino":"0x1000000045a"},{"tid":620,"created_ino":"0x1000000045b"},{"tid":621,"created_ino":"0x1000000045c"},{"tid":622,"created_ino":"0x1000000045d"},{"tid":623,"created_ino":"0x1000000045e"},{"tid":624,"created_ino":"0x1000000045f"},{"tid":625,"created_ino":"0x10000000460"},{"tid":628,"created_ino":"0x10000000461"},{"tid":632,"created_ino":"0x10000000462"},{"tid":635,"created_ino":"0x10000000463"},{"tid":638,"created_ino":"0x10000000464"},{"tid":641,"created_ino":"0x10000000465"},{"tid":644,"created_ino":"0x10000000466"},{"tid":647,"created_ino":"0x10000000467"},{"tid":650,"created_ino":"0x10000000468"},{"tid":653,"created_ino":"0x10000000469"},{"tid":656,"created_ino":"0x1000000046a"},{"tid":659,"created_ino":"0x1000000046b"},{"tid":662,"created_ino":"0x1000000046c"},{"tid":665,"created_ino":"0x1000000046d"},{"tid":668,"created_ino":"0x1000000046e"},{"tid":671,"created_ino":"0x1000000046f"},{"tid":674,"created_ino":"0x10000000470"},{"tid":677,"created_ino":"0x10000000471"},{"tid":680,"created_ino":"0x10000000472"},{"tid":683,"created_ino":"0x10000000473"},{"tid":686,"created_ino":"0x10000000474"},{"tid":689,"created_ino":"0x10000000475"},{"tid":692,"created_ino":"0x10000000476"},{"tid":695,"created_ino":"0x10000000477"},{"tid":698,"created_ino":"0x10000000478"},{"tid":701,"created_ino":"0x10000000479"},{"tid":704,"created_ino":"0x1000000047a"},{"tid":707,"created_ino":"0x1000000047b"},{"tid":710,"created_ino":"0x1000000047c"},{"tid":713,"created_ino":"0x1000000047d"},{"tid":716,"created_ino":"0x1000000047e"},{"tid":719,"created_ino":"0x1000000047f"},{"tid":722,"created_ino":"0x10000000480"},{"tid":725,"created_ino":"0x10000000481"},{"tid":728,"created_ino":"0x10000000482"},{"tid":731,"created_ino":"0x10000000483"},{"tid":734,"created_ino":"0x10000000484"},{"tid":737,"created_ino":"0x10000000485"},{"tid":740,"created_ino":"0x10000000486"},{"tid":743,"created_ino":"0x10000000487"},{"tid":746,"created_ino":"0x10000000488"},{"tid":749,"created_ino":"0x10000000489"},{"tid":752,"created_ino":"0x1000000048a"},{"tid":755,"created_ino":"0x1000000048b"},{"tid":758,"created_ino":"0x1000000048c"},{"tid":761,"created_ino":"0x1000000048d"},{"tid":764,"created_ino":"0x1000000048e"},{"tid":767,"created_ino":"0x1000000048f"},{"tid":770,"created_ino":"0x10000000490"},{"tid":773,"created_ino":"0x10000000491"},{"tid":776,"created_ino":"0x10000000492"},{"tid":779,"created_ino":"0x10000000493"},{"tid":782,"created_ino":"0x10000000494"},{"tid":785,"created_ino":"0x10000000495"},{"tid":788,"created_ino":"0x10000000496"},{"tid":791,"created_ino":"0x10000000497"},{"tid":794,"created_ino":"0x10000000498"},{"tid":797,"created_ino":"0x10000000499"},{"tid":800,"created_ino":"0x1000000049a"},{"tid":803,"created_ino":"0x1000000049b"},{"tid":806,"created_ino":"0x1000000049c"},{"tid":809,"created_ino":"0x1000000049d"},{"tid":812,"created_ino":"0x1000000049e"},{"tid":815,"created_ino":"0x1000000049f"},{"tid":818,"created_ino":"0x100000004a0"},{"tid":821,"created_ino":"0x100000004a1"},{"tid":824,"created_ino":"0x100000004a2"},{"tid":827,"created_ino":"0x100000004a3"},{"tid":830,"created_ino":"0x100000004a4"},{"tid":833,"created_ino":"0x100000004a5"},{"tid":836,"created_ino":"0x100000004a6"},{"tid":839,"created_ino":"0x100000004a7"},{"tid":842,"created_ino":"0x100000004a8"},{"tid":845,"created_ino":"0x100000004a9"},{"tid":848,"created_ino":"0x100000004aa"},{"tid":851,"created_ino":"0x100000004ab"},{"tid":854,"created_ino":"0x100000004ac"},{"tid":857,"created_ino":"0x100000004ad"},{"tid":860,"created_ino":"0x100000004ae"},{"tid":863,"created_ino":"0x100000004af"},{"tid":866,"created_ino":"0x100000004b0"},{"tid":869,"created_ino":"0x100000004b1"},{"tid":872,"created_ino":"0x100000004b2"},{"tid":875,"created_ino":"0x100000004b3"},{"tid":878,"created_ino":"0x100000004b4"},{"tid":881,"created_ino":"0x100000004b5"},{"tid":884,"created_ino":"0x100000004b6"},{"tid":887,"created_ino":"0x100000004b7"},{"tid":890,"created_ino":"0x100000004b8"},{"tid":893,"created_ino":"0x100000004b9"},{"tid":896,"created_ino":"0x100000004ba"},{"tid":899,"created_ino":"0x100000004bb"},{"tid":902,"created_ino":"0x100000004bc"},{"tid":905,"created_ino":"0x100000004bd"},{"tid":908,"created_ino":"0x100000004be"},{"tid":911,"created_ino":"0x100000004bf"},{"tid":914,"created_ino":"0x100000004c0"},{"tid":917,"created_ino":"0x100000004c1"},{"tid":920,"created_ino":"0x100000004c2"},{"tid":923,"created_ino":"0x100000004c3"},{"tid":926,"created_ino":"0x100000004c4"},{"tid":929,"created_ino":"0x100000004c5"},{"tid":932,"created_ino":"0x100000004c6"},{"tid":935,"created_ino":"0x100000004c7"},{"tid":938,"created_ino":"0x100000004c8"},{"tid":941,"created_ino":"0x100000004c9"},{"tid":944,"created_ino":"0x100000004ca"},{"tid":947,"created_ino":"0x100000004cb"},{"tid":950,"created_ino":"0x100000004cc"},{"tid":953,"created_ino":"0x100000004cd"},{"tid":956,"created_ino":"0x100000004ce"},{"tid":959,"created_ino":"0x100000004cf"},{"tid":962,"created_ino":"0x100000004d0"},{"tid":965,"created_ino":"0x100000004d1"},{"tid":968,"created_ino":"0x100000004d2"},{"tid":971,"created_ino":"0x100000004d3"},{"tid":974,"created_ino":"0x100000004d4"},{"tid":977,"created_ino":"0x100000004d5"},{"tid":980,"created_ino":"0x100000004d6"},{"tid":983,"created_ino":"0x100000004d7"},{"tid":986,"created_ino":"0x100000004d8"},{"tid":989,"created_ino":"0x100000004d9"},{"tid":992,"created_ino":"0x100000004da"},{"tid":995,"created_ino":"0x100000004db"},{"tid":998,"created_ino":"0x100000004dc"},{"tid":1001,"created_ino":"0x100000004dd"},{"tid":1004,"created_ino":"0x100000004de"},{"tid":1007,"created_ino":"0x100000004df"},{"tid":1010,"created_ino":"0x100000004e0"},{"tid":1013,"created_ino":"0x100000004e1"},{"tid":1016,"created_ino":"0x100000004e2"},{"tid":1019,"created_ino":"0x100000004e3"},{"tid":1022,"created_ino":"0x100000004e4"},{"tid":1025,"created_ino":"0x100000004e5"},{"tid":1028,"created_ino":"0x100000004e6"},{"tid":1031,"created_ino":"0x100000004e7"},{"tid":1034,"created_ino":"0x100000004e8"},{"tid":1037,"created_ino":"0x100000004e9"},{"tid":1040,"created_ino":"0x100000004ea"},{"tid":1043,"created_ino":"0x100000004eb"},{"tid":1046,"created_ino":"0x100000004ec"},{"tid":1049,"created_ino":"0x100000004ed"},{"tid":1052,"created_ino":"0x100000004ee"},{"tid":1055,"created_ino":"0x100000004ef"},{"tid":1058,"created_ino":"0x100000004f0"},{"tid":1061,"created_ino":"0x100000004f1"},{"tid":1064,"created_ino":"0x100000004f2"},{"tid":1067,"created_ino":"0x100000004f3"},{"tid":1070,"created_ino":"0x100000004f4"},{"tid":1073,"created_ino":"0x100000004f5"},{"tid":1076,"created_ino":"0x100000004f6"},{"tid":1079,"created_ino":"0x100000004f7"},{"tid":1082,"created_ino":"0x100000004f8"},{"tid":1085,"created_ino":"0x100000004f9"},{"tid":1088,"created_ino":"0x100000004fa"},{"tid":1091,"created_ino":"0x100000004fb"},{"tid":1094,"created_ino":"0x100000004fc"},{"tid":1097,"created_ino":"0x100000004fd"},{"tid":1100,"created_ino":"0x100000004fe"},{"tid":1103,"created_ino":"0x100000004ff"},{"tid":1106,"created_ino":"0x10000000500"},{"tid":1109,"created_ino":"0x10000000501"},{"tid":1112,"created_ino":"0x10000000502"},{"tid":1115,"created_ino":"0x10000000503"},{"tid":1118,"created_ino":"0x10000000504"},{"tid":1121,"created_ino":"0x10000000505"},{"tid":1124,"created_ino":"0x10000000506"},{"tid":1127,"created_ino":"0x10000000507"},{"tid":1130,"created_ino":"0x10000000508"},{"tid":1133,"created_ino":"0x10000000509"},{"tid":1136,"created_ino":"0x1000000050a"},{"tid":1139,"created_ino":"0x1000000050b"},{"tid":1142,"created_ino":"0x1000000050c"},{"tid":1145,"created_ino":"0x1000000050d"},{"tid":1148,"created_ino":"0x1000000050e"},{"tid":1151,"created_ino":"0x1000000050f"},{"tid":1154,"created_ino":"0x10000000510"},{"tid":1157,"created_ino":"0x10000000511"},{"tid":1160,"created_ino":"0x10000000512"},{"tid":1163,"created_ino":"0x10000000513"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000514","length":203},{"start":"0x100000005e0","length":501}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"0","hostname":"smithi016","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"69634","root":"/"}}] 2024-09-19T07:54:05.500 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "48393", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3044582375, "type": "any" }, "name": { "num": 6374, "type": "client" } }, "id": 6374, "inst": "client.6374 192.168.0.1:0/3044582375", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9520576185111578 }, "state": "open", "uptime": 9.022873407 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "0", "hostname": "smithi016", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "69634", "root": "/" }, "completed_requests": [ { "created_ino": "0x100000003e9", "tid": 506 }, { "created_ino": "0x100000003ea", "tid": 507 }, { "created_ino": "0x100000003eb", "tid": 508 }, { "created_ino": "0x100000003ec", "tid": 509 }, { "created_ino": "0x100000003ed", "tid": 510 }, { "created_ino": "0x100000003ee", "tid": 511 }, { "created_ino": "0x100000003ef", "tid": 512 }, { "created_ino": "0x100000003f0", "tid": 513 }, { "created_ino": "0x100000003f1", "tid": 514 }, { "created_ino": "0x100000003f2", "tid": 515 }, { "created_ino": "0x100000003f3", "tid": 516 }, { "created_ino": "0x100000003f4", "tid": 517 }, { "created_ino": "0x100000003f5", "tid": 518 }, { "created_ino": "0x100000003f6", "tid": 519 }, { "created_ino": "0x100000003f7", "tid": 520 }, { "created_ino": "0x100000003f8", "tid": 521 }, { "created_ino": "0x100000003f9", "tid": 522 }, { "created_ino": "0x100000003fa", "tid": 523 }, { "created_ino": "0x100000003fb", "tid": 524 }, { "created_ino": "0x100000003fc", "tid": 525 }, { "created_ino": "0x100000003fd", "tid": 526 }, { "created_ino": "0x100000003fe", "tid": 527 }, { "created_ino": "0x100000003ff", "tid": 528 }, { "created_ino": "0x10000000400", "tid": 529 }, { "created_ino": "0x10000000401", "tid": 530 }, { "created_ino": "0x10000000402", "tid": 531 }, { "created_ino": "0x10000000403", "tid": 532 }, { "created_ino": "0x10000000404", "tid": 533 }, { "created_ino": "0x10000000405", "tid": 534 }, { "created_ino": "0x10000000406", "tid": 535 }, { "created_ino": "0x10000000407", "tid": 536 }, { "created_ino": "0x10000000408", "tid": 537 }, { "created_ino": "0x10000000409", "tid": 538 }, { "created_ino": "0x1000000040a", "tid": 539 }, { "created_ino": "0x1000000040b", "tid": 540 }, { "created_ino": "0x1000000040c", "tid": 541 }, { "created_ino": "0x1000000040d", "tid": 542 }, { "created_ino": "0x1000000040e", "tid": 543 }, { "created_ino": "0x1000000040f", "tid": 544 }, { "created_ino": "0x10000000410", "tid": 545 }, { "created_ino": "0x10000000411", "tid": 546 }, { "created_ino": "0x10000000412", "tid": 547 }, { "created_ino": "0x10000000413", "tid": 548 }, { "created_ino": "0x10000000414", "tid": 549 }, { "created_ino": "0x10000000415", "tid": 550 }, { "created_ino": "0x10000000416", "tid": 551 }, { "created_ino": "0x10000000417", "tid": 552 }, { "created_ino": "0x10000000418", "tid": 553 }, { "created_ino": "0x10000000419", "tid": 554 }, { "created_ino": "0x1000000041a", "tid": 555 }, { "created_ino": "0x1000000041b", "tid": 556 }, { "created_ino": "0x1000000041c", "tid": 557 }, { "created_ino": "0x1000000041d", "tid": 558 }, { "created_ino": "0x1000000041e", "tid": 559 }, { "created_ino": "0x1000000041f", "tid": 560 }, { "created_ino": "0x10000000420", "tid": 561 }, { "created_ino": "0x10000000421", "tid": 562 }, { "created_ino": "0x10000000422", "tid": 563 }, { "created_ino": "0x10000000423", "tid": 564 }, { "created_ino": "0x10000000424", "tid": 565 }, { "created_ino": "0x10000000425", "tid": 566 }, { "created_ino": "0x10000000426", "tid": 567 }, { "created_ino": "0x10000000427", "tid": 568 }, { "created_ino": "0x10000000428", "tid": 569 }, { "created_ino": "0x10000000429", "tid": 570 }, { "created_ino": "0x1000000042a", "tid": 571 }, { "created_ino": "0x1000000042b", "tid": 572 }, { "created_ino": "0x1000000042c", "tid": 573 }, { "created_ino": "0x1000000042d", "tid": 574 }, { "created_ino": "0x1000000042e", "tid": 575 }, { "created_ino": "0x1000000042f", "tid": 576 }, { "created_ino": "0x10000000430", "tid": 577 }, { "created_ino": "0x10000000431", "tid": 578 }, { "created_ino": "0x10000000432", "tid": 579 }, { "created_ino": "0x10000000433", "tid": 580 }, { "created_ino": "0x10000000434", "tid": 581 }, { "created_ino": "0x10000000435", "tid": 582 }, { "created_ino": "0x10000000436", "tid": 583 }, { "created_ino": "0x10000000437", "tid": 584 }, { "created_ino": "0x10000000438", "tid": 585 }, { "created_ino": "0x10000000439", "tid": 586 }, { "created_ino": "0x1000000043a", "tid": 587 }, { "created_ino": "0x1000000043b", "tid": 588 }, { "created_ino": "0x1000000043c", "tid": 589 }, { "created_ino": "0x1000000043d", "tid": 590 }, { "created_ino": "0x1000000043e", "tid": 591 }, { "created_ino": "0x1000000043f", "tid": 592 }, { "created_ino": "0x10000000440", "tid": 593 }, { "created_ino": "0x10000000441", "tid": 594 }, { "created_ino": "0x10000000442", "tid": 595 }, { "created_ino": "0x10000000443", "tid": 596 }, { "created_ino": "0x10000000444", "tid": 597 }, { "created_ino": "0x10000000445", "tid": 598 }, { "created_ino": "0x10000000446", "tid": 599 }, { "created_ino": "0x10000000447", "tid": 600 }, { "created_ino": "0x10000000448", "tid": 601 }, { "created_ino": "0x10000000449", "tid": 602 }, { "created_ino": "0x1000000044a", "tid": 603 }, { "created_ino": "0x1000000044b", "tid": 604 }, { "created_ino": "0x1000000044c", "tid": 605 }, { "created_ino": "0x1000000044d", "tid": 606 }, { "created_ino": "0x1000000044e", "tid": 607 }, { "created_ino": "0x1000000044f", "tid": 608 }, { "created_ino": "0x10000000450", "tid": 609 }, { "created_ino": "0x10000000451", "tid": 610 }, { "created_ino": "0x10000000452", "tid": 611 }, { "created_ino": "0x10000000453", "tid": 612 }, { "created_ino": "0x10000000454", "tid": 613 }, { "created_ino": "0x10000000455", "tid": 614 }, { "created_ino": "0x10000000456", "tid": 615 }, { "created_ino": "0x10000000457", "tid": 616 }, { "created_ino": "0x10000000458", "tid": 617 }, { "created_ino": "0x10000000459", "tid": 618 }, { "created_ino": "0x1000000045a", "tid": 619 }, { "created_ino": "0x1000000045b", "tid": 620 }, { "created_ino": "0x1000000045c", "tid": 621 }, { "created_ino": "0x1000000045d", "tid": 622 }, { "created_ino": "0x1000000045e", "tid": 623 }, { "created_ino": "0x1000000045f", "tid": 624 }, { "created_ino": "0x10000000460", "tid": 625 }, { "created_ino": "0x10000000461", "tid": 628 }, { "created_ino": "0x10000000462", "tid": 632 }, { "created_ino": "0x10000000463", "tid": 635 }, { "created_ino": "0x10000000464", "tid": 638 }, { "created_ino": "0x10000000465", "tid": 641 }, { "created_ino": "0x10000000466", "tid": 644 }, { "created_ino": "0x10000000467", "tid": 647 }, { "created_ino": "0x10000000468", "tid": 650 }, { "created_ino": "0x10000000469", "tid": 653 }, { "created_ino": "0x1000000046a", "tid": 656 }, { "created_ino": "0x1000000046b", "tid": 659 }, { "created_ino": "0x1000000046c", "tid": 662 }, { "created_ino": "0x1000000046d", "tid": 665 }, { "created_ino": "0x1000000046e", "tid": 668 }, { "created_ino": "0x1000000046f", "tid": 671 }, { "created_ino": "0x10000000470", "tid": 674 }, { "created_ino": "0x10000000471", "tid": 677 }, { "created_ino": "0x10000000472", "tid": 680 }, { "created_ino": "0x10000000473", "tid": 683 }, { "created_ino": "0x10000000474", "tid": 686 }, { "created_ino": "0x10000000475", "tid": 689 }, { "created_ino": "0x10000000476", "tid": 692 }, { "created_ino": "0x10000000477", "tid": 695 }, { "created_ino": "0x10000000478", "tid": 698 }, { "created_ino": "0x10000000479", "tid": 701 }, { "created_ino": "0x1000000047a", "tid": 704 }, { "created_ino": "0x1000000047b", "tid": 707 }, { "created_ino": "0x1000000047c", "tid": 710 }, { "created_ino": "0x1000000047d", "tid": 713 }, { "created_ino": "0x1000000047e", "tid": 716 }, { "created_ino": "0x1000000047f", "tid": 719 }, { "created_ino": "0x10000000480", "tid": 722 }, { "created_ino": "0x10000000481", "tid": 725 }, { "created_ino": "0x10000000482", "tid": 728 }, { "created_ino": "0x10000000483", "tid": 731 }, { "created_ino": "0x10000000484", "tid": 734 }, { "created_ino": "0x10000000485", "tid": 737 }, { "created_ino": "0x10000000486", "tid": 740 }, { "created_ino": "0x10000000487", "tid": 743 }, { "created_ino": "0x10000000488", "tid": 746 }, { "created_ino": "0x10000000489", "tid": 749 }, { "created_ino": "0x1000000048a", "tid": 752 }, { "created_ino": "0x1000000048b", "tid": 755 }, { "created_ino": "0x1000000048c", "tid": 758 }, { "created_ino": "0x1000000048d", "tid": 761 }, { "created_ino": "0x1000000048e", "tid": 764 }, { "created_ino": "0x1000000048f", "tid": 767 }, { "created_ino": "0x10000000490", "tid": 770 }, { "created_ino": "0x10000000491", "tid": 773 }, { "created_ino": "0x10000000492", "tid": 776 }, { "created_ino": "0x10000000493", "tid": 779 }, { "created_ino": "0x10000000494", "tid": 782 }, { "created_ino": "0x10000000495", "tid": 785 }, { "created_ino": "0x10000000496", "tid": 788 }, { "created_ino": "0x10000000497", "tid": 791 }, { "created_ino": "0x10000000498", "tid": 794 }, { "created_ino": "0x10000000499", "tid": 797 }, { "created_ino": "0x1000000049a", "tid": 800 }, { "created_ino": "0x1000000049b", "tid": 803 }, { "created_ino": "0x1000000049c", "tid": 806 }, { "created_ino": "0x1000000049d", "tid": 809 }, { "created_ino": "0x1000000049e", "tid": 812 }, { "created_ino": "0x1000000049f", "tid": 815 }, { "created_ino": "0x100000004a0", "tid": 818 }, { "created_ino": "0x100000004a1", "tid": 821 }, { "created_ino": "0x100000004a2", "tid": 824 }, { "created_ino": "0x100000004a3", "tid": 827 }, { "created_ino": "0x100000004a4", "tid": 830 }, { "created_ino": "0x100000004a5", "tid": 833 }, { "created_ino": "0x100000004a6", "tid": 836 }, { "created_ino": "0x100000004a7", "tid": 839 }, { "created_ino": "0x100000004a8", "tid": 842 }, { "created_ino": "0x100000004a9", "tid": 845 }, { "created_ino": "0x100000004aa", "tid": 848 }, { "created_ino": "0x100000004ab", "tid": 851 }, { "created_ino": "0x100000004ac", "tid": 854 }, { "created_ino": "0x100000004ad", "tid": 857 }, { "created_ino": "0x100000004ae", "tid": 860 }, { "created_ino": "0x100000004af", "tid": 863 }, { "created_ino": "0x100000004b0", "tid": 866 }, { "created_ino": "0x100000004b1", "tid": 869 }, { "created_ino": "0x100000004b2", "tid": 872 }, { "created_ino": "0x100000004b3", "tid": 875 }, { "created_ino": "0x100000004b4", "tid": 878 }, { "created_ino": "0x100000004b5", "tid": 881 }, { "created_ino": "0x100000004b6", "tid": 884 }, { "created_ino": "0x100000004b7", "tid": 887 }, { "created_ino": "0x100000004b8", "tid": 890 }, { "created_ino": "0x100000004b9", "tid": 893 }, { "created_ino": "0x100000004ba", "tid": 896 }, { "created_ino": "0x100000004bb", "tid": 899 }, { "created_ino": "0x100000004bc", "tid": 902 }, { "created_ino": "0x100000004bd", "tid": 905 }, { "created_ino": "0x100000004be", "tid": 908 }, { "created_ino": "0x100000004bf", "tid": 911 }, { "created_ino": "0x100000004c0", "tid": 914 }, { "created_ino": "0x100000004c1", "tid": 917 }, { "created_ino": "0x100000004c2", "tid": 920 }, { "created_ino": "0x100000004c3", "tid": 923 }, { "created_ino": "0x100000004c4", "tid": 926 }, { "created_ino": "0x100000004c5", "tid": 929 }, { "created_ino": "0x100000004c6", "tid": 932 }, { "created_ino": "0x100000004c7", "tid": 935 }, { "created_ino": "0x100000004c8", "tid": 938 }, { "created_ino": "0x100000004c9", "tid": 941 }, { "created_ino": "0x100000004ca", "tid": 944 }, { "created_ino": "0x100000004cb", "tid": 947 }, { "created_ino": "0x100000004cc", "tid": 950 }, { "created_ino": "0x100000004cd", "tid": 953 }, { "created_ino": "0x100000004ce", "tid": 956 }, { "created_ino": "0x100000004cf", "tid": 959 }, { "created_ino": "0x100000004d0", "tid": 962 }, { "created_ino": "0x100000004d1", "tid": 965 }, { "created_ino": "0x100000004d2", "tid": 968 }, { "created_ino": "0x100000004d3", "tid": 971 }, { "created_ino": "0x100000004d4", "tid": 974 }, { "created_ino": "0x100000004d5", "tid": 977 }, { "created_ino": "0x100000004d6", "tid": 980 }, { "created_ino": "0x100000004d7", "tid": 983 }, { "created_ino": "0x100000004d8", "tid": 986 }, { "created_ino": "0x100000004d9", "tid": 989 }, { "created_ino": "0x100000004da", "tid": 992 }, { "created_ino": "0x100000004db", "tid": 995 }, { "created_ino": "0x100000004dc", "tid": 998 }, { "created_ino": "0x100000004dd", "tid": 1001 }, { "created_ino": "0x100000004de", "tid": 1004 }, { "created_ino": "0x100000004df", "tid": 1007 }, { "created_ino": "0x100000004e0", "tid": 1010 }, { "created_ino": "0x100000004e1", "tid": 1013 }, { "created_ino": "0x100000004e2", "tid": 1016 }, { "created_ino": "0x100000004e3", "tid": 1019 }, { "created_ino": "0x100000004e4", "tid": 1022 }, { "created_ino": "0x100000004e5", "tid": 1025 }, { "created_ino": "0x100000004e6", "tid": 1028 }, { "created_ino": "0x100000004e7", "tid": 1031 }, { "created_ino": "0x100000004e8", "tid": 1034 }, { "created_ino": "0x100000004e9", "tid": 1037 }, { "created_ino": "0x100000004ea", "tid": 1040 }, { "created_ino": "0x100000004eb", "tid": 1043 }, { "created_ino": "0x100000004ec", "tid": 1046 }, { "created_ino": "0x100000004ed", "tid": 1049 }, { "created_ino": "0x100000004ee", "tid": 1052 }, { "created_ino": "0x100000004ef", "tid": 1055 }, { "created_ino": "0x100000004f0", "tid": 1058 }, { "created_ino": "0x100000004f1", "tid": 1061 }, { "created_ino": "0x100000004f2", "tid": 1064 }, { "created_ino": "0x100000004f3", "tid": 1067 }, { "created_ino": "0x100000004f4", "tid": 1070 }, { "created_ino": "0x100000004f5", "tid": 1073 }, { "created_ino": "0x100000004f6", "tid": 1076 }, { "created_ino": "0x100000004f7", "tid": 1079 }, { "created_ino": "0x100000004f8", "tid": 1082 }, { "created_ino": "0x100000004f9", "tid": 1085 }, { "created_ino": "0x100000004fa", "tid": 1088 }, { "created_ino": "0x100000004fb", "tid": 1091 }, { "created_ino": "0x100000004fc", "tid": 1094 }, { "created_ino": "0x100000004fd", "tid": 1097 }, { "created_ino": "0x100000004fe", "tid": 1100 }, { "created_ino": "0x100000004ff", "tid": 1103 }, { "created_ino": "0x10000000500", "tid": 1106 }, { "created_ino": "0x10000000501", "tid": 1109 }, { "created_ino": "0x10000000502", "tid": 1112 }, { "created_ino": "0x10000000503", "tid": 1115 }, { "created_ino": "0x10000000504", "tid": 1118 }, { "created_ino": "0x10000000505", "tid": 1121 }, { "created_ino": "0x10000000506", "tid": 1124 }, { "created_ino": "0x10000000507", "tid": 1127 }, { "created_ino": "0x10000000508", "tid": 1130 }, { "created_ino": "0x10000000509", "tid": 1133 }, { "created_ino": "0x1000000050a", "tid": 1136 }, { "created_ino": "0x1000000050b", "tid": 1139 }, { "created_ino": "0x1000000050c", "tid": 1142 }, { "created_ino": "0x1000000050d", "tid": 1145 }, { "created_ino": "0x1000000050e", "tid": 1148 }, { "created_ino": "0x1000000050f", "tid": 1151 }, { "created_ino": "0x10000000510", "tid": 1154 }, { "created_ino": "0x10000000511", "tid": 1157 }, { "created_ino": "0x10000000512", "tid": 1160 }, { "created_ino": "0x10000000513", "tid": 1163 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1440089416, "type": "any" }, "name": { "num": 7551, "type": "client" } }, "id": 7551, "inst": "client.7551 192.168.0.1:0/1440089416", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 506, "num_caps": 802, "num_completed_flushes": 0, "num_completed_requests": 299, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 203, "start": "0x10000000514" }, { "length": 501, "start": "0x100000005e0" } ], "recall_caps": { "halflife": 60, "value": 2331.4225414343996 }, "recall_caps_throttle": { "halflife": 1.5, "value": 1001.4289494857182 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 514.8624656182676 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1095, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1586.161946597183 }, "state": "open", "uptime": 10.554851914 } ] 2024-09-19T07:54:05.500 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-09-19T07:54:05.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:54:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.600+0000 7fc48f39f640 1 Processor -- start 2024-09-19T07:54:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.601+0000 7fc48f39f640 1 -- start start 2024-09-19T07:54:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.601+0000 7fc48f39f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc488070d90 0x7fc488073460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.601+0000 7fc48f39f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc48806c320 0x7fc4880cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.601+0000 7fc48f39f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4880ccc40 con 0x7fc488071170 2024-09-19T07:54:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.601+0000 7fc48f39f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc4880cd780 con 0x7fc48806c320 2024-09-19T07:54:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.601+0000 7fc48f39f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc4880ccee0 con 0x7fc488070d90 2024-09-19T07:54:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48db9c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc488071170 0x7fc48806bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60558/0 (socket says 172.21.15.16:60558) 2024-09-19T07:54:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48db9c640 1 -- 172.21.15.16:0/2590031614 learned_addr learned my addr 172.21.15.16:0/2590031614 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48e39d640 1 --2- 172.21.15.16:0/2590031614 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc488070d90 0x7fc488073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48eb9e640 1 --2- 172.21.15.16:0/2590031614 >> v2:172.21.15.146:3300/0 conn(0x7fc48806c320 0x7fc4880cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48d39b640 1 -- 172.21.15.16:0/2590031614 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3579120595 0 0) 0x7fc4880ccc40 con 0x7fc488071170 2024-09-19T07:54:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48d39b640 1 -- 172.21.15.16:0/2590031614 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc470003520 con 0x7fc488071170 2024-09-19T07:54:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48d39b640 1 -- 172.21.15.16:0/2590031614 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2955501715 0 0) 0x7fc470003520 con 0x7fc488071170 2024-09-19T07:54:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48d39b640 1 -- 172.21.15.16:0/2590031614 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc488070d90 msgr2=0x7fc488073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48d39b640 1 --2- 172.21.15.16:0/2590031614 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc488070d90 0x7fc488073460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48d39b640 1 -- 172.21.15.16:0/2590031614 >> v2:172.21.15.146:3300/0 conn(0x7fc48806c320 msgr2=0x7fc4880cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:05.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48d39b640 1 --2- 172.21.15.16:0/2590031614 >> v2:172.21.15.146:3300/0 conn(0x7fc48806c320 0x7fc4880cf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48d39b640 1 -- 172.21.15.16:0/2590031614 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4880cd1c0 con 0x7fc488071170 2024-09-19T07:54:05.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48d39b640 1 -- 172.21.15.16:0/2590031614 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fc47c001670 con 0x7fc488071170 2024-09-19T07:54:05.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48d39b640 1 -- 172.21.15.16:0/2590031614 >> v1:172.21.15.16:6789/0 conn(0x7fc488071170 legacy=0x7fc48806bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:05.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48d39b640 1 --2- 172.21.15.16:0/2590031614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4700039b0 0x7fc470003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48d39b640 1 --2- 172.21.15.16:0/2590031614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc470004800 0x7fc470004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48d39b640 1 --2- 172.21.15.16:0/2590031614 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc488070d90 0x7fc4700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48d39b640 1 -- 172.21.15.16:0/2590031614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc4880cd780 con 0x7fc470004800 2024-09-19T07:54:05.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48d39b640 1 -- 172.21.15.16:0/2590031614 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc4880ccee0 con 0x7fc4700039b0 2024-09-19T07:54:05.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48d39b640 1 -- 172.21.15.16:0/2590031614 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc47000b9c0 con 0x7fc488070d90 2024-09-19T07:54:05.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48db9c640 1 --2- 172.21.15.16:0/2590031614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc470004800 0x7fc470004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48e39d640 1 --2- 172.21.15.16:0/2590031614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4700039b0 0x7fc470003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.602+0000 7fc48eb9e640 1 --2- 172.21.15.16:0/2590031614 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc488070d90 0x7fc4700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.603+0000 7fc48db9c640 1 -- 172.21.15.16:0/2590031614 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc488070d90 msgr2=0x7fc4700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.603+0000 7fc48db9c640 1 --2- 172.21.15.16:0/2590031614 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc488070d90 0x7fc4700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.603+0000 7fc48db9c640 1 -- 172.21.15.16:0/2590031614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4700039b0 msgr2=0x7fc470003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.603+0000 7fc48db9c640 1 --2- 172.21.15.16:0/2590031614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4700039b0 0x7fc470003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.603+0000 7fc48db9c640 1 -- 172.21.15.16:0/2590031614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc470003520 con 0x7fc470004800 2024-09-19T07:54:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.603+0000 7fc48db9c640 1 --2- 172.21.15.16:0/2590031614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc470004800 0x7fc470004c80 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fc4880cc3f0 tx=0x7fc470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9466aaff6f6a84b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.604+0000 7fc48d39b640 1 -- 172.21.15.16:0/2590031614 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c007d50 con 0x7fc470004800 2024-09-19T07:54:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.604+0000 7fc48d39b640 1 -- 172.21.15.16:0/2590031614 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc47c0058d0 con 0x7fc470004800 2024-09-19T07:54:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.604+0000 7fc48d39b640 1 -- 172.21.15.16:0/2590031614 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c005cc0 con 0x7fc470004800 2024-09-19T07:54:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.604+0000 7fc48f39f640 1 -- 172.21.15.16:0/2590031614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc470004800 msgr2=0x7fc470004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.604+0000 7fc48f39f640 1 --2- 172.21.15.16:0/2590031614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc470004800 0x7fc470004c80 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fc4880cc3f0 tx=0x7fc470005830 comp rx=0 tx=0).stop 2024-09-19T07:54:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.604+0000 7fc48eb9e640 1 -- 172.21.15.16:0/2590031614 reap_dead start 2024-09-19T07:54:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.604+0000 7fc48f39f640 1 -- 172.21.15.16:0/2590031614 shutdown_connections 2024-09-19T07:54:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.604+0000 7fc48f39f640 1 -- 172.21.15.16:0/2590031614 >> 172.21.15.16:0/2590031614 conn(0x7fc4880d0f30 msgr2=0x7fc4880d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.604+0000 7fc48f39f640 1 -- 172.21.15.16:0/2590031614 shutdown_connections 2024-09-19T07:54:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.604+0000 7fc48f39f640 1 -- 172.21.15.16:0/2590031614 wait complete. 2024-09-19T07:54:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.605+0000 7fc48f39f640 1 Processor -- start 2024-09-19T07:54:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.605+0000 7fc48f39f640 1 -- start start 2024-09-19T07:54:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.606+0000 7fc48f39f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc470004800 0x7fc488158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.606+0000 7fc48f39f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc488159210 0x7fc4880cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.606+0000 7fc48f39f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc48815cf30 0x7fc48815d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.606+0000 7fc48f39f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc4880d1ac0 con 0x7fc48815cf30 2024-09-19T07:54:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.606+0000 7fc48f39f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc4880ceba0 con 0x7fc470004800 2024-09-19T07:54:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.606+0000 7fc48f39f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc4880d1c40 con 0x7fc488159210 2024-09-19T07:54:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.606+0000 7fc48eb9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc48815cf30 0x7fc48815d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.606+0000 7fc48eb9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc48815cf30 0x7fc48815d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38422/0 (socket says 172.21.15.16:38422) 2024-09-19T07:54:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.606+0000 7fc48eb9e640 1 -- 172.21.15.16:0/1461537196 learned_addr learned my addr 172.21.15.16:0/1461537196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.606+0000 7fc48e39d640 1 --2- 172.21.15.16:0/1461537196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc470004800 0x7fc488158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.606+0000 7fc48db9c640 1 --2- 172.21.15.16:0/1461537196 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc488159210 0x7fc4880cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.607+0000 7fc48eb9e640 1 -- 172.21.15.16:0/1461537196 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc488159210 msgr2=0x7fc4880cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.607+0000 7fc48eb9e640 1 --2- 172.21.15.16:0/1461537196 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc488159210 0x7fc4880cabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.607+0000 7fc48eb9e640 1 -- 172.21.15.16:0/1461537196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc470004800 msgr2=0x7fc488158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.607+0000 7fc48eb9e640 1 --2- 172.21.15.16:0/1461537196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc470004800 0x7fc488158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.607+0000 7fc48eb9e640 1 -- 172.21.15.16:0/1461537196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48809f330 con 0x7fc48815cf30 2024-09-19T07:54:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.607+0000 7fc48eb9e640 1 --2- 172.21.15.16:0/1461537196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc48815cf30 0x7fc48815d3b0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fc488072a00 tx=0x7fc480011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.607+0000 7fc486ffd640 1 -- 172.21.15.16:0/1461537196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480004240 con 0x7fc48815cf30 2024-09-19T07:54:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.607+0000 7fc486ffd640 1 -- 172.21.15.16:0/1461537196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc48001b070 con 0x7fc48815cf30 2024-09-19T07:54:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.607+0000 7fc48f39f640 1 -- 172.21.15.16:0/1461537196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48809f610 con 0x7fc48815cf30 2024-09-19T07:54:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.607+0000 7fc486ffd640 1 -- 172.21.15.16:0/1461537196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48000f710 con 0x7fc48815cf30 2024-09-19T07:54:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.607+0000 7fc48f39f640 1 -- 172.21.15.16:0/1461537196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc48809fbd0 con 0x7fc48815cf30 2024-09-19T07:54:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.608+0000 7fc48f39f640 1 -- 172.21.15.16:0/1461537196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc48806bc10 con 0x7fc48815cf30 2024-09-19T07:54:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.609+0000 7fc486ffd640 1 -- 172.21.15.16:0/1461537196 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc480012040 con 0x7fc48815cf30 2024-09-19T07:54:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.609+0000 7fc486ffd640 1 --2- 172.21.15.16:0/1461537196 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc47406c330 0x7fc47406e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.609+0000 7fc486ffd640 1 -- 172.21.15.16:0/1461537196 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fc48008b370 con 0x7fc48815cf30 2024-09-19T07:54:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.610+0000 7fc48e39d640 1 --2- 172.21.15.16:0/1461537196 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc47406c330 0x7fc47406e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.610+0000 7fc48e39d640 1 --2- 172.21.15.16:0/1461537196 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc47406c330 0x7fc47406e7f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc478014370 tx=0x7fc47800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.614+0000 7fc486ffd640 1 -- 172.21.15.16:0/1461537196 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc4800599a0 con 0x7fc48815cf30 2024-09-19T07:54:05.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.826+0000 7fc48f39f640 1 -- 172.21.15.16:0/1461537196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc4880ccc00 con 0x7fc48815cf30 2024-09-19T07:54:05.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.826+0000 7fc486ffd640 1 -- 172.21.15.16:0/1461537196 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7fc4800592a0 con 0x7fc48815cf30 2024-09-19T07:54:05.827 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:05.827 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:54:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.828+0000 7fc48f39f640 1 -- 172.21.15.16:0/1461537196 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc47406c330 msgr2=0x7fc47406e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.828+0000 7fc48f39f640 1 --2- 172.21.15.16:0/1461537196 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc47406c330 0x7fc47406e7f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc478014370 tx=0x7fc47800b040 comp rx=0 tx=0).stop 2024-09-19T07:54:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.828+0000 7fc48f39f640 1 -- 172.21.15.16:0/1461537196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc48815cf30 msgr2=0x7fc48815d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.828+0000 7fc48f39f640 1 --2- 172.21.15.16:0/1461537196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc48815cf30 0x7fc48815d3b0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fc488072a00 tx=0x7fc480011cf0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.828+0000 7fc48f39f640 1 -- 172.21.15.16:0/1461537196 shutdown_connections 2024-09-19T07:54:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.828+0000 7fc48f39f640 1 --2- 172.21.15.16:0/1461537196 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc47406c330 0x7fc47406e7f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.828+0000 7fc48f39f640 1 --2- 172.21.15.16:0/1461537196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc48815cf30 0x7fc48815d3b0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.828+0000 7fc48f39f640 1 --2- 172.21.15.16:0/1461537196 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc488159210 0x7fc4880cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.828+0000 7fc48f39f640 1 --2- 172.21.15.16:0/1461537196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc470004800 0x7fc488158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.828+0000 7fc48f39f640 1 -- 172.21.15.16:0/1461537196 >> 172.21.15.16:0/1461537196 conn(0x7fc4880d0f30 msgr2=0x7fc488070370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:05.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.828+0000 7fc48f39f640 1 -- 172.21.15.16:0/1461537196 shutdown_connections 2024-09-19T07:54:05.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.828+0000 7fc48f39f640 1 -- 172.21.15.16:0/1461537196 wait complete. 2024-09-19T07:54:05.839 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-19T07:54:05.839 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-09-19T07:54:05.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:54:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.939+0000 7f348d865640 1 Processor -- start 2024-09-19T07:54:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.940+0000 7f348d865640 1 -- start start 2024-09-19T07:54:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.940+0000 7f348d865640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3488072590 0x7f348806d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.940+0000 7f348d865640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34881620b0 con 0x7f34880d21f0 2024-09-19T07:54:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.940+0000 7f348d865640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3488162370 con 0x7f3488141ad0 2024-09-19T07:54:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.940+0000 7f348d865640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f3488141750 con 0x7f3488072590 2024-09-19T07:54:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.941+0000 7f34867fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3488072590 0x7f348806d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.941+0000 7f3486ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f34880d21f0 0x7f34880d25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60570/0 (socket says 172.21.15.16:60570) 2024-09-19T07:54:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.941+0000 7f3486ffd640 1 -- 172.21.15.16:0/380679539 learned_addr learned my addr 172.21.15.16:0/380679539 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.941+0000 7f34867fc640 1 -- 172.21.15.16:0/380679539 >> v1:172.21.15.146:6789/0 conn(0x7f3488141ad0 legacy=0x7f3488141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.941+0000 7f34867fc640 1 -- 172.21.15.16:0/380679539 >> v1:172.21.15.16:6789/0 conn(0x7f34880d21f0 legacy=0x7f34880d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.941+0000 7f34867fc640 1 -- 172.21.15.16:0/380679539 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3488162650 con 0x7f3488072590 2024-09-19T07:54:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.941+0000 7f3485ffb640 1 -- 172.21.15.16:0/380679539 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 424642161 0 0) 0x7f34881620b0 con 0x7f34880d21f0 2024-09-19T07:54:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.942+0000 7f34867fc640 1 --2- 172.21.15.16:0/380679539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3488072590 0x7f348806d0e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f34700097a0 tx=0x7f347002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ccef01738adcc7f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.942+0000 7f3485ffb640 1 -- 172.21.15.16:0/380679539 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470037070 con 0x7f3488072590 2024-09-19T07:54:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.942+0000 7f3485ffb640 1 -- 172.21.15.16:0/380679539 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3470032440 con 0x7f3488072590 2024-09-19T07:54:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.942+0000 7f3485ffb640 1 -- 172.21.15.16:0/380679539 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470032810 con 0x7f3488072590 2024-09-19T07:54:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.942+0000 7f348d865640 1 -- 172.21.15.16:0/380679539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3488072590 msgr2=0x7f348806d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.942+0000 7f348d865640 1 --2- 172.21.15.16:0/380679539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3488072590 0x7f348806d0e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f34700097a0 tx=0x7f347002a1a0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.942+0000 7f348d865640 1 -- 172.21.15.16:0/380679539 shutdown_connections 2024-09-19T07:54:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.942+0000 7f348d865640 1 --2- 172.21.15.16:0/380679539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3488072590 0x7f348806d0e0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.942+0000 7f348d865640 1 -- 172.21.15.16:0/380679539 >> 172.21.15.16:0/380679539 conn(0x7f34880b3ed0 msgr2=0x7f34880b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.942+0000 7f348d865640 1 -- 172.21.15.16:0/380679539 shutdown_connections 2024-09-19T07:54:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.942+0000 7f348d865640 1 -- 172.21.15.16:0/380679539 wait complete. 2024-09-19T07:54:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.943+0000 7f348d865640 1 Processor -- start 2024-09-19T07:54:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.944+0000 7f348d865640 1 -- start start 2024-09-19T07:54:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.944+0000 7f348d865640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3488072590 0x7f348815be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.944+0000 7f348d865640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34880d21f0 0x7f3488159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.944+0000 7f348d865640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3488141ad0 0x7f3488140540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.944+0000 7f348d865640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f348806d620 con 0x7f3488141ad0 2024-09-19T07:54:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.944+0000 7f348d865640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3488162c10 con 0x7f34880d21f0 2024-09-19T07:54:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.944+0000 7f348d865640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f348806d7a0 con 0x7f3488072590 2024-09-19T07:54:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.945+0000 7f34877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3488141ad0 0x7f3488140540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.945+0000 7f34877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3488141ad0 0x7f3488140540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38428/0 (socket says 172.21.15.16:38428) 2024-09-19T07:54:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.945+0000 7f34877fe640 1 -- 172.21.15.16:0/1924752080 learned_addr learned my addr 172.21.15.16:0/1924752080 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.945+0000 7f34867fc640 1 --2- 172.21.15.16:0/1924752080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34880d21f0 0x7f3488159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.945+0000 7f3486ffd640 1 --2- 172.21.15.16:0/1924752080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3488072590 0x7f348815be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.945+0000 7f34877fe640 1 -- 172.21.15.16:0/1924752080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3488072590 msgr2=0x7f348815be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:05.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.945+0000 7f34877fe640 1 --2- 172.21.15.16:0/1924752080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3488072590 0x7f348815be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.945+0000 7f34877fe640 1 -- 172.21.15.16:0/1924752080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34880d21f0 msgr2=0x7f3488159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:05.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.945+0000 7f34877fe640 1 --2- 172.21.15.16:0/1924752080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34880d21f0 0x7f3488159800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:05.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.945+0000 7f34877fe640 1 -- 172.21.15.16:0/1924752080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3488159d40 con 0x7f3488141ad0 2024-09-19T07:54:05.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.945+0000 7f34877fe640 1 --2- 172.21.15.16:0/1924752080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3488141ad0 0x7f3488140540 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f348000c390 tx=0x7f348000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.945+0000 7f347ffff640 1 -- 172.21.15.16:0/1924752080 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480017070 con 0x7f3488141ad0 2024-09-19T07:54:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.945+0000 7f347ffff640 1 -- 172.21.15.16:0/1924752080 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34800052d0 con 0x7f3488141ad0 2024-09-19T07:54:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.945+0000 7f348d865640 1 -- 172.21.15.16:0/1924752080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3488159fc0 con 0x7f3488141ad0 2024-09-19T07:54:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.945+0000 7f347ffff640 1 -- 172.21.15.16:0/1924752080 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480003810 con 0x7f3488141ad0 2024-09-19T07:54:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.945+0000 7f348d865640 1 -- 172.21.15.16:0/1924752080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f348815c600 con 0x7f3488141ad0 2024-09-19T07:54:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.946+0000 7f348d865640 1 -- 172.21.15.16:0/1924752080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3488158c30 con 0x7f3488141ad0 2024-09-19T07:54:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.947+0000 7f347ffff640 1 -- 172.21.15.16:0/1924752080 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f348001a070 con 0x7f3488141ad0 2024-09-19T07:54:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.947+0000 7f347ffff640 1 --2- 172.21.15.16:0/1924752080 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f346806c2e0 0x7f346806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.947+0000 7f347ffff640 1 -- 172.21.15.16:0/1924752080 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f348008a7e0 con 0x7f3488141ad0 2024-09-19T07:54:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.947+0000 7f3486ffd640 1 --2- 172.21.15.16:0/1924752080 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f346806c2e0 0x7f346806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.948+0000 7f3486ffd640 1 --2- 172.21.15.16:0/1924752080 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f346806c2e0 0x7f346806e7a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3478001920 tx=0x7f34780018b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:05.951+0000 7f347ffff640 1 -- 172.21.15.16:0/1924752080 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3488159d40 con 0x7f3488141ad0 2024-09-19T07:54:06.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:06.164+0000 7f348d865640 1 -- 172.21.15.16:0/1924752080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3488072290 con 0x7f3488141ad0 2024-09-19T07:54:06.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:06.164+0000 7f347ffff640 1 -- 172.21.15.16:0/1924752080 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7f3480058770 con 0x7f3488141ad0 2024-09-19T07:54:06.165 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:06.165 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:54:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:06.166+0000 7f348d865640 1 -- 172.21.15.16:0/1924752080 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f346806c2e0 msgr2=0x7f346806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:06.166+0000 7f348d865640 1 --2- 172.21.15.16:0/1924752080 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f346806c2e0 0x7f346806e7a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3478001920 tx=0x7f34780018b0 comp rx=0 tx=0).stop 2024-09-19T07:54:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:06.166+0000 7f348d865640 1 -- 172.21.15.16:0/1924752080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3488141ad0 msgr2=0x7f3488140540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:06.166+0000 7f348d865640 1 --2- 172.21.15.16:0/1924752080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3488141ad0 0x7f3488140540 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f348000c390 tx=0x7f348000c850 comp rx=0 tx=0).stop 2024-09-19T07:54:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:06.166+0000 7f348d865640 1 -- 172.21.15.16:0/1924752080 shutdown_connections 2024-09-19T07:54:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:06.166+0000 7f348d865640 1 --2- 172.21.15.16:0/1924752080 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f346806c2e0 0x7f346806e7a0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:06.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:06.166+0000 7f348d865640 1 --2- 172.21.15.16:0/1924752080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3488141ad0 0x7f3488140540 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:06.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:06.166+0000 7f348d865640 1 --2- 172.21.15.16:0/1924752080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34880d21f0 0x7f3488159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:06.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:06.166+0000 7f348d865640 1 --2- 172.21.15.16:0/1924752080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3488072590 0x7f348815be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:06.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:06.166+0000 7f348d865640 1 -- 172.21.15.16:0/1924752080 >> 172.21.15.16:0/1924752080 conn(0x7f34880b3ed0 msgr2=0x7f34880b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:06.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:06.166+0000 7f348d865640 1 -- 172.21.15.16:0/1924752080 shutdown_connections 2024-09-19T07:54:06.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:06.166+0000 7f348d865640 1 -- 172.21.15.16:0/1924752080 wait complete. 2024-09-19T07:54:06.177 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-19T07:54:06.177 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2024-09-19T07:54:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:06 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-19T07:54:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:54:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:stdin closed, goodbye! 2024-09-19T07:54:06.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:54:06.936 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:54:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.083+0000 7f9b0c1e8640 1 Processor -- start 2024-09-19T07:54:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.084+0000 7f9b0c1e8640 1 -- start start 2024-09-19T07:54:07.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.084+0000 7f9b0c1e8640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9b0415e2a0 0x7f9b0406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:07.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.084+0000 7f9b0c1e8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b0406d960 0x7f9b040d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.084+0000 7f9b0c1e8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b040d1880 con 0x7f9b0415e680 2024-09-19T07:54:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.084+0000 7f9b0c1e8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9b040d1b60 con 0x7f9b0406d960 2024-09-19T07:54:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.084+0000 7f9b0c1e8640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f9b040cf2d0 con 0x7f9b0415e2a0 2024-09-19T07:54:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.085+0000 7f9b0a785640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b0406d960 0x7f9b040d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.085+0000 7f9b0a785640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b0406d960 0x7f9b040d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38440/0 (socket says 172.21.15.16:38440) 2024-09-19T07:54:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.085+0000 7f9b09783640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9b0415e680 0x7f9b040cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60576/0 (socket says 172.21.15.16:60576) 2024-09-19T07:54:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.085+0000 7f9b0a785640 1 -- 172.21.15.16:0/3945858251 learned_addr learned my addr 172.21.15.16:0/3945858251 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.085+0000 7f9b09f84640 1 --2- 172.21.15.16:0/3945858251 >> v2:172.21.15.146:3300/0 conn(0x7f9b0415e2a0 0x7f9b0406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.085+0000 7f9b0a785640 1 -- 172.21.15.16:0/3945858251 >> v2:172.21.15.146:3300/0 conn(0x7f9b0415e2a0 msgr2=0x7f9b0406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.085+0000 7f9b0a785640 1 --2- 172.21.15.16:0/3945858251 >> v2:172.21.15.146:3300/0 conn(0x7f9b0415e2a0 0x7f9b0406d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.085+0000 7f9b0a785640 1 -- 172.21.15.16:0/3945858251 >> v1:172.21.15.16:6789/0 conn(0x7f9b0415e680 legacy=0x7f9b040cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.085+0000 7f9b0a785640 1 -- 172.21.15.16:0/3945858251 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b040cf5b0 con 0x7f9b0406d960 2024-09-19T07:54:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.085+0000 7f9b08f82640 1 -- 172.21.15.16:0/3945858251 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 855707389 0 0) 0x7f9b040d1880 con 0x7f9b0415e680 2024-09-19T07:54:07.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.086+0000 7f9b0a785640 1 --2- 172.21.15.16:0/3945858251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b0406d960 0x7f9b040d4370 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f9b040b6fa0 tx=0x7f9afc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b69fe67a9eed1e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:07.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.086+0000 7f9b08f82640 1 -- 172.21.15.16:0/3945858251 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc036070 con 0x7f9b0406d960 2024-09-19T07:54:07.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.086+0000 7f9b08f82640 1 -- 172.21.15.16:0/3945858251 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9afc031600 con 0x7f9b0406d960 2024-09-19T07:54:07.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.086+0000 7f9b08f82640 1 -- 172.21.15.16:0/3945858251 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc0319d0 con 0x7f9b0406d960 2024-09-19T07:54:07.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.086+0000 7f9b0c1e8640 1 -- 172.21.15.16:0/3945858251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b0406d960 msgr2=0x7f9b040d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:07.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.086+0000 7f9b0c1e8640 1 --2- 172.21.15.16:0/3945858251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b0406d960 0x7f9b040d4370 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f9b040b6fa0 tx=0x7f9afc02a730 comp rx=0 tx=0).stop 2024-09-19T07:54:07.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.086+0000 7f9b0c1e8640 1 -- 172.21.15.16:0/3945858251 shutdown_connections 2024-09-19T07:54:07.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.086+0000 7f9b0c1e8640 1 --2- 172.21.15.16:0/3945858251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b0406d960 0x7f9b040d4370 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:07.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.086+0000 7f9b0c1e8640 1 --2- 172.21.15.16:0/3945858251 >> v2:172.21.15.146:3300/0 conn(0x7f9b0415e2a0 0x7f9b0406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:07.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.086+0000 7f9b0c1e8640 1 -- 172.21.15.16:0/3945858251 >> 172.21.15.16:0/3945858251 conn(0x7f9b0409d760 msgr2=0x7f9b0408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:07.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.087+0000 7f9b0c1e8640 1 -- 172.21.15.16:0/3945858251 shutdown_connections 2024-09-19T07:54:07.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.087+0000 7f9b0c1e8640 1 -- 172.21.15.16:0/3945858251 wait complete. 2024-09-19T07:54:07.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.088+0000 7f9b0c1e8640 1 Processor -- start 2024-09-19T07:54:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.088+0000 7f9b0c1e8640 1 -- start start 2024-09-19T07:54:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b0c1e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b0406d960 0x7f9b0415d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b0c1e8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b0415e2a0 0x7f9b041ea1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b0c1e8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b0415e680 0x7f9b041ea6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b0c1e8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b040d0360 con 0x7f9b0406d960 2024-09-19T07:54:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b0c1e8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9b040d01e0 con 0x7f9b0415e2a0 2024-09-19T07:54:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b0c1e8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9b040d48b0 con 0x7f9b0415e680 2024-09-19T07:54:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b09f84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b0406d960 0x7f9b0415d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b09f84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b0406d960 0x7f9b0415d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38454/0 (socket says 172.21.15.16:38454) 2024-09-19T07:54:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b09f84640 1 -- 172.21.15.16:0/3899764438 learned_addr learned my addr 172.21.15.16:0/3899764438 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b09783640 1 --2- 172.21.15.16:0/3899764438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b0415e2a0 0x7f9b041ea1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b0a785640 1 --2- 172.21.15.16:0/3899764438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b0415e680 0x7f9b041ea6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:07.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b09f84640 1 -- 172.21.15.16:0/3899764438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b0415e680 msgr2=0x7f9b041ea6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:07.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b09f84640 1 --2- 172.21.15.16:0/3899764438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b0415e680 0x7f9b041ea6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:07.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b09f84640 1 -- 172.21.15.16:0/3899764438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b0415e2a0 msgr2=0x7f9b041ea1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:07.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b09f84640 1 --2- 172.21.15.16:0/3899764438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b0415e2a0 0x7f9b041ea1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:07.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b09f84640 1 -- 172.21.15.16:0/3899764438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0415d8c0 con 0x7f9b0406d960 2024-09-19T07:54:07.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b09783640 1 --2- 172.21.15.16:0/3899764438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b0415e2a0 0x7f9b041ea1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:07.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b0a785640 1 --2- 172.21.15.16:0/3899764438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b0415e680 0x7f9b041ea6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:07.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b09f84640 1 --2- 172.21.15.16:0/3899764438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b0406d960 0x7f9b0415d380 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000d830 tx=0x7f9b0000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:07.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9afa7fc640 1 -- 172.21.15.16:0/3899764438 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0000b6c0 con 0x7f9b0406d960 2024-09-19T07:54:07.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9afa7fc640 1 -- 172.21.15.16:0/3899764438 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b00019070 con 0x7f9b0406d960 2024-09-19T07:54:07.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b0c1e8640 1 -- 172.21.15.16:0/3899764438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b040c8fd0 con 0x7f9b0406d960 2024-09-19T07:54:07.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9afa7fc640 1 -- 172.21.15.16:0/3899764438 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0000ba80 con 0x7f9b0406d960 2024-09-19T07:54:07.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.089+0000 7f9b0c1e8640 1 -- 172.21.15.16:0/3899764438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b040c9270 con 0x7f9b0406d960 2024-09-19T07:54:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.090+0000 7f9b0c1e8640 1 -- 172.21.15.16:0/3899764438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b040d4c70 con 0x7f9b0406d960 2024-09-19T07:54:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.091+0000 7f9afa7fc640 1 -- 172.21.15.16:0/3899764438 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9b0000bc20 con 0x7f9b0406d960 2024-09-19T07:54:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.092+0000 7f9afa7fc640 1 --2- 172.21.15.16:0/3899764438 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ae806c2b0 0x7f9ae806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:07.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.092+0000 7f9afa7fc640 1 -- 172.21.15.16:0/3899764438 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f9b0008ba90 con 0x7f9b0406d960 2024-09-19T07:54:07.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.092+0000 7f9b09783640 1 --2- 172.21.15.16:0/3899764438 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ae806c2b0 0x7f9ae806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:07.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.092+0000 7f9b09783640 1 --2- 172.21.15.16:0/3899764438 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ae806c2b0 0x7f9ae806e770 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9b040b95a0 tx=0x7f9af4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.096+0000 7f9afa7fc640 1 -- 172.21.15.16:0/3899764438 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9b0415d8c0 con 0x7f9b0406d960 2024-09-19T07:54:07.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.307+0000 7f9b0c1e8640 1 -- 172.21.15.16:0/3899764438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b0406cf90 con 0x7f9b0406d960 2024-09-19T07:54:07.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.307+0000 7f9afa7fc640 1 -- 172.21.15.16:0/3899764438 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5231 (secure 0 0 0) 0x7f9b0001c360 con 0x7f9b0406d960 2024-09-19T07:54:07.308 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:07.308 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":65,"btime":"2024-09-19T07:54:05:204476+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2471408352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2471408352},{"type":"v1","addr":"172.21.15.16:6833","nonce":2471408352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/1281369866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1281369866},{"type":"v1","addr":"172.21.15.146:6836","nonce":1281369866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1377303883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1377303883},{"type":"v1","addr":"172.21.15.146:6837","nonce":1377303883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:53:43.812066+0000","modified":"2024-09-19T07:54:04.488075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7368":{"gid":7368,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/2050126415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2050126415},{"type":"v1","addr":"172.21.15.16:6835","nonce":2050126415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7368,"qdb_cluster":[7368]},"id":8}]} 2024-09-19T07:54:07.308 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 65 2024-09-19T07:54:07.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.309+0000 7f9b0c1e8640 1 -- 172.21.15.16:0/3899764438 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ae806c2b0 msgr2=0x7f9ae806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:07.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.309+0000 7f9b0c1e8640 1 --2- 172.21.15.16:0/3899764438 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ae806c2b0 0x7f9ae806e770 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9b040b95a0 tx=0x7f9af4009040 comp rx=0 tx=0).stop 2024-09-19T07:54:07.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.309+0000 7f9b0c1e8640 1 -- 172.21.15.16:0/3899764438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b0406d960 msgr2=0x7f9b0415d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:07.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.309+0000 7f9b0c1e8640 1 --2- 172.21.15.16:0/3899764438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b0406d960 0x7f9b0415d380 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000d830 tx=0x7f9b0000dcf0 comp rx=0 tx=0).stop 2024-09-19T07:54:07.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.309+0000 7f9b0c1e8640 1 -- 172.21.15.16:0/3899764438 shutdown_connections 2024-09-19T07:54:07.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.309+0000 7f9b0c1e8640 1 --2- 172.21.15.16:0/3899764438 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ae806c2b0 0x7f9ae806e770 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:07.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.309+0000 7f9b0c1e8640 1 --2- 172.21.15.16:0/3899764438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b0415e680 0x7f9b041ea6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:07.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.309+0000 7f9b0c1e8640 1 --2- 172.21.15.16:0/3899764438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b0415e2a0 0x7f9b041ea1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:07.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.309+0000 7f9b0c1e8640 1 --2- 172.21.15.16:0/3899764438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b0406d960 0x7f9b0415d380 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:07.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.309+0000 7f9b0c1e8640 1 -- 172.21.15.16:0/3899764438 >> 172.21.15.16:0/3899764438 conn(0x7f9b0409d760 msgr2=0x7f9b04160970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:07.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.310+0000 7f9b0c1e8640 1 -- 172.21.15.16:0/3899764438 shutdown_connections 2024-09-19T07:54:07.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.310+0000 7f9b0c1e8640 1 -- 172.21.15.16:0/3899764438 wait complete. 2024-09-19T07:54:07.321 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-19T07:54:07.441 INFO:teuthology.orchestra.run.smithi016.stdout:[{"id":6374,"entity":{"name":{"type":"client","num":6374},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3044582375}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":10.968846105000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9388145136504713,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6374 192.168.0.1:0/3044582375","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"48393","root":"/"}},{"id":7551,"entity":{"name":{"type":"client","num":7551},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1440089416}},"state":"open","num_leases":1,"num_caps":802,"request_load_avg":1070,"uptime":12.500824612000001,"requests_in_flight":0,"num_completed_requests":299,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":2675.1332993307042,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":662.84946764578183,"halflife":1.5},"recall_caps_throttle2o":{"value":138.78570995292054,"halflife":0.5},"session_cache_liveness":{"value":1579.0463304069372,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":506,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7551 192.168.0.1:0/1440089416","completed_requests":[{"tid":506,"created_ino":"0x100000003e9"},{"tid":507,"created_ino":"0x100000003ea"},{"tid":508,"created_ino":"0x100000003eb"},{"tid":509,"created_ino":"0x100000003ec"},{"tid":510,"created_ino":"0x100000003ed"},{"tid":511,"created_ino":"0x100000003ee"},{"tid":512,"created_ino":"0x100000003ef"},{"tid":513,"created_ino":"0x100000003f0"},{"tid":514,"created_ino":"0x100000003f1"},{"tid":515,"created_ino":"0x100000003f2"},{"tid":516,"created_ino":"0x100000003f3"},{"tid":517,"created_ino":"0x100000003f4"},{"tid":518,"created_ino":"0x100000003f5"},{"tid":519,"created_ino":"0x100000003f6"},{"tid":520,"created_ino":"0x100000003f7"},{"tid":521,"created_ino":"0x100000003f8"},{"tid":522,"created_ino":"0x100000003f9"},{"tid":523,"created_ino":"0x100000003fa"},{"tid":524,"created_ino":"0x100000003fb"},{"tid":525,"created_ino":"0x100000003fc"},{"tid":526,"created_ino":"0x100000003fd"},{"tid":527,"created_ino":"0x100000003fe"},{"tid":528,"created_ino":"0x100000003ff"},{"tid":529,"created_ino":"0x10000000400"},{"tid":530,"created_ino":"0x10000000401"},{"tid":531,"created_ino":"0x10000000402"},{"tid":532,"created_ino":"0x10000000403"},{"tid":533,"created_ino":"0x10000000404"},{"tid":534,"created_ino":"0x10000000405"},{"tid":535,"created_ino":"0x10000000406"},{"tid":536,"created_ino":"0x10000000407"},{"tid":537,"created_ino":"0x10000000408"},{"tid":538,"created_ino":"0x10000000409"},{"tid":539,"created_ino":"0x1000000040a"},{"tid":540,"created_ino":"0x1000000040b"},{"tid":541,"created_ino":"0x1000000040c"},{"tid":542,"created_ino":"0x1000000040d"},{"tid":543,"created_ino":"0x1000000040e"},{"tid":544,"created_ino":"0x1000000040f"},{"tid":545,"created_ino":"0x10000000410"},{"tid":546,"created_ino":"0x10000000411"},{"tid":547,"created_ino":"0x10000000412"},{"tid":548,"created_ino":"0x10000000413"},{"tid":549,"created_ino":"0x10000000414"},{"tid":550,"created_ino":"0x10000000415"},{"tid":551,"created_ino":"0x10000000416"},{"tid":552,"created_ino":"0x10000000417"},{"tid":553,"created_ino":"0x10000000418"},{"tid":554,"created_ino":"0x10000000419"},{"tid":555,"created_ino":"0x1000000041a"},{"tid":556,"created_ino":"0x1000000041b"},{"tid":557,"created_ino":"0x1000000041c"},{"tid":558,"created_ino":"0x1000000041d"},{"tid":559,"created_ino":"0x1000000041e"},{"tid":560,"created_ino":"0x1000000041f"},{"tid":561,"created_ino":"0x10000000420"},{"tid":562,"created_ino":"0x10000000421"},{"tid":563,"created_ino":"0x10000000422"},{"tid":564,"created_ino":"0x10000000423"},{"tid":565,"created_ino":"0x10000000424"},{"tid":566,"created_ino":"0x10000000425"},{"tid":567,"created_ino":"0x10000000426"},{"tid":568,"created_ino":"0x10000000427"},{"tid":569,"created_ino":"0x10000000428"},{"tid":570,"created_ino":"0x10000000429"},{"tid":571,"created_ino":"0x1000000042a"},{"tid":572,"created_ino":"0x1000000042b"},{"tid":573,"created_ino":"0x1000000042c"},{"tid":574,"created_ino":"0x1000000042d"},{"tid":575,"created_ino":"0x1000000042e"},{"tid":576,"created_ino":"0x1000000042f"},{"tid":577,"created_ino":"0x10000000430"},{"tid":578,"created_ino":"0x10000000431"},{"tid":579,"created_ino":"0x10000000432"},{"tid":580,"created_ino":"0x10000000433"},{"tid":581,"created_ino":"0x10000000434"},{"tid":582,"created_ino":"0x10000000435"},{"tid":583,"created_ino":"0x10000000436"},{"tid":584,"created_ino":"0x10000000437"},{"tid":585,"created_ino":"0x10000000438"},{"tid":586,"created_ino":"0x10000000439"},{"tid":587,"created_ino":"0x1000000043a"},{"tid":588,"created_ino":"0x1000000043b"},{"tid":589,"created_ino":"0x1000000043c"},{"tid":590,"created_ino":"0x1000000043d"},{"tid":591,"created_ino":"0x1000000043e"},{"tid":592,"created_ino":"0x1000000043f"},{"tid":593,"created_ino":"0x10000000440"},{"tid":594,"created_ino":"0x10000000441"},{"tid":595,"created_ino":"0x10000000442"},{"tid":596,"created_ino":"0x10000000443"},{"tid":597,"created_ino":"0x10000000444"},{"tid":598,"created_ino":"0x10000000445"},{"tid":599,"created_ino":"0x10000000446"},{"tid":600,"created_ino":"0x10000000447"},{"tid":601,"created_ino":"0x10000000448"},{"tid":602,"created_ino":"0x10000000449"},{"tid":603,"created_ino":"0x1000000044a"},{"tid":604,"created_ino":"0x1000000044b"},{"tid":605,"created_ino":"0x1000000044c"},{"tid":606,"created_ino":"0x1000000044d"},{"tid":607,"created_ino":"0x1000000044e"},{"tid":608,"created_ino":"0x1000000044f"},{"tid":609,"created_ino":"0x10000000450"},{"tid":610,"created_ino":"0x10000000451"},{"tid":611,"created_ino":"0x10000000452"},{"tid":612,"created_ino":"0x10000000453"},{"tid":613,"created_ino":"0x10000000454"},{"tid":614,"created_ino":"0x10000000455"},{"tid":615,"created_ino":"0x10000000456"},{"tid":616,"created_ino":"0x10000000457"},{"tid":617,"created_ino":"0x10000000458"},{"tid":618,"created_ino":"0x10000000459"},{"tid":619,"created_ino":"0x1000000045a"},{"tid":620,"created_ino":"0x1000000045b"},{"tid":621,"created_ino":"0x1000000045c"},{"tid":622,"created_ino":"0x1000000045d"},{"tid":623,"created_ino":"0x1000000045e"},{"tid":624,"created_ino":"0x1000000045f"},{"tid":625,"created_ino":"0x10000000460"},{"tid":628,"created_ino":"0x10000000461"},{"tid":632,"created_ino":"0x10000000462"},{"tid":635,"created_ino":"0x10000000463"},{"tid":638,"created_ino":"0x10000000464"},{"tid":641,"created_ino":"0x10000000465"},{"tid":644,"created_ino":"0x10000000466"},{"tid":647,"created_ino":"0x10000000467"},{"tid":650,"created_ino":"0x10000000468"},{"tid":653,"created_ino":"0x10000000469"},{"tid":656,"created_ino":"0x1000000046a"},{"tid":659,"created_ino":"0x1000000046b"},{"tid":662,"created_ino":"0x1000000046c"},{"tid":665,"created_ino":"0x1000000046d"},{"tid":668,"created_ino":"0x1000000046e"},{"tid":671,"created_ino":"0x1000000046f"},{"tid":674,"created_ino":"0x10000000470"},{"tid":677,"created_ino":"0x10000000471"},{"tid":680,"created_ino":"0x10000000472"},{"tid":683,"created_ino":"0x10000000473"},{"tid":686,"created_ino":"0x10000000474"},{"tid":689,"created_ino":"0x10000000475"},{"tid":692,"created_ino":"0x10000000476"},{"tid":695,"created_ino":"0x10000000477"},{"tid":698,"created_ino":"0x10000000478"},{"tid":701,"created_ino":"0x10000000479"},{"tid":704,"created_ino":"0x1000000047a"},{"tid":707,"created_ino":"0x1000000047b"},{"tid":710,"created_ino":"0x1000000047c"},{"tid":713,"created_ino":"0x1000000047d"},{"tid":716,"created_ino":"0x1000000047e"},{"tid":719,"created_ino":"0x1000000047f"},{"tid":722,"created_ino":"0x10000000480"},{"tid":725,"created_ino":"0x10000000481"},{"tid":728,"created_ino":"0x10000000482"},{"tid":731,"created_ino":"0x10000000483"},{"tid":734,"created_ino":"0x10000000484"},{"tid":737,"created_ino":"0x10000000485"},{"tid":740,"created_ino":"0x10000000486"},{"tid":743,"created_ino":"0x10000000487"},{"tid":746,"created_ino":"0x10000000488"},{"tid":749,"created_ino":"0x10000000489"},{"tid":752,"created_ino":"0x1000000048a"},{"tid":755,"created_ino":"0x1000000048b"},{"tid":758,"created_ino":"0x1000000048c"},{"tid":761,"created_ino":"0x1000000048d"},{"tid":764,"created_ino":"0x1000000048e"},{"tid":767,"created_ino":"0x1000000048f"},{"tid":770,"created_ino":"0x10000000490"},{"tid":773,"created_ino":"0x10000000491"},{"tid":776,"created_ino":"0x10000000492"},{"tid":779,"created_ino":"0x10000000493"},{"tid":782,"created_ino":"0x10000000494"},{"tid":785,"created_ino":"0x10000000495"},{"tid":788,"created_ino":"0x10000000496"},{"tid":791,"created_ino":"0x10000000497"},{"tid":794,"created_ino":"0x10000000498"},{"tid":797,"created_ino":"0x10000000499"},{"tid":800,"created_ino":"0x1000000049a"},{"tid":803,"created_ino":"0x1000000049b"},{"tid":806,"created_ino":"0x1000000049c"},{"tid":809,"created_ino":"0x1000000049d"},{"tid":812,"created_ino":"0x1000000049e"},{"tid":815,"created_ino":"0x1000000049f"},{"tid":818,"created_ino":"0x100000004a0"},{"tid":821,"created_ino":"0x100000004a1"},{"tid":824,"created_ino":"0x100000004a2"},{"tid":827,"created_ino":"0x100000004a3"},{"tid":830,"created_ino":"0x100000004a4"},{"tid":833,"created_ino":"0x100000004a5"},{"tid":836,"created_ino":"0x100000004a6"},{"tid":839,"created_ino":"0x100000004a7"},{"tid":842,"created_ino":"0x100000004a8"},{"tid":845,"created_ino":"0x100000004a9"},{"tid":848,"created_ino":"0x100000004aa"},{"tid":851,"created_ino":"0x100000004ab"},{"tid":854,"created_ino":"0x100000004ac"},{"tid":857,"created_ino":"0x100000004ad"},{"tid":860,"created_ino":"0x100000004ae"},{"tid":863,"created_ino":"0x100000004af"},{"tid":866,"created_ino":"0x100000004b0"},{"tid":869,"created_ino":"0x100000004b1"},{"tid":872,"created_ino":"0x100000004b2"},{"tid":875,"created_ino":"0x100000004b3"},{"tid":878,"created_ino":"0x100000004b4"},{"tid":881,"created_ino":"0x100000004b5"},{"tid":884,"created_ino":"0x100000004b6"},{"tid":887,"created_ino":"0x100000004b7"},{"tid":890,"created_ino":"0x100000004b8"},{"tid":893,"created_ino":"0x100000004b9"},{"tid":896,"created_ino":"0x100000004ba"},{"tid":899,"created_ino":"0x100000004bb"},{"tid":902,"created_ino":"0x100000004bc"},{"tid":905,"created_ino":"0x100000004bd"},{"tid":908,"created_ino":"0x100000004be"},{"tid":911,"created_ino":"0x100000004bf"},{"tid":914,"created_ino":"0x100000004c0"},{"tid":917,"created_ino":"0x100000004c1"},{"tid":920,"created_ino":"0x100000004c2"},{"tid":923,"created_ino":"0x100000004c3"},{"tid":926,"created_ino":"0x100000004c4"},{"tid":929,"created_ino":"0x100000004c5"},{"tid":932,"created_ino":"0x100000004c6"},{"tid":935,"created_ino":"0x100000004c7"},{"tid":938,"created_ino":"0x100000004c8"},{"tid":941,"created_ino":"0x100000004c9"},{"tid":944,"created_ino":"0x100000004ca"},{"tid":947,"created_ino":"0x100000004cb"},{"tid":950,"created_ino":"0x100000004cc"},{"tid":953,"created_ino":"0x100000004cd"},{"tid":956,"created_ino":"0x100000004ce"},{"tid":959,"created_ino":"0x100000004cf"},{"tid":962,"created_ino":"0x100000004d0"},{"tid":965,"created_ino":"0x100000004d1"},{"tid":968,"created_ino":"0x100000004d2"},{"tid":971,"created_ino":"0x100000004d3"},{"tid":974,"created_ino":"0x100000004d4"},{"tid":977,"created_ino":"0x100000004d5"},{"tid":980,"created_ino":"0x100000004d6"},{"tid":983,"created_ino":"0x100000004d7"},{"tid":986,"created_ino":"0x100000004d8"},{"tid":989,"created_ino":"0x100000004d9"},{"tid":992,"created_ino":"0x100000004da"},{"tid":995,"created_ino":"0x100000004db"},{"tid":998,"created_ino":"0x100000004dc"},{"tid":1001,"created_ino":"0x100000004dd"},{"tid":1004,"created_ino":"0x100000004de"},{"tid":1007,"created_ino":"0x100000004df"},{"tid":1010,"created_ino":"0x100000004e0"},{"tid":1013,"created_ino":"0x100000004e1"},{"tid":1016,"created_ino":"0x100000004e2"},{"tid":1019,"created_ino":"0x100000004e3"},{"tid":1022,"created_ino":"0x100000004e4"},{"tid":1025,"created_ino":"0x100000004e5"},{"tid":1028,"created_ino":"0x100000004e6"},{"tid":1031,"created_ino":"0x100000004e7"},{"tid":1034,"created_ino":"0x100000004e8"},{"tid":1037,"created_ino":"0x100000004e9"},{"tid":1040,"created_ino":"0x100000004ea"},{"tid":1043,"created_ino":"0x100000004eb"},{"tid":1046,"created_ino":"0x100000004ec"},{"tid":1049,"created_ino":"0x100000004ed"},{"tid":1052,"created_ino":"0x100000004ee"},{"tid":1055,"created_ino":"0x100000004ef"},{"tid":1058,"created_ino":"0x100000004f0"},{"tid":1061,"created_ino":"0x100000004f1"},{"tid":1064,"created_ino":"0x100000004f2"},{"tid":1067,"created_ino":"0x100000004f3"},{"tid":1070,"created_ino":"0x100000004f4"},{"tid":1073,"created_ino":"0x100000004f5"},{"tid":1076,"created_ino":"0x100000004f6"},{"tid":1079,"created_ino":"0x100000004f7"},{"tid":1082,"created_ino":"0x100000004f8"},{"tid":1085,"created_ino":"0x100000004f9"},{"tid":1088,"created_ino":"0x100000004fa"},{"tid":1091,"created_ino":"0x100000004fb"},{"tid":1094,"created_ino":"0x100000004fc"},{"tid":1097,"created_ino":"0x100000004fd"},{"tid":1100,"created_ino":"0x100000004fe"},{"tid":1103,"created_ino":"0x100000004ff"},{"tid":1106,"created_ino":"0x10000000500"},{"tid":1109,"created_ino":"0x10000000501"},{"tid":1112,"created_ino":"0x10000000502"},{"tid":1115,"created_ino":"0x10000000503"},{"tid":1118,"created_ino":"0x10000000504"},{"tid":1121,"created_ino":"0x10000000505"},{"tid":1124,"created_ino":"0x10000000506"},{"tid":1127,"created_ino":"0x10000000507"},{"tid":1130,"created_ino":"0x10000000508"},{"tid":1133,"created_ino":"0x10000000509"},{"tid":1136,"created_ino":"0x1000000050a"},{"tid":1139,"created_ino":"0x1000000050b"},{"tid":1142,"created_ino":"0x1000000050c"},{"tid":1145,"created_ino":"0x1000000050d"},{"tid":1148,"created_ino":"0x1000000050e"},{"tid":1151,"created_ino":"0x1000000050f"},{"tid":1154,"created_ino":"0x10000000510"},{"tid":1157,"created_ino":"0x10000000511"},{"tid":1160,"created_ino":"0x10000000512"},{"tid":1163,"created_ino":"0x10000000513"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000514","length":203},{"start":"0x100000005e0","length":501}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"0","hostname":"smithi016","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"69634","root":"/"}}] 2024-09-19T07:54:07.444 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "48393", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3044582375, "type": "any" }, "name": { "num": 6374, "type": "client" } }, "id": 6374, "inst": "client.6374 192.168.0.1:0/3044582375", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9388145136504713 }, "state": "open", "uptime": 10.968846105 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "0", "hostname": "smithi016", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "69634", "root": "/" }, "completed_requests": [ { "created_ino": "0x100000003e9", "tid": 506 }, { "created_ino": "0x100000003ea", "tid": 507 }, { "created_ino": "0x100000003eb", "tid": 508 }, { "created_ino": "0x100000003ec", "tid": 509 }, { "created_ino": "0x100000003ed", "tid": 510 }, { "created_ino": "0x100000003ee", "tid": 511 }, { "created_ino": "0x100000003ef", "tid": 512 }, { "created_ino": "0x100000003f0", "tid": 513 }, { "created_ino": "0x100000003f1", "tid": 514 }, { "created_ino": "0x100000003f2", "tid": 515 }, { "created_ino": "0x100000003f3", "tid": 516 }, { "created_ino": "0x100000003f4", "tid": 517 }, { "created_ino": "0x100000003f5", "tid": 518 }, { "created_ino": "0x100000003f6", "tid": 519 }, { "created_ino": "0x100000003f7", "tid": 520 }, { "created_ino": "0x100000003f8", "tid": 521 }, { "created_ino": "0x100000003f9", "tid": 522 }, { "created_ino": "0x100000003fa", "tid": 523 }, { "created_ino": "0x100000003fb", "tid": 524 }, { "created_ino": "0x100000003fc", "tid": 525 }, { "created_ino": "0x100000003fd", "tid": 526 }, { "created_ino": "0x100000003fe", "tid": 527 }, { "created_ino": "0x100000003ff", "tid": 528 }, { "created_ino": "0x10000000400", "tid": 529 }, { "created_ino": "0x10000000401", "tid": 530 }, { "created_ino": "0x10000000402", "tid": 531 }, { "created_ino": "0x10000000403", "tid": 532 }, { "created_ino": "0x10000000404", "tid": 533 }, { "created_ino": "0x10000000405", "tid": 534 }, { "created_ino": "0x10000000406", "tid": 535 }, { "created_ino": "0x10000000407", "tid": 536 }, { "created_ino": "0x10000000408", "tid": 537 }, { "created_ino": "0x10000000409", "tid": 538 }, { "created_ino": "0x1000000040a", "tid": 539 }, { "created_ino": "0x1000000040b", "tid": 540 }, { "created_ino": "0x1000000040c", "tid": 541 }, { "created_ino": "0x1000000040d", "tid": 542 }, { "created_ino": "0x1000000040e", "tid": 543 }, { "created_ino": "0x1000000040f", "tid": 544 }, { "created_ino": "0x10000000410", "tid": 545 }, { "created_ino": "0x10000000411", "tid": 546 }, { "created_ino": "0x10000000412", "tid": 547 }, { "created_ino": "0x10000000413", "tid": 548 }, { "created_ino": "0x10000000414", "tid": 549 }, { "created_ino": "0x10000000415", "tid": 550 }, { "created_ino": "0x10000000416", "tid": 551 }, { "created_ino": "0x10000000417", "tid": 552 }, { "created_ino": "0x10000000418", "tid": 553 }, { "created_ino": "0x10000000419", "tid": 554 }, { "created_ino": "0x1000000041a", "tid": 555 }, { "created_ino": "0x1000000041b", "tid": 556 }, { "created_ino": "0x1000000041c", "tid": 557 }, { "created_ino": "0x1000000041d", "tid": 558 }, { "created_ino": "0x1000000041e", "tid": 559 }, { "created_ino": "0x1000000041f", "tid": 560 }, { "created_ino": "0x10000000420", "tid": 561 }, { "created_ino": "0x10000000421", "tid": 562 }, { "created_ino": "0x10000000422", "tid": 563 }, { "created_ino": "0x10000000423", "tid": 564 }, { "created_ino": "0x10000000424", "tid": 565 }, { "created_ino": "0x10000000425", "tid": 566 }, { "created_ino": "0x10000000426", "tid": 567 }, { "created_ino": "0x10000000427", "tid": 568 }, { "created_ino": "0x10000000428", "tid": 569 }, { "created_ino": "0x10000000429", "tid": 570 }, { "created_ino": "0x1000000042a", "tid": 571 }, { "created_ino": "0x1000000042b", "tid": 572 }, { "created_ino": "0x1000000042c", "tid": 573 }, { "created_ino": "0x1000000042d", "tid": 574 }, { "created_ino": "0x1000000042e", "tid": 575 }, { "created_ino": "0x1000000042f", "tid": 576 }, { "created_ino": "0x10000000430", "tid": 577 }, { "created_ino": "0x10000000431", "tid": 578 }, { "created_ino": "0x10000000432", "tid": 579 }, { "created_ino": "0x10000000433", "tid": 580 }, { "created_ino": "0x10000000434", "tid": 581 }, { "created_ino": "0x10000000435", "tid": 582 }, { "created_ino": "0x10000000436", "tid": 583 }, { "created_ino": "0x10000000437", "tid": 584 }, { "created_ino": "0x10000000438", "tid": 585 }, { "created_ino": "0x10000000439", "tid": 586 }, { "created_ino": "0x1000000043a", "tid": 587 }, { "created_ino": "0x1000000043b", "tid": 588 }, { "created_ino": "0x1000000043c", "tid": 589 }, { "created_ino": "0x1000000043d", "tid": 590 }, { "created_ino": "0x1000000043e", "tid": 591 }, { "created_ino": "0x1000000043f", "tid": 592 }, { "created_ino": "0x10000000440", "tid": 593 }, { "created_ino": "0x10000000441", "tid": 594 }, { "created_ino": "0x10000000442", "tid": 595 }, { "created_ino": "0x10000000443", "tid": 596 }, { "created_ino": "0x10000000444", "tid": 597 }, { "created_ino": "0x10000000445", "tid": 598 }, { "created_ino": "0x10000000446", "tid": 599 }, { "created_ino": "0x10000000447", "tid": 600 }, { "created_ino": "0x10000000448", "tid": 601 }, { "created_ino": "0x10000000449", "tid": 602 }, { "created_ino": "0x1000000044a", "tid": 603 }, { "created_ino": "0x1000000044b", "tid": 604 }, { "created_ino": "0x1000000044c", "tid": 605 }, { "created_ino": "0x1000000044d", "tid": 606 }, { "created_ino": "0x1000000044e", "tid": 607 }, { "created_ino": "0x1000000044f", "tid": 608 }, { "created_ino": "0x10000000450", "tid": 609 }, { "created_ino": "0x10000000451", "tid": 610 }, { "created_ino": "0x10000000452", "tid": 611 }, { "created_ino": "0x10000000453", "tid": 612 }, { "created_ino": "0x10000000454", "tid": 613 }, { "created_ino": "0x10000000455", "tid": 614 }, { "created_ino": "0x10000000456", "tid": 615 }, { "created_ino": "0x10000000457", "tid": 616 }, { "created_ino": "0x10000000458", "tid": 617 }, { "created_ino": "0x10000000459", "tid": 618 }, { "created_ino": "0x1000000045a", "tid": 619 }, { "created_ino": "0x1000000045b", "tid": 620 }, { "created_ino": "0x1000000045c", "tid": 621 }, { "created_ino": "0x1000000045d", "tid": 622 }, { "created_ino": "0x1000000045e", "tid": 623 }, { "created_ino": "0x1000000045f", "tid": 624 }, { "created_ino": "0x10000000460", "tid": 625 }, { "created_ino": "0x10000000461", "tid": 628 }, { "created_ino": "0x10000000462", "tid": 632 }, { "created_ino": "0x10000000463", "tid": 635 }, { "created_ino": "0x10000000464", "tid": 638 }, { "created_ino": "0x10000000465", "tid": 641 }, { "created_ino": "0x10000000466", "tid": 644 }, { "created_ino": "0x10000000467", "tid": 647 }, { "created_ino": "0x10000000468", "tid": 650 }, { "created_ino": "0x10000000469", "tid": 653 }, { "created_ino": "0x1000000046a", "tid": 656 }, { "created_ino": "0x1000000046b", "tid": 659 }, { "created_ino": "0x1000000046c", "tid": 662 }, { "created_ino": "0x1000000046d", "tid": 665 }, { "created_ino": "0x1000000046e", "tid": 668 }, { "created_ino": "0x1000000046f", "tid": 671 }, { "created_ino": "0x10000000470", "tid": 674 }, { "created_ino": "0x10000000471", "tid": 677 }, { "created_ino": "0x10000000472", "tid": 680 }, { "created_ino": "0x10000000473", "tid": 683 }, { "created_ino": "0x10000000474", "tid": 686 }, { "created_ino": "0x10000000475", "tid": 689 }, { "created_ino": "0x10000000476", "tid": 692 }, { "created_ino": "0x10000000477", "tid": 695 }, { "created_ino": "0x10000000478", "tid": 698 }, { "created_ino": "0x10000000479", "tid": 701 }, { "created_ino": "0x1000000047a", "tid": 704 }, { "created_ino": "0x1000000047b", "tid": 707 }, { "created_ino": "0x1000000047c", "tid": 710 }, { "created_ino": "0x1000000047d", "tid": 713 }, { "created_ino": "0x1000000047e", "tid": 716 }, { "created_ino": "0x1000000047f", "tid": 719 }, { "created_ino": "0x10000000480", "tid": 722 }, { "created_ino": "0x10000000481", "tid": 725 }, { "created_ino": "0x10000000482", "tid": 728 }, { "created_ino": "0x10000000483", "tid": 731 }, { "created_ino": "0x10000000484", "tid": 734 }, { "created_ino": "0x10000000485", "tid": 737 }, { "created_ino": "0x10000000486", "tid": 740 }, { "created_ino": "0x10000000487", "tid": 743 }, { "created_ino": "0x10000000488", "tid": 746 }, { "created_ino": "0x10000000489", "tid": 749 }, { "created_ino": "0x1000000048a", "tid": 752 }, { "created_ino": "0x1000000048b", "tid": 755 }, { "created_ino": "0x1000000048c", "tid": 758 }, { "created_ino": "0x1000000048d", "tid": 761 }, { "created_ino": "0x1000000048e", "tid": 764 }, { "created_ino": "0x1000000048f", "tid": 767 }, { "created_ino": "0x10000000490", "tid": 770 }, { "created_ino": "0x10000000491", "tid": 773 }, { "created_ino": "0x10000000492", "tid": 776 }, { "created_ino": "0x10000000493", "tid": 779 }, { "created_ino": "0x10000000494", "tid": 782 }, { "created_ino": "0x10000000495", "tid": 785 }, { "created_ino": "0x10000000496", "tid": 788 }, { "created_ino": "0x10000000497", "tid": 791 }, { "created_ino": "0x10000000498", "tid": 794 }, { "created_ino": "0x10000000499", "tid": 797 }, { "created_ino": "0x1000000049a", "tid": 800 }, { "created_ino": "0x1000000049b", "tid": 803 }, { "created_ino": "0x1000000049c", "tid": 806 }, { "created_ino": "0x1000000049d", "tid": 809 }, { "created_ino": "0x1000000049e", "tid": 812 }, { "created_ino": "0x1000000049f", "tid": 815 }, { "created_ino": "0x100000004a0", "tid": 818 }, { "created_ino": "0x100000004a1", "tid": 821 }, { "created_ino": "0x100000004a2", "tid": 824 }, { "created_ino": "0x100000004a3", "tid": 827 }, { "created_ino": "0x100000004a4", "tid": 830 }, { "created_ino": "0x100000004a5", "tid": 833 }, { "created_ino": "0x100000004a6", "tid": 836 }, { "created_ino": "0x100000004a7", "tid": 839 }, { "created_ino": "0x100000004a8", "tid": 842 }, { "created_ino": "0x100000004a9", "tid": 845 }, { "created_ino": "0x100000004aa", "tid": 848 }, { "created_ino": "0x100000004ab", "tid": 851 }, { "created_ino": "0x100000004ac", "tid": 854 }, { "created_ino": "0x100000004ad", "tid": 857 }, { "created_ino": "0x100000004ae", "tid": 860 }, { "created_ino": "0x100000004af", "tid": 863 }, { "created_ino": "0x100000004b0", "tid": 866 }, { "created_ino": "0x100000004b1", "tid": 869 }, { "created_ino": "0x100000004b2", "tid": 872 }, { "created_ino": "0x100000004b3", "tid": 875 }, { "created_ino": "0x100000004b4", "tid": 878 }, { "created_ino": "0x100000004b5", "tid": 881 }, { "created_ino": "0x100000004b6", "tid": 884 }, { "created_ino": "0x100000004b7", "tid": 887 }, { "created_ino": "0x100000004b8", "tid": 890 }, { "created_ino": "0x100000004b9", "tid": 893 }, { "created_ino": "0x100000004ba", "tid": 896 }, { "created_ino": "0x100000004bb", "tid": 899 }, { "created_ino": "0x100000004bc", "tid": 902 }, { "created_ino": "0x100000004bd", "tid": 905 }, { "created_ino": "0x100000004be", "tid": 908 }, { "created_ino": "0x100000004bf", "tid": 911 }, { "created_ino": "0x100000004c0", "tid": 914 }, { "created_ino": "0x100000004c1", "tid": 917 }, { "created_ino": "0x100000004c2", "tid": 920 }, { "created_ino": "0x100000004c3", "tid": 923 }, { "created_ino": "0x100000004c4", "tid": 926 }, { "created_ino": "0x100000004c5", "tid": 929 }, { "created_ino": "0x100000004c6", "tid": 932 }, { "created_ino": "0x100000004c7", "tid": 935 }, { "created_ino": "0x100000004c8", "tid": 938 }, { "created_ino": "0x100000004c9", "tid": 941 }, { "created_ino": "0x100000004ca", "tid": 944 }, { "created_ino": "0x100000004cb", "tid": 947 }, { "created_ino": "0x100000004cc", "tid": 950 }, { "created_ino": "0x100000004cd", "tid": 953 }, { "created_ino": "0x100000004ce", "tid": 956 }, { "created_ino": "0x100000004cf", "tid": 959 }, { "created_ino": "0x100000004d0", "tid": 962 }, { "created_ino": "0x100000004d1", "tid": 965 }, { "created_ino": "0x100000004d2", "tid": 968 }, { "created_ino": "0x100000004d3", "tid": 971 }, { "created_ino": "0x100000004d4", "tid": 974 }, { "created_ino": "0x100000004d5", "tid": 977 }, { "created_ino": "0x100000004d6", "tid": 980 }, { "created_ino": "0x100000004d7", "tid": 983 }, { "created_ino": "0x100000004d8", "tid": 986 }, { "created_ino": "0x100000004d9", "tid": 989 }, { "created_ino": "0x100000004da", "tid": 992 }, { "created_ino": "0x100000004db", "tid": 995 }, { "created_ino": "0x100000004dc", "tid": 998 }, { "created_ino": "0x100000004dd", "tid": 1001 }, { "created_ino": "0x100000004de", "tid": 1004 }, { "created_ino": "0x100000004df", "tid": 1007 }, { "created_ino": "0x100000004e0", "tid": 1010 }, { "created_ino": "0x100000004e1", "tid": 1013 }, { "created_ino": "0x100000004e2", "tid": 1016 }, { "created_ino": "0x100000004e3", "tid": 1019 }, { "created_ino": "0x100000004e4", "tid": 1022 }, { "created_ino": "0x100000004e5", "tid": 1025 }, { "created_ino": "0x100000004e6", "tid": 1028 }, { "created_ino": "0x100000004e7", "tid": 1031 }, { "created_ino": "0x100000004e8", "tid": 1034 }, { "created_ino": "0x100000004e9", "tid": 1037 }, { "created_ino": "0x100000004ea", "tid": 1040 }, { "created_ino": "0x100000004eb", "tid": 1043 }, { "created_ino": "0x100000004ec", "tid": 1046 }, { "created_ino": "0x100000004ed", "tid": 1049 }, { "created_ino": "0x100000004ee", "tid": 1052 }, { "created_ino": "0x100000004ef", "tid": 1055 }, { "created_ino": "0x100000004f0", "tid": 1058 }, { "created_ino": "0x100000004f1", "tid": 1061 }, { "created_ino": "0x100000004f2", "tid": 1064 }, { "created_ino": "0x100000004f3", "tid": 1067 }, { "created_ino": "0x100000004f4", "tid": 1070 }, { "created_ino": "0x100000004f5", "tid": 1073 }, { "created_ino": "0x100000004f6", "tid": 1076 }, { "created_ino": "0x100000004f7", "tid": 1079 }, { "created_ino": "0x100000004f8", "tid": 1082 }, { "created_ino": "0x100000004f9", "tid": 1085 }, { "created_ino": "0x100000004fa", "tid": 1088 }, { "created_ino": "0x100000004fb", "tid": 1091 }, { "created_ino": "0x100000004fc", "tid": 1094 }, { "created_ino": "0x100000004fd", "tid": 1097 }, { "created_ino": "0x100000004fe", "tid": 1100 }, { "created_ino": "0x100000004ff", "tid": 1103 }, { "created_ino": "0x10000000500", "tid": 1106 }, { "created_ino": "0x10000000501", "tid": 1109 }, { "created_ino": "0x10000000502", "tid": 1112 }, { "created_ino": "0x10000000503", "tid": 1115 }, { "created_ino": "0x10000000504", "tid": 1118 }, { "created_ino": "0x10000000505", "tid": 1121 }, { "created_ino": "0x10000000506", "tid": 1124 }, { "created_ino": "0x10000000507", "tid": 1127 }, { "created_ino": "0x10000000508", "tid": 1130 }, { "created_ino": "0x10000000509", "tid": 1133 }, { "created_ino": "0x1000000050a", "tid": 1136 }, { "created_ino": "0x1000000050b", "tid": 1139 }, { "created_ino": "0x1000000050c", "tid": 1142 }, { "created_ino": "0x1000000050d", "tid": 1145 }, { "created_ino": "0x1000000050e", "tid": 1148 }, { "created_ino": "0x1000000050f", "tid": 1151 }, { "created_ino": "0x10000000510", "tid": 1154 }, { "created_ino": "0x10000000511", "tid": 1157 }, { "created_ino": "0x10000000512", "tid": 1160 }, { "created_ino": "0x10000000513", "tid": 1163 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1440089416, "type": "any" }, "name": { "num": 7551, "type": "client" } }, "id": 7551, "inst": "client.7551 192.168.0.1:0/1440089416", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 506, "num_caps": 802, "num_completed_flushes": 0, "num_completed_requests": 299, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 203, "start": "0x10000000514" }, { "length": 501, "start": "0x100000005e0" } ], "recall_caps": { "halflife": 60, "value": 2675.133299330704 }, "recall_caps_throttle": { "halflife": 1.5, "value": 662.8494676457818 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 138.78570995292054 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1070, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1579.0463304069372 }, "state": "open", "uptime": 12.500824612 } ] 2024-09-19T07:54:07.444 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-19T07:54:07.444 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:54:07.445 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:54:07.488 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:54:07.538 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:54:07.538 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:54:07.538 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 800\nabs_path = "/home/ubuntu/cephtest/mnt.0/subdir"\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi016') 2024-09-19T07:54:07.538 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:54:07.538 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:54:07.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:54:07.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.697+0000 7f6647d91640 1 Processor -- start 2024-09-19T07:54:07.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.697+0000 7f6647d91640 1 -- start start 2024-09-19T07:54:07.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.698+0000 7f6647d91640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f66400b0a80 0x7f66400cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:07.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.698+0000 7f6647d91640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6640161d90 0x7f66400cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:07.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.698+0000 7f6647d91640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f664006c990 con 0x7f66401619b0 2024-09-19T07:54:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.698+0000 7f6647d91640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f66400cec10 con 0x7f6640161d90 2024-09-19T07:54:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.698+0000 7f6647d91640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f66400ce930 con 0x7f66400b0a80 2024-09-19T07:54:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.698+0000 7f664632e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6640161d90 0x7f66400cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.698+0000 7f664632e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6640161d90 0x7f66400cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38460/0 (socket says 172.21.15.16:38460) 2024-09-19T07:54:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.698+0000 7f6645b2d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f66401619b0 0x7f66400d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60584/0 (socket says 172.21.15.16:60584) 2024-09-19T07:54:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.698+0000 7f664632e640 1 -- 172.21.15.16:0/2710161003 learned_addr learned my addr 172.21.15.16:0/2710161003 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.698+0000 7f664532c640 1 --2- 172.21.15.16:0/2710161003 >> v2:172.21.15.146:3300/0 conn(0x7f66400b0a80 0x7f66400cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.698+0000 7f664632e640 1 -- 172.21.15.16:0/2710161003 >> v2:172.21.15.146:3300/0 conn(0x7f66400b0a80 msgr2=0x7f66400cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.698+0000 7f664632e640 1 --2- 172.21.15.16:0/2710161003 >> v2:172.21.15.146:3300/0 conn(0x7f66400b0a80 0x7f66400cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.698+0000 7f664632e640 1 -- 172.21.15.16:0/2710161003 >> v1:172.21.15.16:6789/0 conn(0x7f66401619b0 legacy=0x7f66400d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.698+0000 7f664632e640 1 -- 172.21.15.16:0/2710161003 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66400b0540 con 0x7f6640161d90 2024-09-19T07:54:07.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.699+0000 7f664632e640 1 --2- 172.21.15.16:0/2710161003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640161d90 0x7f66400cc210 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f66400b39c0 tx=0x7f663802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa1cf4971daf708 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:07.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.699+0000 7f6644b2b640 1 -- 172.21.15.16:0/2710161003 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638036070 con 0x7f6640161d90 2024-09-19T07:54:07.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.699+0000 7f6644b2b640 1 -- 172.21.15.16:0/2710161003 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6638031600 con 0x7f6640161d90 2024-09-19T07:54:07.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.699+0000 7f6644b2b640 1 -- 172.21.15.16:0/2710161003 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66380319d0 con 0x7f6640161d90 2024-09-19T07:54:07.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.699+0000 7f6647d91640 1 -- 172.21.15.16:0/2710161003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640161d90 msgr2=0x7f66400cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:07.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.699+0000 7f6647d91640 1 --2- 172.21.15.16:0/2710161003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640161d90 0x7f66400cc210 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f66400b39c0 tx=0x7f663802a730 comp rx=0 tx=0).stop 2024-09-19T07:54:07.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.700+0000 7f6647d91640 1 -- 172.21.15.16:0/2710161003 shutdown_connections 2024-09-19T07:54:07.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.700+0000 7f6647d91640 1 --2- 172.21.15.16:0/2710161003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640161d90 0x7f66400cc210 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:07.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.700+0000 7f6647d91640 1 --2- 172.21.15.16:0/2710161003 >> v2:172.21.15.146:3300/0 conn(0x7f66400b0a80 0x7f66400cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:07.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.700+0000 7f6647d91640 1 -- 172.21.15.16:0/2710161003 >> 172.21.15.16:0/2710161003 conn(0x7f664009a9f0 msgr2=0x7f66400b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:07.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.700+0000 7f6647d91640 1 -- 172.21.15.16:0/2710161003 shutdown_connections 2024-09-19T07:54:07.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.700+0000 7f6647d91640 1 -- 172.21.15.16:0/2710161003 wait complete. 2024-09-19T07:54:07.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.701+0000 7f6647d91640 1 Processor -- start 2024-09-19T07:54:07.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.701+0000 7f6647d91640 1 -- start start 2024-09-19T07:54:07.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f6647d91640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66400b0a80 0x7f66401f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:07.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f6647d91640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66401619b0 0x7f66400c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:07.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f6647d91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640161d90 0x7f66400c5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:07.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f6647d91640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66400cd330 con 0x7f6640161d90 2024-09-19T07:54:07.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f6647d91640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f664006c620 con 0x7f66400b0a80 2024-09-19T07:54:07.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f6647d91640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f66400cd4b0 con 0x7f66401619b0 2024-09-19T07:54:07.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f664632e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640161d90 0x7f66400c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:07.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f664632e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640161d90 0x7f66400c5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38474/0 (socket says 172.21.15.16:38474) 2024-09-19T07:54:07.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f664632e640 1 -- 172.21.15.16:0/251892984 learned_addr learned my addr 172.21.15.16:0/251892984 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f664632e640 1 -- 172.21.15.16:0/251892984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66401619b0 msgr2=0x7f66400c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f664632e640 1 --2- 172.21.15.16:0/251892984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66401619b0 0x7f66400c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f664632e640 1 -- 172.21.15.16:0/251892984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66400b0a80 msgr2=0x7f66401f3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f6645b2d640 1 --2- 172.21.15.16:0/251892984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66400b0a80 0x7f66401f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f664632e640 1 --2- 172.21.15.16:0/251892984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66400b0a80 0x7f66401f3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f664632e640 1 -- 172.21.15.16:0/251892984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66400c7cf0 con 0x7f6640161d90 2024-09-19T07:54:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f664632e640 1 --2- 172.21.15.16:0/251892984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640161d90 0x7f66400c5930 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f6638009f40 tx=0x7f6638004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f66367fc640 1 -- 172.21.15.16:0/251892984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638036050 con 0x7f6640161d90 2024-09-19T07:54:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f66367fc640 1 -- 172.21.15.16:0/251892984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6638042070 con 0x7f6640161d90 2024-09-19T07:54:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f6647d91640 1 -- 172.21.15.16:0/251892984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66400c5e70 con 0x7f6640161d90 2024-09-19T07:54:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f66367fc640 1 -- 172.21.15.16:0/251892984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638009510 con 0x7f6640161d90 2024-09-19T07:54:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.702+0000 7f6647d91640 1 -- 172.21.15.16:0/251892984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66401f46a0 con 0x7f6640161d90 2024-09-19T07:54:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.703+0000 7f6647d91640 1 -- 172.21.15.16:0/251892984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6640160b50 con 0x7f6640161d90 2024-09-19T07:54:07.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.704+0000 7f66367fc640 1 -- 172.21.15.16:0/251892984 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6638007bf0 con 0x7f6640161d90 2024-09-19T07:54:07.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.705+0000 7f66367fc640 1 --2- 172.21.15.16:0/251892984 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f662406c2b0 0x7f662406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:07.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.705+0000 7f66367fc640 1 -- 172.21.15.16:0/251892984 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f66380ac370 con 0x7f6640161d90 2024-09-19T07:54:07.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.705+0000 7f6645b2d640 1 --2- 172.21.15.16:0/251892984 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f662406c2b0 0x7f662406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:07.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.705+0000 7f6645b2d640 1 --2- 172.21.15.16:0/251892984 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f662406c2b0 0x7f662406e770 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f663c000f60 tx=0x7f663c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:07.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.709+0000 7f66367fc640 1 -- 172.21.15.16:0/251892984 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f66400c5e70 con 0x7f6640161d90 2024-09-19T07:54:07.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.895+0000 7f6647d91640 1 -- 172.21.15.16:0/251892984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f66401f5000 con 0x7f6640161d90 2024-09-19T07:54:07.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.896+0000 7f66367fc640 1 -- 172.21.15.16:0/251892984 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v118) ==== 80+0+0 (secure 0 0 0) 0x7f66400c7cf0 con 0x7f6640161d90 2024-09-19T07:54:07.896 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:54:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.898+0000 7f6647d91640 1 -- 172.21.15.16:0/251892984 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f662406c2b0 msgr2=0x7f662406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.898+0000 7f6647d91640 1 --2- 172.21.15.16:0/251892984 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f662406c2b0 0x7f662406e770 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f663c000f60 tx=0x7f663c009040 comp rx=0 tx=0).stop 2024-09-19T07:54:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.898+0000 7f6647d91640 1 -- 172.21.15.16:0/251892984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640161d90 msgr2=0x7f66400c5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.898+0000 7f6647d91640 1 --2- 172.21.15.16:0/251892984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640161d90 0x7f66400c5930 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f6638009f40 tx=0x7f6638004a40 comp rx=0 tx=0).stop 2024-09-19T07:54:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.898+0000 7f6647d91640 1 -- 172.21.15.16:0/251892984 shutdown_connections 2024-09-19T07:54:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.898+0000 7f6647d91640 1 --2- 172.21.15.16:0/251892984 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f662406c2b0 0x7f662406e770 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.898+0000 7f6647d91640 1 --2- 172.21.15.16:0/251892984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640161d90 0x7f66400c5930 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:07.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.898+0000 7f6647d91640 1 --2- 172.21.15.16:0/251892984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66401619b0 0x7f66400c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:07.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.898+0000 7f6647d91640 1 --2- 172.21.15.16:0/251892984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66400b0a80 0x7f66401f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:07.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.898+0000 7f6647d91640 1 -- 172.21.15.16:0/251892984 >> 172.21.15.16:0/251892984 conn(0x7f664009a9f0 msgr2=0x7f664006fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:07.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.898+0000 7f6647d91640 1 -- 172.21.15.16:0/251892984 shutdown_connections 2024-09-19T07:54:07.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:07.898+0000 7f6647d91640 1 -- 172.21.15.16:0/251892984 wait complete. 2024-09-19T07:54:07.909 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-19T07:54:07.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:07.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T07:54:07.933 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[69634]: fuse finished with error 0 and tester_r 0 2024-09-19T07:54:07.935 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:54:07.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:07.935 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:54:08.131 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:54:08.131 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:54:08.131 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:54:08.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:54:08.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.277+0000 7f4d0668c640 1 Processor -- start 2024-09-19T07:54:08.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.277+0000 7f4d0668c640 1 -- start start 2024-09-19T07:54:08.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.278+0000 7f4d0668c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4d000d0070 0x7f4d0015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:08.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.278+0000 7f4d0668c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d000d4130 0x7f4d000d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:08.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.278+0000 7f4d0668c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d000d4ae0 con 0x7f4d0015f0d0 2024-09-19T07:54:08.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.278+0000 7f4d0668c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4d000d1b10 con 0x7f4d000d0070 2024-09-19T07:54:08.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.278+0000 7f4d0668c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4d000cf2b0 con 0x7f4d000d4130 2024-09-19T07:54:08.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.278+0000 7f4d04c29640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d000d4130 0x7f4d000d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.279+0000 7f4cff7fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4d000d0070 0x7f4d0015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.279+0000 7f4d04c29640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d000d4130 0x7f4d000d45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:33274/0 (socket says 172.21.15.16:33274) 2024-09-19T07:54:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.279+0000 7f4d04c29640 1 -- 172.21.15.16:0/652640780 learned_addr learned my addr 172.21.15.16:0/652640780 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.279+0000 7f4cfffff640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f4d0015f0d0 0x7f4d000d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:53440/0 (socket says 172.21.15.16:53440) 2024-09-19T07:54:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.279+0000 7f4cfeffd640 1 -- 172.21.15.16:0/652640780 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4097214668 0 0) 0x7f4d000d4ae0 con 0x7f4d0015f0d0 2024-09-19T07:54:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.279+0000 7f4cfeffd640 1 -- 172.21.15.16:0/652640780 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ce8003520 con 0x7f4d0015f0d0 2024-09-19T07:54:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.279+0000 7f4cff7fe640 1 -- 172.21.15.16:0/652640780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d000d4130 msgr2=0x7f4d000d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.279+0000 7f4cff7fe640 1 --2- 172.21.15.16:0/652640780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d000d4130 0x7f4d000d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.279+0000 7f4cff7fe640 1 -- 172.21.15.16:0/652640780 >> v1:172.21.15.146:6789/0 conn(0x7f4d0015f0d0 legacy=0x7f4d000d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.279+0000 7f4cff7fe640 1 -- 172.21.15.16:0/652640780 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d000cf590 con 0x7f4d000d0070 2024-09-19T07:54:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.279+0000 7f4d04c29640 1 --2- 172.21.15.16:0/652640780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d000d4130 0x7f4d000d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:54:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.280+0000 7f4cff7fe640 1 --2- 172.21.15.16:0/652640780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d000d0070 0x7f4d0015f4b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40097a0 tx=0x7f4cf402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3211c7fb92a81ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.280+0000 7f4cfeffd640 1 -- 172.21.15.16:0/652640780 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4037070 con 0x7f4d000d0070 2024-09-19T07:54:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.280+0000 7f4cfeffd640 1 -- 172.21.15.16:0/652640780 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf402ae50 con 0x7f4d000d0070 2024-09-19T07:54:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.280+0000 7f4cfeffd640 1 -- 172.21.15.16:0/652640780 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4032660 con 0x7f4d000d0070 2024-09-19T07:54:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.280+0000 7f4d0668c640 1 -- 172.21.15.16:0/652640780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d000d0070 msgr2=0x7f4d0015f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.280+0000 7f4d0668c640 1 --2- 172.21.15.16:0/652640780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d000d0070 0x7f4d0015f4b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40097a0 tx=0x7f4cf402a100 comp rx=0 tx=0).stop 2024-09-19T07:54:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.280+0000 7f4d0668c640 1 -- 172.21.15.16:0/652640780 shutdown_connections 2024-09-19T07:54:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.280+0000 7f4d0668c640 1 --2- 172.21.15.16:0/652640780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d000d4130 0x7f4d000d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.280+0000 7f4d0668c640 1 --2- 172.21.15.16:0/652640780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d000d0070 0x7f4d0015f4b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.280+0000 7f4d0668c640 1 -- 172.21.15.16:0/652640780 >> 172.21.15.16:0/652640780 conn(0x7f4d0009d760 msgr2=0x7f4d0008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.281+0000 7f4d0668c640 1 -- 172.21.15.16:0/652640780 shutdown_connections 2024-09-19T07:54:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.281+0000 7f4d0668c640 1 -- 172.21.15.16:0/652640780 wait complete. 2024-09-19T07:54:08.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.282+0000 7f4d0668c640 1 Processor -- start 2024-09-19T07:54:08.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.283+0000 7f4d0668c640 1 -- start start 2024-09-19T07:54:08.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.284+0000 7f4d0668c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d000d0070 0x7f4d0014e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:08.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.284+0000 7f4d0668c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d000d4130 0x7f4d0015b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:08.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.284+0000 7f4d0668c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d0015f0d0 0x7f4d0015d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:08.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.284+0000 7f4d0668c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d000cef60 con 0x7f4d0015f0d0 2024-09-19T07:54:08.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.284+0000 7f4d0668c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4d000cfb50 con 0x7f4d000d0070 2024-09-19T07:54:08.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.284+0000 7f4d0668c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4d000b2f70 con 0x7f4d000d4130 2024-09-19T07:54:08.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.284+0000 7f4d04c29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d0015f0d0 0x7f4d0015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:08.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.284+0000 7f4d04c29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d0015f0d0 0x7f4d0015d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38482/0 (socket says 172.21.15.16:38482) 2024-09-19T07:54:08.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.284+0000 7f4d04c29640 1 -- 172.21.15.16:0/2548032047 learned_addr learned my addr 172.21.15.16:0/2548032047 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:08.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.284+0000 7f4cfffff640 1 --2- 172.21.15.16:0/2548032047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d000d0070 0x7f4d0014e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:08.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.284+0000 7f4cff7fe640 1 --2- 172.21.15.16:0/2548032047 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d000d4130 0x7f4d0015b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:08.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.285+0000 7f4d04c29640 1 -- 172.21.15.16:0/2548032047 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d000d4130 msgr2=0x7f4d0015b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:08.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.285+0000 7f4d04c29640 1 --2- 172.21.15.16:0/2548032047 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d000d4130 0x7f4d0015b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:08.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.285+0000 7f4d04c29640 1 -- 172.21.15.16:0/2548032047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d000d0070 msgr2=0x7f4d0014e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:08.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.285+0000 7f4d04c29640 1 --2- 172.21.15.16:0/2548032047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d000d0070 0x7f4d0014e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:08.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.285+0000 7f4d04c29640 1 -- 172.21.15.16:0/2548032047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d000cd700 con 0x7f4d0015f0d0 2024-09-19T07:54:08.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.285+0000 7f4d04c29640 1 --2- 172.21.15.16:0/2548032047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d0015f0d0 0x7f4d0015d950 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800efc0 tx=0x7f4cf800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:08.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.285+0000 7f4cfcff9640 1 -- 172.21.15.16:0/2548032047 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf800cbe0 con 0x7f4d0015f0d0 2024-09-19T07:54:08.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.285+0000 7f4cfcff9640 1 -- 172.21.15.16:0/2548032047 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf800cd80 con 0x7f4d0015f0d0 2024-09-19T07:54:08.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.285+0000 7f4d0668c640 1 -- 172.21.15.16:0/2548032047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d000cd9e0 con 0x7f4d0015f0d0 2024-09-19T07:54:08.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.285+0000 7f4cfcff9640 1 -- 172.21.15.16:0/2548032047 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8003bb0 con 0x7f4d0015f0d0 2024-09-19T07:54:08.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.285+0000 7f4d0668c640 1 -- 172.21.15.16:0/2548032047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d000cdfa0 con 0x7f4d0015f0d0 2024-09-19T07:54:08.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.286+0000 7f4d0668c640 1 -- 172.21.15.16:0/2548032047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d0015f4b0 con 0x7f4d0015f0d0 2024-09-19T07:54:08.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.287+0000 7f4cfcff9640 1 -- 172.21.15.16:0/2548032047 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4cf801b030 con 0x7f4d0015f0d0 2024-09-19T07:54:08.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.287+0000 7f4cfcff9640 1 --2- 172.21.15.16:0/2548032047 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4cec06c2b0 0x7f4cec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:08.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.287+0000 7f4cfcff9640 1 -- 172.21.15.16:0/2548032047 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f4cf808b9e0 con 0x7f4d0015f0d0 2024-09-19T07:54:08.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.287+0000 7f4cfffff640 1 --2- 172.21.15.16:0/2548032047 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4cec06c2b0 0x7f4cec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:08.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.288+0000 7f4cfffff640 1 --2- 172.21.15.16:0/2548032047 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4cec06c2b0 0x7f4cec06e770 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4d0015c010 tx=0x7f4cf0007b10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:08.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.291+0000 7f4cfcff9640 1 -- 172.21.15.16:0/2548032047 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4cf8022020 con 0x7f4d0015f0d0 2024-09-19T07:54:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.501+0000 7f4d0668c640 1 -- 172.21.15.16:0/2548032047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4d0006cf30 con 0x7f4d0015f0d0 2024-09-19T07:54:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.501+0000 7f4cfcff9640 1 -- 172.21.15.16:0/2548032047 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v118) ==== 80+0+0 (secure 0 0 0) 0x7f4cf8010040 con 0x7f4d0015f0d0 2024-09-19T07:54:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:54:08.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.503+0000 7f4d0668c640 1 -- 172.21.15.16:0/2548032047 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4cec06c2b0 msgr2=0x7f4cec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:08.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.503+0000 7f4d0668c640 1 --2- 172.21.15.16:0/2548032047 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4cec06c2b0 0x7f4cec06e770 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4d0015c010 tx=0x7f4cf0007b10 comp rx=0 tx=0).stop 2024-09-19T07:54:08.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.503+0000 7f4d0668c640 1 -- 172.21.15.16:0/2548032047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d0015f0d0 msgr2=0x7f4d0015d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.503+0000 7f4d0668c640 1 --2- 172.21.15.16:0/2548032047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d0015f0d0 0x7f4d0015d950 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800efc0 tx=0x7f4cf800c490 comp rx=0 tx=0).stop 2024-09-19T07:54:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.503+0000 7f4d0668c640 1 -- 172.21.15.16:0/2548032047 shutdown_connections 2024-09-19T07:54:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.503+0000 7f4d0668c640 1 --2- 172.21.15.16:0/2548032047 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4cec06c2b0 0x7f4cec06e770 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.503+0000 7f4d0668c640 1 --2- 172.21.15.16:0/2548032047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d0015f0d0 0x7f4d0015d950 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.503+0000 7f4d0668c640 1 --2- 172.21.15.16:0/2548032047 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d000d4130 0x7f4d0015b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.503+0000 7f4d0668c640 1 --2- 172.21.15.16:0/2548032047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d000d0070 0x7f4d0014e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.503+0000 7f4d0668c640 1 -- 172.21.15.16:0/2548032047 >> 172.21.15.16:0/2548032047 conn(0x7f4d0009d760 msgr2=0x7f4d0006dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.503+0000 7f4d0668c640 1 -- 172.21.15.16:0/2548032047 shutdown_connections 2024-09-19T07:54:08.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.503+0000 7f4d0668c640 1 -- 172.21.15.16:0/2548032047 wait complete. 2024-09-19T07:54:08.515 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2024-09-19T07:54:08.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:08.515 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T07:54:08.544 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[48393]: fuse finished with error 0 and tester_r 0 2024-09-19T07:54:08.546 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:54:08.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:08.546 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T07:54:08.667 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:54:08.667 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:54:08.667 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:54:08.684 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:54:08.684 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:54:08.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:08.684 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T07:54:08.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:54:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T07:54:08.741 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:54:08.741 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:54:08.741 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:54:08.759 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:54:08.759 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:54:08.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:08.759 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T07:54:08.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:54:08.818 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T07:54:08.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:54:08.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.922+0000 7f7c6e503640 1 Processor -- start 2024-09-19T07:54:08.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.922+0000 7f7c6e503640 1 -- start start 2024-09-19T07:54:08.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.923+0000 7f7c6e503640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7c6815e2a0 0x7f7c6806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:08.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.923+0000 7f7c6e503640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7c6806d960 0x7f7c680d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:08.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.923+0000 7f7c6e503640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c680d1880 con 0x7f7c6815e680 2024-09-19T07:54:08.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.923+0000 7f7c6e503640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f7c680d1b60 con 0x7f7c6806d960 2024-09-19T07:54:08.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.923+0000 7f7c6e503640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f7c680cf2d0 con 0x7f7c6815e2a0 2024-09-19T07:54:08.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.923+0000 7f7c6caa0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7c6806d960 0x7f7c680d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:08.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.923+0000 7f7c5ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7c6815e680 0x7f7c680cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60600/0 (socket says 172.21.15.16:60600) 2024-09-19T07:54:08.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.923+0000 7f7c5ffff640 1 -- 172.21.15.16:0/2028633766 learned_addr learned my addr 172.21.15.16:0/2028633766 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:08.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.924+0000 7f7c67fff640 1 --2- 172.21.15.16:0/2028633766 >> v2:172.21.15.146:3300/0 conn(0x7f7c6815e2a0 0x7f7c6806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:08.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.924+0000 7f7c6caa0640 1 -- 172.21.15.16:0/2028633766 >> v2:172.21.15.146:3300/0 conn(0x7f7c6815e2a0 msgr2=0x7f7c6806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:08.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.924+0000 7f7c6caa0640 1 --2- 172.21.15.16:0/2028633766 >> v2:172.21.15.146:3300/0 conn(0x7f7c6815e2a0 0x7f7c6806d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:08.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.924+0000 7f7c6caa0640 1 -- 172.21.15.16:0/2028633766 >> v1:172.21.15.16:6789/0 conn(0x7f7c6815e680 legacy=0x7f7c680cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:08.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.924+0000 7f7c6caa0640 1 -- 172.21.15.16:0/2028633766 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c680cf5b0 con 0x7f7c6806d960 2024-09-19T07:54:08.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.924+0000 7f7c677fe640 1 -- 172.21.15.16:0/2028633766 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2188114915 0 0) 0x7f7c680d1880 con 0x7f7c6815e680 2024-09-19T07:54:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.924+0000 7f7c6caa0640 1 --2- 172.21.15.16:0/2028633766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c6806d960 0x7f7c680d4370 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f7c680b6fa0 tx=0x7f7c6002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c4e65f0ecf8624 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.925+0000 7f7c677fe640 1 -- 172.21.15.16:0/2028633766 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c60007740 con 0x7f7c6806d960 2024-09-19T07:54:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.925+0000 7f7c677fe640 1 -- 172.21.15.16:0/2028633766 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c600078e0 con 0x7f7c6806d960 2024-09-19T07:54:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.925+0000 7f7c677fe640 1 -- 172.21.15.16:0/2028633766 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c60007cb0 con 0x7f7c6806d960 2024-09-19T07:54:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.925+0000 7f7c6e503640 1 -- 172.21.15.16:0/2028633766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c6806d960 msgr2=0x7f7c680d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.925+0000 7f7c6e503640 1 --2- 172.21.15.16:0/2028633766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c6806d960 0x7f7c680d4370 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f7c680b6fa0 tx=0x7f7c6002a680 comp rx=0 tx=0).stop 2024-09-19T07:54:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.925+0000 7f7c6e503640 1 -- 172.21.15.16:0/2028633766 shutdown_connections 2024-09-19T07:54:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.925+0000 7f7c6e503640 1 --2- 172.21.15.16:0/2028633766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c6806d960 0x7f7c680d4370 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.925+0000 7f7c6e503640 1 --2- 172.21.15.16:0/2028633766 >> v2:172.21.15.146:3300/0 conn(0x7f7c6815e2a0 0x7f7c6806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.925+0000 7f7c6e503640 1 -- 172.21.15.16:0/2028633766 >> 172.21.15.16:0/2028633766 conn(0x7f7c6809d760 msgr2=0x7f7c6808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:08.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.925+0000 7f7c6e503640 1 -- 172.21.15.16:0/2028633766 shutdown_connections 2024-09-19T07:54:08.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.925+0000 7f7c6e503640 1 -- 172.21.15.16:0/2028633766 wait complete. 2024-09-19T07:54:08.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.926+0000 7f7c6e503640 1 Processor -- start 2024-09-19T07:54:08.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.927+0000 7f7c6e503640 1 -- start start 2024-09-19T07:54:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.927+0000 7f7c6e503640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c6806d960 0x7f7c6815d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.927+0000 7f7c6e503640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c6815e2a0 0x7f7c681ea1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.927+0000 7f7c6e503640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c6815e680 0x7f7c681ea6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.927+0000 7f7c6e503640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c680d0360 con 0x7f7c6806d960 2024-09-19T07:54:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.927+0000 7f7c6e503640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7c680d01e0 con 0x7f7c6815e680 2024-09-19T07:54:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.927+0000 7f7c6e503640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7c680d48b0 con 0x7f7c6815e2a0 2024-09-19T07:54:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.927+0000 7f7c67fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c6806d960 0x7f7c6815d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.927+0000 7f7c67fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c6806d960 0x7f7c6815d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38500/0 (socket says 172.21.15.16:38500) 2024-09-19T07:54:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.927+0000 7f7c67fff640 1 -- 172.21.15.16:0/4000639631 learned_addr learned my addr 172.21.15.16:0/4000639631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.928+0000 7f7c6caa0640 1 --2- 172.21.15.16:0/4000639631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c6815e680 0x7f7c681ea6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.928+0000 7f7c5ffff640 1 --2- 172.21.15.16:0/4000639631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c6815e2a0 0x7f7c681ea1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:08.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.928+0000 7f7c67fff640 1 -- 172.21.15.16:0/4000639631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c6815e2a0 msgr2=0x7f7c681ea1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:08.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.928+0000 7f7c67fff640 1 --2- 172.21.15.16:0/4000639631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c6815e2a0 0x7f7c681ea1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:08.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.928+0000 7f7c67fff640 1 -- 172.21.15.16:0/4000639631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c6815e680 msgr2=0x7f7c681ea6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:08.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.928+0000 7f7c67fff640 1 --2- 172.21.15.16:0/4000639631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c6815e680 0x7f7c681ea6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:08.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.928+0000 7f7c67fff640 1 -- 172.21.15.16:0/4000639631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6815d8c0 con 0x7f7c6806d960 2024-09-19T07:54:08.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.928+0000 7f7c6caa0640 1 --2- 172.21.15.16:0/4000639631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c6815e680 0x7f7c681ea6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:08.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.928+0000 7f7c67fff640 1 --2- 172.21.15.16:0/4000639631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c6806d960 0x7f7c6815d380 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800d830 tx=0x7f7c5800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:08.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.928+0000 7f7c657fa640 1 -- 172.21.15.16:0/4000639631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5800b6c0 con 0x7f7c6806d960 2024-09-19T07:54:08.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.928+0000 7f7c657fa640 1 -- 172.21.15.16:0/4000639631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c58019070 con 0x7f7c6806d960 2024-09-19T07:54:08.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.928+0000 7f7c6e503640 1 -- 172.21.15.16:0/4000639631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c680c8fd0 con 0x7f7c6806d960 2024-09-19T07:54:08.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.928+0000 7f7c657fa640 1 -- 172.21.15.16:0/4000639631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5800ba80 con 0x7f7c6806d960 2024-09-19T07:54:08.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.928+0000 7f7c6e503640 1 -- 172.21.15.16:0/4000639631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c680c9270 con 0x7f7c6806d960 2024-09-19T07:54:08.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.929+0000 7f7c6e503640 1 -- 172.21.15.16:0/4000639631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c680d4c70 con 0x7f7c6806d960 2024-09-19T07:54:08.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.930+0000 7f7c657fa640 1 -- 172.21.15.16:0/4000639631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7c5800bc20 con 0x7f7c6806d960 2024-09-19T07:54:08.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.930+0000 7f7c657fa640 1 --2- 172.21.15.16:0/4000639631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7c4806c2b0 0x7f7c4806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:08.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.930+0000 7f7c657fa640 1 -- 172.21.15.16:0/4000639631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f7c5808ba90 con 0x7f7c6806d960 2024-09-19T07:54:08.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.930+0000 7f7c5ffff640 1 --2- 172.21.15.16:0/4000639631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7c4806c2b0 0x7f7c4806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:08.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.931+0000 7f7c5ffff640 1 --2- 172.21.15.16:0/4000639631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7c4806c2b0 0x7f7c4806e770 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7c680b95a0 tx=0x7f7c50007d00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:08.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:08.934+0000 7f7c657fa640 1 -- 172.21.15.16:0/4000639631 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7c6815d8c0 con 0x7f7c6806d960 2024-09-19T07:54:09.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.143+0000 7f7c6e503640 1 -- 172.21.15.16:0/4000639631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c6806cf90 con 0x7f7c6806d960 2024-09-19T07:54:09.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.144+0000 7f7c657fa640 1 -- 172.21.15.16:0/4000639631 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5231 (secure 0 0 0) 0x7f7c5801c360 con 0x7f7c6806d960 2024-09-19T07:54:09.144 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:09.145 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":65,"btime":"2024-09-19T07:54:05:204476+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2471408352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2471408352},{"type":"v1","addr":"172.21.15.16:6833","nonce":2471408352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/1281369866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1281369866},{"type":"v1","addr":"172.21.15.146:6836","nonce":1281369866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1377303883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1377303883},{"type":"v1","addr":"172.21.15.146:6837","nonce":1377303883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:53:43.812066+0000","modified":"2024-09-19T07:54:04.488075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7368":{"gid":7368,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/2050126415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2050126415},{"type":"v1","addr":"172.21.15.16:6835","nonce":2050126415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7368,"qdb_cluster":[7368]},"id":8}]} 2024-09-19T07:54:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 65 2024-09-19T07:54:09.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.146+0000 7f7c6e503640 1 -- 172.21.15.16:0/4000639631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7c4806c2b0 msgr2=0x7f7c4806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.146+0000 7f7c6e503640 1 --2- 172.21.15.16:0/4000639631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7c4806c2b0 0x7f7c4806e770 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7c680b95a0 tx=0x7f7c50007d00 comp rx=0 tx=0).stop 2024-09-19T07:54:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.146+0000 7f7c6e503640 1 -- 172.21.15.16:0/4000639631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c6806d960 msgr2=0x7f7c6815d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.146+0000 7f7c6e503640 1 --2- 172.21.15.16:0/4000639631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c6806d960 0x7f7c6815d380 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800d830 tx=0x7f7c5800dcf0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.146+0000 7f7c6e503640 1 -- 172.21.15.16:0/4000639631 shutdown_connections 2024-09-19T07:54:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.146+0000 7f7c6e503640 1 --2- 172.21.15.16:0/4000639631 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7c4806c2b0 0x7f7c4806e770 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.146+0000 7f7c6e503640 1 --2- 172.21.15.16:0/4000639631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c6815e680 0x7f7c681ea6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.146+0000 7f7c6e503640 1 --2- 172.21.15.16:0/4000639631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c6815e2a0 0x7f7c681ea1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.146+0000 7f7c6e503640 1 --2- 172.21.15.16:0/4000639631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c6806d960 0x7f7c6815d380 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.146+0000 7f7c6e503640 1 -- 172.21.15.16:0/4000639631 >> 172.21.15.16:0/4000639631 conn(0x7f7c6809d760 msgr2=0x7f7c68160970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.146+0000 7f7c6e503640 1 -- 172.21.15.16:0/4000639631 shutdown_connections 2024-09-19T07:54:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.146+0000 7f7c6e503640 1 -- 172.21.15.16:0/4000639631 wait complete. 2024-09-19T07:54:09.158 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:54:09.201 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:54:09.204 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:54:09.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.261+0000 7f809d39f640 1 Processor -- start 2024-09-19T07:54:09.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.261+0000 7f809d39f640 1 -- start start 2024-09-19T07:54:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.262+0000 7f809d39f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8098161af0 0x7f8098162c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.262+0000 7f809d39f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f80980b47c0 0x7f80980d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.262+0000 7f809d39f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80980d05e0 con 0x7f809809a9f0 2024-09-19T07:54:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.262+0000 7f809d39f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f80980d03b0 con 0x7f8098161af0 2024-09-19T07:54:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.262+0000 7f809d39f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f8098163150 con 0x7f80980b47c0 2024-09-19T07:54:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.262+0000 7f8096ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8098161af0 0x7f8098162c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.262+0000 7f8096ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8098161af0 0x7f8098162c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38506/0 (socket says 172.21.15.16:38506) 2024-09-19T07:54:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.262+0000 7f8096ffd640 1 -- 172.21.15.16:0/4175530358 learned_addr learned my addr 172.21.15.16:0/4175530358 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.263+0000 7f80967fc640 1 --2- 172.21.15.16:0/4175530358 >> v2:172.21.15.146:3300/0 conn(0x7f80980b47c0 0x7f80980d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.263+0000 7f8096ffd640 1 -- 172.21.15.16:0/4175530358 >> v2:172.21.15.146:3300/0 conn(0x7f80980b47c0 msgr2=0x7f80980d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:09.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.263+0000 7f8096ffd640 1 --2- 172.21.15.16:0/4175530358 >> v2:172.21.15.146:3300/0 conn(0x7f80980b47c0 0x7f80980d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.263+0000 7f8096ffd640 1 -- 172.21.15.16:0/4175530358 >> v1:172.21.15.146:6789/0 conn(0x7f809809a9f0 legacy=0x7f80980cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:09.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.263+0000 7f8096ffd640 1 -- 172.21.15.16:0/4175530358 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80980d08c0 con 0x7f8098161af0 2024-09-19T07:54:09.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.264+0000 7f8096ffd640 1 --2- 172.21.15.16:0/4175530358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8098161af0 0x7f8098162c10 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f809806e440 tx=0x7f808802a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87bc25d983c39425 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:09.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.265+0000 7f8095ffb640 1 -- 172.21.15.16:0/4175530358 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8088037070 con 0x7f8098161af0 2024-09-19T07:54:09.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.265+0000 7f8095ffb640 1 -- 172.21.15.16:0/4175530358 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8088032640 con 0x7f8098161af0 2024-09-19T07:54:09.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.265+0000 7f8095ffb640 1 -- 172.21.15.16:0/4175530358 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8088032a10 con 0x7f8098161af0 2024-09-19T07:54:09.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.266+0000 7f809d39f640 1 -- 172.21.15.16:0/4175530358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8098161af0 msgr2=0x7f8098162c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:09.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.266+0000 7f809d39f640 1 --2- 172.21.15.16:0/4175530358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8098161af0 0x7f8098162c10 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f809806e440 tx=0x7f808802a330 comp rx=0 tx=0).stop 2024-09-19T07:54:09.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.266+0000 7f809d39f640 1 -- 172.21.15.16:0/4175530358 shutdown_connections 2024-09-19T07:54:09.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.266+0000 7f809d39f640 1 --2- 172.21.15.16:0/4175530358 >> v2:172.21.15.146:3300/0 conn(0x7f80980b47c0 0x7f80980d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.266+0000 7f809d39f640 1 --2- 172.21.15.16:0/4175530358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8098161af0 0x7f8098162c10 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.266+0000 7f809d39f640 1 -- 172.21.15.16:0/4175530358 >> 172.21.15.16:0/4175530358 conn(0x7f80980b3ca0 msgr2=0x7f80980b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:09.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.266+0000 7f809d39f640 1 -- 172.21.15.16:0/4175530358 shutdown_connections 2024-09-19T07:54:09.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.266+0000 7f809d39f640 1 -- 172.21.15.16:0/4175530358 wait complete. 2024-09-19T07:54:09.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.267+0000 7f809d39f640 1 Processor -- start 2024-09-19T07:54:09.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.267+0000 7f809d39f640 1 -- start start 2024-09-19T07:54:09.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.268+0000 7f809d39f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f809809a9f0 0x7f809814a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.268+0000 7f809d39f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80980b47c0 0x7f809815a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.268+0000 7f809d39f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8098161af0 0x7f809806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.268+0000 7f809d39f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80980d1160 con 0x7f80980b47c0 2024-09-19T07:54:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.268+0000 7f809d39f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f80980d0e80 con 0x7f8098161af0 2024-09-19T07:54:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.268+0000 7f809d39f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8098161ed0 con 0x7f809809a9f0 2024-09-19T07:54:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.268+0000 7f80967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80980b47c0 0x7f809815a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.268+0000 7f80967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80980b47c0 0x7f809815a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38510/0 (socket says 172.21.15.16:38510) 2024-09-19T07:54:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.268+0000 7f80967fc640 1 -- 172.21.15.16:0/483004451 learned_addr learned my addr 172.21.15.16:0/483004451 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.268+0000 7f80977fe640 1 --2- 172.21.15.16:0/483004451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8098161af0 0x7f809806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.268+0000 7f8096ffd640 1 --2- 172.21.15.16:0/483004451 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f809809a9f0 0x7f809814a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:09.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.268+0000 7f80967fc640 1 -- 172.21.15.16:0/483004451 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f809809a9f0 msgr2=0x7f809814a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.268+0000 7f80967fc640 1 --2- 172.21.15.16:0/483004451 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f809809a9f0 0x7f809814a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.268+0000 7f80967fc640 1 -- 172.21.15.16:0/483004451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8098161af0 msgr2=0x7f809806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.268+0000 7f80967fc640 1 --2- 172.21.15.16:0/483004451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8098161af0 0x7f809806cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.268+0000 7f80967fc640 1 -- 172.21.15.16:0/483004451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8098157f90 con 0x7f80980b47c0 2024-09-19T07:54:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.269+0000 7f80967fc640 1 --2- 172.21.15.16:0/483004451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80980b47c0 0x7f809815a0b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f808c00e8a0 tx=0x7f808c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.269+0000 7f807f7fe640 1 -- 172.21.15.16:0/483004451 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c017070 con 0x7f80980b47c0 2024-09-19T07:54:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.269+0000 7f807f7fe640 1 -- 172.21.15.16:0/483004451 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f808c00c8e0 con 0x7f80980b47c0 2024-09-19T07:54:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.269+0000 7f809d39f640 1 -- 172.21.15.16:0/483004451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809815eac0 con 0x7f80980b47c0 2024-09-19T07:54:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.269+0000 7f807f7fe640 1 -- 172.21.15.16:0/483004451 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c00ccd0 con 0x7f80980b47c0 2024-09-19T07:54:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.269+0000 7f809d39f640 1 -- 172.21.15.16:0/483004451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f809815f080 con 0x7f80980b47c0 2024-09-19T07:54:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.270+0000 7f809d39f640 1 -- 172.21.15.16:0/483004451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8098162c10 con 0x7f80980b47c0 2024-09-19T07:54:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.270+0000 7f807f7fe640 1 -- 172.21.15.16:0/483004451 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f808c01a070 con 0x7f80980b47c0 2024-09-19T07:54:09.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.271+0000 7f807f7fe640 1 --2- 172.21.15.16:0/483004451 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f808406c2b0 0x7f808406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:09.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.271+0000 7f807f7fe640 1 -- 172.21.15.16:0/483004451 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f808c08b670 con 0x7f80980b47c0 2024-09-19T07:54:09.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.271+0000 7f8096ffd640 1 --2- 172.21.15.16:0/483004451 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f808406c2b0 0x7f808406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:09.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.271+0000 7f8096ffd640 1 --2- 172.21.15.16:0/483004451 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f808406c2b0 0x7f808406e770 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8088034000 tx=0x7f808802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:09.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.275+0000 7f807f7fe640 1 -- 172.21.15.16:0/483004451 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f808c059690 con 0x7f80980b47c0 2024-09-19T07:54:09.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.486+0000 7f809d39f640 1 -- 172.21.15.16:0/483004451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f809809b850 con 0x7f80980b47c0 2024-09-19T07:54:09.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.487+0000 7f807f7fe640 1 -- 172.21.15.16:0/483004451 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5231 (secure 0 0 0) 0x7f808c05d990 con 0x7f80980b47c0 2024-09-19T07:54:09.487 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:09.487 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":65,"btime":"2024-09-19T07:54:05:204476+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2471408352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2471408352},{"type":"v1","addr":"172.21.15.16:6833","nonce":2471408352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/1281369866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1281369866},{"type":"v1","addr":"172.21.15.146:6836","nonce":1281369866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1377303883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1377303883},{"type":"v1","addr":"172.21.15.146:6837","nonce":1377303883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:53:43.812066+0000","modified":"2024-09-19T07:54:04.488075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7368":{"gid":7368,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/2050126415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2050126415},{"type":"v1","addr":"172.21.15.16:6835","nonce":2050126415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7368,"qdb_cluster":[7368]},"id":8}]} 2024-09-19T07:54:09.488 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 65 2024-09-19T07:54:09.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.489+0000 7f809d39f640 1 -- 172.21.15.16:0/483004451 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f808406c2b0 msgr2=0x7f808406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:09.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.489+0000 7f809d39f640 1 --2- 172.21.15.16:0/483004451 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f808406c2b0 0x7f808406e770 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8088034000 tx=0x7f808802e000 comp rx=0 tx=0).stop 2024-09-19T07:54:09.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.489+0000 7f809d39f640 1 -- 172.21.15.16:0/483004451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80980b47c0 msgr2=0x7f809815a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:09.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.489+0000 7f809d39f640 1 --2- 172.21.15.16:0/483004451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80980b47c0 0x7f809815a0b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f808c00e8a0 tx=0x7f808c00ed60 comp rx=0 tx=0).stop 2024-09-19T07:54:09.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.489+0000 7f809d39f640 1 -- 172.21.15.16:0/483004451 shutdown_connections 2024-09-19T07:54:09.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.489+0000 7f809d39f640 1 --2- 172.21.15.16:0/483004451 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f808406c2b0 0x7f808406e770 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.489+0000 7f809d39f640 1 --2- 172.21.15.16:0/483004451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8098161af0 0x7f809806cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.489+0000 7f809d39f640 1 --2- 172.21.15.16:0/483004451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80980b47c0 0x7f809815a0b0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.489+0000 7f809d39f640 1 --2- 172.21.15.16:0/483004451 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f809809a9f0 0x7f809814a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.489+0000 7f809d39f640 1 -- 172.21.15.16:0/483004451 >> 172.21.15.16:0/483004451 conn(0x7f80980b3ca0 msgr2=0x7f809809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:09.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.489+0000 7f809d39f640 1 -- 172.21.15.16:0/483004451 shutdown_connections 2024-09-19T07:54:09.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.489+0000 7f809d39f640 1 -- 172.21.15.16:0/483004451 wait complete. 2024-09-19T07:54:09.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:54:09.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.642+0000 7f1b8dfe4640 1 Processor -- start 2024-09-19T07:54:09.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.642+0000 7f1b8dfe4640 1 -- start start 2024-09-19T07:54:09.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.643+0000 7f1b8dfe4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b880d0bc0 0x7f1b880cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:09.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.643+0000 7f1b8dfe4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b8815c2c0 0x7f1b880cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:09.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.643+0000 7f1b8dfe4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b880cfc80 con 0x7f1b8815bee0 2024-09-19T07:54:09.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.643+0000 7f1b8dfe4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f1b880ceaf0 con 0x7f1b8815c2c0 2024-09-19T07:54:09.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.643+0000 7f1b8dfe4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1b880cdee0 con 0x7f1b880d0bc0 2024-09-19T07:54:09.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.643+0000 7f1b877fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1b8815bee0 0x7f1b880d04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60610/0 (socket says 172.21.15.16:60610) 2024-09-19T07:54:09.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.643+0000 7f1b877fe640 1 -- 172.21.15.16:0/1551050549 learned_addr learned my addr 172.21.15.16:0/1551050549 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:09.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.643+0000 7f1b87fff640 1 --2- 172.21.15.16:0/1551050549 >> v2:172.21.15.16:3300/0 conn(0x7f1b8815c2c0 0x7f1b880cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:09.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.643+0000 7f1b86ffd640 1 --2- 172.21.15.16:0/1551050549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b880d0bc0 0x7f1b880cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:09.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.643+0000 7f1b867fc640 1 -- 172.21.15.16:0/1551050549 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3060285012 0 0) 0x7f1b880cfc80 con 0x7f1b8815bee0 2024-09-19T07:54:09.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.643+0000 7f1b867fc640 1 -- 172.21.15.16:0/1551050549 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b70003520 con 0x7f1b8815bee0 2024-09-19T07:54:09.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.643+0000 7f1b87fff640 1 -- 172.21.15.16:0/1551050549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b880d0bc0 msgr2=0x7f1b880cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:09.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.643+0000 7f1b87fff640 1 --2- 172.21.15.16:0/1551050549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b880d0bc0 0x7f1b880cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.643+0000 7f1b87fff640 1 -- 172.21.15.16:0/1551050549 >> v1:172.21.15.16:6789/0 conn(0x7f1b8815bee0 legacy=0x7f1b880d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:09.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.643+0000 7f1b87fff640 1 -- 172.21.15.16:0/1551050549 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b880ce1c0 con 0x7f1b8815c2c0 2024-09-19T07:54:09.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.644+0000 7f1b87fff640 1 --2- 172.21.15.16:0/1551050549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b8815c2c0 0x7f1b880cf740 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009d90 tx=0x7f1b8002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0e24d6c2f42ecc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:09.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.644+0000 7f1b867fc640 1 -- 172.21.15.16:0/1551050549 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b80007740 con 0x7f1b8815c2c0 2024-09-19T07:54:09.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.644+0000 7f1b867fc640 1 -- 172.21.15.16:0/1551050549 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b800078e0 con 0x7f1b8815c2c0 2024-09-19T07:54:09.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.644+0000 7f1b867fc640 1 -- 172.21.15.16:0/1551050549 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b80007cb0 con 0x7f1b8815c2c0 2024-09-19T07:54:09.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.645+0000 7f1b8dfe4640 1 -- 172.21.15.16:0/1551050549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b8815c2c0 msgr2=0x7f1b880cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:09.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.645+0000 7f1b8dfe4640 1 --2- 172.21.15.16:0/1551050549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b8815c2c0 0x7f1b880cf740 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009d90 tx=0x7f1b8002a660 comp rx=0 tx=0).stop 2024-09-19T07:54:09.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.645+0000 7f1b8dfe4640 1 -- 172.21.15.16:0/1551050549 shutdown_connections 2024-09-19T07:54:09.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.645+0000 7f1b8dfe4640 1 --2- 172.21.15.16:0/1551050549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b8815c2c0 0x7f1b880cf740 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.645+0000 7f1b8dfe4640 1 --2- 172.21.15.16:0/1551050549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b880d0bc0 0x7f1b880cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.645+0000 7f1b8dfe4640 1 -- 172.21.15.16:0/1551050549 >> 172.21.15.16:0/1551050549 conn(0x7f1b8806bdf0 msgr2=0x7f1b8806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:09.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.645+0000 7f1b8dfe4640 1 -- 172.21.15.16:0/1551050549 shutdown_connections 2024-09-19T07:54:09.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.645+0000 7f1b8dfe4640 1 -- 172.21.15.16:0/1551050549 wait complete. 2024-09-19T07:54:09.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.646+0000 7f1b8dfe4640 1 Processor -- start 2024-09-19T07:54:09.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.646+0000 7f1b8dfe4640 1 -- start start 2024-09-19T07:54:09.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.647+0000 7f1b8dfe4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b880d0bc0 0x7f1b881f13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:09.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.647+0000 7f1b8dfe4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b8815bee0 0x7f1b8815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:09.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.647+0000 7f1b8dfe4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b8815c2c0 0x7f1b880b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:09.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.647+0000 7f1b8dfe4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8806dec0 con 0x7f1b880d0bc0 2024-09-19T07:54:09.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.647+0000 7f1b8dfe4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1b880ce780 con 0x7f1b8815c2c0 2024-09-19T07:54:09.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.647+0000 7f1b8dfe4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1b8806e040 con 0x7f1b8815bee0 2024-09-19T07:54:09.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.647+0000 7f1b877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b880d0bc0 0x7f1b881f13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:09.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.647+0000 7f1b877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b880d0bc0 0x7f1b881f13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38530/0 (socket says 172.21.15.16:38530) 2024-09-19T07:54:09.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.647+0000 7f1b877fe640 1 -- 172.21.15.16:0/514222832 learned_addr learned my addr 172.21.15.16:0/514222832 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:09.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.647+0000 7f1b86ffd640 1 --2- 172.21.15.16:0/514222832 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b8815bee0 0x7f1b8815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:09.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.647+0000 7f1b87fff640 1 --2- 172.21.15.16:0/514222832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b8815c2c0 0x7f1b880b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.647+0000 7f1b877fe640 1 -- 172.21.15.16:0/514222832 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b8815bee0 msgr2=0x7f1b8815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.647+0000 7f1b877fe640 1 --2- 172.21.15.16:0/514222832 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b8815bee0 0x7f1b8815c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.647+0000 7f1b877fe640 1 -- 172.21.15.16:0/514222832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b8815c2c0 msgr2=0x7f1b880b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.647+0000 7f1b877fe640 1 --2- 172.21.15.16:0/514222832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b8815c2c0 0x7f1b880b0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.647+0000 7f1b877fe640 1 -- 172.21.15.16:0/514222832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b88072150 con 0x7f1b880d0bc0 2024-09-19T07:54:09.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.647+0000 7f1b86ffd640 1 --2- 172.21.15.16:0/514222832 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b8815bee0 0x7f1b8815c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:09.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.647+0000 7f1b87fff640 1 --2- 172.21.15.16:0/514222832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b8815c2c0 0x7f1b880b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:09.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.648+0000 7f1b877fe640 1 --2- 172.21.15.16:0/514222832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b880d0bc0 0x7f1b881f13f0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800c430 tx=0x7f1b7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:09.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.648+0000 7f1b6ffff640 1 -- 172.21.15.16:0/514222832 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78017070 con 0x7f1b880d0bc0 2024-09-19T07:54:09.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.648+0000 7f1b6ffff640 1 -- 172.21.15.16:0/514222832 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b78005ce0 con 0x7f1b880d0bc0 2024-09-19T07:54:09.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.648+0000 7f1b8dfe4640 1 -- 172.21.15.16:0/514222832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b88072430 con 0x7f1b880d0bc0 2024-09-19T07:54:09.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.648+0000 7f1b6ffff640 1 -- 172.21.15.16:0/514222832 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b780072f0 con 0x7f1b880d0bc0 2024-09-19T07:54:09.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.648+0000 7f1b8dfe4640 1 -- 172.21.15.16:0/514222832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b8815d280 con 0x7f1b880d0bc0 2024-09-19T07:54:09.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.649+0000 7f1b8dfe4640 1 -- 172.21.15.16:0/514222832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b880d04b0 con 0x7f1b880d0bc0 2024-09-19T07:54:09.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.649+0000 7f1b6ffff640 1 -- 172.21.15.16:0/514222832 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1b7801a070 con 0x7f1b880d0bc0 2024-09-19T07:54:09.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.650+0000 7f1b6ffff640 1 --2- 172.21.15.16:0/514222832 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1b7406c2e0 0x7f1b7406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:09.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.650+0000 7f1b6ffff640 1 -- 172.21.15.16:0/514222832 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f1b7808aa50 con 0x7f1b880d0bc0 2024-09-19T07:54:09.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.650+0000 7f1b86ffd640 1 --2- 172.21.15.16:0/514222832 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1b7406c2e0 0x7f1b7406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:09.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.651+0000 7f1b86ffd640 1 --2- 172.21.15.16:0/514222832 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1b7406c2e0 0x7f1b7406e7a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1b880aea20 tx=0x7f1b7c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:09.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.654+0000 7f1b6ffff640 1 -- 172.21.15.16:0/514222832 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1b78059030 con 0x7f1b880d0bc0 2024-09-19T07:54:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.829+0000 7f1b8dfe4640 1 -- 172.21.15.16:0/514222832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b8806c550 con 0x7f1b880d0bc0 2024-09-19T07:54:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.830+0000 7f1b6ffff640 1 -- 172.21.15.16:0/514222832 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+17036 (secure 0 0 0) 0x7f1b78058930 con 0x7f1b880d0bc0 2024-09-19T07:54:09.830 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:09.831 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":118,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:53:45.447012+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:53:41.474707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-19T07:53:42.798313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:54:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.832+0000 7f1b8dfe4640 1 -- 172.21.15.16:0/514222832 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1b7406c2e0 msgr2=0x7f1b7406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.832+0000 7f1b8dfe4640 1 --2- 172.21.15.16:0/514222832 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1b7406c2e0 0x7f1b7406e7a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1b880aea20 tx=0x7f1b7c008040 comp rx=0 tx=0).stop 2024-09-19T07:54:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.832+0000 7f1b8dfe4640 1 -- 172.21.15.16:0/514222832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b880d0bc0 msgr2=0x7f1b881f13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.832+0000 7f1b8dfe4640 1 --2- 172.21.15.16:0/514222832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b880d0bc0 0x7f1b881f13f0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800c430 tx=0x7f1b7800c8f0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.832+0000 7f1b8dfe4640 1 -- 172.21.15.16:0/514222832 shutdown_connections 2024-09-19T07:54:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.832+0000 7f1b8dfe4640 1 --2- 172.21.15.16:0/514222832 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1b7406c2e0 0x7f1b7406e7a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.832+0000 7f1b8dfe4640 1 --2- 172.21.15.16:0/514222832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b8815c2c0 0x7f1b880b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.832+0000 7f1b8dfe4640 1 --2- 172.21.15.16:0/514222832 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b8815bee0 0x7f1b8815c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.832+0000 7f1b8dfe4640 1 --2- 172.21.15.16:0/514222832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b880d0bc0 0x7f1b881f13f0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.832+0000 7f1b8dfe4640 1 -- 172.21.15.16:0/514222832 >> 172.21.15.16:0/514222832 conn(0x7f1b8806bdf0 msgr2=0x7f1b8806e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.832+0000 7f1b8dfe4640 1 -- 172.21.15.16:0/514222832 shutdown_connections 2024-09-19T07:54:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.832+0000 7f1b8dfe4640 1 -- 172.21.15.16:0/514222832 wait complete. 2024-09-19T07:54:09.844 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T07:54:09.844 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:54:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.945+0000 7f2abd2ba640 1 Processor -- start 2024-09-19T07:54:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.945+0000 7f2abd2ba640 1 -- start start 2024-09-19T07:54:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.946+0000 7f2abd2ba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab815f0d0 0x7f2ab815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.946+0000 7f2abd2ba640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2ab80d4130 0x7f2ab80d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.946+0000 7f2abd2ba640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ab80d4b20 con 0x7f2ab80d0070 2024-09-19T07:54:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.946+0000 7f2abd2ba640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2ab80d1b10 con 0x7f2ab80d4130 2024-09-19T07:54:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.946+0000 7f2abd2ba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab80cf2b0 con 0x7f2ab815f0d0 2024-09-19T07:54:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.946+0000 7f2ab67fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2ab80d0070 0x7f2ab80d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60612/0 (socket says 172.21.15.16:60612) 2024-09-19T07:54:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.946+0000 7f2ab67fc640 1 -- 172.21.15.16:0/4074903855 learned_addr learned my addr 172.21.15.16:0/4074903855 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.946+0000 7f2ab77fe640 1 --2- 172.21.15.16:0/4074903855 >> v2:172.21.15.146:3300/0 conn(0x7f2ab80d4130 0x7f2ab80d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.947+0000 7f2ab6ffd640 1 --2- 172.21.15.16:0/4074903855 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab815f0d0 0x7f2ab815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.947+0000 7f2ab5ffb640 1 -- 172.21.15.16:0/4074903855 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 204548466 0 0) 0x7f2aa0001110 con 0x7f2ab80d0070 2024-09-19T07:54:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.947+0000 7f2ab5ffb640 1 -- 172.21.15.16:0/4074903855 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2aa4003520 con 0x7f2ab80d0070 2024-09-19T07:54:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.947+0000 7f2ab77fe640 1 -- 172.21.15.16:0/4074903855 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab815f0d0 msgr2=0x7f2ab815f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.947+0000 7f2ab77fe640 1 --2- 172.21.15.16:0/4074903855 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab815f0d0 0x7f2ab815f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.947+0000 7f2ab77fe640 1 -- 172.21.15.16:0/4074903855 >> v1:172.21.15.16:6789/0 conn(0x7f2ab80d0070 legacy=0x7f2ab80d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.947+0000 7f2ab77fe640 1 -- 172.21.15.16:0/4074903855 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab80cf590 con 0x7f2ab80d4130 2024-09-19T07:54:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.948+0000 7f2ab77fe640 1 --2- 172.21.15.16:0/4074903855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab80d4130 0x7f2ab80d45e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80d2fc0 tx=0x7f2aa802a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32119893fa2a8d70 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.948+0000 7f2ab5ffb640 1 -- 172.21.15.16:0/4074903855 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8007740 con 0x7f2ab80d4130 2024-09-19T07:54:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.948+0000 7f2ab5ffb640 1 -- 172.21.15.16:0/4074903855 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa80078e0 con 0x7f2ab80d4130 2024-09-19T07:54:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.948+0000 7f2ab5ffb640 1 -- 172.21.15.16:0/4074903855 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8007c90 con 0x7f2ab80d4130 2024-09-19T07:54:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.948+0000 7f2abd2ba640 1 -- 172.21.15.16:0/4074903855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab80d4130 msgr2=0x7f2ab80d45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.948+0000 7f2abd2ba640 1 --2- 172.21.15.16:0/4074903855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab80d4130 0x7f2ab80d45e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80d2fc0 tx=0x7f2aa802a5b0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.948+0000 7f2abd2ba640 1 -- 172.21.15.16:0/4074903855 shutdown_connections 2024-09-19T07:54:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.948+0000 7f2abd2ba640 1 --2- 172.21.15.16:0/4074903855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab80d4130 0x7f2ab80d45e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.948+0000 7f2abd2ba640 1 --2- 172.21.15.16:0/4074903855 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab815f0d0 0x7f2ab815f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.948+0000 7f2abd2ba640 1 -- 172.21.15.16:0/4074903855 >> 172.21.15.16:0/4074903855 conn(0x7f2ab809d760 msgr2=0x7f2ab808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.948+0000 7f2abd2ba640 1 -- 172.21.15.16:0/4074903855 shutdown_connections 2024-09-19T07:54:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.948+0000 7f2abd2ba640 1 -- 172.21.15.16:0/4074903855 wait complete. 2024-09-19T07:54:09.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.950+0000 7f2abd2ba640 1 Processor -- start 2024-09-19T07:54:09.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.950+0000 7f2abd2ba640 1 -- start start 2024-09-19T07:54:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.950+0000 7f2abd2ba640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab80d0070 0x7f2ab815b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.950+0000 7f2abd2ba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab80d4130 0x7f2ab80cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.950+0000 7f2abd2ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab815f0d0 0x7f2ab80cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.951+0000 7f2abd2ba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab80cef60 con 0x7f2ab815f0d0 2024-09-19T07:54:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.951+0000 7f2abd2ba640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab80cfb50 con 0x7f2ab80d0070 2024-09-19T07:54:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.951+0000 7f2abd2ba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab80b2f70 con 0x7f2ab80d4130 2024-09-19T07:54:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.951+0000 7f2ab77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab815f0d0 0x7f2ab80cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.951+0000 7f2ab77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab815f0d0 0x7f2ab80cc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38546/0 (socket says 172.21.15.16:38546) 2024-09-19T07:54:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.951+0000 7f2ab77fe640 1 -- 172.21.15.16:0/1759171195 learned_addr learned my addr 172.21.15.16:0/1759171195 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.951+0000 7f2ab67fc640 1 --2- 172.21.15.16:0/1759171195 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab80d4130 0x7f2ab80cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.951+0000 7f2ab6ffd640 1 --2- 172.21.15.16:0/1759171195 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab80d0070 0x7f2ab815b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.951+0000 7f2ab77fe640 1 -- 172.21.15.16:0/1759171195 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab80d4130 msgr2=0x7f2ab80cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.951+0000 7f2ab77fe640 1 --2- 172.21.15.16:0/1759171195 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab80d4130 0x7f2ab80cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.951+0000 7f2ab77fe640 1 -- 172.21.15.16:0/1759171195 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab80d0070 msgr2=0x7f2ab815b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.951+0000 7f2ab77fe640 1 --2- 172.21.15.16:0/1759171195 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab80d0070 0x7f2ab815b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.951+0000 7f2ab77fe640 1 -- 172.21.15.16:0/1759171195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab815b930 con 0x7f2ab815f0d0 2024-09-19T07:54:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.951+0000 7f2ab77fe640 1 --2- 172.21.15.16:0/1759171195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab815f0d0 0x7f2ab80cc470 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f2aa802aba0 tx=0x7f2aa8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.951+0000 7f2aaf7fe640 1 -- 172.21.15.16:0/1759171195 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8009450 con 0x7f2ab815f0d0 2024-09-19T07:54:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.951+0000 7f2aaf7fe640 1 -- 172.21.15.16:0/1759171195 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa8004240 con 0x7f2ab815f0d0 2024-09-19T07:54:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.951+0000 7f2aaf7fe640 1 -- 172.21.15.16:0/1759171195 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8004630 con 0x7f2ab815f0d0 2024-09-19T07:54:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.951+0000 7f2abd2ba640 1 -- 172.21.15.16:0/1759171195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab80ccd70 con 0x7f2ab815f0d0 2024-09-19T07:54:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.951+0000 7f2abd2ba640 1 -- 172.21.15.16:0/1759171195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab80cd330 con 0x7f2ab815f0d0 2024-09-19T07:54:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.952+0000 7f2abd2ba640 1 -- 172.21.15.16:0/1759171195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab80d45e0 con 0x7f2ab815f0d0 2024-09-19T07:54:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.953+0000 7f2aaf7fe640 1 -- 172.21.15.16:0/1759171195 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2aa8039040 con 0x7f2ab815f0d0 2024-09-19T07:54:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.954+0000 7f2aaf7fe640 1 --2- 172.21.15.16:0/1759171195 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2a9806c2b0 0x7f2a9806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.954+0000 7f2aaf7fe640 1 -- 172.21.15.16:0/1759171195 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f2aa80ac590 con 0x7f2ab815f0d0 2024-09-19T07:54:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.954+0000 7f2ab6ffd640 1 --2- 172.21.15.16:0/1759171195 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2a9806c2b0 0x7f2a9806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.954+0000 7f2ab6ffd640 1 --2- 172.21.15.16:0/1759171195 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2a9806c2b0 0x7f2a9806e770 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80b9f00 tx=0x7f2ab0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:09.958+0000 7f2aaf7fe640 1 -- 172.21.15.16:0/1759171195 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2aa8040030 con 0x7f2ab815f0d0 2024-09-19T07:54:10.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.171+0000 7f2abd2ba640 1 -- 172.21.15.16:0/1759171195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ab806d6f0 con 0x7f2ab815f0d0 2024-09-19T07:54:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.172+0000 7f2aaf7fe640 1 -- 172.21.15.16:0/1759171195 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5231 (secure 0 0 0) 0x7f2aa807a500 con 0x7f2ab815f0d0 2024-09-19T07:54:10.172 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:10.172 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":65,"btime":"2024-09-19T07:54:05:204476+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2471408352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2471408352},{"type":"v1","addr":"172.21.15.16:6833","nonce":2471408352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/1281369866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1281369866},{"type":"v1","addr":"172.21.15.146:6836","nonce":1281369866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1377303883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1377303883},{"type":"v1","addr":"172.21.15.146:6837","nonce":1377303883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:53:43.812066+0000","modified":"2024-09-19T07:54:04.488075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7368":{"gid":7368,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/2050126415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2050126415},{"type":"v1","addr":"172.21.15.16:6835","nonce":2050126415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7368,"qdb_cluster":[7368]},"id":8}]} 2024-09-19T07:54:10.173 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 65 2024-09-19T07:54:10.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.174+0000 7f2abd2ba640 1 -- 172.21.15.16:0/1759171195 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2a9806c2b0 msgr2=0x7f2a9806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:10.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.174+0000 7f2abd2ba640 1 --2- 172.21.15.16:0/1759171195 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2a9806c2b0 0x7f2a9806e770 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80b9f00 tx=0x7f2ab0009190 comp rx=0 tx=0).stop 2024-09-19T07:54:10.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.174+0000 7f2abd2ba640 1 -- 172.21.15.16:0/1759171195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab815f0d0 msgr2=0x7f2ab80cc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:10.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.174+0000 7f2abd2ba640 1 --2- 172.21.15.16:0/1759171195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab815f0d0 0x7f2ab80cc470 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f2aa802aba0 tx=0x7f2aa8004a40 comp rx=0 tx=0).stop 2024-09-19T07:54:10.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.174+0000 7f2abd2ba640 1 -- 172.21.15.16:0/1759171195 shutdown_connections 2024-09-19T07:54:10.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.174+0000 7f2abd2ba640 1 --2- 172.21.15.16:0/1759171195 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2a9806c2b0 0x7f2a9806e770 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.174+0000 7f2abd2ba640 1 --2- 172.21.15.16:0/1759171195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab815f0d0 0x7f2ab80cc470 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.174+0000 7f2abd2ba640 1 --2- 172.21.15.16:0/1759171195 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab80d4130 0x7f2ab80cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.174+0000 7f2abd2ba640 1 --2- 172.21.15.16:0/1759171195 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab80d0070 0x7f2ab815b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.174+0000 7f2abd2ba640 1 -- 172.21.15.16:0/1759171195 >> 172.21.15.16:0/1759171195 conn(0x7f2ab809d760 msgr2=0x7f2ab806daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:10.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.174+0000 7f2abd2ba640 1 -- 172.21.15.16:0/1759171195 shutdown_connections 2024-09-19T07:54:10.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.174+0000 7f2abd2ba640 1 -- 172.21.15.16:0/1759171195 wait complete. 2024-09-19T07:54:10.186 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:54:10.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.287+0000 7f38a7d0a640 1 Processor -- start 2024-09-19T07:54:10.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.288+0000 7f38a7d0a640 1 -- start start 2024-09-19T07:54:10.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.288+0000 7f38a7d0a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f38a015f0d0 0x7f38a015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:10.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.288+0000 7f38a7d0a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38a00d0070 0x7f38a00d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:10.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.289+0000 7f38a7d0a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38a00cf2b0 con 0x7f38a00d1940 2024-09-19T07:54:10.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.289+0000 7f38a7d0a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f38a00cf590 con 0x7f38a015f0d0 2024-09-19T07:54:10.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.289+0000 7f38a7d0a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f38a00cf870 con 0x7f38a00d0070 2024-09-19T07:54:10.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.289+0000 7f38a5aa6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f38a015f0d0 0x7f38a015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:10.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.289+0000 7f38a52a5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38a00d0070 0x7f38a00d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:10.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.289+0000 7f38a5aa6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f38a015f0d0 0x7f38a015f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:56942/0 (socket says 172.21.15.16:56942) 2024-09-19T07:54:10.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.289+0000 7f38a5aa6640 1 -- 172.21.15.16:0/3191573312 learned_addr learned my addr 172.21.15.16:0/3191573312 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:10.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.289+0000 7f38a52a5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38a00d0070 0x7f38a00d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:33332/0 (socket says 172.21.15.16:33332) 2024-09-19T07:54:10.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.289+0000 7f38a5aa6640 1 -- 172.21.15.16:0/3191573312 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38a00d0070 msgr2=0x7f38a00d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:10.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.289+0000 7f38a5aa6640 1 --2- 172.21.15.16:0/3191573312 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38a00d0070 0x7f38a00d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.289+0000 7f38a5aa6640 1 -- 172.21.15.16:0/3191573312 >> v1:172.21.15.146:6789/0 conn(0x7f38a00d1940 legacy=0x7f38a00d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:10.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.289+0000 7f38a5aa6640 1 -- 172.21.15.16:0/3191573312 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a00cfb50 con 0x7f38a015f0d0 2024-09-19T07:54:10.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.290+0000 7f38a5aa6640 1 --2- 172.21.15.16:0/3191573312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38a015f0d0 0x7f38a015f4b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f389c009e50 tx=0x7f389c02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c83559122c03530e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:10.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.290+0000 7f38a4aa4640 1 -- 172.21.15.16:0/3191573312 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c036070 con 0x7f38a015f0d0 2024-09-19T07:54:10.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.290+0000 7f38a4aa4640 1 -- 172.21.15.16:0/3191573312 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f389c031600 con 0x7f38a015f0d0 2024-09-19T07:54:10.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.290+0000 7f38a4aa4640 1 -- 172.21.15.16:0/3191573312 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c0319d0 con 0x7f38a015f0d0 2024-09-19T07:54:10.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.291+0000 7f38a7d0a640 1 -- 172.21.15.16:0/3191573312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38a015f0d0 msgr2=0x7f38a015f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:10.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.291+0000 7f38a7d0a640 1 --2- 172.21.15.16:0/3191573312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38a015f0d0 0x7f38a015f4b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f389c009e50 tx=0x7f389c02a680 comp rx=0 tx=0).stop 2024-09-19T07:54:10.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.291+0000 7f38a7d0a640 1 -- 172.21.15.16:0/3191573312 shutdown_connections 2024-09-19T07:54:10.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.291+0000 7f38a7d0a640 1 --2- 172.21.15.16:0/3191573312 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38a00d0070 0x7f38a00d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.291+0000 7f38a7d0a640 1 --2- 172.21.15.16:0/3191573312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38a015f0d0 0x7f38a015f4b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.291+0000 7f38a7d0a640 1 -- 172.21.15.16:0/3191573312 >> 172.21.15.16:0/3191573312 conn(0x7f38a009d760 msgr2=0x7f38a008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:10.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.291+0000 7f38a7d0a640 1 -- 172.21.15.16:0/3191573312 shutdown_connections 2024-09-19T07:54:10.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.291+0000 7f38a7d0a640 1 -- 172.21.15.16:0/3191573312 wait complete. 2024-09-19T07:54:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.292+0000 7f38a7d0a640 1 Processor -- start 2024-09-19T07:54:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.292+0000 7f38a7d0a640 1 -- start start 2024-09-19T07:54:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.293+0000 7f38a7d0a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38a00d0070 0x7f38a01f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.293+0000 7f38a7d0a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38a00d1940 0x7f38a0159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.293+0000 7f38a7d0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a015f0d0 0x7f38a006de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.293+0000 7f38a7d0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38a00d49c0 con 0x7f38a015f0d0 2024-09-19T07:54:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.293+0000 7f38a7d0a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f38a00d4840 con 0x7f38a00d0070 2024-09-19T07:54:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.293+0000 7f38a7d0a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f38a00d4b40 con 0x7f38a00d1940 2024-09-19T07:54:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.293+0000 7f38a62a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a015f0d0 0x7f38a006de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.293+0000 7f38a62a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a015f0d0 0x7f38a006de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38552/0 (socket says 172.21.15.16:38552) 2024-09-19T07:54:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.293+0000 7f38a62a7640 1 -- 172.21.15.16:0/2232181224 learned_addr learned my addr 172.21.15.16:0/2232181224 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.293+0000 7f38a5aa6640 1 --2- 172.21.15.16:0/2232181224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38a00d0070 0x7f38a01f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.293+0000 7f38a52a5640 1 --2- 172.21.15.16:0/2232181224 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38a00d1940 0x7f38a0159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.293+0000 7f38a62a7640 1 -- 172.21.15.16:0/2232181224 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38a00d1940 msgr2=0x7f38a0159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:10.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.293+0000 7f38a62a7640 1 --2- 172.21.15.16:0/2232181224 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38a00d1940 0x7f38a0159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.293+0000 7f38a62a7640 1 -- 172.21.15.16:0/2232181224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38a00d0070 msgr2=0x7f38a01f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:10.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.293+0000 7f38a62a7640 1 --2- 172.21.15.16:0/2232181224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38a00d0070 0x7f38a01f7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.293+0000 7f38a62a7640 1 -- 172.21.15.16:0/2232181224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a01f7be0 con 0x7f38a015f0d0 2024-09-19T07:54:10.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.294+0000 7f38a62a7640 1 --2- 172.21.15.16:0/2232181224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a015f0d0 0x7f38a006de70 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f389800c3f0 tx=0x7f389800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:10.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.294+0000 7f38967fc640 1 -- 172.21.15.16:0/2232181224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898017070 con 0x7f38a015f0d0 2024-09-19T07:54:10.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.294+0000 7f38967fc640 1 -- 172.21.15.16:0/2232181224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38980052d0 con 0x7f38a015f0d0 2024-09-19T07:54:10.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.294+0000 7f38a7d0a640 1 -- 172.21.15.16:0/2232181224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a01f7ec0 con 0x7f38a015f0d0 2024-09-19T07:54:10.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.294+0000 7f38967fc640 1 -- 172.21.15.16:0/2232181224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898003850 con 0x7f38a015f0d0 2024-09-19T07:54:10.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.294+0000 7f38a7d0a640 1 -- 172.21.15.16:0/2232181224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a01f8480 con 0x7f38a015f0d0 2024-09-19T07:54:10.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.295+0000 7f38a7d0a640 1 -- 172.21.15.16:0/2232181224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a00d1400 con 0x7f38a015f0d0 2024-09-19T07:54:10.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.295+0000 7f38967fc640 1 -- 172.21.15.16:0/2232181224 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f389801a070 con 0x7f38a015f0d0 2024-09-19T07:54:10.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.296+0000 7f38967fc640 1 --2- 172.21.15.16:0/2232181224 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f388406c2b0 0x7f388406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:10.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.296+0000 7f38967fc640 1 -- 172.21.15.16:0/2232181224 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f389808a840 con 0x7f38a015f0d0 2024-09-19T07:54:10.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.296+0000 7f38a5aa6640 1 --2- 172.21.15.16:0/2232181224 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f388406c2b0 0x7f388406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:10.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.297+0000 7f38a5aa6640 1 --2- 172.21.15.16:0/2232181224 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f388406c2b0 0x7f388406e770 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f389c033000 tx=0x7f389c009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:10.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.300+0000 7f38967fc640 1 -- 172.21.15.16:0/2232181224 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f38980587d0 con 0x7f38a015f0d0 2024-09-19T07:54:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.516+0000 7f38a7d0a640 1 -- 172.21.15.16:0/2232181224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38a0074f90 con 0x7f38a015f0d0 2024-09-19T07:54:10.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.517+0000 7f38967fc640 1 -- 172.21.15.16:0/2232181224 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5231 (secure 0 0 0) 0x7f389805cad0 con 0x7f38a015f0d0 2024-09-19T07:54:10.517 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:10.517 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":65,"btime":"2024-09-19T07:54:05:204476+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2471408352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2471408352},{"type":"v1","addr":"172.21.15.16:6833","nonce":2471408352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/1281369866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1281369866},{"type":"v1","addr":"172.21.15.146:6836","nonce":1281369866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1377303883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1377303883},{"type":"v1","addr":"172.21.15.146:6837","nonce":1377303883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:53:43.812066+0000","modified":"2024-09-19T07:54:04.488075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7368},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7368":{"gid":7368,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/2050126415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2050126415},{"type":"v1","addr":"172.21.15.16:6835","nonce":2050126415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7368,"qdb_cluster":[7368]},"id":8}]} 2024-09-19T07:54:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 65 2024-09-19T07:54:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.519+0000 7f38a7d0a640 1 -- 172.21.15.16:0/2232181224 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f388406c2b0 msgr2=0x7f388406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.519+0000 7f38a7d0a640 1 --2- 172.21.15.16:0/2232181224 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f388406c2b0 0x7f388406e770 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f389c033000 tx=0x7f389c009000 comp rx=0 tx=0).stop 2024-09-19T07:54:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.519+0000 7f38a7d0a640 1 -- 172.21.15.16:0/2232181224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a015f0d0 msgr2=0x7f38a006de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.519+0000 7f38a7d0a640 1 --2- 172.21.15.16:0/2232181224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a015f0d0 0x7f38a006de70 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f389800c3f0 tx=0x7f389800c8b0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.519+0000 7f38a7d0a640 1 -- 172.21.15.16:0/2232181224 shutdown_connections 2024-09-19T07:54:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.519+0000 7f38a7d0a640 1 --2- 172.21.15.16:0/2232181224 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f388406c2b0 0x7f388406e770 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.519+0000 7f38a7d0a640 1 --2- 172.21.15.16:0/2232181224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a015f0d0 0x7f38a006de70 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.519+0000 7f38a7d0a640 1 --2- 172.21.15.16:0/2232181224 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38a00d1940 0x7f38a0159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.519+0000 7f38a7d0a640 1 --2- 172.21.15.16:0/2232181224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38a00d0070 0x7f38a01f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.519+0000 7f38a7d0a640 1 -- 172.21.15.16:0/2232181224 >> 172.21.15.16:0/2232181224 conn(0x7f38a009d760 msgr2=0x7f38a0161e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.519+0000 7f38a7d0a640 1 -- 172.21.15.16:0/2232181224 shutdown_connections 2024-09-19T07:54:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.519+0000 7f38a7d0a640 1 -- 172.21.15.16:0/2232181224 wait complete. 2024-09-19T07:54:10.530 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:54:10.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.632+0000 7f1083fff640 1 Processor -- start 2024-09-19T07:54:10.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.632+0000 7f1083fff640 1 -- start start 2024-09-19T07:54:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.633+0000 7f1083fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f10840d1010 0x7f10840cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.633+0000 7f1083fff640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f10840b4f80 0x7f10840cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.633+0000 7f1083fff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10840d2110 con 0x7f1084162a00 2024-09-19T07:54:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.633+0000 7f1083fff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f10840d23f0 con 0x7f10840d1010 2024-09-19T07:54:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.633+0000 7f1083fff640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f10840d26d0 con 0x7f10840b4f80 2024-09-19T07:54:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.633+0000 7f107bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f10840d1010 0x7f10840cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.633+0000 7f1082ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1084162a00 0x7f10840d0900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60622/0 (socket says 172.21.15.16:60622) 2024-09-19T07:54:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.633+0000 7f1082ffd640 1 -- 172.21.15.16:0/1813063163 learned_addr learned my addr 172.21.15.16:0/1813063163 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.633+0000 7f10837fe640 1 --2- 172.21.15.16:0/1813063163 >> v2:172.21.15.146:3300/0 conn(0x7f10840b4f80 0x7f10840cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.633+0000 7f107bfff640 1 -- 172.21.15.16:0/1813063163 >> v2:172.21.15.146:3300/0 conn(0x7f10840b4f80 msgr2=0x7f10840cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.633+0000 7f107bfff640 1 --2- 172.21.15.16:0/1813063163 >> v2:172.21.15.146:3300/0 conn(0x7f10840b4f80 0x7f10840cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.633+0000 7f107bfff640 1 -- 172.21.15.16:0/1813063163 >> v1:172.21.15.16:6789/0 conn(0x7f1084162a00 legacy=0x7f10840d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.633+0000 7f107bfff640 1 -- 172.21.15.16:0/1813063163 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10840d29b0 con 0x7f10840d1010 2024-09-19T07:54:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.633+0000 7f10827fc640 1 -- 172.21.15.16:0/1813063163 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3145488592 0 0) 0x7f10840d2110 con 0x7f1084162a00 2024-09-19T07:54:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.634+0000 7f107bfff640 1 --2- 172.21.15.16:0/1813063163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10840d1010 0x7f10840cee00 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f108408d650 tx=0x7f106c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d670721e86ea04c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.634+0000 7f10827fc640 1 -- 172.21.15.16:0/1813063163 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c037070 con 0x7f10840d1010 2024-09-19T07:54:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.634+0000 7f10827fc640 1 -- 172.21.15.16:0/1813063163 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f106c032440 con 0x7f10840d1010 2024-09-19T07:54:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.634+0000 7f10827fc640 1 -- 172.21.15.16:0/1813063163 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c02ae60 con 0x7f10840d1010 2024-09-19T07:54:10.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.635+0000 7f1083fff640 1 -- 172.21.15.16:0/1813063163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10840d1010 msgr2=0x7f10840cee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:10.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.635+0000 7f1083fff640 1 --2- 172.21.15.16:0/1813063163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10840d1010 0x7f10840cee00 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f108408d650 tx=0x7f106c02a110 comp rx=0 tx=0).stop 2024-09-19T07:54:10.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.635+0000 7f1083fff640 1 -- 172.21.15.16:0/1813063163 shutdown_connections 2024-09-19T07:54:10.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.635+0000 7f1083fff640 1 --2- 172.21.15.16:0/1813063163 >> v2:172.21.15.146:3300/0 conn(0x7f10840b4f80 0x7f10840cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.635+0000 7f1083fff640 1 --2- 172.21.15.16:0/1813063163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10840d1010 0x7f10840cee00 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.635+0000 7f1083fff640 1 -- 172.21.15.16:0/1813063163 >> 172.21.15.16:0/1813063163 conn(0x7f108409d520 msgr2=0x7f10840d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:10.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.635+0000 7f1083fff640 1 -- 172.21.15.16:0/1813063163 shutdown_connections 2024-09-19T07:54:10.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.635+0000 7f1083fff640 1 -- 172.21.15.16:0/1813063163 wait complete. 2024-09-19T07:54:10.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.636+0000 7f1083fff640 1 Processor -- start 2024-09-19T07:54:10.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.636+0000 7f1083fff640 1 -- start start 2024-09-19T07:54:10.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.637+0000 7f1083fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10840b4f80 0x7f10841f4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:10.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.637+0000 7f1083fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10840d1010 0x7f10840cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:10.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.637+0000 7f1083fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1084162a00 0x7f10840cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:10.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.637+0000 7f1083fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f10840d31b0 con 0x7f1084162a00 2024-09-19T07:54:10.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.637+0000 7f1083fff640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f10840b8030 con 0x7f10840b4f80 2024-09-19T07:54:10.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.637+0000 7f1083fff640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f10840d3330 con 0x7f10840d1010 2024-09-19T07:54:10.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.637+0000 7f10837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1084162a00 0x7f10840cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:10.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.637+0000 7f10837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1084162a00 0x7f10840cc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38556/0 (socket says 172.21.15.16:38556) 2024-09-19T07:54:10.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.637+0000 7f10837fe640 1 -- 172.21.15.16:0/2743640343 learned_addr learned my addr 172.21.15.16:0/2743640343 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:10.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.637+0000 7f1082ffd640 1 --2- 172.21.15.16:0/2743640343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10840b4f80 0x7f10841f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:10.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.637+0000 7f107bfff640 1 --2- 172.21.15.16:0/2743640343 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10840d1010 0x7f10840cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:10.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.637+0000 7f10837fe640 1 -- 172.21.15.16:0/2743640343 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10840d1010 msgr2=0x7f10840cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:10.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.637+0000 7f10837fe640 1 --2- 172.21.15.16:0/2743640343 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10840d1010 0x7f10840cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.637+0000 7f10837fe640 1 -- 172.21.15.16:0/2743640343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10840b4f80 msgr2=0x7f10841f4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:10.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.637+0000 7f10837fe640 1 --2- 172.21.15.16:0/2743640343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10840b4f80 0x7f10841f4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.637+0000 7f10837fe640 1 -- 172.21.15.16:0/2743640343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10840cac00 con 0x7f1084162a00 2024-09-19T07:54:10.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.637+0000 7f1082ffd640 1 --2- 172.21.15.16:0/2743640343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10840b4f80 0x7f10841f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:10.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.637+0000 7f10837fe640 1 --2- 172.21.15.16:0/2743640343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1084162a00 0x7f10840cc220 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f107c007e60 tx=0x7f107c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:10.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.638+0000 7f107b7fe640 1 -- 172.21.15.16:0/2743640343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c014420 con 0x7f1084162a00 2024-09-19T07:54:10.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.638+0000 7f107b7fe640 1 -- 172.21.15.16:0/2743640343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f107c019070 con 0x7f1084162a00 2024-09-19T07:54:10.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.638+0000 7f1083fff640 1 -- 172.21.15.16:0/2743640343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10840cb460 con 0x7f1084162a00 2024-09-19T07:54:10.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.638+0000 7f107b7fe640 1 -- 172.21.15.16:0/2743640343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c014820 con 0x7f1084162a00 2024-09-19T07:54:10.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.638+0000 7f1083fff640 1 -- 172.21.15.16:0/2743640343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10840cba20 con 0x7f1084162a00 2024-09-19T07:54:10.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.639+0000 7f1083fff640 1 -- 172.21.15.16:0/2743640343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1084162de0 con 0x7f1084162a00 2024-09-19T07:54:10.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.639+0000 7f107b7fe640 1 -- 172.21.15.16:0/2743640343 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f107c0149c0 con 0x7f1084162a00 2024-09-19T07:54:10.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.640+0000 7f107b7fe640 1 --2- 172.21.15.16:0/2743640343 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f106406c2b0 0x7f106406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:10.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.640+0000 7f107b7fe640 1 -- 172.21.15.16:0/2743640343 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f107c08be00 con 0x7f1084162a00 2024-09-19T07:54:10.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.640+0000 7f1082ffd640 1 --2- 172.21.15.16:0/2743640343 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f106406c2b0 0x7f106406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:10.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.641+0000 7f1082ffd640 1 --2- 172.21.15.16:0/2743640343 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f106406c2b0 0x7f106406e770 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1074000f60 tx=0x7f1074007c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:10.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.644+0000 7f107b7fe640 1 -- 172.21.15.16:0/2743640343 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f107c05a3e0 con 0x7f1084162a00 2024-09-19T07:54:10.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.819+0000 7f1083fff640 1 -- 172.21.15.16:0/2743640343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10840d1870 con 0x7f1084162a00 2024-09-19T07:54:10.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.820+0000 7f107b7fe640 1 -- 172.21.15.16:0/2743640343 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+17036 (secure 0 0 0) 0x7f107c059ce0 con 0x7f1084162a00 2024-09-19T07:54:10.820 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:10.821 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":118,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:53:45.447012+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:53:41.474707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-19T07:53:42.798313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:54:10.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.822+0000 7f1083fff640 1 -- 172.21.15.16:0/2743640343 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f106406c2b0 msgr2=0x7f106406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.822+0000 7f1083fff640 1 --2- 172.21.15.16:0/2743640343 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f106406c2b0 0x7f106406e770 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1074000f60 tx=0x7f1074007c10 comp rx=0 tx=0).stop 2024-09-19T07:54:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.822+0000 7f1083fff640 1 -- 172.21.15.16:0/2743640343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1084162a00 msgr2=0x7f10840cc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.822+0000 7f1083fff640 1 --2- 172.21.15.16:0/2743640343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1084162a00 0x7f10840cc220 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f107c007e60 tx=0x7f107c00e4b0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.822+0000 7f1083fff640 1 -- 172.21.15.16:0/2743640343 shutdown_connections 2024-09-19T07:54:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.822+0000 7f1083fff640 1 --2- 172.21.15.16:0/2743640343 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f106406c2b0 0x7f106406e770 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.822+0000 7f1083fff640 1 --2- 172.21.15.16:0/2743640343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1084162a00 0x7f10840cc220 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.822+0000 7f1083fff640 1 --2- 172.21.15.16:0/2743640343 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10840d1010 0x7f10840cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.822+0000 7f1083fff640 1 --2- 172.21.15.16:0/2743640343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10840b4f80 0x7f10841f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.822+0000 7f1083fff640 1 -- 172.21.15.16:0/2743640343 >> 172.21.15.16:0/2743640343 conn(0x7f108409d520 msgr2=0x7f108406e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.822+0000 7f1083fff640 1 -- 172.21.15.16:0/2743640343 shutdown_connections 2024-09-19T07:54:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.822+0000 7f1083fff640 1 -- 172.21.15.16:0/2743640343 wait complete. 2024-09-19T07:54:10.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T07:54:10.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.937+0000 7f35b3a5e640 1 Processor -- start 2024-09-19T07:54:10.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.937+0000 7f35b3a5e640 1 -- start start 2024-09-19T07:54:10.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.938+0000 7f35b3a5e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f35ac15e680 0x7f35ac0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:10.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.938+0000 7f35b3a5e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35ac0d0010 0x7f35ac0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:10.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.938+0000 7f35b3a5e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35ac0d1880 con 0x7f35ac15e2a0 2024-09-19T07:54:10.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.938+0000 7f35b3a5e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f35ac0d1b60 con 0x7f35ac0d0010 2024-09-19T07:54:10.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.938+0000 7f35b3a5e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f35ac0cf2d0 con 0x7f35ac15e680 2024-09-19T07:54:10.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.938+0000 7f35b1ffb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35ac0d0010 0x7f35ac0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:10.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.938+0000 7f35b1ffb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35ac0d0010 0x7f35ac0d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38570/0 (socket says 172.21.15.16:38570) 2024-09-19T07:54:10.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.938+0000 7f35b1ffb640 1 -- 172.21.15.16:0/2818712280 learned_addr learned my addr 172.21.15.16:0/2818712280 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:10.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.938+0000 7f35b0ff9640 1 --2- 172.21.15.16:0/2818712280 >> v2:172.21.15.146:3300/0 conn(0x7f35ac15e680 0x7f35ac0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:10.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.938+0000 7f35b1ffb640 1 -- 172.21.15.16:0/2818712280 >> v2:172.21.15.146:3300/0 conn(0x7f35ac15e680 msgr2=0x7f35ac0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:10.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.938+0000 7f35b1ffb640 1 --2- 172.21.15.16:0/2818712280 >> v2:172.21.15.146:3300/0 conn(0x7f35ac15e680 0x7f35ac0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.938+0000 7f35b1ffb640 1 -- 172.21.15.16:0/2818712280 >> v1:172.21.15.146:6789/0 conn(0x7f35ac15e2a0 legacy=0x7f35ac06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:10.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.938+0000 7f35b1ffb640 1 -- 172.21.15.16:0/2818712280 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac0cf5b0 con 0x7f35ac0d0010 2024-09-19T07:54:10.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.939+0000 7f35b1ffb640 1 --2- 172.21.15.16:0/2818712280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35ac0d0010 0x7f35ac0d4370 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0a22a0 tx=0x7f35a402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8943c7d160c70a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:10.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.939+0000 7f35a3fff640 1 -- 172.21.15.16:0/2818712280 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4036070 con 0x7f35ac0d0010 2024-09-19T07:54:10.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.939+0000 7f35a3fff640 1 -- 172.21.15.16:0/2818712280 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35a4031600 con 0x7f35ac0d0010 2024-09-19T07:54:10.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.939+0000 7f35a3fff640 1 -- 172.21.15.16:0/2818712280 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a40319d0 con 0x7f35ac0d0010 2024-09-19T07:54:10.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.940+0000 7f35b3a5e640 1 -- 172.21.15.16:0/2818712280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35ac0d0010 msgr2=0x7f35ac0d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:10.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.940+0000 7f35b3a5e640 1 --2- 172.21.15.16:0/2818712280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35ac0d0010 0x7f35ac0d4370 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0a22a0 tx=0x7f35a402a6a0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.940+0000 7f35b3a5e640 1 -- 172.21.15.16:0/2818712280 shutdown_connections 2024-09-19T07:54:10.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.940+0000 7f35b3a5e640 1 --2- 172.21.15.16:0/2818712280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35ac0d0010 0x7f35ac0d4370 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.940+0000 7f35b3a5e640 1 --2- 172.21.15.16:0/2818712280 >> v2:172.21.15.146:3300/0 conn(0x7f35ac15e680 0x7f35ac0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.940+0000 7f35b3a5e640 1 -- 172.21.15.16:0/2818712280 >> 172.21.15.16:0/2818712280 conn(0x7f35ac09d760 msgr2=0x7f35ac08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:10.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.940+0000 7f35b3a5e640 1 -- 172.21.15.16:0/2818712280 shutdown_connections 2024-09-19T07:54:10.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.940+0000 7f35b3a5e640 1 -- 172.21.15.16:0/2818712280 wait complete. 2024-09-19T07:54:10.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.941+0000 7f35b3a5e640 1 Processor -- start 2024-09-19T07:54:10.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.941+0000 7f35b3a5e640 1 -- start start 2024-09-19T07:54:10.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.942+0000 7f35b3a5e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35ac0d0010 0x7f35ac1e6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:10.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.942+0000 7f35b3a5e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35ac15e2a0 0x7f35ac0cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:10.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.942+0000 7f35b3a5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35ac15e680 0x7f35ac1f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:10.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.942+0000 7f35b3a5e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac06dcb0 con 0x7f35ac15e680 2024-09-19T07:54:10.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.942+0000 7f35b3a5e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac06db30 con 0x7f35ac15e2a0 2024-09-19T07:54:10.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.942+0000 7f35b3a5e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f35ac0d48b0 con 0x7f35ac0d0010 2024-09-19T07:54:10.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.942+0000 7f35b1ffb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35ac15e680 0x7f35ac1f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:10.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.942+0000 7f35b1ffb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35ac15e680 0x7f35ac1f88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38572/0 (socket says 172.21.15.16:38572) 2024-09-19T07:54:10.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.942+0000 7f35b1ffb640 1 -- 172.21.15.16:0/2513451121 learned_addr learned my addr 172.21.15.16:0/2513451121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:10.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.942+0000 7f35b0ff9640 1 --2- 172.21.15.16:0/2513451121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35ac15e2a0 0x7f35ac0cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:10.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.942+0000 7f35b1ffb640 1 -- 172.21.15.16:0/2513451121 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35ac0d0010 msgr2=0x7f35ac1e6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:10.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.942+0000 7f35b17fa640 1 --2- 172.21.15.16:0/2513451121 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35ac0d0010 0x7f35ac1e6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:10.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.942+0000 7f35b1ffb640 1 --2- 172.21.15.16:0/2513451121 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35ac0d0010 0x7f35ac1e6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.942+0000 7f35b1ffb640 1 -- 172.21.15.16:0/2513451121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35ac15e2a0 msgr2=0x7f35ac0cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:10.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.942+0000 7f35b1ffb640 1 --2- 172.21.15.16:0/2513451121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35ac15e2a0 0x7f35ac0cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:10.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.942+0000 7f35b1ffb640 1 -- 172.21.15.16:0/2513451121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac1f5310 con 0x7f35ac15e680 2024-09-19T07:54:10.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.942+0000 7f35b0ff9640 1 --2- 172.21.15.16:0/2513451121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35ac15e2a0 0x7f35ac0cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:10.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.942+0000 7f35b17fa640 1 --2- 172.21.15.16:0/2513451121 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35ac0d0010 0x7f35ac1e6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:10.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.942+0000 7f35b1ffb640 1 --2- 172.21.15.16:0/2513451121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35ac15e680 0x7f35ac1f88c0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009f40 tx=0x7f35a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:10.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.943+0000 7f35a1ffb640 1 -- 172.21.15.16:0/2513451121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a403f020 con 0x7f35ac15e680 2024-09-19T07:54:10.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.943+0000 7f35a1ffb640 1 -- 172.21.15.16:0/2513451121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35a4037040 con 0x7f35ac15e680 2024-09-19T07:54:10.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.943+0000 7f35b3a5e640 1 -- 172.21.15.16:0/2513451121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ac1f55f0 con 0x7f35ac15e680 2024-09-19T07:54:10.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.943+0000 7f35a1ffb640 1 -- 172.21.15.16:0/2513451121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4036070 con 0x7f35ac15e680 2024-09-19T07:54:10.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.943+0000 7f35b3a5e640 1 -- 172.21.15.16:0/2513451121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ac1f80c0 con 0x7f35ac15e680 2024-09-19T07:54:10.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.944+0000 7f35b3a5e640 1 -- 172.21.15.16:0/2513451121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ac0d4c70 con 0x7f35ac15e680 2024-09-19T07:54:10.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.944+0000 7f35a1ffb640 1 -- 172.21.15.16:0/2513451121 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f35a4042020 con 0x7f35ac15e680 2024-09-19T07:54:10.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.945+0000 7f35a1ffb640 1 --2- 172.21.15.16:0/2513451121 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f359006c2b0 0x7f359006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:10.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.945+0000 7f35a1ffb640 1 -- 172.21.15.16:0/2513451121 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f35a40ab940 con 0x7f35ac15e680 2024-09-19T07:54:10.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.945+0000 7f35b17fa640 1 --2- 172.21.15.16:0/2513451121 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f359006c2b0 0x7f359006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:10.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.946+0000 7f35b17fa640 1 --2- 172.21.15.16:0/2513451121 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f359006c2b0 0x7f359006e770 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f35a8001680 tx=0x7f35a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:10.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:10.949+0000 7f35a1ffb640 1 -- 172.21.15.16:0/2513451121 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f35a4049030 con 0x7f35ac15e680 2024-09-19T07:54:11.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.162+0000 7f35b3a5e640 1 -- 172.21.15.16:0/2513451121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f35ac06cf90 con 0x7f35ac15e680 2024-09-19T07:54:11.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.162+0000 7f35a1ffb640 1 -- 172.21.15.16:0/2513451121 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=-1 MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it v65) ==== 316+0+0 (secure 0 0 0) 0x7f35a4079960 con 0x7f35ac15e680 2024-09-19T07:54:11.162 INFO:teuthology.orchestra.run.smithi016.stderr:Error EPERM: MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it 2024-09-19T07:54:11.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:54:11.168 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs --yes-i-really-mean-it 2024-09-19T07:54:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.272+0000 7f5bd073b640 1 Processor -- start 2024-09-19T07:54:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.272+0000 7f5bd073b640 1 -- start start 2024-09-19T07:54:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.273+0000 7f5bd073b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5bc815e360 0x7f5bc80ae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.273+0000 7f5bd073b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bc806fb40 0x7f5bc8070180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.273+0000 7f5bd073b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bc806e9e0 con 0x7f5bc80706c0 2024-09-19T07:54:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.273+0000 7f5bd073b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f5bc806e700 con 0x7f5bc815e360 2024-09-19T07:54:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.273+0000 7f5bd073b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5bc80b3cd0 con 0x7f5bc806fb40 2024-09-19T07:54:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.273+0000 7f5bce4d7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5bc815e360 0x7f5bc80ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.273+0000 7f5bcecd8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5bc80706c0 0x7f5bc80b4a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60634/0 (socket says 172.21.15.16:60634) 2024-09-19T07:54:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.273+0000 7f5bcecd8640 1 -- 172.21.15.16:0/1804963220 learned_addr learned my addr 172.21.15.16:0/1804963220 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.273+0000 7f5bcdcd6640 1 --2- 172.21.15.16:0/1804963220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bc806fb40 0x7f5bc8070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.273+0000 7f5bce4d7640 1 -- 172.21.15.16:0/1804963220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bc806fb40 msgr2=0x7f5bc8070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.273+0000 7f5bce4d7640 1 --2- 172.21.15.16:0/1804963220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bc806fb40 0x7f5bc8070180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.273+0000 7f5bce4d7640 1 -- 172.21.15.16:0/1804963220 >> v1:172.21.15.16:6789/0 conn(0x7f5bc80706c0 legacy=0x7f5bc80b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.273+0000 7f5bcd4d5640 1 -- 172.21.15.16:0/1804963220 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4161440962 0 0) 0x7f5bc806e9e0 con 0x7f5bc80706c0 2024-09-19T07:54:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.273+0000 7f5bce4d7640 1 -- 172.21.15.16:0/1804963220 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc80b3fb0 con 0x7f5bc815e360 2024-09-19T07:54:11.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.274+0000 7f5bce4d7640 1 --2- 172.21.15.16:0/1804963220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc815e360 0x7f5bc80ae910 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40097a0 tx=0x7f5bc402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b8c84e8bea6919 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:11.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.274+0000 7f5bcd4d5640 1 -- 172.21.15.16:0/1804963220 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4037070 con 0x7f5bc815e360 2024-09-19T07:54:11.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.274+0000 7f5bcd4d5640 1 -- 172.21.15.16:0/1804963220 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc4032440 con 0x7f5bc815e360 2024-09-19T07:54:11.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.274+0000 7f5bcd4d5640 1 -- 172.21.15.16:0/1804963220 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4032810 con 0x7f5bc815e360 2024-09-19T07:54:11.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.275+0000 7f5bd073b640 1 -- 172.21.15.16:0/1804963220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc815e360 msgr2=0x7f5bc80ae910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:11.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.275+0000 7f5bd073b640 1 --2- 172.21.15.16:0/1804963220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc815e360 0x7f5bc80ae910 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40097a0 tx=0x7f5bc402a180 comp rx=0 tx=0).stop 2024-09-19T07:54:11.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.275+0000 7f5bd073b640 1 -- 172.21.15.16:0/1804963220 shutdown_connections 2024-09-19T07:54:11.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.275+0000 7f5bd073b640 1 --2- 172.21.15.16:0/1804963220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bc806fb40 0x7f5bc8070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:11.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.275+0000 7f5bd073b640 1 --2- 172.21.15.16:0/1804963220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc815e360 0x7f5bc80ae910 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:11.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.275+0000 7f5bd073b640 1 -- 172.21.15.16:0/1804963220 >> 172.21.15.16:0/1804963220 conn(0x7f5bc8073560 msgr2=0x7f5bc806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:11.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.275+0000 7f5bd073b640 1 -- 172.21.15.16:0/1804963220 shutdown_connections 2024-09-19T07:54:11.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.275+0000 7f5bd073b640 1 -- 172.21.15.16:0/1804963220 wait complete. 2024-09-19T07:54:11.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.276+0000 7f5bd073b640 1 Processor -- start 2024-09-19T07:54:11.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.276+0000 7f5bd073b640 1 -- start start 2024-09-19T07:54:11.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.277+0000 7f5bd073b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc806fb40 0x7f5bc81e8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:11.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.277+0000 7f5bd073b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bc80706c0 0x7f5bc80cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:11.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.277+0000 7f5bd073b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bc815e360 0x7f5bc81452a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:11.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.277+0000 7f5bd073b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc80aee50 con 0x7f5bc806fb40 2024-09-19T07:54:11.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.277+0000 7f5bd073b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc806f6d0 con 0x7f5bc80706c0 2024-09-19T07:54:11.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.277+0000 7f5bd073b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5bc815e740 con 0x7f5bc815e360 2024-09-19T07:54:11.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.277+0000 7f5bce4d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc806fb40 0x7f5bc81e8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:11.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.277+0000 7f5bce4d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc806fb40 0x7f5bc81e8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38594/0 (socket says 172.21.15.16:38594) 2024-09-19T07:54:11.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.277+0000 7f5bce4d7640 1 -- 172.21.15.16:0/2429810492 learned_addr learned my addr 172.21.15.16:0/2429810492 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:11.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.277+0000 7f5bcdcd6640 1 --2- 172.21.15.16:0/2429810492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bc80706c0 0x7f5bc80cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:11.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.277+0000 7f5bcecd8640 1 --2- 172.21.15.16:0/2429810492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bc815e360 0x7f5bc81452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:11.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.277+0000 7f5bce4d7640 1 -- 172.21.15.16:0/2429810492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bc815e360 msgr2=0x7f5bc81452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.277+0000 7f5bce4d7640 1 --2- 172.21.15.16:0/2429810492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bc815e360 0x7f5bc81452a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.277+0000 7f5bce4d7640 1 -- 172.21.15.16:0/2429810492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bc80706c0 msgr2=0x7f5bc80cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.277+0000 7f5bce4d7640 1 --2- 172.21.15.16:0/2429810492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bc80706c0 0x7f5bc80cd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.277+0000 7f5bce4d7640 1 -- 172.21.15.16:0/2429810492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc81e9230 con 0x7f5bc806fb40 2024-09-19T07:54:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.277+0000 7f5bcdcd6640 1 --2- 172.21.15.16:0/2429810492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bc80706c0 0x7f5bc80cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.277+0000 7f5bce4d7640 1 --2- 172.21.15.16:0/2429810492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc806fb40 0x7f5bc81e8cf0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40098c0 tx=0x7f5bc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.278+0000 7f5bbeffd640 1 -- 172.21.15.16:0/2429810492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4037040 con 0x7f5bc806fb40 2024-09-19T07:54:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.278+0000 7f5bbeffd640 1 -- 172.21.15.16:0/2429810492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc4030500 con 0x7f5bc806fb40 2024-09-19T07:54:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.278+0000 7f5bd073b640 1 -- 172.21.15.16:0/2429810492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc81e94f0 con 0x7f5bc806fb40 2024-09-19T07:54:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.278+0000 7f5bbeffd640 1 -- 172.21.15.16:0/2429810492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc40308d0 con 0x7f5bc806fb40 2024-09-19T07:54:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.278+0000 7f5bd073b640 1 -- 172.21.15.16:0/2429810492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc81e9ab0 con 0x7f5bc806fb40 2024-09-19T07:54:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.279+0000 7f5bd073b640 1 -- 172.21.15.16:0/2429810492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bc8067d90 con 0x7f5bc806fb40 2024-09-19T07:54:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.279+0000 7f5bbeffd640 1 -- 172.21.15.16:0/2429810492 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5bc4044020 con 0x7f5bc806fb40 2024-09-19T07:54:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.280+0000 7f5bbeffd640 1 --2- 172.21.15.16:0/2429810492 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5bac06c2e0 0x7f5bac06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:11.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.280+0000 7f5bbeffd640 1 -- 172.21.15.16:0/2429810492 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f5bc40ab860 con 0x7f5bc806fb40 2024-09-19T07:54:11.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.280+0000 7f5bcdcd6640 1 --2- 172.21.15.16:0/2429810492 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5bac06c2e0 0x7f5bac06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:11.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.280+0000 7f5bcdcd6640 1 --2- 172.21.15.16:0/2429810492 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5bac06c2e0 0x7f5bac06e7a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5bc814b9b0 tx=0x7f5bb8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:11.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.284+0000 7f5bbeffd640 1 -- 172.21.15.16:0/2429810492 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5bc4079e40 con 0x7f5bc806fb40 2024-09-19T07:54:11.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.496+0000 7f5bd073b640 1 -- 172.21.15.16:0/2429810492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5bc80cd060 con 0x7f5bc806fb40 2024-09-19T07:54:11.497 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-19T07:54:11.497+0000 7f6950fea640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:54:11.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.498+0000 7f5bbeffd640 1 -- 172.21.15.16:0/2429810492 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) ==== 191+0+0 (secure 0 0 0) 0x7f5bc4079740 con 0x7f5bc806fb40 2024-09-19T07:54:11.498 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T07:54:11.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.501+0000 7f5bd073b640 1 -- 172.21.15.16:0/2429810492 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5bac06c2e0 msgr2=0x7f5bac06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:11.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.501+0000 7f5bd073b640 1 --2- 172.21.15.16:0/2429810492 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5bac06c2e0 0x7f5bac06e7a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5bc814b9b0 tx=0x7f5bb8008040 comp rx=0 tx=0).stop 2024-09-19T07:54:11.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.501+0000 7f5bd073b640 1 -- 172.21.15.16:0/2429810492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc806fb40 msgr2=0x7f5bc81e8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:11.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.501+0000 7f5bd073b640 1 --2- 172.21.15.16:0/2429810492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc806fb40 0x7f5bc81e8cf0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40098c0 tx=0x7f5bc40046c0 comp rx=0 tx=0).stop 2024-09-19T07:54:11.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.501+0000 7f5bd073b640 1 -- 172.21.15.16:0/2429810492 shutdown_connections 2024-09-19T07:54:11.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.501+0000 7f5bd073b640 1 --2- 172.21.15.16:0/2429810492 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5bac06c2e0 0x7f5bac06e7a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:11.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.501+0000 7f5bd073b640 1 --2- 172.21.15.16:0/2429810492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bc815e360 0x7f5bc81452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:11.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.501+0000 7f5bd073b640 1 --2- 172.21.15.16:0/2429810492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bc80706c0 0x7f5bc80cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:11.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.501+0000 7f5bd073b640 1 --2- 172.21.15.16:0/2429810492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc806fb40 0x7f5bc81e8cf0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:11.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.501+0000 7f5bd073b640 1 -- 172.21.15.16:0/2429810492 >> 172.21.15.16:0/2429810492 conn(0x7f5bc8073560 msgr2=0x7f5bc80cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:11.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.502+0000 7f5bd073b640 1 -- 172.21.15.16:0/2429810492 shutdown_connections 2024-09-19T07:54:11.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.502+0000 7f5bd073b640 1 -- 172.21.15.16:0/2429810492 wait complete. 2024-09-19T07:54:11.513 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T07:54:11.541 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-19T07:54:11.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.621+0000 7fd97e3c6640 1 Processor -- start 2024-09-19T07:54:11.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.621+0000 7fd97e3c6640 1 -- start start 2024-09-19T07:54:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.622+0000 7fd97e3c6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd97815e2a0 0x7fd97806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.622+0000 7fd97e3c6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd97815e680 0x7fd9780cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.622+0000 7fd97e3c6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9780d1880 con 0x7fd97806d960 2024-09-19T07:54:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.622+0000 7fd97e3c6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd9780d1b60 con 0x7fd97815e680 2024-09-19T07:54:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.622+0000 7fd97e3c6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fd9780cf2d0 con 0x7fd97815e2a0 2024-09-19T07:54:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.622+0000 7fd96ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd97815e680 0x7fd9780cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.622+0000 7fd96ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd97815e680 0x7fd9780cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38618/0 (socket says 172.21.15.16:38618) 2024-09-19T07:54:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.622+0000 7fd96ffff640 1 -- 172.21.15.16:0/1745236351 learned_addr learned my addr 172.21.15.16:0/1745236351 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.622+0000 7fd977fff640 1 --2- 172.21.15.16:0/1745236351 >> v2:172.21.15.146:3300/0 conn(0x7fd97815e2a0 0x7fd97806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:11.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.622+0000 7fd96ffff640 1 -- 172.21.15.16:0/1745236351 >> v2:172.21.15.146:3300/0 conn(0x7fd97815e2a0 msgr2=0x7fd97806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:11.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.622+0000 7fd96ffff640 1 --2- 172.21.15.16:0/1745236351 >> v2:172.21.15.146:3300/0 conn(0x7fd97815e2a0 0x7fd97806d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:11.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.622+0000 7fd96ffff640 1 -- 172.21.15.16:0/1745236351 >> v1:172.21.15.146:6789/0 conn(0x7fd97806d960 legacy=0x7fd9780d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:11.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.623+0000 7fd96ffff640 1 -- 172.21.15.16:0/1745236351 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9780cf5b0 con 0x7fd97815e680 2024-09-19T07:54:11.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.623+0000 7fd96ffff640 1 --2- 172.21.15.16:0/1745236351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97815e680 0x7fd9780cfad0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fd9780b6fa0 tx=0x7fd96002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97ca3eee5ed7b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:11.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.623+0000 7fd9777fe640 1 -- 172.21.15.16:0/1745236351 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd960037070 con 0x7fd97815e680 2024-09-19T07:54:11.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.623+0000 7fd9777fe640 1 -- 172.21.15.16:0/1745236351 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd960032440 con 0x7fd97815e680 2024-09-19T07:54:11.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.623+0000 7fd9777fe640 1 -- 172.21.15.16:0/1745236351 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd960032810 con 0x7fd97815e680 2024-09-19T07:54:11.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.624+0000 7fd97e3c6640 1 -- 172.21.15.16:0/1745236351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97815e680 msgr2=0x7fd9780cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:11.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.624+0000 7fd97e3c6640 1 --2- 172.21.15.16:0/1745236351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97815e680 0x7fd9780cfad0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fd9780b6fa0 tx=0x7fd96002a230 comp rx=0 tx=0).stop 2024-09-19T07:54:11.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.624+0000 7fd97e3c6640 1 -- 172.21.15.16:0/1745236351 shutdown_connections 2024-09-19T07:54:11.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.624+0000 7fd97e3c6640 1 --2- 172.21.15.16:0/1745236351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97815e680 0x7fd9780cfad0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:11.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.624+0000 7fd97e3c6640 1 --2- 172.21.15.16:0/1745236351 >> v2:172.21.15.146:3300/0 conn(0x7fd97815e2a0 0x7fd97806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:11.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.624+0000 7fd97e3c6640 1 -- 172.21.15.16:0/1745236351 >> 172.21.15.16:0/1745236351 conn(0x7fd97809d760 msgr2=0x7fd97808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:11.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.624+0000 7fd97e3c6640 1 -- 172.21.15.16:0/1745236351 shutdown_connections 2024-09-19T07:54:11.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.624+0000 7fd97e3c6640 1 -- 172.21.15.16:0/1745236351 wait complete. 2024-09-19T07:54:11.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.625+0000 7fd97e3c6640 1 Processor -- start 2024-09-19T07:54:11.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.625+0000 7fd97e3c6640 1 -- start start 2024-09-19T07:54:11.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.626+0000 7fd97e3c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97806d960 0x7fd97815d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:11.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.626+0000 7fd97e3c6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd97815e2a0 0x7fd9780cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:11.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.626+0000 7fd97e3c6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd97815e680 0x7fd9780ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:11.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.626+0000 7fd97e3c6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd9780d4c00 con 0x7fd97806d960 2024-09-19T07:54:11.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.626+0000 7fd97e3c6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd9780d4a80 con 0x7fd97815e2a0 2024-09-19T07:54:11.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.626+0000 7fd97e3c6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd9780d0010 con 0x7fd97815e680 2024-09-19T07:54:11.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.626+0000 7fd977fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97806d960 0x7fd97815d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.626+0000 7fd977fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97806d960 0x7fd97815d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38628/0 (socket says 172.21.15.16:38628) 2024-09-19T07:54:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.626+0000 7fd977fff640 1 -- 172.21.15.16:0/2082542170 learned_addr learned my addr 172.21.15.16:0/2082542170 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.626+0000 7fd97c963640 1 --2- 172.21.15.16:0/2082542170 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd97815e680 0x7fd9780ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.626+0000 7fd96ffff640 1 --2- 172.21.15.16:0/2082542170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd97815e2a0 0x7fd9780cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.626+0000 7fd977fff640 1 -- 172.21.15.16:0/2082542170 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd97815e680 msgr2=0x7fd9780ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.626+0000 7fd977fff640 1 --2- 172.21.15.16:0/2082542170 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd97815e680 0x7fd9780ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.626+0000 7fd977fff640 1 -- 172.21.15.16:0/2082542170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd97815e2a0 msgr2=0x7fd9780cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.626+0000 7fd977fff640 1 --2- 172.21.15.16:0/2082542170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd97815e2a0 0x7fd9780cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.626+0000 7fd977fff640 1 -- 172.21.15.16:0/2082542170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9781e7fe0 con 0x7fd97806d960 2024-09-19T07:54:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.626+0000 7fd96ffff640 1 --2- 172.21.15.16:0/2082542170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd97815e2a0 0x7fd9780cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.626+0000 7fd97c963640 1 --2- 172.21.15.16:0/2082542170 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd97815e680 0x7fd9780ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.627+0000 7fd977fff640 1 --2- 172.21.15.16:0/2082542170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97806d960 0x7fd97815d410 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fd96800d800 tx=0x7fd96800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.627+0000 7fd9757fa640 1 -- 172.21.15.16:0/2082542170 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96800b6c0 con 0x7fd97806d960 2024-09-19T07:54:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.627+0000 7fd9757fa640 1 -- 172.21.15.16:0/2082542170 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd968019070 con 0x7fd97806d960 2024-09-19T07:54:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.627+0000 7fd97e3c6640 1 -- 172.21.15.16:0/2082542170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9781e82c0 con 0x7fd97806d960 2024-09-19T07:54:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.627+0000 7fd9757fa640 1 -- 172.21.15.16:0/2082542170 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96800ba80 con 0x7fd97806d960 2024-09-19T07:54:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.627+0000 7fd97e3c6640 1 -- 172.21.15.16:0/2082542170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9781e8880 con 0x7fd97806d960 2024-09-19T07:54:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.628+0000 7fd97e3c6640 1 -- 172.21.15.16:0/2082542170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd97806d420 con 0x7fd97806d960 2024-09-19T07:54:11.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.628+0000 7fd9757fa640 1 -- 172.21.15.16:0/2082542170 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd96800bc20 con 0x7fd97806d960 2024-09-19T07:54:11.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.629+0000 7fd9757fa640 1 --2- 172.21.15.16:0/2082542170 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd95806c2b0 0x7fd95806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:11.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.629+0000 7fd9757fa640 1 -- 172.21.15.16:0/2082542170 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7fd96808bb10 con 0x7fd97806d960 2024-09-19T07:54:11.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.629+0000 7fd96ffff640 1 --2- 172.21.15.16:0/2082542170 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd95806c2b0 0x7fd95806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:11.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.630+0000 7fd96ffff640 1 --2- 172.21.15.16:0/2082542170 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd95806c2b0 0x7fd95806e770 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd960034000 tx=0x7fd96002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:11.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.633+0000 7fd9757fa640 1 -- 172.21.15.16:0/2082542170 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd96805a010 con 0x7fd97806d960 2024-09-19T07:54:11.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:11.846+0000 7fd97e3c6640 1 -- 172.21.15.16:0/2082542170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd97806cf90 con 0x7fd97806d960 2024-09-19T07:54:12.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.500+0000 7fd9757fa640 1 -- 172.21.15.16:0/2082542170 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v67) ==== 104+0+0 (secure 0 0 0) 0x7fd968059910 con 0x7fd97806d960 2024-09-19T07:54:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.503+0000 7fd97e3c6640 1 -- 172.21.15.16:0/2082542170 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd95806c2b0 msgr2=0x7fd95806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.503+0000 7fd97e3c6640 1 --2- 172.21.15.16:0/2082542170 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd95806c2b0 0x7fd95806e770 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd960034000 tx=0x7fd96002e000 comp rx=0 tx=0).stop 2024-09-19T07:54:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.503+0000 7fd97e3c6640 1 -- 172.21.15.16:0/2082542170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97806d960 msgr2=0x7fd97815d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.503+0000 7fd97e3c6640 1 --2- 172.21.15.16:0/2082542170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97806d960 0x7fd97815d410 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fd96800d800 tx=0x7fd96800dcc0 comp rx=0 tx=0).stop 2024-09-19T07:54:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.503+0000 7fd97e3c6640 1 -- 172.21.15.16:0/2082542170 shutdown_connections 2024-09-19T07:54:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.503+0000 7fd97e3c6640 1 --2- 172.21.15.16:0/2082542170 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd95806c2b0 0x7fd95806e770 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.503+0000 7fd97e3c6640 1 --2- 172.21.15.16:0/2082542170 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd97815e680 0x7fd9780ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.503+0000 7fd97e3c6640 1 --2- 172.21.15.16:0/2082542170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd97815e2a0 0x7fd9780cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.503+0000 7fd97e3c6640 1 --2- 172.21.15.16:0/2082542170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97806d960 0x7fd97815d410 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.503+0000 7fd97e3c6640 1 -- 172.21.15.16:0/2082542170 >> 172.21.15.16:0/2082542170 conn(0x7fd97809d760 msgr2=0x7fd9780750a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.503+0000 7fd97e3c6640 1 -- 172.21.15.16:0/2082542170 shutdown_connections 2024-09-19T07:54:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.503+0000 7fd97e3c6640 1 -- 172.21.15.16:0/2082542170 wait complete. 2024-09-19T07:54:12.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T07:54:12.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.672+0000 7f32b4eed640 1 Processor -- start 2024-09-19T07:54:12.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.672+0000 7f32b4eed640 1 -- start start 2024-09-19T07:54:12.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.673+0000 7f32b4eed640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32b015f0d0 0x7f32b015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:12.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.673+0000 7f32b4eed640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f32b00d0070 0x7f32b00d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:12.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.673+0000 7f32b4eed640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32b00cf2b0 con 0x7f32b00d1940 2024-09-19T07:54:12.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.673+0000 7f32b4eed640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f32b00cf590 con 0x7f32b015f0d0 2024-09-19T07:54:12.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.673+0000 7f32b4eed640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f32b00cf870 con 0x7f32b00d0070 2024-09-19T07:54:12.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.673+0000 7f32ae59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32b015f0d0 0x7f32b015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:12.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.673+0000 7f32aed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f32b00d1940 0x7f32b00d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42660/0 (socket says 172.21.15.16:42660) 2024-09-19T07:54:12.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.673+0000 7f32aed9d640 1 -- 172.21.15.16:0/295787331 learned_addr learned my addr 172.21.15.16:0/295787331 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:12.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.673+0000 7f32add9b640 1 --2- 172.21.15.16:0/295787331 >> v2:172.21.15.146:3300/0 conn(0x7f32b00d0070 0x7f32b00d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:12.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.673+0000 7f32ae59c640 1 -- 172.21.15.16:0/295787331 >> v2:172.21.15.146:3300/0 conn(0x7f32b00d0070 msgr2=0x7f32b00d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:12.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.673+0000 7f32ae59c640 1 --2- 172.21.15.16:0/295787331 >> v2:172.21.15.146:3300/0 conn(0x7f32b00d0070 0x7f32b00d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:12.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.673+0000 7f32ae59c640 1 -- 172.21.15.16:0/295787331 >> v1:172.21.15.16:6789/0 conn(0x7f32b00d1940 legacy=0x7f32b00d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:12.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.673+0000 7f32ae59c640 1 -- 172.21.15.16:0/295787331 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b00cfb50 con 0x7f32b015f0d0 2024-09-19T07:54:12.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.673+0000 7f32ad59a640 1 -- 172.21.15.16:0/295787331 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2408048363 0 0) 0x7f32b00cf2b0 con 0x7f32b00d1940 2024-09-19T07:54:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.674+0000 7f32ae59c640 1 --2- 172.21.15.16:0/295787331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b015f0d0 0x7f32b015f4b0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f32b0162fc0 tx=0x7f32a002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc8a10d7a6a314f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.674+0000 7f32ad59a640 1 -- 172.21.15.16:0/295787331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0037070 con 0x7f32b015f0d0 2024-09-19T07:54:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.674+0000 7f32ad59a640 1 -- 172.21.15.16:0/295787331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a0032440 con 0x7f32b015f0d0 2024-09-19T07:54:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.674+0000 7f32ad59a640 1 -- 172.21.15.16:0/295787331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a002ae60 con 0x7f32b015f0d0 2024-09-19T07:54:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.675+0000 7f32b4eed640 1 -- 172.21.15.16:0/295787331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b015f0d0 msgr2=0x7f32b015f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.675+0000 7f32b4eed640 1 --2- 172.21.15.16:0/295787331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b015f0d0 0x7f32b015f4b0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f32b0162fc0 tx=0x7f32a002a110 comp rx=0 tx=0).stop 2024-09-19T07:54:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.675+0000 7f32b4eed640 1 -- 172.21.15.16:0/295787331 shutdown_connections 2024-09-19T07:54:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.675+0000 7f32b4eed640 1 --2- 172.21.15.16:0/295787331 >> v2:172.21.15.146:3300/0 conn(0x7f32b00d0070 0x7f32b00d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.675+0000 7f32b4eed640 1 --2- 172.21.15.16:0/295787331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b015f0d0 0x7f32b015f4b0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.675+0000 7f32b4eed640 1 -- 172.21.15.16:0/295787331 >> 172.21.15.16:0/295787331 conn(0x7f32b009d760 msgr2=0x7f32b008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.675+0000 7f32b4eed640 1 -- 172.21.15.16:0/295787331 shutdown_connections 2024-09-19T07:54:12.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.675+0000 7f32b4eed640 1 -- 172.21.15.16:0/295787331 wait complete. 2024-09-19T07:54:12.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.676+0000 7f32b4eed640 1 Processor -- start 2024-09-19T07:54:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.676+0000 7f32b4eed640 1 -- start start 2024-09-19T07:54:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.677+0000 7f32b4eed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b00d0070 0x7f32b00ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.677+0000 7f32b4eed640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b00d1940 0x7f32b00b92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.677+0000 7f32b4eed640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32b015f0d0 0x7f32b01f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.677+0000 7f32b4eed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32b00d49c0 con 0x7f32b00d0070 2024-09-19T07:54:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.677+0000 7f32b4eed640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f32b00d4840 con 0x7f32b015f0d0 2024-09-19T07:54:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.677+0000 7f32b4eed640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f32b00d4b40 con 0x7f32b00d1940 2024-09-19T07:54:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.677+0000 7f32ae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b00d0070 0x7f32b00ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.677+0000 7f32ae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b00d0070 0x7f32b00ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54620/0 (socket says 172.21.15.16:54620) 2024-09-19T07:54:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.677+0000 7f32ae59c640 1 -- 172.21.15.16:0/16819120 learned_addr learned my addr 172.21.15.16:0/16819120 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.677+0000 7f32ae59c640 1 -- 172.21.15.16:0/16819120 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b00d1940 msgr2=0x7f32b00b92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.677+0000 7f32aed9d640 1 --2- 172.21.15.16:0/16819120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32b015f0d0 0x7f32b01f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:12.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.677+0000 7f32ae59c640 1 --2- 172.21.15.16:0/16819120 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b00d1940 0x7f32b00b92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:12.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.677+0000 7f32ae59c640 1 -- 172.21.15.16:0/16819120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32b015f0d0 msgr2=0x7f32b01f76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:12.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.677+0000 7f32ae59c640 1 --2- 172.21.15.16:0/16819120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32b015f0d0 0x7f32b01f76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:12.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.677+0000 7f32ae59c640 1 -- 172.21.15.16:0/16819120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b00b8000 con 0x7f32b00d0070 2024-09-19T07:54:12.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.677+0000 7f32aed9d640 1 --2- 172.21.15.16:0/16819120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32b015f0d0 0x7f32b01f76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:12.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.678+0000 7f32ae59c640 1 --2- 172.21.15.16:0/16819120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b00d0070 0x7f32b00ce210 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f32a0000c00 tx=0x7f32a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:12.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.678+0000 7f329effd640 1 -- 172.21.15.16:0/16819120 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0037040 con 0x7f32b00d0070 2024-09-19T07:54:12.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.678+0000 7f329effd640 1 -- 172.21.15.16:0/16819120 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a0042070 con 0x7f32b00d0070 2024-09-19T07:54:12.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.678+0000 7f329effd640 1 -- 172.21.15.16:0/16819120 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0030bd0 con 0x7f32b00d0070 2024-09-19T07:54:12.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.678+0000 7f32b4eed640 1 -- 172.21.15.16:0/16819120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b00b82e0 con 0x7f32b00d0070 2024-09-19T07:54:12.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.678+0000 7f32b4eed640 1 -- 172.21.15.16:0/16819120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b00b88a0 con 0x7f32b00d0070 2024-09-19T07:54:12.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.679+0000 7f32b4eed640 1 -- 172.21.15.16:0/16819120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b0075710 con 0x7f32b00d0070 2024-09-19T07:54:12.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.679+0000 7f329effd640 1 -- 172.21.15.16:0/16819120 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f32a0047460 con 0x7f32b00d0070 2024-09-19T07:54:12.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.680+0000 7f329effd640 1 --2- 172.21.15.16:0/16819120 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f329006c2b0 0x7f329006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:12.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.680+0000 7f329effd640 1 -- 172.21.15.16:0/16819120 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7f32a00ac150 con 0x7f32b00d0070 2024-09-19T07:54:12.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.680+0000 7f32add9b640 1 --2- 172.21.15.16:0/16819120 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f329006c2b0 0x7f329006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:12.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.681+0000 7f32add9b640 1 --2- 172.21.15.16:0/16819120 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f329006c2b0 0x7f329006e770 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f32b006d550 tx=0x7f32a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:12.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.684+0000 7f329effd640 1 -- 172.21.15.16:0/16819120 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f32b0075710 con 0x7f32b00d0070 2024-09-19T07:54:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:12.865+0000 7f32b4eed640 1 -- 172.21.15.16:0/16819120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f32b006cf30 con 0x7f32b00d0070 2024-09-19T07:54:13.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.504+0000 7f329effd640 1 -- 172.21.15.16:0/16819120 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v120) ==== 181+0+0 (secure 0 0 0) 0x7f32a0032e20 con 0x7f32b00d0070 2024-09-19T07:54:13.505 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-19T07:54:13.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.507+0000 7f32b4eed640 1 -- 172.21.15.16:0/16819120 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f329006c2b0 msgr2=0x7f329006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:13.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.507+0000 7f32b4eed640 1 --2- 172.21.15.16:0/16819120 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f329006c2b0 0x7f329006e770 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f32b006d550 tx=0x7f32a4008040 comp rx=0 tx=0).stop 2024-09-19T07:54:13.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.507+0000 7f32b4eed640 1 -- 172.21.15.16:0/16819120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b00d0070 msgr2=0x7f32b00ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:13.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.507+0000 7f32b4eed640 1 --2- 172.21.15.16:0/16819120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b00d0070 0x7f32b00ce210 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f32a0000c00 tx=0x7f32a00046c0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.507+0000 7f32b4eed640 1 -- 172.21.15.16:0/16819120 shutdown_connections 2024-09-19T07:54:13.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.507+0000 7f32b4eed640 1 --2- 172.21.15.16:0/16819120 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f329006c2b0 0x7f329006e770 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.507+0000 7f32b4eed640 1 --2- 172.21.15.16:0/16819120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32b015f0d0 0x7f32b01f76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.507+0000 7f32b4eed640 1 --2- 172.21.15.16:0/16819120 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b00d1940 0x7f32b00b92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.507+0000 7f32b4eed640 1 --2- 172.21.15.16:0/16819120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b00d0070 0x7f32b00ce210 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.507+0000 7f32b4eed640 1 -- 172.21.15.16:0/16819120 >> 172.21.15.16:0/16819120 conn(0x7f32b009d760 msgr2=0x7f32b00d1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:13.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.507+0000 7f32b4eed640 1 -- 172.21.15.16:0/16819120 shutdown_connections 2024-09-19T07:54:13.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.508+0000 7f32b4eed640 1 -- 172.21.15.16:0/16819120 wait complete. 2024-09-19T07:54:13.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T07:54:13.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.632+0000 7f9e89e45640 1 Processor -- start 2024-09-19T07:54:13.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.632+0000 7f9e89e45640 1 -- start start 2024-09-19T07:54:13.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.633+0000 7f9e89e45640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e8406d430 0x7f9e841439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.633+0000 7f9e89e45640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e84143f30 0x7f9e840d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.633+0000 7f9e89e45640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e840c7140 con 0x7f9e840c8d20 2024-09-19T07:54:13.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.633+0000 7f9e89e45640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9e84141960 con 0x7f9e8406d430 2024-09-19T07:54:13.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.633+0000 7f9e89e45640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9e84142200 con 0x7f9e84143f30 2024-09-19T07:54:13.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.633+0000 7f9e82ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e8406d430 0x7f9e841439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.633+0000 7f9e82ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e8406d430 0x7f9e841439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54632/0 (socket says 172.21.15.16:54632) 2024-09-19T07:54:13.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.633+0000 7f9e82ffd640 1 -- 172.21.15.16:0/4251570112 learned_addr learned my addr 172.21.15.16:0/4251570112 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:13.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.633+0000 7f9e83fff640 1 --2- 172.21.15.16:0/4251570112 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e84143f30 0x7f9e840d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.634+0000 7f9e82ffd640 1 -- 172.21.15.16:0/4251570112 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e84143f30 msgr2=0x7f9e840d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:13.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.634+0000 7f9e82ffd640 1 --2- 172.21.15.16:0/4251570112 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e84143f30 0x7f9e840d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.634+0000 7f9e82ffd640 1 -- 172.21.15.16:0/4251570112 >> v1:172.21.15.146:6789/0 conn(0x7f9e840c8d20 legacy=0x7f9e8406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:13.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.634+0000 7f9e82ffd640 1 -- 172.21.15.16:0/4251570112 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e840c9100 con 0x7f9e8406d430 2024-09-19T07:54:13.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.634+0000 7f9e82ffd640 1 --2- 172.21.15.16:0/4251570112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e8406d430 0x7f9e841439f0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009870 tx=0x7f9e7802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec386c1327cc94d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:13.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.634+0000 7f9e827fc640 1 -- 172.21.15.16:0/4251570112 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78037070 con 0x7f9e8406d430 2024-09-19T07:54:13.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.634+0000 7f9e827fc640 1 -- 172.21.15.16:0/4251570112 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e78032440 con 0x7f9e8406d430 2024-09-19T07:54:13.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.634+0000 7f9e827fc640 1 -- 172.21.15.16:0/4251570112 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78032810 con 0x7f9e8406d430 2024-09-19T07:54:13.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.635+0000 7f9e89e45640 1 -- 172.21.15.16:0/4251570112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e8406d430 msgr2=0x7f9e841439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:13.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.635+0000 7f9e89e45640 1 --2- 172.21.15.16:0/4251570112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e8406d430 0x7f9e841439f0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009870 tx=0x7f9e7802a130 comp rx=0 tx=0).stop 2024-09-19T07:54:13.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.635+0000 7f9e89e45640 1 -- 172.21.15.16:0/4251570112 shutdown_connections 2024-09-19T07:54:13.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.635+0000 7f9e89e45640 1 --2- 172.21.15.16:0/4251570112 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e84143f30 0x7f9e840d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.635+0000 7f9e89e45640 1 --2- 172.21.15.16:0/4251570112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e8406d430 0x7f9e841439f0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.635+0000 7f9e89e45640 1 -- 172.21.15.16:0/4251570112 >> 172.21.15.16:0/4251570112 conn(0x7f9e840b3ed0 msgr2=0x7f9e840b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:13.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.635+0000 7f9e89e45640 1 -- 172.21.15.16:0/4251570112 shutdown_connections 2024-09-19T07:54:13.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.635+0000 7f9e89e45640 1 -- 172.21.15.16:0/4251570112 wait complete. 2024-09-19T07:54:13.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.636+0000 7f9e89e45640 1 Processor -- start 2024-09-19T07:54:13.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.637+0000 7f9e89e45640 1 -- start start 2024-09-19T07:54:13.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.637+0000 7f9e89e45640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e8406d430 0x7f9e840b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.637+0000 7f9e89e45640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e840c8d20 0x7f9e84140b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.637+0000 7f9e89e45640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e84143f30 0x7f9e840d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.637+0000 7f9e89e45640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9e84141e90 con 0x7f9e84143f30 2024-09-19T07:54:13.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.637+0000 7f9e89e45640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9e84141bb0 con 0x7f9e840c8d20 2024-09-19T07:54:13.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.637+0000 7f9e89e45640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9e84144350 con 0x7f9e8406d430 2024-09-19T07:54:13.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.637+0000 7f9e83fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e84143f30 0x7f9e840d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.637+0000 7f9e83fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e84143f30 0x7f9e840d1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54634/0 (socket says 172.21.15.16:54634) 2024-09-19T07:54:13.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.637+0000 7f9e83fff640 1 -- 172.21.15.16:0/2197511974 learned_addr learned my addr 172.21.15.16:0/2197511974 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:13.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.638+0000 7f9e82ffd640 1 --2- 172.21.15.16:0/2197511974 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e840c8d20 0x7f9e84140b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.637+0000 7f9e837fe640 1 --2- 172.21.15.16:0/2197511974 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e8406d430 0x7f9e840b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.638+0000 7f9e83fff640 1 -- 172.21.15.16:0/2197511974 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e8406d430 msgr2=0x7f9e840b6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:13.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.638+0000 7f9e83fff640 1 --2- 172.21.15.16:0/2197511974 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e8406d430 0x7f9e840b6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.638+0000 7f9e83fff640 1 -- 172.21.15.16:0/2197511974 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e840c8d20 msgr2=0x7f9e84140b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:13.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.638+0000 7f9e83fff640 1 --2- 172.21.15.16:0/2197511974 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e840c8d20 0x7f9e84140b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.638+0000 7f9e83fff640 1 -- 172.21.15.16:0/2197511974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e840b6a70 con 0x7f9e84143f30 2024-09-19T07:54:13.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.638+0000 7f9e83fff640 1 --2- 172.21.15.16:0/2197511974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e84143f30 0x7f9e840d1e20 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c007e60 tx=0x7f9e7c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:13.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.638+0000 7f9e6bfff640 1 -- 172.21.15.16:0/2197511974 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c014420 con 0x7f9e84143f30 2024-09-19T07:54:13.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.638+0000 7f9e6bfff640 1 -- 172.21.15.16:0/2197511974 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e7c019070 con 0x7f9e84143f30 2024-09-19T07:54:13.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.638+0000 7f9e89e45640 1 -- 172.21.15.16:0/2197511974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e840b6d50 con 0x7f9e84143f30 2024-09-19T07:54:13.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.638+0000 7f9e6bfff640 1 -- 172.21.15.16:0/2197511974 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c014820 con 0x7f9e84143f30 2024-09-19T07:54:13.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.638+0000 7f9e89e45640 1 -- 172.21.15.16:0/2197511974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e840b7310 con 0x7f9e84143f30 2024-09-19T07:54:13.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.639+0000 7f9e89e45640 1 -- 172.21.15.16:0/2197511974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e841439f0 con 0x7f9e84143f30 2024-09-19T07:54:13.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.640+0000 7f9e6bfff640 1 -- 172.21.15.16:0/2197511974 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9e7c0149c0 con 0x7f9e84143f30 2024-09-19T07:54:13.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.640+0000 7f9e6bfff640 1 --2- 172.21.15.16:0/2197511974 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9e7006c2b0 0x7f9e7006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.640+0000 7f9e6bfff640 1 -- 172.21.15.16:0/2197511974 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5749+0+0 (secure 0 0 0) 0x7f9e7c08bb70 con 0x7f9e84143f30 2024-09-19T07:54:13.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.640+0000 7f9e837fe640 1 --2- 172.21.15.16:0/2197511974 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9e7006c2b0 0x7f9e7006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.641+0000 7f9e837fe640 1 --2- 172.21.15.16:0/2197511974 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9e7006c2b0 0x7f9e7006e770 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9e740016e0 tx=0x7f9e74009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:13.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.644+0000 7f9e6bfff640 1 -- 172.21.15.16:0/2197511974 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9e7c059c60 con 0x7f9e84143f30 2024-09-19T07:54:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:13.820+0000 7f9e89e45640 1 -- 172.21.15.16:0/2197511974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9e841428a0 con 0x7f9e84143f30 2024-09-19T07:54:14.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.506+0000 7f9e6bfff640 1 -- 172.21.15.16:0/2197511974 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v121) ==== 169+0+0 (secure 0 0 0) 0x7f9e7c05df60 con 0x7f9e84143f30 2024-09-19T07:54:14.506 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-19T07:54:14.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.512+0000 7f9e89e45640 1 -- 172.21.15.16:0/2197511974 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9e7006c2b0 msgr2=0x7f9e7006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:14.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.512+0000 7f9e89e45640 1 --2- 172.21.15.16:0/2197511974 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9e7006c2b0 0x7f9e7006e770 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9e740016e0 tx=0x7f9e74009040 comp rx=0 tx=0).stop 2024-09-19T07:54:14.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.512+0000 7f9e89e45640 1 -- 172.21.15.16:0/2197511974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e84143f30 msgr2=0x7f9e840d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:14.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.512+0000 7f9e89e45640 1 --2- 172.21.15.16:0/2197511974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e84143f30 0x7f9e840d1e20 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c007e60 tx=0x7f9e7c00e4b0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.513+0000 7f9e89e45640 1 -- 172.21.15.16:0/2197511974 shutdown_connections 2024-09-19T07:54:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.513+0000 7f9e89e45640 1 --2- 172.21.15.16:0/2197511974 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9e7006c2b0 0x7f9e7006e770 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.513+0000 7f9e89e45640 1 --2- 172.21.15.16:0/2197511974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e84143f30 0x7f9e840d1e20 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.513+0000 7f9e89e45640 1 --2- 172.21.15.16:0/2197511974 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e840c8d20 0x7f9e84140b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.513+0000 7f9e89e45640 1 --2- 172.21.15.16:0/2197511974 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e8406d430 0x7f9e840b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.513+0000 7f9e89e45640 1 -- 172.21.15.16:0/2197511974 >> 172.21.15.16:0/2197511974 conn(0x7f9e840b3ed0 msgr2=0x7f9e84142dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:14.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.514+0000 7f9e89e45640 1 -- 172.21.15.16:0/2197511974 shutdown_connections 2024-09-19T07:54:14.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.514+0000 7f9e89e45640 1 -- 172.21.15.16:0/2197511974 wait complete. 2024-09-19T07:54:14.529 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2024-09-19T07:54:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.639+0000 7f86f0318640 1 Processor -- start 2024-09-19T07:54:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.639+0000 7f86f0318640 1 -- start start 2024-09-19T07:54:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.640+0000 7f86f0318640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f86e815f0d0 0x7f86e815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.640+0000 7f86f0318640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86e80d4130 0x7f86e80d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.640+0000 7f86f0318640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86e80d4ae0 con 0x7f86e80d0070 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.640+0000 7f86f0318640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f86e80d1b10 con 0x7f86e815f0d0 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.640+0000 7f86f0318640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f86e80cf2b0 con 0x7f86e80d4130 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.640+0000 7f86ee0b4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f86e815f0d0 0x7f86e815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.640+0000 7f86ed8b3640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f86e80d0070 0x7f86e80d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42668/0 (socket says 172.21.15.16:42668) 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.640+0000 7f86ed8b3640 1 -- 172.21.15.16:0/131138191 learned_addr learned my addr 172.21.15.16:0/131138191 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.640+0000 7f86ee8b5640 1 --2- 172.21.15.16:0/131138191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86e80d4130 0x7f86e80d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.641+0000 7f86ee0b4640 1 -- 172.21.15.16:0/131138191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86e80d4130 msgr2=0x7f86e80d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.641+0000 7f86ee0b4640 1 --2- 172.21.15.16:0/131138191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86e80d4130 0x7f86e80d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.641+0000 7f86ee0b4640 1 -- 172.21.15.16:0/131138191 >> v1:172.21.15.16:6789/0 conn(0x7f86e80d0070 legacy=0x7f86e80d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.641+0000 7f86ee0b4640 1 -- 172.21.15.16:0/131138191 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e80cf590 con 0x7f86e815f0d0 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.641+0000 7f86ee0b4640 1 --2- 172.21.15.16:0/131138191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e815f0d0 0x7f86e815f4b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f86e40097a0 tx=0x7f86e402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473c74d907462165 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.641+0000 7f86ed0b2640 1 -- 172.21.15.16:0/131138191 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4037070 con 0x7f86e815f0d0 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.641+0000 7f86ed0b2640 1 -- 172.21.15.16:0/131138191 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e402ae50 con 0x7f86e815f0d0 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.641+0000 7f86ed0b2640 1 -- 172.21.15.16:0/131138191 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4032660 con 0x7f86e815f0d0 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.642+0000 7f86f0318640 1 -- 172.21.15.16:0/131138191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e815f0d0 msgr2=0x7f86e815f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.642+0000 7f86f0318640 1 --2- 172.21.15.16:0/131138191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e815f0d0 0x7f86e815f4b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f86e40097a0 tx=0x7f86e402a100 comp rx=0 tx=0).stop 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.642+0000 7f86f0318640 1 -- 172.21.15.16:0/131138191 shutdown_connections 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.642+0000 7f86f0318640 1 --2- 172.21.15.16:0/131138191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86e80d4130 0x7f86e80d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.642+0000 7f86f0318640 1 --2- 172.21.15.16:0/131138191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e815f0d0 0x7f86e815f4b0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.642+0000 7f86f0318640 1 -- 172.21.15.16:0/131138191 >> 172.21.15.16:0/131138191 conn(0x7f86e809d760 msgr2=0x7f86e808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:14.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.642+0000 7f86f0318640 1 -- 172.21.15.16:0/131138191 shutdown_connections 2024-09-19T07:54:14.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.642+0000 7f86f0318640 1 -- 172.21.15.16:0/131138191 wait complete. 2024-09-19T07:54:14.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.643+0000 7f86f0318640 1 Processor -- start 2024-09-19T07:54:14.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.643+0000 7f86f0318640 1 -- start start 2024-09-19T07:54:14.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.644+0000 7f86f0318640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e80d0070 0x7f86e81f7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:14.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.644+0000 7f86f0318640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86e80d4130 0x7f86e80b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.644+0000 7f86f0318640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86e815f0d0 0x7f86e80b98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.644+0000 7f86f0318640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86e80cef60 con 0x7f86e80d0070 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.644+0000 7f86f0318640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f86e80cfb50 con 0x7f86e815f0d0 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.644+0000 7f86f0318640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f86e80b2f70 con 0x7f86e80d4130 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.644+0000 7f86ee0b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e80d0070 0x7f86e81f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.644+0000 7f86ee0b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e80d0070 0x7f86e81f7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54644/0 (socket says 172.21.15.16:54644) 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.644+0000 7f86ee0b4640 1 -- 172.21.15.16:0/1136012138 learned_addr learned my addr 172.21.15.16:0/1136012138 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.644+0000 7f86ed8b3640 1 --2- 172.21.15.16:0/1136012138 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86e80d4130 0x7f86e80b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.644+0000 7f86ee8b5640 1 --2- 172.21.15.16:0/1136012138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86e815f0d0 0x7f86e80b98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.644+0000 7f86ee0b4640 1 -- 172.21.15.16:0/1136012138 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86e80d4130 msgr2=0x7f86e80b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.644+0000 7f86ee0b4640 1 --2- 172.21.15.16:0/1136012138 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86e80d4130 0x7f86e80b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.644+0000 7f86ee0b4640 1 -- 172.21.15.16:0/1136012138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86e815f0d0 msgr2=0x7f86e80b98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.644+0000 7f86ee0b4640 1 --2- 172.21.15.16:0/1136012138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86e815f0d0 0x7f86e80b98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.644+0000 7f86ee0b4640 1 -- 172.21.15.16:0/1136012138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e814b7f0 con 0x7f86e80d0070 2024-09-19T07:54:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.645+0000 7f86ed8b3640 1 --2- 172.21.15.16:0/1136012138 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86e80d4130 0x7f86e80b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.645+0000 7f86ee0b4640 1 --2- 172.21.15.16:0/1136012138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e80d0070 0x7f86e81f7100 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f86e40098c0 tx=0x7f86e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.645+0000 7f86deffd640 1 -- 172.21.15.16:0/1136012138 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4041030 con 0x7f86e80d0070 2024-09-19T07:54:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.645+0000 7f86deffd640 1 -- 172.21.15.16:0/1136012138 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e402ac50 con 0x7f86e80d0070 2024-09-19T07:54:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.645+0000 7f86deffd640 1 -- 172.21.15.16:0/1136012138 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4037040 con 0x7f86e80d0070 2024-09-19T07:54:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.645+0000 7f86f0318640 1 -- 172.21.15.16:0/1136012138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e81489a0 con 0x7f86e80d0070 2024-09-19T07:54:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.645+0000 7f86f0318640 1 -- 172.21.15.16:0/1136012138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e8148f60 con 0x7f86e80d0070 2024-09-19T07:54:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.646+0000 7f86f0318640 1 -- 172.21.15.16:0/1136012138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e80d45a0 con 0x7f86e80d0070 2024-09-19T07:54:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.646+0000 7f86deffd640 1 -- 172.21.15.16:0/1136012138 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f86e4044070 con 0x7f86e80d0070 2024-09-19T07:54:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.647+0000 7f86deffd640 1 --2- 172.21.15.16:0/1136012138 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f86cc06c2b0 0x7f86cc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.647+0000 7f86deffd640 1 -- 172.21.15.16:0/1136012138 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7f86e402ae00 con 0x7f86e80d0070 2024-09-19T07:54:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.647+0000 7f86ed8b3640 1 --2- 172.21.15.16:0/1136012138 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f86cc06c2b0 0x7f86cc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.648+0000 7f86ed8b3640 1 --2- 172.21.15.16:0/1136012138 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f86cc06c2b0 0x7f86cc06e770 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f86d8000f60 tx=0x7f86d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.651+0000 7f86deffd640 1 -- 172.21.15.16:0/1136012138 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f86e4044e40 con 0x7f86e80d0070 2024-09-19T07:54:14.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.825+0000 7f86f0318640 1 -- 172.21.15.16:0/1136012138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) -- 0x7f86e806cf30 con 0x7f86e80d0070 2024-09-19T07:54:14.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.829+0000 7f86deffd640 1 -- 172.21.15.16:0/1136012138 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v48) ==== 111+0+0 (secure 0 0 0) 0x7f86e407a090 con 0x7f86e80d0070 2024-09-19T07:54:14.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.831+0000 7f86f0318640 1 -- 172.21.15.16:0/1136012138 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f86cc06c2b0 msgr2=0x7f86cc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:14.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.831+0000 7f86f0318640 1 --2- 172.21.15.16:0/1136012138 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f86cc06c2b0 0x7f86cc06e770 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f86d8000f60 tx=0x7f86d8009040 comp rx=0 tx=0).stop 2024-09-19T07:54:14.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.831+0000 7f86f0318640 1 -- 172.21.15.16:0/1136012138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e80d0070 msgr2=0x7f86e81f7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:14.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.831+0000 7f86f0318640 1 --2- 172.21.15.16:0/1136012138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e80d0070 0x7f86e81f7100 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f86e40098c0 tx=0x7f86e40046c0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.831+0000 7f86f0318640 1 -- 172.21.15.16:0/1136012138 shutdown_connections 2024-09-19T07:54:14.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.831+0000 7f86f0318640 1 --2- 172.21.15.16:0/1136012138 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f86cc06c2b0 0x7f86cc06e770 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.831+0000 7f86f0318640 1 --2- 172.21.15.16:0/1136012138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86e815f0d0 0x7f86e80b98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.831+0000 7f86f0318640 1 --2- 172.21.15.16:0/1136012138 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86e80d4130 0x7f86e80b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.831+0000 7f86f0318640 1 --2- 172.21.15.16:0/1136012138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e80d0070 0x7f86e81f7100 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.831+0000 7f86f0318640 1 -- 172.21.15.16:0/1136012138 >> 172.21.15.16:0/1136012138 conn(0x7f86e809d760 msgr2=0x7f86e806e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:14.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.831+0000 7f86f0318640 1 -- 172.21.15.16:0/1136012138 shutdown_connections 2024-09-19T07:54:14.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.831+0000 7f86f0318640 1 -- 172.21.15.16:0/1136012138 wait complete. 2024-09-19T07:54:14.842 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2024-09-19T07:54:14.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.946+0000 7f83e8d9a640 1 Processor -- start 2024-09-19T07:54:14.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.946+0000 7f83e8d9a640 1 -- start start 2024-09-19T07:54:14.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.947+0000 7f83e8d9a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83e40c8d20 0x7f83e406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.947+0000 7f83e8d9a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83e4144100 0x7f83e40d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.947+0000 7f83e8d9a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83e40c7140 con 0x7f83e406d260 2024-09-19T07:54:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.947+0000 7f83e8d9a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f83e4144530 con 0x7f83e4144100 2024-09-19T07:54:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.947+0000 7f83e8d9a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f83e4141960 con 0x7f83e40c8d20 2024-09-19T07:54:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.947+0000 7f83e2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83e4144100 0x7f83e40d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.947+0000 7f83e2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83e4144100 0x7f83e40d1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54656/0 (socket says 172.21.15.16:54656) 2024-09-19T07:54:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.947+0000 7f83e2d9d640 1 -- 172.21.15.16:0/3241269022 learned_addr learned my addr 172.21.15.16:0/3241269022 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.947+0000 7f83e259c640 1 --2- 172.21.15.16:0/3241269022 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83e40c8d20 0x7f83e406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.947+0000 7f83e2d9d640 1 -- 172.21.15.16:0/3241269022 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83e40c8d20 msgr2=0x7f83e406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.947+0000 7f83e2d9d640 1 --2- 172.21.15.16:0/3241269022 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83e40c8d20 0x7f83e406cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.947+0000 7f83e2d9d640 1 -- 172.21.15.16:0/3241269022 >> v1:172.21.15.146:6789/0 conn(0x7f83e406d260 legacy=0x7f83e41439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.947+0000 7f83e2d9d640 1 -- 172.21.15.16:0/3241269022 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e40c9100 con 0x7f83e4144100 2024-09-19T07:54:14.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.948+0000 7f83e2d9d640 1 --2- 172.21.15.16:0/3241269022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83e4144100 0x7f83e40d1e50 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009e50 tx=0x7f83dc02a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9656ad47d2f653 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:14.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.948+0000 7f83e159a640 1 -- 172.21.15.16:0/3241269022 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc036070 con 0x7f83e4144100 2024-09-19T07:54:14.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.948+0000 7f83e159a640 1 -- 172.21.15.16:0/3241269022 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83dc031600 con 0x7f83e4144100 2024-09-19T07:54:14.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.948+0000 7f83e159a640 1 -- 172.21.15.16:0/3241269022 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc0319d0 con 0x7f83e4144100 2024-09-19T07:54:14.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.949+0000 7f83e8d9a640 1 -- 172.21.15.16:0/3241269022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83e4144100 msgr2=0x7f83e40d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:14.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.949+0000 7f83e8d9a640 1 --2- 172.21.15.16:0/3241269022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83e4144100 0x7f83e40d1e50 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009e50 tx=0x7f83dc02a5b0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.949+0000 7f83e8d9a640 1 -- 172.21.15.16:0/3241269022 shutdown_connections 2024-09-19T07:54:14.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.949+0000 7f83e8d9a640 1 --2- 172.21.15.16:0/3241269022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83e4144100 0x7f83e40d1e50 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.949+0000 7f83e8d9a640 1 --2- 172.21.15.16:0/3241269022 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83e40c8d20 0x7f83e406cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.949+0000 7f83e8d9a640 1 -- 172.21.15.16:0/3241269022 >> 172.21.15.16:0/3241269022 conn(0x7f83e40b3ed0 msgr2=0x7f83e40b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:14.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.949+0000 7f83e8d9a640 1 -- 172.21.15.16:0/3241269022 shutdown_connections 2024-09-19T07:54:14.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.949+0000 7f83e8d9a640 1 -- 172.21.15.16:0/3241269022 wait complete. 2024-09-19T07:54:14.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.950+0000 7f83e8d9a640 1 Processor -- start 2024-09-19T07:54:14.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.950+0000 7f83e8d9a640 1 -- start start 2024-09-19T07:54:14.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.951+0000 7f83e8d9a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83e406d260 0x7f83e40b7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:14.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.951+0000 7f83e8d9a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83e40c8d20 0x7f83e4140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:14.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.951+0000 7f83e8d9a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83e4144100 0x7f83e40d0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:14.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.951+0000 7f83e8d9a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83e406d6a0 con 0x7f83e4144100 2024-09-19T07:54:14.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.951+0000 7f83e8d9a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f83e4142200 con 0x7f83e406d260 2024-09-19T07:54:14.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.951+0000 7f83e8d9a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f83e4141bb0 con 0x7f83e40c8d20 2024-09-19T07:54:14.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.951+0000 7f83e2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83e4144100 0x7f83e40d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:14.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.951+0000 7f83e2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83e4144100 0x7f83e40d0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54668/0 (socket says 172.21.15.16:54668) 2024-09-19T07:54:14.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.951+0000 7f83e2d9d640 1 -- 172.21.15.16:0/1439677362 learned_addr learned my addr 172.21.15.16:0/1439677362 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:14.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.951+0000 7f83e259c640 1 --2- 172.21.15.16:0/1439677362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83e406d260 0x7f83e40b7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:14.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.951+0000 7f83e1d9b640 1 --2- 172.21.15.16:0/1439677362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83e40c8d20 0x7f83e4140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:14.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.952+0000 7f83e2d9d640 1 -- 172.21.15.16:0/1439677362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83e40c8d20 msgr2=0x7f83e4140940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:14.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.952+0000 7f83e2d9d640 1 --2- 172.21.15.16:0/1439677362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83e40c8d20 0x7f83e4140940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.952+0000 7f83e2d9d640 1 -- 172.21.15.16:0/1439677362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83e406d260 msgr2=0x7f83e40b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:14.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.952+0000 7f83e2d9d640 1 --2- 172.21.15.16:0/1439677362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83e406d260 0x7f83e40b7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:14.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.952+0000 7f83e2d9d640 1 -- 172.21.15.16:0/1439677362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e4140e80 con 0x7f83e4144100 2024-09-19T07:54:14.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.952+0000 7f83e2d9d640 1 --2- 172.21.15.16:0/1439677362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83e4144100 0x7f83e40d0020 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009e50 tx=0x7f83dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:14.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.952+0000 7f83d2ffd640 1 -- 172.21.15.16:0/1439677362 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc036050 con 0x7f83e4144100 2024-09-19T07:54:14.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.952+0000 7f83d2ffd640 1 -- 172.21.15.16:0/1439677362 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83dc042070 con 0x7f83e4144100 2024-09-19T07:54:14.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.952+0000 7f83e8d9a640 1 -- 172.21.15.16:0/1439677362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e40d0870 con 0x7f83e4144100 2024-09-19T07:54:14.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.952+0000 7f83d2ffd640 1 -- 172.21.15.16:0/1439677362 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc009510 con 0x7f83e4144100 2024-09-19T07:54:14.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.952+0000 7f83e8d9a640 1 -- 172.21.15.16:0/1439677362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e40b74e0 con 0x7f83e4144100 2024-09-19T07:54:14.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.953+0000 7f83e8d9a640 1 -- 172.21.15.16:0/1439677362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83e406cd20 con 0x7f83e4144100 2024-09-19T07:54:14.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.953+0000 7f83d2ffd640 1 -- 172.21.15.16:0/1439677362 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f83dc007bd0 con 0x7f83e4144100 2024-09-19T07:54:14.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.954+0000 7f83d2ffd640 1 --2- 172.21.15.16:0/1439677362 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f83c406c2b0 0x7f83c406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:14.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.954+0000 7f83d2ffd640 1 -- 172.21.15.16:0/1439677362 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7f83dc0abfa0 con 0x7f83e4144100 2024-09-19T07:54:14.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.954+0000 7f83e259c640 1 --2- 172.21.15.16:0/1439677362 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f83c406c2b0 0x7f83c406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:14.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.955+0000 7f83e259c640 1 --2- 172.21.15.16:0/1439677362 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f83c406c2b0 0x7f83c406e770 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f83d40079a0 tx=0x7f83d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:14.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:14.958+0000 7f83d2ffd640 1 -- 172.21.15.16:0/1439677362 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f83e40c7140 con 0x7f83e4144100 2024-09-19T07:54:15.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.136+0000 7f83e8d9a640 1 -- 172.21.15.16:0/1439677362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) -- 0x7f83e40690c0 con 0x7f83e4144100 2024-09-19T07:54:15.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.139+0000 7f83d2ffd640 1 -- 172.21.15.16:0/1439677362 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v49) ==== 107+0+0 (secure 0 0 0) 0x7f83dc07a2c0 con 0x7f83e4144100 2024-09-19T07:54:15.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.142+0000 7f83e8d9a640 1 -- 172.21.15.16:0/1439677362 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f83c406c2b0 msgr2=0x7f83c406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.142+0000 7f83e8d9a640 1 --2- 172.21.15.16:0/1439677362 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f83c406c2b0 0x7f83c406e770 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f83d40079a0 tx=0x7f83d4008040 comp rx=0 tx=0).stop 2024-09-19T07:54:15.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.142+0000 7f83e8d9a640 1 -- 172.21.15.16:0/1439677362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83e4144100 msgr2=0x7f83e40d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.142+0000 7f83e8d9a640 1 --2- 172.21.15.16:0/1439677362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83e4144100 0x7f83e40d0020 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009e50 tx=0x7f83dc004a40 comp rx=0 tx=0).stop 2024-09-19T07:54:15.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.142+0000 7f83e8d9a640 1 -- 172.21.15.16:0/1439677362 shutdown_connections 2024-09-19T07:54:15.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.142+0000 7f83e8d9a640 1 --2- 172.21.15.16:0/1439677362 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f83c406c2b0 0x7f83c406e770 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.142+0000 7f83e8d9a640 1 --2- 172.21.15.16:0/1439677362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83e4144100 0x7f83e40d0020 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.142+0000 7f83e8d9a640 1 --2- 172.21.15.16:0/1439677362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83e40c8d20 0x7f83e4140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.142+0000 7f83e8d9a640 1 --2- 172.21.15.16:0/1439677362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83e406d260 0x7f83e40b7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.142+0000 7f83e8d9a640 1 -- 172.21.15.16:0/1439677362 >> 172.21.15.16:0/1439677362 conn(0x7f83e40b3ed0 msgr2=0x7f83e4154ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:15.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.142+0000 7f83e8d9a640 1 -- 172.21.15.16:0/1439677362 shutdown_connections 2024-09-19T07:54:15.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.142+0000 7f83e8d9a640 1 -- 172.21.15.16:0/1439677362 wait complete. 2024-09-19T07:54:15.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2024-09-19T07:54:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.257+0000 7f5948677640 1 Processor -- start 2024-09-19T07:54:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.257+0000 7f5948677640 1 -- start start 2024-09-19T07:54:15.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.258+0000 7f5948677640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5940144100 0x7f59400d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.258+0000 7f5948677640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59400c7140 con 0x7f594006d430 2024-09-19T07:54:15.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.258+0000 7f5948677640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59400c9100 con 0x7f59400c8d20 2024-09-19T07:54:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.258+0000 7f5948677640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f5940141960 con 0x7f5940144100 2024-09-19T07:54:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.258+0000 7f5946c14640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5940144100 0x7f59400d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.258+0000 7f5945c12640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f594006d430 0x7f59401439f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42676/0 (socket says 172.21.15.16:42676) 2024-09-19T07:54:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.258+0000 7f5945c12640 1 -- 172.21.15.16:0/2988888523 learned_addr learned my addr 172.21.15.16:0/2988888523 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.258+0000 7f5946c14640 1 -- 172.21.15.16:0/2988888523 >> v1:172.21.15.146:6789/0 conn(0x7f59400c8d20 legacy=0x7f594006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.258+0000 7f5946c14640 1 -- 172.21.15.16:0/2988888523 >> v1:172.21.15.16:6789/0 conn(0x7f594006d430 legacy=0x7f59401439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.258+0000 7f5945411640 1 -- 172.21.15.16:0/2988888523 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 30797729 0 0) 0x7f5930001110 con 0x7f594006d430 2024-09-19T07:54:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.258+0000 7f5946c14640 1 -- 172.21.15.16:0/2988888523 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5940144760 con 0x7f5940144100 2024-09-19T07:54:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.259+0000 7f5946c14640 1 --2- 172.21.15.16:0/2988888523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940144100 0x7f59400d1e50 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5940141ae0 tx=0x7f593802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c108945230642c32 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.259+0000 7f5945411640 1 -- 172.21.15.16:0/2988888523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938031600 con 0x7f5940144100 2024-09-19T07:54:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.259+0000 7f5945411640 1 -- 172.21.15.16:0/2988888523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59380317a0 con 0x7f5940144100 2024-09-19T07:54:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.259+0000 7f5945411640 1 -- 172.21.15.16:0/2988888523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938031b70 con 0x7f5940144100 2024-09-19T07:54:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.260+0000 7f5948677640 1 -- 172.21.15.16:0/2988888523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940144100 msgr2=0x7f59400d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.260+0000 7f5948677640 1 --2- 172.21.15.16:0/2988888523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940144100 0x7f59400d1e50 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5940141ae0 tx=0x7f593802a5d0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.260+0000 7f5948677640 1 -- 172.21.15.16:0/2988888523 shutdown_connections 2024-09-19T07:54:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.260+0000 7f5948677640 1 --2- 172.21.15.16:0/2988888523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940144100 0x7f59400d1e50 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.260+0000 7f5948677640 1 -- 172.21.15.16:0/2988888523 >> 172.21.15.16:0/2988888523 conn(0x7f59400b3ed0 msgr2=0x7f59400b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.260+0000 7f5948677640 1 -- 172.21.15.16:0/2988888523 shutdown_connections 2024-09-19T07:54:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.260+0000 7f5948677640 1 -- 172.21.15.16:0/2988888523 wait complete. 2024-09-19T07:54:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.261+0000 7f5948677640 1 Processor -- start 2024-09-19T07:54:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.261+0000 7f5948677640 1 -- start start 2024-09-19T07:54:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.262+0000 7f5948677640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f594006d430 0x7f59401574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.262+0000 7f5948677640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59400c8d20 0x7f59400d1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.262+0000 7f5948677640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940144100 0x7f59400ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.262+0000 7f5948677640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5940141e90 con 0x7f5940144100 2024-09-19T07:54:15.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.262+0000 7f5948677640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5940142200 con 0x7f594006d430 2024-09-19T07:54:15.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.262+0000 7f5948677640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f59401436a0 con 0x7f59400c8d20 2024-09-19T07:54:15.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.262+0000 7f5946c14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940144100 0x7f59400ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.262+0000 7f5946c14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940144100 0x7f59400ceca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54680/0 (socket says 172.21.15.16:54680) 2024-09-19T07:54:15.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.262+0000 7f5946c14640 1 -- 172.21.15.16:0/198367690 learned_addr learned my addr 172.21.15.16:0/198367690 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:15.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.262+0000 7f5945c12640 1 --2- 172.21.15.16:0/198367690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59400c8d20 0x7f59400d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.262+0000 7f5946413640 1 --2- 172.21.15.16:0/198367690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f594006d430 0x7f59401574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.262+0000 7f5946c14640 1 -- 172.21.15.16:0/198367690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59400c8d20 msgr2=0x7f59400d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.262+0000 7f5946c14640 1 --2- 172.21.15.16:0/198367690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59400c8d20 0x7f59400d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.262+0000 7f5946c14640 1 -- 172.21.15.16:0/198367690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f594006d430 msgr2=0x7f59401574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.262+0000 7f5946c14640 1 --2- 172.21.15.16:0/198367690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f594006d430 0x7f59401574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.262+0000 7f5946c14640 1 -- 172.21.15.16:0/198367690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59400d0380 con 0x7f5940144100 2024-09-19T07:54:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.262+0000 7f5946c14640 1 --2- 172.21.15.16:0/198367690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940144100 0x7f59400ceca0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f5938009e80 tx=0x7f5938004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.263+0000 7f5936ffd640 1 -- 172.21.15.16:0/198367690 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593803f070 con 0x7f5940144100 2024-09-19T07:54:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.263+0000 7f5936ffd640 1 -- 172.21.15.16:0/198367690 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5938030be0 con 0x7f5940144100 2024-09-19T07:54:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.263+0000 7f5948677640 1 -- 172.21.15.16:0/198367690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59400d0660 con 0x7f5940144100 2024-09-19T07:54:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.263+0000 7f5936ffd640 1 -- 172.21.15.16:0/198367690 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938003990 con 0x7f5940144100 2024-09-19T07:54:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.263+0000 7f5948677640 1 -- 172.21.15.16:0/198367690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59400d0c20 con 0x7f5940144100 2024-09-19T07:54:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.264+0000 7f5948677640 1 -- 172.21.15.16:0/198367690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59401439f0 con 0x7f5940144100 2024-09-19T07:54:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.264+0000 7f5936ffd640 1 -- 172.21.15.16:0/198367690 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5938003b30 con 0x7f5940144100 2024-09-19T07:54:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.265+0000 7f5936ffd640 1 --2- 172.21.15.16:0/198367690 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f592406c2b0 0x7f592406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.265+0000 7f5936ffd640 1 -- 172.21.15.16:0/198367690 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7f59380abf30 con 0x7f5940144100 2024-09-19T07:54:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.265+0000 7f5946413640 1 --2- 172.21.15.16:0/198367690 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f592406c2b0 0x7f592406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.266+0000 7f5946413640 1 --2- 172.21.15.16:0/198367690 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f592406c2b0 0x7f592406e770 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f593c000f60 tx=0x7f593c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.269+0000 7f5936ffd640 1 -- 172.21.15.16:0/198367690 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f593807a960 con 0x7f5940144100 2024-09-19T07:54:15.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.447+0000 7f5948677640 1 -- 172.21.15.16:0/198367690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) -- 0x7f5940072290 con 0x7f5940144100 2024-09-19T07:54:15.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.450+0000 7f5936ffd640 1 -- 172.21.15.16:0/198367690 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v50) ==== 102+0+0 (secure 0 0 0) 0x7f593807a260 con 0x7f5940144100 2024-09-19T07:54:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.452+0000 7f5948677640 1 -- 172.21.15.16:0/198367690 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f592406c2b0 msgr2=0x7f592406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.452+0000 7f5948677640 1 --2- 172.21.15.16:0/198367690 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f592406c2b0 0x7f592406e770 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f593c000f60 tx=0x7f593c009040 comp rx=0 tx=0).stop 2024-09-19T07:54:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.452+0000 7f5948677640 1 -- 172.21.15.16:0/198367690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940144100 msgr2=0x7f59400ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.452+0000 7f5948677640 1 --2- 172.21.15.16:0/198367690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940144100 0x7f59400ceca0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f5938009e80 tx=0x7f5938004a40 comp rx=0 tx=0).stop 2024-09-19T07:54:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.452+0000 7f5948677640 1 -- 172.21.15.16:0/198367690 shutdown_connections 2024-09-19T07:54:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.452+0000 7f5948677640 1 --2- 172.21.15.16:0/198367690 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f592406c2b0 0x7f592406e770 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.452+0000 7f5948677640 1 --2- 172.21.15.16:0/198367690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940144100 0x7f59400ceca0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.452+0000 7f5948677640 1 --2- 172.21.15.16:0/198367690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f59400c8d20 0x7f59400d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.452+0000 7f5948677640 1 --2- 172.21.15.16:0/198367690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f594006d430 0x7f59401574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.452+0000 7f5948677640 1 -- 172.21.15.16:0/198367690 >> 172.21.15.16:0/198367690 conn(0x7f59400b3ed0 msgr2=0x7f594015feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.452+0000 7f5948677640 1 -- 172.21.15.16:0/198367690 shutdown_connections 2024-09-19T07:54:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.452+0000 7f5948677640 1 -- 172.21.15.16:0/198367690 wait complete. 2024-09-19T07:54:15.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T07:54:15.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.575+0000 7f9b898c9640 1 Processor -- start 2024-09-19T07:54:15.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.575+0000 7f9b898c9640 1 -- start start 2024-09-19T07:54:15.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.576+0000 7f9b898c9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b8415c3c0 0x7f9b840cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.576+0000 7f9b898c9640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9b8409b100 0x7f9b840d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.576+0000 7f9b898c9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b840cd030 con 0x7f9b840d0790 2024-09-19T07:54:15.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.576+0000 7f9b898c9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9b840cd310 con 0x7f9b8415c3c0 2024-09-19T07:54:15.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.576+0000 7f9b898c9640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f9b840cd5f0 con 0x7f9b8409b100 2024-09-19T07:54:15.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.576+0000 7f9b82ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b8415c3c0 0x7f9b840cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.576+0000 7f9b827fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9b840d0790 0x7f9b8409a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42688/0 (socket says 172.21.15.16:42688) 2024-09-19T07:54:15.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.576+0000 7f9b827fc640 1 -- 172.21.15.16:0/1965365903 learned_addr learned my addr 172.21.15.16:0/1965365903 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:15.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.576+0000 7f9b837fe640 1 --2- 172.21.15.16:0/1965365903 >> v2:172.21.15.146:3300/0 conn(0x7f9b8409b100 0x7f9b840d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.576+0000 7f9b82ffd640 1 -- 172.21.15.16:0/1965365903 >> v2:172.21.15.146:3300/0 conn(0x7f9b8409b100 msgr2=0x7f9b840d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:15.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.576+0000 7f9b82ffd640 1 --2- 172.21.15.16:0/1965365903 >> v2:172.21.15.146:3300/0 conn(0x7f9b8409b100 0x7f9b840d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.576+0000 7f9b82ffd640 1 -- 172.21.15.16:0/1965365903 >> v1:172.21.15.16:6789/0 conn(0x7f9b840d0790 legacy=0x7f9b8409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.576+0000 7f9b81ffb640 1 -- 172.21.15.16:0/1965365903 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2849303959 0 0) 0x7f9b840cd030 con 0x7f9b840d0790 2024-09-19T07:54:15.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.576+0000 7f9b82ffd640 1 -- 172.21.15.16:0/1965365903 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b840cd8d0 con 0x7f9b8415c3c0 2024-09-19T07:54:15.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.577+0000 7f9b837fe640 1 --2- 172.21.15.16:0/1965365903 >> v2:172.21.15.146:3300/0 conn(0x7f9b8409b100 0x7f9b840d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:15.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.577+0000 7f9b82ffd640 1 --2- 172.21.15.16:0/1965365903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8415c3c0 0x7f9b840cddf0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9b740097a0 tx=0x7f9b7402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bf29c927a7aad9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:15.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.577+0000 7f9b81ffb640 1 -- 172.21.15.16:0/1965365903 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74037070 con 0x7f9b8415c3c0 2024-09-19T07:54:15.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.577+0000 7f9b81ffb640 1 -- 172.21.15.16:0/1965365903 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b74032440 con 0x7f9b8415c3c0 2024-09-19T07:54:15.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.577+0000 7f9b81ffb640 1 -- 172.21.15.16:0/1965365903 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74032810 con 0x7f9b8415c3c0 2024-09-19T07:54:15.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.578+0000 7f9b898c9640 1 -- 172.21.15.16:0/1965365903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8415c3c0 msgr2=0x7f9b840cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.578+0000 7f9b898c9640 1 --2- 172.21.15.16:0/1965365903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8415c3c0 0x7f9b840cddf0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9b740097a0 tx=0x7f9b7402a1e0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.578+0000 7f9b898c9640 1 -- 172.21.15.16:0/1965365903 shutdown_connections 2024-09-19T07:54:15.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.578+0000 7f9b898c9640 1 --2- 172.21.15.16:0/1965365903 >> v2:172.21.15.146:3300/0 conn(0x7f9b8409b100 0x7f9b840d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.578+0000 7f9b898c9640 1 --2- 172.21.15.16:0/1965365903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8415c3c0 0x7f9b840cddf0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.578+0000 7f9b898c9640 1 -- 172.21.15.16:0/1965365903 >> 172.21.15.16:0/1965365903 conn(0x7f9b840b3ca0 msgr2=0x7f9b840b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:15.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.578+0000 7f9b898c9640 1 -- 172.21.15.16:0/1965365903 shutdown_connections 2024-09-19T07:54:15.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.578+0000 7f9b898c9640 1 -- 172.21.15.16:0/1965365903 wait complete. 2024-09-19T07:54:15.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.579+0000 7f9b898c9640 1 Processor -- start 2024-09-19T07:54:15.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.579+0000 7f9b898c9640 1 -- start start 2024-09-19T07:54:15.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.580+0000 7f9b898c9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b8409b100 0x7f9b8414c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.580+0000 7f9b898c9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b840d0790 0x7f9b8406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.580+0000 7f9b898c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8415c3c0 0x7f9b8415d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.580+0000 7f9b898c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8409bd30 con 0x7f9b8415c3c0 2024-09-19T07:54:15.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.580+0000 7f9b898c9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8409bbb0 con 0x7f9b840d0790 2024-09-19T07:54:15.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.580+0000 7f9b898c9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9b840ccc50 con 0x7f9b8409b100 2024-09-19T07:54:15.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.580+0000 7f9b837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8415c3c0 0x7f9b8415d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.580+0000 7f9b837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8415c3c0 0x7f9b8415d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54704/0 (socket says 172.21.15.16:54704) 2024-09-19T07:54:15.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.580+0000 7f9b837fe640 1 -- 172.21.15.16:0/2219139998 learned_addr learned my addr 172.21.15.16:0/2219139998 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:15.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.580+0000 7f9b827fc640 1 --2- 172.21.15.16:0/2219139998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b840d0790 0x7f9b8406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.580+0000 7f9b82ffd640 1 --2- 172.21.15.16:0/2219139998 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b8409b100 0x7f9b8414c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.581+0000 7f9b837fe640 1 -- 172.21.15.16:0/2219139998 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b8409b100 msgr2=0x7f9b8414c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.581+0000 7f9b837fe640 1 --2- 172.21.15.16:0/2219139998 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b8409b100 0x7f9b8414c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.581+0000 7f9b837fe640 1 -- 172.21.15.16:0/2219139998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b840d0790 msgr2=0x7f9b8406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.581+0000 7f9b837fe640 1 --2- 172.21.15.16:0/2219139998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b840d0790 0x7f9b8406dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.581+0000 7f9b837fe640 1 -- 172.21.15.16:0/2219139998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8414cb30 con 0x7f9b8415c3c0 2024-09-19T07:54:15.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.581+0000 7f9b837fe640 1 --2- 172.21.15.16:0/2219139998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8415c3c0 0x7f9b8415d3a0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00cea0 tx=0x7f9b7c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:15.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.581+0000 7f9b6bfff640 1 -- 172.21.15.16:0/2219139998 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c003b90 con 0x7f9b8415c3c0 2024-09-19T07:54:15.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.581+0000 7f9b6bfff640 1 -- 172.21.15.16:0/2219139998 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b7c003d50 con 0x7f9b8415c3c0 2024-09-19T07:54:15.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.581+0000 7f9b898c9640 1 -- 172.21.15.16:0/2219139998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b84155b80 con 0x7f9b8415c3c0 2024-09-19T07:54:15.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.581+0000 7f9b6bfff640 1 -- 172.21.15.16:0/2219139998 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c004260 con 0x7f9b8415c3c0 2024-09-19T07:54:15.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.581+0000 7f9b898c9640 1 -- 172.21.15.16:0/2219139998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b84155e60 con 0x7f9b8415c3c0 2024-09-19T07:54:15.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.582+0000 7f9b898c9640 1 -- 172.21.15.16:0/2219139998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b840cddf0 con 0x7f9b8415c3c0 2024-09-19T07:54:15.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.583+0000 7f9b6bfff640 1 -- 172.21.15.16:0/2219139998 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9b7c004400 con 0x7f9b8415c3c0 2024-09-19T07:54:15.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.583+0000 7f9b6bfff640 1 --2- 172.21.15.16:0/2219139998 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b7006c2b0 0x7f9b7006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.583+0000 7f9b6bfff640 1 -- 172.21.15.16:0/2219139998 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7f9b7c08bbe0 con 0x7f9b8415c3c0 2024-09-19T07:54:15.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.583+0000 7f9b82ffd640 1 --2- 172.21.15.16:0/2219139998 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b7006c2b0 0x7f9b7006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.584+0000 7f9b82ffd640 1 --2- 172.21.15.16:0/2219139998 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b7006c2b0 0x7f9b7006e770 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9b74034000 tx=0x7f9b7402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:15.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.587+0000 7f9b6bfff640 1 -- 172.21.15.16:0/2219139998 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9b8414cb30 con 0x7f9b8415c3c0 2024-09-19T07:54:15.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.763+0000 7f9b898c9640 1 -- 172.21.15.16:0/2219139998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f9b840c8ab0 con 0x7f9b8415c3c0 2024-09-19T07:54:15.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.766+0000 7f9b6bfff640 1 -- 172.21.15.16:0/2219139998 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v51) ==== 104+0+0 (secure 0 0 0) 0x7f9b7c05a550 con 0x7f9b8415c3c0 2024-09-19T07:54:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.769+0000 7f9b898c9640 1 -- 172.21.15.16:0/2219139998 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b7006c2b0 msgr2=0x7f9b7006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.769+0000 7f9b898c9640 1 --2- 172.21.15.16:0/2219139998 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b7006c2b0 0x7f9b7006e770 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9b74034000 tx=0x7f9b7402e000 comp rx=0 tx=0).stop 2024-09-19T07:54:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.769+0000 7f9b898c9640 1 -- 172.21.15.16:0/2219139998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8415c3c0 msgr2=0x7f9b8415d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.769+0000 7f9b898c9640 1 --2- 172.21.15.16:0/2219139998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8415c3c0 0x7f9b8415d3a0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00cea0 tx=0x7f9b7c00cf90 comp rx=0 tx=0).stop 2024-09-19T07:54:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.769+0000 7f9b898c9640 1 -- 172.21.15.16:0/2219139998 shutdown_connections 2024-09-19T07:54:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.769+0000 7f9b898c9640 1 --2- 172.21.15.16:0/2219139998 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9b7006c2b0 0x7f9b7006e770 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.769+0000 7f9b898c9640 1 --2- 172.21.15.16:0/2219139998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8415c3c0 0x7f9b8415d3a0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.769+0000 7f9b898c9640 1 --2- 172.21.15.16:0/2219139998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b840d0790 0x7f9b8406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.769+0000 7f9b898c9640 1 --2- 172.21.15.16:0/2219139998 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b8409b100 0x7f9b8414c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.769+0000 7f9b898c9640 1 -- 172.21.15.16:0/2219139998 >> 172.21.15.16:0/2219139998 conn(0x7f9b840b3ca0 msgr2=0x7f9b840b62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.769+0000 7f9b898c9640 1 -- 172.21.15.16:0/2219139998 shutdown_connections 2024-09-19T07:54:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.769+0000 7f9b898c9640 1 -- 172.21.15.16:0/2219139998 wait complete. 2024-09-19T07:54:15.781 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2024-09-19T07:54:15.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.885+0000 7fa6c0de4640 1 Processor -- start 2024-09-19T07:54:15.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.885+0000 7fa6c0de4640 1 -- start start 2024-09-19T07:54:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.886+0000 7fa6c0de4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6bc15e680 0x7fa6bc0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.886+0000 7fa6c0de4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa6bc0d0010 0x7fa6bc0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.886+0000 7fa6c0de4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6bc0d1880 con 0x7fa6bc15e2a0 2024-09-19T07:54:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.886+0000 7fa6c0de4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa6bc0d1b60 con 0x7fa6bc0d0010 2024-09-19T07:54:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.886+0000 7fa6c0de4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa6bc0cf2d0 con 0x7fa6bc15e680 2024-09-19T07:54:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.886+0000 7fa6bad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa6bc0d0010 0x7fa6bc0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.886+0000 7fa6ba59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa6bc15e2a0 0x7fa6bc06d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42690/0 (socket says 172.21.15.16:42690) 2024-09-19T07:54:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.886+0000 7fa6ba59c640 1 -- 172.21.15.16:0/930731554 learned_addr learned my addr 172.21.15.16:0/930731554 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.887+0000 7fa6b9d9b640 1 --2- 172.21.15.16:0/930731554 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6bc15e680 0x7fa6bc0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.887+0000 7fa6bad9d640 1 -- 172.21.15.16:0/930731554 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6bc15e680 msgr2=0x7fa6bc0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.887+0000 7fa6bad9d640 1 --2- 172.21.15.16:0/930731554 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6bc15e680 0x7fa6bc0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.887+0000 7fa6bad9d640 1 -- 172.21.15.16:0/930731554 >> v1:172.21.15.16:6789/0 conn(0x7fa6bc15e2a0 legacy=0x7fa6bc06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.887+0000 7fa6bad9d640 1 -- 172.21.15.16:0/930731554 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6bc0cf5b0 con 0x7fa6bc0d0010 2024-09-19T07:54:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.887+0000 7fa6b959a640 1 -- 172.21.15.16:0/930731554 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 10769550 0 0) 0x7fa6bc0d1880 con 0x7fa6bc15e2a0 2024-09-19T07:54:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.887+0000 7fa6b9d9b640 1 --2- 172.21.15.16:0/930731554 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6bc15e680 0x7fa6bc0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:15.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.887+0000 7fa6bad9d640 1 --2- 172.21.15.16:0/930731554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6bc0d0010 0x7fa6bc0d4370 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4009d90 tx=0x7fa6b402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149dd2f3b8922d55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:15.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.887+0000 7fa6b959a640 1 -- 172.21.15.16:0/930731554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4031600 con 0x7fa6bc0d0010 2024-09-19T07:54:15.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.888+0000 7fa6b959a640 1 -- 172.21.15.16:0/930731554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b40317a0 con 0x7fa6bc0d0010 2024-09-19T07:54:15.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.888+0000 7fa6b959a640 1 -- 172.21.15.16:0/930731554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4031b70 con 0x7fa6bc0d0010 2024-09-19T07:54:15.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.888+0000 7fa6c0de4640 1 -- 172.21.15.16:0/930731554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6bc0d0010 msgr2=0x7fa6bc0d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.888+0000 7fa6c0de4640 1 --2- 172.21.15.16:0/930731554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6bc0d0010 0x7fa6bc0d4370 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4009d90 tx=0x7fa6b402a5d0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.888+0000 7fa6c0de4640 1 -- 172.21.15.16:0/930731554 shutdown_connections 2024-09-19T07:54:15.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.888+0000 7fa6c0de4640 1 --2- 172.21.15.16:0/930731554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6bc0d0010 0x7fa6bc0d4370 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.888+0000 7fa6c0de4640 1 --2- 172.21.15.16:0/930731554 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6bc15e680 0x7fa6bc0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.888+0000 7fa6c0de4640 1 -- 172.21.15.16:0/930731554 >> 172.21.15.16:0/930731554 conn(0x7fa6bc09d760 msgr2=0x7fa6bc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:15.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.888+0000 7fa6c0de4640 1 -- 172.21.15.16:0/930731554 shutdown_connections 2024-09-19T07:54:15.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.888+0000 7fa6c0de4640 1 -- 172.21.15.16:0/930731554 wait complete. 2024-09-19T07:54:15.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.890+0000 7fa6c0de4640 1 Processor -- start 2024-09-19T07:54:15.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.890+0000 7fa6c0de4640 1 -- start start 2024-09-19T07:54:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.890+0000 7fa6c0de4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6bc0d0010 0x7fa6bc0b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.890+0000 7fa6c0de4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6bc15e2a0 0x7fa6bc0ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.890+0000 7fa6c0de4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6bc15e680 0x7fa6bc0ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.890+0000 7fa6c0de4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6bc06dcb0 con 0x7fa6bc0d0010 2024-09-19T07:54:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.890+0000 7fa6c0de4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa6bc06db30 con 0x7fa6bc15e680 2024-09-19T07:54:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.890+0000 7fa6c0de4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa6bc0d48b0 con 0x7fa6bc15e2a0 2024-09-19T07:54:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.890+0000 7fa6ba59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6bc0d0010 0x7fa6bc0b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.890+0000 7fa6ba59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6bc0d0010 0x7fa6bc0b7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54724/0 (socket says 172.21.15.16:54724) 2024-09-19T07:54:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.890+0000 7fa6ba59c640 1 -- 172.21.15.16:0/389125638 learned_addr learned my addr 172.21.15.16:0/389125638 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.891+0000 7fa6b9d9b640 1 --2- 172.21.15.16:0/389125638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6bc15e2a0 0x7fa6bc0ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.891+0000 7fa6bad9d640 1 --2- 172.21.15.16:0/389125638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6bc15e680 0x7fa6bc0ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.891+0000 7fa6ba59c640 1 -- 172.21.15.16:0/389125638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6bc15e2a0 msgr2=0x7fa6bc0ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.891+0000 7fa6ba59c640 1 --2- 172.21.15.16:0/389125638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6bc15e2a0 0x7fa6bc0ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.891+0000 7fa6ba59c640 1 -- 172.21.15.16:0/389125638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6bc15e680 msgr2=0x7fa6bc0ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.891+0000 7fa6ba59c640 1 --2- 172.21.15.16:0/389125638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6bc15e680 0x7fa6bc0ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.891+0000 7fa6ba59c640 1 -- 172.21.15.16:0/389125638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6bc0cb010 con 0x7fa6bc0d0010 2024-09-19T07:54:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.891+0000 7fa6bad9d640 1 --2- 172.21.15.16:0/389125638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6bc15e680 0x7fa6bc0ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.891+0000 7fa6ba59c640 1 --2- 172.21.15.16:0/389125638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6bc0d0010 0x7fa6bc0b7ae0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00c500 tx=0x7fa6ac00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.891+0000 7fa6aaffd640 1 -- 172.21.15.16:0/389125638 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac018070 con 0x7fa6bc0d0010 2024-09-19T07:54:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.891+0000 7fa6aaffd640 1 -- 172.21.15.16:0/389125638 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6ac007580 con 0x7fa6bc0d0010 2024-09-19T07:54:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.891+0000 7fa6c0de4640 1 -- 172.21.15.16:0/389125638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6bc0cbae0 con 0x7fa6bc0d0010 2024-09-19T07:54:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.891+0000 7fa6aaffd640 1 -- 172.21.15.16:0/389125638 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac004a20 con 0x7fa6bc0d0010 2024-09-19T07:54:15.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.891+0000 7fa6c0de4640 1 -- 172.21.15.16:0/389125638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6bc0cc0a0 con 0x7fa6bc0d0010 2024-09-19T07:54:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.892+0000 7fa6c0de4640 1 -- 172.21.15.16:0/389125638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6bc0cc2a0 con 0x7fa6bc0d0010 2024-09-19T07:54:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.893+0000 7fa6aaffd640 1 -- 172.21.15.16:0/389125638 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa6ac01b070 con 0x7fa6bc0d0010 2024-09-19T07:54:15.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.893+0000 7fa6aaffd640 1 --2- 172.21.15.16:0/389125638 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa69c06c2b0 0x7fa69c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.894+0000 7fa6aaffd640 1 -- 172.21.15.16:0/389125638 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7fa6ac08a4f0 con 0x7fa6bc0d0010 2024-09-19T07:54:15.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.894+0000 7fa6b9d9b640 1 --2- 172.21.15.16:0/389125638 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa69c06c2b0 0x7fa69c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.894+0000 7fa6b9d9b640 1 --2- 172.21.15.16:0/389125638 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa69c06c2b0 0x7fa69c06e770 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0c7980 tx=0x7fa6b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:15.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:15.898+0000 7fa6aaffd640 1 -- 172.21.15.16:0/389125638 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa6ac058810 con 0x7fa6bc0d0010 2024-09-19T07:54:16.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.075+0000 7fa6c0de4640 1 -- 172.21.15.16:0/389125638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) -- 0x7fa6bc06cf90 con 0x7fa6bc0d0010 2024-09-19T07:54:16.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.078+0000 7fa6aaffd640 1 -- 172.21.15.16:0/389125638 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v52) ==== 105+0+0 (secure 0 0 0) 0x7fa6ac05cb10 con 0x7fa6bc0d0010 2024-09-19T07:54:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.080+0000 7fa6c0de4640 1 -- 172.21.15.16:0/389125638 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa69c06c2b0 msgr2=0x7fa69c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.080+0000 7fa6c0de4640 1 --2- 172.21.15.16:0/389125638 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa69c06c2b0 0x7fa69c06e770 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0c7980 tx=0x7fa6b0008040 comp rx=0 tx=0).stop 2024-09-19T07:54:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.080+0000 7fa6c0de4640 1 -- 172.21.15.16:0/389125638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6bc0d0010 msgr2=0x7fa6bc0b7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.080+0000 7fa6c0de4640 1 --2- 172.21.15.16:0/389125638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6bc0d0010 0x7fa6bc0b7ae0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00c500 tx=0x7fa6ac00c9c0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.080+0000 7fa6c0de4640 1 -- 172.21.15.16:0/389125638 shutdown_connections 2024-09-19T07:54:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.080+0000 7fa6c0de4640 1 --2- 172.21.15.16:0/389125638 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa69c06c2b0 0x7fa69c06e770 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.080+0000 7fa6c0de4640 1 --2- 172.21.15.16:0/389125638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6bc15e680 0x7fa6bc0ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.080+0000 7fa6c0de4640 1 --2- 172.21.15.16:0/389125638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6bc15e2a0 0x7fa6bc0ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.080+0000 7fa6c0de4640 1 --2- 172.21.15.16:0/389125638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6bc0d0010 0x7fa6bc0b7ae0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.080+0000 7fa6c0de4640 1 -- 172.21.15.16:0/389125638 >> 172.21.15.16:0/389125638 conn(0x7fa6bc09d760 msgr2=0x7fa6bc0d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:16.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.080+0000 7fa6c0de4640 1 -- 172.21.15.16:0/389125638 shutdown_connections 2024-09-19T07:54:16.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.080+0000 7fa6c0de4640 1 -- 172.21.15.16:0/389125638 wait complete. 2024-09-19T07:54:16.091 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin' 2024-09-19T07:54:16.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.193+0000 7f6978d86640 1 Processor -- start 2024-09-19T07:54:16.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.193+0000 7f6978d86640 1 -- start start 2024-09-19T07:54:16.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.194+0000 7f6978d86640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6974161eb0 0x7f69741622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.194+0000 7f6978d86640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69741425e0 0x7f6974142a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.194+0000 7f6978d86640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6974162b70 con 0x7f697406cd20 2024-09-19T07:54:16.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.194+0000 7f6978d86640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f6974162e50 con 0x7f6974161eb0 2024-09-19T07:54:16.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.194+0000 7f6978d86640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6974142150 con 0x7f69741425e0 2024-09-19T07:54:16.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.194+0000 7f6971d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6974161eb0 0x7f69741622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.194+0000 7f6971d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6974161eb0 0x7f69741622f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54732/0 (socket says 172.21.15.16:54732) 2024-09-19T07:54:16.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.194+0000 7f6971d9b640 1 -- 172.21.15.16:0/3166263092 learned_addr learned my addr 172.21.15.16:0/3166263092 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:16.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.194+0000 7f6972d9d640 1 --2- 172.21.15.16:0/3166263092 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69741425e0 0x7f6974142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.194+0000 7f6971d9b640 1 -- 172.21.15.16:0/3166263092 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69741425e0 msgr2=0x7f6974142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:16.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.194+0000 7f6971d9b640 1 --2- 172.21.15.16:0/3166263092 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69741425e0 0x7f6974142a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.194+0000 7f6971d9b640 1 -- 172.21.15.16:0/3166263092 >> v1:172.21.15.146:6789/0 conn(0x7f697406cd20 legacy=0x7f697406d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:16.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.194+0000 7f6971d9b640 1 -- 172.21.15.16:0/3166263092 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6974141b00 con 0x7f6974161eb0 2024-09-19T07:54:16.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.195+0000 7f6971d9b640 1 --2- 172.21.15.16:0/3166263092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6974161eb0 0x7f69741622f0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f695c009870 tx=0x7f695c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e7ff10a497eff2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:16.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.195+0000 7f697159a640 1 -- 172.21.15.16:0/3166263092 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c037070 con 0x7f6974161eb0 2024-09-19T07:54:16.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.195+0000 7f697159a640 1 -- 172.21.15.16:0/3166263092 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f695c032440 con 0x7f6974161eb0 2024-09-19T07:54:16.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.195+0000 7f697159a640 1 -- 172.21.15.16:0/3166263092 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c032810 con 0x7f6974161eb0 2024-09-19T07:54:16.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.196+0000 7f6978d86640 1 -- 172.21.15.16:0/3166263092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6974161eb0 msgr2=0x7f69741622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:16.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.196+0000 7f6978d86640 1 --2- 172.21.15.16:0/3166263092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6974161eb0 0x7f69741622f0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f695c009870 tx=0x7f695c02a1c0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.196+0000 7f6978d86640 1 -- 172.21.15.16:0/3166263092 shutdown_connections 2024-09-19T07:54:16.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.196+0000 7f6978d86640 1 --2- 172.21.15.16:0/3166263092 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69741425e0 0x7f6974142a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.196+0000 7f6978d86640 1 --2- 172.21.15.16:0/3166263092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6974161eb0 0x7f69741622f0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.196+0000 7f6978d86640 1 -- 172.21.15.16:0/3166263092 >> 172.21.15.16:0/3166263092 conn(0x7f69740b3ed0 msgr2=0x7f69740b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:16.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.196+0000 7f6978d86640 1 -- 172.21.15.16:0/3166263092 shutdown_connections 2024-09-19T07:54:16.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.196+0000 7f6978d86640 1 -- 172.21.15.16:0/3166263092 wait complete. 2024-09-19T07:54:16.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.197+0000 7f6978d86640 1 Processor -- start 2024-09-19T07:54:16.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.197+0000 7f6978d86640 1 -- start start 2024-09-19T07:54:16.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.198+0000 7f6978d86640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f697406cd20 0x7f69740cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.198+0000 7f6978d86640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69741425e0 0x7f69740b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.198+0000 7f6978d86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6974161eb0 0x7f69740b7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.198+0000 7f6978d86640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6974141600 con 0x7f6974161eb0 2024-09-19T07:54:16.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.198+0000 7f6978d86640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6974162830 con 0x7f697406cd20 2024-09-19T07:54:16.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.198+0000 7f6978d86640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6974142fb0 con 0x7f69741425e0 2024-09-19T07:54:16.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.198+0000 7f6972d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6974161eb0 0x7f69740b7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.198+0000 7f6972d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6974161eb0 0x7f69740b7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54734/0 (socket says 172.21.15.16:54734) 2024-09-19T07:54:16.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.198+0000 7f6972d9d640 1 -- 172.21.15.16:0/2735231649 learned_addr learned my addr 172.21.15.16:0/2735231649 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:16.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.198+0000 7f697259c640 1 --2- 172.21.15.16:0/2735231649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f697406cd20 0x7f69740cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.198+0000 7f6971d9b640 1 --2- 172.21.15.16:0/2735231649 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69741425e0 0x7f69740b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.199+0000 7f6972d9d640 1 -- 172.21.15.16:0/2735231649 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69741425e0 msgr2=0x7f69740b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:16.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.199+0000 7f6972d9d640 1 --2- 172.21.15.16:0/2735231649 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69741425e0 0x7f69740b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.199+0000 7f6972d9d640 1 -- 172.21.15.16:0/2735231649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f697406cd20 msgr2=0x7f69740cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:16.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.199+0000 7f6972d9d640 1 --2- 172.21.15.16:0/2735231649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f697406cd20 0x7f69740cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.199+0000 7f6972d9d640 1 -- 172.21.15.16:0/2735231649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69740b55b0 con 0x7f6974161eb0 2024-09-19T07:54:16.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.199+0000 7f6972d9d640 1 --2- 172.21.15.16:0/2735231649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6974161eb0 0x7f69740b7a80 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f6964007e40 tx=0x7f696400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:16.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.199+0000 7f696affd640 1 -- 172.21.15.16:0/2735231649 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964014620 con 0x7f6974161eb0 2024-09-19T07:54:16.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.199+0000 7f696affd640 1 -- 172.21.15.16:0/2735231649 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6964019070 con 0x7f6974161eb0 2024-09-19T07:54:16.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.199+0000 7f6978d86640 1 -- 172.21.15.16:0/2735231649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69740b5890 con 0x7f6974161eb0 2024-09-19T07:54:16.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.199+0000 7f696affd640 1 -- 172.21.15.16:0/2735231649 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964014a20 con 0x7f6974161eb0 2024-09-19T07:54:16.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.199+0000 7f6978d86640 1 -- 172.21.15.16:0/2735231649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69740cc1e0 con 0x7f6974161eb0 2024-09-19T07:54:16.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.200+0000 7f6978d86640 1 -- 172.21.15.16:0/2735231649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6974068ca0 con 0x7f6974161eb0 2024-09-19T07:54:16.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.201+0000 7f696affd640 1 -- 172.21.15.16:0/2735231649 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6964014bc0 con 0x7f6974161eb0 2024-09-19T07:54:16.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.201+0000 7f696affd640 1 --2- 172.21.15.16:0/2735231649 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f695406c2b0 0x7f695406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.201+0000 7f696affd640 1 -- 172.21.15.16:0/2735231649 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7f696408b9c0 con 0x7f6974161eb0 2024-09-19T07:54:16.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.201+0000 7f697259c640 1 --2- 172.21.15.16:0/2735231649 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f695406c2b0 0x7f695406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.202+0000 7f697259c640 1 --2- 172.21.15.16:0/2735231649 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f695406c2b0 0x7f695406e770 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f696c001780 tx=0x7f696c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:16.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.205+0000 7f696affd640 1 -- 172.21.15.16:0/2735231649 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6974068ca0 con 0x7f6974161eb0 2024-09-19T07:54:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.449+0000 7f6978d86640 1 -- 172.21.15.16:0/2735231649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]} v 0) -- 0x7f6974162650 con 0x7f6974161eb0 2024-09-19T07:54:16.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.512+0000 7f696affd640 1 -- 172.21.15.16:0/2735231649 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]}]=0 v496) ==== 143+0+0 (secure 0 0 0) 0x7f6974162650 con 0x7f6974161eb0 2024-09-19T07:54:16.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.514+0000 7f6978d86640 1 -- 172.21.15.16:0/2735231649 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f695406c2b0 msgr2=0x7f695406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:16.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.514+0000 7f6978d86640 1 --2- 172.21.15.16:0/2735231649 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f695406c2b0 0x7f695406e770 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f696c001780 tx=0x7f696c009040 comp rx=0 tx=0).stop 2024-09-19T07:54:16.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.514+0000 7f6978d86640 1 -- 172.21.15.16:0/2735231649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6974161eb0 msgr2=0x7f69740b7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:16.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.514+0000 7f6978d86640 1 --2- 172.21.15.16:0/2735231649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6974161eb0 0x7f69740b7a80 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f6964007e40 tx=0x7f696400e4b0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.514+0000 7f6978d86640 1 -- 172.21.15.16:0/2735231649 shutdown_connections 2024-09-19T07:54:16.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.514+0000 7f6978d86640 1 --2- 172.21.15.16:0/2735231649 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f695406c2b0 0x7f695406e770 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.514+0000 7f6978d86640 1 --2- 172.21.15.16:0/2735231649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6974161eb0 0x7f69740b7a80 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.514+0000 7f6978d86640 1 --2- 172.21.15.16:0/2735231649 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69741425e0 0x7f69740b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.514+0000 7f6978d86640 1 --2- 172.21.15.16:0/2735231649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f697406cd20 0x7f69740cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.514+0000 7f6978d86640 1 -- 172.21.15.16:0/2735231649 >> 172.21.15.16:0/2735231649 conn(0x7f69740b3ed0 msgr2=0x7f69740d1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:16.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.514+0000 7f6978d86640 1 -- 172.21.15.16:0/2735231649 shutdown_connections 2024-09-19T07:54:16.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.514+0000 7f6978d86640 1 -- 172.21.15.16:0/2735231649 wait complete. 2024-09-19T07:54:16.526 INFO:tasks.cephfs_test_runner:test_client_pin (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2024-09-19T07:54:16.526 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin_mincaps (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-19T07:54:16.527 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:54:16.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.630+0000 7fb2ad5e8640 1 Processor -- start 2024-09-19T07:54:16.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.630+0000 7fb2ad5e8640 1 -- start start 2024-09-19T07:54:16.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.631+0000 7fb2ad5e8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2a815e680 0x7fb2a80cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.631+0000 7fb2ad5e8640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb2a80d0010 0x7fb2a80d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.631+0000 7fb2ad5e8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2a80d1880 con 0x7fb2a815e2a0 2024-09-19T07:54:16.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.631+0000 7fb2ad5e8640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb2a80d1b60 con 0x7fb2a80d0010 2024-09-19T07:54:16.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.631+0000 7fb2ad5e8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a80cf2d0 con 0x7fb2a815e680 2024-09-19T07:54:16.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.631+0000 7fb2a6ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb2a815e2a0 0x7fb2a806d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42698/0 (socket says 172.21.15.16:42698) 2024-09-19T07:54:16.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.631+0000 7fb2a6ffd640 1 -- 172.21.15.16:0/442454986 learned_addr learned my addr 172.21.15.16:0/442454986 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:16.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.631+0000 7fb2a5ffb640 1 -- 172.21.15.16:0/442454986 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1712713642 0 0) 0x7fb2a80d1880 con 0x7fb2a815e2a0 2024-09-19T07:54:16.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.631+0000 7fb2a5ffb640 1 -- 172.21.15.16:0/442454986 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb290003520 con 0x7fb2a815e2a0 2024-09-19T07:54:16.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.631+0000 7fb2a67fc640 1 --2- 172.21.15.16:0/442454986 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2a815e680 0x7fb2a80cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.631+0000 7fb2a77fe640 1 --2- 172.21.15.16:0/442454986 >> v2:172.21.15.146:3300/0 conn(0x7fb2a80d0010 0x7fb2a80d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a5ffb640 1 -- 172.21.15.16:0/442454986 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3863422497 0 0) 0x7fb290003520 con 0x7fb2a815e2a0 2024-09-19T07:54:16.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a5ffb640 1 -- 172.21.15.16:0/442454986 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2a815e680 msgr2=0x7fb2a80cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:16.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a5ffb640 1 --2- 172.21.15.16:0/442454986 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2a815e680 0x7fb2a80cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a5ffb640 1 -- 172.21.15.16:0/442454986 >> v2:172.21.15.146:3300/0 conn(0x7fb2a80d0010 msgr2=0x7fb2a80d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:16.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a5ffb640 1 --2- 172.21.15.16:0/442454986 >> v2:172.21.15.146:3300/0 conn(0x7fb2a80d0010 0x7fb2a80d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a5ffb640 1 -- 172.21.15.16:0/442454986 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a80cf5b0 con 0x7fb2a815e2a0 2024-09-19T07:54:16.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a5ffb640 1 -- 172.21.15.16:0/442454986 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fb298001670 con 0x7fb2a815e2a0 2024-09-19T07:54:16.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a5ffb640 1 -- 172.21.15.16:0/442454986 >> v1:172.21.15.16:6789/0 conn(0x7fb2a815e2a0 legacy=0x7fb2a806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:16.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a5ffb640 1 --2- 172.21.15.16:0/442454986 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2a815e680 0x7fb2900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a5ffb640 1 --2- 172.21.15.16:0/442454986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb290003f30 0x7fb290004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a5ffb640 1 --2- 172.21.15.16:0/442454986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb290004d40 0x7fb2900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a5ffb640 1 -- 172.21.15.16:0/442454986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a80d1b60 con 0x7fb290004d40 2024-09-19T07:54:16.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a5ffb640 1 -- 172.21.15.16:0/442454986 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a80cf2d0 con 0x7fb290003f30 2024-09-19T07:54:16.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a5ffb640 1 -- 172.21.15.16:0/442454986 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb29000ba10 con 0x7fb2a815e680 2024-09-19T07:54:16.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a77fe640 1 --2- 172.21.15.16:0/442454986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb290004d40 0x7fb2900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a6ffd640 1 --2- 172.21.15.16:0/442454986 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2a815e680 0x7fb2900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a67fc640 1 --2- 172.21.15.16:0/442454986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb290003f30 0x7fb290004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a77fe640 1 -- 172.21.15.16:0/442454986 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2a815e680 msgr2=0x7fb2900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:16.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a77fe640 1 --2- 172.21.15.16:0/442454986 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2a815e680 0x7fb2900039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a77fe640 1 -- 172.21.15.16:0/442454986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb290003f30 msgr2=0x7fb290004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:16.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a77fe640 1 --2- 172.21.15.16:0/442454986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb290003f30 0x7fb290004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.632+0000 7fb2a77fe640 1 -- 172.21.15.16:0/442454986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb290003520 con 0x7fb290004d40 2024-09-19T07:54:16.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.633+0000 7fb2a77fe640 1 --2- 172.21.15.16:0/442454986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb290004d40 0x7fb2900051c0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000e170 tx=0x7fb290005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112a335a8e7a7aeb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:16.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.633+0000 7fb2a5ffb640 1 -- 172.21.15.16:0/442454986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0034020 con 0x7fb290004d40 2024-09-19T07:54:16.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.633+0000 7fb2a5ffb640 1 -- 172.21.15.16:0/442454986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a0030430 con 0x7fb290004d40 2024-09-19T07:54:16.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.633+0000 7fb2a5ffb640 1 -- 172.21.15.16:0/442454986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0009c10 con 0x7fb290004d40 2024-09-19T07:54:16.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.634+0000 7fb2ad5e8640 1 -- 172.21.15.16:0/442454986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb290004d40 msgr2=0x7fb2900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:16.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.634+0000 7fb2ad5e8640 1 --2- 172.21.15.16:0/442454986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb290004d40 0x7fb2900051c0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000e170 tx=0x7fb290005830 comp rx=0 tx=0).stop 2024-09-19T07:54:16.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.634+0000 7fb2a77fe640 1 -- 172.21.15.16:0/442454986 reap_dead start 2024-09-19T07:54:16.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.634+0000 7fb2ad5e8640 1 -- 172.21.15.16:0/442454986 shutdown_connections 2024-09-19T07:54:16.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.634+0000 7fb2ad5e8640 1 -- 172.21.15.16:0/442454986 >> 172.21.15.16:0/442454986 conn(0x7fb2a809d760 msgr2=0x7fb2a808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:16.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.634+0000 7fb2ad5e8640 1 -- 172.21.15.16:0/442454986 shutdown_connections 2024-09-19T07:54:16.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.634+0000 7fb2ad5e8640 1 -- 172.21.15.16:0/442454986 wait complete. 2024-09-19T07:54:16.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.635+0000 7fb2ad5e8640 1 Processor -- start 2024-09-19T07:54:16.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.635+0000 7fb2ad5e8640 1 -- start start 2024-09-19T07:54:16.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb2ad5e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2a80b9650 0x7fb2a80c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb2ad5e8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2a80cb4b0 0x7fb2a815c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb2ad5e8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2a80b9a30 0x7fb2a815cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb2ad5e8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a806dcb0 con 0x7fb2a80b9650 2024-09-19T07:54:16.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb2ad5e8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a806db30 con 0x7fb2a80cb4b0 2024-09-19T07:54:16.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb2ad5e8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a80cfad0 con 0x7fb2a80b9a30 2024-09-19T07:54:16.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb2a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2a80b9650 0x7fb2a80c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb2a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2a80b9650 0x7fb2a80c8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54756/0 (socket says 172.21.15.16:54756) 2024-09-19T07:54:16.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb2a6ffd640 1 -- 172.21.15.16:0/3693021164 learned_addr learned my addr 172.21.15.16:0/3693021164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:16.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb2a67fc640 1 --2- 172.21.15.16:0/3693021164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2a80cb4b0 0x7fb2a815c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb2a77fe640 1 --2- 172.21.15.16:0/3693021164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2a80b9a30 0x7fb2a815cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb2a6ffd640 1 -- 172.21.15.16:0/3693021164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2a80b9a30 msgr2=0x7fb2a815cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:16.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb2a6ffd640 1 --2- 172.21.15.16:0/3693021164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2a80b9a30 0x7fb2a815cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb2a6ffd640 1 -- 172.21.15.16:0/3693021164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2a80cb4b0 msgr2=0x7fb2a815c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:16.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb2a6ffd640 1 --2- 172.21.15.16:0/3693021164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2a80cb4b0 0x7fb2a815c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb2a6ffd640 1 -- 172.21.15.16:0/3693021164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a80ccc70 con 0x7fb2a80b9650 2024-09-19T07:54:16.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb2a67fc640 1 --2- 172.21.15.16:0/3693021164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2a80cb4b0 0x7fb2a815c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:16.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb2a6ffd640 1 --2- 172.21.15.16:0/3693021164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2a80b9650 0x7fb2a80c8670 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb298000fb0 tx=0x7fb298007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:16.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb28f7fe640 1 -- 172.21.15.16:0/3693021164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29800f9d0 con 0x7fb2a80b9650 2024-09-19T07:54:16.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb28f7fe640 1 -- 172.21.15.16:0/3693021164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb298018070 con 0x7fb2a80b9650 2024-09-19T07:54:16.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.636+0000 7fb2ad5e8640 1 -- 172.21.15.16:0/3693021164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a80c8bb0 con 0x7fb2a80b9650 2024-09-19T07:54:16.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.637+0000 7fb28f7fe640 1 -- 172.21.15.16:0/3693021164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29800fdf0 con 0x7fb2a80b9650 2024-09-19T07:54:16.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.637+0000 7fb2ad5e8640 1 -- 172.21.15.16:0/3693021164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a80c9160 con 0x7fb2a80b9650 2024-09-19T07:54:16.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.638+0000 7fb2ad5e8640 1 -- 172.21.15.16:0/3693021164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a806d420 con 0x7fb2a80b9650 2024-09-19T07:54:16.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.638+0000 7fb28f7fe640 1 -- 172.21.15.16:0/3693021164 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb29801d460 con 0x7fb2a80b9650 2024-09-19T07:54:16.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.638+0000 7fb28f7fe640 1 --2- 172.21.15.16:0/3693021164 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb29406c2e0 0x7fb29406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.639+0000 7fb28f7fe640 1 -- 172.21.15.16:0/3693021164 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7fb29808b590 con 0x7fb2a80b9650 2024-09-19T07:54:16.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.639+0000 7fb2a67fc640 1 --2- 172.21.15.16:0/3693021164 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb29406c2e0 0x7fb29406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.639+0000 7fb2a67fc640 1 --2- 172.21.15.16:0/3693021164 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb29406c2e0 0x7fb29406e7a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb2a806e110 tx=0x7fb29c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:16.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.643+0000 7fb28f7fe640 1 -- 172.21.15.16:0/3693021164 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb298059dd0 con 0x7fb2a80b9650 2024-09-19T07:54:16.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.823+0000 7fb2ad5e8640 1 -- 172.21.15.16:0/3693021164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2a806cf90 con 0x7fb2a80b9650 2024-09-19T07:54:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.824+0000 7fb28f7fe640 1 -- 172.21.15.16:0/3693021164 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+13339 (secure 0 0 0) 0x7fb2980596d0 con 0x7fb2a80b9650 2024-09-19T07:54:16.826 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:16.826 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":121,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:54:14.505178+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/2050126415":"2024-09-20T07:54:11.498008+0000","172.21.15.16:6834/2050126415":"2024-09-20T07:54:11.498008+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:54:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.826+0000 7fb2ad5e8640 1 -- 172.21.15.16:0/3693021164 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb29406c2e0 msgr2=0x7fb29406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.826+0000 7fb2ad5e8640 1 --2- 172.21.15.16:0/3693021164 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb29406c2e0 0x7fb29406e7a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb2a806e110 tx=0x7fb29c008040 comp rx=0 tx=0).stop 2024-09-19T07:54:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.826+0000 7fb2ad5e8640 1 -- 172.21.15.16:0/3693021164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2a80b9650 msgr2=0x7fb2a80c8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.826+0000 7fb2ad5e8640 1 --2- 172.21.15.16:0/3693021164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2a80b9650 0x7fb2a80c8670 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb298000fb0 tx=0x7fb298007720 comp rx=0 tx=0).stop 2024-09-19T07:54:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.826+0000 7fb2ad5e8640 1 -- 172.21.15.16:0/3693021164 shutdown_connections 2024-09-19T07:54:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.826+0000 7fb2ad5e8640 1 --2- 172.21.15.16:0/3693021164 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb29406c2e0 0x7fb29406e7a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.826+0000 7fb2ad5e8640 1 --2- 172.21.15.16:0/3693021164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2a80b9a30 0x7fb2a815cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.826+0000 7fb2ad5e8640 1 --2- 172.21.15.16:0/3693021164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2a80cb4b0 0x7fb2a815c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.826+0000 7fb2ad5e8640 1 --2- 172.21.15.16:0/3693021164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2a80b9650 0x7fb2a80c8670 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.826+0000 7fb2ad5e8640 1 -- 172.21.15.16:0/3693021164 >> 172.21.15.16:0/3693021164 conn(0x7fb2a809d760 msgr2=0x7fb2a81e8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:16.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.826+0000 7fb2ad5e8640 1 -- 172.21.15.16:0/3693021164 shutdown_connections 2024-09-19T07:54:16.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.826+0000 7fb2ad5e8640 1 -- 172.21.15.16:0/3693021164 wait complete. 2024-09-19T07:54:16.838 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T07:42:54.500338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T07:54:16.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T07:54:16.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.942+0000 7fadd4f31640 1 Processor -- start 2024-09-19T07:54:16.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.942+0000 7fadd4f31640 1 -- start start 2024-09-19T07:54:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.943+0000 7fadd4f31640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fadd00d0bc0 0x7fadd00cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.943+0000 7fadd4f31640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fadd015c2c0 0x7fadd00cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.943+0000 7fadd4f31640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadd00cfc80 con 0x7fadd015bee0 2024-09-19T07:54:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.943+0000 7fadd4f31640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fadd00ceaf0 con 0x7fadd015c2c0 2024-09-19T07:54:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.943+0000 7fadd4f31640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fadd00cdee0 con 0x7fadd00d0bc0 2024-09-19T07:54:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.943+0000 7fadced9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fadd015c2c0 0x7fadd00cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.943+0000 7fadced9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fadd015c2c0 0x7fadd00cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54770/0 (socket says 172.21.15.16:54770) 2024-09-19T07:54:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.943+0000 7fadced9d640 1 -- 172.21.15.16:0/858980315 learned_addr learned my addr 172.21.15.16:0/858980315 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.943+0000 7fadcdd9b640 1 --2- 172.21.15.16:0/858980315 >> v2:172.21.15.146:3300/0 conn(0x7fadd00d0bc0 0x7fadd00cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.943+0000 7fadced9d640 1 -- 172.21.15.16:0/858980315 >> v2:172.21.15.146:3300/0 conn(0x7fadd00d0bc0 msgr2=0x7fadd00cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.943+0000 7fadced9d640 1 --2- 172.21.15.16:0/858980315 >> v2:172.21.15.146:3300/0 conn(0x7fadd00d0bc0 0x7fadd00cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.944+0000 7fadced9d640 1 -- 172.21.15.16:0/858980315 >> v1:172.21.15.146:6789/0 conn(0x7fadd015bee0 legacy=0x7fadd00d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.944+0000 7fadced9d640 1 -- 172.21.15.16:0/858980315 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd00ce1c0 con 0x7fadd015c2c0 2024-09-19T07:54:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.944+0000 7fadcdd9b640 1 --2- 172.21.15.16:0/858980315 >> v2:172.21.15.146:3300/0 conn(0x7fadd00d0bc0 0x7fadd00cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.944+0000 7fadced9d640 1 --2- 172.21.15.16:0/858980315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd015c2c0 0x7fadd00cf740 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fadd00cadd0 tx=0x7fadc802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d03d64b19f37ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.944+0000 7fadcd59a640 1 -- 172.21.15.16:0/858980315 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc8036070 con 0x7fadd015c2c0 2024-09-19T07:54:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.944+0000 7fadcd59a640 1 -- 172.21.15.16:0/858980315 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadc8031600 con 0x7fadd015c2c0 2024-09-19T07:54:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.944+0000 7fadcd59a640 1 -- 172.21.15.16:0/858980315 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc80319d0 con 0x7fadd015c2c0 2024-09-19T07:54:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.945+0000 7fadd4f31640 1 -- 172.21.15.16:0/858980315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd015c2c0 msgr2=0x7fadd00cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.945+0000 7fadd4f31640 1 --2- 172.21.15.16:0/858980315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd015c2c0 0x7fadd00cf740 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fadd00cadd0 tx=0x7fadc802a6f0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.945+0000 7fadd4f31640 1 -- 172.21.15.16:0/858980315 shutdown_connections 2024-09-19T07:54:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.945+0000 7fadd4f31640 1 --2- 172.21.15.16:0/858980315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd015c2c0 0x7fadd00cf740 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.945+0000 7fadd4f31640 1 --2- 172.21.15.16:0/858980315 >> v2:172.21.15.146:3300/0 conn(0x7fadd00d0bc0 0x7fadd00cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.945+0000 7fadd4f31640 1 -- 172.21.15.16:0/858980315 >> 172.21.15.16:0/858980315 conn(0x7fadd006bdf0 msgr2=0x7fadd006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:16.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.945+0000 7fadd4f31640 1 -- 172.21.15.16:0/858980315 shutdown_connections 2024-09-19T07:54:16.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.945+0000 7fadd4f31640 1 -- 172.21.15.16:0/858980315 wait complete. 2024-09-19T07:54:16.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.946+0000 7fadd4f31640 1 Processor -- start 2024-09-19T07:54:16.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.947+0000 7fadd4f31640 1 -- start start 2024-09-19T07:54:16.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.947+0000 7fadd4f31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd00d0bc0 0x7fadd0149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.947+0000 7fadd4f31640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadd015bee0 0x7fadd015f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.947+0000 7fadd4f31640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadd015c2c0 0x7fadd014c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.947+0000 7fadd4f31640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fadd006dec0 con 0x7fadd00d0bc0 2024-09-19T07:54:16.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.947+0000 7fadd4f31640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fadd00ce780 con 0x7fadd015bee0 2024-09-19T07:54:16.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.947+0000 7fadd4f31640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fadd006e040 con 0x7fadd015c2c0 2024-09-19T07:54:16.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.947+0000 7fadce59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd00d0bc0 0x7fadd0149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.947+0000 7fadce59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd00d0bc0 0x7fadd0149250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54772/0 (socket says 172.21.15.16:54772) 2024-09-19T07:54:16.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.947+0000 7fadce59c640 1 -- 172.21.15.16:0/2879101916 learned_addr learned my addr 172.21.15.16:0/2879101916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:16.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.947+0000 7fadcdd9b640 1 --2- 172.21.15.16:0/2879101916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadd015bee0 0x7fadd015f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.948+0000 7fadced9d640 1 --2- 172.21.15.16:0/2879101916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadd015c2c0 0x7fadd014c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.948+0000 7fadce59c640 1 -- 172.21.15.16:0/2879101916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadd015c2c0 msgr2=0x7fadd014c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:16.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.948+0000 7fadce59c640 1 --2- 172.21.15.16:0/2879101916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadd015c2c0 0x7fadd014c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.948+0000 7fadce59c640 1 -- 172.21.15.16:0/2879101916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadd015bee0 msgr2=0x7fadd015f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:16.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.948+0000 7fadce59c640 1 --2- 172.21.15.16:0/2879101916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadd015bee0 0x7fadd015f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:16.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.948+0000 7fadce59c640 1 -- 172.21.15.16:0/2879101916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd00c8e40 con 0x7fadd00d0bc0 2024-09-19T07:54:16.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.948+0000 7fadced9d640 1 --2- 172.21.15.16:0/2879101916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadd015c2c0 0x7fadd014c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:16.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.948+0000 7fadce59c640 1 --2- 172.21.15.16:0/2879101916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd00d0bc0 0x7fadd0149250 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fadc000c460 tx=0x7fadc000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:16.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.948+0000 7fadbeffd640 1 -- 172.21.15.16:0/2879101916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0016020 con 0x7fadd00d0bc0 2024-09-19T07:54:16.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.948+0000 7fadbeffd640 1 -- 172.21.15.16:0/2879101916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadc0012870 con 0x7fadd00d0bc0 2024-09-19T07:54:16.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.948+0000 7fadd4f31640 1 -- 172.21.15.16:0/2879101916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd00c9120 con 0x7fadd00d0bc0 2024-09-19T07:54:16.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.948+0000 7fadbeffd640 1 -- 172.21.15.16:0/2879101916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0012c60 con 0x7fadd00d0bc0 2024-09-19T07:54:16.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.948+0000 7fadd4f31640 1 -- 172.21.15.16:0/2879101916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd00c96e0 con 0x7fadd00d0bc0 2024-09-19T07:54:16.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.949+0000 7fadd4f31640 1 -- 172.21.15.16:0/2879101916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd00d04b0 con 0x7fadd00d0bc0 2024-09-19T07:54:16.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.949+0000 7fadbeffd640 1 -- 172.21.15.16:0/2879101916 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fadc0019020 con 0x7fadd00d0bc0 2024-09-19T07:54:16.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.950+0000 7fadbeffd640 1 --2- 172.21.15.16:0/2879101916 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fadb006c2b0 0x7fadb006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:16.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.950+0000 7fadbeffd640 1 -- 172.21.15.16:0/2879101916 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7fadc008b260 con 0x7fadd00d0bc0 2024-09-19T07:54:16.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.950+0000 7fadcdd9b640 1 --2- 172.21.15.16:0/2879101916 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fadb006c2b0 0x7fadb006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:16.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.951+0000 7fadcdd9b640 1 --2- 172.21.15.16:0/2879101916 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fadb006c2b0 0x7fadb006e770 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fadd00af220 tx=0x7fadc4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:16.954+0000 7fadbeffd640 1 -- 172.21.15.16:0/2879101916 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fadc0020020 con 0x7fadd00d0bc0 2024-09-19T07:54:17.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.128+0000 7fadd4f31640 1 -- 172.21.15.16:0/2879101916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fadd006c1d0 con 0x7fadd00d0bc0 2024-09-19T07:54:17.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.129+0000 7fadbeffd640 1 -- 172.21.15.16:0/2879101916 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) ==== 93+0+10 (secure 0 0 0) 0x7fadc0059b40 con 0x7fadd00d0bc0 2024-09-19T07:54:17.129 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-19T07:54:17.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.131+0000 7fadd4f31640 1 -- 172.21.15.16:0/2879101916 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fadb006c2b0 msgr2=0x7fadb006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.131+0000 7fadd4f31640 1 --2- 172.21.15.16:0/2879101916 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fadb006c2b0 0x7fadb006e770 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fadd00af220 tx=0x7fadc4008040 comp rx=0 tx=0).stop 2024-09-19T07:54:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.131+0000 7fadd4f31640 1 -- 172.21.15.16:0/2879101916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd00d0bc0 msgr2=0x7fadd0149250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.131+0000 7fadd4f31640 1 --2- 172.21.15.16:0/2879101916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd00d0bc0 0x7fadd0149250 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fadc000c460 tx=0x7fadc000c920 comp rx=0 tx=0).stop 2024-09-19T07:54:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.131+0000 7fadd4f31640 1 -- 172.21.15.16:0/2879101916 shutdown_connections 2024-09-19T07:54:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.131+0000 7fadd4f31640 1 --2- 172.21.15.16:0/2879101916 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fadb006c2b0 0x7fadb006e770 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.131+0000 7fadd4f31640 1 --2- 172.21.15.16:0/2879101916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadd015c2c0 0x7fadd014c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.131+0000 7fadd4f31640 1 --2- 172.21.15.16:0/2879101916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadd015bee0 0x7fadd015f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.131+0000 7fadd4f31640 1 --2- 172.21.15.16:0/2879101916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd00d0bc0 0x7fadd0149250 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.131+0000 7fadd4f31640 1 -- 172.21.15.16:0/2879101916 >> 172.21.15.16:0/2879101916 conn(0x7fadd006bdf0 msgr2=0x7fadd00b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.131+0000 7fadd4f31640 1 -- 172.21.15.16:0/2879101916 shutdown_connections 2024-09-19T07:54:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.131+0000 7fadd4f31640 1 -- 172.21.15.16:0/2879101916 wait complete. 2024-09-19T07:54:17.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps' 2024-09-19T07:54:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.244+0000 7fce8670a640 1 Processor -- start 2024-09-19T07:54:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.244+0000 7fce8670a640 1 -- start start 2024-09-19T07:54:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.244+0000 7fce8670a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fce800d1c70 0x7fce800d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.244+0000 7fce8670a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce8015ce30 0x7fce800b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.244+0000 7fce8670a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce800d2590 con 0x7fce800d0b30 2024-09-19T07:54:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.244+0000 7fce8670a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fce8015d4c0 con 0x7fce8015ce30 2024-09-19T07:54:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.244+0000 7fce8670a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fce800cfd70 con 0x7fce800d1c70 2024-09-19T07:54:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.245+0000 7fce84ca7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce8015ce30 0x7fce800b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.245+0000 7fce84ca7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce8015ce30 0x7fce800b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54776/0 (socket says 172.21.15.16:54776) 2024-09-19T07:54:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.245+0000 7fce84ca7640 1 -- 172.21.15.16:0/3878936737 learned_addr learned my addr 172.21.15.16:0/3878936737 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.245+0000 7fce7ffff640 1 --2- 172.21.15.16:0/3878936737 >> v2:172.21.15.146:3300/0 conn(0x7fce800d1c70 0x7fce800d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.245+0000 7fce84ca7640 1 -- 172.21.15.16:0/3878936737 >> v2:172.21.15.146:3300/0 conn(0x7fce800d1c70 msgr2=0x7fce800d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.245+0000 7fce84ca7640 1 --2- 172.21.15.16:0/3878936737 >> v2:172.21.15.146:3300/0 conn(0x7fce800d1c70 0x7fce800d2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.245+0000 7fce84ca7640 1 -- 172.21.15.16:0/3878936737 >> v1:172.21.15.146:6789/0 conn(0x7fce800d0b30 legacy=0x7fce8015c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.245+0000 7fce84ca7640 1 -- 172.21.15.16:0/3878936737 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce800d0050 con 0x7fce8015ce30 2024-09-19T07:54:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.245+0000 7fce7ffff640 1 --2- 172.21.15.16:0/3878936737 >> v2:172.21.15.146:3300/0 conn(0x7fce800d1c70 0x7fce800d2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.246+0000 7fce84ca7640 1 --2- 172.21.15.16:0/3878936737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8015ce30 0x7fce800b6f40 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fce80070fd0 tx=0x7fce7802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8981472de6cd75e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.246+0000 7fce7f7fe640 1 -- 172.21.15.16:0/3878936737 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78036070 con 0x7fce8015ce30 2024-09-19T07:54:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.246+0000 7fce7f7fe640 1 -- 172.21.15.16:0/3878936737 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce78031600 con 0x7fce8015ce30 2024-09-19T07:54:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.246+0000 7fce7f7fe640 1 -- 172.21.15.16:0/3878936737 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780319d0 con 0x7fce8015ce30 2024-09-19T07:54:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.246+0000 7fce8670a640 1 -- 172.21.15.16:0/3878936737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8015ce30 msgr2=0x7fce800b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.246+0000 7fce8670a640 1 --2- 172.21.15.16:0/3878936737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8015ce30 0x7fce800b6f40 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fce80070fd0 tx=0x7fce7802a730 comp rx=0 tx=0).stop 2024-09-19T07:54:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.246+0000 7fce8670a640 1 -- 172.21.15.16:0/3878936737 shutdown_connections 2024-09-19T07:54:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.246+0000 7fce8670a640 1 --2- 172.21.15.16:0/3878936737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8015ce30 0x7fce800b6f40 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.246+0000 7fce8670a640 1 --2- 172.21.15.16:0/3878936737 >> v2:172.21.15.146:3300/0 conn(0x7fce800d1c70 0x7fce800d2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.246+0000 7fce8670a640 1 -- 172.21.15.16:0/3878936737 >> 172.21.15.16:0/3878936737 conn(0x7fce800b3ed0 msgr2=0x7fce800b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.246+0000 7fce8670a640 1 -- 172.21.15.16:0/3878936737 shutdown_connections 2024-09-19T07:54:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.246+0000 7fce8670a640 1 -- 172.21.15.16:0/3878936737 wait complete. 2024-09-19T07:54:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.248+0000 7fce8670a640 1 Processor -- start 2024-09-19T07:54:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.248+0000 7fce8670a640 1 -- start start 2024-09-19T07:54:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.248+0000 7fce8670a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce800d0b30 0x7fce8014d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.248+0000 7fce8670a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce800d1c70 0x7fce8015b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.248+0000 7fce8670a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8015ce30 0x7fce8015b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.248+0000 7fce8670a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce80156a70 con 0x7fce8015ce30 2024-09-19T07:54:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.248+0000 7fce8670a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fce800d0610 con 0x7fce800d1c70 2024-09-19T07:54:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.248+0000 7fce8670a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fce800b7480 con 0x7fce800d0b30 2024-09-19T07:54:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.248+0000 7fce84ca7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8015ce30 0x7fce8015b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.249+0000 7fce84ca7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8015ce30 0x7fce8015b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54784/0 (socket says 172.21.15.16:54784) 2024-09-19T07:54:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.249+0000 7fce84ca7640 1 -- 172.21.15.16:0/1386841957 learned_addr learned my addr 172.21.15.16:0/1386841957 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.249+0000 7fce7ffff640 1 --2- 172.21.15.16:0/1386841957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce800d0b30 0x7fce8014d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.249+0000 7fce77fff640 1 --2- 172.21.15.16:0/1386841957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce800d1c70 0x7fce8015b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.249+0000 7fce84ca7640 1 -- 172.21.15.16:0/1386841957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce800d0b30 msgr2=0x7fce8014d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.249+0000 7fce84ca7640 1 --2- 172.21.15.16:0/1386841957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce800d0b30 0x7fce8014d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.249+0000 7fce84ca7640 1 -- 172.21.15.16:0/1386841957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce800d1c70 msgr2=0x7fce8015b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.249+0000 7fce84ca7640 1 --2- 172.21.15.16:0/1386841957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce800d1c70 0x7fce8015b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.249+0000 7fce84ca7640 1 -- 172.21.15.16:0/1386841957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce801e65c0 con 0x7fce8015ce30 2024-09-19T07:54:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.249+0000 7fce84ca7640 1 --2- 172.21.15.16:0/1386841957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8015ce30 0x7fce8015b540 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fce78034040 tx=0x7fce78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.249+0000 7fce7d7fa640 1 -- 172.21.15.16:0/1386841957 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78036050 con 0x7fce8015ce30 2024-09-19T07:54:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.249+0000 7fce7d7fa640 1 -- 172.21.15.16:0/1386841957 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce78042070 con 0x7fce8015ce30 2024-09-19T07:54:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.249+0000 7fce8670a640 1 -- 172.21.15.16:0/1386841957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce801e68a0 con 0x7fce8015ce30 2024-09-19T07:54:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.249+0000 7fce7d7fa640 1 -- 172.21.15.16:0/1386841957 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78009510 con 0x7fce8015ce30 2024-09-19T07:54:17.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.249+0000 7fce8670a640 1 -- 172.21.15.16:0/1386841957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce801e6e60 con 0x7fce8015ce30 2024-09-19T07:54:17.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.250+0000 7fce8670a640 1 -- 172.21.15.16:0/1386841957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce800d2050 con 0x7fce8015ce30 2024-09-19T07:54:17.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.251+0000 7fce7d7fa640 1 -- 172.21.15.16:0/1386841957 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fce78007bf0 con 0x7fce8015ce30 2024-09-19T07:54:17.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.251+0000 7fce7d7fa640 1 --2- 172.21.15.16:0/1386841957 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fce6006c2b0 0x7fce6006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.251+0000 7fce7d7fa640 1 -- 172.21.15.16:0/1386841957 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7fce780ac030 con 0x7fce8015ce30 2024-09-19T07:54:17.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.251+0000 7fce7ffff640 1 --2- 172.21.15.16:0/1386841957 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fce6006c2b0 0x7fce6006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.252+0000 7fce7ffff640 1 --2- 172.21.15.16:0/1386841957 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fce6006c2b0 0x7fce6006e770 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fce70006fd0 tx=0x7fce70008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:17.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.256+0000 7fce7d7fa640 1 -- 172.21.15.16:0/1386841957 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fce7807a9a0 con 0x7fce8015ce30 2024-09-19T07:54:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.499+0000 7fce8670a640 1 -- 172.21.15.16:0/1386841957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]} v 0) -- 0x7fce800d29b0 con 0x7fce8015ce30 2024-09-19T07:54:17.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.519+0000 7fce7d7fa640 1 -- 172.21.15.16:0/1386841957 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]}]=0 v497) ==== 154+0+0 (secure 0 0 0) 0x7fce7807a2a0 con 0x7fce8015ce30 2024-09-19T07:54:17.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.521+0000 7fce8670a640 1 -- 172.21.15.16:0/1386841957 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fce6006c2b0 msgr2=0x7fce6006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.521+0000 7fce8670a640 1 --2- 172.21.15.16:0/1386841957 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fce6006c2b0 0x7fce6006e770 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fce70006fd0 tx=0x7fce70008040 comp rx=0 tx=0).stop 2024-09-19T07:54:17.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.521+0000 7fce8670a640 1 -- 172.21.15.16:0/1386841957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8015ce30 msgr2=0x7fce8015b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.521+0000 7fce8670a640 1 --2- 172.21.15.16:0/1386841957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8015ce30 0x7fce8015b540 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fce78034040 tx=0x7fce78004a40 comp rx=0 tx=0).stop 2024-09-19T07:54:17.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.521+0000 7fce8670a640 1 -- 172.21.15.16:0/1386841957 shutdown_connections 2024-09-19T07:54:17.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.521+0000 7fce8670a640 1 --2- 172.21.15.16:0/1386841957 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fce6006c2b0 0x7fce6006e770 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.521+0000 7fce8670a640 1 --2- 172.21.15.16:0/1386841957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8015ce30 0x7fce8015b540 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.521+0000 7fce8670a640 1 --2- 172.21.15.16:0/1386841957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce800d1c70 0x7fce8015b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.521+0000 7fce8670a640 1 --2- 172.21.15.16:0/1386841957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fce800d0b30 0x7fce8014d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.521+0000 7fce8670a640 1 -- 172.21.15.16:0/1386841957 >> 172.21.15.16:0/1386841957 conn(0x7fce800b3ed0 msgr2=0x7fce800b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:17.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.521+0000 7fce8670a640 1 -- 172.21.15.16:0/1386841957 shutdown_connections 2024-09-19T07:54:17.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.522+0000 7fce8670a640 1 -- 172.21.15.16:0/1386841957 wait complete. 2024-09-19T07:54:17.533 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T07:54:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.638+0000 7fedc7fe8640 1 Processor -- start 2024-09-19T07:54:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.638+0000 7fedc7fe8640 1 -- start start 2024-09-19T07:54:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.639+0000 7fedc7fe8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fedc015e2a0 0x7fedc006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.639+0000 7fedc7fe8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedc00d1880 con 0x7fedc015e680 2024-09-19T07:54:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.639+0000 7fedc7fe8640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedc00d1b60 con 0x7fedc006d960 2024-09-19T07:54:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.639+0000 7fedc7fe8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fedc00cf2d0 con 0x7fedc015e2a0 2024-09-19T07:54:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.639+0000 7fedc5d84640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fedc015e2a0 0x7fedc006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.639+0000 7fedc5583640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fedc015e680 0x7fedc00cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42700/0 (socket says 172.21.15.16:42700) 2024-09-19T07:54:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.639+0000 7fedc5583640 1 -- 172.21.15.16:0/946610298 learned_addr learned my addr 172.21.15.16:0/946610298 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.639+0000 7fedc5d84640 1 -- 172.21.15.16:0/946610298 >> v1:172.21.15.146:6789/0 conn(0x7fedc006d960 legacy=0x7fedc00d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.639+0000 7fedc5d84640 1 -- 172.21.15.16:0/946610298 >> v1:172.21.15.16:6789/0 conn(0x7fedc015e680 legacy=0x7fedc00cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.639+0000 7fedc4d82640 1 -- 172.21.15.16:0/946610298 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2266642828 0 0) 0x7fedc00d1880 con 0x7fedc015e680 2024-09-19T07:54:17.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.639+0000 7fedc5d84640 1 -- 172.21.15.16:0/946610298 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc00cf5b0 con 0x7fedc015e2a0 2024-09-19T07:54:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.640+0000 7fedc5d84640 1 --2- 172.21.15.16:0/946610298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc015e2a0 0x7fedc006d420 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009d90 tx=0x7fedb802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c6f6385b85e413 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.640+0000 7fedc4d82640 1 -- 172.21.15.16:0/946610298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb8007740 con 0x7fedc015e2a0 2024-09-19T07:54:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.640+0000 7fedc4d82640 1 -- 172.21.15.16:0/946610298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedb80078e0 con 0x7fedc015e2a0 2024-09-19T07:54:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.640+0000 7fedc4d82640 1 -- 172.21.15.16:0/946610298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb8007cb0 con 0x7fedc015e2a0 2024-09-19T07:54:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.641+0000 7fedc7fe8640 1 -- 172.21.15.16:0/946610298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc015e2a0 msgr2=0x7fedc006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.641+0000 7fedc7fe8640 1 --2- 172.21.15.16:0/946610298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc015e2a0 0x7fedc006d420 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009d90 tx=0x7fedb802a6a0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.641+0000 7fedc7fe8640 1 -- 172.21.15.16:0/946610298 shutdown_connections 2024-09-19T07:54:17.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.641+0000 7fedc7fe8640 1 --2- 172.21.15.16:0/946610298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc015e2a0 0x7fedc006d420 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.641+0000 7fedc7fe8640 1 -- 172.21.15.16:0/946610298 >> 172.21.15.16:0/946610298 conn(0x7fedc009d760 msgr2=0x7fedc008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:17.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.641+0000 7fedc7fe8640 1 -- 172.21.15.16:0/946610298 shutdown_connections 2024-09-19T07:54:17.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.641+0000 7fedc7fe8640 1 -- 172.21.15.16:0/946610298 wait complete. 2024-09-19T07:54:17.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.642+0000 7fedc7fe8640 1 Processor -- start 2024-09-19T07:54:17.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.642+0000 7fedc7fe8640 1 -- start start 2024-09-19T07:54:17.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.643+0000 7fedc7fe8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc006d960 0x7fedc00b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.643+0000 7fedc7fe8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fedc015e2a0 0x7fedc00c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.643+0000 7fedc7fe8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedc015e680 0x7fedc015d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.643+0000 7fedc7fe8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fedc00d0360 con 0x7fedc006d960 2024-09-19T07:54:17.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.643+0000 7fedc7fe8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fedc00d01e0 con 0x7fedc015e680 2024-09-19T07:54:17.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.643+0000 7fedc7fe8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fedc00d4a80 con 0x7fedc015e2a0 2024-09-19T07:54:17.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.643+0000 7fedc5d84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc006d960 0x7fedc00b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.643+0000 7fedc5d84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc006d960 0x7fedc00b54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54802/0 (socket says 172.21.15.16:54802) 2024-09-19T07:54:17.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.643+0000 7fedc5d84640 1 -- 172.21.15.16:0/3559181840 learned_addr learned my addr 172.21.15.16:0/3559181840 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:17.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.643+0000 7fedc5583640 1 --2- 172.21.15.16:0/3559181840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fedc015e2a0 0x7fedc00c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.643+0000 7fedc6585640 1 --2- 172.21.15.16:0/3559181840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedc015e680 0x7fedc015d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.643+0000 7fedc5d84640 1 -- 172.21.15.16:0/3559181840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fedc015e2a0 msgr2=0x7fedc00c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.643+0000 7fedc5d84640 1 --2- 172.21.15.16:0/3559181840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fedc015e2a0 0x7fedc00c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.643+0000 7fedc5d84640 1 -- 172.21.15.16:0/3559181840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedc015e680 msgr2=0x7fedc015d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.643+0000 7fedc5d84640 1 --2- 172.21.15.16:0/3559181840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedc015e680 0x7fedc015d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.644+0000 7fedc5d84640 1 -- 172.21.15.16:0/3559181840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc00c8550 con 0x7fedc006d960 2024-09-19T07:54:17.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.644+0000 7fedc5d84640 1 --2- 172.21.15.16:0/3559181840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc006d960 0x7fedc00b54a0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009d90 tx=0x7fedb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:17.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.644+0000 7fedbe7fc640 1 -- 172.21.15.16:0/3559181840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb8040070 con 0x7fedc006d960 2024-09-19T07:54:17.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.644+0000 7fedbe7fc640 1 -- 172.21.15.16:0/3559181840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedb8031c40 con 0x7fedc006d960 2024-09-19T07:54:17.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.644+0000 7fedc7fe8640 1 -- 172.21.15.16:0/3559181840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc00c8830 con 0x7fedc006d960 2024-09-19T07:54:17.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.644+0000 7fedbe7fc640 1 -- 172.21.15.16:0/3559181840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb803c420 con 0x7fedc006d960 2024-09-19T07:54:17.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.644+0000 7fedc7fe8640 1 -- 172.21.15.16:0/3559181840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc00b59e0 con 0x7fedc006d960 2024-09-19T07:54:17.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.645+0000 7fedc7fe8640 1 -- 172.21.15.16:0/3559181840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc006d420 con 0x7fedc006d960 2024-09-19T07:54:17.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.645+0000 7fedbe7fc640 1 -- 172.21.15.16:0/3559181840 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fedb8039040 con 0x7fedc006d960 2024-09-19T07:54:17.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.646+0000 7fedbe7fc640 1 --2- 172.21.15.16:0/3559181840 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fedac06c2b0 0x7fedac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.646+0000 7fedbe7fc640 1 -- 172.21.15.16:0/3559181840 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7fedb80ac980 con 0x7fedc006d960 2024-09-19T07:54:17.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.646+0000 7fedc5583640 1 --2- 172.21.15.16:0/3559181840 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fedac06c2b0 0x7fedac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.647+0000 7fedc5583640 1 --2- 172.21.15.16:0/3559181840 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fedac06c2b0 0x7fedac06e770 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fedc0161440 tx=0x7fedb4007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:17.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.650+0000 7fedbe7fc640 1 -- 172.21.15.16:0/3559181840 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fedb807b2f0 con 0x7fedc006d960 2024-09-19T07:54:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.825+0000 7fedc7fe8640 1 -- 172.21.15.16:0/3559181840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fedc00c7c90 con 0x7fedc006d960 2024-09-19T07:54:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.829+0000 7fedbe7fc640 1 -- 172.21.15.16:0/3559181840 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v53)=0 v53) ==== 122+0+0 (secure 0 0 0) 0x7fedb807abf0 con 0x7fedc006d960 2024-09-19T07:54:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.831+0000 7fedc7fe8640 1 -- 172.21.15.16:0/3559181840 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fedac06c2b0 msgr2=0x7fedac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.831+0000 7fedc7fe8640 1 --2- 172.21.15.16:0/3559181840 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fedac06c2b0 0x7fedac06e770 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fedc0161440 tx=0x7fedb4007c00 comp rx=0 tx=0).stop 2024-09-19T07:54:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.831+0000 7fedc7fe8640 1 -- 172.21.15.16:0/3559181840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc006d960 msgr2=0x7fedc00b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.831+0000 7fedc7fe8640 1 --2- 172.21.15.16:0/3559181840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc006d960 0x7fedc00b54a0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009d90 tx=0x7fedb8004a40 comp rx=0 tx=0).stop 2024-09-19T07:54:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.831+0000 7fedc7fe8640 1 -- 172.21.15.16:0/3559181840 shutdown_connections 2024-09-19T07:54:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.831+0000 7fedc7fe8640 1 --2- 172.21.15.16:0/3559181840 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fedac06c2b0 0x7fedac06e770 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.831+0000 7fedc7fe8640 1 --2- 172.21.15.16:0/3559181840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedc015e680 0x7fedc015d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.831+0000 7fedc7fe8640 1 --2- 172.21.15.16:0/3559181840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fedc015e2a0 0x7fedc00c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.831+0000 7fedc7fe8640 1 --2- 172.21.15.16:0/3559181840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc006d960 0x7fedc00b54a0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.831+0000 7fedc7fe8640 1 -- 172.21.15.16:0/3559181840 >> 172.21.15.16:0/3559181840 conn(0x7fedc009d760 msgr2=0x7fedc00d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.831+0000 7fedc7fe8640 1 -- 172.21.15.16:0/3559181840 shutdown_connections 2024-09-19T07:54:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:17.831+0000 7fedc7fe8640 1 -- 172.21.15.16:0/3559181840 wait complete. 2024-09-19T07:54:17.843 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:54:17.843 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:54:17.881 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:54:17.931 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:54:17.932 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:54:17.932 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:54:17.949 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:54:17.949 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:54:17.949 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:54:17.969 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:54:17.969 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:54:17.969 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:54:18.007 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:54:18.007 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:54:18.007 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:54:18.027 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:54:18.027 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T07:54:18.027 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:54:18.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.149+0000 7f0207fcd640 1 Processor -- start 2024-09-19T07:54:18.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.149+0000 7f0207fcd640 1 -- start start 2024-09-19T07:54:18.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.150+0000 7f0207fcd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0200161eb0 0x7f02001622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:18.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.150+0000 7f0207fcd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0200162b70 con 0x7f020006cd20 2024-09-19T07:54:18.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.150+0000 7f0207fcd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0200141b00 con 0x7f02001425e0 2024-09-19T07:54:18.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.150+0000 7f0207fcd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0200162e50 con 0x7f0200161eb0 2024-09-19T07:54:18.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.150+0000 7f0205d69640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f020006cd20 0x7f020006d100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42712/0 (socket says 172.21.15.16:42712) 2024-09-19T07:54:18.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.150+0000 7f0205d69640 1 -- 172.21.15.16:0/3598565518 learned_addr learned my addr 172.21.15.16:0/3598565518 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.150+0000 7f0205568640 1 --2- 172.21.15.16:0/3598565518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0200161eb0 0x7f02001622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.150+0000 7f0204d67640 1 -- 172.21.15.16:0/3598565518 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1987257514 0 0) 0x7f0200162b70 con 0x7f020006cd20 2024-09-19T07:54:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.150+0000 7f0204d67640 1 -- 172.21.15.16:0/3598565518 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01ec003520 con 0x7f020006cd20 2024-09-19T07:54:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.150+0000 7f0204d67640 1 -- 172.21.15.16:0/3598565518 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1254565719 0 0) 0x7f01ec003520 con 0x7f020006cd20 2024-09-19T07:54:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.150+0000 7f0204d67640 1 -- 172.21.15.16:0/3598565518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0200161eb0 msgr2=0x7f02001622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.150+0000 7f0204d67640 1 --2- 172.21.15.16:0/3598565518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0200161eb0 0x7f02001622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.150+0000 7f0204d67640 1 -- 172.21.15.16:0/3598565518 >> v1:172.21.15.146:6789/0 conn(0x7f02001425e0 legacy=0x7f0200142a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.150+0000 7f0204d67640 1 -- 172.21.15.16:0/3598565518 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0200141de0 con 0x7f020006cd20 2024-09-19T07:54:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.150+0000 7f0204d67640 1 -- 172.21.15.16:0/3598565518 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f01fc001670 con 0x7f020006cd20 2024-09-19T07:54:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.151+0000 7f0204d67640 1 -- 172.21.15.16:0/3598565518 >> v1:172.21.15.16:6789/0 conn(0x7f020006cd20 legacy=0x7f020006d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.151+0000 7f0204d67640 1 --2- 172.21.15.16:0/3598565518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0200161eb0 0x7f01ec0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.151+0000 7f0204d67640 1 --2- 172.21.15.16:0/3598565518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01ec003f60 0x7f01ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.151+0000 7f0204d67640 1 --2- 172.21.15.16:0/3598565518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ec004d40 0x7f01ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.151+0000 7f0204d67640 1 -- 172.21.15.16:0/3598565518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0200162e50 con 0x7f01ec004d40 2024-09-19T07:54:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.151+0000 7f0204d67640 1 -- 172.21.15.16:0/3598565518 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec00bb50 con 0x7f01ec003f60 2024-09-19T07:54:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.151+0000 7f0204d67640 1 -- 172.21.15.16:0/3598565518 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f01ec00bd50 con 0x7f0200161eb0 2024-09-19T07:54:18.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.151+0000 7f020656a640 1 --2- 172.21.15.16:0/3598565518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ec004d40 0x7f01ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.151+0000 7f0205d69640 1 --2- 172.21.15.16:0/3598565518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0200161eb0 0x7f01ec0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.151+0000 7f0205568640 1 --2- 172.21.15.16:0/3598565518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01ec003f60 0x7f01ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.151+0000 7f020656a640 1 -- 172.21.15.16:0/3598565518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0200161eb0 msgr2=0x7f01ec0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.151+0000 7f020656a640 1 --2- 172.21.15.16:0/3598565518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0200161eb0 0x7f01ec0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.151+0000 7f020656a640 1 -- 172.21.15.16:0/3598565518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01ec003f60 msgr2=0x7f01ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.151+0000 7f020656a640 1 --2- 172.21.15.16:0/3598565518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01ec003f60 0x7f01ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.151+0000 7f020656a640 1 -- 172.21.15.16:0/3598565518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ec003520 con 0x7f01ec004d40 2024-09-19T07:54:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.152+0000 7f020656a640 1 --2- 172.21.15.16:0/3598565518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ec004d40 0x7f01ec0051c0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f01f8001e50 tx=0x7f01ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6affa5933fa1cb7d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.152+0000 7f0204d67640 1 -- 172.21.15.16:0/3598565518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8001a70 con 0x7f01ec004d40 2024-09-19T07:54:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.152+0000 7f0204d67640 1 -- 172.21.15.16:0/3598565518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f8003970 con 0x7f01ec004d40 2024-09-19T07:54:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.152+0000 7f0204d67640 1 -- 172.21.15.16:0/3598565518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8003d60 con 0x7f01ec004d40 2024-09-19T07:54:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.152+0000 7f0207fcd640 1 -- 172.21.15.16:0/3598565518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ec004d40 msgr2=0x7f01ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:18.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.152+0000 7f0207fcd640 1 --2- 172.21.15.16:0/3598565518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ec004d40 0x7f01ec0051c0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f01f8001e50 tx=0x7f01ec005830 comp rx=0 tx=0).stop 2024-09-19T07:54:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.152+0000 7f020656a640 1 -- 172.21.15.16:0/3598565518 reap_dead start 2024-09-19T07:54:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.152+0000 7f0207fcd640 1 -- 172.21.15.16:0/3598565518 shutdown_connections 2024-09-19T07:54:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.152+0000 7f0207fcd640 1 -- 172.21.15.16:0/3598565518 >> 172.21.15.16:0/3598565518 conn(0x7f02000b3ed0 msgr2=0x7f02000b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.153+0000 7f0207fcd640 1 -- 172.21.15.16:0/3598565518 shutdown_connections 2024-09-19T07:54:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.153+0000 7f0207fcd640 1 -- 172.21.15.16:0/3598565518 wait complete. 2024-09-19T07:54:18.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.154+0000 7f0207fcd640 1 Processor -- start 2024-09-19T07:54:18.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.154+0000 7f0207fcd640 1 -- start start 2024-09-19T07:54:18.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f0207fcd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01ec004d40 0x7f0200140c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:18.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f0207fcd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02000cabb0 0x7f02000cb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:18.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f0207fcd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02000cb570 0x7f02001f7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:18.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f0207fcd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f02001622f0 con 0x7f02000cb570 2024-09-19T07:54:18.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f0207fcd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0200142150 con 0x7f02000cabb0 2024-09-19T07:54:18.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f0207fcd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0200162470 con 0x7f01ec004d40 2024-09-19T07:54:18.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f020656a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02000cb570 0x7f02001f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:18.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f020656a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02000cb570 0x7f02001f7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54806/0 (socket says 172.21.15.16:54806) 2024-09-19T07:54:18.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f020656a640 1 -- 172.21.15.16:0/1160677904 learned_addr learned my addr 172.21.15.16:0/1160677904 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:18.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f0205568640 1 --2- 172.21.15.16:0/1160677904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02000cabb0 0x7f02000cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:18.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f0205d69640 1 --2- 172.21.15.16:0/1160677904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01ec004d40 0x7f0200140c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:18.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f020656a640 1 -- 172.21.15.16:0/1160677904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01ec004d40 msgr2=0x7f0200140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:18.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f020656a640 1 --2- 172.21.15.16:0/1160677904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01ec004d40 0x7f0200140c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f020656a640 1 -- 172.21.15.16:0/1160677904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02000cabb0 msgr2=0x7f02000cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:18.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f020656a640 1 --2- 172.21.15.16:0/1160677904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02000cabb0 0x7f02000cb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f020656a640 1 -- 172.21.15.16:0/1160677904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02000cb9f0 con 0x7f02000cb570 2024-09-19T07:54:18.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f0205568640 1 --2- 172.21.15.16:0/1160677904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02000cabb0 0x7f02000cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:18.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f020656a640 1 --2- 172.21.15.16:0/1160677904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02000cb570 0x7f02001f7650 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f01f8001f40 tx=0x7f01f80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:18.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f01f67fc640 1 -- 172.21.15.16:0/1160677904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8037040 con 0x7f02000cb570 2024-09-19T07:54:18.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f01f67fc640 1 -- 172.21.15.16:0/1160677904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f80050d0 con 0x7f02000cb570 2024-09-19T07:54:18.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f0207fcd640 1 -- 172.21.15.16:0/1160677904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02000b7ae0 con 0x7f02000cb570 2024-09-19T07:54:18.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f01f67fc640 1 -- 172.21.15.16:0/1160677904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8001a70 con 0x7f02000cb570 2024-09-19T07:54:18.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.155+0000 7f0207fcd640 1 -- 172.21.15.16:0/1160677904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02000ca1c0 con 0x7f02000cb570 2024-09-19T07:54:18.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.156+0000 7f0207fcd640 1 -- 172.21.15.16:0/1160677904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02000c9d10 con 0x7f02000cb570 2024-09-19T07:54:18.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.157+0000 7f01f67fc640 1 -- 172.21.15.16:0/1160677904 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f01f8042020 con 0x7f02000cb570 2024-09-19T07:54:18.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.157+0000 7f01f67fc640 1 --2- 172.21.15.16:0/1160677904 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01e406c2b0 0x7f01e406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:18.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.157+0000 7f01f67fc640 1 -- 172.21.15.16:0/1160677904 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7f01f80aa980 con 0x7f02000cb570 2024-09-19T07:54:18.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.158+0000 7f0205d69640 1 --2- 172.21.15.16:0/1160677904 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01e406c2b0 0x7f01e406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:18.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.158+0000 7f0205d69640 1 --2- 172.21.15.16:0/1160677904 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01e406c2b0 0x7f01e406e770 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f01fc001eb0 tx=0x7f01fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:18.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.162+0000 7f01f67fc640 1 -- 172.21.15.16:0/1160677904 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f01f8079240 con 0x7f02000cb570 2024-09-19T07:54:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.373+0000 7f0207fcd640 1 -- 172.21.15.16:0/1160677904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02000690c0 con 0x7f02000cb570 2024-09-19T07:54:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.373+0000 7f01f67fc640 1 -- 172.21.15.16:0/1160677904 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+3914 (secure 0 0 0) 0x7f01f8078b40 con 0x7f02000cb570 2024-09-19T07:54:18.374 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:18.374 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":67,"btime":"2024-09-19T07:54:12:499686+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2471408352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2471408352},{"type":"v1","addr":"172.21.15.16:6833","nonce":2471408352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/1281369866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1281369866},{"type":"v1","addr":"172.21.15.146:6836","nonce":1281369866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1377303883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":1377303883},{"type":"v1","addr":"172.21.15.146:6837","nonce":1377303883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7722,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/408610558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":408610558},{"type":"v1","addr":"172.21.15.16:6835","nonce":408610558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[]} 2024-09-19T07:54:18.374 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 67 2024-09-19T07:54:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.375+0000 7f0207fcd640 1 -- 172.21.15.16:0/1160677904 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01e406c2b0 msgr2=0x7f01e406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.375+0000 7f0207fcd640 1 --2- 172.21.15.16:0/1160677904 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01e406c2b0 0x7f01e406e770 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f01fc001eb0 tx=0x7f01fc009040 comp rx=0 tx=0).stop 2024-09-19T07:54:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.375+0000 7f0207fcd640 1 -- 172.21.15.16:0/1160677904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02000cb570 msgr2=0x7f02001f7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.375+0000 7f0207fcd640 1 --2- 172.21.15.16:0/1160677904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02000cb570 0x7f02001f7650 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f01f8001f40 tx=0x7f01f80014a0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.375+0000 7f0207fcd640 1 -- 172.21.15.16:0/1160677904 shutdown_connections 2024-09-19T07:54:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.375+0000 7f0207fcd640 1 --2- 172.21.15.16:0/1160677904 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01e406c2b0 0x7f01e406e770 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.375+0000 7f0207fcd640 1 --2- 172.21.15.16:0/1160677904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02000cb570 0x7f02001f7650 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.375+0000 7f0207fcd640 1 --2- 172.21.15.16:0/1160677904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02000cabb0 0x7f02000cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.375+0000 7f0207fcd640 1 --2- 172.21.15.16:0/1160677904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01ec004d40 0x7f0200140c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.375+0000 7f0207fcd640 1 -- 172.21.15.16:0/1160677904 >> 172.21.15.16:0/1160677904 conn(0x7f02000b3ed0 msgr2=0x7f020015dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.375+0000 7f0207fcd640 1 -- 172.21.15.16:0/1160677904 shutdown_connections 2024-09-19T07:54:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.375+0000 7f0207fcd640 1 -- 172.21.15.16:0/1160677904 wait complete. 2024-09-19T07:54:18.387 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T07:54:18.387 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T07:54:18.387 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T07:54:18.387 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:54:18.387 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T07:54:18.387 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T07:54:18.388 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T07:54:18.388 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:54:18.388 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T07:54:18.388 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T07:54:18.388 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T07:54:18.388 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:54:18.389 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T07:54:18.389 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T07:54:18.389 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T07:54:18.389 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:54:18.396 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T07:54:18.396 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T07:54:18.397 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T07:54:18.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T07:54:18.400 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T07:54:18.400 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T07:54:18.401 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T07:54:18.401 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T07:54:18.439 INFO:tasks.ceph.mds.a:Started 2024-09-19T07:54:18.439 DEBUG:teuthology.parallel:result is None 2024-09-19T07:54:18.440 INFO:tasks.ceph.mds.c:Started 2024-09-19T07:54:18.440 DEBUG:teuthology.parallel:result is None 2024-09-19T07:54:18.443 INFO:tasks.ceph.mds.b:Started 2024-09-19T07:54:18.444 INFO:tasks.ceph.mds.d:Started 2024-09-19T07:54:18.444 DEBUG:teuthology.parallel:result is None 2024-09-19T07:54:18.445 DEBUG:teuthology.parallel:result is None 2024-09-19T07:54:18.445 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T07:54:18.517 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-19T07:54:18.517 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-19T07:54:18.525 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2024-09-19T07:54:18.525 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2024-09-19T07:54:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.553+0000 7ff9a0c27640 1 Processor -- start 2024-09-19T07:54:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.553+0000 7ff9a0c27640 1 -- start start 2024-09-19T07:54:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.554+0000 7ff9a0c27640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff99c0d0070 0x7ff99c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.554+0000 7ff9a0c27640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff99c0d4130 0x7ff99c0d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.554+0000 7ff9a0c27640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff99c0d4ae0 con 0x7ff99c15f0d0 2024-09-19T07:54:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.555+0000 7ff9a0c27640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff99c0d1b10 con 0x7ff99c0d0070 2024-09-19T07:54:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.555+0000 7ff9a0c27640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff99c0cf2b0 con 0x7ff99c0d4130 2024-09-19T07:54:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.555+0000 7ff999d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff99c0d0070 0x7ff99c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.555+0000 7ff999d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff99c0d0070 0x7ff99c15f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54834/0 (socket says 172.21.15.16:54834) 2024-09-19T07:54:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.555+0000 7ff999d9b640 1 -- 172.21.15.16:0/3388021982 learned_addr learned my addr 172.21.15.16:0/3388021982 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.555+0000 7ff99ad9d640 1 --2- 172.21.15.16:0/3388021982 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff99c0d4130 0x7ff99c0d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.555+0000 7ff999d9b640 1 -- 172.21.15.16:0/3388021982 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff99c0d4130 msgr2=0x7ff99c0d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.555+0000 7ff999d9b640 1 --2- 172.21.15.16:0/3388021982 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff99c0d4130 0x7ff99c0d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.555+0000 7ff999d9b640 1 -- 172.21.15.16:0/3388021982 >> v1:172.21.15.146:6789/0 conn(0x7ff99c15f0d0 legacy=0x7ff99c0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.555+0000 7ff999d9b640 1 -- 172.21.15.16:0/3388021982 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99c0cf590 con 0x7ff99c0d0070 2024-09-19T07:54:18.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.556+0000 7ff999d9b640 1 --2- 172.21.15.16:0/3388021982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff99c0d0070 0x7ff99c15f4b0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7ff990009870 tx=0x7ff99002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4545cb2d3d260fa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:18.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.556+0000 7ff99959a640 1 -- 172.21.15.16:0/3388021982 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990037070 con 0x7ff99c0d0070 2024-09-19T07:54:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.556+0000 7ff99959a640 1 -- 172.21.15.16:0/3388021982 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff990032440 con 0x7ff99c0d0070 2024-09-19T07:54:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.556+0000 7ff99959a640 1 -- 172.21.15.16:0/3388021982 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990032810 con 0x7ff99c0d0070 2024-09-19T07:54:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.556+0000 7ff9a0c27640 1 -- 172.21.15.16:0/3388021982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff99c0d0070 msgr2=0x7ff99c15f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.556+0000 7ff9a0c27640 1 --2- 172.21.15.16:0/3388021982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff99c0d0070 0x7ff99c15f4b0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7ff990009870 tx=0x7ff99002a1c0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.556+0000 7ff9a0c27640 1 -- 172.21.15.16:0/3388021982 shutdown_connections 2024-09-19T07:54:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.556+0000 7ff9a0c27640 1 --2- 172.21.15.16:0/3388021982 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff99c0d4130 0x7ff99c0d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.556+0000 7ff9a0c27640 1 --2- 172.21.15.16:0/3388021982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff99c0d0070 0x7ff99c15f4b0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.556+0000 7ff9a0c27640 1 -- 172.21.15.16:0/3388021982 >> 172.21.15.16:0/3388021982 conn(0x7ff99c09d760 msgr2=0x7ff99c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.557+0000 7ff9a0c27640 1 -- 172.21.15.16:0/3388021982 shutdown_connections 2024-09-19T07:54:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.557+0000 7ff9a0c27640 1 -- 172.21.15.16:0/3388021982 wait complete. 2024-09-19T07:54:18.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.558+0000 7ff9a0c27640 1 Processor -- start 2024-09-19T07:54:18.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.558+0000 7ff9a0c27640 1 -- start start 2024-09-19T07:54:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff9a0c27640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff99c0d0070 0x7ff99c151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff9a0c27640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff99c0d4130 0x7ff99c0cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff9a0c27640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff99c15f0d0 0x7ff99c15d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff9a0c27640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c0cef60 con 0x7ff99c0d0070 2024-09-19T07:54:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff9a0c27640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c0cfb50 con 0x7ff99c15f0d0 2024-09-19T07:54:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff9a0c27640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff99c0b2f70 con 0x7ff99c0d4130 2024-09-19T07:54:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff99a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff99c0d0070 0x7ff99c151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff99a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff99c0d0070 0x7ff99c151850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54846/0 (socket says 172.21.15.16:54846) 2024-09-19T07:54:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff99a59c640 1 -- 172.21.15.16:0/623938019 learned_addr learned my addr 172.21.15.16:0/623938019 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff999d9b640 1 --2- 172.21.15.16:0/623938019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff99c0d4130 0x7ff99c0cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff99ad9d640 1 --2- 172.21.15.16:0/623938019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff99c15f0d0 0x7ff99c15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff99a59c640 1 -- 172.21.15.16:0/623938019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff99c0d4130 msgr2=0x7ff99c0cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff99a59c640 1 --2- 172.21.15.16:0/623938019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff99c0d4130 0x7ff99c0cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff99a59c640 1 -- 172.21.15.16:0/623938019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff99c15f0d0 msgr2=0x7ff99c15d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff99a59c640 1 --2- 172.21.15.16:0/623938019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff99c15f0d0 0x7ff99c15d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff99a59c640 1 -- 172.21.15.16:0/623938019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99c151d90 con 0x7ff99c0d0070 2024-09-19T07:54:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff99a59c640 1 --2- 172.21.15.16:0/623938019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff99c0d0070 0x7ff99c151850 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00c3f0 tx=0x7ff98c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff98affd640 1 -- 172.21.15.16:0/623938019 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c0052d0 con 0x7ff99c0d0070 2024-09-19T07:54:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff98affd640 1 -- 172.21.15.16:0/623938019 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff98c005490 con 0x7ff99c0d0070 2024-09-19T07:54:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff98affd640 1 -- 172.21.15.16:0/623938019 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c007260 con 0x7ff99c0d0070 2024-09-19T07:54:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff9a0c27640 1 -- 172.21.15.16:0/623938019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99c0c8130 con 0x7ff99c0d0070 2024-09-19T07:54:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.559+0000 7ff9a0c27640 1 -- 172.21.15.16:0/623938019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99c0ce0f0 con 0x7ff99c0d0070 2024-09-19T07:54:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.560+0000 7ff9a0c27640 1 -- 172.21.15.16:0/623938019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff99c0d45a0 con 0x7ff99c0d0070 2024-09-19T07:54:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.561+0000 7ff98affd640 1 -- 172.21.15.16:0/623938019 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff98c007400 con 0x7ff99c0d0070 2024-09-19T07:54:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.561+0000 7ff98affd640 1 --2- 172.21.15.16:0/623938019 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff97c06c2b0 0x7ff97c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.562+0000 7ff98affd640 1 -- 172.21.15.16:0/623938019 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7ff98c08a700 con 0x7ff99c0d0070 2024-09-19T07:54:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.562+0000 7ff999d9b640 1 --2- 172.21.15.16:0/623938019 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff97c06c2b0 0x7ff97c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:18.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.562+0000 7ff999d9b640 1 --2- 172.21.15.16:0/623938019 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff97c06c2b0 0x7ff97c06e770 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff990034000 tx=0x7ff99002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:18.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.566+0000 7ff98affd640 1 -- 172.21.15.16:0/623938019 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff99c151d90 con 0x7ff99c0d0070 2024-09-19T07:54:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:18.756+0000 7ff9a0c27640 1 -- 172.21.15.16:0/623938019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff99c0d3390 con 0x7ff99c0d0070 2024-09-19T07:54:19.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.531+0000 7ff98affd640 1 -- 172.21.15.16:0/623938019 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v123) ==== 97+0+0 (secure 0 0 0) 0x7ff98c05de20 con 0x7ff99c0d0070 2024-09-19T07:54:19.531 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-19T07:54:19.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.534+0000 7ff9a0c27640 1 -- 172.21.15.16:0/623938019 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff97c06c2b0 msgr2=0x7ff97c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:19.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.534+0000 7ff9a0c27640 1 --2- 172.21.15.16:0/623938019 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff97c06c2b0 0x7ff97c06e770 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff990034000 tx=0x7ff99002e000 comp rx=0 tx=0).stop 2024-09-19T07:54:19.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.534+0000 7ff9a0c27640 1 -- 172.21.15.16:0/623938019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff99c0d0070 msgr2=0x7ff99c151850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:19.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.534+0000 7ff9a0c27640 1 --2- 172.21.15.16:0/623938019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff99c0d0070 0x7ff99c151850 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00c3f0 tx=0x7ff98c00c8b0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.535+0000 7ff9a0c27640 1 -- 172.21.15.16:0/623938019 shutdown_connections 2024-09-19T07:54:19.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.535+0000 7ff9a0c27640 1 --2- 172.21.15.16:0/623938019 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff97c06c2b0 0x7ff97c06e770 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.535+0000 7ff9a0c27640 1 --2- 172.21.15.16:0/623938019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff99c15f0d0 0x7ff99c15d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.535+0000 7ff9a0c27640 1 --2- 172.21.15.16:0/623938019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff99c0d4130 0x7ff99c0cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.535+0000 7ff9a0c27640 1 --2- 172.21.15.16:0/623938019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff99c0d0070 0x7ff99c151850 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.535+0000 7ff9a0c27640 1 -- 172.21.15.16:0/623938019 >> 172.21.15.16:0/623938019 conn(0x7ff99c09d760 msgr2=0x7ff99c06dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:19.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.535+0000 7ff9a0c27640 1 -- 172.21.15.16:0/623938019 shutdown_connections 2024-09-19T07:54:19.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.535+0000 7ff9a0c27640 1 -- 172.21.15.16:0/623938019 wait complete. 2024-09-19T07:54:19.550 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T07:54:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.658+0000 7fee87dd6640 1 Processor -- start 2024-09-19T07:54:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.658+0000 7fee87dd6640 1 -- start start 2024-09-19T07:54:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.659+0000 7fee87dd6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fee800d0bc0 0x7fee800cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.659+0000 7fee87dd6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee8015c2c0 0x7fee800cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.659+0000 7fee87dd6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee800cfc80 con 0x7fee8015bee0 2024-09-19T07:54:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.659+0000 7fee87dd6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fee800ceaf0 con 0x7fee800d0bc0 2024-09-19T07:54:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.659+0000 7fee87dd6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fee800cdee0 con 0x7fee8015c2c0 2024-09-19T07:54:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.659+0000 7fee85371640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fee800d0bc0 0x7fee800cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.659+0000 7fee85b72640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fee8015bee0 0x7fee800d04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42746/0 (socket says 172.21.15.16:42746) 2024-09-19T07:54:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.659+0000 7fee85b72640 1 -- 172.21.15.16:0/691301240 learned_addr learned my addr 172.21.15.16:0/691301240 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.660+0000 7fee86373640 1 --2- 172.21.15.16:0/691301240 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee8015c2c0 0x7fee800cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.660+0000 7fee85371640 1 -- 172.21.15.16:0/691301240 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee8015c2c0 msgr2=0x7fee800cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.660+0000 7fee85371640 1 --2- 172.21.15.16:0/691301240 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee8015c2c0 0x7fee800cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.660+0000 7fee85371640 1 -- 172.21.15.16:0/691301240 >> v1:172.21.15.16:6789/0 conn(0x7fee8015bee0 legacy=0x7fee800d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.660+0000 7fee84b70640 1 -- 172.21.15.16:0/691301240 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3710672716 0 0) 0x7fee800cfc80 con 0x7fee8015bee0 2024-09-19T07:54:19.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.660+0000 7fee85371640 1 -- 172.21.15.16:0/691301240 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee800ce1c0 con 0x7fee800d0bc0 2024-09-19T07:54:19.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.660+0000 7fee85371640 1 --2- 172.21.15.16:0/691301240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee800d0bc0 0x7fee800cf200 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fee700097a0 tx=0x7fee7002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f993c52cc2a0d826 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:19.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.661+0000 7fee84b70640 1 -- 172.21.15.16:0/691301240 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70037070 con 0x7fee800d0bc0 2024-09-19T07:54:19.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.661+0000 7fee84b70640 1 -- 172.21.15.16:0/691301240 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee70032440 con 0x7fee800d0bc0 2024-09-19T07:54:19.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.661+0000 7fee84b70640 1 -- 172.21.15.16:0/691301240 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70032810 con 0x7fee800d0bc0 2024-09-19T07:54:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.661+0000 7fee87dd6640 1 -- 172.21.15.16:0/691301240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee800d0bc0 msgr2=0x7fee800cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.661+0000 7fee87dd6640 1 --2- 172.21.15.16:0/691301240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee800d0bc0 0x7fee800cf200 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fee700097a0 tx=0x7fee7002a180 comp rx=0 tx=0).stop 2024-09-19T07:54:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.661+0000 7fee87dd6640 1 -- 172.21.15.16:0/691301240 shutdown_connections 2024-09-19T07:54:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.661+0000 7fee87dd6640 1 --2- 172.21.15.16:0/691301240 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee8015c2c0 0x7fee800cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.661+0000 7fee87dd6640 1 --2- 172.21.15.16:0/691301240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee800d0bc0 0x7fee800cf200 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.661+0000 7fee87dd6640 1 -- 172.21.15.16:0/691301240 >> 172.21.15.16:0/691301240 conn(0x7fee8006bdf0 msgr2=0x7fee8006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.661+0000 7fee87dd6640 1 -- 172.21.15.16:0/691301240 shutdown_connections 2024-09-19T07:54:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.661+0000 7fee87dd6640 1 -- 172.21.15.16:0/691301240 wait complete. 2024-09-19T07:54:19.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.663+0000 7fee87dd6640 1 Processor -- start 2024-09-19T07:54:19.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.663+0000 7fee87dd6640 1 -- start start 2024-09-19T07:54:19.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.663+0000 7fee87dd6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee800d0bc0 0x7fee8015c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.663+0000 7fee87dd6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee8015bee0 0x7fee801f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.663+0000 7fee87dd6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee8015c2c0 0x7fee801f2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.663+0000 7fee87dd6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee8006dec0 con 0x7fee8015c2c0 2024-09-19T07:54:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.663+0000 7fee87dd6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fee800ce780 con 0x7fee800d0bc0 2024-09-19T07:54:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.663+0000 7fee87dd6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fee8006e040 con 0x7fee8015bee0 2024-09-19T07:54:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.663+0000 7fee86373640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee8015c2c0 0x7fee801f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.663+0000 7fee86373640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee8015c2c0 0x7fee801f2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54858/0 (socket says 172.21.15.16:54858) 2024-09-19T07:54:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.663+0000 7fee86373640 1 -- 172.21.15.16:0/3544194262 learned_addr learned my addr 172.21.15.16:0/3544194262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.664+0000 7fee85b72640 1 --2- 172.21.15.16:0/3544194262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee800d0bc0 0x7fee8015c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.664+0000 7fee85371640 1 --2- 172.21.15.16:0/3544194262 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee8015bee0 0x7fee801f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.664+0000 7fee86373640 1 -- 172.21.15.16:0/3544194262 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee8015bee0 msgr2=0x7fee801f1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.664+0000 7fee86373640 1 --2- 172.21.15.16:0/3544194262 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee8015bee0 0x7fee801f1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.664+0000 7fee86373640 1 -- 172.21.15.16:0/3544194262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee800d0bc0 msgr2=0x7fee8015c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:19.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.664+0000 7fee86373640 1 --2- 172.21.15.16:0/3544194262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee800d0bc0 0x7fee8015c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.664+0000 7fee86373640 1 -- 172.21.15.16:0/3544194262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee800afb00 con 0x7fee8015c2c0 2024-09-19T07:54:19.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.664+0000 7fee86373640 1 --2- 172.21.15.16:0/3544194262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee8015c2c0 0x7fee801f2150 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fee78007e40 tx=0x7fee7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:19.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.664+0000 7fee767fc640 1 -- 172.21.15.16:0/3544194262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78014620 con 0x7fee8015c2c0 2024-09-19T07:54:19.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.664+0000 7fee767fc640 1 -- 172.21.15.16:0/3544194262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee78019070 con 0x7fee8015c2c0 2024-09-19T07:54:19.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.664+0000 7fee767fc640 1 -- 172.21.15.16:0/3544194262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78014a20 con 0x7fee8015c2c0 2024-09-19T07:54:19.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.664+0000 7fee87dd6640 1 -- 172.21.15.16:0/3544194262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee800afde0 con 0x7fee8015c2c0 2024-09-19T07:54:19.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.664+0000 7fee87dd6640 1 -- 172.21.15.16:0/3544194262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee800b03a0 con 0x7fee8015c2c0 2024-09-19T07:54:19.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.665+0000 7fee87dd6640 1 -- 172.21.15.16:0/3544194262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee800d04b0 con 0x7fee8015c2c0 2024-09-19T07:54:19.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.666+0000 7fee767fc640 1 -- 172.21.15.16:0/3544194262 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fee78014bc0 con 0x7fee8015c2c0 2024-09-19T07:54:19.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.666+0000 7fee767fc640 1 --2- 172.21.15.16:0/3544194262 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fee6406c2b0 0x7fee6406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:19.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.666+0000 7fee767fc640 1 -- 172.21.15.16:0/3544194262 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5249+0+0 (secure 0 0 0) 0x7fee7808ba30 con 0x7fee8015c2c0 2024-09-19T07:54:19.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.666+0000 7fee85b72640 1 --2- 172.21.15.16:0/3544194262 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fee6406c2b0 0x7fee6406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:19.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.667+0000 7fee85b72640 1 --2- 172.21.15.16:0/3544194262 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fee6406c2b0 0x7fee6406e770 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fee7c000f60 tx=0x7fee7c007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:19.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.671+0000 7fee767fc640 1 -- 172.21.15.16:0/3544194262 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fee78059d10 con 0x7fee8015c2c0 2024-09-19T07:54:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.916+0000 7fee87dd6640 1 -- 172.21.15.16:0/3544194262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fee8006c550 con 0x7fee8015c2c0 2024-09-19T07:54:19.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.917+0000 7fee767fc640 1 -- 172.21.15.16:0/3544194262 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7fee7805e010 con 0x7fee8015c2c0 2024-09-19T07:54:19.917 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:19.920 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:54:19.920 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-19T07:54:19.920 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:19.920 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-19T07:54:19.921 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etms80KDxAAqTrG88LTEJIj9TWJu0witQ==", 2024-09-19T07:54:19.921 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:19.921 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:54:19.921 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:54:19.921 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:54:19.921 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:19.921 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:19.921 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:19.921 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-19T07:54:19.921 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etm4T0KGxAAcfDOKrSdKzhJ4xvTFIyxcQ==", 2024-09-19T07:54:19.921 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:19.921 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:54:19.921 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:54:19.922 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:54:19.922 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:19.922 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:19.922 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:19.922 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-19T07:54:19.922 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNDBGFhAAtBioJJ1+NtIDLhd/Hg7aZA==", 2024-09-19T07:54:19.922 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:19.922 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:54:19.922 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:54:19.922 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:54:19.922 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:19.922 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:19.922 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:19.923 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-19T07:54:19.923 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmaHEiIhAACqqjCRMV+xlGh9WqYsj+MA==", 2024-09-19T07:54:19.923 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:19.923 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:54:19.923 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:54:19.923 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:54:19.923 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:19.923 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:19.923 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:19.923 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-19T07:54:19.923 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDT1etm09plCxAAGvJYF6JrKw49FtXb71ox3A==", 2024-09-19T07:54:19.923 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:19.923 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:54:19.924 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:54:19.924 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:19.924 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:19.924 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:19.924 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-19T07:54:19.924 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDW1etmGy8yHRAAlcDsHLnkjtS6sCCR8Olv2A==", 2024-09-19T07:54:19.924 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:19.924 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:54:19.924 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:54:19.924 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:19.924 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:19.924 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:19.925 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-19T07:54:19.925 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDZ1etmMhGTHxAATQaW2kgJhHKftutQQtUa6A==", 2024-09-19T07:54:19.925 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:19.925 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:54:19.925 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:54:19.925 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:19.925 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:19.925 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:19.925 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-19T07:54:19.925 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDc1etmD3sLIhAAIdY00buvb3bIFA5ZpgxxjA==", 2024-09-19T07:54:19.925 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:19.925 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:54:19.925 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:54:19.926 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:19.926 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:19.926 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:19.926 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-19T07:54:19.926 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDi1etmgutcLRAAHJEl5SfQ7N0T4YoYeBfrQg==", 2024-09-19T07:54:19.926 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:19.926 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:54:19.926 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:54:19.927 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:19.927 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:19.927 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:19.927 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-19T07:54:19.927 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDm1etmCV3xARAA+5yhQEmIpyNId3RL1Xsxhw==", 2024-09-19T07:54:19.927 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:19.927 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:54:19.927 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:54:19.927 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:19.927 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:19.927 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:19.927 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-19T07:54:19.928 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDp1etm9XC+AxAA+Lrs0ExfSKaixgOxfX8Nug==", 2024-09-19T07:54:19.928 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:19.928 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:54:19.928 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:54:19.928 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:19.928 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:19.928 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:19.928 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-19T07:54:19.928 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDs1etmhmvMBhAAhvW/udIE2gCd8IrJCSFuvw==", 2024-09-19T07:54:19.929 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:19.929 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:54:19.929 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:54:19.929 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:19.929 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:19.929 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:19.929 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-19T07:54:19.929 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA==", 2024-09-19T07:54:19.929 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:19.930 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:54:19.930 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-19T07:54:19.930 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T07:54:19.930 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:19.930 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:19.930 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-19T07:54:19.930 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmc05KLxAAPQl6wOHqFIf8LVxskiuLVg==", 2024-09-19T07:54:19.930 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:19.930 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:54:19.930 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:54:19.930 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:54:19.930 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:54:19.930 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:19.931 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:19.931 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-19T07:54:19.931 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmjFsCLBAANa7gCMvDw84Ovo7u6RRJ3Q==", 2024-09-19T07:54:19.931 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:19.931 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:54:19.931 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:54:19.931 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:54:19.931 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:54:19.931 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:19.931 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:19.931 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-19T07:54:19.931 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmdUbgNBAAfwstyOeeoZfetgVZu4z5xg==", 2024-09-19T07:54:19.931 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:19.932 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:54:19.932 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:54:19.932 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:54:19.932 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:54:19.932 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.932 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:19.933 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:19.933 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-19T07:54:19.933 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDQ1etm9/vTLhAA9PRfAOA8R4oXp3x0dU8l3g==", 2024-09-19T07:54:19.933 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:19.933 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:54:19.933 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:54:19.933 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:54:19.933 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:19.933 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:19.934 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:19.934 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-19T07:54:19.934 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNQ68CxAANkK+D3u/oTEz8Su4iuZrCg==", 2024-09-19T07:54:19.934 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:19.934 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:54:19.934 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:54:19.934 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:19.934 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.934 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:19.934 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:19.934 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-19T07:54:19.934 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmp+gSCBAAGXzmKfFNd6K2fWeqYhlHAg==", 2024-09-19T07:54:19.934 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:19.935 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:54:19.935 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:54:19.935 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:19.935 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.935 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:19.935 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T07:54:19.935 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:54:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.919+0000 7fee87dd6640 1 -- 172.21.15.16:0/3544194262 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fee6406c2b0 msgr2=0x7fee6406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.919+0000 7fee87dd6640 1 --2- 172.21.15.16:0/3544194262 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fee6406c2b0 0x7fee6406e770 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fee7c000f60 tx=0x7fee7c007d50 comp rx=0 tx=0).stop 2024-09-19T07:54:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.919+0000 7fee87dd6640 1 -- 172.21.15.16:0/3544194262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee8015c2c0 msgr2=0x7fee801f2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.919+0000 7fee87dd6640 1 --2- 172.21.15.16:0/3544194262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee8015c2c0 0x7fee801f2150 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fee78007e40 tx=0x7fee7800e4b0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.919+0000 7fee87dd6640 1 -- 172.21.15.16:0/3544194262 shutdown_connections 2024-09-19T07:54:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.919+0000 7fee87dd6640 1 --2- 172.21.15.16:0/3544194262 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fee6406c2b0 0x7fee6406e770 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.919+0000 7fee87dd6640 1 --2- 172.21.15.16:0/3544194262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee8015c2c0 0x7fee801f2150 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.919+0000 7fee87dd6640 1 --2- 172.21.15.16:0/3544194262 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee8015bee0 0x7fee801f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.919+0000 7fee87dd6640 1 --2- 172.21.15.16:0/3544194262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee800d0bc0 0x7fee8015c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.919+0000 7fee87dd6640 1 -- 172.21.15.16:0/3544194262 >> 172.21.15.16:0/3544194262 conn(0x7fee8006bdf0 msgr2=0x7fee800b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.919+0000 7fee87dd6640 1 -- 172.21.15.16:0/3544194262 shutdown_connections 2024-09-19T07:54:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:19.919+0000 7fee87dd6640 1 -- 172.21.15.16:0/3544194262 wait complete. 2024-09-19T07:54:19.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T07:54:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.040+0000 7fafc1511640 1 Processor -- start 2024-09-19T07:54:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.040+0000 7fafc1511640 1 -- start start 2024-09-19T07:54:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.041+0000 7fafc1511640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fafbc15e680 0x7fafbc0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.041+0000 7fafc1511640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fafbc0d0010 0x7fafbc0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.041+0000 7fafc1511640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafbc0d1880 con 0x7fafbc15e2a0 2024-09-19T07:54:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.041+0000 7fafc1511640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fafbc0d1b60 con 0x7fafbc15e680 2024-09-19T07:54:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.041+0000 7fafc1511640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fafbc0cf2d0 con 0x7fafbc0d0010 2024-09-19T07:54:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.041+0000 7fafba7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fafbc15e680 0x7fafbc0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.041+0000 7fafba7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fafbc15e680 0x7fafbc0cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54864/0 (socket says 172.21.15.16:54864) 2024-09-19T07:54:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.041+0000 7fafba7fc640 1 -- 172.21.15.16:0/2923146360 learned_addr learned my addr 172.21.15.16:0/2923146360 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.041+0000 7fafba7fc640 1 -- 172.21.15.16:0/2923146360 >> v2:172.21.15.146:3300/0 conn(0x7fafbc0d0010 msgr2=0x7fafbc0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.041+0000 7fafba7fc640 1 --2- 172.21.15.16:0/2923146360 >> v2:172.21.15.146:3300/0 conn(0x7fafbc0d0010 0x7fafbc0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.041+0000 7fafba7fc640 1 -- 172.21.15.16:0/2923146360 >> v1:172.21.15.146:6789/0 conn(0x7fafbc15e2a0 legacy=0x7fafbc06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.041+0000 7fafba7fc640 1 -- 172.21.15.16:0/2923146360 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc0cf5b0 con 0x7fafbc15e680 2024-09-19T07:54:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.042+0000 7fafba7fc640 1 --2- 172.21.15.16:0/2923146360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafbc15e680 0x7fafbc0cfad0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fafbc163050 tx=0x7fafb002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10f6c5092e64a2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.042+0000 7fafb9ffb640 1 -- 172.21.15.16:0/2923146360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0037070 con 0x7fafbc15e680 2024-09-19T07:54:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.042+0000 7fafb9ffb640 1 -- 172.21.15.16:0/2923146360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb0032440 con 0x7fafbc15e680 2024-09-19T07:54:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.042+0000 7fafb9ffb640 1 -- 172.21.15.16:0/2923146360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0032810 con 0x7fafbc15e680 2024-09-19T07:54:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.042+0000 7fafc1511640 1 -- 172.21.15.16:0/2923146360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafbc15e680 msgr2=0x7fafbc0cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.042+0000 7fafc1511640 1 --2- 172.21.15.16:0/2923146360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafbc15e680 0x7fafbc0cfad0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fafbc163050 tx=0x7fafb002a1a0 comp rx=0 tx=0).stop 2024-09-19T07:54:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.043+0000 7fafc1511640 1 -- 172.21.15.16:0/2923146360 shutdown_connections 2024-09-19T07:54:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.043+0000 7fafc1511640 1 --2- 172.21.15.16:0/2923146360 >> v2:172.21.15.146:3300/0 conn(0x7fafbc0d0010 0x7fafbc0d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.043+0000 7fafc1511640 1 --2- 172.21.15.16:0/2923146360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafbc15e680 0x7fafbc0cfad0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.043+0000 7fafc1511640 1 -- 172.21.15.16:0/2923146360 >> 172.21.15.16:0/2923146360 conn(0x7fafbc09d760 msgr2=0x7fafbc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.043+0000 7fafc1511640 1 -- 172.21.15.16:0/2923146360 shutdown_connections 2024-09-19T07:54:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.043+0000 7fafc1511640 1 -- 172.21.15.16:0/2923146360 wait complete. 2024-09-19T07:54:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.044+0000 7fafc1511640 1 Processor -- start 2024-09-19T07:54:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.044+0000 7fafc1511640 1 -- start start 2024-09-19T07:54:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.045+0000 7fafc1511640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafbc0d0010 0x7fafbc1f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.045+0000 7fafc1511640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafbc15e2a0 0x7fafbc0ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.045+0000 7fafc1511640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafbc15e680 0x7fafbc0b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.045+0000 7fafc1511640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc06dcb0 con 0x7fafbc15e2a0 2024-09-19T07:54:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.045+0000 7fafc1511640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc06db30 con 0x7fafbc0d0010 2024-09-19T07:54:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.045+0000 7fafc1511640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fafbc0d48b0 con 0x7fafbc15e680 2024-09-19T07:54:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.045+0000 7fafba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafbc15e2a0 0x7fafbc0ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.045+0000 7fafba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafbc15e2a0 0x7fafbc0ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54874/0 (socket says 172.21.15.16:54874) 2024-09-19T07:54:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.045+0000 7fafba7fc640 1 -- 172.21.15.16:0/1941803303 learned_addr learned my addr 172.21.15.16:0/1941803303 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.045+0000 7fafbaffd640 1 --2- 172.21.15.16:0/1941803303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafbc0d0010 0x7fafbc1f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.045+0000 7fafbb7fe640 1 --2- 172.21.15.16:0/1941803303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafbc15e680 0x7fafbc0b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.045+0000 7fafba7fc640 1 -- 172.21.15.16:0/1941803303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafbc15e680 msgr2=0x7fafbc0b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.045+0000 7fafba7fc640 1 --2- 172.21.15.16:0/1941803303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafbc15e680 0x7fafbc0b8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.045+0000 7fafba7fc640 1 -- 172.21.15.16:0/1941803303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafbc0d0010 msgr2=0x7fafbc1f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.045+0000 7fafba7fc640 1 --2- 172.21.15.16:0/1941803303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafbc0d0010 0x7fafbc1f4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.045+0000 7fafba7fc640 1 -- 172.21.15.16:0/1941803303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc0c7f10 con 0x7fafbc15e2a0 2024-09-19T07:54:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.045+0000 7fafbb7fe640 1 --2- 172.21.15.16:0/1941803303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafbc15e680 0x7fafbc0b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.045+0000 7fafba7fc640 1 --2- 172.21.15.16:0/1941803303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafbc15e2a0 0x7fafbc0ca770 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fafb002a1e0 tx=0x7fafb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.046+0000 7fafa37fe640 1 -- 172.21.15.16:0/1941803303 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0041070 con 0x7fafbc15e2a0 2024-09-19T07:54:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.046+0000 7fafa37fe640 1 -- 172.21.15.16:0/1941803303 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb003c890 con 0x7fafbc15e2a0 2024-09-19T07:54:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.046+0000 7fafc1511640 1 -- 172.21.15.16:0/1941803303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafbc0c81b0 con 0x7fafbc15e2a0 2024-09-19T07:54:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.046+0000 7fafa37fe640 1 -- 172.21.15.16:0/1941803303 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0037040 con 0x7fafbc15e2a0 2024-09-19T07:54:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.046+0000 7fafc1511640 1 -- 172.21.15.16:0/1941803303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafbc0c8770 con 0x7fafbc15e2a0 2024-09-19T07:54:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.047+0000 7fafc1511640 1 -- 172.21.15.16:0/1941803303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafbc0d2440 con 0x7fafbc15e2a0 2024-09-19T07:54:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.047+0000 7fafa37fe640 1 -- 172.21.15.16:0/1941803303 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fafb0044070 con 0x7fafbc15e2a0 2024-09-19T07:54:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.048+0000 7fafa37fe640 1 --2- 172.21.15.16:0/1941803303 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fafa806c2b0 0x7fafa806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.048+0000 7fafa37fe640 1 -- 172.21.15.16:0/1941803303 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5249+0+0 (secure 0 0 0) 0x7fafb00ac070 con 0x7fafbc15e2a0 2024-09-19T07:54:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.048+0000 7fafbaffd640 1 --2- 172.21.15.16:0/1941803303 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fafa806c2b0 0x7fafa806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.049+0000 7fafbaffd640 1 --2- 172.21.15.16:0/1941803303 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fafa806c2b0 0x7fafa806e770 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fafac007fd0 tx=0x7fafac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.052+0000 7fafa37fe640 1 -- 172.21.15.16:0/1941803303 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fafbc0d2440 con 0x7fafbc15e2a0 2024-09-19T07:54:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.229+0000 7fafc1511640 1 -- 172.21.15.16:0/1941803303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fafbc0d4c70 con 0x7fafbc15e2a0 2024-09-19T07:54:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.229+0000 7fafa37fe640 1 -- 172.21.15.16:0/1941803303 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v125) ==== 84+0+65 (secure 0 0 0) 0x7fafb007aac0 con 0x7fafbc15e2a0 2024-09-19T07:54:20.230 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:20.230 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-19T07:54:20.230 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:20.230 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-19T07:54:20.230 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-19T07:54:20.230 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:20.230 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-19T07:54:20.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.231+0000 7fafc1511640 1 -- 172.21.15.16:0/1941803303 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fafa806c2b0 msgr2=0x7fafa806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:20.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.231+0000 7fafc1511640 1 --2- 172.21.15.16:0/1941803303 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fafa806c2b0 0x7fafa806e770 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fafac007fd0 tx=0x7fafac009040 comp rx=0 tx=0).stop 2024-09-19T07:54:20.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.231+0000 7fafc1511640 1 -- 172.21.15.16:0/1941803303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafbc15e2a0 msgr2=0x7fafbc0ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:20.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.231+0000 7fafc1511640 1 --2- 172.21.15.16:0/1941803303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafbc15e2a0 0x7fafbc0ca770 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fafb002a1e0 tx=0x7fafb00046c0 comp rx=0 tx=0).stop 2024-09-19T07:54:20.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.231+0000 7fafc1511640 1 -- 172.21.15.16:0/1941803303 shutdown_connections 2024-09-19T07:54:20.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.231+0000 7fafc1511640 1 --2- 172.21.15.16:0/1941803303 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fafa806c2b0 0x7fafa806e770 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:20.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.231+0000 7fafc1511640 1 --2- 172.21.15.16:0/1941803303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafbc15e680 0x7fafbc0b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:20.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.231+0000 7fafc1511640 1 --2- 172.21.15.16:0/1941803303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafbc15e2a0 0x7fafbc0ca770 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:20.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.231+0000 7fafc1511640 1 --2- 172.21.15.16:0/1941803303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafbc0d0010 0x7fafbc1f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:20.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.231+0000 7fafc1511640 1 -- 172.21.15.16:0/1941803303 >> 172.21.15.16:0/1941803303 conn(0x7fafbc09d760 msgr2=0x7fafbc06d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.231+0000 7fafc1511640 1 -- 172.21.15.16:0/1941803303 shutdown_connections 2024-09-19T07:54:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.231+0000 7fafc1511640 1 -- 172.21.15.16:0/1941803303 wait complete. 2024-09-19T07:54:20.242 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T07:54:20.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T07:54:20.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.349+0000 7ff300ca5640 1 Processor -- start 2024-09-19T07:54:20.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.349+0000 7ff300ca5640 1 -- start start 2024-09-19T07:54:20.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.350+0000 7ff300ca5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2fc1619b0 0x7ff2fc0d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:20.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.350+0000 7ff300ca5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2fc0b0a80 0x7ff2fc0d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:20.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.350+0000 7ff300ca5640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff2fc161d90 0x7ff2fc0cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:20.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.350+0000 7ff300ca5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff2fc0d0ec0 con 0x7ff2fc0b0a80 2024-09-19T07:54:20.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.350+0000 7ff300ca5640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff2fc06c990 con 0x7ff2fc161d90 2024-09-19T07:54:20.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.350+0000 7ff300ca5640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff2fc0cec10 con 0x7ff2fc1619b0 2024-09-19T07:54:20.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.350+0000 7ff2f9d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2fc0b0a80 0x7ff2fc0d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:20.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.350+0000 7ff2f9d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2fc0b0a80 0x7ff2fc0d0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54876/0 (socket says 172.21.15.16:54876) 2024-09-19T07:54:20.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.350+0000 7ff2f9d9b640 1 -- 172.21.15.16:0/1131688588 learned_addr learned my addr 172.21.15.16:0/1131688588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:20.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.350+0000 7ff2fad9d640 1 --2- 172.21.15.16:0/1131688588 >> v2:172.21.15.146:3300/0 conn(0x7ff2fc161d90 0x7ff2fc0cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:20.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.350+0000 7ff2fa59c640 1 --2- 172.21.15.16:0/1131688588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2fc1619b0 0x7ff2fc0d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:20.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.351+0000 7ff2f9d9b640 1 -- 172.21.15.16:0/1131688588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2fc1619b0 msgr2=0x7ff2fc0d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:20.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.351+0000 7ff2f9d9b640 1 --2- 172.21.15.16:0/1131688588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2fc1619b0 0x7ff2fc0d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:20.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.351+0000 7ff2f9d9b640 1 -- 172.21.15.16:0/1131688588 >> v2:172.21.15.146:3300/0 conn(0x7ff2fc161d90 msgr2=0x7ff2fc0cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:20.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.351+0000 7ff2f9d9b640 1 --2- 172.21.15.16:0/1131688588 >> v2:172.21.15.146:3300/0 conn(0x7ff2fc161d90 0x7ff2fc0cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:20.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.351+0000 7ff2f9d9b640 1 -- 172.21.15.16:0/1131688588 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2fc0ce930 con 0x7ff2fc0b0a80 2024-09-19T07:54:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.351+0000 7ff2f9d9b640 1 --2- 172.21.15.16:0/1131688588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc0b0a80 0x7ff2fc0d0980 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0cd2c0 tx=0x7ff2e402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1894835d1a5e908 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.351+0000 7ff2f959a640 1 -- 172.21.15.16:0/1131688588 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4037070 con 0x7ff2fc0b0a80 2024-09-19T07:54:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.352+0000 7ff2f959a640 1 -- 172.21.15.16:0/1131688588 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e4032440 con 0x7ff2fc0b0a80 2024-09-19T07:54:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.352+0000 7ff2f959a640 1 -- 172.21.15.16:0/1131688588 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4032810 con 0x7ff2fc0b0a80 2024-09-19T07:54:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.352+0000 7ff300ca5640 1 -- 172.21.15.16:0/1131688588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc0b0a80 msgr2=0x7ff2fc0d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.352+0000 7ff300ca5640 1 --2- 172.21.15.16:0/1131688588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc0b0a80 0x7ff2fc0d0980 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0cd2c0 tx=0x7ff2e402a1e0 comp rx=0 tx=0).stop 2024-09-19T07:54:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.352+0000 7ff300ca5640 1 -- 172.21.15.16:0/1131688588 shutdown_connections 2024-09-19T07:54:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.352+0000 7ff300ca5640 1 --2- 172.21.15.16:0/1131688588 >> v2:172.21.15.146:3300/0 conn(0x7ff2fc161d90 0x7ff2fc0cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.352+0000 7ff300ca5640 1 --2- 172.21.15.16:0/1131688588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc0b0a80 0x7ff2fc0d0980 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.352+0000 7ff300ca5640 1 --2- 172.21.15.16:0/1131688588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2fc1619b0 0x7ff2fc0d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.352+0000 7ff300ca5640 1 -- 172.21.15.16:0/1131688588 >> 172.21.15.16:0/1131688588 conn(0x7ff2fc09a9f0 msgr2=0x7ff2fc0b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.352+0000 7ff300ca5640 1 -- 172.21.15.16:0/1131688588 shutdown_connections 2024-09-19T07:54:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.352+0000 7ff300ca5640 1 -- 172.21.15.16:0/1131688588 wait complete. 2024-09-19T07:54:20.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.354+0000 7ff300ca5640 1 Processor -- start 2024-09-19T07:54:20.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.354+0000 7ff300ca5640 1 -- start start 2024-09-19T07:54:20.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.354+0000 7ff300ca5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2fc0b0a80 0x7ff2fc1f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:20.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.354+0000 7ff300ca5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2fc1619b0 0x7ff2fc15b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:20.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.354+0000 7ff300ca5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc161d90 0x7ff2fc0c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:20.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.354+0000 7ff300ca5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc06c620 con 0x7ff2fc161d90 2024-09-19T07:54:20.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.354+0000 7ff300ca5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc0b0820 con 0x7ff2fc0b0a80 2024-09-19T07:54:20.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.354+0000 7ff300ca5640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff2fc0cd330 con 0x7ff2fc1619b0 2024-09-19T07:54:20.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.354+0000 7ff2fad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc161d90 0x7ff2fc0c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:20.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.354+0000 7ff2fad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc161d90 0x7ff2fc0c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54878/0 (socket says 172.21.15.16:54878) 2024-09-19T07:54:20.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.354+0000 7ff2fad9d640 1 -- 172.21.15.16:0/1870340061 learned_addr learned my addr 172.21.15.16:0/1870340061 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:20.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.355+0000 7ff2fa59c640 1 --2- 172.21.15.16:0/1870340061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2fc0b0a80 0x7ff2fc1f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:20.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.355+0000 7ff2f9d9b640 1 --2- 172.21.15.16:0/1870340061 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2fc1619b0 0x7ff2fc15b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:20.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.355+0000 7ff2fad9d640 1 -- 172.21.15.16:0/1870340061 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2fc1619b0 msgr2=0x7ff2fc15b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:20.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.355+0000 7ff2fad9d640 1 --2- 172.21.15.16:0/1870340061 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2fc1619b0 0x7ff2fc15b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:20.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.355+0000 7ff2fad9d640 1 -- 172.21.15.16:0/1870340061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2fc0b0a80 msgr2=0x7ff2fc1f3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:20.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.355+0000 7ff2fad9d640 1 --2- 172.21.15.16:0/1870340061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2fc0b0a80 0x7ff2fc1f3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:20.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.355+0000 7ff2fad9d640 1 -- 172.21.15.16:0/1870340061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2fc15b7a0 con 0x7ff2fc161d90 2024-09-19T07:54:20.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.355+0000 7ff2fad9d640 1 --2- 172.21.15.16:0/1870340061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc161d90 0x7ff2fc0c8e80 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400e7f0 tx=0x7ff2f400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:20.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.355+0000 7ff2f2ffd640 1 -- 172.21.15.16:0/1870340061 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4017070 con 0x7ff2fc161d90 2024-09-19T07:54:20.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.355+0000 7ff2f2ffd640 1 -- 172.21.15.16:0/1870340061 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f400ee40 con 0x7ff2fc161d90 2024-09-19T07:54:20.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.355+0000 7ff300ca5640 1 -- 172.21.15.16:0/1870340061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2fc1f4490 con 0x7ff2fc161d90 2024-09-19T07:54:20.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.355+0000 7ff2f2ffd640 1 -- 172.21.15.16:0/1870340061 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f400cb80 con 0x7ff2fc161d90 2024-09-19T07:54:20.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.355+0000 7ff300ca5640 1 -- 172.21.15.16:0/1870340061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2fc1f4a50 con 0x7ff2fc161d90 2024-09-19T07:54:20.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.356+0000 7ff300ca5640 1 -- 172.21.15.16:0/1870340061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2fc0d0440 con 0x7ff2fc161d90 2024-09-19T07:54:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.357+0000 7ff2f2ffd640 1 -- 172.21.15.16:0/1870340061 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff2f401a070 con 0x7ff2fc161d90 2024-09-19T07:54:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.357+0000 7ff2f2ffd640 1 --2- 172.21.15.16:0/1870340061 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff2dc06c2b0 0x7ff2dc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.357+0000 7ff2f2ffd640 1 -- 172.21.15.16:0/1870340061 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5249+0+0 (secure 0 0 0) 0x7ff2f408b4d0 con 0x7ff2fc161d90 2024-09-19T07:54:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.357+0000 7ff2fa59c640 1 --2- 172.21.15.16:0/1870340061 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff2dc06c2b0 0x7ff2dc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.358+0000 7ff2fa59c640 1 --2- 172.21.15.16:0/1870340061 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff2dc06c2b0 0x7ff2dc06e770 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0c7ef0 tx=0x7ff2ec00d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.361+0000 7ff2f2ffd640 1 -- 172.21.15.16:0/1870340061 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff2f4059df0 con 0x7ff2fc161d90 2024-09-19T07:54:20.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:20.540+0000 7ff300ca5640 1 -- 172.21.15.16:0/1870340061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff2fc0753e0 con 0x7ff2fc161d90 2024-09-19T07:54:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.539+0000 7ff2f2ffd640 1 -- 172.21.15.16:0/1870340061 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v126) ==== 138+0+0 (secure 0 0 0) 0x7ff2f40596f0 con 0x7ff2fc161d90 2024-09-19T07:54:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-19T07:54:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.543+0000 7ff300ca5640 1 -- 172.21.15.16:0/1870340061 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff2dc06c2b0 msgr2=0x7ff2dc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.543+0000 7ff300ca5640 1 --2- 172.21.15.16:0/1870340061 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff2dc06c2b0 0x7ff2dc06e770 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0c7ef0 tx=0x7ff2ec00d040 comp rx=0 tx=0).stop 2024-09-19T07:54:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.543+0000 7ff300ca5640 1 -- 172.21.15.16:0/1870340061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc161d90 msgr2=0x7ff2fc0c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.543+0000 7ff300ca5640 1 --2- 172.21.15.16:0/1870340061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc161d90 0x7ff2fc0c8e80 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400e7f0 tx=0x7ff2f400ebf0 comp rx=0 tx=0).stop 2024-09-19T07:54:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.544+0000 7ff300ca5640 1 -- 172.21.15.16:0/1870340061 shutdown_connections 2024-09-19T07:54:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.544+0000 7ff300ca5640 1 --2- 172.21.15.16:0/1870340061 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff2dc06c2b0 0x7ff2dc06e770 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.544+0000 7ff300ca5640 1 --2- 172.21.15.16:0/1870340061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc161d90 0x7ff2fc0c8e80 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.544+0000 7ff300ca5640 1 --2- 172.21.15.16:0/1870340061 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2fc1619b0 0x7ff2fc15b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:21.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.544+0000 7ff300ca5640 1 --2- 172.21.15.16:0/1870340061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2fc0b0a80 0x7ff2fc1f3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:21.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.544+0000 7ff300ca5640 1 -- 172.21.15.16:0/1870340061 >> 172.21.15.16:0/1870340061 conn(0x7ff2fc09a9f0 msgr2=0x7ff2fc0ae270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:21.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.545+0000 7ff300ca5640 1 -- 172.21.15.16:0/1870340061 shutdown_connections 2024-09-19T07:54:21.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.545+0000 7ff300ca5640 1 -- 172.21.15.16:0/1870340061 wait complete. 2024-09-19T07:54:21.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T07:54:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.678+0000 7fb785542640 1 Processor -- start 2024-09-19T07:54:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.678+0000 7fb785542640 1 -- start start 2024-09-19T07:54:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.679+0000 7fb785542640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb7800b5280 0x7fb780162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.679+0000 7fb785542640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb78015f250 con 0x7fb7800d4af0 2024-09-19T07:54:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.679+0000 7fb785542640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb78015f530 con 0x7fb78015e0a0 2024-09-19T07:54:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.679+0000 7fb785542640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb78015f810 con 0x7fb7800b5280 2024-09-19T07:54:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.679+0000 7fb77e7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb7800d4af0 0x7fb7800b4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42760/0 (socket says 172.21.15.16:42760) 2024-09-19T07:54:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.679+0000 7fb77e7fc640 1 -- 172.21.15.16:0/3132868756 learned_addr learned my addr 172.21.15.16:0/3132868756 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.679+0000 7fb77dffb640 1 -- 172.21.15.16:0/3132868756 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1136964764 0 0) 0x7fb78015f250 con 0x7fb7800d4af0 2024-09-19T07:54:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.679+0000 7fb77f7fe640 1 --2- 172.21.15.16:0/3132868756 >> v2:172.21.15.146:3300/0 conn(0x7fb7800b5280 0x7fb780162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.679+0000 7fb77dffb640 1 -- 172.21.15.16:0/3132868756 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb768003520 con 0x7fb7800d4af0 2024-09-19T07:54:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.679+0000 7fb77dffb640 1 -- 172.21.15.16:0/3132868756 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2761116952 0 0) 0x7fb768003520 con 0x7fb7800d4af0 2024-09-19T07:54:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.679+0000 7fb77dffb640 1 -- 172.21.15.16:0/3132868756 >> v2:172.21.15.146:3300/0 conn(0x7fb7800b5280 msgr2=0x7fb780162a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:21.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.679+0000 7fb77dffb640 1 --2- 172.21.15.16:0/3132868756 >> v2:172.21.15.146:3300/0 conn(0x7fb7800b5280 0x7fb780162a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:21.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.679+0000 7fb77dffb640 1 -- 172.21.15.16:0/3132868756 >> v1:172.21.15.146:6789/0 conn(0x7fb78015e0a0 legacy=0x7fb7800d43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:21.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.679+0000 7fb77dffb640 1 -- 172.21.15.16:0/3132868756 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7800b4650 con 0x7fb7800d4af0 2024-09-19T07:54:21.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.679+0000 7fb77dffb640 1 -- 172.21.15.16:0/3132868756 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fb774001810 con 0x7fb7800d4af0 2024-09-19T07:54:21.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.679+0000 7fb77dffb640 1 -- 172.21.15.16:0/3132868756 >> v1:172.21.15.16:6789/0 conn(0x7fb7800d4af0 legacy=0x7fb7800b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:21.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.679+0000 7fb77dffb640 1 --2- 172.21.15.16:0/3132868756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7680039b0 0x7fb768003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:21.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.679+0000 7fb77dffb640 1 --2- 172.21.15.16:0/3132868756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb768004800 0x7fb768004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:21.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.679+0000 7fb77dffb640 1 --2- 172.21.15.16:0/3132868756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7680051c0 0x7fb76800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:21.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.680+0000 7fb77dffb640 1 -- 172.21.15.16:0/3132868756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb78015f810 con 0x7fb7680039b0 2024-09-19T07:54:21.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.680+0000 7fb77dffb640 1 -- 172.21.15.16:0/3132868756 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb76800c060 con 0x7fb7680051c0 2024-09-19T07:54:21.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.680+0000 7fb77dffb640 1 -- 172.21.15.16:0/3132868756 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb76800c260 con 0x7fb768004800 2024-09-19T07:54:21.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.680+0000 7fb77effd640 1 --2- 172.21.15.16:0/3132868756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7680039b0 0x7fb768003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:21.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.680+0000 7fb77e7fc640 1 --2- 172.21.15.16:0/3132868756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb768004800 0x7fb768004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:21.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.680+0000 7fb77f7fe640 1 --2- 172.21.15.16:0/3132868756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7680051c0 0x7fb76800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:21.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.680+0000 7fb77effd640 1 -- 172.21.15.16:0/3132868756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb768004800 msgr2=0x7fb768004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:21.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.680+0000 7fb77effd640 1 --2- 172.21.15.16:0/3132868756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb768004800 0x7fb768004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:21.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.680+0000 7fb77effd640 1 -- 172.21.15.16:0/3132868756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7680051c0 msgr2=0x7fb76800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:21.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.680+0000 7fb77effd640 1 --2- 172.21.15.16:0/3132868756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7680051c0 0x7fb76800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:21.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.680+0000 7fb77effd640 1 -- 172.21.15.16:0/3132868756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb768003520 con 0x7fb7680039b0 2024-09-19T07:54:21.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.680+0000 7fb77f7fe640 1 -- 172.21.15.16:0/3132868756 reap_dead start 2024-09-19T07:54:21.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.680+0000 7fb77e7fc640 1 --2- 172.21.15.16:0/3132868756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb768004800 0x7fb768004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:21.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.681+0000 7fb77effd640 1 --2- 172.21.15.16:0/3132868756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7680039b0 0x7fb768003dd0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fb770001750 tx=0x7fb768005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ae278cd0959d0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:21.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.681+0000 7fb77dffb640 1 -- 172.21.15.16:0/3132868756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770001370 con 0x7fb7680039b0 2024-09-19T07:54:21.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.681+0000 7fb77dffb640 1 -- 172.21.15.16:0/3132868756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7700052d0 con 0x7fb7680039b0 2024-09-19T07:54:21.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.681+0000 7fb77dffb640 1 -- 172.21.15.16:0/3132868756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77002abc0 con 0x7fb7680039b0 2024-09-19T07:54:21.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.681+0000 7fb785542640 1 -- 172.21.15.16:0/3132868756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7680039b0 msgr2=0x7fb768003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:21.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.681+0000 7fb785542640 1 --2- 172.21.15.16:0/3132868756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7680039b0 0x7fb768003dd0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fb770001750 tx=0x7fb768005830 comp rx=0 tx=0).stop 2024-09-19T07:54:21.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.681+0000 7fb785542640 1 -- 172.21.15.16:0/3132868756 shutdown_connections 2024-09-19T07:54:21.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.681+0000 7fb785542640 1 --2- 172.21.15.16:0/3132868756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7680039b0 0x7fb768003dd0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:21.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.681+0000 7fb785542640 1 -- 172.21.15.16:0/3132868756 >> 172.21.15.16:0/3132868756 conn(0x7fb78009d770 msgr2=0x7fb78008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:21.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.681+0000 7fb785542640 1 -- 172.21.15.16:0/3132868756 shutdown_connections 2024-09-19T07:54:21.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.681+0000 7fb785542640 1 -- 172.21.15.16:0/3132868756 wait complete. 2024-09-19T07:54:21.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.683+0000 7fb785542640 1 Processor -- start 2024-09-19T07:54:21.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.683+0000 7fb785542640 1 -- start start 2024-09-19T07:54:21.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb785542640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7680039b0 0x7fb78015beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:21.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb785542640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7800cbe30 0x7fb7800b9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:21.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb785542640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7800cc2b0 0x7fb7800b95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:21.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb785542640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb780162e00 con 0x7fb7680039b0 2024-09-19T07:54:21.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb785542640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb780162c80 con 0x7fb7800cbe30 2024-09-19T07:54:21.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb785542640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb7800b4b70 con 0x7fb7800cc2b0 2024-09-19T07:54:21.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb77effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7680039b0 0x7fb78015beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:21.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb77effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7680039b0 0x7fb78015beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54892/0 (socket says 172.21.15.16:54892) 2024-09-19T07:54:21.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb77effd640 1 -- 172.21.15.16:0/1085159942 learned_addr learned my addr 172.21.15.16:0/1085159942 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:21.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb77e7fc640 1 --2- 172.21.15.16:0/1085159942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7800cbe30 0x7fb7800b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:21.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb77f7fe640 1 --2- 172.21.15.16:0/1085159942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7800cc2b0 0x7fb7800b95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:21.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb77effd640 1 -- 172.21.15.16:0/1085159942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7800cc2b0 msgr2=0x7fb7800b95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:21.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb77effd640 1 --2- 172.21.15.16:0/1085159942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7800cc2b0 0x7fb7800b95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:21.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb77effd640 1 -- 172.21.15.16:0/1085159942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7800cbe30 msgr2=0x7fb7800b9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:21.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb77effd640 1 --2- 172.21.15.16:0/1085159942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7800cbe30 0x7fb7800b9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:21.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb77effd640 1 -- 172.21.15.16:0/1085159942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7800b82c0 con 0x7fb7680039b0 2024-09-19T07:54:21.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb77effd640 1 --2- 172.21.15.16:0/1085159942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7680039b0 0x7fb78015beb0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fb77002a570 tx=0x7fb770001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:21.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb7677fe640 1 -- 172.21.15.16:0/1085159942 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77003f030 con 0x7fb7680039b0 2024-09-19T07:54:21.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb7677fe640 1 -- 172.21.15.16:0/1085159942 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb770005480 con 0x7fb7680039b0 2024-09-19T07:54:21.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb785542640 1 -- 172.21.15.16:0/1085159942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7800b85a0 con 0x7fb7680039b0 2024-09-19T07:54:21.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb7677fe640 1 -- 172.21.15.16:0/1085159942 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770037040 con 0x7fb7680039b0 2024-09-19T07:54:21.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.684+0000 7fb785542640 1 -- 172.21.15.16:0/1085159942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7800b8b60 con 0x7fb7680039b0 2024-09-19T07:54:21.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.685+0000 7fb785542640 1 -- 172.21.15.16:0/1085159942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb780073830 con 0x7fb7680039b0 2024-09-19T07:54:21.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.686+0000 7fb7677fe640 1 -- 172.21.15.16:0/1085159942 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb770042070 con 0x7fb7680039b0 2024-09-19T07:54:21.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.687+0000 7fb7677fe640 1 --2- 172.21.15.16:0/1085159942 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb76c06c2e0 0x7fb76c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:21.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.687+0000 7fb7677fe640 1 -- 172.21.15.16:0/1085159942 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5619+0+0 (secure 0 0 0) 0x7fb7700aad30 con 0x7fb7680039b0 2024-09-19T07:54:21.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.687+0000 7fb77e7fc640 1 --2- 172.21.15.16:0/1085159942 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb76c06c2e0 0x7fb76c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:21.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.687+0000 7fb77e7fc640 1 --2- 172.21.15.16:0/1085159942 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb76c06c2e0 0x7fb76c06e7a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb7800ce720 tx=0x7fb774009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:21.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.692+0000 7fb7677fe640 1 -- 172.21.15.16:0/1085159942 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb780073830 con 0x7fb7680039b0 2024-09-19T07:54:21.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:21.867+0000 7fb785542640 1 -- 172.21.15.16:0/1085159942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb7800b4f00 con 0x7fb7680039b0 2024-09-19T07:54:22.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.542+0000 7fb7677fe640 1 -- 172.21.15.16:0/1085159942 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v127) ==== 170+0+0 (secure 0 0 0) 0x7fb770079450 con 0x7fb7680039b0 2024-09-19T07:54:22.542 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-19T07:54:22.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.545+0000 7fb785542640 1 -- 172.21.15.16:0/1085159942 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb76c06c2e0 msgr2=0x7fb76c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:22.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.545+0000 7fb785542640 1 --2- 172.21.15.16:0/1085159942 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb76c06c2e0 0x7fb76c06e7a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb7800ce720 tx=0x7fb774009040 comp rx=0 tx=0).stop 2024-09-19T07:54:22.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.545+0000 7fb785542640 1 -- 172.21.15.16:0/1085159942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7680039b0 msgr2=0x7fb78015beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:22.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.545+0000 7fb785542640 1 --2- 172.21.15.16:0/1085159942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7680039b0 0x7fb78015beb0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fb77002a570 tx=0x7fb770001120 comp rx=0 tx=0).stop 2024-09-19T07:54:22.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.545+0000 7fb785542640 1 -- 172.21.15.16:0/1085159942 shutdown_connections 2024-09-19T07:54:22.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.545+0000 7fb785542640 1 --2- 172.21.15.16:0/1085159942 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb76c06c2e0 0x7fb76c06e7a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:22.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.545+0000 7fb785542640 1 --2- 172.21.15.16:0/1085159942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb7800cc2b0 0x7fb7800b95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:22.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.545+0000 7fb785542640 1 --2- 172.21.15.16:0/1085159942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7800cbe30 0x7fb7800b9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:22.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.545+0000 7fb785542640 1 --2- 172.21.15.16:0/1085159942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7680039b0 0x7fb78015beb0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:22.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.545+0000 7fb785542640 1 -- 172.21.15.16:0/1085159942 >> 172.21.15.16:0/1085159942 conn(0x7fb78009d770 msgr2=0x7fb7800cfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:22.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.545+0000 7fb785542640 1 -- 172.21.15.16:0/1085159942 shutdown_connections 2024-09-19T07:54:22.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.545+0000 7fb785542640 1 -- 172.21.15.16:0/1085159942 wait complete. 2024-09-19T07:54:22.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T07:54:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.721+0000 7f4254966640 1 Processor -- start 2024-09-19T07:54:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.722+0000 7f4254966640 1 -- start start 2024-09-19T07:54:22.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.722+0000 7f4254966640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f425006cd20 0x7f425006d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:22.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.722+0000 7f4254966640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4250161eb0 0x7f42501622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:22.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.723+0000 7f4254966640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f425006d640 con 0x7f4250143a20 2024-09-19T07:54:22.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.723+0000 7f4254966640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4250162e50 con 0x7f425006cd20 2024-09-19T07:54:22.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.723+0000 7f4254966640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4250142150 con 0x7f4250161eb0 2024-09-19T07:54:22.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.723+0000 7f424ed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4250143a20 0x7f4250143ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56736/0 (socket says 172.21.15.16:56736) 2024-09-19T07:54:22.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.723+0000 7f424ed9d640 1 -- 172.21.15.16:0/6760231 learned_addr learned my addr 172.21.15.16:0/6760231 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:22.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.723+0000 7f424e59c640 1 --2- 172.21.15.16:0/6760231 >> v2:172.21.15.146:3300/0 conn(0x7f425006cd20 0x7f425006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:22.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.723+0000 7f424dd9b640 1 --2- 172.21.15.16:0/6760231 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4250161eb0 0x7f42501622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:22.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.723+0000 7f424d59a640 1 -- 172.21.15.16:0/6760231 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1292781916 0 0) 0x7f425006d640 con 0x7f4250143a20 2024-09-19T07:54:22.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.723+0000 7f424d59a640 1 -- 172.21.15.16:0/6760231 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f423c003520 con 0x7f4250143a20 2024-09-19T07:54:22.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.723+0000 7f424e59c640 1 -- 172.21.15.16:0/6760231 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4250161eb0 msgr2=0x7f42501622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:22.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.723+0000 7f424e59c640 1 --2- 172.21.15.16:0/6760231 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4250161eb0 0x7f42501622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:22.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.723+0000 7f424e59c640 1 -- 172.21.15.16:0/6760231 >> v1:172.21.15.16:6789/0 conn(0x7f4250143a20 legacy=0x7f4250143ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:22.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.723+0000 7f424e59c640 1 -- 172.21.15.16:0/6760231 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4250162b70 con 0x7f425006cd20 2024-09-19T07:54:22.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.724+0000 7f424e59c640 1 --2- 172.21.15.16:0/6760231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f425006cd20 0x7f425006d100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f42480097a0 tx=0x7f424802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c9ec073626117ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:22.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.724+0000 7f424d59a640 1 -- 172.21.15.16:0/6760231 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248037070 con 0x7f425006cd20 2024-09-19T07:54:22.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.724+0000 7f424d59a640 1 -- 172.21.15.16:0/6760231 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f424802ae50 con 0x7f425006cd20 2024-09-19T07:54:22.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.724+0000 7f424d59a640 1 -- 172.21.15.16:0/6760231 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248032660 con 0x7f425006cd20 2024-09-19T07:54:22.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.725+0000 7f4254966640 1 -- 172.21.15.16:0/6760231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f425006cd20 msgr2=0x7f425006d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:22.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.725+0000 7f4254966640 1 --2- 172.21.15.16:0/6760231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f425006cd20 0x7f425006d100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f42480097a0 tx=0x7f424802a100 comp rx=0 tx=0).stop 2024-09-19T07:54:22.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.725+0000 7f4254966640 1 -- 172.21.15.16:0/6760231 shutdown_connections 2024-09-19T07:54:22.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.725+0000 7f4254966640 1 --2- 172.21.15.16:0/6760231 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4250161eb0 0x7f42501622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:22.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.725+0000 7f4254966640 1 --2- 172.21.15.16:0/6760231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f425006cd20 0x7f425006d100 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:22.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.725+0000 7f4254966640 1 -- 172.21.15.16:0/6760231 >> 172.21.15.16:0/6760231 conn(0x7f42500b3ed0 msgr2=0x7f42500b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:22.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.725+0000 7f4254966640 1 -- 172.21.15.16:0/6760231 shutdown_connections 2024-09-19T07:54:22.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.725+0000 7f4254966640 1 -- 172.21.15.16:0/6760231 wait complete. 2024-09-19T07:54:22.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.726+0000 7f4254966640 1 Processor -- start 2024-09-19T07:54:22.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.726+0000 7f4254966640 1 -- start start 2024-09-19T07:54:22.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.727+0000 7f4254966640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f425006cd20 0x7f42500c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:22.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.727+0000 7f4254966640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4250143a20 0x7f4250158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:22.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.727+0000 7f4254966640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4250161eb0 0x7f425015d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:22.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.727+0000 7f4254966640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4250162830 con 0x7f425006cd20 2024-09-19T07:54:22.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.727+0000 7f4254966640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4250141de0 con 0x7f4250161eb0 2024-09-19T07:54:22.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.727+0000 7f4254966640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4250141600 con 0x7f4250143a20 2024-09-19T07:54:22.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.727+0000 7f424e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f425006cd20 0x7f42500c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:22.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.727+0000 7f424e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f425006cd20 0x7f42500c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48284/0 (socket says 172.21.15.16:48284) 2024-09-19T07:54:22.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.727+0000 7f424e59c640 1 -- 172.21.15.16:0/2076218613 learned_addr learned my addr 172.21.15.16:0/2076218613 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:22.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.727+0000 7f424dd9b640 1 --2- 172.21.15.16:0/2076218613 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4250143a20 0x7f4250158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:22.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.727+0000 7f424ed9d640 1 --2- 172.21.15.16:0/2076218613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4250161eb0 0x7f425015d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:22.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.728+0000 7f424e59c640 1 -- 172.21.15.16:0/2076218613 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4250143a20 msgr2=0x7f4250158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:22.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.728+0000 7f424e59c640 1 --2- 172.21.15.16:0/2076218613 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4250143a20 0x7f4250158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:22.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.728+0000 7f424e59c640 1 -- 172.21.15.16:0/2076218613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4250161eb0 msgr2=0x7f425015d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:22.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.728+0000 7f424e59c640 1 --2- 172.21.15.16:0/2076218613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4250161eb0 0x7f425015d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:22.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.728+0000 7f424e59c640 1 -- 172.21.15.16:0/2076218613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42501594c0 con 0x7f425006cd20 2024-09-19T07:54:22.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.728+0000 7f424e59c640 1 --2- 172.21.15.16:0/2076218613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f425006cd20 0x7f42500c9b70 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f4248035040 tx=0x7f42480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:22.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.728+0000 7f4246ffd640 1 -- 172.21.15.16:0/2076218613 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248041020 con 0x7f425006cd20 2024-09-19T07:54:22.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.728+0000 7f4246ffd640 1 -- 172.21.15.16:0/2076218613 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f424803c670 con 0x7f425006cd20 2024-09-19T07:54:22.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.728+0000 7f4254966640 1 -- 172.21.15.16:0/2076218613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4250159760 con 0x7f425006cd20 2024-09-19T07:54:22.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.728+0000 7f4246ffd640 1 -- 172.21.15.16:0/2076218613 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248037040 con 0x7f425006cd20 2024-09-19T07:54:22.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.728+0000 7f4254966640 1 -- 172.21.15.16:0/2076218613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4250159d20 con 0x7f425006cd20 2024-09-19T07:54:22.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.729+0000 7f4254966640 1 -- 172.21.15.16:0/2076218613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42501622f0 con 0x7f425006cd20 2024-09-19T07:54:22.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.729+0000 7f4246ffd640 1 -- 172.21.15.16:0/2076218613 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4248044020 con 0x7f425006cd20 2024-09-19T07:54:22.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.730+0000 7f4246ffd640 1 --2- 172.21.15.16:0/2076218613 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f423006c2e0 0x7f423006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:22.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.730+0000 7f4246ffd640 1 -- 172.21.15.16:0/2076218613 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 6001+0+0 (secure 0 0 0) 0x7f42480abc30 con 0x7f425006cd20 2024-09-19T07:54:22.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.730+0000 7f424dd9b640 1 --2- 172.21.15.16:0/2076218613 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f423006c2e0 0x7f423006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:22.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.731+0000 7f424dd9b640 1 --2- 172.21.15.16:0/2076218613 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f423006c2e0 0x7f423006e7a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f42500b3c50 tx=0x7f4238008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:22.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.735+0000 7f4246ffd640 1 -- 172.21.15.16:0/2076218613 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f424804b030 con 0x7f425006cd20 2024-09-19T07:54:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:22.947+0000 7f4254966640 1 -- 172.21.15.16:0/2076218613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f42500690c0 con 0x7f425006cd20 2024-09-19T07:54:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.557+0000 7f4246ffd640 1 -- 172.21.15.16:0/2076218613 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v70) ==== 174+0+0 (secure 0 0 0) 0x7f424807a270 con 0x7f425006cd20 2024-09-19T07:54:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 19 and data pool 20 2024-09-19T07:54:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.562+0000 7f4254966640 1 -- 172.21.15.16:0/2076218613 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f423006c2e0 msgr2=0x7f423006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.562+0000 7f4254966640 1 --2- 172.21.15.16:0/2076218613 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f423006c2e0 0x7f423006e7a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f42500b3c50 tx=0x7f4238008040 comp rx=0 tx=0).stop 2024-09-19T07:54:23.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.562+0000 7f4254966640 1 -- 172.21.15.16:0/2076218613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f425006cd20 msgr2=0x7f42500c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:23.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.562+0000 7f4254966640 1 --2- 172.21.15.16:0/2076218613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f425006cd20 0x7f42500c9b70 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f4248035040 tx=0x7f42480046c0 comp rx=0 tx=0).stop 2024-09-19T07:54:23.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.563+0000 7f4254966640 1 -- 172.21.15.16:0/2076218613 shutdown_connections 2024-09-19T07:54:23.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.563+0000 7f4254966640 1 --2- 172.21.15.16:0/2076218613 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f423006c2e0 0x7f423006e7a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:23.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.563+0000 7f4254966640 1 --2- 172.21.15.16:0/2076218613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4250161eb0 0x7f425015d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:23.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.563+0000 7f4254966640 1 --2- 172.21.15.16:0/2076218613 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4250143a20 0x7f4250158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:23.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.563+0000 7f4254966640 1 --2- 172.21.15.16:0/2076218613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f425006cd20 0x7f42500c9b70 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:23.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.563+0000 7f4254966640 1 -- 172.21.15.16:0/2076218613 >> 172.21.15.16:0/2076218613 conn(0x7f42500b3ed0 msgr2=0x7f42500b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:23.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.563+0000 7f4254966640 1 -- 172.21.15.16:0/2076218613 shutdown_connections 2024-09-19T07:54:23.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.566+0000 7f4254966640 1 -- 172.21.15.16:0/2076218613 wait complete. 2024-09-19T07:54:23.598 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:54:23.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.704+0000 7fcf6d331640 1 Processor -- start 2024-09-19T07:54:23.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.705+0000 7fcf6d331640 1 -- start start 2024-09-19T07:54:23.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.705+0000 7fcf6d331640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcf6815e7f0 0x7fcf680cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:23.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.705+0000 7fcf6d331640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf680cc680 0x7fcf680ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:23.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.705+0000 7fcf6d331640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf680b3c10 con 0x7fcf680b03c0 2024-09-19T07:54:23.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.705+0000 7fcf6d331640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fcf680cc1f0 con 0x7fcf680cc680 2024-09-19T07:54:23.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.705+0000 7fcf6d331640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fcf680cb900 con 0x7fcf6815e7f0 2024-09-19T07:54:23.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.705+0000 7fcf677fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf680cc680 0x7fcf680ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:23.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.706+0000 7fcf677fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf680cc680 0x7fcf680ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48294/0 (socket says 172.21.15.16:48294) 2024-09-19T07:54:23.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.706+0000 7fcf677fe640 1 -- 172.21.15.16:0/1265836110 learned_addr learned my addr 172.21.15.16:0/1265836110 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:23.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.706+0000 7fcf66ffd640 1 --2- 172.21.15.16:0/1265836110 >> v2:172.21.15.146:3300/0 conn(0x7fcf6815e7f0 0x7fcf680cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:23.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.706+0000 7fcf677fe640 1 -- 172.21.15.16:0/1265836110 >> v2:172.21.15.146:3300/0 conn(0x7fcf6815e7f0 msgr2=0x7fcf680cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:23.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.706+0000 7fcf677fe640 1 --2- 172.21.15.16:0/1265836110 >> v2:172.21.15.146:3300/0 conn(0x7fcf6815e7f0 0x7fcf680cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:23.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.706+0000 7fcf677fe640 1 -- 172.21.15.16:0/1265836110 >> v1:172.21.15.146:6789/0 conn(0x7fcf680b03c0 legacy=0x7fcf680adff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:23.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.706+0000 7fcf677fe640 1 -- 172.21.15.16:0/1265836110 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6815ebd0 con 0x7fcf680cc680 2024-09-19T07:54:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.707+0000 7fcf677fe640 1 --2- 172.21.15.16:0/1265836110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf680cc680 0x7fcf680ccb10 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fcf680cec70 tx=0x7fcf6002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24addfbd93fe983 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.707+0000 7fcf65ffb640 1 -- 172.21.15.16:0/1265836110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60036070 con 0x7fcf680cc680 2024-09-19T07:54:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.707+0000 7fcf65ffb640 1 -- 172.21.15.16:0/1265836110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf60031600 con 0x7fcf680cc680 2024-09-19T07:54:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.707+0000 7fcf65ffb640 1 -- 172.21.15.16:0/1265836110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf600319d0 con 0x7fcf680cc680 2024-09-19T07:54:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.707+0000 7fcf6d331640 1 -- 172.21.15.16:0/1265836110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf680cc680 msgr2=0x7fcf680ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.707+0000 7fcf6d331640 1 --2- 172.21.15.16:0/1265836110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf680cc680 0x7fcf680ccb10 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fcf680cec70 tx=0x7fcf6002a660 comp rx=0 tx=0).stop 2024-09-19T07:54:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.707+0000 7fcf6d331640 1 -- 172.21.15.16:0/1265836110 shutdown_connections 2024-09-19T07:54:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.707+0000 7fcf6d331640 1 --2- 172.21.15.16:0/1265836110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf680cc680 0x7fcf680ccb10 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.707+0000 7fcf6d331640 1 --2- 172.21.15.16:0/1265836110 >> v2:172.21.15.146:3300/0 conn(0x7fcf6815e7f0 0x7fcf680cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.707+0000 7fcf6d331640 1 -- 172.21.15.16:0/1265836110 >> 172.21.15.16:0/1265836110 conn(0x7fcf680b2d10 msgr2=0x7fcf680b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.707+0000 7fcf6d331640 1 -- 172.21.15.16:0/1265836110 shutdown_connections 2024-09-19T07:54:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.707+0000 7fcf6d331640 1 -- 172.21.15.16:0/1265836110 wait complete. 2024-09-19T07:54:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.709+0000 7fcf6d331640 1 Processor -- start 2024-09-19T07:54:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.709+0000 7fcf6d331640 1 -- start start 2024-09-19T07:54:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.709+0000 7fcf6d331640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf680b03c0 0x7fcf681f5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.709+0000 7fcf6d331640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf680cc680 0x7fcf68156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.709+0000 7fcf6d331640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf6815e7f0 0x7fcf680d1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.709+0000 7fcf6d331640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf680ae700 con 0x7fcf6815e7f0 2024-09-19T07:54:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.709+0000 7fcf6d331640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcf680cbe80 con 0x7fcf680b03c0 2024-09-19T07:54:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.709+0000 7fcf6d331640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcf680b3830 con 0x7fcf680cc680 2024-09-19T07:54:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.710+0000 7fcf677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf6815e7f0 0x7fcf680d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.710+0000 7fcf677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf6815e7f0 0x7fcf680d1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48308/0 (socket says 172.21.15.16:48308) 2024-09-19T07:54:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.710+0000 7fcf677fe640 1 -- 172.21.15.16:0/400729684 learned_addr learned my addr 172.21.15.16:0/400729684 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.710+0000 7fcf66ffd640 1 --2- 172.21.15.16:0/400729684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf680b03c0 0x7fcf681f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.710+0000 7fcf667fc640 1 --2- 172.21.15.16:0/400729684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf680cc680 0x7fcf68156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.710+0000 7fcf677fe640 1 -- 172.21.15.16:0/400729684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf680cc680 msgr2=0x7fcf68156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.710+0000 7fcf677fe640 1 --2- 172.21.15.16:0/400729684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf680cc680 0x7fcf68156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.710+0000 7fcf677fe640 1 -- 172.21.15.16:0/400729684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf680b03c0 msgr2=0x7fcf681f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.710+0000 7fcf677fe640 1 --2- 172.21.15.16:0/400729684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf680b03c0 0x7fcf681f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.710+0000 7fcf677fe640 1 -- 172.21.15.16:0/400729684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf68157b30 con 0x7fcf6815e7f0 2024-09-19T07:54:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.710+0000 7fcf667fc640 1 --2- 172.21.15.16:0/400729684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf680cc680 0x7fcf68156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.710+0000 7fcf677fe640 1 --2- 172.21.15.16:0/400729684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf6815e7f0 0x7fcf680d1180 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009f40 tx=0x7fcf60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.710+0000 7fcf4f7fe640 1 -- 172.21.15.16:0/400729684 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60041020 con 0x7fcf6815e7f0 2024-09-19T07:54:23.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.710+0000 7fcf4f7fe640 1 -- 172.21.15.16:0/400729684 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf600077a0 con 0x7fcf6815e7f0 2024-09-19T07:54:23.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.710+0000 7fcf4f7fe640 1 -- 172.21.15.16:0/400729684 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60036040 con 0x7fcf6815e7f0 2024-09-19T07:54:23.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.710+0000 7fcf6d331640 1 -- 172.21.15.16:0/400729684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf681f8370 con 0x7fcf6815e7f0 2024-09-19T07:54:23.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.710+0000 7fcf6d331640 1 -- 172.21.15.16:0/400729684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf681f8930 con 0x7fcf6815e7f0 2024-09-19T07:54:23.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.711+0000 7fcf6d331640 1 -- 172.21.15.16:0/400729684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf680cd360 con 0x7fcf6815e7f0 2024-09-19T07:54:23.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.712+0000 7fcf4f7fe640 1 -- 172.21.15.16:0/400729684 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcf60044020 con 0x7fcf6815e7f0 2024-09-19T07:54:23.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.712+0000 7fcf4f7fe640 1 --2- 172.21.15.16:0/400729684 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcf5406c2b0 0x7fcf5406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:23.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.712+0000 7fcf4f7fe640 1 -- 172.21.15.16:0/400729684 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6101+0+0 (secure 0 0 0) 0x7fcf600ac230 con 0x7fcf6815e7f0 2024-09-19T07:54:23.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.712+0000 7fcf66ffd640 1 --2- 172.21.15.16:0/400729684 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcf5406c2b0 0x7fcf5406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:23.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.713+0000 7fcf66ffd640 1 --2- 172.21.15.16:0/400729684 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcf5406c2b0 0x7fcf5406e770 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fcf58006fd0 tx=0x7fcf58008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:23.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.717+0000 7fcf4f7fe640 1 -- 172.21.15.16:0/400729684 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fcf6004b030 con 0x7fcf6815e7f0 2024-09-19T07:54:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.894+0000 7fcf6d331640 1 -- 172.21.15.16:0/400729684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf680b2ac0 con 0x7fcf6815e7f0 2024-09-19T07:54:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.895+0000 7fcf4f7fe640 1 -- 172.21.15.16:0/400729684 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+16997 (secure 0 0 0) 0x7fcf6007a770 con 0x7fcf6815e7f0 2024-09-19T07:54:23.895 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:23.896 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":128,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:54:22.949191+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:54:20.571485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-09-19T07:54:21.899096+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:54:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.897+0000 7fcf6d331640 1 -- 172.21.15.16:0/400729684 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcf5406c2b0 msgr2=0x7fcf5406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.897+0000 7fcf6d331640 1 --2- 172.21.15.16:0/400729684 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcf5406c2b0 0x7fcf5406e770 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fcf58006fd0 tx=0x7fcf58008040 comp rx=0 tx=0).stop 2024-09-19T07:54:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.897+0000 7fcf6d331640 1 -- 172.21.15.16:0/400729684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf6815e7f0 msgr2=0x7fcf680d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.897+0000 7fcf6d331640 1 --2- 172.21.15.16:0/400729684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf6815e7f0 0x7fcf680d1180 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009f40 tx=0x7fcf60004a40 comp rx=0 tx=0).stop 2024-09-19T07:54:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.897+0000 7fcf6d331640 1 -- 172.21.15.16:0/400729684 shutdown_connections 2024-09-19T07:54:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.897+0000 7fcf6d331640 1 --2- 172.21.15.16:0/400729684 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcf5406c2b0 0x7fcf5406e770 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.897+0000 7fcf6d331640 1 --2- 172.21.15.16:0/400729684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf6815e7f0 0x7fcf680d1180 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.897+0000 7fcf6d331640 1 --2- 172.21.15.16:0/400729684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf680cc680 0x7fcf68156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.897+0000 7fcf6d331640 1 --2- 172.21.15.16:0/400729684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf680b03c0 0x7fcf681f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.897+0000 7fcf6d331640 1 -- 172.21.15.16:0/400729684 >> 172.21.15.16:0/400729684 conn(0x7fcf680b2d10 msgr2=0x7fcf6806f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.897+0000 7fcf6d331640 1 -- 172.21.15.16:0/400729684 shutdown_connections 2024-09-19T07:54:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:23.897+0000 7fcf6d331640 1 -- 172.21.15.16:0/400729684 wait complete. 2024-09-19T07:54:23.908 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:54:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.010+0000 7f7565354640 1 Processor -- start 2024-09-19T07:54:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.010+0000 7f7565354640 1 -- start start 2024-09-19T07:54:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.011+0000 7f7565354640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f756015e2a0 0x7f756006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.011+0000 7f7565354640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f756015e680 0x7f75600cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.011+0000 7f7565354640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75600d1880 con 0x7f756006d960 2024-09-19T07:54:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.011+0000 7f7565354640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f75600d1b60 con 0x7f756015e680 2024-09-19T07:54:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.011+0000 7f7565354640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f75600cf2d0 con 0x7f756015e2a0 2024-09-19T07:54:24.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.011+0000 7f755f7fe640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f756006d960 0x7f75600d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:45946/0 (socket says 172.21.15.16:45946) 2024-09-19T07:54:24.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.011+0000 7f755f7fe640 1 -- 172.21.15.16:0/2304172004 learned_addr learned my addr 172.21.15.16:0/2304172004 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:24.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.011+0000 7f755effd640 1 --2- 172.21.15.16:0/2304172004 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f756015e2a0 0x7f756006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:24.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.012+0000 7f755e7fc640 1 --2- 172.21.15.16:0/2304172004 >> v2:172.21.15.146:3300/0 conn(0x7f756015e680 0x7f75600cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:24.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.012+0000 7f755effd640 1 -- 172.21.15.16:0/2304172004 >> v2:172.21.15.146:3300/0 conn(0x7f756015e680 msgr2=0x7f75600cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:24.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.012+0000 7f755dffb640 1 -- 172.21.15.16:0/2304172004 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1650774586 0 0) 0x7f75600d1880 con 0x7f756006d960 2024-09-19T07:54:24.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.012+0000 7f755effd640 1 --2- 172.21.15.16:0/2304172004 >> v2:172.21.15.146:3300/0 conn(0x7f756015e680 0x7f75600cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.012+0000 7f755effd640 1 -- 172.21.15.16:0/2304172004 >> v1:172.21.15.146:6789/0 conn(0x7f756006d960 legacy=0x7f75600d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:24.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.012+0000 7f755effd640 1 -- 172.21.15.16:0/2304172004 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75600cf5b0 con 0x7f756015e2a0 2024-09-19T07:54:24.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.012+0000 7f755e7fc640 1 --2- 172.21.15.16:0/2304172004 >> v2:172.21.15.146:3300/0 conn(0x7f756015e680 0x7f75600cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:24.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.013+0000 7f755effd640 1 --2- 172.21.15.16:0/2304172004 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f756015e2a0 0x7f756006d420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7550009d90 tx=0x7f755002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85cc6bfbc85b3dfc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:24.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.013+0000 7f755dffb640 1 -- 172.21.15.16:0/2304172004 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550007740 con 0x7f756015e2a0 2024-09-19T07:54:24.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.013+0000 7f755dffb640 1 -- 172.21.15.16:0/2304172004 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75500078e0 con 0x7f756015e2a0 2024-09-19T07:54:24.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.013+0000 7f755dffb640 1 -- 172.21.15.16:0/2304172004 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550007cb0 con 0x7f756015e2a0 2024-09-19T07:54:24.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.013+0000 7f7565354640 1 -- 172.21.15.16:0/2304172004 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f756015e2a0 msgr2=0x7f756006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:24.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.013+0000 7f7565354640 1 --2- 172.21.15.16:0/2304172004 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f756015e2a0 0x7f756006d420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7550009d90 tx=0x7f755002a660 comp rx=0 tx=0).stop 2024-09-19T07:54:24.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.013+0000 7f7565354640 1 -- 172.21.15.16:0/2304172004 shutdown_connections 2024-09-19T07:54:24.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.013+0000 7f7565354640 1 --2- 172.21.15.16:0/2304172004 >> v2:172.21.15.146:3300/0 conn(0x7f756015e680 0x7f75600cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.013+0000 7f7565354640 1 --2- 172.21.15.16:0/2304172004 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f756015e2a0 0x7f756006d420 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.013+0000 7f7565354640 1 -- 172.21.15.16:0/2304172004 >> 172.21.15.16:0/2304172004 conn(0x7f756009d760 msgr2=0x7f756008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:24.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.013+0000 7f7565354640 1 -- 172.21.15.16:0/2304172004 shutdown_connections 2024-09-19T07:54:24.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.014+0000 7f7565354640 1 -- 172.21.15.16:0/2304172004 wait complete. 2024-09-19T07:54:24.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.015+0000 7f7565354640 1 Processor -- start 2024-09-19T07:54:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.015+0000 7f7565354640 1 -- start start 2024-09-19T07:54:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.015+0000 7f7565354640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f756006d960 0x7f75601e7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f7565354640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f756015e2a0 0x7f75601e95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f7565354640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f756015e680 0x7f75600b8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f7565354640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75600d4c00 con 0x7f756015e680 2024-09-19T07:54:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f7565354640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f75600d4a80 con 0x7f756015e2a0 2024-09-19T07:54:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f7565354640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f75600d0010 con 0x7f756006d960 2024-09-19T07:54:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f755f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f756015e680 0x7f75600b8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f755f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f756015e680 0x7f75600b8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48316/0 (socket says 172.21.15.16:48316) 2024-09-19T07:54:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f755f7fe640 1 -- 172.21.15.16:0/4217625127 learned_addr learned my addr 172.21.15.16:0/4217625127 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f755e7fc640 1 --2- 172.21.15.16:0/4217625127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f756015e2a0 0x7f75601e95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f755effd640 1 --2- 172.21.15.16:0/4217625127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f756006d960 0x7f75601e7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f755f7fe640 1 -- 172.21.15.16:0/4217625127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f756006d960 msgr2=0x7f75601e7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:24.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f755f7fe640 1 --2- 172.21.15.16:0/4217625127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f756006d960 0x7f75601e7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f755f7fe640 1 -- 172.21.15.16:0/4217625127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f756015e2a0 msgr2=0x7f75601e95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:24.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f755f7fe640 1 --2- 172.21.15.16:0/4217625127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f756015e2a0 0x7f75601e95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f755f7fe640 1 -- 172.21.15.16:0/4217625127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75601e8520 con 0x7f756015e680 2024-09-19T07:54:24.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f755e7fc640 1 --2- 172.21.15.16:0/4217625127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f756015e2a0 0x7f75601e95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:24.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f755f7fe640 1 --2- 172.21.15.16:0/4217625127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f756015e680 0x7f75600b8f10 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f755800c480 tx=0x7f755800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f75577fe640 1 -- 172.21.15.16:0/4217625127 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558018070 con 0x7f756015e680 2024-09-19T07:54:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f75577fe640 1 -- 172.21.15.16:0/4217625127 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7558007500 con 0x7f756015e680 2024-09-19T07:54:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f75577fe640 1 -- 172.21.15.16:0/4217625127 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558004a20 con 0x7f756015e680 2024-09-19T07:54:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.016+0000 7f7565354640 1 -- 172.21.15.16:0/4217625127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75601e8800 con 0x7f756015e680 2024-09-19T07:54:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.017+0000 7f7565354640 1 -- 172.21.15.16:0/4217625127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75601e8dc0 con 0x7f756015e680 2024-09-19T07:54:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.017+0000 7f7565354640 1 -- 172.21.15.16:0/4217625127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f756006d420 con 0x7f756015e680 2024-09-19T07:54:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.018+0000 7f75577fe640 1 -- 172.21.15.16:0/4217625127 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f755801b070 con 0x7f756015e680 2024-09-19T07:54:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.018+0000 7f75577fe640 1 --2- 172.21.15.16:0/4217625127 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f754006c2e0 0x7f754006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:24.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.019+0000 7f75577fe640 1 -- 172.21.15.16:0/4217625127 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6101+0+0 (secure 0 0 0) 0x7f755808a860 con 0x7f756015e680 2024-09-19T07:54:24.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.019+0000 7f755effd640 1 --2- 172.21.15.16:0/4217625127 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f754006c2e0 0x7f754006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:24.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.019+0000 7f755effd640 1 --2- 172.21.15.16:0/4217625127 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f754006c2e0 0x7f754006e7a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7550034000 tx=0x7f755002f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:24.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.023+0000 7f75577fe640 1 -- 172.21.15.16:0/4217625127 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7558058ed0 con 0x7f756015e680 2024-09-19T07:54:24.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.199+0000 7f7565354640 1 -- 172.21.15.16:0/4217625127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75600c7c90 con 0x7f756015e680 2024-09-19T07:54:24.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.200+0000 7f75577fe640 1 -- 172.21.15.16:0/4217625127 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+16997 (secure 0 0 0) 0x7f75580587d0 con 0x7f756015e680 2024-09-19T07:54:24.201 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:24.201 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":128,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:54:22.949191+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:54:20.571485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-09-19T07:54:21.899096+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:54:24.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.202+0000 7f7565354640 1 -- 172.21.15.16:0/4217625127 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f754006c2e0 msgr2=0x7f754006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:24.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.202+0000 7f7565354640 1 --2- 172.21.15.16:0/4217625127 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f754006c2e0 0x7f754006e7a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7550034000 tx=0x7f755002f000 comp rx=0 tx=0).stop 2024-09-19T07:54:24.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.202+0000 7f7565354640 1 -- 172.21.15.16:0/4217625127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f756015e680 msgr2=0x7f75600b8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:24.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.202+0000 7f7565354640 1 --2- 172.21.15.16:0/4217625127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f756015e680 0x7f75600b8f10 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f755800c480 tx=0x7f755800c940 comp rx=0 tx=0).stop 2024-09-19T07:54:24.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.202+0000 7f7565354640 1 -- 172.21.15.16:0/4217625127 shutdown_connections 2024-09-19T07:54:24.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.202+0000 7f7565354640 1 --2- 172.21.15.16:0/4217625127 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f754006c2e0 0x7f754006e7a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.202+0000 7f7565354640 1 --2- 172.21.15.16:0/4217625127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f756015e680 0x7f75600b8f10 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.202+0000 7f7565354640 1 --2- 172.21.15.16:0/4217625127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f756015e2a0 0x7f75601e95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.202+0000 7f7565354640 1 --2- 172.21.15.16:0/4217625127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f756006d960 0x7f75601e7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.202+0000 7f7565354640 1 -- 172.21.15.16:0/4217625127 >> 172.21.15.16:0/4217625127 conn(0x7f756009d760 msgr2=0x7f7560160c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:24.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.203+0000 7f7565354640 1 -- 172.21.15.16:0/4217625127 shutdown_connections 2024-09-19T07:54:24.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.203+0000 7f7565354640 1 -- 172.21.15.16:0/4217625127 wait complete. 2024-09-19T07:54:24.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T07:54:24.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.320+0000 7fdd335e8640 1 Processor -- start 2024-09-19T07:54:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.320+0000 7fdd335e8640 1 -- start start 2024-09-19T07:54:24.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.321+0000 7fdd335e8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd2c15e2a0 0x7fdd2c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:24.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.321+0000 7fdd335e8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdd2c15e680 0x7fdd2c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:24.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.321+0000 7fdd335e8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd2c0d1880 con 0x7fdd2c06d960 2024-09-19T07:54:24.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.321+0000 7fdd335e8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fdd2c0d1b60 con 0x7fdd2c15e680 2024-09-19T07:54:24.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.321+0000 7fdd335e8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdd2c0cf2d0 con 0x7fdd2c15e2a0 2024-09-19T07:54:24.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.321+0000 7fdd30b83640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdd2c15e680 0x7fdd2c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.321+0000 7fdd31b85640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdd2c06d960 0x7fdd2c0d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56748/0 (socket says 172.21.15.16:56748) 2024-09-19T07:54:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.321+0000 7fdd31b85640 1 -- 172.21.15.16:0/452141893 learned_addr learned my addr 172.21.15.16:0/452141893 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.322+0000 7fdd31384640 1 --2- 172.21.15.16:0/452141893 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd2c15e2a0 0x7fdd2c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.322+0000 7fdd30b83640 1 -- 172.21.15.16:0/452141893 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd2c15e2a0 msgr2=0x7fdd2c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.322+0000 7fdd30b83640 1 --2- 172.21.15.16:0/452141893 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd2c15e2a0 0x7fdd2c06d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.322+0000 7fdd30b83640 1 -- 172.21.15.16:0/452141893 >> v1:172.21.15.16:6789/0 conn(0x7fdd2c06d960 legacy=0x7fdd2c0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.322+0000 7fdd2bfff640 1 -- 172.21.15.16:0/452141893 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 462149825 0 0) 0x7fdd2c0d1880 con 0x7fdd2c06d960 2024-09-19T07:54:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.322+0000 7fdd30b83640 1 -- 172.21.15.16:0/452141893 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2c0cf5b0 con 0x7fdd2c15e680 2024-09-19T07:54:24.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.322+0000 7fdd30b83640 1 --2- 172.21.15.16:0/452141893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd2c15e680 0x7fdd2c0cfad0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fdd200097a0 tx=0x7fdd2002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7e184f8ce66438 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:24.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.323+0000 7fdd2bfff640 1 -- 172.21.15.16:0/452141893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20037070 con 0x7fdd2c15e680 2024-09-19T07:54:24.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.323+0000 7fdd2bfff640 1 -- 172.21.15.16:0/452141893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd20032440 con 0x7fdd2c15e680 2024-09-19T07:54:24.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.323+0000 7fdd2bfff640 1 -- 172.21.15.16:0/452141893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20032810 con 0x7fdd2c15e680 2024-09-19T07:54:24.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.323+0000 7fdd335e8640 1 -- 172.21.15.16:0/452141893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd2c15e680 msgr2=0x7fdd2c0cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:24.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.323+0000 7fdd335e8640 1 --2- 172.21.15.16:0/452141893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd2c15e680 0x7fdd2c0cfad0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fdd200097a0 tx=0x7fdd2002a180 comp rx=0 tx=0).stop 2024-09-19T07:54:24.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.323+0000 7fdd335e8640 1 -- 172.21.15.16:0/452141893 shutdown_connections 2024-09-19T07:54:24.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.323+0000 7fdd335e8640 1 --2- 172.21.15.16:0/452141893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd2c15e680 0x7fdd2c0cfad0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.323+0000 7fdd335e8640 1 --2- 172.21.15.16:0/452141893 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd2c15e2a0 0x7fdd2c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.323+0000 7fdd335e8640 1 -- 172.21.15.16:0/452141893 >> 172.21.15.16:0/452141893 conn(0x7fdd2c09d760 msgr2=0x7fdd2c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:24.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.323+0000 7fdd335e8640 1 -- 172.21.15.16:0/452141893 shutdown_connections 2024-09-19T07:54:24.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.323+0000 7fdd335e8640 1 -- 172.21.15.16:0/452141893 wait complete. 2024-09-19T07:54:24.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.324+0000 7fdd335e8640 1 Processor -- start 2024-09-19T07:54:24.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.325+0000 7fdd335e8640 1 -- start start 2024-09-19T07:54:24.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.325+0000 7fdd335e8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd2c06d960 0x7fdd2c1f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:24.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.325+0000 7fdd335e8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd2c15e2a0 0x7fdd2c0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.325+0000 7fdd335e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd2c15e680 0x7fdd2c0c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.325+0000 7fdd335e8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c0d4c00 con 0x7fdd2c15e680 2024-09-19T07:54:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.325+0000 7fdd335e8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c0d4a80 con 0x7fdd2c15e2a0 2024-09-19T07:54:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.325+0000 7fdd335e8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdd2c0d0010 con 0x7fdd2c06d960 2024-09-19T07:54:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.325+0000 7fdd31b85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd2c15e680 0x7fdd2c0c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.325+0000 7fdd31b85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd2c15e680 0x7fdd2c0c9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48332/0 (socket says 172.21.15.16:48332) 2024-09-19T07:54:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.325+0000 7fdd31b85640 1 -- 172.21.15.16:0/1703335262 learned_addr learned my addr 172.21.15.16:0/1703335262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.326+0000 7fdd30b83640 1 --2- 172.21.15.16:0/1703335262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd2c15e2a0 0x7fdd2c0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.326+0000 7fdd31384640 1 --2- 172.21.15.16:0/1703335262 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd2c06d960 0x7fdd2c1f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.326+0000 7fdd31b85640 1 -- 172.21.15.16:0/1703335262 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd2c06d960 msgr2=0x7fdd2c1f3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.326+0000 7fdd31b85640 1 --2- 172.21.15.16:0/1703335262 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd2c06d960 0x7fdd2c1f3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.326+0000 7fdd31b85640 1 -- 172.21.15.16:0/1703335262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd2c15e2a0 msgr2=0x7fdd2c0c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.326+0000 7fdd31b85640 1 --2- 172.21.15.16:0/1703335262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd2c15e2a0 0x7fdd2c0c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.326+0000 7fdd31b85640 1 -- 172.21.15.16:0/1703335262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2c1e9ba0 con 0x7fdd2c15e680 2024-09-19T07:54:24.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.326+0000 7fdd31b85640 1 --2- 172.21.15.16:0/1703335262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd2c15e680 0x7fdd2c0c9200 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c00c7d0 tx=0x7fdd1c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:24.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.326+0000 7fdd29ffb640 1 -- 172.21.15.16:0/1703335262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c018070 con 0x7fdd2c15e680 2024-09-19T07:54:24.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.326+0000 7fdd29ffb640 1 -- 172.21.15.16:0/1703335262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd1c00a640 con 0x7fdd2c15e680 2024-09-19T07:54:24.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.326+0000 7fdd335e8640 1 -- 172.21.15.16:0/1703335262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd2c0b7340 con 0x7fdd2c15e680 2024-09-19T07:54:24.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.326+0000 7fdd29ffb640 1 -- 172.21.15.16:0/1703335262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c00aa70 con 0x7fdd2c15e680 2024-09-19T07:54:24.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.326+0000 7fdd335e8640 1 -- 172.21.15.16:0/1703335262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd2c0b78c0 con 0x7fdd2c15e680 2024-09-19T07:54:24.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.327+0000 7fdd335e8640 1 -- 172.21.15.16:0/1703335262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd2c06d420 con 0x7fdd2c15e680 2024-09-19T07:54:24.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.328+0000 7fdd29ffb640 1 -- 172.21.15.16:0/1703335262 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdd1c01b070 con 0x7fdd2c15e680 2024-09-19T07:54:24.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.328+0000 7fdd29ffb640 1 --2- 172.21.15.16:0/1703335262 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdd1806c2b0 0x7fdd1806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:24.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.328+0000 7fdd29ffb640 1 -- 172.21.15.16:0/1703335262 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6101+0+0 (secure 0 0 0) 0x7fdd1c08aa90 con 0x7fdd2c15e680 2024-09-19T07:54:24.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.328+0000 7fdd31384640 1 --2- 172.21.15.16:0/1703335262 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdd1806c2b0 0x7fdd1806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:24.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.329+0000 7fdd31384640 1 --2- 172.21.15.16:0/1703335262 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdd1806c2b0 0x7fdd1806e770 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400aac0 tx=0x7fdd24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:24.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.332+0000 7fdd29ffb640 1 -- 172.21.15.16:0/1703335262 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fdd1c058ab0 con 0x7fdd2c15e680 2024-09-19T07:54:24.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.545+0000 7fdd335e8640 1 -- 172.21.15.16:0/1703335262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdd2c0d1100 con 0x7fdd2c15e680 2024-09-19T07:54:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.568+0000 7fdd29ffb640 1 -- 172.21.15.16:0/1703335262 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v71) ==== 118+0+0 (secure 0 0 0) 0x7fdd1c05cdb0 con 0x7fdd2c15e680 2024-09-19T07:54:24.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.569+0000 7fdd335e8640 1 -- 172.21.15.16:0/1703335262 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdd1806c2b0 msgr2=0x7fdd1806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:24.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.570+0000 7fdd335e8640 1 --2- 172.21.15.16:0/1703335262 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdd1806c2b0 0x7fdd1806e770 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400aac0 tx=0x7fdd24008040 comp rx=0 tx=0).stop 2024-09-19T07:54:24.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.570+0000 7fdd335e8640 1 -- 172.21.15.16:0/1703335262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd2c15e680 msgr2=0x7fdd2c0c9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:24.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.570+0000 7fdd335e8640 1 --2- 172.21.15.16:0/1703335262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd2c15e680 0x7fdd2c0c9200 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c00c7d0 tx=0x7fdd1c00cc90 comp rx=0 tx=0).stop 2024-09-19T07:54:24.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.570+0000 7fdd335e8640 1 -- 172.21.15.16:0/1703335262 shutdown_connections 2024-09-19T07:54:24.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.570+0000 7fdd335e8640 1 --2- 172.21.15.16:0/1703335262 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdd1806c2b0 0x7fdd1806e770 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.570+0000 7fdd335e8640 1 --2- 172.21.15.16:0/1703335262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd2c15e680 0x7fdd2c0c9200 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.570+0000 7fdd335e8640 1 --2- 172.21.15.16:0/1703335262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd2c15e2a0 0x7fdd2c0c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.570+0000 7fdd335e8640 1 --2- 172.21.15.16:0/1703335262 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd2c06d960 0x7fdd2c1f3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.570+0000 7fdd335e8640 1 -- 172.21.15.16:0/1703335262 >> 172.21.15.16:0/1703335262 conn(0x7fdd2c09d760 msgr2=0x7fdd2c075180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:24.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.570+0000 7fdd335e8640 1 -- 172.21.15.16:0/1703335262 shutdown_connections 2024-09-19T07:54:24.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.570+0000 7fdd335e8640 1 -- 172.21.15.16:0/1703335262 wait complete. 2024-09-19T07:54:24.583 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T07:54:24.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T07:54:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.688+0000 7fc9cce27640 1 Processor -- start 2024-09-19T07:54:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.688+0000 7fc9cce27640 1 -- start start 2024-09-19T07:54:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.689+0000 7fc9cce27640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc9c815e680 0x7fc9c80cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.689+0000 7fc9cce27640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc9c80d0010 0x7fc9c80d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.689+0000 7fc9cce27640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9c80d1880 con 0x7fc9c815e2a0 2024-09-19T07:54:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.689+0000 7fc9cce27640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc9c80d1b60 con 0x7fc9c80d0010 2024-09-19T07:54:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.689+0000 7fc9cce27640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc9c80cf2d0 con 0x7fc9c815e680 2024-09-19T07:54:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.689+0000 7fc9c6d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc9c80d0010 0x7fc9c80d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.689+0000 7fc9c6d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc9c80d0010 0x7fc9c80d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48344/0 (socket says 172.21.15.16:48344) 2024-09-19T07:54:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.689+0000 7fc9c6d9d640 1 -- 172.21.15.16:0/1066010273 learned_addr learned my addr 172.21.15.16:0/1066010273 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.689+0000 7fc9c5d9b640 1 --2- 172.21.15.16:0/1066010273 >> v2:172.21.15.146:3300/0 conn(0x7fc9c815e680 0x7fc9c80cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.689+0000 7fc9c6d9d640 1 -- 172.21.15.16:0/1066010273 >> v2:172.21.15.146:3300/0 conn(0x7fc9c815e680 msgr2=0x7fc9c80cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.689+0000 7fc9c6d9d640 1 --2- 172.21.15.16:0/1066010273 >> v2:172.21.15.146:3300/0 conn(0x7fc9c815e680 0x7fc9c80cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.689+0000 7fc9c6d9d640 1 -- 172.21.15.16:0/1066010273 >> v1:172.21.15.146:6789/0 conn(0x7fc9c815e2a0 legacy=0x7fc9c806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.690+0000 7fc9c6d9d640 1 -- 172.21.15.16:0/1066010273 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c80cf5b0 con 0x7fc9c80d0010 2024-09-19T07:54:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.690+0000 7fc9c5d9b640 1 --2- 172.21.15.16:0/1066010273 >> v2:172.21.15.146:3300/0 conn(0x7fc9c815e680 0x7fc9c80cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.690+0000 7fc9c6d9d640 1 --2- 172.21.15.16:0/1066010273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c80d0010 0x7fc9c80d4370 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80a22a0 tx=0x7fc9c002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4fa9d01ceee29e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.690+0000 7fc9c559a640 1 -- 172.21.15.16:0/1066010273 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0036070 con 0x7fc9c80d0010 2024-09-19T07:54:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.690+0000 7fc9c559a640 1 -- 172.21.15.16:0/1066010273 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c0031600 con 0x7fc9c80d0010 2024-09-19T07:54:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.690+0000 7fc9c559a640 1 -- 172.21.15.16:0/1066010273 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c00319d0 con 0x7fc9c80d0010 2024-09-19T07:54:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.691+0000 7fc9cce27640 1 -- 172.21.15.16:0/1066010273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c80d0010 msgr2=0x7fc9c80d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.691+0000 7fc9cce27640 1 --2- 172.21.15.16:0/1066010273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c80d0010 0x7fc9c80d4370 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80a22a0 tx=0x7fc9c002a660 comp rx=0 tx=0).stop 2024-09-19T07:54:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.691+0000 7fc9cce27640 1 -- 172.21.15.16:0/1066010273 shutdown_connections 2024-09-19T07:54:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.691+0000 7fc9cce27640 1 --2- 172.21.15.16:0/1066010273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c80d0010 0x7fc9c80d4370 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.691+0000 7fc9cce27640 1 --2- 172.21.15.16:0/1066010273 >> v2:172.21.15.146:3300/0 conn(0x7fc9c815e680 0x7fc9c80cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.691+0000 7fc9cce27640 1 -- 172.21.15.16:0/1066010273 >> 172.21.15.16:0/1066010273 conn(0x7fc9c809d760 msgr2=0x7fc9c808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.691+0000 7fc9cce27640 1 -- 172.21.15.16:0/1066010273 shutdown_connections 2024-09-19T07:54:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.691+0000 7fc9cce27640 1 -- 172.21.15.16:0/1066010273 wait complete. 2024-09-19T07:54:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.692+0000 7fc9cce27640 1 Processor -- start 2024-09-19T07:54:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.692+0000 7fc9cce27640 1 -- start start 2024-09-19T07:54:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.693+0000 7fc9cce27640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9c80d0010 0x7fc9c81e87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.693+0000 7fc9cce27640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c815e2a0 0x7fc9c80ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.693+0000 7fc9cce27640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9c815e680 0x7fc9c80b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.693+0000 7fc9cce27640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c806dcb0 con 0x7fc9c815e2a0 2024-09-19T07:54:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.693+0000 7fc9cce27640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c806db30 con 0x7fc9c815e680 2024-09-19T07:54:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.693+0000 7fc9cce27640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc9c80d48b0 con 0x7fc9c80d0010 2024-09-19T07:54:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.693+0000 7fc9c5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c815e2a0 0x7fc9c80ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.693+0000 7fc9c5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c815e2a0 0x7fc9c80ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48360/0 (socket says 172.21.15.16:48360) 2024-09-19T07:54:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.693+0000 7fc9c5d9b640 1 -- 172.21.15.16:0/605954794 learned_addr learned my addr 172.21.15.16:0/605954794 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.693+0000 7fc9c659c640 1 --2- 172.21.15.16:0/605954794 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9c80d0010 0x7fc9c81e87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:24.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.693+0000 7fc9c6d9d640 1 --2- 172.21.15.16:0/605954794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9c815e680 0x7fc9c80b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:24.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.694+0000 7fc9c5d9b640 1 -- 172.21.15.16:0/605954794 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9c80d0010 msgr2=0x7fc9c81e87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:24.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.694+0000 7fc9c5d9b640 1 --2- 172.21.15.16:0/605954794 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9c80d0010 0x7fc9c81e87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.694+0000 7fc9c5d9b640 1 -- 172.21.15.16:0/605954794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9c815e680 msgr2=0x7fc9c80b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:24.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.694+0000 7fc9c5d9b640 1 --2- 172.21.15.16:0/605954794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9c815e680 0x7fc9c80b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:24.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.694+0000 7fc9c5d9b640 1 -- 172.21.15.16:0/605954794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c81f7ab0 con 0x7fc9c815e2a0 2024-09-19T07:54:24.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.694+0000 7fc9c5d9b640 1 --2- 172.21.15.16:0/605954794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c815e2a0 0x7fc9c80ca9e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00c7c0 tx=0x7fc9bc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:24.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.694+0000 7fc9b6ffd640 1 -- 172.21.15.16:0/605954794 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc015070 con 0x7fc9c815e2a0 2024-09-19T07:54:24.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.694+0000 7fc9b6ffd640 1 -- 172.21.15.16:0/605954794 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9bc0077c0 con 0x7fc9c815e2a0 2024-09-19T07:54:24.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.694+0000 7fc9cce27640 1 -- 172.21.15.16:0/605954794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c81f7d90 con 0x7fc9c815e2a0 2024-09-19T07:54:24.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.694+0000 7fc9b6ffd640 1 -- 172.21.15.16:0/605954794 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc007b90 con 0x7fc9c815e2a0 2024-09-19T07:54:24.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.694+0000 7fc9cce27640 1 -- 172.21.15.16:0/605954794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c81f8350 con 0x7fc9c815e2a0 2024-09-19T07:54:24.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.695+0000 7fc9cce27640 1 -- 172.21.15.16:0/605954794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9c80d4c70 con 0x7fc9c815e2a0 2024-09-19T07:54:24.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.696+0000 7fc9b6ffd640 1 -- 172.21.15.16:0/605954794 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc9bc018070 con 0x7fc9c815e2a0 2024-09-19T07:54:24.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.696+0000 7fc9b6ffd640 1 --2- 172.21.15.16:0/605954794 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc9a806c2e0 0x7fc9a806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:24.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.696+0000 7fc9b6ffd640 1 -- 172.21.15.16:0/605954794 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7fc9bc08b430 con 0x7fc9c815e2a0 2024-09-19T07:54:24.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.696+0000 7fc9c659c640 1 --2- 172.21.15.16:0/605954794 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc9a806c2e0 0x7fc9a806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:24.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.697+0000 7fc9c659c640 1 --2- 172.21.15.16:0/605954794 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc9a806c2e0 0x7fc9a806e7a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8004f60 tx=0x7fc9b8007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:24.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.700+0000 7fc9b6ffd640 1 -- 172.21.15.16:0/605954794 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc9bc059330 con 0x7fc9c815e2a0 2024-09-19T07:54:24.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:24.910+0000 7fc9cce27640 1 -- 172.21.15.16:0/605954794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc9c815b7d0 con 0x7fc9c815e2a0 2024-09-19T07:54:25.541 INFO:tasks.ceph.mds.d.smithi146.stderr:2024-09-19T07:54:25.541+0000 7f7911833640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T07:54:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.577+0000 7fc9b6ffd640 1 -- 172.21.15.16:0/605954794 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v72) ==== 122+0+0 (secure 0 0 0) 0x7fc9bc05d630 con 0x7fc9c815e2a0 2024-09-19T07:54:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.580+0000 7fc9cce27640 1 -- 172.21.15.16:0/605954794 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc9a806c2e0 msgr2=0x7fc9a806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.580+0000 7fc9cce27640 1 --2- 172.21.15.16:0/605954794 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc9a806c2e0 0x7fc9a806e7a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8004f60 tx=0x7fc9b8007e30 comp rx=0 tx=0).stop 2024-09-19T07:54:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.580+0000 7fc9cce27640 1 -- 172.21.15.16:0/605954794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c815e2a0 msgr2=0x7fc9c80ca9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.580+0000 7fc9cce27640 1 --2- 172.21.15.16:0/605954794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c815e2a0 0x7fc9c80ca9e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00c7c0 tx=0x7fc9bc00cc80 comp rx=0 tx=0).stop 2024-09-19T07:54:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.580+0000 7fc9cce27640 1 -- 172.21.15.16:0/605954794 shutdown_connections 2024-09-19T07:54:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.580+0000 7fc9cce27640 1 --2- 172.21.15.16:0/605954794 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc9a806c2e0 0x7fc9a806e7a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.580+0000 7fc9cce27640 1 --2- 172.21.15.16:0/605954794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9c815e680 0x7fc9c80b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.580+0000 7fc9cce27640 1 --2- 172.21.15.16:0/605954794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c815e2a0 0x7fc9c80ca9e0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.580+0000 7fc9cce27640 1 --2- 172.21.15.16:0/605954794 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9c80d0010 0x7fc9c81e87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.580+0000 7fc9cce27640 1 -- 172.21.15.16:0/605954794 >> 172.21.15.16:0/605954794 conn(0x7fc9c809d760 msgr2=0x7fc9c80d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.580+0000 7fc9cce27640 1 -- 172.21.15.16:0/605954794 shutdown_connections 2024-09-19T07:54:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.580+0000 7fc9cce27640 1 -- 172.21.15.16:0/605954794 wait complete. 2024-09-19T07:54:25.594 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T07:54:25.595 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:54:25.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.702+0000 7fef66c9b640 1 Processor -- start 2024-09-19T07:54:25.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.702+0000 7fef66c9b640 1 -- start start 2024-09-19T07:54:25.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.703+0000 7fef66c9b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef600d0070 0x7fef6015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:25.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.703+0000 7fef66c9b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef600d1b10 con 0x7fef600d4130 2024-09-19T07:54:25.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.703+0000 7fef66c9b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef600cf2b0 con 0x7fef6015f0d0 2024-09-19T07:54:25.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.703+0000 7fef66c9b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fef600cf590 con 0x7fef600d0070 2024-09-19T07:54:25.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.703+0000 7fef65238640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fef600d4130 0x7fef600d45c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56758/0 (socket says 172.21.15.16:56758) 2024-09-19T07:54:25.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.703+0000 7fef65238640 1 -- 172.21.15.16:0/3529925874 learned_addr learned my addr 172.21.15.16:0/3529925874 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:25.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.703+0000 7fef5ffff640 1 --2- 172.21.15.16:0/3529925874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef600d0070 0x7fef6015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:25.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.704+0000 7fef5f7fe640 1 -- 172.21.15.16:0/3529925874 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4121296846 0 0) 0x7fef600d1b10 con 0x7fef600d4130 2024-09-19T07:54:25.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.704+0000 7fef5f7fe640 1 -- 172.21.15.16:0/3529925874 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef48003520 con 0x7fef600d4130 2024-09-19T07:54:25.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.704+0000 7fef5f7fe640 1 -- 172.21.15.16:0/3529925874 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2865087071 0 0) 0x7fef600cf2b0 con 0x7fef6015f0d0 2024-09-19T07:54:25.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.704+0000 7fef5f7fe640 1 -- 172.21.15.16:0/3529925874 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef600d1b10 con 0x7fef6015f0d0 2024-09-19T07:54:25.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.704+0000 7fef5ffff640 1 -- 172.21.15.16:0/3529925874 >> v1:172.21.15.146:6789/0 conn(0x7fef6015f0d0 legacy=0x7fef600d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:25.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.704+0000 7fef5ffff640 1 -- 172.21.15.16:0/3529925874 >> v1:172.21.15.16:6789/0 conn(0x7fef600d4130 legacy=0x7fef600d45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:25.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.704+0000 7fef5ffff640 1 -- 172.21.15.16:0/3529925874 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef600cf870 con 0x7fef600d0070 2024-09-19T07:54:25.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.705+0000 7fef5ffff640 1 --2- 172.21.15.16:0/3529925874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef600d0070 0x7fef6015f4b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fef540097d0 tx=0x7fef5402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=175d23fe6a57fcaf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.705+0000 7fef5f7fe640 1 -- 172.21.15.16:0/3529925874 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef54037070 con 0x7fef600d0070 2024-09-19T07:54:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.705+0000 7fef5f7fe640 1 -- 172.21.15.16:0/3529925874 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef54032440 con 0x7fef600d0070 2024-09-19T07:54:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.705+0000 7fef5f7fe640 1 -- 172.21.15.16:0/3529925874 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef540327f0 con 0x7fef600d0070 2024-09-19T07:54:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.705+0000 7fef66c9b640 1 -- 172.21.15.16:0/3529925874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef600d0070 msgr2=0x7fef6015f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.705+0000 7fef66c9b640 1 --2- 172.21.15.16:0/3529925874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef600d0070 0x7fef6015f4b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fef540097d0 tx=0x7fef5402a180 comp rx=0 tx=0).stop 2024-09-19T07:54:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.705+0000 7fef66c9b640 1 -- 172.21.15.16:0/3529925874 shutdown_connections 2024-09-19T07:54:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.705+0000 7fef66c9b640 1 --2- 172.21.15.16:0/3529925874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef600d0070 0x7fef6015f4b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.705+0000 7fef66c9b640 1 -- 172.21.15.16:0/3529925874 >> 172.21.15.16:0/3529925874 conn(0x7fef6009d760 msgr2=0x7fef6008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.705+0000 7fef66c9b640 1 -- 172.21.15.16:0/3529925874 shutdown_connections 2024-09-19T07:54:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.705+0000 7fef66c9b640 1 -- 172.21.15.16:0/3529925874 wait complete. 2024-09-19T07:54:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.707+0000 7fef66c9b640 1 Processor -- start 2024-09-19T07:54:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.707+0000 7fef66c9b640 1 -- start start 2024-09-19T07:54:25.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.707+0000 7fef66c9b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef600d0070 0x7fef60161680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:25.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef66c9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef600d4130 0x7fef600b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:25.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef66c9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6015f0d0 0x7fef600cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:25.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef66c9b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fef600b2f70 con 0x7fef6015f0d0 2024-09-19T07:54:25.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef66c9b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fef600cef60 con 0x7fef600d4130 2024-09-19T07:54:25.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef66c9b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fef600b30f0 con 0x7fef600d0070 2024-09-19T07:54:25.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef65238640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6015f0d0 0x7fef600cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:25.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef65238640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6015f0d0 0x7fef600cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48370/0 (socket says 172.21.15.16:48370) 2024-09-19T07:54:25.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef65238640 1 -- 172.21.15.16:0/3848937187 learned_addr learned my addr 172.21.15.16:0/3848937187 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:25.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef5ffff640 1 --2- 172.21.15.16:0/3848937187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef600d4130 0x7fef600b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:25.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef64a37640 1 --2- 172.21.15.16:0/3848937187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef600d0070 0x7fef60161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:25.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef65238640 1 -- 172.21.15.16:0/3848937187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef600d0070 msgr2=0x7fef60161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:25.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef65238640 1 --2- 172.21.15.16:0/3848937187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef600d0070 0x7fef60161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef65238640 1 -- 172.21.15.16:0/3848937187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef600d4130 msgr2=0x7fef600b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef65238640 1 --2- 172.21.15.16:0/3848937187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef600d4130 0x7fef600b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef65238640 1 -- 172.21.15.16:0/3848937187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef600b7960 con 0x7fef6015f0d0 2024-09-19T07:54:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef64a37640 1 --2- 172.21.15.16:0/3848937187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef600d0070 0x7fef60161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef65238640 1 --2- 172.21.15.16:0/3848937187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6015f0d0 0x7fef600cbae0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fef5800cbc0 tx=0x7fef5800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef5d7fa640 1 -- 172.21.15.16:0/3848937187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef58017070 con 0x7fef6015f0d0 2024-09-19T07:54:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef5d7fa640 1 -- 172.21.15.16:0/3848937187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef5800ace0 con 0x7fef6015f0d0 2024-09-19T07:54:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef66c9b640 1 -- 172.21.15.16:0/3848937187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef600b7c40 con 0x7fef6015f0d0 2024-09-19T07:54:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef5d7fa640 1 -- 172.21.15.16:0/3848937187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef58012600 con 0x7fef6015f0d0 2024-09-19T07:54:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.708+0000 7fef66c9b640 1 -- 172.21.15.16:0/3848937187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef600b8200 con 0x7fef6015f0d0 2024-09-19T07:54:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.709+0000 7fef66c9b640 1 -- 172.21.15.16:0/3848937187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef6015f4b0 con 0x7fef6015f0d0 2024-09-19T07:54:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.710+0000 7fef5d7fa640 1 -- 172.21.15.16:0/3848937187 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fef5801a070 con 0x7fef6015f0d0 2024-09-19T07:54:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.710+0000 7fef5d7fa640 1 --2- 172.21.15.16:0/3848937187 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fef4c06c2b0 0x7fef4c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.711+0000 7fef5d7fa640 1 -- 172.21.15.16:0/3848937187 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7fef5808b310 con 0x7fef6015f0d0 2024-09-19T07:54:25.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.711+0000 7fef64a37640 1 --2- 172.21.15.16:0/3848937187 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fef4c06c2b0 0x7fef4c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:25.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.711+0000 7fef64a37640 1 --2- 172.21.15.16:0/3848937187 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fef4c06c2b0 0x7fef4c06e770 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fef500011e0 tx=0x7fef50009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:25.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.715+0000 7fef5d7fa640 1 -- 172.21.15.16:0/3848937187 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fef580592a0 con 0x7fef6015f0d0 2024-09-19T07:54:25.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.926+0000 7fef66c9b640 1 -- 172.21.15.16:0/3848937187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef6006cf30 con 0x7fef6015f0d0 2024-09-19T07:54:25.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.926+0000 7fef5d7fa640 1 -- 172.21.15.16:0/3848937187 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5219 (secure 0 0 0) 0x7fef5805d5a0 con 0x7fef6015f0d0 2024-09-19T07:54:25.928 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:25.928 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":72,"btime":"2024-09-19T07:54:25:574262+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/2994096322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2994096322},{"type":"v1","addr":"172.21.15.146:6836","nonce":2994096322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7839,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/393564462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":393564462},{"type":"v1","addr":"172.21.15.16:6833","nonce":393564462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":7842,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/631392582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":631392582},{"type":"v1","addr":"172.21.15.16:6835","nonce":631392582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:54:22.949504+0000","modified":"2024-09-19T07:54:24.912891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/79304323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":79304323},{"type":"v1","addr":"172.21.15.146:6837","nonce":79304323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7854,"qdb_cluster":[7854]},"id":9}]} 2024-09-19T07:54:25.928 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 72 2024-09-19T07:54:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.928+0000 7fef66c9b640 1 -- 172.21.15.16:0/3848937187 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fef4c06c2b0 msgr2=0x7fef4c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.928+0000 7fef66c9b640 1 --2- 172.21.15.16:0/3848937187 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fef4c06c2b0 0x7fef4c06e770 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fef500011e0 tx=0x7fef50009040 comp rx=0 tx=0).stop 2024-09-19T07:54:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.928+0000 7fef66c9b640 1 -- 172.21.15.16:0/3848937187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6015f0d0 msgr2=0x7fef600cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.928+0000 7fef66c9b640 1 --2- 172.21.15.16:0/3848937187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6015f0d0 0x7fef600cbae0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fef5800cbc0 tx=0x7fef5800a430 comp rx=0 tx=0).stop 2024-09-19T07:54:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.928+0000 7fef66c9b640 1 -- 172.21.15.16:0/3848937187 shutdown_connections 2024-09-19T07:54:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.928+0000 7fef66c9b640 1 --2- 172.21.15.16:0/3848937187 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fef4c06c2b0 0x7fef4c06e770 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.928+0000 7fef66c9b640 1 --2- 172.21.15.16:0/3848937187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6015f0d0 0x7fef600cbae0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.928+0000 7fef66c9b640 1 --2- 172.21.15.16:0/3848937187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef600d4130 0x7fef600b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.928+0000 7fef66c9b640 1 --2- 172.21.15.16:0/3848937187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef600d0070 0x7fef60161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.928+0000 7fef66c9b640 1 -- 172.21.15.16:0/3848937187 >> 172.21.15.16:0/3848937187 conn(0x7fef6009d760 msgr2=0x7fef600b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:25.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.928+0000 7fef66c9b640 1 -- 172.21.15.16:0/3848937187 shutdown_connections 2024-09-19T07:54:25.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:25.929+0000 7fef66c9b640 1 -- 172.21.15.16:0/3848937187 wait complete. 2024-09-19T07:54:25.941 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:54:26.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.041+0000 7f15a5e5e640 1 Processor -- start 2024-09-19T07:54:26.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.041+0000 7f15a5e5e640 1 -- start start 2024-09-19T07:54:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.041+0000 7f15a5e5e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f15a015e2a0 0x7f15a006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.041+0000 7f15a5e5e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f15a015e680 0x7f15a00cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.041+0000 7f15a5e5e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15a00d1880 con 0x7f15a006d960 2024-09-19T07:54:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.041+0000 7f15a5e5e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f15a00d1b60 con 0x7f15a015e680 2024-09-19T07:54:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.041+0000 7f15a5e5e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f15a00cf2d0 con 0x7f15a015e2a0 2024-09-19T07:54:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.042+0000 7f159effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f15a015e680 0x7f15a00cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.042+0000 7f159ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f15a006d960 0x7f15a00d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56768/0 (socket says 172.21.15.16:56768) 2024-09-19T07:54:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.042+0000 7f159ffff640 1 -- 172.21.15.16:0/2773009914 learned_addr learned my addr 172.21.15.16:0/2773009914 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.042+0000 7f159f7fe640 1 --2- 172.21.15.16:0/2773009914 >> v2:172.21.15.146:3300/0 conn(0x7f15a015e2a0 0x7f15a006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.042+0000 7f159effd640 1 -- 172.21.15.16:0/2773009914 >> v2:172.21.15.146:3300/0 conn(0x7f15a015e2a0 msgr2=0x7f15a006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.042+0000 7f159effd640 1 --2- 172.21.15.16:0/2773009914 >> v2:172.21.15.146:3300/0 conn(0x7f15a015e2a0 0x7f15a006d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.042+0000 7f159effd640 1 -- 172.21.15.16:0/2773009914 >> v1:172.21.15.16:6789/0 conn(0x7f15a006d960 legacy=0x7f15a00d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.042+0000 7f159e7fc640 1 -- 172.21.15.16:0/2773009914 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 856510736 0 0) 0x7f15a00d1880 con 0x7f15a006d960 2024-09-19T07:54:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.042+0000 7f159effd640 1 -- 172.21.15.16:0/2773009914 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a00cf5b0 con 0x7f15a015e680 2024-09-19T07:54:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.043+0000 7f159effd640 1 --2- 172.21.15.16:0/2773009914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a015e680 0x7f15a00cfad0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f15a00b6fa0 tx=0x7f158802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297d926440362ac3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.043+0000 7f159e7fc640 1 -- 172.21.15.16:0/2773009914 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588037070 con 0x7f15a015e680 2024-09-19T07:54:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.043+0000 7f159e7fc640 1 -- 172.21.15.16:0/2773009914 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1588032440 con 0x7f15a015e680 2024-09-19T07:54:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.043+0000 7f159e7fc640 1 -- 172.21.15.16:0/2773009914 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588032810 con 0x7f15a015e680 2024-09-19T07:54:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.043+0000 7f15a5e5e640 1 -- 172.21.15.16:0/2773009914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a015e680 msgr2=0x7f15a00cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.043+0000 7f15a5e5e640 1 --2- 172.21.15.16:0/2773009914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a015e680 0x7f15a00cfad0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f15a00b6fa0 tx=0x7f158802a1a0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.044+0000 7f15a5e5e640 1 -- 172.21.15.16:0/2773009914 shutdown_connections 2024-09-19T07:54:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.044+0000 7f15a5e5e640 1 --2- 172.21.15.16:0/2773009914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a015e680 0x7f15a00cfad0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.044+0000 7f15a5e5e640 1 --2- 172.21.15.16:0/2773009914 >> v2:172.21.15.146:3300/0 conn(0x7f15a015e2a0 0x7f15a006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.044+0000 7f15a5e5e640 1 -- 172.21.15.16:0/2773009914 >> 172.21.15.16:0/2773009914 conn(0x7f15a009d760 msgr2=0x7f15a008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.044+0000 7f15a5e5e640 1 -- 172.21.15.16:0/2773009914 shutdown_connections 2024-09-19T07:54:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.044+0000 7f15a5e5e640 1 -- 172.21.15.16:0/2773009914 wait complete. 2024-09-19T07:54:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.045+0000 7f15a5e5e640 1 Processor -- start 2024-09-19T07:54:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.045+0000 7f15a5e5e640 1 -- start start 2024-09-19T07:54:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.046+0000 7f15a5e5e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15a006d960 0x7f15a00b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.046+0000 7f15a5e5e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15a015e2a0 0x7f15a014cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.046+0000 7f15a5e5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a015e680 0x7f15a01f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.046+0000 7f15a5e5e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15a00d4c00 con 0x7f15a015e680 2024-09-19T07:54:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.046+0000 7f15a5e5e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f15a00d4a80 con 0x7f15a015e2a0 2024-09-19T07:54:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.046+0000 7f15a5e5e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f15a00d0010 con 0x7f15a006d960 2024-09-19T07:54:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.046+0000 7f159ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a015e680 0x7f15a01f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.046+0000 7f159ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a015e680 0x7f15a01f6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48402/0 (socket says 172.21.15.16:48402) 2024-09-19T07:54:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.046+0000 7f159ffff640 1 -- 172.21.15.16:0/4165080764 learned_addr learned my addr 172.21.15.16:0/4165080764 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.046+0000 7f159effd640 1 --2- 172.21.15.16:0/4165080764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15a015e2a0 0x7f15a014cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.046+0000 7f159f7fe640 1 --2- 172.21.15.16:0/4165080764 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15a006d960 0x7f15a00b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.046+0000 7f159ffff640 1 -- 172.21.15.16:0/4165080764 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15a006d960 msgr2=0x7f15a00b97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.046+0000 7f159ffff640 1 --2- 172.21.15.16:0/4165080764 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15a006d960 0x7f15a00b97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.046+0000 7f159ffff640 1 -- 172.21.15.16:0/4165080764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15a015e2a0 msgr2=0x7f15a014cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.046+0000 7f159ffff640 1 --2- 172.21.15.16:0/4165080764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15a015e2a0 0x7f15a014cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.046+0000 7f159ffff640 1 -- 172.21.15.16:0/4165080764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a00cb970 con 0x7f15a015e680 2024-09-19T07:54:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.046+0000 7f159effd640 1 --2- 172.21.15.16:0/4165080764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15a015e2a0 0x7f15a014cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.046+0000 7f159ffff640 1 --2- 172.21.15.16:0/4165080764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a015e680 0x7f15a01f6680 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f159800c7a0 tx=0x7f159800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.046+0000 7f1597fff640 1 -- 172.21.15.16:0/4165080764 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598018070 con 0x7f15a015e680 2024-09-19T07:54:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.047+0000 7f1597fff640 1 -- 172.21.15.16:0/4165080764 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159800a640 con 0x7f15a015e680 2024-09-19T07:54:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.047+0000 7f15a5e5e640 1 -- 172.21.15.16:0/4165080764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a00b8d10 con 0x7f15a015e680 2024-09-19T07:54:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.047+0000 7f1597fff640 1 -- 172.21.15.16:0/4165080764 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159800aab0 con 0x7f15a015e680 2024-09-19T07:54:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.047+0000 7f15a5e5e640 1 -- 172.21.15.16:0/4165080764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a00cbc50 con 0x7f15a015e680 2024-09-19T07:54:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.048+0000 7f15a5e5e640 1 -- 172.21.15.16:0/4165080764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a006d420 con 0x7f15a015e680 2024-09-19T07:54:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.048+0000 7f1597fff640 1 -- 172.21.15.16:0/4165080764 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f159801b070 con 0x7f15a015e680 2024-09-19T07:54:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.048+0000 7f1597fff640 1 --2- 172.21.15.16:0/4165080764 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f158006c2b0 0x7f158006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.049+0000 7f1597fff640 1 -- 172.21.15.16:0/4165080764 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7f159808ab30 con 0x7f15a015e680 2024-09-19T07:54:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.049+0000 7f159f7fe640 1 --2- 172.21.15.16:0/4165080764 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f158006c2b0 0x7f158006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.049+0000 7f159f7fe640 1 --2- 172.21.15.16:0/4165080764 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f158006c2b0 0x7f158006e770 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1590009ad0 tx=0x7f1590008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:26.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.053+0000 7f1597fff640 1 -- 172.21.15.16:0/4165080764 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f15980591a0 con 0x7f15a015e680 2024-09-19T07:54:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.227+0000 7f15a5e5e640 1 -- 172.21.15.16:0/4165080764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15a006cf90 con 0x7f15a015e680 2024-09-19T07:54:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.229+0000 7f1597fff640 1 -- 172.21.15.16:0/4165080764 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+16984 (secure 0 0 0) 0x7f1598058aa0 con 0x7f15a015e680 2024-09-19T07:54:26.230 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:26.230 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":129,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:54:24.558589+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:54:20.571485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-09-19T07:54:21.899096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:54:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.231+0000 7f15a5e5e640 1 -- 172.21.15.16:0/4165080764 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f158006c2b0 msgr2=0x7f158006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.231+0000 7f15a5e5e640 1 --2- 172.21.15.16:0/4165080764 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f158006c2b0 0x7f158006e770 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1590009ad0 tx=0x7f1590008040 comp rx=0 tx=0).stop 2024-09-19T07:54:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.231+0000 7f15a5e5e640 1 -- 172.21.15.16:0/4165080764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a015e680 msgr2=0x7f15a01f6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.231+0000 7f15a5e5e640 1 --2- 172.21.15.16:0/4165080764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a015e680 0x7f15a01f6680 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f159800c7a0 tx=0x7f159800cc60 comp rx=0 tx=0).stop 2024-09-19T07:54:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.231+0000 7f15a5e5e640 1 -- 172.21.15.16:0/4165080764 shutdown_connections 2024-09-19T07:54:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.231+0000 7f15a5e5e640 1 --2- 172.21.15.16:0/4165080764 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f158006c2b0 0x7f158006e770 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.231+0000 7f15a5e5e640 1 --2- 172.21.15.16:0/4165080764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a015e680 0x7f15a01f6680 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.231+0000 7f15a5e5e640 1 --2- 172.21.15.16:0/4165080764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15a015e2a0 0x7f15a014cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.231+0000 7f15a5e5e640 1 --2- 172.21.15.16:0/4165080764 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15a006d960 0x7f15a00b97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.231+0000 7f15a5e5e640 1 -- 172.21.15.16:0/4165080764 >> 172.21.15.16:0/4165080764 conn(0x7f15a009d760 msgr2=0x7f15a0074ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.231+0000 7f15a5e5e640 1 -- 172.21.15.16:0/4165080764 shutdown_connections 2024-09-19T07:54:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.231+0000 7f15a5e5e640 1 -- 172.21.15.16:0/4165080764 wait complete. 2024-09-19T07:54:26.244 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T07:54:26.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:54:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.345+0000 7f529f0ed640 1 Processor -- start 2024-09-19T07:54:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.345+0000 7f529f0ed640 1 -- start start 2024-09-19T07:54:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.345+0000 7f529f0ed640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f52980d4840 0x7f529815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f529f0ed640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52980d1b10 con 0x7f52980d0070 2024-09-19T07:54:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f529f0ed640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52980cf2b0 con 0x7f529815f0d0 2024-09-19T07:54:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f529f0ed640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f52980cf590 con 0x7f52980d4840 2024-09-19T07:54:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f5297fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f52980d0070 0x7f52980d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56774/0 (socket says 172.21.15.16:56774) 2024-09-19T07:54:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f5297fff640 1 -- 172.21.15.16:0/2161392408 learned_addr learned my addr 172.21.15.16:0/2161392408 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f529d68a640 1 --2- 172.21.15.16:0/2161392408 >> v2:172.21.15.146:3300/0 conn(0x7f52980d4840 0x7f529815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f52977fe640 1 -- 172.21.15.16:0/2161392408 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1873922436 0 0) 0x7f52980d1b10 con 0x7f52980d0070 2024-09-19T07:54:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f52977fe640 1 -- 172.21.15.16:0/2161392408 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5288003520 con 0x7f52980d0070 2024-09-19T07:54:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f52977fe640 1 -- 172.21.15.16:0/2161392408 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2399507126 0 0) 0x7f52980cf2b0 con 0x7f529815f0d0 2024-09-19T07:54:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f52977fe640 1 -- 172.21.15.16:0/2161392408 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52980d1b10 con 0x7f529815f0d0 2024-09-19T07:54:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f52977fe640 1 -- 172.21.15.16:0/2161392408 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 223268153 0 0) 0x7f5288003520 con 0x7f52980d0070 2024-09-19T07:54:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f52977fe640 1 -- 172.21.15.16:0/2161392408 >> v2:172.21.15.146:3300/0 conn(0x7f52980d4840 msgr2=0x7f529815f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f52977fe640 1 --2- 172.21.15.16:0/2161392408 >> v2:172.21.15.146:3300/0 conn(0x7f52980d4840 0x7f529815f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f52977fe640 1 -- 172.21.15.16:0/2161392408 >> v1:172.21.15.146:6789/0 conn(0x7f529815f0d0 legacy=0x7f52980d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f52977fe640 1 -- 172.21.15.16:0/2161392408 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52980cf870 con 0x7f52980d0070 2024-09-19T07:54:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f52977fe640 1 -- 172.21.15.16:0/2161392408 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f5284001670 con 0x7f52980d0070 2024-09-19T07:54:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f52977fe640 1 -- 172.21.15.16:0/2161392408 >> v1:172.21.15.16:6789/0 conn(0x7f52980d0070 legacy=0x7f52980d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f52977fe640 1 --2- 172.21.15.16:0/2161392408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5288003b40 0x7f5288003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f52977fe640 1 --2- 172.21.15.16:0/2161392408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5288004800 0x7f5288004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f52977fe640 1 --2- 172.21.15.16:0/2161392408 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52880051c0 0x7f528800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f52977fe640 1 -- 172.21.15.16:0/2161392408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f52980cf590 con 0x7f5288004800 2024-09-19T07:54:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f52977fe640 1 -- 172.21.15.16:0/2161392408 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f528800c0f0 con 0x7f5288003b40 2024-09-19T07:54:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f52977fe640 1 -- 172.21.15.16:0/2161392408 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f528800c2f0 con 0x7f52880051c0 2024-09-19T07:54:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f529d68a640 1 --2- 172.21.15.16:0/2161392408 >> v2:172.21.15.146:3300/0 conn(0x7f52980d4840 0x7f529815f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:54:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.346+0000 7f5297fff640 1 --2- 172.21.15.16:0/2161392408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5288004800 0x7f5288004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.347+0000 7f529ce89640 1 --2- 172.21.15.16:0/2161392408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5288003b40 0x7f5288003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.347+0000 7f529d68a640 1 --2- 172.21.15.16:0/2161392408 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52880051c0 0x7f528800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.347+0000 7f5297fff640 1 -- 172.21.15.16:0/2161392408 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52880051c0 msgr2=0x7f528800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.347+0000 7f5297fff640 1 --2- 172.21.15.16:0/2161392408 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52880051c0 0x7f528800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.347+0000 7f5297fff640 1 -- 172.21.15.16:0/2161392408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5288003b40 msgr2=0x7f5288003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.347+0000 7f5297fff640 1 --2- 172.21.15.16:0/2161392408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5288003b40 0x7f5288003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.347+0000 7f5297fff640 1 -- 172.21.15.16:0/2161392408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5288003520 con 0x7f5288004800 2024-09-19T07:54:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.347+0000 7f529d68a640 1 -- 172.21.15.16:0/2161392408 reap_dead start 2024-09-19T07:54:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.347+0000 7f5297fff640 1 --2- 172.21.15.16:0/2161392408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5288004800 0x7f5288004c80 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f528400c190 tx=0x7f5288005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a29e2f75f5bbce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.348+0000 7f52977fe640 1 -- 172.21.15.16:0/2161392408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284007ce0 con 0x7f5288004800 2024-09-19T07:54:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.348+0000 7f52977fe640 1 -- 172.21.15.16:0/2161392408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5284005ad0 con 0x7f5288004800 2024-09-19T07:54:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.348+0000 7f52977fe640 1 -- 172.21.15.16:0/2161392408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284007070 con 0x7f5288004800 2024-09-19T07:54:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.348+0000 7f529f0ed640 1 -- 172.21.15.16:0/2161392408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5288004800 msgr2=0x7f5288004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.348+0000 7f529f0ed640 1 --2- 172.21.15.16:0/2161392408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5288004800 0x7f5288004c80 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f528400c190 tx=0x7f5288005830 comp rx=0 tx=0).stop 2024-09-19T07:54:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.348+0000 7f529f0ed640 1 -- 172.21.15.16:0/2161392408 shutdown_connections 2024-09-19T07:54:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.348+0000 7f529f0ed640 1 --2- 172.21.15.16:0/2161392408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5288004800 0x7f5288004c80 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.348+0000 7f529f0ed640 1 -- 172.21.15.16:0/2161392408 >> 172.21.15.16:0/2161392408 conn(0x7f529809d760 msgr2=0x7f529808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.348+0000 7f529f0ed640 1 -- 172.21.15.16:0/2161392408 shutdown_connections 2024-09-19T07:54:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.348+0000 7f529f0ed640 1 -- 172.21.15.16:0/2161392408 wait complete. 2024-09-19T07:54:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.349+0000 7f529f0ed640 1 Processor -- start 2024-09-19T07:54:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.350+0000 7f529f0ed640 1 -- start start 2024-09-19T07:54:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.350+0000 7f529f0ed640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5288004800 0x7f52980cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.350+0000 7f529f0ed640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5298154ba0 0x7f52980cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.350+0000 7f529f0ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529815d3b0 0x7f529815d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.350+0000 7f529f0ed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f529815f4b0 con 0x7f529815d3b0 2024-09-19T07:54:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.350+0000 7f529f0ed640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f52980cef60 con 0x7f5298154ba0 2024-09-19T07:54:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.350+0000 7f529f0ed640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f529815f630 con 0x7f5288004800 2024-09-19T07:54:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.351+0000 7f529d68a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529815d3b0 0x7f529815d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.351+0000 7f529d68a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529815d3b0 0x7f529815d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48414/0 (socket says 172.21.15.16:48414) 2024-09-19T07:54:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.351+0000 7f529d68a640 1 -- 172.21.15.16:0/2538943214 learned_addr learned my addr 172.21.15.16:0/2538943214 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.351+0000 7f5297fff640 1 --2- 172.21.15.16:0/2538943214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5298154ba0 0x7f52980cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.351+0000 7f529ce89640 1 --2- 172.21.15.16:0/2538943214 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5288004800 0x7f52980cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.351+0000 7f529d68a640 1 -- 172.21.15.16:0/2538943214 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5288004800 msgr2=0x7f52980cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.351+0000 7f529d68a640 1 --2- 172.21.15.16:0/2538943214 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5288004800 0x7f52980cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.351+0000 7f529d68a640 1 -- 172.21.15.16:0/2538943214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5298154ba0 msgr2=0x7f52980cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.351+0000 7f529d68a640 1 --2- 172.21.15.16:0/2538943214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5298154ba0 0x7f52980cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.351+0000 7f529d68a640 1 -- 172.21.15.16:0/2538943214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529815dfc0 con 0x7f529815d3b0 2024-09-19T07:54:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.351+0000 7f5297fff640 1 --2- 172.21.15.16:0/2538943214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5298154ba0 0x7f52980cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.351+0000 7f529d68a640 1 --2- 172.21.15.16:0/2538943214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529815d3b0 0x7f529815d830 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f5290009f90 tx=0x7f52900115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.351+0000 7f52957fa640 1 -- 172.21.15.16:0/2538943214 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5290003990 con 0x7f529815d3b0 2024-09-19T07:54:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.351+0000 7f52957fa640 1 -- 172.21.15.16:0/2538943214 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f529001b070 con 0x7f529815d3b0 2024-09-19T07:54:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.351+0000 7f529f0ed640 1 -- 172.21.15.16:0/2538943214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52980b84e0 con 0x7f529815d3b0 2024-09-19T07:54:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.351+0000 7f52957fa640 1 -- 172.21.15.16:0/2538943214 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5290003b50 con 0x7f529815d3b0 2024-09-19T07:54:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.351+0000 7f529f0ed640 1 -- 172.21.15.16:0/2538943214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52980b8aa0 con 0x7f529815d3b0 2024-09-19T07:54:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.352+0000 7f529f0ed640 1 -- 172.21.15.16:0/2538943214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52980d4130 con 0x7f529815d3b0 2024-09-19T07:54:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.353+0000 7f52957fa640 1 -- 172.21.15.16:0/2538943214 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5290005d10 con 0x7f529815d3b0 2024-09-19T07:54:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.353+0000 7f52957fa640 1 --2- 172.21.15.16:0/2538943214 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f527c06c2e0 0x7f527c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.353+0000 7f52957fa640 1 -- 172.21.15.16:0/2538943214 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7f529008ba80 con 0x7f529815d3b0 2024-09-19T07:54:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.353+0000 7f529ce89640 1 --2- 172.21.15.16:0/2538943214 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f527c06c2e0 0x7f527c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.354+0000 7f529ce89640 1 --2- 172.21.15.16:0/2538943214 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f527c06c2e0 0x7f527c06e7a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f528c000c00 tx=0x7f528c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.358+0000 7f52957fa640 1 -- 172.21.15.16:0/2538943214 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f529005a0f0 con 0x7f529815d3b0 2024-09-19T07:54:26.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.530+0000 7f529f0ed640 1 -- 172.21.15.16:0/2538943214 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f529815dd70 con 0x7f527c06c2e0 2024-09-19T07:54:26.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.536+0000 7f52957fa640 1 -- 172.21.15.16:0/2538943214 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248919 (secure 0 0 0) 0x7f529815dd70 con 0x7f527c06c2e0 2024-09-19T07:54:26.541 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:26.543 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:54:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.539+0000 7f529f0ed640 1 -- 172.21.15.16:0/2538943214 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f527c06c2e0 msgr2=0x7f527c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.539+0000 7f529f0ed640 1 --2- 172.21.15.16:0/2538943214 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f527c06c2e0 0x7f527c06e7a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f528c000c00 tx=0x7f528c009040 comp rx=0 tx=0).stop 2024-09-19T07:54:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.539+0000 7f529f0ed640 1 -- 172.21.15.16:0/2538943214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529815d3b0 msgr2=0x7f529815d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.539+0000 7f529f0ed640 1 --2- 172.21.15.16:0/2538943214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529815d3b0 0x7f529815d830 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f5290009f90 tx=0x7f52900115a0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.539+0000 7f529f0ed640 1 -- 172.21.15.16:0/2538943214 shutdown_connections 2024-09-19T07:54:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.539+0000 7f529f0ed640 1 --2- 172.21.15.16:0/2538943214 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f527c06c2e0 0x7f527c06e7a0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.539+0000 7f529f0ed640 1 --2- 172.21.15.16:0/2538943214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529815d3b0 0x7f529815d830 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.539+0000 7f529f0ed640 1 --2- 172.21.15.16:0/2538943214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5298154ba0 0x7f52980cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.539+0000 7f529f0ed640 1 --2- 172.21.15.16:0/2538943214 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5288004800 0x7f52980cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.539+0000 7f529f0ed640 1 -- 172.21.15.16:0/2538943214 >> 172.21.15.16:0/2538943214 conn(0x7f529809d760 msgr2=0x7f529815b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:26.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.539+0000 7f529f0ed640 1 -- 172.21.15.16:0/2538943214 shutdown_connections 2024-09-19T07:54:26.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.539+0000 7f529f0ed640 1 -- 172.21.15.16:0/2538943214 wait complete. 2024-09-19T07:54:26.551 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-09-19T07:54:24.619570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":184,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":168,"acting":168,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":763920,"kb_used_data":13584,"kb_used_omap":12,"kb_used_meta":750131,"kb_avail":754210800,"statfs":{"total":773094113280,"available":772311859200,"internally_reserved":0,"allocated":13910016,"data_stored":7060667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":768134704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000844"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:54:22.540405+0000","last_change":"2024-09-19T07:54:22.540405+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.540405+0000","last_undegraded":"2024-09-19T07:54:22.540405+0000","last_fullsized":"2024-09-19T07:54:22.540405+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579942+0000","last_change":"2024-09-19T07:54:23.576333+0000","last_active":"2024-09-19T07:54:23.579942+0000","last_peered":"2024-09-19T07:54:23.579942+0000","last_clean":"2024-09-19T07:54:23.579942+0000","last_became_active":"2024-09-19T07:54:23.576053+0000","last_became_peered":"2024-09-19T07:54:23.576053+0000","last_unstale":"2024-09-19T07:54:23.579942+0000","last_undegraded":"2024-09-19T07:54:23.579942+0000","last_fullsized":"2024-09-19T07:54:23.579942+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:39.147749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.591937+0000","last_change":"2024-09-19T07:54:23.589816+0000","last_active":"2024-09-19T07:54:23.591937+0000","last_peered":"2024-09-19T07:54:23.591937+0000","last_clean":"2024-09-19T07:54:23.591937+0000","last_became_active":"2024-09-19T07:54:23.589423+0000","last_became_peered":"2024-09-19T07:54:23.589423+0000","last_unstale":"2024-09-19T07:54:23.591937+0000","last_undegraded":"2024-09-19T07:54:23.591937+0000","last_fullsized":"2024-09-19T07:54:23.591937+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:31.699850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.582857+0000","last_change":"2024-09-19T07:54:23.579793+0000","last_active":"2024-09-19T07:54:23.582857+0000","last_peered":"2024-09-19T07:54:23.582857+0000","last_clean":"2024-09-19T07:54:23.582857+0000","last_became_active":"2024-09-19T07:54:23.579502+0000","last_became_peered":"2024-09-19T07:54:23.579502+0000","last_unstale":"2024-09-19T07:54:23.582857+0000","last_undegraded":"2024-09-19T07:54:23.582857+0000","last_fullsized":"2024-09-19T07:54:23.582857+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:32.477007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.551618+0000","last_change":"2024-09-19T07:54:22.551618+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.551618+0000","last_undegraded":"2024-09-19T07:54:22.551618+0000","last_fullsized":"2024-09-19T07:54:22.551618+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.555440+0000","last_change":"2024-09-19T07:54:22.555440+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.555440+0000","last_undegraded":"2024-09-19T07:54:22.555440+0000","last_fullsized":"2024-09-19T07:54:22.555440+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.586469+0000","last_change":"2024-09-19T07:54:23.584389+0000","last_active":"2024-09-19T07:54:23.586469+0000","last_peered":"2024-09-19T07:54:23.586469+0000","last_clean":"2024-09-19T07:54:23.586469+0000","last_became_active":"2024-09-19T07:54:23.584121+0000","last_became_peered":"2024-09-19T07:54:23.584121+0000","last_unstale":"2024-09-19T07:54:23.586469+0000","last_undegraded":"2024-09-19T07:54:23.586469+0000","last_fullsized":"2024-09-19T07:54:23.586469+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:35.419442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585626+0000","last_change":"2024-09-19T07:54:23.584963+0000","last_active":"2024-09-19T07:54:23.585626+0000","last_peered":"2024-09-19T07:54:23.585626+0000","last_clean":"2024-09-19T07:54:23.585626+0000","last_became_active":"2024-09-19T07:54:23.583723+0000","last_became_peered":"2024-09-19T07:54:23.583723+0000","last_unstale":"2024-09-19T07:54:23.585626+0000","last_undegraded":"2024-09-19T07:54:23.585626+0000","last_fullsized":"2024-09-19T07:54:23.585626+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:57.397944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.554930+0000","last_change":"2024-09-19T07:54:22.554930+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.554930+0000","last_undegraded":"2024-09-19T07:54:22.554930+0000","last_fullsized":"2024-09-19T07:54:22.554930+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585515+0000","last_change":"2024-09-19T07:54:23.579169+0000","last_active":"2024-09-19T07:54:23.585515+0000","last_peered":"2024-09-19T07:54:23.585515+0000","last_clean":"2024-09-19T07:54:23.585515+0000","last_became_active":"2024-09-19T07:54:23.578888+0000","last_became_peered":"2024-09-19T07:54:23.578888+0000","last_unstale":"2024-09-19T07:54:23.585515+0000","last_undegraded":"2024-09-19T07:54:23.585515+0000","last_fullsized":"2024-09-19T07:54:23.585515+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:09.226711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585479+0000","last_change":"2024-09-19T07:54:23.584332+0000","last_active":"2024-09-19T07:54:23.585479+0000","last_peered":"2024-09-19T07:54:23.585479+0000","last_clean":"2024-09-19T07:54:23.585479+0000","last_became_active":"2024-09-19T07:54:23.584031+0000","last_became_peered":"2024-09-19T07:54:23.584031+0000","last_unstale":"2024-09-19T07:54:23.585479+0000","last_undegraded":"2024-09-19T07:54:23.585479+0000","last_fullsized":"2024-09-19T07:54:23.585479+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:14.753001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:54:22.540405+0000","last_change":"2024-09-19T07:54:22.540405+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.540405+0000","last_undegraded":"2024-09-19T07:54:22.540405+0000","last_fullsized":"2024-09-19T07:54:22.540405+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.554415+0000","last_change":"2024-09-19T07:54:22.554415+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.554415+0000","last_undegraded":"2024-09-19T07:54:22.554415+0000","last_fullsized":"2024-09-19T07:54:22.554415+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:54:22.540405+0000","last_change":"2024-09-19T07:54:22.540405+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.540405+0000","last_undegraded":"2024-09-19T07:54:22.540405+0000","last_fullsized":"2024-09-19T07:54:22.540405+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:54:22.540405+0000","last_change":"2024-09-19T07:54:22.540405+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.540405+0000","last_undegraded":"2024-09-19T07:54:22.540405+0000","last_fullsized":"2024-09-19T07:54:22.540405+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.557847+0000","last_change":"2024-09-19T07:54:22.557847+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.557847+0000","last_undegraded":"2024-09-19T07:54:22.557847+0000","last_fullsized":"2024-09-19T07:54:22.557847+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.553868+0000","last_change":"2024-09-19T07:54:22.553868+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.553868+0000","last_undegraded":"2024-09-19T07:54:22.553868+0000","last_fullsized":"2024-09-19T07:54:22.553868+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.555444+0000","last_change":"2024-09-19T07:54:22.555444+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.555444+0000","last_undegraded":"2024-09-19T07:54:22.555444+0000","last_fullsized":"2024-09-19T07:54:22.555444+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.581149+0000","last_change":"2024-09-19T07:54:23.580157+0000","last_active":"2024-09-19T07:54:23.581149+0000","last_peered":"2024-09-19T07:54:23.581149+0000","last_clean":"2024-09-19T07:54:23.581149+0000","last_became_active":"2024-09-19T07:54:23.579109+0000","last_became_peered":"2024-09-19T07:54:23.579109+0000","last_unstale":"2024-09-19T07:54:23.581149+0000","last_undegraded":"2024-09-19T07:54:23.581149+0000","last_fullsized":"2024-09-19T07:54:23.581149+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:17.886276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.552956+0000","last_change":"2024-09-19T07:54:22.552956+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.552956+0000","last_undegraded":"2024-09-19T07:54:22.552956+0000","last_fullsized":"2024-09-19T07:54:22.552956+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.568515+0000","last_change":"2024-09-19T07:54:23.567173+0000","last_active":"2024-09-19T07:54:23.568515+0000","last_peered":"2024-09-19T07:54:23.568515+0000","last_clean":"2024-09-19T07:54:23.568515+0000","last_became_active":"2024-09-19T07:54:23.566866+0000","last_became_peered":"2024-09-19T07:54:23.566866+0000","last_unstale":"2024-09-19T07:54:23.568515+0000","last_undegraded":"2024-09-19T07:54:23.568515+0000","last_fullsized":"2024-09-19T07:54:23.568515+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:14.329860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:54:22.540405+0000","last_change":"2024-09-19T07:54:22.540405+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.540405+0000","last_undegraded":"2024-09-19T07:54:22.540405+0000","last_fullsized":"2024-09-19T07:54:22.540405+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579831+0000","last_change":"2024-09-19T07:54:23.575696+0000","last_active":"2024-09-19T07:54:23.579831+0000","last_peered":"2024-09-19T07:54:23.579831+0000","last_clean":"2024-09-19T07:54:23.579831+0000","last_became_active":"2024-09-19T07:54:23.575427+0000","last_became_peered":"2024-09-19T07:54:23.575427+0000","last_unstale":"2024-09-19T07:54:23.579831+0000","last_undegraded":"2024-09-19T07:54:23.579831+0000","last_fullsized":"2024-09-19T07:54:23.579831+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:29.663123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.557404+0000","last_change":"2024-09-19T07:54:22.557404+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.557404+0000","last_undegraded":"2024-09-19T07:54:22.557404+0000","last_fullsized":"2024-09-19T07:54:22.557404+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.589396+0000","last_change":"2024-09-19T07:54:23.584866+0000","last_active":"2024-09-19T07:54:23.589396+0000","last_peered":"2024-09-19T07:54:23.589396+0000","last_clean":"2024-09-19T07:54:23.589396+0000","last_became_active":"2024-09-19T07:54:23.582585+0000","last_became_peered":"2024-09-19T07:54:23.582585+0000","last_unstale":"2024-09-19T07:54:23.589396+0000","last_undegraded":"2024-09-19T07:54:23.589396+0000","last_fullsized":"2024-09-19T07:54:23.589396+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:09.647595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.587463+0000","last_change":"2024-09-19T07:54:23.583732+0000","last_active":"2024-09-19T07:54:23.587463+0000","last_peered":"2024-09-19T07:54:23.587463+0000","last_clean":"2024-09-19T07:54:23.587463+0000","last_became_active":"2024-09-19T07:54:23.583460+0000","last_became_peered":"2024-09-19T07:54:23.583460+0000","last_unstale":"2024-09-19T07:54:23.587463+0000","last_undegraded":"2024-09-19T07:54:23.587463+0000","last_fullsized":"2024-09-19T07:54:23.587463+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:01.094793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.581128+0000","last_change":"2024-09-19T07:54:23.579141+0000","last_active":"2024-09-19T07:54:23.581128+0000","last_peered":"2024-09-19T07:54:23.581128+0000","last_clean":"2024-09-19T07:54:23.581128+0000","last_became_active":"2024-09-19T07:54:23.578856+0000","last_became_peered":"2024-09-19T07:54:23.578856+0000","last_unstale":"2024-09-19T07:54:23.581128+0000","last_undegraded":"2024-09-19T07:54:23.581128+0000","last_fullsized":"2024-09-19T07:54:23.581128+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:21.371456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585833+0000","last_change":"2024-09-19T07:54:23.582767+0000","last_active":"2024-09-19T07:54:23.585833+0000","last_peered":"2024-09-19T07:54:23.585833+0000","last_clean":"2024-09-19T07:54:23.585833+0000","last_became_active":"2024-09-19T07:54:23.582486+0000","last_became_peered":"2024-09-19T07:54:23.582486+0000","last_unstale":"2024-09-19T07:54:23.585833+0000","last_undegraded":"2024-09-19T07:54:23.585833+0000","last_fullsized":"2024-09-19T07:54:23.585833+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:32.949629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.589319+0000","last_change":"2024-09-19T07:54:23.587202+0000","last_active":"2024-09-19T07:54:23.589319+0000","last_peered":"2024-09-19T07:54:23.589319+0000","last_clean":"2024-09-19T07:54:23.589319+0000","last_became_active":"2024-09-19T07:54:23.586940+0000","last_became_peered":"2024-09-19T07:54:23.586940+0000","last_unstale":"2024-09-19T07:54:23.589319+0000","last_undegraded":"2024-09-19T07:54:23.589319+0000","last_fullsized":"2024-09-19T07:54:23.589319+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:41.297635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.583319+0000","last_change":"2024-09-19T07:54:23.581592+0000","last_active":"2024-09-19T07:54:23.583319+0000","last_peered":"2024-09-19T07:54:23.583319+0000","last_clean":"2024-09-19T07:54:23.583319+0000","last_became_active":"2024-09-19T07:54:23.581298+0000","last_became_peered":"2024-09-19T07:54:23.581298+0000","last_unstale":"2024-09-19T07:54:23.583319+0000","last_undegraded":"2024-09-19T07:54:23.583319+0000","last_fullsized":"2024-09-19T07:54:23.583319+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:04.317330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576623+0000","last_change":"2024-09-19T07:54:23.575015+0000","last_active":"2024-09-19T07:54:23.576623+0000","last_peered":"2024-09-19T07:54:23.576623+0000","last_clean":"2024-09-19T07:54:23.576623+0000","last_became_active":"2024-09-19T07:54:23.574695+0000","last_became_peered":"2024-09-19T07:54:23.574695+0000","last_unstale":"2024-09-19T07:54:23.576623+0000","last_undegraded":"2024-09-19T07:54:23.576623+0000","last_fullsized":"2024-09-19T07:54:23.576623+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:07.673263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585765+0000","last_change":"2024-09-19T07:54:23.583307+0000","last_active":"2024-09-19T07:54:23.585765+0000","last_peered":"2024-09-19T07:54:23.585765+0000","last_clean":"2024-09-19T07:54:23.585765+0000","last_became_active":"2024-09-19T07:54:23.583021+0000","last_became_peered":"2024-09-19T07:54:23.583021+0000","last_unstale":"2024-09-19T07:54:23.585765+0000","last_undegraded":"2024-09-19T07:54:23.585765+0000","last_fullsized":"2024-09-19T07:54:23.585765+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:38.791089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579025+0000","last_change":"2024-09-19T07:54:23.576203+0000","last_active":"2024-09-19T07:54:23.579025+0000","last_peered":"2024-09-19T07:54:23.579025+0000","last_clean":"2024-09-19T07:54:23.579025+0000","last_became_active":"2024-09-19T07:54:23.575926+0000","last_became_peered":"2024-09-19T07:54:23.575926+0000","last_unstale":"2024-09-19T07:54:23.579025+0000","last_undegraded":"2024-09-19T07:54:23.579025+0000","last_fullsized":"2024-09-19T07:54:23.579025+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:56.777102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"128'4","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.591733+0000","last_change":"2024-09-19T07:54:22.568159+0000","last_active":"2024-09-19T07:54:23.591733+0000","last_peered":"2024-09-19T07:54:23.591733+0000","last_clean":"2024-09-19T07:54:23.591733+0000","last_became_active":"2024-09-19T07:54:22.567832+0000","last_became_peered":"2024-09-19T07:54:22.567832+0000","last_unstale":"2024-09-19T07:54:23.591733+0000","last_undegraded":"2024-09-19T07:54:23.591733+0000","last_fullsized":"2024-09-19T07:54:23.591733+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:46.972795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:54:22.540405+0000","last_change":"2024-09-19T07:54:22.540405+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.540405+0000","last_undegraded":"2024-09-19T07:54:22.540405+0000","last_fullsized":"2024-09-19T07:54:22.540405+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565571+0000","last_change":"2024-09-19T07:54:22.561503+0000","last_active":"2024-09-19T07:54:22.565571+0000","last_peered":"2024-09-19T07:54:22.565571+0000","last_clean":"2024-09-19T07:54:22.565571+0000","last_became_active":"2024-09-19T07:54:22.561219+0000","last_became_peered":"2024-09-19T07:54:22.561219+0000","last_unstale":"2024-09-19T07:54:22.565571+0000","last_undegraded":"2024-09-19T07:54:22.565571+0000","last_fullsized":"2024-09-19T07:54:22.565571+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:18.897568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579803+0000","last_change":"2024-09-19T07:54:23.574954+0000","last_active":"2024-09-19T07:54:23.579803+0000","last_peered":"2024-09-19T07:54:23.579803+0000","last_clean":"2024-09-19T07:54:23.579803+0000","last_became_active":"2024-09-19T07:54:23.574624+0000","last_became_peered":"2024-09-19T07:54:23.574624+0000","last_unstale":"2024-09-19T07:54:23.579803+0000","last_undegraded":"2024-09-19T07:54:23.579803+0000","last_fullsized":"2024-09-19T07:54:23.579803+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:32.420071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"128'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.592293+0000","last_change":"2024-09-19T07:54:22.566018+0000","last_active":"2024-09-19T07:54:23.592293+0000","last_peered":"2024-09-19T07:54:23.592293+0000","last_clean":"2024-09-19T07:54:23.592293+0000","last_became_active":"2024-09-19T07:54:22.565812+0000","last_became_peered":"2024-09-19T07:54:22.565812+0000","last_unstale":"2024-09-19T07:54:23.592293+0000","last_undegraded":"2024-09-19T07:54:23.592293+0000","last_fullsized":"2024-09-19T07:54:23.592293+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:01.728173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:54:22.540405+0000","last_change":"2024-09-19T07:54:22.540405+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.540405+0000","last_undegraded":"2024-09-19T07:54:22.540405+0000","last_fullsized":"2024-09-19T07:54:22.540405+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1c","version":"128'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.591308+0000","last_change":"2024-09-19T07:54:22.560370+0000","last_active":"2024-09-19T07:54:23.591308+0000","last_peered":"2024-09-19T07:54:23.591308+0000","last_clean":"2024-09-19T07:54:23.591308+0000","last_became_active":"2024-09-19T07:54:22.560083+0000","last_became_peered":"2024-09-19T07:54:22.560083+0000","last_unstale":"2024-09-19T07:54:23.591308+0000","last_undegraded":"2024-09-19T07:54:23.591308+0000","last_fullsized":"2024-09-19T07:54:23.591308+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:51.100115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579407+0000","last_change":"2024-09-19T07:54:23.577913+0000","last_active":"2024-09-19T07:54:23.579407+0000","last_peered":"2024-09-19T07:54:23.579407+0000","last_clean":"2024-09-19T07:54:23.579407+0000","last_became_active":"2024-09-19T07:54:23.577102+0000","last_became_peered":"2024-09-19T07:54:23.577102+0000","last_unstale":"2024-09-19T07:54:23.579407+0000","last_undegraded":"2024-09-19T07:54:23.579407+0000","last_fullsized":"2024-09-19T07:54:23.579407+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:49.188816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"128'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.586957+0000","last_change":"2024-09-19T07:54:22.554611+0000","last_active":"2024-09-19T07:54:23.586957+0000","last_peered":"2024-09-19T07:54:23.586957+0000","last_clean":"2024-09-19T07:54:23.586957+0000","last_became_active":"2024-09-19T07:54:22.554332+0000","last_became_peered":"2024-09-19T07:54:22.554332+0000","last_unstale":"2024-09-19T07:54:23.586957+0000","last_undegraded":"2024-09-19T07:54:23.586957+0000","last_fullsized":"2024-09-19T07:54:23.586957+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:09.508289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576791+0000","last_change":"2024-09-19T07:54:23.574521+0000","last_active":"2024-09-19T07:54:23.576791+0000","last_peered":"2024-09-19T07:54:23.576791+0000","last_clean":"2024-09-19T07:54:23.576791+0000","last_became_active":"2024-09-19T07:54:23.574251+0000","last_became_peered":"2024-09-19T07:54:23.574251+0000","last_unstale":"2024-09-19T07:54:23.576791+0000","last_undegraded":"2024-09-19T07:54:23.576791+0000","last_fullsized":"2024-09-19T07:54:23.576791+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:31.699850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565667+0000","last_change":"2024-09-19T07:54:22.564044+0000","last_active":"2024-09-19T07:54:22.565667+0000","last_peered":"2024-09-19T07:54:22.565667+0000","last_clean":"2024-09-19T07:54:22.565667+0000","last_became_active":"2024-09-19T07:54:22.562851+0000","last_became_peered":"2024-09-19T07:54:22.562851+0000","last_unstale":"2024-09-19T07:54:22.565667+0000","last_undegraded":"2024-09-19T07:54:22.565667+0000","last_fullsized":"2024-09-19T07:54:22.565667+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:34.414794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.555018+0000","last_change":"2024-09-19T07:54:22.555018+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.555018+0000","last_undegraded":"2024-09-19T07:54:22.555018+0000","last_fullsized":"2024-09-19T07:54:22.555018+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.566646+0000","last_change":"2024-09-19T07:54:22.557995+0000","last_active":"2024-09-19T07:54:22.566646+0000","last_peered":"2024-09-19T07:54:22.566646+0000","last_clean":"2024-09-19T07:54:22.566646+0000","last_became_active":"2024-09-19T07:54:22.557692+0000","last_became_peered":"2024-09-19T07:54:22.557692+0000","last_unstale":"2024-09-19T07:54:22.566646+0000","last_undegraded":"2024-09-19T07:54:22.566646+0000","last_fullsized":"2024-09-19T07:54:22.566646+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:01.620005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576699+0000","last_change":"2024-09-19T07:54:23.573943+0000","last_active":"2024-09-19T07:54:23.576699+0000","last_peered":"2024-09-19T07:54:23.576699+0000","last_clean":"2024-09-19T07:54:23.576699+0000","last_became_active":"2024-09-19T07:54:23.573668+0000","last_became_peered":"2024-09-19T07:54:23.573668+0000","last_unstale":"2024-09-19T07:54:23.576699+0000","last_undegraded":"2024-09-19T07:54:23.576699+0000","last_fullsized":"2024-09-19T07:54:23.576699+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:13:01.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.566981+0000","last_change":"2024-09-19T07:54:22.562903+0000","last_active":"2024-09-19T07:54:23.566981+0000","last_peered":"2024-09-19T07:54:23.566981+0000","last_clean":"2024-09-19T07:54:23.566981+0000","last_became_active":"2024-09-19T07:54:22.562599+0000","last_became_peered":"2024-09-19T07:54:22.562599+0000","last_unstale":"2024-09-19T07:54:23.566981+0000","last_undegraded":"2024-09-19T07:54:23.566981+0000","last_fullsized":"2024-09-19T07:54:23.566981+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:44.078560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:54:22.540405+0000","last_change":"2024-09-19T07:54:22.540405+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.540405+0000","last_undegraded":"2024-09-19T07:54:22.540405+0000","last_fullsized":"2024-09-19T07:54:22.540405+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.563273+0000","last_change":"2024-09-19T07:54:22.556612+0000","last_active":"2024-09-19T07:54:23.563273+0000","last_peered":"2024-09-19T07:54:23.563273+0000","last_clean":"2024-09-19T07:54:23.563273+0000","last_became_active":"2024-09-19T07:54:22.556177+0000","last_became_peered":"2024-09-19T07:54:22.556177+0000","last_unstale":"2024-09-19T07:54:23.563273+0000","last_undegraded":"2024-09-19T07:54:23.563273+0000","last_fullsized":"2024-09-19T07:54:23.563273+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:25.294939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.556350+0000","last_change":"2024-09-19T07:54:22.556350+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.556350+0000","last_undegraded":"2024-09-19T07:54:22.556350+0000","last_fullsized":"2024-09-19T07:54:22.556350+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:54:21.535757+0000","last_change":"2024-09-19T07:54:21.535757+0000","last_active":"2024-09-19T07:54:21.535757+0000","last_peered":"2024-09-19T07:54:21.535757+0000","last_clean":"2024-09-19T07:54:21.535757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:21.535757+0000","last_undegraded":"2024-09-19T07:54:21.535757+0000","last_fullsized":"2024-09-19T07:54:21.535757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.584721+0000","last_change":"2024-09-19T07:54:23.583678+0000","last_active":"2024-09-19T07:54:23.584721+0000","last_peered":"2024-09-19T07:54:23.584721+0000","last_clean":"2024-09-19T07:54:23.584721+0000","last_became_active":"2024-09-19T07:54:23.583254+0000","last_became_peered":"2024-09-19T07:54:23.583254+0000","last_unstale":"2024-09-19T07:54:23.584721+0000","last_undegraded":"2024-09-19T07:54:23.584721+0000","last_fullsized":"2024-09-19T07:54:23.584721+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:46.495632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"128'3","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.593450+0000","last_change":"2024-09-19T07:54:22.558733+0000","last_active":"2024-09-19T07:54:23.593450+0000","last_peered":"2024-09-19T07:54:23.593450+0000","last_clean":"2024-09-19T07:54:23.593450+0000","last_became_active":"2024-09-19T07:54:22.558481+0000","last_became_peered":"2024-09-19T07:54:22.558481+0000","last_unstale":"2024-09-19T07:54:23.593450+0000","last_undegraded":"2024-09-19T07:54:23.593450+0000","last_fullsized":"2024-09-19T07:54:23.593450+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:41.801926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.590832+0000","last_change":"2024-09-19T07:54:23.589083+0000","last_active":"2024-09-19T07:54:23.590832+0000","last_peered":"2024-09-19T07:54:23.590832+0000","last_clean":"2024-09-19T07:54:23.590832+0000","last_became_active":"2024-09-19T07:54:23.588728+0000","last_became_peered":"2024-09-19T07:54:23.588728+0000","last_unstale":"2024-09-19T07:54:23.590832+0000","last_undegraded":"2024-09-19T07:54:23.590832+0000","last_fullsized":"2024-09-19T07:54:23.590832+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:44.660536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"128'6","reported_seq":18,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.594644+0000","last_change":"2024-09-19T07:54:22.564291+0000","last_active":"2024-09-19T07:54:23.594644+0000","last_peered":"2024-09-19T07:54:23.594644+0000","last_clean":"2024-09-19T07:54:23.594644+0000","last_became_active":"2024-09-19T07:54:22.564012+0000","last_became_peered":"2024-09-19T07:54:22.564012+0000","last_unstale":"2024-09-19T07:54:23.594644+0000","last_undegraded":"2024-09-19T07:54:23.594644+0000","last_fullsized":"2024-09-19T07:54:23.594644+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:00.025001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.582649+0000","last_change":"2024-09-19T07:54:23.578794+0000","last_active":"2024-09-19T07:54:23.582649+0000","last_peered":"2024-09-19T07:54:23.582649+0000","last_clean":"2024-09-19T07:54:23.582649+0000","last_became_active":"2024-09-19T07:54:23.578465+0000","last_became_peered":"2024-09-19T07:54:23.578465+0000","last_unstale":"2024-09-19T07:54:23.582649+0000","last_undegraded":"2024-09-19T07:54:23.582649+0000","last_fullsized":"2024-09-19T07:54:23.582649+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:57.295016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:54:21.535757+0000","last_change":"2024-09-19T07:54:21.535757+0000","last_active":"2024-09-19T07:54:21.535757+0000","last_peered":"2024-09-19T07:54:21.535757+0000","last_clean":"2024-09-19T07:54:21.535757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:21.535757+0000","last_undegraded":"2024-09-19T07:54:21.535757+0000","last_fullsized":"2024-09-19T07:54:21.535757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576888+0000","last_change":"2024-09-19T07:54:23.574862+0000","last_active":"2024-09-19T07:54:23.576888+0000","last_peered":"2024-09-19T07:54:23.576888+0000","last_clean":"2024-09-19T07:54:23.576888+0000","last_became_active":"2024-09-19T07:54:23.574431+0000","last_became_peered":"2024-09-19T07:54:23.574431+0000","last_unstale":"2024-09-19T07:54:23.576888+0000","last_undegraded":"2024-09-19T07:54:23.576888+0000","last_fullsized":"2024-09-19T07:54:23.576888+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:29.913764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.567482+0000","last_change":"2024-09-19T07:54:22.559040+0000","last_active":"2024-09-19T07:54:23.567482+0000","last_peered":"2024-09-19T07:54:23.567482+0000","last_clean":"2024-09-19T07:54:23.567482+0000","last_became_active":"2024-09-19T07:54:22.558757+0000","last_became_peered":"2024-09-19T07:54:22.558757+0000","last_unstale":"2024-09-19T07:54:23.567482+0000","last_undegraded":"2024-09-19T07:54:23.567482+0000","last_fullsized":"2024-09-19T07:54:23.567482+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:28:24.370408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.551690+0000","last_change":"2024-09-19T07:54:22.551690+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.551690+0000","last_undegraded":"2024-09-19T07:54:22.551690+0000","last_fullsized":"2024-09-19T07:54:22.551690+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565358+0000","last_change":"2024-09-19T07:54:22.559203+0000","last_active":"2024-09-19T07:54:22.565358+0000","last_peered":"2024-09-19T07:54:22.565358+0000","last_clean":"2024-09-19T07:54:22.565358+0000","last_became_active":"2024-09-19T07:54:22.558920+0000","last_became_peered":"2024-09-19T07:54:22.558920+0000","last_unstale":"2024-09-19T07:54:22.565358+0000","last_undegraded":"2024-09-19T07:54:22.565358+0000","last_fullsized":"2024-09-19T07:54:22.565358+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:43.655888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.589459+0000","last_change":"2024-09-19T07:54:23.587980+0000","last_active":"2024-09-19T07:54:23.589459+0000","last_peered":"2024-09-19T07:54:23.589459+0000","last_clean":"2024-09-19T07:54:23.589459+0000","last_became_active":"2024-09-19T07:54:23.587702+0000","last_became_peered":"2024-09-19T07:54:23.587702+0000","last_unstale":"2024-09-19T07:54:23.589459+0000","last_undegraded":"2024-09-19T07:54:23.589459+0000","last_fullsized":"2024-09-19T07:54:23.589459+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:02.624652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"128'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.578744+0000","last_change":"2024-09-19T07:54:22.553965+0000","last_active":"2024-09-19T07:54:23.578744+0000","last_peered":"2024-09-19T07:54:23.578744+0000","last_clean":"2024-09-19T07:54:23.578744+0000","last_became_active":"2024-09-19T07:54:22.553678+0000","last_became_peered":"2024-09-19T07:54:22.553678+0000","last_unstale":"2024-09-19T07:54:23.578744+0000","last_undegraded":"2024-09-19T07:54:23.578744+0000","last_fullsized":"2024-09-19T07:54:23.578744+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:06.364113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576816+0000","last_change":"2024-09-19T07:54:23.574769+0000","last_active":"2024-09-19T07:54:23.576816+0000","last_peered":"2024-09-19T07:54:23.576816+0000","last_clean":"2024-09-19T07:54:23.576816+0000","last_became_active":"2024-09-19T07:54:23.574488+0000","last_became_peered":"2024-09-19T07:54:23.574488+0000","last_unstale":"2024-09-19T07:54:23.576816+0000","last_undegraded":"2024-09-19T07:54:23.576816+0000","last_fullsized":"2024-09-19T07:54:23.576816+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:41.287209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"128'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.591321+0000","last_change":"2024-09-19T07:54:22.567957+0000","last_active":"2024-09-19T07:54:23.591321+0000","last_peered":"2024-09-19T07:54:23.591321+0000","last_clean":"2024-09-19T07:54:23.591321+0000","last_became_active":"2024-09-19T07:54:22.567503+0000","last_became_peered":"2024-09-19T07:54:22.567503+0000","last_unstale":"2024-09-19T07:54:23.591321+0000","last_undegraded":"2024-09-19T07:54:23.591321+0000","last_fullsized":"2024-09-19T07:54:23.591321+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:06.620754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.580616+0000","last_change":"2024-09-19T07:54:23.578211+0000","last_active":"2024-09-19T07:54:23.580616+0000","last_peered":"2024-09-19T07:54:23.580616+0000","last_clean":"2024-09-19T07:54:23.580616+0000","last_became_active":"2024-09-19T07:54:23.577649+0000","last_became_peered":"2024-09-19T07:54:23.577649+0000","last_unstale":"2024-09-19T07:54:23.580616+0000","last_undegraded":"2024-09-19T07:54:23.580616+0000","last_fullsized":"2024-09-19T07:54:23.580616+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:19.902216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565179+0000","last_change":"2024-09-19T07:54:22.559404+0000","last_active":"2024-09-19T07:54:22.565179+0000","last_peered":"2024-09-19T07:54:22.565179+0000","last_clean":"2024-09-19T07:54:22.565179+0000","last_became_active":"2024-09-19T07:54:22.559101+0000","last_became_peered":"2024-09-19T07:54:22.559101+0000","last_unstale":"2024-09-19T07:54:22.565179+0000","last_undegraded":"2024-09-19T07:54:22.565179+0000","last_fullsized":"2024-09-19T07:54:22.565179+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:30.695202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.587444+0000","last_change":"2024-09-19T07:54:23.585316+0000","last_active":"2024-09-19T07:54:23.587444+0000","last_peered":"2024-09-19T07:54:23.587444+0000","last_clean":"2024-09-19T07:54:23.587444+0000","last_became_active":"2024-09-19T07:54:23.585029+0000","last_became_peered":"2024-09-19T07:54:23.585029+0000","last_unstale":"2024-09-19T07:54:23.587444+0000","last_undegraded":"2024-09-19T07:54:23.587444+0000","last_fullsized":"2024-09-19T07:54:23.587444+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:35.538510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565841+0000","last_change":"2024-09-19T07:54:22.563785+0000","last_active":"2024-09-19T07:54:22.565841+0000","last_peered":"2024-09-19T07:54:22.565841+0000","last_clean":"2024-09-19T07:54:22.565841+0000","last_became_active":"2024-09-19T07:54:22.563471+0000","last_became_peered":"2024-09-19T07:54:22.563471+0000","last_unstale":"2024-09-19T07:54:22.565841+0000","last_undegraded":"2024-09-19T07:54:22.565841+0000","last_fullsized":"2024-09-19T07:54:22.565841+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:16.881628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.552495+0000","last_change":"2024-09-19T07:54:22.552495+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.552495+0000","last_undegraded":"2024-09-19T07:54:22.552495+0000","last_fullsized":"2024-09-19T07:54:22.552495+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"128'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.586262+0000","last_change":"2024-09-19T07:54:22.556136+0000","last_active":"2024-09-19T07:54:23.586262+0000","last_peered":"2024-09-19T07:54:23.586262+0000","last_clean":"2024-09-19T07:54:23.586262+0000","last_became_active":"2024-09-19T07:54:22.555829+0000","last_became_peered":"2024-09-19T07:54:22.555829+0000","last_unstale":"2024-09-19T07:54:23.586262+0000","last_undegraded":"2024-09-19T07:54:23.586262+0000","last_fullsized":"2024-09-19T07:54:23.586262+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:37.052983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:54:22.540405+0000","last_change":"2024-09-19T07:54:22.540405+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.540405+0000","last_undegraded":"2024-09-19T07:54:22.540405+0000","last_fullsized":"2024-09-19T07:54:22.540405+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:54:21.535757+0000","last_change":"2024-09-19T07:54:21.535757+0000","last_active":"2024-09-19T07:54:21.535757+0000","last_peered":"2024-09-19T07:54:21.535757+0000","last_clean":"2024-09-19T07:54:21.535757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:21.535757+0000","last_undegraded":"2024-09-19T07:54:21.535757+0000","last_fullsized":"2024-09-19T07:54:21.535757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.554597+0000","last_change":"2024-09-19T07:54:22.554597+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.554597+0000","last_undegraded":"2024-09-19T07:54:22.554597+0000","last_fullsized":"2024-09-19T07:54:22.554597+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.568962+0000","last_change":"2024-09-19T07:54:22.564048+0000","last_active":"2024-09-19T07:54:23.568962+0000","last_peered":"2024-09-19T07:54:23.568962+0000","last_clean":"2024-09-19T07:54:23.568962+0000","last_became_active":"2024-09-19T07:54:22.563775+0000","last_became_peered":"2024-09-19T07:54:22.563775+0000","last_unstale":"2024-09-19T07:54:23.568962+0000","last_undegraded":"2024-09-19T07:54:23.568962+0000","last_fullsized":"2024-09-19T07:54:23.568962+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:57.190350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585516+0000","last_change":"2024-09-19T07:54:23.584566+0000","last_active":"2024-09-19T07:54:23.585516+0000","last_peered":"2024-09-19T07:54:23.585516+0000","last_clean":"2024-09-19T07:54:23.585516+0000","last_became_active":"2024-09-19T07:54:23.584002+0000","last_became_peered":"2024-09-19T07:54:23.584002+0000","last_unstale":"2024-09-19T07:54:23.585516+0000","last_undegraded":"2024-09-19T07:54:23.585516+0000","last_fullsized":"2024-09-19T07:54:23.585516+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:58.194998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565335+0000","last_change":"2024-09-19T07:54:22.558583+0000","last_active":"2024-09-19T07:54:22.565335+0000","last_peered":"2024-09-19T07:54:22.565335+0000","last_clean":"2024-09-19T07:54:22.565335+0000","last_became_active":"2024-09-19T07:54:22.558239+0000","last_became_peered":"2024-09-19T07:54:22.558239+0000","last_unstale":"2024-09-19T07:54:22.565335+0000","last_undegraded":"2024-09-19T07:54:22.565335+0000","last_fullsized":"2024-09-19T07:54:22.565335+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:37.052983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.553358+0000","last_change":"2024-09-19T07:54:22.553358+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.553358+0000","last_undegraded":"2024-09-19T07:54:22.553358+0000","last_fullsized":"2024-09-19T07:54:22.553358+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:54:21.535757+0000","last_change":"2024-09-19T07:54:21.535757+0000","last_active":"2024-09-19T07:54:21.535757+0000","last_peered":"2024-09-19T07:54:21.535757+0000","last_clean":"2024-09-19T07:54:21.535757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:21.535757+0000","last_undegraded":"2024-09-19T07:54:21.535757+0000","last_fullsized":"2024-09-19T07:54:21.535757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.590668+0000","last_change":"2024-09-19T07:54:23.588433+0000","last_active":"2024-09-19T07:54:23.590668+0000","last_peered":"2024-09-19T07:54:23.590668+0000","last_clean":"2024-09-19T07:54:23.590668+0000","last_became_active":"2024-09-19T07:54:23.587803+0000","last_became_peered":"2024-09-19T07:54:23.587803+0000","last_unstale":"2024-09-19T07:54:23.590668+0000","last_undegraded":"2024-09-19T07:54:23.590668+0000","last_fullsized":"2024-09-19T07:54:23.590668+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:10.512937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.562219+0000","last_change":"2024-09-19T07:54:22.555487+0000","last_active":"2024-09-19T07:54:23.562219+0000","last_peered":"2024-09-19T07:54:23.562219+0000","last_clean":"2024-09-19T07:54:23.562219+0000","last_became_active":"2024-09-19T07:54:22.555215+0000","last_became_peered":"2024-09-19T07:54:22.555215+0000","last_unstale":"2024-09-19T07:54:23.562219+0000","last_undegraded":"2024-09-19T07:54:23.562219+0000","last_fullsized":"2024-09-19T07:54:23.562219+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:01.620005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.586495+0000","last_change":"2024-09-19T07:54:23.585093+0000","last_active":"2024-09-19T07:54:23.586495+0000","last_peered":"2024-09-19T07:54:23.586495+0000","last_clean":"2024-09-19T07:54:23.586495+0000","last_became_active":"2024-09-19T07:54:23.584837+0000","last_became_peered":"2024-09-19T07:54:23.584837+0000","last_unstale":"2024-09-19T07:54:23.586495+0000","last_undegraded":"2024-09-19T07:54:23.586495+0000","last_fullsized":"2024-09-19T07:54:23.586495+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:43.491587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.565590+0000","last_change":"2024-09-19T07:54:22.564147+0000","last_active":"2024-09-19T07:54:23.565590+0000","last_peered":"2024-09-19T07:54:23.565590+0000","last_clean":"2024-09-19T07:54:23.565590+0000","last_became_active":"2024-09-19T07:54:22.563844+0000","last_became_peered":"2024-09-19T07:54:22.563844+0000","last_unstale":"2024-09-19T07:54:23.565590+0000","last_undegraded":"2024-09-19T07:54:23.565590+0000","last_fullsized":"2024-09-19T07:54:23.565590+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:43.655888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585486+0000","last_change":"2024-09-19T07:54:23.582012+0000","last_active":"2024-09-19T07:54:23.585486+0000","last_peered":"2024-09-19T07:54:23.585486+0000","last_clean":"2024-09-19T07:54:23.585486+0000","last_became_active":"2024-09-19T07:54:23.581143+0000","last_became_peered":"2024-09-19T07:54:23.581143+0000","last_unstale":"2024-09-19T07:54:23.585486+0000","last_undegraded":"2024-09-19T07:54:23.585486+0000","last_fullsized":"2024-09-19T07:54:23.585486+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:20.783132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565313+0000","last_change":"2024-09-19T07:54:22.558393+0000","last_active":"2024-09-19T07:54:22.565313+0000","last_peered":"2024-09-19T07:54:22.565313+0000","last_clean":"2024-09-19T07:54:22.565313+0000","last_became_active":"2024-09-19T07:54:22.558091+0000","last_became_peered":"2024-09-19T07:54:22.558091+0000","last_unstale":"2024-09-19T07:54:22.565313+0000","last_undegraded":"2024-09-19T07:54:22.565313+0000","last_fullsized":"2024-09-19T07:54:22.565313+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:09.508289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.549695+0000","last_change":"2024-09-19T07:54:22.549695+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.549695+0000","last_undegraded":"2024-09-19T07:54:22.549695+0000","last_fullsized":"2024-09-19T07:54:22.549695+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.562667+0000","last_change":"2024-09-19T07:54:22.567926+0000","last_active":"2024-09-19T07:54:23.562667+0000","last_peered":"2024-09-19T07:54:23.562667+0000","last_clean":"2024-09-19T07:54:23.562667+0000","last_became_active":"2024-09-19T07:54:22.567617+0000","last_became_peered":"2024-09-19T07:54:22.567617+0000","last_unstale":"2024-09-19T07:54:23.562667+0000","last_undegraded":"2024-09-19T07:54:23.562667+0000","last_fullsized":"2024-09-19T07:54:23.562667+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:41:15.024194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.554395+0000","last_change":"2024-09-19T07:54:22.554395+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.554395+0000","last_undegraded":"2024-09-19T07:54:22.554395+0000","last_fullsized":"2024-09-19T07:54:22.554395+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.570070+0000","last_change":"2024-09-19T07:54:22.558104+0000","last_active":"2024-09-19T07:54:23.570070+0000","last_peered":"2024-09-19T07:54:23.570070+0000","last_clean":"2024-09-19T07:54:23.570070+0000","last_became_active":"2024-09-19T07:54:22.557825+0000","last_became_peered":"2024-09-19T07:54:22.557825+0000","last_unstale":"2024-09-19T07:54:23.570070+0000","last_undegraded":"2024-09-19T07:54:23.570070+0000","last_fullsized":"2024-09-19T07:54:23.570070+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:36:36.185111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.553286+0000","last_change":"2024-09-19T07:54:22.553286+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.553286+0000","last_undegraded":"2024-09-19T07:54:22.553286+0000","last_fullsized":"2024-09-19T07:54:22.553286+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.566668+0000","last_change":"2024-09-19T07:54:22.560118+0000","last_active":"2024-09-19T07:54:22.566668+0000","last_peered":"2024-09-19T07:54:22.566668+0000","last_clean":"2024-09-19T07:54:22.566668+0000","last_became_active":"2024-09-19T07:54:22.559838+0000","last_became_peered":"2024-09-19T07:54:22.559838+0000","last_unstale":"2024-09-19T07:54:22.566668+0000","last_undegraded":"2024-09-19T07:54:22.566668+0000","last_fullsized":"2024-09-19T07:54:22.566668+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:06.668616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.587234+0000","last_change":"2024-09-19T07:54:23.586194+0000","last_active":"2024-09-19T07:54:23.587234+0000","last_peered":"2024-09-19T07:54:23.587234+0000","last_clean":"2024-09-19T07:54:23.587234+0000","last_became_active":"2024-09-19T07:54:23.585946+0000","last_became_peered":"2024-09-19T07:54:23.585946+0000","last_unstale":"2024-09-19T07:54:23.587234+0000","last_undegraded":"2024-09-19T07:54:23.587234+0000","last_fullsized":"2024-09-19T07:54:23.587234+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:27.045701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565875+0000","last_change":"2024-09-19T07:54:22.562538+0000","last_active":"2024-09-19T07:54:22.565875+0000","last_peered":"2024-09-19T07:54:22.565875+0000","last_clean":"2024-09-19T07:54:22.565875+0000","last_became_active":"2024-09-19T07:54:22.562257+0000","last_became_peered":"2024-09-19T07:54:22.562257+0000","last_unstale":"2024-09-19T07:54:22.565875+0000","last_undegraded":"2024-09-19T07:54:22.565875+0000","last_fullsized":"2024-09-19T07:54:22.565875+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:20.366808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.590798+0000","last_change":"2024-09-19T07:54:23.588887+0000","last_active":"2024-09-19T07:54:23.590798+0000","last_peered":"2024-09-19T07:54:23.590798+0000","last_clean":"2024-09-19T07:54:23.590798+0000","last_became_active":"2024-09-19T07:54:23.588623+0000","last_became_peered":"2024-09-19T07:54:23.588623+0000","last_unstale":"2024-09-19T07:54:23.590798+0000","last_undegraded":"2024-09-19T07:54:23.590798+0000","last_fullsized":"2024-09-19T07:54:23.590798+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:38.057631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.569582+0000","last_change":"2024-09-19T07:54:22.565171+0000","last_active":"2024-09-19T07:54:23.569582+0000","last_peered":"2024-09-19T07:54:23.569582+0000","last_clean":"2024-09-19T07:54:23.569582+0000","last_became_active":"2024-09-19T07:54:22.564876+0000","last_became_peered":"2024-09-19T07:54:22.564876+0000","last_unstale":"2024-09-19T07:54:23.569582+0000","last_undegraded":"2024-09-19T07:54:23.569582+0000","last_fullsized":"2024-09-19T07:54:23.569582+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:36.291156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":364,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.567647+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:54:23.567647+0000","last_peered":"2024-09-19T07:54:23.567647+0000","last_clean":"2024-09-19T07:54:23.567647+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:54:23.567647+0000","last_undegraded":"2024-09-19T07:54:23.567647+0000","last_fullsized":"2024-09-19T07:54:23.567647+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":110,"acting":110,"num_store_stats":7},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":56,"acting":56,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509581,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83268,"kb_used_data":2152,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288572,"statfs":{"total":96636764160,"available":96551497728,"internally_reserved":0,"allocated":2203648,"data_stored":1331051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509580,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104952,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266888,"statfs":{"total":96636764160,"available":96529293312,"internally_reserved":0,"allocated":1601536,"data_stored":739653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509581,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72944,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298896,"statfs":{"total":96636764160,"available":96562069504,"internally_reserved":0,"allocated":1597440,"data_stored":739003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509581,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86832,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285008,"statfs":{"total":96636764160,"available":96547848192,"internally_reserved":0,"allocated":1593344,"data_stored":738981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509580,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105460,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266380,"statfs":{"total":96636764160,"available":96528773120,"internally_reserved":0,"allocated":2195456,"data_stored":1329405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542284,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118748,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253092,"statfs":{"total":96636764160,"available":96515166208,"internally_reserved":0,"allocated":1581056,"data_stored":731814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542284,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82720,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94289120,"statfs":{"total":96636764160,"available":96552058880,"internally_reserved":0,"allocated":1581056,"data_stored":731814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":108996,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262844,"statfs":{"total":96636764160,"available":96525152256,"internally_reserved":0,"allocated":1556480,"data_stored":718946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:54:26.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:54:26.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.693+0000 7fcd5f5d6640 1 Processor -- start 2024-09-19T07:54:26.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.693+0000 7fcd5f5d6640 1 -- start start 2024-09-19T07:54:26.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.694+0000 7fcd5f5d6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd5806d580 0x7fcd58141b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.694+0000 7fcd5f5d6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd58161f10 con 0x7fcd580d2130 2024-09-19T07:54:26.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.694+0000 7fcd5f5d6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd58141850 con 0x7fcd58141470 2024-09-19T07:54:26.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.694+0000 7fcd5f5d6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcd581621b0 con 0x7fcd5806d580 2024-09-19T07:54:26.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.694+0000 7fcd5db73640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcd580d2130 0x7fcd580d2580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56782/0 (socket says 172.21.15.16:56782) 2024-09-19T07:54:26.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.694+0000 7fcd5db73640 1 -- 172.21.15.16:0/2147097426 learned_addr learned my addr 172.21.15.16:0/2147097426 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:26.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd5cb71640 1 --2- 172.21.15.16:0/2147097426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd5806d580 0x7fcd58141b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd57fff640 1 -- 172.21.15.16:0/2147097426 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3118125632 0 0) 0x7fcd58161f10 con 0x7fcd580d2130 2024-09-19T07:54:26.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd57fff640 1 -- 172.21.15.16:0/2147097426 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd40003520 con 0x7fcd580d2130 2024-09-19T07:54:26.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd57fff640 1 -- 172.21.15.16:0/2147097426 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1786627745 0 0) 0x7fcd58141850 con 0x7fcd58141470 2024-09-19T07:54:26.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd57fff640 1 -- 172.21.15.16:0/2147097426 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd58161f10 con 0x7fcd58141470 2024-09-19T07:54:26.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd57fff640 1 -- 172.21.15.16:0/2147097426 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 704021754 0 0) 0x7fcd40003520 con 0x7fcd580d2130 2024-09-19T07:54:26.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd57fff640 1 -- 172.21.15.16:0/2147097426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd5806d580 msgr2=0x7fcd58141b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:26.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd57fff640 1 --2- 172.21.15.16:0/2147097426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd5806d580 0x7fcd58141b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd57fff640 1 -- 172.21.15.16:0/2147097426 >> v1:172.21.15.146:6789/0 conn(0x7fcd58141470 legacy=0x7fcd5806ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd57fff640 1 -- 172.21.15.16:0/2147097426 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd58162490 con 0x7fcd580d2130 2024-09-19T07:54:26.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd57fff640 1 -- 172.21.15.16:0/2147097426 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fcd480019f0 con 0x7fcd580d2130 2024-09-19T07:54:26.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd57fff640 1 -- 172.21.15.16:0/2147097426 >> v1:172.21.15.16:6789/0 conn(0x7fcd580d2130 legacy=0x7fcd580d2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd57fff640 1 --2- 172.21.15.16:0/2147097426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd40003b70 0x7fcd40003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd57fff640 1 --2- 172.21.15.16:0/2147097426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd40004800 0x7fcd40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd57fff640 1 --2- 172.21.15.16:0/2147097426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd5806d580 0x7fcd400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd57fff640 1 -- 172.21.15.16:0/2147097426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcd581621b0 con 0x7fcd40004800 2024-09-19T07:54:26.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd57fff640 1 -- 172.21.15.16:0/2147097426 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4000ba70 con 0x7fcd40003b70 2024-09-19T07:54:26.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd57fff640 1 -- 172.21.15.16:0/2147097426 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcd4000bc70 con 0x7fcd5806d580 2024-09-19T07:54:26.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd5cb71640 1 --2- 172.21.15.16:0/2147097426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd40004800 0x7fcd40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd5d372640 1 --2- 172.21.15.16:0/2147097426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd40003b70 0x7fcd40003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd5db73640 1 --2- 172.21.15.16:0/2147097426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd5806d580 0x7fcd400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd5cb71640 1 -- 172.21.15.16:0/2147097426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd5806d580 msgr2=0x7fcd400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:26.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd5cb71640 1 --2- 172.21.15.16:0/2147097426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd5806d580 0x7fcd400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd5cb71640 1 -- 172.21.15.16:0/2147097426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd40003b70 msgr2=0x7fcd40003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:26.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.695+0000 7fcd5cb71640 1 --2- 172.21.15.16:0/2147097426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd40003b70 0x7fcd40003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.696+0000 7fcd5cb71640 1 -- 172.21.15.16:0/2147097426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd40003520 con 0x7fcd40004800 2024-09-19T07:54:26.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.696+0000 7fcd5db73640 1 --2- 172.21.15.16:0/2147097426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd5806d580 0x7fcd400051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:26.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.696+0000 7fcd5cb71640 1 --2- 172.21.15.16:0/2147097426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd40004800 0x7fcd40004c80 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009a90 tx=0x7fcd40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938f948c2086f75f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:26.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.696+0000 7fcd57fff640 1 -- 172.21.15.16:0/2147097426 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c036070 con 0x7fcd40004800 2024-09-19T07:54:26.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.696+0000 7fcd57fff640 1 -- 172.21.15.16:0/2147097426 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4c031400 con 0x7fcd40004800 2024-09-19T07:54:26.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.696+0000 7fcd57fff640 1 -- 172.21.15.16:0/2147097426 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c009690 con 0x7fcd40004800 2024-09-19T07:54:26.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.697+0000 7fcd5f5d6640 1 -- 172.21.15.16:0/2147097426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd40004800 msgr2=0x7fcd40004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.697+0000 7fcd5f5d6640 1 --2- 172.21.15.16:0/2147097426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd40004800 0x7fcd40004c80 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009a90 tx=0x7fcd40005830 comp rx=0 tx=0).stop 2024-09-19T07:54:26.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.697+0000 7fcd5db73640 1 -- 172.21.15.16:0/2147097426 reap_dead start 2024-09-19T07:54:26.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.697+0000 7fcd5f5d6640 1 -- 172.21.15.16:0/2147097426 shutdown_connections 2024-09-19T07:54:26.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.697+0000 7fcd5f5d6640 1 -- 172.21.15.16:0/2147097426 >> 172.21.15.16:0/2147097426 conn(0x7fcd580b4040 msgr2=0x7fcd580b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:26.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.697+0000 7fcd5f5d6640 1 -- 172.21.15.16:0/2147097426 shutdown_connections 2024-09-19T07:54:26.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.697+0000 7fcd5f5d6640 1 -- 172.21.15.16:0/2147097426 wait complete. 2024-09-19T07:54:26.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.698+0000 7fcd5f5d6640 1 Processor -- start 2024-09-19T07:54:26.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.698+0000 7fcd5f5d6640 1 -- start start 2024-09-19T07:54:26.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.699+0000 7fcd5f5d6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd40004800 0x7fcd581f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.699+0000 7fcd5f5d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd5815e3a0 0x7fcd580d1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.699+0000 7fcd5f5d6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd580b5b70 0x7fcd580b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.699+0000 7fcd5f5d6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcd58162d30 con 0x7fcd5815e3a0 2024-09-19T07:54:26.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.699+0000 7fcd5f5d6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcd58162a50 con 0x7fcd580b5b70 2024-09-19T07:54:26.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.699+0000 7fcd5f5d6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcd58141b70 con 0x7fcd40004800 2024-09-19T07:54:26.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.699+0000 7fcd5cb71640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd5815e3a0 0x7fcd580d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.699+0000 7fcd5cb71640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd5815e3a0 0x7fcd580d1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48434/0 (socket says 172.21.15.16:48434) 2024-09-19T07:54:26.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.699+0000 7fcd5cb71640 1 -- 172.21.15.16:0/48912547 learned_addr learned my addr 172.21.15.16:0/48912547 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:26.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.699+0000 7fcd5d372640 1 --2- 172.21.15.16:0/48912547 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd40004800 0x7fcd581f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.699+0000 7fcd5db73640 1 --2- 172.21.15.16:0/48912547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd580b5b70 0x7fcd580b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.699+0000 7fcd5cb71640 1 -- 172.21.15.16:0/48912547 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd40004800 msgr2=0x7fcd581f4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.699+0000 7fcd5cb71640 1 --2- 172.21.15.16:0/48912547 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd40004800 0x7fcd581f4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.699+0000 7fcd5cb71640 1 -- 172.21.15.16:0/48912547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd580b5b70 msgr2=0x7fcd580b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.699+0000 7fcd5cb71640 1 --2- 172.21.15.16:0/48912547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd580b5b70 0x7fcd580b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.699+0000 7fcd5cb71640 1 -- 172.21.15.16:0/48912547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd581f51d0 con 0x7fcd5815e3a0 2024-09-19T07:54:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.699+0000 7fcd5d372640 1 --2- 172.21.15.16:0/48912547 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd40004800 0x7fcd581f4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.700+0000 7fcd5cb71640 1 --2- 172.21.15.16:0/48912547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd5815e3a0 0x7fcd580d1540 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c02ab50 tx=0x7fcd4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.700+0000 7fcd55ffb640 1 -- 172.21.15.16:0/48912547 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c041020 con 0x7fcd5815e3a0 2024-09-19T07:54:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.700+0000 7fcd55ffb640 1 -- 172.21.15.16:0/48912547 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4c03c470 con 0x7fcd5815e3a0 2024-09-19T07:54:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.700+0000 7fcd5f5d6640 1 -- 172.21.15.16:0/48912547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd580d0200 con 0x7fcd5815e3a0 2024-09-19T07:54:26.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.700+0000 7fcd55ffb640 1 -- 172.21.15.16:0/48912547 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c036040 con 0x7fcd5815e3a0 2024-09-19T07:54:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.700+0000 7fcd5f5d6640 1 -- 172.21.15.16:0/48912547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd580d0780 con 0x7fcd5815e3a0 2024-09-19T07:54:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.701+0000 7fcd5f5d6640 1 -- 172.21.15.16:0/48912547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd580d2580 con 0x7fcd5815e3a0 2024-09-19T07:54:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.702+0000 7fcd55ffb640 1 -- 172.21.15.16:0/48912547 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcd4c044020 con 0x7fcd5815e3a0 2024-09-19T07:54:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.702+0000 7fcd55ffb640 1 --2- 172.21.15.16:0/48912547 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcd4406c2b0 0x7fcd4406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.702+0000 7fcd55ffb640 1 -- 172.21.15.16:0/48912547 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7fcd4c0abc80 con 0x7fcd5815e3a0 2024-09-19T07:54:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.702+0000 7fcd5d372640 1 --2- 172.21.15.16:0/48912547 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcd4406c2b0 0x7fcd4406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.703+0000 7fcd5d372640 1 --2- 172.21.15.16:0/48912547 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcd4406c2b0 0x7fcd4406e770 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcd500055e0 tx=0x7fcd50009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:26.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.706+0000 7fcd55ffb640 1 -- 172.21.15.16:0/48912547 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fcd4c04b030 con 0x7fcd5815e3a0 2024-09-19T07:54:26.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.876+0000 7fcd5f5d6640 1 -- 172.21.15.16:0/48912547 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcd5813d6a0 con 0x7fcd4406c2b0 2024-09-19T07:54:26.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.884+0000 7fcd55ffb640 1 -- 172.21.15.16:0/48912547 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250945 (secure 0 0 0) 0x7fcd5813d6a0 con 0x7fcd4406c2b0 2024-09-19T07:54:26.888 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:26.889 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:54:26.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.887+0000 7fcd5f5d6640 1 -- 172.21.15.16:0/48912547 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcd4406c2b0 msgr2=0x7fcd4406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.887+0000 7fcd5f5d6640 1 --2- 172.21.15.16:0/48912547 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcd4406c2b0 0x7fcd4406e770 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcd500055e0 tx=0x7fcd50009040 comp rx=0 tx=0).stop 2024-09-19T07:54:26.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.887+0000 7fcd5f5d6640 1 -- 172.21.15.16:0/48912547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd5815e3a0 msgr2=0x7fcd580d1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:26.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.887+0000 7fcd5f5d6640 1 --2- 172.21.15.16:0/48912547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd5815e3a0 0x7fcd580d1540 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c02ab50 tx=0x7fcd4c004730 comp rx=0 tx=0).stop 2024-09-19T07:54:26.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.887+0000 7fcd5f5d6640 1 -- 172.21.15.16:0/48912547 shutdown_connections 2024-09-19T07:54:26.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.887+0000 7fcd5f5d6640 1 --2- 172.21.15.16:0/48912547 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcd4406c2b0 0x7fcd4406e770 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.887+0000 7fcd5f5d6640 1 --2- 172.21.15.16:0/48912547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd580b5b70 0x7fcd580b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.887+0000 7fcd5f5d6640 1 --2- 172.21.15.16:0/48912547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd5815e3a0 0x7fcd580d1540 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.887+0000 7fcd5f5d6640 1 --2- 172.21.15.16:0/48912547 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd40004800 0x7fcd581f4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:26.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.887+0000 7fcd5f5d6640 1 -- 172.21.15.16:0/48912547 >> 172.21.15.16:0/48912547 conn(0x7fcd580b4040 msgr2=0x7fcd581427f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:26.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.887+0000 7fcd5f5d6640 1 -- 172.21.15.16:0/48912547 shutdown_connections 2024-09-19T07:54:26.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:26.887+0000 7fcd5f5d6640 1 -- 172.21.15.16:0/48912547 wait complete. 2024-09-19T07:54:26.899 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2024-09-19T07:54:26.619972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":190,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":763976,"kb_used_data":13640,"kb_used_omap":12,"kb_used_meta":750131,"kb_avail":754210744,"statfs":{"total":773094113280,"available":772311801856,"internally_reserved":0,"allocated":13967360,"data_stored":7088010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":768134704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.078490"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572572+0000","last_change":"2024-09-19T07:54:23.583787+0000","last_active":"2024-09-19T07:54:24.572572+0000","last_peered":"2024-09-19T07:54:24.572572+0000","last_clean":"2024-09-19T07:54:24.572572+0000","last_became_active":"2024-09-19T07:54:23.583453+0000","last_became_peered":"2024-09-19T07:54:23.583453+0000","last_unstale":"2024-09-19T07:54:24.572572+0000","last_undegraded":"2024-09-19T07:54:24.572572+0000","last_fullsized":"2024-09-19T07:54:24.572572+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:23.821546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579942+0000","last_change":"2024-09-19T07:54:23.576333+0000","last_active":"2024-09-19T07:54:23.579942+0000","last_peered":"2024-09-19T07:54:23.579942+0000","last_clean":"2024-09-19T07:54:23.579942+0000","last_became_active":"2024-09-19T07:54:23.576053+0000","last_became_peered":"2024-09-19T07:54:23.576053+0000","last_unstale":"2024-09-19T07:54:23.579942+0000","last_undegraded":"2024-09-19T07:54:23.579942+0000","last_fullsized":"2024-09-19T07:54:23.579942+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:39.147749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.591937+0000","last_change":"2024-09-19T07:54:23.589816+0000","last_active":"2024-09-19T07:54:23.591937+0000","last_peered":"2024-09-19T07:54:23.591937+0000","last_clean":"2024-09-19T07:54:23.591937+0000","last_became_active":"2024-09-19T07:54:23.589423+0000","last_became_peered":"2024-09-19T07:54:23.589423+0000","last_unstale":"2024-09-19T07:54:23.591937+0000","last_undegraded":"2024-09-19T07:54:23.591937+0000","last_fullsized":"2024-09-19T07:54:23.591937+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:31.699850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.582857+0000","last_change":"2024-09-19T07:54:23.579793+0000","last_active":"2024-09-19T07:54:23.582857+0000","last_peered":"2024-09-19T07:54:23.582857+0000","last_clean":"2024-09-19T07:54:23.582857+0000","last_became_active":"2024-09-19T07:54:23.579502+0000","last_became_peered":"2024-09-19T07:54:23.579502+0000","last_unstale":"2024-09-19T07:54:23.582857+0000","last_undegraded":"2024-09-19T07:54:23.582857+0000","last_fullsized":"2024-09-19T07:54:23.582857+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:32.477007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.551618+0000","last_change":"2024-09-19T07:54:22.551618+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.551618+0000","last_undegraded":"2024-09-19T07:54:22.551618+0000","last_fullsized":"2024-09-19T07:54:22.551618+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.555440+0000","last_change":"2024-09-19T07:54:22.555440+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.555440+0000","last_undegraded":"2024-09-19T07:54:22.555440+0000","last_fullsized":"2024-09-19T07:54:22.555440+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.586469+0000","last_change":"2024-09-19T07:54:23.584389+0000","last_active":"2024-09-19T07:54:23.586469+0000","last_peered":"2024-09-19T07:54:23.586469+0000","last_clean":"2024-09-19T07:54:23.586469+0000","last_became_active":"2024-09-19T07:54:23.584121+0000","last_became_peered":"2024-09-19T07:54:23.584121+0000","last_unstale":"2024-09-19T07:54:23.586469+0000","last_undegraded":"2024-09-19T07:54:23.586469+0000","last_fullsized":"2024-09-19T07:54:23.586469+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:35.419442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585626+0000","last_change":"2024-09-19T07:54:23.584963+0000","last_active":"2024-09-19T07:54:23.585626+0000","last_peered":"2024-09-19T07:54:23.585626+0000","last_clean":"2024-09-19T07:54:23.585626+0000","last_became_active":"2024-09-19T07:54:23.583723+0000","last_became_peered":"2024-09-19T07:54:23.583723+0000","last_unstale":"2024-09-19T07:54:23.585626+0000","last_undegraded":"2024-09-19T07:54:23.585626+0000","last_fullsized":"2024-09-19T07:54:23.585626+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:57.397944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.554930+0000","last_change":"2024-09-19T07:54:22.554930+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.554930+0000","last_undegraded":"2024-09-19T07:54:22.554930+0000","last_fullsized":"2024-09-19T07:54:22.554930+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585515+0000","last_change":"2024-09-19T07:54:23.579169+0000","last_active":"2024-09-19T07:54:23.585515+0000","last_peered":"2024-09-19T07:54:23.585515+0000","last_clean":"2024-09-19T07:54:23.585515+0000","last_became_active":"2024-09-19T07:54:23.578888+0000","last_became_peered":"2024-09-19T07:54:23.578888+0000","last_unstale":"2024-09-19T07:54:23.585515+0000","last_undegraded":"2024-09-19T07:54:23.585515+0000","last_fullsized":"2024-09-19T07:54:23.585515+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:09.226711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585479+0000","last_change":"2024-09-19T07:54:23.584332+0000","last_active":"2024-09-19T07:54:23.585479+0000","last_peered":"2024-09-19T07:54:23.585479+0000","last_clean":"2024-09-19T07:54:23.585479+0000","last_became_active":"2024-09-19T07:54:23.584031+0000","last_became_peered":"2024-09-19T07:54:23.584031+0000","last_unstale":"2024-09-19T07:54:23.585479+0000","last_undegraded":"2024-09-19T07:54:23.585479+0000","last_fullsized":"2024-09-19T07:54:23.585479+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:14.753001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572521+0000","last_change":"2024-09-19T07:54:23.574847+0000","last_active":"2024-09-19T07:54:24.572521+0000","last_peered":"2024-09-19T07:54:24.572521+0000","last_clean":"2024-09-19T07:54:24.572521+0000","last_became_active":"2024-09-19T07:54:23.574423+0000","last_became_peered":"2024-09-19T07:54:23.574423+0000","last_unstale":"2024-09-19T07:54:24.572521+0000","last_undegraded":"2024-09-19T07:54:24.572521+0000","last_fullsized":"2024-09-19T07:54:24.572521+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:35:20.465621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.554415+0000","last_change":"2024-09-19T07:54:22.554415+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.554415+0000","last_undegraded":"2024-09-19T07:54:22.554415+0000","last_fullsized":"2024-09-19T07:54:22.554415+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572583+0000","last_change":"2024-09-19T07:54:23.583768+0000","last_active":"2024-09-19T07:54:24.572583+0000","last_peered":"2024-09-19T07:54:24.572583+0000","last_clean":"2024-09-19T07:54:24.572583+0000","last_became_active":"2024-09-19T07:54:23.579038+0000","last_became_peered":"2024-09-19T07:54:23.579038+0000","last_unstale":"2024-09-19T07:54:24.572583+0000","last_undegraded":"2024-09-19T07:54:24.572583+0000","last_fullsized":"2024-09-19T07:54:24.572583+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:16:28.056548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572220+0000","last_change":"2024-09-19T07:54:23.582464+0000","last_active":"2024-09-19T07:54:24.572220+0000","last_peered":"2024-09-19T07:54:24.572220+0000","last_clean":"2024-09-19T07:54:24.572220+0000","last_became_active":"2024-09-19T07:54:23.582176+0000","last_became_peered":"2024-09-19T07:54:23.582176+0000","last_unstale":"2024-09-19T07:54:24.572220+0000","last_undegraded":"2024-09-19T07:54:24.572220+0000","last_fullsized":"2024-09-19T07:54:24.572220+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:25.500256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.557847+0000","last_change":"2024-09-19T07:54:22.557847+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.557847+0000","last_undegraded":"2024-09-19T07:54:22.557847+0000","last_fullsized":"2024-09-19T07:54:22.557847+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.553868+0000","last_change":"2024-09-19T07:54:22.553868+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.553868+0000","last_undegraded":"2024-09-19T07:54:22.553868+0000","last_fullsized":"2024-09-19T07:54:22.553868+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.555444+0000","last_change":"2024-09-19T07:54:22.555444+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.555444+0000","last_undegraded":"2024-09-19T07:54:22.555444+0000","last_fullsized":"2024-09-19T07:54:22.555444+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.581149+0000","last_change":"2024-09-19T07:54:23.580157+0000","last_active":"2024-09-19T07:54:23.581149+0000","last_peered":"2024-09-19T07:54:23.581149+0000","last_clean":"2024-09-19T07:54:23.581149+0000","last_became_active":"2024-09-19T07:54:23.579109+0000","last_became_peered":"2024-09-19T07:54:23.579109+0000","last_unstale":"2024-09-19T07:54:23.581149+0000","last_undegraded":"2024-09-19T07:54:23.581149+0000","last_fullsized":"2024-09-19T07:54:23.581149+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:17.886276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.552956+0000","last_change":"2024-09-19T07:54:22.552956+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.552956+0000","last_undegraded":"2024-09-19T07:54:22.552956+0000","last_fullsized":"2024-09-19T07:54:22.552956+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.568515+0000","last_change":"2024-09-19T07:54:23.567173+0000","last_active":"2024-09-19T07:54:23.568515+0000","last_peered":"2024-09-19T07:54:23.568515+0000","last_clean":"2024-09-19T07:54:23.568515+0000","last_became_active":"2024-09-19T07:54:23.566866+0000","last_became_peered":"2024-09-19T07:54:23.566866+0000","last_unstale":"2024-09-19T07:54:23.568515+0000","last_undegraded":"2024-09-19T07:54:23.568515+0000","last_fullsized":"2024-09-19T07:54:23.568515+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:14.329860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572411+0000","last_change":"2024-09-19T07:54:23.586941+0000","last_active":"2024-09-19T07:54:24.572411+0000","last_peered":"2024-09-19T07:54:24.572411+0000","last_clean":"2024-09-19T07:54:24.572411+0000","last_became_active":"2024-09-19T07:54:23.586367+0000","last_became_peered":"2024-09-19T07:54:23.586367+0000","last_unstale":"2024-09-19T07:54:24.572411+0000","last_undegraded":"2024-09-19T07:54:24.572411+0000","last_fullsized":"2024-09-19T07:54:24.572411+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:22.782628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579831+0000","last_change":"2024-09-19T07:54:23.575696+0000","last_active":"2024-09-19T07:54:23.579831+0000","last_peered":"2024-09-19T07:54:23.579831+0000","last_clean":"2024-09-19T07:54:23.579831+0000","last_became_active":"2024-09-19T07:54:23.575427+0000","last_became_peered":"2024-09-19T07:54:23.575427+0000","last_unstale":"2024-09-19T07:54:23.579831+0000","last_undegraded":"2024-09-19T07:54:23.579831+0000","last_fullsized":"2024-09-19T07:54:23.579831+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:29.663123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.557404+0000","last_change":"2024-09-19T07:54:22.557404+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.557404+0000","last_undegraded":"2024-09-19T07:54:22.557404+0000","last_fullsized":"2024-09-19T07:54:22.557404+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.589396+0000","last_change":"2024-09-19T07:54:23.584866+0000","last_active":"2024-09-19T07:54:23.589396+0000","last_peered":"2024-09-19T07:54:23.589396+0000","last_clean":"2024-09-19T07:54:23.589396+0000","last_became_active":"2024-09-19T07:54:23.582585+0000","last_became_peered":"2024-09-19T07:54:23.582585+0000","last_unstale":"2024-09-19T07:54:23.589396+0000","last_undegraded":"2024-09-19T07:54:23.589396+0000","last_fullsized":"2024-09-19T07:54:23.589396+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:09.647595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.587463+0000","last_change":"2024-09-19T07:54:23.583732+0000","last_active":"2024-09-19T07:54:23.587463+0000","last_peered":"2024-09-19T07:54:23.587463+0000","last_clean":"2024-09-19T07:54:23.587463+0000","last_became_active":"2024-09-19T07:54:23.583460+0000","last_became_peered":"2024-09-19T07:54:23.583460+0000","last_unstale":"2024-09-19T07:54:23.587463+0000","last_undegraded":"2024-09-19T07:54:23.587463+0000","last_fullsized":"2024-09-19T07:54:23.587463+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:01.094793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.581128+0000","last_change":"2024-09-19T07:54:23.579141+0000","last_active":"2024-09-19T07:54:23.581128+0000","last_peered":"2024-09-19T07:54:23.581128+0000","last_clean":"2024-09-19T07:54:23.581128+0000","last_became_active":"2024-09-19T07:54:23.578856+0000","last_became_peered":"2024-09-19T07:54:23.578856+0000","last_unstale":"2024-09-19T07:54:23.581128+0000","last_undegraded":"2024-09-19T07:54:23.581128+0000","last_fullsized":"2024-09-19T07:54:23.581128+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:21.371456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585833+0000","last_change":"2024-09-19T07:54:23.582767+0000","last_active":"2024-09-19T07:54:23.585833+0000","last_peered":"2024-09-19T07:54:23.585833+0000","last_clean":"2024-09-19T07:54:23.585833+0000","last_became_active":"2024-09-19T07:54:23.582486+0000","last_became_peered":"2024-09-19T07:54:23.582486+0000","last_unstale":"2024-09-19T07:54:23.585833+0000","last_undegraded":"2024-09-19T07:54:23.585833+0000","last_fullsized":"2024-09-19T07:54:23.585833+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:32.949629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.589319+0000","last_change":"2024-09-19T07:54:23.587202+0000","last_active":"2024-09-19T07:54:23.589319+0000","last_peered":"2024-09-19T07:54:23.589319+0000","last_clean":"2024-09-19T07:54:23.589319+0000","last_became_active":"2024-09-19T07:54:23.586940+0000","last_became_peered":"2024-09-19T07:54:23.586940+0000","last_unstale":"2024-09-19T07:54:23.589319+0000","last_undegraded":"2024-09-19T07:54:23.589319+0000","last_fullsized":"2024-09-19T07:54:23.589319+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:41.297635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.583319+0000","last_change":"2024-09-19T07:54:23.581592+0000","last_active":"2024-09-19T07:54:23.583319+0000","last_peered":"2024-09-19T07:54:23.583319+0000","last_clean":"2024-09-19T07:54:23.583319+0000","last_became_active":"2024-09-19T07:54:23.581298+0000","last_became_peered":"2024-09-19T07:54:23.581298+0000","last_unstale":"2024-09-19T07:54:23.583319+0000","last_undegraded":"2024-09-19T07:54:23.583319+0000","last_fullsized":"2024-09-19T07:54:23.583319+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:04.317330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576623+0000","last_change":"2024-09-19T07:54:23.575015+0000","last_active":"2024-09-19T07:54:23.576623+0000","last_peered":"2024-09-19T07:54:23.576623+0000","last_clean":"2024-09-19T07:54:23.576623+0000","last_became_active":"2024-09-19T07:54:23.574695+0000","last_became_peered":"2024-09-19T07:54:23.574695+0000","last_unstale":"2024-09-19T07:54:23.576623+0000","last_undegraded":"2024-09-19T07:54:23.576623+0000","last_fullsized":"2024-09-19T07:54:23.576623+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:07.673263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585765+0000","last_change":"2024-09-19T07:54:23.583307+0000","last_active":"2024-09-19T07:54:23.585765+0000","last_peered":"2024-09-19T07:54:23.585765+0000","last_clean":"2024-09-19T07:54:23.585765+0000","last_became_active":"2024-09-19T07:54:23.583021+0000","last_became_peered":"2024-09-19T07:54:23.583021+0000","last_unstale":"2024-09-19T07:54:23.585765+0000","last_undegraded":"2024-09-19T07:54:23.585765+0000","last_fullsized":"2024-09-19T07:54:23.585765+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:38.791089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579025+0000","last_change":"2024-09-19T07:54:23.576203+0000","last_active":"2024-09-19T07:54:23.579025+0000","last_peered":"2024-09-19T07:54:23.579025+0000","last_clean":"2024-09-19T07:54:23.579025+0000","last_became_active":"2024-09-19T07:54:23.575926+0000","last_became_peered":"2024-09-19T07:54:23.575926+0000","last_unstale":"2024-09-19T07:54:23.579025+0000","last_undegraded":"2024-09-19T07:54:23.579025+0000","last_fullsized":"2024-09-19T07:54:23.579025+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:56.777102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"128'4","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.591733+0000","last_change":"2024-09-19T07:54:22.568159+0000","last_active":"2024-09-19T07:54:23.591733+0000","last_peered":"2024-09-19T07:54:23.591733+0000","last_clean":"2024-09-19T07:54:23.591733+0000","last_became_active":"2024-09-19T07:54:22.567832+0000","last_became_peered":"2024-09-19T07:54:22.567832+0000","last_unstale":"2024-09-19T07:54:23.591733+0000","last_undegraded":"2024-09-19T07:54:23.591733+0000","last_fullsized":"2024-09-19T07:54:23.591733+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:46.972795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572726+0000","last_change":"2024-09-19T07:54:23.585429+0000","last_active":"2024-09-19T07:54:24.572726+0000","last_peered":"2024-09-19T07:54:24.572726+0000","last_clean":"2024-09-19T07:54:24.572726+0000","last_became_active":"2024-09-19T07:54:23.585096+0000","last_became_peered":"2024-09-19T07:54:23.585096+0000","last_unstale":"2024-09-19T07:54:24.572726+0000","last_undegraded":"2024-09-19T07:54:24.572726+0000","last_fullsized":"2024-09-19T07:54:24.572726+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:29.192911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565571+0000","last_change":"2024-09-19T07:54:22.561503+0000","last_active":"2024-09-19T07:54:22.565571+0000","last_peered":"2024-09-19T07:54:22.565571+0000","last_clean":"2024-09-19T07:54:22.565571+0000","last_became_active":"2024-09-19T07:54:22.561219+0000","last_became_peered":"2024-09-19T07:54:22.561219+0000","last_unstale":"2024-09-19T07:54:22.565571+0000","last_undegraded":"2024-09-19T07:54:22.565571+0000","last_fullsized":"2024-09-19T07:54:22.565571+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:18.897568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579803+0000","last_change":"2024-09-19T07:54:23.574954+0000","last_active":"2024-09-19T07:54:23.579803+0000","last_peered":"2024-09-19T07:54:23.579803+0000","last_clean":"2024-09-19T07:54:23.579803+0000","last_became_active":"2024-09-19T07:54:23.574624+0000","last_became_peered":"2024-09-19T07:54:23.574624+0000","last_unstale":"2024-09-19T07:54:23.579803+0000","last_undegraded":"2024-09-19T07:54:23.579803+0000","last_fullsized":"2024-09-19T07:54:23.579803+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:32.420071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"128'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.592293+0000","last_change":"2024-09-19T07:54:22.566018+0000","last_active":"2024-09-19T07:54:23.592293+0000","last_peered":"2024-09-19T07:54:23.592293+0000","last_clean":"2024-09-19T07:54:23.592293+0000","last_became_active":"2024-09-19T07:54:22.565812+0000","last_became_peered":"2024-09-19T07:54:22.565812+0000","last_unstale":"2024-09-19T07:54:23.592293+0000","last_undegraded":"2024-09-19T07:54:23.592293+0000","last_fullsized":"2024-09-19T07:54:23.592293+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:01.728173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572245+0000","last_change":"2024-09-19T07:54:23.582120+0000","last_active":"2024-09-19T07:54:24.572245+0000","last_peered":"2024-09-19T07:54:24.572245+0000","last_clean":"2024-09-19T07:54:24.572245+0000","last_became_active":"2024-09-19T07:54:23.581830+0000","last_became_peered":"2024-09-19T07:54:23.581830+0000","last_unstale":"2024-09-19T07:54:24.572245+0000","last_undegraded":"2024-09-19T07:54:24.572245+0000","last_fullsized":"2024-09-19T07:54:24.572245+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:12.630683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"128'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.591308+0000","last_change":"2024-09-19T07:54:22.560370+0000","last_active":"2024-09-19T07:54:23.591308+0000","last_peered":"2024-09-19T07:54:23.591308+0000","last_clean":"2024-09-19T07:54:23.591308+0000","last_became_active":"2024-09-19T07:54:22.560083+0000","last_became_peered":"2024-09-19T07:54:22.560083+0000","last_unstale":"2024-09-19T07:54:23.591308+0000","last_undegraded":"2024-09-19T07:54:23.591308+0000","last_fullsized":"2024-09-19T07:54:23.591308+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:51.100115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579407+0000","last_change":"2024-09-19T07:54:23.577913+0000","last_active":"2024-09-19T07:54:23.579407+0000","last_peered":"2024-09-19T07:54:23.579407+0000","last_clean":"2024-09-19T07:54:23.579407+0000","last_became_active":"2024-09-19T07:54:23.577102+0000","last_became_peered":"2024-09-19T07:54:23.577102+0000","last_unstale":"2024-09-19T07:54:23.579407+0000","last_undegraded":"2024-09-19T07:54:23.579407+0000","last_fullsized":"2024-09-19T07:54:23.579407+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:49.188816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"128'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.586957+0000","last_change":"2024-09-19T07:54:22.554611+0000","last_active":"2024-09-19T07:54:23.586957+0000","last_peered":"2024-09-19T07:54:23.586957+0000","last_clean":"2024-09-19T07:54:23.586957+0000","last_became_active":"2024-09-19T07:54:22.554332+0000","last_became_peered":"2024-09-19T07:54:22.554332+0000","last_unstale":"2024-09-19T07:54:23.586957+0000","last_undegraded":"2024-09-19T07:54:23.586957+0000","last_fullsized":"2024-09-19T07:54:23.586957+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:09.508289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576791+0000","last_change":"2024-09-19T07:54:23.574521+0000","last_active":"2024-09-19T07:54:23.576791+0000","last_peered":"2024-09-19T07:54:23.576791+0000","last_clean":"2024-09-19T07:54:23.576791+0000","last_became_active":"2024-09-19T07:54:23.574251+0000","last_became_peered":"2024-09-19T07:54:23.574251+0000","last_unstale":"2024-09-19T07:54:23.576791+0000","last_undegraded":"2024-09-19T07:54:23.576791+0000","last_fullsized":"2024-09-19T07:54:23.576791+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:31.699850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565667+0000","last_change":"2024-09-19T07:54:22.564044+0000","last_active":"2024-09-19T07:54:22.565667+0000","last_peered":"2024-09-19T07:54:22.565667+0000","last_clean":"2024-09-19T07:54:22.565667+0000","last_became_active":"2024-09-19T07:54:22.562851+0000","last_became_peered":"2024-09-19T07:54:22.562851+0000","last_unstale":"2024-09-19T07:54:22.565667+0000","last_undegraded":"2024-09-19T07:54:22.565667+0000","last_fullsized":"2024-09-19T07:54:22.565667+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:34.414794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.555018+0000","last_change":"2024-09-19T07:54:22.555018+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.555018+0000","last_undegraded":"2024-09-19T07:54:22.555018+0000","last_fullsized":"2024-09-19T07:54:22.555018+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.566646+0000","last_change":"2024-09-19T07:54:22.557995+0000","last_active":"2024-09-19T07:54:22.566646+0000","last_peered":"2024-09-19T07:54:22.566646+0000","last_clean":"2024-09-19T07:54:22.566646+0000","last_became_active":"2024-09-19T07:54:22.557692+0000","last_became_peered":"2024-09-19T07:54:22.557692+0000","last_unstale":"2024-09-19T07:54:22.566646+0000","last_undegraded":"2024-09-19T07:54:22.566646+0000","last_fullsized":"2024-09-19T07:54:22.566646+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:01.620005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576699+0000","last_change":"2024-09-19T07:54:23.573943+0000","last_active":"2024-09-19T07:54:23.576699+0000","last_peered":"2024-09-19T07:54:23.576699+0000","last_clean":"2024-09-19T07:54:23.576699+0000","last_became_active":"2024-09-19T07:54:23.573668+0000","last_became_peered":"2024-09-19T07:54:23.573668+0000","last_unstale":"2024-09-19T07:54:23.576699+0000","last_undegraded":"2024-09-19T07:54:23.576699+0000","last_fullsized":"2024-09-19T07:54:23.576699+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:13:01.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.566981+0000","last_change":"2024-09-19T07:54:22.562903+0000","last_active":"2024-09-19T07:54:23.566981+0000","last_peered":"2024-09-19T07:54:23.566981+0000","last_clean":"2024-09-19T07:54:23.566981+0000","last_became_active":"2024-09-19T07:54:22.562599+0000","last_became_peered":"2024-09-19T07:54:22.562599+0000","last_unstale":"2024-09-19T07:54:23.566981+0000","last_undegraded":"2024-09-19T07:54:23.566981+0000","last_fullsized":"2024-09-19T07:54:23.566981+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:44.078560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572071+0000","last_change":"2024-09-19T07:54:23.586564+0000","last_active":"2024-09-19T07:54:24.572071+0000","last_peered":"2024-09-19T07:54:24.572071+0000","last_clean":"2024-09-19T07:54:24.572071+0000","last_became_active":"2024-09-19T07:54:23.586246+0000","last_became_peered":"2024-09-19T07:54:23.586246+0000","last_unstale":"2024-09-19T07:54:24.572071+0000","last_undegraded":"2024-09-19T07:54:24.572071+0000","last_fullsized":"2024-09-19T07:54:24.572071+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:14:59.560377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.563273+0000","last_change":"2024-09-19T07:54:22.556612+0000","last_active":"2024-09-19T07:54:23.563273+0000","last_peered":"2024-09-19T07:54:23.563273+0000","last_clean":"2024-09-19T07:54:23.563273+0000","last_became_active":"2024-09-19T07:54:22.556177+0000","last_became_peered":"2024-09-19T07:54:22.556177+0000","last_unstale":"2024-09-19T07:54:23.563273+0000","last_undegraded":"2024-09-19T07:54:23.563273+0000","last_fullsized":"2024-09-19T07:54:23.563273+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:25.294939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.556350+0000","last_change":"2024-09-19T07:54:22.556350+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.556350+0000","last_undegraded":"2024-09-19T07:54:22.556350+0000","last_fullsized":"2024-09-19T07:54:22.556350+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572966+0000","last_change":"2024-09-19T07:54:22.564375+0000","last_active":"2024-09-19T07:54:24.572966+0000","last_peered":"2024-09-19T07:54:24.572966+0000","last_clean":"2024-09-19T07:54:24.572966+0000","last_became_active":"2024-09-19T07:54:22.564096+0000","last_became_peered":"2024-09-19T07:54:22.564096+0000","last_unstale":"2024-09-19T07:54:24.572966+0000","last_undegraded":"2024-09-19T07:54:24.572966+0000","last_fullsized":"2024-09-19T07:54:24.572966+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:05.452375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.584721+0000","last_change":"2024-09-19T07:54:23.583678+0000","last_active":"2024-09-19T07:54:23.584721+0000","last_peered":"2024-09-19T07:54:23.584721+0000","last_clean":"2024-09-19T07:54:23.584721+0000","last_became_active":"2024-09-19T07:54:23.583254+0000","last_became_peered":"2024-09-19T07:54:23.583254+0000","last_unstale":"2024-09-19T07:54:23.584721+0000","last_undegraded":"2024-09-19T07:54:23.584721+0000","last_fullsized":"2024-09-19T07:54:23.584721+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:46.495632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"128'3","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.593450+0000","last_change":"2024-09-19T07:54:22.558733+0000","last_active":"2024-09-19T07:54:23.593450+0000","last_peered":"2024-09-19T07:54:23.593450+0000","last_clean":"2024-09-19T07:54:23.593450+0000","last_became_active":"2024-09-19T07:54:22.558481+0000","last_became_peered":"2024-09-19T07:54:22.558481+0000","last_unstale":"2024-09-19T07:54:23.593450+0000","last_undegraded":"2024-09-19T07:54:23.593450+0000","last_fullsized":"2024-09-19T07:54:23.593450+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:41.801926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.590832+0000","last_change":"2024-09-19T07:54:23.589083+0000","last_active":"2024-09-19T07:54:23.590832+0000","last_peered":"2024-09-19T07:54:23.590832+0000","last_clean":"2024-09-19T07:54:23.590832+0000","last_became_active":"2024-09-19T07:54:23.588728+0000","last_became_peered":"2024-09-19T07:54:23.588728+0000","last_unstale":"2024-09-19T07:54:23.590832+0000","last_undegraded":"2024-09-19T07:54:23.590832+0000","last_fullsized":"2024-09-19T07:54:23.590832+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:44.660536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"128'6","reported_seq":18,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.594644+0000","last_change":"2024-09-19T07:54:22.564291+0000","last_active":"2024-09-19T07:54:23.594644+0000","last_peered":"2024-09-19T07:54:23.594644+0000","last_clean":"2024-09-19T07:54:23.594644+0000","last_became_active":"2024-09-19T07:54:22.564012+0000","last_became_peered":"2024-09-19T07:54:22.564012+0000","last_unstale":"2024-09-19T07:54:23.594644+0000","last_undegraded":"2024-09-19T07:54:23.594644+0000","last_fullsized":"2024-09-19T07:54:23.594644+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:00.025001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.582649+0000","last_change":"2024-09-19T07:54:23.578794+0000","last_active":"2024-09-19T07:54:23.582649+0000","last_peered":"2024-09-19T07:54:23.582649+0000","last_clean":"2024-09-19T07:54:23.582649+0000","last_became_active":"2024-09-19T07:54:23.578465+0000","last_became_peered":"2024-09-19T07:54:23.578465+0000","last_unstale":"2024-09-19T07:54:23.582649+0000","last_undegraded":"2024-09-19T07:54:23.582649+0000","last_fullsized":"2024-09-19T07:54:23.582649+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:57.295016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"128'2","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572360+0000","last_change":"2024-09-19T07:54:22.565043+0000","last_active":"2024-09-19T07:54:24.572360+0000","last_peered":"2024-09-19T07:54:24.572360+0000","last_clean":"2024-09-19T07:54:24.572360+0000","last_became_active":"2024-09-19T07:54:22.564758+0000","last_became_peered":"2024-09-19T07:54:22.564758+0000","last_unstale":"2024-09-19T07:54:24.572360+0000","last_undegraded":"2024-09-19T07:54:24.572360+0000","last_fullsized":"2024-09-19T07:54:24.572360+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:58:28.017221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576888+0000","last_change":"2024-09-19T07:54:23.574862+0000","last_active":"2024-09-19T07:54:23.576888+0000","last_peered":"2024-09-19T07:54:23.576888+0000","last_clean":"2024-09-19T07:54:23.576888+0000","last_became_active":"2024-09-19T07:54:23.574431+0000","last_became_peered":"2024-09-19T07:54:23.574431+0000","last_unstale":"2024-09-19T07:54:23.576888+0000","last_undegraded":"2024-09-19T07:54:23.576888+0000","last_fullsized":"2024-09-19T07:54:23.576888+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:29.913764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.567482+0000","last_change":"2024-09-19T07:54:22.559040+0000","last_active":"2024-09-19T07:54:23.567482+0000","last_peered":"2024-09-19T07:54:23.567482+0000","last_clean":"2024-09-19T07:54:23.567482+0000","last_became_active":"2024-09-19T07:54:22.558757+0000","last_became_peered":"2024-09-19T07:54:22.558757+0000","last_unstale":"2024-09-19T07:54:23.567482+0000","last_undegraded":"2024-09-19T07:54:23.567482+0000","last_fullsized":"2024-09-19T07:54:23.567482+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:28:24.370408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.551690+0000","last_change":"2024-09-19T07:54:22.551690+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.551690+0000","last_undegraded":"2024-09-19T07:54:22.551690+0000","last_fullsized":"2024-09-19T07:54:22.551690+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565358+0000","last_change":"2024-09-19T07:54:22.559203+0000","last_active":"2024-09-19T07:54:22.565358+0000","last_peered":"2024-09-19T07:54:22.565358+0000","last_clean":"2024-09-19T07:54:22.565358+0000","last_became_active":"2024-09-19T07:54:22.558920+0000","last_became_peered":"2024-09-19T07:54:22.558920+0000","last_unstale":"2024-09-19T07:54:22.565358+0000","last_undegraded":"2024-09-19T07:54:22.565358+0000","last_fullsized":"2024-09-19T07:54:22.565358+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:43.655888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.589459+0000","last_change":"2024-09-19T07:54:23.587980+0000","last_active":"2024-09-19T07:54:23.589459+0000","last_peered":"2024-09-19T07:54:23.589459+0000","last_clean":"2024-09-19T07:54:23.589459+0000","last_became_active":"2024-09-19T07:54:23.587702+0000","last_became_peered":"2024-09-19T07:54:23.587702+0000","last_unstale":"2024-09-19T07:54:23.589459+0000","last_undegraded":"2024-09-19T07:54:23.589459+0000","last_fullsized":"2024-09-19T07:54:23.589459+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:02.624652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"128'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.578744+0000","last_change":"2024-09-19T07:54:22.553965+0000","last_active":"2024-09-19T07:54:23.578744+0000","last_peered":"2024-09-19T07:54:23.578744+0000","last_clean":"2024-09-19T07:54:23.578744+0000","last_became_active":"2024-09-19T07:54:22.553678+0000","last_became_peered":"2024-09-19T07:54:22.553678+0000","last_unstale":"2024-09-19T07:54:23.578744+0000","last_undegraded":"2024-09-19T07:54:23.578744+0000","last_fullsized":"2024-09-19T07:54:23.578744+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:06.364113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576816+0000","last_change":"2024-09-19T07:54:23.574769+0000","last_active":"2024-09-19T07:54:23.576816+0000","last_peered":"2024-09-19T07:54:23.576816+0000","last_clean":"2024-09-19T07:54:23.576816+0000","last_became_active":"2024-09-19T07:54:23.574488+0000","last_became_peered":"2024-09-19T07:54:23.574488+0000","last_unstale":"2024-09-19T07:54:23.576816+0000","last_undegraded":"2024-09-19T07:54:23.576816+0000","last_fullsized":"2024-09-19T07:54:23.576816+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:41.287209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"128'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.591321+0000","last_change":"2024-09-19T07:54:22.567957+0000","last_active":"2024-09-19T07:54:23.591321+0000","last_peered":"2024-09-19T07:54:23.591321+0000","last_clean":"2024-09-19T07:54:23.591321+0000","last_became_active":"2024-09-19T07:54:22.567503+0000","last_became_peered":"2024-09-19T07:54:22.567503+0000","last_unstale":"2024-09-19T07:54:23.591321+0000","last_undegraded":"2024-09-19T07:54:23.591321+0000","last_fullsized":"2024-09-19T07:54:23.591321+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:06.620754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.580616+0000","last_change":"2024-09-19T07:54:23.578211+0000","last_active":"2024-09-19T07:54:23.580616+0000","last_peered":"2024-09-19T07:54:23.580616+0000","last_clean":"2024-09-19T07:54:23.580616+0000","last_became_active":"2024-09-19T07:54:23.577649+0000","last_became_peered":"2024-09-19T07:54:23.577649+0000","last_unstale":"2024-09-19T07:54:23.580616+0000","last_undegraded":"2024-09-19T07:54:23.580616+0000","last_fullsized":"2024-09-19T07:54:23.580616+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:19.902216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565179+0000","last_change":"2024-09-19T07:54:22.559404+0000","last_active":"2024-09-19T07:54:22.565179+0000","last_peered":"2024-09-19T07:54:22.565179+0000","last_clean":"2024-09-19T07:54:22.565179+0000","last_became_active":"2024-09-19T07:54:22.559101+0000","last_became_peered":"2024-09-19T07:54:22.559101+0000","last_unstale":"2024-09-19T07:54:22.565179+0000","last_undegraded":"2024-09-19T07:54:22.565179+0000","last_fullsized":"2024-09-19T07:54:22.565179+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:30.695202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.587444+0000","last_change":"2024-09-19T07:54:23.585316+0000","last_active":"2024-09-19T07:54:23.587444+0000","last_peered":"2024-09-19T07:54:23.587444+0000","last_clean":"2024-09-19T07:54:23.587444+0000","last_became_active":"2024-09-19T07:54:23.585029+0000","last_became_peered":"2024-09-19T07:54:23.585029+0000","last_unstale":"2024-09-19T07:54:23.587444+0000","last_undegraded":"2024-09-19T07:54:23.587444+0000","last_fullsized":"2024-09-19T07:54:23.587444+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:35.538510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565841+0000","last_change":"2024-09-19T07:54:22.563785+0000","last_active":"2024-09-19T07:54:22.565841+0000","last_peered":"2024-09-19T07:54:22.565841+0000","last_clean":"2024-09-19T07:54:22.565841+0000","last_became_active":"2024-09-19T07:54:22.563471+0000","last_became_peered":"2024-09-19T07:54:22.563471+0000","last_unstale":"2024-09-19T07:54:22.565841+0000","last_undegraded":"2024-09-19T07:54:22.565841+0000","last_fullsized":"2024-09-19T07:54:22.565841+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:16.881628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.552495+0000","last_change":"2024-09-19T07:54:22.552495+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.552495+0000","last_undegraded":"2024-09-19T07:54:22.552495+0000","last_fullsized":"2024-09-19T07:54:22.552495+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"128'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.586262+0000","last_change":"2024-09-19T07:54:22.556136+0000","last_active":"2024-09-19T07:54:23.586262+0000","last_peered":"2024-09-19T07:54:23.586262+0000","last_clean":"2024-09-19T07:54:23.586262+0000","last_became_active":"2024-09-19T07:54:22.555829+0000","last_became_peered":"2024-09-19T07:54:22.555829+0000","last_unstale":"2024-09-19T07:54:23.586262+0000","last_undegraded":"2024-09-19T07:54:23.586262+0000","last_fullsized":"2024-09-19T07:54:23.586262+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:37.052983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572425+0000","last_change":"2024-09-19T07:54:23.582629+0000","last_active":"2024-09-19T07:54:24.572425+0000","last_peered":"2024-09-19T07:54:24.572425+0000","last_clean":"2024-09-19T07:54:24.572425+0000","last_became_active":"2024-09-19T07:54:23.582343+0000","last_became_peered":"2024-09-19T07:54:23.582343+0000","last_unstale":"2024-09-19T07:54:24.572425+0000","last_undegraded":"2024-09-19T07:54:24.572425+0000","last_fullsized":"2024-09-19T07:54:24.572425+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:38.135921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572535+0000","last_change":"2024-09-19T07:54:22.563856+0000","last_active":"2024-09-19T07:54:24.572535+0000","last_peered":"2024-09-19T07:54:24.572535+0000","last_clean":"2024-09-19T07:54:24.572535+0000","last_became_active":"2024-09-19T07:54:22.563515+0000","last_became_peered":"2024-09-19T07:54:22.563515+0000","last_unstale":"2024-09-19T07:54:24.572535+0000","last_undegraded":"2024-09-19T07:54:24.572535+0000","last_fullsized":"2024-09-19T07:54:24.572535+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:32.880703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.554597+0000","last_change":"2024-09-19T07:54:22.554597+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.554597+0000","last_undegraded":"2024-09-19T07:54:22.554597+0000","last_fullsized":"2024-09-19T07:54:22.554597+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.568962+0000","last_change":"2024-09-19T07:54:22.564048+0000","last_active":"2024-09-19T07:54:23.568962+0000","last_peered":"2024-09-19T07:54:23.568962+0000","last_clean":"2024-09-19T07:54:23.568962+0000","last_became_active":"2024-09-19T07:54:22.563775+0000","last_became_peered":"2024-09-19T07:54:22.563775+0000","last_unstale":"2024-09-19T07:54:23.568962+0000","last_undegraded":"2024-09-19T07:54:23.568962+0000","last_fullsized":"2024-09-19T07:54:23.568962+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:57.190350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585516+0000","last_change":"2024-09-19T07:54:23.584566+0000","last_active":"2024-09-19T07:54:23.585516+0000","last_peered":"2024-09-19T07:54:23.585516+0000","last_clean":"2024-09-19T07:54:23.585516+0000","last_became_active":"2024-09-19T07:54:23.584002+0000","last_became_peered":"2024-09-19T07:54:23.584002+0000","last_unstale":"2024-09-19T07:54:23.585516+0000","last_undegraded":"2024-09-19T07:54:23.585516+0000","last_fullsized":"2024-09-19T07:54:23.585516+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:58.194998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565335+0000","last_change":"2024-09-19T07:54:22.558583+0000","last_active":"2024-09-19T07:54:22.565335+0000","last_peered":"2024-09-19T07:54:22.565335+0000","last_clean":"2024-09-19T07:54:22.565335+0000","last_became_active":"2024-09-19T07:54:22.558239+0000","last_became_peered":"2024-09-19T07:54:22.558239+0000","last_unstale":"2024-09-19T07:54:22.565335+0000","last_undegraded":"2024-09-19T07:54:22.565335+0000","last_fullsized":"2024-09-19T07:54:22.565335+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:37.052983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.553358+0000","last_change":"2024-09-19T07:54:22.553358+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.553358+0000","last_undegraded":"2024-09-19T07:54:22.553358+0000","last_fullsized":"2024-09-19T07:54:22.553358+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"128'2","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572673+0000","last_change":"2024-09-19T07:54:22.561535+0000","last_active":"2024-09-19T07:54:24.572673+0000","last_peered":"2024-09-19T07:54:24.572673+0000","last_clean":"2024-09-19T07:54:24.572673+0000","last_became_active":"2024-09-19T07:54:22.561210+0000","last_became_peered":"2024-09-19T07:54:22.561210+0000","last_unstale":"2024-09-19T07:54:24.572673+0000","last_undegraded":"2024-09-19T07:54:24.572673+0000","last_fullsized":"2024-09-19T07:54:24.572673+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:13.281363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.590668+0000","last_change":"2024-09-19T07:54:23.588433+0000","last_active":"2024-09-19T07:54:23.590668+0000","last_peered":"2024-09-19T07:54:23.590668+0000","last_clean":"2024-09-19T07:54:23.590668+0000","last_became_active":"2024-09-19T07:54:23.587803+0000","last_became_peered":"2024-09-19T07:54:23.587803+0000","last_unstale":"2024-09-19T07:54:23.590668+0000","last_undegraded":"2024-09-19T07:54:23.590668+0000","last_fullsized":"2024-09-19T07:54:23.590668+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:10.512937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.562219+0000","last_change":"2024-09-19T07:54:22.555487+0000","last_active":"2024-09-19T07:54:23.562219+0000","last_peered":"2024-09-19T07:54:23.562219+0000","last_clean":"2024-09-19T07:54:23.562219+0000","last_became_active":"2024-09-19T07:54:22.555215+0000","last_became_peered":"2024-09-19T07:54:22.555215+0000","last_unstale":"2024-09-19T07:54:23.562219+0000","last_undegraded":"2024-09-19T07:54:23.562219+0000","last_fullsized":"2024-09-19T07:54:23.562219+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:01.620005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.586495+0000","last_change":"2024-09-19T07:54:23.585093+0000","last_active":"2024-09-19T07:54:23.586495+0000","last_peered":"2024-09-19T07:54:23.586495+0000","last_clean":"2024-09-19T07:54:23.586495+0000","last_became_active":"2024-09-19T07:54:23.584837+0000","last_became_peered":"2024-09-19T07:54:23.584837+0000","last_unstale":"2024-09-19T07:54:23.586495+0000","last_undegraded":"2024-09-19T07:54:23.586495+0000","last_fullsized":"2024-09-19T07:54:23.586495+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:43.491587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.565590+0000","last_change":"2024-09-19T07:54:22.564147+0000","last_active":"2024-09-19T07:54:23.565590+0000","last_peered":"2024-09-19T07:54:23.565590+0000","last_clean":"2024-09-19T07:54:23.565590+0000","last_became_active":"2024-09-19T07:54:22.563844+0000","last_became_peered":"2024-09-19T07:54:22.563844+0000","last_unstale":"2024-09-19T07:54:23.565590+0000","last_undegraded":"2024-09-19T07:54:23.565590+0000","last_fullsized":"2024-09-19T07:54:23.565590+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:43.655888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585486+0000","last_change":"2024-09-19T07:54:23.582012+0000","last_active":"2024-09-19T07:54:23.585486+0000","last_peered":"2024-09-19T07:54:23.585486+0000","last_clean":"2024-09-19T07:54:23.585486+0000","last_became_active":"2024-09-19T07:54:23.581143+0000","last_became_peered":"2024-09-19T07:54:23.581143+0000","last_unstale":"2024-09-19T07:54:23.585486+0000","last_undegraded":"2024-09-19T07:54:23.585486+0000","last_fullsized":"2024-09-19T07:54:23.585486+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:20.783132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565313+0000","last_change":"2024-09-19T07:54:22.558393+0000","last_active":"2024-09-19T07:54:22.565313+0000","last_peered":"2024-09-19T07:54:22.565313+0000","last_clean":"2024-09-19T07:54:22.565313+0000","last_became_active":"2024-09-19T07:54:22.558091+0000","last_became_peered":"2024-09-19T07:54:22.558091+0000","last_unstale":"2024-09-19T07:54:22.565313+0000","last_undegraded":"2024-09-19T07:54:22.565313+0000","last_fullsized":"2024-09-19T07:54:22.565313+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:09.508289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.549695+0000","last_change":"2024-09-19T07:54:22.549695+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.549695+0000","last_undegraded":"2024-09-19T07:54:22.549695+0000","last_fullsized":"2024-09-19T07:54:22.549695+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.562667+0000","last_change":"2024-09-19T07:54:22.567926+0000","last_active":"2024-09-19T07:54:23.562667+0000","last_peered":"2024-09-19T07:54:23.562667+0000","last_clean":"2024-09-19T07:54:23.562667+0000","last_became_active":"2024-09-19T07:54:22.567617+0000","last_became_peered":"2024-09-19T07:54:22.567617+0000","last_unstale":"2024-09-19T07:54:23.562667+0000","last_undegraded":"2024-09-19T07:54:23.562667+0000","last_fullsized":"2024-09-19T07:54:23.562667+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:41:15.024194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.554395+0000","last_change":"2024-09-19T07:54:22.554395+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.554395+0000","last_undegraded":"2024-09-19T07:54:22.554395+0000","last_fullsized":"2024-09-19T07:54:22.554395+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.570070+0000","last_change":"2024-09-19T07:54:22.558104+0000","last_active":"2024-09-19T07:54:23.570070+0000","last_peered":"2024-09-19T07:54:23.570070+0000","last_clean":"2024-09-19T07:54:23.570070+0000","last_became_active":"2024-09-19T07:54:22.557825+0000","last_became_peered":"2024-09-19T07:54:22.557825+0000","last_unstale":"2024-09-19T07:54:23.570070+0000","last_undegraded":"2024-09-19T07:54:23.570070+0000","last_fullsized":"2024-09-19T07:54:23.570070+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:36:36.185111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.553286+0000","last_change":"2024-09-19T07:54:22.553286+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.553286+0000","last_undegraded":"2024-09-19T07:54:22.553286+0000","last_fullsized":"2024-09-19T07:54:22.553286+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.566668+0000","last_change":"2024-09-19T07:54:22.560118+0000","last_active":"2024-09-19T07:54:22.566668+0000","last_peered":"2024-09-19T07:54:22.566668+0000","last_clean":"2024-09-19T07:54:22.566668+0000","last_became_active":"2024-09-19T07:54:22.559838+0000","last_became_peered":"2024-09-19T07:54:22.559838+0000","last_unstale":"2024-09-19T07:54:22.566668+0000","last_undegraded":"2024-09-19T07:54:22.566668+0000","last_fullsized":"2024-09-19T07:54:22.566668+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:06.668616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.587234+0000","last_change":"2024-09-19T07:54:23.586194+0000","last_active":"2024-09-19T07:54:23.587234+0000","last_peered":"2024-09-19T07:54:23.587234+0000","last_clean":"2024-09-19T07:54:23.587234+0000","last_became_active":"2024-09-19T07:54:23.585946+0000","last_became_peered":"2024-09-19T07:54:23.585946+0000","last_unstale":"2024-09-19T07:54:23.587234+0000","last_undegraded":"2024-09-19T07:54:23.587234+0000","last_fullsized":"2024-09-19T07:54:23.587234+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:27.045701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565875+0000","last_change":"2024-09-19T07:54:22.562538+0000","last_active":"2024-09-19T07:54:22.565875+0000","last_peered":"2024-09-19T07:54:22.565875+0000","last_clean":"2024-09-19T07:54:22.565875+0000","last_became_active":"2024-09-19T07:54:22.562257+0000","last_became_peered":"2024-09-19T07:54:22.562257+0000","last_unstale":"2024-09-19T07:54:22.565875+0000","last_undegraded":"2024-09-19T07:54:22.565875+0000","last_fullsized":"2024-09-19T07:54:22.565875+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:20.366808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.590798+0000","last_change":"2024-09-19T07:54:23.588887+0000","last_active":"2024-09-19T07:54:23.590798+0000","last_peered":"2024-09-19T07:54:23.590798+0000","last_clean":"2024-09-19T07:54:23.590798+0000","last_became_active":"2024-09-19T07:54:23.588623+0000","last_became_peered":"2024-09-19T07:54:23.588623+0000","last_unstale":"2024-09-19T07:54:23.590798+0000","last_undegraded":"2024-09-19T07:54:23.590798+0000","last_fullsized":"2024-09-19T07:54:23.590798+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:38.057631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.569582+0000","last_change":"2024-09-19T07:54:22.565171+0000","last_active":"2024-09-19T07:54:23.569582+0000","last_peered":"2024-09-19T07:54:23.569582+0000","last_clean":"2024-09-19T07:54:23.569582+0000","last_became_active":"2024-09-19T07:54:22.564876+0000","last_became_peered":"2024-09-19T07:54:22.564876+0000","last_unstale":"2024-09-19T07:54:23.569582+0000","last_undegraded":"2024-09-19T07:54:23.569582+0000","last_fullsized":"2024-09-19T07:54:23.569582+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:36.291156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":364,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.567647+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:54:23.567647+0000","last_peered":"2024-09-19T07:54:23.567647+0000","last_clean":"2024-09-19T07:54:23.567647+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:54:23.567647+0000","last_undegraded":"2024-09-19T07:54:23.567647+0000","last_fullsized":"2024-09-19T07:54:23.567647+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509581,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83268,"kb_used_data":2152,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288572,"statfs":{"total":96636764160,"available":96551497728,"internally_reserved":0,"allocated":2203648,"data_stored":1331051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509580,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104952,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266888,"statfs":{"total":96636764160,"available":96529293312,"internally_reserved":0,"allocated":1601536,"data_stored":739653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509581,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72944,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298896,"statfs":{"total":96636764160,"available":96562069504,"internally_reserved":0,"allocated":1597440,"data_stored":739003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509581,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86832,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285008,"statfs":{"total":96636764160,"available":96547848192,"internally_reserved":0,"allocated":1593344,"data_stored":738981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509580,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105460,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266380,"statfs":{"total":96636764160,"available":96528773120,"internally_reserved":0,"allocated":2195456,"data_stored":1329405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542284,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118748,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253092,"statfs":{"total":96636764160,"available":96515166208,"internally_reserved":0,"allocated":1581056,"data_stored":731814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542284,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82720,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94289120,"statfs":{"total":96636764160,"available":96552058880,"internally_reserved":0,"allocated":1581056,"data_stored":731814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574990,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109052,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262788,"statfs":{"total":96636764160,"available":96525094912,"internally_reserved":0,"allocated":1613824,"data_stored":746289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:54:26.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:54:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.040+0000 7fd3029fd640 1 Processor -- start 2024-09-19T07:54:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.040+0000 7fd3029fd640 1 -- start start 2024-09-19T07:54:27.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.041+0000 7fd3029fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2fc15e2a0 0x7fd2fc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.041+0000 7fd3029fd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd2fc15e680 0x7fd2fc0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.041+0000 7fd3029fd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2fc0d1880 con 0x7fd2fc06d960 2024-09-19T07:54:27.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.041+0000 7fd3029fd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fd2fc0d1b60 con 0x7fd2fc15e680 2024-09-19T07:54:27.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.041+0000 7fd3029fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd2fc0cf2d0 con 0x7fd2fc15e2a0 2024-09-19T07:54:27.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.041+0000 7fd2fbfff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2fc15e2a0 0x7fd2fc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.041+0000 7fd2fbfff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2fc15e2a0 0x7fd2fc06d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:37238/0 (socket says 172.21.15.16:37238) 2024-09-19T07:54:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.041+0000 7fd2fbfff640 1 -- 172.21.15.16:0/2811781104 learned_addr learned my addr 172.21.15.16:0/2811781104 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.041+0000 7fd2fb7fe640 1 --2- 172.21.15.16:0/2811781104 >> v2:172.21.15.146:3300/0 conn(0x7fd2fc15e680 0x7fd2fc0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.042+0000 7fd2faffd640 1 -- 172.21.15.16:0/2811781104 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1911966433 0 0) 0x7fd2fc0d1880 con 0x7fd2fc06d960 2024-09-19T07:54:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.042+0000 7fd2faffd640 1 -- 172.21.15.16:0/2811781104 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2e4003520 con 0x7fd2fc06d960 2024-09-19T07:54:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.042+0000 7fd2fbfff640 1 -- 172.21.15.16:0/2811781104 >> v2:172.21.15.146:3300/0 conn(0x7fd2fc15e680 msgr2=0x7fd2fc0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.042+0000 7fd2fbfff640 1 --2- 172.21.15.16:0/2811781104 >> v2:172.21.15.146:3300/0 conn(0x7fd2fc15e680 0x7fd2fc0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.042+0000 7fd2fbfff640 1 -- 172.21.15.16:0/2811781104 >> v1:172.21.15.146:6789/0 conn(0x7fd2fc06d960 legacy=0x7fd2fc0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.042+0000 7fd2fbfff640 1 -- 172.21.15.16:0/2811781104 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc0cf5b0 con 0x7fd2fc15e2a0 2024-09-19T07:54:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.042+0000 7fd2fbfff640 1 --2- 172.21.15.16:0/2811781104 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2fc15e2a0 0x7fd2fc06d420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009870 tx=0x7fd2ec02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=312f7a0a032d1aa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:27.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.043+0000 7fd2faffd640 1 -- 172.21.15.16:0/2811781104 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec037070 con 0x7fd2fc15e2a0 2024-09-19T07:54:27.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.043+0000 7fd2faffd640 1 -- 172.21.15.16:0/2811781104 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ec032440 con 0x7fd2fc15e2a0 2024-09-19T07:54:27.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.043+0000 7fd2faffd640 1 -- 172.21.15.16:0/2811781104 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec0327f0 con 0x7fd2fc15e2a0 2024-09-19T07:54:27.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.043+0000 7fd3029fd640 1 -- 172.21.15.16:0/2811781104 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2fc15e2a0 msgr2=0x7fd2fc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:27.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.043+0000 7fd3029fd640 1 --2- 172.21.15.16:0/2811781104 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2fc15e2a0 0x7fd2fc06d420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009870 tx=0x7fd2ec02a270 comp rx=0 tx=0).stop 2024-09-19T07:54:27.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.043+0000 7fd3029fd640 1 -- 172.21.15.16:0/2811781104 shutdown_connections 2024-09-19T07:54:27.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.043+0000 7fd3029fd640 1 --2- 172.21.15.16:0/2811781104 >> v2:172.21.15.146:3300/0 conn(0x7fd2fc15e680 0x7fd2fc0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.043+0000 7fd3029fd640 1 --2- 172.21.15.16:0/2811781104 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2fc15e2a0 0x7fd2fc06d420 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.043+0000 7fd3029fd640 1 -- 172.21.15.16:0/2811781104 >> 172.21.15.16:0/2811781104 conn(0x7fd2fc09d760 msgr2=0x7fd2fc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:27.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.043+0000 7fd3029fd640 1 -- 172.21.15.16:0/2811781104 shutdown_connections 2024-09-19T07:54:27.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.043+0000 7fd3029fd640 1 -- 172.21.15.16:0/2811781104 wait complete. 2024-09-19T07:54:27.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.044+0000 7fd3029fd640 1 Processor -- start 2024-09-19T07:54:27.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.045+0000 7fd3029fd640 1 -- start start 2024-09-19T07:54:27.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.045+0000 7fd3029fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2fc06d960 0x7fd2fc0b8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.045+0000 7fd3029fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2fc15e2a0 0x7fd2fc14c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.045+0000 7fd3029fd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2fc15e680 0x7fd2fc0c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.045+0000 7fd3029fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc0d4c00 con 0x7fd2fc06d960 2024-09-19T07:54:27.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.045+0000 7fd3029fd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc0d4a80 con 0x7fd2fc15e680 2024-09-19T07:54:27.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.045+0000 7fd3029fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd2fc0d0010 con 0x7fd2fc15e2a0 2024-09-19T07:54:27.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.045+0000 7fd2fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2fc06d960 0x7fd2fc0b8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.045+0000 7fd2fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2fc06d960 0x7fd2fc0b8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48440/0 (socket says 172.21.15.16:48440) 2024-09-19T07:54:27.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.046+0000 7fd2fbfff640 1 -- 172.21.15.16:0/3116754101 learned_addr learned my addr 172.21.15.16:0/3116754101 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:27.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.046+0000 7fd300f9a640 1 --2- 172.21.15.16:0/3116754101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2fc15e680 0x7fd2fc0c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.046+0000 7fd2fb7fe640 1 --2- 172.21.15.16:0/3116754101 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2fc15e2a0 0x7fd2fc14c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.046+0000 7fd2fbfff640 1 -- 172.21.15.16:0/3116754101 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2fc15e2a0 msgr2=0x7fd2fc14c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:27.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.046+0000 7fd2fbfff640 1 --2- 172.21.15.16:0/3116754101 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2fc15e2a0 0x7fd2fc14c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.046+0000 7fd2fbfff640 1 -- 172.21.15.16:0/3116754101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2fc15e680 msgr2=0x7fd2fc0c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:27.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.046+0000 7fd2fbfff640 1 --2- 172.21.15.16:0/3116754101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2fc15e680 0x7fd2fc0c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.046+0000 7fd2fbfff640 1 -- 172.21.15.16:0/3116754101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc0b8f80 con 0x7fd2fc06d960 2024-09-19T07:54:27.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.046+0000 7fd2fbfff640 1 --2- 172.21.15.16:0/3116754101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2fc06d960 0x7fd2fc0b8920 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec000c00 tx=0x7fd2ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:27.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.046+0000 7fd2f8ff9640 1 -- 172.21.15.16:0/3116754101 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec037040 con 0x7fd2fc06d960 2024-09-19T07:54:27.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.046+0000 7fd2f8ff9640 1 -- 172.21.15.16:0/3116754101 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ec004230 con 0x7fd2fc06d960 2024-09-19T07:54:27.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.046+0000 7fd2f8ff9640 1 -- 172.21.15.16:0/3116754101 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec003610 con 0x7fd2fc06d960 2024-09-19T07:54:27.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.046+0000 7fd3029fd640 1 -- 172.21.15.16:0/3116754101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2fc0b9260 con 0x7fd2fc06d960 2024-09-19T07:54:27.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.046+0000 7fd3029fd640 1 -- 172.21.15.16:0/3116754101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2fc0b9820 con 0x7fd2fc06d960 2024-09-19T07:54:27.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.047+0000 7fd3029fd640 1 -- 172.21.15.16:0/3116754101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2fc06d720 con 0x7fd2fc06d960 2024-09-19T07:54:27.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.048+0000 7fd2f8ff9640 1 -- 172.21.15.16:0/3116754101 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd2ec03a070 con 0x7fd2fc06d960 2024-09-19T07:54:27.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.048+0000 7fd2f8ff9640 1 --2- 172.21.15.16:0/3116754101 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd2e806c2b0 0x7fd2e806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.049+0000 7fd2f8ff9640 1 -- 172.21.15.16:0/3116754101 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7fd2ec0abce0 con 0x7fd2fc06d960 2024-09-19T07:54:27.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.049+0000 7fd2fb7fe640 1 --2- 172.21.15.16:0/3116754101 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd2e806c2b0 0x7fd2e806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.049+0000 7fd2fb7fe640 1 --2- 172.21.15.16:0/3116754101 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd2e806c2b0 0x7fd2e806e770 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0b5950 tx=0x7fd2f0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:27.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.053+0000 7fd2f8ff9640 1 -- 172.21.15.16:0/3116754101 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd2fc06d720 con 0x7fd2fc06d960 2024-09-19T07:54:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.223+0000 7fd3029fd640 1 -- 172.21.15.16:0/3116754101 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd2fc06cf90 con 0x7fd2e806c2b0 2024-09-19T07:54:27.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.230+0000 7fd2f8ff9640 1 -- 172.21.15.16:0/3116754101 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250945 (secure 0 0 0) 0x7fd2fc06cf90 con 0x7fd2e806c2b0 2024-09-19T07:54:27.234 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:27.235 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:54:27.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.233+0000 7fd3029fd640 1 -- 172.21.15.16:0/3116754101 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd2e806c2b0 msgr2=0x7fd2e806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:27.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.233+0000 7fd3029fd640 1 --2- 172.21.15.16:0/3116754101 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd2e806c2b0 0x7fd2e806e770 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0b5950 tx=0x7fd2f0009190 comp rx=0 tx=0).stop 2024-09-19T07:54:27.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.233+0000 7fd3029fd640 1 -- 172.21.15.16:0/3116754101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2fc06d960 msgr2=0x7fd2fc0b8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:27.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.233+0000 7fd3029fd640 1 --2- 172.21.15.16:0/3116754101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2fc06d960 0x7fd2fc0b8920 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec000c00 tx=0x7fd2ec0046c0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.233+0000 7fd3029fd640 1 -- 172.21.15.16:0/3116754101 shutdown_connections 2024-09-19T07:54:27.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.233+0000 7fd3029fd640 1 --2- 172.21.15.16:0/3116754101 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd2e806c2b0 0x7fd2e806e770 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.233+0000 7fd3029fd640 1 --2- 172.21.15.16:0/3116754101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2fc15e680 0x7fd2fc0c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.233+0000 7fd3029fd640 1 --2- 172.21.15.16:0/3116754101 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2fc15e2a0 0x7fd2fc14c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.233+0000 7fd3029fd640 1 --2- 172.21.15.16:0/3116754101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2fc06d960 0x7fd2fc0b8920 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.233+0000 7fd3029fd640 1 -- 172.21.15.16:0/3116754101 >> 172.21.15.16:0/3116754101 conn(0x7fd2fc09d760 msgr2=0x7fd2fc15ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:27.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.233+0000 7fd3029fd640 1 -- 172.21.15.16:0/3116754101 shutdown_connections 2024-09-19T07:54:27.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.233+0000 7fd3029fd640 1 -- 172.21.15.16:0/3116754101 wait complete. 2024-09-19T07:54:27.245 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2024-09-19T07:54:26.619972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":190,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":763976,"kb_used_data":13640,"kb_used_omap":12,"kb_used_meta":750131,"kb_avail":754210744,"statfs":{"total":773094113280,"available":772311801856,"internally_reserved":0,"allocated":13967360,"data_stored":7088010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":768134704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.078490"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572572+0000","last_change":"2024-09-19T07:54:23.583787+0000","last_active":"2024-09-19T07:54:24.572572+0000","last_peered":"2024-09-19T07:54:24.572572+0000","last_clean":"2024-09-19T07:54:24.572572+0000","last_became_active":"2024-09-19T07:54:23.583453+0000","last_became_peered":"2024-09-19T07:54:23.583453+0000","last_unstale":"2024-09-19T07:54:24.572572+0000","last_undegraded":"2024-09-19T07:54:24.572572+0000","last_fullsized":"2024-09-19T07:54:24.572572+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:23.821546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579942+0000","last_change":"2024-09-19T07:54:23.576333+0000","last_active":"2024-09-19T07:54:23.579942+0000","last_peered":"2024-09-19T07:54:23.579942+0000","last_clean":"2024-09-19T07:54:23.579942+0000","last_became_active":"2024-09-19T07:54:23.576053+0000","last_became_peered":"2024-09-19T07:54:23.576053+0000","last_unstale":"2024-09-19T07:54:23.579942+0000","last_undegraded":"2024-09-19T07:54:23.579942+0000","last_fullsized":"2024-09-19T07:54:23.579942+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:39.147749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.591937+0000","last_change":"2024-09-19T07:54:23.589816+0000","last_active":"2024-09-19T07:54:23.591937+0000","last_peered":"2024-09-19T07:54:23.591937+0000","last_clean":"2024-09-19T07:54:23.591937+0000","last_became_active":"2024-09-19T07:54:23.589423+0000","last_became_peered":"2024-09-19T07:54:23.589423+0000","last_unstale":"2024-09-19T07:54:23.591937+0000","last_undegraded":"2024-09-19T07:54:23.591937+0000","last_fullsized":"2024-09-19T07:54:23.591937+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:31.699850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.582857+0000","last_change":"2024-09-19T07:54:23.579793+0000","last_active":"2024-09-19T07:54:23.582857+0000","last_peered":"2024-09-19T07:54:23.582857+0000","last_clean":"2024-09-19T07:54:23.582857+0000","last_became_active":"2024-09-19T07:54:23.579502+0000","last_became_peered":"2024-09-19T07:54:23.579502+0000","last_unstale":"2024-09-19T07:54:23.582857+0000","last_undegraded":"2024-09-19T07:54:23.582857+0000","last_fullsized":"2024-09-19T07:54:23.582857+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:32.477007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.551618+0000","last_change":"2024-09-19T07:54:22.551618+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.551618+0000","last_undegraded":"2024-09-19T07:54:22.551618+0000","last_fullsized":"2024-09-19T07:54:22.551618+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.555440+0000","last_change":"2024-09-19T07:54:22.555440+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.555440+0000","last_undegraded":"2024-09-19T07:54:22.555440+0000","last_fullsized":"2024-09-19T07:54:22.555440+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.586469+0000","last_change":"2024-09-19T07:54:23.584389+0000","last_active":"2024-09-19T07:54:23.586469+0000","last_peered":"2024-09-19T07:54:23.586469+0000","last_clean":"2024-09-19T07:54:23.586469+0000","last_became_active":"2024-09-19T07:54:23.584121+0000","last_became_peered":"2024-09-19T07:54:23.584121+0000","last_unstale":"2024-09-19T07:54:23.586469+0000","last_undegraded":"2024-09-19T07:54:23.586469+0000","last_fullsized":"2024-09-19T07:54:23.586469+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:35.419442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585626+0000","last_change":"2024-09-19T07:54:23.584963+0000","last_active":"2024-09-19T07:54:23.585626+0000","last_peered":"2024-09-19T07:54:23.585626+0000","last_clean":"2024-09-19T07:54:23.585626+0000","last_became_active":"2024-09-19T07:54:23.583723+0000","last_became_peered":"2024-09-19T07:54:23.583723+0000","last_unstale":"2024-09-19T07:54:23.585626+0000","last_undegraded":"2024-09-19T07:54:23.585626+0000","last_fullsized":"2024-09-19T07:54:23.585626+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:57.397944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.554930+0000","last_change":"2024-09-19T07:54:22.554930+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.554930+0000","last_undegraded":"2024-09-19T07:54:22.554930+0000","last_fullsized":"2024-09-19T07:54:22.554930+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585515+0000","last_change":"2024-09-19T07:54:23.579169+0000","last_active":"2024-09-19T07:54:23.585515+0000","last_peered":"2024-09-19T07:54:23.585515+0000","last_clean":"2024-09-19T07:54:23.585515+0000","last_became_active":"2024-09-19T07:54:23.578888+0000","last_became_peered":"2024-09-19T07:54:23.578888+0000","last_unstale":"2024-09-19T07:54:23.585515+0000","last_undegraded":"2024-09-19T07:54:23.585515+0000","last_fullsized":"2024-09-19T07:54:23.585515+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:09.226711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585479+0000","last_change":"2024-09-19T07:54:23.584332+0000","last_active":"2024-09-19T07:54:23.585479+0000","last_peered":"2024-09-19T07:54:23.585479+0000","last_clean":"2024-09-19T07:54:23.585479+0000","last_became_active":"2024-09-19T07:54:23.584031+0000","last_became_peered":"2024-09-19T07:54:23.584031+0000","last_unstale":"2024-09-19T07:54:23.585479+0000","last_undegraded":"2024-09-19T07:54:23.585479+0000","last_fullsized":"2024-09-19T07:54:23.585479+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:14.753001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572521+0000","last_change":"2024-09-19T07:54:23.574847+0000","last_active":"2024-09-19T07:54:24.572521+0000","last_peered":"2024-09-19T07:54:24.572521+0000","last_clean":"2024-09-19T07:54:24.572521+0000","last_became_active":"2024-09-19T07:54:23.574423+0000","last_became_peered":"2024-09-19T07:54:23.574423+0000","last_unstale":"2024-09-19T07:54:24.572521+0000","last_undegraded":"2024-09-19T07:54:24.572521+0000","last_fullsized":"2024-09-19T07:54:24.572521+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:35:20.465621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.554415+0000","last_change":"2024-09-19T07:54:22.554415+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.554415+0000","last_undegraded":"2024-09-19T07:54:22.554415+0000","last_fullsized":"2024-09-19T07:54:22.554415+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572583+0000","last_change":"2024-09-19T07:54:23.583768+0000","last_active":"2024-09-19T07:54:24.572583+0000","last_peered":"2024-09-19T07:54:24.572583+0000","last_clean":"2024-09-19T07:54:24.572583+0000","last_became_active":"2024-09-19T07:54:23.579038+0000","last_became_peered":"2024-09-19T07:54:23.579038+0000","last_unstale":"2024-09-19T07:54:24.572583+0000","last_undegraded":"2024-09-19T07:54:24.572583+0000","last_fullsized":"2024-09-19T07:54:24.572583+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:16:28.056548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572220+0000","last_change":"2024-09-19T07:54:23.582464+0000","last_active":"2024-09-19T07:54:24.572220+0000","last_peered":"2024-09-19T07:54:24.572220+0000","last_clean":"2024-09-19T07:54:24.572220+0000","last_became_active":"2024-09-19T07:54:23.582176+0000","last_became_peered":"2024-09-19T07:54:23.582176+0000","last_unstale":"2024-09-19T07:54:24.572220+0000","last_undegraded":"2024-09-19T07:54:24.572220+0000","last_fullsized":"2024-09-19T07:54:24.572220+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:25.500256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.557847+0000","last_change":"2024-09-19T07:54:22.557847+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.557847+0000","last_undegraded":"2024-09-19T07:54:22.557847+0000","last_fullsized":"2024-09-19T07:54:22.557847+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.553868+0000","last_change":"2024-09-19T07:54:22.553868+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.553868+0000","last_undegraded":"2024-09-19T07:54:22.553868+0000","last_fullsized":"2024-09-19T07:54:22.553868+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.555444+0000","last_change":"2024-09-19T07:54:22.555444+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.555444+0000","last_undegraded":"2024-09-19T07:54:22.555444+0000","last_fullsized":"2024-09-19T07:54:22.555444+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.581149+0000","last_change":"2024-09-19T07:54:23.580157+0000","last_active":"2024-09-19T07:54:23.581149+0000","last_peered":"2024-09-19T07:54:23.581149+0000","last_clean":"2024-09-19T07:54:23.581149+0000","last_became_active":"2024-09-19T07:54:23.579109+0000","last_became_peered":"2024-09-19T07:54:23.579109+0000","last_unstale":"2024-09-19T07:54:23.581149+0000","last_undegraded":"2024-09-19T07:54:23.581149+0000","last_fullsized":"2024-09-19T07:54:23.581149+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:17.886276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.552956+0000","last_change":"2024-09-19T07:54:22.552956+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.552956+0000","last_undegraded":"2024-09-19T07:54:22.552956+0000","last_fullsized":"2024-09-19T07:54:22.552956+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.568515+0000","last_change":"2024-09-19T07:54:23.567173+0000","last_active":"2024-09-19T07:54:23.568515+0000","last_peered":"2024-09-19T07:54:23.568515+0000","last_clean":"2024-09-19T07:54:23.568515+0000","last_became_active":"2024-09-19T07:54:23.566866+0000","last_became_peered":"2024-09-19T07:54:23.566866+0000","last_unstale":"2024-09-19T07:54:23.568515+0000","last_undegraded":"2024-09-19T07:54:23.568515+0000","last_fullsized":"2024-09-19T07:54:23.568515+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:14.329860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572411+0000","last_change":"2024-09-19T07:54:23.586941+0000","last_active":"2024-09-19T07:54:24.572411+0000","last_peered":"2024-09-19T07:54:24.572411+0000","last_clean":"2024-09-19T07:54:24.572411+0000","last_became_active":"2024-09-19T07:54:23.586367+0000","last_became_peered":"2024-09-19T07:54:23.586367+0000","last_unstale":"2024-09-19T07:54:24.572411+0000","last_undegraded":"2024-09-19T07:54:24.572411+0000","last_fullsized":"2024-09-19T07:54:24.572411+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:22.782628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579831+0000","last_change":"2024-09-19T07:54:23.575696+0000","last_active":"2024-09-19T07:54:23.579831+0000","last_peered":"2024-09-19T07:54:23.579831+0000","last_clean":"2024-09-19T07:54:23.579831+0000","last_became_active":"2024-09-19T07:54:23.575427+0000","last_became_peered":"2024-09-19T07:54:23.575427+0000","last_unstale":"2024-09-19T07:54:23.579831+0000","last_undegraded":"2024-09-19T07:54:23.579831+0000","last_fullsized":"2024-09-19T07:54:23.579831+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:29.663123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.557404+0000","last_change":"2024-09-19T07:54:22.557404+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.557404+0000","last_undegraded":"2024-09-19T07:54:22.557404+0000","last_fullsized":"2024-09-19T07:54:22.557404+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.589396+0000","last_change":"2024-09-19T07:54:23.584866+0000","last_active":"2024-09-19T07:54:23.589396+0000","last_peered":"2024-09-19T07:54:23.589396+0000","last_clean":"2024-09-19T07:54:23.589396+0000","last_became_active":"2024-09-19T07:54:23.582585+0000","last_became_peered":"2024-09-19T07:54:23.582585+0000","last_unstale":"2024-09-19T07:54:23.589396+0000","last_undegraded":"2024-09-19T07:54:23.589396+0000","last_fullsized":"2024-09-19T07:54:23.589396+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:09.647595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.587463+0000","last_change":"2024-09-19T07:54:23.583732+0000","last_active":"2024-09-19T07:54:23.587463+0000","last_peered":"2024-09-19T07:54:23.587463+0000","last_clean":"2024-09-19T07:54:23.587463+0000","last_became_active":"2024-09-19T07:54:23.583460+0000","last_became_peered":"2024-09-19T07:54:23.583460+0000","last_unstale":"2024-09-19T07:54:23.587463+0000","last_undegraded":"2024-09-19T07:54:23.587463+0000","last_fullsized":"2024-09-19T07:54:23.587463+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:01.094793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.581128+0000","last_change":"2024-09-19T07:54:23.579141+0000","last_active":"2024-09-19T07:54:23.581128+0000","last_peered":"2024-09-19T07:54:23.581128+0000","last_clean":"2024-09-19T07:54:23.581128+0000","last_became_active":"2024-09-19T07:54:23.578856+0000","last_became_peered":"2024-09-19T07:54:23.578856+0000","last_unstale":"2024-09-19T07:54:23.581128+0000","last_undegraded":"2024-09-19T07:54:23.581128+0000","last_fullsized":"2024-09-19T07:54:23.581128+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:21.371456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585833+0000","last_change":"2024-09-19T07:54:23.582767+0000","last_active":"2024-09-19T07:54:23.585833+0000","last_peered":"2024-09-19T07:54:23.585833+0000","last_clean":"2024-09-19T07:54:23.585833+0000","last_became_active":"2024-09-19T07:54:23.582486+0000","last_became_peered":"2024-09-19T07:54:23.582486+0000","last_unstale":"2024-09-19T07:54:23.585833+0000","last_undegraded":"2024-09-19T07:54:23.585833+0000","last_fullsized":"2024-09-19T07:54:23.585833+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:32.949629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.589319+0000","last_change":"2024-09-19T07:54:23.587202+0000","last_active":"2024-09-19T07:54:23.589319+0000","last_peered":"2024-09-19T07:54:23.589319+0000","last_clean":"2024-09-19T07:54:23.589319+0000","last_became_active":"2024-09-19T07:54:23.586940+0000","last_became_peered":"2024-09-19T07:54:23.586940+0000","last_unstale":"2024-09-19T07:54:23.589319+0000","last_undegraded":"2024-09-19T07:54:23.589319+0000","last_fullsized":"2024-09-19T07:54:23.589319+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:41.297635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.583319+0000","last_change":"2024-09-19T07:54:23.581592+0000","last_active":"2024-09-19T07:54:23.583319+0000","last_peered":"2024-09-19T07:54:23.583319+0000","last_clean":"2024-09-19T07:54:23.583319+0000","last_became_active":"2024-09-19T07:54:23.581298+0000","last_became_peered":"2024-09-19T07:54:23.581298+0000","last_unstale":"2024-09-19T07:54:23.583319+0000","last_undegraded":"2024-09-19T07:54:23.583319+0000","last_fullsized":"2024-09-19T07:54:23.583319+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:04.317330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576623+0000","last_change":"2024-09-19T07:54:23.575015+0000","last_active":"2024-09-19T07:54:23.576623+0000","last_peered":"2024-09-19T07:54:23.576623+0000","last_clean":"2024-09-19T07:54:23.576623+0000","last_became_active":"2024-09-19T07:54:23.574695+0000","last_became_peered":"2024-09-19T07:54:23.574695+0000","last_unstale":"2024-09-19T07:54:23.576623+0000","last_undegraded":"2024-09-19T07:54:23.576623+0000","last_fullsized":"2024-09-19T07:54:23.576623+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:07.673263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585765+0000","last_change":"2024-09-19T07:54:23.583307+0000","last_active":"2024-09-19T07:54:23.585765+0000","last_peered":"2024-09-19T07:54:23.585765+0000","last_clean":"2024-09-19T07:54:23.585765+0000","last_became_active":"2024-09-19T07:54:23.583021+0000","last_became_peered":"2024-09-19T07:54:23.583021+0000","last_unstale":"2024-09-19T07:54:23.585765+0000","last_undegraded":"2024-09-19T07:54:23.585765+0000","last_fullsized":"2024-09-19T07:54:23.585765+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:38.791089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579025+0000","last_change":"2024-09-19T07:54:23.576203+0000","last_active":"2024-09-19T07:54:23.579025+0000","last_peered":"2024-09-19T07:54:23.579025+0000","last_clean":"2024-09-19T07:54:23.579025+0000","last_became_active":"2024-09-19T07:54:23.575926+0000","last_became_peered":"2024-09-19T07:54:23.575926+0000","last_unstale":"2024-09-19T07:54:23.579025+0000","last_undegraded":"2024-09-19T07:54:23.579025+0000","last_fullsized":"2024-09-19T07:54:23.579025+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:56.777102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"128'4","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.591733+0000","last_change":"2024-09-19T07:54:22.568159+0000","last_active":"2024-09-19T07:54:23.591733+0000","last_peered":"2024-09-19T07:54:23.591733+0000","last_clean":"2024-09-19T07:54:23.591733+0000","last_became_active":"2024-09-19T07:54:22.567832+0000","last_became_peered":"2024-09-19T07:54:22.567832+0000","last_unstale":"2024-09-19T07:54:23.591733+0000","last_undegraded":"2024-09-19T07:54:23.591733+0000","last_fullsized":"2024-09-19T07:54:23.591733+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:46.972795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572726+0000","last_change":"2024-09-19T07:54:23.585429+0000","last_active":"2024-09-19T07:54:24.572726+0000","last_peered":"2024-09-19T07:54:24.572726+0000","last_clean":"2024-09-19T07:54:24.572726+0000","last_became_active":"2024-09-19T07:54:23.585096+0000","last_became_peered":"2024-09-19T07:54:23.585096+0000","last_unstale":"2024-09-19T07:54:24.572726+0000","last_undegraded":"2024-09-19T07:54:24.572726+0000","last_fullsized":"2024-09-19T07:54:24.572726+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:29.192911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565571+0000","last_change":"2024-09-19T07:54:22.561503+0000","last_active":"2024-09-19T07:54:22.565571+0000","last_peered":"2024-09-19T07:54:22.565571+0000","last_clean":"2024-09-19T07:54:22.565571+0000","last_became_active":"2024-09-19T07:54:22.561219+0000","last_became_peered":"2024-09-19T07:54:22.561219+0000","last_unstale":"2024-09-19T07:54:22.565571+0000","last_undegraded":"2024-09-19T07:54:22.565571+0000","last_fullsized":"2024-09-19T07:54:22.565571+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:18.897568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579803+0000","last_change":"2024-09-19T07:54:23.574954+0000","last_active":"2024-09-19T07:54:23.579803+0000","last_peered":"2024-09-19T07:54:23.579803+0000","last_clean":"2024-09-19T07:54:23.579803+0000","last_became_active":"2024-09-19T07:54:23.574624+0000","last_became_peered":"2024-09-19T07:54:23.574624+0000","last_unstale":"2024-09-19T07:54:23.579803+0000","last_undegraded":"2024-09-19T07:54:23.579803+0000","last_fullsized":"2024-09-19T07:54:23.579803+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:32.420071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"128'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.592293+0000","last_change":"2024-09-19T07:54:22.566018+0000","last_active":"2024-09-19T07:54:23.592293+0000","last_peered":"2024-09-19T07:54:23.592293+0000","last_clean":"2024-09-19T07:54:23.592293+0000","last_became_active":"2024-09-19T07:54:22.565812+0000","last_became_peered":"2024-09-19T07:54:22.565812+0000","last_unstale":"2024-09-19T07:54:23.592293+0000","last_undegraded":"2024-09-19T07:54:23.592293+0000","last_fullsized":"2024-09-19T07:54:23.592293+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:01.728173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572245+0000","last_change":"2024-09-19T07:54:23.582120+0000","last_active":"2024-09-19T07:54:24.572245+0000","last_peered":"2024-09-19T07:54:24.572245+0000","last_clean":"2024-09-19T07:54:24.572245+0000","last_became_active":"2024-09-19T07:54:23.581830+0000","last_became_peered":"2024-09-19T07:54:23.581830+0000","last_unstale":"2024-09-19T07:54:24.572245+0000","last_undegraded":"2024-09-19T07:54:24.572245+0000","last_fullsized":"2024-09-19T07:54:24.572245+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:12.630683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"128'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.591308+0000","last_change":"2024-09-19T07:54:22.560370+0000","last_active":"2024-09-19T07:54:23.591308+0000","last_peered":"2024-09-19T07:54:23.591308+0000","last_clean":"2024-09-19T07:54:23.591308+0000","last_became_active":"2024-09-19T07:54:22.560083+0000","last_became_peered":"2024-09-19T07:54:22.560083+0000","last_unstale":"2024-09-19T07:54:23.591308+0000","last_undegraded":"2024-09-19T07:54:23.591308+0000","last_fullsized":"2024-09-19T07:54:23.591308+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:51.100115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579407+0000","last_change":"2024-09-19T07:54:23.577913+0000","last_active":"2024-09-19T07:54:23.579407+0000","last_peered":"2024-09-19T07:54:23.579407+0000","last_clean":"2024-09-19T07:54:23.579407+0000","last_became_active":"2024-09-19T07:54:23.577102+0000","last_became_peered":"2024-09-19T07:54:23.577102+0000","last_unstale":"2024-09-19T07:54:23.579407+0000","last_undegraded":"2024-09-19T07:54:23.579407+0000","last_fullsized":"2024-09-19T07:54:23.579407+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:49.188816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"128'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.586957+0000","last_change":"2024-09-19T07:54:22.554611+0000","last_active":"2024-09-19T07:54:23.586957+0000","last_peered":"2024-09-19T07:54:23.586957+0000","last_clean":"2024-09-19T07:54:23.586957+0000","last_became_active":"2024-09-19T07:54:22.554332+0000","last_became_peered":"2024-09-19T07:54:22.554332+0000","last_unstale":"2024-09-19T07:54:23.586957+0000","last_undegraded":"2024-09-19T07:54:23.586957+0000","last_fullsized":"2024-09-19T07:54:23.586957+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:09.508289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576791+0000","last_change":"2024-09-19T07:54:23.574521+0000","last_active":"2024-09-19T07:54:23.576791+0000","last_peered":"2024-09-19T07:54:23.576791+0000","last_clean":"2024-09-19T07:54:23.576791+0000","last_became_active":"2024-09-19T07:54:23.574251+0000","last_became_peered":"2024-09-19T07:54:23.574251+0000","last_unstale":"2024-09-19T07:54:23.576791+0000","last_undegraded":"2024-09-19T07:54:23.576791+0000","last_fullsized":"2024-09-19T07:54:23.576791+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:31.699850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565667+0000","last_change":"2024-09-19T07:54:22.564044+0000","last_active":"2024-09-19T07:54:22.565667+0000","last_peered":"2024-09-19T07:54:22.565667+0000","last_clean":"2024-09-19T07:54:22.565667+0000","last_became_active":"2024-09-19T07:54:22.562851+0000","last_became_peered":"2024-09-19T07:54:22.562851+0000","last_unstale":"2024-09-19T07:54:22.565667+0000","last_undegraded":"2024-09-19T07:54:22.565667+0000","last_fullsized":"2024-09-19T07:54:22.565667+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:34.414794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.555018+0000","last_change":"2024-09-19T07:54:22.555018+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.555018+0000","last_undegraded":"2024-09-19T07:54:22.555018+0000","last_fullsized":"2024-09-19T07:54:22.555018+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.566646+0000","last_change":"2024-09-19T07:54:22.557995+0000","last_active":"2024-09-19T07:54:22.566646+0000","last_peered":"2024-09-19T07:54:22.566646+0000","last_clean":"2024-09-19T07:54:22.566646+0000","last_became_active":"2024-09-19T07:54:22.557692+0000","last_became_peered":"2024-09-19T07:54:22.557692+0000","last_unstale":"2024-09-19T07:54:22.566646+0000","last_undegraded":"2024-09-19T07:54:22.566646+0000","last_fullsized":"2024-09-19T07:54:22.566646+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:01.620005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576699+0000","last_change":"2024-09-19T07:54:23.573943+0000","last_active":"2024-09-19T07:54:23.576699+0000","last_peered":"2024-09-19T07:54:23.576699+0000","last_clean":"2024-09-19T07:54:23.576699+0000","last_became_active":"2024-09-19T07:54:23.573668+0000","last_became_peered":"2024-09-19T07:54:23.573668+0000","last_unstale":"2024-09-19T07:54:23.576699+0000","last_undegraded":"2024-09-19T07:54:23.576699+0000","last_fullsized":"2024-09-19T07:54:23.576699+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:13:01.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.566981+0000","last_change":"2024-09-19T07:54:22.562903+0000","last_active":"2024-09-19T07:54:23.566981+0000","last_peered":"2024-09-19T07:54:23.566981+0000","last_clean":"2024-09-19T07:54:23.566981+0000","last_became_active":"2024-09-19T07:54:22.562599+0000","last_became_peered":"2024-09-19T07:54:22.562599+0000","last_unstale":"2024-09-19T07:54:23.566981+0000","last_undegraded":"2024-09-19T07:54:23.566981+0000","last_fullsized":"2024-09-19T07:54:23.566981+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:44.078560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572071+0000","last_change":"2024-09-19T07:54:23.586564+0000","last_active":"2024-09-19T07:54:24.572071+0000","last_peered":"2024-09-19T07:54:24.572071+0000","last_clean":"2024-09-19T07:54:24.572071+0000","last_became_active":"2024-09-19T07:54:23.586246+0000","last_became_peered":"2024-09-19T07:54:23.586246+0000","last_unstale":"2024-09-19T07:54:24.572071+0000","last_undegraded":"2024-09-19T07:54:24.572071+0000","last_fullsized":"2024-09-19T07:54:24.572071+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:14:59.560377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.563273+0000","last_change":"2024-09-19T07:54:22.556612+0000","last_active":"2024-09-19T07:54:23.563273+0000","last_peered":"2024-09-19T07:54:23.563273+0000","last_clean":"2024-09-19T07:54:23.563273+0000","last_became_active":"2024-09-19T07:54:22.556177+0000","last_became_peered":"2024-09-19T07:54:22.556177+0000","last_unstale":"2024-09-19T07:54:23.563273+0000","last_undegraded":"2024-09-19T07:54:23.563273+0000","last_fullsized":"2024-09-19T07:54:23.563273+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:25.294939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.556350+0000","last_change":"2024-09-19T07:54:22.556350+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.556350+0000","last_undegraded":"2024-09-19T07:54:22.556350+0000","last_fullsized":"2024-09-19T07:54:22.556350+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572966+0000","last_change":"2024-09-19T07:54:22.564375+0000","last_active":"2024-09-19T07:54:24.572966+0000","last_peered":"2024-09-19T07:54:24.572966+0000","last_clean":"2024-09-19T07:54:24.572966+0000","last_became_active":"2024-09-19T07:54:22.564096+0000","last_became_peered":"2024-09-19T07:54:22.564096+0000","last_unstale":"2024-09-19T07:54:24.572966+0000","last_undegraded":"2024-09-19T07:54:24.572966+0000","last_fullsized":"2024-09-19T07:54:24.572966+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:05.452375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.584721+0000","last_change":"2024-09-19T07:54:23.583678+0000","last_active":"2024-09-19T07:54:23.584721+0000","last_peered":"2024-09-19T07:54:23.584721+0000","last_clean":"2024-09-19T07:54:23.584721+0000","last_became_active":"2024-09-19T07:54:23.583254+0000","last_became_peered":"2024-09-19T07:54:23.583254+0000","last_unstale":"2024-09-19T07:54:23.584721+0000","last_undegraded":"2024-09-19T07:54:23.584721+0000","last_fullsized":"2024-09-19T07:54:23.584721+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:46.495632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"128'3","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.593450+0000","last_change":"2024-09-19T07:54:22.558733+0000","last_active":"2024-09-19T07:54:23.593450+0000","last_peered":"2024-09-19T07:54:23.593450+0000","last_clean":"2024-09-19T07:54:23.593450+0000","last_became_active":"2024-09-19T07:54:22.558481+0000","last_became_peered":"2024-09-19T07:54:22.558481+0000","last_unstale":"2024-09-19T07:54:23.593450+0000","last_undegraded":"2024-09-19T07:54:23.593450+0000","last_fullsized":"2024-09-19T07:54:23.593450+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:41.801926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.590832+0000","last_change":"2024-09-19T07:54:23.589083+0000","last_active":"2024-09-19T07:54:23.590832+0000","last_peered":"2024-09-19T07:54:23.590832+0000","last_clean":"2024-09-19T07:54:23.590832+0000","last_became_active":"2024-09-19T07:54:23.588728+0000","last_became_peered":"2024-09-19T07:54:23.588728+0000","last_unstale":"2024-09-19T07:54:23.590832+0000","last_undegraded":"2024-09-19T07:54:23.590832+0000","last_fullsized":"2024-09-19T07:54:23.590832+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:44.660536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"128'6","reported_seq":18,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.594644+0000","last_change":"2024-09-19T07:54:22.564291+0000","last_active":"2024-09-19T07:54:23.594644+0000","last_peered":"2024-09-19T07:54:23.594644+0000","last_clean":"2024-09-19T07:54:23.594644+0000","last_became_active":"2024-09-19T07:54:22.564012+0000","last_became_peered":"2024-09-19T07:54:22.564012+0000","last_unstale":"2024-09-19T07:54:23.594644+0000","last_undegraded":"2024-09-19T07:54:23.594644+0000","last_fullsized":"2024-09-19T07:54:23.594644+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:00.025001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.582649+0000","last_change":"2024-09-19T07:54:23.578794+0000","last_active":"2024-09-19T07:54:23.582649+0000","last_peered":"2024-09-19T07:54:23.582649+0000","last_clean":"2024-09-19T07:54:23.582649+0000","last_became_active":"2024-09-19T07:54:23.578465+0000","last_became_peered":"2024-09-19T07:54:23.578465+0000","last_unstale":"2024-09-19T07:54:23.582649+0000","last_undegraded":"2024-09-19T07:54:23.582649+0000","last_fullsized":"2024-09-19T07:54:23.582649+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:57.295016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"128'2","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572360+0000","last_change":"2024-09-19T07:54:22.565043+0000","last_active":"2024-09-19T07:54:24.572360+0000","last_peered":"2024-09-19T07:54:24.572360+0000","last_clean":"2024-09-19T07:54:24.572360+0000","last_became_active":"2024-09-19T07:54:22.564758+0000","last_became_peered":"2024-09-19T07:54:22.564758+0000","last_unstale":"2024-09-19T07:54:24.572360+0000","last_undegraded":"2024-09-19T07:54:24.572360+0000","last_fullsized":"2024-09-19T07:54:24.572360+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:58:28.017221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576888+0000","last_change":"2024-09-19T07:54:23.574862+0000","last_active":"2024-09-19T07:54:23.576888+0000","last_peered":"2024-09-19T07:54:23.576888+0000","last_clean":"2024-09-19T07:54:23.576888+0000","last_became_active":"2024-09-19T07:54:23.574431+0000","last_became_peered":"2024-09-19T07:54:23.574431+0000","last_unstale":"2024-09-19T07:54:23.576888+0000","last_undegraded":"2024-09-19T07:54:23.576888+0000","last_fullsized":"2024-09-19T07:54:23.576888+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:29.913764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.567482+0000","last_change":"2024-09-19T07:54:22.559040+0000","last_active":"2024-09-19T07:54:23.567482+0000","last_peered":"2024-09-19T07:54:23.567482+0000","last_clean":"2024-09-19T07:54:23.567482+0000","last_became_active":"2024-09-19T07:54:22.558757+0000","last_became_peered":"2024-09-19T07:54:22.558757+0000","last_unstale":"2024-09-19T07:54:23.567482+0000","last_undegraded":"2024-09-19T07:54:23.567482+0000","last_fullsized":"2024-09-19T07:54:23.567482+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:28:24.370408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.551690+0000","last_change":"2024-09-19T07:54:22.551690+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.551690+0000","last_undegraded":"2024-09-19T07:54:22.551690+0000","last_fullsized":"2024-09-19T07:54:22.551690+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565358+0000","last_change":"2024-09-19T07:54:22.559203+0000","last_active":"2024-09-19T07:54:22.565358+0000","last_peered":"2024-09-19T07:54:22.565358+0000","last_clean":"2024-09-19T07:54:22.565358+0000","last_became_active":"2024-09-19T07:54:22.558920+0000","last_became_peered":"2024-09-19T07:54:22.558920+0000","last_unstale":"2024-09-19T07:54:22.565358+0000","last_undegraded":"2024-09-19T07:54:22.565358+0000","last_fullsized":"2024-09-19T07:54:22.565358+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:43.655888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.589459+0000","last_change":"2024-09-19T07:54:23.587980+0000","last_active":"2024-09-19T07:54:23.589459+0000","last_peered":"2024-09-19T07:54:23.589459+0000","last_clean":"2024-09-19T07:54:23.589459+0000","last_became_active":"2024-09-19T07:54:23.587702+0000","last_became_peered":"2024-09-19T07:54:23.587702+0000","last_unstale":"2024-09-19T07:54:23.589459+0000","last_undegraded":"2024-09-19T07:54:23.589459+0000","last_fullsized":"2024-09-19T07:54:23.589459+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:02.624652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"128'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.578744+0000","last_change":"2024-09-19T07:54:22.553965+0000","last_active":"2024-09-19T07:54:23.578744+0000","last_peered":"2024-09-19T07:54:23.578744+0000","last_clean":"2024-09-19T07:54:23.578744+0000","last_became_active":"2024-09-19T07:54:22.553678+0000","last_became_peered":"2024-09-19T07:54:22.553678+0000","last_unstale":"2024-09-19T07:54:23.578744+0000","last_undegraded":"2024-09-19T07:54:23.578744+0000","last_fullsized":"2024-09-19T07:54:23.578744+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:06.364113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576816+0000","last_change":"2024-09-19T07:54:23.574769+0000","last_active":"2024-09-19T07:54:23.576816+0000","last_peered":"2024-09-19T07:54:23.576816+0000","last_clean":"2024-09-19T07:54:23.576816+0000","last_became_active":"2024-09-19T07:54:23.574488+0000","last_became_peered":"2024-09-19T07:54:23.574488+0000","last_unstale":"2024-09-19T07:54:23.576816+0000","last_undegraded":"2024-09-19T07:54:23.576816+0000","last_fullsized":"2024-09-19T07:54:23.576816+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:41.287209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"128'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.591321+0000","last_change":"2024-09-19T07:54:22.567957+0000","last_active":"2024-09-19T07:54:23.591321+0000","last_peered":"2024-09-19T07:54:23.591321+0000","last_clean":"2024-09-19T07:54:23.591321+0000","last_became_active":"2024-09-19T07:54:22.567503+0000","last_became_peered":"2024-09-19T07:54:22.567503+0000","last_unstale":"2024-09-19T07:54:23.591321+0000","last_undegraded":"2024-09-19T07:54:23.591321+0000","last_fullsized":"2024-09-19T07:54:23.591321+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:06.620754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.580616+0000","last_change":"2024-09-19T07:54:23.578211+0000","last_active":"2024-09-19T07:54:23.580616+0000","last_peered":"2024-09-19T07:54:23.580616+0000","last_clean":"2024-09-19T07:54:23.580616+0000","last_became_active":"2024-09-19T07:54:23.577649+0000","last_became_peered":"2024-09-19T07:54:23.577649+0000","last_unstale":"2024-09-19T07:54:23.580616+0000","last_undegraded":"2024-09-19T07:54:23.580616+0000","last_fullsized":"2024-09-19T07:54:23.580616+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:19.902216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565179+0000","last_change":"2024-09-19T07:54:22.559404+0000","last_active":"2024-09-19T07:54:22.565179+0000","last_peered":"2024-09-19T07:54:22.565179+0000","last_clean":"2024-09-19T07:54:22.565179+0000","last_became_active":"2024-09-19T07:54:22.559101+0000","last_became_peered":"2024-09-19T07:54:22.559101+0000","last_unstale":"2024-09-19T07:54:22.565179+0000","last_undegraded":"2024-09-19T07:54:22.565179+0000","last_fullsized":"2024-09-19T07:54:22.565179+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:30.695202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.587444+0000","last_change":"2024-09-19T07:54:23.585316+0000","last_active":"2024-09-19T07:54:23.587444+0000","last_peered":"2024-09-19T07:54:23.587444+0000","last_clean":"2024-09-19T07:54:23.587444+0000","last_became_active":"2024-09-19T07:54:23.585029+0000","last_became_peered":"2024-09-19T07:54:23.585029+0000","last_unstale":"2024-09-19T07:54:23.587444+0000","last_undegraded":"2024-09-19T07:54:23.587444+0000","last_fullsized":"2024-09-19T07:54:23.587444+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:35.538510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565841+0000","last_change":"2024-09-19T07:54:22.563785+0000","last_active":"2024-09-19T07:54:22.565841+0000","last_peered":"2024-09-19T07:54:22.565841+0000","last_clean":"2024-09-19T07:54:22.565841+0000","last_became_active":"2024-09-19T07:54:22.563471+0000","last_became_peered":"2024-09-19T07:54:22.563471+0000","last_unstale":"2024-09-19T07:54:22.565841+0000","last_undegraded":"2024-09-19T07:54:22.565841+0000","last_fullsized":"2024-09-19T07:54:22.565841+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:16.881628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.552495+0000","last_change":"2024-09-19T07:54:22.552495+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.552495+0000","last_undegraded":"2024-09-19T07:54:22.552495+0000","last_fullsized":"2024-09-19T07:54:22.552495+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"128'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.586262+0000","last_change":"2024-09-19T07:54:22.556136+0000","last_active":"2024-09-19T07:54:23.586262+0000","last_peered":"2024-09-19T07:54:23.586262+0000","last_clean":"2024-09-19T07:54:23.586262+0000","last_became_active":"2024-09-19T07:54:22.555829+0000","last_became_peered":"2024-09-19T07:54:22.555829+0000","last_unstale":"2024-09-19T07:54:23.586262+0000","last_undegraded":"2024-09-19T07:54:23.586262+0000","last_fullsized":"2024-09-19T07:54:23.586262+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:37.052983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572425+0000","last_change":"2024-09-19T07:54:23.582629+0000","last_active":"2024-09-19T07:54:24.572425+0000","last_peered":"2024-09-19T07:54:24.572425+0000","last_clean":"2024-09-19T07:54:24.572425+0000","last_became_active":"2024-09-19T07:54:23.582343+0000","last_became_peered":"2024-09-19T07:54:23.582343+0000","last_unstale":"2024-09-19T07:54:24.572425+0000","last_undegraded":"2024-09-19T07:54:24.572425+0000","last_fullsized":"2024-09-19T07:54:24.572425+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:38.135921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572535+0000","last_change":"2024-09-19T07:54:22.563856+0000","last_active":"2024-09-19T07:54:24.572535+0000","last_peered":"2024-09-19T07:54:24.572535+0000","last_clean":"2024-09-19T07:54:24.572535+0000","last_became_active":"2024-09-19T07:54:22.563515+0000","last_became_peered":"2024-09-19T07:54:22.563515+0000","last_unstale":"2024-09-19T07:54:24.572535+0000","last_undegraded":"2024-09-19T07:54:24.572535+0000","last_fullsized":"2024-09-19T07:54:24.572535+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:32.880703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.554597+0000","last_change":"2024-09-19T07:54:22.554597+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.554597+0000","last_undegraded":"2024-09-19T07:54:22.554597+0000","last_fullsized":"2024-09-19T07:54:22.554597+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.568962+0000","last_change":"2024-09-19T07:54:22.564048+0000","last_active":"2024-09-19T07:54:23.568962+0000","last_peered":"2024-09-19T07:54:23.568962+0000","last_clean":"2024-09-19T07:54:23.568962+0000","last_became_active":"2024-09-19T07:54:22.563775+0000","last_became_peered":"2024-09-19T07:54:22.563775+0000","last_unstale":"2024-09-19T07:54:23.568962+0000","last_undegraded":"2024-09-19T07:54:23.568962+0000","last_fullsized":"2024-09-19T07:54:23.568962+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:57.190350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585516+0000","last_change":"2024-09-19T07:54:23.584566+0000","last_active":"2024-09-19T07:54:23.585516+0000","last_peered":"2024-09-19T07:54:23.585516+0000","last_clean":"2024-09-19T07:54:23.585516+0000","last_became_active":"2024-09-19T07:54:23.584002+0000","last_became_peered":"2024-09-19T07:54:23.584002+0000","last_unstale":"2024-09-19T07:54:23.585516+0000","last_undegraded":"2024-09-19T07:54:23.585516+0000","last_fullsized":"2024-09-19T07:54:23.585516+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:58.194998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565335+0000","last_change":"2024-09-19T07:54:22.558583+0000","last_active":"2024-09-19T07:54:22.565335+0000","last_peered":"2024-09-19T07:54:22.565335+0000","last_clean":"2024-09-19T07:54:22.565335+0000","last_became_active":"2024-09-19T07:54:22.558239+0000","last_became_peered":"2024-09-19T07:54:22.558239+0000","last_unstale":"2024-09-19T07:54:22.565335+0000","last_undegraded":"2024-09-19T07:54:22.565335+0000","last_fullsized":"2024-09-19T07:54:22.565335+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:37.052983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.553358+0000","last_change":"2024-09-19T07:54:22.553358+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.553358+0000","last_undegraded":"2024-09-19T07:54:22.553358+0000","last_fullsized":"2024-09-19T07:54:22.553358+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"128'2","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572673+0000","last_change":"2024-09-19T07:54:22.561535+0000","last_active":"2024-09-19T07:54:24.572673+0000","last_peered":"2024-09-19T07:54:24.572673+0000","last_clean":"2024-09-19T07:54:24.572673+0000","last_became_active":"2024-09-19T07:54:22.561210+0000","last_became_peered":"2024-09-19T07:54:22.561210+0000","last_unstale":"2024-09-19T07:54:24.572673+0000","last_undegraded":"2024-09-19T07:54:24.572673+0000","last_fullsized":"2024-09-19T07:54:24.572673+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:13.281363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.590668+0000","last_change":"2024-09-19T07:54:23.588433+0000","last_active":"2024-09-19T07:54:23.590668+0000","last_peered":"2024-09-19T07:54:23.590668+0000","last_clean":"2024-09-19T07:54:23.590668+0000","last_became_active":"2024-09-19T07:54:23.587803+0000","last_became_peered":"2024-09-19T07:54:23.587803+0000","last_unstale":"2024-09-19T07:54:23.590668+0000","last_undegraded":"2024-09-19T07:54:23.590668+0000","last_fullsized":"2024-09-19T07:54:23.590668+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:10.512937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.562219+0000","last_change":"2024-09-19T07:54:22.555487+0000","last_active":"2024-09-19T07:54:23.562219+0000","last_peered":"2024-09-19T07:54:23.562219+0000","last_clean":"2024-09-19T07:54:23.562219+0000","last_became_active":"2024-09-19T07:54:22.555215+0000","last_became_peered":"2024-09-19T07:54:22.555215+0000","last_unstale":"2024-09-19T07:54:23.562219+0000","last_undegraded":"2024-09-19T07:54:23.562219+0000","last_fullsized":"2024-09-19T07:54:23.562219+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:01.620005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.586495+0000","last_change":"2024-09-19T07:54:23.585093+0000","last_active":"2024-09-19T07:54:23.586495+0000","last_peered":"2024-09-19T07:54:23.586495+0000","last_clean":"2024-09-19T07:54:23.586495+0000","last_became_active":"2024-09-19T07:54:23.584837+0000","last_became_peered":"2024-09-19T07:54:23.584837+0000","last_unstale":"2024-09-19T07:54:23.586495+0000","last_undegraded":"2024-09-19T07:54:23.586495+0000","last_fullsized":"2024-09-19T07:54:23.586495+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:43.491587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.565590+0000","last_change":"2024-09-19T07:54:22.564147+0000","last_active":"2024-09-19T07:54:23.565590+0000","last_peered":"2024-09-19T07:54:23.565590+0000","last_clean":"2024-09-19T07:54:23.565590+0000","last_became_active":"2024-09-19T07:54:22.563844+0000","last_became_peered":"2024-09-19T07:54:22.563844+0000","last_unstale":"2024-09-19T07:54:23.565590+0000","last_undegraded":"2024-09-19T07:54:23.565590+0000","last_fullsized":"2024-09-19T07:54:23.565590+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:43.655888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585486+0000","last_change":"2024-09-19T07:54:23.582012+0000","last_active":"2024-09-19T07:54:23.585486+0000","last_peered":"2024-09-19T07:54:23.585486+0000","last_clean":"2024-09-19T07:54:23.585486+0000","last_became_active":"2024-09-19T07:54:23.581143+0000","last_became_peered":"2024-09-19T07:54:23.581143+0000","last_unstale":"2024-09-19T07:54:23.585486+0000","last_undegraded":"2024-09-19T07:54:23.585486+0000","last_fullsized":"2024-09-19T07:54:23.585486+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:20.783132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565313+0000","last_change":"2024-09-19T07:54:22.558393+0000","last_active":"2024-09-19T07:54:22.565313+0000","last_peered":"2024-09-19T07:54:22.565313+0000","last_clean":"2024-09-19T07:54:22.565313+0000","last_became_active":"2024-09-19T07:54:22.558091+0000","last_became_peered":"2024-09-19T07:54:22.558091+0000","last_unstale":"2024-09-19T07:54:22.565313+0000","last_undegraded":"2024-09-19T07:54:22.565313+0000","last_fullsized":"2024-09-19T07:54:22.565313+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:09.508289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.549695+0000","last_change":"2024-09-19T07:54:22.549695+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.549695+0000","last_undegraded":"2024-09-19T07:54:22.549695+0000","last_fullsized":"2024-09-19T07:54:22.549695+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.562667+0000","last_change":"2024-09-19T07:54:22.567926+0000","last_active":"2024-09-19T07:54:23.562667+0000","last_peered":"2024-09-19T07:54:23.562667+0000","last_clean":"2024-09-19T07:54:23.562667+0000","last_became_active":"2024-09-19T07:54:22.567617+0000","last_became_peered":"2024-09-19T07:54:22.567617+0000","last_unstale":"2024-09-19T07:54:23.562667+0000","last_undegraded":"2024-09-19T07:54:23.562667+0000","last_fullsized":"2024-09-19T07:54:23.562667+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:41:15.024194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.554395+0000","last_change":"2024-09-19T07:54:22.554395+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.554395+0000","last_undegraded":"2024-09-19T07:54:22.554395+0000","last_fullsized":"2024-09-19T07:54:22.554395+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.570070+0000","last_change":"2024-09-19T07:54:22.558104+0000","last_active":"2024-09-19T07:54:23.570070+0000","last_peered":"2024-09-19T07:54:23.570070+0000","last_clean":"2024-09-19T07:54:23.570070+0000","last_became_active":"2024-09-19T07:54:22.557825+0000","last_became_peered":"2024-09-19T07:54:22.557825+0000","last_unstale":"2024-09-19T07:54:23.570070+0000","last_undegraded":"2024-09-19T07:54:23.570070+0000","last_fullsized":"2024-09-19T07:54:23.570070+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:36:36.185111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.553286+0000","last_change":"2024-09-19T07:54:22.553286+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.553286+0000","last_undegraded":"2024-09-19T07:54:22.553286+0000","last_fullsized":"2024-09-19T07:54:22.553286+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.566668+0000","last_change":"2024-09-19T07:54:22.560118+0000","last_active":"2024-09-19T07:54:22.566668+0000","last_peered":"2024-09-19T07:54:22.566668+0000","last_clean":"2024-09-19T07:54:22.566668+0000","last_became_active":"2024-09-19T07:54:22.559838+0000","last_became_peered":"2024-09-19T07:54:22.559838+0000","last_unstale":"2024-09-19T07:54:22.566668+0000","last_undegraded":"2024-09-19T07:54:22.566668+0000","last_fullsized":"2024-09-19T07:54:22.566668+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:06.668616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.587234+0000","last_change":"2024-09-19T07:54:23.586194+0000","last_active":"2024-09-19T07:54:23.587234+0000","last_peered":"2024-09-19T07:54:23.587234+0000","last_clean":"2024-09-19T07:54:23.587234+0000","last_became_active":"2024-09-19T07:54:23.585946+0000","last_became_peered":"2024-09-19T07:54:23.585946+0000","last_unstale":"2024-09-19T07:54:23.587234+0000","last_undegraded":"2024-09-19T07:54:23.587234+0000","last_fullsized":"2024-09-19T07:54:23.587234+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:27.045701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565875+0000","last_change":"2024-09-19T07:54:22.562538+0000","last_active":"2024-09-19T07:54:22.565875+0000","last_peered":"2024-09-19T07:54:22.565875+0000","last_clean":"2024-09-19T07:54:22.565875+0000","last_became_active":"2024-09-19T07:54:22.562257+0000","last_became_peered":"2024-09-19T07:54:22.562257+0000","last_unstale":"2024-09-19T07:54:22.565875+0000","last_undegraded":"2024-09-19T07:54:22.565875+0000","last_fullsized":"2024-09-19T07:54:22.565875+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:20.366808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.590798+0000","last_change":"2024-09-19T07:54:23.588887+0000","last_active":"2024-09-19T07:54:23.590798+0000","last_peered":"2024-09-19T07:54:23.590798+0000","last_clean":"2024-09-19T07:54:23.590798+0000","last_became_active":"2024-09-19T07:54:23.588623+0000","last_became_peered":"2024-09-19T07:54:23.588623+0000","last_unstale":"2024-09-19T07:54:23.590798+0000","last_undegraded":"2024-09-19T07:54:23.590798+0000","last_fullsized":"2024-09-19T07:54:23.590798+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:38.057631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.569582+0000","last_change":"2024-09-19T07:54:22.565171+0000","last_active":"2024-09-19T07:54:23.569582+0000","last_peered":"2024-09-19T07:54:23.569582+0000","last_clean":"2024-09-19T07:54:23.569582+0000","last_became_active":"2024-09-19T07:54:22.564876+0000","last_became_peered":"2024-09-19T07:54:22.564876+0000","last_unstale":"2024-09-19T07:54:23.569582+0000","last_undegraded":"2024-09-19T07:54:23.569582+0000","last_fullsized":"2024-09-19T07:54:23.569582+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:36.291156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":364,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.567647+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:54:23.567647+0000","last_peered":"2024-09-19T07:54:23.567647+0000","last_clean":"2024-09-19T07:54:23.567647+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:54:23.567647+0000","last_undegraded":"2024-09-19T07:54:23.567647+0000","last_fullsized":"2024-09-19T07:54:23.567647+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509581,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83268,"kb_used_data":2152,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288572,"statfs":{"total":96636764160,"available":96551497728,"internally_reserved":0,"allocated":2203648,"data_stored":1331051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509580,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104952,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266888,"statfs":{"total":96636764160,"available":96529293312,"internally_reserved":0,"allocated":1601536,"data_stored":739653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509581,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72944,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298896,"statfs":{"total":96636764160,"available":96562069504,"internally_reserved":0,"allocated":1597440,"data_stored":739003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509581,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86832,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285008,"statfs":{"total":96636764160,"available":96547848192,"internally_reserved":0,"allocated":1593344,"data_stored":738981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509580,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105460,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266380,"statfs":{"total":96636764160,"available":96528773120,"internally_reserved":0,"allocated":2195456,"data_stored":1329405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542284,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118748,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253092,"statfs":{"total":96636764160,"available":96515166208,"internally_reserved":0,"allocated":1581056,"data_stored":731814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542284,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82720,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94289120,"statfs":{"total":96636764160,"available":96552058880,"internally_reserved":0,"allocated":1581056,"data_stored":731814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574990,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109052,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262788,"statfs":{"total":96636764160,"available":96525094912,"internally_reserved":0,"allocated":1613824,"data_stored":746289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:54:27.250 INFO:tasks.ceph.ceph_manager.ceph:PG 20.3b is not active+clean 2024-09-19T07:54:27.250 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:54:22.551618+0000', 'last_change': '2024-09-19T07:54:22.551618+0000', 'last_active': '2024-09-19T07:54:22.540405+0000', 'last_peered': '2024-09-19T07:54:22.540405+0000', 'last_clean': '2024-09-19T07:54:22.540405+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:54:22.551618+0000', 'last_undegraded': '2024-09-19T07:54:22.551618+0000', 'last_fullsized': '2024-09-19T07:54:22.551618+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_clean_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T07:54:27.250 INFO:tasks.ceph.ceph_manager.ceph:PG 20.3a is not active+clean 2024-09-19T07:54:27.250 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:54:22.555440+0000', 'last_change': '2024-09-19T07:54:22.555440+0000', 'last_active': '2024-09-19T07:54:22.540405+0000', 'last_peered': '2024-09-19T07:54:22.540405+0000', 'last_clean': '2024-09-19T07:54:22.540405+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:54:22.555440+0000', 'last_undegraded': '2024-09-19T07:54:22.555440+0000', 'last_fullsized': '2024-09-19T07:54:22.555440+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_clean_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T07:54:27.250 INFO:tasks.ceph.ceph_manager.ceph:PG 20.37 is not active+clean 2024-09-19T07:54:27.250 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:54:22.554930+0000', 'last_change': '2024-09-19T07:54:22.554930+0000', 'last_active': '2024-09-19T07:54:22.540405+0000', 'last_peered': '2024-09-19T07:54:22.540405+0000', 'last_clean': '2024-09-19T07:54:22.540405+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:54:22.554930+0000', 'last_undegraded': '2024-09-19T07:54:22.554930+0000', 'last_fullsized': '2024-09-19T07:54:22.554930+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_clean_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T07:54:27.250 INFO:tasks.ceph.ceph_manager.ceph:PG 20.33 is not active+clean 2024-09-19T07:54:27.251 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:54:22.554415+0000', 'last_change': '2024-09-19T07:54:22.554415+0000', 'last_active': '2024-09-19T07:54:22.540405+0000', 'last_peered': '2024-09-19T07:54:22.540405+0000', 'last_clean': '2024-09-19T07:54:22.540405+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:54:22.554415+0000', 'last_undegraded': '2024-09-19T07:54:22.554415+0000', 'last_fullsized': '2024-09-19T07:54:22.554415+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_clean_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T07:54:27.251 INFO:tasks.ceph.ceph_manager.ceph:PG 20.30 is not active+clean 2024-09-19T07:54:27.251 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:54:22.557847+0000', 'last_change': '2024-09-19T07:54:22.557847+0000', 'last_active': '2024-09-19T07:54:22.540405+0000', 'last_peered': '2024-09-19T07:54:22.540405+0000', 'last_clean': '2024-09-19T07:54:22.540405+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:54:22.557847+0000', 'last_undegraded': '2024-09-19T07:54:22.557847+0000', 'last_fullsized': '2024-09-19T07:54:22.557847+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_clean_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T07:54:27.251 INFO:tasks.ceph.ceph_manager.ceph:PG 20.2f is not active+clean 2024-09-19T07:54:27.251 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:54:22.553868+0000', 'last_change': '2024-09-19T07:54:22.553868+0000', 'last_active': '2024-09-19T07:54:22.540405+0000', 'last_peered': '2024-09-19T07:54:22.540405+0000', 'last_clean': '2024-09-19T07:54:22.540405+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:54:22.553868+0000', 'last_undegraded': '2024-09-19T07:54:22.553868+0000', 'last_fullsized': '2024-09-19T07:54:22.553868+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_clean_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T07:54:27.251 INFO:tasks.ceph.ceph_manager.ceph:PG 20.2e is not active+clean 2024-09-19T07:54:27.251 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:54:22.555444+0000', 'last_change': '2024-09-19T07:54:22.555444+0000', 'last_active': '2024-09-19T07:54:22.540405+0000', 'last_peered': '2024-09-19T07:54:22.540405+0000', 'last_clean': '2024-09-19T07:54:22.540405+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:54:22.555444+0000', 'last_undegraded': '2024-09-19T07:54:22.555444+0000', 'last_fullsized': '2024-09-19T07:54:22.555444+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_clean_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T07:54:27.251 INFO:tasks.ceph.ceph_manager.ceph:PG 20.2c is not active+clean 2024-09-19T07:54:27.251 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:54:22.552956+0000', 'last_change': '2024-09-19T07:54:22.552956+0000', 'last_active': '2024-09-19T07:54:22.540405+0000', 'last_peered': '2024-09-19T07:54:22.540405+0000', 'last_clean': '2024-09-19T07:54:22.540405+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:54:22.552956+0000', 'last_undegraded': '2024-09-19T07:54:22.552956+0000', 'last_fullsized': '2024-09-19T07:54:22.552956+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_clean_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T07:54:27.251 INFO:tasks.ceph.ceph_manager.ceph:PG 20.28 is not active+clean 2024-09-19T07:54:27.252 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:54:22.557404+0000', 'last_change': '2024-09-19T07:54:22.557404+0000', 'last_active': '2024-09-19T07:54:22.540405+0000', 'last_peered': '2024-09-19T07:54:22.540405+0000', 'last_clean': '2024-09-19T07:54:22.540405+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:54:22.557404+0000', 'last_undegraded': '2024-09-19T07:54:22.557404+0000', 'last_fullsized': '2024-09-19T07:54:22.557404+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_clean_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T07:54:27.252 INFO:tasks.ceph.ceph_manager.ceph:PG 20.1e is not active+clean 2024-09-19T07:54:27.252 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:54:22.555018+0000', 'last_change': '2024-09-19T07:54:22.555018+0000', 'last_active': '2024-09-19T07:54:22.540405+0000', 'last_peered': '2024-09-19T07:54:22.540405+0000', 'last_clean': '2024-09-19T07:54:22.540405+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:54:22.555018+0000', 'last_undegraded': '2024-09-19T07:54:22.555018+0000', 'last_fullsized': '2024-09-19T07:54:22.555018+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_clean_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T07:54:27.252 INFO:tasks.ceph.ceph_manager.ceph:PG 20.11 is not active+clean 2024-09-19T07:54:27.252 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:54:22.556350+0000', 'last_change': '2024-09-19T07:54:22.556350+0000', 'last_active': '2024-09-19T07:54:22.540405+0000', 'last_peered': '2024-09-19T07:54:22.540405+0000', 'last_clean': '2024-09-19T07:54:22.540405+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:54:22.556350+0000', 'last_undegraded': '2024-09-19T07:54:22.556350+0000', 'last_fullsized': '2024-09-19T07:54:22.556350+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_clean_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T07:54:27.252 INFO:tasks.ceph.ceph_manager.ceph:PG 20.17 is not active+clean 2024-09-19T07:54:27.252 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:54:22.551690+0000', 'last_change': '2024-09-19T07:54:22.551690+0000', 'last_active': '2024-09-19T07:54:22.540405+0000', 'last_peered': '2024-09-19T07:54:22.540405+0000', 'last_clean': '2024-09-19T07:54:22.540405+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:54:22.551690+0000', 'last_undegraded': '2024-09-19T07:54:22.551690+0000', 'last_fullsized': '2024-09-19T07:54:22.551690+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_clean_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T07:54:27.252 INFO:tasks.ceph.ceph_manager.ceph:PG 20.c is not active+clean 2024-09-19T07:54:27.252 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:54:22.552495+0000', 'last_change': '2024-09-19T07:54:22.552495+0000', 'last_active': '2024-09-19T07:54:22.540405+0000', 'last_peered': '2024-09-19T07:54:22.540405+0000', 'last_clean': '2024-09-19T07:54:22.540405+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:54:22.552495+0000', 'last_undegraded': '2024-09-19T07:54:22.552495+0000', 'last_fullsized': '2024-09-19T07:54:22.552495+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_clean_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T07:54:27.252 INFO:tasks.ceph.ceph_manager.ceph:PG 20.e is not active+clean 2024-09-19T07:54:27.253 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:54:22.554597+0000', 'last_change': '2024-09-19T07:54:22.554597+0000', 'last_active': '2024-09-19T07:54:22.540405+0000', 'last_peered': '2024-09-19T07:54:22.540405+0000', 'last_clean': '2024-09-19T07:54:22.540405+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:54:22.554597+0000', 'last_undegraded': '2024-09-19T07:54:22.554597+0000', 'last_fullsized': '2024-09-19T07:54:22.554597+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_clean_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T07:54:27.253 INFO:tasks.ceph.ceph_manager.ceph:PG 20.0 is not active+clean 2024-09-19T07:54:27.253 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:54:22.553358+0000', 'last_change': '2024-09-19T07:54:22.553358+0000', 'last_active': '2024-09-19T07:54:22.540405+0000', 'last_peered': '2024-09-19T07:54:22.540405+0000', 'last_clean': '2024-09-19T07:54:22.540405+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:54:22.553358+0000', 'last_undegraded': '2024-09-19T07:54:22.553358+0000', 'last_fullsized': '2024-09-19T07:54:22.553358+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_clean_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T07:54:27.253 INFO:tasks.ceph.ceph_manager.ceph:PG 20.4 is not active+clean 2024-09-19T07:54:27.253 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:54:22.549695+0000', 'last_change': '2024-09-19T07:54:22.549695+0000', 'last_active': '2024-09-19T07:54:22.540405+0000', 'last_peered': '2024-09-19T07:54:22.540405+0000', 'last_clean': '2024-09-19T07:54:22.540405+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:54:22.549695+0000', 'last_undegraded': '2024-09-19T07:54:22.549695+0000', 'last_fullsized': '2024-09-19T07:54:22.549695+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_clean_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T07:54:27.253 INFO:tasks.ceph.ceph_manager.ceph:PG 20.5 is not active+clean 2024-09-19T07:54:27.253 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:54:22.554395+0000', 'last_change': '2024-09-19T07:54:22.554395+0000', 'last_active': '2024-09-19T07:54:22.540405+0000', 'last_peered': '2024-09-19T07:54:22.540405+0000', 'last_clean': '2024-09-19T07:54:22.540405+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:54:22.554395+0000', 'last_undegraded': '2024-09-19T07:54:22.554395+0000', 'last_fullsized': '2024-09-19T07:54:22.554395+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_clean_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T07:54:27.253 INFO:tasks.ceph.ceph_manager.ceph:PG 20.6 is not active+clean 2024-09-19T07:54:27.253 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 127, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:54:22.553286+0000', 'last_change': '2024-09-19T07:54:22.553286+0000', 'last_active': '2024-09-19T07:54:22.540405+0000', 'last_peered': '2024-09-19T07:54:22.540405+0000', 'last_clean': '2024-09-19T07:54:22.540405+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:54:22.553286+0000', 'last_undegraded': '2024-09-19T07:54:22.553286+0000', 'last_fullsized': '2024-09-19T07:54:22.553286+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'last_clean_scrub_stamp': '2024-09-19T07:54:22.540405+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T07:54:27.254 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T07:54:27.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.388+0000 7fd15a9b7640 1 Processor -- start 2024-09-19T07:54:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.388+0000 7fd15a9b7640 1 -- start start 2024-09-19T07:54:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.389+0000 7fd15a9b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1540b0a80 0x7fd1540cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.389+0000 7fd15a9b7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd154161d90 0x7fd1540cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.389+0000 7fd15a9b7640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd15406c990 con 0x7fd1541619b0 2024-09-19T07:54:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.389+0000 7fd15a9b7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fd1540cec10 con 0x7fd154161d90 2024-09-19T07:54:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.389+0000 7fd15a9b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd1540ce930 con 0x7fd1540b0a80 2024-09-19T07:54:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.389+0000 7fd158f54640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd154161d90 0x7fd1540cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.389+0000 7fd153fff640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fd1541619b0 0x7fd1540d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:45992/0 (socket says 172.21.15.16:45992) 2024-09-19T07:54:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.389+0000 7fd1537fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1540b0a80 0x7fd1540cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.389+0000 7fd153fff640 1 -- 172.21.15.16:0/2324471553 learned_addr learned my addr 172.21.15.16:0/2324471553 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:27.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.390+0000 7fd152ffd640 1 -- 172.21.15.16:0/2324471553 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1697309925 0 0) 0x7fd15406c990 con 0x7fd1541619b0 2024-09-19T07:54:27.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.390+0000 7fd152ffd640 1 -- 172.21.15.16:0/2324471553 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd13c003520 con 0x7fd1541619b0 2024-09-19T07:54:27.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.390+0000 7fd1537fe640 1 -- 172.21.15.16:0/2324471553 >> v2:172.21.15.146:3300/0 conn(0x7fd154161d90 msgr2=0x7fd1540cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:27.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.390+0000 7fd1537fe640 1 --2- 172.21.15.16:0/2324471553 >> v2:172.21.15.146:3300/0 conn(0x7fd154161d90 0x7fd1540cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.390+0000 7fd1537fe640 1 -- 172.21.15.16:0/2324471553 >> v1:172.21.15.146:6789/0 conn(0x7fd1541619b0 legacy=0x7fd1540d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:27.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.390+0000 7fd1537fe640 1 -- 172.21.15.16:0/2324471553 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1540b0540 con 0x7fd1540b0a80 2024-09-19T07:54:27.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.391+0000 7fd1537fe640 1 --2- 172.21.15.16:0/2324471553 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1540b0a80 0x7fd1540cbcd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd1480097a0 tx=0x7fd14802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dbdfb09f06267b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:27.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.391+0000 7fd152ffd640 1 -- 172.21.15.16:0/2324471553 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148037070 con 0x7fd1540b0a80 2024-09-19T07:54:27.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.391+0000 7fd152ffd640 1 -- 172.21.15.16:0/2324471553 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd148032440 con 0x7fd1540b0a80 2024-09-19T07:54:27.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.391+0000 7fd152ffd640 1 -- 172.21.15.16:0/2324471553 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1480327f0 con 0x7fd1540b0a80 2024-09-19T07:54:27.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.391+0000 7fd15a9b7640 1 -- 172.21.15.16:0/2324471553 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1540b0a80 msgr2=0x7fd1540cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:27.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.391+0000 7fd15a9b7640 1 --2- 172.21.15.16:0/2324471553 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1540b0a80 0x7fd1540cbcd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd1480097a0 tx=0x7fd14802a180 comp rx=0 tx=0).stop 2024-09-19T07:54:27.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.392+0000 7fd15a9b7640 1 -- 172.21.15.16:0/2324471553 shutdown_connections 2024-09-19T07:54:27.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.392+0000 7fd15a9b7640 1 --2- 172.21.15.16:0/2324471553 >> v2:172.21.15.146:3300/0 conn(0x7fd154161d90 0x7fd1540cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.392+0000 7fd15a9b7640 1 --2- 172.21.15.16:0/2324471553 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1540b0a80 0x7fd1540cbcd0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.392+0000 7fd15a9b7640 1 -- 172.21.15.16:0/2324471553 >> 172.21.15.16:0/2324471553 conn(0x7fd15409a9f0 msgr2=0x7fd1540b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:27.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.392+0000 7fd15a9b7640 1 -- 172.21.15.16:0/2324471553 shutdown_connections 2024-09-19T07:54:27.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.392+0000 7fd15a9b7640 1 -- 172.21.15.16:0/2324471553 wait complete. 2024-09-19T07:54:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.393+0000 7fd15a9b7640 1 Processor -- start 2024-09-19T07:54:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.393+0000 7fd15a9b7640 1 -- start start 2024-09-19T07:54:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.394+0000 7fd15a9b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1540b0a80 0x7fd1541f7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.394+0000 7fd15a9b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1541619b0 0x7fd1540b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.394+0000 7fd15a9b7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd154161d90 0x7fd1540b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.394+0000 7fd15a9b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1540cd330 con 0x7fd1541619b0 2024-09-19T07:54:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.394+0000 7fd15a9b7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd15406c620 con 0x7fd154161d90 2024-09-19T07:54:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.394+0000 7fd15a9b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd1540cd4b0 con 0x7fd1540b0a80 2024-09-19T07:54:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.394+0000 7fd1537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1541619b0 0x7fd1540b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.394+0000 7fd1537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1541619b0 0x7fd1540b1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48450/0 (socket says 172.21.15.16:48450) 2024-09-19T07:54:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.394+0000 7fd1537fe640 1 -- 172.21.15.16:0/3110666580 learned_addr learned my addr 172.21.15.16:0/3110666580 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.394+0000 7fd153fff640 1 --2- 172.21.15.16:0/3110666580 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1540b0a80 0x7fd1541f7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.394+0000 7fd158f54640 1 --2- 172.21.15.16:0/3110666580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd154161d90 0x7fd1540b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.394+0000 7fd1537fe640 1 -- 172.21.15.16:0/3110666580 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1540b0a80 msgr2=0x7fd1541f7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.394+0000 7fd1537fe640 1 --2- 172.21.15.16:0/3110666580 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1540b0a80 0x7fd1541f7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.394+0000 7fd1537fe640 1 -- 172.21.15.16:0/3110666580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd154161d90 msgr2=0x7fd1540b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.394+0000 7fd1537fe640 1 --2- 172.21.15.16:0/3110666580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd154161d90 0x7fd1540b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.394+0000 7fd1537fe640 1 -- 172.21.15.16:0/3110666580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1541f48b0 con 0x7fd1541619b0 2024-09-19T07:54:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.394+0000 7fd1537fe640 1 --2- 172.21.15.16:0/3110666580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1541619b0 0x7fd1540b1eb0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fd14802a1c0 tx=0x7fd1480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.394+0000 7fd150ff9640 1 -- 172.21.15.16:0/3110666580 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148037040 con 0x7fd1541619b0 2024-09-19T07:54:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.394+0000 7fd150ff9640 1 -- 172.21.15.16:0/3110666580 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd148042070 con 0x7fd1541619b0 2024-09-19T07:54:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.395+0000 7fd150ff9640 1 -- 172.21.15.16:0/3110666580 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14803b930 con 0x7fd1541619b0 2024-09-19T07:54:27.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.395+0000 7fd15a9b7640 1 -- 172.21.15.16:0/3110666580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1541f4b90 con 0x7fd1541619b0 2024-09-19T07:54:27.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.395+0000 7fd15a9b7640 1 -- 172.21.15.16:0/3110666580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1541f7960 con 0x7fd1541619b0 2024-09-19T07:54:27.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.396+0000 7fd15a9b7640 1 -- 172.21.15.16:0/3110666580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1540d0440 con 0x7fd1541619b0 2024-09-19T07:54:27.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.396+0000 7fd150ff9640 1 -- 172.21.15.16:0/3110666580 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd14803bad0 con 0x7fd1541619b0 2024-09-19T07:54:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.397+0000 7fd150ff9640 1 --2- 172.21.15.16:0/3110666580 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd14006c2b0 0x7fd14006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.397+0000 7fd150ff9640 1 -- 172.21.15.16:0/3110666580 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7fd1480abe00 con 0x7fd1541619b0 2024-09-19T07:54:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.397+0000 7fd153fff640 1 --2- 172.21.15.16:0/3110666580 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd14006c2b0 0x7fd14006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.397+0000 7fd153fff640 1 --2- 172.21.15.16:0/3110666580 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd14006c2b0 0x7fd14006e770 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd144005880 tx=0x7fd144009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.401+0000 7fd150ff9640 1 -- 172.21.15.16:0/3110666580 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd14807a3d0 con 0x7fd1541619b0 2024-09-19T07:54:27.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.651+0000 7fd15a9b7640 1 -- 172.21.15.16:0/3110666580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd1540d0d40 con 0x7fd1541619b0 2024-09-19T07:54:27.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.651+0000 7fd150ff9640 1 -- 172.21.15.16:0/3110666580 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1562 (secure 0 0 0) 0x7fd148079cd0 con 0x7fd1541619b0 2024-09-19T07:54:27.653 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:27.653 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":705,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":129,"num_osds":8,"num_up_osds":8,"osd_up_since":1726731773,"num_in_osds":8,"osd_in_since":1726731764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":79},{"state_name":"creating+peering","count":18}],"num_pgs":97,"num_pools":3,"num_objects":18,"data_bytes":592708,"bytes_used":782311424,"bytes_avail":772311801856,"bytes_total":773094113280,"inactive_pgs_ratio":0.18556700646877289,"write_bytes_sec":2621,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":72,"btime":"2024-09-19T07:54:25:574262+0000","id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"d","status":"up:active","gid":7854}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-19T07:53:32.607149+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T07:54:27.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.653+0000 7fd15a9b7640 1 -- 172.21.15.16:0/3110666580 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd14006c2b0 msgr2=0x7fd14006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:27.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.653+0000 7fd15a9b7640 1 --2- 172.21.15.16:0/3110666580 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd14006c2b0 0x7fd14006e770 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd144005880 tx=0x7fd144009040 comp rx=0 tx=0).stop 2024-09-19T07:54:27.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.653+0000 7fd15a9b7640 1 -- 172.21.15.16:0/3110666580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1541619b0 msgr2=0x7fd1540b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:27.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.653+0000 7fd15a9b7640 1 --2- 172.21.15.16:0/3110666580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1541619b0 0x7fd1540b1eb0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fd14802a1c0 tx=0x7fd1480046c0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.653+0000 7fd15a9b7640 1 -- 172.21.15.16:0/3110666580 shutdown_connections 2024-09-19T07:54:27.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.653+0000 7fd15a9b7640 1 --2- 172.21.15.16:0/3110666580 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd14006c2b0 0x7fd14006e770 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.653+0000 7fd15a9b7640 1 --2- 172.21.15.16:0/3110666580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd154161d90 0x7fd1540b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.653+0000 7fd15a9b7640 1 --2- 172.21.15.16:0/3110666580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1541619b0 0x7fd1540b1eb0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.653+0000 7fd15a9b7640 1 --2- 172.21.15.16:0/3110666580 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1540b0a80 0x7fd1541f7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.653+0000 7fd15a9b7640 1 -- 172.21.15.16:0/3110666580 >> 172.21.15.16:0/3110666580 conn(0x7fd15409a9f0 msgr2=0x7fd1540cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:27.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.654+0000 7fd15a9b7640 1 -- 172.21.15.16:0/3110666580 shutdown_connections 2024-09-19T07:54:27.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.654+0000 7fd15a9b7640 1 -- 172.21.15.16:0/3110666580 wait complete. 2024-09-19T07:54:27.665 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-19T07:54:27.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:54:27.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.767+0000 7f6a52f8d640 1 Processor -- start 2024-09-19T07:54:27.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.767+0000 7f6a52f8d640 1 -- start start 2024-09-19T07:54:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.768+0000 7f6a52f8d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6a4c0d4370 0x7f6a4c0d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.768+0000 7f6a52f8d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a4c0d1880 con 0x7f6a4c15e2a0 2024-09-19T07:54:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.768+0000 7f6a52f8d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a4c0d1b60 con 0x7f6a4c15e680 2024-09-19T07:54:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.768+0000 7f6a52f8d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f6a4c0cf2d0 con 0x7f6a4c0d4370 2024-09-19T07:54:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.768+0000 7f6a50d29640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6a4c15e2a0 0x7f6a4c06d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56784/0 (socket says 172.21.15.16:56784) 2024-09-19T07:54:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.768+0000 7f6a50d29640 1 -- 172.21.15.16:0/2140690219 learned_addr learned my addr 172.21.15.16:0/2140690219 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.768+0000 7f6a437fe640 1 -- 172.21.15.16:0/2140690219 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 314030791 0 0) 0x7f6a4c0d1880 con 0x7f6a4c15e2a0 2024-09-19T07:54:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.768+0000 7f6a5152a640 1 --2- 172.21.15.16:0/2140690219 >> v2:172.21.15.146:3300/0 conn(0x7f6a4c0d4370 0x7f6a4c0d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.768+0000 7f6a437fe640 1 -- 172.21.15.16:0/2140690219 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a34003520 con 0x7f6a4c15e2a0 2024-09-19T07:54:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.768+0000 7f6a437fe640 1 -- 172.21.15.16:0/2140690219 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 194810057 0 0) 0x7f6a34003520 con 0x7f6a4c15e2a0 2024-09-19T07:54:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.768+0000 7f6a437fe640 1 -- 172.21.15.16:0/2140690219 >> v2:172.21.15.146:3300/0 conn(0x7f6a4c0d4370 msgr2=0x7f6a4c0d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.768+0000 7f6a437fe640 1 --2- 172.21.15.16:0/2140690219 >> v2:172.21.15.146:3300/0 conn(0x7f6a4c0d4370 0x7f6a4c0d4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.768+0000 7f6a437fe640 1 -- 172.21.15.16:0/2140690219 >> v1:172.21.15.146:6789/0 conn(0x7f6a4c15e680 legacy=0x7f6a4c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.768+0000 7f6a437fe640 1 -- 172.21.15.16:0/2140690219 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4c0cf5b0 con 0x7f6a4c15e2a0 2024-09-19T07:54:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.768+0000 7f6a437fe640 1 -- 172.21.15.16:0/2140690219 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f6a48001670 con 0x7f6a4c15e2a0 2024-09-19T07:54:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.768+0000 7f6a437fe640 1 -- 172.21.15.16:0/2140690219 >> v1:172.21.15.16:6789/0 conn(0x7f6a4c15e2a0 legacy=0x7f6a4c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.768+0000 7f6a437fe640 1 --2- 172.21.15.16:0/2140690219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a340039b0 0x7f6a34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.769+0000 7f6a437fe640 1 --2- 172.21.15.16:0/2140690219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a34004800 0x7f6a34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.769+0000 7f6a437fe640 1 --2- 172.21.15.16:0/2140690219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a340051c0 0x7f6a3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.769+0000 7f6a437fe640 1 -- 172.21.15.16:0/2140690219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c0cf2d0 con 0x7f6a340039b0 2024-09-19T07:54:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.769+0000 7f6a437fe640 1 -- 172.21.15.16:0/2140690219 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3400c060 con 0x7f6a34004800 2024-09-19T07:54:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.769+0000 7f6a437fe640 1 -- 172.21.15.16:0/2140690219 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6a3400c260 con 0x7f6a340051c0 2024-09-19T07:54:27.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.769+0000 7f6a50d29640 1 --2- 172.21.15.16:0/2140690219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a340039b0 0x7f6a34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.769+0000 7f6a43fff640 1 --2- 172.21.15.16:0/2140690219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a34004800 0x7f6a34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.769+0000 7f6a5152a640 1 --2- 172.21.15.16:0/2140690219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a340051c0 0x7f6a3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.769+0000 7f6a50d29640 1 -- 172.21.15.16:0/2140690219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a340051c0 msgr2=0x7f6a3400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:27.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.769+0000 7f6a50d29640 1 --2- 172.21.15.16:0/2140690219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a340051c0 0x7f6a3400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.769+0000 7f6a50d29640 1 -- 172.21.15.16:0/2140690219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a34004800 msgr2=0x7f6a34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:27.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.769+0000 7f6a50d29640 1 --2- 172.21.15.16:0/2140690219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a34004800 0x7f6a34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.769+0000 7f6a50d29640 1 -- 172.21.15.16:0/2140690219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a34003520 con 0x7f6a340039b0 2024-09-19T07:54:27.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.769+0000 7f6a5152a640 1 --2- 172.21.15.16:0/2140690219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a340051c0 0x7f6a3400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:27.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.769+0000 7f6a5152a640 1 -- 172.21.15.16:0/2140690219 reap_dead start 2024-09-19T07:54:27.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.770+0000 7f6a50d29640 1 --2- 172.21.15.16:0/2140690219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a340039b0 0x7f6a34003dd0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800c290 tx=0x7f6a34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82131536dd615638 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:27.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.770+0000 7f6a437fe640 1 -- 172.21.15.16:0/2140690219 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48007d30 con 0x7f6a340039b0 2024-09-19T07:54:27.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.770+0000 7f6a437fe640 1 -- 172.21.15.16:0/2140690219 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a48005af0 con 0x7f6a340039b0 2024-09-19T07:54:27.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.770+0000 7f6a437fe640 1 -- 172.21.15.16:0/2140690219 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48007070 con 0x7f6a340039b0 2024-09-19T07:54:27.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.770+0000 7f6a52f8d640 1 -- 172.21.15.16:0/2140690219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a340039b0 msgr2=0x7f6a34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:27.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.770+0000 7f6a52f8d640 1 --2- 172.21.15.16:0/2140690219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a340039b0 0x7f6a34003dd0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800c290 tx=0x7f6a34005830 comp rx=0 tx=0).stop 2024-09-19T07:54:27.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.770+0000 7f6a52f8d640 1 -- 172.21.15.16:0/2140690219 shutdown_connections 2024-09-19T07:54:27.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.770+0000 7f6a52f8d640 1 --2- 172.21.15.16:0/2140690219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a340039b0 0x7f6a34003dd0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.770+0000 7f6a52f8d640 1 -- 172.21.15.16:0/2140690219 >> 172.21.15.16:0/2140690219 conn(0x7f6a4c09d760 msgr2=0x7f6a4c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:27.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.770+0000 7f6a52f8d640 1 -- 172.21.15.16:0/2140690219 shutdown_connections 2024-09-19T07:54:27.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.770+0000 7f6a52f8d640 1 -- 172.21.15.16:0/2140690219 wait complete. 2024-09-19T07:54:27.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.772+0000 7f6a52f8d640 1 Processor -- start 2024-09-19T07:54:27.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.772+0000 7f6a52f8d640 1 -- start start 2024-09-19T07:54:27.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.772+0000 7f6a52f8d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a340039b0 0x7f6a4c1598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.772+0000 7f6a52f8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a4c1f5740 0x7f6a4c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.772+0000 7f6a52f8d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a4c0cdd00 0x7f6a4c0cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.772+0000 7f6a52f8d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c0d49a0 con 0x7f6a4c1f5740 2024-09-19T07:54:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.772+0000 7f6a52f8d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c0d4820 con 0x7f6a340039b0 2024-09-19T07:54:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.772+0000 7f6a52f8d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6a4c0d4b20 con 0x7f6a4c0cdd00 2024-09-19T07:54:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.773+0000 7f6a43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a4c1f5740 0x7f6a4c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.773+0000 7f6a43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a4c1f5740 0x7f6a4c0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48480/0 (socket says 172.21.15.16:48480) 2024-09-19T07:54:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.773+0000 7f6a43fff640 1 -- 172.21.15.16:0/243995608 learned_addr learned my addr 172.21.15.16:0/243995608 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.773+0000 7f6a50d29640 1 --2- 172.21.15.16:0/243995608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a340039b0 0x7f6a4c1598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.773+0000 7f6a5152a640 1 --2- 172.21.15.16:0/243995608 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a4c0cdd00 0x7f6a4c0cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.773+0000 7f6a43fff640 1 -- 172.21.15.16:0/243995608 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a4c0cdd00 msgr2=0x7f6a4c0cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.773+0000 7f6a43fff640 1 --2- 172.21.15.16:0/243995608 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a4c0cdd00 0x7f6a4c0cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.773+0000 7f6a43fff640 1 -- 172.21.15.16:0/243995608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a340039b0 msgr2=0x7f6a4c1598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.773+0000 7f6a43fff640 1 --2- 172.21.15.16:0/243995608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a340039b0 0x7f6a4c1598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.773+0000 7f6a43fff640 1 -- 172.21.15.16:0/243995608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4c1e7fe0 con 0x7f6a4c1f5740 2024-09-19T07:54:27.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.773+0000 7f6a5152a640 1 --2- 172.21.15.16:0/243995608 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a4c0cdd00 0x7f6a4c0cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.773+0000 7f6a43fff640 1 --2- 172.21.15.16:0/243995608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a4c1f5740 0x7f6a4c0cd7c0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00f5c0 tx=0x7f6a3c00fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.773+0000 7f6a417fa640 1 -- 172.21.15.16:0/243995608 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0052e0 con 0x7f6a4c1f5740 2024-09-19T07:54:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.773+0000 7f6a417fa640 1 -- 172.21.15.16:0/243995608 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a3c018070 con 0x7f6a4c1f5740 2024-09-19T07:54:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.773+0000 7f6a52f8d640 1 -- 172.21.15.16:0/243995608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4c0b81f0 con 0x7f6a4c1f5740 2024-09-19T07:54:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.773+0000 7f6a417fa640 1 -- 172.21.15.16:0/243995608 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c003830 con 0x7f6a4c1f5740 2024-09-19T07:54:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.773+0000 7f6a52f8d640 1 -- 172.21.15.16:0/243995608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a4c0b87b0 con 0x7f6a4c1f5740 2024-09-19T07:54:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.774+0000 7f6a52f8d640 1 -- 172.21.15.16:0/243995608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a4c06d420 con 0x7f6a4c1f5740 2024-09-19T07:54:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.775+0000 7f6a417fa640 1 -- 172.21.15.16:0/243995608 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6a3c0039d0 con 0x7f6a4c1f5740 2024-09-19T07:54:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.775+0000 7f6a417fa640 1 --2- 172.21.15.16:0/243995608 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6a3806c2e0 0x7f6a3806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.775+0000 7f6a417fa640 1 -- 172.21.15.16:0/243995608 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7f6a3c08b2f0 con 0x7f6a4c1f5740 2024-09-19T07:54:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.775+0000 7f6a50d29640 1 --2- 172.21.15.16:0/243995608 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6a3806c2e0 0x7f6a3806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.776+0000 7f6a50d29640 1 --2- 172.21.15.16:0/243995608 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6a3806c2e0 0x7f6a3806e7a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6a48032000 tx=0x7f6a48009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:27.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.779+0000 7f6a417fa640 1 -- 172.21.15.16:0/243995608 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6a3c0598d0 con 0x7f6a4c1f5740 2024-09-19T07:54:27.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.951+0000 7f6a52f8d640 1 -- 172.21.15.16:0/243995608 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6a4c0ce0e0 con 0x7f6a3806c2e0 2024-09-19T07:54:27.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.957+0000 7f6a417fa640 1 -- 172.21.15.16:0/243995608 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250945 (secure 0 0 0) 0x7f6a4c0ce0e0 con 0x7f6a3806c2e0 2024-09-19T07:54:27.961 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:54:27.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.960+0000 7f6a52f8d640 1 -- 172.21.15.16:0/243995608 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6a3806c2e0 msgr2=0x7f6a3806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:27.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.960+0000 7f6a52f8d640 1 --2- 172.21.15.16:0/243995608 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6a3806c2e0 0x7f6a3806e7a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6a48032000 tx=0x7f6a48009030 comp rx=0 tx=0).stop 2024-09-19T07:54:27.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.960+0000 7f6a52f8d640 1 -- 172.21.15.16:0/243995608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a4c1f5740 msgr2=0x7f6a4c0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:27.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.960+0000 7f6a52f8d640 1 --2- 172.21.15.16:0/243995608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a4c1f5740 0x7f6a4c0cd7c0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00f5c0 tx=0x7f6a3c00fa80 comp rx=0 tx=0).stop 2024-09-19T07:54:27.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.960+0000 7f6a52f8d640 1 -- 172.21.15.16:0/243995608 shutdown_connections 2024-09-19T07:54:27.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.960+0000 7f6a52f8d640 1 --2- 172.21.15.16:0/243995608 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6a3806c2e0 0x7f6a3806e7a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.960+0000 7f6a52f8d640 1 --2- 172.21.15.16:0/243995608 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a4c0cdd00 0x7f6a4c0cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.960+0000 7f6a52f8d640 1 --2- 172.21.15.16:0/243995608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a4c1f5740 0x7f6a4c0cd7c0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.960+0000 7f6a52f8d640 1 --2- 172.21.15.16:0/243995608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a340039b0 0x7f6a4c1598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:27.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.960+0000 7f6a52f8d640 1 -- 172.21.15.16:0/243995608 >> 172.21.15.16:0/243995608 conn(0x7f6a4c09d760 msgr2=0x7f6a4c0d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:27.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.960+0000 7f6a52f8d640 1 -- 172.21.15.16:0/243995608 shutdown_connections 2024-09-19T07:54:27.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:27.960+0000 7f6a52f8d640 1 -- 172.21.15.16:0/243995608 wait complete. 2024-09-19T07:54:27.972 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2024-09-19T07:54:26.619972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":190,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":763976,"kb_used_data":13640,"kb_used_omap":12,"kb_used_meta":750131,"kb_avail":754210744,"statfs":{"total":773094113280,"available":772311801856,"internally_reserved":0,"allocated":13967360,"data_stored":7088010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":768134704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.078490"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572572+0000","last_change":"2024-09-19T07:54:23.583787+0000","last_active":"2024-09-19T07:54:24.572572+0000","last_peered":"2024-09-19T07:54:24.572572+0000","last_clean":"2024-09-19T07:54:24.572572+0000","last_became_active":"2024-09-19T07:54:23.583453+0000","last_became_peered":"2024-09-19T07:54:23.583453+0000","last_unstale":"2024-09-19T07:54:24.572572+0000","last_undegraded":"2024-09-19T07:54:24.572572+0000","last_fullsized":"2024-09-19T07:54:24.572572+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:23.821546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579942+0000","last_change":"2024-09-19T07:54:23.576333+0000","last_active":"2024-09-19T07:54:23.579942+0000","last_peered":"2024-09-19T07:54:23.579942+0000","last_clean":"2024-09-19T07:54:23.579942+0000","last_became_active":"2024-09-19T07:54:23.576053+0000","last_became_peered":"2024-09-19T07:54:23.576053+0000","last_unstale":"2024-09-19T07:54:23.579942+0000","last_undegraded":"2024-09-19T07:54:23.579942+0000","last_fullsized":"2024-09-19T07:54:23.579942+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:39.147749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.591937+0000","last_change":"2024-09-19T07:54:23.589816+0000","last_active":"2024-09-19T07:54:23.591937+0000","last_peered":"2024-09-19T07:54:23.591937+0000","last_clean":"2024-09-19T07:54:23.591937+0000","last_became_active":"2024-09-19T07:54:23.589423+0000","last_became_peered":"2024-09-19T07:54:23.589423+0000","last_unstale":"2024-09-19T07:54:23.591937+0000","last_undegraded":"2024-09-19T07:54:23.591937+0000","last_fullsized":"2024-09-19T07:54:23.591937+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:31.699850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.582857+0000","last_change":"2024-09-19T07:54:23.579793+0000","last_active":"2024-09-19T07:54:23.582857+0000","last_peered":"2024-09-19T07:54:23.582857+0000","last_clean":"2024-09-19T07:54:23.582857+0000","last_became_active":"2024-09-19T07:54:23.579502+0000","last_became_peered":"2024-09-19T07:54:23.579502+0000","last_unstale":"2024-09-19T07:54:23.582857+0000","last_undegraded":"2024-09-19T07:54:23.582857+0000","last_fullsized":"2024-09-19T07:54:23.582857+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:32.477007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.551618+0000","last_change":"2024-09-19T07:54:22.551618+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.551618+0000","last_undegraded":"2024-09-19T07:54:22.551618+0000","last_fullsized":"2024-09-19T07:54:22.551618+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.555440+0000","last_change":"2024-09-19T07:54:22.555440+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.555440+0000","last_undegraded":"2024-09-19T07:54:22.555440+0000","last_fullsized":"2024-09-19T07:54:22.555440+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.586469+0000","last_change":"2024-09-19T07:54:23.584389+0000","last_active":"2024-09-19T07:54:23.586469+0000","last_peered":"2024-09-19T07:54:23.586469+0000","last_clean":"2024-09-19T07:54:23.586469+0000","last_became_active":"2024-09-19T07:54:23.584121+0000","last_became_peered":"2024-09-19T07:54:23.584121+0000","last_unstale":"2024-09-19T07:54:23.586469+0000","last_undegraded":"2024-09-19T07:54:23.586469+0000","last_fullsized":"2024-09-19T07:54:23.586469+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:35.419442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585626+0000","last_change":"2024-09-19T07:54:23.584963+0000","last_active":"2024-09-19T07:54:23.585626+0000","last_peered":"2024-09-19T07:54:23.585626+0000","last_clean":"2024-09-19T07:54:23.585626+0000","last_became_active":"2024-09-19T07:54:23.583723+0000","last_became_peered":"2024-09-19T07:54:23.583723+0000","last_unstale":"2024-09-19T07:54:23.585626+0000","last_undegraded":"2024-09-19T07:54:23.585626+0000","last_fullsized":"2024-09-19T07:54:23.585626+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:57.397944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.554930+0000","last_change":"2024-09-19T07:54:22.554930+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.554930+0000","last_undegraded":"2024-09-19T07:54:22.554930+0000","last_fullsized":"2024-09-19T07:54:22.554930+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585515+0000","last_change":"2024-09-19T07:54:23.579169+0000","last_active":"2024-09-19T07:54:23.585515+0000","last_peered":"2024-09-19T07:54:23.585515+0000","last_clean":"2024-09-19T07:54:23.585515+0000","last_became_active":"2024-09-19T07:54:23.578888+0000","last_became_peered":"2024-09-19T07:54:23.578888+0000","last_unstale":"2024-09-19T07:54:23.585515+0000","last_undegraded":"2024-09-19T07:54:23.585515+0000","last_fullsized":"2024-09-19T07:54:23.585515+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:09.226711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585479+0000","last_change":"2024-09-19T07:54:23.584332+0000","last_active":"2024-09-19T07:54:23.585479+0000","last_peered":"2024-09-19T07:54:23.585479+0000","last_clean":"2024-09-19T07:54:23.585479+0000","last_became_active":"2024-09-19T07:54:23.584031+0000","last_became_peered":"2024-09-19T07:54:23.584031+0000","last_unstale":"2024-09-19T07:54:23.585479+0000","last_undegraded":"2024-09-19T07:54:23.585479+0000","last_fullsized":"2024-09-19T07:54:23.585479+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:14.753001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572521+0000","last_change":"2024-09-19T07:54:23.574847+0000","last_active":"2024-09-19T07:54:24.572521+0000","last_peered":"2024-09-19T07:54:24.572521+0000","last_clean":"2024-09-19T07:54:24.572521+0000","last_became_active":"2024-09-19T07:54:23.574423+0000","last_became_peered":"2024-09-19T07:54:23.574423+0000","last_unstale":"2024-09-19T07:54:24.572521+0000","last_undegraded":"2024-09-19T07:54:24.572521+0000","last_fullsized":"2024-09-19T07:54:24.572521+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:35:20.465621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.554415+0000","last_change":"2024-09-19T07:54:22.554415+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.554415+0000","last_undegraded":"2024-09-19T07:54:22.554415+0000","last_fullsized":"2024-09-19T07:54:22.554415+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572583+0000","last_change":"2024-09-19T07:54:23.583768+0000","last_active":"2024-09-19T07:54:24.572583+0000","last_peered":"2024-09-19T07:54:24.572583+0000","last_clean":"2024-09-19T07:54:24.572583+0000","last_became_active":"2024-09-19T07:54:23.579038+0000","last_became_peered":"2024-09-19T07:54:23.579038+0000","last_unstale":"2024-09-19T07:54:24.572583+0000","last_undegraded":"2024-09-19T07:54:24.572583+0000","last_fullsized":"2024-09-19T07:54:24.572583+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:16:28.056548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572220+0000","last_change":"2024-09-19T07:54:23.582464+0000","last_active":"2024-09-19T07:54:24.572220+0000","last_peered":"2024-09-19T07:54:24.572220+0000","last_clean":"2024-09-19T07:54:24.572220+0000","last_became_active":"2024-09-19T07:54:23.582176+0000","last_became_peered":"2024-09-19T07:54:23.582176+0000","last_unstale":"2024-09-19T07:54:24.572220+0000","last_undegraded":"2024-09-19T07:54:24.572220+0000","last_fullsized":"2024-09-19T07:54:24.572220+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:25.500256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.557847+0000","last_change":"2024-09-19T07:54:22.557847+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.557847+0000","last_undegraded":"2024-09-19T07:54:22.557847+0000","last_fullsized":"2024-09-19T07:54:22.557847+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.553868+0000","last_change":"2024-09-19T07:54:22.553868+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.553868+0000","last_undegraded":"2024-09-19T07:54:22.553868+0000","last_fullsized":"2024-09-19T07:54:22.553868+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.555444+0000","last_change":"2024-09-19T07:54:22.555444+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.555444+0000","last_undegraded":"2024-09-19T07:54:22.555444+0000","last_fullsized":"2024-09-19T07:54:22.555444+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.581149+0000","last_change":"2024-09-19T07:54:23.580157+0000","last_active":"2024-09-19T07:54:23.581149+0000","last_peered":"2024-09-19T07:54:23.581149+0000","last_clean":"2024-09-19T07:54:23.581149+0000","last_became_active":"2024-09-19T07:54:23.579109+0000","last_became_peered":"2024-09-19T07:54:23.579109+0000","last_unstale":"2024-09-19T07:54:23.581149+0000","last_undegraded":"2024-09-19T07:54:23.581149+0000","last_fullsized":"2024-09-19T07:54:23.581149+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:17.886276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.552956+0000","last_change":"2024-09-19T07:54:22.552956+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.552956+0000","last_undegraded":"2024-09-19T07:54:22.552956+0000","last_fullsized":"2024-09-19T07:54:22.552956+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.568515+0000","last_change":"2024-09-19T07:54:23.567173+0000","last_active":"2024-09-19T07:54:23.568515+0000","last_peered":"2024-09-19T07:54:23.568515+0000","last_clean":"2024-09-19T07:54:23.568515+0000","last_became_active":"2024-09-19T07:54:23.566866+0000","last_became_peered":"2024-09-19T07:54:23.566866+0000","last_unstale":"2024-09-19T07:54:23.568515+0000","last_undegraded":"2024-09-19T07:54:23.568515+0000","last_fullsized":"2024-09-19T07:54:23.568515+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:14.329860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572411+0000","last_change":"2024-09-19T07:54:23.586941+0000","last_active":"2024-09-19T07:54:24.572411+0000","last_peered":"2024-09-19T07:54:24.572411+0000","last_clean":"2024-09-19T07:54:24.572411+0000","last_became_active":"2024-09-19T07:54:23.586367+0000","last_became_peered":"2024-09-19T07:54:23.586367+0000","last_unstale":"2024-09-19T07:54:24.572411+0000","last_undegraded":"2024-09-19T07:54:24.572411+0000","last_fullsized":"2024-09-19T07:54:24.572411+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:22.782628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579831+0000","last_change":"2024-09-19T07:54:23.575696+0000","last_active":"2024-09-19T07:54:23.579831+0000","last_peered":"2024-09-19T07:54:23.579831+0000","last_clean":"2024-09-19T07:54:23.579831+0000","last_became_active":"2024-09-19T07:54:23.575427+0000","last_became_peered":"2024-09-19T07:54:23.575427+0000","last_unstale":"2024-09-19T07:54:23.579831+0000","last_undegraded":"2024-09-19T07:54:23.579831+0000","last_fullsized":"2024-09-19T07:54:23.579831+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:29.663123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.557404+0000","last_change":"2024-09-19T07:54:22.557404+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.557404+0000","last_undegraded":"2024-09-19T07:54:22.557404+0000","last_fullsized":"2024-09-19T07:54:22.557404+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.589396+0000","last_change":"2024-09-19T07:54:23.584866+0000","last_active":"2024-09-19T07:54:23.589396+0000","last_peered":"2024-09-19T07:54:23.589396+0000","last_clean":"2024-09-19T07:54:23.589396+0000","last_became_active":"2024-09-19T07:54:23.582585+0000","last_became_peered":"2024-09-19T07:54:23.582585+0000","last_unstale":"2024-09-19T07:54:23.589396+0000","last_undegraded":"2024-09-19T07:54:23.589396+0000","last_fullsized":"2024-09-19T07:54:23.589396+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:09.647595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.587463+0000","last_change":"2024-09-19T07:54:23.583732+0000","last_active":"2024-09-19T07:54:23.587463+0000","last_peered":"2024-09-19T07:54:23.587463+0000","last_clean":"2024-09-19T07:54:23.587463+0000","last_became_active":"2024-09-19T07:54:23.583460+0000","last_became_peered":"2024-09-19T07:54:23.583460+0000","last_unstale":"2024-09-19T07:54:23.587463+0000","last_undegraded":"2024-09-19T07:54:23.587463+0000","last_fullsized":"2024-09-19T07:54:23.587463+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:01.094793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.581128+0000","last_change":"2024-09-19T07:54:23.579141+0000","last_active":"2024-09-19T07:54:23.581128+0000","last_peered":"2024-09-19T07:54:23.581128+0000","last_clean":"2024-09-19T07:54:23.581128+0000","last_became_active":"2024-09-19T07:54:23.578856+0000","last_became_peered":"2024-09-19T07:54:23.578856+0000","last_unstale":"2024-09-19T07:54:23.581128+0000","last_undegraded":"2024-09-19T07:54:23.581128+0000","last_fullsized":"2024-09-19T07:54:23.581128+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:21.371456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585833+0000","last_change":"2024-09-19T07:54:23.582767+0000","last_active":"2024-09-19T07:54:23.585833+0000","last_peered":"2024-09-19T07:54:23.585833+0000","last_clean":"2024-09-19T07:54:23.585833+0000","last_became_active":"2024-09-19T07:54:23.582486+0000","last_became_peered":"2024-09-19T07:54:23.582486+0000","last_unstale":"2024-09-19T07:54:23.585833+0000","last_undegraded":"2024-09-19T07:54:23.585833+0000","last_fullsized":"2024-09-19T07:54:23.585833+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:32.949629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.589319+0000","last_change":"2024-09-19T07:54:23.587202+0000","last_active":"2024-09-19T07:54:23.589319+0000","last_peered":"2024-09-19T07:54:23.589319+0000","last_clean":"2024-09-19T07:54:23.589319+0000","last_became_active":"2024-09-19T07:54:23.586940+0000","last_became_peered":"2024-09-19T07:54:23.586940+0000","last_unstale":"2024-09-19T07:54:23.589319+0000","last_undegraded":"2024-09-19T07:54:23.589319+0000","last_fullsized":"2024-09-19T07:54:23.589319+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:41.297635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.583319+0000","last_change":"2024-09-19T07:54:23.581592+0000","last_active":"2024-09-19T07:54:23.583319+0000","last_peered":"2024-09-19T07:54:23.583319+0000","last_clean":"2024-09-19T07:54:23.583319+0000","last_became_active":"2024-09-19T07:54:23.581298+0000","last_became_peered":"2024-09-19T07:54:23.581298+0000","last_unstale":"2024-09-19T07:54:23.583319+0000","last_undegraded":"2024-09-19T07:54:23.583319+0000","last_fullsized":"2024-09-19T07:54:23.583319+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:04.317330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576623+0000","last_change":"2024-09-19T07:54:23.575015+0000","last_active":"2024-09-19T07:54:23.576623+0000","last_peered":"2024-09-19T07:54:23.576623+0000","last_clean":"2024-09-19T07:54:23.576623+0000","last_became_active":"2024-09-19T07:54:23.574695+0000","last_became_peered":"2024-09-19T07:54:23.574695+0000","last_unstale":"2024-09-19T07:54:23.576623+0000","last_undegraded":"2024-09-19T07:54:23.576623+0000","last_fullsized":"2024-09-19T07:54:23.576623+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:07.673263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585765+0000","last_change":"2024-09-19T07:54:23.583307+0000","last_active":"2024-09-19T07:54:23.585765+0000","last_peered":"2024-09-19T07:54:23.585765+0000","last_clean":"2024-09-19T07:54:23.585765+0000","last_became_active":"2024-09-19T07:54:23.583021+0000","last_became_peered":"2024-09-19T07:54:23.583021+0000","last_unstale":"2024-09-19T07:54:23.585765+0000","last_undegraded":"2024-09-19T07:54:23.585765+0000","last_fullsized":"2024-09-19T07:54:23.585765+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:38.791089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579025+0000","last_change":"2024-09-19T07:54:23.576203+0000","last_active":"2024-09-19T07:54:23.579025+0000","last_peered":"2024-09-19T07:54:23.579025+0000","last_clean":"2024-09-19T07:54:23.579025+0000","last_became_active":"2024-09-19T07:54:23.575926+0000","last_became_peered":"2024-09-19T07:54:23.575926+0000","last_unstale":"2024-09-19T07:54:23.579025+0000","last_undegraded":"2024-09-19T07:54:23.579025+0000","last_fullsized":"2024-09-19T07:54:23.579025+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:56.777102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"128'4","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.591733+0000","last_change":"2024-09-19T07:54:22.568159+0000","last_active":"2024-09-19T07:54:23.591733+0000","last_peered":"2024-09-19T07:54:23.591733+0000","last_clean":"2024-09-19T07:54:23.591733+0000","last_became_active":"2024-09-19T07:54:22.567832+0000","last_became_peered":"2024-09-19T07:54:22.567832+0000","last_unstale":"2024-09-19T07:54:23.591733+0000","last_undegraded":"2024-09-19T07:54:23.591733+0000","last_fullsized":"2024-09-19T07:54:23.591733+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:46.972795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572726+0000","last_change":"2024-09-19T07:54:23.585429+0000","last_active":"2024-09-19T07:54:24.572726+0000","last_peered":"2024-09-19T07:54:24.572726+0000","last_clean":"2024-09-19T07:54:24.572726+0000","last_became_active":"2024-09-19T07:54:23.585096+0000","last_became_peered":"2024-09-19T07:54:23.585096+0000","last_unstale":"2024-09-19T07:54:24.572726+0000","last_undegraded":"2024-09-19T07:54:24.572726+0000","last_fullsized":"2024-09-19T07:54:24.572726+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:29.192911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565571+0000","last_change":"2024-09-19T07:54:22.561503+0000","last_active":"2024-09-19T07:54:22.565571+0000","last_peered":"2024-09-19T07:54:22.565571+0000","last_clean":"2024-09-19T07:54:22.565571+0000","last_became_active":"2024-09-19T07:54:22.561219+0000","last_became_peered":"2024-09-19T07:54:22.561219+0000","last_unstale":"2024-09-19T07:54:22.565571+0000","last_undegraded":"2024-09-19T07:54:22.565571+0000","last_fullsized":"2024-09-19T07:54:22.565571+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:18.897568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579803+0000","last_change":"2024-09-19T07:54:23.574954+0000","last_active":"2024-09-19T07:54:23.579803+0000","last_peered":"2024-09-19T07:54:23.579803+0000","last_clean":"2024-09-19T07:54:23.579803+0000","last_became_active":"2024-09-19T07:54:23.574624+0000","last_became_peered":"2024-09-19T07:54:23.574624+0000","last_unstale":"2024-09-19T07:54:23.579803+0000","last_undegraded":"2024-09-19T07:54:23.579803+0000","last_fullsized":"2024-09-19T07:54:23.579803+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:32.420071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"128'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.592293+0000","last_change":"2024-09-19T07:54:22.566018+0000","last_active":"2024-09-19T07:54:23.592293+0000","last_peered":"2024-09-19T07:54:23.592293+0000","last_clean":"2024-09-19T07:54:23.592293+0000","last_became_active":"2024-09-19T07:54:22.565812+0000","last_became_peered":"2024-09-19T07:54:22.565812+0000","last_unstale":"2024-09-19T07:54:23.592293+0000","last_undegraded":"2024-09-19T07:54:23.592293+0000","last_fullsized":"2024-09-19T07:54:23.592293+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:01.728173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572245+0000","last_change":"2024-09-19T07:54:23.582120+0000","last_active":"2024-09-19T07:54:24.572245+0000","last_peered":"2024-09-19T07:54:24.572245+0000","last_clean":"2024-09-19T07:54:24.572245+0000","last_became_active":"2024-09-19T07:54:23.581830+0000","last_became_peered":"2024-09-19T07:54:23.581830+0000","last_unstale":"2024-09-19T07:54:24.572245+0000","last_undegraded":"2024-09-19T07:54:24.572245+0000","last_fullsized":"2024-09-19T07:54:24.572245+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:12.630683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"128'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.591308+0000","last_change":"2024-09-19T07:54:22.560370+0000","last_active":"2024-09-19T07:54:23.591308+0000","last_peered":"2024-09-19T07:54:23.591308+0000","last_clean":"2024-09-19T07:54:23.591308+0000","last_became_active":"2024-09-19T07:54:22.560083+0000","last_became_peered":"2024-09-19T07:54:22.560083+0000","last_unstale":"2024-09-19T07:54:23.591308+0000","last_undegraded":"2024-09-19T07:54:23.591308+0000","last_fullsized":"2024-09-19T07:54:23.591308+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:51.100115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.579407+0000","last_change":"2024-09-19T07:54:23.577913+0000","last_active":"2024-09-19T07:54:23.579407+0000","last_peered":"2024-09-19T07:54:23.579407+0000","last_clean":"2024-09-19T07:54:23.579407+0000","last_became_active":"2024-09-19T07:54:23.577102+0000","last_became_peered":"2024-09-19T07:54:23.577102+0000","last_unstale":"2024-09-19T07:54:23.579407+0000","last_undegraded":"2024-09-19T07:54:23.579407+0000","last_fullsized":"2024-09-19T07:54:23.579407+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:49.188816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"128'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.586957+0000","last_change":"2024-09-19T07:54:22.554611+0000","last_active":"2024-09-19T07:54:23.586957+0000","last_peered":"2024-09-19T07:54:23.586957+0000","last_clean":"2024-09-19T07:54:23.586957+0000","last_became_active":"2024-09-19T07:54:22.554332+0000","last_became_peered":"2024-09-19T07:54:22.554332+0000","last_unstale":"2024-09-19T07:54:23.586957+0000","last_undegraded":"2024-09-19T07:54:23.586957+0000","last_fullsized":"2024-09-19T07:54:23.586957+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:09.508289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576791+0000","last_change":"2024-09-19T07:54:23.574521+0000","last_active":"2024-09-19T07:54:23.576791+0000","last_peered":"2024-09-19T07:54:23.576791+0000","last_clean":"2024-09-19T07:54:23.576791+0000","last_became_active":"2024-09-19T07:54:23.574251+0000","last_became_peered":"2024-09-19T07:54:23.574251+0000","last_unstale":"2024-09-19T07:54:23.576791+0000","last_undegraded":"2024-09-19T07:54:23.576791+0000","last_fullsized":"2024-09-19T07:54:23.576791+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:31.699850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565667+0000","last_change":"2024-09-19T07:54:22.564044+0000","last_active":"2024-09-19T07:54:22.565667+0000","last_peered":"2024-09-19T07:54:22.565667+0000","last_clean":"2024-09-19T07:54:22.565667+0000","last_became_active":"2024-09-19T07:54:22.562851+0000","last_became_peered":"2024-09-19T07:54:22.562851+0000","last_unstale":"2024-09-19T07:54:22.565667+0000","last_undegraded":"2024-09-19T07:54:22.565667+0000","last_fullsized":"2024-09-19T07:54:22.565667+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:34.414794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.555018+0000","last_change":"2024-09-19T07:54:22.555018+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.555018+0000","last_undegraded":"2024-09-19T07:54:22.555018+0000","last_fullsized":"2024-09-19T07:54:22.555018+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.566646+0000","last_change":"2024-09-19T07:54:22.557995+0000","last_active":"2024-09-19T07:54:22.566646+0000","last_peered":"2024-09-19T07:54:22.566646+0000","last_clean":"2024-09-19T07:54:22.566646+0000","last_became_active":"2024-09-19T07:54:22.557692+0000","last_became_peered":"2024-09-19T07:54:22.557692+0000","last_unstale":"2024-09-19T07:54:22.566646+0000","last_undegraded":"2024-09-19T07:54:22.566646+0000","last_fullsized":"2024-09-19T07:54:22.566646+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:01.620005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576699+0000","last_change":"2024-09-19T07:54:23.573943+0000","last_active":"2024-09-19T07:54:23.576699+0000","last_peered":"2024-09-19T07:54:23.576699+0000","last_clean":"2024-09-19T07:54:23.576699+0000","last_became_active":"2024-09-19T07:54:23.573668+0000","last_became_peered":"2024-09-19T07:54:23.573668+0000","last_unstale":"2024-09-19T07:54:23.576699+0000","last_undegraded":"2024-09-19T07:54:23.576699+0000","last_fullsized":"2024-09-19T07:54:23.576699+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:13:01.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.566981+0000","last_change":"2024-09-19T07:54:22.562903+0000","last_active":"2024-09-19T07:54:23.566981+0000","last_peered":"2024-09-19T07:54:23.566981+0000","last_clean":"2024-09-19T07:54:23.566981+0000","last_became_active":"2024-09-19T07:54:22.562599+0000","last_became_peered":"2024-09-19T07:54:22.562599+0000","last_unstale":"2024-09-19T07:54:23.566981+0000","last_undegraded":"2024-09-19T07:54:23.566981+0000","last_fullsized":"2024-09-19T07:54:23.566981+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:44.078560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572071+0000","last_change":"2024-09-19T07:54:23.586564+0000","last_active":"2024-09-19T07:54:24.572071+0000","last_peered":"2024-09-19T07:54:24.572071+0000","last_clean":"2024-09-19T07:54:24.572071+0000","last_became_active":"2024-09-19T07:54:23.586246+0000","last_became_peered":"2024-09-19T07:54:23.586246+0000","last_unstale":"2024-09-19T07:54:24.572071+0000","last_undegraded":"2024-09-19T07:54:24.572071+0000","last_fullsized":"2024-09-19T07:54:24.572071+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:14:59.560377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.563273+0000","last_change":"2024-09-19T07:54:22.556612+0000","last_active":"2024-09-19T07:54:23.563273+0000","last_peered":"2024-09-19T07:54:23.563273+0000","last_clean":"2024-09-19T07:54:23.563273+0000","last_became_active":"2024-09-19T07:54:22.556177+0000","last_became_peered":"2024-09-19T07:54:22.556177+0000","last_unstale":"2024-09-19T07:54:23.563273+0000","last_undegraded":"2024-09-19T07:54:23.563273+0000","last_fullsized":"2024-09-19T07:54:23.563273+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:25.294939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.556350+0000","last_change":"2024-09-19T07:54:22.556350+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.556350+0000","last_undegraded":"2024-09-19T07:54:22.556350+0000","last_fullsized":"2024-09-19T07:54:22.556350+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572966+0000","last_change":"2024-09-19T07:54:22.564375+0000","last_active":"2024-09-19T07:54:24.572966+0000","last_peered":"2024-09-19T07:54:24.572966+0000","last_clean":"2024-09-19T07:54:24.572966+0000","last_became_active":"2024-09-19T07:54:22.564096+0000","last_became_peered":"2024-09-19T07:54:22.564096+0000","last_unstale":"2024-09-19T07:54:24.572966+0000","last_undegraded":"2024-09-19T07:54:24.572966+0000","last_fullsized":"2024-09-19T07:54:24.572966+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:05.452375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.584721+0000","last_change":"2024-09-19T07:54:23.583678+0000","last_active":"2024-09-19T07:54:23.584721+0000","last_peered":"2024-09-19T07:54:23.584721+0000","last_clean":"2024-09-19T07:54:23.584721+0000","last_became_active":"2024-09-19T07:54:23.583254+0000","last_became_peered":"2024-09-19T07:54:23.583254+0000","last_unstale":"2024-09-19T07:54:23.584721+0000","last_undegraded":"2024-09-19T07:54:23.584721+0000","last_fullsized":"2024-09-19T07:54:23.584721+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:46.495632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"128'3","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.593450+0000","last_change":"2024-09-19T07:54:22.558733+0000","last_active":"2024-09-19T07:54:23.593450+0000","last_peered":"2024-09-19T07:54:23.593450+0000","last_clean":"2024-09-19T07:54:23.593450+0000","last_became_active":"2024-09-19T07:54:22.558481+0000","last_became_peered":"2024-09-19T07:54:22.558481+0000","last_unstale":"2024-09-19T07:54:23.593450+0000","last_undegraded":"2024-09-19T07:54:23.593450+0000","last_fullsized":"2024-09-19T07:54:23.593450+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:41.801926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.590832+0000","last_change":"2024-09-19T07:54:23.589083+0000","last_active":"2024-09-19T07:54:23.590832+0000","last_peered":"2024-09-19T07:54:23.590832+0000","last_clean":"2024-09-19T07:54:23.590832+0000","last_became_active":"2024-09-19T07:54:23.588728+0000","last_became_peered":"2024-09-19T07:54:23.588728+0000","last_unstale":"2024-09-19T07:54:23.590832+0000","last_undegraded":"2024-09-19T07:54:23.590832+0000","last_fullsized":"2024-09-19T07:54:23.590832+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:44.660536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"128'6","reported_seq":18,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.594644+0000","last_change":"2024-09-19T07:54:22.564291+0000","last_active":"2024-09-19T07:54:23.594644+0000","last_peered":"2024-09-19T07:54:23.594644+0000","last_clean":"2024-09-19T07:54:23.594644+0000","last_became_active":"2024-09-19T07:54:22.564012+0000","last_became_peered":"2024-09-19T07:54:22.564012+0000","last_unstale":"2024-09-19T07:54:23.594644+0000","last_undegraded":"2024-09-19T07:54:23.594644+0000","last_fullsized":"2024-09-19T07:54:23.594644+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:00.025001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.582649+0000","last_change":"2024-09-19T07:54:23.578794+0000","last_active":"2024-09-19T07:54:23.582649+0000","last_peered":"2024-09-19T07:54:23.582649+0000","last_clean":"2024-09-19T07:54:23.582649+0000","last_became_active":"2024-09-19T07:54:23.578465+0000","last_became_peered":"2024-09-19T07:54:23.578465+0000","last_unstale":"2024-09-19T07:54:23.582649+0000","last_undegraded":"2024-09-19T07:54:23.582649+0000","last_fullsized":"2024-09-19T07:54:23.582649+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:57.295016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"128'2","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572360+0000","last_change":"2024-09-19T07:54:22.565043+0000","last_active":"2024-09-19T07:54:24.572360+0000","last_peered":"2024-09-19T07:54:24.572360+0000","last_clean":"2024-09-19T07:54:24.572360+0000","last_became_active":"2024-09-19T07:54:22.564758+0000","last_became_peered":"2024-09-19T07:54:22.564758+0000","last_unstale":"2024-09-19T07:54:24.572360+0000","last_undegraded":"2024-09-19T07:54:24.572360+0000","last_fullsized":"2024-09-19T07:54:24.572360+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:58:28.017221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576888+0000","last_change":"2024-09-19T07:54:23.574862+0000","last_active":"2024-09-19T07:54:23.576888+0000","last_peered":"2024-09-19T07:54:23.576888+0000","last_clean":"2024-09-19T07:54:23.576888+0000","last_became_active":"2024-09-19T07:54:23.574431+0000","last_became_peered":"2024-09-19T07:54:23.574431+0000","last_unstale":"2024-09-19T07:54:23.576888+0000","last_undegraded":"2024-09-19T07:54:23.576888+0000","last_fullsized":"2024-09-19T07:54:23.576888+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:29.913764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.567482+0000","last_change":"2024-09-19T07:54:22.559040+0000","last_active":"2024-09-19T07:54:23.567482+0000","last_peered":"2024-09-19T07:54:23.567482+0000","last_clean":"2024-09-19T07:54:23.567482+0000","last_became_active":"2024-09-19T07:54:22.558757+0000","last_became_peered":"2024-09-19T07:54:22.558757+0000","last_unstale":"2024-09-19T07:54:23.567482+0000","last_undegraded":"2024-09-19T07:54:23.567482+0000","last_fullsized":"2024-09-19T07:54:23.567482+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:28:24.370408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.551690+0000","last_change":"2024-09-19T07:54:22.551690+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.551690+0000","last_undegraded":"2024-09-19T07:54:22.551690+0000","last_fullsized":"2024-09-19T07:54:22.551690+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565358+0000","last_change":"2024-09-19T07:54:22.559203+0000","last_active":"2024-09-19T07:54:22.565358+0000","last_peered":"2024-09-19T07:54:22.565358+0000","last_clean":"2024-09-19T07:54:22.565358+0000","last_became_active":"2024-09-19T07:54:22.558920+0000","last_became_peered":"2024-09-19T07:54:22.558920+0000","last_unstale":"2024-09-19T07:54:22.565358+0000","last_undegraded":"2024-09-19T07:54:22.565358+0000","last_fullsized":"2024-09-19T07:54:22.565358+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:43.655888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.589459+0000","last_change":"2024-09-19T07:54:23.587980+0000","last_active":"2024-09-19T07:54:23.589459+0000","last_peered":"2024-09-19T07:54:23.589459+0000","last_clean":"2024-09-19T07:54:23.589459+0000","last_became_active":"2024-09-19T07:54:23.587702+0000","last_became_peered":"2024-09-19T07:54:23.587702+0000","last_unstale":"2024-09-19T07:54:23.589459+0000","last_undegraded":"2024-09-19T07:54:23.589459+0000","last_fullsized":"2024-09-19T07:54:23.589459+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:02.624652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"128'1","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.578744+0000","last_change":"2024-09-19T07:54:22.553965+0000","last_active":"2024-09-19T07:54:23.578744+0000","last_peered":"2024-09-19T07:54:23.578744+0000","last_clean":"2024-09-19T07:54:23.578744+0000","last_became_active":"2024-09-19T07:54:22.553678+0000","last_became_peered":"2024-09-19T07:54:22.553678+0000","last_unstale":"2024-09-19T07:54:23.578744+0000","last_undegraded":"2024-09-19T07:54:23.578744+0000","last_fullsized":"2024-09-19T07:54:23.578744+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:06.364113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.576816+0000","last_change":"2024-09-19T07:54:23.574769+0000","last_active":"2024-09-19T07:54:23.576816+0000","last_peered":"2024-09-19T07:54:23.576816+0000","last_clean":"2024-09-19T07:54:23.576816+0000","last_became_active":"2024-09-19T07:54:23.574488+0000","last_became_peered":"2024-09-19T07:54:23.574488+0000","last_unstale":"2024-09-19T07:54:23.576816+0000","last_undegraded":"2024-09-19T07:54:23.576816+0000","last_fullsized":"2024-09-19T07:54:23.576816+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:41.287209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"128'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.591321+0000","last_change":"2024-09-19T07:54:22.567957+0000","last_active":"2024-09-19T07:54:23.591321+0000","last_peered":"2024-09-19T07:54:23.591321+0000","last_clean":"2024-09-19T07:54:23.591321+0000","last_became_active":"2024-09-19T07:54:22.567503+0000","last_became_peered":"2024-09-19T07:54:22.567503+0000","last_unstale":"2024-09-19T07:54:23.591321+0000","last_undegraded":"2024-09-19T07:54:23.591321+0000","last_fullsized":"2024-09-19T07:54:23.591321+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:06.620754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.580616+0000","last_change":"2024-09-19T07:54:23.578211+0000","last_active":"2024-09-19T07:54:23.580616+0000","last_peered":"2024-09-19T07:54:23.580616+0000","last_clean":"2024-09-19T07:54:23.580616+0000","last_became_active":"2024-09-19T07:54:23.577649+0000","last_became_peered":"2024-09-19T07:54:23.577649+0000","last_unstale":"2024-09-19T07:54:23.580616+0000","last_undegraded":"2024-09-19T07:54:23.580616+0000","last_fullsized":"2024-09-19T07:54:23.580616+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:19.902216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565179+0000","last_change":"2024-09-19T07:54:22.559404+0000","last_active":"2024-09-19T07:54:22.565179+0000","last_peered":"2024-09-19T07:54:22.565179+0000","last_clean":"2024-09-19T07:54:22.565179+0000","last_became_active":"2024-09-19T07:54:22.559101+0000","last_became_peered":"2024-09-19T07:54:22.559101+0000","last_unstale":"2024-09-19T07:54:22.565179+0000","last_undegraded":"2024-09-19T07:54:22.565179+0000","last_fullsized":"2024-09-19T07:54:22.565179+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:30.695202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.587444+0000","last_change":"2024-09-19T07:54:23.585316+0000","last_active":"2024-09-19T07:54:23.587444+0000","last_peered":"2024-09-19T07:54:23.587444+0000","last_clean":"2024-09-19T07:54:23.587444+0000","last_became_active":"2024-09-19T07:54:23.585029+0000","last_became_peered":"2024-09-19T07:54:23.585029+0000","last_unstale":"2024-09-19T07:54:23.587444+0000","last_undegraded":"2024-09-19T07:54:23.587444+0000","last_fullsized":"2024-09-19T07:54:23.587444+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:35.538510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565841+0000","last_change":"2024-09-19T07:54:22.563785+0000","last_active":"2024-09-19T07:54:22.565841+0000","last_peered":"2024-09-19T07:54:22.565841+0000","last_clean":"2024-09-19T07:54:22.565841+0000","last_became_active":"2024-09-19T07:54:22.563471+0000","last_became_peered":"2024-09-19T07:54:22.563471+0000","last_unstale":"2024-09-19T07:54:22.565841+0000","last_undegraded":"2024-09-19T07:54:22.565841+0000","last_fullsized":"2024-09-19T07:54:22.565841+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:16.881628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.552495+0000","last_change":"2024-09-19T07:54:22.552495+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.552495+0000","last_undegraded":"2024-09-19T07:54:22.552495+0000","last_fullsized":"2024-09-19T07:54:22.552495+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"128'2","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.586262+0000","last_change":"2024-09-19T07:54:22.556136+0000","last_active":"2024-09-19T07:54:23.586262+0000","last_peered":"2024-09-19T07:54:23.586262+0000","last_clean":"2024-09-19T07:54:23.586262+0000","last_became_active":"2024-09-19T07:54:22.555829+0000","last_became_peered":"2024-09-19T07:54:22.555829+0000","last_unstale":"2024-09-19T07:54:23.586262+0000","last_undegraded":"2024-09-19T07:54:23.586262+0000","last_fullsized":"2024-09-19T07:54:23.586262+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:37.052983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572425+0000","last_change":"2024-09-19T07:54:23.582629+0000","last_active":"2024-09-19T07:54:24.572425+0000","last_peered":"2024-09-19T07:54:24.572425+0000","last_clean":"2024-09-19T07:54:24.572425+0000","last_became_active":"2024-09-19T07:54:23.582343+0000","last_became_peered":"2024-09-19T07:54:23.582343+0000","last_unstale":"2024-09-19T07:54:24.572425+0000","last_undegraded":"2024-09-19T07:54:24.572425+0000","last_fullsized":"2024-09-19T07:54:24.572425+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:38.135921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572535+0000","last_change":"2024-09-19T07:54:22.563856+0000","last_active":"2024-09-19T07:54:24.572535+0000","last_peered":"2024-09-19T07:54:24.572535+0000","last_clean":"2024-09-19T07:54:24.572535+0000","last_became_active":"2024-09-19T07:54:22.563515+0000","last_became_peered":"2024-09-19T07:54:22.563515+0000","last_unstale":"2024-09-19T07:54:24.572535+0000","last_undegraded":"2024-09-19T07:54:24.572535+0000","last_fullsized":"2024-09-19T07:54:24.572535+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:32.880703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.554597+0000","last_change":"2024-09-19T07:54:22.554597+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.554597+0000","last_undegraded":"2024-09-19T07:54:22.554597+0000","last_fullsized":"2024-09-19T07:54:22.554597+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.568962+0000","last_change":"2024-09-19T07:54:22.564048+0000","last_active":"2024-09-19T07:54:23.568962+0000","last_peered":"2024-09-19T07:54:23.568962+0000","last_clean":"2024-09-19T07:54:23.568962+0000","last_became_active":"2024-09-19T07:54:22.563775+0000","last_became_peered":"2024-09-19T07:54:22.563775+0000","last_unstale":"2024-09-19T07:54:23.568962+0000","last_undegraded":"2024-09-19T07:54:23.568962+0000","last_fullsized":"2024-09-19T07:54:23.568962+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:57.190350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585516+0000","last_change":"2024-09-19T07:54:23.584566+0000","last_active":"2024-09-19T07:54:23.585516+0000","last_peered":"2024-09-19T07:54:23.585516+0000","last_clean":"2024-09-19T07:54:23.585516+0000","last_became_active":"2024-09-19T07:54:23.584002+0000","last_became_peered":"2024-09-19T07:54:23.584002+0000","last_unstale":"2024-09-19T07:54:23.585516+0000","last_undegraded":"2024-09-19T07:54:23.585516+0000","last_fullsized":"2024-09-19T07:54:23.585516+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:58.194998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565335+0000","last_change":"2024-09-19T07:54:22.558583+0000","last_active":"2024-09-19T07:54:22.565335+0000","last_peered":"2024-09-19T07:54:22.565335+0000","last_clean":"2024-09-19T07:54:22.565335+0000","last_became_active":"2024-09-19T07:54:22.558239+0000","last_became_peered":"2024-09-19T07:54:22.558239+0000","last_unstale":"2024-09-19T07:54:22.565335+0000","last_undegraded":"2024-09-19T07:54:22.565335+0000","last_fullsized":"2024-09-19T07:54:22.565335+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:37.052983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.553358+0000","last_change":"2024-09-19T07:54:22.553358+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.553358+0000","last_undegraded":"2024-09-19T07:54:22.553358+0000","last_fullsized":"2024-09-19T07:54:22.553358+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"128'2","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572673+0000","last_change":"2024-09-19T07:54:22.561535+0000","last_active":"2024-09-19T07:54:24.572673+0000","last_peered":"2024-09-19T07:54:24.572673+0000","last_clean":"2024-09-19T07:54:24.572673+0000","last_became_active":"2024-09-19T07:54:22.561210+0000","last_became_peered":"2024-09-19T07:54:22.561210+0000","last_unstale":"2024-09-19T07:54:24.572673+0000","last_undegraded":"2024-09-19T07:54:24.572673+0000","last_fullsized":"2024-09-19T07:54:24.572673+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:13.281363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.590668+0000","last_change":"2024-09-19T07:54:23.588433+0000","last_active":"2024-09-19T07:54:23.590668+0000","last_peered":"2024-09-19T07:54:23.590668+0000","last_clean":"2024-09-19T07:54:23.590668+0000","last_became_active":"2024-09-19T07:54:23.587803+0000","last_became_peered":"2024-09-19T07:54:23.587803+0000","last_unstale":"2024-09-19T07:54:23.590668+0000","last_undegraded":"2024-09-19T07:54:23.590668+0000","last_fullsized":"2024-09-19T07:54:23.590668+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:10.512937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.562219+0000","last_change":"2024-09-19T07:54:22.555487+0000","last_active":"2024-09-19T07:54:23.562219+0000","last_peered":"2024-09-19T07:54:23.562219+0000","last_clean":"2024-09-19T07:54:23.562219+0000","last_became_active":"2024-09-19T07:54:22.555215+0000","last_became_peered":"2024-09-19T07:54:22.555215+0000","last_unstale":"2024-09-19T07:54:23.562219+0000","last_undegraded":"2024-09-19T07:54:23.562219+0000","last_fullsized":"2024-09-19T07:54:23.562219+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:01.620005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.586495+0000","last_change":"2024-09-19T07:54:23.585093+0000","last_active":"2024-09-19T07:54:23.586495+0000","last_peered":"2024-09-19T07:54:23.586495+0000","last_clean":"2024-09-19T07:54:23.586495+0000","last_became_active":"2024-09-19T07:54:23.584837+0000","last_became_peered":"2024-09-19T07:54:23.584837+0000","last_unstale":"2024-09-19T07:54:23.586495+0000","last_undegraded":"2024-09-19T07:54:23.586495+0000","last_fullsized":"2024-09-19T07:54:23.586495+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:43.491587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.565590+0000","last_change":"2024-09-19T07:54:22.564147+0000","last_active":"2024-09-19T07:54:23.565590+0000","last_peered":"2024-09-19T07:54:23.565590+0000","last_clean":"2024-09-19T07:54:23.565590+0000","last_became_active":"2024-09-19T07:54:22.563844+0000","last_became_peered":"2024-09-19T07:54:22.563844+0000","last_unstale":"2024-09-19T07:54:23.565590+0000","last_undegraded":"2024-09-19T07:54:23.565590+0000","last_fullsized":"2024-09-19T07:54:23.565590+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:43.655888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.585486+0000","last_change":"2024-09-19T07:54:23.582012+0000","last_active":"2024-09-19T07:54:23.585486+0000","last_peered":"2024-09-19T07:54:23.585486+0000","last_clean":"2024-09-19T07:54:23.585486+0000","last_became_active":"2024-09-19T07:54:23.581143+0000","last_became_peered":"2024-09-19T07:54:23.581143+0000","last_unstale":"2024-09-19T07:54:23.585486+0000","last_undegraded":"2024-09-19T07:54:23.585486+0000","last_fullsized":"2024-09-19T07:54:23.585486+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:20.783132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565313+0000","last_change":"2024-09-19T07:54:22.558393+0000","last_active":"2024-09-19T07:54:22.565313+0000","last_peered":"2024-09-19T07:54:22.565313+0000","last_clean":"2024-09-19T07:54:22.565313+0000","last_became_active":"2024-09-19T07:54:22.558091+0000","last_became_peered":"2024-09-19T07:54:22.558091+0000","last_unstale":"2024-09-19T07:54:22.565313+0000","last_undegraded":"2024-09-19T07:54:22.565313+0000","last_fullsized":"2024-09-19T07:54:22.565313+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:09.508289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.549695+0000","last_change":"2024-09-19T07:54:22.549695+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.549695+0000","last_undegraded":"2024-09-19T07:54:22.549695+0000","last_fullsized":"2024-09-19T07:54:22.549695+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.562667+0000","last_change":"2024-09-19T07:54:22.567926+0000","last_active":"2024-09-19T07:54:23.562667+0000","last_peered":"2024-09-19T07:54:23.562667+0000","last_clean":"2024-09-19T07:54:23.562667+0000","last_became_active":"2024-09-19T07:54:22.567617+0000","last_became_peered":"2024-09-19T07:54:22.567617+0000","last_unstale":"2024-09-19T07:54:23.562667+0000","last_undegraded":"2024-09-19T07:54:23.562667+0000","last_fullsized":"2024-09-19T07:54:23.562667+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:41:15.024194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.554395+0000","last_change":"2024-09-19T07:54:22.554395+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.554395+0000","last_undegraded":"2024-09-19T07:54:22.554395+0000","last_fullsized":"2024-09-19T07:54:22.554395+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.570070+0000","last_change":"2024-09-19T07:54:22.558104+0000","last_active":"2024-09-19T07:54:23.570070+0000","last_peered":"2024-09-19T07:54:23.570070+0000","last_clean":"2024-09-19T07:54:23.570070+0000","last_became_active":"2024-09-19T07:54:22.557825+0000","last_became_peered":"2024-09-19T07:54:22.557825+0000","last_unstale":"2024-09-19T07:54:23.570070+0000","last_undegraded":"2024-09-19T07:54:23.570070+0000","last_fullsized":"2024-09-19T07:54:23.570070+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:36:36.185111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":127,"state":"creating+peering","last_fresh":"2024-09-19T07:54:22.553286+0000","last_change":"2024-09-19T07:54:22.553286+0000","last_active":"2024-09-19T07:54:22.540405+0000","last_peered":"2024-09-19T07:54:22.540405+0000","last_clean":"2024-09-19T07:54:22.540405+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:54:22.553286+0000","last_undegraded":"2024-09-19T07:54:22.553286+0000","last_fullsized":"2024-09-19T07:54:22.553286+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.566668+0000","last_change":"2024-09-19T07:54:22.560118+0000","last_active":"2024-09-19T07:54:22.566668+0000","last_peered":"2024-09-19T07:54:22.566668+0000","last_clean":"2024-09-19T07:54:22.566668+0000","last_became_active":"2024-09-19T07:54:22.559838+0000","last_became_peered":"2024-09-19T07:54:22.559838+0000","last_unstale":"2024-09-19T07:54:22.566668+0000","last_undegraded":"2024-09-19T07:54:22.566668+0000","last_fullsized":"2024-09-19T07:54:22.566668+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:06.668616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.587234+0000","last_change":"2024-09-19T07:54:23.586194+0000","last_active":"2024-09-19T07:54:23.587234+0000","last_peered":"2024-09-19T07:54:23.587234+0000","last_clean":"2024-09-19T07:54:23.587234+0000","last_became_active":"2024-09-19T07:54:23.585946+0000","last_became_peered":"2024-09-19T07:54:23.585946+0000","last_unstale":"2024-09-19T07:54:23.587234+0000","last_undegraded":"2024-09-19T07:54:23.587234+0000","last_fullsized":"2024-09-19T07:54:23.587234+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:27.045701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-19T07:54:22.565875+0000","last_change":"2024-09-19T07:54:22.562538+0000","last_active":"2024-09-19T07:54:22.565875+0000","last_peered":"2024-09-19T07:54:22.565875+0000","last_clean":"2024-09-19T07:54:22.565875+0000","last_became_active":"2024-09-19T07:54:22.562257+0000","last_became_peered":"2024-09-19T07:54:22.562257+0000","last_unstale":"2024-09-19T07:54:22.565875+0000","last_undegraded":"2024-09-19T07:54:22.565875+0000","last_fullsized":"2024-09-19T07:54:22.565875+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:20.366808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.590798+0000","last_change":"2024-09-19T07:54:23.588887+0000","last_active":"2024-09-19T07:54:23.590798+0000","last_peered":"2024-09-19T07:54:23.590798+0000","last_clean":"2024-09-19T07:54:23.590798+0000","last_became_active":"2024-09-19T07:54:23.588623+0000","last_became_peered":"2024-09-19T07:54:23.588623+0000","last_unstale":"2024-09-19T07:54:23.590798+0000","last_undegraded":"2024-09-19T07:54:23.590798+0000","last_fullsized":"2024-09-19T07:54:23.590798+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:38.057631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.569582+0000","last_change":"2024-09-19T07:54:22.565171+0000","last_active":"2024-09-19T07:54:23.569582+0000","last_peered":"2024-09-19T07:54:23.569582+0000","last_clean":"2024-09-19T07:54:23.569582+0000","last_became_active":"2024-09-19T07:54:22.564876+0000","last_became_peered":"2024-09-19T07:54:22.564876+0000","last_unstale":"2024-09-19T07:54:23.569582+0000","last_undegraded":"2024-09-19T07:54:23.569582+0000","last_fullsized":"2024-09-19T07:54:23.569582+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:36.291156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":364,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-19T07:54:23.567647+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:54:23.567647+0000","last_peered":"2024-09-19T07:54:23.567647+0000","last_clean":"2024-09-19T07:54:23.567647+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:54:23.567647+0000","last_undegraded":"2024-09-19T07:54:23.567647+0000","last_fullsized":"2024-09-19T07:54:23.567647+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509581,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83268,"kb_used_data":2152,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288572,"statfs":{"total":96636764160,"available":96551497728,"internally_reserved":0,"allocated":2203648,"data_stored":1331051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509580,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104952,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266888,"statfs":{"total":96636764160,"available":96529293312,"internally_reserved":0,"allocated":1601536,"data_stored":739653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509581,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72944,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298896,"statfs":{"total":96636764160,"available":96562069504,"internally_reserved":0,"allocated":1597440,"data_stored":739003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509581,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86832,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285008,"statfs":{"total":96636764160,"available":96547848192,"internally_reserved":0,"allocated":1593344,"data_stored":738981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509580,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105460,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266380,"statfs":{"total":96636764160,"available":96528773120,"internally_reserved":0,"allocated":2195456,"data_stored":1329405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542284,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118748,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253092,"statfs":{"total":96636764160,"available":96515166208,"internally_reserved":0,"allocated":1581056,"data_stored":731814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542284,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82720,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94289120,"statfs":{"total":96636764160,"available":96552058880,"internally_reserved":0,"allocated":1581056,"data_stored":731814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574990,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109052,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262788,"statfs":{"total":96636764160,"available":96525094912,"internally_reserved":0,"allocated":1613824,"data_stored":746289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:54:30.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:54:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.091+0000 7f1979186640 1 Processor -- start 2024-09-19T07:54:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.091+0000 7f1979186640 1 -- start start 2024-09-19T07:54:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.092+0000 7f1979186640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19740d0070 0x7f197415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.092+0000 7f1979186640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19740d4130 0x7f19740d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.092+0000 7f1979186640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19740d4ae0 con 0x7f197415f0d0 2024-09-19T07:54:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.092+0000 7f1979186640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f19740d1b10 con 0x7f19740d0070 2024-09-19T07:54:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.092+0000 7f1979186640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f19740cf2b0 con 0x7f19740d4130 2024-09-19T07:54:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.092+0000 7f197259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19740d0070 0x7f197415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.092+0000 7f197259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19740d0070 0x7f197415f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48488/0 (socket says 172.21.15.16:48488) 2024-09-19T07:54:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.092+0000 7f197259c640 1 -- 172.21.15.16:0/2876459428 learned_addr learned my addr 172.21.15.16:0/2876459428 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.093+0000 7f197359e640 1 --2- 172.21.15.16:0/2876459428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19740d4130 0x7f19740d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.093+0000 7f197259c640 1 -- 172.21.15.16:0/2876459428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19740d4130 msgr2=0x7f19740d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.093+0000 7f197259c640 1 --2- 172.21.15.16:0/2876459428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19740d4130 0x7f19740d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.093+0000 7f197259c640 1 -- 172.21.15.16:0/2876459428 >> v1:172.21.15.146:6789/0 conn(0x7f197415f0d0 legacy=0x7f19740d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.093+0000 7f197259c640 1 -- 172.21.15.16:0/2876459428 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19740cf590 con 0x7f19740d0070 2024-09-19T07:54:31.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.094+0000 7f197259c640 1 --2- 172.21.15.16:0/2876459428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19740d0070 0x7f197415f4b0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f1968009870 tx=0x7f196802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db36c39263f77b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:31.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.094+0000 7f1971d9b640 1 -- 172.21.15.16:0/2876459428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968037070 con 0x7f19740d0070 2024-09-19T07:54:31.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.094+0000 7f1971d9b640 1 -- 172.21.15.16:0/2876459428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1968032440 con 0x7f19740d0070 2024-09-19T07:54:31.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.094+0000 7f1971d9b640 1 -- 172.21.15.16:0/2876459428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968032810 con 0x7f19740d0070 2024-09-19T07:54:31.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.094+0000 7f1979186640 1 -- 172.21.15.16:0/2876459428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19740d0070 msgr2=0x7f197415f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:31.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.094+0000 7f1979186640 1 --2- 172.21.15.16:0/2876459428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19740d0070 0x7f197415f4b0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f1968009870 tx=0x7f196802a180 comp rx=0 tx=0).stop 2024-09-19T07:54:31.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.094+0000 7f1979186640 1 -- 172.21.15.16:0/2876459428 shutdown_connections 2024-09-19T07:54:31.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.094+0000 7f1979186640 1 --2- 172.21.15.16:0/2876459428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19740d4130 0x7f19740d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.094+0000 7f1979186640 1 --2- 172.21.15.16:0/2876459428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19740d0070 0x7f197415f4b0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.094+0000 7f1979186640 1 -- 172.21.15.16:0/2876459428 >> 172.21.15.16:0/2876459428 conn(0x7f197409d760 msgr2=0x7f197408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:31.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.094+0000 7f1979186640 1 -- 172.21.15.16:0/2876459428 shutdown_connections 2024-09-19T07:54:31.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.094+0000 7f1979186640 1 -- 172.21.15.16:0/2876459428 wait complete. 2024-09-19T07:54:31.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.096+0000 7f1979186640 1 Processor -- start 2024-09-19T07:54:31.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.096+0000 7f1979186640 1 -- start start 2024-09-19T07:54:31.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f1979186640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19740d0070 0x7f197415a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:31.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f1979186640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19740d4130 0x7f1974161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:31.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f1979186640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f197415f0d0 0x7f19741623f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:31.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f1979186640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19740cef60 con 0x7f19740d4130 2024-09-19T07:54:31.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f1979186640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f19740cfb50 con 0x7f19740d0070 2024-09-19T07:54:31.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f1979186640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f19740b2f70 con 0x7f197415f0d0 2024-09-19T07:54:31.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f197259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19740d4130 0x7f1974161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:31.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f197259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19740d4130 0x7f1974161eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48492/0 (socket says 172.21.15.16:48492) 2024-09-19T07:54:31.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f197259c640 1 -- 172.21.15.16:0/780976103 learned_addr learned my addr 172.21.15.16:0/780976103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:31.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f1972d9d640 1 --2- 172.21.15.16:0/780976103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19740d0070 0x7f197415a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:31.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f197359e640 1 --2- 172.21.15.16:0/780976103 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f197415f0d0 0x7f19741623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:31.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f197259c640 1 -- 172.21.15.16:0/780976103 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f197415f0d0 msgr2=0x7f19741623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:31.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f197259c640 1 --2- 172.21.15.16:0/780976103 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f197415f0d0 0x7f19741623f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f197259c640 1 -- 172.21.15.16:0/780976103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19740d0070 msgr2=0x7f197415a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:31.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f197259c640 1 --2- 172.21.15.16:0/780976103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19740d0070 0x7f197415a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f197259c640 1 -- 172.21.15.16:0/780976103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19740c8020 con 0x7f19740d4130 2024-09-19T07:54:31.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f197259c640 1 --2- 172.21.15.16:0/780976103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19740d4130 0x7f1974161eb0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f1968009870 tx=0x7f19680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f195b7fe640 1 -- 172.21.15.16:0/780976103 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968041020 con 0x7f19740d4130 2024-09-19T07:54:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f195b7fe640 1 -- 172.21.15.16:0/780976103 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19680306c0 con 0x7f19740d4130 2024-09-19T07:54:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f1979186640 1 -- 172.21.15.16:0/780976103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19740c8300 con 0x7f19740d4130 2024-09-19T07:54:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f195b7fe640 1 -- 172.21.15.16:0/780976103 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968037040 con 0x7f19740d4130 2024-09-19T07:54:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.098+0000 7f1979186640 1 -- 172.21.15.16:0/780976103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19740c88c0 con 0x7f19740d4130 2024-09-19T07:54:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.099+0000 7f1979186640 1 -- 172.21.15.16:0/780976103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19740d45a0 con 0x7f19740d4130 2024-09-19T07:54:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.099+0000 7f195b7fe640 1 -- 172.21.15.16:0/780976103 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1968044020 con 0x7f19740d4130 2024-09-19T07:54:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.100+0000 7f195b7fe640 1 --2- 172.21.15.16:0/780976103 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f196006c2b0 0x7f196006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.100+0000 7f195b7fe640 1 -- 172.21.15.16:0/780976103 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7f19680abd80 con 0x7f19740d4130 2024-09-19T07:54:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.100+0000 7f1972d9d640 1 --2- 172.21.15.16:0/780976103 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f196006c2b0 0x7f196006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.102+0000 7f1972d9d640 1 --2- 172.21.15.16:0/780976103 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f196006c2b0 0x7f196006e770 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f196400a750 tx=0x7f1964008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.105+0000 7f195b7fe640 1 -- 172.21.15.16:0/780976103 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f196804b030 con 0x7f19740d4130 2024-09-19T07:54:31.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.278+0000 7f1979186640 1 -- 172.21.15.16:0/780976103 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f19740c7cb0 con 0x7f196006c2b0 2024-09-19T07:54:31.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.284+0000 7f195b7fe640 1 -- 172.21.15.16:0/780976103 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252458 (secure 0 0 0) 0x7f19740c7cb0 con 0x7f196006c2b0 2024-09-19T07:54:31.288 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:54:31.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.287+0000 7f1979186640 1 -- 172.21.15.16:0/780976103 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f196006c2b0 msgr2=0x7f196006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:31.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.287+0000 7f1979186640 1 --2- 172.21.15.16:0/780976103 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f196006c2b0 0x7f196006e770 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f196400a750 tx=0x7f1964008040 comp rx=0 tx=0).stop 2024-09-19T07:54:31.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.287+0000 7f1979186640 1 -- 172.21.15.16:0/780976103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19740d4130 msgr2=0x7f1974161eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:31.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.287+0000 7f1979186640 1 --2- 172.21.15.16:0/780976103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19740d4130 0x7f1974161eb0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f1968009870 tx=0x7f19680046c0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.287+0000 7f1979186640 1 -- 172.21.15.16:0/780976103 shutdown_connections 2024-09-19T07:54:31.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.287+0000 7f1979186640 1 --2- 172.21.15.16:0/780976103 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f196006c2b0 0x7f196006e770 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.287+0000 7f1979186640 1 --2- 172.21.15.16:0/780976103 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f197415f0d0 0x7f19741623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.287+0000 7f1979186640 1 --2- 172.21.15.16:0/780976103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19740d4130 0x7f1974161eb0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.287+0000 7f1979186640 1 --2- 172.21.15.16:0/780976103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19740d0070 0x7f197415a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.287+0000 7f1979186640 1 -- 172.21.15.16:0/780976103 >> 172.21.15.16:0/780976103 conn(0x7f197409d760 msgr2=0x7f197408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:31.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.287+0000 7f1979186640 1 -- 172.21.15.16:0/780976103 shutdown_connections 2024-09-19T07:54:31.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.287+0000 7f1979186640 1 -- 172.21.15.16:0/780976103 wait complete. 2024-09-19T07:54:31.300 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2024-09-19T07:54:30.620993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":764092,"kb_used_data":13768,"kb_used_omap":12,"kb_used_meta":750131,"kb_avail":754210628,"statfs":{"total":773094113280,"available":772311683072,"internally_reserved":0,"allocated":14098432,"data_stored":7150940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":768134704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001878"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572572+0000","last_change":"2024-09-19T07:54:23.583787+0000","last_active":"2024-09-19T07:54:24.572572+0000","last_peered":"2024-09-19T07:54:24.572572+0000","last_clean":"2024-09-19T07:54:24.572572+0000","last_became_active":"2024-09-19T07:54:23.583453+0000","last_became_peered":"2024-09-19T07:54:23.583453+0000","last_unstale":"2024-09-19T07:54:24.572572+0000","last_undegraded":"2024-09-19T07:54:24.572572+0000","last_fullsized":"2024-09-19T07:54:24.572572+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:23.821546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.563161+0000","last_change":"2024-09-19T07:54:23.576333+0000","last_active":"2024-09-19T07:54:24.563161+0000","last_peered":"2024-09-19T07:54:24.563161+0000","last_clean":"2024-09-19T07:54:24.563161+0000","last_became_active":"2024-09-19T07:54:23.576053+0000","last_became_peered":"2024-09-19T07:54:23.576053+0000","last_unstale":"2024-09-19T07:54:24.563161+0000","last_undegraded":"2024-09-19T07:54:24.563161+0000","last_fullsized":"2024-09-19T07:54:24.563161+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:36:37.189759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573368+0000","last_change":"2024-09-19T07:54:23.589816+0000","last_active":"2024-09-19T07:54:24.573368+0000","last_peered":"2024-09-19T07:54:24.573368+0000","last_clean":"2024-09-19T07:54:24.573368+0000","last_became_active":"2024-09-19T07:54:23.589423+0000","last_became_peered":"2024-09-19T07:54:23.589423+0000","last_unstale":"2024-09-19T07:54:24.573368+0000","last_undegraded":"2024-09-19T07:54:24.573368+0000","last_fullsized":"2024-09-19T07:54:24.573368+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:57.397944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.562713+0000","last_change":"2024-09-19T07:54:23.579793+0000","last_active":"2024-09-19T07:54:24.562713+0000","last_peered":"2024-09-19T07:54:24.562713+0000","last_clean":"2024-09-19T07:54:24.562713+0000","last_became_active":"2024-09-19T07:54:23.579502+0000","last_became_peered":"2024-09-19T07:54:23.579502+0000","last_unstale":"2024-09-19T07:54:24.562713+0000","last_undegraded":"2024-09-19T07:54:24.562713+0000","last_fullsized":"2024-09-19T07:54:24.562713+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:52.104763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572850+0000","last_change":"2024-09-19T07:54:23.584685+0000","last_active":"2024-09-19T07:54:24.572850+0000","last_peered":"2024-09-19T07:54:24.572850+0000","last_clean":"2024-09-19T07:54:24.572850+0000","last_became_active":"2024-09-19T07:54:23.584387+0000","last_became_peered":"2024-09-19T07:54:23.584387+0000","last_unstale":"2024-09-19T07:54:24.572850+0000","last_undegraded":"2024-09-19T07:54:24.572850+0000","last_fullsized":"2024-09-19T07:54:24.572850+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:59.290129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.561778+0000","last_change":"2024-09-19T07:54:23.583123+0000","last_active":"2024-09-19T07:54:24.561778+0000","last_peered":"2024-09-19T07:54:24.561778+0000","last_clean":"2024-09-19T07:54:24.561778+0000","last_became_active":"2024-09-19T07:54:23.582473+0000","last_became_peered":"2024-09-19T07:54:23.582473+0000","last_unstale":"2024-09-19T07:54:24.561778+0000","last_undegraded":"2024-09-19T07:54:24.561778+0000","last_fullsized":"2024-09-19T07:54:24.561778+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:42.806574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.574005+0000","last_change":"2024-09-19T07:54:23.584389+0000","last_active":"2024-09-19T07:54:24.574005+0000","last_peered":"2024-09-19T07:54:24.574005+0000","last_clean":"2024-09-19T07:54:24.574005+0000","last_became_active":"2024-09-19T07:54:23.584121+0000","last_became_peered":"2024-09-19T07:54:23.584121+0000","last_unstale":"2024-09-19T07:54:24.574005+0000","last_undegraded":"2024-09-19T07:54:24.574005+0000","last_fullsized":"2024-09-19T07:54:24.574005+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:48.898237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.562828+0000","last_change":"2024-09-19T07:54:23.584963+0000","last_active":"2024-09-19T07:54:24.562828+0000","last_peered":"2024-09-19T07:54:24.562828+0000","last_clean":"2024-09-19T07:54:24.562828+0000","last_became_active":"2024-09-19T07:54:23.583723+0000","last_became_peered":"2024-09-19T07:54:23.583723+0000","last_unstale":"2024-09-19T07:54:24.562828+0000","last_undegraded":"2024-09-19T07:54:24.562828+0000","last_fullsized":"2024-09-19T07:54:24.562828+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:56:05.465775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.561913+0000","last_change":"2024-09-19T07:54:23.566064+0000","last_active":"2024-09-19T07:54:24.561913+0000","last_peered":"2024-09-19T07:54:24.561913+0000","last_clean":"2024-09-19T07:54:24.561913+0000","last_became_active":"2024-09-19T07:54:23.565781+0000","last_became_peered":"2024-09-19T07:54:23.565781+0000","last_unstale":"2024-09-19T07:54:24.561913+0000","last_undegraded":"2024-09-19T07:54:24.561913+0000","last_fullsized":"2024-09-19T07:54:24.561913+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:29.913764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572441+0000","last_change":"2024-09-19T07:54:23.579169+0000","last_active":"2024-09-19T07:54:24.572441+0000","last_peered":"2024-09-19T07:54:24.572441+0000","last_clean":"2024-09-19T07:54:24.572441+0000","last_became_active":"2024-09-19T07:54:23.578888+0000","last_became_peered":"2024-09-19T07:54:23.578888+0000","last_unstale":"2024-09-19T07:54:24.572441+0000","last_undegraded":"2024-09-19T07:54:24.572441+0000","last_fullsized":"2024-09-19T07:54:24.572441+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:33:53.440423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.563035+0000","last_change":"2024-09-19T07:54:23.584332+0000","last_active":"2024-09-19T07:54:24.563035+0000","last_peered":"2024-09-19T07:54:24.563035+0000","last_clean":"2024-09-19T07:54:24.563035+0000","last_became_active":"2024-09-19T07:54:23.584031+0000","last_became_peered":"2024-09-19T07:54:23.584031+0000","last_unstale":"2024-09-19T07:54:24.563035+0000","last_undegraded":"2024-09-19T07:54:24.563035+0000","last_fullsized":"2024-09-19T07:54:24.563035+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:01.029649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572521+0000","last_change":"2024-09-19T07:54:23.574847+0000","last_active":"2024-09-19T07:54:24.572521+0000","last_peered":"2024-09-19T07:54:24.572521+0000","last_clean":"2024-09-19T07:54:24.572521+0000","last_became_active":"2024-09-19T07:54:23.574423+0000","last_became_peered":"2024-09-19T07:54:23.574423+0000","last_unstale":"2024-09-19T07:54:24.572521+0000","last_undegraded":"2024-09-19T07:54:24.572521+0000","last_fullsized":"2024-09-19T07:54:24.572521+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:35:20.465621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.561833+0000","last_change":"2024-09-19T07:54:23.581042+0000","last_active":"2024-09-19T07:54:24.561833+0000","last_peered":"2024-09-19T07:54:24.561833+0000","last_clean":"2024-09-19T07:54:24.561833+0000","last_became_active":"2024-09-19T07:54:23.580757+0000","last_became_peered":"2024-09-19T07:54:23.580757+0000","last_unstale":"2024-09-19T07:54:24.561833+0000","last_undegraded":"2024-09-19T07:54:24.561833+0000","last_fullsized":"2024-09-19T07:54:24.561833+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:13:01.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572583+0000","last_change":"2024-09-19T07:54:23.583768+0000","last_active":"2024-09-19T07:54:24.572583+0000","last_peered":"2024-09-19T07:54:24.572583+0000","last_clean":"2024-09-19T07:54:24.572583+0000","last_became_active":"2024-09-19T07:54:23.579038+0000","last_became_peered":"2024-09-19T07:54:23.579038+0000","last_unstale":"2024-09-19T07:54:24.572583+0000","last_undegraded":"2024-09-19T07:54:24.572583+0000","last_fullsized":"2024-09-19T07:54:24.572583+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:16:28.056548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572220+0000","last_change":"2024-09-19T07:54:23.582464+0000","last_active":"2024-09-19T07:54:24.572220+0000","last_peered":"2024-09-19T07:54:24.572220+0000","last_clean":"2024-09-19T07:54:24.572220+0000","last_became_active":"2024-09-19T07:54:23.582176+0000","last_became_peered":"2024-09-19T07:54:23.582176+0000","last_unstale":"2024-09-19T07:54:24.572220+0000","last_undegraded":"2024-09-19T07:54:24.572220+0000","last_fullsized":"2024-09-19T07:54:24.572220+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:25.500256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.561590+0000","last_change":"2024-09-19T07:54:23.573617+0000","last_active":"2024-09-19T07:54:24.561590+0000","last_peered":"2024-09-19T07:54:24.561590+0000","last_clean":"2024-09-19T07:54:24.561590+0000","last_became_active":"2024-09-19T07:54:23.573111+0000","last_became_peered":"2024-09-19T07:54:23.573111+0000","last_unstale":"2024-09-19T07:54:24.561590+0000","last_undegraded":"2024-09-19T07:54:24.561590+0000","last_fullsized":"2024-09-19T07:54:24.561590+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:37.295804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.561984+0000","last_change":"2024-09-19T07:54:23.573594+0000","last_active":"2024-09-19T07:54:24.561984+0000","last_peered":"2024-09-19T07:54:24.561984+0000","last_clean":"2024-09-19T07:54:24.561984+0000","last_became_active":"2024-09-19T07:54:23.573292+0000","last_became_peered":"2024-09-19T07:54:23.573292+0000","last_unstale":"2024-09-19T07:54:24.561984+0000","last_undegraded":"2024-09-19T07:54:24.561984+0000","last_fullsized":"2024-09-19T07:54:24.561984+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:19.902216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573174+0000","last_change":"2024-09-19T07:54:23.577157+0000","last_active":"2024-09-19T07:54:24.573174+0000","last_peered":"2024-09-19T07:54:24.573174+0000","last_clean":"2024-09-19T07:54:24.573174+0000","last_became_active":"2024-09-19T07:54:23.576873+0000","last_became_peered":"2024-09-19T07:54:23.576873+0000","last_unstale":"2024-09-19T07:54:24.573174+0000","last_undegraded":"2024-09-19T07:54:24.573174+0000","last_fullsized":"2024-09-19T07:54:24.573174+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:32.949629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573683+0000","last_change":"2024-09-19T07:54:23.580157+0000","last_active":"2024-09-19T07:54:24.573683+0000","last_peered":"2024-09-19T07:54:24.573683+0000","last_clean":"2024-09-19T07:54:24.573683+0000","last_became_active":"2024-09-19T07:54:23.579109+0000","last_became_peered":"2024-09-19T07:54:23.579109+0000","last_unstale":"2024-09-19T07:54:24.573683+0000","last_undegraded":"2024-09-19T07:54:24.573683+0000","last_fullsized":"2024-09-19T07:54:24.573683+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:55.409800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573051+0000","last_change":"2024-09-19T07:54:23.576506+0000","last_active":"2024-09-19T07:54:24.573051+0000","last_peered":"2024-09-19T07:54:24.573051+0000","last_clean":"2024-09-19T07:54:24.573051+0000","last_became_active":"2024-09-19T07:54:23.574887+0000","last_became_peered":"2024-09-19T07:54:23.574887+0000","last_unstale":"2024-09-19T07:54:24.573051+0000","last_undegraded":"2024-09-19T07:54:24.573051+0000","last_fullsized":"2024-09-19T07:54:24.573051+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:57.295016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.563305+0000","last_change":"2024-09-19T07:54:23.567173+0000","last_active":"2024-09-19T07:54:24.563305+0000","last_peered":"2024-09-19T07:54:24.563305+0000","last_clean":"2024-09-19T07:54:24.563305+0000","last_became_active":"2024-09-19T07:54:23.566866+0000","last_became_peered":"2024-09-19T07:54:23.566866+0000","last_unstale":"2024-09-19T07:54:24.563305+0000","last_undegraded":"2024-09-19T07:54:24.563305+0000","last_fullsized":"2024-09-19T07:54:24.563305+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:42.806574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572411+0000","last_change":"2024-09-19T07:54:23.586941+0000","last_active":"2024-09-19T07:54:24.572411+0000","last_peered":"2024-09-19T07:54:24.572411+0000","last_clean":"2024-09-19T07:54:24.572411+0000","last_became_active":"2024-09-19T07:54:23.586367+0000","last_became_peered":"2024-09-19T07:54:23.586367+0000","last_unstale":"2024-09-19T07:54:24.572411+0000","last_undegraded":"2024-09-19T07:54:24.572411+0000","last_fullsized":"2024-09-19T07:54:24.572411+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:22.782628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.563217+0000","last_change":"2024-09-19T07:54:23.575696+0000","last_active":"2024-09-19T07:54:24.563217+0000","last_peered":"2024-09-19T07:54:24.563217+0000","last_clean":"2024-09-19T07:54:24.563217+0000","last_became_active":"2024-09-19T07:54:23.575427+0000","last_became_peered":"2024-09-19T07:54:23.575427+0000","last_unstale":"2024-09-19T07:54:24.563217+0000","last_undegraded":"2024-09-19T07:54:24.563217+0000","last_fullsized":"2024-09-19T07:54:24.563217+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:02.732821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.561656+0000","last_change":"2024-09-19T07:54:23.568377+0000","last_active":"2024-09-19T07:54:24.561656+0000","last_peered":"2024-09-19T07:54:24.561656+0000","last_clean":"2024-09-19T07:54:24.561656+0000","last_became_active":"2024-09-19T07:54:23.568068+0000","last_became_peered":"2024-09-19T07:54:23.568068+0000","last_unstale":"2024-09-19T07:54:24.561656+0000","last_undegraded":"2024-09-19T07:54:24.561656+0000","last_fullsized":"2024-09-19T07:54:24.561656+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:36:37.189759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573813+0000","last_change":"2024-09-19T07:54:23.584866+0000","last_active":"2024-09-19T07:54:24.573813+0000","last_peered":"2024-09-19T07:54:24.573813+0000","last_clean":"2024-09-19T07:54:24.573813+0000","last_became_active":"2024-09-19T07:54:23.582585+0000","last_became_peered":"2024-09-19T07:54:23.582585+0000","last_unstale":"2024-09-19T07:54:24.573813+0000","last_undegraded":"2024-09-19T07:54:24.573813+0000","last_fullsized":"2024-09-19T07:54:24.573813+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:32.420071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573480+0000","last_change":"2024-09-19T07:54:23.583732+0000","last_active":"2024-09-19T07:54:24.573480+0000","last_peered":"2024-09-19T07:54:24.573480+0000","last_clean":"2024-09-19T07:54:24.573480+0000","last_became_active":"2024-09-19T07:54:23.583460+0000","last_became_peered":"2024-09-19T07:54:23.583460+0000","last_unstale":"2024-09-19T07:54:24.573480+0000","last_undegraded":"2024-09-19T07:54:24.573480+0000","last_fullsized":"2024-09-19T07:54:24.573480+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:41:16.028842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.574072+0000","last_change":"2024-09-19T07:54:23.579141+0000","last_active":"2024-09-19T07:54:24.574072+0000","last_peered":"2024-09-19T07:54:24.574072+0000","last_clean":"2024-09-19T07:54:24.574072+0000","last_became_active":"2024-09-19T07:54:23.578856+0000","last_became_peered":"2024-09-19T07:54:23.578856+0000","last_unstale":"2024-09-19T07:54:24.574072+0000","last_undegraded":"2024-09-19T07:54:24.574072+0000","last_fullsized":"2024-09-19T07:54:24.574072+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:49.410443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573313+0000","last_change":"2024-09-19T07:54:23.582767+0000","last_active":"2024-09-19T07:54:24.573313+0000","last_peered":"2024-09-19T07:54:24.573313+0000","last_clean":"2024-09-19T07:54:24.573313+0000","last_became_active":"2024-09-19T07:54:23.582486+0000","last_became_peered":"2024-09-19T07:54:23.582486+0000","last_unstale":"2024-09-19T07:54:24.573313+0000","last_undegraded":"2024-09-19T07:54:24.573313+0000","last_fullsized":"2024-09-19T07:54:24.573313+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:52:32.044984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573507+0000","last_change":"2024-09-19T07:54:23.587202+0000","last_active":"2024-09-19T07:54:24.573507+0000","last_peered":"2024-09-19T07:54:24.573507+0000","last_clean":"2024-09-19T07:54:24.573507+0000","last_became_active":"2024-09-19T07:54:23.586940+0000","last_became_peered":"2024-09-19T07:54:23.586940+0000","last_unstale":"2024-09-19T07:54:24.573507+0000","last_undegraded":"2024-09-19T07:54:24.573507+0000","last_fullsized":"2024-09-19T07:54:24.573507+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:26.299587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573462+0000","last_change":"2024-09-19T07:54:23.581592+0000","last_active":"2024-09-19T07:54:24.573462+0000","last_peered":"2024-09-19T07:54:24.573462+0000","last_clean":"2024-09-19T07:54:24.573462+0000","last_became_active":"2024-09-19T07:54:23.581298+0000","last_became_peered":"2024-09-19T07:54:23.581298+0000","last_unstale":"2024-09-19T07:54:24.573462+0000","last_undegraded":"2024-09-19T07:54:24.573462+0000","last_fullsized":"2024-09-19T07:54:24.573462+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:56:05.465775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.562623+0000","last_change":"2024-09-19T07:54:23.575015+0000","last_active":"2024-09-19T07:54:24.562623+0000","last_peered":"2024-09-19T07:54:24.562623+0000","last_clean":"2024-09-19T07:54:24.562623+0000","last_became_active":"2024-09-19T07:54:23.574695+0000","last_became_peered":"2024-09-19T07:54:23.574695+0000","last_unstale":"2024-09-19T07:54:24.562623+0000","last_undegraded":"2024-09-19T07:54:24.562623+0000","last_fullsized":"2024-09-19T07:54:24.562623+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:50.463382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572033+0000","last_change":"2024-09-19T07:54:23.583307+0000","last_active":"2024-09-19T07:54:24.572033+0000","last_peered":"2024-09-19T07:54:24.572033+0000","last_clean":"2024-09-19T07:54:24.572033+0000","last_became_active":"2024-09-19T07:54:23.583021+0000","last_became_peered":"2024-09-19T07:54:23.583021+0000","last_unstale":"2024-09-19T07:54:24.572033+0000","last_undegraded":"2024-09-19T07:54:24.572033+0000","last_fullsized":"2024-09-19T07:54:24.572033+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:11.438853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.562972+0000","last_change":"2024-09-19T07:54:23.576203+0000","last_active":"2024-09-19T07:54:24.562972+0000","last_peered":"2024-09-19T07:54:24.562972+0000","last_clean":"2024-09-19T07:54:24.562972+0000","last_became_active":"2024-09-19T07:54:23.575926+0000","last_became_peered":"2024-09-19T07:54:23.575926+0000","last_unstale":"2024-09-19T07:54:24.562972+0000","last_undegraded":"2024-09-19T07:54:24.562972+0000","last_fullsized":"2024-09-19T07:54:24.562972+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:41:16.028842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"128'4","reported_seq":18,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573848+0000","last_change":"2024-09-19T07:54:22.568159+0000","last_active":"2024-09-19T07:54:24.573848+0000","last_peered":"2024-09-19T07:54:24.573848+0000","last_clean":"2024-09-19T07:54:24.573848+0000","last_became_active":"2024-09-19T07:54:22.567832+0000","last_became_peered":"2024-09-19T07:54:22.567832+0000","last_unstale":"2024-09-19T07:54:24.573848+0000","last_undegraded":"2024-09-19T07:54:24.573848+0000","last_fullsized":"2024-09-19T07:54:24.573848+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:46.972795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572726+0000","last_change":"2024-09-19T07:54:23.585429+0000","last_active":"2024-09-19T07:54:24.572726+0000","last_peered":"2024-09-19T07:54:24.572726+0000","last_clean":"2024-09-19T07:54:24.572726+0000","last_became_active":"2024-09-19T07:54:23.585096+0000","last_became_peered":"2024-09-19T07:54:23.585096+0000","last_unstale":"2024-09-19T07:54:24.572726+0000","last_undegraded":"2024-09-19T07:54:24.572726+0000","last_fullsized":"2024-09-19T07:54:24.572726+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:29.192911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"128'1","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573359+0000","last_change":"2024-09-19T07:54:22.561503+0000","last_active":"2024-09-19T07:54:24.573359+0000","last_peered":"2024-09-19T07:54:24.573359+0000","last_clean":"2024-09-19T07:54:24.573359+0000","last_became_active":"2024-09-19T07:54:22.561219+0000","last_became_peered":"2024-09-19T07:54:22.561219+0000","last_unstale":"2024-09-19T07:54:24.573359+0000","last_undegraded":"2024-09-19T07:54:24.573359+0000","last_fullsized":"2024-09-19T07:54:24.573359+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:26.041053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.562866+0000","last_change":"2024-09-19T07:54:23.574954+0000","last_active":"2024-09-19T07:54:24.562866+0000","last_peered":"2024-09-19T07:54:24.562866+0000","last_clean":"2024-09-19T07:54:24.562866+0000","last_became_active":"2024-09-19T07:54:23.574624+0000","last_became_peered":"2024-09-19T07:54:23.574624+0000","last_unstale":"2024-09-19T07:54:24.562866+0000","last_undegraded":"2024-09-19T07:54:24.562866+0000","last_fullsized":"2024-09-19T07:54:24.562866+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:37.295804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"128'2","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573799+0000","last_change":"2024-09-19T07:54:22.566018+0000","last_active":"2024-09-19T07:54:24.573799+0000","last_peered":"2024-09-19T07:54:24.573799+0000","last_clean":"2024-09-19T07:54:24.573799+0000","last_became_active":"2024-09-19T07:54:22.565812+0000","last_became_peered":"2024-09-19T07:54:22.565812+0000","last_unstale":"2024-09-19T07:54:24.573799+0000","last_undegraded":"2024-09-19T07:54:24.573799+0000","last_fullsized":"2024-09-19T07:54:24.573799+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:01.728173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572245+0000","last_change":"2024-09-19T07:54:23.582120+0000","last_active":"2024-09-19T07:54:24.572245+0000","last_peered":"2024-09-19T07:54:24.572245+0000","last_clean":"2024-09-19T07:54:24.572245+0000","last_became_active":"2024-09-19T07:54:23.581830+0000","last_became_peered":"2024-09-19T07:54:23.581830+0000","last_unstale":"2024-09-19T07:54:24.572245+0000","last_undegraded":"2024-09-19T07:54:24.572245+0000","last_fullsized":"2024-09-19T07:54:24.572245+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:12.630683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"128'2","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573567+0000","last_change":"2024-09-19T07:54:22.560370+0000","last_active":"2024-09-19T07:54:24.573567+0000","last_peered":"2024-09-19T07:54:24.573567+0000","last_clean":"2024-09-19T07:54:24.573567+0000","last_became_active":"2024-09-19T07:54:22.560083+0000","last_became_peered":"2024-09-19T07:54:22.560083+0000","last_unstale":"2024-09-19T07:54:24.573567+0000","last_undegraded":"2024-09-19T07:54:24.573567+0000","last_fullsized":"2024-09-19T07:54:24.573567+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:51.100115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.563100+0000","last_change":"2024-09-19T07:54:23.577913+0000","last_active":"2024-09-19T07:54:24.563100+0000","last_peered":"2024-09-19T07:54:24.563100+0000","last_clean":"2024-09-19T07:54:24.563100+0000","last_became_active":"2024-09-19T07:54:23.577102+0000","last_became_peered":"2024-09-19T07:54:23.577102+0000","last_unstale":"2024-09-19T07:54:24.563100+0000","last_undegraded":"2024-09-19T07:54:24.563100+0000","last_fullsized":"2024-09-19T07:54:24.563100+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:26.299587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"128'1","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.563295+0000","last_change":"2024-09-19T07:54:22.554611+0000","last_active":"2024-09-19T07:54:24.563295+0000","last_peered":"2024-09-19T07:54:24.563295+0000","last_clean":"2024-09-19T07:54:24.563295+0000","last_became_active":"2024-09-19T07:54:22.554332+0000","last_became_peered":"2024-09-19T07:54:22.554332+0000","last_unstale":"2024-09-19T07:54:24.563295+0000","last_undegraded":"2024-09-19T07:54:24.563295+0000","last_fullsized":"2024-09-19T07:54:24.563295+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:09.508289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.563172+0000","last_change":"2024-09-19T07:54:23.574521+0000","last_active":"2024-09-19T07:54:24.563172+0000","last_peered":"2024-09-19T07:54:24.563172+0000","last_clean":"2024-09-19T07:54:24.563172+0000","last_became_active":"2024-09-19T07:54:23.574251+0000","last_became_peered":"2024-09-19T07:54:23.574251+0000","last_unstale":"2024-09-19T07:54:24.563172+0000","last_undegraded":"2024-09-19T07:54:24.563172+0000","last_fullsized":"2024-09-19T07:54:24.563172+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:07.368761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"128'1","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572813+0000","last_change":"2024-09-19T07:54:22.564044+0000","last_active":"2024-09-19T07:54:24.572813+0000","last_peered":"2024-09-19T07:54:24.572813+0000","last_clean":"2024-09-19T07:54:24.572813+0000","last_became_active":"2024-09-19T07:54:22.562851+0000","last_became_peered":"2024-09-19T07:54:22.562851+0000","last_unstale":"2024-09-19T07:54:24.572813+0000","last_undegraded":"2024-09-19T07:54:24.572813+0000","last_fullsized":"2024-09-19T07:54:24.572813+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:42.486939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573268+0000","last_change":"2024-09-19T07:54:23.580128+0000","last_active":"2024-09-19T07:54:24.573268+0000","last_peered":"2024-09-19T07:54:24.573268+0000","last_clean":"2024-09-19T07:54:24.573268+0000","last_became_active":"2024-09-19T07:54:23.579842+0000","last_became_peered":"2024-09-19T07:54:23.579842+0000","last_unstale":"2024-09-19T07:54:24.573268+0000","last_undegraded":"2024-09-19T07:54:24.573268+0000","last_fullsized":"2024-09-19T07:54:24.573268+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:38.791089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.561498+0000","last_change":"2024-09-19T07:54:22.557995+0000","last_active":"2024-09-19T07:54:24.561498+0000","last_peered":"2024-09-19T07:54:24.561498+0000","last_clean":"2024-09-19T07:54:24.561498+0000","last_became_active":"2024-09-19T07:54:22.557692+0000","last_became_peered":"2024-09-19T07:54:22.557692+0000","last_unstale":"2024-09-19T07:54:24.561498+0000","last_undegraded":"2024-09-19T07:54:24.561498+0000","last_fullsized":"2024-09-19T07:54:24.561498+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:55.772454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.574000+0000","last_change":"2024-09-19T07:54:23.573943+0000","last_active":"2024-09-19T07:54:24.574000+0000","last_peered":"2024-09-19T07:54:24.574000+0000","last_clean":"2024-09-19T07:54:24.574000+0000","last_became_active":"2024-09-19T07:54:23.573668+0000","last_became_peered":"2024-09-19T07:54:23.573668+0000","last_unstale":"2024-09-19T07:54:24.574000+0000","last_undegraded":"2024-09-19T07:54:24.574000+0000","last_fullsized":"2024-09-19T07:54:24.574000+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:36.619462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573624+0000","last_change":"2024-09-19T07:54:22.562903+0000","last_active":"2024-09-19T07:54:24.573624+0000","last_peered":"2024-09-19T07:54:24.573624+0000","last_clean":"2024-09-19T07:54:24.573624+0000","last_became_active":"2024-09-19T07:54:22.562599+0000","last_became_peered":"2024-09-19T07:54:22.562599+0000","last_unstale":"2024-09-19T07:54:24.573624+0000","last_undegraded":"2024-09-19T07:54:24.573624+0000","last_fullsized":"2024-09-19T07:54:24.573624+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:44.078560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572071+0000","last_change":"2024-09-19T07:54:23.586564+0000","last_active":"2024-09-19T07:54:24.572071+0000","last_peered":"2024-09-19T07:54:24.572071+0000","last_clean":"2024-09-19T07:54:24.572071+0000","last_became_active":"2024-09-19T07:54:23.586246+0000","last_became_peered":"2024-09-19T07:54:23.586246+0000","last_unstale":"2024-09-19T07:54:24.572071+0000","last_undegraded":"2024-09-19T07:54:24.572071+0000","last_fullsized":"2024-09-19T07:54:24.572071+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:14:59.560377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.563761+0000","last_change":"2024-09-19T07:54:22.556612+0000","last_active":"2024-09-19T07:54:24.563761+0000","last_peered":"2024-09-19T07:54:24.563761+0000","last_clean":"2024-09-19T07:54:24.563761+0000","last_became_active":"2024-09-19T07:54:22.556177+0000","last_became_peered":"2024-09-19T07:54:22.556177+0000","last_unstale":"2024-09-19T07:54:24.563761+0000","last_undegraded":"2024-09-19T07:54:24.563761+0000","last_fullsized":"2024-09-19T07:54:24.563761+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:25.294939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.561645+0000","last_change":"2024-09-19T07:54:23.577640+0000","last_active":"2024-09-19T07:54:24.561645+0000","last_peered":"2024-09-19T07:54:24.561645+0000","last_clean":"2024-09-19T07:54:24.561645+0000","last_became_active":"2024-09-19T07:54:23.575726+0000","last_became_peered":"2024-09-19T07:54:23.575726+0000","last_unstale":"2024-09-19T07:54:24.561645+0000","last_undegraded":"2024-09-19T07:54:24.561645+0000","last_fullsized":"2024-09-19T07:54:24.561645+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:01.029649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572966+0000","last_change":"2024-09-19T07:54:22.564375+0000","last_active":"2024-09-19T07:54:24.572966+0000","last_peered":"2024-09-19T07:54:24.572966+0000","last_clean":"2024-09-19T07:54:24.572966+0000","last_became_active":"2024-09-19T07:54:22.564096+0000","last_became_peered":"2024-09-19T07:54:22.564096+0000","last_unstale":"2024-09-19T07:54:24.572966+0000","last_undegraded":"2024-09-19T07:54:24.572966+0000","last_fullsized":"2024-09-19T07:54:24.572966+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:05.452375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573445+0000","last_change":"2024-09-19T07:54:23.583678+0000","last_active":"2024-09-19T07:54:24.573445+0000","last_peered":"2024-09-19T07:54:24.573445+0000","last_clean":"2024-09-19T07:54:24.573445+0000","last_became_active":"2024-09-19T07:54:23.583254+0000","last_became_peered":"2024-09-19T07:54:23.583254+0000","last_unstale":"2024-09-19T07:54:24.573445+0000","last_undegraded":"2024-09-19T07:54:24.573445+0000","last_fullsized":"2024-09-19T07:54:24.573445+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:50.463382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"128'3","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573922+0000","last_change":"2024-09-19T07:54:22.558733+0000","last_active":"2024-09-19T07:54:24.573922+0000","last_peered":"2024-09-19T07:54:24.573922+0000","last_clean":"2024-09-19T07:54:24.573922+0000","last_became_active":"2024-09-19T07:54:22.558481+0000","last_became_peered":"2024-09-19T07:54:22.558481+0000","last_unstale":"2024-09-19T07:54:24.573922+0000","last_undegraded":"2024-09-19T07:54:24.573922+0000","last_fullsized":"2024-09-19T07:54:24.573922+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:41.801926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573661+0000","last_change":"2024-09-19T07:54:23.589083+0000","last_active":"2024-09-19T07:54:24.573661+0000","last_peered":"2024-09-19T07:54:24.573661+0000","last_clean":"2024-09-19T07:54:24.573661+0000","last_became_active":"2024-09-19T07:54:23.588728+0000","last_became_peered":"2024-09-19T07:54:23.588728+0000","last_unstale":"2024-09-19T07:54:24.573661+0000","last_undegraded":"2024-09-19T07:54:24.573661+0000","last_fullsized":"2024-09-19T07:54:24.573661+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:07.368761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"128'6","reported_seq":20,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573564+0000","last_change":"2024-09-19T07:54:22.564291+0000","last_active":"2024-09-19T07:54:24.573564+0000","last_peered":"2024-09-19T07:54:24.573564+0000","last_clean":"2024-09-19T07:54:24.573564+0000","last_became_active":"2024-09-19T07:54:22.564012+0000","last_became_peered":"2024-09-19T07:54:22.564012+0000","last_unstale":"2024-09-19T07:54:24.573564+0000","last_undegraded":"2024-09-19T07:54:24.573564+0000","last_fullsized":"2024-09-19T07:54:24.573564+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:00.025001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573411+0000","last_change":"2024-09-19T07:54:23.578794+0000","last_active":"2024-09-19T07:54:24.573411+0000","last_peered":"2024-09-19T07:54:24.573411+0000","last_clean":"2024-09-19T07:54:24.573411+0000","last_became_active":"2024-09-19T07:54:23.578465+0000","last_became_peered":"2024-09-19T07:54:23.578465+0000","last_unstale":"2024-09-19T07:54:24.573411+0000","last_undegraded":"2024-09-19T07:54:24.573411+0000","last_fullsized":"2024-09-19T07:54:24.573411+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:34.477730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"128'2","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572360+0000","last_change":"2024-09-19T07:54:22.565043+0000","last_active":"2024-09-19T07:54:24.572360+0000","last_peered":"2024-09-19T07:54:24.572360+0000","last_clean":"2024-09-19T07:54:24.572360+0000","last_became_active":"2024-09-19T07:54:22.564758+0000","last_became_peered":"2024-09-19T07:54:22.564758+0000","last_unstale":"2024-09-19T07:54:24.572360+0000","last_undegraded":"2024-09-19T07:54:24.572360+0000","last_fullsized":"2024-09-19T07:54:24.572360+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:58:28.017221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.574193+0000","last_change":"2024-09-19T07:54:23.574862+0000","last_active":"2024-09-19T07:54:24.574193+0000","last_peered":"2024-09-19T07:54:24.574193+0000","last_clean":"2024-09-19T07:54:24.574193+0000","last_became_active":"2024-09-19T07:54:23.574431+0000","last_became_peered":"2024-09-19T07:54:23.574431+0000","last_unstale":"2024-09-19T07:54:24.574193+0000","last_undegraded":"2024-09-19T07:54:24.574193+0000","last_fullsized":"2024-09-19T07:54:24.574193+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:40.687672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573181+0000","last_change":"2024-09-19T07:54:22.559040+0000","last_active":"2024-09-19T07:54:24.573181+0000","last_peered":"2024-09-19T07:54:24.573181+0000","last_clean":"2024-09-19T07:54:24.573181+0000","last_became_active":"2024-09-19T07:54:22.558757+0000","last_became_peered":"2024-09-19T07:54:22.558757+0000","last_unstale":"2024-09-19T07:54:24.573181+0000","last_undegraded":"2024-09-19T07:54:24.573181+0000","last_fullsized":"2024-09-19T07:54:24.573181+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:28:24.370408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573086+0000","last_change":"2024-09-19T07:54:23.570832+0000","last_active":"2024-09-19T07:54:24.573086+0000","last_peered":"2024-09-19T07:54:24.573086+0000","last_clean":"2024-09-19T07:54:24.573086+0000","last_became_active":"2024-09-19T07:54:23.570518+0000","last_became_peered":"2024-09-19T07:54:23.570518+0000","last_unstale":"2024-09-19T07:54:24.573086+0000","last_undegraded":"2024-09-19T07:54:24.573086+0000","last_fullsized":"2024-09-19T07:54:24.573086+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:09.226711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"128'8","reported_seq":22,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.561780+0000","last_change":"2024-09-19T07:54:22.559203+0000","last_active":"2024-09-19T07:54:24.561780+0000","last_peered":"2024-09-19T07:54:24.561780+0000","last_clean":"2024-09-19T07:54:24.561780+0000","last_became_active":"2024-09-19T07:54:22.558920+0000","last_became_peered":"2024-09-19T07:54:22.558920+0000","last_unstale":"2024-09-19T07:54:24.561780+0000","last_undegraded":"2024-09-19T07:54:24.561780+0000","last_fullsized":"2024-09-19T07:54:24.561780+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:56:04.461128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573172+0000","last_change":"2024-09-19T07:54:23.587980+0000","last_active":"2024-09-19T07:54:24.573172+0000","last_peered":"2024-09-19T07:54:24.573172+0000","last_clean":"2024-09-19T07:54:24.573172+0000","last_became_active":"2024-09-19T07:54:23.587702+0000","last_became_peered":"2024-09-19T07:54:23.587702+0000","last_unstale":"2024-09-19T07:54:24.573172+0000","last_undegraded":"2024-09-19T07:54:24.573172+0000","last_fullsized":"2024-09-19T07:54:24.573172+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:49.188816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"128'1","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.563872+0000","last_change":"2024-09-19T07:54:22.553965+0000","last_active":"2024-09-19T07:54:24.563872+0000","last_peered":"2024-09-19T07:54:24.563872+0000","last_clean":"2024-09-19T07:54:24.563872+0000","last_became_active":"2024-09-19T07:54:22.553678+0000","last_became_peered":"2024-09-19T07:54:22.553678+0000","last_unstale":"2024-09-19T07:54:24.563872+0000","last_undegraded":"2024-09-19T07:54:24.563872+0000","last_fullsized":"2024-09-19T07:54:24.563872+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:06:06.364113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573285+0000","last_change":"2024-09-19T07:54:23.574769+0000","last_active":"2024-09-19T07:54:24.573285+0000","last_peered":"2024-09-19T07:54:24.573285+0000","last_clean":"2024-09-19T07:54:24.573285+0000","last_became_active":"2024-09-19T07:54:23.574488+0000","last_became_peered":"2024-09-19T07:54:23.574488+0000","last_unstale":"2024-09-19T07:54:24.573285+0000","last_undegraded":"2024-09-19T07:54:24.573285+0000","last_fullsized":"2024-09-19T07:54:24.573285+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:27:17.728264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"128'2","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573596+0000","last_change":"2024-09-19T07:54:22.567957+0000","last_active":"2024-09-19T07:54:24.573596+0000","last_peered":"2024-09-19T07:54:24.573596+0000","last_clean":"2024-09-19T07:54:24.573596+0000","last_became_active":"2024-09-19T07:54:22.567503+0000","last_became_peered":"2024-09-19T07:54:22.567503+0000","last_unstale":"2024-09-19T07:54:24.573596+0000","last_undegraded":"2024-09-19T07:54:24.573596+0000","last_fullsized":"2024-09-19T07:54:24.573596+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:06.620754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573746+0000","last_change":"2024-09-19T07:54:23.578211+0000","last_active":"2024-09-19T07:54:24.573746+0000","last_peered":"2024-09-19T07:54:24.573746+0000","last_clean":"2024-09-19T07:54:24.573746+0000","last_became_active":"2024-09-19T07:54:23.577649+0000","last_became_peered":"2024-09-19T07:54:23.577649+0000","last_unstale":"2024-09-19T07:54:24.573746+0000","last_undegraded":"2024-09-19T07:54:24.573746+0000","last_fullsized":"2024-09-19T07:54:24.573746+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:45.651261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"128'1","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.562023+0000","last_change":"2024-09-19T07:54:22.559404+0000","last_active":"2024-09-19T07:54:24.562023+0000","last_peered":"2024-09-19T07:54:24.562023+0000","last_clean":"2024-09-19T07:54:24.562023+0000","last_became_active":"2024-09-19T07:54:22.559101+0000","last_became_peered":"2024-09-19T07:54:22.559101+0000","last_unstale":"2024-09-19T07:54:24.562023+0000","last_undegraded":"2024-09-19T07:54:24.562023+0000","last_fullsized":"2024-09-19T07:54:24.562023+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:56.393297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572972+0000","last_change":"2024-09-19T07:54:23.585316+0000","last_active":"2024-09-19T07:54:24.572972+0000","last_peered":"2024-09-19T07:54:24.572972+0000","last_clean":"2024-09-19T07:54:24.572972+0000","last_became_active":"2024-09-19T07:54:23.585029+0000","last_became_peered":"2024-09-19T07:54:23.585029+0000","last_unstale":"2024-09-19T07:54:24.572972+0000","last_undegraded":"2024-09-19T07:54:24.572972+0000","last_fullsized":"2024-09-19T07:54:24.572972+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:15.730926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573144+0000","last_change":"2024-09-19T07:54:22.563785+0000","last_active":"2024-09-19T07:54:24.573144+0000","last_peered":"2024-09-19T07:54:24.573144+0000","last_clean":"2024-09-19T07:54:24.573144+0000","last_became_active":"2024-09-19T07:54:22.563471+0000","last_became_peered":"2024-09-19T07:54:22.563471+0000","last_unstale":"2024-09-19T07:54:24.573144+0000","last_undegraded":"2024-09-19T07:54:24.573144+0000","last_fullsized":"2024-09-19T07:54:24.573144+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:27:16.723616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573101+0000","last_change":"2024-09-19T07:54:23.576840+0000","last_active":"2024-09-19T07:54:24.573101+0000","last_peered":"2024-09-19T07:54:24.573101+0000","last_clean":"2024-09-19T07:54:24.573101+0000","last_became_active":"2024-09-19T07:54:23.576512+0000","last_became_peered":"2024-09-19T07:54:23.576512+0000","last_unstale":"2024-09-19T07:54:24.573101+0000","last_undegraded":"2024-09-19T07:54:24.573101+0000","last_fullsized":"2024-09-19T07:54:24.573101+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:20.783132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"128'2","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.563303+0000","last_change":"2024-09-19T07:54:22.556136+0000","last_active":"2024-09-19T07:54:24.563303+0000","last_peered":"2024-09-19T07:54:24.563303+0000","last_clean":"2024-09-19T07:54:24.563303+0000","last_became_active":"2024-09-19T07:54:22.555829+0000","last_became_peered":"2024-09-19T07:54:22.555829+0000","last_unstale":"2024-09-19T07:54:24.563303+0000","last_undegraded":"2024-09-19T07:54:24.563303+0000","last_fullsized":"2024-09-19T07:54:24.563303+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:37.052983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572425+0000","last_change":"2024-09-19T07:54:23.582629+0000","last_active":"2024-09-19T07:54:24.572425+0000","last_peered":"2024-09-19T07:54:24.572425+0000","last_clean":"2024-09-19T07:54:24.572425+0000","last_became_active":"2024-09-19T07:54:23.582343+0000","last_became_peered":"2024-09-19T07:54:23.582343+0000","last_unstale":"2024-09-19T07:54:24.572425+0000","last_undegraded":"2024-09-19T07:54:24.572425+0000","last_fullsized":"2024-09-19T07:54:24.572425+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:38.135921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572535+0000","last_change":"2024-09-19T07:54:22.563856+0000","last_active":"2024-09-19T07:54:24.572535+0000","last_peered":"2024-09-19T07:54:24.572535+0000","last_clean":"2024-09-19T07:54:24.572535+0000","last_became_active":"2024-09-19T07:54:22.563515+0000","last_became_peered":"2024-09-19T07:54:22.563515+0000","last_unstale":"2024-09-19T07:54:24.572535+0000","last_undegraded":"2024-09-19T07:54:24.572535+0000","last_fullsized":"2024-09-19T07:54:24.572535+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:32.880703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573352+0000","last_change":"2024-09-19T07:54:23.585848+0000","last_active":"2024-09-19T07:54:24.573352+0000","last_peered":"2024-09-19T07:54:24.573352+0000","last_clean":"2024-09-19T07:54:24.573352+0000","last_became_active":"2024-09-19T07:54:23.585575+0000","last_became_peered":"2024-09-19T07:54:23.585575+0000","last_unstale":"2024-09-19T07:54:24.573352+0000","last_undegraded":"2024-09-19T07:54:24.573352+0000","last_fullsized":"2024-09-19T07:54:24.573352+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:35.538510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573433+0000","last_change":"2024-09-19T07:54:22.564048+0000","last_active":"2024-09-19T07:54:24.573433+0000","last_peered":"2024-09-19T07:54:24.573433+0000","last_clean":"2024-09-19T07:54:24.573433+0000","last_became_active":"2024-09-19T07:54:22.563775+0000","last_became_peered":"2024-09-19T07:54:22.563775+0000","last_unstale":"2024-09-19T07:54:24.573433+0000","last_undegraded":"2024-09-19T07:54:24.573433+0000","last_fullsized":"2024-09-19T07:54:24.573433+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:57.190350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.563666+0000","last_change":"2024-09-19T07:54:23.584566+0000","last_active":"2024-09-19T07:54:24.563666+0000","last_peered":"2024-09-19T07:54:24.563666+0000","last_clean":"2024-09-19T07:54:24.563666+0000","last_became_active":"2024-09-19T07:54:23.584002+0000","last_became_peered":"2024-09-19T07:54:23.584002+0000","last_unstale":"2024-09-19T07:54:24.563666+0000","last_undegraded":"2024-09-19T07:54:24.563666+0000","last_fullsized":"2024-09-19T07:54:24.563666+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:13:01.144966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.561917+0000","last_change":"2024-09-19T07:54:22.558583+0000","last_active":"2024-09-19T07:54:24.561917+0000","last_peered":"2024-09-19T07:54:24.561917+0000","last_clean":"2024-09-19T07:54:24.561917+0000","last_became_active":"2024-09-19T07:54:22.558239+0000","last_became_peered":"2024-09-19T07:54:22.558239+0000","last_unstale":"2024-09-19T07:54:24.561917+0000","last_undegraded":"2024-09-19T07:54:24.561917+0000","last_fullsized":"2024-09-19T07:54:24.561917+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:41:15.024194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572873+0000","last_change":"2024-09-19T07:54:23.576205+0000","last_active":"2024-09-19T07:54:24.572873+0000","last_peered":"2024-09-19T07:54:24.572873+0000","last_clean":"2024-09-19T07:54:24.572873+0000","last_became_active":"2024-09-19T07:54:23.575362+0000","last_became_peered":"2024-09-19T07:54:23.575362+0000","last_unstale":"2024-09-19T07:54:24.572873+0000","last_undegraded":"2024-09-19T07:54:24.572873+0000","last_fullsized":"2024-09-19T07:54:24.572873+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:56:28.570535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"128'2","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.572673+0000","last_change":"2024-09-19T07:54:22.561535+0000","last_active":"2024-09-19T07:54:24.572673+0000","last_peered":"2024-09-19T07:54:24.572673+0000","last_clean":"2024-09-19T07:54:24.572673+0000","last_became_active":"2024-09-19T07:54:22.561210+0000","last_became_peered":"2024-09-19T07:54:22.561210+0000","last_unstale":"2024-09-19T07:54:24.572673+0000","last_undegraded":"2024-09-19T07:54:24.572673+0000","last_fullsized":"2024-09-19T07:54:24.572673+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:13.281363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573159+0000","last_change":"2024-09-19T07:54:23.588433+0000","last_active":"2024-09-19T07:54:24.573159+0000","last_peered":"2024-09-19T07:54:24.573159+0000","last_clean":"2024-09-19T07:54:24.573159+0000","last_became_active":"2024-09-19T07:54:23.587803+0000","last_became_peered":"2024-09-19T07:54:23.587803+0000","last_unstale":"2024-09-19T07:54:24.573159+0000","last_undegraded":"2024-09-19T07:54:24.573159+0000","last_fullsized":"2024-09-19T07:54:24.573159+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:56.777102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"129'1","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.563548+0000","last_change":"2024-09-19T07:54:22.555487+0000","last_active":"2024-09-19T07:54:24.563548+0000","last_peered":"2024-09-19T07:54:24.563548+0000","last_clean":"2024-09-19T07:54:24.563548+0000","last_became_active":"2024-09-19T07:54:22.555215+0000","last_became_peered":"2024-09-19T07:54:22.555215+0000","last_unstale":"2024-09-19T07:54:24.563548+0000","last_undegraded":"2024-09-19T07:54:24.563548+0000","last_fullsized":"2024-09-19T07:54:24.563548+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:01.620005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573417+0000","last_change":"2024-09-19T07:54:23.585093+0000","last_active":"2024-09-19T07:54:24.573417+0000","last_peered":"2024-09-19T07:54:24.573417+0000","last_clean":"2024-09-19T07:54:24.573417+0000","last_became_active":"2024-09-19T07:54:23.584837+0000","last_became_peered":"2024-09-19T07:54:23.584837+0000","last_unstale":"2024-09-19T07:54:24.573417+0000","last_undegraded":"2024-09-19T07:54:24.573417+0000","last_fullsized":"2024-09-19T07:54:24.573417+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:53.694112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.563215+0000","last_change":"2024-09-19T07:54:22.564147+0000","last_active":"2024-09-19T07:54:24.563215+0000","last_peered":"2024-09-19T07:54:24.563215+0000","last_clean":"2024-09-19T07:54:24.563215+0000","last_became_active":"2024-09-19T07:54:22.563844+0000","last_became_peered":"2024-09-19T07:54:22.563844+0000","last_unstale":"2024-09-19T07:54:24.563215+0000","last_undegraded":"2024-09-19T07:54:24.563215+0000","last_fullsized":"2024-09-19T07:54:24.563215+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:43.655888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573104+0000","last_change":"2024-09-19T07:54:23.582012+0000","last_active":"2024-09-19T07:54:24.573104+0000","last_peered":"2024-09-19T07:54:24.573104+0000","last_clean":"2024-09-19T07:54:24.573104+0000","last_became_active":"2024-09-19T07:54:23.581143+0000","last_became_peered":"2024-09-19T07:54:23.581143+0000","last_unstale":"2024-09-19T07:54:24.573104+0000","last_undegraded":"2024-09-19T07:54:24.573104+0000","last_fullsized":"2024-09-19T07:54:24.573104+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:31.705022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.561747+0000","last_change":"2024-09-19T07:54:22.558393+0000","last_active":"2024-09-19T07:54:24.561747+0000","last_peered":"2024-09-19T07:54:24.561747+0000","last_clean":"2024-09-19T07:54:24.561747+0000","last_became_active":"2024-09-19T07:54:22.558091+0000","last_became_peered":"2024-09-19T07:54:22.558091+0000","last_unstale":"2024-09-19T07:54:24.561747+0000","last_undegraded":"2024-09-19T07:54:24.561747+0000","last_fullsized":"2024-09-19T07:54:24.561747+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:49.458734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.561892+0000","last_change":"2024-09-19T07:54:23.580255+0000","last_active":"2024-09-19T07:54:24.561892+0000","last_peered":"2024-09-19T07:54:24.561892+0000","last_clean":"2024-09-19T07:54:24.561892+0000","last_became_active":"2024-09-19T07:54:23.579952+0000","last_became_peered":"2024-09-19T07:54:23.579952+0000","last_unstale":"2024-09-19T07:54:24.561892+0000","last_undegraded":"2024-09-19T07:54:24.561892+0000","last_fullsized":"2024-09-19T07:54:24.561892+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:41.287209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.563403+0000","last_change":"2024-09-19T07:54:22.567926+0000","last_active":"2024-09-19T07:54:24.563403+0000","last_peered":"2024-09-19T07:54:24.563403+0000","last_clean":"2024-09-19T07:54:24.563403+0000","last_became_active":"2024-09-19T07:54:22.567617+0000","last_became_peered":"2024-09-19T07:54:22.567617+0000","last_unstale":"2024-09-19T07:54:24.563403+0000","last_undegraded":"2024-09-19T07:54:24.563403+0000","last_fullsized":"2024-09-19T07:54:24.563403+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:41:15.024194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.562007+0000","last_change":"2024-09-19T07:54:23.579433+0000","last_active":"2024-09-19T07:54:24.562007+0000","last_peered":"2024-09-19T07:54:24.562007+0000","last_clean":"2024-09-19T07:54:24.562007+0000","last_became_active":"2024-09-19T07:54:23.579086+0000","last_became_peered":"2024-09-19T07:54:23.579086+0000","last_unstale":"2024-09-19T07:54:24.562007+0000","last_undegraded":"2024-09-19T07:54:24.562007+0000","last_fullsized":"2024-09-19T07:54:24.562007+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:21.371456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573850+0000","last_change":"2024-09-19T07:54:22.558104+0000","last_active":"2024-09-19T07:54:24.573850+0000","last_peered":"2024-09-19T07:54:24.573850+0000","last_clean":"2024-09-19T07:54:24.573850+0000","last_became_active":"2024-09-19T07:54:22.557825+0000","last_became_peered":"2024-09-19T07:54:22.557825+0000","last_unstale":"2024-09-19T07:54:24.573850+0000","last_undegraded":"2024-09-19T07:54:24.573850+0000","last_fullsized":"2024-09-19T07:54:24.573850+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:36:36.185111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.561682+0000","last_change":"2024-09-19T07:54:23.575561+0000","last_active":"2024-09-19T07:54:24.561682+0000","last_peered":"2024-09-19T07:54:24.561682+0000","last_clean":"2024-09-19T07:54:24.561682+0000","last_became_active":"2024-09-19T07:54:23.575268+0000","last_became_peered":"2024-09-19T07:54:23.575268+0000","last_unstale":"2024-09-19T07:54:24.561682+0000","last_undegraded":"2024-09-19T07:54:24.561682+0000","last_fullsized":"2024-09-19T07:54:24.561682+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:02.732821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.561629+0000","last_change":"2024-09-19T07:54:22.560118+0000","last_active":"2024-09-19T07:54:24.561629+0000","last_peered":"2024-09-19T07:54:24.561629+0000","last_clean":"2024-09-19T07:54:24.561629+0000","last_became_active":"2024-09-19T07:54:22.559838+0000","last_became_peered":"2024-09-19T07:54:22.559838+0000","last_unstale":"2024-09-19T07:54:24.561629+0000","last_undegraded":"2024-09-19T07:54:24.561629+0000","last_fullsized":"2024-09-19T07:54:24.561629+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:48.184168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.574100+0000","last_change":"2024-09-19T07:54:23.586194+0000","last_active":"2024-09-19T07:54:24.574100+0000","last_peered":"2024-09-19T07:54:24.574100+0000","last_clean":"2024-09-19T07:54:24.574100+0000","last_became_active":"2024-09-19T07:54:23.585946+0000","last_became_peered":"2024-09-19T07:54:23.585946+0000","last_unstale":"2024-09-19T07:54:24.574100+0000","last_undegraded":"2024-09-19T07:54:24.574100+0000","last_fullsized":"2024-09-19T07:54:24.574100+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:18:21.447818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573073+0000","last_change":"2024-09-19T07:54:22.562538+0000","last_active":"2024-09-19T07:54:24.573073+0000","last_peered":"2024-09-19T07:54:24.573073+0000","last_clean":"2024-09-19T07:54:24.573073+0000","last_became_active":"2024-09-19T07:54:22.562257+0000","last_became_peered":"2024-09-19T07:54:22.562257+0000","last_unstale":"2024-09-19T07:54:24.573073+0000","last_undegraded":"2024-09-19T07:54:24.573073+0000","last_fullsized":"2024-09-19T07:54:24.573073+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:19.547633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573773+0000","last_change":"2024-09-19T07:54:23.588887+0000","last_active":"2024-09-19T07:54:24.573773+0000","last_peered":"2024-09-19T07:54:24.573773+0000","last_clean":"2024-09-19T07:54:24.573773+0000","last_became_active":"2024-09-19T07:54:23.588623+0000","last_became_peered":"2024-09-19T07:54:23.588623+0000","last_unstale":"2024-09-19T07:54:24.573773+0000","last_undegraded":"2024-09-19T07:54:24.573773+0000","last_fullsized":"2024-09-19T07:54:24.573773+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:22.540405+0000","last_clean_scrub_stamp":"2024-09-19T07:54:22.540405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:14.329860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573601+0000","last_change":"2024-09-19T07:54:22.565171+0000","last_active":"2024-09-19T07:54:24.573601+0000","last_peered":"2024-09-19T07:54:24.573601+0000","last_clean":"2024-09-19T07:54:24.573601+0000","last_became_active":"2024-09-19T07:54:22.564876+0000","last_became_peered":"2024-09-19T07:54:22.564876+0000","last_unstale":"2024-09-19T07:54:24.573601+0000","last_undegraded":"2024-09-19T07:54:24.573601+0000","last_fullsized":"2024-09-19T07:54:24.573601+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:54:21.535757+0000","last_clean_scrub_stamp":"2024-09-19T07:54:21.535757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:36.291156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":366,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-19T07:54:24.573082+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:54:24.573082+0000","last_peered":"2024-09-19T07:54:24.573082+0000","last_clean":"2024-09-19T07:54:24.573082+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:54:24.573082+0000","last_undegraded":"2024-09-19T07:54:24.573082+0000","last_fullsized":"2024-09-19T07:54:24.573082+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509582,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83272,"kb_used_data":2164,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288568,"statfs":{"total":96636764160,"available":96551493632,"internally_reserved":0,"allocated":2215936,"data_stored":1337743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509581,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104956,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266884,"statfs":{"total":96636764160,"available":96529289216,"internally_reserved":0,"allocated":1613824,"data_stored":746345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509582,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72960,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298880,"statfs":{"total":96636764160,"available":96562053120,"internally_reserved":0,"allocated":1609728,"data_stored":745695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86836,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285004,"statfs":{"total":96636764160,"available":96547844096,"internally_reserved":0,"allocated":1605632,"data_stored":745673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509581,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105476,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266364,"statfs":{"total":96636764160,"available":96528756736,"internally_reserved":0,"allocated":2207744,"data_stored":1336097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542285,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118796,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253044,"statfs":{"total":96636764160,"available":96515117056,"internally_reserved":0,"allocated":1622016,"data_stored":747355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542285,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82744,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94289096,"statfs":{"total":96636764160,"available":96552034304,"internally_reserved":0,"allocated":1609728,"data_stored":745743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574990,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109052,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262788,"statfs":{"total":96636764160,"available":96525094912,"internally_reserved":0,"allocated":1613824,"data_stored":746289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:54:31.304 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T07:54:31.304 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T07:54:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.450+0000 7f989e959640 1 Processor -- start 2024-09-19T07:54:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.450+0000 7f989e959640 1 -- start start 2024-09-19T07:54:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.451+0000 7f989e959640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f989815e0a0 0x7f9898162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.451+0000 7f989e959640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f98980b5280 0x7f98980d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.451+0000 7f989e959640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f989815f250 con 0x7f98980d43e0 2024-09-19T07:54:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.451+0000 7f989e959640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f98980d4d60 con 0x7f989815e0a0 2024-09-19T07:54:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.451+0000 7f989e959640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f989815f530 con 0x7f98980b5280 2024-09-19T07:54:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.451+0000 7f9897fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f989815e0a0 0x7f9898162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.451+0000 7f9897fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f989815e0a0 0x7f9898162a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48508/0 (socket says 172.21.15.16:48508) 2024-09-19T07:54:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.451+0000 7f9897fff640 1 -- 172.21.15.16:0/1298696854 learned_addr learned my addr 172.21.15.16:0/1298696854 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.451+0000 7f989cef6640 1 --2- 172.21.15.16:0/1298696854 >> v2:172.21.15.146:3300/0 conn(0x7f98980b5280 0x7f98980d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.451+0000 7f9897fff640 1 -- 172.21.15.16:0/1298696854 >> v2:172.21.15.146:3300/0 conn(0x7f98980b5280 msgr2=0x7f98980d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.451+0000 7f9897fff640 1 --2- 172.21.15.16:0/1298696854 >> v2:172.21.15.146:3300/0 conn(0x7f98980b5280 0x7f98980d4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.451+0000 7f9897fff640 1 -- 172.21.15.16:0/1298696854 >> v1:172.21.15.146:6789/0 conn(0x7f98980d43e0 legacy=0x7f98980b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.451+0000 7f9897fff640 1 -- 172.21.15.16:0/1298696854 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989815f810 con 0x7f989815e0a0 2024-09-19T07:54:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.451+0000 7f989cef6640 1 --2- 172.21.15.16:0/1298696854 >> v2:172.21.15.146:3300/0 conn(0x7f98980b5280 0x7f98980d4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.451+0000 7f9897fff640 1 --2- 172.21.15.16:0/1298696854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f989815e0a0 0x7f9898162a70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f989809ddf0 tx=0x7f988802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4029907bd12ceaf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:31.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.451+0000 7f9896ffd640 1 -- 172.21.15.16:0/1298696854 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888037070 con 0x7f989815e0a0 2024-09-19T07:54:31.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.451+0000 7f9896ffd640 1 -- 172.21.15.16:0/1298696854 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9888032440 con 0x7f989815e0a0 2024-09-19T07:54:31.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.451+0000 7f9896ffd640 1 -- 172.21.15.16:0/1298696854 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888032810 con 0x7f989815e0a0 2024-09-19T07:54:31.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.453+0000 7f989e959640 1 -- 172.21.15.16:0/1298696854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f989815e0a0 msgr2=0x7f9898162a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:31.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.453+0000 7f989e959640 1 --2- 172.21.15.16:0/1298696854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f989815e0a0 0x7f9898162a70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f989809ddf0 tx=0x7f988802a230 comp rx=0 tx=0).stop 2024-09-19T07:54:31.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.453+0000 7f989e959640 1 -- 172.21.15.16:0/1298696854 shutdown_connections 2024-09-19T07:54:31.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.453+0000 7f989e959640 1 --2- 172.21.15.16:0/1298696854 >> v2:172.21.15.146:3300/0 conn(0x7f98980b5280 0x7f98980d4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.453+0000 7f989e959640 1 --2- 172.21.15.16:0/1298696854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f989815e0a0 0x7f9898162a70 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.453+0000 7f989e959640 1 -- 172.21.15.16:0/1298696854 >> 172.21.15.16:0/1298696854 conn(0x7f989809d770 msgr2=0x7f989808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:31.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.453+0000 7f989e959640 1 -- 172.21.15.16:0/1298696854 shutdown_connections 2024-09-19T07:54:31.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.453+0000 7f989e959640 1 -- 172.21.15.16:0/1298696854 wait complete. 2024-09-19T07:54:31.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.454+0000 7f989e959640 1 Processor -- start 2024-09-19T07:54:31.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.454+0000 7f989e959640 1 -- start start 2024-09-19T07:54:31.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.454+0000 7f989e959640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98980b5280 0x7f989815d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:31.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.454+0000 7f989e959640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98980d43e0 0x7f9898160dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:31.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.454+0000 7f989e959640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f989815e0a0 0x7f9898161300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:31.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.454+0000 7f989e959640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f989815e600 con 0x7f98980b5280 2024-09-19T07:54:31.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.454+0000 7f989e959640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f989815e480 con 0x7f98980d43e0 2024-09-19T07:54:31.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.454+0000 7f989e959640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f989815e780 con 0x7f989815e0a0 2024-09-19T07:54:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.454+0000 7f9897fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98980b5280 0x7f989815d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.454+0000 7f9897fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98980b5280 0x7f989815d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48524/0 (socket says 172.21.15.16:48524) 2024-09-19T07:54:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.454+0000 7f9897fff640 1 -- 172.21.15.16:0/476465465 learned_addr learned my addr 172.21.15.16:0/476465465 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.454+0000 7f98977fe640 1 --2- 172.21.15.16:0/476465465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98980d43e0 0x7f9898160dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.454+0000 7f989cef6640 1 --2- 172.21.15.16:0/476465465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f989815e0a0 0x7f9898161300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.454+0000 7f9897fff640 1 -- 172.21.15.16:0/476465465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f989815e0a0 msgr2=0x7f9898161300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.454+0000 7f9897fff640 1 --2- 172.21.15.16:0/476465465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f989815e0a0 0x7f9898161300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.454+0000 7f9897fff640 1 -- 172.21.15.16:0/476465465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98980d43e0 msgr2=0x7f9898160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.454+0000 7f9897fff640 1 --2- 172.21.15.16:0/476465465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98980d43e0 0x7f9898160dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.454+0000 7f9897fff640 1 -- 172.21.15.16:0/476465465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98981f76d0 con 0x7f98980b5280 2024-09-19T07:54:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.454+0000 7f989cef6640 1 --2- 172.21.15.16:0/476465465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f989815e0a0 0x7f9898161300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:31.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.456+0000 7f9897fff640 1 --2- 172.21.15.16:0/476465465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98980b5280 0x7f989815d9f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f9888000c00 tx=0x7f98880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:31.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.456+0000 7f9894ff9640 1 -- 172.21.15.16:0/476465465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888037040 con 0x7f98980b5280 2024-09-19T07:54:31.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.456+0000 7f9894ff9640 1 -- 172.21.15.16:0/476465465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9888042070 con 0x7f98980b5280 2024-09-19T07:54:31.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.456+0000 7f9894ff9640 1 -- 172.21.15.16:0/476465465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888030bb0 con 0x7f98980b5280 2024-09-19T07:54:31.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.456+0000 7f989e959640 1 -- 172.21.15.16:0/476465465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98980cd9c0 con 0x7f98980b5280 2024-09-19T07:54:31.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.456+0000 7f989e959640 1 -- 172.21.15.16:0/476465465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98981f7970 con 0x7f98980b5280 2024-09-19T07:54:31.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.457+0000 7f989e959640 1 -- 172.21.15.16:0/476465465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98980ba3d0 con 0x7f98980b5280 2024-09-19T07:54:31.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.457+0000 7f9894ff9640 1 -- 172.21.15.16:0/476465465 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9888047460 con 0x7f98980b5280 2024-09-19T07:54:31.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.458+0000 7f9894ff9640 1 --2- 172.21.15.16:0/476465465 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f988406c2b0 0x7f988406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:31.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.458+0000 7f9894ff9640 1 -- 172.21.15.16:0/476465465 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7f98880ac210 con 0x7f98980b5280 2024-09-19T07:54:31.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.458+0000 7f98977fe640 1 --2- 172.21.15.16:0/476465465 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f988406c2b0 0x7f988406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:31.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.458+0000 7f98977fe640 1 --2- 172.21.15.16:0/476465465 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f988406c2b0 0x7f988406e770 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f98980d1ca0 tx=0x7f988c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.462+0000 7f9894ff9640 1 -- 172.21.15.16:0/476465465 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9888045360 con 0x7f98980b5280 2024-09-19T07:54:31.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.706+0000 7f989e959640 1 -- 172.21.15.16:0/476465465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f98980d4800 con 0x7f98980b5280 2024-09-19T07:54:31.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.708+0000 7f9894ff9640 1 -- 172.21.15.16:0/476465465 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f988807a7f0 con 0x7f98980b5280 2024-09-19T07:54:31.708 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-19T07:54:31.708 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA== 2024-09-19T07:54:31.708 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-19T07:54:31.708 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-19T07:54:31.708 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T07:54:31.709 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-19T07:54:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.711+0000 7f989e959640 1 -- 172.21.15.16:0/476465465 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f988406c2b0 msgr2=0x7f988406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.711+0000 7f989e959640 1 --2- 172.21.15.16:0/476465465 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f988406c2b0 0x7f988406e770 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f98980d1ca0 tx=0x7f988c009040 comp rx=0 tx=0).stop 2024-09-19T07:54:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.711+0000 7f989e959640 1 -- 172.21.15.16:0/476465465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98980b5280 msgr2=0x7f989815d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.711+0000 7f989e959640 1 --2- 172.21.15.16:0/476465465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98980b5280 0x7f989815d9f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f9888000c00 tx=0x7f98880046c0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.711+0000 7f989e959640 1 -- 172.21.15.16:0/476465465 shutdown_connections 2024-09-19T07:54:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.711+0000 7f989e959640 1 --2- 172.21.15.16:0/476465465 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f988406c2b0 0x7f988406e770 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.711+0000 7f989e959640 1 --2- 172.21.15.16:0/476465465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f989815e0a0 0x7f9898161300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.711+0000 7f989e959640 1 --2- 172.21.15.16:0/476465465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98980d43e0 0x7f9898160dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.711+0000 7f989e959640 1 --2- 172.21.15.16:0/476465465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98980b5280 0x7f989815d9f0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.711+0000 7f989e959640 1 -- 172.21.15.16:0/476465465 >> 172.21.15.16:0/476465465 conn(0x7f989809d770 msgr2=0x7f98980b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.711+0000 7f989e959640 1 -- 172.21.15.16:0/476465465 shutdown_connections 2024-09-19T07:54:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.711+0000 7f989e959640 1 -- 172.21.15.16:0/476465465 wait complete. 2024-09-19T07:54:31.722 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:54:31.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.823+0000 7f247c31a640 1 Processor -- start 2024-09-19T07:54:31.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.823+0000 7f247c31a640 1 -- start start 2024-09-19T07:54:31.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.824+0000 7f247c31a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f247415f0d0 0x7f247415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:31.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.824+0000 7f247c31a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f24740d0070 0x7f24740d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:31.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.824+0000 7f247c31a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24740cf2b0 con 0x7f24740d1940 2024-09-19T07:54:31.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.824+0000 7f247c31a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f24740cf590 con 0x7f24740d0070 2024-09-19T07:54:31.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.824+0000 7f247c31a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f24740cf870 con 0x7f247415f0d0 2024-09-19T07:54:31.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.825+0000 7f247a0b6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f247415f0d0 0x7f247415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:31.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.825+0000 7f24798b5640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f24740d0070 0x7f24740d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:31.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.825+0000 7f247a8b7640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f24740d1940 0x7f24740d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:46020/0 (socket says 172.21.15.16:46020) 2024-09-19T07:54:31.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.825+0000 7f247a8b7640 1 -- 172.21.15.16:0/2238426289 learned_addr learned my addr 172.21.15.16:0/2238426289 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:31.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.825+0000 7f24798b5640 1 -- 172.21.15.16:0/2238426289 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f247415f0d0 msgr2=0x7f247415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:31.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.825+0000 7f24798b5640 1 --2- 172.21.15.16:0/2238426289 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f247415f0d0 0x7f247415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.825+0000 7f24798b5640 1 -- 172.21.15.16:0/2238426289 >> v1:172.21.15.146:6789/0 conn(0x7f24740d1940 legacy=0x7f24740d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:31.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.825+0000 7f24798b5640 1 -- 172.21.15.16:0/2238426289 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24740cfb50 con 0x7f24740d0070 2024-09-19T07:54:31.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.825+0000 7f247a0b6640 1 --2- 172.21.15.16:0/2238426289 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f247415f0d0 0x7f247415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:54:31.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.826+0000 7f24798b5640 1 --2- 172.21.15.16:0/2238426289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24740d0070 0x7f24740d1400 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f24640097a0 tx=0x7f246402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22d8750b56771d38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:31.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.826+0000 7f24790b4640 1 -- 172.21.15.16:0/2238426289 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464037070 con 0x7f24740d0070 2024-09-19T07:54:31.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.826+0000 7f24790b4640 1 -- 172.21.15.16:0/2238426289 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f246402ae50 con 0x7f24740d0070 2024-09-19T07:54:31.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.826+0000 7f24790b4640 1 -- 172.21.15.16:0/2238426289 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464032660 con 0x7f24740d0070 2024-09-19T07:54:31.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.827+0000 7f247c31a640 1 -- 172.21.15.16:0/2238426289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24740d0070 msgr2=0x7f24740d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:31.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.827+0000 7f247c31a640 1 --2- 172.21.15.16:0/2238426289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24740d0070 0x7f24740d1400 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f24640097a0 tx=0x7f246402a100 comp rx=0 tx=0).stop 2024-09-19T07:54:31.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.827+0000 7f247c31a640 1 -- 172.21.15.16:0/2238426289 shutdown_connections 2024-09-19T07:54:31.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.827+0000 7f247c31a640 1 --2- 172.21.15.16:0/2238426289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24740d0070 0x7f24740d1400 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.827+0000 7f247c31a640 1 --2- 172.21.15.16:0/2238426289 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f247415f0d0 0x7f247415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.827+0000 7f247c31a640 1 -- 172.21.15.16:0/2238426289 >> 172.21.15.16:0/2238426289 conn(0x7f247409d760 msgr2=0x7f247408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:31.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.827+0000 7f247c31a640 1 -- 172.21.15.16:0/2238426289 shutdown_connections 2024-09-19T07:54:31.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.827+0000 7f247c31a640 1 -- 172.21.15.16:0/2238426289 wait complete. 2024-09-19T07:54:31.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.828+0000 7f247c31a640 1 Processor -- start 2024-09-19T07:54:31.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.828+0000 7f247c31a640 1 -- start start 2024-09-19T07:54:31.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.829+0000 7f247c31a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24740d0070 0x7f24741589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:31.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.829+0000 7f247c31a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24740d1940 0x7f247415af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:31.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.829+0000 7f247c31a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f247415f0d0 0x7f24741577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:31.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.829+0000 7f247c31a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f24740d49c0 con 0x7f24740d0070 2024-09-19T07:54:31.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.829+0000 7f247c31a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f24740d4840 con 0x7f247415f0d0 2024-09-19T07:54:31.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.829+0000 7f247c31a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f24740d4b40 con 0x7f24740d1940 2024-09-19T07:54:31.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.829+0000 7f247a0b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24740d0070 0x7f24741589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:31.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.829+0000 7f247a0b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24740d0070 0x7f24741589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48538/0 (socket says 172.21.15.16:48538) 2024-09-19T07:54:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.829+0000 7f247a0b6640 1 -- 172.21.15.16:0/3613275654 learned_addr learned my addr 172.21.15.16:0/3613275654 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.829+0000 7f24798b5640 1 --2- 172.21.15.16:0/3613275654 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24740d1940 0x7f247415af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.829+0000 7f247a8b7640 1 --2- 172.21.15.16:0/3613275654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f247415f0d0 0x7f24741577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.830+0000 7f247a0b6640 1 -- 172.21.15.16:0/3613275654 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24740d1940 msgr2=0x7f247415af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.830+0000 7f247a0b6640 1 --2- 172.21.15.16:0/3613275654 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24740d1940 0x7f247415af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.830+0000 7f247a0b6640 1 -- 172.21.15.16:0/3613275654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f247415f0d0 msgr2=0x7f24741577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.830+0000 7f247a0b6640 1 --2- 172.21.15.16:0/3613275654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f247415f0d0 0x7f24741577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.830+0000 7f247a0b6640 1 -- 172.21.15.16:0/3613275654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24740ccdb0 con 0x7f24740d0070 2024-09-19T07:54:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.830+0000 7f247a0b6640 1 --2- 172.21.15.16:0/3613275654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24740d0070 0x7f24741589c0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f247000e990 tx=0x7f247000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.830+0000 7f246affd640 1 -- 172.21.15.16:0/3613275654 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470017070 con 0x7f24740d0070 2024-09-19T07:54:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.830+0000 7f246affd640 1 -- 172.21.15.16:0/3613275654 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f247000c9e0 con 0x7f24740d0070 2024-09-19T07:54:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.830+0000 7f247c31a640 1 -- 172.21.15.16:0/3613275654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24740cd090 con 0x7f24740d0070 2024-09-19T07:54:31.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.830+0000 7f246affd640 1 -- 172.21.15.16:0/3613275654 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247000cdb0 con 0x7f24740d0070 2024-09-19T07:54:31.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.830+0000 7f247c31a640 1 -- 172.21.15.16:0/3613275654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24740cd930 con 0x7f24740d0070 2024-09-19T07:54:31.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.831+0000 7f247c31a640 1 -- 172.21.15.16:0/3613275654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24740d1400 con 0x7f24740d0070 2024-09-19T07:54:31.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.831+0000 7f246affd640 1 -- 172.21.15.16:0/3613275654 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f247001a070 con 0x7f24740d0070 2024-09-19T07:54:31.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.832+0000 7f246affd640 1 --2- 172.21.15.16:0/3613275654 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f245806c2b0 0x7f245806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:31.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.832+0000 7f246affd640 1 -- 172.21.15.16:0/3613275654 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7f247008b4b0 con 0x7f24740d0070 2024-09-19T07:54:31.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.832+0000 7f24798b5640 1 --2- 172.21.15.16:0/3613275654 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f245806c2b0 0x7f245806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:31.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.833+0000 7f24798b5640 1 --2- 172.21.15.16:0/3613275654 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f245806c2b0 0x7f245806e770 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2464034000 tx=0x7f246402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:31.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:31.836+0000 7f246affd640 1 -- 172.21.15.16:0/3613275654 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f24700593c0 con 0x7f24740d0070 2024-09-19T07:54:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.048+0000 7f247c31a640 1 -- 172.21.15.16:0/3613275654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f247406cf30 con 0x7f24740d0070 2024-09-19T07:54:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.049+0000 7f246affd640 1 -- 172.21.15.16:0/3613275654 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5219 (secure 0 0 0) 0x7f247005d6c0 con 0x7f24740d0070 2024-09-19T07:54:32.049 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:32.050 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":72,"btime":"2024-09-19T07:54:25:574262+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/2994096322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2994096322},{"type":"v1","addr":"172.21.15.146:6836","nonce":2994096322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7839,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/393564462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":393564462},{"type":"v1","addr":"172.21.15.16:6833","nonce":393564462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":7842,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/631392582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":631392582},{"type":"v1","addr":"172.21.15.16:6835","nonce":631392582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:54:22.949504+0000","modified":"2024-09-19T07:54:24.912891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/79304323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":79304323},{"type":"v1","addr":"172.21.15.146:6837","nonce":79304323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7854,"qdb_cluster":[7854]},"id":9}]} 2024-09-19T07:54:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 72 2024-09-19T07:54:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.051+0000 7f247c31a640 1 -- 172.21.15.16:0/3613275654 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f245806c2b0 msgr2=0x7f245806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.051+0000 7f247c31a640 1 --2- 172.21.15.16:0/3613275654 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f245806c2b0 0x7f245806e770 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2464034000 tx=0x7f246402e000 comp rx=0 tx=0).stop 2024-09-19T07:54:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.051+0000 7f247c31a640 1 -- 172.21.15.16:0/3613275654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24740d0070 msgr2=0x7f24741589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.051+0000 7f247c31a640 1 --2- 172.21.15.16:0/3613275654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24740d0070 0x7f24741589c0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f247000e990 tx=0x7f247000ee50 comp rx=0 tx=0).stop 2024-09-19T07:54:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.051+0000 7f247c31a640 1 -- 172.21.15.16:0/3613275654 shutdown_connections 2024-09-19T07:54:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.051+0000 7f247c31a640 1 --2- 172.21.15.16:0/3613275654 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f245806c2b0 0x7f245806e770 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.051+0000 7f247c31a640 1 --2- 172.21.15.16:0/3613275654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f247415f0d0 0x7f24741577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.051+0000 7f247c31a640 1 --2- 172.21.15.16:0/3613275654 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24740d1940 0x7f247415af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.051+0000 7f247c31a640 1 --2- 172.21.15.16:0/3613275654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24740d0070 0x7f24741589c0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.051+0000 7f247c31a640 1 -- 172.21.15.16:0/3613275654 >> 172.21.15.16:0/3613275654 conn(0x7f247409d760 msgr2=0x7f247406d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.051+0000 7f247c31a640 1 -- 172.21.15.16:0/3613275654 shutdown_connections 2024-09-19T07:54:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.051+0000 7f247c31a640 1 -- 172.21.15.16:0/3613275654 wait complete. 2024-09-19T07:54:32.466 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T07:54:22.949504+0000', 'modified': '2024-09-19T07:54:24.912891+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7854}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7854': {'gid': 7854, 'name': 'd', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.146:6837/79304323', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6835', 'nonce': 79304323}, {'type': 'v1', 'addr': '172.21.15.146:6837', 'nonce': 79304323}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7854, 'qdb_cluster': [7854]} 2024-09-19T07:54:32.466 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T07:54:32.466 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-19T07:54:32.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.571+0000 7fea1ede4640 1 Processor -- start 2024-09-19T07:54:32.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.571+0000 7fea1ede4640 1 -- start start 2024-09-19T07:54:32.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.572+0000 7fea1ede4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea1815c140 0x7fea18160650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.572+0000 7fea1ede4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea180b6650 0x7fea1815d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.572+0000 7fea1ede4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea18160b90 con 0x7fea1806f9d0 2024-09-19T07:54:32.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.572+0000 7fea1ede4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fea1815c520 con 0x7fea1815c140 2024-09-19T07:54:32.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.572+0000 7fea1ede4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fea1815c7f0 con 0x7fea180b6650 2024-09-19T07:54:32.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.572+0000 7fea1cb80640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea1815c140 0x7fea18160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.572+0000 7fea17fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fea1806f9d0 0x7fea180b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35510/0 (socket says 172.21.15.16:35510) 2024-09-19T07:54:32.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.572+0000 7fea17fff640 1 -- 172.21.15.16:0/1585567543 learned_addr learned my addr 172.21.15.16:0/1585567543 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:32.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.572+0000 7fea1d381640 1 --2- 172.21.15.16:0/1585567543 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea180b6650 0x7fea1815d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.572+0000 7fea1cb80640 1 -- 172.21.15.16:0/1585567543 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea180b6650 msgr2=0x7fea1815d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:32.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.572+0000 7fea1cb80640 1 --2- 172.21.15.16:0/1585567543 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea180b6650 0x7fea1815d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.572+0000 7fea1cb80640 1 -- 172.21.15.16:0/1585567543 >> v1:172.21.15.16:6789/0 conn(0x7fea1806f9d0 legacy=0x7fea180b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:32.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.572+0000 7fea1cb80640 1 -- 172.21.15.16:0/1585567543 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1815cad0 con 0x7fea1815c140 2024-09-19T07:54:32.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.573+0000 7fea1cb80640 1 --2- 172.21.15.16:0/1585567543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1815c140 0x7fea18160650 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fea080097a0 tx=0x7fea0802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2afb45f9032a68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:32.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.573+0000 7fea177fe640 1 -- 172.21.15.16:0/1585567543 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08037070 con 0x7fea1815c140 2024-09-19T07:54:32.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.573+0000 7fea177fe640 1 -- 172.21.15.16:0/1585567543 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea0802ad10 con 0x7fea1815c140 2024-09-19T07:54:32.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.573+0000 7fea177fe640 1 -- 172.21.15.16:0/1585567543 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08032660 con 0x7fea1815c140 2024-09-19T07:54:32.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.574+0000 7fea1ede4640 1 -- 172.21.15.16:0/1585567543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1815c140 msgr2=0x7fea18160650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:32.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.574+0000 7fea1ede4640 1 --2- 172.21.15.16:0/1585567543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1815c140 0x7fea18160650 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fea080097a0 tx=0x7fea0802a080 comp rx=0 tx=0).stop 2024-09-19T07:54:32.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.574+0000 7fea1ede4640 1 -- 172.21.15.16:0/1585567543 shutdown_connections 2024-09-19T07:54:32.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.574+0000 7fea1ede4640 1 --2- 172.21.15.16:0/1585567543 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea180b6650 0x7fea1815d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.574+0000 7fea1ede4640 1 --2- 172.21.15.16:0/1585567543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1815c140 0x7fea18160650 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.574+0000 7fea1ede4640 1 -- 172.21.15.16:0/1585567543 >> 172.21.15.16:0/1585567543 conn(0x7fea1806bc10 msgr2=0x7fea1806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:32.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.574+0000 7fea1ede4640 1 -- 172.21.15.16:0/1585567543 shutdown_connections 2024-09-19T07:54:32.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.574+0000 7fea1ede4640 1 -- 172.21.15.16:0/1585567543 wait complete. 2024-09-19T07:54:32.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.576+0000 7fea1ede4640 1 Processor -- start 2024-09-19T07:54:32.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.576+0000 7fea1ede4640 1 -- start start 2024-09-19T07:54:32.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea1ede4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1806f9d0 0x7fea180a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea1ede4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea180b6650 0x7fea180af760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea1ede4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea1815c140 0x7fea18142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea1ede4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fea1815d370 con 0x7fea1806f9d0 2024-09-19T07:54:32.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea1ede4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fea1815d090 con 0x7fea180b6650 2024-09-19T07:54:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea1ede4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fea1815ddd0 con 0x7fea1815c140 2024-09-19T07:54:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea1cb80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1806f9d0 0x7fea180a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea1cb80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1806f9d0 0x7fea180a9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60094/0 (socket says 172.21.15.16:60094) 2024-09-19T07:54:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea1cb80640 1 -- 172.21.15.16:0/3606251167 learned_addr learned my addr 172.21.15.16:0/3606251167 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea17fff640 1 --2- 172.21.15.16:0/3606251167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea180b6650 0x7fea180af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea1d381640 1 --2- 172.21.15.16:0/3606251167 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea1815c140 0x7fea18142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea1cb80640 1 -- 172.21.15.16:0/3606251167 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea1815c140 msgr2=0x7fea18142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea1cb80640 1 --2- 172.21.15.16:0/3606251167 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea1815c140 0x7fea18142e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea1cb80640 1 -- 172.21.15.16:0/3606251167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea180b6650 msgr2=0x7fea180af760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea1cb80640 1 --2- 172.21.15.16:0/3606251167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea180b6650 0x7fea180af760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea1cb80640 1 -- 172.21.15.16:0/3606251167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea180aa380 con 0x7fea1806f9d0 2024-09-19T07:54:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea1cb80640 1 --2- 172.21.15.16:0/3606251167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1806f9d0 0x7fea180a9450 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fea08000c00 tx=0x7fea080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea157fa640 1 -- 172.21.15.16:0/3606251167 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08037040 con 0x7fea1806f9d0 2024-09-19T07:54:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea157fa640 1 -- 172.21.15.16:0/3606251167 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea08004250 con 0x7fea1806f9d0 2024-09-19T07:54:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea157fa640 1 -- 172.21.15.16:0/3606251167 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08003610 con 0x7fea1806f9d0 2024-09-19T07:54:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.577+0000 7fea1ede4640 1 -- 172.21.15.16:0/3606251167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea180a9ae0 con 0x7fea1806f9d0 2024-09-19T07:54:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.578+0000 7fea1ede4640 1 -- 172.21.15.16:0/3606251167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea180aa0a0 con 0x7fea1806f9d0 2024-09-19T07:54:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.580+0000 7fea157fa640 1 -- 172.21.15.16:0/3606251167 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fea0802abd0 con 0x7fea1806f9d0 2024-09-19T07:54:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.580+0000 7fea157fa640 1 --2- 172.21.15.16:0/3606251167 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fea0406c2e0 0x7fea0406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.580+0000 7fea157fa640 1 -- 172.21.15.16:0/3606251167 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7fea080abdf0 con 0x7fea1806f9d0 2024-09-19T07:54:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.580+0000 7fea17fff640 1 --2- 172.21.15.16:0/3606251167 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fea0406c2e0 0x7fea0406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.581+0000 7fea21c59740 1 Processor -- start 2024-09-19T07:54:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.581+0000 7fea21c59740 1 -- start start 2024-09-19T07:54:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.581+0000 7fea17fff640 1 --2- 172.21.15.16:0/3606251167 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fea0406c2e0 0x7fea0406e7a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fea180afe60 tx=0x7fea0c007e20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fea21c59740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd940b120 0x557cd940b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fea21c59740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd9425ae0 0x557cd9427290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fea21c59740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd93dea80 0x557cd94277d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fea21c59740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557cd9343520 con 0x557cd93dea80 2024-09-19T07:54:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fea21c59740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x557cd92eb3e0 con 0x557cd940b120 2024-09-19T07:54:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fea21c59740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x557cd933f4e0 con 0x557cd9425ae0 2024-09-19T07:54:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fea1d381640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd93dea80 0x557cd94277d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fea1d381640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd93dea80 0x557cd94277d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60110/0 (socket says 172.21.15.16:60110) 2024-09-19T07:54:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fea1d381640 1 -- 172.21.15.16:0/1128050847 learned_addr learned my addr 172.21.15.16:0/1128050847 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fea17fff640 1 --2- 172.21.15.16:0/1128050847 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd9425ae0 0x557cd9427290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fea1cb80640 1 --2- 172.21.15.16:0/1128050847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd940b120 0x557cd940b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fea1d381640 1 -- 172.21.15.16:0/1128050847 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd9425ae0 msgr2=0x557cd9427290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fea1d381640 1 --2- 172.21.15.16:0/1128050847 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd9425ae0 0x557cd9427290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fea1d381640 1 -- 172.21.15.16:0/1128050847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd940b120 msgr2=0x557cd940b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fea1d381640 1 --2- 172.21.15.16:0/1128050847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd940b120 0x557cd940b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fea1d381640 1 -- 172.21.15.16:0/1128050847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cd93062f0 con 0x557cd93dea80 2024-09-19T07:54:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fea1d381640 1 --2- 172.21.15.16:0/1128050847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd93dea80 0x557cd94277d0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fea10009f70 tx=0x7fea10003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867d5481827b076e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fe9feffd640 1 -- 172.21.15.16:0/1128050847 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10015420 con 0x557cd93dea80 2024-09-19T07:54:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fe9feffd640 1 -- 172.21.15.16:0/1128050847 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea1001a070 con 0x557cd93dea80 2024-09-19T07:54:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.582+0000 7fe9feffd640 1 -- 172.21.15.16:0/1128050847 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea100157e0 con 0x557cd93dea80 2024-09-19T07:54:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.583+0000 7fea21c59740 1 -- 172.21.15.16:0/1128050847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd93dea80 msgr2=0x557cd94277d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.583+0000 7fea21c59740 1 --2- 172.21.15.16:0/1128050847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd93dea80 0x557cd94277d0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fea10009f70 tx=0x7fea10003e20 comp rx=0 tx=0).stop 2024-09-19T07:54:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.583+0000 7fea21c59740 1 -- 172.21.15.16:0/1128050847 shutdown_connections 2024-09-19T07:54:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.583+0000 7fea21c59740 1 --2- 172.21.15.16:0/1128050847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd93dea80 0x557cd94277d0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.583+0000 7fea21c59740 1 --2- 172.21.15.16:0/1128050847 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd9425ae0 0x557cd9427290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.583+0000 7fea21c59740 1 --2- 172.21.15.16:0/1128050847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd940b120 0x557cd940b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.583+0000 7fea21c59740 1 -- 172.21.15.16:0/1128050847 >> 172.21.15.16:0/1128050847 conn(0x557cd93dda80 msgr2=0x557cd9425f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.583+0000 7fea21c59740 1 -- 172.21.15.16:0/1128050847 shutdown_connections 2024-09-19T07:54:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.583+0000 7fea21c59740 1 -- 172.21.15.16:0/1128050847 wait complete. 2024-09-19T07:54:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.584+0000 7fea21c59740 1 -- start start 2024-09-19T07:54:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.584+0000 7fea21c59740 1 Processor -- start 2024-09-19T07:54:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.584+0000 7fea21c59740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd940b120 0x557cd9406d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.584+0000 7fea21c59740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd9425ae0 0x557cd9421d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.584+0000 7fea21c59740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd94222c0 0x557cd94074a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.584+0000 7fea21c59740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557cd940bee0 con 0x557cd9425ae0 2024-09-19T07:54:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.584+0000 7fea21c59740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x557cd92cc450 con 0x557cd94222c0 2024-09-19T07:54:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.584+0000 7fea21c59740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x557cd9426770 con 0x557cd940b120 2024-09-19T07:54:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.585+0000 7fea21c59740 10 client.0 _ll_register_callbacks cb 0x557cd9055ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.585+0000 7fea17fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd9425ae0 0x557cd9421d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.585+0000 7fea17fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd9425ae0 0x557cd9421d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60116/0 (socket says 172.21.15.16:60116) 2024-09-19T07:54:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.585+0000 7fea17fff640 1 -- 172.21.15.16:0/308726350 learned_addr learned my addr 172.21.15.16:0/308726350 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.585+0000 7fea1cb80640 1 --2- 172.21.15.16:0/308726350 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd940b120 0x557cd9406d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.585+0000 7fea1d381640 1 --2- 172.21.15.16:0/308726350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd94222c0 0x557cd94074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.585+0000 7fea17fff640 1 -- 172.21.15.16:0/308726350 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd940b120 msgr2=0x557cd9406d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:32.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.585+0000 7fea17fff640 1 --2- 172.21.15.16:0/308726350 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd940b120 0x557cd9406d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.585+0000 7fea17fff640 1 -- 172.21.15.16:0/308726350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd94222c0 msgr2=0x557cd94074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:32.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.585+0000 7fea17fff640 1 --2- 172.21.15.16:0/308726350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd94222c0 0x557cd94074a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.585+0000 7fea17fff640 1 -- 172.21.15.16:0/308726350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cd90562d0 con 0x557cd9425ae0 2024-09-19T07:54:32.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.585+0000 7fea17fff640 1 --2- 172.21.15.16:0/308726350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd9425ae0 0x557cd9421d80 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fea0c007fd0 tx=0x7fea0c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15f1c3890f268ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:32.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.585+0000 7fe9f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:54:32.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.585+0000 7fe9f77fe640 1 -- 172.21.15.16:0/308726350 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c012a30 con 0x557cd9425ae0 2024-09-19T07:54:32.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.585+0000 7fe9f77fe640 1 -- 172.21.15.16:0/308726350 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7fea0c012bd0 con 0x557cd9425ae0 2024-09-19T07:54:32.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.585+0000 7fea21c59740 1 -- 172.21.15.16:0/308726350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea1001a070 con 0x557cd9425ae0 2024-09-19T07:54:32.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.585+0000 7fe9f77fe640 1 -- 172.21.15.16:0/308726350 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fea0c02e050 con 0x557cd9425ae0 2024-09-19T07:54:32.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.585+0000 7fea21c59740 10 client.7986 fetch_fsmap learned FSMap version 72 2024-09-19T07:54:32.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.585+0000 7fea21c59740 1 -- 172.21.15.16:0/308726350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x557cd91206b0 con 0x557cd9425ae0 2024-09-19T07:54:32.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.586+0000 7fe9f77fe640 1 -- 172.21.15.16:0/308726350 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea1001a070 con 0x557cd9425ae0 2024-09-19T07:54:32.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.586+0000 7fe9f77fe640 1 -- 172.21.15.16:0/308726350 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 72) ==== 3502+0+0 (secure 0 0 0) 0x7fea0c031020 con 0x557cd9425ae0 2024-09-19T07:54:32.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.586+0000 7fea21c59740 10 client.7986 fetch_fsmap finished waiting for FSMap version 72 2024-09-19T07:54:32.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.586+0000 7fea21c59740 10 client.7986 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-19T07:54:32.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.586+0000 7fea21c59740 20 client.7986 populate_metadata read hostname 'smithi016' 2024-09-19T07:54:32.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.586+0000 7fea21c59740 1 --2- 172.21.15.16:0/308726350 >> [v2:172.21.15.146:6835/79304323,v1:172.21.15.146:6837/79304323] conn(0x557cd940cde0 0x557cd94207a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.586+0000 7fea21c59740 4 client.7986 mds_command: new command op to 7854 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:54:32.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.586+0000 7fea21c59740 1 -- 172.21.15.16:0/308726350 --> [v2:172.21.15.146:6835/79304323,v1:172.21.15.146:6837/79304323] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cd9350c50 con 0x557cd940cde0 2024-09-19T07:54:32.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.586+0000 7fea1cb80640 1 --2- 172.21.15.16:0/308726350 >> [v2:172.21.15.146:6835/79304323,v1:172.21.15.146:6837/79304323] conn(0x557cd940cde0 0x557cd94207a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.586+0000 7fe9f77fe640 10 client.7986 ms_handle_connect on v2:172.21.15.146:6835/79304323 2024-09-19T07:54:32.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.586+0000 7fea1cb80640 1 --2- 172.21.15.16:0/308726350 >> [v2:172.21.15.146:6835/79304323,v1:172.21.15.146:6837/79304323] conn(0x557cd940cde0 0x557cd94207a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=890897ff0248970 server_cookie=717e3d3267d044c2 in_seq=0 out_seq=0 2024-09-19T07:54:32.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.587+0000 7fe9f77fe640 1 -- 172.21.15.16:0/308726350 <== mds.0 v2:172.21.15.146:6835/79304323 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557cd9350c50 con 0x557cd940cde0 2024-09-19T07:54:32.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.587+0000 7fe9f77fe640 10 client.7986 handle_command_reply: tid=0 2024-09-19T07:54:32.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.587+0000 7fea21c59740 1 client.7986 shutdown 2024-09-19T07:54:32.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.587+0000 7fea21c59740 1 -- 172.21.15.16:0/308726350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd9425ae0 msgr2=0x557cd9421d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:32.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.587+0000 7fea21c59740 1 --2- 172.21.15.16:0/308726350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd9425ae0 0x557cd9421d80 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fea0c007fd0 tx=0x7fea0c000ed0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.587+0000 7fea21c59740 1 -- 172.21.15.16:0/308726350 shutdown_connections 2024-09-19T07:54:32.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.587+0000 7fea21c59740 1 --2- 172.21.15.16:0/308726350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd94222c0 0x557cd94074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.587+0000 7fea21c59740 1 --2- 172.21.15.16:0/308726350 >> [v2:172.21.15.146:6835/79304323,v1:172.21.15.146:6837/79304323] conn(0x557cd940cde0 0x557cd94207a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.587+0000 7fea21c59740 1 --2- 172.21.15.16:0/308726350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd9425ae0 0x557cd9421d80 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.587+0000 7fea21c59740 1 --2- 172.21.15.16:0/308726350 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd940b120 0x557cd9406d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.587+0000 7fe9f77fe640 0 client.7986 ms_handle_reset on v2:172.21.15.146:6835/79304323 2024-09-19T07:54:32.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.587+0000 7fea21c59740 1 -- 172.21.15.16:0/308726350 >> 172.21.15.16:0/308726350 conn(0x557cd93dea80 msgr2=0x557cd9429360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:32.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.587+0000 7fea21c59740 1 -- 172.21.15.16:0/308726350 shutdown_connections 2024-09-19T07:54:32.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.587+0000 7fea21c59740 1 -- 172.21.15.16:0/308726350 wait complete. 2024-09-19T07:54:32.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.588+0000 7fea21c59740 20 client.7986 trim_cache size 0 max 16384 2024-09-19T07:54:32.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.605+0000 7fea21c59740 1 Processor -- start 2024-09-19T07:54:32.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.606+0000 7fea21c59740 1 -- start start 2024-09-19T07:54:32.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.606+0000 7fea21c59740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd940b120 0x557cd94a3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.606+0000 7fea21c59740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd94222c0 0x557cd9424ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.606+0000 7fea21c59740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd9425ae0 0x557cd9425400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.606+0000 7fea21c59740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fea180b5390 con 0x557cd94222c0 2024-09-19T07:54:32.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.606+0000 7fea21c59740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fea181449a0 con 0x557cd9425ae0 2024-09-19T07:54:32.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.606+0000 7fea21c59740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fea18144ff0 con 0x557cd940b120 2024-09-19T07:54:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.606+0000 7fea17fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd94222c0 0x557cd9424ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.606+0000 7fea17fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd94222c0 0x557cd9424ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60128/0 (socket says 172.21.15.16:60128) 2024-09-19T07:54:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.606+0000 7fea17fff640 1 -- 172.21.15.16:0/872396502 learned_addr learned my addr 172.21.15.16:0/872396502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.606+0000 7fea1d381640 1 --2- 172.21.15.16:0/872396502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd9425ae0 0x557cd9425400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.606+0000 7fea1cb80640 1 --2- 172.21.15.16:0/872396502 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd940b120 0x557cd94a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.607+0000 7fea17fff640 1 -- 172.21.15.16:0/872396502 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd940b120 msgr2=0x557cd94a3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.607+0000 7fea17fff640 1 --2- 172.21.15.16:0/872396502 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd940b120 0x557cd94a3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.607+0000 7fea17fff640 1 -- 172.21.15.16:0/872396502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd9425ae0 msgr2=0x557cd9425400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.607+0000 7fea17fff640 1 --2- 172.21.15.16:0/872396502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd9425ae0 0x557cd9425400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.607+0000 7fea17fff640 1 -- 172.21.15.16:0/872396502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cd92c9f70 con 0x557cd94222c0 2024-09-19T07:54:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.607+0000 7fea1cb80640 1 --2- 172.21.15.16:0/872396502 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd940b120 0x557cd94a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.607+0000 7fea17fff640 1 --2- 172.21.15.16:0/872396502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd94222c0 0x557cd9424ec0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0214a0 tx=0x7fea0c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e22983069e236ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.607+0000 7fe9fe7fc640 1 -- 172.21.15.16:0/872396502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c012a30 con 0x557cd94222c0 2024-09-19T07:54:32.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.607+0000 7fe9fe7fc640 1 -- 172.21.15.16:0/872396502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea0c026070 con 0x557cd94222c0 2024-09-19T07:54:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.607+0000 7fe9fe7fc640 1 -- 172.21.15.16:0/872396502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c012df0 con 0x557cd94222c0 2024-09-19T07:54:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.607+0000 7fea21c59740 1 -- 172.21.15.16:0/872396502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd94222c0 msgr2=0x557cd9424ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.607+0000 7fea21c59740 1 --2- 172.21.15.16:0/872396502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd94222c0 0x557cd9424ec0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0214a0 tx=0x7fea0c021000 comp rx=0 tx=0).stop 2024-09-19T07:54:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.607+0000 7fea21c59740 1 -- 172.21.15.16:0/872396502 shutdown_connections 2024-09-19T07:54:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.607+0000 7fea21c59740 1 --2- 172.21.15.16:0/872396502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd9425ae0 0x557cd9425400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.607+0000 7fea21c59740 1 --2- 172.21.15.16:0/872396502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd94222c0 0x557cd9424ec0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.607+0000 7fea21c59740 1 --2- 172.21.15.16:0/872396502 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd940b120 0x557cd94a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.607+0000 7fea21c59740 1 -- 172.21.15.16:0/872396502 >> 172.21.15.16:0/872396502 conn(0x557cd93dea80 msgr2=0x557cd93e1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.607+0000 7fea21c59740 1 -- 172.21.15.16:0/872396502 shutdown_connections 2024-09-19T07:54:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.607+0000 7fea21c59740 1 -- 172.21.15.16:0/872396502 wait complete. 2024-09-19T07:54:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.608+0000 7fea21c59740 1 -- start start 2024-09-19T07:54:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.608+0000 7fea21c59740 1 Processor -- start 2024-09-19T07:54:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.609+0000 7fea21c59740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd940b120 0x557cd9420fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.609+0000 7fea21c59740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd94222c0 0x557cd949f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.609+0000 7fea21c59740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd9425ae0 0x557cd94ace30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.609+0000 7fea21c59740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557cd940cc60 con 0x557cd9425ae0 2024-09-19T07:54:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.609+0000 7fea21c59740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x557cd93df730 con 0x557cd940b120 2024-09-19T07:54:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.609+0000 7fea21c59740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x557cd940d280 con 0x557cd94222c0 2024-09-19T07:54:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.609+0000 7fea21c59740 10 client.0 _ll_register_callbacks cb 0x7fea0c001c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.609+0000 7fea1d381640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd9425ae0 0x557cd94ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.609+0000 7fea1d381640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd9425ae0 0x557cd94ace30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60132/0 (socket says 172.21.15.16:60132) 2024-09-19T07:54:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.609+0000 7fea1d381640 1 -- 172.21.15.16:0/375985847 learned_addr learned my addr 172.21.15.16:0/375985847 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.609+0000 7fea1cb80640 1 --2- 172.21.15.16:0/375985847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd940b120 0x557cd9420fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.609+0000 7fea17fff640 1 --2- 172.21.15.16:0/375985847 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd94222c0 0x557cd949f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.609+0000 7fea1d381640 1 -- 172.21.15.16:0/375985847 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd94222c0 msgr2=0x557cd949f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.609+0000 7fea1d381640 1 --2- 172.21.15.16:0/375985847 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd94222c0 0x557cd949f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.609+0000 7fea1d381640 1 -- 172.21.15.16:0/375985847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd940b120 msgr2=0x557cd9420fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:32.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.609+0000 7fea1d381640 1 --2- 172.21.15.16:0/375985847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd940b120 0x557cd9420fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.609+0000 7fea1d381640 1 -- 172.21.15.16:0/375985847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cd9429090 con 0x557cd9425ae0 2024-09-19T07:54:32.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.609+0000 7fea1d381640 1 --2- 172.21.15.16:0/375985847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd9425ae0 0x557cd94ace30 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fea10017040 tx=0x7fea10004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f073e81d514bf9cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:32.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.609+0000 7fe9f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:54:32.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.610+0000 7fe9f77fe640 1 -- 172.21.15.16:0/375985847 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10015630 con 0x557cd9425ae0 2024-09-19T07:54:32.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.610+0000 7fe9f77fe640 1 -- 172.21.15.16:0/375985847 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7fea10029c80 con 0x557cd9425ae0 2024-09-19T07:54:32.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.610+0000 7fea21c59740 1 -- 172.21.15.16:0/375985847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea0c026070 con 0x557cd9425ae0 2024-09-19T07:54:32.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.610+0000 7fe9f77fe640 1 -- 172.21.15.16:0/375985847 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fea10030050 con 0x557cd9425ae0 2024-09-19T07:54:32.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.610+0000 7fea21c59740 10 client.7992 fetch_fsmap learned FSMap version 72 2024-09-19T07:54:32.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.610+0000 7fea21c59740 1 -- 172.21.15.16:0/375985847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x557cd9423a20 con 0x557cd9425ae0 2024-09-19T07:54:32.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.610+0000 7fe9f77fe640 1 -- 172.21.15.16:0/375985847 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea0c026070 con 0x557cd9425ae0 2024-09-19T07:54:32.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.610+0000 7fe9f77fe640 1 -- 172.21.15.16:0/375985847 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 72) ==== 3502+0+0 (secure 0 0 0) 0x7fea10033020 con 0x557cd9425ae0 2024-09-19T07:54:32.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.610+0000 7fea21c59740 10 client.7992 fetch_fsmap finished waiting for FSMap version 72 2024-09-19T07:54:32.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.610+0000 7fea21c59740 10 client.7992 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-19T07:54:32.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.610+0000 7fea21c59740 20 client.7992 populate_metadata read hostname 'smithi016' 2024-09-19T07:54:32.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.610+0000 7fea21c59740 1 --2- 172.21.15.16:0/375985847 >> [v2:172.21.15.146:6835/79304323,v1:172.21.15.146:6837/79304323] conn(0x557cd940df20 0x557cd9427060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.610+0000 7fea21c59740 4 client.7992 mds_command: new command op to 7854 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:54:32.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.610+0000 7fea21c59740 1 -- 172.21.15.16:0/375985847 --> [v2:172.21.15.146:6835/79304323,v1:172.21.15.146:6837/79304323] -- command(tid 0: {"prefix": "status"}) -- 0x557cd934afa0 con 0x557cd940df20 2024-09-19T07:54:32.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.610+0000 7fea1cb80640 1 --2- 172.21.15.16:0/375985847 >> [v2:172.21.15.146:6835/79304323,v1:172.21.15.146:6837/79304323] conn(0x557cd940df20 0x557cd9427060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.611+0000 7fe9f77fe640 10 client.7992 ms_handle_connect on v2:172.21.15.146:6835/79304323 2024-09-19T07:54:32.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.611+0000 7fea1cb80640 1 --2- 172.21.15.16:0/375985847 >> [v2:172.21.15.146:6835/79304323,v1:172.21.15.146:6837/79304323] conn(0x557cd940df20 0x557cd9427060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ee5ecdd76ae2ea6 server_cookie=27781d76dfccd389 in_seq=0 out_seq=0 2024-09-19T07:54:32.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.611+0000 7fe9f77fe640 1 -- 172.21.15.16:0/375985847 <== mds.0 v2:172.21.15.146:6835/79304323 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557cd934afa0 con 0x557cd940df20 2024-09-19T07:54:32.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.611+0000 7fe9f77fe640 10 client.7992 handle_command_reply: tid=0 2024-09-19T07:54:32.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.611+0000 7fea21c59740 1 client.7992 shutdown 2024-09-19T07:54:32.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.611+0000 7fea21c59740 1 -- 172.21.15.16:0/375985847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd9425ae0 msgr2=0x557cd94ace30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:32.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.611+0000 7fea21c59740 1 --2- 172.21.15.16:0/375985847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd9425ae0 0x557cd94ace30 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fea10017040 tx=0x7fea10004ab0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.611+0000 7fea21c59740 1 -- 172.21.15.16:0/375985847 shutdown_connections 2024-09-19T07:54:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.611+0000 7fea21c59740 1 --2- 172.21.15.16:0/375985847 >> [v2:172.21.15.146:6835/79304323,v1:172.21.15.146:6837/79304323] conn(0x557cd940df20 0x557cd9427060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.611+0000 7fea21c59740 1 --2- 172.21.15.16:0/375985847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cd9425ae0 0x557cd94ace30 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.611+0000 7fea21c59740 1 --2- 172.21.15.16:0/375985847 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557cd94222c0 0x557cd949f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.611+0000 7fe9f77fe640 0 client.7992 ms_handle_reset on v2:172.21.15.146:6835/79304323 2024-09-19T07:54:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.611+0000 7fea21c59740 1 --2- 172.21.15.16:0/375985847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557cd940b120 0x557cd9420fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.611+0000 7fea21c59740 1 -- 172.21.15.16:0/375985847 >> 172.21.15.16:0/375985847 conn(0x557cd93dea80 msgr2=0x557cd9423ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.611+0000 7fea21c59740 1 -- 172.21.15.16:0/375985847 shutdown_connections 2024-09-19T07:54:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.611+0000 7fea21c59740 1 -- 172.21.15.16:0/375985847 wait complete. 2024-09-19T07:54:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.611+0000 7fea21c59740 20 client.7992 trim_cache size 0 max 16384 2024-09-19T07:54:32.623 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:54:32.623 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb", 2024-09-19T07:54:32.623 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-19T07:54:32.623 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7854, 2024-09-19T07:54:32.623 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T07:54:32.623 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T07:54:32.623 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T07:54:32.623 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 9.0527121749999999, 2024-09-19T07:54:32.623 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 72, 2024-09-19T07:54:32.623 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 129, 2024-09-19T07:54:32.623 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 129, 2024-09-19T07:54:32.623 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 14.084756188 2024-09-19T07:54:32.623 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:54:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.612+0000 7fea1ede4640 1 -- 172.21.15.16:0/3606251167 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fea0406c2e0 msgr2=0x7fea0406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.612+0000 7fea1ede4640 1 --2- 172.21.15.16:0/3606251167 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fea0406c2e0 0x7fea0406e7a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fea180afe60 tx=0x7fea0c007e20 comp rx=0 tx=0).stop 2024-09-19T07:54:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.612+0000 7fea1ede4640 1 -- 172.21.15.16:0/3606251167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1806f9d0 msgr2=0x7fea180a9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.612+0000 7fea1ede4640 1 --2- 172.21.15.16:0/3606251167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1806f9d0 0x7fea180a9450 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fea08000c00 tx=0x7fea080046c0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.612+0000 7fea1ede4640 1 -- 172.21.15.16:0/3606251167 shutdown_connections 2024-09-19T07:54:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.612+0000 7fea1ede4640 1 --2- 172.21.15.16:0/3606251167 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fea0406c2e0 0x7fea0406e7a0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.612+0000 7fea1ede4640 1 --2- 172.21.15.16:0/3606251167 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea1815c140 0x7fea18142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.612+0000 7fea1ede4640 1 --2- 172.21.15.16:0/3606251167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea180b6650 0x7fea180af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.612+0000 7fea1ede4640 1 --2- 172.21.15.16:0/3606251167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1806f9d0 0x7fea180a9450 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.612+0000 7fea1ede4640 1 -- 172.21.15.16:0/3606251167 >> 172.21.15.16:0/3606251167 conn(0x7fea1806bc10 msgr2=0x7fea180b5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:32.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.612+0000 7fea1ede4640 1 -- 172.21.15.16:0/3606251167 shutdown_connections 2024-09-19T07:54:32.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.612+0000 7fea1ede4640 1 -- 172.21.15.16:0/3606251167 wait complete. 2024-09-19T07:54:32.626 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:54:32.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.731+0000 7fbfd7e65640 1 Processor -- start 2024-09-19T07:54:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.732+0000 7fbfd7e65640 1 -- start start 2024-09-19T07:54:32.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.732+0000 7fbfd7e65640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfd015e680 0x7fbfd00cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.732+0000 7fbfd7e65640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfd00d1880 con 0x7fbfd015e2a0 2024-09-19T07:54:32.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.732+0000 7fbfd7e65640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfd00d1b60 con 0x7fbfd00d0010 2024-09-19T07:54:32.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.732+0000 7fbfd7e65640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd00cf2d0 con 0x7fbfd015e680 2024-09-19T07:54:32.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd5c01640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbfd015e2a0 0x7fbfd006d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35526/0 (socket says 172.21.15.16:35526) 2024-09-19T07:54:32.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd5c01640 1 -- 172.21.15.16:0/2822577660 learned_addr learned my addr 172.21.15.16:0/2822577660 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:32.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd5400640 1 --2- 172.21.15.16:0/2822577660 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfd015e680 0x7fbfd00cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd4bff640 1 -- 172.21.15.16:0/2822577660 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4023979783 0 0) 0x7fbfd00d1880 con 0x7fbfd015e2a0 2024-09-19T07:54:32.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd4bff640 1 -- 172.21.15.16:0/2822577660 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfbc003520 con 0x7fbfd015e2a0 2024-09-19T07:54:32.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd4bff640 1 -- 172.21.15.16:0/2822577660 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4182843338 0 0) 0x7fbfd00d1b60 con 0x7fbfd00d0010 2024-09-19T07:54:32.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd4bff640 1 -- 172.21.15.16:0/2822577660 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfd00d1880 con 0x7fbfd00d0010 2024-09-19T07:54:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd4bff640 1 -- 172.21.15.16:0/2822577660 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 415770433 0 0) 0x7fbfbc003520 con 0x7fbfd015e2a0 2024-09-19T07:54:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd4bff640 1 -- 172.21.15.16:0/2822577660 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfd015e680 msgr2=0x7fbfd00cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd4bff640 1 --2- 172.21.15.16:0/2822577660 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfd015e680 0x7fbfd00cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd4bff640 1 -- 172.21.15.16:0/2822577660 >> v1:172.21.15.146:6789/0 conn(0x7fbfd00d0010 legacy=0x7fbfd00d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd4bff640 1 -- 172.21.15.16:0/2822577660 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd00cf5b0 con 0x7fbfd015e2a0 2024-09-19T07:54:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd4bff640 1 -- 172.21.15.16:0/2822577660 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fbfcc001670 con 0x7fbfd015e2a0 2024-09-19T07:54:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd4bff640 1 -- 172.21.15.16:0/2822577660 >> v1:172.21.15.16:6789/0 conn(0x7fbfd015e2a0 legacy=0x7fbfd006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd4bff640 1 --2- 172.21.15.16:0/2822577660 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfd015e680 0x7fbfbc003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd4bff640 1 --2- 172.21.15.16:0/2822577660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfbc0040d0 0x7fbfbc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd4bff640 1 --2- 172.21.15.16:0/2822577660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfbc004d40 0x7fbfbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd4bff640 1 -- 172.21.15.16:0/2822577660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd00cf2d0 con 0x7fbfbc0040d0 2024-09-19T07:54:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd4bff640 1 -- 172.21.15.16:0/2822577660 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc00bb50 con 0x7fbfbc004d40 2024-09-19T07:54:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd4bff640 1 -- 172.21.15.16:0/2822577660 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbfbc00bd20 con 0x7fbfd015e680 2024-09-19T07:54:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd5400640 1 --2- 172.21.15.16:0/2822577660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfbc0040d0 0x7fbfbc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd5c01640 1 --2- 172.21.15.16:0/2822577660 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfd015e680 0x7fbfbc003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.733+0000 7fbfd6402640 1 --2- 172.21.15.16:0/2822577660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfbc004d40 0x7fbfbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.734+0000 7fbfd5400640 1 -- 172.21.15.16:0/2822577660 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfd015e680 msgr2=0x7fbfbc003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.734+0000 7fbfd5400640 1 --2- 172.21.15.16:0/2822577660 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfd015e680 0x7fbfbc003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.734+0000 7fbfd5400640 1 -- 172.21.15.16:0/2822577660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfbc004d40 msgr2=0x7fbfbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.734+0000 7fbfd5400640 1 --2- 172.21.15.16:0/2822577660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfbc004d40 0x7fbfbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.734+0000 7fbfd5400640 1 -- 172.21.15.16:0/2822577660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfbc003520 con 0x7fbfbc0040d0 2024-09-19T07:54:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.734+0000 7fbfd5400640 1 --2- 172.21.15.16:0/2822577660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfbc0040d0 0x7fbfbc004800 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00a22a0 tx=0x7fbfbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4d57172bedaa02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.735+0000 7fbfd4bff640 1 -- 172.21.15.16:0/2822577660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0036070 con 0x7fbfbc0040d0 2024-09-19T07:54:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.735+0000 7fbfd4bff640 1 -- 172.21.15.16:0/2822577660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc0031400 con 0x7fbfbc0040d0 2024-09-19T07:54:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.735+0000 7fbfd4bff640 1 -- 172.21.15.16:0/2822577660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0009690 con 0x7fbfbc0040d0 2024-09-19T07:54:32.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.735+0000 7fbfd7e65640 1 -- 172.21.15.16:0/2822577660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfbc0040d0 msgr2=0x7fbfbc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:32.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.735+0000 7fbfd7e65640 1 --2- 172.21.15.16:0/2822577660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfbc0040d0 0x7fbfbc004800 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00a22a0 tx=0x7fbfbc005830 comp rx=0 tx=0).stop 2024-09-19T07:54:32.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.735+0000 7fbfd6402640 1 -- 172.21.15.16:0/2822577660 reap_dead start 2024-09-19T07:54:32.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.735+0000 7fbfd7e65640 1 -- 172.21.15.16:0/2822577660 shutdown_connections 2024-09-19T07:54:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.735+0000 7fbfd7e65640 1 -- 172.21.15.16:0/2822577660 >> 172.21.15.16:0/2822577660 conn(0x7fbfd009d760 msgr2=0x7fbfd008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.735+0000 7fbfd7e65640 1 -- 172.21.15.16:0/2822577660 shutdown_connections 2024-09-19T07:54:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.735+0000 7fbfd7e65640 1 -- 172.21.15.16:0/2822577660 wait complete. 2024-09-19T07:54:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.737+0000 7fbfd7e65640 1 Processor -- start 2024-09-19T07:54:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.737+0000 7fbfd7e65640 1 -- start start 2024-09-19T07:54:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.737+0000 7fbfd7e65640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfbc0040d0 0x7fbfd015c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.737+0000 7fbfd7e65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd00c95f0 0x7fbfd009e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.737+0000 7fbfd7e65640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfd00c9a70 0x7fbfd00b8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.737+0000 7fbfd7e65640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd006dcb0 con 0x7fbfd00c95f0 2024-09-19T07:54:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.737+0000 7fbfd7e65640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd006db30 con 0x7fbfd00c9a70 2024-09-19T07:54:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.737+0000 7fbfd7e65640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd00d4a80 con 0x7fbfbc0040d0 2024-09-19T07:54:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.738+0000 7fbfd5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd00c95f0 0x7fbfd009e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.738+0000 7fbfd5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd00c95f0 0x7fbfd009e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60146/0 (socket says 172.21.15.16:60146) 2024-09-19T07:54:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.738+0000 7fbfd5400640 1 -- 172.21.15.16:0/2539145298 learned_addr learned my addr 172.21.15.16:0/2539145298 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.738+0000 7fbfd5c01640 1 --2- 172.21.15.16:0/2539145298 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfbc0040d0 0x7fbfd015c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.738+0000 7fbfd6402640 1 --2- 172.21.15.16:0/2539145298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfd00c9a70 0x7fbfd00b8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.738+0000 7fbfd5400640 1 -- 172.21.15.16:0/2539145298 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfbc0040d0 msgr2=0x7fbfd015c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.738+0000 7fbfd5400640 1 --2- 172.21.15.16:0/2539145298 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfbc0040d0 0x7fbfd015c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.738+0000 7fbfd5400640 1 -- 172.21.15.16:0/2539145298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfd00c9a70 msgr2=0x7fbfd00b8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.738+0000 7fbfd5400640 1 --2- 172.21.15.16:0/2539145298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfd00c9a70 0x7fbfd00b8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.738+0000 7fbfd5400640 1 -- 172.21.15.16:0/2539145298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd015c7c0 con 0x7fbfd00c95f0 2024-09-19T07:54:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.738+0000 7fbfd6402640 1 --2- 172.21.15.16:0/2539145298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfd00c9a70 0x7fbfd00b8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.738+0000 7fbfd5400640 1 --2- 172.21.15.16:0/2539145298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd00c95f0 0x7fbfd009e470 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0034040 tx=0x7fbfc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.738+0000 7fbfc67fc640 1 -- 172.21.15.16:0/2539145298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0040020 con 0x7fbfd00c95f0 2024-09-19T07:54:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.738+0000 7fbfc67fc640 1 -- 172.21.15.16:0/2539145298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc003b6a0 con 0x7fbfd00c95f0 2024-09-19T07:54:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.738+0000 7fbfc67fc640 1 -- 172.21.15.16:0/2539145298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0036040 con 0x7fbfd00c95f0 2024-09-19T07:54:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.738+0000 7fbfd7e65640 1 -- 172.21.15.16:0/2539145298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd00cab30 con 0x7fbfd00c95f0 2024-09-19T07:54:32.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.738+0000 7fbfd7e65640 1 -- 172.21.15.16:0/2539145298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd00cb0f0 con 0x7fbfd00c95f0 2024-09-19T07:54:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.739+0000 7fbfd7e65640 1 -- 172.21.15.16:0/2539145298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd00cfad0 con 0x7fbfd00c95f0 2024-09-19T07:54:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.740+0000 7fbfc67fc640 1 -- 172.21.15.16:0/2539145298 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbfc0043020 con 0x7fbfd00c95f0 2024-09-19T07:54:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.740+0000 7fbfc67fc640 1 --2- 172.21.15.16:0/2539145298 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbfb406c2b0 0x7fbfb406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.741+0000 7fbfc67fc640 1 -- 172.21.15.16:0/2539145298 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7fbfc00abcb0 con 0x7fbfd00c95f0 2024-09-19T07:54:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.741+0000 7fbfd5c01640 1 --2- 172.21.15.16:0/2539145298 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbfb406c2b0 0x7fbfb406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.742+0000 7fbfd5c01640 1 --2- 172.21.15.16:0/2539145298 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbfb406c2b0 0x7fbfb406e770 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc001200 tx=0x7fbfcc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:32.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.745+0000 7fbfc67fc640 1 -- 172.21.15.16:0/2539145298 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fbfc004a030 con 0x7fbfd00c95f0 2024-09-19T07:54:32.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.956+0000 7fbfd7e65640 1 -- 172.21.15.16:0/2539145298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbfd006cf90 con 0x7fbfd00c95f0 2024-09-19T07:54:32.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.957+0000 7fbfc67fc640 1 -- 172.21.15.16:0/2539145298 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) ==== 53+0+73 (secure 0 0 0) 0x7fbfc007a1f0 con 0x7fbfd00c95f0 2024-09-19T07:54:32.957 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:54:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.959+0000 7fbfd7e65640 1 -- 172.21.15.16:0/2539145298 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbfb406c2b0 msgr2=0x7fbfb406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.959+0000 7fbfd7e65640 1 --2- 172.21.15.16:0/2539145298 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbfb406c2b0 0x7fbfb406e770 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc001200 tx=0x7fbfcc009040 comp rx=0 tx=0).stop 2024-09-19T07:54:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.959+0000 7fbfd7e65640 1 -- 172.21.15.16:0/2539145298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd00c95f0 msgr2=0x7fbfd009e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.959+0000 7fbfd7e65640 1 --2- 172.21.15.16:0/2539145298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd00c95f0 0x7fbfd009e470 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0034040 tx=0x7fbfc0004730 comp rx=0 tx=0).stop 2024-09-19T07:54:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.959+0000 7fbfd7e65640 1 -- 172.21.15.16:0/2539145298 shutdown_connections 2024-09-19T07:54:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.959+0000 7fbfd7e65640 1 --2- 172.21.15.16:0/2539145298 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbfb406c2b0 0x7fbfb406e770 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.959+0000 7fbfd7e65640 1 --2- 172.21.15.16:0/2539145298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfd00c9a70 0x7fbfd00b8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.959+0000 7fbfd7e65640 1 --2- 172.21.15.16:0/2539145298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd00c95f0 0x7fbfd009e470 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.959+0000 7fbfd7e65640 1 --2- 172.21.15.16:0/2539145298 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfbc0040d0 0x7fbfd015c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.959+0000 7fbfd7e65640 1 -- 172.21.15.16:0/2539145298 >> 172.21.15.16:0/2539145298 conn(0x7fbfd009d760 msgr2=0x7fbfd015d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.959+0000 7fbfd7e65640 1 -- 172.21.15.16:0/2539145298 shutdown_connections 2024-09-19T07:54:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:32.959+0000 7fbfd7e65640 1 -- 172.21.15.16:0/2539145298 wait complete. 2024-09-19T07:54:32.971 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:54:32.971 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:54:32.971 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T07:54:32.971 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T07:54:32.971 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:54:32.971 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:54:32.971 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:54:32.971 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:54:32.971 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:54:32.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:32.972 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T07:54:32.988 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:54:32.988 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:54:32.988 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:54:32.988 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:54:32.988 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:54:32.988 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:54:32.988 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:54:32.989 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:54:32.989 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T07:54:32.989 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:54:32.989 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:54:32.989 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T07:54:32.989 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:54:32.989 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:54:32.989 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T07:54:32.989 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T07:54:32.989 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42116sec preferred_lft 42116sec 2024-09-19T07:54:32.989 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T07:54:32.989 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:54:32.989 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:54:32.990 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:54:32.990 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T07:54:32.990 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:54:32.990 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:54:32.990 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:54:32.990 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:54:32.990 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::32:7bff:fe92:8895/64 scope link 2024-09-19T07:54:32.990 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:54:32.990 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:54:32.990 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether e6:fd:21:6a:43:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:54:32.990 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e4fd:21ff:fe6a:4369/64 scope link 2024-09-19T07:54:32.990 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:54:32.990 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:54:32.991 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:54:32.991 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::80f4:81ff:fe7d:b7f0/64 scope link 2024-09-19T07:54:32.991 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:54:32.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:32.991 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:54:33.053 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:54:33.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:33.055 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T07:54:33.112 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T07:54:33.112 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:54:33.112 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T07:54:33.112 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T07:54:33.112 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:54:33.113 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:54:33.113 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:54:33.113 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T07:54:33.113 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T07:54:33.113 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T07:54:33.113 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:54:33.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:54:33.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:54:33.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:33.241 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:54:33.257 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T07:54:33.257 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:54:33.257 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:54:33.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:33.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-19T07:54:33.323 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T07:54:33.323 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:54:33.323 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T07:54:33.379 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:54:33.379 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:54:33.379 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:54:33.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:54:33.501 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:54:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:54:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:54:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:54:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:54:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:54:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:54:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:54:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:54:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:54:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:54:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:54:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:54:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:54:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:54:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:54:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:54:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:54:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:54:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:54:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:54:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:54:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:54:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:54:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:54:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:54:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:54:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:54:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:54:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:54:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:54:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:54:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:54:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:54:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:54:33.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:33.562 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:54:33.614 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:54:33.615 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T07:54:33.658 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:54:33.683 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:54:33.740 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:54:33.740 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:54:33.740 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:54:33.740 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:54:33.740 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:54:33.740 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:54:33.740 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:54:33.740 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:54:33.740 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:54:33.740 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:54:33.741 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:54:33.741 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:54:33.741 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:54:33.741 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:54:33.741 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:54:33.741 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:54:33.741 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:54:33.741 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:54:33.741 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:54:33.741 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:54:33.741 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:54:33.741 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:54:33.741 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:54:33.742 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:54:33.742 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:54:33.742 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:54:33.742 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:54:33.742 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:54:33.742 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:54:33.742 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:54:33.742 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:54:33.742 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:54:33.742 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:54:33.742 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:54:33.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:33.743 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:54:33.743 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-19T07:54:33.741+0000 7f2dc843d480 -1 init, newargv = 0x564f2f92e820 newargc=15 2024-09-19T07:54:33.743 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[72929]: starting ceph client 2024-09-19T07:54:33.746 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[72929]: starting fuse 2024-09-19T07:54:33.759 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:54:33.759 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-19T07:54:33.760 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:54:33.760 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:54:33.760 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:54:33.760 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:54:33.760 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:54:33.760 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:54:33.760 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:54:33.760 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:54:33.760 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:54:33.760 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:54:33.760 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:54:33.761 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:54:33.761 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:54:33.761 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:54:33.761 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:54:33.761 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:54:33.761 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:54:33.761 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:54:33.761 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:54:33.761 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:54:33.761 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:54:33.761 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:54:33.761 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:54:33.761 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:54:33.761 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:54:33.762 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:54:33.762 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:54:33.880 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.72929.asok 2024-09-19T07:54:33.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:54:33.890 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.72929.asok 2024-09-19T07:54:33.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:33.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.72929.asok status 2024-09-19T07:54:34.028 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:54:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:54:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:54:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:54:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:54:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:54:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:54:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "72929", 2024-09-19T07:54:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:54:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-19T07:54:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-19T07:54:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 8007, 2024-09-19T07:54:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:54:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:54:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:54:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 8007 2024-09-19T07:54:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:54:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:54:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:54:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1934386859 2024-09-19T07:54:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:54:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:54:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:54:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1934386859 2024-09-19T07:54:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.8007 192.168.0.1:0/1934386859", 2024-09-19T07:54:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/1934386859", 2024-09-19T07:54:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-19T07:54:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 72, 2024-09-19T07:54:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 129, 2024-09-19T07:54:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:54:34.031 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:54:34.031 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:54:34.031 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:54:34.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:34.037 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-19T07:54:34.037 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T07:54:34.053 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-19T07:54:34.053 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T07:54:34.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:34.054 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:54:34.121 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:54:34.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.263+0000 7f1a9365e640 1 Processor -- start 2024-09-19T07:54:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.263+0000 7f1a9365e640 1 -- start start 2024-09-19T07:54:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.264+0000 7f1a9365e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1a8c15e2a0 0x7f1a8c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.264+0000 7f1a9365e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a8c15e680 0x7f1a8c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.264+0000 7f1a9365e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a8c0d1880 con 0x7f1a8c06d960 2024-09-19T07:54:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.264+0000 7f1a9365e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f1a8c0d1b60 con 0x7f1a8c15e2a0 2024-09-19T07:54:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.264+0000 7f1a9365e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8c0cf2d0 con 0x7f1a8c15e680 2024-09-19T07:54:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.264+0000 7f1a913fa640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1a8c15e2a0 0x7f1a8c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.264+0000 7f1a913fa640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1a8c15e2a0 0x7f1a8c06d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60160/0 (socket says 172.21.15.16:60160) 2024-09-19T07:54:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.264+0000 7f1a913fa640 1 -- 172.21.15.16:0/3334840235 learned_addr learned my addr 172.21.15.16:0/3334840235 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.264+0000 7f1a913fa640 1 -- 172.21.15.16:0/3334840235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a8c15e680 msgr2=0x7f1a8c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.264+0000 7f1a913fa640 1 --2- 172.21.15.16:0/3334840235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a8c15e680 0x7f1a8c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:34.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.264+0000 7f1a913fa640 1 -- 172.21.15.16:0/3334840235 >> v1:172.21.15.146:6789/0 conn(0x7f1a8c06d960 legacy=0x7f1a8c0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:34.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.264+0000 7f1a913fa640 1 -- 172.21.15.16:0/3334840235 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8c0cf5b0 con 0x7f1a8c15e2a0 2024-09-19T07:54:34.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.265+0000 7f1a913fa640 1 --2- 172.21.15.16:0/3334840235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c15e2a0 0x7f1a8c06d420 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009870 tx=0x7f1a8002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae2936056eab147 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:34.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.265+0000 7f1a7ffff640 1 -- 172.21.15.16:0/3334840235 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80037070 con 0x7f1a8c15e2a0 2024-09-19T07:54:34.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.265+0000 7f1a7ffff640 1 -- 172.21.15.16:0/3334840235 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a80032440 con 0x7f1a8c15e2a0 2024-09-19T07:54:34.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.265+0000 7f1a7ffff640 1 -- 172.21.15.16:0/3334840235 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80032810 con 0x7f1a8c15e2a0 2024-09-19T07:54:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.266+0000 7f1a9365e640 1 -- 172.21.15.16:0/3334840235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c15e2a0 msgr2=0x7f1a8c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.266+0000 7f1a9365e640 1 --2- 172.21.15.16:0/3334840235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c15e2a0 0x7f1a8c06d420 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009870 tx=0x7f1a8002a230 comp rx=0 tx=0).stop 2024-09-19T07:54:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.266+0000 7f1a9365e640 1 -- 172.21.15.16:0/3334840235 shutdown_connections 2024-09-19T07:54:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.266+0000 7f1a9365e640 1 --2- 172.21.15.16:0/3334840235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a8c15e680 0x7f1a8c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.266+0000 7f1a9365e640 1 --2- 172.21.15.16:0/3334840235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c15e2a0 0x7f1a8c06d420 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.266+0000 7f1a9365e640 1 -- 172.21.15.16:0/3334840235 >> 172.21.15.16:0/3334840235 conn(0x7f1a8c09d760 msgr2=0x7f1a8c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.266+0000 7f1a9365e640 1 -- 172.21.15.16:0/3334840235 shutdown_connections 2024-09-19T07:54:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.266+0000 7f1a9365e640 1 -- 172.21.15.16:0/3334840235 wait complete. 2024-09-19T07:54:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.267+0000 7f1a9365e640 1 Processor -- start 2024-09-19T07:54:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.267+0000 7f1a9365e640 1 -- start start 2024-09-19T07:54:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.268+0000 7f1a9365e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a8c06d960 0x7f1a8c0cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.268+0000 7f1a9365e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a8c15e2a0 0x7f1a8c15cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.268+0000 7f1a9365e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c15e680 0x7f1a8c15c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.268+0000 7f1a9365e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c0d4c00 con 0x7f1a8c15e680 2024-09-19T07:54:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.268+0000 7f1a9365e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c0d4a80 con 0x7f1a8c15e2a0 2024-09-19T07:54:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.268+0000 7f1a9365e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8c0d0010 con 0x7f1a8c06d960 2024-09-19T07:54:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.268+0000 7f1a91bfb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c15e680 0x7f1a8c15c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.268+0000 7f1a91bfb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c15e680 0x7f1a8c15c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60162/0 (socket says 172.21.15.16:60162) 2024-09-19T07:54:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.268+0000 7f1a91bfb640 1 -- 172.21.15.16:0/362648698 learned_addr learned my addr 172.21.15.16:0/362648698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.268+0000 7f1a91bfb640 1 -- 172.21.15.16:0/362648698 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a8c06d960 msgr2=0x7f1a8c0cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.268+0000 7f1a91bfb640 1 --2- 172.21.15.16:0/362648698 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a8c06d960 0x7f1a8c0cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.268+0000 7f1a91bfb640 1 -- 172.21.15.16:0/362648698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a8c15e2a0 msgr2=0x7f1a8c15cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.268+0000 7f1a90bf9640 1 --2- 172.21.15.16:0/362648698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a8c15e2a0 0x7f1a8c15cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.268+0000 7f1a91bfb640 1 --2- 172.21.15.16:0/362648698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a8c15e2a0 0x7f1a8c15cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.268+0000 7f1a91bfb640 1 -- 172.21.15.16:0/362648698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8c0cb110 con 0x7f1a8c15e680 2024-09-19T07:54:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.268+0000 7f1a90bf9640 1 --2- 172.21.15.16:0/362648698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a8c15e2a0 0x7f1a8c15cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.268+0000 7f1a91bfb640 1 --2- 172.21.15.16:0/362648698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c15e680 0x7f1a8c15c740 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400c7b0 tx=0x7f1a8400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.269+0000 7f1a7dffb640 1 -- 172.21.15.16:0/362648698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84004b60 con 0x7f1a8c15e680 2024-09-19T07:54:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.269+0000 7f1a9365e640 1 -- 172.21.15.16:0/362648698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a8c0cd680 con 0x7f1a8c15e680 2024-09-19T07:54:34.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.269+0000 7f1a7dffb640 1 -- 172.21.15.16:0/362648698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a84004d00 con 0x7f1a8c15e680 2024-09-19T07:54:34.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.269+0000 7f1a7dffb640 1 -- 172.21.15.16:0/362648698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8400aa90 con 0x7f1a8c15e680 2024-09-19T07:54:34.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.269+0000 7f1a9365e640 1 -- 172.21.15.16:0/362648698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a8c0cdc40 con 0x7f1a8c15e680 2024-09-19T07:54:34.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.270+0000 7f1a9365e640 1 -- 172.21.15.16:0/362648698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a8c06d420 con 0x7f1a8c15e680 2024-09-19T07:54:34.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.270+0000 7f1a7dffb640 1 -- 172.21.15.16:0/362648698 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1a8400ac30 con 0x7f1a8c15e680 2024-09-19T07:54:34.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.271+0000 7f1a7dffb640 1 --2- 172.21.15.16:0/362648698 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1a7006c2b0 0x7f1a7006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:34.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.271+0000 7f1a7dffb640 1 -- 172.21.15.16:0/362648698 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7f1a8408b890 con 0x7f1a8c15e680 2024-09-19T07:54:34.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.271+0000 7f1a913fa640 1 --2- 172.21.15.16:0/362648698 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1a7006c2b0 0x7f1a7006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:34.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.272+0000 7f1a913fa640 1 --2- 172.21.15.16:0/362648698 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1a7006c2b0 0x7f1a7006e770 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1a80034000 tx=0x7f1a8002c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.275+0000 7f1a7dffb640 1 -- 172.21.15.16:0/362648698 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1a84020070 con 0x7f1a8c15e680 2024-09-19T07:54:34.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.487+0000 7f1a9365e640 1 -- 172.21.15.16:0/362648698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1a8c06cf90 con 0x7f1a8c15e680 2024-09-19T07:54:34.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.487+0000 7f1a7dffb640 1 -- 172.21.15.16:0/362648698 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) ==== 53+0+73 (secure 0 0 0) 0x7f1a84059e80 con 0x7f1a8c15e680 2024-09-19T07:54:34.488 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:54:34.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.489+0000 7f1a9365e640 1 -- 172.21.15.16:0/362648698 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1a7006c2b0 msgr2=0x7f1a7006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:34.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.489+0000 7f1a9365e640 1 --2- 172.21.15.16:0/362648698 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1a7006c2b0 0x7f1a7006e770 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1a80034000 tx=0x7f1a8002c000 comp rx=0 tx=0).stop 2024-09-19T07:54:34.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.489+0000 7f1a9365e640 1 -- 172.21.15.16:0/362648698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c15e680 msgr2=0x7f1a8c15c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:34.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.489+0000 7f1a9365e640 1 --2- 172.21.15.16:0/362648698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c15e680 0x7f1a8c15c740 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400c7b0 tx=0x7f1a8400cc70 comp rx=0 tx=0).stop 2024-09-19T07:54:34.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.489+0000 7f1a9365e640 1 -- 172.21.15.16:0/362648698 shutdown_connections 2024-09-19T07:54:34.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.489+0000 7f1a9365e640 1 --2- 172.21.15.16:0/362648698 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1a7006c2b0 0x7f1a7006e770 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:34.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.489+0000 7f1a9365e640 1 --2- 172.21.15.16:0/362648698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c15e680 0x7f1a8c15c740 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:34.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.489+0000 7f1a9365e640 1 --2- 172.21.15.16:0/362648698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a8c15e2a0 0x7f1a8c15cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:34.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.489+0000 7f1a9365e640 1 --2- 172.21.15.16:0/362648698 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a8c06d960 0x7f1a8c0cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:34.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.489+0000 7f1a9365e640 1 -- 172.21.15.16:0/362648698 >> 172.21.15.16:0/362648698 conn(0x7f1a8c09d760 msgr2=0x7f1a8c160a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:34.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.489+0000 7f1a9365e640 1 -- 172.21.15.16:0/362648698 shutdown_connections 2024-09-19T07:54:34.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:34.489+0000 7f1a9365e640 1 -- 172.21.15.16:0/362648698 wait complete. 2024-09-19T07:54:34.500 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:54:34.500 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:54:34.501 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-09-19T07:54:34.501 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T07:54:34.501 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T07:54:34.501 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:54:34.501 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:54:34.501 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:54:34.501 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:54:34.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:34.501 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-09-19T07:54:34.521 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:54:34.521 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:54:34.521 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:54:34.522 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:54:34.522 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-09-19T07:54:34.522 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:54:34.522 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:54:34.522 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-09-19T07:54:34.522 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2024-09-19T07:54:34.522 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:54:34.522 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-09-19T07:54:34.522 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2024-09-19T07:54:34.522 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:54:34.522 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-09-19T07:54:34.522 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:54:34.522 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-09-19T07:54:34.522 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T07:54:34.523 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42082sec preferred_lft 42082sec 2024-09-19T07:54:34.523 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link noprefixroute 2024-09-19T07:54:34.523 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:54:34.523 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:54:34.523 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:54:34.523 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:54:34.523 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:54:34.523 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::e47f:32ff:fe68:31ca/64 scope link 2024-09-19T07:54:34.523 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:54:34.523 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:54:34.523 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether f6:1f:22:44:9d:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:54:34.523 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::f41f:22ff:fe44:9dcd/64 scope link 2024-09-19T07:54:34.523 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:54:34.523 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:54:34.524 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:54:34.524 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::8459:eeff:fe21:a113/64 scope link 2024-09-19T07:54:34.524 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:54:34.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:34.524 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:54:34.590 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-09-19T07:54:34.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:34.592 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-09-19T07:54:34.654 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-09-19T07:54:34.655 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:54:34.655 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T07:54:34.655 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T07:54:34.655 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:54:34.656 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:54:34.656 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-19T07:54:34.656 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T07:54:34.656 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T07:54:34.657 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T07:54:34.657 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-19T07:54:34.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:54:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:54:34.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:54:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:54:34.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:34.791 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-09-19T07:54:34.807 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T07:54:34.807 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T07:54:34.807 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:54:34.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:34.807 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2024-09-19T07:54:34.873 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T07:54:34.873 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:54:34.873 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T07:54:34.928 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:54:34.928 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:54:34.929 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:54:34.985 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:54:35.051 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:54:35.108 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:54:35.108 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:54:35.109 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:54:35.109 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:54:35.109 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:54:35.109 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:54:35.109 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:54:35.109 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:54:35.110 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:54:35.110 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:54:35.110 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:54:35.110 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:54:35.110 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:54:35.110 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:54:35.111 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:54:35.111 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:54:35.111 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:54:35.111 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:54:35.111 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:54:35.111 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:54:35.112 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:54:35.112 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:54:35.112 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:54:35.112 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:54:35.112 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:54:35.112 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:54:35.113 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:54:35.113 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:54:35.113 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:54:35.113 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:54:35.114 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:54:35.114 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:54:35.114 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:54:35.114 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:54:35.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:35.115 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:54:35.164 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:54:35.165 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T07:54:35.207 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:54:35.232 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:54:35.288 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:54:35.289 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:54:35.289 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:54:35.289 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:54:35.289 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:54:35.289 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:54:35.289 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:54:35.289 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:54:35.289 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:54:35.289 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:54:35.289 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:54:35.290 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:54:35.290 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:54:35.290 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:54:35.290 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:54:35.290 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:54:35.290 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:54:35.290 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:54:35.290 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:54:35.290 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:54:35.290 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:54:35.290 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:54:35.290 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:54:35.291 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:54:35.291 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:54:35.291 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:54:35.291 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:54:35.291 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:54:35.291 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:54:35.291 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:54:35.291 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:54:35.291 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:54:35.291 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:54:35.291 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:54:35.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:35.292 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:54:35.292 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2024-09-19T07:54:35.289+0000 7eff8dc36480 -1 init, newargv = 0x55a743eaf730 newargc=15 2024-09-19T07:54:35.292 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[49038]: starting ceph client 2024-09-19T07:54:35.293 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[49038]: starting fuse 2024-09-19T07:54:35.307 INFO:teuthology.orchestra.run.smithi146.stdout:42 2024-09-19T07:54:35.308 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-19T07:54:35.308 DEBUG:teuthology.orchestra.run.smithi146:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:54:35.308 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2024-09-19T07:54:35.308 DEBUG:teuthology.orchestra.run.smithi146:> import re 2024-09-19T07:54:35.308 DEBUG:teuthology.orchestra.run.smithi146:> import os 2024-09-19T07:54:35.309 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2024-09-19T07:54:35.309 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:54:35.309 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2024-09-19T07:54:35.309 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T07:54:35.309 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2024-09-19T07:54:35.309 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T07:54:35.309 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:54:35.309 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2024-09-19T07:54:35.309 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2024-09-19T07:54:35.309 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2024-09-19T07:54:35.309 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2024-09-19T07:54:35.309 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:54:35.309 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2024-09-19T07:54:35.309 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:54:35.309 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:54:35.310 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:54:35.310 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2024-09-19T07:54:35.310 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2024-09-19T07:54:35.310 DEBUG:teuthology.orchestra.run.smithi146:> return f 2024-09-19T07:54:35.310 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:54:35.310 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:54:35.310 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2024-09-19T07:54:35.310 DEBUG:teuthology.orchestra.run.smithi146:> ' 2024-09-19T07:54:35.425 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.49038.asok 2024-09-19T07:54:35.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:54:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:54:35.435 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.49038.asok 2024-09-19T07:54:35.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:35.435 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.49038.asok status 2024-09-19T07:54:35.569 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-19T07:54:35.570 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2024-09-19T07:54:35.570 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:54:35.570 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:54:35.570 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2024-09-19T07:54:35.570 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2024-09-19T07:54:35.570 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T07:54:35.570 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "49038", 2024-09-19T07:54:35.570 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2024-09-19T07:54:35.570 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:54:35.570 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2024-09-19T07:54:35.570 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2024-09-19T07:54:35.571 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 6635, 2024-09-19T07:54:35.571 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2024-09-19T07:54:35.571 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2024-09-19T07:54:35.571 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2024-09-19T07:54:35.571 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 6635 2024-09-19T07:54:35.571 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:54:35.571 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:54:35.571 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:54:35.571 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:54:35.571 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 2776231670 2024-09-19T07:54:35.571 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-19T07:54:35.571 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:54:35.571 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:54:35.572 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:54:35.572 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:54:35.572 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 2776231670 2024-09-19T07:54:35.572 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:54:35.572 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.6635 192.168.0.1:0/2776231670", 2024-09-19T07:54:35.572 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/2776231670", 2024-09-19T07:54:35.572 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2024-09-19T07:54:35.572 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 72, 2024-09-19T07:54:35.572 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 129, 2024-09-19T07:54:35.572 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:54:35.572 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2024-09-19T07:54:35.572 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2024-09-19T07:54:35.572 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-19T07:54:35.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:35.578 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2024-09-19T07:54:35.578 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T07:54:35.594 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2024-09-19T07:54:35.594 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T07:54:35.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:35.594 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:54:35.660 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-09-19T07:54:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.779+0000 7f76f38ba640 1 Processor -- start 2024-09-19T07:54:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.779+0000 7f76f38ba640 1 -- start start 2024-09-19T07:54:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.780+0000 7f76f38ba640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f76ec15e2a0 0x7f76ec06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.780+0000 7f76f38ba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76ec06d960 0x7f76ec0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.780+0000 7f76f38ba640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76ec0d1880 con 0x7f76ec15e680 2024-09-19T07:54:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.780+0000 7f76f38ba640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f76ec0d1b60 con 0x7f76ec06d960 2024-09-19T07:54:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.780+0000 7f76f38ba640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f76ec0cf2d0 con 0x7f76ec15e2a0 2024-09-19T07:54:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.780+0000 7f76f1e57640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76ec06d960 0x7f76ec0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.780+0000 7f76f0e55640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f76ec15e680 0x7f76ec0cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35534/0 (socket says 172.21.15.16:35534) 2024-09-19T07:54:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.780+0000 7f76f0e55640 1 -- 172.21.15.16:0/1644531089 learned_addr learned my addr 172.21.15.16:0/1644531089 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.781+0000 7f76f1656640 1 --2- 172.21.15.16:0/1644531089 >> v2:172.21.15.146:3300/0 conn(0x7f76ec15e2a0 0x7f76ec06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.781+0000 7f76f1e57640 1 -- 172.21.15.16:0/1644531089 >> v2:172.21.15.146:3300/0 conn(0x7f76ec15e2a0 msgr2=0x7f76ec06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.781+0000 7f76f1e57640 1 --2- 172.21.15.16:0/1644531089 >> v2:172.21.15.146:3300/0 conn(0x7f76ec15e2a0 0x7f76ec06d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.781+0000 7f76f1e57640 1 -- 172.21.15.16:0/1644531089 >> v1:172.21.15.16:6789/0 conn(0x7f76ec15e680 legacy=0x7f76ec0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.781+0000 7f76e3fff640 1 -- 172.21.15.16:0/1644531089 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2942040489 0 0) 0x7f76ec0d1880 con 0x7f76ec15e680 2024-09-19T07:54:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.781+0000 7f76f1e57640 1 -- 172.21.15.16:0/1644531089 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ec0cf5b0 con 0x7f76ec06d960 2024-09-19T07:54:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.782+0000 7f76f1e57640 1 --2- 172.21.15.16:0/1644531089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec06d960 0x7f76ec0d4370 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0b6fa0 tx=0x7f76e402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d05728ccd6f80e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.782+0000 7f76e3fff640 1 -- 172.21.15.16:0/1644531089 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4007740 con 0x7f76ec06d960 2024-09-19T07:54:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.782+0000 7f76e3fff640 1 -- 172.21.15.16:0/1644531089 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e40078e0 con 0x7f76ec06d960 2024-09-19T07:54:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.782+0000 7f76e3fff640 1 -- 172.21.15.16:0/1644531089 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4007cb0 con 0x7f76ec06d960 2024-09-19T07:54:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.782+0000 7f76f38ba640 1 -- 172.21.15.16:0/1644531089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec06d960 msgr2=0x7f76ec0d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.782+0000 7f76f38ba640 1 --2- 172.21.15.16:0/1644531089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec06d960 0x7f76ec0d4370 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0b6fa0 tx=0x7f76e402a680 comp rx=0 tx=0).stop 2024-09-19T07:54:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.782+0000 7f76f38ba640 1 -- 172.21.15.16:0/1644531089 shutdown_connections 2024-09-19T07:54:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.782+0000 7f76f38ba640 1 --2- 172.21.15.16:0/1644531089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec06d960 0x7f76ec0d4370 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.782+0000 7f76f38ba640 1 --2- 172.21.15.16:0/1644531089 >> v2:172.21.15.146:3300/0 conn(0x7f76ec15e2a0 0x7f76ec06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.782+0000 7f76f38ba640 1 -- 172.21.15.16:0/1644531089 >> 172.21.15.16:0/1644531089 conn(0x7f76ec09d760 msgr2=0x7f76ec08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.783+0000 7f76f38ba640 1 -- 172.21.15.16:0/1644531089 shutdown_connections 2024-09-19T07:54:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.783+0000 7f76f38ba640 1 -- 172.21.15.16:0/1644531089 wait complete. 2024-09-19T07:54:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.784+0000 7f76f38ba640 1 Processor -- start 2024-09-19T07:54:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.784+0000 7f76f38ba640 1 -- start start 2024-09-19T07:54:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.784+0000 7f76f38ba640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76ec06d960 0x7f76ec157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.784+0000 7f76f38ba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76ec15e2a0 0x7f76ec1f4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.784+0000 7f76f38ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec15e680 0x7f76ec161f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.784+0000 7f76f38ba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec0d0360 con 0x7f76ec15e680 2024-09-19T07:54:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.784+0000 7f76f38ba640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec0d01e0 con 0x7f76ec06d960 2024-09-19T07:54:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.784+0000 7f76f38ba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f76ec0d48b0 con 0x7f76ec15e2a0 2024-09-19T07:54:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.785+0000 7f76f1e57640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec15e680 0x7f76ec161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.785+0000 7f76f1e57640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec15e680 0x7f76ec161f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60184/0 (socket says 172.21.15.16:60184) 2024-09-19T07:54:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.785+0000 7f76f1e57640 1 -- 172.21.15.16:0/3447749820 learned_addr learned my addr 172.21.15.16:0/3447749820 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.785+0000 7f76f1656640 1 --2- 172.21.15.16:0/3447749820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76ec06d960 0x7f76ec157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.785+0000 7f76f0e55640 1 --2- 172.21.15.16:0/3447749820 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76ec15e2a0 0x7f76ec1f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.785+0000 7f76f1e57640 1 -- 172.21.15.16:0/3447749820 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76ec15e2a0 msgr2=0x7f76ec1f4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.785+0000 7f76f1e57640 1 --2- 172.21.15.16:0/3447749820 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76ec15e2a0 0x7f76ec1f4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.785+0000 7f76f1e57640 1 -- 172.21.15.16:0/3447749820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76ec06d960 msgr2=0x7f76ec157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.785+0000 7f76f1e57640 1 --2- 172.21.15.16:0/3447749820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76ec06d960 0x7f76ec157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.785+0000 7f76f1e57640 1 -- 172.21.15.16:0/3447749820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ec0b8520 con 0x7f76ec15e680 2024-09-19T07:54:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.785+0000 7f76f1656640 1 --2- 172.21.15.16:0/3447749820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76ec06d960 0x7f76ec157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:54:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.785+0000 7f76f1e57640 1 --2- 172.21.15.16:0/3447749820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec15e680 0x7f76ec161f40 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009e80 tx=0x7f76e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.785+0000 7f76e1ffb640 1 -- 172.21.15.16:0/3447749820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4040070 con 0x7f76ec15e680 2024-09-19T07:54:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.785+0000 7f76e1ffb640 1 -- 172.21.15.16:0/3447749820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e4031c40 con 0x7f76ec15e680 2024-09-19T07:54:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.785+0000 7f76f38ba640 1 -- 172.21.15.16:0/3447749820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ec0b87c0 con 0x7f76ec15e680 2024-09-19T07:54:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.785+0000 7f76e1ffb640 1 -- 172.21.15.16:0/3447749820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4009450 con 0x7f76ec15e680 2024-09-19T07:54:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.786+0000 7f76f38ba640 1 -- 172.21.15.16:0/3447749820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76ec0b8d80 con 0x7f76ec15e680 2024-09-19T07:54:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.787+0000 7f76f38ba640 1 -- 172.21.15.16:0/3447749820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76ec0d4c70 con 0x7f76ec15e680 2024-09-19T07:54:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.787+0000 7f76e1ffb640 1 -- 172.21.15.16:0/3447749820 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f76e4039040 con 0x7f76ec15e680 2024-09-19T07:54:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.788+0000 7f76e1ffb640 1 --2- 172.21.15.16:0/3447749820 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f76d006c2b0 0x7f76d006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.788+0000 7f76e1ffb640 1 -- 172.21.15.16:0/3447749820 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7f76e40ac660 con 0x7f76ec15e680 2024-09-19T07:54:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.788+0000 7f76f1656640 1 --2- 172.21.15.16:0/3447749820 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f76d006c2b0 0x7f76d006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.789+0000 7f76f1656640 1 --2- 172.21.15.16:0/3447749820 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f76d006c2b0 0x7f76d006e770 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0cb900 tx=0x7f76e8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.792+0000 7f76e1ffb640 1 -- 172.21.15.16:0/3447749820 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f76e407a680 con 0x7f76ec15e680 2024-09-19T07:54:35.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.968+0000 7f76f38ba640 1 -- 172.21.15.16:0/3447749820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7f76ec06cf90 con 0x7f76ec15e680 2024-09-19T07:54:35.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.971+0000 7f76e1ffb640 1 -- 172.21.15.16:0/3447749820 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v54)=0 v54) ==== 121+0+0 (secure 0 0 0) 0x7f76e40aae20 con 0x7f76ec15e680 2024-09-19T07:54:35.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.974+0000 7f76f38ba640 1 -- 172.21.15.16:0/3447749820 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f76d006c2b0 msgr2=0x7f76d006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:35.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.974+0000 7f76f38ba640 1 --2- 172.21.15.16:0/3447749820 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f76d006c2b0 0x7f76d006e770 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0cb900 tx=0x7f76e8009190 comp rx=0 tx=0).stop 2024-09-19T07:54:35.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.974+0000 7f76f38ba640 1 -- 172.21.15.16:0/3447749820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec15e680 msgr2=0x7f76ec161f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:35.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.974+0000 7f76f38ba640 1 --2- 172.21.15.16:0/3447749820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec15e680 0x7f76ec161f40 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009e80 tx=0x7f76e4004a40 comp rx=0 tx=0).stop 2024-09-19T07:54:35.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.974+0000 7f76f38ba640 1 -- 172.21.15.16:0/3447749820 shutdown_connections 2024-09-19T07:54:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.974+0000 7f76f38ba640 1 --2- 172.21.15.16:0/3447749820 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f76d006c2b0 0x7f76d006e770 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.974+0000 7f76f38ba640 1 --2- 172.21.15.16:0/3447749820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec15e680 0x7f76ec161f40 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.974+0000 7f76f38ba640 1 --2- 172.21.15.16:0/3447749820 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76ec15e2a0 0x7f76ec1f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.974+0000 7f76f38ba640 1 --2- 172.21.15.16:0/3447749820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76ec06d960 0x7f76ec157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.974+0000 7f76f38ba640 1 -- 172.21.15.16:0/3447749820 >> 172.21.15.16:0/3447749820 conn(0x7f76ec09d760 msgr2=0x7f76ec0d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.974+0000 7f76f38ba640 1 -- 172.21.15.16:0/3447749820 shutdown_connections 2024-09-19T07:54:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:35.974+0000 7f76f38ba640 1 -- 172.21.15.16:0/3447749820 wait complete. 2024-09-19T07:54:35.985 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 100 2024-09-19T07:54:36.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.089+0000 7f90f7eba640 1 Processor -- start 2024-09-19T07:54:36.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.089+0000 7f90f7eba640 1 -- start start 2024-09-19T07:54:36.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.090+0000 7f90f7eba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90f015e2a0 0x7f90f006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.090+0000 7f90f7eba640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f90f015e680 0x7f90f00cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.090+0000 7f90f7eba640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90f00d1880 con 0x7f90f006d960 2024-09-19T07:54:36.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.090+0000 7f90f7eba640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f90f00d1b60 con 0x7f90f015e680 2024-09-19T07:54:36.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.090+0000 7f90f7eba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f90f00cf2d0 con 0x7f90f015e2a0 2024-09-19T07:54:36.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.090+0000 7f90f6457640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f90f006d960 0x7f90f00d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35542/0 (socket says 172.21.15.16:35542) 2024-09-19T07:54:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.090+0000 7f90f6457640 1 -- 172.21.15.16:0/3185119266 learned_addr learned my addr 172.21.15.16:0/3185119266 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.090+0000 7f90f5455640 1 --2- 172.21.15.16:0/3185119266 >> v2:172.21.15.146:3300/0 conn(0x7f90f015e680 0x7f90f00cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.090+0000 7f90f5c56640 1 --2- 172.21.15.16:0/3185119266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90f015e2a0 0x7f90f006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.091+0000 7f90f4c54640 1 -- 172.21.15.16:0/3185119266 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2018891194 0 0) 0x7f90f00d1880 con 0x7f90f006d960 2024-09-19T07:54:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.091+0000 7f90f4c54640 1 -- 172.21.15.16:0/3185119266 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90dc003520 con 0x7f90f006d960 2024-09-19T07:54:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.091+0000 7f90f5455640 1 -- 172.21.15.16:0/3185119266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90f015e2a0 msgr2=0x7f90f006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.091+0000 7f90f5455640 1 --2- 172.21.15.16:0/3185119266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90f015e2a0 0x7f90f006d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.091+0000 7f90f5455640 1 -- 172.21.15.16:0/3185119266 >> v1:172.21.15.16:6789/0 conn(0x7f90f006d960 legacy=0x7f90f00d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.091+0000 7f90f5455640 1 -- 172.21.15.16:0/3185119266 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f00cf5b0 con 0x7f90f015e680 2024-09-19T07:54:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.091+0000 7f90f5c56640 1 --2- 172.21.15.16:0/3185119266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90f015e2a0 0x7f90f006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:54:36.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.092+0000 7f90f5455640 1 --2- 172.21.15.16:0/3185119266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90f015e680 0x7f90f00cfad0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f90e00097a0 tx=0x7f90e002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12c884a789d863d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:36.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.092+0000 7f90f4c54640 1 -- 172.21.15.16:0/3185119266 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0037070 con 0x7f90f015e680 2024-09-19T07:54:36.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.092+0000 7f90f4c54640 1 -- 172.21.15.16:0/3185119266 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e0032440 con 0x7f90f015e680 2024-09-19T07:54:36.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.092+0000 7f90f4c54640 1 -- 172.21.15.16:0/3185119266 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0032810 con 0x7f90f015e680 2024-09-19T07:54:36.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.092+0000 7f90f7eba640 1 -- 172.21.15.16:0/3185119266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90f015e680 msgr2=0x7f90f00cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.092+0000 7f90f7eba640 1 --2- 172.21.15.16:0/3185119266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90f015e680 0x7f90f00cfad0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f90e00097a0 tx=0x7f90e002a180 comp rx=0 tx=0).stop 2024-09-19T07:54:36.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.092+0000 7f90f7eba640 1 -- 172.21.15.16:0/3185119266 shutdown_connections 2024-09-19T07:54:36.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.092+0000 7f90f7eba640 1 --2- 172.21.15.16:0/3185119266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90f015e680 0x7f90f00cfad0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.092+0000 7f90f7eba640 1 --2- 172.21.15.16:0/3185119266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90f015e2a0 0x7f90f006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.092+0000 7f90f7eba640 1 -- 172.21.15.16:0/3185119266 >> 172.21.15.16:0/3185119266 conn(0x7f90f009d760 msgr2=0x7f90f008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:36.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.092+0000 7f90f7eba640 1 -- 172.21.15.16:0/3185119266 shutdown_connections 2024-09-19T07:54:36.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.092+0000 7f90f7eba640 1 -- 172.21.15.16:0/3185119266 wait complete. 2024-09-19T07:54:36.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.093+0000 7f90f7eba640 1 Processor -- start 2024-09-19T07:54:36.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.094+0000 7f90f7eba640 1 -- start start 2024-09-19T07:54:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.094+0000 7f90f7eba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90f006d960 0x7f90f00cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.094+0000 7f90f7eba640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90f015e2a0 0x7f90f00cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.094+0000 7f90f7eba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f015e680 0x7f90f00cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.094+0000 7f90f7eba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f90f00d4c00 con 0x7f90f015e680 2024-09-19T07:54:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.094+0000 7f90f7eba640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f90f00d4a80 con 0x7f90f015e2a0 2024-09-19T07:54:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.094+0000 7f90f7eba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f90f00d0010 con 0x7f90f006d960 2024-09-19T07:54:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.094+0000 7f90f6457640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f015e680 0x7f90f00cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.094+0000 7f90f6457640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f015e680 0x7f90f00cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60186/0 (socket says 172.21.15.16:60186) 2024-09-19T07:54:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.094+0000 7f90f6457640 1 -- 172.21.15.16:0/1892916459 learned_addr learned my addr 172.21.15.16:0/1892916459 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.095+0000 7f90f5c56640 1 --2- 172.21.15.16:0/1892916459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90f006d960 0x7f90f00cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.095+0000 7f90f5455640 1 --2- 172.21.15.16:0/1892916459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90f015e2a0 0x7f90f00cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.095+0000 7f90f6457640 1 -- 172.21.15.16:0/1892916459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90f006d960 msgr2=0x7f90f00cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.095+0000 7f90f6457640 1 --2- 172.21.15.16:0/1892916459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90f006d960 0x7f90f00cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.095+0000 7f90f6457640 1 -- 172.21.15.16:0/1892916459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90f015e2a0 msgr2=0x7f90f00cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.095+0000 7f90f6457640 1 --2- 172.21.15.16:0/1892916459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90f015e2a0 0x7f90f00cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.095+0000 7f90f6457640 1 -- 172.21.15.16:0/1892916459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f00cd880 con 0x7f90f015e680 2024-09-19T07:54:36.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.095+0000 7f90f6457640 1 --2- 172.21.15.16:0/1892916459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f015e680 0x7f90f00cc570 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f90e800cb40 tx=0x7f90e800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:36.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.095+0000 7f90e67fc640 1 -- 172.21.15.16:0/1892916459 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8017070 con 0x7f90f015e680 2024-09-19T07:54:36.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.095+0000 7f90e67fc640 1 -- 172.21.15.16:0/1892916459 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e800aca0 con 0x7f90f015e680 2024-09-19T07:54:36.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.095+0000 7f90e67fc640 1 -- 172.21.15.16:0/1892916459 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8012620 con 0x7f90f015e680 2024-09-19T07:54:36.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.095+0000 7f90f7eba640 1 -- 172.21.15.16:0/1892916459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f00b72b0 con 0x7f90f015e680 2024-09-19T07:54:36.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.095+0000 7f90f7eba640 1 -- 172.21.15.16:0/1892916459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f00cdb60 con 0x7f90f015e680 2024-09-19T07:54:36.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.096+0000 7f90f7eba640 1 -- 172.21.15.16:0/1892916459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90f006de70 con 0x7f90f015e680 2024-09-19T07:54:36.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.097+0000 7f90e67fc640 1 -- 172.21.15.16:0/1892916459 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f90e801a070 con 0x7f90f015e680 2024-09-19T07:54:36.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.097+0000 7f90e67fc640 1 --2- 172.21.15.16:0/1892916459 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f90d406c2b0 0x7f90d406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.097+0000 7f90e67fc640 1 -- 172.21.15.16:0/1892916459 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7f90e808b4a0 con 0x7f90f015e680 2024-09-19T07:54:36.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.097+0000 7f90f5c56640 1 --2- 172.21.15.16:0/1892916459 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f90d406c2b0 0x7f90d406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.098+0000 7f90f5c56640 1 --2- 172.21.15.16:0/1892916459 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f90d406c2b0 0x7f90d406e770 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009560 tx=0x7f90ec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:36.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.101+0000 7f90e67fc640 1 -- 172.21.15.16:0/1892916459 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f90f006de70 con 0x7f90f015e680 2024-09-19T07:54:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.276+0000 7f90f7eba640 1 -- 172.21.15.16:0/1892916459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7f90f006d420 con 0x7f90f015e680 2024-09-19T07:54:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.279+0000 7f90e67fc640 1 -- 172.21.15.16:0/1892916459 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v55)=0 v55) ==== 119+0+0 (secure 0 0 0) 0x7f90e8059a80 con 0x7f90f015e680 2024-09-19T07:54:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.281+0000 7f90f7eba640 1 -- 172.21.15.16:0/1892916459 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f90d406c2b0 msgr2=0x7f90d406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.281+0000 7f90f7eba640 1 --2- 172.21.15.16:0/1892916459 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f90d406c2b0 0x7f90d406e770 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009560 tx=0x7f90ec008040 comp rx=0 tx=0).stop 2024-09-19T07:54:36.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.281+0000 7f90f7eba640 1 -- 172.21.15.16:0/1892916459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f015e680 msgr2=0x7f90f00cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.281+0000 7f90f7eba640 1 --2- 172.21.15.16:0/1892916459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f015e680 0x7f90f00cc570 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f90e800cb40 tx=0x7f90e800a060 comp rx=0 tx=0).stop 2024-09-19T07:54:36.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.281+0000 7f90f7eba640 1 -- 172.21.15.16:0/1892916459 shutdown_connections 2024-09-19T07:54:36.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.281+0000 7f90f7eba640 1 --2- 172.21.15.16:0/1892916459 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f90d406c2b0 0x7f90d406e770 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.281+0000 7f90f7eba640 1 --2- 172.21.15.16:0/1892916459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f015e680 0x7f90f00cc570 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.281+0000 7f90f7eba640 1 --2- 172.21.15.16:0/1892916459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90f015e2a0 0x7f90f00cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.281+0000 7f90f7eba640 1 --2- 172.21.15.16:0/1892916459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90f006d960 0x7f90f00cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.281+0000 7f90f7eba640 1 -- 172.21.15.16:0/1892916459 >> 172.21.15.16:0/1892916459 conn(0x7f90f009d760 msgr2=0x7f90f015c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:36.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.281+0000 7f90f7eba640 1 -- 172.21.15.16:0/1892916459 shutdown_connections 2024-09-19T07:54:36.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.281+0000 7f90f7eba640 1 -- 172.21.15.16:0/1892916459 wait complete. 2024-09-19T07:54:36.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 200 2024-09-19T07:54:36.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.398+0000 7f35223bd640 1 Processor -- start 2024-09-19T07:54:36.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.398+0000 7f35223bd640 1 -- start start 2024-09-19T07:54:36.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.399+0000 7f35223bd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f351c0d21f0 0x7f351c0d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.399+0000 7f35223bd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f351c0d2b10 con 0x7f351c141ad0 2024-09-19T07:54:36.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.399+0000 7f35223bd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f351c1620b0 con 0x7f351c072590 2024-09-19T07:54:36.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.399+0000 7f35223bd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f351c141750 con 0x7f351c0d21f0 2024-09-19T07:54:36.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.399+0000 7f351bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f351c0d21f0 0x7f351c0d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.399+0000 7f352095a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f351c141ad0 0x7f351c141f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35552/0 (socket says 172.21.15.16:35552) 2024-09-19T07:54:36.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.399+0000 7f352095a640 1 -- 172.21.15.16:0/107086003 learned_addr learned my addr 172.21.15.16:0/107086003 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:36.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.399+0000 7f351bfff640 1 -- 172.21.15.16:0/107086003 >> v1:172.21.15.146:6789/0 conn(0x7f351c072590 legacy=0x7f351c06d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.399+0000 7f351bfff640 1 -- 172.21.15.16:0/107086003 >> v1:172.21.15.16:6789/0 conn(0x7f351c141ad0 legacy=0x7f351c141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.399+0000 7f351affd640 1 -- 172.21.15.16:0/107086003 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 166822042 0 0) 0x7f351c0d2b10 con 0x7f351c141ad0 2024-09-19T07:54:36.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.399+0000 7f351bfff640 1 -- 172.21.15.16:0/107086003 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351c162370 con 0x7f351c0d21f0 2024-09-19T07:54:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.400+0000 7f351bfff640 1 --2- 172.21.15.16:0/107086003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0d21f0 0x7f351c0d25d0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f350c0097a0 tx=0x7f350c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375c2f522cdda16f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.400+0000 7f351affd640 1 -- 172.21.15.16:0/107086003 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c037070 con 0x7f351c0d21f0 2024-09-19T07:54:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.400+0000 7f351affd640 1 -- 172.21.15.16:0/107086003 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f350c032440 con 0x7f351c0d21f0 2024-09-19T07:54:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.400+0000 7f351affd640 1 -- 172.21.15.16:0/107086003 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c032810 con 0x7f351c0d21f0 2024-09-19T07:54:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.401+0000 7f35223bd640 1 -- 172.21.15.16:0/107086003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0d21f0 msgr2=0x7f351c0d25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.401+0000 7f35223bd640 1 --2- 172.21.15.16:0/107086003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0d21f0 0x7f351c0d25d0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f350c0097a0 tx=0x7f350c02a1a0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.401+0000 7f35223bd640 1 -- 172.21.15.16:0/107086003 shutdown_connections 2024-09-19T07:54:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.401+0000 7f35223bd640 1 --2- 172.21.15.16:0/107086003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0d21f0 0x7f351c0d25d0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.401+0000 7f35223bd640 1 -- 172.21.15.16:0/107086003 >> 172.21.15.16:0/107086003 conn(0x7f351c0b3ed0 msgr2=0x7f351c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.401+0000 7f35223bd640 1 -- 172.21.15.16:0/107086003 shutdown_connections 2024-09-19T07:54:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.401+0000 7f35223bd640 1 -- 172.21.15.16:0/107086003 wait complete. 2024-09-19T07:54:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.402+0000 7f35223bd640 1 Processor -- start 2024-09-19T07:54:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.402+0000 7f35223bd640 1 -- start start 2024-09-19T07:54:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.403+0000 7f35223bd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f351c072590 0x7f351c157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.403+0000 7f35223bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0d21f0 0x7f351c0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.403+0000 7f35223bd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f351c141ad0 0x7f351c15b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.403+0000 7f35223bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f351c162930 con 0x7f351c0d21f0 2024-09-19T07:54:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.403+0000 7f35223bd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f351c06d7f0 con 0x7f351c141ad0 2024-09-19T07:54:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.403+0000 7f35223bd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f351c162c10 con 0x7f351c072590 2024-09-19T07:54:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.403+0000 7f351b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0d21f0 0x7f351c0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.403+0000 7f351b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0d21f0 0x7f351c0b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60204/0 (socket says 172.21.15.16:60204) 2024-09-19T07:54:36.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.403+0000 7f351b7fe640 1 -- 172.21.15.16:0/2621903137 learned_addr learned my addr 172.21.15.16:0/2621903137 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:36.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.403+0000 7f351bfff640 1 --2- 172.21.15.16:0/2621903137 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f351c072590 0x7f351c157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.403+0000 7f352095a640 1 --2- 172.21.15.16:0/2621903137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f351c141ad0 0x7f351c15b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.403+0000 7f351b7fe640 1 -- 172.21.15.16:0/2621903137 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f351c072590 msgr2=0x7f351c157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.403+0000 7f351b7fe640 1 --2- 172.21.15.16:0/2621903137 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f351c072590 0x7f351c157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.403+0000 7f351b7fe640 1 -- 172.21.15.16:0/2621903137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f351c141ad0 msgr2=0x7f351c15b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.403+0000 7f351b7fe640 1 --2- 172.21.15.16:0/2621903137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f351c141ad0 0x7f351c15b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.403+0000 7f351b7fe640 1 -- 172.21.15.16:0/2621903137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351c0b73e0 con 0x7f351c0d21f0 2024-09-19T07:54:36.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.403+0000 7f351b7fe640 1 --2- 172.21.15.16:0/2621903137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0d21f0 0x7f351c0b6b00 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f351000c480 tx=0x7f351000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:36.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.403+0000 7f3518ff9640 1 -- 172.21.15.16:0/2621903137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35100052d0 con 0x7f351c0d21f0 2024-09-19T07:54:36.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.404+0000 7f3518ff9640 1 -- 172.21.15.16:0/2621903137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3510005470 con 0x7f351c0d21f0 2024-09-19T07:54:36.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.404+0000 7f35223bd640 1 -- 172.21.15.16:0/2621903137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351c0b76c0 con 0x7f351c0d21f0 2024-09-19T07:54:36.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.404+0000 7f3518ff9640 1 -- 172.21.15.16:0/2621903137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510012870 con 0x7f351c0d21f0 2024-09-19T07:54:36.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.404+0000 7f35223bd640 1 -- 172.21.15.16:0/2621903137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f351c0b7c80 con 0x7f351c0d21f0 2024-09-19T07:54:36.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.405+0000 7f35223bd640 1 -- 172.21.15.16:0/2621903137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f351c158c30 con 0x7f351c0d21f0 2024-09-19T07:54:36.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.405+0000 7f3518ff9640 1 -- 172.21.15.16:0/2621903137 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3510012a10 con 0x7f351c0d21f0 2024-09-19T07:54:36.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.405+0000 7f3518ff9640 1 --2- 172.21.15.16:0/2621903137 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f350806c2e0 0x7f350806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.406+0000 7f3518ff9640 1 -- 172.21.15.16:0/2621903137 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7f351008b070 con 0x7f351c0d21f0 2024-09-19T07:54:36.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.406+0000 7f351bfff640 1 --2- 172.21.15.16:0/2621903137 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f350806c2e0 0x7f350806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.406+0000 7f351bfff640 1 --2- 172.21.15.16:0/2621903137 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f350806c2e0 0x7f350806e7a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f350c034000 tx=0x7f350c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:36.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.410+0000 7f3518ff9640 1 -- 172.21.15.16:0/2621903137 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3510059090 con 0x7f351c0d21f0 2024-09-19T07:54:36.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.588+0000 7f35223bd640 1 -- 172.21.15.16:0/2621903137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7f351c0690c0 con 0x7f351c0d21f0 2024-09-19T07:54:36.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.591+0000 7f3518ff9640 1 -- 172.21.15.16:0/2621903137 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v56)=0 v56) ==== 128+0+0 (secure 0 0 0) 0x7f351005d390 con 0x7f351c0d21f0 2024-09-19T07:54:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.594+0000 7f35223bd640 1 -- 172.21.15.16:0/2621903137 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f350806c2e0 msgr2=0x7f350806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.594+0000 7f35223bd640 1 --2- 172.21.15.16:0/2621903137 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f350806c2e0 0x7f350806e7a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f350c034000 tx=0x7f350c02e000 comp rx=0 tx=0).stop 2024-09-19T07:54:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.594+0000 7f35223bd640 1 -- 172.21.15.16:0/2621903137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0d21f0 msgr2=0x7f351c0b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.594+0000 7f35223bd640 1 --2- 172.21.15.16:0/2621903137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0d21f0 0x7f351c0b6b00 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f351000c480 tx=0x7f351000c940 comp rx=0 tx=0).stop 2024-09-19T07:54:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.594+0000 7f35223bd640 1 -- 172.21.15.16:0/2621903137 shutdown_connections 2024-09-19T07:54:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.594+0000 7f35223bd640 1 --2- 172.21.15.16:0/2621903137 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f350806c2e0 0x7f350806e7a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.594+0000 7f35223bd640 1 --2- 172.21.15.16:0/2621903137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f351c141ad0 0x7f351c15b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.594+0000 7f35223bd640 1 --2- 172.21.15.16:0/2621903137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351c0d21f0 0x7f351c0b6b00 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.594+0000 7f35223bd640 1 --2- 172.21.15.16:0/2621903137 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f351c072590 0x7f351c157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.594+0000 7f35223bd640 1 -- 172.21.15.16:0/2621903137 >> 172.21.15.16:0/2621903137 conn(0x7f351c0b3ed0 msgr2=0x7f351c160f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.594+0000 7f35223bd640 1 -- 172.21.15.16:0/2621903137 shutdown_connections 2024-09-19T07:54:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.594+0000 7f35223bd640 1 -- 172.21.15.16:0/2621903137 wait complete. 2024-09-19T07:54:36.605 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-09-19T07:54:36.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.711+0000 7fd77f253640 1 Processor -- start 2024-09-19T07:54:36.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.711+0000 7fd77f253640 1 -- start start 2024-09-19T07:54:36.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.711+0000 7fd77f253640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd778162730 0x7fd7780b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.711+0000 7fd77f253640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd7780d45f0 0x7fd7781667b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.711+0000 7fd77f253640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7780d4ba0 con 0x7fd778166370 2024-09-19T07:54:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.711+0000 7fd77f253640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd778166cf0 con 0x7fd7780d45f0 2024-09-19T07:54:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.711+0000 7fd77f253640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd7780d4e80 con 0x7fd778162730 2024-09-19T07:54:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.712+0000 7fd77d7f0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd7780d45f0 0x7fd7781667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.712+0000 7fd777fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd778166370 0x7fd7780d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35566/0 (socket says 172.21.15.16:35566) 2024-09-19T07:54:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.712+0000 7fd777fff640 1 -- 172.21.15.16:0/4084445209 learned_addr learned my addr 172.21.15.16:0/4084445209 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.712+0000 7fd77cfef640 1 --2- 172.21.15.16:0/4084445209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd778162730 0x7fd7780b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.712+0000 7fd77d7f0640 1 -- 172.21.15.16:0/4084445209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd778162730 msgr2=0x7fd7780b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.712+0000 7fd77d7f0640 1 --2- 172.21.15.16:0/4084445209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd778162730 0x7fd7780b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.712+0000 7fd77d7f0640 1 -- 172.21.15.16:0/4084445209 >> v1:172.21.15.16:6789/0 conn(0x7fd778166370 legacy=0x7fd7780d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.712+0000 7fd77d7f0640 1 -- 172.21.15.16:0/4084445209 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7780d2590 con 0x7fd7780d45f0 2024-09-19T07:54:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.712+0000 7fd7777fe640 1 -- 172.21.15.16:0/4084445209 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3724211627 0 0) 0x7fd7780d4ba0 con 0x7fd778166370 2024-09-19T07:54:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.712+0000 7fd77cfef640 1 --2- 172.21.15.16:0/4084445209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd778162730 0x7fd7780b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.713+0000 7fd77d7f0640 1 --2- 172.21.15.16:0/4084445209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7780d45f0 0x7fd7781667b0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fd770009d90 tx=0x7fd77002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfd2a04b7b228a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.713+0000 7fd7777fe640 1 -- 172.21.15.16:0/4084445209 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770007740 con 0x7fd7780d45f0 2024-09-19T07:54:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.713+0000 7fd7777fe640 1 -- 172.21.15.16:0/4084445209 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7700078e0 con 0x7fd7780d45f0 2024-09-19T07:54:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.713+0000 7fd7777fe640 1 -- 172.21.15.16:0/4084445209 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770007cb0 con 0x7fd7780d45f0 2024-09-19T07:54:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.713+0000 7fd77f253640 1 -- 172.21.15.16:0/4084445209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7780d45f0 msgr2=0x7fd7781667b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.713+0000 7fd77f253640 1 --2- 172.21.15.16:0/4084445209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7780d45f0 0x7fd7781667b0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fd770009d90 tx=0x7fd77002a6f0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.713+0000 7fd77f253640 1 -- 172.21.15.16:0/4084445209 shutdown_connections 2024-09-19T07:54:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.713+0000 7fd77f253640 1 --2- 172.21.15.16:0/4084445209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7780d45f0 0x7fd7781667b0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.713+0000 7fd77f253640 1 --2- 172.21.15.16:0/4084445209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd778162730 0x7fd7780b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.713+0000 7fd77f253640 1 -- 172.21.15.16:0/4084445209 >> 172.21.15.16:0/4084445209 conn(0x7fd77809d760 msgr2=0x7fd77808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.714+0000 7fd77f253640 1 -- 172.21.15.16:0/4084445209 shutdown_connections 2024-09-19T07:54:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.714+0000 7fd77f253640 1 -- 172.21.15.16:0/4084445209 wait complete. 2024-09-19T07:54:36.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.715+0000 7fd77f253640 1 Processor -- start 2024-09-19T07:54:36.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.715+0000 7fd77f253640 1 -- start start 2024-09-19T07:54:36.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.716+0000 7fd77f253640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd7780d45f0 0x7fd7781612d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.716+0000 7fd77f253640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd778162730 0x7fd7780ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.716+0000 7fd77f253640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd778166370 0x7fd7781f8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.716+0000 7fd77f253640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd778162c90 con 0x7fd778166370 2024-09-19T07:54:36.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.716+0000 7fd77f253640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd778162b10 con 0x7fd778162730 2024-09-19T07:54:36.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.716+0000 7fd77f253640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd7780d2d90 con 0x7fd7780d45f0 2024-09-19T07:54:36.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.716+0000 7fd77d7f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd778166370 0x7fd7781f8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.716+0000 7fd77d7f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd778166370 0x7fd7781f8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60218/0 (socket says 172.21.15.16:60218) 2024-09-19T07:54:36.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.716+0000 7fd77d7f0640 1 -- 172.21.15.16:0/1694387794 learned_addr learned my addr 172.21.15.16:0/1694387794 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:36.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.716+0000 7fd777fff640 1 --2- 172.21.15.16:0/1694387794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd778162730 0x7fd7780ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.716+0000 7fd77cfef640 1 --2- 172.21.15.16:0/1694387794 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd7780d45f0 0x7fd7781612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.716+0000 7fd77d7f0640 1 -- 172.21.15.16:0/1694387794 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd7780d45f0 msgr2=0x7fd7781612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.716+0000 7fd77d7f0640 1 --2- 172.21.15.16:0/1694387794 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd7780d45f0 0x7fd7781612d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.716+0000 7fd77d7f0640 1 -- 172.21.15.16:0/1694387794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd778162730 msgr2=0x7fd7780ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.716+0000 7fd77d7f0640 1 --2- 172.21.15.16:0/1694387794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd778162730 0x7fd7780ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.716+0000 7fd77d7f0640 1 -- 172.21.15.16:0/1694387794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7781f8940 con 0x7fd778166370 2024-09-19T07:54:36.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.716+0000 7fd777fff640 1 --2- 172.21.15.16:0/1694387794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd778162730 0x7fd7780ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:36.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.716+0000 7fd77d7f0640 1 --2- 172.21.15.16:0/1694387794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd778166370 0x7fd7781f8400 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fd77002a730 tx=0x7fd770004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:36.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.717+0000 7fd7757fa640 1 -- 172.21.15.16:0/1694387794 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770040070 con 0x7fd778166370 2024-09-19T07:54:36.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.717+0000 7fd7757fa640 1 -- 172.21.15.16:0/1694387794 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd770031bc0 con 0x7fd778166370 2024-09-19T07:54:36.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.717+0000 7fd77f253640 1 -- 172.21.15.16:0/1694387794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7781f8be0 con 0x7fd778166370 2024-09-19T07:54:36.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.717+0000 7fd7757fa640 1 -- 172.21.15.16:0/1694387794 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770009450 con 0x7fd778166370 2024-09-19T07:54:36.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.717+0000 7fd77f253640 1 -- 172.21.15.16:0/1694387794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7781f91a0 con 0x7fd778166370 2024-09-19T07:54:36.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.718+0000 7fd77f253640 1 -- 172.21.15.16:0/1694387794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7781667b0 con 0x7fd778166370 2024-09-19T07:54:36.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.718+0000 7fd7757fa640 1 -- 172.21.15.16:0/1694387794 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd770039040 con 0x7fd778166370 2024-09-19T07:54:36.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.719+0000 7fd7757fa640 1 --2- 172.21.15.16:0/1694387794 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd76406c2b0 0x7fd76406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.719+0000 7fd7757fa640 1 -- 172.21.15.16:0/1694387794 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7fd7700ac820 con 0x7fd778166370 2024-09-19T07:54:36.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.719+0000 7fd77cfef640 1 --2- 172.21.15.16:0/1694387794 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd76406c2b0 0x7fd76406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.719+0000 7fd77cfef640 1 --2- 172.21.15.16:0/1694387794 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd76406c2b0 0x7fd76406e770 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd7781f5ff0 tx=0x7fd768008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:36.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.723+0000 7fd7757fa640 1 -- 172.21.15.16:0/1694387794 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd77007ae00 con 0x7fd778166370 2024-09-19T07:54:36.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.900+0000 7fd77f253640 1 -- 172.21.15.16:0/1694387794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7fd7780d21f0 con 0x7fd778166370 2024-09-19T07:54:36.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.900+0000 7fd7757fa640 1 -- 172.21.15.16:0/1694387794 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v56) ==== 106+0+4 (secure 0 0 0) 0x7fd77007a700 con 0x7fd778166370 2024-09-19T07:54:36.900 INFO:teuthology.orchestra.run.smithi016.stdout:100 2024-09-19T07:54:36.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.902+0000 7fd77f253640 1 -- 172.21.15.16:0/1694387794 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd76406c2b0 msgr2=0x7fd76406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.902+0000 7fd77f253640 1 --2- 172.21.15.16:0/1694387794 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd76406c2b0 0x7fd76406e770 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd7781f5ff0 tx=0x7fd768008040 comp rx=0 tx=0).stop 2024-09-19T07:54:36.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.902+0000 7fd77f253640 1 -- 172.21.15.16:0/1694387794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd778166370 msgr2=0x7fd7781f8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.902+0000 7fd77f253640 1 --2- 172.21.15.16:0/1694387794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd778166370 0x7fd7781f8400 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fd77002a730 tx=0x7fd770004a40 comp rx=0 tx=0).stop 2024-09-19T07:54:36.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.902+0000 7fd77f253640 1 -- 172.21.15.16:0/1694387794 shutdown_connections 2024-09-19T07:54:36.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.902+0000 7fd77f253640 1 --2- 172.21.15.16:0/1694387794 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd76406c2b0 0x7fd76406e770 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.902+0000 7fd77f253640 1 --2- 172.21.15.16:0/1694387794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd778166370 0x7fd7781f8400 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.902+0000 7fd77f253640 1 --2- 172.21.15.16:0/1694387794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd778162730 0x7fd7780ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.902+0000 7fd77f253640 1 --2- 172.21.15.16:0/1694387794 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd7780d45f0 0x7fd7781612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.902+0000 7fd77f253640 1 -- 172.21.15.16:0/1694387794 >> 172.21.15.16:0/1694387794 conn(0x7fd77809d760 msgr2=0x7fd77808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:36.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.902+0000 7fd77f253640 1 -- 172.21.15.16:0/1694387794 shutdown_connections 2024-09-19T07:54:36.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:36.902+0000 7fd77f253640 1 -- 172.21.15.16:0/1694387794 wait complete. 2024-09-19T07:54:36.913 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2024-09-19T07:54:37.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.019+0000 7f5b76398640 1 Processor -- start 2024-09-19T07:54:37.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.019+0000 7f5b76398640 1 -- start start 2024-09-19T07:54:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.020+0000 7f5b76398640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b701619b0 0x7f5b700d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.020+0000 7f5b76398640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5b700b0a80 0x7f5b700d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.020+0000 7f5b76398640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b7006c990 con 0x7f5b70161d90 2024-09-19T07:54:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.020+0000 7f5b76398640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f5b700d0ec0 con 0x7f5b700b0a80 2024-09-19T07:54:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.020+0000 7f5b76398640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5b700cec10 con 0x7f5b701619b0 2024-09-19T07:54:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.020+0000 7f5b6f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5b700b0a80 0x7f5b700d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.020+0000 7f5b74935640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5b70161d90 0x7f5b700cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35580/0 (socket says 172.21.15.16:35580) 2024-09-19T07:54:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.020+0000 7f5b74935640 1 -- 172.21.15.16:0/1739937667 learned_addr learned my addr 172.21.15.16:0/1739937667 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.020+0000 7f5b6ffff640 1 --2- 172.21.15.16:0/1739937667 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b701619b0 0x7f5b700d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.020+0000 7f5b6f7fe640 1 -- 172.21.15.16:0/1739937667 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b701619b0 msgr2=0x7f5b700d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.020+0000 7f5b6f7fe640 1 --2- 172.21.15.16:0/1739937667 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b701619b0 0x7f5b700d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.020+0000 7f5b6f7fe640 1 -- 172.21.15.16:0/1739937667 >> v1:172.21.15.16:6789/0 conn(0x7f5b70161d90 legacy=0x7f5b700cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.020+0000 7f5b6f7fe640 1 -- 172.21.15.16:0/1739937667 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b700ce930 con 0x7f5b700b0a80 2024-09-19T07:54:37.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.020+0000 7f5b6effd640 1 -- 172.21.15.16:0/1739937667 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2532867722 0 0) 0x7f5b7006c990 con 0x7f5b70161d90 2024-09-19T07:54:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.021+0000 7f5b6f7fe640 1 --2- 172.21.15.16:0/1739937667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b700b0a80 0x7f5b700d0980 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f5b640097a0 tx=0x7f5b6402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c32727cd72c63e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.021+0000 7f5b6effd640 1 -- 172.21.15.16:0/1739937667 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64037070 con 0x7f5b700b0a80 2024-09-19T07:54:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.021+0000 7f5b6effd640 1 -- 172.21.15.16:0/1739937667 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b64032440 con 0x7f5b700b0a80 2024-09-19T07:54:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.021+0000 7f5b6effd640 1 -- 172.21.15.16:0/1739937667 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64032810 con 0x7f5b700b0a80 2024-09-19T07:54:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.022+0000 7f5b76398640 1 -- 172.21.15.16:0/1739937667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b700b0a80 msgr2=0x7f5b700d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.022+0000 7f5b76398640 1 --2- 172.21.15.16:0/1739937667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b700b0a80 0x7f5b700d0980 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f5b640097a0 tx=0x7f5b6402a180 comp rx=0 tx=0).stop 2024-09-19T07:54:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.022+0000 7f5b76398640 1 -- 172.21.15.16:0/1739937667 shutdown_connections 2024-09-19T07:54:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.022+0000 7f5b76398640 1 --2- 172.21.15.16:0/1739937667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b700b0a80 0x7f5b700d0980 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.022+0000 7f5b76398640 1 --2- 172.21.15.16:0/1739937667 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b701619b0 0x7f5b700d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.022+0000 7f5b76398640 1 -- 172.21.15.16:0/1739937667 >> 172.21.15.16:0/1739937667 conn(0x7f5b7009a9f0 msgr2=0x7f5b700b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.022+0000 7f5b76398640 1 -- 172.21.15.16:0/1739937667 shutdown_connections 2024-09-19T07:54:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.022+0000 7f5b76398640 1 -- 172.21.15.16:0/1739937667 wait complete. 2024-09-19T07:54:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.023+0000 7f5b76398640 1 Processor -- start 2024-09-19T07:54:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.023+0000 7f5b76398640 1 -- start start 2024-09-19T07:54:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.024+0000 7f5b76398640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b700b0a80 0x7f5b701f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.024+0000 7f5b76398640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b701619b0 0x7f5b700c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.024+0000 7f5b76398640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b70161d90 0x7f5b701f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.024+0000 7f5b76398640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7006c620 con 0x7f5b700b0a80 2024-09-19T07:54:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.024+0000 7f5b76398640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5b700b0820 con 0x7f5b701619b0 2024-09-19T07:54:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.024+0000 7f5b76398640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5b700ccce0 con 0x7f5b70161d90 2024-09-19T07:54:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.024+0000 7f5b6ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b700b0a80 0x7f5b701f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.024+0000 7f5b6ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b700b0a80 0x7f5b701f50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60234/0 (socket says 172.21.15.16:60234) 2024-09-19T07:54:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.024+0000 7f5b6ffff640 1 -- 172.21.15.16:0/110445862 learned_addr learned my addr 172.21.15.16:0/110445862 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.024+0000 7f5b6f7fe640 1 --2- 172.21.15.16:0/110445862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b701619b0 0x7f5b700c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.024+0000 7f5b74935640 1 --2- 172.21.15.16:0/110445862 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b70161d90 0x7f5b701f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.024+0000 7f5b6ffff640 1 -- 172.21.15.16:0/110445862 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b70161d90 msgr2=0x7f5b701f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.024+0000 7f5b6ffff640 1 --2- 172.21.15.16:0/110445862 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b70161d90 0x7f5b701f7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.024+0000 7f5b6ffff640 1 -- 172.21.15.16:0/110445862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b701619b0 msgr2=0x7f5b700c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.024+0000 7f5b6ffff640 1 --2- 172.21.15.16:0/110445862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b701619b0 0x7f5b700c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.024+0000 7f5b6ffff640 1 -- 172.21.15.16:0/110445862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b700c5400 con 0x7f5b700b0a80 2024-09-19T07:54:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.025+0000 7f5b6ffff640 1 --2- 172.21.15.16:0/110445862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b700b0a80 0x7f5b701f50e0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000d800 tx=0x7f5b6000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.025+0000 7f5b6cff9640 1 -- 172.21.15.16:0/110445862 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6000b660 con 0x7f5b700b0a80 2024-09-19T07:54:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.025+0000 7f5b6cff9640 1 -- 172.21.15.16:0/110445862 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b60019070 con 0x7f5b700b0a80 2024-09-19T07:54:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.025+0000 7f5b76398640 1 -- 172.21.15.16:0/110445862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b700b1090 con 0x7f5b700b0a80 2024-09-19T07:54:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.025+0000 7f5b6cff9640 1 -- 172.21.15.16:0/110445862 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6000ba20 con 0x7f5b700b0a80 2024-09-19T07:54:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.025+0000 7f5b76398640 1 -- 172.21.15.16:0/110445862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b700b1650 con 0x7f5b700b0a80 2024-09-19T07:54:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.026+0000 7f5b76398640 1 -- 172.21.15.16:0/110445862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b700cd330 con 0x7f5b700b0a80 2024-09-19T07:54:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.026+0000 7f5b6cff9640 1 -- 172.21.15.16:0/110445862 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5b6000bbc0 con 0x7f5b700b0a80 2024-09-19T07:54:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.027+0000 7f5b6cff9640 1 --2- 172.21.15.16:0/110445862 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5b5c06c2e0 0x7f5b5c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.027+0000 7f5b6cff9640 1 -- 172.21.15.16:0/110445862 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7f5b6008b970 con 0x7f5b700b0a80 2024-09-19T07:54:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.027+0000 7f5b6f7fe640 1 --2- 172.21.15.16:0/110445862 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5b5c06c2e0 0x7f5b5c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.028+0000 7f5b6f7fe640 1 --2- 172.21.15.16:0/110445862 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5b5c06c2e0 0x7f5b5c06e7a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5b64034000 tx=0x7f5b6402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.031+0000 7f5b6cff9640 1 -- 172.21.15.16:0/110445862 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5b6001c360 con 0x7f5b700b0a80 2024-09-19T07:54:37.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.206+0000 7f5b76398640 1 -- 172.21.15.16:0/110445862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7f5b700b0e60 con 0x7f5b700b0a80 2024-09-19T07:54:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.209+0000 7f5b6cff9640 1 -- 172.21.15.16:0/110445862 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v57)=0 v57) ==== 124+0+0 (secure 0 0 0) 0x7f5b6005ee20 con 0x7f5b700b0a80 2024-09-19T07:54:37.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.211+0000 7f5b76398640 1 -- 172.21.15.16:0/110445862 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5b5c06c2e0 msgr2=0x7f5b5c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.211+0000 7f5b76398640 1 --2- 172.21.15.16:0/110445862 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5b5c06c2e0 0x7f5b5c06e7a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5b64034000 tx=0x7f5b6402e000 comp rx=0 tx=0).stop 2024-09-19T07:54:37.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.211+0000 7f5b76398640 1 -- 172.21.15.16:0/110445862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b700b0a80 msgr2=0x7f5b701f50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.211+0000 7f5b76398640 1 --2- 172.21.15.16:0/110445862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b700b0a80 0x7f5b701f50e0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000d800 tx=0x7f5b6000dcc0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.211+0000 7f5b76398640 1 -- 172.21.15.16:0/110445862 shutdown_connections 2024-09-19T07:54:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.211+0000 7f5b76398640 1 --2- 172.21.15.16:0/110445862 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5b5c06c2e0 0x7f5b5c06e7a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.211+0000 7f5b76398640 1 --2- 172.21.15.16:0/110445862 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b70161d90 0x7f5b701f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.211+0000 7f5b76398640 1 --2- 172.21.15.16:0/110445862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b701619b0 0x7f5b700c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.211+0000 7f5b76398640 1 --2- 172.21.15.16:0/110445862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b700b0a80 0x7f5b701f50e0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.211+0000 7f5b76398640 1 -- 172.21.15.16:0/110445862 >> 172.21.15.16:0/110445862 conn(0x7f5b7009a9f0 msgr2=0x7f5b70160aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.212+0000 7f5b76398640 1 -- 172.21.15.16:0/110445862 shutdown_connections 2024-09-19T07:54:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.212+0000 7f5b76398640 1 -- 172.21.15.16:0/110445862 wait complete. 2024-09-19T07:54:37.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2024-09-19T07:54:37.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.326+0000 7fda04d66640 1 Processor -- start 2024-09-19T07:54:37.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.326+0000 7fda04d66640 1 -- start start 2024-09-19T07:54:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.327+0000 7fda04d66640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fda0015f0d0 0x7fda0015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.327+0000 7fda04d66640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fda000d0070 0x7fda000d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.327+0000 7fda04d66640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda000d1940 0x7fda000d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.327+0000 7fda04d66640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fda000cf2b0 con 0x7fda0015f0d0 2024-09-19T07:54:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.327+0000 7fda04d66640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fda000cf590 con 0x7fda000d0070 2024-09-19T07:54:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.327+0000 7fda04d66640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fda000cf870 con 0x7fda000d1940 2024-09-19T07:54:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.327+0000 7fd9fe59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fda0015f0d0 0x7fda0015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.327+0000 7fd9fe59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fda0015f0d0 0x7fda0015f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60246/0 (socket says 172.21.15.16:60246) 2024-09-19T07:54:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.327+0000 7fd9fe59c640 1 -- 172.21.15.16:0/2808141486 learned_addr learned my addr 172.21.15.16:0/2808141486 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.327+0000 7fd9fdd9b640 1 --2- 172.21.15.16:0/2808141486 >> v2:172.21.15.146:3300/0 conn(0x7fda000d0070 0x7fda000d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.327+0000 7fd9fed9d640 1 --2- 172.21.15.16:0/2808141486 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda000d1940 0x7fda000d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.327+0000 7fd9fe59c640 1 -- 172.21.15.16:0/2808141486 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda000d1940 msgr2=0x7fda000d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:37.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.327+0000 7fd9fe59c640 1 --2- 172.21.15.16:0/2808141486 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda000d1940 0x7fda000d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.327+0000 7fd9fe59c640 1 -- 172.21.15.16:0/2808141486 >> v2:172.21.15.146:3300/0 conn(0x7fda000d0070 msgr2=0x7fda000d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:37.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.327+0000 7fd9fe59c640 1 --2- 172.21.15.16:0/2808141486 >> v2:172.21.15.146:3300/0 conn(0x7fda000d0070 0x7fda000d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.328+0000 7fd9fe59c640 1 -- 172.21.15.16:0/2808141486 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda000cfb50 con 0x7fda0015f0d0 2024-09-19T07:54:37.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.328+0000 7fd9fe59c640 1 --2- 172.21.15.16:0/2808141486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0015f0d0 0x7fda0015f4b0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fda00162fc0 tx=0x7fd9f002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf2daca9a2cf441 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:37.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.328+0000 7fd9fd59a640 1 -- 172.21.15.16:0/2808141486 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0037070 con 0x7fda0015f0d0 2024-09-19T07:54:37.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.328+0000 7fd9fd59a640 1 -- 172.21.15.16:0/2808141486 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f0032440 con 0x7fda0015f0d0 2024-09-19T07:54:37.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.328+0000 7fd9fd59a640 1 -- 172.21.15.16:0/2808141486 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0032810 con 0x7fda0015f0d0 2024-09-19T07:54:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.329+0000 7fda04d66640 1 -- 172.21.15.16:0/2808141486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0015f0d0 msgr2=0x7fda0015f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.329+0000 7fda04d66640 1 --2- 172.21.15.16:0/2808141486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0015f0d0 0x7fda0015f4b0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fda00162fc0 tx=0x7fd9f002a140 comp rx=0 tx=0).stop 2024-09-19T07:54:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.329+0000 7fda04d66640 1 -- 172.21.15.16:0/2808141486 shutdown_connections 2024-09-19T07:54:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.329+0000 7fda04d66640 1 --2- 172.21.15.16:0/2808141486 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda000d1940 0x7fda000d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.329+0000 7fda04d66640 1 --2- 172.21.15.16:0/2808141486 >> v2:172.21.15.146:3300/0 conn(0x7fda000d0070 0x7fda000d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.329+0000 7fda04d66640 1 --2- 172.21.15.16:0/2808141486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda0015f0d0 0x7fda0015f4b0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.329+0000 7fda04d66640 1 -- 172.21.15.16:0/2808141486 >> 172.21.15.16:0/2808141486 conn(0x7fda0009d760 msgr2=0x7fda0008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.329+0000 7fda04d66640 1 -- 172.21.15.16:0/2808141486 shutdown_connections 2024-09-19T07:54:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.329+0000 7fda04d66640 1 -- 172.21.15.16:0/2808141486 wait complete. 2024-09-19T07:54:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.330+0000 7fda04d66640 1 Processor -- start 2024-09-19T07:54:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.330+0000 7fda04d66640 1 -- start start 2024-09-19T07:54:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.331+0000 7fda04d66640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda000d0070 0x7fda001547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.331+0000 7fda04d66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda000d1940 0x7fda000d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.331+0000 7fda04d66640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda0015f0d0 0x7fda000d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.331+0000 7fda04d66640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda000d47f0 con 0x7fda000d1940 2024-09-19T07:54:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.331+0000 7fda04d66640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fda000d4670 con 0x7fda000d0070 2024-09-19T07:54:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.331+0000 7fda04d66640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fda000d4970 con 0x7fda0015f0d0 2024-09-19T07:54:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.331+0000 7fd9fdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda000d1940 0x7fda000d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.331+0000 7fd9fdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda000d1940 0x7fda000d1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60254/0 (socket says 172.21.15.16:60254) 2024-09-19T07:54:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.331+0000 7fd9fdd9b640 1 -- 172.21.15.16:0/2739862933 learned_addr learned my addr 172.21.15.16:0/2739862933 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.331+0000 7fd9fed9d640 1 --2- 172.21.15.16:0/2739862933 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda0015f0d0 0x7fda000d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.331+0000 7fd9fe59c640 1 --2- 172.21.15.16:0/2739862933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda000d0070 0x7fda001547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.332+0000 7fd9fdd9b640 1 -- 172.21.15.16:0/2739862933 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda0015f0d0 msgr2=0x7fda000d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.332+0000 7fd9fdd9b640 1 --2- 172.21.15.16:0/2739862933 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda0015f0d0 0x7fda000d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.332+0000 7fd9fdd9b640 1 -- 172.21.15.16:0/2739862933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda000d0070 msgr2=0x7fda001547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.332+0000 7fd9fdd9b640 1 --2- 172.21.15.16:0/2739862933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda000d0070 0x7fda001547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.332+0000 7fd9fdd9b640 1 -- 172.21.15.16:0/2739862933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda000c8db0 con 0x7fda000d1940 2024-09-19T07:54:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.332+0000 7fd9fdd9b640 1 --2- 172.21.15.16:0/2739862933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda000d1940 0x7fda000d1f90 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400d800 tx=0x7fd9f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.332+0000 7fd9eeffd640 1 -- 172.21.15.16:0/2739862933 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4018030 con 0x7fda000d1940 2024-09-19T07:54:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.332+0000 7fd9eeffd640 1 -- 172.21.15.16:0/2739862933 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f40096a0 con 0x7fda000d1940 2024-09-19T07:54:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.332+0000 7fda04d66640 1 -- 172.21.15.16:0/2739862933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda000c8ad0 con 0x7fda000d1940 2024-09-19T07:54:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.332+0000 7fd9eeffd640 1 -- 172.21.15.16:0/2739862933 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4009870 con 0x7fda000d1940 2024-09-19T07:54:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.332+0000 7fda04d66640 1 -- 172.21.15.16:0/2739862933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda000cb830 con 0x7fda000d1940 2024-09-19T07:54:37.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.333+0000 7fda04d66640 1 -- 172.21.15.16:0/2739862933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda000d1400 con 0x7fda000d1940 2024-09-19T07:54:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.334+0000 7fd9eeffd640 1 -- 172.21.15.16:0/2739862933 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd9f401b070 con 0x7fda000d1940 2024-09-19T07:54:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.334+0000 7fd9eeffd640 1 --2- 172.21.15.16:0/2739862933 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd9e006c2b0 0x7fd9e006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.334+0000 7fd9eeffd640 1 -- 172.21.15.16:0/2739862933 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7fd9f408b780 con 0x7fda000d1940 2024-09-19T07:54:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.334+0000 7fd9fe59c640 1 --2- 172.21.15.16:0/2739862933 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd9e006c2b0 0x7fd9e006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.335+0000 7fd9fe59c640 1 --2- 172.21.15.16:0/2739862933 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd9e006c2b0 0x7fd9e006e770 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0034000 tx=0x7fd9f002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.339+0000 7fd9eeffd640 1 -- 172.21.15.16:0/2739862933 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd9f4059680 con 0x7fda000d1940 2024-09-19T07:54:37.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.516+0000 7fda04d66640 1 -- 172.21.15.16:0/2739862933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) -- 0x7fda0006d2f0 con 0x7fda000d1940 2024-09-19T07:54:37.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.516+0000 7fd9eeffd640 1 -- 172.21.15.16:0/2739862933 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v57) ==== 106+0+8 (secure 0 0 0) 0x7fd9f405d980 con 0x7fda000d1940 2024-09-19T07:54:37.517 INFO:teuthology.orchestra.run.smithi016.stdout:1048576 2024-09-19T07:54:37.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.518+0000 7fda04d66640 1 -- 172.21.15.16:0/2739862933 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd9e006c2b0 msgr2=0x7fd9e006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.518+0000 7fda04d66640 1 --2- 172.21.15.16:0/2739862933 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd9e006c2b0 0x7fd9e006e770 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0034000 tx=0x7fd9f002e000 comp rx=0 tx=0).stop 2024-09-19T07:54:37.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.518+0000 7fda04d66640 1 -- 172.21.15.16:0/2739862933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda000d1940 msgr2=0x7fda000d1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.518+0000 7fda04d66640 1 --2- 172.21.15.16:0/2739862933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda000d1940 0x7fda000d1f90 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400d800 tx=0x7fd9f400dcc0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.518+0000 7fda04d66640 1 -- 172.21.15.16:0/2739862933 shutdown_connections 2024-09-19T07:54:37.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.518+0000 7fda04d66640 1 --2- 172.21.15.16:0/2739862933 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd9e006c2b0 0x7fd9e006e770 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.518+0000 7fda04d66640 1 --2- 172.21.15.16:0/2739862933 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda0015f0d0 0x7fda000d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.518+0000 7fda04d66640 1 --2- 172.21.15.16:0/2739862933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda000d1940 0x7fda000d1f90 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.518+0000 7fda04d66640 1 --2- 172.21.15.16:0/2739862933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda000d0070 0x7fda001547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.518+0000 7fda04d66640 1 -- 172.21.15.16:0/2739862933 >> 172.21.15.16:0/2739862933 conn(0x7fda0009d760 msgr2=0x7fda0006d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:37.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.518+0000 7fda04d66640 1 -- 172.21.15.16:0/2739862933 shutdown_connections 2024-09-19T07:54:37.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.518+0000 7fda04d66640 1 -- 172.21.15.16:0/2739862933 wait complete. 2024-09-19T07:54:37.530 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-09-19T07:54:37.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.633+0000 7ffb6073a640 1 Processor -- start 2024-09-19T07:54:37.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.633+0000 7ffb6073a640 1 -- start start 2024-09-19T07:54:37.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.634+0000 7ffb6073a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffb5815e2a0 0x7ffb5806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.634+0000 7ffb6073a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffb5815e680 0x7ffb580cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.634+0000 7ffb6073a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb580d1880 con 0x7ffb5806d960 2024-09-19T07:54:37.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.634+0000 7ffb6073a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ffb580d1b60 con 0x7ffb5815e680 2024-09-19T07:54:37.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.634+0000 7ffb6073a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ffb580cf2d0 con 0x7ffb5815e2a0 2024-09-19T07:54:37.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.634+0000 7ffb5dcd5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffb5815e680 0x7ffb580cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.634+0000 7ffb5ecd7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffb5806d960 0x7ffb580d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35596/0 (socket says 172.21.15.16:35596) 2024-09-19T07:54:37.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.634+0000 7ffb5ecd7640 1 -- 172.21.15.16:0/3863951737 learned_addr learned my addr 172.21.15.16:0/3863951737 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:37.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.634+0000 7ffb5e4d6640 1 --2- 172.21.15.16:0/3863951737 >> v2:172.21.15.146:3300/0 conn(0x7ffb5815e2a0 0x7ffb5806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.635+0000 7ffb5dcd5640 1 -- 172.21.15.16:0/3863951737 >> v2:172.21.15.146:3300/0 conn(0x7ffb5815e2a0 msgr2=0x7ffb5806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:37.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.635+0000 7ffb5dcd5640 1 --2- 172.21.15.16:0/3863951737 >> v2:172.21.15.146:3300/0 conn(0x7ffb5815e2a0 0x7ffb5806d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.635+0000 7ffb5dcd5640 1 -- 172.21.15.16:0/3863951737 >> v1:172.21.15.16:6789/0 conn(0x7ffb5806d960 legacy=0x7ffb580d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.635+0000 7ffb5dcd5640 1 -- 172.21.15.16:0/3863951737 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb580cf5b0 con 0x7ffb5815e680 2024-09-19T07:54:37.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.635+0000 7ffb5d4d4640 1 -- 172.21.15.16:0/3863951737 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3563451364 0 0) 0x7ffb580d1880 con 0x7ffb5806d960 2024-09-19T07:54:37.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.635+0000 7ffb5dcd5640 1 --2- 172.21.15.16:0/3863951737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5815e680 0x7ffb580cfad0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7ffb580b6fa0 tx=0x7ffb4802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d823650298a737 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:37.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.635+0000 7ffb5d4d4640 1 -- 172.21.15.16:0/3863951737 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb48037070 con 0x7ffb5815e680 2024-09-19T07:54:37.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.635+0000 7ffb5d4d4640 1 -- 172.21.15.16:0/3863951737 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb48032440 con 0x7ffb5815e680 2024-09-19T07:54:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.635+0000 7ffb5d4d4640 1 -- 172.21.15.16:0/3863951737 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb48032810 con 0x7ffb5815e680 2024-09-19T07:54:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.636+0000 7ffb6073a640 1 -- 172.21.15.16:0/3863951737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5815e680 msgr2=0x7ffb580cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.636+0000 7ffb6073a640 1 --2- 172.21.15.16:0/3863951737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5815e680 0x7ffb580cfad0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7ffb580b6fa0 tx=0x7ffb4802a1a0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.636+0000 7ffb6073a640 1 -- 172.21.15.16:0/3863951737 shutdown_connections 2024-09-19T07:54:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.636+0000 7ffb6073a640 1 --2- 172.21.15.16:0/3863951737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5815e680 0x7ffb580cfad0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.636+0000 7ffb6073a640 1 --2- 172.21.15.16:0/3863951737 >> v2:172.21.15.146:3300/0 conn(0x7ffb5815e2a0 0x7ffb5806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.636+0000 7ffb6073a640 1 -- 172.21.15.16:0/3863951737 >> 172.21.15.16:0/3863951737 conn(0x7ffb5809d760 msgr2=0x7ffb5808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.636+0000 7ffb6073a640 1 -- 172.21.15.16:0/3863951737 shutdown_connections 2024-09-19T07:54:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.636+0000 7ffb6073a640 1 -- 172.21.15.16:0/3863951737 wait complete. 2024-09-19T07:54:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.637+0000 7ffb6073a640 1 Processor -- start 2024-09-19T07:54:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.637+0000 7ffb6073a640 1 -- start start 2024-09-19T07:54:37.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.638+0000 7ffb6073a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5806d960 0x7ffb580b9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.638+0000 7ffb6073a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb5815e2a0 0x7ffb580b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.638+0000 7ffb6073a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb5815e680 0x7ffb580ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.638+0000 7ffb6073a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffb580d4c00 con 0x7ffb5806d960 2024-09-19T07:54:37.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.638+0000 7ffb6073a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ffb580d4a80 con 0x7ffb5815e680 2024-09-19T07:54:37.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.638+0000 7ffb6073a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ffb580d0010 con 0x7ffb5815e2a0 2024-09-19T07:54:37.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.638+0000 7ffb5e4d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5806d960 0x7ffb580b9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.638+0000 7ffb5e4d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5806d960 0x7ffb580b9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60272/0 (socket says 172.21.15.16:60272) 2024-09-19T07:54:37.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.638+0000 7ffb5e4d6640 1 -- 172.21.15.16:0/4016405912 learned_addr learned my addr 172.21.15.16:0/4016405912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:37.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.638+0000 7ffb5ecd7640 1 --2- 172.21.15.16:0/4016405912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb5815e680 0x7ffb580ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.638+0000 7ffb5dcd5640 1 --2- 172.21.15.16:0/4016405912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb5815e2a0 0x7ffb580b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.639+0000 7ffb5e4d6640 1 -- 172.21.15.16:0/4016405912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb5815e2a0 msgr2=0x7ffb580b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.639+0000 7ffb5e4d6640 1 --2- 172.21.15.16:0/4016405912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb5815e2a0 0x7ffb580b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.639+0000 7ffb5e4d6640 1 -- 172.21.15.16:0/4016405912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb5815e680 msgr2=0x7ffb580ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.639+0000 7ffb5e4d6640 1 --2- 172.21.15.16:0/4016405912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb5815e680 0x7ffb580ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.639+0000 7ffb5e4d6640 1 -- 172.21.15.16:0/4016405912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb580b5a40 con 0x7ffb5806d960 2024-09-19T07:54:37.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.639+0000 7ffb5e4d6640 1 --2- 172.21.15.16:0/4016405912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5806d960 0x7ffb580b9f60 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400c7f0 tx=0x7ffb5400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:37.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.639+0000 7ffb4effd640 1 -- 172.21.15.16:0/4016405912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb54015070 con 0x7ffb5806d960 2024-09-19T07:54:37.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.639+0000 7ffb4effd640 1 -- 172.21.15.16:0/4016405912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb54004240 con 0x7ffb5806d960 2024-09-19T07:54:37.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.639+0000 7ffb6073a640 1 -- 172.21.15.16:0/4016405912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb580cad40 con 0x7ffb5806d960 2024-09-19T07:54:37.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.639+0000 7ffb4effd640 1 -- 172.21.15.16:0/4016405912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb54007620 con 0x7ffb5806d960 2024-09-19T07:54:37.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.639+0000 7ffb6073a640 1 -- 172.21.15.16:0/4016405912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb580b7670 con 0x7ffb5806d960 2024-09-19T07:54:37.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.640+0000 7ffb6073a640 1 -- 172.21.15.16:0/4016405912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb5806d420 con 0x7ffb5806d960 2024-09-19T07:54:37.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.640+0000 7ffb4effd640 1 -- 172.21.15.16:0/4016405912 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ffb540079d0 con 0x7ffb5806d960 2024-09-19T07:54:37.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.641+0000 7ffb4effd640 1 --2- 172.21.15.16:0/4016405912 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffb3c06c2b0 0x7ffb3c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.641+0000 7ffb4effd640 1 -- 172.21.15.16:0/4016405912 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7ffb5408b9d0 con 0x7ffb5806d960 2024-09-19T07:54:37.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.641+0000 7ffb5dcd5640 1 --2- 172.21.15.16:0/4016405912 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffb3c06c2b0 0x7ffb3c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.642+0000 7ffb5dcd5640 1 --2- 172.21.15.16:0/4016405912 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffb3c06c2b0 0x7ffb3c06e770 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ffb48034000 tx=0x7ffb4802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:37.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.645+0000 7ffb4effd640 1 -- 172.21.15.16:0/4016405912 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ffb5405a040 con 0x7ffb5806d960 2024-09-19T07:54:37.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.821+0000 7ffb6073a640 1 -- 172.21.15.16:0/4016405912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7ffb580cfb70 con 0x7ffb5806d960 2024-09-19T07:54:37.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.821+0000 7ffb4effd640 1 -- 172.21.15.16:0/4016405912 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v57) ==== 112+0+10 (secure 0 0 0) 0x7ffb54059940 con 0x7ffb5806d960 2024-09-19T07:54:37.822 INFO:teuthology.orchestra.run.smithi016.stdout:60.000000 2024-09-19T07:54:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.823+0000 7ffb6073a640 1 -- 172.21.15.16:0/4016405912 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffb3c06c2b0 msgr2=0x7ffb3c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.823+0000 7ffb6073a640 1 --2- 172.21.15.16:0/4016405912 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffb3c06c2b0 0x7ffb3c06e770 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ffb48034000 tx=0x7ffb4802e000 comp rx=0 tx=0).stop 2024-09-19T07:54:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.823+0000 7ffb6073a640 1 -- 172.21.15.16:0/4016405912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5806d960 msgr2=0x7ffb580b9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.823+0000 7ffb6073a640 1 --2- 172.21.15.16:0/4016405912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5806d960 0x7ffb580b9f60 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400c7f0 tx=0x7ffb5400ccb0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.823+0000 7ffb6073a640 1 -- 172.21.15.16:0/4016405912 shutdown_connections 2024-09-19T07:54:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.823+0000 7ffb6073a640 1 --2- 172.21.15.16:0/4016405912 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffb3c06c2b0 0x7ffb3c06e770 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.823+0000 7ffb6073a640 1 --2- 172.21.15.16:0/4016405912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb5815e680 0x7ffb580ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.823+0000 7ffb6073a640 1 --2- 172.21.15.16:0/4016405912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb5815e2a0 0x7ffb580b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.823+0000 7ffb6073a640 1 --2- 172.21.15.16:0/4016405912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5806d960 0x7ffb580b9f60 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.823+0000 7ffb6073a640 1 -- 172.21.15.16:0/4016405912 >> 172.21.15.16:0/4016405912 conn(0x7ffb5809d760 msgr2=0x7ffb580cd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.823+0000 7ffb6073a640 1 -- 172.21.15.16:0/4016405912 shutdown_connections 2024-09-19T07:54:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37.823+0000 7ffb6073a640 1 -- 172.21.15.16:0/4016405912 wait complete. 2024-09-19T07:54:37.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:54:37.834 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:54:37.834 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:54:37.834 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:54:37.834 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:54:37.835 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:54:37.835 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:54:37.835 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:54:37.835 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:54:37.835 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:54:37.835 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:54:37.835 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:54:37.835 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:54:37.835 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:54:37.835 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:54:37.835 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:54:37.835 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:54:37.835 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:54:37.835 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:54:37.835 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:54:37.836 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:54:37.836 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:54:37.836 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:54:37.836 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:54:37.836 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:54:37.836 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:54:37.836 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:54:37.917 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.72929.asok 2024-09-19T07:54:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:54:37.926 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.72929.asok 2024-09-19T07:54:37.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:37.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.72929.asok mds_sessions 2024-09-19T07:54:38.061 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:54:38.062 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 8007, 2024-09-19T07:54:38.062 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:54:38.062 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:54:38.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:54:38.062 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 8007 2024-09-19T07:54:38.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:38.062 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:54:38.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:54:38.062 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:54:38.062 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1934386859 2024-09-19T07:54:38.062 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:38.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:38.063 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.8007 v1:192.168.0.1:0/1934386859", 2024-09-19T07:54:38.063 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "v1:192.168.0.1:0/1934386859", 2024-09-19T07:54:38.063 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions": [ 2024-09-19T07:54:38.063 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:38.063 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": 0, 2024-09-19T07:54:38.063 INFO:teuthology.orchestra.run.smithi016.stdout: "addrs": { 2024-09-19T07:54:38.063 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-19T07:54:38.063 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:38.063 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-19T07:54:38.063 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.146:6835", 2024-09-19T07:54:38.064 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 79304323 2024-09-19T07:54:38.064 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:38.064 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:38.064 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:54:38.064 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.146:6837", 2024-09-19T07:54:38.064 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 79304323 2024-09-19T07:54:38.064 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:38.064 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T07:54:38.064 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:38.064 INFO:teuthology.orchestra.run.smithi016.stdout: "seq": 0, 2024-09-19T07:54:38.064 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_gen": 0, 2024-09-19T07:54:38.064 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_ttl": "2024-09-19T07:55:34.744370+0000", 2024-09-19T07:54:38.064 INFO:teuthology.orchestra.run.smithi016.stdout: "last_cap_renew_request": "2024-09-19T07:54:34.744370+0000", 2024-09-19T07:54:38.064 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_renew_seq": 2, 2024-09-19T07:54:38.065 INFO:teuthology.orchestra.run.smithi016.stdout: "num_caps": 1, 2024-09-19T07:54:38.065 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "open" 2024-09-19T07:54:38.065 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:38.065 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:54:38.065 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 72 2024-09-19T07:54:38.065 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:54:38.070 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:54:38.071 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:54:38.087 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-19T07:54:38.087 DEBUG:teuthology.orchestra.run.smithi016:> import fcntl 2024-09-19T07:54:38.087 DEBUG:teuthology.orchestra.run.smithi016:> import sys 2024-09-19T07:54:38.088 DEBUG:teuthology.orchestra.run.smithi016:> import time 2024-09-19T07:54:38.088 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:54:38.088 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:54:38.088 DEBUG:teuthology.orchestra.run.smithi016:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-19T07:54:38.088 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:54:38.088 DEBUG:teuthology.orchestra.run.smithi016:> n = 200 2024-09-19T07:54:38.088 DEBUG:teuthology.orchestra.run.smithi016:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2024-09-19T07:54:38.088 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:54:38.088 DEBUG:teuthology.orchestra.run.smithi016:> if not os.path.exists(abs_path): 2024-09-19T07:54:38.088 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(abs_path) 2024-09-19T07:54:38.088 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:54:38.088 DEBUG:teuthology.orchestra.run.smithi016:> handles = [] 2024-09-19T07:54:38.088 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, n): 2024-09-19T07:54:38.088 DEBUG:teuthology.orchestra.run.smithi016:> fname = "file_"+str(i) 2024-09-19T07:54:38.088 DEBUG:teuthology.orchestra.run.smithi016:> path = os.path.join(abs_path, fname) 2024-09-19T07:54:38.089 DEBUG:teuthology.orchestra.run.smithi016:> handles.append(open(path, '"'"'w'"'"')) 2024-09-19T07:54:38.089 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:54:38.089 DEBUG:teuthology.orchestra.run.smithi016:> print("waiting with handles open", file=sys.stderr) 2024-09-19T07:54:38.089 DEBUG:teuthology.orchestra.run.smithi016:> while True: 2024-09-19T07:54:38.089 DEBUG:teuthology.orchestra.run.smithi016:> print("open_n_background: keeping files open", file=sys.stderr) 2024-09-19T07:54:38.089 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-19T07:54:38.089 DEBUG:teuthology.orchestra.run.smithi016:> if os.read(0, 4096) == b"": 2024-09-19T07:54:38.089 DEBUG:teuthology.orchestra.run.smithi016:> break 2024-09-19T07:54:38.089 DEBUG:teuthology.orchestra.run.smithi016:> except BlockingIOError: 2024-09-19T07:54:38.089 DEBUG:teuthology.orchestra.run.smithi016:> pass 2024-09-19T07:54:38.089 DEBUG:teuthology.orchestra.run.smithi016:> time.sleep(2) 2024-09-19T07:54:38.089 DEBUG:teuthology.orchestra.run.smithi016:> print("stdin closed, goodbye!", file=sys.stderr) 2024-09-19T07:54:38.089 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:54:38.129 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:54:38.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.234+0000 7fabda442640 1 Processor -- start 2024-09-19T07:54:38.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.234+0000 7fabda442640 1 -- start start 2024-09-19T07:54:38.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.236+0000 7fabda442640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fabd406c320 0x7fabd40b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:38.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.236+0000 7fabda442640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabd406ca00 con 0x7fabd40b0460 2024-09-19T07:54:38.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.236+0000 7fabda442640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabd40b0960 con 0x7fabd4145a20 2024-09-19T07:54:38.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.236+0000 7fabda442640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fabd40b1570 con 0x7fabd406c320 2024-09-19T07:54:38.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.236+0000 7fabd37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fabd406c320 0x7fabd40b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:38.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.236+0000 7fabd89df640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fabd40b0460 0x7fabd40b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35602/0 (socket says 172.21.15.16:35602) 2024-09-19T07:54:38.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.236+0000 7fabd89df640 1 -- 172.21.15.16:0/764009310 learned_addr learned my addr 172.21.15.16:0/764009310 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:38.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.236+0000 7fabd37fe640 1 -- 172.21.15.16:0/764009310 >> v1:172.21.15.146:6789/0 conn(0x7fabd4145a20 legacy=0x7fabd406bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:38.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.236+0000 7fabd37fe640 1 -- 172.21.15.16:0/764009310 >> v1:172.21.15.16:6789/0 conn(0x7fabd40b0460 legacy=0x7fabd40b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:38.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.236+0000 7fabd2ffd640 1 -- 172.21.15.16:0/764009310 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3952494032 0 0) 0x7fabd406ca00 con 0x7fabd40b0460 2024-09-19T07:54:38.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.236+0000 7fabd37fe640 1 -- 172.21.15.16:0/764009310 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd40b0c40 con 0x7fabd406c320 2024-09-19T07:54:38.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.237+0000 7fabd37fe640 1 --2- 172.21.15.16:0/764009310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabd406c320 0x7fabd40b6cb0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fabc80097a0 tx=0x7fabc802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f7b949bf6ecfb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:38.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.237+0000 7fabd2ffd640 1 -- 172.21.15.16:0/764009310 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc8037070 con 0x7fabd406c320 2024-09-19T07:54:38.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.237+0000 7fabd2ffd640 1 -- 172.21.15.16:0/764009310 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabc8032440 con 0x7fabd406c320 2024-09-19T07:54:38.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.237+0000 7fabd2ffd640 1 -- 172.21.15.16:0/764009310 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc8032810 con 0x7fabd406c320 2024-09-19T07:54:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.238+0000 7fabda442640 1 -- 172.21.15.16:0/764009310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabd406c320 msgr2=0x7fabd40b6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.238+0000 7fabda442640 1 --2- 172.21.15.16:0/764009310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabd406c320 0x7fabd40b6cb0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fabc80097a0 tx=0x7fabc802a230 comp rx=0 tx=0).stop 2024-09-19T07:54:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.238+0000 7fabda442640 1 -- 172.21.15.16:0/764009310 shutdown_connections 2024-09-19T07:54:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.238+0000 7fabda442640 1 --2- 172.21.15.16:0/764009310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabd406c320 0x7fabd40b6cb0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.238+0000 7fabda442640 1 -- 172.21.15.16:0/764009310 >> 172.21.15.16:0/764009310 conn(0x7fabd409a9f0 msgr2=0x7fabd40b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.238+0000 7fabda442640 1 -- 172.21.15.16:0/764009310 shutdown_connections 2024-09-19T07:54:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.238+0000 7fabda442640 1 -- 172.21.15.16:0/764009310 wait complete. 2024-09-19T07:54:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.239+0000 7fabda442640 1 Processor -- start 2024-09-19T07:54:38.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.239+0000 7fabda442640 1 -- start start 2024-09-19T07:54:38.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.240+0000 7fabda442640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabd406c320 0x7fabd41e1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.240+0000 7fabda442640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fabd40b0460 0x7fabd41e5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.240+0000 7fabda442640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabd4145a20 0x7fabd40cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.240+0000 7fabda442640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fabd4145e00 con 0x7fabd406c320 2024-09-19T07:54:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.240+0000 7fabda442640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fabd40b1200 con 0x7fabd4145a20 2024-09-19T07:54:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.240+0000 7fabda442640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fabd4145f80 con 0x7fabd40b0460 2024-09-19T07:54:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.240+0000 7fabd3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabd406c320 0x7fabd41e1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.240+0000 7fabd3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabd406c320 0x7fabd41e1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60290/0 (socket says 172.21.15.16:60290) 2024-09-19T07:54:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.240+0000 7fabd3fff640 1 -- 172.21.15.16:0/186723067 learned_addr learned my addr 172.21.15.16:0/186723067 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.240+0000 7fabd37fe640 1 --2- 172.21.15.16:0/186723067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fabd40b0460 0x7fabd41e5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.240+0000 7fabd89df640 1 --2- 172.21.15.16:0/186723067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabd4145a20 0x7fabd40cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.241+0000 7fabd3fff640 1 -- 172.21.15.16:0/186723067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fabd40b0460 msgr2=0x7fabd41e5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.241+0000 7fabd3fff640 1 --2- 172.21.15.16:0/186723067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fabd40b0460 0x7fabd41e5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.241+0000 7fabd3fff640 1 -- 172.21.15.16:0/186723067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabd4145a20 msgr2=0x7fabd40cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:38.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.241+0000 7fabd3fff640 1 --2- 172.21.15.16:0/186723067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabd4145a20 0x7fabd40cfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:38.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.241+0000 7fabd3fff640 1 -- 172.21.15.16:0/186723067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd41e7ff0 con 0x7fabd406c320 2024-09-19T07:54:38.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.241+0000 7fabd3fff640 1 --2- 172.21.15.16:0/186723067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabd406c320 0x7fabd41e1820 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fabc400c450 tx=0x7fabc400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:38.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.241+0000 7fabd0ff9640 1 -- 172.21.15.16:0/186723067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc40052d0 con 0x7fabd406c320 2024-09-19T07:54:38.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.241+0000 7fabd0ff9640 1 -- 172.21.15.16:0/186723067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabc4005470 con 0x7fabd406c320 2024-09-19T07:54:38.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.241+0000 7fabda442640 1 -- 172.21.15.16:0/186723067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd41e82d0 con 0x7fabd406c320 2024-09-19T07:54:38.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.241+0000 7fabd0ff9640 1 -- 172.21.15.16:0/186723067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4012870 con 0x7fabd406c320 2024-09-19T07:54:38.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.241+0000 7fabda442640 1 -- 172.21.15.16:0/186723067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd41e5890 con 0x7fabd406c320 2024-09-19T07:54:38.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.242+0000 7fabda442640 1 -- 172.21.15.16:0/186723067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabd41e84a0 con 0x7fabd406c320 2024-09-19T07:54:38.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.243+0000 7fabd0ff9640 1 -- 172.21.15.16:0/186723067 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fabc4012a10 con 0x7fabd406c320 2024-09-19T07:54:38.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.243+0000 7fabd0ff9640 1 --2- 172.21.15.16:0/186723067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fabc006c2b0 0x7fabc006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:38.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.243+0000 7fabd0ff9640 1 -- 172.21.15.16:0/186723067 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7fabc408afe0 con 0x7fabd406c320 2024-09-19T07:54:38.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.243+0000 7fabd37fe640 1 --2- 172.21.15.16:0/186723067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fabc006c2b0 0x7fabc006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:38.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.244+0000 7fabd37fe640 1 --2- 172.21.15.16:0/186723067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fabc006c2b0 0x7fabc006e770 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fabc8034000 tx=0x7fabc802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:38.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.248+0000 7fabd0ff9640 1 -- 172.21.15.16:0/186723067 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fabc4059000 con 0x7fabd406c320 2024-09-19T07:54:38.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.469+0000 7fabda442640 1 -- 172.21.15.16:0/186723067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabd41e86c0 con 0x7fabd406c320 2024-09-19T07:54:38.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.470+0000 7fabd0ff9640 1 -- 172.21.15.16:0/186723067 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5219 (secure 0 0 0) 0x7fabc405d300 con 0x7fabd406c320 2024-09-19T07:54:38.470 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:38.470 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":72,"btime":"2024-09-19T07:54:25:574262+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/2994096322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2994096322},{"type":"v1","addr":"172.21.15.146:6836","nonce":2994096322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7839,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/393564462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":393564462},{"type":"v1","addr":"172.21.15.16:6833","nonce":393564462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":7842,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/631392582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":631392582},{"type":"v1","addr":"172.21.15.16:6835","nonce":631392582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:54:22.949504+0000","modified":"2024-09-19T07:54:24.912891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6837/79304323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":79304323},{"type":"v1","addr":"172.21.15.146:6837","nonce":79304323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7854,"qdb_cluster":[7854]},"id":9}]} 2024-09-19T07:54:38.471 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 72 2024-09-19T07:54:38.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.472+0000 7fabda442640 1 -- 172.21.15.16:0/186723067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fabc006c2b0 msgr2=0x7fabc006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:38.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.472+0000 7fabda442640 1 --2- 172.21.15.16:0/186723067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fabc006c2b0 0x7fabc006e770 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fabc8034000 tx=0x7fabc802e000 comp rx=0 tx=0).stop 2024-09-19T07:54:38.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.472+0000 7fabda442640 1 -- 172.21.15.16:0/186723067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabd406c320 msgr2=0x7fabd41e1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:38.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.472+0000 7fabda442640 1 --2- 172.21.15.16:0/186723067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabd406c320 0x7fabd41e1820 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fabc400c450 tx=0x7fabc400c910 comp rx=0 tx=0).stop 2024-09-19T07:54:38.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.472+0000 7fabda442640 1 -- 172.21.15.16:0/186723067 shutdown_connections 2024-09-19T07:54:38.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.472+0000 7fabda442640 1 --2- 172.21.15.16:0/186723067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fabc006c2b0 0x7fabc006e770 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:38.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.472+0000 7fabda442640 1 --2- 172.21.15.16:0/186723067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabd4145a20 0x7fabd40cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:38.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.472+0000 7fabda442640 1 --2- 172.21.15.16:0/186723067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fabd40b0460 0x7fabd41e5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:38.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.472+0000 7fabda442640 1 --2- 172.21.15.16:0/186723067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabd406c320 0x7fabd41e1820 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:38.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.472+0000 7fabda442640 1 -- 172.21.15.16:0/186723067 >> 172.21.15.16:0/186723067 conn(0x7fabd409a9f0 msgr2=0x7fabd41442f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:38.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.472+0000 7fabda442640 1 -- 172.21.15.16:0/186723067 shutdown_connections 2024-09-19T07:54:38.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.472+0000 7fabda442640 1 -- 172.21.15.16:0/186723067 wait complete. 2024-09-19T07:54:38.483 INFO:teuthology.orchestra.run.smithi016.stderr:waiting with handles open 2024-09-19T07:54:38.484 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:54:38.485 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-19T07:54:38.612 INFO:teuthology.orchestra.run.smithi146.stdout:[{"id":6635,"entity":{"name":{"type":"client","num":6635},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2776231670}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":3.3119502000000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9905017325798471,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6635 192.168.0.1:0/2776231670","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"49038","root":"/"}},{"id":8007,"entity":{"name":{"type":"client","num":8007},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1934386859}},"state":"open","num_leases":1,"num_caps":202,"request_load_avg":204,"uptime":4.8589269399999999,"requests_in_flight":0,"num_completed_requests":15,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":99.963039538105335,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":98.532187200207261,"halflife":1.5},"recall_caps_throttle2o":{"value":95.660879596566531,"halflife":0.5},"session_cache_liveness":{"value":405.74306526495559,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":190,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8007 192.168.0.1:0/1934386859","completed_requests":[{"tid":190,"created_ino":"0x100000002ae"},{"tid":191,"created_ino":"0x100000002af"},{"tid":192,"created_ino":"0x100000002b0"},{"tid":193,"created_ino":"0x100000002b1"},{"tid":194,"created_ino":"0x100000002b2"},{"tid":195,"created_ino":"0x100000002b3"},{"tid":196,"created_ino":"0x100000002b4"},{"tid":197,"created_ino":"0x100000002b5"},{"tid":198,"created_ino":"0x100000002b6"},{"tid":199,"created_ino":"0x100000002b7"},{"tid":200,"created_ino":"0x100000002b8"},{"tid":201,"created_ino":"0x100000002b9"},{"tid":202,"created_ino":"0x100000002ba"},{"tid":203,"created_ino":"0x100000002bb"},{"tid":204,"created_ino":"0x100000002bc"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x100000002bd","length":802}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"0","hostname":"smithi016","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"72929","root":"/"}}] 2024-09-19T07:54:38.877 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "49038", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2776231670, "type": "any" }, "name": { "num": 6635, "type": "client" } }, "id": 6635, "inst": "client.6635 192.168.0.1:0/2776231670", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.990501732579847 }, "state": "open", "uptime": 3.3119502 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "0", "hostname": "smithi016", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "72929", "root": "/" }, "completed_requests": [ { "created_ino": "0x100000002ae", "tid": 190 }, { "created_ino": "0x100000002af", "tid": 191 }, { "created_ino": "0x100000002b0", "tid": 192 }, { "created_ino": "0x100000002b1", "tid": 193 }, { "created_ino": "0x100000002b2", "tid": 194 }, { "created_ino": "0x100000002b3", "tid": 195 }, { "created_ino": "0x100000002b4", "tid": 196 }, { "created_ino": "0x100000002b5", "tid": 197 }, { "created_ino": "0x100000002b6", "tid": 198 }, { "created_ino": "0x100000002b7", "tid": 199 }, { "created_ino": "0x100000002b8", "tid": 200 }, { "created_ino": "0x100000002b9", "tid": 201 }, { "created_ino": "0x100000002ba", "tid": 202 }, { "created_ino": "0x100000002bb", "tid": 203 }, { "created_ino": "0x100000002bc", "tid": 204 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1934386859, "type": "any" }, "name": { "num": 8007, "type": "client" } }, "id": 8007, "inst": "client.8007 192.168.0.1:0/1934386859", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 190, "num_caps": 202, "num_completed_flushes": 0, "num_completed_requests": 15, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 802, "start": "0x100000002bd" } ], "recall_caps": { "halflife": 60, "value": 99.96303953810533 }, "recall_caps_throttle": { "halflife": 1.5, "value": 98.53218720020726 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 95.66087959656653 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 204, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 405.7430652649556 }, "state": "open", "uptime": 4.85892694 } ] 2024-09-19T07:54:38.877 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-09-19T07:54:38.877 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:54:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.980+0000 7f209f650640 1 Processor -- start 2024-09-19T07:54:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.981+0000 7f209f650640 1 -- start start 2024-09-19T07:54:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.981+0000 7f209f650640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f209806d430 0x7f20981439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.981+0000 7f209f650640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2098143f30 0x7f20980d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:38.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.981+0000 7f209f650640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20980c7140 con 0x7f20980c8d20 2024-09-19T07:54:38.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.981+0000 7f209f650640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2098141960 con 0x7f2098143f30 2024-09-19T07:54:38.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.981+0000 7f209f650640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2098142200 con 0x7f209806d430 2024-09-19T07:54:38.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f209d3ec640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f20980c8d20 0x7f209806cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35610/0 (socket says 172.21.15.16:35610) 2024-09-19T07:54:38.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f209d3ec640 1 -- 172.21.15.16:0/85018213 learned_addr learned my addr 172.21.15.16:0/85018213 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:38.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f209dbed640 1 --2- 172.21.15.16:0/85018213 >> v2:172.21.15.146:3300/0 conn(0x7f2098143f30 0x7f20980d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:38.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f209cbeb640 1 --2- 172.21.15.16:0/85018213 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f209806d430 0x7f20981439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:38.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f208bfff640 1 -- 172.21.15.16:0/85018213 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1784766031 0 0) 0x7f2094001110 con 0x7f20980c8d20 2024-09-19T07:54:38.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f208bfff640 1 -- 172.21.15.16:0/85018213 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2084003520 con 0x7f20980c8d20 2024-09-19T07:54:38.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f208bfff640 1 -- 172.21.15.16:0/85018213 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4208685184 0 0) 0x7f2084003520 con 0x7f20980c8d20 2024-09-19T07:54:38.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f208bfff640 1 -- 172.21.15.16:0/85018213 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f209806d430 msgr2=0x7f20981439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:38.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f208bfff640 1 --2- 172.21.15.16:0/85018213 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f209806d430 0x7f20981439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:38.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f208bfff640 1 -- 172.21.15.16:0/85018213 >> v2:172.21.15.146:3300/0 conn(0x7f2098143f30 msgr2=0x7f20980d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f208bfff640 1 --2- 172.21.15.16:0/85018213 >> v2:172.21.15.146:3300/0 conn(0x7f2098143f30 0x7f20980d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f208bfff640 1 -- 172.21.15.16:0/85018213 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20980c9100 con 0x7f20980c8d20 2024-09-19T07:54:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f208bfff640 1 -- 172.21.15.16:0/85018213 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f2094001840 con 0x7f20980c8d20 2024-09-19T07:54:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f208bfff640 1 -- 172.21.15.16:0/85018213 >> v1:172.21.15.16:6789/0 conn(0x7f20980c8d20 legacy=0x7f209806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f208bfff640 1 --2- 172.21.15.16:0/85018213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20840039b0 0x7f2084003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f208bfff640 1 --2- 172.21.15.16:0/85018213 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f209806d430 0x7f2084004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f208bfff640 1 --2- 172.21.15.16:0/85018213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2084004d40 0x7f20840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f208bfff640 1 -- 172.21.15.16:0/85018213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2098141960 con 0x7f20840039b0 2024-09-19T07:54:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f208bfff640 1 -- 172.21.15.16:0/85018213 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2098142200 con 0x7f2084004d40 2024-09-19T07:54:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f208bfff640 1 -- 172.21.15.16:0/85018213 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f208400ba70 con 0x7f209806d430 2024-09-19T07:54:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f209d3ec640 1 --2- 172.21.15.16:0/85018213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20840039b0 0x7f2084003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f209cbeb640 1 --2- 172.21.15.16:0/85018213 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f209806d430 0x7f2084004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.982+0000 7f209dbed640 1 --2- 172.21.15.16:0/85018213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2084004d40 0x7f20840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.983+0000 7f209d3ec640 1 -- 172.21.15.16:0/85018213 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f209806d430 msgr2=0x7f2084004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.983+0000 7f209d3ec640 1 --2- 172.21.15.16:0/85018213 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f209806d430 0x7f2084004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.983+0000 7f209d3ec640 1 -- 172.21.15.16:0/85018213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2084004d40 msgr2=0x7f20840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.983+0000 7f209d3ec640 1 --2- 172.21.15.16:0/85018213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2084004d40 0x7f20840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.983+0000 7f209d3ec640 1 -- 172.21.15.16:0/85018213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2084003520 con 0x7f20840039b0 2024-09-19T07:54:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.983+0000 7f209d3ec640 1 --2- 172.21.15.16:0/85018213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20840039b0 0x7f2084003dd0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f209400c3b0 tx=0x7f2084005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791e202f6d2f60b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.983+0000 7f208bfff640 1 -- 172.21.15.16:0/85018213 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209400c0e0 con 0x7f20840039b0 2024-09-19T07:54:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.983+0000 7f208bfff640 1 -- 172.21.15.16:0/85018213 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2094034070 con 0x7f20840039b0 2024-09-19T07:54:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.983+0000 7f208bfff640 1 -- 172.21.15.16:0/85018213 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094005d10 con 0x7f20840039b0 2024-09-19T07:54:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.984+0000 7f209f650640 1 -- 172.21.15.16:0/85018213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20840039b0 msgr2=0x7f2084003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.984+0000 7f209f650640 1 --2- 172.21.15.16:0/85018213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20840039b0 0x7f2084003dd0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f209400c3b0 tx=0x7f2084005830 comp rx=0 tx=0).stop 2024-09-19T07:54:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.984+0000 7f209dbed640 1 -- 172.21.15.16:0/85018213 reap_dead start 2024-09-19T07:54:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.984+0000 7f209f650640 1 -- 172.21.15.16:0/85018213 shutdown_connections 2024-09-19T07:54:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.984+0000 7f209f650640 1 -- 172.21.15.16:0/85018213 >> 172.21.15.16:0/85018213 conn(0x7f20980b3ed0 msgr2=0x7f20980b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.984+0000 7f209f650640 1 -- 172.21.15.16:0/85018213 shutdown_connections 2024-09-19T07:54:38.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.984+0000 7f209f650640 1 -- 172.21.15.16:0/85018213 wait complete. 2024-09-19T07:54:38.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.985+0000 7f209f650640 1 Processor -- start 2024-09-19T07:54:38.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.985+0000 7f209f650640 1 -- start start 2024-09-19T07:54:38.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.986+0000 7f209f650640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20840039b0 0x7f20980b7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:38.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.986+0000 7f209f650640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20980cf1d0 0x7f20980cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:38.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.986+0000 7f209f650640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f20980cf650 0x7f20980cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:38.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.986+0000 7f209f650640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2098141e90 con 0x7f20980cf1d0 2024-09-19T07:54:38.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.986+0000 7f209f650640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2098141bb0 con 0x7f20840039b0 2024-09-19T07:54:38.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.986+0000 7f209f650640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2098144380 con 0x7f20980cf650 2024-09-19T07:54:38.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.986+0000 7f209cbeb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20980cf1d0 0x7f20980cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:38.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.986+0000 7f209cbeb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20980cf1d0 0x7f20980cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60308/0 (socket says 172.21.15.16:60308) 2024-09-19T07:54:38.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.986+0000 7f209cbeb640 1 -- 172.21.15.16:0/23596659 learned_addr learned my addr 172.21.15.16:0/23596659 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:38.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.986+0000 7f209d3ec640 1 --2- 172.21.15.16:0/23596659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20840039b0 0x7f20980b7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:38.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.987+0000 7f209cbeb640 1 -- 172.21.15.16:0/23596659 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f20980cf650 msgr2=0x7f20980cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:38.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.987+0000 7f209cbeb640 1 --2- 172.21.15.16:0/23596659 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f20980cf650 0x7f20980cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:38.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.987+0000 7f209cbeb640 1 -- 172.21.15.16:0/23596659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20840039b0 msgr2=0x7f20980b7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:38.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.987+0000 7f209cbeb640 1 --2- 172.21.15.16:0/23596659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20840039b0 0x7f20980b7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.987+0000 7f209cbeb640 1 -- 172.21.15.16:0/23596659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20980cc810 con 0x7f20980cf1d0 2024-09-19T07:54:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.987+0000 7f209cbeb640 1 --2- 172.21.15.16:0/23596659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20980cf1d0 0x7f20980cb500 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f208c0118e0 tx=0x7f208c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.987+0000 7f2089ffb640 1 -- 172.21.15.16:0/23596659 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c00fa70 con 0x7f20980cf1d0 2024-09-19T07:54:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.987+0000 7f2089ffb640 1 -- 172.21.15.16:0/23596659 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f208c00fc10 con 0x7f20980cf1d0 2024-09-19T07:54:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.987+0000 7f2089ffb640 1 -- 172.21.15.16:0/23596659 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c014440 con 0x7f20980cf1d0 2024-09-19T07:54:38.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.987+0000 7f209f650640 1 -- 172.21.15.16:0/23596659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20980ccaf0 con 0x7f20980cf1d0 2024-09-19T07:54:38.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.987+0000 7f209f650640 1 -- 172.21.15.16:0/23596659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20980cd0b0 con 0x7f20980cf1d0 2024-09-19T07:54:38.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.988+0000 7f209f650640 1 -- 172.21.15.16:0/23596659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20981439f0 con 0x7f20980cf1d0 2024-09-19T07:54:38.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.989+0000 7f2089ffb640 1 -- 172.21.15.16:0/23596659 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f208c012040 con 0x7f20980cf1d0 2024-09-19T07:54:38.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.989+0000 7f2089ffb640 1 --2- 172.21.15.16:0/23596659 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f207c06c2b0 0x7f207c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:38.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.989+0000 7f2089ffb640 1 -- 172.21.15.16:0/23596659 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7f208c08bbf0 con 0x7f20980cf1d0 2024-09-19T07:54:38.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.989+0000 7f209d3ec640 1 --2- 172.21.15.16:0/23596659 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f207c06c2b0 0x7f207c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:38.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.990+0000 7f209d3ec640 1 --2- 172.21.15.16:0/23596659 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f207c06c2b0 0x7f207c06e770 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2094033000 tx=0x7f209403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:38.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:38.993+0000 7f2089ffb640 1 -- 172.21.15.16:0/23596659 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f208c019030 con 0x7f20980cf1d0 2024-09-19T07:54:39.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:39.206+0000 7f209f650640 1 -- 172.21.15.16:0/23596659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2098072290 con 0x7f20980cf1d0 2024-09-19T07:54:39.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:39.207+0000 7f2089ffb640 1 -- 172.21.15.16:0/23596659 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f208c05a1d0 con 0x7f20980cf1d0 2024-09-19T07:54:39.207 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:39.207 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T07:54:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:39.209+0000 7f209f650640 1 -- 172.21.15.16:0/23596659 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f207c06c2b0 msgr2=0x7f207c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:39.209+0000 7f209f650640 1 --2- 172.21.15.16:0/23596659 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f207c06c2b0 0x7f207c06e770 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2094033000 tx=0x7f209403d040 comp rx=0 tx=0).stop 2024-09-19T07:54:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:39.209+0000 7f209f650640 1 -- 172.21.15.16:0/23596659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20980cf1d0 msgr2=0x7f20980cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:39.209+0000 7f209f650640 1 --2- 172.21.15.16:0/23596659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20980cf1d0 0x7f20980cb500 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f208c0118e0 tx=0x7f208c011da0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:39.209+0000 7f209f650640 1 -- 172.21.15.16:0/23596659 shutdown_connections 2024-09-19T07:54:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:39.209+0000 7f209f650640 1 --2- 172.21.15.16:0/23596659 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f207c06c2b0 0x7f207c06e770 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:39.209+0000 7f209f650640 1 --2- 172.21.15.16:0/23596659 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f20980cf650 0x7f20980cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:39.209+0000 7f209f650640 1 --2- 172.21.15.16:0/23596659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20980cf1d0 0x7f20980cb500 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:39.209+0000 7f209f650640 1 --2- 172.21.15.16:0/23596659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20840039b0 0x7f20980b7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:39.209+0000 7f209f650640 1 -- 172.21.15.16:0/23596659 >> 172.21.15.16:0/23596659 conn(0x7f20980b3ed0 msgr2=0x7f20980b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:39.209+0000 7f209f650640 1 -- 172.21.15.16:0/23596659 shutdown_connections 2024-09-19T07:54:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:39.209+0000 7f209f650640 1 -- 172.21.15.16:0/23596659 wait complete. 2024-09-19T07:54:39.222 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-19T07:54:39.222 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-19T07:54:39.280 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:54:39.283 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:54:40.486 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:54:42.488 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:54:44.223 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:54:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.340+0000 7fb07e313640 1 Processor -- start 2024-09-19T07:54:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.340+0000 7fb07e313640 1 -- start start 2024-09-19T07:54:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.341+0000 7fb07e313640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb07815e680 0x7fb0780cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.341+0000 7fb07e313640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0780d1880 con 0x7fb07815e2a0 2024-09-19T07:54:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.341+0000 7fb07e313640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0780d1b60 con 0x7fb0780d0010 2024-09-19T07:54:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.341+0000 7fb07e313640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb0780cf2d0 con 0x7fb07815e680 2024-09-19T07:54:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.341+0000 7fb077fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb07815e2a0 0x7fb07806d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47802/0 (socket says 172.21.15.16:47802) 2024-09-19T07:54:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.341+0000 7fb077fff640 1 -- 172.21.15.16:0/2566523026 learned_addr learned my addr 172.21.15.16:0/2566523026 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:44.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.341+0000 7fb076ffd640 1 -- 172.21.15.16:0/2566523026 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2877062505 0 0) 0x7fb0780d1880 con 0x7fb07815e2a0 2024-09-19T07:54:44.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.341+0000 7fb076ffd640 1 -- 172.21.15.16:0/2566523026 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb060003520 con 0x7fb07815e2a0 2024-09-19T07:54:44.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb0777fe640 1 --2- 172.21.15.16:0/2566523026 >> v2:172.21.15.146:3300/0 conn(0x7fb07815e680 0x7fb0780cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb076ffd640 1 -- 172.21.15.16:0/2566523026 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4128114131 0 0) 0x7fb060003520 con 0x7fb07815e2a0 2024-09-19T07:54:44.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb076ffd640 1 -- 172.21.15.16:0/2566523026 >> v2:172.21.15.146:3300/0 conn(0x7fb07815e680 msgr2=0x7fb0780cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:44.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb076ffd640 1 --2- 172.21.15.16:0/2566523026 >> v2:172.21.15.146:3300/0 conn(0x7fb07815e680 0x7fb0780cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb076ffd640 1 -- 172.21.15.16:0/2566523026 >> v1:172.21.15.146:6789/0 conn(0x7fb0780d0010 legacy=0x7fb0780d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb076ffd640 1 -- 172.21.15.16:0/2566523026 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0780cf5b0 con 0x7fb07815e2a0 2024-09-19T07:54:44.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb076ffd640 1 -- 172.21.15.16:0/2566523026 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fb068001670 con 0x7fb07815e2a0 2024-09-19T07:54:44.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb076ffd640 1 -- 172.21.15.16:0/2566523026 >> v1:172.21.15.16:6789/0 conn(0x7fb07815e2a0 legacy=0x7fb07806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb076ffd640 1 --2- 172.21.15.16:0/2566523026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0600039b0 0x7fb060003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb076ffd640 1 --2- 172.21.15.16:0/2566523026 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb060004800 0x7fb060004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb076ffd640 1 --2- 172.21.15.16:0/2566523026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0600052e0 0x7fb06000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb076ffd640 1 -- 172.21.15.16:0/2566523026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0780cf2d0 con 0x7fb0600052e0 2024-09-19T07:54:44.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb076ffd640 1 -- 172.21.15.16:0/2566523026 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb06000c1b0 con 0x7fb0600039b0 2024-09-19T07:54:44.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb076ffd640 1 -- 172.21.15.16:0/2566523026 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb06000c3b0 con 0x7fb060004800 2024-09-19T07:54:44.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb07c8b0640 1 --2- 172.21.15.16:0/2566523026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0600052e0 0x7fb06000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb077fff640 1 --2- 172.21.15.16:0/2566523026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0600039b0 0x7fb060003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb0777fe640 1 --2- 172.21.15.16:0/2566523026 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb060004800 0x7fb060004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb07c8b0640 1 -- 172.21.15.16:0/2566523026 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb060004800 msgr2=0x7fb060004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb07c8b0640 1 --2- 172.21.15.16:0/2566523026 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb060004800 0x7fb060004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb07c8b0640 1 -- 172.21.15.16:0/2566523026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0600039b0 msgr2=0x7fb060003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb07c8b0640 1 --2- 172.21.15.16:0/2566523026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0600039b0 0x7fb060003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.342+0000 7fb07c8b0640 1 -- 172.21.15.16:0/2566523026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb060003520 con 0x7fb0600052e0 2024-09-19T07:54:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.343+0000 7fb07c8b0640 1 -- 172.21.15.16:0/2566523026 reap_dead start 2024-09-19T07:54:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.343+0000 7fb07c8b0640 1 --2- 172.21.15.16:0/2566523026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0600052e0 0x7fb06000b9c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fb070001e20 tx=0x7fb060005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead6a667b9385d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.343+0000 7fb076ffd640 1 -- 172.21.15.16:0/2566523026 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070001a70 con 0x7fb0600052e0 2024-09-19T07:54:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.343+0000 7fb076ffd640 1 -- 172.21.15.16:0/2566523026 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb07002ae40 con 0x7fb0600052e0 2024-09-19T07:54:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.343+0000 7fb076ffd640 1 -- 172.21.15.16:0/2566523026 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070003bb0 con 0x7fb0600052e0 2024-09-19T07:54:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.344+0000 7fb07e313640 1 -- 172.21.15.16:0/2566523026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0600052e0 msgr2=0x7fb06000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.344+0000 7fb07e313640 1 --2- 172.21.15.16:0/2566523026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0600052e0 0x7fb06000b9c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fb070001e20 tx=0x7fb060005830 comp rx=0 tx=0).stop 2024-09-19T07:54:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.344+0000 7fb07e313640 1 -- 172.21.15.16:0/2566523026 shutdown_connections 2024-09-19T07:54:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.344+0000 7fb07e313640 1 --2- 172.21.15.16:0/2566523026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0600052e0 0x7fb06000b9c0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.344+0000 7fb07e313640 1 -- 172.21.15.16:0/2566523026 >> 172.21.15.16:0/2566523026 conn(0x7fb07809d760 msgr2=0x7fb07808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.344+0000 7fb07e313640 1 -- 172.21.15.16:0/2566523026 shutdown_connections 2024-09-19T07:54:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.344+0000 7fb07e313640 1 -- 172.21.15.16:0/2566523026 wait complete. 2024-09-19T07:54:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.345+0000 7fb07e313640 1 Processor -- start 2024-09-19T07:54:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.345+0000 7fb07e313640 1 -- start start 2024-09-19T07:54:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.346+0000 7fb07e313640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0600052e0 0x7fb078156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.346+0000 7fb07e313640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0780b7210 0x7fb0780cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.346+0000 7fb07e313640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0780cca40 0x7fb0780ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.346+0000 7fb07e313640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0780cfc50 con 0x7fb0780cca40 2024-09-19T07:54:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.346+0000 7fb07e313640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb0780cfad0 con 0x7fb0780b7210 2024-09-19T07:54:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.346+0000 7fb07e313640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb0780cfdd0 con 0x7fb0600052e0 2024-09-19T07:54:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.346+0000 7fb07c8b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0780cca40 0x7fb0780ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.346+0000 7fb07c8b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0780cca40 0x7fb0780ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56596/0 (socket says 172.21.15.16:56596) 2024-09-19T07:54:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.346+0000 7fb07c8b0640 1 -- 172.21.15.16:0/962222376 learned_addr learned my addr 172.21.15.16:0/962222376 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.346+0000 7fb077fff640 1 --2- 172.21.15.16:0/962222376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0600052e0 0x7fb078156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.346+0000 7fb0777fe640 1 --2- 172.21.15.16:0/962222376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0780b7210 0x7fb0780cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.347+0000 7fb07c8b0640 1 -- 172.21.15.16:0/962222376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0600052e0 msgr2=0x7fb078156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.347+0000 7fb07c8b0640 1 --2- 172.21.15.16:0/962222376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0600052e0 0x7fb078156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.347+0000 7fb07c8b0640 1 -- 172.21.15.16:0/962222376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0780b7210 msgr2=0x7fb0780cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.347+0000 7fb07c8b0640 1 --2- 172.21.15.16:0/962222376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0780b7210 0x7fb0780cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.347+0000 7fb07c8b0640 1 -- 172.21.15.16:0/962222376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0780ba110 con 0x7fb0780cca40 2024-09-19T07:54:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.347+0000 7fb07c8b0640 1 --2- 172.21.15.16:0/962222376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0780cca40 0x7fb0780ccef0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fb070005040 tx=0x7fb0700014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.347+0000 7fb074ff9640 1 -- 172.21.15.16:0/962222376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070007db0 con 0x7fb0780cca40 2024-09-19T07:54:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.347+0000 7fb074ff9640 1 -- 172.21.15.16:0/962222376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb070041070 con 0x7fb0780cca40 2024-09-19T07:54:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.347+0000 7fb07e313640 1 -- 172.21.15.16:0/962222376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0781e8380 con 0x7fb0780cca40 2024-09-19T07:54:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.347+0000 7fb074ff9640 1 -- 172.21.15.16:0/962222376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070001a70 con 0x7fb0780cca40 2024-09-19T07:54:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.347+0000 7fb07e313640 1 -- 172.21.15.16:0/962222376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb078156990 con 0x7fb0780cca40 2024-09-19T07:54:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.348+0000 7fb07e313640 1 -- 172.21.15.16:0/962222376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb07806dbe0 con 0x7fb0780cca40 2024-09-19T07:54:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.349+0000 7fb074ff9640 1 -- 172.21.15.16:0/962222376 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb070032660 con 0x7fb0780cca40 2024-09-19T07:54:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.349+0000 7fb074ff9640 1 --2- 172.21.15.16:0/962222376 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb06406c2b0 0x7fb06406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.349+0000 7fb074ff9640 1 -- 172.21.15.16:0/962222376 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7fb0700ab240 con 0x7fb0780cca40 2024-09-19T07:54:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.349+0000 7fb077fff640 1 --2- 172.21.15.16:0/962222376 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb06406c2b0 0x7fb06406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.350+0000 7fb077fff640 1 --2- 172.21.15.16:0/962222376 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb06406c2b0 0x7fb06406e770 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb0680017e0 tx=0x7fb0680015d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:44.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.354+0000 7fb074ff9640 1 -- 172.21.15.16:0/962222376 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb070039070 con 0x7fb0780cca40 2024-09-19T07:54:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:54:44.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.566+0000 7fb07e313640 1 -- 172.21.15.16:0/962222376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb07806cf90 con 0x7fb0780cca40 2024-09-19T07:54:44.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.566+0000 7fb074ff9640 1 -- 172.21.15.16:0/962222376 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7fb0700798b0 con 0x7fb0780cca40 2024-09-19T07:54:44.566 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:44.566 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:54:44.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.568+0000 7fb07e313640 1 -- 172.21.15.16:0/962222376 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb06406c2b0 msgr2=0x7fb06406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.568+0000 7fb07e313640 1 --2- 172.21.15.16:0/962222376 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb06406c2b0 0x7fb06406e770 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb0680017e0 tx=0x7fb0680015d0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.568+0000 7fb07e313640 1 -- 172.21.15.16:0/962222376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0780cca40 msgr2=0x7fb0780ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.568+0000 7fb07e313640 1 --2- 172.21.15.16:0/962222376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0780cca40 0x7fb0780ccef0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fb070005040 tx=0x7fb0700014a0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.568+0000 7fb07e313640 1 -- 172.21.15.16:0/962222376 shutdown_connections 2024-09-19T07:54:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.568+0000 7fb07e313640 1 --2- 172.21.15.16:0/962222376 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb06406c2b0 0x7fb06406e770 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.568+0000 7fb07e313640 1 --2- 172.21.15.16:0/962222376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0780cca40 0x7fb0780ccef0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.568+0000 7fb07e313640 1 --2- 172.21.15.16:0/962222376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0780b7210 0x7fb0780cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.568+0000 7fb07e313640 1 --2- 172.21.15.16:0/962222376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0600052e0 0x7fb078156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.568+0000 7fb07e313640 1 -- 172.21.15.16:0/962222376 >> 172.21.15.16:0/962222376 conn(0x7fb07809d760 msgr2=0x7fb078074ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.568+0000 7fb07e313640 1 -- 172.21.15.16:0/962222376 shutdown_connections 2024-09-19T07:54:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.568+0000 7fb07e313640 1 -- 172.21.15.16:0/962222376 wait complete. 2024-09-19T07:54:44.581 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-19T07:54:44.581 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-09-19T07:54:44.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:54:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.689+0000 7f8dd05b7640 1 Processor -- start 2024-09-19T07:54:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.689+0000 7f8dd05b7640 1 -- start start 2024-09-19T07:54:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.689+0000 7f8dd05b7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8dc815e2a0 0x7f8dc806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.690+0000 7f8dd05b7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8dc806d960 0x7f8dc80d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.690+0000 7f8dd05b7640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dc80d1880 con 0x7f8dc815e680 2024-09-19T07:54:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.690+0000 7f8dd05b7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f8dc80d1b60 con 0x7f8dc806d960 2024-09-19T07:54:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.690+0000 7f8dd05b7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f8dc80cf2d0 con 0x7f8dc815e2a0 2024-09-19T07:54:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.690+0000 7f8dceb54640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8dc806d960 0x7f8dc80d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.690+0000 7f8dceb54640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8dc806d960 0x7f8dc80d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56602/0 (socket says 172.21.15.16:56602) 2024-09-19T07:54:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.690+0000 7f8dceb54640 1 -- 172.21.15.16:0/2497477703 learned_addr learned my addr 172.21.15.16:0/2497477703 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.690+0000 7f8dce353640 1 --2- 172.21.15.16:0/2497477703 >> v2:172.21.15.146:3300/0 conn(0x7f8dc815e2a0 0x7f8dc806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.690+0000 7f8dceb54640 1 -- 172.21.15.16:0/2497477703 >> v2:172.21.15.146:3300/0 conn(0x7f8dc815e2a0 msgr2=0x7f8dc806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.690+0000 7f8dceb54640 1 --2- 172.21.15.16:0/2497477703 >> v2:172.21.15.146:3300/0 conn(0x7f8dc815e2a0 0x7f8dc806d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.690+0000 7f8dceb54640 1 -- 172.21.15.16:0/2497477703 >> v1:172.21.15.146:6789/0 conn(0x7f8dc815e680 legacy=0x7f8dc80cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.690+0000 7f8dceb54640 1 -- 172.21.15.16:0/2497477703 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc80cf5b0 con 0x7f8dc806d960 2024-09-19T07:54:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.691+0000 7f8dceb54640 1 --2- 172.21.15.16:0/2497477703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc806d960 0x7f8dc80d4370 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80b6fa0 tx=0x7f8dc002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37591b2a98a955db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.691+0000 7f8dcd351640 1 -- 172.21.15.16:0/2497477703 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0036070 con 0x7f8dc806d960 2024-09-19T07:54:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.691+0000 7f8dcd351640 1 -- 172.21.15.16:0/2497477703 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc0031600 con 0x7f8dc806d960 2024-09-19T07:54:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.691+0000 7f8dcd351640 1 -- 172.21.15.16:0/2497477703 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc00319d0 con 0x7f8dc806d960 2024-09-19T07:54:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.691+0000 7f8dd05b7640 1 -- 172.21.15.16:0/2497477703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc806d960 msgr2=0x7f8dc80d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.691+0000 7f8dd05b7640 1 --2- 172.21.15.16:0/2497477703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc806d960 0x7f8dc80d4370 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80b6fa0 tx=0x7f8dc002a6f0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.691+0000 7f8dd05b7640 1 -- 172.21.15.16:0/2497477703 shutdown_connections 2024-09-19T07:54:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.691+0000 7f8dd05b7640 1 --2- 172.21.15.16:0/2497477703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc806d960 0x7f8dc80d4370 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.691+0000 7f8dd05b7640 1 --2- 172.21.15.16:0/2497477703 >> v2:172.21.15.146:3300/0 conn(0x7f8dc815e2a0 0x7f8dc806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.692+0000 7f8dd05b7640 1 -- 172.21.15.16:0/2497477703 >> 172.21.15.16:0/2497477703 conn(0x7f8dc809d760 msgr2=0x7f8dc808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.692+0000 7f8dd05b7640 1 -- 172.21.15.16:0/2497477703 shutdown_connections 2024-09-19T07:54:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.692+0000 7f8dd05b7640 1 -- 172.21.15.16:0/2497477703 wait complete. 2024-09-19T07:54:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.693+0000 7f8dd05b7640 1 Processor -- start 2024-09-19T07:54:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.693+0000 7f8dd05b7640 1 -- start start 2024-09-19T07:54:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.693+0000 7f8dd05b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8dc806d960 0x7f8dc80cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.693+0000 7f8dd05b7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8dc815e2a0 0x7f8dc815a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.693+0000 7f8dd05b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc815e680 0x7f8dc815f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.693+0000 7f8dd05b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc80d0360 con 0x7f8dc815e680 2024-09-19T07:54:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.693+0000 7f8dd05b7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc80d01e0 con 0x7f8dc815e2a0 2024-09-19T07:54:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.693+0000 7f8dd05b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8dc80d48b0 con 0x7f8dc806d960 2024-09-19T07:54:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.694+0000 7f8dceb54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc815e680 0x7f8dc815f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.694+0000 7f8dceb54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc815e680 0x7f8dc815f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56614/0 (socket says 172.21.15.16:56614) 2024-09-19T07:54:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.694+0000 7f8dceb54640 1 -- 172.21.15.16:0/4179830299 learned_addr learned my addr 172.21.15.16:0/4179830299 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.694+0000 7f8dcdb52640 1 --2- 172.21.15.16:0/4179830299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8dc815e2a0 0x7f8dc815a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.694+0000 7f8dce353640 1 --2- 172.21.15.16:0/4179830299 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8dc806d960 0x7f8dc80cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.694+0000 7f8dceb54640 1 -- 172.21.15.16:0/4179830299 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8dc806d960 msgr2=0x7f8dc80cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.694+0000 7f8dceb54640 1 --2- 172.21.15.16:0/4179830299 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8dc806d960 0x7f8dc80cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.694+0000 7f8dceb54640 1 -- 172.21.15.16:0/4179830299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8dc815e2a0 msgr2=0x7f8dc815a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.694+0000 7f8dceb54640 1 --2- 172.21.15.16:0/4179830299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8dc815e2a0 0x7f8dc815a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.694+0000 7f8dceb54640 1 -- 172.21.15.16:0/4179830299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc815ae20 con 0x7f8dc815e680 2024-09-19T07:54:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.694+0000 7f8dcdb52640 1 --2- 172.21.15.16:0/4179830299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8dc815e2a0 0x7f8dc815a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.694+0000 7f8dce353640 1 --2- 172.21.15.16:0/4179830299 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8dc806d960 0x7f8dc80cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.694+0000 7f8dceb54640 1 --2- 172.21.15.16:0/4179830299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc815e680 0x7f8dc815f8a0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009f40 tx=0x7f8dc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.694+0000 7f8dbeffd640 1 -- 172.21.15.16:0/4179830299 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0036050 con 0x7f8dc815e680 2024-09-19T07:54:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.694+0000 7f8dbeffd640 1 -- 172.21.15.16:0/4179830299 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc0042070 con 0x7f8dc815e680 2024-09-19T07:54:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.694+0000 7f8dd05b7640 1 -- 172.21.15.16:0/4179830299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc80cb700 con 0x7f8dc815e680 2024-09-19T07:54:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.694+0000 7f8dbeffd640 1 -- 172.21.15.16:0/4179830299 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0009510 con 0x7f8dc815e680 2024-09-19T07:54:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.694+0000 7f8dd05b7640 1 -- 172.21.15.16:0/4179830299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc80cdb20 con 0x7f8dc815e680 2024-09-19T07:54:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.695+0000 7f8dd05b7640 1 -- 172.21.15.16:0/4179830299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc80d4c70 con 0x7f8dc815e680 2024-09-19T07:54:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.696+0000 7f8dbeffd640 1 -- 172.21.15.16:0/4179830299 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8dc0007c50 con 0x7f8dc815e680 2024-09-19T07:54:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.696+0000 7f8dbeffd640 1 --2- 172.21.15.16:0/4179830299 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8dac06c2b0 0x7f8dac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.696+0000 7f8dbeffd640 1 -- 172.21.15.16:0/4179830299 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7f8dc00ac450 con 0x7f8dc815e680 2024-09-19T07:54:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.696+0000 7f8dce353640 1 --2- 172.21.15.16:0/4179830299 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8dac06c2b0 0x7f8dac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.697+0000 7f8dce353640 1 --2- 172.21.15.16:0/4179830299 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8dac06c2b0 0x7f8dac06e770 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4006fd0 tx=0x7f8dc4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.701+0000 7f8dbeffd640 1 -- 172.21.15.16:0/4179830299 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8dc007aa30 con 0x7f8dc815e680 2024-09-19T07:54:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.915+0000 7f8dd05b7640 1 -- 172.21.15.16:0/4179830299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8dc80c7c90 con 0x7f8dc815e680 2024-09-19T07:54:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.915+0000 7f8dbeffd640 1 -- 172.21.15.16:0/4179830299 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7f8dc007a330 con 0x7f8dc815e680 2024-09-19T07:54:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:44.916 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:54:44.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.917+0000 7f8dd05b7640 1 -- 172.21.15.16:0/4179830299 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8dac06c2b0 msgr2=0x7f8dac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.917+0000 7f8dd05b7640 1 --2- 172.21.15.16:0/4179830299 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8dac06c2b0 0x7f8dac06e770 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4006fd0 tx=0x7f8dc4008040 comp rx=0 tx=0).stop 2024-09-19T07:54:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.917+0000 7f8dd05b7640 1 -- 172.21.15.16:0/4179830299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc815e680 msgr2=0x7f8dc815f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.917+0000 7f8dd05b7640 1 --2- 172.21.15.16:0/4179830299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc815e680 0x7f8dc815f8a0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009f40 tx=0x7f8dc0004a40 comp rx=0 tx=0).stop 2024-09-19T07:54:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.917+0000 7f8dd05b7640 1 -- 172.21.15.16:0/4179830299 shutdown_connections 2024-09-19T07:54:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.917+0000 7f8dd05b7640 1 --2- 172.21.15.16:0/4179830299 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8dac06c2b0 0x7f8dac06e770 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.917+0000 7f8dd05b7640 1 --2- 172.21.15.16:0/4179830299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc815e680 0x7f8dc815f8a0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.917+0000 7f8dd05b7640 1 --2- 172.21.15.16:0/4179830299 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8dc815e2a0 0x7f8dc815a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.917+0000 7f8dd05b7640 1 --2- 172.21.15.16:0/4179830299 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8dc806d960 0x7f8dc80cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.917+0000 7f8dd05b7640 1 -- 172.21.15.16:0/4179830299 >> 172.21.15.16:0/4179830299 conn(0x7f8dc809d760 msgr2=0x7f8dc80d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.917+0000 7f8dd05b7640 1 -- 172.21.15.16:0/4179830299 shutdown_connections 2024-09-19T07:54:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44.917+0000 7f8dd05b7640 1 -- 172.21.15.16:0/4179830299 wait complete. 2024-09-19T07:54:44.928 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-19T07:54:44.928 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2024-09-19T07:54:44.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:44 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-19T07:54:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:54:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:stdin closed, goodbye! 2024-09-19T07:54:46.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:54:46.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:54:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.674+0000 7ff890cdd640 1 Processor -- start 2024-09-19T07:54:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.674+0000 7ff890cdd640 1 -- start start 2024-09-19T07:54:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.675+0000 7ff890cdd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff88c15c140 0x7ff88c160650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.675+0000 7ff890cdd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff88c0b6650 0x7ff88c15d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.675+0000 7ff890cdd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff88c160b90 con 0x7ff88c06f9d0 2024-09-19T07:54:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.675+0000 7ff890cdd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff88c15c520 con 0x7ff88c15c140 2024-09-19T07:54:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.675+0000 7ff890cdd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff88c15c7f0 con 0x7ff88c0b6650 2024-09-19T07:54:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.675+0000 7ff889d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff88c06f9d0 0x7ff88c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47808/0 (socket says 172.21.15.16:47808) 2024-09-19T07:54:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.675+0000 7ff889d9b640 1 -- 172.21.15.16:0/3017497353 learned_addr learned my addr 172.21.15.16:0/3017497353 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.675+0000 7ff88a59c640 1 --2- 172.21.15.16:0/3017497353 >> v2:172.21.15.146:3300/0 conn(0x7ff88c15c140 0x7ff88c160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.675+0000 7ff88ad9d640 1 --2- 172.21.15.16:0/3017497353 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff88c0b6650 0x7ff88c15d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:46.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.675+0000 7ff88959a640 1 -- 172.21.15.16:0/3017497353 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1752240776 0 0) 0x7ff880001110 con 0x7ff88c06f9d0 2024-09-19T07:54:46.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.675+0000 7ff88959a640 1 -- 172.21.15.16:0/3017497353 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff874003520 con 0x7ff88c06f9d0 2024-09-19T07:54:46.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.675+0000 7ff88a59c640 1 -- 172.21.15.16:0/3017497353 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff88c0b6650 msgr2=0x7ff88c15d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:46.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.675+0000 7ff88a59c640 1 --2- 172.21.15.16:0/3017497353 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff88c0b6650 0x7ff88c15d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.675+0000 7ff88a59c640 1 -- 172.21.15.16:0/3017497353 >> v1:172.21.15.16:6789/0 conn(0x7ff88c06f9d0 legacy=0x7ff88c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.676+0000 7ff88a59c640 1 -- 172.21.15.16:0/3017497353 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88c15cad0 con 0x7ff88c15c140 2024-09-19T07:54:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.676+0000 7ff88ad9d640 1 --2- 172.21.15.16:0/3017497353 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff88c0b6650 0x7ff88c15d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:54:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.676+0000 7ff88a59c640 1 --2- 172.21.15.16:0/3017497353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff88c15c140 0x7ff88c160650 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0097a0 tx=0x7ff87c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3582e83674abbd71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.677+0000 7ff88959a640 1 -- 172.21.15.16:0/3017497353 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c037070 con 0x7ff88c15c140 2024-09-19T07:54:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.677+0000 7ff88959a640 1 -- 172.21.15.16:0/3017497353 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff87c032440 con 0x7ff88c15c140 2024-09-19T07:54:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.677+0000 7ff88959a640 1 -- 172.21.15.16:0/3017497353 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c0327f0 con 0x7ff88c15c140 2024-09-19T07:54:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.677+0000 7ff890cdd640 1 -- 172.21.15.16:0/3017497353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff88c15c140 msgr2=0x7ff88c160650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.677+0000 7ff890cdd640 1 --2- 172.21.15.16:0/3017497353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff88c15c140 0x7ff88c160650 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0097a0 tx=0x7ff87c02a180 comp rx=0 tx=0).stop 2024-09-19T07:54:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.677+0000 7ff890cdd640 1 -- 172.21.15.16:0/3017497353 shutdown_connections 2024-09-19T07:54:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.677+0000 7ff890cdd640 1 --2- 172.21.15.16:0/3017497353 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff88c0b6650 0x7ff88c15d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.677+0000 7ff890cdd640 1 --2- 172.21.15.16:0/3017497353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff88c15c140 0x7ff88c160650 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.677+0000 7ff890cdd640 1 -- 172.21.15.16:0/3017497353 >> 172.21.15.16:0/3017497353 conn(0x7ff88c06bc10 msgr2=0x7ff88c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:46.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.677+0000 7ff890cdd640 1 -- 172.21.15.16:0/3017497353 shutdown_connections 2024-09-19T07:54:46.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.677+0000 7ff890cdd640 1 -- 172.21.15.16:0/3017497353 wait complete. 2024-09-19T07:54:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.679+0000 7ff890cdd640 1 Processor -- start 2024-09-19T07:54:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.679+0000 7ff890cdd640 1 -- start start 2024-09-19T07:54:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.679+0000 7ff890cdd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff88c06f9d0 0x7ff88c14c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.679+0000 7ff890cdd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff88c0b6650 0x7ff88c0ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.679+0000 7ff890cdd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff88c15c140 0x7ff88c0adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.679+0000 7ff890cdd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c15d370 con 0x7ff88c0b6650 2024-09-19T07:54:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.679+0000 7ff890cdd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c15d090 con 0x7ff88c06f9d0 2024-09-19T07:54:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.679+0000 7ff890cdd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff88c15ddd0 con 0x7ff88c15c140 2024-09-19T07:54:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.679+0000 7ff889d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff88c0b6650 0x7ff88c0ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.679+0000 7ff889d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff88c0b6650 0x7ff88c0ad9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56618/0 (socket says 172.21.15.16:56618) 2024-09-19T07:54:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.679+0000 7ff889d9b640 1 -- 172.21.15.16:0/745150325 learned_addr learned my addr 172.21.15.16:0/745150325 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.680+0000 7ff88a59c640 1 --2- 172.21.15.16:0/745150325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff88c06f9d0 0x7ff88c14c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.680+0000 7ff88ad9d640 1 --2- 172.21.15.16:0/745150325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff88c15c140 0x7ff88c0adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.680+0000 7ff889d9b640 1 -- 172.21.15.16:0/745150325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff88c15c140 msgr2=0x7ff88c0adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.680+0000 7ff889d9b640 1 --2- 172.21.15.16:0/745150325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff88c15c140 0x7ff88c0adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.680+0000 7ff889d9b640 1 -- 172.21.15.16:0/745150325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff88c06f9d0 msgr2=0x7ff88c14c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.680+0000 7ff889d9b640 1 --2- 172.21.15.16:0/745150325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff88c06f9d0 0x7ff88c14c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.680+0000 7ff889d9b640 1 -- 172.21.15.16:0/745150325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88c0ae760 con 0x7ff88c0b6650 2024-09-19T07:54:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.680+0000 7ff88ad9d640 1 --2- 172.21.15.16:0/745150325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff88c15c140 0x7ff88c0adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.680+0000 7ff889d9b640 1 --2- 172.21.15.16:0/745150325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff88c0b6650 0x7ff88c0ad9c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7ff88000c730 tx=0x7ff88000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.680+0000 7ff87affd640 1 -- 172.21.15.16:0/745150325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880017070 con 0x7ff88c0b6650 2024-09-19T07:54:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.680+0000 7ff87affd640 1 -- 172.21.15.16:0/745150325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff880005ad0 con 0x7ff88c0b6650 2024-09-19T07:54:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.680+0000 7ff890cdd640 1 -- 172.21.15.16:0/745150325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88c0aea40 con 0x7ff88c0b6650 2024-09-19T07:54:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.680+0000 7ff87affd640 1 -- 172.21.15.16:0/745150325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880007e20 con 0x7ff88c0b6650 2024-09-19T07:54:46.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.680+0000 7ff890cdd640 1 -- 172.21.15.16:0/745150325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff88c0af2e0 con 0x7ff88c0b6650 2024-09-19T07:54:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.681+0000 7ff890cdd640 1 -- 172.21.15.16:0/745150325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff88c15d890 con 0x7ff88c0b6650 2024-09-19T07:54:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.682+0000 7ff87affd640 1 -- 172.21.15.16:0/745150325 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff88001a070 con 0x7ff88c0b6650 2024-09-19T07:54:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.682+0000 7ff87affd640 1 --2- 172.21.15.16:0/745150325 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff86c06c2b0 0x7ff86c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.682+0000 7ff87affd640 1 -- 172.21.15.16:0/745150325 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7ff88008a840 con 0x7ff88c0b6650 2024-09-19T07:54:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.682+0000 7ff88a59c640 1 --2- 172.21.15.16:0/745150325 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff86c06c2b0 0x7ff86c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:46.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.683+0000 7ff88a59c640 1 --2- 172.21.15.16:0/745150325 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff86c06c2b0 0x7ff86c06e770 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff87c034000 tx=0x7ff87c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:46.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.687+0000 7ff87affd640 1 -- 172.21.15.16:0/745150325 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff88c160b90 con 0x7ff88c0b6650 2024-09-19T07:54:46.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.898+0000 7ff890cdd640 1 -- 172.21.15.16:0/745150325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff88c067fc0 con 0x7ff88c0b6650 2024-09-19T07:54:46.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.899+0000 7ff87affd640 1 -- 172.21.15.16:0/745150325 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5219 (secure 0 0 0) 0x7ff88005f080 con 0x7ff88c0b6650 2024-09-19T07:54:46.899 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:46.899 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":74,"btime":"2024-09-19T07:54:44:229158+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/2994096322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2994096322},{"type":"v1","addr":"172.21.15.146:6836","nonce":2994096322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7839,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/393564462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":393564462},{"type":"v1","addr":"172.21.15.16:6833","nonce":393564462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":7842,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/631392582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":631392582},{"type":"v1","addr":"172.21.15.16:6835","nonce":631392582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:54:22.949504+0000","modified":"2024-09-19T07:54:43.595934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6837/79304323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":79304323},{"type":"v1","addr":"172.21.15.146:6837","nonce":79304323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7854,"qdb_cluster":[7854]},"id":9}]} 2024-09-19T07:54:46.900 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 74 2024-09-19T07:54:46.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.900+0000 7ff890cdd640 1 -- 172.21.15.16:0/745150325 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff86c06c2b0 msgr2=0x7ff86c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:46.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.900+0000 7ff890cdd640 1 --2- 172.21.15.16:0/745150325 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff86c06c2b0 0x7ff86c06e770 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff87c034000 tx=0x7ff87c02e000 comp rx=0 tx=0).stop 2024-09-19T07:54:46.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.901+0000 7ff890cdd640 1 -- 172.21.15.16:0/745150325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff88c0b6650 msgr2=0x7ff88c0ad9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:46.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.901+0000 7ff890cdd640 1 --2- 172.21.15.16:0/745150325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff88c0b6650 0x7ff88c0ad9c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7ff88000c730 tx=0x7ff88000cbf0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.901+0000 7ff890cdd640 1 -- 172.21.15.16:0/745150325 shutdown_connections 2024-09-19T07:54:46.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.901+0000 7ff890cdd640 1 --2- 172.21.15.16:0/745150325 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff86c06c2b0 0x7ff86c06e770 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.901+0000 7ff890cdd640 1 --2- 172.21.15.16:0/745150325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff88c15c140 0x7ff88c0adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.901+0000 7ff890cdd640 1 --2- 172.21.15.16:0/745150325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff88c0b6650 0x7ff88c0ad9c0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.901+0000 7ff890cdd640 1 --2- 172.21.15.16:0/745150325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff88c06f9d0 0x7ff88c14c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.901+0000 7ff890cdd640 1 -- 172.21.15.16:0/745150325 >> 172.21.15.16:0/745150325 conn(0x7ff88c06bc10 msgr2=0x7ff88c15f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:46.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.901+0000 7ff890cdd640 1 -- 172.21.15.16:0/745150325 shutdown_connections 2024-09-19T07:54:46.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:46.901+0000 7ff890cdd640 1 -- 172.21.15.16:0/745150325 wait complete. 2024-09-19T07:54:46.912 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-19T07:54:47.041 INFO:teuthology.orchestra.run.smithi146.stdout:[{"id":6635,"entity":{"name":{"type":"client","num":6635},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2776231670}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":11.740823463,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9328256512660307,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6635 192.168.0.1:0/2776231670","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"49038","root":"/"}},{"id":8007,"entity":{"name":{"type":"client","num":8007},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1934386859}},"state":"open","num_leases":1,"num_caps":202,"request_load_avg":185,"uptime":13.287800203,"requests_in_flight":0,"num_completed_requests":15,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":855.1941028623404,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":215.31600765941621,"halflife":1.5},"recall_caps_throttle2o":{"value":70.755833559843111,"halflife":0.5},"session_cache_liveness":{"value":397.91773289019358,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":190,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8007 192.168.0.1:0/1934386859","completed_requests":[{"tid":190,"created_ino":"0x100000002ae"},{"tid":191,"created_ino":"0x100000002af"},{"tid":192,"created_ino":"0x100000002b0"},{"tid":193,"created_ino":"0x100000002b1"},{"tid":194,"created_ino":"0x100000002b2"},{"tid":195,"created_ino":"0x100000002b3"},{"tid":196,"created_ino":"0x100000002b4"},{"tid":197,"created_ino":"0x100000002b5"},{"tid":198,"created_ino":"0x100000002b6"},{"tid":199,"created_ino":"0x100000002b7"},{"tid":200,"created_ino":"0x100000002b8"},{"tid":201,"created_ino":"0x100000002b9"},{"tid":202,"created_ino":"0x100000002ba"},{"tid":203,"created_ino":"0x100000002bb"},{"tid":204,"created_ino":"0x100000002bc"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x100000002bd","length":802}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"0","hostname":"smithi016","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"72929","root":"/"}}] 2024-09-19T07:54:47.042 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "49038", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2776231670, "type": "any" }, "name": { "num": 6635, "type": "client" } }, "id": 6635, "inst": "client.6635 192.168.0.1:0/2776231670", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9328256512660307 }, "state": "open", "uptime": 11.740823463 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "0", "hostname": "smithi016", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "72929", "root": "/" }, "completed_requests": [ { "created_ino": "0x100000002ae", "tid": 190 }, { "created_ino": "0x100000002af", "tid": 191 }, { "created_ino": "0x100000002b0", "tid": 192 }, { "created_ino": "0x100000002b1", "tid": 193 }, { "created_ino": "0x100000002b2", "tid": 194 }, { "created_ino": "0x100000002b3", "tid": 195 }, { "created_ino": "0x100000002b4", "tid": 196 }, { "created_ino": "0x100000002b5", "tid": 197 }, { "created_ino": "0x100000002b6", "tid": 198 }, { "created_ino": "0x100000002b7", "tid": 199 }, { "created_ino": "0x100000002b8", "tid": 200 }, { "created_ino": "0x100000002b9", "tid": 201 }, { "created_ino": "0x100000002ba", "tid": 202 }, { "created_ino": "0x100000002bb", "tid": 203 }, { "created_ino": "0x100000002bc", "tid": 204 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1934386859, "type": "any" }, "name": { "num": 8007, "type": "client" } }, "id": 8007, "inst": "client.8007 192.168.0.1:0/1934386859", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 190, "num_caps": 202, "num_completed_flushes": 0, "num_completed_requests": 15, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 802, "start": "0x100000002bd" } ], "recall_caps": { "halflife": 60, "value": 855.1941028623404 }, "recall_caps_throttle": { "halflife": 1.5, "value": 215.31600765941621 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 70.75583355984311 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 185, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 397.9177328901936 }, "state": "open", "uptime": 13.287800203 } ] 2024-09-19T07:54:47.042 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-19T07:54:47.042 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:54:47.042 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:54:47.083 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:54:47.120 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:54:47.120 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:54:47.121 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 200\nabs_path = "/home/ubuntu/cephtest/mnt.0/subdir"\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi016') 2024-09-19T07:54:47.121 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:54:47.121 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:54:47.139 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:54:47.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.284+0000 7fbfd3006640 1 Processor -- start 2024-09-19T07:54:47.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.284+0000 7fbfd3006640 1 -- start start 2024-09-19T07:54:47.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.285+0000 7fbfd3006640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbfcc0d0540 0x7fbfcc141a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.285+0000 7fbfd3006640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfcc141fa0 0x7fbfcc13f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.285+0000 7fbfd3006640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfcc15d710 con 0x7fbfcc141680 2024-09-19T07:54:47.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.285+0000 7fbfd3006640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fbfcc15c6f0 con 0x7fbfcc0d0540 2024-09-19T07:54:47.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.285+0000 7fbfd3006640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbfcc15c950 con 0x7fbfcc141fa0 2024-09-19T07:54:47.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.285+0000 7fbfd0da2640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbfcc141680 0x7fbfcc0cfe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47810/0 (socket says 172.21.15.16:47810) 2024-09-19T07:54:47.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.285+0000 7fbfd0da2640 1 -- 172.21.15.16:0/2214429639 learned_addr learned my addr 172.21.15.16:0/2214429639 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:47.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.285+0000 7fbfcbfff640 1 --2- 172.21.15.16:0/2214429639 >> v2:172.21.15.146:3300/0 conn(0x7fbfcc0d0540 0x7fbfcc141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.285+0000 7fbfd15a3640 1 --2- 172.21.15.16:0/2214429639 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfcc141fa0 0x7fbfcc13f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.285+0000 7fbfcb7fe640 1 -- 172.21.15.16:0/2214429639 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3700615752 0 0) 0x7fbfcc15d710 con 0x7fbfcc141680 2024-09-19T07:54:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.285+0000 7fbfcb7fe640 1 -- 172.21.15.16:0/2214429639 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfb4003520 con 0x7fbfcc141680 2024-09-19T07:54:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.286+0000 7fbfd15a3640 1 -- 172.21.15.16:0/2214429639 >> v2:172.21.15.146:3300/0 conn(0x7fbfcc0d0540 msgr2=0x7fbfcc141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.286+0000 7fbfcb7fe640 1 -- 172.21.15.16:0/2214429639 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3306124584 0 0) 0x7fbfb4003520 con 0x7fbfcc141680 2024-09-19T07:54:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.286+0000 7fbfd15a3640 1 --2- 172.21.15.16:0/2214429639 >> v2:172.21.15.146:3300/0 conn(0x7fbfcc0d0540 0x7fbfcc141a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.286+0000 7fbfd15a3640 1 -- 172.21.15.16:0/2214429639 >> v1:172.21.15.16:6789/0 conn(0x7fbfcc141680 legacy=0x7fbfcc0cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.286+0000 7fbfd15a3640 1 -- 172.21.15.16:0/2214429639 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfcc15cc30 con 0x7fbfcc141fa0 2024-09-19T07:54:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.286+0000 7fbfcbfff640 1 --2- 172.21.15.16:0/2214429639 >> v2:172.21.15.146:3300/0 conn(0x7fbfcc0d0540 0x7fbfcc141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:54:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.286+0000 7fbfd15a3640 1 --2- 172.21.15.16:0/2214429639 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfcc141fa0 0x7fbfcc13f8b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0097a0 tx=0x7fbfbc02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd03ed85c5b15bdf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.287+0000 7fbfcb7fe640 1 -- 172.21.15.16:0/2214429639 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc037070 con 0x7fbfcc141fa0 2024-09-19T07:54:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.287+0000 7fbfcb7fe640 1 -- 172.21.15.16:0/2214429639 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfbc02aba0 con 0x7fbfcc141fa0 2024-09-19T07:54:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.287+0000 7fbfcb7fe640 1 -- 172.21.15.16:0/2214429639 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4001a10 con 0x7fbfcc141fa0 2024-09-19T07:54:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.287+0000 7fbfd3006640 1 -- 172.21.15.16:0/2214429639 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfcc141fa0 msgr2=0x7fbfcc13f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.287+0000 7fbfd3006640 1 --2- 172.21.15.16:0/2214429639 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfcc141fa0 0x7fbfcc13f8b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0097a0 tx=0x7fbfbc02a110 comp rx=0 tx=0).stop 2024-09-19T07:54:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.287+0000 7fbfd3006640 1 -- 172.21.15.16:0/2214429639 shutdown_connections 2024-09-19T07:54:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.287+0000 7fbfd3006640 1 --2- 172.21.15.16:0/2214429639 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfcc141fa0 0x7fbfcc13f8b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.287+0000 7fbfd3006640 1 --2- 172.21.15.16:0/2214429639 >> v2:172.21.15.146:3300/0 conn(0x7fbfcc0d0540 0x7fbfcc141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.287+0000 7fbfd3006640 1 -- 172.21.15.16:0/2214429639 >> 172.21.15.16:0/2214429639 conn(0x7fbfcc0b3ed0 msgr2=0x7fbfcc0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.287+0000 7fbfd3006640 1 -- 172.21.15.16:0/2214429639 shutdown_connections 2024-09-19T07:54:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.287+0000 7fbfd3006640 1 -- 172.21.15.16:0/2214429639 wait complete. 2024-09-19T07:54:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.288+0000 7fbfd3006640 1 Processor -- start 2024-09-19T07:54:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.288+0000 7fbfd3006640 1 -- start start 2024-09-19T07:54:47.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.289+0000 7fbfd3006640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfcc0d0540 0x7fbfcc1e6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.289+0000 7fbfd3006640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfcc141680 0x7fbfcc145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.289+0000 7fbfd3006640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfcc141fa0 0x7fbfcc1460b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.289+0000 7fbfd3006640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc1423f0 con 0x7fbfcc141fa0 2024-09-19T07:54:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.289+0000 7fbfd3006640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc15d1f0 con 0x7fbfcc141680 2024-09-19T07:54:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.289+0000 7fbfd3006640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbfcc142570 con 0x7fbfcc0d0540 2024-09-19T07:54:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.289+0000 7fbfd15a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfcc141fa0 0x7fbfcc1460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.290+0000 7fbfd15a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfcc141fa0 0x7fbfcc1460b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56632/0 (socket says 172.21.15.16:56632) 2024-09-19T07:54:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.290+0000 7fbfd15a3640 1 -- 172.21.15.16:0/2885392258 learned_addr learned my addr 172.21.15.16:0/2885392258 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.290+0000 7fbfcbfff640 1 --2- 172.21.15.16:0/2885392258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfcc141680 0x7fbfcc145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.290+0000 7fbfd0da2640 1 --2- 172.21.15.16:0/2885392258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfcc0d0540 0x7fbfcc1e6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.290+0000 7fbfd15a3640 1 -- 172.21.15.16:0/2885392258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfcc0d0540 msgr2=0x7fbfcc1e6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.290+0000 7fbfd15a3640 1 --2- 172.21.15.16:0/2885392258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfcc0d0540 0x7fbfcc1e6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.290+0000 7fbfd15a3640 1 -- 172.21.15.16:0/2885392258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfcc141680 msgr2=0x7fbfcc145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.290+0000 7fbfd15a3640 1 --2- 172.21.15.16:0/2885392258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfcc141680 0x7fbfcc145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.290+0000 7fbfd15a3640 1 -- 172.21.15.16:0/2885392258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfcc0b7920 con 0x7fbfcc141fa0 2024-09-19T07:54:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.290+0000 7fbfcbfff640 1 --2- 172.21.15.16:0/2885392258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfcc141680 0x7fbfcc145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.290+0000 7fbfd15a3640 1 --2- 172.21.15.16:0/2885392258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfcc141fa0 0x7fbfcc1460b0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0097a0 tx=0x7fbfbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.290+0000 7fbfc97fa640 1 -- 172.21.15.16:0/2885392258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc037050 con 0x7fbfcc141fa0 2024-09-19T07:54:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.290+0000 7fbfc97fa640 1 -- 172.21.15.16:0/2885392258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfbc038040 con 0x7fbfcc141fa0 2024-09-19T07:54:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.290+0000 7fbfd3006640 1 -- 172.21.15.16:0/2885392258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfcc147170 con 0x7fbfcc141fa0 2024-09-19T07:54:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.290+0000 7fbfc97fa640 1 -- 172.21.15.16:0/2885392258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc006e10 con 0x7fbfcc141fa0 2024-09-19T07:54:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.290+0000 7fbfd3006640 1 -- 172.21.15.16:0/2885392258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfcc147730 con 0x7fbfcc141fa0 2024-09-19T07:54:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.291+0000 7fbfd3006640 1 -- 172.21.15.16:0/2885392258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfcc141a60 con 0x7fbfcc141fa0 2024-09-19T07:54:47.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.292+0000 7fbfc97fa640 1 -- 172.21.15.16:0/2885392258 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbfbc003610 con 0x7fbfcc141fa0 2024-09-19T07:54:47.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.292+0000 7fbfc97fa640 1 --2- 172.21.15.16:0/2885392258 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbfb806c2b0 0x7fbfb806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.292+0000 7fbfc97fa640 1 -- 172.21.15.16:0/2885392258 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7fbfbc0abe40 con 0x7fbfcc141fa0 2024-09-19T07:54:47.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.292+0000 7fbfd0da2640 1 --2- 172.21.15.16:0/2885392258 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbfb806c2b0 0x7fbfb806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.293+0000 7fbfd0da2640 1 --2- 172.21.15.16:0/2885392258 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbfb806c2b0 0x7fbfb806e770 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4001330 tx=0x7fbfc4007e10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:47.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.296+0000 7fbfc97fa640 1 -- 172.21.15.16:0/2885392258 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fbfbc07a540 con 0x7fbfcc141fa0 2024-09-19T07:54:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.486+0000 7fbfd3006640 1 -- 172.21.15.16:0/2885392258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbfcc0690c0 con 0x7fbfcc141fa0 2024-09-19T07:54:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.486+0000 7fbfc97fa640 1 -- 172.21.15.16:0/2885392258 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v129) ==== 80+0+0 (secure 0 0 0) 0x7fbfbc079e40 con 0x7fbfcc141fa0 2024-09-19T07:54:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:54:47.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.488+0000 7fbfd3006640 1 -- 172.21.15.16:0/2885392258 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbfb806c2b0 msgr2=0x7fbfb806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.488+0000 7fbfd3006640 1 --2- 172.21.15.16:0/2885392258 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbfb806c2b0 0x7fbfb806e770 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4001330 tx=0x7fbfc4007e10 comp rx=0 tx=0).stop 2024-09-19T07:54:47.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.488+0000 7fbfd3006640 1 -- 172.21.15.16:0/2885392258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfcc141fa0 msgr2=0x7fbfcc1460b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.488+0000 7fbfd3006640 1 --2- 172.21.15.16:0/2885392258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfcc141fa0 0x7fbfcc1460b0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0097a0 tx=0x7fbfbc0046c0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.488+0000 7fbfd3006640 1 -- 172.21.15.16:0/2885392258 shutdown_connections 2024-09-19T07:54:47.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.488+0000 7fbfd3006640 1 --2- 172.21.15.16:0/2885392258 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbfb806c2b0 0x7fbfb806e770 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.488+0000 7fbfd3006640 1 --2- 172.21.15.16:0/2885392258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfcc141fa0 0x7fbfcc1460b0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.488+0000 7fbfd3006640 1 --2- 172.21.15.16:0/2885392258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfcc141680 0x7fbfcc145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.488+0000 7fbfd3006640 1 --2- 172.21.15.16:0/2885392258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfcc0d0540 0x7fbfcc1e6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.488+0000 7fbfd3006640 1 -- 172.21.15.16:0/2885392258 >> 172.21.15.16:0/2885392258 conn(0x7fbfcc0b3ed0 msgr2=0x7fbfcc13e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.488+0000 7fbfd3006640 1 -- 172.21.15.16:0/2885392258 shutdown_connections 2024-09-19T07:54:47.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.488+0000 7fbfd3006640 1 -- 172.21.15.16:0/2885392258 wait complete. 2024-09-19T07:54:47.500 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-19T07:54:47.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:47.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T07:54:47.525 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[72929]: fuse finished with error 0 and tester_r 0 2024-09-19T07:54:47.527 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:54:47.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:47.527 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:54:47.697 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:54:47.697 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:54:47.697 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:54:47.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:54:47.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.841+0000 7fc0e121e640 1 Processor -- start 2024-09-19T07:54:47.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.841+0000 7fc0e121e640 1 -- start start 2024-09-19T07:54:47.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.841+0000 7fc0e121e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0dc15e680 0x7fc0dc0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.841+0000 7fc0e121e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0dc0d1880 con 0x7fc0dc0d0010 2024-09-19T07:54:47.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.841+0000 7fc0e121e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0dc0d1b60 con 0x7fc0dc15e2a0 2024-09-19T07:54:47.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.841+0000 7fc0e121e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc0dc0cf2d0 con 0x7fc0dc15e680 2024-09-19T07:54:47.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0db59e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc0dc0d0010 0x7fc0dc0d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47824/0 (socket says 172.21.15.16:47824) 2024-09-19T07:54:47.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0db59e640 1 -- 172.21.15.16:0/3415610783 learned_addr learned my addr 172.21.15.16:0/3415610783 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:47.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0da59c640 1 --2- 172.21.15.16:0/3415610783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0dc15e680 0x7fc0dc0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0d9d9b640 1 -- 172.21.15.16:0/3415610783 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1668251175 0 0) 0x7fc0dc0d1880 con 0x7fc0dc0d0010 2024-09-19T07:54:47.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0d9d9b640 1 -- 172.21.15.16:0/3415610783 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0c4003520 con 0x7fc0dc0d0010 2024-09-19T07:54:47.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0d9d9b640 1 -- 172.21.15.16:0/3415610783 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4043713354 0 0) 0x7fc0c4003520 con 0x7fc0dc0d0010 2024-09-19T07:54:47.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0d9d9b640 1 -- 172.21.15.16:0/3415610783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0dc15e680 msgr2=0x7fc0dc0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:47.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0d9d9b640 1 --2- 172.21.15.16:0/3415610783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0dc15e680 0x7fc0dc0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0d9d9b640 1 -- 172.21.15.16:0/3415610783 >> v1:172.21.15.146:6789/0 conn(0x7fc0dc15e2a0 legacy=0x7fc0dc06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0d9d9b640 1 -- 172.21.15.16:0/3415610783 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0dc0cf5b0 con 0x7fc0dc0d0010 2024-09-19T07:54:47.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0d9d9b640 1 -- 172.21.15.16:0/3415610783 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fc0d40019f0 con 0x7fc0dc0d0010 2024-09-19T07:54:47.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0d9d9b640 1 -- 172.21.15.16:0/3415610783 >> v1:172.21.15.16:6789/0 conn(0x7fc0dc0d0010 legacy=0x7fc0dc0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0d9d9b640 1 --2- 172.21.15.16:0/3415610783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0dc15e680 0x7fc0c40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0d9d9b640 1 --2- 172.21.15.16:0/3415610783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0c4003f20 0x7fc0c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0d9d9b640 1 --2- 172.21.15.16:0/3415610783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0c4004d40 0x7fc0c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0d9d9b640 1 -- 172.21.15.16:0/3415610783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0dc0cf2d0 con 0x7fc0c4004d40 2024-09-19T07:54:47.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0d9d9b640 1 -- 172.21.15.16:0/3415610783 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c400bad0 con 0x7fc0c4003f20 2024-09-19T07:54:47.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0d9d9b640 1 -- 172.21.15.16:0/3415610783 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c400bcd0 con 0x7fc0dc15e680 2024-09-19T07:54:47.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0db59e640 1 --2- 172.21.15.16:0/3415610783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0c4004d40 0x7fc0c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0dad9d640 1 --2- 172.21.15.16:0/3415610783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0dc15e680 0x7fc0c40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.842+0000 7fc0da59c640 1 --2- 172.21.15.16:0/3415610783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0c4003f20 0x7fc0c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.843+0000 7fc0db59e640 1 -- 172.21.15.16:0/3415610783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0dc15e680 msgr2=0x7fc0c40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:47.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.843+0000 7fc0db59e640 1 --2- 172.21.15.16:0/3415610783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0dc15e680 0x7fc0c40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.843+0000 7fc0db59e640 1 -- 172.21.15.16:0/3415610783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0c4003f20 msgr2=0x7fc0c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:47.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.843+0000 7fc0db59e640 1 --2- 172.21.15.16:0/3415610783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0c4003f20 0x7fc0c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.843+0000 7fc0db59e640 1 -- 172.21.15.16:0/3415610783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c4003520 con 0x7fc0c4004d40 2024-09-19T07:54:47.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.843+0000 7fc0db59e640 1 --2- 172.21.15.16:0/3415610783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0c4004d40 0x7fc0c40051c0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400a920 tx=0x7fc0c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fbf7fe94bd02a46 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:47.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.843+0000 7fc0d9d9b640 1 -- 172.21.15.16:0/3415610783 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d400a4c0 con 0x7fc0c4004d40 2024-09-19T07:54:47.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.844+0000 7fc0d9d9b640 1 -- 172.21.15.16:0/3415610783 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d402f070 con 0x7fc0c4004d40 2024-09-19T07:54:47.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.844+0000 7fc0d9d9b640 1 -- 172.21.15.16:0/3415610783 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d402bd20 con 0x7fc0c4004d40 2024-09-19T07:54:47.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.844+0000 7fc0e121e640 1 -- 172.21.15.16:0/3415610783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0c4004d40 msgr2=0x7fc0c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.844+0000 7fc0e121e640 1 --2- 172.21.15.16:0/3415610783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0c4004d40 0x7fc0c40051c0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400a920 tx=0x7fc0c4005830 comp rx=0 tx=0).stop 2024-09-19T07:54:47.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.844+0000 7fc0db59e640 1 -- 172.21.15.16:0/3415610783 reap_dead start 2024-09-19T07:54:47.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.844+0000 7fc0e121e640 1 -- 172.21.15.16:0/3415610783 shutdown_connections 2024-09-19T07:54:47.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.844+0000 7fc0e121e640 1 -- 172.21.15.16:0/3415610783 >> 172.21.15.16:0/3415610783 conn(0x7fc0dc09d760 msgr2=0x7fc0dc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:47.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.844+0000 7fc0e121e640 1 -- 172.21.15.16:0/3415610783 shutdown_connections 2024-09-19T07:54:47.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.844+0000 7fc0e121e640 1 -- 172.21.15.16:0/3415610783 wait complete. 2024-09-19T07:54:47.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.846+0000 7fc0e121e640 1 Processor -- start 2024-09-19T07:54:47.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.846+0000 7fc0e121e640 1 -- start start 2024-09-19T07:54:47.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.846+0000 7fc0e121e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0dc14a160 0x7fc0dc14d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.846+0000 7fc0e121e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0dc0ce610 0x7fc0dc1e8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.846+0000 7fc0e121e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0dc1e91a0 0x7fc0dc0b93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.846+0000 7fc0e121e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0dc06dcb0 con 0x7fc0dc0ce610 2024-09-19T07:54:47.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.846+0000 7fc0e121e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc0dc06db30 con 0x7fc0dc14a160 2024-09-19T07:54:47.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.846+0000 7fc0e121e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc0dc0d4a80 con 0x7fc0dc1e91a0 2024-09-19T07:54:47.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.846+0000 7fc0da59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0dc0ce610 0x7fc0dc1e8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.846+0000 7fc0da59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0dc0ce610 0x7fc0dc1e8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56642/0 (socket says 172.21.15.16:56642) 2024-09-19T07:54:47.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.847+0000 7fc0da59c640 1 -- 172.21.15.16:0/4111848689 learned_addr learned my addr 172.21.15.16:0/4111848689 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:47.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.847+0000 7fc0dad9d640 1 --2- 172.21.15.16:0/4111848689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0dc14a160 0x7fc0dc14d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.847+0000 7fc0db59e640 1 --2- 172.21.15.16:0/4111848689 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0dc1e91a0 0x7fc0dc0b93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.847+0000 7fc0da59c640 1 -- 172.21.15.16:0/4111848689 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0dc1e91a0 msgr2=0x7fc0dc0b93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.847+0000 7fc0da59c640 1 --2- 172.21.15.16:0/4111848689 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0dc1e91a0 0x7fc0dc0b93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.847+0000 7fc0da59c640 1 -- 172.21.15.16:0/4111848689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0dc14a160 msgr2=0x7fc0dc14d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.847+0000 7fc0da59c640 1 --2- 172.21.15.16:0/4111848689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0dc14a160 0x7fc0dc14d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.847+0000 7fc0da59c640 1 -- 172.21.15.16:0/4111848689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0dc0cd570 con 0x7fc0dc0ce610 2024-09-19T07:54:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.847+0000 7fc0dad9d640 1 --2- 172.21.15.16:0/4111848689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0dc14a160 0x7fc0dc14d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.847+0000 7fc0da59c640 1 --2- 172.21.15.16:0/4111848689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0dc0ce610 0x7fc0dc1e8c60 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0011960 tx=0x7fc0d0011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.847+0000 7fc0c37fe640 1 -- 172.21.15.16:0/4111848689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0019070 con 0x7fc0dc0ce610 2024-09-19T07:54:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.847+0000 7fc0c37fe640 1 -- 172.21.15.16:0/4111848689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d0004300 con 0x7fc0dc0ce610 2024-09-19T07:54:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.847+0000 7fc0e121e640 1 -- 172.21.15.16:0/4111848689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0dc0cd850 con 0x7fc0dc0ce610 2024-09-19T07:54:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.847+0000 7fc0c37fe640 1 -- 172.21.15.16:0/4111848689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d000fa60 con 0x7fc0dc0ce610 2024-09-19T07:54:47.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.847+0000 7fc0e121e640 1 -- 172.21.15.16:0/4111848689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0dc0cde10 con 0x7fc0dc0ce610 2024-09-19T07:54:47.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.848+0000 7fc0e121e640 1 -- 172.21.15.16:0/4111848689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0dc1e95b0 con 0x7fc0dc0ce610 2024-09-19T07:54:47.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.849+0000 7fc0c37fe640 1 -- 172.21.15.16:0/4111848689 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc0d001c070 con 0x7fc0dc0ce610 2024-09-19T07:54:47.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.849+0000 7fc0c37fe640 1 --2- 172.21.15.16:0/4111848689 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc0c806c2b0 0x7fc0c806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.849+0000 7fc0c37fe640 1 -- 172.21.15.16:0/4111848689 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7fc0d008ba50 con 0x7fc0dc0ce610 2024-09-19T07:54:47.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.850+0000 7fc0dad9d640 1 --2- 172.21.15.16:0/4111848689 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc0c806c2b0 0x7fc0c806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.850+0000 7fc0dad9d640 1 --2- 172.21.15.16:0/4111848689 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc0c806c2b0 0x7fc0c806e770 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc003700 tx=0x7fc0cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:47.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:47.854+0000 7fc0c37fe640 1 -- 172.21.15.16:0/4111848689 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc0dc1e95b0 con 0x7fc0dc0ce610 2024-09-19T07:54:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.041+0000 7fc0e121e640 1 -- 172.21.15.16:0/4111848689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc0dc06cf90 con 0x7fc0dc0ce610 2024-09-19T07:54:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.041+0000 7fc0c37fe640 1 -- 172.21.15.16:0/4111848689 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v129) ==== 80+0+0 (secure 0 0 0) 0x7fc0d005a040 con 0x7fc0dc0ce610 2024-09-19T07:54:48.042 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:54:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.043+0000 7fc0e121e640 1 -- 172.21.15.16:0/4111848689 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc0c806c2b0 msgr2=0x7fc0c806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.043+0000 7fc0e121e640 1 --2- 172.21.15.16:0/4111848689 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc0c806c2b0 0x7fc0c806e770 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc003700 tx=0x7fc0cc009040 comp rx=0 tx=0).stop 2024-09-19T07:54:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.043+0000 7fc0e121e640 1 -- 172.21.15.16:0/4111848689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0dc0ce610 msgr2=0x7fc0dc1e8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:48.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.043+0000 7fc0e121e640 1 --2- 172.21.15.16:0/4111848689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0dc0ce610 0x7fc0dc1e8c60 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0011960 tx=0x7fc0d0011d60 comp rx=0 tx=0).stop 2024-09-19T07:54:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.043+0000 7fc0e121e640 1 -- 172.21.15.16:0/4111848689 shutdown_connections 2024-09-19T07:54:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.043+0000 7fc0e121e640 1 --2- 172.21.15.16:0/4111848689 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc0c806c2b0 0x7fc0c806e770 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.043+0000 7fc0e121e640 1 --2- 172.21.15.16:0/4111848689 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0dc1e91a0 0x7fc0dc0b93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.043+0000 7fc0e121e640 1 --2- 172.21.15.16:0/4111848689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0dc0ce610 0x7fc0dc1e8c60 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.043+0000 7fc0e121e640 1 --2- 172.21.15.16:0/4111848689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0dc14a160 0x7fc0dc14d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.043+0000 7fc0e121e640 1 -- 172.21.15.16:0/4111848689 >> 172.21.15.16:0/4111848689 conn(0x7fc0dc09d760 msgr2=0x7fc0dc0d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.044+0000 7fc0e121e640 1 -- 172.21.15.16:0/4111848689 shutdown_connections 2024-09-19T07:54:48.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.044+0000 7fc0e121e640 1 -- 172.21.15.16:0/4111848689 wait complete. 2024-09-19T07:54:48.055 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2024-09-19T07:54:48.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:48.055 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T07:54:48.080 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[49038]: fuse finished with error 0 and tester_r 0 2024-09-19T07:54:48.082 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:54:48.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:48.082 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T07:54:48.148 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:54:48.148 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:54:48.148 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:54:48.165 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:54:48.168 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:54:48.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:48.168 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T07:54:48.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:54:48.223 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T07:54:48.223 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:54:48.223 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:54:48.223 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:54:48.239 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:54:48.239 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:54:48.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:54:48.239 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T07:54:48.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:54:48.296 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T07:54:48.296 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:54:48.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.398+0000 7f32baaed640 1 Processor -- start 2024-09-19T07:54:48.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.398+0000 7f32baaed640 1 -- start start 2024-09-19T07:54:48.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.399+0000 7f32baaed640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32b415e2a0 0x7f32b406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:48.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.399+0000 7f32baaed640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f32b415e680 0x7f32b40cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:48.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.399+0000 7f32baaed640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32b40d1880 con 0x7f32b406d960 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.399+0000 7f32baaed640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f32b40d1b60 con 0x7f32b415e2a0 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.399+0000 7f32baaed640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f32b40cf2d0 con 0x7f32b415e680 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.399+0000 7f32b908a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f32b406d960 0x7f32b40d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47830/0 (socket says 172.21.15.16:47830) 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.399+0000 7f32b8889640 1 --2- 172.21.15.16:0/786376629 >> v2:172.21.15.16:3300/0 conn(0x7f32b415e2a0 0x7f32b406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.399+0000 7f32b908a640 1 -- 172.21.15.16:0/786376629 learned_addr learned my addr 172.21.15.16:0/786376629 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.399+0000 7f32b3fff640 1 --2- 172.21.15.16:0/786376629 >> v2:172.21.15.146:3300/0 conn(0x7f32b415e680 0x7f32b40cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.400+0000 7f32b8889640 1 -- 172.21.15.16:0/786376629 >> v2:172.21.15.146:3300/0 conn(0x7f32b415e680 msgr2=0x7f32b40cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.400+0000 7f32b8889640 1 --2- 172.21.15.16:0/786376629 >> v2:172.21.15.146:3300/0 conn(0x7f32b415e680 0x7f32b40cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.400+0000 7f32b8889640 1 -- 172.21.15.16:0/786376629 >> v1:172.21.15.16:6789/0 conn(0x7f32b406d960 legacy=0x7f32b40d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.400+0000 7f32b8889640 1 -- 172.21.15.16:0/786376629 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b40cf5b0 con 0x7f32b415e2a0 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.400+0000 7f32b3fff640 1 --2- 172.21.15.16:0/786376629 >> v2:172.21.15.146:3300/0 conn(0x7f32b415e680 0x7f32b40cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.400+0000 7f32b8889640 1 --2- 172.21.15.16:0/786376629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b415e2a0 0x7f32b406d420 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f32b40a22a0 tx=0x7f32a402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6950be26c2d0f09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.401+0000 7f32b37fe640 1 -- 172.21.15.16:0/786376629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4037070 con 0x7f32b415e2a0 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.401+0000 7f32b37fe640 1 -- 172.21.15.16:0/786376629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a4032440 con 0x7f32b415e2a0 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.401+0000 7f32b37fe640 1 -- 172.21.15.16:0/786376629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4032810 con 0x7f32b415e2a0 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.401+0000 7f32baaed640 1 -- 172.21.15.16:0/786376629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b415e2a0 msgr2=0x7f32b406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.401+0000 7f32baaed640 1 --2- 172.21.15.16:0/786376629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b415e2a0 0x7f32b406d420 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f32b40a22a0 tx=0x7f32a402a1a0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.401+0000 7f32baaed640 1 -- 172.21.15.16:0/786376629 shutdown_connections 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.401+0000 7f32baaed640 1 --2- 172.21.15.16:0/786376629 >> v2:172.21.15.146:3300/0 conn(0x7f32b415e680 0x7f32b40cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.401+0000 7f32baaed640 1 --2- 172.21.15.16:0/786376629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b415e2a0 0x7f32b406d420 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.401+0000 7f32baaed640 1 -- 172.21.15.16:0/786376629 >> 172.21.15.16:0/786376629 conn(0x7f32b409d760 msgr2=0x7f32b408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.401+0000 7f32baaed640 1 -- 172.21.15.16:0/786376629 shutdown_connections 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.401+0000 7f32baaed640 1 -- 172.21.15.16:0/786376629 wait complete. 2024-09-19T07:54:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.402+0000 7f32baaed640 1 Processor -- start 2024-09-19T07:54:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.402+0000 7f32baaed640 1 -- start start 2024-09-19T07:54:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.403+0000 7f32baaed640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b406d960 0x7f32b415c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.403+0000 7f32baaed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b415e2a0 0x7f32b40a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.403+0000 7f32baaed640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32b415e680 0x7f32b4160e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.403+0000 7f32baaed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32b40d4c00 con 0x7f32b415e2a0 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.403+0000 7f32baaed640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f32b40d4a80 con 0x7f32b415e680 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.403+0000 7f32baaed640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f32b40d0010 con 0x7f32b406d960 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.403+0000 7f32b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b415e2a0 0x7f32b40a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.403+0000 7f32b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b415e2a0 0x7f32b40a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56664/0 (socket says 172.21.15.16:56664) 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.403+0000 7f32b3fff640 1 -- 172.21.15.16:0/1692451786 learned_addr learned my addr 172.21.15.16:0/1692451786 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.404+0000 7f32b908a640 1 --2- 172.21.15.16:0/1692451786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32b415e680 0x7f32b4160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.404+0000 7f32b8889640 1 --2- 172.21.15.16:0/1692451786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b406d960 0x7f32b415c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.404+0000 7f32b3fff640 1 -- 172.21.15.16:0/1692451786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b406d960 msgr2=0x7f32b415c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.404+0000 7f32b3fff640 1 --2- 172.21.15.16:0/1692451786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b406d960 0x7f32b415c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.404+0000 7f32b3fff640 1 -- 172.21.15.16:0/1692451786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32b415e680 msgr2=0x7f32b4160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.404+0000 7f32b3fff640 1 --2- 172.21.15.16:0/1692451786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32b415e680 0x7f32b4160e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.404+0000 7f32b3fff640 1 -- 172.21.15.16:0/1692451786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b40a10b0 con 0x7f32b415e2a0 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.404+0000 7f32b8889640 1 --2- 172.21.15.16:0/1692451786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b406d960 0x7f32b415c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.404+0000 7f32b3fff640 1 --2- 172.21.15.16:0/1692451786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b415e2a0 0x7f32b40a0600 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f32a800d800 tx=0x7f32a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.404+0000 7f32b17fa640 1 -- 172.21.15.16:0/1692451786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8004030 con 0x7f32b415e2a0 2024-09-19T07:54:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.404+0000 7f32b17fa640 1 -- 172.21.15.16:0/1692451786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a80041d0 con 0x7f32b415e2a0 2024-09-19T07:54:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.404+0000 7f32b17fa640 1 -- 172.21.15.16:0/1692451786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a800b6c0 con 0x7f32b415e2a0 2024-09-19T07:54:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.404+0000 7f32baaed640 1 -- 172.21.15.16:0/1692451786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b40a1390 con 0x7f32b415e2a0 2024-09-19T07:54:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.404+0000 7f32baaed640 1 -- 172.21.15.16:0/1692451786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b40a1950 con 0x7f32b415e2a0 2024-09-19T07:54:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.405+0000 7f32baaed640 1 -- 172.21.15.16:0/1692451786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b40b8390 con 0x7f32b415e2a0 2024-09-19T07:54:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.405+0000 7f32b17fa640 1 -- 172.21.15.16:0/1692451786 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f32a800b860 con 0x7f32b415e2a0 2024-09-19T07:54:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.406+0000 7f32b17fa640 1 --2- 172.21.15.16:0/1692451786 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f32a006c2b0 0x7f32a006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.406+0000 7f32b17fa640 1 -- 172.21.15.16:0/1692451786 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7f32a808b840 con 0x7f32b415e2a0 2024-09-19T07:54:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.406+0000 7f32b8889640 1 --2- 172.21.15.16:0/1692451786 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f32a006c2b0 0x7f32a006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.407+0000 7f32b8889640 1 --2- 172.21.15.16:0/1692451786 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f32a006c2b0 0x7f32a006e770 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f32a4034000 tx=0x7f32a402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.410+0000 7f32b17fa640 1 -- 172.21.15.16:0/1692451786 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f32a8059e20 con 0x7f32b415e2a0 2024-09-19T07:54:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.622+0000 7f32baaed640 1 -- 172.21.15.16:0/1692451786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32b406d420 con 0x7f32b415e2a0 2024-09-19T07:54:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.622+0000 7f32b17fa640 1 -- 172.21.15.16:0/1692451786 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5219 (secure 0 0 0) 0x7f32a8059720 con 0x7f32b415e2a0 2024-09-19T07:54:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:48.623 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":74,"btime":"2024-09-19T07:54:44:229158+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/2994096322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2994096322},{"type":"v1","addr":"172.21.15.146:6836","nonce":2994096322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7839,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/393564462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":393564462},{"type":"v1","addr":"172.21.15.16:6833","nonce":393564462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":7842,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/631392582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":631392582},{"type":"v1","addr":"172.21.15.16:6835","nonce":631392582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:54:22.949504+0000","modified":"2024-09-19T07:54:43.595934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6837/79304323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":79304323},{"type":"v1","addr":"172.21.15.146:6837","nonce":79304323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7854,"qdb_cluster":[7854]},"id":9}]} 2024-09-19T07:54:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 74 2024-09-19T07:54:48.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.625+0000 7f32baaed640 1 -- 172.21.15.16:0/1692451786 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f32a006c2b0 msgr2=0x7f32a006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:48.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.625+0000 7f32baaed640 1 --2- 172.21.15.16:0/1692451786 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f32a006c2b0 0x7f32a006e770 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f32a4034000 tx=0x7f32a402e000 comp rx=0 tx=0).stop 2024-09-19T07:54:48.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.625+0000 7f32baaed640 1 -- 172.21.15.16:0/1692451786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b415e2a0 msgr2=0x7f32b40a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:48.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.625+0000 7f32baaed640 1 --2- 172.21.15.16:0/1692451786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b415e2a0 0x7f32b40a0600 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f32a800d800 tx=0x7f32a800dcc0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.625+0000 7f32baaed640 1 -- 172.21.15.16:0/1692451786 shutdown_connections 2024-09-19T07:54:48.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.625+0000 7f32baaed640 1 --2- 172.21.15.16:0/1692451786 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f32a006c2b0 0x7f32a006e770 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.625+0000 7f32baaed640 1 --2- 172.21.15.16:0/1692451786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32b415e680 0x7f32b4160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.625+0000 7f32baaed640 1 --2- 172.21.15.16:0/1692451786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b415e2a0 0x7f32b40a0600 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.625+0000 7f32baaed640 1 --2- 172.21.15.16:0/1692451786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b406d960 0x7f32b415c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.625+0000 7f32baaed640 1 -- 172.21.15.16:0/1692451786 >> 172.21.15.16:0/1692451786 conn(0x7f32b409d760 msgr2=0x7f32b40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:48.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.625+0000 7f32baaed640 1 -- 172.21.15.16:0/1692451786 shutdown_connections 2024-09-19T07:54:48.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.625+0000 7f32baaed640 1 -- 172.21.15.16:0/1692451786 wait complete. 2024-09-19T07:54:48.636 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:54:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.738+0000 7fdd49aac640 1 Processor -- start 2024-09-19T07:54:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.738+0000 7fdd49aac640 1 -- start start 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.739+0000 7fdd49aac640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdd4415e2a0 0x7fdd4406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.739+0000 7fdd49aac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdd4415e680 0x7fdd440cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.739+0000 7fdd49aac640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd4406d960 0x7fdd440d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.739+0000 7fdd49aac640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fdd440d1880 con 0x7fdd4415e680 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.739+0000 7fdd49aac640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fdd440d1b60 con 0x7fdd4415e2a0 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.739+0000 7fdd49aac640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdd440cf2d0 con 0x7fdd4406d960 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.739+0000 7fdd42ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdd4415e680 0x7fdd440cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.739+0000 7fdd42ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdd4415e680 0x7fdd440cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56680/0 (socket says 172.21.15.16:56680) 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.739+0000 7fdd42ffd640 1 -- 172.21.15.16:0/435121370 learned_addr learned my addr 172.21.15.16:0/435121370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.739+0000 7fdd437fe640 1 --2- 172.21.15.16:0/435121370 >> v2:172.21.15.146:3300/0 conn(0x7fdd4415e2a0 0x7fdd4406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.739+0000 7fdd43fff640 1 --2- 172.21.15.16:0/435121370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd4406d960 0x7fdd440d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.739+0000 7fdd42ffd640 1 -- 172.21.15.16:0/435121370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd4406d960 msgr2=0x7fdd440d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.739+0000 7fdd42ffd640 1 --2- 172.21.15.16:0/435121370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd4406d960 0x7fdd440d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.739+0000 7fdd42ffd640 1 -- 172.21.15.16:0/435121370 >> v2:172.21.15.146:3300/0 conn(0x7fdd4415e2a0 msgr2=0x7fdd4406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.739+0000 7fdd42ffd640 1 --2- 172.21.15.16:0/435121370 >> v2:172.21.15.146:3300/0 conn(0x7fdd4415e2a0 0x7fdd4406d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.739+0000 7fdd42ffd640 1 -- 172.21.15.16:0/435121370 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd440cf5b0 con 0x7fdd4415e680 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.739+0000 7fdd43fff640 1 --2- 172.21.15.16:0/435121370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd4406d960 0x7fdd440d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.740+0000 7fdd42ffd640 1 --2- 172.21.15.16:0/435121370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4415e680 0x7fdd440cfad0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fdd440b6fa0 tx=0x7fdd3802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1000c1fd837d1dd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:48.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.740+0000 7fdd427fc640 1 -- 172.21.15.16:0/435121370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38037070 con 0x7fdd4415e680 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.740+0000 7fdd427fc640 1 -- 172.21.15.16:0/435121370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd38032440 con 0x7fdd4415e680 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.740+0000 7fdd427fc640 1 -- 172.21.15.16:0/435121370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38032810 con 0x7fdd4415e680 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.741+0000 7fdd49aac640 1 -- 172.21.15.16:0/435121370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4415e680 msgr2=0x7fdd440cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.741+0000 7fdd49aac640 1 --2- 172.21.15.16:0/435121370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4415e680 0x7fdd440cfad0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fdd440b6fa0 tx=0x7fdd3802a1e0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.741+0000 7fdd49aac640 1 -- 172.21.15.16:0/435121370 shutdown_connections 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.741+0000 7fdd49aac640 1 --2- 172.21.15.16:0/435121370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd4406d960 0x7fdd440d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.741+0000 7fdd49aac640 1 --2- 172.21.15.16:0/435121370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4415e680 0x7fdd440cfad0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.741+0000 7fdd49aac640 1 --2- 172.21.15.16:0/435121370 >> v2:172.21.15.146:3300/0 conn(0x7fdd4415e2a0 0x7fdd4406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.741+0000 7fdd49aac640 1 -- 172.21.15.16:0/435121370 >> 172.21.15.16:0/435121370 conn(0x7fdd4409d760 msgr2=0x7fdd4408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.741+0000 7fdd49aac640 1 -- 172.21.15.16:0/435121370 shutdown_connections 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.741+0000 7fdd49aac640 1 -- 172.21.15.16:0/435121370 wait complete. 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.742+0000 7fdd49aac640 1 Processor -- start 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.742+0000 7fdd49aac640 1 -- start start 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.743+0000 7fdd49aac640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd4406d960 0x7fdd44146620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.743+0000 7fdd49aac640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd4415e2a0 0x7fdd440ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.743+0000 7fdd49aac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4415e680 0x7fdd440b9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.743+0000 7fdd49aac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd440d44f0 con 0x7fdd4415e680 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.743+0000 7fdd49aac640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdd440d4370 con 0x7fdd4415e2a0 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.743+0000 7fdd49aac640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdd440d4670 con 0x7fdd4406d960 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.743+0000 7fdd43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4415e680 0x7fdd440b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.743+0000 7fdd43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4415e680 0x7fdd440b9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56696/0 (socket says 172.21.15.16:56696) 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.743+0000 7fdd43fff640 1 -- 172.21.15.16:0/2613893356 learned_addr learned my addr 172.21.15.16:0/2613893356 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.743+0000 7fdd42ffd640 1 --2- 172.21.15.16:0/2613893356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd4415e2a0 0x7fdd440ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.743+0000 7fdd437fe640 1 --2- 172.21.15.16:0/2613893356 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd4406d960 0x7fdd44146620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.744+0000 7fdd43fff640 1 -- 172.21.15.16:0/2613893356 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd4406d960 msgr2=0x7fdd44146620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.744+0000 7fdd43fff640 1 --2- 172.21.15.16:0/2613893356 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd4406d960 0x7fdd44146620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.744+0000 7fdd43fff640 1 -- 172.21.15.16:0/2613893356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd4415e2a0 msgr2=0x7fdd440ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.744+0000 7fdd43fff640 1 --2- 172.21.15.16:0/2613893356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd4415e2a0 0x7fdd440ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.744+0000 7fdd43fff640 1 -- 172.21.15.16:0/2613893356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd44146e00 con 0x7fdd4415e680 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.744+0000 7fdd43fff640 1 --2- 172.21.15.16:0/2613893356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4415e680 0x7fdd440b9780 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009fa0 tx=0x7fdd3c00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.744+0000 7fdd2bfff640 1 -- 172.21.15.16:0/2613893356 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c01a070 con 0x7fdd4415e680 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.744+0000 7fdd2bfff640 1 -- 172.21.15.16:0/2613893356 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd3c003bb0 con 0x7fdd4415e680 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.744+0000 7fdd49aac640 1 -- 172.21.15.16:0/2613893356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd44146b60 con 0x7fdd4415e680 2024-09-19T07:54:48.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.744+0000 7fdd2bfff640 1 -- 172.21.15.16:0/2613893356 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c004030 con 0x7fdd4415e680 2024-09-19T07:54:48.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.744+0000 7fdd49aac640 1 -- 172.21.15.16:0/2613893356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd441470a0 con 0x7fdd4415e680 2024-09-19T07:54:48.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.745+0000 7fdd49aac640 1 -- 172.21.15.16:0/2613893356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd4406d420 con 0x7fdd4415e680 2024-09-19T07:54:48.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.745+0000 7fdd2bfff640 1 -- 172.21.15.16:0/2613893356 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdd3c01d070 con 0x7fdd4415e680 2024-09-19T07:54:48.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.746+0000 7fdd2bfff640 1 --2- 172.21.15.16:0/2613893356 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdd3006c2b0 0x7fdd3006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:48.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.746+0000 7fdd2bfff640 1 -- 172.21.15.16:0/2613893356 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7fdd3c08b900 con 0x7fdd4415e680 2024-09-19T07:54:48.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.746+0000 7fdd437fe640 1 --2- 172.21.15.16:0/2613893356 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdd3006c2b0 0x7fdd3006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:48.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.746+0000 7fdd437fe640 1 --2- 172.21.15.16:0/2613893356 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdd3006c2b0 0x7fdd3006e770 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009a50 tx=0x7fdd34008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:48.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.750+0000 7fdd2bfff640 1 -- 172.21.15.16:0/2613893356 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fdd3c010070 con 0x7fdd4415e680 2024-09-19T07:54:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.962+0000 7fdd49aac640 1 -- 172.21.15.16:0/2613893356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd4406cf90 con 0x7fdd4415e680 2024-09-19T07:54:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.962+0000 7fdd2bfff640 1 -- 172.21.15.16:0/2613893356 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5219 (secure 0 0 0) 0x7fdd3c059ee0 con 0x7fdd4415e680 2024-09-19T07:54:48.963 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:48.963 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":74,"btime":"2024-09-19T07:54:44:229158+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/2994096322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2994096322},{"type":"v1","addr":"172.21.15.146:6836","nonce":2994096322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7839,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/393564462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":393564462},{"type":"v1","addr":"172.21.15.16:6833","nonce":393564462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":7842,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/631392582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":631392582},{"type":"v1","addr":"172.21.15.16:6835","nonce":631392582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:54:22.949504+0000","modified":"2024-09-19T07:54:43.595934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6837/79304323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":79304323},{"type":"v1","addr":"172.21.15.146:6837","nonce":79304323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7854,"qdb_cluster":[7854]},"id":9}]} 2024-09-19T07:54:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 74 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.964+0000 7fdd49aac640 1 -- 172.21.15.16:0/2613893356 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdd3006c2b0 msgr2=0x7fdd3006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.964+0000 7fdd49aac640 1 --2- 172.21.15.16:0/2613893356 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdd3006c2b0 0x7fdd3006e770 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009a50 tx=0x7fdd34008040 comp rx=0 tx=0).stop 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.964+0000 7fdd49aac640 1 -- 172.21.15.16:0/2613893356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4415e680 msgr2=0x7fdd440b9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.964+0000 7fdd49aac640 1 --2- 172.21.15.16:0/2613893356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4415e680 0x7fdd440b9780 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009fa0 tx=0x7fdd3c00d560 comp rx=0 tx=0).stop 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.964+0000 7fdd49aac640 1 -- 172.21.15.16:0/2613893356 shutdown_connections 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.964+0000 7fdd49aac640 1 --2- 172.21.15.16:0/2613893356 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdd3006c2b0 0x7fdd3006e770 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.964+0000 7fdd49aac640 1 --2- 172.21.15.16:0/2613893356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd4415e680 0x7fdd440b9780 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.964+0000 7fdd49aac640 1 --2- 172.21.15.16:0/2613893356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd4415e2a0 0x7fdd440ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.964+0000 7fdd49aac640 1 --2- 172.21.15.16:0/2613893356 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd4406d960 0x7fdd44146620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.964+0000 7fdd49aac640 1 -- 172.21.15.16:0/2613893356 >> 172.21.15.16:0/2613893356 conn(0x7fdd4409d760 msgr2=0x7fdd4415f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.964+0000 7fdd49aac640 1 -- 172.21.15.16:0/2613893356 shutdown_connections 2024-09-19T07:54:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:48.964+0000 7fdd49aac640 1 -- 172.21.15.16:0/2613893356 wait complete. 2024-09-19T07:54:48.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:54:49.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.076+0000 7fe239cba640 1 Processor -- start 2024-09-19T07:54:49.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.077+0000 7fe239cba640 1 -- start start 2024-09-19T07:54:49.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.077+0000 7fe239cba640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe23415b120 0x7fe23406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.077+0000 7fe239cba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe23406dec0 0x7fe23406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.077+0000 7fe239cba640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe23415b500 con 0x7fe2340d04b0 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.077+0000 7fe239cba640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fe23406e940 con 0x7fe23406dec0 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.077+0000 7fe239cba640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe23406ec20 con 0x7fe23415b120 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.078+0000 7fe232ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe23406dec0 0x7fe23406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.078+0000 7fe233fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe2340d04b0 0x7fe2340c5160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47846/0 (socket says 172.21.15.16:47846) 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.078+0000 7fe233fff640 1 -- 172.21.15.16:0/2476073507 learned_addr learned my addr 172.21.15.16:0/2476073507 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.078+0000 7fe2337fe640 1 --2- 172.21.15.16:0/2476073507 >> v2:172.21.15.146:3300/0 conn(0x7fe23415b120 0x7fe23406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.078+0000 7fe232ffd640 1 -- 172.21.15.16:0/2476073507 >> v2:172.21.15.146:3300/0 conn(0x7fe23415b120 msgr2=0x7fe23406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.078+0000 7fe232ffd640 1 --2- 172.21.15.16:0/2476073507 >> v2:172.21.15.146:3300/0 conn(0x7fe23415b120 0x7fe23406f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.078+0000 7fe232ffd640 1 -- 172.21.15.16:0/2476073507 >> v1:172.21.15.16:6789/0 conn(0x7fe2340d04b0 legacy=0x7fe2340c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.078+0000 7fe232ffd640 1 -- 172.21.15.16:0/2476073507 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23406ef00 con 0x7fe23406dec0 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.078+0000 7fe2327fc640 1 -- 172.21.15.16:0/2476073507 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2015481327 0 0) 0x7fe23415b500 con 0x7fe2340d04b0 2024-09-19T07:54:49.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.079+0000 7fe232ffd640 1 --2- 172.21.15.16:0/2476073507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe23406dec0 0x7fe23406e330 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fe23406cb50 tx=0x7fe21c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c83c30f4ae7c04b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:49.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.079+0000 7fe2327fc640 1 -- 172.21.15.16:0/2476073507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c037070 con 0x7fe23406dec0 2024-09-19T07:54:49.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.079+0000 7fe2327fc640 1 -- 172.21.15.16:0/2476073507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe21c032440 con 0x7fe23406dec0 2024-09-19T07:54:49.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.079+0000 7fe2327fc640 1 -- 172.21.15.16:0/2476073507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c032810 con 0x7fe23406dec0 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.079+0000 7fe239cba640 1 -- 172.21.15.16:0/2476073507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe23406dec0 msgr2=0x7fe23406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.079+0000 7fe239cba640 1 --2- 172.21.15.16:0/2476073507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe23406dec0 0x7fe23406e330 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fe23406cb50 tx=0x7fe21c02a1a0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.079+0000 7fe239cba640 1 -- 172.21.15.16:0/2476073507 shutdown_connections 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.079+0000 7fe239cba640 1 --2- 172.21.15.16:0/2476073507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe23406dec0 0x7fe23406e330 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.079+0000 7fe239cba640 1 --2- 172.21.15.16:0/2476073507 >> v2:172.21.15.146:3300/0 conn(0x7fe23415b120 0x7fe23406f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.079+0000 7fe239cba640 1 -- 172.21.15.16:0/2476073507 >> 172.21.15.16:0/2476073507 conn(0x7fe23406bdf0 msgr2=0x7fe23406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.079+0000 7fe239cba640 1 -- 172.21.15.16:0/2476073507 shutdown_connections 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.079+0000 7fe239cba640 1 -- 172.21.15.16:0/2476073507 wait complete. 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.080+0000 7fe239cba640 1 Processor -- start 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.081+0000 7fe239cba640 1 -- start start 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.081+0000 7fe239cba640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe23406dec0 0x7fe234071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.081+0000 7fe239cba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2340d04b0 0x7fe2340afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.081+0000 7fe239cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe23415b120 0x7fe2340b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.081+0000 7fe239cba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2340ce060 con 0x7fe23415b120 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.081+0000 7fe239cba640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe23406f4c0 con 0x7fe23406dec0 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.081+0000 7fe239cba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe2340d0900 con 0x7fe2340d04b0 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.082+0000 7fe233fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe23415b120 0x7fe2340b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.082+0000 7fe233fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe23415b120 0x7fe2340b25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56710/0 (socket says 172.21.15.16:56710) 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.082+0000 7fe233fff640 1 -- 172.21.15.16:0/3749686005 learned_addr learned my addr 172.21.15.16:0/3749686005 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.082+0000 7fe232ffd640 1 --2- 172.21.15.16:0/3749686005 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2340d04b0 0x7fe2340afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.082+0000 7fe2337fe640 1 --2- 172.21.15.16:0/3749686005 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe23406dec0 0x7fe234071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.082+0000 7fe233fff640 1 -- 172.21.15.16:0/3749686005 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2340d04b0 msgr2=0x7fe2340afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.082+0000 7fe233fff640 1 --2- 172.21.15.16:0/3749686005 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2340d04b0 0x7fe2340afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.082+0000 7fe233fff640 1 -- 172.21.15.16:0/3749686005 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe23406dec0 msgr2=0x7fe234071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.082+0000 7fe233fff640 1 --2- 172.21.15.16:0/3749686005 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe23406dec0 0x7fe234071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.082+0000 7fe233fff640 1 -- 172.21.15.16:0/3749686005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23414bea0 con 0x7fe23415b120 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.082+0000 7fe233fff640 1 --2- 172.21.15.16:0/3749686005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe23415b120 0x7fe2340b25c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00c7d0 tx=0x7fe22c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.082+0000 7fe22bfff640 1 -- 172.21.15.16:0/3749686005 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c018070 con 0x7fe23415b120 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.082+0000 7fe22bfff640 1 -- 172.21.15.16:0/3749686005 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe22c00a640 con 0x7fe23415b120 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.082+0000 7fe239cba640 1 -- 172.21.15.16:0/3749686005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe234072380 con 0x7fe23415b120 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.082+0000 7fe22bfff640 1 -- 172.21.15.16:0/3749686005 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c00aa70 con 0x7fe23415b120 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.082+0000 7fe239cba640 1 -- 172.21.15.16:0/3749686005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2340b0bd0 con 0x7fe23415b120 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.083+0000 7fe239cba640 1 -- 172.21.15.16:0/3749686005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2340d0cd0 con 0x7fe23415b120 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.084+0000 7fe22bfff640 1 -- 172.21.15.16:0/3749686005 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe22c01b070 con 0x7fe23415b120 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.084+0000 7fe22bfff640 1 --2- 172.21.15.16:0/3749686005 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe21406c2b0 0x7fe21406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.084+0000 7fe22bfff640 1 -- 172.21.15.16:0/3749686005 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7fe22c08ac00 con 0x7fe23415b120 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.084+0000 7fe2337fe640 1 --2- 172.21.15.16:0/3749686005 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe21406c2b0 0x7fe21406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.085+0000 7fe2337fe640 1 --2- 172.21.15.16:0/3749686005 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe21406c2b0 0x7fe21406e770 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe234159300 tx=0x7fe224008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:49.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.089+0000 7fe22bfff640 1 -- 172.21.15.16:0/3749686005 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe22c0591d0 con 0x7fe23415b120 2024-09-19T07:54:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.267+0000 7fe239cba640 1 -- 172.21.15.16:0/3749686005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2340cbd60 con 0x7fe23415b120 2024-09-19T07:54:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.268+0000 7fe22bfff640 1 -- 172.21.15.16:0/3749686005 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+16984 (secure 0 0 0) 0x7fe22c058ad0 con 0x7fe23415b120 2024-09-19T07:54:49.269 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:49.269 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":129,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:54:24.558589+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:54:20.571485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-09-19T07:54:21.899096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:54:49.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.270+0000 7fe239cba640 1 -- 172.21.15.16:0/3749686005 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe21406c2b0 msgr2=0x7fe21406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.270+0000 7fe239cba640 1 --2- 172.21.15.16:0/3749686005 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe21406c2b0 0x7fe21406e770 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe234159300 tx=0x7fe224008040 comp rx=0 tx=0).stop 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.270+0000 7fe239cba640 1 -- 172.21.15.16:0/3749686005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe23415b120 msgr2=0x7fe2340b25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.270+0000 7fe239cba640 1 --2- 172.21.15.16:0/3749686005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe23415b120 0x7fe2340b25c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00c7d0 tx=0x7fe22c00cc90 comp rx=0 tx=0).stop 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.270+0000 7fe239cba640 1 -- 172.21.15.16:0/3749686005 shutdown_connections 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.270+0000 7fe239cba640 1 --2- 172.21.15.16:0/3749686005 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe21406c2b0 0x7fe21406e770 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.270+0000 7fe239cba640 1 --2- 172.21.15.16:0/3749686005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe23415b120 0x7fe2340b25c0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.270+0000 7fe239cba640 1 --2- 172.21.15.16:0/3749686005 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2340d04b0 0x7fe2340afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.270+0000 7fe239cba640 1 --2- 172.21.15.16:0/3749686005 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe23406dec0 0x7fe234071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.270+0000 7fe239cba640 1 -- 172.21.15.16:0/3749686005 >> 172.21.15.16:0/3749686005 conn(0x7fe23406bdf0 msgr2=0x7fe2340ca900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.270+0000 7fe239cba640 1 -- 172.21.15.16:0/3749686005 shutdown_connections 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.270+0000 7fe239cba640 1 -- 172.21.15.16:0/3749686005 wait complete. 2024-09-19T07:54:49.281 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T07:54:49.282 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:54:49.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.383+0000 7ff115253640 1 Processor -- start 2024-09-19T07:54:49.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.383+0000 7ff115253640 1 -- start start 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.384+0000 7ff115253640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff11006d430 0x7ff1101439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.384+0000 7ff115253640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff110143f30 0x7ff1100d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.384+0000 7ff115253640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1100c7140 con 0x7ff1100c8d20 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.384+0000 7ff115253640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff110141960 con 0x7ff110143f30 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.384+0000 7ff115253640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff110142200 con 0x7ff11006d430 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.384+0000 7ff10f59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff110143f30 0x7ff1100d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.384+0000 7ff10f59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff110143f30 0x7ff1100d1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56724/0 (socket says 172.21.15.16:56724) 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.384+0000 7ff10f59e640 1 -- 172.21.15.16:0/4246923575 learned_addr learned my addr 172.21.15.16:0/4246923575 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.384+0000 7ff10f59e640 1 -- 172.21.15.16:0/4246923575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff11006d430 msgr2=0x7ff1101439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.384+0000 7ff10f59e640 1 --2- 172.21.15.16:0/4246923575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff11006d430 0x7ff1101439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.384+0000 7ff10f59e640 1 -- 172.21.15.16:0/4246923575 >> v1:172.21.15.146:6789/0 conn(0x7ff1100c8d20 legacy=0x7ff11006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:49.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.384+0000 7ff10f59e640 1 -- 172.21.15.16:0/4246923575 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1100c9100 con 0x7ff110143f30 2024-09-19T07:54:49.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.385+0000 7ff10f59e640 1 --2- 172.21.15.16:0/4246923575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff110143f30 0x7ff1100d1e50 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7ff100009e50 tx=0x7ff10002a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a4694db96bcef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:49.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.385+0000 7ff10dd9b640 1 -- 172.21.15.16:0/4246923575 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100036070 con 0x7ff110143f30 2024-09-19T07:54:49.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.385+0000 7ff10dd9b640 1 -- 172.21.15.16:0/4246923575 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff100031600 con 0x7ff110143f30 2024-09-19T07:54:49.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.385+0000 7ff10dd9b640 1 -- 172.21.15.16:0/4246923575 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1000319d0 con 0x7ff110143f30 2024-09-19T07:54:49.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.386+0000 7ff115253640 1 -- 172.21.15.16:0/4246923575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff110143f30 msgr2=0x7ff1100d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.386+0000 7ff115253640 1 --2- 172.21.15.16:0/4246923575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff110143f30 0x7ff1100d1e50 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7ff100009e50 tx=0x7ff10002a5b0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.386+0000 7ff115253640 1 -- 172.21.15.16:0/4246923575 shutdown_connections 2024-09-19T07:54:49.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.386+0000 7ff115253640 1 --2- 172.21.15.16:0/4246923575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff110143f30 0x7ff1100d1e50 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.386+0000 7ff115253640 1 --2- 172.21.15.16:0/4246923575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff11006d430 0x7ff1101439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.386+0000 7ff115253640 1 -- 172.21.15.16:0/4246923575 >> 172.21.15.16:0/4246923575 conn(0x7ff1100b3ed0 msgr2=0x7ff1100b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:49.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.386+0000 7ff115253640 1 -- 172.21.15.16:0/4246923575 shutdown_connections 2024-09-19T07:54:49.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.386+0000 7ff115253640 1 -- 172.21.15.16:0/4246923575 wait complete. 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.387+0000 7ff115253640 1 Processor -- start 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.387+0000 7ff115253640 1 -- start start 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.388+0000 7ff115253640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff11006d430 0x7ff1100b7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.388+0000 7ff115253640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1100c8d20 0x7ff1100b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.388+0000 7ff115253640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff110143f30 0x7ff1100b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.388+0000 7ff115253640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff110141e90 con 0x7ff110143f30 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.388+0000 7ff115253640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff110141bb0 con 0x7ff11006d430 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.388+0000 7ff115253640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff110144380 con 0x7ff1100c8d20 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.388+0000 7ff10f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff110143f30 0x7ff1100b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.388+0000 7ff10f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff110143f30 0x7ff1100b2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56728/0 (socket says 172.21.15.16:56728) 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.388+0000 7ff10f59e640 1 -- 172.21.15.16:0/4110066418 learned_addr learned my addr 172.21.15.16:0/4110066418 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.388+0000 7ff10ed9d640 1 --2- 172.21.15.16:0/4110066418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff11006d430 0x7ff1100b7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.388+0000 7ff10e59c640 1 --2- 172.21.15.16:0/4110066418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1100c8d20 0x7ff1100b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.389+0000 7ff10f59e640 1 -- 172.21.15.16:0/4110066418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1100c8d20 msgr2=0x7ff1100b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.389+0000 7ff10f59e640 1 --2- 172.21.15.16:0/4110066418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1100c8d20 0x7ff1100b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.389+0000 7ff10f59e640 1 -- 172.21.15.16:0/4110066418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff11006d430 msgr2=0x7ff1100b7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.389+0000 7ff10f59e640 1 --2- 172.21.15.16:0/4110066418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff11006d430 0x7ff1100b7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.389+0000 7ff10f59e640 1 -- 172.21.15.16:0/4110066418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1100cd480 con 0x7ff110143f30 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.389+0000 7ff10f59e640 1 --2- 172.21.15.16:0/4110066418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff110143f30 0x7ff1100b2b10 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7ff100034040 tx=0x7ff100004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.389+0000 7ff1077fe640 1 -- 172.21.15.16:0/4110066418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100041020 con 0x7ff110143f30 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.389+0000 7ff1077fe640 1 -- 172.21.15.16:0/4110066418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff100007780 con 0x7ff110143f30 2024-09-19T07:54:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.389+0000 7ff115253640 1 -- 172.21.15.16:0/4110066418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1100cc8c0 con 0x7ff110143f30 2024-09-19T07:54:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.389+0000 7ff1077fe640 1 -- 172.21.15.16:0/4110066418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100036040 con 0x7ff110143f30 2024-09-19T07:54:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.389+0000 7ff115253640 1 -- 172.21.15.16:0/4110066418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1100cd760 con 0x7ff110143f30 2024-09-19T07:54:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.390+0000 7ff115253640 1 -- 172.21.15.16:0/4110066418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1101439f0 con 0x7ff110143f30 2024-09-19T07:54:49.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.391+0000 7ff1077fe640 1 -- 172.21.15.16:0/4110066418 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff100044020 con 0x7ff110143f30 2024-09-19T07:54:49.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.391+0000 7ff1077fe640 1 --2- 172.21.15.16:0/4110066418 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff0f006c2b0 0x7ff0f006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.391+0000 7ff1077fe640 1 -- 172.21.15.16:0/4110066418 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7ff1000ac280 con 0x7ff110143f30 2024-09-19T07:54:49.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.391+0000 7ff10ed9d640 1 --2- 172.21.15.16:0/4110066418 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff0f006c2b0 0x7ff0f006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:49.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.392+0000 7ff10ed9d640 1 --2- 172.21.15.16:0/4110066418 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff0f006c2b0 0x7ff0f006e770 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff108006fd0 tx=0x7ff108009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.395+0000 7ff1077fe640 1 -- 172.21.15.16:0/4110066418 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff10004b030 con 0x7ff110143f30 2024-09-19T07:54:49.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.606+0000 7ff115253640 1 -- 172.21.15.16:0/4110066418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1100690c0 con 0x7ff110143f30 2024-09-19T07:54:49.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.606+0000 7ff1077fe640 1 -- 172.21.15.16:0/4110066418 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5219 (secure 0 0 0) 0x7ff1100c7140 con 0x7ff110143f30 2024-09-19T07:54:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:49.607 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":74,"btime":"2024-09-19T07:54:44:229158+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/2994096322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2994096322},{"type":"v1","addr":"172.21.15.146:6836","nonce":2994096322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7839,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/393564462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":393564462},{"type":"v1","addr":"172.21.15.16:6833","nonce":393564462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":7842,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/631392582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":631392582},{"type":"v1","addr":"172.21.15.16:6835","nonce":631392582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:54:22.949504+0000","modified":"2024-09-19T07:54:43.595934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6837/79304323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":79304323},{"type":"v1","addr":"172.21.15.146:6837","nonce":79304323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7854,"qdb_cluster":[7854]},"id":9}]} 2024-09-19T07:54:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 74 2024-09-19T07:54:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.608+0000 7ff115253640 1 -- 172.21.15.16:0/4110066418 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff0f006c2b0 msgr2=0x7ff0f006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.608+0000 7ff115253640 1 --2- 172.21.15.16:0/4110066418 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff0f006c2b0 0x7ff0f006e770 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff108006fd0 tx=0x7ff108009060 comp rx=0 tx=0).stop 2024-09-19T07:54:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.608+0000 7ff115253640 1 -- 172.21.15.16:0/4110066418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff110143f30 msgr2=0x7ff1100b2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.608+0000 7ff115253640 1 --2- 172.21.15.16:0/4110066418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff110143f30 0x7ff1100b2b10 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7ff100034040 tx=0x7ff100004a40 comp rx=0 tx=0).stop 2024-09-19T07:54:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.608+0000 7ff115253640 1 -- 172.21.15.16:0/4110066418 shutdown_connections 2024-09-19T07:54:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.608+0000 7ff115253640 1 --2- 172.21.15.16:0/4110066418 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff0f006c2b0 0x7ff0f006e770 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.608+0000 7ff115253640 1 --2- 172.21.15.16:0/4110066418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff110143f30 0x7ff1100b2b10 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.608+0000 7ff115253640 1 --2- 172.21.15.16:0/4110066418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1100c8d20 0x7ff1100b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.609+0000 7ff115253640 1 --2- 172.21.15.16:0/4110066418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff11006d430 0x7ff1100b7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.609+0000 7ff115253640 1 -- 172.21.15.16:0/4110066418 >> 172.21.15.16:0/4110066418 conn(0x7ff1100b3ed0 msgr2=0x7ff110140840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.609+0000 7ff115253640 1 -- 172.21.15.16:0/4110066418 shutdown_connections 2024-09-19T07:54:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.609+0000 7ff115253640 1 -- 172.21.15.16:0/4110066418 wait complete. 2024-09-19T07:54:49.620 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:54:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.722+0000 7f4f37328640 1 Processor -- start 2024-09-19T07:54:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.722+0000 7f4f37328640 1 -- start start 2024-09-19T07:54:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.723+0000 7f4f37328640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4f3015e680 0x7f4f300cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.723+0000 7f4f37328640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f300d1880 con 0x7f4f3015e2a0 2024-09-19T07:54:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.723+0000 7f4f37328640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f300d1b60 con 0x7f4f300d0010 2024-09-19T07:54:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.723+0000 7f4f37328640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4f300cf2d0 con 0x7f4f3015e680 2024-09-19T07:54:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.723+0000 7f4f350c4640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4f3015e2a0 0x7f4f3006d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47862/0 (socket says 172.21.15.16:47862) 2024-09-19T07:54:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.723+0000 7f4f350c4640 1 -- 172.21.15.16:0/231216143 learned_addr learned my addr 172.21.15.16:0/231216143 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.723+0000 7f4f348c3640 1 --2- 172.21.15.16:0/231216143 >> v2:172.21.15.146:3300/0 conn(0x7f4f3015e680 0x7f4f300cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.723+0000 7f4f27fff640 1 -- 172.21.15.16:0/231216143 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 377598027 0 0) 0x7f4f300d1880 con 0x7f4f3015e2a0 2024-09-19T07:54:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.723+0000 7f4f27fff640 1 -- 172.21.15.16:0/231216143 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f1c003520 con 0x7f4f3015e2a0 2024-09-19T07:54:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f27fff640 1 -- 172.21.15.16:0/231216143 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3833884426 0 0) 0x7f4f1c003520 con 0x7f4f3015e2a0 2024-09-19T07:54:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f27fff640 1 -- 172.21.15.16:0/231216143 >> v2:172.21.15.146:3300/0 conn(0x7f4f3015e680 msgr2=0x7f4f300cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f27fff640 1 --2- 172.21.15.16:0/231216143 >> v2:172.21.15.146:3300/0 conn(0x7f4f3015e680 0x7f4f300cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f27fff640 1 -- 172.21.15.16:0/231216143 >> v1:172.21.15.146:6789/0 conn(0x7f4f300d0010 legacy=0x7f4f300d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f27fff640 1 -- 172.21.15.16:0/231216143 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f300cf5b0 con 0x7f4f3015e2a0 2024-09-19T07:54:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f27fff640 1 -- 172.21.15.16:0/231216143 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f4f2c001670 con 0x7f4f3015e2a0 2024-09-19T07:54:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f27fff640 1 -- 172.21.15.16:0/231216143 >> v1:172.21.15.16:6789/0 conn(0x7f4f3015e2a0 legacy=0x7f4f3006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f27fff640 1 --2- 172.21.15.16:0/231216143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f1c0039b0 0x7f4f1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f27fff640 1 --2- 172.21.15.16:0/231216143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f1c004800 0x7f4f1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f27fff640 1 --2- 172.21.15.16:0/231216143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f1c0051c0 0x7f4f1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f27fff640 1 -- 172.21.15.16:0/231216143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f300cf2d0 con 0x7f4f1c0039b0 2024-09-19T07:54:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f27fff640 1 -- 172.21.15.16:0/231216143 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c00c160 con 0x7f4f1c0051c0 2024-09-19T07:54:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f27fff640 1 -- 172.21.15.16:0/231216143 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4f1c00c360 con 0x7f4f1c004800 2024-09-19T07:54:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f350c4640 1 --2- 172.21.15.16:0/231216143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f1c0039b0 0x7f4f1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f358c5640 1 --2- 172.21.15.16:0/231216143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f1c0051c0 0x7f4f1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f348c3640 1 --2- 172.21.15.16:0/231216143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f1c004800 0x7f4f1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f350c4640 1 -- 172.21.15.16:0/231216143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f1c004800 msgr2=0x7f4f1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f350c4640 1 --2- 172.21.15.16:0/231216143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f1c004800 0x7f4f1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f350c4640 1 -- 172.21.15.16:0/231216143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f1c0051c0 msgr2=0x7f4f1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f350c4640 1 --2- 172.21.15.16:0/231216143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f1c0051c0 0x7f4f1c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f350c4640 1 -- 172.21.15.16:0/231216143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1c003520 con 0x7f4f1c0039b0 2024-09-19T07:54:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.724+0000 7f4f358c5640 1 -- 172.21.15.16:0/231216143 reap_dead start 2024-09-19T07:54:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.725+0000 7f4f350c4640 1 --2- 172.21.15.16:0/231216143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f1c0039b0 0x7f4f1c003dd0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00c210 tx=0x7f4f1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b7b17054e83c02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.725+0000 7f4f27fff640 1 -- 172.21.15.16:0/231216143 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c007d30 con 0x7f4f1c0039b0 2024-09-19T07:54:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.725+0000 7f4f27fff640 1 -- 172.21.15.16:0/231216143 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f2c005ad0 con 0x7f4f1c0039b0 2024-09-19T07:54:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.725+0000 7f4f27fff640 1 -- 172.21.15.16:0/231216143 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c007070 con 0x7f4f1c0039b0 2024-09-19T07:54:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.726+0000 7f4f37328640 1 -- 172.21.15.16:0/231216143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f1c0039b0 msgr2=0x7f4f1c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.726+0000 7f4f37328640 1 --2- 172.21.15.16:0/231216143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f1c0039b0 0x7f4f1c003dd0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00c210 tx=0x7f4f1c005830 comp rx=0 tx=0).stop 2024-09-19T07:54:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.726+0000 7f4f37328640 1 -- 172.21.15.16:0/231216143 shutdown_connections 2024-09-19T07:54:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.726+0000 7f4f37328640 1 --2- 172.21.15.16:0/231216143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f1c0039b0 0x7f4f1c003dd0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.726+0000 7f4f37328640 1 -- 172.21.15.16:0/231216143 >> 172.21.15.16:0/231216143 conn(0x7f4f3009d760 msgr2=0x7f4f3008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.726+0000 7f4f37328640 1 -- 172.21.15.16:0/231216143 shutdown_connections 2024-09-19T07:54:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.726+0000 7f4f37328640 1 -- 172.21.15.16:0/231216143 wait complete. 2024-09-19T07:54:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.727+0000 7f4f37328640 1 Processor -- start 2024-09-19T07:54:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.727+0000 7f4f37328640 1 -- start start 2024-09-19T07:54:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.728+0000 7f4f37328640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f1c0039b0 0x7f4f300c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.728+0000 7f4f37328640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f301f4990 0x7f4f301f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.728+0000 7f4f37328640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f300ccee0 0x7f4f300cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.728+0000 7f4f37328640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f300cfc50 con 0x7f4f300ccee0 2024-09-19T07:54:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.728+0000 7f4f37328640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4f300cfad0 con 0x7f4f301f4990 2024-09-19T07:54:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.728+0000 7f4f37328640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4f300cfdd0 con 0x7f4f1c0039b0 2024-09-19T07:54:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.728+0000 7f4f358c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f300ccee0 0x7f4f300cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.728+0000 7f4f358c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f300ccee0 0x7f4f300cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56736/0 (socket says 172.21.15.16:56736) 2024-09-19T07:54:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.728+0000 7f4f358c5640 1 -- 172.21.15.16:0/1491590482 learned_addr learned my addr 172.21.15.16:0/1491590482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.728+0000 7f4f348c3640 1 --2- 172.21.15.16:0/1491590482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f301f4990 0x7f4f301f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.728+0000 7f4f350c4640 1 --2- 172.21.15.16:0/1491590482 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f1c0039b0 0x7f4f300c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.728+0000 7f4f358c5640 1 -- 172.21.15.16:0/1491590482 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f1c0039b0 msgr2=0x7f4f300c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.728+0000 7f4f358c5640 1 --2- 172.21.15.16:0/1491590482 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f1c0039b0 0x7f4f300c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.728+0000 7f4f358c5640 1 -- 172.21.15.16:0/1491590482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f301f4990 msgr2=0x7f4f301f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.728+0000 7f4f358c5640 1 --2- 172.21.15.16:0/1491590482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f301f4990 0x7f4f301f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.728+0000 7f4f358c5640 1 -- 172.21.15.16:0/1491590482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f300cde70 con 0x7f4f300ccee0 2024-09-19T07:54:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.728+0000 7f4f358c5640 1 --2- 172.21.15.16:0/1491590482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f300ccee0 0x7f4f300cd360 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f4f3006cd80 tx=0x7f4f2800fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.729+0000 7f4f25ffb640 1 -- 172.21.15.16:0/1491590482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2801a070 con 0x7f4f300ccee0 2024-09-19T07:54:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.729+0000 7f4f25ffb640 1 -- 172.21.15.16:0/1491590482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f28006b50 con 0x7f4f300ccee0 2024-09-19T07:54:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.729+0000 7f4f37328640 1 -- 172.21.15.16:0/1491590482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f301f5350 con 0x7f4f300ccee0 2024-09-19T07:54:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.729+0000 7f4f25ffb640 1 -- 172.21.15.16:0/1491590482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28003970 con 0x7f4f300ccee0 2024-09-19T07:54:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.729+0000 7f4f37328640 1 -- 172.21.15.16:0/1491590482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f301f5bb0 con 0x7f4f300ccee0 2024-09-19T07:54:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.730+0000 7f4f37328640 1 -- 172.21.15.16:0/1491590482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f3006d420 con 0x7f4f300ccee0 2024-09-19T07:54:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.730+0000 7f4f25ffb640 1 -- 172.21.15.16:0/1491590482 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4f2801d070 con 0x7f4f300ccee0 2024-09-19T07:54:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.731+0000 7f4f25ffb640 1 --2- 172.21.15.16:0/1491590482 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4f1406c2b0 0x7f4f1406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.731+0000 7f4f25ffb640 1 -- 172.21.15.16:0/1491590482 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7f4f2808a970 con 0x7f4f300ccee0 2024-09-19T07:54:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.731+0000 7f4f350c4640 1 --2- 172.21.15.16:0/1491590482 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4f1406c2b0 0x7f4f1406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.731+0000 7f4f350c4640 1 --2- 172.21.15.16:0/1491590482 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4f1406c2b0 0x7f4f1406e770 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c033000 tx=0x7f4f2c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:49.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.735+0000 7f4f25ffb640 1 -- 172.21.15.16:0/1491590482 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4f301f5350 con 0x7f4f300ccee0 2024-09-19T07:54:49.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.946+0000 7f4f37328640 1 -- 172.21.15.16:0/1491590482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f3006cf90 con 0x7f4f300ccee0 2024-09-19T07:54:49.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.946+0000 7f4f25ffb640 1 -- 172.21.15.16:0/1491590482 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5219 (secure 0 0 0) 0x7f4f280588f0 con 0x7f4f300ccee0 2024-09-19T07:54:49.947 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:49.947 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":74,"btime":"2024-09-19T07:54:44:229158+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/2994096322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2994096322},{"type":"v1","addr":"172.21.15.146:6836","nonce":2994096322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7839,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/393564462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":393564462},{"type":"v1","addr":"172.21.15.16:6833","nonce":393564462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":7842,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/631392582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":631392582},{"type":"v1","addr":"172.21.15.16:6835","nonce":631392582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:54:22.949504+0000","modified":"2024-09-19T07:54:43.595934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6837/79304323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":79304323},{"type":"v1","addr":"172.21.15.146:6837","nonce":79304323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7854,"qdb_cluster":[7854]},"id":9}]} 2024-09-19T07:54:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 74 2024-09-19T07:54:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.948+0000 7f4f37328640 1 -- 172.21.15.16:0/1491590482 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4f1406c2b0 msgr2=0x7f4f1406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.948+0000 7f4f37328640 1 --2- 172.21.15.16:0/1491590482 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4f1406c2b0 0x7f4f1406e770 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c033000 tx=0x7f4f2c02e000 comp rx=0 tx=0).stop 2024-09-19T07:54:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.948+0000 7f4f37328640 1 -- 172.21.15.16:0/1491590482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f300ccee0 msgr2=0x7f4f300cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.948+0000 7f4f37328640 1 --2- 172.21.15.16:0/1491590482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f300ccee0 0x7f4f300cd360 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f4f3006cd80 tx=0x7f4f2800fb20 comp rx=0 tx=0).stop 2024-09-19T07:54:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.948+0000 7f4f37328640 1 -- 172.21.15.16:0/1491590482 shutdown_connections 2024-09-19T07:54:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.948+0000 7f4f37328640 1 --2- 172.21.15.16:0/1491590482 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4f1406c2b0 0x7f4f1406e770 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.948+0000 7f4f37328640 1 --2- 172.21.15.16:0/1491590482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f300ccee0 0x7f4f300cd360 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.948+0000 7f4f37328640 1 --2- 172.21.15.16:0/1491590482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f301f4990 0x7f4f301f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.948+0000 7f4f37328640 1 --2- 172.21.15.16:0/1491590482 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f1c0039b0 0x7f4f300c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.948+0000 7f4f37328640 1 -- 172.21.15.16:0/1491590482 >> 172.21.15.16:0/1491590482 conn(0x7f4f3009d760 msgr2=0x7f4f300d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.949+0000 7f4f37328640 1 -- 172.21.15.16:0/1491590482 shutdown_connections 2024-09-19T07:54:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:49.949+0000 7f4f37328640 1 -- 172.21.15.16:0/1491590482 wait complete. 2024-09-19T07:54:49.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:54:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.062+0000 7f6ada91a640 1 Processor -- start 2024-09-19T07:54:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.062+0000 7f6ada91a640 1 -- start start 2024-09-19T07:54:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.062+0000 7f6ada91a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6ad415e2a0 0x7f6ad406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.062+0000 7f6ada91a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6ad406d960 0x7f6ad40d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.062+0000 7f6ada91a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ad40d1880 con 0x7f6ad415e680 2024-09-19T07:54:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.063+0000 7f6ada91a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f6ad40d1b60 con 0x7f6ad415e2a0 2024-09-19T07:54:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.063+0000 7f6ada91a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f6ad40cf2d0 con 0x7f6ad406d960 2024-09-19T07:54:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.063+0000 7f6ad3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6ad415e2a0 0x7f6ad406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.063+0000 7f6ad37fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6ad415e680 0x7f6ad40cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47868/0 (socket says 172.21.15.16:47868) 2024-09-19T07:54:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.063+0000 7f6ad3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6ad415e2a0 0x7f6ad406d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56738/0 (socket says 172.21.15.16:56738) 2024-09-19T07:54:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.063+0000 7f6ad37fe640 1 -- 172.21.15.16:0/3635859142 learned_addr learned my addr 172.21.15.16:0/3635859142 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.063+0000 7f6ad8eb7640 1 --2- 172.21.15.16:0/3635859142 >> v2:172.21.15.146:3300/0 conn(0x7f6ad406d960 0x7f6ad40d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.063+0000 7f6ad3fff640 1 -- 172.21.15.16:0/3635859142 >> v2:172.21.15.146:3300/0 conn(0x7f6ad406d960 msgr2=0x7f6ad40d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.063+0000 7f6ad3fff640 1 --2- 172.21.15.16:0/3635859142 >> v2:172.21.15.146:3300/0 conn(0x7f6ad406d960 0x7f6ad40d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.063+0000 7f6ad3fff640 1 -- 172.21.15.16:0/3635859142 >> v1:172.21.15.16:6789/0 conn(0x7f6ad415e680 legacy=0x7f6ad40cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.063+0000 7f6ad3fff640 1 -- 172.21.15.16:0/3635859142 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad40cf5b0 con 0x7f6ad415e2a0 2024-09-19T07:54:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.063+0000 7f6ad2ffd640 1 -- 172.21.15.16:0/3635859142 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 398964327 0 0) 0x7f6ad40d1880 con 0x7f6ad415e680 2024-09-19T07:54:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.063+0000 7f6ad8eb7640 1 --2- 172.21.15.16:0/3635859142 >> v2:172.21.15.146:3300/0 conn(0x7f6ad406d960 0x7f6ad40d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.064+0000 7f6ad3fff640 1 --2- 172.21.15.16:0/3635859142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad415e2a0 0x7f6ad406d420 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4163050 tx=0x7f6ac402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b13fe78ef01d42e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.064+0000 7f6ad2ffd640 1 -- 172.21.15.16:0/3635859142 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4037070 con 0x7f6ad415e2a0 2024-09-19T07:54:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.064+0000 7f6ad2ffd640 1 -- 172.21.15.16:0/3635859142 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ac4032440 con 0x7f6ad415e2a0 2024-09-19T07:54:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.064+0000 7f6ad2ffd640 1 -- 172.21.15.16:0/3635859142 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4032810 con 0x7f6ad415e2a0 2024-09-19T07:54:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.064+0000 7f6ada91a640 1 -- 172.21.15.16:0/3635859142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad415e2a0 msgr2=0x7f6ad406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.064+0000 7f6ada91a640 1 --2- 172.21.15.16:0/3635859142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad415e2a0 0x7f6ad406d420 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4163050 tx=0x7f6ac402a1a0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.064+0000 7f6ada91a640 1 -- 172.21.15.16:0/3635859142 shutdown_connections 2024-09-19T07:54:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.064+0000 7f6ada91a640 1 --2- 172.21.15.16:0/3635859142 >> v2:172.21.15.146:3300/0 conn(0x7f6ad406d960 0x7f6ad40d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.065+0000 7f6ada91a640 1 --2- 172.21.15.16:0/3635859142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad415e2a0 0x7f6ad406d420 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.065+0000 7f6ada91a640 1 -- 172.21.15.16:0/3635859142 >> 172.21.15.16:0/3635859142 conn(0x7f6ad409d760 msgr2=0x7f6ad408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.065+0000 7f6ada91a640 1 -- 172.21.15.16:0/3635859142 shutdown_connections 2024-09-19T07:54:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.065+0000 7f6ada91a640 1 -- 172.21.15.16:0/3635859142 wait complete. 2024-09-19T07:54:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.066+0000 7f6ada91a640 1 Processor -- start 2024-09-19T07:54:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.066+0000 7f6ada91a640 1 -- start start 2024-09-19T07:54:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ada91a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ad406d960 0x7f6ad415d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ada91a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad415e2a0 0x7f6ad41f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ada91a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ad415e680 0x7f6ad40c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ada91a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad40d0360 con 0x7f6ad415e2a0 2024-09-19T07:54:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ada91a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad40d01e0 con 0x7f6ad415e680 2024-09-19T07:54:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ada91a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6ad40d48b0 con 0x7f6ad406d960 2024-09-19T07:54:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ad37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad415e2a0 0x7f6ad41f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ad37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad415e2a0 0x7f6ad41f42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56752/0 (socket says 172.21.15.16:56752) 2024-09-19T07:54:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ad37fe640 1 -- 172.21.15.16:0/3470351840 learned_addr learned my addr 172.21.15.16:0/3470351840 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ad37fe640 1 -- 172.21.15.16:0/3470351840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ad406d960 msgr2=0x7f6ad415d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ad37fe640 1 --2- 172.21.15.16:0/3470351840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ad406d960 0x7f6ad415d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ad37fe640 1 -- 172.21.15.16:0/3470351840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ad415e680 msgr2=0x7f6ad40c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ad37fe640 1 --2- 172.21.15.16:0/3470351840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ad415e680 0x7f6ad40c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ad37fe640 1 -- 172.21.15.16:0/3470351840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad41f47f0 con 0x7f6ad415e2a0 2024-09-19T07:54:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ad37fe640 1 --2- 172.21.15.16:0/3470351840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad415e2a0 0x7f6ad41f42b0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800c440 tx=0x7f6ac800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ad0ff9640 1 -- 172.21.15.16:0/3470351840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8018070 con 0x7f6ad415e2a0 2024-09-19T07:54:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ad0ff9640 1 -- 172.21.15.16:0/3470351840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ac80074e0 con 0x7f6ad415e2a0 2024-09-19T07:54:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ada91a640 1 -- 172.21.15.16:0/3470351840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad40ce640 con 0x7f6ad415e2a0 2024-09-19T07:54:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ad0ff9640 1 -- 172.21.15.16:0/3470351840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8004a20 con 0x7f6ad415e2a0 2024-09-19T07:54:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.067+0000 7f6ada91a640 1 -- 172.21.15.16:0/3470351840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad41e7140 con 0x7f6ad415e2a0 2024-09-19T07:54:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.068+0000 7f6ada91a640 1 -- 172.21.15.16:0/3470351840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad41e6c50 con 0x7f6ad415e2a0 2024-09-19T07:54:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.069+0000 7f6ad0ff9640 1 -- 172.21.15.16:0/3470351840 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6ac801b070 con 0x7f6ad415e2a0 2024-09-19T07:54:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.069+0000 7f6ad0ff9640 1 --2- 172.21.15.16:0/3470351840 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6ac006c2b0 0x7f6ac006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.069+0000 7f6ad0ff9640 1 -- 172.21.15.16:0/3470351840 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7f6ac808a7d0 con 0x7f6ad415e2a0 2024-09-19T07:54:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.070+0000 7f6ad3fff640 1 --2- 172.21.15.16:0/3470351840 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6ac006c2b0 0x7f6ac006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.070+0000 7f6ad3fff640 1 --2- 172.21.15.16:0/3470351840 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6ac006c2b0 0x7f6ac006e770 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4034000 tx=0x7f6ac402c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:50.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.074+0000 7f6ad0ff9640 1 -- 172.21.15.16:0/3470351840 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6ac8058e40 con 0x7f6ad415e2a0 2024-09-19T07:54:50.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.249+0000 7f6ada91a640 1 -- 172.21.15.16:0/3470351840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ad406cf90 con 0x7f6ad415e2a0 2024-09-19T07:54:50.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.251+0000 7f6ad0ff9640 1 -- 172.21.15.16:0/3470351840 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+16984 (secure 0 0 0) 0x7f6ac8058740 con 0x7f6ad415e2a0 2024-09-19T07:54:50.251 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:50.251 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":129,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:54:24.558589+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:54:20.571485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-09-19T07:54:21.899096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:54:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.252+0000 7f6ada91a640 1 -- 172.21.15.16:0/3470351840 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6ac006c2b0 msgr2=0x7f6ac006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.252+0000 7f6ada91a640 1 --2- 172.21.15.16:0/3470351840 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6ac006c2b0 0x7f6ac006e770 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4034000 tx=0x7f6ac402c000 comp rx=0 tx=0).stop 2024-09-19T07:54:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.253+0000 7f6ada91a640 1 -- 172.21.15.16:0/3470351840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad415e2a0 msgr2=0x7f6ad41f42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.253+0000 7f6ada91a640 1 --2- 172.21.15.16:0/3470351840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad415e2a0 0x7f6ad41f42b0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800c440 tx=0x7f6ac800c900 comp rx=0 tx=0).stop 2024-09-19T07:54:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.253+0000 7f6ada91a640 1 -- 172.21.15.16:0/3470351840 shutdown_connections 2024-09-19T07:54:50.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.253+0000 7f6ada91a640 1 --2- 172.21.15.16:0/3470351840 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6ac006c2b0 0x7f6ac006e770 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.253+0000 7f6ada91a640 1 --2- 172.21.15.16:0/3470351840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ad415e680 0x7f6ad40c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.253+0000 7f6ada91a640 1 --2- 172.21.15.16:0/3470351840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ad415e2a0 0x7f6ad41f42b0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.253+0000 7f6ada91a640 1 --2- 172.21.15.16:0/3470351840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ad406d960 0x7f6ad415d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.253+0000 7f6ada91a640 1 -- 172.21.15.16:0/3470351840 >> 172.21.15.16:0/3470351840 conn(0x7f6ad409d760 msgr2=0x7f6ad40b92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.253+0000 7f6ada91a640 1 -- 172.21.15.16:0/3470351840 shutdown_connections 2024-09-19T07:54:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.253+0000 7f6ada91a640 1 -- 172.21.15.16:0/3470351840 wait complete. 2024-09-19T07:54:50.264 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T07:54:50.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.367+0000 7f31054d0640 1 Processor -- start 2024-09-19T07:54:50.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.367+0000 7f31054d0640 1 -- start start 2024-09-19T07:54:50.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.367+0000 7f31054d0640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f310015bee0 0x7f31000d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:50.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.367+0000 7f31054d0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f31000cf910 0x7f31000d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:50.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.367+0000 7f31054d0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31000cdee0 con 0x7f310015c2c0 2024-09-19T07:54:50.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.367+0000 7f31054d0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f31000ceaf0 con 0x7f31000cf910 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.367+0000 7f31054d0640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f31000ce1c0 con 0x7f310015bee0 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.368+0000 7f30ff7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f31000cf910 0x7f31000d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.368+0000 7f30ff7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f31000cf910 0x7f31000d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56764/0 (socket says 172.21.15.16:56764) 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.368+0000 7f30f67fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f310015c2c0 0x7f31000cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47870/0 (socket says 172.21.15.16:47870) 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.368+0000 7f30ff7fe640 1 -- 172.21.15.16:0/2465545317 learned_addr learned my addr 172.21.15.16:0/2465545317 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.368+0000 7f30feffd640 1 --2- 172.21.15.16:0/2465545317 >> v2:172.21.15.146:3300/0 conn(0x7f310015bee0 0x7f31000d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.368+0000 7f30ff7fe640 1 -- 172.21.15.16:0/2465545317 >> v2:172.21.15.146:3300/0 conn(0x7f310015bee0 msgr2=0x7f31000d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.368+0000 7f30ff7fe640 1 --2- 172.21.15.16:0/2465545317 >> v2:172.21.15.146:3300/0 conn(0x7f310015bee0 0x7f31000d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.368+0000 7f30ff7fe640 1 -- 172.21.15.16:0/2465545317 >> v1:172.21.15.16:6789/0 conn(0x7f310015c2c0 legacy=0x7f31000cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.368+0000 7f30ff7fe640 1 -- 172.21.15.16:0/2465545317 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31000ce4a0 con 0x7f31000cf910 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.368+0000 7f30feffd640 1 --2- 172.21.15.16:0/2465545317 >> v2:172.21.15.146:3300/0 conn(0x7f310015bee0 0x7f31000d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.369+0000 7f30ff7fe640 1 --2- 172.21.15.16:0/2465545317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31000cf910 0x7f31000d09f0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f31000b2f40 tx=0x7f30f802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49df5cc5a70b33f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.369+0000 7f30fe7fc640 1 -- 172.21.15.16:0/2465545317 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8036070 con 0x7f31000cf910 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.369+0000 7f30fe7fc640 1 -- 172.21.15.16:0/2465545317 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30f8031600 con 0x7f31000cf910 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.369+0000 7f30fe7fc640 1 -- 172.21.15.16:0/2465545317 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f80319d0 con 0x7f31000cf910 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.369+0000 7f31054d0640 1 -- 172.21.15.16:0/2465545317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31000cf910 msgr2=0x7f31000d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.369+0000 7f31054d0640 1 --2- 172.21.15.16:0/2465545317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31000cf910 0x7f31000d09f0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f31000b2f40 tx=0x7f30f802a730 comp rx=0 tx=0).stop 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.369+0000 7f31054d0640 1 -- 172.21.15.16:0/2465545317 shutdown_connections 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.369+0000 7f31054d0640 1 --2- 172.21.15.16:0/2465545317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31000cf910 0x7f31000d09f0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.369+0000 7f31054d0640 1 --2- 172.21.15.16:0/2465545317 >> v2:172.21.15.146:3300/0 conn(0x7f310015bee0 0x7f31000d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.369+0000 7f31054d0640 1 -- 172.21.15.16:0/2465545317 >> 172.21.15.16:0/2465545317 conn(0x7f310006bdf0 msgr2=0x7f310006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:50.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.369+0000 7f31054d0640 1 -- 172.21.15.16:0/2465545317 shutdown_connections 2024-09-19T07:54:50.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.369+0000 7f31054d0640 1 -- 172.21.15.16:0/2465545317 wait complete. 2024-09-19T07:54:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.371+0000 7f31054d0640 1 Processor -- start 2024-09-19T07:54:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.371+0000 7f31054d0640 1 -- start start 2024-09-19T07:54:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.371+0000 7f31054d0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31000cf910 0x7f31000af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.371+0000 7f31054d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310015bee0 0x7f31001f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.371+0000 7f31054d0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f310015c2c0 0x7f31001421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.371+0000 7f31054d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f310006e040 con 0x7f310015bee0 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.371+0000 7f31054d0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f310006dec0 con 0x7f31000cf910 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.371+0000 7f31054d0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f310006e1c0 con 0x7f310015c2c0 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.372+0000 7f30f67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310015bee0 0x7f31001f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.372+0000 7f30f67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310015bee0 0x7f31001f23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56778/0 (socket says 172.21.15.16:56778) 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.372+0000 7f30f67fc640 1 -- 172.21.15.16:0/128346133 learned_addr learned my addr 172.21.15.16:0/128346133 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.372+0000 7f30feffd640 1 --2- 172.21.15.16:0/128346133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31000cf910 0x7f31000af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:50.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.372+0000 7f30ff7fe640 1 --2- 172.21.15.16:0/128346133 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f310015c2c0 0x7f31001421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:50.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.372+0000 7f30f67fc640 1 -- 172.21.15.16:0/128346133 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f310015c2c0 msgr2=0x7f31001421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.372+0000 7f30f67fc640 1 --2- 172.21.15.16:0/128346133 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f310015c2c0 0x7f31001421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.372+0000 7f30f67fc640 1 -- 172.21.15.16:0/128346133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31000cf910 msgr2=0x7f31000af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.372+0000 7f30f67fc640 1 --2- 172.21.15.16:0/128346133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31000cf910 0x7f31000af620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.372+0000 7f30f67fc640 1 -- 172.21.15.16:0/128346133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31000afb60 con 0x7f310015bee0 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.372+0000 7f30feffd640 1 --2- 172.21.15.16:0/128346133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31000cf910 0x7f31000af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.372+0000 7f30ff7fe640 1 --2- 172.21.15.16:0/128346133 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f310015c2c0 0x7f31001421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.372+0000 7f30f67fc640 1 --2- 172.21.15.16:0/128346133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310015bee0 0x7f31001f23a0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f30e800c400 tx=0x7f30e800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.372+0000 7f30f7fff640 1 -- 172.21.15.16:0/128346133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e80074e0 con 0x7f310015bee0 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.372+0000 7f30f7fff640 1 -- 172.21.15.16:0/128346133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e8007680 con 0x7f310015bee0 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.372+0000 7f31054d0640 1 -- 172.21.15.16:0/128346133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31000c9490 con 0x7f310015bee0 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.372+0000 7f30f7fff640 1 -- 172.21.15.16:0/128346133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8004a00 con 0x7f310015bee0 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.372+0000 7f31054d0640 1 -- 172.21.15.16:0/128346133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31000c9d30 con 0x7f310015bee0 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.373+0000 7f31054d0640 1 -- 172.21.15.16:0/128346133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31000cf200 con 0x7f310015bee0 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.374+0000 7f30f7fff640 1 -- 172.21.15.16:0/128346133 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f30e801a020 con 0x7f310015bee0 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.374+0000 7f30f7fff640 1 --2- 172.21.15.16:0/128346133 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f30e006c2e0 0x7f30e006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.374+0000 7f30f7fff640 1 -- 172.21.15.16:0/128346133 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7f30e808ada0 con 0x7f310015bee0 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.375+0000 7f30feffd640 1 --2- 172.21.15.16:0/128346133 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f30e006c2e0 0x7f30e006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.375+0000 7f30feffd640 1 --2- 172.21.15.16:0/128346133 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f30e006c2e0 0x7f30e006e7a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f31000cdb90 tx=0x7f30f0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.379+0000 7f30f7fff640 1 -- 172.21.15.16:0/128346133 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f30e8010300 con 0x7f310015bee0 2024-09-19T07:54:50.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.591+0000 7f31054d0640 1 -- 172.21.15.16:0/128346133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f31000b3f00 con 0x7f310015bee0 2024-09-19T07:54:50.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.592+0000 7f30f7fff640 1 -- 172.21.15.16:0/128346133 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=-1 MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it v74) ==== 316+0+0 (secure 0 0 0) 0x7f30e8059380 con 0x7f310015bee0 2024-09-19T07:54:50.592 INFO:teuthology.orchestra.run.smithi016.stderr:Error EPERM: MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it 2024-09-19T07:54:50.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:54:50.597 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs --yes-i-really-mean-it 2024-09-19T07:54:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.702+0000 7f58729fd640 1 Processor -- start 2024-09-19T07:54:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.702+0000 7f58729fd640 1 -- start start 2024-09-19T07:54:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.703+0000 7f58729fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f586c15bee0 0x7f586c0d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.703+0000 7f58729fd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f586c15c2c0 0x7f586c0d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.703+0000 7f58729fd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f586c0cdee0 con 0x7f586c0cf200 2024-09-19T07:54:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.703+0000 7f58729fd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f586c0ceaf0 con 0x7f586c15c2c0 2024-09-19T07:54:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.703+0000 7f58729fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f586c0ce1c0 con 0x7f586c15bee0 2024-09-19T07:54:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.703+0000 7f586bfff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f586c15bee0 0x7f586c0d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.703+0000 7f586b7fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f586c15c2c0 0x7f586c0d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.703+0000 7f586bfff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f586c15bee0 0x7f586c0d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:58618/0 (socket says 172.21.15.16:58618) 2024-09-19T07:54:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.703+0000 7f5870f9a640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f586c0cf200 0x7f586c0cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:44272/0 (socket says 172.21.15.16:44272) 2024-09-19T07:54:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.703+0000 7f586bfff640 1 -- 172.21.15.16:0/213955099 learned_addr learned my addr 172.21.15.16:0/213955099 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.704+0000 7f586bfff640 1 -- 172.21.15.16:0/213955099 >> v2:172.21.15.146:3300/0 conn(0x7f586c15c2c0 msgr2=0x7f586c0d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.704+0000 7f586bfff640 1 --2- 172.21.15.16:0/213955099 >> v2:172.21.15.146:3300/0 conn(0x7f586c15c2c0 0x7f586c0d09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.704+0000 7f586bfff640 1 -- 172.21.15.16:0/213955099 >> v1:172.21.15.146:6789/0 conn(0x7f586c0cf200 legacy=0x7f586c0cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.704+0000 7f586bfff640 1 -- 172.21.15.16:0/213955099 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586c0ce4a0 con 0x7f586c15bee0 2024-09-19T07:54:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.704+0000 7f586b7fe640 1 --2- 172.21.15.16:0/213955099 >> v2:172.21.15.146:3300/0 conn(0x7f586c15c2c0 0x7f586c0d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:54:50.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.705+0000 7f586bfff640 1 --2- 172.21.15.16:0/213955099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f586c15bee0 0x7f586c0d04b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f585c009870 tx=0x7f585c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dbff05391783d84 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:50.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.705+0000 7f586affd640 1 -- 172.21.15.16:0/213955099 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c037070 con 0x7f586c15bee0 2024-09-19T07:54:50.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.705+0000 7f586affd640 1 -- 172.21.15.16:0/213955099 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f585c032440 con 0x7f586c15bee0 2024-09-19T07:54:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.705+0000 7f586affd640 1 -- 172.21.15.16:0/213955099 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c032810 con 0x7f586c15bee0 2024-09-19T07:54:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.705+0000 7f58729fd640 1 -- 172.21.15.16:0/213955099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f586c15bee0 msgr2=0x7f586c0d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.705+0000 7f58729fd640 1 --2- 172.21.15.16:0/213955099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f586c15bee0 0x7f586c0d04b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f585c009870 tx=0x7f585c02a270 comp rx=0 tx=0).stop 2024-09-19T07:54:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.705+0000 7f58729fd640 1 -- 172.21.15.16:0/213955099 shutdown_connections 2024-09-19T07:54:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.705+0000 7f58729fd640 1 --2- 172.21.15.16:0/213955099 >> v2:172.21.15.146:3300/0 conn(0x7f586c15c2c0 0x7f586c0d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.705+0000 7f58729fd640 1 --2- 172.21.15.16:0/213955099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f586c15bee0 0x7f586c0d04b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.705+0000 7f58729fd640 1 -- 172.21.15.16:0/213955099 >> 172.21.15.16:0/213955099 conn(0x7f586c06bdf0 msgr2=0x7f586c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.705+0000 7f58729fd640 1 -- 172.21.15.16:0/213955099 shutdown_connections 2024-09-19T07:54:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.705+0000 7f58729fd640 1 -- 172.21.15.16:0/213955099 wait complete. 2024-09-19T07:54:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.706+0000 7f58729fd640 1 Processor -- start 2024-09-19T07:54:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.707+0000 7f58729fd640 1 -- start start 2024-09-19T07:54:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.707+0000 7f58729fd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f586c0cf200 0x7f586c150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.707+0000 7f58729fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f586c15bee0 0x7f586c15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.707+0000 7f58729fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f586c15c2c0 0x7f586c0c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.707+0000 7f58729fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f586c06e040 con 0x7f586c15bee0 2024-09-19T07:54:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.707+0000 7f58729fd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f586c06dec0 con 0x7f586c0cf200 2024-09-19T07:54:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.707+0000 7f58729fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f586c06e1c0 con 0x7f586c15c2c0 2024-09-19T07:54:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.708+0000 7f586b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f586c15bee0 0x7f586c15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.708+0000 7f586b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f586c15bee0 0x7f586c15ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56786/0 (socket says 172.21.15.16:56786) 2024-09-19T07:54:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.708+0000 7f586b7fe640 1 -- 172.21.15.16:0/1777845342 learned_addr learned my addr 172.21.15.16:0/1777845342 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.708+0000 7f5870f9a640 1 --2- 172.21.15.16:0/1777845342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f586c15c2c0 0x7f586c0c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.708+0000 7f586bfff640 1 --2- 172.21.15.16:0/1777845342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f586c0cf200 0x7f586c150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.708+0000 7f586b7fe640 1 -- 172.21.15.16:0/1777845342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f586c15c2c0 msgr2=0x7f586c0c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.708+0000 7f586b7fe640 1 --2- 172.21.15.16:0/1777845342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f586c15c2c0 0x7f586c0c7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.708+0000 7f586b7fe640 1 -- 172.21.15.16:0/1777845342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f586c0cf200 msgr2=0x7f586c150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.708+0000 7f586b7fe640 1 --2- 172.21.15.16:0/1777845342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f586c0cf200 0x7f586c150780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.708+0000 7f586b7fe640 1 -- 172.21.15.16:0/1777845342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586c15deb0 con 0x7f586c15bee0 2024-09-19T07:54:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.708+0000 7f586b7fe640 1 --2- 172.21.15.16:0/1777845342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f586c15bee0 0x7f586c15ef50 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f585400ebb0 tx=0x7f585400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.708+0000 7f5868ff9640 1 -- 172.21.15.16:0/1777845342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854017070 con 0x7f586c15bee0 2024-09-19T07:54:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.708+0000 7f5868ff9640 1 -- 172.21.15.16:0/1777845342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f585400cbd0 con 0x7f586c15bee0 2024-09-19T07:54:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.708+0000 7f58729fd640 1 -- 172.21.15.16:0/1777845342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586c15e190 con 0x7f586c15bee0 2024-09-19T07:54:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.708+0000 7f5868ff9640 1 -- 172.21.15.16:0/1777845342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854013470 con 0x7f586c15bee0 2024-09-19T07:54:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.708+0000 7f58729fd640 1 -- 172.21.15.16:0/1777845342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f586c15e750 con 0x7f586c15bee0 2024-09-19T07:54:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.709+0000 7f58729fd640 1 -- 172.21.15.16:0/1777845342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f586c0cf6b0 con 0x7f586c15bee0 2024-09-19T07:54:50.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.710+0000 7f5868ff9640 1 -- 172.21.15.16:0/1777845342 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f585401a070 con 0x7f586c15bee0 2024-09-19T07:54:50.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.710+0000 7f5868ff9640 1 --2- 172.21.15.16:0/1777845342 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f584c06c2b0 0x7f584c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:50.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.710+0000 7f5868ff9640 1 -- 172.21.15.16:0/1777845342 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7f585408b760 con 0x7f586c15bee0 2024-09-19T07:54:50.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.711+0000 7f586bfff640 1 --2- 172.21.15.16:0/1777845342 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f584c06c2b0 0x7f584c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:50.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.712+0000 7f586bfff640 1 --2- 172.21.15.16:0/1777845342 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f584c06c2b0 0x7f584c06e770 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f585c034000 tx=0x7f585c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:50.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.715+0000 7f5868ff9640 1 -- 172.21.15.16:0/1777845342 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5854059700 con 0x7f586c15bee0 2024-09-19T07:54:50.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.926+0000 7f58729fd640 1 -- 172.21.15.16:0/1777845342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f586c15b1a0 con 0x7f586c15bee0 2024-09-19T07:54:50.926 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-19T07:54:50.926+0000 7f6950fea640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:54:50.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.928+0000 7f5868ff9640 1 -- 172.21.15.16:0/1777845342 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) ==== 191+0+0 (secure 0 0 0) 0x7f585405da00 con 0x7f586c15bee0 2024-09-19T07:54:50.928 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T07:54:50.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.931+0000 7f58729fd640 1 -- 172.21.15.16:0/1777845342 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f584c06c2b0 msgr2=0x7f584c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:50.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.931+0000 7f58729fd640 1 --2- 172.21.15.16:0/1777845342 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f584c06c2b0 0x7f584c06e770 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f585c034000 tx=0x7f585c02e000 comp rx=0 tx=0).stop 2024-09-19T07:54:50.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.931+0000 7f58729fd640 1 -- 172.21.15.16:0/1777845342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f586c15bee0 msgr2=0x7f586c15ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:50.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.931+0000 7f58729fd640 1 --2- 172.21.15.16:0/1777845342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f586c15bee0 0x7f586c15ef50 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f585400ebb0 tx=0x7f585400c520 comp rx=0 tx=0).stop 2024-09-19T07:54:50.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.931+0000 7f58729fd640 1 -- 172.21.15.16:0/1777845342 shutdown_connections 2024-09-19T07:54:50.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.931+0000 7f58729fd640 1 --2- 172.21.15.16:0/1777845342 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f584c06c2b0 0x7f584c06e770 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.931+0000 7f58729fd640 1 --2- 172.21.15.16:0/1777845342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f586c15c2c0 0x7f586c0c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.931+0000 7f58729fd640 1 --2- 172.21.15.16:0/1777845342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f586c15bee0 0x7f586c15ef50 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.931+0000 7f58729fd640 1 --2- 172.21.15.16:0/1777845342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f586c0cf200 0x7f586c150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:50.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.931+0000 7f58729fd640 1 -- 172.21.15.16:0/1777845342 >> 172.21.15.16:0/1777845342 conn(0x7f586c06bdf0 msgr2=0x7f586c0b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:50.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.931+0000 7f58729fd640 1 -- 172.21.15.16:0/1777845342 shutdown_connections 2024-09-19T07:54:50.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:50.931+0000 7f58729fd640 1 -- 172.21.15.16:0/1777845342 wait complete. 2024-09-19T07:54:50.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T07:54:50.970 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2024-09-19T07:54:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.048+0000 7f6b40853640 1 Processor -- start 2024-09-19T07:54:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.048+0000 7f6b40853640 1 -- start start 2024-09-19T07:54:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.049+0000 7f6b40853640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6b3c1619b0 0x7f6b3c0d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.049+0000 7f6b40853640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b3c0ce6a0 con 0x7f6b3c0b0a80 2024-09-19T07:54:51.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.049+0000 7f6b40853640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b3c06c990 con 0x7f6b3c161d90 2024-09-19T07:54:51.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.049+0000 7f6b40853640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f6b3c0cec10 con 0x7f6b3c1619b0 2024-09-19T07:54:51.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.049+0000 7f6b3a59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6b3c1619b0 0x7f6b3c0d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:51.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.049+0000 7f6b39d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6b3c0b0a80 0x7f6b3c0cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47886/0 (socket says 172.21.15.16:47886) 2024-09-19T07:54:51.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.049+0000 7f6b39d9b640 1 -- 172.21.15.16:0/1526997792 learned_addr learned my addr 172.21.15.16:0/1526997792 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:51.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.049+0000 7f6b3a59c640 1 -- 172.21.15.16:0/1526997792 >> v1:172.21.15.146:6789/0 conn(0x7f6b3c161d90 legacy=0x7f6b3c0cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:51.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.049+0000 7f6b3959a640 1 -- 172.21.15.16:0/1526997792 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 728114515 0 0) 0x7f6b3c0ce6a0 con 0x7f6b3c0b0a80 2024-09-19T07:54:51.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.049+0000 7f6b3a59c640 1 -- 172.21.15.16:0/1526997792 >> v1:172.21.15.16:6789/0 conn(0x7f6b3c0b0a80 legacy=0x7f6b3c0cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:51.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.049+0000 7f6b3a59c640 1 -- 172.21.15.16:0/1526997792 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3c0ce930 con 0x7f6b3c1619b0 2024-09-19T07:54:51.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.050+0000 7f6b3a59c640 1 --2- 172.21.15.16:0/1526997792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b3c1619b0 0x7f6b3c0d0440 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0097a0 tx=0x7f6b2c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f33ab037580743 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:51.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.050+0000 7f6b3959a640 1 -- 172.21.15.16:0/1526997792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c037070 con 0x7f6b3c1619b0 2024-09-19T07:54:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.050+0000 7f6b3959a640 1 -- 172.21.15.16:0/1526997792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b2c032440 con 0x7f6b3c1619b0 2024-09-19T07:54:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.050+0000 7f6b3959a640 1 -- 172.21.15.16:0/1526997792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c032810 con 0x7f6b3c1619b0 2024-09-19T07:54:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.051+0000 7f6b40853640 1 -- 172.21.15.16:0/1526997792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b3c1619b0 msgr2=0x7f6b3c0d0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.051+0000 7f6b40853640 1 --2- 172.21.15.16:0/1526997792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b3c1619b0 0x7f6b3c0d0440 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0097a0 tx=0x7f6b2c02a230 comp rx=0 tx=0).stop 2024-09-19T07:54:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.051+0000 7f6b40853640 1 -- 172.21.15.16:0/1526997792 shutdown_connections 2024-09-19T07:54:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.051+0000 7f6b40853640 1 --2- 172.21.15.16:0/1526997792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b3c1619b0 0x7f6b3c0d0440 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.051+0000 7f6b40853640 1 -- 172.21.15.16:0/1526997792 >> 172.21.15.16:0/1526997792 conn(0x7f6b3c09a9f0 msgr2=0x7f6b3c0b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.051+0000 7f6b40853640 1 -- 172.21.15.16:0/1526997792 shutdown_connections 2024-09-19T07:54:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.051+0000 7f6b40853640 1 -- 172.21.15.16:0/1526997792 wait complete. 2024-09-19T07:54:51.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.052+0000 7f6b40853640 1 Processor -- start 2024-09-19T07:54:51.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.052+0000 7f6b40853640 1 -- start start 2024-09-19T07:54:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.053+0000 7f6b40853640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b3c0b0a80 0x7f6b3c0b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.053+0000 7f6b40853640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b3c1619b0 0x7f6b3c158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.053+0000 7f6b40853640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b3c161d90 0x7f6b3c0c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.053+0000 7f6b40853640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c06c620 con 0x7f6b3c1619b0 2024-09-19T07:54:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.053+0000 7f6b40853640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c0b0820 con 0x7f6b3c0b0a80 2024-09-19T07:54:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.053+0000 7f6b40853640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6b3c0ccce0 con 0x7f6b3c161d90 2024-09-19T07:54:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.053+0000 7f6b39d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b3c1619b0 0x7f6b3c158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.053+0000 7f6b39d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b3c1619b0 0x7f6b3c158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56812/0 (socket says 172.21.15.16:56812) 2024-09-19T07:54:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.053+0000 7f6b39d9b640 1 -- 172.21.15.16:0/3111599723 learned_addr learned my addr 172.21.15.16:0/3111599723 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.053+0000 7f6b3a59c640 1 --2- 172.21.15.16:0/3111599723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b3c0b0a80 0x7f6b3c0b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.053+0000 7f6b3ad9d640 1 --2- 172.21.15.16:0/3111599723 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b3c161d90 0x7f6b3c0c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.053+0000 7f6b39d9b640 1 -- 172.21.15.16:0/3111599723 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b3c161d90 msgr2=0x7f6b3c0c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.053+0000 7f6b39d9b640 1 --2- 172.21.15.16:0/3111599723 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b3c161d90 0x7f6b3c0c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.053+0000 7f6b39d9b640 1 -- 172.21.15.16:0/3111599723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b3c0b0a80 msgr2=0x7f6b3c0b66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.053+0000 7f6b39d9b640 1 --2- 172.21.15.16:0/3111599723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b3c0b0a80 0x7f6b3c0b66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.053+0000 7f6b39d9b640 1 -- 172.21.15.16:0/3111599723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3c0b6c10 con 0x7f6b3c1619b0 2024-09-19T07:54:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.054+0000 7f6b39d9b640 1 --2- 172.21.15.16:0/3111599723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b3c1619b0 0x7f6b3c158460 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f6b3000c460 tx=0x7f6b3000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.054+0000 7f6b2affd640 1 -- 172.21.15.16:0/3111599723 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30018070 con 0x7f6b3c1619b0 2024-09-19T07:54:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.054+0000 7f6b2affd640 1 -- 172.21.15.16:0/3111599723 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b30007520 con 0x7f6b3c1619b0 2024-09-19T07:54:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.054+0000 7f6b2affd640 1 -- 172.21.15.16:0/3111599723 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30004a20 con 0x7f6b3c1619b0 2024-09-19T07:54:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.054+0000 7f6b40853640 1 -- 172.21.15.16:0/3111599723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3c1589a0 con 0x7f6b3c1619b0 2024-09-19T07:54:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.054+0000 7f6b40853640 1 -- 172.21.15.16:0/3111599723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b3c0b5bf0 con 0x7f6b3c1619b0 2024-09-19T07:54:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.055+0000 7f6b40853640 1 -- 172.21.15.16:0/3111599723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b3c0cd330 con 0x7f6b3c1619b0 2024-09-19T07:54:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.055+0000 7f6b2affd640 1 -- 172.21.15.16:0/3111599723 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6b3001b070 con 0x7f6b3c1619b0 2024-09-19T07:54:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.056+0000 7f6b2affd640 1 --2- 172.21.15.16:0/3111599723 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6b1c06c2b0 0x7f6b1c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.056+0000 7f6b2affd640 1 -- 172.21.15.16:0/3111599723 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6187+0+0 (secure 0 0 0) 0x7f6b3008a960 con 0x7f6b3c1619b0 2024-09-19T07:54:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.056+0000 7f6b3a59c640 1 --2- 172.21.15.16:0/3111599723 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6b1c06c2b0 0x7f6b1c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.057+0000 7f6b3a59c640 1 --2- 172.21.15.16:0/3111599723 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6b1c06c2b0 0x7f6b1c06e770 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c034000 tx=0x7f6b2c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.060+0000 7f6b2affd640 1 -- 172.21.15.16:0/3111599723 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6b300588a0 con 0x7f6b3c1619b0 2024-09-19T07:54:51.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.275+0000 7f6b40853640 1 -- 172.21.15.16:0/3111599723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6b3c06bdb0 con 0x7f6b3c1619b0 2024-09-19T07:54:51.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.929+0000 7f6b2affd640 1 -- 172.21.15.16:0/3111599723 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) ==== 104+0+0 (secure 0 0 0) 0x7f6b3005cba0 con 0x7f6b3c1619b0 2024-09-19T07:54:51.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.932+0000 7f6b40853640 1 -- 172.21.15.16:0/3111599723 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6b1c06c2b0 msgr2=0x7f6b1c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:51.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.932+0000 7f6b40853640 1 --2- 172.21.15.16:0/3111599723 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6b1c06c2b0 0x7f6b1c06e770 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c034000 tx=0x7f6b2c02e000 comp rx=0 tx=0).stop 2024-09-19T07:54:51.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.932+0000 7f6b40853640 1 -- 172.21.15.16:0/3111599723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b3c1619b0 msgr2=0x7f6b3c158460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:51.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.932+0000 7f6b40853640 1 --2- 172.21.15.16:0/3111599723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b3c1619b0 0x7f6b3c158460 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f6b3000c460 tx=0x7f6b3000c920 comp rx=0 tx=0).stop 2024-09-19T07:54:51.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.932+0000 7f6b40853640 1 -- 172.21.15.16:0/3111599723 shutdown_connections 2024-09-19T07:54:51.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.932+0000 7f6b40853640 1 --2- 172.21.15.16:0/3111599723 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6b1c06c2b0 0x7f6b1c06e770 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:51.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.932+0000 7f6b40853640 1 --2- 172.21.15.16:0/3111599723 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b3c161d90 0x7f6b3c0c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:51.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.932+0000 7f6b40853640 1 --2- 172.21.15.16:0/3111599723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b3c1619b0 0x7f6b3c158460 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:51.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.932+0000 7f6b40853640 1 --2- 172.21.15.16:0/3111599723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b3c0b0a80 0x7f6b3c0b66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.932+0000 7f6b40853640 1 -- 172.21.15.16:0/3111599723 >> 172.21.15.16:0/3111599723 conn(0x7f6b3c09a9f0 msgr2=0x7f6b3c0707d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.932+0000 7f6b40853640 1 -- 172.21.15.16:0/3111599723 shutdown_connections 2024-09-19T07:54:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:51.932+0000 7f6b40853640 1 -- 172.21.15.16:0/3111599723 wait complete. 2024-09-19T07:54:51.946 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T07:54:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.059+0000 7f96a6db7640 1 Processor -- start 2024-09-19T07:54:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.059+0000 7f96a6db7640 1 -- start start 2024-09-19T07:54:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.060+0000 7f96a6db7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f96a00c8d20 0x7f96a006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.060+0000 7f96a6db7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96a00c7140 con 0x7f96a006d260 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.060+0000 7f96a6db7640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96a00c9100 con 0x7f96a0144100 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.060+0000 7f96a6db7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f96a006d6d0 con 0x7f96a00c8d20 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.060+0000 7f96a4b53640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f96a00c8d20 0x7f96a006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.060+0000 7f96a4b53640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f96a00c8d20 0x7f96a006cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56822/0 (socket says 172.21.15.16:56822) 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.060+0000 7f9697fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f96a006d260 0x7f96a01439f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47892/0 (socket says 172.21.15.16:47892) 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.060+0000 7f96a4b53640 1 -- 172.21.15.16:0/1755716156 learned_addr learned my addr 172.21.15.16:0/1755716156 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.060+0000 7f96a4b53640 1 -- 172.21.15.16:0/1755716156 >> v1:172.21.15.146:6789/0 conn(0x7f96a0144100 legacy=0x7f96a00d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.060+0000 7f96a4b53640 1 -- 172.21.15.16:0/1755716156 >> v1:172.21.15.16:6789/0 conn(0x7f96a006d260 legacy=0x7f96a01439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.060+0000 7f96a4b53640 1 -- 172.21.15.16:0/1755716156 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a0144760 con 0x7f96a00c8d20 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.060+0000 7f96977fe640 1 -- 172.21.15.16:0/1755716156 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3648501101 0 0) 0x7f9690001110 con 0x7f96a006d260 2024-09-19T07:54:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.061+0000 7f96a4b53640 1 --2- 172.21.15.16:0/1755716156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a00c8d20 0x7f96a006cd20 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f969c009870 tx=0x7f969c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e86ffd4c7f25050 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.061+0000 7f96977fe640 1 -- 172.21.15.16:0/1755716156 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c037070 con 0x7f96a00c8d20 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.061+0000 7f96977fe640 1 -- 172.21.15.16:0/1755716156 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f969c032440 con 0x7f96a00c8d20 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.061+0000 7f96977fe640 1 -- 172.21.15.16:0/1755716156 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c032810 con 0x7f96a00c8d20 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.062+0000 7f96a6db7640 1 -- 172.21.15.16:0/1755716156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a00c8d20 msgr2=0x7f96a006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.062+0000 7f96a6db7640 1 --2- 172.21.15.16:0/1755716156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a00c8d20 0x7f96a006cd20 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f969c009870 tx=0x7f969c02a180 comp rx=0 tx=0).stop 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.062+0000 7f96a6db7640 1 -- 172.21.15.16:0/1755716156 shutdown_connections 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.062+0000 7f96a6db7640 1 --2- 172.21.15.16:0/1755716156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a00c8d20 0x7f96a006cd20 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.062+0000 7f96a6db7640 1 -- 172.21.15.16:0/1755716156 >> 172.21.15.16:0/1755716156 conn(0x7f96a00b3ed0 msgr2=0x7f96a00b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.062+0000 7f96a6db7640 1 -- 172.21.15.16:0/1755716156 shutdown_connections 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.062+0000 7f96a6db7640 1 -- 172.21.15.16:0/1755716156 wait complete. 2024-09-19T07:54:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.063+0000 7f96a6db7640 1 Processor -- start 2024-09-19T07:54:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.063+0000 7f96a6db7640 1 -- start start 2024-09-19T07:54:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.064+0000 7f96a6db7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96a006d260 0x7f96a0140790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.064+0000 7f96a6db7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a00c8d20 0x7f96a00b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.064+0000 7f96a6db7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a0144100 0x7f96a015fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.064+0000 7f96a6db7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96a0142200 con 0x7f96a00c8d20 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.064+0000 7f96a6db7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f96a0141960 con 0x7f96a0144100 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.064+0000 7f96a6db7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f96a0144510 con 0x7f96a006d260 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.064+0000 7f9697fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a00c8d20 0x7f96a00b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.064+0000 7f9697fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a00c8d20 0x7f96a00b7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56826/0 (socket says 172.21.15.16:56826) 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.064+0000 7f9697fff640 1 -- 172.21.15.16:0/485749166 learned_addr learned my addr 172.21.15.16:0/485749166 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.064+0000 7f96a5354640 1 --2- 172.21.15.16:0/485749166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a0144100 0x7f96a015fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.064+0000 7f96a4b53640 1 --2- 172.21.15.16:0/485749166 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96a006d260 0x7f96a0140790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.064+0000 7f9697fff640 1 -- 172.21.15.16:0/485749166 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96a006d260 msgr2=0x7f96a0140790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.064+0000 7f9697fff640 1 --2- 172.21.15.16:0/485749166 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96a006d260 0x7f96a0140790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.064+0000 7f9697fff640 1 -- 172.21.15.16:0/485749166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a0144100 msgr2=0x7f96a015fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.064+0000 7f9697fff640 1 --2- 172.21.15.16:0/485749166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a0144100 0x7f96a015fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.064+0000 7f9697fff640 1 -- 172.21.15.16:0/485749166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a01557a0 con 0x7f96a00c8d20 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.064+0000 7f96a4b53640 1 --2- 172.21.15.16:0/485749166 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96a006d260 0x7f96a0140790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.064+0000 7f9697fff640 1 --2- 172.21.15.16:0/485749166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a00c8d20 0x7f96a00b7330 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f969000c460 tx=0x7f969000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.065+0000 7f96957fa640 1 -- 172.21.15.16:0/485749166 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9690018070 con 0x7f96a00c8d20 2024-09-19T07:54:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.065+0000 7f96957fa640 1 -- 172.21.15.16:0/485749166 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9690007500 con 0x7f96a00c8d20 2024-09-19T07:54:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.065+0000 7f96a6db7640 1 -- 172.21.15.16:0/485749166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a0155a80 con 0x7f96a00c8d20 2024-09-19T07:54:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.065+0000 7f96957fa640 1 -- 172.21.15.16:0/485749166 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9690004a20 con 0x7f96a00c8d20 2024-09-19T07:54:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.065+0000 7f96a6db7640 1 -- 172.21.15.16:0/485749166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a0156040 con 0x7f96a00c8d20 2024-09-19T07:54:52.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.066+0000 7f96a6db7640 1 -- 172.21.15.16:0/485749166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a006cd20 con 0x7f96a00c8d20 2024-09-19T07:54:52.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.066+0000 7f96957fa640 1 -- 172.21.15.16:0/485749166 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f969001b070 con 0x7f96a00c8d20 2024-09-19T07:54:52.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.067+0000 7f96957fa640 1 --2- 172.21.15.16:0/485749166 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f968c06c2b0 0x7f968c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:52.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.067+0000 7f96957fa640 1 -- 172.21.15.16:0/485749166 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6187+0+0 (secure 0 0 0) 0x7f969008a830 con 0x7f96a00c8d20 2024-09-19T07:54:52.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.067+0000 7f96a4b53640 1 --2- 172.21.15.16:0/485749166 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f968c06c2b0 0x7f968c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:52.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.067+0000 7f96a4b53640 1 --2- 172.21.15.16:0/485749166 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f968c06c2b0 0x7f968c06e770 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f969c034000 tx=0x7f969c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.071+0000 7f96957fa640 1 -- 172.21.15.16:0/485749166 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f96a00c7140 con 0x7f96a00c8d20 2024-09-19T07:54:52.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.244+0000 7f96a6db7640 1 -- 172.21.15.16:0/485749166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f96a0072290 con 0x7f96a00c8d20 2024-09-19T07:54:52.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.931+0000 7f96957fa640 1 -- 172.21.15.16:0/485749166 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v131) ==== 181+0+0 (secure 0 0 0) 0x7f9690058e50 con 0x7f96a00c8d20 2024-09-19T07:54:52.931 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-19T07:54:52.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.934+0000 7f96a6db7640 1 -- 172.21.15.16:0/485749166 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f968c06c2b0 msgr2=0x7f968c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:52.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.934+0000 7f96a6db7640 1 --2- 172.21.15.16:0/485749166 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f968c06c2b0 0x7f968c06e770 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f969c034000 tx=0x7f969c02e000 comp rx=0 tx=0).stop 2024-09-19T07:54:52.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.934+0000 7f96a6db7640 1 -- 172.21.15.16:0/485749166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a00c8d20 msgr2=0x7f96a00b7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:52.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.934+0000 7f96a6db7640 1 --2- 172.21.15.16:0/485749166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a00c8d20 0x7f96a00b7330 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f969000c460 tx=0x7f969000c920 comp rx=0 tx=0).stop 2024-09-19T07:54:52.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.934+0000 7f96a6db7640 1 -- 172.21.15.16:0/485749166 shutdown_connections 2024-09-19T07:54:52.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.934+0000 7f96a6db7640 1 --2- 172.21.15.16:0/485749166 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f968c06c2b0 0x7f968c06e770 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:52.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.934+0000 7f96a6db7640 1 --2- 172.21.15.16:0/485749166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a0144100 0x7f96a015fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:52.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.934+0000 7f96a6db7640 1 --2- 172.21.15.16:0/485749166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a00c8d20 0x7f96a00b7330 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:52.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.934+0000 7f96a6db7640 1 --2- 172.21.15.16:0/485749166 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96a006d260 0x7f96a0140790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:52.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.934+0000 7f96a6db7640 1 -- 172.21.15.16:0/485749166 >> 172.21.15.16:0/485749166 conn(0x7f96a00b3ed0 msgr2=0x7f96a00b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:52.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.935+0000 7f96a6db7640 1 -- 172.21.15.16:0/485749166 shutdown_connections 2024-09-19T07:54:52.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:52.935+0000 7f96a6db7640 1 -- 172.21.15.16:0/485749166 wait complete. 2024-09-19T07:54:52.949 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T07:54:53.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.063+0000 7f139f749640 1 Processor -- start 2024-09-19T07:54:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.063+0000 7f139f749640 1 -- start start 2024-09-19T07:54:53.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.064+0000 7f139f749640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1398072590 0x7f139806d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:53.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.064+0000 7f139f749640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1398141ad0 0x7f1398141f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:53.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.064+0000 7f139f749640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13981620b0 con 0x7f13980d21f0 2024-09-19T07:54:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.064+0000 7f139f749640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f1398141750 con 0x7f1398141ad0 2024-09-19T07:54:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.064+0000 7f139f749640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1398162370 con 0x7f1398072590 2024-09-19T07:54:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.064+0000 7f139dce6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1398141ad0 0x7f1398141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.064+0000 7f139d4e5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f13980d21f0 0x7f13980d25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33742/0 (socket says 172.21.15.16:33742) 2024-09-19T07:54:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.064+0000 7f139dce6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1398141ad0 0x7f1398141f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50906/0 (socket says 172.21.15.16:50906) 2024-09-19T07:54:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.064+0000 7f139d4e5640 1 -- 172.21.15.16:0/2832089095 learned_addr learned my addr 172.21.15.16:0/2832089095 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.064+0000 7f139cce4640 1 --2- 172.21.15.16:0/2832089095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1398072590 0x7f139806d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.065+0000 7f139dce6640 1 -- 172.21.15.16:0/2832089095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1398072590 msgr2=0x7f139806d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.065+0000 7f139dce6640 1 --2- 172.21.15.16:0/2832089095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1398072590 0x7f139806d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.065+0000 7f139dce6640 1 -- 172.21.15.16:0/2832089095 >> v1:172.21.15.16:6789/0 conn(0x7f13980d21f0 legacy=0x7f13980d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.065+0000 7f1397fff640 1 -- 172.21.15.16:0/2832089095 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 187321392 0 0) 0x7f13981620b0 con 0x7f13980d21f0 2024-09-19T07:54:53.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.065+0000 7f139dce6640 1 -- 172.21.15.16:0/2832089095 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1398162650 con 0x7f1398141ad0 2024-09-19T07:54:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.065+0000 7f139dce6640 1 --2- 172.21.15.16:0/2832089095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398141ad0 0x7f1398141f50 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f13900097a0 tx=0x7f139002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257813aec6c8a5ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.066+0000 7f1397fff640 1 -- 172.21.15.16:0/2832089095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390037070 con 0x7f1398141ad0 2024-09-19T07:54:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.066+0000 7f1397fff640 1 -- 172.21.15.16:0/2832089095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f139002ae50 con 0x7f1398141ad0 2024-09-19T07:54:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.066+0000 7f1397fff640 1 -- 172.21.15.16:0/2832089095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390032660 con 0x7f1398141ad0 2024-09-19T07:54:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.066+0000 7f139f749640 1 -- 172.21.15.16:0/2832089095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398141ad0 msgr2=0x7f1398141f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.066+0000 7f139f749640 1 --2- 172.21.15.16:0/2832089095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398141ad0 0x7f1398141f50 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f13900097a0 tx=0x7f139002a100 comp rx=0 tx=0).stop 2024-09-19T07:54:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.066+0000 7f139f749640 1 -- 172.21.15.16:0/2832089095 shutdown_connections 2024-09-19T07:54:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.066+0000 7f139f749640 1 --2- 172.21.15.16:0/2832089095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398141ad0 0x7f1398141f50 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.066+0000 7f139f749640 1 --2- 172.21.15.16:0/2832089095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1398072590 0x7f139806d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:53.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.066+0000 7f139f749640 1 -- 172.21.15.16:0/2832089095 >> 172.21.15.16:0/2832089095 conn(0x7f13980b3ed0 msgr2=0x7f13980b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:53.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.066+0000 7f139f749640 1 -- 172.21.15.16:0/2832089095 shutdown_connections 2024-09-19T07:54:53.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.066+0000 7f139f749640 1 -- 172.21.15.16:0/2832089095 wait complete. 2024-09-19T07:54:53.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.067+0000 7f139f749640 1 Processor -- start 2024-09-19T07:54:53.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.068+0000 7f139f749640 1 -- start start 2024-09-19T07:54:53.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.068+0000 7f139f749640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1398072590 0x7f139815b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:53.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.068+0000 7f139f749640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13980d21f0 0x7f13981f6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:53.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.068+0000 7f139f749640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398141ad0 0x7f13981f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:53.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.068+0000 7f139f749640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f139806d620 con 0x7f1398141ad0 2024-09-19T07:54:53.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.068+0000 7f139f749640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1398162c10 con 0x7f1398072590 2024-09-19T07:54:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.068+0000 7f139f749640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f139806d7a0 con 0x7f13980d21f0 2024-09-19T07:54:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.068+0000 7f139dce6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398141ad0 0x7f13981f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.068+0000 7f139dce6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398141ad0 0x7f13981f70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50914/0 (socket says 172.21.15.16:50914) 2024-09-19T07:54:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.068+0000 7f139dce6640 1 -- 172.21.15.16:0/3915459296 learned_addr learned my addr 172.21.15.16:0/3915459296 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.069+0000 7f139cce4640 1 --2- 172.21.15.16:0/3915459296 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13980d21f0 0x7f13981f6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.069+0000 7f139d4e5640 1 --2- 172.21.15.16:0/3915459296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1398072590 0x7f139815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.069+0000 7f139dce6640 1 -- 172.21.15.16:0/3915459296 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13980d21f0 msgr2=0x7f13981f6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.069+0000 7f139dce6640 1 --2- 172.21.15.16:0/3915459296 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13980d21f0 0x7f13981f6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.069+0000 7f139dce6640 1 -- 172.21.15.16:0/3915459296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1398072590 msgr2=0x7f139815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.069+0000 7f139dce6640 1 --2- 172.21.15.16:0/3915459296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1398072590 0x7f139815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.069+0000 7f139dce6640 1 -- 172.21.15.16:0/3915459296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13981f4f00 con 0x7f1398141ad0 2024-09-19T07:54:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.069+0000 7f139dce6640 1 --2- 172.21.15.16:0/3915459296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398141ad0 0x7f13981f70a0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f13900097a0 tx=0x7f13900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.069+0000 7f1395ffb640 1 -- 172.21.15.16:0/3915459296 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390037040 con 0x7f1398141ad0 2024-09-19T07:54:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.069+0000 7f1395ffb640 1 -- 172.21.15.16:0/3915459296 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1390004030 con 0x7f1398141ad0 2024-09-19T07:54:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.069+0000 7f1395ffb640 1 -- 172.21.15.16:0/3915459296 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390004420 con 0x7f1398141ad0 2024-09-19T07:54:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.069+0000 7f139f749640 1 -- 172.21.15.16:0/3915459296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f139815d4a0 con 0x7f1398141ad0 2024-09-19T07:54:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.069+0000 7f139f749640 1 -- 172.21.15.16:0/3915459296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f139815b740 con 0x7f1398141ad0 2024-09-19T07:54:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.070+0000 7f139f749640 1 -- 172.21.15.16:0/3915459296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f139815dbb0 con 0x7f1398141ad0 2024-09-19T07:54:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.071+0000 7f1395ffb640 1 -- 172.21.15.16:0/3915459296 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1390003610 con 0x7f1398141ad0 2024-09-19T07:54:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.071+0000 7f1395ffb640 1 --2- 172.21.15.16:0/3915459296 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f138406c2b0 0x7f138406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.071+0000 7f1395ffb640 1 -- 172.21.15.16:0/3915459296 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5749+0+0 (secure 0 0 0) 0x7f13900abf10 con 0x7f1398141ad0 2024-09-19T07:54:53.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.072+0000 7f139d4e5640 1 --2- 172.21.15.16:0/3915459296 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f138406c2b0 0x7f138406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:53.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.072+0000 7f139d4e5640 1 --2- 172.21.15.16:0/3915459296 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f138406c2b0 0x7f138406e770 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f13880012e0 tx=0x7f1388001700 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:53.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.076+0000 7f1395ffb640 1 -- 172.21.15.16:0/3915459296 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f139004c070 con 0x7f1398141ad0 2024-09-19T07:54:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.249+0000 7f139f749640 1 -- 172.21.15.16:0/3915459296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1398068ca0 con 0x7f1398141ad0 2024-09-19T07:54:53.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.932+0000 7f1395ffb640 1 -- 172.21.15.16:0/3915459296 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v132) ==== 169+0+0 (secure 0 0 0) 0x7f139007a090 con 0x7f1398141ad0 2024-09-19T07:54:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-19T07:54:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.937+0000 7f139f749640 1 -- 172.21.15.16:0/3915459296 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f138406c2b0 msgr2=0x7f138406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.937+0000 7f139f749640 1 --2- 172.21.15.16:0/3915459296 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f138406c2b0 0x7f138406e770 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f13880012e0 tx=0x7f1388001700 comp rx=0 tx=0).stop 2024-09-19T07:54:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.937+0000 7f139f749640 1 -- 172.21.15.16:0/3915459296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398141ad0 msgr2=0x7f13981f70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.937+0000 7f139f749640 1 --2- 172.21.15.16:0/3915459296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398141ad0 0x7f13981f70a0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f13900097a0 tx=0x7f13900046c0 comp rx=0 tx=0).stop 2024-09-19T07:54:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.937+0000 7f139f749640 1 -- 172.21.15.16:0/3915459296 shutdown_connections 2024-09-19T07:54:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.937+0000 7f139f749640 1 --2- 172.21.15.16:0/3915459296 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f138406c2b0 0x7f138406e770 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.937+0000 7f139f749640 1 --2- 172.21.15.16:0/3915459296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398141ad0 0x7f13981f70a0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.937+0000 7f139f749640 1 --2- 172.21.15.16:0/3915459296 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13980d21f0 0x7f13981f6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.937+0000 7f139f749640 1 --2- 172.21.15.16:0/3915459296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1398072590 0x7f139815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.937+0000 7f139f749640 1 -- 172.21.15.16:0/3915459296 >> 172.21.15.16:0/3915459296 conn(0x7f13980b3ed0 msgr2=0x7f13980b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.937+0000 7f139f749640 1 -- 172.21.15.16:0/3915459296 shutdown_connections 2024-09-19T07:54:53.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:53.937+0000 7f139f749640 1 -- 172.21.15.16:0/3915459296 wait complete. 2024-09-19T07:54:53.953 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2024-09-19T07:54:54.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.064+0000 7fb97b59f640 1 Processor -- start 2024-09-19T07:54:54.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.064+0000 7fb97b59f640 1 -- start start 2024-09-19T07:54:54.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.065+0000 7fb97b59f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb97406d430 0x7fb9741439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.065+0000 7fb97b59f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9740c7140 con 0x7fb974143f30 2024-09-19T07:54:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.065+0000 7fb97b59f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9740c9100 con 0x7fb9740c8d20 2024-09-19T07:54:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.065+0000 7fb97b59f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb974141960 con 0x7fb97406d430 2024-09-19T07:54:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.065+0000 7fb979b3c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb974143f30 0x7fb9740d1e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33756/0 (socket says 172.21.15.16:33756) 2024-09-19T07:54:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.065+0000 7fb979b3c640 1 -- 172.21.15.16:0/3820983275 learned_addr learned my addr 172.21.15.16:0/3820983275 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.065+0000 7fb96bfff640 1 -- 172.21.15.16:0/3820983275 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3714695739 0 0) 0x7fb96c001490 con 0x7fb974143f30 2024-09-19T07:54:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.065+0000 7fb978b3a640 1 --2- 172.21.15.16:0/3820983275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb97406d430 0x7fb9741439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.065+0000 7fb96bfff640 1 -- 172.21.15.16:0/3820983275 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb960003520 con 0x7fb974143f30 2024-09-19T07:54:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.065+0000 7fb96bfff640 1 -- 172.21.15.16:0/3820983275 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 954011731 0 0) 0x7fb960003520 con 0x7fb974143f30 2024-09-19T07:54:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb96bfff640 1 -- 172.21.15.16:0/3820983275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb97406d430 msgr2=0x7fb9741439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb96bfff640 1 --2- 172.21.15.16:0/3820983275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb97406d430 0x7fb9741439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb96bfff640 1 -- 172.21.15.16:0/3820983275 >> v1:172.21.15.146:6789/0 conn(0x7fb9740c8d20 legacy=0x7fb97406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:54.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb96bfff640 1 -- 172.21.15.16:0/3820983275 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb974144760 con 0x7fb974143f30 2024-09-19T07:54:54.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb96bfff640 1 -- 172.21.15.16:0/3820983275 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fb96c001bc0 con 0x7fb974143f30 2024-09-19T07:54:54.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb96bfff640 1 -- 172.21.15.16:0/3820983275 >> v1:172.21.15.16:6789/0 conn(0x7fb974143f30 legacy=0x7fb9740d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:54.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb96bfff640 1 --2- 172.21.15.16:0/3820983275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9600039e0 0x7fb960003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb96bfff640 1 --2- 172.21.15.16:0/3820983275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb960004800 0x7fb960004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb96bfff640 1 --2- 172.21.15.16:0/3820983275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb97406d430 0x7fb96000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb96bfff640 1 -- 172.21.15.16:0/3820983275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb974141960 con 0x7fb960004800 2024-09-19T07:54:54.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb96bfff640 1 -- 172.21.15.16:0/3820983275 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb960005550 con 0x7fb9600039e0 2024-09-19T07:54:54.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb96bfff640 1 -- 172.21.15.16:0/3820983275 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb96000bf00 con 0x7fb97406d430 2024-09-19T07:54:54.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb978b3a640 1 --2- 172.21.15.16:0/3820983275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb960004800 0x7fb960004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb97933b640 1 --2- 172.21.15.16:0/3820983275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9600039e0 0x7fb960003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb979b3c640 1 --2- 172.21.15.16:0/3820983275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb97406d430 0x7fb96000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb978b3a640 1 -- 172.21.15.16:0/3820983275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb97406d430 msgr2=0x7fb96000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:54.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb978b3a640 1 --2- 172.21.15.16:0/3820983275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb97406d430 0x7fb96000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb978b3a640 1 -- 172.21.15.16:0/3820983275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9600039e0 msgr2=0x7fb960003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:54.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb978b3a640 1 --2- 172.21.15.16:0/3820983275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9600039e0 0x7fb960003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.066+0000 7fb978b3a640 1 -- 172.21.15.16:0/3820983275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb960003520 con 0x7fb960004800 2024-09-19T07:54:54.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.067+0000 7fb978b3a640 1 --2- 172.21.15.16:0/3820983275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb960004800 0x7fb960004c80 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fb964009ac0 tx=0x7fb960005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8d4974a36f2d45 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:54.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.067+0000 7fb96bfff640 1 -- 172.21.15.16:0/3820983275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964036070 con 0x7fb960004800 2024-09-19T07:54:54.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.067+0000 7fb96bfff640 1 -- 172.21.15.16:0/3820983275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb964031400 con 0x7fb960004800 2024-09-19T07:54:54.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.067+0000 7fb96bfff640 1 -- 172.21.15.16:0/3820983275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964009690 con 0x7fb960004800 2024-09-19T07:54:54.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.067+0000 7fb97b59f640 1 -- 172.21.15.16:0/3820983275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb960004800 msgr2=0x7fb960004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:54.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.067+0000 7fb97b59f640 1 --2- 172.21.15.16:0/3820983275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb960004800 0x7fb960004c80 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fb964009ac0 tx=0x7fb960005830 comp rx=0 tx=0).stop 2024-09-19T07:54:54.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.067+0000 7fb979b3c640 1 -- 172.21.15.16:0/3820983275 reap_dead start 2024-09-19T07:54:54.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.067+0000 7fb97b59f640 1 -- 172.21.15.16:0/3820983275 shutdown_connections 2024-09-19T07:54:54.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.067+0000 7fb97b59f640 1 -- 172.21.15.16:0/3820983275 >> 172.21.15.16:0/3820983275 conn(0x7fb9740b3ed0 msgr2=0x7fb9740b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:54.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.068+0000 7fb97b59f640 1 -- 172.21.15.16:0/3820983275 shutdown_connections 2024-09-19T07:54:54.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.068+0000 7fb97b59f640 1 -- 172.21.15.16:0/3820983275 wait complete. 2024-09-19T07:54:54.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.069+0000 7fb97b59f640 1 Processor -- start 2024-09-19T07:54:54.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.069+0000 7fb97b59f640 1 -- start start 2024-09-19T07:54:54.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.069+0000 7fb97b59f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb960004800 0x7fb974160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.069+0000 7fb97b59f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9741f4920 0x7fb9740b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.069+0000 7fb97b59f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb97415f600 0x7fb9741572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.069+0000 7fb97b59f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb974141e90 con 0x7fb97415f600 2024-09-19T07:54:54.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.069+0000 7fb97b59f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb974142200 con 0x7fb960004800 2024-09-19T07:54:54.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.069+0000 7fb97b59f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb9741436a0 con 0x7fb9741f4920 2024-09-19T07:54:54.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.070+0000 7fb979b3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb97415f600 0x7fb9741572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.070+0000 7fb979b3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb97415f600 0x7fb9741572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50928/0 (socket says 172.21.15.16:50928) 2024-09-19T07:54:54.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.070+0000 7fb979b3c640 1 -- 172.21.15.16:0/684985003 learned_addr learned my addr 172.21.15.16:0/684985003 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:54.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.070+0000 7fb97933b640 1 --2- 172.21.15.16:0/684985003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb960004800 0x7fb974160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.070+0000 7fb978b3a640 1 --2- 172.21.15.16:0/684985003 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9741f4920 0x7fb9740b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.070+0000 7fb979b3c640 1 -- 172.21.15.16:0/684985003 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9741f4920 msgr2=0x7fb9740b7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:54.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.070+0000 7fb979b3c640 1 --2- 172.21.15.16:0/684985003 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9741f4920 0x7fb9740b7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.070+0000 7fb979b3c640 1 -- 172.21.15.16:0/684985003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb960004800 msgr2=0x7fb974160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:54.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.070+0000 7fb979b3c640 1 --2- 172.21.15.16:0/684985003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb960004800 0x7fb974160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.070+0000 7fb979b3c640 1 -- 172.21.15.16:0/684985003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb974160ad0 con 0x7fb97415f600 2024-09-19T07:54:54.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.070+0000 7fb979b3c640 1 --2- 172.21.15.16:0/684985003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb97415f600 0x7fb9741572c0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fb97406ef00 tx=0x7fb96c007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:54.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.070+0000 7fb969ffb640 1 -- 172.21.15.16:0/684985003 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c017020 con 0x7fb97415f600 2024-09-19T07:54:54.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.070+0000 7fb969ffb640 1 -- 172.21.15.16:0/684985003 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96c00eb00 con 0x7fb97415f600 2024-09-19T07:54:54.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.070+0000 7fb97b59f640 1 -- 172.21.15.16:0/684985003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb974160d90 con 0x7fb97415f600 2024-09-19T07:54:54.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.070+0000 7fb969ffb640 1 -- 172.21.15.16:0/684985003 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c013420 con 0x7fb97415f600 2024-09-19T07:54:54.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.071+0000 7fb97b59f640 1 -- 172.21.15.16:0/684985003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9741e7880 con 0x7fb97415f600 2024-09-19T07:54:54.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.072+0000 7fb97b59f640 1 -- 172.21.15.16:0/684985003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9740d0f90 con 0x7fb97415f600 2024-09-19T07:54:54.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.072+0000 7fb969ffb640 1 -- 172.21.15.16:0/684985003 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb96c01a020 con 0x7fb97415f600 2024-09-19T07:54:54.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.072+0000 7fb969ffb640 1 --2- 172.21.15.16:0/684985003 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb95806c2e0 0x7fb95806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.072+0000 7fb969ffb640 1 -- 172.21.15.16:0/684985003 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5335+0+0 (secure 0 0 0) 0x7fb96c08b1e0 con 0x7fb97415f600 2024-09-19T07:54:54.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.073+0000 7fb97933b640 1 --2- 172.21.15.16:0/684985003 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb95806c2e0 0x7fb95806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.073+0000 7fb97933b640 1 --2- 172.21.15.16:0/684985003 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb95806c2e0 0x7fb95806e7a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb9700017e0 tx=0x7fb970009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:54.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.077+0000 7fb969ffb640 1 -- 172.21.15.16:0/684985003 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb9740d0f90 con 0x7fb97415f600 2024-09-19T07:54:54.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.255+0000 7fb97b59f640 1 -- 172.21.15.16:0/684985003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) -- 0x7fb9740690c0 con 0x7fb97415f600 2024-09-19T07:54:54.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.259+0000 7fb969ffb640 1 -- 172.21.15.16:0/684985003 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v58) ==== 111+0+0 (secure 0 0 0) 0x7fb96c021030 con 0x7fb97415f600 2024-09-19T07:54:54.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.261+0000 7fb97b59f640 1 -- 172.21.15.16:0/684985003 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb95806c2e0 msgr2=0x7fb95806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:54.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.261+0000 7fb97b59f640 1 --2- 172.21.15.16:0/684985003 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb95806c2e0 0x7fb95806e7a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb9700017e0 tx=0x7fb970009040 comp rx=0 tx=0).stop 2024-09-19T07:54:54.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.261+0000 7fb97b59f640 1 -- 172.21.15.16:0/684985003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb97415f600 msgr2=0x7fb9741572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:54.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.261+0000 7fb97b59f640 1 --2- 172.21.15.16:0/684985003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb97415f600 0x7fb9741572c0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fb97406ef00 tx=0x7fb96c007bc0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.261+0000 7fb97b59f640 1 -- 172.21.15.16:0/684985003 shutdown_connections 2024-09-19T07:54:54.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.261+0000 7fb97b59f640 1 --2- 172.21.15.16:0/684985003 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb95806c2e0 0x7fb95806e7a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.261+0000 7fb97b59f640 1 --2- 172.21.15.16:0/684985003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb97415f600 0x7fb9741572c0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.261+0000 7fb97b59f640 1 --2- 172.21.15.16:0/684985003 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9741f4920 0x7fb9740b7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.261+0000 7fb97b59f640 1 --2- 172.21.15.16:0/684985003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb960004800 0x7fb974160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.261+0000 7fb97b59f640 1 -- 172.21.15.16:0/684985003 >> 172.21.15.16:0/684985003 conn(0x7fb9740b3ed0 msgr2=0x7fb9740b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:54.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.261+0000 7fb97b59f640 1 -- 172.21.15.16:0/684985003 shutdown_connections 2024-09-19T07:54:54.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.261+0000 7fb97b59f640 1 -- 172.21.15.16:0/684985003 wait complete. 2024-09-19T07:54:54.272 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2024-09-19T07:54:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.376+0000 7ffaa9111640 1 Processor -- start 2024-09-19T07:54:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.376+0000 7ffaa9111640 1 -- start start 2024-09-19T07:54:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.376+0000 7ffaa9111640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffaa41619b0 0x7ffaa40d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.376+0000 7ffaa9111640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaa40ce6a0 con 0x7ffaa4161d90 2024-09-19T07:54:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.376+0000 7ffaa9111640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaa406c990 con 0x7ffaa40b0a80 2024-09-19T07:54:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.376+0000 7ffaa9111640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ffaa40cec10 con 0x7ffaa41619b0 2024-09-19T07:54:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa359e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffaa4161d90 0x7ffaa40cdf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33762/0 (socket says 172.21.15.16:33762) 2024-09-19T07:54:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa359e640 1 -- 172.21.15.16:0/85455038 learned_addr learned my addr 172.21.15.16:0/85455038 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa2d9d640 1 --2- 172.21.15.16:0/85455038 >> v2:172.21.15.146:3300/0 conn(0x7ffaa41619b0 0x7ffaa40d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa1d9b640 1 -- 172.21.15.16:0/85455038 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2858777830 0 0) 0x7ffaa40ce6a0 con 0x7ffaa4161d90 2024-09-19T07:54:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa1d9b640 1 -- 172.21.15.16:0/85455038 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa90003520 con 0x7ffaa4161d90 2024-09-19T07:54:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa1d9b640 1 -- 172.21.15.16:0/85455038 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 44931363 0 0) 0x7ffa90003520 con 0x7ffaa4161d90 2024-09-19T07:54:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa1d9b640 1 -- 172.21.15.16:0/85455038 >> v2:172.21.15.146:3300/0 conn(0x7ffaa41619b0 msgr2=0x7ffaa40d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa1d9b640 1 --2- 172.21.15.16:0/85455038 >> v2:172.21.15.146:3300/0 conn(0x7ffaa41619b0 0x7ffaa40d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa1d9b640 1 -- 172.21.15.16:0/85455038 >> v1:172.21.15.146:6789/0 conn(0x7ffaa40b0a80 legacy=0x7ffaa40cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:54.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa1d9b640 1 -- 172.21.15.16:0/85455038 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa40ce930 con 0x7ffaa4161d90 2024-09-19T07:54:54.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa1d9b640 1 -- 172.21.15.16:0/85455038 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7ffa9c001670 con 0x7ffaa4161d90 2024-09-19T07:54:54.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa1d9b640 1 -- 172.21.15.16:0/85455038 >> v1:172.21.15.16:6789/0 conn(0x7ffaa4161d90 legacy=0x7ffaa40cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:54.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa1d9b640 1 --2- 172.21.15.16:0/85455038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa900039b0 0x7ffa90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa1d9b640 1 --2- 172.21.15.16:0/85455038 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa90004800 0x7ffa90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa1d9b640 1 --2- 172.21.15.16:0/85455038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa900051c0 0x7ffa9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa1d9b640 1 -- 172.21.15.16:0/85455038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa40cec10 con 0x7ffa900039b0 2024-09-19T07:54:54.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa1d9b640 1 -- 172.21.15.16:0/85455038 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9000c060 con 0x7ffa900051c0 2024-09-19T07:54:54.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa1d9b640 1 -- 172.21.15.16:0/85455038 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ffa9000c260 con 0x7ffa90004800 2024-09-19T07:54:54.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa2d9d640 1 --2- 172.21.15.16:0/85455038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa900039b0 0x7ffa90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa259c640 1 --2- 172.21.15.16:0/85455038 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa90004800 0x7ffa90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.377+0000 7ffaa359e640 1 --2- 172.21.15.16:0/85455038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa900051c0 0x7ffa9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.378+0000 7ffaa2d9d640 1 -- 172.21.15.16:0/85455038 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa90004800 msgr2=0x7ffa90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:54.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.378+0000 7ffaa2d9d640 1 --2- 172.21.15.16:0/85455038 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa90004800 0x7ffa90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.378+0000 7ffaa2d9d640 1 -- 172.21.15.16:0/85455038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa900051c0 msgr2=0x7ffa9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:54.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.378+0000 7ffaa2d9d640 1 --2- 172.21.15.16:0/85455038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa900051c0 0x7ffa9000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.378+0000 7ffaa2d9d640 1 -- 172.21.15.16:0/85455038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa90003520 con 0x7ffa900039b0 2024-09-19T07:54:54.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.378+0000 7ffaa359e640 1 --2- 172.21.15.16:0/85455038 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa900051c0 0x7ffa9000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:54.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.378+0000 7ffaa359e640 1 -- 172.21.15.16:0/85455038 reap_dead start 2024-09-19T07:54:54.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.378+0000 7ffaa2d9d640 1 --2- 172.21.15.16:0/85455038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa900039b0 0x7ffa90003dd0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009fc0 tx=0x7ffa90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c80c45581c8676 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:54.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.378+0000 7ffaa1d9b640 1 -- 172.21.15.16:0/85455038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa94009c10 con 0x7ffa900039b0 2024-09-19T07:54:54.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.379+0000 7ffaa1d9b640 1 -- 172.21.15.16:0/85455038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa94036070 con 0x7ffa900039b0 2024-09-19T07:54:54.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.379+0000 7ffaa1d9b640 1 -- 172.21.15.16:0/85455038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa94031880 con 0x7ffa900039b0 2024-09-19T07:54:54.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.379+0000 7ffaa9111640 1 -- 172.21.15.16:0/85455038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa900039b0 msgr2=0x7ffa90003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:54.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.379+0000 7ffaa9111640 1 --2- 172.21.15.16:0/85455038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa900039b0 0x7ffa90003dd0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009fc0 tx=0x7ffa90005830 comp rx=0 tx=0).stop 2024-09-19T07:54:54.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.379+0000 7ffaa9111640 1 -- 172.21.15.16:0/85455038 shutdown_connections 2024-09-19T07:54:54.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.379+0000 7ffaa9111640 1 --2- 172.21.15.16:0/85455038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa900039b0 0x7ffa90003dd0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.379+0000 7ffaa9111640 1 -- 172.21.15.16:0/85455038 >> 172.21.15.16:0/85455038 conn(0x7ffaa409a9f0 msgr2=0x7ffaa40b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:54.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.379+0000 7ffaa9111640 1 -- 172.21.15.16:0/85455038 shutdown_connections 2024-09-19T07:54:54.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.379+0000 7ffaa9111640 1 -- 172.21.15.16:0/85455038 wait complete. 2024-09-19T07:54:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.381+0000 7ffaa9111640 1 Processor -- start 2024-09-19T07:54:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.381+0000 7ffaa9111640 1 -- start start 2024-09-19T07:54:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.381+0000 7ffaa9111640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa900039b0 0x7ffaa40b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.381+0000 7ffaa9111640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffaa414bec0 0x7ffaa414c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.381+0000 7ffaa9111640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaa414ca20 0x7ffaa414e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.381+0000 7ffaa9111640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa406c620 con 0x7ffa900039b0 2024-09-19T07:54:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.381+0000 7ffaa9111640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa40b0820 con 0x7ffaa414ca20 2024-09-19T07:54:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.381+0000 7ffaa9111640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa40cd330 con 0x7ffaa414bec0 2024-09-19T07:54:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.381+0000 7ffaa2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa900039b0 0x7ffaa40b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.382+0000 7ffaa2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa900039b0 0x7ffaa40b2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50938/0 (socket says 172.21.15.16:50938) 2024-09-19T07:54:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.382+0000 7ffaa2d9d640 1 -- 172.21.15.16:0/1595907191 learned_addr learned my addr 172.21.15.16:0/1595907191 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.382+0000 7ffaa359e640 1 --2- 172.21.15.16:0/1595907191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaa414ca20 0x7ffaa414e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.382+0000 7ffaa259c640 1 --2- 172.21.15.16:0/1595907191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffaa414bec0 0x7ffaa414c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.382+0000 7ffaa2d9d640 1 -- 172.21.15.16:0/1595907191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffaa414bec0 msgr2=0x7ffaa414c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.382+0000 7ffaa2d9d640 1 --2- 172.21.15.16:0/1595907191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffaa414bec0 0x7ffaa414c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.382+0000 7ffaa2d9d640 1 -- 172.21.15.16:0/1595907191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaa414ca20 msgr2=0x7ffaa414e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.382+0000 7ffaa2d9d640 1 --2- 172.21.15.16:0/1595907191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaa414ca20 0x7ffaa414e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.382+0000 7ffaa2d9d640 1 -- 172.21.15.16:0/1595907191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa40b2a60 con 0x7ffa900039b0 2024-09-19T07:54:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.382+0000 7ffaa2d9d640 1 --2- 172.21.15.16:0/1595907191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa900039b0 0x7ffaa40b2520 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400e200 tx=0x7ffa94009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.382+0000 7ffa9b7fe640 1 -- 172.21.15.16:0/1595907191 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa94009c10 con 0x7ffa900039b0 2024-09-19T07:54:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.382+0000 7ffa9b7fe640 1 -- 172.21.15.16:0/1595907191 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa94003990 con 0x7ffa900039b0 2024-09-19T07:54:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.382+0000 7ffaa9111640 1 -- 172.21.15.16:0/1595907191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa40b7040 con 0x7ffa900039b0 2024-09-19T07:54:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.382+0000 7ffa9b7fe640 1 -- 172.21.15.16:0/1595907191 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa94003d60 con 0x7ffa900039b0 2024-09-19T07:54:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.382+0000 7ffaa9111640 1 -- 172.21.15.16:0/1595907191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa40b1d20 con 0x7ffa900039b0 2024-09-19T07:54:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.383+0000 7ffaa9111640 1 -- 172.21.15.16:0/1595907191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa40b7320 con 0x7ffa900039b0 2024-09-19T07:54:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.384+0000 7ffa9b7fe640 1 -- 172.21.15.16:0/1595907191 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ffa94005d10 con 0x7ffa900039b0 2024-09-19T07:54:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.384+0000 7ffa9b7fe640 1 --2- 172.21.15.16:0/1595907191 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffa8406c2e0 0x7ffa8406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.384+0000 7ffa9b7fe640 1 -- 172.21.15.16:0/1595907191 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5335+0+0 (secure 0 0 0) 0x7ffa940ac5d0 con 0x7ffa900039b0 2024-09-19T07:54:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.384+0000 7ffaa259c640 1 --2- 172.21.15.16:0/1595907191 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffa8406c2e0 0x7ffa8406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.385+0000 7ffaa259c640 1 --2- 172.21.15.16:0/1595907191 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffa8406c2e0 0x7ffa8406e7a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c000c00 tx=0x7ffa8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:54.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.388+0000 7ffa9b7fe640 1 -- 172.21.15.16:0/1595907191 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ffa9407afd0 con 0x7ffa900039b0 2024-09-19T07:54:54.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.567+0000 7ffaa9111640 1 -- 172.21.15.16:0/1595907191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) -- 0x7ffaa40cd080 con 0x7ffa900039b0 2024-09-19T07:54:54.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.570+0000 7ffa9b7fe640 1 -- 172.21.15.16:0/1595907191 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v59) ==== 107+0+0 (secure 0 0 0) 0x7ffa9407a8d0 con 0x7ffa900039b0 2024-09-19T07:54:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.572+0000 7ffaa9111640 1 -- 172.21.15.16:0/1595907191 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffa8406c2e0 msgr2=0x7ffa8406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.572+0000 7ffaa9111640 1 --2- 172.21.15.16:0/1595907191 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffa8406c2e0 0x7ffa8406e7a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c000c00 tx=0x7ffa8c009040 comp rx=0 tx=0).stop 2024-09-19T07:54:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.572+0000 7ffaa9111640 1 -- 172.21.15.16:0/1595907191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa900039b0 msgr2=0x7ffaa40b2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.572+0000 7ffaa9111640 1 --2- 172.21.15.16:0/1595907191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa900039b0 0x7ffaa40b2520 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400e200 tx=0x7ffa94009750 comp rx=0 tx=0).stop 2024-09-19T07:54:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.572+0000 7ffaa9111640 1 -- 172.21.15.16:0/1595907191 shutdown_connections 2024-09-19T07:54:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.572+0000 7ffaa9111640 1 --2- 172.21.15.16:0/1595907191 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ffa8406c2e0 0x7ffa8406e7a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.572+0000 7ffaa9111640 1 --2- 172.21.15.16:0/1595907191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffaa414ca20 0x7ffaa414e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.572+0000 7ffaa9111640 1 --2- 172.21.15.16:0/1595907191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffaa414bec0 0x7ffaa414c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.572+0000 7ffaa9111640 1 --2- 172.21.15.16:0/1595907191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa900039b0 0x7ffaa40b2520 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.572+0000 7ffaa9111640 1 -- 172.21.15.16:0/1595907191 >> 172.21.15.16:0/1595907191 conn(0x7ffaa409a9f0 msgr2=0x7ffaa40d1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:54.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.572+0000 7ffaa9111640 1 -- 172.21.15.16:0/1595907191 shutdown_connections 2024-09-19T07:54:54.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.572+0000 7ffaa9111640 1 -- 172.21.15.16:0/1595907191 wait complete. 2024-09-19T07:54:54.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2024-09-19T07:54:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.688+0000 7f1e24926640 1 Processor -- start 2024-09-19T07:54:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.688+0000 7f1e24926640 1 -- start start 2024-09-19T07:54:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.689+0000 7f1e24926640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1e201619b0 0x7f1e200d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.689+0000 7f1e24926640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1e200b0a80 0x7f1e200d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.689+0000 7f1e24926640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e2006c990 con 0x7f1e20161d90 2024-09-19T07:54:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.689+0000 7f1e24926640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f1e200d0ec0 con 0x7f1e201619b0 2024-09-19T07:54:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.689+0000 7f1e24926640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f1e200cec10 con 0x7f1e200b0a80 2024-09-19T07:54:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.689+0000 7f1e1e59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1e201619b0 0x7f1e200d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.689+0000 7f1e1e59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1e201619b0 0x7f1e200d0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50954/0 (socket says 172.21.15.16:50954) 2024-09-19T07:54:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.689+0000 7f1e1e59c640 1 -- 172.21.15.16:0/584953190 learned_addr learned my addr 172.21.15.16:0/584953190 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.689+0000 7f1e1dd9b640 1 --2- 172.21.15.16:0/584953190 >> v2:172.21.15.146:3300/0 conn(0x7f1e200b0a80 0x7f1e200d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.689+0000 7f1e1e59c640 1 -- 172.21.15.16:0/584953190 >> v2:172.21.15.146:3300/0 conn(0x7f1e200b0a80 msgr2=0x7f1e200d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.689+0000 7f1e1e59c640 1 --2- 172.21.15.16:0/584953190 >> v2:172.21.15.146:3300/0 conn(0x7f1e200b0a80 0x7f1e200d0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.689+0000 7f1e1e59c640 1 -- 172.21.15.16:0/584953190 >> v1:172.21.15.146:6789/0 conn(0x7f1e20161d90 legacy=0x7f1e200cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.689+0000 7f1e1e59c640 1 -- 172.21.15.16:0/584953190 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e200ce930 con 0x7f1e201619b0 2024-09-19T07:54:54.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.690+0000 7f1e1e59c640 1 --2- 172.21.15.16:0/584953190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e201619b0 0x7f1e200d0440 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f1e200d1040 tx=0x7f1e1002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70d313a6a04977d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:54.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.690+0000 7f1e1d59a640 1 -- 172.21.15.16:0/584953190 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10037070 con 0x7f1e201619b0 2024-09-19T07:54:54.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.690+0000 7f1e1d59a640 1 -- 172.21.15.16:0/584953190 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e10032440 con 0x7f1e201619b0 2024-09-19T07:54:54.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.690+0000 7f1e1d59a640 1 -- 172.21.15.16:0/584953190 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10032810 con 0x7f1e201619b0 2024-09-19T07:54:54.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.691+0000 7f1e24926640 1 -- 172.21.15.16:0/584953190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e201619b0 msgr2=0x7f1e200d0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:54.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.691+0000 7f1e24926640 1 --2- 172.21.15.16:0/584953190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e201619b0 0x7f1e200d0440 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f1e200d1040 tx=0x7f1e1002a210 comp rx=0 tx=0).stop 2024-09-19T07:54:54.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.691+0000 7f1e24926640 1 -- 172.21.15.16:0/584953190 shutdown_connections 2024-09-19T07:54:54.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.691+0000 7f1e24926640 1 --2- 172.21.15.16:0/584953190 >> v2:172.21.15.146:3300/0 conn(0x7f1e200b0a80 0x7f1e200d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.691+0000 7f1e24926640 1 --2- 172.21.15.16:0/584953190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e201619b0 0x7f1e200d0440 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.691+0000 7f1e24926640 1 -- 172.21.15.16:0/584953190 >> 172.21.15.16:0/584953190 conn(0x7f1e2009a9f0 msgr2=0x7f1e200b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:54.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.691+0000 7f1e24926640 1 -- 172.21.15.16:0/584953190 shutdown_connections 2024-09-19T07:54:54.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.691+0000 7f1e24926640 1 -- 172.21.15.16:0/584953190 wait complete. 2024-09-19T07:54:54.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.692+0000 7f1e24926640 1 Processor -- start 2024-09-19T07:54:54.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.692+0000 7f1e24926640 1 -- start start 2024-09-19T07:54:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.693+0000 7f1e24926640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e200b0a80 0x7f1e200b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.693+0000 7f1e24926640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e201619b0 0x7f1e200c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.693+0000 7f1e24926640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e20161d90 0x7f1e20152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.693+0000 7f1e24926640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2006c620 con 0x7f1e200b0a80 2024-09-19T07:54:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.693+0000 7f1e24926640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1e200b0820 con 0x7f1e201619b0 2024-09-19T07:54:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.693+0000 7f1e24926640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1e200ccce0 con 0x7f1e20161d90 2024-09-19T07:54:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.693+0000 7f1e1e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e200b0a80 0x7f1e200b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.693+0000 7f1e1e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e200b0a80 0x7f1e200b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50962/0 (socket says 172.21.15.16:50962) 2024-09-19T07:54:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.693+0000 7f1e1e59c640 1 -- 172.21.15.16:0/3814131332 learned_addr learned my addr 172.21.15.16:0/3814131332 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.693+0000 7f1e1dd9b640 1 --2- 172.21.15.16:0/3814131332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e201619b0 0x7f1e200c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.693+0000 7f1e1ed9d640 1 --2- 172.21.15.16:0/3814131332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e20161d90 0x7f1e20152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.693+0000 7f1e1e59c640 1 -- 172.21.15.16:0/3814131332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e20161d90 msgr2=0x7f1e20152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.693+0000 7f1e1e59c640 1 --2- 172.21.15.16:0/3814131332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e20161d90 0x7f1e20152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.693+0000 7f1e1e59c640 1 -- 172.21.15.16:0/3814131332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e201619b0 msgr2=0x7f1e200c4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.693+0000 7f1e1e59c640 1 --2- 172.21.15.16:0/3814131332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e201619b0 0x7f1e200c4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.693+0000 7f1e1e59c640 1 -- 172.21.15.16:0/3814131332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e200c54d0 con 0x7f1e200b0a80 2024-09-19T07:54:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.693+0000 7f1e1ed9d640 1 --2- 172.21.15.16:0/3814131332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e20161d90 0x7f1e20152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.694+0000 7f1e1e59c640 1 --2- 172.21.15.16:0/3814131332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e200b0a80 0x7f1e200b6d00 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f1e1002a250 tx=0x7f1e100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.694+0000 7f1e0effd640 1 -- 172.21.15.16:0/3814131332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10037040 con 0x7f1e200b0a80 2024-09-19T07:54:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.694+0000 7f1e0effd640 1 -- 172.21.15.16:0/3814131332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e10042070 con 0x7f1e200b0a80 2024-09-19T07:54:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.694+0000 7f1e24926640 1 -- 172.21.15.16:0/3814131332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e200c57b0 con 0x7f1e200b0a80 2024-09-19T07:54:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.694+0000 7f1e0effd640 1 -- 172.21.15.16:0/3814131332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10030bb0 con 0x7f1e200b0a80 2024-09-19T07:54:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.694+0000 7f1e24926640 1 -- 172.21.15.16:0/3814131332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e200c5d70 con 0x7f1e200b0a80 2024-09-19T07:54:54.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.695+0000 7f1e24926640 1 -- 172.21.15.16:0/3814131332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e200cd330 con 0x7f1e200b0a80 2024-09-19T07:54:54.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.695+0000 7f1e0effd640 1 -- 172.21.15.16:0/3814131332 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1e10047460 con 0x7f1e200b0a80 2024-09-19T07:54:54.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.696+0000 7f1e0effd640 1 --2- 172.21.15.16:0/3814131332 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1e0006c2b0 0x7f1e0006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:54.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.696+0000 7f1e0effd640 1 -- 172.21.15.16:0/3814131332 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5335+0+0 (secure 0 0 0) 0x7f1e100abf20 con 0x7f1e200b0a80 2024-09-19T07:54:54.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.696+0000 7f1e1dd9b640 1 --2- 172.21.15.16:0/3814131332 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1e0006c2b0 0x7f1e0006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:54.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.697+0000 7f1e1dd9b640 1 --2- 172.21.15.16:0/3814131332 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1e0006c2b0 0x7f1e0006e770 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1e2006f460 tx=0x7f1e14008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.700+0000 7f1e0effd640 1 -- 172.21.15.16:0/3814131332 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1e1007a7f0 con 0x7f1e200b0a80 2024-09-19T07:54:54.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.875+0000 7f1e24926640 1 -- 172.21.15.16:0/3814131332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) -- 0x7f1e2006f1b0 con 0x7f1e200b0a80 2024-09-19T07:54:54.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.878+0000 7f1e0effd640 1 -- 172.21.15.16:0/3814131332 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v60) ==== 102+0+0 (secure 0 0 0) 0x7f1e1007a0f0 con 0x7f1e200b0a80 2024-09-19T07:54:54.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.880+0000 7f1e24926640 1 -- 172.21.15.16:0/3814131332 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1e0006c2b0 msgr2=0x7f1e0006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:54.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.880+0000 7f1e24926640 1 --2- 172.21.15.16:0/3814131332 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1e0006c2b0 0x7f1e0006e770 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1e2006f460 tx=0x7f1e14008040 comp rx=0 tx=0).stop 2024-09-19T07:54:54.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.880+0000 7f1e24926640 1 -- 172.21.15.16:0/3814131332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e200b0a80 msgr2=0x7f1e200b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:54.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.880+0000 7f1e24926640 1 --2- 172.21.15.16:0/3814131332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e200b0a80 0x7f1e200b6d00 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f1e1002a250 tx=0x7f1e100046c0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.880+0000 7f1e24926640 1 -- 172.21.15.16:0/3814131332 shutdown_connections 2024-09-19T07:54:54.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.880+0000 7f1e24926640 1 --2- 172.21.15.16:0/3814131332 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1e0006c2b0 0x7f1e0006e770 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.880+0000 7f1e24926640 1 --2- 172.21.15.16:0/3814131332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e20161d90 0x7f1e20152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.880+0000 7f1e24926640 1 --2- 172.21.15.16:0/3814131332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e201619b0 0x7f1e200c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.880+0000 7f1e24926640 1 --2- 172.21.15.16:0/3814131332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e200b0a80 0x7f1e200b6d00 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:54.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.880+0000 7f1e24926640 1 -- 172.21.15.16:0/3814131332 >> 172.21.15.16:0/3814131332 conn(0x7f1e2009a9f0 msgr2=0x7f1e200ae370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:54.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.880+0000 7f1e24926640 1 -- 172.21.15.16:0/3814131332 shutdown_connections 2024-09-19T07:54:54.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:54.880+0000 7f1e24926640 1 -- 172.21.15.16:0/3814131332 wait complete. 2024-09-19T07:54:54.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T07:54:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.000+0000 7f3c3440e640 1 Processor -- start 2024-09-19T07:54:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.000+0000 7f3c3440e640 1 -- start start 2024-09-19T07:54:55.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.001+0000 7f3c3440e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3c2c0c8d20 0x7f3c2c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:55.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.001+0000 7f3c3440e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c2c06d260 0x7f3c2c1439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:55.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.001+0000 7f3c3440e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c2c0c7140 con 0x7f3c2c143f30 2024-09-19T07:54:55.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.001+0000 7f3c3440e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f3c2c141960 con 0x7f3c2c0c8d20 2024-09-19T07:54:55.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.001+0000 7f3c3440e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2c142200 con 0x7f3c2c06d260 2024-09-19T07:54:55.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.001+0000 7f3c319a9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c2c06d260 0x7f3c2c1439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.001+0000 7f3c321aa640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3c2c0c8d20 0x7f3c2c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.001+0000 7f3c319a9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c2c06d260 0x7f3c2c1439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:60068/0 (socket says 172.21.15.16:60068) 2024-09-19T07:54:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.001+0000 7f3c319a9640 1 -- 172.21.15.16:0/1211077220 learned_addr learned my addr 172.21.15.16:0/1211077220 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.002+0000 7f3c319a9640 1 -- 172.21.15.16:0/1211077220 >> v2:172.21.15.146:3300/0 conn(0x7f3c2c0c8d20 msgr2=0x7f3c2c06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.002+0000 7f3c319a9640 1 --2- 172.21.15.16:0/1211077220 >> v2:172.21.15.146:3300/0 conn(0x7f3c2c0c8d20 0x7f3c2c06cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.002+0000 7f3c319a9640 1 -- 172.21.15.16:0/1211077220 >> v1:172.21.15.146:6789/0 conn(0x7f3c2c143f30 legacy=0x7f3c2c0d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.002+0000 7f3c311a8640 1 -- 172.21.15.16:0/1211077220 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3209724094 0 0) 0x7f3c24001490 con 0x7f3c2c143f30 2024-09-19T07:54:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.002+0000 7f3c319a9640 1 -- 172.21.15.16:0/1211077220 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2c0c9100 con 0x7f3c2c06d260 2024-09-19T07:54:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.002+0000 7f3c321aa640 1 --2- 172.21.15.16:0/1211077220 >> v2:172.21.15.146:3300/0 conn(0x7f3c2c0c8d20 0x7f3c2c06cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:54:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.002+0000 7f3c319a9640 1 --2- 172.21.15.16:0/1211077220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c2c06d260 0x7f3c2c1439f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c009870 tx=0x7f3c1c02a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28b9ee3c6aa88010 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.003+0000 7f3c311a8640 1 -- 172.21.15.16:0/1211077220 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c037070 con 0x7f3c2c06d260 2024-09-19T07:54:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.003+0000 7f3c311a8640 1 -- 172.21.15.16:0/1211077220 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c1c032440 con 0x7f3c2c06d260 2024-09-19T07:54:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.003+0000 7f3c311a8640 1 -- 172.21.15.16:0/1211077220 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c032810 con 0x7f3c2c06d260 2024-09-19T07:54:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.003+0000 7f3c3440e640 1 -- 172.21.15.16:0/1211077220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c2c06d260 msgr2=0x7f3c2c1439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.003+0000 7f3c3440e640 1 --2- 172.21.15.16:0/1211077220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c2c06d260 0x7f3c2c1439f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c009870 tx=0x7f3c1c02a130 comp rx=0 tx=0).stop 2024-09-19T07:54:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.003+0000 7f3c3440e640 1 -- 172.21.15.16:0/1211077220 shutdown_connections 2024-09-19T07:54:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.003+0000 7f3c3440e640 1 --2- 172.21.15.16:0/1211077220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c2c06d260 0x7f3c2c1439f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.003+0000 7f3c3440e640 1 --2- 172.21.15.16:0/1211077220 >> v2:172.21.15.146:3300/0 conn(0x7f3c2c0c8d20 0x7f3c2c06cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.003+0000 7f3c3440e640 1 -- 172.21.15.16:0/1211077220 >> 172.21.15.16:0/1211077220 conn(0x7f3c2c0b3ed0 msgr2=0x7f3c2c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:55.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.003+0000 7f3c3440e640 1 -- 172.21.15.16:0/1211077220 shutdown_connections 2024-09-19T07:54:55.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.003+0000 7f3c3440e640 1 -- 172.21.15.16:0/1211077220 wait complete. 2024-09-19T07:54:55.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.004+0000 7f3c3440e640 1 Processor -- start 2024-09-19T07:54:55.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.004+0000 7f3c3440e640 1 -- start start 2024-09-19T07:54:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.005+0000 7f3c3440e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c06d260 0x7f3c2c0cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.005+0000 7f3c3440e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c2c0c8d20 0x7f3c2c0cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.005+0000 7f3c3440e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c2c143f30 0x7f3c2c0b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.005+0000 7f3c3440e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c141bb0 con 0x7f3c2c06d260 2024-09-19T07:54:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.005+0000 7f3c3440e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c06d6a0 con 0x7f3c2c143f30 2024-09-19T07:54:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.005+0000 7f3c3440e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2c141e90 con 0x7f3c2c0c8d20 2024-09-19T07:54:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.005+0000 7f3c321aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c06d260 0x7f3c2c0cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.005+0000 7f3c321aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c06d260 0x7f3c2c0cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50972/0 (socket says 172.21.15.16:50972) 2024-09-19T07:54:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.005+0000 7f3c321aa640 1 -- 172.21.15.16:0/1475504471 learned_addr learned my addr 172.21.15.16:0/1475504471 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.005+0000 7f3c319a9640 1 --2- 172.21.15.16:0/1475504471 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c2c0c8d20 0x7f3c2c0cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.005+0000 7f3c329ab640 1 --2- 172.21.15.16:0/1475504471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c2c143f30 0x7f3c2c0b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.006+0000 7f3c321aa640 1 -- 172.21.15.16:0/1475504471 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c2c0c8d20 msgr2=0x7f3c2c0cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.006+0000 7f3c321aa640 1 --2- 172.21.15.16:0/1475504471 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c2c0c8d20 0x7f3c2c0cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.006+0000 7f3c321aa640 1 -- 172.21.15.16:0/1475504471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c2c143f30 msgr2=0x7f3c2c0b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.006+0000 7f3c321aa640 1 --2- 172.21.15.16:0/1475504471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c2c143f30 0x7f3c2c0b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.006+0000 7f3c321aa640 1 -- 172.21.15.16:0/1475504471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2c0d1420 con 0x7f3c2c06d260 2024-09-19T07:54:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.006+0000 7f3c321aa640 1 --2- 172.21.15.16:0/1475504471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c06d260 0x7f3c2c0cd230 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800e960 tx=0x7f3c2800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.006+0000 7f3c22ffd640 1 -- 172.21.15.16:0/1475504471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28017070 con 0x7f3c2c06d260 2024-09-19T07:54:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.006+0000 7f3c22ffd640 1 -- 172.21.15.16:0/1475504471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c2800ca00 con 0x7f3c2c06d260 2024-09-19T07:54:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.006+0000 7f3c3440e640 1 -- 172.21.15.16:0/1475504471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2c0d16c0 con 0x7f3c2c06d260 2024-09-19T07:54:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.006+0000 7f3c22ffd640 1 -- 172.21.15.16:0/1475504471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2800cdd0 con 0x7f3c2c06d260 2024-09-19T07:54:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.006+0000 7f3c3440e640 1 -- 172.21.15.16:0/1475504471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c2c0d1c80 con 0x7f3c2c06d260 2024-09-19T07:54:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.007+0000 7f3c3440e640 1 -- 172.21.15.16:0/1475504471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c2c144360 con 0x7f3c2c06d260 2024-09-19T07:54:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.008+0000 7f3c22ffd640 1 -- 172.21.15.16:0/1475504471 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3c2801a070 con 0x7f3c2c06d260 2024-09-19T07:54:55.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.008+0000 7f3c22ffd640 1 --2- 172.21.15.16:0/1475504471 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3c1006c2b0 0x7f3c1006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:55.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.008+0000 7f3c22ffd640 1 -- 172.21.15.16:0/1475504471 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5335+0+0 (secure 0 0 0) 0x7f3c2808b200 con 0x7f3c2c06d260 2024-09-19T07:54:55.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.008+0000 7f3c319a9640 1 --2- 172.21.15.16:0/1475504471 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3c1006c2b0 0x7f3c1006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:55.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.009+0000 7f3c319a9640 1 --2- 172.21.15.16:0/1475504471 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3c1006c2b0 0x7f3c1006e770 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c034000 tx=0x7f3c1c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.012+0000 7f3c22ffd640 1 -- 172.21.15.16:0/1475504471 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3c28059a50 con 0x7f3c2c06d260 2024-09-19T07:54:55.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.188+0000 7f3c3440e640 1 -- 172.21.15.16:0/1475504471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f3c2c0d10d0 con 0x7f3c2c06d260 2024-09-19T07:54:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.191+0000 7f3c22ffd640 1 -- 172.21.15.16:0/1475504471 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v61) ==== 104+0+0 (secure 0 0 0) 0x7f3c28059350 con 0x7f3c2c06d260 2024-09-19T07:54:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.193+0000 7f3c3440e640 1 -- 172.21.15.16:0/1475504471 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3c1006c2b0 msgr2=0x7f3c1006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.193+0000 7f3c3440e640 1 --2- 172.21.15.16:0/1475504471 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3c1006c2b0 0x7f3c1006e770 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c034000 tx=0x7f3c1c02e000 comp rx=0 tx=0).stop 2024-09-19T07:54:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.193+0000 7f3c3440e640 1 -- 172.21.15.16:0/1475504471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c06d260 msgr2=0x7f3c2c0cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.193+0000 7f3c3440e640 1 --2- 172.21.15.16:0/1475504471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c06d260 0x7f3c2c0cd230 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800e960 tx=0x7f3c2800ee20 comp rx=0 tx=0).stop 2024-09-19T07:54:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.193+0000 7f3c3440e640 1 -- 172.21.15.16:0/1475504471 shutdown_connections 2024-09-19T07:54:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.193+0000 7f3c3440e640 1 --2- 172.21.15.16:0/1475504471 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3c1006c2b0 0x7f3c1006e770 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.193+0000 7f3c3440e640 1 --2- 172.21.15.16:0/1475504471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c2c143f30 0x7f3c2c0b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.193+0000 7f3c3440e640 1 --2- 172.21.15.16:0/1475504471 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c2c0c8d20 0x7f3c2c0cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.193+0000 7f3c3440e640 1 --2- 172.21.15.16:0/1475504471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c06d260 0x7f3c2c0cd230 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.193+0000 7f3c3440e640 1 -- 172.21.15.16:0/1475504471 >> 172.21.15.16:0/1475504471 conn(0x7f3c2c0b3ed0 msgr2=0x7f3c2c15fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.193+0000 7f3c3440e640 1 -- 172.21.15.16:0/1475504471 shutdown_connections 2024-09-19T07:54:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.193+0000 7f3c3440e640 1 -- 172.21.15.16:0/1475504471 wait complete. 2024-09-19T07:54:55.205 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2024-09-19T07:54:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.309+0000 7efea6aba640 1 Processor -- start 2024-09-19T07:54:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.309+0000 7efea6aba640 1 -- start start 2024-09-19T07:54:55.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.309+0000 7efea6aba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efea0142170 0x7efea0166d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:55.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.310+0000 7efea6aba640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efea015d710 con 0x7efea00d0540 2024-09-19T07:54:55.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.310+0000 7efea6aba640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efea015c950 con 0x7efea0141680 2024-09-19T07:54:55.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.310+0000 7efea6aba640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7efea015c6f0 con 0x7efea0142170 2024-09-19T07:54:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.310+0000 7efea5057640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efea0142170 0x7efea0166d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.310+0000 7efe9ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efea00d0540 0x7efea0141a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33774/0 (socket says 172.21.15.16:33774) 2024-09-19T07:54:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.310+0000 7efe9ffff640 1 -- 172.21.15.16:0/64394132 learned_addr learned my addr 172.21.15.16:0/64394132 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.310+0000 7efea5057640 1 -- 172.21.15.16:0/64394132 >> v1:172.21.15.146:6789/0 conn(0x7efea0141680 legacy=0x7efea00cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.310+0000 7efea5057640 1 -- 172.21.15.16:0/64394132 >> v1:172.21.15.16:6789/0 conn(0x7efea00d0540 legacy=0x7efea0141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.310+0000 7efea5057640 1 -- 172.21.15.16:0/64394132 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea015cc30 con 0x7efea0142170 2024-09-19T07:54:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.310+0000 7efe9f7fe640 1 -- 172.21.15.16:0/64394132 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1223544808 0 0) 0x7efea015d710 con 0x7efea00d0540 2024-09-19T07:54:55.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.311+0000 7efea5057640 1 --2- 172.21.15.16:0/64394132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea0142170 0x7efea0166d70 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7efe98009d90 tx=0x7efe9802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5257b6aeb00e977 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:55.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.311+0000 7efe9f7fe640 1 -- 172.21.15.16:0/64394132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98007740 con 0x7efea0142170 2024-09-19T07:54:55.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.311+0000 7efe9f7fe640 1 -- 172.21.15.16:0/64394132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe980078e0 con 0x7efea0142170 2024-09-19T07:54:55.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.311+0000 7efe9f7fe640 1 -- 172.21.15.16:0/64394132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98007cb0 con 0x7efea0142170 2024-09-19T07:54:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.311+0000 7efea6aba640 1 -- 172.21.15.16:0/64394132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea0142170 msgr2=0x7efea0166d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.311+0000 7efea6aba640 1 --2- 172.21.15.16:0/64394132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea0142170 0x7efea0166d70 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7efe98009d90 tx=0x7efe9802a6f0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.311+0000 7efea6aba640 1 -- 172.21.15.16:0/64394132 shutdown_connections 2024-09-19T07:54:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.311+0000 7efea6aba640 1 --2- 172.21.15.16:0/64394132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea0142170 0x7efea0166d70 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.311+0000 7efea6aba640 1 -- 172.21.15.16:0/64394132 >> 172.21.15.16:0/64394132 conn(0x7efea00b3ed0 msgr2=0x7efea00b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.311+0000 7efea6aba640 1 -- 172.21.15.16:0/64394132 shutdown_connections 2024-09-19T07:54:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.312+0000 7efea6aba640 1 -- 172.21.15.16:0/64394132 wait complete. 2024-09-19T07:54:55.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.313+0000 7efea6aba640 1 Processor -- start 2024-09-19T07:54:55.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.313+0000 7efea6aba640 1 -- start start 2024-09-19T07:54:55.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efea6aba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efea00d0540 0x7efea0155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:55.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efea6aba640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efea0141680 0x7efea00c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:55.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efea6aba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea0142170 0x7efea00b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:55.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efea6aba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efea01425c0 con 0x7efea0142170 2024-09-19T07:54:55.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efea6aba640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efea015d1f0 con 0x7efea0141680 2024-09-19T07:54:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efea6aba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efea01408c0 con 0x7efea00d0540 2024-09-19T07:54:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efea5057640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea0142170 0x7efea00b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efea5057640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea0142170 0x7efea00b51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50996/0 (socket says 172.21.15.16:50996) 2024-09-19T07:54:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efea5057640 1 -- 172.21.15.16:0/743284595 learned_addr learned my addr 172.21.15.16:0/743284595 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efe9ffff640 1 --2- 172.21.15.16:0/743284595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efea0141680 0x7efea00c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efea4856640 1 --2- 172.21.15.16:0/743284595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efea00d0540 0x7efea0155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efea5057640 1 -- 172.21.15.16:0/743284595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efea00d0540 msgr2=0x7efea0155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efea5057640 1 --2- 172.21.15.16:0/743284595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efea00d0540 0x7efea0155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efea5057640 1 -- 172.21.15.16:0/743284595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efea0141680 msgr2=0x7efea00c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efea5057640 1 --2- 172.21.15.16:0/743284595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efea0141680 0x7efea00c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efea5057640 1 -- 172.21.15.16:0/743284595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea01e41f0 con 0x7efea0142170 2024-09-19T07:54:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efe9ffff640 1 --2- 172.21.15.16:0/743284595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efea0141680 0x7efea00c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efea5057640 1 --2- 172.21.15.16:0/743284595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea0142170 0x7efea00b51d0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7efe98009eb0 tx=0x7efe98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efe9d7fa640 1 -- 172.21.15.16:0/743284595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98040070 con 0x7efea0142170 2024-09-19T07:54:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efe9d7fa640 1 -- 172.21.15.16:0/743284595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe98031c40 con 0x7efea0142170 2024-09-19T07:54:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efea6aba640 1 -- 172.21.15.16:0/743284595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea01e3430 con 0x7efea0142170 2024-09-19T07:54:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.314+0000 7efe9d7fa640 1 -- 172.21.15.16:0/743284595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98003990 con 0x7efea0142170 2024-09-19T07:54:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.315+0000 7efea6aba640 1 -- 172.21.15.16:0/743284595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea01e39f0 con 0x7efea0142170 2024-09-19T07:54:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.315+0000 7efea6aba640 1 -- 172.21.15.16:0/743284595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efea0141a60 con 0x7efea0142170 2024-09-19T07:54:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.316+0000 7efe9d7fa640 1 -- 172.21.15.16:0/743284595 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7efe98039040 con 0x7efea0142170 2024-09-19T07:54:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.316+0000 7efe9d7fa640 1 --2- 172.21.15.16:0/743284595 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efe8406c2b0 0x7efe8406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.316+0000 7efe9d7fa640 1 -- 172.21.15.16:0/743284595 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5335+0+0 (secure 0 0 0) 0x7efe980ac4a0 con 0x7efea0142170 2024-09-19T07:54:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.317+0000 7efea4856640 1 --2- 172.21.15.16:0/743284595 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efe8406c2b0 0x7efe8406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.317+0000 7efea4856640 1 --2- 172.21.15.16:0/743284595 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efe8406c2b0 0x7efe8406e770 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7efe94000f60 tx=0x7efe94009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:55.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.321+0000 7efe9d7fa640 1 -- 172.21.15.16:0/743284595 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7efe9807a7c0 con 0x7efea0142170 2024-09-19T07:54:55.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.497+0000 7efea6aba640 1 -- 172.21.15.16:0/743284595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) -- 0x7efea00690c0 con 0x7efea0142170 2024-09-19T07:54:55.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.501+0000 7efe9d7fa640 1 -- 172.21.15.16:0/743284595 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v62) ==== 105+0+0 (secure 0 0 0) 0x7efe9807eac0 con 0x7efea0142170 2024-09-19T07:54:55.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.503+0000 7efea6aba640 1 -- 172.21.15.16:0/743284595 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efe8406c2b0 msgr2=0x7efe8406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:55.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.503+0000 7efea6aba640 1 --2- 172.21.15.16:0/743284595 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efe8406c2b0 0x7efe8406e770 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7efe94000f60 tx=0x7efe94009040 comp rx=0 tx=0).stop 2024-09-19T07:54:55.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.503+0000 7efea6aba640 1 -- 172.21.15.16:0/743284595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea0142170 msgr2=0x7efea00b51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:55.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.503+0000 7efea6aba640 1 --2- 172.21.15.16:0/743284595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea0142170 0x7efea00b51d0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7efe98009eb0 tx=0x7efe98004a40 comp rx=0 tx=0).stop 2024-09-19T07:54:55.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.503+0000 7efea6aba640 1 -- 172.21.15.16:0/743284595 shutdown_connections 2024-09-19T07:54:55.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.503+0000 7efea6aba640 1 --2- 172.21.15.16:0/743284595 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efe8406c2b0 0x7efe8406e770 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.503+0000 7efea6aba640 1 --2- 172.21.15.16:0/743284595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea0142170 0x7efea00b51d0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.503+0000 7efea6aba640 1 --2- 172.21.15.16:0/743284595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efea0141680 0x7efea00c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.503+0000 7efea6aba640 1 --2- 172.21.15.16:0/743284595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efea00d0540 0x7efea0155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.503+0000 7efea6aba640 1 -- 172.21.15.16:0/743284595 >> 172.21.15.16:0/743284595 conn(0x7efea00b3ed0 msgr2=0x7efea013e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:55.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.503+0000 7efea6aba640 1 -- 172.21.15.16:0/743284595 shutdown_connections 2024-09-19T07:54:55.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.503+0000 7efea6aba640 1 -- 172.21.15.16:0/743284595 wait complete. 2024-09-19T07:54:55.514 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps' 2024-09-19T07:54:55.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.617+0000 7f87d2566640 1 Processor -- start 2024-09-19T07:54:55.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.618+0000 7f87d2566640 1 -- start start 2024-09-19T07:54:55.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.618+0000 7f87d2566640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f87cc0d0070 0x7f87cc15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:55.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.618+0000 7f87d2566640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87cc0d1b10 con 0x7f87cc0d4130 2024-09-19T07:54:55.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.618+0000 7f87d2566640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87cc0cf2b0 con 0x7f87cc15f0d0 2024-09-19T07:54:55.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.618+0000 7f87d2566640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f87cc0cf590 con 0x7f87cc0d0070 2024-09-19T07:54:55.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.618+0000 7f87d0b03640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f87cc0d4130 0x7f87cc0d4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33782/0 (socket says 172.21.15.16:33782) 2024-09-19T07:54:55.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.618+0000 7f87d0b03640 1 -- 172.21.15.16:0/3146248411 learned_addr learned my addr 172.21.15.16:0/3146248411 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:55.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87caffd640 1 -- 172.21.15.16:0/3146248411 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1226645580 0 0) 0x7f87cc0d1b10 con 0x7f87cc0d4130 2024-09-19T07:54:55.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87caffd640 1 -- 172.21.15.16:0/3146248411 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87b4003520 con 0x7f87cc0d4130 2024-09-19T07:54:55.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87cb7fe640 1 --2- 172.21.15.16:0/3146248411 >> v2:172.21.15.146:3300/0 conn(0x7f87cc0d0070 0x7f87cc15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:55.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87caffd640 1 -- 172.21.15.16:0/3146248411 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4160991665 0 0) 0x7f87b4003520 con 0x7f87cc0d4130 2024-09-19T07:54:55.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87caffd640 1 -- 172.21.15.16:0/3146248411 >> v2:172.21.15.146:3300/0 conn(0x7f87cc0d0070 msgr2=0x7f87cc15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:55.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87caffd640 1 --2- 172.21.15.16:0/3146248411 >> v2:172.21.15.146:3300/0 conn(0x7f87cc0d0070 0x7f87cc15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87caffd640 1 -- 172.21.15.16:0/3146248411 >> v1:172.21.15.146:6789/0 conn(0x7f87cc15f0d0 legacy=0x7f87cc0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:55.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87caffd640 1 -- 172.21.15.16:0/3146248411 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87cc0cf870 con 0x7f87cc0d4130 2024-09-19T07:54:55.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87caffd640 1 -- 172.21.15.16:0/3146248411 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f87c4001a10 con 0x7f87cc0d4130 2024-09-19T07:54:55.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87caffd640 1 -- 172.21.15.16:0/3146248411 >> v1:172.21.15.16:6789/0 conn(0x7f87cc0d4130 legacy=0x7f87cc0d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:55.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87caffd640 1 --2- 172.21.15.16:0/3146248411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87b40039b0 0x7f87b4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:55.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87caffd640 1 --2- 172.21.15.16:0/3146248411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4004800 0x7f87b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:55.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87caffd640 1 --2- 172.21.15.16:0/3146248411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87b40052e0 0x7f87b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:55.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87caffd640 1 -- 172.21.15.16:0/3146248411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc0cf590 con 0x7f87b4004800 2024-09-19T07:54:55.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87caffd640 1 -- 172.21.15.16:0/3146248411 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f87b400c230 con 0x7f87b40052e0 2024-09-19T07:54:55.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87caffd640 1 -- 172.21.15.16:0/3146248411 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f87b400c430 con 0x7f87b40039b0 2024-09-19T07:54:55.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87cb7fe640 1 --2- 172.21.15.16:0/3146248411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4004800 0x7f87b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:55.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87cbfff640 1 --2- 172.21.15.16:0/3146248411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87b40039b0 0x7f87b4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:55.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87cb7fe640 1 -- 172.21.15.16:0/3146248411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87b40039b0 msgr2=0x7f87b4003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:55.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87cb7fe640 1 --2- 172.21.15.16:0/3146248411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87b40039b0 0x7f87b4003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87cb7fe640 1 -- 172.21.15.16:0/3146248411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87b40052e0 msgr2=0x7f87b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:55.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.619+0000 7f87cb7fe640 1 --2- 172.21.15.16:0/3146248411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87b40052e0 0x7f87b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.620+0000 7f87d0b03640 1 -- 172.21.15.16:0/3146248411 reap_dead start 2024-09-19T07:54:55.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.620+0000 7f87cb7fe640 1 -- 172.21.15.16:0/3146248411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b4003520 con 0x7f87b4004800 2024-09-19T07:54:55.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.620+0000 7f87cb7fe640 1 --2- 172.21.15.16:0/3146248411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4004800 0x7f87b4004c80 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f87c000ba00 tx=0x7f87b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746c1e5b7acb589d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:55.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.620+0000 7f87caffd640 1 -- 172.21.15.16:0/3146248411 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c0030070 con 0x7f87b4004800 2024-09-19T07:54:55.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.620+0000 7f87caffd640 1 -- 172.21.15.16:0/3146248411 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c002adb0 con 0x7f87b4004800 2024-09-19T07:54:55.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.620+0000 7f87caffd640 1 -- 172.21.15.16:0/3146248411 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c000b600 con 0x7f87b4004800 2024-09-19T07:54:55.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.621+0000 7f87d2566640 1 -- 172.21.15.16:0/3146248411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4004800 msgr2=0x7f87b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:55.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.621+0000 7f87d2566640 1 --2- 172.21.15.16:0/3146248411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4004800 0x7f87b4004c80 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f87c000ba00 tx=0x7f87b4005830 comp rx=0 tx=0).stop 2024-09-19T07:54:55.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.621+0000 7f87d2566640 1 -- 172.21.15.16:0/3146248411 shutdown_connections 2024-09-19T07:54:55.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.621+0000 7f87d2566640 1 --2- 172.21.15.16:0/3146248411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4004800 0x7f87b4004c80 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.621+0000 7f87d2566640 1 -- 172.21.15.16:0/3146248411 >> 172.21.15.16:0/3146248411 conn(0x7f87cc09d760 msgr2=0x7f87cc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:55.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.621+0000 7f87d2566640 1 -- 172.21.15.16:0/3146248411 shutdown_connections 2024-09-19T07:54:55.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.621+0000 7f87d2566640 1 -- 172.21.15.16:0/3146248411 wait complete. 2024-09-19T07:54:55.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.622+0000 7f87d2566640 1 Processor -- start 2024-09-19T07:54:55.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.622+0000 7f87d2566640 1 -- start start 2024-09-19T07:54:55.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.623+0000 7f87d2566640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4004800 0x7f87cc154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.623+0000 7f87d2566640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87cc157fc0 0x7f87cc0b96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.623+0000 7f87d2566640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87cc157a70 0x7f87cc0b9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.623+0000 7f87d2566640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc15f4b0 con 0x7f87b4004800 2024-09-19T07:54:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.623+0000 7f87d2566640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc0cef60 con 0x7f87cc157fc0 2024-09-19T07:54:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.623+0000 7f87d2566640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f87cc15f630 con 0x7f87cc157a70 2024-09-19T07:54:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.623+0000 7f87cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4004800 0x7f87cc154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.623+0000 7f87cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4004800 0x7f87cc154aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51020/0 (socket says 172.21.15.16:51020) 2024-09-19T07:54:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.623+0000 7f87cbfff640 1 -- 172.21.15.16:0/2597555784 learned_addr learned my addr 172.21.15.16:0/2597555784 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.623+0000 7f87cb7fe640 1 --2- 172.21.15.16:0/2597555784 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87cc157fc0 0x7f87cc0b96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.623+0000 7f87cbfff640 1 -- 172.21.15.16:0/2597555784 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87cc157a70 msgr2=0x7f87cc0b9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.624+0000 7f87cbfff640 1 --2- 172.21.15.16:0/2597555784 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87cc157a70 0x7f87cc0b9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.624+0000 7f87cbfff640 1 -- 172.21.15.16:0/2597555784 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87cc157fc0 msgr2=0x7f87cc0b96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.624+0000 7f87cbfff640 1 --2- 172.21.15.16:0/2597555784 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87cc157fc0 0x7f87cc0b96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.624+0000 7f87cbfff640 1 -- 172.21.15.16:0/2597555784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87cc0b73b0 con 0x7f87b4004800 2024-09-19T07:54:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.624+0000 7f87cb7fe640 1 --2- 172.21.15.16:0/2597555784 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87cc157fc0 0x7f87cc0b96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.624+0000 7f87cbfff640 1 --2- 172.21.15.16:0/2597555784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4004800 0x7f87cc154aa0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00f870 tx=0x7f87bc00fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.624+0000 7f87c8ff9640 1 -- 172.21.15.16:0/2597555784 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc018020 con 0x7f87b4004800 2024-09-19T07:54:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.624+0000 7f87c8ff9640 1 -- 172.21.15.16:0/2597555784 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87bc005490 con 0x7f87b4004800 2024-09-19T07:54:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.624+0000 7f87c8ff9640 1 -- 172.21.15.16:0/2597555784 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc00db80 con 0x7f87b4004800 2024-09-19T07:54:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.624+0000 7f87d2566640 1 -- 172.21.15.16:0/2597555784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87cc0b7d00 con 0x7f87b4004800 2024-09-19T07:54:55.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.624+0000 7f87d2566640 1 -- 172.21.15.16:0/2597555784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87cc1611f0 con 0x7f87b4004800 2024-09-19T07:54:55.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.625+0000 7f87d2566640 1 -- 172.21.15.16:0/2597555784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87cc0d4560 con 0x7f87b4004800 2024-09-19T07:54:55.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.625+0000 7f87c8ff9640 1 -- 172.21.15.16:0/2597555784 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f87bc01b020 con 0x7f87b4004800 2024-09-19T07:54:55.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.626+0000 7f87c8ff9640 1 --2- 172.21.15.16:0/2597555784 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f87b806c2e0 0x7f87b806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:55.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.626+0000 7f87c8ff9640 1 -- 172.21.15.16:0/2597555784 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5335+0+0 (secure 0 0 0) 0x7f87bc08b6e0 con 0x7f87b4004800 2024-09-19T07:54:55.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.626+0000 7f87cb7fe640 1 --2- 172.21.15.16:0/2597555784 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f87b806c2e0 0x7f87b806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:55.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.627+0000 7f87cb7fe640 1 --2- 172.21.15.16:0/2597555784 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f87b806c2e0 0x7f87b806e7a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f87b40057e0 tx=0x7f87c0030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:55.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.630+0000 7f87c8ff9640 1 -- 172.21.15.16:0/2597555784 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f87bc022020 con 0x7f87b4004800 2024-09-19T07:54:55.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.877+0000 7f87d2566640 1 -- 172.21.15.16:0/2597555784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]} v 0) -- 0x7f87cc06cf30 con 0x7f87b4004800 2024-09-19T07:54:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.939+0000 7f87c8ff9640 1 -- 172.21.15.16:0/2597555784 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]}]=0 v532) ==== 151+0+0 (secure 0 0 0) 0x7f87bc05a110 con 0x7f87b4004800 2024-09-19T07:54:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.941+0000 7f87d2566640 1 -- 172.21.15.16:0/2597555784 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f87b806c2e0 msgr2=0x7f87b806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.941+0000 7f87d2566640 1 --2- 172.21.15.16:0/2597555784 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f87b806c2e0 0x7f87b806e7a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f87b40057e0 tx=0x7f87c0030000 comp rx=0 tx=0).stop 2024-09-19T07:54:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.941+0000 7f87d2566640 1 -- 172.21.15.16:0/2597555784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4004800 msgr2=0x7f87cc154aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.941+0000 7f87d2566640 1 --2- 172.21.15.16:0/2597555784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4004800 0x7f87cc154aa0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00f870 tx=0x7f87bc00fd30 comp rx=0 tx=0).stop 2024-09-19T07:54:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.942+0000 7f87d2566640 1 -- 172.21.15.16:0/2597555784 shutdown_connections 2024-09-19T07:54:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.942+0000 7f87d2566640 1 --2- 172.21.15.16:0/2597555784 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f87b806c2e0 0x7f87b806e7a0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.942+0000 7f87d2566640 1 --2- 172.21.15.16:0/2597555784 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87cc157a70 0x7f87cc0b9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.942+0000 7f87d2566640 1 --2- 172.21.15.16:0/2597555784 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87cc157fc0 0x7f87cc0b96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.942+0000 7f87d2566640 1 --2- 172.21.15.16:0/2597555784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4004800 0x7f87cc154aa0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.942+0000 7f87d2566640 1 -- 172.21.15.16:0/2597555784 >> 172.21.15.16:0/2597555784 conn(0x7f87cc09d760 msgr2=0x7f87cc15fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.942+0000 7f87d2566640 1 -- 172.21.15.16:0/2597555784 shutdown_connections 2024-09-19T07:54:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:55.942+0000 7f87d2566640 1 -- 172.21.15.16:0/2597555784 wait complete. 2024-09-19T07:54:55.953 INFO:tasks.cephfs_test_runner:test_client_pin_mincaps (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2024-09-19T07:54:55.953 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin_root (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-19T07:54:55.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:54:56.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.056+0000 7f7f18fd8640 1 Processor -- start 2024-09-19T07:54:56.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.056+0000 7f7f18fd8640 1 -- start start 2024-09-19T07:54:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.057+0000 7f7f18fd8640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7f140c8d20 0x7f7f1406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.057+0000 7f7f18fd8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f140c7140 con 0x7f7f1406d260 2024-09-19T07:54:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.057+0000 7f7f18fd8640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f140c9100 con 0x7f7f14144100 2024-09-19T07:54:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.057+0000 7f7f18fd8640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f7f1406d6d0 con 0x7f7f140c8d20 2024-09-19T07:54:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.057+0000 7f7f0bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7f1406d260 0x7f7f141439f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33798/0 (socket says 172.21.15.16:33798) 2024-09-19T07:54:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.057+0000 7f7f0bfff640 1 -- 172.21.15.16:0/382535476 learned_addr learned my addr 172.21.15.16:0/382535476 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.057+0000 7f7f1259c640 1 --2- 172.21.15.16:0/382535476 >> v2:172.21.15.146:3300/0 conn(0x7f7f140c8d20 0x7f7f1406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.057+0000 7f7f11d9b640 1 -- 172.21.15.16:0/382535476 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2435711211 0 0) 0x7f7efc001110 con 0x7f7f1406d260 2024-09-19T07:54:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.057+0000 7f7f11d9b640 1 -- 172.21.15.16:0/382535476 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f00003520 con 0x7f7f1406d260 2024-09-19T07:54:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.057+0000 7f7f11d9b640 1 -- 172.21.15.16:0/382535476 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 64751298 0 0) 0x7f7f00003520 con 0x7f7f1406d260 2024-09-19T07:54:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f11d9b640 1 -- 172.21.15.16:0/382535476 >> v2:172.21.15.146:3300/0 conn(0x7f7f140c8d20 msgr2=0x7f7f1406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f11d9b640 1 --2- 172.21.15.16:0/382535476 >> v2:172.21.15.146:3300/0 conn(0x7f7f140c8d20 0x7f7f1406cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f11d9b640 1 -- 172.21.15.16:0/382535476 >> v1:172.21.15.146:6789/0 conn(0x7f7f14144100 legacy=0x7f7f140d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f11d9b640 1 -- 172.21.15.16:0/382535476 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f14144760 con 0x7f7f1406d260 2024-09-19T07:54:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f11d9b640 1 -- 172.21.15.16:0/382535476 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f7efc001840 con 0x7f7f1406d260 2024-09-19T07:54:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f11d9b640 1 -- 172.21.15.16:0/382535476 >> v1:172.21.15.16:6789/0 conn(0x7f7f1406d260 legacy=0x7f7f141439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f11d9b640 1 --2- 172.21.15.16:0/382535476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f000039b0 0x7f7f00003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f11d9b640 1 --2- 172.21.15.16:0/382535476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f00004800 0x7f7f00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f11d9b640 1 --2- 172.21.15.16:0/382535476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f000051c0 0x7f7f0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f11d9b640 1 -- 172.21.15.16:0/382535476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1406d6d0 con 0x7f7f000051c0 2024-09-19T07:54:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f11d9b640 1 -- 172.21.15.16:0/382535476 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0000c110 con 0x7f7f00004800 2024-09-19T07:54:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f11d9b640 1 -- 172.21.15.16:0/382535476 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7f0000c310 con 0x7f7f000039b0 2024-09-19T07:54:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f12d9d640 1 --2- 172.21.15.16:0/382535476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f000051c0 0x7f7f0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f1259c640 1 --2- 172.21.15.16:0/382535476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f000039b0 0x7f7f00003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f0bfff640 1 --2- 172.21.15.16:0/382535476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f00004800 0x7f7f00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f12d9d640 1 -- 172.21.15.16:0/382535476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f000039b0 msgr2=0x7f7f00003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f12d9d640 1 --2- 172.21.15.16:0/382535476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f000039b0 0x7f7f00003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f12d9d640 1 -- 172.21.15.16:0/382535476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f00004800 msgr2=0x7f7f00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f12d9d640 1 --2- 172.21.15.16:0/382535476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f00004800 0x7f7f00004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.058+0000 7f7f12d9d640 1 -- 172.21.15.16:0/382535476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f00003520 con 0x7f7f000051c0 2024-09-19T07:54:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.059+0000 7f7f12d9d640 1 -- 172.21.15.16:0/382535476 reap_dead start 2024-09-19T07:54:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.059+0000 7f7f12d9d640 1 --2- 172.21.15.16:0/382535476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f000051c0 0x7f7f0000b9c0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c001f60 tx=0x7f7f00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd896fd21e3323f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.059+0000 7f7f11d9b640 1 -- 172.21.15.16:0/382535476 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c001b30 con 0x7f7f000051c0 2024-09-19T07:54:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.059+0000 7f7f11d9b640 1 -- 172.21.15.16:0/382535476 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f0c031600 con 0x7f7f000051c0 2024-09-19T07:54:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.059+0000 7f7f11d9b640 1 -- 172.21.15.16:0/382535476 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0319f0 con 0x7f7f000051c0 2024-09-19T07:54:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.059+0000 7f7f18fd8640 1 -- 172.21.15.16:0/382535476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f000051c0 msgr2=0x7f7f0000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:56.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.059+0000 7f7f18fd8640 1 --2- 172.21.15.16:0/382535476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f000051c0 0x7f7f0000b9c0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c001f60 tx=0x7f7f00005830 comp rx=0 tx=0).stop 2024-09-19T07:54:56.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.059+0000 7f7f18fd8640 1 -- 172.21.15.16:0/382535476 shutdown_connections 2024-09-19T07:54:56.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.059+0000 7f7f18fd8640 1 --2- 172.21.15.16:0/382535476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f000051c0 0x7f7f0000b9c0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.059+0000 7f7f18fd8640 1 -- 172.21.15.16:0/382535476 >> 172.21.15.16:0/382535476 conn(0x7f7f140b3ed0 msgr2=0x7f7f140b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:56.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.060+0000 7f7f18fd8640 1 -- 172.21.15.16:0/382535476 shutdown_connections 2024-09-19T07:54:56.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.060+0000 7f7f18fd8640 1 -- 172.21.15.16:0/382535476 wait complete. 2024-09-19T07:54:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.061+0000 7f7f18fd8640 1 Processor -- start 2024-09-19T07:54:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.061+0000 7f7f18fd8640 1 -- start start 2024-09-19T07:54:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f18fd8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f000051c0 0x7f7f14157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f18fd8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f140d1e20 0x7f7f141f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f18fd8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f140d22a0 0x7f7f141f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f18fd8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7f14142200 con 0x7f7f140d22a0 2024-09-19T07:54:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f18fd8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7f14141960 con 0x7f7f140d1e20 2024-09-19T07:54:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f18fd8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7f14144510 con 0x7f7f000051c0 2024-09-19T07:54:56.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f12d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f140d22a0 0x7f7f141f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f12d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f140d22a0 0x7f7f141f6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51038/0 (socket says 172.21.15.16:51038) 2024-09-19T07:54:56.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f12d9d640 1 -- 172.21.15.16:0/929853521 learned_addr learned my addr 172.21.15.16:0/929853521 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:56.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f0bfff640 1 --2- 172.21.15.16:0/929853521 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f140d1e20 0x7f7f141f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f1259c640 1 --2- 172.21.15.16:0/929853521 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f000051c0 0x7f7f14157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f12d9d640 1 -- 172.21.15.16:0/929853521 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f000051c0 msgr2=0x7f7f14157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:56.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f12d9d640 1 --2- 172.21.15.16:0/929853521 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f000051c0 0x7f7f14157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f12d9d640 1 -- 172.21.15.16:0/929853521 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f140d1e20 msgr2=0x7f7f141f68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:56.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f12d9d640 1 --2- 172.21.15.16:0/929853521 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f140d1e20 0x7f7f141f68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f12d9d640 1 -- 172.21.15.16:0/929853521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f141f4090 con 0x7f7f140d22a0 2024-09-19T07:54:56.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f1259c640 1 --2- 172.21.15.16:0/929853521 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f000051c0 0x7f7f14157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:56.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f12d9d640 1 --2- 172.21.15.16:0/929853521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f140d22a0 0x7f7f141f6e10 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c02aa00 tx=0x7f7f0c0015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f0affd640 1 -- 172.21.15.16:0/929853521 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c001b30 con 0x7f7f140d22a0 2024-09-19T07:54:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f0affd640 1 -- 172.21.15.16:0/929853521 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f0c040070 con 0x7f7f140d22a0 2024-09-19T07:54:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f18fd8640 1 -- 172.21.15.16:0/929853521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f141f4370 con 0x7f7f140d22a0 2024-09-19T07:54:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.062+0000 7f7f0affd640 1 -- 172.21.15.16:0/929853521 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c005070 con 0x7f7f140d22a0 2024-09-19T07:54:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.063+0000 7f7f18fd8640 1 -- 172.21.15.16:0/929853521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f141f4930 con 0x7f7f140d22a0 2024-09-19T07:54:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.063+0000 7f7f18fd8640 1 -- 172.21.15.16:0/929853521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f1406cd20 con 0x7f7f140d22a0 2024-09-19T07:54:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.064+0000 7f7f0affd640 1 -- 172.21.15.16:0/929853521 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7f0c039040 con 0x7f7f140d22a0 2024-09-19T07:54:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.064+0000 7f7f0affd640 1 --2- 172.21.15.16:0/929853521 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7ef406c2b0 0x7f7ef406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.065+0000 7f7f0affd640 1 -- 172.21.15.16:0/929853521 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5335+0+0 (secure 0 0 0) 0x7f7f0c0ab380 con 0x7f7f140d22a0 2024-09-19T07:54:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.065+0000 7f7f1259c640 1 --2- 172.21.15.16:0/929853521 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7ef406c2b0 0x7f7ef406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.065+0000 7f7f1259c640 1 --2- 172.21.15.16:0/929853521 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7ef406c2b0 0x7f7ef406e770 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009b40 tx=0x7f7f04008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.069+0000 7f7f0affd640 1 -- 172.21.15.16:0/929853521 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7f0c079c60 con 0x7f7f140d22a0 2024-09-19T07:54:56.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.245+0000 7f7f18fd8640 1 -- 172.21.15.16:0/929853521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f14072290 con 0x7f7f140d22a0 2024-09-19T07:54:56.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.245+0000 7f7f0affd640 1 -- 172.21.15.16:0/929853521 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+13337 (secure 0 0 0) 0x7f7f0c079560 con 0x7f7f140d22a0 2024-09-19T07:54:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:56.246 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":132,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:54:53.931834+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.146:6837/79304323":"2024-09-20T07:54:50.927373+0000","172.21.15.146:6835/79304323":"2024-09-20T07:54:50.927373+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:54:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.248+0000 7f7f18fd8640 1 -- 172.21.15.16:0/929853521 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7ef406c2b0 msgr2=0x7f7ef406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.248+0000 7f7f18fd8640 1 --2- 172.21.15.16:0/929853521 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7ef406c2b0 0x7f7ef406e770 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009b40 tx=0x7f7f04008040 comp rx=0 tx=0).stop 2024-09-19T07:54:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.248+0000 7f7f18fd8640 1 -- 172.21.15.16:0/929853521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f140d22a0 msgr2=0x7f7f141f6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.248+0000 7f7f18fd8640 1 --2- 172.21.15.16:0/929853521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f140d22a0 0x7f7f141f6e10 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c02aa00 tx=0x7f7f0c0015b0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.248+0000 7f7f18fd8640 1 -- 172.21.15.16:0/929853521 shutdown_connections 2024-09-19T07:54:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.248+0000 7f7f18fd8640 1 --2- 172.21.15.16:0/929853521 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7ef406c2b0 0x7f7ef406e770 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.248+0000 7f7f18fd8640 1 --2- 172.21.15.16:0/929853521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f140d22a0 0x7f7f141f6e10 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.248+0000 7f7f18fd8640 1 --2- 172.21.15.16:0/929853521 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f140d1e20 0x7f7f141f68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.248+0000 7f7f18fd8640 1 --2- 172.21.15.16:0/929853521 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f000051c0 0x7f7f14157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.248+0000 7f7f18fd8640 1 -- 172.21.15.16:0/929853521 >> 172.21.15.16:0/929853521 conn(0x7f7f140b3ed0 msgr2=0x7f7f14142590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.248+0000 7f7f18fd8640 1 -- 172.21.15.16:0/929853521 shutdown_connections 2024-09-19T07:54:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.248+0000 7f7f18fd8640 1 -- 172.21.15.16:0/929853521 wait complete. 2024-09-19T07:54:56.259 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T07:42:54.500338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T07:54:56.260 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T07:54:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.365+0000 7f3285518640 1 Processor -- start 2024-09-19T07:54:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.365+0000 7f3285518640 1 -- start start 2024-09-19T07:54:56.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.366+0000 7f3285518640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f328015e2a0 0x7f328006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.366+0000 7f3285518640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f328006d960 0x7f32800d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.366+0000 7f3285518640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32800d1880 con 0x7f328015e680 2024-09-19T07:54:56.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.366+0000 7f3285518640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f32800d1b60 con 0x7f328006d960 2024-09-19T07:54:56.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.366+0000 7f3285518640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f32800cf2d0 con 0x7f328015e2a0 2024-09-19T07:54:56.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.366+0000 7f327f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f328006d960 0x7f32800d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.366+0000 7f327f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f328006d960 0x7f32800d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51040/0 (socket says 172.21.15.16:51040) 2024-09-19T07:54:56.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.366+0000 7f327f7fe640 1 -- 172.21.15.16:0/3120892221 learned_addr learned my addr 172.21.15.16:0/3120892221 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:56.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.366+0000 7f327effd640 1 --2- 172.21.15.16:0/3120892221 >> v2:172.21.15.146:3300/0 conn(0x7f328015e2a0 0x7f328006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.366+0000 7f327f7fe640 1 -- 172.21.15.16:0/3120892221 >> v2:172.21.15.146:3300/0 conn(0x7f328015e2a0 msgr2=0x7f328006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:56.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.366+0000 7f327f7fe640 1 --2- 172.21.15.16:0/3120892221 >> v2:172.21.15.146:3300/0 conn(0x7f328015e2a0 0x7f328006d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.366+0000 7f327f7fe640 1 -- 172.21.15.16:0/3120892221 >> v1:172.21.15.146:6789/0 conn(0x7f328015e680 legacy=0x7f32800cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:56.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.366+0000 7f327f7fe640 1 -- 172.21.15.16:0/3120892221 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32800cf5b0 con 0x7f328006d960 2024-09-19T07:54:56.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.367+0000 7f327f7fe640 1 --2- 172.21.15.16:0/3120892221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f328006d960 0x7f32800d4370 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f32800b6fa0 tx=0x7f327802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72fa184336efb6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:56.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.367+0000 7f327e7fc640 1 -- 172.21.15.16:0/3120892221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278036070 con 0x7f328006d960 2024-09-19T07:54:56.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.367+0000 7f327e7fc640 1 -- 172.21.15.16:0/3120892221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3278031600 con 0x7f328006d960 2024-09-19T07:54:56.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.367+0000 7f327e7fc640 1 -- 172.21.15.16:0/3120892221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32780319d0 con 0x7f328006d960 2024-09-19T07:54:56.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.368+0000 7f3285518640 1 -- 172.21.15.16:0/3120892221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f328006d960 msgr2=0x7f32800d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:56.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.368+0000 7f3285518640 1 --2- 172.21.15.16:0/3120892221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f328006d960 0x7f32800d4370 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f32800b6fa0 tx=0x7f327802a6f0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.368+0000 7f3285518640 1 -- 172.21.15.16:0/3120892221 shutdown_connections 2024-09-19T07:54:56.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.368+0000 7f3285518640 1 --2- 172.21.15.16:0/3120892221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f328006d960 0x7f32800d4370 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.368+0000 7f3285518640 1 --2- 172.21.15.16:0/3120892221 >> v2:172.21.15.146:3300/0 conn(0x7f328015e2a0 0x7f328006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.368+0000 7f3285518640 1 -- 172.21.15.16:0/3120892221 >> 172.21.15.16:0/3120892221 conn(0x7f328009d760 msgr2=0x7f328008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:56.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.368+0000 7f3285518640 1 -- 172.21.15.16:0/3120892221 shutdown_connections 2024-09-19T07:54:56.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.368+0000 7f3285518640 1 -- 172.21.15.16:0/3120892221 wait complete. 2024-09-19T07:54:56.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.369+0000 7f3285518640 1 Processor -- start 2024-09-19T07:54:56.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.369+0000 7f3285518640 1 -- start start 2024-09-19T07:54:56.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.370+0000 7f3285518640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f328006d960 0x7f32800d3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.370+0000 7f3285518640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f328015e2a0 0x7f3280150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.370+0000 7f3285518640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f328015e680 0x7f32800cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.370+0000 7f3285518640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32800d0360 con 0x7f328006d960 2024-09-19T07:54:56.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.370+0000 7f3285518640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f32800d01e0 con 0x7f328015e2a0 2024-09-19T07:54:56.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.370+0000 7f3285518640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f32800d48b0 con 0x7f328015e680 2024-09-19T07:54:56.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.370+0000 7f327effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f328006d960 0x7f32800d3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.370+0000 7f327effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f328006d960 0x7f32800d3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51052/0 (socket says 172.21.15.16:51052) 2024-09-19T07:54:56.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.370+0000 7f327effd640 1 -- 172.21.15.16:0/1826645321 learned_addr learned my addr 172.21.15.16:0/1826645321 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:56.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.370+0000 7f327f7fe640 1 --2- 172.21.15.16:0/1826645321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f328015e680 0x7f32800cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.370+0000 7f32767fc640 1 --2- 172.21.15.16:0/1826645321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f328015e2a0 0x7f3280150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.370+0000 7f327effd640 1 -- 172.21.15.16:0/1826645321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f328015e680 msgr2=0x7f32800cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:56.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.370+0000 7f327effd640 1 --2- 172.21.15.16:0/1826645321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f328015e680 0x7f32800cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.370+0000 7f327effd640 1 -- 172.21.15.16:0/1826645321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f328015e2a0 msgr2=0x7f3280150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:56.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.370+0000 7f327effd640 1 --2- 172.21.15.16:0/1826645321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f328015e2a0 0x7f3280150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.370+0000 7f327effd640 1 -- 172.21.15.16:0/1826645321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32800d3ab0 con 0x7f328006d960 2024-09-19T07:54:56.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.370+0000 7f327effd640 1 --2- 172.21.15.16:0/1826645321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f328006d960 0x7f32800d3570 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f327000d800 tx=0x7f327000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:56.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.371+0000 7f3277fff640 1 -- 172.21.15.16:0/1826645321 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327000b6c0 con 0x7f328006d960 2024-09-19T07:54:56.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.371+0000 7f3277fff640 1 -- 172.21.15.16:0/1826645321 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3270019070 con 0x7f328006d960 2024-09-19T07:54:56.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.371+0000 7f3285518640 1 -- 172.21.15.16:0/1826645321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3280150dd0 con 0x7f328006d960 2024-09-19T07:54:56.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.371+0000 7f3277fff640 1 -- 172.21.15.16:0/1826645321 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327000ba80 con 0x7f328006d960 2024-09-19T07:54:56.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.371+0000 7f3285518640 1 -- 172.21.15.16:0/1826645321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32800c84d0 con 0x7f328006d960 2024-09-19T07:54:56.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.372+0000 7f3285518640 1 -- 172.21.15.16:0/1826645321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32800d4c70 con 0x7f328006d960 2024-09-19T07:54:56.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.372+0000 7f3277fff640 1 -- 172.21.15.16:0/1826645321 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f327000bc20 con 0x7f328006d960 2024-09-19T07:54:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.373+0000 7f3277fff640 1 --2- 172.21.15.16:0/1826645321 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f326006c2b0 0x7f326006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.373+0000 7f3277fff640 1 -- 172.21.15.16:0/1826645321 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5335+0+0 (secure 0 0 0) 0x7f327008b880 con 0x7f328006d960 2024-09-19T07:54:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.373+0000 7f32767fc640 1 --2- 172.21.15.16:0/1826645321 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f326006c2b0 0x7f326006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.374+0000 7f32767fc640 1 --2- 172.21.15.16:0/1826645321 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f326006c2b0 0x7f326006e770 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3268006fd0 tx=0x7f3268008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:56.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.377+0000 7f3277fff640 1 -- 172.21.15.16:0/1826645321 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f327005a030 con 0x7f328006d960 2024-09-19T07:54:56.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.553+0000 7f3285518640 1 -- 172.21.15.16:0/1826645321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3280162af0 con 0x7f328006d960 2024-09-19T07:54:56.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.553+0000 7f3277fff640 1 -- 172.21.15.16:0/1826645321 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v132) ==== 93+0+10 (secure 0 0 0) 0x7f3270059930 con 0x7f328006d960 2024-09-19T07:54:56.554 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-19T07:54:56.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.555+0000 7f3285518640 1 -- 172.21.15.16:0/1826645321 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f326006c2b0 msgr2=0x7f326006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:56.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.555+0000 7f3285518640 1 --2- 172.21.15.16:0/1826645321 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f326006c2b0 0x7f326006e770 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3268006fd0 tx=0x7f3268008040 comp rx=0 tx=0).stop 2024-09-19T07:54:56.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.555+0000 7f3285518640 1 -- 172.21.15.16:0/1826645321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f328006d960 msgr2=0x7f32800d3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:56.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.555+0000 7f3285518640 1 --2- 172.21.15.16:0/1826645321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f328006d960 0x7f32800d3570 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f327000d800 tx=0x7f327000dcc0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.555+0000 7f3285518640 1 -- 172.21.15.16:0/1826645321 shutdown_connections 2024-09-19T07:54:56.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.555+0000 7f3285518640 1 --2- 172.21.15.16:0/1826645321 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f326006c2b0 0x7f326006e770 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.555+0000 7f3285518640 1 --2- 172.21.15.16:0/1826645321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f328015e680 0x7f32800cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.555+0000 7f3285518640 1 --2- 172.21.15.16:0/1826645321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f328015e2a0 0x7f3280150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.555+0000 7f3285518640 1 --2- 172.21.15.16:0/1826645321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f328006d960 0x7f32800d3570 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.555+0000 7f3285518640 1 -- 172.21.15.16:0/1826645321 >> 172.21.15.16:0/1826645321 conn(0x7f328009d760 msgr2=0x7f32800d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:56.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.555+0000 7f3285518640 1 -- 172.21.15.16:0/1826645321 shutdown_connections 2024-09-19T07:54:56.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.555+0000 7f3285518640 1 -- 172.21.15.16:0/1826645321 wait complete. 2024-09-19T07:54:56.566 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root' 2024-09-19T07:54:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.669+0000 7f1db4e3c640 1 Processor -- start 2024-09-19T07:54:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.669+0000 7f1db4e3c640 1 -- start start 2024-09-19T07:54:56.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.670+0000 7f1db4e3c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1db015bee0 0x7f1db00d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.670+0000 7f1db4e3c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1db00cf910 0x7f1db00d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.670+0000 7f1db4e3c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1db00cdee0 con 0x7f1db015c2c0 2024-09-19T07:54:56.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.670+0000 7f1db4e3c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f1db00ceaf0 con 0x7f1db00cf910 2024-09-19T07:54:56.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.670+0000 7f1db4e3c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f1db00ce1c0 con 0x7f1db015bee0 2024-09-19T07:54:56.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.670+0000 7f1daed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1db00cf910 0x7f1db00d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.670+0000 7f1daed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1db00cf910 0x7f1db00d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51062/0 (socket says 172.21.15.16:51062) 2024-09-19T07:54:56.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.670+0000 7f1daed9d640 1 -- 172.21.15.16:0/2982269545 learned_addr learned my addr 172.21.15.16:0/2982269545 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:56.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.670+0000 7f1dae59c640 1 --2- 172.21.15.16:0/2982269545 >> v2:172.21.15.146:3300/0 conn(0x7f1db015bee0 0x7f1db00d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.670+0000 7f1daed9d640 1 -- 172.21.15.16:0/2982269545 >> v2:172.21.15.146:3300/0 conn(0x7f1db015bee0 msgr2=0x7f1db00d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:56.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.670+0000 7f1daed9d640 1 --2- 172.21.15.16:0/2982269545 >> v2:172.21.15.146:3300/0 conn(0x7f1db015bee0 0x7f1db00d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.670+0000 7f1daed9d640 1 -- 172.21.15.16:0/2982269545 >> v1:172.21.15.146:6789/0 conn(0x7f1db015c2c0 legacy=0x7f1db00cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:56.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.670+0000 7f1daed9d640 1 -- 172.21.15.16:0/2982269545 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db00ce4a0 con 0x7f1db00cf910 2024-09-19T07:54:56.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.670+0000 7f1dae59c640 1 --2- 172.21.15.16:0/2982269545 >> v2:172.21.15.146:3300/0 conn(0x7f1db015bee0 0x7f1db00d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.671+0000 7f1daed9d640 1 --2- 172.21.15.16:0/2982269545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db00cf910 0x7f1db00d09f0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f1db00b2f40 tx=0x7f1da802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17779624fa8ef24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.671+0000 7f1dad59a640 1 -- 172.21.15.16:0/2982269545 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8037070 con 0x7f1db00cf910 2024-09-19T07:54:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.671+0000 7f1dad59a640 1 -- 172.21.15.16:0/2982269545 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da8032440 con 0x7f1db00cf910 2024-09-19T07:54:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.671+0000 7f1dad59a640 1 -- 172.21.15.16:0/2982269545 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8032810 con 0x7f1db00cf910 2024-09-19T07:54:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.672+0000 7f1db4e3c640 1 -- 172.21.15.16:0/2982269545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db00cf910 msgr2=0x7f1db00d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.672+0000 7f1db4e3c640 1 --2- 172.21.15.16:0/2982269545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db00cf910 0x7f1db00d09f0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f1db00b2f40 tx=0x7f1da802a230 comp rx=0 tx=0).stop 2024-09-19T07:54:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.672+0000 7f1db4e3c640 1 -- 172.21.15.16:0/2982269545 shutdown_connections 2024-09-19T07:54:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.672+0000 7f1db4e3c640 1 --2- 172.21.15.16:0/2982269545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db00cf910 0x7f1db00d09f0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.672+0000 7f1db4e3c640 1 --2- 172.21.15.16:0/2982269545 >> v2:172.21.15.146:3300/0 conn(0x7f1db015bee0 0x7f1db00d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.672+0000 7f1db4e3c640 1 -- 172.21.15.16:0/2982269545 >> 172.21.15.16:0/2982269545 conn(0x7f1db006bdf0 msgr2=0x7f1db006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.672+0000 7f1db4e3c640 1 -- 172.21.15.16:0/2982269545 shutdown_connections 2024-09-19T07:54:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.672+0000 7f1db4e3c640 1 -- 172.21.15.16:0/2982269545 wait complete. 2024-09-19T07:54:56.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.673+0000 7f1db4e3c640 1 Processor -- start 2024-09-19T07:54:56.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.673+0000 7f1db4e3c640 1 -- start start 2024-09-19T07:54:56.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.674+0000 7f1db4e3c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1db00cf910 0x7f1db0150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.674+0000 7f1db4e3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db015bee0 0x7f1db00aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.674+0000 7f1db4e3c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1db015c2c0 0x7f1db0159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.674+0000 7f1db4e3c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1db006e040 con 0x7f1db015bee0 2024-09-19T07:54:56.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.674+0000 7f1db4e3c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1db006dec0 con 0x7f1db015c2c0 2024-09-19T07:54:56.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.674+0000 7f1db4e3c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1db006e1c0 con 0x7f1db00cf910 2024-09-19T07:54:56.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.674+0000 7f1dadd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db015bee0 0x7f1db00aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.674+0000 7f1dadd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db015bee0 0x7f1db00aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51074/0 (socket says 172.21.15.16:51074) 2024-09-19T07:54:56.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.674+0000 7f1dadd9b640 1 -- 172.21.15.16:0/4055583464 learned_addr learned my addr 172.21.15.16:0/4055583464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:56.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.674+0000 7f1dae59c640 1 --2- 172.21.15.16:0/4055583464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1db00cf910 0x7f1db0150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.674+0000 7f1daed9d640 1 --2- 172.21.15.16:0/4055583464 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1db015c2c0 0x7f1db0159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.675+0000 7f1dadd9b640 1 -- 172.21.15.16:0/4055583464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1db00cf910 msgr2=0x7f1db0150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:56.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.675+0000 7f1dadd9b640 1 --2- 172.21.15.16:0/4055583464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1db00cf910 0x7f1db0150810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.675+0000 7f1dadd9b640 1 -- 172.21.15.16:0/4055583464 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1db015c2c0 msgr2=0x7f1db0159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:56.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.675+0000 7f1dadd9b640 1 --2- 172.21.15.16:0/4055583464 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1db015c2c0 0x7f1db0159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.675+0000 7f1dadd9b640 1 -- 172.21.15.16:0/4055583464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db00af340 con 0x7f1db015bee0 2024-09-19T07:54:56.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.675+0000 7f1dadd9b640 1 --2- 172.21.15.16:0/4055583464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db015bee0 0x7f1db00aee00 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800c460 tx=0x7f1d9800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:56.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.675+0000 7f1da6ffd640 1 -- 172.21.15.16:0/4055583464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98016020 con 0x7f1db015bee0 2024-09-19T07:54:56.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.675+0000 7f1da6ffd640 1 -- 172.21.15.16:0/4055583464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d98012870 con 0x7f1db015bee0 2024-09-19T07:54:56.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.675+0000 7f1db4e3c640 1 -- 172.21.15.16:0/4055583464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db015f170 con 0x7f1db015bee0 2024-09-19T07:54:56.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.675+0000 7f1da6ffd640 1 -- 172.21.15.16:0/4055583464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98012c60 con 0x7f1db015bee0 2024-09-19T07:54:56.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.675+0000 7f1db4e3c640 1 -- 172.21.15.16:0/4055583464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db00ae600 con 0x7f1db015bee0 2024-09-19T07:54:56.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.676+0000 7f1db4e3c640 1 -- 172.21.15.16:0/4055583464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db015f690 con 0x7f1db015bee0 2024-09-19T07:54:56.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.676+0000 7f1da6ffd640 1 -- 172.21.15.16:0/4055583464 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1d98019020 con 0x7f1db015bee0 2024-09-19T07:54:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.677+0000 7f1da6ffd640 1 --2- 172.21.15.16:0/4055583464 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1d9006c2e0 0x7f1d9006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.677+0000 7f1da6ffd640 1 -- 172.21.15.16:0/4055583464 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5335+0+0 (secure 0 0 0) 0x7f1d9808aba0 con 0x7f1db015bee0 2024-09-19T07:54:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.677+0000 7f1dae59c640 1 --2- 172.21.15.16:0/4055583464 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1d9006c2e0 0x7f1d9006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.678+0000 7f1dae59c640 1 --2- 172.21.15.16:0/4055583464 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1d9006c2e0 0x7f1d9006e7a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1db00ca5b0 tx=0x7f1da0007810 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.681+0000 7f1da6ffd640 1 -- 172.21.15.16:0/4055583464 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1db015f690 con 0x7f1db015bee0 2024-09-19T07:54:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.928+0000 7f1db4e3c640 1 -- 172.21.15.16:0/4055583464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]} v 0) -- 0x7f1db00b1a50 con 0x7f1db015bee0 2024-09-19T07:54:56.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.944+0000 7f1da6ffd640 1 -- 172.21.15.16:0/4055583464 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]}]=0 v533) ==== 151+0+0 (secure 0 0 0) 0x7f1db00b1a50 con 0x7f1db015bee0 2024-09-19T07:54:56.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.947+0000 7f1db4e3c640 1 -- 172.21.15.16:0/4055583464 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1d9006c2e0 msgr2=0x7f1d9006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:56.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.947+0000 7f1db4e3c640 1 --2- 172.21.15.16:0/4055583464 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1d9006c2e0 0x7f1d9006e7a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1db00ca5b0 tx=0x7f1da0007810 comp rx=0 tx=0).stop 2024-09-19T07:54:56.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.947+0000 7f1db4e3c640 1 -- 172.21.15.16:0/4055583464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db015bee0 msgr2=0x7f1db00aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:56.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.947+0000 7f1db4e3c640 1 --2- 172.21.15.16:0/4055583464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db015bee0 0x7f1db00aee00 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800c460 tx=0x7f1d9800c920 comp rx=0 tx=0).stop 2024-09-19T07:54:56.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.947+0000 7f1db4e3c640 1 -- 172.21.15.16:0/4055583464 shutdown_connections 2024-09-19T07:54:56.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.947+0000 7f1db4e3c640 1 --2- 172.21.15.16:0/4055583464 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1d9006c2e0 0x7f1d9006e7a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.947+0000 7f1db4e3c640 1 --2- 172.21.15.16:0/4055583464 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1db015c2c0 0x7f1db0159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.947+0000 7f1db4e3c640 1 --2- 172.21.15.16:0/4055583464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db015bee0 0x7f1db00aee00 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.947+0000 7f1db4e3c640 1 --2- 172.21.15.16:0/4055583464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1db00cf910 0x7f1db0150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:56.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.947+0000 7f1db4e3c640 1 -- 172.21.15.16:0/4055583464 >> 172.21.15.16:0/4055583464 conn(0x7f1db006bdf0 msgr2=0x7f1db00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:56.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.947+0000 7f1db4e3c640 1 -- 172.21.15.16:0/4055583464 shutdown_connections 2024-09-19T07:54:56.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:56.947+0000 7f1db4e3c640 1 -- 172.21.15.16:0/4055583464 wait complete. 2024-09-19T07:54:56.958 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T07:54:57.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.063+0000 7f9cfb023640 1 Processor -- start 2024-09-19T07:54:57.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.063+0000 7f9cfb023640 1 -- start start 2024-09-19T07:54:57.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.064+0000 7f9cfb023640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9cf415e2a0 0x7f9cf406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.064+0000 7f9cfb023640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9cf415e680 0x7f9cf40cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.064+0000 7f9cfb023640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cf40d1880 con 0x7f9cf406d960 2024-09-19T07:54:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.064+0000 7f9cfb023640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9cf40d1b60 con 0x7f9cf415e680 2024-09-19T07:54:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.064+0000 7f9cfb023640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf40cf2d0 con 0x7f9cf415e2a0 2024-09-19T07:54:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.064+0000 7f9cf3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9cf415e680 0x7f9cf40cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.064+0000 7f9cf3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9cf415e680 0x7f9cf40cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51088/0 (socket says 172.21.15.16:51088) 2024-09-19T07:54:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.064+0000 7f9cf3fff640 1 -- 172.21.15.16:0/2790410905 learned_addr learned my addr 172.21.15.16:0/2790410905 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.064+0000 7f9cf8dbf640 1 --2- 172.21.15.16:0/2790410905 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9cf415e2a0 0x7f9cf406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.064+0000 7f9cf3fff640 1 -- 172.21.15.16:0/2790410905 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9cf415e2a0 msgr2=0x7f9cf406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.064+0000 7f9cf3fff640 1 --2- 172.21.15.16:0/2790410905 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9cf415e2a0 0x7f9cf406d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.064+0000 7f9cf3fff640 1 -- 172.21.15.16:0/2790410905 >> v1:172.21.15.146:6789/0 conn(0x7f9cf406d960 legacy=0x7f9cf40d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.064+0000 7f9cf3fff640 1 -- 172.21.15.16:0/2790410905 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf40cf5b0 con 0x7f9cf415e680 2024-09-19T07:54:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.064+0000 7f9cf8dbf640 1 --2- 172.21.15.16:0/2790410905 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9cf415e2a0 0x7f9cf406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:57.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.065+0000 7f9cf3fff640 1 --2- 172.21.15.16:0/2790410905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415e680 0x7f9cf40cfad0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009870 tx=0x7f9ce802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1053bfe394d87b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:57.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.065+0000 7f9cf37fe640 1 -- 172.21.15.16:0/2790410905 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8037070 con 0x7f9cf415e680 2024-09-19T07:54:57.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.065+0000 7f9cf37fe640 1 -- 172.21.15.16:0/2790410905 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce8032440 con 0x7f9cf415e680 2024-09-19T07:54:57.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.065+0000 7f9cf37fe640 1 -- 172.21.15.16:0/2790410905 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8032810 con 0x7f9cf415e680 2024-09-19T07:54:57.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.066+0000 7f9cfb023640 1 -- 172.21.15.16:0/2790410905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415e680 msgr2=0x7f9cf40cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:57.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.066+0000 7f9cfb023640 1 --2- 172.21.15.16:0/2790410905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415e680 0x7f9cf40cfad0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009870 tx=0x7f9ce802a210 comp rx=0 tx=0).stop 2024-09-19T07:54:57.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.066+0000 7f9cfb023640 1 -- 172.21.15.16:0/2790410905 shutdown_connections 2024-09-19T07:54:57.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.066+0000 7f9cfb023640 1 --2- 172.21.15.16:0/2790410905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415e680 0x7f9cf40cfad0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.066+0000 7f9cfb023640 1 --2- 172.21.15.16:0/2790410905 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9cf415e2a0 0x7f9cf406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.066+0000 7f9cfb023640 1 -- 172.21.15.16:0/2790410905 >> 172.21.15.16:0/2790410905 conn(0x7f9cf409d760 msgr2=0x7f9cf408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:57.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.066+0000 7f9cfb023640 1 -- 172.21.15.16:0/2790410905 shutdown_connections 2024-09-19T07:54:57.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.066+0000 7f9cfb023640 1 -- 172.21.15.16:0/2790410905 wait complete. 2024-09-19T07:54:57.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.067+0000 7f9cfb023640 1 Processor -- start 2024-09-19T07:54:57.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.067+0000 7f9cfb023640 1 -- start start 2024-09-19T07:54:57.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.068+0000 7f9cfb023640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9cf406d960 0x7f9cf40a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.068+0000 7f9cfb023640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9cf415e2a0 0x7f9cf40ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.068+0000 7f9cfb023640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415e680 0x7f9cf41f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.068+0000 7f9cfb023640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf40d4c00 con 0x7f9cf415e680 2024-09-19T07:54:57.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.068+0000 7f9cfb023640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf40d4a80 con 0x7f9cf415e2a0 2024-09-19T07:54:57.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.068+0000 7f9cfb023640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf40d0010 con 0x7f9cf406d960 2024-09-19T07:54:57.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.068+0000 7f9cf95c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415e680 0x7f9cf41f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.068+0000 7f9cf95c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415e680 0x7f9cf41f4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51098/0 (socket says 172.21.15.16:51098) 2024-09-19T07:54:57.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.068+0000 7f9cf95c0640 1 -- 172.21.15.16:0/3311219030 learned_addr learned my addr 172.21.15.16:0/3311219030 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:57.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.068+0000 7f9cf8dbf640 1 --2- 172.21.15.16:0/3311219030 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9cf406d960 0x7f9cf40a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.068+0000 7f9cf3fff640 1 --2- 172.21.15.16:0/3311219030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9cf415e2a0 0x7f9cf40ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.068+0000 7f9cf95c0640 1 -- 172.21.15.16:0/3311219030 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9cf406d960 msgr2=0x7f9cf40a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:57.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.068+0000 7f9cf95c0640 1 --2- 172.21.15.16:0/3311219030 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9cf406d960 0x7f9cf40a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.068+0000 7f9cf95c0640 1 -- 172.21.15.16:0/3311219030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9cf415e2a0 msgr2=0x7f9cf40ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:57.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.068+0000 7f9cf95c0640 1 --2- 172.21.15.16:0/3311219030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9cf415e2a0 0x7f9cf40ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.068+0000 7f9cf95c0640 1 -- 172.21.15.16:0/3311219030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf40a1d80 con 0x7f9cf415e680 2024-09-19T07:54:57.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.068+0000 7f9cf3fff640 1 --2- 172.21.15.16:0/3311219030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9cf415e2a0 0x7f9cf40ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:57.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.069+0000 7f9cf95c0640 1 --2- 172.21.15.16:0/3311219030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415e680 0x7f9cf41f4f50 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00c7b0 tx=0x7f9cec00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:57.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.069+0000 7f9cf17fa640 1 -- 172.21.15.16:0/3311219030 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec004b60 con 0x7f9cf415e680 2024-09-19T07:54:57.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.069+0000 7f9cf17fa640 1 -- 172.21.15.16:0/3311219030 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cec004d00 con 0x7f9cf415e680 2024-09-19T07:54:57.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.069+0000 7f9cfb023640 1 -- 172.21.15.16:0/3311219030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf40a0ce0 con 0x7f9cf415e680 2024-09-19T07:54:57.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.069+0000 7f9cf17fa640 1 -- 172.21.15.16:0/3311219030 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec00aa90 con 0x7f9cf415e680 2024-09-19T07:54:57.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.069+0000 7f9cfb023640 1 -- 172.21.15.16:0/3311219030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf40a12a0 con 0x7f9cf415e680 2024-09-19T07:54:57.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.070+0000 7f9cfb023640 1 -- 172.21.15.16:0/3311219030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf406d420 con 0x7f9cf415e680 2024-09-19T07:54:57.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.070+0000 7f9cf17fa640 1 -- 172.21.15.16:0/3311219030 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9cec00ac30 con 0x7f9cf415e680 2024-09-19T07:54:57.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.071+0000 7f9cf17fa640 1 --2- 172.21.15.16:0/3311219030 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ce006c2b0 0x7f9ce006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.071+0000 7f9cf17fa640 1 -- 172.21.15.16:0/3311219030 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5335+0+0 (secure 0 0 0) 0x7f9cec08b4d0 con 0x7f9cf415e680 2024-09-19T07:54:57.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.071+0000 7f9cf8dbf640 1 --2- 172.21.15.16:0/3311219030 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ce006c2b0 0x7f9ce006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.072+0000 7f9cf8dbf640 1 --2- 172.21.15.16:0/3311219030 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ce006c2b0 0x7f9ce006e770 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400a990 tx=0x7f9ce4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:57.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.075+0000 7f9cf17fa640 1 -- 172.21.15.16:0/3311219030 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9cec020070 con 0x7f9cf415e680 2024-09-19T07:54:57.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.249+0000 7f9cfb023640 1 -- 172.21.15.16:0/3311219030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f9cf41619d0 con 0x7f9cf415e680 2024-09-19T07:54:57.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.252+0000 7f9cf17fa640 1 -- 172.21.15.16:0/3311219030 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v63)=0 v63) ==== 122+0+0 (secure 0 0 0) 0x7f9cec059e40 con 0x7f9cf415e680 2024-09-19T07:54:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.255+0000 7f9cfb023640 1 -- 172.21.15.16:0/3311219030 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ce006c2b0 msgr2=0x7f9ce006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.255+0000 7f9cfb023640 1 --2- 172.21.15.16:0/3311219030 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ce006c2b0 0x7f9ce006e770 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400a990 tx=0x7f9ce4008040 comp rx=0 tx=0).stop 2024-09-19T07:54:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.255+0000 7f9cfb023640 1 -- 172.21.15.16:0/3311219030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415e680 msgr2=0x7f9cf41f4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.255+0000 7f9cfb023640 1 --2- 172.21.15.16:0/3311219030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415e680 0x7f9cf41f4f50 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00c7b0 tx=0x7f9cec00cc70 comp rx=0 tx=0).stop 2024-09-19T07:54:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.255+0000 7f9cfb023640 1 -- 172.21.15.16:0/3311219030 shutdown_connections 2024-09-19T07:54:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.255+0000 7f9cfb023640 1 --2- 172.21.15.16:0/3311219030 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ce006c2b0 0x7f9ce006e770 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.255+0000 7f9cfb023640 1 --2- 172.21.15.16:0/3311219030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415e680 0x7f9cf41f4f50 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.255+0000 7f9cfb023640 1 --2- 172.21.15.16:0/3311219030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9cf415e2a0 0x7f9cf40ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.255+0000 7f9cfb023640 1 --2- 172.21.15.16:0/3311219030 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9cf406d960 0x7f9cf40a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.255+0000 7f9cfb023640 1 -- 172.21.15.16:0/3311219030 >> 172.21.15.16:0/3311219030 conn(0x7f9cf409d760 msgr2=0x7f9cf4161f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.255+0000 7f9cfb023640 1 -- 172.21.15.16:0/3311219030 shutdown_connections 2024-09-19T07:54:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.255+0000 7f9cfb023640 1 -- 172.21.15.16:0/3311219030 wait complete. 2024-09-19T07:54:57.266 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:54:57.266 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:54:57.310 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:54:57.353 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:54:57.353 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:54:57.353 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:54:57.370 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:54:57.370 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:54:57.370 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:54:57.411 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:54:57.412 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:54:57.412 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:54:57.428 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:54:57.428 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:54:57.428 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:54:57.469 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:54:57.470 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T07:54:57.470 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:54:57.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.571+0000 7fb07b986640 1 Processor -- start 2024-09-19T07:54:57.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.571+0000 7fb07b986640 1 -- start start 2024-09-19T07:54:57.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.572+0000 7fb07b986640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb07415e680 0x7fb0740cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.572+0000 7fb07b986640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0740d1880 con 0x7fb07415e2a0 2024-09-19T07:54:57.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.572+0000 7fb07b986640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0740d1b60 con 0x7fb0740d0010 2024-09-19T07:54:57.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.572+0000 7fb07b986640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb0740cf2d0 con 0x7fb07415e680 2024-09-19T07:54:57.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.572+0000 7fb079722640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb07415e2a0 0x7fb07406d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33804/0 (socket says 172.21.15.16:33804) 2024-09-19T07:54:57.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.572+0000 7fb079722640 1 -- 172.21.15.16:0/4108813592 learned_addr learned my addr 172.21.15.16:0/4108813592 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:57.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.572+0000 7fb078f21640 1 --2- 172.21.15.16:0/4108813592 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb07415e680 0x7fb0740cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.572+0000 7fb06bfff640 1 -- 172.21.15.16:0/4108813592 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3208856628 0 0) 0x7fb0740d1880 con 0x7fb07415e2a0 2024-09-19T07:54:57.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.572+0000 7fb06bfff640 1 -- 172.21.15.16:0/4108813592 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb060003520 con 0x7fb07415e2a0 2024-09-19T07:54:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb06bfff640 1 -- 172.21.15.16:0/4108813592 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2788801230 0 0) 0x7fb060003520 con 0x7fb07415e2a0 2024-09-19T07:54:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb06bfff640 1 -- 172.21.15.16:0/4108813592 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb07415e680 msgr2=0x7fb0740cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb06bfff640 1 --2- 172.21.15.16:0/4108813592 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb07415e680 0x7fb0740cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb06bfff640 1 -- 172.21.15.16:0/4108813592 >> v1:172.21.15.146:6789/0 conn(0x7fb0740d0010 legacy=0x7fb0740d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb06bfff640 1 -- 172.21.15.16:0/4108813592 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0740cf5b0 con 0x7fb07415e2a0 2024-09-19T07:54:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb06bfff640 1 -- 172.21.15.16:0/4108813592 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fb070001670 con 0x7fb07415e2a0 2024-09-19T07:54:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb06bfff640 1 -- 172.21.15.16:0/4108813592 >> v1:172.21.15.16:6789/0 conn(0x7fb07415e2a0 legacy=0x7fb07406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb06bfff640 1 --2- 172.21.15.16:0/4108813592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0600039e0 0x7fb060003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb06bfff640 1 --2- 172.21.15.16:0/4108813592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb060004800 0x7fb060004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb06bfff640 1 --2- 172.21.15.16:0/4108813592 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb07415e680 0x7fb060005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb06bfff640 1 -- 172.21.15.16:0/4108813592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0740cf2d0 con 0x7fb060004800 2024-09-19T07:54:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb06bfff640 1 -- 172.21.15.16:0/4108813592 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb06000bb20 con 0x7fb0600039e0 2024-09-19T07:54:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb06bfff640 1 -- 172.21.15.16:0/4108813592 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb06000bd20 con 0x7fb07415e680 2024-09-19T07:54:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb078f21640 1 --2- 172.21.15.16:0/4108813592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb060004800 0x7fb060004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb079f23640 1 --2- 172.21.15.16:0/4108813592 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb07415e680 0x7fb060005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb079722640 1 --2- 172.21.15.16:0/4108813592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0600039e0 0x7fb060003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb078f21640 1 -- 172.21.15.16:0/4108813592 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb07415e680 msgr2=0x7fb060005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb078f21640 1 --2- 172.21.15.16:0/4108813592 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb07415e680 0x7fb060005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb078f21640 1 -- 172.21.15.16:0/4108813592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0600039e0 msgr2=0x7fb060003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb078f21640 1 --2- 172.21.15.16:0/4108813592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0600039e0 0x7fb060003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.573+0000 7fb078f21640 1 -- 172.21.15.16:0/4108813592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb060003520 con 0x7fb060004800 2024-09-19T07:54:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.574+0000 7fb078f21640 1 --2- 172.21.15.16:0/4108813592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb060004800 0x7fb060004c80 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fb0740a22a0 tx=0x7fb060005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8aacfbfc9b31eb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.574+0000 7fb06bfff640 1 -- 172.21.15.16:0/4108813592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064036070 con 0x7fb060004800 2024-09-19T07:54:57.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.574+0000 7fb06bfff640 1 -- 172.21.15.16:0/4108813592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb064031400 con 0x7fb060004800 2024-09-19T07:54:57.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.574+0000 7fb06bfff640 1 -- 172.21.15.16:0/4108813592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064009690 con 0x7fb060004800 2024-09-19T07:54:57.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.574+0000 7fb07b986640 1 -- 172.21.15.16:0/4108813592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb060004800 msgr2=0x7fb060004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:57.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.574+0000 7fb07b986640 1 --2- 172.21.15.16:0/4108813592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb060004800 0x7fb060004c80 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fb0740a22a0 tx=0x7fb060005830 comp rx=0 tx=0).stop 2024-09-19T07:54:57.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.574+0000 7fb079f23640 1 -- 172.21.15.16:0/4108813592 reap_dead start 2024-09-19T07:54:57.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.574+0000 7fb07b986640 1 -- 172.21.15.16:0/4108813592 shutdown_connections 2024-09-19T07:54:57.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.574+0000 7fb07b986640 1 -- 172.21.15.16:0/4108813592 >> 172.21.15.16:0/4108813592 conn(0x7fb07409d760 msgr2=0x7fb07408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:57.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.575+0000 7fb07b986640 1 -- 172.21.15.16:0/4108813592 shutdown_connections 2024-09-19T07:54:57.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.575+0000 7fb07b986640 1 -- 172.21.15.16:0/4108813592 wait complete. 2024-09-19T07:54:57.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.576+0000 7fb07b986640 1 Processor -- start 2024-09-19T07:54:57.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.576+0000 7fb07b986640 1 -- start start 2024-09-19T07:54:57.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.576+0000 7fb07b986640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb060004800 0x7fb0741e9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.576+0000 7fb07b986640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0741e87c0 0x7fb074154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.576+0000 7fb07b986640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0740b9220 0x7fb07415d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.576+0000 7fb07b986640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb07406dcb0 con 0x7fb0741e87c0 2024-09-19T07:54:57.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.576+0000 7fb07b986640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb07406db30 con 0x7fb060004800 2024-09-19T07:54:57.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.576+0000 7fb07b986640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb0740d4a80 con 0x7fb0740b9220 2024-09-19T07:54:57.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.577+0000 7fb078f21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0741e87c0 0x7fb074154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.577+0000 7fb078f21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0741e87c0 0x7fb074154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51112/0 (socket says 172.21.15.16:51112) 2024-09-19T07:54:57.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.577+0000 7fb078f21640 1 -- 172.21.15.16:0/151433 learned_addr learned my addr 172.21.15.16:0/151433 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:57.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.577+0000 7fb079f23640 1 --2- 172.21.15.16:0/151433 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0740b9220 0x7fb07415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.577+0000 7fb079722640 1 --2- 172.21.15.16:0/151433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb060004800 0x7fb0741e9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.577+0000 7fb078f21640 1 -- 172.21.15.16:0/151433 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0740b9220 msgr2=0x7fb07415d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:57.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.577+0000 7fb078f21640 1 --2- 172.21.15.16:0/151433 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0740b9220 0x7fb07415d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.577+0000 7fb078f21640 1 -- 172.21.15.16:0/151433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb060004800 msgr2=0x7fb0741e9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:57.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.577+0000 7fb078f21640 1 --2- 172.21.15.16:0/151433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb060004800 0x7fb0741e9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.577+0000 7fb078f21640 1 -- 172.21.15.16:0/151433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0741ea230 con 0x7fb0741e87c0 2024-09-19T07:54:57.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.577+0000 7fb078f21640 1 --2- 172.21.15.16:0/151433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0741e87c0 0x7fb074154b10 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fb064034040 tx=0x7fb064004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:57.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.577+0000 7fb069ffb640 1 -- 172.21.15.16:0/151433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064041020 con 0x7fb0741e87c0 2024-09-19T07:54:57.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.577+0000 7fb069ffb640 1 -- 172.21.15.16:0/151433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb06403c470 con 0x7fb0741e87c0 2024-09-19T07:54:57.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.577+0000 7fb069ffb640 1 -- 172.21.15.16:0/151433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064036040 con 0x7fb0741e87c0 2024-09-19T07:54:57.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.577+0000 7fb07b986640 1 -- 172.21.15.16:0/151433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb074155050 con 0x7fb0741e87c0 2024-09-19T07:54:57.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.578+0000 7fb07b986640 1 -- 172.21.15.16:0/151433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0741e8f30 con 0x7fb0741e87c0 2024-09-19T07:54:57.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.579+0000 7fb07b986640 1 -- 172.21.15.16:0/151433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0740cfad0 con 0x7fb0741e87c0 2024-09-19T07:54:57.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.579+0000 7fb069ffb640 1 -- 172.21.15.16:0/151433 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb064044020 con 0x7fb0741e87c0 2024-09-19T07:54:57.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.580+0000 7fb069ffb640 1 --2- 172.21.15.16:0/151433 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb05806c2e0 0x7fb05806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.580+0000 7fb069ffb640 1 -- 172.21.15.16:0/151433 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5335+0+0 (secure 0 0 0) 0x7fb0640abb50 con 0x7fb0741e87c0 2024-09-19T07:54:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.580+0000 7fb079722640 1 --2- 172.21.15.16:0/151433 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb05806c2e0 0x7fb05806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.581+0000 7fb079722640 1 --2- 172.21.15.16:0/151433 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb05806c2e0 0x7fb05806e7a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb0700017e0 tx=0x7fb070007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.584+0000 7fb069ffb640 1 -- 172.21.15.16:0/151433 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb06404b030 con 0x7fb0741e87c0 2024-09-19T07:54:57.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.795+0000 7fb07b986640 1 -- 172.21.15.16:0/151433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb07406cf90 con 0x7fb0741e87c0 2024-09-19T07:54:57.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.796+0000 7fb069ffb640 1 -- 172.21.15.16:0/151433 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+3911 (secure 0 0 0) 0x7fb064079ea0 con 0x7fb0741e87c0 2024-09-19T07:54:57.796 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:57.796 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":76,"btime":"2024-09-19T07:54:51:928941+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/2994096322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2994096322},{"type":"v1","addr":"172.21.15.146:6836","nonce":2994096322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7444,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3892966105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":3892966105},{"type":"v1","addr":"172.21.15.146:6837","nonce":3892966105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7839,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/393564462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":393564462},{"type":"v1","addr":"172.21.15.16:6833","nonce":393564462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":7842,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/631392582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":631392582},{"type":"v1","addr":"172.21.15.16:6835","nonce":631392582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[]} 2024-09-19T07:54:57.797 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 76 2024-09-19T07:54:57.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.798+0000 7fb07b986640 1 -- 172.21.15.16:0/151433 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb05806c2e0 msgr2=0x7fb05806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:57.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.798+0000 7fb07b986640 1 --2- 172.21.15.16:0/151433 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb05806c2e0 0x7fb05806e7a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb0700017e0 tx=0x7fb070007040 comp rx=0 tx=0).stop 2024-09-19T07:54:57.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.798+0000 7fb07b986640 1 -- 172.21.15.16:0/151433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0741e87c0 msgr2=0x7fb074154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:57.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.798+0000 7fb07b986640 1 --2- 172.21.15.16:0/151433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0741e87c0 0x7fb074154b10 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fb064034040 tx=0x7fb064004730 comp rx=0 tx=0).stop 2024-09-19T07:54:57.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.798+0000 7fb07b986640 1 -- 172.21.15.16:0/151433 shutdown_connections 2024-09-19T07:54:57.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.798+0000 7fb07b986640 1 --2- 172.21.15.16:0/151433 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb05806c2e0 0x7fb05806e7a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.798+0000 7fb07b986640 1 --2- 172.21.15.16:0/151433 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0740b9220 0x7fb07415d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.798+0000 7fb07b986640 1 --2- 172.21.15.16:0/151433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0741e87c0 0x7fb074154b10 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.798+0000 7fb07b986640 1 --2- 172.21.15.16:0/151433 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb060004800 0x7fb0741e9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.798+0000 7fb07b986640 1 -- 172.21.15.16:0/151433 >> 172.21.15.16:0/151433 conn(0x7fb07409d760 msgr2=0x7fb074160ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:57.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.798+0000 7fb07b986640 1 -- 172.21.15.16:0/151433 shutdown_connections 2024-09-19T07:54:57.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.798+0000 7fb07b986640 1 -- 172.21.15.16:0/151433 wait complete. 2024-09-19T07:54:57.810 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T07:54:57.810 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T07:54:57.810 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T07:54:57.810 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:54:57.810 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T07:54:57.810 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T07:54:57.811 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T07:54:57.811 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:54:57.811 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T07:54:57.811 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T07:54:57.811 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T07:54:57.811 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:54:57.811 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T07:54:57.811 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T07:54:57.812 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T07:54:57.812 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:54:57.820 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T07:54:57.820 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T07:54:57.821 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T07:54:57.821 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T07:54:57.822 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T07:54:57.823 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T07:54:57.823 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T07:54:57.823 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T07:54:57.863 INFO:tasks.ceph.mds.a:Started 2024-09-19T07:54:57.863 DEBUG:teuthology.parallel:result is None 2024-09-19T07:54:57.865 INFO:tasks.ceph.mds.c:Started 2024-09-19T07:54:57.865 DEBUG:teuthology.parallel:result is None 2024-09-19T07:54:57.866 INFO:tasks.ceph.mds.b:Started 2024-09-19T07:54:57.866 DEBUG:teuthology.parallel:result is None 2024-09-19T07:54:57.867 INFO:tasks.ceph.mds.d:Started 2024-09-19T07:54:57.867 DEBUG:teuthology.parallel:result is None 2024-09-19T07:54:57.867 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T07:54:57.940 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-19T07:54:57.941 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-19T07:54:57.951 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2024-09-19T07:54:57.951 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2024-09-19T07:54:57.952 INFO:tasks.ceph.mds.b.smithi146.stderr:2024-09-19T07:54:57.951+0000 7f1edc394640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-19T07:54:57.952 INFO:tasks.ceph.mds.d.smithi146.stderr:2024-09-19T07:54:57.952+0000 7f5d80341640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6836/0: (98) Address already in use 2024-09-19T07:54:57.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.975+0000 7f51f1f3b640 1 Processor -- start 2024-09-19T07:54:57.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.976+0000 7f51f1f3b640 1 -- start start 2024-09-19T07:54:57.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.976+0000 7f51f1f3b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51ec15f0d0 0x7f51ec15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.976+0000 7f51f1f3b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f51ec0d0070 0x7f51ec0d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.976+0000 7f51f1f3b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f51ec0d1940 0x7f51ec0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.976+0000 7f51f1f3b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f51ec0cf2b0 con 0x7f51ec0d1940 2024-09-19T07:54:57.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.976+0000 7f51f1f3b640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f51ec0cf590 con 0x7f51ec0d0070 2024-09-19T07:54:57.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.976+0000 7f51f1f3b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f51ec0cf870 con 0x7f51ec15f0d0 2024-09-19T07:54:57.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.977+0000 7f51ebfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f51ec0d1940 0x7f51ec0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.977+0000 7f51ebfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f51ec0d1940 0x7f51ec0d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51160/0 (socket says 172.21.15.16:51160) 2024-09-19T07:54:57.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.977+0000 7f51ebfff640 1 -- 172.21.15.16:0/2620135400 learned_addr learned my addr 172.21.15.16:0/2620135400 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:57.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.977+0000 7f51eb7fe640 1 --2- 172.21.15.16:0/2620135400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51ec15f0d0 0x7f51ec15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.977+0000 7f51eaffd640 1 --2- 172.21.15.16:0/2620135400 >> v2:172.21.15.146:3300/0 conn(0x7f51ec0d0070 0x7f51ec0d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.977+0000 7f51ebfff640 1 -- 172.21.15.16:0/2620135400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51ec15f0d0 msgr2=0x7f51ec15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:57.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.977+0000 7f51ebfff640 1 --2- 172.21.15.16:0/2620135400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51ec15f0d0 0x7f51ec15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.977+0000 7f51ebfff640 1 -- 172.21.15.16:0/2620135400 >> v2:172.21.15.146:3300/0 conn(0x7f51ec0d0070 msgr2=0x7f51ec0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:57.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.977+0000 7f51ebfff640 1 --2- 172.21.15.16:0/2620135400 >> v2:172.21.15.146:3300/0 conn(0x7f51ec0d0070 0x7f51ec0d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.977+0000 7f51ebfff640 1 -- 172.21.15.16:0/2620135400 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ec0cfb50 con 0x7f51ec0d1940 2024-09-19T07:54:57.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.978+0000 7f51ebfff640 1 --2- 172.21.15.16:0/2620135400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec0d1940 0x7f51ec0d4130 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0d1f60 tx=0x7f51e402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6714da3377de418 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:57.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.978+0000 7f51ea7fc640 1 -- 172.21.15.16:0/2620135400 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4036070 con 0x7f51ec0d1940 2024-09-19T07:54:57.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.978+0000 7f51ea7fc640 1 -- 172.21.15.16:0/2620135400 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e4031600 con 0x7f51ec0d1940 2024-09-19T07:54:57.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.978+0000 7f51ea7fc640 1 -- 172.21.15.16:0/2620135400 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e40319d0 con 0x7f51ec0d1940 2024-09-19T07:54:57.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.978+0000 7f51f1f3b640 1 -- 172.21.15.16:0/2620135400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec0d1940 msgr2=0x7f51ec0d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:57.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.978+0000 7f51f1f3b640 1 --2- 172.21.15.16:0/2620135400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec0d1940 0x7f51ec0d4130 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0d1f60 tx=0x7f51e402a610 comp rx=0 tx=0).stop 2024-09-19T07:54:57.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.978+0000 7f51f1f3b640 1 -- 172.21.15.16:0/2620135400 shutdown_connections 2024-09-19T07:54:57.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.978+0000 7f51f1f3b640 1 --2- 172.21.15.16:0/2620135400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec0d1940 0x7f51ec0d4130 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.978+0000 7f51f1f3b640 1 --2- 172.21.15.16:0/2620135400 >> v2:172.21.15.146:3300/0 conn(0x7f51ec0d0070 0x7f51ec0d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.978+0000 7f51f1f3b640 1 --2- 172.21.15.16:0/2620135400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51ec15f0d0 0x7f51ec15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.978+0000 7f51f1f3b640 1 -- 172.21.15.16:0/2620135400 >> 172.21.15.16:0/2620135400 conn(0x7f51ec09d760 msgr2=0x7f51ec08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.978+0000 7f51f1f3b640 1 -- 172.21.15.16:0/2620135400 shutdown_connections 2024-09-19T07:54:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.978+0000 7f51f1f3b640 1 -- 172.21.15.16:0/2620135400 wait complete. 2024-09-19T07:54:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.980+0000 7f51f1f3b640 1 Processor -- start 2024-09-19T07:54:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.980+0000 7f51f1f3b640 1 -- start start 2024-09-19T07:54:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.980+0000 7f51f1f3b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51ec0d0070 0x7f51ec14d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51f1f3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec0d1940 0x7f51ec0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51f1f3b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51ec15f0d0 0x7f51ec14aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51f1f3b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec0d47f0 con 0x7f51ec0d1940 2024-09-19T07:54:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51f1f3b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec0d4670 con 0x7f51ec15f0d0 2024-09-19T07:54:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51f1f3b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f51ec0d4970 con 0x7f51ec0d0070 2024-09-19T07:54:57.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51eaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec0d1940 0x7f51ec0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51eaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec0d1940 0x7f51ec0ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51168/0 (socket says 172.21.15.16:51168) 2024-09-19T07:54:57.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51eaffd640 1 -- 172.21.15.16:0/3961669890 learned_addr learned my addr 172.21.15.16:0/3961669890 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:57.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51eb7fe640 1 --2- 172.21.15.16:0/3961669890 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51ec0d0070 0x7f51ec14d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51ebfff640 1 --2- 172.21.15.16:0/3961669890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51ec15f0d0 0x7f51ec14aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51eaffd640 1 -- 172.21.15.16:0/3961669890 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51ec0d0070 msgr2=0x7f51ec14d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:57.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51eaffd640 1 --2- 172.21.15.16:0/3961669890 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51ec0d0070 0x7f51ec14d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51eaffd640 1 -- 172.21.15.16:0/3961669890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51ec15f0d0 msgr2=0x7f51ec14aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:57.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51eaffd640 1 --2- 172.21.15.16:0/3961669890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51ec15f0d0 0x7f51ec14aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:57.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51eaffd640 1 -- 172.21.15.16:0/3961669890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ec14de30 con 0x7f51ec0d1940 2024-09-19T07:54:57.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51eb7fe640 1 --2- 172.21.15.16:0/3961669890 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51ec0d0070 0x7f51ec14d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:54:57.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51eaffd640 1 --2- 172.21.15.16:0/3961669890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec0d1940 0x7f51ec0ce6c0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f51e000e840 tx=0x7f51e000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:57.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51d3fff640 1 -- 172.21.15.16:0/3961669890 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e0017070 con 0x7f51ec0d1940 2024-09-19T07:54:57.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51d3fff640 1 -- 172.21.15.16:0/3961669890 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e000c730 con 0x7f51ec0d1940 2024-09-19T07:54:57.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51d3fff640 1 -- 172.21.15.16:0/3961669890 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e000cb00 con 0x7f51ec0d1940 2024-09-19T07:54:57.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.981+0000 7f51f1f3b640 1 -- 172.21.15.16:0/3961669890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ec14e110 con 0x7f51ec0d1940 2024-09-19T07:54:57.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.982+0000 7f51f1f3b640 1 -- 172.21.15.16:0/3961669890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ec0c8090 con 0x7f51ec0d1940 2024-09-19T07:54:57.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.983+0000 7f51f1f3b640 1 -- 172.21.15.16:0/3961669890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51ec14e630 con 0x7f51ec0d1940 2024-09-19T07:54:57.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.983+0000 7f51d3fff640 1 -- 172.21.15.16:0/3961669890 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f51e001a070 con 0x7f51ec0d1940 2024-09-19T07:54:57.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.984+0000 7f51d3fff640 1 --2- 172.21.15.16:0/3961669890 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f51d806c2b0 0x7f51d806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:57.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.984+0000 7f51d3fff640 1 -- 172.21.15.16:0/3961669890 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5335+0+0 (secure 0 0 0) 0x7f51e008b120 con 0x7f51ec0d1940 2024-09-19T07:54:57.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.984+0000 7f51eb7fe640 1 --2- 172.21.15.16:0/3961669890 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f51d806c2b0 0x7f51d806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:57.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.985+0000 7f51eb7fe640 1 --2- 172.21.15.16:0/3961669890 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f51d806c2b0 0x7f51d806e770 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009f30 tx=0x7f51dc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:57.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:57.988+0000 7f51d3fff640 1 -- 172.21.15.16:0/3961669890 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f51ec14e630 con 0x7f51ec0d1940 2024-09-19T07:54:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:58.177+0000 7f51f1f3b640 1 -- 172.21.15.16:0/3961669890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f51ec0d1400 con 0x7f51ec0d1940 2024-09-19T07:54:58.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:58.951+0000 7f51d3fff640 1 -- 172.21.15.16:0/3961669890 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v136) ==== 97+0+0 (secure 0 0 0) 0x7f51e0059310 con 0x7f51ec0d1940 2024-09-19T07:54:58.952 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-19T07:54:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:58.954+0000 7f51f1f3b640 1 -- 172.21.15.16:0/3961669890 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f51d806c2b0 msgr2=0x7f51d806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:58.954+0000 7f51f1f3b640 1 --2- 172.21.15.16:0/3961669890 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f51d806c2b0 0x7f51d806e770 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009f30 tx=0x7f51dc009190 comp rx=0 tx=0).stop 2024-09-19T07:54:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:58.954+0000 7f51f1f3b640 1 -- 172.21.15.16:0/3961669890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec0d1940 msgr2=0x7f51ec0ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:58.954+0000 7f51f1f3b640 1 --2- 172.21.15.16:0/3961669890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec0d1940 0x7f51ec0ce6c0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f51e000e840 tx=0x7f51e000ed00 comp rx=0 tx=0).stop 2024-09-19T07:54:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:58.954+0000 7f51f1f3b640 1 -- 172.21.15.16:0/3961669890 shutdown_connections 2024-09-19T07:54:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:58.954+0000 7f51f1f3b640 1 --2- 172.21.15.16:0/3961669890 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f51d806c2b0 0x7f51d806e770 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:58.954+0000 7f51f1f3b640 1 --2- 172.21.15.16:0/3961669890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51ec15f0d0 0x7f51ec14aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:58.954+0000 7f51f1f3b640 1 --2- 172.21.15.16:0/3961669890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec0d1940 0x7f51ec0ce6c0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:58.954+0000 7f51f1f3b640 1 --2- 172.21.15.16:0/3961669890 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51ec0d0070 0x7f51ec14d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:58.954+0000 7f51f1f3b640 1 -- 172.21.15.16:0/3961669890 >> 172.21.15.16:0/3961669890 conn(0x7f51ec09d760 msgr2=0x7f51ec161eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:58.954+0000 7f51f1f3b640 1 -- 172.21.15.16:0/3961669890 shutdown_connections 2024-09-19T07:54:58.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:58.954+0000 7f51f1f3b640 1 -- 172.21.15.16:0/3961669890 wait complete. 2024-09-19T07:54:58.968 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T07:54:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.077+0000 7f314c2d0640 1 Processor -- start 2024-09-19T07:54:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.077+0000 7f314c2d0640 1 -- start start 2024-09-19T07:54:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.078+0000 7f314c2d0640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f314415e2a0 0x7f314406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.078+0000 7f314c2d0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31440d1880 con 0x7f314415e680 2024-09-19T07:54:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.078+0000 7f314c2d0640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31440d1b60 con 0x7f314406d960 2024-09-19T07:54:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.078+0000 7f314c2d0640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f31440cf2d0 con 0x7f314415e2a0 2024-09-19T07:54:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.078+0000 7f314986b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f314415e680 0x7f31440cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33826/0 (socket says 172.21.15.16:33826) 2024-09-19T07:54:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.078+0000 7f314986b640 1 -- 172.21.15.16:0/1620928776 learned_addr learned my addr 172.21.15.16:0/1620928776 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.078+0000 7f314a06c640 1 --2- 172.21.15.16:0/1620928776 >> v2:172.21.15.146:3300/0 conn(0x7f314415e2a0 0x7f314406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.078+0000 7f314906a640 1 -- 172.21.15.16:0/1620928776 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3633214156 0 0) 0x7f31440d1880 con 0x7f314415e680 2024-09-19T07:54:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.078+0000 7f314906a640 1 -- 172.21.15.16:0/1620928776 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3130003520 con 0x7f314415e680 2024-09-19T07:54:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314906a640 1 -- 172.21.15.16:0/1620928776 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4285728695 0 0) 0x7f3130003520 con 0x7f314415e680 2024-09-19T07:54:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314906a640 1 -- 172.21.15.16:0/1620928776 >> v2:172.21.15.146:3300/0 conn(0x7f314415e2a0 msgr2=0x7f314406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314906a640 1 --2- 172.21.15.16:0/1620928776 >> v2:172.21.15.146:3300/0 conn(0x7f314415e2a0 0x7f314406d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314906a640 1 -- 172.21.15.16:0/1620928776 >> v1:172.21.15.146:6789/0 conn(0x7f314406d960 legacy=0x7f31440d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314906a640 1 -- 172.21.15.16:0/1620928776 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31440cf5b0 con 0x7f314415e680 2024-09-19T07:54:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314906a640 1 -- 172.21.15.16:0/1620928776 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f3134001670 con 0x7f314415e680 2024-09-19T07:54:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314906a640 1 -- 172.21.15.16:0/1620928776 >> v1:172.21.15.16:6789/0 conn(0x7f314415e680 legacy=0x7f31440cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314906a640 1 --2- 172.21.15.16:0/1620928776 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31300039b0 0x7f3130003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314906a640 1 --2- 172.21.15.16:0/1620928776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31300043b0 0x7f3130004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314906a640 1 --2- 172.21.15.16:0/1620928776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3130005080 0x7f313000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314906a640 1 -- 172.21.15.16:0/1620928776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f31440cf2d0 con 0x7f3130005080 2024-09-19T07:54:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314906a640 1 -- 172.21.15.16:0/1620928776 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f313000bfe0 con 0x7f31300043b0 2024-09-19T07:54:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314906a640 1 -- 172.21.15.16:0/1620928776 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f313000c1e0 con 0x7f31300039b0 2024-09-19T07:54:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314a86d640 1 --2- 172.21.15.16:0/1620928776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3130005080 0x7f313000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314986b640 1 --2- 172.21.15.16:0/1620928776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31300043b0 0x7f3130004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314a06c640 1 --2- 172.21.15.16:0/1620928776 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31300039b0 0x7f3130003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314a86d640 1 -- 172.21.15.16:0/1620928776 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31300039b0 msgr2=0x7f3130003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314a86d640 1 --2- 172.21.15.16:0/1620928776 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31300039b0 0x7f3130003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314a86d640 1 -- 172.21.15.16:0/1620928776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31300043b0 msgr2=0x7f3130004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314a86d640 1 --2- 172.21.15.16:0/1620928776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31300043b0 0x7f3130004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314a86d640 1 -- 172.21.15.16:0/1620928776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3130003520 con 0x7f3130005080 2024-09-19T07:54:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314a06c640 1 --2- 172.21.15.16:0/1620928776 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31300039b0 0x7f3130003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.079+0000 7f314986b640 1 --2- 172.21.15.16:0/1620928776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31300043b0 0x7f3130004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.080+0000 7f314a86d640 1 -- 172.21.15.16:0/1620928776 reap_dead start 2024-09-19T07:54:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.080+0000 7f314a86d640 1 --2- 172.21.15.16:0/1620928776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3130005080 0x7f313000b9c0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f313c002010 tx=0x7f3130005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2285757ac5b24b9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.080+0000 7f314906a640 1 -- 172.21.15.16:0/1620928776 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c001be0 con 0x7f3130005080 2024-09-19T07:54:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.080+0000 7f314906a640 1 -- 172.21.15.16:0/1620928776 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f313c003b70 con 0x7f3130005080 2024-09-19T07:54:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.080+0000 7f314906a640 1 -- 172.21.15.16:0/1620928776 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c005110 con 0x7f3130005080 2024-09-19T07:54:59.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.081+0000 7f314c2d0640 1 -- 172.21.15.16:0/1620928776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3130005080 msgr2=0x7f313000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:59.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.081+0000 7f314c2d0640 1 --2- 172.21.15.16:0/1620928776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3130005080 0x7f313000b9c0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f313c002010 tx=0x7f3130005830 comp rx=0 tx=0).stop 2024-09-19T07:54:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.081+0000 7f314c2d0640 1 -- 172.21.15.16:0/1620928776 shutdown_connections 2024-09-19T07:54:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.081+0000 7f314c2d0640 1 --2- 172.21.15.16:0/1620928776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3130005080 0x7f313000b9c0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.081+0000 7f314c2d0640 1 -- 172.21.15.16:0/1620928776 >> 172.21.15.16:0/1620928776 conn(0x7f314409d760 msgr2=0x7f314408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.081+0000 7f314c2d0640 1 -- 172.21.15.16:0/1620928776 shutdown_connections 2024-09-19T07:54:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.081+0000 7f314c2d0640 1 -- 172.21.15.16:0/1620928776 wait complete. 2024-09-19T07:54:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.082+0000 7f314c2d0640 1 Processor -- start 2024-09-19T07:54:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.082+0000 7f314c2d0640 1 -- start start 2024-09-19T07:54:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.083+0000 7f314c2d0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3130005080 0x7f31440cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.083+0000 7f314c2d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31440b7210 0x7f31441564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.083+0000 7f314c2d0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31440c9f60 0x7f31440ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.083+0000 7f314c2d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f314406d5a0 con 0x7f31440b7210 2024-09-19T07:54:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.083+0000 7f314c2d0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f314406d420 con 0x7f3130005080 2024-09-19T07:54:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.083+0000 7f314c2d0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f314406d720 con 0x7f31440c9f60 2024-09-19T07:54:59.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.083+0000 7f314986b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31440b7210 0x7f31441564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:59.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.083+0000 7f314986b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31440b7210 0x7f31441564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51186/0 (socket says 172.21.15.16:51186) 2024-09-19T07:54:59.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.083+0000 7f314986b640 1 -- 172.21.15.16:0/3940214684 learned_addr learned my addr 172.21.15.16:0/3940214684 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:59.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.083+0000 7f314a06c640 1 --2- 172.21.15.16:0/3940214684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3130005080 0x7f31440cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:59.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.083+0000 7f314a86d640 1 --2- 172.21.15.16:0/3940214684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31440c9f60 0x7f31440ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:59.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.083+0000 7f314986b640 1 -- 172.21.15.16:0/3940214684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31440c9f60 msgr2=0x7f31440ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:59.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.083+0000 7f314986b640 1 --2- 172.21.15.16:0/3940214684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31440c9f60 0x7f31440ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.083+0000 7f314986b640 1 -- 172.21.15.16:0/3940214684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3130005080 msgr2=0x7f31440cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:59.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.083+0000 7f314986b640 1 --2- 172.21.15.16:0/3940214684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3130005080 0x7f31440cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.083+0000 7f314986b640 1 -- 172.21.15.16:0/3940214684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31440c8f20 con 0x7f31440b7210 2024-09-19T07:54:59.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.084+0000 7f314986b640 1 --2- 172.21.15.16:0/3940214684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31440b7210 0x7f31441564e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f313400e760 tx=0x7f313400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:59.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.084+0000 7f313affd640 1 -- 172.21.15.16:0/3940214684 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313400c660 con 0x7f31440b7210 2024-09-19T07:54:59.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.084+0000 7f313affd640 1 -- 172.21.15.16:0/3940214684 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3134018070 con 0x7f31440b7210 2024-09-19T07:54:59.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.084+0000 7f314c2d0640 1 -- 172.21.15.16:0/3940214684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31440c9200 con 0x7f31440b7210 2024-09-19T07:54:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.084+0000 7f313affd640 1 -- 172.21.15.16:0/3940214684 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313400ca40 con 0x7f31440b7210 2024-09-19T07:54:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.084+0000 7f314c2d0640 1 -- 172.21.15.16:0/3940214684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31440b7690 con 0x7f31440b7210 2024-09-19T07:54:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.085+0000 7f314c2d0640 1 -- 172.21.15.16:0/3940214684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31440d0c50 con 0x7f31440b7210 2024-09-19T07:54:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.085+0000 7f313affd640 1 -- 172.21.15.16:0/3940214684 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f313400cbe0 con 0x7f31440b7210 2024-09-19T07:54:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.086+0000 7f313affd640 1 --2- 172.21.15.16:0/3940214684 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f312806c2e0 0x7f312806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.086+0000 7f313affd640 1 -- 172.21.15.16:0/3940214684 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5249+0+0 (secure 0 0 0) 0x7f313408ab80 con 0x7f31440b7210 2024-09-19T07:54:59.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.086+0000 7f314a06c640 1 --2- 172.21.15.16:0/3940214684 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f312806c2e0 0x7f312806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:59.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.087+0000 7f314a06c640 1 --2- 172.21.15.16:0/3940214684 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f312806c2e0 0x7f312806e7a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f31441ea240 tx=0x7f3140008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:59.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.090+0000 7f313affd640 1 -- 172.21.15.16:0/3940214684 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3134058e60 con 0x7f31440b7210 2024-09-19T07:54:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.333+0000 7f314c2d0640 1 -- 172.21.15.16:0/3940214684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f314406cf90 con 0x7f31440b7210 2024-09-19T07:54:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.334+0000 7f313affd640 1 -- 172.21.15.16:0/3940214684 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7f313405d160 con 0x7f31440b7210 2024-09-19T07:54:59.334 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:59.334 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:54:59.334 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-19T07:54:59.335 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.335 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-19T07:54:59.335 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etms80KDxAAqTrG88LTEJIj9TWJu0witQ==", 2024-09-19T07:54:59.335 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:59.335 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:54:59.335 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:54:59.335 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:54:59.335 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:59.335 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:59.335 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.335 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-19T07:54:59.335 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etm4T0KGxAAcfDOKrSdKzhJ4xvTFIyxcQ==", 2024-09-19T07:54:59.335 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:59.336 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:54:59.336 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:54:59.336 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:54:59.336 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:59.336 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.336 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:59.336 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.336 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-19T07:54:59.336 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNDBGFhAAtBioJJ1+NtIDLhd/Hg7aZA==", 2024-09-19T07:54:59.336 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:59.337 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:54:59.337 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:54:59.337 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:54:59.337 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:59.337 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.337 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:59.337 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.337 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-19T07:54:59.337 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmaHEiIhAACqqjCRMV+xlGh9WqYsj+MA==", 2024-09-19T07:54:59.338 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:59.338 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:54:59.338 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:54:59.338 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:54:59.338 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:59.338 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.338 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:59.338 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.338 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-19T07:54:59.339 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDT1etm09plCxAAGvJYF6JrKw49FtXb71ox3A==", 2024-09-19T07:54:59.339 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:59.339 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:54:59.339 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:54:59.339 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:59.339 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.340 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:59.340 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.340 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-19T07:54:59.340 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDW1etmGy8yHRAAlcDsHLnkjtS6sCCR8Olv2A==", 2024-09-19T07:54:59.340 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:59.341 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:54:59.341 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:54:59.341 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:59.341 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.341 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:59.341 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.341 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-19T07:54:59.342 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDZ1etmMhGTHxAATQaW2kgJhHKftutQQtUa6A==", 2024-09-19T07:54:59.342 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:59.342 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:54:59.342 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:54:59.342 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:59.342 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.342 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:59.342 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.343 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-19T07:54:59.343 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDc1etmD3sLIhAAIdY00buvb3bIFA5ZpgxxjA==", 2024-09-19T07:54:59.343 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:59.343 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:54:59.343 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:54:59.343 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:59.343 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.344 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:59.344 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.344 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-19T07:54:59.344 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDi1etmgutcLRAAHJEl5SfQ7N0T4YoYeBfrQg==", 2024-09-19T07:54:59.344 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:59.344 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:54:59.345 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:54:59.345 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:59.345 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.345 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:59.346 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.346 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-19T07:54:59.346 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDm1etmCV3xARAA+5yhQEmIpyNId3RL1Xsxhw==", 2024-09-19T07:54:59.346 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:59.346 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:54:59.346 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:54:59.346 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:59.347 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.347 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:59.347 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.347 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-19T07:54:59.347 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDp1etm9XC+AxAA+Lrs0ExfSKaixgOxfX8Nug==", 2024-09-19T07:54:59.347 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:59.347 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:54:59.347 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:54:59.347 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:59.348 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:59.348 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.348 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-19T07:54:59.348 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDs1etmhmvMBhAAhvW/udIE2gCd8IrJCSFuvw==", 2024-09-19T07:54:59.348 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:59.348 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:54:59.348 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:54:59.348 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:59.348 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:59.348 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.348 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-19T07:54:59.348 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA==", 2024-09-19T07:54:59.349 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:59.349 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:54:59.349 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-19T07:54:59.349 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T07:54:59.349 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.349 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:59.349 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.349 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-19T07:54:59.349 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmc05KLxAAPQl6wOHqFIf8LVxskiuLVg==", 2024-09-19T07:54:59.349 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:59.349 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:54:59.350 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:54:59.350 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:54:59.350 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:54:59.350 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.350 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:59.350 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.350 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-19T07:54:59.350 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmjFsCLBAANa7gCMvDw84Ovo7u6RRJ3Q==", 2024-09-19T07:54:59.350 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:59.350 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:54:59.350 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:54:59.350 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:54:59.350 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:54:59.350 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:59.351 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.351 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-19T07:54:59.351 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmdUbgNBAAfwstyOeeoZfetgVZu4z5xg==", 2024-09-19T07:54:59.351 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:59.351 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:54:59.351 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:54:59.351 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:54:59.351 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:54:59.351 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:59.351 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.352 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-19T07:54:59.352 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDQ1etm9/vTLhAA9PRfAOA8R4oXp3x0dU8l3g==", 2024-09-19T07:54:59.352 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:59.352 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:54:59.352 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:54:59.352 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:54:59.352 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:59.352 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.352 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:59.352 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.352 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-19T07:54:59.352 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNQ68CxAANkK+D3u/oTEz8Su4iuZrCg==", 2024-09-19T07:54:59.353 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:59.353 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:54:59.353 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:54:59.353 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:59.353 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:54:59.353 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.353 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-19T07:54:59.354 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmp+gSCBAAGXzmKfFNd6K2fWeqYhlHAg==", 2024-09-19T07:54:59.354 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:54:59.354 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:54:59.354 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:54:59.354 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:54:59.354 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.354 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.355 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T07:54:59.355 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:54:59.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.336+0000 7f314c2d0640 1 -- 172.21.15.16:0/3940214684 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f312806c2e0 msgr2=0x7f312806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:59.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.336+0000 7f314c2d0640 1 --2- 172.21.15.16:0/3940214684 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f312806c2e0 0x7f312806e7a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f31441ea240 tx=0x7f3140008040 comp rx=0 tx=0).stop 2024-09-19T07:54:59.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.336+0000 7f314c2d0640 1 -- 172.21.15.16:0/3940214684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31440b7210 msgr2=0x7f31441564e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:59.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.336+0000 7f314c2d0640 1 --2- 172.21.15.16:0/3940214684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31440b7210 0x7f31441564e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f313400e760 tx=0x7f313400ec20 comp rx=0 tx=0).stop 2024-09-19T07:54:59.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.336+0000 7f314c2d0640 1 -- 172.21.15.16:0/3940214684 shutdown_connections 2024-09-19T07:54:59.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.336+0000 7f314c2d0640 1 --2- 172.21.15.16:0/3940214684 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f312806c2e0 0x7f312806e7a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.336+0000 7f314c2d0640 1 --2- 172.21.15.16:0/3940214684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31440c9f60 0x7f31440ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.336+0000 7f314c2d0640 1 --2- 172.21.15.16:0/3940214684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31440b7210 0x7f31441564e0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.336+0000 7f314c2d0640 1 --2- 172.21.15.16:0/3940214684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3130005080 0x7f31440cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.336+0000 7f314c2d0640 1 -- 172.21.15.16:0/3940214684 >> 172.21.15.16:0/3940214684 conn(0x7f314409d760 msgr2=0x7f3144162850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:59.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.336+0000 7f314c2d0640 1 -- 172.21.15.16:0/3940214684 shutdown_connections 2024-09-19T07:54:59.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.336+0000 7f314c2d0640 1 -- 172.21.15.16:0/3940214684 wait complete. 2024-09-19T07:54:59.358 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T07:54:59.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.460+0000 7f9dc54ba640 1 Processor -- start 2024-09-19T07:54:59.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.460+0000 7f9dc54ba640 1 -- start start 2024-09-19T07:54:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.460+0000 7f9dc54ba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9dc015f0d0 0x7f9dc015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.460+0000 7f9dc54ba640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9dc00d4130 0x7f9dc00d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.460+0000 7f9dc54ba640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dc00d1b10 con 0x7f9dc00d0070 2024-09-19T07:54:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.460+0000 7f9dc54ba640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9dc00cf2b0 con 0x7f9dc015f0d0 2024-09-19T07:54:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.460+0000 7f9dc54ba640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f9dc00cf590 con 0x7f9dc00d4130 2024-09-19T07:54:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.461+0000 7f9dbeffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9dc015f0d0 0x7f9dc015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.461+0000 7f9db7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9dc00d0070 0x7f9dc00d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33842/0 (socket says 172.21.15.16:33842) 2024-09-19T07:54:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.461+0000 7f9db7fff640 1 -- 172.21.15.16:0/537947289 learned_addr learned my addr 172.21.15.16:0/537947289 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:54:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.461+0000 7f9dbeffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9dc015f0d0 0x7f9dc015f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51200/0 (socket says 172.21.15.16:51200) 2024-09-19T07:54:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.461+0000 7f9dbeffd640 1 -- 172.21.15.16:0/537947289 >> v2:172.21.15.146:3300/0 conn(0x7f9dc00d4130 msgr2=0x7f9dc00d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.461+0000 7f9dbeffd640 1 --2- 172.21.15.16:0/537947289 >> v2:172.21.15.146:3300/0 conn(0x7f9dc00d4130 0x7f9dc00d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.461+0000 7f9dbeffd640 1 -- 172.21.15.16:0/537947289 >> v1:172.21.15.16:6789/0 conn(0x7f9dc00d0070 legacy=0x7f9dc00d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.461+0000 7f9dbe7fc640 1 -- 172.21.15.16:0/537947289 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1967471962 0 0) 0x7f9dc00d1b10 con 0x7f9dc00d0070 2024-09-19T07:54:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.461+0000 7f9dbeffd640 1 -- 172.21.15.16:0/537947289 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc00cf870 con 0x7f9dc015f0d0 2024-09-19T07:54:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.462+0000 7f9dbeffd640 1 --2- 172.21.15.16:0/537947289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc015f0d0 0x7f9dc015f4b0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00d04e0 tx=0x7f9db002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b8ab9db136f2d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.462+0000 7f9dbe7fc640 1 -- 172.21.15.16:0/537947289 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0037070 con 0x7f9dc015f0d0 2024-09-19T07:54:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.462+0000 7f9dbe7fc640 1 -- 172.21.15.16:0/537947289 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db0032440 con 0x7f9dc015f0d0 2024-09-19T07:54:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.462+0000 7f9dbe7fc640 1 -- 172.21.15.16:0/537947289 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db002ae60 con 0x7f9dc015f0d0 2024-09-19T07:54:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.462+0000 7f9dc54ba640 1 -- 172.21.15.16:0/537947289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc015f0d0 msgr2=0x7f9dc015f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.462+0000 7f9dc54ba640 1 --2- 172.21.15.16:0/537947289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc015f0d0 0x7f9dc015f4b0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00d04e0 tx=0x7f9db002a110 comp rx=0 tx=0).stop 2024-09-19T07:54:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.462+0000 7f9dc54ba640 1 -- 172.21.15.16:0/537947289 shutdown_connections 2024-09-19T07:54:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.462+0000 7f9dc54ba640 1 --2- 172.21.15.16:0/537947289 >> v2:172.21.15.146:3300/0 conn(0x7f9dc00d4130 0x7f9dc00d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.462+0000 7f9dc54ba640 1 --2- 172.21.15.16:0/537947289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc015f0d0 0x7f9dc015f4b0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.462+0000 7f9dc54ba640 1 -- 172.21.15.16:0/537947289 >> 172.21.15.16:0/537947289 conn(0x7f9dc009d760 msgr2=0x7f9dc008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.463+0000 7f9dc54ba640 1 -- 172.21.15.16:0/537947289 shutdown_connections 2024-09-19T07:54:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.463+0000 7f9dc54ba640 1 -- 172.21.15.16:0/537947289 wait complete. 2024-09-19T07:54:59.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.464+0000 7f9dc54ba640 1 Processor -- start 2024-09-19T07:54:59.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.464+0000 7f9dc54ba640 1 -- start start 2024-09-19T07:54:59.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.464+0000 7f9dc54ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc00d0070 0x7f9dc00ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.464+0000 7f9dc54ba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9dc00d4130 0x7f9dc015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9dc54ba640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dc015f0d0 0x7f9dc01578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9dc54ba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc00cef60 con 0x7f9dc00d0070 2024-09-19T07:54:59.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9dc54ba640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc00d4aa0 con 0x7f9dc015f0d0 2024-09-19T07:54:59.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9dc54ba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9dc00b2f70 con 0x7f9dc00d4130 2024-09-19T07:54:59.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9dbeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc00d0070 0x7f9dc00ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:59.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9dbeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc00d0070 0x7f9dc00ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51208/0 (socket says 172.21.15.16:51208) 2024-09-19T07:54:59.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9dbeffd640 1 -- 172.21.15.16:0/3878184671 learned_addr learned my addr 172.21.15.16:0/3878184671 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:59.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9db7fff640 1 --2- 172.21.15.16:0/3878184671 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9dc00d4130 0x7f9dc015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:59.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9dbf7fe640 1 --2- 172.21.15.16:0/3878184671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dc015f0d0 0x7f9dc01578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:59.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9dbeffd640 1 -- 172.21.15.16:0/3878184671 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9dc00d4130 msgr2=0x7f9dc015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:59.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9dbeffd640 1 --2- 172.21.15.16:0/3878184671 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9dc00d4130 0x7f9dc015a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9dbeffd640 1 -- 172.21.15.16:0/3878184671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dc015f0d0 msgr2=0x7f9dc01578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:59.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9dbeffd640 1 --2- 172.21.15.16:0/3878184671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dc015f0d0 0x7f9dc01578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9dbeffd640 1 -- 172.21.15.16:0/3878184671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc00cd6b0 con 0x7f9dc00d0070 2024-09-19T07:54:59.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9dbeffd640 1 --2- 172.21.15.16:0/3878184671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc00d0070 0x7f9dc00ce020 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f9db0035040 tx=0x7f9db00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:59.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9db77fe640 1 -- 172.21.15.16:0/3878184671 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0037040 con 0x7f9dc00d0070 2024-09-19T07:54:59.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9db77fe640 1 -- 172.21.15.16:0/3878184671 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db0004230 con 0x7f9dc00d0070 2024-09-19T07:54:59.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9dc54ba640 1 -- 172.21.15.16:0/3878184671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc00cce10 con 0x7f9dc00d0070 2024-09-19T07:54:59.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9db77fe640 1 -- 172.21.15.16:0/3878184671 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0032e20 con 0x7f9dc00d0070 2024-09-19T07:54:59.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.465+0000 7f9dc54ba640 1 -- 172.21.15.16:0/3878184671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc00cd3d0 con 0x7f9dc00d0070 2024-09-19T07:54:59.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.466+0000 7f9dc54ba640 1 -- 172.21.15.16:0/3878184671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc00d4560 con 0x7f9dc00d0070 2024-09-19T07:54:59.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.467+0000 7f9db77fe640 1 -- 172.21.15.16:0/3878184671 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9db0030770 con 0x7f9dc00d0070 2024-09-19T07:54:59.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.467+0000 7f9db77fe640 1 --2- 172.21.15.16:0/3878184671 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9da006c2b0 0x7f9da006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.467+0000 7f9db77fe640 1 -- 172.21.15.16:0/3878184671 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5249+0+0 (secure 0 0 0) 0x7f9db00ac5f0 con 0x7f9dc00d0070 2024-09-19T07:54:59.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.468+0000 7f9db7fff640 1 --2- 172.21.15.16:0/3878184671 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9da006c2b0 0x7f9da006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:59.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.468+0000 7f9db7fff640 1 --2- 172.21.15.16:0/3878184671 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9da006c2b0 0x7f9da006e770 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00b1aa0 tx=0x7f9da8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:59.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.472+0000 7f9db77fe640 1 -- 172.21.15.16:0/3878184671 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9db007b0d0 con 0x7f9dc00d0070 2024-09-19T07:54:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.649+0000 7f9dc54ba640 1 -- 172.21.15.16:0/3878184671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9dc006cf30 con 0x7f9dc00d0070 2024-09-19T07:54:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.650+0000 7f9db77fe640 1 -- 172.21.15.16:0/3878184671 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v136) ==== 84+0+65 (secure 0 0 0) 0x7f9db007a9d0 con 0x7f9dc00d0070 2024-09-19T07:54:59.650 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:54:59.650 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-19T07:54:59.651 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:54:59.651 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-19T07:54:59.651 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-19T07:54:59.651 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:54:59.651 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-19T07:54:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.652+0000 7f9dc54ba640 1 -- 172.21.15.16:0/3878184671 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9da006c2b0 msgr2=0x7f9da006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.652+0000 7f9dc54ba640 1 --2- 172.21.15.16:0/3878184671 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9da006c2b0 0x7f9da006e770 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00b1aa0 tx=0x7f9da8009040 comp rx=0 tx=0).stop 2024-09-19T07:54:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.652+0000 7f9dc54ba640 1 -- 172.21.15.16:0/3878184671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc00d0070 msgr2=0x7f9dc00ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.652+0000 7f9dc54ba640 1 --2- 172.21.15.16:0/3878184671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc00d0070 0x7f9dc00ce020 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f9db0035040 tx=0x7f9db00046c0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.652+0000 7f9dc54ba640 1 -- 172.21.15.16:0/3878184671 shutdown_connections 2024-09-19T07:54:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.652+0000 7f9dc54ba640 1 --2- 172.21.15.16:0/3878184671 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9da006c2b0 0x7f9da006e770 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.652+0000 7f9dc54ba640 1 --2- 172.21.15.16:0/3878184671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dc015f0d0 0x7f9dc01578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.652+0000 7f9dc54ba640 1 --2- 172.21.15.16:0/3878184671 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9dc00d4130 0x7f9dc015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.652+0000 7f9dc54ba640 1 --2- 172.21.15.16:0/3878184671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc00d0070 0x7f9dc00ce020 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.652+0000 7f9dc54ba640 1 -- 172.21.15.16:0/3878184671 >> 172.21.15.16:0/3878184671 conn(0x7f9dc009d760 msgr2=0x7f9dc00d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.652+0000 7f9dc54ba640 1 -- 172.21.15.16:0/3878184671 shutdown_connections 2024-09-19T07:54:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.652+0000 7f9dc54ba640 1 -- 172.21.15.16:0/3878184671 wait complete. 2024-09-19T07:54:59.663 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T07:54:59.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T07:54:59.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.770+0000 7fddbf83c640 1 Processor -- start 2024-09-19T07:54:59.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.770+0000 7fddbf83c640 1 -- start start 2024-09-19T07:54:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.771+0000 7fddbf83c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fddb806cd20 0x7fddb806d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.771+0000 7fddbf83c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fddb8161eb0 0x7fddb81622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.771+0000 7fddbf83c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddb806d640 con 0x7fddb8143a20 2024-09-19T07:54:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.771+0000 7fddbf83c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fddb8162e50 con 0x7fddb8161eb0 2024-09-19T07:54:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.771+0000 7fddbf83c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fddb8142150 con 0x7fddb806cd20 2024-09-19T07:54:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.771+0000 7fddbcdd7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fddb8161eb0 0x7fddb81622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.771+0000 7fddbcdd7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fddb8161eb0 0x7fddb81622f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51222/0 (socket says 172.21.15.16:51222) 2024-09-19T07:54:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.771+0000 7fddbcdd7640 1 -- 172.21.15.16:0/1270863873 learned_addr learned my addr 172.21.15.16:0/1270863873 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.771+0000 7fddbd5d8640 1 --2- 172.21.15.16:0/1270863873 >> v2:172.21.15.146:3300/0 conn(0x7fddb806cd20 0x7fddb806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.771+0000 7fddbcdd7640 1 -- 172.21.15.16:0/1270863873 >> v2:172.21.15.146:3300/0 conn(0x7fddb806cd20 msgr2=0x7fddb806d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.771+0000 7fddbcdd7640 1 --2- 172.21.15.16:0/1270863873 >> v2:172.21.15.146:3300/0 conn(0x7fddb806cd20 0x7fddb806d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.771+0000 7fddbcdd7640 1 -- 172.21.15.16:0/1270863873 >> v1:172.21.15.146:6789/0 conn(0x7fddb8143a20 legacy=0x7fddb8143eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:59.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.772+0000 7fddbcdd7640 1 -- 172.21.15.16:0/1270863873 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb8162b70 con 0x7fddb8161eb0 2024-09-19T07:54:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.772+0000 7fddbcdd7640 1 --2- 172.21.15.16:0/1270863873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb8161eb0 0x7fddb81622f0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fddb8070ff0 tx=0x7fddac02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebe6d4114f4dd18 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.772+0000 7fddb7fff640 1 -- 172.21.15.16:0/1270863873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac037070 con 0x7fddb8161eb0 2024-09-19T07:54:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.772+0000 7fddb7fff640 1 -- 172.21.15.16:0/1270863873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddac032440 con 0x7fddb8161eb0 2024-09-19T07:54:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.772+0000 7fddb7fff640 1 -- 172.21.15.16:0/1270863873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac032810 con 0x7fddb8161eb0 2024-09-19T07:54:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.773+0000 7fddbf83c640 1 -- 172.21.15.16:0/1270863873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb8161eb0 msgr2=0x7fddb81622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.773+0000 7fddbf83c640 1 --2- 172.21.15.16:0/1270863873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb8161eb0 0x7fddb81622f0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fddb8070ff0 tx=0x7fddac02a1a0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.773+0000 7fddbf83c640 1 -- 172.21.15.16:0/1270863873 shutdown_connections 2024-09-19T07:54:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.773+0000 7fddbf83c640 1 --2- 172.21.15.16:0/1270863873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb8161eb0 0x7fddb81622f0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.773+0000 7fddbf83c640 1 --2- 172.21.15.16:0/1270863873 >> v2:172.21.15.146:3300/0 conn(0x7fddb806cd20 0x7fddb806d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.773+0000 7fddbf83c640 1 -- 172.21.15.16:0/1270863873 >> 172.21.15.16:0/1270863873 conn(0x7fddb80b3ed0 msgr2=0x7fddb80b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.773+0000 7fddbf83c640 1 -- 172.21.15.16:0/1270863873 shutdown_connections 2024-09-19T07:54:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.773+0000 7fddbf83c640 1 -- 172.21.15.16:0/1270863873 wait complete. 2024-09-19T07:54:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.774+0000 7fddbf83c640 1 Processor -- start 2024-09-19T07:54:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.774+0000 7fddbf83c640 1 -- start start 2024-09-19T07:54:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.775+0000 7fddbf83c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb806cd20 0x7fddb813fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.775+0000 7fddbf83c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddb8143a20 0x7fddb8140840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.775+0000 7fddbf83c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddb8161eb0 0x7fddb8140d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.775+0000 7fddbf83c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fddb8162830 con 0x7fddb806cd20 2024-09-19T07:54:59.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.775+0000 7fddbf83c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fddb8141de0 con 0x7fddb8143a20 2024-09-19T07:54:59.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.775+0000 7fddbf83c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fddb8141600 con 0x7fddb8161eb0 2024-09-19T07:54:59.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.775+0000 7fddbd5d8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb806cd20 0x7fddb813fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:59.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.775+0000 7fddbd5d8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb806cd20 0x7fddb813fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51228/0 (socket says 172.21.15.16:51228) 2024-09-19T07:54:59.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.775+0000 7fddbd5d8640 1 -- 172.21.15.16:0/1911503023 learned_addr learned my addr 172.21.15.16:0/1911503023 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:54:59.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.775+0000 7fddbddd9640 1 --2- 172.21.15.16:0/1911503023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddb8161eb0 0x7fddb8140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.775+0000 7fddbcdd7640 1 --2- 172.21.15.16:0/1911503023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddb8143a20 0x7fddb8140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.776+0000 7fddbd5d8640 1 -- 172.21.15.16:0/1911503023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddb8161eb0 msgr2=0x7fddb8140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.776+0000 7fddbd5d8640 1 --2- 172.21.15.16:0/1911503023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddb8161eb0 0x7fddb8140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.776+0000 7fddbd5d8640 1 -- 172.21.15.16:0/1911503023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddb8143a20 msgr2=0x7fddb8140840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.776+0000 7fddbd5d8640 1 --2- 172.21.15.16:0/1911503023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddb8143a20 0x7fddb8140840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.776+0000 7fddbd5d8640 1 -- 172.21.15.16:0/1911503023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb815a4d0 con 0x7fddb806cd20 2024-09-19T07:54:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.776+0000 7fddbd5d8640 1 --2- 172.21.15.16:0/1911503023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb806cd20 0x7fddb813fbf0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fdda8007e40 tx=0x7fdda800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.776+0000 7fddb5ffb640 1 -- 172.21.15.16:0/1911503023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8014620 con 0x7fddb806cd20 2024-09-19T07:54:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.776+0000 7fddb5ffb640 1 -- 172.21.15.16:0/1911503023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda8019070 con 0x7fddb806cd20 2024-09-19T07:54:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.776+0000 7fddbf83c640 1 -- 172.21.15.16:0/1911503023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb815a7b0 con 0x7fddb806cd20 2024-09-19T07:54:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.776+0000 7fddb5ffb640 1 -- 172.21.15.16:0/1911503023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8014a00 con 0x7fddb806cd20 2024-09-19T07:54:59.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.776+0000 7fddbf83c640 1 -- 172.21.15.16:0/1911503023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb815ad70 con 0x7fddb806cd20 2024-09-19T07:54:59.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.777+0000 7fddbf83c640 1 -- 172.21.15.16:0/1911503023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb806d100 con 0x7fddb806cd20 2024-09-19T07:54:59.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.778+0000 7fddb5ffb640 1 -- 172.21.15.16:0/1911503023 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdda8014ba0 con 0x7fddb806cd20 2024-09-19T07:54:59.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.778+0000 7fddb5ffb640 1 --2- 172.21.15.16:0/1911503023 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdda406c2b0 0x7fdda406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.778+0000 7fddb5ffb640 1 -- 172.21.15.16:0/1911503023 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5249+0+0 (secure 0 0 0) 0x7fdda808b9c0 con 0x7fddb806cd20 2024-09-19T07:54:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.778+0000 7fddbcdd7640 1 --2- 172.21.15.16:0/1911503023 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdda406c2b0 0x7fdda406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.779+0000 7fddbcdd7640 1 --2- 172.21.15.16:0/1911503023 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdda406c2b0 0x7fdda406e770 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fddac034000 tx=0x7fddac02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.782+0000 7fddb5ffb640 1 -- 172.21.15.16:0/1911503023 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fdda808de20 con 0x7fddb806cd20 2024-09-19T07:54:59.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:54:59.962+0000 7fddbf83c640 1 -- 172.21.15.16:0/1911503023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fddb80690c0 con 0x7fddb806cd20 2024-09-19T07:55:00.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:00.958+0000 7fddb5ffb640 1 -- 172.21.15.16:0/1911503023 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v137) ==== 138+0+0 (secure 0 0 0) 0x7fdda8059ca0 con 0x7fddb806cd20 2024-09-19T07:55:00.959 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-19T07:55:00.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:00.962+0000 7fddbf83c640 1 -- 172.21.15.16:0/1911503023 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdda406c2b0 msgr2=0x7fdda406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:00.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:00.962+0000 7fddbf83c640 1 --2- 172.21.15.16:0/1911503023 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdda406c2b0 0x7fdda406e770 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fddac034000 tx=0x7fddac02e000 comp rx=0 tx=0).stop 2024-09-19T07:55:00.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:00.962+0000 7fddbf83c640 1 -- 172.21.15.16:0/1911503023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb806cd20 msgr2=0x7fddb813fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:00.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:00.962+0000 7fddbf83c640 1 --2- 172.21.15.16:0/1911503023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb806cd20 0x7fddb813fbf0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fdda8007e40 tx=0x7fdda800e4b0 comp rx=0 tx=0).stop 2024-09-19T07:55:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:00.962+0000 7fddbf83c640 1 -- 172.21.15.16:0/1911503023 shutdown_connections 2024-09-19T07:55:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:00.962+0000 7fddbf83c640 1 --2- 172.21.15.16:0/1911503023 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdda406c2b0 0x7fdda406e770 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:00.962+0000 7fddbf83c640 1 --2- 172.21.15.16:0/1911503023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddb8161eb0 0x7fddb8140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:00.962+0000 7fddbf83c640 1 --2- 172.21.15.16:0/1911503023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddb8143a20 0x7fddb8140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:00.962+0000 7fddbf83c640 1 --2- 172.21.15.16:0/1911503023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb806cd20 0x7fddb813fbf0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:00.962+0000 7fddbf83c640 1 -- 172.21.15.16:0/1911503023 >> 172.21.15.16:0/1911503023 conn(0x7fddb80b3ed0 msgr2=0x7fddb8143040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:00.962+0000 7fddbf83c640 1 -- 172.21.15.16:0/1911503023 shutdown_connections 2024-09-19T07:55:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:00.962+0000 7fddbf83c640 1 -- 172.21.15.16:0/1911503023 wait complete. 2024-09-19T07:55:00.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T07:55:01.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.095+0000 7f2c3ffe4640 1 Processor -- start 2024-09-19T07:55:01.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.095+0000 7f2c3ffe4640 1 -- start start 2024-09-19T07:55:01.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.096+0000 7f2c3ffe4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c3815e0a0 0x7f2c38162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:01.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.096+0000 7f2c3ffe4640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2c380d43e0 0x7f2c380d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:01.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.096+0000 7f2c3ffe4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c3815f250 con 0x7f2c380b4b70 2024-09-19T07:55:01.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.096+0000 7f2c3ffe4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2c380d4d60 con 0x7f2c3815e0a0 2024-09-19T07:55:01.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.096+0000 7f2c3ffe4640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2c3815f530 con 0x7f2c380d43e0 2024-09-19T07:55:01.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.096+0000 7f2c3dd80640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c3815e0a0 0x7f2c38162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:01.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.096+0000 7f2c3dd80640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c3815e0a0 0x7f2c38162a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51232/0 (socket says 172.21.15.16:51232) 2024-09-19T07:55:01.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.096+0000 7f2c3dd80640 1 -- 172.21.15.16:0/86230482 learned_addr learned my addr 172.21.15.16:0/86230482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:01.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.096+0000 7f2c3e581640 1 --1- 172.21.15.16:0/86230482 >> v1:172.21.15.16:6789/0 conn(0x7f2c380b4b70 0x7f2c380b5000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33852/0 (socket says 172.21.15.16:33852) 2024-09-19T07:55:01.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.097+0000 7f2c3d57f640 1 --2- 172.21.15.16:0/86230482 >> v2:172.21.15.146:3300/0 conn(0x7f2c380d43e0 0x7f2c380d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:01.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.097+0000 7f2c3dd80640 1 -- 172.21.15.16:0/86230482 >> v2:172.21.15.146:3300/0 conn(0x7f2c380d43e0 msgr2=0x7f2c380d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:01.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.097+0000 7f2c3dd80640 1 --2- 172.21.15.16:0/86230482 >> v2:172.21.15.146:3300/0 conn(0x7f2c380d43e0 0x7f2c380d4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:01.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.097+0000 7f2c3dd80640 1 -- 172.21.15.16:0/86230482 >> v1:172.21.15.16:6789/0 conn(0x7f2c380b4b70 legacy=0x7f2c380b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:01.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.097+0000 7f2c3dd80640 1 -- 172.21.15.16:0/86230482 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3815f810 con 0x7f2c3815e0a0 2024-09-19T07:55:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.098+0000 7f2c3dd80640 1 --2- 172.21.15.16:0/86230482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3815e0a0 0x7f2c38162a70 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f2c380a22d0 tx=0x7f2c2802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a114d4d639063e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.098+0000 7f2c3cd7e640 1 -- 172.21.15.16:0/86230482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28036070 con 0x7f2c3815e0a0 2024-09-19T07:55:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.098+0000 7f2c3cd7e640 1 -- 172.21.15.16:0/86230482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c28031600 con 0x7f2c3815e0a0 2024-09-19T07:55:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.098+0000 7f2c3cd7e640 1 -- 172.21.15.16:0/86230482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c280319d0 con 0x7f2c3815e0a0 2024-09-19T07:55:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.098+0000 7f2c3ffe4640 1 -- 172.21.15.16:0/86230482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3815e0a0 msgr2=0x7f2c38162a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.098+0000 7f2c3ffe4640 1 --2- 172.21.15.16:0/86230482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3815e0a0 0x7f2c38162a70 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f2c380a22d0 tx=0x7f2c2802a730 comp rx=0 tx=0).stop 2024-09-19T07:55:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.098+0000 7f2c3ffe4640 1 -- 172.21.15.16:0/86230482 shutdown_connections 2024-09-19T07:55:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.098+0000 7f2c3ffe4640 1 --2- 172.21.15.16:0/86230482 >> v2:172.21.15.146:3300/0 conn(0x7f2c380d43e0 0x7f2c380d4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.098+0000 7f2c3ffe4640 1 --2- 172.21.15.16:0/86230482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3815e0a0 0x7f2c38162a70 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.098+0000 7f2c3ffe4640 1 -- 172.21.15.16:0/86230482 >> 172.21.15.16:0/86230482 conn(0x7f2c3809d770 msgr2=0x7f2c3808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.098+0000 7f2c3ffe4640 1 -- 172.21.15.16:0/86230482 shutdown_connections 2024-09-19T07:55:01.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.098+0000 7f2c3ffe4640 1 -- 172.21.15.16:0/86230482 wait complete. 2024-09-19T07:55:01.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.100+0000 7f2c3ffe4640 1 Processor -- start 2024-09-19T07:55:01.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.100+0000 7f2c3ffe4640 1 -- start start 2024-09-19T07:55:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.100+0000 7f2c3ffe4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c380b4b70 0x7f2c380b9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.100+0000 7f2c3ffe4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c380d43e0 0x7f2c38159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.100+0000 7f2c3ffe4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3815e0a0 0x7f2c380d1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.100+0000 7f2c3ffe4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3815e600 con 0x7f2c3815e0a0 2024-09-19T07:55:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.100+0000 7f2c3ffe4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3815e480 con 0x7f2c380b4b70 2024-09-19T07:55:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.100+0000 7f2c3ffe4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3815e780 con 0x7f2c380d43e0 2024-09-19T07:55:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.100+0000 7f2c3e581640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3815e0a0 0x7f2c380d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.100+0000 7f2c3e581640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3815e0a0 0x7f2c380d1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51238/0 (socket says 172.21.15.16:51238) 2024-09-19T07:55:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.100+0000 7f2c3e581640 1 -- 172.21.15.16:0/1290661475 learned_addr learned my addr 172.21.15.16:0/1290661475 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.101+0000 7f2c3d57f640 1 --2- 172.21.15.16:0/1290661475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c380d43e0 0x7f2c38159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.101+0000 7f2c3dd80640 1 --2- 172.21.15.16:0/1290661475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c380b4b70 0x7f2c380b9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.101+0000 7f2c3e581640 1 -- 172.21.15.16:0/1290661475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c380d43e0 msgr2=0x7f2c38159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.101+0000 7f2c3e581640 1 --2- 172.21.15.16:0/1290661475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c380d43e0 0x7f2c38159460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.101+0000 7f2c3e581640 1 -- 172.21.15.16:0/1290661475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c380b4b70 msgr2=0x7f2c380b9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.101+0000 7f2c3e581640 1 --2- 172.21.15.16:0/1290661475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c380b4b70 0x7f2c380b9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.101+0000 7f2c3e581640 1 -- 172.21.15.16:0/1290661475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c381599a0 con 0x7f2c3815e0a0 2024-09-19T07:55:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.101+0000 7f2c3e581640 1 --2- 172.21.15.16:0/1290661475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3815e0a0 0x7f2c380d1040 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000c400 tx=0x7f2c3000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.101+0000 7f2c367fc640 1 -- 172.21.15.16:0/1290661475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c300074e0 con 0x7f2c3815e0a0 2024-09-19T07:55:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.101+0000 7f2c367fc640 1 -- 172.21.15.16:0/1290661475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c30007680 con 0x7f2c3815e0a0 2024-09-19T07:55:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.101+0000 7f2c3ffe4640 1 -- 172.21.15.16:0/1290661475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c380d0280 con 0x7f2c3815e0a0 2024-09-19T07:55:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.101+0000 7f2c367fc640 1 -- 172.21.15.16:0/1290661475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c300049e0 con 0x7f2c3815e0a0 2024-09-19T07:55:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.101+0000 7f2c3ffe4640 1 -- 172.21.15.16:0/1290661475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c380d0840 con 0x7f2c3815e0a0 2024-09-19T07:55:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.102+0000 7f2c3ffe4640 1 -- 172.21.15.16:0/1290661475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c380d4820 con 0x7f2c3815e0a0 2024-09-19T07:55:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.103+0000 7f2c367fc640 1 -- 172.21.15.16:0/1290661475 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2c3001a020 con 0x7f2c3815e0a0 2024-09-19T07:55:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.103+0000 7f2c367fc640 1 --2- 172.21.15.16:0/1290661475 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2c2406c2b0 0x7f2c2406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.104+0000 7f2c367fc640 1 -- 172.21.15.16:0/1290661475 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5619+0+0 (secure 0 0 0) 0x7f2c3008ab60 con 0x7f2c3815e0a0 2024-09-19T07:55:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.104+0000 7f2c3dd80640 1 --2- 172.21.15.16:0/1290661475 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2c2406c2b0 0x7f2c2406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.104+0000 7f2c3dd80640 1 --2- 172.21.15.16:0/1290661475 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2c2406c2b0 0x7f2c2406e770 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2c28033000 tx=0x7f2c28009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.108+0000 7f2c367fc640 1 -- 172.21.15.16:0/1290661475 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2c300593b0 con 0x7f2c3815e0a0 2024-09-19T07:55:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.284+0000 7f2c3ffe4640 1 -- 172.21.15.16:0/1290661475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2c380ba3d0 con 0x7f2c3815e0a0 2024-09-19T07:55:01.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.964+0000 7f2c367fc640 1 -- 172.21.15.16:0/1290661475 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v138) ==== 170+0+0 (secure 0 0 0) 0x7f2c30058cb0 con 0x7f2c3815e0a0 2024-09-19T07:55:01.965 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-19T07:55:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.968+0000 7f2c3ffe4640 1 -- 172.21.15.16:0/1290661475 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2c2406c2b0 msgr2=0x7f2c2406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.968+0000 7f2c3ffe4640 1 --2- 172.21.15.16:0/1290661475 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2c2406c2b0 0x7f2c2406e770 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2c28033000 tx=0x7f2c28009000 comp rx=0 tx=0).stop 2024-09-19T07:55:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.968+0000 7f2c3ffe4640 1 -- 172.21.15.16:0/1290661475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3815e0a0 msgr2=0x7f2c380d1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.968+0000 7f2c3ffe4640 1 --2- 172.21.15.16:0/1290661475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3815e0a0 0x7f2c380d1040 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000c400 tx=0x7f2c3000c8c0 comp rx=0 tx=0).stop 2024-09-19T07:55:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.968+0000 7f2c3ffe4640 1 -- 172.21.15.16:0/1290661475 shutdown_connections 2024-09-19T07:55:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.968+0000 7f2c3ffe4640 1 --2- 172.21.15.16:0/1290661475 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2c2406c2b0 0x7f2c2406e770 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.968+0000 7f2c3ffe4640 1 --2- 172.21.15.16:0/1290661475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3815e0a0 0x7f2c380d1040 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.968+0000 7f2c3ffe4640 1 --2- 172.21.15.16:0/1290661475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c380d43e0 0x7f2c38159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.968+0000 7f2c3ffe4640 1 --2- 172.21.15.16:0/1290661475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c380b4b70 0x7f2c380b9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.968+0000 7f2c3ffe4640 1 -- 172.21.15.16:0/1290661475 >> 172.21.15.16:0/1290661475 conn(0x7f2c3809d770 msgr2=0x7f2c381600b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.968+0000 7f2c3ffe4640 1 -- 172.21.15.16:0/1290661475 shutdown_connections 2024-09-19T07:55:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:01.969+0000 7f2c3ffe4640 1 -- 172.21.15.16:0/1290661475 wait complete. 2024-09-19T07:55:01.995 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T07:55:02.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.101+0000 7faf14077640 1 Processor -- start 2024-09-19T07:55:02.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.101+0000 7faf14077640 1 -- start start 2024-09-19T07:55:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.102+0000 7faf14077640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf0c15e2a0 0x7faf0c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.102+0000 7faf14077640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf0c06d960 0x7faf0c0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.102+0000 7faf14077640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf0c0d1880 con 0x7faf0c15e680 2024-09-19T07:55:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.102+0000 7faf14077640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7faf0c0d1b60 con 0x7faf0c06d960 2024-09-19T07:55:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.102+0000 7faf14077640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faf0c0cf2d0 con 0x7faf0c15e2a0 2024-09-19T07:55:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.102+0000 7faf12614640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf0c06d960 0x7faf0c0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.102+0000 7faf12614640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf0c06d960 0x7faf0c0d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51248/0 (socket says 172.21.15.16:51248) 2024-09-19T07:55:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.102+0000 7faf12614640 1 -- 172.21.15.16:0/3328799274 learned_addr learned my addr 172.21.15.16:0/3328799274 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.102+0000 7faf11e13640 1 --2- 172.21.15.16:0/3328799274 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf0c15e2a0 0x7faf0c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.102+0000 7faf12614640 1 -- 172.21.15.16:0/3328799274 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf0c15e2a0 msgr2=0x7faf0c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.102+0000 7faf12614640 1 --2- 172.21.15.16:0/3328799274 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf0c15e2a0 0x7faf0c06d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.102+0000 7faf12614640 1 -- 172.21.15.16:0/3328799274 >> v1:172.21.15.146:6789/0 conn(0x7faf0c15e680 legacy=0x7faf0c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:02.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.102+0000 7faf12614640 1 -- 172.21.15.16:0/3328799274 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0c0cf5b0 con 0x7faf0c06d960 2024-09-19T07:55:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.103+0000 7faf12614640 1 --2- 172.21.15.16:0/3328799274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf0c06d960 0x7faf0c0d4370 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7faf04009e50 tx=0x7faf0402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7c6eac669af75f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.103+0000 7faf10e11640 1 -- 172.21.15.16:0/3328799274 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04036070 con 0x7faf0c06d960 2024-09-19T07:55:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.103+0000 7faf10e11640 1 -- 172.21.15.16:0/3328799274 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf04031600 con 0x7faf0c06d960 2024-09-19T07:55:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.103+0000 7faf10e11640 1 -- 172.21.15.16:0/3328799274 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf040319d0 con 0x7faf0c06d960 2024-09-19T07:55:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.104+0000 7faf14077640 1 -- 172.21.15.16:0/3328799274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf0c06d960 msgr2=0x7faf0c0d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.104+0000 7faf14077640 1 --2- 172.21.15.16:0/3328799274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf0c06d960 0x7faf0c0d4370 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7faf04009e50 tx=0x7faf0402a710 comp rx=0 tx=0).stop 2024-09-19T07:55:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.104+0000 7faf14077640 1 -- 172.21.15.16:0/3328799274 shutdown_connections 2024-09-19T07:55:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.104+0000 7faf14077640 1 --2- 172.21.15.16:0/3328799274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf0c06d960 0x7faf0c0d4370 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:02.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.104+0000 7faf14077640 1 --2- 172.21.15.16:0/3328799274 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf0c15e2a0 0x7faf0c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:02.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.104+0000 7faf14077640 1 -- 172.21.15.16:0/3328799274 >> 172.21.15.16:0/3328799274 conn(0x7faf0c09d760 msgr2=0x7faf0c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:02.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.104+0000 7faf14077640 1 -- 172.21.15.16:0/3328799274 shutdown_connections 2024-09-19T07:55:02.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.104+0000 7faf14077640 1 -- 172.21.15.16:0/3328799274 wait complete. 2024-09-19T07:55:02.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.105+0000 7faf14077640 1 Processor -- start 2024-09-19T07:55:02.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.105+0000 7faf14077640 1 -- start start 2024-09-19T07:55:02.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.106+0000 7faf14077640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf0c06d960 0x7faf0c0cd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:02.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.106+0000 7faf14077640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf0c15e2a0 0x7faf0c0cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:02.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.106+0000 7faf14077640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf0c15e680 0x7faf0c0cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:02.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.106+0000 7faf14077640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c0d0360 con 0x7faf0c06d960 2024-09-19T07:55:02.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.106+0000 7faf14077640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c0d01e0 con 0x7faf0c15e680 2024-09-19T07:55:02.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.106+0000 7faf14077640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faf0c0d48b0 con 0x7faf0c15e2a0 2024-09-19T07:55:02.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.106+0000 7faf11e13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf0c06d960 0x7faf0c0cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:02.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.106+0000 7faf11e13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf0c06d960 0x7faf0c0cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51254/0 (socket says 172.21.15.16:51254) 2024-09-19T07:55:02.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.106+0000 7faf11e13640 1 -- 172.21.15.16:0/805066244 learned_addr learned my addr 172.21.15.16:0/805066244 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:02.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.106+0000 7faf11612640 1 --2- 172.21.15.16:0/805066244 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf0c15e2a0 0x7faf0c0cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:02.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.106+0000 7faf12614640 1 --2- 172.21.15.16:0/805066244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf0c15e680 0x7faf0c0cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:02.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.106+0000 7faf11e13640 1 -- 172.21.15.16:0/805066244 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf0c15e2a0 msgr2=0x7faf0c0cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:02.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.106+0000 7faf11e13640 1 --2- 172.21.15.16:0/805066244 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf0c15e2a0 0x7faf0c0cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:02.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.106+0000 7faf11e13640 1 -- 172.21.15.16:0/805066244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf0c15e680 msgr2=0x7faf0c0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:02.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.106+0000 7faf11e13640 1 --2- 172.21.15.16:0/805066244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf0c15e680 0x7faf0c0cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:02.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.106+0000 7faf11e13640 1 -- 172.21.15.16:0/805066244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0c0cd220 con 0x7faf0c06d960 2024-09-19T07:55:02.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.106+0000 7faf11e13640 1 --2- 172.21.15.16:0/805066244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf0c06d960 0x7faf0c0cd470 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7faf0800e9b0 tx=0x7faf0800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:02.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.106+0000 7faf027fc640 1 -- 172.21.15.16:0/805066244 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08017070 con 0x7faf0c06d960 2024-09-19T07:55:02.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.106+0000 7faf027fc640 1 -- 172.21.15.16:0/805066244 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0800c9a0 con 0x7faf0c06d960 2024-09-19T07:55:02.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.107+0000 7faf14077640 1 -- 172.21.15.16:0/805066244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf0c0c9a10 con 0x7faf0c06d960 2024-09-19T07:55:02.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.107+0000 7faf027fc640 1 -- 172.21.15.16:0/805066244 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0800cd90 con 0x7faf0c06d960 2024-09-19T07:55:02.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.107+0000 7faf14077640 1 -- 172.21.15.16:0/805066244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf0c0c9fd0 con 0x7faf0c06d960 2024-09-19T07:55:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.108+0000 7faf14077640 1 -- 172.21.15.16:0/805066244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf0c0ca400 con 0x7faf0c06d960 2024-09-19T07:55:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.108+0000 7faf027fc640 1 -- 172.21.15.16:0/805066244 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7faf0801a070 con 0x7faf0c06d960 2024-09-19T07:55:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.109+0000 7faf027fc640 1 --2- 172.21.15.16:0/805066244 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faef006c2e0 0x7faef006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.109+0000 7faf027fc640 1 -- 172.21.15.16:0/805066244 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 6001+0+0 (secure 0 0 0) 0x7faf0808b3c0 con 0x7faf0c06d960 2024-09-19T07:55:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.109+0000 7faf11612640 1 --2- 172.21.15.16:0/805066244 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faef006c2e0 0x7faef006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.109+0000 7faf11612640 1 --2- 172.21.15.16:0/805066244 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faef006c2e0 0x7faef006e7a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7faf0c160220 tx=0x7faefc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:02.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.113+0000 7faf027fc640 1 -- 172.21.15.16:0/805066244 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7faf08059440 con 0x7faf0c06d960 2024-09-19T07:55:02.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.323+0000 7faf14077640 1 -- 172.21.15.16:0/805066244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7faf0c0d4370 con 0x7faf0c06d960 2024-09-19T07:55:02.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.975+0000 7faf027fc640 1 -- 172.21.15.16:0/805066244 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v79) ==== 174+0+0 (secure 0 0 0) 0x7faf0805d740 con 0x7faf0c06d960 2024-09-19T07:55:02.976 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 21 and data pool 22 2024-09-19T07:55:02.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.994+0000 7faf14077640 1 -- 172.21.15.16:0/805066244 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faef006c2e0 msgr2=0x7faef006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:02.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.994+0000 7faf14077640 1 --2- 172.21.15.16:0/805066244 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faef006c2e0 0x7faef006e7a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7faf0c160220 tx=0x7faefc009040 comp rx=0 tx=0).stop 2024-09-19T07:55:02.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.994+0000 7faf14077640 1 -- 172.21.15.16:0/805066244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf0c06d960 msgr2=0x7faf0c0cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:02.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.994+0000 7faf14077640 1 --2- 172.21.15.16:0/805066244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf0c06d960 0x7faf0c0cd470 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7faf0800e9b0 tx=0x7faf0800ee70 comp rx=0 tx=0).stop 2024-09-19T07:55:02.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.996+0000 7faf14077640 1 -- 172.21.15.16:0/805066244 shutdown_connections 2024-09-19T07:55:02.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.996+0000 7faf14077640 1 --2- 172.21.15.16:0/805066244 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7faef006c2e0 0x7faef006e7a0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:02.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.996+0000 7faf14077640 1 --2- 172.21.15.16:0/805066244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf0c15e680 0x7faf0c0cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:02.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.996+0000 7faf14077640 1 --2- 172.21.15.16:0/805066244 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf0c15e2a0 0x7faf0c0cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:02.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.996+0000 7faf14077640 1 --2- 172.21.15.16:0/805066244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf0c06d960 0x7faf0c0cd470 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:02.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.996+0000 7faf14077640 1 -- 172.21.15.16:0/805066244 >> 172.21.15.16:0/805066244 conn(0x7faf0c09d760 msgr2=0x7faf0c15f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:02.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.998+0000 7faf14077640 1 -- 172.21.15.16:0/805066244 shutdown_connections 2024-09-19T07:55:02.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:02.998+0000 7faf14077640 1 -- 172.21.15.16:0/805066244 wait complete. 2024-09-19T07:55:03.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:55:03.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.119+0000 7f534ce5e640 1 Processor -- start 2024-09-19T07:55:03.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.119+0000 7f534ce5e640 1 -- start start 2024-09-19T07:55:03.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f534ce5e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f53480d0bc0 0x7f53480cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f534ce5e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53480cdee0 con 0x7f534815c2c0 2024-09-19T07:55:03.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f534ce5e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53480ce1c0 con 0x7f534815bee0 2024-09-19T07:55:03.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f534ce5e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f53480ceaf0 con 0x7f53480d0bc0 2024-09-19T07:55:03.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f5346d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f534815c2c0 0x7f53480cf740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33974/0 (socket says 172.21.15.16:33974) 2024-09-19T07:55:03.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f5346d9d640 1 -- 172.21.15.16:0/3946560368 learned_addr learned my addr 172.21.15.16:0/3946560368 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:03.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f533dd9b640 1 --2- 172.21.15.16:0/3946560368 >> v2:172.21.15.146:3300/0 conn(0x7f53480d0bc0 0x7f53480cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f5345d9b640 1 -- 172.21.15.16:0/3946560368 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2111174844 0 0) 0x7f53480cdee0 con 0x7f534815c2c0 2024-09-19T07:55:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f5345d9b640 1 -- 172.21.15.16:0/3946560368 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5334003520 con 0x7f534815c2c0 2024-09-19T07:55:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f5345d9b640 1 -- 172.21.15.16:0/3946560368 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3453942154 0 0) 0x7f5334003520 con 0x7f534815c2c0 2024-09-19T07:55:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f5345d9b640 1 -- 172.21.15.16:0/3946560368 >> v2:172.21.15.146:3300/0 conn(0x7f53480d0bc0 msgr2=0x7f53480cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f5345d9b640 1 --2- 172.21.15.16:0/3946560368 >> v2:172.21.15.146:3300/0 conn(0x7f53480d0bc0 0x7f53480cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f5345d9b640 1 -- 172.21.15.16:0/3946560368 >> v1:172.21.15.146:6789/0 conn(0x7f534815bee0 legacy=0x7f53480d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f5345d9b640 1 -- 172.21.15.16:0/3946560368 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53480ce4a0 con 0x7f534815c2c0 2024-09-19T07:55:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f5345d9b640 1 -- 172.21.15.16:0/3946560368 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f53400019f0 con 0x7f534815c2c0 2024-09-19T07:55:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f5345d9b640 1 -- 172.21.15.16:0/3946560368 >> v1:172.21.15.16:6789/0 conn(0x7f534815c2c0 legacy=0x7f53480cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f5345d9b640 1 --2- 172.21.15.16:0/3946560368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53340039b0 0x7f5334003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f5345d9b640 1 --2- 172.21.15.16:0/3946560368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5334004800 0x7f5334004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f5345d9b640 1 --2- 172.21.15.16:0/3946560368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f53340051c0 0x7f533400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f5345d9b640 1 -- 172.21.15.16:0/3946560368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f53480ceaf0 con 0x7f5334004800 2024-09-19T07:55:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f5345d9b640 1 -- 172.21.15.16:0/3946560368 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f533400c060 con 0x7f53340039b0 2024-09-19T07:55:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.120+0000 7f5345d9b640 1 -- 172.21.15.16:0/3946560368 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f533400c260 con 0x7f53340051c0 2024-09-19T07:55:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.121+0000 7f533dd9b640 1 --2- 172.21.15.16:0/3946560368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5334004800 0x7f5334004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.121+0000 7f534659c640 1 --2- 172.21.15.16:0/3946560368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53340039b0 0x7f5334003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.121+0000 7f5346d9d640 1 --2- 172.21.15.16:0/3946560368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f53340051c0 0x7f533400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.121+0000 7f533dd9b640 1 -- 172.21.15.16:0/3946560368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f53340051c0 msgr2=0x7f533400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.121+0000 7f533dd9b640 1 --2- 172.21.15.16:0/3946560368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f53340051c0 0x7f533400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.121+0000 7f533dd9b640 1 -- 172.21.15.16:0/3946560368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53340039b0 msgr2=0x7f5334003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.121+0000 7f533dd9b640 1 --2- 172.21.15.16:0/3946560368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53340039b0 0x7f5334003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.121+0000 7f533dd9b640 1 -- 172.21.15.16:0/3946560368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5334003520 con 0x7f5334004800 2024-09-19T07:55:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.121+0000 7f5346d9d640 1 -- 172.21.15.16:0/3946560368 reap_dead start 2024-09-19T07:55:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.122+0000 7f533dd9b640 1 --2- 172.21.15.16:0/3946560368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5334004800 0x7f5334004c80 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f5330009ac0 tx=0x7f5334005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e214ee6c1011e942 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.122+0000 7f5345d9b640 1 -- 172.21.15.16:0/3946560368 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330036070 con 0x7f5334004800 2024-09-19T07:55:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.122+0000 7f5345d9b640 1 -- 172.21.15.16:0/3946560368 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5330031400 con 0x7f5334004800 2024-09-19T07:55:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.122+0000 7f5345d9b640 1 -- 172.21.15.16:0/3946560368 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330009690 con 0x7f5334004800 2024-09-19T07:55:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.122+0000 7f534ce5e640 1 -- 172.21.15.16:0/3946560368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5334004800 msgr2=0x7f5334004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.122+0000 7f534ce5e640 1 --2- 172.21.15.16:0/3946560368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5334004800 0x7f5334004c80 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f5330009ac0 tx=0x7f5334005830 comp rx=0 tx=0).stop 2024-09-19T07:55:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.122+0000 7f534ce5e640 1 -- 172.21.15.16:0/3946560368 shutdown_connections 2024-09-19T07:55:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.122+0000 7f534ce5e640 1 --2- 172.21.15.16:0/3946560368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5334004800 0x7f5334004c80 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.122+0000 7f534ce5e640 1 -- 172.21.15.16:0/3946560368 >> 172.21.15.16:0/3946560368 conn(0x7f534806bdf0 msgr2=0x7f534806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.122+0000 7f534ce5e640 1 -- 172.21.15.16:0/3946560368 shutdown_connections 2024-09-19T07:55:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.122+0000 7f534ce5e640 1 -- 172.21.15.16:0/3946560368 wait complete. 2024-09-19T07:55:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.124+0000 7f534ce5e640 1 Processor -- start 2024-09-19T07:55:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.124+0000 7f534ce5e640 1 -- start start 2024-09-19T07:55:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f534ce5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5334004800 0x7f5348151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f534ce5e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f53480c8310 0x7f53480c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f534ce5e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53480ad160 0x7f53480ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f534ce5e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f53480d0630 con 0x7f5334004800 2024-09-19T07:55:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f534ce5e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f53480d04b0 con 0x7f53480ad160 2024-09-19T07:55:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f534ce5e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f53480d07b0 con 0x7f53480c8310 2024-09-19T07:55:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f534659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5334004800 0x7f5348151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f534659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5334004800 0x7f5348151b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43632/0 (socket says 172.21.15.16:43632) 2024-09-19T07:55:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f534659c640 1 -- 172.21.15.16:0/2867113467 learned_addr learned my addr 172.21.15.16:0/2867113467 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f5346d9d640 1 --2- 172.21.15.16:0/2867113467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53480ad160 0x7f53480ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f533dd9b640 1 --2- 172.21.15.16:0/2867113467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f53480c8310 0x7f53480c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f534659c640 1 -- 172.21.15.16:0/2867113467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f53480c8310 msgr2=0x7f53480c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f534659c640 1 --2- 172.21.15.16:0/2867113467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f53480c8310 0x7f53480c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f534659c640 1 -- 172.21.15.16:0/2867113467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53480ad160 msgr2=0x7f53480ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f534659c640 1 --2- 172.21.15.16:0/2867113467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53480ad160 0x7f53480ad5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f534659c640 1 -- 172.21.15.16:0/2867113467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534815dab0 con 0x7f5334004800 2024-09-19T07:55:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f5346d9d640 1 --2- 172.21.15.16:0/2867113467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53480ad160 0x7f53480ad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f533dd9b640 1 --2- 172.21.15.16:0/2867113467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f53480c8310 0x7f53480c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f534659c640 1 --2- 172.21.15.16:0/2867113467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5334004800 0x7f5348151b40 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f533800f5f0 tx=0x7f533800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f533f7fe640 1 -- 172.21.15.16:0/2867113467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53380052e0 con 0x7f5334004800 2024-09-19T07:55:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f533f7fe640 1 -- 172.21.15.16:0/2867113467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5338018070 con 0x7f5334004800 2024-09-19T07:55:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f533f7fe640 1 -- 172.21.15.16:0/2867113467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338003810 con 0x7f5334004800 2024-09-19T07:55:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.125+0000 7f534ce5e640 1 -- 172.21.15.16:0/2867113467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f534815dd70 con 0x7f5334004800 2024-09-19T07:55:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.126+0000 7f534ce5e640 1 -- 172.21.15.16:0/2867113467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f534815e330 con 0x7f5334004800 2024-09-19T07:55:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.127+0000 7f534ce5e640 1 -- 172.21.15.16:0/2867113467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53480cbb10 con 0x7f5334004800 2024-09-19T07:55:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.127+0000 7f533f7fe640 1 -- 172.21.15.16:0/2867113467 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f53380039b0 con 0x7f5334004800 2024-09-19T07:55:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.127+0000 7f533f7fe640 1 --2- 172.21.15.16:0/2867113467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f532806c2b0 0x7f532806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.128+0000 7f533f7fe640 1 -- 172.21.15.16:0/2867113467 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 6101+0+0 (secure 0 0 0) 0x7f533808b2a0 con 0x7f5334004800 2024-09-19T07:55:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.128+0000 7f533dd9b640 1 --2- 172.21.15.16:0/2867113467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f532806c2b0 0x7f532806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.128+0000 7f533dd9b640 1 --2- 172.21.15.16:0/2867113467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f532806c2b0 0x7f532806e770 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5330033000 tx=0x7f5330008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.132+0000 7f533f7fe640 1 -- 172.21.15.16:0/2867113467 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5338059910 con 0x7f5334004800 2024-09-19T07:55:03.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.308+0000 7f534ce5e640 1 -- 172.21.15.16:0/2867113467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53480cf180 con 0x7f5334004800 2024-09-19T07:55:03.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.309+0000 7f533f7fe640 1 -- 172.21.15.16:0/2867113467 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) ==== 74+0+16989 (secure 0 0 0) 0x7f5338059210 con 0x7f5334004800 2024-09-19T07:55:03.310 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:03.310 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":139,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:55:02.324943+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:54:59.994353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-09-19T07:55:01.315629+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:55:03.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.311+0000 7f534ce5e640 1 -- 172.21.15.16:0/2867113467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f532806c2b0 msgr2=0x7f532806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:03.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.311+0000 7f534ce5e640 1 --2- 172.21.15.16:0/2867113467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f532806c2b0 0x7f532806e770 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5330033000 tx=0x7f5330008030 comp rx=0 tx=0).stop 2024-09-19T07:55:03.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.311+0000 7f534ce5e640 1 -- 172.21.15.16:0/2867113467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5334004800 msgr2=0x7f5348151b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:03.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.311+0000 7f534ce5e640 1 --2- 172.21.15.16:0/2867113467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5334004800 0x7f5348151b40 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f533800f5f0 tx=0x7f533800fab0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.311+0000 7f534ce5e640 1 -- 172.21.15.16:0/2867113467 shutdown_connections 2024-09-19T07:55:03.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.311+0000 7f534ce5e640 1 --2- 172.21.15.16:0/2867113467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f532806c2b0 0x7f532806e770 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.311+0000 7f534ce5e640 1 --2- 172.21.15.16:0/2867113467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53480ad160 0x7f53480ad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.311+0000 7f534ce5e640 1 --2- 172.21.15.16:0/2867113467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f53480c8310 0x7f53480c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.311+0000 7f534ce5e640 1 --2- 172.21.15.16:0/2867113467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5334004800 0x7f5348151b40 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.311+0000 7f534ce5e640 1 -- 172.21.15.16:0/2867113467 >> 172.21.15.16:0/2867113467 conn(0x7f534806bdf0 msgr2=0x7f53480b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:03.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.311+0000 7f534ce5e640 1 -- 172.21.15.16:0/2867113467 shutdown_connections 2024-09-19T07:55:03.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.311+0000 7f534ce5e640 1 -- 172.21.15.16:0/2867113467 wait complete. 2024-09-19T07:55:03.327 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:55:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.430+0000 7fa94cc65640 1 Processor -- start 2024-09-19T07:55:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.430+0000 7fa94cc65640 1 -- start start 2024-09-19T07:55:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.431+0000 7fa94cc65640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa94815c140 0x7fa948160650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.431+0000 7fa94cc65640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa9480b6650 0x7fa94815d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.431+0000 7fa94cc65640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa948160b90 con 0x7fa94806f9d0 2024-09-19T07:55:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.431+0000 7fa94cc65640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa94815c520 con 0x7fa94815c140 2024-09-19T07:55:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.431+0000 7fa94cc65640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa94815c7f0 con 0x7fa9480b6650 2024-09-19T07:55:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.431+0000 7fa94659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa94815c140 0x7fa948160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.431+0000 7fa94659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa94815c140 0x7fa948160650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43646/0 (socket says 172.21.15.16:43646) 2024-09-19T07:55:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.431+0000 7fa94659c640 1 -- 172.21.15.16:0/99796095 learned_addr learned my addr 172.21.15.16:0/99796095 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.431+0000 7fa946d9d640 1 --2- 172.21.15.16:0/99796095 >> v2:172.21.15.146:3300/0 conn(0x7fa9480b6650 0x7fa94815d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.431+0000 7fa94659c640 1 -- 172.21.15.16:0/99796095 >> v2:172.21.15.146:3300/0 conn(0x7fa9480b6650 msgr2=0x7fa94815d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.431+0000 7fa94659c640 1 --2- 172.21.15.16:0/99796095 >> v2:172.21.15.146:3300/0 conn(0x7fa9480b6650 0x7fa94815d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.431+0000 7fa94659c640 1 -- 172.21.15.16:0/99796095 >> v1:172.21.15.16:6789/0 conn(0x7fa94806f9d0 legacy=0x7fa9480b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.431+0000 7fa94659c640 1 -- 172.21.15.16:0/99796095 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94815cad0 con 0x7fa94815c140 2024-09-19T07:55:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.431+0000 7fa94559a640 1 -- 172.21.15.16:0/99796095 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 198308195 0 0) 0x7fa93c001110 con 0x7fa94806f9d0 2024-09-19T07:55:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.432+0000 7fa94659c640 1 --2- 172.21.15.16:0/99796095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa94815c140 0x7fa948160650 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fa9480b0420 tx=0x7fa93802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf933ca9b89d58d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.432+0000 7fa94559a640 1 -- 172.21.15.16:0/99796095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938037070 con 0x7fa94815c140 2024-09-19T07:55:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.432+0000 7fa94559a640 1 -- 172.21.15.16:0/99796095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa938032440 con 0x7fa94815c140 2024-09-19T07:55:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.432+0000 7fa94559a640 1 -- 172.21.15.16:0/99796095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938032810 con 0x7fa94815c140 2024-09-19T07:55:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.433+0000 7fa94cc65640 1 -- 172.21.15.16:0/99796095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa94815c140 msgr2=0x7fa948160650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.433+0000 7fa94cc65640 1 --2- 172.21.15.16:0/99796095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa94815c140 0x7fa948160650 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fa9480b0420 tx=0x7fa93802a270 comp rx=0 tx=0).stop 2024-09-19T07:55:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.433+0000 7fa94cc65640 1 -- 172.21.15.16:0/99796095 shutdown_connections 2024-09-19T07:55:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.433+0000 7fa94cc65640 1 --2- 172.21.15.16:0/99796095 >> v2:172.21.15.146:3300/0 conn(0x7fa9480b6650 0x7fa94815d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.433+0000 7fa94cc65640 1 --2- 172.21.15.16:0/99796095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa94815c140 0x7fa948160650 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.433+0000 7fa94cc65640 1 -- 172.21.15.16:0/99796095 >> 172.21.15.16:0/99796095 conn(0x7fa94806bc10 msgr2=0x7fa94806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.433+0000 7fa94cc65640 1 -- 172.21.15.16:0/99796095 shutdown_connections 2024-09-19T07:55:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.433+0000 7fa94cc65640 1 -- 172.21.15.16:0/99796095 wait complete. 2024-09-19T07:55:03.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.434+0000 7fa94cc65640 1 Processor -- start 2024-09-19T07:55:03.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.434+0000 7fa94cc65640 1 -- start start 2024-09-19T07:55:03.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.435+0000 7fa94cc65640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa94806f9d0 0x7fa94815a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.435+0000 7fa94cc65640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9480b6650 0x7fa9480b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.435+0000 7fa94cc65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa94815c140 0x7fa948153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.435+0000 7fa94cc65640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa94815d370 con 0x7fa94815c140 2024-09-19T07:55:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.435+0000 7fa94cc65640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa94815d090 con 0x7fa9480b6650 2024-09-19T07:55:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.435+0000 7fa94cc65640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa94815ddd0 con 0x7fa94806f9d0 2024-09-19T07:55:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.435+0000 7fa946d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa94815c140 0x7fa948153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.435+0000 7fa946d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa94815c140 0x7fa948153d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43650/0 (socket says 172.21.15.16:43650) 2024-09-19T07:55:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.435+0000 7fa946d9d640 1 -- 172.21.15.16:0/1537240189 learned_addr learned my addr 172.21.15.16:0/1537240189 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.435+0000 7fa945d9b640 1 --2- 172.21.15.16:0/1537240189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9480b6650 0x7fa9480b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.435+0000 7fa94659c640 1 --2- 172.21.15.16:0/1537240189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa94806f9d0 0x7fa94815a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.436+0000 7fa946d9d640 1 -- 172.21.15.16:0/1537240189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa94806f9d0 msgr2=0x7fa94815a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.436+0000 7fa946d9d640 1 --2- 172.21.15.16:0/1537240189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa94806f9d0 0x7fa94815a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.436+0000 7fa946d9d640 1 -- 172.21.15.16:0/1537240189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9480b6650 msgr2=0x7fa9480b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.436+0000 7fa946d9d640 1 --2- 172.21.15.16:0/1537240189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9480b6650 0x7fa9480b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.436+0000 7fa946d9d640 1 -- 172.21.15.16:0/1537240189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9480a96a0 con 0x7fa94815c140 2024-09-19T07:55:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.436+0000 7fa946d9d640 1 --2- 172.21.15.16:0/1537240189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa94815c140 0x7fa948153d90 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fa940009e30 tx=0x7fa94000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.436+0000 7fa936ffd640 1 -- 172.21.15.16:0/1537240189 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940018020 con 0x7fa94815c140 2024-09-19T07:55:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.436+0000 7fa936ffd640 1 -- 172.21.15.16:0/1537240189 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa940014480 con 0x7fa94815c140 2024-09-19T07:55:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.436+0000 7fa94cc65640 1 -- 172.21.15.16:0/1537240189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9480a9980 con 0x7fa94815c140 2024-09-19T07:55:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.436+0000 7fa936ffd640 1 -- 172.21.15.16:0/1537240189 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940014870 con 0x7fa94815c140 2024-09-19T07:55:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.436+0000 7fa94cc65640 1 -- 172.21.15.16:0/1537240189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9480a9f40 con 0x7fa94815c140 2024-09-19T07:55:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.437+0000 7fa94cc65640 1 -- 172.21.15.16:0/1537240189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa94815d890 con 0x7fa94815c140 2024-09-19T07:55:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.438+0000 7fa936ffd640 1 -- 172.21.15.16:0/1537240189 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa940005d10 con 0x7fa94815c140 2024-09-19T07:55:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.438+0000 7fa936ffd640 1 --2- 172.21.15.16:0/1537240189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa92806c2b0 0x7fa92806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.438+0000 7fa936ffd640 1 -- 172.21.15.16:0/1537240189 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 6101+0+0 (secure 0 0 0) 0x7fa94008c290 con 0x7fa94815c140 2024-09-19T07:55:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.438+0000 7fa94659c640 1 --2- 172.21.15.16:0/1537240189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa92806c2b0 0x7fa92806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.439+0000 7fa94659c640 1 --2- 172.21.15.16:0/1537240189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa92806c2b0 0x7fa92806e770 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa938034000 tx=0x7fa93802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.442+0000 7fa936ffd640 1 -- 172.21.15.16:0/1537240189 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa940022030 con 0x7fa94815c140 2024-09-19T07:55:03.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.619+0000 7fa94cc65640 1 -- 172.21.15.16:0/1537240189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9480aa110 con 0x7fa94815c140 2024-09-19T07:55:03.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.620+0000 7fa936ffd640 1 -- 172.21.15.16:0/1537240189 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) ==== 74+0+16989 (secure 0 0 0) 0x7fa940010040 con 0x7fa94815c140 2024-09-19T07:55:03.621 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:03.621 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":139,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:55:02.324943+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:54:59.994353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-09-19T07:55:01.315629+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:55:03.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.623+0000 7fa94cc65640 1 -- 172.21.15.16:0/1537240189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa92806c2b0 msgr2=0x7fa92806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:03.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.623+0000 7fa94cc65640 1 --2- 172.21.15.16:0/1537240189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa92806c2b0 0x7fa92806e770 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa938034000 tx=0x7fa93802e000 comp rx=0 tx=0).stop 2024-09-19T07:55:03.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.623+0000 7fa94cc65640 1 -- 172.21.15.16:0/1537240189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa94815c140 msgr2=0x7fa948153d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:03.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.623+0000 7fa94cc65640 1 --2- 172.21.15.16:0/1537240189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa94815c140 0x7fa948153d90 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fa940009e30 tx=0x7fa94000c490 comp rx=0 tx=0).stop 2024-09-19T07:55:03.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.623+0000 7fa94cc65640 1 -- 172.21.15.16:0/1537240189 shutdown_connections 2024-09-19T07:55:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.623+0000 7fa94cc65640 1 --2- 172.21.15.16:0/1537240189 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa92806c2b0 0x7fa92806e770 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.623+0000 7fa94cc65640 1 --2- 172.21.15.16:0/1537240189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa94815c140 0x7fa948153d90 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.623+0000 7fa94cc65640 1 --2- 172.21.15.16:0/1537240189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9480b6650 0x7fa9480b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.623+0000 7fa94cc65640 1 --2- 172.21.15.16:0/1537240189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa94806f9d0 0x7fa94815a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.623+0000 7fa94cc65640 1 -- 172.21.15.16:0/1537240189 >> 172.21.15.16:0/1537240189 conn(0x7fa94806bc10 msgr2=0x7fa94806cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.623+0000 7fa94cc65640 1 -- 172.21.15.16:0/1537240189 shutdown_connections 2024-09-19T07:55:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.623+0000 7fa94cc65640 1 -- 172.21.15.16:0/1537240189 wait complete. 2024-09-19T07:55:03.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T07:55:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.739+0000 7feeda4f1640 1 Processor -- start 2024-09-19T07:55:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.739+0000 7feeda4f1640 1 -- start start 2024-09-19T07:55:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.740+0000 7feeda4f1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feed4072250 0x7feed40cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.740+0000 7feeda4f1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feed40726c0 0x7feed40d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.740+0000 7feeda4f1640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed40d2b30 con 0x7feed40d2750 2024-09-19T07:55:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.740+0000 7feeda4f1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7feed4145bd0 con 0x7feed4072250 2024-09-19T07:55:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.740+0000 7feeda4f1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7feed41427e0 con 0x7feed40726c0 2024-09-19T07:55:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.740+0000 7feed37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feed4072250 0x7feed40cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.740+0000 7feed37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feed4072250 0x7feed40cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43654/0 (socket says 172.21.15.16:43654) 2024-09-19T07:55:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.740+0000 7feed37fe640 1 -- 172.21.15.16:0/2034108210 learned_addr learned my addr 172.21.15.16:0/2034108210 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.741+0000 7feed8a8e640 1 --2- 172.21.15.16:0/2034108210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feed40726c0 0x7feed40d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.741+0000 7feed37fe640 1 -- 172.21.15.16:0/2034108210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feed40726c0 msgr2=0x7feed40d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.741+0000 7feed37fe640 1 --2- 172.21.15.16:0/2034108210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feed40726c0 0x7feed40d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.741+0000 7feed37fe640 1 -- 172.21.15.16:0/2034108210 >> v1:172.21.15.146:6789/0 conn(0x7feed40d2750 legacy=0x7feed413f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.741+0000 7feed37fe640 1 -- 172.21.15.16:0/2034108210 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed4141c60 con 0x7feed4072250 2024-09-19T07:55:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.741+0000 7feed37fe640 1 --2- 172.21.15.16:0/2034108210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed4072250 0x7feed40cfdc0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7feec8009870 tx=0x7feec802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491acda5d8fc73ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.742+0000 7feed2ffd640 1 -- 172.21.15.16:0/2034108210 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8037070 con 0x7feed4072250 2024-09-19T07:55:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.742+0000 7feed2ffd640 1 -- 172.21.15.16:0/2034108210 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec8032440 con 0x7feed4072250 2024-09-19T07:55:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.742+0000 7feed2ffd640 1 -- 172.21.15.16:0/2034108210 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8032810 con 0x7feed4072250 2024-09-19T07:55:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.742+0000 7feeda4f1640 1 -- 172.21.15.16:0/2034108210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed4072250 msgr2=0x7feed40cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.742+0000 7feeda4f1640 1 --2- 172.21.15.16:0/2034108210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed4072250 0x7feed40cfdc0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7feec8009870 tx=0x7feec802a180 comp rx=0 tx=0).stop 2024-09-19T07:55:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.742+0000 7feeda4f1640 1 -- 172.21.15.16:0/2034108210 shutdown_connections 2024-09-19T07:55:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.742+0000 7feeda4f1640 1 --2- 172.21.15.16:0/2034108210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feed40726c0 0x7feed40d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.742+0000 7feeda4f1640 1 --2- 172.21.15.16:0/2034108210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed4072250 0x7feed40cfdc0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.742+0000 7feeda4f1640 1 -- 172.21.15.16:0/2034108210 >> 172.21.15.16:0/2034108210 conn(0x7feed40b4040 msgr2=0x7feed40b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:03.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.742+0000 7feeda4f1640 1 -- 172.21.15.16:0/2034108210 shutdown_connections 2024-09-19T07:55:03.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.742+0000 7feeda4f1640 1 -- 172.21.15.16:0/2034108210 wait complete. 2024-09-19T07:55:03.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.743+0000 7feeda4f1640 1 Processor -- start 2024-09-19T07:55:03.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.744+0000 7feeda4f1640 1 -- start start 2024-09-19T07:55:03.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.744+0000 7feeda4f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed4072250 0x7feed40b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.744+0000 7feeda4f1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feed40726c0 0x7feed41400f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.744+0000 7feeda4f1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feed40d2750 0x7feed4140630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.744+0000 7feeda4f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feed4142500 con 0x7feed4072250 2024-09-19T07:55:03.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.744+0000 7feeda4f1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7feed4142220 con 0x7feed40726c0 2024-09-19T07:55:03.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.744+0000 7feeda4f1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7feed41418b0 con 0x7feed40d2750 2024-09-19T07:55:03.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.745+0000 7feed3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed4072250 0x7feed40b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.745+0000 7feed3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed4072250 0x7feed40b10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43664/0 (socket says 172.21.15.16:43664) 2024-09-19T07:55:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.745+0000 7feed3fff640 1 -- 172.21.15.16:0/2235124290 learned_addr learned my addr 172.21.15.16:0/2235124290 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.745+0000 7feed8a8e640 1 --2- 172.21.15.16:0/2235124290 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feed40d2750 0x7feed4140630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.745+0000 7feed37fe640 1 --2- 172.21.15.16:0/2235124290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feed40726c0 0x7feed41400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.745+0000 7feed3fff640 1 -- 172.21.15.16:0/2235124290 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feed40d2750 msgr2=0x7feed4140630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.745+0000 7feed3fff640 1 --2- 172.21.15.16:0/2235124290 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feed40d2750 0x7feed4140630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.745+0000 7feed3fff640 1 -- 172.21.15.16:0/2235124290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feed40726c0 msgr2=0x7feed41400f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.745+0000 7feed3fff640 1 --2- 172.21.15.16:0/2235124290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feed40726c0 0x7feed41400f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.745+0000 7feed3fff640 1 -- 172.21.15.16:0/2235124290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed406ddd0 con 0x7feed4072250 2024-09-19T07:55:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.745+0000 7feed3fff640 1 --2- 172.21.15.16:0/2235124290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed4072250 0x7feed40b10e0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7feec400b3f0 tx=0x7feec400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.745+0000 7feed0ff9640 1 -- 172.21.15.16:0/2235124290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4015020 con 0x7feed4072250 2024-09-19T07:55:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.745+0000 7feed0ff9640 1 -- 172.21.15.16:0/2235124290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec4011830 con 0x7feed4072250 2024-09-19T07:55:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.745+0000 7feed0ff9640 1 -- 172.21.15.16:0/2235124290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4011c20 con 0x7feed4072250 2024-09-19T07:55:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.745+0000 7feeda4f1640 1 -- 172.21.15.16:0/2235124290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed41f5280 con 0x7feed4072250 2024-09-19T07:55:03.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.745+0000 7feeda4f1640 1 -- 172.21.15.16:0/2235124290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed4142a00 con 0x7feed4072250 2024-09-19T07:55:03.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.746+0000 7feeda4f1640 1 -- 172.21.15.16:0/2235124290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed413f860 con 0x7feed4072250 2024-09-19T07:55:03.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.747+0000 7feed0ff9640 1 -- 172.21.15.16:0/2235124290 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7feec4018020 con 0x7feed4072250 2024-09-19T07:55:03.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.747+0000 7feed0ff9640 1 --2- 172.21.15.16:0/2235124290 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7feec006c2b0 0x7feec006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:03.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.747+0000 7feed0ff9640 1 -- 172.21.15.16:0/2235124290 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 6101+0+0 (secure 0 0 0) 0x7feec408b160 con 0x7feed4072250 2024-09-19T07:55:03.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.748+0000 7feed37fe640 1 --2- 172.21.15.16:0/2235124290 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7feec006c2b0 0x7feec006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:03.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.748+0000 7feed37fe640 1 --2- 172.21.15.16:0/2235124290 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7feec006c2b0 0x7feec006e770 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7feec8034000 tx=0x7feec802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:03.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.752+0000 7feed0ff9640 1 -- 172.21.15.16:0/2235124290 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7feec401f030 con 0x7feed4072250 2024-09-19T07:55:03.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.960+0000 7feeda4f1640 1 -- 172.21.15.16:0/2235124290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7feed40cf5f0 con 0x7feed4072250 2024-09-19T07:55:03.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.975+0000 7feed0ff9640 1 -- 172.21.15.16:0/2235124290 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v80) ==== 118+0+0 (secure 0 0 0) 0x7feed406ddd0 con 0x7feed4072250 2024-09-19T07:55:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.977+0000 7feeda4f1640 1 -- 172.21.15.16:0/2235124290 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7feec006c2b0 msgr2=0x7feec006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.977+0000 7feeda4f1640 1 --2- 172.21.15.16:0/2235124290 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7feec006c2b0 0x7feec006e770 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7feec8034000 tx=0x7feec802e000 comp rx=0 tx=0).stop 2024-09-19T07:55:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.977+0000 7feeda4f1640 1 -- 172.21.15.16:0/2235124290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed4072250 msgr2=0x7feed40b10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.977+0000 7feeda4f1640 1 --2- 172.21.15.16:0/2235124290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed4072250 0x7feed40b10e0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7feec400b3f0 tx=0x7feec400b8b0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.977+0000 7feeda4f1640 1 -- 172.21.15.16:0/2235124290 shutdown_connections 2024-09-19T07:55:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.977+0000 7feeda4f1640 1 --2- 172.21.15.16:0/2235124290 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7feec006c2b0 0x7feec006e770 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.977+0000 7feeda4f1640 1 --2- 172.21.15.16:0/2235124290 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feed40d2750 0x7feed4140630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.977+0000 7feeda4f1640 1 --2- 172.21.15.16:0/2235124290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feed40726c0 0x7feed41400f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.977+0000 7feeda4f1640 1 --2- 172.21.15.16:0/2235124290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed4072250 0x7feed40b10e0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.977+0000 7feeda4f1640 1 -- 172.21.15.16:0/2235124290 >> 172.21.15.16:0/2235124290 conn(0x7feed40b4040 msgr2=0x7feed40ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.977+0000 7feeda4f1640 1 -- 172.21.15.16:0/2235124290 shutdown_connections 2024-09-19T07:55:03.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:03.977+0000 7feeda4f1640 1 -- 172.21.15.16:0/2235124290 wait complete. 2024-09-19T07:55:03.989 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T07:55:03.989 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T07:55:04.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.094+0000 7f3eb48e6640 1 Processor -- start 2024-09-19T07:55:04.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.094+0000 7f3eb48e6640 1 -- start start 2024-09-19T07:55:04.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.095+0000 7f3eb48e6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3eb006cd20 0x7f3eb006d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:04.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.095+0000 7f3eb48e6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3eb006d640 con 0x7f3eb01425e0 2024-09-19T07:55:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.095+0000 7f3eb48e6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3eb0162b70 con 0x7f3eb0161eb0 2024-09-19T07:55:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.095+0000 7f3eb48e6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f3eb0162e50 con 0x7f3eb006cd20 2024-09-19T07:55:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.095+0000 7f3eae59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3eb006cd20 0x7f3eb006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.095+0000 7f3eaed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3eb01425e0 0x7f3eb0142a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33998/0 (socket says 172.21.15.16:33998) 2024-09-19T07:55:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.095+0000 7f3eaed9d640 1 -- 172.21.15.16:0/3606502011 learned_addr learned my addr 172.21.15.16:0/3606502011 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.095+0000 7f3eae59c640 1 -- 172.21.15.16:0/3606502011 >> v1:172.21.15.146:6789/0 conn(0x7f3eb0161eb0 legacy=0x7f3eb01622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.095+0000 7f3eae59c640 1 -- 172.21.15.16:0/3606502011 >> v1:172.21.15.16:6789/0 conn(0x7f3eb01425e0 legacy=0x7f3eb0142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.095+0000 7f3eae59c640 1 -- 172.21.15.16:0/3606502011 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb0141b00 con 0x7f3eb006cd20 2024-09-19T07:55:04.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.096+0000 7f3eae59c640 1 --2- 172.21.15.16:0/3606502011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb006cd20 0x7f3eb006d100 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00097a0 tx=0x7f3ea002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2bda39cb5c342 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:04.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.096+0000 7f3ead59a640 1 -- 172.21.15.16:0/3606502011 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0037070 con 0x7f3eb006cd20 2024-09-19T07:55:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.096+0000 7f3ead59a640 1 -- 172.21.15.16:0/3606502011 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ea002ae50 con 0x7f3eb006cd20 2024-09-19T07:55:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.096+0000 7f3ead59a640 1 -- 172.21.15.16:0/3606502011 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0032660 con 0x7f3eb006cd20 2024-09-19T07:55:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.097+0000 7f3eb48e6640 1 -- 172.21.15.16:0/3606502011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb006cd20 msgr2=0x7f3eb006d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.097+0000 7f3eb48e6640 1 --2- 172.21.15.16:0/3606502011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb006cd20 0x7f3eb006d100 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00097a0 tx=0x7f3ea002a140 comp rx=0 tx=0).stop 2024-09-19T07:55:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.097+0000 7f3eb48e6640 1 -- 172.21.15.16:0/3606502011 shutdown_connections 2024-09-19T07:55:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.097+0000 7f3eb48e6640 1 --2- 172.21.15.16:0/3606502011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb006cd20 0x7f3eb006d100 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.097+0000 7f3eb48e6640 1 -- 172.21.15.16:0/3606502011 >> 172.21.15.16:0/3606502011 conn(0x7f3eb00b3ed0 msgr2=0x7f3eb00b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.097+0000 7f3eb48e6640 1 -- 172.21.15.16:0/3606502011 shutdown_connections 2024-09-19T07:55:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.097+0000 7f3eb48e6640 1 -- 172.21.15.16:0/3606502011 wait complete. 2024-09-19T07:55:04.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.098+0000 7f3eb48e6640 1 Processor -- start 2024-09-19T07:55:04.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.098+0000 7f3eb48e6640 1 -- start start 2024-09-19T07:55:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.099+0000 7f3eb48e6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3eb006cd20 0x7f3eb01409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.099+0000 7f3eb48e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb01425e0 0x7f3eb00cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.099+0000 7f3eb48e6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3eb0161eb0 0x7f3eb00cec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.099+0000 7f3eb48e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb0141600 con 0x7f3eb01425e0 2024-09-19T07:55:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.099+0000 7f3eb48e6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb0142150 con 0x7f3eb006cd20 2024-09-19T07:55:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.099+0000 7f3eb48e6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3eb0143180 con 0x7f3eb0161eb0 2024-09-19T07:55:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.099+0000 7f3eadd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb01425e0 0x7f3eb00cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.099+0000 7f3eadd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb01425e0 0x7f3eb00cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43686/0 (socket says 172.21.15.16:43686) 2024-09-19T07:55:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.099+0000 7f3eadd9b640 1 -- 172.21.15.16:0/3665822727 learned_addr learned my addr 172.21.15.16:0/3665822727 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.099+0000 7f3eaed9d640 1 --2- 172.21.15.16:0/3665822727 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3eb0161eb0 0x7f3eb00cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.099+0000 7f3eae59c640 1 --2- 172.21.15.16:0/3665822727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3eb006cd20 0x7f3eb01409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.099+0000 7f3eadd9b640 1 -- 172.21.15.16:0/3665822727 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3eb0161eb0 msgr2=0x7f3eb00cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.099+0000 7f3eadd9b640 1 --2- 172.21.15.16:0/3665822727 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3eb0161eb0 0x7f3eb00cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.099+0000 7f3eadd9b640 1 -- 172.21.15.16:0/3665822727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3eb006cd20 msgr2=0x7f3eb01409a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.099+0000 7f3eadd9b640 1 --2- 172.21.15.16:0/3665822727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3eb006cd20 0x7f3eb01409a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.099+0000 7f3eadd9b640 1 -- 172.21.15.16:0/3665822727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb0140ee0 con 0x7f3eb01425e0 2024-09-19T07:55:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.099+0000 7f3eae59c640 1 --2- 172.21.15.16:0/3665822727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3eb006cd20 0x7f3eb01409a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.099+0000 7f3eaed9d640 1 --2- 172.21.15.16:0/3665822727 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3eb0161eb0 0x7f3eb00cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.100+0000 7f3eadd9b640 1 --2- 172.21.15.16:0/3665822727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb01425e0 0x7f3eb00cbd20 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400b3f0 tx=0x7f3ea400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.100+0000 7f3e9effd640 1 -- 172.21.15.16:0/3665822727 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4015020 con 0x7f3eb01425e0 2024-09-19T07:55:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.100+0000 7f3e9effd640 1 -- 172.21.15.16:0/3665822727 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ea4011830 con 0x7f3eb01425e0 2024-09-19T07:55:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.100+0000 7f3eb48e6640 1 -- 172.21.15.16:0/3665822727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb00d20e0 con 0x7f3eb01425e0 2024-09-19T07:55:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.100+0000 7f3e9effd640 1 -- 172.21.15.16:0/3665822727 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4011c20 con 0x7f3eb01425e0 2024-09-19T07:55:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.100+0000 7f3eb48e6640 1 -- 172.21.15.16:0/3665822727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb00cd3d0 con 0x7f3eb01425e0 2024-09-19T07:55:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.101+0000 7f3eb48e6640 1 -- 172.21.15.16:0/3665822727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb0143510 con 0x7f3eb01425e0 2024-09-19T07:55:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.101+0000 7f3e9effd640 1 -- 172.21.15.16:0/3665822727 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3ea4018020 con 0x7f3eb01425e0 2024-09-19T07:55:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.102+0000 7f3e9effd640 1 --2- 172.21.15.16:0/3665822727 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3e9006c2b0 0x7f3e9006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.102+0000 7f3e9effd640 1 -- 172.21.15.16:0/3665822727 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f3ea408b530 con 0x7f3eb01425e0 2024-09-19T07:55:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.102+0000 7f3eae59c640 1 --2- 172.21.15.16:0/3665822727 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3e9006c2b0 0x7f3e9006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.103+0000 7f3eae59c640 1 --2- 172.21.15.16:0/3665822727 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3e9006c2b0 0x7f3e9006e770 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0034000 tx=0x7f3ea002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:04.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.106+0000 7f3e9effd640 1 -- 172.21.15.16:0/3665822727 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3ea401f020 con 0x7f3eb01425e0 2024-09-19T07:55:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.317+0000 7f3eb48e6640 1 -- 172.21.15.16:0/3665822727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3eb0072220 con 0x7f3eb01425e0 2024-09-19T07:55:04.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.984+0000 7f3e9effd640 1 -- 172.21.15.16:0/3665822727 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v81) ==== 122+0+0 (secure 0 0 0) 0x7f3ea40172c0 con 0x7f3eb01425e0 2024-09-19T07:55:04.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.987+0000 7f3eb48e6640 1 -- 172.21.15.16:0/3665822727 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3e9006c2b0 msgr2=0x7f3e9006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:04.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.987+0000 7f3eb48e6640 1 --2- 172.21.15.16:0/3665822727 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3e9006c2b0 0x7f3e9006e770 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0034000 tx=0x7f3ea002e000 comp rx=0 tx=0).stop 2024-09-19T07:55:04.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.987+0000 7f3eb48e6640 1 -- 172.21.15.16:0/3665822727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb01425e0 msgr2=0x7f3eb00cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.987+0000 7f3eb48e6640 1 --2- 172.21.15.16:0/3665822727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb01425e0 0x7f3eb00cbd20 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400b3f0 tx=0x7f3ea400b8b0 comp rx=0 tx=0).stop 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.987+0000 7f3eb48e6640 1 -- 172.21.15.16:0/3665822727 shutdown_connections 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.987+0000 7f3eb48e6640 1 --2- 172.21.15.16:0/3665822727 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3e9006c2b0 0x7f3e9006e770 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.987+0000 7f3eb48e6640 1 --2- 172.21.15.16:0/3665822727 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3eb0161eb0 0x7f3eb00cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.987+0000 7f3eb48e6640 1 --2- 172.21.15.16:0/3665822727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb01425e0 0x7f3eb00cbd20 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.987+0000 7f3eb48e6640 1 --2- 172.21.15.16:0/3665822727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3eb006cd20 0x7f3eb01409a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.987+0000 7f3eb48e6640 1 -- 172.21.15.16:0/3665822727 >> 172.21.15.16:0/3665822727 conn(0x7f3eb00b3ed0 msgr2=0x7f3eb0160a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.987+0000 7f3eb48e6640 1 -- 172.21.15.16:0/3665822727 shutdown_connections 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:04.987+0000 7f3eb48e6640 1 -- 172.21.15.16:0/3665822727 wait complete. 2024-09-19T07:55:05.001 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T07:55:05.001 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:55:05.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.109+0000 7f8c51e1e640 1 Processor -- start 2024-09-19T07:55:05.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.109+0000 7f8c51e1e640 1 -- start start 2024-09-19T07:55:05.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.110+0000 7f8c51e1e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8c4c15e2a0 0x7f8c4c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:05.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.110+0000 7f8c51e1e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c4c15e680 0x7f8c4c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:05.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.110+0000 7f8c51e1e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c4c0d1880 con 0x7f8c4c06d960 2024-09-19T07:55:05.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.110+0000 7f8c51e1e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f8c4c0d1b60 con 0x7f8c4c15e680 2024-09-19T07:55:05.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.110+0000 7f8c51e1e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f8c4c0cf2d0 con 0x7f8c4c15e2a0 2024-09-19T07:55:05.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.110+0000 7f8c4affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c4c15e680 0x7f8c4c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:05.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.110+0000 7f8c4affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c4c15e680 0x7f8c4c0cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43696/0 (socket says 172.21.15.16:43696) 2024-09-19T07:55:05.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.110+0000 7f8c4affd640 1 -- 172.21.15.16:0/3491381856 learned_addr learned my addr 172.21.15.16:0/3491381856 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:05.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.110+0000 7f8c4affd640 1 -- 172.21.15.16:0/3491381856 >> v2:172.21.15.146:3300/0 conn(0x7f8c4c15e2a0 msgr2=0x7f8c4c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:05.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.110+0000 7f8c4affd640 1 --2- 172.21.15.16:0/3491381856 >> v2:172.21.15.146:3300/0 conn(0x7f8c4c15e2a0 0x7f8c4c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.110+0000 7f8c4affd640 1 -- 172.21.15.16:0/3491381856 >> v1:172.21.15.16:6789/0 conn(0x7f8c4c06d960 legacy=0x7f8c4c0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.110+0000 7f8c4affd640 1 -- 172.21.15.16:0/3491381856 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4c0cf5b0 con 0x7f8c4c15e680 2024-09-19T07:55:05.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.110+0000 7f8c4a7fc640 1 -- 172.21.15.16:0/3491381856 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2662586416 0 0) 0x7f8c4c0d1880 con 0x7f8c4c06d960 2024-09-19T07:55:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.111+0000 7f8c4affd640 1 --2- 172.21.15.16:0/3491381856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c4c15e680 0x7f8c4c0cfad0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0b6fa0 tx=0x7f8c4002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ce9ad1e8f2ee1e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.111+0000 7f8c4a7fc640 1 -- 172.21.15.16:0/3491381856 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40037070 con 0x7f8c4c15e680 2024-09-19T07:55:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.111+0000 7f8c4a7fc640 1 -- 172.21.15.16:0/3491381856 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c40032440 con 0x7f8c4c15e680 2024-09-19T07:55:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.111+0000 7f8c4a7fc640 1 -- 172.21.15.16:0/3491381856 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40032810 con 0x7f8c4c15e680 2024-09-19T07:55:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.112+0000 7f8c51e1e640 1 -- 172.21.15.16:0/3491381856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c4c15e680 msgr2=0x7f8c4c0cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.112+0000 7f8c51e1e640 1 --2- 172.21.15.16:0/3491381856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c4c15e680 0x7f8c4c0cfad0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0b6fa0 tx=0x7f8c4002a270 comp rx=0 tx=0).stop 2024-09-19T07:55:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.112+0000 7f8c51e1e640 1 -- 172.21.15.16:0/3491381856 shutdown_connections 2024-09-19T07:55:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.112+0000 7f8c51e1e640 1 --2- 172.21.15.16:0/3491381856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c4c15e680 0x7f8c4c0cfad0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.112+0000 7f8c51e1e640 1 --2- 172.21.15.16:0/3491381856 >> v2:172.21.15.146:3300/0 conn(0x7f8c4c15e2a0 0x7f8c4c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.112+0000 7f8c51e1e640 1 -- 172.21.15.16:0/3491381856 >> 172.21.15.16:0/3491381856 conn(0x7f8c4c09d760 msgr2=0x7f8c4c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.112+0000 7f8c51e1e640 1 -- 172.21.15.16:0/3491381856 shutdown_connections 2024-09-19T07:55:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.112+0000 7f8c51e1e640 1 -- 172.21.15.16:0/3491381856 wait complete. 2024-09-19T07:55:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.113+0000 7f8c51e1e640 1 Processor -- start 2024-09-19T07:55:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.113+0000 7f8c51e1e640 1 -- start start 2024-09-19T07:55:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.114+0000 7f8c51e1e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c4c06d960 0x7f8c4c15c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.114+0000 7f8c51e1e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c4c15e2a0 0x7f8c4c15c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.114+0000 7f8c51e1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c4c15e680 0x7f8c4c1e9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.114+0000 7f8c51e1e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c0d4c00 con 0x7f8c4c15e680 2024-09-19T07:55:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.114+0000 7f8c51e1e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c0d4a80 con 0x7f8c4c06d960 2024-09-19T07:55:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.114+0000 7f8c51e1e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8c4c0d0010 con 0x7f8c4c15e2a0 2024-09-19T07:55:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.114+0000 7f8c4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c4c15e680 0x7f8c4c1e9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.114+0000 7f8c4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c4c15e680 0x7f8c4c1e9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43708/0 (socket says 172.21.15.16:43708) 2024-09-19T07:55:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.114+0000 7f8c4bfff640 1 -- 172.21.15.16:0/352038132 learned_addr learned my addr 172.21.15.16:0/352038132 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.114+0000 7f8c4affd640 1 --2- 172.21.15.16:0/352038132 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c4c15e2a0 0x7f8c4c15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.114+0000 7f8c4b7fe640 1 --2- 172.21.15.16:0/352038132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c4c06d960 0x7f8c4c15c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.114+0000 7f8c4bfff640 1 -- 172.21.15.16:0/352038132 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c4c15e2a0 msgr2=0x7f8c4c15c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.114+0000 7f8c4bfff640 1 --2- 172.21.15.16:0/352038132 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c4c15e2a0 0x7f8c4c15c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.114+0000 7f8c4bfff640 1 -- 172.21.15.16:0/352038132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c4c06d960 msgr2=0x7f8c4c15c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.114+0000 7f8c4bfff640 1 --2- 172.21.15.16:0/352038132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c4c06d960 0x7f8c4c15c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.114+0000 7f8c4bfff640 1 -- 172.21.15.16:0/352038132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4c0cc1f0 con 0x7f8c4c15e680 2024-09-19T07:55:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.114+0000 7f8c4b7fe640 1 --2- 172.21.15.16:0/352038132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c4c06d960 0x7f8c4c15c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.115+0000 7f8c4bfff640 1 --2- 172.21.15.16:0/352038132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c4c15e680 0x7f8c4c1e9080 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f8c4400c790 tx=0x7f8c4400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.115+0000 7f8c33fff640 1 -- 172.21.15.16:0/352038132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c44003da0 con 0x7f8c4c15e680 2024-09-19T07:55:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.115+0000 7f8c33fff640 1 -- 172.21.15.16:0/352038132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c44005070 con 0x7f8c4c15e680 2024-09-19T07:55:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.115+0000 7f8c51e1e640 1 -- 172.21.15.16:0/352038132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c4c0cb5e0 con 0x7f8c4c15e680 2024-09-19T07:55:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.115+0000 7f8c33fff640 1 -- 172.21.15.16:0/352038132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c44005460 con 0x7f8c4c15e680 2024-09-19T07:55:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.115+0000 7f8c51e1e640 1 -- 172.21.15.16:0/352038132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c4c0cbba0 con 0x7f8c4c15e680 2024-09-19T07:55:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.116+0000 7f8c51e1e640 1 -- 172.21.15.16:0/352038132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c4c06d420 con 0x7f8c4c15e680 2024-09-19T07:55:05.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.116+0000 7f8c33fff640 1 -- 172.21.15.16:0/352038132 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8c44005600 con 0x7f8c4c15e680 2024-09-19T07:55:05.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.117+0000 7f8c33fff640 1 --2- 172.21.15.16:0/352038132 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8c3806c2b0 0x7f8c3806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:05.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.117+0000 7f8c33fff640 1 -- 172.21.15.16:0/352038132 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f8c4408b4c0 con 0x7f8c4c15e680 2024-09-19T07:55:05.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.117+0000 7f8c4b7fe640 1 --2- 172.21.15.16:0/352038132 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8c3806c2b0 0x7f8c3806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:05.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.118+0000 7f8c4b7fe640 1 --2- 172.21.15.16:0/352038132 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8c3806c2b0 0x7f8c3806e770 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c161270 tx=0x7f8c3c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:05.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.121+0000 7f8c33fff640 1 -- 172.21.15.16:0/352038132 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8c44059ab0 con 0x7f8c4c15e680 2024-09-19T07:55:05.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.333+0000 7f8c51e1e640 1 -- 172.21.15.16:0/352038132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c4c06cf90 con 0x7f8c4c15e680 2024-09-19T07:55:05.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.333+0000 7f8c33fff640 1 -- 172.21.15.16:0/352038132 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5230 (secure 0 0 0) 0x7f8c440593b0 con 0x7f8c4c15e680 2024-09-19T07:55:05.334 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:05.334 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"btime":"2024-09-19T07:55:04:980435+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/172459502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":172459502},{"type":"v1","addr":"172.21.15.146:6836","nonce":172459502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3417061308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3417061308},{"type":"v1","addr":"172.21.15.146:6837","nonce":3417061308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1810646409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1810646409},{"type":"v1","addr":"172.21.15.16:6834","nonce":1810646409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:55:02.325256+0000","modified":"2024-09-19T07:55:04.318652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8286":{"gid":8286,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/1929965275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1929965275},{"type":"v1","addr":"172.21.15.16:6835","nonce":1929965275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8286,"qdb_cluster":[8286]},"id":10}]} 2024-09-19T07:55:05.334 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 81 2024-09-19T07:55:05.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.335+0000 7f8c51e1e640 1 -- 172.21.15.16:0/352038132 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8c3806c2b0 msgr2=0x7f8c3806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.335+0000 7f8c51e1e640 1 --2- 172.21.15.16:0/352038132 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8c3806c2b0 0x7f8c3806e770 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c161270 tx=0x7f8c3c008040 comp rx=0 tx=0).stop 2024-09-19T07:55:05.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.335+0000 7f8c51e1e640 1 -- 172.21.15.16:0/352038132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c4c15e680 msgr2=0x7f8c4c1e9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.335+0000 7f8c51e1e640 1 --2- 172.21.15.16:0/352038132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c4c15e680 0x7f8c4c1e9080 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f8c4400c790 tx=0x7f8c4400cc50 comp rx=0 tx=0).stop 2024-09-19T07:55:05.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.335+0000 7f8c51e1e640 1 -- 172.21.15.16:0/352038132 shutdown_connections 2024-09-19T07:55:05.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.335+0000 7f8c51e1e640 1 --2- 172.21.15.16:0/352038132 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8c3806c2b0 0x7f8c3806e770 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.335+0000 7f8c51e1e640 1 --2- 172.21.15.16:0/352038132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c4c15e680 0x7f8c4c1e9080 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.335+0000 7f8c51e1e640 1 --2- 172.21.15.16:0/352038132 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c4c15e2a0 0x7f8c4c15c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.335+0000 7f8c51e1e640 1 --2- 172.21.15.16:0/352038132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c4c06d960 0x7f8c4c15c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.335+0000 7f8c51e1e640 1 -- 172.21.15.16:0/352038132 >> 172.21.15.16:0/352038132 conn(0x7f8c4c09d760 msgr2=0x7f8c4c15f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:05.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.336+0000 7f8c51e1e640 1 -- 172.21.15.16:0/352038132 shutdown_connections 2024-09-19T07:55:05.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.336+0000 7f8c51e1e640 1 -- 172.21.15.16:0/352038132 wait complete. 2024-09-19T07:55:05.347 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:55:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.449+0000 7fbeecaae640 1 Processor -- start 2024-09-19T07:55:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.449+0000 7fbeecaae640 1 -- start start 2024-09-19T07:55:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.450+0000 7fbeecaae640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbee815bee0 0x7fbee80d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.450+0000 7fbeecaae640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee80cdee0 con 0x7fbee815c2c0 2024-09-19T07:55:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.450+0000 7fbeecaae640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee80ce1c0 con 0x7fbee80cf910 2024-09-19T07:55:05.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.450+0000 7fbeecaae640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fbee80ceaf0 con 0x7fbee815bee0 2024-09-19T07:55:05.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.450+0000 7fbee659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbee815bee0 0x7fbee80d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:05.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.450+0000 7fbee659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbee815bee0 0x7fbee80d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43718/0 (socket says 172.21.15.16:43718) 2024-09-19T07:55:05.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.450+0000 7fbee659c640 1 -- 172.21.15.16:0/312634759 learned_addr learned my addr 172.21.15.16:0/312634759 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:05.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.450+0000 7fbedffff640 1 --1- 172.21.15.16:0/312634759 >> v1:172.21.15.16:6789/0 conn(0x7fbee815c2c0 0x7fbee80cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34010/0 (socket says 172.21.15.16:34010) 2024-09-19T07:55:05.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.450+0000 7fbee659c640 1 -- 172.21.15.16:0/312634759 >> v1:172.21.15.146:6789/0 conn(0x7fbee80cf910 legacy=0x7fbee806dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.450+0000 7fbee659c640 1 -- 172.21.15.16:0/312634759 >> v1:172.21.15.16:6789/0 conn(0x7fbee815c2c0 legacy=0x7fbee80cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.450+0000 7fbee5d9b640 1 -- 172.21.15.16:0/312634759 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3372208178 0 0) 0x7fbee80cdee0 con 0x7fbee815c2c0 2024-09-19T07:55:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.450+0000 7fbee659c640 1 -- 172.21.15.16:0/312634759 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee80ce4a0 con 0x7fbee815bee0 2024-09-19T07:55:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.451+0000 7fbee659c640 1 --2- 172.21.15.16:0/312634759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee815bee0 0x7fbee80d04b0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009870 tx=0x7fbed802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4abed957483ce9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.451+0000 7fbee5d9b640 1 -- 172.21.15.16:0/312634759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8037070 con 0x7fbee815bee0 2024-09-19T07:55:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.451+0000 7fbee5d9b640 1 -- 172.21.15.16:0/312634759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed802ae50 con 0x7fbee815bee0 2024-09-19T07:55:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.451+0000 7fbee5d9b640 1 -- 172.21.15.16:0/312634759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8032860 con 0x7fbee815bee0 2024-09-19T07:55:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.452+0000 7fbeecaae640 1 -- 172.21.15.16:0/312634759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee815bee0 msgr2=0x7fbee80d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.452+0000 7fbeecaae640 1 --2- 172.21.15.16:0/312634759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee815bee0 0x7fbee80d04b0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009870 tx=0x7fbed802a300 comp rx=0 tx=0).stop 2024-09-19T07:55:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.452+0000 7fbeecaae640 1 -- 172.21.15.16:0/312634759 shutdown_connections 2024-09-19T07:55:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.452+0000 7fbeecaae640 1 --2- 172.21.15.16:0/312634759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee815bee0 0x7fbee80d04b0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.452+0000 7fbeecaae640 1 -- 172.21.15.16:0/312634759 >> 172.21.15.16:0/312634759 conn(0x7fbee806bdf0 msgr2=0x7fbee806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.452+0000 7fbeecaae640 1 -- 172.21.15.16:0/312634759 shutdown_connections 2024-09-19T07:55:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.452+0000 7fbeecaae640 1 -- 172.21.15.16:0/312634759 wait complete. 2024-09-19T07:55:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.453+0000 7fbeecaae640 1 Processor -- start 2024-09-19T07:55:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.453+0000 7fbeecaae640 1 -- start start 2024-09-19T07:55:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.454+0000 7fbeecaae640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbee80cf910 0x7fbee80af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.454+0000 7fbeecaae640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbee815bee0 0x7fbee81f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.454+0000 7fbeecaae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee815c2c0 0x7fbee81ef1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.454+0000 7fbeecaae640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbee80d0b70 con 0x7fbee815c2c0 2024-09-19T07:55:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.454+0000 7fbeecaae640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbee80d09f0 con 0x7fbee815bee0 2024-09-19T07:55:05.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.454+0000 7fbeecaae640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbee80d0cf0 con 0x7fbee80cf910 2024-09-19T07:55:05.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.454+0000 7fbee6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee815c2c0 0x7fbee81ef1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:05.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.454+0000 7fbee6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee815c2c0 0x7fbee81ef1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43726/0 (socket says 172.21.15.16:43726) 2024-09-19T07:55:05.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.454+0000 7fbee6d9d640 1 -- 172.21.15.16:0/4253504493 learned_addr learned my addr 172.21.15.16:0/4253504493 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:05.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.454+0000 7fbedffff640 1 --2- 172.21.15.16:0/4253504493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbee815bee0 0x7fbee81f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:05.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.454+0000 7fbee659c640 1 --2- 172.21.15.16:0/4253504493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbee80cf910 0x7fbee80af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:05.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.454+0000 7fbee6d9d640 1 -- 172.21.15.16:0/4253504493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbee80cf910 msgr2=0x7fbee80af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.454+0000 7fbee6d9d640 1 --2- 172.21.15.16:0/4253504493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbee80cf910 0x7fbee80af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.454+0000 7fbee6d9d640 1 -- 172.21.15.16:0/4253504493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbee815bee0 msgr2=0x7fbee81f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.454+0000 7fbee6d9d640 1 --2- 172.21.15.16:0/4253504493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbee815bee0 0x7fbee81f1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.454+0000 7fbee6d9d640 1 -- 172.21.15.16:0/4253504493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee80adfa0 con 0x7fbee815c2c0 2024-09-19T07:55:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.454+0000 7fbedffff640 1 --2- 172.21.15.16:0/4253504493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbee815bee0 0x7fbee81f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.454+0000 7fbee659c640 1 --2- 172.21.15.16:0/4253504493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbee80cf910 0x7fbee80af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.455+0000 7fbee6d9d640 1 --2- 172.21.15.16:0/4253504493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee815c2c0 0x7fbee81ef1d0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fbee000c6b0 tx=0x7fbee000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.455+0000 7fbedeffd640 1 -- 172.21.15.16:0/4253504493 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0003b70 con 0x7fbee815c2c0 2024-09-19T07:55:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.455+0000 7fbedeffd640 1 -- 172.21.15.16:0/4253504493 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee0003d30 con 0x7fbee815c2c0 2024-09-19T07:55:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.455+0000 7fbedeffd640 1 -- 172.21.15.16:0/4253504493 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0005310 con 0x7fbee815c2c0 2024-09-19T07:55:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.455+0000 7fbeecaae640 1 -- 172.21.15.16:0/4253504493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee80ae280 con 0x7fbee815c2c0 2024-09-19T07:55:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.455+0000 7fbeecaae640 1 -- 172.21.15.16:0/4253504493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee80ae840 con 0x7fbee815c2c0 2024-09-19T07:55:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.456+0000 7fbeecaae640 1 -- 172.21.15.16:0/4253504493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee80d04b0 con 0x7fbee815c2c0 2024-09-19T07:55:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.456+0000 7fbedeffd640 1 -- 172.21.15.16:0/4253504493 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbee00054b0 con 0x7fbee815c2c0 2024-09-19T07:55:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.457+0000 7fbedeffd640 1 --2- 172.21.15.16:0/4253504493 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbec806c2e0 0x7fbec806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.457+0000 7fbedeffd640 1 -- 172.21.15.16:0/4253504493 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7fbee008b410 con 0x7fbee815c2c0 2024-09-19T07:55:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.457+0000 7fbee659c640 1 --2- 172.21.15.16:0/4253504493 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbec806c2e0 0x7fbec806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.458+0000 7fbee659c640 1 --2- 172.21.15.16:0/4253504493 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbec806c2e0 0x7fbec806e7a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbed8034000 tx=0x7fbed802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.461+0000 7fbedeffd640 1 -- 172.21.15.16:0/4253504493 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fbee00599f0 con 0x7fbee815c2c0 2024-09-19T07:55:05.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.638+0000 7fbeecaae640 1 -- 172.21.15.16:0/4253504493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbee806ecc0 con 0x7fbee815c2c0 2024-09-19T07:55:05.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.640+0000 7fbedeffd640 1 -- 172.21.15.16:0/4253504493 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) ==== 74+0+16976 (secure 0 0 0) 0x7fbee00592f0 con 0x7fbee815c2c0 2024-09-19T07:55:05.640 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:05.640 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":140,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:55:03.967221+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:54:59.994353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-09-19T07:55:01.315629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:55:05.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.642+0000 7fbeecaae640 1 -- 172.21.15.16:0/4253504493 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbec806c2e0 msgr2=0x7fbec806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.642+0000 7fbeecaae640 1 --2- 172.21.15.16:0/4253504493 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbec806c2e0 0x7fbec806e7a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbed8034000 tx=0x7fbed802e000 comp rx=0 tx=0).stop 2024-09-19T07:55:05.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.642+0000 7fbeecaae640 1 -- 172.21.15.16:0/4253504493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee815c2c0 msgr2=0x7fbee81ef1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.642+0000 7fbeecaae640 1 --2- 172.21.15.16:0/4253504493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee815c2c0 0x7fbee81ef1d0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fbee000c6b0 tx=0x7fbee000cb70 comp rx=0 tx=0).stop 2024-09-19T07:55:05.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.642+0000 7fbeecaae640 1 -- 172.21.15.16:0/4253504493 shutdown_connections 2024-09-19T07:55:05.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.642+0000 7fbeecaae640 1 --2- 172.21.15.16:0/4253504493 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbec806c2e0 0x7fbec806e7a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.642+0000 7fbeecaae640 1 --2- 172.21.15.16:0/4253504493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee815c2c0 0x7fbee81ef1d0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.642+0000 7fbeecaae640 1 --2- 172.21.15.16:0/4253504493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbee815bee0 0x7fbee81f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.642+0000 7fbeecaae640 1 --2- 172.21.15.16:0/4253504493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbee80cf910 0x7fbee80af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.642+0000 7fbeecaae640 1 -- 172.21.15.16:0/4253504493 >> 172.21.15.16:0/4253504493 conn(0x7fbee806bdf0 msgr2=0x7fbee806dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:05.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.642+0000 7fbeecaae640 1 -- 172.21.15.16:0/4253504493 shutdown_connections 2024-09-19T07:55:05.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.642+0000 7fbeecaae640 1 -- 172.21.15.16:0/4253504493 wait complete. 2024-09-19T07:55:05.654 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T07:55:05.654 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:55:05.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.755+0000 7f2bb416d640 1 Processor -- start 2024-09-19T07:55:05.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.756+0000 7f2bb416d640 1 -- start start 2024-09-19T07:55:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb416d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2bac0d21f0 0x7f2bac0d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb416d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bac0d2b10 con 0x7f2bac141ad0 2024-09-19T07:55:05.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb416d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bac1620b0 con 0x7f2bac072590 2024-09-19T07:55:05.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb416d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2bac141750 con 0x7f2bac0d21f0 2024-09-19T07:55:05.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb270a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2bac141ad0 0x7f2bac141f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34012/0 (socket says 172.21.15.16:34012) 2024-09-19T07:55:05.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb270a640 1 -- 172.21.15.16:0/862564808 learned_addr learned my addr 172.21.15.16:0/862564808 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:05.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb1f09640 1 --2- 172.21.15.16:0/862564808 >> v2:172.21.15.146:3300/0 conn(0x7f2bac0d21f0 0x7f2bac0d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:05.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb0f07640 1 -- 172.21.15.16:0/862564808 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1578977364 0 0) 0x7f2bac0d2b10 con 0x7f2bac141ad0 2024-09-19T07:55:05.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb0f07640 1 -- 172.21.15.16:0/862564808 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b98003520 con 0x7f2bac141ad0 2024-09-19T07:55:05.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb0f07640 1 -- 172.21.15.16:0/862564808 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2895804132 0 0) 0x7f2b98003520 con 0x7f2bac141ad0 2024-09-19T07:55:05.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb0f07640 1 -- 172.21.15.16:0/862564808 >> v2:172.21.15.146:3300/0 conn(0x7f2bac0d21f0 msgr2=0x7f2bac0d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:05.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb0f07640 1 --2- 172.21.15.16:0/862564808 >> v2:172.21.15.146:3300/0 conn(0x7f2bac0d21f0 0x7f2bac0d25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb0f07640 1 -- 172.21.15.16:0/862564808 >> v1:172.21.15.146:6789/0 conn(0x7f2bac072590 legacy=0x7f2bac06d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb0f07640 1 -- 172.21.15.16:0/862564808 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bac162370 con 0x7f2bac141ad0 2024-09-19T07:55:05.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb0f07640 1 -- 172.21.15.16:0/862564808 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f2ba40019f0 con 0x7f2bac141ad0 2024-09-19T07:55:05.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb0f07640 1 -- 172.21.15.16:0/862564808 >> v1:172.21.15.16:6789/0 conn(0x7f2bac141ad0 legacy=0x7f2bac141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb0f07640 1 --2- 172.21.15.16:0/862564808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b980039b0 0x7f2b98003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:05.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb0f07640 1 --2- 172.21.15.16:0/862564808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b980043b0 0x7f2b98004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:05.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb0f07640 1 --2- 172.21.15.16:0/862564808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b98005120 0x7f2b9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:05.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb0f07640 1 -- 172.21.15.16:0/862564808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac141750 con 0x7f2b980039b0 2024-09-19T07:55:05.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb0f07640 1 -- 172.21.15.16:0/862564808 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9800c060 con 0x7f2b98005120 2024-09-19T07:55:05.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.757+0000 7f2bb0f07640 1 -- 172.21.15.16:0/862564808 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2b9800c260 con 0x7f2b980043b0 2024-09-19T07:55:05.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.758+0000 7f2bb1f09640 1 --2- 172.21.15.16:0/862564808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b980039b0 0x7f2b98003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:05.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.758+0000 7f2bb270a640 1 --2- 172.21.15.16:0/862564808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b98005120 0x7f2b9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:05.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.758+0000 7f2bb1708640 1 --2- 172.21.15.16:0/862564808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b980043b0 0x7f2b98004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:05.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.758+0000 7f2bb1f09640 1 -- 172.21.15.16:0/862564808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b980043b0 msgr2=0x7f2b98004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:05.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.758+0000 7f2bb1f09640 1 --2- 172.21.15.16:0/862564808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b980043b0 0x7f2b98004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.758+0000 7f2bb1f09640 1 -- 172.21.15.16:0/862564808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b98005120 msgr2=0x7f2b9800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:05.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.758+0000 7f2bb1f09640 1 --2- 172.21.15.16:0/862564808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b98005120 0x7f2b9800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.758+0000 7f2bb1f09640 1 -- 172.21.15.16:0/862564808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b98003520 con 0x7f2b980039b0 2024-09-19T07:55:05.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.758+0000 7f2bb270a640 1 --2- 172.21.15.16:0/862564808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b98005120 0x7f2b9800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:05.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.758+0000 7f2bb270a640 1 -- 172.21.15.16:0/862564808 reap_dead start 2024-09-19T07:55:05.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.759+0000 7f2bb1f09640 1 --2- 172.21.15.16:0/862564808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b980039b0 0x7f2b98003d90 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009ac0 tx=0x7f2b98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d19819156af44d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:05.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.759+0000 7f2bb0f07640 1 -- 172.21.15.16:0/862564808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8036070 con 0x7f2b980039b0 2024-09-19T07:55:05.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.759+0000 7f2bb0f07640 1 -- 172.21.15.16:0/862564808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ba8031400 con 0x7f2b980039b0 2024-09-19T07:55:05.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.759+0000 7f2bb0f07640 1 -- 172.21.15.16:0/862564808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8009690 con 0x7f2b980039b0 2024-09-19T07:55:05.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.759+0000 7f2bb416d640 1 -- 172.21.15.16:0/862564808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b980039b0 msgr2=0x7f2b98003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.759+0000 7f2bb416d640 1 --2- 172.21.15.16:0/862564808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b980039b0 0x7f2b98003d90 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009ac0 tx=0x7f2b98005830 comp rx=0 tx=0).stop 2024-09-19T07:55:05.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.759+0000 7f2bb416d640 1 -- 172.21.15.16:0/862564808 shutdown_connections 2024-09-19T07:55:05.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.759+0000 7f2bb416d640 1 --2- 172.21.15.16:0/862564808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b980039b0 0x7f2b98003d90 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.759+0000 7f2bb416d640 1 -- 172.21.15.16:0/862564808 >> 172.21.15.16:0/862564808 conn(0x7f2bac0b3ed0 msgr2=0x7f2bac0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:05.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.759+0000 7f2bb416d640 1 -- 172.21.15.16:0/862564808 shutdown_connections 2024-09-19T07:55:05.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.759+0000 7f2bb416d640 1 -- 172.21.15.16:0/862564808 wait complete. 2024-09-19T07:55:05.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.760+0000 7f2bb416d640 1 Processor -- start 2024-09-19T07:55:05.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.761+0000 7f2bb416d640 1 -- start start 2024-09-19T07:55:05.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.761+0000 7f2bb416d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b980039b0 0x7f2bac140e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:05.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.761+0000 7f2bb416d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bac0b7c20 0x7f2bac159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:05.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.761+0000 7f2bb416d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bac15ad20 0x7f2bac15b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.761+0000 7f2bb416d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac162930 con 0x7f2bac0b7c20 2024-09-19T07:55:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.761+0000 7f2bb416d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac06d7f0 con 0x7f2bac15ad20 2024-09-19T07:55:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.761+0000 7f2bb416d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2bac162c10 con 0x7f2b980039b0 2024-09-19T07:55:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.761+0000 7f2bb1708640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bac0b7c20 0x7f2bac159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.761+0000 7f2bb1708640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bac0b7c20 0x7f2bac159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43742/0 (socket says 172.21.15.16:43742) 2024-09-19T07:55:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.761+0000 7f2bb1708640 1 -- 172.21.15.16:0/3544453420 learned_addr learned my addr 172.21.15.16:0/3544453420 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.761+0000 7f2bb1f09640 1 --2- 172.21.15.16:0/3544453420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b980039b0 0x7f2bac140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.761+0000 7f2bb270a640 1 --2- 172.21.15.16:0/3544453420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bac15ad20 0x7f2bac15b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.762+0000 7f2bb1708640 1 -- 172.21.15.16:0/3544453420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b980039b0 msgr2=0x7f2bac140e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.762+0000 7f2bb1708640 1 --2- 172.21.15.16:0/3544453420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b980039b0 0x7f2bac140e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.762+0000 7f2bb1708640 1 -- 172.21.15.16:0/3544453420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bac15ad20 msgr2=0x7f2bac15b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.762+0000 7f2bb1708640 1 --2- 172.21.15.16:0/3544453420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bac15ad20 0x7f2bac15b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.762+0000 7f2bb1708640 1 -- 172.21.15.16:0/3544453420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bac0b5270 con 0x7f2bac0b7c20 2024-09-19T07:55:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.762+0000 7f2bb270a640 1 --2- 172.21.15.16:0/3544453420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bac15ad20 0x7f2bac15b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:05.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.762+0000 7f2bb1708640 1 --2- 172.21.15.16:0/3544453420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bac0b7c20 0x7f2bac159640 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00e540 tx=0x7f2b9c00ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:05.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.762+0000 7f2ba27fc640 1 -- 172.21.15.16:0/3544453420 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c0052e0 con 0x7f2bac0b7c20 2024-09-19T07:55:05.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.762+0000 7f2ba27fc640 1 -- 172.21.15.16:0/3544453420 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b9c018070 con 0x7f2bac0b7c20 2024-09-19T07:55:05.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.762+0000 7f2bb416d640 1 -- 172.21.15.16:0/3544453420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bac0ae250 con 0x7f2bac0b7c20 2024-09-19T07:55:05.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.762+0000 7f2ba27fc640 1 -- 172.21.15.16:0/3544453420 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c003830 con 0x7f2bac0b7c20 2024-09-19T07:55:05.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.762+0000 7f2bb416d640 1 -- 172.21.15.16:0/3544453420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bac15a220 con 0x7f2bac0b7c20 2024-09-19T07:55:05.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.763+0000 7f2bb416d640 1 -- 172.21.15.16:0/3544453420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bac068ca0 con 0x7f2bac0b7c20 2024-09-19T07:55:05.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.764+0000 7f2ba27fc640 1 -- 172.21.15.16:0/3544453420 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2b9c0039d0 con 0x7f2bac0b7c20 2024-09-19T07:55:05.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.764+0000 7f2ba27fc640 1 --2- 172.21.15.16:0/3544453420 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2b9006c2b0 0x7f2b9006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:05.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.765+0000 7f2ba27fc640 1 -- 172.21.15.16:0/3544453420 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f2b9c08b1f0 con 0x7f2bac0b7c20 2024-09-19T07:55:05.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.765+0000 7f2bb1f09640 1 --2- 172.21.15.16:0/3544453420 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2b9006c2b0 0x7f2b9006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:05.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.765+0000 7f2bb1f09640 1 --2- 172.21.15.16:0/3544453420 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2b9006c2b0 0x7f2b9006e770 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8033000 tx=0x7f2ba8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.769+0000 7f2ba27fc640 1 -- 172.21.15.16:0/3544453420 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2b9c059920 con 0x7f2bac0b7c20 2024-09-19T07:55:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.944+0000 7f2bb416d640 1 -- 172.21.15.16:0/3544453420 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2bac0690c0 con 0x7f2b9006c2b0 2024-09-19T07:55:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.950+0000 7f2ba27fc640 1 -- 172.21.15.16:0/3544453420 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251139 (secure 0 0 0) 0x7f2bac0690c0 con 0x7f2b9006c2b0 2024-09-19T07:55:05.954 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:55:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.953+0000 7f2bb416d640 1 -- 172.21.15.16:0/3544453420 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2b9006c2b0 msgr2=0x7f2b9006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.953+0000 7f2bb416d640 1 --2- 172.21.15.16:0/3544453420 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2b9006c2b0 0x7f2b9006e770 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8033000 tx=0x7f2ba8008030 comp rx=0 tx=0).stop 2024-09-19T07:55:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.953+0000 7f2bb416d640 1 -- 172.21.15.16:0/3544453420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bac0b7c20 msgr2=0x7f2bac159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.953+0000 7f2bb416d640 1 --2- 172.21.15.16:0/3544453420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bac0b7c20 0x7f2bac159640 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00e540 tx=0x7f2b9c00ea00 comp rx=0 tx=0).stop 2024-09-19T07:55:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.953+0000 7f2bb416d640 1 -- 172.21.15.16:0/3544453420 shutdown_connections 2024-09-19T07:55:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.953+0000 7f2bb416d640 1 --2- 172.21.15.16:0/3544453420 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2b9006c2b0 0x7f2b9006e770 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.953+0000 7f2bb416d640 1 --2- 172.21.15.16:0/3544453420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bac15ad20 0x7f2bac15b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.953+0000 7f2bb416d640 1 --2- 172.21.15.16:0/3544453420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bac0b7c20 0x7f2bac159640 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.953+0000 7f2bb416d640 1 --2- 172.21.15.16:0/3544453420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b980039b0 0x7f2bac140e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.953+0000 7f2bb416d640 1 -- 172.21.15.16:0/3544453420 >> 172.21.15.16:0/3544453420 conn(0x7f2bac0b3ed0 msgr2=0x7f2bac160300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.954+0000 7f2bb416d640 1 -- 172.21.15.16:0/3544453420 shutdown_connections 2024-09-19T07:55:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:05.954+0000 7f2bb416d640 1 -- 172.21.15.16:0/3544453420 wait complete. 2024-09-19T07:55:05.965 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-09-19T07:55:04.629044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592666,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":199,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":46,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":765104,"kb_used_data":14772,"kb_used_omap":12,"kb_used_meta":750131,"kb_avail":754209616,"statfs":{"total":773094113280,"available":772310646784,"internally_reserved":0,"allocated":15126528,"data_stored":7631622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":768134704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2298,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000733"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972662+0000","last_change":"2024-09-19T07:55:02.977838+0000","last_active":"2024-09-19T07:55:03.972662+0000","last_peered":"2024-09-19T07:55:03.972662+0000","last_clean":"2024-09-19T07:55:03.972662+0000","last_became_active":"2024-09-19T07:55:02.977562+0000","last_became_peered":"2024-09-19T07:55:02.977562+0000","last_unstale":"2024-09-19T07:55:03.972662+0000","last_undegraded":"2024-09-19T07:55:03.972662+0000","last_fullsized":"2024-09-19T07:55:03.972662+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:18.348990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.994497+0000","last_change":"2024-09-19T07:55:02.988730+0000","last_active":"2024-09-19T07:55:02.994497+0000","last_peered":"2024-09-19T07:55:02.994497+0000","last_clean":"2024-09-19T07:55:02.994497+0000","last_became_active":"2024-09-19T07:55:02.988454+0000","last_became_peered":"2024-09-19T07:55:02.988454+0000","last_unstale":"2024-09-19T07:55:02.994497+0000","last_undegraded":"2024-09-19T07:55:02.994497+0000","last_fullsized":"2024-09-19T07:55:02.994497+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:55.150368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998989+0000","last_change":"2024-09-19T07:55:02.995513+0000","last_active":"2024-09-19T07:55:02.998989+0000","last_peered":"2024-09-19T07:55:02.998989+0000","last_clean":"2024-09-19T07:55:02.998989+0000","last_became_active":"2024-09-19T07:55:02.995162+0000","last_became_peered":"2024-09-19T07:55:02.995162+0000","last_unstale":"2024-09-19T07:55:02.998989+0000","last_undegraded":"2024-09-19T07:55:02.998989+0000","last_fullsized":"2024-09-19T07:55:02.998989+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:08.826237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972077+0000","last_change":"2024-09-19T07:55:02.991747+0000","last_active":"2024-09-19T07:55:03.972077+0000","last_peered":"2024-09-19T07:55:03.972077+0000","last_clean":"2024-09-19T07:55:03.972077+0000","last_became_active":"2024-09-19T07:55:02.991487+0000","last_became_peered":"2024-09-19T07:55:02.991487+0000","last_unstale":"2024-09-19T07:55:03.972077+0000","last_undegraded":"2024-09-19T07:55:03.972077+0000","last_fullsized":"2024-09-19T07:55:03.972077+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:00.867259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.978550+0000","last_change":"2024-09-19T07:55:02.974381+0000","last_active":"2024-09-19T07:55:02.978550+0000","last_peered":"2024-09-19T07:55:02.978550+0000","last_clean":"2024-09-19T07:55:02.978550+0000","last_became_active":"2024-09-19T07:55:02.974105+0000","last_became_peered":"2024-09-19T07:55:02.974105+0000","last_unstale":"2024-09-19T07:55:02.978550+0000","last_undegraded":"2024-09-19T07:55:02.978550+0000","last_fullsized":"2024-09-19T07:55:02.978550+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:08.519276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.980270+0000","last_change":"2024-09-19T07:55:02.975703+0000","last_active":"2024-09-19T07:55:02.980270+0000","last_peered":"2024-09-19T07:55:02.980270+0000","last_clean":"2024-09-19T07:55:02.980270+0000","last_became_active":"2024-09-19T07:55:02.975379+0000","last_became_peered":"2024-09-19T07:55:02.975379+0000","last_unstale":"2024-09-19T07:55:02.980270+0000","last_undegraded":"2024-09-19T07:55:02.980270+0000","last_fullsized":"2024-09-19T07:55:02.980270+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:11.846213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972184+0000","last_change":"2024-09-19T07:55:02.985662+0000","last_active":"2024-09-19T07:55:03.972184+0000","last_peered":"2024-09-19T07:55:03.972184+0000","last_clean":"2024-09-19T07:55:03.972184+0000","last_became_active":"2024-09-19T07:55:02.985378+0000","last_became_peered":"2024-09-19T07:55:02.985378+0000","last_unstale":"2024-09-19T07:55:03.972184+0000","last_undegraded":"2024-09-19T07:55:03.972184+0000","last_fullsized":"2024-09-19T07:55:03.972184+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:33.679405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.992880+0000","last_change":"2024-09-19T07:55:02.991159+0000","last_active":"2024-09-19T07:55:02.992880+0000","last_peered":"2024-09-19T07:55:02.992880+0000","last_clean":"2024-09-19T07:55:02.992880+0000","last_became_active":"2024-09-19T07:55:02.990631+0000","last_became_peered":"2024-09-19T07:55:02.990631+0000","last_unstale":"2024-09-19T07:55:02.992880+0000","last_undegraded":"2024-09-19T07:55:02.992880+0000","last_fullsized":"2024-09-19T07:55:02.992880+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:16.038904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998379+0000","last_change":"2024-09-19T07:55:02.994532+0000","last_active":"2024-09-19T07:55:02.998379+0000","last_peered":"2024-09-19T07:55:02.998379+0000","last_clean":"2024-09-19T07:55:02.998379+0000","last_became_active":"2024-09-19T07:55:02.994201+0000","last_became_peered":"2024-09-19T07:55:02.994201+0000","last_unstale":"2024-09-19T07:55:02.998379+0000","last_undegraded":"2024-09-19T07:55:02.998379+0000","last_fullsized":"2024-09-19T07:55:02.998379+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:12.950630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.984004+0000","last_change":"2024-09-19T07:55:02.981864+0000","last_active":"2024-09-19T07:55:02.984004+0000","last_peered":"2024-09-19T07:55:02.984004+0000","last_clean":"2024-09-19T07:55:02.984004+0000","last_became_active":"2024-09-19T07:55:02.981541+0000","last_became_peered":"2024-09-19T07:55:02.981541+0000","last_unstale":"2024-09-19T07:55:02.984004+0000","last_undegraded":"2024-09-19T07:55:02.984004+0000","last_fullsized":"2024-09-19T07:55:02.984004+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:22.255417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:01.959847+0000","last_change":"2024-09-19T07:55:01.959847+0000","last_active":"2024-09-19T07:55:01.959847+0000","last_peered":"2024-09-19T07:55:01.959847+0000","last_clean":"2024-09-19T07:55:01.959847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:01.959847+0000","last_undegraded":"2024-09-19T07:55:01.959847+0000","last_fullsized":"2024-09-19T07:55:01.959847+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972187+0000","last_change":"2024-09-19T07:55:02.989911+0000","last_active":"2024-09-19T07:55:03.972187+0000","last_peered":"2024-09-19T07:55:03.972187+0000","last_clean":"2024-09-19T07:55:03.972187+0000","last_became_active":"2024-09-19T07:55:02.989574+0000","last_became_peered":"2024-09-19T07:55:02.989574+0000","last_unstale":"2024-09-19T07:55:03.972187+0000","last_undegraded":"2024-09-19T07:55:03.972187+0000","last_fullsized":"2024-09-19T07:55:03.972187+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:20.107114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.985584+0000","last_change":"2024-09-19T07:55:02.983842+0000","last_active":"2024-09-19T07:55:02.985584+0000","last_peered":"2024-09-19T07:55:02.985584+0000","last_clean":"2024-09-19T07:55:02.985584+0000","last_became_active":"2024-09-19T07:55:02.983583+0000","last_became_peered":"2024-09-19T07:55:02.983583+0000","last_unstale":"2024-09-19T07:55:02.985584+0000","last_undegraded":"2024-09-19T07:55:02.985584+0000","last_fullsized":"2024-09-19T07:55:02.985584+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:58.534832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.971850+0000","last_change":"2024-09-19T07:55:02.991151+0000","last_active":"2024-09-19T07:55:03.971850+0000","last_peered":"2024-09-19T07:55:03.971850+0000","last_clean":"2024-09-19T07:55:03.971850+0000","last_became_active":"2024-09-19T07:55:02.990791+0000","last_became_peered":"2024-09-19T07:55:02.990791+0000","last_unstale":"2024-09-19T07:55:03.971850+0000","last_undegraded":"2024-09-19T07:55:03.971850+0000","last_fullsized":"2024-09-19T07:55:03.971850+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:16.038904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.997603+0000","last_change":"2024-09-19T07:55:02.993711+0000","last_active":"2024-09-19T07:55:02.997603+0000","last_peered":"2024-09-19T07:55:02.997603+0000","last_clean":"2024-09-19T07:55:02.997603+0000","last_became_active":"2024-09-19T07:55:02.993391+0000","last_became_peered":"2024-09-19T07:55:02.993391+0000","last_unstale":"2024-09-19T07:55:02.997603+0000","last_undegraded":"2024-09-19T07:55:02.997603+0000","last_fullsized":"2024-09-19T07:55:02.997603+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:53:11.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.981640+0000","last_change":"2024-09-19T07:55:02.977783+0000","last_active":"2024-09-19T07:55:02.981640+0000","last_peered":"2024-09-19T07:55:02.981640+0000","last_clean":"2024-09-19T07:55:02.981640+0000","last_became_active":"2024-09-19T07:55:02.977494+0000","last_became_peered":"2024-09-19T07:55:02.977494+0000","last_unstale":"2024-09-19T07:55:02.981640+0000","last_undegraded":"2024-09-19T07:55:02.981640+0000","last_fullsized":"2024-09-19T07:55:02.981640+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:42.284168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972758+0000","last_change":"2024-09-19T07:55:02.989847+0000","last_active":"2024-09-19T07:55:03.972758+0000","last_peered":"2024-09-19T07:55:03.972758+0000","last_clean":"2024-09-19T07:55:03.972758+0000","last_became_active":"2024-09-19T07:55:02.989566+0000","last_became_peered":"2024-09-19T07:55:02.989566+0000","last_unstale":"2024-09-19T07:55:03.972758+0000","last_undegraded":"2024-09-19T07:55:03.972758+0000","last_fullsized":"2024-09-19T07:55:03.972758+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:57:07.989977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:01.959847+0000","last_change":"2024-09-19T07:55:01.959847+0000","last_active":"2024-09-19T07:55:01.959847+0000","last_peered":"2024-09-19T07:55:01.959847+0000","last_clean":"2024-09-19T07:55:01.959847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:01.959847+0000","last_undegraded":"2024-09-19T07:55:01.959847+0000","last_fullsized":"2024-09-19T07:55:01.959847+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983370+0000","last_change":"2024-09-19T07:55:02.988233+0000","last_active":"2024-09-19T07:55:03.983370+0000","last_peered":"2024-09-19T07:55:03.983370+0000","last_clean":"2024-09-19T07:55:03.983370+0000","last_became_active":"2024-09-19T07:55:02.987586+0000","last_became_peered":"2024-09-19T07:55:02.987586+0000","last_unstale":"2024-09-19T07:55:03.983370+0000","last_undegraded":"2024-09-19T07:55:03.983370+0000","last_fullsized":"2024-09-19T07:55:03.983370+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:37.091603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.989123+0000","last_change":"2024-09-19T07:55:02.984872+0000","last_active":"2024-09-19T07:55:02.989123+0000","last_peered":"2024-09-19T07:55:02.989123+0000","last_clean":"2024-09-19T07:55:02.989123+0000","last_became_active":"2024-09-19T07:55:02.984588+0000","last_became_peered":"2024-09-19T07:55:02.984588+0000","last_unstale":"2024-09-19T07:55:02.989123+0000","last_undegraded":"2024-09-19T07:55:02.989123+0000","last_fullsized":"2024-09-19T07:55:02.989123+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:37.091603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.986844+0000","last_change":"2024-09-19T07:55:02.985444+0000","last_active":"2024-09-19T07:55:02.986844+0000","last_peered":"2024-09-19T07:55:02.986844+0000","last_clean":"2024-09-19T07:55:02.986844+0000","last_became_active":"2024-09-19T07:55:02.985161+0000","last_became_peered":"2024-09-19T07:55:02.985161+0000","last_unstale":"2024-09-19T07:55:02.986844+0000","last_undegraded":"2024-09-19T07:55:02.986844+0000","last_fullsized":"2024-09-19T07:55:02.986844+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:19.760108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972782+0000","last_change":"2024-09-19T07:55:02.980246+0000","last_active":"2024-09-19T07:55:03.972782+0000","last_peered":"2024-09-19T07:55:03.972782+0000","last_clean":"2024-09-19T07:55:03.972782+0000","last_became_active":"2024-09-19T07:55:02.979970+0000","last_became_peered":"2024-09-19T07:55:02.979970+0000","last_unstale":"2024-09-19T07:55:03.972782+0000","last_undegraded":"2024-09-19T07:55:03.972782+0000","last_fullsized":"2024-09-19T07:55:03.972782+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:19.717109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.986210+0000","last_change":"2024-09-19T07:55:02.979612+0000","last_active":"2024-09-19T07:55:02.986210+0000","last_peered":"2024-09-19T07:55:02.986210+0000","last_clean":"2024-09-19T07:55:02.986210+0000","last_became_active":"2024-09-19T07:55:02.978902+0000","last_became_peered":"2024-09-19T07:55:02.978902+0000","last_unstale":"2024-09-19T07:55:02.986210+0000","last_undegraded":"2024-09-19T07:55:02.986210+0000","last_fullsized":"2024-09-19T07:55:02.986210+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:57.305718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:01.959847+0000","last_change":"2024-09-19T07:55:01.959847+0000","last_active":"2024-09-19T07:55:01.959847+0000","last_peered":"2024-09-19T07:55:01.959847+0000","last_clean":"2024-09-19T07:55:01.959847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:01.959847+0000","last_undegraded":"2024-09-19T07:55:01.959847+0000","last_fullsized":"2024-09-19T07:55:01.959847+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972885+0000","last_change":"2024-09-19T07:55:02.981645+0000","last_active":"2024-09-19T07:55:03.972885+0000","last_peered":"2024-09-19T07:55:03.972885+0000","last_clean":"2024-09-19T07:55:03.972885+0000","last_became_active":"2024-09-19T07:55:02.981377+0000","last_became_peered":"2024-09-19T07:55:02.981377+0000","last_unstale":"2024-09-19T07:55:03.972885+0000","last_undegraded":"2024-09-19T07:55:03.972885+0000","last_fullsized":"2024-09-19T07:55:03.972885+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:28.867264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972976+0000","last_change":"2024-09-19T07:55:02.991055+0000","last_active":"2024-09-19T07:55:03.972976+0000","last_peered":"2024-09-19T07:55:03.972976+0000","last_clean":"2024-09-19T07:55:03.972976+0000","last_became_active":"2024-09-19T07:55:02.990517+0000","last_became_peered":"2024-09-19T07:55:02.990517+0000","last_unstale":"2024-09-19T07:55:03.972976+0000","last_undegraded":"2024-09-19T07:55:03.972976+0000","last_fullsized":"2024-09-19T07:55:03.972976+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:52.586504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.999462+0000","last_change":"2024-09-19T07:55:02.998488+0000","last_active":"2024-09-19T07:55:02.999462+0000","last_peered":"2024-09-19T07:55:02.999462+0000","last_clean":"2024-09-19T07:55:02.999462+0000","last_became_active":"2024-09-19T07:55:02.998192+0000","last_became_peered":"2024-09-19T07:55:02.998192+0000","last_unstale":"2024-09-19T07:55:02.999462+0000","last_undegraded":"2024-09-19T07:55:02.999462+0000","last_fullsized":"2024-09-19T07:55:02.999462+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:20.318482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:01.959847+0000","last_change":"2024-09-19T07:55:01.959847+0000","last_active":"2024-09-19T07:55:01.959847+0000","last_peered":"2024-09-19T07:55:01.959847+0000","last_clean":"2024-09-19T07:55:01.959847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:01.959847+0000","last_undegraded":"2024-09-19T07:55:01.959847+0000","last_fullsized":"2024-09-19T07:55:01.959847+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.987062+0000","last_change":"2024-09-19T07:55:02.985372+0000","last_active":"2024-09-19T07:55:02.987062+0000","last_peered":"2024-09-19T07:55:02.987062+0000","last_clean":"2024-09-19T07:55:02.987062+0000","last_became_active":"2024-09-19T07:55:02.985109+0000","last_became_peered":"2024-09-19T07:55:02.985109+0000","last_unstale":"2024-09-19T07:55:02.987062+0000","last_undegraded":"2024-09-19T07:55:02.987062+0000","last_fullsized":"2024-09-19T07:55:02.987062+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:59.971722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:01.959847+0000","last_change":"2024-09-19T07:55:01.959847+0000","last_active":"2024-09-19T07:55:01.959847+0000","last_peered":"2024-09-19T07:55:01.959847+0000","last_clean":"2024-09-19T07:55:01.959847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:01.959847+0000","last_undegraded":"2024-09-19T07:55:01.959847+0000","last_fullsized":"2024-09-19T07:55:01.959847+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.981739+0000","last_change":"2024-09-19T07:55:02.978386+0000","last_active":"2024-09-19T07:55:02.981739+0000","last_peered":"2024-09-19T07:55:02.981739+0000","last_clean":"2024-09-19T07:55:02.981739+0000","last_became_active":"2024-09-19T07:55:02.978134+0000","last_became_peered":"2024-09-19T07:55:02.978134+0000","last_unstale":"2024-09-19T07:55:02.981739+0000","last_undegraded":"2024-09-19T07:55:02.981739+0000","last_fullsized":"2024-09-19T07:55:02.981739+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:06.762003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.992859+0000","last_change":"2024-09-19T07:55:02.990915+0000","last_active":"2024-09-19T07:55:02.992859+0000","last_peered":"2024-09-19T07:55:02.992859+0000","last_clean":"2024-09-19T07:55:02.992859+0000","last_became_active":"2024-09-19T07:55:02.990273+0000","last_became_peered":"2024-09-19T07:55:02.990273+0000","last_unstale":"2024-09-19T07:55:02.992859+0000","last_undegraded":"2024-09-19T07:55:02.992859+0000","last_fullsized":"2024-09-19T07:55:02.992859+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:25.070703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998804+0000","last_change":"2024-09-19T07:55:02.994240+0000","last_active":"2024-09-19T07:55:02.998804+0000","last_peered":"2024-09-19T07:55:02.998804+0000","last_clean":"2024-09-19T07:55:02.998804+0000","last_became_active":"2024-09-19T07:55:02.993966+0000","last_became_peered":"2024-09-19T07:55:02.993966+0000","last_unstale":"2024-09-19T07:55:02.998804+0000","last_undegraded":"2024-09-19T07:55:02.998804+0000","last_fullsized":"2024-09-19T07:55:02.998804+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:18.497803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"139'4","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000708+0000","last_change":"2024-09-19T07:55:01.982819+0000","last_active":"2024-09-19T07:55:03.000708+0000","last_peered":"2024-09-19T07:55:03.000708+0000","last_clean":"2024-09-19T07:55:03.000708+0000","last_became_active":"2024-09-19T07:55:01.982455+0000","last_became_peered":"2024-09-19T07:55:01.982455+0000","last_unstale":"2024-09-19T07:55:03.000708+0000","last_undegraded":"2024-09-19T07:55:03.000708+0000","last_fullsized":"2024-09-19T07:55:03.000708+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:01.483855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.995361+0000","last_change":"2024-09-19T07:55:02.988748+0000","last_active":"2024-09-19T07:55:02.995361+0000","last_peered":"2024-09-19T07:55:02.995361+0000","last_clean":"2024-09-19T07:55:02.995361+0000","last_became_active":"2024-09-19T07:55:02.988417+0000","last_became_peered":"2024-09-19T07:55:02.988417+0000","last_unstale":"2024-09-19T07:55:02.995361+0000","last_undegraded":"2024-09-19T07:55:02.995361+0000","last_fullsized":"2024-09-19T07:55:02.995361+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:11.124464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983330+0000","last_change":"2024-09-19T07:55:01.986317+0000","last_active":"2024-09-19T07:55:03.983330+0000","last_peered":"2024-09-19T07:55:03.983330+0000","last_clean":"2024-09-19T07:55:03.983330+0000","last_became_active":"2024-09-19T07:55:01.986050+0000","last_became_peered":"2024-09-19T07:55:01.986050+0000","last_unstale":"2024-09-19T07:55:03.983330+0000","last_undegraded":"2024-09-19T07:55:03.983330+0000","last_fullsized":"2024-09-19T07:55:03.983330+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:20.699942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.988705+0000","last_change":"2024-09-19T07:55:02.988084+0000","last_active":"2024-09-19T07:55:02.988705+0000","last_peered":"2024-09-19T07:55:02.988705+0000","last_clean":"2024-09-19T07:55:02.988705+0000","last_became_active":"2024-09-19T07:55:02.987819+0000","last_became_peered":"2024-09-19T07:55:02.987819+0000","last_unstale":"2024-09-19T07:55:02.988705+0000","last_undegraded":"2024-09-19T07:55:02.988705+0000","last_fullsized":"2024-09-19T07:55:02.988705+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:34.829241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983598+0000","last_change":"2024-09-19T07:55:01.983959+0000","last_active":"2024-09-19T07:55:03.983598+0000","last_peered":"2024-09-19T07:55:03.983598+0000","last_clean":"2024-09-19T07:55:03.983598+0000","last_became_active":"2024-09-19T07:55:01.983682+0000","last_became_peered":"2024-09-19T07:55:01.983682+0000","last_unstale":"2024-09-19T07:55:03.983598+0000","last_undegraded":"2024-09-19T07:55:03.983598+0000","last_fullsized":"2024-09-19T07:55:03.983598+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:17.343679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983514+0000","last_change":"2024-09-19T07:55:02.988272+0000","last_active":"2024-09-19T07:55:03.983514+0000","last_peered":"2024-09-19T07:55:03.983514+0000","last_clean":"2024-09-19T07:55:03.983514+0000","last_became_active":"2024-09-19T07:55:02.987972+0000","last_became_peered":"2024-09-19T07:55:02.987972+0000","last_unstale":"2024-09-19T07:55:03.983514+0000","last_undegraded":"2024-09-19T07:55:03.983514+0000","last_fullsized":"2024-09-19T07:55:03.983514+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:55.597883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"139'2","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000148+0000","last_change":"2024-09-19T07:55:01.975504+0000","last_active":"2024-09-19T07:55:03.000148+0000","last_peered":"2024-09-19T07:55:03.000148+0000","last_clean":"2024-09-19T07:55:03.000148+0000","last_became_active":"2024-09-19T07:55:01.975216+0000","last_became_peered":"2024-09-19T07:55:01.975216+0000","last_unstale":"2024-09-19T07:55:03.000148+0000","last_undegraded":"2024-09-19T07:55:03.000148+0000","last_fullsized":"2024-09-19T07:55:03.000148+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:47.640841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.990742+0000","last_change":"2024-09-19T07:55:02.990235+0000","last_active":"2024-09-19T07:55:02.990742+0000","last_peered":"2024-09-19T07:55:02.990742+0000","last_clean":"2024-09-19T07:55:02.990742+0000","last_became_active":"2024-09-19T07:55:02.989920+0000","last_became_peered":"2024-09-19T07:55:02.989920+0000","last_unstale":"2024-09-19T07:55:02.990742+0000","last_undegraded":"2024-09-19T07:55:02.990742+0000","last_fullsized":"2024-09-19T07:55:02.990742+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:55.597883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"139'1","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.997333+0000","last_change":"2024-09-19T07:55:01.984133+0000","last_active":"2024-09-19T07:55:02.997333+0000","last_peered":"2024-09-19T07:55:02.997333+0000","last_clean":"2024-09-19T07:55:02.997333+0000","last_became_active":"2024-09-19T07:55:01.983829+0000","last_became_peered":"2024-09-19T07:55:01.983829+0000","last_unstale":"2024-09-19T07:55:02.997333+0000","last_undegraded":"2024-09-19T07:55:02.997333+0000","last_fullsized":"2024-09-19T07:55:02.997333+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:08.327895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.971878+0000","last_change":"2024-09-19T07:55:02.990053+0000","last_active":"2024-09-19T07:55:03.971878+0000","last_peered":"2024-09-19T07:55:03.971878+0000","last_clean":"2024-09-19T07:55:03.971878+0000","last_became_active":"2024-09-19T07:55:02.987609+0000","last_became_peered":"2024-09-19T07:55:02.987609+0000","last_unstale":"2024-09-19T07:55:03.971878+0000","last_undegraded":"2024-09-19T07:55:03.971878+0000","last_fullsized":"2024-09-19T07:55:03.971878+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:02:28.317679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"139'1","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998584+0000","last_change":"2024-09-19T07:55:01.984540+0000","last_active":"2024-09-19T07:55:02.998584+0000","last_peered":"2024-09-19T07:55:02.998584+0000","last_clean":"2024-09-19T07:55:02.998584+0000","last_became_active":"2024-09-19T07:55:01.984218+0000","last_became_peered":"2024-09-19T07:55:01.984218+0000","last_unstale":"2024-09-19T07:55:02.998584+0000","last_undegraded":"2024-09-19T07:55:02.998584+0000","last_fullsized":"2024-09-19T07:55:02.998584+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:59:17.842847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.987109+0000","last_change":"2024-09-19T07:55:02.983739+0000","last_active":"2024-09-19T07:55:02.987109+0000","last_peered":"2024-09-19T07:55:02.987109+0000","last_clean":"2024-09-19T07:55:02.987109+0000","last_became_active":"2024-09-19T07:55:02.983448+0000","last_became_peered":"2024-09-19T07:55:02.983448+0000","last_unstale":"2024-09-19T07:55:02.987109+0000","last_undegraded":"2024-09-19T07:55:02.987109+0000","last_fullsized":"2024-09-19T07:55:02.987109+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:50.858294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972280+0000","last_change":"2024-09-19T07:55:01.978901+0000","last_active":"2024-09-19T07:55:03.972280+0000","last_peered":"2024-09-19T07:55:03.972280+0000","last_clean":"2024-09-19T07:55:03.972280+0000","last_became_active":"2024-09-19T07:55:01.978399+0000","last_became_peered":"2024-09-19T07:55:01.978399+0000","last_unstale":"2024-09-19T07:55:03.972280+0000","last_undegraded":"2024-09-19T07:55:03.972280+0000","last_fullsized":"2024-09-19T07:55:03.972280+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:11.363759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972118+0000","last_change":"2024-09-19T07:55:02.978549+0000","last_active":"2024-09-19T07:55:03.972118+0000","last_peered":"2024-09-19T07:55:03.972118+0000","last_clean":"2024-09-19T07:55:03.972118+0000","last_became_active":"2024-09-19T07:55:02.978270+0000","last_became_peered":"2024-09-19T07:55:02.978270+0000","last_unstale":"2024-09-19T07:55:03.972118+0000","last_undegraded":"2024-09-19T07:55:03.972118+0000","last_fullsized":"2024-09-19T07:55:03.972118+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:21.705254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.969267+0000","last_change":"2024-09-19T07:55:01.983833+0000","last_active":"2024-09-19T07:55:02.969267+0000","last_peered":"2024-09-19T07:55:02.969267+0000","last_clean":"2024-09-19T07:55:02.969267+0000","last_became_active":"2024-09-19T07:55:01.983553+0000","last_became_peered":"2024-09-19T07:55:01.983553+0000","last_unstale":"2024-09-19T07:55:02.969267+0000","last_undegraded":"2024-09-19T07:55:02.969267+0000","last_fullsized":"2024-09-19T07:55:02.969267+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:02.576442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.986319+0000","last_change":"2024-09-19T07:55:02.983084+0000","last_active":"2024-09-19T07:55:02.986319+0000","last_peered":"2024-09-19T07:55:02.986319+0000","last_clean":"2024-09-19T07:55:02.986319+0000","last_became_active":"2024-09-19T07:55:02.982793+0000","last_became_peered":"2024-09-19T07:55:02.982793+0000","last_unstale":"2024-09-19T07:55:02.986319+0000","last_undegraded":"2024-09-19T07:55:02.986319+0000","last_fullsized":"2024-09-19T07:55:02.986319+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:39:22.911029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983489+0000","last_change":"2024-09-19T07:55:01.980547+0000","last_active":"2024-09-19T07:55:03.983489+0000","last_peered":"2024-09-19T07:55:03.983489+0000","last_clean":"2024-09-19T07:55:03.983489+0000","last_became_active":"2024-09-19T07:55:01.980253+0000","last_became_peered":"2024-09-19T07:55:01.980253+0000","last_unstale":"2024-09-19T07:55:03.983489+0000","last_undegraded":"2024-09-19T07:55:03.983489+0000","last_fullsized":"2024-09-19T07:55:03.983489+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:32.674094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:01.959847+0000","last_change":"2024-09-19T07:55:01.959847+0000","last_active":"2024-09-19T07:55:01.959847+0000","last_peered":"2024-09-19T07:55:01.959847+0000","last_clean":"2024-09-19T07:55:01.959847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:01.959847+0000","last_undegraded":"2024-09-19T07:55:01.959847+0000","last_fullsized":"2024-09-19T07:55:01.959847+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.973223+0000","last_change":"2024-09-19T07:55:01.980870+0000","last_active":"2024-09-19T07:55:03.973223+0000","last_peered":"2024-09-19T07:55:03.973223+0000","last_clean":"2024-09-19T07:55:03.973223+0000","last_became_active":"2024-09-19T07:55:01.980569+0000","last_became_peered":"2024-09-19T07:55:01.980569+0000","last_unstale":"2024-09-19T07:55:03.973223+0000","last_undegraded":"2024-09-19T07:55:03.973223+0000","last_fullsized":"2024-09-19T07:55:03.973223+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:56:13.952641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:01.959847+0000","last_change":"2024-09-19T07:55:01.959847+0000","last_active":"2024-09-19T07:55:01.959847+0000","last_peered":"2024-09-19T07:55:01.959847+0000","last_clean":"2024-09-19T07:55:01.959847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:01.959847+0000","last_undegraded":"2024-09-19T07:55:01.959847+0000","last_fullsized":"2024-09-19T07:55:01.959847+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-19T07:55:00.965215+0000","last_change":"2024-09-19T07:55:00.965215+0000","last_active":"2024-09-19T07:55:00.954535+0000","last_peered":"2024-09-19T07:55:00.954535+0000","last_clean":"2024-09-19T07:55:00.954535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:00.965215+0000","last_undegraded":"2024-09-19T07:55:00.965215+0000","last_fullsized":"2024-09-19T07:55:00.965215+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.986247+0000","last_change":"2024-09-19T07:55:02.980794+0000","last_active":"2024-09-19T07:55:02.986247+0000","last_peered":"2024-09-19T07:55:02.986247+0000","last_clean":"2024-09-19T07:55:02.986247+0000","last_became_active":"2024-09-19T07:55:02.980530+0000","last_became_peered":"2024-09-19T07:55:02.980530+0000","last_unstale":"2024-09-19T07:55:02.986247+0000","last_undegraded":"2024-09-19T07:55:02.986247+0000","last_fullsized":"2024-09-19T07:55:02.986247+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:14.838884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"139'2","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000534+0000","last_change":"2024-09-19T07:55:01.984318+0000","last_active":"2024-09-19T07:55:03.000534+0000","last_peered":"2024-09-19T07:55:03.000534+0000","last_clean":"2024-09-19T07:55:03.000534+0000","last_became_active":"2024-09-19T07:55:01.983325+0000","last_became_peered":"2024-09-19T07:55:01.983325+0000","last_unstale":"2024-09-19T07:55:03.000534+0000","last_undegraded":"2024-09-19T07:55:03.000534+0000","last_fullsized":"2024-09-19T07:55:03.000534+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:36.086291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.999112+0000","last_change":"2024-09-19T07:55:02.996430+0000","last_active":"2024-09-19T07:55:02.999112+0000","last_peered":"2024-09-19T07:55:02.999112+0000","last_clean":"2024-09-19T07:55:02.999112+0000","last_became_active":"2024-09-19T07:55:02.996148+0000","last_became_peered":"2024-09-19T07:55:02.996148+0000","last_unstale":"2024-09-19T07:55:02.999112+0000","last_undegraded":"2024-09-19T07:55:02.999112+0000","last_fullsized":"2024-09-19T07:55:02.999112+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:12.728223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972021+0000","last_change":"2024-09-19T07:55:01.988068+0000","last_active":"2024-09-19T07:55:03.972021+0000","last_peered":"2024-09-19T07:55:03.972021+0000","last_clean":"2024-09-19T07:55:03.972021+0000","last_became_active":"2024-09-19T07:55:01.987761+0000","last_became_peered":"2024-09-19T07:55:01.987761+0000","last_unstale":"2024-09-19T07:55:03.972021+0000","last_undegraded":"2024-09-19T07:55:03.972021+0000","last_fullsized":"2024-09-19T07:55:03.972021+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:56.300407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.994831+0000","last_change":"2024-09-19T07:55:02.989642+0000","last_active":"2024-09-19T07:55:02.994831+0000","last_peered":"2024-09-19T07:55:02.994831+0000","last_clean":"2024-09-19T07:55:02.994831+0000","last_became_active":"2024-09-19T07:55:02.989315+0000","last_became_peered":"2024-09-19T07:55:02.989315+0000","last_unstale":"2024-09-19T07:55:02.994831+0000","last_undegraded":"2024-09-19T07:55:02.994831+0000","last_fullsized":"2024-09-19T07:55:02.994831+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:29.434870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972162+0000","last_change":"2024-09-19T07:55:01.983411+0000","last_active":"2024-09-19T07:55:03.972162+0000","last_peered":"2024-09-19T07:55:03.972162+0000","last_clean":"2024-09-19T07:55:03.972162+0000","last_became_active":"2024-09-19T07:55:01.982413+0000","last_became_peered":"2024-09-19T07:55:01.982413+0000","last_unstale":"2024-09-19T07:55:03.972162+0000","last_undegraded":"2024-09-19T07:55:03.972162+0000","last_fullsized":"2024-09-19T07:55:03.972162+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:58.316347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.985547+0000","last_change":"2024-09-19T07:55:02.983361+0000","last_active":"2024-09-19T07:55:02.985547+0000","last_peered":"2024-09-19T07:55:02.985547+0000","last_clean":"2024-09-19T07:55:02.985547+0000","last_became_active":"2024-09-19T07:55:02.983032+0000","last_became_peered":"2024-09-19T07:55:02.983032+0000","last_unstale":"2024-09-19T07:55:02.985547+0000","last_undegraded":"2024-09-19T07:55:02.985547+0000","last_fullsized":"2024-09-19T07:55:02.985547+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:56:14.957952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"139'8","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.971849+0000","last_change":"2024-09-19T07:55:01.978831+0000","last_active":"2024-09-19T07:55:03.971849+0000","last_peered":"2024-09-19T07:55:03.971849+0000","last_clean":"2024-09-19T07:55:03.971849+0000","last_became_active":"2024-09-19T07:55:01.978370+0000","last_became_peered":"2024-09-19T07:55:01.978370+0000","last_unstale":"2024-09-19T07:55:03.971849+0000","last_undegraded":"2024-09-19T07:55:03.971849+0000","last_fullsized":"2024-09-19T07:55:03.971849+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:47.640841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983367+0000","last_change":"2024-09-19T07:55:02.998722+0000","last_active":"2024-09-19T07:55:03.983367+0000","last_peered":"2024-09-19T07:55:03.983367+0000","last_clean":"2024-09-19T07:55:03.983367+0000","last_became_active":"2024-09-19T07:55:02.998420+0000","last_became_peered":"2024-09-19T07:55:02.998420+0000","last_unstale":"2024-09-19T07:55:03.983367+0000","last_undegraded":"2024-09-19T07:55:03.983367+0000","last_fullsized":"2024-09-19T07:55:03.983367+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:02.489166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-19T07:55:00.965654+0000","last_change":"2024-09-19T07:55:00.965654+0000","last_active":"2024-09-19T07:55:00.954535+0000","last_peered":"2024-09-19T07:55:00.954535+0000","last_clean":"2024-09-19T07:55:00.954535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:00.965654+0000","last_undegraded":"2024-09-19T07:55:00.965654+0000","last_fullsized":"2024-09-19T07:55:00.965654+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983513+0000","last_change":"2024-09-19T07:55:02.988579+0000","last_active":"2024-09-19T07:55:03.983513+0000","last_peered":"2024-09-19T07:55:03.983513+0000","last_clean":"2024-09-19T07:55:03.983513+0000","last_became_active":"2024-09-19T07:55:02.987958+0000","last_became_peered":"2024-09-19T07:55:02.987958+0000","last_unstale":"2024-09-19T07:55:03.983513+0000","last_undegraded":"2024-09-19T07:55:03.983513+0000","last_fullsized":"2024-09-19T07:55:03.983513+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:08.519276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983438+0000","last_change":"2024-09-19T07:55:01.981723+0000","last_active":"2024-09-19T07:55:03.983438+0000","last_peered":"2024-09-19T07:55:03.983438+0000","last_clean":"2024-09-19T07:55:03.983438+0000","last_became_active":"2024-09-19T07:55:01.981465+0000","last_became_peered":"2024-09-19T07:55:01.981465+0000","last_unstale":"2024-09-19T07:55:03.983438+0000","last_undegraded":"2024-09-19T07:55:03.983438+0000","last_fullsized":"2024-09-19T07:55:03.983438+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:12.838363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.973123+0000","last_change":"2024-09-19T07:55:02.979256+0000","last_active":"2024-09-19T07:55:03.973123+0000","last_peered":"2024-09-19T07:55:03.973123+0000","last_clean":"2024-09-19T07:55:03.973123+0000","last_became_active":"2024-09-19T07:55:02.978921+0000","last_became_peered":"2024-09-19T07:55:02.978921+0000","last_unstale":"2024-09-19T07:55:03.973123+0000","last_undegraded":"2024-09-19T07:55:03.973123+0000","last_fullsized":"2024-09-19T07:55:03.973123+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:51.978120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"139'1","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000384+0000","last_change":"2024-09-19T07:55:01.982720+0000","last_active":"2024-09-19T07:55:03.000384+0000","last_peered":"2024-09-19T07:55:03.000384+0000","last_clean":"2024-09-19T07:55:03.000384+0000","last_became_active":"2024-09-19T07:55:01.982277+0000","last_became_peered":"2024-09-19T07:55:01.982277+0000","last_unstale":"2024-09-19T07:55:03.000384+0000","last_undegraded":"2024-09-19T07:55:03.000384+0000","last_fullsized":"2024-09-19T07:55:03.000384+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:59.197262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972931+0000","last_change":"2024-09-19T07:55:02.989059+0000","last_active":"2024-09-19T07:55:03.972931+0000","last_peered":"2024-09-19T07:55:03.972931+0000","last_clean":"2024-09-19T07:55:03.972931+0000","last_became_active":"2024-09-19T07:55:02.988577+0000","last_became_peered":"2024-09-19T07:55:02.988577+0000","last_unstale":"2024-09-19T07:55:03.972931+0000","last_undegraded":"2024-09-19T07:55:03.972931+0000","last_fullsized":"2024-09-19T07:55:03.972931+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:49.502676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972169+0000","last_change":"2024-09-19T07:55:01.988779+0000","last_active":"2024-09-19T07:55:03.972169+0000","last_peered":"2024-09-19T07:55:03.972169+0000","last_clean":"2024-09-19T07:55:03.972169+0000","last_became_active":"2024-09-19T07:55:01.988419+0000","last_became_peered":"2024-09-19T07:55:01.988419+0000","last_unstale":"2024-09-19T07:55:03.972169+0000","last_undegraded":"2024-09-19T07:55:03.972169+0000","last_fullsized":"2024-09-19T07:55:03.972169+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:59.785587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.982978+0000","last_change":"2024-09-19T07:55:02.987148+0000","last_active":"2024-09-19T07:55:03.982978+0000","last_peered":"2024-09-19T07:55:03.982978+0000","last_clean":"2024-09-19T07:55:03.982978+0000","last_became_active":"2024-09-19T07:55:02.986889+0000","last_became_peered":"2024-09-19T07:55:02.986889+0000","last_unstale":"2024-09-19T07:55:03.982978+0000","last_undegraded":"2024-09-19T07:55:03.982978+0000","last_fullsized":"2024-09-19T07:55:03.982978+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:26:18.944316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"139'2","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.997309+0000","last_change":"2024-09-19T07:55:01.986115+0000","last_active":"2024-09-19T07:55:02.997309+0000","last_peered":"2024-09-19T07:55:02.997309+0000","last_clean":"2024-09-19T07:55:02.997309+0000","last_became_active":"2024-09-19T07:55:01.985827+0000","last_became_peered":"2024-09-19T07:55:01.985827+0000","last_unstale":"2024-09-19T07:55:02.997309+0000","last_undegraded":"2024-09-19T07:55:02.997309+0000","last_fullsized":"2024-09-19T07:55:02.997309+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:01.483855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.989398+0000","last_change":"2024-09-19T07:55:02.987365+0000","last_active":"2024-09-19T07:55:02.989398+0000","last_peered":"2024-09-19T07:55:02.989398+0000","last_clean":"2024-09-19T07:55:02.989398+0000","last_became_active":"2024-09-19T07:55:02.986480+0000","last_became_peered":"2024-09-19T07:55:02.986480+0000","last_unstale":"2024-09-19T07:55:02.989398+0000","last_undegraded":"2024-09-19T07:55:02.989398+0000","last_fullsized":"2024-09-19T07:55:02.989398+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:32.859864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.972920+0000","last_change":"2024-09-19T07:55:01.981717+0000","last_active":"2024-09-19T07:55:02.972920+0000","last_peered":"2024-09-19T07:55:02.972920+0000","last_clean":"2024-09-19T07:55:02.972920+0000","last_became_active":"2024-09-19T07:55:01.981004+0000","last_became_peered":"2024-09-19T07:55:01.981004+0000","last_unstale":"2024-09-19T07:55:02.972920+0000","last_undegraded":"2024-09-19T07:55:02.972920+0000","last_fullsized":"2024-09-19T07:55:02.972920+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:58.316347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983112+0000","last_change":"2024-09-19T07:55:02.992093+0000","last_active":"2024-09-19T07:55:03.983112+0000","last_peered":"2024-09-19T07:55:03.983112+0000","last_clean":"2024-09-19T07:55:03.983112+0000","last_became_active":"2024-09-19T07:55:02.991829+0000","last_became_peered":"2024-09-19T07:55:02.991829+0000","last_unstale":"2024-09-19T07:55:03.983112+0000","last_undegraded":"2024-09-19T07:55:03.983112+0000","last_fullsized":"2024-09-19T07:55:03.983112+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:59:18.848159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.974687+0000","last_change":"2024-09-19T07:55:01.983244+0000","last_active":"2024-09-19T07:55:02.974687+0000","last_peered":"2024-09-19T07:55:02.974687+0000","last_clean":"2024-09-19T07:55:02.974687+0000","last_became_active":"2024-09-19T07:55:01.982960+0000","last_became_peered":"2024-09-19T07:55:01.982960+0000","last_unstale":"2024-09-19T07:55:02.974687+0000","last_undegraded":"2024-09-19T07:55:02.974687+0000","last_fullsized":"2024-09-19T07:55:02.974687+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:26:17.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998969+0000","last_change":"2024-09-19T07:55:02.993715+0000","last_active":"2024-09-19T07:55:02.998969+0000","last_peered":"2024-09-19T07:55:02.998969+0000","last_clean":"2024-09-19T07:55:02.998969+0000","last_became_active":"2024-09-19T07:55:02.993409+0000","last_became_peered":"2024-09-19T07:55:02.993409+0000","last_unstale":"2024-09-19T07:55:02.998969+0000","last_undegraded":"2024-09-19T07:55:02.998969+0000","last_fullsized":"2024-09-19T07:55:02.998969+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:13.897172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.974976+0000","last_change":"2024-09-19T07:55:01.982335+0000","last_active":"2024-09-19T07:55:02.974976+0000","last_peered":"2024-09-19T07:55:02.974976+0000","last_clean":"2024-09-19T07:55:02.974976+0000","last_became_active":"2024-09-19T07:55:01.982052+0000","last_became_peered":"2024-09-19T07:55:01.982052+0000","last_unstale":"2024-09-19T07:55:02.974976+0000","last_undegraded":"2024-09-19T07:55:02.974976+0000","last_fullsized":"2024-09-19T07:55:02.974976+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:59.785587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.975028+0000","last_change":"2024-09-19T07:55:02.974014+0000","last_active":"2024-09-19T07:55:02.975028+0000","last_peered":"2024-09-19T07:55:02.975028+0000","last_clean":"2024-09-19T07:55:02.975028+0000","last_became_active":"2024-09-19T07:55:02.973536+0000","last_became_peered":"2024-09-19T07:55:02.973536+0000","last_unstale":"2024-09-19T07:55:02.975028+0000","last_undegraded":"2024-09-19T07:55:02.975028+0000","last_fullsized":"2024-09-19T07:55:02.975028+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:55.597883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972720+0000","last_change":"2024-09-19T07:55:01.988008+0000","last_active":"2024-09-19T07:55:03.972720+0000","last_peered":"2024-09-19T07:55:03.972720+0000","last_clean":"2024-09-19T07:55:03.972720+0000","last_became_active":"2024-09-19T07:55:01.987725+0000","last_became_peered":"2024-09-19T07:55:01.987725+0000","last_unstale":"2024-09-19T07:55:03.972720+0000","last_undegraded":"2024-09-19T07:55:03.972720+0000","last_fullsized":"2024-09-19T07:55:03.972720+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:13.833572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972516+0000","last_change":"2024-09-19T07:55:02.991309+0000","last_active":"2024-09-19T07:55:03.972516+0000","last_peered":"2024-09-19T07:55:03.972516+0000","last_clean":"2024-09-19T07:55:03.972516+0000","last_became_active":"2024-09-19T07:55:02.991045+0000","last_became_peered":"2024-09-19T07:55:02.991045+0000","last_unstale":"2024-09-19T07:55:03.972516+0000","last_undegraded":"2024-09-19T07:55:03.972516+0000","last_fullsized":"2024-09-19T07:55:03.972516+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:38.709571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"140'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.982444+0000","last_change":"2024-09-19T07:55:01.978226+0000","last_active":"2024-09-19T07:55:03.982444+0000","last_peered":"2024-09-19T07:55:03.982444+0000","last_clean":"2024-09-19T07:55:03.982444+0000","last_became_active":"2024-09-19T07:55:01.977803+0000","last_became_peered":"2024-09-19T07:55:01.977803+0000","last_unstale":"2024-09-19T07:55:03.982444+0000","last_undegraded":"2024-09-19T07:55:03.982444+0000","last_fullsized":"2024-09-19T07:55:03.982444+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:49.852983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.973122+0000","last_change":"2024-09-19T07:55:02.977147+0000","last_active":"2024-09-19T07:55:03.973122+0000","last_peered":"2024-09-19T07:55:03.973122+0000","last_clean":"2024-09-19T07:55:03.973122+0000","last_became_active":"2024-09-19T07:55:02.976832+0000","last_became_peered":"2024-09-19T07:55:02.976832+0000","last_unstale":"2024-09-19T07:55:03.973122+0000","last_undegraded":"2024-09-19T07:55:03.973122+0000","last_fullsized":"2024-09-19T07:55:03.973122+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:03.581754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.974160+0000","last_change":"2024-09-19T07:55:01.974880+0000","last_active":"2024-09-19T07:55:02.974160+0000","last_peered":"2024-09-19T07:55:02.974160+0000","last_clean":"2024-09-19T07:55:02.974160+0000","last_became_active":"2024-09-19T07:55:01.974599+0000","last_became_peered":"2024-09-19T07:55:01.974599+0000","last_unstale":"2024-09-19T07:55:02.974160+0000","last_undegraded":"2024-09-19T07:55:02.974160+0000","last_fullsized":"2024-09-19T07:55:02.974160+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:11.363759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.994783+0000","last_change":"2024-09-19T07:55:02.989699+0000","last_active":"2024-09-19T07:55:02.994783+0000","last_peered":"2024-09-19T07:55:02.994783+0000","last_clean":"2024-09-19T07:55:02.994783+0000","last_became_active":"2024-09-19T07:55:02.989398+0000","last_became_peered":"2024-09-19T07:55:02.989398+0000","last_unstale":"2024-09-19T07:55:02.994783+0000","last_undegraded":"2024-09-19T07:55:02.994783+0000","last_fullsized":"2024-09-19T07:55:02.994783+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:11.153885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.970064+0000","last_change":"2024-09-19T07:55:01.981690+0000","last_active":"2024-09-19T07:55:02.970064+0000","last_peered":"2024-09-19T07:55:02.970064+0000","last_clean":"2024-09-19T07:55:02.970064+0000","last_became_active":"2024-09-19T07:55:01.981417+0000","last_became_peered":"2024-09-19T07:55:01.981417+0000","last_unstale":"2024-09-19T07:55:02.970064+0000","last_undegraded":"2024-09-19T07:55:02.970064+0000","last_fullsized":"2024-09-19T07:55:02.970064+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:26:17.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.988647+0000","last_change":"2024-09-19T07:55:02.986076+0000","last_active":"2024-09-19T07:55:02.988647+0000","last_peered":"2024-09-19T07:55:02.988647+0000","last_clean":"2024-09-19T07:55:02.988647+0000","last_became_active":"2024-09-19T07:55:02.984885+0000","last_became_peered":"2024-09-19T07:55:02.984885+0000","last_unstale":"2024-09-19T07:55:02.988647+0000","last_undegraded":"2024-09-19T07:55:02.988647+0000","last_fullsized":"2024-09-19T07:55:02.988647+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:33.113553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972219+0000","last_change":"2024-09-19T07:55:01.986920+0000","last_active":"2024-09-19T07:55:03.972219+0000","last_peered":"2024-09-19T07:55:03.972219+0000","last_clean":"2024-09-19T07:55:03.972219+0000","last_became_active":"2024-09-19T07:55:01.986618+0000","last_became_peered":"2024-09-19T07:55:01.986618+0000","last_unstale":"2024-09-19T07:55:03.972219+0000","last_undegraded":"2024-09-19T07:55:03.972219+0000","last_fullsized":"2024-09-19T07:55:03.972219+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:59.197262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972053+0000","last_change":"2024-09-19T07:55:02.989902+0000","last_active":"2024-09-19T07:55:03.972053+0000","last_peered":"2024-09-19T07:55:03.972053+0000","last_clean":"2024-09-19T07:55:03.972053+0000","last_became_active":"2024-09-19T07:55:02.987293+0000","last_became_peered":"2024-09-19T07:55:02.987293+0000","last_unstale":"2024-09-19T07:55:03.972053+0000","last_undegraded":"2024-09-19T07:55:03.972053+0000","last_fullsized":"2024-09-19T07:55:03.972053+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:54:28.829885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-19T07:55:00.965812+0000","last_change":"2024-09-19T07:55:00.965812+0000","last_active":"2024-09-19T07:55:00.954535+0000","last_peered":"2024-09-19T07:55:00.954535+0000","last_clean":"2024-09-19T07:55:00.954535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:00.965812+0000","last_undegraded":"2024-09-19T07:55:00.965812+0000","last_fullsized":"2024-09-19T07:55:00.965812+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.987084+0000","last_change":"2024-09-19T07:55:02.983221+0000","last_active":"2024-09-19T07:55:02.987084+0000","last_peered":"2024-09-19T07:55:02.987084+0000","last_clean":"2024-09-19T07:55:02.987084+0000","last_became_active":"2024-09-19T07:55:02.982829+0000","last_became_peered":"2024-09-19T07:55:02.982829+0000","last_unstale":"2024-09-19T07:55:02.987084+0000","last_undegraded":"2024-09-19T07:55:02.987084+0000","last_fullsized":"2024-09-19T07:55:02.987084+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:18.210530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972383+0000","last_change":"2024-09-19T07:55:01.980107+0000","last_active":"2024-09-19T07:55:03.972383+0000","last_peered":"2024-09-19T07:55:03.972383+0000","last_clean":"2024-09-19T07:55:03.972383+0000","last_became_active":"2024-09-19T07:55:01.979828+0000","last_became_peered":"2024-09-19T07:55:01.979828+0000","last_unstale":"2024-09-19T07:55:03.972383+0000","last_undegraded":"2024-09-19T07:55:03.972383+0000","last_fullsized":"2024-09-19T07:55:03.972383+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:17.205219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.988702+0000","last_change":"2024-09-19T07:55:02.986047+0000","last_active":"2024-09-19T07:55:02.988702+0000","last_peered":"2024-09-19T07:55:02.988702+0000","last_clean":"2024-09-19T07:55:02.988702+0000","last_became_active":"2024-09-19T07:55:02.985768+0000","last_became_peered":"2024-09-19T07:55:02.985768+0000","last_unstale":"2024-09-19T07:55:02.988702+0000","last_undegraded":"2024-09-19T07:55:02.988702+0000","last_fullsized":"2024-09-19T07:55:02.988702+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:27:57.147706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.982949+0000","last_change":"2024-09-19T07:55:01.980963+0000","last_active":"2024-09-19T07:55:03.982949+0000","last_peered":"2024-09-19T07:55:03.982949+0000","last_clean":"2024-09-19T07:55:03.982949+0000","last_became_active":"2024-09-19T07:55:01.980672+0000","last_became_peered":"2024-09-19T07:55:01.980672+0000","last_unstale":"2024-09-19T07:55:03.982949+0000","last_undegraded":"2024-09-19T07:55:03.982949+0000","last_fullsized":"2024-09-19T07:55:03.982949+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:11.722912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.988060+0000","last_change":"2024-09-19T07:55:02.985023+0000","last_active":"2024-09-19T07:55:02.988060+0000","last_peered":"2024-09-19T07:55:02.988060+0000","last_clean":"2024-09-19T07:55:02.988060+0000","last_became_active":"2024-09-19T07:55:02.984699+0000","last_became_peered":"2024-09-19T07:55:02.984699+0000","last_unstale":"2024-09-19T07:55:02.988060+0000","last_undegraded":"2024-09-19T07:55:02.988060+0000","last_fullsized":"2024-09-19T07:55:02.988060+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:06.465142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"139'6","reported_seq":18,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000873+0000","last_change":"2024-09-19T07:55:01.982519+0000","last_active":"2024-09-19T07:55:03.000873+0000","last_peered":"2024-09-19T07:55:03.000873+0000","last_clean":"2024-09-19T07:55:03.000873+0000","last_became_active":"2024-09-19T07:55:01.982223+0000","last_became_peered":"2024-09-19T07:55:01.982223+0000","last_unstale":"2024-09-19T07:55:03.000873+0000","last_undegraded":"2024-09-19T07:55:03.000873+0000","last_fullsized":"2024-09-19T07:55:03.000873+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:59:17.842847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":387,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983009+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:55:03.983009+0000","last_peered":"2024-09-19T07:55:03.983009+0000","last_clean":"2024-09-19T07:55:03.983009+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:55:03.983009+0000","last_undegraded":"2024-09-19T07:55:03.983009+0000","last_fullsized":"2024-09-19T07:55:03.983009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2298,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83384,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288456,"statfs":{"total":96636764160,"available":96551378944,"internally_reserved":0,"allocated":2334720,"data_stored":1397055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509588,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105080,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266760,"statfs":{"total":96636764160,"available":96529162240,"internally_reserved":0,"allocated":1740800,"data_stored":806735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509589,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73088,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298752,"statfs":{"total":96636764160,"available":96561922048,"internally_reserved":0,"allocated":1748992,"data_stored":813357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509589,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86984,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284856,"statfs":{"total":96636764160,"available":96547692544,"internally_reserved":0,"allocated":1748992,"data_stored":808277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509588,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105620,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266220,"statfs":{"total":96636764160,"available":96528609280,"internally_reserved":0,"allocated":2355200,"data_stored":1399351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542292,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118916,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252924,"statfs":{"total":96636764160,"available":96514994176,"internally_reserved":0,"allocated":1744896,"data_stored":807281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542292,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82880,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288960,"statfs":{"total":96636764160,"available":96551895040,"internally_reserved":0,"allocated":1740800,"data_stored":806735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109152,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262688,"statfs":{"total":96636764160,"available":96524992512,"internally_reserved":0,"allocated":1712128,"data_stored":792831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:55:05.970 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:55:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.107+0000 7ff44911f640 1 Processor -- start 2024-09-19T07:55:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.107+0000 7ff44911f640 1 -- start start 2024-09-19T07:55:06.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.108+0000 7ff44911f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff44415e2a0 0x7ff44406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:06.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.108+0000 7ff44911f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff44415e680 0x7ff4440cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:06.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.108+0000 7ff44911f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4440d1880 con 0x7ff44406d960 2024-09-19T07:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.108+0000 7ff44911f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff4440d1b60 con 0x7ff44415e680 2024-09-19T07:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.108+0000 7ff44911f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff4440cf2d0 con 0x7ff44415e2a0 2024-09-19T07:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.108+0000 7ff44259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff44415e680 0x7ff4440cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.108+0000 7ff44259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff44415e680 0x7ff4440cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43748/0 (socket says 172.21.15.16:43748) 2024-09-19T07:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.108+0000 7ff44259c640 1 -- 172.21.15.16:0/3813703607 learned_addr learned my addr 172.21.15.16:0/3813703607 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.108+0000 7ff442d9d640 1 --2- 172.21.15.16:0/3813703607 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff44415e2a0 0x7ff44406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.108+0000 7ff44259c640 1 -- 172.21.15.16:0/3813703607 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff44415e2a0 msgr2=0x7ff44406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.108+0000 7ff44259c640 1 --2- 172.21.15.16:0/3813703607 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff44415e2a0 0x7ff44406d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.108+0000 7ff44259c640 1 -- 172.21.15.16:0/3813703607 >> v1:172.21.15.146:6789/0 conn(0x7ff44406d960 legacy=0x7ff4440d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.108+0000 7ff44259c640 1 -- 172.21.15.16:0/3813703607 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4440cf5b0 con 0x7ff44415e680 2024-09-19T07:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.108+0000 7ff442d9d640 1 --2- 172.21.15.16:0/3813703607 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff44415e2a0 0x7ff44406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.109+0000 7ff44259c640 1 --2- 172.21.15.16:0/3813703607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff44415e680 0x7ff4440cfad0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009870 tx=0x7ff42c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b82b0c4943ca84a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.109+0000 7ff441d9b640 1 -- 172.21.15.16:0/3813703607 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c037070 con 0x7ff44415e680 2024-09-19T07:55:06.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.109+0000 7ff441d9b640 1 -- 172.21.15.16:0/3813703607 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff42c032440 con 0x7ff44415e680 2024-09-19T07:55:06.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.109+0000 7ff441d9b640 1 -- 172.21.15.16:0/3813703607 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c032810 con 0x7ff44415e680 2024-09-19T07:55:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.110+0000 7ff44911f640 1 -- 172.21.15.16:0/3813703607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff44415e680 msgr2=0x7ff4440cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.110+0000 7ff44911f640 1 --2- 172.21.15.16:0/3813703607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff44415e680 0x7ff4440cfad0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009870 tx=0x7ff42c02a200 comp rx=0 tx=0).stop 2024-09-19T07:55:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.110+0000 7ff44911f640 1 -- 172.21.15.16:0/3813703607 shutdown_connections 2024-09-19T07:55:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.110+0000 7ff44911f640 1 --2- 172.21.15.16:0/3813703607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff44415e680 0x7ff4440cfad0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.110+0000 7ff44911f640 1 --2- 172.21.15.16:0/3813703607 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff44415e2a0 0x7ff44406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.110+0000 7ff44911f640 1 -- 172.21.15.16:0/3813703607 >> 172.21.15.16:0/3813703607 conn(0x7ff44409d760 msgr2=0x7ff44408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.110+0000 7ff44911f640 1 -- 172.21.15.16:0/3813703607 shutdown_connections 2024-09-19T07:55:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.110+0000 7ff44911f640 1 -- 172.21.15.16:0/3813703607 wait complete. 2024-09-19T07:55:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.111+0000 7ff44911f640 1 Processor -- start 2024-09-19T07:55:06.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.111+0000 7ff44911f640 1 -- start start 2024-09-19T07:55:06.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.112+0000 7ff44911f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff44406d960 0x7ff44415ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:06.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.112+0000 7ff44911f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff44415e2a0 0x7ff44415d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:06.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.112+0000 7ff44911f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff44415e680 0x7ff4440cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:06.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.112+0000 7ff44911f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4440d4c00 con 0x7ff44415e2a0 2024-09-19T07:55:06.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.112+0000 7ff44911f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff4440d4a80 con 0x7ff44406d960 2024-09-19T07:55:06.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.112+0000 7ff44911f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff4440d0010 con 0x7ff44415e680 2024-09-19T07:55:06.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.112+0000 7ff44259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff44415e2a0 0x7ff44415d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:06.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.112+0000 7ff44259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff44415e2a0 0x7ff44415d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43762/0 (socket says 172.21.15.16:43762) 2024-09-19T07:55:06.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.112+0000 7ff44259c640 1 -- 172.21.15.16:0/1409461199 learned_addr learned my addr 172.21.15.16:0/1409461199 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:06.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.112+0000 7ff442d9d640 1 --2- 172.21.15.16:0/1409461199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff44406d960 0x7ff44415ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:06.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.112+0000 7ff44359e640 1 --2- 172.21.15.16:0/1409461199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff44415e680 0x7ff4440cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:06.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.113+0000 7ff44259c640 1 -- 172.21.15.16:0/1409461199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff44415e680 msgr2=0x7ff4440cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:06.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.113+0000 7ff44259c640 1 --2- 172.21.15.16:0/1409461199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff44415e680 0x7ff4440cdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.113+0000 7ff44259c640 1 -- 172.21.15.16:0/1409461199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff44406d960 msgr2=0x7ff44415ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:06.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.113+0000 7ff44259c640 1 --2- 172.21.15.16:0/1409461199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff44406d960 0x7ff44415ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.113+0000 7ff44259c640 1 -- 172.21.15.16:0/1409461199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4440cc060 con 0x7ff44415e2a0 2024-09-19T07:55:06.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.113+0000 7ff442d9d640 1 --2- 172.21.15.16:0/1409461199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff44406d960 0x7ff44415ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:06.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.113+0000 7ff44259c640 1 --2- 172.21.15.16:0/1409461199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff44415e2a0 0x7ff44415d3b0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7ff42c0090b0 tx=0x7ff42c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:06.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.113+0000 7ff43b7fe640 1 -- 172.21.15.16:0/1409461199 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c041070 con 0x7ff44415e2a0 2024-09-19T07:55:06.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.113+0000 7ff43b7fe640 1 -- 172.21.15.16:0/1409461199 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff42c03c890 con 0x7ff44415e2a0 2024-09-19T07:55:06.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.113+0000 7ff43b7fe640 1 -- 172.21.15.16:0/1409461199 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c037040 con 0x7ff44415e2a0 2024-09-19T07:55:06.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.113+0000 7ff44911f640 1 -- 172.21.15.16:0/1409461199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4440cc340 con 0x7ff44415e2a0 2024-09-19T07:55:06.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.113+0000 7ff44911f640 1 -- 172.21.15.16:0/1409461199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4440b8d70 con 0x7ff44415e2a0 2024-09-19T07:55:06.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.114+0000 7ff44911f640 1 -- 172.21.15.16:0/1409461199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4440cbdb0 con 0x7ff44415e2a0 2024-09-19T07:55:06.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.114+0000 7ff43b7fe640 1 -- 172.21.15.16:0/1409461199 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff42c044070 con 0x7ff44415e2a0 2024-09-19T07:55:06.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.115+0000 7ff43b7fe640 1 --2- 172.21.15.16:0/1409461199 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff42406c2b0 0x7ff42406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:06.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.115+0000 7ff43b7fe640 1 -- 172.21.15.16:0/1409461199 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7ff42c0ac370 con 0x7ff44415e2a0 2024-09-19T07:55:06.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.115+0000 7ff442d9d640 1 --2- 172.21.15.16:0/1409461199 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff42406c2b0 0x7ff42406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:06.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.116+0000 7ff442d9d640 1 --2- 172.21.15.16:0/1409461199 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff42406c2b0 0x7ff42406e770 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff434006fd0 tx=0x7ff434008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:06.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.119+0000 7ff43b7fe640 1 -- 172.21.15.16:0/1409461199 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff42c07ab00 con 0x7ff44415e2a0 2024-09-19T07:55:06.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.291+0000 7ff44911f640 1 -- 172.21.15.16:0/1409461199 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff44406cf90 con 0x7ff42406c2b0 2024-09-19T07:55:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.298+0000 7ff43b7fe640 1 -- 172.21.15.16:0/1409461199 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251139 (secure 0 0 0) 0x7ff44406cf90 con 0x7ff42406c2b0 2024-09-19T07:55:06.300 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:06.302 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:55:06.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.300+0000 7ff44911f640 1 -- 172.21.15.16:0/1409461199 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff42406c2b0 msgr2=0x7ff42406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:06.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.300+0000 7ff44911f640 1 --2- 172.21.15.16:0/1409461199 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff42406c2b0 0x7ff42406e770 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff434006fd0 tx=0x7ff434008040 comp rx=0 tx=0).stop 2024-09-19T07:55:06.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.300+0000 7ff44911f640 1 -- 172.21.15.16:0/1409461199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff44415e2a0 msgr2=0x7ff44415d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:06.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.300+0000 7ff44911f640 1 --2- 172.21.15.16:0/1409461199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff44415e2a0 0x7ff44415d3b0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7ff42c0090b0 tx=0x7ff42c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.301+0000 7ff44911f640 1 -- 172.21.15.16:0/1409461199 shutdown_connections 2024-09-19T07:55:06.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.301+0000 7ff44911f640 1 --2- 172.21.15.16:0/1409461199 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff42406c2b0 0x7ff42406e770 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.301+0000 7ff44911f640 1 --2- 172.21.15.16:0/1409461199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff44415e680 0x7ff4440cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.301+0000 7ff44911f640 1 --2- 172.21.15.16:0/1409461199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff44415e2a0 0x7ff44415d3b0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.301+0000 7ff44911f640 1 --2- 172.21.15.16:0/1409461199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff44406d960 0x7ff44415ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.301+0000 7ff44911f640 1 -- 172.21.15.16:0/1409461199 >> 172.21.15.16:0/1409461199 conn(0x7ff44409d760 msgr2=0x7ff44415ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:06.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.301+0000 7ff44911f640 1 -- 172.21.15.16:0/1409461199 shutdown_connections 2024-09-19T07:55:06.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.301+0000 7ff44911f640 1 -- 172.21.15.16:0/1409461199 wait complete. 2024-09-19T07:55:06.312 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-09-19T07:55:04.629044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592666,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":199,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":46,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":765104,"kb_used_data":14772,"kb_used_omap":12,"kb_used_meta":750131,"kb_avail":754209616,"statfs":{"total":773094113280,"available":772310646784,"internally_reserved":0,"allocated":15126528,"data_stored":7631622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":768134704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2298,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000733"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972662+0000","last_change":"2024-09-19T07:55:02.977838+0000","last_active":"2024-09-19T07:55:03.972662+0000","last_peered":"2024-09-19T07:55:03.972662+0000","last_clean":"2024-09-19T07:55:03.972662+0000","last_became_active":"2024-09-19T07:55:02.977562+0000","last_became_peered":"2024-09-19T07:55:02.977562+0000","last_unstale":"2024-09-19T07:55:03.972662+0000","last_undegraded":"2024-09-19T07:55:03.972662+0000","last_fullsized":"2024-09-19T07:55:03.972662+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:18.348990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.994497+0000","last_change":"2024-09-19T07:55:02.988730+0000","last_active":"2024-09-19T07:55:02.994497+0000","last_peered":"2024-09-19T07:55:02.994497+0000","last_clean":"2024-09-19T07:55:02.994497+0000","last_became_active":"2024-09-19T07:55:02.988454+0000","last_became_peered":"2024-09-19T07:55:02.988454+0000","last_unstale":"2024-09-19T07:55:02.994497+0000","last_undegraded":"2024-09-19T07:55:02.994497+0000","last_fullsized":"2024-09-19T07:55:02.994497+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:55.150368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998989+0000","last_change":"2024-09-19T07:55:02.995513+0000","last_active":"2024-09-19T07:55:02.998989+0000","last_peered":"2024-09-19T07:55:02.998989+0000","last_clean":"2024-09-19T07:55:02.998989+0000","last_became_active":"2024-09-19T07:55:02.995162+0000","last_became_peered":"2024-09-19T07:55:02.995162+0000","last_unstale":"2024-09-19T07:55:02.998989+0000","last_undegraded":"2024-09-19T07:55:02.998989+0000","last_fullsized":"2024-09-19T07:55:02.998989+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:08.826237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972077+0000","last_change":"2024-09-19T07:55:02.991747+0000","last_active":"2024-09-19T07:55:03.972077+0000","last_peered":"2024-09-19T07:55:03.972077+0000","last_clean":"2024-09-19T07:55:03.972077+0000","last_became_active":"2024-09-19T07:55:02.991487+0000","last_became_peered":"2024-09-19T07:55:02.991487+0000","last_unstale":"2024-09-19T07:55:03.972077+0000","last_undegraded":"2024-09-19T07:55:03.972077+0000","last_fullsized":"2024-09-19T07:55:03.972077+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:00.867259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.978550+0000","last_change":"2024-09-19T07:55:02.974381+0000","last_active":"2024-09-19T07:55:02.978550+0000","last_peered":"2024-09-19T07:55:02.978550+0000","last_clean":"2024-09-19T07:55:02.978550+0000","last_became_active":"2024-09-19T07:55:02.974105+0000","last_became_peered":"2024-09-19T07:55:02.974105+0000","last_unstale":"2024-09-19T07:55:02.978550+0000","last_undegraded":"2024-09-19T07:55:02.978550+0000","last_fullsized":"2024-09-19T07:55:02.978550+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:08.519276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.980270+0000","last_change":"2024-09-19T07:55:02.975703+0000","last_active":"2024-09-19T07:55:02.980270+0000","last_peered":"2024-09-19T07:55:02.980270+0000","last_clean":"2024-09-19T07:55:02.980270+0000","last_became_active":"2024-09-19T07:55:02.975379+0000","last_became_peered":"2024-09-19T07:55:02.975379+0000","last_unstale":"2024-09-19T07:55:02.980270+0000","last_undegraded":"2024-09-19T07:55:02.980270+0000","last_fullsized":"2024-09-19T07:55:02.980270+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:11.846213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972184+0000","last_change":"2024-09-19T07:55:02.985662+0000","last_active":"2024-09-19T07:55:03.972184+0000","last_peered":"2024-09-19T07:55:03.972184+0000","last_clean":"2024-09-19T07:55:03.972184+0000","last_became_active":"2024-09-19T07:55:02.985378+0000","last_became_peered":"2024-09-19T07:55:02.985378+0000","last_unstale":"2024-09-19T07:55:03.972184+0000","last_undegraded":"2024-09-19T07:55:03.972184+0000","last_fullsized":"2024-09-19T07:55:03.972184+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:33.679405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.992880+0000","last_change":"2024-09-19T07:55:02.991159+0000","last_active":"2024-09-19T07:55:02.992880+0000","last_peered":"2024-09-19T07:55:02.992880+0000","last_clean":"2024-09-19T07:55:02.992880+0000","last_became_active":"2024-09-19T07:55:02.990631+0000","last_became_peered":"2024-09-19T07:55:02.990631+0000","last_unstale":"2024-09-19T07:55:02.992880+0000","last_undegraded":"2024-09-19T07:55:02.992880+0000","last_fullsized":"2024-09-19T07:55:02.992880+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:16.038904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998379+0000","last_change":"2024-09-19T07:55:02.994532+0000","last_active":"2024-09-19T07:55:02.998379+0000","last_peered":"2024-09-19T07:55:02.998379+0000","last_clean":"2024-09-19T07:55:02.998379+0000","last_became_active":"2024-09-19T07:55:02.994201+0000","last_became_peered":"2024-09-19T07:55:02.994201+0000","last_unstale":"2024-09-19T07:55:02.998379+0000","last_undegraded":"2024-09-19T07:55:02.998379+0000","last_fullsized":"2024-09-19T07:55:02.998379+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:12.950630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.984004+0000","last_change":"2024-09-19T07:55:02.981864+0000","last_active":"2024-09-19T07:55:02.984004+0000","last_peered":"2024-09-19T07:55:02.984004+0000","last_clean":"2024-09-19T07:55:02.984004+0000","last_became_active":"2024-09-19T07:55:02.981541+0000","last_became_peered":"2024-09-19T07:55:02.981541+0000","last_unstale":"2024-09-19T07:55:02.984004+0000","last_undegraded":"2024-09-19T07:55:02.984004+0000","last_fullsized":"2024-09-19T07:55:02.984004+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:22.255417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:01.959847+0000","last_change":"2024-09-19T07:55:01.959847+0000","last_active":"2024-09-19T07:55:01.959847+0000","last_peered":"2024-09-19T07:55:01.959847+0000","last_clean":"2024-09-19T07:55:01.959847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:01.959847+0000","last_undegraded":"2024-09-19T07:55:01.959847+0000","last_fullsized":"2024-09-19T07:55:01.959847+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972187+0000","last_change":"2024-09-19T07:55:02.989911+0000","last_active":"2024-09-19T07:55:03.972187+0000","last_peered":"2024-09-19T07:55:03.972187+0000","last_clean":"2024-09-19T07:55:03.972187+0000","last_became_active":"2024-09-19T07:55:02.989574+0000","last_became_peered":"2024-09-19T07:55:02.989574+0000","last_unstale":"2024-09-19T07:55:03.972187+0000","last_undegraded":"2024-09-19T07:55:03.972187+0000","last_fullsized":"2024-09-19T07:55:03.972187+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:20.107114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.985584+0000","last_change":"2024-09-19T07:55:02.983842+0000","last_active":"2024-09-19T07:55:02.985584+0000","last_peered":"2024-09-19T07:55:02.985584+0000","last_clean":"2024-09-19T07:55:02.985584+0000","last_became_active":"2024-09-19T07:55:02.983583+0000","last_became_peered":"2024-09-19T07:55:02.983583+0000","last_unstale":"2024-09-19T07:55:02.985584+0000","last_undegraded":"2024-09-19T07:55:02.985584+0000","last_fullsized":"2024-09-19T07:55:02.985584+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:58.534832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.971850+0000","last_change":"2024-09-19T07:55:02.991151+0000","last_active":"2024-09-19T07:55:03.971850+0000","last_peered":"2024-09-19T07:55:03.971850+0000","last_clean":"2024-09-19T07:55:03.971850+0000","last_became_active":"2024-09-19T07:55:02.990791+0000","last_became_peered":"2024-09-19T07:55:02.990791+0000","last_unstale":"2024-09-19T07:55:03.971850+0000","last_undegraded":"2024-09-19T07:55:03.971850+0000","last_fullsized":"2024-09-19T07:55:03.971850+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:16.038904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.997603+0000","last_change":"2024-09-19T07:55:02.993711+0000","last_active":"2024-09-19T07:55:02.997603+0000","last_peered":"2024-09-19T07:55:02.997603+0000","last_clean":"2024-09-19T07:55:02.997603+0000","last_became_active":"2024-09-19T07:55:02.993391+0000","last_became_peered":"2024-09-19T07:55:02.993391+0000","last_unstale":"2024-09-19T07:55:02.997603+0000","last_undegraded":"2024-09-19T07:55:02.997603+0000","last_fullsized":"2024-09-19T07:55:02.997603+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:53:11.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.981640+0000","last_change":"2024-09-19T07:55:02.977783+0000","last_active":"2024-09-19T07:55:02.981640+0000","last_peered":"2024-09-19T07:55:02.981640+0000","last_clean":"2024-09-19T07:55:02.981640+0000","last_became_active":"2024-09-19T07:55:02.977494+0000","last_became_peered":"2024-09-19T07:55:02.977494+0000","last_unstale":"2024-09-19T07:55:02.981640+0000","last_undegraded":"2024-09-19T07:55:02.981640+0000","last_fullsized":"2024-09-19T07:55:02.981640+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:42.284168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972758+0000","last_change":"2024-09-19T07:55:02.989847+0000","last_active":"2024-09-19T07:55:03.972758+0000","last_peered":"2024-09-19T07:55:03.972758+0000","last_clean":"2024-09-19T07:55:03.972758+0000","last_became_active":"2024-09-19T07:55:02.989566+0000","last_became_peered":"2024-09-19T07:55:02.989566+0000","last_unstale":"2024-09-19T07:55:03.972758+0000","last_undegraded":"2024-09-19T07:55:03.972758+0000","last_fullsized":"2024-09-19T07:55:03.972758+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:57:07.989977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:01.959847+0000","last_change":"2024-09-19T07:55:01.959847+0000","last_active":"2024-09-19T07:55:01.959847+0000","last_peered":"2024-09-19T07:55:01.959847+0000","last_clean":"2024-09-19T07:55:01.959847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:01.959847+0000","last_undegraded":"2024-09-19T07:55:01.959847+0000","last_fullsized":"2024-09-19T07:55:01.959847+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983370+0000","last_change":"2024-09-19T07:55:02.988233+0000","last_active":"2024-09-19T07:55:03.983370+0000","last_peered":"2024-09-19T07:55:03.983370+0000","last_clean":"2024-09-19T07:55:03.983370+0000","last_became_active":"2024-09-19T07:55:02.987586+0000","last_became_peered":"2024-09-19T07:55:02.987586+0000","last_unstale":"2024-09-19T07:55:03.983370+0000","last_undegraded":"2024-09-19T07:55:03.983370+0000","last_fullsized":"2024-09-19T07:55:03.983370+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:37.091603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.989123+0000","last_change":"2024-09-19T07:55:02.984872+0000","last_active":"2024-09-19T07:55:02.989123+0000","last_peered":"2024-09-19T07:55:02.989123+0000","last_clean":"2024-09-19T07:55:02.989123+0000","last_became_active":"2024-09-19T07:55:02.984588+0000","last_became_peered":"2024-09-19T07:55:02.984588+0000","last_unstale":"2024-09-19T07:55:02.989123+0000","last_undegraded":"2024-09-19T07:55:02.989123+0000","last_fullsized":"2024-09-19T07:55:02.989123+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:37.091603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.986844+0000","last_change":"2024-09-19T07:55:02.985444+0000","last_active":"2024-09-19T07:55:02.986844+0000","last_peered":"2024-09-19T07:55:02.986844+0000","last_clean":"2024-09-19T07:55:02.986844+0000","last_became_active":"2024-09-19T07:55:02.985161+0000","last_became_peered":"2024-09-19T07:55:02.985161+0000","last_unstale":"2024-09-19T07:55:02.986844+0000","last_undegraded":"2024-09-19T07:55:02.986844+0000","last_fullsized":"2024-09-19T07:55:02.986844+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:19.760108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972782+0000","last_change":"2024-09-19T07:55:02.980246+0000","last_active":"2024-09-19T07:55:03.972782+0000","last_peered":"2024-09-19T07:55:03.972782+0000","last_clean":"2024-09-19T07:55:03.972782+0000","last_became_active":"2024-09-19T07:55:02.979970+0000","last_became_peered":"2024-09-19T07:55:02.979970+0000","last_unstale":"2024-09-19T07:55:03.972782+0000","last_undegraded":"2024-09-19T07:55:03.972782+0000","last_fullsized":"2024-09-19T07:55:03.972782+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:19.717109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.986210+0000","last_change":"2024-09-19T07:55:02.979612+0000","last_active":"2024-09-19T07:55:02.986210+0000","last_peered":"2024-09-19T07:55:02.986210+0000","last_clean":"2024-09-19T07:55:02.986210+0000","last_became_active":"2024-09-19T07:55:02.978902+0000","last_became_peered":"2024-09-19T07:55:02.978902+0000","last_unstale":"2024-09-19T07:55:02.986210+0000","last_undegraded":"2024-09-19T07:55:02.986210+0000","last_fullsized":"2024-09-19T07:55:02.986210+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:57.305718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:01.959847+0000","last_change":"2024-09-19T07:55:01.959847+0000","last_active":"2024-09-19T07:55:01.959847+0000","last_peered":"2024-09-19T07:55:01.959847+0000","last_clean":"2024-09-19T07:55:01.959847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:01.959847+0000","last_undegraded":"2024-09-19T07:55:01.959847+0000","last_fullsized":"2024-09-19T07:55:01.959847+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972885+0000","last_change":"2024-09-19T07:55:02.981645+0000","last_active":"2024-09-19T07:55:03.972885+0000","last_peered":"2024-09-19T07:55:03.972885+0000","last_clean":"2024-09-19T07:55:03.972885+0000","last_became_active":"2024-09-19T07:55:02.981377+0000","last_became_peered":"2024-09-19T07:55:02.981377+0000","last_unstale":"2024-09-19T07:55:03.972885+0000","last_undegraded":"2024-09-19T07:55:03.972885+0000","last_fullsized":"2024-09-19T07:55:03.972885+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:28.867264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972976+0000","last_change":"2024-09-19T07:55:02.991055+0000","last_active":"2024-09-19T07:55:03.972976+0000","last_peered":"2024-09-19T07:55:03.972976+0000","last_clean":"2024-09-19T07:55:03.972976+0000","last_became_active":"2024-09-19T07:55:02.990517+0000","last_became_peered":"2024-09-19T07:55:02.990517+0000","last_unstale":"2024-09-19T07:55:03.972976+0000","last_undegraded":"2024-09-19T07:55:03.972976+0000","last_fullsized":"2024-09-19T07:55:03.972976+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:52.586504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.999462+0000","last_change":"2024-09-19T07:55:02.998488+0000","last_active":"2024-09-19T07:55:02.999462+0000","last_peered":"2024-09-19T07:55:02.999462+0000","last_clean":"2024-09-19T07:55:02.999462+0000","last_became_active":"2024-09-19T07:55:02.998192+0000","last_became_peered":"2024-09-19T07:55:02.998192+0000","last_unstale":"2024-09-19T07:55:02.999462+0000","last_undegraded":"2024-09-19T07:55:02.999462+0000","last_fullsized":"2024-09-19T07:55:02.999462+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:20.318482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:01.959847+0000","last_change":"2024-09-19T07:55:01.959847+0000","last_active":"2024-09-19T07:55:01.959847+0000","last_peered":"2024-09-19T07:55:01.959847+0000","last_clean":"2024-09-19T07:55:01.959847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:01.959847+0000","last_undegraded":"2024-09-19T07:55:01.959847+0000","last_fullsized":"2024-09-19T07:55:01.959847+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.987062+0000","last_change":"2024-09-19T07:55:02.985372+0000","last_active":"2024-09-19T07:55:02.987062+0000","last_peered":"2024-09-19T07:55:02.987062+0000","last_clean":"2024-09-19T07:55:02.987062+0000","last_became_active":"2024-09-19T07:55:02.985109+0000","last_became_peered":"2024-09-19T07:55:02.985109+0000","last_unstale":"2024-09-19T07:55:02.987062+0000","last_undegraded":"2024-09-19T07:55:02.987062+0000","last_fullsized":"2024-09-19T07:55:02.987062+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:59.971722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:01.959847+0000","last_change":"2024-09-19T07:55:01.959847+0000","last_active":"2024-09-19T07:55:01.959847+0000","last_peered":"2024-09-19T07:55:01.959847+0000","last_clean":"2024-09-19T07:55:01.959847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:01.959847+0000","last_undegraded":"2024-09-19T07:55:01.959847+0000","last_fullsized":"2024-09-19T07:55:01.959847+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.981739+0000","last_change":"2024-09-19T07:55:02.978386+0000","last_active":"2024-09-19T07:55:02.981739+0000","last_peered":"2024-09-19T07:55:02.981739+0000","last_clean":"2024-09-19T07:55:02.981739+0000","last_became_active":"2024-09-19T07:55:02.978134+0000","last_became_peered":"2024-09-19T07:55:02.978134+0000","last_unstale":"2024-09-19T07:55:02.981739+0000","last_undegraded":"2024-09-19T07:55:02.981739+0000","last_fullsized":"2024-09-19T07:55:02.981739+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:06.762003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.992859+0000","last_change":"2024-09-19T07:55:02.990915+0000","last_active":"2024-09-19T07:55:02.992859+0000","last_peered":"2024-09-19T07:55:02.992859+0000","last_clean":"2024-09-19T07:55:02.992859+0000","last_became_active":"2024-09-19T07:55:02.990273+0000","last_became_peered":"2024-09-19T07:55:02.990273+0000","last_unstale":"2024-09-19T07:55:02.992859+0000","last_undegraded":"2024-09-19T07:55:02.992859+0000","last_fullsized":"2024-09-19T07:55:02.992859+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:25.070703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998804+0000","last_change":"2024-09-19T07:55:02.994240+0000","last_active":"2024-09-19T07:55:02.998804+0000","last_peered":"2024-09-19T07:55:02.998804+0000","last_clean":"2024-09-19T07:55:02.998804+0000","last_became_active":"2024-09-19T07:55:02.993966+0000","last_became_peered":"2024-09-19T07:55:02.993966+0000","last_unstale":"2024-09-19T07:55:02.998804+0000","last_undegraded":"2024-09-19T07:55:02.998804+0000","last_fullsized":"2024-09-19T07:55:02.998804+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:18.497803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"139'4","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000708+0000","last_change":"2024-09-19T07:55:01.982819+0000","last_active":"2024-09-19T07:55:03.000708+0000","last_peered":"2024-09-19T07:55:03.000708+0000","last_clean":"2024-09-19T07:55:03.000708+0000","last_became_active":"2024-09-19T07:55:01.982455+0000","last_became_peered":"2024-09-19T07:55:01.982455+0000","last_unstale":"2024-09-19T07:55:03.000708+0000","last_undegraded":"2024-09-19T07:55:03.000708+0000","last_fullsized":"2024-09-19T07:55:03.000708+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:01.483855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.995361+0000","last_change":"2024-09-19T07:55:02.988748+0000","last_active":"2024-09-19T07:55:02.995361+0000","last_peered":"2024-09-19T07:55:02.995361+0000","last_clean":"2024-09-19T07:55:02.995361+0000","last_became_active":"2024-09-19T07:55:02.988417+0000","last_became_peered":"2024-09-19T07:55:02.988417+0000","last_unstale":"2024-09-19T07:55:02.995361+0000","last_undegraded":"2024-09-19T07:55:02.995361+0000","last_fullsized":"2024-09-19T07:55:02.995361+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:11.124464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983330+0000","last_change":"2024-09-19T07:55:01.986317+0000","last_active":"2024-09-19T07:55:03.983330+0000","last_peered":"2024-09-19T07:55:03.983330+0000","last_clean":"2024-09-19T07:55:03.983330+0000","last_became_active":"2024-09-19T07:55:01.986050+0000","last_became_peered":"2024-09-19T07:55:01.986050+0000","last_unstale":"2024-09-19T07:55:03.983330+0000","last_undegraded":"2024-09-19T07:55:03.983330+0000","last_fullsized":"2024-09-19T07:55:03.983330+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:20.699942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.988705+0000","last_change":"2024-09-19T07:55:02.988084+0000","last_active":"2024-09-19T07:55:02.988705+0000","last_peered":"2024-09-19T07:55:02.988705+0000","last_clean":"2024-09-19T07:55:02.988705+0000","last_became_active":"2024-09-19T07:55:02.987819+0000","last_became_peered":"2024-09-19T07:55:02.987819+0000","last_unstale":"2024-09-19T07:55:02.988705+0000","last_undegraded":"2024-09-19T07:55:02.988705+0000","last_fullsized":"2024-09-19T07:55:02.988705+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:34.829241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983598+0000","last_change":"2024-09-19T07:55:01.983959+0000","last_active":"2024-09-19T07:55:03.983598+0000","last_peered":"2024-09-19T07:55:03.983598+0000","last_clean":"2024-09-19T07:55:03.983598+0000","last_became_active":"2024-09-19T07:55:01.983682+0000","last_became_peered":"2024-09-19T07:55:01.983682+0000","last_unstale":"2024-09-19T07:55:03.983598+0000","last_undegraded":"2024-09-19T07:55:03.983598+0000","last_fullsized":"2024-09-19T07:55:03.983598+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:17.343679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983514+0000","last_change":"2024-09-19T07:55:02.988272+0000","last_active":"2024-09-19T07:55:03.983514+0000","last_peered":"2024-09-19T07:55:03.983514+0000","last_clean":"2024-09-19T07:55:03.983514+0000","last_became_active":"2024-09-19T07:55:02.987972+0000","last_became_peered":"2024-09-19T07:55:02.987972+0000","last_unstale":"2024-09-19T07:55:03.983514+0000","last_undegraded":"2024-09-19T07:55:03.983514+0000","last_fullsized":"2024-09-19T07:55:03.983514+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:55.597883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"139'2","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000148+0000","last_change":"2024-09-19T07:55:01.975504+0000","last_active":"2024-09-19T07:55:03.000148+0000","last_peered":"2024-09-19T07:55:03.000148+0000","last_clean":"2024-09-19T07:55:03.000148+0000","last_became_active":"2024-09-19T07:55:01.975216+0000","last_became_peered":"2024-09-19T07:55:01.975216+0000","last_unstale":"2024-09-19T07:55:03.000148+0000","last_undegraded":"2024-09-19T07:55:03.000148+0000","last_fullsized":"2024-09-19T07:55:03.000148+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:47.640841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.990742+0000","last_change":"2024-09-19T07:55:02.990235+0000","last_active":"2024-09-19T07:55:02.990742+0000","last_peered":"2024-09-19T07:55:02.990742+0000","last_clean":"2024-09-19T07:55:02.990742+0000","last_became_active":"2024-09-19T07:55:02.989920+0000","last_became_peered":"2024-09-19T07:55:02.989920+0000","last_unstale":"2024-09-19T07:55:02.990742+0000","last_undegraded":"2024-09-19T07:55:02.990742+0000","last_fullsized":"2024-09-19T07:55:02.990742+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:55.597883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"139'1","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.997333+0000","last_change":"2024-09-19T07:55:01.984133+0000","last_active":"2024-09-19T07:55:02.997333+0000","last_peered":"2024-09-19T07:55:02.997333+0000","last_clean":"2024-09-19T07:55:02.997333+0000","last_became_active":"2024-09-19T07:55:01.983829+0000","last_became_peered":"2024-09-19T07:55:01.983829+0000","last_unstale":"2024-09-19T07:55:02.997333+0000","last_undegraded":"2024-09-19T07:55:02.997333+0000","last_fullsized":"2024-09-19T07:55:02.997333+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:08.327895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.971878+0000","last_change":"2024-09-19T07:55:02.990053+0000","last_active":"2024-09-19T07:55:03.971878+0000","last_peered":"2024-09-19T07:55:03.971878+0000","last_clean":"2024-09-19T07:55:03.971878+0000","last_became_active":"2024-09-19T07:55:02.987609+0000","last_became_peered":"2024-09-19T07:55:02.987609+0000","last_unstale":"2024-09-19T07:55:03.971878+0000","last_undegraded":"2024-09-19T07:55:03.971878+0000","last_fullsized":"2024-09-19T07:55:03.971878+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:02:28.317679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"139'1","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998584+0000","last_change":"2024-09-19T07:55:01.984540+0000","last_active":"2024-09-19T07:55:02.998584+0000","last_peered":"2024-09-19T07:55:02.998584+0000","last_clean":"2024-09-19T07:55:02.998584+0000","last_became_active":"2024-09-19T07:55:01.984218+0000","last_became_peered":"2024-09-19T07:55:01.984218+0000","last_unstale":"2024-09-19T07:55:02.998584+0000","last_undegraded":"2024-09-19T07:55:02.998584+0000","last_fullsized":"2024-09-19T07:55:02.998584+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:59:17.842847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.987109+0000","last_change":"2024-09-19T07:55:02.983739+0000","last_active":"2024-09-19T07:55:02.987109+0000","last_peered":"2024-09-19T07:55:02.987109+0000","last_clean":"2024-09-19T07:55:02.987109+0000","last_became_active":"2024-09-19T07:55:02.983448+0000","last_became_peered":"2024-09-19T07:55:02.983448+0000","last_unstale":"2024-09-19T07:55:02.987109+0000","last_undegraded":"2024-09-19T07:55:02.987109+0000","last_fullsized":"2024-09-19T07:55:02.987109+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:50.858294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972280+0000","last_change":"2024-09-19T07:55:01.978901+0000","last_active":"2024-09-19T07:55:03.972280+0000","last_peered":"2024-09-19T07:55:03.972280+0000","last_clean":"2024-09-19T07:55:03.972280+0000","last_became_active":"2024-09-19T07:55:01.978399+0000","last_became_peered":"2024-09-19T07:55:01.978399+0000","last_unstale":"2024-09-19T07:55:03.972280+0000","last_undegraded":"2024-09-19T07:55:03.972280+0000","last_fullsized":"2024-09-19T07:55:03.972280+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:11.363759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972118+0000","last_change":"2024-09-19T07:55:02.978549+0000","last_active":"2024-09-19T07:55:03.972118+0000","last_peered":"2024-09-19T07:55:03.972118+0000","last_clean":"2024-09-19T07:55:03.972118+0000","last_became_active":"2024-09-19T07:55:02.978270+0000","last_became_peered":"2024-09-19T07:55:02.978270+0000","last_unstale":"2024-09-19T07:55:03.972118+0000","last_undegraded":"2024-09-19T07:55:03.972118+0000","last_fullsized":"2024-09-19T07:55:03.972118+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:21.705254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.969267+0000","last_change":"2024-09-19T07:55:01.983833+0000","last_active":"2024-09-19T07:55:02.969267+0000","last_peered":"2024-09-19T07:55:02.969267+0000","last_clean":"2024-09-19T07:55:02.969267+0000","last_became_active":"2024-09-19T07:55:01.983553+0000","last_became_peered":"2024-09-19T07:55:01.983553+0000","last_unstale":"2024-09-19T07:55:02.969267+0000","last_undegraded":"2024-09-19T07:55:02.969267+0000","last_fullsized":"2024-09-19T07:55:02.969267+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:02.576442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.986319+0000","last_change":"2024-09-19T07:55:02.983084+0000","last_active":"2024-09-19T07:55:02.986319+0000","last_peered":"2024-09-19T07:55:02.986319+0000","last_clean":"2024-09-19T07:55:02.986319+0000","last_became_active":"2024-09-19T07:55:02.982793+0000","last_became_peered":"2024-09-19T07:55:02.982793+0000","last_unstale":"2024-09-19T07:55:02.986319+0000","last_undegraded":"2024-09-19T07:55:02.986319+0000","last_fullsized":"2024-09-19T07:55:02.986319+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:39:22.911029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983489+0000","last_change":"2024-09-19T07:55:01.980547+0000","last_active":"2024-09-19T07:55:03.983489+0000","last_peered":"2024-09-19T07:55:03.983489+0000","last_clean":"2024-09-19T07:55:03.983489+0000","last_became_active":"2024-09-19T07:55:01.980253+0000","last_became_peered":"2024-09-19T07:55:01.980253+0000","last_unstale":"2024-09-19T07:55:03.983489+0000","last_undegraded":"2024-09-19T07:55:03.983489+0000","last_fullsized":"2024-09-19T07:55:03.983489+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:32.674094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:01.959847+0000","last_change":"2024-09-19T07:55:01.959847+0000","last_active":"2024-09-19T07:55:01.959847+0000","last_peered":"2024-09-19T07:55:01.959847+0000","last_clean":"2024-09-19T07:55:01.959847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:01.959847+0000","last_undegraded":"2024-09-19T07:55:01.959847+0000","last_fullsized":"2024-09-19T07:55:01.959847+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.973223+0000","last_change":"2024-09-19T07:55:01.980870+0000","last_active":"2024-09-19T07:55:03.973223+0000","last_peered":"2024-09-19T07:55:03.973223+0000","last_clean":"2024-09-19T07:55:03.973223+0000","last_became_active":"2024-09-19T07:55:01.980569+0000","last_became_peered":"2024-09-19T07:55:01.980569+0000","last_unstale":"2024-09-19T07:55:03.973223+0000","last_undegraded":"2024-09-19T07:55:03.973223+0000","last_fullsized":"2024-09-19T07:55:03.973223+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:56:13.952641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:01.959847+0000","last_change":"2024-09-19T07:55:01.959847+0000","last_active":"2024-09-19T07:55:01.959847+0000","last_peered":"2024-09-19T07:55:01.959847+0000","last_clean":"2024-09-19T07:55:01.959847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:01.959847+0000","last_undegraded":"2024-09-19T07:55:01.959847+0000","last_fullsized":"2024-09-19T07:55:01.959847+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-19T07:55:00.965215+0000","last_change":"2024-09-19T07:55:00.965215+0000","last_active":"2024-09-19T07:55:00.954535+0000","last_peered":"2024-09-19T07:55:00.954535+0000","last_clean":"2024-09-19T07:55:00.954535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:00.965215+0000","last_undegraded":"2024-09-19T07:55:00.965215+0000","last_fullsized":"2024-09-19T07:55:00.965215+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.986247+0000","last_change":"2024-09-19T07:55:02.980794+0000","last_active":"2024-09-19T07:55:02.986247+0000","last_peered":"2024-09-19T07:55:02.986247+0000","last_clean":"2024-09-19T07:55:02.986247+0000","last_became_active":"2024-09-19T07:55:02.980530+0000","last_became_peered":"2024-09-19T07:55:02.980530+0000","last_unstale":"2024-09-19T07:55:02.986247+0000","last_undegraded":"2024-09-19T07:55:02.986247+0000","last_fullsized":"2024-09-19T07:55:02.986247+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:14.838884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"139'2","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000534+0000","last_change":"2024-09-19T07:55:01.984318+0000","last_active":"2024-09-19T07:55:03.000534+0000","last_peered":"2024-09-19T07:55:03.000534+0000","last_clean":"2024-09-19T07:55:03.000534+0000","last_became_active":"2024-09-19T07:55:01.983325+0000","last_became_peered":"2024-09-19T07:55:01.983325+0000","last_unstale":"2024-09-19T07:55:03.000534+0000","last_undegraded":"2024-09-19T07:55:03.000534+0000","last_fullsized":"2024-09-19T07:55:03.000534+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:36.086291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.999112+0000","last_change":"2024-09-19T07:55:02.996430+0000","last_active":"2024-09-19T07:55:02.999112+0000","last_peered":"2024-09-19T07:55:02.999112+0000","last_clean":"2024-09-19T07:55:02.999112+0000","last_became_active":"2024-09-19T07:55:02.996148+0000","last_became_peered":"2024-09-19T07:55:02.996148+0000","last_unstale":"2024-09-19T07:55:02.999112+0000","last_undegraded":"2024-09-19T07:55:02.999112+0000","last_fullsized":"2024-09-19T07:55:02.999112+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:12.728223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972021+0000","last_change":"2024-09-19T07:55:01.988068+0000","last_active":"2024-09-19T07:55:03.972021+0000","last_peered":"2024-09-19T07:55:03.972021+0000","last_clean":"2024-09-19T07:55:03.972021+0000","last_became_active":"2024-09-19T07:55:01.987761+0000","last_became_peered":"2024-09-19T07:55:01.987761+0000","last_unstale":"2024-09-19T07:55:03.972021+0000","last_undegraded":"2024-09-19T07:55:03.972021+0000","last_fullsized":"2024-09-19T07:55:03.972021+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:56.300407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.994831+0000","last_change":"2024-09-19T07:55:02.989642+0000","last_active":"2024-09-19T07:55:02.994831+0000","last_peered":"2024-09-19T07:55:02.994831+0000","last_clean":"2024-09-19T07:55:02.994831+0000","last_became_active":"2024-09-19T07:55:02.989315+0000","last_became_peered":"2024-09-19T07:55:02.989315+0000","last_unstale":"2024-09-19T07:55:02.994831+0000","last_undegraded":"2024-09-19T07:55:02.994831+0000","last_fullsized":"2024-09-19T07:55:02.994831+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:29.434870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972162+0000","last_change":"2024-09-19T07:55:01.983411+0000","last_active":"2024-09-19T07:55:03.972162+0000","last_peered":"2024-09-19T07:55:03.972162+0000","last_clean":"2024-09-19T07:55:03.972162+0000","last_became_active":"2024-09-19T07:55:01.982413+0000","last_became_peered":"2024-09-19T07:55:01.982413+0000","last_unstale":"2024-09-19T07:55:03.972162+0000","last_undegraded":"2024-09-19T07:55:03.972162+0000","last_fullsized":"2024-09-19T07:55:03.972162+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:58.316347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.985547+0000","last_change":"2024-09-19T07:55:02.983361+0000","last_active":"2024-09-19T07:55:02.985547+0000","last_peered":"2024-09-19T07:55:02.985547+0000","last_clean":"2024-09-19T07:55:02.985547+0000","last_became_active":"2024-09-19T07:55:02.983032+0000","last_became_peered":"2024-09-19T07:55:02.983032+0000","last_unstale":"2024-09-19T07:55:02.985547+0000","last_undegraded":"2024-09-19T07:55:02.985547+0000","last_fullsized":"2024-09-19T07:55:02.985547+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:56:14.957952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"139'8","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.971849+0000","last_change":"2024-09-19T07:55:01.978831+0000","last_active":"2024-09-19T07:55:03.971849+0000","last_peered":"2024-09-19T07:55:03.971849+0000","last_clean":"2024-09-19T07:55:03.971849+0000","last_became_active":"2024-09-19T07:55:01.978370+0000","last_became_peered":"2024-09-19T07:55:01.978370+0000","last_unstale":"2024-09-19T07:55:03.971849+0000","last_undegraded":"2024-09-19T07:55:03.971849+0000","last_fullsized":"2024-09-19T07:55:03.971849+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:47.640841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983367+0000","last_change":"2024-09-19T07:55:02.998722+0000","last_active":"2024-09-19T07:55:03.983367+0000","last_peered":"2024-09-19T07:55:03.983367+0000","last_clean":"2024-09-19T07:55:03.983367+0000","last_became_active":"2024-09-19T07:55:02.998420+0000","last_became_peered":"2024-09-19T07:55:02.998420+0000","last_unstale":"2024-09-19T07:55:03.983367+0000","last_undegraded":"2024-09-19T07:55:03.983367+0000","last_fullsized":"2024-09-19T07:55:03.983367+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:02.489166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-19T07:55:00.965654+0000","last_change":"2024-09-19T07:55:00.965654+0000","last_active":"2024-09-19T07:55:00.954535+0000","last_peered":"2024-09-19T07:55:00.954535+0000","last_clean":"2024-09-19T07:55:00.954535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:00.965654+0000","last_undegraded":"2024-09-19T07:55:00.965654+0000","last_fullsized":"2024-09-19T07:55:00.965654+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983513+0000","last_change":"2024-09-19T07:55:02.988579+0000","last_active":"2024-09-19T07:55:03.983513+0000","last_peered":"2024-09-19T07:55:03.983513+0000","last_clean":"2024-09-19T07:55:03.983513+0000","last_became_active":"2024-09-19T07:55:02.987958+0000","last_became_peered":"2024-09-19T07:55:02.987958+0000","last_unstale":"2024-09-19T07:55:03.983513+0000","last_undegraded":"2024-09-19T07:55:03.983513+0000","last_fullsized":"2024-09-19T07:55:03.983513+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:08.519276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983438+0000","last_change":"2024-09-19T07:55:01.981723+0000","last_active":"2024-09-19T07:55:03.983438+0000","last_peered":"2024-09-19T07:55:03.983438+0000","last_clean":"2024-09-19T07:55:03.983438+0000","last_became_active":"2024-09-19T07:55:01.981465+0000","last_became_peered":"2024-09-19T07:55:01.981465+0000","last_unstale":"2024-09-19T07:55:03.983438+0000","last_undegraded":"2024-09-19T07:55:03.983438+0000","last_fullsized":"2024-09-19T07:55:03.983438+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:12.838363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.973123+0000","last_change":"2024-09-19T07:55:02.979256+0000","last_active":"2024-09-19T07:55:03.973123+0000","last_peered":"2024-09-19T07:55:03.973123+0000","last_clean":"2024-09-19T07:55:03.973123+0000","last_became_active":"2024-09-19T07:55:02.978921+0000","last_became_peered":"2024-09-19T07:55:02.978921+0000","last_unstale":"2024-09-19T07:55:03.973123+0000","last_undegraded":"2024-09-19T07:55:03.973123+0000","last_fullsized":"2024-09-19T07:55:03.973123+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:51.978120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"139'1","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000384+0000","last_change":"2024-09-19T07:55:01.982720+0000","last_active":"2024-09-19T07:55:03.000384+0000","last_peered":"2024-09-19T07:55:03.000384+0000","last_clean":"2024-09-19T07:55:03.000384+0000","last_became_active":"2024-09-19T07:55:01.982277+0000","last_became_peered":"2024-09-19T07:55:01.982277+0000","last_unstale":"2024-09-19T07:55:03.000384+0000","last_undegraded":"2024-09-19T07:55:03.000384+0000","last_fullsized":"2024-09-19T07:55:03.000384+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:59.197262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972931+0000","last_change":"2024-09-19T07:55:02.989059+0000","last_active":"2024-09-19T07:55:03.972931+0000","last_peered":"2024-09-19T07:55:03.972931+0000","last_clean":"2024-09-19T07:55:03.972931+0000","last_became_active":"2024-09-19T07:55:02.988577+0000","last_became_peered":"2024-09-19T07:55:02.988577+0000","last_unstale":"2024-09-19T07:55:03.972931+0000","last_undegraded":"2024-09-19T07:55:03.972931+0000","last_fullsized":"2024-09-19T07:55:03.972931+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:49.502676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972169+0000","last_change":"2024-09-19T07:55:01.988779+0000","last_active":"2024-09-19T07:55:03.972169+0000","last_peered":"2024-09-19T07:55:03.972169+0000","last_clean":"2024-09-19T07:55:03.972169+0000","last_became_active":"2024-09-19T07:55:01.988419+0000","last_became_peered":"2024-09-19T07:55:01.988419+0000","last_unstale":"2024-09-19T07:55:03.972169+0000","last_undegraded":"2024-09-19T07:55:03.972169+0000","last_fullsized":"2024-09-19T07:55:03.972169+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:59.785587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.982978+0000","last_change":"2024-09-19T07:55:02.987148+0000","last_active":"2024-09-19T07:55:03.982978+0000","last_peered":"2024-09-19T07:55:03.982978+0000","last_clean":"2024-09-19T07:55:03.982978+0000","last_became_active":"2024-09-19T07:55:02.986889+0000","last_became_peered":"2024-09-19T07:55:02.986889+0000","last_unstale":"2024-09-19T07:55:03.982978+0000","last_undegraded":"2024-09-19T07:55:03.982978+0000","last_fullsized":"2024-09-19T07:55:03.982978+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:26:18.944316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"139'2","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.997309+0000","last_change":"2024-09-19T07:55:01.986115+0000","last_active":"2024-09-19T07:55:02.997309+0000","last_peered":"2024-09-19T07:55:02.997309+0000","last_clean":"2024-09-19T07:55:02.997309+0000","last_became_active":"2024-09-19T07:55:01.985827+0000","last_became_peered":"2024-09-19T07:55:01.985827+0000","last_unstale":"2024-09-19T07:55:02.997309+0000","last_undegraded":"2024-09-19T07:55:02.997309+0000","last_fullsized":"2024-09-19T07:55:02.997309+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:01.483855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.989398+0000","last_change":"2024-09-19T07:55:02.987365+0000","last_active":"2024-09-19T07:55:02.989398+0000","last_peered":"2024-09-19T07:55:02.989398+0000","last_clean":"2024-09-19T07:55:02.989398+0000","last_became_active":"2024-09-19T07:55:02.986480+0000","last_became_peered":"2024-09-19T07:55:02.986480+0000","last_unstale":"2024-09-19T07:55:02.989398+0000","last_undegraded":"2024-09-19T07:55:02.989398+0000","last_fullsized":"2024-09-19T07:55:02.989398+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:32.859864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.972920+0000","last_change":"2024-09-19T07:55:01.981717+0000","last_active":"2024-09-19T07:55:02.972920+0000","last_peered":"2024-09-19T07:55:02.972920+0000","last_clean":"2024-09-19T07:55:02.972920+0000","last_became_active":"2024-09-19T07:55:01.981004+0000","last_became_peered":"2024-09-19T07:55:01.981004+0000","last_unstale":"2024-09-19T07:55:02.972920+0000","last_undegraded":"2024-09-19T07:55:02.972920+0000","last_fullsized":"2024-09-19T07:55:02.972920+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:58.316347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983112+0000","last_change":"2024-09-19T07:55:02.992093+0000","last_active":"2024-09-19T07:55:03.983112+0000","last_peered":"2024-09-19T07:55:03.983112+0000","last_clean":"2024-09-19T07:55:03.983112+0000","last_became_active":"2024-09-19T07:55:02.991829+0000","last_became_peered":"2024-09-19T07:55:02.991829+0000","last_unstale":"2024-09-19T07:55:03.983112+0000","last_undegraded":"2024-09-19T07:55:03.983112+0000","last_fullsized":"2024-09-19T07:55:03.983112+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:59:18.848159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.974687+0000","last_change":"2024-09-19T07:55:01.983244+0000","last_active":"2024-09-19T07:55:02.974687+0000","last_peered":"2024-09-19T07:55:02.974687+0000","last_clean":"2024-09-19T07:55:02.974687+0000","last_became_active":"2024-09-19T07:55:01.982960+0000","last_became_peered":"2024-09-19T07:55:01.982960+0000","last_unstale":"2024-09-19T07:55:02.974687+0000","last_undegraded":"2024-09-19T07:55:02.974687+0000","last_fullsized":"2024-09-19T07:55:02.974687+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:26:17.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998969+0000","last_change":"2024-09-19T07:55:02.993715+0000","last_active":"2024-09-19T07:55:02.998969+0000","last_peered":"2024-09-19T07:55:02.998969+0000","last_clean":"2024-09-19T07:55:02.998969+0000","last_became_active":"2024-09-19T07:55:02.993409+0000","last_became_peered":"2024-09-19T07:55:02.993409+0000","last_unstale":"2024-09-19T07:55:02.998969+0000","last_undegraded":"2024-09-19T07:55:02.998969+0000","last_fullsized":"2024-09-19T07:55:02.998969+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:13.897172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.974976+0000","last_change":"2024-09-19T07:55:01.982335+0000","last_active":"2024-09-19T07:55:02.974976+0000","last_peered":"2024-09-19T07:55:02.974976+0000","last_clean":"2024-09-19T07:55:02.974976+0000","last_became_active":"2024-09-19T07:55:01.982052+0000","last_became_peered":"2024-09-19T07:55:01.982052+0000","last_unstale":"2024-09-19T07:55:02.974976+0000","last_undegraded":"2024-09-19T07:55:02.974976+0000","last_fullsized":"2024-09-19T07:55:02.974976+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:59.785587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.975028+0000","last_change":"2024-09-19T07:55:02.974014+0000","last_active":"2024-09-19T07:55:02.975028+0000","last_peered":"2024-09-19T07:55:02.975028+0000","last_clean":"2024-09-19T07:55:02.975028+0000","last_became_active":"2024-09-19T07:55:02.973536+0000","last_became_peered":"2024-09-19T07:55:02.973536+0000","last_unstale":"2024-09-19T07:55:02.975028+0000","last_undegraded":"2024-09-19T07:55:02.975028+0000","last_fullsized":"2024-09-19T07:55:02.975028+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:55.597883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972720+0000","last_change":"2024-09-19T07:55:01.988008+0000","last_active":"2024-09-19T07:55:03.972720+0000","last_peered":"2024-09-19T07:55:03.972720+0000","last_clean":"2024-09-19T07:55:03.972720+0000","last_became_active":"2024-09-19T07:55:01.987725+0000","last_became_peered":"2024-09-19T07:55:01.987725+0000","last_unstale":"2024-09-19T07:55:03.972720+0000","last_undegraded":"2024-09-19T07:55:03.972720+0000","last_fullsized":"2024-09-19T07:55:03.972720+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:13.833572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972516+0000","last_change":"2024-09-19T07:55:02.991309+0000","last_active":"2024-09-19T07:55:03.972516+0000","last_peered":"2024-09-19T07:55:03.972516+0000","last_clean":"2024-09-19T07:55:03.972516+0000","last_became_active":"2024-09-19T07:55:02.991045+0000","last_became_peered":"2024-09-19T07:55:02.991045+0000","last_unstale":"2024-09-19T07:55:03.972516+0000","last_undegraded":"2024-09-19T07:55:03.972516+0000","last_fullsized":"2024-09-19T07:55:03.972516+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:38.709571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"140'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.982444+0000","last_change":"2024-09-19T07:55:01.978226+0000","last_active":"2024-09-19T07:55:03.982444+0000","last_peered":"2024-09-19T07:55:03.982444+0000","last_clean":"2024-09-19T07:55:03.982444+0000","last_became_active":"2024-09-19T07:55:01.977803+0000","last_became_peered":"2024-09-19T07:55:01.977803+0000","last_unstale":"2024-09-19T07:55:03.982444+0000","last_undegraded":"2024-09-19T07:55:03.982444+0000","last_fullsized":"2024-09-19T07:55:03.982444+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:49.852983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.973122+0000","last_change":"2024-09-19T07:55:02.977147+0000","last_active":"2024-09-19T07:55:03.973122+0000","last_peered":"2024-09-19T07:55:03.973122+0000","last_clean":"2024-09-19T07:55:03.973122+0000","last_became_active":"2024-09-19T07:55:02.976832+0000","last_became_peered":"2024-09-19T07:55:02.976832+0000","last_unstale":"2024-09-19T07:55:03.973122+0000","last_undegraded":"2024-09-19T07:55:03.973122+0000","last_fullsized":"2024-09-19T07:55:03.973122+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:03.581754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.974160+0000","last_change":"2024-09-19T07:55:01.974880+0000","last_active":"2024-09-19T07:55:02.974160+0000","last_peered":"2024-09-19T07:55:02.974160+0000","last_clean":"2024-09-19T07:55:02.974160+0000","last_became_active":"2024-09-19T07:55:01.974599+0000","last_became_peered":"2024-09-19T07:55:01.974599+0000","last_unstale":"2024-09-19T07:55:02.974160+0000","last_undegraded":"2024-09-19T07:55:02.974160+0000","last_fullsized":"2024-09-19T07:55:02.974160+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:11.363759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.994783+0000","last_change":"2024-09-19T07:55:02.989699+0000","last_active":"2024-09-19T07:55:02.994783+0000","last_peered":"2024-09-19T07:55:02.994783+0000","last_clean":"2024-09-19T07:55:02.994783+0000","last_became_active":"2024-09-19T07:55:02.989398+0000","last_became_peered":"2024-09-19T07:55:02.989398+0000","last_unstale":"2024-09-19T07:55:02.994783+0000","last_undegraded":"2024-09-19T07:55:02.994783+0000","last_fullsized":"2024-09-19T07:55:02.994783+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:11.153885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.970064+0000","last_change":"2024-09-19T07:55:01.981690+0000","last_active":"2024-09-19T07:55:02.970064+0000","last_peered":"2024-09-19T07:55:02.970064+0000","last_clean":"2024-09-19T07:55:02.970064+0000","last_became_active":"2024-09-19T07:55:01.981417+0000","last_became_peered":"2024-09-19T07:55:01.981417+0000","last_unstale":"2024-09-19T07:55:02.970064+0000","last_undegraded":"2024-09-19T07:55:02.970064+0000","last_fullsized":"2024-09-19T07:55:02.970064+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:26:17.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.988647+0000","last_change":"2024-09-19T07:55:02.986076+0000","last_active":"2024-09-19T07:55:02.988647+0000","last_peered":"2024-09-19T07:55:02.988647+0000","last_clean":"2024-09-19T07:55:02.988647+0000","last_became_active":"2024-09-19T07:55:02.984885+0000","last_became_peered":"2024-09-19T07:55:02.984885+0000","last_unstale":"2024-09-19T07:55:02.988647+0000","last_undegraded":"2024-09-19T07:55:02.988647+0000","last_fullsized":"2024-09-19T07:55:02.988647+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:33.113553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972219+0000","last_change":"2024-09-19T07:55:01.986920+0000","last_active":"2024-09-19T07:55:03.972219+0000","last_peered":"2024-09-19T07:55:03.972219+0000","last_clean":"2024-09-19T07:55:03.972219+0000","last_became_active":"2024-09-19T07:55:01.986618+0000","last_became_peered":"2024-09-19T07:55:01.986618+0000","last_unstale":"2024-09-19T07:55:03.972219+0000","last_undegraded":"2024-09-19T07:55:03.972219+0000","last_fullsized":"2024-09-19T07:55:03.972219+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:59.197262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972053+0000","last_change":"2024-09-19T07:55:02.989902+0000","last_active":"2024-09-19T07:55:03.972053+0000","last_peered":"2024-09-19T07:55:03.972053+0000","last_clean":"2024-09-19T07:55:03.972053+0000","last_became_active":"2024-09-19T07:55:02.987293+0000","last_became_peered":"2024-09-19T07:55:02.987293+0000","last_unstale":"2024-09-19T07:55:03.972053+0000","last_undegraded":"2024-09-19T07:55:03.972053+0000","last_fullsized":"2024-09-19T07:55:03.972053+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:54:28.829885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-19T07:55:00.965812+0000","last_change":"2024-09-19T07:55:00.965812+0000","last_active":"2024-09-19T07:55:00.954535+0000","last_peered":"2024-09-19T07:55:00.954535+0000","last_clean":"2024-09-19T07:55:00.954535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:00.965812+0000","last_undegraded":"2024-09-19T07:55:00.965812+0000","last_fullsized":"2024-09-19T07:55:00.965812+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.987084+0000","last_change":"2024-09-19T07:55:02.983221+0000","last_active":"2024-09-19T07:55:02.987084+0000","last_peered":"2024-09-19T07:55:02.987084+0000","last_clean":"2024-09-19T07:55:02.987084+0000","last_became_active":"2024-09-19T07:55:02.982829+0000","last_became_peered":"2024-09-19T07:55:02.982829+0000","last_unstale":"2024-09-19T07:55:02.987084+0000","last_undegraded":"2024-09-19T07:55:02.987084+0000","last_fullsized":"2024-09-19T07:55:02.987084+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:18.210530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972383+0000","last_change":"2024-09-19T07:55:01.980107+0000","last_active":"2024-09-19T07:55:03.972383+0000","last_peered":"2024-09-19T07:55:03.972383+0000","last_clean":"2024-09-19T07:55:03.972383+0000","last_became_active":"2024-09-19T07:55:01.979828+0000","last_became_peered":"2024-09-19T07:55:01.979828+0000","last_unstale":"2024-09-19T07:55:03.972383+0000","last_undegraded":"2024-09-19T07:55:03.972383+0000","last_fullsized":"2024-09-19T07:55:03.972383+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:17.205219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.988702+0000","last_change":"2024-09-19T07:55:02.986047+0000","last_active":"2024-09-19T07:55:02.988702+0000","last_peered":"2024-09-19T07:55:02.988702+0000","last_clean":"2024-09-19T07:55:02.988702+0000","last_became_active":"2024-09-19T07:55:02.985768+0000","last_became_peered":"2024-09-19T07:55:02.985768+0000","last_unstale":"2024-09-19T07:55:02.988702+0000","last_undegraded":"2024-09-19T07:55:02.988702+0000","last_fullsized":"2024-09-19T07:55:02.988702+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:27:57.147706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.982949+0000","last_change":"2024-09-19T07:55:01.980963+0000","last_active":"2024-09-19T07:55:03.982949+0000","last_peered":"2024-09-19T07:55:03.982949+0000","last_clean":"2024-09-19T07:55:03.982949+0000","last_became_active":"2024-09-19T07:55:01.980672+0000","last_became_peered":"2024-09-19T07:55:01.980672+0000","last_unstale":"2024-09-19T07:55:03.982949+0000","last_undegraded":"2024-09-19T07:55:03.982949+0000","last_fullsized":"2024-09-19T07:55:03.982949+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:11.722912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.988060+0000","last_change":"2024-09-19T07:55:02.985023+0000","last_active":"2024-09-19T07:55:02.988060+0000","last_peered":"2024-09-19T07:55:02.988060+0000","last_clean":"2024-09-19T07:55:02.988060+0000","last_became_active":"2024-09-19T07:55:02.984699+0000","last_became_peered":"2024-09-19T07:55:02.984699+0000","last_unstale":"2024-09-19T07:55:02.988060+0000","last_undegraded":"2024-09-19T07:55:02.988060+0000","last_fullsized":"2024-09-19T07:55:02.988060+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:06.465142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"139'6","reported_seq":18,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000873+0000","last_change":"2024-09-19T07:55:01.982519+0000","last_active":"2024-09-19T07:55:03.000873+0000","last_peered":"2024-09-19T07:55:03.000873+0000","last_clean":"2024-09-19T07:55:03.000873+0000","last_became_active":"2024-09-19T07:55:01.982223+0000","last_became_peered":"2024-09-19T07:55:01.982223+0000","last_unstale":"2024-09-19T07:55:03.000873+0000","last_undegraded":"2024-09-19T07:55:03.000873+0000","last_fullsized":"2024-09-19T07:55:03.000873+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:59:17.842847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":387,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983009+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:55:03.983009+0000","last_peered":"2024-09-19T07:55:03.983009+0000","last_clean":"2024-09-19T07:55:03.983009+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:55:03.983009+0000","last_undegraded":"2024-09-19T07:55:03.983009+0000","last_fullsized":"2024-09-19T07:55:03.983009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2298,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83384,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288456,"statfs":{"total":96636764160,"available":96551378944,"internally_reserved":0,"allocated":2334720,"data_stored":1397055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509588,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105080,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266760,"statfs":{"total":96636764160,"available":96529162240,"internally_reserved":0,"allocated":1740800,"data_stored":806735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509589,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73088,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298752,"statfs":{"total":96636764160,"available":96561922048,"internally_reserved":0,"allocated":1748992,"data_stored":813357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509589,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86984,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284856,"statfs":{"total":96636764160,"available":96547692544,"internally_reserved":0,"allocated":1748992,"data_stored":808277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509588,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105620,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266220,"statfs":{"total":96636764160,"available":96528609280,"internally_reserved":0,"allocated":2355200,"data_stored":1399351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542292,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118916,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252924,"statfs":{"total":96636764160,"available":96514994176,"internally_reserved":0,"allocated":1744896,"data_stored":807281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542292,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82880,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288960,"statfs":{"total":96636764160,"available":96551895040,"internally_reserved":0,"allocated":1740800,"data_stored":806735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109152,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262688,"statfs":{"total":96636764160,"available":96524992512,"internally_reserved":0,"allocated":1712128,"data_stored":792831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:55:06.316 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:55:06.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.454+0000 7f07f0d8f640 1 Processor -- start 2024-09-19T07:55:06.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.454+0000 7f07f0d8f640 1 -- start start 2024-09-19T07:55:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.455+0000 7f07f0d8f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07ec13f0c0 0x7f07ec15d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.455+0000 7f07f0d8f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07ec1419d0 0x7f07ec141e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.455+0000 7f07f0d8f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07ec15d6b0 con 0x7f07ec15fce0 2024-09-19T07:55:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.455+0000 7f07f0d8f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f07ec142370 con 0x7f07ec1419d0 2024-09-19T07:55:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.455+0000 7f07f0d8f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f07ec072450 con 0x7f07ec13f0c0 2024-09-19T07:55:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.455+0000 7f07e3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07ec1419d0 0x7f07ec141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.455+0000 7f07ead9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f07ec15fce0 0x7f07ec160100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34020/0 (socket says 172.21.15.16:34020) 2024-09-19T07:55:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.455+0000 7f07ead9d640 1 -- 172.21.15.16:0/3637926469 learned_addr learned my addr 172.21.15.16:0/3637926469 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.455+0000 7f07ea59c640 1 --2- 172.21.15.16:0/3637926469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07ec13f0c0 0x7f07ec15d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.455+0000 7f07e3fff640 1 -- 172.21.15.16:0/3637926469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07ec13f0c0 msgr2=0x7f07ec15d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.455+0000 7f07e3fff640 1 --2- 172.21.15.16:0/3637926469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07ec13f0c0 0x7f07ec15d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.455+0000 7f07e3fff640 1 -- 172.21.15.16:0/3637926469 >> v1:172.21.15.16:6789/0 conn(0x7f07ec15fce0 legacy=0x7f07ec160100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.455+0000 7f07e9d9b640 1 -- 172.21.15.16:0/3637926469 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2950171520 0 0) 0x7f07e4001490 con 0x7f07ec15fce0 2024-09-19T07:55:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.455+0000 7f07e3fff640 1 -- 172.21.15.16:0/3637926469 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ec072730 con 0x7f07ec1419d0 2024-09-19T07:55:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.456+0000 7f07e3fff640 1 --2- 172.21.15.16:0/3637926469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ec1419d0 0x7f07ec141e30 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f07d40097a0 tx=0x7f07d402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad87321e57bcd781 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.456+0000 7f07e9d9b640 1 -- 172.21.15.16:0/3637926469 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4037070 con 0x7f07ec1419d0 2024-09-19T07:55:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.456+0000 7f07e9d9b640 1 -- 172.21.15.16:0/3637926469 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07d4032440 con 0x7f07ec1419d0 2024-09-19T07:55:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.456+0000 7f07e9d9b640 1 -- 172.21.15.16:0/3637926469 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4032810 con 0x7f07ec1419d0 2024-09-19T07:55:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.457+0000 7f07f0d8f640 1 -- 172.21.15.16:0/3637926469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ec1419d0 msgr2=0x7f07ec141e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.457+0000 7f07f0d8f640 1 --2- 172.21.15.16:0/3637926469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ec1419d0 0x7f07ec141e30 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f07d40097a0 tx=0x7f07d402a180 comp rx=0 tx=0).stop 2024-09-19T07:55:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.457+0000 7f07f0d8f640 1 -- 172.21.15.16:0/3637926469 shutdown_connections 2024-09-19T07:55:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.457+0000 7f07f0d8f640 1 --2- 172.21.15.16:0/3637926469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ec1419d0 0x7f07ec141e30 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.457+0000 7f07f0d8f640 1 --2- 172.21.15.16:0/3637926469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07ec13f0c0 0x7f07ec15d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.457+0000 7f07f0d8f640 1 -- 172.21.15.16:0/3637926469 >> 172.21.15.16:0/3637926469 conn(0x7f07ec067d70 msgr2=0x7f07ec06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.457+0000 7f07f0d8f640 1 -- 172.21.15.16:0/3637926469 shutdown_connections 2024-09-19T07:55:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.457+0000 7f07f0d8f640 1 -- 172.21.15.16:0/3637926469 wait complete. 2024-09-19T07:55:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.458+0000 7f07f0d8f640 1 Processor -- start 2024-09-19T07:55:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.458+0000 7f07f0d8f640 1 -- start start 2024-09-19T07:55:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.459+0000 7f07f0d8f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ec13f0c0 0x7f07ec1e65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.459+0000 7f07f0d8f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07ec1419d0 0x7f07ec0b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.459+0000 7f07f0d8f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07ec15fce0 0x7f07ec06f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.459+0000 7f07f0d8f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec072fd0 con 0x7f07ec13f0c0 2024-09-19T07:55:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.459+0000 7f07f0d8f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec072cf0 con 0x7f07ec1419d0 2024-09-19T07:55:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.459+0000 7f07f0d8f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f07ec1427a0 con 0x7f07ec15fce0 2024-09-19T07:55:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.459+0000 7f07ea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ec13f0c0 0x7f07ec1e65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.459+0000 7f07ea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ec13f0c0 0x7f07ec1e65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43794/0 (socket says 172.21.15.16:43794) 2024-09-19T07:55:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.459+0000 7f07ea59c640 1 -- 172.21.15.16:0/947786116 learned_addr learned my addr 172.21.15.16:0/947786116 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.459+0000 7f07ead9d640 1 --2- 172.21.15.16:0/947786116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07ec15fce0 0x7f07ec06f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.459+0000 7f07ea59c640 1 -- 172.21.15.16:0/947786116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07ec15fce0 msgr2=0x7f07ec06f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.459+0000 7f07e3fff640 1 --2- 172.21.15.16:0/947786116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07ec1419d0 0x7f07ec0b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.459+0000 7f07ea59c640 1 --2- 172.21.15.16:0/947786116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07ec15fce0 0x7f07ec06f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.459+0000 7f07ea59c640 1 -- 172.21.15.16:0/947786116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07ec1419d0 msgr2=0x7f07ec0b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.459+0000 7f07ea59c640 1 --2- 172.21.15.16:0/947786116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07ec1419d0 0x7f07ec0b20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.459+0000 7f07ea59c640 1 -- 172.21.15.16:0/947786116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ec1e5b20 con 0x7f07ec13f0c0 2024-09-19T07:55:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.460+0000 7f07ea59c640 1 --2- 172.21.15.16:0/947786116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ec13f0c0 0x7f07ec1e65c0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00d800 tx=0x7f07dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.460+0000 7f07e2ffd640 1 -- 172.21.15.16:0/947786116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc00b660 con 0x7f07ec13f0c0 2024-09-19T07:55:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.460+0000 7f07e2ffd640 1 -- 172.21.15.16:0/947786116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07dc019070 con 0x7f07ec13f0c0 2024-09-19T07:55:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.460+0000 7f07e2ffd640 1 -- 172.21.15.16:0/947786116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc00ba20 con 0x7f07ec13f0c0 2024-09-19T07:55:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.460+0000 7f07f0d8f640 1 -- 172.21.15.16:0/947786116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07ec0b2ba0 con 0x7f07ec13f0c0 2024-09-19T07:55:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.460+0000 7f07f0d8f640 1 -- 172.21.15.16:0/947786116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07ec1e60a0 con 0x7f07ec13f0c0 2024-09-19T07:55:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.461+0000 7f07f0d8f640 1 -- 172.21.15.16:0/947786116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07ec0720b0 con 0x7f07ec13f0c0 2024-09-19T07:55:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.461+0000 7f07e2ffd640 1 -- 172.21.15.16:0/947786116 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f07dc00bbc0 con 0x7f07ec13f0c0 2024-09-19T07:55:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.462+0000 7f07e2ffd640 1 --2- 172.21.15.16:0/947786116 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f07cc06c2e0 0x7f07cc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.462+0000 7f07e2ffd640 1 -- 172.21.15.16:0/947786116 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f07dc08b970 con 0x7f07ec13f0c0 2024-09-19T07:55:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.462+0000 7f07e3fff640 1 --2- 172.21.15.16:0/947786116 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f07cc06c2e0 0x7f07cc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:06.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.463+0000 7f07e3fff640 1 --2- 172.21.15.16:0/947786116 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f07cc06c2e0 0x7f07cc06e7a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f07d4034000 tx=0x7f07d402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:06.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.466+0000 7f07e2ffd640 1 -- 172.21.15.16:0/947786116 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f07dc01c360 con 0x7f07ec13f0c0 2024-09-19T07:55:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.638+0000 7f07f0d8f640 1 -- 172.21.15.16:0/947786116 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f07ec15a730 con 0x7f07cc06c2e0 2024-09-19T07:55:06.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.645+0000 7f07e2ffd640 1 -- 172.21.15.16:0/947786116 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252457 (secure 0 0 0) 0x7f07ec15a730 con 0x7f07cc06c2e0 2024-09-19T07:55:06.647 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:06.649 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:55:06.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.647+0000 7f07f0d8f640 1 -- 172.21.15.16:0/947786116 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f07cc06c2e0 msgr2=0x7f07cc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.647+0000 7f07f0d8f640 1 --2- 172.21.15.16:0/947786116 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f07cc06c2e0 0x7f07cc06e7a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f07d4034000 tx=0x7f07d402e000 comp rx=0 tx=0).stop 2024-09-19T07:55:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.647+0000 7f07f0d8f640 1 -- 172.21.15.16:0/947786116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ec13f0c0 msgr2=0x7f07ec1e65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.647+0000 7f07f0d8f640 1 --2- 172.21.15.16:0/947786116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ec13f0c0 0x7f07ec1e65c0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00d800 tx=0x7f07dc00dcc0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.647+0000 7f07f0d8f640 1 -- 172.21.15.16:0/947786116 shutdown_connections 2024-09-19T07:55:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.647+0000 7f07f0d8f640 1 --2- 172.21.15.16:0/947786116 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f07cc06c2e0 0x7f07cc06e7a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.647+0000 7f07f0d8f640 1 --2- 172.21.15.16:0/947786116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07ec15fce0 0x7f07ec06f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.647+0000 7f07f0d8f640 1 --2- 172.21.15.16:0/947786116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07ec1419d0 0x7f07ec0b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.647+0000 7f07f0d8f640 1 --2- 172.21.15.16:0/947786116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ec13f0c0 0x7f07ec1e65c0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.648+0000 7f07f0d8f640 1 -- 172.21.15.16:0/947786116 >> 172.21.15.16:0/947786116 conn(0x7f07ec067d70 msgr2=0x7f07ec15f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.648+0000 7f07f0d8f640 1 -- 172.21.15.16:0/947786116 shutdown_connections 2024-09-19T07:55:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.648+0000 7f07f0d8f640 1 -- 172.21.15.16:0/947786116 wait complete. 2024-09-19T07:55:06.659 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":501,"stamp":"2024-09-19T07:55:06.629540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":765144,"kb_used_data":14816,"kb_used_omap":12,"kb_used_meta":750131,"kb_avail":754209576,"statfs":{"total":773094113280,"available":772310605824,"internally_reserved":0,"allocated":15171584,"data_stored":7652260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":768134704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.668548"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972662+0000","last_change":"2024-09-19T07:55:02.977838+0000","last_active":"2024-09-19T07:55:03.972662+0000","last_peered":"2024-09-19T07:55:03.972662+0000","last_clean":"2024-09-19T07:55:03.972662+0000","last_became_active":"2024-09-19T07:55:02.977562+0000","last_became_peered":"2024-09-19T07:55:02.977562+0000","last_unstale":"2024-09-19T07:55:03.972662+0000","last_undegraded":"2024-09-19T07:55:03.972662+0000","last_fullsized":"2024-09-19T07:55:03.972662+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:18.348990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.994497+0000","last_change":"2024-09-19T07:55:02.988730+0000","last_active":"2024-09-19T07:55:02.994497+0000","last_peered":"2024-09-19T07:55:02.994497+0000","last_clean":"2024-09-19T07:55:02.994497+0000","last_became_active":"2024-09-19T07:55:02.988454+0000","last_became_peered":"2024-09-19T07:55:02.988454+0000","last_unstale":"2024-09-19T07:55:02.994497+0000","last_undegraded":"2024-09-19T07:55:02.994497+0000","last_fullsized":"2024-09-19T07:55:02.994497+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:55.150368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998989+0000","last_change":"2024-09-19T07:55:02.995513+0000","last_active":"2024-09-19T07:55:02.998989+0000","last_peered":"2024-09-19T07:55:02.998989+0000","last_clean":"2024-09-19T07:55:02.998989+0000","last_became_active":"2024-09-19T07:55:02.995162+0000","last_became_peered":"2024-09-19T07:55:02.995162+0000","last_unstale":"2024-09-19T07:55:02.998989+0000","last_undegraded":"2024-09-19T07:55:02.998989+0000","last_fullsized":"2024-09-19T07:55:02.998989+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:08.826237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972077+0000","last_change":"2024-09-19T07:55:02.991747+0000","last_active":"2024-09-19T07:55:03.972077+0000","last_peered":"2024-09-19T07:55:03.972077+0000","last_clean":"2024-09-19T07:55:03.972077+0000","last_became_active":"2024-09-19T07:55:02.991487+0000","last_became_peered":"2024-09-19T07:55:02.991487+0000","last_unstale":"2024-09-19T07:55:03.972077+0000","last_undegraded":"2024-09-19T07:55:03.972077+0000","last_fullsized":"2024-09-19T07:55:03.972077+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:00.867259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.978550+0000","last_change":"2024-09-19T07:55:02.974381+0000","last_active":"2024-09-19T07:55:02.978550+0000","last_peered":"2024-09-19T07:55:02.978550+0000","last_clean":"2024-09-19T07:55:02.978550+0000","last_became_active":"2024-09-19T07:55:02.974105+0000","last_became_peered":"2024-09-19T07:55:02.974105+0000","last_unstale":"2024-09-19T07:55:02.978550+0000","last_undegraded":"2024-09-19T07:55:02.978550+0000","last_fullsized":"2024-09-19T07:55:02.978550+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:08.519276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.980270+0000","last_change":"2024-09-19T07:55:02.975703+0000","last_active":"2024-09-19T07:55:02.980270+0000","last_peered":"2024-09-19T07:55:02.980270+0000","last_clean":"2024-09-19T07:55:02.980270+0000","last_became_active":"2024-09-19T07:55:02.975379+0000","last_became_peered":"2024-09-19T07:55:02.975379+0000","last_unstale":"2024-09-19T07:55:02.980270+0000","last_undegraded":"2024-09-19T07:55:02.980270+0000","last_fullsized":"2024-09-19T07:55:02.980270+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:11.846213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972184+0000","last_change":"2024-09-19T07:55:02.985662+0000","last_active":"2024-09-19T07:55:03.972184+0000","last_peered":"2024-09-19T07:55:03.972184+0000","last_clean":"2024-09-19T07:55:03.972184+0000","last_became_active":"2024-09-19T07:55:02.985378+0000","last_became_peered":"2024-09-19T07:55:02.985378+0000","last_unstale":"2024-09-19T07:55:03.972184+0000","last_undegraded":"2024-09-19T07:55:03.972184+0000","last_fullsized":"2024-09-19T07:55:03.972184+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:33.679405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.992880+0000","last_change":"2024-09-19T07:55:02.991159+0000","last_active":"2024-09-19T07:55:02.992880+0000","last_peered":"2024-09-19T07:55:02.992880+0000","last_clean":"2024-09-19T07:55:02.992880+0000","last_became_active":"2024-09-19T07:55:02.990631+0000","last_became_peered":"2024-09-19T07:55:02.990631+0000","last_unstale":"2024-09-19T07:55:02.992880+0000","last_undegraded":"2024-09-19T07:55:02.992880+0000","last_fullsized":"2024-09-19T07:55:02.992880+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:16.038904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998379+0000","last_change":"2024-09-19T07:55:02.994532+0000","last_active":"2024-09-19T07:55:02.998379+0000","last_peered":"2024-09-19T07:55:02.998379+0000","last_clean":"2024-09-19T07:55:02.998379+0000","last_became_active":"2024-09-19T07:55:02.994201+0000","last_became_peered":"2024-09-19T07:55:02.994201+0000","last_unstale":"2024-09-19T07:55:02.998379+0000","last_undegraded":"2024-09-19T07:55:02.998379+0000","last_fullsized":"2024-09-19T07:55:02.998379+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:12.950630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.984004+0000","last_change":"2024-09-19T07:55:02.981864+0000","last_active":"2024-09-19T07:55:02.984004+0000","last_peered":"2024-09-19T07:55:02.984004+0000","last_clean":"2024-09-19T07:55:02.984004+0000","last_became_active":"2024-09-19T07:55:02.981541+0000","last_became_peered":"2024-09-19T07:55:02.981541+0000","last_unstale":"2024-09-19T07:55:02.984004+0000","last_undegraded":"2024-09-19T07:55:02.984004+0000","last_fullsized":"2024-09-19T07:55:02.984004+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:22.255417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845763+0000","last_change":"2024-09-19T07:55:02.991467+0000","last_active":"2024-09-19T07:55:04.845763+0000","last_peered":"2024-09-19T07:55:04.845763+0000","last_clean":"2024-09-19T07:55:04.845763+0000","last_became_active":"2024-09-19T07:55:02.991185+0000","last_became_peered":"2024-09-19T07:55:02.991185+0000","last_unstale":"2024-09-19T07:55:04.845763+0000","last_undegraded":"2024-09-19T07:55:04.845763+0000","last_fullsized":"2024-09-19T07:55:04.845763+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:25:03.509906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972187+0000","last_change":"2024-09-19T07:55:02.989911+0000","last_active":"2024-09-19T07:55:03.972187+0000","last_peered":"2024-09-19T07:55:03.972187+0000","last_clean":"2024-09-19T07:55:03.972187+0000","last_became_active":"2024-09-19T07:55:02.989574+0000","last_became_peered":"2024-09-19T07:55:02.989574+0000","last_unstale":"2024-09-19T07:55:03.972187+0000","last_undegraded":"2024-09-19T07:55:03.972187+0000","last_fullsized":"2024-09-19T07:55:03.972187+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:20.107114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.985584+0000","last_change":"2024-09-19T07:55:02.983842+0000","last_active":"2024-09-19T07:55:02.985584+0000","last_peered":"2024-09-19T07:55:02.985584+0000","last_clean":"2024-09-19T07:55:02.985584+0000","last_became_active":"2024-09-19T07:55:02.983583+0000","last_became_peered":"2024-09-19T07:55:02.983583+0000","last_unstale":"2024-09-19T07:55:02.985584+0000","last_undegraded":"2024-09-19T07:55:02.985584+0000","last_fullsized":"2024-09-19T07:55:02.985584+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:58.534832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.971850+0000","last_change":"2024-09-19T07:55:02.991151+0000","last_active":"2024-09-19T07:55:03.971850+0000","last_peered":"2024-09-19T07:55:03.971850+0000","last_clean":"2024-09-19T07:55:03.971850+0000","last_became_active":"2024-09-19T07:55:02.990791+0000","last_became_peered":"2024-09-19T07:55:02.990791+0000","last_unstale":"2024-09-19T07:55:03.971850+0000","last_undegraded":"2024-09-19T07:55:03.971850+0000","last_fullsized":"2024-09-19T07:55:03.971850+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:16.038904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.997603+0000","last_change":"2024-09-19T07:55:02.993711+0000","last_active":"2024-09-19T07:55:02.997603+0000","last_peered":"2024-09-19T07:55:02.997603+0000","last_clean":"2024-09-19T07:55:02.997603+0000","last_became_active":"2024-09-19T07:55:02.993391+0000","last_became_peered":"2024-09-19T07:55:02.993391+0000","last_unstale":"2024-09-19T07:55:02.997603+0000","last_undegraded":"2024-09-19T07:55:02.997603+0000","last_fullsized":"2024-09-19T07:55:02.997603+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:53:11.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.981640+0000","last_change":"2024-09-19T07:55:02.977783+0000","last_active":"2024-09-19T07:55:02.981640+0000","last_peered":"2024-09-19T07:55:02.981640+0000","last_clean":"2024-09-19T07:55:02.981640+0000","last_became_active":"2024-09-19T07:55:02.977494+0000","last_became_peered":"2024-09-19T07:55:02.977494+0000","last_unstale":"2024-09-19T07:55:02.981640+0000","last_undegraded":"2024-09-19T07:55:02.981640+0000","last_fullsized":"2024-09-19T07:55:02.981640+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:42.284168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972758+0000","last_change":"2024-09-19T07:55:02.989847+0000","last_active":"2024-09-19T07:55:03.972758+0000","last_peered":"2024-09-19T07:55:03.972758+0000","last_clean":"2024-09-19T07:55:03.972758+0000","last_became_active":"2024-09-19T07:55:02.989566+0000","last_became_peered":"2024-09-19T07:55:02.989566+0000","last_unstale":"2024-09-19T07:55:03.972758+0000","last_undegraded":"2024-09-19T07:55:03.972758+0000","last_fullsized":"2024-09-19T07:55:03.972758+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:57:07.989977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845741+0000","last_change":"2024-09-19T07:55:02.988518+0000","last_active":"2024-09-19T07:55:04.845741+0000","last_peered":"2024-09-19T07:55:04.845741+0000","last_clean":"2024-09-19T07:55:04.845741+0000","last_became_active":"2024-09-19T07:55:02.988215+0000","last_became_peered":"2024-09-19T07:55:02.988215+0000","last_unstale":"2024-09-19T07:55:04.845741+0000","last_undegraded":"2024-09-19T07:55:04.845741+0000","last_fullsized":"2024-09-19T07:55:04.845741+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:33.944874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983370+0000","last_change":"2024-09-19T07:55:02.988233+0000","last_active":"2024-09-19T07:55:03.983370+0000","last_peered":"2024-09-19T07:55:03.983370+0000","last_clean":"2024-09-19T07:55:03.983370+0000","last_became_active":"2024-09-19T07:55:02.987586+0000","last_became_peered":"2024-09-19T07:55:02.987586+0000","last_unstale":"2024-09-19T07:55:03.983370+0000","last_undegraded":"2024-09-19T07:55:03.983370+0000","last_fullsized":"2024-09-19T07:55:03.983370+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:37.091603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.989123+0000","last_change":"2024-09-19T07:55:02.984872+0000","last_active":"2024-09-19T07:55:02.989123+0000","last_peered":"2024-09-19T07:55:02.989123+0000","last_clean":"2024-09-19T07:55:02.989123+0000","last_became_active":"2024-09-19T07:55:02.984588+0000","last_became_peered":"2024-09-19T07:55:02.984588+0000","last_unstale":"2024-09-19T07:55:02.989123+0000","last_undegraded":"2024-09-19T07:55:02.989123+0000","last_fullsized":"2024-09-19T07:55:02.989123+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:37.091603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.986844+0000","last_change":"2024-09-19T07:55:02.985444+0000","last_active":"2024-09-19T07:55:02.986844+0000","last_peered":"2024-09-19T07:55:02.986844+0000","last_clean":"2024-09-19T07:55:02.986844+0000","last_became_active":"2024-09-19T07:55:02.985161+0000","last_became_peered":"2024-09-19T07:55:02.985161+0000","last_unstale":"2024-09-19T07:55:02.986844+0000","last_undegraded":"2024-09-19T07:55:02.986844+0000","last_fullsized":"2024-09-19T07:55:02.986844+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:19.760108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972782+0000","last_change":"2024-09-19T07:55:02.980246+0000","last_active":"2024-09-19T07:55:03.972782+0000","last_peered":"2024-09-19T07:55:03.972782+0000","last_clean":"2024-09-19T07:55:03.972782+0000","last_became_active":"2024-09-19T07:55:02.979970+0000","last_became_peered":"2024-09-19T07:55:02.979970+0000","last_unstale":"2024-09-19T07:55:03.972782+0000","last_undegraded":"2024-09-19T07:55:03.972782+0000","last_fullsized":"2024-09-19T07:55:03.972782+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:19.717109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.986210+0000","last_change":"2024-09-19T07:55:02.979612+0000","last_active":"2024-09-19T07:55:02.986210+0000","last_peered":"2024-09-19T07:55:02.986210+0000","last_clean":"2024-09-19T07:55:02.986210+0000","last_became_active":"2024-09-19T07:55:02.978902+0000","last_became_peered":"2024-09-19T07:55:02.978902+0000","last_unstale":"2024-09-19T07:55:02.986210+0000","last_undegraded":"2024-09-19T07:55:02.986210+0000","last_fullsized":"2024-09-19T07:55:02.986210+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:57.305718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845596+0000","last_change":"2024-09-19T07:55:02.987092+0000","last_active":"2024-09-19T07:55:04.845596+0000","last_peered":"2024-09-19T07:55:04.845596+0000","last_clean":"2024-09-19T07:55:04.845596+0000","last_became_active":"2024-09-19T07:55:02.986604+0000","last_became_peered":"2024-09-19T07:55:02.986604+0000","last_unstale":"2024-09-19T07:55:04.845596+0000","last_undegraded":"2024-09-19T07:55:04.845596+0000","last_fullsized":"2024-09-19T07:55:04.845596+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:30:29.389217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972885+0000","last_change":"2024-09-19T07:55:02.981645+0000","last_active":"2024-09-19T07:55:03.972885+0000","last_peered":"2024-09-19T07:55:03.972885+0000","last_clean":"2024-09-19T07:55:03.972885+0000","last_became_active":"2024-09-19T07:55:02.981377+0000","last_became_peered":"2024-09-19T07:55:02.981377+0000","last_unstale":"2024-09-19T07:55:03.972885+0000","last_undegraded":"2024-09-19T07:55:03.972885+0000","last_fullsized":"2024-09-19T07:55:03.972885+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:28.867264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972976+0000","last_change":"2024-09-19T07:55:02.991055+0000","last_active":"2024-09-19T07:55:03.972976+0000","last_peered":"2024-09-19T07:55:03.972976+0000","last_clean":"2024-09-19T07:55:03.972976+0000","last_became_active":"2024-09-19T07:55:02.990517+0000","last_became_peered":"2024-09-19T07:55:02.990517+0000","last_unstale":"2024-09-19T07:55:03.972976+0000","last_undegraded":"2024-09-19T07:55:03.972976+0000","last_fullsized":"2024-09-19T07:55:03.972976+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:52.586504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.999462+0000","last_change":"2024-09-19T07:55:02.998488+0000","last_active":"2024-09-19T07:55:02.999462+0000","last_peered":"2024-09-19T07:55:02.999462+0000","last_clean":"2024-09-19T07:55:02.999462+0000","last_became_active":"2024-09-19T07:55:02.998192+0000","last_became_peered":"2024-09-19T07:55:02.998192+0000","last_unstale":"2024-09-19T07:55:02.999462+0000","last_undegraded":"2024-09-19T07:55:02.999462+0000","last_fullsized":"2024-09-19T07:55:02.999462+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:20.318482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845428+0000","last_change":"2024-09-19T07:55:02.988808+0000","last_active":"2024-09-19T07:55:04.845428+0000","last_peered":"2024-09-19T07:55:04.845428+0000","last_clean":"2024-09-19T07:55:04.845428+0000","last_became_active":"2024-09-19T07:55:02.988524+0000","last_became_peered":"2024-09-19T07:55:02.988524+0000","last_unstale":"2024-09-19T07:55:04.845428+0000","last_undegraded":"2024-09-19T07:55:04.845428+0000","last_fullsized":"2024-09-19T07:55:04.845428+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:14.948481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.987062+0000","last_change":"2024-09-19T07:55:02.985372+0000","last_active":"2024-09-19T07:55:02.987062+0000","last_peered":"2024-09-19T07:55:02.987062+0000","last_clean":"2024-09-19T07:55:02.987062+0000","last_became_active":"2024-09-19T07:55:02.985109+0000","last_became_peered":"2024-09-19T07:55:02.985109+0000","last_unstale":"2024-09-19T07:55:02.987062+0000","last_undegraded":"2024-09-19T07:55:02.987062+0000","last_fullsized":"2024-09-19T07:55:02.987062+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:59.971722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845582+0000","last_change":"2024-09-19T07:55:02.993268+0000","last_active":"2024-09-19T07:55:04.845582+0000","last_peered":"2024-09-19T07:55:04.845582+0000","last_clean":"2024-09-19T07:55:04.845582+0000","last_became_active":"2024-09-19T07:55:02.992933+0000","last_became_peered":"2024-09-19T07:55:02.992933+0000","last_unstale":"2024-09-19T07:55:04.845582+0000","last_undegraded":"2024-09-19T07:55:04.845582+0000","last_fullsized":"2024-09-19T07:55:04.845582+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:34:34.252831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.981739+0000","last_change":"2024-09-19T07:55:02.978386+0000","last_active":"2024-09-19T07:55:02.981739+0000","last_peered":"2024-09-19T07:55:02.981739+0000","last_clean":"2024-09-19T07:55:02.981739+0000","last_became_active":"2024-09-19T07:55:02.978134+0000","last_became_peered":"2024-09-19T07:55:02.978134+0000","last_unstale":"2024-09-19T07:55:02.981739+0000","last_undegraded":"2024-09-19T07:55:02.981739+0000","last_fullsized":"2024-09-19T07:55:02.981739+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:06.762003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.992859+0000","last_change":"2024-09-19T07:55:02.990915+0000","last_active":"2024-09-19T07:55:02.992859+0000","last_peered":"2024-09-19T07:55:02.992859+0000","last_clean":"2024-09-19T07:55:02.992859+0000","last_became_active":"2024-09-19T07:55:02.990273+0000","last_became_peered":"2024-09-19T07:55:02.990273+0000","last_unstale":"2024-09-19T07:55:02.992859+0000","last_undegraded":"2024-09-19T07:55:02.992859+0000","last_fullsized":"2024-09-19T07:55:02.992859+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:25.070703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998804+0000","last_change":"2024-09-19T07:55:02.994240+0000","last_active":"2024-09-19T07:55:02.998804+0000","last_peered":"2024-09-19T07:55:02.998804+0000","last_clean":"2024-09-19T07:55:02.998804+0000","last_became_active":"2024-09-19T07:55:02.993966+0000","last_became_peered":"2024-09-19T07:55:02.993966+0000","last_unstale":"2024-09-19T07:55:02.998804+0000","last_undegraded":"2024-09-19T07:55:02.998804+0000","last_fullsized":"2024-09-19T07:55:02.998804+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:18.497803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"139'4","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000708+0000","last_change":"2024-09-19T07:55:01.982819+0000","last_active":"2024-09-19T07:55:03.000708+0000","last_peered":"2024-09-19T07:55:03.000708+0000","last_clean":"2024-09-19T07:55:03.000708+0000","last_became_active":"2024-09-19T07:55:01.982455+0000","last_became_peered":"2024-09-19T07:55:01.982455+0000","last_unstale":"2024-09-19T07:55:03.000708+0000","last_undegraded":"2024-09-19T07:55:03.000708+0000","last_fullsized":"2024-09-19T07:55:03.000708+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:01.483855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.995361+0000","last_change":"2024-09-19T07:55:02.988748+0000","last_active":"2024-09-19T07:55:02.995361+0000","last_peered":"2024-09-19T07:55:02.995361+0000","last_clean":"2024-09-19T07:55:02.995361+0000","last_became_active":"2024-09-19T07:55:02.988417+0000","last_became_peered":"2024-09-19T07:55:02.988417+0000","last_unstale":"2024-09-19T07:55:02.995361+0000","last_undegraded":"2024-09-19T07:55:02.995361+0000","last_fullsized":"2024-09-19T07:55:02.995361+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:11.124464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983330+0000","last_change":"2024-09-19T07:55:01.986317+0000","last_active":"2024-09-19T07:55:03.983330+0000","last_peered":"2024-09-19T07:55:03.983330+0000","last_clean":"2024-09-19T07:55:03.983330+0000","last_became_active":"2024-09-19T07:55:01.986050+0000","last_became_peered":"2024-09-19T07:55:01.986050+0000","last_unstale":"2024-09-19T07:55:03.983330+0000","last_undegraded":"2024-09-19T07:55:03.983330+0000","last_fullsized":"2024-09-19T07:55:03.983330+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:20.699942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.988705+0000","last_change":"2024-09-19T07:55:02.988084+0000","last_active":"2024-09-19T07:55:02.988705+0000","last_peered":"2024-09-19T07:55:02.988705+0000","last_clean":"2024-09-19T07:55:02.988705+0000","last_became_active":"2024-09-19T07:55:02.987819+0000","last_became_peered":"2024-09-19T07:55:02.987819+0000","last_unstale":"2024-09-19T07:55:02.988705+0000","last_undegraded":"2024-09-19T07:55:02.988705+0000","last_fullsized":"2024-09-19T07:55:02.988705+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:34.829241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983598+0000","last_change":"2024-09-19T07:55:01.983959+0000","last_active":"2024-09-19T07:55:03.983598+0000","last_peered":"2024-09-19T07:55:03.983598+0000","last_clean":"2024-09-19T07:55:03.983598+0000","last_became_active":"2024-09-19T07:55:01.983682+0000","last_became_peered":"2024-09-19T07:55:01.983682+0000","last_unstale":"2024-09-19T07:55:03.983598+0000","last_undegraded":"2024-09-19T07:55:03.983598+0000","last_fullsized":"2024-09-19T07:55:03.983598+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:17.343679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983514+0000","last_change":"2024-09-19T07:55:02.988272+0000","last_active":"2024-09-19T07:55:03.983514+0000","last_peered":"2024-09-19T07:55:03.983514+0000","last_clean":"2024-09-19T07:55:03.983514+0000","last_became_active":"2024-09-19T07:55:02.987972+0000","last_became_peered":"2024-09-19T07:55:02.987972+0000","last_unstale":"2024-09-19T07:55:03.983514+0000","last_undegraded":"2024-09-19T07:55:03.983514+0000","last_fullsized":"2024-09-19T07:55:03.983514+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:55.597883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"139'2","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000148+0000","last_change":"2024-09-19T07:55:01.975504+0000","last_active":"2024-09-19T07:55:03.000148+0000","last_peered":"2024-09-19T07:55:03.000148+0000","last_clean":"2024-09-19T07:55:03.000148+0000","last_became_active":"2024-09-19T07:55:01.975216+0000","last_became_peered":"2024-09-19T07:55:01.975216+0000","last_unstale":"2024-09-19T07:55:03.000148+0000","last_undegraded":"2024-09-19T07:55:03.000148+0000","last_fullsized":"2024-09-19T07:55:03.000148+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:47.640841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.990742+0000","last_change":"2024-09-19T07:55:02.990235+0000","last_active":"2024-09-19T07:55:02.990742+0000","last_peered":"2024-09-19T07:55:02.990742+0000","last_clean":"2024-09-19T07:55:02.990742+0000","last_became_active":"2024-09-19T07:55:02.989920+0000","last_became_peered":"2024-09-19T07:55:02.989920+0000","last_unstale":"2024-09-19T07:55:02.990742+0000","last_undegraded":"2024-09-19T07:55:02.990742+0000","last_fullsized":"2024-09-19T07:55:02.990742+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:55.597883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"139'1","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.997333+0000","last_change":"2024-09-19T07:55:01.984133+0000","last_active":"2024-09-19T07:55:02.997333+0000","last_peered":"2024-09-19T07:55:02.997333+0000","last_clean":"2024-09-19T07:55:02.997333+0000","last_became_active":"2024-09-19T07:55:01.983829+0000","last_became_peered":"2024-09-19T07:55:01.983829+0000","last_unstale":"2024-09-19T07:55:02.997333+0000","last_undegraded":"2024-09-19T07:55:02.997333+0000","last_fullsized":"2024-09-19T07:55:02.997333+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:08.327895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.971878+0000","last_change":"2024-09-19T07:55:02.990053+0000","last_active":"2024-09-19T07:55:03.971878+0000","last_peered":"2024-09-19T07:55:03.971878+0000","last_clean":"2024-09-19T07:55:03.971878+0000","last_became_active":"2024-09-19T07:55:02.987609+0000","last_became_peered":"2024-09-19T07:55:02.987609+0000","last_unstale":"2024-09-19T07:55:03.971878+0000","last_undegraded":"2024-09-19T07:55:03.971878+0000","last_fullsized":"2024-09-19T07:55:03.971878+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:02:28.317679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"139'1","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998584+0000","last_change":"2024-09-19T07:55:01.984540+0000","last_active":"2024-09-19T07:55:02.998584+0000","last_peered":"2024-09-19T07:55:02.998584+0000","last_clean":"2024-09-19T07:55:02.998584+0000","last_became_active":"2024-09-19T07:55:01.984218+0000","last_became_peered":"2024-09-19T07:55:01.984218+0000","last_unstale":"2024-09-19T07:55:02.998584+0000","last_undegraded":"2024-09-19T07:55:02.998584+0000","last_fullsized":"2024-09-19T07:55:02.998584+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:59:17.842847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.987109+0000","last_change":"2024-09-19T07:55:02.983739+0000","last_active":"2024-09-19T07:55:02.987109+0000","last_peered":"2024-09-19T07:55:02.987109+0000","last_clean":"2024-09-19T07:55:02.987109+0000","last_became_active":"2024-09-19T07:55:02.983448+0000","last_became_peered":"2024-09-19T07:55:02.983448+0000","last_unstale":"2024-09-19T07:55:02.987109+0000","last_undegraded":"2024-09-19T07:55:02.987109+0000","last_fullsized":"2024-09-19T07:55:02.987109+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:50.858294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972280+0000","last_change":"2024-09-19T07:55:01.978901+0000","last_active":"2024-09-19T07:55:03.972280+0000","last_peered":"2024-09-19T07:55:03.972280+0000","last_clean":"2024-09-19T07:55:03.972280+0000","last_became_active":"2024-09-19T07:55:01.978399+0000","last_became_peered":"2024-09-19T07:55:01.978399+0000","last_unstale":"2024-09-19T07:55:03.972280+0000","last_undegraded":"2024-09-19T07:55:03.972280+0000","last_fullsized":"2024-09-19T07:55:03.972280+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:11.363759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972118+0000","last_change":"2024-09-19T07:55:02.978549+0000","last_active":"2024-09-19T07:55:03.972118+0000","last_peered":"2024-09-19T07:55:03.972118+0000","last_clean":"2024-09-19T07:55:03.972118+0000","last_became_active":"2024-09-19T07:55:02.978270+0000","last_became_peered":"2024-09-19T07:55:02.978270+0000","last_unstale":"2024-09-19T07:55:03.972118+0000","last_undegraded":"2024-09-19T07:55:03.972118+0000","last_fullsized":"2024-09-19T07:55:03.972118+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:21.705254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.969267+0000","last_change":"2024-09-19T07:55:01.983833+0000","last_active":"2024-09-19T07:55:02.969267+0000","last_peered":"2024-09-19T07:55:02.969267+0000","last_clean":"2024-09-19T07:55:02.969267+0000","last_became_active":"2024-09-19T07:55:01.983553+0000","last_became_peered":"2024-09-19T07:55:01.983553+0000","last_unstale":"2024-09-19T07:55:02.969267+0000","last_undegraded":"2024-09-19T07:55:02.969267+0000","last_fullsized":"2024-09-19T07:55:02.969267+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:02.576442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.986319+0000","last_change":"2024-09-19T07:55:02.983084+0000","last_active":"2024-09-19T07:55:02.986319+0000","last_peered":"2024-09-19T07:55:02.986319+0000","last_clean":"2024-09-19T07:55:02.986319+0000","last_became_active":"2024-09-19T07:55:02.982793+0000","last_became_peered":"2024-09-19T07:55:02.982793+0000","last_unstale":"2024-09-19T07:55:02.986319+0000","last_undegraded":"2024-09-19T07:55:02.986319+0000","last_fullsized":"2024-09-19T07:55:02.986319+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:39:22.911029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983489+0000","last_change":"2024-09-19T07:55:01.980547+0000","last_active":"2024-09-19T07:55:03.983489+0000","last_peered":"2024-09-19T07:55:03.983489+0000","last_clean":"2024-09-19T07:55:03.983489+0000","last_became_active":"2024-09-19T07:55:01.980253+0000","last_became_peered":"2024-09-19T07:55:01.980253+0000","last_unstale":"2024-09-19T07:55:03.983489+0000","last_undegraded":"2024-09-19T07:55:03.983489+0000","last_fullsized":"2024-09-19T07:55:03.983489+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:32.674094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845937+0000","last_change":"2024-09-19T07:55:02.990524+0000","last_active":"2024-09-19T07:55:04.845937+0000","last_peered":"2024-09-19T07:55:04.845937+0000","last_clean":"2024-09-19T07:55:04.845937+0000","last_became_active":"2024-09-19T07:55:02.990227+0000","last_became_peered":"2024-09-19T07:55:02.990227+0000","last_unstale":"2024-09-19T07:55:04.845937+0000","last_undegraded":"2024-09-19T07:55:04.845937+0000","last_fullsized":"2024-09-19T07:55:04.845937+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:14:38.518293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.973223+0000","last_change":"2024-09-19T07:55:01.980870+0000","last_active":"2024-09-19T07:55:03.973223+0000","last_peered":"2024-09-19T07:55:03.973223+0000","last_clean":"2024-09-19T07:55:03.973223+0000","last_became_active":"2024-09-19T07:55:01.980569+0000","last_became_peered":"2024-09-19T07:55:01.980569+0000","last_unstale":"2024-09-19T07:55:03.973223+0000","last_undegraded":"2024-09-19T07:55:03.973223+0000","last_fullsized":"2024-09-19T07:55:03.973223+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:56:13.952641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845892+0000","last_change":"2024-09-19T07:55:02.987773+0000","last_active":"2024-09-19T07:55:04.845892+0000","last_peered":"2024-09-19T07:55:04.845892+0000","last_clean":"2024-09-19T07:55:04.845892+0000","last_became_active":"2024-09-19T07:55:02.987513+0000","last_became_peered":"2024-09-19T07:55:02.987513+0000","last_unstale":"2024-09-19T07:55:04.845892+0000","last_undegraded":"2024-09-19T07:55:04.845892+0000","last_fullsized":"2024-09-19T07:55:04.845892+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:25.245393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"139'3","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845998+0000","last_change":"2024-09-19T07:55:01.978949+0000","last_active":"2024-09-19T07:55:04.845998+0000","last_peered":"2024-09-19T07:55:04.845998+0000","last_clean":"2024-09-19T07:55:04.845998+0000","last_became_active":"2024-09-19T07:55:01.978508+0000","last_became_peered":"2024-09-19T07:55:01.978508+0000","last_unstale":"2024-09-19T07:55:04.845998+0000","last_undegraded":"2024-09-19T07:55:04.845998+0000","last_fullsized":"2024-09-19T07:55:04.845998+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:33:49.871640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.986247+0000","last_change":"2024-09-19T07:55:02.980794+0000","last_active":"2024-09-19T07:55:02.986247+0000","last_peered":"2024-09-19T07:55:02.986247+0000","last_clean":"2024-09-19T07:55:02.986247+0000","last_became_active":"2024-09-19T07:55:02.980530+0000","last_became_peered":"2024-09-19T07:55:02.980530+0000","last_unstale":"2024-09-19T07:55:02.986247+0000","last_undegraded":"2024-09-19T07:55:02.986247+0000","last_fullsized":"2024-09-19T07:55:02.986247+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:14.838884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"139'2","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000534+0000","last_change":"2024-09-19T07:55:01.984318+0000","last_active":"2024-09-19T07:55:03.000534+0000","last_peered":"2024-09-19T07:55:03.000534+0000","last_clean":"2024-09-19T07:55:03.000534+0000","last_became_active":"2024-09-19T07:55:01.983325+0000","last_became_peered":"2024-09-19T07:55:01.983325+0000","last_unstale":"2024-09-19T07:55:03.000534+0000","last_undegraded":"2024-09-19T07:55:03.000534+0000","last_fullsized":"2024-09-19T07:55:03.000534+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:36.086291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.999112+0000","last_change":"2024-09-19T07:55:02.996430+0000","last_active":"2024-09-19T07:55:02.999112+0000","last_peered":"2024-09-19T07:55:02.999112+0000","last_clean":"2024-09-19T07:55:02.999112+0000","last_became_active":"2024-09-19T07:55:02.996148+0000","last_became_peered":"2024-09-19T07:55:02.996148+0000","last_unstale":"2024-09-19T07:55:02.999112+0000","last_undegraded":"2024-09-19T07:55:02.999112+0000","last_fullsized":"2024-09-19T07:55:02.999112+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:12.728223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972021+0000","last_change":"2024-09-19T07:55:01.988068+0000","last_active":"2024-09-19T07:55:03.972021+0000","last_peered":"2024-09-19T07:55:03.972021+0000","last_clean":"2024-09-19T07:55:03.972021+0000","last_became_active":"2024-09-19T07:55:01.987761+0000","last_became_peered":"2024-09-19T07:55:01.987761+0000","last_unstale":"2024-09-19T07:55:03.972021+0000","last_undegraded":"2024-09-19T07:55:03.972021+0000","last_fullsized":"2024-09-19T07:55:03.972021+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:56.300407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.994831+0000","last_change":"2024-09-19T07:55:02.989642+0000","last_active":"2024-09-19T07:55:02.994831+0000","last_peered":"2024-09-19T07:55:02.994831+0000","last_clean":"2024-09-19T07:55:02.994831+0000","last_became_active":"2024-09-19T07:55:02.989315+0000","last_became_peered":"2024-09-19T07:55:02.989315+0000","last_unstale":"2024-09-19T07:55:02.994831+0000","last_undegraded":"2024-09-19T07:55:02.994831+0000","last_fullsized":"2024-09-19T07:55:02.994831+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:29.434870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972162+0000","last_change":"2024-09-19T07:55:01.983411+0000","last_active":"2024-09-19T07:55:03.972162+0000","last_peered":"2024-09-19T07:55:03.972162+0000","last_clean":"2024-09-19T07:55:03.972162+0000","last_became_active":"2024-09-19T07:55:01.982413+0000","last_became_peered":"2024-09-19T07:55:01.982413+0000","last_unstale":"2024-09-19T07:55:03.972162+0000","last_undegraded":"2024-09-19T07:55:03.972162+0000","last_fullsized":"2024-09-19T07:55:03.972162+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:58.316347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.985547+0000","last_change":"2024-09-19T07:55:02.983361+0000","last_active":"2024-09-19T07:55:02.985547+0000","last_peered":"2024-09-19T07:55:02.985547+0000","last_clean":"2024-09-19T07:55:02.985547+0000","last_became_active":"2024-09-19T07:55:02.983032+0000","last_became_peered":"2024-09-19T07:55:02.983032+0000","last_unstale":"2024-09-19T07:55:02.985547+0000","last_undegraded":"2024-09-19T07:55:02.985547+0000","last_fullsized":"2024-09-19T07:55:02.985547+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:56:14.957952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"139'8","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.971849+0000","last_change":"2024-09-19T07:55:01.978831+0000","last_active":"2024-09-19T07:55:03.971849+0000","last_peered":"2024-09-19T07:55:03.971849+0000","last_clean":"2024-09-19T07:55:03.971849+0000","last_became_active":"2024-09-19T07:55:01.978370+0000","last_became_peered":"2024-09-19T07:55:01.978370+0000","last_unstale":"2024-09-19T07:55:03.971849+0000","last_undegraded":"2024-09-19T07:55:03.971849+0000","last_fullsized":"2024-09-19T07:55:03.971849+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:47.640841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983367+0000","last_change":"2024-09-19T07:55:02.998722+0000","last_active":"2024-09-19T07:55:03.983367+0000","last_peered":"2024-09-19T07:55:03.983367+0000","last_clean":"2024-09-19T07:55:03.983367+0000","last_became_active":"2024-09-19T07:55:02.998420+0000","last_became_peered":"2024-09-19T07:55:02.998420+0000","last_unstale":"2024-09-19T07:55:03.983367+0000","last_undegraded":"2024-09-19T07:55:03.983367+0000","last_fullsized":"2024-09-19T07:55:03.983367+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:02.489166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845663+0000","last_change":"2024-09-19T07:55:01.983892+0000","last_active":"2024-09-19T07:55:04.845663+0000","last_peered":"2024-09-19T07:55:04.845663+0000","last_clean":"2024-09-19T07:55:04.845663+0000","last_became_active":"2024-09-19T07:55:01.983621+0000","last_became_peered":"2024-09-19T07:55:01.983621+0000","last_unstale":"2024-09-19T07:55:04.845663+0000","last_undegraded":"2024-09-19T07:55:04.845663+0000","last_fullsized":"2024-09-19T07:55:04.845663+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:35:59.714032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983513+0000","last_change":"2024-09-19T07:55:02.988579+0000","last_active":"2024-09-19T07:55:03.983513+0000","last_peered":"2024-09-19T07:55:03.983513+0000","last_clean":"2024-09-19T07:55:03.983513+0000","last_became_active":"2024-09-19T07:55:02.987958+0000","last_became_peered":"2024-09-19T07:55:02.987958+0000","last_unstale":"2024-09-19T07:55:03.983513+0000","last_undegraded":"2024-09-19T07:55:03.983513+0000","last_fullsized":"2024-09-19T07:55:03.983513+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:08.519276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983438+0000","last_change":"2024-09-19T07:55:01.981723+0000","last_active":"2024-09-19T07:55:03.983438+0000","last_peered":"2024-09-19T07:55:03.983438+0000","last_clean":"2024-09-19T07:55:03.983438+0000","last_became_active":"2024-09-19T07:55:01.981465+0000","last_became_peered":"2024-09-19T07:55:01.981465+0000","last_unstale":"2024-09-19T07:55:03.983438+0000","last_undegraded":"2024-09-19T07:55:03.983438+0000","last_fullsized":"2024-09-19T07:55:03.983438+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:12.838363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.973123+0000","last_change":"2024-09-19T07:55:02.979256+0000","last_active":"2024-09-19T07:55:03.973123+0000","last_peered":"2024-09-19T07:55:03.973123+0000","last_clean":"2024-09-19T07:55:03.973123+0000","last_became_active":"2024-09-19T07:55:02.978921+0000","last_became_peered":"2024-09-19T07:55:02.978921+0000","last_unstale":"2024-09-19T07:55:03.973123+0000","last_undegraded":"2024-09-19T07:55:03.973123+0000","last_fullsized":"2024-09-19T07:55:03.973123+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:51.978120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"139'1","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000384+0000","last_change":"2024-09-19T07:55:01.982720+0000","last_active":"2024-09-19T07:55:03.000384+0000","last_peered":"2024-09-19T07:55:03.000384+0000","last_clean":"2024-09-19T07:55:03.000384+0000","last_became_active":"2024-09-19T07:55:01.982277+0000","last_became_peered":"2024-09-19T07:55:01.982277+0000","last_unstale":"2024-09-19T07:55:03.000384+0000","last_undegraded":"2024-09-19T07:55:03.000384+0000","last_fullsized":"2024-09-19T07:55:03.000384+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:59.197262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972931+0000","last_change":"2024-09-19T07:55:02.989059+0000","last_active":"2024-09-19T07:55:03.972931+0000","last_peered":"2024-09-19T07:55:03.972931+0000","last_clean":"2024-09-19T07:55:03.972931+0000","last_became_active":"2024-09-19T07:55:02.988577+0000","last_became_peered":"2024-09-19T07:55:02.988577+0000","last_unstale":"2024-09-19T07:55:03.972931+0000","last_undegraded":"2024-09-19T07:55:03.972931+0000","last_fullsized":"2024-09-19T07:55:03.972931+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:49.502676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972169+0000","last_change":"2024-09-19T07:55:01.988779+0000","last_active":"2024-09-19T07:55:03.972169+0000","last_peered":"2024-09-19T07:55:03.972169+0000","last_clean":"2024-09-19T07:55:03.972169+0000","last_became_active":"2024-09-19T07:55:01.988419+0000","last_became_peered":"2024-09-19T07:55:01.988419+0000","last_unstale":"2024-09-19T07:55:03.972169+0000","last_undegraded":"2024-09-19T07:55:03.972169+0000","last_fullsized":"2024-09-19T07:55:03.972169+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:59.785587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.982978+0000","last_change":"2024-09-19T07:55:02.987148+0000","last_active":"2024-09-19T07:55:03.982978+0000","last_peered":"2024-09-19T07:55:03.982978+0000","last_clean":"2024-09-19T07:55:03.982978+0000","last_became_active":"2024-09-19T07:55:02.986889+0000","last_became_peered":"2024-09-19T07:55:02.986889+0000","last_unstale":"2024-09-19T07:55:03.982978+0000","last_undegraded":"2024-09-19T07:55:03.982978+0000","last_fullsized":"2024-09-19T07:55:03.982978+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:26:18.944316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"139'2","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.997309+0000","last_change":"2024-09-19T07:55:01.986115+0000","last_active":"2024-09-19T07:55:02.997309+0000","last_peered":"2024-09-19T07:55:02.997309+0000","last_clean":"2024-09-19T07:55:02.997309+0000","last_became_active":"2024-09-19T07:55:01.985827+0000","last_became_peered":"2024-09-19T07:55:01.985827+0000","last_unstale":"2024-09-19T07:55:02.997309+0000","last_undegraded":"2024-09-19T07:55:02.997309+0000","last_fullsized":"2024-09-19T07:55:02.997309+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:01.483855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.989398+0000","last_change":"2024-09-19T07:55:02.987365+0000","last_active":"2024-09-19T07:55:02.989398+0000","last_peered":"2024-09-19T07:55:02.989398+0000","last_clean":"2024-09-19T07:55:02.989398+0000","last_became_active":"2024-09-19T07:55:02.986480+0000","last_became_peered":"2024-09-19T07:55:02.986480+0000","last_unstale":"2024-09-19T07:55:02.989398+0000","last_undegraded":"2024-09-19T07:55:02.989398+0000","last_fullsized":"2024-09-19T07:55:02.989398+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:32.859864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.972920+0000","last_change":"2024-09-19T07:55:01.981717+0000","last_active":"2024-09-19T07:55:02.972920+0000","last_peered":"2024-09-19T07:55:02.972920+0000","last_clean":"2024-09-19T07:55:02.972920+0000","last_became_active":"2024-09-19T07:55:01.981004+0000","last_became_peered":"2024-09-19T07:55:01.981004+0000","last_unstale":"2024-09-19T07:55:02.972920+0000","last_undegraded":"2024-09-19T07:55:02.972920+0000","last_fullsized":"2024-09-19T07:55:02.972920+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:58.316347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983112+0000","last_change":"2024-09-19T07:55:02.992093+0000","last_active":"2024-09-19T07:55:03.983112+0000","last_peered":"2024-09-19T07:55:03.983112+0000","last_clean":"2024-09-19T07:55:03.983112+0000","last_became_active":"2024-09-19T07:55:02.991829+0000","last_became_peered":"2024-09-19T07:55:02.991829+0000","last_unstale":"2024-09-19T07:55:03.983112+0000","last_undegraded":"2024-09-19T07:55:03.983112+0000","last_fullsized":"2024-09-19T07:55:03.983112+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:59:18.848159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.974687+0000","last_change":"2024-09-19T07:55:01.983244+0000","last_active":"2024-09-19T07:55:02.974687+0000","last_peered":"2024-09-19T07:55:02.974687+0000","last_clean":"2024-09-19T07:55:02.974687+0000","last_became_active":"2024-09-19T07:55:01.982960+0000","last_became_peered":"2024-09-19T07:55:01.982960+0000","last_unstale":"2024-09-19T07:55:02.974687+0000","last_undegraded":"2024-09-19T07:55:02.974687+0000","last_fullsized":"2024-09-19T07:55:02.974687+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:26:17.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998969+0000","last_change":"2024-09-19T07:55:02.993715+0000","last_active":"2024-09-19T07:55:02.998969+0000","last_peered":"2024-09-19T07:55:02.998969+0000","last_clean":"2024-09-19T07:55:02.998969+0000","last_became_active":"2024-09-19T07:55:02.993409+0000","last_became_peered":"2024-09-19T07:55:02.993409+0000","last_unstale":"2024-09-19T07:55:02.998969+0000","last_undegraded":"2024-09-19T07:55:02.998969+0000","last_fullsized":"2024-09-19T07:55:02.998969+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:13.897172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.974976+0000","last_change":"2024-09-19T07:55:01.982335+0000","last_active":"2024-09-19T07:55:02.974976+0000","last_peered":"2024-09-19T07:55:02.974976+0000","last_clean":"2024-09-19T07:55:02.974976+0000","last_became_active":"2024-09-19T07:55:01.982052+0000","last_became_peered":"2024-09-19T07:55:01.982052+0000","last_unstale":"2024-09-19T07:55:02.974976+0000","last_undegraded":"2024-09-19T07:55:02.974976+0000","last_fullsized":"2024-09-19T07:55:02.974976+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:59.785587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.975028+0000","last_change":"2024-09-19T07:55:02.974014+0000","last_active":"2024-09-19T07:55:02.975028+0000","last_peered":"2024-09-19T07:55:02.975028+0000","last_clean":"2024-09-19T07:55:02.975028+0000","last_became_active":"2024-09-19T07:55:02.973536+0000","last_became_peered":"2024-09-19T07:55:02.973536+0000","last_unstale":"2024-09-19T07:55:02.975028+0000","last_undegraded":"2024-09-19T07:55:02.975028+0000","last_fullsized":"2024-09-19T07:55:02.975028+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:55.597883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972720+0000","last_change":"2024-09-19T07:55:01.988008+0000","last_active":"2024-09-19T07:55:03.972720+0000","last_peered":"2024-09-19T07:55:03.972720+0000","last_clean":"2024-09-19T07:55:03.972720+0000","last_became_active":"2024-09-19T07:55:01.987725+0000","last_became_peered":"2024-09-19T07:55:01.987725+0000","last_unstale":"2024-09-19T07:55:03.972720+0000","last_undegraded":"2024-09-19T07:55:03.972720+0000","last_fullsized":"2024-09-19T07:55:03.972720+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:13.833572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972516+0000","last_change":"2024-09-19T07:55:02.991309+0000","last_active":"2024-09-19T07:55:03.972516+0000","last_peered":"2024-09-19T07:55:03.972516+0000","last_clean":"2024-09-19T07:55:03.972516+0000","last_became_active":"2024-09-19T07:55:02.991045+0000","last_became_peered":"2024-09-19T07:55:02.991045+0000","last_unstale":"2024-09-19T07:55:03.972516+0000","last_undegraded":"2024-09-19T07:55:03.972516+0000","last_fullsized":"2024-09-19T07:55:03.972516+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:38.709571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"140'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.982444+0000","last_change":"2024-09-19T07:55:01.978226+0000","last_active":"2024-09-19T07:55:03.982444+0000","last_peered":"2024-09-19T07:55:03.982444+0000","last_clean":"2024-09-19T07:55:03.982444+0000","last_became_active":"2024-09-19T07:55:01.977803+0000","last_became_peered":"2024-09-19T07:55:01.977803+0000","last_unstale":"2024-09-19T07:55:03.982444+0000","last_undegraded":"2024-09-19T07:55:03.982444+0000","last_fullsized":"2024-09-19T07:55:03.982444+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:49.852983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.973122+0000","last_change":"2024-09-19T07:55:02.977147+0000","last_active":"2024-09-19T07:55:03.973122+0000","last_peered":"2024-09-19T07:55:03.973122+0000","last_clean":"2024-09-19T07:55:03.973122+0000","last_became_active":"2024-09-19T07:55:02.976832+0000","last_became_peered":"2024-09-19T07:55:02.976832+0000","last_unstale":"2024-09-19T07:55:03.973122+0000","last_undegraded":"2024-09-19T07:55:03.973122+0000","last_fullsized":"2024-09-19T07:55:03.973122+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:03.581754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.974160+0000","last_change":"2024-09-19T07:55:01.974880+0000","last_active":"2024-09-19T07:55:02.974160+0000","last_peered":"2024-09-19T07:55:02.974160+0000","last_clean":"2024-09-19T07:55:02.974160+0000","last_became_active":"2024-09-19T07:55:01.974599+0000","last_became_peered":"2024-09-19T07:55:01.974599+0000","last_unstale":"2024-09-19T07:55:02.974160+0000","last_undegraded":"2024-09-19T07:55:02.974160+0000","last_fullsized":"2024-09-19T07:55:02.974160+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:11.363759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.994783+0000","last_change":"2024-09-19T07:55:02.989699+0000","last_active":"2024-09-19T07:55:02.994783+0000","last_peered":"2024-09-19T07:55:02.994783+0000","last_clean":"2024-09-19T07:55:02.994783+0000","last_became_active":"2024-09-19T07:55:02.989398+0000","last_became_peered":"2024-09-19T07:55:02.989398+0000","last_unstale":"2024-09-19T07:55:02.994783+0000","last_undegraded":"2024-09-19T07:55:02.994783+0000","last_fullsized":"2024-09-19T07:55:02.994783+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:11.153885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.970064+0000","last_change":"2024-09-19T07:55:01.981690+0000","last_active":"2024-09-19T07:55:02.970064+0000","last_peered":"2024-09-19T07:55:02.970064+0000","last_clean":"2024-09-19T07:55:02.970064+0000","last_became_active":"2024-09-19T07:55:01.981417+0000","last_became_peered":"2024-09-19T07:55:01.981417+0000","last_unstale":"2024-09-19T07:55:02.970064+0000","last_undegraded":"2024-09-19T07:55:02.970064+0000","last_fullsized":"2024-09-19T07:55:02.970064+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:26:17.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.988647+0000","last_change":"2024-09-19T07:55:02.986076+0000","last_active":"2024-09-19T07:55:02.988647+0000","last_peered":"2024-09-19T07:55:02.988647+0000","last_clean":"2024-09-19T07:55:02.988647+0000","last_became_active":"2024-09-19T07:55:02.984885+0000","last_became_peered":"2024-09-19T07:55:02.984885+0000","last_unstale":"2024-09-19T07:55:02.988647+0000","last_undegraded":"2024-09-19T07:55:02.988647+0000","last_fullsized":"2024-09-19T07:55:02.988647+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:33.113553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972219+0000","last_change":"2024-09-19T07:55:01.986920+0000","last_active":"2024-09-19T07:55:03.972219+0000","last_peered":"2024-09-19T07:55:03.972219+0000","last_clean":"2024-09-19T07:55:03.972219+0000","last_became_active":"2024-09-19T07:55:01.986618+0000","last_became_peered":"2024-09-19T07:55:01.986618+0000","last_unstale":"2024-09-19T07:55:03.972219+0000","last_undegraded":"2024-09-19T07:55:03.972219+0000","last_fullsized":"2024-09-19T07:55:03.972219+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:59.197262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972053+0000","last_change":"2024-09-19T07:55:02.989902+0000","last_active":"2024-09-19T07:55:03.972053+0000","last_peered":"2024-09-19T07:55:03.972053+0000","last_clean":"2024-09-19T07:55:03.972053+0000","last_became_active":"2024-09-19T07:55:02.987293+0000","last_became_peered":"2024-09-19T07:55:02.987293+0000","last_unstale":"2024-09-19T07:55:03.972053+0000","last_undegraded":"2024-09-19T07:55:03.972053+0000","last_fullsized":"2024-09-19T07:55:03.972053+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:54:28.829885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845473+0000","last_change":"2024-09-19T07:55:01.977073+0000","last_active":"2024-09-19T07:55:04.845473+0000","last_peered":"2024-09-19T07:55:04.845473+0000","last_clean":"2024-09-19T07:55:04.845473+0000","last_became_active":"2024-09-19T07:55:01.976782+0000","last_became_peered":"2024-09-19T07:55:01.976782+0000","last_unstale":"2024-09-19T07:55:04.845473+0000","last_undegraded":"2024-09-19T07:55:04.845473+0000","last_fullsized":"2024-09-19T07:55:04.845473+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:38.579539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.987084+0000","last_change":"2024-09-19T07:55:02.983221+0000","last_active":"2024-09-19T07:55:02.987084+0000","last_peered":"2024-09-19T07:55:02.987084+0000","last_clean":"2024-09-19T07:55:02.987084+0000","last_became_active":"2024-09-19T07:55:02.982829+0000","last_became_peered":"2024-09-19T07:55:02.982829+0000","last_unstale":"2024-09-19T07:55:02.987084+0000","last_undegraded":"2024-09-19T07:55:02.987084+0000","last_fullsized":"2024-09-19T07:55:02.987084+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:18.210530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972383+0000","last_change":"2024-09-19T07:55:01.980107+0000","last_active":"2024-09-19T07:55:03.972383+0000","last_peered":"2024-09-19T07:55:03.972383+0000","last_clean":"2024-09-19T07:55:03.972383+0000","last_became_active":"2024-09-19T07:55:01.979828+0000","last_became_peered":"2024-09-19T07:55:01.979828+0000","last_unstale":"2024-09-19T07:55:03.972383+0000","last_undegraded":"2024-09-19T07:55:03.972383+0000","last_fullsized":"2024-09-19T07:55:03.972383+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:17.205219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.988702+0000","last_change":"2024-09-19T07:55:02.986047+0000","last_active":"2024-09-19T07:55:02.988702+0000","last_peered":"2024-09-19T07:55:02.988702+0000","last_clean":"2024-09-19T07:55:02.988702+0000","last_became_active":"2024-09-19T07:55:02.985768+0000","last_became_peered":"2024-09-19T07:55:02.985768+0000","last_unstale":"2024-09-19T07:55:02.988702+0000","last_undegraded":"2024-09-19T07:55:02.988702+0000","last_fullsized":"2024-09-19T07:55:02.988702+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:27:57.147706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.982949+0000","last_change":"2024-09-19T07:55:01.980963+0000","last_active":"2024-09-19T07:55:03.982949+0000","last_peered":"2024-09-19T07:55:03.982949+0000","last_clean":"2024-09-19T07:55:03.982949+0000","last_became_active":"2024-09-19T07:55:01.980672+0000","last_became_peered":"2024-09-19T07:55:01.980672+0000","last_unstale":"2024-09-19T07:55:03.982949+0000","last_undegraded":"2024-09-19T07:55:03.982949+0000","last_fullsized":"2024-09-19T07:55:03.982949+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:11.722912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.988060+0000","last_change":"2024-09-19T07:55:02.985023+0000","last_active":"2024-09-19T07:55:02.988060+0000","last_peered":"2024-09-19T07:55:02.988060+0000","last_clean":"2024-09-19T07:55:02.988060+0000","last_became_active":"2024-09-19T07:55:02.984699+0000","last_became_peered":"2024-09-19T07:55:02.984699+0000","last_unstale":"2024-09-19T07:55:02.988060+0000","last_undegraded":"2024-09-19T07:55:02.988060+0000","last_fullsized":"2024-09-19T07:55:02.988060+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:06.465142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"139'6","reported_seq":18,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000873+0000","last_change":"2024-09-19T07:55:01.982519+0000","last_active":"2024-09-19T07:55:03.000873+0000","last_peered":"2024-09-19T07:55:03.000873+0000","last_clean":"2024-09-19T07:55:03.000873+0000","last_became_active":"2024-09-19T07:55:01.982223+0000","last_became_peered":"2024-09-19T07:55:01.982223+0000","last_unstale":"2024-09-19T07:55:03.000873+0000","last_undegraded":"2024-09-19T07:55:03.000873+0000","last_fullsized":"2024-09-19T07:55:03.000873+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:59:17.842847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":387,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983009+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:55:03.983009+0000","last_peered":"2024-09-19T07:55:03.983009+0000","last_clean":"2024-09-19T07:55:03.983009+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:55:03.983009+0000","last_undegraded":"2024-09-19T07:55:03.983009+0000","last_fullsized":"2024-09-19T07:55:03.983009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83384,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288456,"statfs":{"total":96636764160,"available":96551378944,"internally_reserved":0,"allocated":2334720,"data_stored":1397055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509588,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105080,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266760,"statfs":{"total":96636764160,"available":96529162240,"internally_reserved":0,"allocated":1740800,"data_stored":806735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509589,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73088,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298752,"statfs":{"total":96636764160,"available":96561922048,"internally_reserved":0,"allocated":1748992,"data_stored":813357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509589,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86984,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284856,"statfs":{"total":96636764160,"available":96547692544,"internally_reserved":0,"allocated":1748992,"data_stored":808277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509588,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105620,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266220,"statfs":{"total":96636764160,"available":96528609280,"internally_reserved":0,"allocated":2355200,"data_stored":1399351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542292,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118916,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252924,"statfs":{"total":96636764160,"available":96514994176,"internally_reserved":0,"allocated":1744896,"data_stored":807281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542292,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82880,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288960,"statfs":{"total":96636764160,"available":96551895040,"internally_reserved":0,"allocated":1740800,"data_stored":806735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574998,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109192,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262648,"statfs":{"total":96636764160,"available":96524951552,"internally_reserved":0,"allocated":1757184,"data_stored":813469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:55:06.663 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T07:55:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.800+0000 7f778ecba640 1 Processor -- start 2024-09-19T07:55:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.801+0000 7f778ecba640 1 -- start start 2024-09-19T07:55:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.801+0000 7f778ecba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f778815f0d0 0x7f778815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.801+0000 7f778ecba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77880d4130 0x7f77880d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.801+0000 7f778ecba640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77880d4ae0 con 0x7f77880d0070 2024-09-19T07:55:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.801+0000 7f778ecba640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f77880d1b10 con 0x7f778815f0d0 2024-09-19T07:55:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.801+0000 7f778ecba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f77880cf2b0 con 0x7f77880d4130 2024-09-19T07:55:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.801+0000 7f778ca56640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f778815f0d0 0x7f778815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.802+0000 7f778ca56640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f778815f0d0 0x7f778815f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43804/0 (socket says 172.21.15.16:43804) 2024-09-19T07:55:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.802+0000 7f778ca56640 1 -- 172.21.15.16:0/2125260802 learned_addr learned my addr 172.21.15.16:0/2125260802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.802+0000 7f778d257640 1 --2- 172.21.15.16:0/2125260802 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77880d4130 0x7f77880d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.802+0000 7f778ca56640 1 -- 172.21.15.16:0/2125260802 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77880d4130 msgr2=0x7f77880d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.802+0000 7f778ca56640 1 --2- 172.21.15.16:0/2125260802 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77880d4130 0x7f77880d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.802+0000 7f778ca56640 1 -- 172.21.15.16:0/2125260802 >> v1:172.21.15.146:6789/0 conn(0x7f77880d0070 legacy=0x7f77880d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.802+0000 7f778ca56640 1 -- 172.21.15.16:0/2125260802 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77880cf590 con 0x7f778815f0d0 2024-09-19T07:55:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.802+0000 7f778d257640 1 --2- 172.21.15.16:0/2125260802 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77880d4130 0x7f77880d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.803+0000 7f778ca56640 1 --2- 172.21.15.16:0/2125260802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f778815f0d0 0x7f778815f4b0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f7780009870 tx=0x7f778002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b4ec5d5d4134e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.803+0000 7f77877fe640 1 -- 172.21.15.16:0/2125260802 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780037070 con 0x7f778815f0d0 2024-09-19T07:55:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.803+0000 7f77877fe640 1 -- 172.21.15.16:0/2125260802 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7780032440 con 0x7f778815f0d0 2024-09-19T07:55:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.803+0000 7f77877fe640 1 -- 172.21.15.16:0/2125260802 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780032810 con 0x7f778815f0d0 2024-09-19T07:55:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.803+0000 7f778ecba640 1 -- 172.21.15.16:0/2125260802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f778815f0d0 msgr2=0x7f778815f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.803+0000 7f778ecba640 1 --2- 172.21.15.16:0/2125260802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f778815f0d0 0x7f778815f4b0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f7780009870 tx=0x7f778002a180 comp rx=0 tx=0).stop 2024-09-19T07:55:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.803+0000 7f778ecba640 1 -- 172.21.15.16:0/2125260802 shutdown_connections 2024-09-19T07:55:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.803+0000 7f778ecba640 1 --2- 172.21.15.16:0/2125260802 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77880d4130 0x7f77880d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.803+0000 7f778ecba640 1 --2- 172.21.15.16:0/2125260802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f778815f0d0 0x7f778815f4b0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.803+0000 7f778ecba640 1 -- 172.21.15.16:0/2125260802 >> 172.21.15.16:0/2125260802 conn(0x7f778809d760 msgr2=0x7f778808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.803+0000 7f778ecba640 1 -- 172.21.15.16:0/2125260802 shutdown_connections 2024-09-19T07:55:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.804+0000 7f778ecba640 1 -- 172.21.15.16:0/2125260802 wait complete. 2024-09-19T07:55:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.805+0000 7f778ecba640 1 Processor -- start 2024-09-19T07:55:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.805+0000 7f778ecba640 1 -- start start 2024-09-19T07:55:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f778ecba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77880d0070 0x7f77881e7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f778ecba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77880d4130 0x7f778815da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f778ecba640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f778815f0d0 0x7f778815acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f778ecba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77880cef60 con 0x7f77880d4130 2024-09-19T07:55:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f778ecba640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f77880cfb50 con 0x7f778815f0d0 2024-09-19T07:55:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f778ecba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f77880b2f70 con 0x7f77880d0070 2024-09-19T07:55:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f7787fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77880d4130 0x7f778815da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f7787fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77880d4130 0x7f778815da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43818/0 (socket says 172.21.15.16:43818) 2024-09-19T07:55:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f7787fff640 1 -- 172.21.15.16:0/1047180674 learned_addr learned my addr 172.21.15.16:0/1047180674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f778d257640 1 --2- 172.21.15.16:0/1047180674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f778815f0d0 0x7f778815acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f778ca56640 1 --2- 172.21.15.16:0/1047180674 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77880d0070 0x7f77881e7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f7787fff640 1 -- 172.21.15.16:0/1047180674 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77880d0070 msgr2=0x7f77881e7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f7787fff640 1 --2- 172.21.15.16:0/1047180674 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77880d0070 0x7f77881e7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f7787fff640 1 -- 172.21.15.16:0/1047180674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f778815f0d0 msgr2=0x7f778815acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f7787fff640 1 --2- 172.21.15.16:0/1047180674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f778815f0d0 0x7f778815acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f7787fff640 1 -- 172.21.15.16:0/1047180674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77881e85b0 con 0x7f77880d4130 2024-09-19T07:55:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f778ca56640 1 --2- 172.21.15.16:0/1047180674 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77880d0070 0x7f77881e7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f7787fff640 1 --2- 172.21.15.16:0/1047180674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77880d4130 0x7f778815da00 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f777800c7c0 tx=0x7f777800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f77857fa640 1 -- 172.21.15.16:0/1047180674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7778016020 con 0x7f77880d4130 2024-09-19T07:55:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f77857fa640 1 -- 172.21.15.16:0/1047180674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f777800aa50 con 0x7f77880d4130 2024-09-19T07:55:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f778ecba640 1 -- 172.21.15.16:0/1047180674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77880cc660 con 0x7f77880d4130 2024-09-19T07:55:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f77857fa640 1 -- 172.21.15.16:0/1047180674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7778012420 con 0x7f77880d4130 2024-09-19T07:55:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.806+0000 7f778ecba640 1 -- 172.21.15.16:0/1047180674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77880ccf00 con 0x7f77880d4130 2024-09-19T07:55:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.807+0000 7f778ecba640 1 -- 172.21.15.16:0/1047180674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77880cc830 con 0x7f77880d4130 2024-09-19T07:55:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.808+0000 7f77857fa640 1 -- 172.21.15.16:0/1047180674 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7778019020 con 0x7f77880d4130 2024-09-19T07:55:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.809+0000 7f77857fa640 1 --2- 172.21.15.16:0/1047180674 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f776806c2b0 0x7f776806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.809+0000 7f77857fa640 1 -- 172.21.15.16:0/1047180674 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f777808b380 con 0x7f77880d4130 2024-09-19T07:55:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.809+0000 7f778ca56640 1 --2- 172.21.15.16:0/1047180674 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f776806c2b0 0x7f776806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.809+0000 7f778ca56640 1 --2- 172.21.15.16:0/1047180674 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f776806c2b0 0x7f776806e770 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7780034000 tx=0x7f778002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:06.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:06.813+0000 7f77857fa640 1 -- 172.21.15.16:0/1047180674 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7778020070 con 0x7f77880d4130 2024-09-19T07:55:07.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.056+0000 7f778ecba640 1 -- 172.21.15.16:0/1047180674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f77880d45a0 con 0x7f77880d4130 2024-09-19T07:55:07.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.056+0000 7f77857fa640 1 -- 172.21.15.16:0/1047180674 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1478 (secure 0 0 0) 0x7f77780598c0 con 0x7f77880d4130 2024-09-19T07:55:07.056 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:07.057 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":744,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":140,"num_osds":8,"num_up_osds":8,"osd_up_since":1726731773,"num_in_osds":8,"osd_in_since":1726731764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":783507456,"bytes_avail":772310605824,"bytes_total":773094113280,"write_bytes_sec":2529,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":81,"btime":"2024-09-19T07:55:04:980435+0000","id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"a","status":"up:active","gid":8286}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-19T07:53:32.607149+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T07:55:07.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.058+0000 7f778ecba640 1 -- 172.21.15.16:0/1047180674 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f776806c2b0 msgr2=0x7f776806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:07.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.058+0000 7f778ecba640 1 --2- 172.21.15.16:0/1047180674 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f776806c2b0 0x7f776806e770 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7780034000 tx=0x7f778002e000 comp rx=0 tx=0).stop 2024-09-19T07:55:07.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.058+0000 7f778ecba640 1 -- 172.21.15.16:0/1047180674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77880d4130 msgr2=0x7f778815da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:07.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.058+0000 7f778ecba640 1 --2- 172.21.15.16:0/1047180674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77880d4130 0x7f778815da00 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f777800c7c0 tx=0x7f777800cc80 comp rx=0 tx=0).stop 2024-09-19T07:55:07.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.058+0000 7f778ecba640 1 -- 172.21.15.16:0/1047180674 shutdown_connections 2024-09-19T07:55:07.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.058+0000 7f778ecba640 1 --2- 172.21.15.16:0/1047180674 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f776806c2b0 0x7f776806e770 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:07.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.058+0000 7f778ecba640 1 --2- 172.21.15.16:0/1047180674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f778815f0d0 0x7f778815acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:07.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.058+0000 7f778ecba640 1 --2- 172.21.15.16:0/1047180674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77880d4130 0x7f778815da00 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:07.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.058+0000 7f778ecba640 1 --2- 172.21.15.16:0/1047180674 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77880d0070 0x7f77881e7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:07.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.058+0000 7f778ecba640 1 -- 172.21.15.16:0/1047180674 >> 172.21.15.16:0/1047180674 conn(0x7f778809d760 msgr2=0x7f7788161eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:07.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.058+0000 7f778ecba640 1 -- 172.21.15.16:0/1047180674 shutdown_connections 2024-09-19T07:55:07.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.058+0000 7f778ecba640 1 -- 172.21.15.16:0/1047180674 wait complete. 2024-09-19T07:55:07.069 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-19T07:55:07.069 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:55:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.171+0000 7f7e1cc53640 1 Processor -- start 2024-09-19T07:55:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.171+0000 7f7e1cc53640 1 -- start start 2024-09-19T07:55:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.172+0000 7f7e1cc53640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7e180d21f0 0x7f7e180d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.172+0000 7f7e1cc53640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e18072590 0x7f7e1806d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.172+0000 7f7e1cc53640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e180d2b10 con 0x7f7e18143a60 2024-09-19T07:55:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.172+0000 7f7e1cc53640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f7e18141750 con 0x7f7e18072590 2024-09-19T07:55:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.172+0000 7f7e1cc53640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f7e181620b0 con 0x7f7e180d21f0 2024-09-19T07:55:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.172+0000 7f7e15d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e18072590 0x7f7e1806d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.172+0000 7f7e16d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7e18143a60 0x7f7e18143f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34022/0 (socket says 172.21.15.16:34022) 2024-09-19T07:55:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.172+0000 7f7e16d9d640 1 -- 172.21.15.16:0/2366794151 learned_addr learned my addr 172.21.15.16:0/2366794151 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.172+0000 7f7e15d9b640 1 -- 172.21.15.16:0/2366794151 >> v2:172.21.15.146:3300/0 conn(0x7f7e180d21f0 msgr2=0x7f7e180d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.172+0000 7f7e15d9b640 1 --2- 172.21.15.16:0/2366794151 >> v2:172.21.15.146:3300/0 conn(0x7f7e180d21f0 0x7f7e180d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.172+0000 7f7e15d9b640 1 -- 172.21.15.16:0/2366794151 >> v1:172.21.15.16:6789/0 conn(0x7f7e18143a60 legacy=0x7f7e18143f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.172+0000 7f7e1559a640 1 -- 172.21.15.16:0/2366794151 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 60765627 0 0) 0x7f7e180d2b10 con 0x7f7e18143a60 2024-09-19T07:55:07.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.172+0000 7f7e15d9b640 1 -- 172.21.15.16:0/2366794151 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e18162370 con 0x7f7e18072590 2024-09-19T07:55:07.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.173+0000 7f7e15d9b640 1 --2- 172.21.15.16:0/2366794151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e18072590 0x7f7e1806d0e0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f7e180cfd80 tx=0x7f7e0002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e077675ffc6d787c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:07.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.173+0000 7f7e1559a640 1 -- 172.21.15.16:0/2366794151 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00037070 con 0x7f7e18072590 2024-09-19T07:55:07.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.173+0000 7f7e1559a640 1 -- 172.21.15.16:0/2366794151 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e00032440 con 0x7f7e18072590 2024-09-19T07:55:07.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.173+0000 7f7e1559a640 1 -- 172.21.15.16:0/2366794151 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0002ae60 con 0x7f7e18072590 2024-09-19T07:55:07.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.174+0000 7f7e1cc53640 1 -- 172.21.15.16:0/2366794151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e18072590 msgr2=0x7f7e1806d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:07.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.174+0000 7f7e1cc53640 1 --2- 172.21.15.16:0/2366794151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e18072590 0x7f7e1806d0e0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f7e180cfd80 tx=0x7f7e0002a110 comp rx=0 tx=0).stop 2024-09-19T07:55:07.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.174+0000 7f7e1cc53640 1 -- 172.21.15.16:0/2366794151 shutdown_connections 2024-09-19T07:55:07.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.174+0000 7f7e1cc53640 1 --2- 172.21.15.16:0/2366794151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e18072590 0x7f7e1806d0e0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:07.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.174+0000 7f7e1cc53640 1 --2- 172.21.15.16:0/2366794151 >> v2:172.21.15.146:3300/0 conn(0x7f7e180d21f0 0x7f7e180d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:07.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.174+0000 7f7e1cc53640 1 -- 172.21.15.16:0/2366794151 >> 172.21.15.16:0/2366794151 conn(0x7f7e180b3ed0 msgr2=0x7f7e180b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:07.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.174+0000 7f7e1cc53640 1 -- 172.21.15.16:0/2366794151 shutdown_connections 2024-09-19T07:55:07.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.174+0000 7f7e1cc53640 1 -- 172.21.15.16:0/2366794151 wait complete. 2024-09-19T07:55:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.175+0000 7f7e1cc53640 1 Processor -- start 2024-09-19T07:55:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.175+0000 7f7e1cc53640 1 -- start start 2024-09-19T07:55:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.176+0000 7f7e1cc53640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e18072590 0x7f7e181590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.176+0000 7f7e1cc53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e180d21f0 0x7f7e1815b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.176+0000 7f7e1cc53640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e18143a60 0x7f7e1815bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.176+0000 7f7e1cc53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18162c10 con 0x7f7e180d21f0 2024-09-19T07:55:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.176+0000 7f7e1cc53640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18162930 con 0x7f7e18143a60 2024-09-19T07:55:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.176+0000 7f7e1cc53640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7e1806d620 con 0x7f7e18072590 2024-09-19T07:55:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.176+0000 7f7e15d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e180d21f0 0x7f7e1815b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.176+0000 7f7e15d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e180d21f0 0x7f7e1815b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43830/0 (socket says 172.21.15.16:43830) 2024-09-19T07:55:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.176+0000 7f7e15d9b640 1 -- 172.21.15.16:0/1126278897 learned_addr learned my addr 172.21.15.16:0/1126278897 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.176+0000 7f7e16d9d640 1 --2- 172.21.15.16:0/1126278897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e18143a60 0x7f7e1815bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.176+0000 7f7e1659c640 1 --2- 172.21.15.16:0/1126278897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e18072590 0x7f7e181590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.177+0000 7f7e15d9b640 1 -- 172.21.15.16:0/1126278897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e18072590 msgr2=0x7f7e181590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.177+0000 7f7e15d9b640 1 --2- 172.21.15.16:0/1126278897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e18072590 0x7f7e181590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.177+0000 7f7e15d9b640 1 -- 172.21.15.16:0/1126278897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e18143a60 msgr2=0x7f7e1815bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.177+0000 7f7e15d9b640 1 --2- 172.21.15.16:0/1126278897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e18143a60 0x7f7e1815bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.177+0000 7f7e15d9b640 1 -- 172.21.15.16:0/1126278897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e181e9e20 con 0x7f7e180d21f0 2024-09-19T07:55:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.177+0000 7f7e15d9b640 1 --2- 172.21.15.16:0/1126278897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e180d21f0 0x7f7e1815b620 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009890 tx=0x7f7e000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.177+0000 7f7e0effd640 1 -- 172.21.15.16:0/1126278897 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00041070 con 0x7f7e180d21f0 2024-09-19T07:55:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.177+0000 7f7e0effd640 1 -- 172.21.15.16:0/1126278897 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e0003c890 con 0x7f7e180d21f0 2024-09-19T07:55:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.177+0000 7f7e1cc53640 1 -- 172.21.15.16:0/1126278897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e181ea100 con 0x7f7e180d21f0 2024-09-19T07:55:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.177+0000 7f7e0effd640 1 -- 172.21.15.16:0/1126278897 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00037040 con 0x7f7e180d21f0 2024-09-19T07:55:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.177+0000 7f7e1cc53640 1 -- 172.21.15.16:0/1126278897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e181ea6c0 con 0x7f7e180d21f0 2024-09-19T07:55:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.178+0000 7f7e1cc53640 1 -- 172.21.15.16:0/1126278897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e18158c30 con 0x7f7e180d21f0 2024-09-19T07:55:07.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.178+0000 7f7e0effd640 1 -- 172.21.15.16:0/1126278897 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7e00044070 con 0x7f7e180d21f0 2024-09-19T07:55:07.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.179+0000 7f7e0effd640 1 --2- 172.21.15.16:0/1126278897 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7df806c2b0 0x7f7df806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:07.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.179+0000 7f7e0effd640 1 -- 172.21.15.16:0/1126278897 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f7e000ac440 con 0x7f7e180d21f0 2024-09-19T07:55:07.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.179+0000 7f7e1659c640 1 --2- 172.21.15.16:0/1126278897 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7df806c2b0 0x7f7df806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:07.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.180+0000 7f7e1659c640 1 --2- 172.21.15.16:0/1126278897 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7df806c2b0 0x7f7df806e770 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009f20 tx=0x7f7e080094f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:07.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.183+0000 7f7e0effd640 1 -- 172.21.15.16:0/1126278897 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7e0007aab0 con 0x7f7e180d21f0 2024-09-19T07:55:07.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.356+0000 7f7e1cc53640 1 -- 172.21.15.16:0/1126278897 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e18072290 con 0x7f7df806c2b0 2024-09-19T07:55:07.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.362+0000 7f7e0effd640 1 -- 172.21.15.16:0/1126278897 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252457 (secure 0 0 0) 0x7f7e18072290 con 0x7f7df806c2b0 2024-09-19T07:55:07.365 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:07.366 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:55:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.365+0000 7f7e1cc53640 1 -- 172.21.15.16:0/1126278897 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7df806c2b0 msgr2=0x7f7df806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.365+0000 7f7e1cc53640 1 --2- 172.21.15.16:0/1126278897 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7df806c2b0 0x7f7df806e770 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009f20 tx=0x7f7e080094f0 comp rx=0 tx=0).stop 2024-09-19T07:55:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.365+0000 7f7e1cc53640 1 -- 172.21.15.16:0/1126278897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e180d21f0 msgr2=0x7f7e1815b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.365+0000 7f7e1cc53640 1 --2- 172.21.15.16:0/1126278897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e180d21f0 0x7f7e1815b620 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009890 tx=0x7f7e000046c0 comp rx=0 tx=0).stop 2024-09-19T07:55:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.365+0000 7f7e1cc53640 1 -- 172.21.15.16:0/1126278897 shutdown_connections 2024-09-19T07:55:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.365+0000 7f7e1cc53640 1 --2- 172.21.15.16:0/1126278897 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7df806c2b0 0x7f7df806e770 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.365+0000 7f7e1cc53640 1 --2- 172.21.15.16:0/1126278897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e18143a60 0x7f7e1815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.365+0000 7f7e1cc53640 1 --2- 172.21.15.16:0/1126278897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e180d21f0 0x7f7e1815b620 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.365+0000 7f7e1cc53640 1 --2- 172.21.15.16:0/1126278897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e18072590 0x7f7e181590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:07.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.365+0000 7f7e1cc53640 1 -- 172.21.15.16:0/1126278897 >> 172.21.15.16:0/1126278897 conn(0x7f7e180b3ed0 msgr2=0x7f7e181422f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:07.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.365+0000 7f7e1cc53640 1 -- 172.21.15.16:0/1126278897 shutdown_connections 2024-09-19T07:55:07.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:07.365+0000 7f7e1cc53640 1 -- 172.21.15.16:0/1126278897 wait complete. 2024-09-19T07:55:07.377 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":501,"stamp":"2024-09-19T07:55:06.629540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":765144,"kb_used_data":14816,"kb_used_omap":12,"kb_used_meta":750131,"kb_avail":754209576,"statfs":{"total":773094113280,"available":772310605824,"internally_reserved":0,"allocated":15171584,"data_stored":7652260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":768134704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.668548"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972662+0000","last_change":"2024-09-19T07:55:02.977838+0000","last_active":"2024-09-19T07:55:03.972662+0000","last_peered":"2024-09-19T07:55:03.972662+0000","last_clean":"2024-09-19T07:55:03.972662+0000","last_became_active":"2024-09-19T07:55:02.977562+0000","last_became_peered":"2024-09-19T07:55:02.977562+0000","last_unstale":"2024-09-19T07:55:03.972662+0000","last_undegraded":"2024-09-19T07:55:03.972662+0000","last_fullsized":"2024-09-19T07:55:03.972662+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:18.348990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.994497+0000","last_change":"2024-09-19T07:55:02.988730+0000","last_active":"2024-09-19T07:55:02.994497+0000","last_peered":"2024-09-19T07:55:02.994497+0000","last_clean":"2024-09-19T07:55:02.994497+0000","last_became_active":"2024-09-19T07:55:02.988454+0000","last_became_peered":"2024-09-19T07:55:02.988454+0000","last_unstale":"2024-09-19T07:55:02.994497+0000","last_undegraded":"2024-09-19T07:55:02.994497+0000","last_fullsized":"2024-09-19T07:55:02.994497+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:55.150368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998989+0000","last_change":"2024-09-19T07:55:02.995513+0000","last_active":"2024-09-19T07:55:02.998989+0000","last_peered":"2024-09-19T07:55:02.998989+0000","last_clean":"2024-09-19T07:55:02.998989+0000","last_became_active":"2024-09-19T07:55:02.995162+0000","last_became_peered":"2024-09-19T07:55:02.995162+0000","last_unstale":"2024-09-19T07:55:02.998989+0000","last_undegraded":"2024-09-19T07:55:02.998989+0000","last_fullsized":"2024-09-19T07:55:02.998989+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:08.826237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972077+0000","last_change":"2024-09-19T07:55:02.991747+0000","last_active":"2024-09-19T07:55:03.972077+0000","last_peered":"2024-09-19T07:55:03.972077+0000","last_clean":"2024-09-19T07:55:03.972077+0000","last_became_active":"2024-09-19T07:55:02.991487+0000","last_became_peered":"2024-09-19T07:55:02.991487+0000","last_unstale":"2024-09-19T07:55:03.972077+0000","last_undegraded":"2024-09-19T07:55:03.972077+0000","last_fullsized":"2024-09-19T07:55:03.972077+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:00.867259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.978550+0000","last_change":"2024-09-19T07:55:02.974381+0000","last_active":"2024-09-19T07:55:02.978550+0000","last_peered":"2024-09-19T07:55:02.978550+0000","last_clean":"2024-09-19T07:55:02.978550+0000","last_became_active":"2024-09-19T07:55:02.974105+0000","last_became_peered":"2024-09-19T07:55:02.974105+0000","last_unstale":"2024-09-19T07:55:02.978550+0000","last_undegraded":"2024-09-19T07:55:02.978550+0000","last_fullsized":"2024-09-19T07:55:02.978550+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:08.519276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.980270+0000","last_change":"2024-09-19T07:55:02.975703+0000","last_active":"2024-09-19T07:55:02.980270+0000","last_peered":"2024-09-19T07:55:02.980270+0000","last_clean":"2024-09-19T07:55:02.980270+0000","last_became_active":"2024-09-19T07:55:02.975379+0000","last_became_peered":"2024-09-19T07:55:02.975379+0000","last_unstale":"2024-09-19T07:55:02.980270+0000","last_undegraded":"2024-09-19T07:55:02.980270+0000","last_fullsized":"2024-09-19T07:55:02.980270+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:11.846213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972184+0000","last_change":"2024-09-19T07:55:02.985662+0000","last_active":"2024-09-19T07:55:03.972184+0000","last_peered":"2024-09-19T07:55:03.972184+0000","last_clean":"2024-09-19T07:55:03.972184+0000","last_became_active":"2024-09-19T07:55:02.985378+0000","last_became_peered":"2024-09-19T07:55:02.985378+0000","last_unstale":"2024-09-19T07:55:03.972184+0000","last_undegraded":"2024-09-19T07:55:03.972184+0000","last_fullsized":"2024-09-19T07:55:03.972184+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:33.679405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.992880+0000","last_change":"2024-09-19T07:55:02.991159+0000","last_active":"2024-09-19T07:55:02.992880+0000","last_peered":"2024-09-19T07:55:02.992880+0000","last_clean":"2024-09-19T07:55:02.992880+0000","last_became_active":"2024-09-19T07:55:02.990631+0000","last_became_peered":"2024-09-19T07:55:02.990631+0000","last_unstale":"2024-09-19T07:55:02.992880+0000","last_undegraded":"2024-09-19T07:55:02.992880+0000","last_fullsized":"2024-09-19T07:55:02.992880+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:16.038904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998379+0000","last_change":"2024-09-19T07:55:02.994532+0000","last_active":"2024-09-19T07:55:02.998379+0000","last_peered":"2024-09-19T07:55:02.998379+0000","last_clean":"2024-09-19T07:55:02.998379+0000","last_became_active":"2024-09-19T07:55:02.994201+0000","last_became_peered":"2024-09-19T07:55:02.994201+0000","last_unstale":"2024-09-19T07:55:02.998379+0000","last_undegraded":"2024-09-19T07:55:02.998379+0000","last_fullsized":"2024-09-19T07:55:02.998379+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:12.950630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.984004+0000","last_change":"2024-09-19T07:55:02.981864+0000","last_active":"2024-09-19T07:55:02.984004+0000","last_peered":"2024-09-19T07:55:02.984004+0000","last_clean":"2024-09-19T07:55:02.984004+0000","last_became_active":"2024-09-19T07:55:02.981541+0000","last_became_peered":"2024-09-19T07:55:02.981541+0000","last_unstale":"2024-09-19T07:55:02.984004+0000","last_undegraded":"2024-09-19T07:55:02.984004+0000","last_fullsized":"2024-09-19T07:55:02.984004+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:22.255417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845763+0000","last_change":"2024-09-19T07:55:02.991467+0000","last_active":"2024-09-19T07:55:04.845763+0000","last_peered":"2024-09-19T07:55:04.845763+0000","last_clean":"2024-09-19T07:55:04.845763+0000","last_became_active":"2024-09-19T07:55:02.991185+0000","last_became_peered":"2024-09-19T07:55:02.991185+0000","last_unstale":"2024-09-19T07:55:04.845763+0000","last_undegraded":"2024-09-19T07:55:04.845763+0000","last_fullsized":"2024-09-19T07:55:04.845763+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:25:03.509906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972187+0000","last_change":"2024-09-19T07:55:02.989911+0000","last_active":"2024-09-19T07:55:03.972187+0000","last_peered":"2024-09-19T07:55:03.972187+0000","last_clean":"2024-09-19T07:55:03.972187+0000","last_became_active":"2024-09-19T07:55:02.989574+0000","last_became_peered":"2024-09-19T07:55:02.989574+0000","last_unstale":"2024-09-19T07:55:03.972187+0000","last_undegraded":"2024-09-19T07:55:03.972187+0000","last_fullsized":"2024-09-19T07:55:03.972187+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:20.107114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.985584+0000","last_change":"2024-09-19T07:55:02.983842+0000","last_active":"2024-09-19T07:55:02.985584+0000","last_peered":"2024-09-19T07:55:02.985584+0000","last_clean":"2024-09-19T07:55:02.985584+0000","last_became_active":"2024-09-19T07:55:02.983583+0000","last_became_peered":"2024-09-19T07:55:02.983583+0000","last_unstale":"2024-09-19T07:55:02.985584+0000","last_undegraded":"2024-09-19T07:55:02.985584+0000","last_fullsized":"2024-09-19T07:55:02.985584+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:58.534832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.971850+0000","last_change":"2024-09-19T07:55:02.991151+0000","last_active":"2024-09-19T07:55:03.971850+0000","last_peered":"2024-09-19T07:55:03.971850+0000","last_clean":"2024-09-19T07:55:03.971850+0000","last_became_active":"2024-09-19T07:55:02.990791+0000","last_became_peered":"2024-09-19T07:55:02.990791+0000","last_unstale":"2024-09-19T07:55:03.971850+0000","last_undegraded":"2024-09-19T07:55:03.971850+0000","last_fullsized":"2024-09-19T07:55:03.971850+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:16.038904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.997603+0000","last_change":"2024-09-19T07:55:02.993711+0000","last_active":"2024-09-19T07:55:02.997603+0000","last_peered":"2024-09-19T07:55:02.997603+0000","last_clean":"2024-09-19T07:55:02.997603+0000","last_became_active":"2024-09-19T07:55:02.993391+0000","last_became_peered":"2024-09-19T07:55:02.993391+0000","last_unstale":"2024-09-19T07:55:02.997603+0000","last_undegraded":"2024-09-19T07:55:02.997603+0000","last_fullsized":"2024-09-19T07:55:02.997603+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:53:11.464426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.981640+0000","last_change":"2024-09-19T07:55:02.977783+0000","last_active":"2024-09-19T07:55:02.981640+0000","last_peered":"2024-09-19T07:55:02.981640+0000","last_clean":"2024-09-19T07:55:02.981640+0000","last_became_active":"2024-09-19T07:55:02.977494+0000","last_became_peered":"2024-09-19T07:55:02.977494+0000","last_unstale":"2024-09-19T07:55:02.981640+0000","last_undegraded":"2024-09-19T07:55:02.981640+0000","last_fullsized":"2024-09-19T07:55:02.981640+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:42.284168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972758+0000","last_change":"2024-09-19T07:55:02.989847+0000","last_active":"2024-09-19T07:55:03.972758+0000","last_peered":"2024-09-19T07:55:03.972758+0000","last_clean":"2024-09-19T07:55:03.972758+0000","last_became_active":"2024-09-19T07:55:02.989566+0000","last_became_peered":"2024-09-19T07:55:02.989566+0000","last_unstale":"2024-09-19T07:55:03.972758+0000","last_undegraded":"2024-09-19T07:55:03.972758+0000","last_fullsized":"2024-09-19T07:55:03.972758+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:57:07.989977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845741+0000","last_change":"2024-09-19T07:55:02.988518+0000","last_active":"2024-09-19T07:55:04.845741+0000","last_peered":"2024-09-19T07:55:04.845741+0000","last_clean":"2024-09-19T07:55:04.845741+0000","last_became_active":"2024-09-19T07:55:02.988215+0000","last_became_peered":"2024-09-19T07:55:02.988215+0000","last_unstale":"2024-09-19T07:55:04.845741+0000","last_undegraded":"2024-09-19T07:55:04.845741+0000","last_fullsized":"2024-09-19T07:55:04.845741+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:33.944874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983370+0000","last_change":"2024-09-19T07:55:02.988233+0000","last_active":"2024-09-19T07:55:03.983370+0000","last_peered":"2024-09-19T07:55:03.983370+0000","last_clean":"2024-09-19T07:55:03.983370+0000","last_became_active":"2024-09-19T07:55:02.987586+0000","last_became_peered":"2024-09-19T07:55:02.987586+0000","last_unstale":"2024-09-19T07:55:03.983370+0000","last_undegraded":"2024-09-19T07:55:03.983370+0000","last_fullsized":"2024-09-19T07:55:03.983370+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:37.091603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.989123+0000","last_change":"2024-09-19T07:55:02.984872+0000","last_active":"2024-09-19T07:55:02.989123+0000","last_peered":"2024-09-19T07:55:02.989123+0000","last_clean":"2024-09-19T07:55:02.989123+0000","last_became_active":"2024-09-19T07:55:02.984588+0000","last_became_peered":"2024-09-19T07:55:02.984588+0000","last_unstale":"2024-09-19T07:55:02.989123+0000","last_undegraded":"2024-09-19T07:55:02.989123+0000","last_fullsized":"2024-09-19T07:55:02.989123+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:37.091603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.986844+0000","last_change":"2024-09-19T07:55:02.985444+0000","last_active":"2024-09-19T07:55:02.986844+0000","last_peered":"2024-09-19T07:55:02.986844+0000","last_clean":"2024-09-19T07:55:02.986844+0000","last_became_active":"2024-09-19T07:55:02.985161+0000","last_became_peered":"2024-09-19T07:55:02.985161+0000","last_unstale":"2024-09-19T07:55:02.986844+0000","last_undegraded":"2024-09-19T07:55:02.986844+0000","last_fullsized":"2024-09-19T07:55:02.986844+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:19.760108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972782+0000","last_change":"2024-09-19T07:55:02.980246+0000","last_active":"2024-09-19T07:55:03.972782+0000","last_peered":"2024-09-19T07:55:03.972782+0000","last_clean":"2024-09-19T07:55:03.972782+0000","last_became_active":"2024-09-19T07:55:02.979970+0000","last_became_peered":"2024-09-19T07:55:02.979970+0000","last_unstale":"2024-09-19T07:55:03.972782+0000","last_undegraded":"2024-09-19T07:55:03.972782+0000","last_fullsized":"2024-09-19T07:55:03.972782+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:19.717109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.986210+0000","last_change":"2024-09-19T07:55:02.979612+0000","last_active":"2024-09-19T07:55:02.986210+0000","last_peered":"2024-09-19T07:55:02.986210+0000","last_clean":"2024-09-19T07:55:02.986210+0000","last_became_active":"2024-09-19T07:55:02.978902+0000","last_became_peered":"2024-09-19T07:55:02.978902+0000","last_unstale":"2024-09-19T07:55:02.986210+0000","last_undegraded":"2024-09-19T07:55:02.986210+0000","last_fullsized":"2024-09-19T07:55:02.986210+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:57.305718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845596+0000","last_change":"2024-09-19T07:55:02.987092+0000","last_active":"2024-09-19T07:55:04.845596+0000","last_peered":"2024-09-19T07:55:04.845596+0000","last_clean":"2024-09-19T07:55:04.845596+0000","last_became_active":"2024-09-19T07:55:02.986604+0000","last_became_peered":"2024-09-19T07:55:02.986604+0000","last_unstale":"2024-09-19T07:55:04.845596+0000","last_undegraded":"2024-09-19T07:55:04.845596+0000","last_fullsized":"2024-09-19T07:55:04.845596+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:30:29.389217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972885+0000","last_change":"2024-09-19T07:55:02.981645+0000","last_active":"2024-09-19T07:55:03.972885+0000","last_peered":"2024-09-19T07:55:03.972885+0000","last_clean":"2024-09-19T07:55:03.972885+0000","last_became_active":"2024-09-19T07:55:02.981377+0000","last_became_peered":"2024-09-19T07:55:02.981377+0000","last_unstale":"2024-09-19T07:55:03.972885+0000","last_undegraded":"2024-09-19T07:55:03.972885+0000","last_fullsized":"2024-09-19T07:55:03.972885+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:28.867264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972976+0000","last_change":"2024-09-19T07:55:02.991055+0000","last_active":"2024-09-19T07:55:03.972976+0000","last_peered":"2024-09-19T07:55:03.972976+0000","last_clean":"2024-09-19T07:55:03.972976+0000","last_became_active":"2024-09-19T07:55:02.990517+0000","last_became_peered":"2024-09-19T07:55:02.990517+0000","last_unstale":"2024-09-19T07:55:03.972976+0000","last_undegraded":"2024-09-19T07:55:03.972976+0000","last_fullsized":"2024-09-19T07:55:03.972976+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:52.586504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.999462+0000","last_change":"2024-09-19T07:55:02.998488+0000","last_active":"2024-09-19T07:55:02.999462+0000","last_peered":"2024-09-19T07:55:02.999462+0000","last_clean":"2024-09-19T07:55:02.999462+0000","last_became_active":"2024-09-19T07:55:02.998192+0000","last_became_peered":"2024-09-19T07:55:02.998192+0000","last_unstale":"2024-09-19T07:55:02.999462+0000","last_undegraded":"2024-09-19T07:55:02.999462+0000","last_fullsized":"2024-09-19T07:55:02.999462+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:20.318482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845428+0000","last_change":"2024-09-19T07:55:02.988808+0000","last_active":"2024-09-19T07:55:04.845428+0000","last_peered":"2024-09-19T07:55:04.845428+0000","last_clean":"2024-09-19T07:55:04.845428+0000","last_became_active":"2024-09-19T07:55:02.988524+0000","last_became_peered":"2024-09-19T07:55:02.988524+0000","last_unstale":"2024-09-19T07:55:04.845428+0000","last_undegraded":"2024-09-19T07:55:04.845428+0000","last_fullsized":"2024-09-19T07:55:04.845428+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:14.948481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.987062+0000","last_change":"2024-09-19T07:55:02.985372+0000","last_active":"2024-09-19T07:55:02.987062+0000","last_peered":"2024-09-19T07:55:02.987062+0000","last_clean":"2024-09-19T07:55:02.987062+0000","last_became_active":"2024-09-19T07:55:02.985109+0000","last_became_peered":"2024-09-19T07:55:02.985109+0000","last_unstale":"2024-09-19T07:55:02.987062+0000","last_undegraded":"2024-09-19T07:55:02.987062+0000","last_fullsized":"2024-09-19T07:55:02.987062+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:59.971722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845582+0000","last_change":"2024-09-19T07:55:02.993268+0000","last_active":"2024-09-19T07:55:04.845582+0000","last_peered":"2024-09-19T07:55:04.845582+0000","last_clean":"2024-09-19T07:55:04.845582+0000","last_became_active":"2024-09-19T07:55:02.992933+0000","last_became_peered":"2024-09-19T07:55:02.992933+0000","last_unstale":"2024-09-19T07:55:04.845582+0000","last_undegraded":"2024-09-19T07:55:04.845582+0000","last_fullsized":"2024-09-19T07:55:04.845582+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:34:34.252831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.981739+0000","last_change":"2024-09-19T07:55:02.978386+0000","last_active":"2024-09-19T07:55:02.981739+0000","last_peered":"2024-09-19T07:55:02.981739+0000","last_clean":"2024-09-19T07:55:02.981739+0000","last_became_active":"2024-09-19T07:55:02.978134+0000","last_became_peered":"2024-09-19T07:55:02.978134+0000","last_unstale":"2024-09-19T07:55:02.981739+0000","last_undegraded":"2024-09-19T07:55:02.981739+0000","last_fullsized":"2024-09-19T07:55:02.981739+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:06.762003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.992859+0000","last_change":"2024-09-19T07:55:02.990915+0000","last_active":"2024-09-19T07:55:02.992859+0000","last_peered":"2024-09-19T07:55:02.992859+0000","last_clean":"2024-09-19T07:55:02.992859+0000","last_became_active":"2024-09-19T07:55:02.990273+0000","last_became_peered":"2024-09-19T07:55:02.990273+0000","last_unstale":"2024-09-19T07:55:02.992859+0000","last_undegraded":"2024-09-19T07:55:02.992859+0000","last_fullsized":"2024-09-19T07:55:02.992859+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:25.070703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998804+0000","last_change":"2024-09-19T07:55:02.994240+0000","last_active":"2024-09-19T07:55:02.998804+0000","last_peered":"2024-09-19T07:55:02.998804+0000","last_clean":"2024-09-19T07:55:02.998804+0000","last_became_active":"2024-09-19T07:55:02.993966+0000","last_became_peered":"2024-09-19T07:55:02.993966+0000","last_unstale":"2024-09-19T07:55:02.998804+0000","last_undegraded":"2024-09-19T07:55:02.998804+0000","last_fullsized":"2024-09-19T07:55:02.998804+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:18.497803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"139'4","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000708+0000","last_change":"2024-09-19T07:55:01.982819+0000","last_active":"2024-09-19T07:55:03.000708+0000","last_peered":"2024-09-19T07:55:03.000708+0000","last_clean":"2024-09-19T07:55:03.000708+0000","last_became_active":"2024-09-19T07:55:01.982455+0000","last_became_peered":"2024-09-19T07:55:01.982455+0000","last_unstale":"2024-09-19T07:55:03.000708+0000","last_undegraded":"2024-09-19T07:55:03.000708+0000","last_fullsized":"2024-09-19T07:55:03.000708+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:01.483855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.995361+0000","last_change":"2024-09-19T07:55:02.988748+0000","last_active":"2024-09-19T07:55:02.995361+0000","last_peered":"2024-09-19T07:55:02.995361+0000","last_clean":"2024-09-19T07:55:02.995361+0000","last_became_active":"2024-09-19T07:55:02.988417+0000","last_became_peered":"2024-09-19T07:55:02.988417+0000","last_unstale":"2024-09-19T07:55:02.995361+0000","last_undegraded":"2024-09-19T07:55:02.995361+0000","last_fullsized":"2024-09-19T07:55:02.995361+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:11.124464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983330+0000","last_change":"2024-09-19T07:55:01.986317+0000","last_active":"2024-09-19T07:55:03.983330+0000","last_peered":"2024-09-19T07:55:03.983330+0000","last_clean":"2024-09-19T07:55:03.983330+0000","last_became_active":"2024-09-19T07:55:01.986050+0000","last_became_peered":"2024-09-19T07:55:01.986050+0000","last_unstale":"2024-09-19T07:55:03.983330+0000","last_undegraded":"2024-09-19T07:55:03.983330+0000","last_fullsized":"2024-09-19T07:55:03.983330+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:20.699942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.988705+0000","last_change":"2024-09-19T07:55:02.988084+0000","last_active":"2024-09-19T07:55:02.988705+0000","last_peered":"2024-09-19T07:55:02.988705+0000","last_clean":"2024-09-19T07:55:02.988705+0000","last_became_active":"2024-09-19T07:55:02.987819+0000","last_became_peered":"2024-09-19T07:55:02.987819+0000","last_unstale":"2024-09-19T07:55:02.988705+0000","last_undegraded":"2024-09-19T07:55:02.988705+0000","last_fullsized":"2024-09-19T07:55:02.988705+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:34.829241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983598+0000","last_change":"2024-09-19T07:55:01.983959+0000","last_active":"2024-09-19T07:55:03.983598+0000","last_peered":"2024-09-19T07:55:03.983598+0000","last_clean":"2024-09-19T07:55:03.983598+0000","last_became_active":"2024-09-19T07:55:01.983682+0000","last_became_peered":"2024-09-19T07:55:01.983682+0000","last_unstale":"2024-09-19T07:55:03.983598+0000","last_undegraded":"2024-09-19T07:55:03.983598+0000","last_fullsized":"2024-09-19T07:55:03.983598+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:17.343679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983514+0000","last_change":"2024-09-19T07:55:02.988272+0000","last_active":"2024-09-19T07:55:03.983514+0000","last_peered":"2024-09-19T07:55:03.983514+0000","last_clean":"2024-09-19T07:55:03.983514+0000","last_became_active":"2024-09-19T07:55:02.987972+0000","last_became_peered":"2024-09-19T07:55:02.987972+0000","last_unstale":"2024-09-19T07:55:03.983514+0000","last_undegraded":"2024-09-19T07:55:03.983514+0000","last_fullsized":"2024-09-19T07:55:03.983514+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:55.597883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"139'2","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000148+0000","last_change":"2024-09-19T07:55:01.975504+0000","last_active":"2024-09-19T07:55:03.000148+0000","last_peered":"2024-09-19T07:55:03.000148+0000","last_clean":"2024-09-19T07:55:03.000148+0000","last_became_active":"2024-09-19T07:55:01.975216+0000","last_became_peered":"2024-09-19T07:55:01.975216+0000","last_unstale":"2024-09-19T07:55:03.000148+0000","last_undegraded":"2024-09-19T07:55:03.000148+0000","last_fullsized":"2024-09-19T07:55:03.000148+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:47.640841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.990742+0000","last_change":"2024-09-19T07:55:02.990235+0000","last_active":"2024-09-19T07:55:02.990742+0000","last_peered":"2024-09-19T07:55:02.990742+0000","last_clean":"2024-09-19T07:55:02.990742+0000","last_became_active":"2024-09-19T07:55:02.989920+0000","last_became_peered":"2024-09-19T07:55:02.989920+0000","last_unstale":"2024-09-19T07:55:02.990742+0000","last_undegraded":"2024-09-19T07:55:02.990742+0000","last_fullsized":"2024-09-19T07:55:02.990742+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:55.597883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"139'1","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.997333+0000","last_change":"2024-09-19T07:55:01.984133+0000","last_active":"2024-09-19T07:55:02.997333+0000","last_peered":"2024-09-19T07:55:02.997333+0000","last_clean":"2024-09-19T07:55:02.997333+0000","last_became_active":"2024-09-19T07:55:01.983829+0000","last_became_peered":"2024-09-19T07:55:01.983829+0000","last_unstale":"2024-09-19T07:55:02.997333+0000","last_undegraded":"2024-09-19T07:55:02.997333+0000","last_fullsized":"2024-09-19T07:55:02.997333+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:08.327895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.971878+0000","last_change":"2024-09-19T07:55:02.990053+0000","last_active":"2024-09-19T07:55:03.971878+0000","last_peered":"2024-09-19T07:55:03.971878+0000","last_clean":"2024-09-19T07:55:03.971878+0000","last_became_active":"2024-09-19T07:55:02.987609+0000","last_became_peered":"2024-09-19T07:55:02.987609+0000","last_unstale":"2024-09-19T07:55:03.971878+0000","last_undegraded":"2024-09-19T07:55:03.971878+0000","last_fullsized":"2024-09-19T07:55:03.971878+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:02:28.317679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"139'1","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998584+0000","last_change":"2024-09-19T07:55:01.984540+0000","last_active":"2024-09-19T07:55:02.998584+0000","last_peered":"2024-09-19T07:55:02.998584+0000","last_clean":"2024-09-19T07:55:02.998584+0000","last_became_active":"2024-09-19T07:55:01.984218+0000","last_became_peered":"2024-09-19T07:55:01.984218+0000","last_unstale":"2024-09-19T07:55:02.998584+0000","last_undegraded":"2024-09-19T07:55:02.998584+0000","last_fullsized":"2024-09-19T07:55:02.998584+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:59:17.842847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.987109+0000","last_change":"2024-09-19T07:55:02.983739+0000","last_active":"2024-09-19T07:55:02.987109+0000","last_peered":"2024-09-19T07:55:02.987109+0000","last_clean":"2024-09-19T07:55:02.987109+0000","last_became_active":"2024-09-19T07:55:02.983448+0000","last_became_peered":"2024-09-19T07:55:02.983448+0000","last_unstale":"2024-09-19T07:55:02.987109+0000","last_undegraded":"2024-09-19T07:55:02.987109+0000","last_fullsized":"2024-09-19T07:55:02.987109+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:50.858294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972280+0000","last_change":"2024-09-19T07:55:01.978901+0000","last_active":"2024-09-19T07:55:03.972280+0000","last_peered":"2024-09-19T07:55:03.972280+0000","last_clean":"2024-09-19T07:55:03.972280+0000","last_became_active":"2024-09-19T07:55:01.978399+0000","last_became_peered":"2024-09-19T07:55:01.978399+0000","last_unstale":"2024-09-19T07:55:03.972280+0000","last_undegraded":"2024-09-19T07:55:03.972280+0000","last_fullsized":"2024-09-19T07:55:03.972280+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:11.363759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972118+0000","last_change":"2024-09-19T07:55:02.978549+0000","last_active":"2024-09-19T07:55:03.972118+0000","last_peered":"2024-09-19T07:55:03.972118+0000","last_clean":"2024-09-19T07:55:03.972118+0000","last_became_active":"2024-09-19T07:55:02.978270+0000","last_became_peered":"2024-09-19T07:55:02.978270+0000","last_unstale":"2024-09-19T07:55:03.972118+0000","last_undegraded":"2024-09-19T07:55:03.972118+0000","last_fullsized":"2024-09-19T07:55:03.972118+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:21.705254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.969267+0000","last_change":"2024-09-19T07:55:01.983833+0000","last_active":"2024-09-19T07:55:02.969267+0000","last_peered":"2024-09-19T07:55:02.969267+0000","last_clean":"2024-09-19T07:55:02.969267+0000","last_became_active":"2024-09-19T07:55:01.983553+0000","last_became_peered":"2024-09-19T07:55:01.983553+0000","last_unstale":"2024-09-19T07:55:02.969267+0000","last_undegraded":"2024-09-19T07:55:02.969267+0000","last_fullsized":"2024-09-19T07:55:02.969267+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:02.576442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.986319+0000","last_change":"2024-09-19T07:55:02.983084+0000","last_active":"2024-09-19T07:55:02.986319+0000","last_peered":"2024-09-19T07:55:02.986319+0000","last_clean":"2024-09-19T07:55:02.986319+0000","last_became_active":"2024-09-19T07:55:02.982793+0000","last_became_peered":"2024-09-19T07:55:02.982793+0000","last_unstale":"2024-09-19T07:55:02.986319+0000","last_undegraded":"2024-09-19T07:55:02.986319+0000","last_fullsized":"2024-09-19T07:55:02.986319+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:39:22.911029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983489+0000","last_change":"2024-09-19T07:55:01.980547+0000","last_active":"2024-09-19T07:55:03.983489+0000","last_peered":"2024-09-19T07:55:03.983489+0000","last_clean":"2024-09-19T07:55:03.983489+0000","last_became_active":"2024-09-19T07:55:01.980253+0000","last_became_peered":"2024-09-19T07:55:01.980253+0000","last_unstale":"2024-09-19T07:55:03.983489+0000","last_undegraded":"2024-09-19T07:55:03.983489+0000","last_fullsized":"2024-09-19T07:55:03.983489+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:32.674094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845937+0000","last_change":"2024-09-19T07:55:02.990524+0000","last_active":"2024-09-19T07:55:04.845937+0000","last_peered":"2024-09-19T07:55:04.845937+0000","last_clean":"2024-09-19T07:55:04.845937+0000","last_became_active":"2024-09-19T07:55:02.990227+0000","last_became_peered":"2024-09-19T07:55:02.990227+0000","last_unstale":"2024-09-19T07:55:04.845937+0000","last_undegraded":"2024-09-19T07:55:04.845937+0000","last_fullsized":"2024-09-19T07:55:04.845937+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:14:38.518293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.973223+0000","last_change":"2024-09-19T07:55:01.980870+0000","last_active":"2024-09-19T07:55:03.973223+0000","last_peered":"2024-09-19T07:55:03.973223+0000","last_clean":"2024-09-19T07:55:03.973223+0000","last_became_active":"2024-09-19T07:55:01.980569+0000","last_became_peered":"2024-09-19T07:55:01.980569+0000","last_unstale":"2024-09-19T07:55:03.973223+0000","last_undegraded":"2024-09-19T07:55:03.973223+0000","last_fullsized":"2024-09-19T07:55:03.973223+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:56:13.952641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845892+0000","last_change":"2024-09-19T07:55:02.987773+0000","last_active":"2024-09-19T07:55:04.845892+0000","last_peered":"2024-09-19T07:55:04.845892+0000","last_clean":"2024-09-19T07:55:04.845892+0000","last_became_active":"2024-09-19T07:55:02.987513+0000","last_became_peered":"2024-09-19T07:55:02.987513+0000","last_unstale":"2024-09-19T07:55:04.845892+0000","last_undegraded":"2024-09-19T07:55:04.845892+0000","last_fullsized":"2024-09-19T07:55:04.845892+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:25.245393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"139'3","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845998+0000","last_change":"2024-09-19T07:55:01.978949+0000","last_active":"2024-09-19T07:55:04.845998+0000","last_peered":"2024-09-19T07:55:04.845998+0000","last_clean":"2024-09-19T07:55:04.845998+0000","last_became_active":"2024-09-19T07:55:01.978508+0000","last_became_peered":"2024-09-19T07:55:01.978508+0000","last_unstale":"2024-09-19T07:55:04.845998+0000","last_undegraded":"2024-09-19T07:55:04.845998+0000","last_fullsized":"2024-09-19T07:55:04.845998+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:33:49.871640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.986247+0000","last_change":"2024-09-19T07:55:02.980794+0000","last_active":"2024-09-19T07:55:02.986247+0000","last_peered":"2024-09-19T07:55:02.986247+0000","last_clean":"2024-09-19T07:55:02.986247+0000","last_became_active":"2024-09-19T07:55:02.980530+0000","last_became_peered":"2024-09-19T07:55:02.980530+0000","last_unstale":"2024-09-19T07:55:02.986247+0000","last_undegraded":"2024-09-19T07:55:02.986247+0000","last_fullsized":"2024-09-19T07:55:02.986247+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:14.838884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"139'2","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000534+0000","last_change":"2024-09-19T07:55:01.984318+0000","last_active":"2024-09-19T07:55:03.000534+0000","last_peered":"2024-09-19T07:55:03.000534+0000","last_clean":"2024-09-19T07:55:03.000534+0000","last_became_active":"2024-09-19T07:55:01.983325+0000","last_became_peered":"2024-09-19T07:55:01.983325+0000","last_unstale":"2024-09-19T07:55:03.000534+0000","last_undegraded":"2024-09-19T07:55:03.000534+0000","last_fullsized":"2024-09-19T07:55:03.000534+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:36.086291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.999112+0000","last_change":"2024-09-19T07:55:02.996430+0000","last_active":"2024-09-19T07:55:02.999112+0000","last_peered":"2024-09-19T07:55:02.999112+0000","last_clean":"2024-09-19T07:55:02.999112+0000","last_became_active":"2024-09-19T07:55:02.996148+0000","last_became_peered":"2024-09-19T07:55:02.996148+0000","last_unstale":"2024-09-19T07:55:02.999112+0000","last_undegraded":"2024-09-19T07:55:02.999112+0000","last_fullsized":"2024-09-19T07:55:02.999112+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:12.728223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972021+0000","last_change":"2024-09-19T07:55:01.988068+0000","last_active":"2024-09-19T07:55:03.972021+0000","last_peered":"2024-09-19T07:55:03.972021+0000","last_clean":"2024-09-19T07:55:03.972021+0000","last_became_active":"2024-09-19T07:55:01.987761+0000","last_became_peered":"2024-09-19T07:55:01.987761+0000","last_unstale":"2024-09-19T07:55:03.972021+0000","last_undegraded":"2024-09-19T07:55:03.972021+0000","last_fullsized":"2024-09-19T07:55:03.972021+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:56.300407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.994831+0000","last_change":"2024-09-19T07:55:02.989642+0000","last_active":"2024-09-19T07:55:02.994831+0000","last_peered":"2024-09-19T07:55:02.994831+0000","last_clean":"2024-09-19T07:55:02.994831+0000","last_became_active":"2024-09-19T07:55:02.989315+0000","last_became_peered":"2024-09-19T07:55:02.989315+0000","last_unstale":"2024-09-19T07:55:02.994831+0000","last_undegraded":"2024-09-19T07:55:02.994831+0000","last_fullsized":"2024-09-19T07:55:02.994831+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:29.434870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972162+0000","last_change":"2024-09-19T07:55:01.983411+0000","last_active":"2024-09-19T07:55:03.972162+0000","last_peered":"2024-09-19T07:55:03.972162+0000","last_clean":"2024-09-19T07:55:03.972162+0000","last_became_active":"2024-09-19T07:55:01.982413+0000","last_became_peered":"2024-09-19T07:55:01.982413+0000","last_unstale":"2024-09-19T07:55:03.972162+0000","last_undegraded":"2024-09-19T07:55:03.972162+0000","last_fullsized":"2024-09-19T07:55:03.972162+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:58.316347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.985547+0000","last_change":"2024-09-19T07:55:02.983361+0000","last_active":"2024-09-19T07:55:02.985547+0000","last_peered":"2024-09-19T07:55:02.985547+0000","last_clean":"2024-09-19T07:55:02.985547+0000","last_became_active":"2024-09-19T07:55:02.983032+0000","last_became_peered":"2024-09-19T07:55:02.983032+0000","last_unstale":"2024-09-19T07:55:02.985547+0000","last_undegraded":"2024-09-19T07:55:02.985547+0000","last_fullsized":"2024-09-19T07:55:02.985547+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:56:14.957952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"139'8","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.971849+0000","last_change":"2024-09-19T07:55:01.978831+0000","last_active":"2024-09-19T07:55:03.971849+0000","last_peered":"2024-09-19T07:55:03.971849+0000","last_clean":"2024-09-19T07:55:03.971849+0000","last_became_active":"2024-09-19T07:55:01.978370+0000","last_became_peered":"2024-09-19T07:55:01.978370+0000","last_unstale":"2024-09-19T07:55:03.971849+0000","last_undegraded":"2024-09-19T07:55:03.971849+0000","last_fullsized":"2024-09-19T07:55:03.971849+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:47.640841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983367+0000","last_change":"2024-09-19T07:55:02.998722+0000","last_active":"2024-09-19T07:55:03.983367+0000","last_peered":"2024-09-19T07:55:03.983367+0000","last_clean":"2024-09-19T07:55:03.983367+0000","last_became_active":"2024-09-19T07:55:02.998420+0000","last_became_peered":"2024-09-19T07:55:02.998420+0000","last_unstale":"2024-09-19T07:55:03.983367+0000","last_undegraded":"2024-09-19T07:55:03.983367+0000","last_fullsized":"2024-09-19T07:55:03.983367+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:02.489166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845663+0000","last_change":"2024-09-19T07:55:01.983892+0000","last_active":"2024-09-19T07:55:04.845663+0000","last_peered":"2024-09-19T07:55:04.845663+0000","last_clean":"2024-09-19T07:55:04.845663+0000","last_became_active":"2024-09-19T07:55:01.983621+0000","last_became_peered":"2024-09-19T07:55:01.983621+0000","last_unstale":"2024-09-19T07:55:04.845663+0000","last_undegraded":"2024-09-19T07:55:04.845663+0000","last_fullsized":"2024-09-19T07:55:04.845663+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:35:59.714032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983513+0000","last_change":"2024-09-19T07:55:02.988579+0000","last_active":"2024-09-19T07:55:03.983513+0000","last_peered":"2024-09-19T07:55:03.983513+0000","last_clean":"2024-09-19T07:55:03.983513+0000","last_became_active":"2024-09-19T07:55:02.987958+0000","last_became_peered":"2024-09-19T07:55:02.987958+0000","last_unstale":"2024-09-19T07:55:03.983513+0000","last_undegraded":"2024-09-19T07:55:03.983513+0000","last_fullsized":"2024-09-19T07:55:03.983513+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:08.519276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983438+0000","last_change":"2024-09-19T07:55:01.981723+0000","last_active":"2024-09-19T07:55:03.983438+0000","last_peered":"2024-09-19T07:55:03.983438+0000","last_clean":"2024-09-19T07:55:03.983438+0000","last_became_active":"2024-09-19T07:55:01.981465+0000","last_became_peered":"2024-09-19T07:55:01.981465+0000","last_unstale":"2024-09-19T07:55:03.983438+0000","last_undegraded":"2024-09-19T07:55:03.983438+0000","last_fullsized":"2024-09-19T07:55:03.983438+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:12.838363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.973123+0000","last_change":"2024-09-19T07:55:02.979256+0000","last_active":"2024-09-19T07:55:03.973123+0000","last_peered":"2024-09-19T07:55:03.973123+0000","last_clean":"2024-09-19T07:55:03.973123+0000","last_became_active":"2024-09-19T07:55:02.978921+0000","last_became_peered":"2024-09-19T07:55:02.978921+0000","last_unstale":"2024-09-19T07:55:03.973123+0000","last_undegraded":"2024-09-19T07:55:03.973123+0000","last_fullsized":"2024-09-19T07:55:03.973123+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:51.978120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"139'1","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000384+0000","last_change":"2024-09-19T07:55:01.982720+0000","last_active":"2024-09-19T07:55:03.000384+0000","last_peered":"2024-09-19T07:55:03.000384+0000","last_clean":"2024-09-19T07:55:03.000384+0000","last_became_active":"2024-09-19T07:55:01.982277+0000","last_became_peered":"2024-09-19T07:55:01.982277+0000","last_unstale":"2024-09-19T07:55:03.000384+0000","last_undegraded":"2024-09-19T07:55:03.000384+0000","last_fullsized":"2024-09-19T07:55:03.000384+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:59.197262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972931+0000","last_change":"2024-09-19T07:55:02.989059+0000","last_active":"2024-09-19T07:55:03.972931+0000","last_peered":"2024-09-19T07:55:03.972931+0000","last_clean":"2024-09-19T07:55:03.972931+0000","last_became_active":"2024-09-19T07:55:02.988577+0000","last_became_peered":"2024-09-19T07:55:02.988577+0000","last_unstale":"2024-09-19T07:55:03.972931+0000","last_undegraded":"2024-09-19T07:55:03.972931+0000","last_fullsized":"2024-09-19T07:55:03.972931+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:49.502676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972169+0000","last_change":"2024-09-19T07:55:01.988779+0000","last_active":"2024-09-19T07:55:03.972169+0000","last_peered":"2024-09-19T07:55:03.972169+0000","last_clean":"2024-09-19T07:55:03.972169+0000","last_became_active":"2024-09-19T07:55:01.988419+0000","last_became_peered":"2024-09-19T07:55:01.988419+0000","last_unstale":"2024-09-19T07:55:03.972169+0000","last_undegraded":"2024-09-19T07:55:03.972169+0000","last_fullsized":"2024-09-19T07:55:03.972169+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:59.785587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.982978+0000","last_change":"2024-09-19T07:55:02.987148+0000","last_active":"2024-09-19T07:55:03.982978+0000","last_peered":"2024-09-19T07:55:03.982978+0000","last_clean":"2024-09-19T07:55:03.982978+0000","last_became_active":"2024-09-19T07:55:02.986889+0000","last_became_peered":"2024-09-19T07:55:02.986889+0000","last_unstale":"2024-09-19T07:55:03.982978+0000","last_undegraded":"2024-09-19T07:55:03.982978+0000","last_fullsized":"2024-09-19T07:55:03.982978+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:26:18.944316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"139'2","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.997309+0000","last_change":"2024-09-19T07:55:01.986115+0000","last_active":"2024-09-19T07:55:02.997309+0000","last_peered":"2024-09-19T07:55:02.997309+0000","last_clean":"2024-09-19T07:55:02.997309+0000","last_became_active":"2024-09-19T07:55:01.985827+0000","last_became_peered":"2024-09-19T07:55:01.985827+0000","last_unstale":"2024-09-19T07:55:02.997309+0000","last_undegraded":"2024-09-19T07:55:02.997309+0000","last_fullsized":"2024-09-19T07:55:02.997309+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:01.483855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.989398+0000","last_change":"2024-09-19T07:55:02.987365+0000","last_active":"2024-09-19T07:55:02.989398+0000","last_peered":"2024-09-19T07:55:02.989398+0000","last_clean":"2024-09-19T07:55:02.989398+0000","last_became_active":"2024-09-19T07:55:02.986480+0000","last_became_peered":"2024-09-19T07:55:02.986480+0000","last_unstale":"2024-09-19T07:55:02.989398+0000","last_undegraded":"2024-09-19T07:55:02.989398+0000","last_fullsized":"2024-09-19T07:55:02.989398+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:32.859864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.972920+0000","last_change":"2024-09-19T07:55:01.981717+0000","last_active":"2024-09-19T07:55:02.972920+0000","last_peered":"2024-09-19T07:55:02.972920+0000","last_clean":"2024-09-19T07:55:02.972920+0000","last_became_active":"2024-09-19T07:55:01.981004+0000","last_became_peered":"2024-09-19T07:55:01.981004+0000","last_unstale":"2024-09-19T07:55:02.972920+0000","last_undegraded":"2024-09-19T07:55:02.972920+0000","last_fullsized":"2024-09-19T07:55:02.972920+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:58.316347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983112+0000","last_change":"2024-09-19T07:55:02.992093+0000","last_active":"2024-09-19T07:55:03.983112+0000","last_peered":"2024-09-19T07:55:03.983112+0000","last_clean":"2024-09-19T07:55:03.983112+0000","last_became_active":"2024-09-19T07:55:02.991829+0000","last_became_peered":"2024-09-19T07:55:02.991829+0000","last_unstale":"2024-09-19T07:55:03.983112+0000","last_undegraded":"2024-09-19T07:55:03.983112+0000","last_fullsized":"2024-09-19T07:55:03.983112+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:59:18.848159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.974687+0000","last_change":"2024-09-19T07:55:01.983244+0000","last_active":"2024-09-19T07:55:02.974687+0000","last_peered":"2024-09-19T07:55:02.974687+0000","last_clean":"2024-09-19T07:55:02.974687+0000","last_became_active":"2024-09-19T07:55:01.982960+0000","last_became_peered":"2024-09-19T07:55:01.982960+0000","last_unstale":"2024-09-19T07:55:02.974687+0000","last_undegraded":"2024-09-19T07:55:02.974687+0000","last_fullsized":"2024-09-19T07:55:02.974687+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:26:17.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.998969+0000","last_change":"2024-09-19T07:55:02.993715+0000","last_active":"2024-09-19T07:55:02.998969+0000","last_peered":"2024-09-19T07:55:02.998969+0000","last_clean":"2024-09-19T07:55:02.998969+0000","last_became_active":"2024-09-19T07:55:02.993409+0000","last_became_peered":"2024-09-19T07:55:02.993409+0000","last_unstale":"2024-09-19T07:55:02.998969+0000","last_undegraded":"2024-09-19T07:55:02.998969+0000","last_fullsized":"2024-09-19T07:55:02.998969+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:13.897172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.974976+0000","last_change":"2024-09-19T07:55:01.982335+0000","last_active":"2024-09-19T07:55:02.974976+0000","last_peered":"2024-09-19T07:55:02.974976+0000","last_clean":"2024-09-19T07:55:02.974976+0000","last_became_active":"2024-09-19T07:55:01.982052+0000","last_became_peered":"2024-09-19T07:55:01.982052+0000","last_unstale":"2024-09-19T07:55:02.974976+0000","last_undegraded":"2024-09-19T07:55:02.974976+0000","last_fullsized":"2024-09-19T07:55:02.974976+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:59.785587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.975028+0000","last_change":"2024-09-19T07:55:02.974014+0000","last_active":"2024-09-19T07:55:02.975028+0000","last_peered":"2024-09-19T07:55:02.975028+0000","last_clean":"2024-09-19T07:55:02.975028+0000","last_became_active":"2024-09-19T07:55:02.973536+0000","last_became_peered":"2024-09-19T07:55:02.973536+0000","last_unstale":"2024-09-19T07:55:02.975028+0000","last_undegraded":"2024-09-19T07:55:02.975028+0000","last_fullsized":"2024-09-19T07:55:02.975028+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:55.597883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972720+0000","last_change":"2024-09-19T07:55:01.988008+0000","last_active":"2024-09-19T07:55:03.972720+0000","last_peered":"2024-09-19T07:55:03.972720+0000","last_clean":"2024-09-19T07:55:03.972720+0000","last_became_active":"2024-09-19T07:55:01.987725+0000","last_became_peered":"2024-09-19T07:55:01.987725+0000","last_unstale":"2024-09-19T07:55:03.972720+0000","last_undegraded":"2024-09-19T07:55:03.972720+0000","last_fullsized":"2024-09-19T07:55:03.972720+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:13.833572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972516+0000","last_change":"2024-09-19T07:55:02.991309+0000","last_active":"2024-09-19T07:55:03.972516+0000","last_peered":"2024-09-19T07:55:03.972516+0000","last_clean":"2024-09-19T07:55:03.972516+0000","last_became_active":"2024-09-19T07:55:02.991045+0000","last_became_peered":"2024-09-19T07:55:02.991045+0000","last_unstale":"2024-09-19T07:55:03.972516+0000","last_undegraded":"2024-09-19T07:55:03.972516+0000","last_fullsized":"2024-09-19T07:55:03.972516+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:38.709571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"140'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.982444+0000","last_change":"2024-09-19T07:55:01.978226+0000","last_active":"2024-09-19T07:55:03.982444+0000","last_peered":"2024-09-19T07:55:03.982444+0000","last_clean":"2024-09-19T07:55:03.982444+0000","last_became_active":"2024-09-19T07:55:01.977803+0000","last_became_peered":"2024-09-19T07:55:01.977803+0000","last_unstale":"2024-09-19T07:55:03.982444+0000","last_undegraded":"2024-09-19T07:55:03.982444+0000","last_fullsized":"2024-09-19T07:55:03.982444+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:49.852983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.973122+0000","last_change":"2024-09-19T07:55:02.977147+0000","last_active":"2024-09-19T07:55:03.973122+0000","last_peered":"2024-09-19T07:55:03.973122+0000","last_clean":"2024-09-19T07:55:03.973122+0000","last_became_active":"2024-09-19T07:55:02.976832+0000","last_became_peered":"2024-09-19T07:55:02.976832+0000","last_unstale":"2024-09-19T07:55:03.973122+0000","last_undegraded":"2024-09-19T07:55:03.973122+0000","last_fullsized":"2024-09-19T07:55:03.973122+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:03.581754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.974160+0000","last_change":"2024-09-19T07:55:01.974880+0000","last_active":"2024-09-19T07:55:02.974160+0000","last_peered":"2024-09-19T07:55:02.974160+0000","last_clean":"2024-09-19T07:55:02.974160+0000","last_became_active":"2024-09-19T07:55:01.974599+0000","last_became_peered":"2024-09-19T07:55:01.974599+0000","last_unstale":"2024-09-19T07:55:02.974160+0000","last_undegraded":"2024-09-19T07:55:02.974160+0000","last_fullsized":"2024-09-19T07:55:02.974160+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:11.363759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.994783+0000","last_change":"2024-09-19T07:55:02.989699+0000","last_active":"2024-09-19T07:55:02.994783+0000","last_peered":"2024-09-19T07:55:02.994783+0000","last_clean":"2024-09-19T07:55:02.994783+0000","last_became_active":"2024-09-19T07:55:02.989398+0000","last_became_peered":"2024-09-19T07:55:02.989398+0000","last_unstale":"2024-09-19T07:55:02.994783+0000","last_undegraded":"2024-09-19T07:55:02.994783+0000","last_fullsized":"2024-09-19T07:55:02.994783+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:11.153885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.970064+0000","last_change":"2024-09-19T07:55:01.981690+0000","last_active":"2024-09-19T07:55:02.970064+0000","last_peered":"2024-09-19T07:55:02.970064+0000","last_clean":"2024-09-19T07:55:02.970064+0000","last_became_active":"2024-09-19T07:55:01.981417+0000","last_became_peered":"2024-09-19T07:55:01.981417+0000","last_unstale":"2024-09-19T07:55:02.970064+0000","last_undegraded":"2024-09-19T07:55:02.970064+0000","last_fullsized":"2024-09-19T07:55:02.970064+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:26:17.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.988647+0000","last_change":"2024-09-19T07:55:02.986076+0000","last_active":"2024-09-19T07:55:02.988647+0000","last_peered":"2024-09-19T07:55:02.988647+0000","last_clean":"2024-09-19T07:55:02.988647+0000","last_became_active":"2024-09-19T07:55:02.984885+0000","last_became_peered":"2024-09-19T07:55:02.984885+0000","last_unstale":"2024-09-19T07:55:02.988647+0000","last_undegraded":"2024-09-19T07:55:02.988647+0000","last_fullsized":"2024-09-19T07:55:02.988647+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:33.113553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972219+0000","last_change":"2024-09-19T07:55:01.986920+0000","last_active":"2024-09-19T07:55:03.972219+0000","last_peered":"2024-09-19T07:55:03.972219+0000","last_clean":"2024-09-19T07:55:03.972219+0000","last_became_active":"2024-09-19T07:55:01.986618+0000","last_became_peered":"2024-09-19T07:55:01.986618+0000","last_unstale":"2024-09-19T07:55:03.972219+0000","last_undegraded":"2024-09-19T07:55:03.972219+0000","last_fullsized":"2024-09-19T07:55:03.972219+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:59.197262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972053+0000","last_change":"2024-09-19T07:55:02.989902+0000","last_active":"2024-09-19T07:55:03.972053+0000","last_peered":"2024-09-19T07:55:03.972053+0000","last_clean":"2024-09-19T07:55:03.972053+0000","last_became_active":"2024-09-19T07:55:02.987293+0000","last_became_peered":"2024-09-19T07:55:02.987293+0000","last_unstale":"2024-09-19T07:55:03.972053+0000","last_undegraded":"2024-09-19T07:55:03.972053+0000","last_fullsized":"2024-09-19T07:55:03.972053+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:54:28.829885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845473+0000","last_change":"2024-09-19T07:55:01.977073+0000","last_active":"2024-09-19T07:55:04.845473+0000","last_peered":"2024-09-19T07:55:04.845473+0000","last_clean":"2024-09-19T07:55:04.845473+0000","last_became_active":"2024-09-19T07:55:01.976782+0000","last_became_peered":"2024-09-19T07:55:01.976782+0000","last_unstale":"2024-09-19T07:55:04.845473+0000","last_undegraded":"2024-09-19T07:55:04.845473+0000","last_fullsized":"2024-09-19T07:55:04.845473+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:38.579539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.987084+0000","last_change":"2024-09-19T07:55:02.983221+0000","last_active":"2024-09-19T07:55:02.987084+0000","last_peered":"2024-09-19T07:55:02.987084+0000","last_clean":"2024-09-19T07:55:02.987084+0000","last_became_active":"2024-09-19T07:55:02.982829+0000","last_became_peered":"2024-09-19T07:55:02.982829+0000","last_unstale":"2024-09-19T07:55:02.987084+0000","last_undegraded":"2024-09-19T07:55:02.987084+0000","last_fullsized":"2024-09-19T07:55:02.987084+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:18.210530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972383+0000","last_change":"2024-09-19T07:55:01.980107+0000","last_active":"2024-09-19T07:55:03.972383+0000","last_peered":"2024-09-19T07:55:03.972383+0000","last_clean":"2024-09-19T07:55:03.972383+0000","last_became_active":"2024-09-19T07:55:01.979828+0000","last_became_peered":"2024-09-19T07:55:01.979828+0000","last_unstale":"2024-09-19T07:55:03.972383+0000","last_undegraded":"2024-09-19T07:55:03.972383+0000","last_fullsized":"2024-09-19T07:55:03.972383+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:17.205219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.988702+0000","last_change":"2024-09-19T07:55:02.986047+0000","last_active":"2024-09-19T07:55:02.988702+0000","last_peered":"2024-09-19T07:55:02.988702+0000","last_clean":"2024-09-19T07:55:02.988702+0000","last_became_active":"2024-09-19T07:55:02.985768+0000","last_became_peered":"2024-09-19T07:55:02.985768+0000","last_unstale":"2024-09-19T07:55:02.988702+0000","last_undegraded":"2024-09-19T07:55:02.988702+0000","last_fullsized":"2024-09-19T07:55:02.988702+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:27:57.147706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.982949+0000","last_change":"2024-09-19T07:55:01.980963+0000","last_active":"2024-09-19T07:55:03.982949+0000","last_peered":"2024-09-19T07:55:03.982949+0000","last_clean":"2024-09-19T07:55:03.982949+0000","last_became_active":"2024-09-19T07:55:01.980672+0000","last_became_peered":"2024-09-19T07:55:01.980672+0000","last_unstale":"2024-09-19T07:55:03.982949+0000","last_undegraded":"2024-09-19T07:55:03.982949+0000","last_fullsized":"2024-09-19T07:55:03.982949+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:11.722912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:02.988060+0000","last_change":"2024-09-19T07:55:02.985023+0000","last_active":"2024-09-19T07:55:02.988060+0000","last_peered":"2024-09-19T07:55:02.988060+0000","last_clean":"2024-09-19T07:55:02.988060+0000","last_became_active":"2024-09-19T07:55:02.984699+0000","last_became_peered":"2024-09-19T07:55:02.984699+0000","last_unstale":"2024-09-19T07:55:02.988060+0000","last_undegraded":"2024-09-19T07:55:02.988060+0000","last_fullsized":"2024-09-19T07:55:02.988060+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:06.465142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"139'6","reported_seq":18,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.000873+0000","last_change":"2024-09-19T07:55:01.982519+0000","last_active":"2024-09-19T07:55:03.000873+0000","last_peered":"2024-09-19T07:55:03.000873+0000","last_clean":"2024-09-19T07:55:03.000873+0000","last_became_active":"2024-09-19T07:55:01.982223+0000","last_became_peered":"2024-09-19T07:55:01.982223+0000","last_unstale":"2024-09-19T07:55:03.000873+0000","last_undegraded":"2024-09-19T07:55:03.000873+0000","last_fullsized":"2024-09-19T07:55:03.000873+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:59:17.842847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":387,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983009+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:55:03.983009+0000","last_peered":"2024-09-19T07:55:03.983009+0000","last_clean":"2024-09-19T07:55:03.983009+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:55:03.983009+0000","last_undegraded":"2024-09-19T07:55:03.983009+0000","last_fullsized":"2024-09-19T07:55:03.983009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83384,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288456,"statfs":{"total":96636764160,"available":96551378944,"internally_reserved":0,"allocated":2334720,"data_stored":1397055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509588,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105080,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266760,"statfs":{"total":96636764160,"available":96529162240,"internally_reserved":0,"allocated":1740800,"data_stored":806735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509589,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73088,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298752,"statfs":{"total":96636764160,"available":96561922048,"internally_reserved":0,"allocated":1748992,"data_stored":813357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509589,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86984,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284856,"statfs":{"total":96636764160,"available":96547692544,"internally_reserved":0,"allocated":1748992,"data_stored":808277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509588,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105620,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266220,"statfs":{"total":96636764160,"available":96528609280,"internally_reserved":0,"allocated":2355200,"data_stored":1399351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542292,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118916,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252924,"statfs":{"total":96636764160,"available":96514994176,"internally_reserved":0,"allocated":1744896,"data_stored":807281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542292,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82880,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288960,"statfs":{"total":96636764160,"available":96551895040,"internally_reserved":0,"allocated":1740800,"data_stored":806735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574998,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109192,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262648,"statfs":{"total":96636764160,"available":96524951552,"internally_reserved":0,"allocated":1757184,"data_stored":813469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:55:08.259 INFO:tasks.ceph.mds.a.smithi016.stderr:2024-09-19T07:55:08.259+0000 7f9e5f53a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T07:55:09.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:55:09.345 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:55:10.382 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:55:10.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.497+0000 7f7349396640 1 Processor -- start 2024-09-19T07:55:10.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.497+0000 7f7349396640 1 -- start start 2024-09-19T07:55:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.498+0000 7f7349396640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7344141680 0x7f73440cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.498+0000 7f7349396640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f73440d0370 0x7f7344141a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.498+0000 7f7349396640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7344141fa0 0x7f73440cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.498+0000 7f7349396640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f73440d07e0 con 0x7f73440d0370 2024-09-19T07:55:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.498+0000 7f7349396640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f734415c6f0 con 0x7f7344141680 2024-09-19T07:55:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.498+0000 7f7349396640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f734415d710 con 0x7f7344141fa0 2024-09-19T07:55:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.498+0000 7f73427fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f73440d0370 0x7f7344141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.498+0000 7f73427fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f73440d0370 0x7f7344141a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43838/0 (socket says 172.21.15.16:43838) 2024-09-19T07:55:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.498+0000 7f73427fc640 1 -- 172.21.15.16:0/3146040975 learned_addr learned my addr 172.21.15.16:0/3146040975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.498+0000 7f7342ffd640 1 --2- 172.21.15.16:0/3146040975 >> v2:172.21.15.146:3300/0 conn(0x7f7344141680 0x7f73440cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.498+0000 7f73437fe640 1 --2- 172.21.15.16:0/3146040975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7344141fa0 0x7f73440cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.499+0000 7f73427fc640 1 -- 172.21.15.16:0/3146040975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7344141fa0 msgr2=0x7f73440cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.499+0000 7f73427fc640 1 --2- 172.21.15.16:0/3146040975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7344141fa0 0x7f73440cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.499+0000 7f73427fc640 1 -- 172.21.15.16:0/3146040975 >> v2:172.21.15.146:3300/0 conn(0x7f7344141680 msgr2=0x7f73440cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.499+0000 7f73427fc640 1 --2- 172.21.15.16:0/3146040975 >> v2:172.21.15.146:3300/0 conn(0x7f7344141680 0x7f73440cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.499+0000 7f73427fc640 1 -- 172.21.15.16:0/3146040975 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734415c950 con 0x7f73440d0370 2024-09-19T07:55:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.499+0000 7f73427fc640 1 --2- 172.21.15.16:0/3146040975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73440d0370 0x7f7344141a60 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f7344070fc0 tx=0x7f732c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdf6ee18e56d452 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.500+0000 7f7341ffb640 1 -- 172.21.15.16:0/3146040975 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c037070 con 0x7f73440d0370 2024-09-19T07:55:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.500+0000 7f7341ffb640 1 -- 172.21.15.16:0/3146040975 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f732c032440 con 0x7f73440d0370 2024-09-19T07:55:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.500+0000 7f7341ffb640 1 -- 172.21.15.16:0/3146040975 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c032810 con 0x7f73440d0370 2024-09-19T07:55:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.500+0000 7f7349396640 1 -- 172.21.15.16:0/3146040975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73440d0370 msgr2=0x7f7344141a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.500+0000 7f7349396640 1 --2- 172.21.15.16:0/3146040975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73440d0370 0x7f7344141a60 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f7344070fc0 tx=0x7f732c02a210 comp rx=0 tx=0).stop 2024-09-19T07:55:10.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.500+0000 7f7349396640 1 -- 172.21.15.16:0/3146040975 shutdown_connections 2024-09-19T07:55:10.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.500+0000 7f7349396640 1 --2- 172.21.15.16:0/3146040975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7344141fa0 0x7f73440cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.500+0000 7f7349396640 1 --2- 172.21.15.16:0/3146040975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73440d0370 0x7f7344141a60 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.500+0000 7f7349396640 1 --2- 172.21.15.16:0/3146040975 >> v2:172.21.15.146:3300/0 conn(0x7f7344141680 0x7f73440cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.500+0000 7f7349396640 1 -- 172.21.15.16:0/3146040975 >> 172.21.15.16:0/3146040975 conn(0x7f73440b3ed0 msgr2=0x7f73440b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:10.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.500+0000 7f7349396640 1 -- 172.21.15.16:0/3146040975 shutdown_connections 2024-09-19T07:55:10.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.500+0000 7f7349396640 1 -- 172.21.15.16:0/3146040975 wait complete. 2024-09-19T07:55:10.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.502+0000 7f7349396640 1 Processor -- start 2024-09-19T07:55:10.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.502+0000 7f7349396640 1 -- start start 2024-09-19T07:55:10.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.502+0000 7f7349396640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73440d0370 0x7f734413e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:10.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.502+0000 7f7349396640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7344141680 0x7f73440b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:10.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.502+0000 7f7349396640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7344141fa0 0x7f73441f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:10.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.502+0000 7f7349396640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f734415d1f0 con 0x7f73440d0370 2024-09-19T07:55:10.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.502+0000 7f7349396640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f734415cf10 con 0x7f7344141fa0 2024-09-19T07:55:10.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.502+0000 7f7349396640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f73441423f0 con 0x7f7344141680 2024-09-19T07:55:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.503+0000 7f7342ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73440d0370 0x7f734413e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.503+0000 7f7342ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73440d0370 0x7f734413e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43854/0 (socket says 172.21.15.16:43854) 2024-09-19T07:55:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.503+0000 7f7342ffd640 1 -- 172.21.15.16:0/2142325943 learned_addr learned my addr 172.21.15.16:0/2142325943 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.503+0000 7f73427fc640 1 --2- 172.21.15.16:0/2142325943 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7344141680 0x7f73440b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.503+0000 7f73437fe640 1 --2- 172.21.15.16:0/2142325943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7344141fa0 0x7f73441f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.503+0000 7f7342ffd640 1 -- 172.21.15.16:0/2142325943 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7344141680 msgr2=0x7f73440b7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.503+0000 7f7342ffd640 1 --2- 172.21.15.16:0/2142325943 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7344141680 0x7f73440b7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.503+0000 7f7342ffd640 1 -- 172.21.15.16:0/2142325943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7344141fa0 msgr2=0x7f73441f42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.503+0000 7f7342ffd640 1 --2- 172.21.15.16:0/2142325943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7344141fa0 0x7f73441f42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.503+0000 7f7342ffd640 1 -- 172.21.15.16:0/2142325943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73440cc890 con 0x7f73440d0370 2024-09-19T07:55:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.503+0000 7f7342ffd640 1 --2- 172.21.15.16:0/2142325943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73440d0370 0x7f734413e5d0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f733400e8c0 tx=0x7f733400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.503+0000 7f733b7fe640 1 -- 172.21.15.16:0/2142325943 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334017070 con 0x7f73440d0370 2024-09-19T07:55:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.503+0000 7f733b7fe640 1 -- 172.21.15.16:0/2142325943 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f733400c940 con 0x7f73440d0370 2024-09-19T07:55:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.503+0000 7f7349396640 1 -- 172.21.15.16:0/2142325943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73440b5f50 con 0x7f73440d0370 2024-09-19T07:55:10.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.503+0000 7f733b7fe640 1 -- 172.21.15.16:0/2142325943 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733400cd10 con 0x7f73440d0370 2024-09-19T07:55:10.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.503+0000 7f7349396640 1 -- 172.21.15.16:0/2142325943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73440b6510 con 0x7f73440d0370 2024-09-19T07:55:10.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.505+0000 7f7349396640 1 -- 172.21.15.16:0/2142325943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7344141a60 con 0x7f73440d0370 2024-09-19T07:55:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.505+0000 7f733b7fe640 1 -- 172.21.15.16:0/2142325943 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f733401a070 con 0x7f73440d0370 2024-09-19T07:55:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.506+0000 7f733b7fe640 1 --2- 172.21.15.16:0/2142325943 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f732406c2b0 0x7f732406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.506+0000 7f733b7fe640 1 -- 172.21.15.16:0/2142325943 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f733408b4d0 con 0x7f73440d0370 2024-09-19T07:55:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.506+0000 7f73427fc640 1 --2- 172.21.15.16:0/2142325943 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f732406c2b0 0x7f732406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.506+0000 7f73427fc640 1 --2- 172.21.15.16:0/2142325943 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f732406c2b0 0x7f732406e770 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f732c034000 tx=0x7f732c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:10.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.510+0000 7f733b7fe640 1 -- 172.21.15.16:0/2142325943 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7334059460 con 0x7f73440d0370 2024-09-19T07:55:10.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.683+0000 7f7349396640 1 -- 172.21.15.16:0/2142325943 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f73440690c0 con 0x7f732406c2b0 2024-09-19T07:55:10.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.689+0000 7f733b7fe640 1 -- 172.21.15.16:0/2142325943 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252410 (secure 0 0 0) 0x7f73440690c0 con 0x7f732406c2b0 2024-09-19T07:55:10.693 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:10.694 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:55:10.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.692+0000 7f7349396640 1 -- 172.21.15.16:0/2142325943 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f732406c2b0 msgr2=0x7f732406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:10.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.692+0000 7f7349396640 1 --2- 172.21.15.16:0/2142325943 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f732406c2b0 0x7f732406e770 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f732c034000 tx=0x7f732c02e000 comp rx=0 tx=0).stop 2024-09-19T07:55:10.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.692+0000 7f7349396640 1 -- 172.21.15.16:0/2142325943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73440d0370 msgr2=0x7f734413e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.692+0000 7f7349396640 1 --2- 172.21.15.16:0/2142325943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73440d0370 0x7f734413e5d0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f733400e8c0 tx=0x7f733400ed80 comp rx=0 tx=0).stop 2024-09-19T07:55:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.692+0000 7f7349396640 1 -- 172.21.15.16:0/2142325943 shutdown_connections 2024-09-19T07:55:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.692+0000 7f7349396640 1 --2- 172.21.15.16:0/2142325943 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f732406c2b0 0x7f732406e770 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.692+0000 7f7349396640 1 --2- 172.21.15.16:0/2142325943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7344141fa0 0x7f73441f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.692+0000 7f7349396640 1 --2- 172.21.15.16:0/2142325943 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7344141680 0x7f73440b7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.692+0000 7f7349396640 1 --2- 172.21.15.16:0/2142325943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73440d0370 0x7f734413e5d0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.692+0000 7f7349396640 1 -- 172.21.15.16:0/2142325943 >> 172.21.15.16:0/2142325943 conn(0x7f73440b3ed0 msgr2=0x7f73440b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.692+0000 7f7349396640 1 -- 172.21.15.16:0/2142325943 shutdown_connections 2024-09-19T07:55:10.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.692+0000 7f7349396640 1 -- 172.21.15.16:0/2142325943 wait complete. 2024-09-19T07:55:10.704 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":503,"stamp":"2024-09-19T07:55:10.630520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":765196,"kb_used_data":14888,"kb_used_omap":12,"kb_used_meta":750131,"kb_avail":754209524,"statfs":{"total":773094113280,"available":772310552576,"internally_reserved":0,"allocated":15245312,"data_stored":7692412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":768134704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001861"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972662+0000","last_change":"2024-09-19T07:55:02.977838+0000","last_active":"2024-09-19T07:55:03.972662+0000","last_peered":"2024-09-19T07:55:03.972662+0000","last_clean":"2024-09-19T07:55:03.972662+0000","last_became_active":"2024-09-19T07:55:02.977562+0000","last_became_peered":"2024-09-19T07:55:02.977562+0000","last_unstale":"2024-09-19T07:55:03.972662+0000","last_undegraded":"2024-09-19T07:55:03.972662+0000","last_fullsized":"2024-09-19T07:55:03.972662+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:18.348990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.970646+0000","last_change":"2024-09-19T07:55:02.988730+0000","last_active":"2024-09-19T07:55:03.970646+0000","last_peered":"2024-09-19T07:55:03.970646+0000","last_clean":"2024-09-19T07:55:03.970646+0000","last_became_active":"2024-09-19T07:55:02.988454+0000","last_became_peered":"2024-09-19T07:55:02.988454+0000","last_unstale":"2024-09-19T07:55:03.970646+0000","last_undegraded":"2024-09-19T07:55:03.970646+0000","last_fullsized":"2024-09-19T07:55:03.970646+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:52.586504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.970772+0000","last_change":"2024-09-19T07:55:02.995513+0000","last_active":"2024-09-19T07:55:03.970772+0000","last_peered":"2024-09-19T07:55:03.970772+0000","last_clean":"2024-09-19T07:55:03.970772+0000","last_became_active":"2024-09-19T07:55:02.995162+0000","last_became_peered":"2024-09-19T07:55:02.995162+0000","last_unstale":"2024-09-19T07:55:03.970772+0000","last_undegraded":"2024-09-19T07:55:03.970772+0000","last_fullsized":"2024-09-19T07:55:03.970772+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:51.978120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972077+0000","last_change":"2024-09-19T07:55:02.991747+0000","last_active":"2024-09-19T07:55:03.972077+0000","last_peered":"2024-09-19T07:55:03.972077+0000","last_clean":"2024-09-19T07:55:03.972077+0000","last_became_active":"2024-09-19T07:55:02.991487+0000","last_became_peered":"2024-09-19T07:55:02.991487+0000","last_unstale":"2024-09-19T07:55:03.972077+0000","last_undegraded":"2024-09-19T07:55:03.972077+0000","last_fullsized":"2024-09-19T07:55:03.972077+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:00.867259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845963+0000","last_change":"2024-09-19T07:55:02.974381+0000","last_active":"2024-09-19T07:55:04.845963+0000","last_peered":"2024-09-19T07:55:04.845963+0000","last_clean":"2024-09-19T07:55:04.845963+0000","last_became_active":"2024-09-19T07:55:02.974105+0000","last_became_peered":"2024-09-19T07:55:02.974105+0000","last_unstale":"2024-09-19T07:55:04.845963+0000","last_undegraded":"2024-09-19T07:55:04.845963+0000","last_fullsized":"2024-09-19T07:55:04.845963+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:07.375870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845667+0000","last_change":"2024-09-19T07:55:02.975703+0000","last_active":"2024-09-19T07:55:04.845667+0000","last_peered":"2024-09-19T07:55:04.845667+0000","last_clean":"2024-09-19T07:55:04.845667+0000","last_became_active":"2024-09-19T07:55:02.975379+0000","last_became_peered":"2024-09-19T07:55:02.975379+0000","last_unstale":"2024-09-19T07:55:04.845667+0000","last_undegraded":"2024-09-19T07:55:04.845667+0000","last_fullsized":"2024-09-19T07:55:04.845667+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:19:35.656331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972184+0000","last_change":"2024-09-19T07:55:02.985662+0000","last_active":"2024-09-19T07:55:03.972184+0000","last_peered":"2024-09-19T07:55:03.972184+0000","last_clean":"2024-09-19T07:55:03.972184+0000","last_became_active":"2024-09-19T07:55:02.985378+0000","last_became_peered":"2024-09-19T07:55:02.985378+0000","last_unstale":"2024-09-19T07:55:03.972184+0000","last_undegraded":"2024-09-19T07:55:03.972184+0000","last_fullsized":"2024-09-19T07:55:03.972184+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:33.679405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846129+0000","last_change":"2024-09-19T07:55:02.991159+0000","last_active":"2024-09-19T07:55:04.846129+0000","last_peered":"2024-09-19T07:55:04.846129+0000","last_clean":"2024-09-19T07:55:04.846129+0000","last_became_active":"2024-09-19T07:55:02.990631+0000","last_became_peered":"2024-09-19T07:55:02.990631+0000","last_unstale":"2024-09-19T07:55:04.846129+0000","last_undegraded":"2024-09-19T07:55:04.846129+0000","last_fullsized":"2024-09-19T07:55:04.846129+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:02:28.317679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845888+0000","last_change":"2024-09-19T07:55:02.994532+0000","last_active":"2024-09-19T07:55:04.845888+0000","last_peered":"2024-09-19T07:55:04.845888+0000","last_clean":"2024-09-19T07:55:04.845888+0000","last_became_active":"2024-09-19T07:55:02.994201+0000","last_became_peered":"2024-09-19T07:55:02.994201+0000","last_unstale":"2024-09-19T07:55:04.845888+0000","last_undegraded":"2024-09-19T07:55:04.845888+0000","last_fullsized":"2024-09-19T07:55:04.845888+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:05.551206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846075+0000","last_change":"2024-09-19T07:55:02.981864+0000","last_active":"2024-09-19T07:55:04.846075+0000","last_peered":"2024-09-19T07:55:04.846075+0000","last_clean":"2024-09-19T07:55:04.846075+0000","last_became_active":"2024-09-19T07:55:02.981541+0000","last_became_peered":"2024-09-19T07:55:02.981541+0000","last_unstale":"2024-09-19T07:55:04.846075+0000","last_undegraded":"2024-09-19T07:55:04.846075+0000","last_fullsized":"2024-09-19T07:55:04.846075+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:39:05.129409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845763+0000","last_change":"2024-09-19T07:55:02.991467+0000","last_active":"2024-09-19T07:55:04.845763+0000","last_peered":"2024-09-19T07:55:04.845763+0000","last_clean":"2024-09-19T07:55:04.845763+0000","last_became_active":"2024-09-19T07:55:02.991185+0000","last_became_peered":"2024-09-19T07:55:02.991185+0000","last_unstale":"2024-09-19T07:55:04.845763+0000","last_undegraded":"2024-09-19T07:55:04.845763+0000","last_fullsized":"2024-09-19T07:55:04.845763+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:25:03.509906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972187+0000","last_change":"2024-09-19T07:55:02.989911+0000","last_active":"2024-09-19T07:55:03.972187+0000","last_peered":"2024-09-19T07:55:03.972187+0000","last_clean":"2024-09-19T07:55:03.972187+0000","last_became_active":"2024-09-19T07:55:02.989574+0000","last_became_peered":"2024-09-19T07:55:02.989574+0000","last_unstale":"2024-09-19T07:55:03.972187+0000","last_undegraded":"2024-09-19T07:55:03.972187+0000","last_fullsized":"2024-09-19T07:55:03.972187+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:20.107114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845619+0000","last_change":"2024-09-19T07:55:02.983842+0000","last_active":"2024-09-19T07:55:04.845619+0000","last_peered":"2024-09-19T07:55:04.845619+0000","last_clean":"2024-09-19T07:55:04.845619+0000","last_became_active":"2024-09-19T07:55:02.983583+0000","last_became_peered":"2024-09-19T07:55:02.983583+0000","last_unstale":"2024-09-19T07:55:04.845619+0000","last_undegraded":"2024-09-19T07:55:04.845619+0000","last_fullsized":"2024-09-19T07:55:04.845619+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:36.856459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.971850+0000","last_change":"2024-09-19T07:55:02.991151+0000","last_active":"2024-09-19T07:55:03.971850+0000","last_peered":"2024-09-19T07:55:03.971850+0000","last_clean":"2024-09-19T07:55:03.971850+0000","last_became_active":"2024-09-19T07:55:02.990791+0000","last_became_peered":"2024-09-19T07:55:02.990791+0000","last_unstale":"2024-09-19T07:55:03.971850+0000","last_undegraded":"2024-09-19T07:55:03.971850+0000","last_fullsized":"2024-09-19T07:55:03.971850+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:16.038904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.970352+0000","last_change":"2024-09-19T07:55:02.993711+0000","last_active":"2024-09-19T07:55:03.970352+0000","last_peered":"2024-09-19T07:55:03.970352+0000","last_clean":"2024-09-19T07:55:03.970352+0000","last_became_active":"2024-09-19T07:55:02.993391+0000","last_became_peered":"2024-09-19T07:55:02.993391+0000","last_unstale":"2024-09-19T07:55:03.970352+0000","last_undegraded":"2024-09-19T07:55:03.970352+0000","last_fullsized":"2024-09-19T07:55:03.970352+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:13.843674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845448+0000","last_change":"2024-09-19T07:55:02.977783+0000","last_active":"2024-09-19T07:55:04.845448+0000","last_peered":"2024-09-19T07:55:04.845448+0000","last_clean":"2024-09-19T07:55:04.845448+0000","last_became_active":"2024-09-19T07:55:02.977494+0000","last_became_peered":"2024-09-19T07:55:02.977494+0000","last_unstale":"2024-09-19T07:55:04.845448+0000","last_undegraded":"2024-09-19T07:55:04.845448+0000","last_fullsized":"2024-09-19T07:55:04.845448+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:24.887955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972758+0000","last_change":"2024-09-19T07:55:02.989847+0000","last_active":"2024-09-19T07:55:03.972758+0000","last_peered":"2024-09-19T07:55:03.972758+0000","last_clean":"2024-09-19T07:55:03.972758+0000","last_became_active":"2024-09-19T07:55:02.989566+0000","last_became_peered":"2024-09-19T07:55:02.989566+0000","last_unstale":"2024-09-19T07:55:03.972758+0000","last_undegraded":"2024-09-19T07:55:03.972758+0000","last_fullsized":"2024-09-19T07:55:03.972758+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:57:07.989977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845741+0000","last_change":"2024-09-19T07:55:02.988518+0000","last_active":"2024-09-19T07:55:04.845741+0000","last_peered":"2024-09-19T07:55:04.845741+0000","last_clean":"2024-09-19T07:55:04.845741+0000","last_became_active":"2024-09-19T07:55:02.988215+0000","last_became_peered":"2024-09-19T07:55:02.988215+0000","last_unstale":"2024-09-19T07:55:04.845741+0000","last_undegraded":"2024-09-19T07:55:04.845741+0000","last_fullsized":"2024-09-19T07:55:04.845741+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:33.944874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983370+0000","last_change":"2024-09-19T07:55:02.988233+0000","last_active":"2024-09-19T07:55:03.983370+0000","last_peered":"2024-09-19T07:55:03.983370+0000","last_clean":"2024-09-19T07:55:03.983370+0000","last_became_active":"2024-09-19T07:55:02.987586+0000","last_became_peered":"2024-09-19T07:55:02.987586+0000","last_unstale":"2024-09-19T07:55:03.983370+0000","last_undegraded":"2024-09-19T07:55:03.983370+0000","last_fullsized":"2024-09-19T07:55:03.983370+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:37.091603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.764265+0000","last_change":"2024-09-19T07:55:02.984872+0000","last_active":"2024-09-19T07:55:04.764265+0000","last_peered":"2024-09-19T07:55:04.764265+0000","last_clean":"2024-09-19T07:55:04.764265+0000","last_became_active":"2024-09-19T07:55:02.984588+0000","last_became_peered":"2024-09-19T07:55:02.984588+0000","last_unstale":"2024-09-19T07:55:04.764265+0000","last_undegraded":"2024-09-19T07:55:04.764265+0000","last_fullsized":"2024-09-19T07:55:04.764265+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:08.519276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845791+0000","last_change":"2024-09-19T07:55:02.985444+0000","last_active":"2024-09-19T07:55:04.845791+0000","last_peered":"2024-09-19T07:55:04.845791+0000","last_clean":"2024-09-19T07:55:04.845791+0000","last_became_active":"2024-09-19T07:55:02.985161+0000","last_became_peered":"2024-09-19T07:55:02.985161+0000","last_unstale":"2024-09-19T07:55:04.845791+0000","last_undegraded":"2024-09-19T07:55:04.845791+0000","last_fullsized":"2024-09-19T07:55:04.845791+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:48.740266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972782+0000","last_change":"2024-09-19T07:55:02.980246+0000","last_active":"2024-09-19T07:55:03.972782+0000","last_peered":"2024-09-19T07:55:03.972782+0000","last_clean":"2024-09-19T07:55:03.972782+0000","last_became_active":"2024-09-19T07:55:02.979970+0000","last_became_peered":"2024-09-19T07:55:02.979970+0000","last_unstale":"2024-09-19T07:55:03.972782+0000","last_undegraded":"2024-09-19T07:55:03.972782+0000","last_fullsized":"2024-09-19T07:55:03.972782+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:19.717109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846168+0000","last_change":"2024-09-19T07:55:02.979612+0000","last_active":"2024-09-19T07:55:04.846168+0000","last_peered":"2024-09-19T07:55:04.846168+0000","last_clean":"2024-09-19T07:55:04.846168+0000","last_became_active":"2024-09-19T07:55:02.978902+0000","last_became_peered":"2024-09-19T07:55:02.978902+0000","last_unstale":"2024-09-19T07:55:04.846168+0000","last_undegraded":"2024-09-19T07:55:04.846168+0000","last_fullsized":"2024-09-19T07:55:04.846168+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:00.867259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845596+0000","last_change":"2024-09-19T07:55:02.987092+0000","last_active":"2024-09-19T07:55:04.845596+0000","last_peered":"2024-09-19T07:55:04.845596+0000","last_clean":"2024-09-19T07:55:04.845596+0000","last_became_active":"2024-09-19T07:55:02.986604+0000","last_became_peered":"2024-09-19T07:55:02.986604+0000","last_unstale":"2024-09-19T07:55:04.845596+0000","last_undegraded":"2024-09-19T07:55:04.845596+0000","last_fullsized":"2024-09-19T07:55:04.845596+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:30:29.389217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972885+0000","last_change":"2024-09-19T07:55:02.981645+0000","last_active":"2024-09-19T07:55:03.972885+0000","last_peered":"2024-09-19T07:55:03.972885+0000","last_clean":"2024-09-19T07:55:03.972885+0000","last_became_active":"2024-09-19T07:55:02.981377+0000","last_became_peered":"2024-09-19T07:55:02.981377+0000","last_unstale":"2024-09-19T07:55:03.972885+0000","last_undegraded":"2024-09-19T07:55:03.972885+0000","last_fullsized":"2024-09-19T07:55:03.972885+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:28.867264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972976+0000","last_change":"2024-09-19T07:55:02.991055+0000","last_active":"2024-09-19T07:55:03.972976+0000","last_peered":"2024-09-19T07:55:03.972976+0000","last_clean":"2024-09-19T07:55:03.972976+0000","last_became_active":"2024-09-19T07:55:02.990517+0000","last_became_peered":"2024-09-19T07:55:02.990517+0000","last_unstale":"2024-09-19T07:55:03.972976+0000","last_undegraded":"2024-09-19T07:55:03.972976+0000","last_fullsized":"2024-09-19T07:55:03.972976+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:52.586504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846309+0000","last_change":"2024-09-19T07:55:02.998488+0000","last_active":"2024-09-19T07:55:04.846309+0000","last_peered":"2024-09-19T07:55:04.846309+0000","last_clean":"2024-09-19T07:55:04.846309+0000","last_became_active":"2024-09-19T07:55:02.998192+0000","last_became_peered":"2024-09-19T07:55:02.998192+0000","last_unstale":"2024-09-19T07:55:04.846309+0000","last_undegraded":"2024-09-19T07:55:04.846309+0000","last_fullsized":"2024-09-19T07:55:04.846309+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:52.672239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845428+0000","last_change":"2024-09-19T07:55:02.988808+0000","last_active":"2024-09-19T07:55:04.845428+0000","last_peered":"2024-09-19T07:55:04.845428+0000","last_clean":"2024-09-19T07:55:04.845428+0000","last_became_active":"2024-09-19T07:55:02.988524+0000","last_became_peered":"2024-09-19T07:55:02.988524+0000","last_unstale":"2024-09-19T07:55:04.845428+0000","last_undegraded":"2024-09-19T07:55:04.845428+0000","last_fullsized":"2024-09-19T07:55:04.845428+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:14.948481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846465+0000","last_change":"2024-09-19T07:55:02.985372+0000","last_active":"2024-09-19T07:55:04.846465+0000","last_peered":"2024-09-19T07:55:04.846465+0000","last_clean":"2024-09-19T07:55:04.846465+0000","last_became_active":"2024-09-19T07:55:02.985109+0000","last_became_peered":"2024-09-19T07:55:02.985109+0000","last_unstale":"2024-09-19T07:55:04.846465+0000","last_undegraded":"2024-09-19T07:55:04.846465+0000","last_fullsized":"2024-09-19T07:55:04.846465+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:36.714457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845582+0000","last_change":"2024-09-19T07:55:02.993268+0000","last_active":"2024-09-19T07:55:04.845582+0000","last_peered":"2024-09-19T07:55:04.845582+0000","last_clean":"2024-09-19T07:55:04.845582+0000","last_became_active":"2024-09-19T07:55:02.992933+0000","last_became_peered":"2024-09-19T07:55:02.992933+0000","last_unstale":"2024-09-19T07:55:04.845582+0000","last_undegraded":"2024-09-19T07:55:04.845582+0000","last_fullsized":"2024-09-19T07:55:04.845582+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:34:34.252831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845410+0000","last_change":"2024-09-19T07:55:02.978386+0000","last_active":"2024-09-19T07:55:04.845410+0000","last_peered":"2024-09-19T07:55:04.845410+0000","last_clean":"2024-09-19T07:55:04.845410+0000","last_became_active":"2024-09-19T07:55:02.978134+0000","last_became_peered":"2024-09-19T07:55:02.978134+0000","last_unstale":"2024-09-19T07:55:04.845410+0000","last_undegraded":"2024-09-19T07:55:04.845410+0000","last_fullsized":"2024-09-19T07:55:04.845410+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:39:38.939449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846160+0000","last_change":"2024-09-19T07:55:02.990915+0000","last_active":"2024-09-19T07:55:04.846160+0000","last_peered":"2024-09-19T07:55:04.846160+0000","last_clean":"2024-09-19T07:55:04.846160+0000","last_became_active":"2024-09-19T07:55:02.990273+0000","last_became_peered":"2024-09-19T07:55:02.990273+0000","last_unstale":"2024-09-19T07:55:04.846160+0000","last_undegraded":"2024-09-19T07:55:04.846160+0000","last_fullsized":"2024-09-19T07:55:04.846160+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:54:28.829885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.970864+0000","last_change":"2024-09-19T07:55:02.994240+0000","last_active":"2024-09-19T07:55:03.970864+0000","last_peered":"2024-09-19T07:55:03.970864+0000","last_clean":"2024-09-19T07:55:03.970864+0000","last_became_active":"2024-09-19T07:55:02.993966+0000","last_became_peered":"2024-09-19T07:55:02.993966+0000","last_unstale":"2024-09-19T07:55:03.970864+0000","last_undegraded":"2024-09-19T07:55:03.970864+0000","last_fullsized":"2024-09-19T07:55:03.970864+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:26:18.944316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"139'4","reported_seq":18,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846054+0000","last_change":"2024-09-19T07:55:01.982819+0000","last_active":"2024-09-19T07:55:04.846054+0000","last_peered":"2024-09-19T07:55:04.846054+0000","last_clean":"2024-09-19T07:55:04.846054+0000","last_became_active":"2024-09-19T07:55:01.982455+0000","last_became_peered":"2024-09-19T07:55:01.982455+0000","last_unstale":"2024-09-19T07:55:04.846054+0000","last_undegraded":"2024-09-19T07:55:04.846054+0000","last_fullsized":"2024-09-19T07:55:04.846054+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:01.483855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.970806+0000","last_change":"2024-09-19T07:55:02.988748+0000","last_active":"2024-09-19T07:55:03.970806+0000","last_peered":"2024-09-19T07:55:03.970806+0000","last_clean":"2024-09-19T07:55:03.970806+0000","last_became_active":"2024-09-19T07:55:02.988417+0000","last_became_peered":"2024-09-19T07:55:02.988417+0000","last_unstale":"2024-09-19T07:55:03.970806+0000","last_undegraded":"2024-09-19T07:55:03.970806+0000","last_fullsized":"2024-09-19T07:55:03.970806+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:03.581754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983330+0000","last_change":"2024-09-19T07:55:01.986317+0000","last_active":"2024-09-19T07:55:03.983330+0000","last_peered":"2024-09-19T07:55:03.983330+0000","last_clean":"2024-09-19T07:55:03.983330+0000","last_became_active":"2024-09-19T07:55:01.986050+0000","last_became_peered":"2024-09-19T07:55:01.986050+0000","last_unstale":"2024-09-19T07:55:03.983330+0000","last_undegraded":"2024-09-19T07:55:03.983330+0000","last_fullsized":"2024-09-19T07:55:03.983330+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:20.699942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846613+0000","last_change":"2024-09-19T07:55:02.988084+0000","last_active":"2024-09-19T07:55:04.846613+0000","last_peered":"2024-09-19T07:55:04.846613+0000","last_clean":"2024-09-19T07:55:04.846613+0000","last_became_active":"2024-09-19T07:55:02.987819+0000","last_became_peered":"2024-09-19T07:55:02.987819+0000","last_unstale":"2024-09-19T07:55:04.846613+0000","last_undegraded":"2024-09-19T07:55:04.846613+0000","last_fullsized":"2024-09-19T07:55:04.846613+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:48.646152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983598+0000","last_change":"2024-09-19T07:55:01.983959+0000","last_active":"2024-09-19T07:55:03.983598+0000","last_peered":"2024-09-19T07:55:03.983598+0000","last_clean":"2024-09-19T07:55:03.983598+0000","last_became_active":"2024-09-19T07:55:01.983682+0000","last_became_peered":"2024-09-19T07:55:01.983682+0000","last_unstale":"2024-09-19T07:55:03.983598+0000","last_undegraded":"2024-09-19T07:55:03.983598+0000","last_fullsized":"2024-09-19T07:55:03.983598+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:17.343679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983514+0000","last_change":"2024-09-19T07:55:02.988272+0000","last_active":"2024-09-19T07:55:03.983514+0000","last_peered":"2024-09-19T07:55:03.983514+0000","last_clean":"2024-09-19T07:55:03.983514+0000","last_became_active":"2024-09-19T07:55:02.987972+0000","last_became_peered":"2024-09-19T07:55:02.987972+0000","last_unstale":"2024-09-19T07:55:03.983514+0000","last_undegraded":"2024-09-19T07:55:03.983514+0000","last_fullsized":"2024-09-19T07:55:03.983514+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:55.597883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.970711+0000","last_change":"2024-09-19T07:55:01.975504+0000","last_active":"2024-09-19T07:55:03.970711+0000","last_peered":"2024-09-19T07:55:03.970711+0000","last_clean":"2024-09-19T07:55:03.970711+0000","last_became_active":"2024-09-19T07:55:01.975216+0000","last_became_peered":"2024-09-19T07:55:01.975216+0000","last_unstale":"2024-09-19T07:55:03.970711+0000","last_undegraded":"2024-09-19T07:55:03.970711+0000","last_fullsized":"2024-09-19T07:55:03.970711+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:47.640841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.764307+0000","last_change":"2024-09-19T07:55:02.990235+0000","last_active":"2024-09-19T07:55:04.764307+0000","last_peered":"2024-09-19T07:55:04.764307+0000","last_clean":"2024-09-19T07:55:04.764307+0000","last_became_active":"2024-09-19T07:55:02.989920+0000","last_became_peered":"2024-09-19T07:55:02.989920+0000","last_unstale":"2024-09-19T07:55:04.764307+0000","last_undegraded":"2024-09-19T07:55:04.764307+0000","last_fullsized":"2024-09-19T07:55:04.764307+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:11.846213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846504+0000","last_change":"2024-09-19T07:55:01.984133+0000","last_active":"2024-09-19T07:55:04.846504+0000","last_peered":"2024-09-19T07:55:04.846504+0000","last_clean":"2024-09-19T07:55:04.846504+0000","last_became_active":"2024-09-19T07:55:01.983829+0000","last_became_peered":"2024-09-19T07:55:01.983829+0000","last_unstale":"2024-09-19T07:55:04.846504+0000","last_undegraded":"2024-09-19T07:55:04.846504+0000","last_fullsized":"2024-09-19T07:55:04.846504+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:08.327895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.971878+0000","last_change":"2024-09-19T07:55:02.990053+0000","last_active":"2024-09-19T07:55:03.971878+0000","last_peered":"2024-09-19T07:55:03.971878+0000","last_clean":"2024-09-19T07:55:03.971878+0000","last_became_active":"2024-09-19T07:55:02.987609+0000","last_became_peered":"2024-09-19T07:55:02.987609+0000","last_unstale":"2024-09-19T07:55:03.971878+0000","last_undegraded":"2024-09-19T07:55:03.971878+0000","last_fullsized":"2024-09-19T07:55:03.971878+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:02:28.317679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845900+0000","last_change":"2024-09-19T07:55:01.984540+0000","last_active":"2024-09-19T07:55:04.845900+0000","last_peered":"2024-09-19T07:55:04.845900+0000","last_clean":"2024-09-19T07:55:04.845900+0000","last_became_active":"2024-09-19T07:55:01.984218+0000","last_became_peered":"2024-09-19T07:55:01.984218+0000","last_unstale":"2024-09-19T07:55:04.845900+0000","last_undegraded":"2024-09-19T07:55:04.845900+0000","last_fullsized":"2024-09-19T07:55:04.845900+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:59:17.842847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.970383+0000","last_change":"2024-09-19T07:55:02.983739+0000","last_active":"2024-09-19T07:55:03.970383+0000","last_peered":"2024-09-19T07:55:03.970383+0000","last_clean":"2024-09-19T07:55:03.970383+0000","last_became_active":"2024-09-19T07:55:02.983448+0000","last_became_peered":"2024-09-19T07:55:02.983448+0000","last_unstale":"2024-09-19T07:55:03.970383+0000","last_undegraded":"2024-09-19T07:55:03.970383+0000","last_fullsized":"2024-09-19T07:55:03.970383+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:21.705254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972280+0000","last_change":"2024-09-19T07:55:01.978901+0000","last_active":"2024-09-19T07:55:03.972280+0000","last_peered":"2024-09-19T07:55:03.972280+0000","last_clean":"2024-09-19T07:55:03.972280+0000","last_became_active":"2024-09-19T07:55:01.978399+0000","last_became_peered":"2024-09-19T07:55:01.978399+0000","last_unstale":"2024-09-19T07:55:03.972280+0000","last_undegraded":"2024-09-19T07:55:03.972280+0000","last_fullsized":"2024-09-19T07:55:03.972280+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:11.363759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972118+0000","last_change":"2024-09-19T07:55:02.978549+0000","last_active":"2024-09-19T07:55:03.972118+0000","last_peered":"2024-09-19T07:55:03.972118+0000","last_clean":"2024-09-19T07:55:03.972118+0000","last_became_active":"2024-09-19T07:55:02.978270+0000","last_became_peered":"2024-09-19T07:55:02.978270+0000","last_unstale":"2024-09-19T07:55:03.972118+0000","last_undegraded":"2024-09-19T07:55:03.972118+0000","last_fullsized":"2024-09-19T07:55:03.972118+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:21.705254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845539+0000","last_change":"2024-09-19T07:55:01.983833+0000","last_active":"2024-09-19T07:55:04.845539+0000","last_peered":"2024-09-19T07:55:04.845539+0000","last_clean":"2024-09-19T07:55:04.845539+0000","last_became_active":"2024-09-19T07:55:01.983553+0000","last_became_peered":"2024-09-19T07:55:01.983553+0000","last_unstale":"2024-09-19T07:55:04.845539+0000","last_undegraded":"2024-09-19T07:55:04.845539+0000","last_fullsized":"2024-09-19T07:55:04.845539+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:02.576442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846601+0000","last_change":"2024-09-19T07:55:02.983084+0000","last_active":"2024-09-19T07:55:04.846601+0000","last_peered":"2024-09-19T07:55:04.846601+0000","last_clean":"2024-09-19T07:55:04.846601+0000","last_became_active":"2024-09-19T07:55:02.982793+0000","last_became_peered":"2024-09-19T07:55:02.982793+0000","last_unstale":"2024-09-19T07:55:04.846601+0000","last_undegraded":"2024-09-19T07:55:04.846601+0000","last_fullsized":"2024-09-19T07:55:04.846601+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:24:00.202573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983489+0000","last_change":"2024-09-19T07:55:01.980547+0000","last_active":"2024-09-19T07:55:03.983489+0000","last_peered":"2024-09-19T07:55:03.983489+0000","last_clean":"2024-09-19T07:55:03.983489+0000","last_became_active":"2024-09-19T07:55:01.980253+0000","last_became_peered":"2024-09-19T07:55:01.980253+0000","last_unstale":"2024-09-19T07:55:03.983489+0000","last_undegraded":"2024-09-19T07:55:03.983489+0000","last_fullsized":"2024-09-19T07:55:03.983489+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:32.674094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845937+0000","last_change":"2024-09-19T07:55:02.990524+0000","last_active":"2024-09-19T07:55:04.845937+0000","last_peered":"2024-09-19T07:55:04.845937+0000","last_clean":"2024-09-19T07:55:04.845937+0000","last_became_active":"2024-09-19T07:55:02.990227+0000","last_became_peered":"2024-09-19T07:55:02.990227+0000","last_unstale":"2024-09-19T07:55:04.845937+0000","last_undegraded":"2024-09-19T07:55:04.845937+0000","last_fullsized":"2024-09-19T07:55:04.845937+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:14:38.518293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.973223+0000","last_change":"2024-09-19T07:55:01.980870+0000","last_active":"2024-09-19T07:55:03.973223+0000","last_peered":"2024-09-19T07:55:03.973223+0000","last_clean":"2024-09-19T07:55:03.973223+0000","last_became_active":"2024-09-19T07:55:01.980569+0000","last_became_peered":"2024-09-19T07:55:01.980569+0000","last_unstale":"2024-09-19T07:55:03.973223+0000","last_undegraded":"2024-09-19T07:55:03.973223+0000","last_fullsized":"2024-09-19T07:55:03.973223+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:56:13.952641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845892+0000","last_change":"2024-09-19T07:55:02.987773+0000","last_active":"2024-09-19T07:55:04.845892+0000","last_peered":"2024-09-19T07:55:04.845892+0000","last_clean":"2024-09-19T07:55:04.845892+0000","last_became_active":"2024-09-19T07:55:02.987513+0000","last_became_peered":"2024-09-19T07:55:02.987513+0000","last_unstale":"2024-09-19T07:55:04.845892+0000","last_undegraded":"2024-09-19T07:55:04.845892+0000","last_fullsized":"2024-09-19T07:55:04.845892+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:25.245393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"139'3","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845998+0000","last_change":"2024-09-19T07:55:01.978949+0000","last_active":"2024-09-19T07:55:04.845998+0000","last_peered":"2024-09-19T07:55:04.845998+0000","last_clean":"2024-09-19T07:55:04.845998+0000","last_became_active":"2024-09-19T07:55:01.978508+0000","last_became_peered":"2024-09-19T07:55:01.978508+0000","last_unstale":"2024-09-19T07:55:04.845998+0000","last_undegraded":"2024-09-19T07:55:04.845998+0000","last_fullsized":"2024-09-19T07:55:04.845998+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:33:49.871640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846298+0000","last_change":"2024-09-19T07:55:02.980794+0000","last_active":"2024-09-19T07:55:04.846298+0000","last_peered":"2024-09-19T07:55:04.846298+0000","last_clean":"2024-09-19T07:55:04.846298+0000","last_became_active":"2024-09-19T07:55:02.980530+0000","last_became_peered":"2024-09-19T07:55:02.980530+0000","last_unstale":"2024-09-19T07:55:04.846298+0000","last_undegraded":"2024-09-19T07:55:04.846298+0000","last_fullsized":"2024-09-19T07:55:04.846298+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:20.107114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846063+0000","last_change":"2024-09-19T07:55:01.984318+0000","last_active":"2024-09-19T07:55:04.846063+0000","last_peered":"2024-09-19T07:55:04.846063+0000","last_clean":"2024-09-19T07:55:04.846063+0000","last_became_active":"2024-09-19T07:55:01.983325+0000","last_became_peered":"2024-09-19T07:55:01.983325+0000","last_unstale":"2024-09-19T07:55:04.846063+0000","last_undegraded":"2024-09-19T07:55:04.846063+0000","last_fullsized":"2024-09-19T07:55:04.846063+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:36.086291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.970426+0000","last_change":"2024-09-19T07:55:02.996430+0000","last_active":"2024-09-19T07:55:03.970426+0000","last_peered":"2024-09-19T07:55:03.970426+0000","last_clean":"2024-09-19T07:55:03.970426+0000","last_became_active":"2024-09-19T07:55:02.996148+0000","last_became_peered":"2024-09-19T07:55:02.996148+0000","last_unstale":"2024-09-19T07:55:03.970426+0000","last_undegraded":"2024-09-19T07:55:03.970426+0000","last_fullsized":"2024-09-19T07:55:03.970426+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:33.679405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972021+0000","last_change":"2024-09-19T07:55:01.988068+0000","last_active":"2024-09-19T07:55:03.972021+0000","last_peered":"2024-09-19T07:55:03.972021+0000","last_clean":"2024-09-19T07:55:03.972021+0000","last_became_active":"2024-09-19T07:55:01.987761+0000","last_became_peered":"2024-09-19T07:55:01.987761+0000","last_unstale":"2024-09-19T07:55:03.972021+0000","last_undegraded":"2024-09-19T07:55:03.972021+0000","last_fullsized":"2024-09-19T07:55:03.972021+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:56.300407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845771+0000","last_change":"2024-09-19T07:55:02.989642+0000","last_active":"2024-09-19T07:55:04.845771+0000","last_peered":"2024-09-19T07:55:04.845771+0000","last_clean":"2024-09-19T07:55:04.845771+0000","last_became_active":"2024-09-19T07:55:02.989315+0000","last_became_peered":"2024-09-19T07:55:02.989315+0000","last_unstale":"2024-09-19T07:55:04.845771+0000","last_undegraded":"2024-09-19T07:55:04.845771+0000","last_fullsized":"2024-09-19T07:55:04.845771+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:45.805819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972162+0000","last_change":"2024-09-19T07:55:01.983411+0000","last_active":"2024-09-19T07:55:03.972162+0000","last_peered":"2024-09-19T07:55:03.972162+0000","last_clean":"2024-09-19T07:55:03.972162+0000","last_became_active":"2024-09-19T07:55:01.982413+0000","last_became_peered":"2024-09-19T07:55:01.982413+0000","last_unstale":"2024-09-19T07:55:03.972162+0000","last_undegraded":"2024-09-19T07:55:03.972162+0000","last_fullsized":"2024-09-19T07:55:03.972162+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:58.316347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.970450+0000","last_change":"2024-09-19T07:55:02.983361+0000","last_active":"2024-09-19T07:55:03.970450+0000","last_peered":"2024-09-19T07:55:03.970450+0000","last_clean":"2024-09-19T07:55:03.970450+0000","last_became_active":"2024-09-19T07:55:02.983032+0000","last_became_peered":"2024-09-19T07:55:02.983032+0000","last_unstale":"2024-09-19T07:55:03.970450+0000","last_undegraded":"2024-09-19T07:55:03.970450+0000","last_fullsized":"2024-09-19T07:55:03.970450+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:18.348990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"139'8","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.971849+0000","last_change":"2024-09-19T07:55:01.978831+0000","last_active":"2024-09-19T07:55:03.971849+0000","last_peered":"2024-09-19T07:55:03.971849+0000","last_clean":"2024-09-19T07:55:03.971849+0000","last_became_active":"2024-09-19T07:55:01.978370+0000","last_became_peered":"2024-09-19T07:55:01.978370+0000","last_unstale":"2024-09-19T07:55:03.971849+0000","last_undegraded":"2024-09-19T07:55:03.971849+0000","last_fullsized":"2024-09-19T07:55:03.971849+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:47.640841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983367+0000","last_change":"2024-09-19T07:55:02.998722+0000","last_active":"2024-09-19T07:55:03.983367+0000","last_peered":"2024-09-19T07:55:03.983367+0000","last_clean":"2024-09-19T07:55:03.983367+0000","last_became_active":"2024-09-19T07:55:02.998420+0000","last_became_peered":"2024-09-19T07:55:02.998420+0000","last_unstale":"2024-09-19T07:55:03.983367+0000","last_undegraded":"2024-09-19T07:55:03.983367+0000","last_fullsized":"2024-09-19T07:55:03.983367+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:02.489166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845663+0000","last_change":"2024-09-19T07:55:01.983892+0000","last_active":"2024-09-19T07:55:04.845663+0000","last_peered":"2024-09-19T07:55:04.845663+0000","last_clean":"2024-09-19T07:55:04.845663+0000","last_became_active":"2024-09-19T07:55:01.983621+0000","last_became_peered":"2024-09-19T07:55:01.983621+0000","last_unstale":"2024-09-19T07:55:04.845663+0000","last_undegraded":"2024-09-19T07:55:04.845663+0000","last_fullsized":"2024-09-19T07:55:04.845663+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:35:59.714032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983513+0000","last_change":"2024-09-19T07:55:02.988579+0000","last_active":"2024-09-19T07:55:03.983513+0000","last_peered":"2024-09-19T07:55:03.983513+0000","last_clean":"2024-09-19T07:55:03.983513+0000","last_became_active":"2024-09-19T07:55:02.987958+0000","last_became_peered":"2024-09-19T07:55:02.987958+0000","last_unstale":"2024-09-19T07:55:03.983513+0000","last_undegraded":"2024-09-19T07:55:03.983513+0000","last_fullsized":"2024-09-19T07:55:03.983513+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:08.519276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983438+0000","last_change":"2024-09-19T07:55:01.981723+0000","last_active":"2024-09-19T07:55:03.983438+0000","last_peered":"2024-09-19T07:55:03.983438+0000","last_clean":"2024-09-19T07:55:03.983438+0000","last_became_active":"2024-09-19T07:55:01.981465+0000","last_became_peered":"2024-09-19T07:55:01.981465+0000","last_unstale":"2024-09-19T07:55:03.983438+0000","last_undegraded":"2024-09-19T07:55:03.983438+0000","last_fullsized":"2024-09-19T07:55:03.983438+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:12.838363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.973123+0000","last_change":"2024-09-19T07:55:02.979256+0000","last_active":"2024-09-19T07:55:03.973123+0000","last_peered":"2024-09-19T07:55:03.973123+0000","last_clean":"2024-09-19T07:55:03.973123+0000","last_became_active":"2024-09-19T07:55:02.978921+0000","last_became_peered":"2024-09-19T07:55:02.978921+0000","last_unstale":"2024-09-19T07:55:03.973123+0000","last_undegraded":"2024-09-19T07:55:03.973123+0000","last_fullsized":"2024-09-19T07:55:03.973123+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:51.978120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.970858+0000","last_change":"2024-09-19T07:55:01.982720+0000","last_active":"2024-09-19T07:55:03.970858+0000","last_peered":"2024-09-19T07:55:03.970858+0000","last_clean":"2024-09-19T07:55:03.970858+0000","last_became_active":"2024-09-19T07:55:01.982277+0000","last_became_peered":"2024-09-19T07:55:01.982277+0000","last_unstale":"2024-09-19T07:55:03.970858+0000","last_undegraded":"2024-09-19T07:55:03.970858+0000","last_fullsized":"2024-09-19T07:55:03.970858+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:59.197262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972931+0000","last_change":"2024-09-19T07:55:02.989059+0000","last_active":"2024-09-19T07:55:03.972931+0000","last_peered":"2024-09-19T07:55:03.972931+0000","last_clean":"2024-09-19T07:55:03.972931+0000","last_became_active":"2024-09-19T07:55:02.988577+0000","last_became_peered":"2024-09-19T07:55:02.988577+0000","last_unstale":"2024-09-19T07:55:03.972931+0000","last_undegraded":"2024-09-19T07:55:03.972931+0000","last_fullsized":"2024-09-19T07:55:03.972931+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:49.502676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972169+0000","last_change":"2024-09-19T07:55:01.988779+0000","last_active":"2024-09-19T07:55:03.972169+0000","last_peered":"2024-09-19T07:55:03.972169+0000","last_clean":"2024-09-19T07:55:03.972169+0000","last_became_active":"2024-09-19T07:55:01.988419+0000","last_became_peered":"2024-09-19T07:55:01.988419+0000","last_unstale":"2024-09-19T07:55:03.972169+0000","last_undegraded":"2024-09-19T07:55:03.972169+0000","last_fullsized":"2024-09-19T07:55:03.972169+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:59.785587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.982978+0000","last_change":"2024-09-19T07:55:02.987148+0000","last_active":"2024-09-19T07:55:03.982978+0000","last_peered":"2024-09-19T07:55:03.982978+0000","last_clean":"2024-09-19T07:55:03.982978+0000","last_became_active":"2024-09-19T07:55:02.986889+0000","last_became_peered":"2024-09-19T07:55:02.986889+0000","last_unstale":"2024-09-19T07:55:03.982978+0000","last_undegraded":"2024-09-19T07:55:03.982978+0000","last_fullsized":"2024-09-19T07:55:03.982978+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:26:18.944316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.764243+0000","last_change":"2024-09-19T07:55:01.986115+0000","last_active":"2024-09-19T07:55:04.764243+0000","last_peered":"2024-09-19T07:55:04.764243+0000","last_clean":"2024-09-19T07:55:04.764243+0000","last_became_active":"2024-09-19T07:55:01.985827+0000","last_became_peered":"2024-09-19T07:55:01.985827+0000","last_unstale":"2024-09-19T07:55:04.764243+0000","last_undegraded":"2024-09-19T07:55:04.764243+0000","last_fullsized":"2024-09-19T07:55:04.764243+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:01.483855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.970593+0000","last_change":"2024-09-19T07:55:02.987365+0000","last_active":"2024-09-19T07:55:03.970593+0000","last_peered":"2024-09-19T07:55:03.970593+0000","last_clean":"2024-09-19T07:55:03.970593+0000","last_became_active":"2024-09-19T07:55:02.986480+0000","last_became_peered":"2024-09-19T07:55:02.986480+0000","last_unstale":"2024-09-19T07:55:03.970593+0000","last_undegraded":"2024-09-19T07:55:03.970593+0000","last_fullsized":"2024-09-19T07:55:03.970593+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:49.502676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846212+0000","last_change":"2024-09-19T07:55:01.981717+0000","last_active":"2024-09-19T07:55:04.846212+0000","last_peered":"2024-09-19T07:55:04.846212+0000","last_clean":"2024-09-19T07:55:04.846212+0000","last_became_active":"2024-09-19T07:55:01.981004+0000","last_became_peered":"2024-09-19T07:55:01.981004+0000","last_unstale":"2024-09-19T07:55:04.846212+0000","last_undegraded":"2024-09-19T07:55:04.846212+0000","last_fullsized":"2024-09-19T07:55:04.846212+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:58.316347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983112+0000","last_change":"2024-09-19T07:55:02.992093+0000","last_active":"2024-09-19T07:55:03.983112+0000","last_peered":"2024-09-19T07:55:03.983112+0000","last_clean":"2024-09-19T07:55:03.983112+0000","last_became_active":"2024-09-19T07:55:02.991829+0000","last_became_peered":"2024-09-19T07:55:02.991829+0000","last_unstale":"2024-09-19T07:55:03.983112+0000","last_undegraded":"2024-09-19T07:55:03.983112+0000","last_fullsized":"2024-09-19T07:55:03.983112+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:59:18.848159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.764051+0000","last_change":"2024-09-19T07:55:01.983244+0000","last_active":"2024-09-19T07:55:04.764051+0000","last_peered":"2024-09-19T07:55:04.764051+0000","last_clean":"2024-09-19T07:55:04.764051+0000","last_became_active":"2024-09-19T07:55:01.982960+0000","last_became_peered":"2024-09-19T07:55:01.982960+0000","last_unstale":"2024-09-19T07:55:04.764051+0000","last_undegraded":"2024-09-19T07:55:04.764051+0000","last_fullsized":"2024-09-19T07:55:04.764051+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:26:17.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.970559+0000","last_change":"2024-09-19T07:55:02.993715+0000","last_active":"2024-09-19T07:55:03.970559+0000","last_peered":"2024-09-19T07:55:03.970559+0000","last_clean":"2024-09-19T07:55:03.970559+0000","last_became_active":"2024-09-19T07:55:02.993409+0000","last_became_peered":"2024-09-19T07:55:02.993409+0000","last_unstale":"2024-09-19T07:55:03.970559+0000","last_undegraded":"2024-09-19T07:55:03.970559+0000","last_fullsized":"2024-09-19T07:55:03.970559+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:28.867264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846412+0000","last_change":"2024-09-19T07:55:01.982335+0000","last_active":"2024-09-19T07:55:04.846412+0000","last_peered":"2024-09-19T07:55:04.846412+0000","last_clean":"2024-09-19T07:55:04.846412+0000","last_became_active":"2024-09-19T07:55:01.982052+0000","last_became_peered":"2024-09-19T07:55:01.982052+0000","last_unstale":"2024-09-19T07:55:04.846412+0000","last_undegraded":"2024-09-19T07:55:04.846412+0000","last_fullsized":"2024-09-19T07:55:04.846412+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:59.785587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846217+0000","last_change":"2024-09-19T07:55:02.974014+0000","last_active":"2024-09-19T07:55:04.846217+0000","last_peered":"2024-09-19T07:55:04.846217+0000","last_clean":"2024-09-19T07:55:04.846217+0000","last_became_active":"2024-09-19T07:55:02.973536+0000","last_became_peered":"2024-09-19T07:55:02.973536+0000","last_unstale":"2024-09-19T07:55:04.846217+0000","last_undegraded":"2024-09-19T07:55:04.846217+0000","last_fullsized":"2024-09-19T07:55:04.846217+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:34.283267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972720+0000","last_change":"2024-09-19T07:55:01.988008+0000","last_active":"2024-09-19T07:55:03.972720+0000","last_peered":"2024-09-19T07:55:03.972720+0000","last_clean":"2024-09-19T07:55:03.972720+0000","last_became_active":"2024-09-19T07:55:01.987725+0000","last_became_peered":"2024-09-19T07:55:01.987725+0000","last_unstale":"2024-09-19T07:55:03.972720+0000","last_undegraded":"2024-09-19T07:55:03.972720+0000","last_fullsized":"2024-09-19T07:55:03.972720+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:13.833572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972516+0000","last_change":"2024-09-19T07:55:02.991309+0000","last_active":"2024-09-19T07:55:03.972516+0000","last_peered":"2024-09-19T07:55:03.972516+0000","last_clean":"2024-09-19T07:55:03.972516+0000","last_became_active":"2024-09-19T07:55:02.991045+0000","last_became_peered":"2024-09-19T07:55:02.991045+0000","last_unstale":"2024-09-19T07:55:03.972516+0000","last_undegraded":"2024-09-19T07:55:03.972516+0000","last_fullsized":"2024-09-19T07:55:03.972516+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:38.709571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"140'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.982444+0000","last_change":"2024-09-19T07:55:01.978226+0000","last_active":"2024-09-19T07:55:03.982444+0000","last_peered":"2024-09-19T07:55:03.982444+0000","last_clean":"2024-09-19T07:55:03.982444+0000","last_became_active":"2024-09-19T07:55:01.977803+0000","last_became_peered":"2024-09-19T07:55:01.977803+0000","last_unstale":"2024-09-19T07:55:03.982444+0000","last_undegraded":"2024-09-19T07:55:03.982444+0000","last_fullsized":"2024-09-19T07:55:03.982444+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:49.852983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.973122+0000","last_change":"2024-09-19T07:55:02.977147+0000","last_active":"2024-09-19T07:55:03.973122+0000","last_peered":"2024-09-19T07:55:03.973122+0000","last_clean":"2024-09-19T07:55:03.973122+0000","last_became_active":"2024-09-19T07:55:02.976832+0000","last_became_peered":"2024-09-19T07:55:02.976832+0000","last_unstale":"2024-09-19T07:55:03.973122+0000","last_undegraded":"2024-09-19T07:55:03.973122+0000","last_fullsized":"2024-09-19T07:55:03.973122+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:03.581754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.970892+0000","last_change":"2024-09-19T07:55:01.974880+0000","last_active":"2024-09-19T07:55:03.970892+0000","last_peered":"2024-09-19T07:55:03.970892+0000","last_clean":"2024-09-19T07:55:03.970892+0000","last_became_active":"2024-09-19T07:55:01.974599+0000","last_became_peered":"2024-09-19T07:55:01.974599+0000","last_unstale":"2024-09-19T07:55:03.970892+0000","last_undegraded":"2024-09-19T07:55:03.970892+0000","last_fullsized":"2024-09-19T07:55:03.970892+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:11.363759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846089+0000","last_change":"2024-09-19T07:55:02.989699+0000","last_active":"2024-09-19T07:55:04.846089+0000","last_peered":"2024-09-19T07:55:04.846089+0000","last_clean":"2024-09-19T07:55:04.846089+0000","last_became_active":"2024-09-19T07:55:02.989398+0000","last_became_peered":"2024-09-19T07:55:02.989398+0000","last_unstale":"2024-09-19T07:55:04.846089+0000","last_undegraded":"2024-09-19T07:55:04.846089+0000","last_fullsized":"2024-09-19T07:55:04.846089+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:56:23.308468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846196+0000","last_change":"2024-09-19T07:55:01.981690+0000","last_active":"2024-09-19T07:55:04.846196+0000","last_peered":"2024-09-19T07:55:04.846196+0000","last_clean":"2024-09-19T07:55:04.846196+0000","last_became_active":"2024-09-19T07:55:01.981417+0000","last_became_peered":"2024-09-19T07:55:01.981417+0000","last_unstale":"2024-09-19T07:55:04.846196+0000","last_undegraded":"2024-09-19T07:55:04.846196+0000","last_fullsized":"2024-09-19T07:55:04.846196+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:26:17.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846365+0000","last_change":"2024-09-19T07:55:02.986076+0000","last_active":"2024-09-19T07:55:04.846365+0000","last_peered":"2024-09-19T07:55:04.846365+0000","last_clean":"2024-09-19T07:55:04.846365+0000","last_became_active":"2024-09-19T07:55:02.984885+0000","last_became_peered":"2024-09-19T07:55:02.984885+0000","last_unstale":"2024-09-19T07:55:04.846365+0000","last_undegraded":"2024-09-19T07:55:04.846365+0000","last_fullsized":"2024-09-19T07:55:04.846365+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:57:07.989977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972219+0000","last_change":"2024-09-19T07:55:01.986920+0000","last_active":"2024-09-19T07:55:03.972219+0000","last_peered":"2024-09-19T07:55:03.972219+0000","last_clean":"2024-09-19T07:55:03.972219+0000","last_became_active":"2024-09-19T07:55:01.986618+0000","last_became_peered":"2024-09-19T07:55:01.986618+0000","last_unstale":"2024-09-19T07:55:03.972219+0000","last_undegraded":"2024-09-19T07:55:03.972219+0000","last_fullsized":"2024-09-19T07:55:03.972219+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:59.197262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972053+0000","last_change":"2024-09-19T07:55:02.989902+0000","last_active":"2024-09-19T07:55:03.972053+0000","last_peered":"2024-09-19T07:55:03.972053+0000","last_clean":"2024-09-19T07:55:03.972053+0000","last_became_active":"2024-09-19T07:55:02.987293+0000","last_became_peered":"2024-09-19T07:55:02.987293+0000","last_unstale":"2024-09-19T07:55:03.972053+0000","last_undegraded":"2024-09-19T07:55:03.972053+0000","last_fullsized":"2024-09-19T07:55:03.972053+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:54:28.829885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.845473+0000","last_change":"2024-09-19T07:55:01.977073+0000","last_active":"2024-09-19T07:55:04.845473+0000","last_peered":"2024-09-19T07:55:04.845473+0000","last_clean":"2024-09-19T07:55:04.845473+0000","last_became_active":"2024-09-19T07:55:01.976782+0000","last_became_peered":"2024-09-19T07:55:01.976782+0000","last_unstale":"2024-09-19T07:55:04.845473+0000","last_undegraded":"2024-09-19T07:55:04.845473+0000","last_fullsized":"2024-09-19T07:55:04.845473+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:38.579539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.970487+0000","last_change":"2024-09-19T07:55:02.983221+0000","last_active":"2024-09-19T07:55:03.970487+0000","last_peered":"2024-09-19T07:55:03.970487+0000","last_clean":"2024-09-19T07:55:03.970487+0000","last_became_active":"2024-09-19T07:55:02.982829+0000","last_became_peered":"2024-09-19T07:55:02.982829+0000","last_unstale":"2024-09-19T07:55:03.970487+0000","last_undegraded":"2024-09-19T07:55:03.970487+0000","last_fullsized":"2024-09-19T07:55:03.970487+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:19.717109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.972383+0000","last_change":"2024-09-19T07:55:01.980107+0000","last_active":"2024-09-19T07:55:03.972383+0000","last_peered":"2024-09-19T07:55:03.972383+0000","last_clean":"2024-09-19T07:55:03.972383+0000","last_became_active":"2024-09-19T07:55:01.979828+0000","last_became_peered":"2024-09-19T07:55:01.979828+0000","last_unstale":"2024-09-19T07:55:03.972383+0000","last_undegraded":"2024-09-19T07:55:03.972383+0000","last_fullsized":"2024-09-19T07:55:03.972383+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:17.205219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846347+0000","last_change":"2024-09-19T07:55:02.986047+0000","last_active":"2024-09-19T07:55:04.846347+0000","last_peered":"2024-09-19T07:55:04.846347+0000","last_clean":"2024-09-19T07:55:04.846347+0000","last_became_active":"2024-09-19T07:55:02.985768+0000","last_became_peered":"2024-09-19T07:55:02.985768+0000","last_unstale":"2024-09-19T07:55:04.846347+0000","last_undegraded":"2024-09-19T07:55:04.846347+0000","last_fullsized":"2024-09-19T07:55:04.846347+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:38.709571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.982949+0000","last_change":"2024-09-19T07:55:01.980963+0000","last_active":"2024-09-19T07:55:03.982949+0000","last_peered":"2024-09-19T07:55:03.982949+0000","last_clean":"2024-09-19T07:55:03.982949+0000","last_became_active":"2024-09-19T07:55:01.980672+0000","last_became_peered":"2024-09-19T07:55:01.980672+0000","last_unstale":"2024-09-19T07:55:03.982949+0000","last_undegraded":"2024-09-19T07:55:03.982949+0000","last_fullsized":"2024-09-19T07:55:03.982949+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:11.722912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.846732+0000","last_change":"2024-09-19T07:55:02.985023+0000","last_active":"2024-09-19T07:55:04.846732+0000","last_peered":"2024-09-19T07:55:04.846732+0000","last_clean":"2024-09-19T07:55:04.846732+0000","last_became_active":"2024-09-19T07:55:02.984699+0000","last_became_peered":"2024-09-19T07:55:02.984699+0000","last_unstale":"2024-09-19T07:55:04.846732+0000","last_undegraded":"2024-09-19T07:55:04.846732+0000","last_fullsized":"2024-09-19T07:55:04.846732+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:01.959847+0000","last_clean_scrub_stamp":"2024-09-19T07:55:01.959847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:12.369071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"139'6","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:04.764369+0000","last_change":"2024-09-19T07:55:01.982519+0000","last_active":"2024-09-19T07:55:04.764369+0000","last_peered":"2024-09-19T07:55:04.764369+0000","last_clean":"2024-09-19T07:55:04.764369+0000","last_became_active":"2024-09-19T07:55:01.982223+0000","last_became_peered":"2024-09-19T07:55:01.982223+0000","last_unstale":"2024-09-19T07:55:04.764369+0000","last_undegraded":"2024-09-19T07:55:04.764369+0000","last_fullsized":"2024-09-19T07:55:04.764369+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:00.954535+0000","last_clean_scrub_stamp":"2024-09-19T07:55:00.954535+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:59:17.842847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":387,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-19T07:55:03.983009+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:55:03.983009+0000","last_peered":"2024-09-19T07:55:03.983009+0000","last_clean":"2024-09-19T07:55:03.983009+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:55:03.983009+0000","last_undegraded":"2024-09-19T07:55:03.983009+0000","last_fullsized":"2024-09-19T07:55:03.983009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509590,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83400,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288440,"statfs":{"total":96636764160,"available":96551362560,"internally_reserved":0,"allocated":2347008,"data_stored":1403747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509589,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105096,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266744,"statfs":{"total":96636764160,"available":96529145856,"internally_reserved":0,"allocated":1753088,"data_stored":813427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509590,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73088,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298752,"statfs":{"total":96636764160,"available":96561922048,"internally_reserved":0,"allocated":1748992,"data_stored":813357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509590,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86988,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284852,"statfs":{"total":96636764160,"available":96547688448,"internally_reserved":0,"allocated":1761280,"data_stored":814969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509589,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105628,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266212,"statfs":{"total":96636764160,"available":96528601088,"internally_reserved":0,"allocated":2367488,"data_stored":1406043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542293,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118920,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252920,"statfs":{"total":96636764160,"available":96514990080,"internally_reserved":0,"allocated":1757184,"data_stored":813973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542293,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82884,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288956,"statfs":{"total":96636764160,"available":96551890944,"internally_reserved":0,"allocated":1753088,"data_stored":813427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574998,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109192,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262648,"statfs":{"total":96636764160,"available":96524951552,"internally_reserved":0,"allocated":1757184,"data_stored":813469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:55:10.708 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T07:55:10.709 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T07:55:10.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.853+0000 7fda6242e640 1 Processor -- start 2024-09-19T07:55:10.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.853+0000 7fda6242e640 1 -- start start 2024-09-19T07:55:10.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.854+0000 7fda6242e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda5c15cae0 0x7fda5c0d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:10.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.854+0000 7fda6242e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda5c15d0b0 con 0x7fda5c0d2550 2024-09-19T07:55:10.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.854+0000 7fda6242e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda5c15d390 con 0x7fda5c15c680 2024-09-19T07:55:10.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.854+0000 7fda6242e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fda5c15d670 con 0x7fda5c15cae0 2024-09-19T07:55:10.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.854+0000 7fda5bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fda5c0d2550 0x7fda5c0d2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34034/0 (socket says 172.21.15.16:34034) 2024-09-19T07:55:10.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.854+0000 7fda5bfff640 1 -- 172.21.15.16:0/3960182169 learned_addr learned my addr 172.21.15.16:0/3960182169 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:10.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.854+0000 7fda609cb640 1 --2- 172.21.15.16:0/3960182169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda5c15cae0 0x7fda5c0d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:10.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.854+0000 7fda5affd640 1 -- 172.21.15.16:0/3960182169 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3610338553 0 0) 0x7fda5c15d0b0 con 0x7fda5c0d2550 2024-09-19T07:55:10.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.854+0000 7fda5affd640 1 -- 172.21.15.16:0/3960182169 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda44003520 con 0x7fda5c0d2550 2024-09-19T07:55:10.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.855+0000 7fda5affd640 1 -- 172.21.15.16:0/3960182169 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1503316832 0 0) 0x7fda44003520 con 0x7fda5c0d2550 2024-09-19T07:55:10.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.855+0000 7fda609cb640 1 -- 172.21.15.16:0/3960182169 >> v1:172.21.15.146:6789/0 conn(0x7fda5c15c680 legacy=0x7fda5c15d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:10.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.855+0000 7fda609cb640 1 -- 172.21.15.16:0/3960182169 >> v1:172.21.15.16:6789/0 conn(0x7fda5c0d2550 legacy=0x7fda5c0d2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:10.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.855+0000 7fda609cb640 1 -- 172.21.15.16:0/3960182169 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5c0cff20 con 0x7fda5c15cae0 2024-09-19T07:55:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.855+0000 7fda609cb640 1 --2- 172.21.15.16:0/3960182169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda5c15cae0 0x7fda5c0d0720 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fda54009dc0 tx=0x7fda5402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66908203184ecf51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.856+0000 7fda5affd640 1 -- 172.21.15.16:0/3960182169 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c001670 con 0x7fda5c15cae0 2024-09-19T07:55:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.856+0000 7fda5affd640 1 -- 172.21.15.16:0/3960182169 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda54007540 con 0x7fda5c15cae0 2024-09-19T07:55:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.856+0000 7fda5affd640 1 -- 172.21.15.16:0/3960182169 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda540078f0 con 0x7fda5c15cae0 2024-09-19T07:55:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.856+0000 7fda6242e640 1 -- 172.21.15.16:0/3960182169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda5c15cae0 msgr2=0x7fda5c0d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.856+0000 7fda6242e640 1 --2- 172.21.15.16:0/3960182169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda5c15cae0 0x7fda5c0d0720 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fda54009dc0 tx=0x7fda5402a5d0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.856+0000 7fda6242e640 1 -- 172.21.15.16:0/3960182169 shutdown_connections 2024-09-19T07:55:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.856+0000 7fda6242e640 1 --2- 172.21.15.16:0/3960182169 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda5c15cae0 0x7fda5c0d0720 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.856+0000 7fda6242e640 1 -- 172.21.15.16:0/3960182169 >> 172.21.15.16:0/3960182169 conn(0x7fda5c0701d0 msgr2=0x7fda5c0705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.856+0000 7fda6242e640 1 -- 172.21.15.16:0/3960182169 shutdown_connections 2024-09-19T07:55:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.856+0000 7fda6242e640 1 -- 172.21.15.16:0/3960182169 wait complete. 2024-09-19T07:55:10.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.857+0000 7fda6242e640 1 Processor -- start 2024-09-19T07:55:10.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.858+0000 7fda6242e640 1 -- start start 2024-09-19T07:55:10.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.858+0000 7fda6242e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda5c0d2550 0x7fda5c0b7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:10.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.858+0000 7fda6242e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda5c15c680 0x7fda5c1418c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:10.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.858+0000 7fda6242e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda5c15cae0 0x7fda5c0b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.858+0000 7fda6242e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c15df50 con 0x7fda5c0d2550 2024-09-19T07:55:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.858+0000 7fda6242e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c1569d0 con 0x7fda5c15c680 2024-09-19T07:55:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.858+0000 7fda6242e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fda5c1514e0 con 0x7fda5c15cae0 2024-09-19T07:55:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.858+0000 7fda5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda5c0d2550 0x7fda5c0b7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.858+0000 7fda5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda5c0d2550 0x7fda5c0b7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43866/0 (socket says 172.21.15.16:43866) 2024-09-19T07:55:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.858+0000 7fda5bfff640 1 -- 172.21.15.16:0/910942579 learned_addr learned my addr 172.21.15.16:0/910942579 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.859+0000 7fda609cb640 1 --2- 172.21.15.16:0/910942579 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda5c15cae0 0x7fda5c0b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.859+0000 7fda5b7fe640 1 --2- 172.21.15.16:0/910942579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda5c15c680 0x7fda5c1418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.859+0000 7fda5bfff640 1 -- 172.21.15.16:0/910942579 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda5c15cae0 msgr2=0x7fda5c0b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.859+0000 7fda5bfff640 1 --2- 172.21.15.16:0/910942579 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda5c15cae0 0x7fda5c0b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.859+0000 7fda5bfff640 1 -- 172.21.15.16:0/910942579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda5c15c680 msgr2=0x7fda5c1418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.859+0000 7fda5bfff640 1 --2- 172.21.15.16:0/910942579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda5c15c680 0x7fda5c1418c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.859+0000 7fda5bfff640 1 -- 172.21.15.16:0/910942579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5c0b3860 con 0x7fda5c0d2550 2024-09-19T07:55:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.859+0000 7fda5b7fe640 1 --2- 172.21.15.16:0/910942579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda5c15c680 0x7fda5c1418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.859+0000 7fda5bfff640 1 --2- 172.21.15.16:0/910942579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda5c0d2550 0x7fda5c0b7c80 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00c750 tx=0x7fda4c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.859+0000 7fda58ff9640 1 -- 172.21.15.16:0/910942579 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c017070 con 0x7fda5c0d2550 2024-09-19T07:55:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.859+0000 7fda58ff9640 1 -- 172.21.15.16:0/910942579 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4c005ad0 con 0x7fda5c0d2550 2024-09-19T07:55:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.859+0000 7fda6242e640 1 -- 172.21.15.16:0/910942579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda5c151730 con 0x7fda5c0d2550 2024-09-19T07:55:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.859+0000 7fda58ff9640 1 -- 172.21.15.16:0/910942579 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c007070 con 0x7fda5c0d2550 2024-09-19T07:55:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.859+0000 7fda6242e640 1 -- 172.21.15.16:0/910942579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda5c0b3b00 con 0x7fda5c0d2550 2024-09-19T07:55:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.860+0000 7fda6242e640 1 -- 172.21.15.16:0/910942579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda5c0b3d00 con 0x7fda5c0d2550 2024-09-19T07:55:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.861+0000 7fda58ff9640 1 -- 172.21.15.16:0/910942579 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fda4c01a070 con 0x7fda5c0d2550 2024-09-19T07:55:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.861+0000 7fda58ff9640 1 --2- 172.21.15.16:0/910942579 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fda4806c2e0 0x7fda4806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.861+0000 7fda58ff9640 1 -- 172.21.15.16:0/910942579 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7fda4c08aab0 con 0x7fda5c0d2550 2024-09-19T07:55:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.861+0000 7fda5b7fe640 1 --2- 172.21.15.16:0/910942579 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fda4806c2e0 0x7fda4806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.862+0000 7fda5b7fe640 1 --2- 172.21.15.16:0/910942579 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fda4806c2e0 0x7fda4806e7a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fda5c15d890 tx=0x7fda50009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:10.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:10.865+0000 7fda58ff9640 1 -- 172.21.15.16:0/910942579 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fda5c0b3d00 con 0x7fda5c0d2550 2024-09-19T07:55:11.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.113+0000 7fda6242e640 1 -- 172.21.15.16:0/910942579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fda5c0ce810 con 0x7fda5c0d2550 2024-09-19T07:55:11.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.115+0000 7fda58ff9640 1 -- 172.21.15.16:0/910942579 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7fda4c059090 con 0x7fda5c0d2550 2024-09-19T07:55:11.116 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-19T07:55:11.116 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA== 2024-09-19T07:55:11.116 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-19T07:55:11.116 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-19T07:55:11.116 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T07:55:11.116 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-19T07:55:11.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.118+0000 7fda6242e640 1 -- 172.21.15.16:0/910942579 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fda4806c2e0 msgr2=0x7fda4806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:11.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.118+0000 7fda6242e640 1 --2- 172.21.15.16:0/910942579 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fda4806c2e0 0x7fda4806e7a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fda5c15d890 tx=0x7fda50009040 comp rx=0 tx=0).stop 2024-09-19T07:55:11.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.118+0000 7fda6242e640 1 -- 172.21.15.16:0/910942579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda5c0d2550 msgr2=0x7fda5c0b7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:11.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.118+0000 7fda6242e640 1 --2- 172.21.15.16:0/910942579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda5c0d2550 0x7fda5c0b7c80 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00c750 tx=0x7fda4c00cc10 comp rx=0 tx=0).stop 2024-09-19T07:55:11.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.118+0000 7fda6242e640 1 -- 172.21.15.16:0/910942579 shutdown_connections 2024-09-19T07:55:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.118+0000 7fda6242e640 1 --2- 172.21.15.16:0/910942579 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fda4806c2e0 0x7fda4806e7a0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.118+0000 7fda6242e640 1 --2- 172.21.15.16:0/910942579 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda5c15cae0 0x7fda5c0b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.118+0000 7fda6242e640 1 --2- 172.21.15.16:0/910942579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda5c15c680 0x7fda5c1418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.118+0000 7fda6242e640 1 --2- 172.21.15.16:0/910942579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda5c0d2550 0x7fda5c0b7c80 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.118+0000 7fda6242e640 1 -- 172.21.15.16:0/910942579 >> 172.21.15.16:0/910942579 conn(0x7fda5c0701d0 msgr2=0x7fda5c074ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.118+0000 7fda6242e640 1 -- 172.21.15.16:0/910942579 shutdown_connections 2024-09-19T07:55:11.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.118+0000 7fda6242e640 1 -- 172.21.15.16:0/910942579 wait complete. 2024-09-19T07:55:11.129 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:55:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.230+0000 7f86639b9640 1 Processor -- start 2024-09-19T07:55:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.231+0000 7f86639b9640 1 -- start start 2024-09-19T07:55:11.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.231+0000 7f86639b9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f865c15f0d0 0x7f865c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:11.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.231+0000 7f86639b9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f865c0d0070 0x7f865c0d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:11.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.231+0000 7f86639b9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f865c0cf2b0 con 0x7f865c0d1940 2024-09-19T07:55:11.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.231+0000 7f86639b9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f865c0cf590 con 0x7f865c0d0070 2024-09-19T07:55:11.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.231+0000 7f86639b9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f865c0cf870 con 0x7f865c15f0d0 2024-09-19T07:55:11.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.231+0000 7f8660f54640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f865c0d0070 0x7f865c0d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:11.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.231+0000 7f8661f56640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f865c0d1940 0x7f865c0d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34040/0 (socket says 172.21.15.16:34040) 2024-09-19T07:55:11.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.232+0000 7f8661f56640 1 -- 172.21.15.16:0/1072188956 learned_addr learned my addr 172.21.15.16:0/1072188956 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:11.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.232+0000 7f8660f54640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f865c0d0070 0x7f865c0d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43870/0 (socket says 172.21.15.16:43870) 2024-09-19T07:55:11.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.232+0000 7f8661755640 1 --2- 172.21.15.16:0/1072188956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f865c15f0d0 0x7f865c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:11.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.232+0000 7f8660f54640 1 -- 172.21.15.16:0/1072188956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f865c15f0d0 msgr2=0x7f865c15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:11.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.232+0000 7f8660f54640 1 --2- 172.21.15.16:0/1072188956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f865c15f0d0 0x7f865c15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.232+0000 7f8660f54640 1 -- 172.21.15.16:0/1072188956 >> v1:172.21.15.16:6789/0 conn(0x7f865c0d1940 legacy=0x7f865c0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:11.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.232+0000 7f865bfff640 1 -- 172.21.15.16:0/1072188956 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1799958489 0 0) 0x7f865c0cf2b0 con 0x7f865c0d1940 2024-09-19T07:55:11.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.232+0000 7f8660f54640 1 -- 172.21.15.16:0/1072188956 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c0cfb50 con 0x7f865c0d0070 2024-09-19T07:55:11.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.232+0000 7f8661755640 1 --2- 172.21.15.16:0/1072188956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f865c15f0d0 0x7f865c15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:11.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.233+0000 7f8660f54640 1 --2- 172.21.15.16:0/1072188956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c0d0070 0x7f865c0d1400 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f86500097a0 tx=0x7f865002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6053e7cac7df4b16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:11.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.233+0000 7f865bfff640 1 -- 172.21.15.16:0/1072188956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650037070 con 0x7f865c0d0070 2024-09-19T07:55:11.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.233+0000 7f865bfff640 1 -- 172.21.15.16:0/1072188956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f865002ae50 con 0x7f865c0d0070 2024-09-19T07:55:11.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.233+0000 7f865bfff640 1 -- 172.21.15.16:0/1072188956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650032660 con 0x7f865c0d0070 2024-09-19T07:55:11.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.233+0000 7f86639b9640 1 -- 172.21.15.16:0/1072188956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c0d0070 msgr2=0x7f865c0d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:11.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.233+0000 7f86639b9640 1 --2- 172.21.15.16:0/1072188956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c0d0070 0x7f865c0d1400 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f86500097a0 tx=0x7f865002a100 comp rx=0 tx=0).stop 2024-09-19T07:55:11.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.233+0000 7f86639b9640 1 -- 172.21.15.16:0/1072188956 shutdown_connections 2024-09-19T07:55:11.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.233+0000 7f86639b9640 1 --2- 172.21.15.16:0/1072188956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c0d0070 0x7f865c0d1400 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.233+0000 7f86639b9640 1 --2- 172.21.15.16:0/1072188956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f865c15f0d0 0x7f865c15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.233+0000 7f86639b9640 1 -- 172.21.15.16:0/1072188956 >> 172.21.15.16:0/1072188956 conn(0x7f865c09d760 msgr2=0x7f865c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:11.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.233+0000 7f86639b9640 1 -- 172.21.15.16:0/1072188956 shutdown_connections 2024-09-19T07:55:11.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.233+0000 7f86639b9640 1 -- 172.21.15.16:0/1072188956 wait complete. 2024-09-19T07:55:11.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.235+0000 7f86639b9640 1 Processor -- start 2024-09-19T07:55:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.235+0000 7f86639b9640 1 -- start start 2024-09-19T07:55:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.235+0000 7f86639b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c0d0070 0x7f865c0cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.235+0000 7f86639b9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f865c0d1940 0x7f865c0c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.235+0000 7f86639b9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f865c15f0d0 0x7f865c0cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.235+0000 7f86639b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f865c0d49c0 con 0x7f865c0d0070 2024-09-19T07:55:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.235+0000 7f86639b9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f865c0d4840 con 0x7f865c15f0d0 2024-09-19T07:55:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.235+0000 7f86639b9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f865c0d4b40 con 0x7f865c0d1940 2024-09-19T07:55:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.235+0000 7f8661755640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c0d0070 0x7f865c0cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.235+0000 7f8661755640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c0d0070 0x7f865c0cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43880/0 (socket says 172.21.15.16:43880) 2024-09-19T07:55:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.235+0000 7f8661755640 1 -- 172.21.15.16:0/2616185864 learned_addr learned my addr 172.21.15.16:0/2616185864 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.236+0000 7f8660f54640 1 --2- 172.21.15.16:0/2616185864 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f865c0d1940 0x7f865c0c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.236+0000 7f8661755640 1 -- 172.21.15.16:0/2616185864 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f865c0d1940 msgr2=0x7f865c0c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:11.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.236+0000 7f8661755640 1 --2- 172.21.15.16:0/2616185864 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f865c0d1940 0x7f865c0c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.236+0000 7f8661755640 1 -- 172.21.15.16:0/2616185864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f865c15f0d0 msgr2=0x7f865c0cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:11.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.236+0000 7f8661755640 1 --2- 172.21.15.16:0/2616185864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f865c15f0d0 0x7f865c0cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.236+0000 7f8661755640 1 -- 172.21.15.16:0/2616185864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c0ce4e0 con 0x7f865c0d0070 2024-09-19T07:55:11.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.236+0000 7f8660f54640 1 --2- 172.21.15.16:0/2616185864 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f865c0d1940 0x7f865c0c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:11.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.236+0000 7f8661755640 1 --2- 172.21.15.16:0/2616185864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c0d0070 0x7f865c0cc840 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f865400c7f0 tx=0x7f865400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:11.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.236+0000 7f8659ffb640 1 -- 172.21.15.16:0/2616185864 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654015070 con 0x7f865c0d0070 2024-09-19T07:55:11.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.236+0000 7f8659ffb640 1 -- 172.21.15.16:0/2616185864 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8654007820 con 0x7f865c0d0070 2024-09-19T07:55:11.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.236+0000 7f86639b9640 1 -- 172.21.15.16:0/2616185864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865c0ccf00 con 0x7f865c0d0070 2024-09-19T07:55:11.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.236+0000 7f8659ffb640 1 -- 172.21.15.16:0/2616185864 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654007bd0 con 0x7f865c0d0070 2024-09-19T07:55:11.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.236+0000 7f86639b9640 1 -- 172.21.15.16:0/2616185864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865c0cd4c0 con 0x7f865c0d0070 2024-09-19T07:55:11.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.237+0000 7f86639b9640 1 -- 172.21.15.16:0/2616185864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f865c0d1400 con 0x7f865c0d0070 2024-09-19T07:55:11.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.238+0000 7f8659ffb640 1 -- 172.21.15.16:0/2616185864 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8654018070 con 0x7f865c0d0070 2024-09-19T07:55:11.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.238+0000 7f8659ffb640 1 --2- 172.21.15.16:0/2616185864 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f864806c2b0 0x7f864806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:11.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.238+0000 7f8659ffb640 1 -- 172.21.15.16:0/2616185864 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f865408b6c0 con 0x7f865c0d0070 2024-09-19T07:55:11.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.239+0000 7f8660f54640 1 --2- 172.21.15.16:0/2616185864 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f864806c2b0 0x7f864806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:11.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.239+0000 7f8660f54640 1 --2- 172.21.15.16:0/2616185864 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f864806c2b0 0x7f864806e770 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8650034000 tx=0x7f865002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:11.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.242+0000 7f8659ffb640 1 -- 172.21.15.16:0/2616185864 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8654059ca0 con 0x7f865c0d0070 2024-09-19T07:55:11.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.453+0000 7f86639b9640 1 -- 172.21.15.16:0/2616185864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f865c0d0830 con 0x7f865c0d0070 2024-09-19T07:55:11.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.453+0000 7f8659ffb640 1 -- 172.21.15.16:0/2616185864 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5230 (secure 0 0 0) 0x7f86540595a0 con 0x7f865c0d0070 2024-09-19T07:55:11.454 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:11.454 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"btime":"2024-09-19T07:55:04:980435+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/172459502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":172459502},{"type":"v1","addr":"172.21.15.146:6836","nonce":172459502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3417061308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3417061308},{"type":"v1","addr":"172.21.15.146:6837","nonce":3417061308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1810646409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1810646409},{"type":"v1","addr":"172.21.15.16:6834","nonce":1810646409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:55:02.325256+0000","modified":"2024-09-19T07:55:04.318652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8286":{"gid":8286,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/1929965275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1929965275},{"type":"v1","addr":"172.21.15.16:6835","nonce":1929965275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8286,"qdb_cluster":[8286]},"id":10}]} 2024-09-19T07:55:11.454 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 81 2024-09-19T07:55:11.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.456+0000 7f86639b9640 1 -- 172.21.15.16:0/2616185864 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f864806c2b0 msgr2=0x7f864806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:11.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.456+0000 7f86639b9640 1 --2- 172.21.15.16:0/2616185864 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f864806c2b0 0x7f864806e770 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8650034000 tx=0x7f865002e000 comp rx=0 tx=0).stop 2024-09-19T07:55:11.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.456+0000 7f86639b9640 1 -- 172.21.15.16:0/2616185864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c0d0070 msgr2=0x7f865c0cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:11.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.456+0000 7f86639b9640 1 --2- 172.21.15.16:0/2616185864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c0d0070 0x7f865c0cc840 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f865400c7f0 tx=0x7f865400ccb0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.456+0000 7f86639b9640 1 -- 172.21.15.16:0/2616185864 shutdown_connections 2024-09-19T07:55:11.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.456+0000 7f86639b9640 1 --2- 172.21.15.16:0/2616185864 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f864806c2b0 0x7f864806e770 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.456+0000 7f86639b9640 1 --2- 172.21.15.16:0/2616185864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f865c15f0d0 0x7f865c0cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.456+0000 7f86639b9640 1 --2- 172.21.15.16:0/2616185864 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f865c0d1940 0x7f865c0c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.456+0000 7f86639b9640 1 --2- 172.21.15.16:0/2616185864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c0d0070 0x7f865c0cc840 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.456+0000 7f86639b9640 1 -- 172.21.15.16:0/2616185864 >> 172.21.15.16:0/2616185864 conn(0x7f865c09d760 msgr2=0x7f865c06d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:11.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.456+0000 7f86639b9640 1 -- 172.21.15.16:0/2616185864 shutdown_connections 2024-09-19T07:55:11.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.456+0000 7f86639b9640 1 -- 172.21.15.16:0/2616185864 wait complete. 2024-09-19T07:55:11.870 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T07:55:02.325256+0000', 'modified': '2024-09-19T07:55:04.318652+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8286}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8286': {'gid': 8286, 'name': 'a', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6835/1929965275', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6833', 'nonce': 1929965275}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 1929965275}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8286, 'qdb_cluster': [8286]} 2024-09-19T07:55:11.870 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T07:55:11.871 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T07:55:11.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.979+0000 7f98c32ba640 1 Processor -- start 2024-09-19T07:55:11.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.979+0000 7f98c32ba640 1 -- start start 2024-09-19T07:55:11.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.980+0000 7f98c32ba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98bc06cd20 0x7f98bc06d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:11.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.980+0000 7f98c32ba640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98bc06d640 con 0x7f98bc1425e0 2024-09-19T07:55:11.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.980+0000 7f98c32ba640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98bc162b70 con 0x7f98bc161eb0 2024-09-19T07:55:11.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.980+0000 7f98c32ba640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f98bc162e50 con 0x7f98bc06cd20 2024-09-19T07:55:11.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.980+0000 7f98c1056640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98bc06cd20 0x7f98bc06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:11.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.980+0000 7f98c1056640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98bc06cd20 0x7f98bc06d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43892/0 (socket says 172.21.15.16:43892) 2024-09-19T07:55:11.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.980+0000 7f98c1857640 1 --1- 172.21.15.16:0/1550170196 >> v1:172.21.15.16:6789/0 conn(0x7f98bc1425e0 0x7f98bc142a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34042/0 (socket says 172.21.15.16:34042) 2024-09-19T07:55:11.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.980+0000 7f98c1056640 1 -- 172.21.15.16:0/1550170196 learned_addr learned my addr 172.21.15.16:0/1550170196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:11.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.980+0000 7f98c1056640 1 -- 172.21.15.16:0/1550170196 >> v1:172.21.15.146:6789/0 conn(0x7f98bc161eb0 legacy=0x7f98bc1622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:11.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.980+0000 7f98c1056640 1 -- 172.21.15.16:0/1550170196 >> v1:172.21.15.16:6789/0 conn(0x7f98bc1425e0 legacy=0x7f98bc142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:11.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.980+0000 7f98c1056640 1 -- 172.21.15.16:0/1550170196 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc141b00 con 0x7f98bc06cd20 2024-09-19T07:55:11.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.981+0000 7f98c1056640 1 --2- 172.21.15.16:0/1550170196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc06cd20 0x7f98bc06d100 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009870 tx=0x7f98b802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c41ea84463e0ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:11.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.981+0000 7f98b3fff640 1 -- 172.21.15.16:0/1550170196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8037070 con 0x7f98bc06cd20 2024-09-19T07:55:11.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.981+0000 7f98b3fff640 1 -- 172.21.15.16:0/1550170196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b8032440 con 0x7f98bc06cd20 2024-09-19T07:55:11.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.981+0000 7f98b3fff640 1 -- 172.21.15.16:0/1550170196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8032810 con 0x7f98bc06cd20 2024-09-19T07:55:11.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.982+0000 7f98c32ba640 1 -- 172.21.15.16:0/1550170196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc06cd20 msgr2=0x7f98bc06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:11.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.982+0000 7f98c32ba640 1 --2- 172.21.15.16:0/1550170196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc06cd20 0x7f98bc06d100 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009870 tx=0x7f98b802a210 comp rx=0 tx=0).stop 2024-09-19T07:55:11.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.982+0000 7f98c32ba640 1 -- 172.21.15.16:0/1550170196 shutdown_connections 2024-09-19T07:55:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.982+0000 7f98c32ba640 1 --2- 172.21.15.16:0/1550170196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc06cd20 0x7f98bc06d100 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.982+0000 7f98c32ba640 1 -- 172.21.15.16:0/1550170196 >> 172.21.15.16:0/1550170196 conn(0x7f98bc0b3ed0 msgr2=0x7f98bc0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.982+0000 7f98c32ba640 1 -- 172.21.15.16:0/1550170196 shutdown_connections 2024-09-19T07:55:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.982+0000 7f98c32ba640 1 -- 172.21.15.16:0/1550170196 wait complete. 2024-09-19T07:55:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.983+0000 7f98c32ba640 1 Processor -- start 2024-09-19T07:55:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.983+0000 7f98c32ba640 1 -- start start 2024-09-19T07:55:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.984+0000 7f98c32ba640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98bc06cd20 0x7f98bc0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.984+0000 7f98c32ba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98bc1425e0 0x7f98bc0c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.984+0000 7f98c32ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc161eb0 0x7f98bc0c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.984+0000 7f98c32ba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc141600 con 0x7f98bc161eb0 2024-09-19T07:55:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.984+0000 7f98c32ba640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc142150 con 0x7f98bc06cd20 2024-09-19T07:55:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.984+0000 7f98c32ba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f98bc143180 con 0x7f98bc1425e0 2024-09-19T07:55:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.984+0000 7f98c1857640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc161eb0 0x7f98bc0c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.984+0000 7f98c1857640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc161eb0 0x7f98bc0c8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43898/0 (socket says 172.21.15.16:43898) 2024-09-19T07:55:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.984+0000 7f98c1857640 1 -- 172.21.15.16:0/4254511467 learned_addr learned my addr 172.21.15.16:0/4254511467 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.984+0000 7f98c1056640 1 --2- 172.21.15.16:0/4254511467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98bc06cd20 0x7f98bc0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.984+0000 7f98c0855640 1 --2- 172.21.15.16:0/4254511467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98bc1425e0 0x7f98bc0c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.984+0000 7f98c1857640 1 -- 172.21.15.16:0/4254511467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98bc1425e0 msgr2=0x7f98bc0c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.984+0000 7f98c1857640 1 --2- 172.21.15.16:0/4254511467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98bc1425e0 0x7f98bc0c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.984+0000 7f98c1857640 1 -- 172.21.15.16:0/4254511467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98bc06cd20 msgr2=0x7f98bc0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.984+0000 7f98c1857640 1 --2- 172.21.15.16:0/4254511467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98bc06cd20 0x7f98bc0b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.984+0000 7f98c1857640 1 -- 172.21.15.16:0/4254511467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc0b7f30 con 0x7f98bc161eb0 2024-09-19T07:55:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.984+0000 7f98c1857640 1 --2- 172.21.15.16:0/4254511467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc161eb0 0x7f98bc0c8800 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f98b400c7b0 tx=0x7f98b400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.985+0000 7f98b1ffb640 1 -- 172.21.15.16:0/4254511467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4017070 con 0x7f98bc161eb0 2024-09-19T07:55:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.985+0000 7f98b1ffb640 1 -- 172.21.15.16:0/4254511467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b400a640 con 0x7f98bc161eb0 2024-09-19T07:55:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.985+0000 7f98c32ba640 1 -- 172.21.15.16:0/4254511467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98bc0cd010 con 0x7f98bc161eb0 2024-09-19T07:55:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.985+0000 7f98b1ffb640 1 -- 172.21.15.16:0/4254511467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b400a9f0 con 0x7f98bc161eb0 2024-09-19T07:55:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.985+0000 7f98c32ba640 1 -- 172.21.15.16:0/4254511467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98bc0cd5d0 con 0x7f98bc161eb0 2024-09-19T07:55:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.986+0000 7f98b1ffb640 1 -- 172.21.15.16:0/4254511467 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f98b401a070 con 0x7f98bc161eb0 2024-09-19T07:55:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.987+0000 7f98b1ffb640 1 --2- 172.21.15.16:0/4254511467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f98a006c2b0 0x7f98a006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.987+0000 7f98b1ffb640 1 -- 172.21.15.16:0/4254511467 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f98b408ab10 con 0x7f98bc161eb0 2024-09-19T07:55:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.987+0000 7f98c1056640 1 --2- 172.21.15.16:0/4254511467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f98a006c2b0 0x7f98a006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.988+0000 7f98c1056640 1 --2- 172.21.15.16:0/4254511467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f98a006c2b0 0x7f98a006e770 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f98b8034000 tx=0x7f98b802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.988+0000 7f98c605c740 1 Processor -- start 2024-09-19T07:55:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.988+0000 7f98c605c740 1 -- start start 2024-09-19T07:55:11.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.989+0000 7f98c605c740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560eb7e302c0 0x560eb7e75a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:11.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.989+0000 7f98c605c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e2da80 0x560eb7e306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:11.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.989+0000 7f98c605c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e31a50 0x560eb7e2a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.989+0000 7f98c605c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560eb7d92520 con 0x560eb7e31a50 2024-09-19T07:55:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.989+0000 7f98c605c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560eb7d3a3e0 con 0x560eb7e2da80 2024-09-19T07:55:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.989+0000 7f98c605c740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x560eb7d8e4e0 con 0x560eb7e302c0 2024-09-19T07:55:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.989+0000 7f98c1857640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e31a50 0x560eb7e2a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.989+0000 7f98c1857640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e31a50 0x560eb7e2a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43904/0 (socket says 172.21.15.16:43904) 2024-09-19T07:55:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.989+0000 7f98c1857640 1 -- 172.21.15.16:0/2191070893 learned_addr learned my addr 172.21.15.16:0/2191070893 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.989+0000 7f98c0855640 1 --2- 172.21.15.16:0/2191070893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e2da80 0x560eb7e306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.989+0000 7f98c1857640 1 -- 172.21.15.16:0/2191070893 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560eb7e302c0 msgr2=0x560eb7e75a60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:55:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.989+0000 7f98c1857640 1 --2- 172.21.15.16:0/2191070893 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560eb7e302c0 0x560eb7e75a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.989+0000 7f98c1857640 1 -- 172.21.15.16:0/2191070893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e2da80 msgr2=0x560eb7e306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.989+0000 7f98c1857640 1 --2- 172.21.15.16:0/2191070893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e2da80 0x560eb7e306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.989+0000 7f98c1857640 1 -- 172.21.15.16:0/2191070893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eb7d552f0 con 0x560eb7e31a50 2024-09-19T07:55:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.989+0000 7f98c0855640 1 --2- 172.21.15.16:0/2191070893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e2da80 0x560eb7e306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.989+0000 7f98c1857640 1 --2- 172.21.15.16:0/2191070893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e31a50 0x560eb7e2a8b0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f98b4058c60 tx=0x7f98b405ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c498c526010ec922 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.989+0000 7f98a6ffd640 1 -- 172.21.15.16:0/2191070893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4055110 con 0x560eb7e31a50 2024-09-19T07:55:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.990+0000 7f98a6ffd640 1 -- 172.21.15.16:0/2191070893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b405f0c0 con 0x560eb7e31a50 2024-09-19T07:55:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.990+0000 7f98a6ffd640 1 -- 172.21.15.16:0/2191070893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b40552b0 con 0x560eb7e31a50 2024-09-19T07:55:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.990+0000 7f98c605c740 1 -- 172.21.15.16:0/2191070893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e31a50 msgr2=0x560eb7e2a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.990+0000 7f98c605c740 1 --2- 172.21.15.16:0/2191070893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e31a50 0x560eb7e2a8b0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f98b4058c60 tx=0x7f98b405ce60 comp rx=0 tx=0).stop 2024-09-19T07:55:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.990+0000 7f98c605c740 1 -- 172.21.15.16:0/2191070893 shutdown_connections 2024-09-19T07:55:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.990+0000 7f98c605c740 1 --2- 172.21.15.16:0/2191070893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e31a50 0x560eb7e2a8b0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.990+0000 7f98c605c740 1 --2- 172.21.15.16:0/2191070893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e2da80 0x560eb7e306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.990+0000 7f98c605c740 1 --2- 172.21.15.16:0/2191070893 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560eb7e302c0 0x560eb7e75a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.990+0000 7f98c605c740 1 -- 172.21.15.16:0/2191070893 >> 172.21.15.16:0/2191070893 conn(0x560eb7e2ca80 msgr2=0x560eb7e74f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.990+0000 7f98c605c740 1 -- 172.21.15.16:0/2191070893 shutdown_connections 2024-09-19T07:55:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.990+0000 7f98c605c740 1 -- 172.21.15.16:0/2191070893 wait complete. 2024-09-19T07:55:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.991+0000 7f98c605c740 1 -- start start 2024-09-19T07:55:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.991+0000 7f98c605c740 1 Processor -- start 2024-09-19T07:55:11.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f98c605c740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560eb7e302c0 0x560eb7e4ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:11.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f98c605c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e31a50 0x560eb7e50e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:11.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f98c605c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e2e920 0x560eb7e2eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:11.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f98c605c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560eb7d92520 con 0x560eb7e2e920 2024-09-19T07:55:11.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f98c605c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560eb7d1b450 con 0x560eb7e31a50 2024-09-19T07:55:11.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f98c605c740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x560eb7e2bf20 con 0x560eb7e302c0 2024-09-19T07:55:11.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f98c605c740 10 client.0 _ll_register_callbacks cb 0x560eb7aa4ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:55:11.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f98c1857640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e2e920 0x560eb7e2eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:11.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f98c1857640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e2e920 0x560eb7e2eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43916/0 (socket says 172.21.15.16:43916) 2024-09-19T07:55:11.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f98c1857640 1 -- 172.21.15.16:0/2813530546 learned_addr learned my addr 172.21.15.16:0/2813530546 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:11.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f98c1056640 1 --2- 172.21.15.16:0/2813530546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560eb7e302c0 0x560eb7e4ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:11.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f98c0855640 1 --2- 172.21.15.16:0/2813530546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e31a50 0x560eb7e50e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:11.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f98c1857640 1 -- 172.21.15.16:0/2813530546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560eb7e302c0 msgr2=0x560eb7e4ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f98c1857640 1 --2- 172.21.15.16:0/2813530546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560eb7e302c0 0x560eb7e4ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f98c1857640 1 -- 172.21.15.16:0/2813530546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e31a50 msgr2=0x560eb7e50e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f98c1857640 1 --2- 172.21.15.16:0/2813530546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e31a50 0x560eb7e50e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f98c1857640 1 -- 172.21.15.16:0/2813530546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eb7aa52d0 con 0x560eb7e2e920 2024-09-19T07:55:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f98c1857640 1 --2- 172.21.15.16:0/2813530546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e2e920 0x560eb7e2eda0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f98b400c780 tx=0x7f98b401e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4e99e4d17b6a28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f989bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:55:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.992+0000 7f989bfff640 1 -- 172.21.15.16:0/2813530546 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4075060 con 0x560eb7e2e920 2024-09-19T07:55:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.993+0000 7f989bfff640 1 -- 172.21.15.16:0/2813530546 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f98b400ab90 con 0x560eb7e2e920 2024-09-19T07:55:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.993+0000 7f98c605c740 1 -- 172.21.15.16:0/2813530546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98b405f0c0 con 0x560eb7e2e920 2024-09-19T07:55:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.993+0000 7f989bfff640 1 -- 172.21.15.16:0/2813530546 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f98b400ad90 con 0x560eb7e2e920 2024-09-19T07:55:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.993+0000 7f98c605c740 10 client.8460 fetch_fsmap learned FSMap version 81 2024-09-19T07:55:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.993+0000 7f98c605c740 1 -- 172.21.15.16:0/2813530546 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x560eb7b6f6b0 con 0x560eb7e2e920 2024-09-19T07:55:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.993+0000 7f989bfff640 1 -- 172.21.15.16:0/2813530546 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b405f0c0 con 0x560eb7e2e920 2024-09-19T07:55:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.993+0000 7f989bfff640 1 -- 172.21.15.16:0/2813530546 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 81) ==== 3502+0+0 (secure 0 0 0) 0x7f98b4057ef0 con 0x560eb7e2e920 2024-09-19T07:55:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.993+0000 7f98c605c740 10 client.8460 fetch_fsmap finished waiting for FSMap version 81 2024-09-19T07:55:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.993+0000 7f98c605c740 10 client.8460 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T07:55:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.993+0000 7f98c605c740 20 client.8460 populate_metadata read hostname 'smithi016' 2024-09-19T07:55:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.993+0000 7f98c605c740 1 --2- 172.21.15.16:0/2813530546 >> [v2:172.21.15.16:6833/1929965275,v1:172.21.15.16:6835/1929965275] conn(0x560eb7e59e60 0x560eb7e297d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.993+0000 7f98c605c740 4 client.8460 mds_command: new command op to 8286 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:55:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.993+0000 7f98c605c740 1 -- 172.21.15.16:0/2813530546 --> [v2:172.21.15.16:6833/1929965275,v1:172.21.15.16:6835/1929965275] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560eb7d9fc50 con 0x560eb7e59e60 2024-09-19T07:55:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.993+0000 7f98c1056640 1 --2- 172.21.15.16:0/2813530546 >> [v2:172.21.15.16:6833/1929965275,v1:172.21.15.16:6835/1929965275] conn(0x560eb7e59e60 0x560eb7e297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.993+0000 7f98c1056640 1 --2- 172.21.15.16:0/2813530546 >> [v2:172.21.15.16:6833/1929965275,v1:172.21.15.16:6835/1929965275] conn(0x560eb7e59e60 0x560eb7e297d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bc59f1b078d29b8 server_cookie=2cc9d273b908a120 in_seq=0 out_seq=0 2024-09-19T07:55:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.993+0000 7f989bfff640 10 client.8460 ms_handle_connect on v2:172.21.15.16:6833/1929965275 2024-09-19T07:55:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.994+0000 7f989bfff640 1 -- 172.21.15.16:0/2813530546 <== mds.0 v2:172.21.15.16:6833/1929965275 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560eb7d9fc50 con 0x560eb7e59e60 2024-09-19T07:55:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.994+0000 7f989bfff640 10 client.8460 handle_command_reply: tid=0 2024-09-19T07:55:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.994+0000 7f98c605c740 1 client.8460 shutdown 2024-09-19T07:55:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.994+0000 7f98c605c740 1 -- 172.21.15.16:0/2813530546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e2e920 msgr2=0x560eb7e2eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.994+0000 7f98c605c740 1 --2- 172.21.15.16:0/2813530546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e2e920 0x560eb7e2eda0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f98b400c780 tx=0x7f98b401e070 comp rx=0 tx=0).stop 2024-09-19T07:55:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.994+0000 7f98c605c740 1 -- 172.21.15.16:0/2813530546 shutdown_connections 2024-09-19T07:55:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.994+0000 7f98c605c740 1 --2- 172.21.15.16:0/2813530546 >> [v2:172.21.15.16:6833/1929965275,v1:172.21.15.16:6835/1929965275] conn(0x560eb7e59e60 0x560eb7e297d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.994+0000 7f98c605c740 1 --2- 172.21.15.16:0/2813530546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e2e920 0x560eb7e2eda0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.994+0000 7f98c605c740 1 --2- 172.21.15.16:0/2813530546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e31a50 0x560eb7e50e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.994+0000 7f989bfff640 0 client.8460 ms_handle_reset on v2:172.21.15.16:6833/1929965275 2024-09-19T07:55:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.994+0000 7f98c605c740 1 --2- 172.21.15.16:0/2813530546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560eb7e302c0 0x560eb7e4ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.994+0000 7f98c605c740 1 -- 172.21.15.16:0/2813530546 >> 172.21.15.16:0/2813530546 conn(0x560eb7e2da80 msgr2=0x560eb7e72fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.994+0000 7f98c605c740 1 -- 172.21.15.16:0/2813530546 shutdown_connections 2024-09-19T07:55:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.994+0000 7f98c605c740 1 -- 172.21.15.16:0/2813530546 wait complete. 2024-09-19T07:55:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:11.995+0000 7f98c605c740 20 client.8460 trim_cache size 0 max 16384 2024-09-19T07:55:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.014+0000 7f98c605c740 1 Processor -- start 2024-09-19T07:55:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.014+0000 7f98c605c740 1 -- start start 2024-09-19T07:55:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.015+0000 7f98c605c740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560eb7e2e920 0x560eb7e87460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.015+0000 7f98c605c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e302c0 0x560eb7eedc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.015+0000 7f98c605c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e31a50 0x560eb7eee1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.015+0000 7f98c605c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc200170 con 0x560eb7e302c0 2024-09-19T07:55:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.015+0000 7f98c605c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc0ce730 con 0x560eb7e31a50 2024-09-19T07:55:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.015+0000 7f98c605c740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f98bc0ced80 con 0x560eb7e2e920 2024-09-19T07:55:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.015+0000 7f98c0855640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e302c0 0x560eb7eedc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.015+0000 7f98c0855640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e302c0 0x560eb7eedc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43932/0 (socket says 172.21.15.16:43932) 2024-09-19T07:55:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.015+0000 7f98c0855640 1 -- 172.21.15.16:0/1810106025 learned_addr learned my addr 172.21.15.16:0/1810106025 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.015+0000 7f98c1056640 1 --2- 172.21.15.16:0/1810106025 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560eb7e2e920 0x560eb7e87460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.015+0000 7f98c1857640 1 --2- 172.21.15.16:0/1810106025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e31a50 0x560eb7eee1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.016+0000 7f98c0855640 1 -- 172.21.15.16:0/1810106025 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560eb7e2e920 msgr2=0x560eb7e87460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.016+0000 7f98c0855640 1 --2- 172.21.15.16:0/1810106025 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560eb7e2e920 0x560eb7e87460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.016+0000 7f98c0855640 1 -- 172.21.15.16:0/1810106025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e31a50 msgr2=0x560eb7eee1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.016+0000 7f98c0855640 1 --2- 172.21.15.16:0/1810106025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e31a50 0x560eb7eee1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.016+0000 7f98c0855640 1 -- 172.21.15.16:0/1810106025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eb7d9db30 con 0x560eb7e302c0 2024-09-19T07:55:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.016+0000 7f98c0855640 1 --2- 172.21.15.16:0/1810106025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e302c0 0x560eb7eedc70 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00d940 tx=0x7f98ac00de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55281c5f241e2e68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.016+0000 7f98a67fc640 1 -- 172.21.15.16:0/1810106025 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac019020 con 0x560eb7e302c0 2024-09-19T07:55:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.016+0000 7f98a67fc640 1 -- 172.21.15.16:0/1810106025 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ac00fbf0 con 0x560eb7e302c0 2024-09-19T07:55:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.016+0000 7f98a67fc640 1 -- 172.21.15.16:0/1810106025 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac015470 con 0x560eb7e302c0 2024-09-19T07:55:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.017+0000 7f98c605c740 1 -- 172.21.15.16:0/1810106025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e302c0 msgr2=0x560eb7eedc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.017+0000 7f98c605c740 1 --2- 172.21.15.16:0/1810106025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e302c0 0x560eb7eedc70 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00d940 tx=0x7f98ac00de00 comp rx=0 tx=0).stop 2024-09-19T07:55:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.017+0000 7f98c605c740 1 -- 172.21.15.16:0/1810106025 shutdown_connections 2024-09-19T07:55:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.017+0000 7f98c605c740 1 --2- 172.21.15.16:0/1810106025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e31a50 0x560eb7eee1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.017+0000 7f98c605c740 1 --2- 172.21.15.16:0/1810106025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e302c0 0x560eb7eedc70 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.017+0000 7f98c605c740 1 --2- 172.21.15.16:0/1810106025 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560eb7e2e920 0x560eb7e87460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.017+0000 7f98c605c740 1 -- 172.21.15.16:0/1810106025 >> 172.21.15.16:0/1810106025 conn(0x560eb7e2da80 msgr2=0x560eb7e844a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.017+0000 7f98c605c740 1 -- 172.21.15.16:0/1810106025 shutdown_connections 2024-09-19T07:55:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.017+0000 7f98c605c740 1 -- 172.21.15.16:0/1810106025 wait complete. 2024-09-19T07:55:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.017+0000 7f98c605c740 1 -- start start 2024-09-19T07:55:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.017+0000 7f98c605c740 1 Processor -- start 2024-09-19T07:55:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.018+0000 7f98c605c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e2e920 0x560eb7ef2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.018+0000 7f98c605c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e302c0 0x560eb7ef2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.018+0000 7f98c605c740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560eb7e31a50 0x560eb7e70ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.018+0000 7f98c605c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560eb7d92520 con 0x560eb7e2e920 2024-09-19T07:55:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.018+0000 7f98c605c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560eb7e2bf20 con 0x560eb7e302c0 2024-09-19T07:55:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.018+0000 7f98c605c740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x560eb7d1b450 con 0x560eb7e31a50 2024-09-19T07:55:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.018+0000 7f98c605c740 10 client.0 _ll_register_callbacks cb 0x7f98b406f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:55:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.018+0000 7f98c1056640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e2e920 0x560eb7ef2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.018+0000 7f98c1056640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e2e920 0x560eb7ef2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43946/0 (socket says 172.21.15.16:43946) 2024-09-19T07:55:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.018+0000 7f98c1056640 1 -- 172.21.15.16:0/3540816907 learned_addr learned my addr 172.21.15.16:0/3540816907 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.018+0000 7f98c1857640 1 --2- 172.21.15.16:0/3540816907 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560eb7e31a50 0x560eb7e70ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.018+0000 7f98c0855640 1 --2- 172.21.15.16:0/3540816907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e302c0 0x560eb7ef2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:12.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f98c1056640 1 -- 172.21.15.16:0/3540816907 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560eb7e31a50 msgr2=0x560eb7e70ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:12.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f98c1056640 1 --2- 172.21.15.16:0/3540816907 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560eb7e31a50 0x560eb7e70ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f98c1056640 1 -- 172.21.15.16:0/3540816907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e302c0 msgr2=0x560eb7ef2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:12.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f98c1056640 1 --2- 172.21.15.16:0/3540816907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e302c0 0x560eb7ef2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f98c1056640 1 -- 172.21.15.16:0/3540816907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eb7e88cf0 con 0x560eb7e2e920 2024-09-19T07:55:12.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f98c0855640 1 --2- 172.21.15.16:0/3540816907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e302c0 0x560eb7ef2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:12.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f98c1056640 1 --2- 172.21.15.16:0/3540816907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e2e920 0x560eb7ef2a70 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f98b80090b0 tx=0x7f98b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d51229ede40b58 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:12.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f989bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:55:12.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f989bfff640 1 -- 172.21.15.16:0/3540816907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8037040 con 0x560eb7e2e920 2024-09-19T07:55:12.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f989bfff640 1 -- 172.21.15.16:0/3540816907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f98b8056c80 con 0x560eb7e2e920 2024-09-19T07:55:12.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f98c605c740 1 -- 172.21.15.16:0/3540816907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98ac00fbf0 con 0x560eb7e2e920 2024-09-19T07:55:12.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f989bfff640 1 -- 172.21.15.16:0/3540816907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f98b803c7e0 con 0x560eb7e2e920 2024-09-19T07:55:12.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f98c605c740 10 client.8466 fetch_fsmap learned FSMap version 81 2024-09-19T07:55:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f98c605c740 1 -- 172.21.15.16:0/3540816907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x560eb7ef35a0 con 0x560eb7e2e920 2024-09-19T07:55:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f989bfff640 1 -- 172.21.15.16:0/3540816907 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ac00fbf0 con 0x560eb7e2e920 2024-09-19T07:55:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f989bfff640 1 -- 172.21.15.16:0/3540816907 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 81) ==== 3502+0+0 (secure 0 0 0) 0x7f98b805d050 con 0x560eb7e2e920 2024-09-19T07:55:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f98c605c740 10 client.8466 fetch_fsmap finished waiting for FSMap version 81 2024-09-19T07:55:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f98c605c740 10 client.8466 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T07:55:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f98c605c740 20 client.8466 populate_metadata read hostname 'smithi016' 2024-09-19T07:55:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f98c605c740 1 --2- 172.21.15.16:0/3540816907 >> [v2:172.21.15.16:6833/1929965275,v1:172.21.15.16:6835/1929965275] conn(0x560eb7e4c9c0 0x560eb7ef68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f98c605c740 4 client.8466 mds_command: new command op to 8286 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:55:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.019+0000 7f98c605c740 1 -- 172.21.15.16:0/3540816907 --> [v2:172.21.15.16:6833/1929965275,v1:172.21.15.16:6835/1929965275] -- command(tid 0: {"prefix": "status"}) -- 0x560eb7d99fa0 con 0x560eb7e4c9c0 2024-09-19T07:55:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.020+0000 7f98c0855640 1 --2- 172.21.15.16:0/3540816907 >> [v2:172.21.15.16:6833/1929965275,v1:172.21.15.16:6835/1929965275] conn(0x560eb7e4c9c0 0x560eb7ef68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.020+0000 7f98c0855640 1 --2- 172.21.15.16:0/3540816907 >> [v2:172.21.15.16:6833/1929965275,v1:172.21.15.16:6835/1929965275] conn(0x560eb7e4c9c0 0x560eb7ef68b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bc081d668179fce server_cookie=9398cd995b635dd0 in_seq=0 out_seq=0 2024-09-19T07:55:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.020+0000 7f989bfff640 10 client.8466 ms_handle_connect on v2:172.21.15.16:6833/1929965275 2024-09-19T07:55:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.020+0000 7f989bfff640 1 -- 172.21.15.16:0/3540816907 <== mds.0 v2:172.21.15.16:6833/1929965275 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x560eb7d99fa0 con 0x560eb7e4c9c0 2024-09-19T07:55:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.020+0000 7f989bfff640 10 client.8466 handle_command_reply: tid=0 2024-09-19T07:55:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.020+0000 7f98c605c740 1 client.8466 shutdown 2024-09-19T07:55:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.020+0000 7f98c605c740 1 -- 172.21.15.16:0/3540816907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e2e920 msgr2=0x560eb7ef2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.020+0000 7f98c605c740 1 --2- 172.21.15.16:0/3540816907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e2e920 0x560eb7ef2a70 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f98b80090b0 tx=0x7f98b8004730 comp rx=0 tx=0).stop 2024-09-19T07:55:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.020+0000 7f98c605c740 1 -- 172.21.15.16:0/3540816907 shutdown_connections 2024-09-19T07:55:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.020+0000 7f98c605c740 1 --2- 172.21.15.16:0/3540816907 >> [v2:172.21.15.16:6833/1929965275,v1:172.21.15.16:6835/1929965275] conn(0x560eb7e4c9c0 0x560eb7ef68b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.020+0000 7f98c605c740 1 --2- 172.21.15.16:0/3540816907 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560eb7e31a50 0x560eb7e70ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.020+0000 7f98c605c740 1 --2- 172.21.15.16:0/3540816907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eb7e302c0 0x560eb7ef2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.020+0000 7f98c605c740 1 --2- 172.21.15.16:0/3540816907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560eb7e2e920 0x560eb7ef2a70 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.020+0000 7f98c605c740 1 -- 172.21.15.16:0/3540816907 >> 172.21.15.16:0/3540816907 conn(0x560eb7e2da80 msgr2=0x560eb7e87460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.020+0000 7f989bfff640 0 client.8466 ms_handle_reset on v2:172.21.15.16:6833/1929965275 2024-09-19T07:55:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.020+0000 7f98c605c740 1 -- 172.21.15.16:0/3540816907 shutdown_connections 2024-09-19T07:55:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.020+0000 7f98c605c740 1 -- 172.21.15.16:0/3540816907 wait complete. 2024-09-19T07:55:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.021+0000 7f98c605c740 20 client.8466 trim_cache size 0 max 16384 2024-09-19T07:55:12.031 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:55:12.031 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb", 2024-09-19T07:55:12.031 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-19T07:55:12.031 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 8286, 2024-09-19T07:55:12.031 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T07:55:12.031 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T07:55:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T07:55:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 9.0449904080000003, 2024-09-19T07:55:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 81, 2024-09-19T07:55:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 140, 2024-09-19T07:55:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 140, 2024-09-19T07:55:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 14.077973352000001 2024-09-19T07:55:12.032 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:55:12.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.021+0000 7f98c32ba640 1 -- 172.21.15.16:0/4254511467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f98a006c2b0 msgr2=0x7f98a006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:12.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.021+0000 7f98c32ba640 1 --2- 172.21.15.16:0/4254511467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f98a006c2b0 0x7f98a006e770 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f98b8034000 tx=0x7f98b802e000 comp rx=0 tx=0).stop 2024-09-19T07:55:12.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.021+0000 7f98c32ba640 1 -- 172.21.15.16:0/4254511467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc161eb0 msgr2=0x7f98bc0c8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:12.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.021+0000 7f98c32ba640 1 --2- 172.21.15.16:0/4254511467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc161eb0 0x7f98bc0c8800 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f98b400c7b0 tx=0x7f98b400cc70 comp rx=0 tx=0).stop 2024-09-19T07:55:12.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.021+0000 7f98c32ba640 1 -- 172.21.15.16:0/4254511467 shutdown_connections 2024-09-19T07:55:12.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.021+0000 7f98c32ba640 1 --2- 172.21.15.16:0/4254511467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f98a006c2b0 0x7f98a006e770 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.021+0000 7f98c32ba640 1 --2- 172.21.15.16:0/4254511467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc161eb0 0x7f98bc0c8800 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.021+0000 7f98c32ba640 1 --2- 172.21.15.16:0/4254511467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98bc1425e0 0x7f98bc0c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.021+0000 7f98c32ba640 1 --2- 172.21.15.16:0/4254511467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98bc06cd20 0x7f98bc0b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.021+0000 7f98c32ba640 1 -- 172.21.15.16:0/4254511467 >> 172.21.15.16:0/4254511467 conn(0x7f98bc0b3ed0 msgr2=0x7f98bc160cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:12.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.021+0000 7f98c32ba640 1 -- 172.21.15.16:0/4254511467 shutdown_connections 2024-09-19T07:55:12.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.021+0000 7f98c32ba640 1 -- 172.21.15.16:0/4254511467 wait complete. 2024-09-19T07:55:12.035 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:55:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.137+0000 7fb53f69b640 1 Processor -- start 2024-09-19T07:55:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.137+0000 7fb53f69b640 1 -- start start 2024-09-19T07:55:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.138+0000 7fb53f69b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53806d430 0x7fb5381439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.138+0000 7fb53f69b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb538143f30 0x7fb5380d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.138+0000 7fb53f69b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5380c7140 con 0x7fb5380c8d20 2024-09-19T07:55:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.138+0000 7fb53f69b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb538141960 con 0x7fb538143f30 2024-09-19T07:55:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.138+0000 7fb53f69b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb538142200 con 0x7fb53806d430 2024-09-19T07:55:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.138+0000 7fb53dc38640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb538143f30 0x7fb5380d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.138+0000 7fb53d437640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb5380c8d20 0x7fb53806cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34048/0 (socket says 172.21.15.16:34048) 2024-09-19T07:55:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.138+0000 7fb53d437640 1 -- 172.21.15.16:0/3888090602 learned_addr learned my addr 172.21.15.16:0/3888090602 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.138+0000 7fb53dc38640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb538143f30 0x7fb5380d1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43948/0 (socket says 172.21.15.16:43948) 2024-09-19T07:55:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.138+0000 7fb53cc36640 1 --2- 172.21.15.16:0/3888090602 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53806d430 0x7fb5381439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.138+0000 7fb53dc38640 1 -- 172.21.15.16:0/3888090602 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53806d430 msgr2=0x7fb5381439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.138+0000 7fb53dc38640 1 --2- 172.21.15.16:0/3888090602 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53806d430 0x7fb5381439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.138+0000 7fb53dc38640 1 -- 172.21.15.16:0/3888090602 >> v1:172.21.15.16:6789/0 conn(0x7fb5380c8d20 legacy=0x7fb53806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.138+0000 7fb52ffff640 1 -- 172.21.15.16:0/3888090602 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3489386743 0 0) 0x7fb534001110 con 0x7fb5380c8d20 2024-09-19T07:55:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.138+0000 7fb53dc38640 1 -- 172.21.15.16:0/3888090602 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5380c9100 con 0x7fb538143f30 2024-09-19T07:55:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.139+0000 7fb53dc38640 1 --2- 172.21.15.16:0/3888090602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb538143f30 0x7fb5380d1e50 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fb530009d90 tx=0x7fb53002a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f800d2e81c4f9ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.139+0000 7fb52ffff640 1 -- 172.21.15.16:0/3888090602 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530007740 con 0x7fb538143f30 2024-09-19T07:55:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.139+0000 7fb52ffff640 1 -- 172.21.15.16:0/3888090602 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5300078e0 con 0x7fb538143f30 2024-09-19T07:55:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.139+0000 7fb52ffff640 1 -- 172.21.15.16:0/3888090602 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530007cb0 con 0x7fb538143f30 2024-09-19T07:55:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.140+0000 7fb53f69b640 1 -- 172.21.15.16:0/3888090602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb538143f30 msgr2=0x7fb5380d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.140+0000 7fb53f69b640 1 --2- 172.21.15.16:0/3888090602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb538143f30 0x7fb5380d1e50 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fb530009d90 tx=0x7fb53002a530 comp rx=0 tx=0).stop 2024-09-19T07:55:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.140+0000 7fb53f69b640 1 -- 172.21.15.16:0/3888090602 shutdown_connections 2024-09-19T07:55:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.140+0000 7fb53f69b640 1 --2- 172.21.15.16:0/3888090602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb538143f30 0x7fb5380d1e50 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.140+0000 7fb53f69b640 1 --2- 172.21.15.16:0/3888090602 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53806d430 0x7fb5381439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.140+0000 7fb53f69b640 1 -- 172.21.15.16:0/3888090602 >> 172.21.15.16:0/3888090602 conn(0x7fb5380b3ed0 msgr2=0x7fb5380b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.140+0000 7fb53f69b640 1 -- 172.21.15.16:0/3888090602 shutdown_connections 2024-09-19T07:55:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.140+0000 7fb53f69b640 1 -- 172.21.15.16:0/3888090602 wait complete. 2024-09-19T07:55:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.141+0000 7fb53f69b640 1 Processor -- start 2024-09-19T07:55:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.141+0000 7fb53f69b640 1 -- start start 2024-09-19T07:55:12.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.142+0000 7fb53f69b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53806d430 0x7fb538140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.142+0000 7fb53f69b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5380c8d20 0x7fb5380b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.142+0000 7fb53f69b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb538143f30 0x7fb53815e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.142+0000 7fb53f69b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb538141e90 con 0x7fb5380c8d20 2024-09-19T07:55:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.142+0000 7fb53f69b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb538141bb0 con 0x7fb538143f30 2024-09-19T07:55:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.142+0000 7fb53f69b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb538144380 con 0x7fb53806d430 2024-09-19T07:55:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.142+0000 7fb53cc36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5380c8d20 0x7fb5380b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.142+0000 7fb53cc36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5380c8d20 0x7fb5380b6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43956/0 (socket says 172.21.15.16:43956) 2024-09-19T07:55:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.142+0000 7fb53cc36640 1 -- 172.21.15.16:0/4204784907 learned_addr learned my addr 172.21.15.16:0/4204784907 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.142+0000 7fb53d437640 1 --2- 172.21.15.16:0/4204784907 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53806d430 0x7fb538140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.142+0000 7fb53dc38640 1 --2- 172.21.15.16:0/4204784907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb538143f30 0x7fb53815e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.142+0000 7fb53cc36640 1 -- 172.21.15.16:0/4204784907 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53806d430 msgr2=0x7fb538140be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.142+0000 7fb53cc36640 1 --2- 172.21.15.16:0/4204784907 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53806d430 0x7fb538140be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.142+0000 7fb53cc36640 1 -- 172.21.15.16:0/4204784907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb538143f30 msgr2=0x7fb53815e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.142+0000 7fb53cc36640 1 --2- 172.21.15.16:0/4204784907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb538143f30 0x7fb53815e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.142+0000 7fb53cc36640 1 -- 172.21.15.16:0/4204784907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5380d2820 con 0x7fb5380c8d20 2024-09-19T07:55:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.142+0000 7fb53cc36640 1 --2- 172.21.15.16:0/4204784907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5380c8d20 0x7fb5380b6e40 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fb52800d620 tx=0x7fb52800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.143+0000 7fb52dffb640 1 -- 172.21.15.16:0/4204784907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5280125f0 con 0x7fb5380c8d20 2024-09-19T07:55:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.143+0000 7fb52dffb640 1 -- 172.21.15.16:0/4204784907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb528019070 con 0x7fb5380c8d20 2024-09-19T07:55:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.143+0000 7fb53f69b640 1 -- 172.21.15.16:0/4204784907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5380d2b00 con 0x7fb5380c8d20 2024-09-19T07:55:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.143+0000 7fb52dffb640 1 -- 172.21.15.16:0/4204784907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5280129d0 con 0x7fb5380c8d20 2024-09-19T07:55:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.143+0000 7fb53f69b640 1 -- 172.21.15.16:0/4204784907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb538141120 con 0x7fb5380c8d20 2024-09-19T07:55:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.144+0000 7fb53f69b640 1 -- 172.21.15.16:0/4204784907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5381439f0 con 0x7fb5380c8d20 2024-09-19T07:55:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.144+0000 7fb52dffb640 1 -- 172.21.15.16:0/4204784907 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb528012b70 con 0x7fb5380c8d20 2024-09-19T07:55:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.145+0000 7fb52dffb640 1 --2- 172.21.15.16:0/4204784907 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb51c06c2b0 0x7fb51c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.145+0000 7fb52dffb640 1 -- 172.21.15.16:0/4204784907 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7fb52808b8e0 con 0x7fb5380c8d20 2024-09-19T07:55:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.145+0000 7fb53d437640 1 --2- 172.21.15.16:0/4204784907 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb51c06c2b0 0x7fb51c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.146+0000 7fb53d437640 1 --2- 172.21.15.16:0/4204784907 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb51c06c2b0 0x7fb51c06e770 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb534007fd0 tx=0x7fb534007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.149+0000 7fb52dffb640 1 -- 172.21.15.16:0/4204784907 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb528059870 con 0x7fb5380c8d20 2024-09-19T07:55:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.364+0000 7fb53f69b640 1 -- 172.21.15.16:0/4204784907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb5380690c0 con 0x7fb5380c8d20 2024-09-19T07:55:12.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.365+0000 7fb52dffb640 1 -- 172.21.15.16:0/4204784907 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) ==== 53+0+73 (secure 0 0 0) 0x7fb52805db70 con 0x7fb5380c8d20 2024-09-19T07:55:12.365 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:55:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.367+0000 7fb53f69b640 1 -- 172.21.15.16:0/4204784907 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb51c06c2b0 msgr2=0x7fb51c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.367+0000 7fb53f69b640 1 --2- 172.21.15.16:0/4204784907 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb51c06c2b0 0x7fb51c06e770 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb534007fd0 tx=0x7fb534007de0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.367+0000 7fb53f69b640 1 -- 172.21.15.16:0/4204784907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5380c8d20 msgr2=0x7fb5380b6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.367+0000 7fb53f69b640 1 --2- 172.21.15.16:0/4204784907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5380c8d20 0x7fb5380b6e40 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fb52800d620 tx=0x7fb52800dae0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.367+0000 7fb53f69b640 1 -- 172.21.15.16:0/4204784907 shutdown_connections 2024-09-19T07:55:12.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.367+0000 7fb53f69b640 1 --2- 172.21.15.16:0/4204784907 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb51c06c2b0 0x7fb51c06e770 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.367+0000 7fb53f69b640 1 --2- 172.21.15.16:0/4204784907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb538143f30 0x7fb53815e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.367+0000 7fb53f69b640 1 --2- 172.21.15.16:0/4204784907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5380c8d20 0x7fb5380b6e40 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.367+0000 7fb53f69b640 1 --2- 172.21.15.16:0/4204784907 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53806d430 0x7fb538140be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.367+0000 7fb53f69b640 1 -- 172.21.15.16:0/4204784907 >> 172.21.15.16:0/4204784907 conn(0x7fb5380b3ed0 msgr2=0x7fb53815fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.367+0000 7fb53f69b640 1 -- 172.21.15.16:0/4204784907 shutdown_connections 2024-09-19T07:55:12.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12.367+0000 7fb53f69b640 1 -- 172.21.15.16:0/4204784907 wait complete. 2024-09-19T07:55:12.378 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:55:12.378 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:55:12.378 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T07:55:12.378 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T07:55:12.378 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:55:12.378 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:55:12.378 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:55:12.378 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:55:12.378 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:55:12.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:12.379 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T07:55:12.395 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:55:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:55:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:55:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:55:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:12.395 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:55:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T07:55:12.395 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:55:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T07:55:12.396 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:55:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T07:55:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T07:55:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42077sec preferred_lft 42077sec 2024-09-19T07:55:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T07:55:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:12.396 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:55:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T07:55:12.396 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:55:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:55:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::32:7bff:fe92:8895/64 scope link 2024-09-19T07:55:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:12.397 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:55:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether e6:fd:21:6a:43:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:55:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e4fd:21ff:fe6a:4369/64 scope link 2024-09-19T07:55:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:12.397 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:55:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:55:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::80f4:81ff:fe7d:b7f0/64 scope link 2024-09-19T07:55:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:12.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:12.398 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:55:12.459 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:55:12.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:12.461 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T07:55:12.518 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T07:55:12.518 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:55:12.518 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T07:55:12.518 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T07:55:12.518 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:55:12.519 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:55:12.519 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:55:12.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T07:55:12.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T07:55:12.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T07:55:12.519 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:55:12.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:55:12.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:55:12.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:12.647 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:55:12.663 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T07:55:12.663 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:55:12.663 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:55:12.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:12.663 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-19T07:55:12.730 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T07:55:12.730 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:55:12.731 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T07:55:12.786 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:55:12.786 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:55:12.786 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:55:12.842 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:55:12.909 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:55:12.966 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:55:12.966 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:55:12.966 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:55:12.966 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:55:12.966 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:55:12.966 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:55:12.966 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:55:12.967 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:55:12.967 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:55:12.967 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:55:12.967 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:55:12.967 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:55:12.967 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:55:12.967 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:55:12.967 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:55:12.967 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:55:12.967 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:55:12.967 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:55:12.967 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:55:12.967 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:55:12.967 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:55:12.968 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:55:12.968 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:55:12.968 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:55:12.968 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:55:12.968 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:55:12.968 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:55:12.968 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:55:12.968 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:55:12.968 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:55:12.968 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:55:12.968 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:55:12.968 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:55:12.968 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:55:12.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:12.969 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:55:13.022 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:55:13.022 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T07:55:13.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:55:13.090 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:55:13.147 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:55:13.147 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:55:13.147 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:55:13.147 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:55:13.147 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:55:13.147 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:55:13.147 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:55:13.148 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:55:13.148 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:55:13.148 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:55:13.148 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:55:13.148 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:55:13.148 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:55:13.149 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:55:13.149 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:55:13.149 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:55:13.149 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:55:13.149 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:55:13.149 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:55:13.149 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:55:13.149 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:55:13.149 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:55:13.149 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:55:13.150 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:55:13.150 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:55:13.150 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:55:13.150 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:55:13.150 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:55:13.150 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:55:13.150 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:55:13.151 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:55:13.151 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:55:13.151 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:55:13.151 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:55:13.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:13.152 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:55:13.153 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-19T07:55:13.148+0000 7faa15120480 -1 init, newargv = 0x55f441b796f0 newargc=15 2024-09-19T07:55:13.153 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[76175]: starting ceph client 2024-09-19T07:55:13.153 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[76175]: starting fuse 2024-09-19T07:55:13.168 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:55:13.168 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-19T07:55:13.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:55:13.169 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:55:13.169 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:55:13.169 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:55:13.169 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:55:13.169 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:55:13.169 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:55:13.169 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:55:13.169 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:55:13.169 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:55:13.169 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:55:13.169 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:55:13.169 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:55:13.170 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:55:13.170 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:55:13.170 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:55:13.170 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:55:13.170 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:55:13.170 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:55:13.170 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:55:13.170 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:55:13.170 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:55:13.170 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:55:13.170 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:55:13.170 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:55:13.170 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:55:13.170 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:55:13.289 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.76175.asok 2024-09-19T07:55:13.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:55:13.300 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.76175.asok 2024-09-19T07:55:13.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:13.300 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.76175.asok status 2024-09-19T07:55:13.437 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:55:14.027 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:55:14.028 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:55:14.028 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:55:14.028 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:55:14.028 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:55:14.028 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:55:14.028 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "76175", 2024-09-19T07:55:14.028 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:55:14.028 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:14.028 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-19T07:55:14.028 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-19T07:55:14.028 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 8484, 2024-09-19T07:55:14.029 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:55:14.029 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:55:14.029 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:55:14.029 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 8484 2024-09-19T07:55:14.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:14.029 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:55:14.029 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:55:14.029 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:55:14.029 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3786269155 2024-09-19T07:55:14.029 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:14.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:14.029 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:55:14.029 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:55:14.030 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:55:14.030 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3786269155 2024-09-19T07:55:14.030 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:14.030 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.8484 192.168.0.1:0/3786269155", 2024-09-19T07:55:14.030 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/3786269155", 2024-09-19T07:55:14.030 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-19T07:55:14.030 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 81, 2024-09-19T07:55:14.030 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 140, 2024-09-19T07:55:14.030 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:55:14.030 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:55:14.030 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:55:14.030 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:55:14.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:14.032 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-19T07:55:14.032 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T07:55:14.050 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-19T07:55:14.050 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T07:55:14.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:14.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:55:14.120 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:55:14.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.266+0000 7fb6d73cf640 1 Processor -- start 2024-09-19T07:55:14.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.266+0000 7fb6d73cf640 1 -- start start 2024-09-19T07:55:14.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.267+0000 7fb6d73cf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6d00c8d20 0x7fb6d006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:14.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.267+0000 7fb6d73cf640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb6d006d260 0x7fb6d01439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:14.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.267+0000 7fb6d73cf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6d00c7140 con 0x7fb6d0143f30 2024-09-19T07:55:14.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.267+0000 7fb6d73cf640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb6d0141960 con 0x7fb6d006d260 2024-09-19T07:55:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.267+0000 7fb6d73cf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d0142200 con 0x7fb6d00c8d20 2024-09-19T07:55:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.267+0000 7fb6d596c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb6d0143f30 0x7fb6d00d1e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41264/0 (socket says 172.21.15.16:41264) 2024-09-19T07:55:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.267+0000 7fb6d596c640 1 -- 172.21.15.16:0/3051070309 learned_addr learned my addr 172.21.15.16:0/3051070309 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.267+0000 7fb6d516b640 1 --2- 172.21.15.16:0/3051070309 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6d00c8d20 0x7fb6d006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.267+0000 7fb6d496a640 1 --2- 172.21.15.16:0/3051070309 >> v2:172.21.15.146:3300/0 conn(0x7fb6d006d260 0x7fb6d01439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.268+0000 7fb6cffff640 1 -- 172.21.15.16:0/3051070309 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1345418342 0 0) 0x7fb6c0001490 con 0x7fb6d0143f30 2024-09-19T07:55:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.268+0000 7fb6cffff640 1 -- 172.21.15.16:0/3051070309 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6b8003520 con 0x7fb6d0143f30 2024-09-19T07:55:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.268+0000 7fb6d516b640 1 -- 172.21.15.16:0/3051070309 >> v2:172.21.15.146:3300/0 conn(0x7fb6d006d260 msgr2=0x7fb6d01439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.268+0000 7fb6d516b640 1 --2- 172.21.15.16:0/3051070309 >> v2:172.21.15.146:3300/0 conn(0x7fb6d006d260 0x7fb6d01439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.268+0000 7fb6d516b640 1 -- 172.21.15.16:0/3051070309 >> v1:172.21.15.16:6789/0 conn(0x7fb6d0143f30 legacy=0x7fb6d00d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.268+0000 7fb6d516b640 1 -- 172.21.15.16:0/3051070309 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d00c9100 con 0x7fb6d00c8d20 2024-09-19T07:55:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.268+0000 7fb6d496a640 1 --2- 172.21.15.16:0/3051070309 >> v2:172.21.15.146:3300/0 conn(0x7fb6d006d260 0x7fb6d01439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:55:14.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.269+0000 7fb6d516b640 1 --2- 172.21.15.16:0/3051070309 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6d00c8d20 0x7fb6d006cd20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80097a0 tx=0x7fb6c802a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dedd18970f821763 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:14.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.269+0000 7fb6cffff640 1 -- 172.21.15.16:0/3051070309 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8037070 con 0x7fb6d00c8d20 2024-09-19T07:55:14.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.269+0000 7fb6cffff640 1 -- 172.21.15.16:0/3051070309 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c802adb0 con 0x7fb6d00c8d20 2024-09-19T07:55:14.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.269+0000 7fb6cffff640 1 -- 172.21.15.16:0/3051070309 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8032640 con 0x7fb6d00c8d20 2024-09-19T07:55:14.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.269+0000 7fb6d73cf640 1 -- 172.21.15.16:0/3051070309 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6d00c8d20 msgr2=0x7fb6d006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:14.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.269+0000 7fb6d73cf640 1 --2- 172.21.15.16:0/3051070309 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6d00c8d20 0x7fb6d006cd20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80097a0 tx=0x7fb6c802a060 comp rx=0 tx=0).stop 2024-09-19T07:55:14.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.269+0000 7fb6d73cf640 1 -- 172.21.15.16:0/3051070309 shutdown_connections 2024-09-19T07:55:14.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.269+0000 7fb6d73cf640 1 --2- 172.21.15.16:0/3051070309 >> v2:172.21.15.146:3300/0 conn(0x7fb6d006d260 0x7fb6d01439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:14.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.269+0000 7fb6d73cf640 1 --2- 172.21.15.16:0/3051070309 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6d00c8d20 0x7fb6d006cd20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:14.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.269+0000 7fb6d73cf640 1 -- 172.21.15.16:0/3051070309 >> 172.21.15.16:0/3051070309 conn(0x7fb6d00b3ed0 msgr2=0x7fb6d00b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:14.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.269+0000 7fb6d73cf640 1 -- 172.21.15.16:0/3051070309 shutdown_connections 2024-09-19T07:55:14.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.269+0000 7fb6d73cf640 1 -- 172.21.15.16:0/3051070309 wait complete. 2024-09-19T07:55:14.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.271+0000 7fb6d73cf640 1 Processor -- start 2024-09-19T07:55:14.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.271+0000 7fb6d73cf640 1 -- start start 2024-09-19T07:55:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.271+0000 7fb6d73cf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6d006d260 0x7fb6d0155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.271+0000 7fb6d73cf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6d00c8d20 0x7fb6d00b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.271+0000 7fb6d73cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d0143f30 0x7fb6d0158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.271+0000 7fb6d73cf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d0141bb0 con 0x7fb6d0143f30 2024-09-19T07:55:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.271+0000 7fb6d73cf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d006d6a0 con 0x7fb6d00c8d20 2024-09-19T07:55:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.271+0000 7fb6d73cf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d0141e90 con 0x7fb6d006d260 2024-09-19T07:55:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.271+0000 7fb6d596c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d0143f30 0x7fb6d0158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.271+0000 7fb6d596c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d0143f30 0x7fb6d0158470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40392/0 (socket says 172.21.15.16:40392) 2024-09-19T07:55:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.271+0000 7fb6d596c640 1 -- 172.21.15.16:0/3458928358 learned_addr learned my addr 172.21.15.16:0/3458928358 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.272+0000 7fb6d516b640 1 --2- 172.21.15.16:0/3458928358 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6d006d260 0x7fb6d0155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.272+0000 7fb6d496a640 1 --2- 172.21.15.16:0/3458928358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6d00c8d20 0x7fb6d00b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.272+0000 7fb6d596c640 1 -- 172.21.15.16:0/3458928358 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6d006d260 msgr2=0x7fb6d0155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.272+0000 7fb6d596c640 1 --2- 172.21.15.16:0/3458928358 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6d006d260 0x7fb6d0155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.272+0000 7fb6d596c640 1 -- 172.21.15.16:0/3458928358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6d00c8d20 msgr2=0x7fb6d00b7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.272+0000 7fb6d596c640 1 --2- 172.21.15.16:0/3458928358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6d00c8d20 0x7fb6d00b7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.272+0000 7fb6d596c640 1 -- 172.21.15.16:0/3458928358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d00b67c0 con 0x7fb6d0143f30 2024-09-19T07:55:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.272+0000 7fb6d516b640 1 --2- 172.21.15.16:0/3458928358 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6d006d260 0x7fb6d0155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.272+0000 7fb6d496a640 1 --2- 172.21.15.16:0/3458928358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6d00c8d20 0x7fb6d00b7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.272+0000 7fb6d596c640 1 --2- 172.21.15.16:0/3458928358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d0143f30 0x7fb6d0158470 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000ca30 tx=0x7fb6c000cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.272+0000 7fb6cdffb640 1 -- 172.21.15.16:0/3458928358 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0017070 con 0x7fb6d0143f30 2024-09-19T07:55:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.272+0000 7fb6cdffb640 1 -- 172.21.15.16:0/3458928358 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c000abd0 con 0x7fb6d0143f30 2024-09-19T07:55:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.272+0000 7fb6d73cf640 1 -- 172.21.15.16:0/3458928358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d00b6aa0 con 0x7fb6d0143f30 2024-09-19T07:55:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.272+0000 7fb6cdffb640 1 -- 172.21.15.16:0/3458928358 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0012400 con 0x7fb6d0143f30 2024-09-19T07:55:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.272+0000 7fb6d73cf640 1 -- 172.21.15.16:0/3458928358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d00b7060 con 0x7fb6d0143f30 2024-09-19T07:55:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.273+0000 7fb6d73cf640 1 -- 172.21.15.16:0/3458928358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d0144360 con 0x7fb6d0143f30 2024-09-19T07:55:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.274+0000 7fb6cdffb640 1 -- 172.21.15.16:0/3458928358 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb6c001a070 con 0x7fb6d0143f30 2024-09-19T07:55:14.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.274+0000 7fb6cdffb640 1 --2- 172.21.15.16:0/3458928358 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb6bc06c2e0 0x7fb6bc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:14.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.275+0000 7fb6cdffb640 1 -- 172.21.15.16:0/3458928358 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7fb6c008b270 con 0x7fb6d0143f30 2024-09-19T07:55:14.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.275+0000 7fb6d516b640 1 --2- 172.21.15.16:0/3458928358 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb6bc06c2e0 0x7fb6bc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:14.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.275+0000 7fb6d516b640 1 --2- 172.21.15.16:0/3458928358 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb6bc06c2e0 0x7fb6bc06e7a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8034000 tx=0x7fb6c802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.279+0000 7fb6cdffb640 1 -- 172.21.15.16:0/3458928358 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb6d00c7140 con 0x7fb6d0143f30 2024-09-19T07:55:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.489+0000 7fb6d73cf640 1 -- 172.21.15.16:0/3458928358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb6d0072290 con 0x7fb6d0143f30 2024-09-19T07:55:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.489+0000 7fb6cdffb640 1 -- 172.21.15.16:0/3458928358 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) ==== 53+0+73 (secure 0 0 0) 0x7fb6c00598e0 con 0x7fb6d0143f30 2024-09-19T07:55:14.490 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:55:14.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.491+0000 7fb6d73cf640 1 -- 172.21.15.16:0/3458928358 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb6bc06c2e0 msgr2=0x7fb6bc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:14.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.491+0000 7fb6d73cf640 1 --2- 172.21.15.16:0/3458928358 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb6bc06c2e0 0x7fb6bc06e7a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8034000 tx=0x7fb6c802e000 comp rx=0 tx=0).stop 2024-09-19T07:55:14.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.491+0000 7fb6d73cf640 1 -- 172.21.15.16:0/3458928358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d0143f30 msgr2=0x7fb6d0158470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:14.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.491+0000 7fb6d73cf640 1 --2- 172.21.15.16:0/3458928358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d0143f30 0x7fb6d0158470 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000ca30 tx=0x7fb6c000cef0 comp rx=0 tx=0).stop 2024-09-19T07:55:14.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.491+0000 7fb6d73cf640 1 -- 172.21.15.16:0/3458928358 shutdown_connections 2024-09-19T07:55:14.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.491+0000 7fb6d73cf640 1 --2- 172.21.15.16:0/3458928358 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb6bc06c2e0 0x7fb6bc06e7a0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:14.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.491+0000 7fb6d73cf640 1 --2- 172.21.15.16:0/3458928358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d0143f30 0x7fb6d0158470 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:14.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.491+0000 7fb6d73cf640 1 --2- 172.21.15.16:0/3458928358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6d00c8d20 0x7fb6d00b7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:14.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.491+0000 7fb6d73cf640 1 --2- 172.21.15.16:0/3458928358 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6d006d260 0x7fb6d0155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:14.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.491+0000 7fb6d73cf640 1 -- 172.21.15.16:0/3458928358 >> 172.21.15.16:0/3458928358 conn(0x7fb6d00b3ed0 msgr2=0x7fb6d00d1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:14.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.491+0000 7fb6d73cf640 1 -- 172.21.15.16:0/3458928358 shutdown_connections 2024-09-19T07:55:14.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:14.491+0000 7fb6d73cf640 1 -- 172.21.15.16:0/3458928358 wait complete. 2024-09-19T07:55:14.502 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:55:14.503 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:55:14.503 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-09-19T07:55:14.503 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T07:55:14.503 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T07:55:14.503 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:55:14.503 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:55:14.503 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:55:14.503 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:55:14.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:14.504 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-09-19T07:55:14.523 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:55:14.524 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:55:14.524 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:55:14.524 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:14.524 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-09-19T07:55:14.524 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:14.524 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:55:14.524 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:14.524 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2024-09-19T07:55:14.524 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:55:14.524 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:14.524 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2024-09-19T07:55:14.525 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:55:14.525 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:14.525 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:55:14.525 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:14.525 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T07:55:14.525 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42042sec preferred_lft 42042sec 2024-09-19T07:55:14.525 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link noprefixroute 2024-09-19T07:55:14.525 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:14.525 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:55:14.525 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:14.525 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:55:14.525 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:14.526 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::e47f:32ff:fe68:31ca/64 scope link 2024-09-19T07:55:14.526 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:14.526 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:55:14.526 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether f6:1f:22:44:9d:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:55:14.526 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::f41f:22ff:fe44:9dcd/64 scope link 2024-09-19T07:55:14.526 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:14.526 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:55:14.526 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:55:14.526 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::8459:eeff:fe21:a113/64 scope link 2024-09-19T07:55:14.526 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:14.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:14.527 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:55:14.592 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-09-19T07:55:14.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:14.594 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-09-19T07:55:14.654 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-09-19T07:55:14.655 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:55:14.655 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T07:55:14.655 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T07:55:14.655 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:55:14.655 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:55:14.656 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-19T07:55:14.656 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T07:55:14.656 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T07:55:14.656 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T07:55:14.656 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-19T07:55:14.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:55:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:55:14.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:55:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:55:14.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:14.788 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-09-19T07:55:14.804 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T07:55:14.804 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T07:55:14.804 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:55:14.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:14.805 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2024-09-19T07:55:14.871 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T07:55:14.871 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:55:14.871 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T07:55:14.927 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:55:14.928 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:55:14.928 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:55:14.984 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:55:15.051 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:55:15.106 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:55:15.106 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:55:15.107 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:55:15.107 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:55:15.107 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:55:15.107 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:55:15.107 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:55:15.107 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:55:15.107 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:55:15.107 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:55:15.107 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:55:15.107 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:55:15.107 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:55:15.107 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:55:15.107 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:55:15.108 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:55:15.108 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:55:15.108 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:55:15.108 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:55:15.108 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:55:15.108 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:55:15.108 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:55:15.108 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:55:15.108 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:55:15.108 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:55:15.108 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:55:15.108 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:55:15.108 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:55:15.108 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:55:15.109 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:55:15.109 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:55:15.109 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:55:15.109 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:55:15.109 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:55:15.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:15.110 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:55:15.165 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:55:15.165 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T07:55:15.207 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:55:15.231 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:55:15.288 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:55:15.288 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:55:15.289 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:55:15.289 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:55:15.289 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:55:15.289 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:55:15.289 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:55:15.289 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:55:15.289 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:55:15.289 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:55:15.289 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:55:15.289 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:55:15.289 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:55:15.289 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:55:15.290 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:55:15.290 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:55:15.290 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:55:15.290 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:55:15.290 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:55:15.290 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:55:15.290 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:55:15.290 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:55:15.290 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:55:15.290 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:55:15.290 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:55:15.291 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:55:15.291 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:55:15.291 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:55:15.291 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:55:15.291 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:55:15.291 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:55:15.291 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:55:15.291 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:55:15.291 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:55:15.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:15.292 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:55:15.294 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2024-09-19T07:55:15.287+0000 7f613cd11480 -1 init, newargv = 0x55d4f1eb64b0 newargc=15 2024-09-19T07:55:15.294 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[49748]: starting ceph client 2024-09-19T07:55:15.294 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[49748]: starting fuse 2024-09-19T07:55:15.308 INFO:teuthology.orchestra.run.smithi146.stdout:42 2024-09-19T07:55:15.308 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-19T07:55:15.308 DEBUG:teuthology.orchestra.run.smithi146:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:55:15.308 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2024-09-19T07:55:15.308 DEBUG:teuthology.orchestra.run.smithi146:> import re 2024-09-19T07:55:15.308 DEBUG:teuthology.orchestra.run.smithi146:> import os 2024-09-19T07:55:15.309 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2024-09-19T07:55:15.309 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:55:15.309 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2024-09-19T07:55:15.309 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T07:55:15.309 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2024-09-19T07:55:15.309 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T07:55:15.309 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:55:15.309 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2024-09-19T07:55:15.309 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2024-09-19T07:55:15.309 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2024-09-19T07:55:15.309 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2024-09-19T07:55:15.309 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:55:15.309 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2024-09-19T07:55:15.309 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:55:15.310 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:55:15.310 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:55:15.310 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2024-09-19T07:55:15.310 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2024-09-19T07:55:15.310 DEBUG:teuthology.orchestra.run.smithi146:> return f 2024-09-19T07:55:15.310 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:55:15.310 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:55:15.310 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2024-09-19T07:55:15.310 DEBUG:teuthology.orchestra.run.smithi146:> ' 2024-09-19T07:55:15.425 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.49748.asok 2024-09-19T07:55:15.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:55:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:55:15.436 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.49748.asok 2024-09-19T07:55:15.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:15.436 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.49748.asok status 2024-09-19T07:55:15.568 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-19T07:55:15.568 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2024-09-19T07:55:15.569 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:55:15.569 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:55:15.569 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2024-09-19T07:55:15.569 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2024-09-19T07:55:15.569 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T07:55:15.569 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "49748", 2024-09-19T07:55:15.569 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2024-09-19T07:55:15.569 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:55:15.569 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2024-09-19T07:55:15.569 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2024-09-19T07:55:15.570 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 7654, 2024-09-19T07:55:15.570 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2024-09-19T07:55:15.570 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2024-09-19T07:55:15.570 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2024-09-19T07:55:15.570 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 7654 2024-09-19T07:55:15.570 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:55:15.570 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:55:15.570 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:55:15.570 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:55:15.570 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 2861814065 2024-09-19T07:55:15.570 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-19T07:55:15.570 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:55:15.570 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:55:15.570 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:55:15.571 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:55:15.571 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 2861814065 2024-09-19T07:55:15.571 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:55:15.571 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.7654 192.168.0.1:0/2861814065", 2024-09-19T07:55:15.571 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/2861814065", 2024-09-19T07:55:15.571 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2024-09-19T07:55:15.571 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 81, 2024-09-19T07:55:15.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 140, 2024-09-19T07:55:15.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:55:15.571 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2024-09-19T07:55:15.571 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2024-09-19T07:55:15.571 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-19T07:55:15.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:15.577 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2024-09-19T07:55:15.577 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T07:55:15.593 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2024-09-19T07:55:15.593 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T07:55:15.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:15.593 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:55:15.658 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-09-19T07:55:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.776+0000 7f3d2c949640 1 Processor -- start 2024-09-19T07:55:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.776+0000 7f3d2c949640 1 -- start start 2024-09-19T07:55:15.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.777+0000 7f3d2c949640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d280d0070 0x7f3d2815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:15.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.777+0000 7f3d2c949640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d280d1b10 con 0x7f3d2815f0d0 2024-09-19T07:55:15.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.777+0000 7f3d2c949640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d280cf2b0 con 0x7f3d280d4130 2024-09-19T07:55:15.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.777+0000 7f3d2c949640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3d280cf590 con 0x7f3d280d0070 2024-09-19T07:55:15.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.777+0000 7f3d2659c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3d2815f0d0 0x7f3d280d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41278/0 (socket says 172.21.15.16:41278) 2024-09-19T07:55:15.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.777+0000 7f3d2659c640 1 -- 172.21.15.16:0/509149010 learned_addr learned my addr 172.21.15.16:0/509149010 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:15.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.777+0000 7f3d25d9b640 1 --2- 172.21.15.16:0/509149010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d280d0070 0x7f3d2815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:15.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.778+0000 7f3d2559a640 1 -- 172.21.15.16:0/509149010 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4214523683 0 0) 0x7f3d280d1b10 con 0x7f3d2815f0d0 2024-09-19T07:55:15.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.778+0000 7f3d2559a640 1 -- 172.21.15.16:0/509149010 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d10003520 con 0x7f3d2815f0d0 2024-09-19T07:55:15.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.778+0000 7f3d2559a640 1 -- 172.21.15.16:0/509149010 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3350005985 0 0) 0x7f3d280cf2b0 con 0x7f3d280d4130 2024-09-19T07:55:15.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.778+0000 7f3d2559a640 1 -- 172.21.15.16:0/509149010 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d280d1b10 con 0x7f3d280d4130 2024-09-19T07:55:15.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.778+0000 7f3d25d9b640 1 -- 172.21.15.16:0/509149010 >> v1:172.21.15.146:6789/0 conn(0x7f3d280d4130 legacy=0x7f3d280d45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:15.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.778+0000 7f3d25d9b640 1 -- 172.21.15.16:0/509149010 >> v1:172.21.15.16:6789/0 conn(0x7f3d2815f0d0 legacy=0x7f3d280d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:15.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.778+0000 7f3d25d9b640 1 -- 172.21.15.16:0/509149010 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d280cf870 con 0x7f3d280d0070 2024-09-19T07:55:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.779+0000 7f3d25d9b640 1 --2- 172.21.15.16:0/509149010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d280d0070 0x7f3d2815f4b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0097d0 tx=0x7f3d1c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=914483470cfda813 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.779+0000 7f3d2559a640 1 -- 172.21.15.16:0/509149010 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c037070 con 0x7f3d280d0070 2024-09-19T07:55:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.779+0000 7f3d2559a640 1 -- 172.21.15.16:0/509149010 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d1c032440 con 0x7f3d280d0070 2024-09-19T07:55:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.779+0000 7f3d2559a640 1 -- 172.21.15.16:0/509149010 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c032810 con 0x7f3d280d0070 2024-09-19T07:55:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.779+0000 7f3d2c949640 1 -- 172.21.15.16:0/509149010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d280d0070 msgr2=0x7f3d2815f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.779+0000 7f3d2c949640 1 --2- 172.21.15.16:0/509149010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d280d0070 0x7f3d2815f4b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0097d0 tx=0x7f3d1c02a180 comp rx=0 tx=0).stop 2024-09-19T07:55:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.780+0000 7f3d2c949640 1 -- 172.21.15.16:0/509149010 shutdown_connections 2024-09-19T07:55:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.780+0000 7f3d2c949640 1 --2- 172.21.15.16:0/509149010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d280d0070 0x7f3d2815f4b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.780+0000 7f3d2c949640 1 -- 172.21.15.16:0/509149010 >> 172.21.15.16:0/509149010 conn(0x7f3d2809d760 msgr2=0x7f3d2808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.780+0000 7f3d2c949640 1 -- 172.21.15.16:0/509149010 shutdown_connections 2024-09-19T07:55:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.780+0000 7f3d2c949640 1 -- 172.21.15.16:0/509149010 wait complete. 2024-09-19T07:55:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.781+0000 7f3d2c949640 1 Processor -- start 2024-09-19T07:55:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.781+0000 7f3d2c949640 1 -- start start 2024-09-19T07:55:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.782+0000 7f3d2c949640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d280d0070 0x7f3d2815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.782+0000 7f3d2c949640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d280d4130 0x7f3d2815a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.782+0000 7f3d2c949640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d2815f0d0 0x7f3d280cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.782+0000 7f3d2c949640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3d280b2f70 con 0x7f3d2815f0d0 2024-09-19T07:55:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.782+0000 7f3d2c949640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3d280cef60 con 0x7f3d280d0070 2024-09-19T07:55:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.782+0000 7f3d2c949640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3d280b30f0 con 0x7f3d280d4130 2024-09-19T07:55:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.782+0000 7f3d26d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d2815f0d0 0x7f3d280cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.782+0000 7f3d26d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d2815f0d0 0x7f3d280cdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40394/0 (socket says 172.21.15.16:40394) 2024-09-19T07:55:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.782+0000 7f3d26d9d640 1 -- 172.21.15.16:0/2507955289 learned_addr learned my addr 172.21.15.16:0/2507955289 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.782+0000 7f3d2659c640 1 --2- 172.21.15.16:0/2507955289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d280d0070 0x7f3d2815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.782+0000 7f3d25d9b640 1 --2- 172.21.15.16:0/2507955289 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d280d4130 0x7f3d2815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.782+0000 7f3d26d9d640 1 -- 172.21.15.16:0/2507955289 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d280d4130 msgr2=0x7f3d2815a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.782+0000 7f3d26d9d640 1 --2- 172.21.15.16:0/2507955289 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d280d4130 0x7f3d2815a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.782+0000 7f3d26d9d640 1 -- 172.21.15.16:0/2507955289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d280d0070 msgr2=0x7f3d2815b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.782+0000 7f3d26d9d640 1 --2- 172.21.15.16:0/2507955289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d280d0070 0x7f3d2815b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.782+0000 7f3d26d9d640 1 -- 172.21.15.16:0/2507955289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2815a7a0 con 0x7f3d2815f0d0 2024-09-19T07:55:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.782+0000 7f3d25d9b640 1 --2- 172.21.15.16:0/2507955289 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d280d4130 0x7f3d2815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.782+0000 7f3d2659c640 1 --2- 172.21.15.16:0/2507955289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d280d0070 0x7f3d2815b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.782+0000 7f3d26d9d640 1 --2- 172.21.15.16:0/2507955289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d2815f0d0 0x7f3d280cdc80 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000c430 tx=0x7f3d2000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.783+0000 7f3d16ffd640 1 -- 172.21.15.16:0/2507955289 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20017070 con 0x7f3d2815f0d0 2024-09-19T07:55:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.783+0000 7f3d16ffd640 1 -- 172.21.15.16:0/2507955289 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d200052d0 con 0x7f3d2815f0d0 2024-09-19T07:55:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.783+0000 7f3d2c949640 1 -- 172.21.15.16:0/2507955289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d28161110 con 0x7f3d2815f0d0 2024-09-19T07:55:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.783+0000 7f3d16ffd640 1 -- 172.21.15.16:0/2507955289 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20003830 con 0x7f3d2815f0d0 2024-09-19T07:55:15.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.783+0000 7f3d2c949640 1 -- 172.21.15.16:0/2507955289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d281616d0 con 0x7f3d2815f0d0 2024-09-19T07:55:15.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.784+0000 7f3d2c949640 1 -- 172.21.15.16:0/2507955289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d2806cf30 con 0x7f3d2815f0d0 2024-09-19T07:55:15.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.784+0000 7f3d16ffd640 1 -- 172.21.15.16:0/2507955289 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3d2001a070 con 0x7f3d2815f0d0 2024-09-19T07:55:15.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.785+0000 7f3d16ffd640 1 --2- 172.21.15.16:0/2507955289 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3d0806c2b0 0x7f3d0806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:15.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.785+0000 7f3d16ffd640 1 -- 172.21.15.16:0/2507955289 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f3d2008a920 con 0x7f3d2815f0d0 2024-09-19T07:55:15.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.785+0000 7f3d2659c640 1 --2- 172.21.15.16:0/2507955289 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3d0806c2b0 0x7f3d0806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:15.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.786+0000 7f3d2659c640 1 --2- 172.21.15.16:0/2507955289 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3d0806c2b0 0x7f3d0806e770 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3d18001560 tx=0x7f3d18009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:15.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.789+0000 7f3d16ffd640 1 -- 172.21.15.16:0/2507955289 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3d20058f90 con 0x7f3d2815f0d0 2024-09-19T07:55:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.965+0000 7f3d2c949640 1 -- 172.21.15.16:0/2507955289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7f3d280d3820 con 0x7f3d2815f0d0 2024-09-19T07:55:15.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.968+0000 7f3d16ffd640 1 -- 172.21.15.16:0/2507955289 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v64)=0 v64) ==== 121+0+0 (secure 0 0 0) 0x7f3d20058890 con 0x7f3d2815f0d0 2024-09-19T07:55:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.970+0000 7f3d2c949640 1 -- 172.21.15.16:0/2507955289 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3d0806c2b0 msgr2=0x7f3d0806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.970+0000 7f3d2c949640 1 --2- 172.21.15.16:0/2507955289 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3d0806c2b0 0x7f3d0806e770 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3d18001560 tx=0x7f3d18009040 comp rx=0 tx=0).stop 2024-09-19T07:55:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.970+0000 7f3d2c949640 1 -- 172.21.15.16:0/2507955289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d2815f0d0 msgr2=0x7f3d280cdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.970+0000 7f3d2c949640 1 --2- 172.21.15.16:0/2507955289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d2815f0d0 0x7f3d280cdc80 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000c430 tx=0x7f3d2000c8f0 comp rx=0 tx=0).stop 2024-09-19T07:55:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.970+0000 7f3d2c949640 1 -- 172.21.15.16:0/2507955289 shutdown_connections 2024-09-19T07:55:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.970+0000 7f3d2c949640 1 --2- 172.21.15.16:0/2507955289 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3d0806c2b0 0x7f3d0806e770 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.970+0000 7f3d2c949640 1 --2- 172.21.15.16:0/2507955289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d2815f0d0 0x7f3d280cdc80 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.970+0000 7f3d2c949640 1 --2- 172.21.15.16:0/2507955289 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d280d4130 0x7f3d2815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.970+0000 7f3d2c949640 1 --2- 172.21.15.16:0/2507955289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d280d0070 0x7f3d2815b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.970+0000 7f3d2c949640 1 -- 172.21.15.16:0/2507955289 >> 172.21.15.16:0/2507955289 conn(0x7f3d2809d760 msgr2=0x7f3d281626d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.970+0000 7f3d2c949640 1 -- 172.21.15.16:0/2507955289 shutdown_connections 2024-09-19T07:55:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:15.970+0000 7f3d2c949640 1 -- 172.21.15.16:0/2507955289 wait complete. 2024-09-19T07:55:15.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 200 2024-09-19T07:55:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.087+0000 7ff049df1640 1 Processor -- start 2024-09-19T07:55:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.088+0000 7ff049df1640 1 -- start start 2024-09-19T07:55:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.088+0000 7ff049df1640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff04415e680 0x7ff0440cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.088+0000 7ff049df1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff0440d0010 0x7ff0440d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.088+0000 7ff049df1640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0440d1880 con 0x7ff04415e2a0 2024-09-19T07:55:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.088+0000 7ff049df1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff0440d1b60 con 0x7ff0440d0010 2024-09-19T07:55:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.088+0000 7ff049df1640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff0440cf2d0 con 0x7ff04415e680 2024-09-19T07:55:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.088+0000 7ff043fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff0440d0010 0x7ff0440d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.088+0000 7ff043fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff0440d0010 0x7ff0440d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40408/0 (socket says 172.21.15.16:40408) 2024-09-19T07:55:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.088+0000 7ff043fff640 1 -- 172.21.15.16:0/2560736912 learned_addr learned my addr 172.21.15.16:0/2560736912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.089+0000 7ff042ffd640 1 --2- 172.21.15.16:0/2560736912 >> v2:172.21.15.146:3300/0 conn(0x7ff04415e680 0x7ff0440cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.089+0000 7ff043fff640 1 -- 172.21.15.16:0/2560736912 >> v2:172.21.15.146:3300/0 conn(0x7ff04415e680 msgr2=0x7ff0440cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.089+0000 7ff043fff640 1 --2- 172.21.15.16:0/2560736912 >> v2:172.21.15.146:3300/0 conn(0x7ff04415e680 0x7ff0440cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.089+0000 7ff043fff640 1 -- 172.21.15.16:0/2560736912 >> v1:172.21.15.146:6789/0 conn(0x7ff04415e2a0 legacy=0x7ff04406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.089+0000 7ff043fff640 1 -- 172.21.15.16:0/2560736912 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0440cf5b0 con 0x7ff0440d0010 2024-09-19T07:55:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.089+0000 7ff043fff640 1 --2- 172.21.15.16:0/2560736912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0440d0010 0x7ff0440d4370 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7ff0440a22a0 tx=0x7ff03c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae18e6ab113eee33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.090+0000 7ff0427fc640 1 -- 172.21.15.16:0/2560736912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c036070 con 0x7ff0440d0010 2024-09-19T07:55:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.090+0000 7ff0427fc640 1 -- 172.21.15.16:0/2560736912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff03c031600 con 0x7ff0440d0010 2024-09-19T07:55:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.090+0000 7ff0427fc640 1 -- 172.21.15.16:0/2560736912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c0319d0 con 0x7ff0440d0010 2024-09-19T07:55:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.090+0000 7ff049df1640 1 -- 172.21.15.16:0/2560736912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0440d0010 msgr2=0x7ff0440d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.090+0000 7ff049df1640 1 --2- 172.21.15.16:0/2560736912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0440d0010 0x7ff0440d4370 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7ff0440a22a0 tx=0x7ff03c02a6a0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.090+0000 7ff049df1640 1 -- 172.21.15.16:0/2560736912 shutdown_connections 2024-09-19T07:55:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.090+0000 7ff049df1640 1 --2- 172.21.15.16:0/2560736912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0440d0010 0x7ff0440d4370 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.090+0000 7ff049df1640 1 --2- 172.21.15.16:0/2560736912 >> v2:172.21.15.146:3300/0 conn(0x7ff04415e680 0x7ff0440cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.090+0000 7ff049df1640 1 -- 172.21.15.16:0/2560736912 >> 172.21.15.16:0/2560736912 conn(0x7ff04409d760 msgr2=0x7ff04408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.090+0000 7ff049df1640 1 -- 172.21.15.16:0/2560736912 shutdown_connections 2024-09-19T07:55:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.090+0000 7ff049df1640 1 -- 172.21.15.16:0/2560736912 wait complete. 2024-09-19T07:55:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.091+0000 7ff049df1640 1 Processor -- start 2024-09-19T07:55:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.092+0000 7ff049df1640 1 -- start start 2024-09-19T07:55:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.092+0000 7ff049df1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0440d0010 0x7ff0441528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.092+0000 7ff049df1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff04415e2a0 0x7ff0440c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.092+0000 7ff049df1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04415e680 0x7ff04415fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.092+0000 7ff049df1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff04406dcb0 con 0x7ff04415e680 2024-09-19T07:55:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.092+0000 7ff049df1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff04406db30 con 0x7ff04415e2a0 2024-09-19T07:55:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.092+0000 7ff049df1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff0440d48b0 con 0x7ff0440d0010 2024-09-19T07:55:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.092+0000 7ff043fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04415e680 0x7ff04415fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.093+0000 7ff043fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04415e680 0x7ff04415fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40414/0 (socket says 172.21.15.16:40414) 2024-09-19T07:55:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.093+0000 7ff043fff640 1 -- 172.21.15.16:0/1868310648 learned_addr learned my addr 172.21.15.16:0/1868310648 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.093+0000 7ff042ffd640 1 --2- 172.21.15.16:0/1868310648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff04415e2a0 0x7ff0440c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.093+0000 7ff0437fe640 1 --2- 172.21.15.16:0/1868310648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0440d0010 0x7ff0441528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.093+0000 7ff043fff640 1 -- 172.21.15.16:0/1868310648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0440d0010 msgr2=0x7ff0441528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.093+0000 7ff043fff640 1 --2- 172.21.15.16:0/1868310648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0440d0010 0x7ff0441528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.093+0000 7ff043fff640 1 -- 172.21.15.16:0/1868310648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff04415e2a0 msgr2=0x7ff0440c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.093+0000 7ff043fff640 1 --2- 172.21.15.16:0/1868310648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff04415e2a0 0x7ff0440c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.093+0000 7ff043fff640 1 -- 172.21.15.16:0/1868310648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0440ca510 con 0x7ff04415e680 2024-09-19T07:55:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.093+0000 7ff043fff640 1 --2- 172.21.15.16:0/1868310648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04415e680 0x7ff04415fed0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7ff03c034040 tx=0x7ff03c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.093+0000 7ff02bfff640 1 -- 172.21.15.16:0/1868310648 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c036050 con 0x7ff04415e680 2024-09-19T07:55:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.093+0000 7ff02bfff640 1 -- 172.21.15.16:0/1868310648 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff03c042070 con 0x7ff04415e680 2024-09-19T07:55:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.093+0000 7ff02bfff640 1 -- 172.21.15.16:0/1868310648 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c009510 con 0x7ff04415e680 2024-09-19T07:55:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.093+0000 7ff049df1640 1 -- 172.21.15.16:0/1868310648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0440ca7f0 con 0x7ff04415e680 2024-09-19T07:55:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.093+0000 7ff049df1640 1 -- 172.21.15.16:0/1868310648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0440cadb0 con 0x7ff04415e680 2024-09-19T07:55:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.094+0000 7ff049df1640 1 -- 172.21.15.16:0/1868310648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0440d4c70 con 0x7ff04415e680 2024-09-19T07:55:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.095+0000 7ff02bfff640 1 -- 172.21.15.16:0/1868310648 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff03c007bd0 con 0x7ff04415e680 2024-09-19T07:55:16.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.095+0000 7ff02bfff640 1 --2- 172.21.15.16:0/1868310648 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff03006c2b0 0x7ff03006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:16.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.095+0000 7ff02bfff640 1 -- 172.21.15.16:0/1868310648 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7ff03c0ac2f0 con 0x7ff04415e680 2024-09-19T07:55:16.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.095+0000 7ff0437fe640 1 --2- 172.21.15.16:0/1868310648 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff03006c2b0 0x7ff03006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:16.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.096+0000 7ff0437fe640 1 --2- 172.21.15.16:0/1868310648 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff03006c2b0 0x7ff03006e770 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff044160ac0 tx=0x7ff034009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.100+0000 7ff02bfff640 1 -- 172.21.15.16:0/1868310648 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff03c07a290 con 0x7ff04415e680 2024-09-19T07:55:16.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.274+0000 7ff049df1640 1 -- 172.21.15.16:0/1868310648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7ff04406cf90 con 0x7ff04415e680 2024-09-19T07:55:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.277+0000 7ff02bfff640 1 -- 172.21.15.16:0/1868310648 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v65)=0 v65) ==== 119+0+0 (secure 0 0 0) 0x7ff03c07e590 con 0x7ff04415e680 2024-09-19T07:55:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.279+0000 7ff049df1640 1 -- 172.21.15.16:0/1868310648 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff03006c2b0 msgr2=0x7ff03006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.279+0000 7ff049df1640 1 --2- 172.21.15.16:0/1868310648 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff03006c2b0 0x7ff03006e770 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff044160ac0 tx=0x7ff034009040 comp rx=0 tx=0).stop 2024-09-19T07:55:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.279+0000 7ff049df1640 1 -- 172.21.15.16:0/1868310648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04415e680 msgr2=0x7ff04415fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.279+0000 7ff049df1640 1 --2- 172.21.15.16:0/1868310648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04415e680 0x7ff04415fed0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7ff03c034040 tx=0x7ff03c004a40 comp rx=0 tx=0).stop 2024-09-19T07:55:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.279+0000 7ff049df1640 1 -- 172.21.15.16:0/1868310648 shutdown_connections 2024-09-19T07:55:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.279+0000 7ff049df1640 1 --2- 172.21.15.16:0/1868310648 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff03006c2b0 0x7ff03006e770 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.279+0000 7ff049df1640 1 --2- 172.21.15.16:0/1868310648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04415e680 0x7ff04415fed0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.279+0000 7ff049df1640 1 --2- 172.21.15.16:0/1868310648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff04415e2a0 0x7ff0440c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.279+0000 7ff049df1640 1 --2- 172.21.15.16:0/1868310648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0440d0010 0x7ff0441528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.279+0000 7ff049df1640 1 -- 172.21.15.16:0/1868310648 >> 172.21.15.16:0/1868310648 conn(0x7ff04409d760 msgr2=0x7ff044161020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.279+0000 7ff049df1640 1 -- 172.21.15.16:0/1868310648 shutdown_connections 2024-09-19T07:55:16.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.279+0000 7ff049df1640 1 -- 172.21.15.16:0/1868310648 wait complete. 2024-09-19T07:55:16.290 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 400 2024-09-19T07:55:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.395+0000 7f01d53fd640 1 Processor -- start 2024-09-19T07:55:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.395+0000 7f01d53fd640 1 -- start start 2024-09-19T07:55:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.396+0000 7f01d53fd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f01d015c280 0x7f01d00d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.396+0000 7f01d53fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01d01608f0 0x7f01d006bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.396+0000 7f01d53fd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f01d006c150 0x7f01d00cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.396+0000 7f01d53fd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f01d00b4010 con 0x7f01d015c280 2024-09-19T07:55:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.396+0000 7f01d53fd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f01d00b45d0 con 0x7f01d006c150 2024-09-19T07:55:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.396+0000 7f01d53fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f01d006c580 con 0x7f01d01608f0 2024-09-19T07:55:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.396+0000 7f01ceffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f01d015c280 0x7f01d00d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:16.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.396+0000 7f01ceffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f01d015c280 0x7f01d00d0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40416/0 (socket says 172.21.15.16:40416) 2024-09-19T07:55:16.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.396+0000 7f01ceffd640 1 -- 172.21.15.16:0/2983808083 learned_addr learned my addr 172.21.15.16:0/2983808083 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:16.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.397+0000 7f01ce7fc640 1 --2- 172.21.15.16:0/2983808083 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01d01608f0 0x7f01d006bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:16.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.397+0000 7f01cf7fe640 1 --2- 172.21.15.16:0/2983808083 >> v2:172.21.15.146:3300/0 conn(0x7f01d006c150 0x7f01d00cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:16.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.397+0000 7f01ceffd640 1 -- 172.21.15.16:0/2983808083 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01d01608f0 msgr2=0x7f01d006bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:16.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.397+0000 7f01ceffd640 1 --2- 172.21.15.16:0/2983808083 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01d01608f0 0x7f01d006bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.397+0000 7f01ceffd640 1 -- 172.21.15.16:0/2983808083 >> v2:172.21.15.146:3300/0 conn(0x7f01d006c150 msgr2=0x7f01d00cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:16.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.397+0000 7f01ceffd640 1 --2- 172.21.15.16:0/2983808083 >> v2:172.21.15.146:3300/0 conn(0x7f01d006c150 0x7f01d00cdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.397+0000 7f01ceffd640 1 -- 172.21.15.16:0/2983808083 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d00d0a30 con 0x7f01d015c280 2024-09-19T07:55:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.397+0000 7f01ceffd640 1 --2- 172.21.15.16:0/2983808083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d015c280 0x7f01d00d0350 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f01d00c81e0 tx=0x7f01c002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b5502d4c728bc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.398+0000 7f01cdffb640 1 -- 172.21.15.16:0/2983808083 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c0036070 con 0x7f01d015c280 2024-09-19T07:55:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.398+0000 7f01cdffb640 1 -- 172.21.15.16:0/2983808083 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c0031600 con 0x7f01d015c280 2024-09-19T07:55:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.398+0000 7f01cdffb640 1 -- 172.21.15.16:0/2983808083 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c00319d0 con 0x7f01d015c280 2024-09-19T07:55:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.398+0000 7f01d53fd640 1 -- 172.21.15.16:0/2983808083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d015c280 msgr2=0x7f01d00d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.398+0000 7f01d53fd640 1 --2- 172.21.15.16:0/2983808083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d015c280 0x7f01d00d0350 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f01d00c81e0 tx=0x7f01c002a6a0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.398+0000 7f01d53fd640 1 -- 172.21.15.16:0/2983808083 shutdown_connections 2024-09-19T07:55:16.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.398+0000 7f01d53fd640 1 --2- 172.21.15.16:0/2983808083 >> v2:172.21.15.146:3300/0 conn(0x7f01d006c150 0x7f01d00cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.398+0000 7f01d53fd640 1 --2- 172.21.15.16:0/2983808083 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01d01608f0 0x7f01d006bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.398+0000 7f01d53fd640 1 --2- 172.21.15.16:0/2983808083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d015c280 0x7f01d00d0350 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.398+0000 7f01d53fd640 1 -- 172.21.15.16:0/2983808083 >> 172.21.15.16:0/2983808083 conn(0x7f01d00b2d10 msgr2=0x7f01d00b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:16.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.398+0000 7f01d53fd640 1 -- 172.21.15.16:0/2983808083 shutdown_connections 2024-09-19T07:55:16.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.398+0000 7f01d53fd640 1 -- 172.21.15.16:0/2983808083 wait complete. 2024-09-19T07:55:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.400+0000 7f01d53fd640 1 Processor -- start 2024-09-19T07:55:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.400+0000 7f01d53fd640 1 -- start start 2024-09-19T07:55:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.400+0000 7f01d53fd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01d006c150 0x7f01d015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.400+0000 7f01d53fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01d015c280 0x7f01d00c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.400+0000 7f01d53fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d01608f0 0x7f01d00c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.400+0000 7f01d53fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01d0160d10 con 0x7f01d01608f0 2024-09-19T07:55:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.400+0000 7f01d53fd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f01d00b4260 con 0x7f01d006c150 2024-09-19T07:55:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.400+0000 7f01d53fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f01d0160e90 con 0x7f01d015c280 2024-09-19T07:55:16.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.400+0000 7f01cf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d01608f0 0x7f01d00c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:16.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.400+0000 7f01cf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d01608f0 0x7f01d00c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40418/0 (socket says 172.21.15.16:40418) 2024-09-19T07:55:16.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.401+0000 7f01cf7fe640 1 -- 172.21.15.16:0/1265468335 learned_addr learned my addr 172.21.15.16:0/1265468335 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:16.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.401+0000 7f01ce7fc640 1 --2- 172.21.15.16:0/1265468335 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01d015c280 0x7f01d00c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:16.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.401+0000 7f01ceffd640 1 --2- 172.21.15.16:0/1265468335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01d006c150 0x7f01d015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:16.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.401+0000 7f01cf7fe640 1 -- 172.21.15.16:0/1265468335 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01d015c280 msgr2=0x7f01d00c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:16.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.401+0000 7f01cf7fe640 1 --2- 172.21.15.16:0/1265468335 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01d015c280 0x7f01d00c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.401+0000 7f01cf7fe640 1 -- 172.21.15.16:0/1265468335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01d006c150 msgr2=0x7f01d015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:16.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.401+0000 7f01cf7fe640 1 --2- 172.21.15.16:0/1265468335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01d006c150 0x7f01d015b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.401+0000 7f01cf7fe640 1 -- 172.21.15.16:0/1265468335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d00c5c30 con 0x7f01d01608f0 2024-09-19T07:55:16.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.401+0000 7f01cf7fe640 1 --2- 172.21.15.16:0/1265468335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d01608f0 0x7f01d00c97e0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f01c800d800 tx=0x7f01c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:16.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.401+0000 7f01b77fe640 1 -- 172.21.15.16:0/1265468335 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8018020 con 0x7f01d01608f0 2024-09-19T07:55:16.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.401+0000 7f01b77fe640 1 -- 172.21.15.16:0/1265468335 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c80096c0 con 0x7f01d01608f0 2024-09-19T07:55:16.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.401+0000 7f01d53fd640 1 -- 172.21.15.16:0/1265468335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d00c5f10 con 0x7f01d01608f0 2024-09-19T07:55:16.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.401+0000 7f01b77fe640 1 -- 172.21.15.16:0/1265468335 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8009a90 con 0x7f01d01608f0 2024-09-19T07:55:16.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.401+0000 7f01d53fd640 1 -- 172.21.15.16:0/1265468335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d00c64d0 con 0x7f01d01608f0 2024-09-19T07:55:16.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.402+0000 7f01d53fd640 1 -- 172.21.15.16:0/1265468335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d006c8b0 con 0x7f01d01608f0 2024-09-19T07:55:16.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.403+0000 7f01b77fe640 1 -- 172.21.15.16:0/1265468335 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f01c801b020 con 0x7f01d01608f0 2024-09-19T07:55:16.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.403+0000 7f01b77fe640 1 --2- 172.21.15.16:0/1265468335 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01bc06c2b0 0x7f01bc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:16.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.403+0000 7f01b77fe640 1 -- 172.21.15.16:0/1265468335 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f01c808b950 con 0x7f01d01608f0 2024-09-19T07:55:16.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.404+0000 7f01ceffd640 1 --2- 172.21.15.16:0/1265468335 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01bc06c2b0 0x7f01bc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:16.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.404+0000 7f01ceffd640 1 --2- 172.21.15.16:0/1265468335 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01bc06c2b0 0x7f01bc06e770 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f01c0033000 tx=0x7f01c0009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:16.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.408+0000 7f01b77fe640 1 -- 172.21.15.16:0/1265468335 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f01c8022020 con 0x7f01d01608f0 2024-09-19T07:55:16.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.582+0000 7f01d53fd640 1 -- 172.21.15.16:0/1265468335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7f01d015bc00 con 0x7f01d01608f0 2024-09-19T07:55:16.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.585+0000 7f01b77fe640 1 -- 172.21.15.16:0/1265468335 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v66)=0 v66) ==== 128+0+0 (secure 0 0 0) 0x7f01c8059970 con 0x7f01d01608f0 2024-09-19T07:55:16.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.587+0000 7f01d53fd640 1 -- 172.21.15.16:0/1265468335 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01bc06c2b0 msgr2=0x7f01bc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:16.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.587+0000 7f01d53fd640 1 --2- 172.21.15.16:0/1265468335 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01bc06c2b0 0x7f01bc06e770 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f01c0033000 tx=0x7f01c0009000 comp rx=0 tx=0).stop 2024-09-19T07:55:16.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.587+0000 7f01d53fd640 1 -- 172.21.15.16:0/1265468335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d01608f0 msgr2=0x7f01d00c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:16.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.587+0000 7f01d53fd640 1 --2- 172.21.15.16:0/1265468335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d01608f0 0x7f01d00c97e0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f01c800d800 tx=0x7f01c800dcc0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.587+0000 7f01d53fd640 1 -- 172.21.15.16:0/1265468335 shutdown_connections 2024-09-19T07:55:16.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.587+0000 7f01d53fd640 1 --2- 172.21.15.16:0/1265468335 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f01bc06c2b0 0x7f01bc06e770 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.587+0000 7f01d53fd640 1 --2- 172.21.15.16:0/1265468335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d01608f0 0x7f01d00c97e0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.587+0000 7f01d53fd640 1 --2- 172.21.15.16:0/1265468335 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01d015c280 0x7f01d00c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.587+0000 7f01d53fd640 1 --2- 172.21.15.16:0/1265468335 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01d006c150 0x7f01d015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.587+0000 7f01d53fd640 1 -- 172.21.15.16:0/1265468335 >> 172.21.15.16:0/1265468335 conn(0x7f01d00b2d10 msgr2=0x7f01d01e7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:16.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.587+0000 7f01d53fd640 1 -- 172.21.15.16:0/1265468335 shutdown_connections 2024-09-19T07:55:16.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.587+0000 7f01d53fd640 1 -- 172.21.15.16:0/1265468335 wait complete. 2024-09-19T07:55:16.599 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-09-19T07:55:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.702+0000 7fe2ec04c640 1 Processor -- start 2024-09-19T07:55:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.703+0000 7fe2ec04c640 1 -- start start 2024-09-19T07:55:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.703+0000 7fe2ec04c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe2e40c5b00 0x7fe2e415a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.703+0000 7fe2ec04c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2e40cc930 0x7fe2e40ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.703+0000 7fe2ec04c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2e40b4540 con 0x7fe2e40cde40 2024-09-19T07:55:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.703+0000 7fe2ec04c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fe2e40d0bc0 con 0x7fe2e40c5b00 2024-09-19T07:55:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.703+0000 7fe2ec04c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe2e40b4870 con 0x7fe2e40cc930 2024-09-19T07:55:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.704+0000 7fe2e9de8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe2e40c5b00 0x7fe2e415a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:16.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.704+0000 7fe2e9de8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe2e40c5b00 0x7fe2e415a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40434/0 (socket says 172.21.15.16:40434) 2024-09-19T07:55:16.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.704+0000 7fe2e9de8640 1 -- 172.21.15.16:0/240688808 learned_addr learned my addr 172.21.15.16:0/240688808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:16.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.704+0000 7fe2ea5e9640 1 --2- 172.21.15.16:0/240688808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2e40cc930 0x7fe2e40ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:16.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.704+0000 7fe2e9de8640 1 -- 172.21.15.16:0/240688808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2e40cc930 msgr2=0x7fe2e40ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:16.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.704+0000 7fe2e9de8640 1 --2- 172.21.15.16:0/240688808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2e40cc930 0x7fe2e40ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.704+0000 7fe2e9de8640 1 -- 172.21.15.16:0/240688808 >> v1:172.21.15.146:6789/0 conn(0x7fe2e40cde40 legacy=0x7fe2e40d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:16.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.704+0000 7fe2e9de8640 1 -- 172.21.15.16:0/240688808 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e40c5ee0 con 0x7fe2e40c5b00 2024-09-19T07:55:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.705+0000 7fe2e9de8640 1 --2- 172.21.15.16:0/240688808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e40c5b00 0x7fe2e415a7b0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009870 tx=0x7fe2e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac0c1fd5eff360f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.705+0000 7fe2e8de6640 1 -- 172.21.15.16:0/240688808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0037070 con 0x7fe2e40c5b00 2024-09-19T07:55:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.705+0000 7fe2e8de6640 1 -- 172.21.15.16:0/240688808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e0032440 con 0x7fe2e40c5b00 2024-09-19T07:55:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.705+0000 7fe2e8de6640 1 -- 172.21.15.16:0/240688808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0032810 con 0x7fe2e40c5b00 2024-09-19T07:55:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.706+0000 7fe2ec04c640 1 -- 172.21.15.16:0/240688808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e40c5b00 msgr2=0x7fe2e415a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:16.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.706+0000 7fe2ec04c640 1 --2- 172.21.15.16:0/240688808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e40c5b00 0x7fe2e415a7b0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009870 tx=0x7fe2e002a270 comp rx=0 tx=0).stop 2024-09-19T07:55:16.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.706+0000 7fe2ec04c640 1 -- 172.21.15.16:0/240688808 shutdown_connections 2024-09-19T07:55:16.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.706+0000 7fe2ec04c640 1 --2- 172.21.15.16:0/240688808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2e40cc930 0x7fe2e40ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.706+0000 7fe2ec04c640 1 --2- 172.21.15.16:0/240688808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e40c5b00 0x7fe2e415a7b0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.706+0000 7fe2ec04c640 1 -- 172.21.15.16:0/240688808 >> 172.21.15.16:0/240688808 conn(0x7fe2e406bdf0 msgr2=0x7fe2e406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:16.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.706+0000 7fe2ec04c640 1 -- 172.21.15.16:0/240688808 shutdown_connections 2024-09-19T07:55:16.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.706+0000 7fe2ec04c640 1 -- 172.21.15.16:0/240688808 wait complete. 2024-09-19T07:55:16.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.707+0000 7fe2ec04c640 1 Processor -- start 2024-09-19T07:55:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.707+0000 7fe2ec04c640 1 -- start start 2024-09-19T07:55:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.708+0000 7fe2ec04c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e40c5b00 0x7fe2e4158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.708+0000 7fe2ec04c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2e40cc930 0x7fe2e40b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.708+0000 7fe2ec04c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2e40cde40 0x7fe2e40b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.708+0000 7fe2ec04c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e40cff70 con 0x7fe2e40c5b00 2024-09-19T07:55:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.708+0000 7fe2ec04c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e40cfc90 con 0x7fe2e40cde40 2024-09-19T07:55:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.708+0000 7fe2ec04c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe2e40ca8c0 con 0x7fe2e40cc930 2024-09-19T07:55:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.708+0000 7fe2e9de8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e40c5b00 0x7fe2e4158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.708+0000 7fe2e9de8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e40c5b00 0x7fe2e4158b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40444/0 (socket says 172.21.15.16:40444) 2024-09-19T07:55:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.708+0000 7fe2e9de8640 1 -- 172.21.15.16:0/3830996067 learned_addr learned my addr 172.21.15.16:0/3830996067 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.708+0000 7fe2ea5e9640 1 --2- 172.21.15.16:0/3830996067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2e40cde40 0x7fe2e40b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.708+0000 7fe2e95e7640 1 --2- 172.21.15.16:0/3830996067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2e40cc930 0x7fe2e40b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.708+0000 7fe2e9de8640 1 -- 172.21.15.16:0/3830996067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2e40cc930 msgr2=0x7fe2e40b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.708+0000 7fe2e9de8640 1 --2- 172.21.15.16:0/3830996067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2e40cc930 0x7fe2e40b09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.708+0000 7fe2e9de8640 1 -- 172.21.15.16:0/3830996067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2e40cde40 msgr2=0x7fe2e40b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.708+0000 7fe2e9de8640 1 --2- 172.21.15.16:0/3830996067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2e40cde40 0x7fe2e40b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.708+0000 7fe2e9de8640 1 -- 172.21.15.16:0/3830996067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e40adea0 con 0x7fe2e40c5b00 2024-09-19T07:55:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.708+0000 7fe2e95e7640 1 --2- 172.21.15.16:0/3830996067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2e40cc930 0x7fe2e40b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.708+0000 7fe2e9de8640 1 --2- 172.21.15.16:0/3830996067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e40c5b00 0x7fe2e4158b20 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fe2e002a2b0 tx=0x7fe2e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.708+0000 7fe2da7fc640 1 -- 172.21.15.16:0/3830996067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0041020 con 0x7fe2e40c5b00 2024-09-19T07:55:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.709+0000 7fe2da7fc640 1 -- 172.21.15.16:0/3830996067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e00306c0 con 0x7fe2e40c5b00 2024-09-19T07:55:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.709+0000 7fe2da7fc640 1 -- 172.21.15.16:0/3830996067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0037040 con 0x7fe2e40c5b00 2024-09-19T07:55:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.709+0000 7fe2ec04c640 1 -- 172.21.15.16:0/3830996067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2e40c7590 con 0x7fe2e40c5b00 2024-09-19T07:55:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.709+0000 7fe2ec04c640 1 -- 172.21.15.16:0/3830996067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2e4159060 con 0x7fe2e40c5b00 2024-09-19T07:55:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.710+0000 7fe2ec04c640 1 -- 172.21.15.16:0/3830996067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2e40ce280 con 0x7fe2e40c5b00 2024-09-19T07:55:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.710+0000 7fe2da7fc640 1 -- 172.21.15.16:0/3830996067 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe2e0044020 con 0x7fe2e40c5b00 2024-09-19T07:55:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.711+0000 7fe2da7fc640 1 --2- 172.21.15.16:0/3830996067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe2c806c2b0 0x7fe2c806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.711+0000 7fe2da7fc640 1 -- 172.21.15.16:0/3830996067 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7fe2e00abd50 con 0x7fe2e40c5b00 2024-09-19T07:55:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.711+0000 7fe2e95e7640 1 --2- 172.21.15.16:0/3830996067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe2c806c2b0 0x7fe2c806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.711+0000 7fe2e95e7640 1 --2- 172.21.15.16:0/3830996067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe2c806c2b0 0x7fe2c806e770 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40ae830 tx=0x7fe2d4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.715+0000 7fe2da7fc640 1 -- 172.21.15.16:0/3830996067 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe2e004b030 con 0x7fe2e40c5b00 2024-09-19T07:55:16.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.893+0000 7fe2ec04c640 1 -- 172.21.15.16:0/3830996067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7fe2e40c7320 con 0x7fe2e40c5b00 2024-09-19T07:55:16.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.893+0000 7fe2da7fc640 1 -- 172.21.15.16:0/3830996067 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v66) ==== 106+0+4 (secure 0 0 0) 0x7fe2e007a320 con 0x7fe2e40c5b00 2024-09-19T07:55:16.894 INFO:teuthology.orchestra.run.smithi016.stdout:100 2024-09-19T07:55:16.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.895+0000 7fe2ec04c640 1 -- 172.21.15.16:0/3830996067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe2c806c2b0 msgr2=0x7fe2c806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:16.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.895+0000 7fe2ec04c640 1 --2- 172.21.15.16:0/3830996067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe2c806c2b0 0x7fe2c806e770 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40ae830 tx=0x7fe2d4009040 comp rx=0 tx=0).stop 2024-09-19T07:55:16.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.895+0000 7fe2ec04c640 1 -- 172.21.15.16:0/3830996067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e40c5b00 msgr2=0x7fe2e4158b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:16.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.895+0000 7fe2ec04c640 1 --2- 172.21.15.16:0/3830996067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e40c5b00 0x7fe2e4158b20 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fe2e002a2b0 tx=0x7fe2e00046c0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.895+0000 7fe2ec04c640 1 -- 172.21.15.16:0/3830996067 shutdown_connections 2024-09-19T07:55:16.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.895+0000 7fe2ec04c640 1 --2- 172.21.15.16:0/3830996067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe2c806c2b0 0x7fe2c806e770 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.895+0000 7fe2ec04c640 1 --2- 172.21.15.16:0/3830996067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2e40cde40 0x7fe2e40b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.895+0000 7fe2ec04c640 1 --2- 172.21.15.16:0/3830996067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2e40cc930 0x7fe2e40b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.895+0000 7fe2ec04c640 1 --2- 172.21.15.16:0/3830996067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2e40c5b00 0x7fe2e4158b20 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:16.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.895+0000 7fe2ec04c640 1 -- 172.21.15.16:0/3830996067 >> 172.21.15.16:0/3830996067 conn(0x7fe2e406bdf0 msgr2=0x7fe2e40c60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:16.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.895+0000 7fe2ec04c640 1 -- 172.21.15.16:0/3830996067 shutdown_connections 2024-09-19T07:55:16.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:16.895+0000 7fe2ec04c640 1 -- 172.21.15.16:0/3830996067 wait complete. 2024-09-19T07:55:16.907 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2024-09-19T07:55:17.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.011+0000 7f475e78f640 1 Processor -- start 2024-09-19T07:55:17.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.011+0000 7f475e78f640 1 -- start start 2024-09-19T07:55:17.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.012+0000 7f475e78f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f475815e680 0x7f47580cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.012+0000 7f475e78f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47580d1880 con 0x7f47580d0010 2024-09-19T07:55:17.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.012+0000 7f475e78f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47580d1b60 con 0x7f475815e2a0 2024-09-19T07:55:17.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.012+0000 7f475e78f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f47580cf2d0 con 0x7f475815e680 2024-09-19T07:55:17.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.012+0000 7f47577fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f475815e680 0x7f47580cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:17.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.012+0000 7f475cd2c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f47580d0010 0x7f47580d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41290/0 (socket says 172.21.15.16:41290) 2024-09-19T07:55:17.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.012+0000 7f475cd2c640 1 -- 172.21.15.16:0/712567675 learned_addr learned my addr 172.21.15.16:0/712567675 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:17.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.013+0000 7f47577fe640 1 -- 172.21.15.16:0/712567675 >> v1:172.21.15.146:6789/0 conn(0x7f475815e2a0 legacy=0x7f475806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:17.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.013+0000 7f47577fe640 1 -- 172.21.15.16:0/712567675 >> v1:172.21.15.16:6789/0 conn(0x7f47580d0010 legacy=0x7f47580d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.013+0000 7f47577fe640 1 -- 172.21.15.16:0/712567675 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47580cf5b0 con 0x7f475815e680 2024-09-19T07:55:17.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.013+0000 7f4756ffd640 1 -- 172.21.15.16:0/712567675 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3740796233 0 0) 0x7f47580d1880 con 0x7f47580d0010 2024-09-19T07:55:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.013+0000 7f47577fe640 1 --2- 172.21.15.16:0/712567675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f475815e680 0x7f47580cfad0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f474c0097a0 tx=0x7f474c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56e3cd7fb037971 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.013+0000 7f4756ffd640 1 -- 172.21.15.16:0/712567675 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c037070 con 0x7f475815e680 2024-09-19T07:55:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.013+0000 7f4756ffd640 1 -- 172.21.15.16:0/712567675 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f474c032440 con 0x7f475815e680 2024-09-19T07:55:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.013+0000 7f4756ffd640 1 -- 172.21.15.16:0/712567675 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c032810 con 0x7f475815e680 2024-09-19T07:55:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.014+0000 7f475e78f640 1 -- 172.21.15.16:0/712567675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f475815e680 msgr2=0x7f47580cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.014+0000 7f475e78f640 1 --2- 172.21.15.16:0/712567675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f475815e680 0x7f47580cfad0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f474c0097a0 tx=0x7f474c02a1a0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.014+0000 7f475e78f640 1 -- 172.21.15.16:0/712567675 shutdown_connections 2024-09-19T07:55:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.014+0000 7f475e78f640 1 --2- 172.21.15.16:0/712567675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f475815e680 0x7f47580cfad0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.014+0000 7f475e78f640 1 -- 172.21.15.16:0/712567675 >> 172.21.15.16:0/712567675 conn(0x7f475809d760 msgr2=0x7f475808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.014+0000 7f475e78f640 1 -- 172.21.15.16:0/712567675 shutdown_connections 2024-09-19T07:55:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.014+0000 7f475e78f640 1 -- 172.21.15.16:0/712567675 wait complete. 2024-09-19T07:55:17.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.015+0000 7f475e78f640 1 Processor -- start 2024-09-19T07:55:17.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.015+0000 7f475e78f640 1 -- start start 2024-09-19T07:55:17.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.016+0000 7f475e78f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47580d0010 0x7f4758156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.016+0000 7f475e78f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f475815e2a0 0x7f47580cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.016+0000 7f475e78f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f475815e680 0x7f47580cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.016+0000 7f475e78f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f475806dcb0 con 0x7f47580d0010 2024-09-19T07:55:17.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.016+0000 7f475e78f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f475806db30 con 0x7f475815e680 2024-09-19T07:55:17.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.016+0000 7f475e78f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f47580d4a80 con 0x7f475815e2a0 2024-09-19T07:55:17.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.016+0000 7f4757fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47580d0010 0x7f4758156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:17.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.016+0000 7f4757fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47580d0010 0x7f4758156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40466/0 (socket says 172.21.15.16:40466) 2024-09-19T07:55:17.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.016+0000 7f4757fff640 1 -- 172.21.15.16:0/210641319 learned_addr learned my addr 172.21.15.16:0/210641319 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:17.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.016+0000 7f475cd2c640 1 --2- 172.21.15.16:0/210641319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f475815e680 0x7f47580cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:17.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.016+0000 7f47577fe640 1 --2- 172.21.15.16:0/210641319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f475815e2a0 0x7f47580cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.017+0000 7f4757fff640 1 -- 172.21.15.16:0/210641319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f475815e2a0 msgr2=0x7f47580cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.017+0000 7f4757fff640 1 --2- 172.21.15.16:0/210641319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f475815e2a0 0x7f47580cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.017+0000 7f4757fff640 1 -- 172.21.15.16:0/210641319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f475815e680 msgr2=0x7f47580cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.017+0000 7f4757fff640 1 --2- 172.21.15.16:0/210641319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f475815e680 0x7f47580cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.017+0000 7f4757fff640 1 -- 172.21.15.16:0/210641319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4758156e50 con 0x7f47580d0010 2024-09-19T07:55:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.017+0000 7f4757fff640 1 --2- 172.21.15.16:0/210641319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47580d0010 0x7f4758156910 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f474800c480 tx=0x7f474800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.017+0000 7f4754ff9640 1 -- 172.21.15.16:0/210641319 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47480052d0 con 0x7f47580d0010 2024-09-19T07:55:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.017+0000 7f4754ff9640 1 -- 172.21.15.16:0/210641319 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4748005470 con 0x7f47580d0010 2024-09-19T07:55:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.017+0000 7f475e78f640 1 -- 172.21.15.16:0/210641319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47580c82f0 con 0x7f47580d0010 2024-09-19T07:55:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.017+0000 7f4754ff9640 1 -- 172.21.15.16:0/210641319 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748012870 con 0x7f47580d0010 2024-09-19T07:55:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.017+0000 7f475e78f640 1 -- 172.21.15.16:0/210641319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47580c88b0 con 0x7f47580d0010 2024-09-19T07:55:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.018+0000 7f475e78f640 1 -- 172.21.15.16:0/210641319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47580cfad0 con 0x7f47580d0010 2024-09-19T07:55:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.018+0000 7f4754ff9640 1 -- 172.21.15.16:0/210641319 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4748012a10 con 0x7f47580d0010 2024-09-19T07:55:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.019+0000 7f4754ff9640 1 --2- 172.21.15.16:0/210641319 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f474406c2b0 0x7f474406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.019+0000 7f4754ff9640 1 -- 172.21.15.16:0/210641319 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f474808b1e0 con 0x7f47580d0010 2024-09-19T07:55:17.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.019+0000 7f47577fe640 1 --2- 172.21.15.16:0/210641319 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f474406c2b0 0x7f474406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:17.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.020+0000 7f47577fe640 1 --2- 172.21.15.16:0/210641319 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f474406c2b0 0x7f474406e770 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f474c034000 tx=0x7f474c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:17.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.023+0000 7f4754ff9640 1 -- 172.21.15.16:0/210641319 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4758156e50 con 0x7f47580d0010 2024-09-19T07:55:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.198+0000 7f475e78f640 1 -- 172.21.15.16:0/210641319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7f475815d080 con 0x7f47580d0010 2024-09-19T07:55:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.201+0000 7f4754ff9640 1 -- 172.21.15.16:0/210641319 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v67)=0 v67) ==== 124+0+0 (secure 0 0 0) 0x7f4748059850 con 0x7f47580d0010 2024-09-19T07:55:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.203+0000 7f475e78f640 1 -- 172.21.15.16:0/210641319 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f474406c2b0 msgr2=0x7f474406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.203+0000 7f475e78f640 1 --2- 172.21.15.16:0/210641319 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f474406c2b0 0x7f474406e770 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f474c034000 tx=0x7f474c02e000 comp rx=0 tx=0).stop 2024-09-19T07:55:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.203+0000 7f475e78f640 1 -- 172.21.15.16:0/210641319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47580d0010 msgr2=0x7f4758156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.203+0000 7f475e78f640 1 --2- 172.21.15.16:0/210641319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47580d0010 0x7f4758156910 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f474800c480 tx=0x7f474800c940 comp rx=0 tx=0).stop 2024-09-19T07:55:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.203+0000 7f475e78f640 1 -- 172.21.15.16:0/210641319 shutdown_connections 2024-09-19T07:55:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.203+0000 7f475e78f640 1 --2- 172.21.15.16:0/210641319 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f474406c2b0 0x7f474406e770 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.203+0000 7f475e78f640 1 --2- 172.21.15.16:0/210641319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f475815e680 0x7f47580cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.203+0000 7f475e78f640 1 --2- 172.21.15.16:0/210641319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f475815e2a0 0x7f47580cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.203+0000 7f475e78f640 1 --2- 172.21.15.16:0/210641319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47580d0010 0x7f4758156910 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.203+0000 7f475e78f640 1 -- 172.21.15.16:0/210641319 >> 172.21.15.16:0/210641319 conn(0x7f475809d760 msgr2=0x7f47580d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.203+0000 7f475e78f640 1 -- 172.21.15.16:0/210641319 shutdown_connections 2024-09-19T07:55:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.203+0000 7f475e78f640 1 -- 172.21.15.16:0/210641319 wait complete. 2024-09-19T07:55:17.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2024-09-19T07:55:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.319+0000 7fb10d34b640 1 Processor -- start 2024-09-19T07:55:17.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.319+0000 7fb10d34b640 1 -- start start 2024-09-19T07:55:17.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.320+0000 7fb10d34b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb10815c2c0 0x7fb1080cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.320+0000 7fb10d34b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1080cdee0 con 0x7fb1080d0bc0 2024-09-19T07:55:17.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.320+0000 7fb10d34b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1080ce1c0 con 0x7fb10815bee0 2024-09-19T07:55:17.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.320+0000 7fb10d34b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb1080ceaf0 con 0x7fb10815c2c0 2024-09-19T07:55:17.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.320+0000 7fb1067fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb1080d0bc0 0x7fb1080cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41292/0 (socket says 172.21.15.16:41292) 2024-09-19T07:55:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.320+0000 7fb1067fc640 1 -- 172.21.15.16:0/3421208038 learned_addr learned my addr 172.21.15.16:0/3421208038 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.320+0000 7fb1077fe640 1 --2- 172.21.15.16:0/3421208038 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb10815c2c0 0x7fb1080cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.320+0000 7fb105ffb640 1 -- 172.21.15.16:0/3421208038 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3137056440 0 0) 0x7fb1080cdee0 con 0x7fb1080d0bc0 2024-09-19T07:55:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.320+0000 7fb105ffb640 1 -- 172.21.15.16:0/3421208038 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0f0003520 con 0x7fb1080d0bc0 2024-09-19T07:55:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.321+0000 7fb105ffb640 1 -- 172.21.15.16:0/3421208038 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 306352803 0 0) 0x7fb1080ce1c0 con 0x7fb10815bee0 2024-09-19T07:55:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.321+0000 7fb105ffb640 1 -- 172.21.15.16:0/3421208038 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1080cdee0 con 0x7fb10815bee0 2024-09-19T07:55:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.321+0000 7fb105ffb640 1 -- 172.21.15.16:0/3421208038 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2673286534 0 0) 0x7fb0f0003520 con 0x7fb1080d0bc0 2024-09-19T07:55:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.321+0000 7fb1077fe640 1 -- 172.21.15.16:0/3421208038 >> v1:172.21.15.146:6789/0 conn(0x7fb10815bee0 legacy=0x7fb1080d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.321+0000 7fb1077fe640 1 -- 172.21.15.16:0/3421208038 >> v1:172.21.15.16:6789/0 conn(0x7fb1080d0bc0 legacy=0x7fb1080cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.321+0000 7fb1077fe640 1 -- 172.21.15.16:0/3421208038 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1080ce4a0 con 0x7fb10815c2c0 2024-09-19T07:55:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.322+0000 7fb1077fe640 1 --2- 172.21.15.16:0/3421208038 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb10815c2c0 0x7fb1080cf910 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb100009dc0 tx=0x7fb10002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff63bbb55be5fb11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.322+0000 7fb105ffb640 1 -- 172.21.15.16:0/3421208038 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc001670 con 0x7fb10815c2c0 2024-09-19T07:55:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.322+0000 7fb105ffb640 1 -- 172.21.15.16:0/3421208038 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb100007540 con 0x7fb10815c2c0 2024-09-19T07:55:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.322+0000 7fb105ffb640 1 -- 172.21.15.16:0/3421208038 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100007910 con 0x7fb10815c2c0 2024-09-19T07:55:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.322+0000 7fb10d34b640 1 -- 172.21.15.16:0/3421208038 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb10815c2c0 msgr2=0x7fb1080cf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.322+0000 7fb10d34b640 1 --2- 172.21.15.16:0/3421208038 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb10815c2c0 0x7fb1080cf910 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb100009dc0 tx=0x7fb10002a660 comp rx=0 tx=0).stop 2024-09-19T07:55:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.322+0000 7fb10d34b640 1 -- 172.21.15.16:0/3421208038 shutdown_connections 2024-09-19T07:55:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.322+0000 7fb10d34b640 1 --2- 172.21.15.16:0/3421208038 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb10815c2c0 0x7fb1080cf910 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.322+0000 7fb10d34b640 1 -- 172.21.15.16:0/3421208038 >> 172.21.15.16:0/3421208038 conn(0x7fb10806bdf0 msgr2=0x7fb10806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.322+0000 7fb10d34b640 1 -- 172.21.15.16:0/3421208038 shutdown_connections 2024-09-19T07:55:17.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.322+0000 7fb10d34b640 1 -- 172.21.15.16:0/3421208038 wait complete. 2024-09-19T07:55:17.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.324+0000 7fb10d34b640 1 Processor -- start 2024-09-19T07:55:17.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.324+0000 7fb10d34b640 1 -- start start 2024-09-19T07:55:17.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.324+0000 7fb10d34b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1080d0bc0 0x7fb10815c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.324+0000 7fb10d34b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb10815bee0 0x7fb1081538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.324+0000 7fb10d34b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb10815c2c0 0x7fb108072b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.324+0000 7fb10d34b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb10806f050 con 0x7fb1080d0bc0 2024-09-19T07:55:17.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.324+0000 7fb10d34b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb10806eed0 con 0x7fb10815bee0 2024-09-19T07:55:17.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.324+0000 7fb10d34b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb108166ea0 con 0x7fb10815c2c0 2024-09-19T07:55:17.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.325+0000 7fb106ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1080d0bc0 0x7fb10815c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:17.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.325+0000 7fb106ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1080d0bc0 0x7fb10815c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40474/0 (socket says 172.21.15.16:40474) 2024-09-19T07:55:17.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.325+0000 7fb106ffd640 1 -- 172.21.15.16:0/2777070584 learned_addr learned my addr 172.21.15.16:0/2777070584 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:17.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.325+0000 7fb1077fe640 1 --2- 172.21.15.16:0/2777070584 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb10815c2c0 0x7fb108072b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:17.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.325+0000 7fb1067fc640 1 --2- 172.21.15.16:0/2777070584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb10815bee0 0x7fb1081538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:17.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.325+0000 7fb106ffd640 1 -- 172.21.15.16:0/2777070584 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb10815c2c0 msgr2=0x7fb108072b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.325+0000 7fb106ffd640 1 --2- 172.21.15.16:0/2777070584 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb10815c2c0 0x7fb108072b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.325+0000 7fb106ffd640 1 -- 172.21.15.16:0/2777070584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb10815bee0 msgr2=0x7fb1081538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.325+0000 7fb106ffd640 1 --2- 172.21.15.16:0/2777070584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb10815bee0 0x7fb1081538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.325+0000 7fb106ffd640 1 -- 172.21.15.16:0/2777070584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb108157f80 con 0x7fb1080d0bc0 2024-09-19T07:55:17.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.325+0000 7fb106ffd640 1 --2- 172.21.15.16:0/2777070584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1080d0bc0 0x7fb10815c6d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800c490 tx=0x7fb0f800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:17.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.325+0000 7fb0ef7fe640 1 -- 172.21.15.16:0/2777070584 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f80052d0 con 0x7fb1080d0bc0 2024-09-19T07:55:17.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.325+0000 7fb0ef7fe640 1 -- 172.21.15.16:0/2777070584 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f8005470 con 0x7fb1080d0bc0 2024-09-19T07:55:17.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.325+0000 7fb10d34b640 1 -- 172.21.15.16:0/2777070584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1080720b0 con 0x7fb1080d0bc0 2024-09-19T07:55:17.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.325+0000 7fb0ef7fe640 1 -- 172.21.15.16:0/2777070584 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8012850 con 0x7fb1080d0bc0 2024-09-19T07:55:17.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.325+0000 7fb10d34b640 1 -- 172.21.15.16:0/2777070584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb108072670 con 0x7fb1080d0bc0 2024-09-19T07:55:17.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.326+0000 7fb10d34b640 1 -- 172.21.15.16:0/2777070584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1080d04b0 con 0x7fb1080d0bc0 2024-09-19T07:55:17.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.327+0000 7fb0ef7fe640 1 -- 172.21.15.16:0/2777070584 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb0f80129f0 con 0x7fb1080d0bc0 2024-09-19T07:55:17.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.327+0000 7fb0ef7fe640 1 --2- 172.21.15.16:0/2777070584 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb0f406c2b0 0x7fb0f406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.328+0000 7fb0ef7fe640 1 -- 172.21.15.16:0/2777070584 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7fb0f808b120 con 0x7fb1080d0bc0 2024-09-19T07:55:17.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.328+0000 7fb1067fc640 1 --2- 172.21.15.16:0/2777070584 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb0f406c2b0 0x7fb0f406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:17.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.328+0000 7fb1067fc640 1 --2- 172.21.15.16:0/2777070584 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb0f406c2b0 0x7fb0f406e770 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb1080af680 tx=0x7fb0fc001610 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:17.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.332+0000 7fb0ef7fe640 1 -- 172.21.15.16:0/2777070584 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb0f80590b0 con 0x7fb1080d0bc0 2024-09-19T07:55:17.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.512+0000 7fb10d34b640 1 -- 172.21.15.16:0/2777070584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) -- 0x7fb108158750 con 0x7fb1080d0bc0 2024-09-19T07:55:17.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.512+0000 7fb0ef7fe640 1 -- 172.21.15.16:0/2777070584 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v67) ==== 106+0+8 (secure 0 0 0) 0x7fb0f805d3b0 con 0x7fb1080d0bc0 2024-09-19T07:55:17.513 INFO:teuthology.orchestra.run.smithi016.stdout:1048576 2024-09-19T07:55:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.514+0000 7fb10d34b640 1 -- 172.21.15.16:0/2777070584 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb0f406c2b0 msgr2=0x7fb0f406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.514+0000 7fb10d34b640 1 --2- 172.21.15.16:0/2777070584 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb0f406c2b0 0x7fb0f406e770 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb1080af680 tx=0x7fb0fc001610 comp rx=0 tx=0).stop 2024-09-19T07:55:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.514+0000 7fb10d34b640 1 -- 172.21.15.16:0/2777070584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1080d0bc0 msgr2=0x7fb10815c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.514+0000 7fb10d34b640 1 --2- 172.21.15.16:0/2777070584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1080d0bc0 0x7fb10815c6d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800c490 tx=0x7fb0f800c950 comp rx=0 tx=0).stop 2024-09-19T07:55:17.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.514+0000 7fb10d34b640 1 -- 172.21.15.16:0/2777070584 shutdown_connections 2024-09-19T07:55:17.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.514+0000 7fb10d34b640 1 --2- 172.21.15.16:0/2777070584 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb0f406c2b0 0x7fb0f406e770 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.514+0000 7fb10d34b640 1 --2- 172.21.15.16:0/2777070584 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb10815c2c0 0x7fb108072b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.514+0000 7fb10d34b640 1 --2- 172.21.15.16:0/2777070584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb10815bee0 0x7fb1081538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.514+0000 7fb10d34b640 1 --2- 172.21.15.16:0/2777070584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1080d0bc0 0x7fb10815c6d0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.514+0000 7fb10d34b640 1 -- 172.21.15.16:0/2777070584 >> 172.21.15.16:0/2777070584 conn(0x7fb10806bdf0 msgr2=0x7fb1080cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:17.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.514+0000 7fb10d34b640 1 -- 172.21.15.16:0/2777070584 shutdown_connections 2024-09-19T07:55:17.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.514+0000 7fb10d34b640 1 -- 172.21.15.16:0/2777070584 wait complete. 2024-09-19T07:55:17.526 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-09-19T07:55:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.629+0000 7fbee8903640 1 Processor -- start 2024-09-19T07:55:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.629+0000 7fbee8903640 1 -- start start 2024-09-19T07:55:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.630+0000 7fbee8903640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbee4161eb0 0x7fbee41622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.630+0000 7fbee8903640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbee41425e0 0x7fbee4142a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.630+0000 7fbee8903640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee4162b70 con 0x7fbee406cd20 2024-09-19T07:55:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.630+0000 7fbee8903640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fbee4162e50 con 0x7fbee41425e0 2024-09-19T07:55:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.630+0000 7fbee8903640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fbee4142150 con 0x7fbee4161eb0 2024-09-19T07:55:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.630+0000 7fbee259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbee406cd20 0x7fbee406d100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41298/0 (socket says 172.21.15.16:41298) 2024-09-19T07:55:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.630+0000 7fbee2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbee41425e0 0x7fbee4142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.630+0000 7fbee259c640 1 -- 172.21.15.16:0/1153035541 learned_addr learned my addr 172.21.15.16:0/1153035541 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.631+0000 7fbee2d9d640 1 -- 172.21.15.16:0/1153035541 >> v2:172.21.15.146:3300/0 conn(0x7fbee4161eb0 msgr2=0x7fbee41622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.631+0000 7fbee2d9d640 1 --2- 172.21.15.16:0/1153035541 >> v2:172.21.15.146:3300/0 conn(0x7fbee4161eb0 0x7fbee41622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.631+0000 7fbee2d9d640 1 -- 172.21.15.16:0/1153035541 >> v1:172.21.15.16:6789/0 conn(0x7fbee406cd20 legacy=0x7fbee406d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.631+0000 7fbee2d9d640 1 -- 172.21.15.16:0/1153035541 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee4141b00 con 0x7fbee41425e0 2024-09-19T07:55:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.631+0000 7fbee159a640 1 -- 172.21.15.16:0/1153035541 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2499109939 0 0) 0x7fbee4162b70 con 0x7fbee406cd20 2024-09-19T07:55:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.631+0000 7fbee2d9d640 1 --2- 172.21.15.16:0/1153035541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee41425e0 0x7fbee4142a70 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fbee406ef00 tx=0x7fbedc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78085f1cfb1918c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.631+0000 7fbee159a640 1 -- 172.21.15.16:0/1153035541 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc007740 con 0x7fbee41425e0 2024-09-19T07:55:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.632+0000 7fbee159a640 1 -- 172.21.15.16:0/1153035541 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbedc0078e0 con 0x7fbee41425e0 2024-09-19T07:55:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.632+0000 7fbee159a640 1 -- 172.21.15.16:0/1153035541 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc007cb0 con 0x7fbee41425e0 2024-09-19T07:55:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.632+0000 7fbee8903640 1 -- 172.21.15.16:0/1153035541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee41425e0 msgr2=0x7fbee4142a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.632+0000 7fbee8903640 1 --2- 172.21.15.16:0/1153035541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee41425e0 0x7fbee4142a70 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fbee406ef00 tx=0x7fbedc02a5f0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.632+0000 7fbee8903640 1 -- 172.21.15.16:0/1153035541 shutdown_connections 2024-09-19T07:55:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.632+0000 7fbee8903640 1 --2- 172.21.15.16:0/1153035541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee41425e0 0x7fbee4142a70 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.632+0000 7fbee8903640 1 --2- 172.21.15.16:0/1153035541 >> v2:172.21.15.146:3300/0 conn(0x7fbee4161eb0 0x7fbee41622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.632+0000 7fbee8903640 1 -- 172.21.15.16:0/1153035541 >> 172.21.15.16:0/1153035541 conn(0x7fbee40b3ed0 msgr2=0x7fbee40b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.632+0000 7fbee8903640 1 -- 172.21.15.16:0/1153035541 shutdown_connections 2024-09-19T07:55:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.632+0000 7fbee8903640 1 -- 172.21.15.16:0/1153035541 wait complete. 2024-09-19T07:55:17.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.633+0000 7fbee8903640 1 Processor -- start 2024-09-19T07:55:17.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.634+0000 7fbee8903640 1 -- start start 2024-09-19T07:55:17.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.634+0000 7fbee8903640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee406cd20 0x7fbee4140820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.634+0000 7fbee8903640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbee41425e0 0x7fbee41569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.634+0000 7fbee8903640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbee4161eb0 0x7fbee40cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.634+0000 7fbee8903640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbee4141600 con 0x7fbee406cd20 2024-09-19T07:55:17.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.634+0000 7fbee8903640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbee4162830 con 0x7fbee4161eb0 2024-09-19T07:55:17.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.634+0000 7fbee8903640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbee4142fb0 con 0x7fbee41425e0 2024-09-19T07:55:17.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.634+0000 7fbee259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee406cd20 0x7fbee4140820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:17.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.634+0000 7fbee259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee406cd20 0x7fbee4140820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40490/0 (socket says 172.21.15.16:40490) 2024-09-19T07:55:17.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.634+0000 7fbee259c640 1 -- 172.21.15.16:0/2334841537 learned_addr learned my addr 172.21.15.16:0/2334841537 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:17.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.635+0000 7fbee1d9b640 1 --2- 172.21.15.16:0/2334841537 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbee41425e0 0x7fbee41569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:17.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.635+0000 7fbee2d9d640 1 --2- 172.21.15.16:0/2334841537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbee4161eb0 0x7fbee40cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.635+0000 7fbee259c640 1 -- 172.21.15.16:0/2334841537 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbee41425e0 msgr2=0x7fbee41569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.635+0000 7fbee259c640 1 --2- 172.21.15.16:0/2334841537 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbee41425e0 0x7fbee41569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.635+0000 7fbee259c640 1 -- 172.21.15.16:0/2334841537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbee4161eb0 msgr2=0x7fbee40cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.635+0000 7fbee259c640 1 --2- 172.21.15.16:0/2334841537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbee4161eb0 0x7fbee40cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.635+0000 7fbee259c640 1 -- 172.21.15.16:0/2334841537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee4155c30 con 0x7fbee406cd20 2024-09-19T07:55:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.635+0000 7fbee1d9b640 1 --2- 172.21.15.16:0/2334841537 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbee41425e0 0x7fbee41569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.635+0000 7fbee259c640 1 --2- 172.21.15.16:0/2334841537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee406cd20 0x7fbee4140820 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fbed400c4d0 tx=0x7fbed400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.635+0000 7fbedaffd640 1 -- 172.21.15.16:0/2334841537 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4018070 con 0x7fbee406cd20 2024-09-19T07:55:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.635+0000 7fbedaffd640 1 -- 172.21.15.16:0/2334841537 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed4007500 con 0x7fbee406cd20 2024-09-19T07:55:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.635+0000 7fbedaffd640 1 -- 172.21.15.16:0/2334841537 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4004a20 con 0x7fbee406cd20 2024-09-19T07:55:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.635+0000 7fbee8903640 1 -- 172.21.15.16:0/2334841537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee4155f10 con 0x7fbee406cd20 2024-09-19T07:55:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.635+0000 7fbee8903640 1 -- 172.21.15.16:0/2334841537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee41564d0 con 0x7fbee406cd20 2024-09-19T07:55:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.636+0000 7fbee8903640 1 -- 172.21.15.16:0/2334841537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee41566d0 con 0x7fbee406cd20 2024-09-19T07:55:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.637+0000 7fbedaffd640 1 -- 172.21.15.16:0/2334841537 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbed401b070 con 0x7fbee406cd20 2024-09-19T07:55:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.637+0000 7fbedaffd640 1 --2- 172.21.15.16:0/2334841537 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbec406c2b0 0x7fbec406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.637+0000 7fbedaffd640 1 -- 172.21.15.16:0/2334841537 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7fbed408a940 con 0x7fbee406cd20 2024-09-19T07:55:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.637+0000 7fbee1d9b640 1 --2- 172.21.15.16:0/2334841537 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbec406c2b0 0x7fbec406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.638+0000 7fbee1d9b640 1 --2- 172.21.15.16:0/2334841537 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbec406c2b0 0x7fbec406e770 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fbee41558d0 tx=0x7fbecc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:17.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.641+0000 7fbedaffd640 1 -- 172.21.15.16:0/2334841537 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fbee41566d0 con 0x7fbee406cd20 2024-09-19T07:55:17.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.818+0000 7fbee8903640 1 -- 172.21.15.16:0/2334841537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7fbee4162b70 con 0x7fbee406cd20 2024-09-19T07:55:17.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.818+0000 7fbedaffd640 1 -- 172.21.15.16:0/2334841537 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v67) ==== 112+0+10 (secure 0 0 0) 0x7fbed4058f20 con 0x7fbee406cd20 2024-09-19T07:55:17.819 INFO:teuthology.orchestra.run.smithi016.stdout:60.000000 2024-09-19T07:55:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.821+0000 7fbee8903640 1 -- 172.21.15.16:0/2334841537 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbec406c2b0 msgr2=0x7fbec406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.821+0000 7fbee8903640 1 --2- 172.21.15.16:0/2334841537 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbec406c2b0 0x7fbec406e770 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fbee41558d0 tx=0x7fbecc008040 comp rx=0 tx=0).stop 2024-09-19T07:55:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.821+0000 7fbee8903640 1 -- 172.21.15.16:0/2334841537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee406cd20 msgr2=0x7fbee4140820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.821+0000 7fbee8903640 1 --2- 172.21.15.16:0/2334841537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee406cd20 0x7fbee4140820 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fbed400c4d0 tx=0x7fbed400c990 comp rx=0 tx=0).stop 2024-09-19T07:55:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.821+0000 7fbee8903640 1 -- 172.21.15.16:0/2334841537 shutdown_connections 2024-09-19T07:55:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.821+0000 7fbee8903640 1 --2- 172.21.15.16:0/2334841537 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbec406c2b0 0x7fbec406e770 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.821+0000 7fbee8903640 1 --2- 172.21.15.16:0/2334841537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbee4161eb0 0x7fbee40cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.821+0000 7fbee8903640 1 --2- 172.21.15.16:0/2334841537 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbee41425e0 0x7fbee41569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.821+0000 7fbee8903640 1 --2- 172.21.15.16:0/2334841537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee406cd20 0x7fbee4140820 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.821+0000 7fbee8903640 1 -- 172.21.15.16:0/2334841537 >> 172.21.15.16:0/2334841537 conn(0x7fbee40b3ed0 msgr2=0x7fbee415d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.821+0000 7fbee8903640 1 -- 172.21.15.16:0/2334841537 shutdown_connections 2024-09-19T07:55:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17.821+0000 7fbee8903640 1 -- 172.21.15.16:0/2334841537 wait complete. 2024-09-19T07:55:17.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:55:17.832 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:55:17.832 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:55:17.832 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:55:17.832 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:55:17.832 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:55:17.833 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:55:17.833 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:55:17.833 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:55:17.833 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:55:17.833 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:55:17.833 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:55:17.833 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:55:17.833 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:55:17.833 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:55:17.833 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:55:17.833 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:55:17.833 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:55:17.834 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:55:17.834 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:55:17.834 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:55:17.834 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:55:17.834 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:55:17.834 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:55:17.834 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:55:17.834 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:55:17.834 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:55:17.914 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.76175.asok 2024-09-19T07:55:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:55:17.924 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.76175.asok 2024-09-19T07:55:17.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:17.924 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.76175.asok mds_sessions 2024-09-19T07:55:18.061 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:55:18.061 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 8484, 2024-09-19T07:55:18.061 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:55:18.061 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:55:18.061 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:55:18.061 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 8484 2024-09-19T07:55:18.061 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:18.061 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:55:18.061 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:55:18.062 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:55:18.062 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3786269155 2024-09-19T07:55:18.062 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:18.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:18.062 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.8484 v1:192.168.0.1:0/3786269155", 2024-09-19T07:55:18.062 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "v1:192.168.0.1:0/3786269155", 2024-09-19T07:55:18.062 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions": [ 2024-09-19T07:55:18.062 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:18.062 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": 0, 2024-09-19T07:55:18.062 INFO:teuthology.orchestra.run.smithi016.stdout: "addrs": { 2024-09-19T07:55:18.062 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-19T07:55:18.062 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:18.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-19T07:55:18.062 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6833", 2024-09-19T07:55:18.063 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1929965275 2024-09-19T07:55:18.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:18.063 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:18.063 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:55:18.063 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6835", 2024-09-19T07:55:18.063 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1929965275 2024-09-19T07:55:18.063 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:18.063 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T07:55:18.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:18.063 INFO:teuthology.orchestra.run.smithi016.stdout: "seq": 0, 2024-09-19T07:55:18.063 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_gen": 0, 2024-09-19T07:55:18.063 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_ttl": "2024-09-19T07:56:14.150826+0000", 2024-09-19T07:55:18.063 INFO:teuthology.orchestra.run.smithi016.stdout: "last_cap_renew_request": "2024-09-19T07:55:14.150826+0000", 2024-09-19T07:55:18.064 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_renew_seq": 2, 2024-09-19T07:55:18.064 INFO:teuthology.orchestra.run.smithi016.stdout: "num_caps": 1, 2024-09-19T07:55:18.064 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "open" 2024-09-19T07:55:18.064 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:18.064 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:55:18.064 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 81 2024-09-19T07:55:18.064 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:55:18.069 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:55:18.069 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:55:18.086 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-19T07:55:18.086 DEBUG:teuthology.orchestra.run.smithi016:> import fcntl 2024-09-19T07:55:18.086 DEBUG:teuthology.orchestra.run.smithi016:> import sys 2024-09-19T07:55:18.086 DEBUG:teuthology.orchestra.run.smithi016:> import time 2024-09-19T07:55:18.086 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:55:18.086 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:55:18.086 DEBUG:teuthology.orchestra.run.smithi016:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-19T07:55:18.086 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:55:18.086 DEBUG:teuthology.orchestra.run.smithi016:> n = 400 2024-09-19T07:55:18.086 DEBUG:teuthology.orchestra.run.smithi016:> abs_path = "/home/ubuntu/cephtest/mnt.0/." 2024-09-19T07:55:18.086 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:55:18.087 DEBUG:teuthology.orchestra.run.smithi016:> if not os.path.exists(abs_path): 2024-09-19T07:55:18.087 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(abs_path) 2024-09-19T07:55:18.087 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:55:18.087 DEBUG:teuthology.orchestra.run.smithi016:> handles = [] 2024-09-19T07:55:18.087 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, n): 2024-09-19T07:55:18.087 DEBUG:teuthology.orchestra.run.smithi016:> fname = "file_"+str(i) 2024-09-19T07:55:18.087 DEBUG:teuthology.orchestra.run.smithi016:> path = os.path.join(abs_path, fname) 2024-09-19T07:55:18.087 DEBUG:teuthology.orchestra.run.smithi016:> handles.append(open(path, '"'"'w'"'"')) 2024-09-19T07:55:18.087 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:55:18.087 DEBUG:teuthology.orchestra.run.smithi016:> print("waiting with handles open", file=sys.stderr) 2024-09-19T07:55:18.087 DEBUG:teuthology.orchestra.run.smithi016:> while True: 2024-09-19T07:55:18.087 DEBUG:teuthology.orchestra.run.smithi016:> print("open_n_background: keeping files open", file=sys.stderr) 2024-09-19T07:55:18.087 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-19T07:55:18.087 DEBUG:teuthology.orchestra.run.smithi016:> if os.read(0, 4096) == b"": 2024-09-19T07:55:18.087 DEBUG:teuthology.orchestra.run.smithi016:> break 2024-09-19T07:55:18.088 DEBUG:teuthology.orchestra.run.smithi016:> except BlockingIOError: 2024-09-19T07:55:18.088 DEBUG:teuthology.orchestra.run.smithi016:> pass 2024-09-19T07:55:18.088 DEBUG:teuthology.orchestra.run.smithi016:> time.sleep(2) 2024-09-19T07:55:18.088 DEBUG:teuthology.orchestra.run.smithi016:> print("stdin closed, goodbye!", file=sys.stderr) 2024-09-19T07:55:18.088 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:55:18.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:55:18.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.240+0000 7f32fa5fd640 1 Processor -- start 2024-09-19T07:55:18.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.240+0000 7f32fa5fd640 1 -- start start 2024-09-19T07:55:18.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.241+0000 7f32fa5fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32f415c280 0x7f32f40d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:18.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.241+0000 7f32fa5fd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f32f406c320 0x7f32f40b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:18.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.241+0000 7f32fa5fd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32f40d0a30 con 0x7f32f41608f0 2024-09-19T07:55:18.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.241+0000 7f32fa5fd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f32f40b4010 con 0x7f32f406c320 2024-09-19T07:55:18.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.241+0000 7f32fa5fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f32f40b45d0 con 0x7f32f415c280 2024-09-19T07:55:18.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.241+0000 7f32f37fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f32f41608f0 0x7f32f406bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41306/0 (socket says 172.21.15.16:41306) 2024-09-19T07:55:18.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.241+0000 7f32f37fe640 1 -- 172.21.15.16:0/2344217283 learned_addr learned my addr 172.21.15.16:0/2344217283 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:18.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f8b9a640 1 --2- 172.21.15.16:0/2344217283 >> v2:172.21.15.146:3300/0 conn(0x7f32f406c320 0x7f32f40b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:18.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f3fff640 1 --2- 172.21.15.16:0/2344217283 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32f415c280 0x7f32f40d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:18.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f2ffd640 1 -- 172.21.15.16:0/2344217283 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2420938569 0 0) 0x7f32f40d0a30 con 0x7f32f41608f0 2024-09-19T07:55:18.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f2ffd640 1 -- 172.21.15.16:0/2344217283 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32dc003520 con 0x7f32f41608f0 2024-09-19T07:55:18.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f2ffd640 1 -- 172.21.15.16:0/2344217283 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 300993069 0 0) 0x7f32dc003520 con 0x7f32f41608f0 2024-09-19T07:55:18.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f2ffd640 1 -- 172.21.15.16:0/2344217283 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32f415c280 msgr2=0x7f32f40d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:18.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f2ffd640 1 --2- 172.21.15.16:0/2344217283 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32f415c280 0x7f32f40d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:18.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f2ffd640 1 -- 172.21.15.16:0/2344217283 >> v2:172.21.15.146:3300/0 conn(0x7f32f406c320 msgr2=0x7f32f40b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:18.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f2ffd640 1 --2- 172.21.15.16:0/2344217283 >> v2:172.21.15.146:3300/0 conn(0x7f32f406c320 0x7f32f40b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:18.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f2ffd640 1 -- 172.21.15.16:0/2344217283 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f40d0d10 con 0x7f32f41608f0 2024-09-19T07:55:18.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f2ffd640 1 -- 172.21.15.16:0/2344217283 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f32e8001670 con 0x7f32f41608f0 2024-09-19T07:55:18.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f2ffd640 1 -- 172.21.15.16:0/2344217283 >> v1:172.21.15.16:6789/0 conn(0x7f32f41608f0 legacy=0x7f32f406bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:18.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f2ffd640 1 --2- 172.21.15.16:0/2344217283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32dc0039b0 0x7f32dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:18.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f2ffd640 1 --2- 172.21.15.16:0/2344217283 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32f415c280 0x7f32dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:18.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f2ffd640 1 --2- 172.21.15.16:0/2344217283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32dc004d40 0x7f32dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:18.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f2ffd640 1 -- 172.21.15.16:0/2344217283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32f40b4010 con 0x7f32dc0039b0 2024-09-19T07:55:18.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f2ffd640 1 -- 172.21.15.16:0/2344217283 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f32f40b45d0 con 0x7f32dc004d40 2024-09-19T07:55:18.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f2ffd640 1 -- 172.21.15.16:0/2344217283 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f32dc00b9c0 con 0x7f32f415c280 2024-09-19T07:55:18.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f3fff640 1 --2- 172.21.15.16:0/2344217283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32dc0039b0 0x7f32dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:18.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f37fe640 1 --2- 172.21.15.16:0/2344217283 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32f415c280 0x7f32dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:18.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.242+0000 7f32f8b9a640 1 --2- 172.21.15.16:0/2344217283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32dc004d40 0x7f32dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:18.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.243+0000 7f32f3fff640 1 -- 172.21.15.16:0/2344217283 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32f415c280 msgr2=0x7f32dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:18.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.243+0000 7f32f3fff640 1 --2- 172.21.15.16:0/2344217283 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32f415c280 0x7f32dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:18.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.243+0000 7f32f3fff640 1 -- 172.21.15.16:0/2344217283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32dc004d40 msgr2=0x7f32dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:18.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.243+0000 7f32f3fff640 1 --2- 172.21.15.16:0/2344217283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32dc004d40 0x7f32dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:18.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.243+0000 7f32f3fff640 1 -- 172.21.15.16:0/2344217283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32dc003520 con 0x7f32dc0039b0 2024-09-19T07:55:18.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.243+0000 7f32f3fff640 1 --2- 172.21.15.16:0/2344217283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32dc0039b0 0x7f32dc003dd0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f32e4009ac0 tx=0x7f32dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f467e4c0d01aafa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:18.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.243+0000 7f32f2ffd640 1 -- 172.21.15.16:0/2344217283 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4036070 con 0x7f32dc0039b0 2024-09-19T07:55:18.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.243+0000 7f32f2ffd640 1 -- 172.21.15.16:0/2344217283 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32e4031400 con 0x7f32dc0039b0 2024-09-19T07:55:18.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.243+0000 7f32f2ffd640 1 -- 172.21.15.16:0/2344217283 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4009690 con 0x7f32dc0039b0 2024-09-19T07:55:18.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.244+0000 7f32fa5fd640 1 -- 172.21.15.16:0/2344217283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32dc0039b0 msgr2=0x7f32dc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:18.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.244+0000 7f32fa5fd640 1 --2- 172.21.15.16:0/2344217283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32dc0039b0 0x7f32dc003dd0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f32e4009ac0 tx=0x7f32dc005830 comp rx=0 tx=0).stop 2024-09-19T07:55:18.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.244+0000 7f32f8b9a640 1 -- 172.21.15.16:0/2344217283 reap_dead start 2024-09-19T07:55:18.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.244+0000 7f32fa5fd640 1 -- 172.21.15.16:0/2344217283 shutdown_connections 2024-09-19T07:55:18.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.244+0000 7f32fa5fd640 1 -- 172.21.15.16:0/2344217283 >> 172.21.15.16:0/2344217283 conn(0x7f32f40b2d10 msgr2=0x7f32f40b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:18.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.244+0000 7f32fa5fd640 1 -- 172.21.15.16:0/2344217283 shutdown_connections 2024-09-19T07:55:18.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.244+0000 7f32fa5fd640 1 -- 172.21.15.16:0/2344217283 wait complete. 2024-09-19T07:55:18.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.246+0000 7f32fa5fd640 1 Processor -- start 2024-09-19T07:55:18.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.246+0000 7f32fa5fd640 1 -- start start 2024-09-19T07:55:18.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.247+0000 7f32fa5fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32dc0039b0 0x7f32f415b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:18.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.247+0000 7f32fa5fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32f409ebc0 0x7f32f40c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:18.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.247+0000 7f32fa5fd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32f40c73c0 0x7f32f415e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:18.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.247+0000 7f32fa5fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32f4160e90 con 0x7f32dc0039b0 2024-09-19T07:55:18.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.247+0000 7f32fa5fd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f32f4160d10 con 0x7f32f40c73c0 2024-09-19T07:55:18.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.247+0000 7f32fa5fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f32f415c660 con 0x7f32f409ebc0 2024-09-19T07:55:18.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.247+0000 7f32f3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32dc0039b0 0x7f32f415b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:18.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.247+0000 7f32f3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32dc0039b0 0x7f32f415b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40502/0 (socket says 172.21.15.16:40502) 2024-09-19T07:55:18.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.247+0000 7f32f3fff640 1 -- 172.21.15.16:0/4091320872 learned_addr learned my addr 172.21.15.16:0/4091320872 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:18.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.247+0000 7f32f8b9a640 1 --2- 172.21.15.16:0/4091320872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32f40c73c0 0x7f32f415e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:18.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.247+0000 7f32f37fe640 1 --2- 172.21.15.16:0/4091320872 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32f409ebc0 0x7f32f40c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:18.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.247+0000 7f32f3fff640 1 -- 172.21.15.16:0/4091320872 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32f409ebc0 msgr2=0x7f32f40c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:18.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.247+0000 7f32f3fff640 1 --2- 172.21.15.16:0/4091320872 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32f409ebc0 0x7f32f40c6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:18.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.247+0000 7f32f3fff640 1 -- 172.21.15.16:0/4091320872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32f40c73c0 msgr2=0x7f32f415e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:18.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.247+0000 7f32f3fff640 1 --2- 172.21.15.16:0/4091320872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32f40c73c0 0x7f32f415e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:18.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.247+0000 7f32f3fff640 1 -- 172.21.15.16:0/4091320872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f409db20 con 0x7f32dc0039b0 2024-09-19T07:55:18.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.247+0000 7f32f3fff640 1 --2- 172.21.15.16:0/4091320872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32dc0039b0 0x7f32f415b550 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f32e4009ac0 tx=0x7f32e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:18.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.248+0000 7f32f0ff9640 1 -- 172.21.15.16:0/4091320872 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4036050 con 0x7f32dc0039b0 2024-09-19T07:55:18.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.248+0000 7f32f0ff9640 1 -- 172.21.15.16:0/4091320872 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32e4042070 con 0x7f32dc0039b0 2024-09-19T07:55:18.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.248+0000 7f32fa5fd640 1 -- 172.21.15.16:0/4091320872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f409de00 con 0x7f32dc0039b0 2024-09-19T07:55:18.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.248+0000 7f32f0ff9640 1 -- 172.21.15.16:0/4091320872 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4009690 con 0x7f32dc0039b0 2024-09-19T07:55:18.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.248+0000 7f32fa5fd640 1 -- 172.21.15.16:0/4091320872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f409e3c0 con 0x7f32dc0039b0 2024-09-19T07:55:18.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.249+0000 7f32fa5fd640 1 -- 172.21.15.16:0/4091320872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f40d0350 con 0x7f32dc0039b0 2024-09-19T07:55:18.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.250+0000 7f32f0ff9640 1 -- 172.21.15.16:0/4091320872 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f32e4047460 con 0x7f32dc0039b0 2024-09-19T07:55:18.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.250+0000 7f32f0ff9640 1 --2- 172.21.15.16:0/4091320872 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f32e006c2b0 0x7f32e006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:18.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.250+0000 7f32f0ff9640 1 -- 172.21.15.16:0/4091320872 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f32e40abde0 con 0x7f32dc0039b0 2024-09-19T07:55:18.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.250+0000 7f32f37fe640 1 --2- 172.21.15.16:0/4091320872 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f32e006c2b0 0x7f32e006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:18.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.251+0000 7f32f37fe640 1 --2- 172.21.15.16:0/4091320872 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f32e006c2b0 0x7f32e006e770 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f32f40b3c00 tx=0x7f32e8007970 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:18.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.257+0000 7f32f0ff9640 1 -- 172.21.15.16:0/4091320872 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f32e40044e0 con 0x7f32dc0039b0 2024-09-19T07:55:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.514+0000 7f32fa5fd640 1 -- 172.21.15.16:0/4091320872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32f406c8b0 con 0x7f32dc0039b0 2024-09-19T07:55:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.515+0000 7f32f0ff9640 1 -- 172.21.15.16:0/4091320872 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5230 (secure 0 0 0) 0x7f32e4079e00 con 0x7f32dc0039b0 2024-09-19T07:55:18.515 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:18.516 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"btime":"2024-09-19T07:55:04:980435+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/172459502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":172459502},{"type":"v1","addr":"172.21.15.146:6836","nonce":172459502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3417061308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3417061308},{"type":"v1","addr":"172.21.15.146:6837","nonce":3417061308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1810646409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1810646409},{"type":"v1","addr":"172.21.15.16:6834","nonce":1810646409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:55:02.325256+0000","modified":"2024-09-19T07:55:04.318652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8286":{"gid":8286,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/1929965275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1929965275},{"type":"v1","addr":"172.21.15.16:6835","nonce":1929965275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8286,"qdb_cluster":[8286]},"id":10}]} 2024-09-19T07:55:18.516 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 81 2024-09-19T07:55:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.517+0000 7f32fa5fd640 1 -- 172.21.15.16:0/4091320872 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f32e006c2b0 msgr2=0x7f32e006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.517+0000 7f32fa5fd640 1 --2- 172.21.15.16:0/4091320872 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f32e006c2b0 0x7f32e006e770 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f32f40b3c00 tx=0x7f32e8007970 comp rx=0 tx=0).stop 2024-09-19T07:55:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.517+0000 7f32fa5fd640 1 -- 172.21.15.16:0/4091320872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32dc0039b0 msgr2=0x7f32f415b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.517+0000 7f32fa5fd640 1 --2- 172.21.15.16:0/4091320872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32dc0039b0 0x7f32f415b550 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f32e4009ac0 tx=0x7f32e4004730 comp rx=0 tx=0).stop 2024-09-19T07:55:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.517+0000 7f32fa5fd640 1 -- 172.21.15.16:0/4091320872 shutdown_connections 2024-09-19T07:55:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.518+0000 7f32fa5fd640 1 --2- 172.21.15.16:0/4091320872 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f32e006c2b0 0x7f32e006e770 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.518+0000 7f32fa5fd640 1 --2- 172.21.15.16:0/4091320872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32f40c73c0 0x7f32f415e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.518+0000 7f32fa5fd640 1 --2- 172.21.15.16:0/4091320872 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32f409ebc0 0x7f32f40c6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.518+0000 7f32fa5fd640 1 --2- 172.21.15.16:0/4091320872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32dc0039b0 0x7f32f415b550 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.518+0000 7f32fa5fd640 1 -- 172.21.15.16:0/4091320872 >> 172.21.15.16:0/4091320872 conn(0x7f32f40b2d10 msgr2=0x7f32f40b64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.518+0000 7f32fa5fd640 1 -- 172.21.15.16:0/4091320872 shutdown_connections 2024-09-19T07:55:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:18.518+0000 7f32fa5fd640 1 -- 172.21.15.16:0/4091320872 wait complete. 2024-09-19T07:55:18.531 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-19T07:55:18.667 INFO:teuthology.orchestra.run.smithi016.stdout:[{"id":7654,"entity":{"name":{"type":"client","num":7654},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2861814065}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":3.3669528729999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9901194202744286,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7654 192.168.0.1:0/2861814065","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"49748","root":"/"}},{"id":8484,"entity":{"name":{"type":"client","num":8484},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3786269155}},"state":"open","num_leases":253,"num_caps":254,"request_load_avg":757,"uptime":5.5069229210000001,"requests_in_flight":252,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":759.58403147378101,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":7,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8484 192.168.0.1:0/3786269155","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"0","hostname":"smithi016","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"76175","root":"/"}}] 2024-09-19T07:55:18.668 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "49748", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2861814065, "type": "any" }, "name": { "num": 7654, "type": "client" } }, "id": 7654, "inst": "client.7654 192.168.0.1:0/2861814065", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9901194202744286 }, "state": "open", "uptime": 3.366952873 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "0", "hostname": "smithi016", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "76175", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3786269155, "type": "any" }, "name": { "num": 8484, "type": "client" } }, "id": 8484, "inst": "client.8484 192.168.0.1:0/3786269155", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 7, "num_caps": 254, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 253, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 757, "requests_in_flight": 252, "session_cache_liveness": { "halflife": 300, "value": 759.584031473781 }, "state": "open", "uptime": 5.506922921 } ] 2024-09-19T07:55:18.668 DEBUG:tasks.ceph_test_case:wait_until_equal: 254 != 401, waiting (timeout=600)... 2024-09-19T07:55:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:waiting with handles open 2024-09-19T07:55:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:55:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:55:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:55:23.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:55:23.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.785+0000 7ff96e3fd640 1 Processor -- start 2024-09-19T07:55:23.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.785+0000 7ff96e3fd640 1 -- start start 2024-09-19T07:55:23.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.786+0000 7ff96e3fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff968141680 0x7ff9680cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:23.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.786+0000 7ff96e3fd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff9680d0370 0x7ff968141a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:23.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.786+0000 7ff96e3fd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff96815d710 con 0x7ff968141fa0 2024-09-19T07:55:23.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.786+0000 7ff96e3fd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff9680d07b0 con 0x7ff9680d0370 2024-09-19T07:55:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.786+0000 7ff96e3fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff96815c6f0 con 0x7ff968141680 2024-09-19T07:55:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.786+0000 7ff967fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff968141680 0x7ff9680cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.787+0000 7ff967fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff968141680 0x7ff9680cfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:49310/0 (socket says 172.21.15.16:49310) 2024-09-19T07:55:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.787+0000 7ff967fff640 1 -- 172.21.15.16:0/2448731400 learned_addr learned my addr 172.21.15.16:0/2448731400 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.787+0000 7ff9677fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff9680d0370 0x7ff968141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.787+0000 7ff96c99a640 1 --1- 172.21.15.16:0/2448731400 >> v1:172.21.15.146:6789/0 conn(0x7ff968141fa0 0x7ff9680cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:34570/0 (socket says 172.21.15.16:34570) 2024-09-19T07:55:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.787+0000 7ff966ffd640 1 -- 172.21.15.16:0/2448731400 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1305263112 0 0) 0x7ff96815d710 con 0x7ff968141fa0 2024-09-19T07:55:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.787+0000 7ff966ffd640 1 -- 172.21.15.16:0/2448731400 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff950003520 con 0x7ff968141fa0 2024-09-19T07:55:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.787+0000 7ff9677fe640 1 -- 172.21.15.16:0/2448731400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff968141680 msgr2=0x7ff9680cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.787+0000 7ff9677fe640 1 --2- 172.21.15.16:0/2448731400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff968141680 0x7ff9680cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.787+0000 7ff9677fe640 1 -- 172.21.15.16:0/2448731400 >> v1:172.21.15.146:6789/0 conn(0x7ff968141fa0 legacy=0x7ff9680cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:23.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.787+0000 7ff9677fe640 1 -- 172.21.15.16:0/2448731400 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96815c950 con 0x7ff9680d0370 2024-09-19T07:55:23.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.787+0000 7ff967fff640 1 --2- 172.21.15.16:0/2448731400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff968141680 0x7ff9680cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:55:23.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.788+0000 7ff9677fe640 1 --2- 172.21.15.16:0/2448731400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9680d0370 0x7ff968141a60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0097a0 tx=0x7ff95c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad6345c76b6a0136 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:23.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.788+0000 7ff966ffd640 1 -- 172.21.15.16:0/2448731400 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c037070 con 0x7ff9680d0370 2024-09-19T07:55:23.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.788+0000 7ff966ffd640 1 -- 172.21.15.16:0/2448731400 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c032440 con 0x7ff9680d0370 2024-09-19T07:55:23.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.788+0000 7ff966ffd640 1 -- 172.21.15.16:0/2448731400 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c032810 con 0x7ff9680d0370 2024-09-19T07:55:23.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.788+0000 7ff96e3fd640 1 -- 172.21.15.16:0/2448731400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9680d0370 msgr2=0x7ff968141a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:23.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.788+0000 7ff96e3fd640 1 --2- 172.21.15.16:0/2448731400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9680d0370 0x7ff968141a60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0097a0 tx=0x7ff95c02a180 comp rx=0 tx=0).stop 2024-09-19T07:55:23.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.789+0000 7ff96e3fd640 1 -- 172.21.15.16:0/2448731400 shutdown_connections 2024-09-19T07:55:23.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.789+0000 7ff96e3fd640 1 --2- 172.21.15.16:0/2448731400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9680d0370 0x7ff968141a60 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:23.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.789+0000 7ff96e3fd640 1 --2- 172.21.15.16:0/2448731400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff968141680 0x7ff9680cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:23.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.789+0000 7ff96e3fd640 1 -- 172.21.15.16:0/2448731400 >> 172.21.15.16:0/2448731400 conn(0x7ff9680b3ed0 msgr2=0x7ff9680b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:23.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.789+0000 7ff96e3fd640 1 -- 172.21.15.16:0/2448731400 shutdown_connections 2024-09-19T07:55:23.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.789+0000 7ff96e3fd640 1 -- 172.21.15.16:0/2448731400 wait complete. 2024-09-19T07:55:23.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.790+0000 7ff96e3fd640 1 Processor -- start 2024-09-19T07:55:23.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.790+0000 7ff96e3fd640 1 -- start start 2024-09-19T07:55:23.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.791+0000 7ff96e3fd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9680d0370 0x7ff96806d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.791+0000 7ff96e3fd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff968141680 0x7ff9681f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.791+0000 7ff96e3fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff968141fa0 0x7ff9681f9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.791+0000 7ff96e3fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff96815d1f0 con 0x7ff968141fa0 2024-09-19T07:55:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.791+0000 7ff96e3fd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff96815cf10 con 0x7ff968141680 2024-09-19T07:55:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.791+0000 7ff96e3fd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff9681408c0 con 0x7ff9680d0370 2024-09-19T07:55:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.791+0000 7ff96c99a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff968141fa0 0x7ff9681f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.791+0000 7ff96c99a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff968141fa0 0x7ff9681f9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41062/0 (socket says 172.21.15.16:41062) 2024-09-19T07:55:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.791+0000 7ff96c99a640 1 -- 172.21.15.16:0/4053633680 learned_addr learned my addr 172.21.15.16:0/4053633680 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.791+0000 7ff967fff640 1 --2- 172.21.15.16:0/4053633680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9680d0370 0x7ff96806d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.791+0000 7ff9677fe640 1 --2- 172.21.15.16:0/4053633680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff968141680 0x7ff9681f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.791+0000 7ff96c99a640 1 -- 172.21.15.16:0/4053633680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9680d0370 msgr2=0x7ff96806d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.791+0000 7ff96c99a640 1 --2- 172.21.15.16:0/4053633680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9680d0370 0x7ff96806d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.791+0000 7ff96c99a640 1 -- 172.21.15.16:0/4053633680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff968141680 msgr2=0x7ff9681f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.791+0000 7ff96c99a640 1 --2- 172.21.15.16:0/4053633680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff968141680 0x7ff9681f5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.791+0000 7ff96c99a640 1 -- 172.21.15.16:0/4053633680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96806de00 con 0x7ff968141fa0 2024-09-19T07:55:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.792+0000 7ff96c99a640 1 --2- 172.21.15.16:0/4053633680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff968141fa0 0x7ff9681f9010 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7ff96000c740 tx=0x7ff96000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.792+0000 7ff964ff9640 1 -- 172.21.15.16:0/4053633680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960017070 con 0x7ff968141fa0 2024-09-19T07:55:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.792+0000 7ff964ff9640 1 -- 172.21.15.16:0/4053633680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff96000ce50 con 0x7ff968141fa0 2024-09-19T07:55:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.792+0000 7ff96e3fd640 1 -- 172.21.15.16:0/4053633680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96806e0e0 con 0x7ff968141fa0 2024-09-19T07:55:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.792+0000 7ff964ff9640 1 -- 172.21.15.16:0/4053633680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96000a860 con 0x7ff968141fa0 2024-09-19T07:55:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.792+0000 7ff96e3fd640 1 -- 172.21.15.16:0/4053633680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96806e6a0 con 0x7ff968141fa0 2024-09-19T07:55:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.793+0000 7ff96e3fd640 1 -- 172.21.15.16:0/4053633680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff968142430 con 0x7ff968141fa0 2024-09-19T07:55:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.793+0000 7ff964ff9640 1 -- 172.21.15.16:0/4053633680 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff96001a070 con 0x7ff968141fa0 2024-09-19T07:55:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.794+0000 7ff964ff9640 1 --2- 172.21.15.16:0/4053633680 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff95406c2b0 0x7ff95406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.794+0000 7ff964ff9640 1 -- 172.21.15.16:0/4053633680 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7ff96008ac40 con 0x7ff968141fa0 2024-09-19T07:55:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.794+0000 7ff967fff640 1 --2- 172.21.15.16:0/4053633680 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff95406c2b0 0x7ff95406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.795+0000 7ff967fff640 1 --2- 172.21.15.16:0/4053633680 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff95406c2b0 0x7ff95406e770 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff958009b70 tx=0x7ff958008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:23.798+0000 7ff964ff9640 1 -- 172.21.15.16:0/4053633680 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff960059220 con 0x7ff968141fa0 2024-09-19T07:55:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.012+0000 7ff96e3fd640 1 -- 172.21.15.16:0/4053633680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9680690c0 con 0x7ff968141fa0 2024-09-19T07:55:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.012+0000 7ff964ff9640 1 -- 172.21.15.16:0/4053633680 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5230 (secure 0 0 0) 0x7ff960058b20 con 0x7ff968141fa0 2024-09-19T07:55:24.012 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:24.013 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":83,"btime":"2024-09-19T07:55:23:288383+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/172459502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":172459502},{"type":"v1","addr":"172.21.15.146:6836","nonce":172459502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3417061308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3417061308},{"type":"v1","addr":"172.21.15.146:6837","nonce":3417061308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1810646409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1810646409},{"type":"v1","addr":"172.21.15.16:6834","nonce":1810646409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:55:02.325256+0000","modified":"2024-09-19T07:55:23.002436+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8286":{"gid":8286,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/1929965275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1929965275},{"type":"v1","addr":"172.21.15.16:6835","nonce":1929965275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8286,"qdb_cluster":[8286]},"id":10}]} 2024-09-19T07:55:24.013 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 83 2024-09-19T07:55:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.014+0000 7ff96e3fd640 1 -- 172.21.15.16:0/4053633680 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff95406c2b0 msgr2=0x7ff95406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.014+0000 7ff96e3fd640 1 --2- 172.21.15.16:0/4053633680 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff95406c2b0 0x7ff95406e770 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff958009b70 tx=0x7ff958008040 comp rx=0 tx=0).stop 2024-09-19T07:55:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.014+0000 7ff96e3fd640 1 -- 172.21.15.16:0/4053633680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff968141fa0 msgr2=0x7ff9681f9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.014+0000 7ff96e3fd640 1 --2- 172.21.15.16:0/4053633680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff968141fa0 0x7ff9681f9010 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7ff96000c740 tx=0x7ff96000cc00 comp rx=0 tx=0).stop 2024-09-19T07:55:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.014+0000 7ff96e3fd640 1 -- 172.21.15.16:0/4053633680 shutdown_connections 2024-09-19T07:55:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.014+0000 7ff96e3fd640 1 --2- 172.21.15.16:0/4053633680 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff95406c2b0 0x7ff95406e770 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.014+0000 7ff96e3fd640 1 --2- 172.21.15.16:0/4053633680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff968141fa0 0x7ff9681f9010 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.014+0000 7ff96e3fd640 1 --2- 172.21.15.16:0/4053633680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff968141680 0x7ff9681f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.014+0000 7ff96e3fd640 1 --2- 172.21.15.16:0/4053633680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9680d0370 0x7ff96806d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.014+0000 7ff96e3fd640 1 -- 172.21.15.16:0/4053633680 >> 172.21.15.16:0/4053633680 conn(0x7ff9680b3ed0 msgr2=0x7ff9680cd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:24.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.015+0000 7ff96e3fd640 1 -- 172.21.15.16:0/4053633680 shutdown_connections 2024-09-19T07:55:24.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.015+0000 7ff96e3fd640 1 -- 172.21.15.16:0/4053633680 wait complete. 2024-09-19T07:55:24.026 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-19T07:55:24.146 INFO:teuthology.orchestra.run.smithi016.stdout:[{"id":7654,"entity":{"name":{"type":"client","num":7654},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2861814065}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":8.846876172,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9524992504355252,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7654 192.168.0.1:0/2861814065","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"49748","root":"/"}},{"id":8484,"entity":{"name":{"type":"client","num":8484},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3786269155}},"state":"open","num_leases":400,"num_caps":401,"request_load_avg":1126,"uptime":10.98684622,"requests_in_flight":0,"num_completed_requests":399,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":1164.071803139562,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":473.33459924190254,"halflife":1.5},"recall_caps_throttle2o":{"value":217.44276769443005,"halflife":0.5},"session_cache_liveness":{"value":1185.6038318100507,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":7,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8484 192.168.0.1:0/3786269155","completed_requests":[{"tid":7,"created_ino":"0x10000000001"},{"tid":10,"created_ino":"0x100000001f6"},{"tid":13,"created_ino":"0x100000001f7"},{"tid":16,"created_ino":"0x100000001f8"},{"tid":19,"created_ino":"0x100000001f9"},{"tid":22,"created_ino":"0x100000001fa"},{"tid":25,"created_ino":"0x100000001fb"},{"tid":28,"created_ino":"0x100000001fc"},{"tid":31,"created_ino":"0x100000001fd"},{"tid":34,"created_ino":"0x100000001fe"},{"tid":37,"created_ino":"0x100000001ff"},{"tid":40,"created_ino":"0x10000000200"},{"tid":43,"created_ino":"0x10000000201"},{"tid":46,"created_ino":"0x10000000202"},{"tid":49,"created_ino":"0x10000000203"},{"tid":52,"created_ino":"0x10000000204"},{"tid":55,"created_ino":"0x10000000205"},{"tid":58,"created_ino":"0x10000000206"},{"tid":61,"created_ino":"0x10000000207"},{"tid":64,"created_ino":"0x10000000208"},{"tid":67,"created_ino":"0x10000000209"},{"tid":70,"created_ino":"0x1000000020a"},{"tid":73,"created_ino":"0x1000000020b"},{"tid":76,"created_ino":"0x1000000020c"},{"tid":79,"created_ino":"0x1000000020d"},{"tid":82,"created_ino":"0x1000000020e"},{"tid":85,"created_ino":"0x1000000020f"},{"tid":88,"created_ino":"0x10000000210"},{"tid":91,"created_ino":"0x10000000211"},{"tid":94,"created_ino":"0x10000000212"},{"tid":97,"created_ino":"0x10000000213"},{"tid":100,"created_ino":"0x10000000214"},{"tid":103,"created_ino":"0x10000000215"},{"tid":106,"created_ino":"0x10000000216"},{"tid":109,"created_ino":"0x10000000217"},{"tid":112,"created_ino":"0x10000000218"},{"tid":115,"created_ino":"0x10000000219"},{"tid":118,"created_ino":"0x1000000021a"},{"tid":121,"created_ino":"0x1000000021b"},{"tid":124,"created_ino":"0x1000000021c"},{"tid":127,"created_ino":"0x1000000021d"},{"tid":130,"created_ino":"0x1000000021e"},{"tid":133,"created_ino":"0x1000000021f"},{"tid":136,"created_ino":"0x10000000220"},{"tid":139,"created_ino":"0x10000000221"},{"tid":142,"created_ino":"0x10000000222"},{"tid":145,"created_ino":"0x10000000223"},{"tid":148,"created_ino":"0x10000000224"},{"tid":151,"created_ino":"0x10000000225"},{"tid":154,"created_ino":"0x10000000226"},{"tid":157,"created_ino":"0x10000000227"},{"tid":160,"created_ino":"0x10000000228"},{"tid":163,"created_ino":"0x10000000229"},{"tid":166,"created_ino":"0x1000000022a"},{"tid":169,"created_ino":"0x1000000022b"},{"tid":172,"created_ino":"0x1000000022c"},{"tid":175,"created_ino":"0x1000000022d"},{"tid":178,"created_ino":"0x1000000022e"},{"tid":181,"created_ino":"0x1000000022f"},{"tid":184,"created_ino":"0x10000000230"},{"tid":187,"created_ino":"0x10000000231"},{"tid":190,"created_ino":"0x10000000232"},{"tid":193,"created_ino":"0x10000000233"},{"tid":196,"created_ino":"0x10000000234"},{"tid":199,"created_ino":"0x10000000235"},{"tid":202,"created_ino":"0x10000000236"},{"tid":205,"created_ino":"0x10000000237"},{"tid":208,"created_ino":"0x10000000238"},{"tid":211,"created_ino":"0x10000000239"},{"tid":214,"created_ino":"0x1000000023a"},{"tid":217,"created_ino":"0x1000000023b"},{"tid":220,"created_ino":"0x1000000023c"},{"tid":223,"created_ino":"0x1000000023d"},{"tid":226,"created_ino":"0x1000000023e"},{"tid":229,"created_ino":"0x1000000023f"},{"tid":232,"created_ino":"0x10000000240"},{"tid":235,"created_ino":"0x10000000241"},{"tid":238,"created_ino":"0x10000000242"},{"tid":241,"created_ino":"0x10000000243"},{"tid":244,"created_ino":"0x10000000244"},{"tid":247,"created_ino":"0x10000000245"},{"tid":250,"created_ino":"0x10000000246"},{"tid":253,"created_ino":"0x10000000247"},{"tid":256,"created_ino":"0x10000000248"},{"tid":259,"created_ino":"0x10000000249"},{"tid":262,"created_ino":"0x1000000024a"},{"tid":265,"created_ino":"0x1000000024b"},{"tid":268,"created_ino":"0x1000000024c"},{"tid":271,"created_ino":"0x1000000024d"},{"tid":274,"created_ino":"0x1000000024e"},{"tid":277,"created_ino":"0x1000000024f"},{"tid":280,"created_ino":"0x10000000250"},{"tid":283,"created_ino":"0x10000000251"},{"tid":286,"created_ino":"0x10000000252"},{"tid":289,"created_ino":"0x10000000253"},{"tid":292,"created_ino":"0x10000000254"},{"tid":295,"created_ino":"0x10000000255"},{"tid":298,"created_ino":"0x10000000256"},{"tid":301,"created_ino":"0x10000000257"},{"tid":304,"created_ino":"0x10000000258"},{"tid":307,"created_ino":"0x10000000259"},{"tid":310,"created_ino":"0x1000000025a"},{"tid":313,"created_ino":"0x1000000025b"},{"tid":316,"created_ino":"0x1000000025c"},{"tid":319,"created_ino":"0x1000000025d"},{"tid":322,"created_ino":"0x1000000025e"},{"tid":325,"created_ino":"0x1000000025f"},{"tid":328,"created_ino":"0x10000000260"},{"tid":331,"created_ino":"0x10000000261"},{"tid":334,"created_ino":"0x10000000262"},{"tid":337,"created_ino":"0x10000000263"},{"tid":340,"created_ino":"0x10000000264"},{"tid":343,"created_ino":"0x10000000265"},{"tid":346,"created_ino":"0x10000000266"},{"tid":349,"created_ino":"0x10000000267"},{"tid":352,"created_ino":"0x10000000268"},{"tid":355,"created_ino":"0x10000000269"},{"tid":358,"created_ino":"0x1000000026a"},{"tid":361,"created_ino":"0x1000000026b"},{"tid":364,"created_ino":"0x1000000026c"},{"tid":367,"created_ino":"0x1000000026d"},{"tid":370,"created_ino":"0x1000000026e"},{"tid":373,"created_ino":"0x1000000026f"},{"tid":376,"created_ino":"0x10000000270"},{"tid":379,"created_ino":"0x10000000271"},{"tid":382,"created_ino":"0x10000000272"},{"tid":385,"created_ino":"0x10000000273"},{"tid":388,"created_ino":"0x10000000274"},{"tid":391,"created_ino":"0x10000000275"},{"tid":394,"created_ino":"0x10000000276"},{"tid":397,"created_ino":"0x10000000277"},{"tid":400,"created_ino":"0x10000000278"},{"tid":403,"created_ino":"0x10000000279"},{"tid":406,"created_ino":"0x1000000027a"},{"tid":409,"created_ino":"0x1000000027b"},{"tid":412,"created_ino":"0x1000000027c"},{"tid":415,"created_ino":"0x1000000027d"},{"tid":418,"created_ino":"0x1000000027e"},{"tid":421,"created_ino":"0x1000000027f"},{"tid":424,"created_ino":"0x10000000280"},{"tid":427,"created_ino":"0x10000000281"},{"tid":430,"created_ino":"0x10000000282"},{"tid":433,"created_ino":"0x10000000283"},{"tid":436,"created_ino":"0x10000000284"},{"tid":439,"created_ino":"0x10000000285"},{"tid":442,"created_ino":"0x10000000286"},{"tid":445,"created_ino":"0x10000000287"},{"tid":448,"created_ino":"0x10000000288"},{"tid":451,"created_ino":"0x10000000289"},{"tid":454,"created_ino":"0x1000000028a"},{"tid":457,"created_ino":"0x1000000028b"},{"tid":460,"created_ino":"0x1000000028c"},{"tid":463,"created_ino":"0x1000000028d"},{"tid":466,"created_ino":"0x1000000028e"},{"tid":469,"created_ino":"0x1000000028f"},{"tid":472,"created_ino":"0x10000000290"},{"tid":475,"created_ino":"0x10000000291"},{"tid":478,"created_ino":"0x10000000292"},{"tid":481,"created_ino":"0x10000000293"},{"tid":484,"created_ino":"0x10000000294"},{"tid":487,"created_ino":"0x10000000295"},{"tid":490,"created_ino":"0x10000000296"},{"tid":493,"created_ino":"0x10000000297"},{"tid":496,"created_ino":"0x10000000298"},{"tid":499,"created_ino":"0x10000000299"},{"tid":502,"created_ino":"0x1000000029a"},{"tid":505,"created_ino":"0x1000000029b"},{"tid":508,"created_ino":"0x1000000029c"},{"tid":511,"created_ino":"0x1000000029d"},{"tid":514,"created_ino":"0x1000000029e"},{"tid":517,"created_ino":"0x1000000029f"},{"tid":520,"created_ino":"0x100000002a0"},{"tid":523,"created_ino":"0x100000002a1"},{"tid":526,"created_ino":"0x100000002a2"},{"tid":529,"created_ino":"0x100000002a3"},{"tid":532,"created_ino":"0x100000002a4"},{"tid":535,"created_ino":"0x100000002a5"},{"tid":538,"created_ino":"0x100000002a6"},{"tid":541,"created_ino":"0x100000002a7"},{"tid":544,"created_ino":"0x100000002a8"},{"tid":547,"created_ino":"0x100000002a9"},{"tid":550,"created_ino":"0x100000002aa"},{"tid":553,"created_ino":"0x100000002ab"},{"tid":556,"created_ino":"0x100000002ac"},{"tid":559,"created_ino":"0x100000002ad"},{"tid":562,"created_ino":"0x100000002ae"},{"tid":565,"created_ino":"0x100000002af"},{"tid":568,"created_ino":"0x100000002b0"},{"tid":571,"created_ino":"0x100000002b1"},{"tid":574,"created_ino":"0x100000002b2"},{"tid":577,"created_ino":"0x100000002b3"},{"tid":580,"created_ino":"0x100000002b4"},{"tid":583,"created_ino":"0x100000002b5"},{"tid":586,"created_ino":"0x100000002b6"},{"tid":589,"created_ino":"0x100000002b7"},{"tid":592,"created_ino":"0x100000002b8"},{"tid":595,"created_ino":"0x100000002b9"},{"tid":598,"created_ino":"0x100000002ba"},{"tid":601,"created_ino":"0x100000002bb"},{"tid":604,"created_ino":"0x100000002bc"},{"tid":607,"created_ino":"0x100000002bd"},{"tid":610,"created_ino":"0x100000002be"},{"tid":613,"created_ino":"0x100000002bf"},{"tid":616,"created_ino":"0x100000002c0"},{"tid":619,"created_ino":"0x100000002c1"},{"tid":622,"created_ino":"0x100000002c2"},{"tid":625,"created_ino":"0x100000002c3"},{"tid":628,"created_ino":"0x100000002c4"},{"tid":631,"created_ino":"0x100000002c5"},{"tid":634,"created_ino":"0x100000002c6"},{"tid":637,"created_ino":"0x100000002c7"},{"tid":640,"created_ino":"0x100000002c8"},{"tid":643,"created_ino":"0x100000002c9"},{"tid":646,"created_ino":"0x100000002ca"},{"tid":649,"created_ino":"0x100000002cb"},{"tid":652,"created_ino":"0x100000002cc"},{"tid":655,"created_ino":"0x100000002cd"},{"tid":658,"created_ino":"0x100000002ce"},{"tid":661,"created_ino":"0x100000002cf"},{"tid":664,"created_ino":"0x100000002d0"},{"tid":667,"created_ino":"0x100000002d1"},{"tid":670,"created_ino":"0x100000002d2"},{"tid":673,"created_ino":"0x100000002d3"},{"tid":676,"created_ino":"0x100000002d4"},{"tid":679,"created_ino":"0x100000002d5"},{"tid":682,"created_ino":"0x100000002d6"},{"tid":685,"created_ino":"0x100000002d7"},{"tid":688,"created_ino":"0x100000002d8"},{"tid":691,"created_ino":"0x100000002d9"},{"tid":694,"created_ino":"0x100000002da"},{"tid":697,"created_ino":"0x100000002db"},{"tid":700,"created_ino":"0x100000002dc"},{"tid":703,"created_ino":"0x100000002dd"},{"tid":706,"created_ino":"0x100000002de"},{"tid":709,"created_ino":"0x100000002df"},{"tid":712,"created_ino":"0x100000002e0"},{"tid":715,"created_ino":"0x100000002e1"},{"tid":718,"created_ino":"0x100000002e2"},{"tid":721,"created_ino":"0x100000002e3"},{"tid":724,"created_ino":"0x100000002e4"},{"tid":727,"created_ino":"0x100000002e5"},{"tid":730,"created_ino":"0x100000002e6"},{"tid":733,"created_ino":"0x100000002e7"},{"tid":736,"created_ino":"0x100000002e8"},{"tid":739,"created_ino":"0x100000002e9"},{"tid":742,"created_ino":"0x100000002ea"},{"tid":745,"created_ino":"0x100000002eb"},{"tid":748,"created_ino":"0x100000002ec"},{"tid":751,"created_ino":"0x100000002ed"},{"tid":754,"created_ino":"0x100000002ee"},{"tid":757,"created_ino":"0x100000002ef"},{"tid":760,"created_ino":"0x100000002f0"},{"tid":763,"created_ino":"0x100000002f1"},{"tid":766,"created_ino":"0x100000002f2"},{"tid":769,"created_ino":"0x100000002f3"},{"tid":772,"created_ino":"0x100000002f4"},{"tid":775,"created_ino":"0x100000002f5"},{"tid":778,"created_ino":"0x100000002f6"},{"tid":781,"created_ino":"0x100000002f7"},{"tid":784,"created_ino":"0x100000002f8"},{"tid":787,"created_ino":"0x100000002f9"},{"tid":790,"created_ino":"0x100000002fa"},{"tid":793,"created_ino":"0x100000002fb"},{"tid":796,"created_ino":"0x100000002fc"},{"tid":799,"created_ino":"0x100000002fd"},{"tid":802,"created_ino":"0x100000002fe"},{"tid":805,"created_ino":"0x100000002ff"},{"tid":808,"created_ino":"0x10000000300"},{"tid":811,"created_ino":"0x10000000301"},{"tid":814,"created_ino":"0x10000000302"},{"tid":817,"created_ino":"0x10000000303"},{"tid":820,"created_ino":"0x10000000304"},{"tid":823,"created_ino":"0x10000000305"},{"tid":826,"created_ino":"0x10000000306"},{"tid":829,"created_ino":"0x10000000307"},{"tid":832,"created_ino":"0x10000000308"},{"tid":835,"created_ino":"0x10000000309"},{"tid":838,"created_ino":"0x1000000030a"},{"tid":841,"created_ino":"0x1000000030b"},{"tid":844,"created_ino":"0x1000000030c"},{"tid":847,"created_ino":"0x1000000030d"},{"tid":850,"created_ino":"0x1000000030e"},{"tid":853,"created_ino":"0x1000000030f"},{"tid":856,"created_ino":"0x10000000310"},{"tid":859,"created_ino":"0x10000000311"},{"tid":862,"created_ino":"0x10000000312"},{"tid":865,"created_ino":"0x10000000313"},{"tid":868,"created_ino":"0x10000000314"},{"tid":871,"created_ino":"0x10000000315"},{"tid":874,"created_ino":"0x10000000316"},{"tid":877,"created_ino":"0x10000000317"},{"tid":880,"created_ino":"0x10000000318"},{"tid":883,"created_ino":"0x10000000319"},{"tid":886,"created_ino":"0x1000000031a"},{"tid":889,"created_ino":"0x1000000031b"},{"tid":892,"created_ino":"0x1000000031c"},{"tid":895,"created_ino":"0x1000000031d"},{"tid":898,"created_ino":"0x1000000031e"},{"tid":901,"created_ino":"0x1000000031f"},{"tid":904,"created_ino":"0x10000000320"},{"tid":907,"created_ino":"0x10000000321"},{"tid":910,"created_ino":"0x10000000322"},{"tid":913,"created_ino":"0x10000000323"},{"tid":916,"created_ino":"0x10000000324"},{"tid":919,"created_ino":"0x10000000325"},{"tid":922,"created_ino":"0x10000000326"},{"tid":925,"created_ino":"0x10000000327"},{"tid":928,"created_ino":"0x10000000328"},{"tid":931,"created_ino":"0x10000000329"},{"tid":934,"created_ino":"0x1000000032a"},{"tid":937,"created_ino":"0x1000000032b"},{"tid":940,"created_ino":"0x1000000032c"},{"tid":943,"created_ino":"0x1000000032d"},{"tid":946,"created_ino":"0x1000000032e"},{"tid":949,"created_ino":"0x1000000032f"},{"tid":952,"created_ino":"0x10000000330"},{"tid":955,"created_ino":"0x10000000331"},{"tid":958,"created_ino":"0x10000000332"},{"tid":961,"created_ino":"0x10000000333"},{"tid":964,"created_ino":"0x10000000334"},{"tid":967,"created_ino":"0x10000000335"},{"tid":970,"created_ino":"0x10000000336"},{"tid":973,"created_ino":"0x10000000337"},{"tid":976,"created_ino":"0x10000000338"},{"tid":979,"created_ino":"0x10000000339"},{"tid":982,"created_ino":"0x1000000033a"},{"tid":985,"created_ino":"0x1000000033b"},{"tid":988,"created_ino":"0x1000000033c"},{"tid":991,"created_ino":"0x1000000033d"},{"tid":994,"created_ino":"0x1000000033e"},{"tid":997,"created_ino":"0x1000000033f"},{"tid":1000,"created_ino":"0x10000000340"},{"tid":1003,"created_ino":"0x10000000341"},{"tid":1006,"created_ino":"0x10000000342"},{"tid":1009,"created_ino":"0x10000000343"},{"tid":1012,"created_ino":"0x10000000344"},{"tid":1015,"created_ino":"0x10000000345"},{"tid":1018,"created_ino":"0x10000000346"},{"tid":1021,"created_ino":"0x10000000347"},{"tid":1024,"created_ino":"0x10000000348"},{"tid":1027,"created_ino":"0x10000000349"},{"tid":1030,"created_ino":"0x1000000034a"},{"tid":1033,"created_ino":"0x1000000034b"},{"tid":1036,"created_ino":"0x1000000034c"},{"tid":1039,"created_ino":"0x1000000034d"},{"tid":1042,"created_ino":"0x1000000034e"},{"tid":1045,"created_ino":"0x1000000034f"},{"tid":1048,"created_ino":"0x10000000350"},{"tid":1051,"created_ino":"0x10000000351"},{"tid":1054,"created_ino":"0x10000000352"},{"tid":1057,"created_ino":"0x10000000353"},{"tid":1060,"created_ino":"0x10000000354"},{"tid":1063,"created_ino":"0x10000000355"},{"tid":1066,"created_ino":"0x10000000356"},{"tid":1069,"created_ino":"0x10000000357"},{"tid":1072,"created_ino":"0x10000000358"},{"tid":1075,"created_ino":"0x10000000359"},{"tid":1078,"created_ino":"0x1000000035a"},{"tid":1081,"created_ino":"0x1000000035b"},{"tid":1084,"created_ino":"0x1000000035c"},{"tid":1087,"created_ino":"0x1000000035d"},{"tid":1090,"created_ino":"0x1000000035e"},{"tid":1093,"created_ino":"0x1000000035f"},{"tid":1096,"created_ino":"0x10000000360"},{"tid":1099,"created_ino":"0x10000000361"},{"tid":1102,"created_ino":"0x10000000362"},{"tid":1105,"created_ino":"0x10000000363"},{"tid":1108,"created_ino":"0x10000000364"},{"tid":1111,"created_ino":"0x10000000365"},{"tid":1114,"created_ino":"0x10000000366"},{"tid":1117,"created_ino":"0x10000000367"},{"tid":1120,"created_ino":"0x10000000368"},{"tid":1123,"created_ino":"0x10000000369"},{"tid":1126,"created_ino":"0x1000000036a"},{"tid":1129,"created_ino":"0x1000000036b"},{"tid":1132,"created_ino":"0x1000000036c"},{"tid":1135,"created_ino":"0x1000000036d"},{"tid":1138,"created_ino":"0x1000000036e"},{"tid":1141,"created_ino":"0x1000000036f"},{"tid":1144,"created_ino":"0x10000000370"},{"tid":1147,"created_ino":"0x10000000371"},{"tid":1150,"created_ino":"0x10000000372"},{"tid":1153,"created_ino":"0x10000000373"},{"tid":1156,"created_ino":"0x10000000374"},{"tid":1159,"created_ino":"0x10000000375"},{"tid":1162,"created_ino":"0x10000000376"},{"tid":1165,"created_ino":"0x10000000377"},{"tid":1168,"created_ino":"0x10000000378"},{"tid":1171,"created_ino":"0x10000000379"},{"tid":1174,"created_ino":"0x1000000037a"},{"tid":1177,"created_ino":"0x1000000037b"},{"tid":1180,"created_ino":"0x1000000037c"},{"tid":1183,"created_ino":"0x1000000037d"},{"tid":1186,"created_ino":"0x1000000037e"},{"tid":1189,"created_ino":"0x1000000037f"},{"tid":1192,"created_ino":"0x10000000380"},{"tid":1195,"created_ino":"0x10000000381"},{"tid":1198,"created_ino":"0x10000000382"},{"tid":1201,"created_ino":"0x10000000383"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000384","length":603}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"0","hostname":"smithi016","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"76175","root":"/"}}] 2024-09-19T07:55:24.149 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "49748", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2861814065, "type": "any" }, "name": { "num": 7654, "type": "client" } }, "id": 7654, "inst": "client.7654 192.168.0.1:0/2861814065", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.952499250435525 }, "state": "open", "uptime": 8.846876172 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "0", "hostname": "smithi016", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "76175", "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000001", "tid": 7 }, { "created_ino": "0x100000001f6", "tid": 10 }, { "created_ino": "0x100000001f7", "tid": 13 }, { "created_ino": "0x100000001f8", "tid": 16 }, { "created_ino": "0x100000001f9", "tid": 19 }, { "created_ino": "0x100000001fa", "tid": 22 }, { "created_ino": "0x100000001fb", "tid": 25 }, { "created_ino": "0x100000001fc", "tid": 28 }, { "created_ino": "0x100000001fd", "tid": 31 }, { "created_ino": "0x100000001fe", "tid": 34 }, { "created_ino": "0x100000001ff", "tid": 37 }, { "created_ino": "0x10000000200", "tid": 40 }, { "created_ino": "0x10000000201", "tid": 43 }, { "created_ino": "0x10000000202", "tid": 46 }, { "created_ino": "0x10000000203", "tid": 49 }, { "created_ino": "0x10000000204", "tid": 52 }, { "created_ino": "0x10000000205", "tid": 55 }, { "created_ino": "0x10000000206", "tid": 58 }, { "created_ino": "0x10000000207", "tid": 61 }, { "created_ino": "0x10000000208", "tid": 64 }, { "created_ino": "0x10000000209", "tid": 67 }, { "created_ino": "0x1000000020a", "tid": 70 }, { "created_ino": "0x1000000020b", "tid": 73 }, { "created_ino": "0x1000000020c", "tid": 76 }, { "created_ino": "0x1000000020d", "tid": 79 }, { "created_ino": "0x1000000020e", "tid": 82 }, { "created_ino": "0x1000000020f", "tid": 85 }, { "created_ino": "0x10000000210", "tid": 88 }, { "created_ino": "0x10000000211", "tid": 91 }, { "created_ino": "0x10000000212", "tid": 94 }, { "created_ino": "0x10000000213", "tid": 97 }, { "created_ino": "0x10000000214", "tid": 100 }, { "created_ino": "0x10000000215", "tid": 103 }, { "created_ino": "0x10000000216", "tid": 106 }, { "created_ino": "0x10000000217", "tid": 109 }, { "created_ino": "0x10000000218", "tid": 112 }, { "created_ino": "0x10000000219", "tid": 115 }, { "created_ino": "0x1000000021a", "tid": 118 }, { "created_ino": "0x1000000021b", "tid": 121 }, { "created_ino": "0x1000000021c", "tid": 124 }, { "created_ino": "0x1000000021d", "tid": 127 }, { "created_ino": "0x1000000021e", "tid": 130 }, { "created_ino": "0x1000000021f", "tid": 133 }, { "created_ino": "0x10000000220", "tid": 136 }, { "created_ino": "0x10000000221", "tid": 139 }, { "created_ino": "0x10000000222", "tid": 142 }, { "created_ino": "0x10000000223", "tid": 145 }, { "created_ino": "0x10000000224", "tid": 148 }, { "created_ino": "0x10000000225", "tid": 151 }, { "created_ino": "0x10000000226", "tid": 154 }, { "created_ino": "0x10000000227", "tid": 157 }, { "created_ino": "0x10000000228", "tid": 160 }, { "created_ino": "0x10000000229", "tid": 163 }, { "created_ino": "0x1000000022a", "tid": 166 }, { "created_ino": "0x1000000022b", "tid": 169 }, { "created_ino": "0x1000000022c", "tid": 172 }, { "created_ino": "0x1000000022d", "tid": 175 }, { "created_ino": "0x1000000022e", "tid": 178 }, { "created_ino": "0x1000000022f", "tid": 181 }, { "created_ino": "0x10000000230", "tid": 184 }, { "created_ino": "0x10000000231", "tid": 187 }, { "created_ino": "0x10000000232", "tid": 190 }, { "created_ino": "0x10000000233", "tid": 193 }, { "created_ino": "0x10000000234", "tid": 196 }, { "created_ino": "0x10000000235", "tid": 199 }, { "created_ino": "0x10000000236", "tid": 202 }, { "created_ino": "0x10000000237", "tid": 205 }, { "created_ino": "0x10000000238", "tid": 208 }, { "created_ino": "0x10000000239", "tid": 211 }, { "created_ino": "0x1000000023a", "tid": 214 }, { "created_ino": "0x1000000023b", "tid": 217 }, { "created_ino": "0x1000000023c", "tid": 220 }, { "created_ino": "0x1000000023d", "tid": 223 }, { "created_ino": "0x1000000023e", "tid": 226 }, { "created_ino": "0x1000000023f", "tid": 229 }, { "created_ino": "0x10000000240", "tid": 232 }, { "created_ino": "0x10000000241", "tid": 235 }, { "created_ino": "0x10000000242", "tid": 238 }, { "created_ino": "0x10000000243", "tid": 241 }, { "created_ino": "0x10000000244", "tid": 244 }, { "created_ino": "0x10000000245", "tid": 247 }, { "created_ino": "0x10000000246", "tid": 250 }, { "created_ino": "0x10000000247", "tid": 253 }, { "created_ino": "0x10000000248", "tid": 256 }, { "created_ino": "0x10000000249", "tid": 259 }, { "created_ino": "0x1000000024a", "tid": 262 }, { "created_ino": "0x1000000024b", "tid": 265 }, { "created_ino": "0x1000000024c", "tid": 268 }, { "created_ino": "0x1000000024d", "tid": 271 }, { "created_ino": "0x1000000024e", "tid": 274 }, { "created_ino": "0x1000000024f", "tid": 277 }, { "created_ino": "0x10000000250", "tid": 280 }, { "created_ino": "0x10000000251", "tid": 283 }, { "created_ino": "0x10000000252", "tid": 286 }, { "created_ino": "0x10000000253", "tid": 289 }, { "created_ino": "0x10000000254", "tid": 292 }, { "created_ino": "0x10000000255", "tid": 295 }, { "created_ino": "0x10000000256", "tid": 298 }, { "created_ino": "0x10000000257", "tid": 301 }, { "created_ino": "0x10000000258", "tid": 304 }, { "created_ino": "0x10000000259", "tid": 307 }, { "created_ino": "0x1000000025a", "tid": 310 }, { "created_ino": "0x1000000025b", "tid": 313 }, { "created_ino": "0x1000000025c", "tid": 316 }, { "created_ino": "0x1000000025d", "tid": 319 }, { "created_ino": "0x1000000025e", "tid": 322 }, { "created_ino": "0x1000000025f", "tid": 325 }, { "created_ino": "0x10000000260", "tid": 328 }, { "created_ino": "0x10000000261", "tid": 331 }, { "created_ino": "0x10000000262", "tid": 334 }, { "created_ino": "0x10000000263", "tid": 337 }, { "created_ino": "0x10000000264", "tid": 340 }, { "created_ino": "0x10000000265", "tid": 343 }, { "created_ino": "0x10000000266", "tid": 346 }, { "created_ino": "0x10000000267", "tid": 349 }, { "created_ino": "0x10000000268", "tid": 352 }, { "created_ino": "0x10000000269", "tid": 355 }, { "created_ino": "0x1000000026a", "tid": 358 }, { "created_ino": "0x1000000026b", "tid": 361 }, { "created_ino": "0x1000000026c", "tid": 364 }, { "created_ino": "0x1000000026d", "tid": 367 }, { "created_ino": "0x1000000026e", "tid": 370 }, { "created_ino": "0x1000000026f", "tid": 373 }, { "created_ino": "0x10000000270", "tid": 376 }, { "created_ino": "0x10000000271", "tid": 379 }, { "created_ino": "0x10000000272", "tid": 382 }, { "created_ino": "0x10000000273", "tid": 385 }, { "created_ino": "0x10000000274", "tid": 388 }, { "created_ino": "0x10000000275", "tid": 391 }, { "created_ino": "0x10000000276", "tid": 394 }, { "created_ino": "0x10000000277", "tid": 397 }, { "created_ino": "0x10000000278", "tid": 400 }, { "created_ino": "0x10000000279", "tid": 403 }, { "created_ino": "0x1000000027a", "tid": 406 }, { "created_ino": "0x1000000027b", "tid": 409 }, { "created_ino": "0x1000000027c", "tid": 412 }, { "created_ino": "0x1000000027d", "tid": 415 }, { "created_ino": "0x1000000027e", "tid": 418 }, { "created_ino": "0x1000000027f", "tid": 421 }, { "created_ino": "0x10000000280", "tid": 424 }, { "created_ino": "0x10000000281", "tid": 427 }, { "created_ino": "0x10000000282", "tid": 430 }, { "created_ino": "0x10000000283", "tid": 433 }, { "created_ino": "0x10000000284", "tid": 436 }, { "created_ino": "0x10000000285", "tid": 439 }, { "created_ino": "0x10000000286", "tid": 442 }, { "created_ino": "0x10000000287", "tid": 445 }, { "created_ino": "0x10000000288", "tid": 448 }, { "created_ino": "0x10000000289", "tid": 451 }, { "created_ino": "0x1000000028a", "tid": 454 }, { "created_ino": "0x1000000028b", "tid": 457 }, { "created_ino": "0x1000000028c", "tid": 460 }, { "created_ino": "0x1000000028d", "tid": 463 }, { "created_ino": "0x1000000028e", "tid": 466 }, { "created_ino": "0x1000000028f", "tid": 469 }, { "created_ino": "0x10000000290", "tid": 472 }, { "created_ino": "0x10000000291", "tid": 475 }, { "created_ino": "0x10000000292", "tid": 478 }, { "created_ino": "0x10000000293", "tid": 481 }, { "created_ino": "0x10000000294", "tid": 484 }, { "created_ino": "0x10000000295", "tid": 487 }, { "created_ino": "0x10000000296", "tid": 490 }, { "created_ino": "0x10000000297", "tid": 493 }, { "created_ino": "0x10000000298", "tid": 496 }, { "created_ino": "0x10000000299", "tid": 499 }, { "created_ino": "0x1000000029a", "tid": 502 }, { "created_ino": "0x1000000029b", "tid": 505 }, { "created_ino": "0x1000000029c", "tid": 508 }, { "created_ino": "0x1000000029d", "tid": 511 }, { "created_ino": "0x1000000029e", "tid": 514 }, { "created_ino": "0x1000000029f", "tid": 517 }, { "created_ino": "0x100000002a0", "tid": 520 }, { "created_ino": "0x100000002a1", "tid": 523 }, { "created_ino": "0x100000002a2", "tid": 526 }, { "created_ino": "0x100000002a3", "tid": 529 }, { "created_ino": "0x100000002a4", "tid": 532 }, { "created_ino": "0x100000002a5", "tid": 535 }, { "created_ino": "0x100000002a6", "tid": 538 }, { "created_ino": "0x100000002a7", "tid": 541 }, { "created_ino": "0x100000002a8", "tid": 544 }, { "created_ino": "0x100000002a9", "tid": 547 }, { "created_ino": "0x100000002aa", "tid": 550 }, { "created_ino": "0x100000002ab", "tid": 553 }, { "created_ino": "0x100000002ac", "tid": 556 }, { "created_ino": "0x100000002ad", "tid": 559 }, { "created_ino": "0x100000002ae", "tid": 562 }, { "created_ino": "0x100000002af", "tid": 565 }, { "created_ino": "0x100000002b0", "tid": 568 }, { "created_ino": "0x100000002b1", "tid": 571 }, { "created_ino": "0x100000002b2", "tid": 574 }, { "created_ino": "0x100000002b3", "tid": 577 }, { "created_ino": "0x100000002b4", "tid": 580 }, { "created_ino": "0x100000002b5", "tid": 583 }, { "created_ino": "0x100000002b6", "tid": 586 }, { "created_ino": "0x100000002b7", "tid": 589 }, { "created_ino": "0x100000002b8", "tid": 592 }, { "created_ino": "0x100000002b9", "tid": 595 }, { "created_ino": "0x100000002ba", "tid": 598 }, { "created_ino": "0x100000002bb", "tid": 601 }, { "created_ino": "0x100000002bc", "tid": 604 }, { "created_ino": "0x100000002bd", "tid": 607 }, { "created_ino": "0x100000002be", "tid": 610 }, { "created_ino": "0x100000002bf", "tid": 613 }, { "created_ino": "0x100000002c0", "tid": 616 }, { "created_ino": "0x100000002c1", "tid": 619 }, { "created_ino": "0x100000002c2", "tid": 622 }, { "created_ino": "0x100000002c3", "tid": 625 }, { "created_ino": "0x100000002c4", "tid": 628 }, { "created_ino": "0x100000002c5", "tid": 631 }, { "created_ino": "0x100000002c6", "tid": 634 }, { "created_ino": "0x100000002c7", "tid": 637 }, { "created_ino": "0x100000002c8", "tid": 640 }, { "created_ino": "0x100000002c9", "tid": 643 }, { "created_ino": "0x100000002ca", "tid": 646 }, { "created_ino": "0x100000002cb", "tid": 649 }, { "created_ino": "0x100000002cc", "tid": 652 }, { "created_ino": "0x100000002cd", "tid": 655 }, { "created_ino": "0x100000002ce", "tid": 658 }, { "created_ino": "0x100000002cf", "tid": 661 }, { "created_ino": "0x100000002d0", "tid": 664 }, { "created_ino": "0x100000002d1", "tid": 667 }, { "created_ino": "0x100000002d2", "tid": 670 }, { "created_ino": "0x100000002d3", "tid": 673 }, { "created_ino": "0x100000002d4", "tid": 676 }, { "created_ino": "0x100000002d5", "tid": 679 }, { "created_ino": "0x100000002d6", "tid": 682 }, { "created_ino": "0x100000002d7", "tid": 685 }, { "created_ino": "0x100000002d8", "tid": 688 }, { "created_ino": "0x100000002d9", "tid": 691 }, { "created_ino": "0x100000002da", "tid": 694 }, { "created_ino": "0x100000002db", "tid": 697 }, { "created_ino": "0x100000002dc", "tid": 700 }, { "created_ino": "0x100000002dd", "tid": 703 }, { "created_ino": "0x100000002de", "tid": 706 }, { "created_ino": "0x100000002df", "tid": 709 }, { "created_ino": "0x100000002e0", "tid": 712 }, { "created_ino": "0x100000002e1", "tid": 715 }, { "created_ino": "0x100000002e2", "tid": 718 }, { "created_ino": "0x100000002e3", "tid": 721 }, { "created_ino": "0x100000002e4", "tid": 724 }, { "created_ino": "0x100000002e5", "tid": 727 }, { "created_ino": "0x100000002e6", "tid": 730 }, { "created_ino": "0x100000002e7", "tid": 733 }, { "created_ino": "0x100000002e8", "tid": 736 }, { "created_ino": "0x100000002e9", "tid": 739 }, { "created_ino": "0x100000002ea", "tid": 742 }, { "created_ino": "0x100000002eb", "tid": 745 }, { "created_ino": "0x100000002ec", "tid": 748 }, { "created_ino": "0x100000002ed", "tid": 751 }, { "created_ino": "0x100000002ee", "tid": 754 }, { "created_ino": "0x100000002ef", "tid": 757 }, { "created_ino": "0x100000002f0", "tid": 760 }, { "created_ino": "0x100000002f1", "tid": 763 }, { "created_ino": "0x100000002f2", "tid": 766 }, { "created_ino": "0x100000002f3", "tid": 769 }, { "created_ino": "0x100000002f4", "tid": 772 }, { "created_ino": "0x100000002f5", "tid": 775 }, { "created_ino": "0x100000002f6", "tid": 778 }, { "created_ino": "0x100000002f7", "tid": 781 }, { "created_ino": "0x100000002f8", "tid": 784 }, { "created_ino": "0x100000002f9", "tid": 787 }, { "created_ino": "0x100000002fa", "tid": 790 }, { "created_ino": "0x100000002fb", "tid": 793 }, { "created_ino": "0x100000002fc", "tid": 796 }, { "created_ino": "0x100000002fd", "tid": 799 }, { "created_ino": "0x100000002fe", "tid": 802 }, { "created_ino": "0x100000002ff", "tid": 805 }, { "created_ino": "0x10000000300", "tid": 808 }, { "created_ino": "0x10000000301", "tid": 811 }, { "created_ino": "0x10000000302", "tid": 814 }, { "created_ino": "0x10000000303", "tid": 817 }, { "created_ino": "0x10000000304", "tid": 820 }, { "created_ino": "0x10000000305", "tid": 823 }, { "created_ino": "0x10000000306", "tid": 826 }, { "created_ino": "0x10000000307", "tid": 829 }, { "created_ino": "0x10000000308", "tid": 832 }, { "created_ino": "0x10000000309", "tid": 835 }, { "created_ino": "0x1000000030a", "tid": 838 }, { "created_ino": "0x1000000030b", "tid": 841 }, { "created_ino": "0x1000000030c", "tid": 844 }, { "created_ino": "0x1000000030d", "tid": 847 }, { "created_ino": "0x1000000030e", "tid": 850 }, { "created_ino": "0x1000000030f", "tid": 853 }, { "created_ino": "0x10000000310", "tid": 856 }, { "created_ino": "0x10000000311", "tid": 859 }, { "created_ino": "0x10000000312", "tid": 862 }, { "created_ino": "0x10000000313", "tid": 865 }, { "created_ino": "0x10000000314", "tid": 868 }, { "created_ino": "0x10000000315", "tid": 871 }, { "created_ino": "0x10000000316", "tid": 874 }, { "created_ino": "0x10000000317", "tid": 877 }, { "created_ino": "0x10000000318", "tid": 880 }, { "created_ino": "0x10000000319", "tid": 883 }, { "created_ino": "0x1000000031a", "tid": 886 }, { "created_ino": "0x1000000031b", "tid": 889 }, { "created_ino": "0x1000000031c", "tid": 892 }, { "created_ino": "0x1000000031d", "tid": 895 }, { "created_ino": "0x1000000031e", "tid": 898 }, { "created_ino": "0x1000000031f", "tid": 901 }, { "created_ino": "0x10000000320", "tid": 904 }, { "created_ino": "0x10000000321", "tid": 907 }, { "created_ino": "0x10000000322", "tid": 910 }, { "created_ino": "0x10000000323", "tid": 913 }, { "created_ino": "0x10000000324", "tid": 916 }, { "created_ino": "0x10000000325", "tid": 919 }, { "created_ino": "0x10000000326", "tid": 922 }, { "created_ino": "0x10000000327", "tid": 925 }, { "created_ino": "0x10000000328", "tid": 928 }, { "created_ino": "0x10000000329", "tid": 931 }, { "created_ino": "0x1000000032a", "tid": 934 }, { "created_ino": "0x1000000032b", "tid": 937 }, { "created_ino": "0x1000000032c", "tid": 940 }, { "created_ino": "0x1000000032d", "tid": 943 }, { "created_ino": "0x1000000032e", "tid": 946 }, { "created_ino": "0x1000000032f", "tid": 949 }, { "created_ino": "0x10000000330", "tid": 952 }, { "created_ino": "0x10000000331", "tid": 955 }, { "created_ino": "0x10000000332", "tid": 958 }, { "created_ino": "0x10000000333", "tid": 961 }, { "created_ino": "0x10000000334", "tid": 964 }, { "created_ino": "0x10000000335", "tid": 967 }, { "created_ino": "0x10000000336", "tid": 970 }, { "created_ino": "0x10000000337", "tid": 973 }, { "created_ino": "0x10000000338", "tid": 976 }, { "created_ino": "0x10000000339", "tid": 979 }, { "created_ino": "0x1000000033a", "tid": 982 }, { "created_ino": "0x1000000033b", "tid": 985 }, { "created_ino": "0x1000000033c", "tid": 988 }, { "created_ino": "0x1000000033d", "tid": 991 }, { "created_ino": "0x1000000033e", "tid": 994 }, { "created_ino": "0x1000000033f", "tid": 997 }, { "created_ino": "0x10000000340", "tid": 1000 }, { "created_ino": "0x10000000341", "tid": 1003 }, { "created_ino": "0x10000000342", "tid": 1006 }, { "created_ino": "0x10000000343", "tid": 1009 }, { "created_ino": "0x10000000344", "tid": 1012 }, { "created_ino": "0x10000000345", "tid": 1015 }, { "created_ino": "0x10000000346", "tid": 1018 }, { "created_ino": "0x10000000347", "tid": 1021 }, { "created_ino": "0x10000000348", "tid": 1024 }, { "created_ino": "0x10000000349", "tid": 1027 }, { "created_ino": "0x1000000034a", "tid": 1030 }, { "created_ino": "0x1000000034b", "tid": 1033 }, { "created_ino": "0x1000000034c", "tid": 1036 }, { "created_ino": "0x1000000034d", "tid": 1039 }, { "created_ino": "0x1000000034e", "tid": 1042 }, { "created_ino": "0x1000000034f", "tid": 1045 }, { "created_ino": "0x10000000350", "tid": 1048 }, { "created_ino": "0x10000000351", "tid": 1051 }, { "created_ino": "0x10000000352", "tid": 1054 }, { "created_ino": "0x10000000353", "tid": 1057 }, { "created_ino": "0x10000000354", "tid": 1060 }, { "created_ino": "0x10000000355", "tid": 1063 }, { "created_ino": "0x10000000356", "tid": 1066 }, { "created_ino": "0x10000000357", "tid": 1069 }, { "created_ino": "0x10000000358", "tid": 1072 }, { "created_ino": "0x10000000359", "tid": 1075 }, { "created_ino": "0x1000000035a", "tid": 1078 }, { "created_ino": "0x1000000035b", "tid": 1081 }, { "created_ino": "0x1000000035c", "tid": 1084 }, { "created_ino": "0x1000000035d", "tid": 1087 }, { "created_ino": "0x1000000035e", "tid": 1090 }, { "created_ino": "0x1000000035f", "tid": 1093 }, { "created_ino": "0x10000000360", "tid": 1096 }, { "created_ino": "0x10000000361", "tid": 1099 }, { "created_ino": "0x10000000362", "tid": 1102 }, { "created_ino": "0x10000000363", "tid": 1105 }, { "created_ino": "0x10000000364", "tid": 1108 }, { "created_ino": "0x10000000365", "tid": 1111 }, { "created_ino": "0x10000000366", "tid": 1114 }, { "created_ino": "0x10000000367", "tid": 1117 }, { "created_ino": "0x10000000368", "tid": 1120 }, { "created_ino": "0x10000000369", "tid": 1123 }, { "created_ino": "0x1000000036a", "tid": 1126 }, { "created_ino": "0x1000000036b", "tid": 1129 }, { "created_ino": "0x1000000036c", "tid": 1132 }, { "created_ino": "0x1000000036d", "tid": 1135 }, { "created_ino": "0x1000000036e", "tid": 1138 }, { "created_ino": "0x1000000036f", "tid": 1141 }, { "created_ino": "0x10000000370", "tid": 1144 }, { "created_ino": "0x10000000371", "tid": 1147 }, { "created_ino": "0x10000000372", "tid": 1150 }, { "created_ino": "0x10000000373", "tid": 1153 }, { "created_ino": "0x10000000374", "tid": 1156 }, { "created_ino": "0x10000000375", "tid": 1159 }, { "created_ino": "0x10000000376", "tid": 1162 }, { "created_ino": "0x10000000377", "tid": 1165 }, { "created_ino": "0x10000000378", "tid": 1168 }, { "created_ino": "0x10000000379", "tid": 1171 }, { "created_ino": "0x1000000037a", "tid": 1174 }, { "created_ino": "0x1000000037b", "tid": 1177 }, { "created_ino": "0x1000000037c", "tid": 1180 }, { "created_ino": "0x1000000037d", "tid": 1183 }, { "created_ino": "0x1000000037e", "tid": 1186 }, { "created_ino": "0x1000000037f", "tid": 1189 }, { "created_ino": "0x10000000380", "tid": 1192 }, { "created_ino": "0x10000000381", "tid": 1195 }, { "created_ino": "0x10000000382", "tid": 1198 }, { "created_ino": "0x10000000383", "tid": 1201 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3786269155, "type": "any" }, "name": { "num": 8484, "type": "client" } }, "id": 8484, "inst": "client.8484 192.168.0.1:0/3786269155", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 7, "num_caps": 401, "num_completed_flushes": 0, "num_completed_requests": 399, "num_leases": 400, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 603, "start": "0x10000000384" } ], "recall_caps": { "halflife": 60, "value": 1164.071803139562 }, "recall_caps_throttle": { "halflife": 1.5, "value": 473.33459924190254 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 217.44276769443005 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1126, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1185.6038318100507 }, "state": "open", "uptime": 10.98684622 } ] 2024-09-19T07:55:24.150 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-09-19T07:55:24.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:55:24.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.252+0000 7f2c8359e640 1 Processor -- start 2024-09-19T07:55:24.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.252+0000 7f2c8359e640 1 -- start start 2024-09-19T07:55:24.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.253+0000 7f2c8359e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2c8415e500 0x7f2c840d1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:24.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.253+0000 7f2c8359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c840d2ee0 0x7f2c840cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:24.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.253+0000 7f2c8359e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c840b5210 con 0x7f2c840b4170 2024-09-19T07:55:24.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.253+0000 7f2c8359e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2c840b4600 con 0x7f2c840d2ee0 2024-09-19T07:55:24.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.253+0000 7f2c8359e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2c840b48e0 con 0x7f2c8415e500 2024-09-19T07:55:24.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.253+0000 7f2c82d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c840d2ee0 0x7f2c840cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:24.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.253+0000 7f2c81d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2c840b4170 0x7f2c840d27d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50820/0 (socket says 172.21.15.16:50820) 2024-09-19T07:55:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.253+0000 7f2c81d9b640 1 -- 172.21.15.16:0/43129054 learned_addr learned my addr 172.21.15.16:0/43129054 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.253+0000 7f2c8259c640 1 --2- 172.21.15.16:0/43129054 >> v2:172.21.15.146:3300/0 conn(0x7f2c8415e500 0x7f2c840d1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.254+0000 7f2c82d9d640 1 -- 172.21.15.16:0/43129054 >> v2:172.21.15.146:3300/0 conn(0x7f2c8415e500 msgr2=0x7f2c840d1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.254+0000 7f2c82d9d640 1 --2- 172.21.15.16:0/43129054 >> v2:172.21.15.146:3300/0 conn(0x7f2c8415e500 0x7f2c840d1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.254+0000 7f2c82d9d640 1 -- 172.21.15.16:0/43129054 >> v1:172.21.15.16:6789/0 conn(0x7f2c840b4170 legacy=0x7f2c840d27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.254+0000 7f2c82d9d640 1 -- 172.21.15.16:0/43129054 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c840b4bc0 con 0x7f2c840d2ee0 2024-09-19T07:55:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.254+0000 7f2c8159a640 1 -- 172.21.15.16:0/43129054 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1579254426 0 0) 0x7f2c840b5210 con 0x7f2c840b4170 2024-09-19T07:55:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.254+0000 7f2c8259c640 1 --2- 172.21.15.16:0/43129054 >> v2:172.21.15.146:3300/0 conn(0x7f2c8415e500 0x7f2c840d1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:24.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.254+0000 7f2c82d9d640 1 --2- 172.21.15.16:0/43129054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c840d2ee0 0x7f2c840cf290 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f2c8408e0b0 tx=0x7f2c7c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c663cf96ec17a2a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:24.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.254+0000 7f2c8159a640 1 -- 172.21.15.16:0/43129054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c007740 con 0x7f2c840d2ee0 2024-09-19T07:55:24.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.255+0000 7f2c8159a640 1 -- 172.21.15.16:0/43129054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c7c0078e0 con 0x7f2c840d2ee0 2024-09-19T07:55:24.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.255+0000 7f2c8159a640 1 -- 172.21.15.16:0/43129054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c007cb0 con 0x7f2c840d2ee0 2024-09-19T07:55:24.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.255+0000 7f2c8359e640 1 -- 172.21.15.16:0/43129054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c840d2ee0 msgr2=0x7f2c840cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:24.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.255+0000 7f2c8359e640 1 --2- 172.21.15.16:0/43129054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c840d2ee0 0x7f2c840cf290 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f2c8408e0b0 tx=0x7f2c7c02a710 comp rx=0 tx=0).stop 2024-09-19T07:55:24.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.255+0000 7f2c8359e640 1 -- 172.21.15.16:0/43129054 shutdown_connections 2024-09-19T07:55:24.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.255+0000 7f2c8359e640 1 --2- 172.21.15.16:0/43129054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c840d2ee0 0x7f2c840cf290 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.255+0000 7f2c8359e640 1 --2- 172.21.15.16:0/43129054 >> v2:172.21.15.146:3300/0 conn(0x7f2c8415e500 0x7f2c840d1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.255+0000 7f2c8359e640 1 -- 172.21.15.16:0/43129054 >> 172.21.15.16:0/43129054 conn(0x7f2c8408dc00 msgr2=0x7f2c8409d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:24.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.255+0000 7f2c8359e640 1 -- 172.21.15.16:0/43129054 shutdown_connections 2024-09-19T07:55:24.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.255+0000 7f2c8359e640 1 -- 172.21.15.16:0/43129054 wait complete. 2024-09-19T07:55:24.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.256+0000 7f2c8359e640 1 Processor -- start 2024-09-19T07:55:24.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.257+0000 7f2c8359e640 1 -- start start 2024-09-19T07:55:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.257+0000 7f2c8359e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c840d2ee0 0x7f2c840b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.257+0000 7f2c8359e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c8415e500 0x7f2c840b95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.257+0000 7f2c8359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c840b9b20 0x7f2c840cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.257+0000 7f2c8359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c840d2480 con 0x7f2c840b9b20 2024-09-19T07:55:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.257+0000 7f2c8359e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2c840d2300 con 0x7f2c840d2ee0 2024-09-19T07:55:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.257+0000 7f2c8359e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2c840cf7d0 con 0x7f2c8415e500 2024-09-19T07:55:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.257+0000 7f2c82d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c840b9b20 0x7f2c840cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.257+0000 7f2c82d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c840b9b20 0x7f2c840cc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41066/0 (socket says 172.21.15.16:41066) 2024-09-19T07:55:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.257+0000 7f2c82d9d640 1 -- 172.21.15.16:0/2502167374 learned_addr learned my addr 172.21.15.16:0/2502167374 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:24.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.258+0000 7f2c8259c640 1 --2- 172.21.15.16:0/2502167374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c840d2ee0 0x7f2c840b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:24.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.258+0000 7f2c81d9b640 1 --2- 172.21.15.16:0/2502167374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c8415e500 0x7f2c840b95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:24.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.258+0000 7f2c82d9d640 1 -- 172.21.15.16:0/2502167374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c8415e500 msgr2=0x7f2c840b95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:24.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.258+0000 7f2c82d9d640 1 --2- 172.21.15.16:0/2502167374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c8415e500 0x7f2c840b95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.258+0000 7f2c82d9d640 1 -- 172.21.15.16:0/2502167374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c840d2ee0 msgr2=0x7f2c840b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:24.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.258+0000 7f2c82d9d640 1 --2- 172.21.15.16:0/2502167374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c840d2ee0 0x7f2c840b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.258+0000 7f2c82d9d640 1 -- 172.21.15.16:0/2502167374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c841f3f70 con 0x7f2c840b9b20 2024-09-19T07:55:24.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.258+0000 7f2c81d9b640 1 --2- 172.21.15.16:0/2502167374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c8415e500 0x7f2c840b95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:24.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.258+0000 7f2c82d9d640 1 --2- 172.21.15.16:0/2502167374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c840b9b20 0x7f2c840cc070 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c02ad00 tx=0x7f2c7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:24.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.258+0000 7f2c72ffd640 1 -- 172.21.15.16:0/2502167374 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c040070 con 0x7f2c840b9b20 2024-09-19T07:55:24.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.258+0000 7f2c72ffd640 1 -- 172.21.15.16:0/2502167374 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c7c031bc0 con 0x7f2c840b9b20 2024-09-19T07:55:24.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.258+0000 7f2c8359e640 1 -- 172.21.15.16:0/2502167374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c841f4210 con 0x7f2c840b9b20 2024-09-19T07:55:24.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.258+0000 7f2c72ffd640 1 -- 172.21.15.16:0/2502167374 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c045460 con 0x7f2c840b9b20 2024-09-19T07:55:24.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.258+0000 7f2c8359e640 1 -- 172.21.15.16:0/2502167374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c841f47d0 con 0x7f2c840b9b20 2024-09-19T07:55:24.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.259+0000 7f2c8359e640 1 -- 172.21.15.16:0/2502167374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c840cf290 con 0x7f2c840b9b20 2024-09-19T07:55:24.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.260+0000 7f2c72ffd640 1 -- 172.21.15.16:0/2502167374 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2c7c039040 con 0x7f2c840b9b20 2024-09-19T07:55:24.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.260+0000 7f2c72ffd640 1 --2- 172.21.15.16:0/2502167374 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2c6406c2b0 0x7f2c6406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:24.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.260+0000 7f2c72ffd640 1 -- 172.21.15.16:0/2502167374 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f2c7c0ace40 con 0x7f2c840b9b20 2024-09-19T07:55:24.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.260+0000 7f2c8259c640 1 --2- 172.21.15.16:0/2502167374 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2c6406c2b0 0x7f2c6406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:24.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.261+0000 7f2c8259c640 1 --2- 172.21.15.16:0/2502167374 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2c6406c2b0 0x7f2c6406e770 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2c74006fd0 tx=0x7f2c74008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:24.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.264+0000 7f2c72ffd640 1 -- 172.21.15.16:0/2502167374 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2c7c07b420 con 0x7f2c840b9b20 2024-09-19T07:55:24.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.478+0000 7f2c8359e640 1 -- 172.21.15.16:0/2502167374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2c840751a0 con 0x7f2c840b9b20 2024-09-19T07:55:24.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.479+0000 7f2c72ffd640 1 -- 172.21.15.16:0/2502167374 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7f2c7c07ad20 con 0x7f2c840b9b20 2024-09-19T07:55:24.479 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:24.479 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:55:24.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.481+0000 7f2c8359e640 1 -- 172.21.15.16:0/2502167374 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2c6406c2b0 msgr2=0x7f2c6406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:24.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.481+0000 7f2c8359e640 1 --2- 172.21.15.16:0/2502167374 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2c6406c2b0 0x7f2c6406e770 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2c74006fd0 tx=0x7f2c74008040 comp rx=0 tx=0).stop 2024-09-19T07:55:24.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.481+0000 7f2c8359e640 1 -- 172.21.15.16:0/2502167374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c840b9b20 msgr2=0x7f2c840cc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:24.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.481+0000 7f2c8359e640 1 --2- 172.21.15.16:0/2502167374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c840b9b20 0x7f2c840cc070 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c02ad00 tx=0x7f2c7c004a40 comp rx=0 tx=0).stop 2024-09-19T07:55:24.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.481+0000 7f2c8359e640 1 -- 172.21.15.16:0/2502167374 shutdown_connections 2024-09-19T07:55:24.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.481+0000 7f2c8359e640 1 --2- 172.21.15.16:0/2502167374 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2c6406c2b0 0x7f2c6406e770 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.481+0000 7f2c8359e640 1 --2- 172.21.15.16:0/2502167374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c840b9b20 0x7f2c840cc070 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.481+0000 7f2c8359e640 1 --2- 172.21.15.16:0/2502167374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c8415e500 0x7f2c840b95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.481+0000 7f2c8359e640 1 --2- 172.21.15.16:0/2502167374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c840d2ee0 0x7f2c840b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.481+0000 7f2c8359e640 1 -- 172.21.15.16:0/2502167374 >> 172.21.15.16:0/2502167374 conn(0x7f2c8408dc00 msgr2=0x7f2c840d27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:24.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.481+0000 7f2c8359e640 1 -- 172.21.15.16:0/2502167374 shutdown_connections 2024-09-19T07:55:24.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.481+0000 7f2c8359e640 1 -- 172.21.15.16:0/2502167374 wait complete. 2024-09-19T07:55:24.492 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-19T07:55:24.492 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-09-19T07:55:24.492 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:55:24.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.593+0000 7f39d57d8640 1 Processor -- start 2024-09-19T07:55:24.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.593+0000 7f39d57d8640 1 -- start start 2024-09-19T07:55:24.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.594+0000 7f39d57d8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f39d015e2a0 0x7f39d006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:24.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.594+0000 7f39d57d8640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f39d015e680 0x7f39d00cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:24.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.594+0000 7f39d57d8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39d00d1880 con 0x7f39d006d960 2024-09-19T07:55:24.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.594+0000 7f39d57d8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f39d00d1b60 con 0x7f39d015e2a0 2024-09-19T07:55:24.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.594+0000 7f39d57d8640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f39d00cf2d0 con 0x7f39d015e680 2024-09-19T07:55:24.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.594+0000 7f39ceffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f39d015e2a0 0x7f39d006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:24.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.594+0000 7f39cf7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f39d006d960 0x7f39d00d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50824/0 (socket says 172.21.15.16:50824) 2024-09-19T07:55:24.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.594+0000 7f39cf7fe640 1 -- 172.21.15.16:0/2632929501 learned_addr learned my addr 172.21.15.16:0/2632929501 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:24.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.594+0000 7f39c7fff640 1 --2- 172.21.15.16:0/2632929501 >> v2:172.21.15.146:3300/0 conn(0x7f39d015e680 0x7f39d00cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:24.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.594+0000 7f39ceffd640 1 -- 172.21.15.16:0/2632929501 >> v2:172.21.15.146:3300/0 conn(0x7f39d015e680 msgr2=0x7f39d00cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:24.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.594+0000 7f39ceffd640 1 --2- 172.21.15.16:0/2632929501 >> v2:172.21.15.146:3300/0 conn(0x7f39d015e680 0x7f39d00cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.594+0000 7f39ceffd640 1 -- 172.21.15.16:0/2632929501 >> v1:172.21.15.16:6789/0 conn(0x7f39d006d960 legacy=0x7f39d00d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:24.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.594+0000 7f39ceffd640 1 -- 172.21.15.16:0/2632929501 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d00cf5b0 con 0x7f39d015e2a0 2024-09-19T07:55:24.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.595+0000 7f39ceffd640 1 --2- 172.21.15.16:0/2632929501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39d015e2a0 0x7f39d006d420 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f39d00a22a0 tx=0x7f39c002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391ab6702d150be5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:24.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.595+0000 7f39ce7fc640 1 -- 172.21.15.16:0/2632929501 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0037070 con 0x7f39d015e2a0 2024-09-19T07:55:24.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.595+0000 7f39ce7fc640 1 -- 172.21.15.16:0/2632929501 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c0032440 con 0x7f39d015e2a0 2024-09-19T07:55:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.595+0000 7f39ce7fc640 1 -- 172.21.15.16:0/2632929501 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0032810 con 0x7f39d015e2a0 2024-09-19T07:55:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.596+0000 7f39d57d8640 1 -- 172.21.15.16:0/2632929501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39d015e2a0 msgr2=0x7f39d006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.596+0000 7f39d57d8640 1 --2- 172.21.15.16:0/2632929501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39d015e2a0 0x7f39d006d420 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f39d00a22a0 tx=0x7f39c002a1a0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.596+0000 7f39d57d8640 1 -- 172.21.15.16:0/2632929501 shutdown_connections 2024-09-19T07:55:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.596+0000 7f39d57d8640 1 --2- 172.21.15.16:0/2632929501 >> v2:172.21.15.146:3300/0 conn(0x7f39d015e680 0x7f39d00cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.596+0000 7f39d57d8640 1 --2- 172.21.15.16:0/2632929501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39d015e2a0 0x7f39d006d420 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.596+0000 7f39d57d8640 1 -- 172.21.15.16:0/2632929501 >> 172.21.15.16:0/2632929501 conn(0x7f39d009d760 msgr2=0x7f39d008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.596+0000 7f39d57d8640 1 -- 172.21.15.16:0/2632929501 shutdown_connections 2024-09-19T07:55:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.596+0000 7f39d57d8640 1 -- 172.21.15.16:0/2632929501 wait complete. 2024-09-19T07:55:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.597+0000 7f39d57d8640 1 Processor -- start 2024-09-19T07:55:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.597+0000 7f39d57d8640 1 -- start start 2024-09-19T07:55:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39d57d8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39d006d960 0x7f39d00cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39d57d8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39d015e2a0 0x7f39d00cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39d57d8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39d015e680 0x7f39d01f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39d57d8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f39d00d4c00 con 0x7f39d006d960 2024-09-19T07:55:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39d57d8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f39d00d4a80 con 0x7f39d015e680 2024-09-19T07:55:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39d57d8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f39d00d0010 con 0x7f39d015e2a0 2024-09-19T07:55:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39d006d960 0x7f39d00cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39d006d960 0x7f39d00cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41086/0 (socket says 172.21.15.16:41086) 2024-09-19T07:55:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39ceffd640 1 -- 172.21.15.16:0/1879199564 learned_addr learned my addr 172.21.15.16:0/1879199564 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:24.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39cf7fe640 1 --2- 172.21.15.16:0/1879199564 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39d015e680 0x7f39d01f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:24.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39c7fff640 1 --2- 172.21.15.16:0/1879199564 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39d015e2a0 0x7f39d00cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:24.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39ceffd640 1 -- 172.21.15.16:0/1879199564 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39d015e2a0 msgr2=0x7f39d00cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:24.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39ceffd640 1 --2- 172.21.15.16:0/1879199564 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39d015e2a0 0x7f39d00cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39ceffd640 1 -- 172.21.15.16:0/1879199564 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39d015e680 msgr2=0x7f39d01f7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:24.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39ceffd640 1 --2- 172.21.15.16:0/1879199564 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39d015e680 0x7f39d01f7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39ceffd640 1 -- 172.21.15.16:0/1879199564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d00cb4b0 con 0x7f39d006d960 2024-09-19T07:55:24.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39ceffd640 1 --2- 172.21.15.16:0/1879199564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39d006d960 0x7f39d00cc510 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009890 tx=0x7f39c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:24.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39c77fe640 1 -- 172.21.15.16:0/1879199564 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0037040 con 0x7f39d006d960 2024-09-19T07:55:24.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39c77fe640 1 -- 172.21.15.16:0/1879199564 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c0042070 con 0x7f39d006d960 2024-09-19T07:55:24.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39d57d8640 1 -- 172.21.15.16:0/1879199564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d00cb770 con 0x7f39d006d960 2024-09-19T07:55:24.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.598+0000 7f39c77fe640 1 -- 172.21.15.16:0/1879199564 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0030bd0 con 0x7f39d006d960 2024-09-19T07:55:24.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.599+0000 7f39d57d8640 1 -- 172.21.15.16:0/1879199564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d00cbd30 con 0x7f39d006d960 2024-09-19T07:55:24.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.599+0000 7f39d57d8640 1 -- 172.21.15.16:0/1879199564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d00cc170 con 0x7f39d006d960 2024-09-19T07:55:24.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.600+0000 7f39c77fe640 1 -- 172.21.15.16:0/1879199564 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f39c0047460 con 0x7f39d006d960 2024-09-19T07:55:24.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.600+0000 7f39c77fe640 1 --2- 172.21.15.16:0/1879199564 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f39b006c2b0 0x7f39b006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:24.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.600+0000 7f39c77fe640 1 -- 172.21.15.16:0/1879199564 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f39c00ac100 con 0x7f39d006d960 2024-09-19T07:55:24.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.601+0000 7f39c7fff640 1 --2- 172.21.15.16:0/1879199564 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f39b006c2b0 0x7f39b006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:24.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.601+0000 7f39c7fff640 1 --2- 172.21.15.16:0/1879199564 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f39b006c2b0 0x7f39b006e770 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f39b8006fd0 tx=0x7f39b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:24.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.605+0000 7f39c77fe640 1 -- 172.21.15.16:0/1879199564 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f39c0032e20 con 0x7f39d006d960 2024-09-19T07:55:24.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.818+0000 7f39d57d8640 1 -- 172.21.15.16:0/1879199564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f39d006d420 con 0x7f39d006d960 2024-09-19T07:55:24.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.818+0000 7f39c77fe640 1 -- 172.21.15.16:0/1879199564 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7f39c007a770 con 0x7f39d006d960 2024-09-19T07:55:24.819 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:24.819 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:55:24.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.820+0000 7f39d57d8640 1 -- 172.21.15.16:0/1879199564 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f39b006c2b0 msgr2=0x7f39b006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:24.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.820+0000 7f39d57d8640 1 --2- 172.21.15.16:0/1879199564 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f39b006c2b0 0x7f39b006e770 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f39b8006fd0 tx=0x7f39b8008040 comp rx=0 tx=0).stop 2024-09-19T07:55:24.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.820+0000 7f39d57d8640 1 -- 172.21.15.16:0/1879199564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39d006d960 msgr2=0x7f39d00cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:24.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.820+0000 7f39d57d8640 1 --2- 172.21.15.16:0/1879199564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39d006d960 0x7f39d00cc510 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009890 tx=0x7f39c00046c0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.820+0000 7f39d57d8640 1 -- 172.21.15.16:0/1879199564 shutdown_connections 2024-09-19T07:55:24.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.820+0000 7f39d57d8640 1 --2- 172.21.15.16:0/1879199564 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f39b006c2b0 0x7f39b006e770 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.820+0000 7f39d57d8640 1 --2- 172.21.15.16:0/1879199564 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39d015e680 0x7f39d01f7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.820+0000 7f39d57d8640 1 --2- 172.21.15.16:0/1879199564 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39d015e2a0 0x7f39d00cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.820+0000 7f39d57d8640 1 --2- 172.21.15.16:0/1879199564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39d006d960 0x7f39d00cc510 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.820+0000 7f39d57d8640 1 -- 172.21.15.16:0/1879199564 >> 172.21.15.16:0/1879199564 conn(0x7f39d009d760 msgr2=0x7f39d015fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:24.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.820+0000 7f39d57d8640 1 -- 172.21.15.16:0/1879199564 shutdown_connections 2024-09-19T07:55:24.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24.820+0000 7f39d57d8640 1 -- 172.21.15.16:0/1879199564 wait complete. 2024-09-19T07:55:24.831 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-19T07:55:24.831 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2024-09-19T07:55:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-19T07:55:24.912 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-19T07:55:24.912 INFO:teuthology.orchestra.run.smithi016.stderr:stdin closed, goodbye! 2024-09-19T07:55:24.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:55:24.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:55:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.090+0000 7f716c8d7640 1 Processor -- start 2024-09-19T07:55:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.090+0000 7f716c8d7640 1 -- start start 2024-09-19T07:55:25.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.090+0000 7f716c8d7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f716815bee0 0x7f71680d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:25.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.091+0000 7f716c8d7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f716815c2c0 0x7f71680d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:25.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.091+0000 7f716c8d7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71680cf200 0x7f71680cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:25.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.091+0000 7f716c8d7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f71680ceaf0 con 0x7f716815c2c0 2024-09-19T07:55:25.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.091+0000 7f716c8d7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f71680cdee0 con 0x7f716815bee0 2024-09-19T07:55:25.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.091+0000 7f716c8d7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f71680ce1c0 con 0x7f71680cf200 2024-09-19T07:55:25.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.091+0000 7f715ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f716815c2c0 0x7f71680d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:25.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.091+0000 7f715ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f716815c2c0 0x7f71680d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41098/0 (socket says 172.21.15.16:41098) 2024-09-19T07:55:25.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.091+0000 7f715ffff640 1 -- 172.21.15.16:0/1050218308 learned_addr learned my addr 172.21.15.16:0/1050218308 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:25.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.091+0000 7f7166d9d640 1 --2- 172.21.15.16:0/1050218308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71680cf200 0x7f71680cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:25.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.091+0000 7f716659c640 1 --2- 172.21.15.16:0/1050218308 >> v2:172.21.15.146:3300/0 conn(0x7f716815bee0 0x7f71680d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:25.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.091+0000 7f715ffff640 1 -- 172.21.15.16:0/1050218308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71680cf200 msgr2=0x7f71680cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:25.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.091+0000 7f715ffff640 1 --2- 172.21.15.16:0/1050218308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71680cf200 0x7f71680cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.091+0000 7f715ffff640 1 -- 172.21.15.16:0/1050218308 >> v2:172.21.15.146:3300/0 conn(0x7f716815bee0 msgr2=0x7f71680d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:25.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.091+0000 7f715ffff640 1 --2- 172.21.15.16:0/1050218308 >> v2:172.21.15.146:3300/0 conn(0x7f716815bee0 0x7f71680d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.091+0000 7f715ffff640 1 -- 172.21.15.16:0/1050218308 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71680ce4a0 con 0x7f716815c2c0 2024-09-19T07:55:25.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.092+0000 7f715ffff640 1 --2- 172.21.15.16:0/1050218308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716815c2c0 0x7f71680d09f0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f71680b2f40 tx=0x7f715002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da85a4038da7ed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:25.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.092+0000 7f7165d9b640 1 -- 172.21.15.16:0/1050218308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7150037070 con 0x7f716815c2c0 2024-09-19T07:55:25.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.092+0000 7f7165d9b640 1 -- 172.21.15.16:0/1050218308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7150032440 con 0x7f716815c2c0 2024-09-19T07:55:25.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.092+0000 7f7165d9b640 1 -- 172.21.15.16:0/1050218308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7150032810 con 0x7f716815c2c0 2024-09-19T07:55:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.092+0000 7f716c8d7640 1 -- 172.21.15.16:0/1050218308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716815c2c0 msgr2=0x7f71680d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.092+0000 7f716c8d7640 1 --2- 172.21.15.16:0/1050218308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716815c2c0 0x7f71680d09f0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f71680b2f40 tx=0x7f715002a1e0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.093+0000 7f716c8d7640 1 -- 172.21.15.16:0/1050218308 shutdown_connections 2024-09-19T07:55:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.093+0000 7f716c8d7640 1 --2- 172.21.15.16:0/1050218308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71680cf200 0x7f71680cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.093+0000 7f716c8d7640 1 --2- 172.21.15.16:0/1050218308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716815c2c0 0x7f71680d09f0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.093+0000 7f716c8d7640 1 --2- 172.21.15.16:0/1050218308 >> v2:172.21.15.146:3300/0 conn(0x7f716815bee0 0x7f71680d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.093+0000 7f716c8d7640 1 -- 172.21.15.16:0/1050218308 >> 172.21.15.16:0/1050218308 conn(0x7f716806bdf0 msgr2=0x7f716806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.093+0000 7f716c8d7640 1 -- 172.21.15.16:0/1050218308 shutdown_connections 2024-09-19T07:55:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.093+0000 7f716c8d7640 1 -- 172.21.15.16:0/1050218308 wait complete. 2024-09-19T07:55:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.094+0000 7f716c8d7640 1 Processor -- start 2024-09-19T07:55:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.094+0000 7f716c8d7640 1 -- start start 2024-09-19T07:55:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f716c8d7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71680cf200 0x7f71680adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f716c8d7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716815bee0 0x7f71680cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f716c8d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716815c2c0 0x7f7168159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f716c8d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f716815fd40 con 0x7f716815c2c0 2024-09-19T07:55:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f716c8d7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f71680cfbd0 con 0x7f716815bee0 2024-09-19T07:55:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f716c8d7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f716815fec0 con 0x7f71680cf200 2024-09-19T07:55:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f7166d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716815c2c0 0x7f7168159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f7166d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716815c2c0 0x7f7168159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41112/0 (socket says 172.21.15.16:41112) 2024-09-19T07:55:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f7166d9d640 1 -- 172.21.15.16:0/3178816972 learned_addr learned my addr 172.21.15.16:0/3178816972 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f716659c640 1 --2- 172.21.15.16:0/3178816972 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71680cf200 0x7f71680adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f715ffff640 1 --2- 172.21.15.16:0/3178816972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716815bee0 0x7f71680cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f7166d9d640 1 -- 172.21.15.16:0/3178816972 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71680cf200 msgr2=0x7f71680adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f7166d9d640 1 --2- 172.21.15.16:0/3178816972 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71680cf200 0x7f71680adb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f7166d9d640 1 -- 172.21.15.16:0/3178816972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716815bee0 msgr2=0x7f71680cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f7166d9d640 1 --2- 172.21.15.16:0/3178816972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716815bee0 0x7f71680cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f7166d9d640 1 -- 172.21.15.16:0/3178816972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71680ae0b0 con 0x7f716815c2c0 2024-09-19T07:55:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f7166d9d640 1 --2- 172.21.15.16:0/3178816972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716815c2c0 0x7f7168159690 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f7160007e60 tx=0x7f716000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f715effd640 1 -- 172.21.15.16:0/3178816972 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160014420 con 0x7f716815c2c0 2024-09-19T07:55:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f715effd640 1 -- 172.21.15.16:0/3178816972 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7160019070 con 0x7f716815c2c0 2024-09-19T07:55:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.095+0000 7f716c8d7640 1 -- 172.21.15.16:0/3178816972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71680ad490 con 0x7f716815c2c0 2024-09-19T07:55:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.096+0000 7f715effd640 1 -- 172.21.15.16:0/3178816972 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160014820 con 0x7f716815c2c0 2024-09-19T07:55:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.096+0000 7f716c8d7640 1 -- 172.21.15.16:0/3178816972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71680cc370 con 0x7f716815c2c0 2024-09-19T07:55:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.096+0000 7f716c8d7640 1 -- 172.21.15.16:0/3178816972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71680cf690 con 0x7f716815c2c0 2024-09-19T07:55:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.097+0000 7f715effd640 1 -- 172.21.15.16:0/3178816972 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f71600149c0 con 0x7f716815c2c0 2024-09-19T07:55:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.097+0000 7f715effd640 1 --2- 172.21.15.16:0/3178816972 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f714806c2b0 0x7f714806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.098+0000 7f715effd640 1 -- 172.21.15.16:0/3178816972 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f716008bce0 con 0x7f716815c2c0 2024-09-19T07:55:25.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.098+0000 7f716659c640 1 --2- 172.21.15.16:0/3178816972 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f714806c2b0 0x7f714806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:25.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.098+0000 7f716659c640 1 --2- 172.21.15.16:0/3178816972 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f714806c2b0 0x7f714806e770 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7158006fd0 tx=0x7f7158008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:25.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.102+0000 7f715effd640 1 -- 172.21.15.16:0/3178816972 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7160059c60 con 0x7f716815c2c0 2024-09-19T07:55:25.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.314+0000 7f716c8d7640 1 -- 172.21.15.16:0/3178816972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71680b47d0 con 0x7f716815c2c0 2024-09-19T07:55:25.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.315+0000 7f715effd640 1 -- 172.21.15.16:0/3178816972 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5230 (secure 0 0 0) 0x7f716005df60 con 0x7f716815c2c0 2024-09-19T07:55:25.315 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:25.315 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":83,"btime":"2024-09-19T07:55:23:288383+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/172459502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":172459502},{"type":"v1","addr":"172.21.15.146:6836","nonce":172459502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3417061308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3417061308},{"type":"v1","addr":"172.21.15.146:6837","nonce":3417061308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1810646409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1810646409},{"type":"v1","addr":"172.21.15.16:6834","nonce":1810646409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:55:02.325256+0000","modified":"2024-09-19T07:55:23.002436+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8286":{"gid":8286,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/1929965275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1929965275},{"type":"v1","addr":"172.21.15.16:6835","nonce":1929965275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8286,"qdb_cluster":[8286]},"id":10}]} 2024-09-19T07:55:25.316 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 83 2024-09-19T07:55:25.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.317+0000 7f716c8d7640 1 -- 172.21.15.16:0/3178816972 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f714806c2b0 msgr2=0x7f714806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.317+0000 7f716c8d7640 1 --2- 172.21.15.16:0/3178816972 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f714806c2b0 0x7f714806e770 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7158006fd0 tx=0x7f7158008040 comp rx=0 tx=0).stop 2024-09-19T07:55:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.317+0000 7f716c8d7640 1 -- 172.21.15.16:0/3178816972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716815c2c0 msgr2=0x7f7168159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.317+0000 7f716c8d7640 1 --2- 172.21.15.16:0/3178816972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716815c2c0 0x7f7168159690 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f7160007e60 tx=0x7f716000e4b0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.317+0000 7f716c8d7640 1 -- 172.21.15.16:0/3178816972 shutdown_connections 2024-09-19T07:55:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.317+0000 7f716c8d7640 1 --2- 172.21.15.16:0/3178816972 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f714806c2b0 0x7f714806e770 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.317+0000 7f716c8d7640 1 --2- 172.21.15.16:0/3178816972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716815c2c0 0x7f7168159690 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.317+0000 7f716c8d7640 1 --2- 172.21.15.16:0/3178816972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716815bee0 0x7f71680cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.317+0000 7f716c8d7640 1 --2- 172.21.15.16:0/3178816972 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71680cf200 0x7f71680adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.317+0000 7f716c8d7640 1 -- 172.21.15.16:0/3178816972 >> 172.21.15.16:0/3178816972 conn(0x7f716806bdf0 msgr2=0x7f716806dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.317+0000 7f716c8d7640 1 -- 172.21.15.16:0/3178816972 shutdown_connections 2024-09-19T07:55:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.317+0000 7f716c8d7640 1 -- 172.21.15.16:0/3178816972 wait complete. 2024-09-19T07:55:25.328 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-19T07:55:25.450 INFO:teuthology.orchestra.run.smithi016.stdout:[{"id":7654,"entity":{"name":{"type":"client","num":7654},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2861814065}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":10.150857921,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9436172365018036,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7654 192.168.0.1:0/2861814065","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"49748","root":"/"}},{"id":8484,"entity":{"name":{"type":"client","num":8484},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3786269155}},"state":"open","num_leases":400,"num_caps":401,"request_load_avg":1110,"uptime":12.290827969,"requests_in_flight":0,"num_completed_requests":399,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":1345.6304188904933,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":421.55637600215539,"halflife":1.5},"recall_caps_throttle2o":{"value":142.84554030034909,"halflife":0.5},"session_cache_liveness":{"value":1182.037175611084,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":7,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8484 192.168.0.1:0/3786269155","completed_requests":[{"tid":7,"created_ino":"0x10000000001"},{"tid":10,"created_ino":"0x100000001f6"},{"tid":13,"created_ino":"0x100000001f7"},{"tid":16,"created_ino":"0x100000001f8"},{"tid":19,"created_ino":"0x100000001f9"},{"tid":22,"created_ino":"0x100000001fa"},{"tid":25,"created_ino":"0x100000001fb"},{"tid":28,"created_ino":"0x100000001fc"},{"tid":31,"created_ino":"0x100000001fd"},{"tid":34,"created_ino":"0x100000001fe"},{"tid":37,"created_ino":"0x100000001ff"},{"tid":40,"created_ino":"0x10000000200"},{"tid":43,"created_ino":"0x10000000201"},{"tid":46,"created_ino":"0x10000000202"},{"tid":49,"created_ino":"0x10000000203"},{"tid":52,"created_ino":"0x10000000204"},{"tid":55,"created_ino":"0x10000000205"},{"tid":58,"created_ino":"0x10000000206"},{"tid":61,"created_ino":"0x10000000207"},{"tid":64,"created_ino":"0x10000000208"},{"tid":67,"created_ino":"0x10000000209"},{"tid":70,"created_ino":"0x1000000020a"},{"tid":73,"created_ino":"0x1000000020b"},{"tid":76,"created_ino":"0x1000000020c"},{"tid":79,"created_ino":"0x1000000020d"},{"tid":82,"created_ino":"0x1000000020e"},{"tid":85,"created_ino":"0x1000000020f"},{"tid":88,"created_ino":"0x10000000210"},{"tid":91,"created_ino":"0x10000000211"},{"tid":94,"created_ino":"0x10000000212"},{"tid":97,"created_ino":"0x10000000213"},{"tid":100,"created_ino":"0x10000000214"},{"tid":103,"created_ino":"0x10000000215"},{"tid":106,"created_ino":"0x10000000216"},{"tid":109,"created_ino":"0x10000000217"},{"tid":112,"created_ino":"0x10000000218"},{"tid":115,"created_ino":"0x10000000219"},{"tid":118,"created_ino":"0x1000000021a"},{"tid":121,"created_ino":"0x1000000021b"},{"tid":124,"created_ino":"0x1000000021c"},{"tid":127,"created_ino":"0x1000000021d"},{"tid":130,"created_ino":"0x1000000021e"},{"tid":133,"created_ino":"0x1000000021f"},{"tid":136,"created_ino":"0x10000000220"},{"tid":139,"created_ino":"0x10000000221"},{"tid":142,"created_ino":"0x10000000222"},{"tid":145,"created_ino":"0x10000000223"},{"tid":148,"created_ino":"0x10000000224"},{"tid":151,"created_ino":"0x10000000225"},{"tid":154,"created_ino":"0x10000000226"},{"tid":157,"created_ino":"0x10000000227"},{"tid":160,"created_ino":"0x10000000228"},{"tid":163,"created_ino":"0x10000000229"},{"tid":166,"created_ino":"0x1000000022a"},{"tid":169,"created_ino":"0x1000000022b"},{"tid":172,"created_ino":"0x1000000022c"},{"tid":175,"created_ino":"0x1000000022d"},{"tid":178,"created_ino":"0x1000000022e"},{"tid":181,"created_ino":"0x1000000022f"},{"tid":184,"created_ino":"0x10000000230"},{"tid":187,"created_ino":"0x10000000231"},{"tid":190,"created_ino":"0x10000000232"},{"tid":193,"created_ino":"0x10000000233"},{"tid":196,"created_ino":"0x10000000234"},{"tid":199,"created_ino":"0x10000000235"},{"tid":202,"created_ino":"0x10000000236"},{"tid":205,"created_ino":"0x10000000237"},{"tid":208,"created_ino":"0x10000000238"},{"tid":211,"created_ino":"0x10000000239"},{"tid":214,"created_ino":"0x1000000023a"},{"tid":217,"created_ino":"0x1000000023b"},{"tid":220,"created_ino":"0x1000000023c"},{"tid":223,"created_ino":"0x1000000023d"},{"tid":226,"created_ino":"0x1000000023e"},{"tid":229,"created_ino":"0x1000000023f"},{"tid":232,"created_ino":"0x10000000240"},{"tid":235,"created_ino":"0x10000000241"},{"tid":238,"created_ino":"0x10000000242"},{"tid":241,"created_ino":"0x10000000243"},{"tid":244,"created_ino":"0x10000000244"},{"tid":247,"created_ino":"0x10000000245"},{"tid":250,"created_ino":"0x10000000246"},{"tid":253,"created_ino":"0x10000000247"},{"tid":256,"created_ino":"0x10000000248"},{"tid":259,"created_ino":"0x10000000249"},{"tid":262,"created_ino":"0x1000000024a"},{"tid":265,"created_ino":"0x1000000024b"},{"tid":268,"created_ino":"0x1000000024c"},{"tid":271,"created_ino":"0x1000000024d"},{"tid":274,"created_ino":"0x1000000024e"},{"tid":277,"created_ino":"0x1000000024f"},{"tid":280,"created_ino":"0x10000000250"},{"tid":283,"created_ino":"0x10000000251"},{"tid":286,"created_ino":"0x10000000252"},{"tid":289,"created_ino":"0x10000000253"},{"tid":292,"created_ino":"0x10000000254"},{"tid":295,"created_ino":"0x10000000255"},{"tid":298,"created_ino":"0x10000000256"},{"tid":301,"created_ino":"0x10000000257"},{"tid":304,"created_ino":"0x10000000258"},{"tid":307,"created_ino":"0x10000000259"},{"tid":310,"created_ino":"0x1000000025a"},{"tid":313,"created_ino":"0x1000000025b"},{"tid":316,"created_ino":"0x1000000025c"},{"tid":319,"created_ino":"0x1000000025d"},{"tid":322,"created_ino":"0x1000000025e"},{"tid":325,"created_ino":"0x1000000025f"},{"tid":328,"created_ino":"0x10000000260"},{"tid":331,"created_ino":"0x10000000261"},{"tid":334,"created_ino":"0x10000000262"},{"tid":337,"created_ino":"0x10000000263"},{"tid":340,"created_ino":"0x10000000264"},{"tid":343,"created_ino":"0x10000000265"},{"tid":346,"created_ino":"0x10000000266"},{"tid":349,"created_ino":"0x10000000267"},{"tid":352,"created_ino":"0x10000000268"},{"tid":355,"created_ino":"0x10000000269"},{"tid":358,"created_ino":"0x1000000026a"},{"tid":361,"created_ino":"0x1000000026b"},{"tid":364,"created_ino":"0x1000000026c"},{"tid":367,"created_ino":"0x1000000026d"},{"tid":370,"created_ino":"0x1000000026e"},{"tid":373,"created_ino":"0x1000000026f"},{"tid":376,"created_ino":"0x10000000270"},{"tid":379,"created_ino":"0x10000000271"},{"tid":382,"created_ino":"0x10000000272"},{"tid":385,"created_ino":"0x10000000273"},{"tid":388,"created_ino":"0x10000000274"},{"tid":391,"created_ino":"0x10000000275"},{"tid":394,"created_ino":"0x10000000276"},{"tid":397,"created_ino":"0x10000000277"},{"tid":400,"created_ino":"0x10000000278"},{"tid":403,"created_ino":"0x10000000279"},{"tid":406,"created_ino":"0x1000000027a"},{"tid":409,"created_ino":"0x1000000027b"},{"tid":412,"created_ino":"0x1000000027c"},{"tid":415,"created_ino":"0x1000000027d"},{"tid":418,"created_ino":"0x1000000027e"},{"tid":421,"created_ino":"0x1000000027f"},{"tid":424,"created_ino":"0x10000000280"},{"tid":427,"created_ino":"0x10000000281"},{"tid":430,"created_ino":"0x10000000282"},{"tid":433,"created_ino":"0x10000000283"},{"tid":436,"created_ino":"0x10000000284"},{"tid":439,"created_ino":"0x10000000285"},{"tid":442,"created_ino":"0x10000000286"},{"tid":445,"created_ino":"0x10000000287"},{"tid":448,"created_ino":"0x10000000288"},{"tid":451,"created_ino":"0x10000000289"},{"tid":454,"created_ino":"0x1000000028a"},{"tid":457,"created_ino":"0x1000000028b"},{"tid":460,"created_ino":"0x1000000028c"},{"tid":463,"created_ino":"0x1000000028d"},{"tid":466,"created_ino":"0x1000000028e"},{"tid":469,"created_ino":"0x1000000028f"},{"tid":472,"created_ino":"0x10000000290"},{"tid":475,"created_ino":"0x10000000291"},{"tid":478,"created_ino":"0x10000000292"},{"tid":481,"created_ino":"0x10000000293"},{"tid":484,"created_ino":"0x10000000294"},{"tid":487,"created_ino":"0x10000000295"},{"tid":490,"created_ino":"0x10000000296"},{"tid":493,"created_ino":"0x10000000297"},{"tid":496,"created_ino":"0x10000000298"},{"tid":499,"created_ino":"0x10000000299"},{"tid":502,"created_ino":"0x1000000029a"},{"tid":505,"created_ino":"0x1000000029b"},{"tid":508,"created_ino":"0x1000000029c"},{"tid":511,"created_ino":"0x1000000029d"},{"tid":514,"created_ino":"0x1000000029e"},{"tid":517,"created_ino":"0x1000000029f"},{"tid":520,"created_ino":"0x100000002a0"},{"tid":523,"created_ino":"0x100000002a1"},{"tid":526,"created_ino":"0x100000002a2"},{"tid":529,"created_ino":"0x100000002a3"},{"tid":532,"created_ino":"0x100000002a4"},{"tid":535,"created_ino":"0x100000002a5"},{"tid":538,"created_ino":"0x100000002a6"},{"tid":541,"created_ino":"0x100000002a7"},{"tid":544,"created_ino":"0x100000002a8"},{"tid":547,"created_ino":"0x100000002a9"},{"tid":550,"created_ino":"0x100000002aa"},{"tid":553,"created_ino":"0x100000002ab"},{"tid":556,"created_ino":"0x100000002ac"},{"tid":559,"created_ino":"0x100000002ad"},{"tid":562,"created_ino":"0x100000002ae"},{"tid":565,"created_ino":"0x100000002af"},{"tid":568,"created_ino":"0x100000002b0"},{"tid":571,"created_ino":"0x100000002b1"},{"tid":574,"created_ino":"0x100000002b2"},{"tid":577,"created_ino":"0x100000002b3"},{"tid":580,"created_ino":"0x100000002b4"},{"tid":583,"created_ino":"0x100000002b5"},{"tid":586,"created_ino":"0x100000002b6"},{"tid":589,"created_ino":"0x100000002b7"},{"tid":592,"created_ino":"0x100000002b8"},{"tid":595,"created_ino":"0x100000002b9"},{"tid":598,"created_ino":"0x100000002ba"},{"tid":601,"created_ino":"0x100000002bb"},{"tid":604,"created_ino":"0x100000002bc"},{"tid":607,"created_ino":"0x100000002bd"},{"tid":610,"created_ino":"0x100000002be"},{"tid":613,"created_ino":"0x100000002bf"},{"tid":616,"created_ino":"0x100000002c0"},{"tid":619,"created_ino":"0x100000002c1"},{"tid":622,"created_ino":"0x100000002c2"},{"tid":625,"created_ino":"0x100000002c3"},{"tid":628,"created_ino":"0x100000002c4"},{"tid":631,"created_ino":"0x100000002c5"},{"tid":634,"created_ino":"0x100000002c6"},{"tid":637,"created_ino":"0x100000002c7"},{"tid":640,"created_ino":"0x100000002c8"},{"tid":643,"created_ino":"0x100000002c9"},{"tid":646,"created_ino":"0x100000002ca"},{"tid":649,"created_ino":"0x100000002cb"},{"tid":652,"created_ino":"0x100000002cc"},{"tid":655,"created_ino":"0x100000002cd"},{"tid":658,"created_ino":"0x100000002ce"},{"tid":661,"created_ino":"0x100000002cf"},{"tid":664,"created_ino":"0x100000002d0"},{"tid":667,"created_ino":"0x100000002d1"},{"tid":670,"created_ino":"0x100000002d2"},{"tid":673,"created_ino":"0x100000002d3"},{"tid":676,"created_ino":"0x100000002d4"},{"tid":679,"created_ino":"0x100000002d5"},{"tid":682,"created_ino":"0x100000002d6"},{"tid":685,"created_ino":"0x100000002d7"},{"tid":688,"created_ino":"0x100000002d8"},{"tid":691,"created_ino":"0x100000002d9"},{"tid":694,"created_ino":"0x100000002da"},{"tid":697,"created_ino":"0x100000002db"},{"tid":700,"created_ino":"0x100000002dc"},{"tid":703,"created_ino":"0x100000002dd"},{"tid":706,"created_ino":"0x100000002de"},{"tid":709,"created_ino":"0x100000002df"},{"tid":712,"created_ino":"0x100000002e0"},{"tid":715,"created_ino":"0x100000002e1"},{"tid":718,"created_ino":"0x100000002e2"},{"tid":721,"created_ino":"0x100000002e3"},{"tid":724,"created_ino":"0x100000002e4"},{"tid":727,"created_ino":"0x100000002e5"},{"tid":730,"created_ino":"0x100000002e6"},{"tid":733,"created_ino":"0x100000002e7"},{"tid":736,"created_ino":"0x100000002e8"},{"tid":739,"created_ino":"0x100000002e9"},{"tid":742,"created_ino":"0x100000002ea"},{"tid":745,"created_ino":"0x100000002eb"},{"tid":748,"created_ino":"0x100000002ec"},{"tid":751,"created_ino":"0x100000002ed"},{"tid":754,"created_ino":"0x100000002ee"},{"tid":757,"created_ino":"0x100000002ef"},{"tid":760,"created_ino":"0x100000002f0"},{"tid":763,"created_ino":"0x100000002f1"},{"tid":766,"created_ino":"0x100000002f2"},{"tid":769,"created_ino":"0x100000002f3"},{"tid":772,"created_ino":"0x100000002f4"},{"tid":775,"created_ino":"0x100000002f5"},{"tid":778,"created_ino":"0x100000002f6"},{"tid":781,"created_ino":"0x100000002f7"},{"tid":784,"created_ino":"0x100000002f8"},{"tid":787,"created_ino":"0x100000002f9"},{"tid":790,"created_ino":"0x100000002fa"},{"tid":793,"created_ino":"0x100000002fb"},{"tid":796,"created_ino":"0x100000002fc"},{"tid":799,"created_ino":"0x100000002fd"},{"tid":802,"created_ino":"0x100000002fe"},{"tid":805,"created_ino":"0x100000002ff"},{"tid":808,"created_ino":"0x10000000300"},{"tid":811,"created_ino":"0x10000000301"},{"tid":814,"created_ino":"0x10000000302"},{"tid":817,"created_ino":"0x10000000303"},{"tid":820,"created_ino":"0x10000000304"},{"tid":823,"created_ino":"0x10000000305"},{"tid":826,"created_ino":"0x10000000306"},{"tid":829,"created_ino":"0x10000000307"},{"tid":832,"created_ino":"0x10000000308"},{"tid":835,"created_ino":"0x10000000309"},{"tid":838,"created_ino":"0x1000000030a"},{"tid":841,"created_ino":"0x1000000030b"},{"tid":844,"created_ino":"0x1000000030c"},{"tid":847,"created_ino":"0x1000000030d"},{"tid":850,"created_ino":"0x1000000030e"},{"tid":853,"created_ino":"0x1000000030f"},{"tid":856,"created_ino":"0x10000000310"},{"tid":859,"created_ino":"0x10000000311"},{"tid":862,"created_ino":"0x10000000312"},{"tid":865,"created_ino":"0x10000000313"},{"tid":868,"created_ino":"0x10000000314"},{"tid":871,"created_ino":"0x10000000315"},{"tid":874,"created_ino":"0x10000000316"},{"tid":877,"created_ino":"0x10000000317"},{"tid":880,"created_ino":"0x10000000318"},{"tid":883,"created_ino":"0x10000000319"},{"tid":886,"created_ino":"0x1000000031a"},{"tid":889,"created_ino":"0x1000000031b"},{"tid":892,"created_ino":"0x1000000031c"},{"tid":895,"created_ino":"0x1000000031d"},{"tid":898,"created_ino":"0x1000000031e"},{"tid":901,"created_ino":"0x1000000031f"},{"tid":904,"created_ino":"0x10000000320"},{"tid":907,"created_ino":"0x10000000321"},{"tid":910,"created_ino":"0x10000000322"},{"tid":913,"created_ino":"0x10000000323"},{"tid":916,"created_ino":"0x10000000324"},{"tid":919,"created_ino":"0x10000000325"},{"tid":922,"created_ino":"0x10000000326"},{"tid":925,"created_ino":"0x10000000327"},{"tid":928,"created_ino":"0x10000000328"},{"tid":931,"created_ino":"0x10000000329"},{"tid":934,"created_ino":"0x1000000032a"},{"tid":937,"created_ino":"0x1000000032b"},{"tid":940,"created_ino":"0x1000000032c"},{"tid":943,"created_ino":"0x1000000032d"},{"tid":946,"created_ino":"0x1000000032e"},{"tid":949,"created_ino":"0x1000000032f"},{"tid":952,"created_ino":"0x10000000330"},{"tid":955,"created_ino":"0x10000000331"},{"tid":958,"created_ino":"0x10000000332"},{"tid":961,"created_ino":"0x10000000333"},{"tid":964,"created_ino":"0x10000000334"},{"tid":967,"created_ino":"0x10000000335"},{"tid":970,"created_ino":"0x10000000336"},{"tid":973,"created_ino":"0x10000000337"},{"tid":976,"created_ino":"0x10000000338"},{"tid":979,"created_ino":"0x10000000339"},{"tid":982,"created_ino":"0x1000000033a"},{"tid":985,"created_ino":"0x1000000033b"},{"tid":988,"created_ino":"0x1000000033c"},{"tid":991,"created_ino":"0x1000000033d"},{"tid":994,"created_ino":"0x1000000033e"},{"tid":997,"created_ino":"0x1000000033f"},{"tid":1000,"created_ino":"0x10000000340"},{"tid":1003,"created_ino":"0x10000000341"},{"tid":1006,"created_ino":"0x10000000342"},{"tid":1009,"created_ino":"0x10000000343"},{"tid":1012,"created_ino":"0x10000000344"},{"tid":1015,"created_ino":"0x10000000345"},{"tid":1018,"created_ino":"0x10000000346"},{"tid":1021,"created_ino":"0x10000000347"},{"tid":1024,"created_ino":"0x10000000348"},{"tid":1027,"created_ino":"0x10000000349"},{"tid":1030,"created_ino":"0x1000000034a"},{"tid":1033,"created_ino":"0x1000000034b"},{"tid":1036,"created_ino":"0x1000000034c"},{"tid":1039,"created_ino":"0x1000000034d"},{"tid":1042,"created_ino":"0x1000000034e"},{"tid":1045,"created_ino":"0x1000000034f"},{"tid":1048,"created_ino":"0x10000000350"},{"tid":1051,"created_ino":"0x10000000351"},{"tid":1054,"created_ino":"0x10000000352"},{"tid":1057,"created_ino":"0x10000000353"},{"tid":1060,"created_ino":"0x10000000354"},{"tid":1063,"created_ino":"0x10000000355"},{"tid":1066,"created_ino":"0x10000000356"},{"tid":1069,"created_ino":"0x10000000357"},{"tid":1072,"created_ino":"0x10000000358"},{"tid":1075,"created_ino":"0x10000000359"},{"tid":1078,"created_ino":"0x1000000035a"},{"tid":1081,"created_ino":"0x1000000035b"},{"tid":1084,"created_ino":"0x1000000035c"},{"tid":1087,"created_ino":"0x1000000035d"},{"tid":1090,"created_ino":"0x1000000035e"},{"tid":1093,"created_ino":"0x1000000035f"},{"tid":1096,"created_ino":"0x10000000360"},{"tid":1099,"created_ino":"0x10000000361"},{"tid":1102,"created_ino":"0x10000000362"},{"tid":1105,"created_ino":"0x10000000363"},{"tid":1108,"created_ino":"0x10000000364"},{"tid":1111,"created_ino":"0x10000000365"},{"tid":1114,"created_ino":"0x10000000366"},{"tid":1117,"created_ino":"0x10000000367"},{"tid":1120,"created_ino":"0x10000000368"},{"tid":1123,"created_ino":"0x10000000369"},{"tid":1126,"created_ino":"0x1000000036a"},{"tid":1129,"created_ino":"0x1000000036b"},{"tid":1132,"created_ino":"0x1000000036c"},{"tid":1135,"created_ino":"0x1000000036d"},{"tid":1138,"created_ino":"0x1000000036e"},{"tid":1141,"created_ino":"0x1000000036f"},{"tid":1144,"created_ino":"0x10000000370"},{"tid":1147,"created_ino":"0x10000000371"},{"tid":1150,"created_ino":"0x10000000372"},{"tid":1153,"created_ino":"0x10000000373"},{"tid":1156,"created_ino":"0x10000000374"},{"tid":1159,"created_ino":"0x10000000375"},{"tid":1162,"created_ino":"0x10000000376"},{"tid":1165,"created_ino":"0x10000000377"},{"tid":1168,"created_ino":"0x10000000378"},{"tid":1171,"created_ino":"0x10000000379"},{"tid":1174,"created_ino":"0x1000000037a"},{"tid":1177,"created_ino":"0x1000000037b"},{"tid":1180,"created_ino":"0x1000000037c"},{"tid":1183,"created_ino":"0x1000000037d"},{"tid":1186,"created_ino":"0x1000000037e"},{"tid":1189,"created_ino":"0x1000000037f"},{"tid":1192,"created_ino":"0x10000000380"},{"tid":1195,"created_ino":"0x10000000381"},{"tid":1198,"created_ino":"0x10000000382"},{"tid":1201,"created_ino":"0x10000000383"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000384","length":603}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"0","hostname":"smithi016","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"76175","root":"/"}}] 2024-09-19T07:55:25.453 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "49748", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2861814065, "type": "any" }, "name": { "num": 7654, "type": "client" } }, "id": 7654, "inst": "client.7654 192.168.0.1:0/2861814065", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9436172365018036 }, "state": "open", "uptime": 10.150857921 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "0", "hostname": "smithi016", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "76175", "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000001", "tid": 7 }, { "created_ino": "0x100000001f6", "tid": 10 }, { "created_ino": "0x100000001f7", "tid": 13 }, { "created_ino": "0x100000001f8", "tid": 16 }, { "created_ino": "0x100000001f9", "tid": 19 }, { "created_ino": "0x100000001fa", "tid": 22 }, { "created_ino": "0x100000001fb", "tid": 25 }, { "created_ino": "0x100000001fc", "tid": 28 }, { "created_ino": "0x100000001fd", "tid": 31 }, { "created_ino": "0x100000001fe", "tid": 34 }, { "created_ino": "0x100000001ff", "tid": 37 }, { "created_ino": "0x10000000200", "tid": 40 }, { "created_ino": "0x10000000201", "tid": 43 }, { "created_ino": "0x10000000202", "tid": 46 }, { "created_ino": "0x10000000203", "tid": 49 }, { "created_ino": "0x10000000204", "tid": 52 }, { "created_ino": "0x10000000205", "tid": 55 }, { "created_ino": "0x10000000206", "tid": 58 }, { "created_ino": "0x10000000207", "tid": 61 }, { "created_ino": "0x10000000208", "tid": 64 }, { "created_ino": "0x10000000209", "tid": 67 }, { "created_ino": "0x1000000020a", "tid": 70 }, { "created_ino": "0x1000000020b", "tid": 73 }, { "created_ino": "0x1000000020c", "tid": 76 }, { "created_ino": "0x1000000020d", "tid": 79 }, { "created_ino": "0x1000000020e", "tid": 82 }, { "created_ino": "0x1000000020f", "tid": 85 }, { "created_ino": "0x10000000210", "tid": 88 }, { "created_ino": "0x10000000211", "tid": 91 }, { "created_ino": "0x10000000212", "tid": 94 }, { "created_ino": "0x10000000213", "tid": 97 }, { "created_ino": "0x10000000214", "tid": 100 }, { "created_ino": "0x10000000215", "tid": 103 }, { "created_ino": "0x10000000216", "tid": 106 }, { "created_ino": "0x10000000217", "tid": 109 }, { "created_ino": "0x10000000218", "tid": 112 }, { "created_ino": "0x10000000219", "tid": 115 }, { "created_ino": "0x1000000021a", "tid": 118 }, { "created_ino": "0x1000000021b", "tid": 121 }, { "created_ino": "0x1000000021c", "tid": 124 }, { "created_ino": "0x1000000021d", "tid": 127 }, { "created_ino": "0x1000000021e", "tid": 130 }, { "created_ino": "0x1000000021f", "tid": 133 }, { "created_ino": "0x10000000220", "tid": 136 }, { "created_ino": "0x10000000221", "tid": 139 }, { "created_ino": "0x10000000222", "tid": 142 }, { "created_ino": "0x10000000223", "tid": 145 }, { "created_ino": "0x10000000224", "tid": 148 }, { "created_ino": "0x10000000225", "tid": 151 }, { "created_ino": "0x10000000226", "tid": 154 }, { "created_ino": "0x10000000227", "tid": 157 }, { "created_ino": "0x10000000228", "tid": 160 }, { "created_ino": "0x10000000229", "tid": 163 }, { "created_ino": "0x1000000022a", "tid": 166 }, { "created_ino": "0x1000000022b", "tid": 169 }, { "created_ino": "0x1000000022c", "tid": 172 }, { "created_ino": "0x1000000022d", "tid": 175 }, { "created_ino": "0x1000000022e", "tid": 178 }, { "created_ino": "0x1000000022f", "tid": 181 }, { "created_ino": "0x10000000230", "tid": 184 }, { "created_ino": "0x10000000231", "tid": 187 }, { "created_ino": "0x10000000232", "tid": 190 }, { "created_ino": "0x10000000233", "tid": 193 }, { "created_ino": "0x10000000234", "tid": 196 }, { "created_ino": "0x10000000235", "tid": 199 }, { "created_ino": "0x10000000236", "tid": 202 }, { "created_ino": "0x10000000237", "tid": 205 }, { "created_ino": "0x10000000238", "tid": 208 }, { "created_ino": "0x10000000239", "tid": 211 }, { "created_ino": "0x1000000023a", "tid": 214 }, { "created_ino": "0x1000000023b", "tid": 217 }, { "created_ino": "0x1000000023c", "tid": 220 }, { "created_ino": "0x1000000023d", "tid": 223 }, { "created_ino": "0x1000000023e", "tid": 226 }, { "created_ino": "0x1000000023f", "tid": 229 }, { "created_ino": "0x10000000240", "tid": 232 }, { "created_ino": "0x10000000241", "tid": 235 }, { "created_ino": "0x10000000242", "tid": 238 }, { "created_ino": "0x10000000243", "tid": 241 }, { "created_ino": "0x10000000244", "tid": 244 }, { "created_ino": "0x10000000245", "tid": 247 }, { "created_ino": "0x10000000246", "tid": 250 }, { "created_ino": "0x10000000247", "tid": 253 }, { "created_ino": "0x10000000248", "tid": 256 }, { "created_ino": "0x10000000249", "tid": 259 }, { "created_ino": "0x1000000024a", "tid": 262 }, { "created_ino": "0x1000000024b", "tid": 265 }, { "created_ino": "0x1000000024c", "tid": 268 }, { "created_ino": "0x1000000024d", "tid": 271 }, { "created_ino": "0x1000000024e", "tid": 274 }, { "created_ino": "0x1000000024f", "tid": 277 }, { "created_ino": "0x10000000250", "tid": 280 }, { "created_ino": "0x10000000251", "tid": 283 }, { "created_ino": "0x10000000252", "tid": 286 }, { "created_ino": "0x10000000253", "tid": 289 }, { "created_ino": "0x10000000254", "tid": 292 }, { "created_ino": "0x10000000255", "tid": 295 }, { "created_ino": "0x10000000256", "tid": 298 }, { "created_ino": "0x10000000257", "tid": 301 }, { "created_ino": "0x10000000258", "tid": 304 }, { "created_ino": "0x10000000259", "tid": 307 }, { "created_ino": "0x1000000025a", "tid": 310 }, { "created_ino": "0x1000000025b", "tid": 313 }, { "created_ino": "0x1000000025c", "tid": 316 }, { "created_ino": "0x1000000025d", "tid": 319 }, { "created_ino": "0x1000000025e", "tid": 322 }, { "created_ino": "0x1000000025f", "tid": 325 }, { "created_ino": "0x10000000260", "tid": 328 }, { "created_ino": "0x10000000261", "tid": 331 }, { "created_ino": "0x10000000262", "tid": 334 }, { "created_ino": "0x10000000263", "tid": 337 }, { "created_ino": "0x10000000264", "tid": 340 }, { "created_ino": "0x10000000265", "tid": 343 }, { "created_ino": "0x10000000266", "tid": 346 }, { "created_ino": "0x10000000267", "tid": 349 }, { "created_ino": "0x10000000268", "tid": 352 }, { "created_ino": "0x10000000269", "tid": 355 }, { "created_ino": "0x1000000026a", "tid": 358 }, { "created_ino": "0x1000000026b", "tid": 361 }, { "created_ino": "0x1000000026c", "tid": 364 }, { "created_ino": "0x1000000026d", "tid": 367 }, { "created_ino": "0x1000000026e", "tid": 370 }, { "created_ino": "0x1000000026f", "tid": 373 }, { "created_ino": "0x10000000270", "tid": 376 }, { "created_ino": "0x10000000271", "tid": 379 }, { "created_ino": "0x10000000272", "tid": 382 }, { "created_ino": "0x10000000273", "tid": 385 }, { "created_ino": "0x10000000274", "tid": 388 }, { "created_ino": "0x10000000275", "tid": 391 }, { "created_ino": "0x10000000276", "tid": 394 }, { "created_ino": "0x10000000277", "tid": 397 }, { "created_ino": "0x10000000278", "tid": 400 }, { "created_ino": "0x10000000279", "tid": 403 }, { "created_ino": "0x1000000027a", "tid": 406 }, { "created_ino": "0x1000000027b", "tid": 409 }, { "created_ino": "0x1000000027c", "tid": 412 }, { "created_ino": "0x1000000027d", "tid": 415 }, { "created_ino": "0x1000000027e", "tid": 418 }, { "created_ino": "0x1000000027f", "tid": 421 }, { "created_ino": "0x10000000280", "tid": 424 }, { "created_ino": "0x10000000281", "tid": 427 }, { "created_ino": "0x10000000282", "tid": 430 }, { "created_ino": "0x10000000283", "tid": 433 }, { "created_ino": "0x10000000284", "tid": 436 }, { "created_ino": "0x10000000285", "tid": 439 }, { "created_ino": "0x10000000286", "tid": 442 }, { "created_ino": "0x10000000287", "tid": 445 }, { "created_ino": "0x10000000288", "tid": 448 }, { "created_ino": "0x10000000289", "tid": 451 }, { "created_ino": "0x1000000028a", "tid": 454 }, { "created_ino": "0x1000000028b", "tid": 457 }, { "created_ino": "0x1000000028c", "tid": 460 }, { "created_ino": "0x1000000028d", "tid": 463 }, { "created_ino": "0x1000000028e", "tid": 466 }, { "created_ino": "0x1000000028f", "tid": 469 }, { "created_ino": "0x10000000290", "tid": 472 }, { "created_ino": "0x10000000291", "tid": 475 }, { "created_ino": "0x10000000292", "tid": 478 }, { "created_ino": "0x10000000293", "tid": 481 }, { "created_ino": "0x10000000294", "tid": 484 }, { "created_ino": "0x10000000295", "tid": 487 }, { "created_ino": "0x10000000296", "tid": 490 }, { "created_ino": "0x10000000297", "tid": 493 }, { "created_ino": "0x10000000298", "tid": 496 }, { "created_ino": "0x10000000299", "tid": 499 }, { "created_ino": "0x1000000029a", "tid": 502 }, { "created_ino": "0x1000000029b", "tid": 505 }, { "created_ino": "0x1000000029c", "tid": 508 }, { "created_ino": "0x1000000029d", "tid": 511 }, { "created_ino": "0x1000000029e", "tid": 514 }, { "created_ino": "0x1000000029f", "tid": 517 }, { "created_ino": "0x100000002a0", "tid": 520 }, { "created_ino": "0x100000002a1", "tid": 523 }, { "created_ino": "0x100000002a2", "tid": 526 }, { "created_ino": "0x100000002a3", "tid": 529 }, { "created_ino": "0x100000002a4", "tid": 532 }, { "created_ino": "0x100000002a5", "tid": 535 }, { "created_ino": "0x100000002a6", "tid": 538 }, { "created_ino": "0x100000002a7", "tid": 541 }, { "created_ino": "0x100000002a8", "tid": 544 }, { "created_ino": "0x100000002a9", "tid": 547 }, { "created_ino": "0x100000002aa", "tid": 550 }, { "created_ino": "0x100000002ab", "tid": 553 }, { "created_ino": "0x100000002ac", "tid": 556 }, { "created_ino": "0x100000002ad", "tid": 559 }, { "created_ino": "0x100000002ae", "tid": 562 }, { "created_ino": "0x100000002af", "tid": 565 }, { "created_ino": "0x100000002b0", "tid": 568 }, { "created_ino": "0x100000002b1", "tid": 571 }, { "created_ino": "0x100000002b2", "tid": 574 }, { "created_ino": "0x100000002b3", "tid": 577 }, { "created_ino": "0x100000002b4", "tid": 580 }, { "created_ino": "0x100000002b5", "tid": 583 }, { "created_ino": "0x100000002b6", "tid": 586 }, { "created_ino": "0x100000002b7", "tid": 589 }, { "created_ino": "0x100000002b8", "tid": 592 }, { "created_ino": "0x100000002b9", "tid": 595 }, { "created_ino": "0x100000002ba", "tid": 598 }, { "created_ino": "0x100000002bb", "tid": 601 }, { "created_ino": "0x100000002bc", "tid": 604 }, { "created_ino": "0x100000002bd", "tid": 607 }, { "created_ino": "0x100000002be", "tid": 610 }, { "created_ino": "0x100000002bf", "tid": 613 }, { "created_ino": "0x100000002c0", "tid": 616 }, { "created_ino": "0x100000002c1", "tid": 619 }, { "created_ino": "0x100000002c2", "tid": 622 }, { "created_ino": "0x100000002c3", "tid": 625 }, { "created_ino": "0x100000002c4", "tid": 628 }, { "created_ino": "0x100000002c5", "tid": 631 }, { "created_ino": "0x100000002c6", "tid": 634 }, { "created_ino": "0x100000002c7", "tid": 637 }, { "created_ino": "0x100000002c8", "tid": 640 }, { "created_ino": "0x100000002c9", "tid": 643 }, { "created_ino": "0x100000002ca", "tid": 646 }, { "created_ino": "0x100000002cb", "tid": 649 }, { "created_ino": "0x100000002cc", "tid": 652 }, { "created_ino": "0x100000002cd", "tid": 655 }, { "created_ino": "0x100000002ce", "tid": 658 }, { "created_ino": "0x100000002cf", "tid": 661 }, { "created_ino": "0x100000002d0", "tid": 664 }, { "created_ino": "0x100000002d1", "tid": 667 }, { "created_ino": "0x100000002d2", "tid": 670 }, { "created_ino": "0x100000002d3", "tid": 673 }, { "created_ino": "0x100000002d4", "tid": 676 }, { "created_ino": "0x100000002d5", "tid": 679 }, { "created_ino": "0x100000002d6", "tid": 682 }, { "created_ino": "0x100000002d7", "tid": 685 }, { "created_ino": "0x100000002d8", "tid": 688 }, { "created_ino": "0x100000002d9", "tid": 691 }, { "created_ino": "0x100000002da", "tid": 694 }, { "created_ino": "0x100000002db", "tid": 697 }, { "created_ino": "0x100000002dc", "tid": 700 }, { "created_ino": "0x100000002dd", "tid": 703 }, { "created_ino": "0x100000002de", "tid": 706 }, { "created_ino": "0x100000002df", "tid": 709 }, { "created_ino": "0x100000002e0", "tid": 712 }, { "created_ino": "0x100000002e1", "tid": 715 }, { "created_ino": "0x100000002e2", "tid": 718 }, { "created_ino": "0x100000002e3", "tid": 721 }, { "created_ino": "0x100000002e4", "tid": 724 }, { "created_ino": "0x100000002e5", "tid": 727 }, { "created_ino": "0x100000002e6", "tid": 730 }, { "created_ino": "0x100000002e7", "tid": 733 }, { "created_ino": "0x100000002e8", "tid": 736 }, { "created_ino": "0x100000002e9", "tid": 739 }, { "created_ino": "0x100000002ea", "tid": 742 }, { "created_ino": "0x100000002eb", "tid": 745 }, { "created_ino": "0x100000002ec", "tid": 748 }, { "created_ino": "0x100000002ed", "tid": 751 }, { "created_ino": "0x100000002ee", "tid": 754 }, { "created_ino": "0x100000002ef", "tid": 757 }, { "created_ino": "0x100000002f0", "tid": 760 }, { "created_ino": "0x100000002f1", "tid": 763 }, { "created_ino": "0x100000002f2", "tid": 766 }, { "created_ino": "0x100000002f3", "tid": 769 }, { "created_ino": "0x100000002f4", "tid": 772 }, { "created_ino": "0x100000002f5", "tid": 775 }, { "created_ino": "0x100000002f6", "tid": 778 }, { "created_ino": "0x100000002f7", "tid": 781 }, { "created_ino": "0x100000002f8", "tid": 784 }, { "created_ino": "0x100000002f9", "tid": 787 }, { "created_ino": "0x100000002fa", "tid": 790 }, { "created_ino": "0x100000002fb", "tid": 793 }, { "created_ino": "0x100000002fc", "tid": 796 }, { "created_ino": "0x100000002fd", "tid": 799 }, { "created_ino": "0x100000002fe", "tid": 802 }, { "created_ino": "0x100000002ff", "tid": 805 }, { "created_ino": "0x10000000300", "tid": 808 }, { "created_ino": "0x10000000301", "tid": 811 }, { "created_ino": "0x10000000302", "tid": 814 }, { "created_ino": "0x10000000303", "tid": 817 }, { "created_ino": "0x10000000304", "tid": 820 }, { "created_ino": "0x10000000305", "tid": 823 }, { "created_ino": "0x10000000306", "tid": 826 }, { "created_ino": "0x10000000307", "tid": 829 }, { "created_ino": "0x10000000308", "tid": 832 }, { "created_ino": "0x10000000309", "tid": 835 }, { "created_ino": "0x1000000030a", "tid": 838 }, { "created_ino": "0x1000000030b", "tid": 841 }, { "created_ino": "0x1000000030c", "tid": 844 }, { "created_ino": "0x1000000030d", "tid": 847 }, { "created_ino": "0x1000000030e", "tid": 850 }, { "created_ino": "0x1000000030f", "tid": 853 }, { "created_ino": "0x10000000310", "tid": 856 }, { "created_ino": "0x10000000311", "tid": 859 }, { "created_ino": "0x10000000312", "tid": 862 }, { "created_ino": "0x10000000313", "tid": 865 }, { "created_ino": "0x10000000314", "tid": 868 }, { "created_ino": "0x10000000315", "tid": 871 }, { "created_ino": "0x10000000316", "tid": 874 }, { "created_ino": "0x10000000317", "tid": 877 }, { "created_ino": "0x10000000318", "tid": 880 }, { "created_ino": "0x10000000319", "tid": 883 }, { "created_ino": "0x1000000031a", "tid": 886 }, { "created_ino": "0x1000000031b", "tid": 889 }, { "created_ino": "0x1000000031c", "tid": 892 }, { "created_ino": "0x1000000031d", "tid": 895 }, { "created_ino": "0x1000000031e", "tid": 898 }, { "created_ino": "0x1000000031f", "tid": 901 }, { "created_ino": "0x10000000320", "tid": 904 }, { "created_ino": "0x10000000321", "tid": 907 }, { "created_ino": "0x10000000322", "tid": 910 }, { "created_ino": "0x10000000323", "tid": 913 }, { "created_ino": "0x10000000324", "tid": 916 }, { "created_ino": "0x10000000325", "tid": 919 }, { "created_ino": "0x10000000326", "tid": 922 }, { "created_ino": "0x10000000327", "tid": 925 }, { "created_ino": "0x10000000328", "tid": 928 }, { "created_ino": "0x10000000329", "tid": 931 }, { "created_ino": "0x1000000032a", "tid": 934 }, { "created_ino": "0x1000000032b", "tid": 937 }, { "created_ino": "0x1000000032c", "tid": 940 }, { "created_ino": "0x1000000032d", "tid": 943 }, { "created_ino": "0x1000000032e", "tid": 946 }, { "created_ino": "0x1000000032f", "tid": 949 }, { "created_ino": "0x10000000330", "tid": 952 }, { "created_ino": "0x10000000331", "tid": 955 }, { "created_ino": "0x10000000332", "tid": 958 }, { "created_ino": "0x10000000333", "tid": 961 }, { "created_ino": "0x10000000334", "tid": 964 }, { "created_ino": "0x10000000335", "tid": 967 }, { "created_ino": "0x10000000336", "tid": 970 }, { "created_ino": "0x10000000337", "tid": 973 }, { "created_ino": "0x10000000338", "tid": 976 }, { "created_ino": "0x10000000339", "tid": 979 }, { "created_ino": "0x1000000033a", "tid": 982 }, { "created_ino": "0x1000000033b", "tid": 985 }, { "created_ino": "0x1000000033c", "tid": 988 }, { "created_ino": "0x1000000033d", "tid": 991 }, { "created_ino": "0x1000000033e", "tid": 994 }, { "created_ino": "0x1000000033f", "tid": 997 }, { "created_ino": "0x10000000340", "tid": 1000 }, { "created_ino": "0x10000000341", "tid": 1003 }, { "created_ino": "0x10000000342", "tid": 1006 }, { "created_ino": "0x10000000343", "tid": 1009 }, { "created_ino": "0x10000000344", "tid": 1012 }, { "created_ino": "0x10000000345", "tid": 1015 }, { "created_ino": "0x10000000346", "tid": 1018 }, { "created_ino": "0x10000000347", "tid": 1021 }, { "created_ino": "0x10000000348", "tid": 1024 }, { "created_ino": "0x10000000349", "tid": 1027 }, { "created_ino": "0x1000000034a", "tid": 1030 }, { "created_ino": "0x1000000034b", "tid": 1033 }, { "created_ino": "0x1000000034c", "tid": 1036 }, { "created_ino": "0x1000000034d", "tid": 1039 }, { "created_ino": "0x1000000034e", "tid": 1042 }, { "created_ino": "0x1000000034f", "tid": 1045 }, { "created_ino": "0x10000000350", "tid": 1048 }, { "created_ino": "0x10000000351", "tid": 1051 }, { "created_ino": "0x10000000352", "tid": 1054 }, { "created_ino": "0x10000000353", "tid": 1057 }, { "created_ino": "0x10000000354", "tid": 1060 }, { "created_ino": "0x10000000355", "tid": 1063 }, { "created_ino": "0x10000000356", "tid": 1066 }, { "created_ino": "0x10000000357", "tid": 1069 }, { "created_ino": "0x10000000358", "tid": 1072 }, { "created_ino": "0x10000000359", "tid": 1075 }, { "created_ino": "0x1000000035a", "tid": 1078 }, { "created_ino": "0x1000000035b", "tid": 1081 }, { "created_ino": "0x1000000035c", "tid": 1084 }, { "created_ino": "0x1000000035d", "tid": 1087 }, { "created_ino": "0x1000000035e", "tid": 1090 }, { "created_ino": "0x1000000035f", "tid": 1093 }, { "created_ino": "0x10000000360", "tid": 1096 }, { "created_ino": "0x10000000361", "tid": 1099 }, { "created_ino": "0x10000000362", "tid": 1102 }, { "created_ino": "0x10000000363", "tid": 1105 }, { "created_ino": "0x10000000364", "tid": 1108 }, { "created_ino": "0x10000000365", "tid": 1111 }, { "created_ino": "0x10000000366", "tid": 1114 }, { "created_ino": "0x10000000367", "tid": 1117 }, { "created_ino": "0x10000000368", "tid": 1120 }, { "created_ino": "0x10000000369", "tid": 1123 }, { "created_ino": "0x1000000036a", "tid": 1126 }, { "created_ino": "0x1000000036b", "tid": 1129 }, { "created_ino": "0x1000000036c", "tid": 1132 }, { "created_ino": "0x1000000036d", "tid": 1135 }, { "created_ino": "0x1000000036e", "tid": 1138 }, { "created_ino": "0x1000000036f", "tid": 1141 }, { "created_ino": "0x10000000370", "tid": 1144 }, { "created_ino": "0x10000000371", "tid": 1147 }, { "created_ino": "0x10000000372", "tid": 1150 }, { "created_ino": "0x10000000373", "tid": 1153 }, { "created_ino": "0x10000000374", "tid": 1156 }, { "created_ino": "0x10000000375", "tid": 1159 }, { "created_ino": "0x10000000376", "tid": 1162 }, { "created_ino": "0x10000000377", "tid": 1165 }, { "created_ino": "0x10000000378", "tid": 1168 }, { "created_ino": "0x10000000379", "tid": 1171 }, { "created_ino": "0x1000000037a", "tid": 1174 }, { "created_ino": "0x1000000037b", "tid": 1177 }, { "created_ino": "0x1000000037c", "tid": 1180 }, { "created_ino": "0x1000000037d", "tid": 1183 }, { "created_ino": "0x1000000037e", "tid": 1186 }, { "created_ino": "0x1000000037f", "tid": 1189 }, { "created_ino": "0x10000000380", "tid": 1192 }, { "created_ino": "0x10000000381", "tid": 1195 }, { "created_ino": "0x10000000382", "tid": 1198 }, { "created_ino": "0x10000000383", "tid": 1201 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3786269155, "type": "any" }, "name": { "num": 8484, "type": "client" } }, "id": 8484, "inst": "client.8484 192.168.0.1:0/3786269155", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 7, "num_caps": 401, "num_completed_flushes": 0, "num_completed_requests": 399, "num_leases": 400, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 603, "start": "0x10000000384" } ], "recall_caps": { "halflife": 60, "value": 1345.6304188904933 }, "recall_caps_throttle": { "halflife": 1.5, "value": 421.5563760021554 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 142.84554030034909 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1110, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1182.037175611084 }, "state": "open", "uptime": 12.290827969 } ] 2024-09-19T07:55:25.454 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-19T07:55:25.454 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:55:25.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:55:25.496 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:55:25.551 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:55:25.551 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:55:25.551 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 400\nabs_path = "/home/ubuntu/cephtest/mnt.0/."\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi016') 2024-09-19T07:55:25.551 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:55:25.552 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:55:25.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:55:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.710+0000 7fdb9ec9b640 1 Processor -- start 2024-09-19T07:55:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.711+0000 7fdb9ec9b640 1 -- start start 2024-09-19T07:55:25.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.711+0000 7fdb9ec9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdb980c8d20 0x7fdb9806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.711+0000 7fdb9ec9b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb9806d260 0x7fdb981439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.711+0000 7fdb9ec9b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdb98143f30 0x7fdb980d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.711+0000 7fdb9ec9b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fdb98141960 con 0x7fdb980c8d20 2024-09-19T07:55:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.711+0000 7fdb9ec9b640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fdb98142200 con 0x7fdb98143f30 2024-09-19T07:55:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.711+0000 7fdb9ec9b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9806d6a0 con 0x7fdb9806d260 2024-09-19T07:55:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.712+0000 7fdb9ca37640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdb980c8d20 0x7fdb9806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.712+0000 7fdb9ca37640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdb980c8d20 0x7fdb9806cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41128/0 (socket says 172.21.15.16:41128) 2024-09-19T07:55:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.712+0000 7fdb9ca37640 1 -- 172.21.15.16:0/669503250 learned_addr learned my addr 172.21.15.16:0/669503250 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.712+0000 7fdb97fff640 1 --2- 172.21.15.16:0/669503250 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb9806d260 0x7fdb981439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.712+0000 7fdb9d238640 1 --2- 172.21.15.16:0/669503250 >> v2:172.21.15.146:3300/0 conn(0x7fdb98143f30 0x7fdb980d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.712+0000 7fdb9ca37640 1 -- 172.21.15.16:0/669503250 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb9806d260 msgr2=0x7fdb981439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.712+0000 7fdb9ca37640 1 --2- 172.21.15.16:0/669503250 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb9806d260 0x7fdb981439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.712+0000 7fdb9ca37640 1 -- 172.21.15.16:0/669503250 >> v2:172.21.15.146:3300/0 conn(0x7fdb98143f30 msgr2=0x7fdb980d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.712+0000 7fdb9ca37640 1 --2- 172.21.15.16:0/669503250 >> v2:172.21.15.146:3300/0 conn(0x7fdb98143f30 0x7fdb980d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.712+0000 7fdb9ca37640 1 -- 172.21.15.16:0/669503250 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb980c7140 con 0x7fdb980c8d20 2024-09-19T07:55:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.713+0000 7fdb9ca37640 1 --2- 172.21.15.16:0/669503250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb980c8d20 0x7fdb9806cd20 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fdb9806d820 tx=0x7fdb8c02a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c1c7e94953c122 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:25.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.713+0000 7fdb977fe640 1 -- 172.21.15.16:0/669503250 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c037070 con 0x7fdb980c8d20 2024-09-19T07:55:25.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.713+0000 7fdb977fe640 1 -- 172.21.15.16:0/669503250 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb8c02adf0 con 0x7fdb980c8d20 2024-09-19T07:55:25.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.713+0000 7fdb977fe640 1 -- 172.21.15.16:0/669503250 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c032660 con 0x7fdb980c8d20 2024-09-19T07:55:25.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.713+0000 7fdb9ec9b640 1 -- 172.21.15.16:0/669503250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb980c8d20 msgr2=0x7fdb9806cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:25.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.713+0000 7fdb9ec9b640 1 --2- 172.21.15.16:0/669503250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb980c8d20 0x7fdb9806cd20 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fdb9806d820 tx=0x7fdb8c02a0a0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.713+0000 7fdb9ec9b640 1 -- 172.21.15.16:0/669503250 shutdown_connections 2024-09-19T07:55:25.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.713+0000 7fdb9ec9b640 1 --2- 172.21.15.16:0/669503250 >> v2:172.21.15.146:3300/0 conn(0x7fdb98143f30 0x7fdb980d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.713+0000 7fdb9ec9b640 1 --2- 172.21.15.16:0/669503250 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb9806d260 0x7fdb981439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.713+0000 7fdb9ec9b640 1 --2- 172.21.15.16:0/669503250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb980c8d20 0x7fdb9806cd20 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.713+0000 7fdb9ec9b640 1 -- 172.21.15.16:0/669503250 >> 172.21.15.16:0/669503250 conn(0x7fdb980b3ed0 msgr2=0x7fdb980b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:25.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.713+0000 7fdb9ec9b640 1 -- 172.21.15.16:0/669503250 shutdown_connections 2024-09-19T07:55:25.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.713+0000 7fdb9ec9b640 1 -- 172.21.15.16:0/669503250 wait complete. 2024-09-19T07:55:25.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.715+0000 7fdb9ec9b640 1 Processor -- start 2024-09-19T07:55:25.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.715+0000 7fdb9ec9b640 1 -- start start 2024-09-19T07:55:25.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.715+0000 7fdb9ec9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb9806d260 0x7fdb980b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:25.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.715+0000 7fdb9ec9b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb980c8d20 0x7fdb98131390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb9ec9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb98143f30 0x7fdb9815fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb9ec9b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb98141bb0 con 0x7fdb98143f30 2024-09-19T07:55:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb9ec9b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdb98144760 con 0x7fdb9806d260 2024-09-19T07:55:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb9ec9b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdb98141e90 con 0x7fdb980c8d20 2024-09-19T07:55:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb9d238640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb98143f30 0x7fdb9815fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb9d238640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb98143f30 0x7fdb9815fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41134/0 (socket says 172.21.15.16:41134) 2024-09-19T07:55:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb9d238640 1 -- 172.21.15.16:0/1273011229 learned_addr learned my addr 172.21.15.16:0/1273011229 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb9ca37640 1 --2- 172.21.15.16:0/1273011229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb9806d260 0x7fdb980b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb97fff640 1 --2- 172.21.15.16:0/1273011229 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb980c8d20 0x7fdb98131390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb9d238640 1 -- 172.21.15.16:0/1273011229 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb980c8d20 msgr2=0x7fdb98131390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb9d238640 1 --2- 172.21.15.16:0/1273011229 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb980c8d20 0x7fdb98131390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb9d238640 1 -- 172.21.15.16:0/1273011229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb9806d260 msgr2=0x7fdb980b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb9d238640 1 --2- 172.21.15.16:0/1273011229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb9806d260 0x7fdb980b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb9d238640 1 -- 172.21.15.16:0/1273011229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb98160730 con 0x7fdb98143f30 2024-09-19T07:55:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb9d238640 1 --2- 172.21.15.16:0/1273011229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb98143f30 0x7fdb9815fce0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fdb90007e60 tx=0x7fdb9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb957fa640 1 -- 172.21.15.16:0/1273011229 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90014420 con 0x7fdb98143f30 2024-09-19T07:55:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb957fa640 1 -- 172.21.15.16:0/1273011229 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb90019070 con 0x7fdb98143f30 2024-09-19T07:55:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb9ec9b640 1 -- 172.21.15.16:0/1273011229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb98160a10 con 0x7fdb98143f30 2024-09-19T07:55:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb957fa640 1 -- 172.21.15.16:0/1273011229 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90014820 con 0x7fdb98143f30 2024-09-19T07:55:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.716+0000 7fdb9ec9b640 1 -- 172.21.15.16:0/1273011229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb98160fd0 con 0x7fdb98143f30 2024-09-19T07:55:25.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.717+0000 7fdb9ec9b640 1 -- 172.21.15.16:0/1273011229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb980d1be0 con 0x7fdb98143f30 2024-09-19T07:55:25.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.718+0000 7fdb957fa640 1 -- 172.21.15.16:0/1273011229 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdb900149c0 con 0x7fdb98143f30 2024-09-19T07:55:25.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.718+0000 7fdb957fa640 1 --2- 172.21.15.16:0/1273011229 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdb7c06c2e0 0x7fdb7c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:25.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.718+0000 7fdb957fa640 1 -- 172.21.15.16:0/1273011229 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7fdb9008bbc0 con 0x7fdb98143f30 2024-09-19T07:55:25.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.719+0000 7fdb9ca37640 1 --2- 172.21.15.16:0/1273011229 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdb7c06c2e0 0x7fdb7c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:25.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.719+0000 7fdb9ca37640 1 --2- 172.21.15.16:0/1273011229 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdb7c06c2e0 0x7fdb7c06e7a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c034000 tx=0x7fdb8c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:25.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.723+0000 7fdb957fa640 1 -- 172.21.15.16:0/1273011229 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fdb980d1be0 con 0x7fdb98143f30 2024-09-19T07:55:25.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.910+0000 7fdb9ec9b640 1 -- 172.21.15.16:0/1273011229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdb980b42b0 con 0x7fdb98143f30 2024-09-19T07:55:25.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.910+0000 7fdb957fa640 1 -- 172.21.15.16:0/1273011229 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v140) ==== 80+0+0 (secure 0 0 0) 0x7fdb980b42b0 con 0x7fdb98143f30 2024-09-19T07:55:25.911 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:55:25.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.913+0000 7fdb9ec9b640 1 -- 172.21.15.16:0/1273011229 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdb7c06c2e0 msgr2=0x7fdb7c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:25.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.913+0000 7fdb9ec9b640 1 --2- 172.21.15.16:0/1273011229 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdb7c06c2e0 0x7fdb7c06e7a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c034000 tx=0x7fdb8c02e000 comp rx=0 tx=0).stop 2024-09-19T07:55:25.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.913+0000 7fdb9ec9b640 1 -- 172.21.15.16:0/1273011229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb98143f30 msgr2=0x7fdb9815fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:25.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.913+0000 7fdb9ec9b640 1 --2- 172.21.15.16:0/1273011229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb98143f30 0x7fdb9815fce0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fdb90007e60 tx=0x7fdb9000e4b0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.913+0000 7fdb9ec9b640 1 -- 172.21.15.16:0/1273011229 shutdown_connections 2024-09-19T07:55:25.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.913+0000 7fdb9ec9b640 1 --2- 172.21.15.16:0/1273011229 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdb7c06c2e0 0x7fdb7c06e7a0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.913+0000 7fdb9ec9b640 1 --2- 172.21.15.16:0/1273011229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb98143f30 0x7fdb9815fce0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.913+0000 7fdb9ec9b640 1 --2- 172.21.15.16:0/1273011229 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb980c8d20 0x7fdb98131390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.913+0000 7fdb9ec9b640 1 --2- 172.21.15.16:0/1273011229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb9806d260 0x7fdb980b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:25.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.913+0000 7fdb9ec9b640 1 -- 172.21.15.16:0/1273011229 >> 172.21.15.16:0/1273011229 conn(0x7fdb980b3ed0 msgr2=0x7fdb980d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:25.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.913+0000 7fdb9ec9b640 1 -- 172.21.15.16:0/1273011229 shutdown_connections 2024-09-19T07:55:25.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:25.913+0000 7fdb9ec9b640 1 -- 172.21.15.16:0/1273011229 wait complete. 2024-09-19T07:55:25.924 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-19T07:55:25.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:25.924 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T07:55:25.948 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[76175]: fuse finished with error 0 and tester_r 0 2024-09-19T07:55:25.950 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:55:25.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:25.950 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:55:26.132 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:55:26.132 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:55:26.133 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:55:26.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:55:26.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.276+0000 7f7166284640 1 Processor -- start 2024-09-19T07:55:26.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.276+0000 7f7166284640 1 -- start start 2024-09-19T07:55:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.276+0000 7f7166284640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f716015f0d0 0x7f716015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.276+0000 7f7166284640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71600d4130 0x7f71600d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.277+0000 7f7166284640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71600d4ae0 con 0x7f71600d0070 2024-09-19T07:55:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.277+0000 7f7166284640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f71600d1b10 con 0x7f716015f0d0 2024-09-19T07:55:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.277+0000 7f7166284640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f71600cf2b0 con 0x7f71600d4130 2024-09-19T07:55:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.277+0000 7f715ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f716015f0d0 0x7f716015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.277+0000 7f715ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f716015f0d0 0x7f716015f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41150/0 (socket says 172.21.15.16:41150) 2024-09-19T07:55:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.277+0000 7f715f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f71600d0070 0x7f71600d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50836/0 (socket says 172.21.15.16:50836) 2024-09-19T07:55:26.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.277+0000 7f715ffff640 1 -- 172.21.15.16:0/263634545 learned_addr learned my addr 172.21.15.16:0/263634545 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:26.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.277+0000 7f7164821640 1 --2- 172.21.15.16:0/263634545 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71600d4130 0x7f71600d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:26.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.277+0000 7f715ffff640 1 -- 172.21.15.16:0/263634545 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71600d4130 msgr2=0x7f71600d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:26.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.277+0000 7f715ffff640 1 --2- 172.21.15.16:0/263634545 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71600d4130 0x7f71600d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:26.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.277+0000 7f715ffff640 1 -- 172.21.15.16:0/263634545 >> v1:172.21.15.16:6789/0 conn(0x7f71600d0070 legacy=0x7f71600d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:26.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.277+0000 7f715ffff640 1 -- 172.21.15.16:0/263634545 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71600cf590 con 0x7f716015f0d0 2024-09-19T07:55:26.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.277+0000 7f715effd640 1 -- 172.21.15.16:0/263634545 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1329586700 0 0) 0x7f71600d4ae0 con 0x7f71600d0070 2024-09-19T07:55:26.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.277+0000 7f7164821640 1 --2- 172.21.15.16:0/263634545 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71600d4130 0x7f71600d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:26.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.278+0000 7f715ffff640 1 --2- 172.21.15.16:0/263634545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716015f0d0 0x7f716015f4b0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f7150009870 tx=0x7f715002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac97f5eaa6c6d383 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:26.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.278+0000 7f715effd640 1 -- 172.21.15.16:0/263634545 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7150037070 con 0x7f716015f0d0 2024-09-19T07:55:26.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.278+0000 7f715effd640 1 -- 172.21.15.16:0/263634545 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7150032440 con 0x7f716015f0d0 2024-09-19T07:55:26.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.278+0000 7f715effd640 1 -- 172.21.15.16:0/263634545 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7150032810 con 0x7f716015f0d0 2024-09-19T07:55:26.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.278+0000 7f7166284640 1 -- 172.21.15.16:0/263634545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716015f0d0 msgr2=0x7f716015f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:26.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.278+0000 7f7166284640 1 --2- 172.21.15.16:0/263634545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716015f0d0 0x7f716015f4b0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f7150009870 tx=0x7f715002a1c0 comp rx=0 tx=0).stop 2024-09-19T07:55:26.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.278+0000 7f7166284640 1 -- 172.21.15.16:0/263634545 shutdown_connections 2024-09-19T07:55:26.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.278+0000 7f7166284640 1 --2- 172.21.15.16:0/263634545 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71600d4130 0x7f71600d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:26.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.278+0000 7f7166284640 1 --2- 172.21.15.16:0/263634545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716015f0d0 0x7f716015f4b0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:26.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.278+0000 7f7166284640 1 -- 172.21.15.16:0/263634545 >> 172.21.15.16:0/263634545 conn(0x7f716009d760 msgr2=0x7f716008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:26.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.278+0000 7f7166284640 1 -- 172.21.15.16:0/263634545 shutdown_connections 2024-09-19T07:55:26.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.279+0000 7f7166284640 1 -- 172.21.15.16:0/263634545 wait complete. 2024-09-19T07:55:26.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.280+0000 7f7166284640 1 Processor -- start 2024-09-19T07:55:26.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.280+0000 7f7166284640 1 -- start start 2024-09-19T07:55:26.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f7166284640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71600d0070 0x7f71601576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:26.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f7166284640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71600d4130 0x7f7160157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:26.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f7166284640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716015f0d0 0x7f71601609e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:26.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f7166284640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f71600cef60 con 0x7f71600d4130 2024-09-19T07:55:26.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f7166284640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f71600cfb50 con 0x7f716015f0d0 2024-09-19T07:55:26.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f7166284640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f71600b2f70 con 0x7f71600d0070 2024-09-19T07:55:26.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f715f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71600d4130 0x7f7160157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:26.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f715f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71600d4130 0x7f7160157080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41164/0 (socket says 172.21.15.16:41164) 2024-09-19T07:55:26.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f715f7fe640 1 -- 172.21.15.16:0/1584824634 learned_addr learned my addr 172.21.15.16:0/1584824634 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:26.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f715ffff640 1 --2- 172.21.15.16:0/1584824634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71600d0070 0x7f71601576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:26.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f7164821640 1 --2- 172.21.15.16:0/1584824634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716015f0d0 0x7f71601609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:26.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f715f7fe640 1 -- 172.21.15.16:0/1584824634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71600d0070 msgr2=0x7f71601576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:26.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f715f7fe640 1 --2- 172.21.15.16:0/1584824634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71600d0070 0x7f71601576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:26.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f715f7fe640 1 -- 172.21.15.16:0/1584824634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716015f0d0 msgr2=0x7f71601609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:26.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f715f7fe640 1 --2- 172.21.15.16:0/1584824634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716015f0d0 0x7f71601609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:26.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f715f7fe640 1 -- 172.21.15.16:0/1584824634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71601f4530 con 0x7f71600d4130 2024-09-19T07:55:26.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f7164821640 1 --2- 172.21.15.16:0/1584824634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716015f0d0 0x7f71601609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:26.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f715f7fe640 1 --2- 172.21.15.16:0/1584824634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71600d4130 0x7f7160157080 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f715400c4d0 tx=0x7f715400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:26.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f715cff9640 1 -- 172.21.15.16:0/1584824634 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154018070 con 0x7f71600d4130 2024-09-19T07:55:26.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f715cff9640 1 -- 172.21.15.16:0/1584824634 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7154007500 con 0x7f71600d4130 2024-09-19T07:55:26.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.281+0000 7f7166284640 1 -- 172.21.15.16:0/1584824634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71601f47d0 con 0x7f71600d4130 2024-09-19T07:55:26.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.282+0000 7f715cff9640 1 -- 172.21.15.16:0/1584824634 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154004a20 con 0x7f71600d4130 2024-09-19T07:55:26.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.282+0000 7f7166284640 1 -- 172.21.15.16:0/1584824634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71601f4d90 con 0x7f71600d4130 2024-09-19T07:55:26.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.282+0000 7f7166284640 1 -- 172.21.15.16:0/1584824634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f716006cf30 con 0x7f71600d4130 2024-09-19T07:55:26.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.283+0000 7f715cff9640 1 -- 172.21.15.16:0/1584824634 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f715401b070 con 0x7f71600d4130 2024-09-19T07:55:26.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.283+0000 7f715cff9640 1 --2- 172.21.15.16:0/1584824634 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f714c06c2b0 0x7f714c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:26.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.284+0000 7f715cff9640 1 -- 172.21.15.16:0/1584824634 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f715408a860 con 0x7f71600d4130 2024-09-19T07:55:26.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.284+0000 7f715ffff640 1 --2- 172.21.15.16:0/1584824634 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f714c06c2b0 0x7f714c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:26.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.284+0000 7f715ffff640 1 --2- 172.21.15.16:0/1584824634 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f714c06c2b0 0x7f714c06e770 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7150034000 tx=0x7f715002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:26.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.288+0000 7f715cff9640 1 -- 172.21.15.16:0/1584824634 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f71540587f0 con 0x7f71600d4130 2024-09-19T07:55:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.477+0000 7f7166284640 1 -- 172.21.15.16:0/1584824634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f716006d7b0 con 0x7f71600d4130 2024-09-19T07:55:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.478+0000 7f715cff9640 1 -- 172.21.15.16:0/1584824634 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v140) ==== 80+0+0 (secure 0 0 0) 0x7f715405caf0 con 0x7f71600d4130 2024-09-19T07:55:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:55:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.479+0000 7f7166284640 1 -- 172.21.15.16:0/1584824634 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f714c06c2b0 msgr2=0x7f714c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.479+0000 7f7166284640 1 --2- 172.21.15.16:0/1584824634 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f714c06c2b0 0x7f714c06e770 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7150034000 tx=0x7f715002e000 comp rx=0 tx=0).stop 2024-09-19T07:55:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.480+0000 7f7166284640 1 -- 172.21.15.16:0/1584824634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71600d4130 msgr2=0x7f7160157080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.480+0000 7f7166284640 1 --2- 172.21.15.16:0/1584824634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71600d4130 0x7f7160157080 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f715400c4d0 tx=0x7f715400c990 comp rx=0 tx=0).stop 2024-09-19T07:55:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.480+0000 7f7166284640 1 -- 172.21.15.16:0/1584824634 shutdown_connections 2024-09-19T07:55:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.480+0000 7f7166284640 1 --2- 172.21.15.16:0/1584824634 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f714c06c2b0 0x7f714c06e770 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.480+0000 7f7166284640 1 --2- 172.21.15.16:0/1584824634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716015f0d0 0x7f71601609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.480+0000 7f7166284640 1 --2- 172.21.15.16:0/1584824634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71600d4130 0x7f7160157080 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.480+0000 7f7166284640 1 --2- 172.21.15.16:0/1584824634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71600d0070 0x7f71601576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.480+0000 7f7166284640 1 -- 172.21.15.16:0/1584824634 >> 172.21.15.16:0/1584824634 conn(0x7f716009d760 msgr2=0x7f716006dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.480+0000 7f7166284640 1 -- 172.21.15.16:0/1584824634 shutdown_connections 2024-09-19T07:55:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.480+0000 7f7166284640 1 -- 172.21.15.16:0/1584824634 wait complete. 2024-09-19T07:55:26.491 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2024-09-19T07:55:26.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:26.491 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T07:55:26.520 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[49748]: fuse finished with error 0 and tester_r 0 2024-09-19T07:55:26.522 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:55:26.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:26.523 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T07:55:26.693 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:55:26.693 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:55:26.693 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:55:26.710 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:55:26.710 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:55:26.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:26.711 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T07:55:26.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:55:26.768 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T07:55:26.768 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:55:26.768 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:55:26.768 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:55:26.788 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:55:26.788 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:55:26.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:26.788 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T07:55:26.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:55:26.849 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T07:55:26.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:55:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.953+0000 7ff655998640 1 Processor -- start 2024-09-19T07:55:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.953+0000 7ff655998640 1 -- start start 2024-09-19T07:55:26.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.953+0000 7ff655998640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff65015bee0 0x7ff6500d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:26.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.953+0000 7ff655998640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6500cf910 0x7ff6500d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.953+0000 7ff655998640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6500cdee0 con 0x7ff65015c2c0 2024-09-19T07:55:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.953+0000 7ff655998640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff6500ceaf0 con 0x7ff65015bee0 2024-09-19T07:55:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.953+0000 7ff655998640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff6500ce1c0 con 0x7ff6500cf910 2024-09-19T07:55:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.954+0000 7ff64f7fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6500cf910 0x7ff6500d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.954+0000 7ff64effd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff65015bee0 0x7ff6500d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.954+0000 7ff64e7fc640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ff65015c2c0 0x7ff6500cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.16:34572/0 (socket says 172.21.15.16:34572) 2024-09-19T07:55:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.954+0000 7ff64f7fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6500cf910 0x7ff6500d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:49386/0 (socket says 172.21.15.16:49386) 2024-09-19T07:55:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.954+0000 7ff64e7fc640 1 -- 172.21.15.16:0/3103721553 learned_addr learned my addr 172.21.15.16:0/3103721553 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.954+0000 7ff64f7fe640 1 -- 172.21.15.16:0/3103721553 >> v2:172.21.15.146:3300/0 conn(0x7ff65015bee0 msgr2=0x7ff6500d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.954+0000 7ff64f7fe640 1 --2- 172.21.15.16:0/3103721553 >> v2:172.21.15.146:3300/0 conn(0x7ff65015bee0 0x7ff6500d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.954+0000 7ff64f7fe640 1 -- 172.21.15.16:0/3103721553 >> v1:172.21.15.146:6789/0 conn(0x7ff65015c2c0 legacy=0x7ff6500cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.954+0000 7ff64f7fe640 1 -- 172.21.15.16:0/3103721553 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6500ce4a0 con 0x7ff6500cf910 2024-09-19T07:55:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.954+0000 7ff64effd640 1 --2- 172.21.15.16:0/3103721553 >> v2:172.21.15.146:3300/0 conn(0x7ff65015bee0 0x7ff6500d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:55:26.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.955+0000 7ff64f7fe640 1 --2- 172.21.15.16:0/3103721553 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6500cf910 0x7ff6500d09f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff648009d90 tx=0x7ff64802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29254fcbfc9f2279 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:26.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.955+0000 7ff64dffb640 1 -- 172.21.15.16:0/3103721553 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648007740 con 0x7ff6500cf910 2024-09-19T07:55:26.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.955+0000 7ff64dffb640 1 -- 172.21.15.16:0/3103721553 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6480078e0 con 0x7ff6500cf910 2024-09-19T07:55:26.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.955+0000 7ff64dffb640 1 -- 172.21.15.16:0/3103721553 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648007cb0 con 0x7ff6500cf910 2024-09-19T07:55:26.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.956+0000 7ff655998640 1 -- 172.21.15.16:0/3103721553 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6500cf910 msgr2=0x7ff6500d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:26.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.956+0000 7ff655998640 1 --2- 172.21.15.16:0/3103721553 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6500cf910 0x7ff6500d09f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff648009d90 tx=0x7ff64802a660 comp rx=0 tx=0).stop 2024-09-19T07:55:26.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.956+0000 7ff655998640 1 -- 172.21.15.16:0/3103721553 shutdown_connections 2024-09-19T07:55:26.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.956+0000 7ff655998640 1 --2- 172.21.15.16:0/3103721553 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6500cf910 0x7ff6500d09f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:26.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.956+0000 7ff655998640 1 --2- 172.21.15.16:0/3103721553 >> v2:172.21.15.146:3300/0 conn(0x7ff65015bee0 0x7ff6500d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:26.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.956+0000 7ff655998640 1 -- 172.21.15.16:0/3103721553 >> 172.21.15.16:0/3103721553 conn(0x7ff65006bdf0 msgr2=0x7ff65006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:26.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.956+0000 7ff655998640 1 -- 172.21.15.16:0/3103721553 shutdown_connections 2024-09-19T07:55:26.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.956+0000 7ff655998640 1 -- 172.21.15.16:0/3103721553 wait complete. 2024-09-19T07:55:26.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.957+0000 7ff655998640 1 Processor -- start 2024-09-19T07:55:26.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.957+0000 7ff655998640 1 -- start start 2024-09-19T07:55:26.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.958+0000 7ff655998640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6500cf910 0x7ff65015d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:26.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.958+0000 7ff655998640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65015bee0 0x7ff65014c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:26.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.958+0000 7ff655998640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff65015c2c0 0x7ff650073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:26.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.958+0000 7ff655998640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff65006e040 con 0x7ff65015bee0 2024-09-19T07:55:26.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.958+0000 7ff655998640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff65006dec0 con 0x7ff65015c2c0 2024-09-19T07:55:26.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.958+0000 7ff655998640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff65006e1c0 con 0x7ff6500cf910 2024-09-19T07:55:26.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.958+0000 7ff64e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65015bee0 0x7ff65014c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:26.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.958+0000 7ff64e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65015bee0 0x7ff65014c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41178/0 (socket says 172.21.15.16:41178) 2024-09-19T07:55:26.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.958+0000 7ff64e7fc640 1 -- 172.21.15.16:0/3274057796 learned_addr learned my addr 172.21.15.16:0/3274057796 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:26.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.958+0000 7ff64f7fe640 1 --2- 172.21.15.16:0/3274057796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff65015c2c0 0x7ff650073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:26.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.958+0000 7ff64effd640 1 --2- 172.21.15.16:0/3274057796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6500cf910 0x7ff65015d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:26.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.958+0000 7ff64e7fc640 1 -- 172.21.15.16:0/3274057796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6500cf910 msgr2=0x7ff65015d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.959+0000 7ff64e7fc640 1 --2- 172.21.15.16:0/3274057796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6500cf910 0x7ff65015d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.959+0000 7ff64e7fc640 1 -- 172.21.15.16:0/3274057796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff65015c2c0 msgr2=0x7ff650073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.959+0000 7ff64e7fc640 1 --2- 172.21.15.16:0/3274057796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff65015c2c0 0x7ff650073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.959+0000 7ff64e7fc640 1 -- 172.21.15.16:0/3274057796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff65014cf30 con 0x7ff65015bee0 2024-09-19T07:55:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.959+0000 7ff64effd640 1 --2- 172.21.15.16:0/3274057796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6500cf910 0x7ff65015d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:55:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.959+0000 7ff64e7fc640 1 --2- 172.21.15.16:0/3274057796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65015bee0 0x7ff65014c9f0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7ff64400c400 tx=0x7ff64400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.959+0000 7ff637fff640 1 -- 172.21.15.16:0/3274057796 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644017070 con 0x7ff65015bee0 2024-09-19T07:55:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.959+0000 7ff637fff640 1 -- 172.21.15.16:0/3274057796 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff644005ce0 con 0x7ff65015bee0 2024-09-19T07:55:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.959+0000 7ff637fff640 1 -- 172.21.15.16:0/3274057796 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6440072f0 con 0x7ff65015bee0 2024-09-19T07:55:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.959+0000 7ff655998640 1 -- 172.21.15.16:0/3274057796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff65015dda0 con 0x7ff65015bee0 2024-09-19T07:55:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.959+0000 7ff655998640 1 -- 172.21.15.16:0/3274057796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff65015e360 con 0x7ff65015bee0 2024-09-19T07:55:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.960+0000 7ff655998640 1 -- 172.21.15.16:0/3274057796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6500cf200 con 0x7ff65015bee0 2024-09-19T07:55:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.961+0000 7ff637fff640 1 -- 172.21.15.16:0/3274057796 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff64401a070 con 0x7ff65015bee0 2024-09-19T07:55:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.961+0000 7ff637fff640 1 --2- 172.21.15.16:0/3274057796 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff63c06c2b0 0x7ff63c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.961+0000 7ff637fff640 1 -- 172.21.15.16:0/3274057796 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7ff64408a910 con 0x7ff65015bee0 2024-09-19T07:55:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.961+0000 7ff64effd640 1 --2- 172.21.15.16:0/3274057796 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff63c06c2b0 0x7ff63c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.962+0000 7ff64effd640 1 --2- 172.21.15.16:0/3274057796 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff63c06c2b0 0x7ff63c06e770 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff640009560 tx=0x7ff640009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:26.966+0000 7ff637fff640 1 -- 172.21.15.16:0/3274057796 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff65014cf30 con 0x7ff65015bee0 2024-09-19T07:55:27.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.179+0000 7ff655998640 1 -- 172.21.15.16:0/3274057796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff65006c6a0 con 0x7ff65015bee0 2024-09-19T07:55:27.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.180+0000 7ff637fff640 1 -- 172.21.15.16:0/3274057796 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5230 (secure 0 0 0) 0x7ff6440588a0 con 0x7ff65015bee0 2024-09-19T07:55:27.180 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:27.180 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":83,"btime":"2024-09-19T07:55:23:288383+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/172459502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":172459502},{"type":"v1","addr":"172.21.15.146:6836","nonce":172459502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3417061308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3417061308},{"type":"v1","addr":"172.21.15.146:6837","nonce":3417061308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1810646409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1810646409},{"type":"v1","addr":"172.21.15.16:6834","nonce":1810646409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:55:02.325256+0000","modified":"2024-09-19T07:55:23.002436+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8286":{"gid":8286,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/1929965275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1929965275},{"type":"v1","addr":"172.21.15.16:6835","nonce":1929965275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8286,"qdb_cluster":[8286]},"id":10}]} 2024-09-19T07:55:27.181 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 83 2024-09-19T07:55:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.182+0000 7ff655998640 1 -- 172.21.15.16:0/3274057796 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff63c06c2b0 msgr2=0x7ff63c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.182+0000 7ff655998640 1 --2- 172.21.15.16:0/3274057796 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff63c06c2b0 0x7ff63c06e770 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff640009560 tx=0x7ff640009190 comp rx=0 tx=0).stop 2024-09-19T07:55:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.182+0000 7ff655998640 1 -- 172.21.15.16:0/3274057796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65015bee0 msgr2=0x7ff65014c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.182+0000 7ff655998640 1 --2- 172.21.15.16:0/3274057796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65015bee0 0x7ff65014c9f0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7ff64400c400 tx=0x7ff64400c8c0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.182+0000 7ff655998640 1 -- 172.21.15.16:0/3274057796 shutdown_connections 2024-09-19T07:55:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.182+0000 7ff655998640 1 --2- 172.21.15.16:0/3274057796 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff63c06c2b0 0x7ff63c06e770 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.182+0000 7ff655998640 1 --2- 172.21.15.16:0/3274057796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff65015c2c0 0x7ff650073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.182+0000 7ff655998640 1 --2- 172.21.15.16:0/3274057796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65015bee0 0x7ff65014c9f0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.182+0000 7ff655998640 1 --2- 172.21.15.16:0/3274057796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6500cf910 0x7ff65015d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.182+0000 7ff655998640 1 -- 172.21.15.16:0/3274057796 >> 172.21.15.16:0/3274057796 conn(0x7ff65006bdf0 msgr2=0x7ff6500b4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.182+0000 7ff655998640 1 -- 172.21.15.16:0/3274057796 shutdown_connections 2024-09-19T07:55:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.182+0000 7ff655998640 1 -- 172.21.15.16:0/3274057796 wait complete. 2024-09-19T07:55:27.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:55:27.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.295+0000 7f2726d03640 1 Processor -- start 2024-09-19T07:55:27.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.295+0000 7f2726d03640 1 -- start start 2024-09-19T07:55:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.296+0000 7f2726d03640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2720161770 0x7f272015fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.296+0000 7f2726d03640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27200b3490 0x7f272015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.296+0000 7f2726d03640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f27200720b0 0x7f2720072520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.296+0000 7f2726d03640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2720072a60 con 0x7f2720161770 2024-09-19T07:55:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.296+0000 7f2726d03640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f27200b3bc0 con 0x7f27200720b0 2024-09-19T07:55:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.296+0000 7f2726d03640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f27200b3ea0 con 0x7f27200b3490 2024-09-19T07:55:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.296+0000 7f2724a9f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2720161770 0x7f272015fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.296+0000 7f2724a9f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2720161770 0x7f272015fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41184/0 (socket says 172.21.15.16:41184) 2024-09-19T07:55:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.296+0000 7f2724a9f640 1 -- 172.21.15.16:0/3480207418 learned_addr learned my addr 172.21.15.16:0/3480207418 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.296+0000 7f2724a9f640 1 -- 172.21.15.16:0/3480207418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27200b3490 msgr2=0x7f272015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.296+0000 7f2724a9f640 1 --2- 172.21.15.16:0/3480207418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27200b3490 0x7f272015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.296+0000 7f2724a9f640 1 -- 172.21.15.16:0/3480207418 >> v2:172.21.15.146:3300/0 conn(0x7f27200720b0 msgr2=0x7f2720072520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.296+0000 7f2724a9f640 1 --2- 172.21.15.16:0/3480207418 >> v2:172.21.15.146:3300/0 conn(0x7f27200720b0 0x7f2720072520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.296+0000 7f2724a9f640 1 -- 172.21.15.16:0/3480207418 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27200b4180 con 0x7f2720161770 2024-09-19T07:55:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.297+0000 7f2724a9f640 1 --2- 172.21.15.16:0/3480207418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2720161770 0x7f272015fa10 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f2720073530 tx=0x7f271402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563a4c01c1c05da0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.297+0000 7f271f7fe640 1 -- 172.21.15.16:0/3480207418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714037070 con 0x7f2720161770 2024-09-19T07:55:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.297+0000 7f271f7fe640 1 -- 172.21.15.16:0/3480207418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2714032440 con 0x7f2720161770 2024-09-19T07:55:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.297+0000 7f271f7fe640 1 -- 172.21.15.16:0/3480207418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714032810 con 0x7f2720161770 2024-09-19T07:55:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.298+0000 7f2726d03640 1 -- 172.21.15.16:0/3480207418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2720161770 msgr2=0x7f272015fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.298+0000 7f2726d03640 1 --2- 172.21.15.16:0/3480207418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2720161770 0x7f272015fa10 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f2720073530 tx=0x7f271402a1e0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.298+0000 7f2726d03640 1 -- 172.21.15.16:0/3480207418 shutdown_connections 2024-09-19T07:55:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.298+0000 7f2726d03640 1 --2- 172.21.15.16:0/3480207418 >> v2:172.21.15.146:3300/0 conn(0x7f27200720b0 0x7f2720072520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.298+0000 7f2726d03640 1 --2- 172.21.15.16:0/3480207418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27200b3490 0x7f272015f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.298+0000 7f2726d03640 1 --2- 172.21.15.16:0/3480207418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2720161770 0x7f272015fa10 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.298+0000 7f2726d03640 1 -- 172.21.15.16:0/3480207418 >> 172.21.15.16:0/3480207418 conn(0x7f2720067d70 msgr2=0x7f272006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.298+0000 7f2726d03640 1 -- 172.21.15.16:0/3480207418 shutdown_connections 2024-09-19T07:55:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.298+0000 7f2726d03640 1 -- 172.21.15.16:0/3480207418 wait complete. 2024-09-19T07:55:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.299+0000 7f2726d03640 1 Processor -- start 2024-09-19T07:55:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.299+0000 7f2726d03640 1 -- start start 2024-09-19T07:55:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.300+0000 7f2726d03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27200720b0 0x7f272014ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.300+0000 7f2726d03640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27200b3490 0x7f272014a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.300+0000 7f2726d03640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2720161770 0x7f272014d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.300+0000 7f2726d03640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27200b38d0 con 0x7f27200720b0 2024-09-19T07:55:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.300+0000 7f2726d03640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f272006e150 con 0x7f27200b3490 2024-09-19T07:55:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.300+0000 7f2726d03640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2720161b50 con 0x7f2720161770 2024-09-19T07:55:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.300+0000 7f2724a9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27200720b0 0x7f272014ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.300+0000 7f2724a9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27200720b0 0x7f272014ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41190/0 (socket says 172.21.15.16:41190) 2024-09-19T07:55:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.300+0000 7f2724a9f640 1 -- 172.21.15.16:0/1728638815 learned_addr learned my addr 172.21.15.16:0/1728638815 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.300+0000 7f27252a0640 1 --2- 172.21.15.16:0/1728638815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2720161770 0x7f272014d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.300+0000 7f271ffff640 1 --2- 172.21.15.16:0/1728638815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27200b3490 0x7f272014a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.300+0000 7f2724a9f640 1 -- 172.21.15.16:0/1728638815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2720161770 msgr2=0x7f272014d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.300+0000 7f2724a9f640 1 --2- 172.21.15.16:0/1728638815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2720161770 0x7f272014d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.300+0000 7f2724a9f640 1 -- 172.21.15.16:0/1728638815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27200b3490 msgr2=0x7f272014a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.300+0000 7f2724a9f640 1 --2- 172.21.15.16:0/1728638815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27200b3490 0x7f272014a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.300+0000 7f2724a9f640 1 -- 172.21.15.16:0/1728638815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2720151de0 con 0x7f27200720b0 2024-09-19T07:55:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.300+0000 7f27252a0640 1 --2- 172.21.15.16:0/1728638815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2720161770 0x7f272014d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.301+0000 7f2724a9f640 1 --2- 172.21.15.16:0/1728638815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27200720b0 0x7f272014ef30 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f2714000c00 tx=0x7f27140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.301+0000 7f271d7fa640 1 -- 172.21.15.16:0/1728638815 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714041070 con 0x7f27200720b0 2024-09-19T07:55:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.301+0000 7f271d7fa640 1 -- 172.21.15.16:0/1728638815 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f271403c870 con 0x7f27200720b0 2024-09-19T07:55:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.301+0000 7f2726d03640 1 -- 172.21.15.16:0/1728638815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2720155990 con 0x7f27200720b0 2024-09-19T07:55:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.301+0000 7f271d7fa640 1 -- 172.21.15.16:0/1728638815 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714037040 con 0x7f27200720b0 2024-09-19T07:55:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.301+0000 7f2726d03640 1 -- 172.21.15.16:0/1728638815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f272014da10 con 0x7f27200720b0 2024-09-19T07:55:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.302+0000 7f2726d03640 1 -- 172.21.15.16:0/1728638815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2720072520 con 0x7f27200720b0 2024-09-19T07:55:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.302+0000 7f271d7fa640 1 -- 172.21.15.16:0/1728638815 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2714044070 con 0x7f27200720b0 2024-09-19T07:55:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.303+0000 7f271d7fa640 1 --2- 172.21.15.16:0/1728638815 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f270406c2b0 0x7f270406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.303+0000 7f271d7fa640 1 -- 172.21.15.16:0/1728638815 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f27140ac1c0 con 0x7f27200720b0 2024-09-19T07:55:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.303+0000 7f271ffff640 1 --2- 172.21.15.16:0/1728638815 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f270406c2b0 0x7f270406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:27.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.304+0000 7f271ffff640 1 --2- 172.21.15.16:0/1728638815 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f270406c2b0 0x7f270406e770 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f272015e820 tx=0x7f270c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.307+0000 7f271d7fa640 1 -- 172.21.15.16:0/1728638815 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f271407a790 con 0x7f27200720b0 2024-09-19T07:55:27.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.519+0000 7f2726d03640 1 -- 172.21.15.16:0/1728638815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2720151b00 con 0x7f27200720b0 2024-09-19T07:55:27.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.519+0000 7f271d7fa640 1 -- 172.21.15.16:0/1728638815 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5230 (secure 0 0 0) 0x7f271407a090 con 0x7f27200720b0 2024-09-19T07:55:27.520 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:27.520 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":83,"btime":"2024-09-19T07:55:23:288383+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/172459502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":172459502},{"type":"v1","addr":"172.21.15.146:6836","nonce":172459502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3417061308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3417061308},{"type":"v1","addr":"172.21.15.146:6837","nonce":3417061308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1810646409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1810646409},{"type":"v1","addr":"172.21.15.16:6834","nonce":1810646409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:55:02.325256+0000","modified":"2024-09-19T07:55:23.002436+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8286":{"gid":8286,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/1929965275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1929965275},{"type":"v1","addr":"172.21.15.16:6835","nonce":1929965275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8286,"qdb_cluster":[8286]},"id":10}]} 2024-09-19T07:55:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 83 2024-09-19T07:55:27.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.521+0000 7f2726d03640 1 -- 172.21.15.16:0/1728638815 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f270406c2b0 msgr2=0x7f270406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:27.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.521+0000 7f2726d03640 1 --2- 172.21.15.16:0/1728638815 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f270406c2b0 0x7f270406e770 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f272015e820 tx=0x7f270c008040 comp rx=0 tx=0).stop 2024-09-19T07:55:27.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.521+0000 7f2726d03640 1 -- 172.21.15.16:0/1728638815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27200720b0 msgr2=0x7f272014ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:27.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.521+0000 7f2726d03640 1 --2- 172.21.15.16:0/1728638815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27200720b0 0x7f272014ef30 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f2714000c00 tx=0x7f27140046c0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.521+0000 7f2726d03640 1 -- 172.21.15.16:0/1728638815 shutdown_connections 2024-09-19T07:55:27.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.521+0000 7f2726d03640 1 --2- 172.21.15.16:0/1728638815 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f270406c2b0 0x7f270406e770 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.521+0000 7f2726d03640 1 --2- 172.21.15.16:0/1728638815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2720161770 0x7f272014d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.521+0000 7f2726d03640 1 --2- 172.21.15.16:0/1728638815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27200b3490 0x7f272014a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.521+0000 7f2726d03640 1 --2- 172.21.15.16:0/1728638815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27200720b0 0x7f272014ef30 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.521+0000 7f2726d03640 1 -- 172.21.15.16:0/1728638815 >> 172.21.15.16:0/1728638815 conn(0x7f2720067d70 msgr2=0x7f27200b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:27.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.521+0000 7f2726d03640 1 -- 172.21.15.16:0/1728638815 shutdown_connections 2024-09-19T07:55:27.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.521+0000 7f2726d03640 1 -- 172.21.15.16:0/1728638815 wait complete. 2024-09-19T07:55:27.533 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:55:27.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.635+0000 7f33d4166640 1 Processor -- start 2024-09-19T07:55:27.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.635+0000 7f33d4166640 1 -- start start 2024-09-19T07:55:27.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.636+0000 7f33d4166640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f33cc0d21f0 0x7f33cc0d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:27.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.636+0000 7f33d4166640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33cc141ad0 0x7f33cc141f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:27.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.636+0000 7f33d4166640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33cc0d2b10 con 0x7f33cc072590 2024-09-19T07:55:27.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.636+0000 7f33d4166640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f33cc141750 con 0x7f33cc141ad0 2024-09-19T07:55:27.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.636+0000 7f33d4166640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f33cc06d7f0 con 0x7f33cc0d21f0 2024-09-19T07:55:27.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.636+0000 7f33d2703640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33cc141ad0 0x7f33cc141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:27.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.636+0000 7f33d2703640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33cc141ad0 0x7f33cc141f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41204/0 (socket says 172.21.15.16:41204) 2024-09-19T07:55:27.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.636+0000 7f33d2703640 1 -- 172.21.15.16:0/826757798 learned_addr learned my addr 172.21.15.16:0/826757798 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:27.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.636+0000 7f33d1701640 1 --1- 172.21.15.16:0/826757798 >> v1:172.21.15.16:6789/0 conn(0x7f33cc072590 0x7f33cc06d0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50838/0 (socket says 172.21.15.16:50838) 2024-09-19T07:55:27.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.636+0000 7f33d2703640 1 -- 172.21.15.16:0/826757798 >> v2:172.21.15.146:3300/0 conn(0x7f33cc0d21f0 msgr2=0x7f33cc0d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:27.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.636+0000 7f33d2703640 1 --2- 172.21.15.16:0/826757798 >> v2:172.21.15.146:3300/0 conn(0x7f33cc0d21f0 0x7f33cc0d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.636+0000 7f33d2703640 1 -- 172.21.15.16:0/826757798 >> v1:172.21.15.16:6789/0 conn(0x7f33cc072590 legacy=0x7f33cc06d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:27.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.636+0000 7f33d2703640 1 -- 172.21.15.16:0/826757798 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc1620b0 con 0x7f33cc141ad0 2024-09-19T07:55:27.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.637+0000 7f33d2703640 1 --2- 172.21.15.16:0/826757798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc141ad0 0x7f33cc141f80 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f33cc0cfd80 tx=0x7f33bc02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397ee7a48f38e153 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:27.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.637+0000 7f33d0f00640 1 -- 172.21.15.16:0/826757798 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc036070 con 0x7f33cc141ad0 2024-09-19T07:55:27.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.637+0000 7f33d0f00640 1 -- 172.21.15.16:0/826757798 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33bc031600 con 0x7f33cc141ad0 2024-09-19T07:55:27.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.637+0000 7f33d0f00640 1 -- 172.21.15.16:0/826757798 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc0319d0 con 0x7f33cc141ad0 2024-09-19T07:55:27.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.638+0000 7f33d4166640 1 -- 172.21.15.16:0/826757798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc141ad0 msgr2=0x7f33cc141f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:27.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.638+0000 7f33d4166640 1 --2- 172.21.15.16:0/826757798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc141ad0 0x7f33cc141f80 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f33cc0cfd80 tx=0x7f33bc02a6d0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.638+0000 7f33d4166640 1 -- 172.21.15.16:0/826757798 shutdown_connections 2024-09-19T07:55:27.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.638+0000 7f33d4166640 1 --2- 172.21.15.16:0/826757798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc141ad0 0x7f33cc141f80 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.638+0000 7f33d4166640 1 --2- 172.21.15.16:0/826757798 >> v2:172.21.15.146:3300/0 conn(0x7f33cc0d21f0 0x7f33cc0d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.638+0000 7f33d4166640 1 -- 172.21.15.16:0/826757798 >> 172.21.15.16:0/826757798 conn(0x7f33cc0b3ed0 msgr2=0x7f33cc0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:27.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.638+0000 7f33d4166640 1 -- 172.21.15.16:0/826757798 shutdown_connections 2024-09-19T07:55:27.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.638+0000 7f33d4166640 1 -- 172.21.15.16:0/826757798 wait complete. 2024-09-19T07:55:27.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.639+0000 7f33d4166640 1 Processor -- start 2024-09-19T07:55:27.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.639+0000 7f33d4166640 1 -- start start 2024-09-19T07:55:27.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.640+0000 7f33d4166640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33cc072590 0x7f33cc1405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:27.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.640+0000 7f33d4166640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33cc0d21f0 0x7f33cc159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:27.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.640+0000 7f33d4166640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc141ad0 0x7f33cc15e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:27.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.640+0000 7f33d4166640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc162930 con 0x7f33cc141ad0 2024-09-19T07:55:27.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.640+0000 7f33d4166640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc162650 con 0x7f33cc072590 2024-09-19T07:55:27.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.640+0000 7f33d4166640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f33cc162c10 con 0x7f33cc0d21f0 2024-09-19T07:55:27.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.640+0000 7f33d2703640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc141ad0 0x7f33cc15e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:27.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.640+0000 7f33d2703640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc141ad0 0x7f33cc15e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41216/0 (socket says 172.21.15.16:41216) 2024-09-19T07:55:27.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.640+0000 7f33d2703640 1 -- 172.21.15.16:0/3772208983 learned_addr learned my addr 172.21.15.16:0/3772208983 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:27.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.640+0000 7f33d2703640 1 -- 172.21.15.16:0/3772208983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33cc0d21f0 msgr2=0x7f33cc159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:27.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.640+0000 7f33d2703640 1 --2- 172.21.15.16:0/3772208983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33cc0d21f0 0x7f33cc159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.640+0000 7f33d2703640 1 -- 172.21.15.16:0/3772208983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33cc072590 msgr2=0x7f33cc1405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:27.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.640+0000 7f33d2703640 1 --2- 172.21.15.16:0/3772208983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33cc072590 0x7f33cc1405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.640+0000 7f33d2703640 1 -- 172.21.15.16:0/3772208983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc159e10 con 0x7f33cc141ad0 2024-09-19T07:55:27.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.640+0000 7f33d2703640 1 --2- 172.21.15.16:0/3772208983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc141ad0 0x7f33cc15e730 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f33bc034040 tx=0x7f33bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:27.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.641+0000 7f33ca7fc640 1 -- 172.21.15.16:0/3772208983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc036050 con 0x7f33cc141ad0 2024-09-19T07:55:27.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.641+0000 7f33ca7fc640 1 -- 172.21.15.16:0/3772208983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33bc042070 con 0x7f33cc141ad0 2024-09-19T07:55:27.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.641+0000 7f33ca7fc640 1 -- 172.21.15.16:0/3772208983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc009510 con 0x7f33cc141ad0 2024-09-19T07:55:27.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.641+0000 7f33d4166640 1 -- 172.21.15.16:0/3772208983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33cc15b7d0 con 0x7f33cc141ad0 2024-09-19T07:55:27.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.641+0000 7f33d4166640 1 -- 172.21.15.16:0/3772208983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33cc15bd90 con 0x7f33cc141ad0 2024-09-19T07:55:27.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.642+0000 7f33d4166640 1 -- 172.21.15.16:0/3772208983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33cc15b5f0 con 0x7f33cc141ad0 2024-09-19T07:55:27.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.642+0000 7f33ca7fc640 1 -- 172.21.15.16:0/3772208983 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f33bc007bf0 con 0x7f33cc141ad0 2024-09-19T07:55:27.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.643+0000 7f33ca7fc640 1 --2- 172.21.15.16:0/3772208983 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f33b806c2e0 0x7f33b806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:27.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.643+0000 7f33ca7fc640 1 -- 172.21.15.16:0/3772208983 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f33bc0abf10 con 0x7f33cc141ad0 2024-09-19T07:55:27.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.643+0000 7f33d1f02640 1 --2- 172.21.15.16:0/3772208983 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f33b806c2e0 0x7f33b806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:27.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.643+0000 7f33d1f02640 1 --2- 172.21.15.16:0/3772208983 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f33b806c2e0 0x7f33b806e7a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009710 tx=0x7f33c4006c90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:27.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.647+0000 7f33ca7fc640 1 -- 172.21.15.16:0/3772208983 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f33cc15b5f0 con 0x7f33cc141ad0 2024-09-19T07:55:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.823+0000 7f33d4166640 1 -- 172.21.15.16:0/3772208983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33cc158c30 con 0x7f33cc141ad0 2024-09-19T07:55:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.824+0000 7f33ca7fc640 1 -- 172.21.15.16:0/3772208983 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) ==== 74+0+16976 (secure 0 0 0) 0x7f33bc04b020 con 0x7f33cc141ad0 2024-09-19T07:55:27.824 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:27.824 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":140,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:55:03.967221+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:54:59.994353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-09-19T07:55:01.315629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:55:27.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.826+0000 7f33d4166640 1 -- 172.21.15.16:0/3772208983 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f33b806c2e0 msgr2=0x7f33b806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:27.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.826+0000 7f33d4166640 1 --2- 172.21.15.16:0/3772208983 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f33b806c2e0 0x7f33b806e7a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009710 tx=0x7f33c4006c90 comp rx=0 tx=0).stop 2024-09-19T07:55:27.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.826+0000 7f33d4166640 1 -- 172.21.15.16:0/3772208983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc141ad0 msgr2=0x7f33cc15e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:27.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.826+0000 7f33d4166640 1 --2- 172.21.15.16:0/3772208983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc141ad0 0x7f33cc15e730 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f33bc034040 tx=0x7f33bc004a40 comp rx=0 tx=0).stop 2024-09-19T07:55:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.826+0000 7f33d4166640 1 -- 172.21.15.16:0/3772208983 shutdown_connections 2024-09-19T07:55:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.826+0000 7f33d4166640 1 --2- 172.21.15.16:0/3772208983 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f33b806c2e0 0x7f33b806e7a0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.826+0000 7f33d4166640 1 --2- 172.21.15.16:0/3772208983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc141ad0 0x7f33cc15e730 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.826+0000 7f33d4166640 1 --2- 172.21.15.16:0/3772208983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33cc0d21f0 0x7f33cc159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.826+0000 7f33d4166640 1 --2- 172.21.15.16:0/3772208983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33cc072590 0x7f33cc1405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.826+0000 7f33d4166640 1 -- 172.21.15.16:0/3772208983 >> 172.21.15.16:0/3772208983 conn(0x7f33cc0b3ed0 msgr2=0x7f33cc141ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.826+0000 7f33d4166640 1 -- 172.21.15.16:0/3772208983 shutdown_connections 2024-09-19T07:55:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.826+0000 7f33d4166640 1 -- 172.21.15.16:0/3772208983 wait complete. 2024-09-19T07:55:27.837 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T07:55:27.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:55:27.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.939+0000 7f295a68b640 1 Processor -- start 2024-09-19T07:55:27.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.939+0000 7f295a68b640 1 -- start start 2024-09-19T07:55:27.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.940+0000 7f295a68b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f295415e2a0 0x7f295406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:27.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.940+0000 7f295a68b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f295415e680 0x7f29540cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:27.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.940+0000 7f295a68b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29540d1880 con 0x7f295406d960 2024-09-19T07:55:27.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.940+0000 7f295a68b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f29540d1b60 con 0x7f295415e680 2024-09-19T07:55:27.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.940+0000 7f295a68b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f29540cf2d0 con 0x7f295415e2a0 2024-09-19T07:55:27.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.940+0000 7f29537fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f295415e680 0x7f29540cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:27.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.940+0000 7f2958c28640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f295406d960 0x7f29540d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50850/0 (socket says 172.21.15.16:50850) 2024-09-19T07:55:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.940+0000 7f2958c28640 1 -- 172.21.15.16:0/898664148 learned_addr learned my addr 172.21.15.16:0/898664148 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.940+0000 7f2953fff640 1 --2- 172.21.15.16:0/898664148 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f295415e2a0 0x7f295406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.940+0000 7f29537fe640 1 -- 172.21.15.16:0/898664148 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f295415e2a0 msgr2=0x7f295406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.940+0000 7f29537fe640 1 --2- 172.21.15.16:0/898664148 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f295415e2a0 0x7f295406d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.940+0000 7f29537fe640 1 -- 172.21.15.16:0/898664148 >> v1:172.21.15.16:6789/0 conn(0x7f295406d960 legacy=0x7f29540d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.940+0000 7f29537fe640 1 -- 172.21.15.16:0/898664148 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29540cf5b0 con 0x7f295415e680 2024-09-19T07:55:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.940+0000 7f2952ffd640 1 -- 172.21.15.16:0/898664148 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1562353326 0 0) 0x7f29540d1880 con 0x7f295406d960 2024-09-19T07:55:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.941+0000 7f29537fe640 1 --2- 172.21.15.16:0/898664148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295415e680 0x7f29540cfad0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f29480097a0 tx=0x7f294802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1e9d3f5f536097 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.941+0000 7f2952ffd640 1 -- 172.21.15.16:0/898664148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948037070 con 0x7f295415e680 2024-09-19T07:55:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.941+0000 7f2952ffd640 1 -- 172.21.15.16:0/898664148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2948032440 con 0x7f295415e680 2024-09-19T07:55:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.941+0000 7f2952ffd640 1 -- 172.21.15.16:0/898664148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948032810 con 0x7f295415e680 2024-09-19T07:55:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.942+0000 7f295a68b640 1 -- 172.21.15.16:0/898664148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295415e680 msgr2=0x7f29540cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.942+0000 7f295a68b640 1 --2- 172.21.15.16:0/898664148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295415e680 0x7f29540cfad0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f29480097a0 tx=0x7f294802a180 comp rx=0 tx=0).stop 2024-09-19T07:55:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.942+0000 7f295a68b640 1 -- 172.21.15.16:0/898664148 shutdown_connections 2024-09-19T07:55:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.942+0000 7f295a68b640 1 --2- 172.21.15.16:0/898664148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295415e680 0x7f29540cfad0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.942+0000 7f295a68b640 1 --2- 172.21.15.16:0/898664148 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f295415e2a0 0x7f295406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.942+0000 7f295a68b640 1 -- 172.21.15.16:0/898664148 >> 172.21.15.16:0/898664148 conn(0x7f295409d760 msgr2=0x7f295408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.942+0000 7f295a68b640 1 -- 172.21.15.16:0/898664148 shutdown_connections 2024-09-19T07:55:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.942+0000 7f295a68b640 1 -- 172.21.15.16:0/898664148 wait complete. 2024-09-19T07:55:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.943+0000 7f295a68b640 1 Processor -- start 2024-09-19T07:55:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.943+0000 7f295a68b640 1 -- start start 2024-09-19T07:55:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.944+0000 7f295a68b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f295406d960 0x7f2954156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.944+0000 7f295a68b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f295415e2a0 0x7f295415d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.944+0000 7f295a68b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295415e680 0x7f295415d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.944+0000 7f295a68b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29540d4c00 con 0x7f295415e680 2024-09-19T07:55:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.944+0000 7f295a68b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f29540d4a80 con 0x7f295406d960 2024-09-19T07:55:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.944+0000 7f295a68b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f29540d0010 con 0x7f295415e2a0 2024-09-19T07:55:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.944+0000 7f2958c28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295415e680 0x7f295415d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.944+0000 7f2958c28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295415e680 0x7f295415d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41224/0 (socket says 172.21.15.16:41224) 2024-09-19T07:55:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.944+0000 7f2958c28640 1 -- 172.21.15.16:0/631826467 learned_addr learned my addr 172.21.15.16:0/631826467 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.944+0000 7f29537fe640 1 --2- 172.21.15.16:0/631826467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f295415e2a0 0x7f295415d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.944+0000 7f2953fff640 1 --2- 172.21.15.16:0/631826467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f295406d960 0x7f2954156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.944+0000 7f2958c28640 1 -- 172.21.15.16:0/631826467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f295415e2a0 msgr2=0x7f295415d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.944+0000 7f2958c28640 1 --2- 172.21.15.16:0/631826467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f295415e2a0 0x7f295415d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.944+0000 7f2958c28640 1 -- 172.21.15.16:0/631826467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f295406d960 msgr2=0x7f2954156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.944+0000 7f2958c28640 1 --2- 172.21.15.16:0/631826467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f295406d960 0x7f2954156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.944+0000 7f2958c28640 1 -- 172.21.15.16:0/631826467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29540cb5e0 con 0x7f295415e680 2024-09-19T07:55:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.945+0000 7f2958c28640 1 --2- 172.21.15.16:0/631826467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295415e680 0x7f295415d6b0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f294c00c7a0 tx=0x7f294c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.945+0000 7f2950ff9640 1 -- 172.21.15.16:0/631826467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c018070 con 0x7f295415e680 2024-09-19T07:55:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.945+0000 7f2950ff9640 1 -- 172.21.15.16:0/631826467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f294c00a5e0 con 0x7f295415e680 2024-09-19T07:55:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.945+0000 7f295a68b640 1 -- 172.21.15.16:0/631826467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29540cb8c0 con 0x7f295415e680 2024-09-19T07:55:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.945+0000 7f2950ff9640 1 -- 172.21.15.16:0/631826467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c00aa50 con 0x7f295415e680 2024-09-19T07:55:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.945+0000 7f295a68b640 1 -- 172.21.15.16:0/631826467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29540cbe80 con 0x7f295415e680 2024-09-19T07:55:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.946+0000 7f295a68b640 1 -- 172.21.15.16:0/631826467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f295406d420 con 0x7f295415e680 2024-09-19T07:55:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.946+0000 7f2950ff9640 1 -- 172.21.15.16:0/631826467 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f294c01b070 con 0x7f295415e680 2024-09-19T07:55:27.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.947+0000 7f2950ff9640 1 --2- 172.21.15.16:0/631826467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f294006c2b0 0x7f294006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:27.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.947+0000 7f2950ff9640 1 -- 172.21.15.16:0/631826467 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f294c08ab60 con 0x7f295415e680 2024-09-19T07:55:27.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.947+0000 7f2953fff640 1 --2- 172.21.15.16:0/631826467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f294006c2b0 0x7f294006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:27.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.948+0000 7f2953fff640 1 --2- 172.21.15.16:0/631826467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f294006c2b0 0x7f294006e770 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2944006fd0 tx=0x7f2944008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:27.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:27.951+0000 7f2950ff9640 1 -- 172.21.15.16:0/631826467 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f294c0591d0 con 0x7f295415e680 2024-09-19T07:55:28.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.164+0000 7f295a68b640 1 -- 172.21.15.16:0/631826467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f295406cf90 con 0x7f295415e680 2024-09-19T07:55:28.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.164+0000 7f2950ff9640 1 -- 172.21.15.16:0/631826467 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5230 (secure 0 0 0) 0x7f294c058ad0 con 0x7f295415e680 2024-09-19T07:55:28.165 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:28.165 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":83,"btime":"2024-09-19T07:55:23:288383+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/172459502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":172459502},{"type":"v1","addr":"172.21.15.146:6836","nonce":172459502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3417061308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3417061308},{"type":"v1","addr":"172.21.15.146:6837","nonce":3417061308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1810646409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1810646409},{"type":"v1","addr":"172.21.15.16:6834","nonce":1810646409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:55:02.325256+0000","modified":"2024-09-19T07:55:23.002436+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8286":{"gid":8286,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/1929965275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1929965275},{"type":"v1","addr":"172.21.15.16:6835","nonce":1929965275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8286,"qdb_cluster":[8286]},"id":10}]} 2024-09-19T07:55:28.165 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 83 2024-09-19T07:55:28.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.167+0000 7f295a68b640 1 -- 172.21.15.16:0/631826467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f294006c2b0 msgr2=0x7f294006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:28.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.167+0000 7f295a68b640 1 --2- 172.21.15.16:0/631826467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f294006c2b0 0x7f294006e770 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2944006fd0 tx=0x7f2944008040 comp rx=0 tx=0).stop 2024-09-19T07:55:28.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.167+0000 7f295a68b640 1 -- 172.21.15.16:0/631826467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295415e680 msgr2=0x7f295415d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:28.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.167+0000 7f295a68b640 1 --2- 172.21.15.16:0/631826467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295415e680 0x7f295415d6b0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f294c00c7a0 tx=0x7f294c00cc60 comp rx=0 tx=0).stop 2024-09-19T07:55:28.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.167+0000 7f295a68b640 1 -- 172.21.15.16:0/631826467 shutdown_connections 2024-09-19T07:55:28.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.167+0000 7f295a68b640 1 --2- 172.21.15.16:0/631826467 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f294006c2b0 0x7f294006e770 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.167+0000 7f295a68b640 1 --2- 172.21.15.16:0/631826467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295415e680 0x7f295415d6b0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.167+0000 7f295a68b640 1 --2- 172.21.15.16:0/631826467 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f295415e2a0 0x7f295415d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.167+0000 7f295a68b640 1 --2- 172.21.15.16:0/631826467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f295406d960 0x7f2954156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.167+0000 7f295a68b640 1 -- 172.21.15.16:0/631826467 >> 172.21.15.16:0/631826467 conn(0x7f295409d760 msgr2=0x7f295415fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:28.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.167+0000 7f295a68b640 1 -- 172.21.15.16:0/631826467 shutdown_connections 2024-09-19T07:55:28.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.167+0000 7f295a68b640 1 -- 172.21.15.16:0/631826467 wait complete. 2024-09-19T07:55:28.178 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:55:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.280+0000 7f7c85bf8640 1 Processor -- start 2024-09-19T07:55:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.280+0000 7f7c85bf8640 1 -- start start 2024-09-19T07:55:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.281+0000 7f7c85bf8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7c800d0070 0x7f7c8015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.281+0000 7f7c85bf8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c800d1b10 con 0x7f7c800d4130 2024-09-19T07:55:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.281+0000 7f7c85bf8640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c800cf2b0 con 0x7f7c8015f0d0 2024-09-19T07:55:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.281+0000 7f7c85bf8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f7c800cf590 con 0x7f7c800d0070 2024-09-19T07:55:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.281+0000 7f7c77fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7c800d0070 0x7f7c8015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.281+0000 7f7c7ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7c800d4130 0x7f7c800d4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50866/0 (socket says 172.21.15.16:50866) 2024-09-19T07:55:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.281+0000 7f7c7ffff640 1 -- 172.21.15.16:0/1124454337 learned_addr learned my addr 172.21.15.16:0/1124454337 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.281+0000 7f7c77fff640 1 -- 172.21.15.16:0/1124454337 >> v1:172.21.15.146:6789/0 conn(0x7f7c8015f0d0 legacy=0x7f7c800d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.281+0000 7f7c77fff640 1 -- 172.21.15.16:0/1124454337 >> v1:172.21.15.16:6789/0 conn(0x7f7c800d4130 legacy=0x7f7c800d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.281+0000 7f7c77fff640 1 -- 172.21.15.16:0/1124454337 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c800cf870 con 0x7f7c800d0070 2024-09-19T07:55:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.282+0000 7f7c77fff640 1 --2- 172.21.15.16:0/1124454337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c800d0070 0x7f7c8015f4b0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f7c680097a0 tx=0x7f7c6802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0b6a139297a38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.282+0000 7f7c7effd640 1 -- 172.21.15.16:0/1124454337 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c68037070 con 0x7f7c800d0070 2024-09-19T07:55:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.282+0000 7f7c7effd640 1 -- 172.21.15.16:0/1124454337 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c68032440 con 0x7f7c800d0070 2024-09-19T07:55:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.282+0000 7f7c7effd640 1 -- 172.21.15.16:0/1124454337 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6802ae60 con 0x7f7c800d0070 2024-09-19T07:55:28.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.283+0000 7f7c85bf8640 1 -- 172.21.15.16:0/1124454337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c800d0070 msgr2=0x7f7c8015f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:28.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.283+0000 7f7c85bf8640 1 --2- 172.21.15.16:0/1124454337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c800d0070 0x7f7c8015f4b0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f7c680097a0 tx=0x7f7c6802a110 comp rx=0 tx=0).stop 2024-09-19T07:55:28.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.283+0000 7f7c85bf8640 1 -- 172.21.15.16:0/1124454337 shutdown_connections 2024-09-19T07:55:28.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.283+0000 7f7c85bf8640 1 --2- 172.21.15.16:0/1124454337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c800d0070 0x7f7c8015f4b0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.283+0000 7f7c85bf8640 1 -- 172.21.15.16:0/1124454337 >> 172.21.15.16:0/1124454337 conn(0x7f7c8009d760 msgr2=0x7f7c8008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:28.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.283+0000 7f7c85bf8640 1 -- 172.21.15.16:0/1124454337 shutdown_connections 2024-09-19T07:55:28.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.283+0000 7f7c85bf8640 1 -- 172.21.15.16:0/1124454337 wait complete. 2024-09-19T07:55:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.284+0000 7f7c85bf8640 1 Processor -- start 2024-09-19T07:55:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.284+0000 7f7c85bf8640 1 -- start start 2024-09-19T07:55:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c85bf8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c800d0070 0x7f7c800cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c85bf8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c800d4130 0x7f7c800b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c85bf8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c8015f0d0 0x7f7c800ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c85bf8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c800b2f70 con 0x7f7c8015f0d0 2024-09-19T07:55:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c85bf8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7c800cef60 con 0x7f7c800d4130 2024-09-19T07:55:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c85bf8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7c800b30f0 con 0x7f7c800d0070 2024-09-19T07:55:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c7ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c8015f0d0 0x7f7c800ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c7ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c8015f0d0 0x7f7c800ca780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41238/0 (socket says 172.21.15.16:41238) 2024-09-19T07:55:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c7ffff640 1 -- 172.21.15.16:0/2343499153 learned_addr learned my addr 172.21.15.16:0/2343499153 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c7f7fe640 1 --2- 172.21.15.16:0/2343499153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c800d0070 0x7f7c800cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c77fff640 1 --2- 172.21.15.16:0/2343499153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c800d4130 0x7f7c800b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c7ffff640 1 -- 172.21.15.16:0/2343499153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c800d0070 msgr2=0x7f7c800cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c7ffff640 1 --2- 172.21.15.16:0/2343499153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c800d0070 0x7f7c800cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c7ffff640 1 -- 172.21.15.16:0/2343499153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c800d4130 msgr2=0x7f7c800b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c7ffff640 1 --2- 172.21.15.16:0/2343499153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c800d4130 0x7f7c800b7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c7ffff640 1 -- 172.21.15.16:0/2343499153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8015fd30 con 0x7f7c8015f0d0 2024-09-19T07:55:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c7ffff640 1 --2- 172.21.15.16:0/2343499153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c8015f0d0 0x7f7c800ca780 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800cb10 tx=0x7f7c7800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c7cff9640 1 -- 172.21.15.16:0/2343499153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78017070 con 0x7f7c8015f0d0 2024-09-19T07:55:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c7cff9640 1 -- 172.21.15.16:0/2343499153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c7800ac70 con 0x7f7c8015f0d0 2024-09-19T07:55:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.285+0000 7f7c7cff9640 1 -- 172.21.15.16:0/2343499153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78012620 con 0x7f7c8015f0d0 2024-09-19T07:55:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.286+0000 7f7c85bf8640 1 -- 172.21.15.16:0/2343499153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c80160010 con 0x7f7c8015f0d0 2024-09-19T07:55:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.286+0000 7f7c85bf8640 1 -- 172.21.15.16:0/2343499153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c801605d0 con 0x7f7c8015f0d0 2024-09-19T07:55:28.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.287+0000 7f7c85bf8640 1 -- 172.21.15.16:0/2343499153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c800ce630 con 0x7f7c8015f0d0 2024-09-19T07:55:28.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.287+0000 7f7c7cff9640 1 -- 172.21.15.16:0/2343499153 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7c7801a070 con 0x7f7c8015f0d0 2024-09-19T07:55:28.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.287+0000 7f7c7cff9640 1 --2- 172.21.15.16:0/2343499153 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7c6006c2e0 0x7f7c6006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:28.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.288+0000 7f7c7cff9640 1 -- 172.21.15.16:0/2343499153 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f7c7808b250 con 0x7f7c8015f0d0 2024-09-19T07:55:28.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.288+0000 7f7c7f7fe640 1 --2- 172.21.15.16:0/2343499153 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7c6006c2e0 0x7f7c6006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:28.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.288+0000 7f7c7f7fe640 1 --2- 172.21.15.16:0/2343499153 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7c6006c2e0 0x7f7c6006e7a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7c700016e0 tx=0x7f7c70009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:28.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.292+0000 7f7c7cff9640 1 -- 172.21.15.16:0/2343499153 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7c800ce630 con 0x7f7c8015f0d0 2024-09-19T07:55:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.503+0000 7f7c85bf8640 1 -- 172.21.15.16:0/2343499153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c8015f4b0 con 0x7f7c8015f0d0 2024-09-19T07:55:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.504+0000 7f7c7cff9640 1 -- 172.21.15.16:0/2343499153 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5230 (secure 0 0 0) 0x7f7c7800ae20 con 0x7f7c8015f0d0 2024-09-19T07:55:28.505 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:28.506 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":83,"btime":"2024-09-19T07:55:23:288383+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/172459502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":172459502},{"type":"v1","addr":"172.21.15.146:6836","nonce":172459502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3417061308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3417061308},{"type":"v1","addr":"172.21.15.146:6837","nonce":3417061308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1810646409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1810646409},{"type":"v1","addr":"172.21.15.16:6834","nonce":1810646409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:55:02.325256+0000","modified":"2024-09-19T07:55:23.002436+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8286":{"gid":8286,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":8,"addr":"172.21.15.16:6835/1929965275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1929965275},{"type":"v1","addr":"172.21.15.16:6835","nonce":1929965275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8286,"qdb_cluster":[8286]},"id":10}]} 2024-09-19T07:55:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 83 2024-09-19T07:55:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.506+0000 7f7c85bf8640 1 -- 172.21.15.16:0/2343499153 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7c6006c2e0 msgr2=0x7f7c6006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.506+0000 7f7c85bf8640 1 --2- 172.21.15.16:0/2343499153 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7c6006c2e0 0x7f7c6006e7a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7c700016e0 tx=0x7f7c70009040 comp rx=0 tx=0).stop 2024-09-19T07:55:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.506+0000 7f7c85bf8640 1 -- 172.21.15.16:0/2343499153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c8015f0d0 msgr2=0x7f7c800ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.506+0000 7f7c85bf8640 1 --2- 172.21.15.16:0/2343499153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c8015f0d0 0x7f7c800ca780 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800cb10 tx=0x7f7c7800a060 comp rx=0 tx=0).stop 2024-09-19T07:55:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.506+0000 7f7c85bf8640 1 -- 172.21.15.16:0/2343499153 shutdown_connections 2024-09-19T07:55:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.506+0000 7f7c85bf8640 1 --2- 172.21.15.16:0/2343499153 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7c6006c2e0 0x7f7c6006e7a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.506+0000 7f7c85bf8640 1 --2- 172.21.15.16:0/2343499153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c8015f0d0 0x7f7c800ca780 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.506+0000 7f7c85bf8640 1 --2- 172.21.15.16:0/2343499153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c800d4130 0x7f7c800b7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.506+0000 7f7c85bf8640 1 --2- 172.21.15.16:0/2343499153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c800d0070 0x7f7c800cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.506+0000 7f7c85bf8640 1 -- 172.21.15.16:0/2343499153 >> 172.21.15.16:0/2343499153 conn(0x7f7c8009d760 msgr2=0x7f7c800d2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.506+0000 7f7c85bf8640 1 -- 172.21.15.16:0/2343499153 shutdown_connections 2024-09-19T07:55:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.506+0000 7f7c85bf8640 1 -- 172.21.15.16:0/2343499153 wait complete. 2024-09-19T07:55:28.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:55:28.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.619+0000 7fb14f79f640 1 Processor -- start 2024-09-19T07:55:28.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.619+0000 7fb14f79f640 1 -- start start 2024-09-19T07:55:28.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.620+0000 7fb14f79f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb14815e2a0 0x7fb14806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:28.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.620+0000 7fb14f79f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb14806d960 0x7fb1480d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:28.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.620+0000 7fb14f79f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1480d1880 con 0x7fb14815e680 2024-09-19T07:55:28.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.620+0000 7fb14f79f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb1480d1b60 con 0x7fb14806d960 2024-09-19T07:55:28.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.620+0000 7fb14f79f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb1480cf2d0 con 0x7fb14815e2a0 2024-09-19T07:55:28.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.620+0000 7fb14dd3c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb14806d960 0x7fb1480d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:28.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.620+0000 7fb14cd3a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb14815e680 0x7fb1480cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50872/0 (socket says 172.21.15.16:50872) 2024-09-19T07:55:28.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.620+0000 7fb14cd3a640 1 -- 172.21.15.16:0/450580424 learned_addr learned my addr 172.21.15.16:0/450580424 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:28.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.620+0000 7fb14d53b640 1 --2- 172.21.15.16:0/450580424 >> v2:172.21.15.146:3300/0 conn(0x7fb14815e2a0 0x7fb14806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:28.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.620+0000 7fb14dd3c640 1 -- 172.21.15.16:0/450580424 >> v2:172.21.15.146:3300/0 conn(0x7fb14815e2a0 msgr2=0x7fb14806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:28.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.620+0000 7fb14dd3c640 1 --2- 172.21.15.16:0/450580424 >> v2:172.21.15.146:3300/0 conn(0x7fb14815e2a0 0x7fb14806d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.620+0000 7fb14dd3c640 1 -- 172.21.15.16:0/450580424 >> v1:172.21.15.16:6789/0 conn(0x7fb14815e680 legacy=0x7fb1480cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:28.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.620+0000 7fb13bfff640 1 -- 172.21.15.16:0/450580424 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2346473595 0 0) 0x7fb1480d1880 con 0x7fb14815e680 2024-09-19T07:55:28.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.620+0000 7fb14dd3c640 1 -- 172.21.15.16:0/450580424 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1480cf5b0 con 0x7fb14806d960 2024-09-19T07:55:28.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.621+0000 7fb14dd3c640 1 --2- 172.21.15.16:0/450580424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb14806d960 0x7fb1480d4370 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fb1480b6fa0 tx=0x7fb14002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7660590b87c36d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:28.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.621+0000 7fb13bfff640 1 -- 172.21.15.16:0/450580424 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb140007740 con 0x7fb14806d960 2024-09-19T07:55:28.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.621+0000 7fb13bfff640 1 -- 172.21.15.16:0/450580424 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1400078e0 con 0x7fb14806d960 2024-09-19T07:55:28.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.621+0000 7fb13bfff640 1 -- 172.21.15.16:0/450580424 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb140007cb0 con 0x7fb14806d960 2024-09-19T07:55:28.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.622+0000 7fb14f79f640 1 -- 172.21.15.16:0/450580424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb14806d960 msgr2=0x7fb1480d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:28.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.622+0000 7fb14f79f640 1 --2- 172.21.15.16:0/450580424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb14806d960 0x7fb1480d4370 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fb1480b6fa0 tx=0x7fb14002a680 comp rx=0 tx=0).stop 2024-09-19T07:55:28.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.622+0000 7fb14f79f640 1 -- 172.21.15.16:0/450580424 shutdown_connections 2024-09-19T07:55:28.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.622+0000 7fb14f79f640 1 --2- 172.21.15.16:0/450580424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb14806d960 0x7fb1480d4370 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.622+0000 7fb14f79f640 1 --2- 172.21.15.16:0/450580424 >> v2:172.21.15.146:3300/0 conn(0x7fb14815e2a0 0x7fb14806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.622+0000 7fb14f79f640 1 -- 172.21.15.16:0/450580424 >> 172.21.15.16:0/450580424 conn(0x7fb14809d760 msgr2=0x7fb14808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:28.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.622+0000 7fb14f79f640 1 -- 172.21.15.16:0/450580424 shutdown_connections 2024-09-19T07:55:28.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.622+0000 7fb14f79f640 1 -- 172.21.15.16:0/450580424 wait complete. 2024-09-19T07:55:28.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.623+0000 7fb14f79f640 1 Processor -- start 2024-09-19T07:55:28.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.623+0000 7fb14f79f640 1 -- start start 2024-09-19T07:55:28.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14f79f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb14806d960 0x7fb1481562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:28.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14f79f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb14815e2a0 0x7fb1480c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:28.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14f79f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb14815e680 0x7fb1481612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:28.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14f79f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb1480d0360 con 0x7fb14815e680 2024-09-19T07:55:28.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14f79f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb1480d01e0 con 0x7fb14806d960 2024-09-19T07:55:28.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14f79f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb1480d48b0 con 0x7fb14815e2a0 2024-09-19T07:55:28.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14dd3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb14815e680 0x7fb1481612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:28.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14dd3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb14815e680 0x7fb1481612a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41260/0 (socket says 172.21.15.16:41260) 2024-09-19T07:55:28.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14dd3c640 1 -- 172.21.15.16:0/1349533477 learned_addr learned my addr 172.21.15.16:0/1349533477 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:28.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14d53b640 1 --2- 172.21.15.16:0/1349533477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb14806d960 0x7fb1481562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:28.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14cd3a640 1 --2- 172.21.15.16:0/1349533477 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb14815e2a0 0x7fb1480c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:28.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14dd3c640 1 -- 172.21.15.16:0/1349533477 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb14815e2a0 msgr2=0x7fb1480c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:28.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14dd3c640 1 --2- 172.21.15.16:0/1349533477 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb14815e2a0 0x7fb1480c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14dd3c640 1 -- 172.21.15.16:0/1349533477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb14806d960 msgr2=0x7fb1481562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:28.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14dd3c640 1 --2- 172.21.15.16:0/1349533477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb14806d960 0x7fb1481562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14dd3c640 1 -- 172.21.15.16:0/1349533477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1480c8840 con 0x7fb14815e680 2024-09-19T07:55:28.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14d53b640 1 --2- 172.21.15.16:0/1349533477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb14806d960 0x7fb1481562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:28.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14cd3a640 1 --2- 172.21.15.16:0/1349533477 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb14815e2a0 0x7fb1480c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:28.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14dd3c640 1 --2- 172.21.15.16:0/1349533477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb14815e680 0x7fb1481612a0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fb14002ac70 tx=0x7fb140004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:28.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb139ffb640 1 -- 172.21.15.16:0/1349533477 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb140040070 con 0x7fb14815e680 2024-09-19T07:55:28.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb139ffb640 1 -- 172.21.15.16:0/1349533477 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb140031bc0 con 0x7fb14815e680 2024-09-19T07:55:28.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.624+0000 7fb14f79f640 1 -- 172.21.15.16:0/1349533477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1480c8b80 con 0x7fb14815e680 2024-09-19T07:55:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.625+0000 7fb139ffb640 1 -- 172.21.15.16:0/1349533477 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb140045460 con 0x7fb14815e680 2024-09-19T07:55:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.625+0000 7fb14f79f640 1 -- 172.21.15.16:0/1349533477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1480b8710 con 0x7fb14815e680 2024-09-19T07:55:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.626+0000 7fb14f79f640 1 -- 172.21.15.16:0/1349533477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1480b8910 con 0x7fb14815e680 2024-09-19T07:55:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.626+0000 7fb139ffb640 1 -- 172.21.15.16:0/1349533477 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb140039040 con 0x7fb14815e680 2024-09-19T07:55:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.627+0000 7fb139ffb640 1 --2- 172.21.15.16:0/1349533477 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb12c06c2b0 0x7fb12c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.627+0000 7fb139ffb640 1 -- 172.21.15.16:0/1349533477 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7fb1400acfd0 con 0x7fb14815e680 2024-09-19T07:55:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.627+0000 7fb14d53b640 1 --2- 172.21.15.16:0/1349533477 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb12c06c2b0 0x7fb12c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.627+0000 7fb14d53b640 1 --2- 172.21.15.16:0/1349533477 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb12c06c2b0 0x7fb12c06e770 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb144006fd0 tx=0x7fb144008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:28.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.631+0000 7fb139ffb640 1 -- 172.21.15.16:0/1349533477 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb1480b8910 con 0x7fb14815e680 2024-09-19T07:55:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.809+0000 7fb14f79f640 1 -- 172.21.15.16:0/1349533477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1480c8d60 con 0x7fb14815e680 2024-09-19T07:55:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.810+0000 7fb139ffb640 1 -- 172.21.15.16:0/1349533477 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) ==== 74+0+16976 (secure 0 0 0) 0x7fb1480c8b80 con 0x7fb14815e680 2024-09-19T07:55:28.810 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:28.811 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":140,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:55:03.967221+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:54:59.994353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-09-19T07:55:01.315629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:55:28.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.812+0000 7fb14f79f640 1 -- 172.21.15.16:0/1349533477 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb12c06c2b0 msgr2=0x7fb12c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:28.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.812+0000 7fb14f79f640 1 --2- 172.21.15.16:0/1349533477 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb12c06c2b0 0x7fb12c06e770 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb144006fd0 tx=0x7fb144008040 comp rx=0 tx=0).stop 2024-09-19T07:55:28.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.812+0000 7fb14f79f640 1 -- 172.21.15.16:0/1349533477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb14815e680 msgr2=0x7fb1481612a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:28.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.812+0000 7fb14f79f640 1 --2- 172.21.15.16:0/1349533477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb14815e680 0x7fb1481612a0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fb14002ac70 tx=0x7fb140004a40 comp rx=0 tx=0).stop 2024-09-19T07:55:28.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.812+0000 7fb14f79f640 1 -- 172.21.15.16:0/1349533477 shutdown_connections 2024-09-19T07:55:28.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.812+0000 7fb14f79f640 1 --2- 172.21.15.16:0/1349533477 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb12c06c2b0 0x7fb12c06e770 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.812+0000 7fb14f79f640 1 --2- 172.21.15.16:0/1349533477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb14815e680 0x7fb1481612a0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.812+0000 7fb14f79f640 1 --2- 172.21.15.16:0/1349533477 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb14815e2a0 0x7fb1480c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.812+0000 7fb14f79f640 1 --2- 172.21.15.16:0/1349533477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb14806d960 0x7fb1481562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.812+0000 7fb14f79f640 1 -- 172.21.15.16:0/1349533477 >> 172.21.15.16:0/1349533477 conn(0x7fb14809d760 msgr2=0x7fb14815f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:28.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.812+0000 7fb14f79f640 1 -- 172.21.15.16:0/1349533477 shutdown_connections 2024-09-19T07:55:28.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.812+0000 7fb14f79f640 1 -- 172.21.15.16:0/1349533477 wait complete. 2024-09-19T07:55:28.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T07:55:28.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.927+0000 7f55f4a27640 1 Processor -- start 2024-09-19T07:55:28.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.927+0000 7f55f4a27640 1 -- start start 2024-09-19T07:55:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.927+0000 7f55f4a27640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55f015bee0 0x7f55f00d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.928+0000 7f55f4a27640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55f00cf910 0x7f55f00d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.928+0000 7f55f4a27640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55f00cdee0 con 0x7f55f015c2c0 2024-09-19T07:55:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.928+0000 7f55f4a27640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f55f00ceaf0 con 0x7f55f00cf910 2024-09-19T07:55:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.928+0000 7f55f4a27640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f55f00ce1c0 con 0x7f55f015bee0 2024-09-19T07:55:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.928+0000 7f55eed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55f00cf910 0x7f55f00d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.928+0000 7f55edd9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f55f015c2c0 0x7f55f00cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50878/0 (socket says 172.21.15.16:50878) 2024-09-19T07:55:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.928+0000 7f55edd9b640 1 -- 172.21.15.16:0/400974331 learned_addr learned my addr 172.21.15.16:0/400974331 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.928+0000 7f55ee59c640 1 --2- 172.21.15.16:0/400974331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55f015bee0 0x7f55f00d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.928+0000 7f55eed9d640 1 -- 172.21.15.16:0/400974331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55f015bee0 msgr2=0x7f55f00d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.928+0000 7f55eed9d640 1 --2- 172.21.15.16:0/400974331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55f015bee0 0x7f55f00d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.928+0000 7f55eed9d640 1 -- 172.21.15.16:0/400974331 >> v1:172.21.15.16:6789/0 conn(0x7f55f015c2c0 legacy=0x7f55f00cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.928+0000 7f55eed9d640 1 -- 172.21.15.16:0/400974331 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f00ce4a0 con 0x7f55f00cf910 2024-09-19T07:55:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.928+0000 7f55ee59c640 1 --2- 172.21.15.16:0/400974331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55f015bee0 0x7f55f00d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.929+0000 7f55eed9d640 1 --2- 172.21.15.16:0/400974331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00cf910 0x7f55f00d09f0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f55e80097a0 tx=0x7f55e802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed8e204265b6d29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.929+0000 7f55ed59a640 1 -- 172.21.15.16:0/400974331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e8037070 con 0x7f55f00cf910 2024-09-19T07:55:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.929+0000 7f55ed59a640 1 -- 172.21.15.16:0/400974331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e8032440 con 0x7f55f00cf910 2024-09-19T07:55:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.929+0000 7f55ed59a640 1 -- 172.21.15.16:0/400974331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e8032810 con 0x7f55f00cf910 2024-09-19T07:55:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.929+0000 7f55f4a27640 1 -- 172.21.15.16:0/400974331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00cf910 msgr2=0x7f55f00d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.929+0000 7f55f4a27640 1 --2- 172.21.15.16:0/400974331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00cf910 0x7f55f00d09f0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f55e80097a0 tx=0x7f55e802a180 comp rx=0 tx=0).stop 2024-09-19T07:55:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.930+0000 7f55f4a27640 1 -- 172.21.15.16:0/400974331 shutdown_connections 2024-09-19T07:55:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.930+0000 7f55f4a27640 1 --2- 172.21.15.16:0/400974331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00cf910 0x7f55f00d09f0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.930+0000 7f55f4a27640 1 --2- 172.21.15.16:0/400974331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55f015bee0 0x7f55f00d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.930+0000 7f55f4a27640 1 -- 172.21.15.16:0/400974331 >> 172.21.15.16:0/400974331 conn(0x7f55f006bdf0 msgr2=0x7f55f006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.930+0000 7f55f4a27640 1 -- 172.21.15.16:0/400974331 shutdown_connections 2024-09-19T07:55:28.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.930+0000 7f55f4a27640 1 -- 172.21.15.16:0/400974331 wait complete. 2024-09-19T07:55:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.931+0000 7f55f4a27640 1 Processor -- start 2024-09-19T07:55:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.931+0000 7f55f4a27640 1 -- start start 2024-09-19T07:55:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55f4a27640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00cf910 0x7f55f00ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55f4a27640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55f015bee0 0x7f55f015f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55f4a27640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f015c2c0 0x7f55f0071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55f4a27640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55f006e040 con 0x7f55f00cf910 2024-09-19T07:55:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55f4a27640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f55f006dec0 con 0x7f55f015c2c0 2024-09-19T07:55:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55f4a27640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f55f006e1c0 con 0x7f55f015bee0 2024-09-19T07:55:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55ee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00cf910 0x7f55f00ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55ee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00cf910 0x7f55f00ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41290/0 (socket says 172.21.15.16:41290) 2024-09-19T07:55:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55ee59c640 1 -- 172.21.15.16:0/548755861 learned_addr learned my addr 172.21.15.16:0/548755861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55ee59c640 1 -- 172.21.15.16:0/548755861 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55f015bee0 msgr2=0x7f55f015f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55edd9b640 1 --2- 172.21.15.16:0/548755861 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55f015bee0 0x7f55f015f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55eed9d640 1 --2- 172.21.15.16:0/548755861 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f015c2c0 0x7f55f0071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55ee59c640 1 --2- 172.21.15.16:0/548755861 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55f015bee0 0x7f55f015f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55ee59c640 1 -- 172.21.15.16:0/548755861 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f015c2c0 msgr2=0x7f55f0071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55ee59c640 1 --2- 172.21.15.16:0/548755861 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f015c2c0 0x7f55f0071ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55ee59c640 1 -- 172.21.15.16:0/548755861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f0072640 con 0x7f55f00cf910 2024-09-19T07:55:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55edd9b640 1 --2- 172.21.15.16:0/548755861 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55f015bee0 0x7f55f015f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55eed9d640 1 --2- 172.21.15.16:0/548755861 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f015c2c0 0x7f55f0071ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55ee59c640 1 --2- 172.21.15.16:0/548755861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00cf910 0x7f55f00ad600 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f55e000d100 tx=0x7f55e000d5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.932+0000 7f55e6ffd640 1 -- 172.21.15.16:0/548755861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e001a070 con 0x7f55f00cf910 2024-09-19T07:55:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.933+0000 7f55e6ffd640 1 -- 172.21.15.16:0/548755861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e0003b90 con 0x7f55f00cf910 2024-09-19T07:55:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.933+0000 7f55e6ffd640 1 -- 172.21.15.16:0/548755861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0004030 con 0x7f55f00cf910 2024-09-19T07:55:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.933+0000 7f55f4a27640 1 -- 172.21.15.16:0/548755861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f0072920 con 0x7f55f00cf910 2024-09-19T07:55:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.933+0000 7f55f4a27640 1 -- 172.21.15.16:0/548755861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f0072ee0 con 0x7f55f00cf910 2024-09-19T07:55:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.934+0000 7f55f4a27640 1 -- 172.21.15.16:0/548755861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f00cf200 con 0x7f55f00cf910 2024-09-19T07:55:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.934+0000 7f55e6ffd640 1 -- 172.21.15.16:0/548755861 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f55e001d070 con 0x7f55f00cf910 2024-09-19T07:55:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.935+0000 7f55e6ffd640 1 --2- 172.21.15.16:0/548755861 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f55d006c2b0 0x7f55d006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.935+0000 7f55e6ffd640 1 -- 172.21.15.16:0/548755861 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f55e008b7b0 con 0x7f55f00cf910 2024-09-19T07:55:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.935+0000 7f55edd9b640 1 --2- 172.21.15.16:0/548755861 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f55d006c2b0 0x7f55d006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:28.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.935+0000 7f55edd9b640 1 --2- 172.21.15.16:0/548755861 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f55d006c2b0 0x7f55d006e770 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f55f0071e50 tx=0x7f55d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:28.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:28.939+0000 7f55e6ffd640 1 -- 172.21.15.16:0/548755861 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f55e0010070 con 0x7f55f00cf910 2024-09-19T07:55:29.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.149+0000 7f55f4a27640 1 -- 172.21.15.16:0/548755861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f55f00b3fc0 con 0x7f55f00cf910 2024-09-19T07:55:29.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.150+0000 7f55e6ffd640 1 -- 172.21.15.16:0/548755861 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=-1 MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it v83) ==== 316+0+0 (secure 0 0 0) 0x7f55e00597d0 con 0x7f55f00cf910 2024-09-19T07:55:29.150 INFO:teuthology.orchestra.run.smithi016.stderr:Error EPERM: MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it 2024-09-19T07:55:29.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:55:29.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs --yes-i-really-mean-it 2024-09-19T07:55:29.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.259+0000 7f8c36bbd640 1 Processor -- start 2024-09-19T07:55:29.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.259+0000 7f8c36bbd640 1 -- start start 2024-09-19T07:55:29.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.260+0000 7f8c36bbd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c3015e2a0 0x7f8c3006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:29.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.260+0000 7f8c36bbd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8c3006d960 0x7f8c300d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:29.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.260+0000 7f8c36bbd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c300d1880 con 0x7f8c3015e680 2024-09-19T07:55:29.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.260+0000 7f8c36bbd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f8c300d1b60 con 0x7f8c3006d960 2024-09-19T07:55:29.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.260+0000 7f8c36bbd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8c300cf2d0 con 0x7f8c3015e2a0 2024-09-19T07:55:29.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.260+0000 7f8c2ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8c3015e680 0x7f8c300cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50894/0 (socket says 172.21.15.16:50894) 2024-09-19T07:55:29.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.260+0000 7f8c2ffff640 1 -- 172.21.15.16:0/1506410546 learned_addr learned my addr 172.21.15.16:0/1506410546 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:29.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.260+0000 7f8c34959640 1 --2- 172.21.15.16:0/1506410546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c3015e2a0 0x7f8c3006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:29.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.260+0000 7f8c3515a640 1 --2- 172.21.15.16:0/1506410546 >> v2:172.21.15.146:3300/0 conn(0x7f8c3006d960 0x7f8c300d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:29.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.261+0000 7f8c2f7fe640 1 -- 172.21.15.16:0/1506410546 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 63831612 0 0) 0x7f8c300d1880 con 0x7f8c3015e680 2024-09-19T07:55:29.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.261+0000 7f8c2f7fe640 1 -- 172.21.15.16:0/1506410546 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c18003520 con 0x7f8c3015e680 2024-09-19T07:55:29.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.261+0000 7f8c2f7fe640 1 -- 172.21.15.16:0/1506410546 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2771543387 0 0) 0x7f8c18003520 con 0x7f8c3015e680 2024-09-19T07:55:29.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.261+0000 7f8c34959640 1 -- 172.21.15.16:0/1506410546 >> v2:172.21.15.146:3300/0 conn(0x7f8c3006d960 msgr2=0x7f8c300d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:29.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.261+0000 7f8c34959640 1 --2- 172.21.15.16:0/1506410546 >> v2:172.21.15.146:3300/0 conn(0x7f8c3006d960 0x7f8c300d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:29.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.261+0000 7f8c34959640 1 -- 172.21.15.16:0/1506410546 >> v1:172.21.15.16:6789/0 conn(0x7f8c3015e680 legacy=0x7f8c300cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:29.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.261+0000 7f8c34959640 1 -- 172.21.15.16:0/1506410546 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c300cf5b0 con 0x7f8c3015e2a0 2024-09-19T07:55:29.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.262+0000 7f8c34959640 1 --2- 172.21.15.16:0/1506410546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c3015e2a0 0x7f8c3006d420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8c200097a0 tx=0x7f8c2002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ade244ad8c8f6234 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:29.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.262+0000 7f8c2f7fe640 1 -- 172.21.15.16:0/1506410546 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20037070 con 0x7f8c3015e2a0 2024-09-19T07:55:29.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.262+0000 7f8c2f7fe640 1 -- 172.21.15.16:0/1506410546 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c2002aba0 con 0x7f8c3015e2a0 2024-09-19T07:55:29.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.262+0000 7f8c2f7fe640 1 -- 172.21.15.16:0/1506410546 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24001670 con 0x7f8c3015e2a0 2024-09-19T07:55:29.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.262+0000 7f8c36bbd640 1 -- 172.21.15.16:0/1506410546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c3015e2a0 msgr2=0x7f8c3006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:29.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.262+0000 7f8c36bbd640 1 --2- 172.21.15.16:0/1506410546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c3015e2a0 0x7f8c3006d420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8c200097a0 tx=0x7f8c2002a110 comp rx=0 tx=0).stop 2024-09-19T07:55:29.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.263+0000 7f8c36bbd640 1 -- 172.21.15.16:0/1506410546 shutdown_connections 2024-09-19T07:55:29.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.263+0000 7f8c36bbd640 1 --2- 172.21.15.16:0/1506410546 >> v2:172.21.15.146:3300/0 conn(0x7f8c3006d960 0x7f8c300d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:29.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.263+0000 7f8c36bbd640 1 --2- 172.21.15.16:0/1506410546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c3015e2a0 0x7f8c3006d420 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:29.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.263+0000 7f8c36bbd640 1 -- 172.21.15.16:0/1506410546 >> 172.21.15.16:0/1506410546 conn(0x7f8c3009d760 msgr2=0x7f8c3008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:29.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.263+0000 7f8c36bbd640 1 -- 172.21.15.16:0/1506410546 shutdown_connections 2024-09-19T07:55:29.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.263+0000 7f8c36bbd640 1 -- 172.21.15.16:0/1506410546 wait complete. 2024-09-19T07:55:29.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.264+0000 7f8c36bbd640 1 Processor -- start 2024-09-19T07:55:29.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.264+0000 7f8c36bbd640 1 -- start start 2024-09-19T07:55:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.265+0000 7f8c36bbd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c3006d960 0x7f8c30155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.265+0000 7f8c36bbd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c3015e2a0 0x7f8c300b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.265+0000 7f8c36bbd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c3015e680 0x7f8c301599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.265+0000 7f8c36bbd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c300d0360 con 0x7f8c3015e680 2024-09-19T07:55:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.265+0000 7f8c36bbd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8c300d01e0 con 0x7f8c3006d960 2024-09-19T07:55:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.265+0000 7f8c36bbd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8c300d48b0 con 0x7f8c3015e2a0 2024-09-19T07:55:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.265+0000 7f8c3515a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c3015e680 0x7f8c301599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.265+0000 7f8c3515a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c3015e680 0x7f8c301599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41306/0 (socket says 172.21.15.16:41306) 2024-09-19T07:55:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.265+0000 7f8c3515a640 1 -- 172.21.15.16:0/3129131241 learned_addr learned my addr 172.21.15.16:0/3129131241 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.265+0000 7f8c2ffff640 1 --2- 172.21.15.16:0/3129131241 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c3015e2a0 0x7f8c300b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.265+0000 7f8c34959640 1 --2- 172.21.15.16:0/3129131241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c3006d960 0x7f8c30155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:29.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.265+0000 7f8c3515a640 1 -- 172.21.15.16:0/3129131241 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c3015e2a0 msgr2=0x7f8c300b76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.265+0000 7f8c3515a640 1 --2- 172.21.15.16:0/3129131241 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c3015e2a0 0x7f8c300b76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.265+0000 7f8c3515a640 1 -- 172.21.15.16:0/3129131241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c3006d960 msgr2=0x7f8c30155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.265+0000 7f8c3515a640 1 --2- 172.21.15.16:0/3129131241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c3006d960 0x7f8c30155ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.265+0000 7f8c3515a640 1 -- 172.21.15.16:0/3129131241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c30158c00 con 0x7f8c3015e680 2024-09-19T07:55:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.265+0000 7f8c3515a640 1 --2- 172.21.15.16:0/3129131241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c3015e680 0x7f8c301599c0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800efc0 tx=0x7f8c2800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.265+0000 7f8c2d7fa640 1 -- 172.21.15.16:0/3129131241 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28018020 con 0x7f8c3015e680 2024-09-19T07:55:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.266+0000 7f8c2d7fa640 1 -- 172.21.15.16:0/3129131241 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c28014480 con 0x7f8c3015e680 2024-09-19T07:55:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.266+0000 7f8c36bbd640 1 -- 172.21.15.16:0/3129131241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c30158ee0 con 0x7f8c3015e680 2024-09-19T07:55:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.266+0000 7f8c2d7fa640 1 -- 172.21.15.16:0/3129131241 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28014870 con 0x7f8c3015e680 2024-09-19T07:55:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.266+0000 7f8c36bbd640 1 -- 172.21.15.16:0/3129131241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c301594a0 con 0x7f8c3015e680 2024-09-19T07:55:29.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.267+0000 7f8c36bbd640 1 -- 172.21.15.16:0/3129131241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c301596a0 con 0x7f8c3015e680 2024-09-19T07:55:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.267+0000 7f8c2d7fa640 1 -- 172.21.15.16:0/3129131241 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8c28005d10 con 0x7f8c3015e680 2024-09-19T07:55:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.268+0000 7f8c2d7fa640 1 --2- 172.21.15.16:0/3129131241 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8c1c06c2b0 0x7f8c1c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.268+0000 7f8c2d7fa640 1 -- 172.21.15.16:0/3129131241 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f8c2808c150 con 0x7f8c3015e680 2024-09-19T07:55:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.268+0000 7f8c34959640 1 --2- 172.21.15.16:0/3129131241 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8c1c06c2b0 0x7f8c1c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:29.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.269+0000 7f8c34959640 1 --2- 172.21.15.16:0/3129131241 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8c1c06c2b0 0x7f8c1c06e770 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8c20034000 tx=0x7f8c2002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:29.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.272+0000 7f8c2d7fa640 1 -- 172.21.15.16:0/3129131241 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8c301596a0 con 0x7f8c3015e680 2024-09-19T07:55:29.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.485+0000 7f8c36bbd640 1 -- 172.21.15.16:0/3129131241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f8c300d4c70 con 0x7f8c3015e680 2024-09-19T07:55:29.486 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-19T07:55:29.486+0000 7f6950fea640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:55:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.487+0000 7f8c2d7fa640 1 -- 172.21.15.16:0/3129131241 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) ==== 191+0+0 (secure 0 0 0) 0x7f8c28022070 con 0x7f8c3015e680 2024-09-19T07:55:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T07:55:29.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.490+0000 7f8c36bbd640 1 -- 172.21.15.16:0/3129131241 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8c1c06c2b0 msgr2=0x7f8c1c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:29.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.490+0000 7f8c36bbd640 1 --2- 172.21.15.16:0/3129131241 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8c1c06c2b0 0x7f8c1c06e770 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8c20034000 tx=0x7f8c2002e000 comp rx=0 tx=0).stop 2024-09-19T07:55:29.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.490+0000 7f8c36bbd640 1 -- 172.21.15.16:0/3129131241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c3015e680 msgr2=0x7f8c301599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:29.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.490+0000 7f8c36bbd640 1 --2- 172.21.15.16:0/3129131241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c3015e680 0x7f8c301599c0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800efc0 tx=0x7f8c2800c490 comp rx=0 tx=0).stop 2024-09-19T07:55:29.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.490+0000 7f8c36bbd640 1 -- 172.21.15.16:0/3129131241 shutdown_connections 2024-09-19T07:55:29.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.490+0000 7f8c36bbd640 1 --2- 172.21.15.16:0/3129131241 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f8c1c06c2b0 0x7f8c1c06e770 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:29.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.490+0000 7f8c36bbd640 1 --2- 172.21.15.16:0/3129131241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c3015e680 0x7f8c301599c0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:29.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.490+0000 7f8c36bbd640 1 --2- 172.21.15.16:0/3129131241 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c3015e2a0 0x7f8c300b76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:29.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.490+0000 7f8c36bbd640 1 --2- 172.21.15.16:0/3129131241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c3006d960 0x7f8c30155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:29.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.491+0000 7f8c36bbd640 1 -- 172.21.15.16:0/3129131241 >> 172.21.15.16:0/3129131241 conn(0x7f8c3009d760 msgr2=0x7f8c30161f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:29.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.491+0000 7f8c36bbd640 1 -- 172.21.15.16:0/3129131241 shutdown_connections 2024-09-19T07:55:29.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.491+0000 7f8c36bbd640 1 -- 172.21.15.16:0/3129131241 wait complete. 2024-09-19T07:55:29.506 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T07:55:29.534 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-19T07:55:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.611+0000 7f21d9af1640 1 Processor -- start 2024-09-19T07:55:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.611+0000 7f21d9af1640 1 -- start start 2024-09-19T07:55:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.612+0000 7f21d9af1640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f21d415b300 0x7f21d40cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.612+0000 7f21d9af1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21d40ccbd0 0x7f21d40cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.612+0000 7f21d9af1640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21d40cd910 con 0x7f21d415b6e0 2024-09-19T07:55:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.612+0000 7f21d9af1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f21d40cd070 con 0x7f21d40ccbd0 2024-09-19T07:55:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.612+0000 7f21d9af1640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f21d40cd350 con 0x7f21d415b300 2024-09-19T07:55:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.612+0000 7f21d2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21d40ccbd0 0x7f21d40cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.612+0000 7f21d2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21d40ccbd0 0x7f21d40cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41330/0 (socket says 172.21.15.16:41330) 2024-09-19T07:55:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.612+0000 7f21d2ffd640 1 -- 172.21.15.16:0/2487050047 learned_addr learned my addr 172.21.15.16:0/2487050047 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.613+0000 7f21d37fe640 1 --2- 172.21.15.16:0/2487050047 >> v2:172.21.15.146:3300/0 conn(0x7f21d415b300 0x7f21d40cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.613+0000 7f21d2ffd640 1 -- 172.21.15.16:0/2487050047 >> v2:172.21.15.146:3300/0 conn(0x7f21d415b300 msgr2=0x7f21d40cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.613+0000 7f21d2ffd640 1 --2- 172.21.15.16:0/2487050047 >> v2:172.21.15.146:3300/0 conn(0x7f21d415b300 0x7f21d40cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.613+0000 7f21d2ffd640 1 -- 172.21.15.16:0/2487050047 >> v1:172.21.15.146:6789/0 conn(0x7f21d415b6e0 legacy=0x7f21d40cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.613+0000 7f21d2ffd640 1 -- 172.21.15.16:0/2487050047 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d40cd630 con 0x7f21d40ccbd0 2024-09-19T07:55:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.613+0000 7f21d2ffd640 1 --2- 172.21.15.16:0/2487050047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d40ccbd0 0x7f21d40cdb50 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f21d40688d0 tx=0x7f21c802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ee1b4fc2d03c83 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.614+0000 7f21d27fc640 1 -- 172.21.15.16:0/2487050047 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c8037070 con 0x7f21d40ccbd0 2024-09-19T07:55:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.614+0000 7f21d27fc640 1 -- 172.21.15.16:0/2487050047 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21c8032440 con 0x7f21d40ccbd0 2024-09-19T07:55:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.614+0000 7f21d27fc640 1 -- 172.21.15.16:0/2487050047 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c8032810 con 0x7f21d40ccbd0 2024-09-19T07:55:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.614+0000 7f21d9af1640 1 -- 172.21.15.16:0/2487050047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d40ccbd0 msgr2=0x7f21d40cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.614+0000 7f21d9af1640 1 --2- 172.21.15.16:0/2487050047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d40ccbd0 0x7f21d40cdb50 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f21d40688d0 tx=0x7f21c802a230 comp rx=0 tx=0).stop 2024-09-19T07:55:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.614+0000 7f21d9af1640 1 -- 172.21.15.16:0/2487050047 shutdown_connections 2024-09-19T07:55:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.614+0000 7f21d9af1640 1 --2- 172.21.15.16:0/2487050047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d40ccbd0 0x7f21d40cdb50 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.614+0000 7f21d9af1640 1 --2- 172.21.15.16:0/2487050047 >> v2:172.21.15.146:3300/0 conn(0x7f21d415b300 0x7f21d40cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.614+0000 7f21d9af1640 1 -- 172.21.15.16:0/2487050047 >> 172.21.15.16:0/2487050047 conn(0x7f21d40b2d10 msgr2=0x7f21d40b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.614+0000 7f21d9af1640 1 -- 172.21.15.16:0/2487050047 shutdown_connections 2024-09-19T07:55:29.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.614+0000 7f21d9af1640 1 -- 172.21.15.16:0/2487050047 wait complete. 2024-09-19T07:55:29.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.615+0000 7f21d9af1640 1 Processor -- start 2024-09-19T07:55:29.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.616+0000 7f21d9af1640 1 -- start start 2024-09-19T07:55:29.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.616+0000 7f21d9af1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21d40ccbd0 0x7f21d40b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:29.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.616+0000 7f21d9af1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d415b300 0x7f21d415cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:29.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.616+0000 7f21d9af1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21d415b6e0 0x7f21d414f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:29.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.616+0000 7f21d9af1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21d40b3b70 con 0x7f21d415b300 2024-09-19T07:55:29.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.616+0000 7f21d9af1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f21d40b39f0 con 0x7f21d40ccbd0 2024-09-19T07:55:29.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.616+0000 7f21d9af1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f21d40b2060 con 0x7f21d415b6e0 2024-09-19T07:55:29.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.617+0000 7f21d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d415b300 0x7f21d415cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:29.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.617+0000 7f21d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d415b300 0x7f21d415cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41346/0 (socket says 172.21.15.16:41346) 2024-09-19T07:55:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.617+0000 7f21d2ffd640 1 -- 172.21.15.16:0/3542042780 learned_addr learned my addr 172.21.15.16:0/3542042780 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.617+0000 7f21d37fe640 1 --2- 172.21.15.16:0/3542042780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21d40ccbd0 0x7f21d40b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.617+0000 7f21d3fff640 1 --2- 172.21.15.16:0/3542042780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21d415b6e0 0x7f21d414f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.617+0000 7f21d2ffd640 1 -- 172.21.15.16:0/3542042780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21d415b6e0 msgr2=0x7f21d414f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.617+0000 7f21d2ffd640 1 --2- 172.21.15.16:0/3542042780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21d415b6e0 0x7f21d414f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.617+0000 7f21d2ffd640 1 -- 172.21.15.16:0/3542042780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21d40ccbd0 msgr2=0x7f21d40b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.617+0000 7f21d2ffd640 1 --2- 172.21.15.16:0/3542042780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21d40ccbd0 0x7f21d40b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.617+0000 7f21d2ffd640 1 -- 172.21.15.16:0/3542042780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d415d2c0 con 0x7f21d415b300 2024-09-19T07:55:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.617+0000 7f21d3fff640 1 --2- 172.21.15.16:0/3542042780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21d415b6e0 0x7f21d414f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.617+0000 7f21d37fe640 1 --2- 172.21.15.16:0/3542042780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21d40ccbd0 0x7f21d40b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.617+0000 7f21d2ffd640 1 --2- 172.21.15.16:0/3542042780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d415b300 0x7f21d415cd80 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f21c802a270 tx=0x7f21c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.617+0000 7f21bbfff640 1 -- 172.21.15.16:0/3542042780 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c8037040 con 0x7f21d415b300 2024-09-19T07:55:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.617+0000 7f21bbfff640 1 -- 172.21.15.16:0/3542042780 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21c8042070 con 0x7f21d415b300 2024-09-19T07:55:29.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.617+0000 7f21d9af1640 1 -- 172.21.15.16:0/3542042780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d40b5780 con 0x7f21d415b300 2024-09-19T07:55:29.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.617+0000 7f21bbfff640 1 -- 172.21.15.16:0/3542042780 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c8030bb0 con 0x7f21d415b300 2024-09-19T07:55:29.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.617+0000 7f21d9af1640 1 -- 172.21.15.16:0/3542042780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d40b5d40 con 0x7f21d415b300 2024-09-19T07:55:29.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.618+0000 7f21d9af1640 1 -- 172.21.15.16:0/3542042780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21d40cc690 con 0x7f21d415b300 2024-09-19T07:55:29.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.619+0000 7f21bbfff640 1 -- 172.21.15.16:0/3542042780 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f21c8047460 con 0x7f21d415b300 2024-09-19T07:55:29.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.619+0000 7f21bbfff640 1 --2- 172.21.15.16:0/3542042780 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f21c006c2e0 0x7f21c006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:29.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.619+0000 7f21bbfff640 1 -- 172.21.15.16:0/3542042780 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6187+0+0 (secure 0 0 0) 0x7f21c80ac260 con 0x7f21d415b300 2024-09-19T07:55:29.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.619+0000 7f21d37fe640 1 --2- 172.21.15.16:0/3542042780 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f21c006c2e0 0x7f21c006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:29.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.620+0000 7f21d37fe640 1 --2- 172.21.15.16:0/3542042780 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f21c006c2e0 0x7f21c006e7a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f21c4007f50 tx=0x7f21c40074a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:29.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.623+0000 7f21bbfff640 1 -- 172.21.15.16:0/3542042780 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f21d415d2c0 con 0x7f21d415b300 2024-09-19T07:55:29.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:29.833+0000 7f21d9af1640 1 -- 172.21.15.16:0/3542042780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f21d40b1980 con 0x7f21d415b300 2024-09-19T07:55:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.488+0000 7f21bbfff640 1 -- 172.21.15.16:0/3542042780 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) ==== 104+0+0 (secure 0 0 0) 0x7f21c8045360 con 0x7f21d415b300 2024-09-19T07:55:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.492+0000 7f21d9af1640 1 -- 172.21.15.16:0/3542042780 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f21c006c2e0 msgr2=0x7f21c006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.492+0000 7f21d9af1640 1 --2- 172.21.15.16:0/3542042780 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f21c006c2e0 0x7f21c006e7a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f21c4007f50 tx=0x7f21c40074a0 comp rx=0 tx=0).stop 2024-09-19T07:55:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.492+0000 7f21d9af1640 1 -- 172.21.15.16:0/3542042780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d415b300 msgr2=0x7f21d415cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.492+0000 7f21d9af1640 1 --2- 172.21.15.16:0/3542042780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d415b300 0x7f21d415cd80 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f21c802a270 tx=0x7f21c80046c0 comp rx=0 tx=0).stop 2024-09-19T07:55:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.492+0000 7f21d9af1640 1 -- 172.21.15.16:0/3542042780 shutdown_connections 2024-09-19T07:55:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.492+0000 7f21d9af1640 1 --2- 172.21.15.16:0/3542042780 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f21c006c2e0 0x7f21c006e7a0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.492+0000 7f21d9af1640 1 --2- 172.21.15.16:0/3542042780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21d415b6e0 0x7f21d414f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.492+0000 7f21d9af1640 1 --2- 172.21.15.16:0/3542042780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d415b300 0x7f21d415cd80 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.492+0000 7f21d9af1640 1 --2- 172.21.15.16:0/3542042780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21d40ccbd0 0x7f21d40b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.492+0000 7f21d9af1640 1 -- 172.21.15.16:0/3542042780 >> 172.21.15.16:0/3542042780 conn(0x7f21d40b2d10 msgr2=0x7f21d406ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.492+0000 7f21d9af1640 1 -- 172.21.15.16:0/3542042780 shutdown_connections 2024-09-19T07:55:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.492+0000 7f21d9af1640 1 -- 172.21.15.16:0/3542042780 wait complete. 2024-09-19T07:55:30.507 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T07:55:30.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.663+0000 7f77695a6640 1 Processor -- start 2024-09-19T07:55:30.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.663+0000 7f77695a6640 1 -- start start 2024-09-19T07:55:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.664+0000 7f77695a6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77640b0a80 0x7f77640cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.664+0000 7f77695a6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f776406c990 con 0x7f77641619b0 2024-09-19T07:55:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.664+0000 7f77695a6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77640cec10 con 0x7f7764161d90 2024-09-19T07:55:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.664+0000 7f77695a6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f77640ce930 con 0x7f77640b0a80 2024-09-19T07:55:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.664+0000 7f7762ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f77641619b0 0x7f77640d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50908/0 (socket says 172.21.15.16:50908) 2024-09-19T07:55:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.664+0000 7f7762ffd640 1 -- 172.21.15.16:0/870021957 learned_addr learned my addr 172.21.15.16:0/870021957 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.664+0000 7f7761ffb640 1 -- 172.21.15.16:0/870021957 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1489381874 0 0) 0x7f776406c990 con 0x7f77641619b0 2024-09-19T07:55:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.664+0000 7f77627fc640 1 --2- 172.21.15.16:0/870021957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77640b0a80 0x7f77640cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.664+0000 7f7761ffb640 1 -- 172.21.15.16:0/870021957 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f774c003520 con 0x7f77641619b0 2024-09-19T07:55:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f7761ffb640 1 -- 172.21.15.16:0/870021957 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3092619996 0 0) 0x7f774c003520 con 0x7f77641619b0 2024-09-19T07:55:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f7761ffb640 1 -- 172.21.15.16:0/870021957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77640b0a80 msgr2=0x7f77640cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f7761ffb640 1 --2- 172.21.15.16:0/870021957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77640b0a80 0x7f77640cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f7761ffb640 1 -- 172.21.15.16:0/870021957 >> v1:172.21.15.146:6789/0 conn(0x7f7764161d90 legacy=0x7f77640cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f7761ffb640 1 -- 172.21.15.16:0/870021957 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77640b0540 con 0x7f77641619b0 2024-09-19T07:55:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f7761ffb640 1 -- 172.21.15.16:0/870021957 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f7754001670 con 0x7f77641619b0 2024-09-19T07:55:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f7761ffb640 1 -- 172.21.15.16:0/870021957 >> v1:172.21.15.16:6789/0 conn(0x7f77641619b0 legacy=0x7f77640d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f7761ffb640 1 --2- 172.21.15.16:0/870021957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77640b0a80 0x7f774c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f7761ffb640 1 --2- 172.21.15.16:0/870021957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f774c003f60 0x7f774c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f7761ffb640 1 --2- 172.21.15.16:0/870021957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f774c004d40 0x7f774c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f7761ffb640 1 -- 172.21.15.16:0/870021957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77640ce930 con 0x7f774c004d40 2024-09-19T07:55:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f7761ffb640 1 -- 172.21.15.16:0/870021957 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f774c00baa0 con 0x7f774c003f60 2024-09-19T07:55:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f7761ffb640 1 -- 172.21.15.16:0/870021957 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f774c00bca0 con 0x7f77640b0a80 2024-09-19T07:55:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f77637fe640 1 --2- 172.21.15.16:0/870021957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f774c004d40 0x7f774c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f7762ffd640 1 --2- 172.21.15.16:0/870021957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77640b0a80 0x7f774c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f77627fc640 1 --2- 172.21.15.16:0/870021957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f774c003f60 0x7f774c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f77637fe640 1 -- 172.21.15.16:0/870021957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77640b0a80 msgr2=0x7f774c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f77637fe640 1 --2- 172.21.15.16:0/870021957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77640b0a80 0x7f774c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f77637fe640 1 -- 172.21.15.16:0/870021957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f774c003f60 msgr2=0x7f774c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f77637fe640 1 --2- 172.21.15.16:0/870021957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f774c003f60 0x7f774c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.665+0000 7f77637fe640 1 -- 172.21.15.16:0/870021957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774c003520 con 0x7f774c004d40 2024-09-19T07:55:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.666+0000 7f77637fe640 1 --2- 172.21.15.16:0/870021957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f774c004d40 0x7f774c0051c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f775c001e50 tx=0x7f774c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d763f510e94ff757 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.666+0000 7f7761ffb640 1 -- 172.21.15.16:0/870021957 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c001a70 con 0x7f774c004d40 2024-09-19T07:55:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.666+0000 7f7761ffb640 1 -- 172.21.15.16:0/870021957 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f775c003970 con 0x7f774c004d40 2024-09-19T07:55:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.666+0000 7f7761ffb640 1 -- 172.21.15.16:0/870021957 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c003da0 con 0x7f774c004d40 2024-09-19T07:55:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.667+0000 7f77695a6640 1 -- 172.21.15.16:0/870021957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f774c004d40 msgr2=0x7f774c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.667+0000 7f77695a6640 1 --2- 172.21.15.16:0/870021957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f774c004d40 0x7f774c0051c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f775c001e50 tx=0x7f774c005830 comp rx=0 tx=0).stop 2024-09-19T07:55:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.667+0000 7f77637fe640 1 -- 172.21.15.16:0/870021957 reap_dead start 2024-09-19T07:55:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.667+0000 7f77695a6640 1 -- 172.21.15.16:0/870021957 shutdown_connections 2024-09-19T07:55:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.667+0000 7f77695a6640 1 -- 172.21.15.16:0/870021957 >> 172.21.15.16:0/870021957 conn(0x7f776409a9f0 msgr2=0x7f77640b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.667+0000 7f77695a6640 1 -- 172.21.15.16:0/870021957 shutdown_connections 2024-09-19T07:55:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.667+0000 7f77695a6640 1 -- 172.21.15.16:0/870021957 wait complete. 2024-09-19T07:55:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.668+0000 7f77695a6640 1 Processor -- start 2024-09-19T07:55:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.668+0000 7f77695a6640 1 -- start start 2024-09-19T07:55:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.669+0000 7f77695a6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7764149590 0x7f77640b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.669+0000 7f77695a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7764149970 0x7f77640b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.669+0000 7f77695a6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77641526a0 0x7f77640b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.669+0000 7f77695a6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77640cd330 con 0x7f7764149970 2024-09-19T07:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.669+0000 7f77695a6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f776406c620 con 0x7f7764149590 2024-09-19T07:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.669+0000 7f77695a6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f77640cd4b0 con 0x7f77641526a0 2024-09-19T07:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.669+0000 7f77627fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7764149970 0x7f77640b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.669+0000 7f77627fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7764149970 0x7f77640b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41364/0 (socket says 172.21.15.16:41364) 2024-09-19T07:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.669+0000 7f77627fc640 1 -- 172.21.15.16:0/3917181552 learned_addr learned my addr 172.21.15.16:0/3917181552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.669+0000 7f7762ffd640 1 --2- 172.21.15.16:0/3917181552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7764149590 0x7f77640b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.669+0000 7f77637fe640 1 --2- 172.21.15.16:0/3917181552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77641526a0 0x7f77640b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.669+0000 7f77627fc640 1 -- 172.21.15.16:0/3917181552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77641526a0 msgr2=0x7f77640b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.669+0000 7f77627fc640 1 --2- 172.21.15.16:0/3917181552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77641526a0 0x7f77640b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.669+0000 7f77627fc640 1 -- 172.21.15.16:0/3917181552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7764149590 msgr2=0x7f77640b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.669+0000 7f77627fc640 1 --2- 172.21.15.16:0/3917181552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7764149590 0x7f77640b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.669+0000 7f77627fc640 1 -- 172.21.15.16:0/3917181552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77640b29f0 con 0x7f7764149970 2024-09-19T07:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.670+0000 7f77627fc640 1 --2- 172.21.15.16:0/3917181552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7764149970 0x7f77640b20f0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f7758011960 tx=0x7f7758011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.670+0000 7f774b7fe640 1 -- 172.21.15.16:0/3917181552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7758019070 con 0x7f7764149970 2024-09-19T07:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.670+0000 7f774b7fe640 1 -- 172.21.15.16:0/3917181552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7758011e90 con 0x7f7764149970 2024-09-19T07:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.670+0000 7f774b7fe640 1 -- 172.21.15.16:0/3917181552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7758004440 con 0x7f7764149970 2024-09-19T07:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.670+0000 7f77695a6640 1 -- 172.21.15.16:0/3917181552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77640b2cd0 con 0x7f7764149970 2024-09-19T07:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.670+0000 7f77695a6640 1 -- 172.21.15.16:0/3917181552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77640b47c0 con 0x7f7764149970 2024-09-19T07:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.671+0000 7f77695a6640 1 -- 172.21.15.16:0/3917181552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77640d15c0 con 0x7f7764149970 2024-09-19T07:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.671+0000 7f774b7fe640 1 -- 172.21.15.16:0/3917181552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f775801c070 con 0x7f7764149970 2024-09-19T07:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.672+0000 7f774b7fe640 1 --2- 172.21.15.16:0/3917181552 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f775006c2b0 0x7f775006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.672+0000 7f774b7fe640 1 -- 172.21.15.16:0/3917181552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6187+0+0 (secure 0 0 0) 0x7f775808b990 con 0x7f7764149970 2024-09-19T07:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.672+0000 7f7762ffd640 1 --2- 172.21.15.16:0/3917181552 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f775006c2b0 0x7f775006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.673+0000 7f7762ffd640 1 --2- 172.21.15.16:0/3917181552 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f775006c2b0 0x7f775006e770 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7754007db0 tx=0x7f77540078f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.676+0000 7f774b7fe640 1 -- 172.21.15.16:0/3917181552 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7758059830 con 0x7f7764149970 2024-09-19T07:55:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:30.852+0000 7f77695a6640 1 -- 172.21.15.16:0/3917181552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f776415f580 con 0x7f7764149970 2024-09-19T07:55:31.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.493+0000 7f774b7fe640 1 -- 172.21.15.16:0/3917181552 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v142) ==== 181+0+0 (secure 0 0 0) 0x7f775805db30 con 0x7f7764149970 2024-09-19T07:55:31.494 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-19T07:55:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.497+0000 7f77695a6640 1 -- 172.21.15.16:0/3917181552 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f775006c2b0 msgr2=0x7f775006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.497+0000 7f77695a6640 1 --2- 172.21.15.16:0/3917181552 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f775006c2b0 0x7f775006e770 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7754007db0 tx=0x7f77540078f0 comp rx=0 tx=0).stop 2024-09-19T07:55:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.497+0000 7f77695a6640 1 -- 172.21.15.16:0/3917181552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7764149970 msgr2=0x7f77640b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.497+0000 7f77695a6640 1 --2- 172.21.15.16:0/3917181552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7764149970 0x7f77640b20f0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f7758011960 tx=0x7f7758011d60 comp rx=0 tx=0).stop 2024-09-19T07:55:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.497+0000 7f77695a6640 1 -- 172.21.15.16:0/3917181552 shutdown_connections 2024-09-19T07:55:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.497+0000 7f77695a6640 1 --2- 172.21.15.16:0/3917181552 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f775006c2b0 0x7f775006e770 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.497+0000 7f77695a6640 1 --2- 172.21.15.16:0/3917181552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f77641526a0 0x7f77640b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.497+0000 7f77695a6640 1 --2- 172.21.15.16:0/3917181552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7764149970 0x7f77640b20f0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.497+0000 7f77695a6640 1 --2- 172.21.15.16:0/3917181552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7764149590 0x7f77640b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.497+0000 7f77695a6640 1 -- 172.21.15.16:0/3917181552 >> 172.21.15.16:0/3917181552 conn(0x7f776409a9f0 msgr2=0x7f776406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.497+0000 7f77695a6640 1 -- 172.21.15.16:0/3917181552 shutdown_connections 2024-09-19T07:55:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.498+0000 7f77695a6640 1 -- 172.21.15.16:0/3917181552 wait complete. 2024-09-19T07:55:31.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T07:55:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.625+0000 7fdb8adbd640 1 Processor -- start 2024-09-19T07:55:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.625+0000 7fdb8adbd640 1 -- start start 2024-09-19T07:55:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.625+0000 7fdb8adbd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdb840c8d20 0x7fdb8406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.625+0000 7fdb8adbd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb84144100 0x7fdb840d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.625+0000 7fdb8adbd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb840c7140 con 0x7fdb8406d260 2024-09-19T07:55:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.625+0000 7fdb8adbd640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fdb8406d6d0 con 0x7fdb840c8d20 2024-09-19T07:55:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.625+0000 7fdb8adbd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdb84141960 con 0x7fdb84144100 2024-09-19T07:55:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.626+0000 7fdb88b59640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdb840c8d20 0x7fdb8406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.626+0000 7fdb88b59640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdb840c8d20 0x7fdb8406cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41366/0 (socket says 172.21.15.16:41366) 2024-09-19T07:55:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.626+0000 7fdb83fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdb8406d260 0x7fdb841439f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50922/0 (socket says 172.21.15.16:50922) 2024-09-19T07:55:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.626+0000 7fdb88b59640 1 -- 172.21.15.16:0/1652403501 learned_addr learned my addr 172.21.15.16:0/1652403501 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.626+0000 7fdb8935a640 1 --2- 172.21.15.16:0/1652403501 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb84144100 0x7fdb840d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:31.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.626+0000 7fdb88b59640 1 -- 172.21.15.16:0/1652403501 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb84144100 msgr2=0x7fdb840d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:31.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.626+0000 7fdb88b59640 1 --2- 172.21.15.16:0/1652403501 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb84144100 0x7fdb840d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:31.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.626+0000 7fdb88b59640 1 -- 172.21.15.16:0/1652403501 >> v1:172.21.15.16:6789/0 conn(0x7fdb8406d260 legacy=0x7fdb841439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:31.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.626+0000 7fdb88b59640 1 -- 172.21.15.16:0/1652403501 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb840c9100 con 0x7fdb840c8d20 2024-09-19T07:55:31.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.626+0000 7fdb837fe640 1 -- 172.21.15.16:0/1652403501 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3253493544 0 0) 0x7fdb74001490 con 0x7fdb8406d260 2024-09-19T07:55:31.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.627+0000 7fdb88b59640 1 --2- 172.21.15.16:0/1652403501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb840c8d20 0x7fdb8406cd20 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009870 tx=0x7fdb7c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98e1e7fdcb4231e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:31.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.627+0000 7fdb837fe640 1 -- 172.21.15.16:0/1652403501 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c037070 con 0x7fdb840c8d20 2024-09-19T07:55:31.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.627+0000 7fdb837fe640 1 -- 172.21.15.16:0/1652403501 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb7c032440 con 0x7fdb840c8d20 2024-09-19T07:55:31.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.627+0000 7fdb837fe640 1 -- 172.21.15.16:0/1652403501 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c032810 con 0x7fdb840c8d20 2024-09-19T07:55:31.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.627+0000 7fdb8adbd640 1 -- 172.21.15.16:0/1652403501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb840c8d20 msgr2=0x7fdb8406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:31.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.627+0000 7fdb8adbd640 1 --2- 172.21.15.16:0/1652403501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb840c8d20 0x7fdb8406cd20 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009870 tx=0x7fdb7c02a130 comp rx=0 tx=0).stop 2024-09-19T07:55:31.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.627+0000 7fdb8adbd640 1 -- 172.21.15.16:0/1652403501 shutdown_connections 2024-09-19T07:55:31.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.627+0000 7fdb8adbd640 1 --2- 172.21.15.16:0/1652403501 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb84144100 0x7fdb840d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:31.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.627+0000 7fdb8adbd640 1 --2- 172.21.15.16:0/1652403501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb840c8d20 0x7fdb8406cd20 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:31.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.627+0000 7fdb8adbd640 1 -- 172.21.15.16:0/1652403501 >> 172.21.15.16:0/1652403501 conn(0x7fdb840b3ed0 msgr2=0x7fdb840b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:31.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.627+0000 7fdb8adbd640 1 -- 172.21.15.16:0/1652403501 shutdown_connections 2024-09-19T07:55:31.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.627+0000 7fdb8adbd640 1 -- 172.21.15.16:0/1652403501 wait complete. 2024-09-19T07:55:31.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.629+0000 7fdb8adbd640 1 Processor -- start 2024-09-19T07:55:31.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.629+0000 7fdb8adbd640 1 -- start start 2024-09-19T07:55:31.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb8adbd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb8406d260 0x7fdb841e9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:31.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb8adbd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb840c8d20 0x7fdb84140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:31.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb8adbd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb84144100 0x7fdb840b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:31.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb8adbd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb84144510 con 0x7fdb84144100 2024-09-19T07:55:31.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb8adbd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdb84142200 con 0x7fdb840c8d20 2024-09-19T07:55:31.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb8adbd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdb84141bb0 con 0x7fdb8406d260 2024-09-19T07:55:31.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb8935a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb84144100 0x7fdb840b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:31.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb8935a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb84144100 0x7fdb840b7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41368/0 (socket says 172.21.15.16:41368) 2024-09-19T07:55:31.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb8935a640 1 -- 172.21.15.16:0/2701739023 learned_addr learned my addr 172.21.15.16:0/2701739023 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb88b59640 1 --2- 172.21.15.16:0/2701739023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb8406d260 0x7fdb841e9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb83fff640 1 --2- 172.21.15.16:0/2701739023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb840c8d20 0x7fdb84140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb8935a640 1 -- 172.21.15.16:0/2701739023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb8406d260 msgr2=0x7fdb841e9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb8935a640 1 --2- 172.21.15.16:0/2701739023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb8406d260 0x7fdb841e9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb8935a640 1 -- 172.21.15.16:0/2701739023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb840c8d20 msgr2=0x7fdb84140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb8935a640 1 --2- 172.21.15.16:0/2701739023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb840c8d20 0x7fdb84140c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb8935a640 1 -- 172.21.15.16:0/2701739023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb841f4210 con 0x7fdb84144100 2024-09-19T07:55:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb83fff640 1 --2- 172.21.15.16:0/2701739023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb840c8d20 0x7fdb84140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb8935a640 1 --2- 172.21.15.16:0/2701739023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb84144100 0x7fdb840b7120 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00d800 tx=0x7fdb6c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb817fa640 1 -- 172.21.15.16:0/2701739023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c00b6c0 con 0x7fdb84144100 2024-09-19T07:55:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb817fa640 1 -- 172.21.15.16:0/2701739023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb6c019070 con 0x7fdb84144100 2024-09-19T07:55:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb8adbd640 1 -- 172.21.15.16:0/2701739023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb841f4f90 con 0x7fdb84144100 2024-09-19T07:55:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb817fa640 1 -- 172.21.15.16:0/2701739023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c00bac0 con 0x7fdb84144100 2024-09-19T07:55:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.630+0000 7fdb8adbd640 1 -- 172.21.15.16:0/2701739023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb841f4790 con 0x7fdb84144100 2024-09-19T07:55:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.631+0000 7fdb8adbd640 1 -- 172.21.15.16:0/2701739023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb8406cd20 con 0x7fdb84144100 2024-09-19T07:55:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.632+0000 7fdb817fa640 1 -- 172.21.15.16:0/2701739023 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdb6c00bc60 con 0x7fdb84144100 2024-09-19T07:55:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.632+0000 7fdb817fa640 1 --2- 172.21.15.16:0/2701739023 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdb6406c2e0 0x7fdb6406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.632+0000 7fdb817fa640 1 -- 172.21.15.16:0/2701739023 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5749+0+0 (secure 0 0 0) 0x7fdb6c08b740 con 0x7fdb84144100 2024-09-19T07:55:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.633+0000 7fdb88b59640 1 --2- 172.21.15.16:0/2701739023 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdb6406c2e0 0x7fdb6406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.633+0000 7fdb88b59640 1 --2- 172.21.15.16:0/2701739023 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdb6406c2e0 0x7fdb6406e7a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c034000 tx=0x7fdb7c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:31.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.637+0000 7fdb817fa640 1 -- 172.21.15.16:0/2701739023 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fdb6c059f10 con 0x7fdb84144100 2024-09-19T07:55:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:31.817+0000 7fdb8adbd640 1 -- 172.21.15.16:0/2701739023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdb840690c0 con 0x7fdb84144100 2024-09-19T07:55:32.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.495+0000 7fdb817fa640 1 -- 172.21.15.16:0/2701739023 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v143) ==== 169+0+0 (secure 0 0 0) 0x7fdb6c059810 con 0x7fdb84144100 2024-09-19T07:55:32.496 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-19T07:55:32.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.499+0000 7fdb8adbd640 1 -- 172.21.15.16:0/2701739023 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdb6406c2e0 msgr2=0x7fdb6406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:32.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.499+0000 7fdb8adbd640 1 --2- 172.21.15.16:0/2701739023 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdb6406c2e0 0x7fdb6406e7a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c034000 tx=0x7fdb7c02e000 comp rx=0 tx=0).stop 2024-09-19T07:55:32.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.499+0000 7fdb8adbd640 1 -- 172.21.15.16:0/2701739023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb84144100 msgr2=0x7fdb840b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:32.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.499+0000 7fdb8adbd640 1 --2- 172.21.15.16:0/2701739023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb84144100 0x7fdb840b7120 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00d800 tx=0x7fdb6c00dcc0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.500+0000 7fdb8adbd640 1 -- 172.21.15.16:0/2701739023 shutdown_connections 2024-09-19T07:55:32.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.500+0000 7fdb8adbd640 1 --2- 172.21.15.16:0/2701739023 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdb6406c2e0 0x7fdb6406e7a0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.500+0000 7fdb8adbd640 1 --2- 172.21.15.16:0/2701739023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb84144100 0x7fdb840b7120 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.500+0000 7fdb8adbd640 1 --2- 172.21.15.16:0/2701739023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb840c8d20 0x7fdb84140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.500+0000 7fdb8adbd640 1 --2- 172.21.15.16:0/2701739023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb8406d260 0x7fdb841e9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.500+0000 7fdb8adbd640 1 -- 172.21.15.16:0/2701739023 >> 172.21.15.16:0/2701739023 conn(0x7fdb840b3ed0 msgr2=0x7fdb8415fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:32.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.500+0000 7fdb8adbd640 1 -- 172.21.15.16:0/2701739023 shutdown_connections 2024-09-19T07:55:32.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.500+0000 7fdb8adbd640 1 -- 172.21.15.16:0/2701739023 wait complete. 2024-09-19T07:55:32.515 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2024-09-19T07:55:32.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.626+0000 7f4aa6fd8640 1 Processor -- start 2024-09-19T07:55:32.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.626+0000 7f4aa6fd8640 1 -- start start 2024-09-19T07:55:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.627+0000 7f4aa6fd8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4aa015e2a0 0x7f4aa006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.627+0000 7f4aa6fd8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4aa006d960 0x7f4aa00d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.627+0000 7f4aa6fd8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4aa00d1880 con 0x7f4aa015e680 2024-09-19T07:55:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.627+0000 7f4aa6fd8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f4aa00d1b60 con 0x7f4aa015e2a0 2024-09-19T07:55:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.627+0000 7f4aa6fd8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4aa00cf2d0 con 0x7f4aa006d960 2024-09-19T07:55:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.627+0000 7f4aa4d74640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4aa015e2a0 0x7f4aa006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.627+0000 7f4aa4d74640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4aa015e2a0 0x7f4aa006d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49912/0 (socket says 172.21.15.16:49912) 2024-09-19T07:55:32.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.627+0000 7f4a9ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4aa015e680 0x7f4aa00cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52758/0 (socket says 172.21.15.16:52758) 2024-09-19T07:55:32.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.627+0000 7f4aa4d74640 1 -- 172.21.15.16:0/1000142510 learned_addr learned my addr 172.21.15.16:0/1000142510 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:32.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.628+0000 7f4aa5575640 1 --2- 172.21.15.16:0/1000142510 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4aa006d960 0x7f4aa00d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:32.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.628+0000 7f4aa4d74640 1 -- 172.21.15.16:0/1000142510 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4aa006d960 msgr2=0x7f4aa00d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:32.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.628+0000 7f4aa4d74640 1 --2- 172.21.15.16:0/1000142510 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4aa006d960 0x7f4aa00d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.628+0000 7f4aa4d74640 1 -- 172.21.15.16:0/1000142510 >> v1:172.21.15.16:6789/0 conn(0x7f4aa015e680 legacy=0x7f4aa00cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:32.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.628+0000 7f4aa4d74640 1 -- 172.21.15.16:0/1000142510 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa00cf5b0 con 0x7f4aa015e2a0 2024-09-19T07:55:32.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.628+0000 7f4a9f7fe640 1 -- 172.21.15.16:0/1000142510 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2976135367 0 0) 0x7f4aa00d1880 con 0x7f4aa015e680 2024-09-19T07:55:32.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.628+0000 7f4aa4d74640 1 --2- 172.21.15.16:0/1000142510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa015e2a0 0x7f4aa006d420 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009870 tx=0x7f4a9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bd0a704f0002c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.628+0000 7f4a9f7fe640 1 -- 172.21.15.16:0/1000142510 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90037070 con 0x7f4aa015e2a0 2024-09-19T07:55:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.629+0000 7f4a9f7fe640 1 -- 172.21.15.16:0/1000142510 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a90032440 con 0x7f4aa015e2a0 2024-09-19T07:55:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.629+0000 7f4a9f7fe640 1 -- 172.21.15.16:0/1000142510 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90032810 con 0x7f4aa015e2a0 2024-09-19T07:55:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.629+0000 7f4aa6fd8640 1 -- 172.21.15.16:0/1000142510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa015e2a0 msgr2=0x7f4aa006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.629+0000 7f4aa6fd8640 1 --2- 172.21.15.16:0/1000142510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa015e2a0 0x7f4aa006d420 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009870 tx=0x7f4a9002a270 comp rx=0 tx=0).stop 2024-09-19T07:55:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.629+0000 7f4aa6fd8640 1 -- 172.21.15.16:0/1000142510 shutdown_connections 2024-09-19T07:55:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.629+0000 7f4aa6fd8640 1 --2- 172.21.15.16:0/1000142510 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4aa006d960 0x7f4aa00d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.629+0000 7f4aa6fd8640 1 --2- 172.21.15.16:0/1000142510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa015e2a0 0x7f4aa006d420 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.629+0000 7f4aa6fd8640 1 -- 172.21.15.16:0/1000142510 >> 172.21.15.16:0/1000142510 conn(0x7f4aa009d760 msgr2=0x7f4aa008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.629+0000 7f4aa6fd8640 1 -- 172.21.15.16:0/1000142510 shutdown_connections 2024-09-19T07:55:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.629+0000 7f4aa6fd8640 1 -- 172.21.15.16:0/1000142510 wait complete. 2024-09-19T07:55:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.630+0000 7f4aa6fd8640 1 Processor -- start 2024-09-19T07:55:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.630+0000 7f4aa6fd8640 1 -- start start 2024-09-19T07:55:32.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.631+0000 7f4aa6fd8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4aa006d960 0x7f4aa00a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.631+0000 7f4aa6fd8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4aa015e2a0 0x7f4aa015f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.631+0000 7f4aa6fd8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa015e680 0x7f4aa009e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.631+0000 7f4aa6fd8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa00d0360 con 0x7f4aa015e680 2024-09-19T07:55:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.631+0000 7f4aa6fd8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa00d01e0 con 0x7f4aa006d960 2024-09-19T07:55:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.631+0000 7f4aa6fd8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4aa00d48b0 con 0x7f4aa015e2a0 2024-09-19T07:55:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.631+0000 7f4aa5575640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa015e680 0x7f4aa009e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.631+0000 7f4aa5575640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa015e680 0x7f4aa009e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49920/0 (socket says 172.21.15.16:49920) 2024-09-19T07:55:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.631+0000 7f4aa5575640 1 -- 172.21.15.16:0/1618974868 learned_addr learned my addr 172.21.15.16:0/1618974868 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.632+0000 7f4aa4d74640 1 --2- 172.21.15.16:0/1618974868 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4aa006d960 0x7f4aa00a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.632+0000 7f4a9ffff640 1 --2- 172.21.15.16:0/1618974868 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4aa015e2a0 0x7f4aa015f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.632+0000 7f4aa5575640 1 -- 172.21.15.16:0/1618974868 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4aa015e2a0 msgr2=0x7f4aa015f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.632+0000 7f4aa5575640 1 --2- 172.21.15.16:0/1618974868 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4aa015e2a0 0x7f4aa015f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.632+0000 7f4aa5575640 1 -- 172.21.15.16:0/1618974868 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4aa006d960 msgr2=0x7f4aa00a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.632+0000 7f4aa5575640 1 --2- 172.21.15.16:0/1618974868 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4aa006d960 0x7f4aa00a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.632+0000 7f4aa5575640 1 -- 172.21.15.16:0/1618974868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa00a0750 con 0x7f4aa015e680 2024-09-19T07:55:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.632+0000 7f4aa5575640 1 --2- 172.21.15.16:0/1618974868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa015e680 0x7f4aa009e9c0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800cd70 tx=0x7f4a9800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.632+0000 7f4a9d7fa640 1 -- 172.21.15.16:0/1618974868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98003b90 con 0x7f4aa015e680 2024-09-19T07:55:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.632+0000 7f4a9d7fa640 1 -- 172.21.15.16:0/1618974868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a98003d30 con 0x7f4aa015e680 2024-09-19T07:55:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.632+0000 7f4aa6fd8640 1 -- 172.21.15.16:0/1618974868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa00a09f0 con 0x7f4aa015e680 2024-09-19T07:55:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.632+0000 7f4a9d7fa640 1 -- 172.21.15.16:0/1618974868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98004240 con 0x7f4aa015e680 2024-09-19T07:55:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.632+0000 7f4aa6fd8640 1 -- 172.21.15.16:0/1618974868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa00a0fb0 con 0x7f4aa015e680 2024-09-19T07:55:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.633+0000 7f4aa6fd8640 1 -- 172.21.15.16:0/1618974868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aa0161450 con 0x7f4aa015e680 2024-09-19T07:55:32.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.634+0000 7f4a9d7fa640 1 -- 172.21.15.16:0/1618974868 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4a980043e0 con 0x7f4aa015e680 2024-09-19T07:55:32.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.634+0000 7f4a9d7fa640 1 --2- 172.21.15.16:0/1618974868 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4a8c06c2b0 0x7f4a8c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:32.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.634+0000 7f4a9d7fa640 1 -- 172.21.15.16:0/1618974868 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5335+0+0 (secure 0 0 0) 0x7f4a9808bbe0 con 0x7f4aa015e680 2024-09-19T07:55:32.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.635+0000 7f4aa4d74640 1 --2- 172.21.15.16:0/1618974868 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4a8c06c2b0 0x7f4a8c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:32.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.635+0000 7f4aa4d74640 1 --2- 172.21.15.16:0/1618974868 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4a8c06c2b0 0x7f4a8c06e770 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4a90034000 tx=0x7f4a9002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:32.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.639+0000 7f4a9d7fa640 1 -- 172.21.15.16:0/1618974868 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4aa0161450 con 0x7f4aa015e680 2024-09-19T07:55:32.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.818+0000 7f4aa6fd8640 1 -- 172.21.15.16:0/1618974868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) -- 0x7f4aa00d4c50 con 0x7f4aa015e680 2024-09-19T07:55:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.821+0000 7f4a9d7fa640 1 -- 172.21.15.16:0/1618974868 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v68) ==== 111+0+0 (secure 0 0 0) 0x7f4a98059f00 con 0x7f4aa015e680 2024-09-19T07:55:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.823+0000 7f4aa6fd8640 1 -- 172.21.15.16:0/1618974868 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4a8c06c2b0 msgr2=0x7f4a8c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.823+0000 7f4aa6fd8640 1 --2- 172.21.15.16:0/1618974868 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4a8c06c2b0 0x7f4a8c06e770 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4a90034000 tx=0x7f4a9002e000 comp rx=0 tx=0).stop 2024-09-19T07:55:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.823+0000 7f4aa6fd8640 1 -- 172.21.15.16:0/1618974868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa015e680 msgr2=0x7f4aa009e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.823+0000 7f4aa6fd8640 1 --2- 172.21.15.16:0/1618974868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa015e680 0x7f4aa009e9c0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800cd70 tx=0x7f4a9800a660 comp rx=0 tx=0).stop 2024-09-19T07:55:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.824+0000 7f4aa6fd8640 1 -- 172.21.15.16:0/1618974868 shutdown_connections 2024-09-19T07:55:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.824+0000 7f4aa6fd8640 1 --2- 172.21.15.16:0/1618974868 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4a8c06c2b0 0x7f4a8c06e770 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.824+0000 7f4aa6fd8640 1 --2- 172.21.15.16:0/1618974868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4aa015e680 0x7f4aa009e9c0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.824+0000 7f4aa6fd8640 1 --2- 172.21.15.16:0/1618974868 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4aa015e2a0 0x7f4aa015f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.824+0000 7f4aa6fd8640 1 --2- 172.21.15.16:0/1618974868 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4aa006d960 0x7f4aa00a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.824+0000 7f4aa6fd8640 1 -- 172.21.15.16:0/1618974868 >> 172.21.15.16:0/1618974868 conn(0x7f4aa009d760 msgr2=0x7f4aa00d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.824+0000 7f4aa6fd8640 1 -- 172.21.15.16:0/1618974868 shutdown_connections 2024-09-19T07:55:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.824+0000 7f4aa6fd8640 1 -- 172.21.15.16:0/1618974868 wait complete. 2024-09-19T07:55:32.835 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2024-09-19T07:55:32.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.939+0000 7fb02505e640 1 Processor -- start 2024-09-19T07:55:32.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.939+0000 7fb02505e640 1 -- start start 2024-09-19T07:55:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.940+0000 7fb02505e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb02015e2a0 0x7fb02006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.940+0000 7fb02505e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb02015e680 0x7fb0200cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.940+0000 7fb02505e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb02006d960 0x7fb0200d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.940+0000 7fb02505e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb0200d1880 con 0x7fb02015e2a0 2024-09-19T07:55:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.940+0000 7fb02505e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb0200d1b60 con 0x7fb02015e680 2024-09-19T07:55:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.940+0000 7fb02505e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb0200cf2d0 con 0x7fb02006d960 2024-09-19T07:55:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.940+0000 7fb01ed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb02015e2a0 0x7fb02006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.940+0000 7fb01ed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb02015e2a0 0x7fb02006d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49934/0 (socket says 172.21.15.16:49934) 2024-09-19T07:55:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.940+0000 7fb01ed9d640 1 -- 172.21.15.16:0/2651397632 learned_addr learned my addr 172.21.15.16:0/2651397632 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.940+0000 7fb01f59e640 1 --2- 172.21.15.16:0/2651397632 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb02006d960 0x7fb0200d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.940+0000 7fb01e59c640 1 --2- 172.21.15.16:0/2651397632 >> v2:172.21.15.146:3300/0 conn(0x7fb02015e680 0x7fb0200cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.940+0000 7fb01ed9d640 1 -- 172.21.15.16:0/2651397632 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb02006d960 msgr2=0x7fb0200d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.940+0000 7fb01ed9d640 1 --2- 172.21.15.16:0/2651397632 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb02006d960 0x7fb0200d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.940+0000 7fb01ed9d640 1 -- 172.21.15.16:0/2651397632 >> v2:172.21.15.146:3300/0 conn(0x7fb02015e680 msgr2=0x7fb0200cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.940+0000 7fb01ed9d640 1 --2- 172.21.15.16:0/2651397632 >> v2:172.21.15.146:3300/0 conn(0x7fb02015e680 0x7fb0200cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.940+0000 7fb01ed9d640 1 -- 172.21.15.16:0/2651397632 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0200cf5b0 con 0x7fb02015e2a0 2024-09-19T07:55:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.941+0000 7fb01ed9d640 1 --2- 172.21.15.16:0/2651397632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb02015e2a0 0x7fb02006d420 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb0200a22a0 tx=0x7fb01002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296bfbe63b1e3045 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.941+0000 7fb01dd9b640 1 -- 172.21.15.16:0/2651397632 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010037070 con 0x7fb02015e2a0 2024-09-19T07:55:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.941+0000 7fb01dd9b640 1 -- 172.21.15.16:0/2651397632 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb010032440 con 0x7fb02015e2a0 2024-09-19T07:55:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.941+0000 7fb01dd9b640 1 -- 172.21.15.16:0/2651397632 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010032810 con 0x7fb02015e2a0 2024-09-19T07:55:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.942+0000 7fb02505e640 1 -- 172.21.15.16:0/2651397632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb02015e2a0 msgr2=0x7fb02006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.942+0000 7fb02505e640 1 --2- 172.21.15.16:0/2651397632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb02015e2a0 0x7fb02006d420 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb0200a22a0 tx=0x7fb01002a1e0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.942+0000 7fb02505e640 1 -- 172.21.15.16:0/2651397632 shutdown_connections 2024-09-19T07:55:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.942+0000 7fb02505e640 1 --2- 172.21.15.16:0/2651397632 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb02006d960 0x7fb0200d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.942+0000 7fb02505e640 1 --2- 172.21.15.16:0/2651397632 >> v2:172.21.15.146:3300/0 conn(0x7fb02015e680 0x7fb0200cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.942+0000 7fb02505e640 1 --2- 172.21.15.16:0/2651397632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb02015e2a0 0x7fb02006d420 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.942+0000 7fb02505e640 1 -- 172.21.15.16:0/2651397632 >> 172.21.15.16:0/2651397632 conn(0x7fb02009d760 msgr2=0x7fb02008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.942+0000 7fb02505e640 1 -- 172.21.15.16:0/2651397632 shutdown_connections 2024-09-19T07:55:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.942+0000 7fb02505e640 1 -- 172.21.15.16:0/2651397632 wait complete. 2024-09-19T07:55:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.943+0000 7fb02505e640 1 Processor -- start 2024-09-19T07:55:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.943+0000 7fb02505e640 1 -- start start 2024-09-19T07:55:32.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.944+0000 7fb02505e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb02006d960 0x7fb0201564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:32.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.944+0000 7fb02505e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb02015e2a0 0x7fb0200ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:32.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.944+0000 7fb02505e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb02015e680 0x7fb0200ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.944+0000 7fb02505e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0200d44f0 con 0x7fb02015e2a0 2024-09-19T07:55:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.944+0000 7fb02505e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb0200d4370 con 0x7fb02015e680 2024-09-19T07:55:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.944+0000 7fb02505e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb0200d4670 con 0x7fb02006d960 2024-09-19T07:55:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.944+0000 7fb01e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb02015e2a0 0x7fb0200ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.944+0000 7fb01e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb02015e2a0 0x7fb0200ca460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49936/0 (socket says 172.21.15.16:49936) 2024-09-19T07:55:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.944+0000 7fb01e59c640 1 -- 172.21.15.16:0/1482405220 learned_addr learned my addr 172.21.15.16:0/1482405220 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.944+0000 7fb01ed9d640 1 --2- 172.21.15.16:0/1482405220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb02006d960 0x7fb0201564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.944+0000 7fb01f59e640 1 --2- 172.21.15.16:0/1482405220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb02015e680 0x7fb0200ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.944+0000 7fb01e59c640 1 -- 172.21.15.16:0/1482405220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb02006d960 msgr2=0x7fb0201564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.944+0000 7fb01e59c640 1 --2- 172.21.15.16:0/1482405220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb02006d960 0x7fb0201564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.944+0000 7fb01e59c640 1 -- 172.21.15.16:0/1482405220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb02015e680 msgr2=0x7fb0200ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.944+0000 7fb01e59c640 1 --2- 172.21.15.16:0/1482405220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb02015e680 0x7fb0200ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.944+0000 7fb01e59c640 1 -- 172.21.15.16:0/1482405220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0201597d0 con 0x7fb02015e2a0 2024-09-19T07:55:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.945+0000 7fb01e59c640 1 --2- 172.21.15.16:0/1482405220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb02015e2a0 0x7fb0200ca460 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fb01400d5c0 tx=0x7fb01400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.945+0000 7fb0077fe640 1 -- 172.21.15.16:0/1482405220 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140125f0 con 0x7fb02015e2a0 2024-09-19T07:55:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.945+0000 7fb0077fe640 1 -- 172.21.15.16:0/1482405220 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb014019070 con 0x7fb02015e2a0 2024-09-19T07:55:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.945+0000 7fb02505e640 1 -- 172.21.15.16:0/1482405220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb020159ab0 con 0x7fb02015e2a0 2024-09-19T07:55:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.945+0000 7fb0077fe640 1 -- 172.21.15.16:0/1482405220 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140129d0 con 0x7fb02015e2a0 2024-09-19T07:55:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.945+0000 7fb02505e640 1 -- 172.21.15.16:0/1482405220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb02015a570 con 0x7fb02015e2a0 2024-09-19T07:55:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.946+0000 7fb02505e640 1 -- 172.21.15.16:0/1482405220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb02006d420 con 0x7fb02015e2a0 2024-09-19T07:55:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.946+0000 7fb0077fe640 1 -- 172.21.15.16:0/1482405220 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb014012b70 con 0x7fb02015e2a0 2024-09-19T07:55:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.947+0000 7fb0077fe640 1 --2- 172.21.15.16:0/1482405220 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb00c06c2b0 0x7fb00c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.947+0000 7fb0077fe640 1 -- 172.21.15.16:0/1482405220 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5335+0+0 (secure 0 0 0) 0x7fb01408b510 con 0x7fb02015e2a0 2024-09-19T07:55:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.947+0000 7fb01ed9d640 1 --2- 172.21.15.16:0/1482405220 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb00c06c2b0 0x7fb00c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.948+0000 7fb01ed9d640 1 --2- 172.21.15.16:0/1482405220 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb00c06c2b0 0x7fb00c06e770 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb010034000 tx=0x7fb01002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:32.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:32.952+0000 7fb0077fe640 1 -- 172.21.15.16:0/1482405220 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb014059e80 con 0x7fb02015e2a0 2024-09-19T07:55:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.130+0000 7fb02505e640 1 -- 172.21.15.16:0/1482405220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) -- 0x7fb0200d3c00 con 0x7fb02015e2a0 2024-09-19T07:55:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.133+0000 7fb0077fe640 1 -- 172.21.15.16:0/1482405220 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v69) ==== 107+0+0 (secure 0 0 0) 0x7fb014059780 con 0x7fb02015e2a0 2024-09-19T07:55:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.136+0000 7fb02505e640 1 -- 172.21.15.16:0/1482405220 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb00c06c2b0 msgr2=0x7fb00c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.136+0000 7fb02505e640 1 --2- 172.21.15.16:0/1482405220 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb00c06c2b0 0x7fb00c06e770 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb010034000 tx=0x7fb01002e000 comp rx=0 tx=0).stop 2024-09-19T07:55:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.136+0000 7fb02505e640 1 -- 172.21.15.16:0/1482405220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb02015e2a0 msgr2=0x7fb0200ca460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.136+0000 7fb02505e640 1 --2- 172.21.15.16:0/1482405220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb02015e2a0 0x7fb0200ca460 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fb01400d5c0 tx=0x7fb01400da80 comp rx=0 tx=0).stop 2024-09-19T07:55:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.136+0000 7fb02505e640 1 -- 172.21.15.16:0/1482405220 shutdown_connections 2024-09-19T07:55:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.136+0000 7fb02505e640 1 --2- 172.21.15.16:0/1482405220 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb00c06c2b0 0x7fb00c06e770 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.136+0000 7fb02505e640 1 --2- 172.21.15.16:0/1482405220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb02015e680 0x7fb0200ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.136+0000 7fb02505e640 1 --2- 172.21.15.16:0/1482405220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb02015e2a0 0x7fb0200ca460 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.136+0000 7fb02505e640 1 --2- 172.21.15.16:0/1482405220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb02006d960 0x7fb0201564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.136+0000 7fb02505e640 1 -- 172.21.15.16:0/1482405220 >> 172.21.15.16:0/1482405220 conn(0x7fb02009d760 msgr2=0x7fb0200d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.136+0000 7fb02505e640 1 -- 172.21.15.16:0/1482405220 shutdown_connections 2024-09-19T07:55:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.136+0000 7fb02505e640 1 -- 172.21.15.16:0/1482405220 wait complete. 2024-09-19T07:55:33.147 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2024-09-19T07:55:33.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.250+0000 7f41e3baf640 1 Processor -- start 2024-09-19T07:55:33.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.250+0000 7f41e3baf640 1 -- start start 2024-09-19T07:55:33.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.251+0000 7f41e3baf640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f41dc161eb0 0x7f41dc1622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.251+0000 7f41e3baf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41dc162b70 con 0x7f41dc06cd20 2024-09-19T07:55:33.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.251+0000 7f41e3baf640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41dc141b00 con 0x7f41dc1425e0 2024-09-19T07:55:33.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.251+0000 7f41e3baf640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f41dc162e50 con 0x7f41dc161eb0 2024-09-19T07:55:33.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.251+0000 7f41e194b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f41dc06cd20 0x7f41dc06d100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52766/0 (socket says 172.21.15.16:52766) 2024-09-19T07:55:33.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.251+0000 7f41e194b640 1 -- 172.21.15.16:0/2937928412 learned_addr learned my addr 172.21.15.16:0/2937928412 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:33.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.251+0000 7f41e0949640 1 -- 172.21.15.16:0/2937928412 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3837127777 0 0) 0x7f41dc162b70 con 0x7f41dc06cd20 2024-09-19T07:55:33.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.251+0000 7f41e114a640 1 --2- 172.21.15.16:0/2937928412 >> v2:172.21.15.146:3300/0 conn(0x7f41dc161eb0 0x7f41dc1622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.251+0000 7f41e0949640 1 -- 172.21.15.16:0/2937928412 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41c8003520 con 0x7f41dc06cd20 2024-09-19T07:55:33.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.251+0000 7f41e0949640 1 -- 172.21.15.16:0/2937928412 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2014804310 0 0) 0x7f41c8003520 con 0x7f41dc06cd20 2024-09-19T07:55:33.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.251+0000 7f41e0949640 1 -- 172.21.15.16:0/2937928412 >> v2:172.21.15.146:3300/0 conn(0x7f41dc161eb0 msgr2=0x7f41dc1622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:33.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e0949640 1 --2- 172.21.15.16:0/2937928412 >> v2:172.21.15.146:3300/0 conn(0x7f41dc161eb0 0x7f41dc1622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e0949640 1 -- 172.21.15.16:0/2937928412 >> v1:172.21.15.146:6789/0 conn(0x7f41dc1425e0 legacy=0x7f41dc142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:33.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e0949640 1 -- 172.21.15.16:0/2937928412 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41dc141de0 con 0x7f41dc06cd20 2024-09-19T07:55:33.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e0949640 1 -- 172.21.15.16:0/2937928412 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f41d8001670 con 0x7f41dc06cd20 2024-09-19T07:55:33.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e0949640 1 -- 172.21.15.16:0/2937928412 >> v1:172.21.15.16:6789/0 conn(0x7f41dc06cd20 legacy=0x7f41dc06d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:33.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e0949640 1 --2- 172.21.15.16:0/2937928412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c80039b0 0x7f41c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e0949640 1 --2- 172.21.15.16:0/2937928412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41c8004800 0x7f41c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e0949640 1 --2- 172.21.15.16:0/2937928412 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41c80051c0 0x7f41c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e0949640 1 -- 172.21.15.16:0/2937928412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc162e50 con 0x7f41c80039b0 2024-09-19T07:55:33.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e0949640 1 -- 172.21.15.16:0/2937928412 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f41c800c160 con 0x7f41c8004800 2024-09-19T07:55:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e0949640 1 -- 172.21.15.16:0/2937928412 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f41c800c360 con 0x7f41c80051c0 2024-09-19T07:55:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e194b640 1 --2- 172.21.15.16:0/2937928412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c80039b0 0x7f41c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e114a640 1 --2- 172.21.15.16:0/2937928412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41c8004800 0x7f41c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e214c640 1 --2- 172.21.15.16:0/2937928412 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41c80051c0 0x7f41c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e194b640 1 -- 172.21.15.16:0/2937928412 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41c80051c0 msgr2=0x7f41c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e194b640 1 --2- 172.21.15.16:0/2937928412 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41c80051c0 0x7f41c800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e194b640 1 -- 172.21.15.16:0/2937928412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41c8004800 msgr2=0x7f41c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e194b640 1 --2- 172.21.15.16:0/2937928412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41c8004800 0x7f41c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e194b640 1 -- 172.21.15.16:0/2937928412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c8003520 con 0x7f41c80039b0 2024-09-19T07:55:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.252+0000 7f41e214c640 1 -- 172.21.15.16:0/2937928412 reap_dead start 2024-09-19T07:55:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.253+0000 7f41e194b640 1 --2- 172.21.15.16:0/2937928412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c80039b0 0x7f41c8003dd0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f41d800c290 tx=0x7f41c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8dc35d93f38fa3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.253+0000 7f41e0949640 1 -- 172.21.15.16:0/2937928412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8007d30 con 0x7f41c80039b0 2024-09-19T07:55:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.253+0000 7f41e0949640 1 -- 172.21.15.16:0/2937928412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d80058d0 con 0x7f41c80039b0 2024-09-19T07:55:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.253+0000 7f41e0949640 1 -- 172.21.15.16:0/2937928412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8005ce0 con 0x7f41c80039b0 2024-09-19T07:55:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.253+0000 7f41e3baf640 1 -- 172.21.15.16:0/2937928412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c80039b0 msgr2=0x7f41c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.253+0000 7f41e3baf640 1 --2- 172.21.15.16:0/2937928412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c80039b0 0x7f41c8003dd0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f41d800c290 tx=0x7f41c8005830 comp rx=0 tx=0).stop 2024-09-19T07:55:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.253+0000 7f41e3baf640 1 -- 172.21.15.16:0/2937928412 shutdown_connections 2024-09-19T07:55:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.253+0000 7f41e3baf640 1 --2- 172.21.15.16:0/2937928412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c80039b0 0x7f41c8003dd0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.253+0000 7f41e3baf640 1 -- 172.21.15.16:0/2937928412 >> 172.21.15.16:0/2937928412 conn(0x7f41dc0b3ed0 msgr2=0x7f41dc0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.254+0000 7f41e3baf640 1 -- 172.21.15.16:0/2937928412 shutdown_connections 2024-09-19T07:55:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.254+0000 7f41e3baf640 1 -- 172.21.15.16:0/2937928412 wait complete. 2024-09-19T07:55:33.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.255+0000 7f41e3baf640 1 Processor -- start 2024-09-19T07:55:33.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.255+0000 7f41e3baf640 1 -- start start 2024-09-19T07:55:33.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.255+0000 7f41e3baf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41c80039b0 0x7f41dc0d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.255+0000 7f41e3baf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41dc144a30 0x7f41dc146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.255+0000 7f41e3baf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41dc1f39a0 0x7f41dc1f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.255+0000 7f41e3baf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc06d100 con 0x7f41dc144a30 2024-09-19T07:55:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.255+0000 7f41e3baf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc142150 con 0x7f41dc1f39a0 2024-09-19T07:55:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.255+0000 7f41e3baf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f41dc06d280 con 0x7f41c80039b0 2024-09-19T07:55:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.256+0000 7f41e114a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41dc144a30 0x7f41dc146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.256+0000 7f41e114a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41dc144a30 0x7f41dc146e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49960/0 (socket says 172.21.15.16:49960) 2024-09-19T07:55:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.256+0000 7f41e114a640 1 -- 172.21.15.16:0/2410536 learned_addr learned my addr 172.21.15.16:0/2410536 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.256+0000 7f41e214c640 1 --2- 172.21.15.16:0/2410536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41dc1f39a0 0x7f41dc1f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.256+0000 7f41e194b640 1 --2- 172.21.15.16:0/2410536 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41c80039b0 0x7f41dc0d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.256+0000 7f41e114a640 1 -- 172.21.15.16:0/2410536 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41c80039b0 msgr2=0x7f41dc0d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:33.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.256+0000 7f41e114a640 1 --2- 172.21.15.16:0/2410536 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41c80039b0 0x7f41dc0d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.256+0000 7f41e114a640 1 -- 172.21.15.16:0/2410536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41dc1f39a0 msgr2=0x7f41dc1f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.256+0000 7f41e114a640 1 --2- 172.21.15.16:0/2410536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41dc1f39a0 0x7f41dc1f3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.256+0000 7f41e114a640 1 -- 172.21.15.16:0/2410536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41dc1f4370 con 0x7f41dc144a30 2024-09-19T07:55:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.256+0000 7f41e114a640 1 --2- 172.21.15.16:0/2410536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41dc144a30 0x7f41dc146e20 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f41cc011700 tx=0x7f41cc011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.256+0000 7f41d27fc640 1 -- 172.21.15.16:0/2410536 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc00f960 con 0x7f41dc144a30 2024-09-19T07:55:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.256+0000 7f41d27fc640 1 -- 172.21.15.16:0/2410536 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41cc00fb00 con 0x7f41dc144a30 2024-09-19T07:55:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.256+0000 7f41d27fc640 1 -- 172.21.15.16:0/2410536 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc015460 con 0x7f41dc144a30 2024-09-19T07:55:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.256+0000 7f41e3baf640 1 -- 172.21.15.16:0/2410536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41dc144f20 con 0x7f41dc144a30 2024-09-19T07:55:33.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.256+0000 7f41e3baf640 1 -- 172.21.15.16:0/2410536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41dc145760 con 0x7f41dc144a30 2024-09-19T07:55:33.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.257+0000 7f41e3baf640 1 -- 172.21.15.16:0/2410536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41dc162b70 con 0x7f41dc144a30 2024-09-19T07:55:33.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.258+0000 7f41d27fc640 1 -- 172.21.15.16:0/2410536 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f41cc015600 con 0x7f41dc144a30 2024-09-19T07:55:33.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.258+0000 7f41d27fc640 1 --2- 172.21.15.16:0/2410536 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f41c006c2b0 0x7f41c006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.258+0000 7f41d27fc640 1 -- 172.21.15.16:0/2410536 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5335+0+0 (secure 0 0 0) 0x7f41cc08bb60 con 0x7f41dc144a30 2024-09-19T07:55:33.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.258+0000 7f41e194b640 1 --2- 172.21.15.16:0/2410536 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f41c006c2b0 0x7f41c006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.259+0000 7f41e194b640 1 --2- 172.21.15.16:0/2410536 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f41c006c2b0 0x7f41c006e770 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f41d8033000 tx=0x7f41d8009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:33.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.263+0000 7f41d27fc640 1 -- 172.21.15.16:0/2410536 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f41dc144f20 con 0x7f41dc144a30 2024-09-19T07:55:33.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.437+0000 7f41e3baf640 1 -- 172.21.15.16:0/2410536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) -- 0x7f41dc072220 con 0x7f41dc144a30 2024-09-19T07:55:33.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.440+0000 7f41d27fc640 1 -- 172.21.15.16:0/2410536 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v70) ==== 102+0+0 (secure 0 0 0) 0x7f41cc019030 con 0x7f41dc144a30 2024-09-19T07:55:33.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.442+0000 7f41e3baf640 1 -- 172.21.15.16:0/2410536 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f41c006c2b0 msgr2=0x7f41c006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:33.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.442+0000 7f41e3baf640 1 --2- 172.21.15.16:0/2410536 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f41c006c2b0 0x7f41c006e770 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f41d8033000 tx=0x7f41d8009030 comp rx=0 tx=0).stop 2024-09-19T07:55:33.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.442+0000 7f41e3baf640 1 -- 172.21.15.16:0/2410536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41dc144a30 msgr2=0x7f41dc146e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.442+0000 7f41e3baf640 1 --2- 172.21.15.16:0/2410536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41dc144a30 0x7f41dc146e20 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f41cc011700 tx=0x7f41cc011bc0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.442+0000 7f41e3baf640 1 -- 172.21.15.16:0/2410536 shutdown_connections 2024-09-19T07:55:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.442+0000 7f41e3baf640 1 --2- 172.21.15.16:0/2410536 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f41c006c2b0 0x7f41c006e770 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.442+0000 7f41e3baf640 1 --2- 172.21.15.16:0/2410536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41dc1f39a0 0x7f41dc1f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.442+0000 7f41e3baf640 1 --2- 172.21.15.16:0/2410536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41dc144a30 0x7f41dc146e20 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.442+0000 7f41e3baf640 1 --2- 172.21.15.16:0/2410536 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41c80039b0 0x7f41dc0d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.442+0000 7f41e3baf640 1 -- 172.21.15.16:0/2410536 >> 172.21.15.16:0/2410536 conn(0x7f41dc0b3ed0 msgr2=0x7f41dc15e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.442+0000 7f41e3baf640 1 -- 172.21.15.16:0/2410536 shutdown_connections 2024-09-19T07:55:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.442+0000 7f41e3baf640 1 -- 172.21.15.16:0/2410536 wait complete. 2024-09-19T07:55:33.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T07:55:33.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.559+0000 7f114ef03640 1 Processor -- start 2024-09-19T07:55:33.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.559+0000 7f114ef03640 1 -- start start 2024-09-19T07:55:33.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.560+0000 7f114ef03640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f114815e680 0x7f11480cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.560+0000 7f114ef03640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f11480d0010 0x7f11480d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.560+0000 7f114ef03640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11480d1880 con 0x7f114815e2a0 2024-09-19T07:55:33.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.560+0000 7f114ef03640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f11480d1b60 con 0x7f11480d0010 2024-09-19T07:55:33.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.560+0000 7f114ef03640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f11480cf2d0 con 0x7f114815e680 2024-09-19T07:55:33.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.560+0000 7f114cc9f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f114815e2a0 0x7f114806d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52768/0 (socket says 172.21.15.16:52768) 2024-09-19T07:55:33.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.560+0000 7f114cc9f640 1 -- 172.21.15.16:0/3647928534 learned_addr learned my addr 172.21.15.16:0/3647928534 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:33.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f1147fff640 1 --2- 172.21.15.16:0/3647928534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f114815e680 0x7f11480cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f114d4a0640 1 --2- 172.21.15.16:0/3647928534 >> v2:172.21.15.146:3300/0 conn(0x7f11480d0010 0x7f11480d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f11477fe640 1 -- 172.21.15.16:0/3647928534 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 171662175 0 0) 0x7f11480d1880 con 0x7f114815e2a0 2024-09-19T07:55:33.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f11477fe640 1 -- 172.21.15.16:0/3647928534 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1134003520 con 0x7f114815e2a0 2024-09-19T07:55:33.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f11477fe640 1 -- 172.21.15.16:0/3647928534 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 703411933 0 0) 0x7f1134003520 con 0x7f114815e2a0 2024-09-19T07:55:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f11477fe640 1 -- 172.21.15.16:0/3647928534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f114815e680 msgr2=0x7f11480cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f11477fe640 1 --2- 172.21.15.16:0/3647928534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f114815e680 0x7f11480cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f11477fe640 1 -- 172.21.15.16:0/3647928534 >> v2:172.21.15.146:3300/0 conn(0x7f11480d0010 msgr2=0x7f11480d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f11477fe640 1 --2- 172.21.15.16:0/3647928534 >> v2:172.21.15.146:3300/0 conn(0x7f11480d0010 0x7f11480d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f11477fe640 1 -- 172.21.15.16:0/3647928534 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11480cf5b0 con 0x7f114815e2a0 2024-09-19T07:55:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f11477fe640 1 -- 172.21.15.16:0/3647928534 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f1138001670 con 0x7f114815e2a0 2024-09-19T07:55:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f11477fe640 1 -- 172.21.15.16:0/3647928534 >> v1:172.21.15.16:6789/0 conn(0x7f114815e2a0 legacy=0x7f114806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f11477fe640 1 --2- 172.21.15.16:0/3647928534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f114815e680 0x7f11340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f11477fe640 1 --2- 172.21.15.16:0/3647928534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1134003f30 0x7f1134004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f11477fe640 1 --2- 172.21.15.16:0/3647928534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1134004d40 0x7f11340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f11477fe640 1 -- 172.21.15.16:0/3647928534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11480d1b60 con 0x7f1134003f30 2024-09-19T07:55:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f11477fe640 1 -- 172.21.15.16:0/3647928534 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f11480cf2d0 con 0x7f1134004d40 2024-09-19T07:55:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f11477fe640 1 -- 172.21.15.16:0/3647928534 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f113400ba10 con 0x7f114815e680 2024-09-19T07:55:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f1147fff640 1 --2- 172.21.15.16:0/3647928534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1134003f30 0x7f1134004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f114cc9f640 1 --2- 172.21.15.16:0/3647928534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f114815e680 0x7f11340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f114d4a0640 1 --2- 172.21.15.16:0/3647928534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1134004d40 0x7f11340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f1147fff640 1 -- 172.21.15.16:0/3647928534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f114815e680 msgr2=0x7f11340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f1147fff640 1 --2- 172.21.15.16:0/3647928534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f114815e680 0x7f11340039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f1147fff640 1 -- 172.21.15.16:0/3647928534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1134004d40 msgr2=0x7f11340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f1147fff640 1 --2- 172.21.15.16:0/3647928534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1134004d40 0x7f11340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.561+0000 7f1147fff640 1 -- 172.21.15.16:0/3647928534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1134003520 con 0x7f1134003f30 2024-09-19T07:55:33.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.562+0000 7f1147fff640 1 --2- 172.21.15.16:0/3647928534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1134003f30 0x7f1134004800 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f114815ea80 tx=0x7f1134005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8bd9da24e88c1e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.562+0000 7f11477fe640 1 -- 172.21.15.16:0/3647928534 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1130037070 con 0x7f1134003f30 2024-09-19T07:55:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.562+0000 7f11477fe640 1 -- 172.21.15.16:0/3647928534 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f113002e050 con 0x7f1134003f30 2024-09-19T07:55:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.562+0000 7f11477fe640 1 -- 172.21.15.16:0/3647928534 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1130009c10 con 0x7f1134003f30 2024-09-19T07:55:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.563+0000 7f114ef03640 1 -- 172.21.15.16:0/3647928534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1134003f30 msgr2=0x7f1134004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.563+0000 7f114ef03640 1 --2- 172.21.15.16:0/3647928534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1134003f30 0x7f1134004800 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f114815ea80 tx=0x7f1134005830 comp rx=0 tx=0).stop 2024-09-19T07:55:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.563+0000 7f114d4a0640 1 -- 172.21.15.16:0/3647928534 reap_dead start 2024-09-19T07:55:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.563+0000 7f114ef03640 1 -- 172.21.15.16:0/3647928534 shutdown_connections 2024-09-19T07:55:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.563+0000 7f114ef03640 1 -- 172.21.15.16:0/3647928534 >> 172.21.15.16:0/3647928534 conn(0x7f114809d760 msgr2=0x7f114808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.563+0000 7f114ef03640 1 -- 172.21.15.16:0/3647928534 shutdown_connections 2024-09-19T07:55:33.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.563+0000 7f114ef03640 1 -- 172.21.15.16:0/3647928534 wait complete. 2024-09-19T07:55:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.564+0000 7f114ef03640 1 Processor -- start 2024-09-19T07:55:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.564+0000 7f114ef03640 1 -- start start 2024-09-19T07:55:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.565+0000 7f114ef03640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1134003f30 0x7f1148159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.565+0000 7f114ef03640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11480cd080 0x7f11480cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.565+0000 7f114ef03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11481e9710 0x7f11481e9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.565+0000 7f114ef03640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f114806dcb0 con 0x7f11481e9710 2024-09-19T07:55:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.565+0000 7f114ef03640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f114806db30 con 0x7f1134003f30 2024-09-19T07:55:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.565+0000 7f114ef03640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f11480cfad0 con 0x7f11480cd080 2024-09-19T07:55:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.565+0000 7f114d4a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11481e9710 0x7f11481e9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.565+0000 7f114d4a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11481e9710 0x7f11481e9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49968/0 (socket says 172.21.15.16:49968) 2024-09-19T07:55:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.565+0000 7f114d4a0640 1 -- 172.21.15.16:0/2550867134 learned_addr learned my addr 172.21.15.16:0/2550867134 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.565+0000 7f1147fff640 1 --2- 172.21.15.16:0/2550867134 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11480cd080 0x7f11480cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.565+0000 7f114cc9f640 1 --2- 172.21.15.16:0/2550867134 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1134003f30 0x7f1148159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.565+0000 7f114d4a0640 1 -- 172.21.15.16:0/2550867134 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11480cd080 msgr2=0x7f11480cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.565+0000 7f114d4a0640 1 --2- 172.21.15.16:0/2550867134 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11480cd080 0x7f11480cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.565+0000 7f114d4a0640 1 -- 172.21.15.16:0/2550867134 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1134003f30 msgr2=0x7f1148159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.565+0000 7f114d4a0640 1 --2- 172.21.15.16:0/2550867134 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1134003f30 0x7f1148159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.565+0000 7f114d4a0640 1 -- 172.21.15.16:0/2550867134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11480ba210 con 0x7f11481e9710 2024-09-19T07:55:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.565+0000 7f114cc9f640 1 --2- 172.21.15.16:0/2550867134 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1134003f30 0x7f1148159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.566+0000 7f114d4a0640 1 --2- 172.21.15.16:0/2550867134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11481e9710 0x7f11481e9bc0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f114806eda0 tx=0x7f1140011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.566+0000 7f11457fa640 1 -- 172.21.15.16:0/2550867134 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140019020 con 0x7f11481e9710 2024-09-19T07:55:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.566+0000 7f11457fa640 1 -- 172.21.15.16:0/2550867134 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1140003610 con 0x7f11481e9710 2024-09-19T07:55:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.566+0000 7f11457fa640 1 -- 172.21.15.16:0/2550867134 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140003a00 con 0x7f11481e9710 2024-09-19T07:55:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.566+0000 7f114ef03640 1 -- 172.21.15.16:0/2550867134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11480b9c50 con 0x7f11481e9710 2024-09-19T07:55:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.566+0000 7f114ef03640 1 -- 172.21.15.16:0/2550867134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11481e8f10 con 0x7f11481e9710 2024-09-19T07:55:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.567+0000 7f114ef03640 1 -- 172.21.15.16:0/2550867134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f114806d420 con 0x7f11481e9710 2024-09-19T07:55:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.567+0000 7f11457fa640 1 -- 172.21.15.16:0/2550867134 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f114001c020 con 0x7f11481e9710 2024-09-19T07:55:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.568+0000 7f11457fa640 1 --2- 172.21.15.16:0/2550867134 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f112806c2e0 0x7f112806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.568+0000 7f11457fa640 1 -- 172.21.15.16:0/2550867134 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5335+0+0 (secure 0 0 0) 0x7f114008b400 con 0x7f11481e9710 2024-09-19T07:55:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.568+0000 7f114cc9f640 1 --2- 172.21.15.16:0/2550867134 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f112806c2e0 0x7f112806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.569+0000 7f114cc9f640 1 --2- 172.21.15.16:0/2550867134 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f112806c2e0 0x7f112806e7a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f11481eab20 tx=0x7f1138009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.572+0000 7f11457fa640 1 -- 172.21.15.16:0/2550867134 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1140023030 con 0x7f11481e9710 2024-09-19T07:55:33.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.746+0000 7f114ef03640 1 -- 172.21.15.16:0/2550867134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f114806d910 con 0x7f11481e9710 2024-09-19T07:55:33.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.749+0000 7f11457fa640 1 -- 172.21.15.16:0/2550867134 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v71) ==== 104+0+0 (secure 0 0 0) 0x7f1140059d70 con 0x7f11481e9710 2024-09-19T07:55:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.751+0000 7f114ef03640 1 -- 172.21.15.16:0/2550867134 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f112806c2e0 msgr2=0x7f112806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.751+0000 7f114ef03640 1 --2- 172.21.15.16:0/2550867134 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f112806c2e0 0x7f112806e7a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f11481eab20 tx=0x7f1138009040 comp rx=0 tx=0).stop 2024-09-19T07:55:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.751+0000 7f114ef03640 1 -- 172.21.15.16:0/2550867134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11481e9710 msgr2=0x7f11481e9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.751+0000 7f114ef03640 1 --2- 172.21.15.16:0/2550867134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11481e9710 0x7f11481e9bc0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f114806eda0 tx=0x7f1140011920 comp rx=0 tx=0).stop 2024-09-19T07:55:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.751+0000 7f114ef03640 1 -- 172.21.15.16:0/2550867134 shutdown_connections 2024-09-19T07:55:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.751+0000 7f114ef03640 1 --2- 172.21.15.16:0/2550867134 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f112806c2e0 0x7f112806e7a0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.751+0000 7f114ef03640 1 --2- 172.21.15.16:0/2550867134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11481e9710 0x7f11481e9bc0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.751+0000 7f114ef03640 1 --2- 172.21.15.16:0/2550867134 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11480cd080 0x7f11480cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.751+0000 7f114ef03640 1 --2- 172.21.15.16:0/2550867134 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1134003f30 0x7f1148159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.751+0000 7f114ef03640 1 -- 172.21.15.16:0/2550867134 >> 172.21.15.16:0/2550867134 conn(0x7f114809d760 msgr2=0x7f11480d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.751+0000 7f114ef03640 1 -- 172.21.15.16:0/2550867134 shutdown_connections 2024-09-19T07:55:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.751+0000 7f114ef03640 1 -- 172.21.15.16:0/2550867134 wait complete. 2024-09-19T07:55:33.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2024-09-19T07:55:33.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.866+0000 7f13c7853640 1 Processor -- start 2024-09-19T07:55:33.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.866+0000 7f13c7853640 1 -- start start 2024-09-19T07:55:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.867+0000 7f13c7853640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13c015e680 0x7f13c00cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.867+0000 7f13c7853640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f13c00d0010 0x7f13c00d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.867+0000 7f13c7853640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c00d1880 con 0x7f13c015e2a0 2024-09-19T07:55:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.867+0000 7f13c7853640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f13c00d1b60 con 0x7f13c015e680 2024-09-19T07:55:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.867+0000 7f13c7853640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f13c00cf2d0 con 0x7f13c00d0010 2024-09-19T07:55:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.867+0000 7f13c4dee640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13c015e680 0x7f13c00cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.867+0000 7f13c4dee640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13c015e680 0x7f13c00cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49976/0 (socket says 172.21.15.16:49976) 2024-09-19T07:55:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.867+0000 7f13c4dee640 1 -- 172.21.15.16:0/2212593205 learned_addr learned my addr 172.21.15.16:0/2212593205 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.867+0000 7f13c5df0640 1 --2- 172.21.15.16:0/2212593205 >> v2:172.21.15.146:3300/0 conn(0x7f13c00d0010 0x7f13c00d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.867+0000 7f13c4dee640 1 -- 172.21.15.16:0/2212593205 >> v2:172.21.15.146:3300/0 conn(0x7f13c00d0010 msgr2=0x7f13c00d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.868+0000 7f13c4dee640 1 --2- 172.21.15.16:0/2212593205 >> v2:172.21.15.146:3300/0 conn(0x7f13c00d0010 0x7f13c00d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.868+0000 7f13c4dee640 1 -- 172.21.15.16:0/2212593205 >> v1:172.21.15.146:6789/0 conn(0x7f13c015e2a0 legacy=0x7f13c006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:33.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.868+0000 7f13c4dee640 1 -- 172.21.15.16:0/2212593205 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c00cf5b0 con 0x7f13c015e680 2024-09-19T07:55:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.868+0000 7f13c4dee640 1 --2- 172.21.15.16:0/2212593205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c015e680 0x7f13c00cfad0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f13c0163050 tx=0x7f13b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988882c07260b526 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.868+0000 7f13b7fff640 1 -- 172.21.15.16:0/2212593205 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0037070 con 0x7f13c015e680 2024-09-19T07:55:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.868+0000 7f13b7fff640 1 -- 172.21.15.16:0/2212593205 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b0032440 con 0x7f13c015e680 2024-09-19T07:55:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.868+0000 7f13b7fff640 1 -- 172.21.15.16:0/2212593205 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0032810 con 0x7f13c015e680 2024-09-19T07:55:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.869+0000 7f13c7853640 1 -- 172.21.15.16:0/2212593205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c015e680 msgr2=0x7f13c00cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.869+0000 7f13c7853640 1 --2- 172.21.15.16:0/2212593205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c015e680 0x7f13c00cfad0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f13c0163050 tx=0x7f13b002a1e0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.869+0000 7f13c7853640 1 -- 172.21.15.16:0/2212593205 shutdown_connections 2024-09-19T07:55:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.869+0000 7f13c7853640 1 --2- 172.21.15.16:0/2212593205 >> v2:172.21.15.146:3300/0 conn(0x7f13c00d0010 0x7f13c00d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.869+0000 7f13c7853640 1 --2- 172.21.15.16:0/2212593205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c015e680 0x7f13c00cfad0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.869+0000 7f13c7853640 1 -- 172.21.15.16:0/2212593205 >> 172.21.15.16:0/2212593205 conn(0x7f13c009d760 msgr2=0x7f13c008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.869+0000 7f13c7853640 1 -- 172.21.15.16:0/2212593205 shutdown_connections 2024-09-19T07:55:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.869+0000 7f13c7853640 1 -- 172.21.15.16:0/2212593205 wait complete. 2024-09-19T07:55:33.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.870+0000 7f13c7853640 1 Processor -- start 2024-09-19T07:55:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.870+0000 7f13c7853640 1 -- start start 2024-09-19T07:55:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.871+0000 7f13c7853640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c00d0010 0x7f13c0155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.871+0000 7f13c7853640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13c015e2a0 0x7f13c00b95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.871+0000 7f13c7853640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13c015e680 0x7f13c00b9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.871+0000 7f13c7853640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13c006dcb0 con 0x7f13c00d0010 2024-09-19T07:55:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.871+0000 7f13c7853640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f13c006db30 con 0x7f13c015e2a0 2024-09-19T07:55:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.871+0000 7f13c7853640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f13c00d48b0 con 0x7f13c015e680 2024-09-19T07:55:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.871+0000 7f13c55ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c00d0010 0x7f13c0155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.871+0000 7f13c55ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c00d0010 0x7f13c0155a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49988/0 (socket says 172.21.15.16:49988) 2024-09-19T07:55:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.871+0000 7f13c55ef640 1 -- 172.21.15.16:0/2135700370 learned_addr learned my addr 172.21.15.16:0/2135700370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.871+0000 7f13c5df0640 1 --2- 172.21.15.16:0/2135700370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13c015e680 0x7f13c00b9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.871+0000 7f13c4dee640 1 --2- 172.21.15.16:0/2135700370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13c015e2a0 0x7f13c00b95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.872+0000 7f13c55ef640 1 -- 172.21.15.16:0/2135700370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13c015e680 msgr2=0x7f13c00b9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:33.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.872+0000 7f13c55ef640 1 --2- 172.21.15.16:0/2135700370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13c015e680 0x7f13c00b9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.872+0000 7f13c55ef640 1 -- 172.21.15.16:0/2135700370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13c015e2a0 msgr2=0x7f13c00b95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:33.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.872+0000 7f13c55ef640 1 --2- 172.21.15.16:0/2135700370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13c015e2a0 0x7f13c00b95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:33.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.872+0000 7f13c55ef640 1 -- 172.21.15.16:0/2135700370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c0155f60 con 0x7f13c00d0010 2024-09-19T07:55:33.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.872+0000 7f13c55ef640 1 --2- 172.21.15.16:0/2135700370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c00d0010 0x7f13c0155a20 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00c400 tx=0x7f13bc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:33.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.872+0000 7f13b5ffb640 1 -- 172.21.15.16:0/2135700370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc0052d0 con 0x7f13c00d0010 2024-09-19T07:55:33.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.872+0000 7f13b5ffb640 1 -- 172.21.15.16:0/2135700370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13bc005490 con 0x7f13c00d0010 2024-09-19T07:55:33.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.872+0000 7f13c7853640 1 -- 172.21.15.16:0/2135700370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c00cb5e0 con 0x7f13c00d0010 2024-09-19T07:55:33.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.872+0000 7f13b5ffb640 1 -- 172.21.15.16:0/2135700370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc007260 con 0x7f13c00d0010 2024-09-19T07:55:33.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.872+0000 7f13c7853640 1 -- 172.21.15.16:0/2135700370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c00cbb60 con 0x7f13c00d0010 2024-09-19T07:55:33.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.873+0000 7f13c7853640 1 -- 172.21.15.16:0/2135700370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c00d4c70 con 0x7f13c00d0010 2024-09-19T07:55:33.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.874+0000 7f13b5ffb640 1 -- 172.21.15.16:0/2135700370 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f13bc007400 con 0x7f13c00d0010 2024-09-19T07:55:33.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.874+0000 7f13b5ffb640 1 --2- 172.21.15.16:0/2135700370 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f13a406c2b0 0x7f13a406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:33.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.874+0000 7f13b5ffb640 1 -- 172.21.15.16:0/2135700370 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5335+0+0 (secure 0 0 0) 0x7f13bc08a710 con 0x7f13c00d0010 2024-09-19T07:55:33.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.874+0000 7f13c4dee640 1 --2- 172.21.15.16:0/2135700370 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f13a406c2b0 0x7f13a406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:33.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.875+0000 7f13c4dee640 1 --2- 172.21.15.16:0/2135700370 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f13a406c2b0 0x7f13a406e770 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f13b0034000 tx=0x7f13b002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:33.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:33.878+0000 7f13b5ffb640 1 -- 172.21.15.16:0/2135700370 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f13c0155f60 con 0x7f13c00d0010 2024-09-19T07:55:34.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.055+0000 7f13c7853640 1 -- 172.21.15.16:0/2135700370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) -- 0x7f13c006bc10 con 0x7f13c00d0010 2024-09-19T07:55:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.058+0000 7f13b5ffb640 1 -- 172.21.15.16:0/2135700370 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v72) ==== 105+0+0 (secure 0 0 0) 0x7f13bc058a30 con 0x7f13c00d0010 2024-09-19T07:55:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.061+0000 7f13c7853640 1 -- 172.21.15.16:0/2135700370 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f13a406c2b0 msgr2=0x7f13a406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.061+0000 7f13c7853640 1 --2- 172.21.15.16:0/2135700370 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f13a406c2b0 0x7f13a406e770 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f13b0034000 tx=0x7f13b002e000 comp rx=0 tx=0).stop 2024-09-19T07:55:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.061+0000 7f13c7853640 1 -- 172.21.15.16:0/2135700370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c00d0010 msgr2=0x7f13c0155a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.061+0000 7f13c7853640 1 --2- 172.21.15.16:0/2135700370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c00d0010 0x7f13c0155a20 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00c400 tx=0x7f13bc00c8c0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.061+0000 7f13c7853640 1 -- 172.21.15.16:0/2135700370 shutdown_connections 2024-09-19T07:55:34.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.061+0000 7f13c7853640 1 --2- 172.21.15.16:0/2135700370 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f13a406c2b0 0x7f13a406e770 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.061+0000 7f13c7853640 1 --2- 172.21.15.16:0/2135700370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13c015e680 0x7f13c00b9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.061+0000 7f13c7853640 1 --2- 172.21.15.16:0/2135700370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13c015e2a0 0x7f13c00b95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.061+0000 7f13c7853640 1 --2- 172.21.15.16:0/2135700370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c00d0010 0x7f13c0155a20 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.061+0000 7f13c7853640 1 -- 172.21.15.16:0/2135700370 >> 172.21.15.16:0/2135700370 conn(0x7f13c009d760 msgr2=0x7f13c0160ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:34.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.061+0000 7f13c7853640 1 -- 172.21.15.16:0/2135700370 shutdown_connections 2024-09-19T07:55:34.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.061+0000 7f13c7853640 1 -- 172.21.15.16:0/2135700370 wait complete. 2024-09-19T07:55:34.072 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root' 2024-09-19T07:55:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.178+0000 7f57cebe8640 1 Processor -- start 2024-09-19T07:55:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.178+0000 7f57cebe8640 1 -- start start 2024-09-19T07:55:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.179+0000 7f57cebe8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57c815e680 0x7f57c80cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.179+0000 7f57cebe8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f57c80d0010 0x7f57c80d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.179+0000 7f57cebe8640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57c80d1880 con 0x7f57c815e2a0 2024-09-19T07:55:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.179+0000 7f57cebe8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f57c80d1b60 con 0x7f57c80d0010 2024-09-19T07:55:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.179+0000 7f57cebe8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f57c80cf2d0 con 0x7f57c815e680 2024-09-19T07:55:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.179+0000 7f57cd185640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f57c80d0010 0x7f57c80d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.179+0000 7f57cd185640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f57c80d0010 0x7f57c80d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50004/0 (socket says 172.21.15.16:50004) 2024-09-19T07:55:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.179+0000 7f57cd185640 1 -- 172.21.15.16:0/4106221456 learned_addr learned my addr 172.21.15.16:0/4106221456 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.179+0000 7f57c7fff640 1 --2- 172.21.15.16:0/4106221456 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57c815e680 0x7f57c80cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.179+0000 7f57cd185640 1 -- 172.21.15.16:0/4106221456 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57c815e680 msgr2=0x7f57c80cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.179+0000 7f57cd185640 1 --2- 172.21.15.16:0/4106221456 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57c815e680 0x7f57c80cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.179+0000 7f57cd185640 1 -- 172.21.15.16:0/4106221456 >> v1:172.21.15.146:6789/0 conn(0x7f57c815e2a0 legacy=0x7f57c806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:34.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.179+0000 7f57cd185640 1 -- 172.21.15.16:0/4106221456 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c80cf5b0 con 0x7f57c80d0010 2024-09-19T07:55:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.180+0000 7f57cd185640 1 --2- 172.21.15.16:0/4106221456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c80d0010 0x7f57c80d4370 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009e50 tx=0x7f57c002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bbd50bfa521a0f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.180+0000 7f57c77fe640 1 -- 172.21.15.16:0/4106221456 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0036070 con 0x7f57c80d0010 2024-09-19T07:55:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.180+0000 7f57c77fe640 1 -- 172.21.15.16:0/4106221456 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c0031600 con 0x7f57c80d0010 2024-09-19T07:55:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.180+0000 7f57c77fe640 1 -- 172.21.15.16:0/4106221456 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c00319d0 con 0x7f57c80d0010 2024-09-19T07:55:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.181+0000 7f57cebe8640 1 -- 172.21.15.16:0/4106221456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c80d0010 msgr2=0x7f57c80d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.181+0000 7f57cebe8640 1 --2- 172.21.15.16:0/4106221456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c80d0010 0x7f57c80d4370 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009e50 tx=0x7f57c002a630 comp rx=0 tx=0).stop 2024-09-19T07:55:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.181+0000 7f57cebe8640 1 -- 172.21.15.16:0/4106221456 shutdown_connections 2024-09-19T07:55:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.181+0000 7f57cebe8640 1 --2- 172.21.15.16:0/4106221456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c80d0010 0x7f57c80d4370 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.181+0000 7f57cebe8640 1 --2- 172.21.15.16:0/4106221456 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57c815e680 0x7f57c80cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.181+0000 7f57cebe8640 1 -- 172.21.15.16:0/4106221456 >> 172.21.15.16:0/4106221456 conn(0x7f57c809d760 msgr2=0x7f57c808bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.181+0000 7f57cebe8640 1 -- 172.21.15.16:0/4106221456 shutdown_connections 2024-09-19T07:55:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.181+0000 7f57cebe8640 1 -- 172.21.15.16:0/4106221456 wait complete. 2024-09-19T07:55:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.182+0000 7f57cebe8640 1 Processor -- start 2024-09-19T07:55:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.182+0000 7f57cebe8640 1 -- start start 2024-09-19T07:55:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.183+0000 7f57cebe8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c80d0010 0x7f57c8160970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.183+0000 7f57cebe8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c815e2a0 0x7f57c8160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.183+0000 7f57cebe8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57c815e680 0x7f57c814c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.183+0000 7f57cebe8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57c806dcb0 con 0x7f57c80d0010 2024-09-19T07:55:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.183+0000 7f57cebe8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f57c806db30 con 0x7f57c815e2a0 2024-09-19T07:55:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.183+0000 7f57cebe8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f57c80d48b0 con 0x7f57c815e680 2024-09-19T07:55:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.183+0000 7f57cc984640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c80d0010 0x7f57c8160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.183+0000 7f57cc984640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c80d0010 0x7f57c8160970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50018/0 (socket says 172.21.15.16:50018) 2024-09-19T07:55:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.183+0000 7f57cc984640 1 -- 172.21.15.16:0/2524052001 learned_addr learned my addr 172.21.15.16:0/2524052001 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.183+0000 7f57cd185640 1 --2- 172.21.15.16:0/2524052001 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57c815e680 0x7f57c814c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.183+0000 7f57c7fff640 1 --2- 172.21.15.16:0/2524052001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c815e2a0 0x7f57c8160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.183+0000 7f57cc984640 1 -- 172.21.15.16:0/2524052001 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57c815e680 msgr2=0x7f57c814c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.183+0000 7f57cc984640 1 --2- 172.21.15.16:0/2524052001 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57c815e680 0x7f57c814c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.183+0000 7f57cc984640 1 -- 172.21.15.16:0/2524052001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c815e2a0 msgr2=0x7f57c8160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.183+0000 7f57cc984640 1 --2- 172.21.15.16:0/2524052001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c815e2a0 0x7f57c8160eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.183+0000 7f57cc984640 1 -- 172.21.15.16:0/2524052001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c814d120 con 0x7f57c80d0010 2024-09-19T07:55:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.184+0000 7f57cc984640 1 --2- 172.21.15.16:0/2524052001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c80d0010 0x7f57c8160970 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00c430 tx=0x7f57bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.184+0000 7f57c57fa640 1 -- 172.21.15.16:0/2524052001 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc011790 con 0x7f57c80d0010 2024-09-19T07:55:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.184+0000 7f57c57fa640 1 -- 172.21.15.16:0/2524052001 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57bc017070 con 0x7f57c80d0010 2024-09-19T07:55:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.184+0000 7f57cebe8640 1 -- 172.21.15.16:0/2524052001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c81f4720 con 0x7f57c80d0010 2024-09-19T07:55:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.184+0000 7f57c57fa640 1 -- 172.21.15.16:0/2524052001 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc011bb0 con 0x7f57c80d0010 2024-09-19T07:55:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.184+0000 7f57cebe8640 1 -- 172.21.15.16:0/2524052001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c81f4ce0 con 0x7f57c80d0010 2024-09-19T07:55:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.185+0000 7f57cebe8640 1 -- 172.21.15.16:0/2524052001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57c80d4370 con 0x7f57c80d0010 2024-09-19T07:55:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.185+0000 7f57c57fa640 1 -- 172.21.15.16:0/2524052001 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f57bc01c460 con 0x7f57c80d0010 2024-09-19T07:55:34.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.186+0000 7f57c57fa640 1 --2- 172.21.15.16:0/2524052001 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f57ac06c2b0 0x7f57ac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.186+0000 7f57c57fa640 1 -- 172.21.15.16:0/2524052001 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5335+0+0 (secure 0 0 0) 0x7f57bc08b1d0 con 0x7f57c80d0010 2024-09-19T07:55:34.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.186+0000 7f57c7fff640 1 --2- 172.21.15.16:0/2524052001 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f57ac06c2b0 0x7f57ac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.187+0000 7f57c7fff640 1 --2- 172.21.15.16:0/2524052001 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f57ac06c2b0 0x7f57ac06e770 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f57b4006fd0 tx=0x7f57b4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:34.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.190+0000 7f57c57fa640 1 -- 172.21.15.16:0/2524052001 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f57bc059aa0 con 0x7f57c80d0010 2024-09-19T07:55:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.434+0000 7f57cebe8640 1 -- 172.21.15.16:0/2524052001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]} v 0) -- 0x7f57c815de10 con 0x7f57c80d0010 2024-09-19T07:55:34.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.501+0000 7f57c57fa640 1 -- 172.21.15.16:0/2524052001 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]}]=0 v567) ==== 148+0+0 (secure 0 0 0) 0x7f57bc0593a0 con 0x7f57c80d0010 2024-09-19T07:55:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.503+0000 7f57cebe8640 1 -- 172.21.15.16:0/2524052001 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f57ac06c2b0 msgr2=0x7f57ac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.503+0000 7f57cebe8640 1 --2- 172.21.15.16:0/2524052001 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f57ac06c2b0 0x7f57ac06e770 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f57b4006fd0 tx=0x7f57b4008040 comp rx=0 tx=0).stop 2024-09-19T07:55:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.503+0000 7f57cebe8640 1 -- 172.21.15.16:0/2524052001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c80d0010 msgr2=0x7f57c8160970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.503+0000 7f57cebe8640 1 --2- 172.21.15.16:0/2524052001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c80d0010 0x7f57c8160970 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00c430 tx=0x7f57bc00c8f0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.503+0000 7f57cebe8640 1 -- 172.21.15.16:0/2524052001 shutdown_connections 2024-09-19T07:55:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.503+0000 7f57cebe8640 1 --2- 172.21.15.16:0/2524052001 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f57ac06c2b0 0x7f57ac06e770 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.503+0000 7f57cebe8640 1 --2- 172.21.15.16:0/2524052001 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57c815e680 0x7f57c814c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.503+0000 7f57cebe8640 1 --2- 172.21.15.16:0/2524052001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c815e2a0 0x7f57c8160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.503+0000 7f57cebe8640 1 --2- 172.21.15.16:0/2524052001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c80d0010 0x7f57c8160970 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.503+0000 7f57cebe8640 1 -- 172.21.15.16:0/2524052001 >> 172.21.15.16:0/2524052001 conn(0x7f57c809d760 msgr2=0x7f57c806d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:34.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.503+0000 7f57cebe8640 1 -- 172.21.15.16:0/2524052001 shutdown_connections 2024-09-19T07:55:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.503+0000 7f57cebe8640 1 -- 172.21.15.16:0/2524052001 wait complete. 2024-09-19T07:55:34.515 INFO:tasks.cephfs_test_runner:test_client_pin_root (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2024-09-19T07:55:34.515 INFO:tasks.cephfs_test_runner:Starting test: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) When a client has a bug (which we will simulate) preventing it from releasing caps, 2024-09-19T07:55:34.515 INFO:tasks.cephfs_test_runner:test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-19T07:55:34.516 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:55:34.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.618+0000 7fda794d7640 1 Processor -- start 2024-09-19T07:55:34.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.619+0000 7fda794d7640 1 -- start start 2024-09-19T07:55:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.619+0000 7fda794d7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fda7415f0d0 0x7fda7415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.619+0000 7fda794d7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda740d0070 0x7fda740d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.619+0000 7fda794d7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda740cf2b0 con 0x7fda740d1940 2024-09-19T07:55:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.619+0000 7fda794d7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fda740cf590 con 0x7fda7415f0d0 2024-09-19T07:55:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.619+0000 7fda794d7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fda740cf870 con 0x7fda740d0070 2024-09-19T07:55:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.619+0000 7fda737fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fda740d1940 0x7fda740d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52780/0 (socket says 172.21.15.16:52780) 2024-09-19T07:55:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.619+0000 7fda737fe640 1 -- 172.21.15.16:0/1560112346 learned_addr learned my addr 172.21.15.16:0/1560112346 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda71ffb640 1 -- 172.21.15.16:0/1560112346 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1384193844 0 0) 0x7fda740cf2b0 con 0x7fda740d1940 2024-09-19T07:55:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda727fc640 1 --2- 172.21.15.16:0/1560112346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda740d0070 0x7fda740d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda71ffb640 1 -- 172.21.15.16:0/1560112346 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda60003520 con 0x7fda740d1940 2024-09-19T07:55:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda72ffd640 1 --2- 172.21.15.16:0/1560112346 >> v2:172.21.15.146:3300/0 conn(0x7fda7415f0d0 0x7fda7415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda71ffb640 1 -- 172.21.15.16:0/1560112346 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2283262922 0 0) 0x7fda60003520 con 0x7fda740d1940 2024-09-19T07:55:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda71ffb640 1 -- 172.21.15.16:0/1560112346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda740d0070 msgr2=0x7fda740d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda71ffb640 1 --2- 172.21.15.16:0/1560112346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda740d0070 0x7fda740d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda71ffb640 1 -- 172.21.15.16:0/1560112346 >> v2:172.21.15.146:3300/0 conn(0x7fda7415f0d0 msgr2=0x7fda7415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda71ffb640 1 --2- 172.21.15.16:0/1560112346 >> v2:172.21.15.146:3300/0 conn(0x7fda7415f0d0 0x7fda7415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda71ffb640 1 -- 172.21.15.16:0/1560112346 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda740cfb50 con 0x7fda740d1940 2024-09-19T07:55:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda71ffb640 1 -- 172.21.15.16:0/1560112346 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fda64001a10 con 0x7fda740d1940 2024-09-19T07:55:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda71ffb640 1 -- 172.21.15.16:0/1560112346 >> v1:172.21.15.16:6789/0 conn(0x7fda740d1940 legacy=0x7fda740d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda71ffb640 1 --2- 172.21.15.16:0/1560112346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda600039b0 0x7fda60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda71ffb640 1 --2- 172.21.15.16:0/1560112346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda740d0070 0x7fda60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda71ffb640 1 --2- 172.21.15.16:0/1560112346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda60004d40 0x7fda600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda71ffb640 1 -- 172.21.15.16:0/1560112346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda740cf590 con 0x7fda600039b0 2024-09-19T07:55:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda71ffb640 1 -- 172.21.15.16:0/1560112346 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fda740cf870 con 0x7fda60004d40 2024-09-19T07:55:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda71ffb640 1 -- 172.21.15.16:0/1560112346 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fda6000b9f0 con 0x7fda740d0070 2024-09-19T07:55:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda72ffd640 1 --2- 172.21.15.16:0/1560112346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda600039b0 0x7fda60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda727fc640 1 --2- 172.21.15.16:0/1560112346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda740d0070 0x7fda60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda737fe640 1 --2- 172.21.15.16:0/1560112346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda60004d40 0x7fda600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda72ffd640 1 -- 172.21.15.16:0/1560112346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda740d0070 msgr2=0x7fda60004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda72ffd640 1 --2- 172.21.15.16:0/1560112346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda740d0070 0x7fda60004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda72ffd640 1 -- 172.21.15.16:0/1560112346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda60004d40 msgr2=0x7fda600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda72ffd640 1 --2- 172.21.15.16:0/1560112346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda60004d40 0x7fda600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.620+0000 7fda72ffd640 1 -- 172.21.15.16:0/1560112346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda60003520 con 0x7fda600039b0 2024-09-19T07:55:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.621+0000 7fda72ffd640 1 --2- 172.21.15.16:0/1560112346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda600039b0 0x7fda60003dd0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009ac0 tx=0x7fda60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602ef7d4d6670bd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.621+0000 7fda71ffb640 1 -- 172.21.15.16:0/1560112346 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c036070 con 0x7fda600039b0 2024-09-19T07:55:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.621+0000 7fda71ffb640 1 -- 172.21.15.16:0/1560112346 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda6c031400 con 0x7fda600039b0 2024-09-19T07:55:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.621+0000 7fda71ffb640 1 -- 172.21.15.16:0/1560112346 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c009690 con 0x7fda600039b0 2024-09-19T07:55:34.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.622+0000 7fda794d7640 1 -- 172.21.15.16:0/1560112346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda600039b0 msgr2=0x7fda60003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:34.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.622+0000 7fda794d7640 1 --2- 172.21.15.16:0/1560112346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda600039b0 0x7fda60003dd0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009ac0 tx=0x7fda60005830 comp rx=0 tx=0).stop 2024-09-19T07:55:34.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.622+0000 7fda737fe640 1 -- 172.21.15.16:0/1560112346 reap_dead start 2024-09-19T07:55:34.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.622+0000 7fda794d7640 1 -- 172.21.15.16:0/1560112346 shutdown_connections 2024-09-19T07:55:34.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.622+0000 7fda794d7640 1 -- 172.21.15.16:0/1560112346 >> 172.21.15.16:0/1560112346 conn(0x7fda7409d760 msgr2=0x7fda7408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:34.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.622+0000 7fda794d7640 1 -- 172.21.15.16:0/1560112346 shutdown_connections 2024-09-19T07:55:34.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.622+0000 7fda794d7640 1 -- 172.21.15.16:0/1560112346 wait complete. 2024-09-19T07:55:34.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.623+0000 7fda794d7640 1 Processor -- start 2024-09-19T07:55:34.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.623+0000 7fda794d7640 1 -- start start 2024-09-19T07:55:34.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.624+0000 7fda794d7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda600039b0 0x7fda74155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.624+0000 7fda794d7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda74154aa0 0x7fda740b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.624+0000 7fda794d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda740b7750 0x7fda740b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.624+0000 7fda794d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda740d49c0 con 0x7fda740b7750 2024-09-19T07:55:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.624+0000 7fda794d7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fda740d4840 con 0x7fda600039b0 2024-09-19T07:55:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.624+0000 7fda794d7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fda740d4b40 con 0x7fda74154aa0 2024-09-19T07:55:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.624+0000 7fda737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda740b7750 0x7fda740b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.624+0000 7fda737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda740b7750 0x7fda740b7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50038/0 (socket says 172.21.15.16:50038) 2024-09-19T07:55:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.624+0000 7fda737fe640 1 -- 172.21.15.16:0/3276414365 learned_addr learned my addr 172.21.15.16:0/3276414365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.624+0000 7fda72ffd640 1 --2- 172.21.15.16:0/3276414365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda600039b0 0x7fda74155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.624+0000 7fda727fc640 1 --2- 172.21.15.16:0/3276414365 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda74154aa0 0x7fda740b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.624+0000 7fda737fe640 1 -- 172.21.15.16:0/3276414365 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda74154aa0 msgr2=0x7fda740b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.624+0000 7fda737fe640 1 --2- 172.21.15.16:0/3276414365 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda74154aa0 0x7fda740b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.624+0000 7fda737fe640 1 -- 172.21.15.16:0/3276414365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda600039b0 msgr2=0x7fda74155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.624+0000 7fda737fe640 1 --2- 172.21.15.16:0/3276414365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda600039b0 0x7fda74155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.624+0000 7fda737fe640 1 -- 172.21.15.16:0/3276414365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda740cc730 con 0x7fda740b7750 2024-09-19T07:55:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.625+0000 7fda737fe640 1 --2- 172.21.15.16:0/3276414365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda740b7750 0x7fda740b7c90 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fda64002230 tx=0x7fda6400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:34.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.625+0000 7fda6b7fe640 1 -- 172.21.15.16:0/3276414365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6400edc0 con 0x7fda740b7750 2024-09-19T07:55:34.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.625+0000 7fda6b7fe640 1 -- 172.21.15.16:0/3276414365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda64014430 con 0x7fda740b7750 2024-09-19T07:55:34.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.625+0000 7fda6b7fe640 1 -- 172.21.15.16:0/3276414365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda64014820 con 0x7fda740b7750 2024-09-19T07:55:34.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.625+0000 7fda794d7640 1 -- 172.21.15.16:0/3276414365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda740cca10 con 0x7fda740b7750 2024-09-19T07:55:34.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.625+0000 7fda794d7640 1 -- 172.21.15.16:0/3276414365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda740ccfd0 con 0x7fda740b7750 2024-09-19T07:55:34.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.626+0000 7fda794d7640 1 -- 172.21.15.16:0/3276414365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda7415f4b0 con 0x7fda740b7750 2024-09-19T07:55:34.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.627+0000 7fda6b7fe640 1 -- 172.21.15.16:0/3276414365 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fda6401b060 con 0x7fda740b7750 2024-09-19T07:55:34.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.627+0000 7fda6b7fe640 1 --2- 172.21.15.16:0/3276414365 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fda5406c2b0 0x7fda5406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.627+0000 7fda6b7fe640 1 -- 172.21.15.16:0/3276414365 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5335+0+0 (secure 0 0 0) 0x7fda6408abc0 con 0x7fda740b7750 2024-09-19T07:55:34.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.627+0000 7fda72ffd640 1 --2- 172.21.15.16:0/3276414365 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fda5406c2b0 0x7fda5406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.628+0000 7fda72ffd640 1 --2- 172.21.15.16:0/3276414365 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fda5406c2b0 0x7fda5406e770 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fda6c033000 tx=0x7fda6c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:34.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.632+0000 7fda6b7fe640 1 -- 172.21.15.16:0/3276414365 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fda64058ee0 con 0x7fda740b7750 2024-09-19T07:55:34.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.807+0000 7fda794d7640 1 -- 172.21.15.16:0/3276414365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda7406cf30 con 0x7fda740b7750 2024-09-19T07:55:34.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.808+0000 7fda6b7fe640 1 -- 172.21.15.16:0/3276414365 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) ==== 74+0+13339 (secure 0 0 0) 0x7fda6405d1e0 con 0x7fda740b7750 2024-09-19T07:55:34.809 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:34.809 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":143,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:55:32.494748+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/1929965275":"2024-09-20T07:55:29.486855+0000","172.21.15.16:6833/1929965275":"2024-09-20T07:55:29.486855+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:55:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.810+0000 7fda794d7640 1 -- 172.21.15.16:0/3276414365 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fda5406c2b0 msgr2=0x7fda5406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.810+0000 7fda794d7640 1 --2- 172.21.15.16:0/3276414365 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fda5406c2b0 0x7fda5406e770 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fda6c033000 tx=0x7fda6c008030 comp rx=0 tx=0).stop 2024-09-19T07:55:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.810+0000 7fda794d7640 1 -- 172.21.15.16:0/3276414365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda740b7750 msgr2=0x7fda740b7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.810+0000 7fda794d7640 1 --2- 172.21.15.16:0/3276414365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda740b7750 0x7fda740b7c90 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fda64002230 tx=0x7fda6400e530 comp rx=0 tx=0).stop 2024-09-19T07:55:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.811+0000 7fda794d7640 1 -- 172.21.15.16:0/3276414365 shutdown_connections 2024-09-19T07:55:34.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.811+0000 7fda794d7640 1 --2- 172.21.15.16:0/3276414365 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fda5406c2b0 0x7fda5406e770 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.811+0000 7fda794d7640 1 --2- 172.21.15.16:0/3276414365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda740b7750 0x7fda740b7c90 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.811+0000 7fda794d7640 1 --2- 172.21.15.16:0/3276414365 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda74154aa0 0x7fda740b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.811+0000 7fda794d7640 1 --2- 172.21.15.16:0/3276414365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda600039b0 0x7fda74155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.811+0000 7fda794d7640 1 -- 172.21.15.16:0/3276414365 >> 172.21.15.16:0/3276414365 conn(0x7fda7409d760 msgr2=0x7fda7415fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.811+0000 7fda794d7640 1 -- 172.21.15.16:0/3276414365 shutdown_connections 2024-09-19T07:55:34.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.811+0000 7fda794d7640 1 -- 172.21.15.16:0/3276414365 wait complete. 2024-09-19T07:55:34.822 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T07:42:54.500338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T07:55:34.822 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T07:55:34.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.927+0000 7f6042caf640 1 Processor -- start 2024-09-19T07:55:34.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.927+0000 7f6042caf640 1 -- start start 2024-09-19T07:55:34.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.928+0000 7f6042caf640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f603c15f0d0 0x7f603c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.928+0000 7f6042caf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f603c0d0070 0x7f603c0d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.928+0000 7f6042caf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f603c0d1940 0x7f603c0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.928+0000 7f6042caf640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f603c0cf2b0 con 0x7f603c0d0070 2024-09-19T07:55:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.928+0000 7f6042caf640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f603c0cf590 con 0x7f603c15f0d0 2024-09-19T07:55:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.928+0000 7f6042caf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f603c0cf870 con 0x7f603c0d1940 2024-09-19T07:55:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.928+0000 7f603bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f603c0d0070 0x7f603c0d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.928+0000 7f603bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f603c0d0070 0x7f603c0d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50042/0 (socket says 172.21.15.16:50042) 2024-09-19T07:55:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.928+0000 7f603bfff640 1 -- 172.21.15.16:0/3163822222 learned_addr learned my addr 172.21.15.16:0/3163822222 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.929+0000 7f604124c640 1 --2- 172.21.15.16:0/3163822222 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f603c0d1940 0x7f603c0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.929+0000 7f6040a4b640 1 --2- 172.21.15.16:0/3163822222 >> v2:172.21.15.146:3300/0 conn(0x7f603c15f0d0 0x7f603c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.929+0000 7f603bfff640 1 -- 172.21.15.16:0/3163822222 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f603c0d1940 msgr2=0x7f603c0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.929+0000 7f603bfff640 1 --2- 172.21.15.16:0/3163822222 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f603c0d1940 0x7f603c0d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.929+0000 7f603bfff640 1 -- 172.21.15.16:0/3163822222 >> v2:172.21.15.146:3300/0 conn(0x7f603c15f0d0 msgr2=0x7f603c15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.929+0000 7f603bfff640 1 --2- 172.21.15.16:0/3163822222 >> v2:172.21.15.146:3300/0 conn(0x7f603c15f0d0 0x7f603c15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.929+0000 7f603bfff640 1 -- 172.21.15.16:0/3163822222 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603c0cfb50 con 0x7f603c0d0070 2024-09-19T07:55:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.929+0000 7f6040a4b640 1 --2- 172.21.15.16:0/3163822222 >> v2:172.21.15.146:3300/0 conn(0x7f603c15f0d0 0x7f603c15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:34.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.929+0000 7f604124c640 1 --2- 172.21.15.16:0/3163822222 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f603c0d1940 0x7f603c0d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:34.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.929+0000 7f603bfff640 1 --2- 172.21.15.16:0/3163822222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f603c0d0070 0x7f603c0d1400 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f603c0b6f40 tx=0x7f603002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7145361b2c55d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:34.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.930+0000 7f603b7fe640 1 -- 172.21.15.16:0/3163822222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030037070 con 0x7f603c0d0070 2024-09-19T07:55:34.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.930+0000 7f603b7fe640 1 -- 172.21.15.16:0/3163822222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6030032440 con 0x7f603c0d0070 2024-09-19T07:55:34.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.930+0000 7f603b7fe640 1 -- 172.21.15.16:0/3163822222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030032810 con 0x7f603c0d0070 2024-09-19T07:55:34.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.930+0000 7f6042caf640 1 -- 172.21.15.16:0/3163822222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f603c0d0070 msgr2=0x7f603c0d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:34.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.930+0000 7f6042caf640 1 --2- 172.21.15.16:0/3163822222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f603c0d0070 0x7f603c0d1400 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f603c0b6f40 tx=0x7f603002a140 comp rx=0 tx=0).stop 2024-09-19T07:55:34.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.930+0000 7f6042caf640 1 -- 172.21.15.16:0/3163822222 shutdown_connections 2024-09-19T07:55:34.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.930+0000 7f6042caf640 1 --2- 172.21.15.16:0/3163822222 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f603c0d1940 0x7f603c0d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.930+0000 7f6042caf640 1 --2- 172.21.15.16:0/3163822222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f603c0d0070 0x7f603c0d1400 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.930+0000 7f6042caf640 1 --2- 172.21.15.16:0/3163822222 >> v2:172.21.15.146:3300/0 conn(0x7f603c15f0d0 0x7f603c15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.930+0000 7f6042caf640 1 -- 172.21.15.16:0/3163822222 >> 172.21.15.16:0/3163822222 conn(0x7f603c09d760 msgr2=0x7f603c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:34.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.930+0000 7f6042caf640 1 -- 172.21.15.16:0/3163822222 shutdown_connections 2024-09-19T07:55:34.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.930+0000 7f6042caf640 1 -- 172.21.15.16:0/3163822222 wait complete. 2024-09-19T07:55:34.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.931+0000 7f6042caf640 1 Processor -- start 2024-09-19T07:55:34.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.931+0000 7f6042caf640 1 -- start start 2024-09-19T07:55:34.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.932+0000 7f6042caf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f603c0d0070 0x7f603c1f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.932+0000 7f6042caf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f603c0d1940 0x7f603c15a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.932+0000 7f6042caf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f603c15f0d0 0x7f603c15c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.932+0000 7f6042caf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f603c0d47f0 con 0x7f603c0d1940 2024-09-19T07:55:34.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.932+0000 7f6042caf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f603c0d4670 con 0x7f603c15f0d0 2024-09-19T07:55:34.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.932+0000 7f6042caf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f603c0d4970 con 0x7f603c0d0070 2024-09-19T07:55:34.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.932+0000 7f603bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f603c0d1940 0x7f603c15a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.932+0000 7f603bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f603c0d1940 0x7f603c15a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50054/0 (socket says 172.21.15.16:50054) 2024-09-19T07:55:34.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.932+0000 7f603bfff640 1 -- 172.21.15.16:0/2480987807 learned_addr learned my addr 172.21.15.16:0/2480987807 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:34.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.932+0000 7f6040a4b640 1 --2- 172.21.15.16:0/2480987807 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f603c0d0070 0x7f603c1f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.932+0000 7f604124c640 1 --2- 172.21.15.16:0/2480987807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f603c15f0d0 0x7f603c15c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.933+0000 7f603bfff640 1 -- 172.21.15.16:0/2480987807 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f603c0d0070 msgr2=0x7f603c1f4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:34.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.933+0000 7f603bfff640 1 --2- 172.21.15.16:0/2480987807 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f603c0d0070 0x7f603c1f4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.933+0000 7f603bfff640 1 -- 172.21.15.16:0/2480987807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f603c15f0d0 msgr2=0x7f603c15c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.933+0000 7f603bfff640 1 --2- 172.21.15.16:0/2480987807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f603c15f0d0 0x7f603c15c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.933+0000 7f603bfff640 1 -- 172.21.15.16:0/2480987807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603c0b8810 con 0x7f603c0d1940 2024-09-19T07:55:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.933+0000 7f603bfff640 1 --2- 172.21.15.16:0/2480987807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f603c0d1940 0x7f603c15a950 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f603002a180 tx=0x7f60300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.933+0000 7f60397fa640 1 -- 172.21.15.16:0/2480987807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030037040 con 0x7f603c0d1940 2024-09-19T07:55:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.933+0000 7f60397fa640 1 -- 172.21.15.16:0/2480987807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6030042070 con 0x7f603c0d1940 2024-09-19T07:55:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.933+0000 7f6042caf640 1 -- 172.21.15.16:0/2480987807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603c0b8af0 con 0x7f603c0d1940 2024-09-19T07:55:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.933+0000 7f60397fa640 1 -- 172.21.15.16:0/2480987807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030030bb0 con 0x7f603c0d1940 2024-09-19T07:55:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.933+0000 7f6042caf640 1 -- 172.21.15.16:0/2480987807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f603c0b90b0 con 0x7f603c0d1940 2024-09-19T07:55:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.934+0000 7f6042caf640 1 -- 172.21.15.16:0/2480987807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f603c0d1400 con 0x7f603c0d1940 2024-09-19T07:55:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.935+0000 7f60397fa640 1 -- 172.21.15.16:0/2480987807 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6030047460 con 0x7f603c0d1940 2024-09-19T07:55:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.935+0000 7f60397fa640 1 --2- 172.21.15.16:0/2480987807 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f602806c2e0 0x7f602806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:34.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.935+0000 7f60397fa640 1 -- 172.21.15.16:0/2480987807 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5335+0+0 (secure 0 0 0) 0x7f60300abcb0 con 0x7f603c0d1940 2024-09-19T07:55:34.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.935+0000 7f6040a4b640 1 --2- 172.21.15.16:0/2480987807 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f602806c2e0 0x7f602806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:34.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.936+0000 7f6040a4b640 1 --2- 172.21.15.16:0/2480987807 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f602806c2e0 0x7f602806e7a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f602c00ab60 tx=0x7f602c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:34.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:34.939+0000 7f60397fa640 1 -- 172.21.15.16:0/2480987807 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6030079fd0 con 0x7f603c0d1940 2024-09-19T07:55:35.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.115+0000 7f6042caf640 1 -- 172.21.15.16:0/2480987807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f603c06cf30 con 0x7f603c0d1940 2024-09-19T07:55:35.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.115+0000 7f60397fa640 1 -- 172.21.15.16:0/2480987807 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) ==== 93+0+10 (secure 0 0 0) 0x7f603007e2d0 con 0x7f603c0d1940 2024-09-19T07:55:35.116 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-19T07:55:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.117+0000 7f6042caf640 1 -- 172.21.15.16:0/2480987807 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f602806c2e0 msgr2=0x7f602806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.117+0000 7f6042caf640 1 --2- 172.21.15.16:0/2480987807 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f602806c2e0 0x7f602806e7a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f602c00ab60 tx=0x7f602c008040 comp rx=0 tx=0).stop 2024-09-19T07:55:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.117+0000 7f6042caf640 1 -- 172.21.15.16:0/2480987807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f603c0d1940 msgr2=0x7f603c15a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.117+0000 7f6042caf640 1 --2- 172.21.15.16:0/2480987807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f603c0d1940 0x7f603c15a950 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f603002a180 tx=0x7f60300046c0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.117+0000 7f6042caf640 1 -- 172.21.15.16:0/2480987807 shutdown_connections 2024-09-19T07:55:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.117+0000 7f6042caf640 1 --2- 172.21.15.16:0/2480987807 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f602806c2e0 0x7f602806e7a0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.117+0000 7f6042caf640 1 --2- 172.21.15.16:0/2480987807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f603c15f0d0 0x7f603c15c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.117+0000 7f6042caf640 1 --2- 172.21.15.16:0/2480987807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f603c0d1940 0x7f603c15a950 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.117+0000 7f6042caf640 1 --2- 172.21.15.16:0/2480987807 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f603c0d0070 0x7f603c1f4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.117+0000 7f6042caf640 1 -- 172.21.15.16:0/2480987807 >> 172.21.15.16:0/2480987807 conn(0x7f603c09d760 msgr2=0x7f603c161eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.117+0000 7f6042caf640 1 -- 172.21.15.16:0/2480987807 shutdown_connections 2024-09-19T07:55:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.117+0000 7f6042caf640 1 -- 172.21.15.16:0/2480987807 wait complete. 2024-09-19T07:55:35.129 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug' 2024-09-19T07:55:35.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.230+0000 7fd327de4640 1 Processor -- start 2024-09-19T07:55:35.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.230+0000 7fd327de4640 1 -- start start 2024-09-19T07:55:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.231+0000 7fd327de4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd32006db40 0x7fd32006df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.231+0000 7fd327de4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd32015b8b0 0x7fd32015bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.231+0000 7fd327de4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3200cebe0 con 0x7fd320072cb0 2024-09-19T07:55:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.231+0000 7fd327de4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd32006e460 con 0x7fd32015b8b0 2024-09-19T07:55:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.231+0000 7fd327de4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd32013f6e0 con 0x7fd32006db40 2024-09-19T07:55:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.231+0000 7fd32537f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd32015b8b0 0x7fd32015bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.231+0000 7fd326381640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd320072cb0 0x7fd320073140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52790/0 (socket says 172.21.15.16:52790) 2024-09-19T07:55:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.231+0000 7fd326381640 1 -- 172.21.15.16:0/3113853304 learned_addr learned my addr 172.21.15.16:0/3113853304 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.231+0000 7fd325b80640 1 --2- 172.21.15.16:0/3113853304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd32006db40 0x7fd32006df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.231+0000 7fd32537f640 1 -- 172.21.15.16:0/3113853304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd32006db40 msgr2=0x7fd32006df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.231+0000 7fd32537f640 1 --2- 172.21.15.16:0/3113853304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd32006db40 0x7fd32006df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.231+0000 7fd32537f640 1 -- 172.21.15.16:0/3113853304 >> v1:172.21.15.16:6789/0 conn(0x7fd320072cb0 legacy=0x7fd320073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:35.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.231+0000 7fd32537f640 1 -- 172.21.15.16:0/3113853304 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3200cee60 con 0x7fd32015b8b0 2024-09-19T07:55:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.232+0000 7fd32537f640 1 --2- 172.21.15.16:0/3113853304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32015b8b0 0x7fd32015bd10 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fd3140097a0 tx=0x7fd31402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e76688fc5d0ffb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.232+0000 7fd324b7e640 1 -- 172.21.15.16:0/3113853304 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314037070 con 0x7fd32015b8b0 2024-09-19T07:55:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.232+0000 7fd324b7e640 1 -- 172.21.15.16:0/3113853304 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31402ae50 con 0x7fd32015b8b0 2024-09-19T07:55:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.232+0000 7fd324b7e640 1 -- 172.21.15.16:0/3113853304 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314032660 con 0x7fd32015b8b0 2024-09-19T07:55:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.233+0000 7fd327de4640 1 -- 172.21.15.16:0/3113853304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32015b8b0 msgr2=0x7fd32015bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.233+0000 7fd327de4640 1 --2- 172.21.15.16:0/3113853304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32015b8b0 0x7fd32015bd10 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fd3140097a0 tx=0x7fd31402a100 comp rx=0 tx=0).stop 2024-09-19T07:55:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.233+0000 7fd327de4640 1 -- 172.21.15.16:0/3113853304 shutdown_connections 2024-09-19T07:55:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.233+0000 7fd327de4640 1 --2- 172.21.15.16:0/3113853304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32015b8b0 0x7fd32015bd10 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.233+0000 7fd327de4640 1 --2- 172.21.15.16:0/3113853304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd32006db40 0x7fd32006df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.233+0000 7fd327de4640 1 -- 172.21.15.16:0/3113853304 >> 172.21.15.16:0/3113853304 conn(0x7fd32006f6b0 msgr2=0x7fd32006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.233+0000 7fd327de4640 1 -- 172.21.15.16:0/3113853304 shutdown_connections 2024-09-19T07:55:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.233+0000 7fd327de4640 1 -- 172.21.15.16:0/3113853304 wait complete. 2024-09-19T07:55:35.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.234+0000 7fd327de4640 1 Processor -- start 2024-09-19T07:55:35.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.234+0000 7fd327de4640 1 -- start start 2024-09-19T07:55:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.235+0000 7fd327de4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd32006db40 0x7fd32015a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.235+0000 7fd327de4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd320072cb0 0x7fd32015c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.235+0000 7fd327de4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd32015b8b0 0x7fd3200a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.235+0000 7fd327de4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3200cf700 con 0x7fd320072cb0 2024-09-19T07:55:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.235+0000 7fd327de4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd3200cf420 con 0x7fd32015b8b0 2024-09-19T07:55:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.235+0000 7fd327de4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd3200cfc20 con 0x7fd32006db40 2024-09-19T07:55:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.235+0000 7fd32537f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd320072cb0 0x7fd32015c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.235+0000 7fd32537f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd320072cb0 0x7fd32015c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50072/0 (socket says 172.21.15.16:50072) 2024-09-19T07:55:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.235+0000 7fd32537f640 1 -- 172.21.15.16:0/4038415653 learned_addr learned my addr 172.21.15.16:0/4038415653 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.235+0000 7fd325b80640 1 --2- 172.21.15.16:0/4038415653 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd32006db40 0x7fd32015a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.235+0000 7fd326381640 1 --2- 172.21.15.16:0/4038415653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd32015b8b0 0x7fd3200a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.235+0000 7fd32537f640 1 -- 172.21.15.16:0/4038415653 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd32006db40 msgr2=0x7fd32015a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.235+0000 7fd32537f640 1 --2- 172.21.15.16:0/4038415653 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd32006db40 0x7fd32015a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.235+0000 7fd32537f640 1 -- 172.21.15.16:0/4038415653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd32015b8b0 msgr2=0x7fd3200a97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.235+0000 7fd32537f640 1 --2- 172.21.15.16:0/4038415653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd32015b8b0 0x7fd3200a97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.235+0000 7fd32537f640 1 -- 172.21.15.16:0/4038415653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3200b31f0 con 0x7fd320072cb0 2024-09-19T07:55:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.235+0000 7fd32537f640 1 --2- 172.21.15.16:0/4038415653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd320072cb0 0x7fd32015c620 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fd314000c00 tx=0x7fd3140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.236+0000 7fd31e7fc640 1 -- 172.21.15.16:0/4038415653 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314041070 con 0x7fd320072cb0 2024-09-19T07:55:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.236+0000 7fd31e7fc640 1 -- 172.21.15.16:0/4038415653 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31403c670 con 0x7fd320072cb0 2024-09-19T07:55:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.236+0000 7fd327de4640 1 -- 172.21.15.16:0/4038415653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32015d5d0 con 0x7fd320072cb0 2024-09-19T07:55:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.236+0000 7fd31e7fc640 1 -- 172.21.15.16:0/4038415653 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314037040 con 0x7fd320072cb0 2024-09-19T07:55:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.236+0000 7fd327de4640 1 -- 172.21.15.16:0/4038415653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3200b3490 con 0x7fd320072cb0 2024-09-19T07:55:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.237+0000 7fd327de4640 1 -- 172.21.15.16:0/4038415653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd32013e260 con 0x7fd320072cb0 2024-09-19T07:55:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.237+0000 7fd31e7fc640 1 -- 172.21.15.16:0/4038415653 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd314044070 con 0x7fd320072cb0 2024-09-19T07:55:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.238+0000 7fd31e7fc640 1 --2- 172.21.15.16:0/4038415653 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd30c06c2b0 0x7fd30c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.238+0000 7fd31e7fc640 1 -- 172.21.15.16:0/4038415653 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5335+0+0 (secure 0 0 0) 0x7fd3140ab7f0 con 0x7fd320072cb0 2024-09-19T07:55:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.238+0000 7fd325b80640 1 --2- 172.21.15.16:0/4038415653 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd30c06c2b0 0x7fd30c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:35.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.239+0000 7fd325b80640 1 --2- 172.21.15.16:0/4038415653 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd30c06c2b0 0x7fd30c06e770 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd310009a60 tx=0x7fd310003990 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:35.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.242+0000 7fd31e7fc640 1 -- 172.21.15.16:0/4038415653 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd31407a160 con 0x7fd320072cb0 2024-09-19T07:55:35.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.490+0000 7fd327de4640 1 -- 172.21.15.16:0/4038415653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]} v 0) -- 0x7fd320140770 con 0x7fd320072cb0 2024-09-19T07:55:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.505+0000 7fd31e7fc640 1 -- 172.21.15.16:0/4038415653 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]}]=0 v568) ==== 154+0+0 (secure 0 0 0) 0x7fd314079a60 con 0x7fd320072cb0 2024-09-19T07:55:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.508+0000 7fd327de4640 1 -- 172.21.15.16:0/4038415653 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd30c06c2b0 msgr2=0x7fd30c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.508+0000 7fd327de4640 1 --2- 172.21.15.16:0/4038415653 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd30c06c2b0 0x7fd30c06e770 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd310009a60 tx=0x7fd310003990 comp rx=0 tx=0).stop 2024-09-19T07:55:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.508+0000 7fd327de4640 1 -- 172.21.15.16:0/4038415653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd320072cb0 msgr2=0x7fd32015c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.508+0000 7fd327de4640 1 --2- 172.21.15.16:0/4038415653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd320072cb0 0x7fd32015c620 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fd314000c00 tx=0x7fd3140046c0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.508+0000 7fd327de4640 1 -- 172.21.15.16:0/4038415653 shutdown_connections 2024-09-19T07:55:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.508+0000 7fd327de4640 1 --2- 172.21.15.16:0/4038415653 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd30c06c2b0 0x7fd30c06e770 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.508+0000 7fd327de4640 1 --2- 172.21.15.16:0/4038415653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd32015b8b0 0x7fd3200a97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.508+0000 7fd327de4640 1 --2- 172.21.15.16:0/4038415653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd320072cb0 0x7fd32015c620 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.508+0000 7fd327de4640 1 --2- 172.21.15.16:0/4038415653 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd32006db40 0x7fd32015a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.508+0000 7fd327de4640 1 -- 172.21.15.16:0/4038415653 >> 172.21.15.16:0/4038415653 conn(0x7fd32006f6b0 msgr2=0x7fd3200b11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.508+0000 7fd327de4640 1 -- 172.21.15.16:0/4038415653 shutdown_connections 2024-09-19T07:55:35.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.508+0000 7fd327de4640 1 -- 172.21.15.16:0/4038415653 wait complete. 2024-09-19T07:55:35.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T07:55:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.623+0000 7fcd02b03640 1 Processor -- start 2024-09-19T07:55:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.623+0000 7fcd02b03640 1 -- start start 2024-09-19T07:55:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.624+0000 7fcd02b03640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fccfc15e2a0 0x7fccfc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.624+0000 7fcd02b03640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fccfc15e680 0x7fccfc0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.624+0000 7fcd02b03640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccfc0d1880 con 0x7fccfc06d960 2024-09-19T07:55:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.624+0000 7fcd02b03640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fccfc0d1b60 con 0x7fccfc15e680 2024-09-19T07:55:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.624+0000 7fcd02b03640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fccfc0cf2d0 con 0x7fccfc15e2a0 2024-09-19T07:55:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.624+0000 7fccfbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fccfc15e680 0x7fccfc0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.624+0000 7fccfbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fccfc15e680 0x7fccfc0cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50078/0 (socket says 172.21.15.16:50078) 2024-09-19T07:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.624+0000 7fccfbfff640 1 -- 172.21.15.16:0/2660497085 learned_addr learned my addr 172.21.15.16:0/2660497085 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.624+0000 7fcd0089f640 1 --2- 172.21.15.16:0/2660497085 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fccfc15e2a0 0x7fccfc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.625+0000 7fccfbfff640 1 -- 172.21.15.16:0/2660497085 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fccfc15e2a0 msgr2=0x7fccfc06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.625+0000 7fccfbfff640 1 --2- 172.21.15.16:0/2660497085 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fccfc15e2a0 0x7fccfc06d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.625+0000 7fccfbfff640 1 -- 172.21.15.16:0/2660497085 >> v1:172.21.15.146:6789/0 conn(0x7fccfc06d960 legacy=0x7fccfc0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.625+0000 7fccfbfff640 1 -- 172.21.15.16:0/2660497085 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccfc0cf5b0 con 0x7fccfc15e680 2024-09-19T07:55:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.625+0000 7fccfbfff640 1 --2- 172.21.15.16:0/2660497085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccfc15e680 0x7fccfc0cfad0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009870 tx=0x7fccf002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24baa4c320441257 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.626+0000 7fccfb7fe640 1 -- 172.21.15.16:0/2660497085 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf0037070 con 0x7fccfc15e680 2024-09-19T07:55:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.626+0000 7fccfb7fe640 1 -- 172.21.15.16:0/2660497085 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf0032440 con 0x7fccfc15e680 2024-09-19T07:55:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.626+0000 7fccfb7fe640 1 -- 172.21.15.16:0/2660497085 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf0032810 con 0x7fccfc15e680 2024-09-19T07:55:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.626+0000 7fcd02b03640 1 -- 172.21.15.16:0/2660497085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccfc15e680 msgr2=0x7fccfc0cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.626+0000 7fcd02b03640 1 --2- 172.21.15.16:0/2660497085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccfc15e680 0x7fccfc0cfad0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009870 tx=0x7fccf002a270 comp rx=0 tx=0).stop 2024-09-19T07:55:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.626+0000 7fcd02b03640 1 -- 172.21.15.16:0/2660497085 shutdown_connections 2024-09-19T07:55:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.626+0000 7fcd02b03640 1 --2- 172.21.15.16:0/2660497085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccfc15e680 0x7fccfc0cfad0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.626+0000 7fcd02b03640 1 --2- 172.21.15.16:0/2660497085 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fccfc15e2a0 0x7fccfc06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.626+0000 7fcd02b03640 1 -- 172.21.15.16:0/2660497085 >> 172.21.15.16:0/2660497085 conn(0x7fccfc09d760 msgr2=0x7fccfc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.626+0000 7fcd02b03640 1 -- 172.21.15.16:0/2660497085 shutdown_connections 2024-09-19T07:55:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.626+0000 7fcd02b03640 1 -- 172.21.15.16:0/2660497085 wait complete. 2024-09-19T07:55:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.627+0000 7fcd02b03640 1 Processor -- start 2024-09-19T07:55:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.627+0000 7fcd02b03640 1 -- start start 2024-09-19T07:55:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.628+0000 7fcd02b03640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fccfc06d960 0x7fccfc0cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.628+0000 7fcd02b03640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fccfc15e2a0 0x7fccfc0b8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.628+0000 7fcd02b03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccfc15e680 0x7fccfc0cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.628+0000 7fcd02b03640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc0d4c00 con 0x7fccfc15e680 2024-09-19T07:55:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.628+0000 7fcd02b03640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc0d4a80 con 0x7fccfc06d960 2024-09-19T07:55:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.628+0000 7fcd02b03640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fccfc0d0010 con 0x7fccfc15e2a0 2024-09-19T07:55:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.628+0000 7fcd010a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccfc15e680 0x7fccfc0cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.628+0000 7fcd010a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccfc15e680 0x7fccfc0cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50082/0 (socket says 172.21.15.16:50082) 2024-09-19T07:55:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.628+0000 7fcd010a0640 1 -- 172.21.15.16:0/3862707100 learned_addr learned my addr 172.21.15.16:0/3862707100 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.629+0000 7fccfbfff640 1 --2- 172.21.15.16:0/3862707100 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fccfc15e2a0 0x7fccfc0b8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.629+0000 7fcd0089f640 1 --2- 172.21.15.16:0/3862707100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fccfc06d960 0x7fccfc0cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.629+0000 7fcd010a0640 1 -- 172.21.15.16:0/3862707100 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fccfc15e2a0 msgr2=0x7fccfc0b8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.629+0000 7fcd010a0640 1 --2- 172.21.15.16:0/3862707100 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fccfc15e2a0 0x7fccfc0b8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.629+0000 7fcd010a0640 1 -- 172.21.15.16:0/3862707100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fccfc06d960 msgr2=0x7fccfc0cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.629+0000 7fcd010a0640 1 --2- 172.21.15.16:0/3862707100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fccfc06d960 0x7fccfc0cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.629+0000 7fcd010a0640 1 -- 172.21.15.16:0/3862707100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccfc0b9090 con 0x7fccfc15e680 2024-09-19T07:55:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.629+0000 7fcd010a0640 1 --2- 172.21.15.16:0/3862707100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccfc15e680 0x7fccfc0cbde0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fccf400c7a0 tx=0x7fccf400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.629+0000 7fccf97fa640 1 -- 172.21.15.16:0/3862707100 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4003b70 con 0x7fccfc15e680 2024-09-19T07:55:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.629+0000 7fccf97fa640 1 -- 172.21.15.16:0/3862707100 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf4003d30 con 0x7fccfc15e680 2024-09-19T07:55:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.629+0000 7fcd02b03640 1 -- 172.21.15.16:0/3862707100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccfc0b8ad0 con 0x7fccfc15e680 2024-09-19T07:55:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.629+0000 7fccf97fa640 1 -- 172.21.15.16:0/3862707100 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf400aa90 con 0x7fccfc15e680 2024-09-19T07:55:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.629+0000 7fcd02b03640 1 -- 172.21.15.16:0/3862707100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccfc0cb5e0 con 0x7fccfc15e680 2024-09-19T07:55:35.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.630+0000 7fcd02b03640 1 -- 172.21.15.16:0/3862707100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccfc06d420 con 0x7fccfc15e680 2024-09-19T07:55:35.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.631+0000 7fccf97fa640 1 -- 172.21.15.16:0/3862707100 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fccf400ac30 con 0x7fccfc15e680 2024-09-19T07:55:35.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.631+0000 7fccf97fa640 1 --2- 172.21.15.16:0/3862707100 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcce806c2e0 0x7fcce806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:35.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.631+0000 7fccf97fa640 1 -- 172.21.15.16:0/3862707100 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5335+0+0 (secure 0 0 0) 0x7fccf408b000 con 0x7fccfc15e680 2024-09-19T07:55:35.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.631+0000 7fcd0089f640 1 --2- 172.21.15.16:0/3862707100 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcce806c2e0 0x7fcce806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:35.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.632+0000 7fcd0089f640 1 --2- 172.21.15.16:0/3862707100 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcce806c2e0 0x7fcce806e7a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fccec006fd0 tx=0x7fccec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.635+0000 7fccf97fa640 1 -- 172.21.15.16:0/3862707100 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fccf4059320 con 0x7fccfc15e680 2024-09-19T07:55:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.810+0000 7fcd02b03640 1 -- 172.21.15.16:0/3862707100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fccfc15d6c0 con 0x7fccfc15e680 2024-09-19T07:55:35.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.814+0000 7fccf97fa640 1 -- 172.21.15.16:0/3862707100 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v73)=0 v73) ==== 122+0+0 (secure 0 0 0) 0x7fccf405d620 con 0x7fccfc15e680 2024-09-19T07:55:35.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.816+0000 7fcd02b03640 1 -- 172.21.15.16:0/3862707100 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcce806c2e0 msgr2=0x7fcce806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:35.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.816+0000 7fcd02b03640 1 --2- 172.21.15.16:0/3862707100 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcce806c2e0 0x7fcce806e7a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fccec006fd0 tx=0x7fccec008040 comp rx=0 tx=0).stop 2024-09-19T07:55:35.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.816+0000 7fcd02b03640 1 -- 172.21.15.16:0/3862707100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccfc15e680 msgr2=0x7fccfc0cbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:35.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.816+0000 7fcd02b03640 1 --2- 172.21.15.16:0/3862707100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccfc15e680 0x7fccfc0cbde0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fccf400c7a0 tx=0x7fccf400cc60 comp rx=0 tx=0).stop 2024-09-19T07:55:35.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.816+0000 7fcd02b03640 1 -- 172.21.15.16:0/3862707100 shutdown_connections 2024-09-19T07:55:35.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.816+0000 7fcd02b03640 1 --2- 172.21.15.16:0/3862707100 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fcce806c2e0 0x7fcce806e7a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.816+0000 7fcd02b03640 1 --2- 172.21.15.16:0/3862707100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccfc15e680 0x7fccfc0cbde0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.816+0000 7fcd02b03640 1 --2- 172.21.15.16:0/3862707100 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fccfc15e2a0 0x7fccfc0b8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.816+0000 7fcd02b03640 1 --2- 172.21.15.16:0/3862707100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fccfc06d960 0x7fccfc0cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:35.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.816+0000 7fcd02b03640 1 -- 172.21.15.16:0/3862707100 >> 172.21.15.16:0/3862707100 conn(0x7fccfc09d760 msgr2=0x7fccfc0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:35.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.816+0000 7fcd02b03640 1 -- 172.21.15.16:0/3862707100 shutdown_connections 2024-09-19T07:55:35.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:35.816+0000 7fcd02b03640 1 -- 172.21.15.16:0/3862707100 wait complete. 2024-09-19T07:55:35.828 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:55:35.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:55:35.868 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:55:35.935 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:55:35.935 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:55:35.935 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:55:35.951 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:55:35.952 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:55:35.952 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:55:35.970 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:55:35.970 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:55:35.970 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:55:36.008 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:55:36.008 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:55:36.008 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:55:36.026 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:55:36.027 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T07:55:36.027 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:55:36.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.149+0000 7f7ab292c640 1 Processor -- start 2024-09-19T07:55:36.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.149+0000 7f7ab292c640 1 -- start start 2024-09-19T07:55:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.150+0000 7f7ab292c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7aac0c8d20 0x7f7aac06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.150+0000 7f7ab292c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7aac06d260 0x7f7aac1439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.150+0000 7f7ab292c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7aac0c7140 con 0x7f7aac143f30 2024-09-19T07:55:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.150+0000 7f7ab292c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f7aac141960 con 0x7f7aac0c8d20 2024-09-19T07:55:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.150+0000 7f7ab292c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7aac142200 con 0x7f7aac06d260 2024-09-19T07:55:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.150+0000 7f7aab7fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7aac06d260 0x7f7aac1439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.150+0000 7f7aab7fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7aac06d260 0x7f7aac1439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:54800/0 (socket says 172.21.15.16:54800) 2024-09-19T07:55:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.150+0000 7f7aab7fe640 1 -- 172.21.15.16:0/55282518 learned_addr learned my addr 172.21.15.16:0/55282518 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.150+0000 7f7aabfff640 1 --2- 172.21.15.16:0/55282518 >> v2:172.21.15.146:3300/0 conn(0x7f7aac0c8d20 0x7f7aac06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.151+0000 7f7aab7fe640 1 -- 172.21.15.16:0/55282518 >> v2:172.21.15.146:3300/0 conn(0x7f7aac0c8d20 msgr2=0x7f7aac06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.151+0000 7f7aab7fe640 1 --2- 172.21.15.16:0/55282518 >> v2:172.21.15.146:3300/0 conn(0x7f7aac0c8d20 0x7f7aac06cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.151+0000 7f7aab7fe640 1 -- 172.21.15.16:0/55282518 >> v1:172.21.15.146:6789/0 conn(0x7f7aac143f30 legacy=0x7f7aac0d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.151+0000 7f7aab7fe640 1 -- 172.21.15.16:0/55282518 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac0c9100 con 0x7f7aac06d260 2024-09-19T07:55:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.151+0000 7f7aabfff640 1 --2- 172.21.15.16:0/55282518 >> v2:172.21.15.146:3300/0 conn(0x7f7aac0c8d20 0x7f7aac06cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:55:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.151+0000 7f7aab7fe640 1 --2- 172.21.15.16:0/55282518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7aac06d260 0x7f7aac1439f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009870 tx=0x7f7aa002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ec9abcf993881f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.152+0000 7f7aaaffd640 1 -- 172.21.15.16:0/55282518 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0037070 con 0x7f7aac06d260 2024-09-19T07:55:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.152+0000 7f7aaaffd640 1 -- 172.21.15.16:0/55282518 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa0032440 con 0x7f7aac06d260 2024-09-19T07:55:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.152+0000 7f7aaaffd640 1 -- 172.21.15.16:0/55282518 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0032810 con 0x7f7aac06d260 2024-09-19T07:55:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.152+0000 7f7ab292c640 1 -- 172.21.15.16:0/55282518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7aac06d260 msgr2=0x7f7aac1439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.152+0000 7f7ab292c640 1 --2- 172.21.15.16:0/55282518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7aac06d260 0x7f7aac1439f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009870 tx=0x7f7aa002a130 comp rx=0 tx=0).stop 2024-09-19T07:55:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.152+0000 7f7ab292c640 1 -- 172.21.15.16:0/55282518 shutdown_connections 2024-09-19T07:55:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.152+0000 7f7ab292c640 1 --2- 172.21.15.16:0/55282518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7aac06d260 0x7f7aac1439f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.152+0000 7f7ab292c640 1 --2- 172.21.15.16:0/55282518 >> v2:172.21.15.146:3300/0 conn(0x7f7aac0c8d20 0x7f7aac06cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.152+0000 7f7ab292c640 1 -- 172.21.15.16:0/55282518 >> 172.21.15.16:0/55282518 conn(0x7f7aac0b3ed0 msgr2=0x7f7aac0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.152+0000 7f7ab292c640 1 -- 172.21.15.16:0/55282518 shutdown_connections 2024-09-19T07:55:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.152+0000 7f7ab292c640 1 -- 172.21.15.16:0/55282518 wait complete. 2024-09-19T07:55:36.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.153+0000 7f7ab292c640 1 Processor -- start 2024-09-19T07:55:36.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.154+0000 7f7ab292c640 1 -- start start 2024-09-19T07:55:36.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.154+0000 7f7ab292c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7aac06d260 0x7f7aac0cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:36.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.154+0000 7f7ab292c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7aac0c8d20 0x7f7aac130760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:36.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.154+0000 7f7ab292c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aac143f30 0x7f7aac1327c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:36.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.154+0000 7f7ab292c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac141bb0 con 0x7f7aac143f30 2024-09-19T07:55:36.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.154+0000 7f7ab292c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac06d6a0 con 0x7f7aac06d260 2024-09-19T07:55:36.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.154+0000 7f7ab292c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7aac141e90 con 0x7f7aac0c8d20 2024-09-19T07:55:36.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.154+0000 7f7ab0ec9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aac143f30 0x7f7aac1327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:36.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.154+0000 7f7ab0ec9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aac143f30 0x7f7aac1327c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50094/0 (socket says 172.21.15.16:50094) 2024-09-19T07:55:36.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.154+0000 7f7ab0ec9640 1 -- 172.21.15.16:0/2677199616 learned_addr learned my addr 172.21.15.16:0/2677199616 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:36.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.154+0000 7f7aabfff640 1 --2- 172.21.15.16:0/2677199616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7aac06d260 0x7f7aac0cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:36.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.155+0000 7f7aab7fe640 1 --2- 172.21.15.16:0/2677199616 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7aac0c8d20 0x7f7aac130760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:36.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.155+0000 7f7ab0ec9640 1 -- 172.21.15.16:0/2677199616 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7aac0c8d20 msgr2=0x7f7aac130760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:36.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.155+0000 7f7ab0ec9640 1 --2- 172.21.15.16:0/2677199616 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7aac0c8d20 0x7f7aac130760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:36.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.155+0000 7f7ab0ec9640 1 -- 172.21.15.16:0/2677199616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7aac06d260 msgr2=0x7f7aac0cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:36.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.155+0000 7f7ab0ec9640 1 --2- 172.21.15.16:0/2677199616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7aac06d260 0x7f7aac0cbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:36.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.155+0000 7f7ab0ec9640 1 -- 172.21.15.16:0/2677199616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac0b62c0 con 0x7f7aac143f30 2024-09-19T07:55:36.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.155+0000 7f7ab0ec9640 1 --2- 172.21.15.16:0/2677199616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aac143f30 0x7f7aac1327c0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400c730 tx=0x7f7aa400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:36.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.155+0000 7f7aa8ff9640 1 -- 172.21.15.16:0/2677199616 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4016020 con 0x7f7aac143f30 2024-09-19T07:55:36.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.155+0000 7f7aa8ff9640 1 -- 172.21.15.16:0/2677199616 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa4006d50 con 0x7f7aac143f30 2024-09-19T07:55:36.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.155+0000 7f7ab292c640 1 -- 172.21.15.16:0/2677199616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aac0b6530 con 0x7f7aac143f30 2024-09-19T07:55:36.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.155+0000 7f7aa8ff9640 1 -- 172.21.15.16:0/2677199616 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4003bb0 con 0x7f7aac143f30 2024-09-19T07:55:36.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.155+0000 7f7ab292c640 1 -- 172.21.15.16:0/2677199616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aac0b6af0 con 0x7f7aac143f30 2024-09-19T07:55:36.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.156+0000 7f7ab292c640 1 -- 172.21.15.16:0/2677199616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aac144360 con 0x7f7aac143f30 2024-09-19T07:55:36.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.157+0000 7f7aa8ff9640 1 -- 172.21.15.16:0/2677199616 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7aa40050d0 con 0x7f7aac143f30 2024-09-19T07:55:36.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.157+0000 7f7aa8ff9640 1 --2- 172.21.15.16:0/2677199616 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7a9806c2b0 0x7f7a9806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:36.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.157+0000 7f7aa8ff9640 1 -- 172.21.15.16:0/2677199616 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5335+0+0 (secure 0 0 0) 0x7f7aa408acc0 con 0x7f7aac143f30 2024-09-19T07:55:36.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.157+0000 7f7aabfff640 1 --2- 172.21.15.16:0/2677199616 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7a9806c2b0 0x7f7a9806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:36.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.158+0000 7f7aabfff640 1 --2- 172.21.15.16:0/2677199616 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7a9806c2b0 0x7f7a9806e770 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009560 tx=0x7f7a9c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:36.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.162+0000 7f7aa8ff9640 1 -- 172.21.15.16:0/2677199616 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7aa4020020 con 0x7f7aac143f30 2024-09-19T07:55:36.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.370+0000 7f7ab292c640 1 -- 172.21.15.16:0/2677199616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7aac0690c0 con 0x7f7aac143f30 2024-09-19T07:55:36.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.371+0000 7f7aa8ff9640 1 -- 172.21.15.16:0/2677199616 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+3914 (secure 0 0 0) 0x7f7aac0c7140 con 0x7f7aac143f30 2024-09-19T07:55:36.372 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:36.373 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":85,"btime":"2024-09-19T07:55:30:488301+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7540,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6836/172459502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6835","nonce":172459502},{"type":"v1","addr":"172.21.15.146:6836","nonce":172459502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3417061308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3417061308},{"type":"v1","addr":"172.21.15.146:6837","nonce":3417061308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1810646409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1810646409},{"type":"v1","addr":"172.21.15.16:6834","nonce":1810646409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2163655847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2163655847},{"type":"v1","addr":"172.21.15.16:6835","nonce":2163655847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[]} 2024-09-19T07:55:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 85 2024-09-19T07:55:36.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.373+0000 7f7ab292c640 1 -- 172.21.15.16:0/2677199616 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7a9806c2b0 msgr2=0x7f7a9806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:36.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.373+0000 7f7ab292c640 1 --2- 172.21.15.16:0/2677199616 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7a9806c2b0 0x7f7a9806e770 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009560 tx=0x7f7a9c009190 comp rx=0 tx=0).stop 2024-09-19T07:55:36.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.373+0000 7f7ab292c640 1 -- 172.21.15.16:0/2677199616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aac143f30 msgr2=0x7f7aac1327c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.373+0000 7f7ab292c640 1 --2- 172.21.15.16:0/2677199616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aac143f30 0x7f7aac1327c0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400c730 tx=0x7f7aa400cb30 comp rx=0 tx=0).stop 2024-09-19T07:55:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.373+0000 7f7ab292c640 1 -- 172.21.15.16:0/2677199616 shutdown_connections 2024-09-19T07:55:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.373+0000 7f7ab292c640 1 --2- 172.21.15.16:0/2677199616 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f7a9806c2b0 0x7f7a9806e770 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.373+0000 7f7ab292c640 1 --2- 172.21.15.16:0/2677199616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aac143f30 0x7f7aac1327c0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.373+0000 7f7ab292c640 1 --2- 172.21.15.16:0/2677199616 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7aac0c8d20 0x7f7aac130760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.373+0000 7f7ab292c640 1 --2- 172.21.15.16:0/2677199616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7aac06d260 0x7f7aac0cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.373+0000 7f7ab292c640 1 -- 172.21.15.16:0/2677199616 >> 172.21.15.16:0/2677199616 conn(0x7f7aac0b3ed0 msgr2=0x7f7aac15fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.373+0000 7f7ab292c640 1 -- 172.21.15.16:0/2677199616 shutdown_connections 2024-09-19T07:55:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:36.373+0000 7f7ab292c640 1 -- 172.21.15.16:0/2677199616 wait complete. 2024-09-19T07:55:36.385 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T07:55:36.386 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T07:55:36.386 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T07:55:36.386 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:55:36.386 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T07:55:36.386 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T07:55:36.386 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T07:55:36.386 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:55:36.387 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T07:55:36.387 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T07:55:36.387 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T07:55:36.387 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:55:36.387 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T07:55:36.387 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T07:55:36.387 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T07:55:36.388 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:55:36.395 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T07:55:36.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T07:55:36.396 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T07:55:36.396 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T07:55:36.400 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T07:55:36.400 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T07:55:36.439 INFO:tasks.ceph.mds.a:Started 2024-09-19T07:55:36.439 DEBUG:teuthology.parallel:result is None 2024-09-19T07:55:36.439 INFO:tasks.ceph.mds.c:Started 2024-09-19T07:55:36.440 DEBUG:teuthology.parallel:result is None 2024-09-19T07:55:36.442 INFO:tasks.ceph.mds.d:Started 2024-09-19T07:55:36.442 DEBUG:teuthology.parallel:result is None 2024-09-19T07:55:36.514 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-19T07:55:36.515 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-19T07:55:36.523 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2024-09-19T07:55:39.715 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:55:39.718 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:55:39.801 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-19T07:55:42.400 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T07:55:42.400 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T07:55:42.404 INFO:tasks.ceph.mds.b:Started 2024-09-19T07:55:42.404 DEBUG:teuthology.parallel:result is None 2024-09-19T07:55:42.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T07:55:42.487 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2024-09-19T07:55:42.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.520+0000 7ff9e24c9640 1 Processor -- start 2024-09-19T07:55:42.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.520+0000 7ff9e24c9640 1 -- start start 2024-09-19T07:55:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.521+0000 7ff9e24c9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9dc06cd20 0x7ff9dc06d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.521+0000 7ff9e24c9640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff9dc161eb0 0x7ff9dc1622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.521+0000 7ff9e24c9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff9dc143a20 0x7ff9dc143ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.521+0000 7ff9e24c9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7ff9dc162e50 con 0x7ff9dc143a20 2024-09-19T07:55:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.521+0000 7ff9e24c9640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff9dc142150 con 0x7ff9dc161eb0 2024-09-19T07:55:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.521+0000 7ff9e24c9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc06d640 con 0x7ff9dc06cd20 2024-09-19T07:55:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.521+0000 7ff9e0a66640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff9dc143a20 0x7ff9dc143ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.521+0000 7ff9e0a66640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff9dc143a20 0x7ff9dc143ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49574/0 (socket says 172.21.15.16:49574) 2024-09-19T07:55:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.521+0000 7ff9e0a66640 1 -- 172.21.15.16:0/3908517573 learned_addr learned my addr 172.21.15.16:0/3908517573 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.521+0000 7ff9db7fe640 1 --2- 172.21.15.16:0/3908517573 >> v2:172.21.15.146:3300/0 conn(0x7ff9dc161eb0 0x7ff9dc1622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.521+0000 7ff9dbfff640 1 --2- 172.21.15.16:0/3908517573 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9dc06cd20 0x7ff9dc06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.521+0000 7ff9e0a66640 1 -- 172.21.15.16:0/3908517573 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9dc06cd20 msgr2=0x7ff9dc06d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.521+0000 7ff9e0a66640 1 --2- 172.21.15.16:0/3908517573 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9dc06cd20 0x7ff9dc06d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.521+0000 7ff9e0a66640 1 -- 172.21.15.16:0/3908517573 >> v2:172.21.15.146:3300/0 conn(0x7ff9dc161eb0 msgr2=0x7ff9dc1622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.521+0000 7ff9e0a66640 1 --2- 172.21.15.16:0/3908517573 >> v2:172.21.15.146:3300/0 conn(0x7ff9dc161eb0 0x7ff9dc1622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.522+0000 7ff9e0a66640 1 -- 172.21.15.16:0/3908517573 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc162b70 con 0x7ff9dc143a20 2024-09-19T07:55:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.522+0000 7ff9db7fe640 1 --2- 172.21.15.16:0/3908517573 >> v2:172.21.15.146:3300/0 conn(0x7ff9dc161eb0 0x7ff9dc1622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.522+0000 7ff9e0a66640 1 --2- 172.21.15.16:0/3908517573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc143a20 0x7ff9dc143ed0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0d2fc0 tx=0x7ff9d402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1376aa7d5b44ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.522+0000 7ff9daffd640 1 -- 172.21.15.16:0/3908517573 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4036070 con 0x7ff9dc143a20 2024-09-19T07:55:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.522+0000 7ff9daffd640 1 -- 172.21.15.16:0/3908517573 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d4031600 con 0x7ff9dc143a20 2024-09-19T07:55:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.522+0000 7ff9daffd640 1 -- 172.21.15.16:0/3908517573 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40319d0 con 0x7ff9dc143a20 2024-09-19T07:55:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.523+0000 7ff9e24c9640 1 -- 172.21.15.16:0/3908517573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc143a20 msgr2=0x7ff9dc143ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.523+0000 7ff9e24c9640 1 --2- 172.21.15.16:0/3908517573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc143a20 0x7ff9dc143ed0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0d2fc0 tx=0x7ff9d402a610 comp rx=0 tx=0).stop 2024-09-19T07:55:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.523+0000 7ff9e24c9640 1 -- 172.21.15.16:0/3908517573 shutdown_connections 2024-09-19T07:55:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.523+0000 7ff9e24c9640 1 --2- 172.21.15.16:0/3908517573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc143a20 0x7ff9dc143ed0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.523+0000 7ff9e24c9640 1 --2- 172.21.15.16:0/3908517573 >> v2:172.21.15.146:3300/0 conn(0x7ff9dc161eb0 0x7ff9dc1622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.523+0000 7ff9e24c9640 1 --2- 172.21.15.16:0/3908517573 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9dc06cd20 0x7ff9dc06d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.523+0000 7ff9e24c9640 1 -- 172.21.15.16:0/3908517573 >> 172.21.15.16:0/3908517573 conn(0x7ff9dc0b3ed0 msgr2=0x7ff9dc0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.523+0000 7ff9e24c9640 1 -- 172.21.15.16:0/3908517573 shutdown_connections 2024-09-19T07:55:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.523+0000 7ff9e24c9640 1 -- 172.21.15.16:0/3908517573 wait complete. 2024-09-19T07:55:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.524+0000 7ff9e24c9640 1 Processor -- start 2024-09-19T07:55:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.525+0000 7ff9e24c9640 1 -- start start 2024-09-19T07:55:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.525+0000 7ff9e24c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc06cd20 0x7ff9dc158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.525+0000 7ff9e24c9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9dc143a20 0x7ff9dc0b7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.525+0000 7ff9e24c9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9dc161eb0 0x7ff9dc0b7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.525+0000 7ff9e24c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc162830 con 0x7ff9dc06cd20 2024-09-19T07:55:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.525+0000 7ff9e24c9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc141de0 con 0x7ff9dc143a20 2024-09-19T07:55:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.525+0000 7ff9e24c9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc141600 con 0x7ff9dc161eb0 2024-09-19T07:55:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.525+0000 7ff9dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc06cd20 0x7ff9dc158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.525+0000 7ff9dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc06cd20 0x7ff9dc158380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49582/0 (socket says 172.21.15.16:49582) 2024-09-19T07:55:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.525+0000 7ff9dbfff640 1 -- 172.21.15.16:0/3464014595 learned_addr learned my addr 172.21.15.16:0/3464014595 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.526+0000 7ff9db7fe640 1 --2- 172.21.15.16:0/3464014595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9dc143a20 0x7ff9dc0b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.526+0000 7ff9e0a66640 1 --2- 172.21.15.16:0/3464014595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9dc161eb0 0x7ff9dc0b7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.526+0000 7ff9dbfff640 1 -- 172.21.15.16:0/3464014595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9dc161eb0 msgr2=0x7ff9dc0b7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.526+0000 7ff9dbfff640 1 --2- 172.21.15.16:0/3464014595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9dc161eb0 0x7ff9dc0b7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.526+0000 7ff9dbfff640 1 -- 172.21.15.16:0/3464014595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9dc143a20 msgr2=0x7ff9dc0b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.526+0000 7ff9dbfff640 1 --2- 172.21.15.16:0/3464014595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9dc143a20 0x7ff9dc0b7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.526+0000 7ff9dbfff640 1 -- 172.21.15.16:0/3464014595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc0b57c0 con 0x7ff9dc06cd20 2024-09-19T07:55:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.526+0000 7ff9dbfff640 1 --2- 172.21.15.16:0/3464014595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc06cd20 0x7ff9dc158380 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00e870 tx=0x7ff9cc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.526+0000 7ff9d8ff9640 1 -- 172.21.15.16:0/3464014595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc017070 con 0x7ff9dc06cd20 2024-09-19T07:55:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.526+0000 7ff9d8ff9640 1 -- 172.21.15.16:0/3464014595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9cc00ee60 con 0x7ff9dc06cd20 2024-09-19T07:55:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.526+0000 7ff9e24c9640 1 -- 172.21.15.16:0/3464014595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9dc0b5aa0 con 0x7ff9dc06cd20 2024-09-19T07:55:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.526+0000 7ff9d8ff9640 1 -- 172.21.15.16:0/3464014595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc00cb80 con 0x7ff9dc06cd20 2024-09-19T07:55:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.526+0000 7ff9e24c9640 1 -- 172.21.15.16:0/3464014595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9dc0b6060 con 0x7ff9dc06cd20 2024-09-19T07:55:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.527+0000 7ff9e24c9640 1 -- 172.21.15.16:0/3464014595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9dc06d100 con 0x7ff9dc06cd20 2024-09-19T07:55:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.528+0000 7ff9d8ff9640 1 -- 172.21.15.16:0/3464014595 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff9cc01a070 con 0x7ff9dc06cd20 2024-09-19T07:55:42.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.528+0000 7ff9d8ff9640 1 --2- 172.21.15.16:0/3464014595 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff9c806c2e0 0x7ff9c806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:42.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.528+0000 7ff9d8ff9640 1 -- 172.21.15.16:0/3464014595 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5335+0+0 (secure 0 0 0) 0x7ff9cc08b040 con 0x7ff9dc06cd20 2024-09-19T07:55:42.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.528+0000 7ff9db7fe640 1 --2- 172.21.15.16:0/3464014595 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff9c806c2e0 0x7ff9c806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:42.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.529+0000 7ff9db7fe640 1 --2- 172.21.15.16:0/3464014595 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff9c806c2e0 0x7ff9c806e7a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0b7250 tx=0x7ff9d0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:42.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.534+0000 7ff9d8ff9640 1 -- 172.21.15.16:0/3464014595 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff9cc0592d0 con 0x7ff9dc06cd20 2024-09-19T07:55:42.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:42.722+0000 7ff9e24c9640 1 -- 172.21.15.16:0/3464014595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff9dc0690c0 con 0x7ff9dc06cd20 2024-09-19T07:55:43.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.698+0000 7ff9d8ff9640 1 -- 172.21.15.16:0/3464014595 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v148) ==== 97+0+0 (secure 0 0 0) 0x7ff9cc05d5d0 con 0x7ff9dc06cd20 2024-09-19T07:55:43.699 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-19T07:55:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.702+0000 7ff9e24c9640 1 -- 172.21.15.16:0/3464014595 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff9c806c2e0 msgr2=0x7ff9c806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.702+0000 7ff9e24c9640 1 --2- 172.21.15.16:0/3464014595 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff9c806c2e0 0x7ff9c806e7a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0b7250 tx=0x7ff9d0008040 comp rx=0 tx=0).stop 2024-09-19T07:55:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.702+0000 7ff9e24c9640 1 -- 172.21.15.16:0/3464014595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc06cd20 msgr2=0x7ff9dc158380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.702+0000 7ff9e24c9640 1 --2- 172.21.15.16:0/3464014595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc06cd20 0x7ff9dc158380 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00e870 tx=0x7ff9cc00ed30 comp rx=0 tx=0).stop 2024-09-19T07:55:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.702+0000 7ff9e24c9640 1 -- 172.21.15.16:0/3464014595 shutdown_connections 2024-09-19T07:55:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.702+0000 7ff9e24c9640 1 --2- 172.21.15.16:0/3464014595 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff9c806c2e0 0x7ff9c806e7a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.702+0000 7ff9e24c9640 1 --2- 172.21.15.16:0/3464014595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9dc161eb0 0x7ff9dc0b7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.702+0000 7ff9e24c9640 1 --2- 172.21.15.16:0/3464014595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9dc143a20 0x7ff9dc0b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.702+0000 7ff9e24c9640 1 --2- 172.21.15.16:0/3464014595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc06cd20 0x7ff9dc158380 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.702+0000 7ff9e24c9640 1 -- 172.21.15.16:0/3464014595 >> 172.21.15.16:0/3464014595 conn(0x7ff9dc0b3ed0 msgr2=0x7ff9dc06ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.702+0000 7ff9e24c9640 1 -- 172.21.15.16:0/3464014595 shutdown_connections 2024-09-19T07:55:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.702+0000 7ff9e24c9640 1 -- 172.21.15.16:0/3464014595 wait complete. 2024-09-19T07:55:43.716 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T07:55:43.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.824+0000 7f925345c640 1 Processor -- start 2024-09-19T07:55:43.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.824+0000 7f925345c640 1 -- start start 2024-09-19T07:55:43.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.825+0000 7f925345c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f924c15e2a0 0x7f924c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:43.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.825+0000 7f925345c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f924c0d1880 con 0x7f924c06d960 2024-09-19T07:55:43.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.825+0000 7f925345c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f924c0d1b60 con 0x7f924c15e680 2024-09-19T07:55:43.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.825+0000 7f925345c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f924c0cf2d0 con 0x7f924c15e2a0 2024-09-19T07:55:43.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.825+0000 7f92519f9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f924c06d960 0x7f924c0d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60666/0 (socket says 172.21.15.16:60666) 2024-09-19T07:55:43.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.825+0000 7f92519f9640 1 -- 172.21.15.16:0/2121933183 learned_addr learned my addr 172.21.15.16:0/2121933183 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:43.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.825+0000 7f9243fff640 1 -- 172.21.15.16:0/2121933183 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 216150970 0 0) 0x7f924c0d1880 con 0x7f924c06d960 2024-09-19T07:55:43.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.825+0000 7f92511f8640 1 --2- 172.21.15.16:0/2121933183 >> v2:172.21.15.146:3300/0 conn(0x7f924c15e2a0 0x7f924c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:43.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.825+0000 7f9243fff640 1 -- 172.21.15.16:0/2121933183 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9238003520 con 0x7f924c06d960 2024-09-19T07:55:43.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f9243fff640 1 -- 172.21.15.16:0/2121933183 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3274529865 0 0) 0x7f9238003520 con 0x7f924c06d960 2024-09-19T07:55:43.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f9243fff640 1 -- 172.21.15.16:0/2121933183 >> v2:172.21.15.146:3300/0 conn(0x7f924c15e2a0 msgr2=0x7f924c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:43.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f9243fff640 1 --2- 172.21.15.16:0/2121933183 >> v2:172.21.15.146:3300/0 conn(0x7f924c15e2a0 0x7f924c06d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:43.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f9243fff640 1 -- 172.21.15.16:0/2121933183 >> v1:172.21.15.146:6789/0 conn(0x7f924c15e680 legacy=0x7f924c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:43.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f9243fff640 1 -- 172.21.15.16:0/2121933183 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c0cf5b0 con 0x7f924c06d960 2024-09-19T07:55:43.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f9243fff640 1 -- 172.21.15.16:0/2121933183 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f92440019f0 con 0x7f924c06d960 2024-09-19T07:55:43.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f9243fff640 1 -- 172.21.15.16:0/2121933183 >> v1:172.21.15.16:6789/0 conn(0x7f924c06d960 legacy=0x7f924c0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:43.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f9243fff640 1 --2- 172.21.15.16:0/2121933183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92380039b0 0x7f9238003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:43.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f9243fff640 1 --2- 172.21.15.16:0/2121933183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9238004800 0x7f9238004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:43.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f9243fff640 1 --2- 172.21.15.16:0/2121933183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92380051c0 0x7f923800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:43.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f9243fff640 1 -- 172.21.15.16:0/2121933183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f924c0cf2d0 con 0x7f92380039b0 2024-09-19T07:55:43.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f9243fff640 1 -- 172.21.15.16:0/2121933183 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f923800c060 con 0x7f9238004800 2024-09-19T07:55:43.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f9243fff640 1 -- 172.21.15.16:0/2121933183 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f923800c260 con 0x7f92380051c0 2024-09-19T07:55:43.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f92511f8640 1 --2- 172.21.15.16:0/2121933183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92380039b0 0x7f9238003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:43.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f92509f7640 1 --2- 172.21.15.16:0/2121933183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9238004800 0x7f9238004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:43.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f92519f9640 1 --2- 172.21.15.16:0/2121933183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92380051c0 0x7f923800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f92511f8640 1 -- 172.21.15.16:0/2121933183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92380051c0 msgr2=0x7f923800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f92511f8640 1 --2- 172.21.15.16:0/2121933183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92380051c0 0x7f923800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f92511f8640 1 -- 172.21.15.16:0/2121933183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9238004800 msgr2=0x7f9238004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f92511f8640 1 --2- 172.21.15.16:0/2121933183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9238004800 0x7f9238004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f92511f8640 1 -- 172.21.15.16:0/2121933183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9238003520 con 0x7f92380039b0 2024-09-19T07:55:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f92519f9640 1 --2- 172.21.15.16:0/2121933183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92380051c0 0x7f923800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f92509f7640 1 --2- 172.21.15.16:0/2121933183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9238004800 0x7f9238004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.826+0000 7f92519f9640 1 -- 172.21.15.16:0/2121933183 reap_dead start 2024-09-19T07:55:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.827+0000 7f92511f8640 1 --2- 172.21.15.16:0/2121933183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92380039b0 0x7f9238003dd0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f9248009ac0 tx=0x7f9238005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a068c5997b691af4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.827+0000 7f9243fff640 1 -- 172.21.15.16:0/2121933183 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9248036070 con 0x7f92380039b0 2024-09-19T07:55:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.827+0000 7f9243fff640 1 -- 172.21.15.16:0/2121933183 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9248031400 con 0x7f92380039b0 2024-09-19T07:55:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.827+0000 7f9243fff640 1 -- 172.21.15.16:0/2121933183 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9248009690 con 0x7f92380039b0 2024-09-19T07:55:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.827+0000 7f925345c640 1 -- 172.21.15.16:0/2121933183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92380039b0 msgr2=0x7f9238003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.827+0000 7f925345c640 1 --2- 172.21.15.16:0/2121933183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92380039b0 0x7f9238003dd0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f9248009ac0 tx=0x7f9238005830 comp rx=0 tx=0).stop 2024-09-19T07:55:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.828+0000 7f925345c640 1 -- 172.21.15.16:0/2121933183 shutdown_connections 2024-09-19T07:55:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.828+0000 7f925345c640 1 --2- 172.21.15.16:0/2121933183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92380039b0 0x7f9238003dd0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.828+0000 7f925345c640 1 -- 172.21.15.16:0/2121933183 >> 172.21.15.16:0/2121933183 conn(0x7f924c09d760 msgr2=0x7f924c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.828+0000 7f925345c640 1 -- 172.21.15.16:0/2121933183 shutdown_connections 2024-09-19T07:55:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.828+0000 7f925345c640 1 -- 172.21.15.16:0/2121933183 wait complete. 2024-09-19T07:55:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.829+0000 7f925345c640 1 Processor -- start 2024-09-19T07:55:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.829+0000 7f925345c640 1 -- start start 2024-09-19T07:55:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.830+0000 7f925345c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92380039b0 0x7f924c1528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.830+0000 7f925345c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f924c0b9800 0x7f924c0b9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.830+0000 7f925345c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f924c0cd290 0x7f924c0cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.830+0000 7f925345c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f924c06d5a0 con 0x7f92380039b0 2024-09-19T07:55:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.830+0000 7f925345c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f924c06d420 con 0x7f924c0b9800 2024-09-19T07:55:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.830+0000 7f925345c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f924c06d720 con 0x7f924c0cd290 2024-09-19T07:55:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.830+0000 7f92511f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92380039b0 0x7f924c1528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.830+0000 7f92511f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92380039b0 0x7f924c1528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49606/0 (socket says 172.21.15.16:49606) 2024-09-19T07:55:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.830+0000 7f92511f8640 1 -- 172.21.15.16:0/1945269640 learned_addr learned my addr 172.21.15.16:0/1945269640 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.830+0000 7f92509f7640 1 --2- 172.21.15.16:0/1945269640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f924c0b9800 0x7f924c0b9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.830+0000 7f92511f8640 1 -- 172.21.15.16:0/1945269640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f924c0cd290 msgr2=0x7f924c0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.830+0000 7f92519f9640 1 --2- 172.21.15.16:0/1945269640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f924c0cd290 0x7f924c0cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.830+0000 7f92511f8640 1 --2- 172.21.15.16:0/1945269640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f924c0cd290 0x7f924c0cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.830+0000 7f92511f8640 1 -- 172.21.15.16:0/1945269640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f924c0b9800 msgr2=0x7f924c0b9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.830+0000 7f92511f8640 1 --2- 172.21.15.16:0/1945269640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f924c0b9800 0x7f924c0b9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.830+0000 7f92511f8640 1 -- 172.21.15.16:0/1945269640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c0b7dc0 con 0x7f92380039b0 2024-09-19T07:55:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.830+0000 7f92509f7640 1 --2- 172.21.15.16:0/1945269640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f924c0b9800 0x7f924c0b9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.830+0000 7f92511f8640 1 --2- 172.21.15.16:0/1945269640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92380039b0 0x7f924c1528d0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f924802abb0 tx=0x7f9248004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.830+0000 7f9241ffb640 1 -- 172.21.15.16:0/1945269640 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924803f020 con 0x7f92380039b0 2024-09-19T07:55:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.831+0000 7f9241ffb640 1 -- 172.21.15.16:0/1945269640 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9248037040 con 0x7f92380039b0 2024-09-19T07:55:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.831+0000 7f925345c640 1 -- 172.21.15.16:0/1945269640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924c0b80a0 con 0x7f92380039b0 2024-09-19T07:55:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.831+0000 7f9241ffb640 1 -- 172.21.15.16:0/1945269640 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9248036070 con 0x7f92380039b0 2024-09-19T07:55:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.831+0000 7f925345c640 1 -- 172.21.15.16:0/1945269640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f924c0b8660 con 0x7f92380039b0 2024-09-19T07:55:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.832+0000 7f925345c640 1 -- 172.21.15.16:0/1945269640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f924c0cfad0 con 0x7f92380039b0 2024-09-19T07:55:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.832+0000 7f9241ffb640 1 -- 172.21.15.16:0/1945269640 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9248042020 con 0x7f92380039b0 2024-09-19T07:55:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.833+0000 7f9241ffb640 1 --2- 172.21.15.16:0/1945269640 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f923006c2b0 0x7f923006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.833+0000 7f9241ffb640 1 -- 172.21.15.16:0/1945269640 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5249+0+0 (secure 0 0 0) 0x7f92480ab260 con 0x7f92380039b0 2024-09-19T07:55:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.833+0000 7f92509f7640 1 --2- 172.21.15.16:0/1945269640 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f923006c2b0 0x7f923006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.834+0000 7f92509f7640 1 --2- 172.21.15.16:0/1945269640 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f923006c2b0 0x7f923006e770 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f923c000c00 tx=0x7f923c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:43.837+0000 7f9241ffb640 1 -- 172.21.15.16:0/1945269640 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9248049030 con 0x7f92380039b0 2024-09-19T07:55:44.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.076+0000 7f925345c640 1 -- 172.21.15.16:0/1945269640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f924c0ce8a0 con 0x7f92380039b0 2024-09-19T07:55:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.077+0000 7f9241ffb640 1 -- 172.21.15.16:0/1945269640 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7f9248079c50 con 0x7f92380039b0 2024-09-19T07:55:44.077 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:44.077 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:55:44.077 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-19T07:55:44.078 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.078 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-19T07:55:44.078 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etms80KDxAAqTrG88LTEJIj9TWJu0witQ==", 2024-09-19T07:55:44.078 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:55:44.078 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:55:44.078 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:55:44.078 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:55:44.078 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:55:44.078 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.078 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:44.078 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.078 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-19T07:55:44.078 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etm4T0KGxAAcfDOKrSdKzhJ4xvTFIyxcQ==", 2024-09-19T07:55:44.078 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:55:44.079 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:55:44.079 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:55:44.079 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:55:44.079 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:55:44.079 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.079 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:44.079 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.079 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-19T07:55:44.079 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNDBGFhAAtBioJJ1+NtIDLhd/Hg7aZA==", 2024-09-19T07:55:44.079 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:55:44.079 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:55:44.079 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:55:44.079 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:55:44.080 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:55:44.080 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:44.080 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.080 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-19T07:55:44.080 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmaHEiIhAACqqjCRMV+xlGh9WqYsj+MA==", 2024-09-19T07:55:44.080 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:55:44.080 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:55:44.080 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:55:44.080 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:55:44.080 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:55:44.080 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:44.081 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.081 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-19T07:55:44.081 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDT1etm09plCxAAGvJYF6JrKw49FtXb71ox3A==", 2024-09-19T07:55:44.081 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:55:44.081 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:55:44.081 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:55:44.081 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:55:44.081 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.081 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:44.081 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.081 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-19T07:55:44.081 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDW1etmGy8yHRAAlcDsHLnkjtS6sCCR8Olv2A==", 2024-09-19T07:55:44.081 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:55:44.082 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:55:44.082 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:55:44.082 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:55:44.082 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:44.082 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.082 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-19T07:55:44.082 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDZ1etmMhGTHxAATQaW2kgJhHKftutQQtUa6A==", 2024-09-19T07:55:44.082 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:55:44.082 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:55:44.082 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:55:44.082 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:55:44.082 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:44.083 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.083 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-19T07:55:44.083 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDc1etmD3sLIhAAIdY00buvb3bIFA5ZpgxxjA==", 2024-09-19T07:55:44.083 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:55:44.083 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:55:44.083 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:55:44.083 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:55:44.083 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:44.083 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.083 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-19T07:55:44.083 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDi1etmgutcLRAAHJEl5SfQ7N0T4YoYeBfrQg==", 2024-09-19T07:55:44.084 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:55:44.084 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:55:44.084 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:55:44.084 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:55:44.084 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:44.084 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.084 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-19T07:55:44.084 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDm1etmCV3xARAA+5yhQEmIpyNId3RL1Xsxhw==", 2024-09-19T07:55:44.084 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:55:44.084 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:55:44.084 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:55:44.084 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:55:44.085 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:44.085 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.085 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-19T07:55:44.085 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDp1etm9XC+AxAA+Lrs0ExfSKaixgOxfX8Nug==", 2024-09-19T07:55:44.085 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:55:44.085 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:55:44.085 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:55:44.085 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:55:44.085 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:44.085 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.085 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-19T07:55:44.085 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDs1etmhmvMBhAAhvW/udIE2gCd8IrJCSFuvw==", 2024-09-19T07:55:44.086 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:55:44.086 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-19T07:55:44.086 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-19T07:55:44.086 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:55:44.086 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.086 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:44.086 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.086 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-19T07:55:44.086 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA==", 2024-09-19T07:55:44.086 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:55:44.086 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:55:44.086 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-19T07:55:44.086 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T07:55:44.087 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:44.087 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.087 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-19T07:55:44.087 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmc05KLxAAPQl6wOHqFIf8LVxskiuLVg==", 2024-09-19T07:55:44.087 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:55:44.087 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:55:44.087 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:55:44.087 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:55:44.087 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:55:44.087 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:44.087 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.087 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-19T07:55:44.088 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmjFsCLBAANa7gCMvDw84Ovo7u6RRJ3Q==", 2024-09-19T07:55:44.088 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:55:44.088 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:55:44.088 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:55:44.088 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:55:44.088 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:55:44.088 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:44.088 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.088 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-19T07:55:44.088 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmdUbgNBAAfwstyOeeoZfetgVZu4z5xg==", 2024-09-19T07:55:44.088 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:55:44.088 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-19T07:55:44.089 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-19T07:55:44.089 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-19T07:55:44.089 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-19T07:55:44.089 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:44.089 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.089 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-19T07:55:44.089 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDQ1etm9/vTLhAA9PRfAOA8R4oXp3x0dU8l3g==", 2024-09-19T07:55:44.089 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:55:44.089 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:55:44.089 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-19T07:55:44.089 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-19T07:55:44.089 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:55:44.089 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:44.090 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.090 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-19T07:55:44.090 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmNQ68CxAANkK+D3u/oTEz8Su4iuZrCg==", 2024-09-19T07:55:44.090 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:55:44.090 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:55:44.090 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:55:44.090 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:55:44.090 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:44.090 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.090 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-19T07:55:44.090 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDR1etmp+gSCBAAGXzmKfFNd6K2fWeqYhlHAg==", 2024-09-19T07:55:44.090 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-19T07:55:44.091 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-19T07:55:44.091 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-19T07:55:44.091 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-19T07:55:44.091 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.091 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.091 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T07:55:44.091 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:55:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.079+0000 7f925345c640 1 -- 172.21.15.16:0/1945269640 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f923006c2b0 msgr2=0x7f923006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.079+0000 7f925345c640 1 --2- 172.21.15.16:0/1945269640 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f923006c2b0 0x7f923006e770 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f923c000c00 tx=0x7f923c009040 comp rx=0 tx=0).stop 2024-09-19T07:55:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.079+0000 7f925345c640 1 -- 172.21.15.16:0/1945269640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92380039b0 msgr2=0x7f924c1528d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.079+0000 7f925345c640 1 --2- 172.21.15.16:0/1945269640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92380039b0 0x7f924c1528d0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f924802abb0 tx=0x7f9248004730 comp rx=0 tx=0).stop 2024-09-19T07:55:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.079+0000 7f925345c640 1 -- 172.21.15.16:0/1945269640 shutdown_connections 2024-09-19T07:55:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.079+0000 7f925345c640 1 --2- 172.21.15.16:0/1945269640 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f923006c2b0 0x7f923006e770 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.079+0000 7f925345c640 1 --2- 172.21.15.16:0/1945269640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f924c0cd290 0x7f924c0cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.079+0000 7f925345c640 1 --2- 172.21.15.16:0/1945269640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f924c0b9800 0x7f924c0b9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.079+0000 7f925345c640 1 --2- 172.21.15.16:0/1945269640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92380039b0 0x7f924c1528d0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.079+0000 7f925345c640 1 -- 172.21.15.16:0/1945269640 >> 172.21.15.16:0/1945269640 conn(0x7f924c09d760 msgr2=0x7f924c06de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.079+0000 7f925345c640 1 -- 172.21.15.16:0/1945269640 shutdown_connections 2024-09-19T07:55:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.079+0000 7f925345c640 1 -- 172.21.15.16:0/1945269640 wait complete. 2024-09-19T07:55:44.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T07:55:44.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.195+0000 7fb3b14e4640 1 Processor -- start 2024-09-19T07:55:44.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.195+0000 7fb3b14e4640 1 -- start start 2024-09-19T07:55:44.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.196+0000 7fb3b14e4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb3ac15f0d0 0x7fb3ac15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.196+0000 7fb3b14e4640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb3ac0d0070 0x7fb3ac0d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.196+0000 7fb3b14e4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3ac0d1940 0x7fb3ac0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.196+0000 7fb3b14e4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fb3ac0cf2b0 con 0x7fb3ac15f0d0 2024-09-19T07:55:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.196+0000 7fb3b14e4640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb3ac0cf590 con 0x7fb3ac0d0070 2024-09-19T07:55:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.196+0000 7fb3b14e4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb3ac0cf870 con 0x7fb3ac0d1940 2024-09-19T07:55:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.196+0000 7fb3aaffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb3ac15f0d0 0x7fb3ac15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.196+0000 7fb3aaffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb3ac15f0d0 0x7fb3ac15f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49616/0 (socket says 172.21.15.16:49616) 2024-09-19T07:55:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.196+0000 7fb3aaffd640 1 -- 172.21.15.16:0/4006480006 learned_addr learned my addr 172.21.15.16:0/4006480006 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.196+0000 7fb3ab7fe640 1 --2- 172.21.15.16:0/4006480006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3ac0d1940 0x7fb3ac0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.196+0000 7fb3aa7fc640 1 --2- 172.21.15.16:0/4006480006 >> v2:172.21.15.146:3300/0 conn(0x7fb3ac0d0070 0x7fb3ac0d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:44.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.197+0000 7fb3aaffd640 1 -- 172.21.15.16:0/4006480006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3ac0d1940 msgr2=0x7fb3ac0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:44.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.197+0000 7fb3aaffd640 1 --2- 172.21.15.16:0/4006480006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3ac0d1940 0x7fb3ac0d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.197+0000 7fb3aaffd640 1 -- 172.21.15.16:0/4006480006 >> v2:172.21.15.146:3300/0 conn(0x7fb3ac0d0070 msgr2=0x7fb3ac0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:44.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.197+0000 7fb3aaffd640 1 --2- 172.21.15.16:0/4006480006 >> v2:172.21.15.146:3300/0 conn(0x7fb3ac0d0070 0x7fb3ac0d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.197+0000 7fb3aaffd640 1 -- 172.21.15.16:0/4006480006 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ac0cfb50 con 0x7fb3ac15f0d0 2024-09-19T07:55:44.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.197+0000 7fb3aaffd640 1 --2- 172.21.15.16:0/4006480006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac15f0d0 0x7fb3ac15f4b0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac162fc0 tx=0x7fb39c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ffe47751e128d68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:44.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.197+0000 7fb3a9ffb640 1 -- 172.21.15.16:0/4006480006 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c036070 con 0x7fb3ac15f0d0 2024-09-19T07:55:44.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.197+0000 7fb3a9ffb640 1 -- 172.21.15.16:0/4006480006 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb39c031600 con 0x7fb3ac15f0d0 2024-09-19T07:55:44.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.197+0000 7fb3a9ffb640 1 -- 172.21.15.16:0/4006480006 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c0319d0 con 0x7fb3ac15f0d0 2024-09-19T07:55:44.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.198+0000 7fb3b14e4640 1 -- 172.21.15.16:0/4006480006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac15f0d0 msgr2=0x7fb3ac15f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:44.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.198+0000 7fb3b14e4640 1 --2- 172.21.15.16:0/4006480006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac15f0d0 0x7fb3ac15f4b0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac162fc0 tx=0x7fb39c02a610 comp rx=0 tx=0).stop 2024-09-19T07:55:44.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.198+0000 7fb3b14e4640 1 -- 172.21.15.16:0/4006480006 shutdown_connections 2024-09-19T07:55:44.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.198+0000 7fb3b14e4640 1 --2- 172.21.15.16:0/4006480006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3ac0d1940 0x7fb3ac0d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.198+0000 7fb3b14e4640 1 --2- 172.21.15.16:0/4006480006 >> v2:172.21.15.146:3300/0 conn(0x7fb3ac0d0070 0x7fb3ac0d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.198+0000 7fb3b14e4640 1 --2- 172.21.15.16:0/4006480006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac15f0d0 0x7fb3ac15f4b0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.198+0000 7fb3b14e4640 1 -- 172.21.15.16:0/4006480006 >> 172.21.15.16:0/4006480006 conn(0x7fb3ac09d760 msgr2=0x7fb3ac08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:44.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.198+0000 7fb3b14e4640 1 -- 172.21.15.16:0/4006480006 shutdown_connections 2024-09-19T07:55:44.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.198+0000 7fb3b14e4640 1 -- 172.21.15.16:0/4006480006 wait complete. 2024-09-19T07:55:44.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.199+0000 7fb3b14e4640 1 Processor -- start 2024-09-19T07:55:44.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.199+0000 7fb3b14e4640 1 -- start start 2024-09-19T07:55:44.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.200+0000 7fb3b14e4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3ac0d0070 0x7fb3ac1521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:44.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.200+0000 7fb3b14e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac0d1940 0x7fb3ac1546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:44.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.200+0000 7fb3b14e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3ac15f0d0 0x7fb3ac160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:44.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.200+0000 7fb3b14e4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac0d47f0 con 0x7fb3ac0d1940 2024-09-19T07:55:44.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.200+0000 7fb3b14e4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac0d4670 con 0x7fb3ac15f0d0 2024-09-19T07:55:44.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.200+0000 7fb3b14e4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb3ac0d4970 con 0x7fb3ac0d0070 2024-09-19T07:55:44.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.200+0000 7fb3aa7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac0d1940 0x7fb3ac1546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:44.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.200+0000 7fb3aa7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac0d1940 0x7fb3ac1546e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49628/0 (socket says 172.21.15.16:49628) 2024-09-19T07:55:44.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.200+0000 7fb3aa7fc640 1 -- 172.21.15.16:0/1625099838 learned_addr learned my addr 172.21.15.16:0/1625099838 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:44.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.200+0000 7fb3aaffd640 1 --2- 172.21.15.16:0/1625099838 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3ac0d0070 0x7fb3ac1521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:44.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.200+0000 7fb3ab7fe640 1 --2- 172.21.15.16:0/1625099838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3ac15f0d0 0x7fb3ac160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:44.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.201+0000 7fb3aa7fc640 1 -- 172.21.15.16:0/1625099838 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3ac0d0070 msgr2=0x7fb3ac1521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:44.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.201+0000 7fb3aa7fc640 1 --2- 172.21.15.16:0/1625099838 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3ac0d0070 0x7fb3ac1521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.201+0000 7fb3aa7fc640 1 -- 172.21.15.16:0/1625099838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3ac15f0d0 msgr2=0x7fb3ac160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:44.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.201+0000 7fb3aa7fc640 1 --2- 172.21.15.16:0/1625099838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3ac15f0d0 0x7fb3ac160500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.201+0000 7fb3aa7fc640 1 -- 172.21.15.16:0/1625099838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ac1f4960 con 0x7fb3ac0d1940 2024-09-19T07:55:44.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.201+0000 7fb3aa7fc640 1 --2- 172.21.15.16:0/1625099838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac0d1940 0x7fb3ac1546e0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fb39400d800 tx=0x7fb39400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:44.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.201+0000 7fb3a37fe640 1 -- 172.21.15.16:0/1625099838 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394018030 con 0x7fb3ac0d1940 2024-09-19T07:55:44.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.201+0000 7fb3a37fe640 1 -- 172.21.15.16:0/1625099838 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3940096a0 con 0x7fb3ac0d1940 2024-09-19T07:55:44.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.201+0000 7fb3a37fe640 1 -- 172.21.15.16:0/1625099838 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394009870 con 0x7fb3ac0d1940 2024-09-19T07:55:44.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.201+0000 7fb3b14e4640 1 -- 172.21.15.16:0/1625099838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3ac153920 con 0x7fb3ac0d1940 2024-09-19T07:55:44.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.201+0000 7fb3b14e4640 1 -- 172.21.15.16:0/1625099838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3ac153ee0 con 0x7fb3ac0d1940 2024-09-19T07:55:44.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.202+0000 7fb3b14e4640 1 -- 172.21.15.16:0/1625099838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3ac0d1400 con 0x7fb3ac0d1940 2024-09-19T07:55:44.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.203+0000 7fb3a37fe640 1 -- 172.21.15.16:0/1625099838 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb39401b070 con 0x7fb3ac0d1940 2024-09-19T07:55:44.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.203+0000 7fb3a37fe640 1 --2- 172.21.15.16:0/1625099838 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb38c06c2b0 0x7fb38c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:44.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.203+0000 7fb3a37fe640 1 -- 172.21.15.16:0/1625099838 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5249+0+0 (secure 0 0 0) 0x7fb39408b340 con 0x7fb3ac0d1940 2024-09-19T07:55:44.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.203+0000 7fb3aaffd640 1 --2- 172.21.15.16:0/1625099838 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb38c06c2b0 0x7fb38c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:44.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.204+0000 7fb3aaffd640 1 --2- 172.21.15.16:0/1625099838 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb38c06c2b0 0x7fb38c06e770 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb39c033000 tx=0x7fb39c009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.207+0000 7fb3a37fe640 1 -- 172.21.15.16:0/1625099838 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb3ac1f4960 con 0x7fb3ac0d1940 2024-09-19T07:55:44.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.385+0000 7fb3b14e4640 1 -- 172.21.15.16:0/1625099838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb3ac15be80 con 0x7fb3ac0d1940 2024-09-19T07:55:44.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.385+0000 7fb3a37fe640 1 -- 172.21.15.16:0/1625099838 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v148) ==== 84+0+65 (secure 0 0 0) 0x7fb394059620 con 0x7fb3ac0d1940 2024-09-19T07:55:44.386 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:44.386 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-19T07:55:44.386 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:55:44.386 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-19T07:55:44.386 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-19T07:55:44.386 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:44.386 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-19T07:55:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.387+0000 7fb3b14e4640 1 -- 172.21.15.16:0/1625099838 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb38c06c2b0 msgr2=0x7fb38c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.387+0000 7fb3b14e4640 1 --2- 172.21.15.16:0/1625099838 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb38c06c2b0 0x7fb38c06e770 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb39c033000 tx=0x7fb39c009000 comp rx=0 tx=0).stop 2024-09-19T07:55:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.387+0000 7fb3b14e4640 1 -- 172.21.15.16:0/1625099838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac0d1940 msgr2=0x7fb3ac1546e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.387+0000 7fb3b14e4640 1 --2- 172.21.15.16:0/1625099838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac0d1940 0x7fb3ac1546e0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fb39400d800 tx=0x7fb39400dcc0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.387+0000 7fb3b14e4640 1 -- 172.21.15.16:0/1625099838 shutdown_connections 2024-09-19T07:55:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.387+0000 7fb3b14e4640 1 --2- 172.21.15.16:0/1625099838 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fb38c06c2b0 0x7fb38c06e770 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.387+0000 7fb3b14e4640 1 --2- 172.21.15.16:0/1625099838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3ac15f0d0 0x7fb3ac160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.387+0000 7fb3b14e4640 1 --2- 172.21.15.16:0/1625099838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3ac0d1940 0x7fb3ac1546e0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.387+0000 7fb3b14e4640 1 --2- 172.21.15.16:0/1625099838 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3ac0d0070 0x7fb3ac1521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.388+0000 7fb3b14e4640 1 -- 172.21.15.16:0/1625099838 >> 172.21.15.16:0/1625099838 conn(0x7fb3ac09d760 msgr2=0x7fb3ac15c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.388+0000 7fb3b14e4640 1 -- 172.21.15.16:0/1625099838 shutdown_connections 2024-09-19T07:55:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.388+0000 7fb3b14e4640 1 -- 172.21.15.16:0/1625099838 wait complete. 2024-09-19T07:55:44.399 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T07:55:44.399 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T07:55:44.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.505+0000 7f65116af640 1 Processor -- start 2024-09-19T07:55:44.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.505+0000 7f65116af640 1 -- start start 2024-09-19T07:55:44.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.505+0000 7f65116af640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f650c1619b0 0x7f650c0d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:44.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.505+0000 7f65116af640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f650c0b0a80 0x7f650c0d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:44.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.505+0000 7f65116af640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f650c06c990 con 0x7f650c161d90 2024-09-19T07:55:44.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.506+0000 7f65116af640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f650c0d0ec0 con 0x7f650c1619b0 2024-09-19T07:55:44.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.506+0000 7f65116af640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f650c0cec10 con 0x7f650c0b0a80 2024-09-19T07:55:44.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.506+0000 7f650affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f650c1619b0 0x7f650c0d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:44.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.506+0000 7f650b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f650c161d90 0x7f650c0cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60672/0 (socket says 172.21.15.16:60672) 2024-09-19T07:55:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.506+0000 7f650affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f650c1619b0 0x7f650c0d0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49644/0 (socket says 172.21.15.16:49644) 2024-09-19T07:55:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.506+0000 7f650b7fe640 1 -- 172.21.15.16:0/3497009786 learned_addr learned my addr 172.21.15.16:0/3497009786 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.506+0000 7f650a7fc640 1 --2- 172.21.15.16:0/3497009786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f650c0b0a80 0x7f650c0d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.506+0000 7f650affd640 1 -- 172.21.15.16:0/3497009786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f650c0b0a80 msgr2=0x7f650c0d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.506+0000 7f650affd640 1 --2- 172.21.15.16:0/3497009786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f650c0b0a80 0x7f650c0d0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.506+0000 7f650affd640 1 -- 172.21.15.16:0/3497009786 >> v1:172.21.15.16:6789/0 conn(0x7f650c161d90 legacy=0x7f650c0cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.506+0000 7f650affd640 1 -- 172.21.15.16:0/3497009786 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650c0ce930 con 0x7f650c1619b0 2024-09-19T07:55:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.506+0000 7f6509ffb640 1 -- 172.21.15.16:0/3497009786 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1202824019 0 0) 0x7f650c06c990 con 0x7f650c161d90 2024-09-19T07:55:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.507+0000 7f650affd640 1 --2- 172.21.15.16:0/3497009786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f650c1619b0 0x7f650c0d0440 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0097a0 tx=0x7f64fc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44dd2533122f0e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.507+0000 7f6509ffb640 1 -- 172.21.15.16:0/3497009786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc037070 con 0x7f650c1619b0 2024-09-19T07:55:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.507+0000 7f6509ffb640 1 -- 172.21.15.16:0/3497009786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64fc032440 con 0x7f650c1619b0 2024-09-19T07:55:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.507+0000 7f6509ffb640 1 -- 172.21.15.16:0/3497009786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc032810 con 0x7f650c1619b0 2024-09-19T07:55:44.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.507+0000 7f65116af640 1 -- 172.21.15.16:0/3497009786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f650c1619b0 msgr2=0x7f650c0d0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:44.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.507+0000 7f65116af640 1 --2- 172.21.15.16:0/3497009786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f650c1619b0 0x7f650c0d0440 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0097a0 tx=0x7f64fc02a180 comp rx=0 tx=0).stop 2024-09-19T07:55:44.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.507+0000 7f65116af640 1 -- 172.21.15.16:0/3497009786 shutdown_connections 2024-09-19T07:55:44.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.507+0000 7f65116af640 1 --2- 172.21.15.16:0/3497009786 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f650c0b0a80 0x7f650c0d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.507+0000 7f65116af640 1 --2- 172.21.15.16:0/3497009786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f650c1619b0 0x7f650c0d0440 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.507+0000 7f65116af640 1 -- 172.21.15.16:0/3497009786 >> 172.21.15.16:0/3497009786 conn(0x7f650c09a9f0 msgr2=0x7f650c0b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:44.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.507+0000 7f65116af640 1 -- 172.21.15.16:0/3497009786 shutdown_connections 2024-09-19T07:55:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.508+0000 7f65116af640 1 -- 172.21.15.16:0/3497009786 wait complete. 2024-09-19T07:55:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.509+0000 7f65116af640 1 Processor -- start 2024-09-19T07:55:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.509+0000 7f65116af640 1 -- start start 2024-09-19T07:55:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.509+0000 7f65116af640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f650c0b0a80 0x7f650c0b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f65116af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f650c1619b0 0x7f650c0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f65116af640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f650c161d90 0x7f650c0c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f65116af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f650c06c620 con 0x7f650c1619b0 2024-09-19T07:55:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f65116af640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f650c0b0820 con 0x7f650c0b0a80 2024-09-19T07:55:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f65116af640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f650c0ccce0 con 0x7f650c161d90 2024-09-19T07:55:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f650a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f650c1619b0 0x7f650c0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f650a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f650c1619b0 0x7f650c0caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49648/0 (socket says 172.21.15.16:49648) 2024-09-19T07:55:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f650a7fc640 1 -- 172.21.15.16:0/1151227903 learned_addr learned my addr 172.21.15.16:0/1151227903 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f650affd640 1 --2- 172.21.15.16:0/1151227903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f650c0b0a80 0x7f650c0b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f650b7fe640 1 --2- 172.21.15.16:0/1151227903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f650c161d90 0x7f650c0c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f650a7fc640 1 -- 172.21.15.16:0/1151227903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f650c161d90 msgr2=0x7f650c0c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f650a7fc640 1 --2- 172.21.15.16:0/1151227903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f650c161d90 0x7f650c0c66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f650a7fc640 1 -- 172.21.15.16:0/1151227903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f650c0b0a80 msgr2=0x7f650c0b6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f650a7fc640 1 --2- 172.21.15.16:0/1151227903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f650c0b0a80 0x7f650c0b6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f650a7fc640 1 -- 172.21.15.16:0/1151227903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650c0b5f70 con 0x7f650c1619b0 2024-09-19T07:55:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f650a7fc640 1 --2- 172.21.15.16:0/1151227903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f650c1619b0 0x7f650c0caea0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f650000c820 tx=0x7f650000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f64f37fe640 1 -- 172.21.15.16:0/1151227903 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500004230 con 0x7f650c1619b0 2024-09-19T07:55:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f64f37fe640 1 -- 172.21.15.16:0/1151227903 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65000043d0 con 0x7f650c1619b0 2024-09-19T07:55:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f65116af640 1 -- 172.21.15.16:0/1151227903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f650c0b6250 con 0x7f650c1619b0 2024-09-19T07:55:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f64f37fe640 1 -- 172.21.15.16:0/1151227903 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500007a70 con 0x7f650c1619b0 2024-09-19T07:55:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.510+0000 7f65116af640 1 -- 172.21.15.16:0/1151227903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f650c0b6810 con 0x7f650c1619b0 2024-09-19T07:55:44.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.511+0000 7f65116af640 1 -- 172.21.15.16:0/1151227903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f650c0cd330 con 0x7f650c1619b0 2024-09-19T07:55:44.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.512+0000 7f64f37fe640 1 -- 172.21.15.16:0/1151227903 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6500007c10 con 0x7f650c1619b0 2024-09-19T07:55:44.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.512+0000 7f64f37fe640 1 --2- 172.21.15.16:0/1151227903 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f64f806c2b0 0x7f64f806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:44.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.513+0000 7f64f37fe640 1 -- 172.21.15.16:0/1151227903 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5249+0+0 (secure 0 0 0) 0x7f650008b180 con 0x7f650c1619b0 2024-09-19T07:55:44.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.513+0000 7f650affd640 1 --2- 172.21.15.16:0/1151227903 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f64f806c2b0 0x7f64f806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:44.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.513+0000 7f650affd640 1 --2- 172.21.15.16:0/1151227903 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f64f806c2b0 0x7f64f806e770 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f64fc034000 tx=0x7f64fc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:44.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.517+0000 7f64f37fe640 1 -- 172.21.15.16:0/1151227903 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6500015020 con 0x7f650c1619b0 2024-09-19T07:55:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.696+0000 7f65116af640 1 -- 172.21.15.16:0/1151227903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f650c070640 con 0x7f650c1619b0 2024-09-19T07:55:44.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.778+0000 7f64f37fe640 1 -- 172.21.15.16:0/1151227903 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v149) ==== 138+0+0 (secure 0 0 0) 0x7f65000594f0 con 0x7f650c1619b0 2024-09-19T07:55:44.779 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-19T07:55:44.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.781+0000 7f65116af640 1 -- 172.21.15.16:0/1151227903 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f64f806c2b0 msgr2=0x7f64f806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:44.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.781+0000 7f65116af640 1 --2- 172.21.15.16:0/1151227903 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f64f806c2b0 0x7f64f806e770 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f64fc034000 tx=0x7f64fc02e000 comp rx=0 tx=0).stop 2024-09-19T07:55:44.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.781+0000 7f65116af640 1 -- 172.21.15.16:0/1151227903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f650c1619b0 msgr2=0x7f650c0caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:44.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.781+0000 7f65116af640 1 --2- 172.21.15.16:0/1151227903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f650c1619b0 0x7f650c0caea0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f650000c820 tx=0x7f650000cce0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.781+0000 7f65116af640 1 -- 172.21.15.16:0/1151227903 shutdown_connections 2024-09-19T07:55:44.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.781+0000 7f65116af640 1 --2- 172.21.15.16:0/1151227903 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f64f806c2b0 0x7f64f806e770 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.781+0000 7f65116af640 1 --2- 172.21.15.16:0/1151227903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f650c161d90 0x7f650c0c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.781+0000 7f65116af640 1 --2- 172.21.15.16:0/1151227903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f650c1619b0 0x7f650c0caea0 secure :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f650000c820 tx=0x7f650000cce0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.781+0000 7f65116af640 1 --2- 172.21.15.16:0/1151227903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f650c0b0a80 0x7f650c0b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.781+0000 7f65116af640 1 -- 172.21.15.16:0/1151227903 >> 172.21.15.16:0/1151227903 conn(0x7f650c09a9f0 msgr2=0x7f650c0ae270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:44.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.782+0000 7f65116af640 1 -- 172.21.15.16:0/1151227903 shutdown_connections 2024-09-19T07:55:44.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.782+0000 7f65116af640 1 -- 172.21.15.16:0/1151227903 wait complete. 2024-09-19T07:55:44.799 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T07:55:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.908+0000 7faca0d52640 1 Processor -- start 2024-09-19T07:55:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.908+0000 7faca0d52640 1 -- start start 2024-09-19T07:55:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.909+0000 7faca0d52640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fac9c15e0a0 0x7fac9c162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.909+0000 7faca0d52640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fac9c0d43e0 0x7fac9c0d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.909+0000 7faca0d52640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac9c15f250 con 0x7fac9c0b4b70 2024-09-19T07:55:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.909+0000 7faca0d52640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fac9c0d4d60 con 0x7fac9c0d43e0 2024-09-19T07:55:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.909+0000 7faca0d52640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fac9c15f530 con 0x7fac9c15e0a0 2024-09-19T07:55:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.909+0000 7fac91d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fac9c0d43e0 0x7fac9c0d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.909+0000 7fac91d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fac9c0d43e0 0x7fac9c0d4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49654/0 (socket says 172.21.15.16:49654) 2024-09-19T07:55:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.909+0000 7fac91d9b640 1 -- 172.21.15.16:0/421386867 learned_addr learned my addr 172.21.15.16:0/421386867 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.909+0000 7fac9ad9d640 1 --1- 172.21.15.16:0/421386867 >> v1:172.21.15.16:6789/0 conn(0x7fac9c0b4b70 0x7fac9c0b5000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60674/0 (socket says 172.21.15.16:60674) 2024-09-19T07:55:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.909+0000 7fac9a59c640 1 --2- 172.21.15.16:0/421386867 >> v2:172.21.15.146:3300/0 conn(0x7fac9c15e0a0 0x7fac9c162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.909+0000 7fac91d9b640 1 -- 172.21.15.16:0/421386867 >> v2:172.21.15.146:3300/0 conn(0x7fac9c15e0a0 msgr2=0x7fac9c162a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.909+0000 7fac91d9b640 1 --2- 172.21.15.16:0/421386867 >> v2:172.21.15.146:3300/0 conn(0x7fac9c15e0a0 0x7fac9c162a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.909+0000 7fac91d9b640 1 -- 172.21.15.16:0/421386867 >> v1:172.21.15.16:6789/0 conn(0x7fac9c0b4b70 legacy=0x7fac9c0b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.909+0000 7fac99d9b640 1 -- 172.21.15.16:0/421386867 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1624122143 0 0) 0x7fac9c15f250 con 0x7fac9c0b4b70 2024-09-19T07:55:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.909+0000 7fac91d9b640 1 -- 172.21.15.16:0/421386867 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c15f810 con 0x7fac9c0d43e0 2024-09-19T07:55:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.909+0000 7fac9a59c640 1 --2- 172.21.15.16:0/421386867 >> v2:172.21.15.146:3300/0 conn(0x7fac9c15e0a0 0x7fac9c162a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.910+0000 7fac91d9b640 1 --2- 172.21.15.16:0/421386867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac9c0d43e0 0x7fac9c0d4820 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0d43b0 tx=0x7fac8402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756e5d9d6bd068ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.910+0000 7fac99d9b640 1 -- 172.21.15.16:0/421386867 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84004030 con 0x7fac9c0d43e0 2024-09-19T07:55:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.910+0000 7fac99d9b640 1 -- 172.21.15.16:0/421386867 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac840041d0 con 0x7fac9c0d43e0 2024-09-19T07:55:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.910+0000 7fac99d9b640 1 -- 172.21.15.16:0/421386867 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84003610 con 0x7fac9c0d43e0 2024-09-19T07:55:44.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.910+0000 7faca0d52640 1 -- 172.21.15.16:0/421386867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac9c0d43e0 msgr2=0x7fac9c0d4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:44.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.910+0000 7faca0d52640 1 --2- 172.21.15.16:0/421386867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac9c0d43e0 0x7fac9c0d4820 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0d43b0 tx=0x7fac8402a270 comp rx=0 tx=0).stop 2024-09-19T07:55:44.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.911+0000 7faca0d52640 1 -- 172.21.15.16:0/421386867 shutdown_connections 2024-09-19T07:55:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.911+0000 7faca0d52640 1 --2- 172.21.15.16:0/421386867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac9c0d43e0 0x7fac9c0d4820 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.911+0000 7faca0d52640 1 --2- 172.21.15.16:0/421386867 >> v2:172.21.15.146:3300/0 conn(0x7fac9c15e0a0 0x7fac9c162a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.911+0000 7faca0d52640 1 -- 172.21.15.16:0/421386867 >> 172.21.15.16:0/421386867 conn(0x7fac9c09d770 msgr2=0x7fac9c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.911+0000 7faca0d52640 1 -- 172.21.15.16:0/421386867 shutdown_connections 2024-09-19T07:55:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.911+0000 7faca0d52640 1 -- 172.21.15.16:0/421386867 wait complete. 2024-09-19T07:55:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.912+0000 7faca0d52640 1 Processor -- start 2024-09-19T07:55:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.912+0000 7faca0d52640 1 -- start start 2024-09-19T07:55:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.913+0000 7faca0d52640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac9c0b4b70 0x7fac9c0cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.913+0000 7faca0d52640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac9c0d43e0 0x7fac9c0b8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.913+0000 7faca0d52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac9c15e0a0 0x7fac9c0b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.913+0000 7faca0d52640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c15e600 con 0x7fac9c15e0a0 2024-09-19T07:55:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.913+0000 7faca0d52640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c15e480 con 0x7fac9c0d43e0 2024-09-19T07:55:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.913+0000 7faca0d52640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fac9c15e780 con 0x7fac9c0b4b70 2024-09-19T07:55:44.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.913+0000 7fac9ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac9c15e0a0 0x7fac9c0b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:44.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.913+0000 7fac9ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac9c15e0a0 0x7fac9c0b8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49666/0 (socket says 172.21.15.16:49666) 2024-09-19T07:55:44.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.913+0000 7fac9ad9d640 1 -- 172.21.15.16:0/430820942 learned_addr learned my addr 172.21.15.16:0/430820942 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:44.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.913+0000 7fac91d9b640 1 --2- 172.21.15.16:0/430820942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac9c0d43e0 0x7fac9c0b8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:44.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.913+0000 7fac9a59c640 1 --2- 172.21.15.16:0/430820942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac9c0b4b70 0x7fac9c0cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:44.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.913+0000 7fac9ad9d640 1 -- 172.21.15.16:0/430820942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac9c0b4b70 msgr2=0x7fac9c0cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:44.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.913+0000 7fac9ad9d640 1 --2- 172.21.15.16:0/430820942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac9c0b4b70 0x7fac9c0cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.913+0000 7fac9ad9d640 1 -- 172.21.15.16:0/430820942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac9c0d43e0 msgr2=0x7fac9c0b8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:44.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.913+0000 7fac9ad9d640 1 --2- 172.21.15.16:0/430820942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac9c0d43e0 0x7fac9c0b8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:44.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.913+0000 7fac9ad9d640 1 -- 172.21.15.16:0/430820942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c0cfb70 con 0x7fac9c15e0a0 2024-09-19T07:55:44.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.913+0000 7fac91d9b640 1 --2- 172.21.15.16:0/430820942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac9c0d43e0 0x7fac9c0b8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:44.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.914+0000 7fac9ad9d640 1 --2- 172.21.15.16:0/430820942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac9c15e0a0 0x7fac9c0b8dd0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fac9400c7d0 tx=0x7fac9400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:44.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.914+0000 7fac937fe640 1 -- 172.21.15.16:0/430820942 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94018070 con 0x7fac9c15e0a0 2024-09-19T07:55:44.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.914+0000 7fac937fe640 1 -- 172.21.15.16:0/430820942 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac9400a640 con 0x7fac9c15e0a0 2024-09-19T07:55:44.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.914+0000 7faca0d52640 1 -- 172.21.15.16:0/430820942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9c0cfeb0 con 0x7fac9c15e0a0 2024-09-19T07:55:44.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.914+0000 7fac937fe640 1 -- 172.21.15.16:0/430820942 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9400aa30 con 0x7fac9c15e0a0 2024-09-19T07:55:44.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.914+0000 7faca0d52640 1 -- 172.21.15.16:0/430820942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9c0d06f0 con 0x7fac9c15e0a0 2024-09-19T07:55:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.915+0000 7faca0d52640 1 -- 172.21.15.16:0/430820942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac9c15ed40 con 0x7fac9c15e0a0 2024-09-19T07:55:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.915+0000 7fac937fe640 1 -- 172.21.15.16:0/430820942 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fac9401b070 con 0x7fac9c15e0a0 2024-09-19T07:55:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.916+0000 7fac937fe640 1 --2- 172.21.15.16:0/430820942 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fac7c06c2b0 0x7fac7c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.916+0000 7fac937fe640 1 -- 172.21.15.16:0/430820942 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5619+0+0 (secure 0 0 0) 0x7fac9408a980 con 0x7fac9c15e0a0 2024-09-19T07:55:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.916+0000 7fac9a59c640 1 --2- 172.21.15.16:0/430820942 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fac7c06c2b0 0x7fac7c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.917+0000 7fac9a59c640 1 --2- 172.21.15.16:0/430820942 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fac7c06c2b0 0x7fac7c06e770 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00a950 tx=0x7fac8c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:44.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:44.920+0000 7fac937fe640 1 -- 172.21.15.16:0/430820942 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fac9c15ed40 con 0x7fac9c15e0a0 2024-09-19T07:55:45.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.096+0000 7faca0d52640 1 -- 172.21.15.16:0/430820942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fac9c0d4820 con 0x7fac9c15e0a0 2024-09-19T07:55:45.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.781+0000 7fac937fe640 1 -- 172.21.15.16:0/430820942 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v150) ==== 170+0+0 (secure 0 0 0) 0x7fac9c0cfeb0 con 0x7fac9c15e0a0 2024-09-19T07:55:45.782 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-19T07:55:45.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.785+0000 7faca0d52640 1 -- 172.21.15.16:0/430820942 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fac7c06c2b0 msgr2=0x7fac7c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:45.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.785+0000 7faca0d52640 1 --2- 172.21.15.16:0/430820942 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fac7c06c2b0 0x7fac7c06e770 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00a950 tx=0x7fac8c008040 comp rx=0 tx=0).stop 2024-09-19T07:55:45.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.785+0000 7faca0d52640 1 -- 172.21.15.16:0/430820942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac9c15e0a0 msgr2=0x7fac9c0b8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:45.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.785+0000 7faca0d52640 1 --2- 172.21.15.16:0/430820942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac9c15e0a0 0x7fac9c0b8dd0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fac9400c7d0 tx=0x7fac9400cc90 comp rx=0 tx=0).stop 2024-09-19T07:55:45.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.785+0000 7faca0d52640 1 -- 172.21.15.16:0/430820942 shutdown_connections 2024-09-19T07:55:45.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.785+0000 7faca0d52640 1 --2- 172.21.15.16:0/430820942 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fac7c06c2b0 0x7fac7c06e770 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:45.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.785+0000 7faca0d52640 1 --2- 172.21.15.16:0/430820942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac9c15e0a0 0x7fac9c0b8dd0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:45.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.785+0000 7faca0d52640 1 --2- 172.21.15.16:0/430820942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac9c0d43e0 0x7fac9c0b8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:45.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.785+0000 7faca0d52640 1 --2- 172.21.15.16:0/430820942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac9c0b4b70 0x7fac9c0cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:45.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.785+0000 7faca0d52640 1 -- 172.21.15.16:0/430820942 >> 172.21.15.16:0/430820942 conn(0x7fac9c09d770 msgr2=0x7fac9c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:45.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.785+0000 7faca0d52640 1 -- 172.21.15.16:0/430820942 shutdown_connections 2024-09-19T07:55:45.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.788+0000 7faca0d52640 1 -- 172.21.15.16:0/430820942 wait complete. 2024-09-19T07:55:45.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T07:55:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.928+0000 7f1738827640 1 Processor -- start 2024-09-19T07:55:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.928+0000 7f1738827640 1 -- start start 2024-09-19T07:55:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.929+0000 7f1738827640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f173415e2a0 0x7f173406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:45.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.929+0000 7f1738827640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f173415e680 0x7f17340cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:45.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.929+0000 7f1738827640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f173406d960 0x7f17340d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:45.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.929+0000 7f1738827640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f17340d1880 con 0x7f173415e680 2024-09-19T07:55:45.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.929+0000 7f1738827640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f17340d1b60 con 0x7f173406d960 2024-09-19T07:55:45.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.929+0000 7f1738827640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f17340cf2d0 con 0x7f173415e2a0 2024-09-19T07:55:45.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.929+0000 7f172bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f173415e680 0x7f17340cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:45.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.929+0000 7f172bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f173415e680 0x7f17340cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49680/0 (socket says 172.21.15.16:49680) 2024-09-19T07:55:45.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.929+0000 7f172bfff640 1 -- 172.21.15.16:0/3499470269 learned_addr learned my addr 172.21.15.16:0/3499470269 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:45.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.929+0000 7f173259c640 1 --2- 172.21.15.16:0/3499470269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f173415e2a0 0x7f173406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:45.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.929+0000 7f1732d9d640 1 --2- 172.21.15.16:0/3499470269 >> v2:172.21.15.146:3300/0 conn(0x7f173406d960 0x7f17340d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:45.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.929+0000 7f172bfff640 1 -- 172.21.15.16:0/3499470269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f173415e2a0 msgr2=0x7f173406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:45.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.929+0000 7f172bfff640 1 --2- 172.21.15.16:0/3499470269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f173415e2a0 0x7f173406d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:45.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.929+0000 7f172bfff640 1 -- 172.21.15.16:0/3499470269 >> v2:172.21.15.146:3300/0 conn(0x7f173406d960 msgr2=0x7f17340d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:45.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.929+0000 7f172bfff640 1 --2- 172.21.15.16:0/3499470269 >> v2:172.21.15.146:3300/0 conn(0x7f173406d960 0x7f17340d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:45.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.929+0000 7f172bfff640 1 -- 172.21.15.16:0/3499470269 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17340cf5b0 con 0x7f173415e680 2024-09-19T07:55:45.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.929+0000 7f1732d9d640 1 --2- 172.21.15.16:0/3499470269 >> v2:172.21.15.146:3300/0 conn(0x7f173406d960 0x7f17340d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:45.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.930+0000 7f172bfff640 1 --2- 172.21.15.16:0/3499470269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173415e680 0x7f17340cfad0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f173406cd80 tx=0x7f171c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3404c21b7c6979 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:45.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.930+0000 7f1731d9b640 1 -- 172.21.15.16:0/3499470269 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c037070 con 0x7f173415e680 2024-09-19T07:55:45.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.930+0000 7f1731d9b640 1 -- 172.21.15.16:0/3499470269 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f171c032440 con 0x7f173415e680 2024-09-19T07:55:45.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.930+0000 7f1731d9b640 1 -- 172.21.15.16:0/3499470269 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c032810 con 0x7f173415e680 2024-09-19T07:55:45.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.931+0000 7f1738827640 1 -- 172.21.15.16:0/3499470269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173415e680 msgr2=0x7f17340cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:45.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.931+0000 7f1738827640 1 --2- 172.21.15.16:0/3499470269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173415e680 0x7f17340cfad0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f173406cd80 tx=0x7f171c02a1e0 comp rx=0 tx=0).stop 2024-09-19T07:55:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.931+0000 7f1738827640 1 -- 172.21.15.16:0/3499470269 shutdown_connections 2024-09-19T07:55:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.931+0000 7f1738827640 1 --2- 172.21.15.16:0/3499470269 >> v2:172.21.15.146:3300/0 conn(0x7f173406d960 0x7f17340d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.931+0000 7f1738827640 1 --2- 172.21.15.16:0/3499470269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173415e680 0x7f17340cfad0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.931+0000 7f1738827640 1 --2- 172.21.15.16:0/3499470269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f173415e2a0 0x7f173406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.931+0000 7f1738827640 1 -- 172.21.15.16:0/3499470269 >> 172.21.15.16:0/3499470269 conn(0x7f173409d760 msgr2=0x7f173408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.931+0000 7f1738827640 1 -- 172.21.15.16:0/3499470269 shutdown_connections 2024-09-19T07:55:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.931+0000 7f1738827640 1 -- 172.21.15.16:0/3499470269 wait complete. 2024-09-19T07:55:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.932+0000 7f1738827640 1 Processor -- start 2024-09-19T07:55:45.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.932+0000 7f1738827640 1 -- start start 2024-09-19T07:55:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.933+0000 7f1738827640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f173406d960 0x7f17340cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.933+0000 7f1738827640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f173415e2a0 0x7f1734156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.933+0000 7f1738827640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173415e680 0x7f17340ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.933+0000 7f1738827640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17340d44f0 con 0x7f173415e680 2024-09-19T07:55:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.933+0000 7f1738827640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f17340d4370 con 0x7f173415e2a0 2024-09-19T07:55:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.933+0000 7f1738827640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f17340d4670 con 0x7f173406d960 2024-09-19T07:55:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.933+0000 7f1732d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173415e680 0x7f17340ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.933+0000 7f1732d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173415e680 0x7f17340ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49682/0 (socket says 172.21.15.16:49682) 2024-09-19T07:55:45.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.933+0000 7f1732d9d640 1 -- 172.21.15.16:0/3834557037 learned_addr learned my addr 172.21.15.16:0/3834557037 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:45.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.933+0000 7f173259c640 1 --2- 172.21.15.16:0/3834557037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f173406d960 0x7f17340cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:45.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.933+0000 7f172bfff640 1 --2- 172.21.15.16:0/3834557037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f173415e2a0 0x7f1734156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:45.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.934+0000 7f1732d9d640 1 -- 172.21.15.16:0/3834557037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f173406d960 msgr2=0x7f17340cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:45.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.934+0000 7f1732d9d640 1 --2- 172.21.15.16:0/3834557037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f173406d960 0x7f17340cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:45.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.934+0000 7f1732d9d640 1 -- 172.21.15.16:0/3834557037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f173415e2a0 msgr2=0x7f1734156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:45.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.934+0000 7f1732d9d640 1 --2- 172.21.15.16:0/3834557037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f173415e2a0 0x7f1734156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:45.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.934+0000 7f1732d9d640 1 -- 172.21.15.16:0/3834557037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17340cc530 con 0x7f173415e680 2024-09-19T07:55:45.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.934+0000 7f1732d9d640 1 --2- 172.21.15.16:0/3834557037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173415e680 0x7f17340ccbe0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f172c00cef0 tx=0x7f172c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:45.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.934+0000 7f172affd640 1 -- 172.21.15.16:0/3834557037 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c01a070 con 0x7f173415e680 2024-09-19T07:55:45.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.934+0000 7f172affd640 1 -- 172.21.15.16:0/3834557037 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f172c00d860 con 0x7f173415e680 2024-09-19T07:55:45.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.934+0000 7f1738827640 1 -- 172.21.15.16:0/3834557037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17340c9280 con 0x7f173415e680 2024-09-19T07:55:45.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.934+0000 7f172affd640 1 -- 172.21.15.16:0/3834557037 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c00dc50 con 0x7f173415e680 2024-09-19T07:55:45.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.934+0000 7f1738827640 1 -- 172.21.15.16:0/3834557037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17340cd120 con 0x7f173415e680 2024-09-19T07:55:45.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.935+0000 7f1738827640 1 -- 172.21.15.16:0/3834557037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17340d4a40 con 0x7f173415e680 2024-09-19T07:55:45.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.936+0000 7f172affd640 1 -- 172.21.15.16:0/3834557037 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f172c01d070 con 0x7f173415e680 2024-09-19T07:55:45.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.936+0000 7f172affd640 1 --2- 172.21.15.16:0/3834557037 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f171406c2b0 0x7f171406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:45.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.936+0000 7f172affd640 1 -- 172.21.15.16:0/3834557037 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6001+0+0 (secure 0 0 0) 0x7f172c08bd00 con 0x7f173415e680 2024-09-19T07:55:45.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.936+0000 7f173259c640 1 --2- 172.21.15.16:0/3834557037 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f171406c2b0 0x7f171406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:45.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.937+0000 7f173259c640 1 --2- 172.21.15.16:0/3834557037 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f171406c2b0 0x7f171406e770 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1724006fd0 tx=0x7f1724008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:45.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:45.941+0000 7f172affd640 1 -- 172.21.15.16:0/3834557037 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f172c059e40 con 0x7f173415e680 2024-09-19T07:55:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.153+0000 7f1738827640 1 -- 172.21.15.16:0/3834557037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f173406cf90 con 0x7f173415e680 2024-09-19T07:55:46.514 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-19T07:55:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.820+0000 7f172affd640 1 -- 172.21.15.16:0/3834557037 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v89) ==== 174+0+0 (secure 0 0 0) 0x7f172c05e140 con 0x7f173415e680 2024-09-19T07:55:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 23 and data pool 24 2024-09-19T07:55:46.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.826+0000 7f1738827640 1 -- 172.21.15.16:0/3834557037 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f171406c2b0 msgr2=0x7f171406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:46.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.826+0000 7f1738827640 1 --2- 172.21.15.16:0/3834557037 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f171406c2b0 0x7f171406e770 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1724006fd0 tx=0x7f1724008040 comp rx=0 tx=0).stop 2024-09-19T07:55:46.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.826+0000 7f1738827640 1 -- 172.21.15.16:0/3834557037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173415e680 msgr2=0x7f17340ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:46.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.826+0000 7f1738827640 1 --2- 172.21.15.16:0/3834557037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173415e680 0x7f17340ccbe0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f172c00cef0 tx=0x7f172c00a710 comp rx=0 tx=0).stop 2024-09-19T07:55:46.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.826+0000 7f1738827640 1 -- 172.21.15.16:0/3834557037 shutdown_connections 2024-09-19T07:55:46.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.826+0000 7f1738827640 1 --2- 172.21.15.16:0/3834557037 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f171406c2b0 0x7f171406e770 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:46.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.826+0000 7f1738827640 1 --2- 172.21.15.16:0/3834557037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173415e680 0x7f17340ccbe0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:46.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.826+0000 7f1738827640 1 --2- 172.21.15.16:0/3834557037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f173415e2a0 0x7f1734156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:46.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.826+0000 7f1738827640 1 --2- 172.21.15.16:0/3834557037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f173406d960 0x7f17340cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:46.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.826+0000 7f1738827640 1 -- 172.21.15.16:0/3834557037 >> 172.21.15.16:0/3834557037 conn(0x7f173409d760 msgr2=0x7f173415cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:46.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.827+0000 7f1738827640 1 -- 172.21.15.16:0/3834557037 shutdown_connections 2024-09-19T07:55:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.827+0000 7f1738827640 1 -- 172.21.15.16:0/3834557037 wait complete. 2024-09-19T07:55:46.845 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:55:46.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.952+0000 7f982eaa6640 1 Processor -- start 2024-09-19T07:55:46.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.952+0000 7f982eaa6640 1 -- start start 2024-09-19T07:55:46.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.953+0000 7f982eaa6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9828072590 0x7f982806d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.953+0000 7f982eaa6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9828141ad0 0x7f9828141f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.953+0000 7f982eaa6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98281620b0 con 0x7f98280d21f0 2024-09-19T07:55:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.953+0000 7f982eaa6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f9828141750 con 0x7f9828141ad0 2024-09-19T07:55:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.953+0000 7f982eaa6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9828162370 con 0x7f9828072590 2024-09-19T07:55:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.953+0000 7f982c842640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f98280d21f0 0x7f98280d25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60688/0 (socket says 172.21.15.16:60688) 2024-09-19T07:55:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.953+0000 7f982c842640 1 -- 172.21.15.16:0/1634577872 learned_addr learned my addr 172.21.15.16:0/1634577872 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.953+0000 7f982d043640 1 --2- 172.21.15.16:0/1634577872 >> v2:172.21.15.146:3300/0 conn(0x7f9828141ad0 0x7f9828141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.953+0000 7f9827fff640 1 --2- 172.21.15.16:0/1634577872 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9828072590 0x7f982806d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.953+0000 7f98277fe640 1 -- 172.21.15.16:0/1634577872 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 810865904 0 0) 0x7f98281620b0 con 0x7f98280d21f0 2024-09-19T07:55:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.953+0000 7f98277fe640 1 -- 172.21.15.16:0/1634577872 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9814003520 con 0x7f98280d21f0 2024-09-19T07:55:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.953+0000 7f9827fff640 1 -- 172.21.15.16:0/1634577872 >> v2:172.21.15.146:3300/0 conn(0x7f9828141ad0 msgr2=0x7f9828141f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:46.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.953+0000 7f9827fff640 1 --2- 172.21.15.16:0/1634577872 >> v2:172.21.15.146:3300/0 conn(0x7f9828141ad0 0x7f9828141f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:46.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.953+0000 7f9827fff640 1 -- 172.21.15.16:0/1634577872 >> v1:172.21.15.16:6789/0 conn(0x7f98280d21f0 legacy=0x7f98280d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:46.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.953+0000 7f98277fe640 1 -- 172.21.15.16:0/1634577872 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1912687083 0 0) 0x7f9814003520 con 0x7f98280d21f0 2024-09-19T07:55:46.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.953+0000 7f9827fff640 1 -- 172.21.15.16:0/1634577872 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9828162650 con 0x7f9828072590 2024-09-19T07:55:46.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.953+0000 7f982d043640 1 --2- 172.21.15.16:0/1634577872 >> v2:172.21.15.146:3300/0 conn(0x7f9828141ad0 0x7f9828141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:55:46.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.954+0000 7f9827fff640 1 --2- 172.21.15.16:0/1634577872 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9828072590 0x7f982806d0e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f98100097a0 tx=0x7f981002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c8c71001e6d512d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:46.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.955+0000 7f98277fe640 1 -- 172.21.15.16:0/1634577872 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9810037070 con 0x7f9828072590 2024-09-19T07:55:46.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.955+0000 7f98277fe640 1 -- 172.21.15.16:0/1634577872 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f981002ae50 con 0x7f9828072590 2024-09-19T07:55:46.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.955+0000 7f98277fe640 1 -- 172.21.15.16:0/1634577872 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9810032640 con 0x7f9828072590 2024-09-19T07:55:46.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.955+0000 7f982eaa6640 1 -- 172.21.15.16:0/1634577872 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9828072590 msgr2=0x7f982806d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:46.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.955+0000 7f982eaa6640 1 --2- 172.21.15.16:0/1634577872 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9828072590 0x7f982806d0e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f98100097a0 tx=0x7f981002a100 comp rx=0 tx=0).stop 2024-09-19T07:55:46.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.955+0000 7f982eaa6640 1 -- 172.21.15.16:0/1634577872 shutdown_connections 2024-09-19T07:55:46.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.955+0000 7f982eaa6640 1 --2- 172.21.15.16:0/1634577872 >> v2:172.21.15.146:3300/0 conn(0x7f9828141ad0 0x7f9828141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:46.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.955+0000 7f982eaa6640 1 --2- 172.21.15.16:0/1634577872 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9828072590 0x7f982806d0e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:46.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.955+0000 7f982eaa6640 1 -- 172.21.15.16:0/1634577872 >> 172.21.15.16:0/1634577872 conn(0x7f98280b3ed0 msgr2=0x7f98280b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:46.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.955+0000 7f982eaa6640 1 -- 172.21.15.16:0/1634577872 shutdown_connections 2024-09-19T07:55:46.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.955+0000 7f982eaa6640 1 -- 172.21.15.16:0/1634577872 wait complete. 2024-09-19T07:55:46.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.956+0000 7f982eaa6640 1 Processor -- start 2024-09-19T07:55:46.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.956+0000 7f982eaa6640 1 -- start start 2024-09-19T07:55:46.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.957+0000 7f982eaa6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9828072590 0x7f982813ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:46.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.957+0000 7f982eaa6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98280d21f0 0x7f98280b7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:46.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.957+0000 7f982eaa6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9828141ad0 0x7f98280c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:46.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.957+0000 7f982eaa6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f982806d620 con 0x7f9828141ad0 2024-09-19T07:55:46.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.957+0000 7f982eaa6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9828162c10 con 0x7f98280d21f0 2024-09-19T07:55:46.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.957+0000 7f982eaa6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f982806d7a0 con 0x7f9828072590 2024-09-19T07:55:46.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.957+0000 7f982d043640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9828141ad0 0x7f98280c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:46.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.957+0000 7f982d043640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9828141ad0 0x7f98280c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49690/0 (socket says 172.21.15.16:49690) 2024-09-19T07:55:46.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.957+0000 7f982d043640 1 -- 172.21.15.16:0/770121603 learned_addr learned my addr 172.21.15.16:0/770121603 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:46.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.957+0000 7f982c842640 1 --2- 172.21.15.16:0/770121603 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9828072590 0x7f982813ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:46.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.957+0000 7f9827fff640 1 --2- 172.21.15.16:0/770121603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98280d21f0 0x7f98280b7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:46.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.958+0000 7f982d043640 1 -- 172.21.15.16:0/770121603 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9828072590 msgr2=0x7f982813ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:46.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.958+0000 7f982d043640 1 --2- 172.21.15.16:0/770121603 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9828072590 0x7f982813ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:46.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.958+0000 7f982d043640 1 -- 172.21.15.16:0/770121603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98280d21f0 msgr2=0x7f98280b7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:46.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.958+0000 7f982d043640 1 --2- 172.21.15.16:0/770121603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98280d21f0 0x7f98280b7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:46.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.958+0000 7f982d043640 1 -- 172.21.15.16:0/770121603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98280b5700 con 0x7f9828141ad0 2024-09-19T07:55:46.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.958+0000 7f982d043640 1 --2- 172.21.15.16:0/770121603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9828141ad0 0x7f98280c8430 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f982000e880 tx=0x7f982000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:46.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.958+0000 7f98257fa640 1 -- 172.21.15.16:0/770121603 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820017070 con 0x7f9828141ad0 2024-09-19T07:55:46.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.958+0000 7f98257fa640 1 -- 172.21.15.16:0/770121603 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9820004020 con 0x7f9828141ad0 2024-09-19T07:55:46.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.958+0000 7f982eaa6640 1 -- 172.21.15.16:0/770121603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98280b59e0 con 0x7f9828141ad0 2024-09-19T07:55:46.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.958+0000 7f98257fa640 1 -- 172.21.15.16:0/770121603 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98200043f0 con 0x7f9828141ad0 2024-09-19T07:55:46.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.958+0000 7f982eaa6640 1 -- 172.21.15.16:0/770121603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98280b5fa0 con 0x7f9828141ad0 2024-09-19T07:55:46.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.959+0000 7f982eaa6640 1 -- 172.21.15.16:0/770121603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9828158c30 con 0x7f9828141ad0 2024-09-19T07:55:46.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.960+0000 7f98257fa640 1 -- 172.21.15.16:0/770121603 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f982000c940 con 0x7f9828141ad0 2024-09-19T07:55:46.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.960+0000 7f98257fa640 1 --2- 172.21.15.16:0/770121603 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f980806c2b0 0x7f980806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:46.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.960+0000 7f98257fa640 1 -- 172.21.15.16:0/770121603 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6101+0+0 (secure 0 0 0) 0x7f982008b600 con 0x7f9828141ad0 2024-09-19T07:55:46.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.960+0000 7f982c842640 1 --2- 172.21.15.16:0/770121603 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f980806c2b0 0x7f980806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:46.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.961+0000 7f982c842640 1 --2- 172.21.15.16:0/770121603 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f980806c2b0 0x7f980806e770 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f98280cfbb0 tx=0x7f9818009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:46.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:46.965+0000 7f98257fa640 1 -- 172.21.15.16:0/770121603 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9820059620 con 0x7f9828141ad0 2024-09-19T07:55:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.143+0000 7f982eaa6640 1 -- 172.21.15.16:0/770121603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9828072290 con 0x7f9828141ad0 2024-09-19T07:55:47.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.144+0000 7f98257fa640 1 -- 172.21.15.16:0/770121603 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) ==== 74+0+16981 (secure 0 0 0) 0x7f982005d920 con 0x7f9828141ad0 2024-09-19T07:55:47.145 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:47.145 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":151,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:55:46.155106+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:55:44.728059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-09-19T07:55:45.128249+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:55:47.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.146+0000 7f982eaa6640 1 -- 172.21.15.16:0/770121603 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f980806c2b0 msgr2=0x7f980806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:47.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.146+0000 7f982eaa6640 1 --2- 172.21.15.16:0/770121603 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f980806c2b0 0x7f980806e770 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f98280cfbb0 tx=0x7f9818009040 comp rx=0 tx=0).stop 2024-09-19T07:55:47.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.146+0000 7f982eaa6640 1 -- 172.21.15.16:0/770121603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9828141ad0 msgr2=0x7f98280c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:47.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.146+0000 7f982eaa6640 1 --2- 172.21.15.16:0/770121603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9828141ad0 0x7f98280c8430 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f982000e880 tx=0x7f982000ed40 comp rx=0 tx=0).stop 2024-09-19T07:55:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.146+0000 7f982eaa6640 1 -- 172.21.15.16:0/770121603 shutdown_connections 2024-09-19T07:55:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.146+0000 7f982eaa6640 1 --2- 172.21.15.16:0/770121603 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f980806c2b0 0x7f980806e770 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.146+0000 7f982eaa6640 1 --2- 172.21.15.16:0/770121603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9828141ad0 0x7f98280c8430 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.146+0000 7f982eaa6640 1 --2- 172.21.15.16:0/770121603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98280d21f0 0x7f98280b7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.146+0000 7f982eaa6640 1 --2- 172.21.15.16:0/770121603 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9828072590 0x7f982813ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.146+0000 7f982eaa6640 1 -- 172.21.15.16:0/770121603 >> 172.21.15.16:0/770121603 conn(0x7f98280b3ed0 msgr2=0x7f9828160be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.146+0000 7f982eaa6640 1 -- 172.21.15.16:0/770121603 shutdown_connections 2024-09-19T07:55:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.146+0000 7f982eaa6640 1 -- 172.21.15.16:0/770121603 wait complete. 2024-09-19T07:55:47.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:55:47.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.260+0000 7f377ecc9640 1 Processor -- start 2024-09-19T07:55:47.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.260+0000 7f377ecc9640 1 -- start start 2024-09-19T07:55:47.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.261+0000 7f377ecc9640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3778141680 0x7f37780cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:47.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.261+0000 7f377ecc9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37780d0370 0x7f3778141a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:47.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.261+0000 7f377ecc9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3778141fa0 0x7f37780cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:47.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.261+0000 7f377ecc9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f37780d07b0 con 0x7f3778141fa0 2024-09-19T07:55:47.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.261+0000 7f377ecc9640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f377815c6f0 con 0x7f3778141680 2024-09-19T07:55:47.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.261+0000 7f377ecc9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f377815d710 con 0x7f37780d0370 2024-09-19T07:55:47.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.261+0000 7f377d266640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3778141fa0 0x7f37780cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:47.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.261+0000 7f377d266640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3778141fa0 0x7f37780cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49702/0 (socket says 172.21.15.16:49702) 2024-09-19T07:55:47.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.261+0000 7f377d266640 1 -- 172.21.15.16:0/425324895 learned_addr learned my addr 172.21.15.16:0/425324895 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:47.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.261+0000 7f377ca65640 1 --2- 172.21.15.16:0/425324895 >> v2:172.21.15.146:3300/0 conn(0x7f3778141680 0x7f37780cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:47.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.261+0000 7f3777fff640 1 --2- 172.21.15.16:0/425324895 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37780d0370 0x7f3778141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:47.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.261+0000 7f377d266640 1 -- 172.21.15.16:0/425324895 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37780d0370 msgr2=0x7f3778141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:47.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.261+0000 7f377d266640 1 --2- 172.21.15.16:0/425324895 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37780d0370 0x7f3778141a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.261+0000 7f377d266640 1 -- 172.21.15.16:0/425324895 >> v2:172.21.15.146:3300/0 conn(0x7f3778141680 msgr2=0x7f37780cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:47.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.261+0000 7f377d266640 1 --2- 172.21.15.16:0/425324895 >> v2:172.21.15.146:3300/0 conn(0x7f3778141680 0x7f37780cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.261+0000 7f377d266640 1 -- 172.21.15.16:0/425324895 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377815c950 con 0x7f3778141fa0 2024-09-19T07:55:47.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.261+0000 7f377ca65640 1 --2- 172.21.15.16:0/425324895 >> v2:172.21.15.146:3300/0 conn(0x7f3778141680 0x7f37780cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:47.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.262+0000 7f377d266640 1 --2- 172.21.15.16:0/425324895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3778141fa0 0x7f37780cec70 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f3778070fc0 tx=0x7f377002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6935fe6d0b2573 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:47.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.262+0000 7f37777fe640 1 -- 172.21.15.16:0/425324895 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770036070 con 0x7f3778141fa0 2024-09-19T07:55:47.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.262+0000 7f37777fe640 1 -- 172.21.15.16:0/425324895 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3770031600 con 0x7f3778141fa0 2024-09-19T07:55:47.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.262+0000 7f37777fe640 1 -- 172.21.15.16:0/425324895 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37700319d0 con 0x7f3778141fa0 2024-09-19T07:55:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.263+0000 7f377ecc9640 1 -- 172.21.15.16:0/425324895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3778141fa0 msgr2=0x7f37780cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.263+0000 7f377ecc9640 1 --2- 172.21.15.16:0/425324895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3778141fa0 0x7f37780cec70 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f3778070fc0 tx=0x7f377002a6c0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.263+0000 7f377ecc9640 1 -- 172.21.15.16:0/425324895 shutdown_connections 2024-09-19T07:55:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.263+0000 7f377ecc9640 1 --2- 172.21.15.16:0/425324895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3778141fa0 0x7f37780cec70 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.263+0000 7f377ecc9640 1 --2- 172.21.15.16:0/425324895 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37780d0370 0x7f3778141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.263+0000 7f377ecc9640 1 --2- 172.21.15.16:0/425324895 >> v2:172.21.15.146:3300/0 conn(0x7f3778141680 0x7f37780cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.263+0000 7f377ecc9640 1 -- 172.21.15.16:0/425324895 >> 172.21.15.16:0/425324895 conn(0x7f37780b3ed0 msgr2=0x7f37780b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.263+0000 7f377ecc9640 1 -- 172.21.15.16:0/425324895 shutdown_connections 2024-09-19T07:55:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.263+0000 7f377ecc9640 1 -- 172.21.15.16:0/425324895 wait complete. 2024-09-19T07:55:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.264+0000 7f377ecc9640 1 Processor -- start 2024-09-19T07:55:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.264+0000 7f377ecc9640 1 -- start start 2024-09-19T07:55:47.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.265+0000 7f377ecc9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37780d0370 0x7f377813ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:47.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.265+0000 7f377ecc9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3778141680 0x7f3778157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:47.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.265+0000 7f377ecc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3778141fa0 0x7f37780c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:47.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.265+0000 7f377ecc9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f377815d1f0 con 0x7f3778141fa0 2024-09-19T07:55:47.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.265+0000 7f377ecc9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f377815cf10 con 0x7f37780d0370 2024-09-19T07:55:47.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.265+0000 7f377ecc9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3778142410 con 0x7f3778141680 2024-09-19T07:55:47.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.265+0000 7f377d266640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3778141fa0 0x7f37780c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:47.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.265+0000 7f377d266640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3778141fa0 0x7f37780c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49718/0 (socket says 172.21.15.16:49718) 2024-09-19T07:55:47.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.265+0000 7f377d266640 1 -- 172.21.15.16:0/3368195730 learned_addr learned my addr 172.21.15.16:0/3368195730 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:47.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.265+0000 7f377ca65640 1 --2- 172.21.15.16:0/3368195730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37780d0370 0x7f377813ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:47.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.265+0000 7f3777fff640 1 --2- 172.21.15.16:0/3368195730 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3778141680 0x7f3778157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:47.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.265+0000 7f377d266640 1 -- 172.21.15.16:0/3368195730 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3778141680 msgr2=0x7f3778157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:47.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.265+0000 7f377d266640 1 --2- 172.21.15.16:0/3368195730 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3778141680 0x7f3778157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.265+0000 7f377d266640 1 -- 172.21.15.16:0/3368195730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37780d0370 msgr2=0x7f377813ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:47.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.265+0000 7f377d266640 1 --2- 172.21.15.16:0/3368195730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37780d0370 0x7f377813ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.265+0000 7f377d266640 1 -- 172.21.15.16:0/3368195730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3778158a60 con 0x7f3778141fa0 2024-09-19T07:55:47.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.266+0000 7f377d266640 1 --2- 172.21.15.16:0/3368195730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3778141fa0 0x7f37780c8190 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f3770009e50 tx=0x7f3770004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:47.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.266+0000 7f37757fa640 1 -- 172.21.15.16:0/3368195730 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770036050 con 0x7f3778141fa0 2024-09-19T07:55:47.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.266+0000 7f37757fa640 1 -- 172.21.15.16:0/3368195730 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3770042070 con 0x7f3778141fa0 2024-09-19T07:55:47.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.266+0000 7f377ecc9640 1 -- 172.21.15.16:0/3368195730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3778158d40 con 0x7f3778141fa0 2024-09-19T07:55:47.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.266+0000 7f37757fa640 1 -- 172.21.15.16:0/3368195730 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770009510 con 0x7f3778141fa0 2024-09-19T07:55:47.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.266+0000 7f377ecc9640 1 -- 172.21.15.16:0/3368195730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37780b55e0 con 0x7f3778141fa0 2024-09-19T07:55:47.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.267+0000 7f377ecc9640 1 -- 172.21.15.16:0/3368195730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3778141a60 con 0x7f3778141fa0 2024-09-19T07:55:47.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.267+0000 7f37757fa640 1 -- 172.21.15.16:0/3368195730 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3770007bd0 con 0x7f3778141fa0 2024-09-19T07:55:47.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.268+0000 7f37757fa640 1 --2- 172.21.15.16:0/3368195730 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f375c06c2b0 0x7f375c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:47.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.268+0000 7f37757fa640 1 -- 172.21.15.16:0/3368195730 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6101+0+0 (secure 0 0 0) 0x7f37700ac3a0 con 0x7f3778141fa0 2024-09-19T07:55:47.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.268+0000 7f377ca65640 1 --2- 172.21.15.16:0/3368195730 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f375c06c2b0 0x7f375c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:47.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.269+0000 7f377ca65640 1 --2- 172.21.15.16:0/3368195730 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f375c06c2b0 0x7f375c06e770 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f376c00d8b0 tx=0x7f376c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:47.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.272+0000 7f37757fa640 1 -- 172.21.15.16:0/3368195730 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f377007aa10 con 0x7f3778141fa0 2024-09-19T07:55:47.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.448+0000 7f377ecc9640 1 -- 172.21.15.16:0/3368195730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f377813f590 con 0x7f3778141fa0 2024-09-19T07:55:47.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.449+0000 7f37757fa640 1 -- 172.21.15.16:0/3368195730 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) ==== 74+0+16981 (secure 0 0 0) 0x7f377007a310 con 0x7f3778141fa0 2024-09-19T07:55:47.450 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:47.450 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":151,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:55:46.155106+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:55:44.728059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-09-19T07:55:45.128249+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:55:47.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.451+0000 7f377ecc9640 1 -- 172.21.15.16:0/3368195730 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f375c06c2b0 msgr2=0x7f375c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:47.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.451+0000 7f377ecc9640 1 --2- 172.21.15.16:0/3368195730 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f375c06c2b0 0x7f375c06e770 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f376c00d8b0 tx=0x7f376c008040 comp rx=0 tx=0).stop 2024-09-19T07:55:47.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.451+0000 7f377ecc9640 1 -- 172.21.15.16:0/3368195730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3778141fa0 msgr2=0x7f37780c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:47.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.451+0000 7f377ecc9640 1 --2- 172.21.15.16:0/3368195730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3778141fa0 0x7f37780c8190 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f3770009e50 tx=0x7f3770004a40 comp rx=0 tx=0).stop 2024-09-19T07:55:47.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.451+0000 7f377ecc9640 1 -- 172.21.15.16:0/3368195730 shutdown_connections 2024-09-19T07:55:47.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.451+0000 7f377ecc9640 1 --2- 172.21.15.16:0/3368195730 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f375c06c2b0 0x7f375c06e770 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.451+0000 7f377ecc9640 1 --2- 172.21.15.16:0/3368195730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3778141fa0 0x7f37780c8190 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.451+0000 7f377ecc9640 1 --2- 172.21.15.16:0/3368195730 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3778141680 0x7f3778157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.451+0000 7f377ecc9640 1 --2- 172.21.15.16:0/3368195730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37780d0370 0x7f377813ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.451+0000 7f377ecc9640 1 -- 172.21.15.16:0/3368195730 >> 172.21.15.16:0/3368195730 conn(0x7f37780b3ed0 msgr2=0x7f37780b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:47.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.451+0000 7f377ecc9640 1 -- 172.21.15.16:0/3368195730 shutdown_connections 2024-09-19T07:55:47.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.451+0000 7f377ecc9640 1 -- 172.21.15.16:0/3368195730 wait complete. 2024-09-19T07:55:47.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T07:55:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.568+0000 7fbd85a4c640 1 Processor -- start 2024-09-19T07:55:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.568+0000 7fbd85a4c640 1 -- start start 2024-09-19T07:55:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.569+0000 7fbd85a4c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbd8015f0d0 0x7fbd8015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.569+0000 7fbd85a4c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd800d4130 0x7fbd800d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.569+0000 7fbd85a4c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd800d4ae0 con 0x7fbd800d0070 2024-09-19T07:55:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.569+0000 7fbd85a4c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fbd800d1b10 con 0x7fbd8015f0d0 2024-09-19T07:55:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.569+0000 7fbd85a4c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbd800cf2b0 con 0x7fbd800d4130 2024-09-19T07:55:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.569+0000 7fbd7effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbd8015f0d0 0x7fbd8015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.569+0000 7fbd7effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbd8015f0d0 0x7fbd8015f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49734/0 (socket says 172.21.15.16:49734) 2024-09-19T07:55:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.569+0000 7fbd7e7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbd800d0070 0x7fbd800d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60698/0 (socket says 172.21.15.16:60698) 2024-09-19T07:55:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.569+0000 7fbd7effd640 1 -- 172.21.15.16:0/1995214292 learned_addr learned my addr 172.21.15.16:0/1995214292 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.569+0000 7fbd7f7fe640 1 --2- 172.21.15.16:0/1995214292 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd800d4130 0x7fbd800d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.569+0000 7fbd7effd640 1 -- 172.21.15.16:0/1995214292 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd800d4130 msgr2=0x7fbd800d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.569+0000 7fbd7effd640 1 --2- 172.21.15.16:0/1995214292 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd800d4130 0x7fbd800d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.569+0000 7fbd7effd640 1 -- 172.21.15.16:0/1995214292 >> v1:172.21.15.16:6789/0 conn(0x7fbd800d0070 legacy=0x7fbd800d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.569+0000 7fbd7effd640 1 -- 172.21.15.16:0/1995214292 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd800cf590 con 0x7fbd8015f0d0 2024-09-19T07:55:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.569+0000 7fbd7dffb640 1 -- 172.21.15.16:0/1995214292 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3978454331 0 0) 0x7fbd800d4ae0 con 0x7fbd800d0070 2024-09-19T07:55:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.569+0000 7fbd7f7fe640 1 --2- 172.21.15.16:0/1995214292 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd800d4130 0x7fbd800d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.570+0000 7fbd7effd640 1 --2- 172.21.15.16:0/1995214292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd8015f0d0 0x7fbd8015f4b0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009870 tx=0x7fbd7002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0b6c2273a65bac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.570+0000 7fbd7dffb640 1 -- 172.21.15.16:0/1995214292 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70037070 con 0x7fbd8015f0d0 2024-09-19T07:55:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.570+0000 7fbd7dffb640 1 -- 172.21.15.16:0/1995214292 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd70032440 con 0x7fbd8015f0d0 2024-09-19T07:55:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.570+0000 7fbd7dffb640 1 -- 172.21.15.16:0/1995214292 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70032810 con 0x7fbd8015f0d0 2024-09-19T07:55:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.571+0000 7fbd85a4c640 1 -- 172.21.15.16:0/1995214292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd8015f0d0 msgr2=0x7fbd8015f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.571+0000 7fbd85a4c640 1 --2- 172.21.15.16:0/1995214292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd8015f0d0 0x7fbd8015f4b0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009870 tx=0x7fbd7002a1c0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.571+0000 7fbd85a4c640 1 -- 172.21.15.16:0/1995214292 shutdown_connections 2024-09-19T07:55:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.571+0000 7fbd85a4c640 1 --2- 172.21.15.16:0/1995214292 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd800d4130 0x7fbd800d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.571+0000 7fbd85a4c640 1 --2- 172.21.15.16:0/1995214292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd8015f0d0 0x7fbd8015f4b0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.571+0000 7fbd85a4c640 1 -- 172.21.15.16:0/1995214292 >> 172.21.15.16:0/1995214292 conn(0x7fbd8009d760 msgr2=0x7fbd8008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.571+0000 7fbd85a4c640 1 -- 172.21.15.16:0/1995214292 shutdown_connections 2024-09-19T07:55:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.571+0000 7fbd85a4c640 1 -- 172.21.15.16:0/1995214292 wait complete. 2024-09-19T07:55:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.572+0000 7fbd85a4c640 1 Processor -- start 2024-09-19T07:55:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.572+0000 7fbd85a4c640 1 -- start start 2024-09-19T07:55:47.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.573+0000 7fbd85a4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd800d0070 0x7fbd800ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:47.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.573+0000 7fbd85a4c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd800d4130 0x7fbd80160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:47.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.573+0000 7fbd85a4c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd8015f0d0 0x7fbd80160f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:47.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.573+0000 7fbd85a4c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd800cef60 con 0x7fbd800d0070 2024-09-19T07:55:47.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.573+0000 7fbd85a4c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbd800cfb50 con 0x7fbd800d4130 2024-09-19T07:55:47.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.573+0000 7fbd85a4c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbd800b2f70 con 0x7fbd8015f0d0 2024-09-19T07:55:47.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.573+0000 7fbd7effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd800d0070 0x7fbd800ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.573+0000 7fbd7effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd800d0070 0x7fbd800ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49744/0 (socket says 172.21.15.16:49744) 2024-09-19T07:55:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.573+0000 7fbd7effd640 1 -- 172.21.15.16:0/3576758255 learned_addr learned my addr 172.21.15.16:0/3576758255 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.573+0000 7fbd7e7fc640 1 --2- 172.21.15.16:0/3576758255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd800d4130 0x7fbd80160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.573+0000 7fbd7f7fe640 1 --2- 172.21.15.16:0/3576758255 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd8015f0d0 0x7fbd80160f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.573+0000 7fbd7effd640 1 -- 172.21.15.16:0/3576758255 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd8015f0d0 msgr2=0x7fbd80160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.573+0000 7fbd7effd640 1 --2- 172.21.15.16:0/3576758255 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd8015f0d0 0x7fbd80160f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.574+0000 7fbd7effd640 1 -- 172.21.15.16:0/3576758255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd800d4130 msgr2=0x7fbd80160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.574+0000 7fbd7effd640 1 --2- 172.21.15.16:0/3576758255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd800d4130 0x7fbd80160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.574+0000 7fbd7effd640 1 -- 172.21.15.16:0/3576758255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd800cc490 con 0x7fbd800d0070 2024-09-19T07:55:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.574+0000 7fbd7f7fe640 1 --2- 172.21.15.16:0/3576758255 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd8015f0d0 0x7fbd80160f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.574+0000 7fbd7effd640 1 --2- 172.21.15.16:0/3576758255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd800d0070 0x7fbd800ca4b0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fbd70000c00 tx=0x7fbd700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.574+0000 7fbd67fff640 1 -- 172.21.15.16:0/3576758255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70041020 con 0x7fbd800d0070 2024-09-19T07:55:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.574+0000 7fbd67fff640 1 -- 172.21.15.16:0/3576758255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd700306c0 con 0x7fbd800d0070 2024-09-19T07:55:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.574+0000 7fbd85a4c640 1 -- 172.21.15.16:0/3576758255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd800cc770 con 0x7fbd800d0070 2024-09-19T07:55:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.574+0000 7fbd67fff640 1 -- 172.21.15.16:0/3576758255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70037040 con 0x7fbd800d0070 2024-09-19T07:55:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.574+0000 7fbd85a4c640 1 -- 172.21.15.16:0/3576758255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd800ccd30 con 0x7fbd800d0070 2024-09-19T07:55:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.575+0000 7fbd85a4c640 1 -- 172.21.15.16:0/3576758255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd8015f4b0 con 0x7fbd800d0070 2024-09-19T07:55:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.575+0000 7fbd67fff640 1 -- 172.21.15.16:0/3576758255 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbd70044020 con 0x7fbd800d0070 2024-09-19T07:55:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.576+0000 7fbd67fff640 1 --2- 172.21.15.16:0/3576758255 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbd6c06c2b0 0x7fbd6c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.576+0000 7fbd67fff640 1 -- 172.21.15.16:0/3576758255 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6101+0+0 (secure 0 0 0) 0x7fbd700abd60 con 0x7fbd800d0070 2024-09-19T07:55:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.576+0000 7fbd7e7fc640 1 --2- 172.21.15.16:0/3576758255 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbd6c06c2b0 0x7fbd6c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.577+0000 7fbd7e7fc640 1 --2- 172.21.15.16:0/3576758255 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbd6c06c2b0 0x7fbd6c06e770 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbd800cda90 tx=0x7fbd74007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:47.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.581+0000 7fbd67fff640 1 -- 172.21.15.16:0/3576758255 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fbd7004b030 con 0x7fbd800d0070 2024-09-19T07:55:47.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:47.792+0000 7fbd85a4c640 1 -- 172.21.15.16:0/3576758255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbd800d31d0 con 0x7fbd800d0070 2024-09-19T07:55:48.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.209+0000 7fbd67fff640 1 -- 172.21.15.16:0/3576758255 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v91) ==== 118+0+0 (secure 0 0 0) 0x7fbd7007a340 con 0x7fbd800d0070 2024-09-19T07:55:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.212+0000 7fbd85a4c640 1 -- 172.21.15.16:0/3576758255 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbd6c06c2b0 msgr2=0x7fbd6c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.212+0000 7fbd85a4c640 1 --2- 172.21.15.16:0/3576758255 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbd6c06c2b0 0x7fbd6c06e770 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbd800cda90 tx=0x7fbd74007d50 comp rx=0 tx=0).stop 2024-09-19T07:55:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.212+0000 7fbd85a4c640 1 -- 172.21.15.16:0/3576758255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd800d0070 msgr2=0x7fbd800ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.212+0000 7fbd85a4c640 1 --2- 172.21.15.16:0/3576758255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd800d0070 0x7fbd800ca4b0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fbd70000c00 tx=0x7fbd700046c0 comp rx=0 tx=0).stop 2024-09-19T07:55:48.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.212+0000 7fbd85a4c640 1 -- 172.21.15.16:0/3576758255 shutdown_connections 2024-09-19T07:55:48.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.212+0000 7fbd85a4c640 1 --2- 172.21.15.16:0/3576758255 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbd6c06c2b0 0x7fbd6c06e770 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:48.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.212+0000 7fbd85a4c640 1 --2- 172.21.15.16:0/3576758255 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd8015f0d0 0x7fbd80160f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:48.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.212+0000 7fbd85a4c640 1 --2- 172.21.15.16:0/3576758255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd800d4130 0x7fbd80160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:48.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.212+0000 7fbd85a4c640 1 --2- 172.21.15.16:0/3576758255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd800d0070 0x7fbd800ca4b0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:48.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.212+0000 7fbd85a4c640 1 -- 172.21.15.16:0/3576758255 >> 172.21.15.16:0/3576758255 conn(0x7fbd8009d760 msgr2=0x7fbd800d1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:48.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.212+0000 7fbd85a4c640 1 -- 172.21.15.16:0/3576758255 shutdown_connections 2024-09-19T07:55:48.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.212+0000 7fbd85a4c640 1 -- 172.21.15.16:0/3576758255 wait complete. 2024-09-19T07:55:48.225 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T07:55:48.225 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T07:55:48.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.335+0000 7fa53215f640 1 Processor -- start 2024-09-19T07:55:48.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.335+0000 7fa53215f640 1 -- start start 2024-09-19T07:55:48.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.336+0000 7fa53215f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa52c15e680 0x7fa52c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:48.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.336+0000 7fa53215f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa52c0d0010 0x7fa52c0d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.336+0000 7fa53215f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa52c0d1880 con 0x7fa52c15e2a0 2024-09-19T07:55:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.336+0000 7fa53215f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa52c0d1b60 con 0x7fa52c0d0010 2024-09-19T07:55:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.336+0000 7fa53215f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa52c0cf2d0 con 0x7fa52c15e680 2024-09-19T07:55:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.336+0000 7fa52bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa52c0d0010 0x7fa52c0d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.336+0000 7fa52b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa52c15e2a0 0x7fa52c06d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60710/0 (socket says 172.21.15.16:60710) 2024-09-19T07:55:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.336+0000 7fa52b7fe640 1 -- 172.21.15.16:0/2108808099 learned_addr learned my addr 172.21.15.16:0/2108808099 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.336+0000 7fa52affd640 1 --2- 172.21.15.16:0/2108808099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa52c15e680 0x7fa52c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.336+0000 7fa52bfff640 1 -- 172.21.15.16:0/2108808099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa52c15e680 msgr2=0x7fa52c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.336+0000 7fa52bfff640 1 --2- 172.21.15.16:0/2108808099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa52c15e680 0x7fa52c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.336+0000 7fa52bfff640 1 -- 172.21.15.16:0/2108808099 >> v1:172.21.15.16:6789/0 conn(0x7fa52c15e2a0 legacy=0x7fa52c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.336+0000 7fa52bfff640 1 -- 172.21.15.16:0/2108808099 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c0cf5b0 con 0x7fa52c0d0010 2024-09-19T07:55:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.337+0000 7fa52bfff640 1 --2- 172.21.15.16:0/2108808099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c0d0010 0x7fa52c0d4370 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fa524009d90 tx=0x7fa52402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc1b43e193c30df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.337+0000 7fa52a7fc640 1 -- 172.21.15.16:0/2108808099 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524031600 con 0x7fa52c0d0010 2024-09-19T07:55:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.337+0000 7fa52a7fc640 1 -- 172.21.15.16:0/2108808099 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5240317a0 con 0x7fa52c0d0010 2024-09-19T07:55:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.337+0000 7fa52a7fc640 1 -- 172.21.15.16:0/2108808099 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524031b70 con 0x7fa52c0d0010 2024-09-19T07:55:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.337+0000 7fa53215f640 1 -- 172.21.15.16:0/2108808099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c0d0010 msgr2=0x7fa52c0d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.337+0000 7fa53215f640 1 --2- 172.21.15.16:0/2108808099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c0d0010 0x7fa52c0d4370 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fa524009d90 tx=0x7fa52402a5d0 comp rx=0 tx=0).stop 2024-09-19T07:55:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.337+0000 7fa53215f640 1 -- 172.21.15.16:0/2108808099 shutdown_connections 2024-09-19T07:55:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.337+0000 7fa53215f640 1 --2- 172.21.15.16:0/2108808099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c0d0010 0x7fa52c0d4370 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.337+0000 7fa53215f640 1 --2- 172.21.15.16:0/2108808099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa52c15e680 0x7fa52c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.337+0000 7fa53215f640 1 -- 172.21.15.16:0/2108808099 >> 172.21.15.16:0/2108808099 conn(0x7fa52c09d760 msgr2=0x7fa52c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.338+0000 7fa53215f640 1 -- 172.21.15.16:0/2108808099 shutdown_connections 2024-09-19T07:55:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.338+0000 7fa53215f640 1 -- 172.21.15.16:0/2108808099 wait complete. 2024-09-19T07:55:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.339+0000 7fa53215f640 1 Processor -- start 2024-09-19T07:55:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.339+0000 7fa53215f640 1 -- start start 2024-09-19T07:55:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.339+0000 7fa53215f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa52c0d0010 0x7fa52c0b8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa53215f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa52c15e2a0 0x7fa52c0d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa53215f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c15e680 0x7fa52c0d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa53215f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c06dcb0 con 0x7fa52c15e680 2024-09-19T07:55:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa53215f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c06db30 con 0x7fa52c15e2a0 2024-09-19T07:55:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa53215f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa52c0d48b0 con 0x7fa52c0d0010 2024-09-19T07:55:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa52bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c15e680 0x7fa52c0d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa52bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c15e680 0x7fa52c0d0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49752/0 (socket says 172.21.15.16:49752) 2024-09-19T07:55:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa52bfff640 1 -- 172.21.15.16:0/1628459822 learned_addr learned my addr 172.21.15.16:0/1628459822 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa52b7fe640 1 --2- 172.21.15.16:0/1628459822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa52c0d0010 0x7fa52c0b8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa52affd640 1 --2- 172.21.15.16:0/1628459822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa52c15e2a0 0x7fa52c0d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa52bfff640 1 -- 172.21.15.16:0/1628459822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa52c0d0010 msgr2=0x7fa52c0b8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa52bfff640 1 --2- 172.21.15.16:0/1628459822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa52c0d0010 0x7fa52c0b8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa52bfff640 1 -- 172.21.15.16:0/1628459822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa52c15e2a0 msgr2=0x7fa52c0d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa52bfff640 1 --2- 172.21.15.16:0/1628459822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa52c15e2a0 0x7fa52c0d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa52bfff640 1 -- 172.21.15.16:0/1628459822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c0ca710 con 0x7fa52c15e680 2024-09-19T07:55:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa52bfff640 1 --2- 172.21.15.16:0/1628459822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c15e680 0x7fa52c0d0e00 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fa52402abc0 tx=0x7fa524004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa513fff640 1 -- 172.21.15.16:0/1628459822 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524040070 con 0x7fa52c15e680 2024-09-19T07:55:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa513fff640 1 -- 172.21.15.16:0/1628459822 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa524030c00 con 0x7fa52c15e680 2024-09-19T07:55:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa53215f640 1 -- 172.21.15.16:0/1628459822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52c0ca9f0 con 0x7fa52c15e680 2024-09-19T07:55:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa513fff640 1 -- 172.21.15.16:0/1628459822 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524009450 con 0x7fa52c15e680 2024-09-19T07:55:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.340+0000 7fa53215f640 1 -- 172.21.15.16:0/1628459822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52c0cafb0 con 0x7fa52c15e680 2024-09-19T07:55:48.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.342+0000 7fa53215f640 1 -- 172.21.15.16:0/1628459822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa52c0d4370 con 0x7fa52c15e680 2024-09-19T07:55:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.342+0000 7fa513fff640 1 -- 172.21.15.16:0/1628459822 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa524039040 con 0x7fa52c15e680 2024-09-19T07:55:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.343+0000 7fa513fff640 1 --2- 172.21.15.16:0/1628459822 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa51806c2e0 0x7fa51806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.343+0000 7fa513fff640 1 -- 172.21.15.16:0/1628459822 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7fa5240ac8a0 con 0x7fa52c15e680 2024-09-19T07:55:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.343+0000 7fa52b7fe640 1 --2- 172.21.15.16:0/1628459822 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa51806c2e0 0x7fa51806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.343+0000 7fa52b7fe640 1 --2- 172.21.15.16:0/1628459822 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa51806c2e0 0x7fa51806e7a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa51c007fd0 tx=0x7fa51c007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:48.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.347+0000 7fa513fff640 1 -- 172.21.15.16:0/1628459822 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa52407a830 con 0x7fa52c15e680 2024-09-19T07:55:48.527 INFO:tasks.ceph.mds.c.smithi016.stderr:2024-09-19T07:55:48.526+0000 7fd5975ff640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T07:55:48.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:48.560+0000 7fa53215f640 1 -- 172.21.15.16:0/1628459822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa52c06cf90 con 0x7fa52c15e680 2024-09-19T07:55:49.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.215+0000 7fa513fff640 1 -- 172.21.15.16:0/1628459822 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v92) ==== 122+0+0 (secure 0 0 0) 0x7fa52407eb30 con 0x7fa52c15e680 2024-09-19T07:55:49.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.217+0000 7fa53215f640 1 -- 172.21.15.16:0/1628459822 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa51806c2e0 msgr2=0x7fa51806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:49.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.217+0000 7fa53215f640 1 --2- 172.21.15.16:0/1628459822 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa51806c2e0 0x7fa51806e7a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa51c007fd0 tx=0x7fa51c007e80 comp rx=0 tx=0).stop 2024-09-19T07:55:49.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.217+0000 7fa53215f640 1 -- 172.21.15.16:0/1628459822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c15e680 msgr2=0x7fa52c0d0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:49.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.217+0000 7fa53215f640 1 --2- 172.21.15.16:0/1628459822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c15e680 0x7fa52c0d0e00 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fa52402abc0 tx=0x7fa524004a40 comp rx=0 tx=0).stop 2024-09-19T07:55:49.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.218+0000 7fa53215f640 1 -- 172.21.15.16:0/1628459822 shutdown_connections 2024-09-19T07:55:49.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.218+0000 7fa53215f640 1 --2- 172.21.15.16:0/1628459822 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa51806c2e0 0x7fa51806e7a0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.218+0000 7fa53215f640 1 --2- 172.21.15.16:0/1628459822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa52c15e680 0x7fa52c0d0e00 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.218+0000 7fa53215f640 1 --2- 172.21.15.16:0/1628459822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa52c15e2a0 0x7fa52c0d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.218+0000 7fa53215f640 1 --2- 172.21.15.16:0/1628459822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa52c0d0010 0x7fa52c0b8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.218+0000 7fa53215f640 1 -- 172.21.15.16:0/1628459822 >> 172.21.15.16:0/1628459822 conn(0x7fa52c09d760 msgr2=0x7fa52c1608f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:49.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.218+0000 7fa53215f640 1 -- 172.21.15.16:0/1628459822 shutdown_connections 2024-09-19T07:55:49.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.218+0000 7fa53215f640 1 -- 172.21.15.16:0/1628459822 wait complete. 2024-09-19T07:55:49.232 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T07:55:49.232 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:55:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.340+0000 7f9cc202e640 1 Processor -- start 2024-09-19T07:55:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.340+0000 7f9cc202e640 1 -- start start 2024-09-19T07:55:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.341+0000 7f9cc202e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9cbc0d4370 0x7f9cbc0d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.341+0000 7f9cc202e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cbc0d1880 con 0x7f9cbc15e2a0 2024-09-19T07:55:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.341+0000 7f9cc202e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cbc0d1b60 con 0x7f9cbc15e680 2024-09-19T07:55:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.341+0000 7f9cc202e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9cbc0cf2d0 con 0x7f9cbc0d4370 2024-09-19T07:55:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.341+0000 7f9cbbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9cbc0d4370 0x7f9cbc0d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.341+0000 7f9cbb7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9cbc15e2a0 0x7f9cbc06d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60712/0 (socket says 172.21.15.16:60712) 2024-09-19T07:55:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.341+0000 7f9cbbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9cbc0d4370 0x7f9cbc0d4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49768/0 (socket says 172.21.15.16:49768) 2024-09-19T07:55:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.341+0000 7f9cbb7fe640 1 -- 172.21.15.16:0/1261608416 learned_addr learned my addr 172.21.15.16:0/1261608416 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.341+0000 7f9cbbfff640 1 -- 172.21.15.16:0/1261608416 >> v1:172.21.15.146:6789/0 conn(0x7f9cbc15e680 legacy=0x7f9cbc0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.341+0000 7f9cbbfff640 1 -- 172.21.15.16:0/1261608416 >> v1:172.21.15.16:6789/0 conn(0x7f9cbc15e2a0 legacy=0x7f9cbc06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.341+0000 7f9cbbfff640 1 -- 172.21.15.16:0/1261608416 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cbc0cf5b0 con 0x7f9cbc0d4370 2024-09-19T07:55:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.341+0000 7f9cba7fc640 1 -- 172.21.15.16:0/1261608416 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3743770132 0 0) 0x7f9cbc0d1880 con 0x7f9cbc15e2a0 2024-09-19T07:55:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.342+0000 7f9cbbfff640 1 --2- 172.21.15.16:0/1261608416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cbc0d4370 0x7f9cbc0d4820 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009d90 tx=0x7f9cb402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737c10025e0fa6ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.342+0000 7f9cba7fc640 1 -- 172.21.15.16:0/1261608416 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4007740 con 0x7f9cbc0d4370 2024-09-19T07:55:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.342+0000 7f9cba7fc640 1 -- 172.21.15.16:0/1261608416 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb40078e0 con 0x7f9cbc0d4370 2024-09-19T07:55:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.342+0000 7f9cba7fc640 1 -- 172.21.15.16:0/1261608416 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4007cb0 con 0x7f9cbc0d4370 2024-09-19T07:55:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.343+0000 7f9cc202e640 1 -- 172.21.15.16:0/1261608416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cbc0d4370 msgr2=0x7f9cbc0d4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.343+0000 7f9cc202e640 1 --2- 172.21.15.16:0/1261608416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cbc0d4370 0x7f9cbc0d4820 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009d90 tx=0x7f9cb402a6a0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.343+0000 7f9cc202e640 1 -- 172.21.15.16:0/1261608416 shutdown_connections 2024-09-19T07:55:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.343+0000 7f9cc202e640 1 --2- 172.21.15.16:0/1261608416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cbc0d4370 0x7f9cbc0d4820 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.343+0000 7f9cc202e640 1 -- 172.21.15.16:0/1261608416 >> 172.21.15.16:0/1261608416 conn(0x7f9cbc09d760 msgr2=0x7f9cbc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.343+0000 7f9cc202e640 1 -- 172.21.15.16:0/1261608416 shutdown_connections 2024-09-19T07:55:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.343+0000 7f9cc202e640 1 -- 172.21.15.16:0/1261608416 wait complete. 2024-09-19T07:55:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.344+0000 7f9cc202e640 1 Processor -- start 2024-09-19T07:55:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.344+0000 7f9cc202e640 1 -- start start 2024-09-19T07:55:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.345+0000 7f9cc202e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cbc0d4370 0x7f9cbc0c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.345+0000 7f9cc202e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9cbc15e2a0 0x7f9cbc1f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.345+0000 7f9cc202e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9cbc15e680 0x7f9cbc0c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.345+0000 7f9cc202e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc06dcb0 con 0x7f9cbc0d4370 2024-09-19T07:55:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.345+0000 7f9cc202e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc06db30 con 0x7f9cbc15e680 2024-09-19T07:55:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.345+0000 7f9cc202e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9cbc0d01e0 con 0x7f9cbc15e2a0 2024-09-19T07:55:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.345+0000 7f9cbb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cbc0d4370 0x7f9cbc0c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.345+0000 7f9cbb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cbc0d4370 0x7f9cbc0c98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49770/0 (socket says 172.21.15.16:49770) 2024-09-19T07:55:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.345+0000 7f9cbb7fe640 1 -- 172.21.15.16:0/262022177 learned_addr learned my addr 172.21.15.16:0/262022177 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.345+0000 7f9cbbfff640 1 --2- 172.21.15.16:0/262022177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9cbc15e680 0x7f9cbc0c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.345+0000 7f9cbb7fe640 1 -- 172.21.15.16:0/262022177 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9cbc15e2a0 msgr2=0x7f9cbc1f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.345+0000 7f9cbb7fe640 1 --2- 172.21.15.16:0/262022177 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9cbc15e2a0 0x7f9cbc1f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.345+0000 7f9cbb7fe640 1 -- 172.21.15.16:0/262022177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9cbc15e680 msgr2=0x7f9cbc0c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.345+0000 7f9cbb7fe640 1 --2- 172.21.15.16:0/262022177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9cbc15e680 0x7f9cbc0c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.345+0000 7f9cbb7fe640 1 -- 172.21.15.16:0/262022177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cbc0ca030 con 0x7f9cbc0d4370 2024-09-19T07:55:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.345+0000 7f9cbbfff640 1 --2- 172.21.15.16:0/262022177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9cbc15e680 0x7f9cbc0c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.345+0000 7f9cbb7fe640 1 --2- 172.21.15.16:0/262022177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cbc0d4370 0x7f9cbc0c98e0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00c430 tx=0x7f9cac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.346+0000 7f9ca3fff640 1 -- 172.21.15.16:0/262022177 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac0074e0 con 0x7f9cbc0d4370 2024-09-19T07:55:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.346+0000 7f9ca3fff640 1 -- 172.21.15.16:0/262022177 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cac0047e0 con 0x7f9cbc0d4370 2024-09-19T07:55:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.346+0000 7f9cc202e640 1 -- 172.21.15.16:0/262022177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cbc0ca310 con 0x7f9cbc0d4370 2024-09-19T07:55:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.346+0000 7f9ca3fff640 1 -- 172.21.15.16:0/262022177 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac004bd0 con 0x7f9cbc0d4370 2024-09-19T07:55:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.346+0000 7f9cc202e640 1 -- 172.21.15.16:0/262022177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cbc0ca8d0 con 0x7f9cbc0d4370 2024-09-19T07:55:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.347+0000 7f9cc202e640 1 -- 172.21.15.16:0/262022177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cbc0d4820 con 0x7f9cbc0d4370 2024-09-19T07:55:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.347+0000 7f9ca3fff640 1 -- 172.21.15.16:0/262022177 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9cac011070 con 0x7f9cbc0d4370 2024-09-19T07:55:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.348+0000 7f9ca3fff640 1 --2- 172.21.15.16:0/262022177 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ca806c2b0 0x7f9ca806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.348+0000 7f9ca3fff640 1 -- 172.21.15.16:0/262022177 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f9cac08b260 con 0x7f9cbc0d4370 2024-09-19T07:55:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.348+0000 7f9cbaffd640 1 --2- 172.21.15.16:0/262022177 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ca806c2b0 0x7f9ca806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:49.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.349+0000 7f9cbaffd640 1 --2- 172.21.15.16:0/262022177 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ca806c2b0 0x7f9ca806e770 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc161b00 tx=0x7f9cb0007990 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.352+0000 7f9ca3fff640 1 -- 172.21.15.16:0/262022177 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9cac059840 con 0x7f9cbc0d4370 2024-09-19T07:55:49.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.566+0000 7f9cc202e640 1 -- 172.21.15.16:0/262022177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cbc06cf90 con 0x7f9cbc0d4370 2024-09-19T07:55:49.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.567+0000 7f9ca3fff640 1 -- 172.21.15.16:0/262022177 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5230 (secure 0 0 0) 0x7f9cac059140 con 0x7f9cbc0d4370 2024-09-19T07:55:49.567 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:49.568 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":92,"btime":"2024-09-19T07:55:49:210981+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7064,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3873812166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3873812166},{"type":"v1","addr":"172.21.15.146:6837","nonce":3873812166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":7852,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/951273503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":951273503},{"type":"v1","addr":"172.21.15.146:6835","nonce":951273503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":8757,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2338289617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2338289617},{"type":"v1","addr":"172.21.15.16:6834","nonce":2338289617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:55:46.155417+0000","modified":"2024-09-19T07:55:48.561637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8760":{"gid":8760,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6835/3705993654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":3705993654},{"type":"v1","addr":"172.21.15.16:6835","nonce":3705993654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8760,"qdb_cluster":[8760]},"id":11}]} 2024-09-19T07:55:49.568 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 92 2024-09-19T07:55:49.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.569+0000 7f9cc202e640 1 -- 172.21.15.16:0/262022177 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ca806c2b0 msgr2=0x7f9ca806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:49.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.569+0000 7f9cc202e640 1 --2- 172.21.15.16:0/262022177 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ca806c2b0 0x7f9ca806e770 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc161b00 tx=0x7f9cb0007990 comp rx=0 tx=0).stop 2024-09-19T07:55:49.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.569+0000 7f9cc202e640 1 -- 172.21.15.16:0/262022177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cbc0d4370 msgr2=0x7f9cbc0c98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:49.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.569+0000 7f9cc202e640 1 --2- 172.21.15.16:0/262022177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cbc0d4370 0x7f9cbc0c98e0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00c430 tx=0x7f9cac00c8f0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.569+0000 7f9cc202e640 1 -- 172.21.15.16:0/262022177 shutdown_connections 2024-09-19T07:55:49.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.569+0000 7f9cc202e640 1 --2- 172.21.15.16:0/262022177 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ca806c2b0 0x7f9ca806e770 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.569+0000 7f9cc202e640 1 --2- 172.21.15.16:0/262022177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9cbc15e680 0x7f9cbc0c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.569+0000 7f9cc202e640 1 --2- 172.21.15.16:0/262022177 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9cbc15e2a0 0x7f9cbc1f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.569+0000 7f9cc202e640 1 --2- 172.21.15.16:0/262022177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cbc0d4370 0x7f9cbc0c98e0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.569+0000 7f9cc202e640 1 -- 172.21.15.16:0/262022177 >> 172.21.15.16:0/262022177 conn(0x7f9cbc09d760 msgr2=0x7f9cbc161f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:49.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.569+0000 7f9cc202e640 1 -- 172.21.15.16:0/262022177 shutdown_connections 2024-09-19T07:55:49.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.569+0000 7f9cc202e640 1 -- 172.21.15.16:0/262022177 wait complete. 2024-09-19T07:55:49.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:55:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.683+0000 7f3487923640 1 Processor -- start 2024-09-19T07:55:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.683+0000 7f3487923640 1 -- start start 2024-09-19T07:55:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3487923640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f34800d4840 0x7f348015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3487923640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34800d1b10 con 0x7f348015f0d0 2024-09-19T07:55:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3487923640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34800cf2b0 con 0x7f34800d0070 2024-09-19T07:55:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3487923640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f34800cf590 con 0x7f34800d4840 2024-09-19T07:55:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f34856bf640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f348015f0d0 0x7f34800d1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60716/0 (socket says 172.21.15.16:60716) 2024-09-19T07:55:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f34856bf640 1 -- 172.21.15.16:0/686840900 learned_addr learned my addr 172.21.15.16:0/686840900 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3485ec0640 1 --2- 172.21.15.16:0/686840900 >> v2:172.21.15.146:3300/0 conn(0x7f34800d4840 0x7f348015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3477fff640 1 -- 172.21.15.16:0/686840900 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1668937958 0 0) 0x7f34800d1b10 con 0x7f348015f0d0 2024-09-19T07:55:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3477fff640 1 -- 172.21.15.16:0/686840900 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f346c003520 con 0x7f348015f0d0 2024-09-19T07:55:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3477fff640 1 -- 172.21.15.16:0/686840900 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1529956558 0 0) 0x7f346c003520 con 0x7f348015f0d0 2024-09-19T07:55:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3477fff640 1 -- 172.21.15.16:0/686840900 >> v2:172.21.15.146:3300/0 conn(0x7f34800d4840 msgr2=0x7f348015f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3477fff640 1 --2- 172.21.15.16:0/686840900 >> v2:172.21.15.146:3300/0 conn(0x7f34800d4840 0x7f348015f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3477fff640 1 -- 172.21.15.16:0/686840900 >> v1:172.21.15.146:6789/0 conn(0x7f34800d0070 legacy=0x7f34800d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3477fff640 1 -- 172.21.15.16:0/686840900 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34800cf870 con 0x7f348015f0d0 2024-09-19T07:55:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3477fff640 1 -- 172.21.15.16:0/686840900 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f347c001670 con 0x7f348015f0d0 2024-09-19T07:55:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3477fff640 1 -- 172.21.15.16:0/686840900 >> v1:172.21.15.16:6789/0 conn(0x7f348015f0d0 legacy=0x7f34800d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3477fff640 1 --2- 172.21.15.16:0/686840900 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f346c0039b0 0x7f346c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3477fff640 1 --2- 172.21.15.16:0/686840900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c004800 0x7f346c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3477fff640 1 --2- 172.21.15.16:0/686840900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f346c0052e0 0x7f346c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3477fff640 1 -- 172.21.15.16:0/686840900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34800cf590 con 0x7f346c004800 2024-09-19T07:55:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3477fff640 1 -- 172.21.15.16:0/686840900 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f346c00c190 con 0x7f346c0052e0 2024-09-19T07:55:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3477fff640 1 -- 172.21.15.16:0/686840900 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f346c00c390 con 0x7f346c0039b0 2024-09-19T07:55:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.684+0000 7f3484ebe640 1 --2- 172.21.15.16:0/686840900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c004800 0x7f346c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.685+0000 7f34856bf640 1 --2- 172.21.15.16:0/686840900 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f346c0039b0 0x7f346c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.685+0000 7f3485ec0640 1 --2- 172.21.15.16:0/686840900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f346c0052e0 0x7f346c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.685+0000 7f3484ebe640 1 -- 172.21.15.16:0/686840900 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f346c0039b0 msgr2=0x7f346c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.685+0000 7f3484ebe640 1 --2- 172.21.15.16:0/686840900 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f346c0039b0 0x7f346c003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.685+0000 7f3484ebe640 1 -- 172.21.15.16:0/686840900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f346c0052e0 msgr2=0x7f346c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.685+0000 7f3484ebe640 1 --2- 172.21.15.16:0/686840900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f346c0052e0 0x7f346c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.685+0000 7f3484ebe640 1 -- 172.21.15.16:0/686840900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346c003520 con 0x7f346c004800 2024-09-19T07:55:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.685+0000 7f3485ec0640 1 --2- 172.21.15.16:0/686840900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f346c0052e0 0x7f346c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.685+0000 7f3485ec0640 1 -- 172.21.15.16:0/686840900 reap_dead start 2024-09-19T07:55:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.685+0000 7f3484ebe640 1 --2- 172.21.15.16:0/686840900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c004800 0x7f346c004c80 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f3470003d80 tx=0x7f346c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc49e8aebf719dea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:49.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.686+0000 7f3477fff640 1 -- 172.21.15.16:0/686840900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470037070 con 0x7f346c004800 2024-09-19T07:55:49.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.686+0000 7f3477fff640 1 -- 172.21.15.16:0/686840900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f347002a9c0 con 0x7f346c004800 2024-09-19T07:55:49.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.686+0000 7f3477fff640 1 -- 172.21.15.16:0/686840900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470003980 con 0x7f346c004800 2024-09-19T07:55:49.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.686+0000 7f3487923640 1 -- 172.21.15.16:0/686840900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c004800 msgr2=0x7f346c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:49.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.686+0000 7f3487923640 1 --2- 172.21.15.16:0/686840900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c004800 0x7f346c004c80 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f3470003d80 tx=0x7f346c005830 comp rx=0 tx=0).stop 2024-09-19T07:55:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.686+0000 7f3487923640 1 -- 172.21.15.16:0/686840900 shutdown_connections 2024-09-19T07:55:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.686+0000 7f3487923640 1 --2- 172.21.15.16:0/686840900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c004800 0x7f346c004c80 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.686+0000 7f3487923640 1 -- 172.21.15.16:0/686840900 >> 172.21.15.16:0/686840900 conn(0x7f348009d760 msgr2=0x7f348008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.686+0000 7f3487923640 1 -- 172.21.15.16:0/686840900 shutdown_connections 2024-09-19T07:55:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.686+0000 7f3487923640 1 -- 172.21.15.16:0/686840900 wait complete. 2024-09-19T07:55:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.687+0000 7f3487923640 1 Processor -- start 2024-09-19T07:55:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.688+0000 7f3487923640 1 -- start start 2024-09-19T07:55:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.688+0000 7f3487923640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f346c004800 0x7f3480157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.688+0000 7f3487923640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34800cd4c0 0x7f3480159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.688+0000 7f3487923640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34800cde60 0x7f34801599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.688+0000 7f3487923640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f348015f4b0 con 0x7f34800cde60 2024-09-19T07:55:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.688+0000 7f3487923640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f34800cef60 con 0x7f34800cd4c0 2024-09-19T07:55:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.688+0000 7f3487923640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f348015f630 con 0x7f346c004800 2024-09-19T07:55:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.688+0000 7f3485ec0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34800cde60 0x7f34801599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.688+0000 7f3485ec0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34800cde60 0x7f34801599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49798/0 (socket says 172.21.15.16:49798) 2024-09-19T07:55:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.688+0000 7f3485ec0640 1 -- 172.21.15.16:0/1370276156 learned_addr learned my addr 172.21.15.16:0/1370276156 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.689+0000 7f34856bf640 1 --2- 172.21.15.16:0/1370276156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f346c004800 0x7f3480157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.689+0000 7f3484ebe640 1 --2- 172.21.15.16:0/1370276156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34800cd4c0 0x7f3480159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.689+0000 7f3485ec0640 1 -- 172.21.15.16:0/1370276156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f346c004800 msgr2=0x7f3480157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.689+0000 7f3485ec0640 1 --2- 172.21.15.16:0/1370276156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f346c004800 0x7f3480157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.689+0000 7f3485ec0640 1 -- 172.21.15.16:0/1370276156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34800cd4c0 msgr2=0x7f3480159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.689+0000 7f3485ec0640 1 --2- 172.21.15.16:0/1370276156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34800cd4c0 0x7f3480159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.689+0000 7f3485ec0640 1 -- 172.21.15.16:0/1370276156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34800c8a80 con 0x7f34800cde60 2024-09-19T07:55:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.689+0000 7f3484ebe640 1 --2- 172.21.15.16:0/1370276156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34800cd4c0 0x7f3480159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.689+0000 7f3485ec0640 1 --2- 172.21.15.16:0/1370276156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34800cde60 0x7f34801599b0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f34800a22a0 tx=0x7f3478004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.689+0000 7f3475ffb640 1 -- 172.21.15.16:0/1370276156 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478007b90 con 0x7f34800cde60 2024-09-19T07:55:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.689+0000 7f3475ffb640 1 -- 172.21.15.16:0/1370276156 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3478019070 con 0x7f34800cde60 2024-09-19T07:55:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.689+0000 7f3487923640 1 -- 172.21.15.16:0/1370276156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34800c8020 con 0x7f34800cde60 2024-09-19T07:55:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.689+0000 7f3475ffb640 1 -- 172.21.15.16:0/1370276156 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478014410 con 0x7f34800cde60 2024-09-19T07:55:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.689+0000 7f3487923640 1 -- 172.21.15.16:0/1370276156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3480161770 con 0x7f34800cde60 2024-09-19T07:55:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.690+0000 7f3487923640 1 -- 172.21.15.16:0/1370276156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34800d4130 con 0x7f34800cde60 2024-09-19T07:55:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.691+0000 7f3475ffb640 1 -- 172.21.15.16:0/1370276156 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f34780145b0 con 0x7f34800cde60 2024-09-19T07:55:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.691+0000 7f3475ffb640 1 --2- 172.21.15.16:0/1370276156 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f346406c2e0 0x7f346406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.691+0000 7f3475ffb640 1 -- 172.21.15.16:0/1370276156 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f347808ba10 con 0x7f34800cde60 2024-09-19T07:55:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.691+0000 7f34856bf640 1 --2- 172.21.15.16:0/1370276156 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f346406c2e0 0x7f346406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.692+0000 7f34856bf640 1 --2- 172.21.15.16:0/1370276156 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f346406c2e0 0x7f346406e7a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f347c001520 tx=0x7f347c0014b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.695+0000 7f3475ffb640 1 -- 172.21.15.16:0/1370276156 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f347805a080 con 0x7f34800cde60 2024-09-19T07:55:49.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.875+0000 7f3487923640 1 -- 172.21.15.16:0/1370276156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f348006e0b0 con 0x7f34800cde60 2024-09-19T07:55:49.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.876+0000 7f3475ffb640 1 -- 172.21.15.16:0/1370276156 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) ==== 74+0+16968 (secure 0 0 0) 0x7f3478059980 con 0x7f34800cde60 2024-09-19T07:55:49.877 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:49.877 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":152,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:55:47.797354+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:55:44.728059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-09-19T07:55:45.128249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:55:49.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.879+0000 7f3487923640 1 -- 172.21.15.16:0/1370276156 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f346406c2e0 msgr2=0x7f346406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:49.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.879+0000 7f3487923640 1 --2- 172.21.15.16:0/1370276156 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f346406c2e0 0x7f346406e7a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f347c001520 tx=0x7f347c0014b0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.879+0000 7f3487923640 1 -- 172.21.15.16:0/1370276156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34800cde60 msgr2=0x7f34801599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:49.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.879+0000 7f3487923640 1 --2- 172.21.15.16:0/1370276156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34800cde60 0x7f34801599b0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f34800a22a0 tx=0x7f3478004780 comp rx=0 tx=0).stop 2024-09-19T07:55:49.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.879+0000 7f3487923640 1 -- 172.21.15.16:0/1370276156 shutdown_connections 2024-09-19T07:55:49.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.879+0000 7f3487923640 1 --2- 172.21.15.16:0/1370276156 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f346406c2e0 0x7f346406e7a0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.879+0000 7f3487923640 1 --2- 172.21.15.16:0/1370276156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34800cde60 0x7f34801599b0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.879+0000 7f3487923640 1 --2- 172.21.15.16:0/1370276156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34800cd4c0 0x7f3480159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.879+0000 7f3487923640 1 --2- 172.21.15.16:0/1370276156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f346c004800 0x7f3480157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.879+0000 7f3487923640 1 -- 172.21.15.16:0/1370276156 >> 172.21.15.16:0/1370276156 conn(0x7f348009d760 msgr2=0x7f3480161eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:49.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.879+0000 7f3487923640 1 -- 172.21.15.16:0/1370276156 shutdown_connections 2024-09-19T07:55:49.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.879+0000 7f3487923640 1 -- 172.21.15.16:0/1370276156 wait complete. 2024-09-19T07:55:49.890 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T07:55:49.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:55:49.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.992+0000 7f39b340e640 1 Processor -- start 2024-09-19T07:55:49.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.993+0000 7f39b340e640 1 -- start start 2024-09-19T07:55:49.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.993+0000 7f39b340e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39ac0d1ca0 0x7f39ac0cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:49.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.993+0000 7f39b340e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39ac0d22d0 con 0x7f39ac0d02f0 2024-09-19T07:55:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.993+0000 7f39b340e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39ac0d25b0 con 0x7f39ac15cb00 2024-09-19T07:55:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.993+0000 7f39b340e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f39ac0b40b0 con 0x7f39ac0d1ca0 2024-09-19T07:55:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.994+0000 7f39b19ab640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f39ac0d02f0 0x7f39ac1562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60726/0 (socket says 172.21.15.16:60726) 2024-09-19T07:55:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.994+0000 7f39b19ab640 1 -- 172.21.15.16:0/3218743736 learned_addr learned my addr 172.21.15.16:0/3218743736 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.994+0000 7f39b09a9640 1 --2- 172.21.15.16:0/3218743736 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39ac0d1ca0 0x7f39ac0cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.994+0000 7f39a3fff640 1 -- 172.21.15.16:0/3218743736 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1706550196 0 0) 0x7f39ac0d22d0 con 0x7f39ac0d02f0 2024-09-19T07:55:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.994+0000 7f39a3fff640 1 -- 172.21.15.16:0/3218743736 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3998003520 con 0x7f39ac0d02f0 2024-09-19T07:55:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.994+0000 7f39a3fff640 1 -- 172.21.15.16:0/3218743736 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2906747686 0 0) 0x7f39ac0d25b0 con 0x7f39ac15cb00 2024-09-19T07:55:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.994+0000 7f39a3fff640 1 -- 172.21.15.16:0/3218743736 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39ac0d22d0 con 0x7f39ac15cb00 2024-09-19T07:55:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.994+0000 7f39b09a9640 1 -- 172.21.15.16:0/3218743736 >> v1:172.21.15.146:6789/0 conn(0x7f39ac15cb00 legacy=0x7f39ac15cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.994+0000 7f39b09a9640 1 -- 172.21.15.16:0/3218743736 >> v1:172.21.15.16:6789/0 conn(0x7f39ac0d02f0 legacy=0x7f39ac1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.994+0000 7f39b09a9640 1 -- 172.21.15.16:0/3218743736 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39ac0d2890 con 0x7f39ac0d1ca0 2024-09-19T07:55:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.995+0000 7f39b09a9640 1 --2- 172.21.15.16:0/3218743736 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39ac0d1ca0 0x7f39ac0cfdb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f399c0097d0 tx=0x7f399c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=624f758a941f869f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.995+0000 7f39a3fff640 1 -- 172.21.15.16:0/3218743736 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c037070 con 0x7f39ac0d1ca0 2024-09-19T07:55:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.995+0000 7f39a3fff640 1 -- 172.21.15.16:0/3218743736 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f399c032440 con 0x7f39ac0d1ca0 2024-09-19T07:55:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.995+0000 7f39a3fff640 1 -- 172.21.15.16:0/3218743736 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c0327f0 con 0x7f39ac0d1ca0 2024-09-19T07:55:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.996+0000 7f39b340e640 1 -- 172.21.15.16:0/3218743736 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39ac0d1ca0 msgr2=0x7f39ac0cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.996+0000 7f39b340e640 1 --2- 172.21.15.16:0/3218743736 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39ac0d1ca0 0x7f39ac0cfdb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f399c0097d0 tx=0x7f399c02a180 comp rx=0 tx=0).stop 2024-09-19T07:55:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.996+0000 7f39b340e640 1 -- 172.21.15.16:0/3218743736 shutdown_connections 2024-09-19T07:55:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.996+0000 7f39b340e640 1 --2- 172.21.15.16:0/3218743736 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39ac0d1ca0 0x7f39ac0cfdb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.996+0000 7f39b340e640 1 -- 172.21.15.16:0/3218743736 >> 172.21.15.16:0/3218743736 conn(0x7f39ac067fe0 msgr2=0x7f39ac06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.996+0000 7f39b340e640 1 -- 172.21.15.16:0/3218743736 shutdown_connections 2024-09-19T07:55:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.996+0000 7f39b340e640 1 -- 172.21.15.16:0/3218743736 wait complete. 2024-09-19T07:55:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.997+0000 7f39b340e640 1 Processor -- start 2024-09-19T07:55:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.997+0000 7f39b340e640 1 -- start start 2024-09-19T07:55:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.998+0000 7f39b340e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39ac0d02f0 0x7f39ac0cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.998+0000 7f39b340e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39ac0d1ca0 0x7f39ac13eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.998+0000 7f39b340e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39ac15cb00 0x7f39ac0b79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.998+0000 7f39b340e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac15d5f0 con 0x7f39ac0d1ca0 2024-09-19T07:55:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.998+0000 7f39b340e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac0d0760 con 0x7f39ac0d02f0 2024-09-19T07:55:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.998+0000 7f39b340e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f39ac15d770 con 0x7f39ac15cb00 2024-09-19T07:55:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.998+0000 7f39b09a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39ac0d1ca0 0x7f39ac13eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.998+0000 7f39b09a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39ac0d1ca0 0x7f39ac13eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49804/0 (socket says 172.21.15.16:49804) 2024-09-19T07:55:50.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.998+0000 7f39b09a9640 1 -- 172.21.15.16:0/937825453 learned_addr learned my addr 172.21.15.16:0/937825453 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:50.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.998+0000 7f39b11aa640 1 --2- 172.21.15.16:0/937825453 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39ac0d02f0 0x7f39ac0cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:50.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.998+0000 7f39b19ab640 1 --2- 172.21.15.16:0/937825453 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39ac15cb00 0x7f39ac0b79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:50.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.998+0000 7f39b09a9640 1 -- 172.21.15.16:0/937825453 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39ac15cb00 msgr2=0x7f39ac0b79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:50.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.998+0000 7f39b09a9640 1 --2- 172.21.15.16:0/937825453 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39ac15cb00 0x7f39ac0b79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.998+0000 7f39b09a9640 1 -- 172.21.15.16:0/937825453 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39ac0d02f0 msgr2=0x7f39ac0cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:50.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.998+0000 7f39b09a9640 1 --2- 172.21.15.16:0/937825453 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39ac0d02f0 0x7f39ac0cbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.998+0000 7f39b09a9640 1 -- 172.21.15.16:0/937825453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39ac0b6940 con 0x7f39ac0d1ca0 2024-09-19T07:55:50.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.998+0000 7f39b09a9640 1 --2- 172.21.15.16:0/937825453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39ac0d1ca0 0x7f39ac13eaa0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f399c02a1c0 tx=0x7f399c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:50.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.998+0000 7f39a1ffb640 1 -- 172.21.15.16:0/937825453 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c041020 con 0x7f39ac0d1ca0 2024-09-19T07:55:50.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.999+0000 7f39a1ffb640 1 -- 172.21.15.16:0/937825453 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f399c03d6e0 con 0x7f39ac0d1ca0 2024-09-19T07:55:50.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.999+0000 7f39a1ffb640 1 -- 172.21.15.16:0/937825453 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c037040 con 0x7f39ac0d1ca0 2024-09-19T07:55:50.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.999+0000 7f39b340e640 1 -- 172.21.15.16:0/937825453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39ac0b6c20 con 0x7f39ac0d1ca0 2024-09-19T07:55:50.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:49.999+0000 7f39b340e640 1 -- 172.21.15.16:0/937825453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39ac0b71e0 con 0x7f39ac0d1ca0 2024-09-19T07:55:50.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.000+0000 7f39b340e640 1 -- 172.21.15.16:0/937825453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39ac0cfdb0 con 0x7f39ac0d1ca0 2024-09-19T07:55:50.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.000+0000 7f39a1ffb640 1 -- 172.21.15.16:0/937825453 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f399c044020 con 0x7f39ac0d1ca0 2024-09-19T07:55:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.001+0000 7f39a1ffb640 1 --2- 172.21.15.16:0/937825453 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f399006c2b0 0x7f399006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.001+0000 7f39a1ffb640 1 -- 172.21.15.16:0/937825453 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f399c0abda0 con 0x7f39ac0d1ca0 2024-09-19T07:55:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.001+0000 7f39b11aa640 1 --2- 172.21.15.16:0/937825453 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f399006c2b0 0x7f399006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.001+0000 7f39b11aa640 1 --2- 172.21.15.16:0/937825453 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f399006c2b0 0x7f399006e770 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f39a8001650 tx=0x7f39a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.005+0000 7f39a1ffb640 1 -- 172.21.15.16:0/937825453 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f399c04b030 con 0x7f39ac0d1ca0 2024-09-19T07:55:50.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.183+0000 7f39b340e640 1 -- 172.21.15.16:0/937825453 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f39ac0b1a40 con 0x7f399006c2b0 2024-09-19T07:55:50.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.189+0000 7f39a1ffb640 1 -- 172.21.15.16:0/937825453 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243152 (secure 0 0 0) 0x7f39ac0b1a40 con 0x7f399006c2b0 2024-09-19T07:55:50.192 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:55:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.192+0000 7f39b340e640 1 -- 172.21.15.16:0/937825453 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f399006c2b0 msgr2=0x7f399006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.192+0000 7f39b340e640 1 --2- 172.21.15.16:0/937825453 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f399006c2b0 0x7f399006e770 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f39a8001650 tx=0x7f39a8009040 comp rx=0 tx=0).stop 2024-09-19T07:55:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.192+0000 7f39b340e640 1 -- 172.21.15.16:0/937825453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39ac0d1ca0 msgr2=0x7f39ac13eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.192+0000 7f39b340e640 1 --2- 172.21.15.16:0/937825453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39ac0d1ca0 0x7f39ac13eaa0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f399c02a1c0 tx=0x7f399c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.192+0000 7f39b340e640 1 -- 172.21.15.16:0/937825453 shutdown_connections 2024-09-19T07:55:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.192+0000 7f39b340e640 1 --2- 172.21.15.16:0/937825453 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f399006c2b0 0x7f399006e770 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.192+0000 7f39b340e640 1 --2- 172.21.15.16:0/937825453 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39ac15cb00 0x7f39ac0b79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.192+0000 7f39b340e640 1 --2- 172.21.15.16:0/937825453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39ac0d1ca0 0x7f39ac13eaa0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.192+0000 7f39b340e640 1 --2- 172.21.15.16:0/937825453 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39ac0d02f0 0x7f39ac0cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.192+0000 7f39b340e640 1 -- 172.21.15.16:0/937825453 >> 172.21.15.16:0/937825453 conn(0x7f39ac067fe0 msgr2=0x7f39ac070800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.193+0000 7f39b340e640 1 -- 172.21.15.16:0/937825453 shutdown_connections 2024-09-19T07:55:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.193+0000 7f39b340e640 1 -- 172.21.15.16:0/937825453 wait complete. 2024-09-19T07:55:50.204 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":534,"stamp":"2024-09-19T07:55:48.640068+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592002,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":182,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":88,"acting":88,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":766080,"kb_used_data":15760,"kb_used_omap":12,"kb_used_meta":750131,"kb_avail":754208640,"statfs":{"total":773094113280,"available":772309647360,"internally_reserved":0,"allocated":16138240,"data_stored":8107522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":768134704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1634,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000880"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800604+0000","last_change":"2024-09-19T07:55:46.819762+0000","last_active":"2024-09-19T07:55:47.800604+0000","last_peered":"2024-09-19T07:55:47.800604+0000","last_clean":"2024-09-19T07:55:47.800604+0000","last_became_active":"2024-09-19T07:55:46.819203+0000","last_became_peered":"2024-09-19T07:55:46.819203+0000","last_unstale":"2024-09-19T07:55:47.800604+0000","last_undegraded":"2024-09-19T07:55:47.800604+0000","last_fullsized":"2024-09-19T07:55:47.800604+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:59.757492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800582+0000","last_change":"2024-09-19T07:55:46.804795+0000","last_active":"2024-09-19T07:55:47.800582+0000","last_peered":"2024-09-19T07:55:47.800582+0000","last_clean":"2024-09-19T07:55:47.800582+0000","last_became_active":"2024-09-19T07:55:46.804489+0000","last_became_peered":"2024-09-19T07:55:46.804489+0000","last_unstale":"2024-09-19T07:55:47.800582+0000","last_undegraded":"2024-09-19T07:55:47.800582+0000","last_fullsized":"2024-09-19T07:55:47.800582+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:18.647644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.801046+0000","last_change":"2024-09-19T07:55:46.810344+0000","last_active":"2024-09-19T07:55:47.801046+0000","last_peered":"2024-09-19T07:55:47.801046+0000","last_clean":"2024-09-19T07:55:47.801046+0000","last_became_active":"2024-09-19T07:55:46.810035+0000","last_became_peered":"2024-09-19T07:55:46.810035+0000","last_unstale":"2024-09-19T07:55:47.801046+0000","last_undegraded":"2024-09-19T07:55:47.801046+0000","last_fullsized":"2024-09-19T07:55:47.801046+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:29.247081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800997+0000","last_change":"2024-09-19T07:55:46.817698+0000","last_active":"2024-09-19T07:55:47.800997+0000","last_peered":"2024-09-19T07:55:47.800997+0000","last_clean":"2024-09-19T07:55:47.800997+0000","last_became_active":"2024-09-19T07:55:46.817376+0000","last_became_peered":"2024-09-19T07:55:46.817376+0000","last_unstale":"2024-09-19T07:55:47.800997+0000","last_undegraded":"2024-09-19T07:55:47.800997+0000","last_fullsized":"2024-09-19T07:55:47.800997+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:07.805087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827703+0000","last_change":"2024-09-19T07:55:46.811483+0000","last_active":"2024-09-19T07:55:47.827703+0000","last_peered":"2024-09-19T07:55:47.827703+0000","last_clean":"2024-09-19T07:55:47.827703+0000","last_became_active":"2024-09-19T07:55:46.811206+0000","last_became_peered":"2024-09-19T07:55:46.811206+0000","last_unstale":"2024-09-19T07:55:47.827703+0000","last_undegraded":"2024-09-19T07:55:47.827703+0000","last_fullsized":"2024-09-19T07:55:47.827703+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:47.785611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.801098+0000","last_change":"2024-09-19T07:55:46.806494+0000","last_active":"2024-09-19T07:55:47.801098+0000","last_peered":"2024-09-19T07:55:47.801098+0000","last_clean":"2024-09-19T07:55:47.801098+0000","last_became_active":"2024-09-19T07:55:46.806196+0000","last_became_peered":"2024-09-19T07:55:46.806196+0000","last_unstale":"2024-09-19T07:55:47.801098+0000","last_undegraded":"2024-09-19T07:55:47.801098+0000","last_fullsized":"2024-09-19T07:55:47.801098+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:23.021974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.791579+0000","last_change":"2024-09-19T07:55:45.791579+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.791579+0000","last_undegraded":"2024-09-19T07:55:45.791579+0000","last_fullsized":"2024-09-19T07:55:45.791579+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827721+0000","last_change":"2024-09-19T07:55:46.817878+0000","last_active":"2024-09-19T07:55:47.827721+0000","last_peered":"2024-09-19T07:55:47.827721+0000","last_clean":"2024-09-19T07:55:47.827721+0000","last_became_active":"2024-09-19T07:55:46.817518+0000","last_became_peered":"2024-09-19T07:55:46.817518+0000","last_unstale":"2024-09-19T07:55:47.827721+0000","last_undegraded":"2024-09-19T07:55:47.827721+0000","last_fullsized":"2024-09-19T07:55:47.827721+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:07.791830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827831+0000","last_change":"2024-09-19T07:55:46.819719+0000","last_active":"2024-09-19T07:55:47.827831+0000","last_peered":"2024-09-19T07:55:47.827831+0000","last_clean":"2024-09-19T07:55:47.827831+0000","last_became_active":"2024-09-19T07:55:46.811669+0000","last_became_peered":"2024-09-19T07:55:46.811669+0000","last_unstale":"2024-09-19T07:55:47.827831+0000","last_undegraded":"2024-09-19T07:55:47.827831+0000","last_fullsized":"2024-09-19T07:55:47.827831+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:40.791876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.790719+0000","last_change":"2024-09-19T07:55:45.790719+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.790719+0000","last_undegraded":"2024-09-19T07:55:45.790719+0000","last_fullsized":"2024-09-19T07:55:45.790719+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827493+0000","last_change":"2024-09-19T07:55:46.818518+0000","last_active":"2024-09-19T07:55:47.827493+0000","last_peered":"2024-09-19T07:55:47.827493+0000","last_clean":"2024-09-19T07:55:47.827493+0000","last_became_active":"2024-09-19T07:55:46.818264+0000","last_became_peered":"2024-09-19T07:55:46.818264+0000","last_unstale":"2024-09-19T07:55:47.827493+0000","last_undegraded":"2024-09-19T07:55:47.827493+0000","last_fullsized":"2024-09-19T07:55:47.827493+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:53.813959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827377+0000","last_change":"2024-09-19T07:55:46.813694+0000","last_active":"2024-09-19T07:55:47.827377+0000","last_peered":"2024-09-19T07:55:47.827377+0000","last_clean":"2024-09-19T07:55:47.827377+0000","last_became_active":"2024-09-19T07:55:46.813395+0000","last_became_peered":"2024-09-19T07:55:46.813395+0000","last_unstale":"2024-09-19T07:55:47.827377+0000","last_undegraded":"2024-09-19T07:55:47.827377+0000","last_fullsized":"2024-09-19T07:55:47.827377+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:46:54.731529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-19T07:55:45.801899+0000","last_change":"2024-09-19T07:55:45.800564+0000","last_active":"2024-09-19T07:55:45.801899+0000","last_peered":"2024-09-19T07:55:45.801899+0000","last_clean":"2024-09-19T07:55:45.801899+0000","last_became_active":"2024-09-19T07:55:45.800078+0000","last_became_peered":"2024-09-19T07:55:45.800078+0000","last_unstale":"2024-09-19T07:55:45.801899+0000","last_undegraded":"2024-09-19T07:55:45.801899+0000","last_fullsized":"2024-09-19T07:55:45.801899+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:18:15.498014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800845+0000","last_change":"2024-09-19T07:55:46.818421+0000","last_active":"2024-09-19T07:55:47.800845+0000","last_peered":"2024-09-19T07:55:47.800845+0000","last_clean":"2024-09-19T07:55:47.800845+0000","last_became_active":"2024-09-19T07:55:46.818140+0000","last_became_peered":"2024-09-19T07:55:46.818140+0000","last_unstale":"2024-09-19T07:55:47.800845+0000","last_undegraded":"2024-09-19T07:55:47.800845+0000","last_fullsized":"2024-09-19T07:55:47.800845+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:20.053063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"151'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800833+0000","last_change":"2024-09-19T07:55:45.805798+0000","last_active":"2024-09-19T07:55:47.800833+0000","last_peered":"2024-09-19T07:55:47.800833+0000","last_clean":"2024-09-19T07:55:47.800833+0000","last_became_active":"2024-09-19T07:55:45.805511+0000","last_became_peered":"2024-09-19T07:55:45.805511+0000","last_unstale":"2024-09-19T07:55:47.800833+0000","last_undegraded":"2024-09-19T07:55:47.800833+0000","last_fullsized":"2024-09-19T07:55:47.800833+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:18.474678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.790402+0000","last_change":"2024-09-19T07:55:45.790402+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.790402+0000","last_undegraded":"2024-09-19T07:55:45.790402+0000","last_fullsized":"2024-09-19T07:55:45.790402+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.790244+0000","last_change":"2024-09-19T07:55:45.790244+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.790244+0000","last_undegraded":"2024-09-19T07:55:45.790244+0000","last_fullsized":"2024-09-19T07:55:45.790244+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-19T07:55:45.805997+0000","last_change":"2024-09-19T07:55:45.804265+0000","last_active":"2024-09-19T07:55:45.805997+0000","last_peered":"2024-09-19T07:55:45.805997+0000","last_clean":"2024-09-19T07:55:45.805997+0000","last_became_active":"2024-09-19T07:55:45.804000+0000","last_became_peered":"2024-09-19T07:55:45.804000+0000","last_unstale":"2024-09-19T07:55:45.805997+0000","last_undegraded":"2024-09-19T07:55:45.805997+0000","last_fullsized":"2024-09-19T07:55:45.805997+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:00.654293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800706+0000","last_change":"2024-09-19T07:55:45.797824+0000","last_active":"2024-09-19T07:55:47.800706+0000","last_peered":"2024-09-19T07:55:47.800706+0000","last_clean":"2024-09-19T07:55:47.800706+0000","last_became_active":"2024-09-19T07:55:45.797559+0000","last_became_peered":"2024-09-19T07:55:45.797559+0000","last_unstale":"2024-09-19T07:55:47.800706+0000","last_undegraded":"2024-09-19T07:55:47.800706+0000","last_fullsized":"2024-09-19T07:55:47.800706+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:48.369553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827717+0000","last_change":"2024-09-19T07:55:45.800048+0000","last_active":"2024-09-19T07:55:47.827717+0000","last_peered":"2024-09-19T07:55:47.827717+0000","last_clean":"2024-09-19T07:55:47.827717+0000","last_became_active":"2024-09-19T07:55:45.799749+0000","last_became_peered":"2024-09-19T07:55:45.799749+0000","last_unstale":"2024-09-19T07:55:47.827717+0000","last_undegraded":"2024-09-19T07:55:47.827717+0000","last_fullsized":"2024-09-19T07:55:47.827717+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:42.846081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827407+0000","last_change":"2024-09-19T07:55:46.819081+0000","last_active":"2024-09-19T07:55:47.827407+0000","last_peered":"2024-09-19T07:55:47.827407+0000","last_clean":"2024-09-19T07:55:47.827407+0000","last_became_active":"2024-09-19T07:55:46.818785+0000","last_became_peered":"2024-09-19T07:55:46.818785+0000","last_unstale":"2024-09-19T07:55:47.827407+0000","last_undegraded":"2024-09-19T07:55:47.827407+0000","last_fullsized":"2024-09-19T07:55:47.827407+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:48.362583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.791181+0000","last_change":"2024-09-19T07:55:45.791181+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.791181+0000","last_undegraded":"2024-09-19T07:55:45.791181+0000","last_fullsized":"2024-09-19T07:55:45.791181+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"151'6","reported_seq":20,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827602+0000","last_change":"2024-09-19T07:55:45.802798+0000","last_active":"2024-09-19T07:55:47.827602+0000","last_peered":"2024-09-19T07:55:47.827602+0000","last_clean":"2024-09-19T07:55:47.827602+0000","last_became_active":"2024-09-19T07:55:45.802458+0000","last_became_peered":"2024-09-19T07:55:45.802458+0000","last_unstale":"2024-09-19T07:55:47.827602+0000","last_undegraded":"2024-09-19T07:55:47.827602+0000","last_fullsized":"2024-09-19T07:55:47.827602+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:08.281321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800893+0000","last_change":"2024-09-19T07:55:46.817083+0000","last_active":"2024-09-19T07:55:47.800893+0000","last_peered":"2024-09-19T07:55:47.800893+0000","last_clean":"2024-09-19T07:55:47.800893+0000","last_became_active":"2024-09-19T07:55:46.816804+0000","last_became_peered":"2024-09-19T07:55:46.816804+0000","last_unstale":"2024-09-19T07:55:47.800893+0000","last_undegraded":"2024-09-19T07:55:47.800893+0000","last_fullsized":"2024-09-19T07:55:47.800893+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:55.546951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827440+0000","last_change":"2024-09-19T07:55:45.801135+0000","last_active":"2024-09-19T07:55:47.827440+0000","last_peered":"2024-09-19T07:55:47.827440+0000","last_clean":"2024-09-19T07:55:47.827440+0000","last_became_active":"2024-09-19T07:55:45.800837+0000","last_became_peered":"2024-09-19T07:55:45.800837+0000","last_unstale":"2024-09-19T07:55:47.827440+0000","last_undegraded":"2024-09-19T07:55:47.827440+0000","last_fullsized":"2024-09-19T07:55:47.827440+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:30:46.437440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.791295+0000","last_change":"2024-09-19T07:55:45.791295+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.791295+0000","last_undegraded":"2024-09-19T07:55:45.791295+0000","last_fullsized":"2024-09-19T07:55:45.791295+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"151'8","reported_seq":22,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800808+0000","last_change":"2024-09-19T07:55:45.795277+0000","last_active":"2024-09-19T07:55:47.800808+0000","last_peered":"2024-09-19T07:55:47.800808+0000","last_clean":"2024-09-19T07:55:47.800808+0000","last_became_active":"2024-09-19T07:55:45.794876+0000","last_became_peered":"2024-09-19T07:55:45.794876+0000","last_unstale":"2024-09-19T07:55:47.800808+0000","last_undegraded":"2024-09-19T07:55:47.800808+0000","last_fullsized":"2024-09-19T07:55:47.800808+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:50.194217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800585+0000","last_change":"2024-09-19T07:55:46.817868+0000","last_active":"2024-09-19T07:55:47.800585+0000","last_peered":"2024-09-19T07:55:47.800585+0000","last_clean":"2024-09-19T07:55:47.800585+0000","last_became_active":"2024-09-19T07:55:46.817545+0000","last_became_peered":"2024-09-19T07:55:46.817545+0000","last_unstale":"2024-09-19T07:55:47.800585+0000","last_undegraded":"2024-09-19T07:55:47.800585+0000","last_fullsized":"2024-09-19T07:55:47.800585+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:37:45.202649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"151'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827791+0000","last_change":"2024-09-19T07:55:45.797085+0000","last_active":"2024-09-19T07:55:47.827791+0000","last_peered":"2024-09-19T07:55:47.827791+0000","last_clean":"2024-09-19T07:55:47.827791+0000","last_became_active":"2024-09-19T07:55:45.796793+0000","last_became_peered":"2024-09-19T07:55:45.796793+0000","last_unstale":"2024-09-19T07:55:47.827791+0000","last_undegraded":"2024-09-19T07:55:47.827791+0000","last_fullsized":"2024-09-19T07:55:47.827791+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:14.418996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.e","version":"151'2","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.801123+0000","last_change":"2024-09-19T07:55:45.803736+0000","last_active":"2024-09-19T07:55:47.801123+0000","last_peered":"2024-09-19T07:55:47.801123+0000","last_clean":"2024-09-19T07:55:47.801123+0000","last_became_active":"2024-09-19T07:55:45.803419+0000","last_became_peered":"2024-09-19T07:55:45.803419+0000","last_unstale":"2024-09-19T07:55:47.801123+0000","last_undegraded":"2024-09-19T07:55:47.801123+0000","last_fullsized":"2024-09-19T07:55:47.801123+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:39:47.947756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800962+0000","last_change":"2024-09-19T07:55:45.805741+0000","last_active":"2024-09-19T07:55:47.800962+0000","last_peered":"2024-09-19T07:55:47.800962+0000","last_clean":"2024-09-19T07:55:47.800962+0000","last_became_active":"2024-09-19T07:55:45.805464+0000","last_became_peered":"2024-09-19T07:55:45.805464+0000","last_unstale":"2024-09-19T07:55:47.800962+0000","last_undegraded":"2024-09-19T07:55:47.800962+0000","last_fullsized":"2024-09-19T07:55:47.800962+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:31.558613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827844+0000","last_change":"2024-09-19T07:55:46.817735+0000","last_active":"2024-09-19T07:55:47.827844+0000","last_peered":"2024-09-19T07:55:47.827844+0000","last_clean":"2024-09-19T07:55:47.827844+0000","last_became_active":"2024-09-19T07:55:46.817466+0000","last_became_peered":"2024-09-19T07:55:46.817466+0000","last_unstale":"2024-09-19T07:55:47.827844+0000","last_undegraded":"2024-09-19T07:55:47.827844+0000","last_fullsized":"2024-09-19T07:55:47.827844+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:52.010816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-19T07:55:45.798537+0000","last_change":"2024-09-19T07:55:45.797312+0000","last_active":"2024-09-19T07:55:45.798537+0000","last_peered":"2024-09-19T07:55:45.798537+0000","last_clean":"2024-09-19T07:55:45.798537+0000","last_became_active":"2024-09-19T07:55:45.796832+0000","last_became_peered":"2024-09-19T07:55:45.796832+0000","last_unstale":"2024-09-19T07:55:45.798537+0000","last_undegraded":"2024-09-19T07:55:45.798537+0000","last_fullsized":"2024-09-19T07:55:45.798537+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:15.635183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800693+0000","last_change":"2024-09-19T07:55:46.819496+0000","last_active":"2024-09-19T07:55:47.800693+0000","last_peered":"2024-09-19T07:55:47.800693+0000","last_clean":"2024-09-19T07:55:47.800693+0000","last_became_active":"2024-09-19T07:55:46.818787+0000","last_became_peered":"2024-09-19T07:55:46.818787+0000","last_unstale":"2024-09-19T07:55:47.800693+0000","last_undegraded":"2024-09-19T07:55:47.800693+0000","last_fullsized":"2024-09-19T07:55:47.800693+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:58.971945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"152'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.828794+0000","last_change":"2024-09-19T07:55:45.800698+0000","last_active":"2024-09-19T07:55:47.828794+0000","last_peered":"2024-09-19T07:55:47.828794+0000","last_clean":"2024-09-19T07:55:47.828794+0000","last_became_active":"2024-09-19T07:55:45.800012+0000","last_became_peered":"2024-09-19T07:55:45.800012+0000","last_unstale":"2024-09-19T07:55:47.828794+0000","last_undegraded":"2024-09-19T07:55:47.828794+0000","last_fullsized":"2024-09-19T07:55:47.828794+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:21.500902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.794831+0000","last_change":"2024-09-19T07:55:45.794831+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.794831+0000","last_undegraded":"2024-09-19T07:55:45.794831+0000","last_fullsized":"2024-09-19T07:55:45.794831+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827580+0000","last_change":"2024-09-19T07:55:45.802021+0000","last_active":"2024-09-19T07:55:47.827580+0000","last_peered":"2024-09-19T07:55:47.827580+0000","last_clean":"2024-09-19T07:55:47.827580+0000","last_became_active":"2024-09-19T07:55:45.801518+0000","last_became_peered":"2024-09-19T07:55:45.801518+0000","last_unstale":"2024-09-19T07:55:47.827580+0000","last_undegraded":"2024-09-19T07:55:47.827580+0000","last_fullsized":"2024-09-19T07:55:47.827580+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:59.000128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800755+0000","last_change":"2024-09-19T07:55:46.812665+0000","last_active":"2024-09-19T07:55:47.800755+0000","last_peered":"2024-09-19T07:55:47.800755+0000","last_clean":"2024-09-19T07:55:47.800755+0000","last_became_active":"2024-09-19T07:55:46.812424+0000","last_became_peered":"2024-09-19T07:55:46.812424+0000","last_unstale":"2024-09-19T07:55:47.800755+0000","last_undegraded":"2024-09-19T07:55:47.800755+0000","last_fullsized":"2024-09-19T07:55:47.800755+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:50.004806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-19T07:55:45.808438+0000","last_change":"2024-09-19T07:55:45.804911+0000","last_active":"2024-09-19T07:55:45.808438+0000","last_peered":"2024-09-19T07:55:45.808438+0000","last_clean":"2024-09-19T07:55:45.808438+0000","last_became_active":"2024-09-19T07:55:45.804578+0000","last_became_peered":"2024-09-19T07:55:45.804578+0000","last_unstale":"2024-09-19T07:55:45.808438+0000","last_undegraded":"2024-09-19T07:55:45.808438+0000","last_fullsized":"2024-09-19T07:55:45.808438+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:04.552288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827507+0000","last_change":"2024-09-19T07:55:45.801889+0000","last_active":"2024-09-19T07:55:47.827507+0000","last_peered":"2024-09-19T07:55:47.827507+0000","last_clean":"2024-09-19T07:55:47.827507+0000","last_became_active":"2024-09-19T07:55:45.801569+0000","last_became_peered":"2024-09-19T07:55:45.801569+0000","last_unstale":"2024-09-19T07:55:47.827507+0000","last_undegraded":"2024-09-19T07:55:47.827507+0000","last_fullsized":"2024-09-19T07:55:47.827507+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:19.834999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827678+0000","last_change":"2024-09-19T07:55:46.816559+0000","last_active":"2024-09-19T07:55:47.827678+0000","last_peered":"2024-09-19T07:55:47.827678+0000","last_clean":"2024-09-19T07:55:47.827678+0000","last_became_active":"2024-09-19T07:55:46.816305+0000","last_became_peered":"2024-09-19T07:55:46.816305+0000","last_unstale":"2024-09-19T07:55:47.827678+0000","last_undegraded":"2024-09-19T07:55:47.827678+0000","last_fullsized":"2024-09-19T07:55:47.827678+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:27.599531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.790864+0000","last_change":"2024-09-19T07:55:45.790864+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.790864+0000","last_undegraded":"2024-09-19T07:55:45.790864+0000","last_fullsized":"2024-09-19T07:55:45.790864+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827644+0000","last_change":"2024-09-19T07:55:45.800943+0000","last_active":"2024-09-19T07:55:47.827644+0000","last_peered":"2024-09-19T07:55:47.827644+0000","last_clean":"2024-09-19T07:55:47.827644+0000","last_became_active":"2024-09-19T07:55:45.800428+0000","last_became_peered":"2024-09-19T07:55:45.800428+0000","last_unstale":"2024-09-19T07:55:47.827644+0000","last_undegraded":"2024-09-19T07:55:47.827644+0000","last_fullsized":"2024-09-19T07:55:47.827644+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:40.722531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":400,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T07:55:44.084202+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:55:44.084202+0000","last_peered":"2024-09-19T07:55:44.084202+0000","last_clean":"2024-09-19T07:55:44.084202+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:55:44.084202+0000","last_undegraded":"2024-09-19T07:55:44.084202+0000","last_fullsized":"2024-09-19T07:55:44.084202+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":52,"acting":52,"num_store_stats":3},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":1634,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":34,"acting":34,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509597,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83476,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288364,"statfs":{"total":96636764160,"available":96551284736,"internally_reserved":0,"allocated":2424832,"data_stored":1439155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":105180,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266660,"statfs":{"total":96636764160,"available":96529059840,"internally_reserved":0,"allocated":1843200,"data_stored":854322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509597,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73192,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298648,"statfs":{"total":96636764160,"available":96561815552,"internally_reserved":0,"allocated":1855488,"data_stored":859841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509597,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":87084,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284756,"statfs":{"total":96636764160,"available":96547590144,"internally_reserved":0,"allocated":1855488,"data_stored":859841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509596,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105696,"kb_used_data":2380,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266144,"statfs":{"total":96636764160,"available":96528531456,"internally_reserved":0,"allocated":2437120,"data_stored":1444690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542301,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119088,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252752,"statfs":{"total":96636764160,"available":96514818048,"internally_reserved":0,"allocated":1925120,"data_stored":888784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542301,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83052,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288788,"statfs":{"total":96636764160,"available":96551718912,"internally_reserved":0,"allocated":1916928,"data_stored":888180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575006,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109312,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262528,"statfs":{"total":96636764160,"available":96524828672,"internally_reserved":0,"allocated":1880064,"data_stored":872709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:55:50.209 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:55:50.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.346+0000 7f42ecb86640 1 Processor -- start 2024-09-19T07:55:50.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.346+0000 7f42ecb86640 1 -- start start 2024-09-19T07:55:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.347+0000 7f42ecb86640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f42e815bee0 0x7f42e80d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:50.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.347+0000 7f42ecb86640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f42e815c2c0 0x7f42e80d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.347+0000 7f42ecb86640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42e80cf200 0x7f42e80cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.347+0000 7f42ecb86640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f42e80ceaf0 con 0x7f42e815c2c0 2024-09-19T07:55:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.347+0000 7f42ecb86640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f42e80cdee0 con 0x7f42e815bee0 2024-09-19T07:55:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.347+0000 7f42ecb86640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f42e80ce1c0 con 0x7f42e80cf200 2024-09-19T07:55:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.347+0000 7f42ddd9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f42e815c2c0 0x7f42e80d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.347+0000 7f42ddd9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f42e815c2c0 0x7f42e80d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49820/0 (socket says 172.21.15.16:49820) 2024-09-19T07:55:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.347+0000 7f42ddd9b640 1 -- 172.21.15.16:0/369339093 learned_addr learned my addr 172.21.15.16:0/369339093 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.347+0000 7f42e659c640 1 --2- 172.21.15.16:0/369339093 >> v2:172.21.15.146:3300/0 conn(0x7f42e815bee0 0x7f42e80d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.347+0000 7f42e6d9d640 1 --2- 172.21.15.16:0/369339093 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42e80cf200 0x7f42e80cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.347+0000 7f42ddd9b640 1 -- 172.21.15.16:0/369339093 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42e80cf200 msgr2=0x7f42e80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.347+0000 7f42ddd9b640 1 --2- 172.21.15.16:0/369339093 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42e80cf200 0x7f42e80cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.347+0000 7f42ddd9b640 1 -- 172.21.15.16:0/369339093 >> v2:172.21.15.146:3300/0 conn(0x7f42e815bee0 msgr2=0x7f42e80d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:50.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.347+0000 7f42ddd9b640 1 --2- 172.21.15.16:0/369339093 >> v2:172.21.15.146:3300/0 conn(0x7f42e815bee0 0x7f42e80d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.347+0000 7f42ddd9b640 1 -- 172.21.15.16:0/369339093 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e80ce4a0 con 0x7f42e815c2c0 2024-09-19T07:55:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.348+0000 7f42ddd9b640 1 --2- 172.21.15.16:0/369339093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e815c2c0 0x7f42e80d09f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f42e80b2f40 tx=0x7f42d002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c746a90bc2f11cb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.348+0000 7f42e5d9b640 1 -- 172.21.15.16:0/369339093 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0037070 con 0x7f42e815c2c0 2024-09-19T07:55:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.348+0000 7f42e5d9b640 1 -- 172.21.15.16:0/369339093 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d0032440 con 0x7f42e815c2c0 2024-09-19T07:55:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.348+0000 7f42e5d9b640 1 -- 172.21.15.16:0/369339093 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0032810 con 0x7f42e815c2c0 2024-09-19T07:55:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.348+0000 7f42ecb86640 1 -- 172.21.15.16:0/369339093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e815c2c0 msgr2=0x7f42e80d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.348+0000 7f42ecb86640 1 --2- 172.21.15.16:0/369339093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e815c2c0 0x7f42e80d09f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f42e80b2f40 tx=0x7f42d002a1e0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.348+0000 7f42ecb86640 1 -- 172.21.15.16:0/369339093 shutdown_connections 2024-09-19T07:55:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.348+0000 7f42ecb86640 1 --2- 172.21.15.16:0/369339093 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42e80cf200 0x7f42e80cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.348+0000 7f42ecb86640 1 --2- 172.21.15.16:0/369339093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e815c2c0 0x7f42e80d09f0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.348+0000 7f42ecb86640 1 --2- 172.21.15.16:0/369339093 >> v2:172.21.15.146:3300/0 conn(0x7f42e815bee0 0x7f42e80d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.348+0000 7f42ecb86640 1 -- 172.21.15.16:0/369339093 >> 172.21.15.16:0/369339093 conn(0x7f42e806bdf0 msgr2=0x7f42e806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:50.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.349+0000 7f42ecb86640 1 -- 172.21.15.16:0/369339093 shutdown_connections 2024-09-19T07:55:50.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.349+0000 7f42ecb86640 1 -- 172.21.15.16:0/369339093 wait complete. 2024-09-19T07:55:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.350+0000 7f42ecb86640 1 Processor -- start 2024-09-19T07:55:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.350+0000 7f42ecb86640 1 -- start start 2024-09-19T07:55:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.351+0000 7f42ecb86640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42e80cf200 0x7f42e80adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.351+0000 7f42ecb86640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42e815bee0 0x7f42e80cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.351+0000 7f42ecb86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e815c2c0 0x7f42e8159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.351+0000 7f42ecb86640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f42e815fd40 con 0x7f42e815c2c0 2024-09-19T07:55:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.351+0000 7f42ecb86640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f42e80cfbd0 con 0x7f42e815bee0 2024-09-19T07:55:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.351+0000 7f42ecb86640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f42e815fec0 con 0x7f42e80cf200 2024-09-19T07:55:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.351+0000 7f42e6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e815c2c0 0x7f42e8159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:50.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.351+0000 7f42e6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e815c2c0 0x7f42e8159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49834/0 (socket says 172.21.15.16:49834) 2024-09-19T07:55:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.351+0000 7f42e6d9d640 1 -- 172.21.15.16:0/687092039 learned_addr learned my addr 172.21.15.16:0/687092039 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.351+0000 7f42e659c640 1 --2- 172.21.15.16:0/687092039 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42e80cf200 0x7f42e80adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.351+0000 7f42ddd9b640 1 --2- 172.21.15.16:0/687092039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42e815bee0 0x7f42e80cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.351+0000 7f42e6d9d640 1 -- 172.21.15.16:0/687092039 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42e80cf200 msgr2=0x7f42e80adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.351+0000 7f42e6d9d640 1 --2- 172.21.15.16:0/687092039 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42e80cf200 0x7f42e80adb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.351+0000 7f42e6d9d640 1 -- 172.21.15.16:0/687092039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42e815bee0 msgr2=0x7f42e80cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.351+0000 7f42e6d9d640 1 --2- 172.21.15.16:0/687092039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42e815bee0 0x7f42e80cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.351+0000 7f42e6d9d640 1 -- 172.21.15.16:0/687092039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e80ae0b0 con 0x7f42e815c2c0 2024-09-19T07:55:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.351+0000 7f42e659c640 1 --2- 172.21.15.16:0/687092039 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42e80cf200 0x7f42e80adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.351+0000 7f42ddd9b640 1 --2- 172.21.15.16:0/687092039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42e815bee0 0x7f42e80cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.351+0000 7f42e6d9d640 1 --2- 172.21.15.16:0/687092039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e815c2c0 0x7f42e8159690 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f42e0007e60 tx=0x7f42e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.352+0000 7f42df7fe640 1 -- 172.21.15.16:0/687092039 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0014420 con 0x7f42e815c2c0 2024-09-19T07:55:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.352+0000 7f42df7fe640 1 -- 172.21.15.16:0/687092039 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e0019070 con 0x7f42e815c2c0 2024-09-19T07:55:50.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.352+0000 7f42ecb86640 1 -- 172.21.15.16:0/687092039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e80ad490 con 0x7f42e815c2c0 2024-09-19T07:55:50.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.352+0000 7f42df7fe640 1 -- 172.21.15.16:0/687092039 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0014820 con 0x7f42e815c2c0 2024-09-19T07:55:50.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.352+0000 7f42ecb86640 1 -- 172.21.15.16:0/687092039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e80cc370 con 0x7f42e815c2c0 2024-09-19T07:55:50.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.353+0000 7f42ecb86640 1 -- 172.21.15.16:0/687092039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e80cf690 con 0x7f42e815c2c0 2024-09-19T07:55:50.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.353+0000 7f42df7fe640 1 -- 172.21.15.16:0/687092039 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f42e00149c0 con 0x7f42e815c2c0 2024-09-19T07:55:50.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.354+0000 7f42df7fe640 1 --2- 172.21.15.16:0/687092039 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f42c806c2b0 0x7f42c806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:50.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.354+0000 7f42df7fe640 1 -- 172.21.15.16:0/687092039 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f42e008bd70 con 0x7f42e815c2c0 2024-09-19T07:55:50.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.354+0000 7f42e659c640 1 --2- 172.21.15.16:0/687092039 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f42c806c2b0 0x7f42c806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:50.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.355+0000 7f42e659c640 1 --2- 172.21.15.16:0/687092039 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f42c806c2b0 0x7f42c806e770 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f42d8006fd0 tx=0x7f42d8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.358+0000 7f42df7fe640 1 -- 172.21.15.16:0/687092039 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f42e005a3e0 con 0x7f42e815c2c0 2024-09-19T07:55:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.532+0000 7f42ecb86640 1 -- 172.21.15.16:0/687092039 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f42e80b44f0 con 0x7f42c806c2b0 2024-09-19T07:55:50.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.538+0000 7f42df7fe640 1 -- 172.21.15.16:0/687092039 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243152 (secure 0 0 0) 0x7f42e80b44f0 con 0x7f42c806c2b0 2024-09-19T07:55:50.541 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:50.542 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:55:50.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.541+0000 7f42ecb86640 1 -- 172.21.15.16:0/687092039 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f42c806c2b0 msgr2=0x7f42c806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:50.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.541+0000 7f42ecb86640 1 --2- 172.21.15.16:0/687092039 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f42c806c2b0 0x7f42c806e770 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f42d8006fd0 tx=0x7f42d8008040 comp rx=0 tx=0).stop 2024-09-19T07:55:50.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.541+0000 7f42ecb86640 1 -- 172.21.15.16:0/687092039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e815c2c0 msgr2=0x7f42e8159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:50.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.541+0000 7f42ecb86640 1 --2- 172.21.15.16:0/687092039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e815c2c0 0x7f42e8159690 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f42e0007e60 tx=0x7f42e000e4b0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.541+0000 7f42ecb86640 1 -- 172.21.15.16:0/687092039 shutdown_connections 2024-09-19T07:55:50.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.541+0000 7f42ecb86640 1 --2- 172.21.15.16:0/687092039 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f42c806c2b0 0x7f42c806e770 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.541+0000 7f42ecb86640 1 --2- 172.21.15.16:0/687092039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e815c2c0 0x7f42e8159690 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.541+0000 7f42ecb86640 1 --2- 172.21.15.16:0/687092039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42e815bee0 0x7f42e80cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.541+0000 7f42ecb86640 1 --2- 172.21.15.16:0/687092039 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f42e80cf200 0x7f42e80adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.541+0000 7f42ecb86640 1 -- 172.21.15.16:0/687092039 >> 172.21.15.16:0/687092039 conn(0x7f42e806bdf0 msgr2=0x7f42e806dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:50.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.541+0000 7f42ecb86640 1 -- 172.21.15.16:0/687092039 shutdown_connections 2024-09-19T07:55:50.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.541+0000 7f42ecb86640 1 -- 172.21.15.16:0/687092039 wait complete. 2024-09-19T07:55:50.553 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":534,"stamp":"2024-09-19T07:55:48.640068+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592002,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":182,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":88,"acting":88,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":766080,"kb_used_data":15760,"kb_used_omap":12,"kb_used_meta":750131,"kb_avail":754208640,"statfs":{"total":773094113280,"available":772309647360,"internally_reserved":0,"allocated":16138240,"data_stored":8107522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":768134704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1634,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000880"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800604+0000","last_change":"2024-09-19T07:55:46.819762+0000","last_active":"2024-09-19T07:55:47.800604+0000","last_peered":"2024-09-19T07:55:47.800604+0000","last_clean":"2024-09-19T07:55:47.800604+0000","last_became_active":"2024-09-19T07:55:46.819203+0000","last_became_peered":"2024-09-19T07:55:46.819203+0000","last_unstale":"2024-09-19T07:55:47.800604+0000","last_undegraded":"2024-09-19T07:55:47.800604+0000","last_fullsized":"2024-09-19T07:55:47.800604+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:59.757492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800582+0000","last_change":"2024-09-19T07:55:46.804795+0000","last_active":"2024-09-19T07:55:47.800582+0000","last_peered":"2024-09-19T07:55:47.800582+0000","last_clean":"2024-09-19T07:55:47.800582+0000","last_became_active":"2024-09-19T07:55:46.804489+0000","last_became_peered":"2024-09-19T07:55:46.804489+0000","last_unstale":"2024-09-19T07:55:47.800582+0000","last_undegraded":"2024-09-19T07:55:47.800582+0000","last_fullsized":"2024-09-19T07:55:47.800582+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:18.647644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.801046+0000","last_change":"2024-09-19T07:55:46.810344+0000","last_active":"2024-09-19T07:55:47.801046+0000","last_peered":"2024-09-19T07:55:47.801046+0000","last_clean":"2024-09-19T07:55:47.801046+0000","last_became_active":"2024-09-19T07:55:46.810035+0000","last_became_peered":"2024-09-19T07:55:46.810035+0000","last_unstale":"2024-09-19T07:55:47.801046+0000","last_undegraded":"2024-09-19T07:55:47.801046+0000","last_fullsized":"2024-09-19T07:55:47.801046+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:29.247081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800997+0000","last_change":"2024-09-19T07:55:46.817698+0000","last_active":"2024-09-19T07:55:47.800997+0000","last_peered":"2024-09-19T07:55:47.800997+0000","last_clean":"2024-09-19T07:55:47.800997+0000","last_became_active":"2024-09-19T07:55:46.817376+0000","last_became_peered":"2024-09-19T07:55:46.817376+0000","last_unstale":"2024-09-19T07:55:47.800997+0000","last_undegraded":"2024-09-19T07:55:47.800997+0000","last_fullsized":"2024-09-19T07:55:47.800997+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:07.805087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827703+0000","last_change":"2024-09-19T07:55:46.811483+0000","last_active":"2024-09-19T07:55:47.827703+0000","last_peered":"2024-09-19T07:55:47.827703+0000","last_clean":"2024-09-19T07:55:47.827703+0000","last_became_active":"2024-09-19T07:55:46.811206+0000","last_became_peered":"2024-09-19T07:55:46.811206+0000","last_unstale":"2024-09-19T07:55:47.827703+0000","last_undegraded":"2024-09-19T07:55:47.827703+0000","last_fullsized":"2024-09-19T07:55:47.827703+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:47.785611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.801098+0000","last_change":"2024-09-19T07:55:46.806494+0000","last_active":"2024-09-19T07:55:47.801098+0000","last_peered":"2024-09-19T07:55:47.801098+0000","last_clean":"2024-09-19T07:55:47.801098+0000","last_became_active":"2024-09-19T07:55:46.806196+0000","last_became_peered":"2024-09-19T07:55:46.806196+0000","last_unstale":"2024-09-19T07:55:47.801098+0000","last_undegraded":"2024-09-19T07:55:47.801098+0000","last_fullsized":"2024-09-19T07:55:47.801098+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:23.021974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.791579+0000","last_change":"2024-09-19T07:55:45.791579+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.791579+0000","last_undegraded":"2024-09-19T07:55:45.791579+0000","last_fullsized":"2024-09-19T07:55:45.791579+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827721+0000","last_change":"2024-09-19T07:55:46.817878+0000","last_active":"2024-09-19T07:55:47.827721+0000","last_peered":"2024-09-19T07:55:47.827721+0000","last_clean":"2024-09-19T07:55:47.827721+0000","last_became_active":"2024-09-19T07:55:46.817518+0000","last_became_peered":"2024-09-19T07:55:46.817518+0000","last_unstale":"2024-09-19T07:55:47.827721+0000","last_undegraded":"2024-09-19T07:55:47.827721+0000","last_fullsized":"2024-09-19T07:55:47.827721+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:07.791830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827831+0000","last_change":"2024-09-19T07:55:46.819719+0000","last_active":"2024-09-19T07:55:47.827831+0000","last_peered":"2024-09-19T07:55:47.827831+0000","last_clean":"2024-09-19T07:55:47.827831+0000","last_became_active":"2024-09-19T07:55:46.811669+0000","last_became_peered":"2024-09-19T07:55:46.811669+0000","last_unstale":"2024-09-19T07:55:47.827831+0000","last_undegraded":"2024-09-19T07:55:47.827831+0000","last_fullsized":"2024-09-19T07:55:47.827831+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:40.791876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.790719+0000","last_change":"2024-09-19T07:55:45.790719+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.790719+0000","last_undegraded":"2024-09-19T07:55:45.790719+0000","last_fullsized":"2024-09-19T07:55:45.790719+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827493+0000","last_change":"2024-09-19T07:55:46.818518+0000","last_active":"2024-09-19T07:55:47.827493+0000","last_peered":"2024-09-19T07:55:47.827493+0000","last_clean":"2024-09-19T07:55:47.827493+0000","last_became_active":"2024-09-19T07:55:46.818264+0000","last_became_peered":"2024-09-19T07:55:46.818264+0000","last_unstale":"2024-09-19T07:55:47.827493+0000","last_undegraded":"2024-09-19T07:55:47.827493+0000","last_fullsized":"2024-09-19T07:55:47.827493+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:53.813959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827377+0000","last_change":"2024-09-19T07:55:46.813694+0000","last_active":"2024-09-19T07:55:47.827377+0000","last_peered":"2024-09-19T07:55:47.827377+0000","last_clean":"2024-09-19T07:55:47.827377+0000","last_became_active":"2024-09-19T07:55:46.813395+0000","last_became_peered":"2024-09-19T07:55:46.813395+0000","last_unstale":"2024-09-19T07:55:47.827377+0000","last_undegraded":"2024-09-19T07:55:47.827377+0000","last_fullsized":"2024-09-19T07:55:47.827377+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:46:54.731529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-19T07:55:45.801899+0000","last_change":"2024-09-19T07:55:45.800564+0000","last_active":"2024-09-19T07:55:45.801899+0000","last_peered":"2024-09-19T07:55:45.801899+0000","last_clean":"2024-09-19T07:55:45.801899+0000","last_became_active":"2024-09-19T07:55:45.800078+0000","last_became_peered":"2024-09-19T07:55:45.800078+0000","last_unstale":"2024-09-19T07:55:45.801899+0000","last_undegraded":"2024-09-19T07:55:45.801899+0000","last_fullsized":"2024-09-19T07:55:45.801899+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:18:15.498014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800845+0000","last_change":"2024-09-19T07:55:46.818421+0000","last_active":"2024-09-19T07:55:47.800845+0000","last_peered":"2024-09-19T07:55:47.800845+0000","last_clean":"2024-09-19T07:55:47.800845+0000","last_became_active":"2024-09-19T07:55:46.818140+0000","last_became_peered":"2024-09-19T07:55:46.818140+0000","last_unstale":"2024-09-19T07:55:47.800845+0000","last_undegraded":"2024-09-19T07:55:47.800845+0000","last_fullsized":"2024-09-19T07:55:47.800845+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:20.053063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"151'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800833+0000","last_change":"2024-09-19T07:55:45.805798+0000","last_active":"2024-09-19T07:55:47.800833+0000","last_peered":"2024-09-19T07:55:47.800833+0000","last_clean":"2024-09-19T07:55:47.800833+0000","last_became_active":"2024-09-19T07:55:45.805511+0000","last_became_peered":"2024-09-19T07:55:45.805511+0000","last_unstale":"2024-09-19T07:55:47.800833+0000","last_undegraded":"2024-09-19T07:55:47.800833+0000","last_fullsized":"2024-09-19T07:55:47.800833+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:18.474678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.790402+0000","last_change":"2024-09-19T07:55:45.790402+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.790402+0000","last_undegraded":"2024-09-19T07:55:45.790402+0000","last_fullsized":"2024-09-19T07:55:45.790402+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.790244+0000","last_change":"2024-09-19T07:55:45.790244+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.790244+0000","last_undegraded":"2024-09-19T07:55:45.790244+0000","last_fullsized":"2024-09-19T07:55:45.790244+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-19T07:55:45.805997+0000","last_change":"2024-09-19T07:55:45.804265+0000","last_active":"2024-09-19T07:55:45.805997+0000","last_peered":"2024-09-19T07:55:45.805997+0000","last_clean":"2024-09-19T07:55:45.805997+0000","last_became_active":"2024-09-19T07:55:45.804000+0000","last_became_peered":"2024-09-19T07:55:45.804000+0000","last_unstale":"2024-09-19T07:55:45.805997+0000","last_undegraded":"2024-09-19T07:55:45.805997+0000","last_fullsized":"2024-09-19T07:55:45.805997+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:00.654293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800706+0000","last_change":"2024-09-19T07:55:45.797824+0000","last_active":"2024-09-19T07:55:47.800706+0000","last_peered":"2024-09-19T07:55:47.800706+0000","last_clean":"2024-09-19T07:55:47.800706+0000","last_became_active":"2024-09-19T07:55:45.797559+0000","last_became_peered":"2024-09-19T07:55:45.797559+0000","last_unstale":"2024-09-19T07:55:47.800706+0000","last_undegraded":"2024-09-19T07:55:47.800706+0000","last_fullsized":"2024-09-19T07:55:47.800706+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:48.369553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827717+0000","last_change":"2024-09-19T07:55:45.800048+0000","last_active":"2024-09-19T07:55:47.827717+0000","last_peered":"2024-09-19T07:55:47.827717+0000","last_clean":"2024-09-19T07:55:47.827717+0000","last_became_active":"2024-09-19T07:55:45.799749+0000","last_became_peered":"2024-09-19T07:55:45.799749+0000","last_unstale":"2024-09-19T07:55:47.827717+0000","last_undegraded":"2024-09-19T07:55:47.827717+0000","last_fullsized":"2024-09-19T07:55:47.827717+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:42.846081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827407+0000","last_change":"2024-09-19T07:55:46.819081+0000","last_active":"2024-09-19T07:55:47.827407+0000","last_peered":"2024-09-19T07:55:47.827407+0000","last_clean":"2024-09-19T07:55:47.827407+0000","last_became_active":"2024-09-19T07:55:46.818785+0000","last_became_peered":"2024-09-19T07:55:46.818785+0000","last_unstale":"2024-09-19T07:55:47.827407+0000","last_undegraded":"2024-09-19T07:55:47.827407+0000","last_fullsized":"2024-09-19T07:55:47.827407+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:48.362583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.791181+0000","last_change":"2024-09-19T07:55:45.791181+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.791181+0000","last_undegraded":"2024-09-19T07:55:45.791181+0000","last_fullsized":"2024-09-19T07:55:45.791181+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"151'6","reported_seq":20,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827602+0000","last_change":"2024-09-19T07:55:45.802798+0000","last_active":"2024-09-19T07:55:47.827602+0000","last_peered":"2024-09-19T07:55:47.827602+0000","last_clean":"2024-09-19T07:55:47.827602+0000","last_became_active":"2024-09-19T07:55:45.802458+0000","last_became_peered":"2024-09-19T07:55:45.802458+0000","last_unstale":"2024-09-19T07:55:47.827602+0000","last_undegraded":"2024-09-19T07:55:47.827602+0000","last_fullsized":"2024-09-19T07:55:47.827602+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:08.281321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800893+0000","last_change":"2024-09-19T07:55:46.817083+0000","last_active":"2024-09-19T07:55:47.800893+0000","last_peered":"2024-09-19T07:55:47.800893+0000","last_clean":"2024-09-19T07:55:47.800893+0000","last_became_active":"2024-09-19T07:55:46.816804+0000","last_became_peered":"2024-09-19T07:55:46.816804+0000","last_unstale":"2024-09-19T07:55:47.800893+0000","last_undegraded":"2024-09-19T07:55:47.800893+0000","last_fullsized":"2024-09-19T07:55:47.800893+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:55.546951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827440+0000","last_change":"2024-09-19T07:55:45.801135+0000","last_active":"2024-09-19T07:55:47.827440+0000","last_peered":"2024-09-19T07:55:47.827440+0000","last_clean":"2024-09-19T07:55:47.827440+0000","last_became_active":"2024-09-19T07:55:45.800837+0000","last_became_peered":"2024-09-19T07:55:45.800837+0000","last_unstale":"2024-09-19T07:55:47.827440+0000","last_undegraded":"2024-09-19T07:55:47.827440+0000","last_fullsized":"2024-09-19T07:55:47.827440+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:30:46.437440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.791295+0000","last_change":"2024-09-19T07:55:45.791295+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.791295+0000","last_undegraded":"2024-09-19T07:55:45.791295+0000","last_fullsized":"2024-09-19T07:55:45.791295+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"151'8","reported_seq":22,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800808+0000","last_change":"2024-09-19T07:55:45.795277+0000","last_active":"2024-09-19T07:55:47.800808+0000","last_peered":"2024-09-19T07:55:47.800808+0000","last_clean":"2024-09-19T07:55:47.800808+0000","last_became_active":"2024-09-19T07:55:45.794876+0000","last_became_peered":"2024-09-19T07:55:45.794876+0000","last_unstale":"2024-09-19T07:55:47.800808+0000","last_undegraded":"2024-09-19T07:55:47.800808+0000","last_fullsized":"2024-09-19T07:55:47.800808+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:50.194217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800585+0000","last_change":"2024-09-19T07:55:46.817868+0000","last_active":"2024-09-19T07:55:47.800585+0000","last_peered":"2024-09-19T07:55:47.800585+0000","last_clean":"2024-09-19T07:55:47.800585+0000","last_became_active":"2024-09-19T07:55:46.817545+0000","last_became_peered":"2024-09-19T07:55:46.817545+0000","last_unstale":"2024-09-19T07:55:47.800585+0000","last_undegraded":"2024-09-19T07:55:47.800585+0000","last_fullsized":"2024-09-19T07:55:47.800585+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:37:45.202649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"151'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827791+0000","last_change":"2024-09-19T07:55:45.797085+0000","last_active":"2024-09-19T07:55:47.827791+0000","last_peered":"2024-09-19T07:55:47.827791+0000","last_clean":"2024-09-19T07:55:47.827791+0000","last_became_active":"2024-09-19T07:55:45.796793+0000","last_became_peered":"2024-09-19T07:55:45.796793+0000","last_unstale":"2024-09-19T07:55:47.827791+0000","last_undegraded":"2024-09-19T07:55:47.827791+0000","last_fullsized":"2024-09-19T07:55:47.827791+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:14.418996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.e","version":"151'2","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.801123+0000","last_change":"2024-09-19T07:55:45.803736+0000","last_active":"2024-09-19T07:55:47.801123+0000","last_peered":"2024-09-19T07:55:47.801123+0000","last_clean":"2024-09-19T07:55:47.801123+0000","last_became_active":"2024-09-19T07:55:45.803419+0000","last_became_peered":"2024-09-19T07:55:45.803419+0000","last_unstale":"2024-09-19T07:55:47.801123+0000","last_undegraded":"2024-09-19T07:55:47.801123+0000","last_fullsized":"2024-09-19T07:55:47.801123+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:39:47.947756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800962+0000","last_change":"2024-09-19T07:55:45.805741+0000","last_active":"2024-09-19T07:55:47.800962+0000","last_peered":"2024-09-19T07:55:47.800962+0000","last_clean":"2024-09-19T07:55:47.800962+0000","last_became_active":"2024-09-19T07:55:45.805464+0000","last_became_peered":"2024-09-19T07:55:45.805464+0000","last_unstale":"2024-09-19T07:55:47.800962+0000","last_undegraded":"2024-09-19T07:55:47.800962+0000","last_fullsized":"2024-09-19T07:55:47.800962+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:31.558613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827844+0000","last_change":"2024-09-19T07:55:46.817735+0000","last_active":"2024-09-19T07:55:47.827844+0000","last_peered":"2024-09-19T07:55:47.827844+0000","last_clean":"2024-09-19T07:55:47.827844+0000","last_became_active":"2024-09-19T07:55:46.817466+0000","last_became_peered":"2024-09-19T07:55:46.817466+0000","last_unstale":"2024-09-19T07:55:47.827844+0000","last_undegraded":"2024-09-19T07:55:47.827844+0000","last_fullsized":"2024-09-19T07:55:47.827844+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:52.010816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-19T07:55:45.798537+0000","last_change":"2024-09-19T07:55:45.797312+0000","last_active":"2024-09-19T07:55:45.798537+0000","last_peered":"2024-09-19T07:55:45.798537+0000","last_clean":"2024-09-19T07:55:45.798537+0000","last_became_active":"2024-09-19T07:55:45.796832+0000","last_became_peered":"2024-09-19T07:55:45.796832+0000","last_unstale":"2024-09-19T07:55:45.798537+0000","last_undegraded":"2024-09-19T07:55:45.798537+0000","last_fullsized":"2024-09-19T07:55:45.798537+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:15.635183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800693+0000","last_change":"2024-09-19T07:55:46.819496+0000","last_active":"2024-09-19T07:55:47.800693+0000","last_peered":"2024-09-19T07:55:47.800693+0000","last_clean":"2024-09-19T07:55:47.800693+0000","last_became_active":"2024-09-19T07:55:46.818787+0000","last_became_peered":"2024-09-19T07:55:46.818787+0000","last_unstale":"2024-09-19T07:55:47.800693+0000","last_undegraded":"2024-09-19T07:55:47.800693+0000","last_fullsized":"2024-09-19T07:55:47.800693+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:58.971945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"152'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.828794+0000","last_change":"2024-09-19T07:55:45.800698+0000","last_active":"2024-09-19T07:55:47.828794+0000","last_peered":"2024-09-19T07:55:47.828794+0000","last_clean":"2024-09-19T07:55:47.828794+0000","last_became_active":"2024-09-19T07:55:45.800012+0000","last_became_peered":"2024-09-19T07:55:45.800012+0000","last_unstale":"2024-09-19T07:55:47.828794+0000","last_undegraded":"2024-09-19T07:55:47.828794+0000","last_fullsized":"2024-09-19T07:55:47.828794+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:21.500902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.794831+0000","last_change":"2024-09-19T07:55:45.794831+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.794831+0000","last_undegraded":"2024-09-19T07:55:45.794831+0000","last_fullsized":"2024-09-19T07:55:45.794831+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827580+0000","last_change":"2024-09-19T07:55:45.802021+0000","last_active":"2024-09-19T07:55:47.827580+0000","last_peered":"2024-09-19T07:55:47.827580+0000","last_clean":"2024-09-19T07:55:47.827580+0000","last_became_active":"2024-09-19T07:55:45.801518+0000","last_became_peered":"2024-09-19T07:55:45.801518+0000","last_unstale":"2024-09-19T07:55:47.827580+0000","last_undegraded":"2024-09-19T07:55:47.827580+0000","last_fullsized":"2024-09-19T07:55:47.827580+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:59.000128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800755+0000","last_change":"2024-09-19T07:55:46.812665+0000","last_active":"2024-09-19T07:55:47.800755+0000","last_peered":"2024-09-19T07:55:47.800755+0000","last_clean":"2024-09-19T07:55:47.800755+0000","last_became_active":"2024-09-19T07:55:46.812424+0000","last_became_peered":"2024-09-19T07:55:46.812424+0000","last_unstale":"2024-09-19T07:55:47.800755+0000","last_undegraded":"2024-09-19T07:55:47.800755+0000","last_fullsized":"2024-09-19T07:55:47.800755+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:50.004806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-19T07:55:45.808438+0000","last_change":"2024-09-19T07:55:45.804911+0000","last_active":"2024-09-19T07:55:45.808438+0000","last_peered":"2024-09-19T07:55:45.808438+0000","last_clean":"2024-09-19T07:55:45.808438+0000","last_became_active":"2024-09-19T07:55:45.804578+0000","last_became_peered":"2024-09-19T07:55:45.804578+0000","last_unstale":"2024-09-19T07:55:45.808438+0000","last_undegraded":"2024-09-19T07:55:45.808438+0000","last_fullsized":"2024-09-19T07:55:45.808438+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:04.552288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827507+0000","last_change":"2024-09-19T07:55:45.801889+0000","last_active":"2024-09-19T07:55:47.827507+0000","last_peered":"2024-09-19T07:55:47.827507+0000","last_clean":"2024-09-19T07:55:47.827507+0000","last_became_active":"2024-09-19T07:55:45.801569+0000","last_became_peered":"2024-09-19T07:55:45.801569+0000","last_unstale":"2024-09-19T07:55:47.827507+0000","last_undegraded":"2024-09-19T07:55:47.827507+0000","last_fullsized":"2024-09-19T07:55:47.827507+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:19.834999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827678+0000","last_change":"2024-09-19T07:55:46.816559+0000","last_active":"2024-09-19T07:55:47.827678+0000","last_peered":"2024-09-19T07:55:47.827678+0000","last_clean":"2024-09-19T07:55:47.827678+0000","last_became_active":"2024-09-19T07:55:46.816305+0000","last_became_peered":"2024-09-19T07:55:46.816305+0000","last_unstale":"2024-09-19T07:55:47.827678+0000","last_undegraded":"2024-09-19T07:55:47.827678+0000","last_fullsized":"2024-09-19T07:55:47.827678+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:27.599531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.790864+0000","last_change":"2024-09-19T07:55:45.790864+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.790864+0000","last_undegraded":"2024-09-19T07:55:45.790864+0000","last_fullsized":"2024-09-19T07:55:45.790864+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:44.778194+0000","last_change":"2024-09-19T07:55:44.778194+0000","last_active":"2024-09-19T07:55:44.778194+0000","last_peered":"2024-09-19T07:55:44.778194+0000","last_clean":"2024-09-19T07:55:44.778194+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:44.778194+0000","last_undegraded":"2024-09-19T07:55:44.778194+0000","last_fullsized":"2024-09-19T07:55:44.778194+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T07:55:45.779621+0000","last_change":"2024-09-19T07:55:45.779621+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.779621+0000","last_undegraded":"2024-09-19T07:55:45.779621+0000","last_fullsized":"2024-09-19T07:55:45.779621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827644+0000","last_change":"2024-09-19T07:55:45.800943+0000","last_active":"2024-09-19T07:55:47.827644+0000","last_peered":"2024-09-19T07:55:47.827644+0000","last_clean":"2024-09-19T07:55:47.827644+0000","last_became_active":"2024-09-19T07:55:45.800428+0000","last_became_peered":"2024-09-19T07:55:45.800428+0000","last_unstale":"2024-09-19T07:55:47.827644+0000","last_undegraded":"2024-09-19T07:55:47.827644+0000","last_fullsized":"2024-09-19T07:55:47.827644+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:40.722531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":400,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T07:55:44.084202+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:55:44.084202+0000","last_peered":"2024-09-19T07:55:44.084202+0000","last_clean":"2024-09-19T07:55:44.084202+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:55:44.084202+0000","last_undegraded":"2024-09-19T07:55:44.084202+0000","last_fullsized":"2024-09-19T07:55:44.084202+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":52,"acting":52,"num_store_stats":3},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":1634,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":34,"acting":34,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509597,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83476,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288364,"statfs":{"total":96636764160,"available":96551284736,"internally_reserved":0,"allocated":2424832,"data_stored":1439155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":83032522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":105180,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266660,"statfs":{"total":96636764160,"available":96529059840,"internally_reserved":0,"allocated":1843200,"data_stored":854322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509597,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73192,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298648,"statfs":{"total":96636764160,"available":96561815552,"internally_reserved":0,"allocated":1855488,"data_stored":859841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509597,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":87084,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284756,"statfs":{"total":96636764160,"available":96547590144,"internally_reserved":0,"allocated":1855488,"data_stored":859841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509596,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105696,"kb_used_data":2380,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266144,"statfs":{"total":96636764160,"available":96528531456,"internally_reserved":0,"allocated":2437120,"data_stored":1444690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542301,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119088,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252752,"statfs":{"total":96636764160,"available":96514818048,"internally_reserved":0,"allocated":1925120,"data_stored":888784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542301,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83052,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288788,"statfs":{"total":96636764160,"available":96551718912,"internally_reserved":0,"allocated":1916928,"data_stored":888180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575006,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109312,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262528,"statfs":{"total":96636764160,"available":96524828672,"internally_reserved":0,"allocated":1880064,"data_stored":872709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:55:50.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:55:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.695+0000 7f7186552640 1 Processor -- start 2024-09-19T07:55:50.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.695+0000 7f7186552640 1 -- start start 2024-09-19T07:55:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.696+0000 7f7186552640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f718006f9e0 0x7f71800d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.696+0000 7f7186552640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71800d09f0 0x7f71800c5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.696+0000 7f7186552640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f718015b500 con 0x7f718015b120 2024-09-19T07:55:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.696+0000 7f7186552640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f71800d0e40 con 0x7f718006f9e0 2024-09-19T07:55:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.696+0000 7f7186552640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f718006e940 con 0x7f71800d09f0 2024-09-19T07:55:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.696+0000 7f717f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f718006f9e0 0x7f71800d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.696+0000 7f717ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f718015b120 0x7f718006dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60738/0 (socket says 172.21.15.16:60738) 2024-09-19T07:55:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.696+0000 7f717f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f718006f9e0 0x7f71800d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49836/0 (socket says 172.21.15.16:49836) 2024-09-19T07:55:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.696+0000 7f717ffff640 1 -- 172.21.15.16:0/3907231573 learned_addr learned my addr 172.21.15.16:0/3907231573 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.696+0000 7f717f7fe640 1 -- 172.21.15.16:0/3907231573 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71800d09f0 msgr2=0x7f71800c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.696+0000 7f7184aef640 1 --2- 172.21.15.16:0/3907231573 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71800d09f0 0x7f71800c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.696+0000 7f717f7fe640 1 --2- 172.21.15.16:0/3907231573 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71800d09f0 0x7f71800c5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.696+0000 7f717f7fe640 1 -- 172.21.15.16:0/3907231573 >> v1:172.21.15.16:6789/0 conn(0x7f718015b120 legacy=0x7f718006dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.696+0000 7f717f7fe640 1 -- 172.21.15.16:0/3907231573 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718006ec20 con 0x7f718006f9e0 2024-09-19T07:55:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.696+0000 7f717effd640 1 -- 172.21.15.16:0/3907231573 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2094515878 0 0) 0x7f718015b500 con 0x7f718015b120 2024-09-19T07:55:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.697+0000 7f717f7fe640 1 --2- 172.21.15.16:0/3907231573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f718006f9e0 0x7f71800d04b0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f71740097a0 tx=0x7f717402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492ecbe919e07e04 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.697+0000 7f717effd640 1 -- 172.21.15.16:0/3907231573 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174037070 con 0x7f718006f9e0 2024-09-19T07:55:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.697+0000 7f717effd640 1 -- 172.21.15.16:0/3907231573 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7174032440 con 0x7f718006f9e0 2024-09-19T07:55:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.697+0000 7f717effd640 1 -- 172.21.15.16:0/3907231573 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174032810 con 0x7f718006f9e0 2024-09-19T07:55:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.698+0000 7f7186552640 1 -- 172.21.15.16:0/3907231573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f718006f9e0 msgr2=0x7f71800d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.698+0000 7f7186552640 1 --2- 172.21.15.16:0/3907231573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f718006f9e0 0x7f71800d04b0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f71740097a0 tx=0x7f717402a180 comp rx=0 tx=0).stop 2024-09-19T07:55:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.698+0000 7f7186552640 1 -- 172.21.15.16:0/3907231573 shutdown_connections 2024-09-19T07:55:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.698+0000 7f7186552640 1 --2- 172.21.15.16:0/3907231573 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71800d09f0 0x7f71800c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.698+0000 7f7186552640 1 --2- 172.21.15.16:0/3907231573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f718006f9e0 0x7f71800d04b0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.698+0000 7f7186552640 1 -- 172.21.15.16:0/3907231573 >> 172.21.15.16:0/3907231573 conn(0x7f718006bdf0 msgr2=0x7f718006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.698+0000 7f7186552640 1 -- 172.21.15.16:0/3907231573 shutdown_connections 2024-09-19T07:55:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.698+0000 7f7186552640 1 -- 172.21.15.16:0/3907231573 wait complete. 2024-09-19T07:55:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.699+0000 7f7186552640 1 Processor -- start 2024-09-19T07:55:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.699+0000 7f7186552640 1 -- start start 2024-09-19T07:55:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.700+0000 7f7186552640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f718006f9e0 0x7f7180151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.700+0000 7f7186552640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71800d09f0 0x7f71800b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.700+0000 7f7186552640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f718015b120 0x7f71800b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.700+0000 7f7186552640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f718006f4c0 con 0x7f718015b120 2024-09-19T07:55:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.700+0000 7f7186552640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f718006f1e0 con 0x7f718006f9e0 2024-09-19T07:55:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.700+0000 7f7186552640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f718006e5d0 con 0x7f71800d09f0 2024-09-19T07:55:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.700+0000 7f7184aef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f718015b120 0x7f71800b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.700+0000 7f7184aef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f718015b120 0x7f71800b36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49842/0 (socket says 172.21.15.16:49842) 2024-09-19T07:55:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.700+0000 7f7184aef640 1 -- 172.21.15.16:0/1749504449 learned_addr learned my addr 172.21.15.16:0/1749504449 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.700+0000 7f717f7fe640 1 --2- 172.21.15.16:0/1749504449 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71800d09f0 0x7f71800b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.700+0000 7f717ffff640 1 --2- 172.21.15.16:0/1749504449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f718006f9e0 0x7f7180151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.700+0000 7f7184aef640 1 -- 172.21.15.16:0/1749504449 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71800d09f0 msgr2=0x7f71800b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.700+0000 7f7184aef640 1 --2- 172.21.15.16:0/1749504449 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71800d09f0 0x7f71800b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.700+0000 7f7184aef640 1 -- 172.21.15.16:0/1749504449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f718006f9e0 msgr2=0x7f7180151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.700+0000 7f7184aef640 1 --2- 172.21.15.16:0/1749504449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f718006f9e0 0x7f7180151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.700+0000 7f7184aef640 1 -- 172.21.15.16:0/1749504449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7180156520 con 0x7f718015b120 2024-09-19T07:55:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.700+0000 7f7184aef640 1 --2- 172.21.15.16:0/1749504449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f718015b120 0x7f71800b36c0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f717800cda0 tx=0x7f717800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.701+0000 7f717cff9640 1 -- 172.21.15.16:0/1749504449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7178003b90 con 0x7f718015b120 2024-09-19T07:55:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.701+0000 7f717cff9640 1 -- 172.21.15.16:0/1749504449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7178003d30 con 0x7f718015b120 2024-09-19T07:55:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.701+0000 7f7186552640 1 -- 172.21.15.16:0/1749504449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7180156800 con 0x7f718015b120 2024-09-19T07:55:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.701+0000 7f717cff9640 1 -- 172.21.15.16:0/1749504449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7178004240 con 0x7f718015b120 2024-09-19T07:55:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.701+0000 7f7186552640 1 -- 172.21.15.16:0/1749504449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7180156dc0 con 0x7f718015b120 2024-09-19T07:55:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.702+0000 7f7186552640 1 -- 172.21.15.16:0/1749504449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71800ce060 con 0x7f718015b120 2024-09-19T07:55:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.702+0000 7f717cff9640 1 -- 172.21.15.16:0/1749504449 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f71780043e0 con 0x7f718015b120 2024-09-19T07:55:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.703+0000 7f717cff9640 1 --2- 172.21.15.16:0/1749504449 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f716c06c2b0 0x7f716c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.703+0000 7f717cff9640 1 -- 172.21.15.16:0/1749504449 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f717808bf10 con 0x7f718015b120 2024-09-19T07:55:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.703+0000 7f717ffff640 1 --2- 172.21.15.16:0/1749504449 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f716c06c2b0 0x7f716c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.704+0000 7f717ffff640 1 --2- 172.21.15.16:0/1749504449 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f716c06c2b0 0x7f716c06e770 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7170007fd0 tx=0x7f7170007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.707+0000 7f717cff9640 1 -- 172.21.15.16:0/1749504449 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7178059f30 con 0x7f718015b120 2024-09-19T07:55:50.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.879+0000 7f7186552640 1 -- 172.21.15.16:0/1749504449 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f71800c6650 con 0x7f716c06c2b0 2024-09-19T07:55:50.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.886+0000 7f717cff9640 1 -- 172.21.15.16:0/1749504449 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251829 (secure 0 0 0) 0x7f71800c6650 con 0x7f716c06c2b0 2024-09-19T07:55:50.889 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:50.890 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:55:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.889+0000 7f7186552640 1 -- 172.21.15.16:0/1749504449 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f716c06c2b0 msgr2=0x7f716c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.889+0000 7f7186552640 1 --2- 172.21.15.16:0/1749504449 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f716c06c2b0 0x7f716c06e770 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7170007fd0 tx=0x7f7170007de0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.889+0000 7f7186552640 1 -- 172.21.15.16:0/1749504449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f718015b120 msgr2=0x7f71800b36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.889+0000 7f7186552640 1 --2- 172.21.15.16:0/1749504449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f718015b120 0x7f71800b36c0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f717800cda0 tx=0x7f717800a660 comp rx=0 tx=0).stop 2024-09-19T07:55:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.889+0000 7f7186552640 1 -- 172.21.15.16:0/1749504449 shutdown_connections 2024-09-19T07:55:50.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.889+0000 7f7186552640 1 --2- 172.21.15.16:0/1749504449 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f716c06c2b0 0x7f716c06e770 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.889+0000 7f7186552640 1 --2- 172.21.15.16:0/1749504449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f718015b120 0x7f71800b36c0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.889+0000 7f7186552640 1 --2- 172.21.15.16:0/1749504449 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71800d09f0 0x7f71800b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.889+0000 7f7186552640 1 --2- 172.21.15.16:0/1749504449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f718006f9e0 0x7f7180151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:50.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.889+0000 7f7186552640 1 -- 172.21.15.16:0/1749504449 >> 172.21.15.16:0/1749504449 conn(0x7f718006bdf0 msgr2=0x7f718006dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:50.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.889+0000 7f7186552640 1 -- 172.21.15.16:0/1749504449 shutdown_connections 2024-09-19T07:55:50.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:50.889+0000 7f7186552640 1 -- 172.21.15.16:0/1749504449 wait complete. 2024-09-19T07:55:50.900 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":535,"stamp":"2024-09-19T07:55:50.640721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":201,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":770568,"kb_used_data":16080,"kb_used_omap":12,"kb_used_meta":754291,"kb_avail":754204152,"statfs":{"total":773094113280,"available":772305051648,"internally_reserved":0,"allocated":16465920,"data_stored":8264241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":772394544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.860165"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807561+0000","last_change":"2024-09-19T07:55:46.814625+0000","last_active":"2024-09-19T07:55:47.807561+0000","last_peered":"2024-09-19T07:55:47.807561+0000","last_clean":"2024-09-19T07:55:47.807561+0000","last_became_active":"2024-09-19T07:55:46.814365+0000","last_became_peered":"2024-09-19T07:55:46.814365+0000","last_unstale":"2024-09-19T07:55:47.807561+0000","last_undegraded":"2024-09-19T07:55:47.807561+0000","last_fullsized":"2024-09-19T07:55:47.807561+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:13.254644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590303+0000","last_change":"2024-09-19T07:55:46.811993+0000","last_active":"2024-09-19T07:55:48.590303+0000","last_peered":"2024-09-19T07:55:48.590303+0000","last_clean":"2024-09-19T07:55:48.590303+0000","last_became_active":"2024-09-19T07:55:46.811723+0000","last_became_peered":"2024-09-19T07:55:46.811723+0000","last_unstale":"2024-09-19T07:55:48.590303+0000","last_undegraded":"2024-09-19T07:55:48.590303+0000","last_fullsized":"2024-09-19T07:55:48.590303+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:37:45.202649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807311+0000","last_change":"2024-09-19T07:55:46.818859+0000","last_active":"2024-09-19T07:55:47.807311+0000","last_peered":"2024-09-19T07:55:47.807311+0000","last_clean":"2024-09-19T07:55:47.807311+0000","last_became_active":"2024-09-19T07:55:46.818573+0000","last_became_peered":"2024-09-19T07:55:46.818573+0000","last_unstale":"2024-09-19T07:55:47.807311+0000","last_undegraded":"2024-09-19T07:55:47.807311+0000","last_fullsized":"2024-09-19T07:55:47.807311+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:03.579883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806482+0000","last_change":"2024-09-19T07:55:46.813506+0000","last_active":"2024-09-19T07:55:47.806482+0000","last_peered":"2024-09-19T07:55:47.806482+0000","last_clean":"2024-09-19T07:55:47.806482+0000","last_became_active":"2024-09-19T07:55:46.813249+0000","last_became_peered":"2024-09-19T07:55:46.813249+0000","last_unstale":"2024-09-19T07:55:47.806482+0000","last_undegraded":"2024-09-19T07:55:47.806482+0000","last_fullsized":"2024-09-19T07:55:47.806482+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:06.075192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590102+0000","last_change":"2024-09-19T07:55:46.823203+0000","last_active":"2024-09-19T07:55:48.590102+0000","last_peered":"2024-09-19T07:55:48.590102+0000","last_clean":"2024-09-19T07:55:48.590102+0000","last_became_active":"2024-09-19T07:55:46.822707+0000","last_became_peered":"2024-09-19T07:55:46.822707+0000","last_unstale":"2024-09-19T07:55:48.590102+0000","last_undegraded":"2024-09-19T07:55:48.590102+0000","last_fullsized":"2024-09-19T07:55:48.590102+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:23.021974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590780+0000","last_change":"2024-09-19T07:55:46.813957+0000","last_active":"2024-09-19T07:55:48.590780+0000","last_peered":"2024-09-19T07:55:48.590780+0000","last_clean":"2024-09-19T07:55:48.590780+0000","last_became_active":"2024-09-19T07:55:46.813674+0000","last_became_peered":"2024-09-19T07:55:46.813674+0000","last_unstale":"2024-09-19T07:55:48.590780+0000","last_undegraded":"2024-09-19T07:55:48.590780+0000","last_fullsized":"2024-09-19T07:55:48.590780+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:07.805087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800604+0000","last_change":"2024-09-19T07:55:46.819762+0000","last_active":"2024-09-19T07:55:47.800604+0000","last_peered":"2024-09-19T07:55:47.800604+0000","last_clean":"2024-09-19T07:55:47.800604+0000","last_became_active":"2024-09-19T07:55:46.819203+0000","last_became_peered":"2024-09-19T07:55:46.819203+0000","last_unstale":"2024-09-19T07:55:47.800604+0000","last_undegraded":"2024-09-19T07:55:47.800604+0000","last_fullsized":"2024-09-19T07:55:47.800604+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:59.757492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800582+0000","last_change":"2024-09-19T07:55:46.804795+0000","last_active":"2024-09-19T07:55:47.800582+0000","last_peered":"2024-09-19T07:55:47.800582+0000","last_clean":"2024-09-19T07:55:47.800582+0000","last_became_active":"2024-09-19T07:55:46.804489+0000","last_became_peered":"2024-09-19T07:55:46.804489+0000","last_unstale":"2024-09-19T07:55:47.800582+0000","last_undegraded":"2024-09-19T07:55:47.800582+0000","last_fullsized":"2024-09-19T07:55:47.800582+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:18.647644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.801046+0000","last_change":"2024-09-19T07:55:46.810344+0000","last_active":"2024-09-19T07:55:47.801046+0000","last_peered":"2024-09-19T07:55:47.801046+0000","last_clean":"2024-09-19T07:55:47.801046+0000","last_became_active":"2024-09-19T07:55:46.810035+0000","last_became_peered":"2024-09-19T07:55:46.810035+0000","last_unstale":"2024-09-19T07:55:47.801046+0000","last_undegraded":"2024-09-19T07:55:47.801046+0000","last_fullsized":"2024-09-19T07:55:47.801046+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:29.247081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800997+0000","last_change":"2024-09-19T07:55:46.817698+0000","last_active":"2024-09-19T07:55:47.800997+0000","last_peered":"2024-09-19T07:55:47.800997+0000","last_clean":"2024-09-19T07:55:47.800997+0000","last_became_active":"2024-09-19T07:55:46.817376+0000","last_became_peered":"2024-09-19T07:55:46.817376+0000","last_unstale":"2024-09-19T07:55:47.800997+0000","last_undegraded":"2024-09-19T07:55:47.800997+0000","last_fullsized":"2024-09-19T07:55:47.800997+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:07.805087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806932+0000","last_change":"2024-09-19T07:55:46.819681+0000","last_active":"2024-09-19T07:55:47.806932+0000","last_peered":"2024-09-19T07:55:47.806932+0000","last_clean":"2024-09-19T07:55:47.806932+0000","last_became_active":"2024-09-19T07:55:46.819379+0000","last_became_peered":"2024-09-19T07:55:46.819379+0000","last_unstale":"2024-09-19T07:55:47.806932+0000","last_undegraded":"2024-09-19T07:55:47.806932+0000","last_fullsized":"2024-09-19T07:55:47.806932+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:37:45.202649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590306+0000","last_change":"2024-09-19T07:55:46.814981+0000","last_active":"2024-09-19T07:55:48.590306+0000","last_peered":"2024-09-19T07:55:48.590306+0000","last_clean":"2024-09-19T07:55:48.590306+0000","last_became_active":"2024-09-19T07:55:46.814690+0000","last_became_peered":"2024-09-19T07:55:46.814690+0000","last_unstale":"2024-09-19T07:55:48.590306+0000","last_undegraded":"2024-09-19T07:55:48.590306+0000","last_fullsized":"2024-09-19T07:55:48.590306+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:59.757492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827703+0000","last_change":"2024-09-19T07:55:46.811483+0000","last_active":"2024-09-19T07:55:47.827703+0000","last_peered":"2024-09-19T07:55:47.827703+0000","last_clean":"2024-09-19T07:55:47.827703+0000","last_became_active":"2024-09-19T07:55:46.811206+0000","last_became_peered":"2024-09-19T07:55:46.811206+0000","last_unstale":"2024-09-19T07:55:47.827703+0000","last_undegraded":"2024-09-19T07:55:47.827703+0000","last_fullsized":"2024-09-19T07:55:47.827703+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:47.785611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.589975+0000","last_change":"2024-09-19T07:55:46.822810+0000","last_active":"2024-09-19T07:55:48.589975+0000","last_peered":"2024-09-19T07:55:48.589975+0000","last_clean":"2024-09-19T07:55:48.589975+0000","last_became_active":"2024-09-19T07:55:46.822536+0000","last_became_peered":"2024-09-19T07:55:46.822536+0000","last_unstale":"2024-09-19T07:55:48.589975+0000","last_undegraded":"2024-09-19T07:55:48.589975+0000","last_fullsized":"2024-09-19T07:55:48.589975+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:07.805087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806082+0000","last_change":"2024-09-19T07:55:46.815322+0000","last_active":"2024-09-19T07:55:47.806082+0000","last_peered":"2024-09-19T07:55:47.806082+0000","last_clean":"2024-09-19T07:55:47.806082+0000","last_became_active":"2024-09-19T07:55:46.815056+0000","last_became_peered":"2024-09-19T07:55:46.815056+0000","last_unstale":"2024-09-19T07:55:47.806082+0000","last_undegraded":"2024-09-19T07:55:47.806082+0000","last_fullsized":"2024-09-19T07:55:47.806082+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:14.739369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806011+0000","last_change":"2024-09-19T07:55:46.813810+0000","last_active":"2024-09-19T07:55:47.806011+0000","last_peered":"2024-09-19T07:55:47.806011+0000","last_clean":"2024-09-19T07:55:47.806011+0000","last_became_active":"2024-09-19T07:55:46.812769+0000","last_became_peered":"2024-09-19T07:55:46.812769+0000","last_unstale":"2024-09-19T07:55:47.806011+0000","last_undegraded":"2024-09-19T07:55:47.806011+0000","last_fullsized":"2024-09-19T07:55:47.806011+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:05:26.103943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.801098+0000","last_change":"2024-09-19T07:55:46.806494+0000","last_active":"2024-09-19T07:55:47.801098+0000","last_peered":"2024-09-19T07:55:47.801098+0000","last_clean":"2024-09-19T07:55:47.801098+0000","last_became_active":"2024-09-19T07:55:46.806196+0000","last_became_peered":"2024-09-19T07:55:46.806196+0000","last_unstale":"2024-09-19T07:55:47.801098+0000","last_undegraded":"2024-09-19T07:55:47.801098+0000","last_fullsized":"2024-09-19T07:55:47.801098+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:23.021974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590830+0000","last_change":"2024-09-19T07:55:46.817297+0000","last_active":"2024-09-19T07:55:48.590830+0000","last_peered":"2024-09-19T07:55:48.590830+0000","last_clean":"2024-09-19T07:55:48.590830+0000","last_became_active":"2024-09-19T07:55:46.817014+0000","last_became_peered":"2024-09-19T07:55:46.817014+0000","last_unstale":"2024-09-19T07:55:48.590830+0000","last_undegraded":"2024-09-19T07:55:48.590830+0000","last_fullsized":"2024-09-19T07:55:48.590830+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:35.797895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806915+0000","last_change":"2024-09-19T07:55:46.808729+0000","last_active":"2024-09-19T07:55:47.806915+0000","last_peered":"2024-09-19T07:55:47.806915+0000","last_clean":"2024-09-19T07:55:47.806915+0000","last_became_active":"2024-09-19T07:55:46.808438+0000","last_became_peered":"2024-09-19T07:55:46.808438+0000","last_unstale":"2024-09-19T07:55:47.806915+0000","last_undegraded":"2024-09-19T07:55:47.806915+0000","last_fullsized":"2024-09-19T07:55:47.806915+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:18.647644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.791579+0000","last_change":"2024-09-19T07:55:45.791579+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.791579+0000","last_undegraded":"2024-09-19T07:55:45.791579+0000","last_fullsized":"2024-09-19T07:55:45.791579+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827721+0000","last_change":"2024-09-19T07:55:46.817878+0000","last_active":"2024-09-19T07:55:47.827721+0000","last_peered":"2024-09-19T07:55:47.827721+0000","last_clean":"2024-09-19T07:55:47.827721+0000","last_became_active":"2024-09-19T07:55:46.817518+0000","last_became_peered":"2024-09-19T07:55:46.817518+0000","last_unstale":"2024-09-19T07:55:47.827721+0000","last_undegraded":"2024-09-19T07:55:47.827721+0000","last_fullsized":"2024-09-19T07:55:47.827721+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:07.791830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590116+0000","last_change":"2024-09-19T07:55:46.818390+0000","last_active":"2024-09-19T07:55:48.590116+0000","last_peered":"2024-09-19T07:55:48.590116+0000","last_clean":"2024-09-19T07:55:48.590116+0000","last_became_active":"2024-09-19T07:55:46.817865+0000","last_became_peered":"2024-09-19T07:55:46.817865+0000","last_unstale":"2024-09-19T07:55:48.590116+0000","last_undegraded":"2024-09-19T07:55:48.590116+0000","last_fullsized":"2024-09-19T07:55:48.590116+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:02:49.871105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.589704+0000","last_change":"2024-09-19T07:55:46.820984+0000","last_active":"2024-09-19T07:55:48.589704+0000","last_peered":"2024-09-19T07:55:48.589704+0000","last_clean":"2024-09-19T07:55:48.589704+0000","last_became_active":"2024-09-19T07:55:46.816998+0000","last_became_peered":"2024-09-19T07:55:46.816998+0000","last_unstale":"2024-09-19T07:55:48.589704+0000","last_undegraded":"2024-09-19T07:55:48.589704+0000","last_fullsized":"2024-09-19T07:55:48.589704+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:55.546951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806365+0000","last_change":"2024-09-19T07:55:46.812552+0000","last_active":"2024-09-19T07:55:47.806365+0000","last_peered":"2024-09-19T07:55:47.806365+0000","last_clean":"2024-09-19T07:55:47.806365+0000","last_became_active":"2024-09-19T07:55:46.812258+0000","last_became_peered":"2024-09-19T07:55:46.812258+0000","last_unstale":"2024-09-19T07:55:47.806365+0000","last_undegraded":"2024-09-19T07:55:47.806365+0000","last_fullsized":"2024-09-19T07:55:47.806365+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:50.581777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827831+0000","last_change":"2024-09-19T07:55:46.819719+0000","last_active":"2024-09-19T07:55:47.827831+0000","last_peered":"2024-09-19T07:55:47.827831+0000","last_clean":"2024-09-19T07:55:47.827831+0000","last_became_active":"2024-09-19T07:55:46.811669+0000","last_became_peered":"2024-09-19T07:55:46.811669+0000","last_unstale":"2024-09-19T07:55:47.827831+0000","last_undegraded":"2024-09-19T07:55:47.827831+0000","last_fullsized":"2024-09-19T07:55:47.827831+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:40.791876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.790719+0000","last_change":"2024-09-19T07:55:45.790719+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.790719+0000","last_undegraded":"2024-09-19T07:55:45.790719+0000","last_fullsized":"2024-09-19T07:55:45.790719+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590414+0000","last_change":"2024-09-19T07:55:46.820554+0000","last_active":"2024-09-19T07:55:48.590414+0000","last_peered":"2024-09-19T07:55:48.590414+0000","last_clean":"2024-09-19T07:55:48.590414+0000","last_became_active":"2024-09-19T07:55:46.820274+0000","last_became_peered":"2024-09-19T07:55:46.820274+0000","last_unstale":"2024-09-19T07:55:48.590414+0000","last_undegraded":"2024-09-19T07:55:48.590414+0000","last_fullsized":"2024-09-19T07:55:48.590414+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:58.971945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827493+0000","last_change":"2024-09-19T07:55:46.818518+0000","last_active":"2024-09-19T07:55:47.827493+0000","last_peered":"2024-09-19T07:55:47.827493+0000","last_clean":"2024-09-19T07:55:47.827493+0000","last_became_active":"2024-09-19T07:55:46.818264+0000","last_became_peered":"2024-09-19T07:55:46.818264+0000","last_unstale":"2024-09-19T07:55:47.827493+0000","last_undegraded":"2024-09-19T07:55:47.827493+0000","last_fullsized":"2024-09-19T07:55:47.827493+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:53.813959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590685+0000","last_change":"2024-09-19T07:55:46.810596+0000","last_active":"2024-09-19T07:55:48.590685+0000","last_peered":"2024-09-19T07:55:48.590685+0000","last_clean":"2024-09-19T07:55:48.590685+0000","last_became_active":"2024-09-19T07:55:46.810298+0000","last_became_peered":"2024-09-19T07:55:46.810298+0000","last_unstale":"2024-09-19T07:55:48.590685+0000","last_undegraded":"2024-09-19T07:55:48.590685+0000","last_fullsized":"2024-09-19T07:55:48.590685+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:55.546951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590292+0000","last_change":"2024-09-19T07:55:46.817593+0000","last_active":"2024-09-19T07:55:48.590292+0000","last_peered":"2024-09-19T07:55:48.590292+0000","last_clean":"2024-09-19T07:55:48.590292+0000","last_became_active":"2024-09-19T07:55:46.817119+0000","last_became_peered":"2024-09-19T07:55:46.817119+0000","last_unstale":"2024-09-19T07:55:48.590292+0000","last_undegraded":"2024-09-19T07:55:48.590292+0000","last_fullsized":"2024-09-19T07:55:48.590292+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:03.536883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806250+0000","last_change":"2024-09-19T07:55:46.814871+0000","last_active":"2024-09-19T07:55:47.806250+0000","last_peered":"2024-09-19T07:55:47.806250+0000","last_clean":"2024-09-19T07:55:47.806250+0000","last_became_active":"2024-09-19T07:55:46.814602+0000","last_became_peered":"2024-09-19T07:55:46.814602+0000","last_unstale":"2024-09-19T07:55:47.806250+0000","last_undegraded":"2024-09-19T07:55:47.806250+0000","last_fullsized":"2024-09-19T07:55:47.806250+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:38.643240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590672+0000","last_change":"2024-09-19T07:55:46.818472+0000","last_active":"2024-09-19T07:55:48.590672+0000","last_peered":"2024-09-19T07:55:48.590672+0000","last_clean":"2024-09-19T07:55:48.590672+0000","last_became_active":"2024-09-19T07:55:46.818222+0000","last_became_peered":"2024-09-19T07:55:46.818222+0000","last_unstale":"2024-09-19T07:55:48.590672+0000","last_undegraded":"2024-09-19T07:55:48.590672+0000","last_fullsized":"2024-09-19T07:55:48.590672+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:33.322451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590232+0000","last_change":"2024-09-19T07:55:46.815707+0000","last_active":"2024-09-19T07:55:48.590232+0000","last_peered":"2024-09-19T07:55:48.590232+0000","last_clean":"2024-09-19T07:55:48.590232+0000","last_became_active":"2024-09-19T07:55:46.815207+0000","last_became_peered":"2024-09-19T07:55:46.815207+0000","last_unstale":"2024-09-19T07:55:48.590232+0000","last_undegraded":"2024-09-19T07:55:48.590232+0000","last_fullsized":"2024-09-19T07:55:48.590232+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:18.647644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"151'4","reported_seq":18,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807621+0000","last_change":"2024-09-19T07:55:45.799741+0000","last_active":"2024-09-19T07:55:47.807621+0000","last_peered":"2024-09-19T07:55:47.807621+0000","last_clean":"2024-09-19T07:55:47.807621+0000","last_became_active":"2024-09-19T07:55:45.799491+0000","last_became_peered":"2024-09-19T07:55:45.799491+0000","last_unstale":"2024-09-19T07:55:47.807621+0000","last_undegraded":"2024-09-19T07:55:47.807621+0000","last_fullsized":"2024-09-19T07:55:47.807621+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:34.796467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827377+0000","last_change":"2024-09-19T07:55:46.813694+0000","last_active":"2024-09-19T07:55:47.827377+0000","last_peered":"2024-09-19T07:55:47.827377+0000","last_clean":"2024-09-19T07:55:47.827377+0000","last_became_active":"2024-09-19T07:55:46.813395+0000","last_became_peered":"2024-09-19T07:55:46.813395+0000","last_unstale":"2024-09-19T07:55:47.827377+0000","last_undegraded":"2024-09-19T07:55:47.827377+0000","last_fullsized":"2024-09-19T07:55:47.827377+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:46:54.731529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"151'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590239+0000","last_change":"2024-09-19T07:55:45.805741+0000","last_active":"2024-09-19T07:55:48.590239+0000","last_peered":"2024-09-19T07:55:48.590239+0000","last_clean":"2024-09-19T07:55:48.590239+0000","last_became_active":"2024-09-19T07:55:45.805455+0000","last_became_peered":"2024-09-19T07:55:45.805455+0000","last_unstale":"2024-09-19T07:55:48.590239+0000","last_undegraded":"2024-09-19T07:55:48.590239+0000","last_fullsized":"2024-09-19T07:55:48.590239+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:37.641813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.589999+0000","last_change":"2024-09-19T07:55:46.823361+0000","last_active":"2024-09-19T07:55:48.589999+0000","last_peered":"2024-09-19T07:55:48.589999+0000","last_clean":"2024-09-19T07:55:48.589999+0000","last_became_active":"2024-09-19T07:55:46.823078+0000","last_became_peered":"2024-09-19T07:55:46.823078+0000","last_unstale":"2024-09-19T07:55:48.589999+0000","last_undegraded":"2024-09-19T07:55:48.589999+0000","last_fullsized":"2024-09-19T07:55:48.589999+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:29.247081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"151'2","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807032+0000","last_change":"2024-09-19T07:55:45.794756+0000","last_active":"2024-09-19T07:55:47.807032+0000","last_peered":"2024-09-19T07:55:47.807032+0000","last_clean":"2024-09-19T07:55:47.807032+0000","last_became_active":"2024-09-19T07:55:45.794448+0000","last_became_peered":"2024-09-19T07:55:45.794448+0000","last_unstale":"2024-09-19T07:55:47.807032+0000","last_undegraded":"2024-09-19T07:55:47.807032+0000","last_fullsized":"2024-09-19T07:55:47.807032+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:19.674806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590854+0000","last_change":"2024-09-19T07:55:46.815448+0000","last_active":"2024-09-19T07:55:48.590854+0000","last_peered":"2024-09-19T07:55:48.590854+0000","last_clean":"2024-09-19T07:55:48.590854+0000","last_became_active":"2024-09-19T07:55:46.815181+0000","last_became_peered":"2024-09-19T07:55:46.815181+0000","last_unstale":"2024-09-19T07:55:48.590854+0000","last_undegraded":"2024-09-19T07:55:48.590854+0000","last_fullsized":"2024-09-19T07:55:48.590854+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:29.247081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-19T07:55:45.801899+0000","last_change":"2024-09-19T07:55:45.800564+0000","last_active":"2024-09-19T07:55:45.801899+0000","last_peered":"2024-09-19T07:55:45.801899+0000","last_clean":"2024-09-19T07:55:45.801899+0000","last_became_active":"2024-09-19T07:55:45.800078+0000","last_became_peered":"2024-09-19T07:55:45.800078+0000","last_unstale":"2024-09-19T07:55:45.801899+0000","last_undegraded":"2024-09-19T07:55:45.801899+0000","last_fullsized":"2024-09-19T07:55:45.801899+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:18:15.498014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800845+0000","last_change":"2024-09-19T07:55:46.818421+0000","last_active":"2024-09-19T07:55:47.800845+0000","last_peered":"2024-09-19T07:55:47.800845+0000","last_clean":"2024-09-19T07:55:47.800845+0000","last_became_active":"2024-09-19T07:55:46.818140+0000","last_became_peered":"2024-09-19T07:55:46.818140+0000","last_unstale":"2024-09-19T07:55:47.800845+0000","last_undegraded":"2024-09-19T07:55:47.800845+0000","last_fullsized":"2024-09-19T07:55:47.800845+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:20.053063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"151'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800833+0000","last_change":"2024-09-19T07:55:45.805798+0000","last_active":"2024-09-19T07:55:47.800833+0000","last_peered":"2024-09-19T07:55:47.800833+0000","last_clean":"2024-09-19T07:55:47.800833+0000","last_became_active":"2024-09-19T07:55:45.805511+0000","last_became_peered":"2024-09-19T07:55:45.805511+0000","last_unstale":"2024-09-19T07:55:47.800833+0000","last_undegraded":"2024-09-19T07:55:47.800833+0000","last_fullsized":"2024-09-19T07:55:47.800833+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:18.474678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.790402+0000","last_change":"2024-09-19T07:55:45.790402+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.790402+0000","last_undegraded":"2024-09-19T07:55:45.790402+0000","last_fullsized":"2024-09-19T07:55:45.790402+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"151'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590065+0000","last_change":"2024-09-19T07:55:45.803999+0000","last_active":"2024-09-19T07:55:48.590065+0000","last_peered":"2024-09-19T07:55:48.590065+0000","last_clean":"2024-09-19T07:55:48.590065+0000","last_became_active":"2024-09-19T07:55:45.803710+0000","last_became_peered":"2024-09-19T07:55:45.803710+0000","last_unstale":"2024-09-19T07:55:48.590065+0000","last_undegraded":"2024-09-19T07:55:48.590065+0000","last_fullsized":"2024-09-19T07:55:48.590065+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:07.119888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.790244+0000","last_change":"2024-09-19T07:55:45.790244+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.790244+0000","last_undegraded":"2024-09-19T07:55:45.790244+0000","last_fullsized":"2024-09-19T07:55:45.790244+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-19T07:55:45.805997+0000","last_change":"2024-09-19T07:55:45.804265+0000","last_active":"2024-09-19T07:55:45.805997+0000","last_peered":"2024-09-19T07:55:45.805997+0000","last_clean":"2024-09-19T07:55:45.805997+0000","last_became_active":"2024-09-19T07:55:45.804000+0000","last_became_peered":"2024-09-19T07:55:45.804000+0000","last_unstale":"2024-09-19T07:55:45.805997+0000","last_undegraded":"2024-09-19T07:55:45.805997+0000","last_fullsized":"2024-09-19T07:55:45.805997+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:00.654293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807545+0000","last_change":"2024-09-19T07:55:46.804640+0000","last_active":"2024-09-19T07:55:47.807545+0000","last_peered":"2024-09-19T07:55:47.807545+0000","last_clean":"2024-09-19T07:55:47.807545+0000","last_became_active":"2024-09-19T07:55:46.804353+0000","last_became_peered":"2024-09-19T07:55:46.804353+0000","last_unstale":"2024-09-19T07:55:47.807545+0000","last_undegraded":"2024-09-19T07:55:47.807545+0000","last_fullsized":"2024-09-19T07:55:47.807545+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:54.973660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800706+0000","last_change":"2024-09-19T07:55:45.797824+0000","last_active":"2024-09-19T07:55:47.800706+0000","last_peered":"2024-09-19T07:55:47.800706+0000","last_clean":"2024-09-19T07:55:47.800706+0000","last_became_active":"2024-09-19T07:55:45.797559+0000","last_became_peered":"2024-09-19T07:55:45.797559+0000","last_unstale":"2024-09-19T07:55:47.800706+0000","last_undegraded":"2024-09-19T07:55:47.800706+0000","last_fullsized":"2024-09-19T07:55:47.800706+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:48.369553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590859+0000","last_change":"2024-09-19T07:55:46.815474+0000","last_active":"2024-09-19T07:55:48.590859+0000","last_peered":"2024-09-19T07:55:48.590859+0000","last_clean":"2024-09-19T07:55:48.590859+0000","last_became_active":"2024-09-19T07:55:46.814841+0000","last_became_peered":"2024-09-19T07:55:46.814841+0000","last_unstale":"2024-09-19T07:55:48.590859+0000","last_undegraded":"2024-09-19T07:55:48.590859+0000","last_fullsized":"2024-09-19T07:55:48.590859+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:23.021974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827717+0000","last_change":"2024-09-19T07:55:45.800048+0000","last_active":"2024-09-19T07:55:47.827717+0000","last_peered":"2024-09-19T07:55:47.827717+0000","last_clean":"2024-09-19T07:55:47.827717+0000","last_became_active":"2024-09-19T07:55:45.799749+0000","last_became_peered":"2024-09-19T07:55:45.799749+0000","last_unstale":"2024-09-19T07:55:47.827717+0000","last_undegraded":"2024-09-19T07:55:47.827717+0000","last_fullsized":"2024-09-19T07:55:47.827717+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:42.846081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827407+0000","last_change":"2024-09-19T07:55:46.819081+0000","last_active":"2024-09-19T07:55:47.827407+0000","last_peered":"2024-09-19T07:55:47.827407+0000","last_clean":"2024-09-19T07:55:47.827407+0000","last_became_active":"2024-09-19T07:55:46.818785+0000","last_became_peered":"2024-09-19T07:55:46.818785+0000","last_unstale":"2024-09-19T07:55:47.827407+0000","last_undegraded":"2024-09-19T07:55:47.827407+0000","last_fullsized":"2024-09-19T07:55:47.827407+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:48.362583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"151'3","reported_seq":17,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807316+0000","last_change":"2024-09-19T07:55:45.792189+0000","last_active":"2024-09-19T07:55:47.807316+0000","last_peered":"2024-09-19T07:55:47.807316+0000","last_clean":"2024-09-19T07:55:47.807316+0000","last_became_active":"2024-09-19T07:55:45.791906+0000","last_became_peered":"2024-09-19T07:55:45.791906+0000","last_unstale":"2024-09-19T07:55:47.807316+0000","last_undegraded":"2024-09-19T07:55:47.807316+0000","last_fullsized":"2024-09-19T07:55:47.807316+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:32.321023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.791181+0000","last_change":"2024-09-19T07:55:45.791181+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.791181+0000","last_undegraded":"2024-09-19T07:55:45.791181+0000","last_fullsized":"2024-09-19T07:55:45.791181+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"151'6","reported_seq":20,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827602+0000","last_change":"2024-09-19T07:55:45.802798+0000","last_active":"2024-09-19T07:55:47.827602+0000","last_peered":"2024-09-19T07:55:47.827602+0000","last_clean":"2024-09-19T07:55:47.827602+0000","last_became_active":"2024-09-19T07:55:45.802458+0000","last_became_peered":"2024-09-19T07:55:45.802458+0000","last_unstale":"2024-09-19T07:55:47.827602+0000","last_undegraded":"2024-09-19T07:55:47.827602+0000","last_fullsized":"2024-09-19T07:55:47.827602+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:08.281321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590621+0000","last_change":"2024-09-19T07:55:46.817885+0000","last_active":"2024-09-19T07:55:48.590621+0000","last_peered":"2024-09-19T07:55:48.590621+0000","last_clean":"2024-09-19T07:55:48.590621+0000","last_became_active":"2024-09-19T07:55:46.817566+0000","last_became_peered":"2024-09-19T07:55:46.817566+0000","last_unstale":"2024-09-19T07:55:48.590621+0000","last_undegraded":"2024-09-19T07:55:48.590621+0000","last_fullsized":"2024-09-19T07:55:48.590621+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:12.687039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"151'2","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590265+0000","last_change":"2024-09-19T07:55:45.801524+0000","last_active":"2024-09-19T07:55:48.590265+0000","last_peered":"2024-09-19T07:55:48.590265+0000","last_clean":"2024-09-19T07:55:48.590265+0000","last_became_active":"2024-09-19T07:55:45.801161+0000","last_became_peered":"2024-09-19T07:55:45.801161+0000","last_unstale":"2024-09-19T07:55:48.590265+0000","last_undegraded":"2024-09-19T07:55:48.590265+0000","last_fullsized":"2024-09-19T07:55:48.590265+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:13.737941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800893+0000","last_change":"2024-09-19T07:55:46.817083+0000","last_active":"2024-09-19T07:55:47.800893+0000","last_peered":"2024-09-19T07:55:47.800893+0000","last_clean":"2024-09-19T07:55:47.800893+0000","last_became_active":"2024-09-19T07:55:46.816804+0000","last_became_peered":"2024-09-19T07:55:46.816804+0000","last_unstale":"2024-09-19T07:55:47.800893+0000","last_undegraded":"2024-09-19T07:55:47.800893+0000","last_fullsized":"2024-09-19T07:55:47.800893+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:55.546951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827440+0000","last_change":"2024-09-19T07:55:45.801135+0000","last_active":"2024-09-19T07:55:47.827440+0000","last_peered":"2024-09-19T07:55:47.827440+0000","last_clean":"2024-09-19T07:55:47.827440+0000","last_became_active":"2024-09-19T07:55:45.800837+0000","last_became_peered":"2024-09-19T07:55:45.800837+0000","last_unstale":"2024-09-19T07:55:47.827440+0000","last_undegraded":"2024-09-19T07:55:47.827440+0000","last_fullsized":"2024-09-19T07:55:47.827440+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:30:46.437440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807292+0000","last_change":"2024-09-19T07:55:46.813235+0000","last_active":"2024-09-19T07:55:47.807292+0000","last_peered":"2024-09-19T07:55:47.807292+0000","last_clean":"2024-09-19T07:55:47.807292+0000","last_became_active":"2024-09-19T07:55:46.812961+0000","last_became_peered":"2024-09-19T07:55:46.812961+0000","last_unstale":"2024-09-19T07:55:47.807292+0000","last_undegraded":"2024-09-19T07:55:47.807292+0000","last_fullsized":"2024-09-19T07:55:47.807292+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:59.757492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806364+0000","last_change":"2024-09-19T07:55:45.804843+0000","last_active":"2024-09-19T07:55:47.806364+0000","last_peered":"2024-09-19T07:55:47.806364+0000","last_clean":"2024-09-19T07:55:47.806364+0000","last_became_active":"2024-09-19T07:55:45.804613+0000","last_became_peered":"2024-09-19T07:55:45.804613+0000","last_unstale":"2024-09-19T07:55:47.806364+0000","last_undegraded":"2024-09-19T07:55:47.806364+0000","last_fullsized":"2024-09-19T07:55:47.806364+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:18.474678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.791295+0000","last_change":"2024-09-19T07:55:45.791295+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.791295+0000","last_undegraded":"2024-09-19T07:55:45.791295+0000","last_fullsized":"2024-09-19T07:55:45.791295+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"151'8","reported_seq":22,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800808+0000","last_change":"2024-09-19T07:55:45.795277+0000","last_active":"2024-09-19T07:55:47.800808+0000","last_peered":"2024-09-19T07:55:47.800808+0000","last_clean":"2024-09-19T07:55:47.800808+0000","last_became_active":"2024-09-19T07:55:45.794876+0000","last_became_peered":"2024-09-19T07:55:45.794876+0000","last_unstale":"2024-09-19T07:55:47.800808+0000","last_undegraded":"2024-09-19T07:55:47.800808+0000","last_fullsized":"2024-09-19T07:55:47.800808+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:50.194217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800585+0000","last_change":"2024-09-19T07:55:46.817868+0000","last_active":"2024-09-19T07:55:47.800585+0000","last_peered":"2024-09-19T07:55:47.800585+0000","last_clean":"2024-09-19T07:55:47.800585+0000","last_became_active":"2024-09-19T07:55:46.817545+0000","last_became_peered":"2024-09-19T07:55:46.817545+0000","last_unstale":"2024-09-19T07:55:47.800585+0000","last_undegraded":"2024-09-19T07:55:47.800585+0000","last_fullsized":"2024-09-19T07:55:47.800585+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:37:45.202649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"151'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827791+0000","last_change":"2024-09-19T07:55:45.797085+0000","last_active":"2024-09-19T07:55:47.827791+0000","last_peered":"2024-09-19T07:55:47.827791+0000","last_clean":"2024-09-19T07:55:47.827791+0000","last_became_active":"2024-09-19T07:55:45.796793+0000","last_became_peered":"2024-09-19T07:55:45.796793+0000","last_unstale":"2024-09-19T07:55:47.827791+0000","last_undegraded":"2024-09-19T07:55:47.827791+0000","last_fullsized":"2024-09-19T07:55:47.827791+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:14.418996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590734+0000","last_change":"2024-09-19T07:55:46.815815+0000","last_active":"2024-09-19T07:55:48.590734+0000","last_peered":"2024-09-19T07:55:48.590734+0000","last_clean":"2024-09-19T07:55:48.590734+0000","last_became_active":"2024-09-19T07:55:46.815502+0000","last_became_peered":"2024-09-19T07:55:46.815502+0000","last_unstale":"2024-09-19T07:55:48.590734+0000","last_undegraded":"2024-09-19T07:55:48.590734+0000","last_fullsized":"2024-09-19T07:55:48.590734+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:36.406278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"151'2","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.801123+0000","last_change":"2024-09-19T07:55:45.803736+0000","last_active":"2024-09-19T07:55:47.801123+0000","last_peered":"2024-09-19T07:55:47.801123+0000","last_clean":"2024-09-19T07:55:47.801123+0000","last_became_active":"2024-09-19T07:55:45.803419+0000","last_became_peered":"2024-09-19T07:55:45.803419+0000","last_unstale":"2024-09-19T07:55:47.801123+0000","last_undegraded":"2024-09-19T07:55:47.801123+0000","last_fullsized":"2024-09-19T07:55:47.801123+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:39:47.947756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806166+0000","last_change":"2024-09-19T07:55:46.812728+0000","last_active":"2024-09-19T07:55:47.806166+0000","last_peered":"2024-09-19T07:55:47.806166+0000","last_clean":"2024-09-19T07:55:47.806166+0000","last_became_active":"2024-09-19T07:55:46.811051+0000","last_became_peered":"2024-09-19T07:55:46.811051+0000","last_unstale":"2024-09-19T07:55:47.806166+0000","last_undegraded":"2024-09-19T07:55:47.806166+0000","last_fullsized":"2024-09-19T07:55:47.806166+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:25.009628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"151'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590475+0000","last_change":"2024-09-19T07:55:45.801739+0000","last_active":"2024-09-19T07:55:48.590475+0000","last_peered":"2024-09-19T07:55:48.590475+0000","last_clean":"2024-09-19T07:55:48.590475+0000","last_became_active":"2024-09-19T07:55:45.800502+0000","last_became_peered":"2024-09-19T07:55:45.800502+0000","last_unstale":"2024-09-19T07:55:48.590475+0000","last_undegraded":"2024-09-19T07:55:48.590475+0000","last_fullsized":"2024-09-19T07:55:48.590475+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:37.112493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806634+0000","last_change":"2024-09-19T07:55:46.813673+0000","last_active":"2024-09-19T07:55:47.806634+0000","last_peered":"2024-09-19T07:55:47.806634+0000","last_clean":"2024-09-19T07:55:47.806634+0000","last_became_active":"2024-09-19T07:55:46.813361+0000","last_became_peered":"2024-09-19T07:55:46.813361+0000","last_unstale":"2024-09-19T07:55:47.806634+0000","last_undegraded":"2024-09-19T07:55:47.806634+0000","last_fullsized":"2024-09-19T07:55:47.806634+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:02:49.871105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800962+0000","last_change":"2024-09-19T07:55:45.805741+0000","last_active":"2024-09-19T07:55:47.800962+0000","last_peered":"2024-09-19T07:55:47.800962+0000","last_clean":"2024-09-19T07:55:47.800962+0000","last_became_active":"2024-09-19T07:55:45.805464+0000","last_became_peered":"2024-09-19T07:55:45.805464+0000","last_unstale":"2024-09-19T07:55:47.800962+0000","last_undegraded":"2024-09-19T07:55:47.800962+0000","last_fullsized":"2024-09-19T07:55:47.800962+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:31.558613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827844+0000","last_change":"2024-09-19T07:55:46.817735+0000","last_active":"2024-09-19T07:55:47.827844+0000","last_peered":"2024-09-19T07:55:47.827844+0000","last_clean":"2024-09-19T07:55:47.827844+0000","last_became_active":"2024-09-19T07:55:46.817466+0000","last_became_peered":"2024-09-19T07:55:46.817466+0000","last_unstale":"2024-09-19T07:55:47.827844+0000","last_undegraded":"2024-09-19T07:55:47.827844+0000","last_fullsized":"2024-09-19T07:55:47.827844+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:52.010816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"151'2","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590349+0000","last_change":"2024-09-19T07:55:45.801502+0000","last_active":"2024-09-19T07:55:48.590349+0000","last_peered":"2024-09-19T07:55:48.590349+0000","last_clean":"2024-09-19T07:55:48.590349+0000","last_became_active":"2024-09-19T07:55:45.800836+0000","last_became_peered":"2024-09-19T07:55:45.800836+0000","last_unstale":"2024-09-19T07:55:48.590349+0000","last_undegraded":"2024-09-19T07:55:48.590349+0000","last_fullsized":"2024-09-19T07:55:48.590349+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:24.008200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590386+0000","last_change":"2024-09-19T07:55:46.825374+0000","last_active":"2024-09-19T07:55:48.590386+0000","last_peered":"2024-09-19T07:55:48.590386+0000","last_clean":"2024-09-19T07:55:48.590386+0000","last_became_active":"2024-09-19T07:55:46.825083+0000","last_became_peered":"2024-09-19T07:55:46.825083+0000","last_unstale":"2024-09-19T07:55:48.590386+0000","last_undegraded":"2024-09-19T07:55:48.590386+0000","last_fullsized":"2024-09-19T07:55:48.590386+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:47.605737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806267+0000","last_change":"2024-09-19T07:55:45.795584+0000","last_active":"2024-09-19T07:55:47.806267+0000","last_peered":"2024-09-19T07:55:47.806267+0000","last_clean":"2024-09-19T07:55:47.806267+0000","last_became_active":"2024-09-19T07:55:45.795283+0000","last_became_peered":"2024-09-19T07:55:45.795283+0000","last_unstale":"2024-09-19T07:55:47.806267+0000","last_undegraded":"2024-09-19T07:55:47.806267+0000","last_fullsized":"2024-09-19T07:55:47.806267+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:07.706302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590213+0000","last_change":"2024-09-19T07:55:46.824701+0000","last_active":"2024-09-19T07:55:48.590213+0000","last_peered":"2024-09-19T07:55:48.590213+0000","last_clean":"2024-09-19T07:55:48.590213+0000","last_became_active":"2024-09-19T07:55:46.824414+0000","last_became_peered":"2024-09-19T07:55:46.824414+0000","last_unstale":"2024-09-19T07:55:48.590213+0000","last_undegraded":"2024-09-19T07:55:48.590213+0000","last_fullsized":"2024-09-19T07:55:48.590213+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:18.795850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-19T07:55:45.798537+0000","last_change":"2024-09-19T07:55:45.797312+0000","last_active":"2024-09-19T07:55:45.798537+0000","last_peered":"2024-09-19T07:55:45.798537+0000","last_clean":"2024-09-19T07:55:45.798537+0000","last_became_active":"2024-09-19T07:55:45.796832+0000","last_became_peered":"2024-09-19T07:55:45.796832+0000","last_unstale":"2024-09-19T07:55:45.798537+0000","last_undegraded":"2024-09-19T07:55:45.798537+0000","last_fullsized":"2024-09-19T07:55:45.798537+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:15.635183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590487+0000","last_change":"2024-09-19T07:55:46.817406+0000","last_active":"2024-09-19T07:55:48.590487+0000","last_peered":"2024-09-19T07:55:48.590487+0000","last_clean":"2024-09-19T07:55:48.590487+0000","last_became_active":"2024-09-19T07:55:46.817128+0000","last_became_peered":"2024-09-19T07:55:46.817128+0000","last_unstale":"2024-09-19T07:55:48.590487+0000","last_undegraded":"2024-09-19T07:55:48.590487+0000","last_fullsized":"2024-09-19T07:55:48.590487+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:50.004806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590346+0000","last_change":"2024-09-19T07:55:45.795162+0000","last_active":"2024-09-19T07:55:48.590346+0000","last_peered":"2024-09-19T07:55:48.590346+0000","last_clean":"2024-09-19T07:55:48.590346+0000","last_became_active":"2024-09-19T07:55:45.794882+0000","last_became_peered":"2024-09-19T07:55:45.794882+0000","last_unstale":"2024-09-19T07:55:48.590346+0000","last_undegraded":"2024-09-19T07:55:48.590346+0000","last_fullsized":"2024-09-19T07:55:48.590346+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:28.624166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.805948+0000","last_change":"2024-09-19T07:55:46.814034+0000","last_active":"2024-09-19T07:55:47.805948+0000","last_peered":"2024-09-19T07:55:47.805948+0000","last_clean":"2024-09-19T07:55:47.805948+0000","last_became_active":"2024-09-19T07:55:46.813762+0000","last_became_peered":"2024-09-19T07:55:46.813762+0000","last_unstale":"2024-09-19T07:55:47.805948+0000","last_undegraded":"2024-09-19T07:55:47.805948+0000","last_fullsized":"2024-09-19T07:55:47.805948+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:38.113921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"151'2","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590339+0000","last_change":"2024-09-19T07:55:45.801205+0000","last_active":"2024-09-19T07:55:48.590339+0000","last_peered":"2024-09-19T07:55:48.590339+0000","last_clean":"2024-09-19T07:55:48.590339+0000","last_became_active":"2024-09-19T07:55:45.800895+0000","last_became_peered":"2024-09-19T07:55:45.800895+0000","last_unstale":"2024-09-19T07:55:48.590339+0000","last_undegraded":"2024-09-19T07:55:48.590339+0000","last_fullsized":"2024-09-19T07:55:48.590339+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:56.753472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800693+0000","last_change":"2024-09-19T07:55:46.819496+0000","last_active":"2024-09-19T07:55:47.800693+0000","last_peered":"2024-09-19T07:55:47.800693+0000","last_clean":"2024-09-19T07:55:47.800693+0000","last_became_active":"2024-09-19T07:55:46.818787+0000","last_became_peered":"2024-09-19T07:55:46.818787+0000","last_unstale":"2024-09-19T07:55:47.800693+0000","last_undegraded":"2024-09-19T07:55:47.800693+0000","last_fullsized":"2024-09-19T07:55:47.800693+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:58.971945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"152'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.828794+0000","last_change":"2024-09-19T07:55:45.800698+0000","last_active":"2024-09-19T07:55:47.828794+0000","last_peered":"2024-09-19T07:55:47.828794+0000","last_clean":"2024-09-19T07:55:47.828794+0000","last_became_active":"2024-09-19T07:55:45.800012+0000","last_became_peered":"2024-09-19T07:55:45.800012+0000","last_unstale":"2024-09-19T07:55:47.828794+0000","last_undegraded":"2024-09-19T07:55:47.828794+0000","last_fullsized":"2024-09-19T07:55:47.828794+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:21.500902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.794831+0000","last_change":"2024-09-19T07:55:45.794831+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.794831+0000","last_undegraded":"2024-09-19T07:55:45.794831+0000","last_fullsized":"2024-09-19T07:55:45.794831+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827580+0000","last_change":"2024-09-19T07:55:45.802021+0000","last_active":"2024-09-19T07:55:47.827580+0000","last_peered":"2024-09-19T07:55:47.827580+0000","last_clean":"2024-09-19T07:55:47.827580+0000","last_became_active":"2024-09-19T07:55:45.801518+0000","last_became_peered":"2024-09-19T07:55:45.801518+0000","last_unstale":"2024-09-19T07:55:47.827580+0000","last_undegraded":"2024-09-19T07:55:47.827580+0000","last_fullsized":"2024-09-19T07:55:47.827580+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:59.000128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800755+0000","last_change":"2024-09-19T07:55:46.812665+0000","last_active":"2024-09-19T07:55:47.800755+0000","last_peered":"2024-09-19T07:55:47.800755+0000","last_clean":"2024-09-19T07:55:47.800755+0000","last_became_active":"2024-09-19T07:55:46.812424+0000","last_became_peered":"2024-09-19T07:55:46.812424+0000","last_unstale":"2024-09-19T07:55:47.800755+0000","last_undegraded":"2024-09-19T07:55:47.800755+0000","last_fullsized":"2024-09-19T07:55:47.800755+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:50.004806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-19T07:55:45.808438+0000","last_change":"2024-09-19T07:55:45.804911+0000","last_active":"2024-09-19T07:55:45.808438+0000","last_peered":"2024-09-19T07:55:45.808438+0000","last_clean":"2024-09-19T07:55:45.808438+0000","last_became_active":"2024-09-19T07:55:45.804578+0000","last_became_peered":"2024-09-19T07:55:45.804578+0000","last_unstale":"2024-09-19T07:55:45.808438+0000","last_undegraded":"2024-09-19T07:55:45.808438+0000","last_fullsized":"2024-09-19T07:55:45.808438+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:04.552288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806888+0000","last_change":"2024-09-19T07:55:46.815931+0000","last_active":"2024-09-19T07:55:47.806888+0000","last_peered":"2024-09-19T07:55:47.806888+0000","last_clean":"2024-09-19T07:55:47.806888+0000","last_became_active":"2024-09-19T07:55:46.815675+0000","last_became_peered":"2024-09-19T07:55:46.815675+0000","last_unstale":"2024-09-19T07:55:47.806888+0000","last_undegraded":"2024-09-19T07:55:47.806888+0000","last_fullsized":"2024-09-19T07:55:47.806888+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:42.354606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806692+0000","last_change":"2024-09-19T07:55:45.803403+0000","last_active":"2024-09-19T07:55:47.806692+0000","last_peered":"2024-09-19T07:55:47.806692+0000","last_clean":"2024-09-19T07:55:47.806692+0000","last_became_active":"2024-09-19T07:55:45.802988+0000","last_became_peered":"2024-09-19T07:55:45.802988+0000","last_unstale":"2024-09-19T07:55:47.806692+0000","last_undegraded":"2024-09-19T07:55:47.806692+0000","last_fullsized":"2024-09-19T07:55:47.806692+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:35.404851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590533+0000","last_change":"2024-09-19T07:55:46.815044+0000","last_active":"2024-09-19T07:55:48.590533+0000","last_peered":"2024-09-19T07:55:48.590533+0000","last_clean":"2024-09-19T07:55:48.590533+0000","last_became_active":"2024-09-19T07:55:46.814746+0000","last_became_peered":"2024-09-19T07:55:46.814746+0000","last_unstale":"2024-09-19T07:55:48.590533+0000","last_undegraded":"2024-09-19T07:55:48.590533+0000","last_fullsized":"2024-09-19T07:55:48.590533+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:20.053063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827507+0000","last_change":"2024-09-19T07:55:45.801889+0000","last_active":"2024-09-19T07:55:47.827507+0000","last_peered":"2024-09-19T07:55:47.827507+0000","last_clean":"2024-09-19T07:55:47.827507+0000","last_became_active":"2024-09-19T07:55:45.801569+0000","last_became_peered":"2024-09-19T07:55:45.801569+0000","last_unstale":"2024-09-19T07:55:47.827507+0000","last_undegraded":"2024-09-19T07:55:47.827507+0000","last_fullsized":"2024-09-19T07:55:47.827507+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:19.834999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827678+0000","last_change":"2024-09-19T07:55:46.816559+0000","last_active":"2024-09-19T07:55:47.827678+0000","last_peered":"2024-09-19T07:55:47.827678+0000","last_clean":"2024-09-19T07:55:47.827678+0000","last_became_active":"2024-09-19T07:55:46.816305+0000","last_became_peered":"2024-09-19T07:55:46.816305+0000","last_unstale":"2024-09-19T07:55:47.827678+0000","last_undegraded":"2024-09-19T07:55:47.827678+0000","last_fullsized":"2024-09-19T07:55:47.827678+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:27.599531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806465+0000","last_change":"2024-09-19T07:55:45.797679+0000","last_active":"2024-09-19T07:55:47.806465+0000","last_peered":"2024-09-19T07:55:47.806465+0000","last_clean":"2024-09-19T07:55:47.806465+0000","last_became_active":"2024-09-19T07:55:45.797383+0000","last_became_peered":"2024-09-19T07:55:45.797383+0000","last_unstale":"2024-09-19T07:55:47.806465+0000","last_undegraded":"2024-09-19T07:55:47.806465+0000","last_fullsized":"2024-09-19T07:55:47.806465+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:28.624166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.790864+0000","last_change":"2024-09-19T07:55:45.790864+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.790864+0000","last_undegraded":"2024-09-19T07:55:45.790864+0000","last_fullsized":"2024-09-19T07:55:45.790864+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.589573+0000","last_change":"2024-09-19T07:55:45.800775+0000","last_active":"2024-09-19T07:55:48.589573+0000","last_peered":"2024-09-19T07:55:48.589573+0000","last_clean":"2024-09-19T07:55:48.589573+0000","last_became_active":"2024-09-19T07:55:45.800484+0000","last_became_peered":"2024-09-19T07:55:45.800484+0000","last_unstale":"2024-09-19T07:55:48.589573+0000","last_undegraded":"2024-09-19T07:55:48.589573+0000","last_fullsized":"2024-09-19T07:55:48.589573+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:35.490586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590087+0000","last_change":"2024-09-19T07:55:46.818224+0000","last_active":"2024-09-19T07:55:48.590087+0000","last_peered":"2024-09-19T07:55:48.590087+0000","last_clean":"2024-09-19T07:55:48.590087+0000","last_became_active":"2024-09-19T07:55:46.817954+0000","last_became_peered":"2024-09-19T07:55:46.817954+0000","last_unstale":"2024-09-19T07:55:48.590087+0000","last_undegraded":"2024-09-19T07:55:48.590087+0000","last_fullsized":"2024-09-19T07:55:48.590087+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:38.643240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827644+0000","last_change":"2024-09-19T07:55:45.800943+0000","last_active":"2024-09-19T07:55:47.827644+0000","last_peered":"2024-09-19T07:55:47.827644+0000","last_clean":"2024-09-19T07:55:47.827644+0000","last_became_active":"2024-09-19T07:55:45.800428+0000","last_became_peered":"2024-09-19T07:55:45.800428+0000","last_unstale":"2024-09-19T07:55:47.827644+0000","last_undegraded":"2024-09-19T07:55:47.827644+0000","last_fullsized":"2024-09-19T07:55:47.827644+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:40.722531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":408,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590404+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:55:48.590404+0000","last_peered":"2024-09-19T07:55:48.590404+0000","last_clean":"2024-09-19T07:55:48.590404+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:55:48.590404+0000","last_undegraded":"2024-09-19T07:55:48.590404+0000","last_fullsized":"2024-09-19T07:55:48.590404+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509598,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87708,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284132,"statfs":{"total":96636764160,"available":96546951168,"internally_reserved":0,"allocated":2498560,"data_stored":1476936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":87292362},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509597,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105240,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266600,"statfs":{"total":96636764160,"available":96528998400,"internally_reserved":0,"allocated":1904640,"data_stored":886568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509598,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73252,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298588,"statfs":{"total":96636764160,"available":96561754112,"internally_reserved":0,"allocated":1912832,"data_stored":886692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509598,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87152,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284688,"statfs":{"total":96636764160,"available":96547520512,"internally_reserved":0,"allocated":1921024,"data_stored":887332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509597,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105764,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266076,"statfs":{"total":96636764160,"available":96528461824,"internally_reserved":0,"allocated":2506752,"data_stored":1477040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542301,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119088,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252752,"statfs":{"total":96636764160,"available":96514818048,"internally_reserved":0,"allocated":1925120,"data_stored":888784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542301,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83052,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288788,"statfs":{"total":96636764160,"available":96551718912,"internally_reserved":0,"allocated":1916928,"data_stored":888180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575006,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109312,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262528,"statfs":{"total":96636764160,"available":96524828672,"internally_reserved":0,"allocated":1880064,"data_stored":872709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:55:50.904 INFO:tasks.ceph.ceph_manager.ceph:PG 24.2c is not active+clean 2024-09-19T07:55:50.904 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '24.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 150, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:55:45.791579+0000', 'last_change': '2024-09-19T07:55:45.791579+0000', 'last_active': '2024-09-19T07:55:45.779621+0000', 'last_peered': '2024-09-19T07:55:45.779621+0000', 'last_clean': '2024-09-19T07:55:45.779621+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:55:45.791579+0000', 'last_undegraded': '2024-09-19T07:55:45.791579+0000', 'last_fullsized': '2024-09-19T07:55:45.791579+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'last_clean_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T07:55:50.904 INFO:tasks.ceph.ceph_manager.ceph:PG 24.26 is not active+clean 2024-09-19T07:55:50.904 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '24.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 150, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:55:45.790719+0000', 'last_change': '2024-09-19T07:55:45.790719+0000', 'last_active': '2024-09-19T07:55:45.779621+0000', 'last_peered': '2024-09-19T07:55:45.779621+0000', 'last_clean': '2024-09-19T07:55:45.779621+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:55:45.790719+0000', 'last_undegraded': '2024-09-19T07:55:45.790719+0000', 'last_fullsized': '2024-09-19T07:55:45.790719+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'last_clean_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T07:55:50.905 INFO:tasks.ceph.ceph_manager.ceph:PG 24.15 is not active+clean 2024-09-19T07:55:50.905 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '24.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 150, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:55:45.790402+0000', 'last_change': '2024-09-19T07:55:45.790402+0000', 'last_active': '2024-09-19T07:55:45.779621+0000', 'last_peered': '2024-09-19T07:55:45.779621+0000', 'last_clean': '2024-09-19T07:55:45.779621+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:55:45.790402+0000', 'last_undegraded': '2024-09-19T07:55:45.790402+0000', 'last_fullsized': '2024-09-19T07:55:45.790402+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'last_clean_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T07:55:50.905 INFO:tasks.ceph.ceph_manager.ceph:PG 24.16 is not active+clean 2024-09-19T07:55:50.905 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '24.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 150, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:55:45.790244+0000', 'last_change': '2024-09-19T07:55:45.790244+0000', 'last_active': '2024-09-19T07:55:45.779621+0000', 'last_peered': '2024-09-19T07:55:45.779621+0000', 'last_clean': '2024-09-19T07:55:45.779621+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:55:45.790244+0000', 'last_undegraded': '2024-09-19T07:55:45.790244+0000', 'last_fullsized': '2024-09-19T07:55:45.790244+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'last_clean_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T07:55:50.905 INFO:tasks.ceph.ceph_manager.ceph:PG 24.1b is not active+clean 2024-09-19T07:55:50.905 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '24.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 150, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:55:45.791181+0000', 'last_change': '2024-09-19T07:55:45.791181+0000', 'last_active': '2024-09-19T07:55:45.779621+0000', 'last_peered': '2024-09-19T07:55:45.779621+0000', 'last_clean': '2024-09-19T07:55:45.779621+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:55:45.791181+0000', 'last_undegraded': '2024-09-19T07:55:45.791181+0000', 'last_fullsized': '2024-09-19T07:55:45.791181+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'last_clean_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T07:55:50.905 INFO:tasks.ceph.ceph_manager.ceph:PG 24.1f is not active+clean 2024-09-19T07:55:50.905 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '24.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 150, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:55:45.791295+0000', 'last_change': '2024-09-19T07:55:45.791295+0000', 'last_active': '2024-09-19T07:55:45.779621+0000', 'last_peered': '2024-09-19T07:55:45.779621+0000', 'last_clean': '2024-09-19T07:55:45.779621+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:55:45.791295+0000', 'last_undegraded': '2024-09-19T07:55:45.791295+0000', 'last_fullsized': '2024-09-19T07:55:45.791295+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'last_clean_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T07:55:50.905 INFO:tasks.ceph.ceph_manager.ceph:PG 24.a is not active+clean 2024-09-19T07:55:50.905 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '24.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 150, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:55:45.794831+0000', 'last_change': '2024-09-19T07:55:45.794831+0000', 'last_active': '2024-09-19T07:55:45.779621+0000', 'last_peered': '2024-09-19T07:55:45.779621+0000', 'last_clean': '2024-09-19T07:55:45.779621+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:55:45.794831+0000', 'last_undegraded': '2024-09-19T07:55:45.794831+0000', 'last_fullsized': '2024-09-19T07:55:45.794831+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'last_clean_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T07:55:50.906 INFO:tasks.ceph.ceph_manager.ceph:PG 24.f is not active+clean 2024-09-19T07:55:50.906 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '24.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 150, 'state': 'creating+peering', 'last_fresh': '2024-09-19T07:55:45.790864+0000', 'last_change': '2024-09-19T07:55:45.790864+0000', 'last_active': '2024-09-19T07:55:45.779621+0000', 'last_peered': '2024-09-19T07:55:45.779621+0000', 'last_clean': '2024-09-19T07:55:45.779621+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T07:55:45.790864+0000', 'last_undegraded': '2024-09-19T07:55:45.790864+0000', 'last_fullsized': '2024-09-19T07:55:45.790864+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'last_clean_scrub_stamp': '2024-09-19T07:55:45.779621+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T07:55:50.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T07:55:51.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.042+0000 7fd59580e640 1 Processor -- start 2024-09-19T07:55:51.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.042+0000 7fd59580e640 1 -- start start 2024-09-19T07:55:51.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.043+0000 7fd59580e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd59006cd20 0x7fd59006d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:51.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.043+0000 7fd59580e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5901425e0 0x7fd590142a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:51.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.043+0000 7fd59580e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd59006d640 con 0x7fd590161eb0 2024-09-19T07:55:51.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.043+0000 7fd59580e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd590162e50 con 0x7fd5901425e0 2024-09-19T07:55:51.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.043+0000 7fd59580e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fd590142150 con 0x7fd59006cd20 2024-09-19T07:55:51.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.043+0000 7fd58f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5901425e0 0x7fd590142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.043+0000 7fd58f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5901425e0 0x7fd590142a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49844/0 (socket says 172.21.15.16:49844) 2024-09-19T07:55:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.043+0000 7fd58f7fe640 1 -- 172.21.15.16:0/2871300564 learned_addr learned my addr 172.21.15.16:0/2871300564 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.043+0000 7fd58effd640 1 --2- 172.21.15.16:0/2871300564 >> v2:172.21.15.146:3300/0 conn(0x7fd59006cd20 0x7fd59006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.043+0000 7fd58f7fe640 1 -- 172.21.15.16:0/2871300564 >> v2:172.21.15.146:3300/0 conn(0x7fd59006cd20 msgr2=0x7fd59006d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.043+0000 7fd58f7fe640 1 --2- 172.21.15.16:0/2871300564 >> v2:172.21.15.146:3300/0 conn(0x7fd59006cd20 0x7fd59006d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.043+0000 7fd58f7fe640 1 -- 172.21.15.16:0/2871300564 >> v1:172.21.15.146:6789/0 conn(0x7fd590161eb0 legacy=0x7fd5901622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.043+0000 7fd58f7fe640 1 -- 172.21.15.16:0/2871300564 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd590162b70 con 0x7fd5901425e0 2024-09-19T07:55:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.043+0000 7fd58effd640 1 --2- 172.21.15.16:0/2871300564 >> v2:172.21.15.146:3300/0 conn(0x7fd59006cd20 0x7fd59006d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.044+0000 7fd58f7fe640 1 --2- 172.21.15.16:0/2871300564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5901425e0 0x7fd590142a70 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fd590070ff0 tx=0x7fd58002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2552941b8ba98c08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.044+0000 7fd58dffb640 1 -- 172.21.15.16:0/2871300564 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580036070 con 0x7fd5901425e0 2024-09-19T07:55:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.044+0000 7fd58dffb640 1 -- 172.21.15.16:0/2871300564 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd580031600 con 0x7fd5901425e0 2024-09-19T07:55:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.044+0000 7fd58dffb640 1 -- 172.21.15.16:0/2871300564 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5800319d0 con 0x7fd5901425e0 2024-09-19T07:55:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.045+0000 7fd59580e640 1 -- 172.21.15.16:0/2871300564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5901425e0 msgr2=0x7fd590142a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.045+0000 7fd59580e640 1 --2- 172.21.15.16:0/2871300564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5901425e0 0x7fd590142a70 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fd590070ff0 tx=0x7fd58002a660 comp rx=0 tx=0).stop 2024-09-19T07:55:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.045+0000 7fd59580e640 1 -- 172.21.15.16:0/2871300564 shutdown_connections 2024-09-19T07:55:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.045+0000 7fd59580e640 1 --2- 172.21.15.16:0/2871300564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5901425e0 0x7fd590142a70 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.045+0000 7fd59580e640 1 --2- 172.21.15.16:0/2871300564 >> v2:172.21.15.146:3300/0 conn(0x7fd59006cd20 0x7fd59006d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.045+0000 7fd59580e640 1 -- 172.21.15.16:0/2871300564 >> 172.21.15.16:0/2871300564 conn(0x7fd5900b3ed0 msgr2=0x7fd5900b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.045+0000 7fd59580e640 1 -- 172.21.15.16:0/2871300564 shutdown_connections 2024-09-19T07:55:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.045+0000 7fd59580e640 1 -- 172.21.15.16:0/2871300564 wait complete. 2024-09-19T07:55:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.046+0000 7fd59580e640 1 Processor -- start 2024-09-19T07:55:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.046+0000 7fd59580e640 1 -- start start 2024-09-19T07:55:51.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.047+0000 7fd59580e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd59006cd20 0x7fd5900cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.047+0000 7fd59580e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5901425e0 0x7fd5900b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.047+0000 7fd59580e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd590161eb0 0x7fd5901591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.047+0000 7fd59580e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd590141600 con 0x7fd5901425e0 2024-09-19T07:55:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.047+0000 7fd59580e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd590141de0 con 0x7fd59006cd20 2024-09-19T07:55:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.047+0000 7fd59580e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd590142fb0 con 0x7fd590161eb0 2024-09-19T07:55:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.047+0000 7fd58e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5901425e0 0x7fd5900b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.047+0000 7fd58e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5901425e0 0x7fd5900b51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49856/0 (socket says 172.21.15.16:49856) 2024-09-19T07:55:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.047+0000 7fd58e7fc640 1 -- 172.21.15.16:0/3105091263 learned_addr learned my addr 172.21.15.16:0/3105091263 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.047+0000 7fd58f7fe640 1 --2- 172.21.15.16:0/3105091263 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd590161eb0 0x7fd5901591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.047+0000 7fd58effd640 1 --2- 172.21.15.16:0/3105091263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd59006cd20 0x7fd5900cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.048+0000 7fd58e7fc640 1 -- 172.21.15.16:0/3105091263 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd590161eb0 msgr2=0x7fd5901591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.048+0000 7fd58e7fc640 1 --2- 172.21.15.16:0/3105091263 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd590161eb0 0x7fd5901591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.048+0000 7fd58e7fc640 1 -- 172.21.15.16:0/3105091263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd59006cd20 msgr2=0x7fd5900cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.048+0000 7fd58e7fc640 1 --2- 172.21.15.16:0/3105091263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd59006cd20 0x7fd5900cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.048+0000 7fd58e7fc640 1 -- 172.21.15.16:0/3105091263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5900cc350 con 0x7fd5901425e0 2024-09-19T07:55:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.048+0000 7fd58e7fc640 1 --2- 172.21.15.16:0/3105091263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5901425e0 0x7fd5900b51d0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fd57800b3f0 tx=0x7fd57800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.048+0000 7fd587fff640 1 -- 172.21.15.16:0/3105091263 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd578015020 con 0x7fd5901425e0 2024-09-19T07:55:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.048+0000 7fd587fff640 1 -- 172.21.15.16:0/3105091263 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd578011830 con 0x7fd5901425e0 2024-09-19T07:55:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.048+0000 7fd59580e640 1 -- 172.21.15.16:0/3105091263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd590159b30 con 0x7fd5901425e0 2024-09-19T07:55:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.048+0000 7fd587fff640 1 -- 172.21.15.16:0/3105091263 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd578011c20 con 0x7fd5901425e0 2024-09-19T07:55:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.048+0000 7fd59580e640 1 -- 172.21.15.16:0/3105091263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5900c96b0 con 0x7fd5901425e0 2024-09-19T07:55:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.049+0000 7fd59580e640 1 -- 172.21.15.16:0/3105091263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd59006d100 con 0x7fd5901425e0 2024-09-19T07:55:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.050+0000 7fd587fff640 1 -- 172.21.15.16:0/3105091263 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd578018020 con 0x7fd5901425e0 2024-09-19T07:55:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.050+0000 7fd587fff640 1 --2- 172.21.15.16:0/3105091263 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd57006c2b0 0x7fd57006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.050+0000 7fd587fff640 1 -- 172.21.15.16:0/3105091263 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7fd57808b170 con 0x7fd5901425e0 2024-09-19T07:55:51.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.051+0000 7fd58effd640 1 --2- 172.21.15.16:0/3105091263 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd57006c2b0 0x7fd57006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:51.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.051+0000 7fd58effd640 1 --2- 172.21.15.16:0/3105091263 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd57006c2b0 0x7fd57006e770 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd58800a990 tx=0x7fd588008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:51.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.054+0000 7fd587fff640 1 -- 172.21.15.16:0/3105091263 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd57801f030 con 0x7fd5901425e0 2024-09-19T07:55:51.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.304+0000 7fd59580e640 1 -- 172.21.15.16:0/3105091263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd590072220 con 0x7fd5901425e0 2024-09-19T07:55:51.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.304+0000 7fd587fff640 1 -- 172.21.15.16:0/3105091263 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1564 (secure 0 0 0) 0x7fd5780597e0 con 0x7fd5901425e0 2024-09-19T07:55:51.304 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:51.305 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":789,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":152,"num_osds":8,"num_up_osds":8,"osd_up_since":1726731773,"num_in_osds":8,"osd_in_since":1726731764,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":89},{"state_name":"creating+peering","count":8}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":789061632,"bytes_avail":772305051648,"bytes_total":773094113280,"inactive_pgs_ratio":0.082474224269390106,"write_bytes_sec":2446,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":92,"btime":"2024-09-19T07:55:49:210981+0000","id":11,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":11,"rank":0,"name":"c","status":"up:active","gid":8760}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-19T07:53:32.607149+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T07:55:51.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.306+0000 7fd59580e640 1 -- 172.21.15.16:0/3105091263 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd57006c2b0 msgr2=0x7fd57006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:51.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.306+0000 7fd59580e640 1 --2- 172.21.15.16:0/3105091263 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd57006c2b0 0x7fd57006e770 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd58800a990 tx=0x7fd588008040 comp rx=0 tx=0).stop 2024-09-19T07:55:51.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.306+0000 7fd59580e640 1 -- 172.21.15.16:0/3105091263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5901425e0 msgr2=0x7fd5900b51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:51.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.306+0000 7fd59580e640 1 --2- 172.21.15.16:0/3105091263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5901425e0 0x7fd5900b51d0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fd57800b3f0 tx=0x7fd57800b8b0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.306+0000 7fd59580e640 1 -- 172.21.15.16:0/3105091263 shutdown_connections 2024-09-19T07:55:51.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.306+0000 7fd59580e640 1 --2- 172.21.15.16:0/3105091263 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd57006c2b0 0x7fd57006e770 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.306+0000 7fd59580e640 1 --2- 172.21.15.16:0/3105091263 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd590161eb0 0x7fd5901591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.306+0000 7fd59580e640 1 --2- 172.21.15.16:0/3105091263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5901425e0 0x7fd5900b51d0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.306+0000 7fd59580e640 1 --2- 172.21.15.16:0/3105091263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd59006cd20 0x7fd5900cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.306+0000 7fd59580e640 1 -- 172.21.15.16:0/3105091263 >> 172.21.15.16:0/3105091263 conn(0x7fd5900b3ed0 msgr2=0x7fd59015f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:51.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.306+0000 7fd59580e640 1 -- 172.21.15.16:0/3105091263 shutdown_connections 2024-09-19T07:55:51.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.306+0000 7fd59580e640 1 -- 172.21.15.16:0/3105091263 wait complete. 2024-09-19T07:55:51.318 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-19T07:55:51.318 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:55:51.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.420+0000 7f54358a6640 1 Processor -- start 2024-09-19T07:55:51.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.420+0000 7f54358a6640 1 -- start start 2024-09-19T07:55:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.421+0000 7f54358a6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f543015e680 0x7f54300cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.421+0000 7f54358a6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54300d0010 0x7f54300d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.421+0000 7f54358a6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54300d1880 con 0x7f543015e2a0 2024-09-19T07:55:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.421+0000 7f54358a6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f54300d1b60 con 0x7f54300d0010 2024-09-19T07:55:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.421+0000 7f54358a6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f54300cf2d0 con 0x7f543015e680 2024-09-19T07:55:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.421+0000 7f542f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54300d0010 0x7f54300d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.421+0000 7f542f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54300d0010 0x7f54300d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49860/0 (socket says 172.21.15.16:49860) 2024-09-19T07:55:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.421+0000 7f542f7fe640 1 -- 172.21.15.16:0/2005943527 learned_addr learned my addr 172.21.15.16:0/2005943527 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.421+0000 7f542e7fc640 1 --2- 172.21.15.16:0/2005943527 >> v2:172.21.15.146:3300/0 conn(0x7f543015e680 0x7f54300cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.421+0000 7f542f7fe640 1 -- 172.21.15.16:0/2005943527 >> v2:172.21.15.146:3300/0 conn(0x7f543015e680 msgr2=0x7f54300cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.421+0000 7f542f7fe640 1 --2- 172.21.15.16:0/2005943527 >> v2:172.21.15.146:3300/0 conn(0x7f543015e680 0x7f54300cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.421+0000 7f542f7fe640 1 -- 172.21.15.16:0/2005943527 >> v1:172.21.15.146:6789/0 conn(0x7f543015e2a0 legacy=0x7f543006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.421+0000 7f542f7fe640 1 -- 172.21.15.16:0/2005943527 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54300cf5b0 con 0x7f54300d0010 2024-09-19T07:55:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.422+0000 7f542f7fe640 1 --2- 172.21.15.16:0/2005943527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54300d0010 0x7f54300d4370 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f54300a22a0 tx=0x7f542802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a9431cae46f94 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.422+0000 7f542dffb640 1 -- 172.21.15.16:0/2005943527 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428037070 con 0x7f54300d0010 2024-09-19T07:55:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.422+0000 7f542dffb640 1 -- 172.21.15.16:0/2005943527 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5428032440 con 0x7f54300d0010 2024-09-19T07:55:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.422+0000 7f542dffb640 1 -- 172.21.15.16:0/2005943527 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428032810 con 0x7f54300d0010 2024-09-19T07:55:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.423+0000 7f54358a6640 1 -- 172.21.15.16:0/2005943527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54300d0010 msgr2=0x7f54300d4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.423+0000 7f54358a6640 1 --2- 172.21.15.16:0/2005943527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54300d0010 0x7f54300d4370 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f54300a22a0 tx=0x7f542802a1a0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.423+0000 7f54358a6640 1 -- 172.21.15.16:0/2005943527 shutdown_connections 2024-09-19T07:55:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.423+0000 7f54358a6640 1 --2- 172.21.15.16:0/2005943527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54300d0010 0x7f54300d4370 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.423+0000 7f54358a6640 1 --2- 172.21.15.16:0/2005943527 >> v2:172.21.15.146:3300/0 conn(0x7f543015e680 0x7f54300cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.423+0000 7f54358a6640 1 -- 172.21.15.16:0/2005943527 >> 172.21.15.16:0/2005943527 conn(0x7f543009d760 msgr2=0x7f543008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.423+0000 7f54358a6640 1 -- 172.21.15.16:0/2005943527 shutdown_connections 2024-09-19T07:55:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.423+0000 7f54358a6640 1 -- 172.21.15.16:0/2005943527 wait complete. 2024-09-19T07:55:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.424+0000 7f54358a6640 1 Processor -- start 2024-09-19T07:55:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.424+0000 7f54358a6640 1 -- start start 2024-09-19T07:55:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.425+0000 7f54358a6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54300d0010 0x7f543015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.425+0000 7f54358a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f543015e2a0 0x7f54300cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.425+0000 7f54358a6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f543015e680 0x7f54300cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.425+0000 7f54358a6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f543006dcb0 con 0x7f543015e2a0 2024-09-19T07:55:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.425+0000 7f54358a6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f543006db30 con 0x7f54300d0010 2024-09-19T07:55:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.425+0000 7f54358a6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f54300d48b0 con 0x7f543015e680 2024-09-19T07:55:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.425+0000 7f542e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f543015e2a0 0x7f54300cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.425+0000 7f542e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f543015e2a0 0x7f54300cd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49874/0 (socket says 172.21.15.16:49874) 2024-09-19T07:55:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.425+0000 7f542e7fc640 1 -- 172.21.15.16:0/1481326720 learned_addr learned my addr 172.21.15.16:0/1481326720 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.425+0000 7f542effd640 1 --2- 172.21.15.16:0/1481326720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54300d0010 0x7f543015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.425+0000 7f542f7fe640 1 --2- 172.21.15.16:0/1481326720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f543015e680 0x7f54300cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.425+0000 7f542e7fc640 1 -- 172.21.15.16:0/1481326720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f543015e680 msgr2=0x7f54300cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.425+0000 7f542e7fc640 1 --2- 172.21.15.16:0/1481326720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f543015e680 0x7f54300cdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.425+0000 7f542e7fc640 1 -- 172.21.15.16:0/1481326720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54300d0010 msgr2=0x7f543015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.425+0000 7f542e7fc640 1 --2- 172.21.15.16:0/1481326720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54300d0010 0x7f543015c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.425+0000 7f542e7fc640 1 -- 172.21.15.16:0/1481326720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54300ce740 con 0x7f543015e2a0 2024-09-19T07:55:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.425+0000 7f542e7fc640 1 --2- 172.21.15.16:0/1481326720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f543015e2a0 0x7f54300cd5c0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f541800d5c0 tx=0x7f541800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.426+0000 7f5427fff640 1 -- 172.21.15.16:0/1481326720 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54180125f0 con 0x7f543015e2a0 2024-09-19T07:55:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.426+0000 7f5427fff640 1 -- 172.21.15.16:0/1481326720 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5418019070 con 0x7f543015e2a0 2024-09-19T07:55:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.426+0000 7f54358a6640 1 -- 172.21.15.16:0/1481326720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54300ce180 con 0x7f543015e2a0 2024-09-19T07:55:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.426+0000 7f5427fff640 1 -- 172.21.15.16:0/1481326720 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54180129d0 con 0x7f543015e2a0 2024-09-19T07:55:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.426+0000 7f54358a6640 1 -- 172.21.15.16:0/1481326720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54300c9370 con 0x7f543015e2a0 2024-09-19T07:55:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.427+0000 7f54358a6640 1 -- 172.21.15.16:0/1481326720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54300d4c70 con 0x7f543015e2a0 2024-09-19T07:55:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.427+0000 7f5427fff640 1 -- 172.21.15.16:0/1481326720 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5418012b70 con 0x7f543015e2a0 2024-09-19T07:55:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.428+0000 7f5427fff640 1 --2- 172.21.15.16:0/1481326720 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f541006c2b0 0x7f541006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.428+0000 7f5427fff640 1 -- 172.21.15.16:0/1481326720 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f541808b890 con 0x7f543015e2a0 2024-09-19T07:55:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.428+0000 7f542effd640 1 --2- 172.21.15.16:0/1481326720 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f541006c2b0 0x7f541006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:51.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.428+0000 7f542effd640 1 --2- 172.21.15.16:0/1481326720 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f541006c2b0 0x7f541006e770 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5420005660 tx=0x7f5420003d90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:51.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.432+0000 7f5427fff640 1 -- 172.21.15.16:0/1481326720 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5418059f00 con 0x7f543015e2a0 2024-09-19T07:55:51.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.604+0000 7f54358a6640 1 -- 172.21.15.16:0/1481326720 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f543006cf90 con 0x7f541006c2b0 2024-09-19T07:55:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.610+0000 7f5427fff640 1 -- 172.21.15.16:0/1481326720 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251829 (secure 0 0 0) 0x7f543006cf90 con 0x7f541006c2b0 2024-09-19T07:55:51.613 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:55:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.613+0000 7f54358a6640 1 -- 172.21.15.16:0/1481326720 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f541006c2b0 msgr2=0x7f541006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.613+0000 7f54358a6640 1 --2- 172.21.15.16:0/1481326720 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f541006c2b0 0x7f541006e770 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5420005660 tx=0x7f5420003d90 comp rx=0 tx=0).stop 2024-09-19T07:55:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.613+0000 7f54358a6640 1 -- 172.21.15.16:0/1481326720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f543015e2a0 msgr2=0x7f54300cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.613+0000 7f54358a6640 1 --2- 172.21.15.16:0/1481326720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f543015e2a0 0x7f54300cd5c0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f541800d5c0 tx=0x7f541800da80 comp rx=0 tx=0).stop 2024-09-19T07:55:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.613+0000 7f54358a6640 1 -- 172.21.15.16:0/1481326720 shutdown_connections 2024-09-19T07:55:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.613+0000 7f54358a6640 1 --2- 172.21.15.16:0/1481326720 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f541006c2b0 0x7f541006e770 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.613+0000 7f54358a6640 1 --2- 172.21.15.16:0/1481326720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f543015e680 0x7f54300cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.613+0000 7f54358a6640 1 --2- 172.21.15.16:0/1481326720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f543015e2a0 0x7f54300cd5c0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.613+0000 7f54358a6640 1 --2- 172.21.15.16:0/1481326720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54300d0010 0x7f543015c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.613+0000 7f54358a6640 1 -- 172.21.15.16:0/1481326720 >> 172.21.15.16:0/1481326720 conn(0x7f543009d760 msgr2=0x7f5430162320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.613+0000 7f54358a6640 1 -- 172.21.15.16:0/1481326720 shutdown_connections 2024-09-19T07:55:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:51.613+0000 7f54358a6640 1 -- 172.21.15.16:0/1481326720 wait complete. 2024-09-19T07:55:51.625 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":535,"stamp":"2024-09-19T07:55:50.640721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":201,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":770568,"kb_used_data":16080,"kb_used_omap":12,"kb_used_meta":754291,"kb_avail":754204152,"statfs":{"total":773094113280,"available":772305051648,"internally_reserved":0,"allocated":16465920,"data_stored":8264241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":772394544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.860165"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807561+0000","last_change":"2024-09-19T07:55:46.814625+0000","last_active":"2024-09-19T07:55:47.807561+0000","last_peered":"2024-09-19T07:55:47.807561+0000","last_clean":"2024-09-19T07:55:47.807561+0000","last_became_active":"2024-09-19T07:55:46.814365+0000","last_became_peered":"2024-09-19T07:55:46.814365+0000","last_unstale":"2024-09-19T07:55:47.807561+0000","last_undegraded":"2024-09-19T07:55:47.807561+0000","last_fullsized":"2024-09-19T07:55:47.807561+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:13.254644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590303+0000","last_change":"2024-09-19T07:55:46.811993+0000","last_active":"2024-09-19T07:55:48.590303+0000","last_peered":"2024-09-19T07:55:48.590303+0000","last_clean":"2024-09-19T07:55:48.590303+0000","last_became_active":"2024-09-19T07:55:46.811723+0000","last_became_peered":"2024-09-19T07:55:46.811723+0000","last_unstale":"2024-09-19T07:55:48.590303+0000","last_undegraded":"2024-09-19T07:55:48.590303+0000","last_fullsized":"2024-09-19T07:55:48.590303+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:37:45.202649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807311+0000","last_change":"2024-09-19T07:55:46.818859+0000","last_active":"2024-09-19T07:55:47.807311+0000","last_peered":"2024-09-19T07:55:47.807311+0000","last_clean":"2024-09-19T07:55:47.807311+0000","last_became_active":"2024-09-19T07:55:46.818573+0000","last_became_peered":"2024-09-19T07:55:46.818573+0000","last_unstale":"2024-09-19T07:55:47.807311+0000","last_undegraded":"2024-09-19T07:55:47.807311+0000","last_fullsized":"2024-09-19T07:55:47.807311+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:03.579883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806482+0000","last_change":"2024-09-19T07:55:46.813506+0000","last_active":"2024-09-19T07:55:47.806482+0000","last_peered":"2024-09-19T07:55:47.806482+0000","last_clean":"2024-09-19T07:55:47.806482+0000","last_became_active":"2024-09-19T07:55:46.813249+0000","last_became_peered":"2024-09-19T07:55:46.813249+0000","last_unstale":"2024-09-19T07:55:47.806482+0000","last_undegraded":"2024-09-19T07:55:47.806482+0000","last_fullsized":"2024-09-19T07:55:47.806482+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:06.075192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590102+0000","last_change":"2024-09-19T07:55:46.823203+0000","last_active":"2024-09-19T07:55:48.590102+0000","last_peered":"2024-09-19T07:55:48.590102+0000","last_clean":"2024-09-19T07:55:48.590102+0000","last_became_active":"2024-09-19T07:55:46.822707+0000","last_became_peered":"2024-09-19T07:55:46.822707+0000","last_unstale":"2024-09-19T07:55:48.590102+0000","last_undegraded":"2024-09-19T07:55:48.590102+0000","last_fullsized":"2024-09-19T07:55:48.590102+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:23.021974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590780+0000","last_change":"2024-09-19T07:55:46.813957+0000","last_active":"2024-09-19T07:55:48.590780+0000","last_peered":"2024-09-19T07:55:48.590780+0000","last_clean":"2024-09-19T07:55:48.590780+0000","last_became_active":"2024-09-19T07:55:46.813674+0000","last_became_peered":"2024-09-19T07:55:46.813674+0000","last_unstale":"2024-09-19T07:55:48.590780+0000","last_undegraded":"2024-09-19T07:55:48.590780+0000","last_fullsized":"2024-09-19T07:55:48.590780+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:07.805087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800604+0000","last_change":"2024-09-19T07:55:46.819762+0000","last_active":"2024-09-19T07:55:47.800604+0000","last_peered":"2024-09-19T07:55:47.800604+0000","last_clean":"2024-09-19T07:55:47.800604+0000","last_became_active":"2024-09-19T07:55:46.819203+0000","last_became_peered":"2024-09-19T07:55:46.819203+0000","last_unstale":"2024-09-19T07:55:47.800604+0000","last_undegraded":"2024-09-19T07:55:47.800604+0000","last_fullsized":"2024-09-19T07:55:47.800604+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:59.757492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800582+0000","last_change":"2024-09-19T07:55:46.804795+0000","last_active":"2024-09-19T07:55:47.800582+0000","last_peered":"2024-09-19T07:55:47.800582+0000","last_clean":"2024-09-19T07:55:47.800582+0000","last_became_active":"2024-09-19T07:55:46.804489+0000","last_became_peered":"2024-09-19T07:55:46.804489+0000","last_unstale":"2024-09-19T07:55:47.800582+0000","last_undegraded":"2024-09-19T07:55:47.800582+0000","last_fullsized":"2024-09-19T07:55:47.800582+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:18.647644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.801046+0000","last_change":"2024-09-19T07:55:46.810344+0000","last_active":"2024-09-19T07:55:47.801046+0000","last_peered":"2024-09-19T07:55:47.801046+0000","last_clean":"2024-09-19T07:55:47.801046+0000","last_became_active":"2024-09-19T07:55:46.810035+0000","last_became_peered":"2024-09-19T07:55:46.810035+0000","last_unstale":"2024-09-19T07:55:47.801046+0000","last_undegraded":"2024-09-19T07:55:47.801046+0000","last_fullsized":"2024-09-19T07:55:47.801046+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:29.247081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800997+0000","last_change":"2024-09-19T07:55:46.817698+0000","last_active":"2024-09-19T07:55:47.800997+0000","last_peered":"2024-09-19T07:55:47.800997+0000","last_clean":"2024-09-19T07:55:47.800997+0000","last_became_active":"2024-09-19T07:55:46.817376+0000","last_became_peered":"2024-09-19T07:55:46.817376+0000","last_unstale":"2024-09-19T07:55:47.800997+0000","last_undegraded":"2024-09-19T07:55:47.800997+0000","last_fullsized":"2024-09-19T07:55:47.800997+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:07.805087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806932+0000","last_change":"2024-09-19T07:55:46.819681+0000","last_active":"2024-09-19T07:55:47.806932+0000","last_peered":"2024-09-19T07:55:47.806932+0000","last_clean":"2024-09-19T07:55:47.806932+0000","last_became_active":"2024-09-19T07:55:46.819379+0000","last_became_peered":"2024-09-19T07:55:46.819379+0000","last_unstale":"2024-09-19T07:55:47.806932+0000","last_undegraded":"2024-09-19T07:55:47.806932+0000","last_fullsized":"2024-09-19T07:55:47.806932+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:37:45.202649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590306+0000","last_change":"2024-09-19T07:55:46.814981+0000","last_active":"2024-09-19T07:55:48.590306+0000","last_peered":"2024-09-19T07:55:48.590306+0000","last_clean":"2024-09-19T07:55:48.590306+0000","last_became_active":"2024-09-19T07:55:46.814690+0000","last_became_peered":"2024-09-19T07:55:46.814690+0000","last_unstale":"2024-09-19T07:55:48.590306+0000","last_undegraded":"2024-09-19T07:55:48.590306+0000","last_fullsized":"2024-09-19T07:55:48.590306+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:59.757492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827703+0000","last_change":"2024-09-19T07:55:46.811483+0000","last_active":"2024-09-19T07:55:47.827703+0000","last_peered":"2024-09-19T07:55:47.827703+0000","last_clean":"2024-09-19T07:55:47.827703+0000","last_became_active":"2024-09-19T07:55:46.811206+0000","last_became_peered":"2024-09-19T07:55:46.811206+0000","last_unstale":"2024-09-19T07:55:47.827703+0000","last_undegraded":"2024-09-19T07:55:47.827703+0000","last_fullsized":"2024-09-19T07:55:47.827703+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:47.785611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.589975+0000","last_change":"2024-09-19T07:55:46.822810+0000","last_active":"2024-09-19T07:55:48.589975+0000","last_peered":"2024-09-19T07:55:48.589975+0000","last_clean":"2024-09-19T07:55:48.589975+0000","last_became_active":"2024-09-19T07:55:46.822536+0000","last_became_peered":"2024-09-19T07:55:46.822536+0000","last_unstale":"2024-09-19T07:55:48.589975+0000","last_undegraded":"2024-09-19T07:55:48.589975+0000","last_fullsized":"2024-09-19T07:55:48.589975+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:07.805087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806082+0000","last_change":"2024-09-19T07:55:46.815322+0000","last_active":"2024-09-19T07:55:47.806082+0000","last_peered":"2024-09-19T07:55:47.806082+0000","last_clean":"2024-09-19T07:55:47.806082+0000","last_became_active":"2024-09-19T07:55:46.815056+0000","last_became_peered":"2024-09-19T07:55:46.815056+0000","last_unstale":"2024-09-19T07:55:47.806082+0000","last_undegraded":"2024-09-19T07:55:47.806082+0000","last_fullsized":"2024-09-19T07:55:47.806082+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:14.739369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806011+0000","last_change":"2024-09-19T07:55:46.813810+0000","last_active":"2024-09-19T07:55:47.806011+0000","last_peered":"2024-09-19T07:55:47.806011+0000","last_clean":"2024-09-19T07:55:47.806011+0000","last_became_active":"2024-09-19T07:55:46.812769+0000","last_became_peered":"2024-09-19T07:55:46.812769+0000","last_unstale":"2024-09-19T07:55:47.806011+0000","last_undegraded":"2024-09-19T07:55:47.806011+0000","last_fullsized":"2024-09-19T07:55:47.806011+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:05:26.103943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.801098+0000","last_change":"2024-09-19T07:55:46.806494+0000","last_active":"2024-09-19T07:55:47.801098+0000","last_peered":"2024-09-19T07:55:47.801098+0000","last_clean":"2024-09-19T07:55:47.801098+0000","last_became_active":"2024-09-19T07:55:46.806196+0000","last_became_peered":"2024-09-19T07:55:46.806196+0000","last_unstale":"2024-09-19T07:55:47.801098+0000","last_undegraded":"2024-09-19T07:55:47.801098+0000","last_fullsized":"2024-09-19T07:55:47.801098+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:23.021974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590830+0000","last_change":"2024-09-19T07:55:46.817297+0000","last_active":"2024-09-19T07:55:48.590830+0000","last_peered":"2024-09-19T07:55:48.590830+0000","last_clean":"2024-09-19T07:55:48.590830+0000","last_became_active":"2024-09-19T07:55:46.817014+0000","last_became_peered":"2024-09-19T07:55:46.817014+0000","last_unstale":"2024-09-19T07:55:48.590830+0000","last_undegraded":"2024-09-19T07:55:48.590830+0000","last_fullsized":"2024-09-19T07:55:48.590830+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:35.797895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806915+0000","last_change":"2024-09-19T07:55:46.808729+0000","last_active":"2024-09-19T07:55:47.806915+0000","last_peered":"2024-09-19T07:55:47.806915+0000","last_clean":"2024-09-19T07:55:47.806915+0000","last_became_active":"2024-09-19T07:55:46.808438+0000","last_became_peered":"2024-09-19T07:55:46.808438+0000","last_unstale":"2024-09-19T07:55:47.806915+0000","last_undegraded":"2024-09-19T07:55:47.806915+0000","last_fullsized":"2024-09-19T07:55:47.806915+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:18.647644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.791579+0000","last_change":"2024-09-19T07:55:45.791579+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.791579+0000","last_undegraded":"2024-09-19T07:55:45.791579+0000","last_fullsized":"2024-09-19T07:55:45.791579+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827721+0000","last_change":"2024-09-19T07:55:46.817878+0000","last_active":"2024-09-19T07:55:47.827721+0000","last_peered":"2024-09-19T07:55:47.827721+0000","last_clean":"2024-09-19T07:55:47.827721+0000","last_became_active":"2024-09-19T07:55:46.817518+0000","last_became_peered":"2024-09-19T07:55:46.817518+0000","last_unstale":"2024-09-19T07:55:47.827721+0000","last_undegraded":"2024-09-19T07:55:47.827721+0000","last_fullsized":"2024-09-19T07:55:47.827721+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:07.791830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590116+0000","last_change":"2024-09-19T07:55:46.818390+0000","last_active":"2024-09-19T07:55:48.590116+0000","last_peered":"2024-09-19T07:55:48.590116+0000","last_clean":"2024-09-19T07:55:48.590116+0000","last_became_active":"2024-09-19T07:55:46.817865+0000","last_became_peered":"2024-09-19T07:55:46.817865+0000","last_unstale":"2024-09-19T07:55:48.590116+0000","last_undegraded":"2024-09-19T07:55:48.590116+0000","last_fullsized":"2024-09-19T07:55:48.590116+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:02:49.871105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.589704+0000","last_change":"2024-09-19T07:55:46.820984+0000","last_active":"2024-09-19T07:55:48.589704+0000","last_peered":"2024-09-19T07:55:48.589704+0000","last_clean":"2024-09-19T07:55:48.589704+0000","last_became_active":"2024-09-19T07:55:46.816998+0000","last_became_peered":"2024-09-19T07:55:46.816998+0000","last_unstale":"2024-09-19T07:55:48.589704+0000","last_undegraded":"2024-09-19T07:55:48.589704+0000","last_fullsized":"2024-09-19T07:55:48.589704+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:55.546951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806365+0000","last_change":"2024-09-19T07:55:46.812552+0000","last_active":"2024-09-19T07:55:47.806365+0000","last_peered":"2024-09-19T07:55:47.806365+0000","last_clean":"2024-09-19T07:55:47.806365+0000","last_became_active":"2024-09-19T07:55:46.812258+0000","last_became_peered":"2024-09-19T07:55:46.812258+0000","last_unstale":"2024-09-19T07:55:47.806365+0000","last_undegraded":"2024-09-19T07:55:47.806365+0000","last_fullsized":"2024-09-19T07:55:47.806365+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:50.581777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827831+0000","last_change":"2024-09-19T07:55:46.819719+0000","last_active":"2024-09-19T07:55:47.827831+0000","last_peered":"2024-09-19T07:55:47.827831+0000","last_clean":"2024-09-19T07:55:47.827831+0000","last_became_active":"2024-09-19T07:55:46.811669+0000","last_became_peered":"2024-09-19T07:55:46.811669+0000","last_unstale":"2024-09-19T07:55:47.827831+0000","last_undegraded":"2024-09-19T07:55:47.827831+0000","last_fullsized":"2024-09-19T07:55:47.827831+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:40.791876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.790719+0000","last_change":"2024-09-19T07:55:45.790719+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.790719+0000","last_undegraded":"2024-09-19T07:55:45.790719+0000","last_fullsized":"2024-09-19T07:55:45.790719+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590414+0000","last_change":"2024-09-19T07:55:46.820554+0000","last_active":"2024-09-19T07:55:48.590414+0000","last_peered":"2024-09-19T07:55:48.590414+0000","last_clean":"2024-09-19T07:55:48.590414+0000","last_became_active":"2024-09-19T07:55:46.820274+0000","last_became_peered":"2024-09-19T07:55:46.820274+0000","last_unstale":"2024-09-19T07:55:48.590414+0000","last_undegraded":"2024-09-19T07:55:48.590414+0000","last_fullsized":"2024-09-19T07:55:48.590414+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:58.971945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827493+0000","last_change":"2024-09-19T07:55:46.818518+0000","last_active":"2024-09-19T07:55:47.827493+0000","last_peered":"2024-09-19T07:55:47.827493+0000","last_clean":"2024-09-19T07:55:47.827493+0000","last_became_active":"2024-09-19T07:55:46.818264+0000","last_became_peered":"2024-09-19T07:55:46.818264+0000","last_unstale":"2024-09-19T07:55:47.827493+0000","last_undegraded":"2024-09-19T07:55:47.827493+0000","last_fullsized":"2024-09-19T07:55:47.827493+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:53.813959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590685+0000","last_change":"2024-09-19T07:55:46.810596+0000","last_active":"2024-09-19T07:55:48.590685+0000","last_peered":"2024-09-19T07:55:48.590685+0000","last_clean":"2024-09-19T07:55:48.590685+0000","last_became_active":"2024-09-19T07:55:46.810298+0000","last_became_peered":"2024-09-19T07:55:46.810298+0000","last_unstale":"2024-09-19T07:55:48.590685+0000","last_undegraded":"2024-09-19T07:55:48.590685+0000","last_fullsized":"2024-09-19T07:55:48.590685+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:55.546951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590292+0000","last_change":"2024-09-19T07:55:46.817593+0000","last_active":"2024-09-19T07:55:48.590292+0000","last_peered":"2024-09-19T07:55:48.590292+0000","last_clean":"2024-09-19T07:55:48.590292+0000","last_became_active":"2024-09-19T07:55:46.817119+0000","last_became_peered":"2024-09-19T07:55:46.817119+0000","last_unstale":"2024-09-19T07:55:48.590292+0000","last_undegraded":"2024-09-19T07:55:48.590292+0000","last_fullsized":"2024-09-19T07:55:48.590292+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:03.536883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806250+0000","last_change":"2024-09-19T07:55:46.814871+0000","last_active":"2024-09-19T07:55:47.806250+0000","last_peered":"2024-09-19T07:55:47.806250+0000","last_clean":"2024-09-19T07:55:47.806250+0000","last_became_active":"2024-09-19T07:55:46.814602+0000","last_became_peered":"2024-09-19T07:55:46.814602+0000","last_unstale":"2024-09-19T07:55:47.806250+0000","last_undegraded":"2024-09-19T07:55:47.806250+0000","last_fullsized":"2024-09-19T07:55:47.806250+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:38.643240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590672+0000","last_change":"2024-09-19T07:55:46.818472+0000","last_active":"2024-09-19T07:55:48.590672+0000","last_peered":"2024-09-19T07:55:48.590672+0000","last_clean":"2024-09-19T07:55:48.590672+0000","last_became_active":"2024-09-19T07:55:46.818222+0000","last_became_peered":"2024-09-19T07:55:46.818222+0000","last_unstale":"2024-09-19T07:55:48.590672+0000","last_undegraded":"2024-09-19T07:55:48.590672+0000","last_fullsized":"2024-09-19T07:55:48.590672+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:33.322451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590232+0000","last_change":"2024-09-19T07:55:46.815707+0000","last_active":"2024-09-19T07:55:48.590232+0000","last_peered":"2024-09-19T07:55:48.590232+0000","last_clean":"2024-09-19T07:55:48.590232+0000","last_became_active":"2024-09-19T07:55:46.815207+0000","last_became_peered":"2024-09-19T07:55:46.815207+0000","last_unstale":"2024-09-19T07:55:48.590232+0000","last_undegraded":"2024-09-19T07:55:48.590232+0000","last_fullsized":"2024-09-19T07:55:48.590232+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:18.647644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"151'4","reported_seq":18,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807621+0000","last_change":"2024-09-19T07:55:45.799741+0000","last_active":"2024-09-19T07:55:47.807621+0000","last_peered":"2024-09-19T07:55:47.807621+0000","last_clean":"2024-09-19T07:55:47.807621+0000","last_became_active":"2024-09-19T07:55:45.799491+0000","last_became_peered":"2024-09-19T07:55:45.799491+0000","last_unstale":"2024-09-19T07:55:47.807621+0000","last_undegraded":"2024-09-19T07:55:47.807621+0000","last_fullsized":"2024-09-19T07:55:47.807621+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:34.796467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827377+0000","last_change":"2024-09-19T07:55:46.813694+0000","last_active":"2024-09-19T07:55:47.827377+0000","last_peered":"2024-09-19T07:55:47.827377+0000","last_clean":"2024-09-19T07:55:47.827377+0000","last_became_active":"2024-09-19T07:55:46.813395+0000","last_became_peered":"2024-09-19T07:55:46.813395+0000","last_unstale":"2024-09-19T07:55:47.827377+0000","last_undegraded":"2024-09-19T07:55:47.827377+0000","last_fullsized":"2024-09-19T07:55:47.827377+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:46:54.731529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"151'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590239+0000","last_change":"2024-09-19T07:55:45.805741+0000","last_active":"2024-09-19T07:55:48.590239+0000","last_peered":"2024-09-19T07:55:48.590239+0000","last_clean":"2024-09-19T07:55:48.590239+0000","last_became_active":"2024-09-19T07:55:45.805455+0000","last_became_peered":"2024-09-19T07:55:45.805455+0000","last_unstale":"2024-09-19T07:55:48.590239+0000","last_undegraded":"2024-09-19T07:55:48.590239+0000","last_fullsized":"2024-09-19T07:55:48.590239+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:37.641813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.589999+0000","last_change":"2024-09-19T07:55:46.823361+0000","last_active":"2024-09-19T07:55:48.589999+0000","last_peered":"2024-09-19T07:55:48.589999+0000","last_clean":"2024-09-19T07:55:48.589999+0000","last_became_active":"2024-09-19T07:55:46.823078+0000","last_became_peered":"2024-09-19T07:55:46.823078+0000","last_unstale":"2024-09-19T07:55:48.589999+0000","last_undegraded":"2024-09-19T07:55:48.589999+0000","last_fullsized":"2024-09-19T07:55:48.589999+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:29.247081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"151'2","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807032+0000","last_change":"2024-09-19T07:55:45.794756+0000","last_active":"2024-09-19T07:55:47.807032+0000","last_peered":"2024-09-19T07:55:47.807032+0000","last_clean":"2024-09-19T07:55:47.807032+0000","last_became_active":"2024-09-19T07:55:45.794448+0000","last_became_peered":"2024-09-19T07:55:45.794448+0000","last_unstale":"2024-09-19T07:55:47.807032+0000","last_undegraded":"2024-09-19T07:55:47.807032+0000","last_fullsized":"2024-09-19T07:55:47.807032+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:19.674806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590854+0000","last_change":"2024-09-19T07:55:46.815448+0000","last_active":"2024-09-19T07:55:48.590854+0000","last_peered":"2024-09-19T07:55:48.590854+0000","last_clean":"2024-09-19T07:55:48.590854+0000","last_became_active":"2024-09-19T07:55:46.815181+0000","last_became_peered":"2024-09-19T07:55:46.815181+0000","last_unstale":"2024-09-19T07:55:48.590854+0000","last_undegraded":"2024-09-19T07:55:48.590854+0000","last_fullsized":"2024-09-19T07:55:48.590854+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:29.247081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-19T07:55:45.801899+0000","last_change":"2024-09-19T07:55:45.800564+0000","last_active":"2024-09-19T07:55:45.801899+0000","last_peered":"2024-09-19T07:55:45.801899+0000","last_clean":"2024-09-19T07:55:45.801899+0000","last_became_active":"2024-09-19T07:55:45.800078+0000","last_became_peered":"2024-09-19T07:55:45.800078+0000","last_unstale":"2024-09-19T07:55:45.801899+0000","last_undegraded":"2024-09-19T07:55:45.801899+0000","last_fullsized":"2024-09-19T07:55:45.801899+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:18:15.498014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800845+0000","last_change":"2024-09-19T07:55:46.818421+0000","last_active":"2024-09-19T07:55:47.800845+0000","last_peered":"2024-09-19T07:55:47.800845+0000","last_clean":"2024-09-19T07:55:47.800845+0000","last_became_active":"2024-09-19T07:55:46.818140+0000","last_became_peered":"2024-09-19T07:55:46.818140+0000","last_unstale":"2024-09-19T07:55:47.800845+0000","last_undegraded":"2024-09-19T07:55:47.800845+0000","last_fullsized":"2024-09-19T07:55:47.800845+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:20.053063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"151'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800833+0000","last_change":"2024-09-19T07:55:45.805798+0000","last_active":"2024-09-19T07:55:47.800833+0000","last_peered":"2024-09-19T07:55:47.800833+0000","last_clean":"2024-09-19T07:55:47.800833+0000","last_became_active":"2024-09-19T07:55:45.805511+0000","last_became_peered":"2024-09-19T07:55:45.805511+0000","last_unstale":"2024-09-19T07:55:47.800833+0000","last_undegraded":"2024-09-19T07:55:47.800833+0000","last_fullsized":"2024-09-19T07:55:47.800833+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:18.474678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.790402+0000","last_change":"2024-09-19T07:55:45.790402+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.790402+0000","last_undegraded":"2024-09-19T07:55:45.790402+0000","last_fullsized":"2024-09-19T07:55:45.790402+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"151'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590065+0000","last_change":"2024-09-19T07:55:45.803999+0000","last_active":"2024-09-19T07:55:48.590065+0000","last_peered":"2024-09-19T07:55:48.590065+0000","last_clean":"2024-09-19T07:55:48.590065+0000","last_became_active":"2024-09-19T07:55:45.803710+0000","last_became_peered":"2024-09-19T07:55:45.803710+0000","last_unstale":"2024-09-19T07:55:48.590065+0000","last_undegraded":"2024-09-19T07:55:48.590065+0000","last_fullsized":"2024-09-19T07:55:48.590065+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:07.119888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.790244+0000","last_change":"2024-09-19T07:55:45.790244+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.790244+0000","last_undegraded":"2024-09-19T07:55:45.790244+0000","last_fullsized":"2024-09-19T07:55:45.790244+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-19T07:55:45.805997+0000","last_change":"2024-09-19T07:55:45.804265+0000","last_active":"2024-09-19T07:55:45.805997+0000","last_peered":"2024-09-19T07:55:45.805997+0000","last_clean":"2024-09-19T07:55:45.805997+0000","last_became_active":"2024-09-19T07:55:45.804000+0000","last_became_peered":"2024-09-19T07:55:45.804000+0000","last_unstale":"2024-09-19T07:55:45.805997+0000","last_undegraded":"2024-09-19T07:55:45.805997+0000","last_fullsized":"2024-09-19T07:55:45.805997+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:00.654293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807545+0000","last_change":"2024-09-19T07:55:46.804640+0000","last_active":"2024-09-19T07:55:47.807545+0000","last_peered":"2024-09-19T07:55:47.807545+0000","last_clean":"2024-09-19T07:55:47.807545+0000","last_became_active":"2024-09-19T07:55:46.804353+0000","last_became_peered":"2024-09-19T07:55:46.804353+0000","last_unstale":"2024-09-19T07:55:47.807545+0000","last_undegraded":"2024-09-19T07:55:47.807545+0000","last_fullsized":"2024-09-19T07:55:47.807545+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:54.973660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800706+0000","last_change":"2024-09-19T07:55:45.797824+0000","last_active":"2024-09-19T07:55:47.800706+0000","last_peered":"2024-09-19T07:55:47.800706+0000","last_clean":"2024-09-19T07:55:47.800706+0000","last_became_active":"2024-09-19T07:55:45.797559+0000","last_became_peered":"2024-09-19T07:55:45.797559+0000","last_unstale":"2024-09-19T07:55:47.800706+0000","last_undegraded":"2024-09-19T07:55:47.800706+0000","last_fullsized":"2024-09-19T07:55:47.800706+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:48.369553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590859+0000","last_change":"2024-09-19T07:55:46.815474+0000","last_active":"2024-09-19T07:55:48.590859+0000","last_peered":"2024-09-19T07:55:48.590859+0000","last_clean":"2024-09-19T07:55:48.590859+0000","last_became_active":"2024-09-19T07:55:46.814841+0000","last_became_peered":"2024-09-19T07:55:46.814841+0000","last_unstale":"2024-09-19T07:55:48.590859+0000","last_undegraded":"2024-09-19T07:55:48.590859+0000","last_fullsized":"2024-09-19T07:55:48.590859+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:23.021974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827717+0000","last_change":"2024-09-19T07:55:45.800048+0000","last_active":"2024-09-19T07:55:47.827717+0000","last_peered":"2024-09-19T07:55:47.827717+0000","last_clean":"2024-09-19T07:55:47.827717+0000","last_became_active":"2024-09-19T07:55:45.799749+0000","last_became_peered":"2024-09-19T07:55:45.799749+0000","last_unstale":"2024-09-19T07:55:47.827717+0000","last_undegraded":"2024-09-19T07:55:47.827717+0000","last_fullsized":"2024-09-19T07:55:47.827717+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:42.846081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827407+0000","last_change":"2024-09-19T07:55:46.819081+0000","last_active":"2024-09-19T07:55:47.827407+0000","last_peered":"2024-09-19T07:55:47.827407+0000","last_clean":"2024-09-19T07:55:47.827407+0000","last_became_active":"2024-09-19T07:55:46.818785+0000","last_became_peered":"2024-09-19T07:55:46.818785+0000","last_unstale":"2024-09-19T07:55:47.827407+0000","last_undegraded":"2024-09-19T07:55:47.827407+0000","last_fullsized":"2024-09-19T07:55:47.827407+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:48.362583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"151'3","reported_seq":17,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807316+0000","last_change":"2024-09-19T07:55:45.792189+0000","last_active":"2024-09-19T07:55:47.807316+0000","last_peered":"2024-09-19T07:55:47.807316+0000","last_clean":"2024-09-19T07:55:47.807316+0000","last_became_active":"2024-09-19T07:55:45.791906+0000","last_became_peered":"2024-09-19T07:55:45.791906+0000","last_unstale":"2024-09-19T07:55:47.807316+0000","last_undegraded":"2024-09-19T07:55:47.807316+0000","last_fullsized":"2024-09-19T07:55:47.807316+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:32.321023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.791181+0000","last_change":"2024-09-19T07:55:45.791181+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.791181+0000","last_undegraded":"2024-09-19T07:55:45.791181+0000","last_fullsized":"2024-09-19T07:55:45.791181+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"151'6","reported_seq":20,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827602+0000","last_change":"2024-09-19T07:55:45.802798+0000","last_active":"2024-09-19T07:55:47.827602+0000","last_peered":"2024-09-19T07:55:47.827602+0000","last_clean":"2024-09-19T07:55:47.827602+0000","last_became_active":"2024-09-19T07:55:45.802458+0000","last_became_peered":"2024-09-19T07:55:45.802458+0000","last_unstale":"2024-09-19T07:55:47.827602+0000","last_undegraded":"2024-09-19T07:55:47.827602+0000","last_fullsized":"2024-09-19T07:55:47.827602+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:08.281321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590621+0000","last_change":"2024-09-19T07:55:46.817885+0000","last_active":"2024-09-19T07:55:48.590621+0000","last_peered":"2024-09-19T07:55:48.590621+0000","last_clean":"2024-09-19T07:55:48.590621+0000","last_became_active":"2024-09-19T07:55:46.817566+0000","last_became_peered":"2024-09-19T07:55:46.817566+0000","last_unstale":"2024-09-19T07:55:48.590621+0000","last_undegraded":"2024-09-19T07:55:48.590621+0000","last_fullsized":"2024-09-19T07:55:48.590621+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:12.687039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"151'2","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590265+0000","last_change":"2024-09-19T07:55:45.801524+0000","last_active":"2024-09-19T07:55:48.590265+0000","last_peered":"2024-09-19T07:55:48.590265+0000","last_clean":"2024-09-19T07:55:48.590265+0000","last_became_active":"2024-09-19T07:55:45.801161+0000","last_became_peered":"2024-09-19T07:55:45.801161+0000","last_unstale":"2024-09-19T07:55:48.590265+0000","last_undegraded":"2024-09-19T07:55:48.590265+0000","last_fullsized":"2024-09-19T07:55:48.590265+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:13.737941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800893+0000","last_change":"2024-09-19T07:55:46.817083+0000","last_active":"2024-09-19T07:55:47.800893+0000","last_peered":"2024-09-19T07:55:47.800893+0000","last_clean":"2024-09-19T07:55:47.800893+0000","last_became_active":"2024-09-19T07:55:46.816804+0000","last_became_peered":"2024-09-19T07:55:46.816804+0000","last_unstale":"2024-09-19T07:55:47.800893+0000","last_undegraded":"2024-09-19T07:55:47.800893+0000","last_fullsized":"2024-09-19T07:55:47.800893+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:55.546951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827440+0000","last_change":"2024-09-19T07:55:45.801135+0000","last_active":"2024-09-19T07:55:47.827440+0000","last_peered":"2024-09-19T07:55:47.827440+0000","last_clean":"2024-09-19T07:55:47.827440+0000","last_became_active":"2024-09-19T07:55:45.800837+0000","last_became_peered":"2024-09-19T07:55:45.800837+0000","last_unstale":"2024-09-19T07:55:47.827440+0000","last_undegraded":"2024-09-19T07:55:47.827440+0000","last_fullsized":"2024-09-19T07:55:47.827440+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:30:46.437440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807292+0000","last_change":"2024-09-19T07:55:46.813235+0000","last_active":"2024-09-19T07:55:47.807292+0000","last_peered":"2024-09-19T07:55:47.807292+0000","last_clean":"2024-09-19T07:55:47.807292+0000","last_became_active":"2024-09-19T07:55:46.812961+0000","last_became_peered":"2024-09-19T07:55:46.812961+0000","last_unstale":"2024-09-19T07:55:47.807292+0000","last_undegraded":"2024-09-19T07:55:47.807292+0000","last_fullsized":"2024-09-19T07:55:47.807292+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:59.757492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806364+0000","last_change":"2024-09-19T07:55:45.804843+0000","last_active":"2024-09-19T07:55:47.806364+0000","last_peered":"2024-09-19T07:55:47.806364+0000","last_clean":"2024-09-19T07:55:47.806364+0000","last_became_active":"2024-09-19T07:55:45.804613+0000","last_became_peered":"2024-09-19T07:55:45.804613+0000","last_unstale":"2024-09-19T07:55:47.806364+0000","last_undegraded":"2024-09-19T07:55:47.806364+0000","last_fullsized":"2024-09-19T07:55:47.806364+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:18.474678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.791295+0000","last_change":"2024-09-19T07:55:45.791295+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.791295+0000","last_undegraded":"2024-09-19T07:55:45.791295+0000","last_fullsized":"2024-09-19T07:55:45.791295+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"151'8","reported_seq":22,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800808+0000","last_change":"2024-09-19T07:55:45.795277+0000","last_active":"2024-09-19T07:55:47.800808+0000","last_peered":"2024-09-19T07:55:47.800808+0000","last_clean":"2024-09-19T07:55:47.800808+0000","last_became_active":"2024-09-19T07:55:45.794876+0000","last_became_peered":"2024-09-19T07:55:45.794876+0000","last_unstale":"2024-09-19T07:55:47.800808+0000","last_undegraded":"2024-09-19T07:55:47.800808+0000","last_fullsized":"2024-09-19T07:55:47.800808+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:50.194217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800585+0000","last_change":"2024-09-19T07:55:46.817868+0000","last_active":"2024-09-19T07:55:47.800585+0000","last_peered":"2024-09-19T07:55:47.800585+0000","last_clean":"2024-09-19T07:55:47.800585+0000","last_became_active":"2024-09-19T07:55:46.817545+0000","last_became_peered":"2024-09-19T07:55:46.817545+0000","last_unstale":"2024-09-19T07:55:47.800585+0000","last_undegraded":"2024-09-19T07:55:47.800585+0000","last_fullsized":"2024-09-19T07:55:47.800585+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:37:45.202649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"151'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827791+0000","last_change":"2024-09-19T07:55:45.797085+0000","last_active":"2024-09-19T07:55:47.827791+0000","last_peered":"2024-09-19T07:55:47.827791+0000","last_clean":"2024-09-19T07:55:47.827791+0000","last_became_active":"2024-09-19T07:55:45.796793+0000","last_became_peered":"2024-09-19T07:55:45.796793+0000","last_unstale":"2024-09-19T07:55:47.827791+0000","last_undegraded":"2024-09-19T07:55:47.827791+0000","last_fullsized":"2024-09-19T07:55:47.827791+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:14.418996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590734+0000","last_change":"2024-09-19T07:55:46.815815+0000","last_active":"2024-09-19T07:55:48.590734+0000","last_peered":"2024-09-19T07:55:48.590734+0000","last_clean":"2024-09-19T07:55:48.590734+0000","last_became_active":"2024-09-19T07:55:46.815502+0000","last_became_peered":"2024-09-19T07:55:46.815502+0000","last_unstale":"2024-09-19T07:55:48.590734+0000","last_undegraded":"2024-09-19T07:55:48.590734+0000","last_fullsized":"2024-09-19T07:55:48.590734+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:36.406278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"151'2","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.801123+0000","last_change":"2024-09-19T07:55:45.803736+0000","last_active":"2024-09-19T07:55:47.801123+0000","last_peered":"2024-09-19T07:55:47.801123+0000","last_clean":"2024-09-19T07:55:47.801123+0000","last_became_active":"2024-09-19T07:55:45.803419+0000","last_became_peered":"2024-09-19T07:55:45.803419+0000","last_unstale":"2024-09-19T07:55:47.801123+0000","last_undegraded":"2024-09-19T07:55:47.801123+0000","last_fullsized":"2024-09-19T07:55:47.801123+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:39:47.947756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806166+0000","last_change":"2024-09-19T07:55:46.812728+0000","last_active":"2024-09-19T07:55:47.806166+0000","last_peered":"2024-09-19T07:55:47.806166+0000","last_clean":"2024-09-19T07:55:47.806166+0000","last_became_active":"2024-09-19T07:55:46.811051+0000","last_became_peered":"2024-09-19T07:55:46.811051+0000","last_unstale":"2024-09-19T07:55:47.806166+0000","last_undegraded":"2024-09-19T07:55:47.806166+0000","last_fullsized":"2024-09-19T07:55:47.806166+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:25.009628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"151'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590475+0000","last_change":"2024-09-19T07:55:45.801739+0000","last_active":"2024-09-19T07:55:48.590475+0000","last_peered":"2024-09-19T07:55:48.590475+0000","last_clean":"2024-09-19T07:55:48.590475+0000","last_became_active":"2024-09-19T07:55:45.800502+0000","last_became_peered":"2024-09-19T07:55:45.800502+0000","last_unstale":"2024-09-19T07:55:48.590475+0000","last_undegraded":"2024-09-19T07:55:48.590475+0000","last_fullsized":"2024-09-19T07:55:48.590475+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:37.112493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806634+0000","last_change":"2024-09-19T07:55:46.813673+0000","last_active":"2024-09-19T07:55:47.806634+0000","last_peered":"2024-09-19T07:55:47.806634+0000","last_clean":"2024-09-19T07:55:47.806634+0000","last_became_active":"2024-09-19T07:55:46.813361+0000","last_became_peered":"2024-09-19T07:55:46.813361+0000","last_unstale":"2024-09-19T07:55:47.806634+0000","last_undegraded":"2024-09-19T07:55:47.806634+0000","last_fullsized":"2024-09-19T07:55:47.806634+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:02:49.871105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800962+0000","last_change":"2024-09-19T07:55:45.805741+0000","last_active":"2024-09-19T07:55:47.800962+0000","last_peered":"2024-09-19T07:55:47.800962+0000","last_clean":"2024-09-19T07:55:47.800962+0000","last_became_active":"2024-09-19T07:55:45.805464+0000","last_became_peered":"2024-09-19T07:55:45.805464+0000","last_unstale":"2024-09-19T07:55:47.800962+0000","last_undegraded":"2024-09-19T07:55:47.800962+0000","last_fullsized":"2024-09-19T07:55:47.800962+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:31.558613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827844+0000","last_change":"2024-09-19T07:55:46.817735+0000","last_active":"2024-09-19T07:55:47.827844+0000","last_peered":"2024-09-19T07:55:47.827844+0000","last_clean":"2024-09-19T07:55:47.827844+0000","last_became_active":"2024-09-19T07:55:46.817466+0000","last_became_peered":"2024-09-19T07:55:46.817466+0000","last_unstale":"2024-09-19T07:55:47.827844+0000","last_undegraded":"2024-09-19T07:55:47.827844+0000","last_fullsized":"2024-09-19T07:55:47.827844+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:52.010816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"151'2","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590349+0000","last_change":"2024-09-19T07:55:45.801502+0000","last_active":"2024-09-19T07:55:48.590349+0000","last_peered":"2024-09-19T07:55:48.590349+0000","last_clean":"2024-09-19T07:55:48.590349+0000","last_became_active":"2024-09-19T07:55:45.800836+0000","last_became_peered":"2024-09-19T07:55:45.800836+0000","last_unstale":"2024-09-19T07:55:48.590349+0000","last_undegraded":"2024-09-19T07:55:48.590349+0000","last_fullsized":"2024-09-19T07:55:48.590349+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:24.008200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590386+0000","last_change":"2024-09-19T07:55:46.825374+0000","last_active":"2024-09-19T07:55:48.590386+0000","last_peered":"2024-09-19T07:55:48.590386+0000","last_clean":"2024-09-19T07:55:48.590386+0000","last_became_active":"2024-09-19T07:55:46.825083+0000","last_became_peered":"2024-09-19T07:55:46.825083+0000","last_unstale":"2024-09-19T07:55:48.590386+0000","last_undegraded":"2024-09-19T07:55:48.590386+0000","last_fullsized":"2024-09-19T07:55:48.590386+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:47.605737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806267+0000","last_change":"2024-09-19T07:55:45.795584+0000","last_active":"2024-09-19T07:55:47.806267+0000","last_peered":"2024-09-19T07:55:47.806267+0000","last_clean":"2024-09-19T07:55:47.806267+0000","last_became_active":"2024-09-19T07:55:45.795283+0000","last_became_peered":"2024-09-19T07:55:45.795283+0000","last_unstale":"2024-09-19T07:55:47.806267+0000","last_undegraded":"2024-09-19T07:55:47.806267+0000","last_fullsized":"2024-09-19T07:55:47.806267+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:07.706302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590213+0000","last_change":"2024-09-19T07:55:46.824701+0000","last_active":"2024-09-19T07:55:48.590213+0000","last_peered":"2024-09-19T07:55:48.590213+0000","last_clean":"2024-09-19T07:55:48.590213+0000","last_became_active":"2024-09-19T07:55:46.824414+0000","last_became_peered":"2024-09-19T07:55:46.824414+0000","last_unstale":"2024-09-19T07:55:48.590213+0000","last_undegraded":"2024-09-19T07:55:48.590213+0000","last_fullsized":"2024-09-19T07:55:48.590213+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:18.795850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-19T07:55:45.798537+0000","last_change":"2024-09-19T07:55:45.797312+0000","last_active":"2024-09-19T07:55:45.798537+0000","last_peered":"2024-09-19T07:55:45.798537+0000","last_clean":"2024-09-19T07:55:45.798537+0000","last_became_active":"2024-09-19T07:55:45.796832+0000","last_became_peered":"2024-09-19T07:55:45.796832+0000","last_unstale":"2024-09-19T07:55:45.798537+0000","last_undegraded":"2024-09-19T07:55:45.798537+0000","last_fullsized":"2024-09-19T07:55:45.798537+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:15.635183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590487+0000","last_change":"2024-09-19T07:55:46.817406+0000","last_active":"2024-09-19T07:55:48.590487+0000","last_peered":"2024-09-19T07:55:48.590487+0000","last_clean":"2024-09-19T07:55:48.590487+0000","last_became_active":"2024-09-19T07:55:46.817128+0000","last_became_peered":"2024-09-19T07:55:46.817128+0000","last_unstale":"2024-09-19T07:55:48.590487+0000","last_undegraded":"2024-09-19T07:55:48.590487+0000","last_fullsized":"2024-09-19T07:55:48.590487+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:50.004806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590346+0000","last_change":"2024-09-19T07:55:45.795162+0000","last_active":"2024-09-19T07:55:48.590346+0000","last_peered":"2024-09-19T07:55:48.590346+0000","last_clean":"2024-09-19T07:55:48.590346+0000","last_became_active":"2024-09-19T07:55:45.794882+0000","last_became_peered":"2024-09-19T07:55:45.794882+0000","last_unstale":"2024-09-19T07:55:48.590346+0000","last_undegraded":"2024-09-19T07:55:48.590346+0000","last_fullsized":"2024-09-19T07:55:48.590346+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:28.624166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.805948+0000","last_change":"2024-09-19T07:55:46.814034+0000","last_active":"2024-09-19T07:55:47.805948+0000","last_peered":"2024-09-19T07:55:47.805948+0000","last_clean":"2024-09-19T07:55:47.805948+0000","last_became_active":"2024-09-19T07:55:46.813762+0000","last_became_peered":"2024-09-19T07:55:46.813762+0000","last_unstale":"2024-09-19T07:55:47.805948+0000","last_undegraded":"2024-09-19T07:55:47.805948+0000","last_fullsized":"2024-09-19T07:55:47.805948+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:38.113921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"151'2","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590339+0000","last_change":"2024-09-19T07:55:45.801205+0000","last_active":"2024-09-19T07:55:48.590339+0000","last_peered":"2024-09-19T07:55:48.590339+0000","last_clean":"2024-09-19T07:55:48.590339+0000","last_became_active":"2024-09-19T07:55:45.800895+0000","last_became_peered":"2024-09-19T07:55:45.800895+0000","last_unstale":"2024-09-19T07:55:48.590339+0000","last_undegraded":"2024-09-19T07:55:48.590339+0000","last_fullsized":"2024-09-19T07:55:48.590339+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:56.753472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800693+0000","last_change":"2024-09-19T07:55:46.819496+0000","last_active":"2024-09-19T07:55:47.800693+0000","last_peered":"2024-09-19T07:55:47.800693+0000","last_clean":"2024-09-19T07:55:47.800693+0000","last_became_active":"2024-09-19T07:55:46.818787+0000","last_became_peered":"2024-09-19T07:55:46.818787+0000","last_unstale":"2024-09-19T07:55:47.800693+0000","last_undegraded":"2024-09-19T07:55:47.800693+0000","last_fullsized":"2024-09-19T07:55:47.800693+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:58.971945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"152'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.828794+0000","last_change":"2024-09-19T07:55:45.800698+0000","last_active":"2024-09-19T07:55:47.828794+0000","last_peered":"2024-09-19T07:55:47.828794+0000","last_clean":"2024-09-19T07:55:47.828794+0000","last_became_active":"2024-09-19T07:55:45.800012+0000","last_became_peered":"2024-09-19T07:55:45.800012+0000","last_unstale":"2024-09-19T07:55:47.828794+0000","last_undegraded":"2024-09-19T07:55:47.828794+0000","last_fullsized":"2024-09-19T07:55:47.828794+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:21.500902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.794831+0000","last_change":"2024-09-19T07:55:45.794831+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.794831+0000","last_undegraded":"2024-09-19T07:55:45.794831+0000","last_fullsized":"2024-09-19T07:55:45.794831+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827580+0000","last_change":"2024-09-19T07:55:45.802021+0000","last_active":"2024-09-19T07:55:47.827580+0000","last_peered":"2024-09-19T07:55:47.827580+0000","last_clean":"2024-09-19T07:55:47.827580+0000","last_became_active":"2024-09-19T07:55:45.801518+0000","last_became_peered":"2024-09-19T07:55:45.801518+0000","last_unstale":"2024-09-19T07:55:47.827580+0000","last_undegraded":"2024-09-19T07:55:47.827580+0000","last_fullsized":"2024-09-19T07:55:47.827580+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:59.000128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800755+0000","last_change":"2024-09-19T07:55:46.812665+0000","last_active":"2024-09-19T07:55:47.800755+0000","last_peered":"2024-09-19T07:55:47.800755+0000","last_clean":"2024-09-19T07:55:47.800755+0000","last_became_active":"2024-09-19T07:55:46.812424+0000","last_became_peered":"2024-09-19T07:55:46.812424+0000","last_unstale":"2024-09-19T07:55:47.800755+0000","last_undegraded":"2024-09-19T07:55:47.800755+0000","last_fullsized":"2024-09-19T07:55:47.800755+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:50.004806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-19T07:55:45.808438+0000","last_change":"2024-09-19T07:55:45.804911+0000","last_active":"2024-09-19T07:55:45.808438+0000","last_peered":"2024-09-19T07:55:45.808438+0000","last_clean":"2024-09-19T07:55:45.808438+0000","last_became_active":"2024-09-19T07:55:45.804578+0000","last_became_peered":"2024-09-19T07:55:45.804578+0000","last_unstale":"2024-09-19T07:55:45.808438+0000","last_undegraded":"2024-09-19T07:55:45.808438+0000","last_fullsized":"2024-09-19T07:55:45.808438+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:04.552288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806888+0000","last_change":"2024-09-19T07:55:46.815931+0000","last_active":"2024-09-19T07:55:47.806888+0000","last_peered":"2024-09-19T07:55:47.806888+0000","last_clean":"2024-09-19T07:55:47.806888+0000","last_became_active":"2024-09-19T07:55:46.815675+0000","last_became_peered":"2024-09-19T07:55:46.815675+0000","last_unstale":"2024-09-19T07:55:47.806888+0000","last_undegraded":"2024-09-19T07:55:47.806888+0000","last_fullsized":"2024-09-19T07:55:47.806888+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:42.354606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806692+0000","last_change":"2024-09-19T07:55:45.803403+0000","last_active":"2024-09-19T07:55:47.806692+0000","last_peered":"2024-09-19T07:55:47.806692+0000","last_clean":"2024-09-19T07:55:47.806692+0000","last_became_active":"2024-09-19T07:55:45.802988+0000","last_became_peered":"2024-09-19T07:55:45.802988+0000","last_unstale":"2024-09-19T07:55:47.806692+0000","last_undegraded":"2024-09-19T07:55:47.806692+0000","last_fullsized":"2024-09-19T07:55:47.806692+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:35.404851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590533+0000","last_change":"2024-09-19T07:55:46.815044+0000","last_active":"2024-09-19T07:55:48.590533+0000","last_peered":"2024-09-19T07:55:48.590533+0000","last_clean":"2024-09-19T07:55:48.590533+0000","last_became_active":"2024-09-19T07:55:46.814746+0000","last_became_peered":"2024-09-19T07:55:46.814746+0000","last_unstale":"2024-09-19T07:55:48.590533+0000","last_undegraded":"2024-09-19T07:55:48.590533+0000","last_fullsized":"2024-09-19T07:55:48.590533+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:20.053063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827507+0000","last_change":"2024-09-19T07:55:45.801889+0000","last_active":"2024-09-19T07:55:47.827507+0000","last_peered":"2024-09-19T07:55:47.827507+0000","last_clean":"2024-09-19T07:55:47.827507+0000","last_became_active":"2024-09-19T07:55:45.801569+0000","last_became_peered":"2024-09-19T07:55:45.801569+0000","last_unstale":"2024-09-19T07:55:47.827507+0000","last_undegraded":"2024-09-19T07:55:47.827507+0000","last_fullsized":"2024-09-19T07:55:47.827507+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:19.834999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827678+0000","last_change":"2024-09-19T07:55:46.816559+0000","last_active":"2024-09-19T07:55:47.827678+0000","last_peered":"2024-09-19T07:55:47.827678+0000","last_clean":"2024-09-19T07:55:47.827678+0000","last_became_active":"2024-09-19T07:55:46.816305+0000","last_became_peered":"2024-09-19T07:55:46.816305+0000","last_unstale":"2024-09-19T07:55:47.827678+0000","last_undegraded":"2024-09-19T07:55:47.827678+0000","last_fullsized":"2024-09-19T07:55:47.827678+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:27.599531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806465+0000","last_change":"2024-09-19T07:55:45.797679+0000","last_active":"2024-09-19T07:55:47.806465+0000","last_peered":"2024-09-19T07:55:47.806465+0000","last_clean":"2024-09-19T07:55:47.806465+0000","last_became_active":"2024-09-19T07:55:45.797383+0000","last_became_peered":"2024-09-19T07:55:45.797383+0000","last_unstale":"2024-09-19T07:55:47.806465+0000","last_undegraded":"2024-09-19T07:55:47.806465+0000","last_fullsized":"2024-09-19T07:55:47.806465+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:28.624166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":150,"state":"creating+peering","last_fresh":"2024-09-19T07:55:45.790864+0000","last_change":"2024-09-19T07:55:45.790864+0000","last_active":"2024-09-19T07:55:45.779621+0000","last_peered":"2024-09-19T07:55:45.779621+0000","last_clean":"2024-09-19T07:55:45.779621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T07:55:45.790864+0000","last_undegraded":"2024-09-19T07:55:45.790864+0000","last_fullsized":"2024-09-19T07:55:45.790864+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.589573+0000","last_change":"2024-09-19T07:55:45.800775+0000","last_active":"2024-09-19T07:55:48.589573+0000","last_peered":"2024-09-19T07:55:48.589573+0000","last_clean":"2024-09-19T07:55:48.589573+0000","last_became_active":"2024-09-19T07:55:45.800484+0000","last_became_peered":"2024-09-19T07:55:45.800484+0000","last_unstale":"2024-09-19T07:55:48.589573+0000","last_undegraded":"2024-09-19T07:55:48.589573+0000","last_fullsized":"2024-09-19T07:55:48.589573+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:35.490586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590087+0000","last_change":"2024-09-19T07:55:46.818224+0000","last_active":"2024-09-19T07:55:48.590087+0000","last_peered":"2024-09-19T07:55:48.590087+0000","last_clean":"2024-09-19T07:55:48.590087+0000","last_became_active":"2024-09-19T07:55:46.817954+0000","last_became_peered":"2024-09-19T07:55:46.817954+0000","last_unstale":"2024-09-19T07:55:48.590087+0000","last_undegraded":"2024-09-19T07:55:48.590087+0000","last_fullsized":"2024-09-19T07:55:48.590087+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:38.643240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827644+0000","last_change":"2024-09-19T07:55:45.800943+0000","last_active":"2024-09-19T07:55:47.827644+0000","last_peered":"2024-09-19T07:55:47.827644+0000","last_clean":"2024-09-19T07:55:47.827644+0000","last_became_active":"2024-09-19T07:55:45.800428+0000","last_became_peered":"2024-09-19T07:55:45.800428+0000","last_unstale":"2024-09-19T07:55:47.827644+0000","last_undegraded":"2024-09-19T07:55:47.827644+0000","last_fullsized":"2024-09-19T07:55:47.827644+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:40.722531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":408,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590404+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:55:48.590404+0000","last_peered":"2024-09-19T07:55:48.590404+0000","last_clean":"2024-09-19T07:55:48.590404+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:55:48.590404+0000","last_undegraded":"2024-09-19T07:55:48.590404+0000","last_fullsized":"2024-09-19T07:55:48.590404+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509598,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87708,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284132,"statfs":{"total":96636764160,"available":96546951168,"internally_reserved":0,"allocated":2498560,"data_stored":1476936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":87292362},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509597,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105240,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266600,"statfs":{"total":96636764160,"available":96528998400,"internally_reserved":0,"allocated":1904640,"data_stored":886568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509598,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73252,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298588,"statfs":{"total":96636764160,"available":96561754112,"internally_reserved":0,"allocated":1912832,"data_stored":886692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509598,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87152,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284688,"statfs":{"total":96636764160,"available":96547520512,"internally_reserved":0,"allocated":1921024,"data_stored":887332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509597,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105764,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266076,"statfs":{"total":96636764160,"available":96528461824,"internally_reserved":0,"allocated":2506752,"data_stored":1477040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542301,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119088,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252752,"statfs":{"total":96636764160,"available":96514818048,"internally_reserved":0,"allocated":1925120,"data_stored":888784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542301,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83052,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288788,"statfs":{"total":96636764160,"available":96551718912,"internally_reserved":0,"allocated":1916928,"data_stored":888180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575006,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109312,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262528,"statfs":{"total":96636764160,"available":96524828672,"internally_reserved":0,"allocated":1880064,"data_stored":872709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:55:54.631 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:55:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.746+0000 7f1519133640 1 Processor -- start 2024-09-19T07:55:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.746+0000 7f1519133640 1 -- start start 2024-09-19T07:55:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.747+0000 7f1519133640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15140c8d20 0x7f151406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.747+0000 7f1519133640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15140c7140 con 0x7f151406d260 2024-09-19T07:55:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.747+0000 7f1519133640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15140c9100 con 0x7f1514144100 2024-09-19T07:55:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.747+0000 7f1519133640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1514144530 con 0x7f15140c8d20 2024-09-19T07:55:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.747+0000 7f151259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f151406d260 0x7f15141439f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49800/0 (socket says 172.21.15.16:49800) 2024-09-19T07:55:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.747+0000 7f151259c640 1 -- 172.21.15.16:0/4282322428 learned_addr learned my addr 172.21.15.16:0/4282322428 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.747+0000 7f1512d9d640 1 --2- 172.21.15.16:0/4282322428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15140c8d20 0x7f151406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f1511d9b640 1 -- 172.21.15.16:0/4282322428 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 322674504 0 0) 0x7f14fc001110 con 0x7f151406d260 2024-09-19T07:55:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f1511d9b640 1 -- 172.21.15.16:0/4282322428 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1500003520 con 0x7f151406d260 2024-09-19T07:55:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f1511d9b640 1 -- 172.21.15.16:0/4282322428 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2535920337 0 0) 0x7f15140c9100 con 0x7f1514144100 2024-09-19T07:55:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f1511d9b640 1 -- 172.21.15.16:0/4282322428 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14fc001110 con 0x7f1514144100 2024-09-19T07:55:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f1511d9b640 1 -- 172.21.15.16:0/4282322428 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 12102277 0 0) 0x7f1500003520 con 0x7f151406d260 2024-09-19T07:55:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f1511d9b640 1 -- 172.21.15.16:0/4282322428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15140c8d20 msgr2=0x7f151406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f1511d9b640 1 --2- 172.21.15.16:0/4282322428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15140c8d20 0x7f151406cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f1511d9b640 1 -- 172.21.15.16:0/4282322428 >> v1:172.21.15.146:6789/0 conn(0x7f1514144100 legacy=0x7f15140d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f1511d9b640 1 -- 172.21.15.16:0/4282322428 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1514144760 con 0x7f151406d260 2024-09-19T07:55:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f1511d9b640 1 -- 172.21.15.16:0/4282322428 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f14fc001840 con 0x7f151406d260 2024-09-19T07:55:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f1511d9b640 1 -- 172.21.15.16:0/4282322428 >> v1:172.21.15.16:6789/0 conn(0x7f151406d260 legacy=0x7f15141439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f1511d9b640 1 --2- 172.21.15.16:0/4282322428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15140c8d20 0x7f1500003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:54.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f1511d9b640 1 --2- 172.21.15.16:0/4282322428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15000040d0 0x7f1500004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:54.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f1511d9b640 1 --2- 172.21.15.16:0/4282322428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1500004d40 0x7f15000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:54.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f1511d9b640 1 -- 172.21.15.16:0/4282322428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f150000bba0 con 0x7f1500004d40 2024-09-19T07:55:54.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f1511d9b640 1 -- 172.21.15.16:0/4282322428 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f150000bd50 con 0x7f15000040d0 2024-09-19T07:55:54.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f1511d9b640 1 -- 172.21.15.16:0/4282322428 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f150000bf50 con 0x7f15140c8d20 2024-09-19T07:55:54.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f151359e640 1 --2- 172.21.15.16:0/4282322428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1500004d40 0x7f15000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:54.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f1512d9d640 1 --2- 172.21.15.16:0/4282322428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15140c8d20 0x7f1500003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:54.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.748+0000 7f151259c640 1 --2- 172.21.15.16:0/4282322428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15000040d0 0x7f1500004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:54.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.749+0000 7f151359e640 1 -- 172.21.15.16:0/4282322428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15140c8d20 msgr2=0x7f1500003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:54.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.749+0000 7f151359e640 1 --2- 172.21.15.16:0/4282322428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15140c8d20 0x7f1500003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:54.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.749+0000 7f151359e640 1 -- 172.21.15.16:0/4282322428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15000040d0 msgr2=0x7f1500004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:54.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.749+0000 7f151359e640 1 --2- 172.21.15.16:0/4282322428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15000040d0 0x7f1500004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:54.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.749+0000 7f151359e640 1 -- 172.21.15.16:0/4282322428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1500003520 con 0x7f1500004d40 2024-09-19T07:55:54.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.749+0000 7f1512d9d640 1 --2- 172.21.15.16:0/4282322428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15140c8d20 0x7f1500003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:55:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.749+0000 7f151359e640 1 --2- 172.21.15.16:0/4282322428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1500004d40 0x7f15000051c0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f150c0018d0 tx=0x7f1500005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5fe0322d70b3d1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.750+0000 7f1511d9b640 1 -- 172.21.15.16:0/4282322428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c0014f0 con 0x7f1500004d40 2024-09-19T07:55:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.750+0000 7f1511d9b640 1 -- 172.21.15.16:0/4282322428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150c0052d0 con 0x7f1500004d40 2024-09-19T07:55:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.750+0000 7f1511d9b640 1 -- 172.21.15.16:0/4282322428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c02aca0 con 0x7f1500004d40 2024-09-19T07:55:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.750+0000 7f1519133640 1 -- 172.21.15.16:0/4282322428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1500004d40 msgr2=0x7f15000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.750+0000 7f1519133640 1 --2- 172.21.15.16:0/4282322428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1500004d40 0x7f15000051c0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f150c0018d0 tx=0x7f1500005830 comp rx=0 tx=0).stop 2024-09-19T07:55:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.750+0000 7f151359e640 1 -- 172.21.15.16:0/4282322428 reap_dead start 2024-09-19T07:55:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.750+0000 7f1519133640 1 -- 172.21.15.16:0/4282322428 shutdown_connections 2024-09-19T07:55:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.750+0000 7f1519133640 1 -- 172.21.15.16:0/4282322428 >> 172.21.15.16:0/4282322428 conn(0x7f15140b3ed0 msgr2=0x7f15140b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.750+0000 7f1519133640 1 -- 172.21.15.16:0/4282322428 shutdown_connections 2024-09-19T07:55:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.750+0000 7f1519133640 1 -- 172.21.15.16:0/4282322428 wait complete. 2024-09-19T07:55:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.752+0000 7f1519133640 1 Processor -- start 2024-09-19T07:55:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.752+0000 7f1519133640 1 -- start start 2024-09-19T07:55:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.753+0000 7f1519133640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15140b5ba0 0x7f15140b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.753+0000 7f1519133640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15140b64c0 0x7f1514158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.753+0000 7f1519133640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15141587c0 0x7f1514131820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:54.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.753+0000 7f1519133640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1514142200 con 0x7f15140b64c0 2024-09-19T07:55:54.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.753+0000 7f1519133640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1514141960 con 0x7f15141587c0 2024-09-19T07:55:54.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.753+0000 7f1519133640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f151406d6a0 con 0x7f15140b5ba0 2024-09-19T07:55:54.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.753+0000 7f151259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15140b64c0 0x7f1514158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:54.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.753+0000 7f151259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15140b64c0 0x7f1514158280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46810/0 (socket says 172.21.15.16:46810) 2024-09-19T07:55:54.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.753+0000 7f151259c640 1 -- 172.21.15.16:0/4095497006 learned_addr learned my addr 172.21.15.16:0/4095497006 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:54.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.753+0000 7f151359e640 1 --2- 172.21.15.16:0/4095497006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15141587c0 0x7f1514131820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:54.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.753+0000 7f1512d9d640 1 --2- 172.21.15.16:0/4095497006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15140b5ba0 0x7f15140b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:54.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.753+0000 7f151259c640 1 -- 172.21.15.16:0/4095497006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15140b5ba0 msgr2=0x7f15140b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:54.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.753+0000 7f151259c640 1 --2- 172.21.15.16:0/4095497006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15140b5ba0 0x7f15140b5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:54.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.753+0000 7f151259c640 1 -- 172.21.15.16:0/4095497006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15141587c0 msgr2=0x7f1514131820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:54.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.753+0000 7f151259c640 1 --2- 172.21.15.16:0/4095497006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15141587c0 0x7f1514131820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:54.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.753+0000 7f151259c640 1 -- 172.21.15.16:0/4095497006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1514158c40 con 0x7f15140b64c0 2024-09-19T07:55:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.753+0000 7f1512d9d640 1 --2- 172.21.15.16:0/4095497006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15140b5ba0 0x7f15140b5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.753+0000 7f151259c640 1 --2- 172.21.15.16:0/4095497006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15140b64c0 0x7f1514158280 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f14fc004fd0 tx=0x7f14fc0077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.754+0000 7f150b7fe640 1 -- 172.21.15.16:0/4095497006 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc00f990 con 0x7f15140b64c0 2024-09-19T07:55:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.754+0000 7f150b7fe640 1 -- 172.21.15.16:0/4095497006 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14fc018070 con 0x7f15140b64c0 2024-09-19T07:55:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.754+0000 7f150b7fe640 1 -- 172.21.15.16:0/4095497006 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc00fd30 con 0x7f15140b64c0 2024-09-19T07:55:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.754+0000 7f1519133640 1 -- 172.21.15.16:0/4095497006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15140b6b90 con 0x7f15140b64c0 2024-09-19T07:55:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.754+0000 7f1519133640 1 -- 172.21.15.16:0/4095497006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151412f980 con 0x7f15140b64c0 2024-09-19T07:55:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.755+0000 7f1519133640 1 -- 172.21.15.16:0/4095497006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15140b70b0 con 0x7f15140b64c0 2024-09-19T07:55:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.755+0000 7f150b7fe640 1 -- 172.21.15.16:0/4095497006 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f14fc01d460 con 0x7f15140b64c0 2024-09-19T07:55:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.756+0000 7f150b7fe640 1 --2- 172.21.15.16:0/4095497006 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f14f406c2e0 0x7f14f406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.756+0000 7f150b7fe640 1 -- 172.21.15.16:0/4095497006 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f14fc08b7a0 con 0x7f15140b64c0 2024-09-19T07:55:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.756+0000 7f1512d9d640 1 --2- 172.21.15.16:0/4095497006 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f14f406c2e0 0x7f14f406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.756+0000 7f1512d9d640 1 --2- 172.21.15.16:0/4095497006 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f14f406c2e0 0x7f14f406e7a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f15040075e0 tx=0x7f150400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:54.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.760+0000 7f150b7fe640 1 -- 172.21.15.16:0/4095497006 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f15140b70b0 con 0x7f15140b64c0 2024-09-19T07:55:54.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.939+0000 7f1519133640 1 -- 172.21.15.16:0/4095497006 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f15140b6940 con 0x7f14f406c2e0 2024-09-19T07:55:54.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.945+0000 7f150b7fe640 1 -- 172.21.15.16:0/4095497006 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252404 (secure 0 0 0) 0x7f15140b6940 con 0x7f14f406c2e0 2024-09-19T07:55:54.949 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:55:54.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.949+0000 7f1519133640 1 -- 172.21.15.16:0/4095497006 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f14f406c2e0 msgr2=0x7f14f406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:54.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.949+0000 7f1519133640 1 --2- 172.21.15.16:0/4095497006 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f14f406c2e0 0x7f14f406e7a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f15040075e0 tx=0x7f150400b040 comp rx=0 tx=0).stop 2024-09-19T07:55:54.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.949+0000 7f1519133640 1 -- 172.21.15.16:0/4095497006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15140b64c0 msgr2=0x7f1514158280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:54.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.949+0000 7f1519133640 1 --2- 172.21.15.16:0/4095497006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15140b64c0 0x7f1514158280 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f14fc004fd0 tx=0x7f14fc0077a0 comp rx=0 tx=0).stop 2024-09-19T07:55:54.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.949+0000 7f1519133640 1 -- 172.21.15.16:0/4095497006 shutdown_connections 2024-09-19T07:55:54.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.949+0000 7f1519133640 1 --2- 172.21.15.16:0/4095497006 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f14f406c2e0 0x7f14f406e7a0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:54.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.949+0000 7f1519133640 1 --2- 172.21.15.16:0/4095497006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15141587c0 0x7f1514131820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:54.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.949+0000 7f1519133640 1 --2- 172.21.15.16:0/4095497006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15140b64c0 0x7f1514158280 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:54.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.949+0000 7f1519133640 1 --2- 172.21.15.16:0/4095497006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f15140b5ba0 0x7f15140b5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:54.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.949+0000 7f1519133640 1 -- 172.21.15.16:0/4095497006 >> 172.21.15.16:0/4095497006 conn(0x7f15140b3ed0 msgr2=0x7f1514160090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:54.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.949+0000 7f1519133640 1 -- 172.21.15.16:0/4095497006 shutdown_connections 2024-09-19T07:55:54.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:54.949+0000 7f1519133640 1 -- 172.21.15.16:0/4095497006 wait complete. 2024-09-19T07:55:54.960 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":537,"stamp":"2024-09-19T07:55:54.641599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":770592,"kb_used_data":16104,"kb_used_omap":12,"kb_used_meta":754291,"kb_avail":754204128,"statfs":{"total":773094113280,"available":772305027072,"internally_reserved":0,"allocated":16490496,"data_stored":8278100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":772394544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001979"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807561+0000","last_change":"2024-09-19T07:55:46.814625+0000","last_active":"2024-09-19T07:55:47.807561+0000","last_peered":"2024-09-19T07:55:47.807561+0000","last_clean":"2024-09-19T07:55:47.807561+0000","last_became_active":"2024-09-19T07:55:46.814365+0000","last_became_peered":"2024-09-19T07:55:46.814365+0000","last_unstale":"2024-09-19T07:55:47.807561+0000","last_undegraded":"2024-09-19T07:55:47.807561+0000","last_fullsized":"2024-09-19T07:55:47.807561+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:13.254644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590303+0000","last_change":"2024-09-19T07:55:46.811993+0000","last_active":"2024-09-19T07:55:48.590303+0000","last_peered":"2024-09-19T07:55:48.590303+0000","last_clean":"2024-09-19T07:55:48.590303+0000","last_became_active":"2024-09-19T07:55:46.811723+0000","last_became_peered":"2024-09-19T07:55:46.811723+0000","last_unstale":"2024-09-19T07:55:48.590303+0000","last_undegraded":"2024-09-19T07:55:48.590303+0000","last_fullsized":"2024-09-19T07:55:48.590303+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:37:45.202649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807311+0000","last_change":"2024-09-19T07:55:46.818859+0000","last_active":"2024-09-19T07:55:47.807311+0000","last_peered":"2024-09-19T07:55:47.807311+0000","last_clean":"2024-09-19T07:55:47.807311+0000","last_became_active":"2024-09-19T07:55:46.818573+0000","last_became_peered":"2024-09-19T07:55:46.818573+0000","last_unstale":"2024-09-19T07:55:47.807311+0000","last_undegraded":"2024-09-19T07:55:47.807311+0000","last_fullsized":"2024-09-19T07:55:47.807311+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:03.579883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806482+0000","last_change":"2024-09-19T07:55:46.813506+0000","last_active":"2024-09-19T07:55:47.806482+0000","last_peered":"2024-09-19T07:55:47.806482+0000","last_clean":"2024-09-19T07:55:47.806482+0000","last_became_active":"2024-09-19T07:55:46.813249+0000","last_became_peered":"2024-09-19T07:55:46.813249+0000","last_unstale":"2024-09-19T07:55:47.806482+0000","last_undegraded":"2024-09-19T07:55:47.806482+0000","last_fullsized":"2024-09-19T07:55:47.806482+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:06.075192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590102+0000","last_change":"2024-09-19T07:55:46.823203+0000","last_active":"2024-09-19T07:55:48.590102+0000","last_peered":"2024-09-19T07:55:48.590102+0000","last_clean":"2024-09-19T07:55:48.590102+0000","last_became_active":"2024-09-19T07:55:46.822707+0000","last_became_peered":"2024-09-19T07:55:46.822707+0000","last_unstale":"2024-09-19T07:55:48.590102+0000","last_undegraded":"2024-09-19T07:55:48.590102+0000","last_fullsized":"2024-09-19T07:55:48.590102+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:23.021974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590780+0000","last_change":"2024-09-19T07:55:46.813957+0000","last_active":"2024-09-19T07:55:48.590780+0000","last_peered":"2024-09-19T07:55:48.590780+0000","last_clean":"2024-09-19T07:55:48.590780+0000","last_became_active":"2024-09-19T07:55:46.813674+0000","last_became_peered":"2024-09-19T07:55:46.813674+0000","last_unstale":"2024-09-19T07:55:48.590780+0000","last_undegraded":"2024-09-19T07:55:48.590780+0000","last_fullsized":"2024-09-19T07:55:48.590780+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:07.805087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800604+0000","last_change":"2024-09-19T07:55:46.819762+0000","last_active":"2024-09-19T07:55:47.800604+0000","last_peered":"2024-09-19T07:55:47.800604+0000","last_clean":"2024-09-19T07:55:47.800604+0000","last_became_active":"2024-09-19T07:55:46.819203+0000","last_became_peered":"2024-09-19T07:55:46.819203+0000","last_unstale":"2024-09-19T07:55:47.800604+0000","last_undegraded":"2024-09-19T07:55:47.800604+0000","last_fullsized":"2024-09-19T07:55:47.800604+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:59.757492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800582+0000","last_change":"2024-09-19T07:55:46.804795+0000","last_active":"2024-09-19T07:55:47.800582+0000","last_peered":"2024-09-19T07:55:47.800582+0000","last_clean":"2024-09-19T07:55:47.800582+0000","last_became_active":"2024-09-19T07:55:46.804489+0000","last_became_peered":"2024-09-19T07:55:46.804489+0000","last_unstale":"2024-09-19T07:55:47.800582+0000","last_undegraded":"2024-09-19T07:55:47.800582+0000","last_fullsized":"2024-09-19T07:55:47.800582+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:18.647644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.801046+0000","last_change":"2024-09-19T07:55:46.810344+0000","last_active":"2024-09-19T07:55:47.801046+0000","last_peered":"2024-09-19T07:55:47.801046+0000","last_clean":"2024-09-19T07:55:47.801046+0000","last_became_active":"2024-09-19T07:55:46.810035+0000","last_became_peered":"2024-09-19T07:55:46.810035+0000","last_unstale":"2024-09-19T07:55:47.801046+0000","last_undegraded":"2024-09-19T07:55:47.801046+0000","last_fullsized":"2024-09-19T07:55:47.801046+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:29.247081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800997+0000","last_change":"2024-09-19T07:55:46.817698+0000","last_active":"2024-09-19T07:55:47.800997+0000","last_peered":"2024-09-19T07:55:47.800997+0000","last_clean":"2024-09-19T07:55:47.800997+0000","last_became_active":"2024-09-19T07:55:46.817376+0000","last_became_peered":"2024-09-19T07:55:46.817376+0000","last_unstale":"2024-09-19T07:55:47.800997+0000","last_undegraded":"2024-09-19T07:55:47.800997+0000","last_fullsized":"2024-09-19T07:55:47.800997+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:07.805087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806932+0000","last_change":"2024-09-19T07:55:46.819681+0000","last_active":"2024-09-19T07:55:47.806932+0000","last_peered":"2024-09-19T07:55:47.806932+0000","last_clean":"2024-09-19T07:55:47.806932+0000","last_became_active":"2024-09-19T07:55:46.819379+0000","last_became_peered":"2024-09-19T07:55:46.819379+0000","last_unstale":"2024-09-19T07:55:47.806932+0000","last_undegraded":"2024-09-19T07:55:47.806932+0000","last_fullsized":"2024-09-19T07:55:47.806932+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:37:45.202649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590306+0000","last_change":"2024-09-19T07:55:46.814981+0000","last_active":"2024-09-19T07:55:48.590306+0000","last_peered":"2024-09-19T07:55:48.590306+0000","last_clean":"2024-09-19T07:55:48.590306+0000","last_became_active":"2024-09-19T07:55:46.814690+0000","last_became_peered":"2024-09-19T07:55:46.814690+0000","last_unstale":"2024-09-19T07:55:48.590306+0000","last_undegraded":"2024-09-19T07:55:48.590306+0000","last_fullsized":"2024-09-19T07:55:48.590306+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:59.757492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827703+0000","last_change":"2024-09-19T07:55:46.811483+0000","last_active":"2024-09-19T07:55:47.827703+0000","last_peered":"2024-09-19T07:55:47.827703+0000","last_clean":"2024-09-19T07:55:47.827703+0000","last_became_active":"2024-09-19T07:55:46.811206+0000","last_became_peered":"2024-09-19T07:55:46.811206+0000","last_unstale":"2024-09-19T07:55:47.827703+0000","last_undegraded":"2024-09-19T07:55:47.827703+0000","last_fullsized":"2024-09-19T07:55:47.827703+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:47.785611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.589975+0000","last_change":"2024-09-19T07:55:46.822810+0000","last_active":"2024-09-19T07:55:48.589975+0000","last_peered":"2024-09-19T07:55:48.589975+0000","last_clean":"2024-09-19T07:55:48.589975+0000","last_became_active":"2024-09-19T07:55:46.822536+0000","last_became_peered":"2024-09-19T07:55:46.822536+0000","last_unstale":"2024-09-19T07:55:48.589975+0000","last_undegraded":"2024-09-19T07:55:48.589975+0000","last_fullsized":"2024-09-19T07:55:48.589975+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:07.805087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806082+0000","last_change":"2024-09-19T07:55:46.815322+0000","last_active":"2024-09-19T07:55:47.806082+0000","last_peered":"2024-09-19T07:55:47.806082+0000","last_clean":"2024-09-19T07:55:47.806082+0000","last_became_active":"2024-09-19T07:55:46.815056+0000","last_became_peered":"2024-09-19T07:55:46.815056+0000","last_unstale":"2024-09-19T07:55:47.806082+0000","last_undegraded":"2024-09-19T07:55:47.806082+0000","last_fullsized":"2024-09-19T07:55:47.806082+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:14.739369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806011+0000","last_change":"2024-09-19T07:55:46.813810+0000","last_active":"2024-09-19T07:55:47.806011+0000","last_peered":"2024-09-19T07:55:47.806011+0000","last_clean":"2024-09-19T07:55:47.806011+0000","last_became_active":"2024-09-19T07:55:46.812769+0000","last_became_peered":"2024-09-19T07:55:46.812769+0000","last_unstale":"2024-09-19T07:55:47.806011+0000","last_undegraded":"2024-09-19T07:55:47.806011+0000","last_fullsized":"2024-09-19T07:55:47.806011+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:05:26.103943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.801098+0000","last_change":"2024-09-19T07:55:46.806494+0000","last_active":"2024-09-19T07:55:47.801098+0000","last_peered":"2024-09-19T07:55:47.801098+0000","last_clean":"2024-09-19T07:55:47.801098+0000","last_became_active":"2024-09-19T07:55:46.806196+0000","last_became_peered":"2024-09-19T07:55:46.806196+0000","last_unstale":"2024-09-19T07:55:47.801098+0000","last_undegraded":"2024-09-19T07:55:47.801098+0000","last_fullsized":"2024-09-19T07:55:47.801098+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:23.021974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590830+0000","last_change":"2024-09-19T07:55:46.817297+0000","last_active":"2024-09-19T07:55:48.590830+0000","last_peered":"2024-09-19T07:55:48.590830+0000","last_clean":"2024-09-19T07:55:48.590830+0000","last_became_active":"2024-09-19T07:55:46.817014+0000","last_became_peered":"2024-09-19T07:55:46.817014+0000","last_unstale":"2024-09-19T07:55:48.590830+0000","last_undegraded":"2024-09-19T07:55:48.590830+0000","last_fullsized":"2024-09-19T07:55:48.590830+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:35.797895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806915+0000","last_change":"2024-09-19T07:55:46.808729+0000","last_active":"2024-09-19T07:55:47.806915+0000","last_peered":"2024-09-19T07:55:47.806915+0000","last_clean":"2024-09-19T07:55:47.806915+0000","last_became_active":"2024-09-19T07:55:46.808438+0000","last_became_peered":"2024-09-19T07:55:46.808438+0000","last_unstale":"2024-09-19T07:55:47.806915+0000","last_undegraded":"2024-09-19T07:55:47.806915+0000","last_fullsized":"2024-09-19T07:55:47.806915+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:18.647644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.589683+0000","last_change":"2024-09-19T07:55:46.817293+0000","last_active":"2024-09-19T07:55:48.589683+0000","last_peered":"2024-09-19T07:55:48.589683+0000","last_clean":"2024-09-19T07:55:48.589683+0000","last_became_active":"2024-09-19T07:55:46.817004+0000","last_became_peered":"2024-09-19T07:55:46.817004+0000","last_unstale":"2024-09-19T07:55:48.589683+0000","last_undegraded":"2024-09-19T07:55:48.589683+0000","last_fullsized":"2024-09-19T07:55:48.589683+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:45:41.474574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827721+0000","last_change":"2024-09-19T07:55:46.817878+0000","last_active":"2024-09-19T07:55:47.827721+0000","last_peered":"2024-09-19T07:55:47.827721+0000","last_clean":"2024-09-19T07:55:47.827721+0000","last_became_active":"2024-09-19T07:55:46.817518+0000","last_became_peered":"2024-09-19T07:55:46.817518+0000","last_unstale":"2024-09-19T07:55:47.827721+0000","last_undegraded":"2024-09-19T07:55:47.827721+0000","last_fullsized":"2024-09-19T07:55:47.827721+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:07.791830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590116+0000","last_change":"2024-09-19T07:55:46.818390+0000","last_active":"2024-09-19T07:55:48.590116+0000","last_peered":"2024-09-19T07:55:48.590116+0000","last_clean":"2024-09-19T07:55:48.590116+0000","last_became_active":"2024-09-19T07:55:46.817865+0000","last_became_peered":"2024-09-19T07:55:46.817865+0000","last_unstale":"2024-09-19T07:55:48.590116+0000","last_undegraded":"2024-09-19T07:55:48.590116+0000","last_fullsized":"2024-09-19T07:55:48.590116+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:02:49.871105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.589704+0000","last_change":"2024-09-19T07:55:46.820984+0000","last_active":"2024-09-19T07:55:48.589704+0000","last_peered":"2024-09-19T07:55:48.589704+0000","last_clean":"2024-09-19T07:55:48.589704+0000","last_became_active":"2024-09-19T07:55:46.816998+0000","last_became_peered":"2024-09-19T07:55:46.816998+0000","last_unstale":"2024-09-19T07:55:48.589704+0000","last_undegraded":"2024-09-19T07:55:48.589704+0000","last_fullsized":"2024-09-19T07:55:48.589704+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:55.546951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806365+0000","last_change":"2024-09-19T07:55:46.812552+0000","last_active":"2024-09-19T07:55:47.806365+0000","last_peered":"2024-09-19T07:55:47.806365+0000","last_clean":"2024-09-19T07:55:47.806365+0000","last_became_active":"2024-09-19T07:55:46.812258+0000","last_became_peered":"2024-09-19T07:55:46.812258+0000","last_unstale":"2024-09-19T07:55:47.806365+0000","last_undegraded":"2024-09-19T07:55:47.806365+0000","last_fullsized":"2024-09-19T07:55:47.806365+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:50.581777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827831+0000","last_change":"2024-09-19T07:55:46.819719+0000","last_active":"2024-09-19T07:55:47.827831+0000","last_peered":"2024-09-19T07:55:47.827831+0000","last_clean":"2024-09-19T07:55:47.827831+0000","last_became_active":"2024-09-19T07:55:46.811669+0000","last_became_peered":"2024-09-19T07:55:46.811669+0000","last_unstale":"2024-09-19T07:55:47.827831+0000","last_undegraded":"2024-09-19T07:55:47.827831+0000","last_fullsized":"2024-09-19T07:55:47.827831+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:40.791876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590332+0000","last_change":"2024-09-19T07:55:46.814916+0000","last_active":"2024-09-19T07:55:48.590332+0000","last_peered":"2024-09-19T07:55:48.590332+0000","last_clean":"2024-09-19T07:55:48.590332+0000","last_became_active":"2024-09-19T07:55:46.814595+0000","last_became_peered":"2024-09-19T07:55:46.814595+0000","last_unstale":"2024-09-19T07:55:48.590332+0000","last_undegraded":"2024-09-19T07:55:48.590332+0000","last_fullsized":"2024-09-19T07:55:48.590332+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:24:13.517967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590414+0000","last_change":"2024-09-19T07:55:46.820554+0000","last_active":"2024-09-19T07:55:48.590414+0000","last_peered":"2024-09-19T07:55:48.590414+0000","last_clean":"2024-09-19T07:55:48.590414+0000","last_became_active":"2024-09-19T07:55:46.820274+0000","last_became_peered":"2024-09-19T07:55:46.820274+0000","last_unstale":"2024-09-19T07:55:48.590414+0000","last_undegraded":"2024-09-19T07:55:48.590414+0000","last_fullsized":"2024-09-19T07:55:48.590414+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:58.971945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827493+0000","last_change":"2024-09-19T07:55:46.818518+0000","last_active":"2024-09-19T07:55:47.827493+0000","last_peered":"2024-09-19T07:55:47.827493+0000","last_clean":"2024-09-19T07:55:47.827493+0000","last_became_active":"2024-09-19T07:55:46.818264+0000","last_became_peered":"2024-09-19T07:55:46.818264+0000","last_unstale":"2024-09-19T07:55:47.827493+0000","last_undegraded":"2024-09-19T07:55:47.827493+0000","last_fullsized":"2024-09-19T07:55:47.827493+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:53.813959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590685+0000","last_change":"2024-09-19T07:55:46.810596+0000","last_active":"2024-09-19T07:55:48.590685+0000","last_peered":"2024-09-19T07:55:48.590685+0000","last_clean":"2024-09-19T07:55:48.590685+0000","last_became_active":"2024-09-19T07:55:46.810298+0000","last_became_peered":"2024-09-19T07:55:46.810298+0000","last_unstale":"2024-09-19T07:55:48.590685+0000","last_undegraded":"2024-09-19T07:55:48.590685+0000","last_fullsized":"2024-09-19T07:55:48.590685+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:55.546951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590292+0000","last_change":"2024-09-19T07:55:46.817593+0000","last_active":"2024-09-19T07:55:48.590292+0000","last_peered":"2024-09-19T07:55:48.590292+0000","last_clean":"2024-09-19T07:55:48.590292+0000","last_became_active":"2024-09-19T07:55:46.817119+0000","last_became_peered":"2024-09-19T07:55:46.817119+0000","last_unstale":"2024-09-19T07:55:48.590292+0000","last_undegraded":"2024-09-19T07:55:48.590292+0000","last_fullsized":"2024-09-19T07:55:48.590292+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:03.536883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806250+0000","last_change":"2024-09-19T07:55:46.814871+0000","last_active":"2024-09-19T07:55:47.806250+0000","last_peered":"2024-09-19T07:55:47.806250+0000","last_clean":"2024-09-19T07:55:47.806250+0000","last_became_active":"2024-09-19T07:55:46.814602+0000","last_became_peered":"2024-09-19T07:55:46.814602+0000","last_unstale":"2024-09-19T07:55:47.806250+0000","last_undegraded":"2024-09-19T07:55:47.806250+0000","last_fullsized":"2024-09-19T07:55:47.806250+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:38.643240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590672+0000","last_change":"2024-09-19T07:55:46.818472+0000","last_active":"2024-09-19T07:55:48.590672+0000","last_peered":"2024-09-19T07:55:48.590672+0000","last_clean":"2024-09-19T07:55:48.590672+0000","last_became_active":"2024-09-19T07:55:46.818222+0000","last_became_peered":"2024-09-19T07:55:46.818222+0000","last_unstale":"2024-09-19T07:55:48.590672+0000","last_undegraded":"2024-09-19T07:55:48.590672+0000","last_fullsized":"2024-09-19T07:55:48.590672+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:33.322451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590232+0000","last_change":"2024-09-19T07:55:46.815707+0000","last_active":"2024-09-19T07:55:48.590232+0000","last_peered":"2024-09-19T07:55:48.590232+0000","last_clean":"2024-09-19T07:55:48.590232+0000","last_became_active":"2024-09-19T07:55:46.815207+0000","last_became_peered":"2024-09-19T07:55:46.815207+0000","last_unstale":"2024-09-19T07:55:48.590232+0000","last_undegraded":"2024-09-19T07:55:48.590232+0000","last_fullsized":"2024-09-19T07:55:48.590232+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:18.647644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"151'4","reported_seq":18,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807621+0000","last_change":"2024-09-19T07:55:45.799741+0000","last_active":"2024-09-19T07:55:47.807621+0000","last_peered":"2024-09-19T07:55:47.807621+0000","last_clean":"2024-09-19T07:55:47.807621+0000","last_became_active":"2024-09-19T07:55:45.799491+0000","last_became_peered":"2024-09-19T07:55:45.799491+0000","last_unstale":"2024-09-19T07:55:47.807621+0000","last_undegraded":"2024-09-19T07:55:47.807621+0000","last_fullsized":"2024-09-19T07:55:47.807621+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:00:34.796467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827377+0000","last_change":"2024-09-19T07:55:46.813694+0000","last_active":"2024-09-19T07:55:47.827377+0000","last_peered":"2024-09-19T07:55:47.827377+0000","last_clean":"2024-09-19T07:55:47.827377+0000","last_became_active":"2024-09-19T07:55:46.813395+0000","last_became_peered":"2024-09-19T07:55:46.813395+0000","last_unstale":"2024-09-19T07:55:47.827377+0000","last_undegraded":"2024-09-19T07:55:47.827377+0000","last_fullsized":"2024-09-19T07:55:47.827377+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:46:54.731529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"151'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590239+0000","last_change":"2024-09-19T07:55:45.805741+0000","last_active":"2024-09-19T07:55:48.590239+0000","last_peered":"2024-09-19T07:55:48.590239+0000","last_clean":"2024-09-19T07:55:48.590239+0000","last_became_active":"2024-09-19T07:55:45.805455+0000","last_became_peered":"2024-09-19T07:55:45.805455+0000","last_unstale":"2024-09-19T07:55:48.590239+0000","last_undegraded":"2024-09-19T07:55:48.590239+0000","last_fullsized":"2024-09-19T07:55:48.590239+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:37.641813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.589999+0000","last_change":"2024-09-19T07:55:46.823361+0000","last_active":"2024-09-19T07:55:48.589999+0000","last_peered":"2024-09-19T07:55:48.589999+0000","last_clean":"2024-09-19T07:55:48.589999+0000","last_became_active":"2024-09-19T07:55:46.823078+0000","last_became_peered":"2024-09-19T07:55:46.823078+0000","last_unstale":"2024-09-19T07:55:48.589999+0000","last_undegraded":"2024-09-19T07:55:48.589999+0000","last_fullsized":"2024-09-19T07:55:48.589999+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:29.247081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"151'2","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807032+0000","last_change":"2024-09-19T07:55:45.794756+0000","last_active":"2024-09-19T07:55:47.807032+0000","last_peered":"2024-09-19T07:55:47.807032+0000","last_clean":"2024-09-19T07:55:47.807032+0000","last_became_active":"2024-09-19T07:55:45.794448+0000","last_became_peered":"2024-09-19T07:55:45.794448+0000","last_unstale":"2024-09-19T07:55:47.807032+0000","last_undegraded":"2024-09-19T07:55:47.807032+0000","last_fullsized":"2024-09-19T07:55:47.807032+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:19.674806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590854+0000","last_change":"2024-09-19T07:55:46.815448+0000","last_active":"2024-09-19T07:55:48.590854+0000","last_peered":"2024-09-19T07:55:48.590854+0000","last_clean":"2024-09-19T07:55:48.590854+0000","last_became_active":"2024-09-19T07:55:46.815181+0000","last_became_peered":"2024-09-19T07:55:46.815181+0000","last_unstale":"2024-09-19T07:55:48.590854+0000","last_undegraded":"2024-09-19T07:55:48.590854+0000","last_fullsized":"2024-09-19T07:55:48.590854+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:29.247081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"151'2","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.589866+0000","last_change":"2024-09-19T07:55:45.800564+0000","last_active":"2024-09-19T07:55:48.589866+0000","last_peered":"2024-09-19T07:55:48.589866+0000","last_clean":"2024-09-19T07:55:48.589866+0000","last_became_active":"2024-09-19T07:55:45.800078+0000","last_became_peered":"2024-09-19T07:55:45.800078+0000","last_unstale":"2024-09-19T07:55:48.589866+0000","last_undegraded":"2024-09-19T07:55:48.589866+0000","last_fullsized":"2024-09-19T07:55:48.589866+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:20.838299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800845+0000","last_change":"2024-09-19T07:55:46.818421+0000","last_active":"2024-09-19T07:55:47.800845+0000","last_peered":"2024-09-19T07:55:47.800845+0000","last_clean":"2024-09-19T07:55:47.800845+0000","last_became_active":"2024-09-19T07:55:46.818140+0000","last_became_peered":"2024-09-19T07:55:46.818140+0000","last_unstale":"2024-09-19T07:55:47.800845+0000","last_undegraded":"2024-09-19T07:55:47.800845+0000","last_fullsized":"2024-09-19T07:55:47.800845+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:20.053063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"151'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800833+0000","last_change":"2024-09-19T07:55:45.805798+0000","last_active":"2024-09-19T07:55:47.800833+0000","last_peered":"2024-09-19T07:55:47.800833+0000","last_clean":"2024-09-19T07:55:47.800833+0000","last_became_active":"2024-09-19T07:55:45.805511+0000","last_became_peered":"2024-09-19T07:55:45.805511+0000","last_unstale":"2024-09-19T07:55:47.800833+0000","last_undegraded":"2024-09-19T07:55:47.800833+0000","last_fullsized":"2024-09-19T07:55:47.800833+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:18.474678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590082+0000","last_change":"2024-09-19T07:55:46.812838+0000","last_active":"2024-09-19T07:55:48.590082+0000","last_peered":"2024-09-19T07:55:48.590082+0000","last_clean":"2024-09-19T07:55:48.590082+0000","last_became_active":"2024-09-19T07:55:46.812492+0000","last_became_peered":"2024-09-19T07:55:46.812492+0000","last_unstale":"2024-09-19T07:55:48.590082+0000","last_undegraded":"2024-09-19T07:55:48.590082+0000","last_fullsized":"2024-09-19T07:55:48.590082+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:19.096829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"151'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590065+0000","last_change":"2024-09-19T07:55:45.803999+0000","last_active":"2024-09-19T07:55:48.590065+0000","last_peered":"2024-09-19T07:55:48.590065+0000","last_clean":"2024-09-19T07:55:48.590065+0000","last_became_active":"2024-09-19T07:55:45.803710+0000","last_became_peered":"2024-09-19T07:55:45.803710+0000","last_unstale":"2024-09-19T07:55:48.590065+0000","last_undegraded":"2024-09-19T07:55:48.590065+0000","last_fullsized":"2024-09-19T07:55:48.590065+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:07.119888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590432+0000","last_change":"2024-09-19T07:55:46.815536+0000","last_active":"2024-09-19T07:55:48.590432+0000","last_peered":"2024-09-19T07:55:48.590432+0000","last_clean":"2024-09-19T07:55:48.590432+0000","last_became_active":"2024-09-19T07:55:46.815255+0000","last_became_peered":"2024-09-19T07:55:46.815255+0000","last_unstale":"2024-09-19T07:55:48.590432+0000","last_undegraded":"2024-09-19T07:55:48.590432+0000","last_fullsized":"2024-09-19T07:55:48.590432+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:33:20.776395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.589628+0000","last_change":"2024-09-19T07:55:45.804265+0000","last_active":"2024-09-19T07:55:48.589628+0000","last_peered":"2024-09-19T07:55:48.589628+0000","last_clean":"2024-09-19T07:55:48.589628+0000","last_became_active":"2024-09-19T07:55:45.804000+0000","last_became_peered":"2024-09-19T07:55:45.804000+0000","last_unstale":"2024-09-19T07:55:48.589628+0000","last_undegraded":"2024-09-19T07:55:48.589628+0000","last_fullsized":"2024-09-19T07:55:48.589628+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:36:36.433763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807545+0000","last_change":"2024-09-19T07:55:46.804640+0000","last_active":"2024-09-19T07:55:47.807545+0000","last_peered":"2024-09-19T07:55:47.807545+0000","last_clean":"2024-09-19T07:55:47.807545+0000","last_became_active":"2024-09-19T07:55:46.804353+0000","last_became_peered":"2024-09-19T07:55:46.804353+0000","last_unstale":"2024-09-19T07:55:47.807545+0000","last_undegraded":"2024-09-19T07:55:47.807545+0000","last_fullsized":"2024-09-19T07:55:47.807545+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:54.973660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800706+0000","last_change":"2024-09-19T07:55:45.797824+0000","last_active":"2024-09-19T07:55:47.800706+0000","last_peered":"2024-09-19T07:55:47.800706+0000","last_clean":"2024-09-19T07:55:47.800706+0000","last_became_active":"2024-09-19T07:55:45.797559+0000","last_became_peered":"2024-09-19T07:55:45.797559+0000","last_unstale":"2024-09-19T07:55:47.800706+0000","last_undegraded":"2024-09-19T07:55:47.800706+0000","last_fullsized":"2024-09-19T07:55:47.800706+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:48.369553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590859+0000","last_change":"2024-09-19T07:55:46.815474+0000","last_active":"2024-09-19T07:55:48.590859+0000","last_peered":"2024-09-19T07:55:48.590859+0000","last_clean":"2024-09-19T07:55:48.590859+0000","last_became_active":"2024-09-19T07:55:46.814841+0000","last_became_peered":"2024-09-19T07:55:46.814841+0000","last_unstale":"2024-09-19T07:55:48.590859+0000","last_undegraded":"2024-09-19T07:55:48.590859+0000","last_fullsized":"2024-09-19T07:55:48.590859+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:23.021974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827717+0000","last_change":"2024-09-19T07:55:45.800048+0000","last_active":"2024-09-19T07:55:47.827717+0000","last_peered":"2024-09-19T07:55:47.827717+0000","last_clean":"2024-09-19T07:55:47.827717+0000","last_became_active":"2024-09-19T07:55:45.799749+0000","last_became_peered":"2024-09-19T07:55:45.799749+0000","last_unstale":"2024-09-19T07:55:47.827717+0000","last_undegraded":"2024-09-19T07:55:47.827717+0000","last_fullsized":"2024-09-19T07:55:47.827717+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:42.846081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827407+0000","last_change":"2024-09-19T07:55:46.819081+0000","last_active":"2024-09-19T07:55:47.827407+0000","last_peered":"2024-09-19T07:55:47.827407+0000","last_clean":"2024-09-19T07:55:47.827407+0000","last_became_active":"2024-09-19T07:55:46.818785+0000","last_became_peered":"2024-09-19T07:55:46.818785+0000","last_unstale":"2024-09-19T07:55:47.827407+0000","last_undegraded":"2024-09-19T07:55:47.827407+0000","last_fullsized":"2024-09-19T07:55:47.827407+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:48.362583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"151'3","reported_seq":17,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807316+0000","last_change":"2024-09-19T07:55:45.792189+0000","last_active":"2024-09-19T07:55:47.807316+0000","last_peered":"2024-09-19T07:55:47.807316+0000","last_clean":"2024-09-19T07:55:47.807316+0000","last_became_active":"2024-09-19T07:55:45.791906+0000","last_became_peered":"2024-09-19T07:55:45.791906+0000","last_unstale":"2024-09-19T07:55:47.807316+0000","last_undegraded":"2024-09-19T07:55:47.807316+0000","last_fullsized":"2024-09-19T07:55:47.807316+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:32.321023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.589900+0000","last_change":"2024-09-19T07:55:46.815581+0000","last_active":"2024-09-19T07:55:48.589900+0000","last_peered":"2024-09-19T07:55:48.589900+0000","last_clean":"2024-09-19T07:55:48.589900+0000","last_became_active":"2024-09-19T07:55:46.814058+0000","last_became_peered":"2024-09-19T07:55:46.814058+0000","last_unstale":"2024-09-19T07:55:48.589900+0000","last_undegraded":"2024-09-19T07:55:48.589900+0000","last_fullsized":"2024-09-19T07:55:48.589900+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:26.435190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"151'6","reported_seq":20,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827602+0000","last_change":"2024-09-19T07:55:45.802798+0000","last_active":"2024-09-19T07:55:47.827602+0000","last_peered":"2024-09-19T07:55:47.827602+0000","last_clean":"2024-09-19T07:55:47.827602+0000","last_became_active":"2024-09-19T07:55:45.802458+0000","last_became_peered":"2024-09-19T07:55:45.802458+0000","last_unstale":"2024-09-19T07:55:47.827602+0000","last_undegraded":"2024-09-19T07:55:47.827602+0000","last_fullsized":"2024-09-19T07:55:47.827602+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:08.281321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590621+0000","last_change":"2024-09-19T07:55:46.817885+0000","last_active":"2024-09-19T07:55:48.590621+0000","last_peered":"2024-09-19T07:55:48.590621+0000","last_clean":"2024-09-19T07:55:48.590621+0000","last_became_active":"2024-09-19T07:55:46.817566+0000","last_became_peered":"2024-09-19T07:55:46.817566+0000","last_unstale":"2024-09-19T07:55:48.590621+0000","last_undegraded":"2024-09-19T07:55:48.590621+0000","last_fullsized":"2024-09-19T07:55:48.590621+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:12.687039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"151'2","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590265+0000","last_change":"2024-09-19T07:55:45.801524+0000","last_active":"2024-09-19T07:55:48.590265+0000","last_peered":"2024-09-19T07:55:48.590265+0000","last_clean":"2024-09-19T07:55:48.590265+0000","last_became_active":"2024-09-19T07:55:45.801161+0000","last_became_peered":"2024-09-19T07:55:45.801161+0000","last_unstale":"2024-09-19T07:55:48.590265+0000","last_undegraded":"2024-09-19T07:55:48.590265+0000","last_fullsized":"2024-09-19T07:55:48.590265+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:13.737941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800893+0000","last_change":"2024-09-19T07:55:46.817083+0000","last_active":"2024-09-19T07:55:47.800893+0000","last_peered":"2024-09-19T07:55:47.800893+0000","last_clean":"2024-09-19T07:55:47.800893+0000","last_became_active":"2024-09-19T07:55:46.816804+0000","last_became_peered":"2024-09-19T07:55:46.816804+0000","last_unstale":"2024-09-19T07:55:47.800893+0000","last_undegraded":"2024-09-19T07:55:47.800893+0000","last_fullsized":"2024-09-19T07:55:47.800893+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:55.546951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827440+0000","last_change":"2024-09-19T07:55:45.801135+0000","last_active":"2024-09-19T07:55:47.827440+0000","last_peered":"2024-09-19T07:55:47.827440+0000","last_clean":"2024-09-19T07:55:47.827440+0000","last_became_active":"2024-09-19T07:55:45.800837+0000","last_became_peered":"2024-09-19T07:55:45.800837+0000","last_unstale":"2024-09-19T07:55:47.827440+0000","last_undegraded":"2024-09-19T07:55:47.827440+0000","last_fullsized":"2024-09-19T07:55:47.827440+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:30:46.437440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.807292+0000","last_change":"2024-09-19T07:55:46.813235+0000","last_active":"2024-09-19T07:55:47.807292+0000","last_peered":"2024-09-19T07:55:47.807292+0000","last_clean":"2024-09-19T07:55:47.807292+0000","last_became_active":"2024-09-19T07:55:46.812961+0000","last_became_peered":"2024-09-19T07:55:46.812961+0000","last_unstale":"2024-09-19T07:55:47.807292+0000","last_undegraded":"2024-09-19T07:55:47.807292+0000","last_fullsized":"2024-09-19T07:55:47.807292+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:59.757492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806364+0000","last_change":"2024-09-19T07:55:45.804843+0000","last_active":"2024-09-19T07:55:47.806364+0000","last_peered":"2024-09-19T07:55:47.806364+0000","last_clean":"2024-09-19T07:55:47.806364+0000","last_became_active":"2024-09-19T07:55:45.804613+0000","last_became_peered":"2024-09-19T07:55:45.804613+0000","last_unstale":"2024-09-19T07:55:47.806364+0000","last_undegraded":"2024-09-19T07:55:47.806364+0000","last_fullsized":"2024-09-19T07:55:47.806364+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:18.474678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590125+0000","last_change":"2024-09-19T07:55:46.813129+0000","last_active":"2024-09-19T07:55:48.590125+0000","last_peered":"2024-09-19T07:55:48.590125+0000","last_clean":"2024-09-19T07:55:48.590125+0000","last_became_active":"2024-09-19T07:55:46.812651+0000","last_became_peered":"2024-09-19T07:55:46.812651+0000","last_unstale":"2024-09-19T07:55:48.590125+0000","last_undegraded":"2024-09-19T07:55:48.590125+0000","last_fullsized":"2024-09-19T07:55:48.590125+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:20.176535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"151'8","reported_seq":22,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800808+0000","last_change":"2024-09-19T07:55:45.795277+0000","last_active":"2024-09-19T07:55:47.800808+0000","last_peered":"2024-09-19T07:55:47.800808+0000","last_clean":"2024-09-19T07:55:47.800808+0000","last_became_active":"2024-09-19T07:55:45.794876+0000","last_became_peered":"2024-09-19T07:55:45.794876+0000","last_unstale":"2024-09-19T07:55:47.800808+0000","last_undegraded":"2024-09-19T07:55:47.800808+0000","last_fullsized":"2024-09-19T07:55:47.800808+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:50.194217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800585+0000","last_change":"2024-09-19T07:55:46.817868+0000","last_active":"2024-09-19T07:55:47.800585+0000","last_peered":"2024-09-19T07:55:47.800585+0000","last_clean":"2024-09-19T07:55:47.800585+0000","last_became_active":"2024-09-19T07:55:46.817545+0000","last_became_peered":"2024-09-19T07:55:46.817545+0000","last_unstale":"2024-09-19T07:55:47.800585+0000","last_undegraded":"2024-09-19T07:55:47.800585+0000","last_fullsized":"2024-09-19T07:55:47.800585+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:37:45.202649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"151'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827791+0000","last_change":"2024-09-19T07:55:45.797085+0000","last_active":"2024-09-19T07:55:47.827791+0000","last_peered":"2024-09-19T07:55:47.827791+0000","last_clean":"2024-09-19T07:55:47.827791+0000","last_became_active":"2024-09-19T07:55:45.796793+0000","last_became_peered":"2024-09-19T07:55:45.796793+0000","last_unstale":"2024-09-19T07:55:47.827791+0000","last_undegraded":"2024-09-19T07:55:47.827791+0000","last_fullsized":"2024-09-19T07:55:47.827791+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:14.418996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590734+0000","last_change":"2024-09-19T07:55:46.815815+0000","last_active":"2024-09-19T07:55:48.590734+0000","last_peered":"2024-09-19T07:55:48.590734+0000","last_clean":"2024-09-19T07:55:48.590734+0000","last_became_active":"2024-09-19T07:55:46.815502+0000","last_became_peered":"2024-09-19T07:55:46.815502+0000","last_unstale":"2024-09-19T07:55:48.590734+0000","last_undegraded":"2024-09-19T07:55:48.590734+0000","last_fullsized":"2024-09-19T07:55:48.590734+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:36.406278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"151'2","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.801123+0000","last_change":"2024-09-19T07:55:45.803736+0000","last_active":"2024-09-19T07:55:47.801123+0000","last_peered":"2024-09-19T07:55:47.801123+0000","last_clean":"2024-09-19T07:55:47.801123+0000","last_became_active":"2024-09-19T07:55:45.803419+0000","last_became_peered":"2024-09-19T07:55:45.803419+0000","last_unstale":"2024-09-19T07:55:47.801123+0000","last_undegraded":"2024-09-19T07:55:47.801123+0000","last_fullsized":"2024-09-19T07:55:47.801123+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:39:47.947756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806166+0000","last_change":"2024-09-19T07:55:46.812728+0000","last_active":"2024-09-19T07:55:47.806166+0000","last_peered":"2024-09-19T07:55:47.806166+0000","last_clean":"2024-09-19T07:55:47.806166+0000","last_became_active":"2024-09-19T07:55:46.811051+0000","last_became_peered":"2024-09-19T07:55:46.811051+0000","last_unstale":"2024-09-19T07:55:47.806166+0000","last_undegraded":"2024-09-19T07:55:47.806166+0000","last_fullsized":"2024-09-19T07:55:47.806166+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:25.009628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"151'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590475+0000","last_change":"2024-09-19T07:55:45.801739+0000","last_active":"2024-09-19T07:55:48.590475+0000","last_peered":"2024-09-19T07:55:48.590475+0000","last_clean":"2024-09-19T07:55:48.590475+0000","last_became_active":"2024-09-19T07:55:45.800502+0000","last_became_peered":"2024-09-19T07:55:45.800502+0000","last_unstale":"2024-09-19T07:55:48.590475+0000","last_undegraded":"2024-09-19T07:55:48.590475+0000","last_fullsized":"2024-09-19T07:55:48.590475+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:37.112493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806634+0000","last_change":"2024-09-19T07:55:46.813673+0000","last_active":"2024-09-19T07:55:47.806634+0000","last_peered":"2024-09-19T07:55:47.806634+0000","last_clean":"2024-09-19T07:55:47.806634+0000","last_became_active":"2024-09-19T07:55:46.813361+0000","last_became_peered":"2024-09-19T07:55:46.813361+0000","last_unstale":"2024-09-19T07:55:47.806634+0000","last_undegraded":"2024-09-19T07:55:47.806634+0000","last_fullsized":"2024-09-19T07:55:47.806634+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:02:49.871105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800962+0000","last_change":"2024-09-19T07:55:45.805741+0000","last_active":"2024-09-19T07:55:47.800962+0000","last_peered":"2024-09-19T07:55:47.800962+0000","last_clean":"2024-09-19T07:55:47.800962+0000","last_became_active":"2024-09-19T07:55:45.805464+0000","last_became_peered":"2024-09-19T07:55:45.805464+0000","last_unstale":"2024-09-19T07:55:47.800962+0000","last_undegraded":"2024-09-19T07:55:47.800962+0000","last_fullsized":"2024-09-19T07:55:47.800962+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:31.558613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827844+0000","last_change":"2024-09-19T07:55:46.817735+0000","last_active":"2024-09-19T07:55:47.827844+0000","last_peered":"2024-09-19T07:55:47.827844+0000","last_clean":"2024-09-19T07:55:47.827844+0000","last_became_active":"2024-09-19T07:55:46.817466+0000","last_became_peered":"2024-09-19T07:55:46.817466+0000","last_unstale":"2024-09-19T07:55:47.827844+0000","last_undegraded":"2024-09-19T07:55:47.827844+0000","last_fullsized":"2024-09-19T07:55:47.827844+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:52.010816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"151'2","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590349+0000","last_change":"2024-09-19T07:55:45.801502+0000","last_active":"2024-09-19T07:55:48.590349+0000","last_peered":"2024-09-19T07:55:48.590349+0000","last_clean":"2024-09-19T07:55:48.590349+0000","last_became_active":"2024-09-19T07:55:45.800836+0000","last_became_peered":"2024-09-19T07:55:45.800836+0000","last_unstale":"2024-09-19T07:55:48.590349+0000","last_undegraded":"2024-09-19T07:55:48.590349+0000","last_fullsized":"2024-09-19T07:55:48.590349+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:24.008200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590386+0000","last_change":"2024-09-19T07:55:46.825374+0000","last_active":"2024-09-19T07:55:48.590386+0000","last_peered":"2024-09-19T07:55:48.590386+0000","last_clean":"2024-09-19T07:55:48.590386+0000","last_became_active":"2024-09-19T07:55:46.825083+0000","last_became_peered":"2024-09-19T07:55:46.825083+0000","last_unstale":"2024-09-19T07:55:48.590386+0000","last_undegraded":"2024-09-19T07:55:48.590386+0000","last_fullsized":"2024-09-19T07:55:48.590386+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:47.605737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806267+0000","last_change":"2024-09-19T07:55:45.795584+0000","last_active":"2024-09-19T07:55:47.806267+0000","last_peered":"2024-09-19T07:55:47.806267+0000","last_clean":"2024-09-19T07:55:47.806267+0000","last_became_active":"2024-09-19T07:55:45.795283+0000","last_became_peered":"2024-09-19T07:55:45.795283+0000","last_unstale":"2024-09-19T07:55:47.806267+0000","last_undegraded":"2024-09-19T07:55:47.806267+0000","last_fullsized":"2024-09-19T07:55:47.806267+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:07.706302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590213+0000","last_change":"2024-09-19T07:55:46.824701+0000","last_active":"2024-09-19T07:55:48.590213+0000","last_peered":"2024-09-19T07:55:48.590213+0000","last_clean":"2024-09-19T07:55:48.590213+0000","last_became_active":"2024-09-19T07:55:46.824414+0000","last_became_peered":"2024-09-19T07:55:46.824414+0000","last_unstale":"2024-09-19T07:55:48.590213+0000","last_undegraded":"2024-09-19T07:55:48.590213+0000","last_fullsized":"2024-09-19T07:55:48.590213+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:18.795850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.589735+0000","last_change":"2024-09-19T07:55:45.797312+0000","last_active":"2024-09-19T07:55:48.589735+0000","last_peered":"2024-09-19T07:55:48.589735+0000","last_clean":"2024-09-19T07:55:48.589735+0000","last_became_active":"2024-09-19T07:55:45.796832+0000","last_became_peered":"2024-09-19T07:55:45.796832+0000","last_unstale":"2024-09-19T07:55:48.589735+0000","last_undegraded":"2024-09-19T07:55:48.589735+0000","last_fullsized":"2024-09-19T07:55:48.589735+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:56.429223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590487+0000","last_change":"2024-09-19T07:55:46.817406+0000","last_active":"2024-09-19T07:55:48.590487+0000","last_peered":"2024-09-19T07:55:48.590487+0000","last_clean":"2024-09-19T07:55:48.590487+0000","last_became_active":"2024-09-19T07:55:46.817128+0000","last_became_peered":"2024-09-19T07:55:46.817128+0000","last_unstale":"2024-09-19T07:55:48.590487+0000","last_undegraded":"2024-09-19T07:55:48.590487+0000","last_fullsized":"2024-09-19T07:55:48.590487+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:50.004806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590346+0000","last_change":"2024-09-19T07:55:45.795162+0000","last_active":"2024-09-19T07:55:48.590346+0000","last_peered":"2024-09-19T07:55:48.590346+0000","last_clean":"2024-09-19T07:55:48.590346+0000","last_became_active":"2024-09-19T07:55:45.794882+0000","last_became_peered":"2024-09-19T07:55:45.794882+0000","last_unstale":"2024-09-19T07:55:48.590346+0000","last_undegraded":"2024-09-19T07:55:48.590346+0000","last_fullsized":"2024-09-19T07:55:48.590346+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:28.624166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.805948+0000","last_change":"2024-09-19T07:55:46.814034+0000","last_active":"2024-09-19T07:55:47.805948+0000","last_peered":"2024-09-19T07:55:47.805948+0000","last_clean":"2024-09-19T07:55:47.805948+0000","last_became_active":"2024-09-19T07:55:46.813762+0000","last_became_peered":"2024-09-19T07:55:46.813762+0000","last_unstale":"2024-09-19T07:55:47.805948+0000","last_undegraded":"2024-09-19T07:55:47.805948+0000","last_fullsized":"2024-09-19T07:55:47.805948+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:38.113921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"151'2","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590339+0000","last_change":"2024-09-19T07:55:45.801205+0000","last_active":"2024-09-19T07:55:48.590339+0000","last_peered":"2024-09-19T07:55:48.590339+0000","last_clean":"2024-09-19T07:55:48.590339+0000","last_became_active":"2024-09-19T07:55:45.800895+0000","last_became_peered":"2024-09-19T07:55:45.800895+0000","last_unstale":"2024-09-19T07:55:48.590339+0000","last_undegraded":"2024-09-19T07:55:48.590339+0000","last_fullsized":"2024-09-19T07:55:48.590339+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:56.753472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800693+0000","last_change":"2024-09-19T07:55:46.819496+0000","last_active":"2024-09-19T07:55:47.800693+0000","last_peered":"2024-09-19T07:55:47.800693+0000","last_clean":"2024-09-19T07:55:47.800693+0000","last_became_active":"2024-09-19T07:55:46.818787+0000","last_became_peered":"2024-09-19T07:55:46.818787+0000","last_unstale":"2024-09-19T07:55:47.800693+0000","last_undegraded":"2024-09-19T07:55:47.800693+0000","last_fullsized":"2024-09-19T07:55:47.800693+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:58.971945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"152'1","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.828794+0000","last_change":"2024-09-19T07:55:45.800698+0000","last_active":"2024-09-19T07:55:47.828794+0000","last_peered":"2024-09-19T07:55:47.828794+0000","last_clean":"2024-09-19T07:55:47.828794+0000","last_became_active":"2024-09-19T07:55:45.800012+0000","last_became_peered":"2024-09-19T07:55:45.800012+0000","last_unstale":"2024-09-19T07:55:47.828794+0000","last_undegraded":"2024-09-19T07:55:47.828794+0000","last_fullsized":"2024-09-19T07:55:47.828794+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:21.500902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590440+0000","last_change":"2024-09-19T07:55:46.816168+0000","last_active":"2024-09-19T07:55:48.590440+0000","last_peered":"2024-09-19T07:55:48.590440+0000","last_clean":"2024-09-19T07:55:48.590440+0000","last_became_active":"2024-09-19T07:55:46.815913+0000","last_became_peered":"2024-09-19T07:55:46.815913+0000","last_unstale":"2024-09-19T07:55:48.590440+0000","last_undegraded":"2024-09-19T07:55:48.590440+0000","last_fullsized":"2024-09-19T07:55:48.590440+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:08.744555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827580+0000","last_change":"2024-09-19T07:55:45.802021+0000","last_active":"2024-09-19T07:55:47.827580+0000","last_peered":"2024-09-19T07:55:47.827580+0000","last_clean":"2024-09-19T07:55:47.827580+0000","last_became_active":"2024-09-19T07:55:45.801518+0000","last_became_peered":"2024-09-19T07:55:45.801518+0000","last_unstale":"2024-09-19T07:55:47.827580+0000","last_undegraded":"2024-09-19T07:55:47.827580+0000","last_fullsized":"2024-09-19T07:55:47.827580+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:59.000128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.800755+0000","last_change":"2024-09-19T07:55:46.812665+0000","last_active":"2024-09-19T07:55:47.800755+0000","last_peered":"2024-09-19T07:55:47.800755+0000","last_clean":"2024-09-19T07:55:47.800755+0000","last_became_active":"2024-09-19T07:55:46.812424+0000","last_became_peered":"2024-09-19T07:55:46.812424+0000","last_unstale":"2024-09-19T07:55:47.800755+0000","last_undegraded":"2024-09-19T07:55:47.800755+0000","last_fullsized":"2024-09-19T07:55:47.800755+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:50.004806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.589985+0000","last_change":"2024-09-19T07:55:45.804911+0000","last_active":"2024-09-19T07:55:48.589985+0000","last_peered":"2024-09-19T07:55:48.589985+0000","last_clean":"2024-09-19T07:55:48.589985+0000","last_became_active":"2024-09-19T07:55:45.804578+0000","last_became_peered":"2024-09-19T07:55:45.804578+0000","last_unstale":"2024-09-19T07:55:48.589985+0000","last_undegraded":"2024-09-19T07:55:48.589985+0000","last_fullsized":"2024-09-19T07:55:48.589985+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:12.455456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806888+0000","last_change":"2024-09-19T07:55:46.815931+0000","last_active":"2024-09-19T07:55:47.806888+0000","last_peered":"2024-09-19T07:55:47.806888+0000","last_clean":"2024-09-19T07:55:47.806888+0000","last_became_active":"2024-09-19T07:55:46.815675+0000","last_became_peered":"2024-09-19T07:55:46.815675+0000","last_unstale":"2024-09-19T07:55:47.806888+0000","last_undegraded":"2024-09-19T07:55:47.806888+0000","last_fullsized":"2024-09-19T07:55:47.806888+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:42.354606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806692+0000","last_change":"2024-09-19T07:55:45.803403+0000","last_active":"2024-09-19T07:55:47.806692+0000","last_peered":"2024-09-19T07:55:47.806692+0000","last_clean":"2024-09-19T07:55:47.806692+0000","last_became_active":"2024-09-19T07:55:45.802988+0000","last_became_peered":"2024-09-19T07:55:45.802988+0000","last_unstale":"2024-09-19T07:55:47.806692+0000","last_undegraded":"2024-09-19T07:55:47.806692+0000","last_fullsized":"2024-09-19T07:55:47.806692+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:35.404851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590533+0000","last_change":"2024-09-19T07:55:46.815044+0000","last_active":"2024-09-19T07:55:48.590533+0000","last_peered":"2024-09-19T07:55:48.590533+0000","last_clean":"2024-09-19T07:55:48.590533+0000","last_became_active":"2024-09-19T07:55:46.814746+0000","last_became_peered":"2024-09-19T07:55:46.814746+0000","last_unstale":"2024-09-19T07:55:48.590533+0000","last_undegraded":"2024-09-19T07:55:48.590533+0000","last_fullsized":"2024-09-19T07:55:48.590533+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:20.053063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827507+0000","last_change":"2024-09-19T07:55:45.801889+0000","last_active":"2024-09-19T07:55:47.827507+0000","last_peered":"2024-09-19T07:55:47.827507+0000","last_clean":"2024-09-19T07:55:47.827507+0000","last_became_active":"2024-09-19T07:55:45.801569+0000","last_became_peered":"2024-09-19T07:55:45.801569+0000","last_unstale":"2024-09-19T07:55:47.827507+0000","last_undegraded":"2024-09-19T07:55:47.827507+0000","last_fullsized":"2024-09-19T07:55:47.827507+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:19.834999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827678+0000","last_change":"2024-09-19T07:55:46.816559+0000","last_active":"2024-09-19T07:55:47.827678+0000","last_peered":"2024-09-19T07:55:47.827678+0000","last_clean":"2024-09-19T07:55:47.827678+0000","last_became_active":"2024-09-19T07:55:46.816305+0000","last_became_peered":"2024-09-19T07:55:46.816305+0000","last_unstale":"2024-09-19T07:55:47.827678+0000","last_undegraded":"2024-09-19T07:55:47.827678+0000","last_fullsized":"2024-09-19T07:55:47.827678+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:27.599531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.806465+0000","last_change":"2024-09-19T07:55:45.797679+0000","last_active":"2024-09-19T07:55:47.806465+0000","last_peered":"2024-09-19T07:55:47.806465+0000","last_clean":"2024-09-19T07:55:47.806465+0000","last_became_active":"2024-09-19T07:55:45.797383+0000","last_became_peered":"2024-09-19T07:55:45.797383+0000","last_unstale":"2024-09-19T07:55:47.806465+0000","last_undegraded":"2024-09-19T07:55:47.806465+0000","last_fullsized":"2024-09-19T07:55:47.806465+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:28.624166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590257+0000","last_change":"2024-09-19T07:55:46.813583+0000","last_active":"2024-09-19T07:55:48.590257+0000","last_peered":"2024-09-19T07:55:48.590257+0000","last_clean":"2024-09-19T07:55:48.590257+0000","last_became_active":"2024-09-19T07:55:46.813311+0000","last_became_peered":"2024-09-19T07:55:46.813311+0000","last_unstale":"2024-09-19T07:55:48.590257+0000","last_undegraded":"2024-09-19T07:55:48.590257+0000","last_fullsized":"2024-09-19T07:55:48.590257+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:28.942805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.589573+0000","last_change":"2024-09-19T07:55:45.800775+0000","last_active":"2024-09-19T07:55:48.589573+0000","last_peered":"2024-09-19T07:55:48.589573+0000","last_clean":"2024-09-19T07:55:48.589573+0000","last_became_active":"2024-09-19T07:55:45.800484+0000","last_became_peered":"2024-09-19T07:55:45.800484+0000","last_unstale":"2024-09-19T07:55:48.589573+0000","last_undegraded":"2024-09-19T07:55:48.589573+0000","last_fullsized":"2024-09-19T07:55:48.589573+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:35.490586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590087+0000","last_change":"2024-09-19T07:55:46.818224+0000","last_active":"2024-09-19T07:55:48.590087+0000","last_peered":"2024-09-19T07:55:48.590087+0000","last_clean":"2024-09-19T07:55:48.590087+0000","last_became_active":"2024-09-19T07:55:46.817954+0000","last_became_peered":"2024-09-19T07:55:46.817954+0000","last_unstale":"2024-09-19T07:55:48.590087+0000","last_undegraded":"2024-09-19T07:55:48.590087+0000","last_fullsized":"2024-09-19T07:55:48.590087+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:45.779621+0000","last_clean_scrub_stamp":"2024-09-19T07:55:45.779621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:38.643240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:47.827644+0000","last_change":"2024-09-19T07:55:45.800943+0000","last_active":"2024-09-19T07:55:47.827644+0000","last_peered":"2024-09-19T07:55:47.827644+0000","last_clean":"2024-09-19T07:55:47.827644+0000","last_became_active":"2024-09-19T07:55:45.800428+0000","last_became_peered":"2024-09-19T07:55:45.800428+0000","last_unstale":"2024-09-19T07:55:47.827644+0000","last_undegraded":"2024-09-19T07:55:47.827644+0000","last_fullsized":"2024-09-19T07:55:47.827644+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:55:44.778194+0000","last_clean_scrub_stamp":"2024-09-19T07:55:44.778194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:40.722531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":408,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-19T07:55:48.590404+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:55:48.590404+0000","last_peered":"2024-09-19T07:55:48.590404+0000","last_clean":"2024-09-19T07:55:48.590404+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:55:48.590404+0000","last_undegraded":"2024-09-19T07:55:48.590404+0000","last_fullsized":"2024-09-19T07:55:48.590404+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509599,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87708,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284132,"statfs":{"total":96636764160,"available":96546951168,"internally_reserved":0,"allocated":2498560,"data_stored":1476936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":87292362},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509598,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105240,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266600,"statfs":{"total":96636764160,"available":96528998400,"internally_reserved":0,"allocated":1904640,"data_stored":886568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509599,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73252,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298588,"statfs":{"total":96636764160,"available":96561754112,"internally_reserved":0,"allocated":1912832,"data_stored":886692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509599,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87152,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284688,"statfs":{"total":96636764160,"available":96547520512,"internally_reserved":0,"allocated":1921024,"data_stored":887332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509598,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105764,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266076,"statfs":{"total":96636764160,"available":96528461824,"internally_reserved":0,"allocated":2506752,"data_stored":1477040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542302,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119088,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252752,"statfs":{"total":96636764160,"available":96514818048,"internally_reserved":0,"allocated":1925120,"data_stored":888784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542302,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83052,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288788,"statfs":{"total":96636764160,"available":96551718912,"internally_reserved":0,"allocated":1916928,"data_stored":888180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575007,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109336,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262504,"statfs":{"total":96636764160,"available":96524804096,"internally_reserved":0,"allocated":1904640,"data_stored":886568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:55:54.964 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T07:55:54.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T07:55:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.110+0000 7f17d93e8640 1 Processor -- start 2024-09-19T07:55:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.110+0000 7f17d93e8640 1 -- start start 2024-09-19T07:55:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.111+0000 7f17d93e8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f17d415e0a0 0x7f17d4162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.111+0000 7f17d93e8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17d40b5280 0x7f17d40d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.111+0000 7f17d93e8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17d415f250 con 0x7f17d40d43e0 2024-09-19T07:55:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.111+0000 7f17d93e8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f17d40d4d60 con 0x7f17d415e0a0 2024-09-19T07:55:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.111+0000 7f17d93e8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f17d415f530 con 0x7f17d40b5280 2024-09-19T07:55:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.111+0000 7f17d2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f17d415e0a0 0x7f17d4162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.111+0000 7f17d2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f17d415e0a0 0x7f17d4162a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46812/0 (socket says 172.21.15.16:46812) 2024-09-19T07:55:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.111+0000 7f17d27fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f17d40d43e0 0x7f17d40b4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49808/0 (socket says 172.21.15.16:49808) 2024-09-19T07:55:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.111+0000 7f17d2ffd640 1 -- 172.21.15.16:0/943929284 learned_addr learned my addr 172.21.15.16:0/943929284 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.112+0000 7f17d37fe640 1 --2- 172.21.15.16:0/943929284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17d40b5280 0x7f17d40d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.112+0000 7f17d2ffd640 1 -- 172.21.15.16:0/943929284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17d40b5280 msgr2=0x7f17d40d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.112+0000 7f17d2ffd640 1 --2- 172.21.15.16:0/943929284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17d40b5280 0x7f17d40d4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.112+0000 7f17d2ffd640 1 -- 172.21.15.16:0/943929284 >> v1:172.21.15.16:6789/0 conn(0x7f17d40d43e0 legacy=0x7f17d40b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.112+0000 7f17d1ffb640 1 -- 172.21.15.16:0/943929284 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 843722892 0 0) 0x7f17d415f250 con 0x7f17d40d43e0 2024-09-19T07:55:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.112+0000 7f17d2ffd640 1 -- 172.21.15.16:0/943929284 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d415f810 con 0x7f17d415e0a0 2024-09-19T07:55:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.112+0000 7f17d2ffd640 1 --2- 172.21.15.16:0/943929284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d415e0a0 0x7f17d4162a70 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009870 tx=0x7f17c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c6b926de344570 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.113+0000 7f17d1ffb640 1 -- 172.21.15.16:0/943929284 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4037070 con 0x7f17d415e0a0 2024-09-19T07:55:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.113+0000 7f17d1ffb640 1 -- 172.21.15.16:0/943929284 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c4032440 con 0x7f17d415e0a0 2024-09-19T07:55:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.113+0000 7f17d1ffb640 1 -- 172.21.15.16:0/943929284 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4032810 con 0x7f17d415e0a0 2024-09-19T07:55:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.113+0000 7f17d93e8640 1 -- 172.21.15.16:0/943929284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d415e0a0 msgr2=0x7f17d4162a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.113+0000 7f17d93e8640 1 --2- 172.21.15.16:0/943929284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d415e0a0 0x7f17d4162a70 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009870 tx=0x7f17c402a270 comp rx=0 tx=0).stop 2024-09-19T07:55:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.113+0000 7f17d93e8640 1 -- 172.21.15.16:0/943929284 shutdown_connections 2024-09-19T07:55:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.113+0000 7f17d93e8640 1 --2- 172.21.15.16:0/943929284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17d40b5280 0x7f17d40d4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.113+0000 7f17d93e8640 1 --2- 172.21.15.16:0/943929284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d415e0a0 0x7f17d4162a70 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.113+0000 7f17d93e8640 1 -- 172.21.15.16:0/943929284 >> 172.21.15.16:0/943929284 conn(0x7f17d409d770 msgr2=0x7f17d408bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.113+0000 7f17d93e8640 1 -- 172.21.15.16:0/943929284 shutdown_connections 2024-09-19T07:55:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.113+0000 7f17d93e8640 1 -- 172.21.15.16:0/943929284 wait complete. 2024-09-19T07:55:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.115+0000 7f17d93e8640 1 Processor -- start 2024-09-19T07:55:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.115+0000 7f17d93e8640 1 -- start start 2024-09-19T07:55:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.115+0000 7f17d93e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d40b5280 0x7f17d40b97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.115+0000 7f17d93e8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17d40d43e0 0x7f17d41e9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.115+0000 7f17d93e8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17d415e0a0 0x7f17d414e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.115+0000 7f17d93e8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17d415e600 con 0x7f17d40b5280 2024-09-19T07:55:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.115+0000 7f17d93e8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f17d415e480 con 0x7f17d415e0a0 2024-09-19T07:55:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.115+0000 7f17d93e8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f17d415e780 con 0x7f17d40d43e0 2024-09-19T07:55:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.115+0000 7f17d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d40b5280 0x7f17d40b97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.115+0000 7f17d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d40b5280 0x7f17d40b97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46826/0 (socket says 172.21.15.16:46826) 2024-09-19T07:55:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.115+0000 7f17d2ffd640 1 -- 172.21.15.16:0/3763515531 learned_addr learned my addr 172.21.15.16:0/3763515531 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.116+0000 7f17d27fc640 1 --2- 172.21.15.16:0/3763515531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17d40d43e0 0x7f17d41e9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.116+0000 7f17d37fe640 1 --2- 172.21.15.16:0/3763515531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17d415e0a0 0x7f17d414e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.116+0000 7f17d2ffd640 1 -- 172.21.15.16:0/3763515531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17d40d43e0 msgr2=0x7f17d41e9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.116+0000 7f17d2ffd640 1 --2- 172.21.15.16:0/3763515531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17d40d43e0 0x7f17d41e9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.116+0000 7f17d2ffd640 1 -- 172.21.15.16:0/3763515531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17d415e0a0 msgr2=0x7f17d414e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.116+0000 7f17d2ffd640 1 --2- 172.21.15.16:0/3763515531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17d415e0a0 0x7f17d414e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.116+0000 7f17d2ffd640 1 -- 172.21.15.16:0/3763515531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d414b530 con 0x7f17d40b5280 2024-09-19T07:55:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.116+0000 7f17d2ffd640 1 --2- 172.21.15.16:0/3763515531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d40b5280 0x7f17d40b97d0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f17c4035040 tx=0x7f17c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.116+0000 7f17bb7fe640 1 -- 172.21.15.16:0/3763515531 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4037040 con 0x7f17d40b5280 2024-09-19T07:55:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.116+0000 7f17bb7fe640 1 -- 172.21.15.16:0/3763515531 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c4004030 con 0x7f17d40b5280 2024-09-19T07:55:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.116+0000 7f17d93e8640 1 -- 172.21.15.16:0/3763515531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d414b810 con 0x7f17d40b5280 2024-09-19T07:55:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.116+0000 7f17bb7fe640 1 -- 172.21.15.16:0/3763515531 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4004420 con 0x7f17d40b5280 2024-09-19T07:55:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.116+0000 7f17d93e8640 1 -- 172.21.15.16:0/3763515531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d414bdd0 con 0x7f17d40b5280 2024-09-19T07:55:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.117+0000 7f17d93e8640 1 -- 172.21.15.16:0/3763515531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d406d840 con 0x7f17d40b5280 2024-09-19T07:55:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.118+0000 7f17bb7fe640 1 -- 172.21.15.16:0/3763515531 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f17c403a070 con 0x7f17d40b5280 2024-09-19T07:55:55.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.118+0000 7f17bb7fe640 1 --2- 172.21.15.16:0/3763515531 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f17c006c2b0 0x7f17c006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:55.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.118+0000 7f17bb7fe640 1 -- 172.21.15.16:0/3763515531 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f17c40abb10 con 0x7f17d40b5280 2024-09-19T07:55:55.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.119+0000 7f17d27fc640 1 --2- 172.21.15.16:0/3763515531 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f17c006c2b0 0x7f17c006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:55.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.119+0000 7f17d27fc640 1 --2- 172.21.15.16:0/3763515531 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f17c006c2b0 0x7f17c006e770 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f17d406da90 tx=0x7f17c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:55.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.123+0000 7f17bb7fe640 1 -- 172.21.15.16:0/3763515531 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f17d406d840 con 0x7f17d40b5280 2024-09-19T07:55:55.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.365+0000 7f17d93e8640 1 -- 172.21.15.16:0/3763515531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f17d40d4820 con 0x7f17d40b5280 2024-09-19T07:55:55.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.367+0000 7f17bb7fe640 1 -- 172.21.15.16:0/3763515531 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7f17c4041020 con 0x7f17d40b5280 2024-09-19T07:55:55.368 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-19T07:55:55.368 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDR1etmDedOJhAAkuTyoHfnow1B0O4GO3DdGA== 2024-09-19T07:55:55.368 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-19T07:55:55.368 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-19T07:55:55.368 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T07:55:55.368 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-19T07:55:55.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.369+0000 7f17d93e8640 1 -- 172.21.15.16:0/3763515531 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f17c006c2b0 msgr2=0x7f17c006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:55.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.369+0000 7f17d93e8640 1 --2- 172.21.15.16:0/3763515531 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f17c006c2b0 0x7f17c006e770 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f17d406da90 tx=0x7f17c8009040 comp rx=0 tx=0).stop 2024-09-19T07:55:55.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.369+0000 7f17d93e8640 1 -- 172.21.15.16:0/3763515531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d40b5280 msgr2=0x7f17d40b97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:55.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.369+0000 7f17d93e8640 1 --2- 172.21.15.16:0/3763515531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d40b5280 0x7f17d40b97d0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f17c4035040 tx=0x7f17c40046c0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.369+0000 7f17d93e8640 1 -- 172.21.15.16:0/3763515531 shutdown_connections 2024-09-19T07:55:55.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.369+0000 7f17d93e8640 1 --2- 172.21.15.16:0/3763515531 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f17c006c2b0 0x7f17c006e770 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.369+0000 7f17d93e8640 1 --2- 172.21.15.16:0/3763515531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17d415e0a0 0x7f17d414e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.369+0000 7f17d93e8640 1 --2- 172.21.15.16:0/3763515531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17d40d43e0 0x7f17d41e9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.369+0000 7f17d93e8640 1 --2- 172.21.15.16:0/3763515531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17d40b5280 0x7f17d40b97d0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.369+0000 7f17d93e8640 1 -- 172.21.15.16:0/3763515531 >> 172.21.15.16:0/3763515531 conn(0x7f17d409d770 msgr2=0x7f17d40b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:55.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.370+0000 7f17d93e8640 1 -- 172.21.15.16:0/3763515531 shutdown_connections 2024-09-19T07:55:55.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.370+0000 7f17d93e8640 1 -- 172.21.15.16:0/3763515531 wait complete. 2024-09-19T07:55:55.381 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:55:55.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.482+0000 7f4ad351a640 1 Processor -- start 2024-09-19T07:55:55.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.482+0000 7f4ad351a640 1 -- start start 2024-09-19T07:55:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.483+0000 7f4ad351a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4acc15e2a0 0x7f4acc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.483+0000 7f4ad351a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4acc15e680 0x7f4acc0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.483+0000 7f4ad351a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4acc0d1880 con 0x7f4acc06d960 2024-09-19T07:55:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.483+0000 7f4ad351a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f4acc0d1b60 con 0x7f4acc15e680 2024-09-19T07:55:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.483+0000 7f4ad351a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4acc0cf2d0 con 0x7f4acc15e2a0 2024-09-19T07:55:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.483+0000 7f4ad0ab5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4acc15e680 0x7f4acc0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.483+0000 7f4ad0ab5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4acc15e680 0x7f4acc0cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46834/0 (socket says 172.21.15.16:46834) 2024-09-19T07:55:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.483+0000 7f4ad0ab5640 1 -- 172.21.15.16:0/2950479327 learned_addr learned my addr 172.21.15.16:0/2950479327 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.483+0000 7f4ad12b6640 1 --2- 172.21.15.16:0/2950479327 >> v2:172.21.15.146:3300/0 conn(0x7f4acc15e2a0 0x7f4acc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.483+0000 7f4ad0ab5640 1 -- 172.21.15.16:0/2950479327 >> v2:172.21.15.146:3300/0 conn(0x7f4acc15e2a0 msgr2=0x7f4acc06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.483+0000 7f4ad0ab5640 1 --2- 172.21.15.16:0/2950479327 >> v2:172.21.15.146:3300/0 conn(0x7f4acc15e2a0 0x7f4acc06d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.483+0000 7f4ad0ab5640 1 -- 172.21.15.16:0/2950479327 >> v1:172.21.15.16:6789/0 conn(0x7f4acc06d960 legacy=0x7f4acc0d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.483+0000 7f4ac3fff640 1 -- 172.21.15.16:0/2950479327 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3453615608 0 0) 0x7f4acc0d1880 con 0x7f4acc06d960 2024-09-19T07:55:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.483+0000 7f4ad0ab5640 1 -- 172.21.15.16:0/2950479327 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4acc0cf5b0 con 0x7f4acc15e680 2024-09-19T07:55:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.484+0000 7f4ad0ab5640 1 --2- 172.21.15.16:0/2950479327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc15e680 0x7f4acc0cfad0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0b6fa0 tx=0x7f4abc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1180a13fe1de8a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.484+0000 7f4ac3fff640 1 -- 172.21.15.16:0/2950479327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc037070 con 0x7f4acc15e680 2024-09-19T07:55:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.484+0000 7f4ac3fff640 1 -- 172.21.15.16:0/2950479327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4abc032440 con 0x7f4acc15e680 2024-09-19T07:55:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.484+0000 7f4ac3fff640 1 -- 172.21.15.16:0/2950479327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc032810 con 0x7f4acc15e680 2024-09-19T07:55:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.485+0000 7f4ad351a640 1 -- 172.21.15.16:0/2950479327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc15e680 msgr2=0x7f4acc0cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.485+0000 7f4ad351a640 1 --2- 172.21.15.16:0/2950479327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc15e680 0x7f4acc0cfad0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0b6fa0 tx=0x7f4abc02a270 comp rx=0 tx=0).stop 2024-09-19T07:55:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.485+0000 7f4ad351a640 1 -- 172.21.15.16:0/2950479327 shutdown_connections 2024-09-19T07:55:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.485+0000 7f4ad351a640 1 --2- 172.21.15.16:0/2950479327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc15e680 0x7f4acc0cfad0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.485+0000 7f4ad351a640 1 --2- 172.21.15.16:0/2950479327 >> v2:172.21.15.146:3300/0 conn(0x7f4acc15e2a0 0x7f4acc06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.485+0000 7f4ad351a640 1 -- 172.21.15.16:0/2950479327 >> 172.21.15.16:0/2950479327 conn(0x7f4acc09d760 msgr2=0x7f4acc08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.485+0000 7f4ad351a640 1 -- 172.21.15.16:0/2950479327 shutdown_connections 2024-09-19T07:55:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.485+0000 7f4ad351a640 1 -- 172.21.15.16:0/2950479327 wait complete. 2024-09-19T07:55:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.486+0000 7f4ad351a640 1 Processor -- start 2024-09-19T07:55:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.486+0000 7f4ad351a640 1 -- start start 2024-09-19T07:55:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.487+0000 7f4ad351a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4acc06d960 0x7f4acc0b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.487+0000 7f4ad351a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc15e2a0 0x7f4acc14cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.487+0000 7f4ad351a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4acc15e680 0x7f4acc1f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.487+0000 7f4ad351a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc0d4c00 con 0x7f4acc15e2a0 2024-09-19T07:55:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.487+0000 7f4ad351a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc0d4a80 con 0x7f4acc06d960 2024-09-19T07:55:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.487+0000 7f4ad351a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4acc0d0010 con 0x7f4acc15e680 2024-09-19T07:55:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.487+0000 7f4ad0ab5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc15e2a0 0x7f4acc14cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.487+0000 7f4ad0ab5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc15e2a0 0x7f4acc14cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46842/0 (socket says 172.21.15.16:46842) 2024-09-19T07:55:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.487+0000 7f4ad0ab5640 1 -- 172.21.15.16:0/2568307244 learned_addr learned my addr 172.21.15.16:0/2568307244 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.487+0000 7f4ad0ab5640 1 -- 172.21.15.16:0/2568307244 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4acc15e680 msgr2=0x7f4acc1f6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.487+0000 7f4ad0ab5640 1 --2- 172.21.15.16:0/2568307244 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4acc15e680 0x7f4acc1f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.487+0000 7f4ad0ab5640 1 -- 172.21.15.16:0/2568307244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4acc06d960 msgr2=0x7f4acc0b97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.487+0000 7f4ad0ab5640 1 --2- 172.21.15.16:0/2568307244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4acc06d960 0x7f4acc0b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.487+0000 7f4ad0ab5640 1 -- 172.21.15.16:0/2568307244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4acc0cb970 con 0x7f4acc15e2a0 2024-09-19T07:55:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.487+0000 7f4ad0ab5640 1 --2- 172.21.15.16:0/2568307244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc15e2a0 0x7f4acc14cd40 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f4abc02a2b0 tx=0x7f4abc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.488+0000 7f4ac1ffb640 1 -- 172.21.15.16:0/2568307244 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc037040 con 0x7f4acc15e2a0 2024-09-19T07:55:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.488+0000 7f4ac1ffb640 1 -- 172.21.15.16:0/2568307244 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4abc042070 con 0x7f4acc15e2a0 2024-09-19T07:55:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.488+0000 7f4ac1ffb640 1 -- 172.21.15.16:0/2568307244 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc030880 con 0x7f4acc15e2a0 2024-09-19T07:55:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.488+0000 7f4ad351a640 1 -- 172.21.15.16:0/2568307244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4acc0b8d10 con 0x7f4acc15e2a0 2024-09-19T07:55:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.488+0000 7f4ad351a640 1 -- 172.21.15.16:0/2568307244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4acc0cbc50 con 0x7f4acc15e2a0 2024-09-19T07:55:55.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.489+0000 7f4ad351a640 1 -- 172.21.15.16:0/2568307244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4acc06d420 con 0x7f4acc15e2a0 2024-09-19T07:55:55.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.489+0000 7f4ac1ffb640 1 -- 172.21.15.16:0/2568307244 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4abc030a20 con 0x7f4acc15e2a0 2024-09-19T07:55:55.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.490+0000 7f4ac1ffb640 1 --2- 172.21.15.16:0/2568307244 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4ab006c2b0 0x7f4ab006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:55.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.490+0000 7f4ac1ffb640 1 -- 172.21.15.16:0/2568307244 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f4abc0abeb0 con 0x7f4acc15e2a0 2024-09-19T07:55:55.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.490+0000 7f4ad12b6640 1 --2- 172.21.15.16:0/2568307244 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4ab006c2b0 0x7f4ab006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:55.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.491+0000 7f4ad12b6640 1 --2- 172.21.15.16:0/2568307244 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4ab006c2b0 0x7f4ab006e770 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8006a90 tx=0x7f4ac8006a20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:55.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.495+0000 7f4ac1ffb640 1 -- 172.21.15.16:0/2568307244 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4abc07a480 con 0x7f4acc15e2a0 2024-09-19T07:55:55.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.706+0000 7f4ad351a640 1 -- 172.21.15.16:0/2568307244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4acc06cf90 con 0x7f4acc15e2a0 2024-09-19T07:55:55.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.707+0000 7f4ac1ffb640 1 -- 172.21.15.16:0/2568307244 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5230 (secure 0 0 0) 0x7f4abc079d80 con 0x7f4acc15e2a0 2024-09-19T07:55:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:55.707 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":92,"btime":"2024-09-19T07:55:49:210981+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7064,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3873812166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3873812166},{"type":"v1","addr":"172.21.15.146:6837","nonce":3873812166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":7852,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/951273503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":951273503},{"type":"v1","addr":"172.21.15.146:6835","nonce":951273503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":8757,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2338289617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2338289617},{"type":"v1","addr":"172.21.15.16:6834","nonce":2338289617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:55:46.155417+0000","modified":"2024-09-19T07:55:48.561637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8760":{"gid":8760,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6835/3705993654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":3705993654},{"type":"v1","addr":"172.21.15.16:6835","nonce":3705993654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8760,"qdb_cluster":[8760]},"id":11}]} 2024-09-19T07:55:55.707 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 92 2024-09-19T07:55:55.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.709+0000 7f4ad351a640 1 -- 172.21.15.16:0/2568307244 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4ab006c2b0 msgr2=0x7f4ab006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:55.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.709+0000 7f4ad351a640 1 --2- 172.21.15.16:0/2568307244 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4ab006c2b0 0x7f4ab006e770 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8006a90 tx=0x7f4ac8006a20 comp rx=0 tx=0).stop 2024-09-19T07:55:55.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.709+0000 7f4ad351a640 1 -- 172.21.15.16:0/2568307244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc15e2a0 msgr2=0x7f4acc14cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:55.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.709+0000 7f4ad351a640 1 --2- 172.21.15.16:0/2568307244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc15e2a0 0x7f4acc14cd40 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f4abc02a2b0 tx=0x7f4abc0046c0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.709+0000 7f4ad351a640 1 -- 172.21.15.16:0/2568307244 shutdown_connections 2024-09-19T07:55:55.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.709+0000 7f4ad351a640 1 --2- 172.21.15.16:0/2568307244 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4ab006c2b0 0x7f4ab006e770 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.709+0000 7f4ad351a640 1 --2- 172.21.15.16:0/2568307244 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4acc15e680 0x7f4acc1f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.709+0000 7f4ad351a640 1 --2- 172.21.15.16:0/2568307244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc15e2a0 0x7f4acc14cd40 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.709+0000 7f4ad351a640 1 --2- 172.21.15.16:0/2568307244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4acc06d960 0x7f4acc0b97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:55.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.709+0000 7f4ad351a640 1 -- 172.21.15.16:0/2568307244 >> 172.21.15.16:0/2568307244 conn(0x7f4acc09d760 msgr2=0x7f4acc074ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:55.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.709+0000 7f4ad351a640 1 -- 172.21.15.16:0/2568307244 shutdown_connections 2024-09-19T07:55:55.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:55.709+0000 7f4ad351a640 1 -- 172.21.15.16:0/2568307244 wait complete. 2024-09-19T07:55:56.123 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 92, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T07:55:46.155417+0000', 'modified': '2024-09-19T07:55:48.561637+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8760}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8760': {'gid': 8760, 'name': 'c', 'rank': 0, 'incarnation': 89, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.16:6835/3705993654', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6833', 'nonce': 3705993654}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 3705993654}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8760, 'qdb_cluster': [8760]} 2024-09-19T07:55:56.123 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T07:55:56.123 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-19T07:55:56.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.231+0000 7fc01939d640 1 Processor -- start 2024-09-19T07:55:56.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.231+0000 7fc01939d640 1 -- start start 2024-09-19T07:55:56.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.232+0000 7fc01939d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc01415e500 0x7fc0140d1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.232+0000 7fc01939d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc0140b4170 0x7fc0140d27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.232+0000 7fc01939d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0140d2d10 0x7fc0140d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.232+0000 7fc01939d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc0140b5210 con 0x7fc0140b4170 2024-09-19T07:55:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.232+0000 7fc01939d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc0140b4600 con 0x7fc01415e500 2024-09-19T07:55:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.232+0000 7fc01939d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc0140b48e0 con 0x7fc0140d2d10 2024-09-19T07:55:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.232+0000 7fc0137fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc0140b4170 0x7fc0140d27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.232+0000 7fc0137fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc0140b4170 0x7fc0140d27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46844/0 (socket says 172.21.15.16:46844) 2024-09-19T07:55:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.232+0000 7fc0137fe640 1 -- 172.21.15.16:0/603639059 learned_addr learned my addr 172.21.15.16:0/603639059 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.232+0000 7fc013fff640 1 --2- 172.21.15.16:0/603639059 >> v2:172.21.15.146:3300/0 conn(0x7fc01415e500 0x7fc0140d1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.233+0000 7fc0137fe640 1 -- 172.21.15.16:0/603639059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0140d2d10 msgr2=0x7fc0140d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.233+0000 7fc018b9c640 1 --2- 172.21.15.16:0/603639059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0140d2d10 0x7fc0140d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.233+0000 7fc0137fe640 1 --2- 172.21.15.16:0/603639059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0140d2d10 0x7fc0140d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.233+0000 7fc0137fe640 1 -- 172.21.15.16:0/603639059 >> v2:172.21.15.146:3300/0 conn(0x7fc01415e500 msgr2=0x7fc0140d1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.233+0000 7fc0137fe640 1 --2- 172.21.15.16:0/603639059 >> v2:172.21.15.146:3300/0 conn(0x7fc01415e500 0x7fc0140d1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.233+0000 7fc0137fe640 1 -- 172.21.15.16:0/603639059 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0140b4bc0 con 0x7fc0140b4170 2024-09-19T07:55:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.233+0000 7fc018b9c640 1 --2- 172.21.15.16:0/603639059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0140d2d10 0x7fc0140d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:56.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.233+0000 7fc0137fe640 1 --2- 172.21.15.16:0/603639059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0140b4170 0x7fc0140d27d0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fc01408e0b0 tx=0x7fc00802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65205fe1cc6dcba8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:56.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.235+0000 7fc012ffd640 1 -- 172.21.15.16:0/603639059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc008037070 con 0x7fc0140b4170 2024-09-19T07:55:56.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.235+0000 7fc012ffd640 1 -- 172.21.15.16:0/603639059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc008032440 con 0x7fc0140b4170 2024-09-19T07:55:56.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.235+0000 7fc012ffd640 1 -- 172.21.15.16:0/603639059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc008032810 con 0x7fc0140b4170 2024-09-19T07:55:56.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.236+0000 7fc01939d640 1 -- 172.21.15.16:0/603639059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0140b4170 msgr2=0x7fc0140d27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:56.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.236+0000 7fc01939d640 1 --2- 172.21.15.16:0/603639059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0140b4170 0x7fc0140d27d0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fc01408e0b0 tx=0x7fc00802a270 comp rx=0 tx=0).stop 2024-09-19T07:55:56.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.236+0000 7fc01939d640 1 -- 172.21.15.16:0/603639059 shutdown_connections 2024-09-19T07:55:56.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.236+0000 7fc01939d640 1 --2- 172.21.15.16:0/603639059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc0140d2d10 0x7fc0140d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.236+0000 7fc01939d640 1 --2- 172.21.15.16:0/603639059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0140b4170 0x7fc0140d27d0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.236+0000 7fc01939d640 1 --2- 172.21.15.16:0/603639059 >> v2:172.21.15.146:3300/0 conn(0x7fc01415e500 0x7fc0140d1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.236+0000 7fc01939d640 1 -- 172.21.15.16:0/603639059 >> 172.21.15.16:0/603639059 conn(0x7fc01408dc00 msgr2=0x7fc01409d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:56.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.236+0000 7fc01939d640 1 -- 172.21.15.16:0/603639059 shutdown_connections 2024-09-19T07:55:56.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.236+0000 7fc01939d640 1 -- 172.21.15.16:0/603639059 wait complete. 2024-09-19T07:55:56.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.237+0000 7fc01939d640 1 Processor -- start 2024-09-19T07:55:56.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.238+0000 7fc01939d640 1 -- start start 2024-09-19T07:55:56.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.238+0000 7fc01939d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0140d2d10 0x7fc0140b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.238+0000 7fc01939d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01415e500 0x7fc014155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.238+0000 7fc01939d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0141606f0 0x7fc0140cf190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.238+0000 7fc01939d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0140d2480 con 0x7fc0141606f0 2024-09-19T07:55:56.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.238+0000 7fc01939d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc0140d2300 con 0x7fc0140d2d10 2024-09-19T07:55:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.238+0000 7fc01939d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc0140d1120 con 0x7fc01415e500 2024-09-19T07:55:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.238+0000 7fc018b9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0141606f0 0x7fc0140cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.238+0000 7fc018b9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0141606f0 0x7fc0140cf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46856/0 (socket says 172.21.15.16:46856) 2024-09-19T07:55:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.238+0000 7fc018b9c640 1 -- 172.21.15.16:0/856352278 learned_addr learned my addr 172.21.15.16:0/856352278 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.239+0000 7fc013fff640 1 --2- 172.21.15.16:0/856352278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0140d2d10 0x7fc0140b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.239+0000 7fc0137fe640 1 --2- 172.21.15.16:0/856352278 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01415e500 0x7fc014155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.239+0000 7fc018b9c640 1 -- 172.21.15.16:0/856352278 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01415e500 msgr2=0x7fc014155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.239+0000 7fc018b9c640 1 --2- 172.21.15.16:0/856352278 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01415e500 0x7fc014155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.239+0000 7fc018b9c640 1 -- 172.21.15.16:0/856352278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0140d2d10 msgr2=0x7fc0140b7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.239+0000 7fc018b9c640 1 --2- 172.21.15.16:0/856352278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0140d2d10 0x7fc0140b7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.239+0000 7fc018b9c640 1 -- 172.21.15.16:0/856352278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0140b76d0 con 0x7fc0141606f0 2024-09-19T07:55:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.239+0000 7fc018b9c640 1 --2- 172.21.15.16:0/856352278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0141606f0 0x7fc0140cf190 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00cef0 tx=0x7fc00c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.239+0000 7fc010ff9640 1 -- 172.21.15.16:0/856352278 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c01a070 con 0x7fc0141606f0 2024-09-19T07:55:56.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.239+0000 7fc010ff9640 1 -- 172.21.15.16:0/856352278 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc00c00d840 con 0x7fc0141606f0 2024-09-19T07:55:56.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.239+0000 7fc01939d640 1 -- 172.21.15.16:0/856352278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0140b79b0 con 0x7fc0141606f0 2024-09-19T07:55:56.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.239+0000 7fc010ff9640 1 -- 172.21.15.16:0/856352278 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c00dc30 con 0x7fc0141606f0 2024-09-19T07:55:56.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.239+0000 7fc01939d640 1 -- 172.21.15.16:0/856352278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0140b7f70 con 0x7fc0141606f0 2024-09-19T07:55:56.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.241+0000 7fc010ff9640 1 -- 172.21.15.16:0/856352278 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc00c01d070 con 0x7fc0141606f0 2024-09-19T07:55:56.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.241+0000 7fc010ff9640 1 --2- 172.21.15.16:0/856352278 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc00006c2b0 0x7fc00006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.241+0000 7fc010ff9640 1 -- 172.21.15.16:0/856352278 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7fc00c08bd30 con 0x7fc0141606f0 2024-09-19T07:55:56.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.242+0000 7fc013fff640 1 --2- 172.21.15.16:0/856352278 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc00006c2b0 0x7fc00006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.242+0000 7fc01d46e740 1 Processor -- start 2024-09-19T07:55:56.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.242+0000 7fc013fff640 1 --2- 172.21.15.16:0/856352278 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc00006c2b0 0x7fc00006e770 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc0040079a0 tx=0x7fc004008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:56.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.242+0000 7fc01d46e740 1 -- start start 2024-09-19T07:55:56.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fc01d46e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e5394b0 0x555b7e57ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fc01d46e740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e536c70 0x555b7e539890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fc01d46e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e53ac40 0x555b7e533aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fc01d46e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555b7e49b520 con 0x555b7e53ac40 2024-09-19T07:55:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fc01d46e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x555b7e4433e0 con 0x555b7e5394b0 2024-09-19T07:55:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fc01d46e740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x555b7e4974e0 con 0x555b7e536c70 2024-09-19T07:55:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fc018b9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e53ac40 0x555b7e533aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fc018b9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e53ac40 0x555b7e533aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46862/0 (socket says 172.21.15.16:46862) 2024-09-19T07:55:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fc018b9c640 1 -- 172.21.15.16:0/2089243567 learned_addr learned my addr 172.21.15.16:0/2089243567 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fc013fff640 1 --2- 172.21.15.16:0/2089243567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e5394b0 0x555b7e57ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fc0137fe640 1 --2- 172.21.15.16:0/2089243567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e536c70 0x555b7e539890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fc018b9c640 1 -- 172.21.15.16:0/2089243567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e536c70 msgr2=0x555b7e539890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fc018b9c640 1 --2- 172.21.15.16:0/2089243567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e536c70 0x555b7e539890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fc018b9c640 1 -- 172.21.15.16:0/2089243567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e5394b0 msgr2=0x555b7e57ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:56.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fc018b9c640 1 --2- 172.21.15.16:0/2089243567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e5394b0 0x555b7e57ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fc018b9c640 1 -- 172.21.15.16:0/2089243567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b7e45e2f0 con 0x555b7e53ac40 2024-09-19T07:55:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fc013fff640 1 --2- 172.21.15.16:0/2089243567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e5394b0 0x555b7e57ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fc018b9c640 1 --2- 172.21.15.16:0/2089243567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e53ac40 0x555b7e533aa0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fc00c08a0d0 tx=0x7fc00c059770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9befec72d95b9dbd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fbff9ffb640 1 -- 172.21.15.16:0/2089243567 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c062880 con 0x555b7e53ac40 2024-09-19T07:55:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fbff9ffb640 1 -- 172.21.15.16:0/2089243567 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc00c070030 con 0x555b7e53ac40 2024-09-19T07:55:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.243+0000 7fbff9ffb640 1 -- 172.21.15.16:0/2089243567 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c057df0 con 0x555b7e53ac40 2024-09-19T07:55:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.244+0000 7fc01d46e740 1 -- 172.21.15.16:0/2089243567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e53ac40 msgr2=0x555b7e533aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.244+0000 7fc01d46e740 1 --2- 172.21.15.16:0/2089243567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e53ac40 0x555b7e533aa0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fc00c08a0d0 tx=0x7fc00c059770 comp rx=0 tx=0).stop 2024-09-19T07:55:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.244+0000 7fc01d46e740 1 -- 172.21.15.16:0/2089243567 shutdown_connections 2024-09-19T07:55:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.244+0000 7fc01d46e740 1 --2- 172.21.15.16:0/2089243567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e53ac40 0x555b7e533aa0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.244+0000 7fc01d46e740 1 --2- 172.21.15.16:0/2089243567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e536c70 0x555b7e539890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.244+0000 7fc01d46e740 1 --2- 172.21.15.16:0/2089243567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e5394b0 0x555b7e57ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.244+0000 7fc01d46e740 1 -- 172.21.15.16:0/2089243567 >> 172.21.15.16:0/2089243567 conn(0x555b7e535c70 msgr2=0x555b7e57e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.244+0000 7fc01d46e740 1 -- 172.21.15.16:0/2089243567 shutdown_connections 2024-09-19T07:55:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.244+0000 7fc01d46e740 1 -- 172.21.15.16:0/2089243567 wait complete. 2024-09-19T07:55:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.245+0000 7fc01d46e740 1 -- start start 2024-09-19T07:55:56.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.245+0000 7fc01d46e740 1 Processor -- start 2024-09-19T07:55:56.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fc01d46e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e5394b0 0x555b7e57b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fc01d46e740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e53ac40 0x555b7e5619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fc01d46e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e534160 0x555b7e561ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fc01d46e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555b7e535110 con 0x555b7e5394b0 2024-09-19T07:55:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fc01d46e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x555b7e424450 con 0x555b7e534160 2024-09-19T07:55:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fc01d46e740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x555b7e57f480 con 0x555b7e53ac40 2024-09-19T07:55:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fc01d46e740 10 client.0 _ll_register_callbacks cb 0x555b7e1adac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:55:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fc013fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e5394b0 0x555b7e57b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fc013fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e5394b0 0x555b7e57b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46868/0 (socket says 172.21.15.16:46868) 2024-09-19T07:55:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fc013fff640 1 -- 172.21.15.16:0/2786785750 learned_addr learned my addr 172.21.15.16:0/2786785750 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fc018b9c640 1 --2- 172.21.15.16:0/2786785750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e534160 0x555b7e561ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fc0137fe640 1 --2- 172.21.15.16:0/2786785750 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e53ac40 0x555b7e5619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fc013fff640 1 -- 172.21.15.16:0/2786785750 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e53ac40 msgr2=0x555b7e5619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fc013fff640 1 --2- 172.21.15.16:0/2786785750 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e53ac40 0x555b7e5619a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fc013fff640 1 -- 172.21.15.16:0/2786785750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e534160 msgr2=0x555b7e561ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fc013fff640 1 --2- 172.21.15.16:0/2786785750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e534160 0x555b7e561ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fc013fff640 1 -- 172.21.15.16:0/2786785750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b7e1ae2d0 con 0x555b7e5394b0 2024-09-19T07:55:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fc018b9c640 1 --2- 172.21.15.16:0/2786785750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e534160 0x555b7e561ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fc013fff640 1 --2- 172.21.15.16:0/2786785750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e5394b0 0x555b7e57b7a0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fc004006a70 tx=0x7fc004004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762c9f097e4996b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fbff2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:55:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.246+0000 7fbff2ffd640 1 -- 172.21.15.16:0/2786785750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004026020 con 0x555b7e5394b0 2024-09-19T07:55:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.247+0000 7fbff2ffd640 1 -- 172.21.15.16:0/2786785750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7fc004014800 con 0x555b7e5394b0 2024-09-19T07:55:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.247+0000 7fc01d46e740 1 -- 172.21.15.16:0/2786785750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc00c070030 con 0x555b7e5394b0 2024-09-19T07:55:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.247+0000 7fbff2ffd640 1 -- 172.21.15.16:0/2786785750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fc004030050 con 0x555b7e5394b0 2024-09-19T07:55:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.247+0000 7fc01d46e740 10 client.8928 fetch_fsmap learned FSMap version 92 2024-09-19T07:55:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.247+0000 7fc01d46e740 1 -- 172.21.15.16:0/2786785750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x555b7e2786b0 con 0x555b7e5394b0 2024-09-19T07:55:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.247+0000 7fbff2ffd640 1 -- 172.21.15.16:0/2786785750 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc00c070030 con 0x555b7e5394b0 2024-09-19T07:55:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.247+0000 7fbff2ffd640 1 -- 172.21.15.16:0/2786785750 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 92) ==== 3502+0+0 (secure 0 0 0) 0x7fc004033020 con 0x555b7e5394b0 2024-09-19T07:55:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.247+0000 7fc01d46e740 10 client.8928 fetch_fsmap finished waiting for FSMap version 92 2024-09-19T07:55:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.247+0000 7fc01d46e740 10 client.8928 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T07:55:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.247+0000 7fc01d46e740 20 client.8928 populate_metadata read hostname 'smithi016' 2024-09-19T07:55:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.247+0000 7fc01d46e740 1 --2- 172.21.15.16:0/2786785750 >> [v2:172.21.15.16:6833/3705993654,v1:172.21.15.16:6835/3705993654] conn(0x555b7e584d70 0x555b7e585150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.247+0000 7fc01d46e740 4 client.8928 mds_command: new command op to 8760 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:55:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.247+0000 7fc01d46e740 1 -- 172.21.15.16:0/2786785750 --> [v2:172.21.15.16:6833/3705993654,v1:172.21.15.16:6835/3705993654] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b7e4a8c50 con 0x555b7e584d70 2024-09-19T07:55:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.247+0000 7fc0137fe640 1 --2- 172.21.15.16:0/2786785750 >> [v2:172.21.15.16:6833/3705993654,v1:172.21.15.16:6835/3705993654] conn(0x555b7e584d70 0x555b7e585150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.247+0000 7fc0137fe640 1 --2- 172.21.15.16:0/2786785750 >> [v2:172.21.15.16:6833/3705993654,v1:172.21.15.16:6835/3705993654] conn(0x555b7e584d70 0x555b7e585150 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34d21599c5c8c5ff server_cookie=27fac5ae61a61c40 in_seq=0 out_seq=0 2024-09-19T07:55:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.247+0000 7fbff2ffd640 10 client.8928 ms_handle_connect on v2:172.21.15.16:6833/3705993654 2024-09-19T07:55:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.248+0000 7fbff2ffd640 1 -- 172.21.15.16:0/2786785750 <== mds.0 v2:172.21.15.16:6833/3705993654 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b7e4a8c50 con 0x555b7e584d70 2024-09-19T07:55:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.248+0000 7fbff2ffd640 10 client.8928 handle_command_reply: tid=0 2024-09-19T07:55:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.248+0000 7fc01d46e740 1 client.8928 shutdown 2024-09-19T07:55:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.248+0000 7fc01d46e740 1 -- 172.21.15.16:0/2786785750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e5394b0 msgr2=0x555b7e57b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.248+0000 7fc01d46e740 1 --2- 172.21.15.16:0/2786785750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e5394b0 0x555b7e57b7a0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fc004006a70 tx=0x7fc004004730 comp rx=0 tx=0).stop 2024-09-19T07:55:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.248+0000 7fc01d46e740 1 -- 172.21.15.16:0/2786785750 shutdown_connections 2024-09-19T07:55:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.248+0000 7fc01d46e740 1 --2- 172.21.15.16:0/2786785750 >> [v2:172.21.15.16:6833/3705993654,v1:172.21.15.16:6835/3705993654] conn(0x555b7e584d70 0x555b7e585150 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.248+0000 7fc01d46e740 1 --2- 172.21.15.16:0/2786785750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e534160 0x555b7e561ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.248+0000 7fc01d46e740 1 --2- 172.21.15.16:0/2786785750 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e53ac40 0x555b7e5619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.248+0000 7fc01d46e740 1 --2- 172.21.15.16:0/2786785750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e5394b0 0x555b7e57b7a0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.248+0000 7fbff2ffd640 0 client.8928 ms_handle_reset on v2:172.21.15.16:6833/3705993654 2024-09-19T07:55:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.248+0000 7fc01d46e740 1 -- 172.21.15.16:0/2786785750 >> 172.21.15.16:0/2786785750 conn(0x555b7e536c70 msgr2=0x555b7e57d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.248+0000 7fc01d46e740 1 -- 172.21.15.16:0/2786785750 shutdown_connections 2024-09-19T07:55:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.248+0000 7fc01d46e740 1 -- 172.21.15.16:0/2786785750 wait complete. 2024-09-19T07:55:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.249+0000 7fc01d46e740 20 client.8928 trim_cache size 0 max 16384 2024-09-19T07:55:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.266+0000 7fc01d46e740 1 Processor -- start 2024-09-19T07:55:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.266+0000 7fc01d46e740 1 -- start start 2024-09-19T07:55:56.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.267+0000 7fc01d46e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e534160 0x555b7e561d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.267+0000 7fc01d46e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e5394b0 0x555b7e5f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.267+0000 7fc01d46e740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e53ac40 0x555b7e583bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.267+0000 7fc01d46e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141ffd40 con 0x555b7e5394b0 2024-09-19T07:55:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.267+0000 7fc01d46e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc0140ccbb0 con 0x555b7e534160 2024-09-19T07:55:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.267+0000 7fc01d46e740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc0141ff7b0 con 0x555b7e53ac40 2024-09-19T07:55:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.267+0000 7fc0137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e5394b0 0x555b7e5f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.267+0000 7fc0137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e5394b0 0x555b7e5f4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46870/0 (socket says 172.21.15.16:46870) 2024-09-19T07:55:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.267+0000 7fc0137fe640 1 -- 172.21.15.16:0/2016063679 learned_addr learned my addr 172.21.15.16:0/2016063679 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.267+0000 7fc013fff640 1 --2- 172.21.15.16:0/2016063679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e534160 0x555b7e561d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.267+0000 7fc018b9c640 1 --2- 172.21.15.16:0/2016063679 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e53ac40 0x555b7e583bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.267+0000 7fc0137fe640 1 -- 172.21.15.16:0/2016063679 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e53ac40 msgr2=0x555b7e583bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.268+0000 7fc0137fe640 1 --2- 172.21.15.16:0/2016063679 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e53ac40 0x555b7e583bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.268+0000 7fc0137fe640 1 -- 172.21.15.16:0/2016063679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e534160 msgr2=0x555b7e561d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.268+0000 7fc0137fe640 1 --2- 172.21.15.16:0/2016063679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e534160 0x555b7e561d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.268+0000 7fc0137fe640 1 -- 172.21.15.16:0/2016063679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b7e560620 con 0x555b7e5394b0 2024-09-19T07:55:56.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.268+0000 7fc0137fe640 1 --2- 172.21.15.16:0/2016063679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e5394b0 0x555b7e5f4f30 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fc008009960 tx=0x7fc00802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb9a7ec249c1226 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:56.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.268+0000 7fbff97fa640 1 -- 172.21.15.16:0/2016063679 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc008037040 con 0x555b7e5394b0 2024-09-19T07:55:56.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.268+0000 7fbff97fa640 1 -- 172.21.15.16:0/2016063679 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0080327c0 con 0x555b7e5394b0 2024-09-19T07:55:56.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.268+0000 7fbff97fa640 1 -- 172.21.15.16:0/2016063679 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc008032bb0 con 0x555b7e5394b0 2024-09-19T07:55:56.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.268+0000 7fc01d46e740 1 -- 172.21.15.16:0/2016063679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e5394b0 msgr2=0x555b7e5f4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:56.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.268+0000 7fc01d46e740 1 --2- 172.21.15.16:0/2016063679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e5394b0 0x555b7e5f4f30 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fc008009960 tx=0x7fc00802c000 comp rx=0 tx=0).stop 2024-09-19T07:55:56.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.268+0000 7fc01d46e740 1 -- 172.21.15.16:0/2016063679 shutdown_connections 2024-09-19T07:55:56.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.268+0000 7fc01d46e740 1 --2- 172.21.15.16:0/2016063679 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e53ac40 0x555b7e583bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.268+0000 7fc01d46e740 1 --2- 172.21.15.16:0/2016063679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e5394b0 0x555b7e5f4f30 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.268+0000 7fc01d46e740 1 --2- 172.21.15.16:0/2016063679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e534160 0x555b7e561d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.268+0000 7fc01d46e740 1 -- 172.21.15.16:0/2016063679 >> 172.21.15.16:0/2016063679 conn(0x555b7e536c70 msgr2=0x555b7e57fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:56.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.268+0000 7fc01d46e740 1 -- 172.21.15.16:0/2016063679 shutdown_connections 2024-09-19T07:55:56.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.268+0000 7fc01d46e740 1 -- 172.21.15.16:0/2016063679 wait complete. 2024-09-19T07:55:56.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.269+0000 7fc01d46e740 1 -- start start 2024-09-19T07:55:56.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.269+0000 7fc01d46e740 1 Processor -- start 2024-09-19T07:55:56.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.270+0000 7fc01d46e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e534160 0x555b7e5f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.270+0000 7fc01d46e740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e5394b0 0x555b7e538850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.270+0000 7fc01d46e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e53ac40 0x555b7e583bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.270+0000 7fc01d46e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555b7e5624e0 con 0x555b7e53ac40 2024-09-19T07:55:56.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.270+0000 7fc01d46e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x555b7e533960 con 0x555b7e534160 2024-09-19T07:55:56.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.270+0000 7fc01d46e740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x555b7e582ae0 con 0x555b7e5394b0 2024-09-19T07:55:56.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.270+0000 7fc01d46e740 10 client.0 _ll_register_callbacks cb 0x7fc004007040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:55:56.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.270+0000 7fc018b9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e53ac40 0x555b7e583bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.270+0000 7fc013fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e534160 0x555b7e5f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.270+0000 7fc018b9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e53ac40 0x555b7e583bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46884/0 (socket says 172.21.15.16:46884) 2024-09-19T07:55:56.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.270+0000 7fc018b9c640 1 -- 172.21.15.16:0/483803166 learned_addr learned my addr 172.21.15.16:0/483803166 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:56.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.270+0000 7fc0137fe640 1 --2- 172.21.15.16:0/483803166 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e5394b0 0x555b7e538850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.270+0000 7fc018b9c640 1 -- 172.21.15.16:0/483803166 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e5394b0 msgr2=0x555b7e538850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:56.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.270+0000 7fc018b9c640 1 --2- 172.21.15.16:0/483803166 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e5394b0 0x555b7e538850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.270+0000 7fc018b9c640 1 -- 172.21.15.16:0/483803166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e534160 msgr2=0x555b7e5f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:56.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.270+0000 7fc018b9c640 1 --2- 172.21.15.16:0/483803166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e534160 0x555b7e5f3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.270+0000 7fc018b9c640 1 -- 172.21.15.16:0/483803166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b7e57a7e0 con 0x555b7e53ac40 2024-09-19T07:55:56.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.271+0000 7fc018b9c640 1 --2- 172.21.15.16:0/483803166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e53ac40 0x555b7e583bc0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0597b0 tx=0x7fc00c021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b97b1511999a98 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:56.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.271+0000 7fbff2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T07:55:56.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.271+0000 7fbff2ffd640 1 -- 172.21.15.16:0/483803166 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c075070 con 0x555b7e53ac40 2024-09-19T07:55:56.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.271+0000 7fc01d46e740 1 -- 172.21.15.16:0/483803166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0080327c0 con 0x555b7e53ac40 2024-09-19T07:55:56.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.271+0000 7fbff2ffd640 1 -- 172.21.15.16:0/483803166 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7fc00c057df0 con 0x555b7e53ac40 2024-09-19T07:55:56.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.271+0000 7fbff2ffd640 1 -- 172.21.15.16:0/483803166 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fc00c081050 con 0x555b7e53ac40 2024-09-19T07:55:56.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.271+0000 7fc01d46e740 10 client.8934 fetch_fsmap learned FSMap version 92 2024-09-19T07:55:56.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.271+0000 7fc01d46e740 1 -- 172.21.15.16:0/483803166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x555b7e538d90 con 0x555b7e53ac40 2024-09-19T07:55:56.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.271+0000 7fbff2ffd640 1 -- 172.21.15.16:0/483803166 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0080327c0 con 0x555b7e53ac40 2024-09-19T07:55:56.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.271+0000 7fbff2ffd640 1 -- 172.21.15.16:0/483803166 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 92) ==== 3502+0+0 (secure 0 0 0) 0x7fc00c0623f0 con 0x555b7e53ac40 2024-09-19T07:55:56.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.271+0000 7fc01d46e740 10 client.8934 fetch_fsmap finished waiting for FSMap version 92 2024-09-19T07:55:56.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.271+0000 7fc01d46e740 10 client.8934 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T07:55:56.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.271+0000 7fc01d46e740 20 client.8934 populate_metadata read hostname 'smithi016' 2024-09-19T07:55:56.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.271+0000 7fc01d46e740 1 --2- 172.21.15.16:0/483803166 >> [v2:172.21.15.16:6833/3705993654,v1:172.21.15.16:6835/3705993654] conn(0x555b7e562e60 0x555b7e563240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.271+0000 7fc01d46e740 4 client.8934 mds_command: new command op to 8760 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:55:56.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.271+0000 7fc01d46e740 1 -- 172.21.15.16:0/483803166 --> [v2:172.21.15.16:6833/3705993654,v1:172.21.15.16:6835/3705993654] -- command(tid 0: {"prefix": "status"}) -- 0x555b7e4a2fa0 con 0x555b7e562e60 2024-09-19T07:55:56.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.271+0000 7fc013fff640 1 --2- 172.21.15.16:0/483803166 >> [v2:172.21.15.16:6833/3705993654,v1:172.21.15.16:6835/3705993654] conn(0x555b7e562e60 0x555b7e563240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.272+0000 7fbff2ffd640 10 client.8934 ms_handle_connect on v2:172.21.15.16:6833/3705993654 2024-09-19T07:55:56.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.272+0000 7fc013fff640 1 --2- 172.21.15.16:0/483803166 >> [v2:172.21.15.16:6833/3705993654,v1:172.21.15.16:6835/3705993654] conn(0x555b7e562e60 0x555b7e563240 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3674903d6381e56f server_cookie=df0d6e83d972f371 in_seq=0 out_seq=0 2024-09-19T07:55:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.272+0000 7fbff2ffd640 1 -- 172.21.15.16:0/483803166 <== mds.0 v2:172.21.15.16:6833/3705993654 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x555b7e4a2fa0 con 0x555b7e562e60 2024-09-19T07:55:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.272+0000 7fbff2ffd640 10 client.8934 handle_command_reply: tid=0 2024-09-19T07:55:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.272+0000 7fc01d46e740 1 client.8934 shutdown 2024-09-19T07:55:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.272+0000 7fc01d46e740 1 -- 172.21.15.16:0/483803166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e53ac40 msgr2=0x555b7e583bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.272+0000 7fc01d46e740 1 --2- 172.21.15.16:0/483803166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e53ac40 0x555b7e583bc0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0597b0 tx=0x7fc00c021040 comp rx=0 tx=0).stop 2024-09-19T07:55:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.272+0000 7fc01d46e740 1 -- 172.21.15.16:0/483803166 shutdown_connections 2024-09-19T07:55:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.272+0000 7fc01d46e740 1 --2- 172.21.15.16:0/483803166 >> [v2:172.21.15.16:6833/3705993654,v1:172.21.15.16:6835/3705993654] conn(0x555b7e562e60 0x555b7e563240 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.272+0000 7fc01d46e740 1 --2- 172.21.15.16:0/483803166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b7e53ac40 0x555b7e583bc0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.272+0000 7fc01d46e740 1 --2- 172.21.15.16:0/483803166 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x555b7e5394b0 0x555b7e538850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.272+0000 7fbff2ffd640 0 client.8934 ms_handle_reset on v2:172.21.15.16:6833/3705993654 2024-09-19T07:55:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.272+0000 7fc01d46e740 1 --2- 172.21.15.16:0/483803166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555b7e534160 0x555b7e5f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.272+0000 7fc01d46e740 1 -- 172.21.15.16:0/483803166 >> 172.21.15.16:0/483803166 conn(0x555b7e536c70 msgr2=0x555b7e561d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.272+0000 7fc01d46e740 1 -- 172.21.15.16:0/483803166 shutdown_connections 2024-09-19T07:55:56.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.272+0000 7fc01d46e740 1 -- 172.21.15.16:0/483803166 wait complete. 2024-09-19T07:55:56.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.273+0000 7fc01d46e740 20 client.8934 trim_cache size 0 max 16384 2024-09-19T07:55:56.280 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:55:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb", 2024-09-19T07:55:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-19T07:55:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 8760, 2024-09-19T07:55:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T07:55:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T07:55:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T07:55:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 9.4660160040000001, 2024-09-19T07:55:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 92, 2024-09-19T07:55:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 152, 2024-09-19T07:55:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 152, 2024-09-19T07:55:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 19.756209137999999 2024-09-19T07:55:56.281 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:55:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.273+0000 7fc01939d640 1 -- 172.21.15.16:0/856352278 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc00006c2b0 msgr2=0x7fc00006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.273+0000 7fc01939d640 1 --2- 172.21.15.16:0/856352278 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc00006c2b0 0x7fc00006e770 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc0040079a0 tx=0x7fc004008040 comp rx=0 tx=0).stop 2024-09-19T07:55:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.273+0000 7fc01939d640 1 -- 172.21.15.16:0/856352278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0141606f0 msgr2=0x7fc0140cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.273+0000 7fc01939d640 1 --2- 172.21.15.16:0/856352278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0141606f0 0x7fc0140cf190 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00cef0 tx=0x7fc00c00a710 comp rx=0 tx=0).stop 2024-09-19T07:55:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.273+0000 7fc01939d640 1 -- 172.21.15.16:0/856352278 shutdown_connections 2024-09-19T07:55:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.273+0000 7fc01939d640 1 --2- 172.21.15.16:0/856352278 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc00006c2b0 0x7fc00006e770 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.273+0000 7fc01939d640 1 --2- 172.21.15.16:0/856352278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0141606f0 0x7fc0140cf190 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.273+0000 7fc01939d640 1 --2- 172.21.15.16:0/856352278 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc01415e500 0x7fc014155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.273+0000 7fc01939d640 1 --2- 172.21.15.16:0/856352278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0140d2d10 0x7fc0140b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.273+0000 7fc01939d640 1 -- 172.21.15.16:0/856352278 >> 172.21.15.16:0/856352278 conn(0x7fc01408dc00 msgr2=0x7fc01406e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.273+0000 7fc01939d640 1 -- 172.21.15.16:0/856352278 shutdown_connections 2024-09-19T07:55:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.273+0000 7fc01939d640 1 -- 172.21.15.16:0/856352278 wait complete. 2024-09-19T07:55:56.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:55:56.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.386+0000 7f40a5e3c640 1 Processor -- start 2024-09-19T07:55:56.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.386+0000 7f40a5e3c640 1 -- start start 2024-09-19T07:55:56.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.386+0000 7f40a5e3c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f40a015f0d0 0x7f40a015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.386+0000 7f40a5e3c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40a00d0070 0x7f40a00d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.386+0000 7f40a5e3c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f40a00d1940 0x7f40a00d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.386+0000 7f40a5e3c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f40a00cf2b0 con 0x7f40a00d1940 2024-09-19T07:55:56.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.386+0000 7f40a5e3c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f40a00cf590 con 0x7f40a015f0d0 2024-09-19T07:55:56.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.386+0000 7f40a5e3c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f40a00cf870 con 0x7f40a00d0070 2024-09-19T07:55:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.387+0000 7f409ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f40a00d1940 0x7f40a00d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.387+0000 7f409ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f40a00d1940 0x7f40a00d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46898/0 (socket says 172.21.15.16:46898) 2024-09-19T07:55:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.387+0000 7f409ffff640 1 -- 172.21.15.16:0/1591179269 learned_addr learned my addr 172.21.15.16:0/1591179269 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.387+0000 7f409effd640 1 --2- 172.21.15.16:0/1591179269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40a00d0070 0x7f40a00d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.387+0000 7f409f7fe640 1 --2- 172.21.15.16:0/1591179269 >> v2:172.21.15.146:3300/0 conn(0x7f40a015f0d0 0x7f40a015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.387+0000 7f409ffff640 1 -- 172.21.15.16:0/1591179269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40a00d0070 msgr2=0x7f40a00d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:56.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.387+0000 7f409ffff640 1 --2- 172.21.15.16:0/1591179269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40a00d0070 0x7f40a00d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.387+0000 7f409ffff640 1 -- 172.21.15.16:0/1591179269 >> v2:172.21.15.146:3300/0 conn(0x7f40a015f0d0 msgr2=0x7f40a015f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:56.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.387+0000 7f409ffff640 1 --2- 172.21.15.16:0/1591179269 >> v2:172.21.15.146:3300/0 conn(0x7f40a015f0d0 0x7f40a015f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.387+0000 7f409ffff640 1 -- 172.21.15.16:0/1591179269 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a00cfb50 con 0x7f40a00d1940 2024-09-19T07:55:56.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.387+0000 7f409f7fe640 1 --2- 172.21.15.16:0/1591179269 >> v2:172.21.15.146:3300/0 conn(0x7f40a015f0d0 0x7f40a015f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:56.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.388+0000 7f409ffff640 1 --2- 172.21.15.16:0/1591179269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a00d1940 0x7f40a00d4130 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f40a00b6f40 tx=0x7f409802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673255fa968dbc02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:56.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.388+0000 7f409e7fc640 1 -- 172.21.15.16:0/1591179269 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4098036070 con 0x7f40a00d1940 2024-09-19T07:55:56.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.388+0000 7f409e7fc640 1 -- 172.21.15.16:0/1591179269 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4098031600 con 0x7f40a00d1940 2024-09-19T07:55:56.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.388+0000 7f409e7fc640 1 -- 172.21.15.16:0/1591179269 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40980319d0 con 0x7f40a00d1940 2024-09-19T07:55:56.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.388+0000 7f40a5e3c640 1 -- 172.21.15.16:0/1591179269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a00d1940 msgr2=0x7f40a00d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:56.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.388+0000 7f40a5e3c640 1 --2- 172.21.15.16:0/1591179269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a00d1940 0x7f40a00d4130 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f40a00b6f40 tx=0x7f409802a610 comp rx=0 tx=0).stop 2024-09-19T07:55:56.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.389+0000 7f40a5e3c640 1 -- 172.21.15.16:0/1591179269 shutdown_connections 2024-09-19T07:55:56.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.389+0000 7f40a5e3c640 1 --2- 172.21.15.16:0/1591179269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a00d1940 0x7f40a00d4130 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.389+0000 7f40a5e3c640 1 --2- 172.21.15.16:0/1591179269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40a00d0070 0x7f40a00d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.389+0000 7f40a5e3c640 1 --2- 172.21.15.16:0/1591179269 >> v2:172.21.15.146:3300/0 conn(0x7f40a015f0d0 0x7f40a015f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.389+0000 7f40a5e3c640 1 -- 172.21.15.16:0/1591179269 >> 172.21.15.16:0/1591179269 conn(0x7f40a009d760 msgr2=0x7f40a008bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.389+0000 7f40a5e3c640 1 -- 172.21.15.16:0/1591179269 shutdown_connections 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.389+0000 7f40a5e3c640 1 -- 172.21.15.16:0/1591179269 wait complete. 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.390+0000 7f40a5e3c640 1 Processor -- start 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.390+0000 7f40a5e3c640 1 -- start start 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.391+0000 7f40a5e3c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40a00d0070 0x7f40a015bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.391+0000 7f40a5e3c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40a00d1940 0x7f40a0151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.391+0000 7f40a5e3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a015f0d0 0x7f40a00b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.391+0000 7f40a5e3c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40a00d47f0 con 0x7f40a015f0d0 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.391+0000 7f40a5e3c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f40a00d4670 con 0x7f40a00d0070 2024-09-19T07:55:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.391+0000 7f40a5e3c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f40a00d4970 con 0x7f40a00d1940 2024-09-19T07:55:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.391+0000 7f409ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a015f0d0 0x7f40a00b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.391+0000 7f409ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a015f0d0 0x7f40a00b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46900/0 (socket says 172.21.15.16:46900) 2024-09-19T07:55:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.391+0000 7f409ffff640 1 -- 172.21.15.16:0/1528131762 learned_addr learned my addr 172.21.15.16:0/1528131762 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.391+0000 7f409f7fe640 1 --2- 172.21.15.16:0/1528131762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40a00d0070 0x7f40a015bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.391+0000 7f409effd640 1 --2- 172.21.15.16:0/1528131762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40a00d1940 0x7f40a0151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.391+0000 7f409ffff640 1 -- 172.21.15.16:0/1528131762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40a00d1940 msgr2=0x7f40a0151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.391+0000 7f409ffff640 1 --2- 172.21.15.16:0/1528131762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40a00d1940 0x7f40a0151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.391+0000 7f409ffff640 1 -- 172.21.15.16:0/1528131762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40a00d0070 msgr2=0x7f40a015bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.391+0000 7f409ffff640 1 --2- 172.21.15.16:0/1528131762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40a00d0070 0x7f40a015bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.391+0000 7f409ffff640 1 -- 172.21.15.16:0/1528131762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a01518b0 con 0x7f40a015f0d0 2024-09-19T07:55:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.391+0000 7f409effd640 1 --2- 172.21.15.16:0/1528131762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40a00d1940 0x7f40a0151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.391+0000 7f409ffff640 1 --2- 172.21.15.16:0/1528131762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a015f0d0 0x7f40a00b1930 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f4098009f40 tx=0x7f4098004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.391+0000 7f4087fff640 1 -- 172.21.15.16:0/1528131762 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4098036050 con 0x7f40a015f0d0 2024-09-19T07:55:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.392+0000 7f4087fff640 1 -- 172.21.15.16:0/1528131762 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4098042070 con 0x7f40a015f0d0 2024-09-19T07:55:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.392+0000 7f40a5e3c640 1 -- 172.21.15.16:0/1528131762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40a00b92b0 con 0x7f40a015f0d0 2024-09-19T07:55:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.392+0000 7f4087fff640 1 -- 172.21.15.16:0/1528131762 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4098009510 con 0x7f40a015f0d0 2024-09-19T07:55:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.392+0000 7f40a5e3c640 1 -- 172.21.15.16:0/1528131762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40a00b9870 con 0x7f40a015f0d0 2024-09-19T07:55:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.392+0000 7f40a5e3c640 1 -- 172.21.15.16:0/1528131762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40a00d1400 con 0x7f40a015f0d0 2024-09-19T07:55:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.393+0000 7f4087fff640 1 -- 172.21.15.16:0/1528131762 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4098007c50 con 0x7f40a015f0d0 2024-09-19T07:55:56.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.394+0000 7f4087fff640 1 --2- 172.21.15.16:0/1528131762 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f408c06c2e0 0x7f408c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:56.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.394+0000 7f4087fff640 1 -- 172.21.15.16:0/1528131762 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f40980ac430 con 0x7f40a015f0d0 2024-09-19T07:55:56.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.394+0000 7f409f7fe640 1 --2- 172.21.15.16:0/1528131762 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f408c06c2e0 0x7f408c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:56.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.394+0000 7f409f7fe640 1 --2- 172.21.15.16:0/1528131762 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f408c06c2e0 0x7f408c06e7a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f409000ab60 tx=0x7f4090008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:56.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.398+0000 7f4087fff640 1 -- 172.21.15.16:0/1528131762 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f409807aa10 con 0x7f40a015f0d0 2024-09-19T07:55:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.611+0000 7f40a5e3c640 1 -- 172.21.15.16:0/1528131762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f40a006cf30 con 0x7f40a015f0d0 2024-09-19T07:55:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.612+0000 7f4087fff640 1 -- 172.21.15.16:0/1528131762 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v92) ==== 53+0+73 (secure 0 0 0) 0x7f409807a310 con 0x7f40a015f0d0 2024-09-19T07:55:56.612 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:55:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.614+0000 7f40a5e3c640 1 -- 172.21.15.16:0/1528131762 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f408c06c2e0 msgr2=0x7f408c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.614+0000 7f40a5e3c640 1 --2- 172.21.15.16:0/1528131762 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f408c06c2e0 0x7f408c06e7a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f409000ab60 tx=0x7f4090008040 comp rx=0 tx=0).stop 2024-09-19T07:55:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.614+0000 7f40a5e3c640 1 -- 172.21.15.16:0/1528131762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a015f0d0 msgr2=0x7f40a00b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.614+0000 7f40a5e3c640 1 --2- 172.21.15.16:0/1528131762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a015f0d0 0x7f40a00b1930 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f4098009f40 tx=0x7f4098004a40 comp rx=0 tx=0).stop 2024-09-19T07:55:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.614+0000 7f40a5e3c640 1 -- 172.21.15.16:0/1528131762 shutdown_connections 2024-09-19T07:55:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.614+0000 7f40a5e3c640 1 --2- 172.21.15.16:0/1528131762 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f408c06c2e0 0x7f408c06e7a0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.614+0000 7f40a5e3c640 1 --2- 172.21.15.16:0/1528131762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a015f0d0 0x7f40a00b1930 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.614+0000 7f40a5e3c640 1 --2- 172.21.15.16:0/1528131762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40a00d1940 0x7f40a0151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.614+0000 7f40a5e3c640 1 --2- 172.21.15.16:0/1528131762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40a00d0070 0x7f40a015bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.614+0000 7f40a5e3c640 1 -- 172.21.15.16:0/1528131762 >> 172.21.15.16:0/1528131762 conn(0x7f40a009d760 msgr2=0x7f40a01614c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.614+0000 7f40a5e3c640 1 -- 172.21.15.16:0/1528131762 shutdown_connections 2024-09-19T07:55:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56.614+0000 7f40a5e3c640 1 -- 172.21.15.16:0/1528131762 wait complete. 2024-09-19T07:55:56.625 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:55:56.625 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:55:56.625 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T07:55:56.626 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T07:55:56.626 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:55:56.626 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:55:56.626 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:55:56.626 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:55:56.626 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:55:56.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:56.627 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T07:55:56.644 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:55:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:55:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:55:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:55:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:56.645 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:55:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T07:55:56.645 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:55:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T07:55:56.645 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:55:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T07:55:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T07:55:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42032sec preferred_lft 42032sec 2024-09-19T07:55:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T07:55:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:56.646 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:55:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T07:55:56.646 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:55:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:55:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::32:7bff:fe92:8895/64 scope link 2024-09-19T07:55:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:56.646 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:55:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether e6:fd:21:6a:43:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:55:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e4fd:21ff:fe6a:4369/64 scope link 2024-09-19T07:55:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:56.646 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:55:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:55:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::80f4:81ff:fe7d:b7f0/64 scope link 2024-09-19T07:55:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:56.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:56.647 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:55:56.708 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:55:56.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:56.710 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T07:55:56.766 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T07:55:56.767 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:55:56.767 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T07:55:56.767 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T07:55:56.767 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:55:56.767 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:55:56.767 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:55:56.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T07:55:56.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T07:55:56.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T07:55:56.768 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:55:56.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:55:56.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:55:56.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:56.897 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:55:56.913 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T07:55:56.913 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:55:56.914 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:55:56.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:56.914 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-19T07:55:56.980 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T07:55:56.980 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:55:56.981 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T07:55:57.037 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:55:57.038 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:55:57.038 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:55:57.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:55:57.160 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:55:57.218 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:55:57.218 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:55:57.218 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:55:57.218 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:55:57.218 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:55:57.218 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:55:57.218 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:55:57.218 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:55:57.218 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:55:57.218 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:55:57.219 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:55:57.219 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:55:57.219 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:55:57.219 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:55:57.219 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:55:57.219 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:55:57.219 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:55:57.219 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:55:57.219 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:55:57.219 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:55:57.219 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:55:57.219 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:55:57.219 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:55:57.219 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:55:57.220 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:55:57.220 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:55:57.220 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:55:57.220 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:55:57.220 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:55:57.220 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:55:57.220 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:55:57.220 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:55:57.220 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:55:57.220 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:55:57.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:57.221 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:55:57.274 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:55:57.274 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T07:55:57.316 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:55:57.341 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:55:57.399 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:55:57.399 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:55:57.399 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:55:57.399 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:55:57.399 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:55:57.399 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:55:57.399 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:55:57.400 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:55:57.400 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:55:57.400 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:55:57.400 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:55:57.400 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:55:57.400 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:55:57.400 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:55:57.400 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:55:57.400 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:55:57.400 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:55:57.400 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:55:57.400 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:55:57.400 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:55:57.401 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:55:57.401 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:55:57.401 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:55:57.401 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:55:57.401 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:55:57.401 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:55:57.401 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:55:57.401 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:55:57.401 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:55:57.401 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:55:57.401 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:55:57.401 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:55:57.401 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:55:57.402 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:55:57.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:57.402 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:55:57.403 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-19T07:55:57.400+0000 7f278400e480 -1 init, newargv = 0x560b27370be0 newargc=15 2024-09-19T07:55:57.403 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[79481]: starting ceph client 2024-09-19T07:55:57.405 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[79481]: starting fuse 2024-09-19T07:55:57.418 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:55:57.418 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-19T07:55:57.418 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:55:57.419 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:55:57.419 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:55:57.419 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:55:57.419 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:55:57.419 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:55:57.419 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:55:57.419 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:55:57.419 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:55:57.419 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:55:57.419 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:55:57.419 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:55:57.419 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:55:57.419 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:55:57.419 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:55:57.420 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:55:57.420 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:55:57.420 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:55:57.420 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:55:57.420 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:55:57.420 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:55:57.420 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:55:57.420 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:55:57.420 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:55:57.420 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:55:57.420 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:55:57.420 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:55:57.538 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.79481.asok 2024-09-19T07:55:57.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:55:57.549 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.79481.asok 2024-09-19T07:55:57.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:57.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.79481.asok status 2024-09-19T07:55:57.685 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:55:57.685 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:55:57.685 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:55:57.686 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:55:57.686 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:55:57.686 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:55:57.686 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:55:57.686 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "79481", 2024-09-19T07:55:57.686 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:55:57.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:57.686 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-19T07:55:57.686 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-19T07:55:57.686 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 8946, 2024-09-19T07:55:57.686 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:55:57.686 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:55:57.686 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:55:57.686 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 8946 2024-09-19T07:55:57.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:57.687 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:55:57.687 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:55:57.687 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:55:57.687 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3102101924 2024-09-19T07:55:57.687 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:55:57.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:57.687 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:55:57.687 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:55:57.687 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:55:57.687 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3102101924 2024-09-19T07:55:57.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:55:57.687 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.8946 192.168.0.1:0/3102101924", 2024-09-19T07:55:57.687 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/3102101924", 2024-09-19T07:55:57.688 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-19T07:55:57.688 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 92, 2024-09-19T07:55:57.688 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 152, 2024-09-19T07:55:57.688 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:55:57.688 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:55:57.688 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:55:57.688 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:55:57.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:57.694 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-19T07:55:57.694 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T07:55:57.710 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-19T07:55:57.711 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T07:55:57.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:57.711 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:55:57.778 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:55:57.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.920+0000 7f51ce9b9640 1 Processor -- start 2024-09-19T07:55:57.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.920+0000 7f51ce9b9640 1 -- start start 2024-09-19T07:55:57.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.921+0000 7f51ce9b9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f51c806cd20 0x7f51c806d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.921+0000 7f51ce9b9640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f51c8161eb0 0x7f51c81622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.921+0000 7f51ce9b9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51c806d640 con 0x7f51c8143a20 2024-09-19T07:55:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.921+0000 7f51ce9b9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f51c8162e50 con 0x7f51c806cd20 2024-09-19T07:55:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.921+0000 7f51ce9b9640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f51c8142150 con 0x7f51c8161eb0 2024-09-19T07:55:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.921+0000 7f51c7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f51c806cd20 0x7f51c806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.921+0000 7f51ccf56640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f51c8143a20 0x7f51c8143eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49824/0 (socket says 172.21.15.16:49824) 2024-09-19T07:55:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.921+0000 7f51ccf56640 1 -- 172.21.15.16:0/2422175995 learned_addr learned my addr 172.21.15.16:0/2422175995 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.921+0000 7f51c7fff640 1 -- 172.21.15.16:0/2422175995 >> v2:172.21.15.146:3300/0 conn(0x7f51c8161eb0 msgr2=0x7f51c81622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.921+0000 7f51c77fe640 1 --2- 172.21.15.16:0/2422175995 >> v2:172.21.15.146:3300/0 conn(0x7f51c8161eb0 0x7f51c81622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.921+0000 7f51c7fff640 1 --2- 172.21.15.16:0/2422175995 >> v2:172.21.15.146:3300/0 conn(0x7f51c8161eb0 0x7f51c81622f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.921+0000 7f51c7fff640 1 -- 172.21.15.16:0/2422175995 >> v1:172.21.15.16:6789/0 conn(0x7f51c8143a20 legacy=0x7f51c8143eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.921+0000 7f51c6ffd640 1 -- 172.21.15.16:0/2422175995 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3273904179 0 0) 0x7f51c806d640 con 0x7f51c8143a20 2024-09-19T07:55:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.921+0000 7f51c7fff640 1 -- 172.21.15.16:0/2422175995 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c8162b70 con 0x7f51c806cd20 2024-09-19T07:55:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.922+0000 7f51c7fff640 1 --2- 172.21.15.16:0/2422175995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51c806cd20 0x7f51c806d100 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f51c806ef00 tx=0x7f51b802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcdaf05ea086dbf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.922+0000 7f51c6ffd640 1 -- 172.21.15.16:0/2422175995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8037070 con 0x7f51c806cd20 2024-09-19T07:55:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.922+0000 7f51c6ffd640 1 -- 172.21.15.16:0/2422175995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51b8032440 con 0x7f51c806cd20 2024-09-19T07:55:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.922+0000 7f51c6ffd640 1 -- 172.21.15.16:0/2422175995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b802ae60 con 0x7f51c806cd20 2024-09-19T07:55:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.923+0000 7f51ce9b9640 1 -- 172.21.15.16:0/2422175995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51c806cd20 msgr2=0x7f51c806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.923+0000 7f51ce9b9640 1 --2- 172.21.15.16:0/2422175995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51c806cd20 0x7f51c806d100 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f51c806ef00 tx=0x7f51b802a110 comp rx=0 tx=0).stop 2024-09-19T07:55:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.923+0000 7f51ce9b9640 1 -- 172.21.15.16:0/2422175995 shutdown_connections 2024-09-19T07:55:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.923+0000 7f51ce9b9640 1 --2- 172.21.15.16:0/2422175995 >> v2:172.21.15.146:3300/0 conn(0x7f51c8161eb0 0x7f51c81622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.923+0000 7f51ce9b9640 1 --2- 172.21.15.16:0/2422175995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51c806cd20 0x7f51c806d100 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.923+0000 7f51ce9b9640 1 -- 172.21.15.16:0/2422175995 >> 172.21.15.16:0/2422175995 conn(0x7f51c80b3ed0 msgr2=0x7f51c80b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.923+0000 7f51ce9b9640 1 -- 172.21.15.16:0/2422175995 shutdown_connections 2024-09-19T07:55:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.923+0000 7f51ce9b9640 1 -- 172.21.15.16:0/2422175995 wait complete. 2024-09-19T07:55:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.924+0000 7f51ce9b9640 1 Processor -- start 2024-09-19T07:55:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.924+0000 7f51ce9b9640 1 -- start start 2024-09-19T07:55:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.925+0000 7f51ce9b9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51c806cd20 0x7f51c8140b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.925+0000 7f51ce9b9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51c8143a20 0x7f51c81405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.925+0000 7f51ce9b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51c8161eb0 0x7f51c81e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.925+0000 7f51ce9b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f51c8162830 con 0x7f51c8161eb0 2024-09-19T07:55:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.925+0000 7f51ce9b9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f51c8141de0 con 0x7f51c806cd20 2024-09-19T07:55:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.925+0000 7f51ce9b9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f51c8141600 con 0x7f51c8143a20 2024-09-19T07:55:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.925+0000 7f51ccf56640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51c8161eb0 0x7f51c81e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.925+0000 7f51ccf56640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51c8161eb0 0x7f51c81e88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46908/0 (socket says 172.21.15.16:46908) 2024-09-19T07:55:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.925+0000 7f51ccf56640 1 -- 172.21.15.16:0/3872754077 learned_addr learned my addr 172.21.15.16:0/3872754077 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.925+0000 7f51c7fff640 1 --2- 172.21.15.16:0/3872754077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51c806cd20 0x7f51c8140b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.925+0000 7f51c77fe640 1 --2- 172.21.15.16:0/3872754077 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51c8143a20 0x7f51c81405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.925+0000 7f51ccf56640 1 -- 172.21.15.16:0/3872754077 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51c8143a20 msgr2=0x7f51c81405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.925+0000 7f51ccf56640 1 --2- 172.21.15.16:0/3872754077 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51c8143a20 0x7f51c81405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.925+0000 7f51ccf56640 1 -- 172.21.15.16:0/3872754077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51c806cd20 msgr2=0x7f51c8140b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.925+0000 7f51ccf56640 1 --2- 172.21.15.16:0/3872754077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51c806cd20 0x7f51c8140b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.925+0000 7f51ccf56640 1 -- 172.21.15.16:0/3872754077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c81e9e50 con 0x7f51c8161eb0 2024-09-19T07:55:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.926+0000 7f51ccf56640 1 --2- 172.21.15.16:0/3872754077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51c8161eb0 0x7f51c81e88a0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f51c000c700 tx=0x7f51c000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.926+0000 7f51c4ff9640 1 -- 172.21.15.16:0/3872754077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0018070 con 0x7f51c8161eb0 2024-09-19T07:55:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.926+0000 7f51c4ff9640 1 -- 172.21.15.16:0/3872754077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c000ce10 con 0x7f51c8161eb0 2024-09-19T07:55:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.926+0000 7f51ce9b9640 1 -- 172.21.15.16:0/3872754077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c81ea130 con 0x7f51c8161eb0 2024-09-19T07:55:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.926+0000 7f51c4ff9640 1 -- 172.21.15.16:0/3872754077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c000a860 con 0x7f51c8161eb0 2024-09-19T07:55:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.926+0000 7f51ce9b9640 1 -- 172.21.15.16:0/3872754077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c81e8de0 con 0x7f51c8161eb0 2024-09-19T07:55:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.927+0000 7f51ce9b9640 1 -- 172.21.15.16:0/3872754077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c81622f0 con 0x7f51c8161eb0 2024-09-19T07:55:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.927+0000 7f51c4ff9640 1 -- 172.21.15.16:0/3872754077 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f51c001b070 con 0x7f51c8161eb0 2024-09-19T07:55:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.928+0000 7f51c4ff9640 1 --2- 172.21.15.16:0/3872754077 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f51b406c2b0 0x7f51b406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.928+0000 7f51c4ff9640 1 -- 172.21.15.16:0/3872754077 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f51c008ab20 con 0x7f51c8161eb0 2024-09-19T07:55:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.928+0000 7f51c7fff640 1 --2- 172.21.15.16:0/3872754077 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f51b406c2b0 0x7f51b406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.929+0000 7f51c7fff640 1 --2- 172.21.15.16:0/3872754077 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f51b406c2b0 0x7f51b406e770 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f51b8034000 tx=0x7f51b802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:57.932+0000 7f51c4ff9640 1 -- 172.21.15.16:0/3872754077 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f51c0058ab0 con 0x7f51c8161eb0 2024-09-19T07:55:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:58.144+0000 7f51ce9b9640 1 -- 172.21.15.16:0/3872754077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f51c80690c0 con 0x7f51c8161eb0 2024-09-19T07:55:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:58.145+0000 7f51c4ff9640 1 -- 172.21.15.16:0/3872754077 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v92) ==== 53+0+73 (secure 0 0 0) 0x7f51c005cdb0 con 0x7f51c8161eb0 2024-09-19T07:55:58.145 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:55:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:58.147+0000 7f51ce9b9640 1 -- 172.21.15.16:0/3872754077 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f51b406c2b0 msgr2=0x7f51b406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:58.147+0000 7f51ce9b9640 1 --2- 172.21.15.16:0/3872754077 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f51b406c2b0 0x7f51b406e770 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f51b8034000 tx=0x7f51b802e000 comp rx=0 tx=0).stop 2024-09-19T07:55:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:58.147+0000 7f51ce9b9640 1 -- 172.21.15.16:0/3872754077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51c8161eb0 msgr2=0x7f51c81e88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:58.147+0000 7f51ce9b9640 1 --2- 172.21.15.16:0/3872754077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51c8161eb0 0x7f51c81e88a0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f51c000c700 tx=0x7f51c000cbc0 comp rx=0 tx=0).stop 2024-09-19T07:55:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:58.147+0000 7f51ce9b9640 1 -- 172.21.15.16:0/3872754077 shutdown_connections 2024-09-19T07:55:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:58.147+0000 7f51ce9b9640 1 --2- 172.21.15.16:0/3872754077 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f51b406c2b0 0x7f51b406e770 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:58.147+0000 7f51ce9b9640 1 --2- 172.21.15.16:0/3872754077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51c8161eb0 0x7f51c81e88a0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:58.147+0000 7f51ce9b9640 1 --2- 172.21.15.16:0/3872754077 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51c8143a20 0x7f51c81405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:58.147+0000 7f51ce9b9640 1 --2- 172.21.15.16:0/3872754077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51c806cd20 0x7f51c8140b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:58.147+0000 7f51ce9b9640 1 -- 172.21.15.16:0/3872754077 >> 172.21.15.16:0/3872754077 conn(0x7f51c80b3ed0 msgr2=0x7f51c8161030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:58.147+0000 7f51ce9b9640 1 -- 172.21.15.16:0/3872754077 shutdown_connections 2024-09-19T07:55:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:58.147+0000 7f51ce9b9640 1 -- 172.21.15.16:0/3872754077 wait complete. 2024-09-19T07:55:58.158 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:55:58.158 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:55:58.158 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-09-19T07:55:58.158 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T07:55:58.158 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T07:55:58.158 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:55:58.158 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:55:58.158 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:55:58.158 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:55:58.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:58.159 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-09-19T07:55:58.179 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:55:58.179 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:55:58.179 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:55:58.179 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:58.179 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-09-19T07:55:58.179 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:58.179 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:55:58.179 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:58.179 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2024-09-19T07:55:58.180 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:55:58.180 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:58.180 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2024-09-19T07:55:58.180 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:55:58.180 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:58.180 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:55:58.180 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:58.180 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T07:55:58.180 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 41998sec preferred_lft 41998sec 2024-09-19T07:55:58.180 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link noprefixroute 2024-09-19T07:55:58.180 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:58.181 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:55:58.181 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:55:58.181 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:55:58.181 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:58.181 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::e47f:32ff:fe68:31ca/64 scope link 2024-09-19T07:55:58.181 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:58.181 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:55:58.181 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether f6:1f:22:44:9d:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:55:58.181 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::f41f:22ff:fe44:9dcd/64 scope link 2024-09-19T07:55:58.181 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:58.181 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:55:58.181 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 86:59:ee:21:a1:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:55:58.182 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::8459:eeff:fe21:a113/64 scope link 2024-09-19T07:55:58.182 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:55:58.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:58.182 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:55:58.247 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-09-19T07:55:58.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:58.250 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-09-19T07:55:58.308 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-09-19T07:55:58.308 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:55:58.308 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T07:55:58.309 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T07:55:58.309 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:55:58.309 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:55:58.309 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-09-19T07:55:58.309 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T07:55:58.310 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T07:55:58.310 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T07:55:58.310 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-09-19T07:55:58.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:55:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:55:58.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:55:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:55:58.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:58.442 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-09-19T07:55:58.458 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T07:55:58.458 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T07:55:58.458 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:55:58.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:58.458 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2024-09-19T07:55:58.523 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T07:55:58.523 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:55:58.523 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T07:55:58.580 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T07:55:58.580 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:55:58.580 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:55:58.635 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:55:58.700 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:55:58.756 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:55:58.757 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:55:58.757 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:55:58.757 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:55:58.757 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:55:58.757 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:55:58.757 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:55:58.757 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:55:58.757 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:55:58.757 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:55:58.757 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:55:58.757 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:55:58.757 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:55:58.758 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:55:58.758 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:55:58.758 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:55:58.758 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:55:58.758 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:55:58.758 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:55:58.758 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:55:58.758 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:55:58.758 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:55:58.758 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:55:58.758 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:55:58.758 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:55:58.758 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:55:58.758 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:55:58.759 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:55:58.759 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:55:58.759 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:55:58.759 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:55:58.759 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:55:58.759 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:55:58.759 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:55:58.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:58.759 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:55:58.812 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:55:58.813 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T07:55:58.855 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-09-19T07:55:58.880 DEBUG:teuthology.orchestra.run.smithi146:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:55:58.938 INFO:teuthology.orchestra.run.smithi146.stdout:/proc 2024-09-19T07:55:58.938 INFO:teuthology.orchestra.run.smithi146.stdout:/sys 2024-09-19T07:55:58.939 INFO:teuthology.orchestra.run.smithi146.stdout:/dev 2024-09-19T07:55:58.939 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/security 2024-09-19T07:55:58.939 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/shm 2024-09-19T07:55:58.939 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/pts 2024-09-19T07:55:58.939 INFO:teuthology.orchestra.run.smithi146.stdout:/run 2024-09-19T07:55:58.939 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/cgroup 2024-09-19T07:55:58.940 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/pstore 2024-09-19T07:55:58.940 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/bpf 2024-09-19T07:55:58.940 INFO:teuthology.orchestra.run.smithi146.stdout:/ 2024-09-19T07:55:58.940 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/selinux 2024-09-19T07:55:58.940 INFO:teuthology.orchestra.run.smithi146.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:55:58.941 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/hugepages 2024-09-19T07:55:58.941 INFO:teuthology.orchestra.run.smithi146.stdout:/dev/mqueue 2024-09-19T07:55:58.941 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/debug 2024-09-19T07:55:58.941 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/tracing 2024-09-19T07:55:58.941 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/fs/fuse/connections 2024-09-19T07:55:58.942 INFO:teuthology.orchestra.run.smithi146.stdout:/sys/kernel/config 2024-09-19T07:55:58.942 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:55:58.942 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:55:58.942 INFO:teuthology.orchestra.run.smithi146.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:55:58.942 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:55:58.943 INFO:teuthology.orchestra.run.smithi146.stdout:/run/user/1000 2024-09-19T07:55:58.943 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph 2024-09-19T07:55:58.943 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T07:55:58.943 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T07:55:58.943 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T07:55:58.943 INFO:teuthology.orchestra.run.smithi146.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T07:55:58.944 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns 2024-09-19T07:55:58.944 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:55:58.944 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:55:58.944 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:55:58.944 INFO:teuthology.orchestra.run.smithi146.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:55:58.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:58.945 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-09-19T07:55:58.946 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2024-09-19T07:55:58.938+0000 7f1174d2a480 -1 init, newargv = 0x5591a0e3c670 newargc=15 2024-09-19T07:55:58.947 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[50378]: starting ceph client 2024-09-19T07:55:58.947 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[50378]: starting fuse 2024-09-19T07:55:58.961 INFO:teuthology.orchestra.run.smithi146.stdout:42 2024-09-19T07:55:58.962 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-19T07:55:58.962 DEBUG:teuthology.orchestra.run.smithi146:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:55:58.962 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2024-09-19T07:55:58.962 DEBUG:teuthology.orchestra.run.smithi146:> import re 2024-09-19T07:55:58.962 DEBUG:teuthology.orchestra.run.smithi146:> import os 2024-09-19T07:55:58.962 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2024-09-19T07:55:58.962 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:55:58.962 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2024-09-19T07:55:58.962 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T07:55:58.962 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2024-09-19T07:55:58.962 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T07:55:58.963 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:55:58.963 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2024-09-19T07:55:58.963 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2024-09-19T07:55:58.963 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2024-09-19T07:55:58.963 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2024-09-19T07:55:58.963 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:55:58.963 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2024-09-19T07:55:58.963 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:55:58.963 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:55:58.963 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:55:58.963 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2024-09-19T07:55:58.963 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2024-09-19T07:55:58.963 DEBUG:teuthology.orchestra.run.smithi146:> return f 2024-09-19T07:55:58.963 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:55:58.964 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:55:58.964 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2024-09-19T07:55:58.964 DEBUG:teuthology.orchestra.run.smithi146:> ' 2024-09-19T07:55:59.079 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.50378.asok 2024-09-19T07:55:59.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:55:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:55:59.089 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.50378.asok 2024-09-19T07:55:59.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:59.089 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.50378.asok status 2024-09-19T07:55:59.223 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-19T07:55:59.223 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2024-09-19T07:55:59.223 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:55:59.223 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:55:59.223 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2024-09-19T07:55:59.223 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2024-09-19T07:55:59.223 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T07:55:59.223 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "50378", 2024-09-19T07:55:59.223 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2024-09-19T07:55:59.223 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:55:59.223 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2024-09-19T07:55:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2024-09-19T07:55:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 7166, 2024-09-19T07:55:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2024-09-19T07:55:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2024-09-19T07:55:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2024-09-19T07:55:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 7166 2024-09-19T07:55:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:55:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:55:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:55:59.224 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:55:59.225 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 403965297 2024-09-19T07:55:59.225 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-19T07:55:59.225 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:55:59.225 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-09-19T07:55:59.225 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-09-19T07:55:59.225 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:55:59.225 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 403965297 2024-09-19T07:55:59.225 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-19T07:55:59.225 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.7166 192.168.0.1:0/403965297", 2024-09-19T07:55:59.225 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/403965297", 2024-09-19T07:55:59.225 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2024-09-19T07:55:59.225 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 92, 2024-09-19T07:55:59.226 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 152, 2024-09-19T07:55:59.226 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:55:59.226 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2024-09-19T07:55:59.226 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2024-09-19T07:55:59.226 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-19T07:55:59.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:59.231 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2024-09-19T07:55:59.231 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T07:55:59.248 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2024-09-19T07:55:59.248 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T07:55:59.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:59.248 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T07:55:59.314 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T07:55:59.317 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T07:55:59.367 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-09-19T07:55:59.368 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2024-09-19T07:55:59.368 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T07:55:59.368 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2024-09-19T07:55:59.368 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.368 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2024-09-19T07:55:59.368 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.368 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2024-09-19T07:55:59.368 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.369 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2024-09-19T07:55:59.369 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2024-09-19T07:55:59.369 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.369 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2024-09-19T07:55:59.369 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2024-09-19T07:55:59.369 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.369 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T07:55:59.369 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2024-09-19T07:55:59.369 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.369 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T07:55:59.369 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T07:55:59.369 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.369 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2024-09-19T07:55:59.370 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.370 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2024-09-19T07:55:59.370 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2024-09-19T07:55:59.370 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2024-09-19T07:55:59.370 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2024-09-19T07:55:59.370 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2024-09-19T07:55:59.370 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2024-09-19T07:55:59.370 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2024-09-19T07:55:59.370 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T07:55:59.370 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T07:55:59.370 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2024-09-19T07:55:59.370 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.370 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T07:55:59.370 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.370 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2024-09-19T07:55:59.371 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.371 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2024-09-19T07:55:59.371 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.371 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2024-09-19T07:55:59.371 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2024-09-19T07:55:59.371 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2024-09-19T07:55:59.371 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-09-19T07:55:59.371 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb 2024-09-19T07:55:59.371 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.372 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2024-09-19T07:55:59.372 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2024-09-19T07:55:59.372 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.372 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2024-09-19T07:55:59.372 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2024-09-19T07:55:59.372 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2024-09-19T07:55:59.372 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.372 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2024-09-19T07:55:59.372 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2024-09-19T07:55:59.372 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.372 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2024-09-19T07:55:59.372 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2024-09-19T07:55:59.372 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2024-09-19T07:55:59.373 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2024-09-19T07:55:59.373 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.373 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2024-09-19T07:55:59.373 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2024-09-19T07:55:59.373 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.373 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2024-09-19T07:55:59.373 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.373 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2024-09-19T07:55:59.373 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.373 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2024-09-19T07:55:59.373 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.373 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2024-09-19T07:55:59.374 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2024-09-19T07:55:59.374 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.374 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2024-09-19T07:55:59.374 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2024-09-19T07:55:59.374 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2024-09-19T07:55:59.374 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2024-09-19T07:55:59.374 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2024-09-19T07:55:59.374 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2024-09-19T07:55:59.374 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:55:59.374 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2024-09-19T07:55:59.374 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2024-09-19T07:55:59.374 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T07:55:59.374 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2024-09-19T07:55:59.375 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2024-09-19T07:55:59.375 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2024-09-19T07:55:59.375 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2024-09-19T07:55:59.375 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-09-19T07:55:59.375 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.375 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2024-09-19T07:55:59.375 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:55:59.375 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2024-09-19T07:55:59.375 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-19T07:55:59.375 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-19T07:55:59.375 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:55:59.375 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:55:59.375 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2024-09-19T07:55:59.375 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-09-19T07:55:59.376 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.376 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2024-09-19T07:55:59.376 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:55:59.376 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-19T07:55:59.376 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2024-09-19T07:55:59.376 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-19T07:55:59.376 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2024-09-19T07:55:59.376 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2024-09-19T07:55:59.376 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:55:59.376 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2024-09-19T07:55:59.376 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2024-09-19T07:55:59.377 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:55:59.377 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.377 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T07:55:59.377 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T07:55:59.377 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T07:55:59.377 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.377 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T07:55:59.377 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T07:55:59.377 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T07:55:59.377 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.377 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2024-09-19T07:55:59.377 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2024-09-19T07:55:59.377 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.378 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2024-09-19T07:55:59.378 INFO:teuthology.orchestra.run.smithi016.stdout: mon down mkfs grace = 300 2024-09-19T07:55:59.378 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2024-09-19T07:55:59.378 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on pool no app = False 2024-09-19T07:55:59.378 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:55:59.378 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2024-09-19T07:55:59.378 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2024-09-19T07:55:59.378 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2024-09-19T07:55:59.378 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2024-09-19T07:55:59.378 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T07:55:59.378 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T07:55:59.378 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2024-09-19T07:55:59.379 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-09-19T07:55:59.379 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:55:59.379 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2024-09-19T07:55:59.379 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-09-19T07:55:59.379 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-09-19T07:55:59.379 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2024-09-19T07:55:59.379 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2024-09-19T07:55:59.379 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2024-09-19T07:55:59.379 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-09-19T07:55:59.379 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-09-19T07:55:59.379 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-09-19T07:55:59.380 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-09-19T07:55:59.380 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2024-09-19T07:55:59.380 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2024-09-19T07:55:59.380 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2024-09-19T07:55:59.380 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:55:59.380 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2024-09-19T07:55:59.380 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2024-09-19T07:55:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2024-09-19T07:55:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = True 2024-09-19T07:55:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-09-19T07:55:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-09-19T07:55:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-09-19T07:55:59.381 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-19T07:55:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: client inject release failure = true 2024-09-19T07:55:59.397 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2024-09-19T07:55:59.397 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2024-09-19T07:55:59.397 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T07:55:59.397 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2024-09-19T07:55:59.397 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.397 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2024-09-19T07:55:59.397 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.397 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2024-09-19T07:55:59.398 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.398 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2024-09-19T07:55:59.398 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2024-09-19T07:55:59.398 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.398 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2024-09-19T07:55:59.398 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2024-09-19T07:55:59.398 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.398 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T07:55:59.398 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2024-09-19T07:55:59.398 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.398 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T07:55:59.399 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T07:55:59.399 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.399 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2024-09-19T07:55:59.399 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.399 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2024-09-19T07:55:59.399 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2024-09-19T07:55:59.399 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2024-09-19T07:55:59.399 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2024-09-19T07:55:59.399 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2024-09-19T07:55:59.399 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2024-09-19T07:55:59.399 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2024-09-19T07:55:59.399 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T07:55:59.399 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T07:55:59.400 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2024-09-19T07:55:59.400 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.400 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T07:55:59.400 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.400 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2024-09-19T07:55:59.400 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.400 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2024-09-19T07:55:59.400 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.400 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2024-09-19T07:55:59.400 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2024-09-19T07:55:59.400 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2024-09-19T07:55:59.400 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.16,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-09-19T07:55:59.400 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = 6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb 2024-09-19T07:55:59.400 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.401 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2024-09-19T07:55:59.401 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2024-09-19T07:55:59.401 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.401 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2024-09-19T07:55:59.401 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2024-09-19T07:55:59.401 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2024-09-19T07:55:59.401 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.401 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2024-09-19T07:55:59.401 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2024-09-19T07:55:59.401 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.401 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2024-09-19T07:55:59.401 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2024-09-19T07:55:59.401 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2024-09-19T07:55:59.401 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug trim objects = true 2024-09-19T07:55:59.402 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.402 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2024-09-19T07:55:59.402 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2024-09-19T07:55:59.402 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.402 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2024-09-19T07:55:59.402 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.402 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2024-09-19T07:55:59.402 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.402 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2024-09-19T07:55:59.402 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.402 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2024-09-19T07:55:59.402 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2024-09-19T07:55:59.402 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.403 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2024-09-19T07:55:59.403 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2024-09-19T07:55:59.403 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2024-09-19T07:55:59.403 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2024-09-19T07:55:59.403 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 20 2024-09-19T07:55:59.403 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 20 2024-09-19T07:55:59.403 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:55:59.403 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2024-09-19T07:55:59.403 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 10 2024-09-19T07:55:59.403 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T07:55:59.404 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2024-09-19T07:55:59.404 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2024-09-19T07:55:59.404 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2024-09-19T07:55:59.404 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2024-09-19T07:55:59.404 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-09-19T07:55:59.404 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.404 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2024-09-19T07:55:59.404 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:55:59.404 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2024-09-19T07:55:59.404 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-19T07:55:59.404 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-19T07:55:59.404 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:55:59.404 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:55:59.405 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2024-09-19T07:55:59.405 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-09-19T07:55:59.405 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.405 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2024-09-19T07:55:59.405 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:55:59.405 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-19T07:55:59.405 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2024-09-19T07:55:59.405 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-19T07:55:59.405 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2024-09-19T07:55:59.405 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2024-09-19T07:55:59.405 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:55:59.405 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2024-09-19T07:55:59.405 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2024-09-19T07:55:59.405 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:55:59.406 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.406 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T07:55:59.406 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T07:55:59.406 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T07:55:59.406 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.406 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T07:55:59.406 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T07:55:59.406 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T07:55:59.406 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.406 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2024-09-19T07:55:59.406 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2024-09-19T07:55:59.406 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.406 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2024-09-19T07:55:59.407 INFO:teuthology.orchestra.run.smithi146.stdout: mon down mkfs grace = 300 2024-09-19T07:55:59.407 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2024-09-19T07:55:59.407 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on pool no app = False 2024-09-19T07:55:59.407 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:55:59.407 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2024-09-19T07:55:59.407 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2024-09-19T07:55:59.407 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2024-09-19T07:55:59.407 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2024-09-19T07:55:59.407 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T07:55:59.407 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T07:55:59.407 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2024-09-19T07:55:59.407 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-09-19T07:55:59.407 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:55:59.407 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = "" 2024-09-19T07:55:59.407 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-09-19T07:55:59.408 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-09-19T07:55:59.408 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2024-09-19T07:55:59.408 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2024-09-19T07:55:59.408 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2024-09-19T07:55:59.408 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2024-09-19T07:55:59.408 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2024-09-19T07:55:59.408 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2024-09-19T07:55:59.408 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2024-09-19T07:55:59.408 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2024-09-19T07:55:59.408 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2024-09-19T07:55:59.409 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2024-09-19T07:55:59.409 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:55:59.409 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2024-09-19T07:55:59.409 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2024-09-19T07:55:59.409 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2024-09-19T07:55:59.409 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2024-09-19T07:55:59.409 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-09-19T07:55:59.410 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-09-19T07:55:59.410 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-09-19T07:55:59.410 INFO:teuthology.orchestra.run.smithi146.stdout:[client.0] 2024-09-19T07:55:59.410 INFO:teuthology.orchestra.run.smithi146.stdout: client inject release failure = true 2024-09-19T07:55:59.411 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:55:59.412 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:55:59.412 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:55:59.442 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:55:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.591+0000 7f9f052d2640 1 Processor -- start 2024-09-19T07:55:59.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.591+0000 7f9f052d2640 1 -- start start 2024-09-19T07:55:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.592+0000 7f9f052d2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f000d2510 0x7f9f000d2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.592+0000 7f9f052d2640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f000cfdf0 con 0x7f9f0015e600 2024-09-19T07:55:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.592+0000 7f9f052d2640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f000b6aa0 con 0x7f9f000d0c40 2024-09-19T07:55:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.592+0000 7f9f052d2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9f000d3e40 con 0x7f9f000d2510 2024-09-19T07:55:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.592+0000 7f9eff7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f000d2510 0x7f9f000d2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.592+0000 7f9ef7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9f0015e600 0x7f9f000cf6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49826/0 (socket says 172.21.15.16:49826) 2024-09-19T07:55:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.592+0000 7f9ef7fff640 1 -- 172.21.15.16:0/4108041768 learned_addr learned my addr 172.21.15.16:0/4108041768 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:55:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.592+0000 7f9eff7fe640 1 -- 172.21.15.16:0/4108041768 >> v1:172.21.15.146:6789/0 conn(0x7f9f000d0c40 legacy=0x7f9f000d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:55:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.592+0000 7f9eff7fe640 1 -- 172.21.15.16:0/4108041768 >> v1:172.21.15.16:6789/0 conn(0x7f9f0015e600 legacy=0x7f9f000cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.592+0000 7f9eff7fe640 1 -- 172.21.15.16:0/4108041768 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f000cec00 con 0x7f9f000d2510 2024-09-19T07:55:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.592+0000 7f9efe7fc640 1 -- 172.21.15.16:0/4108041768 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1097222818 0 0) 0x7f9f000cfdf0 con 0x7f9f0015e600 2024-09-19T07:55:59.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.593+0000 7f9eff7fe640 1 --2- 172.21.15.16:0/4108041768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f000d2510 0x7f9f000d2930 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8009d90 tx=0x7f9ef802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6937f77a0e0bdb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:59.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.593+0000 7f9efe7fc640 1 -- 172.21.15.16:0/4108041768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8031600 con 0x7f9f000d2510 2024-09-19T07:55:59.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.593+0000 7f9efe7fc640 1 -- 172.21.15.16:0/4108041768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ef80317a0 con 0x7f9f000d2510 2024-09-19T07:55:59.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.593+0000 7f9efe7fc640 1 -- 172.21.15.16:0/4108041768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8031b70 con 0x7f9f000d2510 2024-09-19T07:55:59.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.594+0000 7f9f052d2640 1 -- 172.21.15.16:0/4108041768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f000d2510 msgr2=0x7f9f000d2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:59.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.594+0000 7f9f052d2640 1 --2- 172.21.15.16:0/4108041768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f000d2510 0x7f9f000d2930 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8009d90 tx=0x7f9ef802a610 comp rx=0 tx=0).stop 2024-09-19T07:55:59.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.594+0000 7f9f052d2640 1 -- 172.21.15.16:0/4108041768 shutdown_connections 2024-09-19T07:55:59.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.594+0000 7f9f052d2640 1 --2- 172.21.15.16:0/4108041768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f000d2510 0x7f9f000d2930 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:59.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.594+0000 7f9f052d2640 1 -- 172.21.15.16:0/4108041768 >> 172.21.15.16:0/4108041768 conn(0x7f9f0009d850 msgr2=0x7f9f0008bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:59.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.594+0000 7f9f052d2640 1 -- 172.21.15.16:0/4108041768 shutdown_connections 2024-09-19T07:55:59.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.594+0000 7f9f052d2640 1 -- 172.21.15.16:0/4108041768 wait complete. 2024-09-19T07:55:59.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.595+0000 7f9f052d2640 1 Processor -- start 2024-09-19T07:55:59.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.595+0000 7f9f052d2640 1 -- start start 2024-09-19T07:55:59.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.596+0000 7f9f052d2640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f000d0c40 0x7f9f001f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:59.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.596+0000 7f9f052d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f000d2510 0x7f9f000c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:59.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.596+0000 7f9f052d2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f0015e600 0x7f9f000c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:59.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.596+0000 7f9f052d2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f000b6d80 con 0x7f9f000d2510 2024-09-19T07:55:59.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.596+0000 7f9f052d2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9f000d2e70 con 0x7f9f0015e600 2024-09-19T07:55:59.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.596+0000 7f9f052d2640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9f000b5d50 con 0x7f9f000d0c40 2024-09-19T07:55:59.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.596+0000 7f9ef7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f000d2510 0x7f9f000c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:59.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.596+0000 7f9ef7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f000d2510 0x7f9f000c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46924/0 (socket says 172.21.15.16:46924) 2024-09-19T07:55:59.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.596+0000 7f9ef7fff640 1 -- 172.21.15.16:0/1407571095 learned_addr learned my addr 172.21.15.16:0/1407571095 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:55:59.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.596+0000 7f9eff7fe640 1 --2- 172.21.15.16:0/1407571095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f0015e600 0x7f9f000c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:59.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.596+0000 7f9efeffd640 1 --2- 172.21.15.16:0/1407571095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f000d0c40 0x7f9f001f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:59.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.596+0000 7f9ef7fff640 1 -- 172.21.15.16:0/1407571095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f000d0c40 msgr2=0x7f9f001f37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:59.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.596+0000 7f9ef7fff640 1 --2- 172.21.15.16:0/1407571095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f000d0c40 0x7f9f001f37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:59.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.596+0000 7f9ef7fff640 1 -- 172.21.15.16:0/1407571095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f0015e600 msgr2=0x7f9f000c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:59.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.596+0000 7f9ef7fff640 1 --2- 172.21.15.16:0/1407571095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f0015e600 0x7f9f000c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:59.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.596+0000 7f9ef7fff640 1 -- 172.21.15.16:0/1407571095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f001f7e30 con 0x7f9f000d2510 2024-09-19T07:55:59.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.596+0000 7f9efeffd640 1 --2- 172.21.15.16:0/1407571095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f000d0c40 0x7f9f001f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:59.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.596+0000 7f9eff7fe640 1 --2- 172.21.15.16:0/1407571095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f0015e600 0x7f9f000c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:55:59.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.597+0000 7f9ef7fff640 1 --2- 172.21.15.16:0/1407571095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f000d2510 0x7f9f000c8040 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800c410 tx=0x7f9ee800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:59.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.597+0000 7f9ef77fe640 1 -- 172.21.15.16:0/1407571095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee80074e0 con 0x7f9f000d2510 2024-09-19T07:55:59.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.597+0000 7f9ef77fe640 1 -- 172.21.15.16:0/1407571095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ee80047e0 con 0x7f9f000d2510 2024-09-19T07:55:59.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.597+0000 7f9f052d2640 1 -- 172.21.15.16:0/1407571095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f001f8110 con 0x7f9f000d2510 2024-09-19T07:55:59.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.597+0000 7f9ef77fe640 1 -- 172.21.15.16:0/1407571095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8004bd0 con 0x7f9f000d2510 2024-09-19T07:55:59.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.597+0000 7f9f052d2640 1 -- 172.21.15.16:0/1407571095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f0015b960 con 0x7f9f000d2510 2024-09-19T07:55:59.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.598+0000 7f9f052d2640 1 -- 172.21.15.16:0/1407571095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f000b5610 con 0x7f9f000d2510 2024-09-19T07:55:59.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.598+0000 7f9ef77fe640 1 -- 172.21.15.16:0/1407571095 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9ee8011070 con 0x7f9f000d2510 2024-09-19T07:55:59.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.599+0000 7f9ef77fe640 1 --2- 172.21.15.16:0/1407571095 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ee006c2b0 0x7f9ee006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:59.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.599+0000 7f9ef77fe640 1 -- 172.21.15.16:0/1407571095 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f9ee808b260 con 0x7f9f000d2510 2024-09-19T07:55:59.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.599+0000 7f9efeffd640 1 --2- 172.21.15.16:0/1407571095 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ee006c2b0 0x7f9ee006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:59.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.600+0000 7f9efeffd640 1 --2- 172.21.15.16:0/1407571095 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ee006c2b0 0x7f9ee006e770 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0003eb0 tx=0x7f9ef0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:59.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.604+0000 7f9ef77fe640 1 -- 172.21.15.16:0/1407571095 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9f000b5610 con 0x7f9f000d2510 2024-09-19T07:55:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.790+0000 7f9f052d2640 1 -- 172.21.15.16:0/1407571095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9f000d2930 con 0x7f9f000d2510 2024-09-19T07:55:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.790+0000 7f9ef77fe640 1 -- 172.21.15.16:0/1407571095 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v152) ==== 80+0+0 (secure 0 0 0) 0x7f9ee8059840 con 0x7f9f000d2510 2024-09-19T07:55:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:55:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.792+0000 7f9f052d2640 1 -- 172.21.15.16:0/1407571095 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ee006c2b0 msgr2=0x7f9ee006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.792+0000 7f9f052d2640 1 --2- 172.21.15.16:0/1407571095 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ee006c2b0 0x7f9ee006e770 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0003eb0 tx=0x7f9ef0009040 comp rx=0 tx=0).stop 2024-09-19T07:55:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.792+0000 7f9f052d2640 1 -- 172.21.15.16:0/1407571095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f000d2510 msgr2=0x7f9f000c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.792+0000 7f9f052d2640 1 --2- 172.21.15.16:0/1407571095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f000d2510 0x7f9f000c8040 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800c410 tx=0x7f9ee800c8d0 comp rx=0 tx=0).stop 2024-09-19T07:55:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.792+0000 7f9f052d2640 1 -- 172.21.15.16:0/1407571095 shutdown_connections 2024-09-19T07:55:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.792+0000 7f9f052d2640 1 --2- 172.21.15.16:0/1407571095 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9ee006c2b0 0x7f9ee006e770 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.792+0000 7f9f052d2640 1 --2- 172.21.15.16:0/1407571095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f0015e600 0x7f9f000c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.792+0000 7f9f052d2640 1 --2- 172.21.15.16:0/1407571095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f000d2510 0x7f9f000c8040 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.792+0000 7f9f052d2640 1 --2- 172.21.15.16:0/1407571095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f000d0c40 0x7f9f001f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.792+0000 7f9f052d2640 1 -- 172.21.15.16:0/1407571095 >> 172.21.15.16:0/1407571095 conn(0x7f9f0009d850 msgr2=0x7f9f000b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.792+0000 7f9f052d2640 1 -- 172.21.15.16:0/1407571095 shutdown_connections 2024-09-19T07:55:59.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:55:59.792+0000 7f9f052d2640 1 -- 172.21.15.16:0/1407571095 wait complete. 2024-09-19T07:55:59.803 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-19T07:55:59.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:59.803 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T07:55:59.827 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[79481]: fuse finished with error 0 and tester_r 0 2024-09-19T07:55:59.829 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:55:59.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:59.830 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:55:59.991 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T07:56:00.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.133+0000 7f943649b640 1 Processor -- start 2024-09-19T07:56:00.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.133+0000 7f943649b640 1 -- start start 2024-09-19T07:56:00.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.134+0000 7f943649b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f943006cd80 0x7f94300cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:00.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.134+0000 7f943649b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f943006d280 con 0x7f943006e090 2024-09-19T07:56:00.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.134+0000 7f943649b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f943006d560 con 0x7f9430162290 2024-09-19T07:56:00.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.134+0000 7f943649b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f943006d840 con 0x7f943006cd80 2024-09-19T07:56:00.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.134+0000 7f9434a38640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f943006e090 0x7f94300d0fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49840/0 (socket says 172.21.15.16:49840) 2024-09-19T07:56:00.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.134+0000 7f9434a38640 1 -- 172.21.15.16:0/219815512 learned_addr learned my addr 172.21.15.16:0/219815512 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:56:00.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.134+0000 7f942f7fe640 1 --2- 172.21.15.16:0/219815512 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f943006cd80 0x7f94300cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:00.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.135+0000 7f942effd640 1 -- 172.21.15.16:0/219815512 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2558632375 0 0) 0x7f943006d280 con 0x7f943006e090 2024-09-19T07:56:00.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.135+0000 7f942effd640 1 -- 172.21.15.16:0/219815512 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9418003520 con 0x7f943006e090 2024-09-19T07:56:00.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.135+0000 7f942effd640 1 -- 172.21.15.16:0/219815512 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1629916528 0 0) 0x7f943006d560 con 0x7f9430162290 2024-09-19T07:56:00.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.135+0000 7f942effd640 1 -- 172.21.15.16:0/219815512 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f943006d280 con 0x7f9430162290 2024-09-19T07:56:00.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.135+0000 7f942f7fe640 1 -- 172.21.15.16:0/219815512 >> v1:172.21.15.146:6789/0 conn(0x7f9430162290 legacy=0x7f9430162670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:00.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.135+0000 7f942f7fe640 1 -- 172.21.15.16:0/219815512 >> v1:172.21.15.16:6789/0 conn(0x7f943006e090 legacy=0x7f94300d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:00.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.135+0000 7f942f7fe640 1 -- 172.21.15.16:0/219815512 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f943006db20 con 0x7f943006cd80 2024-09-19T07:56:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.135+0000 7f942f7fe640 1 --2- 172.21.15.16:0/219815512 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f943006cd80 0x7f94300cf1e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9424009800 tx=0x7f942402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16bc912690c9019a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.136+0000 7f942effd640 1 -- 172.21.15.16:0/219815512 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9424037070 con 0x7f943006cd80 2024-09-19T07:56:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.136+0000 7f942effd640 1 -- 172.21.15.16:0/219815512 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9424032440 con 0x7f943006cd80 2024-09-19T07:56:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.136+0000 7f942effd640 1 -- 172.21.15.16:0/219815512 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94240327f0 con 0x7f943006cd80 2024-09-19T07:56:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.136+0000 7f943649b640 1 -- 172.21.15.16:0/219815512 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f943006cd80 msgr2=0x7f94300cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.136+0000 7f943649b640 1 --2- 172.21.15.16:0/219815512 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f943006cd80 0x7f94300cf1e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9424009800 tx=0x7f942402a1a0 comp rx=0 tx=0).stop 2024-09-19T07:56:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.136+0000 7f943649b640 1 -- 172.21.15.16:0/219815512 shutdown_connections 2024-09-19T07:56:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.136+0000 7f943649b640 1 --2- 172.21.15.16:0/219815512 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f943006cd80 0x7f94300cf1e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.136+0000 7f943649b640 1 -- 172.21.15.16:0/219815512 >> 172.21.15.16:0/219815512 conn(0x7f943008bfe0 msgr2=0x7f94300d4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.136+0000 7f943649b640 1 -- 172.21.15.16:0/219815512 shutdown_connections 2024-09-19T07:56:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.136+0000 7f943649b640 1 -- 172.21.15.16:0/219815512 wait complete. 2024-09-19T07:56:00.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.137+0000 7f943649b640 1 Processor -- start 2024-09-19T07:56:00.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.138+0000 7f943649b640 1 -- start start 2024-09-19T07:56:00.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.138+0000 7f943649b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f943006cd80 0x7f9430152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:00.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.138+0000 7f943649b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f943006e090 0x7f94301605d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:00.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.138+0000 7f943649b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9430162290 0x7f9430160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:00.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.138+0000 7f943649b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f943006e6c0 con 0x7f943006cd80 2024-09-19T07:56:00.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.138+0000 7f943649b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f943006e540 con 0x7f943006e090 2024-09-19T07:56:00.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.138+0000 7f943649b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f94300d16d0 con 0x7f9430162290 2024-09-19T07:56:00.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.138+0000 7f942ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f943006cd80 0x7f9430152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:00.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.138+0000 7f942ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f943006cd80 0x7f9430152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46940/0 (socket says 172.21.15.16:46940) 2024-09-19T07:56:00.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.138+0000 7f942ffff640 1 -- 172.21.15.16:0/2554972292 learned_addr learned my addr 172.21.15.16:0/2554972292 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:56:00.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.139+0000 7f9434a38640 1 --2- 172.21.15.16:0/2554972292 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9430162290 0x7f9430160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:00.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.139+0000 7f942f7fe640 1 --2- 172.21.15.16:0/2554972292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f943006e090 0x7f94301605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:00.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.139+0000 7f942ffff640 1 -- 172.21.15.16:0/2554972292 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9430162290 msgr2=0x7f9430160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:00.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.139+0000 7f942ffff640 1 --2- 172.21.15.16:0/2554972292 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9430162290 0x7f9430160b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:00.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.139+0000 7f942ffff640 1 -- 172.21.15.16:0/2554972292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f943006e090 msgr2=0x7f94301605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:00.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.139+0000 7f942ffff640 1 --2- 172.21.15.16:0/2554972292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f943006e090 0x7f94301605d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:00.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.139+0000 7f942ffff640 1 -- 172.21.15.16:0/2554972292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94300c8bc0 con 0x7f943006cd80 2024-09-19T07:56:00.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.139+0000 7f942ffff640 1 --2- 172.21.15.16:0/2554972292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f943006cd80 0x7f9430152880 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f942000c430 tx=0x7f942000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:00.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.139+0000 7f942cff9640 1 -- 172.21.15.16:0/2554972292 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420017070 con 0x7f943006cd80 2024-09-19T07:56:00.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.139+0000 7f942cff9640 1 -- 172.21.15.16:0/2554972292 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94200052f0 con 0x7f943006cd80 2024-09-19T07:56:00.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.139+0000 7f943649b640 1 -- 172.21.15.16:0/2554972292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94300c8e80 con 0x7f943006cd80 2024-09-19T07:56:00.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.139+0000 7f942cff9640 1 -- 172.21.15.16:0/2554972292 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420003830 con 0x7f943006cd80 2024-09-19T07:56:00.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.139+0000 7f943649b640 1 -- 172.21.15.16:0/2554972292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94300c9440 con 0x7f943006cd80 2024-09-19T07:56:00.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.140+0000 7f943649b640 1 -- 172.21.15.16:0/2554972292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f943008c690 con 0x7f943006cd80 2024-09-19T07:56:00.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.141+0000 7f942cff9640 1 -- 172.21.15.16:0/2554972292 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f942001a070 con 0x7f943006cd80 2024-09-19T07:56:00.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.141+0000 7f942cff9640 1 --2- 172.21.15.16:0/2554972292 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f941c06c2b0 0x7f941c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:00.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.141+0000 7f942cff9640 1 -- 172.21.15.16:0/2554972292 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f942008aac0 con 0x7f943006cd80 2024-09-19T07:56:00.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.141+0000 7f942f7fe640 1 --2- 172.21.15.16:0/2554972292 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f941c06c2b0 0x7f941c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:00.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.142+0000 7f942f7fe640 1 --2- 172.21.15.16:0/2554972292 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f941c06c2b0 0x7f941c06e770 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9424034000 tx=0x7f942402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:00.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.145+0000 7f942cff9640 1 -- 172.21.15.16:0/2554972292 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9420059020 con 0x7f943006cd80 2024-09-19T07:56:00.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.358+0000 7f943649b640 1 -- 172.21.15.16:0/2554972292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f94300ce250 con 0x7f943006cd80 2024-09-19T07:56:00.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.358+0000 7f942cff9640 1 -- 172.21.15.16:0/2554972292 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v92) ==== 53+0+73 (secure 0 0 0) 0x7f9420058920 con 0x7f943006cd80 2024-09-19T07:56:00.359 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:56:00.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.360+0000 7f943649b640 1 -- 172.21.15.16:0/2554972292 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f941c06c2b0 msgr2=0x7f941c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:00.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.360+0000 7f943649b640 1 --2- 172.21.15.16:0/2554972292 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f941c06c2b0 0x7f941c06e770 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9424034000 tx=0x7f942402e000 comp rx=0 tx=0).stop 2024-09-19T07:56:00.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.360+0000 7f943649b640 1 -- 172.21.15.16:0/2554972292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f943006cd80 msgr2=0x7f9430152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:00.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.360+0000 7f943649b640 1 --2- 172.21.15.16:0/2554972292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f943006cd80 0x7f9430152880 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f942000c430 tx=0x7f942000c8f0 comp rx=0 tx=0).stop 2024-09-19T07:56:00.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.360+0000 7f943649b640 1 -- 172.21.15.16:0/2554972292 shutdown_connections 2024-09-19T07:56:00.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.360+0000 7f943649b640 1 --2- 172.21.15.16:0/2554972292 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f941c06c2b0 0x7f941c06e770 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:00.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.360+0000 7f943649b640 1 --2- 172.21.15.16:0/2554972292 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9430162290 0x7f9430160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:00.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.360+0000 7f943649b640 1 --2- 172.21.15.16:0/2554972292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f943006e090 0x7f94301605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:00.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.360+0000 7f943649b640 1 --2- 172.21.15.16:0/2554972292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f943006cd80 0x7f9430152880 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:00.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.360+0000 7f943649b640 1 -- 172.21.15.16:0/2554972292 >> 172.21.15.16:0/2554972292 conn(0x7f943008bfe0 msgr2=0x7f94300d2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:00.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.361+0000 7f943649b640 1 -- 172.21.15.16:0/2554972292 shutdown_connections 2024-09-19T07:56:00.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00.361+0000 7f943649b640 1 -- 172.21.15.16:0/2554972292 wait complete. 2024-09-19T07:56:00.372 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:56:00.372 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:56:00.372 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T07:56:00.372 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T07:56:00.372 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:56:00.372 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:56:00.372 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:56:00.372 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:56:00.372 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:56:00.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:56:00.373 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T07:56:00.389 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:56:00.389 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:56:00.389 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:56:00.389 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:56:00.390 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T07:56:00.391 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:56:00.391 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:56:00.391 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:56:00.391 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T07:56:00.391 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:56:00.391 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:56:00.391 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T07:56:00.391 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:56:00.391 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:56:00.391 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T07:56:00.391 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T07:56:00.392 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42029sec preferred_lft 42029sec 2024-09-19T07:56:00.392 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T07:56:00.392 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:56:00.392 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:56:00.392 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:56:00.392 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T07:56:00.392 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T07:56:00.392 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:56:00.392 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T07:56:00.392 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:56:00.392 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::32:7bff:fe92:8895/64 scope link 2024-09-19T07:56:00.392 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:56:00.392 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:56:00.393 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether e6:fd:21:6a:43:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:56:00.393 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e4fd:21ff:fe6a:4369/64 scope link 2024-09-19T07:56:00.393 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:56:00.393 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T07:56:00.393 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 82:f4:81:7d:b7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:56:00.393 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::80f4:81ff:fe7d:b7f0/64 scope link 2024-09-19T07:56:00.393 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:56:00.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:56:00.393 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:56:00.453 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T07:56:00.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:56:00.455 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T07:56:00.511 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T07:56:00.512 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:56:00.512 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T07:56:00.512 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T07:56:00.512 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:56:00.512 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:56:00.512 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T07:56:00.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T07:56:00.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T07:56:00.513 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T07:56:00.513 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T07:56:00.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:56:00.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:56:00.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:56:00.641 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:56:00.657 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T07:56:00.657 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:56:00.657 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:56:00.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:56:00.658 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-19T07:56:00.723 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T07:56:00.723 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:56:00.723 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T07:56:00.779 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:56:00.780 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:56:00.780 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:56:00.836 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:56:00.903 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:56:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:56:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:56:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:56:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:56:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:56:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:56:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:56:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:56:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:56:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:56:00.960 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:56:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:56:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:56:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:56:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:56:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:56:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:56:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:56:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:56:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:56:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:56:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:56:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:56:00.961 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:56:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:56:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:56:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:56:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:56:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:56:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:56:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:56:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:56:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:56:00.962 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:56:00.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:56:00.963 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:56:01.016 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:56:01.016 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T07:56:01.059 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-19T07:56:01.084 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:56:01.142 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-19T07:56:01.142 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-19T07:56:01.142 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-19T07:56:01.142 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-19T07:56:01.142 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-19T07:56:01.142 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-19T07:56:01.142 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-19T07:56:01.143 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-19T07:56:01.143 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-19T07:56:01.143 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-19T07:56:01.143 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-19T07:56:01.143 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-19T07:56:01.143 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:56:01.143 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-19T07:56:01.143 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-19T07:56:01.143 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-19T07:56:01.143 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-19T07:56:01.143 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-19T07:56:01.143 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-19T07:56:01.143 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:56:01.143 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:56:01.144 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:56:01.144 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:56:01.144 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-19T07:56:01.144 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-19T07:56:01.144 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T07:56:01.144 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T07:56:01.144 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T07:56:01.144 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T07:56:01.144 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-19T07:56:01.144 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:56:01.144 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:56:01.144 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:56:01.144 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:56:01.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:56:01.145 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-19T07:56:01.146 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-19T07:56:01.143+0000 7f98ca630480 -1 init, newargv = 0x55a9ad04e9b0 newargc=15 2024-09-19T07:56:01.146 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[80061]: starting ceph client 2024-09-19T07:56:01.148 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[80061]: starting fuse 2024-09-19T07:56:01.162 INFO:teuthology.orchestra.run.smithi016.stdout:41 2024-09-19T07:56:01.163 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-19T07:56:01.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:56:01.163 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:56:01.164 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:56:01.164 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:56:01.164 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:56:01.164 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:56:01.164 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:56:01.164 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:56:01.165 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:56:01.165 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:56:01.165 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:56:01.165 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:56:01.165 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:56:01.165 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:56:01.165 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:56:01.165 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:56:01.166 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:56:01.166 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:56:01.166 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:56:01.166 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:56:01.166 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:56:01.166 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:56:01.166 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:56:01.167 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:56:01.167 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:56:01.167 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:56:01.167 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:56:01.284 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.80061.asok 2024-09-19T07:56:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:56:01.294 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.80061.asok 2024-09-19T07:56:01.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:56:01.295 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.80061.asok status 2024-09-19T07:56:01.430 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:56:01.430 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-19T07:56:01.431 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:56:01.431 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:56:01.431 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-19T07:56:01.431 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-19T07:56:01.431 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T07:56:01.431 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "80061", 2024-09-19T07:56:01.431 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-19T07:56:01.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:56:01.431 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-19T07:56:01.431 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-19T07:56:01.431 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 8985, 2024-09-19T07:56:01.431 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:56:01.431 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:56:01.432 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:56:01.432 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 8985 2024-09-19T07:56:01.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:56:01.432 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:56:01.432 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:56:01.432 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:56:01.432 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2801864998 2024-09-19T07:56:01.432 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:56:01.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:56:01.432 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:56:01.432 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:56:01.432 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:56:01.432 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2801864998 2024-09-19T07:56:01.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:56:01.433 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.8985 192.168.0.1:0/2801864998", 2024-09-19T07:56:01.433 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/2801864998", 2024-09-19T07:56:01.433 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-19T07:56:01.433 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 92, 2024-09-19T07:56:01.433 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 152, 2024-09-19T07:56:01.433 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:56:01.433 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-19T07:56:01.433 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-19T07:56:01.433 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:56:01.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:56:01.439 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-19T07:56:01.439 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T07:56:01.455 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-19T07:56:01.455 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T07:56:01.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:56:01.456 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:56:01.522 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:56:01.522 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-19T07:56:01.522 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-19T07:56:01.522 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-19T07:56:01.523 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-19T07:56:01.523 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:56:01.523 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-19T07:56:01.523 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T07:56:01.523 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-19T07:56:01.523 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T07:56:01.523 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:56:01.523 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-19T07:56:01.523 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-19T07:56:01.524 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-19T07:56:01.524 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-19T07:56:01.524 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:56:01.524 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-19T07:56:01.524 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:56:01.524 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:56:01.524 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:56:01.524 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-19T07:56:01.524 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-19T07:56:01.524 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-19T07:56:01.524 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:56:01.524 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-19T07:56:01.524 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-19T07:56:01.524 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-19T07:56:01.641 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.80061.asok 2024-09-19T07:56:01.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:56:01.653 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.80061.asok 2024-09-19T07:56:01.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:56:01.653 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.80061.asok mds_sessions 2024-09-19T07:56:01.792 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:56:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 8985, 2024-09-19T07:56:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-19T07:56:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-19T07:56:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-19T07:56:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 8985 2024-09-19T07:56:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:56:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-19T07:56:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:56:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-19T07:56:01.792 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2801864998 2024-09-19T07:56:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:56:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:56:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.8985 v1:192.168.0.1:0/2801864998", 2024-09-19T07:56:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "v1:192.168.0.1:0/2801864998", 2024-09-19T07:56:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions": [ 2024-09-19T07:56:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:56:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": 0, 2024-09-19T07:56:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "addrs": { 2024-09-19T07:56:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-19T07:56:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:56:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-19T07:56:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6833", 2024-09-19T07:56:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3705993654 2024-09-19T07:56:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:56:01.793 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T07:56:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T07:56:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6835", 2024-09-19T07:56:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3705993654 2024-09-19T07:56:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:56:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T07:56:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T07:56:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "seq": 0, 2024-09-19T07:56:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_gen": 0, 2024-09-19T07:56:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_ttl": "2024-09-19T07:57:01.147422+0000", 2024-09-19T07:56:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "last_cap_renew_request": "2024-09-19T07:56:01.147422+0000", 2024-09-19T07:56:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_renew_seq": 1, 2024-09-19T07:56:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "num_caps": 1, 2024-09-19T07:56:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "open" 2024-09-19T07:56:01.794 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T07:56:01.795 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T07:56:01.795 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 92 2024-09-19T07:56:01.795 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:56:01.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:56:01.801 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch file1) 2024-09-19T07:56:01.820 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:56:01.820 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:56:01.841 DEBUG:teuthology.orchestra.run.smithi146:> stdin-killer -- python3 -c ' 2024-09-19T07:56:01.841 DEBUG:teuthology.orchestra.run.smithi146:> import fcntl 2024-09-19T07:56:01.841 DEBUG:teuthology.orchestra.run.smithi146:> import os 2024-09-19T07:56:01.841 DEBUG:teuthology.orchestra.run.smithi146:> import sys 2024-09-19T07:56:01.841 DEBUG:teuthology.orchestra.run.smithi146:> import time 2024-09-19T07:56:01.841 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:56:01.841 DEBUG:teuthology.orchestra.run.smithi146:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-19T07:56:01.841 DEBUG:teuthology.orchestra.run.smithi146:> 2024-09-19T07:56:01.841 DEBUG:teuthology.orchestra.run.smithi146:> fd = os.open("/home/ubuntu/cephtest/mnt.1/file1", os.O_RDWR | os.O_CREAT, 0o644) 2024-09-19T07:56:01.842 DEBUG:teuthology.orchestra.run.smithi146:> try: 2024-09-19T07:56:01.842 DEBUG:teuthology.orchestra.run.smithi146:> while True: 2024-09-19T07:56:01.842 DEBUG:teuthology.orchestra.run.smithi146:> print("write_background: writing", file=sys.stderr) 2024-09-19T07:56:01.842 DEBUG:teuthology.orchestra.run.smithi146:> os.write(fd, b'"'"'content'"'"') 2024-09-19T07:56:01.842 DEBUG:teuthology.orchestra.run.smithi146:> if not False: 2024-09-19T07:56:01.842 DEBUG:teuthology.orchestra.run.smithi146:> break 2024-09-19T07:56:01.842 DEBUG:teuthology.orchestra.run.smithi146:> try: 2024-09-19T07:56:01.842 DEBUG:teuthology.orchestra.run.smithi146:> if os.read(0, 4096) == b"": 2024-09-19T07:56:01.842 DEBUG:teuthology.orchestra.run.smithi146:> break 2024-09-19T07:56:01.842 DEBUG:teuthology.orchestra.run.smithi146:> except BlockingIOError: 2024-09-19T07:56:01.842 DEBUG:teuthology.orchestra.run.smithi146:> pass 2024-09-19T07:56:01.842 DEBUG:teuthology.orchestra.run.smithi146:> time.sleep(2) 2024-09-19T07:56:01.842 DEBUG:teuthology.orchestra.run.smithi146:> except IOError as e: 2024-09-19T07:56:01.842 DEBUG:teuthology.orchestra.run.smithi146:> pass 2024-09-19T07:56:01.842 DEBUG:teuthology.orchestra.run.smithi146:> os.close(fd) 2024-09-19T07:56:01.843 DEBUG:teuthology.orchestra.run.smithi146:> ' 2024-09-19T07:56:01.883 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:56:01.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.986+0000 7f4be06f1640 1 Processor -- start 2024-09-19T07:56:01.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.986+0000 7f4be06f1640 1 -- start start 2024-09-19T07:56:01.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.987+0000 7f4be06f1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4bd80ceee0 0x7f4bd815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:01.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.987+0000 7f4be06f1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd80b0370 con 0x7f4bd80afe90 2024-09-19T07:56:01.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.987+0000 7f4be06f1640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd80b3e00 con 0x7f4bd8159db0 2024-09-19T07:56:01.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.987+0000 7f4be06f1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd80b40e0 con 0x7f4bd80ceee0 2024-09-19T07:56:01.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.987+0000 7f4bdec8e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4bd80afe90 0x7f4bd80b48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49850/0 (socket says 172.21.15.16:49850) 2024-09-19T07:56:01.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.987+0000 7f4bdec8e640 1 -- 172.21.15.16:0/529041331 learned_addr learned my addr 172.21.15.16:0/529041331 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:56:01.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.987+0000 7f4bdd48b640 1 -- 172.21.15.16:0/529041331 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 404879068 0 0) 0x7f4bd80b0370 con 0x7f4bd80afe90 2024-09-19T07:56:01.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.987+0000 7f4bddc8c640 1 --2- 172.21.15.16:0/529041331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4bd80ceee0 0x7f4bd815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:01.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.987+0000 7f4bdd48b640 1 -- 172.21.15.16:0/529041331 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bc4003520 con 0x7f4bd80afe90 2024-09-19T07:56:01.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bdd48b640 1 -- 172.21.15.16:0/529041331 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2014000922 0 0) 0x7f4bc4003520 con 0x7f4bd80afe90 2024-09-19T07:56:01.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bdd48b640 1 -- 172.21.15.16:0/529041331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4bd80ceee0 msgr2=0x7f4bd815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:56:01.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bdd48b640 1 --2- 172.21.15.16:0/529041331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4bd80ceee0 0x7f4bd815d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:01.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bdd48b640 1 -- 172.21.15.16:0/529041331 >> v1:172.21.15.146:6789/0 conn(0x7f4bd8159db0 legacy=0x7f4bd80ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:01.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bdd48b640 1 -- 172.21.15.16:0/529041331 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd80b43c0 con 0x7f4bd80afe90 2024-09-19T07:56:01.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bdd48b640 1 -- 172.21.15.16:0/529041331 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f4bd00019f0 con 0x7f4bd80afe90 2024-09-19T07:56:01.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bdd48b640 1 -- 172.21.15.16:0/529041331 >> v1:172.21.15.16:6789/0 conn(0x7f4bd80afe90 legacy=0x7f4bd80b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:01.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bdd48b640 1 --2- 172.21.15.16:0/529041331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4bd80ceee0 0x7f4bc40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:01.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bdd48b640 1 --2- 172.21.15.16:0/529041331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bc4003f20 0x7f4bc4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:01.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bdd48b640 1 --2- 172.21.15.16:0/529041331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bc4004d40 0x7f4bc40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:01.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bdd48b640 1 -- 172.21.15.16:0/529041331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd80b40e0 con 0x7f4bc4003f20 2024-09-19T07:56:01.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bdd48b640 1 -- 172.21.15.16:0/529041331 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc400ba20 con 0x7f4bc4004d40 2024-09-19T07:56:01.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bdd48b640 1 -- 172.21.15.16:0/529041331 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4bc400bc20 con 0x7f4bd80ceee0 2024-09-19T07:56:01.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bddc8c640 1 --2- 172.21.15.16:0/529041331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bc4003f20 0x7f4bc4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:01.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bde48d640 1 --2- 172.21.15.16:0/529041331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4bd80ceee0 0x7f4bc40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:01.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bdec8e640 1 --2- 172.21.15.16:0/529041331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bc4004d40 0x7f4bc40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:01.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bddc8c640 1 -- 172.21.15.16:0/529041331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4bd80ceee0 msgr2=0x7f4bc40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:56:01.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bddc8c640 1 --2- 172.21.15.16:0/529041331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4bd80ceee0 0x7f4bc40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:01.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bddc8c640 1 -- 172.21.15.16:0/529041331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bc4004d40 msgr2=0x7f4bc40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:56:01.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bddc8c640 1 --2- 172.21.15.16:0/529041331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bc4004d40 0x7f4bc40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:01.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.988+0000 7f4bddc8c640 1 -- 172.21.15.16:0/529041331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc4003520 con 0x7f4bc4003f20 2024-09-19T07:56:01.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.989+0000 7f4bddc8c640 1 --2- 172.21.15.16:0/529041331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bc4003f20 0x7f4bc4004800 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80ce780 tx=0x7f4bc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c613a6144fad9972 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:01.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.989+0000 7f4bdd48b640 1 -- 172.21.15.16:0/529041331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8036070 con 0x7f4bc4003f20 2024-09-19T07:56:01.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.989+0000 7f4bdd48b640 1 -- 172.21.15.16:0/529041331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc8031400 con 0x7f4bc4003f20 2024-09-19T07:56:01.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.989+0000 7f4bdd48b640 1 -- 172.21.15.16:0/529041331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8009690 con 0x7f4bc4003f20 2024-09-19T07:56:01.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.990+0000 7f4be06f1640 1 -- 172.21.15.16:0/529041331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bc4003f20 msgr2=0x7f4bc4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:01.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.990+0000 7f4be06f1640 1 --2- 172.21.15.16:0/529041331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bc4003f20 0x7f4bc4004800 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80ce780 tx=0x7f4bc4005830 comp rx=0 tx=0).stop 2024-09-19T07:56:01.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.990+0000 7f4bdec8e640 1 -- 172.21.15.16:0/529041331 reap_dead start 2024-09-19T07:56:01.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.990+0000 7f4be06f1640 1 -- 172.21.15.16:0/529041331 shutdown_connections 2024-09-19T07:56:01.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.990+0000 7f4be06f1640 1 -- 172.21.15.16:0/529041331 >> 172.21.15.16:0/529041331 conn(0x7f4bd8073580 msgr2=0x7f4bd806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:01.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.990+0000 7f4be06f1640 1 -- 172.21.15.16:0/529041331 shutdown_connections 2024-09-19T07:56:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.990+0000 7f4be06f1640 1 -- 172.21.15.16:0/529041331 wait complete. 2024-09-19T07:56:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.991+0000 7f4be06f1640 1 Processor -- start 2024-09-19T07:56:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.991+0000 7f4be06f1640 1 -- start start 2024-09-19T07:56:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4be06f1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bc4003f20 0x7f4bd806fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4be06f1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4bd8160510 0x7f4bd8157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4be06f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bd8157e50 0x7f4bd80ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4be06f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd815d360 con 0x7f4bd8157e50 2024-09-19T07:56:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4be06f1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd815d1e0 con 0x7f4bc4003f20 2024-09-19T07:56:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4be06f1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd815d4e0 con 0x7f4bd8160510 2024-09-19T07:56:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4bdec8e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bd8157e50 0x7f4bd80ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4bdec8e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bd8157e50 0x7f4bd80ad8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46958/0 (socket says 172.21.15.16:46958) 2024-09-19T07:56:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4bdec8e640 1 -- 172.21.15.16:0/3311151420 learned_addr learned my addr 172.21.15.16:0/3311151420 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:56:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4bde48d640 1 --2- 172.21.15.16:0/3311151420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bc4003f20 0x7f4bd806fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4bdec8e640 1 -- 172.21.15.16:0/3311151420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4bd8160510 msgr2=0x7f4bd8157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4bdec8e640 1 --2- 172.21.15.16:0/3311151420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4bd8160510 0x7f4bd8157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4bdec8e640 1 -- 172.21.15.16:0/3311151420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bc4003f20 msgr2=0x7f4bd806fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4bdec8e640 1 --2- 172.21.15.16:0/3311151420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bc4003f20 0x7f4bd806fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4bdec8e640 1 -- 172.21.15.16:0/3311151420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd80ae890 con 0x7f4bd8157e50 2024-09-19T07:56:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4bdec8e640 1 --2- 172.21.15.16:0/3311151420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bd8157e50 0x7f4bd80ad8c0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80b6010 tx=0x7f4bd0007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4bceffd640 1 -- 172.21.15.16:0/3311151420 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd000abc0 con 0x7f4bd8157e50 2024-09-19T07:56:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4bceffd640 1 -- 172.21.15.16:0/3311151420 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd0018070 con 0x7f4bd8157e50 2024-09-19T07:56:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4be06f1640 1 -- 172.21.15.16:0/3311151420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd80ae5b0 con 0x7f4bd8157e50 2024-09-19T07:56:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.992+0000 7f4bceffd640 1 -- 172.21.15.16:0/3311151420 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd00133f0 con 0x7f4bd8157e50 2024-09-19T07:56:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.993+0000 7f4be06f1640 1 -- 172.21.15.16:0/3311151420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd815fd10 con 0x7f4bd8157e50 2024-09-19T07:56:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.993+0000 7f4be06f1640 1 -- 172.21.15.16:0/3311151420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd815f820 con 0x7f4bd8157e50 2024-09-19T07:56:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.994+0000 7f4bceffd640 1 -- 172.21.15.16:0/3311151420 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4bd0013590 con 0x7f4bd8157e50 2024-09-19T07:56:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.995+0000 7f4bceffd640 1 --2- 172.21.15.16:0/3311151420 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4bbc06c2b0 0x7f4bbc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.995+0000 7f4bceffd640 1 -- 172.21.15.16:0/3311151420 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f4bd008bba0 con 0x7f4bd8157e50 2024-09-19T07:56:01.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.995+0000 7f4bde48d640 1 --2- 172.21.15.16:0/3311151420 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4bbc06c2b0 0x7f4bbc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:01.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.995+0000 7f4bde48d640 1 --2- 172.21.15.16:0/3311151420 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4bbc06c2b0 0x7f4bbc06e770 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4000c00 tx=0x7f4bd4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:01.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:01.999+0000 7f4bceffd640 1 -- 172.21.15.16:0/3311151420 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4bd005a180 con 0x7f4bd8157e50 2024-09-19T07:56:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.211+0000 7f4be06f1640 1 -- 172.21.15.16:0/3311151420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4bd80ce7d0 con 0x7f4bd8157e50 2024-09-19T07:56:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.212+0000 7f4bceffd640 1 -- 172.21.15.16:0/3311151420 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5230 (secure 0 0 0) 0x7f4bd0059a80 con 0x7f4bd8157e50 2024-09-19T07:56:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:56:02.212 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":92,"btime":"2024-09-19T07:55:49:210981+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7064,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3873812166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3873812166},{"type":"v1","addr":"172.21.15.146:6837","nonce":3873812166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":7852,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/951273503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":951273503},{"type":"v1","addr":"172.21.15.146:6835","nonce":951273503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":8757,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2338289617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2338289617},{"type":"v1","addr":"172.21.15.16:6834","nonce":2338289617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:55:46.155417+0000","modified":"2024-09-19T07:55:48.561637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8760":{"gid":8760,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6835/3705993654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":3705993654},{"type":"v1","addr":"172.21.15.16:6835","nonce":3705993654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8760,"qdb_cluster":[8760]},"id":11}]} 2024-09-19T07:56:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 92 2024-09-19T07:56:02.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.214+0000 7f4be06f1640 1 -- 172.21.15.16:0/3311151420 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4bbc06c2b0 msgr2=0x7f4bbc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:02.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.214+0000 7f4be06f1640 1 --2- 172.21.15.16:0/3311151420 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4bbc06c2b0 0x7f4bbc06e770 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4000c00 tx=0x7f4bd4009040 comp rx=0 tx=0).stop 2024-09-19T07:56:02.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.214+0000 7f4be06f1640 1 -- 172.21.15.16:0/3311151420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bd8157e50 msgr2=0x7f4bd80ad8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:02.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.214+0000 7f4be06f1640 1 --2- 172.21.15.16:0/3311151420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bd8157e50 0x7f4bd80ad8c0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80b6010 tx=0x7f4bd0007b20 comp rx=0 tx=0).stop 2024-09-19T07:56:02.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.214+0000 7f4be06f1640 1 -- 172.21.15.16:0/3311151420 shutdown_connections 2024-09-19T07:56:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.214+0000 7f4be06f1640 1 --2- 172.21.15.16:0/3311151420 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f4bbc06c2b0 0x7f4bbc06e770 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.214+0000 7f4be06f1640 1 --2- 172.21.15.16:0/3311151420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bd8157e50 0x7f4bd80ad8c0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.214+0000 7f4be06f1640 1 --2- 172.21.15.16:0/3311151420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4bd8160510 0x7f4bd8157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.214+0000 7f4be06f1640 1 --2- 172.21.15.16:0/3311151420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bc4003f20 0x7f4bd806fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.214+0000 7f4be06f1640 1 -- 172.21.15.16:0/3311151420 >> 172.21.15.16:0/3311151420 conn(0x7f4bd8073580 msgr2=0x7f4bd80b1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.214+0000 7f4be06f1640 1 -- 172.21.15.16:0/3311151420 shutdown_connections 2024-09-19T07:56:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.214+0000 7f4be06f1640 1 -- 172.21.15.16:0/3311151420 wait complete. 2024-09-19T07:56:02.225 INFO:tasks.ceph_test_case:waiting 70s for health warning matching MDS_CLIENT_LATE_RELEASE 2024-09-19T07:56:02.225 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:56:02.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.326+0000 7f586df66640 1 Processor -- start 2024-09-19T07:56:02.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.326+0000 7f586df66640 1 -- start start 2024-09-19T07:56:02.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.327+0000 7f586df66640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f58680d10a0 0x7f58680d4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:02.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.327+0000 7f586df66640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58680d5230 con 0x7f58680d1480 2024-09-19T07:56:02.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.327+0000 7f586df66640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58680d2dc0 con 0x7f5868161f30 2024-09-19T07:56:02.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.327+0000 7f586df66640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f58680d30a0 con 0x7f58680d10a0 2024-09-19T07:56:02.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.327+0000 7f5866ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f58680d1480 0x7f5868161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49856/0 (socket says 172.21.15.16:49856) 2024-09-19T07:56:02.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.327+0000 7f5866ffd640 1 -- 172.21.15.16:0/3131972151 learned_addr learned my addr 172.21.15.16:0/3131972151 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:56:02.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.327+0000 7f58677fe640 1 --2- 172.21.15.16:0/3131972151 >> v2:172.21.15.146:3300/0 conn(0x7f58680d10a0 0x7f58680d4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:02.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.327+0000 7f58667fc640 1 -- 172.21.15.16:0/3131972151 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1788106364 0 0) 0x7f58680d5230 con 0x7f58680d1480 2024-09-19T07:56:02.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.328+0000 7f58667fc640 1 -- 172.21.15.16:0/3131972151 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5850003520 con 0x7f58680d1480 2024-09-19T07:56:02.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.328+0000 7f58677fe640 1 -- 172.21.15.16:0/3131972151 >> v1:172.21.15.146:6789/0 conn(0x7f5868161f30 legacy=0x7f58680cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:02.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.328+0000 7f58667fc640 1 -- 172.21.15.16:0/3131972151 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3817314744 0 0) 0x7f5850003520 con 0x7f58680d1480 2024-09-19T07:56:02.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.328+0000 7f58677fe640 1 -- 172.21.15.16:0/3131972151 >> v1:172.21.15.16:6789/0 conn(0x7f58680d1480 legacy=0x7f5868161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:02.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.328+0000 7f58677fe640 1 -- 172.21.15.16:0/3131972151 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58680d3380 con 0x7f58680d10a0 2024-09-19T07:56:02.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.329+0000 7f58677fe640 1 --2- 172.21.15.16:0/3131972151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58680d10a0 0x7f58680d4bc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f58580097a0 tx=0x7f585802a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9a9f25f2229b9be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:02.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.329+0000 7f58667fc640 1 -- 172.21.15.16:0/3131972151 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858037070 con 0x7f58680d10a0 2024-09-19T07:56:02.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.329+0000 7f58667fc640 1 -- 172.21.15.16:0/3131972151 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f585802ab30 con 0x7f58680d10a0 2024-09-19T07:56:02.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.329+0000 7f58667fc640 1 -- 172.21.15.16:0/3131972151 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c001670 con 0x7f58680d10a0 2024-09-19T07:56:02.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.329+0000 7f586df66640 1 -- 172.21.15.16:0/3131972151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58680d10a0 msgr2=0x7f58680d4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.329+0000 7f586df66640 1 --2- 172.21.15.16:0/3131972151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58680d10a0 0x7f58680d4bc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f58580097a0 tx=0x7f585802a0a0 comp rx=0 tx=0).stop 2024-09-19T07:56:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.329+0000 7f586df66640 1 -- 172.21.15.16:0/3131972151 shutdown_connections 2024-09-19T07:56:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.329+0000 7f586df66640 1 --2- 172.21.15.16:0/3131972151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58680d10a0 0x7f58680d4bc0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.329+0000 7f586df66640 1 -- 172.21.15.16:0/3131972151 >> 172.21.15.16:0/3131972151 conn(0x7f586808bfe0 msgr2=0x7f58680d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.329+0000 7f586df66640 1 -- 172.21.15.16:0/3131972151 shutdown_connections 2024-09-19T07:56:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.329+0000 7f586df66640 1 -- 172.21.15.16:0/3131972151 wait complete. 2024-09-19T07:56:02.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.330+0000 7f586df66640 1 Processor -- start 2024-09-19T07:56:02.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.331+0000 7f586df66640 1 -- start start 2024-09-19T07:56:02.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.331+0000 7f586df66640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58680d10a0 0x7f586814f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:02.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.331+0000 7f586df66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58680d1480 0x7f58680ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:02.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.331+0000 7f586df66640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5868161f30 0x7f58680d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:02.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.331+0000 7f586df66640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58680b5350 con 0x7f58680d1480 2024-09-19T07:56:02.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.331+0000 7f586df66640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f58680d3660 con 0x7f58680d10a0 2024-09-19T07:56:02.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.331+0000 7f586df66640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f58680b54d0 con 0x7f5868161f30 2024-09-19T07:56:02.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.331+0000 7f5866ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58680d1480 0x7f58680ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:02.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.332+0000 7f5866ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58680d1480 0x7f58680ca5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46968/0 (socket says 172.21.15.16:46968) 2024-09-19T07:56:02.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.332+0000 7f5866ffd640 1 -- 172.21.15.16:0/1993179273 learned_addr learned my addr 172.21.15.16:0/1993179273 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:56:02.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.332+0000 7f5867fff640 1 --2- 172.21.15.16:0/1993179273 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5868161f30 0x7f58680d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:02.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.332+0000 7f58677fe640 1 --2- 172.21.15.16:0/1993179273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58680d10a0 0x7f586814f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:02.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.332+0000 7f5866ffd640 1 -- 172.21.15.16:0/1993179273 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5868161f30 msgr2=0x7f58680d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:02.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.332+0000 7f5866ffd640 1 --2- 172.21.15.16:0/1993179273 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5868161f30 0x7f58680d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:02.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.332+0000 7f5866ffd640 1 -- 172.21.15.16:0/1993179273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58680d10a0 msgr2=0x7f586814f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:02.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.332+0000 7f5866ffd640 1 --2- 172.21.15.16:0/1993179273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58680d10a0 0x7f586814f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:02.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.332+0000 7f5866ffd640 1 -- 172.21.15.16:0/1993179273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58680cc970 con 0x7f58680d1480 2024-09-19T07:56:02.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.332+0000 7f5866ffd640 1 --2- 172.21.15.16:0/1993179273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58680d1480 0x7f58680ca5c0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f585c00c780 tx=0x7f585c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:02.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.332+0000 7f584ffff640 1 -- 172.21.15.16:0/1993179273 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c017070 con 0x7f58680d1480 2024-09-19T07:56:02.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.332+0000 7f584ffff640 1 -- 172.21.15.16:0/1993179273 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f585c005ad0 con 0x7f58680d1480 2024-09-19T07:56:02.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.332+0000 7f584ffff640 1 -- 172.21.15.16:0/1993179273 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c007070 con 0x7f58680d1480 2024-09-19T07:56:02.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.332+0000 7f586df66640 1 -- 172.21.15.16:0/1993179273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58680c9ae0 con 0x7f58680d1480 2024-09-19T07:56:02.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.332+0000 7f586df66640 1 -- 172.21.15.16:0/1993179273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58680ca0a0 con 0x7f58680d1480 2024-09-19T07:56:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.333+0000 7f586df66640 1 -- 172.21.15.16:0/1993179273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58680ca2a0 con 0x7f58680d1480 2024-09-19T07:56:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.334+0000 7f584ffff640 1 -- 172.21.15.16:0/1993179273 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f585c01a070 con 0x7f58680d1480 2024-09-19T07:56:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.334+0000 7f584ffff640 1 --2- 172.21.15.16:0/1993179273 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f585406c2b0 0x7f585406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.334+0000 7f584ffff640 1 -- 172.21.15.16:0/1993179273 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f585c08ab30 con 0x7f58680d1480 2024-09-19T07:56:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.335+0000 7f58677fe640 1 --2- 172.21.15.16:0/1993179273 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f585406c2b0 0x7f585406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.335+0000 7f58677fe640 1 --2- 172.21.15.16:0/1993179273 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f585406c2b0 0x7f585406e770 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5858034000 tx=0x7f585802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:02.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.339+0000 7f584ffff640 1 -- 172.21.15.16:0/1993179273 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f58680ca2a0 con 0x7f58680d1480 2024-09-19T07:56:02.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.551+0000 7f586df66640 1 -- 172.21.15.16:0/1993179273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f58680d4bc0 con 0x7f58680d1480 2024-09-19T07:56:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.552+0000 7f584ffff640 1 -- 172.21.15.16:0/1993179273 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f585c058a20 con 0x7f58680d1480 2024-09-19T07:56:02.553 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:56:02.553 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T07:56:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.554+0000 7f586df66640 1 -- 172.21.15.16:0/1993179273 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f585406c2b0 msgr2=0x7f585406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.554+0000 7f586df66640 1 --2- 172.21.15.16:0/1993179273 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f585406c2b0 0x7f585406e770 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5858034000 tx=0x7f585802e000 comp rx=0 tx=0).stop 2024-09-19T07:56:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.554+0000 7f586df66640 1 -- 172.21.15.16:0/1993179273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58680d1480 msgr2=0x7f58680ca5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.554+0000 7f586df66640 1 --2- 172.21.15.16:0/1993179273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58680d1480 0x7f58680ca5c0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f585c00c780 tx=0x7f585c00cc40 comp rx=0 tx=0).stop 2024-09-19T07:56:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.554+0000 7f586df66640 1 -- 172.21.15.16:0/1993179273 shutdown_connections 2024-09-19T07:56:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.554+0000 7f586df66640 1 --2- 172.21.15.16:0/1993179273 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f585406c2b0 0x7f585406e770 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.554+0000 7f586df66640 1 --2- 172.21.15.16:0/1993179273 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5868161f30 0x7f58680d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.554+0000 7f586df66640 1 --2- 172.21.15.16:0/1993179273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58680d1480 0x7f58680ca5c0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.554+0000 7f586df66640 1 --2- 172.21.15.16:0/1993179273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58680d10a0 0x7f586814f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.554+0000 7f586df66640 1 -- 172.21.15.16:0/1993179273 >> 172.21.15.16:0/1993179273 conn(0x7f586808bfe0 msgr2=0x7f58680b8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.554+0000 7f586df66640 1 -- 172.21.15.16:0/1993179273 shutdown_connections 2024-09-19T07:56:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:02.554+0000 7f586df66640 1 -- 172.21.15.16:0/1993179273 wait complete. 2024-09-19T07:56:02.565 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-19T07:56:02.565 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2024-09-19T07:56:07.566 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:56:07.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.681+0000 7f231ec4c640 1 Processor -- start 2024-09-19T07:56:07.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.681+0000 7f231ec4c640 1 -- start start 2024-09-19T07:56:07.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.682+0000 7f231ec4c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2318161c30 0x7f23180d4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:07.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.682+0000 7f231ec4c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f23180d13e0 0x7f23180d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:07.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.682+0000 7f231ec4c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23180d50f0 con 0x7f23180b51c0 2024-09-19T07:56:07.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.682+0000 7f231ec4c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f23180d2070 con 0x7f23180d13e0 2024-09-19T07:56:07.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.682+0000 7f231ec4c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f23180d5390 con 0x7f2318161c30 2024-09-19T07:56:07.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.682+0000 7f231d1e9640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f23180d13e0 0x7f23180d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:07.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.682+0000 7f231c9e8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2318161c30 0x7f23180d4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:07.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.682+0000 7f231d1e9640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f23180d13e0 0x7f23180d2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:59828/0 (socket says 172.21.15.16:59828) 2024-09-19T07:56:07.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.682+0000 7f231d1e9640 1 -- 172.21.15.16:0/126513677 learned_addr learned my addr 172.21.15.16:0/126513677 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:56:07.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.682+0000 7f231c9e8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2318161c30 0x7f23180d4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:59550/0 (socket says 172.21.15.16:59550) 2024-09-19T07:56:07.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.683+0000 7f231d1e9640 1 -- 172.21.15.16:0/126513677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2318161c30 msgr2=0x7f23180d4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:56:07.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.683+0000 7f231d1e9640 1 --2- 172.21.15.16:0/126513677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2318161c30 0x7f23180d4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:07.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.683+0000 7f231d1e9640 1 -- 172.21.15.16:0/126513677 >> v1:172.21.15.146:6789/0 conn(0x7f23180b51c0 legacy=0x7f23180d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:07.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.683+0000 7f231d1e9640 1 -- 172.21.15.16:0/126513677 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23180cfda0 con 0x7f23180d13e0 2024-09-19T07:56:07.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.683+0000 7f23177fe640 1 -- 172.21.15.16:0/126513677 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 35427013 0 0) 0x7f23180d50f0 con 0x7f23180b51c0 2024-09-19T07:56:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.683+0000 7f231d1e9640 1 --2- 172.21.15.16:0/126513677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23180d13e0 0x7f23180d2500 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2310009e50 tx=0x7f231002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88035355849d2e46 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.684+0000 7f23177fe640 1 -- 172.21.15.16:0/126513677 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310036070 con 0x7f23180d13e0 2024-09-19T07:56:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.684+0000 7f23177fe640 1 -- 172.21.15.16:0/126513677 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2310031600 con 0x7f23180d13e0 2024-09-19T07:56:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.684+0000 7f23177fe640 1 -- 172.21.15.16:0/126513677 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23100319d0 con 0x7f23180d13e0 2024-09-19T07:56:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.684+0000 7f231ec4c640 1 -- 172.21.15.16:0/126513677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23180d13e0 msgr2=0x7f23180d2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.684+0000 7f231ec4c640 1 --2- 172.21.15.16:0/126513677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23180d13e0 0x7f23180d2500 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2310009e50 tx=0x7f231002a710 comp rx=0 tx=0).stop 2024-09-19T07:56:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.684+0000 7f231ec4c640 1 -- 172.21.15.16:0/126513677 shutdown_connections 2024-09-19T07:56:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.684+0000 7f231ec4c640 1 --2- 172.21.15.16:0/126513677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23180d13e0 0x7f23180d2500 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.684+0000 7f231ec4c640 1 --2- 172.21.15.16:0/126513677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2318161c30 0x7f23180d4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.684+0000 7f231ec4c640 1 -- 172.21.15.16:0/126513677 >> 172.21.15.16:0/126513677 conn(0x7f231808bfe0 msgr2=0x7f23180d44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.684+0000 7f231ec4c640 1 -- 172.21.15.16:0/126513677 shutdown_connections 2024-09-19T07:56:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.684+0000 7f231ec4c640 1 -- 172.21.15.16:0/126513677 wait complete. 2024-09-19T07:56:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.685+0000 7f231ec4c640 1 Processor -- start 2024-09-19T07:56:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.685+0000 7f231ec4c640 1 -- start start 2024-09-19T07:56:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.686+0000 7f231ec4c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23180b51c0 0x7f231815fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:07.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.686+0000 7f231ec4c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23180d13e0 0x7f23180cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:07.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.686+0000 7f231ec4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2318161c30 0x7f23180c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:07.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.686+0000 7f231ec4c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23180b72f0 con 0x7f2318161c30 2024-09-19T07:56:07.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.686+0000 7f231ec4c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f23180d0360 con 0x7f23180d13e0 2024-09-19T07:56:07.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.686+0000 7f231ec4c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2318162010 con 0x7f23180b51c0 2024-09-19T07:56:07.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.687+0000 7f231d1e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2318161c30 0x7f23180c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:07.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.687+0000 7f231d1e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2318161c30 0x7f23180c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48952/0 (socket says 172.21.15.16:48952) 2024-09-19T07:56:07.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.687+0000 7f231d1e9640 1 -- 172.21.15.16:0/4018404626 learned_addr learned my addr 172.21.15.16:0/4018404626 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:56:07.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.687+0000 7f2317fff640 1 --2- 172.21.15.16:0/4018404626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23180d13e0 0x7f23180cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:07.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.687+0000 7f231c9e8640 1 --2- 172.21.15.16:0/4018404626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23180b51c0 0x7f231815fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.687+0000 7f231d1e9640 1 -- 172.21.15.16:0/4018404626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23180b51c0 msgr2=0x7f231815fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.687+0000 7f231d1e9640 1 --2- 172.21.15.16:0/4018404626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23180b51c0 0x7f231815fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.687+0000 7f231d1e9640 1 -- 172.21.15.16:0/4018404626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23180d13e0 msgr2=0x7f23180cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.687+0000 7f231d1e9640 1 --2- 172.21.15.16:0/4018404626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23180d13e0 0x7f23180cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.687+0000 7f231d1e9640 1 -- 172.21.15.16:0/4018404626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23180cc940 con 0x7f2318161c30 2024-09-19T07:56:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.687+0000 7f231d1e9640 1 --2- 172.21.15.16:0/4018404626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2318161c30 0x7f23180c8730 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f2310009e50 tx=0x7f2310004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.687+0000 7f23157fa640 1 -- 172.21.15.16:0/4018404626 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310041020 con 0x7f2318161c30 2024-09-19T07:56:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.687+0000 7f23157fa640 1 -- 172.21.15.16:0/4018404626 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2310037040 con 0x7f2318161c30 2024-09-19T07:56:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.687+0000 7f231ec4c640 1 -- 172.21.15.16:0/4018404626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23180ccc20 con 0x7f2318161c30 2024-09-19T07:56:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.687+0000 7f23157fa640 1 -- 172.21.15.16:0/4018404626 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310036050 con 0x7f2318161c30 2024-09-19T07:56:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.687+0000 7f231ec4c640 1 -- 172.21.15.16:0/4018404626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23180cd1e0 con 0x7f2318161c30 2024-09-19T07:56:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.688+0000 7f231ec4c640 1 -- 172.21.15.16:0/4018404626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23180d4bb0 con 0x7f2318161c30 2024-09-19T07:56:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.689+0000 7f23157fa640 1 -- 172.21.15.16:0/4018404626 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2310007c50 con 0x7f2318161c30 2024-09-19T07:56:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.689+0000 7f23157fa640 1 --2- 172.21.15.16:0/4018404626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22fc06c2b0 0x7f22fc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.690+0000 7f23157fa640 1 -- 172.21.15.16:0/4018404626 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f23100abb60 con 0x7f2318161c30 2024-09-19T07:56:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.690+0000 7f231c9e8640 1 --2- 172.21.15.16:0/4018404626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22fc06c2b0 0x7f22fc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.690+0000 7f231c9e8640 1 --2- 172.21.15.16:0/4018404626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22fc06c2b0 0x7f22fc06e770 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f230c009aa0 tx=0x7f230c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:07.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.694+0000 7f23157fa640 1 -- 172.21.15.16:0/4018404626 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f231004b060 con 0x7f2318161c30 2024-09-19T07:56:07.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.907+0000 7f231ec4c640 1 -- 172.21.15.16:0/4018404626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f23180cb330 con 0x7f2318161c30 2024-09-19T07:56:07.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.908+0000 7f23157fa640 1 -- 172.21.15.16:0/4018404626 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f2310079b80 con 0x7f2318161c30 2024-09-19T07:56:07.908 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:56:07.908 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T07:56:07.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.910+0000 7f231ec4c640 1 -- 172.21.15.16:0/4018404626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22fc06c2b0 msgr2=0x7f22fc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:07.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.910+0000 7f231ec4c640 1 --2- 172.21.15.16:0/4018404626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22fc06c2b0 0x7f22fc06e770 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f230c009aa0 tx=0x7f230c009190 comp rx=0 tx=0).stop 2024-09-19T07:56:07.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.910+0000 7f231ec4c640 1 -- 172.21.15.16:0/4018404626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2318161c30 msgr2=0x7f23180c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:07.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.910+0000 7f231ec4c640 1 --2- 172.21.15.16:0/4018404626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2318161c30 0x7f23180c8730 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f2310009e50 tx=0x7f2310004a40 comp rx=0 tx=0).stop 2024-09-19T07:56:07.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.910+0000 7f231ec4c640 1 -- 172.21.15.16:0/4018404626 shutdown_connections 2024-09-19T07:56:07.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.910+0000 7f231ec4c640 1 --2- 172.21.15.16:0/4018404626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f22fc06c2b0 0x7f22fc06e770 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:07.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.910+0000 7f231ec4c640 1 --2- 172.21.15.16:0/4018404626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2318161c30 0x7f23180c8730 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:07.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.910+0000 7f231ec4c640 1 --2- 172.21.15.16:0/4018404626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23180d13e0 0x7f23180cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:07.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.910+0000 7f231ec4c640 1 --2- 172.21.15.16:0/4018404626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23180b51c0 0x7f231815fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:07.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.910+0000 7f231ec4c640 1 -- 172.21.15.16:0/4018404626 >> 172.21.15.16:0/4018404626 conn(0x7f231808bfe0 msgr2=0x7f231806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:07.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.910+0000 7f231ec4c640 1 -- 172.21.15.16:0/4018404626 shutdown_connections 2024-09-19T07:56:07.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:07.910+0000 7f231ec4c640 1 -- 172.21.15.16:0/4018404626 wait complete. 2024-09-19T07:56:07.921 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-19T07:56:07.921 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2024-09-19T07:56:09.797 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:56:09.801 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:56:12.923 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:56:13.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.037+0000 7f28830ac640 1 Processor -- start 2024-09-19T07:56:13.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.037+0000 7f28830ac640 1 -- start start 2024-09-19T07:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.038+0000 7f28830ac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f287c0d2cd0 0x7f287c0b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.038+0000 7f28830ac640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f287c0b38a0 0x7f287c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.038+0000 7f28830ac640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f287c15ec20 con 0x7f287c15e840 2024-09-19T07:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.038+0000 7f28830ac640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f287c0b2400 con 0x7f287c0d2cd0 2024-09-19T07:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.038+0000 7f28830ac640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f287c0d3390 con 0x7f287c0b38a0 2024-09-19T07:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.038+0000 7f287bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f287c0d2cd0 0x7f287c0b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.038+0000 7f2880e48640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f287c15e840 0x7f287c0d25c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56476/0 (socket says 172.21.15.16:56476) 2024-09-19T07:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.038+0000 7f2880e48640 1 -- 172.21.15.16:0/2238941287 learned_addr learned my addr 172.21.15.16:0/2238941287 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.038+0000 7f2881649640 1 --2- 172.21.15.16:0/2238941287 >> v2:172.21.15.146:3300/0 conn(0x7f287c0b38a0 0x7f287c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.038+0000 7f287bfff640 1 -- 172.21.15.16:0/2238941287 >> v2:172.21.15.146:3300/0 conn(0x7f287c0b38a0 msgr2=0x7f287c06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.038+0000 7f287bfff640 1 --2- 172.21.15.16:0/2238941287 >> v2:172.21.15.146:3300/0 conn(0x7f287c0b38a0 0x7f287c06ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.038+0000 7f287bfff640 1 -- 172.21.15.16:0/2238941287 >> v1:172.21.15.16:6789/0 conn(0x7f287c15e840 legacy=0x7f287c0d25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.038+0000 7f287bfff640 1 -- 172.21.15.16:0/2238941287 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287c0b17f0 con 0x7f287c0d2cd0 2024-09-19T07:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.039+0000 7f287bfff640 1 --2- 172.21.15.16:0/2238941287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0d2cd0 0x7f287c0b10a0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f287c0b5f30 tx=0x7f286802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1f4d385190fac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.039+0000 7f287b7fe640 1 -- 172.21.15.16:0/2238941287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2868037070 con 0x7f287c0d2cd0 2024-09-19T07:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.039+0000 7f287b7fe640 1 -- 172.21.15.16:0/2238941287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2868032440 con 0x7f287c0d2cd0 2024-09-19T07:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.039+0000 7f287b7fe640 1 -- 172.21.15.16:0/2238941287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2868032810 con 0x7f287c0d2cd0 2024-09-19T07:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.040+0000 7f28830ac640 1 -- 172.21.15.16:0/2238941287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0d2cd0 msgr2=0x7f287c0b10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.040+0000 7f28830ac640 1 --2- 172.21.15.16:0/2238941287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0d2cd0 0x7f287c0b10a0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f287c0b5f30 tx=0x7f286802a1a0 comp rx=0 tx=0).stop 2024-09-19T07:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.040+0000 7f28830ac640 1 -- 172.21.15.16:0/2238941287 shutdown_connections 2024-09-19T07:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.040+0000 7f28830ac640 1 --2- 172.21.15.16:0/2238941287 >> v2:172.21.15.146:3300/0 conn(0x7f287c0b38a0 0x7f287c06ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.040+0000 7f28830ac640 1 --2- 172.21.15.16:0/2238941287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0d2cd0 0x7f287c0b10a0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.040+0000 7f28830ac640 1 -- 172.21.15.16:0/2238941287 >> 172.21.15.16:0/2238941287 conn(0x7f287c08bfe0 msgr2=0x7f287c0d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.040+0000 7f28830ac640 1 -- 172.21.15.16:0/2238941287 shutdown_connections 2024-09-19T07:56:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.040+0000 7f28830ac640 1 -- 172.21.15.16:0/2238941287 wait complete. 2024-09-19T07:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.041+0000 7f28830ac640 1 Processor -- start 2024-09-19T07:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.041+0000 7f28830ac640 1 -- start start 2024-09-19T07:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.042+0000 7f28830ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0b38a0 0x7f287c0b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.042+0000 7f28830ac640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f287c0d2cd0 0x7f287c0b9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.042+0000 7f28830ac640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f287c15e840 0x7f287c1545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.042+0000 7f28830ac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f287c0b2090 con 0x7f287c0b38a0 2024-09-19T07:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.042+0000 7f28830ac640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f287c0b1db0 con 0x7f287c15e840 2024-09-19T07:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.042+0000 7f28830ac640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f287c06d210 con 0x7f287c0d2cd0 2024-09-19T07:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.042+0000 7f2880e48640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0b38a0 0x7f287c0b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.042+0000 7f2880e48640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0b38a0 0x7f287c0b72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37598/0 (socket says 172.21.15.16:37598) 2024-09-19T07:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.042+0000 7f2880e48640 1 -- 172.21.15.16:0/3164673743 learned_addr learned my addr 172.21.15.16:0/3164673743 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.043+0000 7f2881649640 1 --2- 172.21.15.16:0/3164673743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f287c15e840 0x7f287c1545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.043+0000 7f287bfff640 1 --2- 172.21.15.16:0/3164673743 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f287c0d2cd0 0x7f287c0b9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.043+0000 7f2880e48640 1 -- 172.21.15.16:0/3164673743 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f287c0d2cd0 msgr2=0x7f287c0b9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.043+0000 7f2880e48640 1 --2- 172.21.15.16:0/3164673743 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f287c0d2cd0 0x7f287c0b9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.043+0000 7f2880e48640 1 -- 172.21.15.16:0/3164673743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f287c15e840 msgr2=0x7f287c1545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.043+0000 7f2880e48640 1 --2- 172.21.15.16:0/3164673743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f287c15e840 0x7f287c1545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.043+0000 7f2880e48640 1 -- 172.21.15.16:0/3164673743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287c0ce380 con 0x7f287c0b38a0 2024-09-19T07:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.043+0000 7f287bfff640 1 --2- 172.21.15.16:0/3164673743 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f287c0d2cd0 0x7f287c0b9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.043+0000 7f2880e48640 1 --2- 172.21.15.16:0/3164673743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0b38a0 0x7f287c0b72e0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f287000c3f0 tx=0x7f287000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.043+0000 7f28797fa640 1 -- 172.21.15.16:0/3164673743 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2870017070 con 0x7f287c0b38a0 2024-09-19T07:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.043+0000 7f28797fa640 1 -- 172.21.15.16:0/3164673743 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2870005ce0 con 0x7f287c0b38a0 2024-09-19T07:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.043+0000 7f28797fa640 1 -- 172.21.15.16:0/3164673743 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28700072f0 con 0x7f287c0b38a0 2024-09-19T07:56:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.043+0000 7f28830ac640 1 -- 172.21.15.16:0/3164673743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f287c0b7970 con 0x7f287c0b38a0 2024-09-19T07:56:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.043+0000 7f28830ac640 1 -- 172.21.15.16:0/3164673743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f287c0b7f30 con 0x7f287c0b38a0 2024-09-19T07:56:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.044+0000 7f28830ac640 1 -- 172.21.15.16:0/3164673743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f287c06ccd0 con 0x7f287c0b38a0 2024-09-19T07:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.045+0000 7f28797fa640 1 -- 172.21.15.16:0/3164673743 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f287001a070 con 0x7f287c0b38a0 2024-09-19T07:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.045+0000 7f28797fa640 1 --2- 172.21.15.16:0/3164673743 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f286006c2b0 0x7f286006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.045+0000 7f28797fa640 1 -- 172.21.15.16:0/3164673743 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f287008a900 con 0x7f287c0b38a0 2024-09-19T07:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.045+0000 7f287bfff640 1 --2- 172.21.15.16:0/3164673743 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f286006c2b0 0x7f286006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.046+0000 7f287bfff640 1 --2- 172.21.15.16:0/3164673743 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f286006c2b0 0x7f286006e770 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2868034000 tx=0x7f286802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:13.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.050+0000 7f28797fa640 1 -- 172.21.15.16:0/3164673743 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2870058800 con 0x7f287c0b38a0 2024-09-19T07:56:13.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.267+0000 7f28830ac640 1 -- 172.21.15.16:0/3164673743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f287c0b5d90 con 0x7f287c0b38a0 2024-09-19T07:56:13.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.268+0000 7f28797fa640 1 -- 172.21.15.16:0/3164673743 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f287005cb00 con 0x7f287c0b38a0 2024-09-19T07:56:13.268 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:56:13.269 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T07:56:13.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.270+0000 7f28830ac640 1 -- 172.21.15.16:0/3164673743 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f286006c2b0 msgr2=0x7f286006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:13.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.270+0000 7f28830ac640 1 --2- 172.21.15.16:0/3164673743 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f286006c2b0 0x7f286006e770 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2868034000 tx=0x7f286802e000 comp rx=0 tx=0).stop 2024-09-19T07:56:13.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.270+0000 7f28830ac640 1 -- 172.21.15.16:0/3164673743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0b38a0 msgr2=0x7f287c0b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:13.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.270+0000 7f28830ac640 1 --2- 172.21.15.16:0/3164673743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0b38a0 0x7f287c0b72e0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f287000c3f0 tx=0x7f287000c8b0 comp rx=0 tx=0).stop 2024-09-19T07:56:13.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.270+0000 7f28830ac640 1 -- 172.21.15.16:0/3164673743 shutdown_connections 2024-09-19T07:56:13.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.270+0000 7f28830ac640 1 --2- 172.21.15.16:0/3164673743 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f286006c2b0 0x7f286006e770 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:13.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.270+0000 7f28830ac640 1 --2- 172.21.15.16:0/3164673743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f287c15e840 0x7f287c1545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:13.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.270+0000 7f28830ac640 1 --2- 172.21.15.16:0/3164673743 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f287c0d2cd0 0x7f287c0b9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:13.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.270+0000 7f28830ac640 1 --2- 172.21.15.16:0/3164673743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0b38a0 0x7f287c0b72e0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:13.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.270+0000 7f28830ac640 1 -- 172.21.15.16:0/3164673743 >> 172.21.15.16:0/3164673743 conn(0x7f287c08bfe0 msgr2=0x7f287c0b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:13.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.270+0000 7f28830ac640 1 -- 172.21.15.16:0/3164673743 shutdown_connections 2024-09-19T07:56:13.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:13.270+0000 7f28830ac640 1 -- 172.21.15.16:0/3164673743 wait complete. 2024-09-19T07:56:13.281 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-19T07:56:13.282 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2024-09-19T07:56:18.283 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:56:18.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.397+0000 7f2dd3de8640 1 Processor -- start 2024-09-19T07:56:18.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.397+0000 7f2dd3de8640 1 -- start start 2024-09-19T07:56:18.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.398+0000 7f2dd3de8640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2dcc15e700 0x7f2dcc06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:18.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.398+0000 7f2dd3de8640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dcc0d09f0 con 0x7f2dcc0d0510 2024-09-19T07:56:18.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.398+0000 7f2dd3de8640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dcc0d0cd0 con 0x7f2dcc09b160 2024-09-19T07:56:18.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.398+0000 7f2dd3de8640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2dcc0d1600 con 0x7f2dcc15e700 2024-09-19T07:56:18.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.398+0000 7f2dd1383640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2dcc0d0510 0x7f2dcc09aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56480/0 (socket says 172.21.15.16:56480) 2024-09-19T07:56:18.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.398+0000 7f2dd1383640 1 -- 172.21.15.16:0/1873163472 learned_addr learned my addr 172.21.15.16:0/1873163472 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:56:18.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.398+0000 7f2dd1b84640 1 --2- 172.21.15.16:0/1873163472 >> v2:172.21.15.146:3300/0 conn(0x7f2dcc15e700 0x7f2dcc06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:18.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.398+0000 7f2dd0b82640 1 -- 172.21.15.16:0/1873163472 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2896226236 0 0) 0x7f2dcc0d09f0 con 0x7f2dcc0d0510 2024-09-19T07:56:18.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.398+0000 7f2dd0b82640 1 -- 172.21.15.16:0/1873163472 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2db8003520 con 0x7f2dcc0d0510 2024-09-19T07:56:18.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd0b82640 1 -- 172.21.15.16:0/1873163472 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1812917955 0 0) 0x7f2db8003520 con 0x7f2dcc0d0510 2024-09-19T07:56:18.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd0b82640 1 -- 172.21.15.16:0/1873163472 >> v2:172.21.15.146:3300/0 conn(0x7f2dcc15e700 msgr2=0x7f2dcc06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:56:18.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd0b82640 1 --2- 172.21.15.16:0/1873163472 >> v2:172.21.15.146:3300/0 conn(0x7f2dcc15e700 0x7f2dcc06cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:18.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd0b82640 1 -- 172.21.15.16:0/1873163472 >> v1:172.21.15.146:6789/0 conn(0x7f2dcc09b160 legacy=0x7f2dcc0cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:18.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd0b82640 1 -- 172.21.15.16:0/1873163472 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dcc0d0fb0 con 0x7f2dcc0d0510 2024-09-19T07:56:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd0b82640 1 -- 172.21.15.16:0/1873163472 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f2dbc001670 con 0x7f2dcc0d0510 2024-09-19T07:56:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd0b82640 1 -- 172.21.15.16:0/1873163472 >> v1:172.21.15.16:6789/0 conn(0x7f2dcc0d0510 legacy=0x7f2dcc09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd0b82640 1 --2- 172.21.15.16:0/1873163472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db80039b0 0x7f2db8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd0b82640 1 --2- 172.21.15.16:0/1873163472 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2db8004800 0x7f2db8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd0b82640 1 --2- 172.21.15.16:0/1873163472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2db80051c0 0x7f2db800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd0b82640 1 -- 172.21.15.16:0/1873163472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc0d1600 con 0x7f2db80039b0 2024-09-19T07:56:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd0b82640 1 -- 172.21.15.16:0/1873163472 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2db800c060 con 0x7f2db80051c0 2024-09-19T07:56:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd0b82640 1 -- 172.21.15.16:0/1873163472 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2db800c260 con 0x7f2db8004800 2024-09-19T07:56:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd1b84640 1 --2- 172.21.15.16:0/1873163472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db80039b0 0x7f2db8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd1383640 1 --2- 172.21.15.16:0/1873163472 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2db8004800 0x7f2db8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd2385640 1 --2- 172.21.15.16:0/1873163472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2db80051c0 0x7f2db800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd1b84640 1 -- 172.21.15.16:0/1873163472 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2db8004800 msgr2=0x7f2db8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:56:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd1b84640 1 --2- 172.21.15.16:0/1873163472 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2db8004800 0x7f2db8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd1b84640 1 -- 172.21.15.16:0/1873163472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2db80051c0 msgr2=0x7f2db800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:56:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd1b84640 1 --2- 172.21.15.16:0/1873163472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2db80051c0 0x7f2db800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd2385640 1 -- 172.21.15.16:0/1873163472 reap_dead start 2024-09-19T07:56:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.399+0000 7f2dd1b84640 1 -- 172.21.15.16:0/1873163472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db8003520 con 0x7f2db80039b0 2024-09-19T07:56:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.400+0000 7f2dd1b84640 1 --2- 172.21.15.16:0/1873163472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db80039b0 0x7f2db8003dd0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009ac0 tx=0x7f2db8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b6f8d4763b0ae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.400+0000 7f2dd0b82640 1 -- 172.21.15.16:0/1873163472 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8036070 con 0x7f2db80039b0 2024-09-19T07:56:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.400+0000 7f2dd0b82640 1 -- 172.21.15.16:0/1873163472 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dc8031400 con 0x7f2db80039b0 2024-09-19T07:56:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.400+0000 7f2dd0b82640 1 -- 172.21.15.16:0/1873163472 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8009690 con 0x7f2db80039b0 2024-09-19T07:56:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.401+0000 7f2dd3de8640 1 -- 172.21.15.16:0/1873163472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db80039b0 msgr2=0x7f2db8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.401+0000 7f2dd3de8640 1 --2- 172.21.15.16:0/1873163472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db80039b0 0x7f2db8003dd0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009ac0 tx=0x7f2db8005830 comp rx=0 tx=0).stop 2024-09-19T07:56:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.401+0000 7f2dd3de8640 1 -- 172.21.15.16:0/1873163472 shutdown_connections 2024-09-19T07:56:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.401+0000 7f2dd3de8640 1 --2- 172.21.15.16:0/1873163472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db80039b0 0x7f2db8003dd0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.401+0000 7f2dd3de8640 1 -- 172.21.15.16:0/1873163472 >> 172.21.15.16:0/1873163472 conn(0x7f2dcc0d22d0 msgr2=0x7f2dcc0d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.401+0000 7f2dd3de8640 1 -- 172.21.15.16:0/1873163472 shutdown_connections 2024-09-19T07:56:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.401+0000 7f2dd3de8640 1 -- 172.21.15.16:0/1873163472 wait complete. 2024-09-19T07:56:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.402+0000 7f2dd3de8640 1 Processor -- start 2024-09-19T07:56:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.402+0000 7f2dd3de8640 1 -- start start 2024-09-19T07:56:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.403+0000 7f2dd3de8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2db80039b0 0x7f2dcc15a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.403+0000 7f2dd3de8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2dcc1e7b50 0x7f2dcc1e7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.403+0000 7f2dd3de8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dcc1e9970 0x7f2dcc1e9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.403+0000 7f2dd3de8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc0d1a90 con 0x7f2dcc1e9970 2024-09-19T07:56:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.403+0000 7f2dd3de8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2dcc09aa50 con 0x7f2db80039b0 2024-09-19T07:56:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.403+0000 7f2dd3de8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2dcc1e85d0 con 0x7f2dcc1e7b50 2024-09-19T07:56:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.403+0000 7f2dd2385640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dcc1e9970 0x7f2dcc1e9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.403+0000 7f2dd2385640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dcc1e9970 0x7f2dcc1e9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37622/0 (socket says 172.21.15.16:37622) 2024-09-19T07:56:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.403+0000 7f2dd2385640 1 -- 172.21.15.16:0/3259259856 learned_addr learned my addr 172.21.15.16:0/3259259856 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:56:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.403+0000 7f2dd1b84640 1 --2- 172.21.15.16:0/3259259856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2db80039b0 0x7f2dcc15a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.403+0000 7f2dd2385640 1 -- 172.21.15.16:0/3259259856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2dcc1e7b50 msgr2=0x7f2dcc1e7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.403+0000 7f2dd1383640 1 --2- 172.21.15.16:0/3259259856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2dcc1e7b50 0x7f2dcc1e7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.403+0000 7f2dd2385640 1 --2- 172.21.15.16:0/3259259856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2dcc1e7b50 0x7f2dcc1e7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.403+0000 7f2dd2385640 1 -- 172.21.15.16:0/3259259856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2db80039b0 msgr2=0x7f2dcc15a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.403+0000 7f2dd2385640 1 --2- 172.21.15.16:0/3259259856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2db80039b0 0x7f2dcc15a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.403+0000 7f2dd2385640 1 -- 172.21.15.16:0/3259259856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dcc1e88b0 con 0x7f2dcc1e9970 2024-09-19T07:56:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.403+0000 7f2dd2385640 1 --2- 172.21.15.16:0/3259259856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dcc1e9970 0x7f2dcc1e9df0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0ced20 tx=0x7f2dc400e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.403+0000 7f2dc27fc640 1 -- 172.21.15.16:0/3259259856 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4019070 con 0x7f2dcc1e9970 2024-09-19T07:56:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.404+0000 7f2dc27fc640 1 -- 172.21.15.16:0/3259259856 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dc4006d60 con 0x7f2dcc1e9970 2024-09-19T07:56:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.404+0000 7f2dd3de8640 1 -- 172.21.15.16:0/3259259856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dcc1e8b90 con 0x7f2dcc1e9970 2024-09-19T07:56:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.404+0000 7f2dc27fc640 1 -- 172.21.15.16:0/3259259856 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4003b90 con 0x7f2dcc1e9970 2024-09-19T07:56:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.404+0000 7f2dd3de8640 1 -- 172.21.15.16:0/3259259856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dcc0b5480 con 0x7f2dcc1e9970 2024-09-19T07:56:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.405+0000 7f2dd3de8640 1 -- 172.21.15.16:0/3259259856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dcc0b5680 con 0x7f2dcc1e9970 2024-09-19T07:56:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.405+0000 7f2dc27fc640 1 -- 172.21.15.16:0/3259259856 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2dc401c070 con 0x7f2dcc1e9970 2024-09-19T07:56:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.406+0000 7f2dc27fc640 1 --2- 172.21.15.16:0/3259259856 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2db006c2e0 0x7f2db006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.406+0000 7f2dc27fc640 1 -- 172.21.15.16:0/3259259856 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f2dc408a9d0 con 0x7f2dcc1e9970 2024-09-19T07:56:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.406+0000 7f2dd1b84640 1 --2- 172.21.15.16:0/3259259856 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2db006c2e0 0x7f2db006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.407+0000 7f2dd1b84640 1 --2- 172.21.15.16:0/3259259856 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2db006c2e0 0x7f2db006e7a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8033000 tx=0x7f2dc8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.410+0000 7f2dc27fc640 1 -- 172.21.15.16:0/3259259856 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2dcc0b5680 con 0x7f2dcc1e9970 2024-09-19T07:56:18.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.623+0000 7f2dd3de8640 1 -- 172.21.15.16:0/3259259856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2dcc09ade0 con 0x7f2dcc1e9970 2024-09-19T07:56:18.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.623+0000 7f2dc27fc640 1 -- 172.21.15.16:0/3259259856 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f2dc4058960 con 0x7f2dcc1e9970 2024-09-19T07:56:18.623 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:56:18.624 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T07:56:18.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.625+0000 7f2dd3de8640 1 -- 172.21.15.16:0/3259259856 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2db006c2e0 msgr2=0x7f2db006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.625+0000 7f2dd3de8640 1 --2- 172.21.15.16:0/3259259856 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2db006c2e0 0x7f2db006e7a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8033000 tx=0x7f2dc8008030 comp rx=0 tx=0).stop 2024-09-19T07:56:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.625+0000 7f2dd3de8640 1 -- 172.21.15.16:0/3259259856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dcc1e9970 msgr2=0x7f2dcc1e9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.625+0000 7f2dd3de8640 1 --2- 172.21.15.16:0/3259259856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dcc1e9970 0x7f2dcc1e9df0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0ced20 tx=0x7f2dc400e960 comp rx=0 tx=0).stop 2024-09-19T07:56:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.625+0000 7f2dd3de8640 1 -- 172.21.15.16:0/3259259856 shutdown_connections 2024-09-19T07:56:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.625+0000 7f2dd3de8640 1 --2- 172.21.15.16:0/3259259856 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2db006c2e0 0x7f2db006e7a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.625+0000 7f2dd3de8640 1 --2- 172.21.15.16:0/3259259856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dcc1e9970 0x7f2dcc1e9df0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.625+0000 7f2dd3de8640 1 --2- 172.21.15.16:0/3259259856 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2dcc1e7b50 0x7f2dcc1e7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.625+0000 7f2dd3de8640 1 --2- 172.21.15.16:0/3259259856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2db80039b0 0x7f2dcc15a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.625+0000 7f2dd3de8640 1 -- 172.21.15.16:0/3259259856 >> 172.21.15.16:0/3259259856 conn(0x7f2dcc0d22d0 msgr2=0x7f2dcc0ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.625+0000 7f2dd3de8640 1 -- 172.21.15.16:0/3259259856 shutdown_connections 2024-09-19T07:56:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:18.625+0000 7f2dd3de8640 1 -- 172.21.15.16:0/3259259856 wait complete. 2024-09-19T07:56:18.636 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-19T07:56:18.636 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2024-09-19T07:56:23.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:56:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.752+0000 7f9c35823640 1 Processor -- start 2024-09-19T07:56:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.752+0000 7f9c35823640 1 -- start start 2024-09-19T07:56:23.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.753+0000 7f9c35823640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c301567b0 0x7f9c300d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:23.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.753+0000 7f9c35823640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9c300d0a80 0x7f9c300aee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:23.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.753+0000 7f9c35823640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c300af360 0x7f9c300b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.753+0000 7f9c35823640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9c300d0ef0 con 0x7f9c301567b0 2024-09-19T07:56:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.753+0000 7f9c35823640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f9c300ad400 con 0x7f9c300d0a80 2024-09-19T07:56:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.753+0000 7f9c35823640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9c3015c880 con 0x7f9c300af360 2024-09-19T07:56:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.753+0000 7f9c2effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c301567b0 0x7f9c300d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.753+0000 7f9c2effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c301567b0 0x7f9c300d0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35380/0 (socket says 172.21.15.16:35380) 2024-09-19T07:56:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.753+0000 7f9c2effd640 1 -- 172.21.15.16:0/3562459954 learned_addr learned my addr 172.21.15.16:0/3562459954 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:56:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.753+0000 7f9c267fc640 1 --2- 172.21.15.16:0/3562459954 >> v2:172.21.15.146:3300/0 conn(0x7f9c300d0a80 0x7f9c300aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.753+0000 7f9c2f7fe640 1 --2- 172.21.15.16:0/3562459954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c300af360 0x7f9c300b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.753+0000 7f9c2effd640 1 -- 172.21.15.16:0/3562459954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c300af360 msgr2=0x7f9c300b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:56:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.753+0000 7f9c2effd640 1 --2- 172.21.15.16:0/3562459954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c300af360 0x7f9c300b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.753+0000 7f9c2effd640 1 -- 172.21.15.16:0/3562459954 >> v2:172.21.15.146:3300/0 conn(0x7f9c300d0a80 msgr2=0x7f9c300aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:56:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.753+0000 7f9c2effd640 1 --2- 172.21.15.16:0/3562459954 >> v2:172.21.15.146:3300/0 conn(0x7f9c300d0a80 0x7f9c300aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.753+0000 7f9c2effd640 1 -- 172.21.15.16:0/3562459954 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c3015cb20 con 0x7f9c301567b0 2024-09-19T07:56:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.753+0000 7f9c267fc640 1 --2- 172.21.15.16:0/3562459954 >> v2:172.21.15.146:3300/0 conn(0x7f9c300d0a80 0x7f9c300aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:56:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.753+0000 7f9c2f7fe640 1 --2- 172.21.15.16:0/3562459954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c300af360 0x7f9c300b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:56:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.754+0000 7f9c2effd640 1 --2- 172.21.15.16:0/3562459954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c301567b0 0x7f9c300d0540 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f9c3006f230 tx=0x7f9c2002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea47f5d10b9ac3dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.754+0000 7f9c2e7fc640 1 -- 172.21.15.16:0/3562459954 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20037070 con 0x7f9c301567b0 2024-09-19T07:56:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.754+0000 7f9c2e7fc640 1 -- 172.21.15.16:0/3562459954 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c20032440 con 0x7f9c301567b0 2024-09-19T07:56:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.754+0000 7f9c2e7fc640 1 -- 172.21.15.16:0/3562459954 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20032810 con 0x7f9c301567b0 2024-09-19T07:56:23.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.755+0000 7f9c35823640 1 -- 172.21.15.16:0/3562459954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c301567b0 msgr2=0x7f9c300d0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.755+0000 7f9c35823640 1 --2- 172.21.15.16:0/3562459954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c301567b0 0x7f9c300d0540 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f9c3006f230 tx=0x7f9c2002a200 comp rx=0 tx=0).stop 2024-09-19T07:56:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.755+0000 7f9c35823640 1 -- 172.21.15.16:0/3562459954 shutdown_connections 2024-09-19T07:56:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.755+0000 7f9c35823640 1 --2- 172.21.15.16:0/3562459954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c300af360 0x7f9c300b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.755+0000 7f9c35823640 1 --2- 172.21.15.16:0/3562459954 >> v2:172.21.15.146:3300/0 conn(0x7f9c300d0a80 0x7f9c300aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.755+0000 7f9c35823640 1 --2- 172.21.15.16:0/3562459954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c301567b0 0x7f9c300d0540 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.755+0000 7f9c35823640 1 -- 172.21.15.16:0/3562459954 >> 172.21.15.16:0/3562459954 conn(0x7f9c300b21f0 msgr2=0x7f9c300b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.755+0000 7f9c35823640 1 -- 172.21.15.16:0/3562459954 shutdown_connections 2024-09-19T07:56:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.755+0000 7f9c35823640 1 -- 172.21.15.16:0/3562459954 wait complete. 2024-09-19T07:56:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.756+0000 7f9c35823640 1 Processor -- start 2024-09-19T07:56:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.756+0000 7f9c35823640 1 -- start start 2024-09-19T07:56:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.757+0000 7f9c35823640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c300af360 0x7f9c30148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.757+0000 7f9c35823640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c300d0a80 0x7f9c30099070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.757+0000 7f9c35823640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c301567b0 0x7f9c30151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.757+0000 7f9c35823640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c300afae0 con 0x7f9c301567b0 2024-09-19T07:56:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.757+0000 7f9c35823640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3015d0e0 con 0x7f9c300af360 2024-09-19T07:56:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.757+0000 7f9c35823640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9c30156b90 con 0x7f9c300d0a80 2024-09-19T07:56:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.757+0000 7f9c2f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c301567b0 0x7f9c30151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.757+0000 7f9c2f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c301567b0 0x7f9c30151840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35388/0 (socket says 172.21.15.16:35388) 2024-09-19T07:56:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.757+0000 7f9c2f7fe640 1 -- 172.21.15.16:0/2800814043 learned_addr learned my addr 172.21.15.16:0/2800814043 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:56:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.757+0000 7f9c2effd640 1 --2- 172.21.15.16:0/2800814043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c300af360 0x7f9c30148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.757+0000 7f9c267fc640 1 --2- 172.21.15.16:0/2800814043 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c300d0a80 0x7f9c30099070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.758+0000 7f9c2f7fe640 1 -- 172.21.15.16:0/2800814043 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c300d0a80 msgr2=0x7f9c30099070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.758+0000 7f9c2f7fe640 1 --2- 172.21.15.16:0/2800814043 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c300d0a80 0x7f9c30099070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.758+0000 7f9c2f7fe640 1 -- 172.21.15.16:0/2800814043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c300af360 msgr2=0x7f9c30148db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:23.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.758+0000 7f9c2f7fe640 1 --2- 172.21.15.16:0/2800814043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c300af360 0x7f9c30148db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:23.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.758+0000 7f9c2f7fe640 1 -- 172.21.15.16:0/2800814043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c30152770 con 0x7f9c301567b0 2024-09-19T07:56:23.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.758+0000 7f9c2f7fe640 1 --2- 172.21.15.16:0/2800814043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c301567b0 0x7f9c30151840 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800cf20 tx=0x7f9c2800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:23.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.758+0000 7f9c2cff9640 1 -- 172.21.15.16:0/2800814043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2801a070 con 0x7f9c301567b0 2024-09-19T07:56:23.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.758+0000 7f9c2cff9640 1 -- 172.21.15.16:0/2800814043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c2800d840 con 0x7f9c301567b0 2024-09-19T07:56:23.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.758+0000 7f9c2cff9640 1 -- 172.21.15.16:0/2800814043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2800dc30 con 0x7f9c301567b0 2024-09-19T07:56:23.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.758+0000 7f9c35823640 1 -- 172.21.15.16:0/2800814043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c301492f0 con 0x7f9c301567b0 2024-09-19T07:56:23.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.758+0000 7f9c35823640 1 -- 172.21.15.16:0/2800814043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c30152490 con 0x7f9c301567b0 2024-09-19T07:56:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.759+0000 7f9c35823640 1 -- 172.21.15.16:0/2800814043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c300ad8f0 con 0x7f9c301567b0 2024-09-19T07:56:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.760+0000 7f9c2cff9640 1 -- 172.21.15.16:0/2800814043 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9c2801d070 con 0x7f9c301567b0 2024-09-19T07:56:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.760+0000 7f9c2cff9640 1 --2- 172.21.15.16:0/2800814043 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9c1006c2b0 0x7f9c1006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.760+0000 7f9c2cff9640 1 -- 172.21.15.16:0/2800814043 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f9c2808bde0 con 0x7f9c301567b0 2024-09-19T07:56:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.760+0000 7f9c2effd640 1 --2- 172.21.15.16:0/2800814043 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9c1006c2b0 0x7f9c1006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.761+0000 7f9c2effd640 1 --2- 172.21.15.16:0/2800814043 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9c1006c2b0 0x7f9c1006e770 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f9c20034000 tx=0x7f9c2002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.765+0000 7f9c2cff9640 1 -- 172.21.15.16:0/2800814043 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9c300ad8f0 con 0x7f9c301567b0 2024-09-19T07:56:23.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.976+0000 7f9c35823640 1 -- 172.21.15.16:0/2800814043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9c300af830 con 0x7f9c301567b0 2024-09-19T07:56:23.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.977+0000 7f9c2cff9640 1 -- 172.21.15.16:0/2800814043 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f9c2805a450 con 0x7f9c301567b0 2024-09-19T07:56:23.977 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:56:23.978 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T07:56:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.979+0000 7f9c35823640 1 -- 172.21.15.16:0/2800814043 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9c1006c2b0 msgr2=0x7f9c1006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.979+0000 7f9c35823640 1 --2- 172.21.15.16:0/2800814043 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9c1006c2b0 0x7f9c1006e770 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f9c20034000 tx=0x7f9c2002e000 comp rx=0 tx=0).stop 2024-09-19T07:56:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.979+0000 7f9c35823640 1 -- 172.21.15.16:0/2800814043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c301567b0 msgr2=0x7f9c30151840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.979+0000 7f9c35823640 1 --2- 172.21.15.16:0/2800814043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c301567b0 0x7f9c30151840 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800cf20 tx=0x7f9c2800a7d0 comp rx=0 tx=0).stop 2024-09-19T07:56:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.979+0000 7f9c35823640 1 -- 172.21.15.16:0/2800814043 shutdown_connections 2024-09-19T07:56:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.979+0000 7f9c35823640 1 --2- 172.21.15.16:0/2800814043 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9c1006c2b0 0x7f9c1006e770 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.979+0000 7f9c35823640 1 --2- 172.21.15.16:0/2800814043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c301567b0 0x7f9c30151840 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.979+0000 7f9c35823640 1 --2- 172.21.15.16:0/2800814043 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c300d0a80 0x7f9c30099070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.979+0000 7f9c35823640 1 --2- 172.21.15.16:0/2800814043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c300af360 0x7f9c30148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.979+0000 7f9c35823640 1 -- 172.21.15.16:0/2800814043 >> 172.21.15.16:0/2800814043 conn(0x7f9c300b21f0 msgr2=0x7f9c3006ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.979+0000 7f9c35823640 1 -- 172.21.15.16:0/2800814043 shutdown_connections 2024-09-19T07:56:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:23.979+0000 7f9c35823640 1 -- 172.21.15.16:0/2800814043 wait complete. 2024-09-19T07:56:23.991 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-19T07:56:23.991 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2024-09-19T07:56:28.991 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:56:29.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.105+0000 7f851b5b7640 1 Processor -- start 2024-09-19T07:56:29.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.105+0000 7f851b5b7640 1 -- start start 2024-09-19T07:56:29.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.106+0000 7f851b5b7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f851415c360 0x7f85140c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:29.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.106+0000 7f851b5b7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f851406e3c0 con 0x7f85140cf3b0 2024-09-19T07:56:29.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.106+0000 7f851b5b7640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f851406e6a0 con 0x7f851406c6c0 2024-09-19T07:56:29.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.106+0000 7f851b5b7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f851406e980 con 0x7f851415c360 2024-09-19T07:56:29.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.106+0000 7f8519b54640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f85140cf3b0 0x7f851406f460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56262/0 (socket says 172.21.15.16:56262) 2024-09-19T07:56:29.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.106+0000 7f8519b54640 1 -- 172.21.15.16:0/1531246813 learned_addr learned my addr 172.21.15.16:0/1531246813 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:56:29.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.106+0000 7f8519353640 1 --2- 172.21.15.16:0/1531246813 >> v2:172.21.15.146:3300/0 conn(0x7f851415c360 0x7f85140c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:29.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.106+0000 7f850bfff640 1 -- 172.21.15.16:0/1531246813 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3247609366 0 0) 0x7f851406e3c0 con 0x7f85140cf3b0 2024-09-19T07:56:29.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.106+0000 7f850bfff640 1 -- 172.21.15.16:0/1531246813 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8500003520 con 0x7f85140cf3b0 2024-09-19T07:56:29.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f850bfff640 1 -- 172.21.15.16:0/1531246813 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3199368252 0 0) 0x7f8500003520 con 0x7f85140cf3b0 2024-09-19T07:56:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f850bfff640 1 -- 172.21.15.16:0/1531246813 >> v2:172.21.15.146:3300/0 conn(0x7f851415c360 msgr2=0x7f85140c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:56:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f850bfff640 1 --2- 172.21.15.16:0/1531246813 >> v2:172.21.15.146:3300/0 conn(0x7f851415c360 0x7f85140c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f850bfff640 1 -- 172.21.15.16:0/1531246813 >> v1:172.21.15.146:6789/0 conn(0x7f851406c6c0 legacy=0x7f851415c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f850bfff640 1 -- 172.21.15.16:0/1531246813 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851406ec60 con 0x7f85140cf3b0 2024-09-19T07:56:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f850bfff640 1 -- 172.21.15.16:0/1531246813 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f850c0019f0 con 0x7f85140cf3b0 2024-09-19T07:56:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f850bfff640 1 -- 172.21.15.16:0/1531246813 >> v1:172.21.15.16:6789/0 conn(0x7f85140cf3b0 legacy=0x7f851406f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f850bfff640 1 --2- 172.21.15.16:0/1531246813 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85000039b0 0x7f8500003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f850bfff640 1 --2- 172.21.15.16:0/1531246813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8500004800 0x7f8500004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f850bfff640 1 --2- 172.21.15.16:0/1531246813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85000051c0 0x7f850000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f850bfff640 1 -- 172.21.15.16:0/1531246813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f851406e980 con 0x7f85000051c0 2024-09-19T07:56:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f850bfff640 1 -- 172.21.15.16:0/1531246813 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f850000c060 con 0x7f8500004800 2024-09-19T07:56:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f850bfff640 1 -- 172.21.15.16:0/1531246813 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f850000c260 con 0x7f85000039b0 2024-09-19T07:56:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f8519b54640 1 --2- 172.21.15.16:0/1531246813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85000051c0 0x7f850000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f8519353640 1 --2- 172.21.15.16:0/1531246813 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85000039b0 0x7f8500003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f8518b52640 1 --2- 172.21.15.16:0/1531246813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8500004800 0x7f8500004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f8519b54640 1 -- 172.21.15.16:0/1531246813 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85000039b0 msgr2=0x7f8500003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:56:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f8519b54640 1 --2- 172.21.15.16:0/1531246813 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85000039b0 0x7f8500003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f8519b54640 1 -- 172.21.15.16:0/1531246813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8500004800 msgr2=0x7f8500004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:56:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f8519b54640 1 --2- 172.21.15.16:0/1531246813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8500004800 0x7f8500004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.107+0000 7f8519b54640 1 -- 172.21.15.16:0/1531246813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8500003520 con 0x7f85000051c0 2024-09-19T07:56:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.108+0000 7f8519b54640 1 -- 172.21.15.16:0/1531246813 reap_dead start 2024-09-19T07:56:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.108+0000 7f8519b54640 1 --2- 172.21.15.16:0/1531246813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85000051c0 0x7f850000b9c0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f850c00a840 tx=0x7f8500005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7bf71017443faf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.108+0000 7f850bfff640 1 -- 172.21.15.16:0/1531246813 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c00a460 con 0x7f85000051c0 2024-09-19T07:56:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.108+0000 7f850bfff640 1 -- 172.21.15.16:0/1531246813 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f850c02f070 con 0x7f85000051c0 2024-09-19T07:56:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.108+0000 7f850bfff640 1 -- 172.21.15.16:0/1531246813 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c02bcb0 con 0x7f85000051c0 2024-09-19T07:56:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.109+0000 7f851b5b7640 1 -- 172.21.15.16:0/1531246813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85000051c0 msgr2=0x7f850000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.109+0000 7f851b5b7640 1 --2- 172.21.15.16:0/1531246813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85000051c0 0x7f850000b9c0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f850c00a840 tx=0x7f8500005830 comp rx=0 tx=0).stop 2024-09-19T07:56:29.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.109+0000 7f851b5b7640 1 -- 172.21.15.16:0/1531246813 shutdown_connections 2024-09-19T07:56:29.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.109+0000 7f851b5b7640 1 --2- 172.21.15.16:0/1531246813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85000051c0 0x7f850000b9c0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:29.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.109+0000 7f851b5b7640 1 -- 172.21.15.16:0/1531246813 >> 172.21.15.16:0/1531246813 conn(0x7f8514070420 msgr2=0x7f851406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:29.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.109+0000 7f851b5b7640 1 -- 172.21.15.16:0/1531246813 shutdown_connections 2024-09-19T07:56:29.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.109+0000 7f851b5b7640 1 -- 172.21.15.16:0/1531246813 wait complete. 2024-09-19T07:56:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.110+0000 7f851b5b7640 1 Processor -- start 2024-09-19T07:56:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.110+0000 7f851b5b7640 1 -- start start 2024-09-19T07:56:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.111+0000 7f851b5b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85000051c0 0x7f85140ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.111+0000 7f851b5b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85140c9070 0x7f85140cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.111+0000 7f851b5b7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8514154aa0 0x7f8514154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.111+0000 7f851b5b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85140cfb70 con 0x7f85140c9070 2024-09-19T07:56:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.111+0000 7f851b5b7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f85140cf9f0 con 0x7f8514154aa0 2024-09-19T07:56:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.111+0000 7f851b5b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f85140cfcf0 con 0x7f85000051c0 2024-09-19T07:56:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.111+0000 7f8518b52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85140c9070 0x7f85140cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.111+0000 7f8518b52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85140c9070 0x7f85140cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35408/0 (socket says 172.21.15.16:35408) 2024-09-19T07:56:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.111+0000 7f8518b52640 1 -- 172.21.15.16:0/494279502 learned_addr learned my addr 172.21.15.16:0/494279502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:56:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.111+0000 7f8518b52640 1 -- 172.21.15.16:0/494279502 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85000051c0 msgr2=0x7f85140ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.111+0000 7f8518b52640 1 --2- 172.21.15.16:0/494279502 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85000051c0 0x7f85140ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.111+0000 7f8519b54640 1 --2- 172.21.15.16:0/494279502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8514154aa0 0x7f8514154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.111+0000 7f8518b52640 1 -- 172.21.15.16:0/494279502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8514154aa0 msgr2=0x7f8514154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.111+0000 7f8518b52640 1 --2- 172.21.15.16:0/494279502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8514154aa0 0x7f8514154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.111+0000 7f8518b52640 1 -- 172.21.15.16:0/494279502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8514155470 con 0x7f85140c9070 2024-09-19T07:56:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.111+0000 7f8519b54640 1 --2- 172.21.15.16:0/494279502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8514154aa0 0x7f8514154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:56:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.111+0000 7f8518b52640 1 --2- 172.21.15.16:0/494279502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85140c9070 0x7f85140cb520 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f850400f5f0 tx=0x7f850400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.112+0000 7f8509ffb640 1 -- 172.21.15.16:0/494279502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85040052e0 con 0x7f85140c9070 2024-09-19T07:56:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.112+0000 7f8509ffb640 1 -- 172.21.15.16:0/494279502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8504018070 con 0x7f85140c9070 2024-09-19T07:56:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.112+0000 7f851b5b7640 1 -- 172.21.15.16:0/494279502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85141e4cd0 con 0x7f85140c9070 2024-09-19T07:56:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.112+0000 7f8509ffb640 1 -- 172.21.15.16:0/494279502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504003810 con 0x7f85140c9070 2024-09-19T07:56:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.112+0000 7f851b5b7640 1 -- 172.21.15.16:0/494279502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85140c9810 con 0x7f85140c9070 2024-09-19T07:56:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.113+0000 7f851b5b7640 1 -- 172.21.15.16:0/494279502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85140c6bf0 con 0x7f85140c9070 2024-09-19T07:56:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.113+0000 7f8509ffb640 1 -- 172.21.15.16:0/494279502 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f85040039b0 con 0x7f85140c9070 2024-09-19T07:56:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.114+0000 7f8509ffb640 1 --2- 172.21.15.16:0/494279502 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f84f806c2b0 0x7f84f806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.114+0000 7f8509ffb640 1 -- 172.21.15.16:0/494279502 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f850408b3a0 con 0x7f85140c9070 2024-09-19T07:56:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.114+0000 7f8519353640 1 --2- 172.21.15.16:0/494279502 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f84f806c2b0 0x7f84f806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:29.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.115+0000 7f8519353640 1 --2- 172.21.15.16:0/494279502 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f84f806c2b0 0x7f84f806e770 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8510009a40 tx=0x7f8510009320 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:29.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.118+0000 7f8509ffb640 1 -- 172.21.15.16:0/494279502 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8504059980 con 0x7f85140c9070 2024-09-19T07:56:29.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.330+0000 7f851b5b7640 1 -- 172.21.15.16:0/494279502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f851406d250 con 0x7f85140c9070 2024-09-19T07:56:29.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.330+0000 7f8509ffb640 1 -- 172.21.15.16:0/494279502 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8504059280 con 0x7f85140c9070 2024-09-19T07:56:29.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:56:29.331 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T07:56:29.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.332+0000 7f851b5b7640 1 -- 172.21.15.16:0/494279502 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f84f806c2b0 msgr2=0x7f84f806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:29.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.332+0000 7f851b5b7640 1 --2- 172.21.15.16:0/494279502 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f84f806c2b0 0x7f84f806e770 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8510009a40 tx=0x7f8510009320 comp rx=0 tx=0).stop 2024-09-19T07:56:29.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.332+0000 7f851b5b7640 1 -- 172.21.15.16:0/494279502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85140c9070 msgr2=0x7f85140cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:29.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.332+0000 7f851b5b7640 1 --2- 172.21.15.16:0/494279502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85140c9070 0x7f85140cb520 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f850400f5f0 tx=0x7f850400fab0 comp rx=0 tx=0).stop 2024-09-19T07:56:29.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.332+0000 7f851b5b7640 1 -- 172.21.15.16:0/494279502 shutdown_connections 2024-09-19T07:56:29.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.332+0000 7f851b5b7640 1 --2- 172.21.15.16:0/494279502 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f84f806c2b0 0x7f84f806e770 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:29.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.332+0000 7f851b5b7640 1 --2- 172.21.15.16:0/494279502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8514154aa0 0x7f8514154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:29.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.332+0000 7f851b5b7640 1 --2- 172.21.15.16:0/494279502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85140c9070 0x7f85140cb520 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:29.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.332+0000 7f851b5b7640 1 --2- 172.21.15.16:0/494279502 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85000051c0 0x7f85140ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:29.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.332+0000 7f851b5b7640 1 -- 172.21.15.16:0/494279502 >> 172.21.15.16:0/494279502 conn(0x7f8514070420 msgr2=0x7f851415c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:29.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.332+0000 7f851b5b7640 1 -- 172.21.15.16:0/494279502 shutdown_connections 2024-09-19T07:56:29.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:29.332+0000 7f851b5b7640 1 -- 172.21.15.16:0/494279502 wait complete. 2024-09-19T07:56:29.345 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-19T07:56:29.345 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2024-09-19T07:56:34.345 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:56:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.460+0000 7f35007c4640 1 Processor -- start 2024-09-19T07:56:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.460+0000 7f35007c4640 1 -- start start 2024-09-19T07:56:34.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.461+0000 7f35007c4640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f34f806f500 0x7f34f8072d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:34.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.461+0000 7f35007c4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34f806dfc0 con 0x7f34f8158e40 2024-09-19T07:56:34.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.461+0000 7f35007c4640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34f806e2a0 con 0x7f34f8072930 2024-09-19T07:56:34.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.461+0000 7f35007c4640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f34f806dd70 con 0x7f34f806f500 2024-09-19T07:56:34.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.461+0000 7f34fe560640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f34f8158e40 0x7f34f80add70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54194/0 (socket says 172.21.15.16:54194) 2024-09-19T07:56:34.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.461+0000 7f34fe560640 1 -- 172.21.15.16:0/563462972 learned_addr learned my addr 172.21.15.16:0/563462972 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:56:34.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.461+0000 7f34fed61640 1 --2- 172.21.15.16:0/563462972 >> v2:172.21.15.146:3300/0 conn(0x7f34f806f500 0x7f34f8072d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:34.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.461+0000 7f34fd55e640 1 -- 172.21.15.16:0/563462972 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2132703483 0 0) 0x7f34f806dfc0 con 0x7f34f8158e40 2024-09-19T07:56:34.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.461+0000 7f34fd55e640 1 -- 172.21.15.16:0/563462972 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34e4003520 con 0x7f34f8158e40 2024-09-19T07:56:34.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.461+0000 7f34fd55e640 1 -- 172.21.15.16:0/563462972 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 698795768 0 0) 0x7f34e4003520 con 0x7f34f8158e40 2024-09-19T07:56:34.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.461+0000 7f34fed61640 1 -- 172.21.15.16:0/563462972 >> v1:172.21.15.146:6789/0 conn(0x7f34f8072930 legacy=0x7f34f806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.461+0000 7f34fed61640 1 -- 172.21.15.16:0/563462972 >> v1:172.21.15.16:6789/0 conn(0x7f34f8158e40 legacy=0x7f34f80add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.461+0000 7f34fed61640 1 -- 172.21.15.16:0/563462972 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f806e580 con 0x7f34f806f500 2024-09-19T07:56:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.462+0000 7f34fed61640 1 --2- 172.21.15.16:0/563462972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34f806f500 0x7f34f8072d80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009d90 tx=0x7f34f002a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24be6c55907bba34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.462+0000 7f34fd55e640 1 -- 172.21.15.16:0/563462972 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4001670 con 0x7f34f806f500 2024-09-19T07:56:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.463+0000 7f34fd55e640 1 -- 172.21.15.16:0/563462972 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f0007540 con 0x7f34f806f500 2024-09-19T07:56:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.463+0000 7f34fd55e640 1 -- 172.21.15.16:0/563462972 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f0007910 con 0x7f34f806f500 2024-09-19T07:56:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.463+0000 7f35007c4640 1 -- 172.21.15.16:0/563462972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34f806f500 msgr2=0x7f34f8072d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.463+0000 7f35007c4640 1 --2- 172.21.15.16:0/563462972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34f806f500 0x7f34f8072d80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009d90 tx=0x7f34f002a680 comp rx=0 tx=0).stop 2024-09-19T07:56:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.463+0000 7f35007c4640 1 -- 172.21.15.16:0/563462972 shutdown_connections 2024-09-19T07:56:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.463+0000 7f35007c4640 1 --2- 172.21.15.16:0/563462972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34f806f500 0x7f34f8072d80 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.463+0000 7f35007c4640 1 -- 172.21.15.16:0/563462972 >> 172.21.15.16:0/563462972 conn(0x7f34f80b21f0 msgr2=0x7f34f80b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.463+0000 7f35007c4640 1 -- 172.21.15.16:0/563462972 shutdown_connections 2024-09-19T07:56:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.463+0000 7f35007c4640 1 -- 172.21.15.16:0/563462972 wait complete. 2024-09-19T07:56:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.464+0000 7f35007c4640 1 Processor -- start 2024-09-19T07:56:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.464+0000 7f35007c4640 1 -- start start 2024-09-19T07:56:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.465+0000 7f35007c4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34f806f500 0x7f34f81f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.465+0000 7f35007c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8072930 0x7f34f814dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.465+0000 7f35007c4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34f8158e40 0x7f34f81576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.465+0000 7f35007c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34f815b9a0 con 0x7f34f8072930 2024-09-19T07:56:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.465+0000 7f35007c4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f34f806ebd0 con 0x7f34f806f500 2024-09-19T07:56:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.465+0000 7f35007c4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f34f815bb20 con 0x7f34f8158e40 2024-09-19T07:56:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.465+0000 7f34fdd5f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8072930 0x7f34f814dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.465+0000 7f34fdd5f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8072930 0x7f34f814dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34802/0 (socket says 172.21.15.16:34802) 2024-09-19T07:56:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.465+0000 7f34fdd5f640 1 -- 172.21.15.16:0/671419943 learned_addr learned my addr 172.21.15.16:0/671419943 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:56:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.466+0000 7f34fed61640 1 --2- 172.21.15.16:0/671419943 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34f8158e40 0x7f34f81576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.466+0000 7f34fe560640 1 --2- 172.21.15.16:0/671419943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34f806f500 0x7f34f81f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.466+0000 7f34fdd5f640 1 -- 172.21.15.16:0/671419943 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34f8158e40 msgr2=0x7f34f81576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.466+0000 7f34fdd5f640 1 --2- 172.21.15.16:0/671419943 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34f8158e40 0x7f34f81576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.466+0000 7f34fdd5f640 1 -- 172.21.15.16:0/671419943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34f806f500 msgr2=0x7f34f81f4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.466+0000 7f34fdd5f640 1 --2- 172.21.15.16:0/671419943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34f806f500 0x7f34f81f4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.466+0000 7f34fdd5f640 1 -- 172.21.15.16:0/671419943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f81525f0 con 0x7f34f8072930 2024-09-19T07:56:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.466+0000 7f34fdd5f640 1 --2- 172.21.15.16:0/671419943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8072930 0x7f34f814dbe0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f34e800c450 tx=0x7f34e800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.466+0000 7f34eeffd640 1 -- 172.21.15.16:0/671419943 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e80052d0 con 0x7f34f8072930 2024-09-19T07:56:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.466+0000 7f34eeffd640 1 -- 172.21.15.16:0/671419943 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e8005470 con 0x7f34f8072930 2024-09-19T07:56:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.466+0000 7f35007c4640 1 -- 172.21.15.16:0/671419943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f81f4760 con 0x7f34f8072930 2024-09-19T07:56:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.466+0000 7f34eeffd640 1 -- 172.21.15.16:0/671419943 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8012870 con 0x7f34f8072930 2024-09-19T07:56:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.466+0000 7f35007c4640 1 -- 172.21.15.16:0/671419943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f81f4d20 con 0x7f34f8072930 2024-09-19T07:56:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.467+0000 7f35007c4640 1 -- 172.21.15.16:0/671419943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f8072d80 con 0x7f34f8072930 2024-09-19T07:56:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.468+0000 7f34eeffd640 1 -- 172.21.15.16:0/671419943 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f34e8012a10 con 0x7f34f8072930 2024-09-19T07:56:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.468+0000 7f34eeffd640 1 --2- 172.21.15.16:0/671419943 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f34dc06c2b0 0x7f34dc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.468+0000 7f34eeffd640 1 -- 172.21.15.16:0/671419943 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f34e808b1f0 con 0x7f34f8072930 2024-09-19T07:56:34.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.468+0000 7f34fe560640 1 --2- 172.21.15.16:0/671419943 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f34dc06c2b0 0x7f34dc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:34.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.469+0000 7f34fe560640 1 --2- 172.21.15.16:0/671419943 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f34dc06c2b0 0x7f34dc06e770 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f34f4005880 tx=0x7f34f4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:34.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.473+0000 7f34eeffd640 1 -- 172.21.15.16:0/671419943 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f34e80597d0 con 0x7f34f8072930 2024-09-19T07:56:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.685+0000 7f35007c4640 1 -- 172.21.15.16:0/671419943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f34f806ff30 con 0x7f34f8072930 2024-09-19T07:56:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.685+0000 7f34eeffd640 1 -- 172.21.15.16:0/671419943 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f34e80590d0 con 0x7f34f8072930 2024-09-19T07:56:34.686 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:56:34.686 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T07:56:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.687+0000 7f35007c4640 1 -- 172.21.15.16:0/671419943 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f34dc06c2b0 msgr2=0x7f34dc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.687+0000 7f35007c4640 1 --2- 172.21.15.16:0/671419943 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f34dc06c2b0 0x7f34dc06e770 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f34f4005880 tx=0x7f34f4009040 comp rx=0 tx=0).stop 2024-09-19T07:56:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.687+0000 7f35007c4640 1 -- 172.21.15.16:0/671419943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8072930 msgr2=0x7f34f814dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.687+0000 7f35007c4640 1 --2- 172.21.15.16:0/671419943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8072930 0x7f34f814dbe0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f34e800c450 tx=0x7f34e800c910 comp rx=0 tx=0).stop 2024-09-19T07:56:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.687+0000 7f35007c4640 1 -- 172.21.15.16:0/671419943 shutdown_connections 2024-09-19T07:56:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.687+0000 7f35007c4640 1 --2- 172.21.15.16:0/671419943 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f34dc06c2b0 0x7f34dc06e770 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.687+0000 7f35007c4640 1 --2- 172.21.15.16:0/671419943 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34f8158e40 0x7f34f81576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.687+0000 7f35007c4640 1 --2- 172.21.15.16:0/671419943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8072930 0x7f34f814dbe0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.687+0000 7f35007c4640 1 --2- 172.21.15.16:0/671419943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34f806f500 0x7f34f81f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.687+0000 7f35007c4640 1 -- 172.21.15.16:0/671419943 >> 172.21.15.16:0/671419943 conn(0x7f34f80b21f0 msgr2=0x7f34f80add30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.687+0000 7f35007c4640 1 -- 172.21.15.16:0/671419943 shutdown_connections 2024-09-19T07:56:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:34.687+0000 7f35007c4640 1 -- 172.21.15.16:0/671419943 wait complete. 2024-09-19T07:56:34.698 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-19T07:56:34.699 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2024-09-19T07:56:39.700 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:56:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.814+0000 7fd3ae09b640 1 Processor -- start 2024-09-19T07:56:39.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.814+0000 7fd3ae09b640 1 -- start start 2024-09-19T07:56:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.815+0000 7fd3ae09b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a806f500 0x7fd3a8072d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.815+0000 7fd3ae09b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3a806dfc0 con 0x7fd3a8158e40 2024-09-19T07:56:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.815+0000 7fd3ae09b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3a806e2a0 con 0x7fd3a8072930 2024-09-19T07:56:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.815+0000 7fd3ae09b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a806dd70 con 0x7fd3a806f500 2024-09-19T07:56:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.815+0000 7fd3a77fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd3a8158e40 0x7fd3a80add70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54204/0 (socket says 172.21.15.16:54204) 2024-09-19T07:56:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.815+0000 7fd3a77fe640 1 -- 172.21.15.16:0/1504265368 learned_addr learned my addr 172.21.15.16:0/1504265368 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:56:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.815+0000 7fd3a7fff640 1 --2- 172.21.15.16:0/1504265368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a806f500 0x7fd3a8072d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.815+0000 7fd3a67fc640 1 -- 172.21.15.16:0/1504265368 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1287463406 0 0) 0x7fd3a806dfc0 con 0x7fd3a8158e40 2024-09-19T07:56:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.815+0000 7fd3a67fc640 1 -- 172.21.15.16:0/1504265368 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd390003520 con 0x7fd3a8158e40 2024-09-19T07:56:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a67fc640 1 -- 172.21.15.16:0/1504265368 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3014442373 0 0) 0x7fd390003520 con 0x7fd3a8158e40 2024-09-19T07:56:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a67fc640 1 -- 172.21.15.16:0/1504265368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a806f500 msgr2=0x7fd3a8072d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:56:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a67fc640 1 --2- 172.21.15.16:0/1504265368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a806f500 0x7fd3a8072d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a67fc640 1 -- 172.21.15.16:0/1504265368 >> v1:172.21.15.146:6789/0 conn(0x7fd3a8072930 legacy=0x7fd3a806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a67fc640 1 -- 172.21.15.16:0/1504265368 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a806e580 con 0x7fd3a8158e40 2024-09-19T07:56:39.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a67fc640 1 -- 172.21.15.16:0/1504265368 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fd398001670 con 0x7fd3a8158e40 2024-09-19T07:56:39.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a67fc640 1 -- 172.21.15.16:0/1504265368 >> v1:172.21.15.16:6789/0 conn(0x7fd3a8158e40 legacy=0x7fd3a80add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:39.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a67fc640 1 --2- 172.21.15.16:0/1504265368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a806f500 0x7fd3900039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:39.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a67fc640 1 --2- 172.21.15.16:0/1504265368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd390003f60 0x7fd390004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:39.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a67fc640 1 --2- 172.21.15.16:0/1504265368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd390004d40 0x7fd3900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:39.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a67fc640 1 -- 172.21.15.16:0/1504265368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a806dd70 con 0x7fd390004d40 2024-09-19T07:56:39.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a67fc640 1 -- 172.21.15.16:0/1504265368 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd39000baa0 con 0x7fd390003f60 2024-09-19T07:56:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a67fc640 1 -- 172.21.15.16:0/1504265368 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd39000bca0 con 0x7fd3a806f500 2024-09-19T07:56:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a7fff640 1 --2- 172.21.15.16:0/1504265368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd390004d40 0x7fd3900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a77fe640 1 --2- 172.21.15.16:0/1504265368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a806f500 0x7fd3900039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a6ffd640 1 --2- 172.21.15.16:0/1504265368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd390003f60 0x7fd390004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a7fff640 1 -- 172.21.15.16:0/1504265368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a806f500 msgr2=0x7fd3900039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:56:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a7fff640 1 --2- 172.21.15.16:0/1504265368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a806f500 0x7fd3900039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a7fff640 1 -- 172.21.15.16:0/1504265368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd390003f60 msgr2=0x7fd390004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:56:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a7fff640 1 --2- 172.21.15.16:0/1504265368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd390003f60 0x7fd390004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.816+0000 7fd3a7fff640 1 -- 172.21.15.16:0/1504265368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd390003520 con 0x7fd390004d40 2024-09-19T07:56:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.817+0000 7fd3a7fff640 1 --2- 172.21.15.16:0/1504265368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd390004d40 0x7fd3900051c0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009fc0 tx=0x7fd390005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473f64a888d6a1e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.817+0000 7fd3a67fc640 1 -- 172.21.15.16:0/1504265368 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0009c10 con 0x7fd390004d40 2024-09-19T07:56:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.817+0000 7fd3a67fc640 1 -- 172.21.15.16:0/1504265368 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3a0036070 con 0x7fd390004d40 2024-09-19T07:56:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.817+0000 7fd3a67fc640 1 -- 172.21.15.16:0/1504265368 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0031880 con 0x7fd390004d40 2024-09-19T07:56:39.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.818+0000 7fd3ae09b640 1 -- 172.21.15.16:0/1504265368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd390004d40 msgr2=0x7fd3900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:39.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.818+0000 7fd3ae09b640 1 --2- 172.21.15.16:0/1504265368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd390004d40 0x7fd3900051c0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009fc0 tx=0x7fd390005830 comp rx=0 tx=0).stop 2024-09-19T07:56:39.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.818+0000 7fd3a7fff640 1 -- 172.21.15.16:0/1504265368 reap_dead start 2024-09-19T07:56:39.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.818+0000 7fd3ae09b640 1 -- 172.21.15.16:0/1504265368 shutdown_connections 2024-09-19T07:56:39.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.818+0000 7fd3ae09b640 1 -- 172.21.15.16:0/1504265368 >> 172.21.15.16:0/1504265368 conn(0x7fd3a80b21f0 msgr2=0x7fd3a80b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:39.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.818+0000 7fd3ae09b640 1 -- 172.21.15.16:0/1504265368 shutdown_connections 2024-09-19T07:56:39.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.818+0000 7fd3ae09b640 1 -- 172.21.15.16:0/1504265368 wait complete. 2024-09-19T07:56:39.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.819+0000 7fd3ae09b640 1 Processor -- start 2024-09-19T07:56:39.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.819+0000 7fd3ae09b640 1 -- start start 2024-09-19T07:56:39.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.820+0000 7fd3ae09b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd390004d40 0x7fd3a80adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:39.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.820+0000 7fd3ae09b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a8146500 0x7fd3a815d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:39.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.820+0000 7fd3ae09b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a81e7cf0 0x7fd3a81e8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:39.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.820+0000 7fd3ae09b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a8072d80 con 0x7fd3a8146500 2024-09-19T07:56:39.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.820+0000 7fd3ae09b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a806ebd0 con 0x7fd390004d40 2024-09-19T07:56:39.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.820+0000 7fd3ae09b640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a8072f00 con 0x7fd3a81e7cf0 2024-09-19T07:56:39.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.820+0000 7fd3a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a8146500 0x7fd3a815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:39.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.820+0000 7fd3a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a8146500 0x7fd3a815d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34830/0 (socket says 172.21.15.16:34830) 2024-09-19T07:56:39.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.820+0000 7fd3a6ffd640 1 -- 172.21.15.16:0/1070342370 learned_addr learned my addr 172.21.15.16:0/1070342370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:56:39.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.820+0000 7fd3a77fe640 1 --2- 172.21.15.16:0/1070342370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd390004d40 0x7fd3a80adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:39.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.820+0000 7fd3a7fff640 1 --2- 172.21.15.16:0/1070342370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a81e7cf0 0x7fd3a81e8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:39.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.821+0000 7fd3a6ffd640 1 -- 172.21.15.16:0/1070342370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a81e7cf0 msgr2=0x7fd3a81e8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:39.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.821+0000 7fd3a6ffd640 1 --2- 172.21.15.16:0/1070342370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a81e7cf0 0x7fd3a81e8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:39.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.821+0000 7fd3a6ffd640 1 -- 172.21.15.16:0/1070342370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd390004d40 msgr2=0x7fd3a80adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:39.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.821+0000 7fd3a6ffd640 1 --2- 172.21.15.16:0/1070342370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd390004d40 0x7fd3a80adf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:39.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.821+0000 7fd3a6ffd640 1 -- 172.21.15.16:0/1070342370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a815d940 con 0x7fd3a8146500 2024-09-19T07:56:39.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.821+0000 7fd3a77fe640 1 --2- 172.21.15.16:0/1070342370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd390004d40 0x7fd3a80adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:56:39.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.821+0000 7fd3a6ffd640 1 --2- 172.21.15.16:0/1070342370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a8146500 0x7fd3a815d400 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00f850 tx=0x7fd39c00fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:39.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.821+0000 7fd38ffff640 1 -- 172.21.15.16:0/1070342370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c00fe40 con 0x7fd3a8146500 2024-09-19T07:56:39.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.821+0000 7fd38ffff640 1 -- 172.21.15.16:0/1070342370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd39c018070 con 0x7fd3a8146500 2024-09-19T07:56:39.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.821+0000 7fd38ffff640 1 -- 172.21.15.16:0/1070342370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c00d730 con 0x7fd3a8146500 2024-09-19T07:56:39.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.821+0000 7fd3ae09b640 1 -- 172.21.15.16:0/1070342370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a815e490 con 0x7fd3a8146500 2024-09-19T07:56:39.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.821+0000 7fd3ae09b640 1 -- 172.21.15.16:0/1070342370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a8146980 con 0x7fd3a8146500 2024-09-19T07:56:39.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.822+0000 7fd3ae09b640 1 -- 172.21.15.16:0/1070342370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a806edf0 con 0x7fd3a8146500 2024-09-19T07:56:39.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.823+0000 7fd38ffff640 1 -- 172.21.15.16:0/1070342370 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd39c00d8d0 con 0x7fd3a8146500 2024-09-19T07:56:39.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.823+0000 7fd38ffff640 1 --2- 172.21.15.16:0/1070342370 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd39406c2e0 0x7fd39406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:39.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.823+0000 7fd38ffff640 1 -- 172.21.15.16:0/1070342370 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7fd39c08b3c0 con 0x7fd3a8146500 2024-09-19T07:56:39.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.823+0000 7fd3a77fe640 1 --2- 172.21.15.16:0/1070342370 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd39406c2e0 0x7fd39406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:39.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.824+0000 7fd3a77fe640 1 --2- 172.21.15.16:0/1070342370 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd39406c2e0 0x7fd39406e7a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd398007db0 tx=0x7fd398009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:39.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:39.828+0000 7fd38ffff640 1 -- 172.21.15.16:0/1070342370 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd39c0599a0 con 0x7fd3a8146500 2024-09-19T07:56:39.871 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:56:39.873 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:56:40.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:40.040+0000 7fd3ae09b640 1 -- 172.21.15.16:0/1070342370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd3a80ceb60 con 0x7fd3a8146500 2024-09-19T07:56:40.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:40.040+0000 7fd38ffff640 1 -- 172.21.15.16:0/1070342370 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd39c0592a0 con 0x7fd3a8146500 2024-09-19T07:56:40.041 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:56:40.041 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T07:56:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:40.042+0000 7fd3ae09b640 1 -- 172.21.15.16:0/1070342370 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd39406c2e0 msgr2=0x7fd39406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:40.042+0000 7fd3ae09b640 1 --2- 172.21.15.16:0/1070342370 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd39406c2e0 0x7fd39406e7a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd398007db0 tx=0x7fd398009040 comp rx=0 tx=0).stop 2024-09-19T07:56:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:40.042+0000 7fd3ae09b640 1 -- 172.21.15.16:0/1070342370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a8146500 msgr2=0x7fd3a815d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:40.042+0000 7fd3ae09b640 1 --2- 172.21.15.16:0/1070342370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a8146500 0x7fd3a815d400 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00f850 tx=0x7fd39c00fd10 comp rx=0 tx=0).stop 2024-09-19T07:56:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:40.042+0000 7fd3ae09b640 1 -- 172.21.15.16:0/1070342370 shutdown_connections 2024-09-19T07:56:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:40.042+0000 7fd3ae09b640 1 --2- 172.21.15.16:0/1070342370 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd39406c2e0 0x7fd39406e7a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:40.042+0000 7fd3ae09b640 1 --2- 172.21.15.16:0/1070342370 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a81e7cf0 0x7fd3a81e8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:40.042+0000 7fd3ae09b640 1 --2- 172.21.15.16:0/1070342370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a8146500 0x7fd3a815d400 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:40.042+0000 7fd3ae09b640 1 --2- 172.21.15.16:0/1070342370 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd390004d40 0x7fd3a80adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:40.042+0000 7fd3ae09b640 1 -- 172.21.15.16:0/1070342370 >> 172.21.15.16:0/1070342370 conn(0x7fd3a80b21f0 msgr2=0x7fd3a8157350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:40.042+0000 7fd3ae09b640 1 -- 172.21.15.16:0/1070342370 shutdown_connections 2024-09-19T07:56:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:40.042+0000 7fd3ae09b640 1 -- 172.21.15.16:0/1070342370 wait complete. 2024-09-19T07:56:40.054 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-19T07:56:40.054 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2024-09-19T07:56:45.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:56:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.169+0000 7f2227fff640 1 Processor -- start 2024-09-19T07:56:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.169+0000 7f2227fff640 1 -- start start 2024-09-19T07:56:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.170+0000 7f2227fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f22280b3400 0x7f22280d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.170+0000 7f2227fff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22280cfc00 con 0x7f222815e5e0 2024-09-19T07:56:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.170+0000 7f2227fff640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22280cfee0 con 0x7f22280d2f20 2024-09-19T07:56:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.170+0000 7f2227fff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f22280b1090 con 0x7f22280b3400 2024-09-19T07:56:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.170+0000 7f22277fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f22280b3400 0x7f22280d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.170+0000 7f2226ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f222815e5e0 0x7f22280d2810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57454/0 (socket says 172.21.15.16:57454) 2024-09-19T07:56:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.170+0000 7f2226ffd640 1 -- 172.21.15.16:0/511942888 learned_addr learned my addr 172.21.15.16:0/511942888 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:56:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.171+0000 7f22277fe640 1 -- 172.21.15.16:0/511942888 >> v1:172.21.15.146:6789/0 conn(0x7f22280d2f20 legacy=0x7f22280b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.171+0000 7f22277fe640 1 -- 172.21.15.16:0/511942888 >> v1:172.21.15.16:6789/0 conn(0x7f222815e5e0 legacy=0x7f22280d2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.171+0000 7f22277fe640 1 -- 172.21.15.16:0/511942888 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22280d01c0 con 0x7f22280b3400 2024-09-19T07:56:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.171+0000 7f22277fe640 1 --2- 172.21.15.16:0/511942888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22280b3400 0x7f22280d1fa0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f2220009d90 tx=0x7f222002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13c080273765b36 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.172+0000 7f2225ffb640 1 -- 172.21.15.16:0/511942888 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220007740 con 0x7f22280b3400 2024-09-19T07:56:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.172+0000 7f2225ffb640 1 -- 172.21.15.16:0/511942888 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22200078e0 con 0x7f22280b3400 2024-09-19T07:56:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.172+0000 7f2225ffb640 1 -- 172.21.15.16:0/511942888 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220007cb0 con 0x7f22280b3400 2024-09-19T07:56:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.172+0000 7f2227fff640 1 -- 172.21.15.16:0/511942888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22280b3400 msgr2=0x7f22280d1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.172+0000 7f2227fff640 1 --2- 172.21.15.16:0/511942888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22280b3400 0x7f22280d1fa0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f2220009d90 tx=0x7f222002a6f0 comp rx=0 tx=0).stop 2024-09-19T07:56:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.172+0000 7f2227fff640 1 -- 172.21.15.16:0/511942888 shutdown_connections 2024-09-19T07:56:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.172+0000 7f2227fff640 1 --2- 172.21.15.16:0/511942888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22280b3400 0x7f22280d1fa0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.172+0000 7f2227fff640 1 -- 172.21.15.16:0/511942888 >> 172.21.15.16:0/511942888 conn(0x7f222809d630 msgr2=0x7f22280d43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.172+0000 7f2227fff640 1 -- 172.21.15.16:0/511942888 shutdown_connections 2024-09-19T07:56:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.172+0000 7f2227fff640 1 -- 172.21.15.16:0/511942888 wait complete. 2024-09-19T07:56:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.174+0000 7f2227fff640 1 Processor -- start 2024-09-19T07:56:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.174+0000 7f2227fff640 1 -- start start 2024-09-19T07:56:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.174+0000 7f2227fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22280b3400 0x7f22280cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.174+0000 7f2227fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22280d2f20 0x7f222815ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.174+0000 7f2227fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222815e5e0 0x7f22281e56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.174+0000 7f2227fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f22280b2040 con 0x7f222815e5e0 2024-09-19T07:56:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.174+0000 7f2227fff640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f22280b1ec0 con 0x7f22280b3400 2024-09-19T07:56:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.174+0000 7f2227fff640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f22280b21c0 con 0x7f22280d2f20 2024-09-19T07:56:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.174+0000 7f22277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222815e5e0 0x7f22281e56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.175+0000 7f22277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222815e5e0 0x7f22281e56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44224/0 (socket says 172.21.15.16:44224) 2024-09-19T07:56:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.175+0000 7f22277fe640 1 -- 172.21.15.16:0/165550567 learned_addr learned my addr 172.21.15.16:0/165550567 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:56:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.175+0000 7f2226ffd640 1 --2- 172.21.15.16:0/165550567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22280b3400 0x7f22280cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.175+0000 7f22267fc640 1 --2- 172.21.15.16:0/165550567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22280d2f20 0x7f222815ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.175+0000 7f22277fe640 1 -- 172.21.15.16:0/165550567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22280d2f20 msgr2=0x7f222815ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.175+0000 7f22277fe640 1 --2- 172.21.15.16:0/165550567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22280d2f20 0x7f222815ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.175+0000 7f22277fe640 1 -- 172.21.15.16:0/165550567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22280b3400 msgr2=0x7f22280cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.175+0000 7f22277fe640 1 --2- 172.21.15.16:0/165550567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22280b3400 0x7f22280cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.175+0000 7f22277fe640 1 -- 172.21.15.16:0/165550567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22280cce60 con 0x7f222815e5e0 2024-09-19T07:56:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.175+0000 7f22277fe640 1 --2- 172.21.15.16:0/165550567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222815e5e0 0x7f22281e56e0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f2220009eb0 tx=0x7f2220004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.175+0000 7f220f7fe640 1 -- 172.21.15.16:0/165550567 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220040070 con 0x7f222815e5e0 2024-09-19T07:56:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.175+0000 7f220f7fe640 1 -- 172.21.15.16:0/165550567 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2220031c90 con 0x7f222815e5e0 2024-09-19T07:56:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.175+0000 7f2227fff640 1 -- 172.21.15.16:0/165550567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22280cd140 con 0x7f222815e5e0 2024-09-19T07:56:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.175+0000 7f220f7fe640 1 -- 172.21.15.16:0/165550567 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220045680 con 0x7f222815e5e0 2024-09-19T07:56:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.175+0000 7f2227fff640 1 -- 172.21.15.16:0/165550567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22280cd700 con 0x7f222815e5e0 2024-09-19T07:56:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.176+0000 7f2227fff640 1 -- 172.21.15.16:0/165550567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22280b25e0 con 0x7f222815e5e0 2024-09-19T07:56:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.177+0000 7f220f7fe640 1 -- 172.21.15.16:0/165550567 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2220039040 con 0x7f222815e5e0 2024-09-19T07:56:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.178+0000 7f220f7fe640 1 --2- 172.21.15.16:0/165550567 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f221406c2e0 0x7f221406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.178+0000 7f220f7fe640 1 -- 172.21.15.16:0/165550567 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f22200acfb0 con 0x7f222815e5e0 2024-09-19T07:56:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.178+0000 7f2226ffd640 1 --2- 172.21.15.16:0/165550567 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f221406c2e0 0x7f221406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.179+0000 7f2226ffd640 1 --2- 172.21.15.16:0/165550567 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f221406c2e0 0x7f221406e7a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2218005880 tx=0x7f2218009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.182+0000 7f220f7fe640 1 -- 172.21.15.16:0/165550567 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2220043360 con 0x7f222815e5e0 2024-09-19T07:56:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.393+0000 7f2227fff640 1 -- 172.21.15.16:0/165550567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f22280d1230 con 0x7f222815e5e0 2024-09-19T07:56:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.394+0000 7f220f7fe640 1 -- 172.21.15.16:0/165550567 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f222007b770 con 0x7f222815e5e0 2024-09-19T07:56:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:56:45.394 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T07:56:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.396+0000 7f2227fff640 1 -- 172.21.15.16:0/165550567 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f221406c2e0 msgr2=0x7f221406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.396+0000 7f2227fff640 1 --2- 172.21.15.16:0/165550567 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f221406c2e0 0x7f221406e7a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2218005880 tx=0x7f2218009040 comp rx=0 tx=0).stop 2024-09-19T07:56:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.396+0000 7f2227fff640 1 -- 172.21.15.16:0/165550567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222815e5e0 msgr2=0x7f22281e56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.396+0000 7f2227fff640 1 --2- 172.21.15.16:0/165550567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222815e5e0 0x7f22281e56e0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f2220009eb0 tx=0x7f2220004a40 comp rx=0 tx=0).stop 2024-09-19T07:56:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.396+0000 7f2227fff640 1 -- 172.21.15.16:0/165550567 shutdown_connections 2024-09-19T07:56:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.396+0000 7f2227fff640 1 --2- 172.21.15.16:0/165550567 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f221406c2e0 0x7f221406e7a0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.396+0000 7f2227fff640 1 --2- 172.21.15.16:0/165550567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222815e5e0 0x7f22281e56e0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.396+0000 7f2227fff640 1 --2- 172.21.15.16:0/165550567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22280d2f20 0x7f222815ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.396+0000 7f2227fff640 1 --2- 172.21.15.16:0/165550567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22280b3400 0x7f22280cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.396+0000 7f2227fff640 1 -- 172.21.15.16:0/165550567 >> 172.21.15.16:0/165550567 conn(0x7f222809d630 msgr2=0x7f22280b7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.396+0000 7f2227fff640 1 -- 172.21.15.16:0/165550567 shutdown_connections 2024-09-19T07:56:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:45.396+0000 7f2227fff640 1 -- 172.21.15.16:0/165550567 wait complete. 2024-09-19T07:56:45.407 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-19T07:56:45.407 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2024-09-19T07:56:50.408 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:56:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.522+0000 7fdaa3786640 1 Processor -- start 2024-09-19T07:56:50.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.522+0000 7fdaa3786640 1 -- start start 2024-09-19T07:56:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.523+0000 7fdaa3786640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fda9c15e700 0x7fda9c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.523+0000 7fdaa3786640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fda9c0d0510 0x7fda9c09aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.523+0000 7fdaa3786640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda9c0d09f0 con 0x7fda9c06d260 2024-09-19T07:56:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.523+0000 7fdaa3786640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fda9c0d1600 con 0x7fda9c15e700 2024-09-19T07:56:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.523+0000 7fdaa3786640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fda9c0cd930 con 0x7fda9c0d0510 2024-09-19T07:56:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.523+0000 7fdaa1522640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fda9c15e700 0x7fda9c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.523+0000 7fdaa1d23640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fda9c06d260 0x7fda9c162060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57462/0 (socket says 172.21.15.16:57462) 2024-09-19T07:56:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.523+0000 7fdaa1d23640 1 -- 172.21.15.16:0/3503643464 learned_addr learned my addr 172.21.15.16:0/3503643464 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:56:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.524+0000 7fdaa0d21640 1 --2- 172.21.15.16:0/3503643464 >> v2:172.21.15.146:3300/0 conn(0x7fda9c0d0510 0x7fda9c09aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.524+0000 7fdaa1522640 1 -- 172.21.15.16:0/3503643464 >> v2:172.21.15.146:3300/0 conn(0x7fda9c0d0510 msgr2=0x7fda9c09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:56:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.524+0000 7fdaa1522640 1 --2- 172.21.15.16:0/3503643464 >> v2:172.21.15.146:3300/0 conn(0x7fda9c0d0510 0x7fda9c09aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.524+0000 7fdaa1522640 1 -- 172.21.15.16:0/3503643464 >> v1:172.21.15.16:6789/0 conn(0x7fda9c06d260 legacy=0x7fda9c162060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.524+0000 7fda93fff640 1 -- 172.21.15.16:0/3503643464 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4058706649 0 0) 0x7fda9c0d09f0 con 0x7fda9c06d260 2024-09-19T07:56:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.524+0000 7fdaa1522640 1 -- 172.21.15.16:0/3503643464 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9c0d0cd0 con 0x7fda9c15e700 2024-09-19T07:56:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.525+0000 7fdaa1522640 1 --2- 172.21.15.16:0/3503643464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda9c15e700 0x7fda9c06cd20 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0d1fc0 tx=0x7fda9802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31366a7f123a7b7d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.525+0000 7fda93fff640 1 -- 172.21.15.16:0/3503643464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98037070 con 0x7fda9c15e700 2024-09-19T07:56:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.525+0000 7fda93fff640 1 -- 172.21.15.16:0/3503643464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda98032440 con 0x7fda9c15e700 2024-09-19T07:56:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.525+0000 7fda93fff640 1 -- 172.21.15.16:0/3503643464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98032810 con 0x7fda9c15e700 2024-09-19T07:56:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.525+0000 7fdaa3786640 1 -- 172.21.15.16:0/3503643464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda9c15e700 msgr2=0x7fda9c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.525+0000 7fdaa3786640 1 --2- 172.21.15.16:0/3503643464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda9c15e700 0x7fda9c06cd20 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0d1fc0 tx=0x7fda9802a1a0 comp rx=0 tx=0).stop 2024-09-19T07:56:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.525+0000 7fdaa3786640 1 -- 172.21.15.16:0/3503643464 shutdown_connections 2024-09-19T07:56:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.525+0000 7fdaa3786640 1 --2- 172.21.15.16:0/3503643464 >> v2:172.21.15.146:3300/0 conn(0x7fda9c0d0510 0x7fda9c09aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.525+0000 7fdaa3786640 1 --2- 172.21.15.16:0/3503643464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda9c15e700 0x7fda9c06cd20 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.525+0000 7fdaa3786640 1 -- 172.21.15.16:0/3503643464 >> 172.21.15.16:0/3503643464 conn(0x7fda9c0d22d0 msgr2=0x7fda9c0d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.525+0000 7fdaa3786640 1 -- 172.21.15.16:0/3503643464 shutdown_connections 2024-09-19T07:56:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.525+0000 7fdaa3786640 1 -- 172.21.15.16:0/3503643464 wait complete. 2024-09-19T07:56:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.527+0000 7fdaa3786640 1 Processor -- start 2024-09-19T07:56:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.527+0000 7fdaa3786640 1 -- start start 2024-09-19T07:56:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.527+0000 7fdaa3786640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda9c06d260 0x7fda9c0c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.527+0000 7fdaa3786640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda9c0d0510 0x7fda9c0b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.527+0000 7fdaa3786640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda9c1f57e0 0x7fda9c1e8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.527+0000 7fdaa3786640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c0d1a90 con 0x7fda9c1f57e0 2024-09-19T07:56:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.527+0000 7fdaa3786640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c0d1290 con 0x7fda9c0d0510 2024-09-19T07:56:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.527+0000 7fdaa3786640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fda9c153d30 con 0x7fda9c06d260 2024-09-19T07:56:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.528+0000 7fdaa1d23640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda9c1f57e0 0x7fda9c1e8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.528+0000 7fdaa1d23640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda9c1f57e0 0x7fda9c1e8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44232/0 (socket says 172.21.15.16:44232) 2024-09-19T07:56:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.528+0000 7fdaa1d23640 1 -- 172.21.15.16:0/296701643 learned_addr learned my addr 172.21.15.16:0/296701643 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:56:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.528+0000 7fdaa0d21640 1 --2- 172.21.15.16:0/296701643 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda9c0d0510 0x7fda9c0b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.528+0000 7fdaa1522640 1 --2- 172.21.15.16:0/296701643 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda9c06d260 0x7fda9c0c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.528+0000 7fdaa1d23640 1 -- 172.21.15.16:0/296701643 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda9c06d260 msgr2=0x7fda9c0c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.528+0000 7fdaa1d23640 1 --2- 172.21.15.16:0/296701643 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda9c06d260 0x7fda9c0c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.528+0000 7fdaa1d23640 1 -- 172.21.15.16:0/296701643 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda9c0d0510 msgr2=0x7fda9c0b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.528+0000 7fdaa1d23640 1 --2- 172.21.15.16:0/296701643 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda9c0d0510 0x7fda9c0b6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.528+0000 7fdaa1d23640 1 -- 172.21.15.16:0/296701643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9c0b70a0 con 0x7fda9c1f57e0 2024-09-19T07:56:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.528+0000 7fdaa0d21640 1 --2- 172.21.15.16:0/296701643 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda9c0d0510 0x7fda9c0b6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:56:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.528+0000 7fdaa1522640 1 --2- 172.21.15.16:0/296701643 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda9c06d260 0x7fda9c0c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:56:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.528+0000 7fdaa1d23640 1 --2- 172.21.15.16:0/296701643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda9c1f57e0 0x7fda9c1e8d30 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fda9400c7d0 tx=0x7fda9400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.528+0000 7fda91ffb640 1 -- 172.21.15.16:0/296701643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94018070 con 0x7fda9c1f57e0 2024-09-19T07:56:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.528+0000 7fda91ffb640 1 -- 172.21.15.16:0/296701643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda9400a5e0 con 0x7fda9c1f57e0 2024-09-19T07:56:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.528+0000 7fdaa3786640 1 -- 172.21.15.16:0/296701643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda9c1f5c90 con 0x7fda9c1f57e0 2024-09-19T07:56:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.528+0000 7fda91ffb640 1 -- 172.21.15.16:0/296701643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9400aa10 con 0x7fda9c1f57e0 2024-09-19T07:56:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.529+0000 7fdaa3786640 1 -- 172.21.15.16:0/296701643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda9c1f6530 con 0x7fda9c1f57e0 2024-09-19T07:56:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.530+0000 7fdaa3786640 1 -- 172.21.15.16:0/296701643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda9c09aa50 con 0x7fda9c1f57e0 2024-09-19T07:56:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.530+0000 7fda91ffb640 1 -- 172.21.15.16:0/296701643 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fda9401b070 con 0x7fda9c1f57e0 2024-09-19T07:56:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.530+0000 7fda91ffb640 1 --2- 172.21.15.16:0/296701643 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fda8006c2b0 0x7fda8006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.530+0000 7fda91ffb640 1 -- 172.21.15.16:0/296701643 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7fda9408abb0 con 0x7fda9c1f57e0 2024-09-19T07:56:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.531+0000 7fdaa1522640 1 --2- 172.21.15.16:0/296701643 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fda8006c2b0 0x7fda8006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.531+0000 7fdaa1522640 1 --2- 172.21.15.16:0/296701643 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fda8006c2b0 0x7fda8006e770 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fda98034000 tx=0x7fda9802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.535+0000 7fda91ffb640 1 -- 172.21.15.16:0/296701643 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fda94059190 con 0x7fda9c1f57e0 2024-09-19T07:56:50.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.753+0000 7fdaa3786640 1 -- 172.21.15.16:0/296701643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fda9c0b3be0 con 0x7fda9c1f57e0 2024-09-19T07:56:50.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.754+0000 7fda91ffb640 1 -- 172.21.15.16:0/296701643 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fda94058a90 con 0x7fda9c1f57e0 2024-09-19T07:56:50.755 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:56:50.755 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T07:56:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.756+0000 7fdaa3786640 1 -- 172.21.15.16:0/296701643 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fda8006c2b0 msgr2=0x7fda8006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.756+0000 7fdaa3786640 1 --2- 172.21.15.16:0/296701643 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fda8006c2b0 0x7fda8006e770 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fda98034000 tx=0x7fda9802e000 comp rx=0 tx=0).stop 2024-09-19T07:56:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.756+0000 7fdaa3786640 1 -- 172.21.15.16:0/296701643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda9c1f57e0 msgr2=0x7fda9c1e8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.756+0000 7fdaa3786640 1 --2- 172.21.15.16:0/296701643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda9c1f57e0 0x7fda9c1e8d30 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fda9400c7d0 tx=0x7fda9400cc90 comp rx=0 tx=0).stop 2024-09-19T07:56:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.756+0000 7fdaa3786640 1 -- 172.21.15.16:0/296701643 shutdown_connections 2024-09-19T07:56:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.756+0000 7fdaa3786640 1 --2- 172.21.15.16:0/296701643 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fda8006c2b0 0x7fda8006e770 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.756+0000 7fdaa3786640 1 --2- 172.21.15.16:0/296701643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda9c1f57e0 0x7fda9c1e8d30 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.756+0000 7fdaa3786640 1 --2- 172.21.15.16:0/296701643 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda9c0d0510 0x7fda9c0b6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.756+0000 7fdaa3786640 1 --2- 172.21.15.16:0/296701643 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda9c06d260 0x7fda9c0c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.756+0000 7fdaa3786640 1 -- 172.21.15.16:0/296701643 >> 172.21.15.16:0/296701643 conn(0x7fda9c0d22d0 msgr2=0x7fda9c0cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.756+0000 7fdaa3786640 1 -- 172.21.15.16:0/296701643 shutdown_connections 2024-09-19T07:56:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:50.756+0000 7fdaa3786640 1 -- 172.21.15.16:0/296701643 wait complete. 2024-09-19T07:56:50.768 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-19T07:56:50.768 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2024-09-19T07:56:55.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:56:55.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.883+0000 7f3c8741e640 1 Processor -- start 2024-09-19T07:56:55.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.883+0000 7f3c8741e640 1 -- start start 2024-09-19T07:56:55.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.884+0000 7f3c8741e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3c800d10a0 0x7f3c800d4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:55.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.884+0000 7f3c8741e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c80161f30 0x7f3c800cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:55.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.884+0000 7f3c8741e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c800d5230 con 0x7f3c800d1480 2024-09-19T07:56:55.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.884+0000 7f3c8741e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f3c800d2dc0 con 0x7f3c800d10a0 2024-09-19T07:56:55.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.884+0000 7f3c8741e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3c800d30a0 con 0x7f3c80161f30 2024-09-19T07:56:55.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.884+0000 7f3c849b9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3c800d1480 0x7f3c80161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58238/0 (socket says 172.21.15.16:58238) 2024-09-19T07:56:55.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.884+0000 7f3c849b9640 1 -- 172.21.15.16:0/3735963944 learned_addr learned my addr 172.21.15.16:0/3735963944 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:56:55.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.884+0000 7f3c859bb640 1 --2- 172.21.15.16:0/3735963944 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c80161f30 0x7f3c800cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.884+0000 7f3c851ba640 1 --2- 172.21.15.16:0/3735963944 >> v2:172.21.15.146:3300/0 conn(0x7f3c800d10a0 0x7f3c800d4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.885+0000 7f3c77fff640 1 -- 172.21.15.16:0/3735963944 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 159461809 0 0) 0x7f3c800d5230 con 0x7f3c800d1480 2024-09-19T07:56:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.885+0000 7f3c77fff640 1 -- 172.21.15.16:0/3735963944 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c6c003520 con 0x7f3c800d1480 2024-09-19T07:56:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.885+0000 7f3c851ba640 1 -- 172.21.15.16:0/3735963944 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c80161f30 msgr2=0x7f3c800cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:56:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.885+0000 7f3c851ba640 1 --2- 172.21.15.16:0/3735963944 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c80161f30 0x7f3c800cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.885+0000 7f3c851ba640 1 -- 172.21.15.16:0/3735963944 >> v1:172.21.15.16:6789/0 conn(0x7f3c800d1480 legacy=0x7f3c80161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.885+0000 7f3c851ba640 1 -- 172.21.15.16:0/3735963944 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c800d3380 con 0x7f3c800d10a0 2024-09-19T07:56:55.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.886+0000 7f3c851ba640 1 --2- 172.21.15.16:0/3735963944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c800d10a0 0x7f3c800d4bc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0097a0 tx=0x7f3c7c02a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94448c09ef7d44d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:55.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.886+0000 7f3c77fff640 1 -- 172.21.15.16:0/3735963944 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c037070 con 0x7f3c800d10a0 2024-09-19T07:56:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.886+0000 7f3c77fff640 1 -- 172.21.15.16:0/3735963944 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c7c02adb0 con 0x7f3c800d10a0 2024-09-19T07:56:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.886+0000 7f3c77fff640 1 -- 172.21.15.16:0/3735963944 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c032640 con 0x7f3c800d10a0 2024-09-19T07:56:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.886+0000 7f3c8741e640 1 -- 172.21.15.16:0/3735963944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c800d10a0 msgr2=0x7f3c800d4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.886+0000 7f3c8741e640 1 --2- 172.21.15.16:0/3735963944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c800d10a0 0x7f3c800d4bc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0097a0 tx=0x7f3c7c02a060 comp rx=0 tx=0).stop 2024-09-19T07:56:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.886+0000 7f3c8741e640 1 -- 172.21.15.16:0/3735963944 shutdown_connections 2024-09-19T07:56:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.886+0000 7f3c8741e640 1 --2- 172.21.15.16:0/3735963944 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c80161f30 0x7f3c800cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.886+0000 7f3c8741e640 1 --2- 172.21.15.16:0/3735963944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c800d10a0 0x7f3c800d4bc0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.886+0000 7f3c8741e640 1 -- 172.21.15.16:0/3735963944 >> 172.21.15.16:0/3735963944 conn(0x7f3c8008bfe0 msgr2=0x7f3c800d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.887+0000 7f3c8741e640 1 -- 172.21.15.16:0/3735963944 shutdown_connections 2024-09-19T07:56:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.887+0000 7f3c8741e640 1 -- 172.21.15.16:0/3735963944 wait complete. 2024-09-19T07:56:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c8741e640 1 Processor -- start 2024-09-19T07:56:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c8741e640 1 -- start start 2024-09-19T07:56:55.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c8741e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c800d10a0 0x7f3c800b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:55.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c8741e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c800d1480 0x7f3c800b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:55.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c8741e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c80161f30 0x7f3c801f3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:55.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c8741e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c800b5350 con 0x7f3c80161f30 2024-09-19T07:56:55.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c8741e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3c800d3660 con 0x7f3c800d10a0 2024-09-19T07:56:55.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c8741e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3c800b54d0 con 0x7f3c800d1480 2024-09-19T07:56:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c859bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c80161f30 0x7f3c801f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c859bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c80161f30 0x7f3c801f3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54388/0 (socket says 172.21.15.16:54388) 2024-09-19T07:56:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c859bb640 1 -- 172.21.15.16:0/175052443 learned_addr learned my addr 172.21.15.16:0/175052443 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:56:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c849b9640 1 --2- 172.21.15.16:0/175052443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c800d1480 0x7f3c800b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c851ba640 1 --2- 172.21.15.16:0/175052443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c800d10a0 0x7f3c800b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c859bb640 1 -- 172.21.15.16:0/175052443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c800d1480 msgr2=0x7f3c800b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c859bb640 1 --2- 172.21.15.16:0/175052443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c800d1480 0x7f3c800b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c859bb640 1 -- 172.21.15.16:0/175052443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c800d10a0 msgr2=0x7f3c800b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c859bb640 1 --2- 172.21.15.16:0/175052443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c800d10a0 0x7f3c800b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c859bb640 1 -- 172.21.15.16:0/175052443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c800b11e0 con 0x7f3c80161f30 2024-09-19T07:56:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c859bb640 1 --2- 172.21.15.16:0/175052443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c80161f30 0x7f3c801f3950 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f3c78009d30 tx=0x7f3c7800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:55.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c75ffb640 1 -- 172.21.15.16:0/175052443 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78018020 con 0x7f3c80161f30 2024-09-19T07:56:55.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c75ffb640 1 -- 172.21.15.16:0/175052443 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c78014480 con 0x7f3c80161f30 2024-09-19T07:56:55.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c8741e640 1 -- 172.21.15.16:0/175052443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c800b14c0 con 0x7f3c80161f30 2024-09-19T07:56:55.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.889+0000 7f3c75ffb640 1 -- 172.21.15.16:0/175052443 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78014870 con 0x7f3c80161f30 2024-09-19T07:56:55.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.890+0000 7f3c8741e640 1 -- 172.21.15.16:0/175052443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c800a1f70 con 0x7f3c80161f30 2024-09-19T07:56:55.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.891+0000 7f3c8741e640 1 -- 172.21.15.16:0/175052443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c800d4bc0 con 0x7f3c80161f30 2024-09-19T07:56:55.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.891+0000 7f3c75ffb640 1 -- 172.21.15.16:0/175052443 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3c78005d10 con 0x7f3c80161f30 2024-09-19T07:56:55.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.892+0000 7f3c75ffb640 1 --2- 172.21.15.16:0/175052443 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3c6406c2b0 0x7f3c6406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:56:55.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.892+0000 7f3c75ffb640 1 -- 172.21.15.16:0/175052443 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f3c7808c120 con 0x7f3c80161f30 2024-09-19T07:56:55.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.892+0000 7f3c851ba640 1 --2- 172.21.15.16:0/175052443 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3c6406c2b0 0x7f3c6406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:56:55.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.894+0000 7f3c851ba640 1 --2- 172.21.15.16:0/175052443 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3c6406c2b0 0x7f3c6406e770 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c034000 tx=0x7f3c7c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:56:55.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:55.897+0000 7f3c75ffb640 1 -- 172.21.15.16:0/175052443 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3c78022070 con 0x7f3c80161f30 2024-09-19T07:56:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:56.111+0000 7f3c8741e640 1 -- 172.21.15.16:0/175052443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3c800ba430 con 0x7f3c80161f30 2024-09-19T07:56:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:56.111+0000 7f3c75ffb640 1 -- 172.21.15.16:0/175052443 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f3c78010040 con 0x7f3c80161f30 2024-09-19T07:56:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:56:56.113 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T07:56:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:56.113+0000 7f3c8741e640 1 -- 172.21.15.16:0/175052443 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3c6406c2b0 msgr2=0x7f3c6406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:56.113+0000 7f3c8741e640 1 --2- 172.21.15.16:0/175052443 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3c6406c2b0 0x7f3c6406e770 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c034000 tx=0x7f3c7c02e000 comp rx=0 tx=0).stop 2024-09-19T07:56:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:56.113+0000 7f3c8741e640 1 -- 172.21.15.16:0/175052443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c80161f30 msgr2=0x7f3c801f3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:56:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:56.113+0000 7f3c8741e640 1 --2- 172.21.15.16:0/175052443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c80161f30 0x7f3c801f3950 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f3c78009d30 tx=0x7f3c7800c490 comp rx=0 tx=0).stop 2024-09-19T07:56:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:56.113+0000 7f3c8741e640 1 -- 172.21.15.16:0/175052443 shutdown_connections 2024-09-19T07:56:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:56.113+0000 7f3c8741e640 1 --2- 172.21.15.16:0/175052443 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3c6406c2b0 0x7f3c6406e770 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:56.113+0000 7f3c8741e640 1 --2- 172.21.15.16:0/175052443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c80161f30 0x7f3c801f3950 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:56.113+0000 7f3c8741e640 1 --2- 172.21.15.16:0/175052443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c800d1480 0x7f3c800b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:56.113+0000 7f3c8741e640 1 --2- 172.21.15.16:0/175052443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c800d10a0 0x7f3c800b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:56:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:56.113+0000 7f3c8741e640 1 -- 172.21.15.16:0/175052443 >> 172.21.15.16:0/175052443 conn(0x7f3c8008bfe0 msgr2=0x7f3c800cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:56:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:56.113+0000 7f3c8741e640 1 -- 172.21.15.16:0/175052443 shutdown_connections 2024-09-19T07:56:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:56:56.113+0000 7f3c8741e640 1 -- 172.21.15.16:0/175052443 wait complete. 2024-09-19T07:56:56.126 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-19T07:56:56.126 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2024-09-19T07:57:01.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:57:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.242+0000 7f6c82077640 1 Processor -- start 2024-09-19T07:57:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.242+0000 7f6c82077640 1 -- start start 2024-09-19T07:57:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.243+0000 7f6c82077640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c7c161c30 0x7f6c7c0d4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.243+0000 7f6c82077640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c7c0d13e0 0x7f6c7c0d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.243+0000 7f6c82077640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c7c0d50f0 con 0x7f6c7c0b51c0 2024-09-19T07:57:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.243+0000 7f6c82077640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f6c7c0d2070 con 0x7f6c7c0d13e0 2024-09-19T07:57:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.243+0000 7f6c82077640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6c7c0d5390 con 0x7f6c7c161c30 2024-09-19T07:57:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.243+0000 7f6c7bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c7c0d13e0 0x7f6c7c0d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.243+0000 7f6c7affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6c7c0b51c0 0x7f6c7c0d0cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58246/0 (socket says 172.21.15.16:58246) 2024-09-19T07:57:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.243+0000 7f6c7affd640 1 -- 172.21.15.16:0/28381109 learned_addr learned my addr 172.21.15.16:0/28381109 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.244+0000 7f6c7b7fe640 1 --2- 172.21.15.16:0/28381109 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c7c161c30 0x7f6c7c0d4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.244+0000 7f6c7bfff640 1 -- 172.21.15.16:0/28381109 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c7c161c30 msgr2=0x7f6c7c0d4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.244+0000 7f6c7bfff640 1 --2- 172.21.15.16:0/28381109 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c7c161c30 0x7f6c7c0d4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.244+0000 7f6c7bfff640 1 -- 172.21.15.16:0/28381109 >> v1:172.21.15.16:6789/0 conn(0x7f6c7c0b51c0 legacy=0x7f6c7c0d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.244+0000 7f6c7bfff640 1 -- 172.21.15.16:0/28381109 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7c0cfda0 con 0x7f6c7c0d13e0 2024-09-19T07:57:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.244+0000 7f6c7a7fc640 1 -- 172.21.15.16:0/28381109 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1445773301 0 0) 0x7f6c7c0d50f0 con 0x7f6c7c0b51c0 2024-09-19T07:57:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.244+0000 7f6c7b7fe640 1 --2- 172.21.15.16:0/28381109 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c7c161c30 0x7f6c7c0d4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:57:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.244+0000 7f6c7bfff640 1 --2- 172.21.15.16:0/28381109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7c0d13e0 0x7f6c7c0d2500 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009d90 tx=0x7f6c7402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0461e72360e17b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.245+0000 7f6c7a7fc640 1 -- 172.21.15.16:0/28381109 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74007740 con 0x7f6c7c0d13e0 2024-09-19T07:57:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.245+0000 7f6c7a7fc640 1 -- 172.21.15.16:0/28381109 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c740078e0 con 0x7f6c7c0d13e0 2024-09-19T07:57:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.245+0000 7f6c7a7fc640 1 -- 172.21.15.16:0/28381109 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74007cb0 con 0x7f6c7c0d13e0 2024-09-19T07:57:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.245+0000 7f6c82077640 1 -- 172.21.15.16:0/28381109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7c0d13e0 msgr2=0x7f6c7c0d2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.245+0000 7f6c82077640 1 --2- 172.21.15.16:0/28381109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7c0d13e0 0x7f6c7c0d2500 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009d90 tx=0x7f6c7402a660 comp rx=0 tx=0).stop 2024-09-19T07:57:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.245+0000 7f6c82077640 1 -- 172.21.15.16:0/28381109 shutdown_connections 2024-09-19T07:57:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.245+0000 7f6c82077640 1 --2- 172.21.15.16:0/28381109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7c0d13e0 0x7f6c7c0d2500 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.245+0000 7f6c82077640 1 --2- 172.21.15.16:0/28381109 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c7c161c30 0x7f6c7c0d4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.245+0000 7f6c82077640 1 -- 172.21.15.16:0/28381109 >> 172.21.15.16:0/28381109 conn(0x7f6c7c08bfe0 msgr2=0x7f6c7c0d44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.245+0000 7f6c82077640 1 -- 172.21.15.16:0/28381109 shutdown_connections 2024-09-19T07:57:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.245+0000 7f6c82077640 1 -- 172.21.15.16:0/28381109 wait complete. 2024-09-19T07:57:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.246+0000 7f6c82077640 1 Processor -- start 2024-09-19T07:57:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.247+0000 7f6c82077640 1 -- start start 2024-09-19T07:57:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.247+0000 7f6c82077640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c7c0b51c0 0x7f6c7c150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.247+0000 7f6c82077640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c7c0d13e0 0x7f6c7c0cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.247+0000 7f6c82077640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7c161c30 0x7f6c7c09faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.247+0000 7f6c82077640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c0b72f0 con 0x7f6c7c161c30 2024-09-19T07:57:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.247+0000 7f6c82077640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c0d0360 con 0x7f6c7c0b51c0 2024-09-19T07:57:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.247+0000 7f6c82077640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6c7c162010 con 0x7f6c7c0d13e0 2024-09-19T07:57:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.248+0000 7f6c7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7c161c30 0x7f6c7c09faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.248+0000 7f6c7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7c161c30 0x7f6c7c09faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54406/0 (socket says 172.21.15.16:54406) 2024-09-19T07:57:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.248+0000 7f6c7bfff640 1 -- 172.21.15.16:0/2549767837 learned_addr learned my addr 172.21.15.16:0/2549767837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.248+0000 7f6c7affd640 1 --2- 172.21.15.16:0/2549767837 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c7c0d13e0 0x7f6c7c0cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.248+0000 7f6c7b7fe640 1 --2- 172.21.15.16:0/2549767837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c7c0b51c0 0x7f6c7c150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.248+0000 7f6c7bfff640 1 -- 172.21.15.16:0/2549767837 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c7c0d13e0 msgr2=0x7f6c7c0cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.248+0000 7f6c7bfff640 1 --2- 172.21.15.16:0/2549767837 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c7c0d13e0 0x7f6c7c0cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.248+0000 7f6c7bfff640 1 -- 172.21.15.16:0/2549767837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c7c0b51c0 msgr2=0x7f6c7c150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.248+0000 7f6c7bfff640 1 --2- 172.21.15.16:0/2549767837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c7c0b51c0 0x7f6c7c150870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.248+0000 7f6c7bfff640 1 -- 172.21.15.16:0/2549767837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7c0ccfa0 con 0x7f6c7c161c30 2024-09-19T07:57:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.248+0000 7f6c7b7fe640 1 --2- 172.21.15.16:0/2549767837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c7c0b51c0 0x7f6c7c150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.248+0000 7f6c7bfff640 1 --2- 172.21.15.16:0/2549767837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7c161c30 0x7f6c7c09faf0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f6c7402ac50 tx=0x7f6c74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.248+0000 7f6c63fff640 1 -- 172.21.15.16:0/2549767837 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74040070 con 0x7f6c7c161c30 2024-09-19T07:57:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.248+0000 7f6c63fff640 1 -- 172.21.15.16:0/2549767837 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c74031c90 con 0x7f6c7c161c30 2024-09-19T07:57:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.248+0000 7f6c82077640 1 -- 172.21.15.16:0/2549767837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7c0cd280 con 0x7f6c7c161c30 2024-09-19T07:57:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.248+0000 7f6c63fff640 1 -- 172.21.15.16:0/2549767837 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74045680 con 0x7f6c7c161c30 2024-09-19T07:57:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.248+0000 7f6c82077640 1 -- 172.21.15.16:0/2549767837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c7c0cd840 con 0x7f6c7c161c30 2024-09-19T07:57:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.249+0000 7f6c82077640 1 -- 172.21.15.16:0/2549767837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c7c0d4bb0 con 0x7f6c7c161c30 2024-09-19T07:57:01.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.250+0000 7f6c63fff640 1 -- 172.21.15.16:0/2549767837 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6c74039040 con 0x7f6c7c161c30 2024-09-19T07:57:01.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.250+0000 7f6c63fff640 1 --2- 172.21.15.16:0/2549767837 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6c6806c2e0 0x7f6c6806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:01.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.250+0000 7f6c63fff640 1 -- 172.21.15.16:0/2549767837 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f6c740acfd0 con 0x7f6c7c161c30 2024-09-19T07:57:01.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.251+0000 7f6c7b7fe640 1 --2- 172.21.15.16:0/2549767837 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6c6806c2e0 0x7f6c6806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:01.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.251+0000 7f6c7b7fe640 1 --2- 172.21.15.16:0/2549767837 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6c6806c2e0 0x7f6c6806e7a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c1f7920 tx=0x7f6c6c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:01.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.255+0000 7f6c63fff640 1 -- 172.21.15.16:0/2549767837 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6c7407b5d0 con 0x7f6c7c161c30 2024-09-19T07:57:01.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.467+0000 7f6c82077640 1 -- 172.21.15.16:0/2549767837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6c7c0d1c30 con 0x7f6c7c161c30 2024-09-19T07:57:01.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.468+0000 7f6c63fff640 1 -- 172.21.15.16:0/2549767837 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f6c7407aed0 con 0x7f6c7c161c30 2024-09-19T07:57:01.468 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:01.468 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T07:57:01.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.470+0000 7f6c82077640 1 -- 172.21.15.16:0/2549767837 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6c6806c2e0 msgr2=0x7f6c6806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:01.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.470+0000 7f6c82077640 1 --2- 172.21.15.16:0/2549767837 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6c6806c2e0 0x7f6c6806e7a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c1f7920 tx=0x7f6c6c008040 comp rx=0 tx=0).stop 2024-09-19T07:57:01.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.470+0000 7f6c82077640 1 -- 172.21.15.16:0/2549767837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7c161c30 msgr2=0x7f6c7c09faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:01.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.470+0000 7f6c82077640 1 --2- 172.21.15.16:0/2549767837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7c161c30 0x7f6c7c09faf0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f6c7402ac50 tx=0x7f6c74004a40 comp rx=0 tx=0).stop 2024-09-19T07:57:01.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.470+0000 7f6c82077640 1 -- 172.21.15.16:0/2549767837 shutdown_connections 2024-09-19T07:57:01.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.470+0000 7f6c82077640 1 --2- 172.21.15.16:0/2549767837 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f6c6806c2e0 0x7f6c6806e7a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:01.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.470+0000 7f6c82077640 1 --2- 172.21.15.16:0/2549767837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7c161c30 0x7f6c7c09faf0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:01.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.470+0000 7f6c82077640 1 --2- 172.21.15.16:0/2549767837 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c7c0d13e0 0x7f6c7c0cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:01.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.470+0000 7f6c82077640 1 --2- 172.21.15.16:0/2549767837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c7c0b51c0 0x7f6c7c150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:01.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.470+0000 7f6c82077640 1 -- 172.21.15.16:0/2549767837 >> 172.21.15.16:0/2549767837 conn(0x7f6c7c08bfe0 msgr2=0x7f6c7c162280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:01.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.470+0000 7f6c82077640 1 -- 172.21.15.16:0/2549767837 shutdown_connections 2024-09-19T07:57:01.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:01.470+0000 7f6c82077640 1 -- 172.21.15.16:0/2549767837 wait complete. 2024-09-19T07:57:01.481 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-19T07:57:01.481 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2024-09-19T07:57:06.482 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:57:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.597+0000 7f0978f13640 1 Processor -- start 2024-09-19T07:57:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.597+0000 7f0978f13640 1 -- start start 2024-09-19T07:57:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.598+0000 7f0978f13640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0974161c30 0x7f097409b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.598+0000 7f0978f13640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f097406dff0 con 0x7f09740ce2f0 2024-09-19T07:57:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.598+0000 7f0978f13640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09740d1300 con 0x7f097409aa50 2024-09-19T07:57:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.598+0000 7f0978f13640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f09740d18c0 con 0x7f0974161c30 2024-09-19T07:57:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.598+0000 7f0972d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f09740ce2f0 0x7f097406d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49752/0 (socket says 172.21.15.16:49752) 2024-09-19T07:57:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.598+0000 7f0972d9d640 1 -- 172.21.15.16:0/1973465574 learned_addr learned my addr 172.21.15.16:0/1973465574 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.598+0000 7f097259c640 1 --2- 172.21.15.16:0/1973465574 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0974161c30 0x7f097409b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.598+0000 7f097159a640 1 -- 172.21.15.16:0/1973465574 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2425694371 0 0) 0x7f097406dff0 con 0x7f09740ce2f0 2024-09-19T07:57:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.598+0000 7f097159a640 1 -- 172.21.15.16:0/1973465574 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f095c003520 con 0x7f09740ce2f0 2024-09-19T07:57:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.598+0000 7f097159a640 1 -- 172.21.15.16:0/1973465574 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 543256355 0 0) 0x7f09740d1300 con 0x7f097409aa50 2024-09-19T07:57:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.598+0000 7f097159a640 1 -- 172.21.15.16:0/1973465574 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f097406dff0 con 0x7f097409aa50 2024-09-19T07:57:06.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.598+0000 7f097259c640 1 -- 172.21.15.16:0/1973465574 >> v1:172.21.15.146:6789/0 conn(0x7f097409aa50 legacy=0x7f097406d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:06.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.598+0000 7f097259c640 1 -- 172.21.15.16:0/1973465574 >> v1:172.21.15.16:6789/0 conn(0x7f09740ce2f0 legacy=0x7f097406d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:06.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.598+0000 7f097259c640 1 -- 172.21.15.16:0/1973465574 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09740d15e0 con 0x7f0974161c30 2024-09-19T07:57:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.599+0000 7f097259c640 1 --2- 172.21.15.16:0/1973465574 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0974161c30 0x7f097409b6d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f09640097d0 tx=0x7f096402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ac344f688be6ca2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.599+0000 7f097159a640 1 -- 172.21.15.16:0/1973465574 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964037070 con 0x7f0974161c30 2024-09-19T07:57:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.599+0000 7f097159a640 1 -- 172.21.15.16:0/1973465574 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0964032440 con 0x7f0974161c30 2024-09-19T07:57:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.600+0000 7f097159a640 1 -- 172.21.15.16:0/1973465574 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09640327f0 con 0x7f0974161c30 2024-09-19T07:57:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.600+0000 7f0978f13640 1 -- 172.21.15.16:0/1973465574 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0974161c30 msgr2=0x7f097409b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.600+0000 7f0978f13640 1 --2- 172.21.15.16:0/1973465574 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0974161c30 0x7f097409b6d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f09640097d0 tx=0x7f096402a180 comp rx=0 tx=0).stop 2024-09-19T07:57:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.600+0000 7f0978f13640 1 -- 172.21.15.16:0/1973465574 shutdown_connections 2024-09-19T07:57:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.600+0000 7f0978f13640 1 --2- 172.21.15.16:0/1973465574 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0974161c30 0x7f097409b6d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.600+0000 7f0978f13640 1 -- 172.21.15.16:0/1973465574 >> 172.21.15.16:0/1973465574 conn(0x7f09740d22f0 msgr2=0x7f09740d2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.600+0000 7f0978f13640 1 -- 172.21.15.16:0/1973465574 shutdown_connections 2024-09-19T07:57:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.600+0000 7f0978f13640 1 -- 172.21.15.16:0/1973465574 wait complete. 2024-09-19T07:57:06.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.602+0000 7f0978f13640 1 Processor -- start 2024-09-19T07:57:06.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.602+0000 7f0978f13640 1 -- start start 2024-09-19T07:57:06.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.602+0000 7f0978f13640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f097409aa50 0x7f09740cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:06.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.602+0000 7f0978f13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09740ce2f0 0x7f09740b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:06.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.602+0000 7f0978f13640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0974161c30 0x7f09740c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:06.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.602+0000 7f0978f13640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09740d0f90 con 0x7f09740ce2f0 2024-09-19T07:57:06.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.602+0000 7f0978f13640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f09740cce50 con 0x7f097409aa50 2024-09-19T07:57:06.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.602+0000 7f0978f13640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f09740d2f60 con 0x7f0974161c30 2024-09-19T07:57:06.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.602+0000 7f0971d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09740ce2f0 0x7f09740b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:06.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.603+0000 7f0971d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09740ce2f0 0x7f09740b7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36568/0 (socket says 172.21.15.16:36568) 2024-09-19T07:57:06.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.603+0000 7f097259c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f097409aa50 0x7f09740cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:06.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.603+0000 7f0971d9b640 1 -- 172.21.15.16:0/273935583 learned_addr learned my addr 172.21.15.16:0/273935583 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.603+0000 7f0972d9d640 1 --2- 172.21.15.16:0/273935583 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0974161c30 0x7f09740c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.603+0000 7f0971d9b640 1 -- 172.21.15.16:0/273935583 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0974161c30 msgr2=0x7f09740c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.603+0000 7f0971d9b640 1 --2- 172.21.15.16:0/273935583 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0974161c30 0x7f09740c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.603+0000 7f0971d9b640 1 -- 172.21.15.16:0/273935583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f097409aa50 msgr2=0x7f09740cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.603+0000 7f0971d9b640 1 --2- 172.21.15.16:0/273935583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f097409aa50 0x7f09740cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.603+0000 7f0971d9b640 1 -- 172.21.15.16:0/273935583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09740cb8f0 con 0x7f09740ce2f0 2024-09-19T07:57:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.603+0000 7f0971d9b640 1 --2- 172.21.15.16:0/273935583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09740ce2f0 0x7f09740b7ac0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f096800c430 tx=0x7f096800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.603+0000 7f0962ffd640 1 -- 172.21.15.16:0/273935583 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968017070 con 0x7f09740ce2f0 2024-09-19T07:57:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.603+0000 7f0962ffd640 1 -- 172.21.15.16:0/273935583 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09680052f0 con 0x7f09740ce2f0 2024-09-19T07:57:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.603+0000 7f0978f13640 1 -- 172.21.15.16:0/273935583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09740cbbb0 con 0x7f09740ce2f0 2024-09-19T07:57:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.603+0000 7f0962ffd640 1 -- 172.21.15.16:0/273935583 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968003830 con 0x7f09740ce2f0 2024-09-19T07:57:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.603+0000 7f0978f13640 1 -- 172.21.15.16:0/273935583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09740cc170 con 0x7f09740ce2f0 2024-09-19T07:57:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.604+0000 7f0978f13640 1 -- 172.21.15.16:0/273935583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09740d1300 con 0x7f09740ce2f0 2024-09-19T07:57:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.605+0000 7f0962ffd640 1 -- 172.21.15.16:0/273935583 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f096801a070 con 0x7f09740ce2f0 2024-09-19T07:57:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.605+0000 7f0962ffd640 1 --2- 172.21.15.16:0/273935583 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f095406c2b0 0x7f095406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.605+0000 7f0962ffd640 1 -- 172.21.15.16:0/273935583 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f096808a890 con 0x7f09740ce2f0 2024-09-19T07:57:06.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.606+0000 7f097259c640 1 --2- 172.21.15.16:0/273935583 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f095406c2b0 0x7f095406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:06.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.606+0000 7f097259c640 1 --2- 172.21.15.16:0/273935583 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f095406c2b0 0x7f095406e770 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0964034000 tx=0x7f096402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:06.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.610+0000 7f0962ffd640 1 -- 172.21.15.16:0/273935583 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f09680588b0 con 0x7f09740ce2f0 2024-09-19T07:57:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.823+0000 7f0978f13640 1 -- 172.21.15.16:0/273935583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f09740d1bc0 con 0x7f09740ce2f0 2024-09-19T07:57:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.823+0000 7f0962ffd640 1 -- 172.21.15.16:0/273935583 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f096805f0d0 con 0x7f09740ce2f0 2024-09-19T07:57:06.823 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:06.824 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T07:57:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.825+0000 7f0978f13640 1 -- 172.21.15.16:0/273935583 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f095406c2b0 msgr2=0x7f095406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.825+0000 7f0978f13640 1 --2- 172.21.15.16:0/273935583 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f095406c2b0 0x7f095406e770 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0964034000 tx=0x7f096402e000 comp rx=0 tx=0).stop 2024-09-19T07:57:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.825+0000 7f0978f13640 1 -- 172.21.15.16:0/273935583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09740ce2f0 msgr2=0x7f09740b7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.825+0000 7f0978f13640 1 --2- 172.21.15.16:0/273935583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09740ce2f0 0x7f09740b7ac0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f096800c430 tx=0x7f096800c8f0 comp rx=0 tx=0).stop 2024-09-19T07:57:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.825+0000 7f0978f13640 1 -- 172.21.15.16:0/273935583 shutdown_connections 2024-09-19T07:57:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.825+0000 7f0978f13640 1 --2- 172.21.15.16:0/273935583 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f095406c2b0 0x7f095406e770 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.825+0000 7f0978f13640 1 --2- 172.21.15.16:0/273935583 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0974161c30 0x7f09740c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.825+0000 7f0978f13640 1 --2- 172.21.15.16:0/273935583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09740ce2f0 0x7f09740b7ac0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.825+0000 7f0978f13640 1 --2- 172.21.15.16:0/273935583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f097409aa50 0x7f09740cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.825+0000 7f0978f13640 1 -- 172.21.15.16:0/273935583 >> 172.21.15.16:0/273935583 conn(0x7f09740d22f0 msgr2=0x7f0974070110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.825+0000 7f0978f13640 1 -- 172.21.15.16:0/273935583 shutdown_connections 2024-09-19T07:57:06.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:06.825+0000 7f0978f13640 1 -- 172.21.15.16:0/273935583 wait complete. 2024-09-19T07:57:06.836 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-19T07:57:06.836 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2024-09-19T07:57:09.952 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:57:09.955 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:57:11.837 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T07:57:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.951+0000 7fbd7ace4640 1 Processor -- start 2024-09-19T07:57:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.951+0000 7fbd7ace4640 1 -- start start 2024-09-19T07:57:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.952+0000 7fbd7ace4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbd74161c00 0x7fbd7406c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.952+0000 7fbd7ace4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd74161fe0 con 0x7fbd740d0b20 2024-09-19T07:57:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.952+0000 7fbd7ace4640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd740b25e0 con 0x7fbd740cd280 2024-09-19T07:57:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.952+0000 7fbd7ace4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fbd740cd6c0 con 0x7fbd74161c00 2024-09-19T07:57:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.952+0000 7fbd78a80640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbd74161c00 0x7fbd7406c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.952+0000 7fbd79281640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbd740d0b20 0x7fbd740cc170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49766/0 (socket says 172.21.15.16:49766) 2024-09-19T07:57:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.952+0000 7fbd79281640 1 -- 172.21.15.16:0/85332080 learned_addr learned my addr 172.21.15.16:0/85332080 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.953+0000 7fbd78a80640 1 -- 172.21.15.16:0/85332080 >> v1:172.21.15.146:6789/0 conn(0x7fbd740cd280 legacy=0x7fbd740d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.953+0000 7fbd78a80640 1 -- 172.21.15.16:0/85332080 >> v1:172.21.15.16:6789/0 conn(0x7fbd740d0b20 legacy=0x7fbd740cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:11.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.953+0000 7fbd78a80640 1 -- 172.21.15.16:0/85332080 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd740b28c0 con 0x7fbd74161c00 2024-09-19T07:57:11.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.953+0000 7fbd78a80640 1 --2- 172.21.15.16:0/85332080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74161c00 0x7fbd7406c620 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fbd640097a0 tx=0x7fbd6402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942b98ada3bd4e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:11.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.954+0000 7fbd737fe640 1 -- 172.21.15.16:0/85332080 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64037070 con 0x7fbd74161c00 2024-09-19T07:57:11.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.954+0000 7fbd737fe640 1 -- 172.21.15.16:0/85332080 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd6402ae50 con 0x7fbd74161c00 2024-09-19T07:57:11.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.954+0000 7fbd737fe640 1 -- 172.21.15.16:0/85332080 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64032660 con 0x7fbd74161c00 2024-09-19T07:57:11.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.954+0000 7fbd7ace4640 1 -- 172.21.15.16:0/85332080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74161c00 msgr2=0x7fbd7406c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:11.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.954+0000 7fbd7ace4640 1 --2- 172.21.15.16:0/85332080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74161c00 0x7fbd7406c620 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fbd640097a0 tx=0x7fbd6402a140 comp rx=0 tx=0).stop 2024-09-19T07:57:11.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.954+0000 7fbd7ace4640 1 -- 172.21.15.16:0/85332080 shutdown_connections 2024-09-19T07:57:11.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.954+0000 7fbd7ace4640 1 --2- 172.21.15.16:0/85332080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74161c00 0x7fbd7406c620 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:11.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.954+0000 7fbd7ace4640 1 -- 172.21.15.16:0/85332080 >> 172.21.15.16:0/85332080 conn(0x7fbd7409aa50 msgr2=0x7fbd740b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:11.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.954+0000 7fbd7ace4640 1 -- 172.21.15.16:0/85332080 shutdown_connections 2024-09-19T07:57:11.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.954+0000 7fbd7ace4640 1 -- 172.21.15.16:0/85332080 wait complete. 2024-09-19T07:57:11.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.955+0000 7fbd7ace4640 1 Processor -- start 2024-09-19T07:57:11.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.956+0000 7fbd7ace4640 1 -- start start 2024-09-19T07:57:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.956+0000 7fbd7ace4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd740cd280 0x7fbd7415ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.956+0000 7fbd7ace4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd740d0b20 0x7fbd741539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.956+0000 7fbd7ace4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74161c00 0x7fbd740ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.956+0000 7fbd7ace4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd740b1e70 con 0x7fbd74161c00 2024-09-19T07:57:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.956+0000 7fbd7ace4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbd740cbce0 con 0x7fbd740d0b20 2024-09-19T07:57:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.956+0000 7fbd7ace4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbd740b1ff0 con 0x7fbd740cd280 2024-09-19T07:57:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.957+0000 7fbd79281640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74161c00 0x7fbd740ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.957+0000 7fbd79281640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74161c00 0x7fbd740ae500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36582/0 (socket says 172.21.15.16:36582) 2024-09-19T07:57:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.957+0000 7fbd79281640 1 -- 172.21.15.16:0/2782621695 learned_addr learned my addr 172.21.15.16:0/2782621695 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.957+0000 7fbd78a80640 1 --2- 172.21.15.16:0/2782621695 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd740cd280 0x7fbd7415ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.957+0000 7fbd73fff640 1 --2- 172.21.15.16:0/2782621695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd740d0b20 0x7fbd741539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.957+0000 7fbd79281640 1 -- 172.21.15.16:0/2782621695 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd740cd280 msgr2=0x7fbd7415ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.957+0000 7fbd79281640 1 --2- 172.21.15.16:0/2782621695 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd740cd280 0x7fbd7415ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.957+0000 7fbd79281640 1 -- 172.21.15.16:0/2782621695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd740d0b20 msgr2=0x7fbd741539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.957+0000 7fbd79281640 1 --2- 172.21.15.16:0/2782621695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd740d0b20 0x7fbd741539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.957+0000 7fbd79281640 1 -- 172.21.15.16:0/2782621695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd740c6d20 con 0x7fbd74161c00 2024-09-19T07:57:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.957+0000 7fbd79281640 1 --2- 172.21.15.16:0/2782621695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74161c00 0x7fbd740ae500 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00cae0 tx=0x7fbd6c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.957+0000 7fbd717fa640 1 -- 172.21.15.16:0/2782621695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c017070 con 0x7fbd74161c00 2024-09-19T07:57:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.957+0000 7fbd717fa640 1 -- 172.21.15.16:0/2782621695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd6c00ac40 con 0x7fbd74161c00 2024-09-19T07:57:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.957+0000 7fbd7ace4640 1 -- 172.21.15.16:0/2782621695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7415bf70 con 0x7fbd74161c00 2024-09-19T07:57:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.957+0000 7fbd717fa640 1 -- 172.21.15.16:0/2782621695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c012400 con 0x7fbd74161c00 2024-09-19T07:57:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.957+0000 7fbd7ace4640 1 -- 172.21.15.16:0/2782621695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd7415c530 con 0x7fbd74161c00 2024-09-19T07:57:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.958+0000 7fbd7ace4640 1 -- 172.21.15.16:0/2782621695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd7415c730 con 0x7fbd74161c00 2024-09-19T07:57:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.959+0000 7fbd717fa640 1 -- 172.21.15.16:0/2782621695 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbd6c01a070 con 0x7fbd74161c00 2024-09-19T07:57:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.959+0000 7fbd717fa640 1 --2- 172.21.15.16:0/2782621695 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbd6006c2b0 0x7fbd6006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.959+0000 7fbd717fa640 1 -- 172.21.15.16:0/2782621695 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7fbd6c08b400 con 0x7fbd74161c00 2024-09-19T07:57:11.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.960+0000 7fbd78a80640 1 --2- 172.21.15.16:0/2782621695 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbd6006c2b0 0x7fbd6006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:11.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.960+0000 7fbd78a80640 1 --2- 172.21.15.16:0/2782621695 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbd6006c2b0 0x7fbd6006e770 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fbd64034000 tx=0x7fbd6402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:11.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:11.964+0000 7fbd717fa640 1 -- 172.21.15.16:0/2782621695 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fbd7415c730 con 0x7fbd74161c00 2024-09-19T07:57:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.178+0000 7fbd7ace4640 1 -- 172.21.15.16:0/2782621695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbd740d1dc0 con 0x7fbd74161c00 2024-09-19T07:57:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.178+0000 7fbd717fa640 1 -- 172.21.15.16:0/2782621695 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+199 (secure 0 0 0) 0x7fbd6c059940 con 0x7fbd74161c00 2024-09-19T07:57:12.179 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:12.179 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CLIENT_LATE_RELEASE":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to capability release","count":1},"muted":false}},"mutes":[]} 2024-09-19T07:57:12.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.180+0000 7fbd7ace4640 1 -- 172.21.15.16:0/2782621695 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbd6006c2b0 msgr2=0x7fbd6006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:12.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.180+0000 7fbd7ace4640 1 --2- 172.21.15.16:0/2782621695 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbd6006c2b0 0x7fbd6006e770 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fbd64034000 tx=0x7fbd6402e000 comp rx=0 tx=0).stop 2024-09-19T07:57:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.180+0000 7fbd7ace4640 1 -- 172.21.15.16:0/2782621695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74161c00 msgr2=0x7fbd740ae500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.180+0000 7fbd7ace4640 1 --2- 172.21.15.16:0/2782621695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74161c00 0x7fbd740ae500 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00cae0 tx=0x7fbd6c00a060 comp rx=0 tx=0).stop 2024-09-19T07:57:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.180+0000 7fbd7ace4640 1 -- 172.21.15.16:0/2782621695 shutdown_connections 2024-09-19T07:57:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.180+0000 7fbd7ace4640 1 --2- 172.21.15.16:0/2782621695 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fbd6006c2b0 0x7fbd6006e770 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.180+0000 7fbd7ace4640 1 --2- 172.21.15.16:0/2782621695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74161c00 0x7fbd740ae500 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.180+0000 7fbd7ace4640 1 --2- 172.21.15.16:0/2782621695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd740d0b20 0x7fbd741539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.180+0000 7fbd7ace4640 1 --2- 172.21.15.16:0/2782621695 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd740cd280 0x7fbd7415ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.180+0000 7fbd7ace4640 1 -- 172.21.15.16:0/2782621695 >> 172.21.15.16:0/2782621695 conn(0x7fbd7409aa50 msgr2=0x7fbd740ce420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.180+0000 7fbd7ace4640 1 -- 172.21.15.16:0/2782621695 shutdown_connections 2024-09-19T07:57:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.180+0000 7fbd7ace4640 1 -- 172.21.15.16:0/2782621695 wait complete. 2024-09-19T07:57:12.191 DEBUG:tasks.ceph_test_case:wait_until_true: success in 65s and 0 retries 2024-09-19T07:57:12.292 INFO:tasks.cephfs.mount:Killing connection on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-19T07:57:12.293 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:57:12.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:57:12.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 down 2024-09-19T07:57:12.320 INFO:tasks.cephfs.mount:Cleaning up killed connection on ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:57:12.320 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:57:12.320 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:57:12.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:57:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.521+0000 7f6089366640 1 Processor -- start 2024-09-19T07:57:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.521+0000 7f6089366640 1 -- start start 2024-09-19T07:57:12.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.521+0000 7f6089366640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f608406bc10 0x7f60840b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:12.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.522+0000 7f6089366640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f60840cddf0 0x7f60840cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:12.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.522+0000 7f6089366640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60840ced90 con 0x7f60840ce970 2024-09-19T07:57:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.522+0000 7f6089366640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f608415ca30 con 0x7f60840cddf0 2024-09-19T07:57:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.522+0000 7f6089366640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f60840b1740 con 0x7f608406bc10 2024-09-19T07:57:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.522+0000 7f60837fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f60840cddf0 0x7f60840cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.522+0000 7f60837fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f60840cddf0 0x7f60840cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44780/0 (socket says 172.21.15.16:44780) 2024-09-19T07:57:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.522+0000 7f60837fe640 1 -- 172.21.15.16:0/1760334003 learned_addr learned my addr 172.21.15.16:0/1760334003 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.522+0000 7f6082ffd640 1 --2- 172.21.15.16:0/1760334003 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f608406bc10 0x7f60840b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.522+0000 7f60837fe640 1 -- 172.21.15.16:0/1760334003 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f608406bc10 msgr2=0x7f60840b6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.522+0000 7f60837fe640 1 --2- 172.21.15.16:0/1760334003 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f608406bc10 0x7f60840b6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.522+0000 7f60837fe640 1 -- 172.21.15.16:0/1760334003 >> v1:172.21.15.146:6789/0 conn(0x7f60840ce970 legacy=0x7f608406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.522+0000 7f60837fe640 1 -- 172.21.15.16:0/1760334003 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60840b1a20 con 0x7f60840cddf0 2024-09-19T07:57:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.523+0000 7f60837fe640 1 --2- 172.21.15.16:0/1760334003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60840cddf0 0x7f60840cf570 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f607c009e50 tx=0x7f607c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4570cd38cba089a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.523+0000 7f6081ffb640 1 -- 172.21.15.16:0/1760334003 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c036070 con 0x7f60840cddf0 2024-09-19T07:57:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.523+0000 7f6081ffb640 1 -- 172.21.15.16:0/1760334003 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f607c031600 con 0x7f60840cddf0 2024-09-19T07:57:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.523+0000 7f6081ffb640 1 -- 172.21.15.16:0/1760334003 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c0319d0 con 0x7f60840cddf0 2024-09-19T07:57:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.523+0000 7f6089366640 1 -- 172.21.15.16:0/1760334003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60840cddf0 msgr2=0x7f60840cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.523+0000 7f6089366640 1 --2- 172.21.15.16:0/1760334003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60840cddf0 0x7f60840cf570 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f607c009e50 tx=0x7f607c02a680 comp rx=0 tx=0).stop 2024-09-19T07:57:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.524+0000 7f6089366640 1 -- 172.21.15.16:0/1760334003 shutdown_connections 2024-09-19T07:57:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.524+0000 7f6089366640 1 --2- 172.21.15.16:0/1760334003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60840cddf0 0x7f60840cf570 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.524+0000 7f6089366640 1 --2- 172.21.15.16:0/1760334003 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f608406bc10 0x7f60840b6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.524+0000 7f6089366640 1 -- 172.21.15.16:0/1760334003 >> 172.21.15.16:0/1760334003 conn(0x7f608409aa50 msgr2=0x7f60840b6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.524+0000 7f6089366640 1 -- 172.21.15.16:0/1760334003 shutdown_connections 2024-09-19T07:57:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.524+0000 7f6089366640 1 -- 172.21.15.16:0/1760334003 wait complete. 2024-09-19T07:57:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.525+0000 7f6089366640 1 Processor -- start 2024-09-19T07:57:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.525+0000 7f6089366640 1 -- start start 2024-09-19T07:57:12.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f6089366640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f608406bc10 0x7f6084157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:12.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f6089366640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f60840cddf0 0x7f60841598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f6089366640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60840ce970 0x7f6084159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f6089366640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f60840b1d00 con 0x7f608406bc10 2024-09-19T07:57:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f6089366640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f60840cd8d0 con 0x7f60840ce970 2024-09-19T07:57:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f6089366640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f60840ce280 con 0x7f60840cddf0 2024-09-19T07:57:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f6082ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f608406bc10 0x7f6084157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f6082ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f608406bc10 0x7f6084157ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44788/0 (socket says 172.21.15.16:44788) 2024-09-19T07:57:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f6082ffd640 1 -- 172.21.15.16:0/2091768599 learned_addr learned my addr 172.21.15.16:0/2091768599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f6082ffd640 1 -- 172.21.15.16:0/2091768599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f60840cddf0 msgr2=0x7f60841598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f60837fe640 1 --2- 172.21.15.16:0/2091768599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60840ce970 0x7f6084159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f60827fc640 1 --2- 172.21.15.16:0/2091768599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f60840cddf0 0x7f60841598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f6082ffd640 1 --2- 172.21.15.16:0/2091768599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f60840cddf0 0x7f60841598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f6082ffd640 1 -- 172.21.15.16:0/2091768599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60840ce970 msgr2=0x7f6084159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f6082ffd640 1 --2- 172.21.15.16:0/2091768599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60840ce970 0x7f6084159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f6082ffd640 1 -- 172.21.15.16:0/2091768599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608415a4c0 con 0x7f608406bc10 2024-09-19T07:57:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f60827fc640 1 --2- 172.21.15.16:0/2091768599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f60840cddf0 0x7f60841598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:57:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f6082ffd640 1 --2- 172.21.15.16:0/2091768599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f608406bc10 0x7f6084157ab0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f607400e9b0 tx=0x7f607400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f607b7fe640 1 -- 172.21.15.16:0/2091768599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074017070 con 0x7f608406bc10 2024-09-19T07:57:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f607b7fe640 1 -- 172.21.15.16:0/2091768599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f607400c9a0 con 0x7f608406bc10 2024-09-19T07:57:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.526+0000 7f607b7fe640 1 -- 172.21.15.16:0/2091768599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607400cd90 con 0x7f608406bc10 2024-09-19T07:57:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.527+0000 7f6089366640 1 -- 172.21.15.16:0/2091768599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f608415a7a0 con 0x7f608406bc10 2024-09-19T07:57:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.527+0000 7f6089366640 1 -- 172.21.15.16:0/2091768599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f608415d1d0 con 0x7f608406bc10 2024-09-19T07:57:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.527+0000 7f6089366640 1 -- 172.21.15.16:0/2091768599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f608415a970 con 0x7f608406bc10 2024-09-19T07:57:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.528+0000 7f607b7fe640 1 -- 172.21.15.16:0/2091768599 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f607401a070 con 0x7f608406bc10 2024-09-19T07:57:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.528+0000 7f607b7fe640 1 --2- 172.21.15.16:0/2091768599 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f606406c2e0 0x7f606406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.529+0000 7f607b7fe640 1 -- 172.21.15.16:0/2091768599 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f607408b920 con 0x7f608406bc10 2024-09-19T07:57:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.529+0000 7f60827fc640 1 --2- 172.21.15.16:0/2091768599 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f606406c2e0 0x7f606406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.529+0000 7f60827fc640 1 --2- 172.21.15.16:0/2091768599 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f606406c2e0 0x7f606406e7a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6084154070 tx=0x7f606c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.533+0000 7f607b7fe640 1 -- 172.21.15.16:0/2091768599 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6074059f10 con 0x7f608406bc10 2024-09-19T07:57:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.720+0000 7f6089366640 1 -- 172.21.15.16:0/2091768599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f60840b6d10 con 0x7f608406bc10 2024-09-19T07:57:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.721+0000 7f607b7fe640 1 -- 172.21.15.16:0/2091768599 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v152) ==== 80+0+0 (secure 0 0 0) 0x7f6074059810 con 0x7f608406bc10 2024-09-19T07:57:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:57:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.723+0000 7f6089366640 1 -- 172.21.15.16:0/2091768599 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f606406c2e0 msgr2=0x7f606406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.723+0000 7f6089366640 1 --2- 172.21.15.16:0/2091768599 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f606406c2e0 0x7f606406e7a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6084154070 tx=0x7f606c009040 comp rx=0 tx=0).stop 2024-09-19T07:57:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.723+0000 7f6089366640 1 -- 172.21.15.16:0/2091768599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f608406bc10 msgr2=0x7f6084157ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.723+0000 7f6089366640 1 --2- 172.21.15.16:0/2091768599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f608406bc10 0x7f6084157ab0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f607400e9b0 tx=0x7f607400ee70 comp rx=0 tx=0).stop 2024-09-19T07:57:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.723+0000 7f6089366640 1 -- 172.21.15.16:0/2091768599 shutdown_connections 2024-09-19T07:57:12.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.723+0000 7f6089366640 1 --2- 172.21.15.16:0/2091768599 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f606406c2e0 0x7f606406e7a0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:12.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.723+0000 7f6089366640 1 --2- 172.21.15.16:0/2091768599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60840ce970 0x7f6084159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:12.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.723+0000 7f6089366640 1 --2- 172.21.15.16:0/2091768599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f60840cddf0 0x7f60841598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:12.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.723+0000 7f6089366640 1 --2- 172.21.15.16:0/2091768599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f608406bc10 0x7f6084157ab0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:12.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.723+0000 7f6089366640 1 -- 172.21.15.16:0/2091768599 >> 172.21.15.16:0/2091768599 conn(0x7f608409aa50 msgr2=0x7f608406c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:12.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.723+0000 7f6089366640 1 -- 172.21.15.16:0/2091768599 shutdown_connections 2024-09-19T07:57:12.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.723+0000 7f6089366640 1 -- 172.21.15.16:0/2091768599 wait complete. 2024-09-19T07:57:12.734 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:57:12.734 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:57:12.752 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:57:12.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.895+0000 7f694d378640 1 Processor -- start 2024-09-19T07:57:12.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.895+0000 7f694d378640 1 -- start start 2024-09-19T07:57:12.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f694d378640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6948160b00 0x7f69480ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:12.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f694d378640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69480d1000 con 0x7f69480ce580 2024-09-19T07:57:12.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f694d378640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69480cc590 con 0x7f6948160010 2024-09-19T07:57:12.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f694d378640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f69480aee50 con 0x7f6948160b00 2024-09-19T07:57:12.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f693e7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f69480ce580 0x7f69480acfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57916/0 (socket says 172.21.15.16:57916) 2024-09-19T07:57:12.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f693e7fc640 1 -- 172.21.15.16:0/3946763254 learned_addr learned my addr 172.21.15.16:0/3946763254 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:12.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f69467fc640 1 -- 172.21.15.16:0/3946763254 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2941118075 0 0) 0x7f69480d1000 con 0x7f69480ce580 2024-09-19T07:57:12.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f69477fe640 1 --2- 172.21.15.16:0/3946763254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6948160b00 0x7f69480ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:12.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f69467fc640 1 -- 172.21.15.16:0/3946763254 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6934003520 con 0x7f69480ce580 2024-09-19T07:57:12.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f69467fc640 1 -- 172.21.15.16:0/3946763254 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1543164294 0 0) 0x7f6934003520 con 0x7f69480ce580 2024-09-19T07:57:12.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f69467fc640 1 -- 172.21.15.16:0/3946763254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6948160b00 msgr2=0x7f69480ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:12.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f69467fc640 1 --2- 172.21.15.16:0/3946763254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6948160b00 0x7f69480ad6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:12.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f69467fc640 1 -- 172.21.15.16:0/3946763254 >> v1:172.21.15.146:6789/0 conn(0x7f6948160010 legacy=0x7f69481603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:12.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f69467fc640 1 -- 172.21.15.16:0/3946763254 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69480cc870 con 0x7f69480ce580 2024-09-19T07:57:12.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f69467fc640 1 -- 172.21.15.16:0/3946763254 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f6930001670 con 0x7f69480ce580 2024-09-19T07:57:12.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f69467fc640 1 -- 172.21.15.16:0/3946763254 >> v1:172.21.15.16:6789/0 conn(0x7f69480ce580 legacy=0x7f69480acfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:12.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f69467fc640 1 --2- 172.21.15.16:0/3946763254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6948160b00 0x7f69340039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:12.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f69467fc640 1 --2- 172.21.15.16:0/3946763254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6934003f60 0x7f6934004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:12.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f69467fc640 1 --2- 172.21.15.16:0/3946763254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6934004d40 0x7f69340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:12.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f69467fc640 1 -- 172.21.15.16:0/3946763254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69480aee50 con 0x7f6934003f60 2024-09-19T07:57:12.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f69467fc640 1 -- 172.21.15.16:0/3946763254 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f693400baa0 con 0x7f6934004d40 2024-09-19T07:57:12.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f69467fc640 1 -- 172.21.15.16:0/3946763254 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f693400bca0 con 0x7f6948160b00 2024-09-19T07:57:12.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.896+0000 7f693e7fc640 1 --2- 172.21.15.16:0/3946763254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6934003f60 0x7f6934004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:12.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.897+0000 7f6946ffd640 1 --2- 172.21.15.16:0/3946763254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6948160b00 0x7f69340039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:12.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.897+0000 7f69477fe640 1 --2- 172.21.15.16:0/3946763254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6934004d40 0x7f69340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:12.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.897+0000 7f693e7fc640 1 -- 172.21.15.16:0/3946763254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6948160b00 msgr2=0x7f69340039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:12.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.897+0000 7f693e7fc640 1 --2- 172.21.15.16:0/3946763254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6948160b00 0x7f69340039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:12.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.897+0000 7f693e7fc640 1 -- 172.21.15.16:0/3946763254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6934004d40 msgr2=0x7f69340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:12.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.897+0000 7f693e7fc640 1 --2- 172.21.15.16:0/3946763254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6934004d40 0x7f69340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:12.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.897+0000 7f693e7fc640 1 -- 172.21.15.16:0/3946763254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6934003520 con 0x7f6934003f60 2024-09-19T07:57:12.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.897+0000 7f69477fe640 1 --2- 172.21.15.16:0/3946763254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6934004d40 0x7f69340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:12.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.897+0000 7f693e7fc640 1 --2- 172.21.15.16:0/3946763254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6934003f60 0x7f6934004800 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f693000c210 tx=0x7f6934005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf923a5334373b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:12.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.898+0000 7f69467fc640 1 -- 172.21.15.16:0/3946763254 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930007d30 con 0x7f6934003f60 2024-09-19T07:57:12.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.898+0000 7f69467fc640 1 -- 172.21.15.16:0/3946763254 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6930005ad0 con 0x7f6934003f60 2024-09-19T07:57:12.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.898+0000 7f69467fc640 1 -- 172.21.15.16:0/3946763254 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930007070 con 0x7f6934003f60 2024-09-19T07:57:12.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.898+0000 7f694d378640 1 -- 172.21.15.16:0/3946763254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6934003f60 msgr2=0x7f6934004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:12.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.898+0000 7f694d378640 1 --2- 172.21.15.16:0/3946763254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6934003f60 0x7f6934004800 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f693000c210 tx=0x7f6934005830 comp rx=0 tx=0).stop 2024-09-19T07:57:12.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.898+0000 7f69477fe640 1 -- 172.21.15.16:0/3946763254 reap_dead start 2024-09-19T07:57:12.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.898+0000 7f694d378640 1 -- 172.21.15.16:0/3946763254 shutdown_connections 2024-09-19T07:57:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.898+0000 7f694d378640 1 -- 172.21.15.16:0/3946763254 >> 172.21.15.16:0/3946763254 conn(0x7f69480b2300 msgr2=0x7f69480d00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.898+0000 7f694d378640 1 -- 172.21.15.16:0/3946763254 shutdown_connections 2024-09-19T07:57:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.898+0000 7f694d378640 1 -- 172.21.15.16:0/3946763254 wait complete. 2024-09-19T07:57:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.899+0000 7f694d378640 1 Processor -- start 2024-09-19T07:57:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.900+0000 7f694d378640 1 -- start start 2024-09-19T07:57:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.900+0000 7f694d378640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6934003f60 0x7f69481f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.900+0000 7f694d378640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f694809c300 0x7f694815f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.900+0000 7f694d378640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f694815ca60 0x7f69481598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.900+0000 7f694d378640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69480cce30 con 0x7f6934003f60 2024-09-19T07:57:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.900+0000 7f694d378640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6948160f90 con 0x7f694815ca60 2024-09-19T07:57:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.900+0000 7f694d378640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f69480cd110 con 0x7f694809c300 2024-09-19T07:57:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.900+0000 7f6946ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6934003f60 0x7f69481f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.900+0000 7f6946ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6934003f60 0x7f69481f29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44808/0 (socket says 172.21.15.16:44808) 2024-09-19T07:57:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.900+0000 7f6946ffd640 1 -- 172.21.15.16:0/2571632067 learned_addr learned my addr 172.21.15.16:0/2571632067 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.901+0000 7f693e7fc640 1 --2- 172.21.15.16:0/2571632067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f694809c300 0x7f694815f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.901+0000 7f69477fe640 1 --2- 172.21.15.16:0/2571632067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f694815ca60 0x7f69481598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.901+0000 7f6946ffd640 1 -- 172.21.15.16:0/2571632067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f694809c300 msgr2=0x7f694815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.901+0000 7f6946ffd640 1 --2- 172.21.15.16:0/2571632067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f694809c300 0x7f694815f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.901+0000 7f6946ffd640 1 -- 172.21.15.16:0/2571632067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f694815ca60 msgr2=0x7f69481598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.901+0000 7f6946ffd640 1 --2- 172.21.15.16:0/2571632067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f694815ca60 0x7f69481598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.901+0000 7f6946ffd640 1 -- 172.21.15.16:0/2571632067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694809aaf0 con 0x7f6934003f60 2024-09-19T07:57:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.901+0000 7f6946ffd640 1 --2- 172.21.15.16:0/2571632067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6934003f60 0x7f69481f29d0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f693800f820 tx=0x7f693800fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.901+0000 7f693ffff640 1 -- 172.21.15.16:0/2571632067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69380050d0 con 0x7f6934003f60 2024-09-19T07:57:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.901+0000 7f693ffff640 1 -- 172.21.15.16:0/2571632067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6938018070 con 0x7f6934003f60 2024-09-19T07:57:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.901+0000 7f694d378640 1 -- 172.21.15.16:0/2571632067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69480ca470 con 0x7f6934003f60 2024-09-19T07:57:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.901+0000 7f693ffff640 1 -- 172.21.15.16:0/2571632067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938005490 con 0x7f6934003f60 2024-09-19T07:57:12.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.901+0000 7f694d378640 1 -- 172.21.15.16:0/2571632067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f694809cac0 con 0x7f6934003f60 2024-09-19T07:57:12.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.902+0000 7f694d378640 1 -- 172.21.15.16:0/2571632067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69481603f0 con 0x7f6934003f60 2024-09-19T07:57:12.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.903+0000 7f693ffff640 1 -- 172.21.15.16:0/2571632067 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f693800d730 con 0x7f6934003f60 2024-09-19T07:57:12.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.903+0000 7f693ffff640 1 --2- 172.21.15.16:0/2571632067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f692806c2b0 0x7f692806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:12.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.903+0000 7f693ffff640 1 -- 172.21.15.16:0/2571632067 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f693808b470 con 0x7f6934003f60 2024-09-19T07:57:12.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.903+0000 7f693e7fc640 1 --2- 172.21.15.16:0/2571632067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f692806c2b0 0x7f692806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:12.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.904+0000 7f693e7fc640 1 --2- 172.21.15.16:0/2571632067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f692806c2b0 0x7f692806e770 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6930033000 tx=0x7f693002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:12.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:12.907+0000 7f693ffff640 1 -- 172.21.15.16:0/2571632067 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6938059a50 con 0x7f6934003f60 2024-09-19T07:57:13.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.096+0000 7f694d378640 1 -- 172.21.15.16:0/2571632067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f694806e420 con 0x7f6934003f60 2024-09-19T07:57:13.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.097+0000 7f693ffff640 1 -- 172.21.15.16:0/2571632067 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v152) ==== 80+0+0 (secure 0 0 0) 0x7f6938059350 con 0x7f6934003f60 2024-09-19T07:57:13.097 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-19T07:57:13.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.099+0000 7f694d378640 1 -- 172.21.15.16:0/2571632067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f692806c2b0 msgr2=0x7f692806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:13.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.099+0000 7f694d378640 1 --2- 172.21.15.16:0/2571632067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f692806c2b0 0x7f692806e770 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6930033000 tx=0x7f693002e000 comp rx=0 tx=0).stop 2024-09-19T07:57:13.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.099+0000 7f694d378640 1 -- 172.21.15.16:0/2571632067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6934003f60 msgr2=0x7f69481f29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:13.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.099+0000 7f694d378640 1 --2- 172.21.15.16:0/2571632067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6934003f60 0x7f69481f29d0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f693800f820 tx=0x7f693800fce0 comp rx=0 tx=0).stop 2024-09-19T07:57:13.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.099+0000 7f694d378640 1 -- 172.21.15.16:0/2571632067 shutdown_connections 2024-09-19T07:57:13.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.099+0000 7f694d378640 1 --2- 172.21.15.16:0/2571632067 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f692806c2b0 0x7f692806e770 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:13.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.099+0000 7f694d378640 1 --2- 172.21.15.16:0/2571632067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f694815ca60 0x7f69481598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:13.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.099+0000 7f694d378640 1 --2- 172.21.15.16:0/2571632067 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f694809c300 0x7f694815f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:13.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.099+0000 7f694d378640 1 --2- 172.21.15.16:0/2571632067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6934003f60 0x7f69481f29d0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:13.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.099+0000 7f694d378640 1 -- 172.21.15.16:0/2571632067 >> 172.21.15.16:0/2571632067 conn(0x7f69480b2300 msgr2=0x7f694806eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:13.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.099+0000 7f694d378640 1 -- 172.21.15.16:0/2571632067 shutdown_connections 2024-09-19T07:57:13.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.099+0000 7f694d378640 1 -- 172.21.15.16:0/2571632067 wait complete. 2024-09-19T07:57:13.110 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-19T07:57:13.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:57:13.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T07:57:13.136 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:57:13.137 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:57:13.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:57:13.137 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:57:13.193 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:57:13.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.334+0000 7f06702c9640 1 Processor -- start 2024-09-19T07:57:13.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.334+0000 7f06702c9640 1 -- start start 2024-09-19T07:57:13.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.335+0000 7f06702c9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f066809aa50 0x7f066809b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:13.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.335+0000 7f06702c9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f066806d3b0 0x7f0668169240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:13.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.335+0000 7f06702c9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06680d1b00 con 0x7f066809ae30 2024-09-19T07:57:13.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.335+0000 7f06702c9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f06680d10b0 con 0x7f066809aa50 2024-09-19T07:57:13.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.335+0000 7f06702c9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f066809bcd0 con 0x7f066806d3b0 2024-09-19T07:57:13.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.336+0000 7f066e065640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f066809aa50 0x7f066809b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:13.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.336+0000 7f066e065640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f066809aa50 0x7f066809b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44820/0 (socket says 172.21.15.16:44820) 2024-09-19T07:57:13.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.336+0000 7f066e065640 1 -- 172.21.15.16:0/2264284528 learned_addr learned my addr 172.21.15.16:0/2264284528 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:13.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.336+0000 7f066d864640 1 --1- 172.21.15.16:0/2264284528 >> v1:172.21.15.16:6789/0 conn(0x7f066809ae30 0x7f066806cca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57932/0 (socket says 172.21.15.16:57932) 2024-09-19T07:57:13.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.336+0000 7f066e866640 1 --2- 172.21.15.16:0/2264284528 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f066806d3b0 0x7f0668169240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:13.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.336+0000 7f066e065640 1 -- 172.21.15.16:0/2264284528 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f066806d3b0 msgr2=0x7f0668169240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:13.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.336+0000 7f066e065640 1 --2- 172.21.15.16:0/2264284528 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f066806d3b0 0x7f0668169240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:13.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.336+0000 7f066e065640 1 -- 172.21.15.16:0/2264284528 >> v1:172.21.15.16:6789/0 conn(0x7f066809ae30 legacy=0x7f066806cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:13.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.336+0000 7f066d063640 1 -- 172.21.15.16:0/2264284528 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 524213913 0 0) 0x7f06680d1b00 con 0x7f066809ae30 2024-09-19T07:57:13.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.336+0000 7f066e065640 1 -- 172.21.15.16:0/2264284528 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06680b30a0 con 0x7f066809aa50 2024-09-19T07:57:13.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.337+0000 7f066e065640 1 --2- 172.21.15.16:0/2264284528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f066809aa50 0x7f066809b790 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f0664009870 tx=0x7f066402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f293fe81da1daeb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:13.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.337+0000 7f066d063640 1 -- 172.21.15.16:0/2264284528 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664037070 con 0x7f066809aa50 2024-09-19T07:57:13.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.337+0000 7f066d063640 1 -- 172.21.15.16:0/2264284528 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0664032440 con 0x7f066809aa50 2024-09-19T07:57:13.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.337+0000 7f066d063640 1 -- 172.21.15.16:0/2264284528 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664032810 con 0x7f066809aa50 2024-09-19T07:57:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.337+0000 7f06702c9640 1 -- 172.21.15.16:0/2264284528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f066809aa50 msgr2=0x7f066809b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.337+0000 7f06702c9640 1 --2- 172.21.15.16:0/2264284528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f066809aa50 0x7f066809b790 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f0664009870 tx=0x7f066402a1c0 comp rx=0 tx=0).stop 2024-09-19T07:57:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.337+0000 7f06702c9640 1 -- 172.21.15.16:0/2264284528 shutdown_connections 2024-09-19T07:57:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.337+0000 7f06702c9640 1 --2- 172.21.15.16:0/2264284528 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f066806d3b0 0x7f0668169240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.337+0000 7f06702c9640 1 --2- 172.21.15.16:0/2264284528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f066809aa50 0x7f066809b790 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.337+0000 7f06702c9640 1 -- 172.21.15.16:0/2264284528 >> 172.21.15.16:0/2264284528 conn(0x7f06680d22d0 msgr2=0x7f06680d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.337+0000 7f06702c9640 1 -- 172.21.15.16:0/2264284528 shutdown_connections 2024-09-19T07:57:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.337+0000 7f06702c9640 1 -- 172.21.15.16:0/2264284528 wait complete. 2024-09-19T07:57:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.339+0000 7f06702c9640 1 Processor -- start 2024-09-19T07:57:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.339+0000 7f06702c9640 1 -- start start 2024-09-19T07:57:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.339+0000 7f06702c9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f066806d3b0 0x7f06680cf470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.339+0000 7f06702c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f066809aa50 0x7f0668157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.339+0000 7f06702c9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f066809ae30 0x7f06681f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.339+0000 7f06702c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06680d1300 con 0x7f066809aa50 2024-09-19T07:57:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.339+0000 7f06702c9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f06680b3660 con 0x7f066806d3b0 2024-09-19T07:57:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.339+0000 7f06702c9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f06680d15e0 con 0x7f066809ae30 2024-09-19T07:57:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.340+0000 7f066d864640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f066809aa50 0x7f0668157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.340+0000 7f066d864640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f066809aa50 0x7f0668157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44824/0 (socket says 172.21.15.16:44824) 2024-09-19T07:57:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.340+0000 7f066d864640 1 -- 172.21.15.16:0/1357516878 learned_addr learned my addr 172.21.15.16:0/1357516878 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.340+0000 7f066e065640 1 --2- 172.21.15.16:0/1357516878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f066806d3b0 0x7f06680cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.340+0000 7f066e866640 1 --2- 172.21.15.16:0/1357516878 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f066809ae30 0x7f06681f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.340+0000 7f066d864640 1 -- 172.21.15.16:0/1357516878 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f066809ae30 msgr2=0x7f06681f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.340+0000 7f066d864640 1 --2- 172.21.15.16:0/1357516878 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f066809ae30 0x7f06681f4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.340+0000 7f066d864640 1 -- 172.21.15.16:0/1357516878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f066806d3b0 msgr2=0x7f06680cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.340+0000 7f066d864640 1 --2- 172.21.15.16:0/1357516878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f066806d3b0 0x7f06680cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.340+0000 7f066d864640 1 -- 172.21.15.16:0/1357516878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06680cfa50 con 0x7f066809aa50 2024-09-19T07:57:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.340+0000 7f066d864640 1 --2- 172.21.15.16:0/1357516878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f066809aa50 0x7f0668157730 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f065800c3f0 tx=0x7f065800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.340+0000 7f065effd640 1 -- 172.21.15.16:0/1357516878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06580074e0 con 0x7f066809aa50 2024-09-19T07:57:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.340+0000 7f065effd640 1 -- 172.21.15.16:0/1357516878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06580047e0 con 0x7f066809aa50 2024-09-19T07:57:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.340+0000 7f06702c9640 1 -- 172.21.15.16:0/1357516878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06680b1060 con 0x7f066809aa50 2024-09-19T07:57:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.340+0000 7f065effd640 1 -- 172.21.15.16:0/1357516878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658004c10 con 0x7f066809aa50 2024-09-19T07:57:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.340+0000 7f06702c9640 1 -- 172.21.15.16:0/1357516878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0668157c70 con 0x7f066809aa50 2024-09-19T07:57:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.341+0000 7f06702c9640 1 -- 172.21.15.16:0/1357516878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f066809b2a0 con 0x7f066809aa50 2024-09-19T07:57:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.342+0000 7f065effd640 1 -- 172.21.15.16:0/1357516878 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0658011070 con 0x7f066809aa50 2024-09-19T07:57:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.342+0000 7f065effd640 1 --2- 172.21.15.16:0/1357516878 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f064c06c2e0 0x7f064c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.342+0000 7f065effd640 1 -- 172.21.15.16:0/1357516878 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6101+0+0 (secure 0 0 0) 0x7f065808b060 con 0x7f066809aa50 2024-09-19T07:57:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.343+0000 7f066e065640 1 --2- 172.21.15.16:0/1357516878 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f064c06c2e0 0x7f064c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.343+0000 7f066e065640 1 --2- 172.21.15.16:0/1357516878 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f064c06c2e0 0x7f064c06e7a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0664034000 tx=0x7f066402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:13.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.347+0000 7f065effd640 1 -- 172.21.15.16:0/1357516878 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f0658058fe0 con 0x7f066809aa50 2024-09-19T07:57:13.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.562+0000 7f06702c9640 1 -- 172.21.15.16:0/1357516878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f066809b890 con 0x7f066809aa50 2024-09-19T07:57:13.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.562+0000 7f065effd640 1 -- 172.21.15.16:0/1357516878 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5231 (secure 0 0 0) 0x7f065805d2e0 con 0x7f066809aa50 2024-09-19T07:57:13.563 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:13.563 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"btime":"2024-09-19T07:57:07:363117+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7064,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3873812166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3873812166},{"type":"v1","addr":"172.21.15.146:6837","nonce":3873812166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":7852,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/951273503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":951273503},{"type":"v1","addr":"172.21.15.146:6835","nonce":951273503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":8757,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2338289617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2338289617},{"type":"v1","addr":"172.21.15.16:6834","nonce":2338289617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:55:46.155417+0000","modified":"2024-09-19T07:57:06.840191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8760":{"gid":8760,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.16:6835/3705993654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":3705993654},{"type":"v1","addr":"172.21.15.16:6835","nonce":3705993654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8760,"qdb_cluster":[8760]},"id":11}]} 2024-09-19T07:57:13.563 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 93 2024-09-19T07:57:13.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.564+0000 7f06702c9640 1 -- 172.21.15.16:0/1357516878 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f064c06c2e0 msgr2=0x7f064c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:13.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.564+0000 7f06702c9640 1 --2- 172.21.15.16:0/1357516878 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f064c06c2e0 0x7f064c06e7a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0664034000 tx=0x7f066402e000 comp rx=0 tx=0).stop 2024-09-19T07:57:13.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.564+0000 7f06702c9640 1 -- 172.21.15.16:0/1357516878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f066809aa50 msgr2=0x7f0668157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:13.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.564+0000 7f06702c9640 1 --2- 172.21.15.16:0/1357516878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f066809aa50 0x7f0668157730 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f065800c3f0 tx=0x7f065800c8b0 comp rx=0 tx=0).stop 2024-09-19T07:57:13.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.564+0000 7f06702c9640 1 -- 172.21.15.16:0/1357516878 shutdown_connections 2024-09-19T07:57:13.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.564+0000 7f06702c9640 1 --2- 172.21.15.16:0/1357516878 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f064c06c2e0 0x7f064c06e7a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:13.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.564+0000 7f06702c9640 1 --2- 172.21.15.16:0/1357516878 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f066809ae30 0x7f06681f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:13.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.564+0000 7f06702c9640 1 --2- 172.21.15.16:0/1357516878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f066809aa50 0x7f0668157730 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:13.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.564+0000 7f06702c9640 1 --2- 172.21.15.16:0/1357516878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f066806d3b0 0x7f06680cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:13.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.564+0000 7f06702c9640 1 -- 172.21.15.16:0/1357516878 >> 172.21.15.16:0/1357516878 conn(0x7f06680d22d0 msgr2=0x7f0668160e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:13.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.564+0000 7f06702c9640 1 -- 172.21.15.16:0/1357516878 shutdown_connections 2024-09-19T07:57:13.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:13.564+0000 7f06702c9640 1 -- 172.21.15.16:0/1357516878 wait complete. 2024-09-19T07:57:13.575 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session evict 8985 2024-09-19T07:57:14.399 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2024-09-19T07:57:16.525 INFO:teuthology.orchestra.run.smithi146.stderr:write_background: writing 2024-09-19T07:57:16.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-19T07:57:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:57:16.540 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T07:57:16.540 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:57:16.587 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T07:57:16.630 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T07:57:16.630 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:57:16.630 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:57:16.630 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:57:16.649 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:57:16.649 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:57:16.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:57:16.650 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:57:16.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:57:16.709 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-19T07:57:16.709 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:57:16.709 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nfd = os.open("/home/ubuntu/cephtest/mnt.1/file1", os.O_RDWR | os.O_CREAT, 0o644)\ntry:\n while True:\n print("write_background: writing", file=sys.stderr)\n os.write(fd, b\'content\')\n if not False:\n break\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nexcept IOError as e:\n pass\nos.close(fd)\n'], check_status=True, hostname='smithi146') 2024-09-19T07:57:16.709 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:57:16.710 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:57:16.729 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T07:57:16.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.862+0000 7fec93aaf640 1 Processor -- start 2024-09-19T07:57:16.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.862+0000 7fec93aaf640 1 -- start start 2024-09-19T07:57:16.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.863+0000 7fec93aaf640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fec8c15f9a0 0x7fec8c09aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:16.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.863+0000 7fec93aaf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec8c0b4d20 con 0x7fec8c0b39a0 2024-09-19T07:57:16.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.863+0000 7fec93aaf640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec8c15fd80 con 0x7fec8c0b2f00 2024-09-19T07:57:16.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.863+0000 7fec93aaf640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fec8c0d0290 con 0x7fec8c15f9a0 2024-09-19T07:57:16.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.863+0000 7fec9104a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fec8c0b39a0 0x7fec8c0b1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57942/0 (socket says 172.21.15.16:57942) 2024-09-19T07:57:16.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.863+0000 7fec9104a640 1 -- 172.21.15.16:0/1047960965 learned_addr learned my addr 172.21.15.16:0/1047960965 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:16.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.863+0000 7fec90849640 1 -- 172.21.15.16:0/1047960965 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 928653444 0 0) 0x7fec8c0b4d20 con 0x7fec8c0b39a0 2024-09-19T07:57:16.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.863+0000 7fec9184b640 1 --2- 172.21.15.16:0/1047960965 >> v2:172.21.15.146:3300/0 conn(0x7fec8c15f9a0 0x7fec8c09aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:16.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.863+0000 7fec90849640 1 -- 172.21.15.16:0/1047960965 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec78003520 con 0x7fec8c0b39a0 2024-09-19T07:57:16.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec90849640 1 -- 172.21.15.16:0/1047960965 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1625923121 0 0) 0x7fec78003520 con 0x7fec8c0b39a0 2024-09-19T07:57:16.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec90849640 1 -- 172.21.15.16:0/1047960965 >> v2:172.21.15.146:3300/0 conn(0x7fec8c15f9a0 msgr2=0x7fec8c09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:16.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec90849640 1 --2- 172.21.15.16:0/1047960965 >> v2:172.21.15.146:3300/0 conn(0x7fec8c15f9a0 0x7fec8c09aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:16.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec90849640 1 -- 172.21.15.16:0/1047960965 >> v1:172.21.15.146:6789/0 conn(0x7fec8c0b2f00 legacy=0x7fec8c0b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:16.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec90849640 1 -- 172.21.15.16:0/1047960965 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8c0b4a40 con 0x7fec8c0b39a0 2024-09-19T07:57:16.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec90849640 1 -- 172.21.15.16:0/1047960965 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fec7c001670 con 0x7fec8c0b39a0 2024-09-19T07:57:16.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec90849640 1 -- 172.21.15.16:0/1047960965 >> v1:172.21.15.16:6789/0 conn(0x7fec8c0b39a0 legacy=0x7fec8c0b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:16.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec90849640 1 --2- 172.21.15.16:0/1047960965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec780039b0 0x7fec78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:16.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec90849640 1 --2- 172.21.15.16:0/1047960965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec78004800 0x7fec78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:16.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec90849640 1 --2- 172.21.15.16:0/1047960965 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec780051c0 0x7fec7800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:16.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec90849640 1 -- 172.21.15.16:0/1047960965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c0d0290 con 0x7fec780039b0 2024-09-19T07:57:16.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec90849640 1 -- 172.21.15.16:0/1047960965 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fec7800c160 con 0x7fec78004800 2024-09-19T07:57:16.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec90849640 1 -- 172.21.15.16:0/1047960965 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fec7800c360 con 0x7fec780051c0 2024-09-19T07:57:16.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec9184b640 1 --2- 172.21.15.16:0/1047960965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec780039b0 0x7fec78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:16.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec9104a640 1 --2- 172.21.15.16:0/1047960965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec78004800 0x7fec78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:16.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec9204c640 1 --2- 172.21.15.16:0/1047960965 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec780051c0 0x7fec7800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:16.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec9184b640 1 -- 172.21.15.16:0/1047960965 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec780051c0 msgr2=0x7fec7800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:16.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec9184b640 1 --2- 172.21.15.16:0/1047960965 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec780051c0 0x7fec7800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:16.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec9184b640 1 -- 172.21.15.16:0/1047960965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec78004800 msgr2=0x7fec78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:16.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec9184b640 1 --2- 172.21.15.16:0/1047960965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec78004800 0x7fec78004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:16.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec9184b640 1 -- 172.21.15.16:0/1047960965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec78003520 con 0x7fec780039b0 2024-09-19T07:57:16.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec9204c640 1 --2- 172.21.15.16:0/1047960965 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec780051c0 0x7fec7800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:16.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.864+0000 7fec9204c640 1 -- 172.21.15.16:0/1047960965 reap_dead start 2024-09-19T07:57:16.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.865+0000 7fec9184b640 1 --2- 172.21.15.16:0/1047960965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec780039b0 0x7fec78003dd0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fec88009ac0 tx=0x7fec78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3263d753a845df4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:16.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.865+0000 7fec90849640 1 -- 172.21.15.16:0/1047960965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88036070 con 0x7fec780039b0 2024-09-19T07:57:16.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.865+0000 7fec90849640 1 -- 172.21.15.16:0/1047960965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec88031400 con 0x7fec780039b0 2024-09-19T07:57:16.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.865+0000 7fec90849640 1 -- 172.21.15.16:0/1047960965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88009690 con 0x7fec780039b0 2024-09-19T07:57:16.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.866+0000 7fec93aaf640 1 -- 172.21.15.16:0/1047960965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec780039b0 msgr2=0x7fec78003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:16.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.866+0000 7fec93aaf640 1 --2- 172.21.15.16:0/1047960965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec780039b0 0x7fec78003dd0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fec88009ac0 tx=0x7fec78005830 comp rx=0 tx=0).stop 2024-09-19T07:57:16.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.866+0000 7fec93aaf640 1 -- 172.21.15.16:0/1047960965 shutdown_connections 2024-09-19T07:57:16.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.866+0000 7fec93aaf640 1 --2- 172.21.15.16:0/1047960965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec780039b0 0x7fec78003dd0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:16.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.866+0000 7fec93aaf640 1 -- 172.21.15.16:0/1047960965 >> 172.21.15.16:0/1047960965 conn(0x7fec8c0b3df0 msgr2=0x7fec8c0b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:16.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.866+0000 7fec93aaf640 1 -- 172.21.15.16:0/1047960965 shutdown_connections 2024-09-19T07:57:16.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.866+0000 7fec93aaf640 1 -- 172.21.15.16:0/1047960965 wait complete. 2024-09-19T07:57:16.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.867+0000 7fec93aaf640 1 Processor -- start 2024-09-19T07:57:16.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.867+0000 7fec93aaf640 1 -- start start 2024-09-19T07:57:16.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.868+0000 7fec93aaf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec780039b0 0x7fec8c0cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:16.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.868+0000 7fec93aaf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec8c0cabb0 0x7fec8c0af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:16.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.868+0000 7fec93aaf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec8c155380 0x7fec8c0b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:16.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.868+0000 7fec93aaf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c0cfed0 con 0x7fec8c0cabb0 2024-09-19T07:57:16.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.868+0000 7fec93aaf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c0b2c50 con 0x7fec780039b0 2024-09-19T07:57:16.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.868+0000 7fec93aaf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fec8c0d0050 con 0x7fec8c155380 2024-09-19T07:57:16.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.868+0000 7fec9104a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec8c0cabb0 0x7fec8c0af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:16.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.868+0000 7fec9104a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec8c0cabb0 0x7fec8c0af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44840/0 (socket says 172.21.15.16:44840) 2024-09-19T07:57:16.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.868+0000 7fec9104a640 1 -- 172.21.15.16:0/729772867 learned_addr learned my addr 172.21.15.16:0/729772867 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:16.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.868+0000 7fec9184b640 1 --2- 172.21.15.16:0/729772867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec780039b0 0x7fec8c0cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:16.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.868+0000 7fec9204c640 1 --2- 172.21.15.16:0/729772867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec8c155380 0x7fec8c0b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:16.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.868+0000 7fec9104a640 1 -- 172.21.15.16:0/729772867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec8c155380 msgr2=0x7fec8c0b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:16.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.868+0000 7fec9104a640 1 --2- 172.21.15.16:0/729772867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec8c155380 0x7fec8c0b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:16.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.868+0000 7fec9104a640 1 -- 172.21.15.16:0/729772867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec780039b0 msgr2=0x7fec8c0cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:16.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.868+0000 7fec9104a640 1 --2- 172.21.15.16:0/729772867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec780039b0 0x7fec8c0cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:16.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.868+0000 7fec9104a640 1 -- 172.21.15.16:0/729772867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8c06cec0 con 0x7fec8c0cabb0 2024-09-19T07:57:16.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.869+0000 7fec9104a640 1 --2- 172.21.15.16:0/729772867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec8c0cabb0 0x7fec8c0af050 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0072c0 tx=0x7fec7c007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:16.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.869+0000 7fec827fc640 1 -- 172.21.15.16:0/729772867 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c00f960 con 0x7fec8c0cabb0 2024-09-19T07:57:16.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.869+0000 7fec827fc640 1 -- 172.21.15.16:0/729772867 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec7c018070 con 0x7fec8c0cabb0 2024-09-19T07:57:16.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.869+0000 7fec93aaf640 1 -- 172.21.15.16:0/729772867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec8c06d1a0 con 0x7fec8c0cabb0 2024-09-19T07:57:16.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.869+0000 7fec827fc640 1 -- 172.21.15.16:0/729772867 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c00fd40 con 0x7fec8c0cabb0 2024-09-19T07:57:16.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.869+0000 7fec93aaf640 1 -- 172.21.15.16:0/729772867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec8c06d760 con 0x7fec8c0cabb0 2024-09-19T07:57:16.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.870+0000 7fec93aaf640 1 -- 172.21.15.16:0/729772867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec8c09aa50 con 0x7fec8c0cabb0 2024-09-19T07:57:16.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.870+0000 7fec827fc640 1 -- 172.21.15.16:0/729772867 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fec7c01d460 con 0x7fec8c0cabb0 2024-09-19T07:57:16.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.871+0000 7fec827fc640 1 --2- 172.21.15.16:0/729772867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fec7006c2b0 0x7fec7006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:16.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.871+0000 7fec827fc640 1 -- 172.21.15.16:0/729772867 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 6144+0+0 (secure 0 0 0) 0x7fec7c08b630 con 0x7fec8c0cabb0 2024-09-19T07:57:16.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.871+0000 7fec9184b640 1 --2- 172.21.15.16:0/729772867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fec7006c2b0 0x7fec7006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:16.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.872+0000 7fec9184b640 1 --2- 172.21.15.16:0/729772867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fec7006c2b0 0x7fec7006e770 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fec88033000 tx=0x7fec88008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:16.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:16.875+0000 7fec827fc640 1 -- 172.21.15.16:0/729772867 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fec7c059c80 con 0x7fec8c0cabb0 2024-09-19T07:57:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.064+0000 7fec93aaf640 1 -- 172.21.15.16:0/729772867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fec8c0cd0b0 con 0x7fec8c0cabb0 2024-09-19T07:57:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.064+0000 7fec827fc640 1 -- 172.21.15.16:0/729772867 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 1 entries v153) ==== 80+0+57 (secure 0 0 0) 0x7fec7c059580 con 0x7fec8c0cabb0 2024-09-19T07:57:17.065 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.1:0/2801864998 2024-09-19T08:57:13.688556+0000 2024-09-19T07:57:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:listed 1 entries 2024-09-19T07:57:17.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.066+0000 7fec93aaf640 1 -- 172.21.15.16:0/729772867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fec7006c2b0 msgr2=0x7fec7006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:17.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.066+0000 7fec93aaf640 1 --2- 172.21.15.16:0/729772867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fec7006c2b0 0x7fec7006e770 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fec88033000 tx=0x7fec88008030 comp rx=0 tx=0).stop 2024-09-19T07:57:17.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.066+0000 7fec93aaf640 1 -- 172.21.15.16:0/729772867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec8c0cabb0 msgr2=0x7fec8c0af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:17.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.066+0000 7fec93aaf640 1 --2- 172.21.15.16:0/729772867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec8c0cabb0 0x7fec8c0af050 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0072c0 tx=0x7fec7c007780 comp rx=0 tx=0).stop 2024-09-19T07:57:17.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.066+0000 7fec93aaf640 1 -- 172.21.15.16:0/729772867 shutdown_connections 2024-09-19T07:57:17.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.066+0000 7fec93aaf640 1 --2- 172.21.15.16:0/729772867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fec7006c2b0 0x7fec7006e770 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.066+0000 7fec93aaf640 1 --2- 172.21.15.16:0/729772867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec8c155380 0x7fec8c0b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.066+0000 7fec93aaf640 1 --2- 172.21.15.16:0/729772867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec8c0cabb0 0x7fec8c0af050 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.066+0000 7fec93aaf640 1 --2- 172.21.15.16:0/729772867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec780039b0 0x7fec8c0cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.066+0000 7fec93aaf640 1 -- 172.21.15.16:0/729772867 >> 172.21.15.16:0/729772867 conn(0x7fec8c0b3df0 msgr2=0x7fec8c0c7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:17.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.066+0000 7fec93aaf640 1 -- 172.21.15.16:0/729772867 shutdown_connections 2024-09-19T07:57:17.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.066+0000 7fec93aaf640 1 -- 172.21.15.16:0/729772867 wait complete. 2024-09-19T07:57:17.078 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2024-09-19T07:57:17.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:57:17.078 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T07:57:17.107 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[50378]: fuse finished with error 0 and tester_r 0 2024-09-19T07:57:17.109 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:57:17.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:57:17.109 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T07:57:17.236 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:57:17.237 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:57:17.237 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:57:17.253 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:57:17.253 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:57:17.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:57:17.254 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T07:57:17.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:57:17.309 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T07:57:17.309 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T07:57:17.309 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:57:17.309 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:57:17.327 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:57:17.328 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:57:17.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:57:17.328 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T07:57:17.385 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T07:57:17.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:57:17.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:57:17.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.488+0000 7f2d3563c640 1 Processor -- start 2024-09-19T07:57:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.488+0000 7f2d3563c640 1 -- start start 2024-09-19T07:57:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.489+0000 7f2d3563c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2d3006cd80 0x7f2d300cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.489+0000 7f2d3563c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d3006e090 0x7f2d300d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.489+0000 7f2d3563c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d3006d280 con 0x7f2d30162290 2024-09-19T07:57:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.489+0000 7f2d3563c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2d3006d560 con 0x7f2d3006e090 2024-09-19T07:57:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.489+0000 7f2d3563c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2d3006d840 con 0x7f2d3006cd80 2024-09-19T07:57:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.489+0000 7f2d2f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d3006e090 0x7f2d300d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.489+0000 7f2d2f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d3006e090 0x7f2d300d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44856/0 (socket says 172.21.15.16:44856) 2024-09-19T07:57:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.489+0000 7f2d2f7fe640 1 -- 172.21.15.16:0/2979267940 learned_addr learned my addr 172.21.15.16:0/2979267940 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.489+0000 7f2d2e7fc640 1 --2- 172.21.15.16:0/2979267940 >> v2:172.21.15.146:3300/0 conn(0x7f2d3006cd80 0x7f2d300cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:17.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.489+0000 7f2d2f7fe640 1 -- 172.21.15.16:0/2979267940 >> v2:172.21.15.146:3300/0 conn(0x7f2d3006cd80 msgr2=0x7f2d300cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:17.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.489+0000 7f2d2f7fe640 1 --2- 172.21.15.16:0/2979267940 >> v2:172.21.15.146:3300/0 conn(0x7f2d3006cd80 0x7f2d300cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.489+0000 7f2d2f7fe640 1 -- 172.21.15.16:0/2979267940 >> v1:172.21.15.146:6789/0 conn(0x7f2d30162290 legacy=0x7f2d30162670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:17.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.490+0000 7f2d2f7fe640 1 -- 172.21.15.16:0/2979267940 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3006db20 con 0x7f2d3006e090 2024-09-19T07:57:17.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.490+0000 7f2d2f7fe640 1 --2- 172.21.15.16:0/2979267940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d3006e090 0x7f2d300d0fc0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f2d300a2390 tx=0x7f2d2802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3448274f88f9c468 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:17.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.490+0000 7f2d2dffb640 1 -- 172.21.15.16:0/2979267940 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28036070 con 0x7f2d3006e090 2024-09-19T07:57:17.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.490+0000 7f2d2dffb640 1 -- 172.21.15.16:0/2979267940 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d28031600 con 0x7f2d3006e090 2024-09-19T07:57:17.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.490+0000 7f2d2dffb640 1 -- 172.21.15.16:0/2979267940 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d280319d0 con 0x7f2d3006e090 2024-09-19T07:57:17.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.491+0000 7f2d3563c640 1 -- 172.21.15.16:0/2979267940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d3006e090 msgr2=0x7f2d300d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:17.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.491+0000 7f2d3563c640 1 --2- 172.21.15.16:0/2979267940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d3006e090 0x7f2d300d0fc0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f2d300a2390 tx=0x7f2d2802a6a0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.491+0000 7f2d3563c640 1 -- 172.21.15.16:0/2979267940 shutdown_connections 2024-09-19T07:57:17.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.491+0000 7f2d3563c640 1 --2- 172.21.15.16:0/2979267940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d3006e090 0x7f2d300d0fc0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.491+0000 7f2d3563c640 1 --2- 172.21.15.16:0/2979267940 >> v2:172.21.15.146:3300/0 conn(0x7f2d3006cd80 0x7f2d300cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.491+0000 7f2d3563c640 1 -- 172.21.15.16:0/2979267940 >> 172.21.15.16:0/2979267940 conn(0x7f2d3008bfe0 msgr2=0x7f2d300d4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:17.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.491+0000 7f2d3563c640 1 -- 172.21.15.16:0/2979267940 shutdown_connections 2024-09-19T07:57:17.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.491+0000 7f2d3563c640 1 -- 172.21.15.16:0/2979267940 wait complete. 2024-09-19T07:57:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.492+0000 7f2d3563c640 1 Processor -- start 2024-09-19T07:57:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.493+0000 7f2d3563c640 1 -- start start 2024-09-19T07:57:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.493+0000 7f2d3563c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d3006cd80 0x7f2d300b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.493+0000 7f2d3563c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d3006e090 0x7f2d300cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.493+0000 7f2d3563c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d30162290 0x7f2d300b8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.493+0000 7f2d3563c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3006e6c0 con 0x7f2d3006e090 2024-09-19T07:57:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.493+0000 7f2d3563c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3006e540 con 0x7f2d30162290 2024-09-19T07:57:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.493+0000 7f2d3563c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2d300d1500 con 0x7f2d3006cd80 2024-09-19T07:57:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.493+0000 7f2d2e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d3006e090 0x7f2d300cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.493+0000 7f2d2e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d3006e090 0x7f2d300cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44858/0 (socket says 172.21.15.16:44858) 2024-09-19T07:57:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.493+0000 7f2d2e7fc640 1 -- 172.21.15.16:0/2572303765 learned_addr learned my addr 172.21.15.16:0/2572303765 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.494+0000 7f2d2effd640 1 --2- 172.21.15.16:0/2572303765 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d3006cd80 0x7f2d300b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.494+0000 7f2d2f7fe640 1 --2- 172.21.15.16:0/2572303765 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d30162290 0x7f2d300b8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.494+0000 7f2d2e7fc640 1 -- 172.21.15.16:0/2572303765 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d3006cd80 msgr2=0x7f2d300b72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.494+0000 7f2d2e7fc640 1 --2- 172.21.15.16:0/2572303765 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d3006cd80 0x7f2d300b72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.494+0000 7f2d2e7fc640 1 -- 172.21.15.16:0/2572303765 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d30162290 msgr2=0x7f2d300b8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.494+0000 7f2d2e7fc640 1 --2- 172.21.15.16:0/2572303765 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d30162290 0x7f2d300b8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.494+0000 7f2d2e7fc640 1 -- 172.21.15.16:0/2572303765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3015bfc0 con 0x7f2d3006e090 2024-09-19T07:57:17.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.494+0000 7f2d2e7fc640 1 --2- 172.21.15.16:0/2572303765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d3006e090 0x7f2d300cd7a0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400e960 tx=0x7f2d2400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:17.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.494+0000 7f2d177fe640 1 -- 172.21.15.16:0/2572303765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24017070 con 0x7f2d3006e090 2024-09-19T07:57:17.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.494+0000 7f2d177fe640 1 -- 172.21.15.16:0/2572303765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d2400ca00 con 0x7f2d3006e090 2024-09-19T07:57:17.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.494+0000 7f2d3563c640 1 -- 172.21.15.16:0/2572303765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d3015c2a0 con 0x7f2d3006e090 2024-09-19T07:57:17.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.494+0000 7f2d177fe640 1 -- 172.21.15.16:0/2572303765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2400cdf0 con 0x7f2d3006e090 2024-09-19T07:57:17.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.494+0000 7f2d3563c640 1 -- 172.21.15.16:0/2572303765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d300cbbd0 con 0x7f2d3006e090 2024-09-19T07:57:17.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.495+0000 7f2d3563c640 1 -- 172.21.15.16:0/2572303765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d300d0fc0 con 0x7f2d3006e090 2024-09-19T07:57:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.496+0000 7f2d177fe640 1 -- 172.21.15.16:0/2572303765 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2d2401a070 con 0x7f2d3006e090 2024-09-19T07:57:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.496+0000 7f2d177fe640 1 --2- 172.21.15.16:0/2572303765 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2d1c06c2b0 0x7f2d1c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.496+0000 7f2d177fe640 1 -- 172.21.15.16:0/2572303765 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 6144+0+0 (secure 0 0 0) 0x7f2d2408ba00 con 0x7f2d3006e090 2024-09-19T07:57:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.496+0000 7f2d2effd640 1 --2- 172.21.15.16:0/2572303765 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2d1c06c2b0 0x7f2d1c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.497+0000 7f2d2effd640 1 --2- 172.21.15.16:0/2572303765 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2d1c06c2b0 0x7f2d1c06e770 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2d20007fd0 tx=0x7f2d20009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:17.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.501+0000 7f2d177fe640 1 -- 172.21.15.16:0/2572303765 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2d24059fc0 con 0x7f2d3006e090 2024-09-19T07:57:17.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.712+0000 7f2d3563c640 1 -- 172.21.15.16:0/2572303765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d3008c690 con 0x7f2d3006e090 2024-09-19T07:57:17.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.713+0000 7f2d177fe640 1 -- 172.21.15.16:0/2572303765 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5231 (secure 0 0 0) 0x7f2d240598c0 con 0x7f2d3006e090 2024-09-19T07:57:17.713 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:17.714 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"btime":"2024-09-19T07:57:07:363117+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7064,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3873812166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3873812166},{"type":"v1","addr":"172.21.15.146:6837","nonce":3873812166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":7852,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/951273503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":951273503},{"type":"v1","addr":"172.21.15.146:6835","nonce":951273503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":8757,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2338289617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2338289617},{"type":"v1","addr":"172.21.15.16:6834","nonce":2338289617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:55:46.155417+0000","modified":"2024-09-19T07:57:06.840191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8760":{"gid":8760,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.16:6835/3705993654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":3705993654},{"type":"v1","addr":"172.21.15.16:6835","nonce":3705993654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8760,"qdb_cluster":[8760]},"id":11}]} 2024-09-19T07:57:17.714 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 93 2024-09-19T07:57:17.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.715+0000 7f2d3563c640 1 -- 172.21.15.16:0/2572303765 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2d1c06c2b0 msgr2=0x7f2d1c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:17.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.715+0000 7f2d3563c640 1 --2- 172.21.15.16:0/2572303765 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2d1c06c2b0 0x7f2d1c06e770 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2d20007fd0 tx=0x7f2d20009040 comp rx=0 tx=0).stop 2024-09-19T07:57:17.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.715+0000 7f2d3563c640 1 -- 172.21.15.16:0/2572303765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d3006e090 msgr2=0x7f2d300cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:17.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.715+0000 7f2d3563c640 1 --2- 172.21.15.16:0/2572303765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d3006e090 0x7f2d300cd7a0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400e960 tx=0x7f2d2400ee20 comp rx=0 tx=0).stop 2024-09-19T07:57:17.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.715+0000 7f2d3563c640 1 -- 172.21.15.16:0/2572303765 shutdown_connections 2024-09-19T07:57:17.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.715+0000 7f2d3563c640 1 --2- 172.21.15.16:0/2572303765 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2d1c06c2b0 0x7f2d1c06e770 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.715+0000 7f2d3563c640 1 --2- 172.21.15.16:0/2572303765 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d30162290 0x7f2d300b8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.715+0000 7f2d3563c640 1 --2- 172.21.15.16:0/2572303765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d3006e090 0x7f2d300cd7a0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.715+0000 7f2d3563c640 1 --2- 172.21.15.16:0/2572303765 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d3006cd80 0x7f2d300b72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.715+0000 7f2d3563c640 1 -- 172.21.15.16:0/2572303765 >> 172.21.15.16:0/2572303765 conn(0x7f2d3008bfe0 msgr2=0x7f2d3015ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:17.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.715+0000 7f2d3563c640 1 -- 172.21.15.16:0/2572303765 shutdown_connections 2024-09-19T07:57:17.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.715+0000 7f2d3563c640 1 -- 172.21.15.16:0/2572303765 wait complete. 2024-09-19T07:57:17.727 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:57:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.829+0000 7f263e6b7640 1 Processor -- start 2024-09-19T07:57:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.829+0000 7f263e6b7640 1 -- start start 2024-09-19T07:57:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.830+0000 7f263e6b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2638162290 0x7f26380cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.830+0000 7f263e6b7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f263806cd80 0x7f2638162670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.830+0000 7f263e6b7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f263806e090 0x7f26380d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.830+0000 7f263e6b7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2638162bb0 con 0x7f263806e090 2024-09-19T07:57:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.830+0000 7f263e6b7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f263806d280 con 0x7f263806cd80 2024-09-19T07:57:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.830+0000 7f263e6b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f263806d560 con 0x7f2638162290 2024-09-19T07:57:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.830+0000 7f263cc54640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f263806e090 0x7f26380d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.830+0000 7f263cc54640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f263806e090 0x7f26380d2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44864/0 (socket says 172.21.15.16:44864) 2024-09-19T07:57:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.830+0000 7f263cc54640 1 -- 172.21.15.16:0/3213828288 learned_addr learned my addr 172.21.15.16:0/3213828288 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.830+0000 7f2637fff640 1 --2- 172.21.15.16:0/3213828288 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2638162290 0x7f26380cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.830+0000 7f26377fe640 1 --2- 172.21.15.16:0/3213828288 >> v2:172.21.15.146:3300/0 conn(0x7f263806cd80 0x7f2638162670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.830+0000 7f263cc54640 1 -- 172.21.15.16:0/3213828288 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2638162290 msgr2=0x7f26380cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.830+0000 7f263cc54640 1 --2- 172.21.15.16:0/3213828288 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2638162290 0x7f26380cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.830+0000 7f263cc54640 1 -- 172.21.15.16:0/3213828288 >> v2:172.21.15.146:3300/0 conn(0x7f263806cd80 msgr2=0x7f2638162670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.830+0000 7f263cc54640 1 --2- 172.21.15.16:0/3213828288 >> v2:172.21.15.146:3300/0 conn(0x7f263806cd80 0x7f2638162670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.830+0000 7f263cc54640 1 -- 172.21.15.16:0/3213828288 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263806d840 con 0x7f263806e090 2024-09-19T07:57:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.830+0000 7f2637fff640 1 --2- 172.21.15.16:0/3213828288 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2638162290 0x7f26380cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.831+0000 7f263cc54640 1 --2- 172.21.15.16:0/3213828288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f263806e090 0x7f26380d2390 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f2630009e50 tx=0x7f263002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f349783f6bc90c8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.831+0000 7f2636ffd640 1 -- 172.21.15.16:0/3213828288 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2630036070 con 0x7f263806e090 2024-09-19T07:57:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.831+0000 7f2636ffd640 1 -- 172.21.15.16:0/3213828288 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2630031600 con 0x7f263806e090 2024-09-19T07:57:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.831+0000 7f2636ffd640 1 -- 172.21.15.16:0/3213828288 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26300319d0 con 0x7f263806e090 2024-09-19T07:57:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.832+0000 7f263e6b7640 1 -- 172.21.15.16:0/3213828288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f263806e090 msgr2=0x7f26380d2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.832+0000 7f263e6b7640 1 --2- 172.21.15.16:0/3213828288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f263806e090 0x7f26380d2390 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f2630009e50 tx=0x7f263002a630 comp rx=0 tx=0).stop 2024-09-19T07:57:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.832+0000 7f263e6b7640 1 -- 172.21.15.16:0/3213828288 shutdown_connections 2024-09-19T07:57:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.832+0000 7f263e6b7640 1 --2- 172.21.15.16:0/3213828288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f263806e090 0x7f26380d2390 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.832+0000 7f263e6b7640 1 --2- 172.21.15.16:0/3213828288 >> v2:172.21.15.146:3300/0 conn(0x7f263806cd80 0x7f2638162670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.832+0000 7f263e6b7640 1 --2- 172.21.15.16:0/3213828288 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2638162290 0x7f26380cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.832+0000 7f263e6b7640 1 -- 172.21.15.16:0/3213828288 >> 172.21.15.16:0/3213828288 conn(0x7f263808bfe0 msgr2=0x7f26380d4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.832+0000 7f263e6b7640 1 -- 172.21.15.16:0/3213828288 shutdown_connections 2024-09-19T07:57:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.832+0000 7f263e6b7640 1 -- 172.21.15.16:0/3213828288 wait complete. 2024-09-19T07:57:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.833+0000 7f263e6b7640 1 Processor -- start 2024-09-19T07:57:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.833+0000 7f263e6b7640 1 -- start start 2024-09-19T07:57:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.834+0000 7f263e6b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f263806cd80 0x7f26380cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.834+0000 7f263e6b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f263806e090 0x7f263814e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.834+0000 7f263e6b7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2638162290 0x7f263814d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.834+0000 7f263e6b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2638162040 con 0x7f263806e090 2024-09-19T07:57:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.834+0000 7f263e6b7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2638161ec0 con 0x7f2638162290 2024-09-19T07:57:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.834+0000 7f263e6b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f263806e540 con 0x7f263806cd80 2024-09-19T07:57:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.834+0000 7f26377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f263806e090 0x7f263814e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.834+0000 7f26377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f263806e090 0x7f263814e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44870/0 (socket says 172.21.15.16:44870) 2024-09-19T07:57:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.834+0000 7f26377fe640 1 -- 172.21.15.16:0/3650018283 learned_addr learned my addr 172.21.15.16:0/3650018283 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.834+0000 7f263cc54640 1 --2- 172.21.15.16:0/3650018283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2638162290 0x7f263814d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.834+0000 7f26377fe640 1 -- 172.21.15.16:0/3650018283 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f263806cd80 msgr2=0x7f26380cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.834+0000 7f26377fe640 1 --2- 172.21.15.16:0/3650018283 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f263806cd80 0x7f26380cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.834+0000 7f26377fe640 1 -- 172.21.15.16:0/3650018283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2638162290 msgr2=0x7f263814d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:17.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.834+0000 7f26377fe640 1 --2- 172.21.15.16:0/3650018283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2638162290 0x7f263814d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.834+0000 7f26377fe640 1 -- 172.21.15.16:0/3650018283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263815f830 con 0x7f263806e090 2024-09-19T07:57:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.834+0000 7f26377fe640 1 --2- 172.21.15.16:0/3650018283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f263806e090 0x7f263814e310 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f262c00e870 tx=0x7f262c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.835+0000 7f2634ff9640 1 -- 172.21.15.16:0/3650018283 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c017070 con 0x7f263806e090 2024-09-19T07:57:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.835+0000 7f2634ff9640 1 -- 172.21.15.16:0/3650018283 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f262c00c940 con 0x7f263806e090 2024-09-19T07:57:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.835+0000 7f263e6b7640 1 -- 172.21.15.16:0/3650018283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263815fb10 con 0x7f263806e090 2024-09-19T07:57:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.835+0000 7f2634ff9640 1 -- 172.21.15.16:0/3650018283 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c00cd10 con 0x7f263806e090 2024-09-19T07:57:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.835+0000 7f263e6b7640 1 -- 172.21.15.16:0/3650018283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26381600d0 con 0x7f263806e090 2024-09-19T07:57:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.836+0000 7f263e6b7640 1 -- 172.21.15.16:0/3650018283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2638162670 con 0x7f263806e090 2024-09-19T07:57:17.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.836+0000 7f2634ff9640 1 -- 172.21.15.16:0/3650018283 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f262c01a070 con 0x7f263806e090 2024-09-19T07:57:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.837+0000 7f2634ff9640 1 --2- 172.21.15.16:0/3650018283 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f262406c2b0 0x7f262406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.837+0000 7f2634ff9640 1 -- 172.21.15.16:0/3650018283 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 6144+0+0 (secure 0 0 0) 0x7f262c08b4b0 con 0x7f263806e090 2024-09-19T07:57:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.837+0000 7f2637fff640 1 --2- 172.21.15.16:0/3650018283 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f262406c2b0 0x7f262406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:17.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.838+0000 7f2637fff640 1 --2- 172.21.15.16:0/3650018283 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f262406c2b0 0x7f262406e770 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2628006fd0 tx=0x7f2628008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:17.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:17.842+0000 7f2634ff9640 1 -- 172.21.15.16:0/3650018283 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f262c059b00 con 0x7f263806e090 2024-09-19T07:57:18.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.051+0000 7f263e6b7640 1 -- 172.21.15.16:0/3650018283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26380ba410 con 0x7f263806e090 2024-09-19T07:57:18.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.051+0000 7f2634ff9640 1 -- 172.21.15.16:0/3650018283 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5231 (secure 0 0 0) 0x7f262c059400 con 0x7f263806e090 2024-09-19T07:57:18.052 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:18.052 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"btime":"2024-09-19T07:57:07:363117+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7064,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3873812166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3873812166},{"type":"v1","addr":"172.21.15.146:6837","nonce":3873812166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":7852,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/951273503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":951273503},{"type":"v1","addr":"172.21.15.146:6835","nonce":951273503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":8757,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2338289617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2338289617},{"type":"v1","addr":"172.21.15.16:6834","nonce":2338289617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:55:46.155417+0000","modified":"2024-09-19T07:57:06.840191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8760":{"gid":8760,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.16:6835/3705993654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":3705993654},{"type":"v1","addr":"172.21.15.16:6835","nonce":3705993654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8760,"qdb_cluster":[8760]},"id":11}]} 2024-09-19T07:57:18.052 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 93 2024-09-19T07:57:18.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.053+0000 7f263e6b7640 1 -- 172.21.15.16:0/3650018283 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f262406c2b0 msgr2=0x7f262406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.053+0000 7f263e6b7640 1 --2- 172.21.15.16:0/3650018283 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f262406c2b0 0x7f262406e770 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2628006fd0 tx=0x7f2628008040 comp rx=0 tx=0).stop 2024-09-19T07:57:18.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.053+0000 7f263e6b7640 1 -- 172.21.15.16:0/3650018283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f263806e090 msgr2=0x7f263814e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.053+0000 7f263e6b7640 1 --2- 172.21.15.16:0/3650018283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f263806e090 0x7f263814e310 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f262c00e870 tx=0x7f262c00ed30 comp rx=0 tx=0).stop 2024-09-19T07:57:18.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.053+0000 7f263e6b7640 1 -- 172.21.15.16:0/3650018283 shutdown_connections 2024-09-19T07:57:18.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.053+0000 7f263e6b7640 1 --2- 172.21.15.16:0/3650018283 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f262406c2b0 0x7f262406e770 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.053+0000 7f263e6b7640 1 --2- 172.21.15.16:0/3650018283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2638162290 0x7f263814d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.053+0000 7f263e6b7640 1 --2- 172.21.15.16:0/3650018283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f263806e090 0x7f263814e310 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.053+0000 7f263e6b7640 1 --2- 172.21.15.16:0/3650018283 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f263806cd80 0x7f26380cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.053+0000 7f263e6b7640 1 -- 172.21.15.16:0/3650018283 >> 172.21.15.16:0/3650018283 conn(0x7f263808bfe0 msgr2=0x7f26380d2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.053+0000 7f263e6b7640 1 -- 172.21.15.16:0/3650018283 shutdown_connections 2024-09-19T07:57:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.053+0000 7f263e6b7640 1 -- 172.21.15.16:0/3650018283 wait complete. 2024-09-19T07:57:18.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:57:18.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.167+0000 7f0f16b0a640 1 Processor -- start 2024-09-19T07:57:18.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.167+0000 7f0f16b0a640 1 -- start start 2024-09-19T07:57:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.168+0000 7f0f16b0a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f100cfc40 0x7f0f100b2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.168+0000 7f0f16b0a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f100cee50 con 0x7f0f100b7320 2024-09-19T07:57:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.168+0000 7f0f16b0a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f100cf130 con 0x7f0f1015e860 2024-09-19T07:57:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.168+0000 7f0f16b0a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f0f100b20e0 con 0x7f0f100cfc40 2024-09-19T07:57:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.168+0000 7f0f150a7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f100cfc40 0x7f0f100b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.168+0000 7f0f0ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0f100b7320 0x7f0f100cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57956/0 (socket says 172.21.15.16:57956) 2024-09-19T07:57:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.168+0000 7f0f0ffff640 1 -- 172.21.15.16:0/3001168730 learned_addr learned my addr 172.21.15.16:0/3001168730 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.168+0000 7f0f150a7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f100cfc40 0x7f0f100b2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44872/0 (socket says 172.21.15.16:44872) 2024-09-19T07:57:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.168+0000 7f0f150a7640 1 -- 172.21.15.16:0/3001168730 >> v1:172.21.15.146:6789/0 conn(0x7f0f1015e860 legacy=0x7f0f100d4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.168+0000 7f0f150a7640 1 -- 172.21.15.16:0/3001168730 >> v1:172.21.15.16:6789/0 conn(0x7f0f100b7320 legacy=0x7f0f100cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.168+0000 7f0f0f7fe640 1 -- 172.21.15.16:0/3001168730 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1039839557 0 0) 0x7f0f100cee50 con 0x7f0f100b7320 2024-09-19T07:57:18.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.168+0000 7f0f150a7640 1 -- 172.21.15.16:0/3001168730 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f100b1a90 con 0x7f0f100cfc40 2024-09-19T07:57:18.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.169+0000 7f0f150a7640 1 --2- 172.21.15.16:0/3001168730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f100cfc40 0x7f0f100b2570 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009d90 tx=0x7f0f0802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f21981af0c0184 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:18.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.169+0000 7f0f0f7fe640 1 -- 172.21.15.16:0/3001168730 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08007740 con 0x7f0f100cfc40 2024-09-19T07:57:18.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.169+0000 7f0f0f7fe640 1 -- 172.21.15.16:0/3001168730 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f080078e0 con 0x7f0f100cfc40 2024-09-19T07:57:18.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.169+0000 7f0f0f7fe640 1 -- 172.21.15.16:0/3001168730 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08007cb0 con 0x7f0f100cfc40 2024-09-19T07:57:18.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.170+0000 7f0f16b0a640 1 -- 172.21.15.16:0/3001168730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f100cfc40 msgr2=0x7f0f100b2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.170+0000 7f0f16b0a640 1 --2- 172.21.15.16:0/3001168730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f100cfc40 0x7f0f100b2570 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009d90 tx=0x7f0f0802a6f0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.170+0000 7f0f16b0a640 1 -- 172.21.15.16:0/3001168730 shutdown_connections 2024-09-19T07:57:18.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.170+0000 7f0f16b0a640 1 --2- 172.21.15.16:0/3001168730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f100cfc40 0x7f0f100b2570 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.170+0000 7f0f16b0a640 1 -- 172.21.15.16:0/3001168730 >> 172.21.15.16:0/3001168730 conn(0x7f0f1008bfe0 msgr2=0x7f0f100d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:18.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.170+0000 7f0f16b0a640 1 -- 172.21.15.16:0/3001168730 shutdown_connections 2024-09-19T07:57:18.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.170+0000 7f0f16b0a640 1 -- 172.21.15.16:0/3001168730 wait complete. 2024-09-19T07:57:18.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.171+0000 7f0f16b0a640 1 Processor -- start 2024-09-19T07:57:18.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.171+0000 7f0f16b0a640 1 -- start start 2024-09-19T07:57:18.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.172+0000 7f0f16b0a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f100b7320 0x7f0f10159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:18.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.172+0000 7f0f16b0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f100cfc40 0x7f0f10161080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:18.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.172+0000 7f0f16b0a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f1015e860 0x7f0f10155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:18.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.172+0000 7f0f16b0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f100b36c0 con 0x7f0f100cfc40 2024-09-19T07:57:18.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.172+0000 7f0f16b0a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0f100ceb20 con 0x7f0f100b7320 2024-09-19T07:57:18.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.172+0000 7f0f16b0a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0f100b3840 con 0x7f0f1015e860 2024-09-19T07:57:18.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.172+0000 7f0f0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f100cfc40 0x7f0f10161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:18.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.172+0000 7f0f0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f100cfc40 0x7f0f10161080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44874/0 (socket says 172.21.15.16:44874) 2024-09-19T07:57:18.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.172+0000 7f0f0ffff640 1 -- 172.21.15.16:0/842022963 learned_addr learned my addr 172.21.15.16:0/842022963 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:18.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.172+0000 7f0f150a7640 1 --2- 172.21.15.16:0/842022963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f1015e860 0x7f0f10155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:18.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.172+0000 7f0f148a6640 1 --2- 172.21.15.16:0/842022963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f100b7320 0x7f0f10159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:18.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.172+0000 7f0f0ffff640 1 -- 172.21.15.16:0/842022963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f1015e860 msgr2=0x7f0f10155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.172+0000 7f0f0ffff640 1 --2- 172.21.15.16:0/842022963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f1015e860 0x7f0f10155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.172+0000 7f0f0ffff640 1 -- 172.21.15.16:0/842022963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f100b7320 msgr2=0x7f0f10159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.172+0000 7f0f0ffff640 1 --2- 172.21.15.16:0/842022963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f100b7320 0x7f0f10159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.172+0000 7f0f0ffff640 1 -- 172.21.15.16:0/842022963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f101615c0 con 0x7f0f100cfc40 2024-09-19T07:57:18.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.172+0000 7f0f0ffff640 1 --2- 172.21.15.16:0/842022963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f100cfc40 0x7f0f10161080 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00c4a0 tx=0x7f0efc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:18.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.173+0000 7f0f0d7fa640 1 -- 172.21.15.16:0/842022963 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc018070 con 0x7f0f100cfc40 2024-09-19T07:57:18.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.173+0000 7f0f0d7fa640 1 -- 172.21.15.16:0/842022963 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0efc0074e0 con 0x7f0f100cfc40 2024-09-19T07:57:18.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.173+0000 7f0f0d7fa640 1 -- 172.21.15.16:0/842022963 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc004a20 con 0x7f0f100cfc40 2024-09-19T07:57:18.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.173+0000 7f0f16b0a640 1 -- 172.21.15.16:0/842022963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f10157f80 con 0x7f0f100cfc40 2024-09-19T07:57:18.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.173+0000 7f0f16b0a640 1 -- 172.21.15.16:0/842022963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f10158540 con 0x7f0f100cfc40 2024-09-19T07:57:18.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.174+0000 7f0f16b0a640 1 -- 172.21.15.16:0/842022963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f100d4fb0 con 0x7f0f100cfc40 2024-09-19T07:57:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.174+0000 7f0f0d7fa640 1 -- 172.21.15.16:0/842022963 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0efc01b070 con 0x7f0f100cfc40 2024-09-19T07:57:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.175+0000 7f0f0d7fa640 1 --2- 172.21.15.16:0/842022963 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0ef406c2e0 0x7f0ef406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.175+0000 7f0f0d7fa640 1 -- 172.21.15.16:0/842022963 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 6144+0+0 (secure 0 0 0) 0x7f0efc08a800 con 0x7f0f100cfc40 2024-09-19T07:57:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.175+0000 7f0f148a6640 1 --2- 172.21.15.16:0/842022963 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0ef406c2e0 0x7f0ef406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:18.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.175+0000 7f0f148a6640 1 --2- 172.21.15.16:0/842022963 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0ef406c2e0 0x7f0ef406e7a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f0f04000f60 tx=0x7f0f04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.179+0000 7f0f0d7fa640 1 -- 172.21.15.16:0/842022963 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f0efc058e50 con 0x7f0f100cfc40 2024-09-19T07:57:18.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.354+0000 7f0f16b0a640 1 -- 172.21.15.16:0/842022963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f100cf370 con 0x7f0f100cfc40 2024-09-19T07:57:18.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.355+0000 7f0f0d7fa640 1 -- 172.21.15.16:0/842022963 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) ==== 74+0+17028 (secure 0 0 0) 0x7f0efc058750 con 0x7f0f100cfc40 2024-09-19T07:57:18.356 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:18.356 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":153,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:57:14.379755+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:55:44.728059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-09-19T07:55:45.128249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/2801864998":"2024-09-19T08:57:13.688556+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:57:18.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.358+0000 7f0f16b0a640 1 -- 172.21.15.16:0/842022963 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0ef406c2e0 msgr2=0x7f0ef406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.358+0000 7f0f16b0a640 1 --2- 172.21.15.16:0/842022963 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0ef406c2e0 0x7f0ef406e7a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f0f04000f60 tx=0x7f0f04009040 comp rx=0 tx=0).stop 2024-09-19T07:57:18.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.358+0000 7f0f16b0a640 1 -- 172.21.15.16:0/842022963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f100cfc40 msgr2=0x7f0f10161080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.358+0000 7f0f16b0a640 1 --2- 172.21.15.16:0/842022963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f100cfc40 0x7f0f10161080 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00c4a0 tx=0x7f0efc00c960 comp rx=0 tx=0).stop 2024-09-19T07:57:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.358+0000 7f0f16b0a640 1 -- 172.21.15.16:0/842022963 shutdown_connections 2024-09-19T07:57:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.358+0000 7f0f16b0a640 1 --2- 172.21.15.16:0/842022963 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f0ef406c2e0 0x7f0ef406e7a0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.358+0000 7f0f16b0a640 1 --2- 172.21.15.16:0/842022963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f1015e860 0x7f0f10155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.358+0000 7f0f16b0a640 1 --2- 172.21.15.16:0/842022963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f100cfc40 0x7f0f10161080 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.358+0000 7f0f16b0a640 1 --2- 172.21.15.16:0/842022963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f100b7320 0x7f0f10159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.358+0000 7f0f16b0a640 1 -- 172.21.15.16:0/842022963 >> 172.21.15.16:0/842022963 conn(0x7f0f1008bfe0 msgr2=0x7f0f100b81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.358+0000 7f0f16b0a640 1 -- 172.21.15.16:0/842022963 shutdown_connections 2024-09-19T07:57:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.358+0000 7f0f16b0a640 1 -- 172.21.15.16:0/842022963 wait complete. 2024-09-19T07:57:18.370 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T07:57:18.370 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:57:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.471+0000 7fea31c1e640 1 Processor -- start 2024-09-19T07:57:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.471+0000 7fea31c1e640 1 -- start start 2024-09-19T07:57:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.472+0000 7fea31c1e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fea2c161bb0 0x7fea2c0b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.472+0000 7fea31c1e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea2c0b38a0 con 0x7fea2c06db20 2024-09-19T07:57:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.472+0000 7fea31c1e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea2c161f90 con 0x7fea2c0b3130 2024-09-19T07:57:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.472+0000 7fea31c1e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fea2c0d10d0 con 0x7fea2c161bb0 2024-09-19T07:57:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.472+0000 7fea2bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fea2c06db20 0x7fea2c0cc950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57964/0 (socket says 172.21.15.16:57964) 2024-09-19T07:57:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.472+0000 7fea2bfff640 1 -- 172.21.15.16:0/2122927803 learned_addr learned my addr 172.21.15.16:0/2122927803 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:18.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.472+0000 7fea2b7fe640 1 --2- 172.21.15.16:0/2122927803 >> v2:172.21.15.146:3300/0 conn(0x7fea2c161bb0 0x7fea2c0b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:18.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.473+0000 7fea2a7fc640 1 -- 172.21.15.16:0/2122927803 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3766175068 0 0) 0x7fea2c0b38a0 con 0x7fea2c06db20 2024-09-19T07:57:18.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.473+0000 7fea2a7fc640 1 -- 172.21.15.16:0/2122927803 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea14003520 con 0x7fea2c06db20 2024-09-19T07:57:18.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.473+0000 7fea2a7fc640 1 -- 172.21.15.16:0/2122927803 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2263848449 0 0) 0x7fea14003520 con 0x7fea2c06db20 2024-09-19T07:57:18.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.473+0000 7fea2b7fe640 1 -- 172.21.15.16:0/2122927803 >> v1:172.21.15.146:6789/0 conn(0x7fea2c0b3130 legacy=0x7fea2c06d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.473+0000 7fea2b7fe640 1 -- 172.21.15.16:0/2122927803 >> v1:172.21.15.16:6789/0 conn(0x7fea2c06db20 legacy=0x7fea2c0cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.473+0000 7fea2b7fe640 1 -- 172.21.15.16:0/2122927803 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2c0d13b0 con 0x7fea2c161bb0 2024-09-19T07:57:18.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.474+0000 7fea2b7fe640 1 --2- 172.21.15.16:0/2122927803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea2c161bb0 0x7fea2c0b4910 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0097a0 tx=0x7fea1c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22b67e03eb3cde6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:18.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.474+0000 7fea2a7fc640 1 -- 172.21.15.16:0/2122927803 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c037070 con 0x7fea2c161bb0 2024-09-19T07:57:18.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.474+0000 7fea2a7fc640 1 -- 172.21.15.16:0/2122927803 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea1c02ac50 con 0x7fea2c161bb0 2024-09-19T07:57:18.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.474+0000 7fea2a7fc640 1 -- 172.21.15.16:0/2122927803 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea240019f0 con 0x7fea2c161bb0 2024-09-19T07:57:18.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.474+0000 7fea31c1e640 1 -- 172.21.15.16:0/2122927803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea2c161bb0 msgr2=0x7fea2c0b4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.474+0000 7fea31c1e640 1 --2- 172.21.15.16:0/2122927803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea2c161bb0 0x7fea2c0b4910 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0097a0 tx=0x7fea1c02a1c0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.474+0000 7fea31c1e640 1 -- 172.21.15.16:0/2122927803 shutdown_connections 2024-09-19T07:57:18.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.474+0000 7fea31c1e640 1 --2- 172.21.15.16:0/2122927803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea2c161bb0 0x7fea2c0b4910 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.474+0000 7fea31c1e640 1 -- 172.21.15.16:0/2122927803 >> 172.21.15.16:0/2122927803 conn(0x7fea2c0b3df0 msgr2=0x7fea2c0b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:18.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.474+0000 7fea31c1e640 1 -- 172.21.15.16:0/2122927803 shutdown_connections 2024-09-19T07:57:18.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.474+0000 7fea31c1e640 1 -- 172.21.15.16:0/2122927803 wait complete. 2024-09-19T07:57:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.476+0000 7fea31c1e640 1 Processor -- start 2024-09-19T07:57:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.476+0000 7fea31c1e640 1 -- start start 2024-09-19T07:57:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.476+0000 7fea31c1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c06db20 0x7fea2c0b7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.476+0000 7fea31c1e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea2c0b3130 0x7fea2c1f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.476+0000 7fea31c1e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea2c161bb0 0x7fea2c0c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.476+0000 7fea31c1e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c0b3570 con 0x7fea2c06db20 2024-09-19T07:57:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.476+0000 7fea31c1e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c0d1970 con 0x7fea2c161bb0 2024-09-19T07:57:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.476+0000 7fea31c1e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fea2c0cd060 con 0x7fea2c0b3130 2024-09-19T07:57:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.477+0000 7fea2b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c06db20 0x7fea2c0b7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.477+0000 7fea2b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c06db20 0x7fea2c0b7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44876/0 (socket says 172.21.15.16:44876) 2024-09-19T07:57:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.477+0000 7fea2b7fe640 1 -- 172.21.15.16:0/153326867 learned_addr learned my addr 172.21.15.16:0/153326867 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.477+0000 7fea2bfff640 1 --2- 172.21.15.16:0/153326867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea2c161bb0 0x7fea2c0c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.477+0000 7fea2affd640 1 --2- 172.21.15.16:0/153326867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea2c0b3130 0x7fea2c1f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.477+0000 7fea2b7fe640 1 -- 172.21.15.16:0/153326867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea2c0b3130 msgr2=0x7fea2c1f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.477+0000 7fea2b7fe640 1 --2- 172.21.15.16:0/153326867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea2c0b3130 0x7fea2c1f36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.477+0000 7fea2b7fe640 1 -- 172.21.15.16:0/153326867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea2c161bb0 msgr2=0x7fea2c0c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.477+0000 7fea2b7fe640 1 --2- 172.21.15.16:0/153326867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea2c161bb0 0x7fea2c0c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.477+0000 7fea2b7fe640 1 -- 172.21.15.16:0/153326867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2c0c6cb0 con 0x7fea2c06db20 2024-09-19T07:57:18.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.477+0000 7fea2bfff640 1 --2- 172.21.15.16:0/153326867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea2c161bb0 0x7fea2c0c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:18.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.477+0000 7fea2b7fe640 1 --2- 172.21.15.16:0/153326867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c06db20 0x7fea2c0b7c50 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0098c0 tx=0x7fea1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:18.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.477+0000 7fea13fff640 1 -- 172.21.15.16:0/153326867 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c041070 con 0x7fea2c06db20 2024-09-19T07:57:18.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.477+0000 7fea13fff640 1 -- 172.21.15.16:0/153326867 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea1c03c460 con 0x7fea2c06db20 2024-09-19T07:57:18.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.477+0000 7fea31c1e640 1 -- 172.21.15.16:0/153326867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2c0c6f50 con 0x7fea2c06db20 2024-09-19T07:57:18.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.477+0000 7fea13fff640 1 -- 172.21.15.16:0/153326867 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c037040 con 0x7fea2c06db20 2024-09-19T07:57:18.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.477+0000 7fea31c1e640 1 -- 172.21.15.16:0/153326867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea2c0b53a0 con 0x7fea2c06db20 2024-09-19T07:57:18.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.479+0000 7fea31c1e640 1 -- 172.21.15.16:0/153326867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea2c0cccb0 con 0x7fea2c06db20 2024-09-19T07:57:18.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.479+0000 7fea13fff640 1 -- 172.21.15.16:0/153326867 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fea1c044070 con 0x7fea2c06db20 2024-09-19T07:57:18.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.479+0000 7fea13fff640 1 --2- 172.21.15.16:0/153326867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fea1806c2b0 0x7fea1806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:18.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.479+0000 7fea13fff640 1 -- 172.21.15.16:0/153326867 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 6144+0+0 (secure 0 0 0) 0x7fea1c0abcb0 con 0x7fea2c06db20 2024-09-19T07:57:18.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.480+0000 7fea2affd640 1 --2- 172.21.15.16:0/153326867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fea1806c2b0 0x7fea1806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:18.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.480+0000 7fea2affd640 1 --2- 172.21.15.16:0/153326867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fea1806c2b0 0x7fea1806e770 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0b7950 tx=0x7fea20009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:18.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.484+0000 7fea13fff640 1 -- 172.21.15.16:0/153326867 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fea1c07a200 con 0x7fea2c06db20 2024-09-19T07:57:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.695+0000 7fea31c1e640 1 -- 172.21.15.16:0/153326867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea2c0b5150 con 0x7fea2c06db20 2024-09-19T07:57:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.695+0000 7fea13fff640 1 -- 172.21.15.16:0/153326867 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5231 (secure 0 0 0) 0x7fea1c079b00 con 0x7fea2c06db20 2024-09-19T07:57:18.696 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:18.696 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"btime":"2024-09-19T07:57:07:363117+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7064,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3873812166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3873812166},{"type":"v1","addr":"172.21.15.146:6837","nonce":3873812166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":7852,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/951273503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":951273503},{"type":"v1","addr":"172.21.15.146:6835","nonce":951273503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":8757,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2338289617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2338289617},{"type":"v1","addr":"172.21.15.16:6834","nonce":2338289617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:55:46.155417+0000","modified":"2024-09-19T07:57:06.840191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8760":{"gid":8760,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.16:6835/3705993654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":3705993654},{"type":"v1","addr":"172.21.15.16:6835","nonce":3705993654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8760,"qdb_cluster":[8760]},"id":11}]} 2024-09-19T07:57:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 93 2024-09-19T07:57:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.697+0000 7fea31c1e640 1 -- 172.21.15.16:0/153326867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fea1806c2b0 msgr2=0x7fea1806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.697+0000 7fea31c1e640 1 --2- 172.21.15.16:0/153326867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fea1806c2b0 0x7fea1806e770 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0b7950 tx=0x7fea20009040 comp rx=0 tx=0).stop 2024-09-19T07:57:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.697+0000 7fea31c1e640 1 -- 172.21.15.16:0/153326867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c06db20 msgr2=0x7fea2c0b7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.697+0000 7fea31c1e640 1 --2- 172.21.15.16:0/153326867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c06db20 0x7fea2c0b7c50 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0098c0 tx=0x7fea1c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.697+0000 7fea31c1e640 1 -- 172.21.15.16:0/153326867 shutdown_connections 2024-09-19T07:57:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.697+0000 7fea31c1e640 1 --2- 172.21.15.16:0/153326867 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fea1806c2b0 0x7fea1806e770 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.697+0000 7fea31c1e640 1 --2- 172.21.15.16:0/153326867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea2c161bb0 0x7fea2c0c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.697+0000 7fea31c1e640 1 --2- 172.21.15.16:0/153326867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea2c0b3130 0x7fea2c1f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.697+0000 7fea31c1e640 1 --2- 172.21.15.16:0/153326867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c06db20 0x7fea2c0b7c50 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.697+0000 7fea31c1e640 1 -- 172.21.15.16:0/153326867 >> 172.21.15.16:0/153326867 conn(0x7fea2c0b3df0 msgr2=0x7fea2c0cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.697+0000 7fea31c1e640 1 -- 172.21.15.16:0/153326867 shutdown_connections 2024-09-19T07:57:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.697+0000 7fea31c1e640 1 -- 172.21.15.16:0/153326867 wait complete. 2024-09-19T07:57:18.709 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:57:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.810+0000 7f2e75ee6640 1 Processor -- start 2024-09-19T07:57:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.810+0000 7f2e75ee6640 1 -- start start 2024-09-19T07:57:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.811+0000 7f2e75ee6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2e7006cd80 0x7f2e700cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.811+0000 7f2e75ee6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e7006e090 0x7f2e700d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.811+0000 7f2e75ee6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e7006d280 con 0x7f2e70162290 2024-09-19T07:57:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.811+0000 7f2e75ee6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f2e7006d560 con 0x7f2e7006cd80 2024-09-19T07:57:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.811+0000 7f2e75ee6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2e7006d840 con 0x7f2e7006e090 2024-09-19T07:57:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.811+0000 7f2e6effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2e7006cd80 0x7f2e700cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.811+0000 7f2e6f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2e70162290 0x7f2e70162670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57978/0 (socket says 172.21.15.16:57978) 2024-09-19T07:57:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.811+0000 7f2e6f7fe640 1 -- 172.21.15.16:0/2895991042 learned_addr learned my addr 172.21.15.16:0/2895991042 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.811+0000 7f2e6effd640 1 -- 172.21.15.16:0/2895991042 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e7006e090 msgr2=0x7f2e700d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.811+0000 7f2e6ffff640 1 --2- 172.21.15.16:0/2895991042 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e7006e090 0x7f2e700d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.811+0000 7f2e6effd640 1 --2- 172.21.15.16:0/2895991042 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e7006e090 0x7f2e700d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.811+0000 7f2e6effd640 1 -- 172.21.15.16:0/2895991042 >> v1:172.21.15.16:6789/0 conn(0x7f2e70162290 legacy=0x7f2e70162670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.811+0000 7f2e6e7fc640 1 -- 172.21.15.16:0/2895991042 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 994310766 0 0) 0x7f2e7006d280 con 0x7f2e70162290 2024-09-19T07:57:18.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.811+0000 7f2e6effd640 1 -- 172.21.15.16:0/2895991042 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7006db20 con 0x7f2e7006cd80 2024-09-19T07:57:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.812+0000 7f2e6effd640 1 --2- 172.21.15.16:0/2895991042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7006cd80 0x7f2e700cf1e0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f2e640097a0 tx=0x7f2e6402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c385190fa18e48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.812+0000 7f2e6e7fc640 1 -- 172.21.15.16:0/2895991042 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e64037070 con 0x7f2e7006cd80 2024-09-19T07:57:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.812+0000 7f2e6e7fc640 1 -- 172.21.15.16:0/2895991042 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e6402ae50 con 0x7f2e7006cd80 2024-09-19T07:57:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.812+0000 7f2e6e7fc640 1 -- 172.21.15.16:0/2895991042 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e64032660 con 0x7f2e7006cd80 2024-09-19T07:57:18.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.813+0000 7f2e75ee6640 1 -- 172.21.15.16:0/2895991042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7006cd80 msgr2=0x7f2e700cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.813+0000 7f2e75ee6640 1 --2- 172.21.15.16:0/2895991042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7006cd80 0x7f2e700cf1e0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f2e640097a0 tx=0x7f2e6402a100 comp rx=0 tx=0).stop 2024-09-19T07:57:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.813+0000 7f2e75ee6640 1 -- 172.21.15.16:0/2895991042 shutdown_connections 2024-09-19T07:57:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.813+0000 7f2e75ee6640 1 --2- 172.21.15.16:0/2895991042 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e7006e090 0x7f2e700d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.813+0000 7f2e75ee6640 1 --2- 172.21.15.16:0/2895991042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7006cd80 0x7f2e700cf1e0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.813+0000 7f2e75ee6640 1 -- 172.21.15.16:0/2895991042 >> 172.21.15.16:0/2895991042 conn(0x7f2e7008bfe0 msgr2=0x7f2e700d4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.813+0000 7f2e75ee6640 1 -- 172.21.15.16:0/2895991042 shutdown_connections 2024-09-19T07:57:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.813+0000 7f2e75ee6640 1 -- 172.21.15.16:0/2895991042 wait complete. 2024-09-19T07:57:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.814+0000 7f2e75ee6640 1 Processor -- start 2024-09-19T07:57:18.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.814+0000 7f2e75ee6640 1 -- start start 2024-09-19T07:57:18.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.815+0000 7f2e75ee6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7006cd80 0x7f2e701f5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:18.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.815+0000 7f2e75ee6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e7006e090 0x7f2e7015c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:18.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.815+0000 7f2e75ee6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e70162290 0x7f2e701f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:18.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.815+0000 7f2e75ee6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7006e6c0 con 0x7f2e7006cd80 2024-09-19T07:57:18.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.815+0000 7f2e75ee6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7006e540 con 0x7f2e70162290 2024-09-19T07:57:18.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.815+0000 7f2e75ee6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2e700d1500 con 0x7f2e7006e090 2024-09-19T07:57:18.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.815+0000 7f2e6f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7006cd80 0x7f2e701f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:18.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.815+0000 7f2e6f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7006cd80 0x7f2e701f5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44892/0 (socket says 172.21.15.16:44892) 2024-09-19T07:57:18.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.815+0000 7f2e6f7fe640 1 -- 172.21.15.16:0/2126751677 learned_addr learned my addr 172.21.15.16:0/2126751677 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:18.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.815+0000 7f2e6ffff640 1 --2- 172.21.15.16:0/2126751677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e70162290 0x7f2e701f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:18.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.815+0000 7f2e6effd640 1 --2- 172.21.15.16:0/2126751677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e7006e090 0x7f2e7015c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.815+0000 7f2e6f7fe640 1 -- 172.21.15.16:0/2126751677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e7006e090 msgr2=0x7f2e7015c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.815+0000 7f2e6f7fe640 1 --2- 172.21.15.16:0/2126751677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e7006e090 0x7f2e7015c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.815+0000 7f2e6f7fe640 1 -- 172.21.15.16:0/2126751677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e70162290 msgr2=0x7f2e701f82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.815+0000 7f2e6f7fe640 1 --2- 172.21.15.16:0/2126751677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e70162290 0x7f2e701f82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.815+0000 7f2e6f7fe640 1 -- 172.21.15.16:0/2126751677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e70160ad0 con 0x7f2e7006cd80 2024-09-19T07:57:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.815+0000 7f2e6effd640 1 --2- 172.21.15.16:0/2126751677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e7006e090 0x7f2e7015c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.816+0000 7f2e6f7fe640 1 --2- 172.21.15.16:0/2126751677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7006cd80 0x7f2e701f5750 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000c430 tx=0x7f2e6000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.816+0000 7f2e57fff640 1 -- 172.21.15.16:0/2126751677 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e600074e0 con 0x7f2e7006cd80 2024-09-19T07:57:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.816+0000 7f2e57fff640 1 -- 172.21.15.16:0/2126751677 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e600047e0 con 0x7f2e7006cd80 2024-09-19T07:57:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.816+0000 7f2e75ee6640 1 -- 172.21.15.16:0/2126751677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e70160db0 con 0x7f2e7006cd80 2024-09-19T07:57:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.816+0000 7f2e57fff640 1 -- 172.21.15.16:0/2126751677 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60004bd0 con 0x7f2e7006cd80 2024-09-19T07:57:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.816+0000 7f2e75ee6640 1 -- 172.21.15.16:0/2126751677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e70161370 con 0x7f2e7006cd80 2024-09-19T07:57:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.817+0000 7f2e75ee6640 1 -- 172.21.15.16:0/2126751677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e7008c690 con 0x7f2e7006cd80 2024-09-19T07:57:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.817+0000 7f2e57fff640 1 -- 172.21.15.16:0/2126751677 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2e60011070 con 0x7f2e7006cd80 2024-09-19T07:57:18.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.818+0000 7f2e57fff640 1 --2- 172.21.15.16:0/2126751677 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2e5c06c2e0 0x7f2e5c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:18.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.818+0000 7f2e57fff640 1 -- 172.21.15.16:0/2126751677 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 6144+0+0 (secure 0 0 0) 0x7f2e6008b120 con 0x7f2e7006cd80 2024-09-19T07:57:18.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.818+0000 7f2e6effd640 1 --2- 172.21.15.16:0/2126751677 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2e5c06c2e0 0x7f2e5c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:18.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.819+0000 7f2e6effd640 1 --2- 172.21.15.16:0/2126751677 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2e5c06c2e0 0x7f2e5c06e7a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2e64034000 tx=0x7f2e6402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:18.822+0000 7f2e57fff640 1 -- 172.21.15.16:0/2126751677 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2e60059770 con 0x7f2e7006cd80 2024-09-19T07:57:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.037+0000 7f2e75ee6640 1 -- 172.21.15.16:0/2126751677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e700ba410 con 0x7f2e7006cd80 2024-09-19T07:57:19.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.038+0000 7f2e57fff640 1 -- 172.21.15.16:0/2126751677 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5231 (secure 0 0 0) 0x7f2e60059070 con 0x7f2e7006cd80 2024-09-19T07:57:19.038 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:19.039 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"btime":"2024-09-19T07:57:07:363117+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7064,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3873812166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3873812166},{"type":"v1","addr":"172.21.15.146:6837","nonce":3873812166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":7852,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/951273503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":951273503},{"type":"v1","addr":"172.21.15.146:6835","nonce":951273503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":8757,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2338289617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2338289617},{"type":"v1","addr":"172.21.15.16:6834","nonce":2338289617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:55:46.155417+0000","modified":"2024-09-19T07:57:06.840191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8760":{"gid":8760,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.16:6835/3705993654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":3705993654},{"type":"v1","addr":"172.21.15.16:6835","nonce":3705993654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8760,"qdb_cluster":[8760]},"id":11}]} 2024-09-19T07:57:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 93 2024-09-19T07:57:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.040+0000 7f2e75ee6640 1 -- 172.21.15.16:0/2126751677 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2e5c06c2e0 msgr2=0x7f2e5c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.040+0000 7f2e75ee6640 1 --2- 172.21.15.16:0/2126751677 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2e5c06c2e0 0x7f2e5c06e7a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2e64034000 tx=0x7f2e6402e000 comp rx=0 tx=0).stop 2024-09-19T07:57:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.040+0000 7f2e75ee6640 1 -- 172.21.15.16:0/2126751677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7006cd80 msgr2=0x7f2e701f5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.040+0000 7f2e75ee6640 1 --2- 172.21.15.16:0/2126751677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7006cd80 0x7f2e701f5750 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000c430 tx=0x7f2e6000c8f0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.040+0000 7f2e75ee6640 1 -- 172.21.15.16:0/2126751677 shutdown_connections 2024-09-19T07:57:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.040+0000 7f2e75ee6640 1 --2- 172.21.15.16:0/2126751677 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f2e5c06c2e0 0x7f2e5c06e7a0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.040+0000 7f2e75ee6640 1 --2- 172.21.15.16:0/2126751677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e70162290 0x7f2e701f82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.040+0000 7f2e75ee6640 1 --2- 172.21.15.16:0/2126751677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e7006e090 0x7f2e7015c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.040+0000 7f2e75ee6640 1 --2- 172.21.15.16:0/2126751677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7006cd80 0x7f2e701f5750 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.040+0000 7f2e75ee6640 1 -- 172.21.15.16:0/2126751677 >> 172.21.15.16:0/2126751677 conn(0x7f2e7008bfe0 msgr2=0x7f2e700d2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:19.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.040+0000 7f2e75ee6640 1 -- 172.21.15.16:0/2126751677 shutdown_connections 2024-09-19T07:57:19.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.040+0000 7f2e75ee6640 1 -- 172.21.15.16:0/2126751677 wait complete. 2024-09-19T07:57:19.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:57:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.153+0000 7ff3509cd640 1 Processor -- start 2024-09-19T07:57:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.153+0000 7ff3509cd640 1 -- start start 2024-09-19T07:57:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff3509cd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff34c06db20 0x7ff34c0b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff3509cd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff34c0b38a0 con 0x7ff34c0b3130 2024-09-19T07:57:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff3509cd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff34c0b35e0 con 0x7ff34c161bb0 2024-09-19T07:57:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff3509cd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff34c161f90 con 0x7ff34c06db20 2024-09-19T07:57:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff34ad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff34c0b3130 0x7ff34c0cc950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57994/0 (socket says 172.21.15.16:57994) 2024-09-19T07:57:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff34ad9d640 1 -- 172.21.15.16:0/2848797533 learned_addr learned my addr 172.21.15.16:0/2848797533 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff349d9b640 1 --2- 172.21.15.16:0/2848797533 >> v2:172.21.15.146:3300/0 conn(0x7ff34c06db20 0x7ff34c0b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff34959a640 1 -- 172.21.15.16:0/2848797533 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 201562402 0 0) 0x7ff34c0b38a0 con 0x7ff34c0b3130 2024-09-19T07:57:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff34959a640 1 -- 172.21.15.16:0/2848797533 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff334003520 con 0x7ff34c0b3130 2024-09-19T07:57:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff34959a640 1 -- 172.21.15.16:0/2848797533 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 991295844 0 0) 0x7ff334003520 con 0x7ff34c0b3130 2024-09-19T07:57:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff34959a640 1 -- 172.21.15.16:0/2848797533 >> v2:172.21.15.146:3300/0 conn(0x7ff34c06db20 msgr2=0x7ff34c0b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff34959a640 1 --2- 172.21.15.16:0/2848797533 >> v2:172.21.15.146:3300/0 conn(0x7ff34c06db20 0x7ff34c0b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff34959a640 1 -- 172.21.15.16:0/2848797533 >> v1:172.21.15.146:6789/0 conn(0x7ff34c161bb0 legacy=0x7ff34c06d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff34959a640 1 -- 172.21.15.16:0/2848797533 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c0d10d0 con 0x7ff34c0b3130 2024-09-19T07:57:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff34959a640 1 -- 172.21.15.16:0/2848797533 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7ff3440019f0 con 0x7ff34c0b3130 2024-09-19T07:57:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff34959a640 1 -- 172.21.15.16:0/2848797533 >> v1:172.21.15.16:6789/0 conn(0x7ff34c0b3130 legacy=0x7ff34c0cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff34959a640 1 --2- 172.21.15.16:0/2848797533 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff334003a00 0x7ff334003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff34959a640 1 --2- 172.21.15.16:0/2848797533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff334004800 0x7ff334004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff34959a640 1 --2- 172.21.15.16:0/2848797533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3340051c0 0x7ff33400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff34959a640 1 -- 172.21.15.16:0/2848797533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c161f90 con 0x7ff3340051c0 2024-09-19T07:57:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff34959a640 1 -- 172.21.15.16:0/2848797533 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff33400c0c0 con 0x7ff334004800 2024-09-19T07:57:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.154+0000 7ff34959a640 1 -- 172.21.15.16:0/2848797533 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff33400c2c0 con 0x7ff334003a00 2024-09-19T07:57:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.155+0000 7ff34ad9d640 1 --2- 172.21.15.16:0/2848797533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3340051c0 0x7ff33400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.155+0000 7ff349d9b640 1 --2- 172.21.15.16:0/2848797533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff334004800 0x7ff334004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.155+0000 7ff34a59c640 1 --2- 172.21.15.16:0/2848797533 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff334003a00 0x7ff334003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.155+0000 7ff34ad9d640 1 -- 172.21.15.16:0/2848797533 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff334003a00 msgr2=0x7ff334003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.155+0000 7ff34ad9d640 1 --2- 172.21.15.16:0/2848797533 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff334003a00 0x7ff334003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.155+0000 7ff34ad9d640 1 -- 172.21.15.16:0/2848797533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff334004800 msgr2=0x7ff334004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.155+0000 7ff34ad9d640 1 --2- 172.21.15.16:0/2848797533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff334004800 0x7ff334004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.155+0000 7ff34ad9d640 1 -- 172.21.15.16:0/2848797533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff334003520 con 0x7ff3340051c0 2024-09-19T07:57:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.156+0000 7ff34ad9d640 1 -- 172.21.15.16:0/2848797533 reap_dead start 2024-09-19T07:57:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.156+0000 7ff34ad9d640 1 --2- 172.21.15.16:0/2848797533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3340051c0 0x7ff33400b9c0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7ff34400a790 tx=0x7ff334005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e57ecd4439b93b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.156+0000 7ff34959a640 1 -- 172.21.15.16:0/2848797533 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34400a3b0 con 0x7ff3340051c0 2024-09-19T07:57:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.156+0000 7ff34959a640 1 -- 172.21.15.16:0/2848797533 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff34402f070 con 0x7ff3340051c0 2024-09-19T07:57:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.156+0000 7ff34959a640 1 -- 172.21.15.16:0/2848797533 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34402bc20 con 0x7ff3340051c0 2024-09-19T07:57:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.156+0000 7ff3509cd640 1 -- 172.21.15.16:0/2848797533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3340051c0 msgr2=0x7ff33400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.156+0000 7ff3509cd640 1 --2- 172.21.15.16:0/2848797533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3340051c0 0x7ff33400b9c0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7ff34400a790 tx=0x7ff334005830 comp rx=0 tx=0).stop 2024-09-19T07:57:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.156+0000 7ff3509cd640 1 -- 172.21.15.16:0/2848797533 shutdown_connections 2024-09-19T07:57:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.156+0000 7ff3509cd640 1 --2- 172.21.15.16:0/2848797533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3340051c0 0x7ff33400b9c0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.156+0000 7ff3509cd640 1 -- 172.21.15.16:0/2848797533 >> 172.21.15.16:0/2848797533 conn(0x7ff34c0b3df0 msgr2=0x7ff34c0b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.157+0000 7ff3509cd640 1 -- 172.21.15.16:0/2848797533 shutdown_connections 2024-09-19T07:57:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.157+0000 7ff3509cd640 1 -- 172.21.15.16:0/2848797533 wait complete. 2024-09-19T07:57:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.158+0000 7ff3509cd640 1 Processor -- start 2024-09-19T07:57:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.158+0000 7ff3509cd640 1 -- start start 2024-09-19T07:57:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff3509cd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3340051c0 0x7ff34c153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff3509cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34c154bf0 0x7ff34c0b5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff3509cd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff34c1545a0 0x7ff34c0b5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff3509cd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c0d1970 con 0x7ff34c154bf0 2024-09-19T07:57:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff3509cd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c0d1690 con 0x7ff3340051c0 2024-09-19T07:57:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff3509cd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff34c0b4910 con 0x7ff34c1545a0 2024-09-19T07:57:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff349d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34c154bf0 0x7ff34c0b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff349d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34c154bf0 0x7ff34c0b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44900/0 (socket says 172.21.15.16:44900) 2024-09-19T07:57:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff349d9b640 1 -- 172.21.15.16:0/3653289836 learned_addr learned my addr 172.21.15.16:0/3653289836 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff34ad9d640 1 --2- 172.21.15.16:0/3653289836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff34c1545a0 0x7ff34c0b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff34a59c640 1 --2- 172.21.15.16:0/3653289836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3340051c0 0x7ff34c153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff349d9b640 1 -- 172.21.15.16:0/3653289836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff34c1545a0 msgr2=0x7ff34c0b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff349d9b640 1 --2- 172.21.15.16:0/3653289836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff34c1545a0 0x7ff34c0b5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff349d9b640 1 -- 172.21.15.16:0/3653289836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3340051c0 msgr2=0x7ff34c153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff349d9b640 1 --2- 172.21.15.16:0/3653289836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3340051c0 0x7ff34c153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff349d9b640 1 -- 172.21.15.16:0/3653289836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c0cb780 con 0x7ff34c154bf0 2024-09-19T07:57:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff349d9b640 1 --2- 172.21.15.16:0/3653289836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34c154bf0 0x7ff34c0b5150 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7ff3400118e0 tx=0x7ff340011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff33affd640 1 -- 172.21.15.16:0/3653289836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34000faf0 con 0x7ff34c154bf0 2024-09-19T07:57:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff33affd640 1 -- 172.21.15.16:0/3653289836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff34000fc90 con 0x7ff34c154bf0 2024-09-19T07:57:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff33affd640 1 -- 172.21.15.16:0/3653289836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340014440 con 0x7ff34c154bf0 2024-09-19T07:57:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.159+0000 7ff3509cd640 1 -- 172.21.15.16:0/3653289836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34c0cba60 con 0x7ff34c154bf0 2024-09-19T07:57:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.160+0000 7ff3509cd640 1 -- 172.21.15.16:0/3653289836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34c154340 con 0x7ff34c154bf0 2024-09-19T07:57:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.161+0000 7ff3509cd640 1 -- 172.21.15.16:0/3653289836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff34c0b38a0 con 0x7ff34c154bf0 2024-09-19T07:57:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.161+0000 7ff33affd640 1 -- 172.21.15.16:0/3653289836 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff340012040 con 0x7ff34c154bf0 2024-09-19T07:57:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.162+0000 7ff33affd640 1 --2- 172.21.15.16:0/3653289836 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff32c06c2b0 0x7ff32c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.162+0000 7ff33affd640 1 -- 172.21.15.16:0/3653289836 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 6144+0+0 (secure 0 0 0) 0x7ff34008bcc0 con 0x7ff34c154bf0 2024-09-19T07:57:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.162+0000 7ff34a59c640 1 --2- 172.21.15.16:0/3653289836 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff32c06c2b0 0x7ff32c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.162+0000 7ff34a59c640 1 --2- 172.21.15.16:0/3653289836 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff32c06c2b0 0x7ff32c06e770 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff33c005590 tx=0x7ff33c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.166+0000 7ff33affd640 1 -- 172.21.15.16:0/3653289836 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff340019030 con 0x7ff34c154bf0 2024-09-19T07:57:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.340+0000 7ff3509cd640 1 -- 172.21.15.16:0/3653289836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff34c0b4550 con 0x7ff34c154bf0 2024-09-19T07:57:19.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.342+0000 7ff33affd640 1 -- 172.21.15.16:0/3653289836 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) ==== 74+0+17028 (secure 0 0 0) 0x7ff34005a310 con 0x7ff34c154bf0 2024-09-19T07:57:19.342 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:19.342 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":153,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:57:14.379755+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:55:44.728059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-09-19T07:55:45.128249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/2801864998":"2024-09-19T08:57:13.688556+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:57:19.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.344+0000 7ff3509cd640 1 -- 172.21.15.16:0/3653289836 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff32c06c2b0 msgr2=0x7ff32c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.344+0000 7ff3509cd640 1 --2- 172.21.15.16:0/3653289836 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff32c06c2b0 0x7ff32c06e770 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff33c005590 tx=0x7ff33c009040 comp rx=0 tx=0).stop 2024-09-19T07:57:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.344+0000 7ff3509cd640 1 -- 172.21.15.16:0/3653289836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34c154bf0 msgr2=0x7ff34c0b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.344+0000 7ff3509cd640 1 --2- 172.21.15.16:0/3653289836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34c154bf0 0x7ff34c0b5150 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7ff3400118e0 tx=0x7ff340011da0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.344+0000 7ff3509cd640 1 -- 172.21.15.16:0/3653289836 shutdown_connections 2024-09-19T07:57:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.344+0000 7ff3509cd640 1 --2- 172.21.15.16:0/3653289836 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7ff32c06c2b0 0x7ff32c06e770 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.344+0000 7ff3509cd640 1 --2- 172.21.15.16:0/3653289836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff34c1545a0 0x7ff34c0b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.344+0000 7ff3509cd640 1 --2- 172.21.15.16:0/3653289836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34c154bf0 0x7ff34c0b5150 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.344+0000 7ff3509cd640 1 --2- 172.21.15.16:0/3653289836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3340051c0 0x7ff34c153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.344+0000 7ff3509cd640 1 -- 172.21.15.16:0/3653289836 >> 172.21.15.16:0/3653289836 conn(0x7ff34c0b3df0 msgr2=0x7ff34c09aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.344+0000 7ff3509cd640 1 -- 172.21.15.16:0/3653289836 shutdown_connections 2024-09-19T07:57:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.344+0000 7ff3509cd640 1 -- 172.21.15.16:0/3653289836 wait complete. 2024-09-19T07:57:19.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T07:57:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.460+0000 7f153c8b7640 1 Processor -- start 2024-09-19T07:57:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.460+0000 7f153c8b7640 1 -- start start 2024-09-19T07:57:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.460+0000 7f153c8b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1538162290 0x7f15380cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f153c8b7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f153806cd80 0x7f1538162670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f153c8b7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1538162bb0 con 0x7f153806e090 2024-09-19T07:57:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f153c8b7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f153806d280 con 0x7f153806cd80 2024-09-19T07:57:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f153c8b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f153806d560 con 0x7f1538162290 2024-09-19T07:57:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f1536d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f153806e090 0x7f15380d2390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57998/0 (socket says 172.21.15.16:57998) 2024-09-19T07:57:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f1536d9d640 1 -- 172.21.15.16:0/201516858 learned_addr learned my addr 172.21.15.16:0/201516858 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f1535d9b640 1 -- 172.21.15.16:0/201516858 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 385572264 0 0) 0x7f1538162bb0 con 0x7f153806e090 2024-09-19T07:57:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f1535d9b640 1 -- 172.21.15.16:0/201516858 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1524003520 con 0x7f153806e090 2024-09-19T07:57:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f152dd9b640 1 --2- 172.21.15.16:0/201516858 >> v2:172.21.15.146:3300/0 conn(0x7f153806cd80 0x7f1538162670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f153659c640 1 --2- 172.21.15.16:0/201516858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1538162290 0x7f15380cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f1535d9b640 1 -- 172.21.15.16:0/201516858 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2757180614 0 0) 0x7f1524003520 con 0x7f153806e090 2024-09-19T07:57:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f1535d9b640 1 -- 172.21.15.16:0/201516858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1538162290 msgr2=0x7f15380cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f1535d9b640 1 --2- 172.21.15.16:0/201516858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1538162290 0x7f15380cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f1535d9b640 1 -- 172.21.15.16:0/201516858 >> v2:172.21.15.146:3300/0 conn(0x7f153806cd80 msgr2=0x7f1538162670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f1535d9b640 1 --2- 172.21.15.16:0/201516858 >> v2:172.21.15.146:3300/0 conn(0x7f153806cd80 0x7f1538162670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f1535d9b640 1 -- 172.21.15.16:0/201516858 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153806d840 con 0x7f153806e090 2024-09-19T07:57:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f1535d9b640 1 -- 172.21.15.16:0/201516858 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f15300019f0 con 0x7f153806e090 2024-09-19T07:57:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f153659c640 1 --2- 172.21.15.16:0/201516858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1538162290 0x7f15380cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f1535d9b640 1 -- 172.21.15.16:0/201516858 >> v1:172.21.15.16:6789/0 conn(0x7f153806e090 legacy=0x7f15380d2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f1535d9b640 1 --2- 172.21.15.16:0/201516858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1538162290 0x7f15240039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f1535d9b640 1 --2- 172.21.15.16:0/201516858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1524003fa0 0x7f1524004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f1535d9b640 1 --2- 172.21.15.16:0/201516858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1524004d40 0x7f15240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f1535d9b640 1 -- 172.21.15.16:0/201516858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f153806d280 con 0x7f1524004d40 2024-09-19T07:57:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f1535d9b640 1 -- 172.21.15.16:0/201516858 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f153806d560 con 0x7f1524003fa0 2024-09-19T07:57:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f1535d9b640 1 -- 172.21.15.16:0/201516858 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f152400bb20 con 0x7f1538162290 2024-09-19T07:57:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f152dd9b640 1 --2- 172.21.15.16:0/201516858 >> v2:172.21.15.146:3300/0 conn(0x7f153806cd80 0x7f1538162670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.461+0000 7f1536d9d640 1 --2- 172.21.15.16:0/201516858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1524004d40 0x7f15240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.462+0000 7f153659c640 1 --2- 172.21.15.16:0/201516858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1538162290 0x7f15240039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.462+0000 7f152dd9b640 1 --2- 172.21.15.16:0/201516858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1524003fa0 0x7f1524004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.462+0000 7f1536d9d640 1 -- 172.21.15.16:0/201516858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1538162290 msgr2=0x7f15240039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.462+0000 7f1536d9d640 1 --2- 172.21.15.16:0/201516858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1538162290 0x7f15240039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.462+0000 7f1536d9d640 1 -- 172.21.15.16:0/201516858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1524003fa0 msgr2=0x7f1524004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.462+0000 7f1536d9d640 1 --2- 172.21.15.16:0/201516858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1524003fa0 0x7f1524004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.462+0000 7f1536d9d640 1 -- 172.21.15.16:0/201516858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1524003520 con 0x7f1524004d40 2024-09-19T07:57:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.462+0000 7f1536d9d640 1 --2- 172.21.15.16:0/201516858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1524004d40 0x7f15240051c0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f153000a840 tx=0x7f1524005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccb87501916d227 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.463+0000 7f1535d9b640 1 -- 172.21.15.16:0/201516858 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153000a460 con 0x7f1524004d40 2024-09-19T07:57:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.463+0000 7f1535d9b640 1 -- 172.21.15.16:0/201516858 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f153002f070 con 0x7f1524004d40 2024-09-19T07:57:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.463+0000 7f1535d9b640 1 -- 172.21.15.16:0/201516858 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153002bcf0 con 0x7f1524004d40 2024-09-19T07:57:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.463+0000 7f153c8b7640 1 -- 172.21.15.16:0/201516858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1524004d40 msgr2=0x7f15240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.463+0000 7f153c8b7640 1 --2- 172.21.15.16:0/201516858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1524004d40 0x7f15240051c0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f153000a840 tx=0x7f1524005830 comp rx=0 tx=0).stop 2024-09-19T07:57:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.463+0000 7f1536d9d640 1 -- 172.21.15.16:0/201516858 reap_dead start 2024-09-19T07:57:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.463+0000 7f153c8b7640 1 -- 172.21.15.16:0/201516858 shutdown_connections 2024-09-19T07:57:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.463+0000 7f153c8b7640 1 -- 172.21.15.16:0/201516858 >> 172.21.15.16:0/201516858 conn(0x7f153808bfe0 msgr2=0x7f15380d4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.463+0000 7f153c8b7640 1 -- 172.21.15.16:0/201516858 shutdown_connections 2024-09-19T07:57:19.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.463+0000 7f153c8b7640 1 -- 172.21.15.16:0/201516858 wait complete. 2024-09-19T07:57:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.465+0000 7f153c8b7640 1 Processor -- start 2024-09-19T07:57:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.465+0000 7f153c8b7640 1 -- start start 2024-09-19T07:57:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.465+0000 7f153c8b7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1524004d40 0x7f15381575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.465+0000 7f153c8b7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1538156500 0x7f15380cff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.465+0000 7f153c8b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15380d04b0 0x7f1538159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.465+0000 7f153c8b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1538162040 con 0x7f15380d04b0 2024-09-19T07:57:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.465+0000 7f153c8b7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1538161ec0 con 0x7f1524004d40 2024-09-19T07:57:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.465+0000 7f153c8b7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f153806e540 con 0x7f1538156500 2024-09-19T07:57:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.465+0000 7f1536d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15380d04b0 0x7f1538159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.465+0000 7f1536d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15380d04b0 0x7f1538159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44912/0 (socket says 172.21.15.16:44912) 2024-09-19T07:57:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.465+0000 7f1536d9d640 1 -- 172.21.15.16:0/3008555916 learned_addr learned my addr 172.21.15.16:0/3008555916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.466+0000 7f153659c640 1 --2- 172.21.15.16:0/3008555916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1524004d40 0x7f15381575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.466+0000 7f152dd9b640 1 --2- 172.21.15.16:0/3008555916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1538156500 0x7f15380cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.466+0000 7f1536d9d640 1 -- 172.21.15.16:0/3008555916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1538156500 msgr2=0x7f15380cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.466+0000 7f1536d9d640 1 --2- 172.21.15.16:0/3008555916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1538156500 0x7f15380cff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.466+0000 7f1536d9d640 1 -- 172.21.15.16:0/3008555916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1524004d40 msgr2=0x7f15381575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.466+0000 7f1536d9d640 1 --2- 172.21.15.16:0/3008555916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1524004d40 0x7f15381575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.466+0000 7f1536d9d640 1 -- 172.21.15.16:0/3008555916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153815f430 con 0x7f15380d04b0 2024-09-19T07:57:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.466+0000 7f1536d9d640 1 --2- 172.21.15.16:0/3008555916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15380d04b0 0x7f1538159b90 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f153000a960 tx=0x7f153806de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.466+0000 7f152f7fe640 1 -- 172.21.15.16:0/3008555916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153000a460 con 0x7f15380d04b0 2024-09-19T07:57:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.466+0000 7f152f7fe640 1 -- 172.21.15.16:0/3008555916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1530004b60 con 0x7f15380d04b0 2024-09-19T07:57:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.466+0000 7f153c8b7640 1 -- 172.21.15.16:0/3008555916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153815a780 con 0x7f15380d04b0 2024-09-19T07:57:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.466+0000 7f152f7fe640 1 -- 172.21.15.16:0/3008555916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530001fb0 con 0x7f15380d04b0 2024-09-19T07:57:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.466+0000 7f153c8b7640 1 -- 172.21.15.16:0/3008555916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f153815f9b0 con 0x7f15380d04b0 2024-09-19T07:57:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.467+0000 7f153c8b7640 1 -- 172.21.15.16:0/3008555916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15380d0930 con 0x7f15380d04b0 2024-09-19T07:57:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.468+0000 7f152f7fe640 1 -- 172.21.15.16:0/3008555916 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1530046060 con 0x7f15380d04b0 2024-09-19T07:57:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.468+0000 7f152f7fe640 1 --2- 172.21.15.16:0/3008555916 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f151806c2b0 0x7f151806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.469+0000 7f152f7fe640 1 -- 172.21.15.16:0/3008555916 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 6144+0+0 (secure 0 0 0) 0x7f15300ab3b0 con 0x7f15380d04b0 2024-09-19T07:57:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.469+0000 7f153659c640 1 --2- 172.21.15.16:0/3008555916 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f151806c2b0 0x7f151806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.469+0000 7f153659c640 1 --2- 172.21.15.16:0/3008555916 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f151806c2b0 0x7f151806e770 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1528006fa0 tx=0x7f1528008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.473+0000 7f152f7fe640 1 -- 172.21.15.16:0/3008555916 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1530079290 con 0x7f15380d04b0 2024-09-19T07:57:19.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.685+0000 7f153c8b7640 1 -- 172.21.15.16:0/3008555916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f15380cf1e0 con 0x7f15380d04b0 2024-09-19T07:57:19.686 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-19T07:57:19.686+0000 7f6950fea640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:57:19.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.687+0000 7f152f7fe640 1 -- 172.21.15.16:0/3008555916 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v95) ==== 161+0+0 (secure 0 0 0) 0x7f153007d590 con 0x7f15380d04b0 2024-09-19T07:57:19.687 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T07:57:19.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.690+0000 7f153c8b7640 1 -- 172.21.15.16:0/3008555916 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f151806c2b0 msgr2=0x7f151806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:19.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.690+0000 7f153c8b7640 1 --2- 172.21.15.16:0/3008555916 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f151806c2b0 0x7f151806e770 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1528006fa0 tx=0x7f1528008040 comp rx=0 tx=0).stop 2024-09-19T07:57:19.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.690+0000 7f153c8b7640 1 -- 172.21.15.16:0/3008555916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15380d04b0 msgr2=0x7f1538159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.690+0000 7f153c8b7640 1 --2- 172.21.15.16:0/3008555916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15380d04b0 0x7f1538159b90 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f153000a960 tx=0x7f153806de00 comp rx=0 tx=0).stop 2024-09-19T07:57:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.690+0000 7f153c8b7640 1 -- 172.21.15.16:0/3008555916 shutdown_connections 2024-09-19T07:57:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.690+0000 7f153c8b7640 1 --2- 172.21.15.16:0/3008555916 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f151806c2b0 0x7f151806e770 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.690+0000 7f153c8b7640 1 --2- 172.21.15.16:0/3008555916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15380d04b0 0x7f1538159b90 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.690+0000 7f153c8b7640 1 --2- 172.21.15.16:0/3008555916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1538156500 0x7f15380cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.690+0000 7f153c8b7640 1 --2- 172.21.15.16:0/3008555916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1524004d40 0x7f15381575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.690+0000 7f153c8b7640 1 -- 172.21.15.16:0/3008555916 >> 172.21.15.16:0/3008555916 conn(0x7f153808bfe0 msgr2=0x7f15380d2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.690+0000 7f153c8b7640 1 -- 172.21.15.16:0/3008555916 shutdown_connections 2024-09-19T07:57:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.690+0000 7f153c8b7640 1 -- 172.21.15.16:0/3008555916 wait complete. 2024-09-19T07:57:19.693 INFO:tasks.ceph.mds.c.smithi016.stderr: -6562> 2024-09-19T07:55:48.526+0000 7fd5975ff640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T07:57:19.702 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T07:57:19.724 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-19T07:57:19.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.807+0000 7f890330a640 1 Processor -- start 2024-09-19T07:57:19.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.807+0000 7f890330a640 1 -- start start 2024-09-19T07:57:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.808+0000 7f890330a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f88fc09aa50 0x7f88fc09ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.808+0000 7f890330a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88fc09b6a0 0x7f88fc0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.808+0000 7f890330a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88fc0cd850 con 0x7f88fc0b27c0 2024-09-19T07:57:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.808+0000 7f890330a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f88fc0b1df0 con 0x7f88fc09aa50 2024-09-19T07:57:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.808+0000 7f890330a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f88fc15c7c0 con 0x7f88fc09b6a0 2024-09-19T07:57:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.808+0000 7f89008a5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88fc09b6a0 0x7f88fc0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.808+0000 7f89010a6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f88fc09aa50 0x7f88fc09ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.808+0000 7f89008a5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88fc09b6a0 0x7f88fc0b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.16:57932/0 (socket says 172.21.15.16:57932) 2024-09-19T07:57:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.808+0000 7f89008a5640 1 -- 172.21.15.16:0/593489223 learned_addr learned my addr 172.21.15.16:0/593489223 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.809+0000 7f89008a5640 1 -- 172.21.15.16:0/593489223 >> v2:172.21.15.146:3300/0 conn(0x7f88fc09aa50 msgr2=0x7f88fc09ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.809+0000 7f89008a5640 1 --2- 172.21.15.16:0/593489223 >> v2:172.21.15.146:3300/0 conn(0x7f88fc09aa50 0x7f88fc09ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.809+0000 7f89008a5640 1 -- 172.21.15.16:0/593489223 >> v1:172.21.15.146:6789/0 conn(0x7f88fc0b27c0 legacy=0x7f88fc0b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.809+0000 7f89008a5640 1 -- 172.21.15.16:0/593489223 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88fc09b370 con 0x7f88fc09b6a0 2024-09-19T07:57:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.810+0000 7f89008a5640 1 --2- 172.21.15.16:0/593489223 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88fc09b6a0 0x7f88fc0b2280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009870 tx=0x7f88ec02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=452e0a200e7e9acf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.810+0000 7f88f3fff640 1 -- 172.21.15.16:0/593489223 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec037070 con 0x7f88fc09b6a0 2024-09-19T07:57:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.810+0000 7f88f3fff640 1 -- 172.21.15.16:0/593489223 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88ec032440 con 0x7f88fc09b6a0 2024-09-19T07:57:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.810+0000 7f88f3fff640 1 -- 172.21.15.16:0/593489223 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec032810 con 0x7f88fc09b6a0 2024-09-19T07:57:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.810+0000 7f890330a640 1 -- 172.21.15.16:0/593489223 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88fc09b6a0 msgr2=0x7f88fc0b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.810+0000 7f890330a640 1 --2- 172.21.15.16:0/593489223 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88fc09b6a0 0x7f88fc0b2280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009870 tx=0x7f88ec02a270 comp rx=0 tx=0).stop 2024-09-19T07:57:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.810+0000 7f890330a640 1 -- 172.21.15.16:0/593489223 shutdown_connections 2024-09-19T07:57:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.810+0000 7f890330a640 1 --2- 172.21.15.16:0/593489223 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88fc09b6a0 0x7f88fc0b2280 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.810+0000 7f890330a640 1 --2- 172.21.15.16:0/593489223 >> v2:172.21.15.146:3300/0 conn(0x7f88fc09aa50 0x7f88fc09ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.810+0000 7f890330a640 1 -- 172.21.15.16:0/593489223 >> 172.21.15.16:0/593489223 conn(0x7f88fc0d22f0 msgr2=0x7f88fc0d2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.810+0000 7f890330a640 1 -- 172.21.15.16:0/593489223 shutdown_connections 2024-09-19T07:57:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.810+0000 7f890330a640 1 -- 172.21.15.16:0/593489223 wait complete. 2024-09-19T07:57:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.812+0000 7f890330a640 1 Processor -- start 2024-09-19T07:57:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.812+0000 7f890330a640 1 -- start start 2024-09-19T07:57:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.812+0000 7f890330a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88fc09aa50 0x7f88fc1526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.812+0000 7f890330a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88fc09b6a0 0x7f88fc1f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.812+0000 7f890330a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88fc0b27c0 0x7f88fc1e3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.812+0000 7f890330a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc0ce090 con 0x7f88fc09aa50 2024-09-19T07:57:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.812+0000 7f890330a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc0cddb0 con 0x7f88fc09b6a0 2024-09-19T07:57:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.812+0000 7f890330a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f88fc0b1a80 con 0x7f88fc0b27c0 2024-09-19T07:57:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.812+0000 7f89010a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88fc09aa50 0x7f88fc1526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.813+0000 7f89010a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88fc09aa50 0x7f88fc1526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44932/0 (socket says 172.21.15.16:44932) 2024-09-19T07:57:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.813+0000 7f89010a6640 1 -- 172.21.15.16:0/1736432954 learned_addr learned my addr 172.21.15.16:0/1736432954 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.813+0000 7f89008a5640 1 --2- 172.21.15.16:0/1736432954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88fc09b6a0 0x7f88fc1f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.813+0000 7f89018a7640 1 --2- 172.21.15.16:0/1736432954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88fc0b27c0 0x7f88fc1e3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.813+0000 7f89010a6640 1 -- 172.21.15.16:0/1736432954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88fc0b27c0 msgr2=0x7f88fc1e3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.813+0000 7f89010a6640 1 --2- 172.21.15.16:0/1736432954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88fc0b27c0 0x7f88fc1e3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.813+0000 7f89010a6640 1 -- 172.21.15.16:0/1736432954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88fc09b6a0 msgr2=0x7f88fc1f1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.813+0000 7f89010a6640 1 --2- 172.21.15.16:0/1736432954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88fc09b6a0 0x7f88fc1f1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.813+0000 7f89010a6640 1 -- 172.21.15.16:0/1736432954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88fc152ef0 con 0x7f88fc09aa50 2024-09-19T07:57:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.813+0000 7f89010a6640 1 --2- 172.21.15.16:0/1736432954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88fc09aa50 0x7f88fc1526a0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f88f800e8a0 tx=0x7f88f800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.813+0000 7f88f1ffb640 1 -- 172.21.15.16:0/1736432954 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8017070 con 0x7f88fc09aa50 2024-09-19T07:57:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.813+0000 7f88f1ffb640 1 -- 172.21.15.16:0/1736432954 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88f800c940 con 0x7f88fc09aa50 2024-09-19T07:57:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.813+0000 7f890330a640 1 -- 172.21.15.16:0/1736432954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88fc1531d0 con 0x7f88fc09aa50 2024-09-19T07:57:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.813+0000 7f88f1ffb640 1 -- 172.21.15.16:0/1736432954 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f800cd30 con 0x7f88fc09aa50 2024-09-19T07:57:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.813+0000 7f890330a640 1 -- 172.21.15.16:0/1736432954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88fc153790 con 0x7f88fc09aa50 2024-09-19T07:57:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.815+0000 7f890330a640 1 -- 172.21.15.16:0/1736432954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88fc1539d0 con 0x7f88fc09aa50 2024-09-19T07:57:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.815+0000 7f88f1ffb640 1 -- 172.21.15.16:0/1736432954 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f88f801a070 con 0x7f88fc09aa50 2024-09-19T07:57:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.815+0000 7f88f1ffb640 1 --2- 172.21.15.16:0/1736432954 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f88e006c2b0 0x7f88e006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.816+0000 7f88f1ffb640 1 -- 172.21.15.16:0/1736432954 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 6230+0+0 (secure 0 0 0) 0x7f88f808b750 con 0x7f88fc09aa50 2024-09-19T07:57:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.816+0000 7f89008a5640 1 --2- 172.21.15.16:0/1736432954 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f88e006c2b0 0x7f88e006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.816+0000 7f89008a5640 1 --2- 172.21.15.16:0/1736432954 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f88e006c2b0 0x7f88e006e770 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f88ec034000 tx=0x7f88ec02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:19.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:19.820+0000 7f88f1ffb640 1 -- 172.21.15.16:0/1736432954 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f88fc1539d0 con 0x7f88fc09aa50 2024-09-19T07:57:20.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.031+0000 7f890330a640 1 -- 172.21.15.16:0/1736432954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f88fc09ae30 con 0x7f88fc09aa50 2024-09-19T07:57:20.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.688+0000 7f88f1ffb640 1 -- 172.21.15.16:0/1736432954 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v96) ==== 104+0+0 (secure 0 0 0) 0x7f88f80596f0 con 0x7f88fc09aa50 2024-09-19T07:57:20.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.691+0000 7f890330a640 1 -- 172.21.15.16:0/1736432954 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f88e006c2b0 msgr2=0x7f88e006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.691+0000 7f890330a640 1 --2- 172.21.15.16:0/1736432954 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f88e006c2b0 0x7f88e006e770 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f88ec034000 tx=0x7f88ec02e000 comp rx=0 tx=0).stop 2024-09-19T07:57:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.691+0000 7f890330a640 1 -- 172.21.15.16:0/1736432954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88fc09aa50 msgr2=0x7f88fc1526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.691+0000 7f890330a640 1 --2- 172.21.15.16:0/1736432954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88fc09aa50 0x7f88fc1526a0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f88f800e8a0 tx=0x7f88f800ed60 comp rx=0 tx=0).stop 2024-09-19T07:57:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.691+0000 7f890330a640 1 -- 172.21.15.16:0/1736432954 shutdown_connections 2024-09-19T07:57:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.691+0000 7f890330a640 1 --2- 172.21.15.16:0/1736432954 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f88e006c2b0 0x7f88e006e770 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.691+0000 7f890330a640 1 --2- 172.21.15.16:0/1736432954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88fc0b27c0 0x7f88fc1e3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.691+0000 7f890330a640 1 --2- 172.21.15.16:0/1736432954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88fc09b6a0 0x7f88fc1f1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.691+0000 7f890330a640 1 --2- 172.21.15.16:0/1736432954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88fc09aa50 0x7f88fc1526a0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.691+0000 7f890330a640 1 -- 172.21.15.16:0/1736432954 >> 172.21.15.16:0/1736432954 conn(0x7f88fc0d22f0 msgr2=0x7f88fc0b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.691+0000 7f890330a640 1 -- 172.21.15.16:0/1736432954 shutdown_connections 2024-09-19T07:57:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.691+0000 7f890330a640 1 -- 172.21.15.16:0/1736432954 wait complete. 2024-09-19T07:57:20.706 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T07:57:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.861+0000 7f9ff2684640 1 Processor -- start 2024-09-19T07:57:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.861+0000 7f9ff2684640 1 -- start start 2024-09-19T07:57:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.862+0000 7f9ff2684640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9fec072060 0x7f9fec0cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.862+0000 7f9ff2684640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9fec0ca8c0 0x7f9fec072440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.862+0000 7f9ff2684640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fec15aee0 con 0x7f9fec072980 2024-09-19T07:57:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.862+0000 7f9ff2684640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9fec0734a0 con 0x7f9fec0ca8c0 2024-09-19T07:57:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.862+0000 7f9ff2684640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f9fec0cb6a0 con 0x7f9fec072060 2024-09-19T07:57:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.862+0000 7f9feb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9fec0ca8c0 0x7f9fec072440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.862+0000 7f9ff0c21640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9fec072980 0x7f9fec0ce480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58012/0 (socket says 172.21.15.16:58012) 2024-09-19T07:57:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.862+0000 7f9ff0c21640 1 -- 172.21.15.16:0/86995159 learned_addr learned my addr 172.21.15.16:0/86995159 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.862+0000 7f9febfff640 1 --2- 172.21.15.16:0/86995159 >> v2:172.21.15.146:3300/0 conn(0x7f9fec072060 0x7f9fec0cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.862+0000 7f9feb7fe640 1 -- 172.21.15.16:0/86995159 >> v2:172.21.15.146:3300/0 conn(0x7f9fec072060 msgr2=0x7f9fec0cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.862+0000 7f9feb7fe640 1 --2- 172.21.15.16:0/86995159 >> v2:172.21.15.146:3300/0 conn(0x7f9fec072060 0x7f9fec0cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.862+0000 7f9feb7fe640 1 -- 172.21.15.16:0/86995159 >> v1:172.21.15.16:6789/0 conn(0x7f9fec072980 legacy=0x7f9fec0ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.863+0000 7f9feb7fe640 1 -- 172.21.15.16:0/86995159 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fec072e50 con 0x7f9fec0ca8c0 2024-09-19T07:57:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.862+0000 7f9feaffd640 1 -- 172.21.15.16:0/86995159 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3581787546 0 0) 0x7f9fec15aee0 con 0x7f9fec072980 2024-09-19T07:57:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.863+0000 7f9feb7fe640 1 --2- 172.21.15.16:0/86995159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fec0ca8c0 0x7f9fec072440 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f9fec06ff10 tx=0x7f9fe002c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c1e0a3be01f82d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.863+0000 7f9feaffd640 1 -- 172.21.15.16:0/86995159 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe003a070 con 0x7f9fec0ca8c0 2024-09-19T07:57:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.863+0000 7f9feaffd640 1 -- 172.21.15.16:0/86995159 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe0032480 con 0x7f9fec0ca8c0 2024-09-19T07:57:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.863+0000 7f9feaffd640 1 -- 172.21.15.16:0/86995159 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe002ce60 con 0x7f9fec0ca8c0 2024-09-19T07:57:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.864+0000 7f9ff2684640 1 -- 172.21.15.16:0/86995159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fec0ca8c0 msgr2=0x7f9fec072440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.864+0000 7f9ff2684640 1 --2- 172.21.15.16:0/86995159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fec0ca8c0 0x7f9fec072440 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f9fec06ff10 tx=0x7f9fe002c640 comp rx=0 tx=0).stop 2024-09-19T07:57:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.864+0000 7f9ff2684640 1 -- 172.21.15.16:0/86995159 shutdown_connections 2024-09-19T07:57:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.864+0000 7f9ff2684640 1 --2- 172.21.15.16:0/86995159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fec0ca8c0 0x7f9fec072440 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.864+0000 7f9ff2684640 1 --2- 172.21.15.16:0/86995159 >> v2:172.21.15.146:3300/0 conn(0x7f9fec072060 0x7f9fec0cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.864+0000 7f9ff2684640 1 -- 172.21.15.16:0/86995159 >> 172.21.15.16:0/86995159 conn(0x7f9fec0b1dd0 msgr2=0x7f9fec0b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.864+0000 7f9ff2684640 1 -- 172.21.15.16:0/86995159 shutdown_connections 2024-09-19T07:57:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.864+0000 7f9ff2684640 1 -- 172.21.15.16:0/86995159 wait complete. 2024-09-19T07:57:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.865+0000 7f9ff2684640 1 Processor -- start 2024-09-19T07:57:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.865+0000 7f9ff2684640 1 -- start start 2024-09-19T07:57:20.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.866+0000 7f9ff2684640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fec072060 0x7f9fec159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:20.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.866+0000 7f9ff2684640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9fec072980 0x7f9fec15d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:20.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.866+0000 7f9ff2684640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fec0ca8c0 0x7f9fec15d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:20.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.866+0000 7f9ff2684640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec0cfcf0 con 0x7f9fec0ca8c0 2024-09-19T07:57:20.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.866+0000 7f9ff2684640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec073ef0 con 0x7f9fec072060 2024-09-19T07:57:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.866+0000 7f9ff2684640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9fec073930 con 0x7f9fec072980 2024-09-19T07:57:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.866+0000 7f9ff0c21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fec0ca8c0 0x7f9fec15d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.866+0000 7f9ff0c21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fec0ca8c0 0x7f9fec15d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44952/0 (socket says 172.21.15.16:44952) 2024-09-19T07:57:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.866+0000 7f9ff0c21640 1 -- 172.21.15.16:0/3461681826 learned_addr learned my addr 172.21.15.16:0/3461681826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.866+0000 7f9febfff640 1 --2- 172.21.15.16:0/3461681826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fec072060 0x7f9fec159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.867+0000 7f9feb7fe640 1 --2- 172.21.15.16:0/3461681826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9fec072980 0x7f9fec15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.867+0000 7f9ff0c21640 1 -- 172.21.15.16:0/3461681826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9fec072980 msgr2=0x7f9fec15d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.867+0000 7f9ff0c21640 1 --2- 172.21.15.16:0/3461681826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9fec072980 0x7f9fec15d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.867+0000 7f9ff0c21640 1 -- 172.21.15.16:0/3461681826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fec072060 msgr2=0x7f9fec159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.867+0000 7f9ff0c21640 1 --2- 172.21.15.16:0/3461681826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fec072060 0x7f9fec159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.867+0000 7f9ff0c21640 1 -- 172.21.15.16:0/3461681826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fec15bc80 con 0x7f9fec0ca8c0 2024-09-19T07:57:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.867+0000 7f9ff0c21640 1 --2- 172.21.15.16:0/3461681826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fec0ca8c0 0x7f9fec15d8a0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400c730 tx=0x7f9fe400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.867+0000 7f9fe8ff9640 1 -- 172.21.15.16:0/3461681826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4018070 con 0x7f9fec0ca8c0 2024-09-19T07:57:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.867+0000 7f9fe8ff9640 1 -- 172.21.15.16:0/3461681826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe400ce40 con 0x7f9fec0ca8c0 2024-09-19T07:57:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.867+0000 7f9ff2684640 1 -- 172.21.15.16:0/3461681826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fec15bf60 con 0x7f9fec0ca8c0 2024-09-19T07:57:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.867+0000 7f9fe8ff9640 1 -- 172.21.15.16:0/3461681826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe400a860 con 0x7f9fec0ca8c0 2024-09-19T07:57:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.867+0000 7f9ff2684640 1 -- 172.21.15.16:0/3461681826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fec15c520 con 0x7f9fec0ca8c0 2024-09-19T07:57:20.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.868+0000 7f9ff2684640 1 -- 172.21.15.16:0/3461681826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fec0ae3a0 con 0x7f9fec0ca8c0 2024-09-19T07:57:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.869+0000 7f9fe8ff9640 1 -- 172.21.15.16:0/3461681826 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9fe401b070 con 0x7f9fec0ca8c0 2024-09-19T07:57:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.869+0000 7f9fe8ff9640 1 --2- 172.21.15.16:0/3461681826 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9fd806c2b0 0x7f9fd806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.869+0000 7f9fe8ff9640 1 -- 172.21.15.16:0/3461681826 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 6230+0+0 (secure 0 0 0) 0x7f9fe408aba0 con 0x7f9fec0ca8c0 2024-09-19T07:57:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.870+0000 7f9febfff640 1 --2- 172.21.15.16:0/3461681826 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9fd806c2b0 0x7f9fd806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.870+0000 7f9febfff640 1 --2- 172.21.15.16:0/3461681826 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9fd806c2b0 0x7f9fd806e770 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc006fd0 tx=0x7f9fdc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:20.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:20.873+0000 7f9fe8ff9640 1 -- 172.21.15.16:0/3461681826 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9fe4058ab0 con 0x7f9fec0ca8c0 2024-09-19T07:57:21.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.048+0000 7f9ff2684640 1 -- 172.21.15.16:0/3461681826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9fec0cd600 con 0x7f9fec0ca8c0 2024-09-19T07:57:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.690+0000 7f9fe8ff9640 1 -- 172.21.15.16:0/3461681826 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v155) ==== 181+0+0 (secure 0 0 0) 0x7f9fe405cdb0 con 0x7f9fec0ca8c0 2024-09-19T07:57:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-19T07:57:21.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.693+0000 7f9ff2684640 1 -- 172.21.15.16:0/3461681826 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9fd806c2b0 msgr2=0x7f9fd806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:21.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.693+0000 7f9ff2684640 1 --2- 172.21.15.16:0/3461681826 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9fd806c2b0 0x7f9fd806e770 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc006fd0 tx=0x7f9fdc008040 comp rx=0 tx=0).stop 2024-09-19T07:57:21.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.693+0000 7f9ff2684640 1 -- 172.21.15.16:0/3461681826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fec0ca8c0 msgr2=0x7f9fec15d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:21.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.693+0000 7f9ff2684640 1 --2- 172.21.15.16:0/3461681826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fec0ca8c0 0x7f9fec15d8a0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400c730 tx=0x7f9fe400cbf0 comp rx=0 tx=0).stop 2024-09-19T07:57:21.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.694+0000 7f9ff2684640 1 -- 172.21.15.16:0/3461681826 shutdown_connections 2024-09-19T07:57:21.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.694+0000 7f9ff2684640 1 --2- 172.21.15.16:0/3461681826 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9fd806c2b0 0x7f9fd806e770 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:21.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.694+0000 7f9ff2684640 1 --2- 172.21.15.16:0/3461681826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fec0ca8c0 0x7f9fec15d8a0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:21.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.694+0000 7f9ff2684640 1 --2- 172.21.15.16:0/3461681826 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9fec072980 0x7f9fec15d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:21.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.694+0000 7f9ff2684640 1 --2- 172.21.15.16:0/3461681826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fec072060 0x7f9fec159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:21.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.694+0000 7f9ff2684640 1 -- 172.21.15.16:0/3461681826 >> 172.21.15.16:0/3461681826 conn(0x7f9fec0b1dd0 msgr2=0x7f9fec0aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:21.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.694+0000 7f9ff2684640 1 -- 172.21.15.16:0/3461681826 shutdown_connections 2024-09-19T07:57:21.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.694+0000 7f9ff2684640 1 -- 172.21.15.16:0/3461681826 wait complete. 2024-09-19T07:57:21.709 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T07:57:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.823+0000 7fa62352c640 1 Processor -- start 2024-09-19T07:57:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.823+0000 7fa62352c640 1 -- start start 2024-09-19T07:57:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.824+0000 7fa62352c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa61c15e450 0x7fa61c06f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.824+0000 7fa62352c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa61c0b3740 0x7fa61c0b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.824+0000 7fa62352c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa61c070140 0x7fa61c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.824+0000 7fa62352c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fa61c15e830 con 0x7fa61c070140 2024-09-19T07:57:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.824+0000 7fa62352c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa61c06e440 con 0x7fa61c0b3740 2024-09-19T07:57:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.824+0000 7fa62352c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa61c06e720 con 0x7fa61c15e450 2024-09-19T07:57:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.824+0000 7fa621ac9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa61c070140 0x7fa61c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.824+0000 7fa621ac9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa61c070140 0x7fa61c0705f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44962/0 (socket says 172.21.15.16:44962) 2024-09-19T07:57:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.824+0000 7fa621ac9640 1 -- 172.21.15.16:0/2962160190 learned_addr learned my addr 172.21.15.16:0/2962160190 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.824+0000 7fa620ac7640 1 --2- 172.21.15.16:0/2962160190 >> v2:172.21.15.146:3300/0 conn(0x7fa61c0b3740 0x7fa61c0b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.824+0000 7fa6212c8640 1 --2- 172.21.15.16:0/2962160190 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa61c15e450 0x7fa61c06f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.824+0000 7fa621ac9640 1 -- 172.21.15.16:0/2962160190 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa61c15e450 msgr2=0x7fa61c06f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.824+0000 7fa621ac9640 1 --2- 172.21.15.16:0/2962160190 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa61c15e450 0x7fa61c06f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.824+0000 7fa621ac9640 1 -- 172.21.15.16:0/2962160190 >> v2:172.21.15.146:3300/0 conn(0x7fa61c0b3740 msgr2=0x7fa61c0b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.824+0000 7fa621ac9640 1 --2- 172.21.15.16:0/2962160190 >> v2:172.21.15.146:3300/0 conn(0x7fa61c0b3740 0x7fa61c0b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.824+0000 7fa621ac9640 1 -- 172.21.15.16:0/2962160190 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c06ea00 con 0x7fa61c070140 2024-09-19T07:57:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.824+0000 7fa620ac7640 1 --2- 172.21.15.16:0/2962160190 >> v2:172.21.15.146:3300/0 conn(0x7fa61c0b3740 0x7fa61c0b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:57:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.825+0000 7fa621ac9640 1 --2- 172.21.15.16:0/2962160190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa61c070140 0x7fa61c0705f0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0690f0 tx=0x7fa61402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3140ffa682e83e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.825+0000 7fa613fff640 1 -- 172.21.15.16:0/2962160190 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614036070 con 0x7fa61c070140 2024-09-19T07:57:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.825+0000 7fa613fff640 1 -- 172.21.15.16:0/2962160190 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa614031600 con 0x7fa61c070140 2024-09-19T07:57:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.825+0000 7fa613fff640 1 -- 172.21.15.16:0/2962160190 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6140319d0 con 0x7fa61c070140 2024-09-19T07:57:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.826+0000 7fa62352c640 1 -- 172.21.15.16:0/2962160190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa61c070140 msgr2=0x7fa61c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.826+0000 7fa62352c640 1 --2- 172.21.15.16:0/2962160190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa61c070140 0x7fa61c0705f0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0690f0 tx=0x7fa61402a6a0 comp rx=0 tx=0).stop 2024-09-19T07:57:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.826+0000 7fa62352c640 1 -- 172.21.15.16:0/2962160190 shutdown_connections 2024-09-19T07:57:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.826+0000 7fa62352c640 1 --2- 172.21.15.16:0/2962160190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa61c070140 0x7fa61c0705f0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.826+0000 7fa62352c640 1 --2- 172.21.15.16:0/2962160190 >> v2:172.21.15.146:3300/0 conn(0x7fa61c0b3740 0x7fa61c0b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.826+0000 7fa62352c640 1 --2- 172.21.15.16:0/2962160190 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa61c15e450 0x7fa61c06f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.826+0000 7fa62352c640 1 -- 172.21.15.16:0/2962160190 >> 172.21.15.16:0/2962160190 conn(0x7fa61c0b23b0 msgr2=0x7fa61c0b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.826+0000 7fa62352c640 1 -- 172.21.15.16:0/2962160190 shutdown_connections 2024-09-19T07:57:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.826+0000 7fa62352c640 1 -- 172.21.15.16:0/2962160190 wait complete. 2024-09-19T07:57:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.827+0000 7fa62352c640 1 Processor -- start 2024-09-19T07:57:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.827+0000 7fa62352c640 1 -- start start 2024-09-19T07:57:21.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa62352c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa61c070140 0x7fa61c144e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:21.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa62352c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa61c0b3740 0x7fa61c1f4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:21.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa62352c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa61c15e450 0x7fa61c151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:21.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa62352c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c0b61b0 con 0x7fa61c070140 2024-09-19T07:57:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa62352c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c0b6030 con 0x7fa61c0b3740 2024-09-19T07:57:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa62352c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa61c0ad8c0 con 0x7fa61c15e450 2024-09-19T07:57:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa6212c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa61c070140 0x7fa61c144e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa6212c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa61c070140 0x7fa61c144e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44978/0 (socket says 172.21.15.16:44978) 2024-09-19T07:57:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa6212c8640 1 -- 172.21.15.16:0/1055348359 learned_addr learned my addr 172.21.15.16:0/1055348359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa620ac7640 1 --2- 172.21.15.16:0/1055348359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa61c0b3740 0x7fa61c1f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa621ac9640 1 --2- 172.21.15.16:0/1055348359 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa61c15e450 0x7fa61c151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa6212c8640 1 -- 172.21.15.16:0/1055348359 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa61c15e450 msgr2=0x7fa61c151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa6212c8640 1 --2- 172.21.15.16:0/1055348359 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa61c15e450 0x7fa61c151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa6212c8640 1 -- 172.21.15.16:0/1055348359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa61c0b3740 msgr2=0x7fa61c1f4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa6212c8640 1 --2- 172.21.15.16:0/1055348359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa61c0b3740 0x7fa61c1f4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa6212c8640 1 -- 172.21.15.16:0/1055348359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c1f5000 con 0x7fa61c070140 2024-09-19T07:57:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa621ac9640 1 --2- 172.21.15.16:0/1055348359 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa61c15e450 0x7fa61c151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa6212c8640 1 --2- 172.21.15.16:0/1055348359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa61c070140 0x7fa61c144e00 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fa61800c7f0 tx=0x7fa61800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa611ffb640 1 -- 172.21.15.16:0/1055348359 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618015070 con 0x7fa61c070140 2024-09-19T07:57:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa611ffb640 1 -- 172.21.15.16:0/1055348359 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa618004220 con 0x7fa61c070140 2024-09-19T07:57:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.828+0000 7fa62352c640 1 -- 172.21.15.16:0/1055348359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61c1f4a40 con 0x7fa61c070140 2024-09-19T07:57:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.829+0000 7fa611ffb640 1 -- 172.21.15.16:0/1055348359 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618003610 con 0x7fa61c070140 2024-09-19T07:57:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.829+0000 7fa62352c640 1 -- 172.21.15.16:0/1055348359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61c1501c0 con 0x7fa61c070140 2024-09-19T07:57:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.830+0000 7fa62352c640 1 -- 172.21.15.16:0/1055348359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa61c0b5af0 con 0x7fa61c070140 2024-09-19T07:57:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.830+0000 7fa611ffb640 1 -- 172.21.15.16:0/1055348359 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa6180037b0 con 0x7fa61c070140 2024-09-19T07:57:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.831+0000 7fa611ffb640 1 --2- 172.21.15.16:0/1055348359 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa60006c2b0 0x7fa60006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.831+0000 7fa611ffb640 1 -- 172.21.15.16:0/1055348359 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5792+0+0 (secure 0 0 0) 0x7fa61808b480 con 0x7fa61c070140 2024-09-19T07:57:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.831+0000 7fa620ac7640 1 --2- 172.21.15.16:0/1055348359 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa60006c2b0 0x7fa60006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.832+0000 7fa620ac7640 1 --2- 172.21.15.16:0/1055348359 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa60006c2b0 0x7fa60006e770 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa61c14f8c0 tx=0x7fa60c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:21.835+0000 7fa611ffb640 1 -- 172.21.15.16:0/1055348359 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa618059560 con 0x7fa61c070140 2024-09-19T07:57:22.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.011+0000 7fa62352c640 1 -- 172.21.15.16:0/1055348359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa61c0ae6a0 con 0x7fa61c070140 2024-09-19T07:57:22.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.691+0000 7fa611ffb640 1 -- 172.21.15.16:0/1055348359 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v156) ==== 169+0+0 (secure 0 0 0) 0x7fa61805d860 con 0x7fa61c070140 2024-09-19T07:57:22.692 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-19T07:57:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.695+0000 7fa62352c640 1 -- 172.21.15.16:0/1055348359 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa60006c2b0 msgr2=0x7fa60006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.695+0000 7fa62352c640 1 --2- 172.21.15.16:0/1055348359 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa60006c2b0 0x7fa60006e770 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa61c14f8c0 tx=0x7fa60c008040 comp rx=0 tx=0).stop 2024-09-19T07:57:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.695+0000 7fa62352c640 1 -- 172.21.15.16:0/1055348359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa61c070140 msgr2=0x7fa61c144e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.695+0000 7fa62352c640 1 --2- 172.21.15.16:0/1055348359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa61c070140 0x7fa61c144e00 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fa61800c7f0 tx=0x7fa61800ccb0 comp rx=0 tx=0).stop 2024-09-19T07:57:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.696+0000 7fa62352c640 1 -- 172.21.15.16:0/1055348359 shutdown_connections 2024-09-19T07:57:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.696+0000 7fa62352c640 1 --2- 172.21.15.16:0/1055348359 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fa60006c2b0 0x7fa60006e770 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.696+0000 7fa62352c640 1 --2- 172.21.15.16:0/1055348359 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa61c15e450 0x7fa61c151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:22.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.696+0000 7fa62352c640 1 --2- 172.21.15.16:0/1055348359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa61c0b3740 0x7fa61c1f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:22.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.696+0000 7fa62352c640 1 --2- 172.21.15.16:0/1055348359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa61c070140 0x7fa61c144e00 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:22.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.696+0000 7fa62352c640 1 -- 172.21.15.16:0/1055348359 >> 172.21.15.16:0/1055348359 conn(0x7fa61c0b23b0 msgr2=0x7fa61c0aebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:22.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.696+0000 7fa62352c640 1 -- 172.21.15.16:0/1055348359 shutdown_connections 2024-09-19T07:57:22.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.696+0000 7fa62352c640 1 -- 172.21.15.16:0/1055348359 wait complete. 2024-09-19T07:57:22.712 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T07:57:22.714 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T07:57:22.761 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-09-19T07:57:22.762 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2024-09-19T07:57:22.762 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T07:57:22.762 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2024-09-19T07:57:22.762 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.762 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2024-09-19T07:57:22.762 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.762 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2024-09-19T07:57:22.762 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.762 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2024-09-19T07:57:22.762 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2024-09-19T07:57:22.762 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.762 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2024-09-19T07:57:22.762 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2024-09-19T07:57:22.763 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.763 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T07:57:22.763 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2024-09-19T07:57:22.763 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.763 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T07:57:22.763 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T07:57:22.763 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.763 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2024-09-19T07:57:22.763 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.763 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2024-09-19T07:57:22.763 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2024-09-19T07:57:22.763 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2024-09-19T07:57:22.763 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2024-09-19T07:57:22.763 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2024-09-19T07:57:22.763 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2024-09-19T07:57:22.764 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2024-09-19T07:57:22.764 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T07:57:22.764 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T07:57:22.764 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2024-09-19T07:57:22.764 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.764 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T07:57:22.764 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.764 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2024-09-19T07:57:22.764 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.764 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2024-09-19T07:57:22.764 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.764 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2024-09-19T07:57:22.764 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2024-09-19T07:57:22.764 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2024-09-19T07:57:22.765 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-09-19T07:57:22.765 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb 2024-09-19T07:57:22.765 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.765 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2024-09-19T07:57:22.765 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2024-09-19T07:57:22.765 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.765 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2024-09-19T07:57:22.765 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2024-09-19T07:57:22.765 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2024-09-19T07:57:22.765 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.765 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2024-09-19T07:57:22.765 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2024-09-19T07:57:22.765 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.766 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2024-09-19T07:57:22.766 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2024-09-19T07:57:22.766 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2024-09-19T07:57:22.766 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2024-09-19T07:57:22.766 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.766 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2024-09-19T07:57:22.766 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2024-09-19T07:57:22.766 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.766 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2024-09-19T07:57:22.766 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.766 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2024-09-19T07:57:22.766 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.766 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2024-09-19T07:57:22.767 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.767 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2024-09-19T07:57:22.767 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2024-09-19T07:57:22.767 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.767 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2024-09-19T07:57:22.767 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2024-09-19T07:57:22.767 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2024-09-19T07:57:22.767 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2024-09-19T07:57:22.767 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2024-09-19T07:57:22.768 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2024-09-19T07:57:22.768 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:57:22.768 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2024-09-19T07:57:22.768 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2024-09-19T07:57:22.768 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T07:57:22.768 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2024-09-19T07:57:22.768 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2024-09-19T07:57:22.768 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2024-09-19T07:57:22.768 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2024-09-19T07:57:22.768 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-09-19T07:57:22.768 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.768 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2024-09-19T07:57:22.768 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:57:22.768 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2024-09-19T07:57:22.769 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-19T07:57:22.769 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-19T07:57:22.769 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:57:22.769 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:57:22.769 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2024-09-19T07:57:22.769 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-09-19T07:57:22.769 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.769 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2024-09-19T07:57:22.769 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:57:22.769 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-19T07:57:22.769 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2024-09-19T07:57:22.769 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-19T07:57:22.769 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2024-09-19T07:57:22.770 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2024-09-19T07:57:22.770 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:57:22.770 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2024-09-19T07:57:22.770 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2024-09-19T07:57:22.770 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:57:22.770 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.770 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T07:57:22.770 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T07:57:22.770 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T07:57:22.770 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.770 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T07:57:22.770 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T07:57:22.770 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T07:57:22.770 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.771 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2024-09-19T07:57:22.771 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2024-09-19T07:57:22.771 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.771 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2024-09-19T07:57:22.771 INFO:teuthology.orchestra.run.smithi016.stdout: mon down mkfs grace = 300 2024-09-19T07:57:22.771 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2024-09-19T07:57:22.771 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on pool no app = False 2024-09-19T07:57:22.771 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:22.771 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2024-09-19T07:57:22.771 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2024-09-19T07:57:22.772 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2024-09-19T07:57:22.772 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2024-09-19T07:57:22.772 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T07:57:22.772 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T07:57:22.772 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2024-09-19T07:57:22.772 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-09-19T07:57:22.772 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:57:22.773 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2024-09-19T07:57:22.773 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-09-19T07:57:22.773 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-09-19T07:57:22.773 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2024-09-19T07:57:22.773 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2024-09-19T07:57:22.773 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2024-09-19T07:57:22.773 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-09-19T07:57:22.773 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-09-19T07:57:22.773 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-09-19T07:57:22.774 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-09-19T07:57:22.774 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2024-09-19T07:57:22.774 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2024-09-19T07:57:22.774 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2024-09-19T07:57:22.774 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-19T07:57:22.774 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2024-09-19T07:57:22.774 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2024-09-19T07:57:22.774 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2024-09-19T07:57:22.774 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = True 2024-09-19T07:57:22.774 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-09-19T07:57:22.774 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-09-19T07:57:22.774 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-09-19T07:57:22.774 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-19T07:57:22.775 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2024-09-19T07:57:22.775 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2024-09-19T07:57:22.775 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T07:57:22.775 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2024-09-19T07:57:22.776 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.776 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2024-09-19T07:57:22.776 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.776 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2024-09-19T07:57:22.776 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.776 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2024-09-19T07:57:22.776 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2024-09-19T07:57:22.777 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.777 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2024-09-19T07:57:22.777 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2024-09-19T07:57:22.777 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.777 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T07:57:22.777 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2024-09-19T07:57:22.777 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.777 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T07:57:22.777 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T07:57:22.778 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.778 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2024-09-19T07:57:22.778 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.778 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2024-09-19T07:57:22.778 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2024-09-19T07:57:22.778 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2024-09-19T07:57:22.778 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2024-09-19T07:57:22.778 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2024-09-19T07:57:22.778 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2024-09-19T07:57:22.778 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2024-09-19T07:57:22.778 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T07:57:22.778 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T07:57:22.779 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2024-09-19T07:57:22.779 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.779 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T07:57:22.779 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.779 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2024-09-19T07:57:22.779 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.779 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2024-09-19T07:57:22.779 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.779 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2024-09-19T07:57:22.779 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2024-09-19T07:57:22.779 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2024-09-19T07:57:22.779 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.16,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-09-19T07:57:22.779 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = 6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb 2024-09-19T07:57:22.779 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.779 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2024-09-19T07:57:22.780 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2024-09-19T07:57:22.780 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.780 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2024-09-19T07:57:22.780 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2024-09-19T07:57:22.780 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2024-09-19T07:57:22.780 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.780 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2024-09-19T07:57:22.780 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2024-09-19T07:57:22.780 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.780 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2024-09-19T07:57:22.780 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2024-09-19T07:57:22.780 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2024-09-19T07:57:22.781 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug trim objects = true 2024-09-19T07:57:22.781 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.781 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2024-09-19T07:57:22.781 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2024-09-19T07:57:22.781 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.781 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2024-09-19T07:57:22.781 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.781 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2024-09-19T07:57:22.781 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.781 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2024-09-19T07:57:22.782 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.782 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2024-09-19T07:57:22.782 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2024-09-19T07:57:22.782 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.782 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2024-09-19T07:57:22.782 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2024-09-19T07:57:22.782 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2024-09-19T07:57:22.783 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2024-09-19T07:57:22.783 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 20 2024-09-19T07:57:22.783 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 20 2024-09-19T07:57:22.783 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:57:22.783 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2024-09-19T07:57:22.783 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 10 2024-09-19T07:57:22.783 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T07:57:22.783 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2024-09-19T07:57:22.783 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2024-09-19T07:57:22.783 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2024-09-19T07:57:22.783 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2024-09-19T07:57:22.783 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-09-19T07:57:22.783 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.784 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2024-09-19T07:57:22.784 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:57:22.784 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2024-09-19T07:57:22.784 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-19T07:57:22.784 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-19T07:57:22.784 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:57:22.784 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:57:22.784 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2024-09-19T07:57:22.784 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-09-19T07:57:22.784 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.784 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2024-09-19T07:57:22.784 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:57:22.784 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-09-19T07:57:22.784 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2024-09-19T07:57:22.785 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-09-19T07:57:22.785 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2024-09-19T07:57:22.785 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2024-09-19T07:57:22.785 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-09-19T07:57:22.785 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2024-09-19T07:57:22.785 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2024-09-19T07:57:22.785 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-09-19T07:57:22.785 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.785 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T07:57:22.785 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T07:57:22.785 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T07:57:22.785 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.785 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T07:57:22.785 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T07:57:22.786 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T07:57:22.786 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.786 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2024-09-19T07:57:22.786 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2024-09-19T07:57:22.786 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.786 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2024-09-19T07:57:22.786 INFO:teuthology.orchestra.run.smithi146.stdout: mon down mkfs grace = 300 2024-09-19T07:57:22.786 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2024-09-19T07:57:22.786 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on pool no app = False 2024-09-19T07:57:22.786 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-19T07:57:22.786 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2024-09-19T07:57:22.786 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2024-09-19T07:57:22.786 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2024-09-19T07:57:22.786 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2024-09-19T07:57:22.787 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T07:57:22.787 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T07:57:22.787 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2024-09-19T07:57:22.787 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-09-19T07:57:22.787 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:57:22.787 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = "" 2024-09-19T07:57:22.787 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-09-19T07:57:22.787 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-09-19T07:57:22.787 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2024-09-19T07:57:22.787 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2024-09-19T07:57:22.787 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2024-09-19T07:57:22.787 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2024-09-19T07:57:22.787 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2024-09-19T07:57:22.787 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2024-09-19T07:57:22.787 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2024-09-19T07:57:22.788 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2024-09-19T07:57:22.788 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2024-09-19T07:57:22.788 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2024-09-19T07:57:22.788 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-09-19T07:57:22.788 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2024-09-19T07:57:22.788 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2024-09-19T07:57:22.788 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2024-09-19T07:57:22.788 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2024-09-19T07:57:22.788 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-09-19T07:57:22.788 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-09-19T07:57:22.788 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-09-19T07:57:22.788 INFO:teuthology.orchestra.run.smithi146.stdout:[client.0] 2024-09-19T07:57:22.790 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T07:57:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.898+0000 7f9c62d86640 1 Processor -- start 2024-09-19T07:57:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.898+0000 7f9c62d86640 1 -- start start 2024-09-19T07:57:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.899+0000 7f9c62d86640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c5c0cff80 0x7f9c5c0b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.899+0000 7f9c62d86640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c5c0b42b0 0x7f9c5c0b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.899+0000 7f9c62d86640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c5c0cef20 con 0x7f9c5c0d27d0 2024-09-19T07:57:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.899+0000 7f9c62d86640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f9c5c0cf1c0 con 0x7f9c5c0cff80 2024-09-19T07:57:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.899+0000 7f9c62d86640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9c5c0cf4a0 con 0x7f9c5c0b42b0 2024-09-19T07:57:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.899+0000 7f9c5bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c5c0cff80 0x7f9c5c0b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.899+0000 7f9c5bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c5c0cff80 0x7f9c5c0b3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42212/0 (socket says 172.21.15.16:42212) 2024-09-19T07:57:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.899+0000 7f9c60b22640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9c5c0d27d0 0x7f9c5c0d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60812/0 (socket says 172.21.15.16:60812) 2024-09-19T07:57:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.899+0000 7f9c5bfff640 1 -- 172.21.15.16:0/3845251189 learned_addr learned my addr 172.21.15.16:0/3845251189 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.899+0000 7f9c61323640 1 --2- 172.21.15.16:0/3845251189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c5c0b42b0 0x7f9c5c0b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.900+0000 7f9c5bfff640 1 -- 172.21.15.16:0/3845251189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c5c0b42b0 msgr2=0x7f9c5c0b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.900+0000 7f9c5bfff640 1 --2- 172.21.15.16:0/3845251189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c5c0b42b0 0x7f9c5c0b46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.900+0000 7f9c5bfff640 1 -- 172.21.15.16:0/3845251189 >> v1:172.21.15.16:6789/0 conn(0x7f9c5c0d27d0 legacy=0x7f9c5c0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.900+0000 7f9c5bfff640 1 -- 172.21.15.16:0/3845251189 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5c0cf780 con 0x7f9c5c0cff80 2024-09-19T07:57:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.900+0000 7f9c5b7fe640 1 -- 172.21.15.16:0/3845251189 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1580681108 0 0) 0x7f9c5c0cef20 con 0x7f9c5c0d27d0 2024-09-19T07:57:22.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.900+0000 7f9c5bfff640 1 --2- 172.21.15.16:0/3845251189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c0cff80 0x7f9c5c0b3d70 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009870 tx=0x7f9c5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6fee86a96b1c68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:22.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.900+0000 7f9c5b7fe640 1 -- 172.21.15.16:0/3845251189 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50037070 con 0x7f9c5c0cff80 2024-09-19T07:57:22.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.901+0000 7f9c5b7fe640 1 -- 172.21.15.16:0/3845251189 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c50032440 con 0x7f9c5c0cff80 2024-09-19T07:57:22.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.901+0000 7f9c5b7fe640 1 -- 172.21.15.16:0/3845251189 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50032810 con 0x7f9c5c0cff80 2024-09-19T07:57:22.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.901+0000 7f9c62d86640 1 -- 172.21.15.16:0/3845251189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c0cff80 msgr2=0x7f9c5c0b3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:22.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.901+0000 7f9c62d86640 1 --2- 172.21.15.16:0/3845251189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c0cff80 0x7f9c5c0b3d70 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009870 tx=0x7f9c5002a270 comp rx=0 tx=0).stop 2024-09-19T07:57:22.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.901+0000 7f9c62d86640 1 -- 172.21.15.16:0/3845251189 shutdown_connections 2024-09-19T07:57:22.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.901+0000 7f9c62d86640 1 --2- 172.21.15.16:0/3845251189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c5c0b42b0 0x7f9c5c0b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:22.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.901+0000 7f9c62d86640 1 --2- 172.21.15.16:0/3845251189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c0cff80 0x7f9c5c0b3d70 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:22.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.901+0000 7f9c62d86640 1 -- 172.21.15.16:0/3845251189 >> 172.21.15.16:0/3845251189 conn(0x7f9c5c09d720 msgr2=0x7f9c5c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:22.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.901+0000 7f9c62d86640 1 -- 172.21.15.16:0/3845251189 shutdown_connections 2024-09-19T07:57:22.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.901+0000 7f9c62d86640 1 -- 172.21.15.16:0/3845251189 wait complete. 2024-09-19T07:57:22.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.902+0000 7f9c62d86640 1 Processor -- start 2024-09-19T07:57:22.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.902+0000 7f9c62d86640 1 -- start start 2024-09-19T07:57:22.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.903+0000 7f9c62d86640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c5c0b42b0 0x7f9c5c0cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:22.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.903+0000 7f9c62d86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c0cff80 0x7f9c5c0b82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:22.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.903+0000 7f9c62d86640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c5c0d27d0 0x7f9c5c152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.903+0000 7f9c62d86640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c0b5970 con 0x7f9c5c0cff80 2024-09-19T07:57:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.903+0000 7f9c62d86640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c0b57f0 con 0x7f9c5c0b42b0 2024-09-19T07:57:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.903+0000 7f9c62d86640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9c5c0d32c0 con 0x7f9c5c0d27d0 2024-09-19T07:57:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.903+0000 7f9c5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c0cff80 0x7f9c5c0b82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.903+0000 7f9c5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c0cff80 0x7f9c5c0b82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42214/0 (socket says 172.21.15.16:42214) 2024-09-19T07:57:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.903+0000 7f9c5bfff640 1 -- 172.21.15.16:0/3259699246 learned_addr learned my addr 172.21.15.16:0/3259699246 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.903+0000 7f9c60b22640 1 --2- 172.21.15.16:0/3259699246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c5c0b42b0 0x7f9c5c0cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.903+0000 7f9c61323640 1 --2- 172.21.15.16:0/3259699246 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c5c0d27d0 0x7f9c5c152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.904+0000 7f9c5bfff640 1 -- 172.21.15.16:0/3259699246 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c5c0d27d0 msgr2=0x7f9c5c152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.904+0000 7f9c5bfff640 1 --2- 172.21.15.16:0/3259699246 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c5c0d27d0 0x7f9c5c152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.904+0000 7f9c5bfff640 1 -- 172.21.15.16:0/3259699246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c5c0b42b0 msgr2=0x7f9c5c0cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.904+0000 7f9c5bfff640 1 --2- 172.21.15.16:0/3259699246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c5c0b42b0 0x7f9c5c0cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.904+0000 7f9c5bfff640 1 -- 172.21.15.16:0/3259699246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5c1f79a0 con 0x7f9c5c0cff80 2024-09-19T07:57:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.904+0000 7f9c5bfff640 1 --2- 172.21.15.16:0/3259699246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c0cff80 0x7f9c5c0b82d0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f9c50035040 tx=0x7f9c500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.904+0000 7f9c597fa640 1 -- 172.21.15.16:0/3259699246 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50037040 con 0x7f9c5c0cff80 2024-09-19T07:57:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.904+0000 7f9c597fa640 1 -- 172.21.15.16:0/3259699246 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c50004030 con 0x7f9c5c0cff80 2024-09-19T07:57:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.904+0000 7f9c597fa640 1 -- 172.21.15.16:0/3259699246 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50004420 con 0x7f9c5c0cff80 2024-09-19T07:57:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.904+0000 7f9c62d86640 1 -- 172.21.15.16:0/3259699246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5c1f7c80 con 0x7f9c5c0cff80 2024-09-19T07:57:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.904+0000 7f9c62d86640 1 -- 172.21.15.16:0/3259699246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c5c1f8240 con 0x7f9c5c0cff80 2024-09-19T07:57:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.905+0000 7f9c62d86640 1 -- 172.21.15.16:0/3259699246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c5c0b3d70 con 0x7f9c5c0cff80 2024-09-19T07:57:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.906+0000 7f9c597fa640 1 -- 172.21.15.16:0/3259699246 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9c5003a070 con 0x7f9c5c0cff80 2024-09-19T07:57:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.906+0000 7f9c597fa640 1 --2- 172.21.15.16:0/3259699246 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9c4806c2b0 0x7f9c4806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.906+0000 7f9c597fa640 1 -- 172.21.15.16:0/3259699246 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f9c500ab7b0 con 0x7f9c5c0cff80 2024-09-19T07:57:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.906+0000 7f9c60b22640 1 --2- 172.21.15.16:0/3259699246 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9c4806c2b0 0x7f9c4806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.907+0000 7f9c60b22640 1 --2- 172.21.15.16:0/3259699246 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9c4806c2b0 0x7f9c4806e770 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c007fd0 tx=0x7f9c4c007e00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:22.911+0000 7f9c597fa640 1 -- 172.21.15.16:0/3259699246 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9c50041020 con 0x7f9c5c0cff80 2024-09-19T07:57:23.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.085+0000 7f9c62d86640 1 -- 172.21.15.16:0/3259699246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f9c5c06dda0 con 0x7f9c5c0cff80 2024-09-19T07:57:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.089+0000 7f9c597fa640 1 -- 172.21.15.16:0/3259699246 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v74) ==== 104+0+0 (secure 0 0 0) 0x7f9c50079a10 con 0x7f9c5c0cff80 2024-09-19T07:57:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.091+0000 7f9c62d86640 1 -- 172.21.15.16:0/3259699246 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9c4806c2b0 msgr2=0x7f9c4806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.091+0000 7f9c62d86640 1 --2- 172.21.15.16:0/3259699246 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9c4806c2b0 0x7f9c4806e770 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c007fd0 tx=0x7f9c4c007e00 comp rx=0 tx=0).stop 2024-09-19T07:57:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.091+0000 7f9c62d86640 1 -- 172.21.15.16:0/3259699246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c0cff80 msgr2=0x7f9c5c0b82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.091+0000 7f9c62d86640 1 --2- 172.21.15.16:0/3259699246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c0cff80 0x7f9c5c0b82d0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f9c50035040 tx=0x7f9c500046c0 comp rx=0 tx=0).stop 2024-09-19T07:57:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.091+0000 7f9c62d86640 1 -- 172.21.15.16:0/3259699246 shutdown_connections 2024-09-19T07:57:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.091+0000 7f9c62d86640 1 --2- 172.21.15.16:0/3259699246 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f9c4806c2b0 0x7f9c4806e770 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.091+0000 7f9c62d86640 1 --2- 172.21.15.16:0/3259699246 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c5c0d27d0 0x7f9c5c152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.091+0000 7f9c62d86640 1 --2- 172.21.15.16:0/3259699246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c0cff80 0x7f9c5c0b82d0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.091+0000 7f9c62d86640 1 --2- 172.21.15.16:0/3259699246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c5c0b42b0 0x7f9c5c0cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.091+0000 7f9c62d86640 1 -- 172.21.15.16:0/3259699246 >> 172.21.15.16:0/3259699246 conn(0x7f9c5c09d720 msgr2=0x7f9c5c06e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.091+0000 7f9c62d86640 1 -- 172.21.15.16:0/3259699246 shutdown_connections 2024-09-19T07:57:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.091+0000 7f9c62d86640 1 -- 172.21.15.16:0/3259699246 wait complete. 2024-09-19T07:57:23.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug' 2024-09-19T07:57:23.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.203+0000 7f30a38c9640 1 Processor -- start 2024-09-19T07:57:23.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.204+0000 7f30a38c9640 1 -- start start 2024-09-19T07:57:23.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.204+0000 7f30a38c9640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f309c15ecf0 0x7f309c0adb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:23.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.205+0000 7f30a38c9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f309c15f1f0 con 0x7f309c06f450 2024-09-19T07:57:23.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.205+0000 7f30a38c9640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f309c15f4d0 con 0x7f309c15b160 2024-09-19T07:57:23.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.205+0000 7f30a38c9640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f309c15f7b0 con 0x7f309c15ecf0 2024-09-19T07:57:23.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.205+0000 7f30a1e66640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f309c06f450 0x7f309c06f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60822/0 (socket says 172.21.15.16:60822) 2024-09-19T07:57:23.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.205+0000 7f30a1e66640 1 -- 172.21.15.16:0/4164536045 learned_addr learned my addr 172.21.15.16:0/4164536045 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:23.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.205+0000 7f30a0e64640 1 --2- 172.21.15.16:0/4164536045 >> v2:172.21.15.146:3300/0 conn(0x7f309c15ecf0 0x7f309c0adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:23.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.205+0000 7f3093fff640 1 -- 172.21.15.16:0/4164536045 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3136778862 0 0) 0x7f309c15f1f0 con 0x7f309c06f450 2024-09-19T07:57:23.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.205+0000 7f3093fff640 1 -- 172.21.15.16:0/4164536045 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3088003520 con 0x7f309c06f450 2024-09-19T07:57:23.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.205+0000 7f3093fff640 1 -- 172.21.15.16:0/4164536045 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1317077033 0 0) 0x7f3088003520 con 0x7f309c06f450 2024-09-19T07:57:23.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.205+0000 7f30a0e64640 1 -- 172.21.15.16:0/4164536045 >> v1:172.21.15.146:6789/0 conn(0x7f309c15b160 legacy=0x7f309c073400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:23.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.205+0000 7f30a0e64640 1 -- 172.21.15.16:0/4164536045 >> v1:172.21.15.16:6789/0 conn(0x7f309c06f450 legacy=0x7f309c06f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:23.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.205+0000 7f30a0e64640 1 -- 172.21.15.16:0/4164536045 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309c15fa90 con 0x7f309c15ecf0 2024-09-19T07:57:23.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.206+0000 7f30a0e64640 1 --2- 172.21.15.16:0/4164536045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f309c15ecf0 0x7f309c0adb70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f308c0097a0 tx=0x7f308c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fefebaba0f3968b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:23.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.206+0000 7f3093fff640 1 -- 172.21.15.16:0/4164536045 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c037070 con 0x7f309c15ecf0 2024-09-19T07:57:23.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.206+0000 7f3093fff640 1 -- 172.21.15.16:0/4164536045 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f308c02abc0 con 0x7f309c15ecf0 2024-09-19T07:57:23.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.206+0000 7f3093fff640 1 -- 172.21.15.16:0/4164536045 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30940019f0 con 0x7f309c15ecf0 2024-09-19T07:57:23.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.207+0000 7f30a38c9640 1 -- 172.21.15.16:0/4164536045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f309c15ecf0 msgr2=0x7f309c0adb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:23.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.207+0000 7f30a38c9640 1 --2- 172.21.15.16:0/4164536045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f309c15ecf0 0x7f309c0adb70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f308c0097a0 tx=0x7f308c02a130 comp rx=0 tx=0).stop 2024-09-19T07:57:23.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.207+0000 7f30a38c9640 1 -- 172.21.15.16:0/4164536045 shutdown_connections 2024-09-19T07:57:23.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.207+0000 7f30a38c9640 1 --2- 172.21.15.16:0/4164536045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f309c15ecf0 0x7f309c0adb70 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:23.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.207+0000 7f30a38c9640 1 -- 172.21.15.16:0/4164536045 >> 172.21.15.16:0/4164536045 conn(0x7f309c0b1b30 msgr2=0x7f309c0b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:23.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.207+0000 7f30a38c9640 1 -- 172.21.15.16:0/4164536045 shutdown_connections 2024-09-19T07:57:23.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.207+0000 7f30a38c9640 1 -- 172.21.15.16:0/4164536045 wait complete. 2024-09-19T07:57:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.208+0000 7f30a38c9640 1 Processor -- start 2024-09-19T07:57:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.208+0000 7f30a38c9640 1 -- start start 2024-09-19T07:57:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.209+0000 7f30a38c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c06f450 0x7f309c1f18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.209+0000 7f30a38c9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f309c15b160 0x7f309c1f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.209+0000 7f30a38c9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f309c15ecf0 0x7f309c0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.209+0000 7f30a38c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f309c0b3310 con 0x7f309c06f450 2024-09-19T07:57:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.209+0000 7f30a38c9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f309c0ae0b0 con 0x7f309c15b160 2024-09-19T07:57:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.209+0000 7f30a38c9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f309c0b3490 con 0x7f309c15ecf0 2024-09-19T07:57:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.209+0000 7f30a1665640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c06f450 0x7f309c1f18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.209+0000 7f30a1665640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c06f450 0x7f309c1f18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42220/0 (socket says 172.21.15.16:42220) 2024-09-19T07:57:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.209+0000 7f30a1665640 1 -- 172.21.15.16:0/1735678813 learned_addr learned my addr 172.21.15.16:0/1735678813 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.209+0000 7f30a1e66640 1 --2- 172.21.15.16:0/1735678813 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f309c15ecf0 0x7f309c0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.209+0000 7f30a0e64640 1 --2- 172.21.15.16:0/1735678813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f309c15b160 0x7f309c1f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.209+0000 7f30a1665640 1 -- 172.21.15.16:0/1735678813 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f309c15ecf0 msgr2=0x7f309c0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.209+0000 7f30a1665640 1 --2- 172.21.15.16:0/1735678813 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f309c15ecf0 0x7f309c0ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.209+0000 7f30a1665640 1 -- 172.21.15.16:0/1735678813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f309c15b160 msgr2=0x7f309c1f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.209+0000 7f30a1665640 1 --2- 172.21.15.16:0/1735678813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f309c15b160 0x7f309c1f45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.209+0000 7f30a1665640 1 -- 172.21.15.16:0/1735678813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309c0cc210 con 0x7f309c06f450 2024-09-19T07:57:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.209+0000 7f30a1665640 1 --2- 172.21.15.16:0/1735678813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c06f450 0x7f309c1f18a0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f309800c450 tx=0x7f309800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.210+0000 7f3091ffb640 1 -- 172.21.15.16:0/1735678813 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30980052d0 con 0x7f309c06f450 2024-09-19T07:57:23.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.210+0000 7f3091ffb640 1 -- 172.21.15.16:0/1735678813 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3098005470 con 0x7f309c06f450 2024-09-19T07:57:23.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.210+0000 7f30a38c9640 1 -- 172.21.15.16:0/1735678813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309c0cc4f0 con 0x7f309c06f450 2024-09-19T07:57:23.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.210+0000 7f3091ffb640 1 -- 172.21.15.16:0/1735678813 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098012870 con 0x7f309c06f450 2024-09-19T07:57:23.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.210+0000 7f30a38c9640 1 -- 172.21.15.16:0/1735678813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f309c0ccab0 con 0x7f309c06f450 2024-09-19T07:57:23.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.211+0000 7f30a38c9640 1 -- 172.21.15.16:0/1735678813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f309c0adb70 con 0x7f309c06f450 2024-09-19T07:57:23.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.211+0000 7f3091ffb640 1 -- 172.21.15.16:0/1735678813 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3098012a10 con 0x7f309c06f450 2024-09-19T07:57:23.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.212+0000 7f3091ffb640 1 --2- 172.21.15.16:0/1735678813 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f308006c2b0 0x7f308006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:23.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.212+0000 7f3091ffb640 1 -- 172.21.15.16:0/1735678813 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f309808ae90 con 0x7f309c06f450 2024-09-19T07:57:23.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.212+0000 7f30a0e64640 1 --2- 172.21.15.16:0/1735678813 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f308006c2b0 0x7f308006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:23.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.212+0000 7f30a0e64640 1 --2- 172.21.15.16:0/1735678813 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f308006c2b0 0x7f308006e770 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f308c034000 tx=0x7f308c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:23.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.216+0000 7f3091ffb640 1 -- 172.21.15.16:0/1735678813 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f30980597d0 con 0x7f309c06f450 2024-09-19T07:57:23.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.465+0000 7f30a38c9640 1 -- 172.21.15.16:0/1735678813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]} v 0) -- 0x7f309c06cbd0 con 0x7f309c06f450 2024-09-19T07:57:23.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.693+0000 7f3091ffb640 1 -- 172.21.15.16:0/1735678813 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]}]=0 v643) ==== 151+0+0 (secure 0 0 0) 0x7f30980590d0 con 0x7f309c06f450 2024-09-19T07:57:23.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.695+0000 7f30a38c9640 1 -- 172.21.15.16:0/1735678813 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f308006c2b0 msgr2=0x7f308006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.695+0000 7f30a38c9640 1 --2- 172.21.15.16:0/1735678813 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f308006c2b0 0x7f308006e770 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f308c034000 tx=0x7f308c02e000 comp rx=0 tx=0).stop 2024-09-19T07:57:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.695+0000 7f30a38c9640 1 -- 172.21.15.16:0/1735678813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c06f450 msgr2=0x7f309c1f18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.695+0000 7f30a38c9640 1 --2- 172.21.15.16:0/1735678813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c06f450 0x7f309c1f18a0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f309800c450 tx=0x7f309800c910 comp rx=0 tx=0).stop 2024-09-19T07:57:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.695+0000 7f30a38c9640 1 -- 172.21.15.16:0/1735678813 shutdown_connections 2024-09-19T07:57:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.695+0000 7f30a38c9640 1 --2- 172.21.15.16:0/1735678813 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f308006c2b0 0x7f308006e770 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.695+0000 7f30a38c9640 1 --2- 172.21.15.16:0/1735678813 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f309c15ecf0 0x7f309c0ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.695+0000 7f30a38c9640 1 --2- 172.21.15.16:0/1735678813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f309c15b160 0x7f309c1f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.695+0000 7f30a38c9640 1 --2- 172.21.15.16:0/1735678813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c06f450 0x7f309c1f18a0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.695+0000 7f30a38c9640 1 -- 172.21.15.16:0/1735678813 >> 172.21.15.16:0/1735678813 conn(0x7f309c0b1b30 msgr2=0x7f309c0730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.696+0000 7f30a38c9640 1 -- 172.21.15.16:0/1735678813 shutdown_connections 2024-09-19T07:57:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:23.696+0000 7f30a38c9640 1 -- 172.21.15.16:0/1735678813 wait complete. 2024-09-19T07:57:23.707 INFO:tasks.cephfs_test_runner:When a client has a bug (which we will simulate) preventing it from releasing caps, ... ok 2024-09-19T07:57:23.708 INFO:tasks.cephfs_test_runner: 2024-09-19T07:57:23.708 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-19T07:57:23.708 INFO:tasks.cephfs_test_runner:Ran 10 tests in 822.091s 2024-09-19T07:57:23.708 INFO:tasks.cephfs_test_runner: 2024-09-19T07:57:23.708 INFO:tasks.cephfs_test_runner:OK 2024-09-19T07:57:23.708 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-19T07:57:23.721 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-19T07:57:23.768 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-19T07:57:23.768 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:57:23.768 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:57:23.787 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:57:23.787 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T07:57:23.787 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:57:23.844 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1454 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T07:57:23.844 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:57:23.845 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:57:23.866 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:57:23.866 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-19T07:57:23.866 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:57:23.926 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056027,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3540 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497464k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T07:57:23.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:57:23.926 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T07:57:23.944 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T07:57:23.944 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:57:23.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:57:23.945 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T07:57:24.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:57:24.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:57:24.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:57:24.080 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-19T07:57:24.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:57:24.184 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-09-19T07:57:24.203 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T07:57:24.203 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T07:57:24.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:57:24.203 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T07:57:24.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:57:24.274 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T07:57:24.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:57:24.342 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link delete ceph-brx 2024-09-19T07:57:24.438 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-19T07:57:24.449 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T07:57:24.449 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:57:24.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.554+0000 7fe48f042640 1 Processor -- start 2024-09-19T07:57:24.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.554+0000 7fe48f042640 1 -- start start 2024-09-19T07:57:24.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.555+0000 7fe48f042640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe488145b30 0x7fe488072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:24.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.555+0000 7fe48f042640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe4880ad520 0x7fe48806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:24.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.555+0000 7fe48f042640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe488073710 con 0x7fe4880af600 2024-09-19T07:57:24.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.555+0000 7fe48f042640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe48806df30 con 0x7fe4880ad520 2024-09-19T07:57:24.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.555+0000 7fe48f042640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe488145f10 con 0x7fe488145b30 2024-09-19T07:57:24.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.555+0000 7fe487fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe4880af600 0x7fe4880ace10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60830/0 (socket says 172.21.15.16:60830) 2024-09-19T07:57:24.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.555+0000 7fe487fff640 1 -- 172.21.15.16:0/1848990612 learned_addr learned my addr 172.21.15.16:0/1848990612 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:24.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.555+0000 7fe48d5df640 1 --2- 172.21.15.16:0/1848990612 >> v2:172.21.15.146:3300/0 conn(0x7fe4880ad520 0x7fe48806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:24.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.555+0000 7fe48cdde640 1 --2- 172.21.15.16:0/1848990612 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe488145b30 0x7fe488072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:24.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe4877fe640 1 -- 172.21.15.16:0/1848990612 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 704329570 0 0) 0x7fe488073710 con 0x7fe4880af600 2024-09-19T07:57:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe4877fe640 1 -- 172.21.15.16:0/1848990612 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe470003520 con 0x7fe4880af600 2024-09-19T07:57:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe4877fe640 1 -- 172.21.15.16:0/1848990612 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 542591569 0 0) 0x7fe470003520 con 0x7fe4880af600 2024-09-19T07:57:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe4877fe640 1 -- 172.21.15.16:0/1848990612 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe488145b30 msgr2=0x7fe488072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe4877fe640 1 --2- 172.21.15.16:0/1848990612 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe488145b30 0x7fe488072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe4877fe640 1 -- 172.21.15.16:0/1848990612 >> v2:172.21.15.146:3300/0 conn(0x7fe4880ad520 msgr2=0x7fe48806e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe4877fe640 1 --2- 172.21.15.16:0/1848990612 >> v2:172.21.15.146:3300/0 conn(0x7fe4880ad520 0x7fe48806e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe4877fe640 1 -- 172.21.15.16:0/1848990612 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe488072e70 con 0x7fe4880af600 2024-09-19T07:57:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe4877fe640 1 -- 172.21.15.16:0/1848990612 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7fe47c001670 con 0x7fe4880af600 2024-09-19T07:57:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe48d5df640 1 --2- 172.21.15.16:0/1848990612 >> v2:172.21.15.146:3300/0 conn(0x7fe4880ad520 0x7fe48806e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:57:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe4877fe640 1 -- 172.21.15.16:0/1848990612 >> v1:172.21.15.16:6789/0 conn(0x7fe4880af600 legacy=0x7fe4880ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe4877fe640 1 --2- 172.21.15.16:0/1848990612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4700039b0 0x7fe470003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe4877fe640 1 --2- 172.21.15.16:0/1848990612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe470004800 0x7fe470004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe4877fe640 1 --2- 172.21.15.16:0/1848990612 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe488145b30 0x7fe4700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe4877fe640 1 -- 172.21.15.16:0/1848990612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe48806df30 con 0x7fe4700039b0 2024-09-19T07:57:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe4877fe640 1 -- 172.21.15.16:0/1848990612 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe488145f10 con 0x7fe470004800 2024-09-19T07:57:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe4877fe640 1 -- 172.21.15.16:0/1848990612 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe47000b9c0 con 0x7fe488145b30 2024-09-19T07:57:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe48cdde640 1 --2- 172.21.15.16:0/1848990612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4700039b0 0x7fe470003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe48d5df640 1 --2- 172.21.15.16:0/1848990612 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe488145b30 0x7fe4700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe487fff640 1 --2- 172.21.15.16:0/1848990612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe470004800 0x7fe470004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe48cdde640 1 -- 172.21.15.16:0/1848990612 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe488145b30 msgr2=0x7fe4700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe48cdde640 1 --2- 172.21.15.16:0/1848990612 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe488145b30 0x7fe4700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe48cdde640 1 -- 172.21.15.16:0/1848990612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe470004800 msgr2=0x7fe470004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe48cdde640 1 --2- 172.21.15.16:0/1848990612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe470004800 0x7fe470004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe48cdde640 1 -- 172.21.15.16:0/1848990612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe470003520 con 0x7fe4700039b0 2024-09-19T07:57:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe48d5df640 1 --2- 172.21.15.16:0/1848990612 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe488145b30 0x7fe4700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.556+0000 7fe487fff640 1 --2- 172.21.15.16:0/1848990612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe470004800 0x7fe470004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.557+0000 7fe48cdde640 1 --2- 172.21.15.16:0/1848990612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4700039b0 0x7fe470003dd0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fe478009ac0 tx=0x7fe470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b92d4289b38870 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.557+0000 7fe4877fe640 1 -- 172.21.15.16:0/1848990612 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478036070 con 0x7fe4700039b0 2024-09-19T07:57:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.557+0000 7fe4877fe640 1 -- 172.21.15.16:0/1848990612 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe478031400 con 0x7fe4700039b0 2024-09-19T07:57:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.557+0000 7fe4877fe640 1 -- 172.21.15.16:0/1848990612 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478009690 con 0x7fe4700039b0 2024-09-19T07:57:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.558+0000 7fe48f042640 1 -- 172.21.15.16:0/1848990612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4700039b0 msgr2=0x7fe470003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.558+0000 7fe48f042640 1 --2- 172.21.15.16:0/1848990612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4700039b0 0x7fe470003dd0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fe478009ac0 tx=0x7fe470005830 comp rx=0 tx=0).stop 2024-09-19T07:57:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.558+0000 7fe48d5df640 1 -- 172.21.15.16:0/1848990612 reap_dead start 2024-09-19T07:57:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.558+0000 7fe48f042640 1 -- 172.21.15.16:0/1848990612 shutdown_connections 2024-09-19T07:57:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.558+0000 7fe48f042640 1 -- 172.21.15.16:0/1848990612 >> 172.21.15.16:0/1848990612 conn(0x7fe4880b1b30 msgr2=0x7fe4880b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.558+0000 7fe48f042640 1 -- 172.21.15.16:0/1848990612 shutdown_connections 2024-09-19T07:57:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.558+0000 7fe48f042640 1 -- 172.21.15.16:0/1848990612 wait complete. 2024-09-19T07:57:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.559+0000 7fe48f042640 1 Processor -- start 2024-09-19T07:57:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.559+0000 7fe48f042640 1 -- start start 2024-09-19T07:57:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.560+0000 7fe48f042640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4700039b0 0x7fe48815b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.560+0000 7fe48f042640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4880c4de0 0x7fe4880c5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.560+0000 7fe48f042640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe488141650 0x7fe488141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.560+0000 7fe48f042640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe488072620 con 0x7fe4700039b0 2024-09-19T07:57:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.560+0000 7fe48f042640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe488073430 con 0x7fe4880c4de0 2024-09-19T07:57:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.560+0000 7fe48f042640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe4880727a0 con 0x7fe488141650 2024-09-19T07:57:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.560+0000 7fe48cdde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4700039b0 0x7fe48815b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.560+0000 7fe48cdde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4700039b0 0x7fe48815b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42230/0 (socket says 172.21.15.16:42230) 2024-09-19T07:57:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.560+0000 7fe48cdde640 1 -- 172.21.15.16:0/1356433552 learned_addr learned my addr 172.21.15.16:0/1356433552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.560+0000 7fe487fff640 1 --2- 172.21.15.16:0/1356433552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4880c4de0 0x7fe4880c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.560+0000 7fe48d5df640 1 --2- 172.21.15.16:0/1356433552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe488141650 0x7fe488141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.560+0000 7fe48cdde640 1 -- 172.21.15.16:0/1356433552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe488141650 msgr2=0x7fe488141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.560+0000 7fe48cdde640 1 --2- 172.21.15.16:0/1356433552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe488141650 0x7fe488141ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.560+0000 7fe48cdde640 1 -- 172.21.15.16:0/1356433552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4880c4de0 msgr2=0x7fe4880c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.560+0000 7fe48cdde640 1 --2- 172.21.15.16:0/1356433552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4880c4de0 0x7fe4880c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.560+0000 7fe48cdde640 1 -- 172.21.15.16:0/1356433552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48815be70 con 0x7fe4700039b0 2024-09-19T07:57:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.561+0000 7fe48cdde640 1 --2- 172.21.15.16:0/1356433552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4700039b0 0x7fe48815b930 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fe478009be0 tx=0x7fe478004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.561+0000 7fe4857fa640 1 -- 172.21.15.16:0/1356433552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478040020 con 0x7fe4700039b0 2024-09-19T07:57:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.561+0000 7fe4857fa640 1 -- 172.21.15.16:0/1356433552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe478003a40 con 0x7fe4700039b0 2024-09-19T07:57:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.561+0000 7fe48f042640 1 -- 172.21.15.16:0/1356433552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe488142380 con 0x7fe4700039b0 2024-09-19T07:57:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.561+0000 7fe4857fa640 1 -- 172.21.15.16:0/1356433552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478036040 con 0x7fe4700039b0 2024-09-19T07:57:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.561+0000 7fe48f042640 1 -- 172.21.15.16:0/1356433552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe48809b9f0 con 0x7fe4700039b0 2024-09-19T07:57:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.562+0000 7fe48f042640 1 -- 172.21.15.16:0/1356433552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe488072b30 con 0x7fe4700039b0 2024-09-19T07:57:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.562+0000 7fe4857fa640 1 -- 172.21.15.16:0/1356433552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe478043020 con 0x7fe4700039b0 2024-09-19T07:57:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.563+0000 7fe4857fa640 1 --2- 172.21.15.16:0/1356433552 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe47406c2b0 0x7fe47406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.563+0000 7fe4857fa640 1 -- 172.21.15.16:0/1356433552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7fe4780ab6e0 con 0x7fe4700039b0 2024-09-19T07:57:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.563+0000 7fe487fff640 1 --2- 172.21.15.16:0/1356433552 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe47406c2b0 0x7fe47406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.563+0000 7fe487fff640 1 --2- 172.21.15.16:0/1356433552 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe47406c2b0 0x7fe47406e770 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe488144e90 tx=0x7fe47c007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.567+0000 7fe4857fa640 1 -- 172.21.15.16:0/1356433552 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe47804a030 con 0x7fe4700039b0 2024-09-19T07:57:24.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.740+0000 7fe48f042640 1 -- 172.21.15.16:0/1356433552 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe488073cf0 con 0x7fe47406c2b0 2024-09-19T07:57:24.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.741+0000 7fe4857fa640 1 -- 172.21.15.16:0/1356433552 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13463 (secure 0 0 0) 0x7fe488073cf0 con 0x7fe47406c2b0 2024-09-19T07:57:24.741 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:24.742 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:57:24.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.743+0000 7fe48f042640 1 -- 172.21.15.16:0/1356433552 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe47406c2b0 msgr2=0x7fe47406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:24.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.743+0000 7fe48f042640 1 --2- 172.21.15.16:0/1356433552 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe47406c2b0 0x7fe47406e770 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe488144e90 tx=0x7fe47c007040 comp rx=0 tx=0).stop 2024-09-19T07:57:24.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.743+0000 7fe48f042640 1 -- 172.21.15.16:0/1356433552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4700039b0 msgr2=0x7fe48815b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:24.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.743+0000 7fe48f042640 1 --2- 172.21.15.16:0/1356433552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4700039b0 0x7fe48815b930 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fe478009be0 tx=0x7fe478004730 comp rx=0 tx=0).stop 2024-09-19T07:57:24.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.743+0000 7fe48f042640 1 -- 172.21.15.16:0/1356433552 shutdown_connections 2024-09-19T07:57:24.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.743+0000 7fe48f042640 1 --2- 172.21.15.16:0/1356433552 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fe47406c2b0 0x7fe47406e770 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:24.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.743+0000 7fe48f042640 1 --2- 172.21.15.16:0/1356433552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe488141650 0x7fe488141ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:24.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.743+0000 7fe48f042640 1 --2- 172.21.15.16:0/1356433552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4880c4de0 0x7fe4880c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:24.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.743+0000 7fe48f042640 1 --2- 172.21.15.16:0/1356433552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4700039b0 0x7fe48815b930 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:24.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.743+0000 7fe48f042640 1 -- 172.21.15.16:0/1356433552 >> 172.21.15.16:0/1356433552 conn(0x7fe4880b1b30 msgr2=0x7fe4880ada80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:24.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.743+0000 7fe48f042640 1 -- 172.21.15.16:0/1356433552 shutdown_connections 2024-09-19T07:57:24.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.743+0000 7fe48f042640 1 -- 172.21.15.16:0/1356433552 wait complete. 2024-09-19T07:57:24.754 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":586,"stamp":"2024-09-19T07:57:24.665369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":770936,"kb_used_data":16456,"kb_used_omap":12,"kb_used_meta":754291,"kb_avail":754203784,"statfs":{"total":773094113280,"available":772304674816,"internally_reserved":0,"allocated":16850944,"data_stored":8463570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":772394544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14414,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-66,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001328"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":417,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-19T07:57:22.703562+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:57:22.703562+0000","last_peered":"2024-09-19T07:57:22.703562+0000","last_clean":"2024-09-19T07:57:22.703562+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:57:22.703562+0000","last_undegraded":"2024-09-19T07:57:22.703562+0000","last_fullsized":"2024-09-19T07:57:22.703562+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509617,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87768,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284072,"statfs":{"total":96636764160,"available":96546889728,"internally_reserved":0,"allocated":2555904,"data_stored":1502277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":87292362},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509616,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":105300,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266540,"statfs":{"total":96636764160,"available":96528936960,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509617,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73296,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298544,"statfs":{"total":96636764160,"available":96561709056,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509617,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":87188,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284652,"statfs":{"total":96636764160,"available":96547483648,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509616,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105812,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266028,"statfs":{"total":96636764160,"available":96528412672,"internally_reserved":0,"allocated":2555904,"data_stored":1502277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542320,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":119124,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252716,"statfs":{"total":96636764160,"available":96514781184,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542320,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":83088,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288752,"statfs":{"total":96636764160,"available":96551682048,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575025,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109360,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262480,"statfs":{"total":96636764160,"available":96524779520,"internally_reserved":0,"allocated":1929216,"data_stored":899471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:57:24.755 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:57:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.857+0000 7f24ed781640 1 Processor -- start 2024-09-19T07:57:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.857+0000 7f24ed781640 1 -- start start 2024-09-19T07:57:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.858+0000 7f24ed781640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24e8155350 0x7f24e80b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.858+0000 7f24ed781640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24e80726e0 0x7f24e806e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.858+0000 7f24ed781640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24e8155730 con 0x7f24e806c750 2024-09-19T07:57:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.858+0000 7f24ed781640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f24e80d01f0 con 0x7f24e80726e0 2024-09-19T07:57:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.858+0000 7f24ed781640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f24e80d04d0 con 0x7f24e8155350 2024-09-19T07:57:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.858+0000 7f24e77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24e80726e0 0x7f24e806e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.858+0000 7f24e77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24e80726e0 0x7f24e806e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42238/0 (socket says 172.21.15.16:42238) 2024-09-19T07:57:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.858+0000 7f24e77fe640 1 -- 172.21.15.16:0/514322789 learned_addr learned my addr 172.21.15.16:0/514322789 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.858+0000 7f24e6ffd640 1 --2- 172.21.15.16:0/514322789 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24e8155350 0x7f24e80b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.858+0000 7f24e77fe640 1 -- 172.21.15.16:0/514322789 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24e8155350 msgr2=0x7f24e80b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.858+0000 7f24e77fe640 1 --2- 172.21.15.16:0/514322789 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24e8155350 0x7f24e80b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.858+0000 7f24e77fe640 1 -- 172.21.15.16:0/514322789 >> v1:172.21.15.146:6789/0 conn(0x7f24e806c750 legacy=0x7f24e8071fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:24.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.858+0000 7f24e77fe640 1 -- 172.21.15.16:0/514322789 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e80d07b0 con 0x7f24e80726e0 2024-09-19T07:57:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.859+0000 7f24e77fe640 1 --2- 172.21.15.16:0/514322789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e80726e0 0x7f24e806e0f0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009e50 tx=0x7f24e002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d8ad3f114ba60c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.859+0000 7f24e5ffb640 1 -- 172.21.15.16:0/514322789 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0036070 con 0x7f24e80726e0 2024-09-19T07:57:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.859+0000 7f24e5ffb640 1 -- 172.21.15.16:0/514322789 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e0031600 con 0x7f24e80726e0 2024-09-19T07:57:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.859+0000 7f24e5ffb640 1 -- 172.21.15.16:0/514322789 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e00319d0 con 0x7f24e80726e0 2024-09-19T07:57:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.860+0000 7f24ed781640 1 -- 172.21.15.16:0/514322789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e80726e0 msgr2=0x7f24e806e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.860+0000 7f24ed781640 1 --2- 172.21.15.16:0/514322789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e80726e0 0x7f24e806e0f0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009e50 tx=0x7f24e002a710 comp rx=0 tx=0).stop 2024-09-19T07:57:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.860+0000 7f24ed781640 1 -- 172.21.15.16:0/514322789 shutdown_connections 2024-09-19T07:57:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.860+0000 7f24ed781640 1 --2- 172.21.15.16:0/514322789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e80726e0 0x7f24e806e0f0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.860+0000 7f24ed781640 1 --2- 172.21.15.16:0/514322789 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24e8155350 0x7f24e80b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.860+0000 7f24ed781640 1 -- 172.21.15.16:0/514322789 >> 172.21.15.16:0/514322789 conn(0x7f24e8067d70 msgr2=0x7f24e806bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.860+0000 7f24ed781640 1 -- 172.21.15.16:0/514322789 shutdown_connections 2024-09-19T07:57:24.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.860+0000 7f24ed781640 1 -- 172.21.15.16:0/514322789 wait complete. 2024-09-19T07:57:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.861+0000 7f24ed781640 1 Processor -- start 2024-09-19T07:57:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.861+0000 7f24ed781640 1 -- start start 2024-09-19T07:57:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.862+0000 7f24ed781640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24e806c750 0x7f24e81f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.862+0000 7f24ed781640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e80726e0 0x7f24e81e7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.862+0000 7f24ed781640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24e8155350 0x7f24e81f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.862+0000 7f24ed781640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f24e80b43b0 con 0x7f24e80726e0 2024-09-19T07:57:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.862+0000 7f24ed781640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f24e80d0d70 con 0x7f24e806c750 2024-09-19T07:57:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.862+0000 7f24ed781640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f24e80b4530 con 0x7f24e8155350 2024-09-19T07:57:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.862+0000 7f24e67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e80726e0 0x7f24e81e7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.862+0000 7f24e67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e80726e0 0x7f24e81e7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42250/0 (socket says 172.21.15.16:42250) 2024-09-19T07:57:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.862+0000 7f24e67fc640 1 -- 172.21.15.16:0/4067645827 learned_addr learned my addr 172.21.15.16:0/4067645827 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.862+0000 7f24e77fe640 1 --2- 172.21.15.16:0/4067645827 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24e8155350 0x7f24e81f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.862+0000 7f24e6ffd640 1 --2- 172.21.15.16:0/4067645827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24e806c750 0x7f24e81f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.862+0000 7f24e67fc640 1 -- 172.21.15.16:0/4067645827 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24e8155350 msgr2=0x7f24e81f76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.862+0000 7f24e67fc640 1 --2- 172.21.15.16:0/4067645827 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24e8155350 0x7f24e81f76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.862+0000 7f24e67fc640 1 -- 172.21.15.16:0/4067645827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24e806c750 msgr2=0x7f24e81f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.862+0000 7f24e67fc640 1 --2- 172.21.15.16:0/4067645827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24e806c750 0x7f24e81f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.862+0000 7f24e67fc640 1 -- 172.21.15.16:0/4067645827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e80b0f40 con 0x7f24e80726e0 2024-09-19T07:57:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.862+0000 7f24e6ffd640 1 --2- 172.21.15.16:0/4067645827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24e806c750 0x7f24e81f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.863+0000 7f24e67fc640 1 --2- 172.21.15.16:0/4067645827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e80726e0 0x7f24e81e7c80 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00c3c0 tx=0x7f24dc00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.863+0000 7f24cf7fe640 1 -- 172.21.15.16:0/4067645827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc0052d0 con 0x7f24e80726e0 2024-09-19T07:57:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.863+0000 7f24cf7fe640 1 -- 172.21.15.16:0/4067645827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24dc005490 con 0x7f24e80726e0 2024-09-19T07:57:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.863+0000 7f24ed781640 1 -- 172.21.15.16:0/4067645827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e80b1220 con 0x7f24e80726e0 2024-09-19T07:57:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.863+0000 7f24cf7fe640 1 -- 172.21.15.16:0/4067645827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc007260 con 0x7f24e80726e0 2024-09-19T07:57:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.863+0000 7f24ed781640 1 -- 172.21.15.16:0/4067645827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e80b17e0 con 0x7f24e80726e0 2024-09-19T07:57:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.864+0000 7f24ed781640 1 -- 172.21.15.16:0/4067645827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e80b3e70 con 0x7f24e80726e0 2024-09-19T07:57:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.865+0000 7f24cf7fe640 1 -- 172.21.15.16:0/4067645827 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f24dc007400 con 0x7f24e80726e0 2024-09-19T07:57:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.865+0000 7f24cf7fe640 1 --2- 172.21.15.16:0/4067645827 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f24d406c2b0 0x7f24d406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.865+0000 7f24cf7fe640 1 -- 172.21.15.16:0/4067645827 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f24dc08a6d0 con 0x7f24e80726e0 2024-09-19T07:57:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.865+0000 7f24e6ffd640 1 --2- 172.21.15.16:0/4067645827 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f24d406c2b0 0x7f24d406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.866+0000 7f24e6ffd640 1 --2- 172.21.15.16:0/4067645827 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f24d406c2b0 0x7f24d406e770 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f24d8006fd0 tx=0x7f24d8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:24.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:24.870+0000 7f24cf7fe640 1 -- 172.21.15.16:0/4067645827 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f24dc0589c0 con 0x7f24e80726e0 2024-09-19T07:57:25.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.042+0000 7f24ed781640 1 -- 172.21.15.16:0/4067645827 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f24e8072150 con 0x7f24d406c2b0 2024-09-19T07:57:25.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.043+0000 7f24cf7fe640 1 -- 172.21.15.16:0/4067645827 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13463 (secure 0 0 0) 0x7f24e8072150 con 0x7f24d406c2b0 2024-09-19T07:57:25.044 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:25.044 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:57:25.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.046+0000 7f24ed781640 1 -- 172.21.15.16:0/4067645827 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f24d406c2b0 msgr2=0x7f24d406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:25.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.046+0000 7f24ed781640 1 --2- 172.21.15.16:0/4067645827 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f24d406c2b0 0x7f24d406e770 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f24d8006fd0 tx=0x7f24d8008040 comp rx=0 tx=0).stop 2024-09-19T07:57:25.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.046+0000 7f24ed781640 1 -- 172.21.15.16:0/4067645827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e80726e0 msgr2=0x7f24e81e7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:25.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.046+0000 7f24ed781640 1 --2- 172.21.15.16:0/4067645827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e80726e0 0x7f24e81e7c80 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00c3c0 tx=0x7f24dc00c7c0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.046+0000 7f24ed781640 1 -- 172.21.15.16:0/4067645827 shutdown_connections 2024-09-19T07:57:25.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.046+0000 7f24ed781640 1 --2- 172.21.15.16:0/4067645827 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f24d406c2b0 0x7f24d406e770 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.046+0000 7f24ed781640 1 --2- 172.21.15.16:0/4067645827 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24e8155350 0x7f24e81f76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.046+0000 7f24ed781640 1 --2- 172.21.15.16:0/4067645827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e80726e0 0x7f24e81e7c80 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.046+0000 7f24ed781640 1 --2- 172.21.15.16:0/4067645827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24e806c750 0x7f24e81f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.046+0000 7f24ed781640 1 -- 172.21.15.16:0/4067645827 >> 172.21.15.16:0/4067645827 conn(0x7f24e8067d70 msgr2=0x7f24e8153430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:25.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.046+0000 7f24ed781640 1 -- 172.21.15.16:0/4067645827 shutdown_connections 2024-09-19T07:57:25.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.046+0000 7f24ed781640 1 -- 172.21.15.16:0/4067645827 wait complete. 2024-09-19T07:57:25.057 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":586,"stamp":"2024-09-19T07:57:24.665369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":770936,"kb_used_data":16456,"kb_used_omap":12,"kb_used_meta":754291,"kb_avail":754203784,"statfs":{"total":773094113280,"available":772304674816,"internally_reserved":0,"allocated":16850944,"data_stored":8463570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":772394544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14414,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-66,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001328"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":417,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-19T07:57:22.703562+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:57:22.703562+0000","last_peered":"2024-09-19T07:57:22.703562+0000","last_clean":"2024-09-19T07:57:22.703562+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:57:22.703562+0000","last_undegraded":"2024-09-19T07:57:22.703562+0000","last_fullsized":"2024-09-19T07:57:22.703562+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509617,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87768,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284072,"statfs":{"total":96636764160,"available":96546889728,"internally_reserved":0,"allocated":2555904,"data_stored":1502277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":87292362},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509616,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":105300,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266540,"statfs":{"total":96636764160,"available":96528936960,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509617,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73296,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298544,"statfs":{"total":96636764160,"available":96561709056,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509617,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":87188,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284652,"statfs":{"total":96636764160,"available":96547483648,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509616,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105812,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266028,"statfs":{"total":96636764160,"available":96528412672,"internally_reserved":0,"allocated":2555904,"data_stored":1502277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542320,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":119124,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252716,"statfs":{"total":96636764160,"available":96514781184,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542320,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":83088,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288752,"statfs":{"total":96636764160,"available":96551682048,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575025,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109360,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262480,"statfs":{"total":96636764160,"available":96524779520,"internally_reserved":0,"allocated":1929216,"data_stored":899471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:57:25.058 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T07:57:25.058 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:57:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.159+0000 7fdcdd2d0640 1 Processor -- start 2024-09-19T07:57:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.159+0000 7fdcdd2d0640 1 -- start start 2024-09-19T07:57:25.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.160+0000 7fdcdd2d0640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdcd80d0e00 0x7fdcd80d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:25.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.160+0000 7fdcdd2d0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcd806cb30 0x7fdcd806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:25.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.160+0000 7fdcdd2d0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcd80d1760 con 0x7fdcd815e790 2024-09-19T07:57:25.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.160+0000 7fdcdd2d0640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fdcd80cf0a0 con 0x7fdcd80d0e00 2024-09-19T07:57:25.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.160+0000 7fdcdd2d0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd80cf380 con 0x7fdcd806cb30 2024-09-19T07:57:25.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.160+0000 7fdcd6ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdcd815e790 0x7fdcd80d4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60844/0 (socket says 172.21.15.16:60844) 2024-09-19T07:57:25.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.160+0000 7fdcd6ffd640 1 -- 172.21.15.16:0/582293126 learned_addr learned my addr 172.21.15.16:0/582293126 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:25.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.160+0000 7fdcd67fc640 1 --2- 172.21.15.16:0/582293126 >> v2:172.21.15.146:3300/0 conn(0x7fdcd80d0e00 0x7fdcd80d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:25.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.160+0000 7fdcd77fe640 1 --2- 172.21.15.16:0/582293126 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcd806cb30 0x7fdcd806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:25.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.160+0000 7fdcd5ffb640 1 -- 172.21.15.16:0/582293126 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3546477507 0 0) 0x7fdcd80d1760 con 0x7fdcd815e790 2024-09-19T07:57:25.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.160+0000 7fdcd5ffb640 1 -- 172.21.15.16:0/582293126 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcc4003520 con 0x7fdcd815e790 2024-09-19T07:57:25.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.161+0000 7fdcd67fc640 1 -- 172.21.15.16:0/582293126 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcd806cb30 msgr2=0x7fdcd806cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:25.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.161+0000 7fdcd67fc640 1 --2- 172.21.15.16:0/582293126 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcd806cb30 0x7fdcd806cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.161+0000 7fdcd67fc640 1 -- 172.21.15.16:0/582293126 >> v1:172.21.15.16:6789/0 conn(0x7fdcd815e790 legacy=0x7fdcd80d4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:25.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.161+0000 7fdcd5ffb640 1 -- 172.21.15.16:0/582293126 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3064487313 0 0) 0x7fdcc4003520 con 0x7fdcd815e790 2024-09-19T07:57:25.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.161+0000 7fdcd67fc640 1 -- 172.21.15.16:0/582293126 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd80cf660 con 0x7fdcd80d0e00 2024-09-19T07:57:25.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.161+0000 7fdcd77fe640 1 --2- 172.21.15.16:0/582293126 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcd806cb30 0x7fdcd806cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:57:25.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.161+0000 7fdcd67fc640 1 --2- 172.21.15.16:0/582293126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcd80d0e00 0x7fdcd80d1220 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00097a0 tx=0x7fdcc002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8bb0bbb7d1510f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:25.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.162+0000 7fdcd5ffb640 1 -- 172.21.15.16:0/582293126 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0037070 con 0x7fdcd80d0e00 2024-09-19T07:57:25.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.162+0000 7fdcd5ffb640 1 -- 172.21.15.16:0/582293126 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc002ac30 con 0x7fdcd80d0e00 2024-09-19T07:57:25.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.162+0000 7fdcd5ffb640 1 -- 172.21.15.16:0/582293126 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0001670 con 0x7fdcd80d0e00 2024-09-19T07:57:25.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.162+0000 7fdcdd2d0640 1 -- 172.21.15.16:0/582293126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcd80d0e00 msgr2=0x7fdcd80d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:25.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.162+0000 7fdcdd2d0640 1 --2- 172.21.15.16:0/582293126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcd80d0e00 0x7fdcd80d1220 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00097a0 tx=0x7fdcc002a1a0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.162+0000 7fdcdd2d0640 1 -- 172.21.15.16:0/582293126 shutdown_connections 2024-09-19T07:57:25.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.162+0000 7fdcdd2d0640 1 --2- 172.21.15.16:0/582293126 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcd806cb30 0x7fdcd806cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.162+0000 7fdcdd2d0640 1 --2- 172.21.15.16:0/582293126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcd80d0e00 0x7fdcd80d1220 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.162+0000 7fdcdd2d0640 1 -- 172.21.15.16:0/582293126 >> 172.21.15.16:0/582293126 conn(0x7fdcd809d740 msgr2=0x7fdcd80d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:25.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.162+0000 7fdcdd2d0640 1 -- 172.21.15.16:0/582293126 shutdown_connections 2024-09-19T07:57:25.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.162+0000 7fdcdd2d0640 1 -- 172.21.15.16:0/582293126 wait complete. 2024-09-19T07:57:25.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.164+0000 7fdcdd2d0640 1 Processor -- start 2024-09-19T07:57:25.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.164+0000 7fdcdd2d0640 1 -- start start 2024-09-19T07:57:25.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.164+0000 7fdcdd2d0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcd806cb30 0x7fdcd8153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:25.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.164+0000 7fdcdd2d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcd80d0e00 0x7fdcd815fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:25.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.164+0000 7fdcdd2d0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcd815e790 0x7fdcd8161f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:25.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.164+0000 7fdcdd2d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd80d0140 con 0x7fdcd80d0e00 2024-09-19T07:57:25.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.164+0000 7fdcdd2d0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd8162f30 con 0x7fdcd815e790 2024-09-19T07:57:25.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.164+0000 7fdcdd2d0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd80b95c0 con 0x7fdcd806cb30 2024-09-19T07:57:25.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.164+0000 7fdcd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcd80d0e00 0x7fdcd815fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:25.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.164+0000 7fdcd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcd80d0e00 0x7fdcd815fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42256/0 (socket says 172.21.15.16:42256) 2024-09-19T07:57:25.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.164+0000 7fdcd67fc640 1 -- 172.21.15.16:0/3279486400 learned_addr learned my addr 172.21.15.16:0/3279486400 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:25.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.164+0000 7fdcd6ffd640 1 --2- 172.21.15.16:0/3279486400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcd806cb30 0x7fdcd8153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:25.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.165+0000 7fdcd77fe640 1 --2- 172.21.15.16:0/3279486400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcd815e790 0x7fdcd8161f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:25.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.165+0000 7fdcd67fc640 1 -- 172.21.15.16:0/3279486400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcd806cb30 msgr2=0x7fdcd8153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:25.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.165+0000 7fdcd67fc640 1 --2- 172.21.15.16:0/3279486400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcd806cb30 0x7fdcd8153430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.165+0000 7fdcd67fc640 1 -- 172.21.15.16:0/3279486400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcd815e790 msgr2=0x7fdcd8161f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:25.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.165+0000 7fdcd67fc640 1 --2- 172.21.15.16:0/3279486400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcd815e790 0x7fdcd8161f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.165+0000 7fdcd67fc640 1 -- 172.21.15.16:0/3279486400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd80b8740 con 0x7fdcd80d0e00 2024-09-19T07:57:25.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.165+0000 7fdcd67fc640 1 --2- 172.21.15.16:0/3279486400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcd80d0e00 0x7fdcd815fb20 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00098c0 tx=0x7fdcc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:25.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.165+0000 7fdccf7fe640 1 -- 172.21.15.16:0/3279486400 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0041020 con 0x7fdcd80d0e00 2024-09-19T07:57:25.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.165+0000 7fdccf7fe640 1 -- 172.21.15.16:0/3279486400 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc003d620 con 0x7fdcd80d0e00 2024-09-19T07:57:25.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.165+0000 7fdcdd2d0640 1 -- 172.21.15.16:0/3279486400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd8160060 con 0x7fdcd80d0e00 2024-09-19T07:57:25.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.165+0000 7fdccf7fe640 1 -- 172.21.15.16:0/3279486400 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0037040 con 0x7fdcd80d0e00 2024-09-19T07:57:25.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.165+0000 7fdcdd2d0640 1 -- 172.21.15.16:0/3279486400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd80b8c90 con 0x7fdcd80d0e00 2024-09-19T07:57:25.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.166+0000 7fdcdd2d0640 1 -- 172.21.15.16:0/3279486400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcd80d1220 con 0x7fdcd80d0e00 2024-09-19T07:57:25.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.167+0000 7fdccf7fe640 1 -- 172.21.15.16:0/3279486400 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdcc0044020 con 0x7fdcd80d0e00 2024-09-19T07:57:25.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.167+0000 7fdccf7fe640 1 --2- 172.21.15.16:0/3279486400 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdcb806c2b0 0x7fdcb806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:25.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.167+0000 7fdccf7fe640 1 -- 172.21.15.16:0/3279486400 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7fdcc00ab610 con 0x7fdcd80d0e00 2024-09-19T07:57:25.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.167+0000 7fdcd6ffd640 1 --2- 172.21.15.16:0/3279486400 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdcb806c2b0 0x7fdcb806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:25.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.168+0000 7fdcd6ffd640 1 --2- 172.21.15.16:0/3279486400 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdcb806c2b0 0x7fdcb806e770 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0005880 tx=0x7fdcd00057f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.172+0000 7fdccf7fe640 1 -- 172.21.15.16:0/3279486400 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fdcc004b030 con 0x7fdcd80d0e00 2024-09-19T07:57:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.346+0000 7fdcdd2d0640 1 -- 172.21.15.16:0/3279486400 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdcd80b9770 con 0x7fdcb806c2b0 2024-09-19T07:57:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.347+0000 7fdccf7fe640 1 -- 172.21.15.16:0/3279486400 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13463 (secure 0 0 0) 0x7fdcd000f940 con 0x7fdcb806c2b0 2024-09-19T07:57:25.347 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:25.348 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:57:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.349+0000 7fdcdd2d0640 1 -- 172.21.15.16:0/3279486400 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdcb806c2b0 msgr2=0x7fdcb806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.349+0000 7fdcdd2d0640 1 --2- 172.21.15.16:0/3279486400 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdcb806c2b0 0x7fdcb806e770 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0005880 tx=0x7fdcd00057f0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.349+0000 7fdcdd2d0640 1 -- 172.21.15.16:0/3279486400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcd80d0e00 msgr2=0x7fdcd815fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.349+0000 7fdcdd2d0640 1 --2- 172.21.15.16:0/3279486400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcd80d0e00 0x7fdcd815fb20 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00098c0 tx=0x7fdcc00046c0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.349+0000 7fdcdd2d0640 1 -- 172.21.15.16:0/3279486400 shutdown_connections 2024-09-19T07:57:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.349+0000 7fdcdd2d0640 1 --2- 172.21.15.16:0/3279486400 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fdcb806c2b0 0x7fdcb806e770 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.349+0000 7fdcdd2d0640 1 --2- 172.21.15.16:0/3279486400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcd815e790 0x7fdcd8161f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.349+0000 7fdcdd2d0640 1 --2- 172.21.15.16:0/3279486400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcd80d0e00 0x7fdcd815fb20 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.349+0000 7fdcdd2d0640 1 --2- 172.21.15.16:0/3279486400 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcd806cb30 0x7fdcd8153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.349+0000 7fdcdd2d0640 1 -- 172.21.15.16:0/3279486400 >> 172.21.15.16:0/3279486400 conn(0x7fdcd809d740 msgr2=0x7fdcd815d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:25.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.350+0000 7fdcdd2d0640 1 -- 172.21.15.16:0/3279486400 shutdown_connections 2024-09-19T07:57:25.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.350+0000 7fdcdd2d0640 1 -- 172.21.15.16:0/3279486400 wait complete. 2024-09-19T07:57:25.360 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":586,"stamp":"2024-09-19T07:57:24.665369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":770936,"kb_used_data":16456,"kb_used_omap":12,"kb_used_meta":754291,"kb_avail":754203784,"statfs":{"total":773094113280,"available":772304674816,"internally_reserved":0,"allocated":16850944,"data_stored":8463570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":772394544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14414,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-66,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001328"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":417,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-19T07:57:22.703562+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:57:22.703562+0000","last_peered":"2024-09-19T07:57:22.703562+0000","last_clean":"2024-09-19T07:57:22.703562+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:57:22.703562+0000","last_undegraded":"2024-09-19T07:57:22.703562+0000","last_fullsized":"2024-09-19T07:57:22.703562+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509617,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87768,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284072,"statfs":{"total":96636764160,"available":96546889728,"internally_reserved":0,"allocated":2555904,"data_stored":1502277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":87292362},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509616,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":105300,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266540,"statfs":{"total":96636764160,"available":96528936960,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509617,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73296,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298544,"statfs":{"total":96636764160,"available":96561709056,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509617,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":87188,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284652,"statfs":{"total":96636764160,"available":96547483648,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509616,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105812,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266028,"statfs":{"total":96636764160,"available":96528412672,"internally_reserved":0,"allocated":2555904,"data_stored":1502277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542320,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":119124,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252716,"statfs":{"total":96636764160,"available":96514781184,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542320,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":83088,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288752,"statfs":{"total":96636764160,"available":96551682048,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575025,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109360,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262480,"statfs":{"total":96636764160,"available":96524779520,"internally_reserved":0,"allocated":1929216,"data_stored":899471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:57:25.361 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:57:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.462+0000 7f231a36d640 1 Processor -- start 2024-09-19T07:57:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.463+0000 7f231a36d640 1 -- start start 2024-09-19T07:57:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.463+0000 7f231a36d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f231415e500 0x7f231415e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.463+0000 7f231a36d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23140cfd90 con 0x7f23140cf040 2024-09-19T07:57:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.463+0000 7f231a36d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23140d0070 con 0x7f23140d41a0 2024-09-19T07:57:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.463+0000 7f231a36d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f23140d26a0 con 0x7f231415e500 2024-09-19T07:57:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.463+0000 7f231890a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f23140cf040 0x7f23140cf4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60854/0 (socket says 172.21.15.16:60854) 2024-09-19T07:57:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.463+0000 7f231890a640 1 -- 172.21.15.16:0/4264511691 learned_addr learned my addr 172.21.15.16:0/4264511691 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.464+0000 7f2313fff640 1 --2- 172.21.15.16:0/4264511691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f231415e500 0x7f231415e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.464+0000 7f2312ffd640 1 -- 172.21.15.16:0/4264511691 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3693861975 0 0) 0x7f23140cfd90 con 0x7f23140cf040 2024-09-19T07:57:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.464+0000 7f2312ffd640 1 -- 172.21.15.16:0/4264511691 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22fc003520 con 0x7f23140cf040 2024-09-19T07:57:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.464+0000 7f2313fff640 1 -- 172.21.15.16:0/4264511691 >> v1:172.21.15.146:6789/0 conn(0x7f23140d41a0 legacy=0x7f23140d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.464+0000 7f2312ffd640 1 -- 172.21.15.16:0/4264511691 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3681514931 0 0) 0x7f23140d0070 con 0x7f23140d41a0 2024-09-19T07:57:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.464+0000 7f2313fff640 1 -- 172.21.15.16:0/4264511691 >> v1:172.21.15.16:6789/0 conn(0x7f23140cf040 legacy=0x7f23140cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.464+0000 7f2313fff640 1 -- 172.21.15.16:0/4264511691 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23140d0350 con 0x7f231415e500 2024-09-19T07:57:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.465+0000 7f2313fff640 1 --2- 172.21.15.16:0/4264511691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f231415e500 0x7f231415e8e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f23040097d0 tx=0x7f230402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca8cbae210530b95 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.465+0000 7f2312ffd640 1 -- 172.21.15.16:0/4264511691 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304037070 con 0x7f231415e500 2024-09-19T07:57:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.465+0000 7f2312ffd640 1 -- 172.21.15.16:0/4264511691 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2304032440 con 0x7f231415e500 2024-09-19T07:57:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.465+0000 7f2312ffd640 1 -- 172.21.15.16:0/4264511691 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23040327f0 con 0x7f231415e500 2024-09-19T07:57:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.465+0000 7f231a36d640 1 -- 172.21.15.16:0/4264511691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f231415e500 msgr2=0x7f231415e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.465+0000 7f231a36d640 1 --2- 172.21.15.16:0/4264511691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f231415e500 0x7f231415e8e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f23040097d0 tx=0x7f230402a210 comp rx=0 tx=0).stop 2024-09-19T07:57:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.465+0000 7f231a36d640 1 -- 172.21.15.16:0/4264511691 shutdown_connections 2024-09-19T07:57:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.466+0000 7f231a36d640 1 --2- 172.21.15.16:0/4264511691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f231415e500 0x7f231415e8e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.466+0000 7f231a36d640 1 -- 172.21.15.16:0/4264511691 >> 172.21.15.16:0/4264511691 conn(0x7f231408bf90 msgr2=0x7f231409d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.466+0000 7f231a36d640 1 -- 172.21.15.16:0/4264511691 shutdown_connections 2024-09-19T07:57:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.466+0000 7f231a36d640 1 -- 172.21.15.16:0/4264511691 wait complete. 2024-09-19T07:57:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.467+0000 7f231a36d640 1 Processor -- start 2024-09-19T07:57:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.467+0000 7f231a36d640 1 -- start start 2024-09-19T07:57:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f231a36d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23140cf040 0x7f23141f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f231a36d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23140d41a0 0x7f23140d2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f231a36d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f231415e500 0x7f23140d2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f231a36d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23140b5990 con 0x7f23140d41a0 2024-09-19T07:57:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f231a36d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f23140b44f0 con 0x7f23140cf040 2024-09-19T07:57:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f231a36d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f23140b5b10 con 0x7f231415e500 2024-09-19T07:57:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f23137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23140d41a0 0x7f23140d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f23137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23140d41a0 0x7f23140d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42266/0 (socket says 172.21.15.16:42266) 2024-09-19T07:57:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f23137fe640 1 -- 172.21.15.16:0/4280587162 learned_addr learned my addr 172.21.15.16:0/4280587162 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f2313fff640 1 --2- 172.21.15.16:0/4280587162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23140cf040 0x7f23141f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f231890a640 1 --2- 172.21.15.16:0/4280587162 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f231415e500 0x7f23140d2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f23137fe640 1 -- 172.21.15.16:0/4280587162 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f231415e500 msgr2=0x7f23140d2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f23137fe640 1 --2- 172.21.15.16:0/4280587162 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f231415e500 0x7f23140d2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f23137fe640 1 -- 172.21.15.16:0/4280587162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23140cf040 msgr2=0x7f23141f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f23137fe640 1 --2- 172.21.15.16:0/4280587162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23140cf040 0x7f23141f3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f23137fe640 1 -- 172.21.15.16:0/4280587162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231415f0a0 con 0x7f23140d41a0 2024-09-19T07:57:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f23137fe640 1 --2- 172.21.15.16:0/4280587162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23140d41a0 0x7f23140d2820 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f230800c3e0 tx=0x7f230800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f2310ff9640 1 -- 172.21.15.16:0/4280587162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308017070 con 0x7f23140d41a0 2024-09-19T07:57:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f2310ff9640 1 -- 172.21.15.16:0/4280587162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23080035f0 con 0x7f23140d41a0 2024-09-19T07:57:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f231a36d640 1 -- 172.21.15.16:0/4280587162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231415f380 con 0x7f23140d41a0 2024-09-19T07:57:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f2310ff9640 1 -- 172.21.15.16:0/4280587162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308003a00 con 0x7f23140d41a0 2024-09-19T07:57:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.468+0000 7f231a36d640 1 -- 172.21.15.16:0/4280587162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f231415f940 con 0x7f23140d41a0 2024-09-19T07:57:25.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.470+0000 7f231a36d640 1 -- 172.21.15.16:0/4280587162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f231415e8e0 con 0x7f23140d41a0 2024-09-19T07:57:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.470+0000 7f2310ff9640 1 -- 172.21.15.16:0/4280587162 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f230801a070 con 0x7f23140d41a0 2024-09-19T07:57:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.470+0000 7f2310ff9640 1 --2- 172.21.15.16:0/4280587162 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f230006c2e0 0x7f230006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.471+0000 7f2310ff9640 1 -- 172.21.15.16:0/4280587162 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f230808a6e0 con 0x7f23140d41a0 2024-09-19T07:57:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.471+0000 7f2313fff640 1 --2- 172.21.15.16:0/4280587162 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f230006c2e0 0x7f230006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.471+0000 7f2313fff640 1 --2- 172.21.15.16:0/4280587162 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f230006c2e0 0x7f230006e7a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2304034000 tx=0x7f230402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.475+0000 7f2310ff9640 1 -- 172.21.15.16:0/4280587162 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2308058f90 con 0x7f23140d41a0 2024-09-19T07:57:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.651+0000 7f231a36d640 1 -- 172.21.15.16:0/4280587162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23140c78f0 con 0x7f23140d41a0 2024-09-19T07:57:25.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.652+0000 7f2310ff9640 1 -- 172.21.15.16:0/4280587162 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+13400 (secure 0 0 0) 0x7f2308058890 con 0x7f23140d41a0 2024-09-19T07:57:25.652 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:25.652 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":156,"fsid":"6abb3d9f-e4b1-4736-a5b9-ddf31a1f81eb","created":"2024-09-19T07:42:42.301106+0000","modified":"2024-09-19T07:57:22.691129+0000","last_up_change":"2024-09-19T07:42:53.935994+0000","last_in_change":"2024-09-19T07:42:44.371446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:42:54.500338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90068366-8d2f-411b-a051-3eb4e2cc6032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6809","nonce":2159059174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6811","nonce":2159059174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6815","nonce":2159059174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2159059174},{"type":"v1","addr":"172.21.15.16:6813","nonce":2159059174}]},"public_addr":"172.21.15.16:6809/2159059174","cluster_addr":"172.21.15.16:6811/2159059174","heartbeat_back_addr":"172.21.15.16:6815/2159059174","heartbeat_front_addr":"172.21.15.16:6813/2159059174","state":["exists","up"]},{"osd":1,"uuid":"df8b4cc6-fe35-4f67-9422-260cb7702f5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6825","nonce":3030558062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6827","nonce":3030558062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6831","nonce":3030558062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3030558062},{"type":"v1","addr":"172.21.15.16:6829","nonce":3030558062}]},"public_addr":"172.21.15.16:6825/3030558062","cluster_addr":"172.21.15.16:6827/3030558062","heartbeat_back_addr":"172.21.15.16:6831/3030558062","heartbeat_front_addr":"172.21.15.16:6829/3030558062","state":["exists","up"]},{"osd":2,"uuid":"049b9893-2f23-45f9-af8e-2e70d2f7ceb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6801","nonce":4074420552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6803","nonce":4074420552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6807","nonce":4074420552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4074420552},{"type":"v1","addr":"172.21.15.16:6805","nonce":4074420552}]},"public_addr":"172.21.15.16:6801/4074420552","cluster_addr":"172.21.15.16:6803/4074420552","heartbeat_back_addr":"172.21.15.16:6807/4074420552","heartbeat_front_addr":"172.21.15.16:6805/4074420552","state":["exists","up"]},{"osd":3,"uuid":"1c8063a9-686f-4a4d-9604-41ddd6ee0660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6817","nonce":3748453908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6819","nonce":3748453908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6823","nonce":3748453908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3748453908},{"type":"v1","addr":"172.21.15.16:6821","nonce":3748453908}]},"public_addr":"172.21.15.16:6817/3748453908","cluster_addr":"172.21.15.16:6819/3748453908","heartbeat_back_addr":"172.21.15.16:6823/3748453908","heartbeat_front_addr":"172.21.15.16:6821/3748453908","state":["exists","up"]},{"osd":4,"uuid":"0e08bb0f-6f52-4c20-b8d2-8716f38382aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6809","nonce":2073273152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6811","nonce":2073273152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6815","nonce":2073273152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":2073273152},{"type":"v1","addr":"172.21.15.146:6813","nonce":2073273152}]},"public_addr":"172.21.15.146:6809/2073273152","cluster_addr":"172.21.15.146:6811/2073273152","heartbeat_back_addr":"172.21.15.146:6815/2073273152","heartbeat_front_addr":"172.21.15.146:6813/2073273152","state":["exists","up"]},{"osd":5,"uuid":"bebed515-c757-4a04-9890-1e2cfc521983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6817","nonce":2651212753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6819","nonce":2651212753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6823","nonce":2651212753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2651212753},{"type":"v1","addr":"172.21.15.146:6821","nonce":2651212753}]},"public_addr":"172.21.15.146:6817/2651212753","cluster_addr":"172.21.15.146:6819/2651212753","heartbeat_back_addr":"172.21.15.146:6823/2651212753","heartbeat_front_addr":"172.21.15.146:6821/2651212753","state":["exists","up"]},{"osd":6,"uuid":"676c8305-aaee-45b7-9c3a-ad1a35785d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6801","nonce":2692587126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6803","nonce":2692587126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6807","nonce":2692587126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2692587126},{"type":"v1","addr":"172.21.15.146:6805","nonce":2692587126}]},"public_addr":"172.21.15.146:6801/2692587126","cluster_addr":"172.21.15.146:6803/2692587126","heartbeat_back_addr":"172.21.15.146:6807/2692587126","heartbeat_front_addr":"172.21.15.146:6805/2692587126","state":["exists","up"]},{"osd":7,"uuid":"afdf6bda-674b-4dfb-a698-ba28e38a22e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6825","nonce":1757996733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6827","nonce":1757996733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6831","nonce":1757996733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1757996733},{"type":"v1","addr":"172.21.15.146:6829","nonce":1757996733}]},"public_addr":"172.21.15.146:6825/1757996733","cluster_addr":"172.21.15.146:6827/1757996733","heartbeat_back_addr":"172.21.15.146:6831/1757996733","heartbeat_front_addr":"172.21.15.146:6829/1757996733","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.994223+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.670627+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.231381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.948467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.663649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.988055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:50.917678+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:42:51.046096+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/3705993654":"2024-09-20T07:57:19.686613+0000","172.21.15.16:6833/3705993654":"2024-09-20T07:57:19.686613+0000","192.168.0.1:0/2801864998":"2024-09-19T08:57:13.688556+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:57:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.654+0000 7f231a36d640 1 -- 172.21.15.16:0/4280587162 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f230006c2e0 msgr2=0x7f230006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.654+0000 7f231a36d640 1 --2- 172.21.15.16:0/4280587162 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f230006c2e0 0x7f230006e7a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2304034000 tx=0x7f230402e000 comp rx=0 tx=0).stop 2024-09-19T07:57:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.654+0000 7f231a36d640 1 -- 172.21.15.16:0/4280587162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23140d41a0 msgr2=0x7f23140d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.654+0000 7f231a36d640 1 --2- 172.21.15.16:0/4280587162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23140d41a0 0x7f23140d2820 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f230800c3e0 tx=0x7f230800c8a0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.654+0000 7f231a36d640 1 -- 172.21.15.16:0/4280587162 shutdown_connections 2024-09-19T07:57:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.654+0000 7f231a36d640 1 --2- 172.21.15.16:0/4280587162 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f230006c2e0 0x7f230006e7a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.654+0000 7f231a36d640 1 --2- 172.21.15.16:0/4280587162 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f231415e500 0x7f23140d2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.654+0000 7f231a36d640 1 --2- 172.21.15.16:0/4280587162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23140d41a0 0x7f23140d2820 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.654+0000 7f231a36d640 1 --2- 172.21.15.16:0/4280587162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23140cf040 0x7f23141f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.654+0000 7f231a36d640 1 -- 172.21.15.16:0/4280587162 >> 172.21.15.16:0/4280587162 conn(0x7f231408bf90 msgr2=0x7f23140d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.654+0000 7f231a36d640 1 -- 172.21.15.16:0/4280587162 shutdown_connections 2024-09-19T07:57:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:25.654+0000 7f231a36d640 1 -- 172.21.15.16:0/4280587162 wait complete. 2024-09-19T07:57:26.666 INFO:tasks.ceph:Scrubbing osd.0 2024-09-19T07:57:26.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-19T07:57:26.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.787+0000 7f1d0afc0640 1 Processor -- start 2024-09-19T07:57:26.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.787+0000 7f1d0afc0640 1 -- start start 2024-09-19T07:57:26.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.788+0000 7f1d0afc0640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1d040cbab0 0x7f1d04165cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.788+0000 7f1d0afc0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d0415b5b0 con 0x7f1d0406f360 2024-09-19T07:57:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.788+0000 7f1d0afc0640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d040ceb90 con 0x7f1d0415b1d0 2024-09-19T07:57:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.788+0000 7f1d0afc0640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f1d040cee30 con 0x7f1d040cbab0 2024-09-19T07:57:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.788+0000 7f1d03fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1d0406f360 0x7f1d040cb3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60866/0 (socket says 172.21.15.16:60866) 2024-09-19T07:57:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.788+0000 7f1d03fff640 1 -- 172.21.15.16:0/667919230 learned_addr learned my addr 172.21.15.16:0/667919230 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.789+0000 7f1d037fe640 1 -- 172.21.15.16:0/667919230 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2009615748 0 0) 0x7f1d0415b5b0 con 0x7f1d0406f360 2024-09-19T07:57:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.789+0000 7f1d037fe640 1 -- 172.21.15.16:0/667919230 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cf0003520 con 0x7f1d0406f360 2024-09-19T07:57:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.789+0000 7f1d0955d640 1 --2- 172.21.15.16:0/667919230 >> v2:172.21.15.146:3300/0 conn(0x7f1d040cbab0 0x7f1d04165cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.789+0000 7f1d037fe640 1 -- 172.21.15.16:0/667919230 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1752690311 0 0) 0x7f1cf0003520 con 0x7f1d0406f360 2024-09-19T07:57:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.789+0000 7f1d037fe640 1 -- 172.21.15.16:0/667919230 >> v2:172.21.15.146:3300/0 conn(0x7f1d040cbab0 msgr2=0x7f1d04165cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.789+0000 7f1d037fe640 1 --2- 172.21.15.16:0/667919230 >> v2:172.21.15.146:3300/0 conn(0x7f1d040cbab0 0x7f1d04165cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.789+0000 7f1d037fe640 1 -- 172.21.15.16:0/667919230 >> v1:172.21.15.146:6789/0 conn(0x7f1d0415b1d0 legacy=0x7f1d0406ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.789+0000 7f1d037fe640 1 -- 172.21.15.16:0/667919230 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d040cf110 con 0x7f1d0406f360 2024-09-19T07:57:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.789+0000 7f1d037fe640 1 -- 172.21.15.16:0/667919230 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f1cf4001670 con 0x7f1d0406f360 2024-09-19T07:57:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.789+0000 7f1d037fe640 1 -- 172.21.15.16:0/667919230 >> v1:172.21.15.16:6789/0 conn(0x7f1d0406f360 legacy=0x7f1d040cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.789+0000 7f1d037fe640 1 --2- 172.21.15.16:0/667919230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf00039b0 0x7f1cf0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.789+0000 7f1d037fe640 1 --2- 172.21.15.16:0/667919230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1cf0004800 0x7f1cf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.789+0000 7f1d037fe640 1 --2- 172.21.15.16:0/667919230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cf00051c0 0x7f1cf000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.789+0000 7f1d037fe640 1 -- 172.21.15.16:0/667919230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1d040cee30 con 0x7f1cf00039b0 2024-09-19T07:57:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.789+0000 7f1d037fe640 1 -- 172.21.15.16:0/667919230 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf000c060 con 0x7f1cf00051c0 2024-09-19T07:57:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.789+0000 7f1d037fe640 1 -- 172.21.15.16:0/667919230 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf000c260 con 0x7f1cf0004800 2024-09-19T07:57:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.789+0000 7f1d08d5c640 1 --2- 172.21.15.16:0/667919230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf00039b0 0x7f1cf0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.789+0000 7f1d0955d640 1 --2- 172.21.15.16:0/667919230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cf00051c0 0x7f1cf000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.790+0000 7f1d08d5c640 1 -- 172.21.15.16:0/667919230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1cf0004800 msgr2=0x7f1cf0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.790+0000 7f1d08d5c640 1 --2- 172.21.15.16:0/667919230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1cf0004800 0x7f1cf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.790+0000 7f1d08d5c640 1 -- 172.21.15.16:0/667919230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cf00051c0 msgr2=0x7f1cf000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.790+0000 7f1d08d5c640 1 --2- 172.21.15.16:0/667919230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cf00051c0 0x7f1cf000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.790+0000 7f1d08d5c640 1 -- 172.21.15.16:0/667919230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf0003520 con 0x7f1cf00039b0 2024-09-19T07:57:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.790+0000 7f1d0955d640 1 --2- 172.21.15.16:0/667919230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cf00051c0 0x7f1cf000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.790+0000 7f1d0955d640 1 -- 172.21.15.16:0/667919230 reap_dead start 2024-09-19T07:57:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.790+0000 7f1d08d5c640 1 --2- 172.21.15.16:0/667919230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf00039b0 0x7f1cf0003dd0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f1cec001750 tx=0x7f1cf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d6252092e90f10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.790+0000 7f1d037fe640 1 -- 172.21.15.16:0/667919230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec001370 con 0x7f1cf00039b0 2024-09-19T07:57:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.790+0000 7f1d037fe640 1 -- 172.21.15.16:0/667919230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cec0052d0 con 0x7f1cf00039b0 2024-09-19T07:57:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.790+0000 7f1d037fe640 1 -- 172.21.15.16:0/667919230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec02abc0 con 0x7f1cf00039b0 2024-09-19T07:57:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.791+0000 7f1d0afc0640 1 -- 172.21.15.16:0/667919230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf00039b0 msgr2=0x7f1cf0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.791+0000 7f1d0afc0640 1 --2- 172.21.15.16:0/667919230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf00039b0 0x7f1cf0003dd0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f1cec001750 tx=0x7f1cf0005830 comp rx=0 tx=0).stop 2024-09-19T07:57:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.791+0000 7f1d0afc0640 1 -- 172.21.15.16:0/667919230 shutdown_connections 2024-09-19T07:57:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.791+0000 7f1d0afc0640 1 --2- 172.21.15.16:0/667919230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf00039b0 0x7f1cf0003dd0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.791+0000 7f1d0afc0640 1 -- 172.21.15.16:0/667919230 >> 172.21.15.16:0/667919230 conn(0x7f1d040b1d80 msgr2=0x7f1d040b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.791+0000 7f1d0afc0640 1 -- 172.21.15.16:0/667919230 shutdown_connections 2024-09-19T07:57:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.791+0000 7f1d0afc0640 1 -- 172.21.15.16:0/667919230 wait complete. 2024-09-19T07:57:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.792+0000 7f1d0afc0640 1 Processor -- start 2024-09-19T07:57:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.792+0000 7f1d0afc0640 1 -- start start 2024-09-19T07:57:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.793+0000 7f1d0afc0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf00039b0 0x7f1d0415cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.793+0000 7f1d0afc0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d0415b5b0 0x7f1d040c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.793+0000 7f1d0afc0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d04156660 0x7f1d040b5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.793+0000 7f1d0afc0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1d040cb3a0 con 0x7f1cf00039b0 2024-09-19T07:57:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.793+0000 7f1d0afc0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1d040cfbf0 con 0x7f1d04156660 2024-09-19T07:57:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.793+0000 7f1d0afc0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1d040cb520 con 0x7f1d0415b5b0 2024-09-19T07:57:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.793+0000 7f1d08d5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf00039b0 0x7f1d0415cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.793+0000 7f1d08d5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf00039b0 0x7f1d0415cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42288/0 (socket says 172.21.15.16:42288) 2024-09-19T07:57:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.793+0000 7f1d08d5c640 1 -- 172.21.15.16:0/3115518962 learned_addr learned my addr 172.21.15.16:0/3115518962 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.793+0000 7f1d03fff640 1 --2- 172.21.15.16:0/3115518962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d0415b5b0 0x7f1d040c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.794+0000 7f1d0955d640 1 --2- 172.21.15.16:0/3115518962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d04156660 0x7f1d040b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.794+0000 7f1d08d5c640 1 -- 172.21.15.16:0/3115518962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d0415b5b0 msgr2=0x7f1d040c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.794+0000 7f1d08d5c640 1 --2- 172.21.15.16:0/3115518962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d0415b5b0 0x7f1d040c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.794+0000 7f1d08d5c640 1 -- 172.21.15.16:0/3115518962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d04156660 msgr2=0x7f1d040b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.794+0000 7f1d08d5c640 1 --2- 172.21.15.16:0/3115518962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d04156660 0x7f1d040b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.794+0000 7f1d08d5c640 1 -- 172.21.15.16:0/3115518962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0415bb20 con 0x7f1cf00039b0 2024-09-19T07:57:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.794+0000 7f1d0955d640 1 --2- 172.21.15.16:0/3115518962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d04156660 0x7f1d040b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.794+0000 7f1d08d5c640 1 --2- 172.21.15.16:0/3115518962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf00039b0 0x7f1d0415cbc0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f1cec000c00 tx=0x7f1cec001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.794+0000 7f1d017fa640 1 -- 172.21.15.16:0/3115518962 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec03f030 con 0x7f1cf00039b0 2024-09-19T07:57:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.794+0000 7f1d017fa640 1 -- 172.21.15.16:0/3115518962 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cec005480 con 0x7f1cf00039b0 2024-09-19T07:57:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.794+0000 7f1d017fa640 1 -- 172.21.15.16:0/3115518962 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec037040 con 0x7f1cf00039b0 2024-09-19T07:57:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.794+0000 7f1d0afc0640 1 -- 172.21.15.16:0/3115518962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0415be00 con 0x7f1cf00039b0 2024-09-19T07:57:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.794+0000 7f1d0afc0640 1 -- 172.21.15.16:0/3115518962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d0415c3c0 con 0x7f1cf00039b0 2024-09-19T07:57:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.795+0000 7f1d0afc0640 1 -- 172.21.15.16:0/3115518962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1d040cc9e0 con 0x7f1cf00039b0 2024-09-19T07:57:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.796+0000 7f1d017fa640 1 -- 172.21.15.16:0/3115518962 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1cec042070 con 0x7f1cf00039b0 2024-09-19T07:57:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.796+0000 7f1d017fa640 1 --2- 172.21.15.16:0/3115518962 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1ce406c2e0 0x7f1ce406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.796+0000 7f1d017fa640 1 -- 172.21.15.16:0/3115518962 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f1cec0aa7a0 con 0x7f1cf00039b0 2024-09-19T07:57:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.796+0000 7f1d017fa640 1 --2- 172.21.15.16:0/3115518962 >> [v2:172.21.15.16:6808/2159059174,v1:172.21.15.16:6809/2159059174] conn(0x7f1ce40727d0 0x7f1ce4074c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.796+0000 7f1d017fa640 1 -- 172.21.15.16:0/3115518962 --> [v2:172.21.15.16:6808/2159059174,v1:172.21.15.16:6809/2159059174] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1cec004e90 con 0x7f1ce40727d0 2024-09-19T07:57:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.796+0000 7f1d017fa640 1 -- 172.21.15.16:0/3115518962 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=156) ==== 24+0+0 (secure 0 0 0) 0x7f1cec0af2c0 con 0x7f1cf00039b0 2024-09-19T07:57:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.797+0000 7f1d0955d640 1 --2- 172.21.15.16:0/3115518962 >> [v2:172.21.15.16:6808/2159059174,v1:172.21.15.16:6809/2159059174] conn(0x7f1ce40727d0 0x7f1ce4074c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.797+0000 7f1d03fff640 1 --2- 172.21.15.16:0/3115518962 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1ce406c2e0 0x7f1ce406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.797+0000 7f1d0955d640 1 --2- 172.21.15.16:0/3115518962 >> [v2:172.21.15.16:6808/2159059174,v1:172.21.15.16:6809/2159059174] conn(0x7f1ce40727d0 0x7f1ce4074c10 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.797+0000 7f1d03fff640 1 --2- 172.21.15.16:0/3115518962 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1ce406c2e0 0x7f1ce406e7a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40011e0 tx=0x7f1cf4001a40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.798+0000 7f1d017fa640 1 -- 172.21.15.16:0/3115518962 <== osd.0 v2:172.21.15.16:6808/2159059174 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1cec004e90 con 0x7f1ce40727d0 2024-09-19T07:57:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.810+0000 7f1d0afc0640 1 -- 172.21.15.16:0/3115518962 --> [v2:172.21.15.16:6808/2159059174,v1:172.21.15.16:6809/2159059174] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1d040c7340 con 0x7f1ce40727d0 2024-09-19T07:57:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.811+0000 7f1d017fa640 1 -- 172.21.15.16:0/3115518962 <== osd.0 v2:172.21.15.16:6808/2159059174 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1d040c7340 con 0x7f1ce40727d0 2024-09-19T07:57:26.811 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:57:26.811 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T07:57:26.811 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:57:26.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.811+0000 7f1d0afc0640 1 -- 172.21.15.16:0/3115518962 >> [v2:172.21.15.16:6808/2159059174,v1:172.21.15.16:6809/2159059174] conn(0x7f1ce40727d0 msgr2=0x7f1ce4074c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:26.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.811+0000 7f1d0afc0640 1 --2- 172.21.15.16:0/3115518962 >> [v2:172.21.15.16:6808/2159059174,v1:172.21.15.16:6809/2159059174] conn(0x7f1ce40727d0 0x7f1ce4074c10 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:26.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.811+0000 7f1d0afc0640 1 -- 172.21.15.16:0/3115518962 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1ce406c2e0 msgr2=0x7f1ce406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:26.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.811+0000 7f1d0afc0640 1 --2- 172.21.15.16:0/3115518962 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1ce406c2e0 0x7f1ce406e7a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40011e0 tx=0x7f1cf4001a40 comp rx=0 tx=0).stop 2024-09-19T07:57:26.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.811+0000 7f1d0afc0640 1 -- 172.21.15.16:0/3115518962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf00039b0 msgr2=0x7f1d0415cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:26.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.811+0000 7f1d0afc0640 1 --2- 172.21.15.16:0/3115518962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf00039b0 0x7f1d0415cbc0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f1cec000c00 tx=0x7f1cec001120 comp rx=0 tx=0).stop 2024-09-19T07:57:26.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.811+0000 7f1d0955d640 1 -- 172.21.15.16:0/3115518962 reap_dead start 2024-09-19T07:57:26.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.811+0000 7f1d0afc0640 1 -- 172.21.15.16:0/3115518962 shutdown_connections 2024-09-19T07:57:26.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.811+0000 7f1d0afc0640 1 -- 172.21.15.16:0/3115518962 >> 172.21.15.16:0/3115518962 conn(0x7f1d040b1d80 msgr2=0x7f1d04072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:26.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.811+0000 7f1d0afc0640 1 -- 172.21.15.16:0/3115518962 shutdown_connections 2024-09-19T07:57:26.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.811+0000 7f1d0afc0640 1 -- 172.21.15.16:0/3115518962 wait complete. 2024-09-19T07:57:26.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-19T07:57:26.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.926+0000 7f094748d640 1 Processor -- start 2024-09-19T07:57:26.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.926+0000 7f094748d640 1 -- start start 2024-09-19T07:57:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.927+0000 7f094748d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f094015c550 0x7f09400d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.927+0000 7f094748d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09401609b0 0x7f09400cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.927+0000 7f094748d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09400ce860 con 0x7f09400cf800 2024-09-19T07:57:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.927+0000 7f094748d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f09400ceae0 con 0x7f094015c550 2024-09-19T07:57:26.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.927+0000 7f094748d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0940160e10 con 0x7f09401609b0 2024-09-19T07:57:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.927+0000 7f0945229640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f094015c550 0x7f09400d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.927+0000 7f0945a2a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f09400cf800 0x7f09400b5090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60882/0 (socket says 172.21.15.16:60882) 2024-09-19T07:57:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.927+0000 7f0945a2a640 1 -- 172.21.15.16:0/3517050260 learned_addr learned my addr 172.21.15.16:0/3517050260 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.928+0000 7f0945229640 1 -- 172.21.15.16:0/3517050260 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09401609b0 msgr2=0x7f09400cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.928+0000 7f0945229640 1 --2- 172.21.15.16:0/3517050260 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09401609b0 0x7f09400cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.928+0000 7f0945229640 1 -- 172.21.15.16:0/3517050260 >> v1:172.21.15.16:6789/0 conn(0x7f09400cf800 legacy=0x7f09400b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.928+0000 7f093ffff640 1 -- 172.21.15.16:0/3517050260 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1769642762 0 0) 0x7f0930001490 con 0x7f09400cf800 2024-09-19T07:57:26.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.928+0000 7f0945229640 1 -- 172.21.15.16:0/3517050260 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09400d27b0 con 0x7f094015c550 2024-09-19T07:57:26.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.929+0000 7f0945229640 1 --2- 172.21.15.16:0/3517050260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f094015c550 0x7f09400d21d0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f09380097a0 tx=0x7f093802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84e9cca80f8c2df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.929+0000 7f093ffff640 1 -- 172.21.15.16:0/3517050260 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938037070 con 0x7f094015c550 2024-09-19T07:57:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.929+0000 7f093ffff640 1 -- 172.21.15.16:0/3517050260 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f093802ae50 con 0x7f094015c550 2024-09-19T07:57:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.929+0000 7f093ffff640 1 -- 172.21.15.16:0/3517050260 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938032660 con 0x7f094015c550 2024-09-19T07:57:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.929+0000 7f094748d640 1 -- 172.21.15.16:0/3517050260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f094015c550 msgr2=0x7f09400d21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.929+0000 7f094748d640 1 --2- 172.21.15.16:0/3517050260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f094015c550 0x7f09400d21d0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f09380097a0 tx=0x7f093802a100 comp rx=0 tx=0).stop 2024-09-19T07:57:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.929+0000 7f094748d640 1 -- 172.21.15.16:0/3517050260 shutdown_connections 2024-09-19T07:57:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.929+0000 7f094748d640 1 --2- 172.21.15.16:0/3517050260 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09401609b0 0x7f09400cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.929+0000 7f094748d640 1 --2- 172.21.15.16:0/3517050260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f094015c550 0x7f09400d21d0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.929+0000 7f094748d640 1 -- 172.21.15.16:0/3517050260 >> 172.21.15.16:0/3517050260 conn(0x7f094006feb0 msgr2=0x7f09400702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.929+0000 7f094748d640 1 -- 172.21.15.16:0/3517050260 shutdown_connections 2024-09-19T07:57:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.929+0000 7f094748d640 1 -- 172.21.15.16:0/3517050260 wait complete. 2024-09-19T07:57:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.931+0000 7f094748d640 1 Processor -- start 2024-09-19T07:57:26.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.931+0000 7f094748d640 1 -- start start 2024-09-19T07:57:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.931+0000 7f094748d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09400cf800 0x7f0940154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.931+0000 7f094748d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f094015c550 0x7f09400c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.931+0000 7f094748d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09401609b0 0x7f09400c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.931+0000 7f094748d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09400b57a0 con 0x7f094015c550 2024-09-19T07:57:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.931+0000 7f094748d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f09400ceda0 con 0x7f09400cf800 2024-09-19T07:57:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.931+0000 7f094748d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f09400b5920 con 0x7f09401609b0 2024-09-19T07:57:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.931+0000 7f0944a28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f094015c550 0x7f09400c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.931+0000 7f0944a28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f094015c550 0x7f09400c9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42308/0 (socket says 172.21.15.16:42308) 2024-09-19T07:57:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.931+0000 7f0944a28640 1 -- 172.21.15.16:0/1704544167 learned_addr learned my addr 172.21.15.16:0/1704544167 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.932+0000 7f0945229640 1 --2- 172.21.15.16:0/1704544167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09400cf800 0x7f0940154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:26.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.932+0000 7f0945a2a640 1 --2- 172.21.15.16:0/1704544167 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09401609b0 0x7f09400c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.932+0000 7f0944a28640 1 -- 172.21.15.16:0/1704544167 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09401609b0 msgr2=0x7f09400c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.932+0000 7f0944a28640 1 --2- 172.21.15.16:0/1704544167 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09401609b0 0x7f09400c9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.932+0000 7f0944a28640 1 -- 172.21.15.16:0/1704544167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09400cf800 msgr2=0x7f0940154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.932+0000 7f0944a28640 1 --2- 172.21.15.16:0/1704544167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09400cf800 0x7f0940154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.932+0000 7f0944a28640 1 -- 172.21.15.16:0/1704544167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094015cc30 con 0x7f094015c550 2024-09-19T07:57:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.932+0000 7f0944a28640 1 --2- 172.21.15.16:0/1704544167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f094015c550 0x7f09400c9310 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f093400b4d0 tx=0x7f093400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.932+0000 7f093dffb640 1 -- 172.21.15.16:0/1704544167 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934014070 con 0x7f094015c550 2024-09-19T07:57:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.932+0000 7f093dffb640 1 -- 172.21.15.16:0/1704544167 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0934010870 con 0x7f094015c550 2024-09-19T07:57:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.932+0000 7f093dffb640 1 -- 172.21.15.16:0/1704544167 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934010c60 con 0x7f094015c550 2024-09-19T07:57:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.932+0000 7f094748d640 1 -- 172.21.15.16:0/1704544167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09400c9e80 con 0x7f094015c550 2024-09-19T07:57:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.932+0000 7f094748d640 1 -- 172.21.15.16:0/1704544167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09400ca440 con 0x7f094015c550 2024-09-19T07:57:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.933+0000 7f094748d640 1 -- 172.21.15.16:0/1704544167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09400b55c0 con 0x7f094015c550 2024-09-19T07:57:26.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.934+0000 7f093dffb640 1 -- 172.21.15.16:0/1704544167 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0934017070 con 0x7f094015c550 2024-09-19T07:57:26.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.934+0000 7f093dffb640 1 --2- 172.21.15.16:0/1704544167 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f092c06c2b0 0x7f092c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:26.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.934+0000 7f093dffb640 1 -- 172.21.15.16:0/1704544167 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f093408afa0 con 0x7f094015c550 2024-09-19T07:57:26.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.934+0000 7f0945229640 1 --2- 172.21.15.16:0/1704544167 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f092c06c2b0 0x7f092c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:26.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.935+0000 7f0945229640 1 --2- 172.21.15.16:0/1704544167 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f092c06c2b0 0x7f092c06e770 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0938034000 tx=0x7f093802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:26.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:26.938+0000 7f093dffb640 1 -- 172.21.15.16:0/1704544167 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f09400b55c0 con 0x7f094015c550 2024-09-19T07:57:27.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.115+0000 7f094748d640 1 -- 172.21.15.16:0/1704544167 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f09400cc020 con 0x7f092c06c2b0 2024-09-19T07:57:27.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.115+0000 7f093dffb640 1 -- 172.21.15.16:0/1704544167 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f09400cc020 con 0x7f092c06c2b0 2024-09-19T07:57:27.116 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 0 to deep-scrub 2024-09-19T07:57:27.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.117+0000 7f094748d640 1 -- 172.21.15.16:0/1704544167 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f092c06c2b0 msgr2=0x7f092c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.117+0000 7f094748d640 1 --2- 172.21.15.16:0/1704544167 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f092c06c2b0 0x7f092c06e770 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0938034000 tx=0x7f093802e000 comp rx=0 tx=0).stop 2024-09-19T07:57:27.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.117+0000 7f094748d640 1 -- 172.21.15.16:0/1704544167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f094015c550 msgr2=0x7f09400c9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.117+0000 7f094748d640 1 --2- 172.21.15.16:0/1704544167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f094015c550 0x7f09400c9310 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f093400b4d0 tx=0x7f093400b990 comp rx=0 tx=0).stop 2024-09-19T07:57:27.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.117+0000 7f094748d640 1 -- 172.21.15.16:0/1704544167 shutdown_connections 2024-09-19T07:57:27.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.117+0000 7f094748d640 1 --2- 172.21.15.16:0/1704544167 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f092c06c2b0 0x7f092c06e770 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.117+0000 7f094748d640 1 --2- 172.21.15.16:0/1704544167 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09401609b0 0x7f09400c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.117+0000 7f094748d640 1 --2- 172.21.15.16:0/1704544167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f094015c550 0x7f09400c9310 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.117+0000 7f094748d640 1 --2- 172.21.15.16:0/1704544167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09400cf800 0x7f0940154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.118+0000 7f094748d640 1 -- 172.21.15.16:0/1704544167 >> 172.21.15.16:0/1704544167 conn(0x7f094006feb0 msgr2=0x7f094015abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:27.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.118+0000 7f094748d640 1 -- 172.21.15.16:0/1704544167 shutdown_connections 2024-09-19T07:57:27.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.118+0000 7f094748d640 1 -- 172.21.15.16:0/1704544167 wait complete. 2024-09-19T07:57:27.129 INFO:tasks.ceph:Scrubbing osd.1 2024-09-19T07:57:27.129 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-19T07:57:27.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.235+0000 7f66c40d7640 1 Processor -- start 2024-09-19T07:57:27.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.235+0000 7f66c40d7640 1 -- start start 2024-09-19T07:57:27.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.236+0000 7f66c40d7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f66bc15e4c0 0x7f66bc0d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.236+0000 7f66c40d7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66bc0d20d0 con 0x7f66bc0d5030 2024-09-19T07:57:27.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.236+0000 7f66c40d7640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66bc0d23b0 con 0x7f66bc162030 2024-09-19T07:57:27.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.236+0000 7f66c40d7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f66bc0d0140 con 0x7f66bc15e4c0 2024-09-19T07:57:27.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.236+0000 7f66c1672640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f66bc0d5030 0x7f66bc0cfa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60898/0 (socket says 172.21.15.16:60898) 2024-09-19T07:57:27.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.236+0000 7f66c1672640 1 -- 172.21.15.16:0/752997713 learned_addr learned my addr 172.21.15.16:0/752997713 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:27.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c1e73640 1 --2- 172.21.15.16:0/752997713 >> v2:172.21.15.146:3300/0 conn(0x7f66bc15e4c0 0x7f66bc0d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c0e71640 1 -- 172.21.15.16:0/752997713 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2771529237 0 0) 0x7f66bc0d20d0 con 0x7f66bc0d5030 2024-09-19T07:57:27.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c0e71640 1 -- 172.21.15.16:0/752997713 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66a4003520 con 0x7f66bc0d5030 2024-09-19T07:57:27.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c0e71640 1 -- 172.21.15.16:0/752997713 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 894062101 0 0) 0x7f66bc0d23b0 con 0x7f66bc162030 2024-09-19T07:57:27.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c0e71640 1 -- 172.21.15.16:0/752997713 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66bc0d20d0 con 0x7f66bc162030 2024-09-19T07:57:27.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c0e71640 1 -- 172.21.15.16:0/752997713 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4132924044 0 0) 0x7f66a4003520 con 0x7f66bc0d5030 2024-09-19T07:57:27.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c0e71640 1 -- 172.21.15.16:0/752997713 >> v2:172.21.15.146:3300/0 conn(0x7f66bc15e4c0 msgr2=0x7f66bc0d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:27.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c0e71640 1 --2- 172.21.15.16:0/752997713 >> v2:172.21.15.146:3300/0 conn(0x7f66bc15e4c0 0x7f66bc0d2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c0e71640 1 -- 172.21.15.16:0/752997713 >> v1:172.21.15.146:6789/0 conn(0x7f66bc162030 legacy=0x7f66bc06cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c0e71640 1 -- 172.21.15.16:0/752997713 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc0d2690 con 0x7f66bc0d5030 2024-09-19T07:57:27.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c0e71640 1 -- 172.21.15.16:0/752997713 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f66b0001670 con 0x7f66bc0d5030 2024-09-19T07:57:27.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c0e71640 1 -- 172.21.15.16:0/752997713 >> v1:172.21.15.16:6789/0 conn(0x7f66bc0d5030 legacy=0x7f66bc0cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c0e71640 1 --2- 172.21.15.16:0/752997713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66a4003b40 0x7f66a4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c0e71640 1 --2- 172.21.15.16:0/752997713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66a4004800 0x7f66a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c0e71640 1 --2- 172.21.15.16:0/752997713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66a40051c0 0x7f66a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c0e71640 1 -- 172.21.15.16:0/752997713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc0d0140 con 0x7f66a40051c0 2024-09-19T07:57:27.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c0e71640 1 -- 172.21.15.16:0/752997713 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f66a400c0f0 con 0x7f66a4004800 2024-09-19T07:57:27.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c0e71640 1 -- 172.21.15.16:0/752997713 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f66a400c2f0 con 0x7f66a4003b40 2024-09-19T07:57:27.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c2674640 1 --2- 172.21.15.16:0/752997713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66a40051c0 0x7f66a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c1672640 1 --2- 172.21.15.16:0/752997713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66a4004800 0x7f66a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c1e73640 1 --2- 172.21.15.16:0/752997713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66a4003b40 0x7f66a4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c2674640 1 -- 172.21.15.16:0/752997713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66a4003b40 msgr2=0x7f66a4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c2674640 1 --2- 172.21.15.16:0/752997713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66a4003b40 0x7f66a4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c2674640 1 -- 172.21.15.16:0/752997713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66a4004800 msgr2=0x7f66a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c2674640 1 --2- 172.21.15.16:0/752997713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66a4004800 0x7f66a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.237+0000 7f66c2674640 1 -- 172.21.15.16:0/752997713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a4003520 con 0x7f66a40051c0 2024-09-19T07:57:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.238+0000 7f66c1e73640 1 --2- 172.21.15.16:0/752997713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66a4003b40 0x7f66a4003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.238+0000 7f66c2674640 1 -- 172.21.15.16:0/752997713 reap_dead start 2024-09-19T07:57:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.238+0000 7f66c2674640 1 --2- 172.21.15.16:0/752997713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66a40051c0 0x7f66a400b9c0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f66b4001fd0 tx=0x7f66a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81dbf48717b814c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.238+0000 7f66c0e71640 1 -- 172.21.15.16:0/752997713 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b4001bf0 con 0x7f66a40051c0 2024-09-19T07:57:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.238+0000 7f66c0e71640 1 -- 172.21.15.16:0/752997713 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b4003b70 con 0x7f66a40051c0 2024-09-19T07:57:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.238+0000 7f66c0e71640 1 -- 172.21.15.16:0/752997713 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b40050d0 con 0x7f66a40051c0 2024-09-19T07:57:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.239+0000 7f66c40d7640 1 -- 172.21.15.16:0/752997713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66a40051c0 msgr2=0x7f66a400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.239+0000 7f66c40d7640 1 --2- 172.21.15.16:0/752997713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66a40051c0 0x7f66a400b9c0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f66b4001fd0 tx=0x7f66a4005830 comp rx=0 tx=0).stop 2024-09-19T07:57:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.239+0000 7f66c40d7640 1 -- 172.21.15.16:0/752997713 shutdown_connections 2024-09-19T07:57:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.239+0000 7f66c40d7640 1 --2- 172.21.15.16:0/752997713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66a40051c0 0x7f66a400b9c0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.239+0000 7f66c40d7640 1 -- 172.21.15.16:0/752997713 >> 172.21.15.16:0/752997713 conn(0x7f66bc09d780 msgr2=0x7f66bc0d43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.239+0000 7f66c40d7640 1 -- 172.21.15.16:0/752997713 shutdown_connections 2024-09-19T07:57:27.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.239+0000 7f66c40d7640 1 -- 172.21.15.16:0/752997713 wait complete. 2024-09-19T07:57:27.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.240+0000 7f66c40d7640 1 Processor -- start 2024-09-19T07:57:27.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.240+0000 7f66c40d7640 1 -- start start 2024-09-19T07:57:27.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.241+0000 7f66c40d7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66a40051c0 0x7f66bc0b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.241+0000 7f66c40d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66bc0cd8b0 0x7f66bc0b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.241+0000 7f66c40d7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66bc0b3fa0 0x7f66bc0b98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.241+0000 7f66c40d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc0d2e90 con 0x7f66bc0cd8b0 2024-09-19T07:57:27.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.241+0000 7f66c40d7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc0d2a00 con 0x7f66a40051c0 2024-09-19T07:57:27.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.241+0000 7f66c40d7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f66bc0d3010 con 0x7f66bc0b3fa0 2024-09-19T07:57:27.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.241+0000 7f66c1672640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66bc0cd8b0 0x7f66bc0b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.241+0000 7f66c1672640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66bc0cd8b0 0x7f66bc0b7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42324/0 (socket says 172.21.15.16:42324) 2024-09-19T07:57:27.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.241+0000 7f66c1672640 1 -- 172.21.15.16:0/2168649735 learned_addr learned my addr 172.21.15.16:0/2168649735 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:27.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.241+0000 7f66c1e73640 1 --2- 172.21.15.16:0/2168649735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66a40051c0 0x7f66bc0b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.241+0000 7f66c2674640 1 --2- 172.21.15.16:0/2168649735 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66bc0b3fa0 0x7f66bc0b98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.241+0000 7f66c1672640 1 -- 172.21.15.16:0/2168649735 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66bc0b3fa0 msgr2=0x7f66bc0b98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.241+0000 7f66c1672640 1 --2- 172.21.15.16:0/2168649735 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66bc0b3fa0 0x7f66bc0b98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.241+0000 7f66c1672640 1 -- 172.21.15.16:0/2168649735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66a40051c0 msgr2=0x7f66bc0b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.241+0000 7f66c1672640 1 --2- 172.21.15.16:0/2168649735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66a40051c0 0x7f66bc0b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.241+0000 7f66c1672640 1 -- 172.21.15.16:0/2168649735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc0b4af0 con 0x7f66bc0cd8b0 2024-09-19T07:57:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.242+0000 7f66c1672640 1 --2- 172.21.15.16:0/2168649735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66bc0cd8b0 0x7f66bc0b7ec0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f66b0004f70 tx=0x7f66b00074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.242+0000 7f66ba7fc640 1 -- 172.21.15.16:0/2168649735 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0001080 con 0x7f66bc0cd8b0 2024-09-19T07:57:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.242+0000 7f66ba7fc640 1 -- 172.21.15.16:0/2168649735 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b0018070 con 0x7f66bc0cd8b0 2024-09-19T07:57:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.242+0000 7f66c40d7640 1 -- 172.21.15.16:0/2168649735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66bc0cb540 con 0x7f66bc0cd8b0 2024-09-19T07:57:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.242+0000 7f66ba7fc640 1 -- 172.21.15.16:0/2168649735 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b000f970 con 0x7f66bc0cd8b0 2024-09-19T07:57:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.242+0000 7f66c40d7640 1 -- 172.21.15.16:0/2168649735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66bc0cbb00 con 0x7f66bc0cd8b0 2024-09-19T07:57:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.243+0000 7f66c40d7640 1 -- 172.21.15.16:0/2168649735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f66bc0cbee0 con 0x7f66bc0cd8b0 2024-09-19T07:57:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.244+0000 7f66ba7fc640 1 -- 172.21.15.16:0/2168649735 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f66b000fb10 con 0x7f66bc0cd8b0 2024-09-19T07:57:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.244+0000 7f66ba7fc640 1 --2- 172.21.15.16:0/2168649735 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f66a806c450 0x7f66a806e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.244+0000 7f66ba7fc640 1 -- 172.21.15.16:0/2168649735 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f66b008bc90 con 0x7f66bc0cd8b0 2024-09-19T07:57:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.244+0000 7f66ba7fc640 1 --2- 172.21.15.16:0/2168649735 >> [v2:172.21.15.16:6824/3030558062,v1:172.21.15.16:6825/3030558062] conn(0x7f66a8072940 0x7f66a8074d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.244+0000 7f66ba7fc640 1 -- 172.21.15.16:0/2168649735 --> [v2:172.21.15.16:6824/3030558062,v1:172.21.15.16:6825/3030558062] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f66b0008e50 con 0x7f66a8072940 2024-09-19T07:57:27.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.244+0000 7f66ba7fc640 1 -- 172.21.15.16:0/2168649735 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=156) ==== 24+0+0 (secure 0 0 0) 0x7f66b008c120 con 0x7f66bc0cd8b0 2024-09-19T07:57:27.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.244+0000 7f66c1e73640 1 --2- 172.21.15.16:0/2168649735 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f66a806c450 0x7f66a806e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.244+0000 7f66c2674640 1 --2- 172.21.15.16:0/2168649735 >> [v2:172.21.15.16:6824/3030558062,v1:172.21.15.16:6825/3030558062] conn(0x7f66a8072940 0x7f66a8074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.245+0000 7f66c2674640 1 --2- 172.21.15.16:0/2168649735 >> [v2:172.21.15.16:6824/3030558062,v1:172.21.15.16:6825/3030558062] conn(0x7f66a8072940 0x7f66a8074d80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:27.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.245+0000 7f66c1e73640 1 --2- 172.21.15.16:0/2168649735 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f66a806c450 0x7f66a806e910 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0cd840 tx=0x7f66ac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:27.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.245+0000 7f66ba7fc640 1 -- 172.21.15.16:0/2168649735 <== osd.1 v2:172.21.15.16:6824/3030558062 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f66b0008e50 con 0x7f66a8072940 2024-09-19T07:57:27.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.258+0000 7f66c40d7640 1 -- 172.21.15.16:0/2168649735 --> [v2:172.21.15.16:6824/3030558062,v1:172.21.15.16:6825/3030558062] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f66bc162480 con 0x7f66a8072940 2024-09-19T07:57:27.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.258+0000 7f66ba7fc640 1 -- 172.21.15.16:0/2168649735 <== osd.1 v2:172.21.15.16:6824/3030558062 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f66bc162480 con 0x7f66a8072940 2024-09-19T07:57:27.259 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:57:27.259 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T07:57:27.259 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:57:27.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.259+0000 7f66c40d7640 1 -- 172.21.15.16:0/2168649735 >> [v2:172.21.15.16:6824/3030558062,v1:172.21.15.16:6825/3030558062] conn(0x7f66a8072940 msgr2=0x7f66a8074d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.259+0000 7f66c40d7640 1 --2- 172.21.15.16:0/2168649735 >> [v2:172.21.15.16:6824/3030558062,v1:172.21.15.16:6825/3030558062] conn(0x7f66a8072940 0x7f66a8074d80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.259+0000 7f66c40d7640 1 -- 172.21.15.16:0/2168649735 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f66a806c450 msgr2=0x7f66a806e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.259+0000 7f66c40d7640 1 --2- 172.21.15.16:0/2168649735 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f66a806c450 0x7f66a806e910 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0cd840 tx=0x7f66ac008040 comp rx=0 tx=0).stop 2024-09-19T07:57:27.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.259+0000 7f66c40d7640 1 -- 172.21.15.16:0/2168649735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66bc0cd8b0 msgr2=0x7f66bc0b7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.259+0000 7f66c40d7640 1 --2- 172.21.15.16:0/2168649735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66bc0cd8b0 0x7f66bc0b7ec0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f66b0004f70 tx=0x7f66b00074f0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.259+0000 7f66c2674640 1 -- 172.21.15.16:0/2168649735 reap_dead start 2024-09-19T07:57:27.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.259+0000 7f66c40d7640 1 -- 172.21.15.16:0/2168649735 shutdown_connections 2024-09-19T07:57:27.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.259+0000 7f66c40d7640 1 -- 172.21.15.16:0/2168649735 >> 172.21.15.16:0/2168649735 conn(0x7f66bc09d780 msgr2=0x7f66bc0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:27.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.259+0000 7f66c40d7640 1 -- 172.21.15.16:0/2168649735 shutdown_connections 2024-09-19T07:57:27.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.259+0000 7f66c40d7640 1 -- 172.21.15.16:0/2168649735 wait complete. 2024-09-19T07:57:27.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-19T07:57:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.371+0000 7f936e328640 1 Processor -- start 2024-09-19T07:57:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.372+0000 7f936e328640 1 -- start start 2024-09-19T07:57:27.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.372+0000 7f936e328640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93680d43d0 0x7f93680d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.372+0000 7f936e328640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93680b75e0 0x7f93680b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.372+0000 7f936e328640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93680d4d30 con 0x7f936815e650 2024-09-19T07:57:27.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.372+0000 7f936e328640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f93680d32c0 con 0x7f93680b75e0 2024-09-19T07:57:27.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.372+0000 7f936e328640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f93680d35a0 con 0x7f93680d43d0 2024-09-19T07:57:27.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.373+0000 7f936c8c5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93680b75e0 0x7f93680b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.373+0000 7f936c8c5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93680b75e0 0x7f93680b7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42330/0 (socket says 172.21.15.16:42330) 2024-09-19T07:57:27.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.373+0000 7f936c8c5640 1 -- 172.21.15.16:0/1163917153 learned_addr learned my addr 172.21.15.16:0/1163917153 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:27.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.373+0000 7f93677fe640 1 --2- 172.21.15.16:0/1163917153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93680d43d0 0x7f93680d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.373+0000 7f936c8c5640 1 -- 172.21.15.16:0/1163917153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93680d43d0 msgr2=0x7f93680d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:27.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.373+0000 7f936c8c5640 1 --2- 172.21.15.16:0/1163917153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93680d43d0 0x7f93680d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.373+0000 7f936c8c5640 1 -- 172.21.15.16:0/1163917153 >> v1:172.21.15.146:6789/0 conn(0x7f936815e650 legacy=0x7f93680cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:27.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.373+0000 7f936c8c5640 1 -- 172.21.15.16:0/1163917153 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93680d3880 con 0x7f93680b75e0 2024-09-19T07:57:27.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.374+0000 7f936c8c5640 1 --2- 172.21.15.16:0/1163917153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93680b75e0 0x7f93680b7a90 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f9360009e50 tx=0x7f936002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b220e8f954bdc887 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:27.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.374+0000 7f9366ffd640 1 -- 172.21.15.16:0/1163917153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360036070 con 0x7f93680b75e0 2024-09-19T07:57:27.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.374+0000 7f9366ffd640 1 -- 172.21.15.16:0/1163917153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9360031600 con 0x7f93680b75e0 2024-09-19T07:57:27.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.374+0000 7f9366ffd640 1 -- 172.21.15.16:0/1163917153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93600319d0 con 0x7f93680b75e0 2024-09-19T07:57:27.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.374+0000 7f936e328640 1 -- 172.21.15.16:0/1163917153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93680b75e0 msgr2=0x7f93680b7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.374+0000 7f936e328640 1 --2- 172.21.15.16:0/1163917153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93680b75e0 0x7f93680b7a90 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f9360009e50 tx=0x7f936002a750 comp rx=0 tx=0).stop 2024-09-19T07:57:27.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.374+0000 7f936e328640 1 -- 172.21.15.16:0/1163917153 shutdown_connections 2024-09-19T07:57:27.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.374+0000 7f936e328640 1 --2- 172.21.15.16:0/1163917153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93680b75e0 0x7f93680b7a90 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.374+0000 7f936e328640 1 --2- 172.21.15.16:0/1163917153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93680d43d0 0x7f93680d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.374+0000 7f936e328640 1 -- 172.21.15.16:0/1163917153 >> 172.21.15.16:0/1163917153 conn(0x7f9368073660 msgr2=0x7f9368073a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:27.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.374+0000 7f936e328640 1 -- 172.21.15.16:0/1163917153 shutdown_connections 2024-09-19T07:57:27.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.375+0000 7f936e328640 1 -- 172.21.15.16:0/1163917153 wait complete. 2024-09-19T07:57:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.376+0000 7f936e328640 1 Processor -- start 2024-09-19T07:57:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.376+0000 7f936e328640 1 -- start start 2024-09-19T07:57:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f936e328640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93680b75e0 0x7f9368157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f936e328640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93680d43d0 0x7f9368161730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f936e328640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f936815e650 0x7f9368158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f936e328640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f93680d2340 con 0x7f93680b75e0 2024-09-19T07:57:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f936e328640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f93680d21c0 con 0x7f936815e650 2024-09-19T07:57:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f936e328640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f93680cfa90 con 0x7f93680d43d0 2024-09-19T07:57:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f9367fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93680b75e0 0x7f9368157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f9367fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93680b75e0 0x7f9368157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42338/0 (socket says 172.21.15.16:42338) 2024-09-19T07:57:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f9367fff640 1 -- 172.21.15.16:0/1766125513 learned_addr learned my addr 172.21.15.16:0/1766125513 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f93677fe640 1 --2- 172.21.15.16:0/1766125513 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93680d43d0 0x7f9368161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f936c8c5640 1 --2- 172.21.15.16:0/1766125513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f936815e650 0x7f9368158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f9367fff640 1 -- 172.21.15.16:0/1766125513 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93680d43d0 msgr2=0x7f9368161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f9367fff640 1 --2- 172.21.15.16:0/1766125513 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93680d43d0 0x7f9368161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f9367fff640 1 -- 172.21.15.16:0/1766125513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f936815e650 msgr2=0x7f9368158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f9367fff640 1 --2- 172.21.15.16:0/1766125513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f936815e650 0x7f9368158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f9367fff640 1 -- 172.21.15.16:0/1766125513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93681456a0 con 0x7f93680b75e0 2024-09-19T07:57:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f9367fff640 1 --2- 172.21.15.16:0/1766125513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93680b75e0 0x7f9368157370 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f935800b3f0 tx=0x7f935800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f9364ff9640 1 -- 172.21.15.16:0/1766125513 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9358015020 con 0x7f93680b75e0 2024-09-19T07:57:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f9364ff9640 1 -- 172.21.15.16:0/1766125513 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9358011830 con 0x7f93680b75e0 2024-09-19T07:57:27.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f936e328640 1 -- 172.21.15.16:0/1766125513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9368145980 con 0x7f93680b75e0 2024-09-19T07:57:27.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f9364ff9640 1 -- 172.21.15.16:0/1766125513 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9358011c20 con 0x7f93680b75e0 2024-09-19T07:57:27.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.377+0000 7f936e328640 1 -- 172.21.15.16:0/1766125513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9368145f40 con 0x7f93680b75e0 2024-09-19T07:57:27.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.378+0000 7f936e328640 1 -- 172.21.15.16:0/1766125513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93680d47f0 con 0x7f93680b75e0 2024-09-19T07:57:27.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.379+0000 7f9364ff9640 1 -- 172.21.15.16:0/1766125513 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9358018020 con 0x7f93680b75e0 2024-09-19T07:57:27.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.379+0000 7f9364ff9640 1 --2- 172.21.15.16:0/1766125513 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f935406c2b0 0x7f935406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.379+0000 7f9364ff9640 1 -- 172.21.15.16:0/1766125513 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f935808b140 con 0x7f93680b75e0 2024-09-19T07:57:27.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.380+0000 7f93677fe640 1 --2- 172.21.15.16:0/1766125513 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f935406c2b0 0x7f935406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.380+0000 7f93677fe640 1 --2- 172.21.15.16:0/1766125513 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f935406c2b0 0x7f935406e770 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f93681605a0 tx=0x7f935c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:27.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.384+0000 7f9364ff9640 1 -- 172.21.15.16:0/1766125513 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f935801f020 con 0x7f93680b75e0 2024-09-19T07:57:27.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.561+0000 7f936e328640 1 -- 172.21.15.16:0/1766125513 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f93680b9050 con 0x7f935406c2b0 2024-09-19T07:57:27.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.562+0000 7f9364ff9640 1 -- 172.21.15.16:0/1766125513 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f93680b9050 con 0x7f935406c2b0 2024-09-19T07:57:27.562 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 1 to deep-scrub 2024-09-19T07:57:27.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.564+0000 7f936e328640 1 -- 172.21.15.16:0/1766125513 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f935406c2b0 msgr2=0x7f935406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.564+0000 7f936e328640 1 --2- 172.21.15.16:0/1766125513 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f935406c2b0 0x7f935406e770 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f93681605a0 tx=0x7f935c008040 comp rx=0 tx=0).stop 2024-09-19T07:57:27.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.564+0000 7f936e328640 1 -- 172.21.15.16:0/1766125513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93680b75e0 msgr2=0x7f9368157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.564+0000 7f936e328640 1 --2- 172.21.15.16:0/1766125513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93680b75e0 0x7f9368157370 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f935800b3f0 tx=0x7f935800b8b0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.564+0000 7f936e328640 1 -- 172.21.15.16:0/1766125513 shutdown_connections 2024-09-19T07:57:27.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.564+0000 7f936e328640 1 --2- 172.21.15.16:0/1766125513 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f935406c2b0 0x7f935406e770 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.564+0000 7f936e328640 1 --2- 172.21.15.16:0/1766125513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f936815e650 0x7f9368158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.564+0000 7f936e328640 1 --2- 172.21.15.16:0/1766125513 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93680d43d0 0x7f9368161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.564+0000 7f936e328640 1 --2- 172.21.15.16:0/1766125513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93680b75e0 0x7f9368157370 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.564+0000 7f936e328640 1 -- 172.21.15.16:0/1766125513 >> 172.21.15.16:0/1766125513 conn(0x7f9368073660 msgr2=0x7f9368073a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:27.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.564+0000 7f936e328640 1 -- 172.21.15.16:0/1766125513 shutdown_connections 2024-09-19T07:57:27.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.564+0000 7f936e328640 1 -- 172.21.15.16:0/1766125513 wait complete. 2024-09-19T07:57:27.575 INFO:tasks.ceph:Scrubbing osd.2 2024-09-19T07:57:27.576 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-19T07:57:27.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.681+0000 7f5d7dc3c640 1 Processor -- start 2024-09-19T07:57:27.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.681+0000 7f5d7dc3c640 1 -- start start 2024-09-19T07:57:27.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.682+0000 7f5d7dc3c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d7815e4c0 0x7f5d780d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.682+0000 7f5d7dc3c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d780d20d0 con 0x7f5d78162030 2024-09-19T07:57:27.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.682+0000 7f5d7dc3c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d780d23b0 con 0x7f5d780d5030 2024-09-19T07:57:27.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.682+0000 7f5d7dc3c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f5d780d0140 con 0x7f5d7815e4c0 2024-09-19T07:57:27.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.682+0000 7f5d777fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d7815e4c0 0x7f5d780d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.682+0000 7f5d77fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5d78162030 0x7f5d7806cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60906/0 (socket says 172.21.15.16:60906) 2024-09-19T07:57:27.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.682+0000 7f5d77fff640 1 -- 172.21.15.16:0/922068969 learned_addr learned my addr 172.21.15.16:0/922068969 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:27.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.682+0000 7f5d777fe640 1 -- 172.21.15.16:0/922068969 >> v1:172.21.15.146:6789/0 conn(0x7f5d780d5030 legacy=0x7f5d780cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:27.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.682+0000 7f5d777fe640 1 -- 172.21.15.16:0/922068969 >> v1:172.21.15.16:6789/0 conn(0x7f5d78162030 legacy=0x7f5d7806cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.682+0000 7f5d777fe640 1 -- 172.21.15.16:0/922068969 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d780d2690 con 0x7f5d7815e4c0 2024-09-19T07:57:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.683+0000 7f5d777fe640 1 --2- 172.21.15.16:0/922068969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d7815e4c0 0x7f5d780d2e90 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f5d680097a0 tx=0x7f5d6802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404ef3977d79f735 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.683+0000 7f5d767fc640 1 -- 172.21.15.16:0/922068969 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68037070 con 0x7f5d7815e4c0 2024-09-19T07:57:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.683+0000 7f5d767fc640 1 -- 172.21.15.16:0/922068969 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d6802ae50 con 0x7f5d7815e4c0 2024-09-19T07:57:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.683+0000 7f5d767fc640 1 -- 172.21.15.16:0/922068969 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68032660 con 0x7f5d7815e4c0 2024-09-19T07:57:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.683+0000 7f5d7dc3c640 1 -- 172.21.15.16:0/922068969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d7815e4c0 msgr2=0x7f5d780d2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.683+0000 7f5d7dc3c640 1 --2- 172.21.15.16:0/922068969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d7815e4c0 0x7f5d780d2e90 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f5d680097a0 tx=0x7f5d6802a140 comp rx=0 tx=0).stop 2024-09-19T07:57:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.684+0000 7f5d7dc3c640 1 -- 172.21.15.16:0/922068969 shutdown_connections 2024-09-19T07:57:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.684+0000 7f5d7dc3c640 1 --2- 172.21.15.16:0/922068969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d7815e4c0 0x7f5d780d2e90 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.684+0000 7f5d7dc3c640 1 -- 172.21.15.16:0/922068969 >> 172.21.15.16:0/922068969 conn(0x7f5d7809d780 msgr2=0x7f5d780d43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:27.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.684+0000 7f5d7dc3c640 1 -- 172.21.15.16:0/922068969 shutdown_connections 2024-09-19T07:57:27.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.684+0000 7f5d7dc3c640 1 -- 172.21.15.16:0/922068969 wait complete. 2024-09-19T07:57:27.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.685+0000 7f5d7dc3c640 1 Processor -- start 2024-09-19T07:57:27.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.685+0000 7f5d7dc3c640 1 -- start start 2024-09-19T07:57:27.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d7dc3c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d780d5030 0x7f5d78157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d7dc3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d7815e4c0 0x7f5d78149ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d7dc3c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d78162030 0x7f5d7814a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d7dc3c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d78162480 con 0x7f5d7815e4c0 2024-09-19T07:57:27.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d7dc3c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5d780d2a00 con 0x7f5d780d5030 2024-09-19T07:57:27.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d7dc3c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5d78162600 con 0x7f5d78162030 2024-09-19T07:57:27.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d76ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d7815e4c0 0x7f5d78149ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d76ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d7815e4c0 0x7f5d78149ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42346/0 (socket says 172.21.15.16:42346) 2024-09-19T07:57:27.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d76ffd640 1 -- 172.21.15.16:0/1351326626 learned_addr learned my addr 172.21.15.16:0/1351326626 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:27.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d777fe640 1 --2- 172.21.15.16:0/1351326626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d780d5030 0x7f5d78157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d77fff640 1 --2- 172.21.15.16:0/1351326626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d78162030 0x7f5d7814a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d76ffd640 1 -- 172.21.15.16:0/1351326626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d78162030 msgr2=0x7f5d7814a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d76ffd640 1 --2- 172.21.15.16:0/1351326626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d78162030 0x7f5d7814a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d76ffd640 1 -- 172.21.15.16:0/1351326626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d780d5030 msgr2=0x7f5d78157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d76ffd640 1 --2- 172.21.15.16:0/1351326626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d780d5030 0x7f5d78157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d76ffd640 1 -- 172.21.15.16:0/1351326626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d780b96b0 con 0x7f5d7815e4c0 2024-09-19T07:57:27.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d77fff640 1 --2- 172.21.15.16:0/1351326626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d78162030 0x7f5d7814a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:27.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d777fe640 1 --2- 172.21.15.16:0/1351326626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d780d5030 0x7f5d78157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:27.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d76ffd640 1 --2- 172.21.15.16:0/1351326626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d7815e4c0 0x7f5d78149ee0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c00c440 tx=0x7f5d6c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:27.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.686+0000 7f5d5ffff640 1 -- 172.21.15.16:0/1351326626 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c017070 con 0x7f5d7815e4c0 2024-09-19T07:57:27.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.687+0000 7f5d5ffff640 1 -- 172.21.15.16:0/1351326626 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d6c0035f0 con 0x7f5d7815e4c0 2024-09-19T07:57:27.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.687+0000 7f5d7dc3c640 1 -- 172.21.15.16:0/1351326626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d780b9990 con 0x7f5d7815e4c0 2024-09-19T07:57:27.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.687+0000 7f5d5ffff640 1 -- 172.21.15.16:0/1351326626 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c0039e0 con 0x7f5d7815e4c0 2024-09-19T07:57:27.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.687+0000 7f5d7dc3c640 1 -- 172.21.15.16:0/1351326626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d781f58f0 con 0x7f5d7815e4c0 2024-09-19T07:57:27.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.687+0000 7f5d7dc3c640 1 -- 172.21.15.16:0/1351326626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5d780b9bd0 con 0x7f5d7815e4c0 2024-09-19T07:57:27.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.688+0000 7f5d5ffff640 1 -- 172.21.15.16:0/1351326626 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5d6c01a070 con 0x7f5d7815e4c0 2024-09-19T07:57:27.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.689+0000 7f5d5ffff640 1 --2- 172.21.15.16:0/1351326626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5d6406c2b0 0x7f5d6406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.689+0000 7f5d5ffff640 1 -- 172.21.15.16:0/1351326626 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f5d6c08aae0 con 0x7f5d7815e4c0 2024-09-19T07:57:27.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.689+0000 7f5d5ffff640 1 --2- 172.21.15.16:0/1351326626 >> [v2:172.21.15.16:6800/4074420552,v1:172.21.15.16:6801/4074420552] conn(0x7f5d640727a0 0x7f5d64074be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.689+0000 7f5d5ffff640 1 -- 172.21.15.16:0/1351326626 --> [v2:172.21.15.16:6800/4074420552,v1:172.21.15.16:6801/4074420552] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d640752b0 con 0x7f5d640727a0 2024-09-19T07:57:27.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.689+0000 7f5d5ffff640 1 -- 172.21.15.16:0/1351326626 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=156) ==== 24+0+0 (secure 0 0 0) 0x7f5d6c08af70 con 0x7f5d7815e4c0 2024-09-19T07:57:27.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.689+0000 7f5d77fff640 1 --2- 172.21.15.16:0/1351326626 >> [v2:172.21.15.16:6800/4074420552,v1:172.21.15.16:6801/4074420552] conn(0x7f5d640727a0 0x7f5d64074be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.689+0000 7f5d777fe640 1 --2- 172.21.15.16:0/1351326626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5d6406c2b0 0x7f5d6406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.689+0000 7f5d77fff640 1 --2- 172.21.15.16:0/1351326626 >> [v2:172.21.15.16:6800/4074420552,v1:172.21.15.16:6801/4074420552] conn(0x7f5d640727a0 0x7f5d64074be0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:27.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.690+0000 7f5d777fe640 1 --2- 172.21.15.16:0/1351326626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5d6406c2b0 0x7f5d6406e770 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5d68034000 tx=0x7f5d6802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:27.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.690+0000 7f5d5ffff640 1 -- 172.21.15.16:0/1351326626 <== osd.2 v2:172.21.15.16:6800/4074420552 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d640752b0 con 0x7f5d640727a0 2024-09-19T07:57:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.704+0000 7f5d7dc3c640 1 -- 172.21.15.16:0/1351326626 --> [v2:172.21.15.16:6800/4074420552,v1:172.21.15.16:6801/4074420552] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5d780d2e90 con 0x7f5d640727a0 2024-09-19T07:57:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.704+0000 7f5d5ffff640 1 -- 172.21.15.16:0/1351326626 <== osd.2 v2:172.21.15.16:6800/4074420552 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5d780d2e90 con 0x7f5d640727a0 2024-09-19T07:57:27.705 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:57:27.705 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T07:57:27.705 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:57:27.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.704+0000 7f5d7dc3c640 1 -- 172.21.15.16:0/1351326626 >> [v2:172.21.15.16:6800/4074420552,v1:172.21.15.16:6801/4074420552] conn(0x7f5d640727a0 msgr2=0x7f5d64074be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.704+0000 7f5d7dc3c640 1 --2- 172.21.15.16:0/1351326626 >> [v2:172.21.15.16:6800/4074420552,v1:172.21.15.16:6801/4074420552] conn(0x7f5d640727a0 0x7f5d64074be0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.705+0000 7f5d7dc3c640 1 -- 172.21.15.16:0/1351326626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5d6406c2b0 msgr2=0x7f5d6406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.705+0000 7f5d7dc3c640 1 --2- 172.21.15.16:0/1351326626 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5d6406c2b0 0x7f5d6406e770 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5d68034000 tx=0x7f5d6802e000 comp rx=0 tx=0).stop 2024-09-19T07:57:27.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.705+0000 7f5d7dc3c640 1 -- 172.21.15.16:0/1351326626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d7815e4c0 msgr2=0x7f5d78149ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.705+0000 7f5d7dc3c640 1 --2- 172.21.15.16:0/1351326626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d7815e4c0 0x7f5d78149ee0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c00c440 tx=0x7f5d6c00c900 comp rx=0 tx=0).stop 2024-09-19T07:57:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.705+0000 7f5d77fff640 1 -- 172.21.15.16:0/1351326626 reap_dead start 2024-09-19T07:57:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.705+0000 7f5d7dc3c640 1 -- 172.21.15.16:0/1351326626 shutdown_connections 2024-09-19T07:57:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.705+0000 7f5d7dc3c640 1 -- 172.21.15.16:0/1351326626 >> 172.21.15.16:0/1351326626 conn(0x7f5d7809d780 msgr2=0x7f5d7815da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.705+0000 7f5d7dc3c640 1 -- 172.21.15.16:0/1351326626 shutdown_connections 2024-09-19T07:57:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.705+0000 7f5d7dc3c640 1 -- 172.21.15.16:0/1351326626 wait complete. 2024-09-19T07:57:27.715 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-19T07:57:27.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.818+0000 7f83850dd640 1 Processor -- start 2024-09-19T07:57:27.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.818+0000 7f83850dd640 1 -- start start 2024-09-19T07:57:27.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.819+0000 7f83850dd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83800d2040 0x7f83800cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.819+0000 7f83850dd640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83800d24d0 con 0x7f838015bfd0 2024-09-19T07:57:27.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.819+0000 7f83850dd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83800d27b0 con 0x7f838015c3b0 2024-09-19T07:57:27.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.819+0000 7f83850dd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f83800ceab0 con 0x7f83800d2040 2024-09-19T07:57:27.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.819+0000 7f837ed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f838015bfd0 0x7f83800d1930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60918/0 (socket says 172.21.15.16:60918) 2024-09-19T07:57:27.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.819+0000 7f837ed9d640 1 -- 172.21.15.16:0/2622887598 learned_addr learned my addr 172.21.15.16:0/2622887598 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:27.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.819+0000 7f837e59c640 1 --2- 172.21.15.16:0/2622887598 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83800d2040 0x7f83800cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.819+0000 7f837dd9b640 1 -- 172.21.15.16:0/2622887598 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1481382372 0 0) 0x7f83800d24d0 con 0x7f838015bfd0 2024-09-19T07:57:27.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.819+0000 7f837dd9b640 1 -- 172.21.15.16:0/2622887598 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8368003520 con 0x7f838015bfd0 2024-09-19T07:57:27.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.819+0000 7f837dd9b640 1 -- 172.21.15.16:0/2622887598 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3622933176 0 0) 0x7f8368003520 con 0x7f838015bfd0 2024-09-19T07:57:27.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.819+0000 7f837dd9b640 1 -- 172.21.15.16:0/2622887598 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83800d2040 msgr2=0x7f83800cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.819+0000 7f837dd9b640 1 --2- 172.21.15.16:0/2622887598 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83800d2040 0x7f83800cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.819+0000 7f837dd9b640 1 -- 172.21.15.16:0/2622887598 >> v1:172.21.15.146:6789/0 conn(0x7f838015c3b0 legacy=0x7f83800b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.819+0000 7f837dd9b640 1 -- 172.21.15.16:0/2622887598 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83800d2a90 con 0x7f838015bfd0 2024-09-19T07:57:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.819+0000 7f837dd9b640 1 -- 172.21.15.16:0/2622887598 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f8370001670 con 0x7f838015bfd0 2024-09-19T07:57:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.820+0000 7f837dd9b640 1 -- 172.21.15.16:0/2622887598 >> v1:172.21.15.16:6789/0 conn(0x7f838015bfd0 legacy=0x7f83800d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.820+0000 7f837dd9b640 1 --2- 172.21.15.16:0/2622887598 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83680039e0 0x7f8368003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.820+0000 7f837dd9b640 1 --2- 172.21.15.16:0/2622887598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8368004340 0x7f8368004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.820+0000 7f837dd9b640 1 --2- 172.21.15.16:0/2622887598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8368005020 0x7f836800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.820+0000 7f837dd9b640 1 -- 172.21.15.16:0/2622887598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f836800bf00 con 0x7f8368005020 2024-09-19T07:57:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.820+0000 7f837dd9b640 1 -- 172.21.15.16:0/2622887598 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f836800c0b0 con 0x7f8368004340 2024-09-19T07:57:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.820+0000 7f837dd9b640 1 -- 172.21.15.16:0/2622887598 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f836800c2b0 con 0x7f83680039e0 2024-09-19T07:57:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.820+0000 7f837f59e640 1 --2- 172.21.15.16:0/2622887598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8368005020 0x7f836800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.820+0000 7f837ed9d640 1 --2- 172.21.15.16:0/2622887598 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83680039e0 0x7f8368003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.820+0000 7f837e59c640 1 --2- 172.21.15.16:0/2622887598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8368004340 0x7f8368004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.820+0000 7f837f59e640 1 -- 172.21.15.16:0/2622887598 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83680039e0 msgr2=0x7f8368003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.820+0000 7f837f59e640 1 --2- 172.21.15.16:0/2622887598 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83680039e0 0x7f8368003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.820+0000 7f837f59e640 1 -- 172.21.15.16:0/2622887598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8368004340 msgr2=0x7f8368004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.820+0000 7f837f59e640 1 --2- 172.21.15.16:0/2622887598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8368004340 0x7f8368004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.820+0000 7f837f59e640 1 -- 172.21.15.16:0/2622887598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8368003520 con 0x7f8368005020 2024-09-19T07:57:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.821+0000 7f837f59e640 1 --2- 172.21.15.16:0/2622887598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8368005020 0x7f836800b9c0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f8378001e50 tx=0x7f8368005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4419a6d2cff3098 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.821+0000 7f837dd9b640 1 -- 172.21.15.16:0/2622887598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8378001a70 con 0x7f8368005020 2024-09-19T07:57:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.821+0000 7f837dd9b640 1 -- 172.21.15.16:0/2622887598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8378003970 con 0x7f8368005020 2024-09-19T07:57:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.821+0000 7f837dd9b640 1 -- 172.21.15.16:0/2622887598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8378003d60 con 0x7f8368005020 2024-09-19T07:57:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.821+0000 7f83850dd640 1 -- 172.21.15.16:0/2622887598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8368005020 msgr2=0x7f836800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.821+0000 7f83850dd640 1 --2- 172.21.15.16:0/2622887598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8368005020 0x7f836800b9c0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f8378001e50 tx=0x7f8368005830 comp rx=0 tx=0).stop 2024-09-19T07:57:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.821+0000 7f837f59e640 1 -- 172.21.15.16:0/2622887598 reap_dead start 2024-09-19T07:57:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.821+0000 7f83850dd640 1 -- 172.21.15.16:0/2622887598 shutdown_connections 2024-09-19T07:57:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.821+0000 7f83850dd640 1 -- 172.21.15.16:0/2622887598 >> 172.21.15.16:0/2622887598 conn(0x7f8380067950 msgr2=0x7f8380067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.821+0000 7f83850dd640 1 -- 172.21.15.16:0/2622887598 shutdown_connections 2024-09-19T07:57:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.821+0000 7f83850dd640 1 -- 172.21.15.16:0/2622887598 wait complete. 2024-09-19T07:57:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.823+0000 7f83850dd640 1 Processor -- start 2024-09-19T07:57:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.823+0000 7f83850dd640 1 -- start start 2024-09-19T07:57:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f83850dd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83801f4ca0 0x7f83800c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f83850dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83801f5080 0x7f838015eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f83850dd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83800c77d0 0x7f838015f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f83850dd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83800cc7f0 con 0x7f83801f5080 2024-09-19T07:57:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f83850dd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f83800d3050 con 0x7f83800c77d0 2024-09-19T07:57:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f83850dd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f83800cc970 con 0x7f83801f4ca0 2024-09-19T07:57:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f837e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83801f5080 0x7f838015eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f837e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83801f5080 0x7f838015eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42356/0 (socket says 172.21.15.16:42356) 2024-09-19T07:57:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f837e59c640 1 -- 172.21.15.16:0/74120094 learned_addr learned my addr 172.21.15.16:0/74120094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f837ed9d640 1 --2- 172.21.15.16:0/74120094 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83801f4ca0 0x7f83800c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f837f59e640 1 --2- 172.21.15.16:0/74120094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83800c77d0 0x7f838015f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f837e59c640 1 -- 172.21.15.16:0/74120094 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83801f4ca0 msgr2=0x7f83800c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f837e59c640 1 --2- 172.21.15.16:0/74120094 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83801f4ca0 0x7f83800c7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f837e59c640 1 -- 172.21.15.16:0/74120094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83800c77d0 msgr2=0x7f838015f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f837e59c640 1 --2- 172.21.15.16:0/74120094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83800c77d0 0x7f838015f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f837e59c640 1 -- 172.21.15.16:0/74120094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83800c7c90 con 0x7f83801f5080 2024-09-19T07:57:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f837e59c640 1 --2- 172.21.15.16:0/74120094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83801f5080 0x7f838015eb90 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f8374011910 tx=0x7f8374011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f83677fe640 1 -- 172.21.15.16:0/74120094 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8374019070 con 0x7f83801f5080 2024-09-19T07:57:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f83677fe640 1 -- 172.21.15.16:0/74120094 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8374004240 con 0x7f83801f5080 2024-09-19T07:57:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f83850dd640 1 -- 172.21.15.16:0/74120094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83800c7f70 con 0x7f83801f5080 2024-09-19T07:57:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.824+0000 7f83677fe640 1 -- 172.21.15.16:0/74120094 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837400f8e0 con 0x7f83801f5080 2024-09-19T07:57:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.825+0000 7f83850dd640 1 -- 172.21.15.16:0/74120094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83800c8530 con 0x7f83801f5080 2024-09-19T07:57:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.825+0000 7f83850dd640 1 -- 172.21.15.16:0/74120094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83800ccd50 con 0x7f83801f5080 2024-09-19T07:57:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.826+0000 7f83677fe640 1 -- 172.21.15.16:0/74120094 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f837401c070 con 0x7f83801f5080 2024-09-19T07:57:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.827+0000 7f83677fe640 1 --2- 172.21.15.16:0/74120094 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f836c06c2b0 0x7f836c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.827+0000 7f83677fe640 1 -- 172.21.15.16:0/74120094 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f837408b6e0 con 0x7f83801f5080 2024-09-19T07:57:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.827+0000 7f837ed9d640 1 --2- 172.21.15.16:0/74120094 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f836c06c2b0 0x7f836c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:27.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.828+0000 7f837ed9d640 1 --2- 172.21.15.16:0/74120094 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f836c06c2b0 0x7f836c06e770 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8370007d50 tx=0x7f83700078f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:27.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:27.831+0000 7f83677fe640 1 -- 172.21.15.16:0/74120094 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f83740599d0 con 0x7f83801f5080 2024-09-19T07:57:28.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.008+0000 7f83850dd640 1 -- 172.21.15.16:0/74120094 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f838015ccd0 con 0x7f836c06c2b0 2024-09-19T07:57:28.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.009+0000 7f83677fe640 1 -- 172.21.15.16:0/74120094 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f838015ccd0 con 0x7f836c06c2b0 2024-09-19T07:57:28.009 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 2 to deep-scrub 2024-09-19T07:57:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.011+0000 7f83850dd640 1 -- 172.21.15.16:0/74120094 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f836c06c2b0 msgr2=0x7f836c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.011+0000 7f83850dd640 1 --2- 172.21.15.16:0/74120094 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f836c06c2b0 0x7f836c06e770 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8370007d50 tx=0x7f83700078f0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.011+0000 7f83850dd640 1 -- 172.21.15.16:0/74120094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83801f5080 msgr2=0x7f838015eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.011+0000 7f83850dd640 1 --2- 172.21.15.16:0/74120094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83801f5080 0x7f838015eb90 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f8374011910 tx=0x7f8374011dd0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.011+0000 7f83850dd640 1 -- 172.21.15.16:0/74120094 shutdown_connections 2024-09-19T07:57:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.011+0000 7f83850dd640 1 --2- 172.21.15.16:0/74120094 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f836c06c2b0 0x7f836c06e770 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.011+0000 7f83850dd640 1 --2- 172.21.15.16:0/74120094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83800c77d0 0x7f838015f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.011+0000 7f83850dd640 1 --2- 172.21.15.16:0/74120094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83801f5080 0x7f838015eb90 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.011+0000 7f83850dd640 1 --2- 172.21.15.16:0/74120094 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83801f4ca0 0x7f83800c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.011+0000 7f83850dd640 1 -- 172.21.15.16:0/74120094 >> 172.21.15.16:0/74120094 conn(0x7f8380067950 msgr2=0x7f83800b7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.011+0000 7f83850dd640 1 -- 172.21.15.16:0/74120094 shutdown_connections 2024-09-19T07:57:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.011+0000 7f83850dd640 1 -- 172.21.15.16:0/74120094 wait complete. 2024-09-19T07:57:28.021 INFO:tasks.ceph:Scrubbing osd.3 2024-09-19T07:57:28.022 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-19T07:57:28.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.127+0000 7f4913b13640 1 Processor -- start 2024-09-19T07:57:28.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.127+0000 7f4913b13640 1 -- start start 2024-09-19T07:57:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.128+0000 7f4913b13640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f490c15c5d0 0x7f490c071940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.128+0000 7f4913b13640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f490c0b63f0 0x7f490c0d1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.128+0000 7f4913b13640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f490c0d04d0 con 0x7f490c0d1310 2024-09-19T07:57:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.128+0000 7f4913b13640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f490c0d1cb0 con 0x7f490c15c5d0 2024-09-19T07:57:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.128+0000 7f4913b13640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f490c0d07b0 con 0x7f490c0b63f0 2024-09-19T07:57:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.128+0000 7f49118af640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f490c15c5d0 0x7f490c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.128+0000 7f49118af640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f490c15c5d0 0x7f490c071940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42360/0 (socket says 172.21.15.16:42360) 2024-09-19T07:57:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.128+0000 7f49118af640 1 -- 172.21.15.16:0/1388510178 learned_addr learned my addr 172.21.15.16:0/1388510178 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.129+0000 7f49120b0640 1 --2- 172.21.15.16:0/1388510178 >> v2:172.21.15.146:3300/0 conn(0x7f490c0b63f0 0x7f490c0d1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.129+0000 7f49118af640 1 -- 172.21.15.16:0/1388510178 >> v2:172.21.15.146:3300/0 conn(0x7f490c0b63f0 msgr2=0x7f490c0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.129+0000 7f49118af640 1 --2- 172.21.15.16:0/1388510178 >> v2:172.21.15.146:3300/0 conn(0x7f490c0b63f0 0x7f490c0d1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.129+0000 7f49118af640 1 -- 172.21.15.16:0/1388510178 >> v1:172.21.15.146:6789/0 conn(0x7f490c0d1310 legacy=0x7f490c15ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.129+0000 7f49118af640 1 -- 172.21.15.16:0/1388510178 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c0d0a90 con 0x7f490c15c5d0 2024-09-19T07:57:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.129+0000 7f49118af640 1 --2- 172.21.15.16:0/1388510178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f490c15c5d0 0x7f490c071940 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f490c0cd130 tx=0x7f490402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05df2744e3d1966 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.130+0000 7f49108ad640 1 -- 172.21.15.16:0/1388510178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904037070 con 0x7f490c15c5d0 2024-09-19T07:57:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.130+0000 7f49108ad640 1 -- 172.21.15.16:0/1388510178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4904032440 con 0x7f490c15c5d0 2024-09-19T07:57:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.130+0000 7f49108ad640 1 -- 172.21.15.16:0/1388510178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904032810 con 0x7f490c15c5d0 2024-09-19T07:57:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.130+0000 7f4913b13640 1 -- 172.21.15.16:0/1388510178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f490c15c5d0 msgr2=0x7f490c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.130+0000 7f4913b13640 1 --2- 172.21.15.16:0/1388510178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f490c15c5d0 0x7f490c071940 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f490c0cd130 tx=0x7f490402a230 comp rx=0 tx=0).stop 2024-09-19T07:57:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.130+0000 7f4913b13640 1 -- 172.21.15.16:0/1388510178 shutdown_connections 2024-09-19T07:57:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.130+0000 7f4913b13640 1 --2- 172.21.15.16:0/1388510178 >> v2:172.21.15.146:3300/0 conn(0x7f490c0b63f0 0x7f490c0d1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.130+0000 7f4913b13640 1 --2- 172.21.15.16:0/1388510178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f490c15c5d0 0x7f490c071940 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.130+0000 7f4913b13640 1 -- 172.21.15.16:0/1388510178 >> 172.21.15.16:0/1388510178 conn(0x7f490c06fef0 msgr2=0x7f490c0702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.130+0000 7f4913b13640 1 -- 172.21.15.16:0/1388510178 shutdown_connections 2024-09-19T07:57:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.130+0000 7f4913b13640 1 -- 172.21.15.16:0/1388510178 wait complete. 2024-09-19T07:57:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.131+0000 7f4913b13640 1 Processor -- start 2024-09-19T07:57:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.132+0000 7f4913b13640 1 -- start start 2024-09-19T07:57:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.132+0000 7f4913b13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f490c0b63f0 0x7f490c149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.132+0000 7f4913b13640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c0d1310 0x7f490c160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.132+0000 7f4913b13640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f490c15c5d0 0x7f490c0c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.132+0000 7f4913b13640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f490c0d0110 con 0x7f490c0b63f0 2024-09-19T07:57:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.132+0000 7f4913b13640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f490c0d1050 con 0x7f490c0d1310 2024-09-19T07:57:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.132+0000 7f4913b13640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f490c0d0290 con 0x7f490c15c5d0 2024-09-19T07:57:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.132+0000 7f49118af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f490c0b63f0 0x7f490c149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.132+0000 7f49118af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f490c0b63f0 0x7f490c149e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42366/0 (socket says 172.21.15.16:42366) 2024-09-19T07:57:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.132+0000 7f49118af640 1 -- 172.21.15.16:0/4000160135 learned_addr learned my addr 172.21.15.16:0/4000160135 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.132+0000 7f49110ae640 1 --2- 172.21.15.16:0/4000160135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c0d1310 0x7f490c160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.133+0000 7f49120b0640 1 --2- 172.21.15.16:0/4000160135 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f490c15c5d0 0x7f490c0c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.133+0000 7f49118af640 1 -- 172.21.15.16:0/4000160135 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f490c15c5d0 msgr2=0x7f490c0c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.133+0000 7f49118af640 1 --2- 172.21.15.16:0/4000160135 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f490c15c5d0 0x7f490c0c9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.133+0000 7f49118af640 1 -- 172.21.15.16:0/4000160135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c0d1310 msgr2=0x7f490c160810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.133+0000 7f49118af640 1 --2- 172.21.15.16:0/4000160135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c0d1310 0x7f490c160810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.133+0000 7f49118af640 1 -- 172.21.15.16:0/4000160135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c14d740 con 0x7f490c0b63f0 2024-09-19T07:57:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.133+0000 7f49118af640 1 --2- 172.21.15.16:0/4000160135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f490c0b63f0 0x7f490c149e40 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f4904000c00 tx=0x7f49040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.133+0000 7f490a7fc640 1 -- 172.21.15.16:0/4000160135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904041020 con 0x7f490c0b63f0 2024-09-19T07:57:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.133+0000 7f490a7fc640 1 -- 172.21.15.16:0/4000160135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49040306c0 con 0x7f490c0b63f0 2024-09-19T07:57:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.133+0000 7f490a7fc640 1 -- 172.21.15.16:0/4000160135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904037040 con 0x7f490c0b63f0 2024-09-19T07:57:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.133+0000 7f4913b13640 1 -- 172.21.15.16:0/4000160135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490c0cc000 con 0x7f490c0b63f0 2024-09-19T07:57:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.133+0000 7f4913b13640 1 -- 172.21.15.16:0/4000160135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f490c0cc5c0 con 0x7f490c0b63f0 2024-09-19T07:57:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.134+0000 7f4913b13640 1 -- 172.21.15.16:0/4000160135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f490c0d1770 con 0x7f490c0b63f0 2024-09-19T07:57:28.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.135+0000 7f490a7fc640 1 -- 172.21.15.16:0/4000160135 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4904044020 con 0x7f490c0b63f0 2024-09-19T07:57:28.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.135+0000 7f490a7fc640 1 --2- 172.21.15.16:0/4000160135 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f48f806c2b0 0x7f48f806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.135+0000 7f490a7fc640 1 -- 172.21.15.16:0/4000160135 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f49040abf40 con 0x7f490c0b63f0 2024-09-19T07:57:28.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.135+0000 7f490a7fc640 1 --2- 172.21.15.16:0/4000160135 >> [v2:172.21.15.16:6816/3748453908,v1:172.21.15.16:6817/3748453908] conn(0x7f48f80727a0 0x7f48f8074be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.135+0000 7f490a7fc640 1 -- 172.21.15.16:0/4000160135 --> [v2:172.21.15.16:6816/3748453908,v1:172.21.15.16:6817/3748453908] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f490403de50 con 0x7f48f80727a0 2024-09-19T07:57:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.135+0000 7f490a7fc640 1 -- 172.21.15.16:0/4000160135 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=156) ==== 24+0+0 (secure 0 0 0) 0x7f49040ac3d0 con 0x7f490c0b63f0 2024-09-19T07:57:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.135+0000 7f49110ae640 1 --2- 172.21.15.16:0/4000160135 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f48f806c2b0 0x7f48f806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.135+0000 7f49120b0640 1 --2- 172.21.15.16:0/4000160135 >> [v2:172.21.15.16:6816/3748453908,v1:172.21.15.16:6817/3748453908] conn(0x7f48f80727a0 0x7f48f8074be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.136+0000 7f49120b0640 1 --2- 172.21.15.16:0/4000160135 >> [v2:172.21.15.16:6816/3748453908,v1:172.21.15.16:6817/3748453908] conn(0x7f48f80727a0 0x7f48f8074be0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.136+0000 7f49110ae640 1 --2- 172.21.15.16:0/4000160135 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f48f806c2b0 0x7f48f806e770 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f490c1f5400 tx=0x7f4900007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.136+0000 7f490a7fc640 1 -- 172.21.15.16:0/4000160135 <== osd.3 v2:172.21.15.16:6816/3748453908 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f490403de50 con 0x7f48f80727a0 2024-09-19T07:57:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.149+0000 7f4913b13640 1 -- 172.21.15.16:0/4000160135 --> [v2:172.21.15.16:6816/3748453908,v1:172.21.15.16:6817/3748453908] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f490c071400 con 0x7f48f80727a0 2024-09-19T07:57:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.149+0000 7f490a7fc640 1 -- 172.21.15.16:0/4000160135 <== osd.3 v2:172.21.15.16:6816/3748453908 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f490c071400 con 0x7f48f80727a0 2024-09-19T07:57:28.150 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:57:28.150 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T07:57:28.150 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:57:28.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.150+0000 7f4913b13640 1 -- 172.21.15.16:0/4000160135 >> [v2:172.21.15.16:6816/3748453908,v1:172.21.15.16:6817/3748453908] conn(0x7f48f80727a0 msgr2=0x7f48f8074be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.150+0000 7f4913b13640 1 --2- 172.21.15.16:0/4000160135 >> [v2:172.21.15.16:6816/3748453908,v1:172.21.15.16:6817/3748453908] conn(0x7f48f80727a0 0x7f48f8074be0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.150+0000 7f4913b13640 1 -- 172.21.15.16:0/4000160135 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f48f806c2b0 msgr2=0x7f48f806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.150+0000 7f4913b13640 1 --2- 172.21.15.16:0/4000160135 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f48f806c2b0 0x7f48f806e770 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f490c1f5400 tx=0x7f4900007e30 comp rx=0 tx=0).stop 2024-09-19T07:57:28.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.150+0000 7f4913b13640 1 -- 172.21.15.16:0/4000160135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f490c0b63f0 msgr2=0x7f490c149e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.150+0000 7f4913b13640 1 --2- 172.21.15.16:0/4000160135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f490c0b63f0 0x7f490c149e40 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f4904000c00 tx=0x7f49040046c0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.150+0000 7f49120b0640 1 -- 172.21.15.16:0/4000160135 reap_dead start 2024-09-19T07:57:28.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.150+0000 7f4913b13640 1 -- 172.21.15.16:0/4000160135 shutdown_connections 2024-09-19T07:57:28.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.150+0000 7f4913b13640 1 -- 172.21.15.16:0/4000160135 >> 172.21.15.16:0/4000160135 conn(0x7f490c06fef0 msgr2=0x7f490c0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:28.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.150+0000 7f4913b13640 1 -- 172.21.15.16:0/4000160135 shutdown_connections 2024-09-19T07:57:28.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.150+0000 7f4913b13640 1 -- 172.21.15.16:0/4000160135 wait complete. 2024-09-19T07:57:28.160 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-19T07:57:28.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.263+0000 7f876100e640 1 Processor -- start 2024-09-19T07:57:28.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.263+0000 7f876100e640 1 -- start start 2024-09-19T07:57:28.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.264+0000 7f876100e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f875c0d0aa0 0x7f875c0d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.264+0000 7f876100e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f875c0d52c0 con 0x7f875c15e660 2024-09-19T07:57:28.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.264+0000 7f876100e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f875c15ea40 con 0x7f875c0d4ea0 2024-09-19T07:57:28.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.264+0000 7f876100e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f875c0d3100 con 0x7f875c0d0aa0 2024-09-19T07:57:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.264+0000 7f875ad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f875c15e660 0x7f875c0d2940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60920/0 (socket says 172.21.15.16:60920) 2024-09-19T07:57:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.264+0000 7f875ad9d640 1 -- 172.21.15.16:0/3642699174 learned_addr learned my addr 172.21.15.16:0/3642699174 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f875b59e640 1 --2- 172.21.15.16:0/3642699174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f875c0d0aa0 0x7f875c0d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f8759d9b640 1 -- 172.21.15.16:0/3642699174 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2929280244 0 0) 0x7f875c0d52c0 con 0x7f875c15e660 2024-09-19T07:57:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f8759d9b640 1 -- 172.21.15.16:0/3642699174 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8744003520 con 0x7f875c15e660 2024-09-19T07:57:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f8759d9b640 1 -- 172.21.15.16:0/3642699174 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 20732823 0 0) 0x7f875c15ea40 con 0x7f875c0d4ea0 2024-09-19T07:57:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f8759d9b640 1 -- 172.21.15.16:0/3642699174 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f875c0d52c0 con 0x7f875c0d4ea0 2024-09-19T07:57:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f8759d9b640 1 -- 172.21.15.16:0/3642699174 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2507341672 0 0) 0x7f8744003520 con 0x7f875c15e660 2024-09-19T07:57:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f8759d9b640 1 -- 172.21.15.16:0/3642699174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f875c0d0aa0 msgr2=0x7f875c0d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f8759d9b640 1 --2- 172.21.15.16:0/3642699174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f875c0d0aa0 0x7f875c0d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f8759d9b640 1 -- 172.21.15.16:0/3642699174 >> v1:172.21.15.146:6789/0 conn(0x7f875c0d4ea0 legacy=0x7f875c0cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f8759d9b640 1 -- 172.21.15.16:0/3642699174 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875c0cf230 con 0x7f875c15e660 2024-09-19T07:57:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f8759d9b640 1 -- 172.21.15.16:0/3642699174 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f874c001670 con 0x7f875c15e660 2024-09-19T07:57:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f8759d9b640 1 -- 172.21.15.16:0/3642699174 >> v1:172.21.15.16:6789/0 conn(0x7f875c15e660 legacy=0x7f875c0d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f8759d9b640 1 --2- 172.21.15.16:0/3642699174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8744003b70 0x7f8744003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f8759d9b640 1 --2- 172.21.15.16:0/3642699174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8744004800 0x7f8744004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f8759d9b640 1 --2- 172.21.15.16:0/3642699174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f875c0d0aa0 0x7f87440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f8759d9b640 1 -- 172.21.15.16:0/3642699174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f875c0d3100 con 0x7f8744003b70 2024-09-19T07:57:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f8759d9b640 1 -- 172.21.15.16:0/3642699174 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f874400baa0 con 0x7f8744004800 2024-09-19T07:57:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f8759d9b640 1 -- 172.21.15.16:0/3642699174 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f874400bca0 con 0x7f875c0d0aa0 2024-09-19T07:57:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f875ad9d640 1 --2- 172.21.15.16:0/3642699174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8744003b70 0x7f8744003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f875b59e640 1 --2- 172.21.15.16:0/3642699174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f875c0d0aa0 0x7f87440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f875a59c640 1 --2- 172.21.15.16:0/3642699174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8744004800 0x7f8744004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f875ad9d640 1 -- 172.21.15.16:0/3642699174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f875c0d0aa0 msgr2=0x7f87440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f875ad9d640 1 --2- 172.21.15.16:0/3642699174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f875c0d0aa0 0x7f87440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f875ad9d640 1 -- 172.21.15.16:0/3642699174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8744004800 msgr2=0x7f8744004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f875ad9d640 1 --2- 172.21.15.16:0/3642699174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8744004800 0x7f8744004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f875ad9d640 1 -- 172.21.15.16:0/3642699174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8744003520 con 0x7f8744003b70 2024-09-19T07:57:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.265+0000 7f875a59c640 1 --2- 172.21.15.16:0/3642699174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8744004800 0x7f8744004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.266+0000 7f875b59e640 1 --2- 172.21.15.16:0/3642699174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f875c0d0aa0 0x7f87440051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.266+0000 7f875ad9d640 1 --2- 172.21.15.16:0/3642699174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8744003b70 0x7f8744003f90 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f874c00c160 tx=0x7f8744005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb115e057625d1c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.266+0000 7f8759d9b640 1 -- 172.21.15.16:0/3642699174 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c007cc0 con 0x7f8744003b70 2024-09-19T07:57:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.266+0000 7f8759d9b640 1 -- 172.21.15.16:0/3642699174 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f874c005af0 con 0x7f8744003b70 2024-09-19T07:57:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.266+0000 7f8759d9b640 1 -- 172.21.15.16:0/3642699174 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c007070 con 0x7f8744003b70 2024-09-19T07:57:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.267+0000 7f876100e640 1 -- 172.21.15.16:0/3642699174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8744003b70 msgr2=0x7f8744003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.267+0000 7f876100e640 1 --2- 172.21.15.16:0/3642699174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8744003b70 0x7f8744003f90 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f874c00c160 tx=0x7f8744005830 comp rx=0 tx=0).stop 2024-09-19T07:57:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.267+0000 7f875b59e640 1 -- 172.21.15.16:0/3642699174 reap_dead start 2024-09-19T07:57:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.267+0000 7f876100e640 1 -- 172.21.15.16:0/3642699174 shutdown_connections 2024-09-19T07:57:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.267+0000 7f876100e640 1 -- 172.21.15.16:0/3642699174 >> 172.21.15.16:0/3642699174 conn(0x7f875c09d720 msgr2=0x7f875c0d4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:28.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.267+0000 7f876100e640 1 -- 172.21.15.16:0/3642699174 shutdown_connections 2024-09-19T07:57:28.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.267+0000 7f876100e640 1 -- 172.21.15.16:0/3642699174 wait complete. 2024-09-19T07:57:28.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.268+0000 7f876100e640 1 Processor -- start 2024-09-19T07:57:28.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.268+0000 7f876100e640 1 -- start start 2024-09-19T07:57:28.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.269+0000 7f876100e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8744003b70 0x7f875c155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.269+0000 7f876100e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f875c1597b0 0x7f875c0cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.269+0000 7f876100e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f875c0b5fe0 0x7f875c0b6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.269+0000 7f876100e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f875c0cee70 con 0x7f8744003b70 2024-09-19T07:57:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.269+0000 7f876100e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f875c0cf7f0 con 0x7f875c0b5fe0 2024-09-19T07:57:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.269+0000 7f876100e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f875c0ceff0 con 0x7f875c1597b0 2024-09-19T07:57:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.269+0000 7f875ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8744003b70 0x7f875c155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.269+0000 7f875ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8744003b70 0x7f875c155bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42378/0 (socket says 172.21.15.16:42378) 2024-09-19T07:57:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.269+0000 7f875ad9d640 1 -- 172.21.15.16:0/3737940427 learned_addr learned my addr 172.21.15.16:0/3737940427 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.269+0000 7f875a59c640 1 --2- 172.21.15.16:0/3737940427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f875c1597b0 0x7f875c0cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.269+0000 7f875b59e640 1 --2- 172.21.15.16:0/3737940427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f875c0b5fe0 0x7f875c0b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.270+0000 7f875ad9d640 1 -- 172.21.15.16:0/3737940427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f875c1597b0 msgr2=0x7f875c0cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.270+0000 7f875ad9d640 1 --2- 172.21.15.16:0/3737940427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f875c1597b0 0x7f875c0cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.270+0000 7f875ad9d640 1 -- 172.21.15.16:0/3737940427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f875c0b5fe0 msgr2=0x7f875c0b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.270+0000 7f875ad9d640 1 --2- 172.21.15.16:0/3737940427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f875c0b5fe0 0x7f875c0b6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.270+0000 7f875ad9d640 1 -- 172.21.15.16:0/3737940427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875c0b4830 con 0x7f8744003b70 2024-09-19T07:57:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.270+0000 7f875a59c640 1 --2- 172.21.15.16:0/3737940427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f875c1597b0 0x7f875c0cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.270+0000 7f875ad9d640 1 --2- 172.21.15.16:0/3737940427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8744003b70 0x7f875c155bb0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f874c02aca0 tx=0x7f875c0cedc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.270+0000 7f87437fe640 1 -- 172.21.15.16:0/3737940427 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c001670 con 0x7f8744003b70 2024-09-19T07:57:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.270+0000 7f87437fe640 1 -- 172.21.15.16:0/3737940427 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f874c042070 con 0x7f8744003b70 2024-09-19T07:57:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.270+0000 7f87437fe640 1 -- 172.21.15.16:0/3737940427 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c007290 con 0x7f8744003b70 2024-09-19T07:57:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.270+0000 7f876100e640 1 -- 172.21.15.16:0/3737940427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f875c0b4b10 con 0x7f8744003b70 2024-09-19T07:57:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.270+0000 7f876100e640 1 -- 172.21.15.16:0/3737940427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f875c0b50d0 con 0x7f8744003b70 2024-09-19T07:57:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.271+0000 7f876100e640 1 -- 172.21.15.16:0/3737940427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f875c0d15d0 con 0x7f8744003b70 2024-09-19T07:57:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.272+0000 7f87437fe640 1 -- 172.21.15.16:0/3737940427 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f874c007430 con 0x7f8744003b70 2024-09-19T07:57:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.272+0000 7f87437fe640 1 --2- 172.21.15.16:0/3737940427 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f874806c2e0 0x7f874806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.272+0000 7f87437fe640 1 -- 172.21.15.16:0/3737940427 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f874c0ab280 con 0x7f8744003b70 2024-09-19T07:57:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.272+0000 7f875a59c640 1 --2- 172.21.15.16:0/3737940427 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f874806c2e0 0x7f874806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.273+0000 7f875a59c640 1 --2- 172.21.15.16:0/3737940427 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f874806c2e0 0x7f874806e7a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f875c1f7de0 tx=0x7f8750009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:28.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.276+0000 7f87437fe640 1 -- 172.21.15.16:0/3737940427 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f875c0d15d0 con 0x7f8744003b70 2024-09-19T07:57:28.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.452+0000 7f876100e640 1 -- 172.21.15.16:0/3737940427 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f875c0b5440 con 0x7f874806c2e0 2024-09-19T07:57:28.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.453+0000 7f87437fe640 1 -- 172.21.15.16:0/3737940427 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f875c0b5440 con 0x7f874806c2e0 2024-09-19T07:57:28.453 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 3 to deep-scrub 2024-09-19T07:57:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.455+0000 7f876100e640 1 -- 172.21.15.16:0/3737940427 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f874806c2e0 msgr2=0x7f874806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.455+0000 7f876100e640 1 --2- 172.21.15.16:0/3737940427 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f874806c2e0 0x7f874806e7a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f875c1f7de0 tx=0x7f8750009040 comp rx=0 tx=0).stop 2024-09-19T07:57:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.455+0000 7f876100e640 1 -- 172.21.15.16:0/3737940427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8744003b70 msgr2=0x7f875c155bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.455+0000 7f876100e640 1 --2- 172.21.15.16:0/3737940427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8744003b70 0x7f875c155bb0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f874c02aca0 tx=0x7f875c0cedc0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.455+0000 7f876100e640 1 -- 172.21.15.16:0/3737940427 shutdown_connections 2024-09-19T07:57:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.455+0000 7f876100e640 1 --2- 172.21.15.16:0/3737940427 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f874806c2e0 0x7f874806e7a0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.455+0000 7f876100e640 1 --2- 172.21.15.16:0/3737940427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f875c0b5fe0 0x7f875c0b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.455+0000 7f876100e640 1 --2- 172.21.15.16:0/3737940427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f875c1597b0 0x7f875c0cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.455+0000 7f876100e640 1 --2- 172.21.15.16:0/3737940427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8744003b70 0x7f875c155bb0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.455+0000 7f876100e640 1 -- 172.21.15.16:0/3737940427 >> 172.21.15.16:0/3737940427 conn(0x7f875c09d720 msgr2=0x7f875c0b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:28.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.455+0000 7f876100e640 1 -- 172.21.15.16:0/3737940427 shutdown_connections 2024-09-19T07:57:28.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.455+0000 7f876100e640 1 -- 172.21.15.16:0/3737940427 wait complete. 2024-09-19T07:57:28.466 INFO:tasks.ceph:Scrubbing osd.4 2024-09-19T07:57:28.466 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-19T07:57:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.572+0000 7efe267e4640 1 Processor -- start 2024-09-19T07:57:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.572+0000 7efe267e4640 1 -- start start 2024-09-19T07:57:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.573+0000 7efe267e4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efe2015e790 0x7efe200d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.573+0000 7efe267e4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efe200d5040 0x7efe200d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.573+0000 7efe267e4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe200cf0a0 con 0x7efe200d1340 2024-09-19T07:57:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.573+0000 7efe267e4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7efe2006db40 con 0x7efe200d5040 2024-09-19T07:57:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.573+0000 7efe267e4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efe200cf380 con 0x7efe2015e790 2024-09-19T07:57:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.573+0000 7efe1f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efe200d5040 0x7efe200d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.573+0000 7efe24d81640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efe200d1340 0x7efe200d2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60934/0 (socket says 172.21.15.16:60934) 2024-09-19T07:57:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.573+0000 7efe1f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efe200d5040 0x7efe200d0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42386/0 (socket says 172.21.15.16:42386) 2024-09-19T07:57:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.573+0000 7efe24d81640 1 -- 172.21.15.16:0/3956173630 learned_addr learned my addr 172.21.15.16:0/3956173630 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.574+0000 7efe1ffff640 1 --2- 172.21.15.16:0/3956173630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efe2015e790 0x7efe200d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.574+0000 7efe1f7fe640 1 -- 172.21.15.16:0/3956173630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efe2015e790 msgr2=0x7efe200d4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.574+0000 7efe1f7fe640 1 --2- 172.21.15.16:0/3956173630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efe2015e790 0x7efe200d4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.574+0000 7efe1f7fe640 1 -- 172.21.15.16:0/3956173630 >> v1:172.21.15.16:6789/0 conn(0x7efe200d1340 legacy=0x7efe200d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.574+0000 7efe1effd640 1 -- 172.21.15.16:0/3956173630 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2958155818 0 0) 0x7efe200cf0a0 con 0x7efe200d1340 2024-09-19T07:57:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.574+0000 7efe1f7fe640 1 -- 172.21.15.16:0/3956173630 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe200cf660 con 0x7efe200d5040 2024-09-19T07:57:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.574+0000 7efe1ffff640 1 --2- 172.21.15.16:0/3956173630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efe2015e790 0x7efe200d4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.575+0000 7efe1f7fe640 1 --2- 172.21.15.16:0/3956173630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe200d5040 0x7efe200d0e00 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7efe140097a0 tx=0x7efe1402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d652e01a5ca5c37e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.575+0000 7efe1effd640 1 -- 172.21.15.16:0/3956173630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14037070 con 0x7efe200d5040 2024-09-19T07:57:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.575+0000 7efe1effd640 1 -- 172.21.15.16:0/3956173630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe14032440 con 0x7efe200d5040 2024-09-19T07:57:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.575+0000 7efe1effd640 1 -- 172.21.15.16:0/3956173630 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14032810 con 0x7efe200d5040 2024-09-19T07:57:28.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.575+0000 7efe267e4640 1 -- 172.21.15.16:0/3956173630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe200d5040 msgr2=0x7efe200d0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.575+0000 7efe267e4640 1 --2- 172.21.15.16:0/3956173630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe200d5040 0x7efe200d0e00 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7efe140097a0 tx=0x7efe1402a210 comp rx=0 tx=0).stop 2024-09-19T07:57:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.575+0000 7efe267e4640 1 -- 172.21.15.16:0/3956173630 shutdown_connections 2024-09-19T07:57:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.575+0000 7efe267e4640 1 --2- 172.21.15.16:0/3956173630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe200d5040 0x7efe200d0e00 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.575+0000 7efe267e4640 1 --2- 172.21.15.16:0/3956173630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efe2015e790 0x7efe200d4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.575+0000 7efe267e4640 1 -- 172.21.15.16:0/3956173630 >> 172.21.15.16:0/3956173630 conn(0x7efe2009d740 msgr2=0x7efe200d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.575+0000 7efe267e4640 1 -- 172.21.15.16:0/3956173630 shutdown_connections 2024-09-19T07:57:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.575+0000 7efe267e4640 1 -- 172.21.15.16:0/3956173630 wait complete. 2024-09-19T07:57:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.576+0000 7efe267e4640 1 Processor -- start 2024-09-19T07:57:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.577+0000 7efe267e4640 1 -- start start 2024-09-19T07:57:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.577+0000 7efe267e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe200d1340 0x7efe200cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.577+0000 7efe267e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe200d5040 0x7efe201617e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.577+0000 7efe267e4640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efe2015e790 0x7efe200b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.577+0000 7efe267e4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efe200d0140 con 0x7efe200d1340 2024-09-19T07:57:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.577+0000 7efe267e4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efe20162f30 con 0x7efe200d5040 2024-09-19T07:57:28.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.577+0000 7efe267e4640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efe200b82e0 con 0x7efe2015e790 2024-09-19T07:57:28.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.578+0000 7efe1ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe200d1340 0x7efe200cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.578+0000 7efe1ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe200d1340 0x7efe200cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42394/0 (socket says 172.21.15.16:42394) 2024-09-19T07:57:28.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.578+0000 7efe1ffff640 1 -- 172.21.15.16:0/2290091418 learned_addr learned my addr 172.21.15.16:0/2290091418 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:28.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.578+0000 7efe1f7fe640 1 --2- 172.21.15.16:0/2290091418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe200d5040 0x7efe201617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.578+0000 7efe24d81640 1 --2- 172.21.15.16:0/2290091418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efe2015e790 0x7efe200b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.578+0000 7efe1ffff640 1 -- 172.21.15.16:0/2290091418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efe2015e790 msgr2=0x7efe200b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.578+0000 7efe1ffff640 1 --2- 172.21.15.16:0/2290091418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efe2015e790 0x7efe200b2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.578+0000 7efe1ffff640 1 -- 172.21.15.16:0/2290091418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe200d5040 msgr2=0x7efe201617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.578+0000 7efe1ffff640 1 --2- 172.21.15.16:0/2290091418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe200d5040 0x7efe201617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.578+0000 7efe1ffff640 1 -- 172.21.15.16:0/2290091418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe200b76d0 con 0x7efe200d1340 2024-09-19T07:57:28.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.578+0000 7efe1ffff640 1 --2- 172.21.15.16:0/2290091418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe200d1340 0x7efe200cd550 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7efe1000d7d0 tx=0x7efe1000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:28.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.578+0000 7efe1cff9640 1 -- 172.21.15.16:0/2290091418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10004030 con 0x7efe200d1340 2024-09-19T07:57:28.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.578+0000 7efe1cff9640 1 -- 172.21.15.16:0/2290091418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe100041d0 con 0x7efe200d1340 2024-09-19T07:57:28.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.578+0000 7efe1cff9640 1 -- 172.21.15.16:0/2290091418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10009640 con 0x7efe200d1340 2024-09-19T07:57:28.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.578+0000 7efe267e4640 1 -- 172.21.15.16:0/2290091418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe200b79b0 con 0x7efe200d1340 2024-09-19T07:57:28.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.578+0000 7efe267e4640 1 -- 172.21.15.16:0/2290091418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe200b7f70 con 0x7efe200d1340 2024-09-19T07:57:28.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.579+0000 7efe267e4640 1 -- 172.21.15.16:0/2290091418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efe200d1be0 con 0x7efe200d1340 2024-09-19T07:57:28.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.580+0000 7efe1cff9640 1 -- 172.21.15.16:0/2290091418 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7efe100097e0 con 0x7efe200d1340 2024-09-19T07:57:28.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.580+0000 7efe1cff9640 1 --2- 172.21.15.16:0/2290091418 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efe0c06c2b0 0x7efe0c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.580+0000 7efe1cff9640 1 -- 172.21.15.16:0/2290091418 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7efe1008b650 con 0x7efe200d1340 2024-09-19T07:57:28.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.581+0000 7efe1cff9640 1 --2- 172.21.15.16:0/2290091418 >> [v2:172.21.15.146:6808/2073273152,v1:172.21.15.146:6809/2073273152] conn(0x7efe0c0727a0 0x7efe0c074be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.581+0000 7efe1cff9640 1 -- 172.21.15.16:0/2290091418 --> [v2:172.21.15.146:6808/2073273152,v1:172.21.15.146:6809/2073273152] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efe1001ee50 con 0x7efe0c0727a0 2024-09-19T07:57:28.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.581+0000 7efe1cff9640 1 -- 172.21.15.16:0/2290091418 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=156) ==== 24+0+0 (secure 0 0 0) 0x7efe10092050 con 0x7efe200d1340 2024-09-19T07:57:28.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.581+0000 7efe1f7fe640 1 --2- 172.21.15.16:0/2290091418 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efe0c06c2b0 0x7efe0c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.581+0000 7efe24d81640 1 --2- 172.21.15.16:0/2290091418 >> [v2:172.21.15.146:6808/2073273152,v1:172.21.15.146:6809/2073273152] conn(0x7efe0c0727a0 0x7efe0c074be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.581+0000 7efe1f7fe640 1 --2- 172.21.15.16:0/2290091418 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efe0c06c2b0 0x7efe0c06e770 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7efe14034000 tx=0x7efe1402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:28.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.581+0000 7efe24d81640 1 --2- 172.21.15.16:0/2290091418 >> [v2:172.21.15.146:6808/2073273152,v1:172.21.15.146:6809/2073273152] conn(0x7efe0c0727a0 0x7efe0c074be0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:28.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.582+0000 7efe1cff9640 1 -- 172.21.15.16:0/2290091418 <== osd.4 v2:172.21.15.146:6808/2073273152 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe1001ee50 con 0x7efe0c0727a0 2024-09-19T07:57:28.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.595+0000 7efe267e4640 1 -- 172.21.15.16:0/2290091418 --> [v2:172.21.15.146:6808/2073273152,v1:172.21.15.146:6809/2073273152] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efe200b66b0 con 0x7efe0c0727a0 2024-09-19T07:57:28.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.595+0000 7efe1cff9640 1 -- 172.21.15.16:0/2290091418 <== osd.4 v2:172.21.15.146:6808/2073273152 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7efe200b66b0 con 0x7efe0c0727a0 2024-09-19T07:57:28.596 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:57:28.596 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T07:57:28.596 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:57:28.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.596+0000 7efe267e4640 1 -- 172.21.15.16:0/2290091418 >> [v2:172.21.15.146:6808/2073273152,v1:172.21.15.146:6809/2073273152] conn(0x7efe0c0727a0 msgr2=0x7efe0c074be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.596+0000 7efe267e4640 1 --2- 172.21.15.16:0/2290091418 >> [v2:172.21.15.146:6808/2073273152,v1:172.21.15.146:6809/2073273152] conn(0x7efe0c0727a0 0x7efe0c074be0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.596+0000 7efe267e4640 1 -- 172.21.15.16:0/2290091418 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efe0c06c2b0 msgr2=0x7efe0c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.596+0000 7efe267e4640 1 --2- 172.21.15.16:0/2290091418 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7efe0c06c2b0 0x7efe0c06e770 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7efe14034000 tx=0x7efe1402e000 comp rx=0 tx=0).stop 2024-09-19T07:57:28.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.596+0000 7efe267e4640 1 -- 172.21.15.16:0/2290091418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe200d1340 msgr2=0x7efe200cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.596+0000 7efe267e4640 1 --2- 172.21.15.16:0/2290091418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe200d1340 0x7efe200cd550 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7efe1000d7d0 tx=0x7efe1000dc90 comp rx=0 tx=0).stop 2024-09-19T07:57:28.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.596+0000 7efe24d81640 1 -- 172.21.15.16:0/2290091418 reap_dead start 2024-09-19T07:57:28.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.596+0000 7efe267e4640 1 -- 172.21.15.16:0/2290091418 shutdown_connections 2024-09-19T07:57:28.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.596+0000 7efe267e4640 1 -- 172.21.15.16:0/2290091418 >> 172.21.15.16:0/2290091418 conn(0x7efe2009d740 msgr2=0x7efe20153020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:28.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.596+0000 7efe267e4640 1 -- 172.21.15.16:0/2290091418 shutdown_connections 2024-09-19T07:57:28.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.596+0000 7efe267e4640 1 -- 172.21.15.16:0/2290091418 wait complete. 2024-09-19T07:57:28.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-19T07:57:28.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.709+0000 7fc64e3f6640 1 Processor -- start 2024-09-19T07:57:28.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.709+0000 7fc64e3f6640 1 -- start start 2024-09-19T07:57:28.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.710+0000 7fc64e3f6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc6480d0930 0x7fc6480ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.710+0000 7fc64e3f6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc64806bb10 0x7fc64806bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.710+0000 7fc64e3f6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc64806c460 con 0x7fc648073810 2024-09-19T07:57:28.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.710+0000 7fc64e3f6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fc648070b30 con 0x7fc64806bb10 2024-09-19T07:57:28.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.710+0000 7fc64e3f6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc6480cd830 con 0x7fc6480d0930 2024-09-19T07:57:28.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.710+0000 7fc64c993640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc64806bb10 0x7fc64806bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.710+0000 7fc64c993640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc64806bb10 0x7fc64806bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42410/0 (socket says 172.21.15.16:42410) 2024-09-19T07:57:28.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.710+0000 7fc64c993640 1 -- 172.21.15.16:0/3888939626 learned_addr learned my addr 172.21.15.16:0/3888939626 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:28.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.710+0000 7fc647fff640 1 --2- 172.21.15.16:0/3888939626 >> v2:172.21.15.146:3300/0 conn(0x7fc6480d0930 0x7fc6480ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.710+0000 7fc64c993640 1 -- 172.21.15.16:0/3888939626 >> v2:172.21.15.146:3300/0 conn(0x7fc6480d0930 msgr2=0x7fc6480ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:28.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.710+0000 7fc64c993640 1 --2- 172.21.15.16:0/3888939626 >> v2:172.21.15.146:3300/0 conn(0x7fc6480d0930 0x7fc6480ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.710+0000 7fc64c993640 1 -- 172.21.15.16:0/3888939626 >> v1:172.21.15.146:6789/0 conn(0x7fc648073810 legacy=0x7fc648070250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.711+0000 7fc64c993640 1 -- 172.21.15.16:0/3888939626 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6480cdad0 con 0x7fc64806bb10 2024-09-19T07:57:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.711+0000 7fc64c993640 1 --2- 172.21.15.16:0/3888939626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc64806bb10 0x7fc64806bf20 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fc6480b3bb0 tx=0x7fc64002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a136b315991835 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.711+0000 7fc646ffd640 1 -- 172.21.15.16:0/3888939626 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640036070 con 0x7fc64806bb10 2024-09-19T07:57:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.711+0000 7fc646ffd640 1 -- 172.21.15.16:0/3888939626 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc640031600 con 0x7fc64806bb10 2024-09-19T07:57:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.711+0000 7fc646ffd640 1 -- 172.21.15.16:0/3888939626 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6400319d0 con 0x7fc64806bb10 2024-09-19T07:57:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.712+0000 7fc64e3f6640 1 -- 172.21.15.16:0/3888939626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc64806bb10 msgr2=0x7fc64806bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.712+0000 7fc64e3f6640 1 --2- 172.21.15.16:0/3888939626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc64806bb10 0x7fc64806bf20 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fc6480b3bb0 tx=0x7fc64002a6a0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.712+0000 7fc64e3f6640 1 -- 172.21.15.16:0/3888939626 shutdown_connections 2024-09-19T07:57:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.712+0000 7fc64e3f6640 1 --2- 172.21.15.16:0/3888939626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc64806bb10 0x7fc64806bf20 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.712+0000 7fc64e3f6640 1 --2- 172.21.15.16:0/3888939626 >> v2:172.21.15.146:3300/0 conn(0x7fc6480d0930 0x7fc6480ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.712+0000 7fc64e3f6640 1 -- 172.21.15.16:0/3888939626 >> 172.21.15.16:0/3888939626 conn(0x7fc6480b2c90 msgr2=0x7fc6480b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.712+0000 7fc64e3f6640 1 -- 172.21.15.16:0/3888939626 shutdown_connections 2024-09-19T07:57:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.712+0000 7fc64e3f6640 1 -- 172.21.15.16:0/3888939626 wait complete. 2024-09-19T07:57:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.713+0000 7fc64e3f6640 1 Processor -- start 2024-09-19T07:57:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.713+0000 7fc64e3f6640 1 -- start start 2024-09-19T07:57:28.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.714+0000 7fc64e3f6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc64806bb10 0x7fc648152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.714+0000 7fc64e3f6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc648073810 0x7fc64815e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.714+0000 7fc64e3f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6480d0930 0x7fc64815e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.714+0000 7fc64e3f6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6480d0d10 con 0x7fc6480d0930 2024-09-19T07:57:28.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.714+0000 7fc64e3f6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc6480ce090 con 0x7fc648073810 2024-09-19T07:57:28.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.714+0000 7fc64e3f6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc6480cc6e0 con 0x7fc64806bb10 2024-09-19T07:57:28.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.714+0000 7fc64c993640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6480d0930 0x7fc64815e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.714+0000 7fc64c993640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6480d0930 0x7fc64815e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42414/0 (socket says 172.21.15.16:42414) 2024-09-19T07:57:28.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.714+0000 7fc64c993640 1 -- 172.21.15.16:0/1646437457 learned_addr learned my addr 172.21.15.16:0/1646437457 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:28.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.714+0000 7fc647fff640 1 --2- 172.21.15.16:0/1646437457 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc64806bb10 0x7fc648152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.714+0000 7fc6477fe640 1 --2- 172.21.15.16:0/1646437457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc648073810 0x7fc64815e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.715+0000 7fc64c993640 1 -- 172.21.15.16:0/1646437457 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc64806bb10 msgr2=0x7fc648152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.715+0000 7fc64c993640 1 --2- 172.21.15.16:0/1646437457 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc64806bb10 0x7fc648152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.715+0000 7fc64c993640 1 -- 172.21.15.16:0/1646437457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc648073810 msgr2=0x7fc64815e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.715+0000 7fc64c993640 1 --2- 172.21.15.16:0/1646437457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc648073810 0x7fc64815e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.715+0000 7fc64c993640 1 -- 172.21.15.16:0/1646437457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6480c8190 con 0x7fc6480d0930 2024-09-19T07:57:28.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.715+0000 7fc647fff640 1 --2- 172.21.15.16:0/1646437457 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc64806bb10 0x7fc648152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:28.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.715+0000 7fc64c993640 1 --2- 172.21.15.16:0/1646437457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6480d0930 0x7fc64815e590 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fc64002ac90 tx=0x7fc640004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:28.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.715+0000 7fc644ff9640 1 -- 172.21.15.16:0/1646437457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64003f020 con 0x7fc6480d0930 2024-09-19T07:57:28.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.715+0000 7fc644ff9640 1 -- 172.21.15.16:0/1646437457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc640037040 con 0x7fc6480d0930 2024-09-19T07:57:28.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.715+0000 7fc64e3f6640 1 -- 172.21.15.16:0/1646437457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6480c8450 con 0x7fc6480d0930 2024-09-19T07:57:28.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.715+0000 7fc644ff9640 1 -- 172.21.15.16:0/1646437457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640036070 con 0x7fc6480d0930 2024-09-19T07:57:28.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.715+0000 7fc64e3f6640 1 -- 172.21.15.16:0/1646437457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6480c8a10 con 0x7fc6480d0930 2024-09-19T07:57:28.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.716+0000 7fc64e3f6640 1 -- 172.21.15.16:0/1646437457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc64806bf20 con 0x7fc6480d0930 2024-09-19T07:57:28.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.717+0000 7fc644ff9640 1 -- 172.21.15.16:0/1646437457 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc640042020 con 0x7fc6480d0930 2024-09-19T07:57:28.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.717+0000 7fc644ff9640 1 --2- 172.21.15.16:0/1646437457 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc63406c2e0 0x7fc63406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:28.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.717+0000 7fc644ff9640 1 -- 172.21.15.16:0/1646437457 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7fc6400ab610 con 0x7fc6480d0930 2024-09-19T07:57:28.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.717+0000 7fc647fff640 1 --2- 172.21.15.16:0/1646437457 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc63406c2e0 0x7fc63406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:28.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.718+0000 7fc647fff640 1 --2- 172.21.15.16:0/1646437457 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc63406c2e0 0x7fc63406e7a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc638009b90 tx=0x7fc638008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.722+0000 7fc644ff9640 1 -- 172.21.15.16:0/1646437457 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc640049030 con 0x7fc6480d0930 2024-09-19T07:57:28.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.896+0000 7fc64e3f6640 1 -- 172.21.15.16:0/1646437457 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fc6481619c0 con 0x7fc63406c2e0 2024-09-19T07:57:28.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.897+0000 7fc644ff9640 1 -- 172.21.15.16:0/1646437457 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc6481619c0 con 0x7fc63406c2e0 2024-09-19T07:57:28.897 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 4 to deep-scrub 2024-09-19T07:57:28.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.899+0000 7fc64e3f6640 1 -- 172.21.15.16:0/1646437457 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc63406c2e0 msgr2=0x7fc63406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.899+0000 7fc64e3f6640 1 --2- 172.21.15.16:0/1646437457 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc63406c2e0 0x7fc63406e7a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc638009b90 tx=0x7fc638008040 comp rx=0 tx=0).stop 2024-09-19T07:57:28.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.899+0000 7fc64e3f6640 1 -- 172.21.15.16:0/1646437457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6480d0930 msgr2=0x7fc64815e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:28.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.899+0000 7fc64e3f6640 1 --2- 172.21.15.16:0/1646437457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6480d0930 0x7fc64815e590 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fc64002ac90 tx=0x7fc640004a40 comp rx=0 tx=0).stop 2024-09-19T07:57:28.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.899+0000 7fc64e3f6640 1 -- 172.21.15.16:0/1646437457 shutdown_connections 2024-09-19T07:57:28.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.899+0000 7fc64e3f6640 1 --2- 172.21.15.16:0/1646437457 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fc63406c2e0 0x7fc63406e7a0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.899+0000 7fc64e3f6640 1 --2- 172.21.15.16:0/1646437457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6480d0930 0x7fc64815e590 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.899+0000 7fc64e3f6640 1 --2- 172.21.15.16:0/1646437457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc648073810 0x7fc64815e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.899+0000 7fc64e3f6640 1 --2- 172.21.15.16:0/1646437457 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc64806bb10 0x7fc648152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:28.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.899+0000 7fc64e3f6640 1 -- 172.21.15.16:0/1646437457 >> 172.21.15.16:0/1646437457 conn(0x7fc6480b2c90 msgr2=0x7fc648074f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:28.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.899+0000 7fc64e3f6640 1 -- 172.21.15.16:0/1646437457 shutdown_connections 2024-09-19T07:57:28.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:28.899+0000 7fc64e3f6640 1 -- 172.21.15.16:0/1646437457 wait complete. 2024-09-19T07:57:28.910 INFO:tasks.ceph:Scrubbing osd.5 2024-09-19T07:57:28.910 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-19T07:57:29.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.015+0000 7f2297b86640 1 Processor -- start 2024-09-19T07:57:29.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.015+0000 7f2297b86640 1 -- start start 2024-09-19T07:57:29.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.016+0000 7f2297b86640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f229009aa00 0x7f229009ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.016+0000 7f2297b86640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f22900b47f0 0x7f229009b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.016+0000 7f2297b86640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f229006d9a0 con 0x7f22900d0420 2024-09-19T07:57:29.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.016+0000 7f2297b86640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f22900ccc40 con 0x7f229009aa00 2024-09-19T07:57:29.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.016+0000 7f2297b86640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f229006dc80 con 0x7f22900b47f0 2024-09-19T07:57:29.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.016+0000 7f2295922640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f229009aa00 0x7f229009ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.016+0000 7f2296123640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f22900d0420 0x7f22900d0870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60938/0 (socket says 172.21.15.16:60938) 2024-09-19T07:57:29.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.016+0000 7f2296123640 1 -- 172.21.15.16:0/3518981616 learned_addr learned my addr 172.21.15.16:0/3518981616 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:29.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.017+0000 7f2295121640 1 --2- 172.21.15.16:0/3518981616 >> v2:172.21.15.146:3300/0 conn(0x7f22900b47f0 0x7f229009b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.017+0000 7f2295922640 1 -- 172.21.15.16:0/3518981616 >> v2:172.21.15.146:3300/0 conn(0x7f22900b47f0 msgr2=0x7f229009b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:29.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.017+0000 7f2295922640 1 --2- 172.21.15.16:0/3518981616 >> v2:172.21.15.146:3300/0 conn(0x7f22900b47f0 0x7f229009b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.017+0000 7f2295922640 1 -- 172.21.15.16:0/3518981616 >> v1:172.21.15.16:6789/0 conn(0x7f22900d0420 legacy=0x7f22900d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.017+0000 7f2294920640 1 -- 172.21.15.16:0/3518981616 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2492744039 0 0) 0x7f229006d9a0 con 0x7f22900d0420 2024-09-19T07:57:29.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.017+0000 7f2295922640 1 -- 172.21.15.16:0/3518981616 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22900cc8d0 con 0x7f229009aa00 2024-09-19T07:57:29.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.017+0000 7f2295121640 1 --2- 172.21.15.16:0/3518981616 >> v2:172.21.15.146:3300/0 conn(0x7f22900b47f0 0x7f229009b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:57:29.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.018+0000 7f2295922640 1 --2- 172.21.15.16:0/3518981616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f229009aa00 0x7f229009ade0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f22900a0200 tx=0x7f228c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988b2044c9bd0af6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.018+0000 7f2294920640 1 -- 172.21.15.16:0/3518981616 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c037070 con 0x7f229009aa00 2024-09-19T07:57:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.018+0000 7f2294920640 1 -- 172.21.15.16:0/3518981616 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f228c032440 con 0x7f229009aa00 2024-09-19T07:57:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.018+0000 7f2294920640 1 -- 172.21.15.16:0/3518981616 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c032810 con 0x7f229009aa00 2024-09-19T07:57:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.018+0000 7f2297b86640 1 -- 172.21.15.16:0/3518981616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f229009aa00 msgr2=0x7f229009ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.018+0000 7f2297b86640 1 --2- 172.21.15.16:0/3518981616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f229009aa00 0x7f229009ade0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f22900a0200 tx=0x7f228c02a1a0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.018+0000 7f2297b86640 1 -- 172.21.15.16:0/3518981616 shutdown_connections 2024-09-19T07:57:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.018+0000 7f2297b86640 1 --2- 172.21.15.16:0/3518981616 >> v2:172.21.15.146:3300/0 conn(0x7f22900b47f0 0x7f229009b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.018+0000 7f2297b86640 1 --2- 172.21.15.16:0/3518981616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f229009aa00 0x7f229009ade0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.018+0000 7f2297b86640 1 -- 172.21.15.16:0/3518981616 >> 172.21.15.16:0/3518981616 conn(0x7f22900b3cd0 msgr2=0x7f22900b40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.018+0000 7f2297b86640 1 -- 172.21.15.16:0/3518981616 shutdown_connections 2024-09-19T07:57:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.018+0000 7f2297b86640 1 -- 172.21.15.16:0/3518981616 wait complete. 2024-09-19T07:57:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.020+0000 7f2297b86640 1 Processor -- start 2024-09-19T07:57:29.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.020+0000 7f2297b86640 1 -- start start 2024-09-19T07:57:29.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.020+0000 7f2297b86640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f229009aa00 0x7f22901f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.020+0000 7f2297b86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22900b47f0 0x7f22900ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.020+0000 7f2297b86640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22900d0420 0x7f22901558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.020+0000 7f2297b86640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f229015e740 con 0x7f22900b47f0 2024-09-19T07:57:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.020+0000 7f2297b86640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f229006d630 con 0x7f22900d0420 2024-09-19T07:57:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.020+0000 7f2297b86640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f229015e8c0 con 0x7f229009aa00 2024-09-19T07:57:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.020+0000 7f2295121640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22900b47f0 0x7f22900ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.020+0000 7f2295121640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22900b47f0 0x7f22900ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42422/0 (socket says 172.21.15.16:42422) 2024-09-19T07:57:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.020+0000 7f2295121640 1 -- 172.21.15.16:0/3198167361 learned_addr learned my addr 172.21.15.16:0/3198167361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.021+0000 7f2295922640 1 --2- 172.21.15.16:0/3198167361 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f229009aa00 0x7f22901f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.021+0000 7f2296123640 1 --2- 172.21.15.16:0/3198167361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22900d0420 0x7f22901558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.021+0000 7f2295121640 1 -- 172.21.15.16:0/3198167361 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f229009aa00 msgr2=0x7f22901f49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.021+0000 7f2295121640 1 --2- 172.21.15.16:0/3198167361 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f229009aa00 0x7f22901f49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.021+0000 7f2295121640 1 -- 172.21.15.16:0/3198167361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22900d0420 msgr2=0x7f22901558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.021+0000 7f2295121640 1 --2- 172.21.15.16:0/3198167361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22900d0420 0x7f22901558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.021+0000 7f2295121640 1 -- 172.21.15.16:0/3198167361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2290155e10 con 0x7f22900b47f0 2024-09-19T07:57:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.021+0000 7f2295922640 1 --2- 172.21.15.16:0/3198167361 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f229009aa00 0x7f22901f49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:57:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.021+0000 7f2295121640 1 --2- 172.21.15.16:0/3198167361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22900b47f0 0x7f22900ca2c0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f228000c7f0 tx=0x7f228000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.021+0000 7f22867fc640 1 -- 172.21.15.16:0/3198167361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280015070 con 0x7f22900b47f0 2024-09-19T07:57:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.021+0000 7f22867fc640 1 -- 172.21.15.16:0/3198167361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2280007a30 con 0x7f22900b47f0 2024-09-19T07:57:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.021+0000 7f2297b86640 1 -- 172.21.15.16:0/3198167361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22900c9500 con 0x7f22900b47f0 2024-09-19T07:57:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.021+0000 7f22867fc640 1 -- 172.21.15.16:0/3198167361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280007e00 con 0x7f22900b47f0 2024-09-19T07:57:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.021+0000 7f2297b86640 1 -- 172.21.15.16:0/3198167361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22900c9ac0 con 0x7f22900b47f0 2024-09-19T07:57:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.022+0000 7f2297b86640 1 -- 172.21.15.16:0/3198167361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f22900c92b0 con 0x7f22900b47f0 2024-09-19T07:57:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.023+0000 7f22867fc640 1 -- 172.21.15.16:0/3198167361 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2280018070 con 0x7f22900b47f0 2024-09-19T07:57:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.023+0000 7f22867fc640 1 --2- 172.21.15.16:0/3198167361 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f227406c2b0 0x7f227406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.023+0000 7f22867fc640 1 -- 172.21.15.16:0/3198167361 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f228008b220 con 0x7f22900b47f0 2024-09-19T07:57:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.023+0000 7f22867fc640 1 --2- 172.21.15.16:0/3198167361 >> [v2:172.21.15.146:6816/2651212753,v1:172.21.15.146:6817/2651212753] conn(0x7f22740727a0 0x7f2274074be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.023+0000 7f22867fc640 1 -- 172.21.15.16:0/3198167361 --> [v2:172.21.15.146:6816/2651212753,v1:172.21.15.146:6817/2651212753] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f228001ce50 con 0x7f22740727a0 2024-09-19T07:57:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.023+0000 7f22867fc640 1 -- 172.21.15.16:0/3198167361 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=156) ==== 24+0+0 (secure 0 0 0) 0x7f2280092050 con 0x7f22900b47f0 2024-09-19T07:57:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.023+0000 7f2295922640 1 --2- 172.21.15.16:0/3198167361 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f227406c2b0 0x7f227406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.024+0000 7f2296123640 1 --2- 172.21.15.16:0/3198167361 >> [v2:172.21.15.146:6816/2651212753,v1:172.21.15.146:6817/2651212753] conn(0x7f22740727a0 0x7f2274074be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.024+0000 7f2295922640 1 --2- 172.21.15.16:0/3198167361 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f227406c2b0 0x7f227406e770 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f228c034000 tx=0x7f228c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.024+0000 7f2296123640 1 --2- 172.21.15.16:0/3198167361 >> [v2:172.21.15.146:6816/2651212753,v1:172.21.15.146:6817/2651212753] conn(0x7f22740727a0 0x7f2274074be0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.025+0000 7f22867fc640 1 -- 172.21.15.16:0/3198167361 <== osd.5 v2:172.21.15.146:6816/2651212753 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f228001ce50 con 0x7f22740727a0 2024-09-19T07:57:29.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.037+0000 7f2297b86640 1 -- 172.21.15.16:0/3198167361 --> [v2:172.21.15.146:6816/2651212753,v1:172.21.15.146:6817/2651212753] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2290068c40 con 0x7f22740727a0 2024-09-19T07:57:29.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.038+0000 7f22867fc640 1 -- 172.21.15.16:0/3198167361 <== osd.5 v2:172.21.15.146:6816/2651212753 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2290068c40 con 0x7f22740727a0 2024-09-19T07:57:29.038 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:57:29.038 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T07:57:29.038 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:57:29.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.038+0000 7f2297b86640 1 -- 172.21.15.16:0/3198167361 >> [v2:172.21.15.146:6816/2651212753,v1:172.21.15.146:6817/2651212753] conn(0x7f22740727a0 msgr2=0x7f2274074be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.038+0000 7f2297b86640 1 --2- 172.21.15.16:0/3198167361 >> [v2:172.21.15.146:6816/2651212753,v1:172.21.15.146:6817/2651212753] conn(0x7f22740727a0 0x7f2274074be0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.038+0000 7f2297b86640 1 -- 172.21.15.16:0/3198167361 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f227406c2b0 msgr2=0x7f227406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.038+0000 7f2297b86640 1 --2- 172.21.15.16:0/3198167361 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f227406c2b0 0x7f227406e770 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f228c034000 tx=0x7f228c02e000 comp rx=0 tx=0).stop 2024-09-19T07:57:29.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.038+0000 7f2297b86640 1 -- 172.21.15.16:0/3198167361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22900b47f0 msgr2=0x7f22900ca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.038+0000 7f2297b86640 1 --2- 172.21.15.16:0/3198167361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22900b47f0 0x7f22900ca2c0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f228000c7f0 tx=0x7f228000cbf0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.038+0000 7f2296123640 1 -- 172.21.15.16:0/3198167361 reap_dead start 2024-09-19T07:57:29.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.038+0000 7f2297b86640 1 -- 172.21.15.16:0/3198167361 shutdown_connections 2024-09-19T07:57:29.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.038+0000 7f2297b86640 1 -- 172.21.15.16:0/3198167361 >> 172.21.15.16:0/3198167361 conn(0x7f22900b3cd0 msgr2=0x7f22900cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:29.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.038+0000 7f2297b86640 1 -- 172.21.15.16:0/3198167361 shutdown_connections 2024-09-19T07:57:29.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.038+0000 7f2297b86640 1 -- 172.21.15.16:0/3198167361 wait complete. 2024-09-19T07:57:29.049 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-19T07:57:29.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.152+0000 7f5df0ab7640 1 Processor -- start 2024-09-19T07:57:29.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.152+0000 7f5df0ab7640 1 -- start start 2024-09-19T07:57:29.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.152+0000 7f5df0ab7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5dec0af460 0x7f5dec073370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.152+0000 7f5df0ab7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5dec15b700 0x7f5dec0d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.152+0000 7f5df0ab7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5dec06ed10 0x7f5dec06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.152+0000 7f5df0ab7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f5dec0738b0 con 0x7f5dec06ed10 2024-09-19T07:57:29.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.152+0000 7f5df0ab7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5dec0722d0 con 0x7f5dec0af460 2024-09-19T07:57:29.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.152+0000 7f5df0ab7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5dec0725b0 con 0x7f5dec15b700 2024-09-19T07:57:29.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.153+0000 7f5dead9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5dec06ed10 0x7f5dec06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.153+0000 7f5dead9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5dec06ed10 0x7f5dec06f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42428/0 (socket says 172.21.15.16:42428) 2024-09-19T07:57:29.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.153+0000 7f5dead9d640 1 -- 172.21.15.16:0/3432282068 learned_addr learned my addr 172.21.15.16:0/3432282068 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:29.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.153+0000 7f5de3fff640 1 --2- 172.21.15.16:0/3432282068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5dec15b700 0x7f5dec0d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.153+0000 7f5dea59c640 1 --2- 172.21.15.16:0/3432282068 >> v2:172.21.15.146:3300/0 conn(0x7f5dec0af460 0x7f5dec073370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.153+0000 7f5dead9d640 1 -- 172.21.15.16:0/3432282068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5dec15b700 msgr2=0x7f5dec0d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:29.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.153+0000 7f5dead9d640 1 --2- 172.21.15.16:0/3432282068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5dec15b700 0x7f5dec0d0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.153+0000 7f5dead9d640 1 -- 172.21.15.16:0/3432282068 >> v2:172.21.15.146:3300/0 conn(0x7f5dec0af460 msgr2=0x7f5dec073370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:29.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.153+0000 7f5dead9d640 1 --2- 172.21.15.16:0/3432282068 >> v2:172.21.15.146:3300/0 conn(0x7f5dec0af460 0x7f5dec073370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.153+0000 7f5dead9d640 1 -- 172.21.15.16:0/3432282068 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec072890 con 0x7f5dec06ed10 2024-09-19T07:57:29.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.154+0000 7f5dead9d640 1 --2- 172.21.15.16:0/3432282068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec06ed10 0x7f5dec06f1c0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0d1080 tx=0x7f5de402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf89edeaf2a15bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:29.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.154+0000 7f5de9d9b640 1 -- 172.21.15.16:0/3432282068 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4036070 con 0x7f5dec06ed10 2024-09-19T07:57:29.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.154+0000 7f5de9d9b640 1 -- 172.21.15.16:0/3432282068 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de4031600 con 0x7f5dec06ed10 2024-09-19T07:57:29.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.154+0000 7f5de9d9b640 1 -- 172.21.15.16:0/3432282068 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de40319d0 con 0x7f5dec06ed10 2024-09-19T07:57:29.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.154+0000 7f5df0ab7640 1 -- 172.21.15.16:0/3432282068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec06ed10 msgr2=0x7f5dec06f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.154+0000 7f5df0ab7640 1 --2- 172.21.15.16:0/3432282068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec06ed10 0x7f5dec06f1c0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0d1080 tx=0x7f5de402a6a0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.154+0000 7f5df0ab7640 1 -- 172.21.15.16:0/3432282068 shutdown_connections 2024-09-19T07:57:29.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.154+0000 7f5df0ab7640 1 --2- 172.21.15.16:0/3432282068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec06ed10 0x7f5dec06f1c0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.154+0000 7f5df0ab7640 1 --2- 172.21.15.16:0/3432282068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5dec15b700 0x7f5dec0d0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.154+0000 7f5df0ab7640 1 --2- 172.21.15.16:0/3432282068 >> v2:172.21.15.146:3300/0 conn(0x7f5dec0af460 0x7f5dec073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.154+0000 7f5df0ab7640 1 -- 172.21.15.16:0/3432282068 >> 172.21.15.16:0/3432282068 conn(0x7f5dec06bcf0 msgr2=0x7f5dec06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:29.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.154+0000 7f5df0ab7640 1 -- 172.21.15.16:0/3432282068 shutdown_connections 2024-09-19T07:57:29.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.154+0000 7f5df0ab7640 1 -- 172.21.15.16:0/3432282068 wait complete. 2024-09-19T07:57:29.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.156+0000 7f5df0ab7640 1 Processor -- start 2024-09-19T07:57:29.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.156+0000 7f5df0ab7640 1 -- start start 2024-09-19T07:57:29.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.156+0000 7f5df0ab7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec06ed10 0x7f5dec0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.156+0000 7f5df0ab7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5dec0af460 0x7f5dec15a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.156+0000 7f5df0ab7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5dec15b700 0x7f5dec15a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.156+0000 7f5df0ab7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec0af840 con 0x7f5dec06ed10 2024-09-19T07:57:29.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.156+0000 7f5df0ab7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec072e50 con 0x7f5dec15b700 2024-09-19T07:57:29.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.156+0000 7f5df0ab7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5dec0d0c50 con 0x7f5dec0af460 2024-09-19T07:57:29.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.157+0000 7f5de3fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5dec0af460 0x7f5dec15a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.157+0000 7f5dea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec06ed10 0x7f5dec0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.157+0000 7f5dea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec06ed10 0x7f5dec0cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42438/0 (socket says 172.21.15.16:42438) 2024-09-19T07:57:29.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.157+0000 7f5dea59c640 1 -- 172.21.15.16:0/1644306308 learned_addr learned my addr 172.21.15.16:0/1644306308 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:29.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.157+0000 7f5dead9d640 1 --2- 172.21.15.16:0/1644306308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5dec15b700 0x7f5dec15a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.157+0000 7f5dea59c640 1 -- 172.21.15.16:0/1644306308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5dec0af460 msgr2=0x7f5dec15a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.157+0000 7f5dea59c640 1 --2- 172.21.15.16:0/1644306308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5dec0af460 0x7f5dec15a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.157+0000 7f5dea59c640 1 -- 172.21.15.16:0/1644306308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5dec15b700 msgr2=0x7f5dec15a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.157+0000 7f5dea59c640 1 --2- 172.21.15.16:0/1644306308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5dec15b700 0x7f5dec15a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.157+0000 7f5dea59c640 1 -- 172.21.15.16:0/1644306308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec0b1430 con 0x7f5dec06ed10 2024-09-19T07:57:29.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.157+0000 7f5dea59c640 1 --2- 172.21.15.16:0/1644306308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec06ed10 0x7f5dec0cdc00 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00d830 tx=0x7f5ddc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:29.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.157+0000 7f5de2ffd640 1 -- 172.21.15.16:0/1644306308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc018020 con 0x7f5dec06ed10 2024-09-19T07:57:29.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.157+0000 7f5de2ffd640 1 -- 172.21.15.16:0/1644306308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ddc0096a0 con 0x7f5dec06ed10 2024-09-19T07:57:29.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.157+0000 7f5de2ffd640 1 -- 172.21.15.16:0/1644306308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc009a70 con 0x7f5dec06ed10 2024-09-19T07:57:29.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.158+0000 7f5df0ab7640 1 -- 172.21.15.16:0/1644306308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dec0b1710 con 0x7f5dec06ed10 2024-09-19T07:57:29.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.158+0000 7f5df0ab7640 1 -- 172.21.15.16:0/1644306308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dec0b1cd0 con 0x7f5dec06ed10 2024-09-19T07:57:29.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.159+0000 7f5df0ab7640 1 -- 172.21.15.16:0/1644306308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dec073370 con 0x7f5dec06ed10 2024-09-19T07:57:29.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.159+0000 7f5de2ffd640 1 -- 172.21.15.16:0/1644306308 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5ddc01b020 con 0x7f5dec06ed10 2024-09-19T07:57:29.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.160+0000 7f5de2ffd640 1 --2- 172.21.15.16:0/1644306308 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5dcc06c2b0 0x7f5dcc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.160+0000 7f5de2ffd640 1 -- 172.21.15.16:0/1644306308 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f5ddc08b490 con 0x7f5dec06ed10 2024-09-19T07:57:29.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.160+0000 7f5de3fff640 1 --2- 172.21.15.16:0/1644306308 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5dcc06c2b0 0x7f5dcc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.161+0000 7f5de3fff640 1 --2- 172.21.15.16:0/1644306308 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5dcc06c2b0 0x7f5dcc06e770 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5dec158c20 tx=0x7f5dd4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:29.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.164+0000 7f5de2ffd640 1 -- 172.21.15.16:0/1644306308 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5ddc022030 con 0x7f5dec06ed10 2024-09-19T07:57:29.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.339+0000 7f5df0ab7640 1 -- 172.21.15.16:0/1644306308 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f5dec14dcb0 con 0x7f5dcc06c2b0 2024-09-19T07:57:29.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.339+0000 7f5de2ffd640 1 -- 172.21.15.16:0/1644306308 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5dec14dcb0 con 0x7f5dcc06c2b0 2024-09-19T07:57:29.340 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 5 to deep-scrub 2024-09-19T07:57:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.342+0000 7f5df0ab7640 1 -- 172.21.15.16:0/1644306308 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5dcc06c2b0 msgr2=0x7f5dcc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.342+0000 7f5df0ab7640 1 --2- 172.21.15.16:0/1644306308 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5dcc06c2b0 0x7f5dcc06e770 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5dec158c20 tx=0x7f5dd4008040 comp rx=0 tx=0).stop 2024-09-19T07:57:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.342+0000 7f5df0ab7640 1 -- 172.21.15.16:0/1644306308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec06ed10 msgr2=0x7f5dec0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.342+0000 7f5df0ab7640 1 --2- 172.21.15.16:0/1644306308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec06ed10 0x7f5dec0cdc00 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00d830 tx=0x7f5ddc00dcf0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.342+0000 7f5df0ab7640 1 -- 172.21.15.16:0/1644306308 shutdown_connections 2024-09-19T07:57:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.342+0000 7f5df0ab7640 1 --2- 172.21.15.16:0/1644306308 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f5dcc06c2b0 0x7f5dcc06e770 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.342+0000 7f5df0ab7640 1 --2- 172.21.15.16:0/1644306308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5dec15b700 0x7f5dec15a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.342+0000 7f5df0ab7640 1 --2- 172.21.15.16:0/1644306308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5dec0af460 0x7f5dec15a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.342+0000 7f5df0ab7640 1 --2- 172.21.15.16:0/1644306308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec06ed10 0x7f5dec0cdc00 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.342+0000 7f5df0ab7640 1 -- 172.21.15.16:0/1644306308 >> 172.21.15.16:0/1644306308 conn(0x7f5dec06bcf0 msgr2=0x7f5dec15e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.342+0000 7f5df0ab7640 1 -- 172.21.15.16:0/1644306308 shutdown_connections 2024-09-19T07:57:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.342+0000 7f5df0ab7640 1 -- 172.21.15.16:0/1644306308 wait complete. 2024-09-19T07:57:29.353 INFO:tasks.ceph:Scrubbing osd.6 2024-09-19T07:57:29.353 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-19T07:57:29.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.459+0000 7f69e5cba640 1 Processor -- start 2024-09-19T07:57:29.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.459+0000 7f69e5cba640 1 -- start start 2024-09-19T07:57:29.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.460+0000 7f69e5cba640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f69e015c330 0x7f69e00adf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.460+0000 7f69e5cba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69e0072d20 0x7f69e00ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.460+0000 7f69e5cba640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69e0073220 con 0x7f69e015bf50 2024-09-19T07:57:29.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.460+0000 7f69e5cba640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f69e006ed50 con 0x7f69e015c330 2024-09-19T07:57:29.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.460+0000 7f69e5cba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f69e0073500 con 0x7f69e0072d20 2024-09-19T07:57:29.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.460+0000 7f69deffd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f69e015c330 0x7f69e00adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.460+0000 7f69dffff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69e0072d20 0x7f69e00ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.460+0000 7f69deffd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f69e015c330 0x7f69e00adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.16:36934/0 (socket says 172.21.15.16:36934) 2024-09-19T07:57:29.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.460+0000 7f69deffd640 1 -- 172.21.15.16:0/3125880801 learned_addr learned my addr 172.21.15.16:0/3125880801 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:29.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.461+0000 7f69dffff640 1 -- 172.21.15.16:0/3125880801 >> v2:172.21.15.146:3300/0 conn(0x7f69e015c330 msgr2=0x7f69e00adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:29.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.461+0000 7f69dffff640 1 --2- 172.21.15.16:0/3125880801 >> v2:172.21.15.146:3300/0 conn(0x7f69e015c330 0x7f69e00adf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.461+0000 7f69dffff640 1 -- 172.21.15.16:0/3125880801 >> v1:172.21.15.146:6789/0 conn(0x7f69e015bf50 legacy=0x7f69e00b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.461+0000 7f69dffff640 1 -- 172.21.15.16:0/3125880801 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e00737e0 con 0x7f69e0072d20 2024-09-19T07:57:29.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.461+0000 7f69deffd640 1 --2- 172.21.15.16:0/3125880801 >> v2:172.21.15.146:3300/0 conn(0x7f69e015c330 0x7f69e00adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:57:29.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.462+0000 7f69dffff640 1 --2- 172.21.15.16:0/3125880801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69e0072d20 0x7f69e00ae4b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009d90 tx=0x7f69d802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=388f95bdd322403d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:29.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.462+0000 7f69de7fc640 1 -- 172.21.15.16:0/3125880801 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d8007740 con 0x7f69e0072d20 2024-09-19T07:57:29.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.462+0000 7f69de7fc640 1 -- 172.21.15.16:0/3125880801 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d80078e0 con 0x7f69e0072d20 2024-09-19T07:57:29.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.462+0000 7f69de7fc640 1 -- 172.21.15.16:0/3125880801 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d8007cb0 con 0x7f69e0072d20 2024-09-19T07:57:29.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.462+0000 7f69e5cba640 1 -- 172.21.15.16:0/3125880801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69e0072d20 msgr2=0x7f69e00ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.462+0000 7f69e5cba640 1 --2- 172.21.15.16:0/3125880801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69e0072d20 0x7f69e00ae4b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009d90 tx=0x7f69d802a660 comp rx=0 tx=0).stop 2024-09-19T07:57:29.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.462+0000 7f69e5cba640 1 -- 172.21.15.16:0/3125880801 shutdown_connections 2024-09-19T07:57:29.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.462+0000 7f69e5cba640 1 --2- 172.21.15.16:0/3125880801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69e0072d20 0x7f69e00ae4b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.462+0000 7f69e5cba640 1 --2- 172.21.15.16:0/3125880801 >> v2:172.21.15.146:3300/0 conn(0x7f69e015c330 0x7f69e00adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.462+0000 7f69e5cba640 1 -- 172.21.15.16:0/3125880801 >> 172.21.15.16:0/3125880801 conn(0x7f69e00b2c90 msgr2=0x7f69e00b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:29.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.462+0000 7f69e5cba640 1 -- 172.21.15.16:0/3125880801 shutdown_connections 2024-09-19T07:57:29.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.462+0000 7f69e5cba640 1 -- 172.21.15.16:0/3125880801 wait complete. 2024-09-19T07:57:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.463+0000 7f69e5cba640 1 Processor -- start 2024-09-19T07:57:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.464+0000 7f69e5cba640 1 -- start start 2024-09-19T07:57:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.464+0000 7f69e5cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0072d20 0x7f69e01f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.464+0000 7f69e5cba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69e015bf50 0x7f69e0158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.464+0000 7f69e5cba640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e015c330 0x7f69e00b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.464+0000 7f69e5cba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69e006f360 con 0x7f69e0072d20 2024-09-19T07:57:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.464+0000 7f69e5cba640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f69e006f1e0 con 0x7f69e015c330 2024-09-19T07:57:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.464+0000 7f69e5cba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f69e006f4e0 con 0x7f69e015bf50 2024-09-19T07:57:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.464+0000 7f69df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0072d20 0x7f69e01f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.465+0000 7f69df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0072d20 0x7f69e01f3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42450/0 (socket says 172.21.15.16:42450) 2024-09-19T07:57:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.465+0000 7f69df7fe640 1 -- 172.21.15.16:0/3636094376 learned_addr learned my addr 172.21.15.16:0/3636094376 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.465+0000 7f69dffff640 1 --2- 172.21.15.16:0/3636094376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e015c330 0x7f69e00b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.465+0000 7f69deffd640 1 --2- 172.21.15.16:0/3636094376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69e015bf50 0x7f69e0158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.465+0000 7f69df7fe640 1 -- 172.21.15.16:0/3636094376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69e015bf50 msgr2=0x7f69e0158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.465+0000 7f69df7fe640 1 --2- 172.21.15.16:0/3636094376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69e015bf50 0x7f69e0158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.465+0000 7f69df7fe640 1 -- 172.21.15.16:0/3636094376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e015c330 msgr2=0x7f69e00b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.465+0000 7f69df7fe640 1 --2- 172.21.15.16:0/3636094376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e015c330 0x7f69e00b57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.465+0000 7f69df7fe640 1 -- 172.21.15.16:0/3636094376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e0153bd0 con 0x7f69e0072d20 2024-09-19T07:57:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.465+0000 7f69df7fe640 1 --2- 172.21.15.16:0/3636094376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0072d20 0x7f69e01f3fe0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f69d000c430 tx=0x7f69d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.465+0000 7f69c7fff640 1 -- 172.21.15.16:0/3636094376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0017070 con 0x7f69e0072d20 2024-09-19T07:57:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.465+0000 7f69c7fff640 1 -- 172.21.15.16:0/3636094376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d00052d0 con 0x7f69e0072d20 2024-09-19T07:57:29.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.465+0000 7f69e5cba640 1 -- 172.21.15.16:0/3636094376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e0153eb0 con 0x7f69e0072d20 2024-09-19T07:57:29.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.465+0000 7f69c7fff640 1 -- 172.21.15.16:0/3636094376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0003850 con 0x7f69e0072d20 2024-09-19T07:57:29.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.465+0000 7f69e5cba640 1 -- 172.21.15.16:0/3636094376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e0154470 con 0x7f69e0072d20 2024-09-19T07:57:29.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.466+0000 7f69e5cba640 1 -- 172.21.15.16:0/3636094376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f69e0153980 con 0x7f69e0072d20 2024-09-19T07:57:29.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.467+0000 7f69c7fff640 1 -- 172.21.15.16:0/3636094376 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f69d001a070 con 0x7f69e0072d20 2024-09-19T07:57:29.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.467+0000 7f69c7fff640 1 --2- 172.21.15.16:0/3636094376 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f69cc06c2b0 0x7f69cc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.467+0000 7f69c7fff640 1 -- 172.21.15.16:0/3636094376 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f69d008a560 con 0x7f69e0072d20 2024-09-19T07:57:29.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.467+0000 7f69c7fff640 1 --2- 172.21.15.16:0/3636094376 >> [v2:172.21.15.146:6800/2692587126,v1:172.21.15.146:6801/2692587126] conn(0x7f69cc0727a0 0x7f69cc074be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.467+0000 7f69c7fff640 1 -- 172.21.15.16:0/3636094376 --> [v2:172.21.15.146:6800/2692587126,v1:172.21.15.146:6801/2692587126] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f69cc0752b0 con 0x7f69cc0727a0 2024-09-19T07:57:29.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.467+0000 7f69deffd640 1 --2- 172.21.15.16:0/3636094376 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f69cc06c2b0 0x7f69cc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.467+0000 7f69c7fff640 1 -- 172.21.15.16:0/3636094376 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=156) ==== 24+0+0 (secure 0 0 0) 0x7f69d0091050 con 0x7f69e0072d20 2024-09-19T07:57:29.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.468+0000 7f69dffff640 1 --2- 172.21.15.16:0/3636094376 >> [v2:172.21.15.146:6800/2692587126,v1:172.21.15.146:6801/2692587126] conn(0x7f69cc0727a0 0x7f69cc074be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.468+0000 7f69deffd640 1 --2- 172.21.15.16:0/3636094376 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f69cc06c2b0 0x7f69cc06e770 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f69e00b4d40 tx=0x7f69d4009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:29.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.468+0000 7f69dffff640 1 --2- 172.21.15.16:0/3636094376 >> [v2:172.21.15.146:6800/2692587126,v1:172.21.15.146:6801/2692587126] conn(0x7f69cc0727a0 0x7f69cc074be0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:29.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.469+0000 7f69c7fff640 1 -- 172.21.15.16:0/3636094376 <== osd.6 v2:172.21.15.146:6800/2692587126 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f69cc0752b0 con 0x7f69cc0727a0 2024-09-19T07:57:29.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.481+0000 7f69e5cba640 1 -- 172.21.15.16:0/3636094376 --> [v2:172.21.15.146:6800/2692587126,v1:172.21.15.146:6801/2692587126] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f69e0075730 con 0x7f69cc0727a0 2024-09-19T07:57:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.482+0000 7f69c7fff640 1 -- 172.21.15.16:0/3636094376 <== osd.6 v2:172.21.15.146:6800/2692587126 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f69e0075730 con 0x7f69cc0727a0 2024-09-19T07:57:29.482 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:57:29.482 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T07:57:29.482 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:57:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.482+0000 7f69e5cba640 1 -- 172.21.15.16:0/3636094376 >> [v2:172.21.15.146:6800/2692587126,v1:172.21.15.146:6801/2692587126] conn(0x7f69cc0727a0 msgr2=0x7f69cc074be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.482+0000 7f69e5cba640 1 --2- 172.21.15.16:0/3636094376 >> [v2:172.21.15.146:6800/2692587126,v1:172.21.15.146:6801/2692587126] conn(0x7f69cc0727a0 0x7f69cc074be0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.482+0000 7f69e5cba640 1 -- 172.21.15.16:0/3636094376 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f69cc06c2b0 msgr2=0x7f69cc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.482+0000 7f69e5cba640 1 --2- 172.21.15.16:0/3636094376 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f69cc06c2b0 0x7f69cc06e770 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f69e00b4d40 tx=0x7f69d4009240 comp rx=0 tx=0).stop 2024-09-19T07:57:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.482+0000 7f69e5cba640 1 -- 172.21.15.16:0/3636094376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0072d20 msgr2=0x7f69e01f3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.482+0000 7f69e5cba640 1 --2- 172.21.15.16:0/3636094376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69e0072d20 0x7f69e01f3fe0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f69d000c430 tx=0x7f69d000c8f0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.482+0000 7f69dffff640 1 -- 172.21.15.16:0/3636094376 reap_dead start 2024-09-19T07:57:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.482+0000 7f69e5cba640 1 -- 172.21.15.16:0/3636094376 shutdown_connections 2024-09-19T07:57:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.482+0000 7f69e5cba640 1 -- 172.21.15.16:0/3636094376 >> 172.21.15.16:0/3636094376 conn(0x7f69e00b2c90 msgr2=0x7f69e015eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.482+0000 7f69e5cba640 1 -- 172.21.15.16:0/3636094376 shutdown_connections 2024-09-19T07:57:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.482+0000 7f69e5cba640 1 -- 172.21.15.16:0/3636094376 wait complete. 2024-09-19T07:57:29.493 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-19T07:57:29.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.596+0000 7f807085e640 1 Processor -- start 2024-09-19T07:57:29.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.596+0000 7f807085e640 1 -- start start 2024-09-19T07:57:29.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f807085e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806c0d0630 0x7f806c0b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f807085e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f806c0af5f0 con 0x7f806c0b4730 2024-09-19T07:57:29.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f807085e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f806c0af8d0 con 0x7f806c159eb0 2024-09-19T07:57:29.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f807085e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f806c0b4b90 con 0x7f806c0d0630 2024-09-19T07:57:29.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f8061d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f806c0b4730 0x7f806c0cff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60946/0 (socket says 172.21.15.16:60946) 2024-09-19T07:57:29.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f8061d9b640 1 -- 172.21.15.16:0/4126054645 learned_addr learned my addr 172.21.15.16:0/4126054645 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:29.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f806ad9d640 1 --2- 172.21.15.16:0/4126054645 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806c0d0630 0x7f806c0b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f8069d9b640 1 -- 172.21.15.16:0/4126054645 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3150666678 0 0) 0x7f806c0af5f0 con 0x7f806c0b4730 2024-09-19T07:57:29.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f8069d9b640 1 -- 172.21.15.16:0/4126054645 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8058003520 con 0x7f806c0b4730 2024-09-19T07:57:29.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f8069d9b640 1 -- 172.21.15.16:0/4126054645 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1102847864 0 0) 0x7f8058003520 con 0x7f806c0b4730 2024-09-19T07:57:29.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f8069d9b640 1 -- 172.21.15.16:0/4126054645 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806c0d0630 msgr2=0x7f806c0b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:29.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f8069d9b640 1 --2- 172.21.15.16:0/4126054645 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806c0d0630 0x7f806c0b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f8069d9b640 1 -- 172.21.15.16:0/4126054645 >> v1:172.21.15.146:6789/0 conn(0x7f806c159eb0 legacy=0x7f806c0aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f8069d9b640 1 -- 172.21.15.16:0/4126054645 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806c0b3220 con 0x7f806c0b4730 2024-09-19T07:57:29.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f8069d9b640 1 -- 172.21.15.16:0/4126054645 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f8054001670 con 0x7f806c0b4730 2024-09-19T07:57:29.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f8069d9b640 1 -- 172.21.15.16:0/4126054645 >> v1:172.21.15.16:6789/0 conn(0x7f806c0b4730 legacy=0x7f806c0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f8069d9b640 1 --2- 172.21.15.16:0/4126054645 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806c0d0630 0x7f80580039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f8069d9b640 1 --2- 172.21.15.16:0/4126054645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8058003f60 0x7f8058004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f8069d9b640 1 --2- 172.21.15.16:0/4126054645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8058004d40 0x7f80580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f8069d9b640 1 -- 172.21.15.16:0/4126054645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f806c0b4b90 con 0x7f8058003f60 2024-09-19T07:57:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f8069d9b640 1 -- 172.21.15.16:0/4126054645 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f805800baa0 con 0x7f8058004d40 2024-09-19T07:57:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f8069d9b640 1 -- 172.21.15.16:0/4126054645 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f805800bca0 con 0x7f806c0d0630 2024-09-19T07:57:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.597+0000 7f8061d9b640 1 --2- 172.21.15.16:0/4126054645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8058003f60 0x7f8058004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.598+0000 7f806a59c640 1 --2- 172.21.15.16:0/4126054645 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806c0d0630 0x7f80580039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.598+0000 7f806ad9d640 1 --2- 172.21.15.16:0/4126054645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8058004d40 0x7f80580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.598+0000 7f8061d9b640 1 -- 172.21.15.16:0/4126054645 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806c0d0630 msgr2=0x7f80580039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.598+0000 7f8061d9b640 1 --2- 172.21.15.16:0/4126054645 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806c0d0630 0x7f80580039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.598+0000 7f8061d9b640 1 -- 172.21.15.16:0/4126054645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8058004d40 msgr2=0x7f80580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.598+0000 7f8061d9b640 1 --2- 172.21.15.16:0/4126054645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8058004d40 0x7f80580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.598+0000 7f8061d9b640 1 -- 172.21.15.16:0/4126054645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8058003520 con 0x7f8058003f60 2024-09-19T07:57:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.598+0000 7f8061d9b640 1 --2- 172.21.15.16:0/4126054645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8058003f60 0x7f8058004800 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f805400c240 tx=0x7f8058005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7136fa5ddfc73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.599+0000 7f8069d9b640 1 -- 172.21.15.16:0/4126054645 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054007cf0 con 0x7f8058003f60 2024-09-19T07:57:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.599+0000 7f8069d9b640 1 -- 172.21.15.16:0/4126054645 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80540058d0 con 0x7f8058003f60 2024-09-19T07:57:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.599+0000 7f8069d9b640 1 -- 172.21.15.16:0/4126054645 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054005d00 con 0x7f8058003f60 2024-09-19T07:57:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.599+0000 7f807085e640 1 -- 172.21.15.16:0/4126054645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8058003f60 msgr2=0x7f8058004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.599+0000 7f807085e640 1 --2- 172.21.15.16:0/4126054645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8058003f60 0x7f8058004800 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f805400c240 tx=0x7f8058005830 comp rx=0 tx=0).stop 2024-09-19T07:57:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.599+0000 7f806ad9d640 1 -- 172.21.15.16:0/4126054645 reap_dead start 2024-09-19T07:57:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.599+0000 7f807085e640 1 -- 172.21.15.16:0/4126054645 shutdown_connections 2024-09-19T07:57:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.599+0000 7f807085e640 1 -- 172.21.15.16:0/4126054645 >> 172.21.15.16:0/4126054645 conn(0x7f806c06bcf0 msgr2=0x7f806c06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.599+0000 7f807085e640 1 -- 172.21.15.16:0/4126054645 shutdown_connections 2024-09-19T07:57:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.599+0000 7f807085e640 1 -- 172.21.15.16:0/4126054645 wait complete. 2024-09-19T07:57:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.601+0000 7f807085e640 1 Processor -- start 2024-09-19T07:57:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.601+0000 7f807085e640 1 -- start start 2024-09-19T07:57:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.601+0000 7f807085e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8058003f60 0x7f806c1539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.601+0000 7f807085e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f806c15f2f0 0x7f806c153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.601+0000 7f807085e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806c072cb0 0x7f806c073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.601+0000 7f807085e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f806c0b3ba0 con 0x7f806c15f2f0 2024-09-19T07:57:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.601+0000 7f807085e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f806c0b3a20 con 0x7f8058003f60 2024-09-19T07:57:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.601+0000 7f807085e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f806c0b3d20 con 0x7f806c072cb0 2024-09-19T07:57:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.602+0000 7f8061d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f806c15f2f0 0x7f806c153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.602+0000 7f8061d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f806c15f2f0 0x7f806c153f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42466/0 (socket says 172.21.15.16:42466) 2024-09-19T07:57:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.602+0000 7f8061d9b640 1 -- 172.21.15.16:0/349443858 learned_addr learned my addr 172.21.15.16:0/349443858 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.602+0000 7f806ad9d640 1 --2- 172.21.15.16:0/349443858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806c072cb0 0x7f806c073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.602+0000 7f806a59c640 1 --2- 172.21.15.16:0/349443858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8058003f60 0x7f806c1539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.602+0000 7f8061d9b640 1 -- 172.21.15.16:0/349443858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806c072cb0 msgr2=0x7f806c073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.602+0000 7f8061d9b640 1 --2- 172.21.15.16:0/349443858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806c072cb0 0x7f806c073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.602+0000 7f8061d9b640 1 -- 172.21.15.16:0/349443858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8058003f60 msgr2=0x7f806c1539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.602+0000 7f8061d9b640 1 --2- 172.21.15.16:0/349443858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8058003f60 0x7f806c1539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.602+0000 7f8061d9b640 1 -- 172.21.15.16:0/349443858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806c155f90 con 0x7f806c15f2f0 2024-09-19T07:57:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.602+0000 7f8061d9b640 1 --2- 172.21.15.16:0/349443858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f806c15f2f0 0x7f806c153f40 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f805400c360 tx=0x7f806c0b33f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.602+0000 7f80637fe640 1 -- 172.21.15.16:0/349443858 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054038040 con 0x7f806c15f2f0 2024-09-19T07:57:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.602+0000 7f80637fe640 1 -- 172.21.15.16:0/349443858 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8054042070 con 0x7f806c15f2f0 2024-09-19T07:57:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.602+0000 7f807085e640 1 -- 172.21.15.16:0/349443858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f806c156270 con 0x7f806c15f2f0 2024-09-19T07:57:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.602+0000 7f80637fe640 1 -- 172.21.15.16:0/349443858 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054001c30 con 0x7f806c15f2f0 2024-09-19T07:57:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.602+0000 7f807085e640 1 -- 172.21.15.16:0/349443858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f806c156830 con 0x7f806c15f2f0 2024-09-19T07:57:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.604+0000 7f807085e640 1 -- 172.21.15.16:0/349443858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f806c15d190 con 0x7f806c15f2f0 2024-09-19T07:57:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.604+0000 7f80637fe640 1 -- 172.21.15.16:0/349443858 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8054007290 con 0x7f806c15f2f0 2024-09-19T07:57:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.604+0000 7f80637fe640 1 --2- 172.21.15.16:0/349443858 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f804c06c2e0 0x7f804c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.604+0000 7f80637fe640 1 -- 172.21.15.16:0/349443858 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f80540ab2f0 con 0x7f806c15f2f0 2024-09-19T07:57:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.605+0000 7f806a59c640 1 --2- 172.21.15.16:0/349443858 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f804c06c2e0 0x7f804c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.605+0000 7f806a59c640 1 --2- 172.21.15.16:0/349443858 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f804c06c2e0 0x7f804c06e7a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f806c09e2c0 tx=0x7f805c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.609+0000 7f80637fe640 1 -- 172.21.15.16:0/349443858 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8054079670 con 0x7f806c15f2f0 2024-09-19T07:57:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.783+0000 7f807085e640 1 -- 172.21.15.16:0/349443858 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f806c0008d0 con 0x7f804c06c2e0 2024-09-19T07:57:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.784+0000 7f80637fe640 1 -- 172.21.15.16:0/349443858 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f806c0008d0 con 0x7f804c06c2e0 2024-09-19T07:57:29.784 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 6 to deep-scrub 2024-09-19T07:57:29.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.786+0000 7f807085e640 1 -- 172.21.15.16:0/349443858 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f804c06c2e0 msgr2=0x7f804c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.786+0000 7f807085e640 1 --2- 172.21.15.16:0/349443858 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f804c06c2e0 0x7f804c06e7a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f806c09e2c0 tx=0x7f805c009040 comp rx=0 tx=0).stop 2024-09-19T07:57:29.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.786+0000 7f807085e640 1 -- 172.21.15.16:0/349443858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f806c15f2f0 msgr2=0x7f806c153f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.786+0000 7f807085e640 1 --2- 172.21.15.16:0/349443858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f806c15f2f0 0x7f806c153f40 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f805400c360 tx=0x7f806c0b33f0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.786+0000 7f807085e640 1 -- 172.21.15.16:0/349443858 shutdown_connections 2024-09-19T07:57:29.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.786+0000 7f807085e640 1 --2- 172.21.15.16:0/349443858 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f804c06c2e0 0x7f804c06e7a0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.786+0000 7f807085e640 1 --2- 172.21.15.16:0/349443858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806c072cb0 0x7f806c073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.786+0000 7f807085e640 1 --2- 172.21.15.16:0/349443858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f806c15f2f0 0x7f806c153f40 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.786+0000 7f807085e640 1 --2- 172.21.15.16:0/349443858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8058003f60 0x7f806c1539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.786+0000 7f807085e640 1 -- 172.21.15.16:0/349443858 >> 172.21.15.16:0/349443858 conn(0x7f806c06bcf0 msgr2=0x7f806c15ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:29.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.786+0000 7f807085e640 1 -- 172.21.15.16:0/349443858 shutdown_connections 2024-09-19T07:57:29.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.786+0000 7f807085e640 1 -- 172.21.15.16:0/349443858 wait complete. 2024-09-19T07:57:29.797 INFO:tasks.ceph:Scrubbing osd.7 2024-09-19T07:57:29.797 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-19T07:57:29.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.903+0000 7f8365b9f640 1 Processor -- start 2024-09-19T07:57:29.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.903+0000 7f8365b9f640 1 -- start start 2024-09-19T07:57:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.903+0000 7f8365b9f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f836015e790 0x7f83600d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.903+0000 7f8365b9f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83600cf0a0 con 0x7f83600d1510 2024-09-19T07:57:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.903+0000 7f8365b9f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83600cf380 con 0x7f83600d5040 2024-09-19T07:57:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.903+0000 7f8365b9f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f83600cf660 con 0x7f836015e790 2024-09-19T07:57:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f83600d1510 0x7f836006cb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60948/0 (socket says 172.21.15.16:60948) 2024-09-19T07:57:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835ffff640 1 -- 172.21.15.16:0/2292373622 learned_addr learned my addr 172.21.15.16:0/2292373622 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835f7fe640 1 --2- 172.21.15.16:0/2292373622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f836015e790 0x7f83600d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835e7fc640 1 -- 172.21.15.16:0/2292373622 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 698377449 0 0) 0x7f83600cf0a0 con 0x7f83600d1510 2024-09-19T07:57:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835e7fc640 1 -- 172.21.15.16:0/2292373622 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8348003520 con 0x7f83600d1510 2024-09-19T07:57:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835e7fc640 1 -- 172.21.15.16:0/2292373622 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1902604526 0 0) 0x7f8348003520 con 0x7f83600d1510 2024-09-19T07:57:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835e7fc640 1 -- 172.21.15.16:0/2292373622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f836015e790 msgr2=0x7f83600d4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835e7fc640 1 --2- 172.21.15.16:0/2292373622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f836015e790 0x7f83600d4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835e7fc640 1 -- 172.21.15.16:0/2292373622 >> v1:172.21.15.146:6789/0 conn(0x7f83600d5040 legacy=0x7f83600d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835e7fc640 1 -- 172.21.15.16:0/2292373622 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83600cf940 con 0x7f83600d1510 2024-09-19T07:57:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835e7fc640 1 -- 172.21.15.16:0/2292373622 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f83580019f0 con 0x7f83600d1510 2024-09-19T07:57:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835e7fc640 1 -- 172.21.15.16:0/2292373622 >> v1:172.21.15.16:6789/0 conn(0x7f83600d1510 legacy=0x7f836006cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835e7fc640 1 --2- 172.21.15.16:0/2292373622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f836015e790 0x7f83480039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835e7fc640 1 --2- 172.21.15.16:0/2292373622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348003f20 0x7f8348004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835e7fc640 1 --2- 172.21.15.16:0/2292373622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8348004d40 0x7f83480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835e7fc640 1 -- 172.21.15.16:0/2292373622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83600cf660 con 0x7f8348003f20 2024-09-19T07:57:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835e7fc640 1 -- 172.21.15.16:0/2292373622 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f834800ba20 con 0x7f8348004d40 2024-09-19T07:57:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835e7fc640 1 -- 172.21.15.16:0/2292373622 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f834800bc20 con 0x7f836015e790 2024-09-19T07:57:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835effd640 1 --2- 172.21.15.16:0/2292373622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348003f20 0x7f8348004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.904+0000 7f835f7fe640 1 --2- 172.21.15.16:0/2292373622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f836015e790 0x7f83480039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.905+0000 7f835ffff640 1 --2- 172.21.15.16:0/2292373622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8348004d40 0x7f83480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.905+0000 7f835effd640 1 -- 172.21.15.16:0/2292373622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f836015e790 msgr2=0x7f83480039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.905+0000 7f835effd640 1 --2- 172.21.15.16:0/2292373622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f836015e790 0x7f83480039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.905+0000 7f835effd640 1 -- 172.21.15.16:0/2292373622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8348004d40 msgr2=0x7f83480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.905+0000 7f835effd640 1 --2- 172.21.15.16:0/2292373622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8348004d40 0x7f83480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.905+0000 7f835effd640 1 -- 172.21.15.16:0/2292373622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8348003520 con 0x7f8348003f20 2024-09-19T07:57:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.905+0000 7f835effd640 1 --2- 172.21.15.16:0/2292373622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348003f20 0x7f8348004800 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f8354001830 tx=0x7f8348005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f74a3c2c7ae4e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.906+0000 7f835e7fc640 1 -- 172.21.15.16:0/2292373622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354037070 con 0x7f8348003f20 2024-09-19T07:57:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.906+0000 7f835e7fc640 1 -- 172.21.15.16:0/2292373622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f835402ab00 con 0x7f8348003f20 2024-09-19T07:57:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.906+0000 7f835e7fc640 1 -- 172.21.15.16:0/2292373622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354001430 con 0x7f8348003f20 2024-09-19T07:57:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.906+0000 7f8365b9f640 1 -- 172.21.15.16:0/2292373622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348003f20 msgr2=0x7f8348004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.906+0000 7f8365b9f640 1 --2- 172.21.15.16:0/2292373622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348003f20 0x7f8348004800 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f8354001830 tx=0x7f8348005830 comp rx=0 tx=0).stop 2024-09-19T07:57:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.906+0000 7f835ffff640 1 -- 172.21.15.16:0/2292373622 reap_dead start 2024-09-19T07:57:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.906+0000 7f8365b9f640 1 -- 172.21.15.16:0/2292373622 shutdown_connections 2024-09-19T07:57:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.906+0000 7f8365b9f640 1 -- 172.21.15.16:0/2292373622 >> 172.21.15.16:0/2292373622 conn(0x7f836009d740 msgr2=0x7f83600d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.906+0000 7f8365b9f640 1 -- 172.21.15.16:0/2292373622 shutdown_connections 2024-09-19T07:57:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.906+0000 7f8365b9f640 1 -- 172.21.15.16:0/2292373622 wait complete. 2024-09-19T07:57:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.907+0000 7f8365b9f640 1 Processor -- start 2024-09-19T07:57:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.908+0000 7f8365b9f640 1 -- start start 2024-09-19T07:57:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.908+0000 7f8365b9f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8348003f20 0x7f836015f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.908+0000 7f8365b9f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8360153720 0x7f8360152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.908+0000 7f8365b9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f836015fee0 0x7f83600b8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.908+0000 7f8365b9f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83600d0140 con 0x7f836015fee0 2024-09-19T07:57:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.908+0000 7f8365b9f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f83600d4b00 con 0x7f8348003f20 2024-09-19T07:57:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.908+0000 7f8365b9f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f836015f4a0 con 0x7f8360153720 2024-09-19T07:57:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.908+0000 7f835ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f836015fee0 0x7f83600b8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.909+0000 7f835ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f836015fee0 0x7f83600b8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42494/0 (socket says 172.21.15.16:42494) 2024-09-19T07:57:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.909+0000 7f835ffff640 1 -- 172.21.15.16:0/1999407713 learned_addr learned my addr 172.21.15.16:0/1999407713 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.909+0000 7f835effd640 1 --2- 172.21.15.16:0/1999407713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8360153720 0x7f8360152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.909+0000 7f835f7fe640 1 --2- 172.21.15.16:0/1999407713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8348003f20 0x7f836015f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.909+0000 7f835ffff640 1 -- 172.21.15.16:0/1999407713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8360153720 msgr2=0x7f8360152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.909+0000 7f835ffff640 1 --2- 172.21.15.16:0/1999407713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8360153720 0x7f8360152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.909+0000 7f835ffff640 1 -- 172.21.15.16:0/1999407713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8348003f20 msgr2=0x7f836015f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.909+0000 7f835ffff640 1 --2- 172.21.15.16:0/1999407713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8348003f20 0x7f836015f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.909+0000 7f835ffff640 1 -- 172.21.15.16:0/1999407713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83601608c0 con 0x7f836015fee0 2024-09-19T07:57:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.909+0000 7f835ffff640 1 --2- 172.21.15.16:0/1999407713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f836015fee0 0x7f83600b8d70 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f836008cee0 tx=0x7f835800a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.909+0000 7f8347fff640 1 -- 172.21.15.16:0/1999407713 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835800acb0 con 0x7f836015fee0 2024-09-19T07:57:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.909+0000 7f8347fff640 1 -- 172.21.15.16:0/1999407713 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8358019070 con 0x7f836015fee0 2024-09-19T07:57:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.909+0000 7f8347fff640 1 -- 172.21.15.16:0/1999407713 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358013630 con 0x7f836015fee0 2024-09-19T07:57:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.909+0000 7f8365b9f640 1 -- 172.21.15.16:0/1999407713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836015eb70 con 0x7f836015fee0 2024-09-19T07:57:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.909+0000 7f8365b9f640 1 -- 172.21.15.16:0/1999407713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f836015f130 con 0x7f836015fee0 2024-09-19T07:57:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.910+0000 7f8365b9f640 1 -- 172.21.15.16:0/1999407713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f83600d1be0 con 0x7f836015fee0 2024-09-19T07:57:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.911+0000 7f8347fff640 1 -- 172.21.15.16:0/1999407713 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f83580137d0 con 0x7f836015fee0 2024-09-19T07:57:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.911+0000 7f8347fff640 1 --2- 172.21.15.16:0/1999407713 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f834c06c2b0 0x7f834c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.911+0000 7f8347fff640 1 -- 172.21.15.16:0/1999407713 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f835808b960 con 0x7f836015fee0 2024-09-19T07:57:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.911+0000 7f835f7fe640 1 --2- 172.21.15.16:0/1999407713 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f834c06c2b0 0x7f834c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.911+0000 7f8347fff640 1 --2- 172.21.15.16:0/1999407713 >> [v2:172.21.15.146:6824/1757996733,v1:172.21.15.146:6825/1757996733] conn(0x7f834c0727a0 0x7f834c074be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.911+0000 7f8347fff640 1 -- 172.21.15.16:0/1999407713 --> [v2:172.21.15.146:6824/1757996733,v1:172.21.15.146:6825/1757996733] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f834c0752b0 con 0x7f834c0727a0 2024-09-19T07:57:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.911+0000 7f8347fff640 1 -- 172.21.15.16:0/1999407713 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=156) ==== 24+0+0 (secure 0 0 0) 0x7f83580902c0 con 0x7f836015fee0 2024-09-19T07:57:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.912+0000 7f835effd640 1 --2- 172.21.15.16:0/1999407713 >> [v2:172.21.15.146:6824/1757996733,v1:172.21.15.146:6825/1757996733] conn(0x7f834c0727a0 0x7f834c074be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.912+0000 7f835f7fe640 1 --2- 172.21.15.16:0/1999407713 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f834c06c2b0 0x7f834c06e770 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f83600ba360 tx=0x7f8350008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.912+0000 7f835effd640 1 --2- 172.21.15.16:0/1999407713 >> [v2:172.21.15.146:6824/1757996733,v1:172.21.15.146:6825/1757996733] conn(0x7f834c0727a0 0x7f834c074be0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.913+0000 7f8347fff640 1 -- 172.21.15.16:0/1999407713 <== osd.7 v2:172.21.15.146:6824/1757996733 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f834c0752b0 con 0x7f834c0727a0 2024-09-19T07:57:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.925+0000 7f8365b9f640 1 -- 172.21.15.16:0/1999407713 --> [v2:172.21.15.146:6824/1757996733,v1:172.21.15.146:6825/1757996733] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f83600d0e30 con 0x7f834c0727a0 2024-09-19T07:57:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.926+0000 7f8347fff640 1 -- 172.21.15.16:0/1999407713 <== osd.7 v2:172.21.15.146:6824/1757996733 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f83600d0e30 con 0x7f834c0727a0 2024-09-19T07:57:29.926 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T07:57:29.926 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T07:57:29.927 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T07:57:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.926+0000 7f8365b9f640 1 -- 172.21.15.16:0/1999407713 >> [v2:172.21.15.146:6824/1757996733,v1:172.21.15.146:6825/1757996733] conn(0x7f834c0727a0 msgr2=0x7f834c074be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.926+0000 7f8365b9f640 1 --2- 172.21.15.16:0/1999407713 >> [v2:172.21.15.146:6824/1757996733,v1:172.21.15.146:6825/1757996733] conn(0x7f834c0727a0 0x7f834c074be0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.926+0000 7f8365b9f640 1 -- 172.21.15.16:0/1999407713 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f834c06c2b0 msgr2=0x7f834c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.926+0000 7f8365b9f640 1 --2- 172.21.15.16:0/1999407713 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f834c06c2b0 0x7f834c06e770 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f83600ba360 tx=0x7f8350008040 comp rx=0 tx=0).stop 2024-09-19T07:57:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.926+0000 7f8365b9f640 1 -- 172.21.15.16:0/1999407713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f836015fee0 msgr2=0x7f83600b8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.926+0000 7f8365b9f640 1 --2- 172.21.15.16:0/1999407713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f836015fee0 0x7f83600b8d70 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f836008cee0 tx=0x7f835800a750 comp rx=0 tx=0).stop 2024-09-19T07:57:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.926+0000 7f835ffff640 1 -- 172.21.15.16:0/1999407713 reap_dead start 2024-09-19T07:57:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.926+0000 7f8365b9f640 1 -- 172.21.15.16:0/1999407713 shutdown_connections 2024-09-19T07:57:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.926+0000 7f8365b9f640 1 -- 172.21.15.16:0/1999407713 >> 172.21.15.16:0/1999407713 conn(0x7f836009d740 msgr2=0x7f836006ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.927+0000 7f8365b9f640 1 -- 172.21.15.16:0/1999407713 shutdown_connections 2024-09-19T07:57:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:29.927+0000 7f8365b9f640 1 -- 172.21.15.16:0/1999407713 wait complete. 2024-09-19T07:57:29.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-19T07:57:30.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.039+0000 7f32a815f640 1 Processor -- start 2024-09-19T07:57:30.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.040+0000 7f32a815f640 1 -- start start 2024-09-19T07:57:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.040+0000 7f32a815f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32a015bfd0 0x7f32a00d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.040+0000 7f32a815f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32a00d24d0 con 0x7f32a015c3b0 2024-09-19T07:57:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.040+0000 7f32a815f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32a00d27b0 con 0x7f32a00ccf00 2024-09-19T07:57:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.040+0000 7f32a815f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f32a00ceab0 con 0x7f32a015bfd0 2024-09-19T07:57:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.041+0000 7f32a5efb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32a015bfd0 0x7f32a00d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.041+0000 7f32a56fa640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f32a015c3b0 0x7f32a00cc7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60954/0 (socket says 172.21.15.16:60954) 2024-09-19T07:57:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.041+0000 7f32a56fa640 1 -- 172.21.15.16:0/3607373927 learned_addr learned my addr 172.21.15.16:0/3607373927 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.041+0000 7f32a5efb640 1 --2- 172.21.15.16:0/3607373927 >> v2:172.21.15.16:3300/0 conn(0x7f32a015bfd0 0x7f32a00d1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42508/0 (socket says 172.21.15.16:42508) 2024-09-19T07:57:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.041+0000 7f32a5efb640 1 -- 172.21.15.16:0/3607373927 >> v1:172.21.15.146:6789/0 conn(0x7f32a00ccf00 legacy=0x7f32a00b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.041+0000 7f32a5efb640 1 -- 172.21.15.16:0/3607373927 >> v1:172.21.15.16:6789/0 conn(0x7f32a015c3b0 legacy=0x7f32a00cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.041+0000 7f32a4ef9640 1 -- 172.21.15.16:0/3607373927 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2029316388 0 0) 0x7f32a00d24d0 con 0x7f32a015c3b0 2024-09-19T07:57:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.041+0000 7f32a5efb640 1 -- 172.21.15.16:0/3607373927 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a00d2a90 con 0x7f32a015bfd0 2024-09-19T07:57:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.042+0000 7f32a5efb640 1 --2- 172.21.15.16:0/3607373927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015bfd0 0x7f32a00d1930 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f32980097a0 tx=0x7f329802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86477fa2e6fda366 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.042+0000 7f32a4ef9640 1 -- 172.21.15.16:0/3607373927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298037070 con 0x7f32a015bfd0 2024-09-19T07:57:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.042+0000 7f32a4ef9640 1 -- 172.21.15.16:0/3607373927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329802ae50 con 0x7f32a015bfd0 2024-09-19T07:57:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.042+0000 7f32a4ef9640 1 -- 172.21.15.16:0/3607373927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298032660 con 0x7f32a015bfd0 2024-09-19T07:57:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.042+0000 7f32a815f640 1 -- 172.21.15.16:0/3607373927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015bfd0 msgr2=0x7f32a00d1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.042+0000 7f32a815f640 1 --2- 172.21.15.16:0/3607373927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015bfd0 0x7f32a00d1930 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f32980097a0 tx=0x7f329802a140 comp rx=0 tx=0).stop 2024-09-19T07:57:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.042+0000 7f32a815f640 1 -- 172.21.15.16:0/3607373927 shutdown_connections 2024-09-19T07:57:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.042+0000 7f32a815f640 1 --2- 172.21.15.16:0/3607373927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015bfd0 0x7f32a00d1930 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.042+0000 7f32a815f640 1 -- 172.21.15.16:0/3607373927 >> 172.21.15.16:0/3607373927 conn(0x7f32a0067950 msgr2=0x7f32a0067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.042+0000 7f32a815f640 1 -- 172.21.15.16:0/3607373927 shutdown_connections 2024-09-19T07:57:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.042+0000 7f32a815f640 1 -- 172.21.15.16:0/3607373927 wait complete. 2024-09-19T07:57:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.044+0000 7f32a815f640 1 Processor -- start 2024-09-19T07:57:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.044+0000 7f32a815f640 1 -- start start 2024-09-19T07:57:30.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.044+0000 7f32a815f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32a00ccf00 0x7f32a00c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:30.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.044+0000 7f32a815f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015bfd0 0x7f32a0155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:30.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.044+0000 7f32a815f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32a015c3b0 0x7f32a01f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:30.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.044+0000 7f32a815f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32a00d1e70 con 0x7f32a015bfd0 2024-09-19T07:57:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.044+0000 7f32a815f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f32a00d3050 con 0x7f32a015c3b0 2024-09-19T07:57:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.044+0000 7f32a815f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f32a00d1ff0 con 0x7f32a00ccf00 2024-09-19T07:57:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.044+0000 7f32a56fa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015bfd0 0x7f32a0155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.044+0000 7f32a56fa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015bfd0 0x7f32a0155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42520/0 (socket says 172.21.15.16:42520) 2024-09-19T07:57:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.044+0000 7f32a56fa640 1 -- 172.21.15.16:0/213803518 learned_addr learned my addr 172.21.15.16:0/213803518 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.045+0000 7f32a66fc640 1 --2- 172.21.15.16:0/213803518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32a015c3b0 0x7f32a01f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.045+0000 7f32a5efb640 1 --2- 172.21.15.16:0/213803518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32a00ccf00 0x7f32a00c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.045+0000 7f32a56fa640 1 -- 172.21.15.16:0/213803518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32a00ccf00 msgr2=0x7f32a00c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.045+0000 7f32a56fa640 1 --2- 172.21.15.16:0/213803518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32a00ccf00 0x7f32a00c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.045+0000 7f32a56fa640 1 -- 172.21.15.16:0/213803518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32a015c3b0 msgr2=0x7f32a01f4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.045+0000 7f32a56fa640 1 --2- 172.21.15.16:0/213803518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32a015c3b0 0x7f32a01f4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.045+0000 7f32a56fa640 1 -- 172.21.15.16:0/213803518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a00c80f0 con 0x7f32a015bfd0 2024-09-19T07:57:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.045+0000 7f32a66fc640 1 --2- 172.21.15.16:0/213803518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32a015c3b0 0x7f32a01f4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.045+0000 7f32a56fa640 1 --2- 172.21.15.16:0/213803518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015bfd0 0x7f32a0155700 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f329400c460 tx=0x7f329400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.045+0000 7f329e7fc640 1 -- 172.21.15.16:0/213803518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294018070 con 0x7f32a015bfd0 2024-09-19T07:57:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.045+0000 7f329e7fc640 1 -- 172.21.15.16:0/213803518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3294007500 con 0x7f32a015bfd0 2024-09-19T07:57:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.045+0000 7f32a815f640 1 -- 172.21.15.16:0/213803518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a00c7b30 con 0x7f32a015bfd0 2024-09-19T07:57:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.045+0000 7f329e7fc640 1 -- 172.21.15.16:0/213803518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294004a20 con 0x7f32a015bfd0 2024-09-19T07:57:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.045+0000 7f32a815f640 1 -- 172.21.15.16:0/213803518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a00c5e00 con 0x7f32a015bfd0 2024-09-19T07:57:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.046+0000 7f32a815f640 1 -- 172.21.15.16:0/213803518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a00c6000 con 0x7f32a015bfd0 2024-09-19T07:57:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.047+0000 7f329e7fc640 1 -- 172.21.15.16:0/213803518 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f329401b070 con 0x7f32a015bfd0 2024-09-19T07:57:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.047+0000 7f329e7fc640 1 --2- 172.21.15.16:0/213803518 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f328c06c2b0 0x7f328c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.047+0000 7f329e7fc640 1 -- 172.21.15.16:0/213803518 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f329408a4c0 con 0x7f32a015bfd0 2024-09-19T07:57:30.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.047+0000 7f32a5efb640 1 --2- 172.21.15.16:0/213803518 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f328c06c2b0 0x7f328c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:30.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.048+0000 7f32a5efb640 1 --2- 172.21.15.16:0/213803518 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f328c06c2b0 0x7f328c06e770 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3298034000 tx=0x7f329802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:30.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.051+0000 7f329e7fc640 1 -- 172.21.15.16:0/213803518 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f32a00c6000 con 0x7f32a015bfd0 2024-09-19T07:57:30.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.227+0000 7f32a815f640 1 -- 172.21.15.16:0/213803518 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f32a01f5410 con 0x7f328c06c2b0 2024-09-19T07:57:30.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.228+0000 7f329e7fc640 1 -- 172.21.15.16:0/213803518 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3298032790 con 0x7f328c06c2b0 2024-09-19T07:57:30.229 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 7 to deep-scrub 2024-09-19T07:57:30.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.230+0000 7f32a815f640 1 -- 172.21.15.16:0/213803518 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f328c06c2b0 msgr2=0x7f328c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:30.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.230+0000 7f32a815f640 1 --2- 172.21.15.16:0/213803518 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f328c06c2b0 0x7f328c06e770 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3298034000 tx=0x7f329802e000 comp rx=0 tx=0).stop 2024-09-19T07:57:30.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.230+0000 7f32a815f640 1 -- 172.21.15.16:0/213803518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015bfd0 msgr2=0x7f32a0155700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:30.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.230+0000 7f32a815f640 1 --2- 172.21.15.16:0/213803518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015bfd0 0x7f32a0155700 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f329400c460 tx=0x7f329400c920 comp rx=0 tx=0).stop 2024-09-19T07:57:30.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.230+0000 7f32a815f640 1 -- 172.21.15.16:0/213803518 shutdown_connections 2024-09-19T07:57:30.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.230+0000 7f32a815f640 1 --2- 172.21.15.16:0/213803518 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f328c06c2b0 0x7f328c06e770 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:30.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.230+0000 7f32a815f640 1 --2- 172.21.15.16:0/213803518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32a015c3b0 0x7f32a01f4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:30.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.230+0000 7f32a815f640 1 --2- 172.21.15.16:0/213803518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a015bfd0 0x7f32a0155700 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:30.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.230+0000 7f32a815f640 1 --2- 172.21.15.16:0/213803518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32a00ccf00 0x7f32a00c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:30.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.230+0000 7f32a815f640 1 -- 172.21.15.16:0/213803518 >> 172.21.15.16:0/213803518 conn(0x7f32a0067950 msgr2=0x7f32a006cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:30.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.230+0000 7f32a815f640 1 -- 172.21.15.16:0/213803518 shutdown_connections 2024-09-19T07:57:30.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.230+0000 7f32a815f640 1 -- 172.21.15.16:0/213803518 wait complete. 2024-09-19T07:57:30.241 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:57:30.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.342+0000 7f4736f91640 1 Processor -- start 2024-09-19T07:57:30.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.342+0000 7f4736f91640 1 -- start start 2024-09-19T07:57:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.343+0000 7f4736f91640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f473015e620 0x7f47300d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.343+0000 7f4736f91640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47300d1930 0x7f473006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.343+0000 7f4736f91640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47300717f0 con 0x7f47300b54a0 2024-09-19T07:57:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.343+0000 7f4736f91640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7f47300d2b80 con 0x7f473015e620 2024-09-19T07:57:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.343+0000 7f4736f91640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4730071a90 con 0x7f47300d1930 2024-09-19T07:57:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.343+0000 7f472ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f473015e620 0x7f47300d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.343+0000 7f472ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f473015e620 0x7f47300d13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42522/0 (socket says 172.21.15.16:42522) 2024-09-19T07:57:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.343+0000 7f472ffff640 1 -- 172.21.15.16:0/1693132768 learned_addr learned my addr 172.21.15.16:0/1693132768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.343+0000 7f4734d2d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f47300b54a0 0x7f47300b5880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60958/0 (socket says 172.21.15.16:60958) 2024-09-19T07:57:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.344+0000 7f473552e640 1 --2- 172.21.15.16:0/1693132768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47300d1930 0x7f473006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.344+0000 7f472ffff640 1 -- 172.21.15.16:0/1693132768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47300d1930 msgr2=0x7f473006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:30.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.344+0000 7f472ffff640 1 --2- 172.21.15.16:0/1693132768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47300d1930 0x7f473006d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.344+0000 7f472ffff640 1 -- 172.21.15.16:0/1693132768 >> v1:172.21.15.16:6789/0 conn(0x7f47300b54a0 legacy=0x7f47300b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.344+0000 7f472ffff640 1 -- 172.21.15.16:0/1693132768 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4730071d70 con 0x7f473015e620 2024-09-19T07:57:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.344+0000 7f472f7fe640 1 -- 172.21.15.16:0/1693132768 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3366293895 0 0) 0x7f47300717f0 con 0x7f47300b54a0 2024-09-19T07:57:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.344+0000 7f473552e640 1 --2- 172.21.15.16:0/1693132768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47300d1930 0x7f473006d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:57:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.344+0000 7f472ffff640 1 --2- 172.21.15.16:0/1693132768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f473015e620 0x7f47300d13f0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f4718009870 tx=0x7f471802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258a36a893f0058 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.344+0000 7f472f7fe640 1 -- 172.21.15.16:0/1693132768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4718037070 con 0x7f473015e620 2024-09-19T07:57:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.345+0000 7f472f7fe640 1 -- 172.21.15.16:0/1693132768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4718032440 con 0x7f473015e620 2024-09-19T07:57:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.345+0000 7f472f7fe640 1 -- 172.21.15.16:0/1693132768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4718032810 con 0x7f473015e620 2024-09-19T07:57:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.345+0000 7f4736f91640 1 -- 172.21.15.16:0/1693132768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f473015e620 msgr2=0x7f47300d13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.345+0000 7f4736f91640 1 --2- 172.21.15.16:0/1693132768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f473015e620 0x7f47300d13f0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f4718009870 tx=0x7f471802a1c0 comp rx=0 tx=0).stop 2024-09-19T07:57:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.345+0000 7f4736f91640 1 -- 172.21.15.16:0/1693132768 shutdown_connections 2024-09-19T07:57:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.345+0000 7f4736f91640 1 --2- 172.21.15.16:0/1693132768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47300d1930 0x7f473006d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.345+0000 7f4736f91640 1 --2- 172.21.15.16:0/1693132768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f473015e620 0x7f47300d13f0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:30.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.345+0000 7f4736f91640 1 -- 172.21.15.16:0/1693132768 >> 172.21.15.16:0/1693132768 conn(0x7f473006feb0 msgr2=0x7f4730070290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.345+0000 7f4736f91640 1 -- 172.21.15.16:0/1693132768 shutdown_connections 2024-09-19T07:57:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.345+0000 7f4736f91640 1 -- 172.21.15.16:0/1693132768 wait complete. 2024-09-19T07:57:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.346+0000 7f4736f91640 1 Processor -- start 2024-09-19T07:57:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.346+0000 7f4736f91640 1 -- start start 2024-09-19T07:57:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.347+0000 7f4736f91640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47300b54a0 0x7f47300b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.347+0000 7f4736f91640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47300d1930 0x7f47300cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.347+0000 7f4736f91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f473015e620 0x7f47300aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.347+0000 7f4736f91640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47300cec20 con 0x7f473015e620 2024-09-19T07:57:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.347+0000 7f4736f91640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4730072330 con 0x7f47300d1930 2024-09-19T07:57:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.347+0000 7f4736f91640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f47300ceda0 con 0x7f47300b54a0 2024-09-19T07:57:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.347+0000 7f473552e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f473015e620 0x7f47300aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.347+0000 7f473552e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f473015e620 0x7f47300aa6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42528/0 (socket says 172.21.15.16:42528) 2024-09-19T07:57:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.347+0000 7f473552e640 1 -- 172.21.15.16:0/2373257059 learned_addr learned my addr 172.21.15.16:0/2373257059 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.347+0000 7f4734d2d640 1 --2- 172.21.15.16:0/2373257059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47300b54a0 0x7f47300b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.347+0000 7f472ffff640 1 --2- 172.21.15.16:0/2373257059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47300d1930 0x7f47300cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.347+0000 7f473552e640 1 -- 172.21.15.16:0/2373257059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47300b54a0 msgr2=0x7f47300b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.347+0000 7f473552e640 1 --2- 172.21.15.16:0/2373257059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47300b54a0 0x7f47300b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.348+0000 7f473552e640 1 -- 172.21.15.16:0/2373257059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47300d1930 msgr2=0x7f47300cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.348+0000 7f473552e640 1 --2- 172.21.15.16:0/2373257059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47300d1930 0x7f47300cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.348+0000 7f473552e640 1 -- 172.21.15.16:0/2373257059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47301f54b0 con 0x7f473015e620 2024-09-19T07:57:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.348+0000 7f472ffff640 1 --2- 172.21.15.16:0/2373257059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47300d1930 0x7f47300cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:57:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.348+0000 7f473552e640 1 --2- 172.21.15.16:0/2373257059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f473015e620 0x7f47300aa6d0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f472000cef0 tx=0x7f472000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.348+0000 7f472d7fa640 1 -- 172.21.15.16:0/2373257059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472001a070 con 0x7f473015e620 2024-09-19T07:57:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.348+0000 7f472d7fa640 1 -- 172.21.15.16:0/2373257059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f472000d860 con 0x7f473015e620 2024-09-19T07:57:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.348+0000 7f4736f91640 1 -- 172.21.15.16:0/2373257059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47300cc390 con 0x7f473015e620 2024-09-19T07:57:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.348+0000 7f472d7fa640 1 -- 172.21.15.16:0/2373257059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472000dc50 con 0x7f473015e620 2024-09-19T07:57:30.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.348+0000 7f4736f91640 1 -- 172.21.15.16:0/2373257059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47300cc950 con 0x7f473015e620 2024-09-19T07:57:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.349+0000 7f4736f91640 1 -- 172.21.15.16:0/2373257059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47300d13f0 con 0x7f473015e620 2024-09-19T07:57:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.349+0000 7f472d7fa640 1 -- 172.21.15.16:0/2373257059 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f472001d070 con 0x7f473015e620 2024-09-19T07:57:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.350+0000 7f472d7fa640 1 --2- 172.21.15.16:0/2373257059 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f471006c2b0 0x7f471006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.350+0000 7f472d7fa640 1 -- 172.21.15.16:0/2373257059 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f472008bb00 con 0x7f473015e620 2024-09-19T07:57:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.350+0000 7f4734d2d640 1 --2- 172.21.15.16:0/2373257059 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f471006c2b0 0x7f471006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.350+0000 7f4734d2d640 1 --2- 172.21.15.16:0/2373257059 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f471006c2b0 0x7f471006e770 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4728000f60 tx=0x7f4728007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.354+0000 7f472d7fa640 1 -- 172.21.15.16:0/2373257059 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f472005a500 con 0x7f473015e620 2024-09-19T07:57:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.528+0000 7f4736f91640 1 -- 172.21.15.16:0/2373257059 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f47300b64c0 con 0x7f471006c2b0 2024-09-19T07:57:30.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.529+0000 7f472d7fa640 1 -- 172.21.15.16:0/2373257059 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13460 (secure 0 0 0) 0x7f47300b64c0 con 0x7f471006c2b0 2024-09-19T07:57:30.529 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:30.529 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:57:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.531+0000 7f4736f91640 1 -- 172.21.15.16:0/2373257059 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f471006c2b0 msgr2=0x7f471006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.531+0000 7f4736f91640 1 --2- 172.21.15.16:0/2373257059 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f471006c2b0 0x7f471006e770 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4728000f60 tx=0x7f4728007c00 comp rx=0 tx=0).stop 2024-09-19T07:57:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.531+0000 7f4736f91640 1 -- 172.21.15.16:0/2373257059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f473015e620 msgr2=0x7f47300aa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.531+0000 7f4736f91640 1 --2- 172.21.15.16:0/2373257059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f473015e620 0x7f47300aa6d0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f472000cef0 tx=0x7f472000a710 comp rx=0 tx=0).stop 2024-09-19T07:57:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.531+0000 7f4736f91640 1 -- 172.21.15.16:0/2373257059 shutdown_connections 2024-09-19T07:57:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.531+0000 7f4736f91640 1 --2- 172.21.15.16:0/2373257059 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f471006c2b0 0x7f471006e770 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.531+0000 7f4736f91640 1 --2- 172.21.15.16:0/2373257059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f473015e620 0x7f47300aa6d0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.531+0000 7f4736f91640 1 --2- 172.21.15.16:0/2373257059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47300d1930 0x7f47300cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.531+0000 7f4736f91640 1 --2- 172.21.15.16:0/2373257059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47300b54a0 0x7f47300b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.531+0000 7f4736f91640 1 -- 172.21.15.16:0/2373257059 >> 172.21.15.16:0/2373257059 conn(0x7f473006feb0 msgr2=0x7f47300b5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.531+0000 7f4736f91640 1 -- 172.21.15.16:0/2373257059 shutdown_connections 2024-09-19T07:57:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:30.531+0000 7f4736f91640 1 -- 172.21.15.16:0/2373257059 wait complete. 2024-09-19T07:57:30.542 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":588,"stamp":"2024-09-19T07:57:28.666248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":770972,"kb_used_data":16488,"kb_used_omap":12,"kb_used_meta":754291,"kb_avail":754203748,"statfs":{"total":773094113280,"available":772304637952,"internally_reserved":0,"allocated":16883712,"data_stored":8476008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":772394544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14421,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-67,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001693"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":417,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-19T07:57:22.703562+0000","last_change":"2024-09-19T07:42:55.992671+0000","last_active":"2024-09-19T07:57:22.703562+0000","last_peered":"2024-09-19T07:57:22.703562+0000","last_clean":"2024-09-19T07:57:22.703562+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:57:22.703562+0000","last_undegraded":"2024-09-19T07:57:22.703562+0000","last_fullsized":"2024-09-19T07:57:22.703562+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:42:54.938297+0000","last_clean_scrub_stamp":"2024-09-19T07:42:54.938297+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:11.187961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509617,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87768,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284072,"statfs":{"total":96636764160,"available":96546889728,"internally_reserved":0,"allocated":2555904,"data_stored":1502277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":87292362},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509616,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":105300,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266540,"statfs":{"total":96636764160,"available":96528936960,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509617,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73296,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298544,"statfs":{"total":96636764160,"available":96561709056,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509617,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":87188,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284652,"statfs":{"total":96636764160,"available":96547483648,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509616,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105812,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266028,"statfs":{"total":96636764160,"available":96528412672,"internally_reserved":0,"allocated":2555904,"data_stored":1502277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542321,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":119124,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252716,"statfs":{"total":96636764160,"available":96514781184,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542321,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":83088,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288752,"statfs":{"total":96636764160,"available":96551682048,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575026,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":109396,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262444,"statfs":{"total":96636764160,"available":96524742656,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:57:30.546 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-19T07:42:54.938297+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=7, tm_min=42, tm_sec=54, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=7, tm_min=57, tm_sec=25, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T07:57:30.546 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-19T07:57:40.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:57:40.069 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:57:50.548 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:57:50.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.665+0000 7f1dc504c640 1 Processor -- start 2024-09-19T07:57:50.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.665+0000 7f1dc504c640 1 -- start start 2024-09-19T07:57:50.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.666+0000 7f1dc504c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dc00b54a0 0x7f1dc00b5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:50.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.666+0000 7f1dc504c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1dc006d1f0 0x7f1dc006d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:50.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.666+0000 7f1dc504c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc00717f0 con 0x7f1dc015e620 2024-09-19T07:57:50.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.666+0000 7f1dc504c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc00d2b80 con 0x7f1dc006d1f0 2024-09-19T07:57:50.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.666+0000 7f1dc504c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc00b5dc0 con 0x7f1dc00b54a0 2024-09-19T07:57:50.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.666+0000 7f1dbe59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1dc015e620 0x7f1dc00d13f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39028/0 (socket says 172.21.15.16:39028) 2024-09-19T07:57:50.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.666+0000 7f1dbe59c640 1 -- 172.21.15.16:0/773200410 learned_addr learned my addr 172.21.15.16:0/773200410 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:50.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.666+0000 7f1dbed9d640 1 --2- 172.21.15.16:0/773200410 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dc00b54a0 0x7f1dc00b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:50.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.666+0000 7f1dbf59e640 1 --2- 172.21.15.16:0/773200410 >> v2:172.21.15.146:3300/0 conn(0x7f1dc006d1f0 0x7f1dc006d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:50.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.666+0000 7f1dbdd9b640 1 -- 172.21.15.16:0/773200410 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 547156854 0 0) 0x7f1dc00717f0 con 0x7f1dc015e620 2024-09-19T07:57:50.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbdd9b640 1 -- 172.21.15.16:0/773200410 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1dac003520 con 0x7f1dc015e620 2024-09-19T07:57:50.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbdd9b640 1 -- 172.21.15.16:0/773200410 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2996958941 0 0) 0x7f1dac003520 con 0x7f1dc015e620 2024-09-19T07:57:50.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbdd9b640 1 -- 172.21.15.16:0/773200410 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dc00b54a0 msgr2=0x7f1dc00b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbdd9b640 1 --2- 172.21.15.16:0/773200410 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dc00b54a0 0x7f1dc00b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbdd9b640 1 -- 172.21.15.16:0/773200410 >> v2:172.21.15.146:3300/0 conn(0x7f1dc006d1f0 msgr2=0x7f1dc006d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbdd9b640 1 --2- 172.21.15.16:0/773200410 >> v2:172.21.15.146:3300/0 conn(0x7f1dc006d1f0 0x7f1dc006d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbdd9b640 1 -- 172.21.15.16:0/773200410 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc0071a90 con 0x7f1dc015e620 2024-09-19T07:57:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbdd9b640 1 -- 172.21.15.16:0/773200410 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f1da8001670 con 0x7f1dc015e620 2024-09-19T07:57:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbdd9b640 1 -- 172.21.15.16:0/773200410 >> v1:172.21.15.16:6789/0 conn(0x7f1dc015e620 legacy=0x7f1dc00d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbdd9b640 1 --2- 172.21.15.16:0/773200410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dac0039b0 0x7f1dac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbdd9b640 1 --2- 172.21.15.16:0/773200410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dac004800 0x7f1dac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbdd9b640 1 --2- 172.21.15.16:0/773200410 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dac0051c0 0x7f1dac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbdd9b640 1 -- 172.21.15.16:0/773200410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc00d2b80 con 0x7f1dac004800 2024-09-19T07:57:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbdd9b640 1 -- 172.21.15.16:0/773200410 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc00b5dc0 con 0x7f1dac0039b0 2024-09-19T07:57:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbdd9b640 1 -- 172.21.15.16:0/773200410 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1dac00bf30 con 0x7f1dac0051c0 2024-09-19T07:57:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbed9d640 1 --2- 172.21.15.16:0/773200410 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dc00b54a0 0x7f1dc00b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:57:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbe59c640 1 --2- 172.21.15.16:0/773200410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dac004800 0x7f1dac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbf59e640 1 --2- 172.21.15.16:0/773200410 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dac0051c0 0x7f1dac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbed9d640 1 --2- 172.21.15.16:0/773200410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dac0039b0 0x7f1dac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbe59c640 1 -- 172.21.15.16:0/773200410 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dac0051c0 msgr2=0x7f1dac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbe59c640 1 --2- 172.21.15.16:0/773200410 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dac0051c0 0x7f1dac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbe59c640 1 -- 172.21.15.16:0/773200410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dac0039b0 msgr2=0x7f1dac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbe59c640 1 --2- 172.21.15.16:0/773200410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dac0039b0 0x7f1dac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbe59c640 1 -- 172.21.15.16:0/773200410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac003520 con 0x7f1dac004800 2024-09-19T07:57:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.667+0000 7f1dbed9d640 1 --2- 172.21.15.16:0/773200410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dac0039b0 0x7f1dac003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:57:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.668+0000 7f1dbe59c640 1 --2- 172.21.15.16:0/773200410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dac004800 0x7f1dac004c80 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0161f30 tx=0x7f1dac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc2b204ef62c5ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.668+0000 7f1dbdd9b640 1 -- 172.21.15.16:0/773200410 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8007d70 con 0x7f1dac004800 2024-09-19T07:57:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.668+0000 7f1dbdd9b640 1 -- 172.21.15.16:0/773200410 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da8005ad0 con 0x7f1dac004800 2024-09-19T07:57:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.668+0000 7f1dbdd9b640 1 -- 172.21.15.16:0/773200410 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8007070 con 0x7f1dac004800 2024-09-19T07:57:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.669+0000 7f1dc504c640 1 -- 172.21.15.16:0/773200410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dac004800 msgr2=0x7f1dac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.669+0000 7f1dc504c640 1 --2- 172.21.15.16:0/773200410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dac004800 0x7f1dac004c80 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0161f30 tx=0x7f1dac005830 comp rx=0 tx=0).stop 2024-09-19T07:57:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.669+0000 7f1dbf59e640 1 -- 172.21.15.16:0/773200410 reap_dead start 2024-09-19T07:57:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.669+0000 7f1dc504c640 1 -- 172.21.15.16:0/773200410 shutdown_connections 2024-09-19T07:57:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.669+0000 7f1dc504c640 1 -- 172.21.15.16:0/773200410 >> 172.21.15.16:0/773200410 conn(0x7f1dc006feb0 msgr2=0x7f1dc0070290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.669+0000 7f1dc504c640 1 -- 172.21.15.16:0/773200410 shutdown_connections 2024-09-19T07:57:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.669+0000 7f1dc504c640 1 -- 172.21.15.16:0/773200410 wait complete. 2024-09-19T07:57:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.671+0000 7f1dc504c640 1 Processor -- start 2024-09-19T07:57:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.671+0000 7f1dc504c640 1 -- start start 2024-09-19T07:57:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.671+0000 7f1dc504c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dac004800 0x7f1dc0159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.671+0000 7f1dc504c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dc0159a70 0x7f1dc015b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.671+0000 7f1dc504c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dc014ca00 0x7f1dc015c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.671+0000 7f1dc504c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc0072330 con 0x7f1dac004800 2024-09-19T07:57:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.671+0000 7f1dc504c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc0072050 con 0x7f1dc014ca00 2024-09-19T07:57:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.671+0000 7f1dc504c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc00d1b00 con 0x7f1dc0159a70 2024-09-19T07:57:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.671+0000 7f1dbed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dac004800 0x7f1dc0159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.672+0000 7f1dbed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dac004800 0x7f1dc0159040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49174/0 (socket says 172.21.15.16:49174) 2024-09-19T07:57:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.672+0000 7f1dbed9d640 1 -- 172.21.15.16:0/303557944 learned_addr learned my addr 172.21.15.16:0/303557944 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.672+0000 7f1dbe59c640 1 --2- 172.21.15.16:0/303557944 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dc0159a70 0x7f1dc015b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.672+0000 7f1dbf59e640 1 --2- 172.21.15.16:0/303557944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dc014ca00 0x7f1dc015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.672+0000 7f1dbed9d640 1 -- 172.21.15.16:0/303557944 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dc0159a70 msgr2=0x7f1dc015b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.672+0000 7f1dbed9d640 1 --2- 172.21.15.16:0/303557944 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dc0159a70 0x7f1dc015b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.672+0000 7f1dbed9d640 1 -- 172.21.15.16:0/303557944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dc014ca00 msgr2=0x7f1dc015c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.672+0000 7f1dbed9d640 1 --2- 172.21.15.16:0/303557944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dc014ca00 0x7f1dc015c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.672+0000 7f1dbed9d640 1 -- 172.21.15.16:0/303557944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc015b750 con 0x7f1dac004800 2024-09-19T07:57:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.672+0000 7f1dbed9d640 1 --2- 172.21.15.16:0/303557944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dac004800 0x7f1dc0159040 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f1db8011680 tx=0x7f1db8011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.672+0000 7f1db77fe640 1 -- 172.21.15.16:0/303557944 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db800f9c0 con 0x7f1dac004800 2024-09-19T07:57:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.672+0000 7f1db77fe640 1 -- 172.21.15.16:0/303557944 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db800fb60 con 0x7f1dac004800 2024-09-19T07:57:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.672+0000 7f1dc504c640 1 -- 172.21.15.16:0/303557944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc0159eb0 con 0x7f1dac004800 2024-09-19T07:57:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.672+0000 7f1db77fe640 1 -- 172.21.15.16:0/303557944 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8015460 con 0x7f1dac004800 2024-09-19T07:57:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.672+0000 7f1dc504c640 1 -- 172.21.15.16:0/303557944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc015a450 con 0x7f1dac004800 2024-09-19T07:57:50.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.674+0000 7f1dc504c640 1 -- 172.21.15.16:0/303557944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc0069350 con 0x7f1dac004800 2024-09-19T07:57:50.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.674+0000 7f1db77fe640 1 -- 172.21.15.16:0/303557944 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1db8015600 con 0x7f1dac004800 2024-09-19T07:57:50.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.674+0000 7f1db77fe640 1 --2- 172.21.15.16:0/303557944 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1da006c2e0 0x7f1da006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:50.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.675+0000 7f1db77fe640 1 -- 172.21.15.16:0/303557944 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f1db808b7f0 con 0x7f1dac004800 2024-09-19T07:57:50.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.675+0000 7f1dbe59c640 1 --2- 172.21.15.16:0/303557944 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1da006c2e0 0x7f1da006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:50.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.675+0000 7f1dbe59c640 1 --2- 172.21.15.16:0/303557944 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1da006c2e0 0x7f1da006e7a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1da8033000 tx=0x7f1da8009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.679+0000 7f1db77fe640 1 -- 172.21.15.16:0/303557944 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1db8019030 con 0x7f1dac004800 2024-09-19T07:57:50.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.853+0000 7f1dc504c640 1 -- 172.21.15.16:0/303557944 --> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1dc00d3030 con 0x7f1da006c2e0 2024-09-19T07:57:50.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.854+0000 7f1db77fe640 1 -- 172.21.15.16:0/303557944 <== mgr.4103 v2:172.21.15.146:6832/413611562 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13391 (secure 0 0 0) 0x7f1da8047460 con 0x7f1da006c2e0 2024-09-19T07:57:50.854 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:50.854 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T07:57:50.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.856+0000 7f1dc504c640 1 -- 172.21.15.16:0/303557944 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1da006c2e0 msgr2=0x7f1da006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:50.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.856+0000 7f1dc504c640 1 --2- 172.21.15.16:0/303557944 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1da006c2e0 0x7f1da006e7a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1da8033000 tx=0x7f1da8009030 comp rx=0 tx=0).stop 2024-09-19T07:57:50.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.856+0000 7f1dc504c640 1 -- 172.21.15.16:0/303557944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dac004800 msgr2=0x7f1dc0159040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:50.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.856+0000 7f1dc504c640 1 --2- 172.21.15.16:0/303557944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dac004800 0x7f1dc0159040 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f1db8011680 tx=0x7f1db8011b40 comp rx=0 tx=0).stop 2024-09-19T07:57:50.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.856+0000 7f1dc504c640 1 -- 172.21.15.16:0/303557944 shutdown_connections 2024-09-19T07:57:50.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.856+0000 7f1dc504c640 1 --2- 172.21.15.16:0/303557944 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f1da006c2e0 0x7f1da006e7a0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:50.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.856+0000 7f1dc504c640 1 --2- 172.21.15.16:0/303557944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dc014ca00 0x7f1dc015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:50.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.856+0000 7f1dc504c640 1 --2- 172.21.15.16:0/303557944 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1dc0159a70 0x7f1dc015b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:50.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.856+0000 7f1dc504c640 1 --2- 172.21.15.16:0/303557944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dac004800 0x7f1dc0159040 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:50.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.856+0000 7f1dc504c640 1 -- 172.21.15.16:0/303557944 >> 172.21.15.16:0/303557944 conn(0x7f1dc006feb0 msgr2=0x7f1dc01615b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:50.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.856+0000 7f1dc504c640 1 -- 172.21.15.16:0/303557944 shutdown_connections 2024-09-19T07:57:50.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.856+0000 7f1dc504c640 1 -- 172.21.15.16:0/303557944 wait complete. 2024-09-19T07:57:50.867 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":599,"stamp":"2024-09-19T07:57:50.670694+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":770960,"kb_used_data":16488,"kb_used_omap":12,"kb_used_meta":754291,"kb_avail":754203760,"statfs":{"total":773094113280,"available":772304650240,"internally_reserved":0,"allocated":16883712,"data_stored":8476008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":772394544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002365"},"pg_stats":[{"pgid":"1.0","version":"156'119","reported_seq":428,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-19T07:57:31.188555+0000","last_change":"2024-09-19T07:57:31.188609+0000","last_active":"2024-09-19T07:57:31.188555+0000","last_peered":"2024-09-19T07:57:31.188555+0000","last_clean":"2024-09-19T07:57:31.188555+0000","last_became_active":"2024-09-19T07:42:55.991699+0000","last_became_peered":"2024-09-19T07:42:55.991699+0000","last_unstale":"2024-09-19T07:57:31.188555+0000","last_undegraded":"2024-09-19T07:57:31.188555+0000","last_fullsized":"2024-09-19T07:57:31.188555+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'119","last_scrub_stamp":"2024-09-19T07:57:31.188470+0000","last_deep_scrub":"156'119","last_deep_scrub_stamp":"2024-09-19T07:57:31.188470+0000","last_clean_scrub_stamp":"2024-09-19T07:57:31.188470+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:58.046441+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509622,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87760,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284080,"statfs":{"total":96636764160,"available":96546897920,"internally_reserved":0,"allocated":2555904,"data_stored":1502277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":87292362},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509621,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":105300,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94266540,"statfs":{"total":96636764160,"available":96528936960,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509622,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73296,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298544,"statfs":{"total":96636764160,"available":96561709056,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509622,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":87188,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284652,"statfs":{"total":96636764160,"available":96547483648,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509621,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105808,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266032,"statfs":{"total":96636764160,"available":96528416768,"internally_reserved":0,"allocated":2555904,"data_stored":1502277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542325,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":119124,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252716,"statfs":{"total":96636764160,"available":96514781184,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542325,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":83088,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288752,"statfs":{"total":96636764160,"available":96551682048,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575030,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":109396,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94262444,"statfs":{"total":96636764160,"available":96524742656,"internally_reserved":0,"allocated":1961984,"data_stored":911909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:57:50.868 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-19T07:57:50.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.970+0000 7fd61964c640 1 Processor -- start 2024-09-19T07:57:50.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.970+0000 7fd61964c640 1 -- start start 2024-09-19T07:57:50.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.971+0000 7fd61964c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd61415a690 0x7fd6140b4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:50.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.971+0000 7fd61964c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd61406f320 0x7fd614072450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:50.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.971+0000 7fd61964c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6140b34c0 con 0x7fd6140b4d60 2024-09-19T07:57:50.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.971+0000 7fd61964c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 -- 0x7fd6140b3780 con 0x7fd61415a690 2024-09-19T07:57:50.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.971+0000 7fd61964c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd6140b3a60 con 0x7fd61406f320 2024-09-19T07:57:50.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.971+0000 7fd612ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd61415a690 0x7fd6140b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:50.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.971+0000 7fd6127fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd6140b4d60 0x7fd61406ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39032/0 (socket says 172.21.15.16:39032) 2024-09-19T07:57:50.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.971+0000 7fd6127fc640 1 -- 172.21.15.16:0/4248138787 learned_addr learned my addr 172.21.15.16:0/4248138787 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:50.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.972+0000 7fd6137fe640 1 --2- 172.21.15.16:0/4248138787 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd61406f320 0x7fd614072450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:50.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.972+0000 7fd612ffd640 1 -- 172.21.15.16:0/4248138787 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd61406f320 msgr2=0x7fd614072450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:50.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.972+0000 7fd612ffd640 1 --2- 172.21.15.16:0/4248138787 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd61406f320 0x7fd614072450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:50.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.972+0000 7fd612ffd640 1 -- 172.21.15.16:0/4248138787 >> v1:172.21.15.16:6789/0 conn(0x7fd6140b4d60 legacy=0x7fd61406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:50.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.972+0000 7fd611ffb640 1 -- 172.21.15.16:0/4248138787 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1383896149 0 0) 0x7fd6140b34c0 con 0x7fd6140b4d60 2024-09-19T07:57:50.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.972+0000 7fd612ffd640 1 -- 172.21.15.16:0/4248138787 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6140b3d40 con 0x7fd61415a690 2024-09-19T07:57:50.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.972+0000 7fd6137fe640 1 --2- 172.21.15.16:0/4248138787 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd61406f320 0x7fd614072450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:57:50.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.972+0000 7fd612ffd640 1 --2- 172.21.15.16:0/4248138787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61415a690 0x7fd6140b4820 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fd6040097a0 tx=0x7fd60402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65441a161f0a7c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:50.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.972+0000 7fd611ffb640 1 -- 172.21.15.16:0/4248138787 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604037070 con 0x7fd61415a690 2024-09-19T07:57:50.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.972+0000 7fd611ffb640 1 -- 172.21.15.16:0/4248138787 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd604032440 con 0x7fd61415a690 2024-09-19T07:57:50.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.972+0000 7fd611ffb640 1 -- 172.21.15.16:0/4248138787 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604032810 con 0x7fd61415a690 2024-09-19T07:57:50.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.973+0000 7fd61964c640 1 -- 172.21.15.16:0/4248138787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61415a690 msgr2=0x7fd6140b4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:50.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.973+0000 7fd61964c640 1 --2- 172.21.15.16:0/4248138787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61415a690 0x7fd6140b4820 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fd6040097a0 tx=0x7fd60402a180 comp rx=0 tx=0).stop 2024-09-19T07:57:50.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.973+0000 7fd61964c640 1 -- 172.21.15.16:0/4248138787 shutdown_connections 2024-09-19T07:57:50.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.973+0000 7fd61964c640 1 --2- 172.21.15.16:0/4248138787 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd61406f320 0x7fd614072450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:50.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.973+0000 7fd61964c640 1 --2- 172.21.15.16:0/4248138787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61415a690 0x7fd6140b4820 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:50.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.973+0000 7fd61964c640 1 -- 172.21.15.16:0/4248138787 >> 172.21.15.16:0/4248138787 conn(0x7fd61406bcf0 msgr2=0x7fd61406c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:50.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.973+0000 7fd61964c640 1 -- 172.21.15.16:0/4248138787 shutdown_connections 2024-09-19T07:57:50.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.973+0000 7fd61964c640 1 -- 172.21.15.16:0/4248138787 wait complete. 2024-09-19T07:57:50.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.974+0000 7fd61964c640 1 Processor -- start 2024-09-19T07:57:50.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.974+0000 7fd61964c640 1 -- start start 2024-09-19T07:57:50.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.975+0000 7fd61964c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61406f320 0x7fd61409cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:50.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.975+0000 7fd61964c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6140b4d60 0x7fd61415cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:50.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.975+0000 7fd61964c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd61415a690 0x7fd61415d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:50.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.975+0000 7fd61964c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd61415aa70 con 0x7fd61406f320 2024-09-19T07:57:50.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.975+0000 7fd61964c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd6140b4300 con 0x7fd61415a690 2024-09-19T07:57:50.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.975+0000 7fd61964c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd61415abf0 con 0x7fd6140b4d60 2024-09-19T07:57:50.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.975+0000 7fd612ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61406f320 0x7fd61409cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:50.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.975+0000 7fd612ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61406f320 0x7fd61409cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49200/0 (socket says 172.21.15.16:49200) 2024-09-19T07:57:50.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.975+0000 7fd612ffd640 1 -- 172.21.15.16:0/1794244183 learned_addr learned my addr 172.21.15.16:0/1794244183 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:50.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.976+0000 7fd6127fc640 1 --2- 172.21.15.16:0/1794244183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6140b4d60 0x7fd61415cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:50.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.976+0000 7fd6137fe640 1 --2- 172.21.15.16:0/1794244183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd61415a690 0x7fd61415d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:50.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.976+0000 7fd612ffd640 1 -- 172.21.15.16:0/1794244183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6140b4d60 msgr2=0x7fd61415cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:50.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.976+0000 7fd612ffd640 1 --2- 172.21.15.16:0/1794244183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6140b4d60 0x7fd61415cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:50.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.976+0000 7fd612ffd640 1 -- 172.21.15.16:0/1794244183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd61415a690 msgr2=0x7fd61415d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:50.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.976+0000 7fd612ffd640 1 --2- 172.21.15.16:0/1794244183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd61415a690 0x7fd61415d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:50.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.976+0000 7fd612ffd640 1 -- 172.21.15.16:0/1794244183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6140aeaa0 con 0x7fd61406f320 2024-09-19T07:57:50.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.976+0000 7fd612ffd640 1 --2- 172.21.15.16:0/1794244183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61406f320 0x7fd61409cc30 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fd604035040 tx=0x7fd6040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:50.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.976+0000 7fd5fb7fe640 1 -- 172.21.15.16:0/1794244183 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604037040 con 0x7fd61406f320 2024-09-19T07:57:50.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.976+0000 7fd5fb7fe640 1 -- 172.21.15.16:0/1794244183 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd604030770 con 0x7fd61406f320 2024-09-19T07:57:50.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.976+0000 7fd61964c640 1 -- 172.21.15.16:0/1794244183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6140aed80 con 0x7fd61406f320 2024-09-19T07:57:50.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.976+0000 7fd5fb7fe640 1 -- 172.21.15.16:0/1794244183 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604030b40 con 0x7fd61406f320 2024-09-19T07:57:50.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.976+0000 7fd61964c640 1 -- 172.21.15.16:0/1794244183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6140af340 con 0x7fd61406f320 2024-09-19T07:57:50.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.977+0000 7fd61964c640 1 -- 172.21.15.16:0/1794244183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6140b4820 con 0x7fd61406f320 2024-09-19T07:57:50.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.977+0000 7fd5fb7fe640 1 -- 172.21.15.16:0/1794244183 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd604044020 con 0x7fd61406f320 2024-09-19T07:57:50.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.978+0000 7fd5fb7fe640 1 --2- 172.21.15.16:0/1794244183 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd60006c2b0 0x7fd60006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:50.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.978+0000 7fd5fb7fe640 1 -- 172.21.15.16:0/1794244183 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7fd6040ab940 con 0x7fd61406f320 2024-09-19T07:57:50.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.978+0000 7fd6127fc640 1 --2- 172.21.15.16:0/1794244183 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd60006c2b0 0x7fd60006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:50.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.979+0000 7fd6127fc640 1 --2- 172.21.15.16:0/1794244183 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd60006c2b0 0x7fd60006e770 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd6140b0aa0 tx=0x7fd608007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:50.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:50.982+0000 7fd5fb7fe640 1 -- 172.21.15.16:0/1794244183 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd604032e20 con 0x7fd61406f320 2024-09-19T07:57:51.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.159+0000 7fd61964c640 1 -- 172.21.15.16:0/1794244183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fd61415e110 con 0x7fd61406f320 2024-09-19T07:57:51.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.162+0000 7fd5fb7fe640 1 -- 172.21.15.16:0/1794244183 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v75)=0 v75) ==== 123+0+0 (secure 0 0 0) 0x7fd60407a1f0 con 0x7fd61406f320 2024-09-19T07:57:51.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.162+0000 7fd5fb7fe640 1 -- 172.21.15.16:0/1794244183 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fd604079af0 con 0x7fd61406f320 2024-09-19T07:57:51.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.164+0000 7fd61964c640 1 -- 172.21.15.16:0/1794244183 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd60006c2b0 msgr2=0x7fd60006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:51.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.164+0000 7fd61964c640 1 --2- 172.21.15.16:0/1794244183 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd60006c2b0 0x7fd60006e770 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd6140b0aa0 tx=0x7fd608007b50 comp rx=0 tx=0).stop 2024-09-19T07:57:51.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.164+0000 7fd61964c640 1 -- 172.21.15.16:0/1794244183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61406f320 msgr2=0x7fd61409cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:51.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.164+0000 7fd61964c640 1 --2- 172.21.15.16:0/1794244183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61406f320 0x7fd61409cc30 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fd604035040 tx=0x7fd6040046c0 comp rx=0 tx=0).stop 2024-09-19T07:57:51.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.164+0000 7fd61964c640 1 -- 172.21.15.16:0/1794244183 shutdown_connections 2024-09-19T07:57:51.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.164+0000 7fd61964c640 1 --2- 172.21.15.16:0/1794244183 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7fd60006c2b0 0x7fd60006e770 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:51.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.164+0000 7fd61964c640 1 --2- 172.21.15.16:0/1794244183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd61415a690 0x7fd61415d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:51.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.164+0000 7fd61964c640 1 --2- 172.21.15.16:0/1794244183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6140b4d60 0x7fd61415cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:51.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.164+0000 7fd61964c640 1 --2- 172.21.15.16:0/1794244183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61406f320 0x7fd61409cc30 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:51.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.164+0000 7fd61964c640 1 -- 172.21.15.16:0/1794244183 >> 172.21.15.16:0/1794244183 conn(0x7fd61406bcf0 msgr2=0x7fd61406e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:51.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.164+0000 7fd61964c640 1 -- 172.21.15.16:0/1794244183 shutdown_connections 2024-09-19T07:57:51.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.164+0000 7fd61964c640 1 -- 172.21.15.16:0/1794244183 wait complete. 2024-09-19T07:57:51.175 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T07:57:51.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T07:57:51.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.277+0000 7f3c102a6640 1 Processor -- start 2024-09-19T07:57:51.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.277+0000 7f3c102a6640 1 -- start start 2024-09-19T07:57:51.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.278+0000 7f3c102a6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c08145b30 0x7f3c08072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:51.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.278+0000 7f3c102a6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3c080af600 0x7f3c080ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:51.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.278+0000 7f3c102a6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c08073710 con 0x7f3c080ad350 2024-09-19T07:57:51.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.278+0000 7f3c102a6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f3c0806df30 con 0x7f3c080af600 2024-09-19T07:57:51.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.278+0000 7f3c102a6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3c08145f10 con 0x7f3c08145b30 2024-09-19T07:57:51.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.278+0000 7f3c0e843640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3c080ad350 0x7f3c0815fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39034/0 (socket says 172.21.15.16:39034) 2024-09-19T07:57:51.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.278+0000 7f3c0e843640 1 -- 172.21.15.16:0/1329384053 learned_addr learned my addr 172.21.15.16:0/1329384053 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-09-19T07:57:51.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.278+0000 7f3c0d841640 1 --2- 172.21.15.16:0/1329384053 >> v2:172.21.15.146:3300/0 conn(0x7f3c080af600 0x7f3c080ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:51.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.278+0000 7f3c0e042640 1 --2- 172.21.15.16:0/1329384053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c08145b30 0x7f3c08072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:51.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.278+0000 7f3c0d040640 1 -- 172.21.15.16:0/1329384053 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3924272110 0 0) 0x7f3c08073710 con 0x7f3c080ad350 2024-09-19T07:57:51.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.278+0000 7f3c0d040640 1 -- 172.21.15.16:0/1329384053 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3bf4003520 con 0x7f3c080ad350 2024-09-19T07:57:51.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.278+0000 7f3c0d040640 1 -- 172.21.15.16:0/1329384053 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2317083424 0 0) 0x7f3bf4003520 con 0x7f3c080ad350 2024-09-19T07:57:51.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.278+0000 7f3c0d040640 1 -- 172.21.15.16:0/1329384053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c08145b30 msgr2=0x7f3c08072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:51.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.278+0000 7f3c0d040640 1 --2- 172.21.15.16:0/1329384053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c08145b30 0x7f3c08072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:51.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.278+0000 7f3c0d040640 1 -- 172.21.15.16:0/1329384053 >> v2:172.21.15.146:3300/0 conn(0x7f3c080af600 msgr2=0x7f3c080ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:51.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.278+0000 7f3c0d040640 1 --2- 172.21.15.16:0/1329384053 >> v2:172.21.15.146:3300/0 conn(0x7f3c080af600 0x7f3c080ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:51.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.278+0000 7f3c0d040640 1 -- 172.21.15.16:0/1329384053 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c08072e70 con 0x7f3c080ad350 2024-09-19T07:57:51.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.278+0000 7f3c0d040640 1 -- 172.21.15.16:0/1329384053 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2217618649 0 0) 0x7f3c000019f0 con 0x7f3c080ad350 2024-09-19T07:57:51.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.278+0000 7f3c0d040640 1 -- 172.21.15.16:0/1329384053 >> v1:172.21.15.16:6789/0 conn(0x7f3c080ad350 legacy=0x7f3c0815fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:51.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.279+0000 7f3c0d040640 1 --2- 172.21.15.16:0/1329384053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c08145b30 0x7f3bf40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:51.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.279+0000 7f3c0d040640 1 --2- 172.21.15.16:0/1329384053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bf4003f30 0x7f3bf4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:51.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.279+0000 7f3c0d040640 1 --2- 172.21.15.16:0/1329384053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4004d40 0x7f3bf40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:51.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.279+0000 7f3c0d040640 1 -- 172.21.15.16:0/1329384053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0806df30 con 0x7f3bf4004d40 2024-09-19T07:57:51.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.279+0000 7f3c0d040640 1 -- 172.21.15.16:0/1329384053 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3c08145f10 con 0x7f3bf4003f30 2024-09-19T07:57:51.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.279+0000 7f3c0d040640 1 -- 172.21.15.16:0/1329384053 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf400b9c0 con 0x7f3c08145b30 2024-09-19T07:57:51.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.279+0000 7f3c0e843640 1 --2- 172.21.15.16:0/1329384053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4004d40 0x7f3bf40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:51.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.279+0000 7f3c0e042640 1 --2- 172.21.15.16:0/1329384053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c08145b30 0x7f3bf40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:51.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.279+0000 7f3c0d841640 1 --2- 172.21.15.16:0/1329384053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bf4003f30 0x7f3bf4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:51.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.279+0000 7f3c0e843640 1 -- 172.21.15.16:0/1329384053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c08145b30 msgr2=0x7f3bf40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:51.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.279+0000 7f3c0e843640 1 --2- 172.21.15.16:0/1329384053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c08145b30 0x7f3bf40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:51.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.279+0000 7f3c0e843640 1 -- 172.21.15.16:0/1329384053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bf4003f30 msgr2=0x7f3bf4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:57:51.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.279+0000 7f3c0e843640 1 --2- 172.21.15.16:0/1329384053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bf4003f30 0x7f3bf4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:51.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.279+0000 7f3c0e843640 1 -- 172.21.15.16:0/1329384053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf4003520 con 0x7f3bf4004d40 2024-09-19T07:57:51.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.280+0000 7f3c0e843640 1 --2- 172.21.15.16:0/1329384053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4004d40 0x7f3bf40051c0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f3c000013d0 tx=0x7f3bf4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b768131e45e0eba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:51.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.280+0000 7f3c0d040640 1 -- 172.21.15.16:0/1329384053 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0000a4e0 con 0x7f3bf4004d40 2024-09-19T07:57:51.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.280+0000 7f3c0d040640 1 -- 172.21.15.16:0/1329384053 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3c0002f070 con 0x7f3bf4004d40 2024-09-19T07:57:51.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.280+0000 7f3c0d040640 1 -- 172.21.15.16:0/1329384053 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0002bcf0 con 0x7f3bf4004d40 2024-09-19T07:57:51.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.280+0000 7f3c102a6640 1 -- 172.21.15.16:0/1329384053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4004d40 msgr2=0x7f3bf40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:51.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.280+0000 7f3c102a6640 1 --2- 172.21.15.16:0/1329384053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4004d40 0x7f3bf40051c0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f3c000013d0 tx=0x7f3bf4005830 comp rx=0 tx=0).stop 2024-09-19T07:57:51.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.280+0000 7f3c0e843640 1 -- 172.21.15.16:0/1329384053 reap_dead start 2024-09-19T07:57:51.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.280+0000 7f3c102a6640 1 -- 172.21.15.16:0/1329384053 shutdown_connections 2024-09-19T07:57:51.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.280+0000 7f3c102a6640 1 -- 172.21.15.16:0/1329384053 >> 172.21.15.16:0/1329384053 conn(0x7f3c080b1b30 msgr2=0x7f3c080b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:51.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.281+0000 7f3c102a6640 1 -- 172.21.15.16:0/1329384053 shutdown_connections 2024-09-19T07:57:51.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.281+0000 7f3c102a6640 1 -- 172.21.15.16:0/1329384053 wait complete. 2024-09-19T07:57:51.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.282+0000 7f3c102a6640 1 Processor -- start 2024-09-19T07:57:51.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.282+0000 7f3c102a6640 1 -- start start 2024-09-19T07:57:51.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.282+0000 7f3c102a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4004d40 0x7f3c0815bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:51.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.282+0000 7f3c102a6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c080af9e0 0x7f3c080afe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:51.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.282+0000 7f3c102a6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c080ca580 0x7f3c080caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:51.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.283+0000 7f3c102a6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0806f3d0 con 0x7f3bf4004d40 2024-09-19T07:57:51.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.283+0000 7f3c102a6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3c08073430 con 0x7f3c080ca580 2024-09-19T07:57:51.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.283+0000 7f3c102a6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3c0806f550 con 0x7f3c080af9e0 2024-09-19T07:57:51.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.283+0000 7f3c0e042640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4004d40 0x7f3c0815bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:51.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.283+0000 7f3c0e042640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4004d40 0x7f3c0815bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49222/0 (socket says 172.21.15.16:49222) 2024-09-19T07:57:51.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.283+0000 7f3c0e042640 1 -- 172.21.15.16:0/2907530285 learned_addr learned my addr 172.21.15.16:0/2907530285 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T07:57:51.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.283+0000 7f3c0e042640 1 -- 172.21.15.16:0/2907530285 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c080af9e0 msgr2=0x7f3c080afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:51.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.283+0000 7f3c0e042640 1 --2- 172.21.15.16:0/2907530285 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c080af9e0 0x7f3c080afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:51.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.283+0000 7f3c0e042640 1 -- 172.21.15.16:0/2907530285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c080ca580 msgr2=0x7f3c080caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:51.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.283+0000 7f3c0e042640 1 --2- 172.21.15.16:0/2907530285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c080ca580 0x7f3c080caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:51.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.283+0000 7f3c0e042640 1 -- 172.21.15.16:0/2907530285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c080b3a50 con 0x7f3bf4004d40 2024-09-19T07:57:51.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.283+0000 7f3c0e042640 1 --2- 172.21.15.16:0/2907530285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4004d40 0x7f3c0815bad0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f3c040119a0 tx=0x7f3c04011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:51.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.283+0000 7f3bfeffd640 1 -- 172.21.15.16:0/2907530285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0400fa70 con 0x7f3bf4004d40 2024-09-19T07:57:51.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.283+0000 7f3bfeffd640 1 -- 172.21.15.16:0/2907530285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3c0400fc10 con 0x7f3bf4004d40 2024-09-19T07:57:51.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.283+0000 7f3c102a6640 1 -- 172.21.15.16:0/2907530285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c080b3d30 con 0x7f3bf4004d40 2024-09-19T07:57:51.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.283+0000 7f3bfeffd640 1 -- 172.21.15.16:0/2907530285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04014440 con 0x7f3bf4004d40 2024-09-19T07:57:51.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.283+0000 7f3c102a6640 1 -- 172.21.15.16:0/2907530285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c080b42f0 con 0x7f3bf4004d40 2024-09-19T07:57:51.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.284+0000 7f3c102a6640 1 -- 172.21.15.16:0/2907530285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c080ace10 con 0x7f3bf4004d40 2024-09-19T07:57:51.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.285+0000 7f3bfeffd640 1 -- 172.21.15.16:0/2907530285 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3c04012040 con 0x7f3bf4004d40 2024-09-19T07:57:51.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.285+0000 7f3bfeffd640 1 --2- 172.21.15.16:0/2907530285 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3bec06c2e0 0x7f3bec06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:57:51.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.285+0000 7f3bfeffd640 1 -- 172.21.15.16:0/2907530285 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5378+0+0 (secure 0 0 0) 0x7f3c0408b9c0 con 0x7f3bf4004d40 2024-09-19T07:57:51.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.286+0000 7f3c0d841640 1 --2- 172.21.15.16:0/2907530285 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3bec06c2e0 0x7f3bec06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:57:51.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.286+0000 7f3c0d841640 1 --2- 172.21.15.16:0/2907530285 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3bec06c2e0 0x7f3bec06e7a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8006e80 tx=0x7f3bf8006df0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:57:51.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.290+0000 7f3bfeffd640 1 -- 172.21.15.16:0/2907530285 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3c0408de20 con 0x7f3bf4004d40 2024-09-19T07:57:51.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.502+0000 7f3c102a6640 1 -- 172.21.15.16:0/2907530285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c08073cf0 con 0x7f3bf4004d40 2024-09-19T07:57:51.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.502+0000 7f3bfeffd640 1 -- 172.21.15.16:0/2907530285 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+3911 (secure 0 0 0) 0x7f3c04019030 con 0x7f3bf4004d40 2024-09-19T07:57:51.503 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T07:57:51.503 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":96,"btime":"2024-09-19T07:57:20:688171+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7064,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3873812166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3873812166},{"type":"v1","addr":"172.21.15.146:6837","nonce":3873812166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":7852,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/951273503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":951273503},{"type":"v1","addr":"172.21.15.146:6835","nonce":951273503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":8757,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2338289617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2338289617},{"type":"v1","addr":"172.21.15.16:6834","nonce":2338289617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":9171,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/483610208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":483610208},{"type":"v1","addr":"172.21.15.16:6835","nonce":483610208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[]} 2024-09-19T07:57:51.503 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 96 2024-09-19T07:57:51.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.505+0000 7f3c102a6640 1 -- 172.21.15.16:0/2907530285 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3bec06c2e0 msgr2=0x7f3bec06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:51.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.505+0000 7f3c102a6640 1 --2- 172.21.15.16:0/2907530285 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3bec06c2e0 0x7f3bec06e7a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8006e80 tx=0x7f3bf8006df0 comp rx=0 tx=0).stop 2024-09-19T07:57:51.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.505+0000 7f3c102a6640 1 -- 172.21.15.16:0/2907530285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4004d40 msgr2=0x7f3c0815bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:57:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.505+0000 7f3c102a6640 1 --2- 172.21.15.16:0/2907530285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4004d40 0x7f3c0815bad0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f3c040119a0 tx=0x7f3c04011e60 comp rx=0 tx=0).stop 2024-09-19T07:57:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.505+0000 7f3c102a6640 1 -- 172.21.15.16:0/2907530285 shutdown_connections 2024-09-19T07:57:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.505+0000 7f3c102a6640 1 --2- 172.21.15.16:0/2907530285 >> [v2:172.21.15.146:6832/413611562,v1:172.21.15.146:6833/413611562] conn(0x7f3bec06c2e0 0x7f3bec06e7a0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.505+0000 7f3c102a6640 1 --2- 172.21.15.16:0/2907530285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c080ca580 0x7f3c080caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.505+0000 7f3c102a6640 1 --2- 172.21.15.16:0/2907530285 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c080af9e0 0x7f3c080afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.505+0000 7f3c102a6640 1 --2- 172.21.15.16:0/2907530285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4004d40 0x7f3c0815bad0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:57:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.505+0000 7f3c102a6640 1 -- 172.21.15.16:0/2907530285 >> 172.21.15.16:0/2907530285 conn(0x7f3c080b1b30 msgr2=0x7f3c08072600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:57:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.505+0000 7f3c102a6640 1 -- 172.21.15.16:0/2907530285 shutdown_connections 2024-09-19T07:57:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T07:57:51.505+0000 7f3c102a6640 1 -- 172.21.15.16:0/2907530285 wait complete. 2024-09-19T07:57:51.516 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-19T07:57:51.516 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-19T07:57:51.516 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T07:57:51.517 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:57:51.525 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T07:57:51.525 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T07:57:51.526 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:57:51.575 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T07:57:51.575 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T07:57:51.576 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:57:51.587 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T07:57:51.587 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T07:57:51.587 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:57:51.640 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T07:57:51.640 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-19T07:57:51.640 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-19T07:57:51.640 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:57:51.682 INFO:tasks.ceph.osd.0:Stopped 2024-09-19T07:57:51.682 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-19T07:57:51.682 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:57:51.722 INFO:tasks.ceph.osd.1:Stopped 2024-09-19T07:57:51.722 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-19T07:57:51.723 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:57:51.800 INFO:tasks.ceph.osd.2:Stopped 2024-09-19T07:57:51.800 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-19T07:57:51.801 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:57:51.873 INFO:tasks.ceph.osd.3:Stopped 2024-09-19T07:57:51.873 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-19T07:57:51.873 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:57:51.912 INFO:tasks.ceph.osd.4:Stopped 2024-09-19T07:57:51.912 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-19T07:57:51.912 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:57:51.986 INFO:tasks.ceph.osd.5:Stopped 2024-09-19T07:57:51.987 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-19T07:57:51.987 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:57:52.074 INFO:tasks.ceph.osd.6:Stopped 2024-09-19T07:57:52.074 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-19T07:57:52.075 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:57:52.122 INFO:tasks.ceph.osd.7:Stopped 2024-09-19T07:57:52.122 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-19T07:57:52.123 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-19T07:57:52.123 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:57:52.149 INFO:tasks.ceph.mgr.y:Stopped 2024-09-19T07:57:52.149 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-19T07:57:52.149 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:57:52.182 INFO:tasks.ceph.mgr.x:Stopped 2024-09-19T07:57:52.182 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-19T07:57:52.182 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-19T07:57:52.182 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:57:52.201 INFO:tasks.ceph.mon.a:Stopped 2024-09-19T07:57:52.201 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-19T07:57:52.201 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:57:52.234 INFO:tasks.ceph.mon.b:Stopped 2024-09-19T07:57:52.234 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-19T07:57:52.235 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:57:52.287 INFO:tasks.ceph.mon.c:Stopped 2024-09-19T07:57:52.287 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-19T07:57:52.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'responding to mclientcaps\(revoke\)' | egrep -v 'not advance its oldest_client_tid' | egrep -v 'failing to advance its oldest client/flush tid' | egrep -v 'Too many inodes in cache' | egrep -v 'failing to respond to cache pressure' | egrep -v 'slow requests are blocked' | egrep -v 'failing to respond to capability release' | egrep -v 'MDS cache is too large' | egrep -v '\(MDS_CLIENT_OLDEST_TID\)' | egrep -v '\(MDS_CACHE_OVERSIZED\)' | head -n 1 2024-09-19T07:57:52.326 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:57:52.327 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-19T07:57:52.537 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:57:52.538 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-19T07:57:52.791 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:57:52.791 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-19T07:57:52.867 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:57:52.867 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-19T07:57:52.942 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:57:52.942 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-19T07:57:53.069 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:57:53.069 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-19T07:57:53.180 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:57:53.181 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-19T07:57:53.240 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:57:53.240 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-19T07:57:53.297 INFO:tasks.ceph:Archiving mon data... 2024-09-19T07:57:53.298 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911819/data/mon.a.tgz 2024-09-19T07:57:53.298 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-09-19T07:57:53.315 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.DtVGtRa2iJ 2024-09-19T07:57:53.316 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.DtVGtRa2iJ 2024-09-19T07:57:53.480 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.DtVGtRa2iJ 2024-09-19T07:57:53.564 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.DtVGtRa2iJ is 467KB 2024-09-19T07:57:53.664 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.DtVGtRa2iJ 2024-09-19T07:57:53.680 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911819/data/mon.b.tgz 2024-09-19T07:57:53.680 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2024-09-19T07:57:53.698 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.nZapOhdjb0 2024-09-19T07:57:53.698 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.nZapOhdjb0 2024-09-19T07:57:53.871 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.nZapOhdjb0 2024-09-19T07:57:53.955 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.nZapOhdjb0 is 476KB 2024-09-19T07:57:54.055 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.nZapOhdjb0 2024-09-19T07:57:54.071 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911819/data/mon.c.tgz 2024-09-19T07:57:54.071 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2024-09-19T07:57:54.126 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.ibSQ1lQDps 2024-09-19T07:57:54.126 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ibSQ1lQDps 2024-09-19T07:57:54.278 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.ibSQ1lQDps 2024-09-19T07:57:54.362 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.ibSQ1lQDps is 476KB 2024-09-19T07:57:54.459 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.ibSQ1lQDps 2024-09-19T07:57:54.475 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-19T07:57:54.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-19T07:57:54.477 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-19T07:57:54.566 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-19T07:57:54.595 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-19T07:57:54.595 INFO:tasks.ceph:Archiving crash dumps... 2024-09-19T07:57:54.598 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911819/remote/smithi016/crash 2024-09-19T07:57:54.598 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-19T07:57:54.637 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911819/remote/smithi146/crash 2024-09-19T07:57:54.638 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-19T07:57:54.662 INFO:tasks.ceph:Shutting down logrotate 2024-09-19T07:57:54.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:57:54.682 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T07:57:54.764 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-19T07:57:54.793 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-19T07:57:54.818 INFO:tasks.ceph:Compressing logs... 2024-09-19T07:57:54.818 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T07:57:54.835 DEBUG:teuthology.orchestra.run.smithi146:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T07:57:54.861 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-19T07:57:54.861 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59052.log 2024-09-19T07:57:54.873 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74675.log 2024-09-19T07:57:54.873 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45533.log 2024-09-19T07:57:54.873 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59052.log.gz 2024-09-19T07:57:54.873 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63758.log 2024-09-19T07:57:54.874 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74675.log.gz 2024-09-19T07:57:54.874 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30139.log 2024-09-19T07:57:54.874 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45533.log: /var/log/ceph/ceph-client.admin.63758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44222.log 2024-09-19T07:57:54.874 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45533.log.gz 2024-09-19T07:57:54.874 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63758.log.gz 2024-09-19T07:57:54.874 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.tmp-client.admin.30139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64112.log 2024-09-19T07:57:54.874 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30139.log.gz 2024-09-19T07:57:54.874 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53643.log 2024-09-19T07:57:54.874 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82614.log 2024-09-19T07:57:54.874 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64112.log.gz 2024-09-19T07:57:54.874 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53643.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44222.log.gz 2024-09-19T07:57:54.874 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51409.log 2024-09-19T07:57:54.875 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53643.log.gz 2024-09-19T07:57:54.875 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44012.log 2024-09-19T07:57:54.875 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82614.log: /var/log/ceph/ceph-client.admin.51409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61839.log 2024-09-19T07:57:54.875 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.82614.log.gz 2024-09-19T07:57:54.875 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51409.log.gz 2024-09-19T07:57:54.875 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64364.log 2024-09-19T07:57:54.875 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44012.log: /var/log/ceph/ceph-client.admin.61839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79071.log 2024-09-19T07:57:54.875 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44012.log.gz 2024-09-19T07:57:54.875 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64364.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61839.log.gz 2024-09-19T07:57:54.875 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55869.log 2024-09-19T07:57:54.875 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64364.log.gz 2024-09-19T07:57:54.875 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.79071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82978.log 2024-09-19T07:57:54.875 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79071.log.gz 2024-09-19T07:57:54.876 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74297.log 2024-09-19T07:57:54.876 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50590.log 2024-09-19T07:57:54.876 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82978.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55869.log.gz 2024-09-19T07:57:54.876 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82446.log 2024-09-19T07:57:54.876 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.82978.log.gz 2024-09-19T07:57:54.876 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74297.log.gz 2024-09-19T07:57:54.876 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54631.log 2024-09-19T07:57:54.876 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82446.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50590.log.gz 2024-09-19T07:57:54.876 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71719.log 2024-09-19T07:57:54.876 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.82446.log.gz 2024-09-19T07:57:54.876 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.54631.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.73491.log 2024-09-19T07:57:54.876 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54631.log.gz 2024-09-19T07:57:54.876 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81673.log 2024-09-19T07:57:54.877 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73491.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71719.log.gz 2024-09-19T07:57:54.877 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43396.log 2024-09-19T07:57:54.877 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73491.log.gz 2024-09-19T07:57:54.877 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.81673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56630.log 2024-09-19T07:57:54.877 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.81673.log.gz 2024-09-19T07:57:54.877 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59369.log 2024-09-19T07:57:54.877 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56630.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43396.log.gz 2024-09-19T07:57:54.877 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50137.log 2024-09-19T07:57:54.877 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56630.log.gz 2024-09-19T07:57:54.877 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57088.log 2024-09-19T07:57:54.877 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59369.log.gz 2024-09-19T07:57:54.878 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81318.log 2024-09-19T07:57:54.878 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64970.log 2024-09-19T07:57:54.878 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50137.log.gz 2024-09-19T07:57:54.878 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57088.log.gz 2024-09-19T07:57:54.878 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.81318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65100.log 2024-09-19T07:57:54.878 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81318.log.gz 2024-09-19T07:57:54.878 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79221.log 2024-09-19T07:57:54.878 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65100.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64970.log.gz 2024-09-19T07:57:54.878 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63632.log 2024-09-19T07:57:54.878 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65100.log.gz 2024-09-19T07:57:54.878 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77499.log 2024-09-19T07:57:54.879 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.79221.log: /var/log/ceph/ceph-client.admin.63632.log: gzip -5 89.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.79221.log.gz 2024-09-19T07:57:54.879 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.75663.log 2024-09-19T07:57:54.879 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77499.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63632.log.gz 2024-09-19T07:57:54.879 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62125.log 2024-09-19T07:57:54.879 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77499.log.gz 2024-09-19T07:57:54.879 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57334.log 2024-09-19T07:57:54.879 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75663.log.gz 2024-09-19T07:57:54.880 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65310.log 2024-09-19T07:57:54.918 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.43443.log 2024-09-19T07:57:54.918 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.50378.log 2024-09-19T07:57:54.918 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.3.43443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36415.log 2024-09-19T07:57:54.918 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.50378.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-19T07:57:54.918 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36415.log: 87.4% -- replaced with /var/log/ceph/ceph-client.3.43443.log.gz 2024-09-19T07:57:54.919 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36495.log 2024-09-19T07:57:54.919 INFO:teuthology.orchestra.run.smithi146.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36415.log.gz 2024-09-19T07:57:54.919 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45545.log 2024-09-19T07:57:54.919 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44802.log 2024-09-19T07:57:54.919 INFO:teuthology.orchestra.run.smithi146.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36495.log.gz 2024-09-19T07:57:54.919 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.45545.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-19T07:57:54.919 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36579.log 2024-09-19T07:57:54.919 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.44802.log: /var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46263.log 2024-09-19T07:57:54.919 INFO:teuthology.orchestra.run.smithi146.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.1.45545.log.gz 2024-09-19T07:57:54.919 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36535.log 2024-09-19T07:57:54.919 INFO:teuthology.orchestra.run.smithi146.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.1.50378.log.gz 2024-09-19T07:57:54.919 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.46263.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-19T07:57:54.919 INFO:teuthology.orchestra.run.smithi146.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36579.log.gz 2024-09-19T07:57:54.920 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36535.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-19T07:57:54.920 INFO:teuthology.orchestra.run.smithi146.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36535.log.gz 2024-09-19T07:57:54.920 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.49038.log 2024-09-19T07:57:54.920 INFO:teuthology.orchestra.run.smithi146.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.1.46263.log.gz 2024-09-19T07:57:54.920 INFO:teuthology.orchestra.run.smithi146.stderr:gzip/var/log/ceph/ceph.audit.log: -5 --verbose -- /var/log/ceph/ceph-client.1.47679.log 2024-09-19T07:57:54.920 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.49038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44154.log 2024-09-19T07:57:54.920 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.47679.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-19T07:57:54.920 INFO:teuthology.orchestra.run.smithi146.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.1.49038.log.gz 2024-09-19T07:57:54.921 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.44154.log: gzip 95.7% -- replaced with /var/log/ceph/ceph-client.1.44802.log.gz 2024-09-19T07:57:54.921 INFO:teuthology.orchestra.run.smithi146.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.1.47679.log.gz 2024-09-19T07:57:54.921 INFO:teuthology.orchestra.run.smithi146.stderr: -5 --verbose -- /var/log/ceph/ceph-client.1.46954.log 2024-09-19T07:57:54.921 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36666.log 2024-09-19T07:57:54.921 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.46954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36626.log 2024-09-19T07:57:54.921 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36666.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36666.log.gz 2024-09-19T07:57:54.922 INFO:teuthology.orchestra.run.smithi146.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.1.44154.log.gz 2024-09-19T07:57:54.922 INFO:teuthology.orchestra.run.smithi146.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-19T07:57:54.922 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43005.log 2024-09-19T07:57:54.922 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-19T07:57:54.922 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.43005.log: /var/log/ceph/ceph-client.admin.36626.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-19T07:57:54.922 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.6.log: 87.8% -- replaced with /var/log/ceph/ceph-client.1.43005.log.gz 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36626.log.gz 2024-09-19T07:57:54.922 INFO:teuthology.orchestra.run.smithi146.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-19T07:57:54.922 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-09-19T07:57:54.922 INFO:teuthology.orchestra.run.smithi146.stderr: 95.3% -- replaced with /var/log/ceph/ceph-client.1.46954.log.gz 2024-09-19T07:57:54.923 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-19T07:57:54.923 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-19T07:57:54.923 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-19T07:57:54.923 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36375.log 2024-09-19T07:57:54.923 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mgr.x.log: 92.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-19T07:57:54.923 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36455.log 2024-09-19T07:57:54.923 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36375.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36375.log.gz 2024-09-19T07:57:54.923 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-19T07:57:54.923 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.36455.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36455.log.gz 2024-09-19T07:57:54.923 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.48393.log 2024-09-19T07:57:54.926 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.49748.log 2024-09-19T07:57:54.926 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.48393.log: 89.6% -- replaced with /var/log/ceph/ceph-client.1.48393.log.gz 2024-09-19T07:57:54.943 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.49748.log: 89.2% -- replaced with /var/log/ceph/ceph-client.1.49748.log.gz 2024-09-19T07:57:55.028 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5/var/log/ceph/ceph-client.admin.57334.log: --verbose -- /var/log/ceph/ceph-client.admin.56993.log 2024-09-19T07:57:55.028 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62125.log.gz 2024-09-19T07:57:55.029 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57334.log.gz 2024-09-19T07:57:55.029 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75239.log 2024-09-19T07:57:55.029 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65310.log.gz 2024-09-19T07:57:55.029 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47988.log 2024-09-19T07:57:55.029 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75239.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56993.log.gz 2024-09-19T07:57:55.029 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78298.log 2024-09-19T07:57:55.029 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75239.log.gz 2024-09-19T07:57:55.029 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50460.log 2024-09-19T07:57:55.029 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47988.log.gz 2024-09-19T07:57:55.029 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.78298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45381.log 2024-09-19T07:57:55.029 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50460.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.78298.log.gz 2024-09-19T07:57:55.029 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58894.log 2024-09-19T07:57:55.029 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50460.log.gz 2024-09-19T07:57:55.030 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80811.log 2024-09-19T07:57:55.030 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45381.log.gz 2024-09-19T07:57:55.030 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53312.log 2024-09-19T07:57:55.030 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.80811.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58894.log.gz 2024-09-19T07:57:55.030 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61713.log 2024-09-19T07:57:55.030 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80811.log.gz 2024-09-19T07:57:55.030 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78159.log 2024-09-19T07:57:55.030 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76332.log 2024-09-19T07:57:55.030 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61713.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53312.log.gz 2024-09-19T07:57:55.030 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.78159.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61713.log.gz 2024-09-19T07:57:55.030 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70001.log 2024-09-19T07:57:55.030 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78159.log.gz 2024-09-19T07:57:55.030 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79155.log 2024-09-19T07:57:55.031 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76332.log.gz 2024-09-19T07:57:55.031 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59579.log 2024-09-19T07:57:55.031 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.79155.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70001.log.gz 2024-09-19T07:57:55.031 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50279.log 2024-09-19T07:57:55.031 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49355.log 2024-09-19T07:57:55.031 INFO:teuthology.orchestra.run.smithi016.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.79155.log.gz 2024-09-19T07:57:55.031 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50279.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59579.log.gz 2024-09-19T07:57:55.031 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81444.log 2024-09-19T07:57:55.031 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50279.log.gz 2024-09-19T07:57:55.031 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46603.log 2024-09-19T07:57:55.031 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49355.log: /var/log/ceph/ceph-client.admin.81444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73086.log 2024-09-19T07:57:55.031 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49355.log.gz 2024-09-19T07:57:55.031 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.81444.log.gz 2024-09-19T07:57:55.031 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60415.log/var/log/ceph/ceph-client.admin.46603.log: 2024-09-19T07:57:55.032 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46603.log.gz/var/log/ceph/ceph-client.admin.73086.log: 2024-09-19T07:57:55.032 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60029.log 2024-09-19T07:57:55.032 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60415.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73086.log.gz 2024-09-19T07:57:55.032 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61950.log 2024-09-19T07:57:55.032 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60415.log.gz 2024-09-19T07:57:55.032 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68930.log 2024-09-19T07:57:55.032 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60029.log.gz 2024-09-19T07:57:55.032 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75323.log 2024-09-19T07:57:55.032 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68930.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61950.log.gz 2024-09-19T07:57:55.032 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74507.log 2024-09-19T07:57:55.032 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68930.log.gz 2024-09-19T07:57:55.032 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42699.log 2024-09-19T07:57:55.032 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75323.log.gz 2024-09-19T07:57:55.033 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64623.log 2024-09-19T07:57:55.033 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42699.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74507.log.gz 2024-09-19T07:57:55.033 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74465.log 2024-09-19T07:57:55.033 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42699.log.gz 2024-09-19T07:57:55.033 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61170.log 2024-09-19T07:57:55.033 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64623.log.gz/var/log/ceph/ceph-client.admin.74465.log: 2024-09-19T07:57:55.033 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44096.log 2024-09-19T07:57:55.033 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61170.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74465.log.gz 2024-09-19T07:57:55.033 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43055.log 2024-09-19T07:57:55.033 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61170.log.gz 2024-09-19T07:57:55.033 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63674.log 2024-09-19T07:57:55.033 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44096.log: /var/log/ceph/ceph-client.admin.43055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42656.log 2024-09-19T07:57:55.033 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44096.log.gz 2024-09-19T07:57:55.033 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43055.log.gz 2024-09-19T07:57:55.034 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61587.log 2024-09-19T07:57:55.034 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63674.log.gz 2024-09-19T07:57:55.034 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50758.log 2024-09-19T07:57:55.034 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42656.log.gz 2024-09-19T07:57:55.034 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49794.log 2024-09-19T07:57:55.034 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50758.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61587.log.gz 2024-09-19T07:57:55.034 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60204.log 2024-09-19T07:57:55.034 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50758.log.gz 2024-09-19T07:57:55.035 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42741.log 2024-09-19T07:57:55.035 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49794.log.gz 2024-09-19T07:57:55.035 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70896.log 2024-09-19T07:57:55.035 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42741.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60204.log.gz 2024-09-19T07:57:55.035 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78512.log 2024-09-19T07:57:55.035 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42741.log.gz 2024-09-19T07:57:55.035 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68576.log 2024-09-19T07:57:55.036 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70896.log.gz/var/log/ceph/ceph-client.admin.78512.log: 2024-09-19T07:57:55.036 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46645.log 2024-09-19T07:57:55.036 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68576.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.78512.log.gz 2024-09-19T07:57:55.036 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-19T07:57:55.036 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68576.log.gz 2024-09-19T07:57:55.036 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55975.log 2024-09-19T07:57:55.036 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46645.log.gz/var/log/ceph/ceph-osd.1.log: 2024-09-19T07:57:55.036 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59094.log 2024-09-19T07:57:55.036 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64322.log 2024-09-19T07:57:55.036 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55975.log.gz 2024-09-19T07:57:55.036 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80541.log 2024-09-19T07:57:55.036 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59094.log.gz 2024-09-19T07:57:55.037 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80685.log 2024-09-19T07:57:55.037 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64322.log.gz 2024-09-19T07:57:55.037 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64070.log 2024-09-19T07:57:55.037 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.80541.log: /var/log/ceph/ceph-client.admin.80685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58285.log 2024-09-19T07:57:55.037 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80541.log.gz 2024-09-19T07:57:55.037 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80685.log.gz 2024-09-19T07:57:55.037 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46851.log 2024-09-19T07:57:55.037 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64543.log 2024-09-19T07:57:55.037 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58285.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64070.log.gz 2024-09-19T07:57:55.037 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46851.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58285.log.gz 2024-09-19T07:57:55.037 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36182.log 2024-09-19T07:57:55.038 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46851.log.gz 2024-09-19T07:57:55.038 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79481.log 2024-09-19T07:57:55.038 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64543.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64543.log.gz 2024-09-19T07:57:55.038 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47760.log 2024-09-19T07:57:55.038 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49752.log 2024-09-19T07:57:55.038 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.79481.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36182.log.gz 2024-09-19T07:57:55.038 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73535.log 2024-09-19T07:57:55.038 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47760.log.gz 2024-09-19T07:57:55.038 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49752.log: 88.6% -- replaced with /var/log/ceph/ceph-client.0.79481.log.gz 2024-09-19T07:57:55.038 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63800.log 2024-09-19T07:57:55.038 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59789.log 2024-09-19T07:57:55.038 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49752.log.gz 2024-09-19T07:57:55.038 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63800.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73535.log.gz 2024-09-19T07:57:55.038 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36446.log 2024-09-19T07:57:55.039 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63800.log.gz 2024-09-19T07:57:55.039 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71124.log 2024-09-19T07:57:55.039 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59789.log.gz 2024-09-19T07:57:55.039 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59705.log 2024-09-19T07:57:55.039 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36446.log: /var/log/ceph/ceph-client.admin.71124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76416.log 2024-09-19T07:57:55.039 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36446.log.gz 2024-09-19T07:57:55.039 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71124.log.gz 2024-09-19T07:57:55.039 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47325.log 2024-09-19T07:57:55.039 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71544.log 2024-09-19T07:57:55.039 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59705.log.gz 2024-09-19T07:57:55.039 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76416.log: /var/log/ceph/ceph-client.admin.47325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50884.log 2024-09-19T07:57:55.039 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76416.log.gz 2024-09-19T07:57:55.039 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47325.log.gz 2024-09-19T07:57:55.040 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72015.log 2024-09-19T07:57:55.040 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71544.log: /var/log/ceph/ceph-client.admin.50884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45241.log 2024-09-19T07:57:55.040 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71544.log.gz 2024-09-19T07:57:55.040 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36545.log 2024-09-19T07:57:55.040 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72015.log.gz 2024-09-19T07:57:55.040 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.2.45241.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50884.log.gz 2024-09-19T07:57:55.040 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66021.log 2024-09-19T07:57:55.040 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49836.log 2024-09-19T07:57:55.040 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36545.log: 89.0% -- replaced with /var/log/ceph/ceph-client.2.45241.log.gz 2024-09-19T07:57:55.040 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43051.log 87.9% 2024-09-19T07:57:55.040 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36545.log.gz 2024-09-19T07:57:55.040 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66021.log.gz 2024-09-19T07:57:55.040 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76584.log 2024-09-19T07:57:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49836.log: /var/log/ceph/ceph-client.admin.43051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70668.log 2024-09-19T07:57:55.041 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49836.log.gz 2024-09-19T07:57:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76584.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43051.log.gz 2024-09-19T07:57:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77416.log 2024-09-19T07:57:55.041 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76584.log.gz 2024-09-19T07:57:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53559.log 2024-09-19T07:57:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70668.log: /var/log/ceph/ceph-client.admin.77416.log: gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.50716.log 2024-09-19T07:57:55.041 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70668.log.gz 2024-09-19T07:57:55.041 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77416.log.gz 2024-09-19T07:57:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42124.log 2024-09-19T07:57:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53559.log: /var/log/ceph/ceph-client.admin.50716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76626.log 2024-09-19T07:57:55.041 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53559.log.gz 2024-09-19T07:57:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42124.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50716.log.gz 2024-09-19T07:57:55.042 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78064.log 2024-09-19T07:57:55.042 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68620.log 2024-09-19T07:57:55.042 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76626.log.gz 2024-09-19T07:57:55.042 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.78064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69308.log 2024-09-19T07:57:55.042 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68620.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78064.log.gz 2024-09-19T07:57:55.042 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75188.log 2024-09-19T07:57:55.042 INFO:teuthology.orchestra.run.smithi016.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.42124.log.gz 2024-09-19T07:57:55.042 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68620.log.gz 2024-09-19T07:57:55.042 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51960.log 2024-09-19T07:57:55.042 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72435.log 2024-09-19T07:57:55.042 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75188.log: /var/log/ceph/ceph-client.admin.51960.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.69308.log.gz 2024-09-19T07:57:55.042 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75188.log.gz 2024-09-19T07:57:55.042 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43732.log 2024-09-19T07:57:55.043 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51960.log.gz 2024-09-19T07:57:55.043 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72225.log 2024-09-19T07:57:55.043 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72435.log.gz 2024-09-19T07:57:55.043 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72183.log 2024-09-19T07:57:55.043 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43732.log: /var/log/ceph/ceph-client.admin.72225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61020.log 2024-09-19T07:57:55.043 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43732.log.gz 2024-09-19T07:57:55.043 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72183.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72225.log.gz 2024-09-19T07:57:55.043 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59915.log 2024-09-19T07:57:55.043 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose 87.8% -- /var/log/ceph/ceph-client.admin.45880.log -- replaced with /var/log/ceph/ceph-client.admin.72183.log.gz 2024-09-19T07:57:55.043 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-19T07:57:55.043 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61020.log: /var/log/ceph/ceph-client.admin.59915.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61020.log.gz 2024-09-19T07:57:55.043 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62544.log 2024-09-19T07:57:55.043 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59915.log.gz 2024-09-19T07:57:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78382.log 2024-09-19T07:57:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45880.log: /var/log/ceph/ceph-client.admin.62544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72561.log 2024-09-19T07:57:55.044 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45880.log.gz 2024-09-19T07:57:55.044 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62544.log.gz 2024-09-19T07:57:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.78382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61104.log 2024-09-19T07:57:55.044 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.78382.log.gz 2024-09-19T07:57:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59327.log 2024-09-19T07:57:55.044 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72561.log.gz 2024-09-19T07:57:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71250.log 2024-09-19T07:57:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61104.log: /var/log/ceph/ceph-client.admin.59327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73767.log 2024-09-19T07:57:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71250.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59327.log.gz 2024-09-19T07:57:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50365.log 2024-09-19T07:57:55.044 INFO:teuthology.orchestra.run.smithi016.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.61104.log.gz 2024-09-19T07:57:55.045 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71250.log.gz 2024-09-19T07:57:55.045 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46561.log 2024-09-19T07:57:55.045 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73767.log: /var/log/ceph/ceph-client.admin.50365.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73767.log.gzgzip 2024-09-19T07:57:55.045 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50409.log 2024-09-19T07:57:55.045 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50365.log.gz 2024-09-19T07:57:55.045 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62751.log 2024-09-19T07:57:55.045 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46561.log: /var/log/ceph/ceph-client.admin.50409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62191.log 2024-09-19T07:57:55.045 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46561.log.gz 2024-09-19T07:57:55.045 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50409.log.gz 2024-09-19T07:57:55.046 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63515.log 2024-09-19T07:57:55.046 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82320.log 2024-09-19T07:57:55.046 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62751.log.gz 2024-09-19T07:57:55.046 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62191.log: /var/log/ceph/ceph-client.admin.63515.log: gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62191.log.gz 2024-09-19T07:57:55.046 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.72519.log 2024-09-19T07:57:55.046 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63515.log.gz 2024-09-19T07:57:55.046 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64154.log 2024-09-19T07:57:55.046 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82320.log: /var/log/ceph/ceph-client.admin.72519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60767.log 2024-09-19T07:57:55.046 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82320.log.gz 2024-09-19T07:57:55.046 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72519.log.gz 2024-09-19T07:57:55.046 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47239.log 2024-09-19T07:57:55.047 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64154.log.gz 2024-09-19T07:57:55.047 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74934.log 2024-09-19T07:57:55.047 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60767.log: /var/log/ceph/ceph-client.admin.47239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57949.log 2024-09-19T07:57:55.047 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60767.log.gz 2024-09-19T07:57:55.047 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74934.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47239.log.gz 2024-09-19T07:57:55.047 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47676.log 2024-09-19T07:57:55.047 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.74934.log.gz 2024-09-19T07:57:55.047 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74088.log 2024-09-19T07:57:55.047 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57949.log.gz 2024-09-19T07:57:55.048 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49533.log 2024-09-19T07:57:55.048 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47676.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.74088.log: /var/log/ceph/ceph-client.admin.75621.log 2024-09-19T07:57:55.048 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47676.log.gz 2024-09-19T07:57:55.048 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49533.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74088.log.gz 2024-09-19T07:57:55.048 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69634.log 2024-09-19T07:57:55.048 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49533.log.gz 2024-09-19T07:57:55.048 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70266.log 2024-09-19T07:57:55.048 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75621.log: /var/log/ceph/ceph-client.0.69634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43395.log 2024-09-19T07:57:55.048 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75621.log.gz 2024-09-19T07:57:55.048 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52130.log 2024-09-19T07:57:55.048 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70266.log.gz 2024-09-19T07:57:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53769.log 2024-09-19T07:57:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52130.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43395.log.gz 2024-09-19T07:57:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78945.log 2024-09-19T07:57:55.049 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52130.log.gz 2024-09-19T07:57:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53853.log 2024-09-19T07:57:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.78945.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53769.log.gz 2024-09-19T07:57:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68715.log 2024-09-19T07:57:55.049 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78945.log.gz 2024-09-19T07:57:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50057.log 2024-09-19T07:57:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68715.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53853.log.gz 2024-09-19T07:57:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71292.log 2024-09-19T07:57:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50057.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68715.log.gz 2024-09-19T07:57:55.050 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42369.log 2024-09-19T07:57:55.050 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50057.log.gz 2024-09-19T07:57:55.050 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74130.log 2024-09-19T07:57:55.050 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48555.log 2024-09-19T07:57:55.050 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71292.log.gz 2024-09-19T07:57:55.050 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42369.log: /var/log/ceph/ceph-client.admin.74130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44054.log 2024-09-19T07:57:55.050 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42369.log.gz 2024-09-19T07:57:55.050 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74130.log.gz 2024-09-19T07:57:55.050 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63308.log 2024-09-19T07:57:55.050 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44054.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48555.log.gz 2024-09-19T07:57:55.051 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54235.log 2024-09-19T07:57:55.051 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63308.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44054.log.gz 2024-09-19T07:57:55.051 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43059.log 2024-09-19T07:57:55.051 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54235.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63308.log.gz 2024-09-19T07:57:55.051 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54459.log 2024-09-19T07:57:55.051 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54235.log.gz 2024-09-19T07:57:55.051 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78563.log 2024-09-19T07:57:55.051 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77549.log 2024-09-19T07:57:55.051 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43059.log.gz 2024-09-19T07:57:55.051 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54459.log: /var/log/ceph/ceph-client.admin.78563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71884.log 2024-09-19T07:57:55.051 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54459.log.gz 2024-09-19T07:57:55.051 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77549.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.78563.log.gz -5 2024-09-19T07:57:55.051 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.82362.log 2024-09-19T07:57:55.051 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77549.log.gz 2024-09-19T07:57:55.052 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62846.log 2024-09-19T07:57:55.052 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75579.log 2024-09-19T07:57:55.052 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82362.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.71884.log.gz 2024-09-19T07:57:55.052 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41517.log 2024-09-19T07:57:55.052 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82362.log.gz 2024-09-19T07:57:55.052 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.75579.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56825.log 2024-09-19T07:57:55.052 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62846.log.gz 2024-09-19T07:57:55.052 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41517.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75579.log.gz 2024-09-19T07:57:55.052 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51210.log 2024-09-19T07:57:55.052 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-client.admin.61797.log 2024-09-19T07:57:55.052 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41517.log.gz 2024-09-19T07:57:55.052 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56825.log: /var/log/ceph/ceph-client.0.51210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51544.log 2024-09-19T07:57:55.053 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56825.log.gz 2024-09-19T07:57:55.053 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54967.log 2024-09-19T07:57:55.053 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61797.log: /var/log/ceph/ceph-client.admin.51544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77374.log 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61797.log.gz 2024-09-19T07:57:55.053 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-19T07:57:55.053 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.0.51210.log.gz 2024-09-19T07:57:55.053 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51544.log.gz 2024-09-19T07:57:55.053 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41391.log 2024-09-19T07:57:55.053 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54967.log.gz 2024-09-19T07:57:55.053 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70414.log 2024-09-19T07:57:55.053 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77374.log: /var/log/ceph/ceph-client.admin.41391.log: gzip 87.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.77374.log.gz -- 2024-09-19T07:57:55.053 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.62238.log 2024-09-19T07:57:55.054 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41391.log.gz/var/log/ceph/ceph-client.admin.70414.log: 2024-09-19T07:57:55.054 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42311.log 2024-09-19T07:57:55.054 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62238.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70414.log.gz 2024-09-19T07:57:55.054 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44138.log 2024-09-19T07:57:55.054 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62238.log.gz 2024-09-19T07:57:55.054 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50632.log 2024-09-19T07:57:55.054 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81630.log 2024-09-19T07:57:55.054 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42311.log.gz 2024-09-19T07:57:55.054 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44138.log: /var/log/ceph/ceph-client.admin.50632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52172.log 2024-09-19T07:57:55.054 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44138.log.gz 2024-09-19T07:57:55.054 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50632.log.gz 2024-09-19T07:57:55.054 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.81630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59831.log 2024-09-19T07:57:55.054 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73962.log 2024-09-19T07:57:55.055 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52172.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81630.log.gz 2024-09-19T07:57:55.055 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59831.log: gzip 87.6% -5 --verbose -- /var/log/ceph/ceph-client.admin.60511.log 2024-09-19T07:57:55.055 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52172.log.gz 2024-09-19T07:57:55.055 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59831.log.gz 2024-09-19T07:57:55.055 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70812.log 2024-09-19T07:57:55.055 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73962.log.gz 2024-09-19T07:57:55.055 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43970.log 2024-09-19T07:57:55.055 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60511.log: /var/log/ceph/ceph-client.admin.70812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60978.log 2024-09-19T07:57:55.055 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60511.log.gz 2024-09-19T07:57:55.055 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70812.log.gz 2024-09-19T07:57:55.055 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43970.log: /var/log/ceph/ceph-client.admin.63883.log 2024-09-19T07:57:55.055 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58852.log 2024-09-19T07:57:55.055 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60978.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43970.log.gz 2024-09-19T07:57:55.056 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80457.log 2024-09-19T07:57:55.056 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60978.log.gz 2024-09-19T07:57:55.056 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58852.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63883.log.gz 2024-09-19T07:57:55.056 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68091.log 2024-09-19T07:57:55.056 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.80457.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58852.log.gz 2024-09-19T07:57:55.056 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49076.log 2024-09-19T07:57:55.056 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0%/var/log/ceph/ceph-client.admin.68091.log: -- replaced with /var/log/ceph/ceph-client.admin.80457.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46931.log 2024-09-19T07:57:55.056 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-19T07:57:55.056 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49076.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68091.log.gz 2024-09-19T07:57:55.057 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47197.log 2024-09-19T07:57:55.057 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49076.log.gz 2024-09-19T07:57:55.057 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54673.log 2024-09-19T07:57:55.057 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47197.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46931.log.gz 2024-09-19T07:57:55.057 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60894.log 2024-09-19T07:57:55.057 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47197.log.gz 2024-09-19T07:57:55.057 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46309.log 2024-09-19T07:57:55.057 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82488.log 2024-09-19T07:57:55.057 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54673.log.gz/var/log/ceph/ceph-client.admin.60894.log: 2024-09-19T07:57:55.057 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43054.log 2024-09-19T07:57:55.057 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60894.log.gz 2024-09-19T07:57:55.057 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46309.log.gz 2024-09-19T07:57:55.057 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67895.log 2024-09-19T07:57:55.058 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43054.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62497.log 2024-09-19T07:57:55.058 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.82488.log.gz 2024-09-19T07:57:55.058 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43054.log.gz 2024-09-19T07:57:55.058 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71842.log 2024-09-19T07:57:55.058 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62497.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67895.log.gz 2024-09-19T07:57:55.058 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41643.log 2024-09-19T07:57:55.058 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61062.log 2024-09-19T07:57:55.058 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71842.log: -- replaced with /var/log/ceph/ceph-client.admin.62497.log.gz 2024-09-19T07:57:55.058 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41643.log: gzip 88.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.71842.log.gz -- 2024-09-19T07:57:55.058 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.47016.log 2024-09-19T07:57:55.058 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68392.log 2024-09-19T07:57:55.058 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61062.log: /var/log/ceph/ceph-client.admin.47016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59537.log 2024-09-19T07:57:55.058 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61062.log.gz 2024-09-19T07:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.41643.log.gz 2024-09-19T07:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56867.log 2024-09-19T07:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47016.log.gz 2024-09-19T07:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59537.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68392.log.gz 2024-09-19T07:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50321.log 2024-09-19T07:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59537.log.gz 2024-09-19T07:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71800.log 2024-09-19T07:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76954.log 2024-09-19T07:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56867.log.gz 2024-09-19T07:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.71800.log: /var/log/ceph/ceph-client.admin.75100.log 2024-09-19T07:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50321.log.gz 2024-09-19T07:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76954.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71800.log.gz 2024-09-19T07:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64448.log 2024-09-19T07:57:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71082.log 2024-09-19T07:57:55.060 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.76954.log.gz 2024-09-19T07:57:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75100.log: /var/log/ceph/ceph-client.admin.64448.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75100.log.gz 2024-09-19T07:57:55.060 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.82152.log 2024-09-19T07:57:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71082.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64448.log.gz 2024-09-19T07:57:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68664.log 2024-09-19T07:57:55.060 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71082.log.gz 2024-09-19T07:57:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60725.log 2024-09-19T07:57:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82152.log: /var/log/ceph/ceph-client.admin.68664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48096.log 2024-09-19T07:57:55.060 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82152.log.gz 2024-09-19T07:57:55.060 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68664.log.gz 2024-09-19T07:57:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65604.log 2024-09-19T07:57:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73170.log 2024-09-19T07:57:55.061 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60725.log.gz 2024-09-19T07:57:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48096.log: /var/log/ceph/ceph-client.admin.65604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75281.log 2024-09-19T07:57:55.061 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48096.log.gz 2024-09-19T07:57:55.061 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65604.log.gz 2024-09-19T07:57:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65496.log 2024-09-19T07:57:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49584.log 2024-09-19T07:57:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75281.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73170.log.gz 2024-09-19T07:57:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82068.log 2024-09-19T07:57:55.061 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75281.log.gz 2024-09-19T07:57:55.061 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65496.log.gz 2024-09-19T07:57:55.061 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53811.log 2024-09-19T07:57:55.061 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49584.log.gz 2024-09-19T07:57:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log/var/log/ceph/ceph-client.admin.82068.log: 2024-09-19T07:57:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68799.log 2024-09-19T07:57:55.062 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.82068.log.gz 2024-09-19T07:57:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56708.log 2024-09-19T07:57:55.062 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53811.log.gz 2024-09-19T07:57:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74854.log 2024-09-19T07:57:55.063 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68799.log.gz 2024-09-19T07:57:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77644.log 2024-09-19T07:57:55.063 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56708.log.gz 2024-09-19T07:57:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54715.log 2024-09-19T07:57:55.063 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74854.log.gz 2024-09-19T07:57:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77602.log 2024-09-19T07:57:55.064 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54715.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77644.log.gz 2024-09-19T07:57:55.064 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68888.log 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54715.log.gz 2024-09-19T07:57:55.064 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-19T07:57:55.064 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77770.log 2024-09-19T07:57:55.064 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77602.log.gz 2024-09-19T07:57:55.065 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68888.log: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-19T07:57:55.065 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41159.log 2024-09-19T07:57:55.065 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68888.log.gz 2024-09-19T07:57:55.065 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49626.log 2024-09-19T07:57:55.065 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77770.log: /var/log/ceph/ceph-client.admin.41159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73380.log 2024-09-19T07:57:55.065 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77770.log.gz 2024-09-19T07:57:55.066 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41159.log.gz/var/log/ceph/ceph-client.admin.49626.log: 2024-09-19T07:57:55.066 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80061.log 2024-09-19T07:57:55.066 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3%/var/log/ceph/ceph-client.admin.73380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68217.log 2024-09-19T07:57:55.066 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49626.log.gz 2024-09-19T07:57:55.066 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73380.log.gz 2024-09-19T07:57:55.066 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.80061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67937.log 2024-09-19T07:57:55.066 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80415.log 2024-09-19T07:57:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68217.log: /var/log/ceph/ceph-client.admin.67937.log: gzip -5 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68217.log.gz 2024-09-19T07:57:55.067 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.62686.log 2024-09-19T07:57:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.80415.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67937.log.gz 2024-09-19T07:57:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54277.log 2024-09-19T07:57:55.067 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80415.log.gz 2024-09-19T07:57:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.62686.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.64406.log 2024-09-19T07:57:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54277.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62686.log.gz 2024-09-19T07:57:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54277.log.gz 2024-09-19T07:57:55.068 INFO:teuthology.orchestra.run.smithi016.stderr: 94.3% --verbose -- /var/log/ceph/ceph-client.admin.57180.log 2024-09-19T07:57:55.068 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.0.80061.log.gz 2024-09-19T07:57:55.068 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70938.log 2024-09-19T07:57:55.068 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64406.log.gz 2024-09-19T07:57:55.068 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58327.log 2024-09-19T07:57:55.068 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57180.log.gz 2024-09-19T07:57:55.068 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74717.log 2024-09-19T07:57:55.068 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71031.log 2024-09-19T07:57:55.068 INFO:teuthology.orchestra.run.smithi016.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.70938.log.gz 2024-09-19T07:57:55.068 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58327.log: /var/log/ceph/ceph-client.admin.74717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61755.log 2024-09-19T07:57:55.068 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58327.log.gz 2024-09-19T07:57:55.068 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74717.log.gz 2024-09-19T07:57:55.068 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64833.log 2024-09-19T07:57:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61755.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71031.log.gz 2024-09-19T07:57:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42835.log 2024-09-19T07:57:55.069 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61755.log.gz 2024-09-19T07:57:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75849.log 2024-09-19T07:57:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71502.log 2024-09-19T07:57:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42835.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64833.log.gz 2024-09-19T07:57:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54543.log 2024-09-19T07:57:55.069 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42835.log.gz 2024-09-19T07:57:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57635.log 2024-09-19T07:57:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64705.log 2024-09-19T07:57:55.069 INFO:teuthology.orchestra.run.smithi016.stderr: 90.9% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75849.log.gz 2024-09-19T07:57:55.069 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71502.log.gz 2024-09-19T07:57:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48849.log 2024-09-19T07:57:55.070 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54543.log.gz 2024-09-19T07:57:55.070 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64705.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57635.log.gz 2024-09-19T07:57:55.070 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68007.log 2024-09-19T07:57:55.070 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58201.log 2024-09-19T07:57:55.070 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64705.log.gz 2024-09-19T07:57:55.070 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81715.log 2024-09-19T07:57:55.070 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48849.log.gz 2024-09-19T07:57:55.070 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41031.log 2024-09-19T07:57:55.070 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68007.log.gz 2024-09-19T07:57:55.070 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58201.log: /var/log/ceph/ceph-client.admin.81715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75723.log 2024-09-19T07:57:55.070 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58201.log.gz 2024-09-19T07:57:55.070 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.81715.log.gz 2024-09-19T07:57:55.070 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75016.log 2024-09-19T07:57:55.071 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69791.log 2024-09-19T07:57:55.071 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41031.log.gz 2024-09-19T07:57:55.071 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75723.log: /var/log/ceph/ceph-client.admin.75016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43053.log 2024-09-19T07:57:55.071 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75723.log.gz 2024-09-19T07:57:55.071 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78987.log 2024-09-19T07:57:55.071 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75016.log.gz 2024-09-19T07:57:55.071 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59873.log 2024-09-19T07:57:55.071 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69791.log.gz 2024-09-19T07:57:55.071 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.78987.log: gzip -5 89.1% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.43053.log.gz /var/log/ceph/ceph-client.admin.57460.log 2024-09-19T07:57:55.071 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-19T07:57:55.071 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59873.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78987.log.gz 2024-09-19T07:57:55.071 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40853.log 2024-09-19T07:57:55.072 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59873.log.gz 2024-09-19T07:57:55.072 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47946.log 2024-09-19T07:57:55.072 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41475.log 2024-09-19T07:57:55.072 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57460.log.gz 2024-09-19T07:57:55.072 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40853.log: /var/log/ceph/ceph-client.admin.47946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70043.log 2024-09-19T07:57:55.072 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40853.log.gz 2024-09-19T07:57:55.072 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47946.log.gz 2024-09-19T07:57:55.072 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72393.log 2024-09-19T07:57:55.072 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54841.log 2024-09-19T07:57:55.072 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70043.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41475.log.gz 2024-09-19T07:57:55.072 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61671.log 2024-09-19T07:57:55.072 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70043.log.gz 2024-09-19T07:57:55.073 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72393.log.gz 2024-09-19T07:57:55.073 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82782.log 2024-09-19T07:57:55.073 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54841.log.gz 2024-09-19T07:57:55.073 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48513.log 2024-09-19T07:57:55.073 INFO:teuthology.orchestra.run.smithi016.stderr:gzip 88.4%/var/log/ceph/ceph-client.admin.82782.log: -- replaced with /var/log/ceph/ceph-client.admin.61671.log.gz 2024-09-19T07:57:55.073 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.64280.log 2024-09-19T07:57:55.073 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48513.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82782.log.gz 2024-09-19T07:57:55.073 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50237.log 2024-09-19T07:57:55.073 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48513.log.gz 2024-09-19T07:57:55.073 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77332.log 2024-09-19T07:57:55.073 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64280.log.gz 2024-09-19T07:57:55.073 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62591.log 2024-09-19T07:57:55.073 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50237.log: /var/log/ceph/ceph-client.admin.77332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67582.log 2024-09-19T07:57:55.073 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50237.log.gz 2024-09-19T07:57:55.074 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77332.log.gz 2024-09-19T07:57:55.074 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60370.log 2024-09-19T07:57:55.074 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67582.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62591.log.gz 2024-09-19T07:57:55.074 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81984.log 2024-09-19T07:57:55.074 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67582.log.gz 2024-09-19T07:57:55.074 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69098.log 2024-09-19T07:57:55.074 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65226.log 2024-09-19T07:57:55.074 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.81984.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60370.log.gz 2024-09-19T07:57:55.074 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69098.log: gzip -5 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81984.log.gz 2024-09-19T07:57:55.074 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.67708.log 2024-09-19T07:57:55.074 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65226.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69098.log.gz 2024-09-19T07:57:55.074 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73920.log 2024-09-19T07:57:55.074 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65226.log.gz 2024-09-19T07:57:55.074 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80625.log 2024-09-19T07:57:55.075 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47904.log 2024-09-19T07:57:55.075 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67708.log.gz 2024-09-19T07:57:55.075 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.80625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47844.log 2024-09-19T07:57:55.075 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73920.log.gz 2024-09-19T07:57:55.075 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69140.log 2024-09-19T07:57:55.075 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80625.log.gz 2024-09-19T07:57:55.075 INFO:teuthology.orchestra.run.smithi016.stderr:gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47904.log.gz 2024-09-19T07:57:55.075 INFO:teuthology.orchestra.run.smithi016.stderr: -5/var/log/ceph/ceph-client.admin.47844.log: --verbose -- /var/log/ceph/ceph-client.admin.73254.log 2024-09-19T07:57:55.075 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69140.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47844.log.gz 2024-09-19T07:57:55.075 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43390.log 2024-09-19T07:57:55.075 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69140.log.gz 2024-09-19T07:57:55.075 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57376.log 2024-09-19T07:57:55.075 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73254.log.gz 2024-09-19T07:57:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63356.log 2024-09-19T07:57:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67624.log 2024-09-19T07:57:55.076 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43390.log.gz 2024-09-19T07:57:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57376.log: /var/log/ceph/ceph-client.admin.63356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68049.log 2024-09-19T07:57:55.076 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57376.log.gz 2024-09-19T07:57:55.076 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63356.log.gz 2024-09-19T07:57:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68312.log 2024-09-19T07:57:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72057.log 2024-09-19T07:57:55.076 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67624.log.gz 2024-09-19T07:57:55.076 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5%/var/log/ceph/ceph-client.admin.68312.log: -- replaced with /var/log/ceph/ceph-client.admin.68049.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71166.log 2024-09-19T07:57:55.076 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-19T07:57:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72057.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68312.log.gz 2024-09-19T07:57:55.076 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63243.log 2024-09-19T07:57:55.077 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72057.log.gz 2024-09-19T07:57:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56783.log 2024-09-19T07:57:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63243.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71166.log.gz 2024-09-19T07:57:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62083.log 2024-09-19T07:57:55.077 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63243.log.gz 2024-09-19T07:57:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71334.log 2024-09-19T07:57:55.077 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7%/var/log/ceph/ceph-client.admin.62083.log: -- replaced with /var/log/ceph/ceph-client.admin.56783.log.gz 2024-09-19T07:57:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48911.log 2024-09-19T07:57:55.077 INFO:teuthology.orchestra.run.smithi016.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.62083.log.gz 2024-09-19T07:57:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66935.log 2024-09-19T07:57:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48911.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71334.log.gz 2024-09-19T07:57:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43058.log 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48911.log.gz 2024-09-19T07:57:55.077 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-19T07:57:55.077 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60936.log 2024-09-19T07:57:55.078 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66935.log.gz 2024-09-19T07:57:55.078 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82110.log 2024-09-19T07:57:55.078 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43058.log.gz 2024-09-19T07:57:55.078 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58075.log 2024-09-19T07:57:55.078 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60936.log.gz 2024-09-19T07:57:55.078 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69224.log 2024-09-19T07:57:55.078 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.82110.log.gz 2024-09-19T07:57:55.078 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58075.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58075.log.gz 2024-09-19T07:57:55.078 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.66184.log 2024-09-19T07:57:55.078 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61430.log 2024-09-19T07:57:55.078 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66184.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69224.log.gz 2024-09-19T07:57:55.078 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64196.log 2024-09-19T07:57:55.078 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.61430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65268.log 2024-09-19T07:57:55.079 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66184.log.gz 2024-09-19T07:57:55.079 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42572.log 2024-09-19T07:57:55.079 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.65268.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64196.log.gz 2024-09-19T07:57:55.079 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53396.log 2024-09-19T07:57:55.079 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42572.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65268.log.gz 2024-09-19T07:57:55.079 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76175.log 2024-09-19T07:57:55.079 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42572.log.gz 2024-09-19T07:57:55.079 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65184.log 2024-09-19T07:57:55.079 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53396.log.gz 2024-09-19T07:57:55.079 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.76175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81049.log 2024-09-19T07:57:55.079 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58369.log 2024-09-19T07:57:55.079 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65184.log.gz/var/log/ceph/ceph-client.admin.81049.log: 2024-09-19T07:57:55.079 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53228.log 2024-09-19T07:57:55.079 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.81049.log.gz 2024-09-19T07:57:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80271.log 2024-09-19T07:57:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55117.log 2024-09-19T07:57:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53228.log: /var/log/ceph/ceph-client.admin.80271.log: 88.7% 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53228.log.gz 2024-09-19T07:57:55.080 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58369.log.gz 2024-09-19T07:57:55.080 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80271.log.gz 2024-09-19T07:57:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55051.log 2024-09-19T07:57:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57907.log 2024-09-19T07:57:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55051.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55117.log.gz 2024-09-19T07:57:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75915.log 2024-09-19T07:57:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.57907.log: /var/log/ceph/ceph-client.admin.46973.log 2024-09-19T07:57:55.080 INFO:teuthology.orchestra.run.smithi016.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.55051.log.gz 2024-09-19T07:57:55.080 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57907.log.gz 2024-09-19T07:57:55.080 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43385.log 2024-09-19T07:57:55.081 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4%/var/log/ceph/ceph-client.admin.46973.log: -- replaced with /var/log/ceph/ceph-client.admin.75915.log.gz 2024-09-19T07:57:55.081 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72929.log 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46973.log.gz 2024-09-19T07:57:55.081 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-19T07:57:55.081 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43385.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-19T07:57:55.081 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43385.log.gz 2024-09-19T07:57:55.081 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.72929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80313.log 2024-09-19T07:57:55.081 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67750.log 2024-09-19T07:57:55.081 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.80313.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80313.log.gz 2024-09-19T07:57:55.081 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81528.log 2024-09-19T07:57:55.081 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67750.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67750.log.gz 2024-09-19T07:57:55.081 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64747.log 2024-09-19T07:57:55.081 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.81528.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81528.log.gz 2024-09-19T07:57:55.081 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78022.log 2024-09-19T07:57:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64747.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64747.log.gz 2024-09-19T07:57:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57991.log 2024-09-19T07:57:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.78022.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78022.log.gz 2024-09-19T07:57:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42530.log 2024-09-19T07:57:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57991.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57991.log.gz 2024-09-19T07:57:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56951.log 2024-09-19T07:57:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42530.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42530.log.gz 2024-09-19T07:57:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46729.log 2024-09-19T07:57:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56951.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56951.log.gz 2024-09-19T07:57:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47369.log 2024-09-19T07:57:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46729.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46729.log.gz 2024-09-19T07:57:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44180.log 2024-09-19T07:57:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47369.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47369.log.gz 2024-09-19T07:57:55.082 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64877.log 2024-09-19T07:57:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44180.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44180.log.gz 2024-09-19T07:57:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58435.log 2024-09-19T07:57:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64877.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64877.log.gz 2024-09-19T07:57:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68534.log 2024-09-19T07:57:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58435.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.58435.log.gz 2024-09-19T07:57:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77206.log 2024-09-19T07:57:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68534.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.68534.log.gz 2024-09-19T07:57:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65864.log 2024-09-19T07:57:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77206.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77206.log.gz 2024-09-19T07:57:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69833.log 2024-09-19T07:57:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.65864.log: 88.6% -- replaced with /var/log/ceph/ceph-client.0.65864.log.gz 2024-09-19T07:57:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70515.log 2024-09-19T07:57:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69833.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69833.log.gz 2024-09-19T07:57:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52383.log 2024-09-19T07:57:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70515.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70515.log.gz 2024-09-19T07:57:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74423.log 2024-09-19T07:57:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52383.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52383.log.gz 2024-09-19T07:57:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58695.log 2024-09-19T07:57:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74423.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74423.log.gz 2024-09-19T07:57:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69056.log 2024-09-19T07:57:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.58695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82740.log 2024-09-19T07:57:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69056.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69056.log.gz 2024-09-19T07:57:55.085 INFO:teuthology.orchestra.run.smithi146.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-19T07:57:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64028.log 2024-09-19T07:57:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82740.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.82740.log.gz 2024-09-19T07:57:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42793.log 2024-09-19T07:57:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64028.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64028.log.gz 2024-09-19T07:57:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54074.log 2024-09-19T07:57:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42793.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42793.log.gz 2024-09-19T07:57:55.095 INFO:teuthology.orchestra.run.smithi016.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.0.72929.log.gz 2024-09-19T07:57:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62285.log 2024-09-19T07:57:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54074.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54074.log.gz 2024-09-19T07:57:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62020.log 2024-09-19T07:57:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62285.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62285.log.gz 2024-09-19T07:57:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55811.log 2024-09-19T07:57:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62020.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62020.log.gz 2024-09-19T07:57:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47154.log 2024-09-19T07:57:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55811.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55811.log.gz 2024-09-19T07:57:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59453.log 2024-09-19T07:57:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47154.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47154.log.gz 2024-09-19T07:57:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79029.log 2024-09-19T07:57:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59453.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59453.log.gz 2024-09-19T07:57:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62334.log 2024-09-19T07:57:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.79029.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79029.log.gz 2024-09-19T07:57:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77686.log 2024-09-19T07:57:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62334.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62334.log.gz 2024-09-19T07:57:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48356.log 2024-09-19T07:57:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77686.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77686.log.gz 2024-09-19T07:57:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57292.log 2024-09-19T07:57:55.097 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.48356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74633.log 2024-09-19T07:57:55.098 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57292.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57292.log.gz 2024-09-19T07:57:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58033.log 2024-09-19T07:57:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74633.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74633.log.gz 2024-09-19T07:57:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45617.log 2024-09-19T07:57:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58033.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58033.log.gz 2024-09-19T07:57:55.122 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82026.log 2024-09-19T07:57:55.123 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45617.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45617.log.gz 2024-09-19T07:57:55.129 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51467.log 2024-09-19T07:57:55.129 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82026.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.82026.log.gz 2024-09-19T07:57:55.146 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47592.log 2024-09-19T07:57:55.146 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51467.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51467.log.gz 2024-09-19T07:57:55.154 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71929.log 2024-09-19T07:57:55.155 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47592.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47592.log.gz 2024-09-19T07:57:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53937.log 2024-09-19T07:57:55.164 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71929.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71929.log.gz 2024-09-19T07:57:55.171 INFO:teuthology.orchestra.run.smithi016.stderr: 95.3% -- replaced with /var/log/ceph/ceph-client.0.76175.log.gz 2024-09-19T07:57:55.173 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80853.log 2024-09-19T07:57:55.173 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53937.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53937.log.gz 2024-09-19T07:57:55.173 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49878.log 2024-09-19T07:57:55.173 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.80853.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80853.log.gz 2024-09-19T07:57:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46519.log 2024-09-19T07:57:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49878.log: 92.6% -- replaced with /var/log/ceph/ceph-client.0.48356.log.gz 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49878.log.gz 2024-09-19T07:57:55.180 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-19T07:57:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82572.log 2024-09-19T07:57:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81360.log 2024-09-19T07:57:55.180 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46519.log.gz 2024-09-19T07:57:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76806.log 2024-09-19T07:57:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.81360.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.82572.log.gz 2024-09-19T07:57:55.180 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81360.log.gz 2024-09-19T07:57:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49481.log 2024-09-19T07:57:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.76806.log: -- /var/log/ceph/ceph-client.admin.59663.log 2024-09-19T07:57:55.180 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76806.log.gz 2024-09-19T07:57:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54799.log 2024-09-19T07:57:55.181 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59663.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49481.log.gz 2024-09-19T07:57:55.181 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59663.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70979.log 2024-09-19T07:57:55.181 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-19T07:57:55.181 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55377.log 2024-09-19T07:57:55.186 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54799.log: /var/log/ceph/ceph-client.admin.70979.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54799.log.gz 2024-09-19T07:57:55.186 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40949.log 2024-09-19T07:57:55.186 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70979.log.gz 2024-09-19T07:57:55.186 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66107.log 2024-09-19T07:57:55.186 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.55377.log: /var/log/ceph/ceph-client.admin.40949.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40949.log.gz 2024-09-19T07:57:55.187 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50502.log 2024-09-19T07:57:55.187 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66107.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66107.log.gz 2024-09-19T07:57:55.187 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42614.log 2024-09-19T07:57:55.187 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50502.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50502.log.gz 2024-09-19T07:57:55.187 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63437.log 2024-09-19T07:57:55.187 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42614.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42614.log.gz 2024-09-19T07:57:55.187 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73600.log 2024-09-19T07:57:55.187 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63437.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63437.log.gz 2024-09-19T07:57:55.189 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-19T07:57:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73600.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73600.log.gz 2024-09-19T07:57:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80583.log 2024-09-19T07:57:55.199 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49154.log 2024-09-19T07:57:55.199 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.80583.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80583.log.gz 2024-09-19T07:57:55.205 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57801.log 2024-09-19T07:57:55.218 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80991.log 2024-09-19T07:57:55.219 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49154.log.gz 2024-09-19T07:57:55.219 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57801.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57801.log.gz 2024-09-19T07:57:55.232 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45423.log 2024-09-19T07:57:55.233 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.80991.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80991.log.gz 2024-09-19T07:57:55.239 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78651.log 2024-09-19T07:57:55.239 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45423.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45423.log.gz 2024-09-19T07:57:55.249 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57555.log 2024-09-19T07:57:55.262 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.78651.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78651.log.gz 2024-09-19T07:57:55.264 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50842.log 2024-09-19T07:57:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57555.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57555.log.gz 2024-09-19T07:57:55.273 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82896.log 2024-09-19T07:57:55.293 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50842.log: 95.5% -- replaced with /var/log/ceph/ceph-client.0.69634.log.gz 2024-09-19T07:57:55.293 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54501.log 2024-09-19T07:57:55.294 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82896.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.82896.log.gz 2024-09-19T07:57:55.294 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43730.log 2024-09-19T07:57:55.294 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54501.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54501.log.gz 2024-09-19T07:57:55.294 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68846.log 2024-09-19T07:57:55.294 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43730.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43730.log.gz 2024-09-19T07:57:55.294 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54364.log 2024-09-19T07:57:55.294 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68846.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.68846.log.gz 2024-09-19T07:57:55.294 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49668.log 2024-09-19T07:57:55.294 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54364.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54364.log.gz 2024-09-19T07:57:55.294 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54589.log 2024-09-19T07:57:55.294 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50842.log.gz 2024-09-19T07:57:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49668.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49668.log.gz 2024-09-19T07:57:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58159.log 2024-09-19T07:57:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54589.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54589.log.gz 2024-09-19T07:57:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45701.log 2024-09-19T07:57:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58159.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58159.log.gz 2024-09-19T07:57:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68972.log 2024-09-19T07:57:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45701.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45701.log.gz 2024-09-19T07:57:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66444.log 2024-09-19T07:57:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68972.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68972.log.gz 2024-09-19T07:57:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52341.log 2024-09-19T07:57:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.66444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81570.log 2024-09-19T07:57:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52341.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52341.log.gz 2024-09-19T07:57:55.313 INFO:teuthology.orchestra.run.smithi146.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-19T07:57:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75058.log 2024-09-19T07:57:55.321 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.81570.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81570.log.gz 2024-09-19T07:57:55.321 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73688.log 2024-09-19T07:57:55.334 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75058.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75058.log.gz 2024-09-19T07:57:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70770.log 2024-09-19T07:57:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73688.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73688.log.gz 2024-09-19T07:57:55.348 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49920.log 2024-09-19T07:57:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70770.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70770.log.gz 2024-09-19T07:57:55.367 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62894.log 2024-09-19T07:57:55.367 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49920.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49920.log.gz 2024-09-19T07:57:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41433.log 2024-09-19T07:57:55.377 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62894.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62894.log.gz 2024-09-19T07:57:55.385 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47718.log 2024-09-19T07:57:55.386 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41433.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41433.log.gz 2024-09-19T07:57:55.404 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69014.log 2024-09-19T07:57:55.404 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47718.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47718.log.gz 2024-09-19T07:57:55.409 INFO:teuthology.orchestra.run.smithi146.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-19T07:57:55.415 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69959.log 2024-09-19T07:57:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69014.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69014.log.gz 2024-09-19T07:57:55.431 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43049.log 2024-09-19T07:57:55.431 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69959.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69959.log.gz 2024-09-19T07:57:55.440 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47462.log 2024-09-19T07:57:55.441 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43049.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43049.log.gz 2024-09-19T07:57:55.485 INFO:teuthology.orchestra.run.smithi016.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.0.58695.log.gz 2024-09-19T07:57:55.485 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66857.log 2024-09-19T07:57:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47462.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47462.log.gz 2024-09-19T07:57:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79638.log 2024-09-19T07:57:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66857.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66857.log.gz 2024-09-19T07:57:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73338.log 2024-09-19T07:57:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.79638.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79638.log.gz 2024-09-19T07:57:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52445.log 2024-09-19T07:57:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73338.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73338.log.gz 2024-09-19T07:57:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49439.log 2024-09-19T07:57:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52445.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52445.log.gz 2024-09-19T07:57:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82194.log 2024-09-19T07:57:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49439.log: 95.0% -- replaced with /var/log/ceph/ceph-client.0.55377.log.gz 2024-09-19T07:57:55.486 INFO:teuthology.orchestra.run.smithi016.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49439.log.gz 2024-09-19T07:57:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70085.log 2024-09-19T07:57:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.82194.log: -- /var/log/ceph/ceph-osd.2.log 2024-09-19T07:57:55.487 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.82194.log.gz 2024-09-19T07:57:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71639.log 2024-09-19T07:57:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.2.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70085.log.gz 2024-09-19T07:57:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72603.log 2024-09-19T07:57:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71639.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71639.log.gz 2024-09-19T07:57:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71418.log 2024-09-19T07:57:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72603.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.72603.log.gz 2024-09-19T07:57:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59747.log 2024-09-19T07:57:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71418.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.71418.log.gz 2024-09-19T07:57:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57250.log 2024-09-19T07:57:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59747.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59747.log.gz 2024-09-19T07:57:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41601.log 2024-09-19T07:57:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57250.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57250.log.gz 2024-09-19T07:57:55.488 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54925.log 2024-09-19T07:57:55.488 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41601.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41601.log.gz 2024-09-19T07:57:55.488 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46435.log 2024-09-19T07:57:55.488 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54925.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54925.log.gz 2024-09-19T07:57:55.488 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78861.log 2024-09-19T07:57:55.488 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46435.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46435.log.gz 2024-09-19T07:57:55.488 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81785.log 2024-09-19T07:57:55.488 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.78861.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.78861.log.gz 2024-09-19T07:57:55.488 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57717.log 2024-09-19T07:57:55.488 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.81785.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81785.log.gz 2024-09-19T07:57:55.488 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70854.log 2024-09-19T07:57:55.488 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57717.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57717.log.gz 2024-09-19T07:57:55.488 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45659.log 2024-09-19T07:57:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70854.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70854.log.gz 2024-09-19T07:57:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65142.log 2024-09-19T07:57:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45659.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45659.log.gz 2024-09-19T07:57:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47508.log 2024-09-19T07:57:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65142.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65142.log.gz 2024-09-19T07:57:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50950.log 2024-09-19T07:57:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47508.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47508.log.gz 2024-09-19T07:57:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71973.log 2024-09-19T07:57:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50950.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50950.log.gz 2024-09-19T07:57:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81243.log 2024-09-19T07:57:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71973.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71973.log.gz 2024-09-19T07:57:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57418.log 2024-09-19T07:57:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.81243.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81243.log.gz 2024-09-19T07:57:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52088.log 2024-09-19T07:57:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57418.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57418.log.gz 2024-09-19T07:57:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41305.log 2024-09-19T07:57:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52088.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52088.log.gz 2024-09-19T07:57:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45938.log 2024-09-19T07:57:55.498 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41305.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41305.log.gz 2024-09-19T07:57:55.498 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67842.log 2024-09-19T07:57:55.498 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45938.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45938.log.gz 2024-09-19T07:57:55.498 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70371.log 2024-09-19T07:57:55.499 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67842.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67842.log.gz 2024-09-19T07:57:55.499 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77938.log 2024-09-19T07:57:55.499 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70371.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70371.log.gz 2024-09-19T07:57:55.499 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53498.log 2024-09-19T07:57:55.499 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77938.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77938.log.gz 2024-09-19T07:57:55.499 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43384.log 2024-09-19T07:57:55.499 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53498.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53498.log.gz 2024-09-19T07:57:55.499 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42878.log 2024-09-19T07:57:55.499 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43384.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43384.log.gz 2024-09-19T07:57:55.499 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76737.log 2024-09-19T07:57:55.499 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42878.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42878.log.gz 2024-09-19T07:57:55.499 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53895.log 2024-09-19T07:57:55.500 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76737.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76737.log.gz 2024-09-19T07:57:55.500 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45575.log 2024-09-19T07:57:55.507 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53895.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53895.log.gz 2024-09-19T07:57:55.508 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62799.log 2024-09-19T07:57:55.508 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45575.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45575.log.gz 2024-09-19T07:57:55.508 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46132.log 2024-09-19T07:57:55.508 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62799.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62799.log.gz 2024-09-19T07:57:55.508 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-19T07:57:55.509 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46132.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46132.log.gz 2024-09-19T07:57:55.509 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50800.log 2024-09-19T07:57:55.517 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47550.log 2024-09-19T07:57:55.517 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50800.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50800.log.gz 2024-09-19T07:57:55.520 INFO:teuthology.orchestra.run.smithi016.stderr: 94.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-19T07:57:55.520 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74046.log 2024-09-19T07:57:55.520 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47550.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47550.log.gz 2024-09-19T07:57:55.521 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78424.log 2024-09-19T07:57:55.522 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74046.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74046.log.gz 2024-09-19T07:57:55.522 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43060.log 2024-09-19T07:57:55.522 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.78424.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78424.log.gz 2024-09-19T07:57:55.523 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74381.log 2024-09-19T07:57:55.524 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43060.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43060.log.gz 2024-09-19T07:57:55.524 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77248.log 2024-09-19T07:57:55.525 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74381.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74381.log.gz 2024-09-19T07:57:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82278.log 2024-09-19T07:57:55.527 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77248.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77248.log.gz 2024-09-19T07:57:55.527 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41073.log 2024-09-19T07:57:55.528 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82278.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.82278.log.gz 2024-09-19T07:57:55.529 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54154.log 2024-09-19T07:57:55.530 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41073.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41073.log.gz 2024-09-19T07:57:55.530 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50674.log 2024-09-19T07:57:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54154.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54154.log.gz 2024-09-19T07:57:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42488.log 2024-09-19T07:57:55.533 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50674.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50674.log.gz 2024-09-19T07:57:55.533 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79724.log 2024-09-19T07:57:55.534 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42488.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42488.log.gz 2024-09-19T07:57:55.534 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45785.log 2024-09-19T07:57:55.535 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.79724.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79724.log.gz 2024-09-19T07:57:55.535 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75411.log 2024-09-19T07:57:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45785.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45785.log.gz 2024-09-19T07:57:55.537 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80499.log 2024-09-19T07:57:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75411.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75411.log.gz 2024-09-19T07:57:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78605.log 2024-09-19T07:57:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.80499.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.80499.log.gz 2024-09-19T07:57:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62381.log 2024-09-19T07:57:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.78605.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.78605.log.gz 2024-09-19T07:57:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-19T07:57:55.542 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62381.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62381.log.gz 2024-09-19T07:57:55.543 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63590.log 2024-09-19T07:57:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.log: 86.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-19T07:57:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75807.log 2024-09-19T07:57:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63590.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63590.log.gz 2024-09-19T07:57:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48597.log 2024-09-19T07:57:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75807.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75807.log.gz 2024-09-19T07:57:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60809.log 2024-09-19T07:57:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48597.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48597.log.gz 2024-09-19T07:57:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66600.log 2024-09-19T07:57:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60809.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60809.log.gz 2024-09-19T07:57:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75144.log 2024-09-19T07:57:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66600.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66600.log.gz 2024-09-19T07:57:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47281.log 2024-09-19T07:57:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75144.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75144.log.gz 2024-09-19T07:57:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41203.log 2024-09-19T07:57:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47281.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47281.log.gz 2024-09-19T07:57:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64928.log 2024-09-19T07:57:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41203.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41203.log.gz 2024-09-19T07:57:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81127.log 2024-09-19T07:57:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64928.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64928.log.gz 2024-09-19T07:57:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63101.log 2024-09-19T07:57:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.81127.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81127.log.gz 2024-09-19T07:57:55.561 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71208.log 2024-09-19T07:57:55.561 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63101.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63101.log.gz 2024-09-19T07:57:55.563 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65058.log 2024-09-19T07:57:55.563 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71208.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71208.log.gz 2024-09-19T07:57:55.563 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43394.log 2024-09-19T07:57:55.565 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65058.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65058.log.gz 2024-09-19T07:57:55.565 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74591.log 2024-09-19T07:57:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43394.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43394.log.gz 2024-09-19T07:57:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42446.log 2024-09-19T07:57:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74591.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74591.log.gz 2024-09-19T07:57:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49313.log 2024-09-19T07:57:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42446.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42446.log.gz 2024-09-19T07:57:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76500.log 2024-09-19T07:57:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49313.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49313.log.gz 2024-09-19T07:57:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55009.log 2024-09-19T07:57:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76500.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76500.log.gz 2024-09-19T07:57:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63842.log 2024-09-19T07:57:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55009.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55009.log.gz 2024-09-19T07:57:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77131.log 2024-09-19T07:57:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63842.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.63842.log.gz 2024-09-19T07:57:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50544.log 2024-09-19T07:57:55.578 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77131.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77131.log.gz 2024-09-19T07:57:55.578 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46771.log 2024-09-19T07:57:55.578 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50544.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50544.log.gz 2024-09-19T07:57:55.578 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46687.log 2024-09-19T07:57:55.579 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46771.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46771.log.gz 2024-09-19T07:57:55.579 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46267.log 2024-09-19T07:57:55.582 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46687.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46687.log.gz 2024-09-19T07:57:55.582 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82236.log 2024-09-19T07:57:55.582 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46267.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46267.log.gz 2024-09-19T07:57:55.582 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76911.log 2024-09-19T07:57:55.583 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82236.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82236.log.gz 2024-09-19T07:57:55.584 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41349.log 2024-09-19T07:57:55.585 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76911.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76911.log.gz 2024-09-19T07:57:55.585 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77896.log 2024-09-19T07:57:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41349.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41349.log.gz 2024-09-19T07:57:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73845.log 2024-09-19T07:57:55.588 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77896.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77896.log.gz 2024-09-19T07:57:55.588 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57035.log 2024-09-19T07:57:55.589 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73845.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73845.log.gz 2024-09-19T07:57:55.589 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75537.log 2024-09-19T07:57:55.590 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57035.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57035.log.gz 2024-09-19T07:57:55.592 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47063.log 2024-09-19T07:57:55.592 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75537.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75537.log.gz 2024-09-19T07:57:55.592 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80355.log 2024-09-19T07:57:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47063.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47063.log.gz 2024-09-19T07:57:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70593.log 2024-09-19T07:57:55.595 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.80355.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80355.log.gz 2024-09-19T07:57:55.595 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49271.log 2024-09-19T07:57:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70593.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70593.log.gz 2024-09-19T07:57:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63986.log 2024-09-19T07:57:55.597 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49271.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49271.log.gz 2024-09-19T07:57:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79113.log 2024-09-19T07:57:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63986.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63986.log.gz 2024-09-19T07:57:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53456.log 2024-09-19T07:57:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.79113.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79113.log.gz 2024-09-19T07:57:55.613 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68133.log 2024-09-19T07:57:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53456.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53456.log.gz 2024-09-19T07:57:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69917.log 2024-09-19T07:57:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68133.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68133.log.gz 2024-09-19T07:57:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73128.log 2024-09-19T07:57:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69917.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69917.log.gz 2024-09-19T07:57:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71376.log 2024-09-19T07:57:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73128.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73128.log.gz 2024-09-19T07:57:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60460.log 2024-09-19T07:57:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71376.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71376.log.gz 2024-09-19T07:57:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76374.log 2024-09-19T07:57:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60460.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60460.log.gz 2024-09-19T07:57:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53186.log 2024-09-19T07:57:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76374.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76374.log.gz 2024-09-19T07:57:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69182.log 2024-09-19T07:57:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53186.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53186.log.gz 2024-09-19T07:57:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47802.log 2024-09-19T07:57:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69182.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69182.log.gz 2024-09-19T07:57:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66729.log 2024-09-19T07:57:55.618 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47802.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47802.log.gz 2024-09-19T07:57:55.618 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72309.log 2024-09-19T07:57:55.618 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.66729.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66729.log.gz 2024-09-19T07:57:55.618 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77854.log 2024-09-19T07:57:55.618 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72309.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72309.log.gz 2024-09-19T07:57:55.618 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43592.log 2024-09-19T07:57:55.620 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77854.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77854.log.gz 2024-09-19T07:57:55.621 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73643.log 2024-09-19T07:57:55.621 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43592.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43592.log.gz 2024-09-19T07:57:55.621 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65412.log 2024-09-19T07:57:55.624 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73643.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73643.log.gz 2024-09-19T07:57:55.625 INFO:teuthology.orchestra.run.smithi016.stderr: 96.0% -- replaced with /var/log/ceph/ceph-client.0.61430.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82404.log 2024-09-19T07:57:55.625 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-19T07:57:55.625 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75453.log 2024-09-19T07:57:55.625 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82404.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65412.log.gz 2024-09-19T07:57:55.625 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.82404.log.gz 2024-09-19T07:57:55.625 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54883.log 2024-09-19T07:57:55.625 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49962.log 2024-09-19T07:57:55.625 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75453.log: /var/log/ceph/ceph-client.admin.54883.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75453.log.gz 2024-09-19T07:57:55.625 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80769.log 2024-09-19T07:57:55.625 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54883.log.gz 2024-09-19T07:57:55.625 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56909.log 2024-09-19T07:57:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49962.log: /var/log/ceph/ceph-client.admin.80769.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49962.log.gz 2024-09-19T07:57:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43726.log 2024-09-19T07:57:55.629 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80769.log.gz 2024-09-19T07:57:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.56909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43731.log 2024-09-19T07:57:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43726.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56909.log.gz 2024-09-19T07:57:55.629 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43726.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60852.log 2024-09-19T07:57:55.629 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-19T07:57:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44351.log 2024-09-19T07:57:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43731.log: /var/log/ceph/ceph-client.admin.60852.log: 88.1% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43731.log.gz 2024-09-19T07:57:55.630 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60852.log.gz 2024-09-19T07:57:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62639.log 2024-09-19T07:57:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5/var/log/ceph/ceph-client.admin.44351.log: --verbose -- /var/log/ceph/ceph-client.admin.48030.log 2024-09-19T07:57:55.630 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44351.log.gz/var/log/ceph/ceph-client.admin.62639.log: 2024-09-19T07:57:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81827.log 2024-09-19T07:57:55.637 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62639.log.gz 2024-09-19T07:57:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44865.log 2024-09-19T07:57:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.81827.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.48030.log.gz 2024-09-19T07:57:55.637 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81827.log.gz 2024-09-19T07:57:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72267.log 2024-09-19T07:57:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46190.log 2024-09-19T07:57:55.638 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44865.log.gz 2024-09-19T07:57:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46393.log 2024-09-19T07:57:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46190.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72267.log.gz 2024-09-19T07:57:55.638 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46190.log.gz 2024-09-19T07:57:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59251.log 2024-09-19T07:57:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69266.log 2024-09-19T07:57:55.638 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46393.log.gz 2024-09-19T07:57:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78693.log 2024-09-19T07:57:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69266.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59251.log.gz 2024-09-19T07:57:55.638 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69266.log.gz 2024-09-19T07:57:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77980.log 2024-09-19T07:57:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60595.log 2024-09-19T07:57:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.78693.log: /var/log/ceph/ceph-client.admin.77980.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78693.log.gz 2024-09-19T07:57:55.638 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.77980.log.gz 2024-09-19T07:57:55.639 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74759.log 2024-09-19T07:57:55.641 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60595.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60595.log.gz 2024-09-19T07:57:55.641 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60286.log 2024-09-19T07:57:55.641 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53685.log 2024-09-19T07:57:55.641 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74759.log: /var/log/ceph/ceph-client.admin.60286.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74759.log.gz 2024-09-19T07:57:55.641 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60286.log.gz 2024-09-19T07:57:55.642 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70196.log 2024-09-19T07:57:55.642 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60683.log 2024-09-19T07:57:55.642 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53685.log: /var/log/ceph/ceph-client.admin.70196.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53685.log.gz 2024-09-19T07:57:55.642 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70196.log.gz 2024-09-19T07:57:55.642 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69374.log 2024-09-19T07:57:55.642 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49229.log 2024-09-19T07:57:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60683.log: /var/log/ceph/ceph-client.admin.69374.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60683.log.gz 2024-09-19T07:57:55.646 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69374.log.gz 2024-09-19T07:57:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53601.log 2024-09-19T07:57:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65454.log 2024-09-19T07:57:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53601.log: /var/log/ceph/ceph-client.admin.49229.log: 88.3% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49229.log.gz 2024-09-19T07:57:55.647 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53601.log.gz 2024-09-19T07:57:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78735.log 2024-09-19T07:57:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67388.log 2024-09-19T07:57:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65454.log: /var/log/ceph/ceph-client.admin.78735.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65454.log.gz 2024-09-19T07:57:55.647 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78735.log.gz 2024-09-19T07:57:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52841.log 2024-09-19T07:57:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.67388.log: -- /var/log/ceph/ceph-client.admin.74213.log 2024-09-19T07:57:55.647 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67388.log.gz 2024-09-19T07:57:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.52841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58936.log 2024-09-19T07:57:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74213.log: 87.0% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74213.log.gz 2024-09-19T07:57:55.653 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.0.52841.log.gz 2024-09-19T07:57:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51367.log 2024-09-19T07:57:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43928.log 2024-09-19T07:57:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58936.log: /var/log/ceph/ceph-client.admin.51367.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58936.log.gz 2024-09-19T07:57:55.653 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51367.log.gz 2024-09-19T07:57:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59173.log 2024-09-19T07:57:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43928.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43928.log.gz 2024-09-19T07:57:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41559.log 2024-09-19T07:57:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63716.log 2024-09-19T07:57:55.653 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59173.log.gz 2024-09-19T07:57:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77812.log 2024-09-19T07:57:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63716.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41559.log.gz 2024-09-19T07:57:55.654 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63716.log.gz 2024-09-19T07:57:55.654 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74172.log 2024-09-19T07:57:55.654 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.77812.log: /var/log/ceph/ceph-client.admin.48973.log 2024-09-19T07:57:55.654 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77812.log.gz 2024-09-19T07:57:55.664 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53979.log 2024-09-19T07:57:55.664 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48973.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.74172.log.gz 2024-09-19T07:57:55.664 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67792.log 2024-09-19T07:57:55.664 INFO:teuthology.orchestra.run.smithi016.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.48973.log.gz 2024-09-19T07:57:55.664 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71460.log 2024-09-19T07:57:55.664 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53979.log: /var/log/ceph/ceph-client.admin.67792.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53979.log.gz 2024-09-19T07:57:55.664 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67792.log.gz 2024-09-19T07:57:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77458.log 2024-09-19T07:57:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.71460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65538.log 2024-09-19T07:57:55.665 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71460.log.gz 2024-09-19T07:57:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69875.log 2024-09-19T07:57:55.665 INFO:teuthology.orchestra.run.smithi016.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.77458.log.gz 2024-09-19T07:57:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68492.log 2024-09-19T07:57:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.69875.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69875.log.gz 2024-09-19T07:57:55.665 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.65538.log.gz 2024-09-19T07:57:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82656.log 2024-09-19T07:57:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43388.log 2024-09-19T07:57:55.666 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68492.log.gz 2024-09-19T07:57:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41709.log 2024-09-19T07:57:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43388.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.82656.log.gz 2024-09-19T07:57:55.666 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56235.log 2024-09-19T07:57:55.666 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43388.log.gz 2024-09-19T07:57:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41709.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41709.log.gz 2024-09-19T07:57:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66771.log 2024-09-19T07:57:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43729.log 2024-09-19T07:57:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.56235.log: /var/log/ceph/ceph-client.admin.66771.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66771.log.gz 2024-09-19T07:57:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82936.log 2024-09-19T07:57:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43729.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43729.log.gz 2024-09-19T07:57:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74339.log 2024-09-19T07:57:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82936.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82936.log.gz 2024-09-19T07:57:55.672 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80913.log 2024-09-19T07:57:55.672 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74339.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74339.log.gz 2024-09-19T07:57:55.672 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74255.log 2024-09-19T07:57:55.672 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.80913.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80913.log.gz 2024-09-19T07:57:55.672 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62941.log 2024-09-19T07:57:55.672 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74255.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74255.log.gz 2024-09-19T07:57:55.672 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60641.log 2024-09-19T07:57:55.672 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62941.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62941.log.gz 2024-09-19T07:57:55.672 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59621.log 2024-09-19T07:57:55.675 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60641.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60641.log.gz 2024-09-19T07:57:55.675 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40772.log 2024-09-19T07:57:55.675 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59621.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59621.log.gz 2024-09-19T07:57:55.675 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68175.log 2024-09-19T07:57:55.675 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40772.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40772.log.gz 2024-09-19T07:57:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53034.log 2024-09-19T07:57:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68175.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68175.log.gz 2024-09-19T07:57:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75369.log 2024-09-19T07:57:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53034.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53034.log.gz 2024-09-19T07:57:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53727.log 2024-09-19T07:57:55.680 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75369.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75369.log.gz 2024-09-19T07:57:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53354.log 2024-09-19T07:57:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53727.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53727.log.gz 2024-09-19T07:57:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82530.log 2024-09-19T07:57:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53354.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53354.log.gz 2024-09-19T07:57:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72477.log 2024-09-19T07:57:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82530.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.82530.log.gz 2024-09-19T07:57:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49710.log 2024-09-19T07:57:55.685 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72477.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72477.log.gz 2024-09-19T07:57:55.686 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78468.log 2024-09-19T07:57:55.687 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49710.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49710.log.gz 2024-09-19T07:57:55.687 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60124.log 2024-09-19T07:57:55.687 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.78468.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78468.log.gz 2024-09-19T07:57:55.689 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54757.log 2024-09-19T07:57:55.689 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60124.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60124.log.gz 2024-09-19T07:57:55.690 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81486.log 2024-09-19T07:57:55.691 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54757.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54757.log.gz 2024-09-19T07:57:55.691 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73296.log 2024-09-19T07:57:55.692 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.81486.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81486.log.gz 2024-09-19T07:57:55.693 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61629.log 2024-09-19T07:57:55.693 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73296.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73296.log.gz 2024-09-19T07:57:55.694 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63935.log 2024-09-19T07:57:55.695 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.61629.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61629.log.gz 2024-09-19T07:57:55.695 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67666.log 2024-09-19T07:57:55.697 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63935.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63935.log.gz 2024-09-19T07:57:55.697 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64238.log 2024-09-19T07:57:55.698 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67666.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67666.log.gz 2024-09-19T07:57:55.698 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78903.log 2024-09-19T07:57:55.699 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64238.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64238.log.gz 2024-09-19T07:57:55.699 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59976.log 2024-09-19T07:57:55.700 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.78903.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78903.log.gz 2024-09-19T07:57:55.701 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80727.log 2024-09-19T07:57:55.703 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59976.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59976.log.gz 2024-09-19T07:57:55.703 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46477.log 2024-09-19T07:57:55.703 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.80727.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80727.log.gz 2024-09-19T07:57:55.704 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62988.log 2024-09-19T07:57:55.705 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46477.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46477.log.gz 2024-09-19T07:57:55.706 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81402.log 2024-09-19T07:57:55.707 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62988.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62988.log.gz 2024-09-19T07:57:55.707 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41245.log 2024-09-19T07:57:55.708 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.81402.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.81402.log.gz 2024-09-19T07:57:55.708 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46351.log 2024-09-19T07:57:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41245.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41245.log.gz 2024-09-19T07:57:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76869.log 2024-09-19T07:57:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46351.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46351.log.gz 2024-09-19T07:57:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47112.log 2024-09-19T07:57:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76869.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76869.log.gz 2024-09-19T07:57:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63054.log 2024-09-19T07:57:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47112.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47112.log.gz 2024-09-19T07:57:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78340.log 2024-09-19T07:57:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63054.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63054.log.gz 2024-09-19T07:57:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73212.log 2024-09-19T07:57:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.78340.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.78340.log.gz 2024-09-19T07:57:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63148.log 2024-09-19T07:57:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.73212.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73212.log.gz 2024-09-19T07:57:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57759.log 2024-09-19T07:57:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63148.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63148.log.gz 2024-09-19T07:57:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77728.log 2024-09-19T07:57:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57759.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57759.log.gz 2024-09-19T07:57:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72351.log 2024-09-19T07:57:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77728.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.77728.log.gz 2024-09-19T07:57:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75765.log 2024-09-19T07:57:55.725 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72351.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72351.log.gz 2024-09-19T07:57:55.725 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46016.log 2024-09-19T07:57:55.725 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75765.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75765.log.gz 2024-09-19T07:57:55.725 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63196.log 2024-09-19T07:57:55.726 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46016.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46016.log.gz 2024-09-19T07:57:55.727 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58117.log 2024-09-19T07:57:55.729 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.63196.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63196.log.gz 2024-09-19T07:57:55.729 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46058.log 2024-09-19T07:57:55.729 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58117.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58117.log.gz 2024-09-19T07:57:55.730 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54319.log 2024-09-19T07:57:55.731 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46058.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46058.log.gz 2024-09-19T07:57:55.731 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68757.log 2024-09-19T07:57:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54319.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54319.log.gz 2024-09-19T07:57:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47634.log 2024-09-19T07:57:55.733 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.68757.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68757.log.gz 2024-09-19T07:57:55.733 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72099.log 2024-09-19T07:57:55.734 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47634.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47634.log.gz 2024-09-19T07:57:55.735 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52581.log 2024-09-19T07:57:55.736 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72099.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72099.log.gz 2024-09-19T07:57:55.737 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52503.log 2024-09-19T07:57:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52581.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52581.log.gz 2024-09-19T07:57:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64789.log 2024-09-19T07:57:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.52503.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52503.log.gz 2024-09-19T07:57:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77053.log 2024-09-19T07:57:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.64789.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64789.log.gz 2024-09-19T07:57:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76458.log 2024-09-19T07:57:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77053.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.77053.log.gz 2024-09-19T07:57:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47420.log 2024-09-19T07:57:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76458.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76458.log.gz 2024-09-19T07:57:55.746 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77290.log 2024-09-19T07:57:55.746 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47420.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47420.log.gz 2024-09-19T07:57:55.746 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76542.log 2024-09-19T07:57:55.748 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.77290.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77290.log.gz 2024-09-19T07:57:55.748 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70710.log 2024-09-19T07:57:55.748 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.76542.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76542.log.gz 2024-09-19T07:57:55.748 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49397.log 2024-09-19T07:57:55.749 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70710.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70710.log.gz 2024-09-19T07:57:55.749 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59495.log 2024-09-19T07:57:55.750 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49397.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49397.log.gz 2024-09-19T07:57:55.750 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82824.log 2024-09-19T07:57:55.751 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59495.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59495.log.gz 2024-09-19T07:57:55.752 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-19T07:57:55.752 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82824.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82824.log.gz 2024-09-19T07:57:55.753 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79801.log 2024-09-19T07:57:55.765 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41115.log 2024-09-19T07:57:55.766 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.79801.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79801.log.gz 2024-09-19T07:57:55.772 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78819.log 2024-09-19T07:57:55.772 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41115.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41115.log.gz 2024-09-19T07:57:55.787 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74004.log 2024-09-19T07:57:55.788 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.78819.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78819.log.gz 2024-09-19T07:57:55.796 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54408.log 2024-09-19T07:57:55.797 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74004.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74004.log.gz 2024-09-19T07:57:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-19T07:57:55.816 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.54408.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54408.log.gz 2024-09-19T07:57:55.829 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57863.log 2024-09-19T07:57:55.849 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74549.log 2024-09-19T07:57:55.850 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57863.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57863.log.gz 2024-09-19T07:57:55.863 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43392.log 2024-09-19T07:57:55.863 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.74549.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74549.log.gz 2024-09-19T07:57:55.874 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44743.log 2024-09-19T07:57:55.875 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43392.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43392.log.gz 2024-09-19T07:57:55.891 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43728.log 2024-09-19T07:57:55.892 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.44743.log: 88.7% -- replaced with /var/log/ceph/ceph-client.0.44743.log.gz 2024-09-19T07:57:55.905 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65012.log 2024-09-19T07:57:55.905 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43728.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43728.log.gz 2024-09-19T07:57:55.914 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59411.log 2024-09-19T07:57:55.914 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65012.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65012.log.gz 2024-09-19T07:57:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60328.log 2024-09-19T07:57:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.59411.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59411.log.gz 2024-09-19T07:57:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62446.log 2024-09-19T07:57:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60328.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60328.log.gz 2024-09-19T07:57:55.951 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45743.log 2024-09-19T07:57:55.952 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.62446.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62446.log.gz 2024-09-19T07:57:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58243.log 2024-09-19T07:57:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45743.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45743.log.gz 2024-09-19T07:57:55.969 INFO:teuthology.orchestra.run.smithi016.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.66444.log.gz 2024-09-19T07:57:55.980 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53270.log 2024-09-19T07:57:55.987 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.58243.log: 88.9% 95.1% -- replaced with /var/log/ceph/ceph-client.admin.58243.log.gz 2024-09-19T07:57:55.987 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.0.56235.log.gz 2024-09-19T07:57:55.991 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75495.log 2024-09-19T07:57:55.991 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53270.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53270.log.gz 2024-09-19T07:57:55.992 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60553.log 2024-09-19T07:57:55.992 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.75495.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75495.log.gz 2024-09-19T07:57:55.992 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65352.log 2024-09-19T07:57:55.992 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.60553.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60553.log.gz 2024-09-19T07:57:55.992 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70329.log 2024-09-19T07:57:55.996 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.65352.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65352.log.gz 2024-09-19T07:57:55.996 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53111.log 2024-09-19T07:57:55.996 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.70329.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70329.log.gz 2024-09-19T07:57:55.996 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57138.log 2024-09-19T07:57:55.996 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.53111.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53111.log.gz 2024-09-19T07:57:55.996 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82698.log 2024-09-19T07:57:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.57138.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57138.log.gz 2024-09-19T07:57:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67195.log 2024-09-19T07:57:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.82698.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82698.log.gz 2024-09-19T07:57:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72669.log 2024-09-19T07:57:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.67195.log: 87.6% -- replaced with /var/log/ceph/ceph-client.0.67195.log.gz 2024-09-19T07:57:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78777.log 2024-09-19T07:57:56.002 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72669.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72669.log.gz 2024-09-19T07:57:56.002 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43721.log 2024-09-19T07:57:56.004 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.78777.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78777.log.gz 2024-09-19T07:57:56.004 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55534.log 2024-09-19T07:57:56.004 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43721.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43721.log.gz 2024-09-19T07:57:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67465.log 2024-09-19T07:57:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.55534.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55534.log.gz 2024-09-19T07:57:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72141.log 2024-09-19T07:57:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.67465.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.67465.log.gz 2024-09-19T07:57:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51804.log 2024-09-19T07:57:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.72141.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72141.log.gz 2024-09-19T07:57:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67540.log 2024-09-19T07:57:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.51804.log: /var/log/ceph/ceph-client.admin.67540.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67540.log.gz 2024-09-19T07:57:56.345 INFO:teuthology.orchestra.run.smithi016.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-19T07:57:56.388 INFO:teuthology.orchestra.run.smithi146.stderr: 94.1% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-19T07:57:59.482 INFO:teuthology.orchestra.run.smithi016.stderr: 94.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-19T07:58:00.652 INFO:teuthology.orchestra.run.smithi146.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-19T07:58:00.737 INFO:teuthology.orchestra.run.smithi146.stderr: 91.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-19T07:58:01.441 INFO:teuthology.orchestra.run.smithi146.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-19T07:58:01.885 INFO:teuthology.orchestra.run.smithi016.stderr: 91.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-19T07:58:01.886 INFO:teuthology.orchestra.run.smithi146.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-19T07:58:01.889 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-09-19T07:58:01.889 INFO:teuthology.orchestra.run.smithi146.stderr:real 0m7.015s 2024-09-19T07:58:01.889 INFO:teuthology.orchestra.run.smithi146.stderr:user 0m26.915s 2024-09-19T07:58:01.889 INFO:teuthology.orchestra.run.smithi146.stderr:sys 0m0.868s 2024-09-19T07:58:03.873 INFO:teuthology.orchestra.run.smithi016.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-19T07:58:05.886 INFO:teuthology.orchestra.run.smithi016.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-19T07:58:07.242 INFO:teuthology.orchestra.run.smithi016.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-19T07:58:14.676 INFO:teuthology.orchestra.run.smithi016.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.0.51804.log.gz 2024-09-19T07:58:45.112 INFO:teuthology.orchestra.run.smithi016.stderr: 94.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-19T07:58:45.114 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-19T07:58:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:real 0m50.264s 2024-09-19T07:58:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:user 1m51.274s 2024-09-19T07:58:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m3.674s 2024-09-19T07:58:45.115 INFO:tasks.ceph:Archiving logs... 2024-09-19T07:58:45.115 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911819/remote/smithi016/log 2024-09-19T07:58:45.116 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T07:59:00.288 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911819/remote/smithi146/log 2024-09-19T07:59:00.289 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T07:59:04.847 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T07:59:04.860 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T07:59:04.861 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T07:59:04.864 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T07:59:04.948 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T07:59:04.957 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T07:59:04.957 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:59:04.960 DEBUG:teuthology.orchestra.run.smithi146:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:59:04.974 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:59:04.977 INFO:teuthology.orchestra.run.smithi146.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:59:04.979 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:59:04.979 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-09-19T07:59:04.980 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv01.front.sepia.ceph.com 3 7 377 14 +1287us[+1292us] +/- 63ms 2024-09-19T07:59:04.980 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 13 -2884us[-2884us] +/- 73ms 2024-09-19T07:59:04.980 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +1380us[+1386us] +/- 71ms 2024-09-19T07:59:04.980 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:59:04.981 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:59:04.982 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2024-09-19T07:59:04.982 INFO:teuthology.orchestra.run.smithi146.stdout:^* hv01.front.sepia.ceph.com 3 6 377 12 +1266us[+1262us] +/- 63ms 2024-09-19T07:59:04.982 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 12 -2911us[-2915us] +/- 73ms 2024-09-19T07:59:04.982 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +1364us[+1360us] +/- 71ms 2024-09-19T07:59:04.982 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:59:04.983 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T07:59:04.994 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T07:59:04.994 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T07:59:05.050 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T07:59:05.152 DEBUG:teuthology.orchestra.run.smithi146:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T07:59:05.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:59:05.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:59:05.260 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:59:05.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:59:05.290 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T07:59:05.300 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T07:59:05.341 INFO:teuthology.task.internal:Duration was 1350.360999 seconds 2024-09-19T07:59:05.341 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T07:59:05.374 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T07:59:05.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T07:59:05.377 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T07:59:05.418 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:59:05.419 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:59:05.635 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T07:59:05.635 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T07:59:05.636 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T07:59:05.697 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi146.front.sepia.ceph.com 2024-09-19T07:59:05.698 DEBUG:teuthology.orchestra.run.smithi146:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T07:59:05.722 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T07:59:05.722 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T07:59:05.740 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T07:59:05.788 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T07:59:05.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T07:59:05.809 DEBUG:teuthology.orchestra.run.smithi146:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T07:59:05.955 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T07:59:05.968 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T07:59:05.968 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T07:59:05.994 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T07:59:06.019 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T07:59:06.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:59:06.036 DEBUG:teuthology.orchestra.run.smithi146:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:59:06.058 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-09-19T07:59:06.082 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = core 2024-09-19T07:59:06.096 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:59:06.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:59:06.127 DEBUG:teuthology.orchestra.run.smithi146:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:59:06.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:59:06.150 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T07:59:06.160 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T07:59:06.161 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911819/remote/smithi016 2024-09-19T07:59:06.161 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T07:59:06.208 DEBUG:teuthology.misc:Transferring archived files from smithi146:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911819/remote/smithi146 2024-09-19T07:59:06.208 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T07:59:06.243 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T07:59:06.243 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T07:59:06.245 DEBUG:teuthology.orchestra.run.smithi146:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T07:59:06.293 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T07:59:06.307 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T07:59:06.307 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T07:59:06.338 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T07:59:06.338 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T07:59:06.342 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T07:59:06.354 INFO:teuthology.orchestra.run.smithi016.stdout: 265836 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 07:59 /home/ubuntu/cephtest 2024-09-19T07:59:06.357 INFO:teuthology.orchestra.run.smithi146.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 07:59 /home/ubuntu/cephtest 2024-09-19T07:59:06.358 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T07:59:06.371 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T07:59:06.446 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/client-limits} duration: 1350.3609991073608 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-19T07:59:06.446 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T07:59:06.544 INFO:teuthology.run:pass